2023-10-07T23:57:57.682 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-10-07T23:57:57.682 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418110 --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full} --name yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi --owner scheduled_yuriw@teuthology --verbose -- /home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418110/orig.config.yaml 2023-10-07T23:57:57.731 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-07T23:57:57.820 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418110 branch: wip-yuri-testing-2023-10-06-0949-quincy description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full} email: yweinste@redhat.com first_in_suite: false job_id: '7418110' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.4' overrides: admin_socket: branch: wip-yuri-testing-2023-10-06-0949-quincy ceph: cephfs: ec_profile: - disabled conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 client.0: debug client: 20 debug objectcacher: 20 debug objecter: 20 client.1: debug client: 20 debug objectcacher: 20 debug objecter: 20 global: ms die on skipped message: false mds: debug mds: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 mon osd backfillfull ratio: 0.6 mon osd full ratio: 0.7 mon osd nearfull ratio: 0.6 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 memstore device bytes: 200000000 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 1.0 osd mon report interval: 5 osd objectstore: memstore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL - PG_RECOVERY_FULL - PG_DEGRADED sha1: 08cac0187919fc3a247efdf445f652e2fb30a647 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 08cac0187919fc3a247efdf445f652e2fb30a647 kclient: syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri-testing-2023-10-07-1508-quincy sha1: abf19fc7a04fe4fceab4e98d4013ae1da7c06006 owner: scheduled_yuriw@teuthology priority: 74 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 8174 sha1: 08cac0187919fc3a247efdf445f652e2fb30a647 sleep_before_teardown: 0 subset: 111/12000 suite: fs suite_branch: wip-yuri-testing-2023-10-07-1508-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_abf19fc7a04fe4fceab4e98d4013ae1da7c06006/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: abf19fc7a04fe4fceab4e98d4013ae1da7c06006 targets: smithi099.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD5LwXEONYcXRmTEhW6npr5lkKPFXj2fVvjuLfb0XPQFOInHhkdkKYBEEFID0gPuAhmpfy5GOshhTa4hwoSPgBk= smithi191.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDoCm6mEVQTmB6VkKOVpObIqmV+5V75Vd5lLvS4FoTXnIeMM/GF1r4oBcMw/DMRs5W3hz8Dp6kZyQGqVobIVqRk= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_full teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-10-07_22:15:20 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-07T23:57:57.821 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_abf19fc7a04fe4fceab4e98d4013ae1da7c06006/qa; will attempt to use it 2023-10-07T23:57:57.822 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_abf19fc7a04fe4fceab4e98d4013ae1da7c06006/qa/tasks 2023-10-07T23:57:57.822 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-07T23:57:57.824 INFO:teuthology.task.internal:Checking packages... 2023-10-07T23:57:57.848 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '08cac0187919fc3a247efdf445f652e2fb30a647' 2023-10-07T23:57:57.848 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-07T23:57:57.848 INFO:teuthology.packaging:ref: None 2023-10-07T23:57:57.848 INFO:teuthology.packaging:tag: None 2023-10-07T23:57:57.848 INFO:teuthology.packaging:branch: wip-yuri-testing-2023-10-06-0949-quincy 2023-10-07T23:57:57.848 INFO:teuthology.packaging:sha1: 08cac0187919fc3a247efdf445f652e2fb30a647 2023-10-07T23:57:57.849 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri-testing-2023-10-06-0949-quincy 2023-10-07T23:57:58.078 INFO:teuthology.task.internal:Found packages for ceph version 17.2.6-1320.g08cac018 2023-10-07T23:57:58.078 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-07T23:57:58.085 INFO:teuthology.task.internal:no buildpackages task found 2023-10-07T23:57:58.085 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-07T23:57:58.092 INFO:teuthology.task.internal:Saving configuration 2023-10-07T23:57:58.109 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-07T23:57:58.115 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-07T23:57:58.141 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi099.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418110', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-07 23:52:52.349362', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD5LwXEONYcXRmTEhW6npr5lkKPFXj2fVvjuLfb0XPQFOInHhkdkKYBEEFID0gPuAhmpfy5GOshhTa4hwoSPgBk='} 2023-10-07T23:57:58.163 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi191.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418110', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-07 23:52:52.347975', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDoCm6mEVQTmB6VkKOVpObIqmV+5V75Vd5lLvS4FoTXnIeMM/GF1r4oBcMw/DMRs5W3hz8Dp6kZyQGqVobIVqRk='} 2023-10-07T23:57:58.163 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-07T23:57:58.169 INFO:teuthology.task.internal:roles: ubuntu@smithi099.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-10-07T23:57:58.169 INFO:teuthology.task.internal:roles: ubuntu@smithi191.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-10-07T23:57:58.169 INFO:teuthology.run_tasks:Running task console_log... 2023-10-07T23:57:58.241 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f73d8286b80>, signals=[15]) 2023-10-07T23:57:58.241 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-07T23:57:58.248 INFO:teuthology.task.internal:Opening connections... 2023-10-07T23:57:58.248 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi099.front.sepia.ceph.com 2023-10-07T23:57:58.249 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-07T23:57:58.317 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi191.front.sepia.ceph.com 2023-10-07T23:57:58.318 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-07T23:57:58.384 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-07T23:57:58.390 DEBUG:teuthology.orchestra.run.smithi099:> uname -m 2023-10-07T23:57:58.408 INFO:teuthology.orchestra.run.smithi099.stdout:x86_64 2023-10-07T23:57:58.408 DEBUG:teuthology.orchestra.run.smithi099:> cat /etc/os-release 2023-10-07T23:57:58.465 INFO:teuthology.orchestra.run.smithi099.stdout:NAME="Red Hat Enterprise Linux" 2023-10-07T23:57:58.465 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION="8.4 (Ootpa)" 2023-10-07T23:57:58.465 INFO:teuthology.orchestra.run.smithi099.stdout:ID="rhel" 2023-10-07T23:57:58.465 INFO:teuthology.orchestra.run.smithi099.stdout:ID_LIKE="fedora" 2023-10-07T23:57:58.465 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION_ID="8.4" 2023-10-07T23:57:58.465 INFO:teuthology.orchestra.run.smithi099.stdout:PLATFORM_ID="platform:el8" 2023-10-07T23:57:58.465 INFO:teuthology.orchestra.run.smithi099.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-10-07T23:57:58.466 INFO:teuthology.orchestra.run.smithi099.stdout:ANSI_COLOR="0;31" 2023-10-07T23:57:58.466 INFO:teuthology.orchestra.run.smithi099.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-10-07T23:57:58.466 INFO:teuthology.orchestra.run.smithi099.stdout:HOME_URL="https://www.redhat.com/" 2023-10-07T23:57:58.466 INFO:teuthology.orchestra.run.smithi099.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-07T23:57:58.466 INFO:teuthology.orchestra.run.smithi099.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-07T23:57:58.466 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-07T23:57:58.466 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-07T23:57:58.466 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-10-07T23:57:58.466 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-07T23:57:58.466 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-10-07T23:57:58.467 INFO:teuthology.lock.ops:Updating smithi099.front.sepia.ceph.com on lock server 2023-10-07T23:57:58.484 DEBUG:teuthology.orchestra.run.smithi191:> uname -m 2023-10-07T23:57:58.501 INFO:teuthology.orchestra.run.smithi191.stdout:x86_64 2023-10-07T23:57:58.502 DEBUG:teuthology.orchestra.run.smithi191:> cat /etc/os-release 2023-10-07T23:57:58.557 INFO:teuthology.orchestra.run.smithi191.stdout:NAME="Red Hat Enterprise Linux" 2023-10-07T23:57:58.558 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION="8.4 (Ootpa)" 2023-10-07T23:57:58.558 INFO:teuthology.orchestra.run.smithi191.stdout:ID="rhel" 2023-10-07T23:57:58.558 INFO:teuthology.orchestra.run.smithi191.stdout:ID_LIKE="fedora" 2023-10-07T23:57:58.558 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION_ID="8.4" 2023-10-07T23:57:58.558 INFO:teuthology.orchestra.run.smithi191.stdout:PLATFORM_ID="platform:el8" 2023-10-07T23:57:58.558 INFO:teuthology.orchestra.run.smithi191.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-10-07T23:57:58.558 INFO:teuthology.orchestra.run.smithi191.stdout:ANSI_COLOR="0;31" 2023-10-07T23:57:58.558 INFO:teuthology.orchestra.run.smithi191.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-10-07T23:57:58.558 INFO:teuthology.orchestra.run.smithi191.stdout:HOME_URL="https://www.redhat.com/" 2023-10-07T23:57:58.559 INFO:teuthology.orchestra.run.smithi191.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-07T23:57:58.559 INFO:teuthology.orchestra.run.smithi191.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-07T23:57:58.559 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-07T23:57:58.559 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-07T23:57:58.559 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-10-07T23:57:58.559 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-07T23:57:58.559 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-10-07T23:57:58.559 INFO:teuthology.lock.ops:Updating smithi191.front.sepia.ceph.com on lock server 2023-10-07T23:57:58.584 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-07T23:57:58.593 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-07T23:57:58.599 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-07T23:57:58.599 DEBUG:teuthology.orchestra.run.smithi099:> test '!' -e /home/ubuntu/cephtest 2023-10-07T23:57:58.601 DEBUG:teuthology.orchestra.run.smithi191:> test '!' -e /home/ubuntu/cephtest 2023-10-07T23:57:58.617 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-07T23:57:58.623 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-07T23:57:58.623 DEBUG:teuthology.orchestra.run.smithi099:> test -z $(ls -A /var/lib/ceph) 2023-10-07T23:57:58.658 DEBUG:teuthology.orchestra.run.smithi191:> test -z $(ls -A /var/lib/ceph) 2023-10-07T23:57:58.673 INFO:teuthology.orchestra.run.smithi099.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-10-07T23:57:58.675 INFO:teuthology.orchestra.run.smithi191.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-10-07T23:57:58.675 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-07T23:57:58.722 INFO:teuthology.run_tasks:Running task kernel... 2023-10-07T23:57:58.737 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2023-10-07T23:57:58.737 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2023-10-07T23:57:58.737 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2023-10-07T23:57:58.738 DEBUG:teuthology.orchestra.run.smithi099:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-07T23:57:58.738 DEBUG:teuthology.orchestra.run.smithi191:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-07T23:57:58.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-07T23:57:58.754 DEBUG:teuthology.orchestra.run.smithi191:> uname -r 2023-10-07T23:57:58.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-07T23:57:58.755 DEBUG:teuthology.orchestra.run.smithi099:> uname -r 2023-10-07T23:57:58.809 INFO:teuthology.orchestra.run.smithi191.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-10-07T23:57:58.810 INFO:teuthology.task.kernel:Running kernel on smithi191: 4.18.0-305.25.1.el8_4.x86_64 2023-10-07T23:57:58.810 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum install -y kernel 2023-10-07T23:57:58.811 INFO:teuthology.orchestra.run.smithi099.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-10-07T23:57:58.811 INFO:teuthology.task.kernel:Running kernel on smithi099: 4.18.0-305.25.1.el8_4.x86_64 2023-10-07T23:57:58.811 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum install -y kernel 2023-10-07T23:58:00.233 INFO:teuthology.orchestra.run.smithi191.stdout:Updating Subscription Management repositories. 2023-10-07T23:58:00.234 INFO:teuthology.orchestra.run.smithi191.stdout:Unable to read consumer identity 2023-10-07T23:58:00.294 INFO:teuthology.orchestra.run.smithi099.stdout:Updating Subscription Management repositories. 2023-10-07T23:58:00.295 INFO:teuthology.orchestra.run.smithi099.stdout:Unable to read consumer identity 2023-10-07T23:58:01.805 INFO:teuthology.orchestra.run.smithi191.stdout:Copr repo for python3-asyncssh owned by ceph 28 kB/s | 3.5 kB 00:00 2023-10-07T23:58:01.837 INFO:teuthology.orchestra.run.smithi099.stdout:Copr repo for python3-asyncssh owned by ceph 14 kB/s | 3.5 kB 00:00 2023-10-07T23:58:02.387 INFO:teuthology.orchestra.run.smithi099.stdout:Extra Packages for Enterprise Linux 29 MB/s | 16 MB 00:00 2023-10-07T23:58:02.437 INFO:teuthology.orchestra.run.smithi191.stdout:Extra Packages for Enterprise Linux 27 MB/s | 16 MB 00:00 2023-10-07T23:58:05.638 INFO:teuthology.orchestra.run.smithi099.stdout:lab-extras 4.3 MB/s | 24 kB 00:00 2023-10-07T23:58:05.745 INFO:teuthology.orchestra.run.smithi191.stdout:lab-extras 4.6 MB/s | 24 kB 00:00 2023-10-07T23:58:06.393 INFO:teuthology.orchestra.run.smithi099.stdout:Package kernel-4.18.0-80.el8.x86_64 is already installed. 2023-10-07T23:58:06.393 INFO:teuthology.orchestra.run.smithi099.stdout:Package kernel-4.18.0-425.3.1.el8.x86_64 is already installed. 2023-10-07T23:58:06.393 INFO:teuthology.orchestra.run.smithi099.stdout:Package kernel-4.18.0-305.25.1.el8_4.x86_64 is already installed. 2023-10-07T23:58:06.445 INFO:teuthology.orchestra.run.smithi191.stdout:Package kernel-4.18.0-80.el8.x86_64 is already installed. 2023-10-07T23:58:06.445 INFO:teuthology.orchestra.run.smithi191.stdout:Package kernel-4.18.0-425.3.1.el8.x86_64 is already installed. 2023-10-07T23:58:06.445 INFO:teuthology.orchestra.run.smithi191.stdout:Package kernel-4.18.0-305.25.1.el8_4.x86_64 is already installed. 2023-10-07T23:58:06.542 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2023-10-07T23:58:06.543 INFO:teuthology.orchestra.run.smithi099.stdout:Nothing to do. 2023-10-07T23:58:06.543 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2023-10-07T23:58:06.591 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2023-10-07T23:58:06.592 INFO:teuthology.orchestra.run.smithi191.stdout:Nothing to do. 2023-10-07T23:58:06.592 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2023-10-07T23:58:06.648 DEBUG:teuthology.orchestra.run.smithi099:> echo no | sudo yum reinstall kernel || true 2023-10-07T23:58:06.708 DEBUG:teuthology.orchestra.run.smithi191:> echo no | sudo yum reinstall kernel || true 2023-10-07T23:58:06.991 INFO:teuthology.orchestra.run.smithi099.stdout:Updating Subscription Management repositories. 2023-10-07T23:58:06.991 INFO:teuthology.orchestra.run.smithi099.stdout:Unable to read consumer identity 2023-10-07T23:58:07.051 INFO:teuthology.orchestra.run.smithi191.stdout:Updating Subscription Management repositories. 2023-10-07T23:58:07.051 INFO:teuthology.orchestra.run.smithi191.stdout:Unable to read consumer identity 2023-10-07T23:58:07.287 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:00:02 ago on Sat 07 Oct 2023 11:58:05 PM UTC. 2023-10-07T23:58:07.354 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:00:02 ago on Sat 07 Oct 2023 11:58:05 PM UTC. 2023-10-07T23:58:07.421 INFO:teuthology.orchestra.run.smithi099.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-10-07T23:58:07.421 INFO:teuthology.orchestra.run.smithi099.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-07T23:58:07.422 INFO:teuthology.orchestra.run.smithi099.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-07T23:58:07.448 INFO:teuthology.orchestra.run.smithi099.stderr:Error: No packages marked for reinstall. 2023-10-07T23:58:07.474 INFO:teuthology.orchestra.run.smithi191.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-10-07T23:58:07.474 INFO:teuthology.orchestra.run.smithi191.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-07T23:58:07.474 INFO:teuthology.orchestra.run.smithi191.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-07T23:58:07.493 INFO:teuthology.orchestra.run.smithi191.stderr:Error: No packages marked for reinstall. 2023-10-07T23:58:07.497 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum reinstall -y kernel || true 2023-10-07T23:58:07.544 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum reinstall -y kernel || true 2023-10-07T23:58:07.840 INFO:teuthology.orchestra.run.smithi099.stdout:Updating Subscription Management repositories. 2023-10-07T23:58:07.840 INFO:teuthology.orchestra.run.smithi099.stdout:Unable to read consumer identity 2023-10-07T23:58:07.885 INFO:teuthology.orchestra.run.smithi191.stdout:Updating Subscription Management repositories. 2023-10-07T23:58:07.886 INFO:teuthology.orchestra.run.smithi191.stdout:Unable to read consumer identity 2023-10-07T23:58:08.128 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:00:03 ago on Sat 07 Oct 2023 11:58:05 PM UTC. 2023-10-07T23:58:08.170 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:00:03 ago on Sat 07 Oct 2023 11:58:05 PM UTC. 2023-10-07T23:58:08.295 INFO:teuthology.orchestra.run.smithi099.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-10-07T23:58:08.295 INFO:teuthology.orchestra.run.smithi099.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-07T23:58:08.295 INFO:teuthology.orchestra.run.smithi099.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-07T23:58:08.309 INFO:teuthology.orchestra.run.smithi099.stderr:Error: No packages marked for reinstall. 2023-10-07T23:58:08.315 INFO:teuthology.orchestra.run.smithi191.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-10-07T23:58:08.315 INFO:teuthology.orchestra.run.smithi191.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-07T23:58:08.315 INFO:teuthology.orchestra.run.smithi191.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-07T23:58:08.349 INFO:teuthology.orchestra.run.smithi191.stderr:Error: No packages marked for reinstall. 2023-10-07T23:58:08.365 DEBUG:teuthology.orchestra.run.smithi099:> rpm -q kernel | sort -rV | head -n 1 2023-10-07T23:58:08.399 DEBUG:teuthology.orchestra.run.smithi191:> rpm -q kernel | sort -rV | head -n 1 2023-10-07T23:58:08.491 INFO:teuthology.orchestra.run.smithi099.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-10-07T23:58:08.491 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-10-07T23:58:08.491 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-305.25.1.el8_4.x86_64 Expected: 4.18.0-425.3.1.el8.x86_64 2023-10-07T23:58:08.491 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-07T23:58:08.491 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-07T23:58:08.492 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2023-10-07T23:58:08.492 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi099.front.sepia.ceph.com, path=None, version=distro) 2023-10-07T23:58:08.492 DEBUG:teuthology.orchestra.run.smithi099:> rpm -q kernel | sort -rV | head -n 1 2023-10-07T23:58:08.524 INFO:teuthology.orchestra.run.smithi191.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-10-07T23:58:08.524 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-10-07T23:58:08.524 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-305.25.1.el8_4.x86_64 Expected: 4.18.0-425.3.1.el8.x86_64 2023-10-07T23:58:08.524 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-07T23:58:08.524 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-07T23:58:08.524 INFO:teuthology.task.kernel:Installing distro kernel on client.1... 2023-10-07T23:58:08.524 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi191.front.sepia.ceph.com, path=None, version=distro) 2023-10-07T23:58:08.525 DEBUG:teuthology.orchestra.run.smithi191:> rpm -q kernel | sort -rV | head -n 1 2023-10-07T23:58:08.616 INFO:teuthology.orchestra.run.smithi099.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-10-07T23:58:08.616 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-10-07T23:58:08.617 DEBUG:teuthology.orchestra.run.smithi099:> sudo rpm -qi grub2-tools 2023-10-07T23:58:08.644 INFO:teuthology.orchestra.run.smithi099.stdout:Name : grub2-tools 2023-10-07T23:58:08.644 INFO:teuthology.orchestra.run.smithi099.stdout:Epoch : 1 2023-10-07T23:58:08.644 INFO:teuthology.orchestra.run.smithi099.stdout:Version : 2.02 2023-10-07T23:58:08.644 INFO:teuthology.orchestra.run.smithi099.stdout:Release : 99.el8_4.1 2023-10-07T23:58:08.645 INFO:teuthology.orchestra.run.smithi099.stdout:Architecture: x86_64 2023-10-07T23:58:08.645 INFO:teuthology.orchestra.run.smithi099.stdout:Install Date: Wed 21 Dec 2022 01:02:32 PM UTC 2023-10-07T23:58:08.645 INFO:teuthology.orchestra.run.smithi099.stdout:Group : System Environment/Base 2023-10-07T23:58:08.645 INFO:teuthology.orchestra.run.smithi099.stdout:Size : 9264631 2023-10-07T23:58:08.645 INFO:teuthology.orchestra.run.smithi099.stdout:License : GPLv3+ 2023-10-07T23:58:08.646 INFO:teuthology.orchestra.run.smithi099.stdout:Signature : RSA/SHA256, Thu 09 Sep 2021 09:41:55 AM UTC, Key ID 199e2f91fd431d51 2023-10-07T23:58:08.646 INFO:teuthology.orchestra.run.smithi099.stdout:Source RPM : grub2-2.02-99.el8_4.1.src.rpm 2023-10-07T23:58:08.646 INFO:teuthology.orchestra.run.smithi099.stdout:Build Date : Tue 18 May 2021 07:46:34 AM UTC 2023-10-07T23:58:08.646 INFO:teuthology.orchestra.run.smithi099.stdout:Build Host : x86-vm-07.build.eng.bos.redhat.com 2023-10-07T23:58:08.646 INFO:teuthology.orchestra.run.smithi099.stdout:Relocations : (not relocatable) 2023-10-07T23:58:08.646 INFO:teuthology.orchestra.run.smithi099.stdout:Packager : Red Hat, Inc. 2023-10-07T23:58:08.647 INFO:teuthology.orchestra.run.smithi099.stdout:Vendor : Red Hat, Inc. 2023-10-07T23:58:08.647 INFO:teuthology.orchestra.run.smithi099.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-07T23:58:08.647 INFO:teuthology.orchestra.run.smithi099.stdout:Summary : Support tools for GRUB. 2023-10-07T23:58:08.647 INFO:teuthology.orchestra.run.smithi099.stdout:Description : 2023-10-07T23:58:08.647 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-07T23:58:08.647 INFO:teuthology.orchestra.run.smithi099.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-07T23:58:08.648 INFO:teuthology.orchestra.run.smithi099.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-07T23:58:08.648 INFO:teuthology.orchestra.run.smithi099.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-07T23:58:08.648 INFO:teuthology.orchestra.run.smithi099.stdout:hardware devices. 2023-10-07T23:58:08.648 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-07T23:58:08.648 INFO:teuthology.orchestra.run.smithi099.stdout:This subpackage provides tools for support of all platforms. 2023-10-07T23:58:08.650 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-07T23:58:08.650 INFO:teuthology.task.kernel:Updating grub on smithi099 to boot 4.18.0-425.3.1.el8.x86_64 2023-10-07T23:58:08.650 DEBUG:teuthology.orchestra.run.smithi099:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-07T23:58:08.652 INFO:teuthology.orchestra.run.smithi191.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-10-07T23:58:08.652 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-10-07T23:58:08.653 DEBUG:teuthology.orchestra.run.smithi191:> sudo rpm -qi grub2-tools 2023-10-07T23:58:08.680 INFO:teuthology.orchestra.run.smithi191.stdout:Name : grub2-tools 2023-10-07T23:58:08.680 INFO:teuthology.orchestra.run.smithi191.stdout:Epoch : 1 2023-10-07T23:58:08.680 INFO:teuthology.orchestra.run.smithi191.stdout:Version : 2.02 2023-10-07T23:58:08.681 INFO:teuthology.orchestra.run.smithi191.stdout:Release : 99.el8_4.1 2023-10-07T23:58:08.681 INFO:teuthology.orchestra.run.smithi191.stdout:Architecture: x86_64 2023-10-07T23:58:08.681 INFO:teuthology.orchestra.run.smithi191.stdout:Install Date: Wed 21 Dec 2022 01:02:32 PM UTC 2023-10-07T23:58:08.681 INFO:teuthology.orchestra.run.smithi191.stdout:Group : System Environment/Base 2023-10-07T23:58:08.681 INFO:teuthology.orchestra.run.smithi191.stdout:Size : 9264631 2023-10-07T23:58:08.681 INFO:teuthology.orchestra.run.smithi191.stdout:License : GPLv3+ 2023-10-07T23:58:08.681 INFO:teuthology.orchestra.run.smithi191.stdout:Signature : RSA/SHA256, Thu 09 Sep 2021 09:41:55 AM UTC, Key ID 199e2f91fd431d51 2023-10-07T23:58:08.681 INFO:teuthology.orchestra.run.smithi191.stdout:Source RPM : grub2-2.02-99.el8_4.1.src.rpm 2023-10-07T23:58:08.681 INFO:teuthology.orchestra.run.smithi191.stdout:Build Date : Tue 18 May 2021 07:46:34 AM UTC 2023-10-07T23:58:08.681 INFO:teuthology.orchestra.run.smithi191.stdout:Build Host : x86-vm-07.build.eng.bos.redhat.com 2023-10-07T23:58:08.682 INFO:teuthology.orchestra.run.smithi191.stdout:Relocations : (not relocatable) 2023-10-07T23:58:08.682 INFO:teuthology.orchestra.run.smithi191.stdout:Packager : Red Hat, Inc. 2023-10-07T23:58:08.682 INFO:teuthology.orchestra.run.smithi191.stdout:Vendor : Red Hat, Inc. 2023-10-07T23:58:08.682 INFO:teuthology.orchestra.run.smithi191.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-07T23:58:08.682 INFO:teuthology.orchestra.run.smithi191.stdout:Summary : Support tools for GRUB. 2023-10-07T23:58:08.682 INFO:teuthology.orchestra.run.smithi191.stdout:Description : 2023-10-07T23:58:08.682 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-07T23:58:08.682 INFO:teuthology.orchestra.run.smithi191.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-07T23:58:08.682 INFO:teuthology.orchestra.run.smithi191.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-07T23:58:08.682 INFO:teuthology.orchestra.run.smithi191.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-07T23:58:08.682 INFO:teuthology.orchestra.run.smithi191.stdout:hardware devices. 2023-10-07T23:58:08.682 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-07T23:58:08.683 INFO:teuthology.orchestra.run.smithi191.stdout:This subpackage provides tools for support of all platforms. 2023-10-07T23:58:08.683 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-07T23:58:08.684 INFO:teuthology.task.kernel:Updating grub on smithi191 to boot 4.18.0-425.3.1.el8.x86_64 2023-10-07T23:58:08.684 DEBUG:teuthology.orchestra.run.smithi191:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-07T23:58:10.056 INFO:teuthology.orchestra.run.smithi099.stderr:Generating grub configuration file ... 2023-10-07T23:58:10.109 INFO:teuthology.orchestra.run.smithi191.stderr:Generating grub configuration file ... 2023-10-07T23:58:11.800 INFO:teuthology.orchestra.run.smithi099.stderr:done 2023-10-07T23:58:11.803 DEBUG:teuthology.orchestra.run.smithi099:> mktemp 2023-10-07T23:58:11.858 INFO:teuthology.orchestra.run.smithi099.stdout:/tmp/tmp.h1GaCsb9iL 2023-10-07T23:58:11.858 DEBUG:teuthology.orchestra.run.smithi099:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.h1GaCsb9iL 2023-10-07T23:58:11.933 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 0666 /tmp/tmp.h1GaCsb9iL 2023-10-07T23:58:12.095 DEBUG:teuthology.orchestra.remote:smithi099:/tmp/tmp.h1GaCsb9iL is 6KB 2023-10-07T23:58:12.143 DEBUG:teuthology.orchestra.run.smithi099:> rm -fr /tmp/tmp.h1GaCsb9iL 2023-10-07T23:58:12.157 DEBUG:teuthology.orchestra.run.smithi099:> sudo /bin/ls /boot/loader/entries || true 2023-10-07T23:58:12.223 INFO:teuthology.orchestra.run.smithi099.stdout:298e228f3b5e401e8ac0f5ae159a2e06-0-rescue.conf 2023-10-07T23:58:12.223 INFO:teuthology.orchestra.run.smithi099.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-305.25.1.el8_4.x86_64.conf 2023-10-07T23:58:12.223 INFO:teuthology.orchestra.run.smithi099.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64.conf 2023-10-07T23:58:12.223 INFO:teuthology.orchestra.run.smithi099.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-80.el8.x86_64.conf 2023-10-07T23:58:12.224 DEBUG:teuthology.orchestra.run.smithi099:> sudo grub2-set-default 298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64 2023-10-07T23:58:12.317 INFO:teuthology.orchestra.run.smithi191.stderr:done 2023-10-07T23:58:12.319 DEBUG:teuthology.orchestra.run.smithi191:> mktemp 2023-10-07T23:58:12.332 INFO:teuthology.orchestra.run.smithi191.stdout:/tmp/tmp.IllPidY2jv 2023-10-07T23:58:12.333 DEBUG:teuthology.orchestra.run.smithi191:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.IllPidY2jv 2023-10-07T23:58:12.381 DEBUG:teuthology.orchestra.run.smithi099:> sudo shutdown -r now 2023-10-07T23:58:12.402 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 0666 /tmp/tmp.IllPidY2jv 2023-10-07T23:58:12.556 DEBUG:teuthology.orchestra.remote:smithi191:/tmp/tmp.IllPidY2jv is 6KB 2023-10-07T23:58:12.604 DEBUG:teuthology.orchestra.run.smithi191:> rm -fr /tmp/tmp.IllPidY2jv 2023-10-07T23:58:12.617 DEBUG:teuthology.orchestra.run.smithi191:> sudo /bin/ls /boot/loader/entries || true 2023-10-07T23:58:12.689 INFO:teuthology.orchestra.run.smithi191.stdout:298e228f3b5e401e8ac0f5ae159a2e06-0-rescue.conf 2023-10-07T23:58:12.689 INFO:teuthology.orchestra.run.smithi191.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-305.25.1.el8_4.x86_64.conf 2023-10-07T23:58:12.689 INFO:teuthology.orchestra.run.smithi191.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64.conf 2023-10-07T23:58:12.689 INFO:teuthology.orchestra.run.smithi191.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-80.el8.x86_64.conf 2023-10-07T23:58:12.690 DEBUG:teuthology.orchestra.run.smithi191:> sudo grub2-set-default 298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64 2023-10-07T23:58:12.863 DEBUG:teuthology.orchestra.run.smithi191:> sudo shutdown -r now 2023-10-07T23:58:42.407 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-10-07T23:58:42.408 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2023-10-07T23:58:42.408 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-07T23:58:42.866 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2023-10-07T23:58:42.866 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com' 2023-10-07T23:58:42.867 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-07T23:59:00.730 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.99 2023-10-07T23:59:01.242 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.191 2023-10-07T23:59:09.738 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2023-10-07T23:59:09.739 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-07T23:59:10.243 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com' 2023-10-07T23:59:10.244 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-07T23:59:12.798 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.99 2023-10-07T23:59:13.306 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.191 2023-10-07T23:59:24.822 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2023-10-07T23:59:24.823 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-07T23:59:25.308 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com' 2023-10-07T23:59:25.308 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-07T23:59:56.608 DEBUG:teuthology.orchestra.run.smithi099:> true 2023-10-07T23:59:56.937 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi099.front.sepia.ceph.com' 2023-10-07T23:59:56.937 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "4.18.0-425.3.1.el8.x86_64"... 2023-10-07T23:59:56.938 DEBUG:teuthology.orchestra.run.smithi099:> uname -r 2023-10-07T23:59:56.951 INFO:teuthology.orchestra.run.smithi099.stdout:4.18.0-425.3.1.el8.x86_64 2023-10-07T23:59:56.951 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-425.3.1.el8.x86_64 vs 4.18.0-425.3.1.el8.x86_64 2023-10-07T23:59:56.952 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-07T23:59:56.952 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-10-07T23:59:57.189 DEBUG:teuthology.orchestra.run.smithi191:> true 2023-10-07T23:59:57.470 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi191.front.sepia.ceph.com' 2023-10-07T23:59:57.470 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "4.18.0-425.3.1.el8.x86_64"... 2023-10-07T23:59:57.471 DEBUG:teuthology.orchestra.run.smithi191:> uname -r 2023-10-07T23:59:57.484 INFO:teuthology.orchestra.run.smithi191.stdout:4.18.0-425.3.1.el8.x86_64 2023-10-07T23:59:57.485 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-425.3.1.el8.x86_64 vs 4.18.0-425.3.1.el8.x86_64 2023-10-07T23:59:57.485 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-07T23:59:57.485 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-10-07T23:59:57.952 DEBUG:teuthology.parallel:result is None 2023-10-07T23:59:58.486 DEBUG:teuthology.parallel:result is None 2023-10-07T23:59:58.486 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-07T23:59:58.493 INFO:teuthology.task.internal:Creating test directory... 2023-10-07T23:59:58.493 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-07T23:59:58.539 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-07T23:59:58.754 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-07T23:59:58.761 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-07T23:59:58.846 INFO:teuthology.task.internal:Creating archive directory... 2023-10-07T23:59:58.846 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-07T23:59:58.873 DEBUG:teuthology.orchestra.run.smithi191:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-07T23:59:59.234 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-07T23:59:59.242 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-07T23:59:59.242 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-07T23:59:59.272 DEBUG:teuthology.orchestra.run.smithi191:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-07T23:59:59.886 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-07T23:59:59.907 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-07T23:59:59.987 INFO:teuthology.orchestra.run.smithi191.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-08T00:00:00.177 INFO:teuthology.orchestra.run.smithi191.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-08T00:00:00.179 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-08T00:00:00.186 INFO:teuthology.task.internal:Configuring sudo... 2023-10-08T00:00:00.186 DEBUG:teuthology.orchestra.run.smithi099:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-08T00:00:00.189 DEBUG:teuthology.orchestra.run.smithi191:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-08T00:00:00.423 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-08T00:00:00.431 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-08T00:00:00.432 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-08T00:00:00.434 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-08T00:00:00.757 DEBUG:teuthology.orchestra.run.smithi099:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-08T00:00:00.815 DEBUG:teuthology.orchestra.run.smithi099:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-08T00:00:00.889 DEBUG:teuthology.orchestra.run.smithi099:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-08T00:00:00.930 DEBUG:teuthology.orchestra.run.smithi099:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-08T00:00:00.972 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-08T00:00:00.972 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-08T00:00:01.037 DEBUG:teuthology.orchestra.run.smithi191:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-08T00:00:01.130 DEBUG:teuthology.orchestra.run.smithi191:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-08T00:00:01.191 DEBUG:teuthology.orchestra.run.smithi191:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-08T00:00:01.264 DEBUG:teuthology.orchestra.run.smithi191:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-08T00:00:01.289 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2023-10-08T00:00:01.289 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-08T00:00:01.353 DEBUG:teuthology.orchestra.run.smithi099:> sudo service rsyslog restart 2023-10-08T00:00:01.356 DEBUG:teuthology.orchestra.run.smithi191:> sudo service rsyslog restart 2023-10-08T00:00:01.431 INFO:teuthology.orchestra.run.smithi099.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-08T00:00:01.546 INFO:teuthology.orchestra.run.smithi191.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-08T00:00:01.927 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-08T00:00:01.935 INFO:teuthology.task.internal:Starting timer... 2023-10-08T00:00:01.935 INFO:teuthology.run_tasks:Running task pcp... 2023-10-08T00:00:01.945 INFO:teuthology.run_tasks:Running task selinux... 2023-10-08T00:00:01.970 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-10-08T00:00:01.971 DEBUG:teuthology.orchestra.run.smithi099:> sudo service auditd rotate 2023-10-08T00:00:02.024 INFO:teuthology.orchestra.run.smithi099.stdout:Rotating logs: [ OK ] 2023-10-08T00:00:02.025 DEBUG:teuthology.orchestra.run.smithi191:> sudo service auditd rotate 2023-10-08T00:00:02.227 INFO:teuthology.orchestra.run.smithi191.stdout:Rotating logs: [ OK ] 2023-10-08T00:00:02.229 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-08T00:00:02.229 DEBUG:teuthology.orchestra.run.smithi099:> /usr/sbin/getenforce 2023-10-08T00:00:02.251 INFO:teuthology.orchestra.run.smithi099.stdout:Permissive 2023-10-08T00:00:02.251 DEBUG:teuthology.orchestra.run.smithi191:> /usr/sbin/getenforce 2023-10-08T00:00:02.266 INFO:teuthology.orchestra.run.smithi191.stdout:Permissive 2023-10-08T00:00:02.266 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi099.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi191.front.sepia.ceph.com': 'permissive'} 2023-10-08T00:00:02.266 DEBUG:teuthology.orchestra.run.smithi099:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-08T00:00:02.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T00:00:02.316 DEBUG:teuthology.orchestra.run.smithi191:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-08T00:00:02.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T00:00:02.341 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-08T00:00:02.342 DEBUG:teuthology.orchestra.run.smithi099:> sudo /usr/sbin/setenforce permissive 2023-10-08T00:00:02.389 DEBUG:teuthology.orchestra.run.smithi191:> sudo /usr/sbin/setenforce permissive 2023-10-08T00:00:02.414 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-08T00:00:02.424 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-10-08T00:00:02.429 INFO:teuthology.repo_utils:Fetching main from origin 2023-10-08T00:00:02.546 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-08T00:00:02.562 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-08T00:00:02.566 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi099.front.sepia.ceph.com,smithi191.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-08T00:08:15.538 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi099.front.sepia.ceph.com'), Remote(name='ubuntu@smithi191.front.sepia.ceph.com')] 2023-10-08T00:08:15.539 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2023-10-08T00:08:15.540 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T00:08:15.609 DEBUG:teuthology.orchestra.run.smithi099:> true 2023-10-08T00:08:15.676 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi099.front.sepia.ceph.com' 2023-10-08T00:08:15.677 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com' 2023-10-08T00:08:15.677 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T00:08:15.746 DEBUG:teuthology.orchestra.run.smithi191:> true 2023-10-08T00:08:15.809 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi191.front.sepia.ceph.com' 2023-10-08T00:08:15.809 INFO:teuthology.run_tasks:Running task clock... 2023-10-08T00:08:15.820 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-08T00:08:15.820 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-08T00:08:15.821 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-08T00:08:15.823 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-08T00:08:15.823 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-08T00:08:15.848 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-08T00:08:15.862 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-08T00:08:15.878 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-08T00:08:15.892 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-08T00:08:15.899 INFO:teuthology.orchestra.run.smithi099.stderr:sudo: ntpd: command not found 2023-10-08T00:08:15.910 INFO:teuthology.orchestra.run.smithi099.stdout:506 Cannot talk to daemon 2023-10-08T00:08:15.922 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-08T00:08:15.924 INFO:teuthology.orchestra.run.smithi191.stderr:sudo: ntpd: command not found 2023-10-08T00:08:15.934 INFO:teuthology.orchestra.run.smithi191.stdout:506 Cannot talk to daemon 2023-10-08T00:08:15.935 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-08T00:08:15.948 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-08T00:08:15.962 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-08T00:08:15.999 INFO:teuthology.orchestra.run.smithi099.stderr:bash: ntpq: command not found 2023-10-08T00:08:16.002 INFO:teuthology.orchestra.run.smithi099.stdout:210 Number of sources = 4 2023-10-08T00:08:16.002 INFO:teuthology.orchestra.run.smithi099.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-08T00:08:16.002 INFO:teuthology.orchestra.run.smithi099.stdout:=============================================================================== 2023-10-08T00:08:16.002 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T00:08:16.002 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T00:08:16.003 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T00:08:16.003 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T00:08:16.029 INFO:teuthology.orchestra.run.smithi191.stderr:bash: ntpq: command not found 2023-10-08T00:08:16.033 INFO:teuthology.orchestra.run.smithi191.stdout:210 Number of sources = 4 2023-10-08T00:08:16.033 INFO:teuthology.orchestra.run.smithi191.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-08T00:08:16.033 INFO:teuthology.orchestra.run.smithi191.stdout:=============================================================================== 2023-10-08T00:08:16.033 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T00:08:16.033 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T00:08:16.033 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T00:08:16.033 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T00:08:16.034 INFO:teuthology.run_tasks:Running task install... 2023-10-08T00:08:16.041 DEBUG:teuthology.task.install:project ceph 2023-10-08T00:08:16.041 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647'}} 2023-10-08T00:08:16.042 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647'} 2023-10-08T00:08:16.042 INFO:teuthology.task.install:Using flavor: default 2023-10-08T00:08:16.047 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-08T00:08:16.048 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-08T00:08:16.048 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647', 'tag': None, 'wait_for_package': False} 2023-10-08T00:08:16.049 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=08cac0187919fc3a247efdf445f652e2fb30a647 2023-10-08T00:08:16.050 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647', 'tag': None, 'wait_for_package': False} 2023-10-08T00:08:16.051 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=08cac0187919fc3a247efdf445f652e2fb30a647 2023-10-08T00:08:16.199 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/ 2023-10-08T00:08:16.200 INFO:teuthology.task.install.rpm:Package version is 17.2.6-1320.g08cac018 2023-10-08T00:08:16.268 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/ 2023-10-08T00:08:16.268 INFO:teuthology.task.install.rpm:Package version is 17.2.6-1320.g08cac018 2023-10-08T00:08:16.309 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-08T00:08:16.309 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-08T00:08:16.309 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-08T00:08:16.335 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-10-08T00:08:16.336 DEBUG:teuthology.orchestra.run.smithi099:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/08cac0187919fc3a247efdf445f652e2fb30a647/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-08T00:08:16.378 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-08T00:08:16.378 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2023-10-08T00:08:16.378 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-08T00:08:16.404 DEBUG:teuthology.orchestra.run.smithi099:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-08T00:08:16.405 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-10-08T00:08:16.405 DEBUG:teuthology.orchestra.run.smithi191:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/08cac0187919fc3a247efdf445f652e2fb30a647/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-08T00:08:16.470 DEBUG:teuthology.orchestra.run.smithi191:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-08T00:08:16.478 DEBUG:teuthology.orchestra.run.smithi099:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-08T00:08:16.542 INFO:teuthology.orchestra.run.smithi099.stdout:check_obsoletes = 1 2023-10-08T00:08:16.543 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum clean all 2023-10-08T00:08:16.545 DEBUG:teuthology.orchestra.run.smithi191:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-08T00:08:16.609 INFO:teuthology.orchestra.run.smithi191.stdout:check_obsoletes = 1 2023-10-08T00:08:16.610 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum clean all 2023-10-08T00:08:16.934 INFO:teuthology.orchestra.run.smithi099.stdout:Updating Subscription Management repositories. 2023-10-08T00:08:17.000 INFO:teuthology.orchestra.run.smithi191.stdout:Updating Subscription Management repositories. 2023-10-08T00:08:17.274 INFO:teuthology.orchestra.run.smithi099.stdout:44 files removed 2023-10-08T00:08:17.315 INFO:teuthology.orchestra.run.smithi191.stdout:44 files removed 2023-10-08T00:08:17.323 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-10-08T00:08:17.367 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-10-08T00:08:17.671 INFO:teuthology.orchestra.run.smithi099.stdout:Updating Subscription Management repositories. 2023-10-08T00:08:17.718 INFO:teuthology.orchestra.run.smithi191.stdout:Updating Subscription Management repositories. 2023-10-08T00:08:18.490 INFO:teuthology.orchestra.run.smithi099.stdout:ceph packages for x86_64 233 kB/s | 78 kB 00:00 2023-10-08T00:08:18.524 INFO:teuthology.orchestra.run.smithi191.stdout:ceph packages for x86_64 251 kB/s | 78 kB 00:00 2023-10-08T00:08:18.771 INFO:teuthology.orchestra.run.smithi099.stdout:ceph noarch packages 38 kB/s | 9.2 kB 00:00 2023-10-08T00:08:18.803 INFO:teuthology.orchestra.run.smithi191.stdout:ceph noarch packages 38 kB/s | 9.2 kB 00:00 2023-10-08T00:08:19.017 INFO:teuthology.orchestra.run.smithi191.stdout:ceph source packages 7.9 kB/s | 1.5 kB 00:00 2023-10-08T00:08:19.047 INFO:teuthology.orchestra.run.smithi099.stdout:ceph source packages 6.0 kB/s | 1.5 kB 00:00 2023-10-08T00:08:19.126 INFO:teuthology.orchestra.run.smithi191.stdout:Copr repo for python3-asyncssh owned by ceph 43 kB/s | 3.5 kB 00:00 2023-10-08T00:08:19.155 INFO:teuthology.orchestra.run.smithi099.stdout:Copr repo for python3-asyncssh owned by ceph 43 kB/s | 3.5 kB 00:00 2023-10-08T00:08:20.973 INFO:teuthology.orchestra.run.smithi099.stdout:Extra Packages for Enterprise Linux 8.9 MB/s | 16 MB 00:01 2023-10-08T00:08:20.984 INFO:teuthology.orchestra.run.smithi191.stdout:Extra Packages for Enterprise Linux 8.7 MB/s | 16 MB 00:01 2023-10-08T00:08:24.287 INFO:teuthology.orchestra.run.smithi099.stdout:lab-extras 4.0 MB/s | 24 kB 00:00 2023-10-08T00:08:24.307 INFO:teuthology.orchestra.run.smithi191.stdout:lab-extras 4.5 MB/s | 24 kB 00:00 2023-10-08T00:08:24.649 INFO:teuthology.orchestra.run.smithi099.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 117 MB/s | 33 MB 00:00 2023-10-08T00:08:24.669 INFO:teuthology.orchestra.run.smithi191.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 118 MB/s | 33 MB 00:00 2023-10-08T00:08:29.788 INFO:teuthology.orchestra.run.smithi099.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 124 MB/s | 38 MB 00:00 2023-10-08T00:08:29.811 INFO:teuthology.orchestra.run.smithi191.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 120 MB/s | 38 MB 00:00 2023-10-08T00:08:38.107 INFO:teuthology.orchestra.run.smithi099.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-08T00:08:38.107 INFO:teuthology.orchestra.run.smithi099.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-08T00:08:38.118 INFO:teuthology.orchestra.run.smithi099.stdout:Package NetworkManager-1:1.30.0-13.el8_4.x86_64 is already installed. 2023-10-08T00:08:38.119 INFO:teuthology.orchestra.run.smithi099.stdout:Package iproute-5.9.0-4.el8.x86_64 is already installed. 2023-10-08T00:08:38.120 INFO:teuthology.orchestra.run.smithi099.stdout:Package util-linux-2.32.1-27.el8.x86_64 is already installed. 2023-10-08T00:08:38.121 INFO:teuthology.orchestra.run.smithi099.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-08T00:08:38.122 INFO:teuthology.orchestra.run.smithi099.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-08T00:08:38.125 INFO:teuthology.orchestra.run.smithi099.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2023-10-08T00:08:38.126 INFO:teuthology.orchestra.run.smithi099.stdout:Package xfsprogs-5.0.0-8.el8.x86_64 is already installed. 2023-10-08T00:08:38.157 INFO:teuthology.orchestra.run.smithi191.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-08T00:08:38.158 INFO:teuthology.orchestra.run.smithi191.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-08T00:08:38.168 INFO:teuthology.orchestra.run.smithi191.stdout:Package NetworkManager-1:1.30.0-13.el8_4.x86_64 is already installed. 2023-10-08T00:08:38.169 INFO:teuthology.orchestra.run.smithi191.stdout:Package iproute-5.9.0-4.el8.x86_64 is already installed. 2023-10-08T00:08:38.171 INFO:teuthology.orchestra.run.smithi191.stdout:Package util-linux-2.32.1-27.el8.x86_64 is already installed. 2023-10-08T00:08:38.171 INFO:teuthology.orchestra.run.smithi191.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-08T00:08:38.173 INFO:teuthology.orchestra.run.smithi191.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-08T00:08:38.176 INFO:teuthology.orchestra.run.smithi191.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2023-10-08T00:08:38.176 INFO:teuthology.orchestra.run.smithi191.stdout:Package xfsprogs-5.0.0-8.el8.x86_64 is already installed. 2023-10-08T00:08:38.616 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2023-10-08T00:08:38.627 INFO:teuthology.orchestra.run.smithi099.stdout:====================================================================================================================== 2023-10-08T00:08:38.628 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repository Size 2023-10-08T00:08:38.628 INFO:teuthology.orchestra.run.smithi099.stdout:====================================================================================================================== 2023-10-08T00:08:38.628 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2023-10-08T00:08:38.628 INFO:teuthology.orchestra.run.smithi099.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-08T00:08:38.628 INFO:teuthology.orchestra.run.smithi099.stdout: ceph x86_64 2:17.2.6-1320.g08cac018.el8 ceph 6.4 k 2023-10-08T00:08:38.628 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-base x86_64 2:17.2.6-1320.g08cac018.el8 ceph 5.2 M 2023-10-08T00:08:38.628 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-fuse x86_64 2:17.2.6-1320.g08cac018.el8 ceph 876 k 2023-10-08T00:08:38.628 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-1320.g08cac018.el8 ceph 162 k 2023-10-08T00:08:38.628 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr x86_64 2:17.2.6-1320.g08cac018.el8 ceph 1.6 M 2023-10-08T00:08:38.628 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-cephadm noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 121 k 2023-10-08T00:08:38.629 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-dashboard noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 1.7 M 2023-10-08T00:08:38.629 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 7.4 M 2023-10-08T00:08:38.629 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-rook noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 47 k 2023-10-08T00:08:38.629 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-radosgw x86_64 2:17.2.6-1320.g08cac018.el8 ceph 17 M 2023-10-08T00:08:38.629 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-test x86_64 2:17.2.6-1320.g08cac018.el8 ceph 52 M 2023-10-08T00:08:38.629 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-volume noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 267 k 2023-10-08T00:08:38.629 INFO:teuthology.orchestra.run.smithi099.stdout: cephadm noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 80 k 2023-10-08T00:08:38.629 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-mirror x86_64 2:17.2.6-1320.g08cac018.el8 ceph 255 k 2023-10-08T00:08:38.629 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-top noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 18 k 2023-10-08T00:08:38.629 INFO:teuthology.orchestra.run.smithi099.stdout: elfutils-libelf-devel x86_64 0.182-3.el8 rhel-8-for-x86_64-baseos-rpms 59 k 2023-10-08T00:08:38.629 INFO:teuthology.orchestra.run.smithi099.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-08T00:08:38.630 INFO:teuthology.orchestra.run.smithi099.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-08T00:08:38.630 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs-devel x86_64 2:17.2.6-1320.g08cac018.el8 ceph 25 k 2023-10-08T00:08:38.630 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs2 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 797 k 2023-10-08T00:08:38.630 INFO:teuthology.orchestra.run.smithi099.stdout: librados-devel x86_64 2:17.2.6-1320.g08cac018.el8 ceph 130 k 2023-10-08T00:08:38.630 INFO:teuthology.orchestra.run.smithi099.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-08T00:08:38.630 INFO:teuthology.orchestra.run.smithi099.stdout: libuuid-devel x86_64 2.32.1-27.el8 rhel-8-for-x86_64-baseos-rpms 97 k 2023-10-08T00:08:38.630 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-devel x86_64 1:1.1.1g-15.el8_3 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-08T00:08:38.630 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cephfs x86_64 2:17.2.6-1320.g08cac018.el8 ceph 190 k 2023-10-08T00:08:38.630 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rados x86_64 2:17.2.6-1320.g08cac018.el8 ceph 371 k 2023-10-08T00:08:38.630 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rbd x86_64 2:17.2.6-1320.g08cac018.el8 ceph 358 k 2023-10-08T00:08:38.630 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rgw x86_64 2:17.2.6-1320.g08cac018.el8 ceph 107 k 2023-10-08T00:08:38.631 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-fuse x86_64 2:17.2.6-1320.g08cac018.el8 ceph 90 k 2023-10-08T00:08:38.631 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-mirror x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.5 M 2023-10-08T00:08:38.631 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-nbd x86_64 2:17.2.6-1320.g08cac018.el8 ceph 173 k 2023-10-08T00:08:38.631 INFO:teuthology.orchestra.run.smithi099.stdout: xfsprogs-devel x86_64 5.0.0-8.el8 rhel-8-for-x86_64-baseos-rpms 70 k 2023-10-08T00:08:38.631 INFO:teuthology.orchestra.run.smithi099.stdout:Upgrading: 2023-10-08T00:08:38.631 INFO:teuthology.orchestra.run.smithi099.stdout: librados2 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.5 M 2023-10-08T00:08:38.631 INFO:teuthology.orchestra.run.smithi099.stdout: librbd1 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.6 M 2023-10-08T00:08:38.631 INFO:teuthology.orchestra.run.smithi099.stdout:Installing dependencies: 2023-10-08T00:08:38.631 INFO:teuthology.orchestra.run.smithi099.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-08T00:08:38.631 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-common x86_64 2:17.2.6-1320.g08cac018.el8 ceph 23 M 2023-10-08T00:08:38.631 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-grafana-dashboards noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 23 k 2023-10-08T00:08:38.631 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mds x86_64 2:17.2.6-1320.g08cac018.el8 ceph 2.3 M 2023-10-08T00:08:38.632 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-modules-core noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 238 k 2023-10-08T00:08:38.632 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mon x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.9 M 2023-10-08T00:08:38.632 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-osd x86_64 2:17.2.6-1320.g08cac018.el8 ceph 17 M 2023-10-08T00:08:38.632 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-prometheus-alerts noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 14 k 2023-10-08T00:08:38.632 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-selinux x86_64 2:17.2.6-1320.g08cac018.el8 ceph 24 k 2023-10-08T00:08:38.632 INFO:teuthology.orchestra.run.smithi099.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-08T00:08:38.632 INFO:teuthology.orchestra.run.smithi099.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-08T00:08:38.632 INFO:teuthology.orchestra.run.smithi099.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-10-08T00:08:38.632 INFO:teuthology.orchestra.run.smithi099.stdout: keyutils-libs-devel x86_64 1.5.10-6.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-08T00:08:38.632 INFO:teuthology.orchestra.run.smithi099.stdout: krb5-devel x86_64 1.18.2-8.3.el8_4 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-08T00:08:38.632 INFO:teuthology.orchestra.run.smithi099.stdout: libcephsqlite x86_64 2:17.2.6-1320.g08cac018.el8 ceph 174 k 2023-10-08T00:08:38.632 INFO:teuthology.orchestra.run.smithi099.stdout: libcom_err-devel x86_64 1.45.6-1.el8 rhel-8-for-x86_64-baseos-rpms 38 k 2023-10-08T00:08:38.633 INFO:teuthology.orchestra.run.smithi099.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-10-08T00:08:38.633 INFO:teuthology.orchestra.run.smithi099.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-08T00:08:38.633 INFO:teuthology.orchestra.run.smithi099.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-10-08T00:08:38.633 INFO:teuthology.orchestra.run.smithi099.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-08T00:08:38.633 INFO:teuthology.orchestra.run.smithi099.stdout: libradosstriper1 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 507 k 2023-10-08T00:08:38.633 INFO:teuthology.orchestra.run.smithi099.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-10-08T00:08:38.633 INFO:teuthology.orchestra.run.smithi099.stdout: librgw2 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 5.6 M 2023-10-08T00:08:38.633 INFO:teuthology.orchestra.run.smithi099.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-08T00:08:38.633 INFO:teuthology.orchestra.run.smithi099.stdout: libsepol-devel x86_64 2.9-2.el8 rhel-8-for-x86_64-baseos-rpms 86 k 2023-10-08T00:08:38.633 INFO:teuthology.orchestra.run.smithi099.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-08T00:08:38.633 INFO:teuthology.orchestra.run.smithi099.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-08T00:08:38.634 INFO:teuthology.orchestra.run.smithi099.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-08T00:08:38.634 INFO:teuthology.orchestra.run.smithi099.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-08T00:08:38.634 INFO:teuthology.orchestra.run.smithi099.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-10-08T00:08:38.634 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-08T00:08:38.634 INFO:teuthology.orchestra.run.smithi099.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-08T00:08:38.634 INFO:teuthology.orchestra.run.smithi099.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-08T00:08:38.634 INFO:teuthology.orchestra.run.smithi099.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-08T00:08:38.634 INFO:teuthology.orchestra.run.smithi099.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-08T00:08:38.634 INFO:teuthology.orchestra.run.smithi099.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-08T00:08:38.634 INFO:teuthology.orchestra.run.smithi099.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-08T00:08:38.634 INFO:teuthology.orchestra.run.smithi099.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-08T00:08:38.634 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-08T00:08:38.635 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-argparse x86_64 2:17.2.6-1320.g08cac018.el8 ceph 45 k 2023-10-08T00:08:38.635 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-common x86_64 2:17.2.6-1320.g08cac018.el8 ceph 115 k 2023-10-08T00:08:38.635 INFO:teuthology.orchestra.run.smithi099.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-08T00:08:38.635 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-08T00:08:38.635 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-08T00:08:38.635 INFO:teuthology.orchestra.run.smithi099.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-08T00:08:38.635 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-08T00:08:38.635 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-08T00:08:38.635 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-08T00:08:38.635 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-08T00:08:38.635 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-08T00:08:38.635 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-08T00:08:38.636 INFO:teuthology.orchestra.run.smithi099.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-08T00:08:38.636 INFO:teuthology.orchestra.run.smithi099.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-08T00:08:38.636 INFO:teuthology.orchestra.run.smithi099.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-08T00:08:38.636 INFO:teuthology.orchestra.run.smithi099.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-08T00:08:38.636 INFO:teuthology.orchestra.run.smithi099.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-08T00:08:38.636 INFO:teuthology.orchestra.run.smithi099.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-08T00:08:38.636 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-08T00:08:38.636 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-08T00:08:38.636 INFO:teuthology.orchestra.run.smithi099.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-08T00:08:38.636 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-08T00:08:38.636 INFO:teuthology.orchestra.run.smithi099.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-08T00:08:38.637 INFO:teuthology.orchestra.run.smithi099.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-08T00:08:38.637 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-08T00:08:38.637 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-08T00:08:38.637 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-08T00:08:38.637 INFO:teuthology.orchestra.run.smithi099.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-08T00:08:38.637 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-08T00:08:38.637 INFO:teuthology.orchestra.run.smithi099.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-08T00:08:38.637 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-08T00:08:38.637 INFO:teuthology.orchestra.run.smithi099.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-10-08T00:08:38.637 INFO:teuthology.orchestra.run.smithi099.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-08T00:08:38.637 INFO:teuthology.orchestra.run.smithi099.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-08T00:08:38.637 INFO:teuthology.orchestra.run.smithi099.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-08T00:08:38.638 INFO:teuthology.orchestra.run.smithi099.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-08T00:08:38.638 INFO:teuthology.orchestra.run.smithi099.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-08T00:08:38.638 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-08T00:08:38.638 INFO:teuthology.orchestra.run.smithi099.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-08T00:08:38.638 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-08T00:08:38.638 INFO:teuthology.orchestra.run.smithi099.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-08T00:08:38.638 INFO:teuthology.orchestra.run.smithi099.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-08T00:08:38.638 INFO:teuthology.orchestra.run.smithi099.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-10-08T00:08:38.638 INFO:teuthology.orchestra.run.smithi099.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-08T00:08:38.638 INFO:teuthology.orchestra.run.smithi099.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-08T00:08:38.638 INFO:teuthology.orchestra.run.smithi099.stdout: zlib-devel x86_64 1.2.11-17.el8 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-08T00:08:38.639 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:08:38.639 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2023-10-08T00:08:38.639 INFO:teuthology.orchestra.run.smithi099.stdout:====================================================================================================================== 2023-10-08T00:08:38.639 INFO:teuthology.orchestra.run.smithi099.stdout:Install 118 Packages 2023-10-08T00:08:38.639 INFO:teuthology.orchestra.run.smithi099.stdout:Upgrade 2 Packages 2023-10-08T00:08:38.639 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:08:38.639 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 202 M 2023-10-08T00:08:38.640 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2023-10-08T00:08:38.661 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2023-10-08T00:08:38.672 INFO:teuthology.orchestra.run.smithi191.stdout:====================================================================================================================== 2023-10-08T00:08:38.672 INFO:teuthology.orchestra.run.smithi191.stdout: Package Arch Version Repository Size 2023-10-08T00:08:38.672 INFO:teuthology.orchestra.run.smithi191.stdout:====================================================================================================================== 2023-10-08T00:08:38.673 INFO:teuthology.orchestra.run.smithi191.stdout:Installing: 2023-10-08T00:08:38.673 INFO:teuthology.orchestra.run.smithi191.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-08T00:08:38.673 INFO:teuthology.orchestra.run.smithi191.stdout: ceph x86_64 2:17.2.6-1320.g08cac018.el8 ceph 6.4 k 2023-10-08T00:08:38.673 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-base x86_64 2:17.2.6-1320.g08cac018.el8 ceph 5.2 M 2023-10-08T00:08:38.673 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-fuse x86_64 2:17.2.6-1320.g08cac018.el8 ceph 876 k 2023-10-08T00:08:38.673 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-1320.g08cac018.el8 ceph 162 k 2023-10-08T00:08:38.673 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr x86_64 2:17.2.6-1320.g08cac018.el8 ceph 1.6 M 2023-10-08T00:08:38.673 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-cephadm noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 121 k 2023-10-08T00:08:38.673 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-dashboard noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 1.7 M 2023-10-08T00:08:38.673 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 7.4 M 2023-10-08T00:08:38.673 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-rook noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 47 k 2023-10-08T00:08:38.674 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-radosgw x86_64 2:17.2.6-1320.g08cac018.el8 ceph 17 M 2023-10-08T00:08:38.674 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-test x86_64 2:17.2.6-1320.g08cac018.el8 ceph 52 M 2023-10-08T00:08:38.674 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-volume noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 267 k 2023-10-08T00:08:38.674 INFO:teuthology.orchestra.run.smithi191.stdout: cephadm noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 80 k 2023-10-08T00:08:38.674 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-mirror x86_64 2:17.2.6-1320.g08cac018.el8 ceph 255 k 2023-10-08T00:08:38.674 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-top noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 18 k 2023-10-08T00:08:38.674 INFO:teuthology.orchestra.run.smithi191.stdout: elfutils-libelf-devel x86_64 0.182-3.el8 rhel-8-for-x86_64-baseos-rpms 59 k 2023-10-08T00:08:38.674 INFO:teuthology.orchestra.run.smithi191.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-08T00:08:38.674 INFO:teuthology.orchestra.run.smithi191.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-08T00:08:38.674 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs-devel x86_64 2:17.2.6-1320.g08cac018.el8 ceph 25 k 2023-10-08T00:08:38.674 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs2 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 797 k 2023-10-08T00:08:38.675 INFO:teuthology.orchestra.run.smithi191.stdout: librados-devel x86_64 2:17.2.6-1320.g08cac018.el8 ceph 130 k 2023-10-08T00:08:38.675 INFO:teuthology.orchestra.run.smithi191.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-08T00:08:38.675 INFO:teuthology.orchestra.run.smithi191.stdout: libuuid-devel x86_64 2.32.1-27.el8 rhel-8-for-x86_64-baseos-rpms 97 k 2023-10-08T00:08:38.675 INFO:teuthology.orchestra.run.smithi191.stdout: openssl-devel x86_64 1:1.1.1g-15.el8_3 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-08T00:08:38.675 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cephfs x86_64 2:17.2.6-1320.g08cac018.el8 ceph 190 k 2023-10-08T00:08:38.675 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rados x86_64 2:17.2.6-1320.g08cac018.el8 ceph 371 k 2023-10-08T00:08:38.675 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rbd x86_64 2:17.2.6-1320.g08cac018.el8 ceph 358 k 2023-10-08T00:08:38.675 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rgw x86_64 2:17.2.6-1320.g08cac018.el8 ceph 107 k 2023-10-08T00:08:38.675 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-fuse x86_64 2:17.2.6-1320.g08cac018.el8 ceph 90 k 2023-10-08T00:08:38.675 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-mirror x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.5 M 2023-10-08T00:08:38.675 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-nbd x86_64 2:17.2.6-1320.g08cac018.el8 ceph 173 k 2023-10-08T00:08:38.675 INFO:teuthology.orchestra.run.smithi191.stdout: xfsprogs-devel x86_64 5.0.0-8.el8 rhel-8-for-x86_64-baseos-rpms 70 k 2023-10-08T00:08:38.676 INFO:teuthology.orchestra.run.smithi191.stdout:Upgrading: 2023-10-08T00:08:38.676 INFO:teuthology.orchestra.run.smithi191.stdout: librados2 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.5 M 2023-10-08T00:08:38.676 INFO:teuthology.orchestra.run.smithi191.stdout: librbd1 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.6 M 2023-10-08T00:08:38.676 INFO:teuthology.orchestra.run.smithi191.stdout:Installing dependencies: 2023-10-08T00:08:38.676 INFO:teuthology.orchestra.run.smithi191.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-08T00:08:38.676 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-common x86_64 2:17.2.6-1320.g08cac018.el8 ceph 23 M 2023-10-08T00:08:38.676 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-grafana-dashboards noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 23 k 2023-10-08T00:08:38.676 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mds x86_64 2:17.2.6-1320.g08cac018.el8 ceph 2.3 M 2023-10-08T00:08:38.676 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-modules-core noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 238 k 2023-10-08T00:08:38.676 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mon x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.9 M 2023-10-08T00:08:38.676 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-osd x86_64 2:17.2.6-1320.g08cac018.el8 ceph 17 M 2023-10-08T00:08:38.677 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-prometheus-alerts noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 14 k 2023-10-08T00:08:38.677 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-selinux x86_64 2:17.2.6-1320.g08cac018.el8 ceph 24 k 2023-10-08T00:08:38.677 INFO:teuthology.orchestra.run.smithi191.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-08T00:08:38.677 INFO:teuthology.orchestra.run.smithi191.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-08T00:08:38.677 INFO:teuthology.orchestra.run.smithi191.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-10-08T00:08:38.677 INFO:teuthology.orchestra.run.smithi191.stdout: keyutils-libs-devel x86_64 1.5.10-6.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-08T00:08:38.677 INFO:teuthology.orchestra.run.smithi191.stdout: krb5-devel x86_64 1.18.2-8.3.el8_4 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-08T00:08:38.677 INFO:teuthology.orchestra.run.smithi191.stdout: libcephsqlite x86_64 2:17.2.6-1320.g08cac018.el8 ceph 174 k 2023-10-08T00:08:38.677 INFO:teuthology.orchestra.run.smithi191.stdout: libcom_err-devel x86_64 1.45.6-1.el8 rhel-8-for-x86_64-baseos-rpms 38 k 2023-10-08T00:08:38.677 INFO:teuthology.orchestra.run.smithi191.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-10-08T00:08:38.677 INFO:teuthology.orchestra.run.smithi191.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-08T00:08:38.678 INFO:teuthology.orchestra.run.smithi191.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-10-08T00:08:38.678 INFO:teuthology.orchestra.run.smithi191.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-08T00:08:38.678 INFO:teuthology.orchestra.run.smithi191.stdout: libradosstriper1 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 507 k 2023-10-08T00:08:38.678 INFO:teuthology.orchestra.run.smithi191.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-10-08T00:08:38.678 INFO:teuthology.orchestra.run.smithi191.stdout: librgw2 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 5.6 M 2023-10-08T00:08:38.678 INFO:teuthology.orchestra.run.smithi191.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-08T00:08:38.678 INFO:teuthology.orchestra.run.smithi191.stdout: libsepol-devel x86_64 2.9-2.el8 rhel-8-for-x86_64-baseos-rpms 86 k 2023-10-08T00:08:38.678 INFO:teuthology.orchestra.run.smithi191.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-08T00:08:38.678 INFO:teuthology.orchestra.run.smithi191.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-08T00:08:38.678 INFO:teuthology.orchestra.run.smithi191.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-08T00:08:38.678 INFO:teuthology.orchestra.run.smithi191.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-08T00:08:38.679 INFO:teuthology.orchestra.run.smithi191.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-10-08T00:08:38.679 INFO:teuthology.orchestra.run.smithi191.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-08T00:08:38.679 INFO:teuthology.orchestra.run.smithi191.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-08T00:08:38.679 INFO:teuthology.orchestra.run.smithi191.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-08T00:08:38.679 INFO:teuthology.orchestra.run.smithi191.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-08T00:08:38.679 INFO:teuthology.orchestra.run.smithi191.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-08T00:08:38.679 INFO:teuthology.orchestra.run.smithi191.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-08T00:08:38.679 INFO:teuthology.orchestra.run.smithi191.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-08T00:08:38.679 INFO:teuthology.orchestra.run.smithi191.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-08T00:08:38.679 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-08T00:08:38.679 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-argparse x86_64 2:17.2.6-1320.g08cac018.el8 ceph 45 k 2023-10-08T00:08:38.680 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-common x86_64 2:17.2.6-1320.g08cac018.el8 ceph 115 k 2023-10-08T00:08:38.680 INFO:teuthology.orchestra.run.smithi191.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-08T00:08:38.680 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-08T00:08:38.680 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-08T00:08:38.680 INFO:teuthology.orchestra.run.smithi191.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-08T00:08:38.680 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-08T00:08:38.680 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-08T00:08:38.680 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-08T00:08:38.680 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-08T00:08:38.680 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-08T00:08:38.680 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-08T00:08:38.681 INFO:teuthology.orchestra.run.smithi191.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-08T00:08:38.681 INFO:teuthology.orchestra.run.smithi191.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-08T00:08:38.681 INFO:teuthology.orchestra.run.smithi191.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-08T00:08:38.681 INFO:teuthology.orchestra.run.smithi191.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-08T00:08:38.681 INFO:teuthology.orchestra.run.smithi191.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-08T00:08:38.681 INFO:teuthology.orchestra.run.smithi191.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-08T00:08:38.681 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-08T00:08:38.681 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-08T00:08:38.681 INFO:teuthology.orchestra.run.smithi191.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-08T00:08:38.681 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-08T00:08:38.681 INFO:teuthology.orchestra.run.smithi191.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-08T00:08:38.681 INFO:teuthology.orchestra.run.smithi191.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-08T00:08:38.682 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-08T00:08:38.682 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-08T00:08:38.682 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-08T00:08:38.682 INFO:teuthology.orchestra.run.smithi191.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-08T00:08:38.682 INFO:teuthology.orchestra.run.smithi191.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-08T00:08:38.682 INFO:teuthology.orchestra.run.smithi191.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-08T00:08:38.682 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-08T00:08:38.682 INFO:teuthology.orchestra.run.smithi191.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-10-08T00:08:38.682 INFO:teuthology.orchestra.run.smithi191.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-08T00:08:38.682 INFO:teuthology.orchestra.run.smithi191.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-08T00:08:38.682 INFO:teuthology.orchestra.run.smithi191.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-08T00:08:38.683 INFO:teuthology.orchestra.run.smithi191.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-08T00:08:38.683 INFO:teuthology.orchestra.run.smithi191.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-08T00:08:38.683 INFO:teuthology.orchestra.run.smithi191.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-08T00:08:38.683 INFO:teuthology.orchestra.run.smithi191.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-08T00:08:38.683 INFO:teuthology.orchestra.run.smithi191.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-08T00:08:38.683 INFO:teuthology.orchestra.run.smithi191.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-08T00:08:38.683 INFO:teuthology.orchestra.run.smithi191.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-08T00:08:38.683 INFO:teuthology.orchestra.run.smithi191.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-10-08T00:08:38.683 INFO:teuthology.orchestra.run.smithi191.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-08T00:08:38.683 INFO:teuthology.orchestra.run.smithi191.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-08T00:08:38.683 INFO:teuthology.orchestra.run.smithi191.stdout: zlib-devel x86_64 1.2.11-17.el8 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-08T00:08:38.684 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T00:08:38.684 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction Summary 2023-10-08T00:08:38.684 INFO:teuthology.orchestra.run.smithi191.stdout:====================================================================================================================== 2023-10-08T00:08:38.684 INFO:teuthology.orchestra.run.smithi191.stdout:Install 118 Packages 2023-10-08T00:08:38.684 INFO:teuthology.orchestra.run.smithi191.stdout:Upgrade 2 Packages 2023-10-08T00:08:38.684 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T00:08:38.684 INFO:teuthology.orchestra.run.smithi191.stdout:Total download size: 202 M 2023-10-08T00:08:38.685 INFO:teuthology.orchestra.run.smithi191.stdout:Downloading Packages: 2023-10-08T00:08:39.154 INFO:teuthology.orchestra.run.smithi099.stdout:(1/120): ceph-17.2.6-1320.g08cac018.el8.x86_64. 42 kB/s | 6.4 kB 00:00 2023-10-08T00:08:39.192 INFO:teuthology.orchestra.run.smithi191.stdout:(1/120): ceph-17.2.6-1320.g08cac018.el8.x86_64. 39 kB/s | 6.4 kB 00:00 2023-10-08T00:08:39.689 INFO:teuthology.orchestra.run.smithi099.stdout:(2/120): ceph-fuse-17.2.6-1320.g08cac018.el8.x8 1.6 MB/s | 876 kB 00:00 2023-10-08T00:08:39.735 INFO:teuthology.orchestra.run.smithi191.stdout:(2/120): ceph-fuse-17.2.6-1320.g08cac018.el8.x8 1.6 MB/s | 876 kB 00:00 2023-10-08T00:08:39.790 INFO:teuthology.orchestra.run.smithi099.stdout:(3/120): ceph-immutable-object-cache-17.2.6-132 1.6 MB/s | 162 kB 00:00 2023-10-08T00:08:39.841 INFO:teuthology.orchestra.run.smithi191.stdout:(3/120): ceph-immutable-object-cache-17.2.6-132 1.5 MB/s | 162 kB 00:00 2023-10-08T00:08:40.648 INFO:teuthology.orchestra.run.smithi099.stdout:(4/120): ceph-common-17.2.6-1320.g08cac018.el8. 14 MB/s | 23 MB 00:01 2023-10-08T00:08:40.674 INFO:teuthology.orchestra.run.smithi191.stdout:(4/120): ceph-mds-17.2.6-1320.g08cac018.el8.x86 2.8 MB/s | 2.3 MB 00:00 2023-10-08T00:08:40.702 INFO:teuthology.orchestra.run.smithi099.stdout:(5/120): ceph-mds-17.2.6-1320.g08cac018.el8.x86 2.5 MB/s | 2.3 MB 00:00 2023-10-08T00:08:40.758 INFO:teuthology.orchestra.run.smithi099.stdout:(6/120): ceph-mgr-17.2.6-1320.g08cac018.el8.x86 14 MB/s | 1.6 MB 00:00 2023-10-08T00:08:41.222 INFO:teuthology.orchestra.run.smithi099.stdout:(7/120): ceph-base-17.2.6-1320.g08cac018.el8.x8 2.3 MB/s | 5.2 MB 00:02 2023-10-08T00:08:41.270 INFO:teuthology.orchestra.run.smithi191.stdout:(5/120): ceph-mgr-17.2.6-1320.g08cac018.el8.x86 2.6 MB/s | 1.6 MB 00:00 2023-10-08T00:08:41.304 INFO:teuthology.orchestra.run.smithi191.stdout:(6/120): ceph-base-17.2.6-1320.g08cac018.el8.x8 2.3 MB/s | 5.2 MB 00:02 2023-10-08T00:08:41.801 INFO:teuthology.orchestra.run.smithi099.stdout:(8/120): ceph-mon-17.2.6-1320.g08cac018.el8.x86 3.6 MB/s | 3.9 MB 00:01 2023-10-08T00:08:41.878 INFO:teuthology.orchestra.run.smithi099.stdout:(9/120): ceph-selinux-17.2.6-1320.g08cac018.el8 315 kB/s | 24 kB 00:00 2023-10-08T00:08:42.138 INFO:teuthology.orchestra.run.smithi099.stdout:(10/120): ceph-osd-17.2.6-1320.g08cac018.el8.x8 12 MB/s | 17 MB 00:01 2023-10-08T00:08:42.161 INFO:teuthology.orchestra.run.smithi099.stdout:(11/120): cephfs-mirror-17.2.6-1320.g08cac018.e 11 MB/s | 255 kB 00:00 2023-10-08T00:08:42.185 INFO:teuthology.orchestra.run.smithi099.stdout:(12/120): libcephfs-devel-17.2.6-1320.g08cac018 1.0 MB/s | 25 kB 00:00 2023-10-08T00:08:42.247 INFO:teuthology.orchestra.run.smithi099.stdout:(13/120): libcephfs2-17.2.6-1320.g08cac018.el8. 13 MB/s | 797 kB 00:00 2023-10-08T00:08:42.293 INFO:teuthology.orchestra.run.smithi099.stdout:(14/120): libcephsqlite-17.2.6-1320.g08cac018.e 3.8 MB/s | 174 kB 00:00 2023-10-08T00:08:42.340 INFO:teuthology.orchestra.run.smithi099.stdout:(15/120): librados-devel-17.2.6-1320.g08cac018. 2.7 MB/s | 130 kB 00:00 2023-10-08T00:08:42.388 INFO:teuthology.orchestra.run.smithi099.stdout:(16/120): libradosstriper1-17.2.6-1320.g08cac01 10 MB/s | 507 kB 00:00 2023-10-08T00:08:42.898 INFO:teuthology.orchestra.run.smithi191.stdout:(7/120): ceph-mon-17.2.6-1320.g08cac018.el8.x86 2.4 MB/s | 3.9 MB 00:01 2023-10-08T00:08:43.007 INFO:teuthology.orchestra.run.smithi099.stdout:(17/120): librgw2-17.2.6-1320.g08cac018.el8.x86 9.0 MB/s | 5.6 MB 00:00 2023-10-08T00:08:43.045 INFO:teuthology.orchestra.run.smithi099.stdout:(18/120): python3-ceph-argparse-17.2.6-1320.g08 1.1 MB/s | 45 kB 00:00 2023-10-08T00:08:43.090 INFO:teuthology.orchestra.run.smithi099.stdout:(19/120): python3-ceph-common-17.2.6-1320.g08ca 2.6 MB/s | 115 kB 00:00 2023-10-08T00:08:43.136 INFO:teuthology.orchestra.run.smithi099.stdout:(20/120): python3-cephfs-17.2.6-1320.g08cac018. 4.0 MB/s | 190 kB 00:00 2023-10-08T00:08:43.192 INFO:teuthology.orchestra.run.smithi099.stdout:(21/120): python3-rados-17.2.6-1320.g08cac018.e 6.5 MB/s | 371 kB 00:00 2023-10-08T00:08:43.253 INFO:teuthology.orchestra.run.smithi099.stdout:(22/120): python3-rbd-17.2.6-1320.g08cac018.el8 5.7 MB/s | 358 kB 00:00 2023-10-08T00:08:43.305 INFO:teuthology.orchestra.run.smithi099.stdout:(23/120): python3-rgw-17.2.6-1320.g08cac018.el8 2.0 MB/s | 107 kB 00:00 2023-10-08T00:08:43.349 INFO:teuthology.orchestra.run.smithi099.stdout:(24/120): rbd-fuse-17.2.6-1320.g08cac018.el8.x8 2.0 MB/s | 90 kB 00:00 2023-10-08T00:08:43.772 INFO:teuthology.orchestra.run.smithi099.stdout:(25/120): rbd-mirror-17.2.6-1320.g08cac018.el8. 8.2 MB/s | 3.5 MB 00:00 2023-10-08T00:08:43.804 INFO:teuthology.orchestra.run.smithi099.stdout:(26/120): rbd-nbd-17.2.6-1320.g08cac018.el8.x86 5.4 MB/s | 173 kB 00:00 2023-10-08T00:08:43.825 INFO:teuthology.orchestra.run.smithi099.stdout:(27/120): ceph-grafana-dashboards-17.2.6-1320.g 1.1 MB/s | 23 kB 00:00 2023-10-08T00:08:43.849 INFO:teuthology.orchestra.run.smithi099.stdout:(28/120): ceph-mgr-cephadm-17.2.6-1320.g08cac01 4.9 MB/s | 121 kB 00:00 2023-10-08T00:08:43.884 INFO:teuthology.orchestra.run.smithi191.stdout:(8/120): ceph-osd-17.2.6-1320.g08cac018.el8.x86 6.5 MB/s | 17 MB 00:02 2023-10-08T00:08:43.949 INFO:teuthology.orchestra.run.smithi191.stdout:(9/120): ceph-selinux-17.2.6-1320.g08cac018.el8 375 kB/s | 24 kB 00:00 2023-10-08T00:08:43.992 INFO:teuthology.orchestra.run.smithi099.stdout:(29/120): ceph-mgr-dashboard-17.2.6-1320.g08cac 12 MB/s | 1.7 MB 00:00 2023-10-08T00:08:44.896 INFO:teuthology.orchestra.run.smithi099.stdout:(30/120): ceph-mgr-diskprediction-local-17.2.6- 8.2 MB/s | 7.4 MB 00:00 2023-10-08T00:08:44.951 INFO:teuthology.orchestra.run.smithi099.stdout:(31/120): ceph-mgr-modules-core-17.2.6-1320.g08 4.2 MB/s | 238 kB 00:00 2023-10-08T00:08:44.998 INFO:teuthology.orchestra.run.smithi099.stdout:(32/120): ceph-mgr-rook-17.2.6-1320.g08cac018.e 1.0 MB/s | 47 kB 00:00 2023-10-08T00:08:45.032 INFO:teuthology.orchestra.run.smithi099.stdout:(33/120): ceph-prometheus-alerts-17.2.6-1320.g0 432 kB/s | 14 kB 00:00 2023-10-08T00:08:45.066 INFO:teuthology.orchestra.run.smithi099.stdout:(34/120): ceph-volume-17.2.6-1320.g08cac018.el8 7.6 MB/s | 267 kB 00:00 2023-10-08T00:08:45.090 INFO:teuthology.orchestra.run.smithi099.stdout:(35/120): cephadm-17.2.6-1320.g08cac018.el8.noa 3.4 MB/s | 80 kB 00:00 2023-10-08T00:08:45.111 INFO:teuthology.orchestra.run.smithi099.stdout:(36/120): cephfs-top-17.2.6-1320.g08cac018.el8. 871 kB/s | 18 kB 00:00 2023-10-08T00:08:45.922 INFO:teuthology.orchestra.run.smithi099.stdout:(37/120): gperftools-libs-2.7-9.el8.x86_64.rpm 378 kB/s | 306 kB 00:00 2023-10-08T00:08:46.071 INFO:teuthology.orchestra.run.smithi099.stdout:(38/120): liboath-2.6.2-3.el8.x86_64.rpm 395 kB/s | 59 kB 00:00 2023-10-08T00:08:46.288 INFO:teuthology.orchestra.run.smithi099.stdout:(39/120): libunwind-1.3.1-3.el8.x86_64.rpm 345 kB/s | 75 kB 00:00 2023-10-08T00:08:46.647 INFO:teuthology.orchestra.run.smithi099.stdout:(40/120): python3-asyncssh-2.7.0-2.el8.noarch.r 1.3 MB/s | 476 kB 00:00 2023-10-08T00:08:46.757 INFO:teuthology.orchestra.run.smithi099.stdout:(41/120): ceph-radosgw-17.2.6-1320.g08cac018.el 3.0 MB/s | 17 MB 00:05 2023-10-08T00:08:46.804 INFO:teuthology.orchestra.run.smithi099.stdout:(42/120): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 279 kB/s | 44 kB 00:00 2023-10-08T00:08:46.976 INFO:teuthology.orchestra.run.smithi099.stdout:(43/120): python3-cachetools-3.1.1-4.el8.noarch 191 kB/s | 33 kB 00:00 2023-10-08T00:08:47.120 INFO:teuthology.orchestra.run.smithi099.stdout:(44/120): python3-certifi-2018.10.15-7.el8.noar 114 kB/s | 16 kB 00:00 2023-10-08T00:08:47.324 INFO:teuthology.orchestra.run.smithi099.stdout:(45/120): python3-cheroot-8.5.2-1.el8.noarch.rp 847 kB/s | 173 kB 00:00 2023-10-08T00:08:47.396 INFO:teuthology.orchestra.run.smithi099.stdout:(46/120): python3-beautifulsoup4-4.6.3-2.el8.1. 290 kB/s | 185 kB 00:00 2023-10-08T00:08:47.537 INFO:teuthology.orchestra.run.smithi099.stdout:(47/120): python3-google-auth-1.1.1-10.el8.noar 578 kB/s | 81 kB 00:00 2023-10-08T00:08:47.627 INFO:teuthology.orchestra.run.smithi099.stdout:(48/120): python3-cherrypy-18.4.0-1.el8.noarch. 1.2 MB/s | 384 kB 00:00 2023-10-08T00:08:47.689 INFO:teuthology.orchestra.run.smithi099.stdout:(49/120): python3-jaraco-6.2-6.el8.noarch.rpm 73 kB/s | 11 kB 00:00 2023-10-08T00:08:47.771 INFO:teuthology.orchestra.run.smithi099.stdout:(50/120): python3-jaraco-functools-2.0-4.el8.no 122 kB/s | 18 kB 00:00 2023-10-08T00:08:47.931 INFO:teuthology.orchestra.run.smithi099.stdout:(51/120): python3-logutils-0.3.5-11.el8.noarch. 304 kB/s | 49 kB 00:00 2023-10-08T00:08:48.070 INFO:teuthology.orchestra.run.smithi099.stdout:(52/120): python3-more-itertools-7.2.0-3.el8.no 423 kB/s | 59 kB 00:00 2023-10-08T00:08:48.216 INFO:teuthology.orchestra.run.smithi099.stdout:(53/120): python3-natsort-7.1.1-2.el8.noarch.rp 414 kB/s | 60 kB 00:00 2023-10-08T00:08:48.262 INFO:teuthology.orchestra.run.smithi099.stdout:(54/120): python3-kubernetes-11.0.0-6.el8.noarc 2.0 MB/s | 1.2 MB 00:00 2023-10-08T00:08:48.395 INFO:teuthology.orchestra.run.smithi099.stdout:(55/120): python3-portend-2.6-1.el8.noarch.rpm 117 kB/s | 16 kB 00:00 2023-10-08T00:08:48.498 INFO:teuthology.orchestra.run.smithi099.stdout:(56/120): python3-pecan-1.3.2-9.el8.noarch.rpm 1.0 MB/s | 283 kB 00:00 2023-10-08T00:08:48.539 INFO:teuthology.orchestra.run.smithi099.stdout:(57/120): python3-repoze-lru-0.7-6.el8.noarch.r 235 kB/s | 34 kB 00:00 2023-10-08T00:08:48.678 INFO:teuthology.orchestra.run.smithi099.stdout:(58/120): python3-rsa-4.9-2.el8.noarch.rpm 460 kB/s | 64 kB 00:00 2023-10-08T00:08:48.680 INFO:teuthology.orchestra.run.smithi191.stdout:(10/120): ceph-radosgw-17.2.6-1320.g08cac018.el 2.9 MB/s | 17 MB 00:05 2023-10-08T00:08:48.705 INFO:teuthology.orchestra.run.smithi099.stdout:(59/120): python3-routes-2.4.1-12.el8.noarch.rp 951 kB/s | 196 kB 00:00 2023-10-08T00:08:48.793 INFO:teuthology.orchestra.run.smithi191.stdout:(11/120): cephfs-mirror-17.2.6-1320.g08cac018.e 2.2 MB/s | 255 kB 00:00 2023-10-08T00:08:48.807 INFO:teuthology.orchestra.run.smithi099.stdout:(60/120): python3-singledispatch-3.4.0.3-18.el8 234 kB/s | 24 kB 00:00 2023-10-08T00:08:48.815 INFO:teuthology.orchestra.run.smithi099.stdout:(61/120): python3-simplegeneric-0.8.1-17.el8.no 137 kB/s | 19 kB 00:00 2023-10-08T00:08:48.848 INFO:teuthology.orchestra.run.smithi191.stdout:(12/120): libcephfs-devel-17.2.6-1320.g08cac018 450 kB/s | 25 kB 00:00 2023-10-08T00:08:48.938 INFO:teuthology.orchestra.run.smithi099.stdout:(62/120): python3-tempora-1.14.1-5.el8.noarch.r 224 kB/s | 29 kB 00:00 2023-10-08T00:08:48.942 INFO:teuthology.orchestra.run.smithi099.stdout:(63/120): python3-trustme-0.6.0-4.el8.noarch.rp 217 kB/s | 27 kB 00:00 2023-10-08T00:08:49.123 INFO:teuthology.orchestra.run.smithi191.stdout:(13/120): libcephfs2-17.2.6-1320.g08cac018.el8. 2.8 MB/s | 797 kB 00:00 2023-10-08T00:08:49.148 INFO:teuthology.orchestra.run.smithi099.stdout:(64/120): python3-webob-1.8.5-1.el8.1.noarch.rp 1.2 MB/s | 251 kB 00:00 2023-10-08T00:08:49.186 INFO:teuthology.orchestra.run.smithi099.stdout:(65/120): python3-waitress-1.4.3-1.el8.noarch.r 1.0 MB/s | 255 kB 00:00 2023-10-08T00:08:49.207 INFO:teuthology.orchestra.run.smithi191.stdout:(14/120): libcephsqlite-17.2.6-1320.g08cac018.e 2.0 MB/s | 174 kB 00:00 2023-10-08T00:08:49.245 INFO:teuthology.orchestra.run.smithi191.stdout:(15/120): librados-devel-17.2.6-1320.g08cac018. 3.4 MB/s | 130 kB 00:00 2023-10-08T00:08:49.365 INFO:teuthology.orchestra.run.smithi099.stdout:(66/120): python3-webtest-2.0.33-1.el8.noarch.r 478 kB/s | 85 kB 00:00 2023-10-08T00:08:49.402 INFO:teuthology.orchestra.run.smithi191.stdout:(16/120): libradosstriper1-17.2.6-1320.g08cac01 3.1 MB/s | 507 kB 00:00 2023-10-08T00:08:49.429 INFO:teuthology.orchestra.run.smithi099.stdout:(67/120): python3-websocket-client-0.56.0-5.el8 218 kB/s | 61 kB 00:00 2023-10-08T00:08:49.507 INFO:teuthology.orchestra.run.smithi099.stdout:(68/120): python3-zc-lockfile-2.0-2.el8.noarch. 161 kB/s | 23 kB 00:00 2023-10-08T00:08:49.514 INFO:teuthology.orchestra.run.smithi099.stdout:(69/120): xmlstarlet-1.6.1-20.el8.x86_64.rpm 13 MB/s | 70 kB 00:00 2023-10-08T00:08:49.895 INFO:teuthology.orchestra.run.smithi099.stdout:(70/120): ceph-test-17.2.6-1320.g08cac018.el8.x 6.4 MB/s | 52 MB 00:08 2023-10-08T00:08:49.929 INFO:teuthology.orchestra.run.smithi099.stdout:(71/120): python3-prettytable-0.7.2-14.el8.noar 1.3 MB/s | 44 kB 00:00 2023-10-08T00:08:49.944 INFO:teuthology.orchestra.run.smithi099.stdout:(72/120): python3-jsonpatch-1.21-2.el8.noarch.r 1.7 MB/s | 27 kB 00:00 2023-10-08T00:08:49.961 INFO:teuthology.orchestra.run.smithi099.stdout:(73/120): jq-1.5-12.el8.x86_64.rpm 9.6 MB/s | 161 kB 00:00 2023-10-08T00:08:49.966 INFO:teuthology.orchestra.run.smithi191.stdout:(17/120): ceph-test-17.2.6-1320.g08cac018.el8.x 8.6 MB/s | 52 MB 00:06 2023-10-08T00:08:49.979 INFO:teuthology.orchestra.run.smithi099.stdout:(74/120): flex-2.6.1-9.el8.x86_64.rpm 17 MB/s | 320 kB 00:00 2023-10-08T00:08:49.999 INFO:teuthology.orchestra.run.smithi099.stdout:(75/120): python3-werkzeug-0.12.2-4.el8.noarch. 23 MB/s | 457 kB 00:00 2023-10-08T00:08:50.023 INFO:teuthology.orchestra.run.smithi191.stdout:(18/120): python3-ceph-argparse-17.2.6-1320.g08 794 kB/s | 45 kB 00:00 2023-10-08T00:08:50.091 INFO:teuthology.orchestra.run.smithi191.stdout:(19/120): python3-ceph-common-17.2.6-1320.g08ca 1.7 MB/s | 115 kB 00:00 2023-10-08T00:08:50.164 INFO:teuthology.orchestra.run.smithi099.stdout:(76/120): python3-scipy-1.0.0-20.module+el8.1.0 22 MB/s | 15 MB 00:00 2023-10-08T00:08:50.185 INFO:teuthology.orchestra.run.smithi099.stdout:(77/120): python3-babel-2.5.1-5.el8.noarch.rpm 26 MB/s | 4.8 MB 00:00 2023-10-08T00:08:50.186 INFO:teuthology.orchestra.run.smithi099.stdout:(78/120): python3-markupsafe-0.23-19.el8.x86_64 1.8 MB/s | 39 kB 00:00 2023-10-08T00:08:50.193 INFO:teuthology.orchestra.run.smithi099.stdout:(79/120): thrift-0.13.0-2.el8.x86_64.rpm 2.3 MB/s | 1.7 MB 00:00 2023-10-08T00:08:50.203 INFO:teuthology.orchestra.run.smithi099.stdout:(80/120): lttng-ust-2.8.1-11.el8.x86_64.rpm 15 MB/s | 259 kB 00:00 2023-10-08T00:08:50.207 INFO:teuthology.orchestra.run.smithi099.stdout:(81/120): bison-3.0.4-10.el8.x86_64.rpm 31 MB/s | 688 kB 00:00 2023-10-08T00:08:50.217 INFO:teuthology.orchestra.run.smithi099.stdout:(82/120): python3-pyasn1-0.3.7-6.el8.noarch.rpm 9.3 MB/s | 126 kB 00:00 2023-10-08T00:08:50.231 INFO:teuthology.orchestra.run.smithi099.stdout:(83/120): python3-numpy-f2py-1.14.3-9.el8.x86_6 5.9 MB/s | 225 kB 00:00 2023-10-08T00:08:50.234 INFO:teuthology.orchestra.run.smithi099.stdout:(84/120): oniguruma-6.8.2-2.el8.x86_64.rpm 11 MB/s | 187 kB 00:00 2023-10-08T00:08:50.251 INFO:teuthology.orchestra.run.smithi099.stdout:(85/120): socat-1.7.3.3-2.el8.x86_64.rpm 18 MB/s | 302 kB 00:00 2023-10-08T00:08:50.256 INFO:teuthology.orchestra.run.smithi099.stdout:(86/120): python3-jinja2-2.10.1-2.el8_0.noarch. 21 MB/s | 538 kB 00:00 2023-10-08T00:08:50.265 INFO:teuthology.orchestra.run.smithi191.stdout:(20/120): ceph-common-17.2.6-1320.g08cac018.el8 2.1 MB/s | 23 MB 00:11 2023-10-08T00:08:50.275 INFO:teuthology.orchestra.run.smithi099.stdout:(87/120): python3-numpy-1.14.3-9.el8.x86_64.rpm 60 MB/s | 3.7 MB 00:00 2023-10-08T00:08:50.277 INFO:teuthology.orchestra.run.smithi099.stdout:(88/120): python3-mako-1.0.6-13.el8.noarch.rpm 6.0 MB/s | 157 kB 00:00 2023-10-08T00:08:50.277 INFO:teuthology.orchestra.run.smithi099.stdout:(89/120): python3-pyasn1-modules-0.3.7-6.el8.no 5.0 MB/s | 110 kB 00:00 2023-10-08T00:08:50.292 INFO:teuthology.orchestra.run.smithi099.stdout:(90/120): librdkafka-0.11.4-1.el8.x86_64.rpm 20 MB/s | 353 kB 00:00 2023-10-08T00:08:50.293 INFO:teuthology.orchestra.run.smithi099.stdout:(91/120): python3-jsonpointer-1.10-11.el8.noarc 1.3 MB/s | 20 kB 00:00 2023-10-08T00:08:50.300 INFO:teuthology.orchestra.run.smithi099.stdout:(92/120): libtool-2.4.6-25.el8.x86_64.rpm 31 MB/s | 709 kB 00:00 2023-10-08T00:08:50.308 INFO:teuthology.orchestra.run.smithi099.stdout:(93/120): python3-pytz-2017.2-9.el8.noarch.rpm 3.5 MB/s | 54 kB 00:00 2023-10-08T00:08:50.334 INFO:teuthology.orchestra.run.smithi099.stdout:(94/120): automake-1.16.1-7.el8.noarch.rpm 28 MB/s | 713 kB 00:00 2023-10-08T00:08:50.340 INFO:teuthology.orchestra.run.smithi191.stdout:(21/120): python3-cephfs-17.2.6-1320.g08cac018. 766 kB/s | 190 kB 00:00 2023-10-08T00:08:50.346 INFO:teuthology.orchestra.run.smithi099.stdout:(95/120): libaio-devel-0.3.112-1.el8.x86_64.rpm 1.4 MB/s | 19 kB 00:00 2023-10-08T00:08:50.358 INFO:teuthology.orchestra.run.smithi099.stdout:(96/120): python3-jwt-1.6.1-2.el8.noarch.rpm 3.4 MB/s | 43 kB 00:00 2023-10-08T00:08:50.374 INFO:teuthology.orchestra.run.smithi099.stdout:(97/120): openblas-threads-0.3.12-1.el8.x86_64. 59 MB/s | 4.7 MB 00:00 2023-10-08T00:08:50.376 INFO:teuthology.orchestra.run.smithi099.stdout:(98/120): python3-oauthlib-2.1.0-1.el8.noarch.r 8.6 MB/s | 155 kB 00:00 2023-10-08T00:08:50.394 INFO:teuthology.orchestra.run.smithi099.stdout:(99/120): openblas-0.3.12-1.el8.x86_64.rpm 49 MB/s | 4.6 MB 00:00 2023-10-08T00:08:50.394 INFO:teuthology.orchestra.run.smithi099.stdout:(100/120): keyutils-libs-devel-1.5.10-6.el8.x86 2.6 MB/s | 48 kB 00:00 2023-10-08T00:08:50.397 INFO:teuthology.orchestra.run.smithi099.stdout:(101/120): pcre2-utf32-10.32-2.el8.x86_64.rpm 9.5 MB/s | 220 kB 00:00 2023-10-08T00:08:50.408 INFO:teuthology.orchestra.run.smithi099.stdout:(102/120): python3-requests-oauthlib-1.0.0-1.el 3.0 MB/s | 43 kB 00:00 2023-10-08T00:08:50.412 INFO:teuthology.orchestra.run.smithi099.stdout:(103/120): pcre2-utf16-10.32-2.el8.x86_64.rpm 12 MB/s | 229 kB 00:00 2023-10-08T00:08:50.413 INFO:teuthology.orchestra.run.smithi191.stdout:(22/120): python3-rados-17.2.6-1320.g08cac018.e 2.4 MB/s | 371 kB 00:00 2023-10-08T00:08:50.416 INFO:teuthology.orchestra.run.smithi191.stdout:(23/120): python3-rbd-17.2.6-1320.g08cac018.el8 4.6 MB/s | 358 kB 00:00 2023-10-08T00:08:50.425 INFO:teuthology.orchestra.run.smithi099.stdout:(104/120): libverto-devel-0.3.0-5.el8.x86_64.rp 1.1 MB/s | 18 kB 00:00 2023-10-08T00:08:50.439 INFO:teuthology.orchestra.run.smithi099.stdout:(105/120): pcre2-devel-10.32-2.el8.x86_64.rpm 22 MB/s | 605 kB 00:00 2023-10-08T00:08:50.445 INFO:teuthology.orchestra.run.smithi099.stdout:(106/120): openssl-devel-1.1.1g-15.el8_3.x86_64 48 MB/s | 2.3 MB 00:00 2023-10-08T00:08:50.446 INFO:teuthology.orchestra.run.smithi099.stdout:(107/120): fuse-2.9.7-12.el8.x86_64.rpm 3.8 MB/s | 83 kB 00:00 2023-10-08T00:08:50.451 INFO:teuthology.orchestra.run.smithi191.stdout:(24/120): python3-rgw-17.2.6-1320.g08cac018.el8 2.8 MB/s | 107 kB 00:00 2023-10-08T00:08:50.452 INFO:teuthology.orchestra.run.smithi099.stdout:(108/120): elfutils-libelf-devel-0.182-3.el8.x8 4.3 MB/s | 59 kB 00:00 2023-10-08T00:08:50.453 INFO:teuthology.orchestra.run.smithi191.stdout:(25/120): rbd-fuse-17.2.6-1320.g08cac018.el8.x8 2.4 MB/s | 90 kB 00:00 2023-10-08T00:08:50.460 INFO:teuthology.orchestra.run.smithi099.stdout:(109/120): libcom_err-devel-1.45.6-1.el8.x86_64 2.7 MB/s | 38 kB 00:00 2023-10-08T00:08:50.462 INFO:teuthology.orchestra.run.smithi099.stdout:(110/120): libquadmath-8.4.1-1.el8.x86_64.rpm 9.9 MB/s | 169 kB 00:00 2023-10-08T00:08:50.467 INFO:teuthology.orchestra.run.smithi099.stdout:(111/120): libsepol-devel-2.9-2.el8.x86_64.rpm 5.8 MB/s | 86 kB 00:00 2023-10-08T00:08:50.474 INFO:teuthology.orchestra.run.smithi099.stdout:(112/120): zlib-devel-1.2.11-17.el8.x86_64.rpm 4.0 MB/s | 58 kB 00:00 2023-10-08T00:08:50.475 INFO:teuthology.orchestra.run.smithi099.stdout:(113/120): librabbitmq-0.9.0-3.el8.x86_64.rpm 3.6 MB/s | 47 kB 00:00 2023-10-08T00:08:50.487 INFO:teuthology.orchestra.run.smithi099.stdout:(114/120): krb5-devel-1.18.2-8.3.el8_4.x86_64.r 28 MB/s | 559 kB 00:00 2023-10-08T00:08:50.489 INFO:teuthology.orchestra.run.smithi099.stdout:(115/120): xfsprogs-devel-5.0.0-8.el8.x86_64.rp 5.3 MB/s | 70 kB 00:00 2023-10-08T00:08:50.492 INFO:teuthology.orchestra.run.smithi191.stdout:(26/120): rbd-nbd-17.2.6-1320.g08cac018.el8.x86 4.4 MB/s | 173 kB 00:00 2023-10-08T00:08:50.495 INFO:teuthology.orchestra.run.smithi099.stdout:(116/120): libgfortran-8.4.1-1.el8.x86_64.rpm 31 MB/s | 639 kB 00:00 2023-10-08T00:08:50.503 INFO:teuthology.orchestra.run.smithi099.stdout:(117/120): libuuid-devel-2.32.1-27.el8.x86_64.r 6.7 MB/s | 97 kB 00:00 2023-10-08T00:08:50.504 INFO:teuthology.orchestra.run.smithi099.stdout:(118/120): libselinux-devel-2.9-5.el8.x86_64.rp 12 MB/s | 200 kB 00:00 2023-10-08T00:08:50.528 INFO:teuthology.orchestra.run.smithi191.stdout:(27/120): ceph-grafana-dashboards-17.2.6-1320.g 644 kB/s | 23 kB 00:00 2023-10-08T00:08:50.565 INFO:teuthology.orchestra.run.smithi191.stdout:(28/120): ceph-mgr-cephadm-17.2.6-1320.g08cac01 3.2 MB/s | 121 kB 00:00 2023-10-08T00:08:50.749 INFO:teuthology.orchestra.run.smithi191.stdout:(29/120): ceph-mgr-dashboard-17.2.6-1320.g08cac 9.2 MB/s | 1.7 MB 00:00 2023-10-08T00:08:50.946 INFO:teuthology.orchestra.run.smithi191.stdout:(30/120): librgw2-17.2.6-1320.g08cac018.el8.x86 3.6 MB/s | 5.6 MB 00:01 2023-10-08T00:08:51.030 INFO:teuthology.orchestra.run.smithi191.stdout:(31/120): ceph-mgr-modules-core-17.2.6-1320.g08 2.8 MB/s | 238 kB 00:00 2023-10-08T00:08:51.068 INFO:teuthology.orchestra.run.smithi191.stdout:(32/120): ceph-mgr-rook-17.2.6-1320.g08cac018.e 1.2 MB/s | 47 kB 00:00 2023-10-08T00:08:51.104 INFO:teuthology.orchestra.run.smithi191.stdout:(33/120): ceph-prometheus-alerts-17.2.6-1320.g0 400 kB/s | 14 kB 00:00 2023-10-08T00:08:51.115 INFO:teuthology.orchestra.run.smithi099.stdout:(119/120): librados2-17.2.6-1320.g08cac018.el8. 5.6 MB/s | 3.5 MB 00:00 2023-10-08T00:08:51.178 INFO:teuthology.orchestra.run.smithi191.stdout:(34/120): ceph-volume-17.2.6-1320.g08cac018.el8 3.5 MB/s | 267 kB 00:00 2023-10-08T00:08:51.216 INFO:teuthology.orchestra.run.smithi191.stdout:(35/120): cephadm-17.2.6-1320.g08cac018.el8.noa 2.1 MB/s | 80 kB 00:00 2023-10-08T00:08:51.252 INFO:teuthology.orchestra.run.smithi191.stdout:(36/120): cephfs-top-17.2.6-1320.g08cac018.el8. 496 kB/s | 18 kB 00:00 2023-10-08T00:08:51.343 INFO:teuthology.orchestra.run.smithi191.stdout:(37/120): rbd-mirror-17.2.6-1320.g08cac018.el8. 3.9 MB/s | 3.5 MB 00:00 2023-10-08T00:08:51.426 INFO:teuthology.orchestra.run.smithi191.stdout:(38/120): gperftools-libs-2.7-9.el8.x86_64.rpm 1.7 MB/s | 306 kB 00:00 2023-10-08T00:08:51.444 INFO:teuthology.orchestra.run.smithi191.stdout:(39/120): libunwind-1.3.1-3.el8.x86_64.rpm 4.1 MB/s | 75 kB 00:00 2023-10-08T00:08:51.470 INFO:teuthology.orchestra.run.smithi191.stdout:(40/120): ceph-mgr-diskprediction-local-17.2.6- 10 MB/s | 7.4 MB 00:00 2023-10-08T00:08:51.470 INFO:teuthology.orchestra.run.smithi191.stdout:(41/120): liboath-2.6.2-3.el8.x86_64.rpm 463 kB/s | 59 kB 00:00 2023-10-08T00:08:51.522 INFO:teuthology.orchestra.run.smithi191.stdout:(42/120): python3-beautifulsoup4-4.6.3-2.el8.1. 3.5 MB/s | 185 kB 00:00 2023-10-08T00:08:51.532 INFO:teuthology.orchestra.run.smithi191.stdout:(43/120): python3-asyncssh-2.7.0-2.el8.noarch.r 5.3 MB/s | 476 kB 00:00 2023-10-08T00:08:51.540 INFO:teuthology.orchestra.run.smithi191.stdout:(44/120): python3-cachetools-3.1.1-4.el8.noarch 1.8 MB/s | 33 kB 00:00 2023-10-08T00:08:51.549 INFO:teuthology.orchestra.run.smithi099.stdout:(120/120): librbd1-17.2.6-1320.g08cac018.el8.x8 3.4 MB/s | 3.6 MB 00:01 2023-10-08T00:08:51.550 INFO:teuthology.orchestra.run.smithi191.stdout:(45/120): python3-certifi-2018.10.15-7.el8.noar 883 kB/s | 16 kB 00:00 2023-10-08T00:08:51.552 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2023-10-08T00:08:51.552 INFO:teuthology.orchestra.run.smithi099.stdout:Total 16 MB/s | 202 MB 00:12 2023-10-08T00:08:51.576 INFO:teuthology.orchestra.run.smithi191.stdout:(46/120): python3-cheroot-8.5.2-1.el8.noarch.rp 4.7 MB/s | 173 kB 00:00 2023-10-08T00:08:51.578 INFO:teuthology.orchestra.run.smithi191.stdout:(47/120): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 405 kB/s | 44 kB 00:00 2023-10-08T00:08:51.595 INFO:teuthology.orchestra.run.smithi191.stdout:(48/120): python3-google-auth-1.1.1-10.el8.noar 4.2 MB/s | 81 kB 00:00 2023-10-08T00:08:51.596 INFO:teuthology.orchestra.run.smithi191.stdout:(49/120): python3-jaraco-6.2-6.el8.noarch.rpm 615 kB/s | 11 kB 00:00 2023-10-08T00:08:51.599 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2023-10-08T00:08:51.614 INFO:teuthology.orchestra.run.smithi191.stdout:(50/120): python3-jaraco-functools-2.0-4.el8.no 988 kB/s | 18 kB 00:00 2023-10-08T00:08:51.622 INFO:teuthology.orchestra.run.smithi191.stdout:(51/120): python3-cherrypy-18.4.0-1.el8.noarch. 5.3 MB/s | 384 kB 00:00 2023-10-08T00:08:51.632 INFO:teuthology.orchestra.run.smithi191.stdout:(52/120): python3-logutils-0.3.5-11.el8.noarch. 2.6 MB/s | 49 kB 00:00 2023-10-08T00:08:51.640 INFO:teuthology.orchestra.run.smithi191.stdout:(53/120): python3-more-itertools-7.2.0-3.el8.no 3.2 MB/s | 59 kB 00:00 2023-10-08T00:08:51.650 INFO:teuthology.orchestra.run.smithi191.stdout:(54/120): python3-natsort-7.1.1-2.el8.noarch.rp 3.3 MB/s | 60 kB 00:00 2023-10-08T00:08:51.668 INFO:teuthology.orchestra.run.smithi191.stdout:(55/120): python3-portend-2.6-1.el8.noarch.rpm 906 kB/s | 16 kB 00:00 2023-10-08T00:08:51.688 INFO:teuthology.orchestra.run.smithi191.stdout:(56/120): python3-repoze-lru-0.7-6.el8.noarch.r 1.6 MB/s | 34 kB 00:00 2023-10-08T00:08:51.700 INFO:teuthology.orchestra.run.smithi191.stdout:(57/120): python3-pecan-1.3.2-9.el8.noarch.rpm 4.6 MB/s | 283 kB 00:00 2023-10-08T00:08:51.711 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2023-10-08T00:08:51.712 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2023-10-08T00:08:51.719 INFO:teuthology.orchestra.run.smithi191.stdout:(58/120): python3-rsa-4.9-2.el8.noarch.rpm 3.4 MB/s | 64 kB 00:00 2023-10-08T00:08:51.729 INFO:teuthology.orchestra.run.smithi191.stdout:(59/120): python3-routes-2.4.1-12.el8.noarch.rp 4.8 MB/s | 196 kB 00:00 2023-10-08T00:08:51.737 INFO:teuthology.orchestra.run.smithi191.stdout:(60/120): python3-simplegeneric-0.8.1-17.el8.no 1.0 MB/s | 19 kB 00:00 2023-10-08T00:08:51.747 INFO:teuthology.orchestra.run.smithi191.stdout:(61/120): python3-singledispatch-3.4.0.3-18.el8 1.3 MB/s | 24 kB 00:00 2023-10-08T00:08:51.755 INFO:teuthology.orchestra.run.smithi191.stdout:(62/120): python3-tempora-1.14.1-5.el8.noarch.r 1.6 MB/s | 29 kB 00:00 2023-10-08T00:08:51.765 INFO:teuthology.orchestra.run.smithi191.stdout:(63/120): python3-trustme-0.6.0-4.el8.noarch.rp 1.5 MB/s | 27 kB 00:00 2023-10-08T00:08:51.808 INFO:teuthology.orchestra.run.smithi191.stdout:(64/120): python3-kubernetes-11.0.0-6.el8.noarc 5.5 MB/s | 1.2 MB 00:00 2023-10-08T00:08:51.809 INFO:teuthology.orchestra.run.smithi191.stdout:(65/120): python3-waitress-1.4.3-1.el8.noarch.r 4.6 MB/s | 255 kB 00:00 2023-10-08T00:08:51.817 INFO:teuthology.orchestra.run.smithi191.stdout:(66/120): python3-webob-1.8.5-1.el8.1.noarch.rp 4.7 MB/s | 251 kB 00:00 2023-10-08T00:08:51.825 INFO:teuthology.orchestra.run.smithi191.stdout:(67/120): python3-websocket-client-0.56.0-5.el8 3.4 MB/s | 61 kB 00:00 2023-10-08T00:08:51.827 INFO:teuthology.orchestra.run.smithi191.stdout:(68/120): python3-webtest-2.0.33-1.el8.noarch.r 4.6 MB/s | 85 kB 00:00 2023-10-08T00:08:51.832 INFO:teuthology.orchestra.run.smithi191.stdout:(69/120): xmlstarlet-1.6.1-20.el8.x86_64.rpm 15 MB/s | 70 kB 00:00 2023-10-08T00:08:51.835 INFO:teuthology.orchestra.run.smithi191.stdout:(70/120): python3-zc-lockfile-2.0-2.el8.noarch. 1.3 MB/s | 23 kB 00:00 2023-10-08T00:08:51.875 INFO:teuthology.orchestra.run.smithi191.stdout:(71/120): python3-prettytable-0.7.2-14.el8.noar 1.1 MB/s | 44 kB 00:00 2023-10-08T00:08:51.890 INFO:teuthology.orchestra.run.smithi191.stdout:(72/120): python3-jsonpatch-1.21-2.el8.noarch.r 1.8 MB/s | 27 kB 00:00 2023-10-08T00:08:51.905 INFO:teuthology.orchestra.run.smithi191.stdout:(73/120): jq-1.5-12.el8.x86_64.rpm 10 MB/s | 161 kB 00:00 2023-10-08T00:08:51.924 INFO:teuthology.orchestra.run.smithi191.stdout:(74/120): flex-2.6.1-9.el8.x86_64.rpm 17 MB/s | 320 kB 00:00 2023-10-08T00:08:51.943 INFO:teuthology.orchestra.run.smithi191.stdout:(75/120): python3-werkzeug-0.12.2-4.el8.noarch. 24 MB/s | 457 kB 00:00 2023-10-08T00:08:52.025 INFO:teuthology.orchestra.run.smithi191.stdout:(76/120): python3-babel-2.5.1-5.el8.noarch.rpm 59 MB/s | 4.8 MB 00:00 2023-10-08T00:08:52.038 INFO:teuthology.orchestra.run.smithi191.stdout:(77/120): python3-markupsafe-0.23-19.el8.x86_64 2.9 MB/s | 39 kB 00:00 2023-10-08T00:08:52.138 INFO:teuthology.orchestra.run.smithi191.stdout:(78/120): python3-scipy-1.0.0-20.module+el8.1.0 48 MB/s | 15 MB 00:00 2023-10-08T00:08:52.146 INFO:teuthology.orchestra.run.smithi191.stdout:(79/120): bison-3.0.4-10.el8.x86_64.rpm 6.3 MB/s | 688 kB 00:00 2023-10-08T00:08:52.153 INFO:teuthology.orchestra.run.smithi191.stdout:(80/120): lttng-ust-2.8.1-11.el8.x86_64.rpm 17 MB/s | 259 kB 00:00 2023-10-08T00:08:52.161 INFO:teuthology.orchestra.run.smithi191.stdout:(81/120): python3-numpy-f2py-1.14.3-9.el8.x86_6 15 MB/s | 225 kB 00:00 2023-10-08T00:08:52.169 INFO:teuthology.orchestra.run.smithi191.stdout:(82/120): python3-pyasn1-0.3.7-6.el8.noarch.rpm 8.8 MB/s | 126 kB 00:00 2023-10-08T00:08:52.172 INFO:teuthology.orchestra.run.smithi191.stdout:(83/120): thrift-0.13.0-2.el8.x86_64.rpm 5.0 MB/s | 1.7 MB 00:00 2023-10-08T00:08:52.188 INFO:teuthology.orchestra.run.smithi191.stdout:(84/120): oniguruma-6.8.2-2.el8.x86_64.rpm 9.2 MB/s | 187 kB 00:00 2023-10-08T00:08:52.205 INFO:teuthology.orchestra.run.smithi191.stdout:(85/120): socat-1.7.3.3-2.el8.x86_64.rpm 17 MB/s | 302 kB 00:00 2023-10-08T00:08:52.208 INFO:teuthology.orchestra.run.smithi191.stdout:(86/120): python3-jinja2-2.10.1-2.el8_0.noarch. 15 MB/s | 538 kB 00:00 2023-10-08T00:08:52.226 INFO:teuthology.orchestra.run.smithi191.stdout:(87/120): python3-numpy-1.14.3-9.el8.x86_64.rpm 58 MB/s | 3.7 MB 00:00 2023-10-08T00:08:52.227 INFO:teuthology.orchestra.run.smithi191.stdout:(88/120): python3-pyasn1-modules-0.3.7-6.el8.no 5.6 MB/s | 110 kB 00:00 2023-10-08T00:08:52.228 INFO:teuthology.orchestra.run.smithi191.stdout:(89/120): python3-mako-1.0.6-13.el8.noarch.rpm 6.7 MB/s | 157 kB 00:00 2023-10-08T00:08:52.242 INFO:teuthology.orchestra.run.smithi191.stdout:(90/120): python3-jsonpointer-1.10-11.el8.noarc 1.3 MB/s | 20 kB 00:00 2023-10-08T00:08:52.246 INFO:teuthology.orchestra.run.smithi191.stdout:(91/120): librdkafka-0.11.4-1.el8.x86_64.rpm 17 MB/s | 353 kB 00:00 2023-10-08T00:08:52.251 INFO:teuthology.orchestra.run.smithi191.stdout:(92/120): libtool-2.4.6-25.el8.x86_64.rpm 31 MB/s | 709 kB 00:00 2023-10-08T00:08:52.256 INFO:teuthology.orchestra.run.smithi191.stdout:(93/120): python3-pytz-2017.2-9.el8.noarch.rpm 3.8 MB/s | 54 kB 00:00 2023-10-08T00:08:52.282 INFO:teuthology.orchestra.run.smithi191.stdout:(94/120): automake-1.16.1-7.el8.noarch.rpm 27 MB/s | 713 kB 00:00 2023-10-08T00:08:52.296 INFO:teuthology.orchestra.run.smithi191.stdout:(95/120): libaio-devel-0.3.112-1.el8.x86_64.rpm 1.3 MB/s | 19 kB 00:00 2023-10-08T00:08:52.309 INFO:teuthology.orchestra.run.smithi191.stdout:(96/120): python3-jwt-1.6.1-2.el8.noarch.rpm 3.4 MB/s | 43 kB 00:00 2023-10-08T00:08:52.329 INFO:teuthology.orchestra.run.smithi191.stdout:(97/120): openblas-threads-0.3.12-1.el8.x86_64. 57 MB/s | 4.7 MB 00:00 2023-10-08T00:08:52.331 INFO:teuthology.orchestra.run.smithi191.stdout:(98/120): python3-oauthlib-2.1.0-1.el8.noarch.r 6.8 MB/s | 155 kB 00:00 2023-10-08T00:08:52.344 INFO:teuthology.orchestra.run.smithi191.stdout:(99/120): openblas-0.3.12-1.el8.x86_64.rpm 49 MB/s | 4.6 MB 00:00 2023-10-08T00:08:52.345 INFO:teuthology.orchestra.run.smithi191.stdout:(100/120): keyutils-libs-devel-1.5.10-6.el8.x86 3.4 MB/s | 48 kB 00:00 2023-10-08T00:08:52.347 INFO:teuthology.orchestra.run.smithi191.stdout:(101/120): pcre2-utf32-10.32-2.el8.x86_64.rpm 12 MB/s | 220 kB 00:00 2023-10-08T00:08:52.359 INFO:teuthology.orchestra.run.smithi191.stdout:(102/120): python3-requests-oauthlib-1.0.0-1.el 3.1 MB/s | 43 kB 00:00 2023-10-08T00:08:52.360 INFO:teuthology.orchestra.run.smithi191.stdout:(103/120): pcre2-utf16-10.32-2.el8.x86_64.rpm 14 MB/s | 229 kB 00:00 2023-10-08T00:08:52.373 INFO:teuthology.orchestra.run.smithi191.stdout:(104/120): libverto-devel-0.3.0-5.el8.x86_64.rp 1.2 MB/s | 18 kB 00:00 2023-10-08T00:08:52.381 INFO:teuthology.orchestra.run.smithi191.stdout:(105/120): pcre2-devel-10.32-2.el8.x86_64.rpm 29 MB/s | 605 kB 00:00 2023-10-08T00:08:52.390 INFO:teuthology.orchestra.run.smithi191.stdout:(106/120): openssl-devel-1.1.1g-15.el8_3.x86_64 54 MB/s | 2.3 MB 00:00 2023-10-08T00:08:52.392 INFO:teuthology.orchestra.run.smithi191.stdout:(107/120): fuse-2.9.7-12.el8.x86_64.rpm 4.4 MB/s | 83 kB 00:00 2023-10-08T00:08:52.397 INFO:teuthology.orchestra.run.smithi191.stdout:(108/120): elfutils-libelf-devel-0.182-3.el8.x8 3.6 MB/s | 59 kB 00:00 2023-10-08T00:08:52.406 INFO:teuthology.orchestra.run.smithi191.stdout:(109/120): libcom_err-devel-1.45.6-1.el8.x86_64 2.6 MB/s | 38 kB 00:00 2023-10-08T00:08:52.408 INFO:teuthology.orchestra.run.smithi191.stdout:(110/120): libquadmath-8.4.1-1.el8.x86_64.rpm 9.6 MB/s | 169 kB 00:00 2023-10-08T00:08:52.413 INFO:teuthology.orchestra.run.smithi191.stdout:(111/120): libsepol-devel-2.9-2.el8.x86_64.rpm 5.3 MB/s | 86 kB 00:00 2023-10-08T00:08:52.419 INFO:teuthology.orchestra.run.smithi191.stdout:(112/120): zlib-devel-1.2.11-17.el8.x86_64.rpm 4.3 MB/s | 58 kB 00:00 2023-10-08T00:08:52.423 INFO:teuthology.orchestra.run.smithi191.stdout:(113/120): librabbitmq-0.9.0-3.el8.x86_64.rpm 3.0 MB/s | 47 kB 00:00 2023-10-08T00:08:52.433 INFO:teuthology.orchestra.run.smithi191.stdout:(114/120): krb5-devel-1.18.2-8.3.el8_4.x86_64.r 27 MB/s | 559 kB 00:00 2023-10-08T00:08:52.438 INFO:teuthology.orchestra.run.smithi191.stdout:(115/120): xfsprogs-devel-5.0.0-8.el8.x86_64.rp 4.7 MB/s | 70 kB 00:00 2023-10-08T00:08:52.441 INFO:teuthology.orchestra.run.smithi191.stdout:(116/120): libgfortran-8.4.1-1.el8.x86_64.rpm 29 MB/s | 639 kB 00:00 2023-10-08T00:08:52.449 INFO:teuthology.orchestra.run.smithi191.stdout:(117/120): libselinux-devel-2.9-5.el8.x86_64.rp 13 MB/s | 200 kB 00:00 2023-10-08T00:08:52.452 INFO:teuthology.orchestra.run.smithi191.stdout:(118/120): libuuid-devel-2.32.1-27.el8.x86_64.r 7.2 MB/s | 97 kB 00:00 2023-10-08T00:08:52.895 INFO:teuthology.orchestra.run.smithi191.stdout:(119/120): librbd1-17.2.6-1320.g08cac018.el8.x8 8.0 MB/s | 3.6 MB 00:00 2023-10-08T00:08:53.243 INFO:teuthology.orchestra.run.smithi191.stdout:(120/120): librados2-17.2.6-1320.g08cac018.el8. 4.3 MB/s | 3.5 MB 00:00 2023-10-08T00:08:53.248 INFO:teuthology.orchestra.run.smithi191.stdout:-------------------------------------------------------------------------------- 2023-10-08T00:08:53.248 INFO:teuthology.orchestra.run.smithi191.stdout:Total 14 MB/s | 202 MB 00:14 2023-10-08T00:08:53.295 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction check 2023-10-08T00:08:53.407 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction check succeeded. 2023-10-08T00:08:53.407 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction test 2023-10-08T00:08:53.470 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2023-10-08T00:08:53.474 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2023-10-08T00:08:55.100 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction test succeeded. 2023-10-08T00:08:55.105 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction 2023-10-08T00:08:58.165 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2023-10-08T00:08:58.395 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/122 2023-10-08T00:08:58.632 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 2/122 2023-10-08T00:08:58.841 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 3/122 2023-10-08T00:08:58.973 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 3/122 2023-10-08T00:08:59.199 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/122 2023-10-08T00:08:59.577 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/122 2023-10-08T00:08:59.808 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : librados2-2:17.2.6-1320.g08cac018.el8.x86_64 5/122 2023-10-08T00:09:00.205 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librados2-2:17.2.6-1320.g08cac018.el8.x86_64 5/122 2023-10-08T00:09:00.331 INFO:teuthology.orchestra.run.smithi191.stdout: Preparing : 1/1 2023-10-08T00:09:00.432 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 6/122 2023-10-08T00:09:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 6/122 2023-10-08T00:09:00.579 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/122 2023-10-08T00:09:00.799 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 7/122 2023-10-08T00:09:00.895 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 7/122 2023-10-08T00:09:00.997 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 2/122 2023-10-08T00:09:01.527 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 3/122 2023-10-08T00:09:01.645 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rados-2:17.2.6-1320.g08cac018.el8.x86_64 8/122 2023-10-08T00:09:01.665 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 3/122 2023-10-08T00:09:01.902 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/122 2023-10-08T00:09:02.276 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/122 2023-10-08T00:09:02.333 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/122 2023-10-08T00:09:02.532 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 10/122 2023-10-08T00:09:02.535 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : librados2-2:17.2.6-1320.g08cac018.el8.x86_64 5/122 2023-10-08T00:09:02.645 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 10/122 2023-10-08T00:09:02.878 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/122 2023-10-08T00:09:02.936 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librados2-2:17.2.6-1320.g08cac018.el8.x86_64 5/122 2023-10-08T00:09:03.039 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/122 2023-10-08T00:09:03.176 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 6/122 2023-10-08T00:09:03.265 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 13/122 2023-10-08T00:09:03.312 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 6/122 2023-10-08T00:09:03.378 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 13/122 2023-10-08T00:09:03.501 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 7/122 2023-10-08T00:09:03.566 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86 14/122 2023-10-08T00:09:03.587 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 7/122 2023-10-08T00:09:03.622 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86 14/122 2023-10-08T00:09:03.820 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : zlib-devel-1.2.11-17.el8.x86_64 15/122 2023-10-08T00:09:03.965 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 16/122 2023-10-08T00:09:03.966 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rados-2:17.2.6-1320.g08cac018.el8.x86_64 8/122 2023-10-08T00:09:04.123 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 16/122 2023-10-08T00:09:04.348 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 17/122 2023-10-08T00:09:04.628 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/122 2023-10-08T00:09:04.785 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 10/122 2023-10-08T00:09:04.878 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 10/122 2023-10-08T00:09:04.900 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 17/122 2023-10-08T00:09:05.065 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 18/122 2023-10-08T00:09:05.122 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/122 2023-10-08T00:09:05.283 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/122 2023-10-08T00:09:05.468 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 13/122 2023-10-08T00:09:05.583 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 13/122 2023-10-08T00:09:05.596 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 18/122 2023-10-08T00:09:05.718 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86 14/122 2023-10-08T00:09:05.774 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86 14/122 2023-10-08T00:09:05.815 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openblas-0.3.12-1.el8.x86_64 19/122 2023-10-08T00:09:05.922 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : zlib-devel-1.2.11-17.el8.x86_64 15/122 2023-10-08T00:09:06.101 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 16/122 2023-10-08T00:09:06.245 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 16/122 2023-10-08T00:09:06.543 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 19/122 2023-10-08T00:09:06.818 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 17/122 2023-10-08T00:09:07.274 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 20/122 2023-10-08T00:09:07.419 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 17/122 2023-10-08T00:09:07.510 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 21/122 2023-10-08T00:09:07.584 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 18/122 2023-10-08T00:09:07.715 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 22/122 2023-10-08T00:09:07.999 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/122 2023-10-08T00:09:08.117 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 18/122 2023-10-08T00:09:08.279 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 24/122 2023-10-08T00:09:08.357 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : openblas-0.3.12-1.el8.x86_64 19/122 2023-10-08T00:09:08.519 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/122 2023-10-08T00:09:08.704 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-ceph-common-2:17.2.6-1320.g08cac018.el8. 26/122 2023-10-08T00:09:08.816 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-ceph-argparse-2:17.2.6-1320.g08cac018.el 27/122 2023-10-08T00:09:09.043 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 19/122 2023-10-08T00:09:09.118 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cephfs-2:17.2.6-1320.g08cac018.el8.x86_6 28/122 2023-10-08T00:09:09.329 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 20/122 2023-10-08T00:09:09.516 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 21/122 2023-10-08T00:09:09.688 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 22/122 2023-10-08T00:09:09.982 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/122 2023-10-08T00:09:10.193 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 29/122 2023-10-08T00:09:10.295 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 24/122 2023-10-08T00:09:10.530 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/122 2023-10-08T00:09:10.564 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 30/122 2023-10-08T00:09:10.732 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-ceph-common-2:17.2.6-1320.g08cac018.el8. 26/122 2023-10-08T00:09:10.877 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-ceph-argparse-2:17.2.6-1320.g08cac018.el 27/122 2023-10-08T00:09:10.891 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 31/122 2023-10-08T00:09:11.134 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cephfs-2:17.2.6-1320.g08cac018.el8.x86_6 28/122 2023-10-08T00:09:11.154 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 32/122 2023-10-08T00:09:11.457 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 33/122 2023-10-08T00:09:11.697 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 34/122 2023-10-08T00:09:12.190 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 29/122 2023-10-08T00:09:12.516 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 30/122 2023-10-08T00:09:12.793 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 31/122 2023-10-08T00:09:13.048 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 32/122 2023-10-08T00:09:13.317 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 33/122 2023-10-08T00:09:13.566 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 34/122 2023-10-08T00:09:13.817 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 34/122 2023-10-08T00:09:14.159 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 35/122 2023-10-08T00:09:14.428 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rsa-4.9-2.el8.noarch 36/122 2023-10-08T00:09:14.675 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/122 2023-10-08T00:09:14.873 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rgw-2:17.2.6-1320.g08cac018.el8.x86_64 38/122 2023-10-08T00:09:15.077 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rbd-2:17.2.6-1320.g08cac018.el8.x86_64 39/122 2023-10-08T00:09:15.306 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librados-devel-2:17.2.6-1320.g08cac018.el8.x86_6 40/122 2023-10-08T00:09:15.479 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libuuid-devel-2.32.1-27.el8.x86_64 41/122 2023-10-08T00:09:15.681 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 34/122 2023-10-08T00:09:15.686 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libsepol-devel-2.9-2.el8.x86_64 42/122 2023-10-08T00:09:15.865 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcom_err-devel-1.45.6-1.el8.x86_64 43/122 2023-10-08T00:09:16.004 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 35/122 2023-10-08T00:09:16.015 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : fuse-2.9.7-12.el8.x86_64 44/122 2023-10-08T00:09:16.209 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 45/122 2023-10-08T00:09:16.257 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rsa-4.9-2.el8.noarch 36/122 2023-10-08T00:09:16.397 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 46/122 2023-10-08T00:09:16.445 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/122 2023-10-08T00:09:16.592 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 47/122 2023-10-08T00:09:16.668 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rgw-2:17.2.6-1320.g08cac018.el8.x86_64 38/122 2023-10-08T00:09:16.861 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rbd-2:17.2.6-1320.g08cac018.el8.x86_64 39/122 2023-10-08T00:09:16.936 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 48/122 2023-10-08T00:09:17.093 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librados-devel-2:17.2.6-1320.g08cac018.el8.x86_6 40/122 2023-10-08T00:09:17.231 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 49/122 2023-10-08T00:09:17.247 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libuuid-devel-2.32.1-27.el8.x86_64 41/122 2023-10-08T00:09:17.404 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libsepol-devel-2.9-2.el8.x86_64 42/122 2023-10-08T00:09:17.513 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 50/122 2023-10-08T00:09:17.600 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libcom_err-devel-1.45.6-1.el8.x86_64 43/122 2023-10-08T00:09:17.760 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : fuse-2.9.7-12.el8.x86_64 44/122 2023-10-08T00:09:17.876 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : krb5-devel-1.18.2-8.3.el8_4.x86_64 51/122 2023-10-08T00:09:17.902 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 45/122 2023-10-08T00:09:18.114 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : automake-1.16.1-7.el8.noarch 52/122 2023-10-08T00:09:18.138 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 46/122 2023-10-08T00:09:18.286 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 53/122 2023-10-08T00:09:18.310 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 47/122 2023-10-08T00:09:18.493 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 54/122 2023-10-08T00:09:18.654 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 48/122 2023-10-08T00:09:18.724 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 55/122 2023-10-08T00:09:18.887 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 56/122 2023-10-08T00:09:18.933 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 49/122 2023-10-08T00:09:19.003 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 56/122 2023-10-08T00:09:19.185 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 50/122 2023-10-08T00:09:19.187 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : jq-1.5-12.el8.x86_64 57/122 2023-10-08T00:09:19.389 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 58/122 2023-10-08T00:09:19.558 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : krb5-devel-1.18.2-8.3.el8_4.x86_64 51/122 2023-10-08T00:09:19.587 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 59/122 2023-10-08T00:09:19.759 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 60/122 2023-10-08T00:09:19.764 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : automake-1.16.1-7.el8.noarch 52/122 2023-10-08T00:09:19.990 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 53/122 2023-10-08T00:09:20.061 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 61/122 2023-10-08T00:09:20.179 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 54/122 2023-10-08T00:09:20.279 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 62/122 2023-10-08T00:09:20.419 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 55/122 2023-10-08T00:09:20.494 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/122 2023-10-08T00:09:20.565 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 56/122 2023-10-08T00:09:20.642 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 64/122 2023-10-08T00:09:20.643 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 56/122 2023-10-08T00:09:20.824 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 65/122 2023-10-08T00:09:20.849 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : jq-1.5-12.el8.x86_64 57/122 2023-10-08T00:09:21.025 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 66/122 2023-10-08T00:09:21.034 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 58/122 2023-10-08T00:09:21.199 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 59/122 2023-10-08T00:09:21.267 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 67/122 2023-10-08T00:09:21.395 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 60/122 2023-10-08T00:09:21.533 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 68/122 2023-10-08T00:09:21.646 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 61/122 2023-10-08T00:09:21.781 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 69/122 2023-10-08T00:09:21.890 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 62/122 2023-10-08T00:09:21.969 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 70/122 2023-10-08T00:09:22.080 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/122 2023-10-08T00:09:22.210 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 71/122 2023-10-08T00:09:22.256 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 64/122 2023-10-08T00:09:22.393 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 65/122 2023-10-08T00:09:22.458 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 72/122 2023-10-08T00:09:22.599 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 66/122 2023-10-08T00:09:22.760 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 73/122 2023-10-08T00:09:22.819 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 67/122 2023-10-08T00:09:23.043 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 68/122 2023-10-08T00:09:23.064 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-portend-2.6-1.el8.noarch 74/122 2023-10-08T00:09:23.251 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 69/122 2023-10-08T00:09:23.328 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 75/122 2023-10-08T00:09:23.412 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 70/122 2023-10-08T00:09:23.553 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 76/122 2023-10-08T00:09:23.654 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 71/122 2023-10-08T00:09:23.856 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 77/122 2023-10-08T00:09:23.877 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 72/122 2023-10-08T00:09:24.121 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 73/122 2023-10-08T00:09:24.421 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-portend-2.6-1.el8.noarch 74/122 2023-10-08T00:09:24.697 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 75/122 2023-10-08T00:09:24.881 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 76/122 2023-10-08T00:09:25.067 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 78/122 2023-10-08T00:09:25.156 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 77/122 2023-10-08T00:09:25.392 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 79/122 2023-10-08T00:09:25.637 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 80/122 2023-10-08T00:09:25.968 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 81/122 2023-10-08T00:09:26.261 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 82/122 2023-10-08T00:09:26.351 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 78/122 2023-10-08T00:09:26.623 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/122 2023-10-08T00:09:26.637 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 79/122 2023-10-08T00:09:26.875 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 80/122 2023-10-08T00:09:26.882 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 84/122 2023-10-08T00:09:27.127 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 85/122 2023-10-08T00:09:27.208 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 81/122 2023-10-08T00:09:27.335 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 86/122 2023-10-08T00:09:27.495 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 82/122 2023-10-08T00:09:27.858 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/122 2023-10-08T00:09:28.076 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 84/122 2023-10-08T00:09:28.299 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 85/122 2023-10-08T00:09:28.514 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 86/122 2023-10-08T00:09:32.465 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 87/122 2023-10-08T00:09:32.684 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 87/122 2023-10-08T00:09:33.326 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 87/122 2023-10-08T00:09:33.687 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 87/122 2023-10-08T00:09:33.699 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 88/122 2023-10-08T00:09:33.815 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 88/122 2023-10-08T00:09:33.905 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 87/122 2023-10-08T00:09:33.934 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 89/122 2023-10-08T00:09:34.490 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 87/122 2023-10-08T00:09:34.808 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 88/122 2023-10-08T00:09:34.907 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 88/122 2023-10-08T00:09:35.113 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 89/122 2023-10-08T00:09:51.798 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 89/122 2023-10-08T00:09:51.798 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /sys 2023-10-08T00:09:51.799 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /proc 2023-10-08T00:09:51.799 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /dev 2023-10-08T00:09:51.799 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /run 2023-10-08T00:09:51.799 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /mnt 2023-10-08T00:09:51.799 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /var/tmp 2023-10-08T00:09:51.799 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /home 2023-10-08T00:09:51.799 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /tmp 2023-10-08T00:09:51.799 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /dev 2023-10-08T00:09:51.799 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:09:52.213 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 90/122 2023-10-08T00:09:53.427 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 89/122 2023-10-08T00:09:53.427 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /sys 2023-10-08T00:09:53.427 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /proc 2023-10-08T00:09:53.427 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /dev 2023-10-08T00:09:53.427 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /run 2023-10-08T00:09:53.427 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /mnt 2023-10-08T00:09:53.427 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /var/tmp 2023-10-08T00:09:53.428 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /home 2023-10-08T00:09:53.428 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /tmp 2023-10-08T00:09:53.428 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /dev 2023-10-08T00:09:53.428 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T00:09:53.710 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 90/122 2023-10-08T00:09:53.846 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 90/122 2023-10-08T00:09:53.878 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 91/122 2023-10-08T00:09:54.110 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 91/122 2023-10-08T00:09:54.336 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 92/122 2023-10-08T00:09:54.727 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 92/122 2023-10-08T00:09:54.921 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 93/122 2023-10-08T00:09:54.938 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 93/122 2023-10-08T00:09:55.329 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 90/122 2023-10-08T00:09:55.505 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 91/122 2023-10-08T00:09:55.728 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 91/122 2023-10-08T00:09:55.940 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 92/122 2023-10-08T00:09:56.332 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 92/122 2023-10-08T00:09:56.534 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 93/122 2023-10-08T00:09:56.551 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 93/122 2023-10-08T00:09:57.755 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: cephadm-2:17.2.6-1320.g08cac018.el8.noarch 94/122 2023-10-08T00:09:57.930 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephadm-2:17.2.6-1320.g08cac018.el8.noarch 94/122 2023-10-08T00:09:58.101 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-1320.g08cac018.e 95/122 2023-10-08T00:09:58.313 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-1320.g08cac018. 96/122 2023-10-08T00:09:58.531 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noa 97/122 2023-10-08T00:09:58.750 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noa 97/122 2023-10-08T00:09:58.951 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.n 98/122 2023-10-08T00:09:59.394 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: cephadm-2:17.2.6-1320.g08cac018.el8.noarch 94/122 2023-10-08T00:09:59.575 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : cephadm-2:17.2.6-1320.g08cac018.el8.noarch 94/122 2023-10-08T00:09:59.924 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-1320.g08cac018.e 95/122 2023-10-08T00:09:59.925 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.n 98/122 2023-10-08T00:09:59.981 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-1320.g08cac018. 96/122 2023-10-08T00:10:00.127 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-1320.g08c 99/122 2023-10-08T00:10:00.200 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noa 97/122 2023-10-08T00:10:00.233 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-1320.g08c 99/122 2023-10-08T00:10:00.420 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noa 97/122 2023-10-08T00:10:00.598 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-1320.g08cac018.el 100/122 2023-10-08T00:10:00.670 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.n 98/122 2023-10-08T00:10:00.784 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 101/122 2023-10-08T00:10:00.823 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 101/122 2023-10-08T00:10:01.010 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 102/122 2023-10-08T00:10:01.021 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 102/122 2023-10-08T00:10:01.642 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.n 98/122 2023-10-08T00:10:01.816 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-1320.g08c 99/122 2023-10-08T00:10:01.917 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-1320.g08c 99/122 2023-10-08T00:10:02.334 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-1320.g08cac018.el 100/122 2023-10-08T00:10:02.570 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 101/122 2023-10-08T00:10:02.607 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-2:17.2.6-1320.g08cac018.el8.x86_64 103/122 2023-10-08T00:10:02.608 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 101/122 2023-10-08T00:10:02.766 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 102/122 2023-10-08T00:10:02.777 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 102/122 2023-10-08T00:10:02.850 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 104/122 2023-10-08T00:10:02.952 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 104/122 2023-10-08T00:10:03.102 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-1320.g08cac 105/122 2023-10-08T00:10:03.145 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-1320.g08cac 105/122 2023-10-08T00:10:03.336 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 106/122 2023-10-08T00:10:03.683 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 106/122 2023-10-08T00:10:03.886 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 107/122 2023-10-08T00:10:04.372 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-2:17.2.6-1320.g08cac018.el8.x86_64 103/122 2023-10-08T00:10:04.603 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 104/122 2023-10-08T00:10:04.693 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 104/122 2023-10-08T00:10:04.893 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-1320.g08cac 105/122 2023-10-08T00:10:04.935 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-1320.g08cac 105/122 2023-10-08T00:10:05.108 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 106/122 2023-10-08T00:10:05.456 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 106/122 2023-10-08T00:10:05.624 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 107/122 2023-10-08T00:10:08.828 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 107/122 2023-10-08T00:10:09.188 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-test-2:17.2.6-1320.g08cac018.el8.x86_64 108/122 2023-10-08T00:10:09.350 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libtool-2.4.6-25.el8.x86_64 109/122 2023-10-08T00:10:10.074 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 109/122 2023-10-08T00:10:10.463 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openssl-devel-1:1.1.1g-15.el8_3.x86_64 110/122 2023-10-08T00:10:10.556 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 107/122 2023-10-08T00:10:10.750 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 111/122 2023-10-08T00:10:10.901 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-test-2:17.2.6-1320.g08cac018.el8.x86_64 108/122 2023-10-08T00:10:10.921 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : xfsprogs-devel-5.0.0-8.el8.x86_64 112/122 2023-10-08T00:10:11.056 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libtool-2.4.6-25.el8.x86_64 109/122 2023-10-08T00:10:11.113 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephfs-devel-2:17.2.6-1320.g08cac018.el8.x86_ 113/122 2023-10-08T00:10:11.322 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : elfutils-libelf-devel-0.182-3.el8.x86_64 114/122 2023-10-08T00:10:11.500 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephfs-top-2:17.2.6-1320.g08cac018.el8.noarch 115/122 2023-10-08T00:10:11.708 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 116/122 2023-10-08T00:10:11.777 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 109/122 2023-10-08T00:10:11.905 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-nbd-2:17.2.6-1320.g08cac018.el8.x86_64 117/122 2023-10-08T00:10:12.136 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 118/122 2023-10-08T00:10:12.165 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : openssl-devel-1:1.1.1g-15.el8_3.x86_64 110/122 2023-10-08T00:10:12.358 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : bison-3.0.4-10.el8.x86_64 119/122 2023-10-08T00:10:12.362 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 111/122 2023-10-08T00:10:12.413 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 119/122 2023-10-08T00:10:12.568 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : xfsprogs-devel-5.0.0-8.el8.x86_64 112/122 2023-10-08T00:10:12.642 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : flex-2.6.1-9.el8.x86_64 120/122 2023-10-08T00:10:12.650 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 120/122 2023-10-08T00:10:12.651 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-08T00:10:12.728 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libcephfs-devel-2:17.2.6-1320.g08cac018.el8.x86_ 113/122 2023-10-08T00:10:12.870 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : elfutils-libelf-devel-0.182-3.el8.x86_64 114/122 2023-10-08T00:10:12.878 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-08T00:10:12.878 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-08T00:10:13.064 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : cephfs-top-2:17.2.6-1320.g08cac018.el8.noarch 115/122 2023-10-08T00:10:13.239 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : rbd-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 116/122 2023-10-08T00:10:13.399 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : rbd-nbd-2:17.2.6-1320.g08cac018.el8.x86_64 117/122 2023-10-08T00:10:13.647 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 118/122 2023-10-08T00:10:13.786 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-08T00:10:13.786 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-2:17.2.6-1320.g08cac018.el8.x86_64 1/122 2023-10-08T00:10:13.787 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 2/122 2023-10-08T00:10:13.787 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 3/122 2023-10-08T00:10:13.787 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 4/122 2023-10-08T00:10:13.787 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-1320.g08cac 5/122 2023-10-08T00:10:13.787 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 6/122 2023-10-08T00:10:13.787 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 7/122 2023-10-08T00:10:13.787 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 8/122 2023-10-08T00:10:13.787 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 9/122 2023-10-08T00:10:13.787 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 10/122 2023-10-08T00:10:13.787 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 11/122 2023-10-08T00:10:13.787 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-test-2:17.2.6-1320.g08cac018.el8.x86_64 12/122 2023-10-08T00:10:13.788 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 13/122 2023-10-08T00:10:13.788 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephfs-devel-2:17.2.6-1320.g08cac018.el8.x86_ 14/122 2023-10-08T00:10:13.788 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 15/122 2023-10-08T00:10:13.788 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 16/122 2023-10-08T00:10:13.788 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados-devel-2:17.2.6-1320.g08cac018.el8.x86_6 17/122 2023-10-08T00:10:13.788 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86 18/122 2023-10-08T00:10:13.788 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 19/122 2023-10-08T00:10:13.788 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-ceph-argparse-2:17.2.6-1320.g08cac018.el 20/122 2023-10-08T00:10:13.788 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-ceph-common-2:17.2.6-1320.g08cac018.el8. 21/122 2023-10-08T00:10:13.788 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cephfs-2:17.2.6-1320.g08cac018.el8.x86_6 22/122 2023-10-08T00:10:13.788 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rados-2:17.2.6-1320.g08cac018.el8.x86_64 23/122 2023-10-08T00:10:13.789 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rbd-2:17.2.6-1320.g08cac018.el8.x86_64 24/122 2023-10-08T00:10:13.789 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rgw-2:17.2.6-1320.g08cac018.el8.x86_64 25/122 2023-10-08T00:10:13.789 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 26/122 2023-10-08T00:10:13.789 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 27/122 2023-10-08T00:10:13.789 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-nbd-2:17.2.6-1320.g08cac018.el8.x86_64 28/122 2023-10-08T00:10:13.789 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-1320.g08cac018. 29/122 2023-10-08T00:10:13.789 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noa 30/122 2023-10-08T00:10:13.797 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.n 31/122 2023-10-08T00:10:13.797 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-1320.g08c 32/122 2023-10-08T00:10:13.797 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-1320.g08cac018.el 33/122 2023-10-08T00:10:13.798 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 34/122 2023-10-08T00:10:13.798 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-1320.g08cac018.e 35/122 2023-10-08T00:10:13.798 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 36/122 2023-10-08T00:10:13.798 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephadm-2:17.2.6-1320.g08cac018.el8.noarch 37/122 2023-10-08T00:10:13.798 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephfs-top-2:17.2.6-1320.g08cac018.el8.noarch 38/122 2023-10-08T00:10:13.798 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/122 2023-10-08T00:10:13.798 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/122 2023-10-08T00:10:13.798 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/122 2023-10-08T00:10:13.799 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/122 2023-10-08T00:10:13.799 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/122 2023-10-08T00:10:13.799 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/122 2023-10-08T00:10:13.799 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/122 2023-10-08T00:10:13.799 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/122 2023-10-08T00:10:13.799 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/122 2023-10-08T00:10:13.799 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/122 2023-10-08T00:10:13.799 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/122 2023-10-08T00:10:13.799 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/122 2023-10-08T00:10:13.799 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/122 2023-10-08T00:10:13.799 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/122 2023-10-08T00:10:13.800 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/122 2023-10-08T00:10:13.800 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/122 2023-10-08T00:10:13.800 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/122 2023-10-08T00:10:13.800 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/122 2023-10-08T00:10:13.800 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/122 2023-10-08T00:10:13.800 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/122 2023-10-08T00:10:13.800 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/122 2023-10-08T00:10:13.800 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/122 2023-10-08T00:10:13.800 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/122 2023-10-08T00:10:13.800 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/122 2023-10-08T00:10:13.800 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/122 2023-10-08T00:10:13.800 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/122 2023-10-08T00:10:13.801 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/122 2023-10-08T00:10:13.801 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/122 2023-10-08T00:10:13.801 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/122 2023-10-08T00:10:13.801 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/122 2023-10-08T00:10:13.801 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/122 2023-10-08T00:10:13.801 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/122 2023-10-08T00:10:13.801 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/122 2023-10-08T00:10:13.801 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 72/122 2023-10-08T00:10:13.801 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 73/122 2023-10-08T00:10:13.801 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 74/122 2023-10-08T00:10:13.801 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : jq-1.5-12.el8.x86_64 75/122 2023-10-08T00:10:13.802 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : flex-2.6.1-9.el8.x86_64 76/122 2023-10-08T00:10:13.802 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 77/122 2023-10-08T00:10:13.802 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 78/122 2023-10-08T00:10:13.802 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 79/122 2023-10-08T00:10:13.802 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : bison-3.0.4-10.el8.x86_64 80/122 2023-10-08T00:10:13.802 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 81/122 2023-10-08T00:10:13.802 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 82/122 2023-10-08T00:10:13.802 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 83/122 2023-10-08T00:10:13.802 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 84/122 2023-10-08T00:10:13.802 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 85/122 2023-10-08T00:10:13.803 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 86/122 2023-10-08T00:10:13.803 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 87/122 2023-10-08T00:10:13.803 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 88/122 2023-10-08T00:10:13.803 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 89/122 2023-10-08T00:10:13.803 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 90/122 2023-10-08T00:10:13.803 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 91/122 2023-10-08T00:10:13.803 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 92/122 2023-10-08T00:10:13.803 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 93/122 2023-10-08T00:10:13.803 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 94/122 2023-10-08T00:10:13.803 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 95/122 2023-10-08T00:10:13.803 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : automake-1.16.1-7.el8.noarch 96/122 2023-10-08T00:10:13.803 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 97/122 2023-10-08T00:10:13.804 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 98/122 2023-10-08T00:10:13.804 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 99/122 2023-10-08T00:10:13.804 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 100/122 2023-10-08T00:10:13.804 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 101/122 2023-10-08T00:10:13.804 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 102/122 2023-10-08T00:10:13.804 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 103/122 2023-10-08T00:10:13.804 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-devel-1:1.1.1g-15.el8_3.x86_64 104/122 2023-10-08T00:10:13.804 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 105/122 2023-10-08T00:10:13.804 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 106/122 2023-10-08T00:10:13.804 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 107/122 2023-10-08T00:10:13.804 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : elfutils-libelf-devel-0.182-3.el8.x86_64 108/122 2023-10-08T00:10:13.804 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 109/122 2023-10-08T00:10:13.805 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcom_err-devel-1.45.6-1.el8.x86_64 110/122 2023-10-08T00:10:13.805 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libsepol-devel-2.9-2.el8.x86_64 111/122 2023-10-08T00:10:13.805 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : zlib-devel-1.2.11-17.el8.x86_64 112/122 2023-10-08T00:10:13.805 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 113/122 2023-10-08T00:10:13.805 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : krb5-devel-1.18.2-8.3.el8_4.x86_64 114/122 2023-10-08T00:10:13.805 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 115/122 2023-10-08T00:10:13.805 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : xfsprogs-devel-5.0.0-8.el8.x86_64 116/122 2023-10-08T00:10:13.805 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 117/122 2023-10-08T00:10:13.805 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libuuid-devel-2.32.1-27.el8.x86_64 118/122 2023-10-08T00:10:13.805 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados2-2:17.2.6-1320.g08cac018.el8.x86_64 119/122 2023-10-08T00:10:13.805 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 120/122 2023-10-08T00:10:13.805 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 121/122 2023-10-08T00:10:13.839 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : bison-3.0.4-10.el8.x86_64 119/122 2023-10-08T00:10:13.901 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 119/122 2023-10-08T00:10:14.089 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : flex-2.6.1-9.el8.x86_64 120/122 2023-10-08T00:10:14.099 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 120/122 2023-10-08T00:10:14.099 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-08T00:10:14.309 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-08T00:10:14.309 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-08T00:10:15.258 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-08T00:10:15.259 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-2:17.2.6-1320.g08cac018.el8.x86_64 1/122 2023-10-08T00:10:15.259 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 2/122 2023-10-08T00:10:15.259 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 3/122 2023-10-08T00:10:15.259 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 4/122 2023-10-08T00:10:15.259 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-1320.g08cac 5/122 2023-10-08T00:10:15.260 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 6/122 2023-10-08T00:10:15.260 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 7/122 2023-10-08T00:10:15.260 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 8/122 2023-10-08T00:10:15.260 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 9/122 2023-10-08T00:10:15.260 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 10/122 2023-10-08T00:10:15.260 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 11/122 2023-10-08T00:10:15.260 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-test-2:17.2.6-1320.g08cac018.el8.x86_64 12/122 2023-10-08T00:10:15.260 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 13/122 2023-10-08T00:10:15.261 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libcephfs-devel-2:17.2.6-1320.g08cac018.el8.x86_ 14/122 2023-10-08T00:10:15.261 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 15/122 2023-10-08T00:10:15.261 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 16/122 2023-10-08T00:10:15.261 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librados-devel-2:17.2.6-1320.g08cac018.el8.x86_6 17/122 2023-10-08T00:10:15.261 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86 18/122 2023-10-08T00:10:15.261 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 19/122 2023-10-08T00:10:15.261 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-ceph-argparse-2:17.2.6-1320.g08cac018.el 20/122 2023-10-08T00:10:15.261 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-ceph-common-2:17.2.6-1320.g08cac018.el8. 21/122 2023-10-08T00:10:15.262 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cephfs-2:17.2.6-1320.g08cac018.el8.x86_6 22/122 2023-10-08T00:10:15.262 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rados-2:17.2.6-1320.g08cac018.el8.x86_64 23/122 2023-10-08T00:10:15.262 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rbd-2:17.2.6-1320.g08cac018.el8.x86_64 24/122 2023-10-08T00:10:15.262 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rgw-2:17.2.6-1320.g08cac018.el8.x86_64 25/122 2023-10-08T00:10:15.262 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : rbd-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 26/122 2023-10-08T00:10:15.262 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 27/122 2023-10-08T00:10:15.262 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : rbd-nbd-2:17.2.6-1320.g08cac018.el8.x86_64 28/122 2023-10-08T00:10:15.263 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-1320.g08cac018. 29/122 2023-10-08T00:10:15.263 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noa 30/122 2023-10-08T00:10:15.263 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.n 31/122 2023-10-08T00:10:15.263 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-1320.g08c 32/122 2023-10-08T00:10:15.263 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-1320.g08cac018.el 33/122 2023-10-08T00:10:15.263 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 34/122 2023-10-08T00:10:15.263 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-1320.g08cac018.e 35/122 2023-10-08T00:10:15.263 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 36/122 2023-10-08T00:10:15.264 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : cephadm-2:17.2.6-1320.g08cac018.el8.noarch 37/122 2023-10-08T00:10:15.264 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : cephfs-top-2:17.2.6-1320.g08cac018.el8.noarch 38/122 2023-10-08T00:10:15.264 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/122 2023-10-08T00:10:15.264 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/122 2023-10-08T00:10:15.264 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/122 2023-10-08T00:10:15.264 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/122 2023-10-08T00:10:15.264 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/122 2023-10-08T00:10:15.264 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/122 2023-10-08T00:10:15.265 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/122 2023-10-08T00:10:15.265 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/122 2023-10-08T00:10:15.265 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/122 2023-10-08T00:10:15.265 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/122 2023-10-08T00:10:15.265 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/122 2023-10-08T00:10:15.265 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/122 2023-10-08T00:10:15.265 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/122 2023-10-08T00:10:15.266 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/122 2023-10-08T00:10:15.266 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/122 2023-10-08T00:10:15.266 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/122 2023-10-08T00:10:15.266 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/122 2023-10-08T00:10:15.266 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/122 2023-10-08T00:10:15.266 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/122 2023-10-08T00:10:15.266 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/122 2023-10-08T00:10:15.266 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/122 2023-10-08T00:10:15.267 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/122 2023-10-08T00:10:15.267 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/122 2023-10-08T00:10:15.267 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/122 2023-10-08T00:10:15.267 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/122 2023-10-08T00:10:15.267 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/122 2023-10-08T00:10:15.267 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/122 2023-10-08T00:10:15.268 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/122 2023-10-08T00:10:15.268 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/122 2023-10-08T00:10:15.268 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/122 2023-10-08T00:10:15.268 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/122 2023-10-08T00:10:15.268 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/122 2023-10-08T00:10:15.268 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/122 2023-10-08T00:10:15.268 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 72/122 2023-10-08T00:10:15.269 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 73/122 2023-10-08T00:10:15.269 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 74/122 2023-10-08T00:10:15.269 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : jq-1.5-12.el8.x86_64 75/122 2023-10-08T00:10:15.269 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : flex-2.6.1-9.el8.x86_64 76/122 2023-10-08T00:10:15.269 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 77/122 2023-10-08T00:10:15.269 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 78/122 2023-10-08T00:10:15.269 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 79/122 2023-10-08T00:10:15.269 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : bison-3.0.4-10.el8.x86_64 80/122 2023-10-08T00:10:15.270 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 81/122 2023-10-08T00:10:15.270 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 82/122 2023-10-08T00:10:15.270 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 83/122 2023-10-08T00:10:15.270 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 84/122 2023-10-08T00:10:15.270 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 85/122 2023-10-08T00:10:15.270 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 86/122 2023-10-08T00:10:15.270 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 87/122 2023-10-08T00:10:15.271 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 88/122 2023-10-08T00:10:15.271 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 89/122 2023-10-08T00:10:15.271 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 90/122 2023-10-08T00:10:15.271 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 91/122 2023-10-08T00:10:15.271 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 92/122 2023-10-08T00:10:15.271 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 93/122 2023-10-08T00:10:15.271 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 94/122 2023-10-08T00:10:15.271 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 95/122 2023-10-08T00:10:15.272 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : automake-1.16.1-7.el8.noarch 96/122 2023-10-08T00:10:15.272 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 97/122 2023-10-08T00:10:15.272 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 98/122 2023-10-08T00:10:15.272 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 99/122 2023-10-08T00:10:15.272 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 100/122 2023-10-08T00:10:15.272 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 101/122 2023-10-08T00:10:15.273 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 102/122 2023-10-08T00:10:15.273 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 103/122 2023-10-08T00:10:15.273 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openssl-devel-1:1.1.1g-15.el8_3.x86_64 104/122 2023-10-08T00:10:15.273 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 105/122 2023-10-08T00:10:15.273 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 106/122 2023-10-08T00:10:15.273 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 107/122 2023-10-08T00:10:15.273 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : elfutils-libelf-devel-0.182-3.el8.x86_64 108/122 2023-10-08T00:10:15.273 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 109/122 2023-10-08T00:10:15.274 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libcom_err-devel-1.45.6-1.el8.x86_64 110/122 2023-10-08T00:10:15.274 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libsepol-devel-2.9-2.el8.x86_64 111/122 2023-10-08T00:10:15.274 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : zlib-devel-1.2.11-17.el8.x86_64 112/122 2023-10-08T00:10:15.274 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 113/122 2023-10-08T00:10:15.274 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : krb5-devel-1.18.2-8.3.el8_4.x86_64 114/122 2023-10-08T00:10:15.274 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 115/122 2023-10-08T00:10:15.274 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : xfsprogs-devel-5.0.0-8.el8.x86_64 116/122 2023-10-08T00:10:15.274 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 117/122 2023-10-08T00:10:15.275 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libuuid-devel-2.32.1-27.el8.x86_64 118/122 2023-10-08T00:10:15.275 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librados2-2:17.2.6-1320.g08cac018.el8.x86_64 119/122 2023-10-08T00:10:15.275 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 120/122 2023-10-08T00:10:15.275 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 121/122 2023-10-08T00:10:16.149 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 122/122 2023-10-08T00:10:16.149 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:01:47 ago on Sun 08 Oct 2023 12:08:29 AM UTC. 2023-10-08T00:10:17.541 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 122/122 2023-10-08T00:10:17.541 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:01:48 ago on Sun 08 Oct 2023 12:08:29 AM UTC. 2023-10-08T00:10:20.138 INFO:teuthology.orchestra.run.smithi191.stdout:Installed products updated. 2023-10-08T00:10:20.290 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T00:10:20.290 INFO:teuthology.orchestra.run.smithi191.stdout:Upgraded: 2023-10-08T00:10:20.290 INFO:teuthology.orchestra.run.smithi191.stdout: librados2-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.290 INFO:teuthology.orchestra.run.smithi191.stdout: librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.291 INFO:teuthology.orchestra.run.smithi191.stdout:Installed: 2023-10-08T00:10:20.291 INFO:teuthology.orchestra.run.smithi191.stdout: automake-1.16.1-7.el8.noarch 2023-10-08T00:10:20.291 INFO:teuthology.orchestra.run.smithi191.stdout: bison-3.0.4-10.el8.x86_64 2023-10-08T00:10:20.291 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.291 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.291 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.291 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.291 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-grafana-dashboards-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T00:10:20.291 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-immutable-object-cache-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.291 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.291 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.292 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T00:10:20.292 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T00:10:20.292 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-diskprediction-local-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T00:10:20.292 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-modules-core-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T00:10:20.292 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T00:10:20.292 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.292 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.292 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-prometheus-alerts-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T00:10:20.292 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.292 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.292 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-test-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.292 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T00:10:20.293 INFO:teuthology.orchestra.run.smithi191.stdout: cephadm-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T00:10:20.293 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.293 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-top-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T00:10:20.293 INFO:teuthology.orchestra.run.smithi191.stdout: elfutils-libelf-devel-0.182-3.el8.x86_64 2023-10-08T00:10:20.293 INFO:teuthology.orchestra.run.smithi191.stdout: flex-2.6.1-9.el8.x86_64 2023-10-08T00:10:20.293 INFO:teuthology.orchestra.run.smithi191.stdout: fuse-2.9.7-12.el8.x86_64 2023-10-08T00:10:20.293 INFO:teuthology.orchestra.run.smithi191.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-08T00:10:20.293 INFO:teuthology.orchestra.run.smithi191.stdout: jq-1.5-12.el8.x86_64 2023-10-08T00:10:20.293 INFO:teuthology.orchestra.run.smithi191.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2023-10-08T00:10:20.293 INFO:teuthology.orchestra.run.smithi191.stdout: krb5-devel-1.18.2-8.3.el8_4.x86_64 2023-10-08T00:10:20.293 INFO:teuthology.orchestra.run.smithi191.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-08T00:10:20.293 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs-devel-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.294 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.294 INFO:teuthology.orchestra.run.smithi191.stdout: libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.294 INFO:teuthology.orchestra.run.smithi191.stdout: libcom_err-devel-1.45.6-1.el8.x86_64 2023-10-08T00:10:20.294 INFO:teuthology.orchestra.run.smithi191.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-10-08T00:10:20.294 INFO:teuthology.orchestra.run.smithi191.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-08T00:10:20.294 INFO:teuthology.orchestra.run.smithi191.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-10-08T00:10:20.294 INFO:teuthology.orchestra.run.smithi191.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-08T00:10:20.294 INFO:teuthology.orchestra.run.smithi191.stdout: librados-devel-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.294 INFO:teuthology.orchestra.run.smithi191.stdout: libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.294 INFO:teuthology.orchestra.run.smithi191.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-10-08T00:10:20.294 INFO:teuthology.orchestra.run.smithi191.stdout: librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.294 INFO:teuthology.orchestra.run.smithi191.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-08T00:10:20.295 INFO:teuthology.orchestra.run.smithi191.stdout: libsepol-devel-2.9-2.el8.x86_64 2023-10-08T00:10:20.295 INFO:teuthology.orchestra.run.smithi191.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-08T00:10:20.295 INFO:teuthology.orchestra.run.smithi191.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-08T00:10:20.295 INFO:teuthology.orchestra.run.smithi191.stdout: libuuid-devel-2.32.1-27.el8.x86_64 2023-10-08T00:10:20.295 INFO:teuthology.orchestra.run.smithi191.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-08T00:10:20.295 INFO:teuthology.orchestra.run.smithi191.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-08T00:10:20.295 INFO:teuthology.orchestra.run.smithi191.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-08T00:10:20.295 INFO:teuthology.orchestra.run.smithi191.stdout: openblas-0.3.12-1.el8.x86_64 2023-10-08T00:10:20.295 INFO:teuthology.orchestra.run.smithi191.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-10-08T00:10:20.295 INFO:teuthology.orchestra.run.smithi191.stdout: openssl-devel-1:1.1.1g-15.el8_3.x86_64 2023-10-08T00:10:20.295 INFO:teuthology.orchestra.run.smithi191.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-08T00:10:20.295 INFO:teuthology.orchestra.run.smithi191.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-08T00:10:20.296 INFO:teuthology.orchestra.run.smithi191.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-08T00:10:20.296 INFO:teuthology.orchestra.run.smithi191.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-08T00:10:20.296 INFO:teuthology.orchestra.run.smithi191.stdout: python3-babel-2.5.1-5.el8.noarch 2023-10-08T00:10:20.296 INFO:teuthology.orchestra.run.smithi191.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-08T00:10:20.296 INFO:teuthology.orchestra.run.smithi191.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-08T00:10:20.296 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-08T00:10:20.296 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-argparse-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.296 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.296 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cephfs-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.296 INFO:teuthology.orchestra.run.smithi191.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-08T00:10:20.296 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-08T00:10:20.296 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-08T00:10:20.297 INFO:teuthology.orchestra.run.smithi191.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-08T00:10:20.297 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-08T00:10:20.297 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-08T00:10:20.297 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-10-08T00:10:20.297 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-08T00:10:20.297 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-08T00:10:20.297 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-08T00:10:20.297 INFO:teuthology.orchestra.run.smithi191.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-08T00:10:20.297 INFO:teuthology.orchestra.run.smithi191.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-08T00:10:20.297 INFO:teuthology.orchestra.run.smithi191.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-08T00:10:20.297 INFO:teuthology.orchestra.run.smithi191.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-08T00:10:20.298 INFO:teuthology.orchestra.run.smithi191.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-08T00:10:20.298 INFO:teuthology.orchestra.run.smithi191.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-08T00:10:20.298 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-10-08T00:10:20.298 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-10-08T00:10:20.298 INFO:teuthology.orchestra.run.smithi191.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-08T00:10:20.298 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-08T00:10:20.298 INFO:teuthology.orchestra.run.smithi191.stdout: python3-portend-2.6-1.el8.noarch 2023-10-08T00:10:20.298 INFO:teuthology.orchestra.run.smithi191.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-08T00:10:20.298 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-08T00:10:20.298 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-08T00:10:20.298 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-08T00:10:20.298 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rados-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.299 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rbd-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.299 INFO:teuthology.orchestra.run.smithi191.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-08T00:10:20.299 INFO:teuthology.orchestra.run.smithi191.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-08T00:10:20.299 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rgw-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.299 INFO:teuthology.orchestra.run.smithi191.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-08T00:10:20.299 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-08T00:10:20.299 INFO:teuthology.orchestra.run.smithi191.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-10-08T00:10:20.299 INFO:teuthology.orchestra.run.smithi191.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-08T00:10:20.299 INFO:teuthology.orchestra.run.smithi191.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-08T00:10:20.299 INFO:teuthology.orchestra.run.smithi191.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-08T00:10:20.299 INFO:teuthology.orchestra.run.smithi191.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-08T00:10:20.299 INFO:teuthology.orchestra.run.smithi191.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-08T00:10:20.300 INFO:teuthology.orchestra.run.smithi191.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-08T00:10:20.300 INFO:teuthology.orchestra.run.smithi191.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-08T00:10:20.300 INFO:teuthology.orchestra.run.smithi191.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-08T00:10:20.300 INFO:teuthology.orchestra.run.smithi191.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-08T00:10:20.300 INFO:teuthology.orchestra.run.smithi191.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-08T00:10:20.300 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.300 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.300 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-nbd-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.300 INFO:teuthology.orchestra.run.smithi191.stdout: socat-1.7.3.3-2.el8.x86_64 2023-10-08T00:10:20.300 INFO:teuthology.orchestra.run.smithi191.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-08T00:10:20.300 INFO:teuthology.orchestra.run.smithi191.stdout: xfsprogs-devel-5.0.0-8.el8.x86_64 2023-10-08T00:10:20.300 INFO:teuthology.orchestra.run.smithi191.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-08T00:10:20.301 INFO:teuthology.orchestra.run.smithi191.stdout: zlib-devel-1.2.11-17.el8.x86_64 2023-10-08T00:10:20.301 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T00:10:20.301 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2023-10-08T00:10:20.435 INFO:teuthology.orchestra.run.smithi099.stdout:Installed products updated. 2023-10-08T00:10:20.532 DEBUG:teuthology.parallel:result is None 2023-10-08T00:10:20.564 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:10:20.565 INFO:teuthology.orchestra.run.smithi099.stdout:Upgraded: 2023-10-08T00:10:20.565 INFO:teuthology.orchestra.run.smithi099.stdout: librados2-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.565 INFO:teuthology.orchestra.run.smithi099.stdout: librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.565 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2023-10-08T00:10:20.565 INFO:teuthology.orchestra.run.smithi099.stdout: automake-1.16.1-7.el8.noarch 2023-10-08T00:10:20.565 INFO:teuthology.orchestra.run.smithi099.stdout: bison-3.0.4-10.el8.x86_64 2023-10-08T00:10:20.565 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.565 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.565 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.565 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.565 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-grafana-dashboards-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T00:10:20.566 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-immutable-object-cache-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.566 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.566 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.566 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T00:10:20.566 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T00:10:20.566 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-diskprediction-local-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T00:10:20.566 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-modules-core-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T00:10:20.566 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T00:10:20.566 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.566 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.566 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-prometheus-alerts-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T00:10:20.567 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.567 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.567 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-test-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.567 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T00:10:20.567 INFO:teuthology.orchestra.run.smithi099.stdout: cephadm-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T00:10:20.567 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.567 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-top-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T00:10:20.567 INFO:teuthology.orchestra.run.smithi099.stdout: elfutils-libelf-devel-0.182-3.el8.x86_64 2023-10-08T00:10:20.567 INFO:teuthology.orchestra.run.smithi099.stdout: flex-2.6.1-9.el8.x86_64 2023-10-08T00:10:20.567 INFO:teuthology.orchestra.run.smithi099.stdout: fuse-2.9.7-12.el8.x86_64 2023-10-08T00:10:20.567 INFO:teuthology.orchestra.run.smithi099.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-08T00:10:20.567 INFO:teuthology.orchestra.run.smithi099.stdout: jq-1.5-12.el8.x86_64 2023-10-08T00:10:20.568 INFO:teuthology.orchestra.run.smithi099.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2023-10-08T00:10:20.568 INFO:teuthology.orchestra.run.smithi099.stdout: krb5-devel-1.18.2-8.3.el8_4.x86_64 2023-10-08T00:10:20.568 INFO:teuthology.orchestra.run.smithi099.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-08T00:10:20.568 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs-devel-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.568 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.568 INFO:teuthology.orchestra.run.smithi099.stdout: libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.569 INFO:teuthology.orchestra.run.smithi099.stdout: libcom_err-devel-1.45.6-1.el8.x86_64 2023-10-08T00:10:20.569 INFO:teuthology.orchestra.run.smithi099.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-10-08T00:10:20.569 INFO:teuthology.orchestra.run.smithi099.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-08T00:10:20.569 INFO:teuthology.orchestra.run.smithi099.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-10-08T00:10:20.569 INFO:teuthology.orchestra.run.smithi099.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-08T00:10:20.569 INFO:teuthology.orchestra.run.smithi099.stdout: librados-devel-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.569 INFO:teuthology.orchestra.run.smithi099.stdout: libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.569 INFO:teuthology.orchestra.run.smithi099.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-10-08T00:10:20.570 INFO:teuthology.orchestra.run.smithi099.stdout: librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.570 INFO:teuthology.orchestra.run.smithi099.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-08T00:10:20.570 INFO:teuthology.orchestra.run.smithi099.stdout: libsepol-devel-2.9-2.el8.x86_64 2023-10-08T00:10:20.570 INFO:teuthology.orchestra.run.smithi099.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-08T00:10:20.570 INFO:teuthology.orchestra.run.smithi099.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-08T00:10:20.570 INFO:teuthology.orchestra.run.smithi099.stdout: libuuid-devel-2.32.1-27.el8.x86_64 2023-10-08T00:10:20.570 INFO:teuthology.orchestra.run.smithi099.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-08T00:10:20.570 INFO:teuthology.orchestra.run.smithi099.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-08T00:10:20.570 INFO:teuthology.orchestra.run.smithi099.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-08T00:10:20.570 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-0.3.12-1.el8.x86_64 2023-10-08T00:10:20.570 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-10-08T00:10:20.570 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-devel-1:1.1.1g-15.el8_3.x86_64 2023-10-08T00:10:20.571 INFO:teuthology.orchestra.run.smithi099.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-08T00:10:20.571 INFO:teuthology.orchestra.run.smithi099.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-08T00:10:20.571 INFO:teuthology.orchestra.run.smithi099.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-08T00:10:20.571 INFO:teuthology.orchestra.run.smithi099.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-08T00:10:20.571 INFO:teuthology.orchestra.run.smithi099.stdout: python3-babel-2.5.1-5.el8.noarch 2023-10-08T00:10:20.571 INFO:teuthology.orchestra.run.smithi099.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-08T00:10:20.571 INFO:teuthology.orchestra.run.smithi099.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-08T00:10:20.571 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-08T00:10:20.571 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-argparse-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.571 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.571 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cephfs-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.572 INFO:teuthology.orchestra.run.smithi099.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-08T00:10:20.572 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-08T00:10:20.572 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-08T00:10:20.572 INFO:teuthology.orchestra.run.smithi099.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-08T00:10:20.572 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-08T00:10:20.572 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-08T00:10:20.572 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-10-08T00:10:20.572 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-08T00:10:20.572 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-08T00:10:20.572 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-08T00:10:20.573 INFO:teuthology.orchestra.run.smithi099.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-08T00:10:20.573 INFO:teuthology.orchestra.run.smithi099.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-08T00:10:20.573 INFO:teuthology.orchestra.run.smithi099.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-08T00:10:20.573 INFO:teuthology.orchestra.run.smithi099.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-08T00:10:20.573 INFO:teuthology.orchestra.run.smithi099.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-08T00:10:20.573 INFO:teuthology.orchestra.run.smithi099.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-08T00:10:20.573 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-10-08T00:10:20.573 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-10-08T00:10:20.573 INFO:teuthology.orchestra.run.smithi099.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-08T00:10:20.573 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-08T00:10:20.573 INFO:teuthology.orchestra.run.smithi099.stdout: python3-portend-2.6-1.el8.noarch 2023-10-08T00:10:20.573 INFO:teuthology.orchestra.run.smithi099.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-08T00:10:20.574 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-08T00:10:20.574 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-08T00:10:20.574 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-08T00:10:20.574 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rados-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.574 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rbd-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.574 INFO:teuthology.orchestra.run.smithi099.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-08T00:10:20.574 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-08T00:10:20.574 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rgw-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.574 INFO:teuthology.orchestra.run.smithi099.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-08T00:10:20.574 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-08T00:10:20.574 INFO:teuthology.orchestra.run.smithi099.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-10-08T00:10:20.575 INFO:teuthology.orchestra.run.smithi099.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-08T00:10:20.575 INFO:teuthology.orchestra.run.smithi099.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-08T00:10:20.575 INFO:teuthology.orchestra.run.smithi099.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-08T00:10:20.575 INFO:teuthology.orchestra.run.smithi099.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-08T00:10:20.575 INFO:teuthology.orchestra.run.smithi099.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-08T00:10:20.575 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-08T00:10:20.575 INFO:teuthology.orchestra.run.smithi099.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-08T00:10:20.575 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-08T00:10:20.575 INFO:teuthology.orchestra.run.smithi099.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-08T00:10:20.575 INFO:teuthology.orchestra.run.smithi099.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-08T00:10:20.575 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.575 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.576 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-nbd-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:10:20.576 INFO:teuthology.orchestra.run.smithi099.stdout: socat-1.7.3.3-2.el8.x86_64 2023-10-08T00:10:20.576 INFO:teuthology.orchestra.run.smithi099.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-08T00:10:20.576 INFO:teuthology.orchestra.run.smithi099.stdout: xfsprogs-devel-5.0.0-8.el8.x86_64 2023-10-08T00:10:20.576 INFO:teuthology.orchestra.run.smithi099.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-08T00:10:20.576 INFO:teuthology.orchestra.run.smithi099.stdout: zlib-devel-1.2.11-17.el8.x86_64 2023-10-08T00:10:20.576 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:10:20.577 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2023-10-08T00:10:20.754 DEBUG:teuthology.parallel:result is None 2023-10-08T00:10:20.755 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=08cac0187919fc3a247efdf445f652e2fb30a647 2023-10-08T00:10:20.987 DEBUG:teuthology.orchestra.run.smithi099:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-08T00:10:21.008 INFO:teuthology.orchestra.run.smithi099.stdout:17.2.6-1320.g08cac018.el8 2023-10-08T00:10:21.009 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1320.g08cac018.el8 2023-10-08T00:10:21.009 INFO:teuthology.task.install:The correct ceph version 17.2.6-1320.g08cac018 is installed. 2023-10-08T00:10:21.010 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=08cac0187919fc3a247efdf445f652e2fb30a647 2023-10-08T00:10:21.233 DEBUG:teuthology.orchestra.run.smithi191:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-08T00:10:21.256 INFO:teuthology.orchestra.run.smithi191.stdout:17.2.6-1320.g08cac018.el8 2023-10-08T00:10:21.256 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1320.g08cac018.el8 2023-10-08T00:10:21.256 INFO:teuthology.task.install:The correct ceph version 17.2.6-1320.g08cac018 is installed. 2023-10-08T00:10:21.256 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-08T00:10:21.256 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-08T00:10:21.256 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-08T00:10:21.295 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2023-10-08T00:10:21.295 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-08T00:10:21.335 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-08T00:10:21.336 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-08T00:10:21.336 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/daemon-helper 2023-10-08T00:10:21.360 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-08T00:10:21.424 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2023-10-08T00:10:21.424 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/usr/bin/daemon-helper 2023-10-08T00:10:21.450 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-08T00:10:21.514 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-08T00:10:21.515 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-08T00:10:21.515 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-08T00:10:21.538 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-08T00:10:21.602 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2023-10-08T00:10:21.603 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-08T00:10:21.627 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-08T00:10:21.692 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-08T00:10:21.692 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-08T00:10:21.693 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/stdin-killer 2023-10-08T00:10:21.717 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-08T00:10:21.781 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2023-10-08T00:10:21.781 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/usr/bin/stdin-killer 2023-10-08T00:10:21.805 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-08T00:10:21.870 INFO:teuthology.run_tasks:Running task ceph... 2023-10-08T00:10:21.936 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-10-08T00:10:21.937 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 777 /var/log/ceph 2023-10-08T00:10:21.938 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 777 /var/log/ceph 2023-10-08T00:10:21.962 INFO:tasks.ceph:Disabling ceph logrotate... 2023-10-08T00:10:21.963 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-08T00:10:22.003 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-08T00:10:22.028 INFO:tasks.ceph:Creating extra log directories... 2023-10-08T00:10:22.029 DEBUG:teuthology.orchestra.run.smithi099:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-08T00:10:22.068 DEBUG:teuthology.orchestra.run.smithi191:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-08T00:10:22.176 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-10-08T00:10:22.177 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-10-08T00:10:22.177 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-10-08T00:10:22.177 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-08T00:10:22.177 DEBUG:teuthology.orchestra.run.smithi099:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-08T00:10:22.192 DEBUG:teuthology.orchestra.run.smithi099:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-08T00:10:22.302 DEBUG:teuthology.orchestra.run.smithi099:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-08T00:10:22.327 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2023-10-08T00:10:22.327 DEBUG:teuthology.orchestra.run.smithi191:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-08T00:10:22.342 DEBUG:teuthology.orchestra.run.smithi191:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-08T00:10:22.451 DEBUG:teuthology.orchestra.run.smithi191:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-08T00:10:22.481 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-10-08T00:10:22.481 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL', 'PG_RECOVERY_FULL', 'PG_DEGRADED'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-10-08T00:10:22.481 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418110', 'branch': 'wip-yuri-testing-2023-10-06-0949-quincy', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7418110', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.4', 'overrides': {'admin_socket': {'branch': 'wip-yuri-testing-2023-10-06-0949-quincy'}, 'ceph': {'cephfs': {'ec_profile': ['disabled']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL', 'PG_RECOVERY_FULL', 'PG_DEGRADED'], 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri-testing-2023-10-07-1508-quincy', 'sha1': 'abf19fc7a04fe4fceab4e98d4013ae1da7c06006'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 74, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 8174, 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647', 'sleep_before_teardown': 0, 'subset': '111/12000', 'suite': 'fs', 'suite_branch': 'wip-yuri-testing-2023-10-07-1508-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_abf19fc7a04fe4fceab4e98d4013ae1da7c06006/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'abf19fc7a04fe4fceab4e98d4013ae1da7c06006', 'targets': {'smithi099.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD5LwXEONYcXRmTEhW6npr5lkKPFXj2fVvjuLfb0XPQFOInHhkdkKYBEEFID0gPuAhmpfy5GOshhTa4hwoSPgBk=', 'smithi191.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDoCm6mEVQTmB6VkKOVpObIqmV+5V75Vd5lLvS4FoTXnIeMM/GF1r4oBcMw/DMRs5W3hz8Dp6kZyQGqVobIVqRk='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['disabled']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL', 'PG_RECOVERY_FULL', 'PG_DEGRADED'], 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_full']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '54e62bcbac4e53d9685e08328b790d3b20d71cae', 'timestamp': '2023-10-07_22:15:20', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-10-08T00:10:22.482 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-08T00:10:22.484 DEBUG:teuthology.orchestra.run.smithi191:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-08T00:10:22.530 DEBUG:teuthology.orchestra.run.smithi099:> sudo install -d -m0777 -- /var/run/ceph 2023-10-08T00:10:22.541 DEBUG:teuthology.orchestra.run.smithi191:> sudo install -d -m0777 -- /var/run/ceph 2023-10-08T00:10:22.593 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-08T00:10:22.593 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/scratch_devs of=/dev/stdout 2023-10-08T00:10:22.618 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-08T00:10:22.619 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_1 2023-10-08T00:10:22.674 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-08T00:10:22.675 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T00:10:22.675 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 6h/6d Inode: 218016 Links: 1 2023-10-08T00:10:22.675 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T00:10:22.676 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T00:10:22.676 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2023-10-08 00:10:13.354132826 +0000 2023-10-08T00:10:22.676 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2023-10-08 00:07:45.097992705 +0000 2023-10-08T00:10:22.676 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2023-10-08 00:07:45.097992705 +0000 2023-10-08T00:10:22.676 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: - 2023-10-08T00:10:22.677 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-08T00:10:22.737 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2023-10-08T00:10:22.737 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2023-10-08T00:10:22.737 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000250474 s, 2.0 MB/s 2023-10-08T00:10:22.738 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-08T00:10:22.794 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_2 2023-10-08T00:10:22.849 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-08T00:10:22.850 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T00:10:22.850 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 6h/6d Inode: 223254 Links: 1 2023-10-08T00:10:22.850 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T00:10:22.850 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T00:10:22.850 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2023-10-08 00:10:13.354132826 +0000 2023-10-08T00:10:22.850 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2023-10-08 00:07:45.596981517 +0000 2023-10-08T00:10:22.850 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2023-10-08 00:07:45.596981517 +0000 2023-10-08T00:10:22.850 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: - 2023-10-08T00:10:22.851 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-08T00:10:22.912 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2023-10-08T00:10:22.912 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2023-10-08T00:10:22.912 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000204397 s, 2.5 MB/s 2023-10-08T00:10:22.913 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-08T00:10:22.969 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_3 2023-10-08T00:10:23.024 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-08T00:10:23.024 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T00:10:23.024 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 6h/6d Inode: 223385 Links: 1 2023-10-08T00:10:23.025 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T00:10:23.025 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T00:10:23.025 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2023-10-08 00:10:13.354132826 +0000 2023-10-08T00:10:23.025 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2023-10-08 00:07:46.097970285 +0000 2023-10-08T00:10:23.025 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2023-10-08 00:07:46.097970285 +0000 2023-10-08T00:10:23.025 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: - 2023-10-08T00:10:23.026 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-08T00:10:23.087 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2023-10-08T00:10:23.087 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2023-10-08T00:10:23.087 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.0002411 s, 2.1 MB/s 2023-10-08T00:10:23.088 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-08T00:10:23.144 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_4 2023-10-08T00:10:23.198 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-08T00:10:23.198 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T00:10:23.198 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 6h/6d Inode: 216762 Links: 1 2023-10-08T00:10:23.198 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T00:10:23.198 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T00:10:23.198 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2023-10-08 00:10:13.354132826 +0000 2023-10-08T00:10:23.198 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2023-10-08 00:07:46.598959053 +0000 2023-10-08T00:10:23.198 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2023-10-08 00:07:46.598959053 +0000 2023-10-08T00:10:23.199 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: - 2023-10-08T00:10:23.199 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-08T00:10:23.261 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2023-10-08T00:10:23.261 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2023-10-08T00:10:23.261 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000198354 s, 2.6 MB/s 2023-10-08T00:10:23.262 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-08T00:10:23.320 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-08T00:10:23.320 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2023-10-08T00:10:23.320 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/scratch_devs of=/dev/stdout 2023-10-08T00:10:23.336 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-08T00:10:23.336 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_1 2023-10-08T00:10:23.392 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-08T00:10:23.392 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T00:10:23.392 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 6h/6d Inode: 217519 Links: 1 2023-10-08T00:10:23.392 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T00:10:23.393 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T00:10:23.393 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2023-10-08 00:10:14.810606683 +0000 2023-10-08T00:10:23.393 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2023-10-08 00:07:45.074891872 +0000 2023-10-08T00:10:23.394 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2023-10-08 00:07:45.074891872 +0000 2023-10-08T00:10:23.394 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: - 2023-10-08T00:10:23.394 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-08T00:10:23.456 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2023-10-08T00:10:23.456 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2023-10-08T00:10:23.456 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000279645 s, 1.8 MB/s 2023-10-08T00:10:23.458 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-08T00:10:23.514 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_2 2023-10-08T00:10:23.570 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-08T00:10:23.570 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T00:10:23.571 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 6h/6d Inode: 223265 Links: 1 2023-10-08T00:10:23.571 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T00:10:23.571 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T00:10:23.571 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2023-10-08 00:10:14.810606683 +0000 2023-10-08T00:10:23.571 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2023-10-08 00:07:45.584883481 +0000 2023-10-08T00:10:23.571 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2023-10-08 00:07:45.584883481 +0000 2023-10-08T00:10:23.571 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: - 2023-10-08T00:10:23.571 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-08T00:10:23.635 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2023-10-08T00:10:23.635 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2023-10-08T00:10:23.635 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000223571 s, 2.3 MB/s 2023-10-08T00:10:23.637 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-08T00:10:23.692 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_3 2023-10-08T00:10:23.747 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-08T00:10:23.748 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T00:10:23.748 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 6h/6d Inode: 217638 Links: 1 2023-10-08T00:10:23.748 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T00:10:23.748 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T00:10:23.748 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2023-10-08 00:10:14.810606683 +0000 2023-10-08T00:10:23.748 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2023-10-08 00:07:46.058875682 +0000 2023-10-08T00:10:23.748 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2023-10-08 00:07:46.058875682 +0000 2023-10-08T00:10:23.749 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: - 2023-10-08T00:10:23.749 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-08T00:10:23.811 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2023-10-08T00:10:23.811 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2023-10-08T00:10:23.812 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000294989 s, 1.7 MB/s 2023-10-08T00:10:23.813 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-08T00:10:23.869 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_4 2023-10-08T00:10:23.924 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-08T00:10:23.924 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T00:10:23.924 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 6h/6d Inode: 223380 Links: 1 2023-10-08T00:10:23.924 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T00:10:23.924 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T00:10:23.924 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2023-10-08 00:10:14.810606683 +0000 2023-10-08T00:10:23.925 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2023-10-08 00:07:46.535867834 +0000 2023-10-08T00:10:23.925 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2023-10-08 00:07:46.535867834 +0000 2023-10-08T00:10:23.925 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: - 2023-10-08T00:10:23.925 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-08T00:10:23.987 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2023-10-08T00:10:23.987 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2023-10-08T00:10:23.987 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000188252 s, 2.7 MB/s 2023-10-08T00:10:23.988 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-08T00:10:24.044 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-08T00:10:24.044 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi099.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi191.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-08T00:10:24.044 INFO:tasks.ceph:Generating config... 2023-10-08T00:10:24.045 INFO:tasks.ceph:[client] client mount timeout = 600 2023-10-08T00:10:24.046 INFO:tasks.ceph:[client] debug client = 20 2023-10-08T00:10:24.046 INFO:tasks.ceph:[client] debug ms = 1 2023-10-08T00:10:24.046 INFO:tasks.ceph:[client] pid file = 2023-10-08T00:10:24.046 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-10-08T00:10:24.046 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-10-08T00:10:24.046 INFO:tasks.ceph:[client.0] debug client = 20 2023-10-08T00:10:24.046 INFO:tasks.ceph:[client.0] debug objectcacher = 20 2023-10-08T00:10:24.046 INFO:tasks.ceph:[client.0] debug objecter = 20 2023-10-08T00:10:24.046 INFO:tasks.ceph:[client.1] debug client = 20 2023-10-08T00:10:24.047 INFO:tasks.ceph:[client.1] debug objectcacher = 20 2023-10-08T00:10:24.047 INFO:tasks.ceph:[client.1] debug objecter = 20 2023-10-08T00:10:24.047 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-10-08T00:10:24.047 INFO:tasks.ceph:[mds] debug mds = 20 2023-10-08T00:10:24.047 INFO:tasks.ceph:[mds] debug ms = 1 2023-10-08T00:10:24.047 INFO:tasks.ceph:[mds] mds debug frag = True 2023-10-08T00:10:24.047 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-10-08T00:10:24.048 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-10-08T00:10:24.048 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-10-08T00:10:24.048 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-10-08T00:10:24.049 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-10-08T00:10:24.049 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-10-08T00:10:24.049 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-10-08T00:10:24.049 INFO:tasks.ceph:[mgr] debug ms = 1 2023-10-08T00:10:24.049 INFO:tasks.ceph:[mon] debug mon = 20 2023-10-08T00:10:24.049 INFO:tasks.ceph:[mon] debug ms = 1 2023-10-08T00:10:24.049 INFO:tasks.ceph:[mon] debug paxos = 20 2023-10-08T00:10:24.049 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-10-08T00:10:24.049 INFO:tasks.ceph:[mon] mon osd backfillfull ratio = 0.6 2023-10-08T00:10:24.050 INFO:tasks.ceph:[mon] mon osd full ratio = 0.7 2023-10-08T00:10:24.050 INFO:tasks.ceph:[mon] mon osd nearfull ratio = 0.6 2023-10-08T00:10:24.050 INFO:tasks.ceph:[osd] bdev async discard = True 2023-10-08T00:10:24.050 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-10-08T00:10:24.050 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-10-08T00:10:24.050 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-10-08T00:10:24.050 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-10-08T00:10:24.050 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-10-08T00:10:24.051 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-10-08T00:10:24.051 INFO:tasks.ceph:[osd] debug ms = 1 2023-10-08T00:10:24.052 INFO:tasks.ceph:[osd] debug osd = 20 2023-10-08T00:10:24.052 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-10-08T00:10:24.052 INFO:tasks.ceph:[osd] memstore device bytes = 200000000 2023-10-08T00:10:24.052 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-10-08T00:10:24.052 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-10-08T00:10:24.053 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-10-08T00:10:24.053 INFO:tasks.ceph:[osd] osd failsafe full ratio = 1.0 2023-10-08T00:10:24.053 INFO:tasks.ceph:[osd] osd mon report interval = 5 2023-10-08T00:10:24.053 INFO:tasks.ceph:[osd] osd objectstore = memstore 2023-10-08T00:10:24.053 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-10-08T00:10:24.054 INFO:tasks.ceph:Setting up mon.a... 2023-10-08T00:10:24.054 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-10-08T00:10:24.102 INFO:teuthology.orchestra.run.smithi099.stdout:creating /etc/ceph/ceph.keyring 2023-10-08T00:10:24.104 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-10-08T00:10:24.144 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-08T00:10:24.210 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.99'), ('mon.b', '172.21.15.191'), ('mon.c', '[v2:172.21.15.191:3301,v1:172.21.15.191:6790]')] 2023-10-08T00:10:24.210 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.99,172.21.15.191,[v2:172.21.15.191:3301,v1:172.21.15.191:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-10-08T00:10:24.211 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-08T00:10:24.211 DEBUG:teuthology.orchestra.run.smithi099:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-10-08T00:10:24.265 DEBUG:teuthology.orchestra.run.smithi099:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.99 --add b 172.21.15.191 --addv c '[v2:172.21.15.191:3301,v1:172.21.15.191:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-10-08T00:10:24.347 INFO:teuthology.orchestra.run.smithi099.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-10-08T00:10:24.347 INFO:teuthology.orchestra.run.smithi099.stdout:monmaptool: generated fsid 06962cdb-9a6f-417f-82fd-9e4e3dedbccf 2023-10-08T00:10:24.348 INFO:teuthology.orchestra.run.smithi099.stdout:setting min_mon_release = octopus 2023-10-08T00:10:24.348 INFO:teuthology.orchestra.run.smithi099.stdout:epoch 0 2023-10-08T00:10:24.348 INFO:teuthology.orchestra.run.smithi099.stdout:fsid 06962cdb-9a6f-417f-82fd-9e4e3dedbccf 2023-10-08T00:10:24.348 INFO:teuthology.orchestra.run.smithi099.stdout:last_changed 2023-10-08T00:10:24.346677+0000 2023-10-08T00:10:24.348 INFO:teuthology.orchestra.run.smithi099.stdout:created 2023-10-08T00:10:24.346677+0000 2023-10-08T00:10:24.348 INFO:teuthology.orchestra.run.smithi099.stdout:min_mon_release 15 (octopus) 2023-10-08T00:10:24.348 INFO:teuthology.orchestra.run.smithi099.stdout:election_strategy: 1 2023-10-08T00:10:24.348 INFO:teuthology.orchestra.run.smithi099.stdout:0: [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] mon.a 2023-10-08T00:10:24.348 INFO:teuthology.orchestra.run.smithi099.stdout:1: [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] mon.b 2023-10-08T00:10:24.348 INFO:teuthology.orchestra.run.smithi099.stdout:2: [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] mon.c 2023-10-08T00:10:24.348 INFO:teuthology.orchestra.run.smithi099.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-10-08T00:10:24.349 DEBUG:teuthology.orchestra.run.smithi099:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-10-08T00:10:24.402 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 06962cdb-9a6f-417f-82fd-9e4e3dedbccf... 2023-10-08T00:10:24.403 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-08T00:10:24.444 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-08T00:10:24.484 INFO:teuthology.orchestra.run.smithi099.stdout:[global] 2023-10-08T00:10:24.484 INFO:teuthology.orchestra.run.smithi099.stdout: chdir = "" 2023-10-08T00:10:24.484 INFO:teuthology.orchestra.run.smithi099.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-08T00:10:24.484 INFO:teuthology.orchestra.run.smithi099.stdout: auth supported = cephx 2023-10-08T00:10:24.484 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:10:24.484 INFO:teuthology.orchestra.run.smithi099.stdout: filestore xattr use omap = true 2023-10-08T00:10:24.484 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:10:24.485 INFO:teuthology.orchestra.run.smithi099.stdout: mon clock drift allowed = 1.000 2023-10-08T00:10:24.485 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:10:24.485 INFO:teuthology.orchestra.run.smithi099.stdout: osd crush chooseleaf type = 0 2023-10-08T00:10:24.485 INFO:teuthology.orchestra.run.smithi099.stdout: auth debug = true 2023-10-08T00:10:24.485 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:10:24.485 INFO:teuthology.orchestra.run.smithi099.stdout: ms die on old message = true 2023-10-08T00:10:24.485 INFO:teuthology.orchestra.run.smithi099.stdout: ms die on bug = true 2023-10-08T00:10:24.485 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:10:24.485 INFO:teuthology.orchestra.run.smithi099.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-08T00:10:24.485 INFO:teuthology.orchestra.run.smithi099.stdout: mon pg warn max object skew = 0 2023-10-08T00:10:24.485 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:10:24.485 INFO:teuthology.orchestra.run.smithi099.stdout: # disable pg_autoscaler by default for new pools 2023-10-08T00:10:24.485 INFO:teuthology.orchestra.run.smithi099.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-08T00:10:24.486 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:10:24.486 INFO:teuthology.orchestra.run.smithi099.stdout: osd pool default size = 2 2023-10-08T00:10:24.486 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:10:24.486 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd allow primary affinity = true 2023-10-08T00:10:24.486 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd allow pg remap = true 2023-10-08T00:10:24.486 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on legacy crush tunables = false 2023-10-08T00:10:24.486 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on crush straw calc version zero = false 2023-10-08T00:10:24.486 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on no sortbitwise = false 2023-10-08T00:10:24.486 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on osd down out interval zero = false 2023-10-08T00:10:24.486 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on too few osds = false 2023-10-08T00:10:24.486 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-08T00:10:24.487 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-08T00:10:24.487 INFO:teuthology.orchestra.run.smithi099.stdout: mon_allow_pool_size_one = true 2023-10-08T00:10:24.487 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:10:24.487 INFO:teuthology.orchestra.run.smithi099.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-08T00:10:24.487 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:10:24.487 INFO:teuthology.orchestra.run.smithi099.stdout: osd default data pool replay window = 5 2023-10-08T00:10:24.487 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:10:24.487 INFO:teuthology.orchestra.run.smithi099.stdout: mon allow pool delete = true 2023-10-08T00:10:24.487 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:10:24.487 INFO:teuthology.orchestra.run.smithi099.stdout: mon cluster log file level = debug 2023-10-08T00:10:24.487 INFO:teuthology.orchestra.run.smithi099.stdout: debug asserts on shutdown = true 2023-10-08T00:10:24.487 INFO:teuthology.orchestra.run.smithi099.stdout: mon health detail to clog = false 2023-10-08T00:10:24.488 INFO:teuthology.orchestra.run.smithi099.stdout: mon host = "172.21.15.99,172.21.15.191,[v2:172.21.15.191:3301,v1:172.21.15.191:6790]" 2023-10-08T00:10:24.488 INFO:teuthology.orchestra.run.smithi099.stdout: ms die on skipped message = False 2023-10-08T00:10:24.488 INFO:teuthology.orchestra.run.smithi099.stdout: fsid = 06962cdb-9a6f-417f-82fd-9e4e3dedbccf 2023-10-08T00:10:24.488 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:10:24.488 INFO:teuthology.orchestra.run.smithi099.stdout:[osd] 2023-10-08T00:10:24.488 INFO:teuthology.orchestra.run.smithi099.stdout: osd journal size = 100 2023-10-08T00:10:24.488 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:10:24.488 INFO:teuthology.orchestra.run.smithi099.stdout: osd scrub load threshold = 5.0 2023-10-08T00:10:24.488 INFO:teuthology.orchestra.run.smithi099.stdout: osd scrub max interval = 600 2023-10-08T00:10:24.488 INFO:teuthology.orchestra.run.smithi099.stdout: osd mclock profile = high_recovery_ops 2023-10-08T00:10:24.488 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:10:24.488 INFO:teuthology.orchestra.run.smithi099.stdout: osd recover clone overlap = true 2023-10-08T00:10:24.488 INFO:teuthology.orchestra.run.smithi099.stdout: osd recovery max chunk = 1048576 2023-10-08T00:10:24.489 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:10:24.489 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug shutdown = true 2023-10-08T00:10:24.489 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug op order = true 2023-10-08T00:10:24.489 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug verify stray on activate = true 2023-10-08T00:10:24.489 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:10:24.489 INFO:teuthology.orchestra.run.smithi099.stdout: osd open classes on start = true 2023-10-08T00:10:24.489 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug pg log writeout = true 2023-10-08T00:10:24.489 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:10:24.489 INFO:teuthology.orchestra.run.smithi099.stdout: osd deep scrub update digest min age = 30 2023-10-08T00:10:24.489 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:10:24.489 INFO:teuthology.orchestra.run.smithi099.stdout: osd map max advance = 10 2023-10-08T00:10:24.489 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:10:24.490 INFO:teuthology.orchestra.run.smithi099.stdout: journal zero on create = true 2023-10-08T00:10:24.490 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:10:24.490 INFO:teuthology.orchestra.run.smithi099.stdout: filestore ondisk finisher threads = 3 2023-10-08T00:10:24.490 INFO:teuthology.orchestra.run.smithi099.stdout: filestore apply finisher threads = 3 2023-10-08T00:10:24.490 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:10:24.490 INFO:teuthology.orchestra.run.smithi099.stdout: bdev debug aio = true 2023-10-08T00:10:24.490 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug misdirected ops = true 2023-10-08T00:10:24.490 INFO:teuthology.orchestra.run.smithi099.stdout: bdev async discard = True 2023-10-08T00:10:24.490 INFO:teuthology.orchestra.run.smithi099.stdout: bdev enable discard = True 2023-10-08T00:10:24.490 INFO:teuthology.orchestra.run.smithi099.stdout: bluestore allocator = bitmap 2023-10-08T00:10:24.490 INFO:teuthology.orchestra.run.smithi099.stdout: bluestore block size = 96636764160 2023-10-08T00:10:24.490 INFO:teuthology.orchestra.run.smithi099.stdout: bluestore fsck on mount = True 2023-10-08T00:10:24.491 INFO:teuthology.orchestra.run.smithi099.stdout: debug bluefs = 1/20 2023-10-08T00:10:24.491 INFO:teuthology.orchestra.run.smithi099.stdout: debug bluestore = 1/20 2023-10-08T00:10:24.491 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2023-10-08T00:10:24.491 INFO:teuthology.orchestra.run.smithi099.stdout: debug osd = 20 2023-10-08T00:10:24.491 INFO:teuthology.orchestra.run.smithi099.stdout: debug rocksdb = 4/10 2023-10-08T00:10:24.491 INFO:teuthology.orchestra.run.smithi099.stdout: memstore device bytes = 200000000 2023-10-08T00:10:24.491 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd backfillfull_ratio = 0.85 2023-10-08T00:10:24.491 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd full ratio = 0.9 2023-10-08T00:10:24.491 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd nearfull ratio = 0.8 2023-10-08T00:10:24.491 INFO:teuthology.orchestra.run.smithi099.stdout: osd failsafe full ratio = 1.0 2023-10-08T00:10:24.491 INFO:teuthology.orchestra.run.smithi099.stdout: osd mon report interval = 5 2023-10-08T00:10:24.491 INFO:teuthology.orchestra.run.smithi099.stdout: osd objectstore = memstore 2023-10-08T00:10:24.492 INFO:teuthology.orchestra.run.smithi099.stdout: osd op complaint time = 180 2023-10-08T00:10:24.492 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:10:24.492 INFO:teuthology.orchestra.run.smithi099.stdout:[mgr] 2023-10-08T00:10:24.492 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2023-10-08T00:10:24.492 INFO:teuthology.orchestra.run.smithi099.stdout: debug mgr = 20 2023-10-08T00:10:24.492 INFO:teuthology.orchestra.run.smithi099.stdout: debug mon = 20 2023-10-08T00:10:24.492 INFO:teuthology.orchestra.run.smithi099.stdout: debug auth = 20 2023-10-08T00:10:24.492 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min pgs per osd = 4 2023-10-08T00:10:24.492 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min bytes per osd = 10 2023-10-08T00:10:24.492 INFO:teuthology.orchestra.run.smithi099.stdout: mgr/telemetry/nag = false 2023-10-08T00:10:24.492 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:10:24.492 INFO:teuthology.orchestra.run.smithi099.stdout:[mon] 2023-10-08T00:10:24.492 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2023-10-08T00:10:24.493 INFO:teuthology.orchestra.run.smithi099.stdout: debug mon = 20 2023-10-08T00:10:24.493 INFO:teuthology.orchestra.run.smithi099.stdout: debug paxos = 20 2023-10-08T00:10:24.493 INFO:teuthology.orchestra.run.smithi099.stdout: debug auth = 20 2023-10-08T00:10:24.493 INFO:teuthology.orchestra.run.smithi099.stdout: mon data avail warn = 5 2023-10-08T00:10:24.493 INFO:teuthology.orchestra.run.smithi099.stdout: mon mgr mkfs grace = 240 2023-10-08T00:10:24.493 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min pgs per osd = 4 2023-10-08T00:10:24.493 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd reporter subtree level = osd 2023-10-08T00:10:24.493 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd prime pg temp = true 2023-10-08T00:10:24.493 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min bytes per osd = 10 2023-10-08T00:10:24.493 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:10:24.493 INFO:teuthology.orchestra.run.smithi099.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-08T00:10:24.493 INFO:teuthology.orchestra.run.smithi099.stdout: auth mon ticket ttl = 660 # 11m 2023-10-08T00:10:24.494 INFO:teuthology.orchestra.run.smithi099.stdout: auth service ticket ttl = 240 # 4m 2023-10-08T00:10:24.494 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:10:24.494 INFO:teuthology.orchestra.run.smithi099.stdout: # don't complain about insecure global_id in the test suite 2023-10-08T00:10:24.494 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-08T00:10:24.494 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-08T00:10:24.494 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:10:24.494 INFO:teuthology.orchestra.run.smithi099.stdout: # 1m isn't quite enough 2023-10-08T00:10:24.494 INFO:teuthology.orchestra.run.smithi099.stdout: mon_down_mkfs_grace = 2m 2023-10-08T00:10:24.494 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:10:24.494 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_filestore_osds = false 2023-10-08T00:10:24.494 INFO:teuthology.orchestra.run.smithi099.stdout: mon op complaint time = 120 2023-10-08T00:10:24.494 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd backfillfull ratio = 0.6 2023-10-08T00:10:24.495 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd full ratio = 0.7 2023-10-08T00:10:24.495 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd nearfull ratio = 0.6 2023-10-08T00:10:24.495 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:10:24.495 INFO:teuthology.orchestra.run.smithi099.stdout:[client] 2023-10-08T00:10:24.495 INFO:teuthology.orchestra.run.smithi099.stdout: rgw cache enabled = true 2023-10-08T00:10:24.495 INFO:teuthology.orchestra.run.smithi099.stdout: rgw enable ops log = true 2023-10-08T00:10:24.495 INFO:teuthology.orchestra.run.smithi099.stdout: rgw enable usage log = true 2023-10-08T00:10:24.495 INFO:teuthology.orchestra.run.smithi099.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-08T00:10:24.495 INFO:teuthology.orchestra.run.smithi099.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-08T00:10:24.495 INFO:teuthology.orchestra.run.smithi099.stdout: client mount timeout = 600 2023-10-08T00:10:24.496 INFO:teuthology.orchestra.run.smithi099.stdout: debug client = 20 2023-10-08T00:10:24.496 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2023-10-08T00:10:24.496 INFO:teuthology.orchestra.run.smithi099.stdout: pid file = "" 2023-10-08T00:10:24.496 INFO:teuthology.orchestra.run.smithi099.stdout: rados mon op timeout = 900 2023-10-08T00:10:24.496 INFO:teuthology.orchestra.run.smithi099.stdout: rados osd op timeout = 900 2023-10-08T00:10:24.496 INFO:teuthology.orchestra.run.smithi099.stdout:[mon.a] 2023-10-08T00:10:24.496 INFO:teuthology.orchestra.run.smithi099.stdout:[mon.b] 2023-10-08T00:10:24.496 INFO:teuthology.orchestra.run.smithi099.stdout:[mon.c] 2023-10-08T00:10:24.496 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.a] 2023-10-08T00:10:24.496 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.c] 2023-10-08T00:10:24.496 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.b] 2023-10-08T00:10:24.497 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.d] 2023-10-08T00:10:24.497 INFO:teuthology.orchestra.run.smithi099.stdout:[client.0] 2023-10-08T00:10:24.497 INFO:teuthology.orchestra.run.smithi099.stdout: debug client = 20 2023-10-08T00:10:24.497 INFO:teuthology.orchestra.run.smithi099.stdout: debug objectcacher = 20 2023-10-08T00:10:24.497 INFO:teuthology.orchestra.run.smithi099.stdout: debug objecter = 20 2023-10-08T00:10:24.497 INFO:teuthology.orchestra.run.smithi099.stdout:[client.1] 2023-10-08T00:10:24.497 INFO:teuthology.orchestra.run.smithi099.stdout: debug client = 20 2023-10-08T00:10:24.497 INFO:teuthology.orchestra.run.smithi099.stdout: debug objectcacher = 20 2023-10-08T00:10:24.497 INFO:teuthology.orchestra.run.smithi099.stdout: debug objecter = 20 2023-10-08T00:10:24.498 INFO:teuthology.orchestra.run.smithi099.stdout:[mds] 2023-10-08T00:10:24.498 INFO:teuthology.orchestra.run.smithi099.stdout: debug mds = 20 2023-10-08T00:10:24.498 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2023-10-08T00:10:24.498 INFO:teuthology.orchestra.run.smithi099.stdout: mds debug frag = True 2023-10-08T00:10:24.498 INFO:teuthology.orchestra.run.smithi099.stdout: mds debug scatterstat = True 2023-10-08T00:10:24.498 INFO:teuthology.orchestra.run.smithi099.stdout: mds op complaint time = 180 2023-10-08T00:10:24.498 INFO:teuthology.orchestra.run.smithi099.stdout: mds verify scatter = True 2023-10-08T00:10:24.498 INFO:teuthology.orchestra.run.smithi099.stdout: osd op complaint time = 180 2023-10-08T00:10:24.498 INFO:teuthology.orchestra.run.smithi099.stdout: rados mon op timeout = 900 2023-10-08T00:10:24.499 INFO:teuthology.orchestra.run.smithi099.stdout: rados osd op timeout = 900 2023-10-08T00:10:24.500 INFO:teuthology.orchestra.run.smithi191.stdout:[global] 2023-10-08T00:10:24.500 INFO:teuthology.orchestra.run.smithi191.stdout: chdir = "" 2023-10-08T00:10:24.501 INFO:teuthology.orchestra.run.smithi191.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-08T00:10:24.501 INFO:teuthology.orchestra.run.smithi191.stdout: auth supported = cephx 2023-10-08T00:10:24.501 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T00:10:24.501 INFO:teuthology.orchestra.run.smithi191.stdout: filestore xattr use omap = true 2023-10-08T00:10:24.501 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T00:10:24.501 INFO:teuthology.orchestra.run.smithi191.stdout: mon clock drift allowed = 1.000 2023-10-08T00:10:24.501 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T00:10:24.501 INFO:teuthology.orchestra.run.smithi191.stdout: osd crush chooseleaf type = 0 2023-10-08T00:10:24.501 INFO:teuthology.orchestra.run.smithi191.stdout: auth debug = true 2023-10-08T00:10:24.501 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T00:10:24.502 INFO:teuthology.orchestra.run.smithi191.stdout: ms die on old message = true 2023-10-08T00:10:24.502 INFO:teuthology.orchestra.run.smithi191.stdout: ms die on bug = true 2023-10-08T00:10:24.502 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T00:10:24.502 INFO:teuthology.orchestra.run.smithi191.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-08T00:10:24.502 INFO:teuthology.orchestra.run.smithi191.stdout: mon pg warn max object skew = 0 2023-10-08T00:10:24.502 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T00:10:24.502 INFO:teuthology.orchestra.run.smithi191.stdout: # disable pg_autoscaler by default for new pools 2023-10-08T00:10:24.502 INFO:teuthology.orchestra.run.smithi191.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-08T00:10:24.502 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T00:10:24.503 INFO:teuthology.orchestra.run.smithi191.stdout: osd pool default size = 2 2023-10-08T00:10:24.503 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T00:10:24.503 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd allow primary affinity = true 2023-10-08T00:10:24.503 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd allow pg remap = true 2023-10-08T00:10:24.503 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on legacy crush tunables = false 2023-10-08T00:10:24.503 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on crush straw calc version zero = false 2023-10-08T00:10:24.503 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on no sortbitwise = false 2023-10-08T00:10:24.503 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on osd down out interval zero = false 2023-10-08T00:10:24.504 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on too few osds = false 2023-10-08T00:10:24.504 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-08T00:10:24.504 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-08T00:10:24.504 INFO:teuthology.orchestra.run.smithi191.stdout: mon_allow_pool_size_one = true 2023-10-08T00:10:24.504 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T00:10:24.504 INFO:teuthology.orchestra.run.smithi191.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-08T00:10:24.504 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T00:10:24.504 INFO:teuthology.orchestra.run.smithi191.stdout: osd default data pool replay window = 5 2023-10-08T00:10:24.504 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T00:10:24.505 INFO:teuthology.orchestra.run.smithi191.stdout: mon allow pool delete = true 2023-10-08T00:10:24.505 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T00:10:24.505 INFO:teuthology.orchestra.run.smithi191.stdout: mon cluster log file level = debug 2023-10-08T00:10:24.505 INFO:teuthology.orchestra.run.smithi191.stdout: debug asserts on shutdown = true 2023-10-08T00:10:24.505 INFO:teuthology.orchestra.run.smithi191.stdout: mon health detail to clog = false 2023-10-08T00:10:24.505 INFO:teuthology.orchestra.run.smithi191.stdout: mon host = "172.21.15.99,172.21.15.191,[v2:172.21.15.191:3301,v1:172.21.15.191:6790]" 2023-10-08T00:10:24.505 INFO:teuthology.orchestra.run.smithi191.stdout: ms die on skipped message = False 2023-10-08T00:10:24.505 INFO:teuthology.orchestra.run.smithi191.stdout: fsid = 06962cdb-9a6f-417f-82fd-9e4e3dedbccf 2023-10-08T00:10:24.505 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T00:10:24.505 INFO:teuthology.orchestra.run.smithi191.stdout:[osd] 2023-10-08T00:10:24.506 INFO:teuthology.orchestra.run.smithi191.stdout: osd journal size = 100 2023-10-08T00:10:24.506 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T00:10:24.506 INFO:teuthology.orchestra.run.smithi191.stdout: osd scrub load threshold = 5.0 2023-10-08T00:10:24.506 INFO:teuthology.orchestra.run.smithi191.stdout: osd scrub max interval = 600 2023-10-08T00:10:24.506 INFO:teuthology.orchestra.run.smithi191.stdout: osd mclock profile = high_recovery_ops 2023-10-08T00:10:24.506 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T00:10:24.506 INFO:teuthology.orchestra.run.smithi191.stdout: osd recover clone overlap = true 2023-10-08T00:10:24.506 INFO:teuthology.orchestra.run.smithi191.stdout: osd recovery max chunk = 1048576 2023-10-08T00:10:24.506 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T00:10:24.506 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug shutdown = true 2023-10-08T00:10:24.506 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug op order = true 2023-10-08T00:10:24.507 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug verify stray on activate = true 2023-10-08T00:10:24.507 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T00:10:24.507 INFO:teuthology.orchestra.run.smithi191.stdout: osd open classes on start = true 2023-10-08T00:10:24.507 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug pg log writeout = true 2023-10-08T00:10:24.507 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T00:10:24.507 INFO:teuthology.orchestra.run.smithi191.stdout: osd deep scrub update digest min age = 30 2023-10-08T00:10:24.507 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T00:10:24.507 INFO:teuthology.orchestra.run.smithi191.stdout: osd map max advance = 10 2023-10-08T00:10:24.507 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T00:10:24.507 INFO:teuthology.orchestra.run.smithi191.stdout: journal zero on create = true 2023-10-08T00:10:24.508 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T00:10:24.508 INFO:teuthology.orchestra.run.smithi191.stdout: filestore ondisk finisher threads = 3 2023-10-08T00:10:24.508 INFO:teuthology.orchestra.run.smithi191.stdout: filestore apply finisher threads = 3 2023-10-08T00:10:24.508 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T00:10:24.508 INFO:teuthology.orchestra.run.smithi191.stdout: bdev debug aio = true 2023-10-08T00:10:24.508 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug misdirected ops = true 2023-10-08T00:10:24.508 INFO:teuthology.orchestra.run.smithi191.stdout: bdev async discard = True 2023-10-08T00:10:24.508 INFO:teuthology.orchestra.run.smithi191.stdout: bdev enable discard = True 2023-10-08T00:10:24.508 INFO:teuthology.orchestra.run.smithi191.stdout: bluestore allocator = bitmap 2023-10-08T00:10:24.508 INFO:teuthology.orchestra.run.smithi191.stdout: bluestore block size = 96636764160 2023-10-08T00:10:24.509 INFO:teuthology.orchestra.run.smithi191.stdout: bluestore fsck on mount = True 2023-10-08T00:10:24.509 INFO:teuthology.orchestra.run.smithi191.stdout: debug bluefs = 1/20 2023-10-08T00:10:24.509 INFO:teuthology.orchestra.run.smithi191.stdout: debug bluestore = 1/20 2023-10-08T00:10:24.509 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2023-10-08T00:10:24.509 INFO:teuthology.orchestra.run.smithi191.stdout: debug osd = 20 2023-10-08T00:10:24.509 INFO:teuthology.orchestra.run.smithi191.stdout: debug rocksdb = 4/10 2023-10-08T00:10:24.509 INFO:teuthology.orchestra.run.smithi191.stdout: memstore device bytes = 200000000 2023-10-08T00:10:24.509 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd backfillfull_ratio = 0.85 2023-10-08T00:10:24.509 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd full ratio = 0.9 2023-10-08T00:10:24.509 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd nearfull ratio = 0.8 2023-10-08T00:10:24.510 INFO:teuthology.orchestra.run.smithi191.stdout: osd failsafe full ratio = 1.0 2023-10-08T00:10:24.510 INFO:teuthology.orchestra.run.smithi191.stdout: osd mon report interval = 5 2023-10-08T00:10:24.510 INFO:teuthology.orchestra.run.smithi191.stdout: osd objectstore = memstore 2023-10-08T00:10:24.510 INFO:teuthology.orchestra.run.smithi191.stdout: osd op complaint time = 180 2023-10-08T00:10:24.510 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T00:10:24.510 INFO:teuthology.orchestra.run.smithi191.stdout:[mgr] 2023-10-08T00:10:24.510 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2023-10-08T00:10:24.510 INFO:teuthology.orchestra.run.smithi191.stdout: debug mgr = 20 2023-10-08T00:10:24.511 INFO:teuthology.orchestra.run.smithi191.stdout: debug mon = 20 2023-10-08T00:10:24.511 INFO:teuthology.orchestra.run.smithi191.stdout: debug auth = 20 2023-10-08T00:10:24.511 INFO:teuthology.orchestra.run.smithi191.stdout: mon reweight min pgs per osd = 4 2023-10-08T00:10:24.511 INFO:teuthology.orchestra.run.smithi191.stdout: mon reweight min bytes per osd = 10 2023-10-08T00:10:24.511 INFO:teuthology.orchestra.run.smithi191.stdout: mgr/telemetry/nag = false 2023-10-08T00:10:24.511 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T00:10:24.511 INFO:teuthology.orchestra.run.smithi191.stdout:[mon] 2023-10-08T00:10:24.511 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2023-10-08T00:10:24.511 INFO:teuthology.orchestra.run.smithi191.stdout: debug mon = 20 2023-10-08T00:10:24.511 INFO:teuthology.orchestra.run.smithi191.stdout: debug paxos = 20 2023-10-08T00:10:24.512 INFO:teuthology.orchestra.run.smithi191.stdout: debug auth = 20 2023-10-08T00:10:24.512 INFO:teuthology.orchestra.run.smithi191.stdout: mon data avail warn = 5 2023-10-08T00:10:24.512 INFO:teuthology.orchestra.run.smithi191.stdout: mon mgr mkfs grace = 240 2023-10-08T00:10:24.512 INFO:teuthology.orchestra.run.smithi191.stdout: mon reweight min pgs per osd = 4 2023-10-08T00:10:24.512 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd reporter subtree level = osd 2023-10-08T00:10:24.512 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd prime pg temp = true 2023-10-08T00:10:24.512 INFO:teuthology.orchestra.run.smithi191.stdout: mon reweight min bytes per osd = 10 2023-10-08T00:10:24.512 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T00:10:24.512 INFO:teuthology.orchestra.run.smithi191.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-08T00:10:24.512 INFO:teuthology.orchestra.run.smithi191.stdout: auth mon ticket ttl = 660 # 11m 2023-10-08T00:10:24.513 INFO:teuthology.orchestra.run.smithi191.stdout: auth service ticket ttl = 240 # 4m 2023-10-08T00:10:24.513 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T00:10:24.513 INFO:teuthology.orchestra.run.smithi191.stdout: # don't complain about insecure global_id in the test suite 2023-10-08T00:10:24.513 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-08T00:10:24.513 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-08T00:10:24.513 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T00:10:24.513 INFO:teuthology.orchestra.run.smithi191.stdout: # 1m isn't quite enough 2023-10-08T00:10:24.513 INFO:teuthology.orchestra.run.smithi191.stdout: mon_down_mkfs_grace = 2m 2023-10-08T00:10:24.513 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T00:10:24.513 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_filestore_osds = false 2023-10-08T00:10:24.514 INFO:teuthology.orchestra.run.smithi191.stdout: mon op complaint time = 120 2023-10-08T00:10:24.514 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd backfillfull ratio = 0.6 2023-10-08T00:10:24.514 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd full ratio = 0.7 2023-10-08T00:10:24.514 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd nearfull ratio = 0.6 2023-10-08T00:10:24.514 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T00:10:24.514 INFO:teuthology.orchestra.run.smithi191.stdout:[client] 2023-10-08T00:10:24.515 INFO:teuthology.orchestra.run.smithi191.stdout: rgw cache enabled = true 2023-10-08T00:10:24.515 INFO:teuthology.orchestra.run.smithi191.stdout: rgw enable ops log = true 2023-10-08T00:10:24.515 INFO:teuthology.orchestra.run.smithi191.stdout: rgw enable usage log = true 2023-10-08T00:10:24.515 INFO:teuthology.orchestra.run.smithi191.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-08T00:10:24.515 INFO:teuthology.orchestra.run.smithi191.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-08T00:10:24.515 INFO:teuthology.orchestra.run.smithi191.stdout: client mount timeout = 600 2023-10-08T00:10:24.515 INFO:teuthology.orchestra.run.smithi191.stdout: debug client = 20 2023-10-08T00:10:24.515 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2023-10-08T00:10:24.515 INFO:teuthology.orchestra.run.smithi191.stdout: pid file = "" 2023-10-08T00:10:24.515 INFO:teuthology.orchestra.run.smithi191.stdout: rados mon op timeout = 900 2023-10-08T00:10:24.516 INFO:teuthology.orchestra.run.smithi191.stdout: rados osd op timeout = 900 2023-10-08T00:10:24.516 INFO:teuthology.orchestra.run.smithi191.stdout:[mon.a] 2023-10-08T00:10:24.516 INFO:teuthology.orchestra.run.smithi191.stdout:[mon.b] 2023-10-08T00:10:24.516 INFO:teuthology.orchestra.run.smithi191.stdout:[mon.c] 2023-10-08T00:10:24.516 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.a] 2023-10-08T00:10:24.516 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.c] 2023-10-08T00:10:24.516 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.b] 2023-10-08T00:10:24.516 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.d] 2023-10-08T00:10:24.516 INFO:teuthology.orchestra.run.smithi191.stdout:[client.0] 2023-10-08T00:10:24.516 INFO:teuthology.orchestra.run.smithi191.stdout: debug client = 20 2023-10-08T00:10:24.516 INFO:teuthology.orchestra.run.smithi191.stdout: debug objectcacher = 20 2023-10-08T00:10:24.516 INFO:teuthology.orchestra.run.smithi191.stdout: debug objecter = 20 2023-10-08T00:10:24.517 INFO:teuthology.orchestra.run.smithi191.stdout:[client.1] 2023-10-08T00:10:24.517 INFO:teuthology.orchestra.run.smithi191.stdout: debug client = 20 2023-10-08T00:10:24.517 INFO:teuthology.orchestra.run.smithi191.stdout: debug objectcacher = 20 2023-10-08T00:10:24.517 INFO:teuthology.orchestra.run.smithi191.stdout: debug objecter = 20 2023-10-08T00:10:24.517 INFO:teuthology.orchestra.run.smithi191.stdout:[mds] 2023-10-08T00:10:24.517 INFO:teuthology.orchestra.run.smithi191.stdout: debug mds = 20 2023-10-08T00:10:24.517 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2023-10-08T00:10:24.517 INFO:teuthology.orchestra.run.smithi191.stdout: mds debug frag = True 2023-10-08T00:10:24.517 INFO:teuthology.orchestra.run.smithi191.stdout: mds debug scatterstat = True 2023-10-08T00:10:24.517 INFO:teuthology.orchestra.run.smithi191.stdout: mds op complaint time = 180 2023-10-08T00:10:24.517 INFO:teuthology.orchestra.run.smithi191.stdout: mds verify scatter = True 2023-10-08T00:10:24.517 INFO:teuthology.orchestra.run.smithi191.stdout: osd op complaint time = 180 2023-10-08T00:10:24.517 INFO:teuthology.orchestra.run.smithi191.stdout: rados mon op timeout = 900 2023-10-08T00:10:24.518 INFO:teuthology.orchestra.run.smithi191.stdout: rados osd op timeout = 900 2023-10-08T00:10:24.518 INFO:tasks.ceph:Creating admin key on mon.a... 2023-10-08T00:10:24.518 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-10-08T00:10:24.591 INFO:tasks.ceph:Copying monmap to all nodes... 2023-10-08T00:10:24.591 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-08T00:10:24.592 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-10-08T00:10:24.607 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-08T00:10:24.607 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-10-08T00:10:24.663 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi099.front.sepia.ceph.com 2023-10-08T00:10:24.663 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-08T00:10:24.663 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-08T00:10:24.664 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-08T00:10:24.737 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-08T00:10:24.737 DEBUG:teuthology.orchestra.run.smithi099:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-08T00:10:24.792 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi191.front.sepia.ceph.com 2023-10-08T00:10:24.792 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2023-10-08T00:10:24.792 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-08T00:10:24.792 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-08T00:10:24.827 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2023-10-08T00:10:24.827 DEBUG:teuthology.orchestra.run.smithi191:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-08T00:10:24.881 INFO:tasks.ceph:Setting up mon nodes... 2023-10-08T00:10:24.881 INFO:tasks.ceph:Setting up mgr nodes... 2023-10-08T00:10:24.882 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-10-08T00:10:24.930 INFO:teuthology.orchestra.run.smithi099.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-10-08T00:10:24.932 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-10-08T00:10:24.990 INFO:teuthology.orchestra.run.smithi191.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-10-08T00:10:24.993 INFO:tasks.ceph:Setting up mds nodes... 2023-10-08T00:10:24.993 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-10-08T00:10:25.041 INFO:teuthology.orchestra.run.smithi099.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-10-08T00:10:25.043 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-10-08T00:10:25.067 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-10-08T00:10:25.154 INFO:teuthology.orchestra.run.smithi099.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-10-08T00:10:25.157 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-10-08T00:10:25.181 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-10-08T00:10:25.229 INFO:teuthology.orchestra.run.smithi191.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-10-08T00:10:25.231 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-10-08T00:10:25.255 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-10-08T00:10:25.342 INFO:teuthology.orchestra.run.smithi191.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-10-08T00:10:25.345 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-10-08T00:10:25.369 INFO:tasks.ceph_client:Setting up client nodes... 2023-10-08T00:10:25.370 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-08T00:10:25.407 INFO:teuthology.orchestra.run.smithi099.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-10-08T00:10:25.420 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-10-08T00:10:25.497 INFO:teuthology.orchestra.run.smithi099.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-10-08T00:10:25.509 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-10-08T00:10:25.547 INFO:teuthology.orchestra.run.smithi191.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-10-08T00:10:25.560 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-10-08T00:10:25.637 INFO:teuthology.orchestra.run.smithi191.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-10-08T00:10:25.650 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-10-08T00:10:25.650 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi099.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi191.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-08T00:10:25.651 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-10-08T00:10:25.674 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-08T00:10:25.675 INFO:tasks.ceph:role: osd.0 2023-10-08T00:10:25.675 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi099.front.sepia.ceph.com 2023-10-08T00:10:25.676 DEBUG:teuthology.orchestra.run.smithi099:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-08T00:10:25.746 INFO:teuthology.orchestra.run.smithi099.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-08T00:10:25.747 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-08T00:10:25.747 INFO:teuthology.orchestra.run.smithi099.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-08T00:10:25.747 INFO:teuthology.orchestra.run.smithi099.stdout: = reflink=1 2023-10-08T00:10:25.747 INFO:teuthology.orchestra.run.smithi099.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-08T00:10:25.748 INFO:teuthology.orchestra.run.smithi099.stdout: = sunit=0 swidth=0 blks 2023-10-08T00:10:25.748 INFO:teuthology.orchestra.run.smithi099.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-08T00:10:25.748 INFO:teuthology.orchestra.run.smithi099.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-08T00:10:25.748 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-08T00:10:25.748 INFO:teuthology.orchestra.run.smithi099.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-08T00:10:26.175 INFO:teuthology.orchestra.run.smithi099.stdout:Discarding blocks...Done. 2023-10-08T00:10:26.177 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi099.front.sepia.ceph.com -o noatime 2023-10-08T00:10:26.177 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-10-08T00:10:26.362 DEBUG:teuthology.orchestra.run.smithi099:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-10-08T00:10:26.392 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-10-08T00:10:26.457 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-08T00:10:26.457 INFO:tasks.ceph:role: osd.1 2023-10-08T00:10:26.457 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi099.front.sepia.ceph.com 2023-10-08T00:10:26.457 DEBUG:teuthology.orchestra.run.smithi099:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-08T00:10:26.529 INFO:teuthology.orchestra.run.smithi099.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-08T00:10:26.529 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-08T00:10:26.529 INFO:teuthology.orchestra.run.smithi099.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-08T00:10:26.529 INFO:teuthology.orchestra.run.smithi099.stdout: = reflink=1 2023-10-08T00:10:26.530 INFO:teuthology.orchestra.run.smithi099.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-08T00:10:26.530 INFO:teuthology.orchestra.run.smithi099.stdout: = sunit=0 swidth=0 blks 2023-10-08T00:10:26.530 INFO:teuthology.orchestra.run.smithi099.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-08T00:10:26.530 INFO:teuthology.orchestra.run.smithi099.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-08T00:10:26.530 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-08T00:10:26.530 INFO:teuthology.orchestra.run.smithi099.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-08T00:10:26.952 INFO:teuthology.orchestra.run.smithi099.stdout:Discarding blocks...Done. 2023-10-08T00:10:26.953 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi099.front.sepia.ceph.com -o noatime 2023-10-08T00:10:26.953 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-10-08T00:10:27.056 DEBUG:teuthology.orchestra.run.smithi099:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-10-08T00:10:27.084 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-10-08T00:10:27.148 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-08T00:10:27.149 INFO:tasks.ceph:role: osd.2 2023-10-08T00:10:27.149 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi099.front.sepia.ceph.com 2023-10-08T00:10:27.149 DEBUG:teuthology.orchestra.run.smithi099:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-08T00:10:27.220 INFO:teuthology.orchestra.run.smithi099.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-08T00:10:27.220 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-08T00:10:27.220 INFO:teuthology.orchestra.run.smithi099.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-08T00:10:27.220 INFO:teuthology.orchestra.run.smithi099.stdout: = reflink=1 2023-10-08T00:10:27.220 INFO:teuthology.orchestra.run.smithi099.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-08T00:10:27.220 INFO:teuthology.orchestra.run.smithi099.stdout: = sunit=0 swidth=0 blks 2023-10-08T00:10:27.220 INFO:teuthology.orchestra.run.smithi099.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-08T00:10:27.220 INFO:teuthology.orchestra.run.smithi099.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-08T00:10:27.220 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-08T00:10:27.220 INFO:teuthology.orchestra.run.smithi099.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-08T00:10:27.672 INFO:teuthology.orchestra.run.smithi099.stdout:Discarding blocks...Done. 2023-10-08T00:10:27.673 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi099.front.sepia.ceph.com -o noatime 2023-10-08T00:10:27.674 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-10-08T00:10:27.845 DEBUG:teuthology.orchestra.run.smithi099:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-10-08T00:10:27.873 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-10-08T00:10:27.938 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-08T00:10:27.938 INFO:tasks.ceph:role: osd.3 2023-10-08T00:10:27.938 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi099.front.sepia.ceph.com 2023-10-08T00:10:27.938 DEBUG:teuthology.orchestra.run.smithi099:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-08T00:10:28.010 INFO:teuthology.orchestra.run.smithi099.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-08T00:10:28.010 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-08T00:10:28.010 INFO:teuthology.orchestra.run.smithi099.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-08T00:10:28.010 INFO:teuthology.orchestra.run.smithi099.stdout: = reflink=1 2023-10-08T00:10:28.010 INFO:teuthology.orchestra.run.smithi099.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-08T00:10:28.010 INFO:teuthology.orchestra.run.smithi099.stdout: = sunit=0 swidth=0 blks 2023-10-08T00:10:28.010 INFO:teuthology.orchestra.run.smithi099.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-08T00:10:28.010 INFO:teuthology.orchestra.run.smithi099.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-08T00:10:28.010 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-08T00:10:28.011 INFO:teuthology.orchestra.run.smithi099.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-08T00:10:28.441 INFO:teuthology.orchestra.run.smithi099.stdout:Discarding blocks...Done. 2023-10-08T00:10:28.443 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi099.front.sepia.ceph.com -o noatime 2023-10-08T00:10:28.443 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-10-08T00:10:28.616 DEBUG:teuthology.orchestra.run.smithi099:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-10-08T00:10:28.647 DEBUG:teuthology.orchestra.run.smithi099:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-08T00:10:28.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:28.735+0000 7f8296645540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-10-08T00:10:28.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:28.735+0000 7f8296645540 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-10-08T00:10:28.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:28.736+0000 7f8296645540 -1 memstore(/var/lib/ceph/osd/ceph-0) /var/lib/ceph/osd/ceph-0 2023-10-08T00:10:28.746 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-08T00:10:28.770 DEBUG:teuthology.orchestra.run.smithi099:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-08T00:10:28.850 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:28.849+0000 7fa48ac83540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-10-08T00:10:28.850 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:28.850+0000 7fa48ac83540 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-10-08T00:10:28.851 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:28.850+0000 7fa48ac83540 -1 memstore(/var/lib/ceph/osd/ceph-1) /var/lib/ceph/osd/ceph-1 2023-10-08T00:10:28.857 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-10-08T00:10:28.922 DEBUG:teuthology.orchestra.run.smithi099:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-08T00:10:29.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:29.002+0000 7f9303d6e540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-10-08T00:10:29.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:29.002+0000 7f9303d6e540 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-10-08T00:10:29.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:29.002+0000 7f9303d6e540 -1 memstore(/var/lib/ceph/osd/ceph-2) /var/lib/ceph/osd/ceph-2 2023-10-08T00:10:29.011 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-10-08T00:10:29.077 DEBUG:teuthology.orchestra.run.smithi099:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-08T00:10:29.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:29.156+0000 7f377b558540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-10-08T00:10:29.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:29.156+0000 7f377b558540 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-10-08T00:10:29.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:29.157+0000 7f377b558540 -1 memstore(/var/lib/ceph/osd/ceph-3) /var/lib/ceph/osd/ceph-3 2023-10-08T00:10:29.163 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-10-08T00:10:29.227 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-10-08T00:10:29.258 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-08T00:10:29.258 INFO:tasks.ceph:role: osd.4 2023-10-08T00:10:29.258 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi191.front.sepia.ceph.com 2023-10-08T00:10:29.258 DEBUG:teuthology.orchestra.run.smithi191:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-08T00:10:29.332 INFO:teuthology.orchestra.run.smithi191.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-08T00:10:29.332 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-08T00:10:29.333 INFO:teuthology.orchestra.run.smithi191.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-08T00:10:29.333 INFO:teuthology.orchestra.run.smithi191.stdout: = reflink=1 2023-10-08T00:10:29.333 INFO:teuthology.orchestra.run.smithi191.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-08T00:10:29.333 INFO:teuthology.orchestra.run.smithi191.stdout: = sunit=0 swidth=0 blks 2023-10-08T00:10:29.333 INFO:teuthology.orchestra.run.smithi191.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-08T00:10:29.333 INFO:teuthology.orchestra.run.smithi191.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-08T00:10:29.333 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-08T00:10:29.333 INFO:teuthology.orchestra.run.smithi191.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-08T00:10:29.767 INFO:teuthology.orchestra.run.smithi191.stdout:Discarding blocks...Done. 2023-10-08T00:10:29.769 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi191.front.sepia.ceph.com -o noatime 2023-10-08T00:10:29.770 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-10-08T00:10:29.941 DEBUG:teuthology.orchestra.run.smithi191:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-10-08T00:10:29.975 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-10-08T00:10:30.041 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-08T00:10:30.041 INFO:tasks.ceph:role: osd.5 2023-10-08T00:10:30.041 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi191.front.sepia.ceph.com 2023-10-08T00:10:30.041 DEBUG:teuthology.orchestra.run.smithi191:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-08T00:10:30.114 INFO:teuthology.orchestra.run.smithi191.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-08T00:10:30.114 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-08T00:10:30.114 INFO:teuthology.orchestra.run.smithi191.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-08T00:10:30.114 INFO:teuthology.orchestra.run.smithi191.stdout: = reflink=1 2023-10-08T00:10:30.114 INFO:teuthology.orchestra.run.smithi191.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-08T00:10:30.114 INFO:teuthology.orchestra.run.smithi191.stdout: = sunit=0 swidth=0 blks 2023-10-08T00:10:30.114 INFO:teuthology.orchestra.run.smithi191.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-08T00:10:30.114 INFO:teuthology.orchestra.run.smithi191.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-08T00:10:30.114 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-08T00:10:30.115 INFO:teuthology.orchestra.run.smithi191.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-08T00:10:30.534 INFO:teuthology.orchestra.run.smithi191.stdout:Discarding blocks...Done. 2023-10-08T00:10:30.536 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi191.front.sepia.ceph.com -o noatime 2023-10-08T00:10:30.536 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-10-08T00:10:30.660 DEBUG:teuthology.orchestra.run.smithi191:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-10-08T00:10:30.848 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-10-08T00:10:30.913 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-08T00:10:30.914 INFO:tasks.ceph:role: osd.6 2023-10-08T00:10:30.914 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi191.front.sepia.ceph.com 2023-10-08T00:10:30.914 DEBUG:teuthology.orchestra.run.smithi191:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-08T00:10:30.986 INFO:teuthology.orchestra.run.smithi191.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-08T00:10:30.986 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-08T00:10:30.986 INFO:teuthology.orchestra.run.smithi191.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-08T00:10:30.986 INFO:teuthology.orchestra.run.smithi191.stdout: = reflink=1 2023-10-08T00:10:30.986 INFO:teuthology.orchestra.run.smithi191.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-08T00:10:30.986 INFO:teuthology.orchestra.run.smithi191.stdout: = sunit=0 swidth=0 blks 2023-10-08T00:10:30.986 INFO:teuthology.orchestra.run.smithi191.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-08T00:10:30.987 INFO:teuthology.orchestra.run.smithi191.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-08T00:10:30.987 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-08T00:10:30.987 INFO:teuthology.orchestra.run.smithi191.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-08T00:10:31.441 INFO:teuthology.orchestra.run.smithi191.stdout:Discarding blocks...Done. 2023-10-08T00:10:31.443 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi191.front.sepia.ceph.com -o noatime 2023-10-08T00:10:31.443 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-10-08T00:10:31.610 DEBUG:teuthology.orchestra.run.smithi191:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-10-08T00:10:31.641 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-10-08T00:10:31.708 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-08T00:10:31.708 INFO:tasks.ceph:role: osd.7 2023-10-08T00:10:31.708 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi191.front.sepia.ceph.com 2023-10-08T00:10:31.708 DEBUG:teuthology.orchestra.run.smithi191:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-08T00:10:31.782 INFO:teuthology.orchestra.run.smithi191.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-08T00:10:31.782 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-08T00:10:31.782 INFO:teuthology.orchestra.run.smithi191.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-08T00:10:31.782 INFO:teuthology.orchestra.run.smithi191.stdout: = reflink=1 2023-10-08T00:10:31.783 INFO:teuthology.orchestra.run.smithi191.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-08T00:10:31.783 INFO:teuthology.orchestra.run.smithi191.stdout: = sunit=0 swidth=0 blks 2023-10-08T00:10:31.783 INFO:teuthology.orchestra.run.smithi191.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-08T00:10:31.783 INFO:teuthology.orchestra.run.smithi191.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-08T00:10:31.783 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-08T00:10:31.783 INFO:teuthology.orchestra.run.smithi191.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-08T00:10:32.216 INFO:teuthology.orchestra.run.smithi191.stdout:Discarding blocks...Done. 2023-10-08T00:10:32.218 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi191.front.sepia.ceph.com -o noatime 2023-10-08T00:10:32.218 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-10-08T00:10:32.315 DEBUG:teuthology.orchestra.run.smithi191:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-10-08T00:10:32.347 DEBUG:teuthology.orchestra.run.smithi191:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-08T00:10:32.441 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:32.441+0000 7f9fc4126540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-10-08T00:10:32.441 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:32.441+0000 7f9fc4126540 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-10-08T00:10:32.442 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:32.442+0000 7f9fc4126540 -1 memstore(/var/lib/ceph/osd/ceph-4) /var/lib/ceph/osd/ceph-4 2023-10-08T00:10:32.451 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-10-08T00:10:32.476 DEBUG:teuthology.orchestra.run.smithi191:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-08T00:10:32.557 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:32.557+0000 7fbd37657540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-10-08T00:10:32.558 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:32.557+0000 7fbd37657540 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-10-08T00:10:32.558 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:32.558+0000 7fbd37657540 -1 memstore(/var/lib/ceph/osd/ceph-5) /var/lib/ceph/osd/ceph-5 2023-10-08T00:10:32.563 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-10-08T00:10:32.628 DEBUG:teuthology.orchestra.run.smithi191:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-08T00:10:32.708 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:32.707+0000 7f185aa2a540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-10-08T00:10:32.708 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:32.708+0000 7f185aa2a540 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-10-08T00:10:32.708 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:32.708+0000 7f185aa2a540 -1 memstore(/var/lib/ceph/osd/ceph-6) /var/lib/ceph/osd/ceph-6 2023-10-08T00:10:32.717 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-10-08T00:10:32.781 DEBUG:teuthology.orchestra.run.smithi191:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-08T00:10:32.860 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:32.860+0000 7f60114d4540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-10-08T00:10:32.861 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:32.861+0000 7f60114d4540 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-10-08T00:10:32.861 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:32.861+0000 7f60114d4540 -1 memstore(/var/lib/ceph/osd/ceph-7) /var/lib/ceph/osd/ceph-7 2023-10-08T00:10:32.867 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-10-08T00:10:32.931 INFO:tasks.ceph:Reading keys from all nodes... 2023-10-08T00:10:32.931 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-08T00:10:32.931 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-10-08T00:10:32.959 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-08T00:10:32.959 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-10-08T00:10:33.026 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-08T00:10:33.026 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-10-08T00:10:33.091 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-08T00:10:33.091 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-10-08T00:10:33.155 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-08T00:10:33.155 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-10-08T00:10:33.220 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-08T00:10:33.220 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-10-08T00:10:33.283 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-08T00:10:33.284 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-10-08T00:10:33.347 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2023-10-08T00:10:33.349 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-10-08T00:10:33.373 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2023-10-08T00:10:33.374 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-10-08T00:10:33.438 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2023-10-08T00:10:33.438 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-10-08T00:10:33.502 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2023-10-08T00:10:33.502 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-10-08T00:10:33.567 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2023-10-08T00:10:33.567 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-10-08T00:10:33.631 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2023-10-08T00:10:33.631 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-10-08T00:10:33.695 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2023-10-08T00:10:33.695 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-10-08T00:10:33.761 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-08T00:10:33.761 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-10-08T00:10:33.777 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-08T00:10:33.777 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-10-08T00:10:33.833 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2023-10-08T00:10:33.833 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-10-08T00:10:33.847 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2023-10-08T00:10:33.848 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-10-08T00:10:33.902 INFO:tasks.ceph:Adding keys to all mons... 2023-10-08T00:10:33.903 DEBUG:teuthology.orchestra.run.smithi099:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-08T00:10:33.906 DEBUG:teuthology.orchestra.run.smithi191:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-08T00:10:33.947 INFO:teuthology.orchestra.run.smithi099.stdout:[mgr.y] 2023-10-08T00:10:33.947 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBw8yFl6ht5NxAAFzXnLDjklCePD6hw/pSfFg== 2023-10-08T00:10:33.947 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.a] 2023-10-08T00:10:33.947 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBx8yFlFlN9AhAAKdbeH0XznSOuyUnReD6RsA== 2023-10-08T00:10:33.947 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.c] 2023-10-08T00:10:33.947 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBx8yFl/M1DCRAANrfHJGZO1s0KOyCdUzZWIA== 2023-10-08T00:10:33.947 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.0] 2023-10-08T00:10:33.947 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQB08yFla7rpKxAAWTOutLcrjDeR69JbPZMoLg== 2023-10-08T00:10:33.947 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.1] 2023-10-08T00:10:33.948 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQB08yFlw+y2MhAAAvLDIB/tb49IlaireTqTFg== 2023-10-08T00:10:33.948 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.2] 2023-10-08T00:10:33.948 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQB18yFlh4wvABAALXOtJZoEfGmt/pOBxReooQ== 2023-10-08T00:10:33.948 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.3] 2023-10-08T00:10:33.948 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQB18yFl8cVfCRAAoGXTACMT+f5Avll5mxG9OA== 2023-10-08T00:10:33.948 INFO:teuthology.orchestra.run.smithi099.stdout:[mgr.x] 2023-10-08T00:10:33.948 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBw8yFl2kMLOxAAsEyuLBLsjQCUS0ERDOi75A== 2023-10-08T00:10:33.948 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.b] 2023-10-08T00:10:33.948 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBx8yFlh2G0DRAA1mYGrUrELHsS9abUxCCG2g== 2023-10-08T00:10:33.948 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.d] 2023-10-08T00:10:33.948 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBx8yFlr5F2FBAAYYeG6EM0/N0FiEAwSbYRyA== 2023-10-08T00:10:33.948 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.4] 2023-10-08T00:10:33.948 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQB48yFlft9WGhAA8JZN4MjSX9wbXiMKJnfCYg== 2023-10-08T00:10:33.949 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.5] 2023-10-08T00:10:33.949 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQB48yFlUf1BIRAA2FMBt1jAcm4ia/vYBDyojw== 2023-10-08T00:10:33.949 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.6] 2023-10-08T00:10:33.949 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQB48yFl0SI6KhAAI6G+Sqfm7OQmGf9nK59x9Q== 2023-10-08T00:10:33.949 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.7] 2023-10-08T00:10:33.949 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQB48yFlo5xWMxAA4p9OSLK0JpWhEExsifX+3A== 2023-10-08T00:10:33.949 INFO:teuthology.orchestra.run.smithi099.stdout:[client.0] 2023-10-08T00:10:33.949 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBx8yFlmzhQGBAAReslU/Pvcsr0dwE6l1N3/Q== 2023-10-08T00:10:33.949 INFO:teuthology.orchestra.run.smithi099.stdout:[client.2] 2023-10-08T00:10:33.949 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBx8yFlMp6rHRAArVgJfA3mY4n78hOMeVlw1Q== 2023-10-08T00:10:33.949 INFO:teuthology.orchestra.run.smithi099.stdout:[client.1] 2023-10-08T00:10:33.949 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBx8yFl2NKsIBAAUg6rXynl1J7qB6BkUizmpA== 2023-10-08T00:10:33.950 INFO:teuthology.orchestra.run.smithi099.stdout:[client.3] 2023-10-08T00:10:33.950 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBx8yFlH0gNJhAAEgGVGiabVIWknpWUohOnHA== 2023-10-08T00:10:33.965 INFO:teuthology.orchestra.run.smithi191.stdout:[mgr.y] 2023-10-08T00:10:33.965 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQBw8yFl6ht5NxAAFzXnLDjklCePD6hw/pSfFg== 2023-10-08T00:10:33.965 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.a] 2023-10-08T00:10:33.966 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQBx8yFlFlN9AhAAKdbeH0XznSOuyUnReD6RsA== 2023-10-08T00:10:33.966 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.c] 2023-10-08T00:10:33.966 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQBx8yFl/M1DCRAANrfHJGZO1s0KOyCdUzZWIA== 2023-10-08T00:10:33.966 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.0] 2023-10-08T00:10:33.966 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQB08yFla7rpKxAAWTOutLcrjDeR69JbPZMoLg== 2023-10-08T00:10:33.966 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.1] 2023-10-08T00:10:33.966 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQB08yFlw+y2MhAAAvLDIB/tb49IlaireTqTFg== 2023-10-08T00:10:33.966 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.2] 2023-10-08T00:10:33.966 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQB18yFlh4wvABAALXOtJZoEfGmt/pOBxReooQ== 2023-10-08T00:10:33.966 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.3] 2023-10-08T00:10:33.967 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQB18yFl8cVfCRAAoGXTACMT+f5Avll5mxG9OA== 2023-10-08T00:10:33.967 INFO:teuthology.orchestra.run.smithi191.stdout:[mgr.x] 2023-10-08T00:10:33.967 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQBw8yFl2kMLOxAAsEyuLBLsjQCUS0ERDOi75A== 2023-10-08T00:10:33.967 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.b] 2023-10-08T00:10:33.967 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQBx8yFlh2G0DRAA1mYGrUrELHsS9abUxCCG2g== 2023-10-08T00:10:33.967 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.d] 2023-10-08T00:10:33.967 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQBx8yFlr5F2FBAAYYeG6EM0/N0FiEAwSbYRyA== 2023-10-08T00:10:33.967 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.4] 2023-10-08T00:10:33.967 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQB48yFlft9WGhAA8JZN4MjSX9wbXiMKJnfCYg== 2023-10-08T00:10:33.967 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.5] 2023-10-08T00:10:33.967 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQB48yFlUf1BIRAA2FMBt1jAcm4ia/vYBDyojw== 2023-10-08T00:10:33.967 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.6] 2023-10-08T00:10:33.968 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQB48yFl0SI6KhAAI6G+Sqfm7OQmGf9nK59x9Q== 2023-10-08T00:10:33.968 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.7] 2023-10-08T00:10:33.968 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQB48yFlo5xWMxAA4p9OSLK0JpWhEExsifX+3A== 2023-10-08T00:10:33.968 INFO:teuthology.orchestra.run.smithi191.stdout:[client.0] 2023-10-08T00:10:33.968 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQBx8yFlmzhQGBAAReslU/Pvcsr0dwE6l1N3/Q== 2023-10-08T00:10:33.968 INFO:teuthology.orchestra.run.smithi191.stdout:[client.2] 2023-10-08T00:10:33.968 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQBx8yFlMp6rHRAArVgJfA3mY4n78hOMeVlw1Q== 2023-10-08T00:10:33.968 INFO:teuthology.orchestra.run.smithi191.stdout:[client.1] 2023-10-08T00:10:33.968 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQBx8yFl2NKsIBAAUg6rXynl1J7qB6BkUizmpA== 2023-10-08T00:10:33.968 INFO:teuthology.orchestra.run.smithi191.stdout:[client.3] 2023-10-08T00:10:33.968 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQBx8yFlH0gNJhAAEgGVGiabVIWknpWUohOnHA== 2023-10-08T00:10:33.969 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-08T00:10:33.993 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-08T00:10:34.125 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-08T00:10:34.127 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-08T00:10:34.206 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-08T00:10:34.209 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-08T00:10:34.449 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T00:10:34.452 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T00:10:34.519 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T00:10:34.521 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T00:10:34.608 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T00:10:34.610 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T00:10:34.677 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T00:10:34.679 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T00:10:34.735 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-08T00:10:34.738 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-08T00:10:34.813 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-08T00:10:34.815 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-08T00:10:34.967 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-08T00:10:34.970 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-08T00:10:35.020 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T00:10:35.024 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T00:10:35.071 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T00:10:35.073 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T00:10:35.122 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T00:10:35.125 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T00:10:35.172 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T00:10:35.175 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T00:10:35.222 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-08T00:10:35.224 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-08T00:10:35.273 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-08T00:10:35.275 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-08T00:10:35.325 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-08T00:10:35.328 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-08T00:10:35.378 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-08T00:10:35.380 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-08T00:10:35.430 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-10-08T00:10:35.430 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-10-08T00:10:35.454 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-08T00:10:35.544 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-10-08T00:10:35.569 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-10-08T00:10:35.593 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-08T00:10:35.685 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-10-08T00:10:35.709 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-10-08T00:10:35.773 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-08T00:10:35.869 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-10-08T00:10:35.894 DEBUG:teuthology.orchestra.run.smithi099:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-08T00:10:35.901 DEBUG:teuthology.orchestra.run.smithi191:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-08T00:10:35.949 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-10-08T00:10:35.949 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-08T00:10:35.950 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-10-08T00:10:35.966 INFO:tasks.ceph.mon.a:Started 2023-10-08T00:10:35.966 INFO:tasks.ceph.mon.b:Restarting daemon 2023-10-08T00:10:35.966 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-10-08T00:10:36.000 INFO:tasks.ceph.mon.b:Started 2023-10-08T00:10:36.000 INFO:tasks.ceph.mon.c:Restarting daemon 2023-10-08T00:10:36.000 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-10-08T00:10:36.002 INFO:tasks.ceph.mon.c:Started 2023-10-08T00:10:36.003 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-10-08T00:10:36.003 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-10-08T00:10:36.003 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-10-08T00:10:36.005 INFO:tasks.ceph.mgr.y:Started 2023-10-08T00:10:36.006 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-08T00:10:36.006 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-08T00:10:36.009 INFO:tasks.ceph.mgr.x:Started 2023-10-08T00:10:36.009 INFO:tasks.ceph:Setting crush tunables to default 2023-10-08T00:10:36.009 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph osd crush tunables default 2023-10-08T00:10:36.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.123+0000 7f47f250a700 1 Processor -- start 2023-10-08T00:10:36.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.123+0000 7f47f250a700 1 -- start start 2023-10-08T00:10:36.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.124+0000 7f47f250a700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f47ec06ca90 0x7f47ec164ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:36.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.124+0000 7f47f250a700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47ec067e30 con 0x7f47ec15a190 2023-10-08T00:10:36.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.124+0000 7f47f250a700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47ec06f8e0 con 0x7f47ec159dc0 2023-10-08T00:10:36.126 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.124+0000 7f47f250a700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47ec0cba70 con 0x7f47ec06ca90 2023-10-08T00:10:36.126 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.124+0000 7f47f0d07700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f47ec15a190 0x7f47ec06ec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:53862/0 (socket says 172.21.15.99:53862) 2023-10-08T00:10:36.126 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.124+0000 7f47f0d07700 1 -- 172.21.15.99:0/3027871588 learned_addr learned my addr 172.21.15.99:0/3027871588 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:10:36.126 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.124+0000 7f47f1d09700 1 --2- 172.21.15.99:0/3027871588 >> v2:172.21.15.191:3300/0 conn(0x7f47ec06ca90 0x7f47ec164ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:36.126 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.125+0000 7f47f1d09700 1 -- 172.21.15.99:0/3027871588 >> v2:172.21.15.191:3300/0 conn(0x7f47ec06ca90 msgr2=0x7f47ec164ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-08T00:10:36.126 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.125+0000 7f47f1d09700 1 -- 172.21.15.99:0/3027871588 >> v2:172.21.15.191:3300/0 conn(0x7f47ec06ca90 msgr2=0x7f47ec164ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-08T00:10:36.126 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.125+0000 7f47f1d09700 1 --2- 172.21.15.99:0/3027871588 >> v2:172.21.15.191:3300/0 conn(0x7f47ec06ca90 0x7f47ec164ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-08T00:10:36.126 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.125+0000 7f47f1d09700 1 --2- 172.21.15.99:0/3027871588 >> v2:172.21.15.191:3300/0 conn(0x7f47ec06ca90 0x7f47ec164ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T00:10:36.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.318+0000 7f47dffff700 1 -- 172.21.15.99:0/3027871588 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3362243365 0 0) 0x7f47ec067e30 con 0x7f47ec15a190 2023-10-08T00:10:36.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.318+0000 7f47dffff700 1 -- 172.21.15.99:0/3027871588 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47d8003570 con 0x7f47ec15a190 2023-10-08T00:10:36.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.319+0000 7f47dffff700 1 -- 172.21.15.99:0/3027871588 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1837698302 0 0) 0x7f47d8003570 con 0x7f47ec15a190 2023-10-08T00:10:36.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.319+0000 7f47dffff700 1 -- 172.21.15.99:0/3027871588 >> v2:172.21.15.191:3300/0 conn(0x7f47ec06ca90 msgr2=0x7f47ec164ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T00:10:36.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.319+0000 7f47dffff700 1 --2- 172.21.15.99:0/3027871588 >> v2:172.21.15.191:3300/0 conn(0x7f47ec06ca90 0x7f47ec164ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:36.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.319+0000 7f47dffff700 1 -- 172.21.15.99:0/3027871588 >> v1:172.21.15.191:6789/0 conn(0x7f47ec159dc0 legacy=0x7f47ec06c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:36.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.319+0000 7f47dffff700 1 -- 172.21.15.99:0/3027871588 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47ec0677e0 con 0x7f47ec15a190 2023-10-08T00:10:36.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.319+0000 7f47dffff700 1 -- 172.21.15.99:0/3027871588 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f47e8001470 con 0x7f47ec15a190 2023-10-08T00:10:36.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.319+0000 7f47dffff700 1 -- 172.21.15.99:0/3027871588 >> v1:172.21.15.99:6789/0 conn(0x7f47ec15a190 legacy=0x7f47ec06ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:36.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.319+0000 7f47dffff700 1 --2- 172.21.15.99:0/3027871588 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f47d8003a60 0x7f47d8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:36.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.319+0000 7f47dffff700 1 --2- 172.21.15.99:0/3027871588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47d8004810 0x7f47d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:36.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.319+0000 7f47dffff700 1 --2- 172.21.15.99:0/3027871588 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f47d80053c0 0x7f47d800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:36.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.319+0000 7f47dffff700 1 -- 172.21.15.99:0/3027871588 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47ec0cba70 con 0x7f47d8004810 2023-10-08T00:10:36.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.319+0000 7f47dffff700 1 -- 172.21.15.99:0/3027871588 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d800c2d0 con 0x7f47d80053c0 2023-10-08T00:10:36.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.319+0000 7f47dffff700 1 -- 172.21.15.99:0/3027871588 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47d800c490 con 0x7f47d8003a60 2023-10-08T00:10:36.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.319+0000 7f47f0d07700 1 --2- 172.21.15.99:0/3027871588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47d8004810 0x7f47d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:36.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.319+0000 7f47f1d09700 1 --2- 172.21.15.99:0/3027871588 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f47d80053c0 0x7f47d800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:36.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.319+0000 7f47f1508700 1 --2- 172.21.15.99:0/3027871588 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f47d8003a60 0x7f47d8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:36.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.319+0000 7f47f0d07700 1 -- 172.21.15.99:0/3027871588 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f47d8003a60 msgr2=0x7f47d8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:36.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.319+0000 7f47f0d07700 1 --2- 172.21.15.99:0/3027871588 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f47d8003a60 0x7f47d8003e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:36.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.319+0000 7f47f0d07700 1 -- 172.21.15.99:0/3027871588 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f47d80053c0 msgr2=0x7f47d800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:36.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.319+0000 7f47f0d07700 1 --2- 172.21.15.99:0/3027871588 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f47d80053c0 0x7f47d800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:36.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.319+0000 7f47f1d09700 1 -- 172.21.15.99:0/3027871588 reap_dead start 2023-10-08T00:10:36.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.319+0000 7f47f0d07700 1 -- 172.21.15.99:0/3027871588 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d8003570 con 0x7f47d8004810 2023-10-08T00:10:36.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.319+0000 7f47f0d07700 1 --2- 172.21.15.99:0/3027871588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47d8004810 0x7f47d8004c80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f47e800c3c0 tx=0x7f47e8001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aeb1faf392e64e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:36.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.320+0000 7f47dffff700 1 -- 172.21.15.99:0/3027871588 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e800cc90 con 0x7f47d8004810 2023-10-08T00:10:36.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.321+0000 7f47dffff700 1 -- 172.21.15.99:0/3027871588 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e800cdf0 con 0x7f47d8004810 2023-10-08T00:10:36.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.321+0000 7f47dffff700 1 -- 172.21.15.99:0/3027871588 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e8005b50 con 0x7f47d8004810 2023-10-08T00:10:36.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.321+0000 7f47f250a700 1 -- 172.21.15.99:0/3027871588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47d8004810 msgr2=0x7f47d8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:36.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.321+0000 7f47f250a700 1 --2- 172.21.15.99:0/3027871588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47d8004810 0x7f47d8004c80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f47e800c3c0 tx=0x7f47e8001e50 comp rx=0 tx=0).stop 2023-10-08T00:10:36.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.321+0000 7f47f250a700 1 -- 172.21.15.99:0/3027871588 shutdown_connections 2023-10-08T00:10:36.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.321+0000 7f47f250a700 1 --2- 172.21.15.99:0/3027871588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47d8004810 0x7f47d8004c80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:36.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.321+0000 7f47f250a700 1 -- 172.21.15.99:0/3027871588 >> 172.21.15.99:0/3027871588 conn(0x7f47ec0cb0a0 msgr2=0x7f47ec06bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:36.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.321+0000 7f47f250a700 1 -- 172.21.15.99:0/3027871588 shutdown_connections 2023-10-08T00:10:36.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.321+0000 7f47f250a700 1 -- 172.21.15.99:0/3027871588 wait complete. 2023-10-08T00:10:36.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.323+0000 7f47f250a700 1 Processor -- start 2023-10-08T00:10:36.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.323+0000 7f47f250a700 1 -- start start 2023-10-08T00:10:36.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.323+0000 7f47f250a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47d8004810 0x7f47ec0adf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:36.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.324+0000 7f47f250a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f47ec14dac0 0x7f47ec14df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:36.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.324+0000 7f47f250a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f47ec14b480 0x7f47ec14b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:36.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.324+0000 7f47f250a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47ec14be30 con 0x7f47d8004810 2023-10-08T00:10:36.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.324+0000 7f47f250a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47ec16eb90 con 0x7f47ec14dac0 2023-10-08T00:10:36.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.324+0000 7f47f250a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47ec16dfc0 con 0x7f47ec14b480 2023-10-08T00:10:36.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.324+0000 7f47f1508700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47d8004810 0x7f47ec0adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:36.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.324+0000 7f47f1508700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47d8004810 0x7f47ec0adf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49002/0 (socket says 172.21.15.99:49002) 2023-10-08T00:10:36.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.324+0000 7f47f1508700 1 -- 172.21.15.99:0/2657408779 learned_addr learned my addr 172.21.15.99:0/2657408779 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:10:36.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.324+0000 7f47f0d07700 1 --2- 172.21.15.99:0/2657408779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f47ec14dac0 0x7f47ec14df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:36.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.324+0000 7f47f1d09700 1 --2- 172.21.15.99:0/2657408779 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f47ec14b480 0x7f47ec14b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:36.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.324+0000 7f47f1508700 1 -- 172.21.15.99:0/2657408779 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f47ec14b480 msgr2=0x7f47ec14b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:36.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.324+0000 7f47f1508700 1 --2- 172.21.15.99:0/2657408779 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f47ec14b480 0x7f47ec14b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:36.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.324+0000 7f47f1508700 1 -- 172.21.15.99:0/2657408779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f47ec14dac0 msgr2=0x7f47ec14df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:36.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.324+0000 7f47f1508700 1 --2- 172.21.15.99:0/2657408779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f47ec14dac0 0x7f47ec14df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:36.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.324+0000 7f47f1508700 1 -- 172.21.15.99:0/2657408779 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47ec16e260 con 0x7f47d8004810 2023-10-08T00:10:36.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.324+0000 7f47f1508700 1 --2- 172.21.15.99:0/2657408779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47d8004810 0x7f47ec0adf50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f47e000d830 tx=0x7f47e0009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:36.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.324+0000 7f47dd7fa700 1 -- 172.21.15.99:0/2657408779 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e0006820 con 0x7f47d8004810 2023-10-08T00:10:36.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.324+0000 7f47dd7fa700 1 -- 172.21.15.99:0/2657408779 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e0006980 con 0x7f47d8004810 2023-10-08T00:10:36.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.324+0000 7f47f250a700 1 -- 172.21.15.99:0/2657408779 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47ec16e540 con 0x7f47d8004810 2023-10-08T00:10:36.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.324+0000 7f47dd7fa700 1 -- 172.21.15.99:0/2657408779 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e0006d40 con 0x7f47d8004810 2023-10-08T00:10:36.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.324+0000 7f47f250a700 1 -- 172.21.15.99:0/2657408779 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47ec0ae490 con 0x7f47d8004810 2023-10-08T00:10:36.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.324+0000 7f47dd7fa700 1 -- 172.21.15.99:0/2657408779 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f47e0007240 con 0x7f47d8004810 2023-10-08T00:10:36.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.324+0000 7f47dd7fa700 1 -- 172.21.15.99:0/2657408779 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f47e0012d80 con 0x7f47d8004810 2023-10-08T00:10:36.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.325+0000 7f47f250a700 1 -- 172.21.15.99:0/2657408779 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47ec16ee70 con 0x7f47d8004810 2023-10-08T00:10:36.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.327+0000 7f47dd7fa700 1 -- 172.21.15.99:0/2657408779 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f47e0001e60 con 0x7f47d8004810 2023-10-08T00:10:36.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.399+0000 7f47f250a700 1 -- 172.21.15.99:0/2657408779 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f47ec16e9e0 con 0x7f47d8004810 2023-10-08T00:10:36.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.402+0000 7f47dd7fa700 1 -- 172.21.15.99:0/2657408779 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f47e00076d0 con 0x7f47d8004810 2023-10-08T00:10:36.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.403+0000 7f47f250a700 1 -- 172.21.15.99:0/2657408779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47d8004810 msgr2=0x7f47ec0adf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:36.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.403+0000 7f47f250a700 1 --2- 172.21.15.99:0/2657408779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47d8004810 0x7f47ec0adf50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f47e000d830 tx=0x7f47e0009040 comp rx=0 tx=0).stop 2023-10-08T00:10:36.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.403+0000 7f47f250a700 1 -- 172.21.15.99:0/2657408779 shutdown_connections 2023-10-08T00:10:36.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.403+0000 7f47f250a700 1 --2- 172.21.15.99:0/2657408779 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f47ec14b480 0x7f47ec14b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:36.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.403+0000 7f47f250a700 1 --2- 172.21.15.99:0/2657408779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47d8004810 0x7f47ec0adf50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:36.406 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.403+0000 7f47f250a700 1 --2- 172.21.15.99:0/2657408779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f47ec14dac0 0x7f47ec14df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:36.406 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.403+0000 7f47f250a700 1 -- 172.21.15.99:0/2657408779 >> 172.21.15.99:0/2657408779 conn(0x7f47ec0cb0a0 msgr2=0x7f47ec154970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:36.406 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.403+0000 7f47f250a700 1 -- 172.21.15.99:0/2657408779 shutdown_connections 2023-10-08T00:10:36.406 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:36.403+0000 7f47f250a700 1 -- 172.21.15.99:0/2657408779 wait complete. 2023-10-08T00:10:36.406 INFO:teuthology.orchestra.run.smithi099.stderr:adjusted tunables profile to default 2023-10-08T00:10:36.415 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-10-08T00:10:36.416 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-08T00:10:36.416 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-10-08T00:10:36.440 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-08T00:10:36.441 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-10-08T00:10:36.505 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-08T00:10:36.505 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-10-08T00:10:36.570 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-08T00:10:36.570 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-10-08T00:10:36.635 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2023-10-08T00:10:36.636 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-10-08T00:10:36.660 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2023-10-08T00:10:36.660 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-10-08T00:10:36.725 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2023-10-08T00:10:36.726 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-10-08T00:10:36.790 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2023-10-08T00:10:36.790 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-10-08T00:10:36.854 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 0 2023-10-08T00:10:37.009 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.009+0000 7f29c291b700 1 Processor -- start 2023-10-08T00:10:37.009 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.009+0000 7f29c291b700 1 -- start start 2023-10-08T00:10:37.016 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.010+0000 7f29c291b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f29bc0cce30 0x7f29bc15aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:37.016 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.010+0000 7f29c291b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f29bc0b6580 0x7f29bc06a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:37.017 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.010+0000 7f29c291b700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29bc0b4d70 con 0x7f29bc06a830 2023-10-08T00:10:37.017 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.010+0000 7f29c291b700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29bc06ac70 con 0x7f29bc0cce30 2023-10-08T00:10:37.017 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.010+0000 7f29c291b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc0cd8d0 con 0x7f29bc0b6580 2023-10-08T00:10:37.017 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.010+0000 7f29c1118700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f29bc0b6580 0x7f29bc06a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:37.017 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.010+0000 7f29c1919700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f29bc0cce30 0x7f29bc15aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:37.017 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.010+0000 7f29c1118700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f29bc0b6580 0x7f29bc06a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.191:59380/0 (socket says 172.21.15.191:59380) 2023-10-08T00:10:37.018 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.010+0000 7f29c1118700 1 -- 172.21.15.191:0/80169726 learned_addr learned my addr 172.21.15.191:0/80169726 (peer_addr_for_me v2:172.21.15.191:0/0) 2023-10-08T00:10:37.018 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.010+0000 7f29c1118700 1 -- 172.21.15.191:0/80169726 >> v2:172.21.15.191:3300/0 conn(0x7f29bc0cce30 msgr2=0x7f29bc15aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:37.018 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.010+0000 7f29c1118700 1 --2- 172.21.15.191:0/80169726 >> v2:172.21.15.191:3300/0 conn(0x7f29bc0cce30 0x7f29bc15aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:37.018 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.010+0000 7f29c1118700 1 -- 172.21.15.191:0/80169726 >> v1:172.21.15.99:6789/0 conn(0x7f29bc06a830 legacy=0x7f29bc15d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:37.019 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.010+0000 7f29c1118700 1 -- 172.21.15.191:0/80169726 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29bc0b4440 con 0x7f29bc0b6580 2023-10-08T00:10:37.019 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.010+0000 7f29c1919700 1 --2- 172.21.15.191:0/80169726 >> v2:172.21.15.191:3300/0 conn(0x7f29bc0cce30 0x7f29bc15aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:10:37.019 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.010+0000 7f29c1118700 1 --2- 172.21.15.191:0/80169726 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f29bc0b6580 0x7f29bc06a2f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f29b80099b0 tx=0x7f29b8009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e60a62be4f8b8305 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:37.019 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.011+0000 7f29c0917700 1 -- 172.21.15.191:0/80169726 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b8009ea0 con 0x7f29bc0b6580 2023-10-08T00:10:37.019 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.011+0000 7f29c0917700 1 -- 172.21.15.191:0/80169726 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b800b860 con 0x7f29bc0b6580 2023-10-08T00:10:37.020 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.011+0000 7f29c0917700 1 -- 172.21.15.191:0/80169726 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b800bc00 con 0x7f29bc0b6580 2023-10-08T00:10:37.020 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.011+0000 7f29c291b700 1 -- 172.21.15.191:0/80169726 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f29bc0b6580 msgr2=0x7f29bc06a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:37.020 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.011+0000 7f29c291b700 1 --2- 172.21.15.191:0/80169726 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f29bc0b6580 0x7f29bc06a2f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f29b80099b0 tx=0x7f29b8009250 comp rx=0 tx=0).stop 2023-10-08T00:10:37.020 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.011+0000 7f29c291b700 1 -- 172.21.15.191:0/80169726 shutdown_connections 2023-10-08T00:10:37.020 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.011+0000 7f29c291b700 1 --2- 172.21.15.191:0/80169726 >> v2:172.21.15.191:3300/0 conn(0x7f29bc0cce30 0x7f29bc15aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:37.020 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.011+0000 7f29c291b700 1 --2- 172.21.15.191:0/80169726 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f29bc0b6580 0x7f29bc06a2f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:37.020 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.011+0000 7f29c291b700 1 -- 172.21.15.191:0/80169726 >> 172.21.15.191:0/80169726 conn(0x7f29bc068cb0 msgr2=0x7f29bc0690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:37.021 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.011+0000 7f29c291b700 1 -- 172.21.15.191:0/80169726 shutdown_connections 2023-10-08T00:10:37.021 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.011+0000 7f29c291b700 1 -- 172.21.15.191:0/80169726 wait complete. 2023-10-08T00:10:37.021 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.012+0000 7f29c291b700 1 Processor -- start 2023-10-08T00:10:37.021 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.012+0000 7f29c291b700 1 -- start start 2023-10-08T00:10:37.021 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.013+0000 7f29c291b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f29bc06a830 0x7f29bc1591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:37.021 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.013+0000 7f29c291b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29bc0b6580 0x7f29bc0b1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:37.021 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.013+0000 7f29c291b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29bc0cce30 0x7f29bc154890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:37.022 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.013+0000 7f29c291b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc0b2df0 con 0x7f29bc0b6580 2023-10-08T00:10:37.022 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.013+0000 7f29c291b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc0b2960 con 0x7f29bc0cce30 2023-10-08T00:10:37.022 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.013+0000 7f29c291b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc0b1d50 con 0x7f29bc06a830 2023-10-08T00:10:37.022 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.013+0000 7f29c1919700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f29bc06a830 0x7f29bc1591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:37.022 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.013+0000 7f29c211a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29bc0cce30 0x7f29bc154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:37.023 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.013+0000 7f29c1919700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f29bc06a830 0x7f29bc1591e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.191:59396/0 (socket says 172.21.15.191:59396) 2023-10-08T00:10:37.023 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.013+0000 7f29c1919700 1 -- 172.21.15.191:0/286274368 learned_addr learned my addr 172.21.15.191:0/286274368 (peer_addr_for_me v2:172.21.15.191:0/0) 2023-10-08T00:10:37.023 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.013+0000 7f29c1118700 1 --2- 172.21.15.191:0/286274368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29bc0b6580 0x7f29bc0b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:37.023 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.014+0000 7f29c1919700 1 -- 172.21.15.191:0/286274368 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29bc0cce30 msgr2=0x7f29bc154890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:37.023 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.014+0000 7f29c1919700 1 --2- 172.21.15.191:0/286274368 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29bc0cce30 0x7f29bc154890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:37.023 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.014+0000 7f29c1919700 1 -- 172.21.15.191:0/286274368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29bc0b6580 msgr2=0x7f29bc0b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:37.024 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.014+0000 7f29c1919700 1 --2- 172.21.15.191:0/286274368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29bc0b6580 0x7f29bc0b1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:37.024 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.014+0000 7f29c1919700 1 -- 172.21.15.191:0/286274368 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29ac009420 con 0x7f29bc06a830 2023-10-08T00:10:37.024 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.014+0000 7f29c1118700 1 --2- 172.21.15.191:0/286274368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29bc0b6580 0x7f29bc0b1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:10:37.024 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.014+0000 7f29c211a700 1 --2- 172.21.15.191:0/286274368 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29bc0cce30 0x7f29bc154890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:10:37.024 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.014+0000 7f29c1919700 1 --2- 172.21.15.191:0/286274368 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f29bc06a830 0x7f29bc1591e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f29ac00ec80 tx=0x7f29ac0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:37.025 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.014+0000 7f29b1ffb700 1 -- 172.21.15.191:0/286274368 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ac00c850 con 0x7f29bc06a830 2023-10-08T00:10:37.025 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.014+0000 7f29b1ffb700 1 -- 172.21.15.191:0/286274368 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29ac00c9b0 con 0x7f29bc06a830 2023-10-08T00:10:37.025 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.014+0000 7f29c291b700 1 -- 172.21.15.191:0/286274368 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29b80094f0 con 0x7f29bc06a830 2023-10-08T00:10:37.025 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.014+0000 7f29b1ffb700 1 -- 172.21.15.191:0/286274368 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ac00cd30 con 0x7f29bc06a830 2023-10-08T00:10:37.025 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.014+0000 7f29c291b700 1 -- 172.21.15.191:0/286274368 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29bc0b2390 con 0x7f29bc06a830 2023-10-08T00:10:37.025 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.014+0000 7f29b1ffb700 1 -- 172.21.15.191:0/286274368 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f29ac005c80 con 0x7f29bc06a830 2023-10-08T00:10:37.025 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.014+0000 7f29b1ffb700 1 -- 172.21.15.191:0/286274368 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7f29ac006020 con 0x7f29bc06a830 2023-10-08T00:10:37.026 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.015+0000 7f29c291b700 1 -- 172.21.15.191:0/286274368 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29bc0b2c40 con 0x7f29bc06a830 2023-10-08T00:10:37.026 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.017+0000 7f29b1ffb700 1 -- 172.21.15.191:0/286274368 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f29ac021060 con 0x7f29bc06a830 2023-10-08T00:10:37.119 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.119+0000 7f29c291b700 1 -- 172.21.15.191:0/286274368 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 0} v 0) v1 -- 0x7f29bc0cd200 con 0x7f29bc06a830 2023-10-08T00:10:37.122 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.122+0000 7f29b1ffb700 1 -- 172.21.15.191:0/286274368 <== mon.2 v2:172.21.15.191:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f29ac017070 con 0x7f29bc06a830 2023-10-08T00:10:37.123 INFO:teuthology.orchestra.run.smithi191.stdout:0 2023-10-08T00:10:37.124 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.123+0000 7f29c291b700 1 -- 172.21.15.191:0/286274368 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f29bc06a830 msgr2=0x7f29bc1591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:37.124 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.123+0000 7f29c291b700 1 --2- 172.21.15.191:0/286274368 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f29bc06a830 0x7f29bc1591e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f29ac00ec80 tx=0x7f29ac0091a0 comp rx=0 tx=0).stop 2023-10-08T00:10:37.124 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.123+0000 7f29c291b700 1 -- 172.21.15.191:0/286274368 shutdown_connections 2023-10-08T00:10:37.124 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.123+0000 7f29c291b700 1 --2- 172.21.15.191:0/286274368 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29bc0cce30 0x7f29bc154890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:37.124 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.123+0000 7f29c291b700 1 --2- 172.21.15.191:0/286274368 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f29bc06a830 0x7f29bc1591e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:37.124 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.123+0000 7f29c291b700 1 --2- 172.21.15.191:0/286274368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29bc0b6580 0x7f29bc0b1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:37.124 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.123+0000 7f29c291b700 1 -- 172.21.15.191:0/286274368 >> 172.21.15.191:0/286274368 conn(0x7f29bc068cb0 msgr2=0x7f29bc15b900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:37.124 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.123+0000 7f29c291b700 1 -- 172.21.15.191:0/286274368 shutdown_connections 2023-10-08T00:10:37.124 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.123+0000 7f29c291b700 1 -- 172.21.15.191:0/286274368 wait complete. 2023-10-08T00:10:37.135 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 1 2023-10-08T00:10:37.250 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.250+0000 7f943e5a9700 1 Processor -- start 2023-10-08T00:10:37.251 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.250+0000 7f943e5a9700 1 -- start start 2023-10-08T00:10:37.259 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.251+0000 7f943e5a9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f943815cec0 0x7f94380cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:37.259 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.251+0000 7f943e5a9700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94380cc760 con 0x7f943815ade0 2023-10-08T00:10:37.259 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.251+0000 7f943e5a9700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f943815b460 con 0x7f943815a300 2023-10-08T00:10:37.259 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.251+0000 7f943e5a9700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f943815ba20 con 0x7f943815cec0 2023-10-08T00:10:37.259 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.251+0000 7f943d5a7700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f943815a300 0x7f943815a6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.191:52232/0 (socket says 172.21.15.191:52232) 2023-10-08T00:10:37.259 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.251+0000 7f943cda6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f943815cec0 0x7f94380cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:37.260 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.251+0000 7f943d5a7700 1 -- 172.21.15.191:0/2281306363 learned_addr learned my addr 172.21.15.191:0/2281306363 (peer_addr_for_me v1:172.21.15.191:0/0) 2023-10-08T00:10:37.260 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.252+0000 7f942ffff700 1 -- 172.21.15.191:0/2281306363 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3455948666 0 0) 0x7f943815b460 con 0x7f943815a300 2023-10-08T00:10:37.260 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.252+0000 7f942ffff700 1 -- 172.21.15.191:0/2281306363 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9424003570 con 0x7f943815a300 2023-10-08T00:10:37.260 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.252+0000 7f943cda6700 1 -- 172.21.15.191:0/2281306363 >> v1:172.21.15.191:6789/0 conn(0x7f943815a300 legacy=0x7f943815a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:37.260 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.252+0000 7f943cda6700 1 -- 172.21.15.191:0/2281306363 >> v1:172.21.15.99:6789/0 conn(0x7f943815ade0 legacy=0x7f94380cb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:37.260 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.252+0000 7f942ffff700 1 -- 172.21.15.191:0/2281306363 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3103885518 0 0) 0x7f94380cc760 con 0x7f943815ade0 2023-10-08T00:10:37.261 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.252+0000 7f943cda6700 1 -- 172.21.15.191:0/2281306363 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94380cc1a0 con 0x7f943815cec0 2023-10-08T00:10:37.261 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.252+0000 7f943cda6700 1 --2- 172.21.15.191:0/2281306363 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f943815cec0 0x7f94380cb430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9434009910 tx=0x7f94340091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45501c128786e28b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:37.261 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.252+0000 7f942ffff700 1 -- 172.21.15.191:0/2281306363 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943400b680 con 0x7f943815cec0 2023-10-08T00:10:37.261 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.252+0000 7f942ffff700 1 -- 172.21.15.191:0/2281306363 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f943400b7e0 con 0x7f943815cec0 2023-10-08T00:10:37.261 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.252+0000 7f942ffff700 1 -- 172.21.15.191:0/2281306363 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943400bba0 con 0x7f943815cec0 2023-10-08T00:10:37.261 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.252+0000 7f943e5a9700 1 -- 172.21.15.191:0/2281306363 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f943815cec0 msgr2=0x7f94380cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:37.262 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.252+0000 7f943e5a9700 1 --2- 172.21.15.191:0/2281306363 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f943815cec0 0x7f94380cb430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9434009910 tx=0x7f94340091a0 comp rx=0 tx=0).stop 2023-10-08T00:10:37.262 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.253+0000 7f943e5a9700 1 -- 172.21.15.191:0/2281306363 shutdown_connections 2023-10-08T00:10:37.262 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.253+0000 7f943e5a9700 1 --2- 172.21.15.191:0/2281306363 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f943815cec0 0x7f94380cb430 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:37.262 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.253+0000 7f943e5a9700 1 -- 172.21.15.191:0/2281306363 >> 172.21.15.191:0/2281306363 conn(0x7f9438099f50 msgr2=0x7f9438067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:37.263 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.253+0000 7f943e5a9700 1 -- 172.21.15.191:0/2281306363 shutdown_connections 2023-10-08T00:10:37.263 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.253+0000 7f943e5a9700 1 -- 172.21.15.191:0/2281306363 wait complete. 2023-10-08T00:10:37.263 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.254+0000 7f943e5a9700 1 Processor -- start 2023-10-08T00:10:37.263 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.254+0000 7f943e5a9700 1 -- start start 2023-10-08T00:10:37.263 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.255+0000 7f943e5a9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f943815a300 0x7f943816d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:37.263 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.255+0000 7f943e5a9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f943815ade0 0x7f94380b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:37.263 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.255+0000 7f943e5a9700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f943815cec0 0x7f94380b2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:37.264 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.255+0000 7f943e5a9700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94380b65d0 con 0x7f943815a300 2023-10-08T00:10:37.264 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.255+0000 7f943e5a9700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94380b27c0 con 0x7f943815cec0 2023-10-08T00:10:37.265 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.255+0000 7f943e5a9700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94380c7e50 con 0x7f943815ade0 2023-10-08T00:10:37.265 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.255+0000 7f943cda6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f943815ade0 0x7f94380b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:37.265 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.255+0000 7f943cda6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f943815ade0 0x7f94380b1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.191:59406/0 (socket says 172.21.15.191:59406) 2023-10-08T00:10:37.266 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.255+0000 7f943dda8700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f943815cec0 0x7f94380b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:37.266 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.255+0000 7f943cda6700 1 -- 172.21.15.191:0/3998013093 learned_addr learned my addr 172.21.15.191:0/3998013093 (peer_addr_for_me v2:172.21.15.191:0/0) 2023-10-08T00:10:37.266 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.255+0000 7f943d5a7700 1 --2- 172.21.15.191:0/3998013093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f943815a300 0x7f943816d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:37.266 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.255+0000 7f943cda6700 1 -- 172.21.15.191:0/3998013093 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f943815cec0 msgr2=0x7f94380b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:37.266 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.255+0000 7f943cda6700 1 --2- 172.21.15.191:0/3998013093 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f943815cec0 0x7f94380b2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:37.266 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.255+0000 7f943cda6700 1 -- 172.21.15.191:0/3998013093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f943815a300 msgr2=0x7f943816d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:37.267 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.255+0000 7f943cda6700 1 --2- 172.21.15.191:0/3998013093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f943815a300 0x7f943816d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:37.267 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.255+0000 7f943cda6700 1 -- 172.21.15.191:0/3998013093 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9434009420 con 0x7f943815ade0 2023-10-08T00:10:37.267 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.255+0000 7f943cda6700 1 --2- 172.21.15.191:0/3998013093 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f943815ade0 0x7f94380b1d40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9438068640 tx=0x7f9434017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:37.267 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.255+0000 7f942d7fa700 1 -- 172.21.15.191:0/3998013093 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943400bb60 con 0x7f943815ade0 2023-10-08T00:10:37.267 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.255+0000 7f942d7fa700 1 -- 172.21.15.191:0/3998013093 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9434000c90 con 0x7f943815ade0 2023-10-08T00:10:37.267 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.255+0000 7f942d7fa700 1 -- 172.21.15.191:0/3998013093 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9434027680 con 0x7f943815ade0 2023-10-08T00:10:37.267 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.255+0000 7f943e5a9700 1 -- 172.21.15.191:0/3998013093 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94380c8130 con 0x7f943815ade0 2023-10-08T00:10:37.267 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.255+0000 7f943e5a9700 1 -- 172.21.15.191:0/3998013093 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94380c86f0 con 0x7f943815ade0 2023-10-08T00:10:37.267 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.257+0000 7f942d7fa700 1 -- 172.21.15.191:0/3998013093 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f94340279f0 con 0x7f943815ade0 2023-10-08T00:10:37.267 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.257+0000 7f943e5a9700 1 -- 172.21.15.191:0/3998013093 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94380c88e0 con 0x7f943815ade0 2023-10-08T00:10:37.268 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.257+0000 7f942d7fa700 1 -- 172.21.15.191:0/3998013093 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7f943401a9d0 con 0x7f943815ade0 2023-10-08T00:10:37.268 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.259+0000 7f942d7fa700 1 -- 172.21.15.191:0/3998013093 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f943402c020 con 0x7f943815ade0 2023-10-08T00:10:37.342 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.342+0000 7f943e5a9700 1 -- 172.21.15.191:0/3998013093 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 1} v 0) v1 -- 0x7f943806ada0 con 0x7f943815ade0 2023-10-08T00:10:37.344 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.345+0000 7f942d7fa700 1 -- 172.21.15.191:0/3998013093 <== mon.2 v2:172.21.15.191:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9434015020 con 0x7f943815ade0 2023-10-08T00:10:37.345 INFO:teuthology.orchestra.run.smithi191.stdout:1 2023-10-08T00:10:37.346 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.346+0000 7f943e5a9700 1 -- 172.21.15.191:0/3998013093 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f943815ade0 msgr2=0x7f94380b1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:37.346 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.346+0000 7f943e5a9700 1 --2- 172.21.15.191:0/3998013093 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f943815ade0 0x7f94380b1d40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9438068640 tx=0x7f9434017000 comp rx=0 tx=0).stop 2023-10-08T00:10:37.346 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.346+0000 7f943e5a9700 1 -- 172.21.15.191:0/3998013093 shutdown_connections 2023-10-08T00:10:37.346 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.346+0000 7f943e5a9700 1 --2- 172.21.15.191:0/3998013093 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f943815cec0 0x7f94380b2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:37.346 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.346+0000 7f943e5a9700 1 --2- 172.21.15.191:0/3998013093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f943815a300 0x7f943816d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:37.346 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.346+0000 7f943e5a9700 1 --2- 172.21.15.191:0/3998013093 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f943815ade0 0x7f94380b1d40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:37.346 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.346+0000 7f943e5a9700 1 -- 172.21.15.191:0/3998013093 >> 172.21.15.191:0/3998013093 conn(0x7f9438099f50 msgr2=0x7f9438150140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:37.346 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.346+0000 7f943e5a9700 1 -- 172.21.15.191:0/3998013093 shutdown_connections 2023-10-08T00:10:37.347 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.346+0000 7f943e5a9700 1 -- 172.21.15.191:0/3998013093 wait complete. 2023-10-08T00:10:37.358 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 2 2023-10-08T00:10:37.474 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.474+0000 7f877a43a700 1 Processor -- start 2023-10-08T00:10:37.475 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.474+0000 7f877a43a700 1 -- start start 2023-10-08T00:10:37.476 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.475+0000 7f877a43a700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8774159e60 0x7f877415e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:37.476 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.475+0000 7f877a43a700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f87740697c0 0x7f87740af730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:37.476 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.475+0000 7f877a43a700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f877415ea90 con 0x7f877415a230 2023-10-08T00:10:37.477 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.475+0000 7f877a43a700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87740d1cf0 con 0x7f8774159e60 2023-10-08T00:10:37.477 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.475+0000 7f877a43a700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87740b3100 con 0x7f87740697c0 2023-10-08T00:10:37.477 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.475+0000 7f87789d7700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f87740697c0 0x7f87740af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:37.477 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.475+0000 7f87737fe700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f877415a230 0x7f87740690b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.191:52234/0 (socket says 172.21.15.191:52234) 2023-10-08T00:10:37.477 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.475+0000 7f87737fe700 1 -- 172.21.15.191:0/1765499491 learned_addr learned my addr 172.21.15.191:0/1765499491 (peer_addr_for_me v1:172.21.15.191:0/0) 2023-10-08T00:10:37.477 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.475+0000 7f8773fff700 1 --2- 172.21.15.191:0/1765499491 >> v2:172.21.15.99:3300/0 conn(0x7f8774159e60 0x7f877415e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:37.477 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.476+0000 7f87789d7700 1 -- 172.21.15.191:0/1765499491 >> v2:172.21.15.99:3300/0 conn(0x7f8774159e60 msgr2=0x7f877415e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:37.477 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.476+0000 7f87789d7700 1 --2- 172.21.15.191:0/1765499491 >> v2:172.21.15.99:3300/0 conn(0x7f8774159e60 0x7f877415e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:37.477 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.476+0000 7f87789d7700 1 -- 172.21.15.191:0/1765499491 >> v1:172.21.15.191:6789/0 conn(0x7f877415a230 legacy=0x7f87740690b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:37.477 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.476+0000 7f87789d7700 1 -- 172.21.15.191:0/1765499491 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87740d10e0 con 0x7f87740697c0 2023-10-08T00:10:37.477 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.476+0000 7f8772ffd700 1 -- 172.21.15.191:0/1765499491 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2864809220 0 0) 0x7f877415ea90 con 0x7f877415a230 2023-10-08T00:10:37.477 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.476+0000 7f87789d7700 1 --2- 172.21.15.191:0/1765499491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87740697c0 0x7f87740af730 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f87740ce810 tx=0x7f876c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dea54d46f2572064 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:37.478 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.476+0000 7f8772ffd700 1 -- 172.21.15.191:0/1765499491 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876c00e070 con 0x7f87740697c0 2023-10-08T00:10:37.478 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.476+0000 7f8772ffd700 1 -- 172.21.15.191:0/1765499491 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f876c00bb60 con 0x7f87740697c0 2023-10-08T00:10:37.478 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.476+0000 7f8772ffd700 1 -- 172.21.15.191:0/1765499491 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876c012430 con 0x7f87740697c0 2023-10-08T00:10:37.481 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.477+0000 7f877a43a700 1 -- 172.21.15.191:0/1765499491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87740697c0 msgr2=0x7f87740af730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:37.481 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.477+0000 7f877a43a700 1 --2- 172.21.15.191:0/1765499491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87740697c0 0x7f87740af730 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f87740ce810 tx=0x7f876c009780 comp rx=0 tx=0).stop 2023-10-08T00:10:37.481 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.477+0000 7f877a43a700 1 -- 172.21.15.191:0/1765499491 shutdown_connections 2023-10-08T00:10:37.481 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.477+0000 7f877a43a700 1 --2- 172.21.15.191:0/1765499491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87740697c0 0x7f87740af730 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:37.481 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.477+0000 7f877a43a700 1 --2- 172.21.15.191:0/1765499491 >> v2:172.21.15.99:3300/0 conn(0x7f8774159e60 0x7f877415e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:37.481 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.477+0000 7f877a43a700 1 -- 172.21.15.191:0/1765499491 >> 172.21.15.191:0/1765499491 conn(0x7f8774070700 msgr2=0x7f8774070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:37.481 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.477+0000 7f877a43a700 1 -- 172.21.15.191:0/1765499491 shutdown_connections 2023-10-08T00:10:37.481 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.477+0000 7f877a43a700 1 -- 172.21.15.191:0/1765499491 wait complete. 2023-10-08T00:10:37.482 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.478+0000 7f877a43a700 1 Processor -- start 2023-10-08T00:10:37.482 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.478+0000 7f877a43a700 1 -- start start 2023-10-08T00:10:37.482 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.479+0000 7f877a43a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87740697c0 0x7f87740c8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:37.482 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.479+0000 7f877a43a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8774159e60 0x7f8774169700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:37.482 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.479+0000 7f877a43a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f877415a230 0x7f877415a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:37.482 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.479+0000 7f877a43a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f877415d440 con 0x7f8774159e60 2023-10-08T00:10:37.482 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.479+0000 7f877a43a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87740b44d0 con 0x7f87740697c0 2023-10-08T00:10:37.482 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.479+0000 7f877a43a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87740c90c0 con 0x7f877415a230 2023-10-08T00:10:37.482 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.479+0000 7f8773fff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87740697c0 0x7f87740c8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:37.482 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.479+0000 7f87789d7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f877415a230 0x7f877415a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:37.482 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.479+0000 7f87789d7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f877415a230 0x7f877415a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.191:59412/0 (socket says 172.21.15.191:59412) 2023-10-08T00:10:37.483 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.479+0000 7f8773fff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87740697c0 0x7f87740c8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:59362/0 (socket says 172.21.15.191:59362) 2023-10-08T00:10:37.483 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.479+0000 7f87789d7700 1 -- 172.21.15.191:0/4100304301 learned_addr learned my addr 172.21.15.191:0/4100304301 (peer_addr_for_me v2:172.21.15.191:0/0) 2023-10-08T00:10:37.483 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.479+0000 7f87737fe700 1 --2- 172.21.15.191:0/4100304301 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8774159e60 0x7f8774169700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:37.483 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.479+0000 7f8773fff700 1 -- 172.21.15.191:0/4100304301 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f877415a230 msgr2=0x7f877415a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:37.483 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.479+0000 7f8773fff700 1 --2- 172.21.15.191:0/4100304301 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f877415a230 0x7f877415a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:37.483 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.479+0000 7f8773fff700 1 -- 172.21.15.191:0/4100304301 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8774159e60 msgr2=0x7f8774169700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:37.483 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.479+0000 7f8773fff700 1 --2- 172.21.15.191:0/4100304301 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8774159e60 0x7f8774169700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:37.483 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.479+0000 7f8773fff700 1 -- 172.21.15.191:0/4100304301 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f876c009a20 con 0x7f87740697c0 2023-10-08T00:10:37.483 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.479+0000 7f87789d7700 1 --2- 172.21.15.191:0/4100304301 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f877415a230 0x7f877415a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:10:37.483 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.479+0000 7f87737fe700 1 --2- 172.21.15.191:0/4100304301 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8774159e60 0x7f8774169700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:10:37.483 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.480+0000 7f8773fff700 1 --2- 172.21.15.191:0/4100304301 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87740697c0 0x7f87740c8b80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f876800ea50 tx=0x7f87680091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:37.483 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.480+0000 7f875bfff700 1 -- 172.21.15.191:0/4100304301 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876800ee90 con 0x7f87740697c0 2023-10-08T00:10:37.484 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.480+0000 7f875bfff700 1 -- 172.21.15.191:0/4100304301 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f876800c7f0 con 0x7f87740697c0 2023-10-08T00:10:37.484 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.480+0000 7f877a43a700 1 -- 172.21.15.191:0/4100304301 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f877415cb10 con 0x7f87740697c0 2023-10-08T00:10:37.484 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.480+0000 7f875bfff700 1 -- 172.21.15.191:0/4100304301 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876800cb90 con 0x7f87740697c0 2023-10-08T00:10:37.484 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.480+0000 7f877a43a700 1 -- 172.21.15.191:0/4100304301 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f877415d0d0 con 0x7f87740697c0 2023-10-08T00:10:37.484 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.480+0000 7f875bfff700 1 -- 172.21.15.191:0/4100304301 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f8768005c80 con 0x7f87740697c0 2023-10-08T00:10:37.484 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.480+0000 7f875bfff700 1 -- 172.21.15.191:0/4100304301 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7f8768013470 con 0x7f87740697c0 2023-10-08T00:10:37.484 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.481+0000 7f877a43a700 1 -- 172.21.15.191:0/4100304301 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f877415d720 con 0x7f87740697c0 2023-10-08T00:10:37.484 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.483+0000 7f875bfff700 1 -- 172.21.15.191:0/4100304301 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f8768021060 con 0x7f87740697c0 2023-10-08T00:10:37.529 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T00:10:37.529+0000 7fa645af8180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T00:10:37.566 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.566+0000 7f877a43a700 1 -- 172.21.15.191:0/4100304301 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 2} v 0) v1 -- 0x7f877404c3c0 con 0x7f87740697c0 2023-10-08T00:10:37.569 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.569+0000 7f875bfff700 1 -- 172.21.15.191:0/4100304301 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8768017070 con 0x7f87740697c0 2023-10-08T00:10:37.569 INFO:teuthology.orchestra.run.smithi191.stdout:2 2023-10-08T00:10:37.571 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.570+0000 7f877a43a700 1 -- 172.21.15.191:0/4100304301 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87740697c0 msgr2=0x7f87740c8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:37.571 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.570+0000 7f877a43a700 1 --2- 172.21.15.191:0/4100304301 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87740697c0 0x7f87740c8b80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f876800ea50 tx=0x7f87680091a0 comp rx=0 tx=0).stop 2023-10-08T00:10:37.571 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.570+0000 7f877a43a700 1 -- 172.21.15.191:0/4100304301 shutdown_connections 2023-10-08T00:10:37.571 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.570+0000 7f877a43a700 1 --2- 172.21.15.191:0/4100304301 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87740697c0 0x7f87740c8b80 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:37.571 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.570+0000 7f877a43a700 1 --2- 172.21.15.191:0/4100304301 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f877415a230 0x7f877415a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:37.571 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.570+0000 7f877a43a700 1 --2- 172.21.15.191:0/4100304301 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8774159e60 0x7f8774169700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:37.571 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.570+0000 7f877a43a700 1 -- 172.21.15.191:0/4100304301 >> 172.21.15.191:0/4100304301 conn(0x7f8774070700 msgr2=0x7f87740cbff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:37.571 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.570+0000 7f877a43a700 1 -- 172.21.15.191:0/4100304301 shutdown_connections 2023-10-08T00:10:37.571 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.570+0000 7f877a43a700 1 -- 172.21.15.191:0/4100304301 wait complete. 2023-10-08T00:10:37.582 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 3 2023-10-08T00:10:37.667 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T00:10:37.667+0000 7fa645af8180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T00:10:37.697 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.697+0000 7f4e42b70700 1 Processor -- start 2023-10-08T00:10:37.697 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.697+0000 7f4e42b70700 1 -- start start 2023-10-08T00:10:37.700 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.698+0000 7f4e42b70700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4e3c0cce30 0x7f4e3c15aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:37.700 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.698+0000 7f4e42b70700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e3c0b6580 0x7f4e3c06a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:37.700 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.698+0000 7f4e42b70700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e3c0b4d70 con 0x7f4e3c06a830 2023-10-08T00:10:37.700 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.698+0000 7f4e42b70700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e3c06ac70 con 0x7f4e3c0cce30 2023-10-08T00:10:37.700 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.698+0000 7f4e42b70700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3c0cd8d0 con 0x7f4e3c0b6580 2023-10-08T00:10:37.700 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.698+0000 7f4e4136d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e3c0b6580 0x7f4e3c06a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:37.701 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.698+0000 7f4e41b6e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4e3c0cce30 0x7f4e3c15aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:37.701 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.698+0000 7f4e41b6e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4e3c0cce30 0x7f4e3c15aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:59364/0 (socket says 172.21.15.191:59364) 2023-10-08T00:10:37.701 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.698+0000 7f4e4136d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e3c0b6580 0x7f4e3c06a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.191:59428/0 (socket says 172.21.15.191:59428) 2023-10-08T00:10:37.701 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.698+0000 7f4e41b6e700 1 -- 172.21.15.191:0/1548039488 learned_addr learned my addr 172.21.15.191:0/1548039488 (peer_addr_for_me v2:172.21.15.191:0/0) 2023-10-08T00:10:37.701 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.698+0000 7f4e4236f700 1 --1- 172.21.15.191:0/1548039488 >> v1:172.21.15.191:6789/0 conn(0x7f4e3c06a830 0x7f4e3c15d880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.191:52240/0 (socket says 172.21.15.191:52240) 2023-10-08T00:10:37.701 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.698+0000 7f4e4136d700 1 -- 172.21.15.191:0/1548039488 >> v2:172.21.15.191:3300/0 conn(0x7f4e3c0cce30 msgr2=0x7f4e3c15aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:37.701 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.698+0000 7f4e4136d700 1 --2- 172.21.15.191:0/1548039488 >> v2:172.21.15.191:3300/0 conn(0x7f4e3c0cce30 0x7f4e3c15aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:37.701 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.698+0000 7f4e4136d700 1 -- 172.21.15.191:0/1548039488 >> v1:172.21.15.191:6789/0 conn(0x7f4e3c06a830 legacy=0x7f4e3c15d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:37.702 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.698+0000 7f4e4136d700 1 -- 172.21.15.191:0/1548039488 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e3c0b4440 con 0x7f4e3c0b6580 2023-10-08T00:10:37.702 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.698+0000 7f4e41b6e700 1 --2- 172.21.15.191:0/1548039488 >> v2:172.21.15.191:3300/0 conn(0x7f4e3c0cce30 0x7f4e3c15aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:10:37.702 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.699+0000 7f4e4136d700 1 --2- 172.21.15.191:0/1548039488 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e3c0b6580 0x7f4e3c06a2f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4e380098e0 tx=0x7f4e380091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57d8272880866be9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:37.702 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.699+0000 7f4e40b6c700 1 -- 172.21.15.191:0/1548039488 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3800b640 con 0x7f4e3c0b6580 2023-10-08T00:10:37.702 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.699+0000 7f4e40b6c700 1 -- 172.21.15.191:0/1548039488 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e3800b7a0 con 0x7f4e3c0b6580 2023-10-08T00:10:37.702 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.699+0000 7f4e40b6c700 1 -- 172.21.15.191:0/1548039488 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3800bb40 con 0x7f4e3c0b6580 2023-10-08T00:10:37.702 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.699+0000 7f4e42b70700 1 -- 172.21.15.191:0/1548039488 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e3c0b6580 msgr2=0x7f4e3c06a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:37.702 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.699+0000 7f4e42b70700 1 --2- 172.21.15.191:0/1548039488 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e3c0b6580 0x7f4e3c06a2f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4e380098e0 tx=0x7f4e380091a0 comp rx=0 tx=0).stop 2023-10-08T00:10:37.702 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.699+0000 7f4e42b70700 1 -- 172.21.15.191:0/1548039488 shutdown_connections 2023-10-08T00:10:37.702 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.699+0000 7f4e42b70700 1 --2- 172.21.15.191:0/1548039488 >> v2:172.21.15.191:3300/0 conn(0x7f4e3c0cce30 0x7f4e3c15aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:37.702 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.699+0000 7f4e42b70700 1 --2- 172.21.15.191:0/1548039488 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e3c0b6580 0x7f4e3c06a2f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:37.702 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.699+0000 7f4e42b70700 1 -- 172.21.15.191:0/1548039488 >> 172.21.15.191:0/1548039488 conn(0x7f4e3c068cb0 msgr2=0x7f4e3c0690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:37.703 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.699+0000 7f4e42b70700 1 -- 172.21.15.191:0/1548039488 shutdown_connections 2023-10-08T00:10:37.703 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.699+0000 7f4e42b70700 1 -- 172.21.15.191:0/1548039488 wait complete. 2023-10-08T00:10:37.705 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.700+0000 7f4e42b70700 1 Processor -- start 2023-10-08T00:10:37.705 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.700+0000 7f4e42b70700 1 -- start start 2023-10-08T00:10:37.705 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.701+0000 7f4e42b70700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e3c06a830 0x7f4e3c14e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:37.705 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.701+0000 7f4e42b70700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e3c0b6580 0x7f4e3c0b2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:37.705 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.701+0000 7f4e42b70700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e3c0cce30 0x7f4e3c0b3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:37.705 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.701+0000 7f4e42b70700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3c0b07d0 con 0x7f4e3c06a830 2023-10-08T00:10:37.706 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.701+0000 7f4e42b70700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3c14ed10 con 0x7f4e3c0cce30 2023-10-08T00:10:37.706 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.701+0000 7f4e42b70700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3c0b1890 con 0x7f4e3c0b6580 2023-10-08T00:10:37.706 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.701+0000 7f4e4136d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e3c0b6580 0x7f4e3c0b2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:37.706 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.701+0000 7f4e4236f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e3c0cce30 0x7f4e3c0b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:37.706 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.701+0000 7f4e4136d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e3c0b6580 0x7f4e3c0b2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.191:59434/0 (socket says 172.21.15.191:59434) 2023-10-08T00:10:37.706 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.701+0000 7f4e4136d700 1 -- 172.21.15.191:0/3413776659 learned_addr learned my addr 172.21.15.191:0/3413776659 (peer_addr_for_me v2:172.21.15.191:0/0) 2023-10-08T00:10:37.706 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.701+0000 7f4e41b6e700 1 --2- 172.21.15.191:0/3413776659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e3c06a830 0x7f4e3c14e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:37.706 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.702+0000 7f4e4136d700 1 -- 172.21.15.191:0/3413776659 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e3c0cce30 msgr2=0x7f4e3c0b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:37.706 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.702+0000 7f4e4136d700 1 --2- 172.21.15.191:0/3413776659 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e3c0cce30 0x7f4e3c0b3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:37.706 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.702+0000 7f4e4136d700 1 -- 172.21.15.191:0/3413776659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e3c06a830 msgr2=0x7f4e3c14e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:37.706 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.702+0000 7f4e4136d700 1 --2- 172.21.15.191:0/3413776659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e3c06a830 0x7f4e3c14e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:37.707 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.702+0000 7f4e4136d700 1 -- 172.21.15.191:0/3413776659 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e2c0094f0 con 0x7f4e3c0b6580 2023-10-08T00:10:37.707 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.702+0000 7f4e41b6e700 1 --2- 172.21.15.191:0/3413776659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e3c06a830 0x7f4e3c14e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:10:37.707 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.702+0000 7f4e4136d700 1 --2- 172.21.15.191:0/3413776659 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e3c0b6580 0x7f4e3c0b2ad0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4e38013000 tx=0x7f4e38017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:37.707 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.702+0000 7f4e31ffb700 1 -- 172.21.15.191:0/3413776659 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3801bc20 con 0x7f4e3c0b6580 2023-10-08T00:10:37.707 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.702+0000 7f4e31ffb700 1 -- 172.21.15.191:0/3413776659 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e3801bd80 con 0x7f4e3c0b6580 2023-10-08T00:10:37.707 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.702+0000 7f4e42b70700 1 -- 172.21.15.191:0/3413776659 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e38009420 con 0x7f4e3c0b6580 2023-10-08T00:10:37.707 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.702+0000 7f4e31ffb700 1 -- 172.21.15.191:0/3413776659 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e38011670 con 0x7f4e3c0b6580 2023-10-08T00:10:37.707 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.702+0000 7f4e42b70700 1 -- 172.21.15.191:0/3413776659 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e3c0b1e50 con 0x7f4e3c0b6580 2023-10-08T00:10:37.707 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.702+0000 7f4e31ffb700 1 -- 172.21.15.191:0/3413776659 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f4e38011ca0 con 0x7f4e3c0b6580 2023-10-08T00:10:37.707 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.702+0000 7f4e31ffb700 1 -- 172.21.15.191:0/3413776659 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7f4e38023890 con 0x7f4e3c0b6580 2023-10-08T00:10:37.707 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.703+0000 7f4e42b70700 1 -- 172.21.15.191:0/3413776659 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e3c089550 con 0x7f4e3c0b6580 2023-10-08T00:10:37.707 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.705+0000 7f4e31ffb700 1 -- 172.21.15.191:0/3413776659 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f4e38015020 con 0x7f4e3c0b6580 2023-10-08T00:10:37.747 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T00:10:37.747+0000 7fa645af8180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T00:10:37.789 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.789+0000 7f4e42b70700 1 -- 172.21.15.191:0/3413776659 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 3} v 0) v1 -- 0x7f4e3c0b4720 con 0x7f4e3c0b6580 2023-10-08T00:10:37.793 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.793+0000 7f4e31ffb700 1 -- 172.21.15.191:0/3413776659 <== mon.2 v2:172.21.15.191:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4e3802c440 con 0x7f4e3c0b6580 2023-10-08T00:10:37.793 INFO:teuthology.orchestra.run.smithi191.stdout:3 2023-10-08T00:10:37.794 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.794+0000 7f4e42b70700 1 -- 172.21.15.191:0/3413776659 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e3c0b6580 msgr2=0x7f4e3c0b2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:37.795 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.794+0000 7f4e42b70700 1 --2- 172.21.15.191:0/3413776659 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e3c0b6580 0x7f4e3c0b2ad0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4e38013000 tx=0x7f4e38017000 comp rx=0 tx=0).stop 2023-10-08T00:10:37.795 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.794+0000 7f4e42b70700 1 -- 172.21.15.191:0/3413776659 shutdown_connections 2023-10-08T00:10:37.795 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.794+0000 7f4e42b70700 1 --2- 172.21.15.191:0/3413776659 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e3c0cce30 0x7f4e3c0b3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:37.795 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.794+0000 7f4e42b70700 1 --2- 172.21.15.191:0/3413776659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e3c06a830 0x7f4e3c14e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:37.795 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.794+0000 7f4e42b70700 1 --2- 172.21.15.191:0/3413776659 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e3c0b6580 0x7f4e3c0b2ad0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:37.795 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.794+0000 7f4e42b70700 1 -- 172.21.15.191:0/3413776659 >> 172.21.15.191:0/3413776659 conn(0x7f4e3c068cb0 msgr2=0x7f4e3c0d1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:37.795 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.794+0000 7f4e42b70700 1 -- 172.21.15.191:0/3413776659 shutdown_connections 2023-10-08T00:10:37.795 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.794+0000 7f4e42b70700 1 -- 172.21.15.191:0/3413776659 wait complete. 2023-10-08T00:10:37.807 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 4 2023-10-08T00:10:37.858 INFO:tasks.ceph.mgr.y.smithi099.stderr:2023-10-08T00:10:37.858+0000 7f3119368180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T00:10:37.923 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.923+0000 7f4e67925700 1 Processor -- start 2023-10-08T00:10:37.923 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.923+0000 7f4e67925700 1 -- start start 2023-10-08T00:10:37.926 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.924+0000 7f4e67925700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4e60159ec0 0x7f4e600af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:37.926 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.924+0000 7f4e67925700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4e6015a290 0x7f4e600afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:37.926 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.924+0000 7f4e67925700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e6006a5a0 0x7f4e6006aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:37.926 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.924+0000 7f4e67925700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e6006af60 con 0x7f4e6015a290 2023-10-08T00:10:37.926 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.924+0000 7f4e67925700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e600b0790 con 0x7f4e60159ec0 2023-10-08T00:10:37.926 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.924+0000 7f4e67925700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e600ce1f0 con 0x7f4e6006a5a0 2023-10-08T00:10:37.926 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.924+0000 7f4e656c1700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4e60159ec0 0x7f4e600af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:37.926 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.924+0000 7f4e656c1700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4e60159ec0 0x7f4e600af940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:59384/0 (socket says 172.21.15.191:59384) 2023-10-08T00:10:37.926 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.924+0000 7f4e65ec2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e6006a5a0 0x7f4e6006aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:37.927 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.924+0000 7f4e656c1700 1 -- 172.21.15.191:0/959929573 learned_addr learned my addr 172.21.15.191:0/959929573 (peer_addr_for_me v2:172.21.15.191:0/0) 2023-10-08T00:10:37.927 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.924+0000 7f4e64ec0700 1 --2- 172.21.15.191:0/959929573 >> v2:172.21.15.99:3300/0 conn(0x7f4e6015a290 0x7f4e600afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:37.927 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.924+0000 7f4e656c1700 1 -- 172.21.15.191:0/959929573 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e6006a5a0 msgr2=0x7f4e6006aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:37.927 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.924+0000 7f4e656c1700 1 --2- 172.21.15.191:0/959929573 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e6006a5a0 0x7f4e6006aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:37.927 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.924+0000 7f4e656c1700 1 -- 172.21.15.191:0/959929573 >> v2:172.21.15.99:3300/0 conn(0x7f4e6015a290 msgr2=0x7f4e600afe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:37.927 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.924+0000 7f4e656c1700 1 --2- 172.21.15.191:0/959929573 >> v2:172.21.15.99:3300/0 conn(0x7f4e6015a290 0x7f4e600afe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:37.927 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.924+0000 7f4e656c1700 1 -- 172.21.15.191:0/959929573 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e600b0b20 con 0x7f4e60159ec0 2023-10-08T00:10:37.927 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.924+0000 7f4e656c1700 1 --2- 172.21.15.191:0/959929573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e60159ec0 0x7f4e600af940 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4e600898e0 tx=0x7f4e5c009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce060dbc26ca9029 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:37.927 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.924+0000 7f4e57fff700 1 -- 172.21.15.191:0/959929573 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c00b640 con 0x7f4e60159ec0 2023-10-08T00:10:37.928 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.924+0000 7f4e57fff700 1 -- 172.21.15.191:0/959929573 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e5c00b7a0 con 0x7f4e60159ec0 2023-10-08T00:10:37.928 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.924+0000 7f4e57fff700 1 -- 172.21.15.191:0/959929573 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c00bb60 con 0x7f4e60159ec0 2023-10-08T00:10:37.928 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.925+0000 7f4e67925700 1 -- 172.21.15.191:0/959929573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e60159ec0 msgr2=0x7f4e600af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:37.928 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.925+0000 7f4e67925700 1 --2- 172.21.15.191:0/959929573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e60159ec0 0x7f4e600af940 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4e600898e0 tx=0x7f4e5c009250 comp rx=0 tx=0).stop 2023-10-08T00:10:37.928 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.925+0000 7f4e67925700 1 -- 172.21.15.191:0/959929573 shutdown_connections 2023-10-08T00:10:37.928 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.925+0000 7f4e67925700 1 --2- 172.21.15.191:0/959929573 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e6006a5a0 0x7f4e6006aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:37.928 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.925+0000 7f4e67925700 1 --2- 172.21.15.191:0/959929573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e60159ec0 0x7f4e600af940 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:37.928 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.925+0000 7f4e67925700 1 --2- 172.21.15.191:0/959929573 >> v2:172.21.15.99:3300/0 conn(0x7f4e6015a290 0x7f4e600afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:37.928 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.925+0000 7f4e67925700 1 -- 172.21.15.191:0/959929573 >> 172.21.15.191:0/959929573 conn(0x7f4e600d1880 msgr2=0x7f4e600cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:37.928 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.925+0000 7f4e67925700 1 -- 172.21.15.191:0/959929573 shutdown_connections 2023-10-08T00:10:37.928 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.925+0000 7f4e67925700 1 -- 172.21.15.191:0/959929573 wait complete. 2023-10-08T00:10:37.931 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.926+0000 7f4e67925700 1 Processor -- start 2023-10-08T00:10:37.931 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.926+0000 7f4e67925700 1 -- start start 2023-10-08T00:10:37.931 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.927+0000 7f4e67925700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e6006a5a0 0x7f4e6015d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:37.931 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.927+0000 7f4e67925700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e60159ec0 0x7f4e6015a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:37.932 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.927+0000 7f4e67925700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e6015a290 0x7f4e6015c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:37.932 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.927+0000 7f4e67925700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6015aeb0 con 0x7f4e6006a5a0 2023-10-08T00:10:37.932 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.927+0000 7f4e67925700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e60158670 con 0x7f4e6015a290 2023-10-08T00:10:37.932 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.927+0000 7f4e67925700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e60158b00 con 0x7f4e60159ec0 2023-10-08T00:10:37.932 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.927+0000 7f4e64ec0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e60159ec0 0x7f4e6015a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:37.932 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.927+0000 7f4e65ec2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e6015a290 0x7f4e6015c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:37.932 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.927+0000 7f4e656c1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e6006a5a0 0x7f4e6015d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:37.932 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.927+0000 7f4e64ec0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e60159ec0 0x7f4e6015a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.191:59458/0 (socket says 172.21.15.191:59458) 2023-10-08T00:10:37.932 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.927+0000 7f4e64ec0700 1 -- 172.21.15.191:0/1626877218 learned_addr learned my addr 172.21.15.191:0/1626877218 (peer_addr_for_me v2:172.21.15.191:0/0) 2023-10-08T00:10:37.932 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.927+0000 7f4e65ec2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e6015a290 0x7f4e6015c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:59388/0 (socket says 172.21.15.191:59388) 2023-10-08T00:10:37.932 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.928+0000 7f4e656c1700 1 -- 172.21.15.191:0/1626877218 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e60159ec0 msgr2=0x7f4e6015a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:37.932 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.928+0000 7f4e656c1700 1 --2- 172.21.15.191:0/1626877218 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e60159ec0 0x7f4e6015a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:37.933 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.928+0000 7f4e656c1700 1 -- 172.21.15.191:0/1626877218 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e6015a290 msgr2=0x7f4e6015c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:37.933 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.928+0000 7f4e656c1700 1 --2- 172.21.15.191:0/1626877218 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e6015a290 0x7f4e6015c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:37.933 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.928+0000 7f4e656c1700 1 -- 172.21.15.191:0/1626877218 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e5c0094f0 con 0x7f4e6006a5a0 2023-10-08T00:10:37.933 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.928+0000 7f4e64ec0700 1 --2- 172.21.15.191:0/1626877218 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e60159ec0 0x7f4e6015a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:10:37.933 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.928+0000 7f4e65ec2700 1 --2- 172.21.15.191:0/1626877218 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e6015a290 0x7f4e6015c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:10:37.933 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.928+0000 7f4e656c1700 1 --2- 172.21.15.191:0/1626877218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e6006a5a0 0x7f4e6015d5a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4e60089910 tx=0x7f4e5c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:37.933 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.928+0000 7f4e557fa700 1 -- 172.21.15.191:0/1626877218 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c00b910 con 0x7f4e6006a5a0 2023-10-08T00:10:37.933 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.928+0000 7f4e557fa700 1 -- 172.21.15.191:0/1626877218 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e5c00ba70 con 0x7f4e6006a5a0 2023-10-08T00:10:37.933 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.928+0000 7f4e557fa700 1 -- 172.21.15.191:0/1626877218 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c011650 con 0x7f4e6006a5a0 2023-10-08T00:10:37.933 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.928+0000 7f4e67925700 1 -- 172.21.15.191:0/1626877218 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e60157d40 con 0x7f4e6006a5a0 2023-10-08T00:10:37.933 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.928+0000 7f4e67925700 1 -- 172.21.15.191:0/1626877218 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e60158300 con 0x7f4e6006a5a0 2023-10-08T00:10:37.934 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.928+0000 7f4e557fa700 1 -- 172.21.15.191:0/1626877218 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f4e5c011c80 con 0x7f4e6006a5a0 2023-10-08T00:10:37.934 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.928+0000 7f4e557fa700 1 -- 172.21.15.191:0/1626877218 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7f4e5c025900 con 0x7f4e6006a5a0 2023-10-08T00:10:37.934 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.929+0000 7f4e67925700 1 -- 172.21.15.191:0/1626877218 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e60158950 con 0x7f4e6006a5a0 2023-10-08T00:10:37.934 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:37.933+0000 7f4e557fa700 1 -- 172.21.15.191:0/1626877218 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f4e5c015020 con 0x7f4e6006a5a0 2023-10-08T00:10:37.942 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T00:10:37.942+0000 7fa645af8180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T00:10:37.992 INFO:tasks.ceph.mgr.y.smithi099.stderr:2023-10-08T00:10:37.991+0000 7f3119368180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T00:10:38.015 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.015+0000 7f4e67925700 1 -- 172.21.15.191:0/1626877218 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 4} v 0) v1 -- 0x7f4e6004c3c0 con 0x7f4e6006a5a0 2023-10-08T00:10:38.018 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.018+0000 7f4e557fa700 1 -- 172.21.15.191:0/1626877218 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4e5c0253f0 con 0x7f4e6006a5a0 2023-10-08T00:10:38.018 INFO:teuthology.orchestra.run.smithi191.stdout:4 2023-10-08T00:10:38.020 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.019+0000 7f4e67925700 1 -- 172.21.15.191:0/1626877218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e6006a5a0 msgr2=0x7f4e6015d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:38.020 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.019+0000 7f4e67925700 1 --2- 172.21.15.191:0/1626877218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e6006a5a0 0x7f4e6015d5a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4e60089910 tx=0x7f4e5c017000 comp rx=0 tx=0).stop 2023-10-08T00:10:38.020 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.019+0000 7f4e67925700 1 -- 172.21.15.191:0/1626877218 shutdown_connections 2023-10-08T00:10:38.020 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.019+0000 7f4e67925700 1 --2- 172.21.15.191:0/1626877218 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e6015a290 0x7f4e6015c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:38.020 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.019+0000 7f4e67925700 1 --2- 172.21.15.191:0/1626877218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e6006a5a0 0x7f4e6015d5a0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:38.020 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.019+0000 7f4e67925700 1 --2- 172.21.15.191:0/1626877218 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e60159ec0 0x7f4e6015a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:38.020 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.019+0000 7f4e67925700 1 -- 172.21.15.191:0/1626877218 >> 172.21.15.191:0/1626877218 conn(0x7f4e600d1880 msgr2=0x7f4e600b1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:38.020 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.020+0000 7f4e67925700 1 -- 172.21.15.191:0/1626877218 shutdown_connections 2023-10-08T00:10:38.021 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.020+0000 7f4e67925700 1 -- 172.21.15.191:0/1626877218 wait complete. 2023-10-08T00:10:38.031 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 5 2023-10-08T00:10:38.069 INFO:tasks.ceph.mgr.y.smithi099.stderr:2023-10-08T00:10:38.068+0000 7f3119368180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T00:10:38.123 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T00:10:38.123+0000 7fa645af8180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T00:10:38.147 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.147+0000 7fadc9328700 1 Processor -- start 2023-10-08T00:10:38.147 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.147+0000 7fadc9328700 1 -- start start 2023-10-08T00:10:38.157 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.148+0000 7fadc9328700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fadc40697c0 0x7fadc415e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:38.158 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.148+0000 7fadc9328700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fadc415a230 0x7fadc40af730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:38.158 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.148+0000 7fadc9328700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadc415ea90 con 0x7fadc4159e60 2023-10-08T00:10:38.158 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.148+0000 7fadc9328700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadc40d1cf0 con 0x7fadc415a230 2023-10-08T00:10:38.158 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.148+0000 7fadc9328700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadc40b3100 con 0x7fadc40697c0 2023-10-08T00:10:38.158 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.148+0000 7fadc27fc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fadc40697c0 0x7fadc415e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:38.158 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.148+0000 7fadc2ffd700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7fadc4159e60 0x7fadc40690b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.191:52252/0 (socket says 172.21.15.191:52252) 2023-10-08T00:10:38.159 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.148+0000 7fadc2ffd700 1 -- 172.21.15.191:0/1897001266 learned_addr learned my addr 172.21.15.191:0/1897001266 (peer_addr_for_me v1:172.21.15.191:0/0) 2023-10-08T00:10:38.159 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.148+0000 7fadc37fe700 1 --2- 172.21.15.191:0/1897001266 >> v2:172.21.15.99:3300/0 conn(0x7fadc415a230 0x7fadc40af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:38.159 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.148+0000 7fadc1ffb700 1 -- 172.21.15.191:0/1897001266 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3913541044 0 0) 0x7fadc415ea90 con 0x7fadc4159e60 2023-10-08T00:10:38.159 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.148+0000 7fadc1ffb700 1 -- 172.21.15.191:0/1897001266 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fadac003570 con 0x7fadc4159e60 2023-10-08T00:10:38.159 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.148+0000 7fadc27fc700 1 -- 172.21.15.191:0/1897001266 >> v2:172.21.15.99:3300/0 conn(0x7fadc415a230 msgr2=0x7fadc40af730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:38.159 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.148+0000 7fadc27fc700 1 --2- 172.21.15.191:0/1897001266 >> v2:172.21.15.99:3300/0 conn(0x7fadc415a230 0x7fadc40af730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:38.159 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.148+0000 7fadc27fc700 1 -- 172.21.15.191:0/1897001266 >> v1:172.21.15.191:6789/0 conn(0x7fadc4159e60 legacy=0x7fadc40690b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:38.160 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.148+0000 7fadc27fc700 1 -- 172.21.15.191:0/1897001266 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc40d10e0 con 0x7fadc40697c0 2023-10-08T00:10:38.160 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.148+0000 7fadc27fc700 1 --2- 172.21.15.191:0/1897001266 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fadc40697c0 0x7fadc415e550 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fadb80098e0 tx=0x7fadb80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55c0b52be66b4147 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:38.160 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.148+0000 7fadc1ffb700 1 -- 172.21.15.191:0/1897001266 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb8011630 con 0x7fadc40697c0 2023-10-08T00:10:38.160 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.148+0000 7fadc1ffb700 1 -- 172.21.15.191:0/1897001266 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb8011790 con 0x7fadc40697c0 2023-10-08T00:10:38.160 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.148+0000 7fadc1ffb700 1 -- 172.21.15.191:0/1897001266 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb8011b30 con 0x7fadc40697c0 2023-10-08T00:10:38.160 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.149+0000 7fadc9328700 1 -- 172.21.15.191:0/1897001266 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fadc40697c0 msgr2=0x7fadc415e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:38.161 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.149+0000 7fadc9328700 1 --2- 172.21.15.191:0/1897001266 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fadc40697c0 0x7fadc415e550 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fadb80098e0 tx=0x7fadb80091a0 comp rx=0 tx=0).stop 2023-10-08T00:10:38.161 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.149+0000 7fadc9328700 1 -- 172.21.15.191:0/1897001266 shutdown_connections 2023-10-08T00:10:38.161 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.149+0000 7fadc9328700 1 --2- 172.21.15.191:0/1897001266 >> v2:172.21.15.99:3300/0 conn(0x7fadc415a230 0x7fadc40af730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:38.161 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.149+0000 7fadc9328700 1 --2- 172.21.15.191:0/1897001266 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fadc40697c0 0x7fadc415e550 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:38.161 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.149+0000 7fadc9328700 1 -- 172.21.15.191:0/1897001266 >> 172.21.15.191:0/1897001266 conn(0x7fadc4070700 msgr2=0x7fadc4070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:38.161 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.149+0000 7fadc9328700 1 -- 172.21.15.191:0/1897001266 shutdown_connections 2023-10-08T00:10:38.162 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.149+0000 7fadc9328700 1 -- 172.21.15.191:0/1897001266 wait complete. 2023-10-08T00:10:38.162 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.150+0000 7fadc9328700 1 Processor -- start 2023-10-08T00:10:38.162 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.150+0000 7fadc9328700 1 -- start start 2023-10-08T00:10:38.162 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.151+0000 7fadc9328700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fadc40697c0 0x7fadc416a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:38.162 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.151+0000 7fadc9328700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fadc4159e60 0x7fadc41582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:38.162 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.151+0000 7fadc9328700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fadc415a230 0x7fadc40b0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:38.163 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.151+0000 7fadc9328700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc415c390 con 0x7fadc4159e60 2023-10-08T00:10:38.163 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.151+0000 7fadc9328700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc415bad0 con 0x7fadc40697c0 2023-10-08T00:10:38.163 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.151+0000 7fadc9328700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadc416a9f0 con 0x7fadc415a230 2023-10-08T00:10:38.163 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.151+0000 7fadc37fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fadc415a230 0x7fadc40b0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:38.163 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.151+0000 7fadc2ffd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fadc40697c0 0x7fadc416a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:38.163 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.151+0000 7fadc2ffd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fadc40697c0 0x7fadc416a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:59394/0 (socket says 172.21.15.191:59394) 2023-10-08T00:10:38.164 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.151+0000 7fadc37fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fadc415a230 0x7fadc40b0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.191:59478/0 (socket says 172.21.15.191:59478) 2023-10-08T00:10:38.164 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.151+0000 7fadc2ffd700 1 -- 172.21.15.191:0/524550187 learned_addr learned my addr 172.21.15.191:0/524550187 (peer_addr_for_me v2:172.21.15.191:0/0) 2023-10-08T00:10:38.164 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.151+0000 7fadc27fc700 1 --2- 172.21.15.191:0/524550187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fadc4159e60 0x7fadc41582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:38.164 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.151+0000 7fadc37fe700 1 -- 172.21.15.191:0/524550187 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fadc40697c0 msgr2=0x7fadc416a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:38.164 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.152+0000 7fadc37fe700 1 --2- 172.21.15.191:0/524550187 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fadc40697c0 0x7fadc416a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:38.164 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.152+0000 7fadc37fe700 1 -- 172.21.15.191:0/524550187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fadc4159e60 msgr2=0x7fadc41582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:38.165 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.152+0000 7fadc37fe700 1 --2- 172.21.15.191:0/524550187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fadc4159e60 0x7fadc41582b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:38.165 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.152+0000 7fadc37fe700 1 -- 172.21.15.191:0/524550187 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb8009420 con 0x7fadc415a230 2023-10-08T00:10:38.165 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.152+0000 7fadc2ffd700 1 --2- 172.21.15.191:0/524550187 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fadc40697c0 0x7fadc416a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:10:38.165 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.152+0000 7fadc37fe700 1 --2- 172.21.15.191:0/524550187 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fadc415a230 0x7fadc40b0a20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fadbc009e50 tx=0x7fadbc00b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:38.165 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.152+0000 7fadaaffd700 1 -- 172.21.15.191:0/524550187 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadbc00caa0 con 0x7fadc415a230 2023-10-08T00:10:38.165 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.152+0000 7fadaaffd700 1 -- 172.21.15.191:0/524550187 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadbc00f040 con 0x7fadc415a230 2023-10-08T00:10:38.166 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.152+0000 7fadc9328700 1 -- 172.21.15.191:0/524550187 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadc415bd40 con 0x7fadc415a230 2023-10-08T00:10:38.166 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.152+0000 7fadaaffd700 1 -- 172.21.15.191:0/524550187 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadbc00ce50 con 0x7fadc415a230 2023-10-08T00:10:38.166 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.152+0000 7fadc9328700 1 -- 172.21.15.191:0/524550187 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadc40c9590 con 0x7fadc415a230 2023-10-08T00:10:38.166 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.152+0000 7fadaaffd700 1 -- 172.21.15.191:0/524550187 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fadbc014690 con 0x7fadc415a230 2023-10-08T00:10:38.166 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.152+0000 7fadaaffd700 1 -- 172.21.15.191:0/524550187 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1296+0+0 (secure 0 0 0) 0x7fadbc01da20 con 0x7fadc415a230 2023-10-08T00:10:38.167 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.153+0000 7fadc9328700 1 -- 172.21.15.191:0/524550187 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadc40d16f0 con 0x7fadc415a230 2023-10-08T00:10:38.167 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.156+0000 7fadaaffd700 1 -- 172.21.15.191:0/524550187 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fadbc03b6d0 con 0x7fadc415a230 2023-10-08T00:10:38.205 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T00:10:38.205+0000 7fa645af8180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T00:10:38.239 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.239+0000 7fadc9328700 1 -- 172.21.15.191:0/524550187 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 5} v 0) v1 -- 0x7fadc404c3c0 con 0x7fadc415a230 2023-10-08T00:10:38.243 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.243+0000 7fadaaffd700 1 -- 172.21.15.191:0/524550187 <== mon.2 v2:172.21.15.191:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fadbc02a070 con 0x7fadc415a230 2023-10-08T00:10:38.244 INFO:teuthology.orchestra.run.smithi191.stdout:5 2023-10-08T00:10:38.245 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.244+0000 7fadc9328700 1 -- 172.21.15.191:0/524550187 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fadc415a230 msgr2=0x7fadc40b0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:38.245 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.244+0000 7fadc9328700 1 --2- 172.21.15.191:0/524550187 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fadc415a230 0x7fadc40b0a20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fadbc009e50 tx=0x7fadbc00b040 comp rx=0 tx=0).stop 2023-10-08T00:10:38.245 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.244+0000 7fadc9328700 1 -- 172.21.15.191:0/524550187 shutdown_connections 2023-10-08T00:10:38.245 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.244+0000 7fadc9328700 1 --2- 172.21.15.191:0/524550187 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fadc40697c0 0x7fadc416a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:38.245 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.244+0000 7fadc9328700 1 --2- 172.21.15.191:0/524550187 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fadc415a230 0x7fadc40b0a20 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:38.245 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.244+0000 7fadc9328700 1 --2- 172.21.15.191:0/524550187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fadc4159e60 0x7fadc41582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:38.246 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.244+0000 7fadc9328700 1 -- 172.21.15.191:0/524550187 >> 172.21.15.191:0/524550187 conn(0x7fadc4070700 msgr2=0x7fadc40cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:38.246 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.244+0000 7fadc9328700 1 -- 172.21.15.191:0/524550187 shutdown_connections 2023-10-08T00:10:38.246 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.244+0000 7fadc9328700 1 -- 172.21.15.191:0/524550187 wait complete. 2023-10-08T00:10:38.253 INFO:tasks.ceph.mgr.y.smithi099.stderr:2023-10-08T00:10:38.252+0000 7f3119368180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T00:10:38.257 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 6 2023-10-08T00:10:38.348 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T00:10:38.348+0000 7fa645af8180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T00:10:38.374 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.374+0000 7f0d64989700 1 Processor -- start 2023-10-08T00:10:38.374 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.374+0000 7f0d64989700 1 -- start start 2023-10-08T00:10:38.376 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.375+0000 7f0d64989700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f0d600b17b0 0x7f0d600aa870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:38.377 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.375+0000 7f0d64989700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d600b03e0 0x7f0d600b0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:38.377 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.375+0000 7f0d64989700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d6006a2b0 con 0x7f0d60159bb0 2023-10-08T00:10:38.377 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.375+0000 7f0d64989700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d600b0d60 con 0x7f0d600b17b0 2023-10-08T00:10:38.377 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.375+0000 7f0d64989700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d60069040 con 0x7f0d600b03e0 2023-10-08T00:10:38.377 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.375+0000 7f0d5e59c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f0d600b17b0 0x7f0d600aa870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:38.377 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.375+0000 7f0d5f59e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d600b03e0 0x7f0d600b0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:38.377 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.375+0000 7f0d5ed9d700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f0d60159bb0 0x7f0d6015e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.191:52258/0 (socket says 172.21.15.191:52258) 2023-10-08T00:10:38.377 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.375+0000 7f0d5ed9d700 1 -- 172.21.15.191:0/1991463602 learned_addr learned my addr 172.21.15.191:0/1991463602 (peer_addr_for_me v1:172.21.15.191:0/0) 2023-10-08T00:10:38.377 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.375+0000 7f0d5e59c700 1 -- 172.21.15.191:0/1991463602 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d600b03e0 msgr2=0x7f0d600b0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:38.377 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.375+0000 7f0d5e59c700 1 --2- 172.21.15.191:0/1991463602 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d600b03e0 0x7f0d600b0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:38.377 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.375+0000 7f0d5e59c700 1 -- 172.21.15.191:0/1991463602 >> v1:172.21.15.191:6789/0 conn(0x7f0d60159bb0 legacy=0x7f0d6015e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:38.378 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.375+0000 7f0d5dd9b700 1 -- 172.21.15.191:0/1991463602 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 741918873 0 0) 0x7f0d6006a2b0 con 0x7f0d60159bb0 2023-10-08T00:10:38.378 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.375+0000 7f0d5e59c700 1 -- 172.21.15.191:0/1991463602 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d60069320 con 0x7f0d600b17b0 2023-10-08T00:10:38.378 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.375+0000 7f0d5f59e700 1 --2- 172.21.15.191:0/1991463602 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d600b03e0 0x7f0d600b0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:10:38.378 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.375+0000 7f0d5e59c700 1 --2- 172.21.15.191:0/1991463602 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d600b17b0 0x7f0d600aa870 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0d500098e0 tx=0x7f0d500091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab250efd6f64ca85 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:38.378 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.376+0000 7f0d5dd9b700 1 -- 172.21.15.191:0/1991463602 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5000b640 con 0x7f0d600b17b0 2023-10-08T00:10:38.378 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.376+0000 7f0d5dd9b700 1 -- 172.21.15.191:0/1991463602 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d5000b7a0 con 0x7f0d600b17b0 2023-10-08T00:10:38.378 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.376+0000 7f0d5dd9b700 1 -- 172.21.15.191:0/1991463602 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5000bb40 con 0x7f0d600b17b0 2023-10-08T00:10:38.378 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.376+0000 7f0d64989700 1 -- 172.21.15.191:0/1991463602 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d600b17b0 msgr2=0x7f0d600aa870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:38.378 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.376+0000 7f0d64989700 1 --2- 172.21.15.191:0/1991463602 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d600b17b0 0x7f0d600aa870 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0d500098e0 tx=0x7f0d500091a0 comp rx=0 tx=0).stop 2023-10-08T00:10:38.378 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.376+0000 7f0d64989700 1 -- 172.21.15.191:0/1991463602 shutdown_connections 2023-10-08T00:10:38.378 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.376+0000 7f0d64989700 1 --2- 172.21.15.191:0/1991463602 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d600b03e0 0x7f0d600b0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:38.379 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.376+0000 7f0d64989700 1 --2- 172.21.15.191:0/1991463602 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d600b17b0 0x7f0d600aa870 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:38.379 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.376+0000 7f0d64989700 1 -- 172.21.15.191:0/1991463602 >> 172.21.15.191:0/1991463602 conn(0x7f0d6006c700 msgr2=0x7f0d60069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:38.379 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.376+0000 7f0d64989700 1 -- 172.21.15.191:0/1991463602 shutdown_connections 2023-10-08T00:10:38.379 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.376+0000 7f0d64989700 1 -- 172.21.15.191:0/1991463602 wait complete. 2023-10-08T00:10:38.381 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.377+0000 7f0d64989700 1 Processor -- start 2023-10-08T00:10:38.381 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.377+0000 7f0d64989700 1 -- start start 2023-10-08T00:10:38.381 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.378+0000 7f0d64989700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d600b03e0 0x7f0d6014c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:38.382 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.378+0000 7f0d64989700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d600b17b0 0x7f0d601559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:38.382 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.378+0000 7f0d64989700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d60159bb0 0x7f0d60067880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:38.382 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.378+0000 7f0d64989700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d600ab7b0 con 0x7f0d600b03e0 2023-10-08T00:10:38.382 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.378+0000 7f0d64989700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6016b1a0 con 0x7f0d60159bb0 2023-10-08T00:10:38.382 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.378+0000 7f0d64989700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d601564b0 con 0x7f0d600b17b0 2023-10-08T00:10:38.382 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.378+0000 7f0d5e59c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d600b17b0 0x7f0d601559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:38.382 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.378+0000 7f0d5f59e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d60159bb0 0x7f0d60067880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:38.382 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.378+0000 7f0d5e59c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d600b17b0 0x7f0d601559b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.191:59500/0 (socket says 172.21.15.191:59500) 2023-10-08T00:10:38.382 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.378+0000 7f0d5f59e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d60159bb0 0x7f0d60067880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:59406/0 (socket says 172.21.15.191:59406) 2023-10-08T00:10:38.382 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.378+0000 7f0d5e59c700 1 -- 172.21.15.191:0/2925034771 learned_addr learned my addr 172.21.15.191:0/2925034771 (peer_addr_for_me v2:172.21.15.191:0/0) 2023-10-08T00:10:38.383 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.378+0000 7f0d5ed9d700 1 --2- 172.21.15.191:0/2925034771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d600b03e0 0x7f0d6014c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:38.383 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.378+0000 7f0d5f59e700 1 -- 172.21.15.191:0/2925034771 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d600b17b0 msgr2=0x7f0d601559b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:38.383 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.378+0000 7f0d5f59e700 1 --2- 172.21.15.191:0/2925034771 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d600b17b0 0x7f0d601559b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:38.383 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.378+0000 7f0d5f59e700 1 -- 172.21.15.191:0/2925034771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d600b03e0 msgr2=0x7f0d6014c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:38.383 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.379+0000 7f0d5f59e700 1 --2- 172.21.15.191:0/2925034771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d600b03e0 0x7f0d6014c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:38.383 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.379+0000 7f0d5f59e700 1 -- 172.21.15.191:0/2925034771 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d58009a20 con 0x7f0d60159bb0 2023-10-08T00:10:38.383 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.379+0000 7f0d5e59c700 1 --2- 172.21.15.191:0/2925034771 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d600b17b0 0x7f0d601559b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:10:38.383 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.379+0000 7f0d5f59e700 1 --2- 172.21.15.191:0/2925034771 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d60159bb0 0x7f0d60067880 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0d58009f10 tx=0x7f0d5800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:38.383 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.379+0000 7f0d46ffd700 1 -- 172.21.15.191:0/2925034771 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5800cb70 con 0x7f0d60159bb0 2023-10-08T00:10:38.383 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.379+0000 7f0d46ffd700 1 -- 172.21.15.191:0/2925034771 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d5800f040 con 0x7f0d60159bb0 2023-10-08T00:10:38.383 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.379+0000 7f0d64989700 1 -- 172.21.15.191:0/2925034771 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d50009420 con 0x7f0d60159bb0 2023-10-08T00:10:38.384 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.379+0000 7f0d46ffd700 1 -- 172.21.15.191:0/2925034771 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d58014480 con 0x7f0d60159bb0 2023-10-08T00:10:38.384 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.379+0000 7f0d64989700 1 -- 172.21.15.191:0/2925034771 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d600ab160 con 0x7f0d60159bb0 2023-10-08T00:10:38.384 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.379+0000 7f0d46ffd700 1 -- 172.21.15.191:0/2925034771 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f0d58003020 con 0x7f0d60159bb0 2023-10-08T00:10:38.384 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.379+0000 7f0d46ffd700 1 -- 172.21.15.191:0/2925034771 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (secure 0 0 0) 0x7f0d580139e0 con 0x7f0d60159bb0 2023-10-08T00:10:38.384 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.380+0000 7f0d64989700 1 -- 172.21.15.191:0/2925034771 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d600ab920 con 0x7f0d60159bb0 2023-10-08T00:10:38.384 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.382+0000 7f0d46ffd700 1 -- 172.21.15.191:0/2925034771 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f0d58018030 con 0x7f0d60159bb0 2023-10-08T00:10:38.425 INFO:tasks.ceph.mgr.y.smithi099.stderr:2023-10-08T00:10:38.425+0000 7f3119368180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T00:10:38.448 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T00:10:38.448+0000 7fa645af8180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T00:10:38.465 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.465+0000 7f0d64989700 1 -- 172.21.15.191:0/2925034771 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 6} v 0) v1 -- 0x7f0d6006e240 con 0x7f0d60159bb0 2023-10-08T00:10:38.469 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.468+0000 7f0d46ffd700 1 -- 172.21.15.191:0/2925034771 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0d580149d0 con 0x7f0d60159bb0 2023-10-08T00:10:38.469 INFO:teuthology.orchestra.run.smithi191.stdout:6 2023-10-08T00:10:38.470 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.469+0000 7f0d64989700 1 -- 172.21.15.191:0/2925034771 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d60159bb0 msgr2=0x7f0d60067880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:38.470 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.469+0000 7f0d64989700 1 --2- 172.21.15.191:0/2925034771 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d60159bb0 0x7f0d60067880 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0d58009f10 tx=0x7f0d5800b040 comp rx=0 tx=0).stop 2023-10-08T00:10:38.470 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.469+0000 7f0d64989700 1 -- 172.21.15.191:0/2925034771 shutdown_connections 2023-10-08T00:10:38.471 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.469+0000 7f0d64989700 1 --2- 172.21.15.191:0/2925034771 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d60159bb0 0x7f0d60067880 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:38.471 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.469+0000 7f0d64989700 1 --2- 172.21.15.191:0/2925034771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d600b03e0 0x7f0d6014c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:38.471 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.469+0000 7f0d64989700 1 --2- 172.21.15.191:0/2925034771 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d600b17b0 0x7f0d601559b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:38.471 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.469+0000 7f0d64989700 1 -- 172.21.15.191:0/2925034771 >> 172.21.15.191:0/2925034771 conn(0x7f0d6006c700 msgr2=0x7f0d600aeea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:38.471 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.469+0000 7f0d64989700 1 -- 172.21.15.191:0/2925034771 shutdown_connections 2023-10-08T00:10:38.471 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.469+0000 7f0d64989700 1 -- 172.21.15.191:0/2925034771 wait complete. 2023-10-08T00:10:38.481 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 7 2023-10-08T00:10:38.503 INFO:tasks.ceph.mgr.y.smithi099.stderr:2023-10-08T00:10:38.502+0000 7f3119368180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T00:10:38.605 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.598+0000 7ff822388700 1 Processor -- start 2023-10-08T00:10:38.606 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.598+0000 7ff822388700 1 -- start start 2023-10-08T00:10:38.606 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.599+0000 7ff822388700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff81c151dc0 0x7ff81c152190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:38.606 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.599+0000 7ff822388700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff81c158e80 0x7ff81c15b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:38.606 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.599+0000 7ff822388700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff81c06bc70 con 0x7ff81c06c3e0 2023-10-08T00:10:38.606 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.599+0000 7ff822388700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff81c06bf50 con 0x7ff81c151dc0 2023-10-08T00:10:38.606 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.599+0000 7ff822388700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff81c1526d0 con 0x7ff81c158e80 2023-10-08T00:10:38.607 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.599+0000 7ff81b7fe700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff81c158e80 0x7ff81c15b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:38.607 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.599+0000 7ff820925700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7ff81c06c3e0 0x7ff81c0ca760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.191:52260/0 (socket says 172.21.15.191:52260) 2023-10-08T00:10:38.607 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.599+0000 7ff820925700 1 -- 172.21.15.191:0/339548860 learned_addr learned my addr 172.21.15.191:0/339548860 (peer_addr_for_me v1:172.21.15.191:0/0) 2023-10-08T00:10:38.607 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.599+0000 7ff81bfff700 1 --2- 172.21.15.191:0/339548860 >> v2:172.21.15.99:3300/0 conn(0x7ff81c151dc0 0x7ff81c152190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:38.607 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.600+0000 7ff81b7fe700 1 -- 172.21.15.191:0/339548860 >> v2:172.21.15.99:3300/0 conn(0x7ff81c151dc0 msgr2=0x7ff81c152190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:38.607 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.600+0000 7ff81b7fe700 1 --2- 172.21.15.191:0/339548860 >> v2:172.21.15.99:3300/0 conn(0x7ff81c151dc0 0x7ff81c152190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:38.607 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.600+0000 7ff81b7fe700 1 -- 172.21.15.191:0/339548860 >> v1:172.21.15.191:6789/0 conn(0x7ff81c06c3e0 legacy=0x7ff81c0ca760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:38.608 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.600+0000 7ff81affd700 1 -- 172.21.15.191:0/339548860 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1271947377 0 0) 0x7ff81c06bc70 con 0x7ff81c06c3e0 2023-10-08T00:10:38.608 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.600+0000 7ff81b7fe700 1 -- 172.21.15.191:0/339548860 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff81c15c000 con 0x7ff81c158e80 2023-10-08T00:10:38.608 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.600+0000 7ff81b7fe700 1 --2- 172.21.15.191:0/339548860 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff81c158e80 0x7ff81c15b7d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff81c06f350 tx=0x7ff8100091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d72514fa2a7e165 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:38.608 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.600+0000 7ff81affd700 1 -- 172.21.15.191:0/339548860 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff810011630 con 0x7ff81c158e80 2023-10-08T00:10:38.608 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.600+0000 7ff81affd700 1 -- 172.21.15.191:0/339548860 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff810011790 con 0x7ff81c158e80 2023-10-08T00:10:38.608 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.600+0000 7ff81affd700 1 -- 172.21.15.191:0/339548860 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff810011b30 con 0x7ff81c158e80 2023-10-08T00:10:38.608 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.600+0000 7ff822388700 1 -- 172.21.15.191:0/339548860 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff81c158e80 msgr2=0x7ff81c15b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:38.609 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.600+0000 7ff822388700 1 --2- 172.21.15.191:0/339548860 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff81c158e80 0x7ff81c15b7d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff81c06f350 tx=0x7ff8100091a0 comp rx=0 tx=0).stop 2023-10-08T00:10:38.609 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.600+0000 7ff822388700 1 -- 172.21.15.191:0/339548860 shutdown_connections 2023-10-08T00:10:38.609 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.600+0000 7ff822388700 1 --2- 172.21.15.191:0/339548860 >> v2:172.21.15.99:3300/0 conn(0x7ff81c151dc0 0x7ff81c152190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:38.609 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.600+0000 7ff822388700 1 --2- 172.21.15.191:0/339548860 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff81c158e80 0x7ff81c15b7d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:38.609 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.600+0000 7ff822388700 1 -- 172.21.15.191:0/339548860 >> 172.21.15.191:0/339548860 conn(0x7ff81c069870 msgr2=0x7ff81c069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:38.609 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.600+0000 7ff822388700 1 -- 172.21.15.191:0/339548860 shutdown_connections 2023-10-08T00:10:38.609 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.600+0000 7ff822388700 1 -- 172.21.15.191:0/339548860 wait complete. 2023-10-08T00:10:38.610 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.601+0000 7ff822388700 1 Processor -- start 2023-10-08T00:10:38.610 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.602+0000 7ff822388700 1 -- start start 2023-10-08T00:10:38.610 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.602+0000 7ff822388700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff81c06c3e0 0x7ff81c14d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:38.610 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.602+0000 7ff822388700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff81c151dc0 0x7ff81c158200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:38.610 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.602+0000 7ff822388700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff81c158e80 0x7ff81c158740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:38.610 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.602+0000 7ff822388700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c0c8bc0 con 0x7ff81c151dc0 2023-10-08T00:10:38.610 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.602+0000 7ff822388700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c0b18e0 con 0x7ff81c158e80 2023-10-08T00:10:38.611 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.602+0000 7ff822388700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c0b1b30 con 0x7ff81c06c3e0 2023-10-08T00:10:38.611 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.602+0000 7ff81bfff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff81c06c3e0 0x7ff81c14d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:38.611 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.603+0000 7ff81bfff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff81c06c3e0 0x7ff81c14d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.191:59512/0 (socket says 172.21.15.191:59512) 2023-10-08T00:10:38.611 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.603+0000 7ff81bfff700 1 -- 172.21.15.191:0/4210803839 learned_addr learned my addr 172.21.15.191:0/4210803839 (peer_addr_for_me v2:172.21.15.191:0/0) 2023-10-08T00:10:38.611 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.603+0000 7ff820925700 1 --2- 172.21.15.191:0/4210803839 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff81c158e80 0x7ff81c158740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:38.611 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.603+0000 7ff81b7fe700 1 --2- 172.21.15.191:0/4210803839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff81c151dc0 0x7ff81c158200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:38.612 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.603+0000 7ff81bfff700 1 -- 172.21.15.191:0/4210803839 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff81c158e80 msgr2=0x7ff81c158740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:38.612 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.603+0000 7ff81bfff700 1 --2- 172.21.15.191:0/4210803839 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff81c158e80 0x7ff81c158740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:38.612 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.603+0000 7ff81bfff700 1 -- 172.21.15.191:0/4210803839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff81c151dc0 msgr2=0x7ff81c158200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:38.612 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.603+0000 7ff81bfff700 1 --2- 172.21.15.191:0/4210803839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff81c151dc0 0x7ff81c158200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:38.612 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.603+0000 7ff81bfff700 1 -- 172.21.15.191:0/4210803839 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff810009420 con 0x7ff81c06c3e0 2023-10-08T00:10:38.612 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.603+0000 7ff820925700 1 --2- 172.21.15.191:0/4210803839 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff81c158e80 0x7ff81c158740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:10:38.612 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.603+0000 7ff81bfff700 1 --2- 172.21.15.191:0/4210803839 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff81c06c3e0 0x7ff81c14d910 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff80c00ec80 tx=0x7ff80c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:38.613 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.603+0000 7ff803fff700 1 -- 172.21.15.191:0/4210803839 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff80c00c850 con 0x7ff81c06c3e0 2023-10-08T00:10:38.613 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.603+0000 7ff803fff700 1 -- 172.21.15.191:0/4210803839 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff80c00c9b0 con 0x7ff81c06c3e0 2023-10-08T00:10:38.613 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.603+0000 7ff803fff700 1 -- 172.21.15.191:0/4210803839 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff80c00cd50 con 0x7ff81c06c3e0 2023-10-08T00:10:38.613 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.603+0000 7ff822388700 1 -- 172.21.15.191:0/4210803839 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff81c0b1e10 con 0x7ff81c06c3e0 2023-10-08T00:10:38.613 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.603+0000 7ff822388700 1 -- 172.21.15.191:0/4210803839 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff81c0b23d0 con 0x7ff81c06c3e0 2023-10-08T00:10:38.613 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.603+0000 7ff803fff700 1 -- 172.21.15.191:0/4210803839 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7ff80c005c80 con 0x7ff81c06c3e0 2023-10-08T00:10:38.614 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.603+0000 7ff803fff700 1 -- 172.21.15.191:0/4210803839 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1526+0+0 (secure 0 0 0) 0x7ff80c006690 con 0x7ff81c06c3e0 2023-10-08T00:10:38.614 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.604+0000 7ff822388700 1 -- 172.21.15.191:0/4210803839 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff81c04c3c0 con 0x7ff81c06c3e0 2023-10-08T00:10:38.614 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.606+0000 7ff803fff700 1 -- 172.21.15.191:0/4210803839 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7ff80c021060 con 0x7ff81c06c3e0 2023-10-08T00:10:38.637 INFO:tasks.ceph.mgr.y.smithi099.stderr:2023-10-08T00:10:38.636+0000 7f3119368180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T00:10:38.689 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.689+0000 7ff822388700 1 -- 172.21.15.191:0/4210803839 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 7} v 0) v1 -- 0x7ff81c15c2e0 con 0x7ff81c06c3e0 2023-10-08T00:10:38.697 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.697+0000 7ff803fff700 1 -- 172.21.15.191:0/4210803839 <== mon.2 v2:172.21.15.191:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff80c017070 con 0x7ff81c06c3e0 2023-10-08T00:10:38.697 INFO:teuthology.orchestra.run.smithi191.stdout:7 2023-10-08T00:10:38.699 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.698+0000 7ff822388700 1 -- 172.21.15.191:0/4210803839 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff81c06c3e0 msgr2=0x7ff81c14d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:38.699 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.698+0000 7ff822388700 1 --2- 172.21.15.191:0/4210803839 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff81c06c3e0 0x7ff81c14d910 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff80c00ec80 tx=0x7ff80c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:10:38.699 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.698+0000 7ff822388700 1 -- 172.21.15.191:0/4210803839 shutdown_connections 2023-10-08T00:10:38.699 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.698+0000 7ff822388700 1 --2- 172.21.15.191:0/4210803839 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff81c158e80 0x7ff81c158740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:38.700 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.698+0000 7ff822388700 1 --2- 172.21.15.191:0/4210803839 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff81c06c3e0 0x7ff81c14d910 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:38.700 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.698+0000 7ff822388700 1 --2- 172.21.15.191:0/4210803839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff81c151dc0 0x7ff81c158200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:38.700 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.698+0000 7ff822388700 1 -- 172.21.15.191:0/4210803839 >> 172.21.15.191:0/4210803839 conn(0x7ff81c069870 msgr2=0x7ff81c151600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:38.700 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.698+0000 7ff822388700 1 -- 172.21.15.191:0/4210803839 shutdown_connections 2023-10-08T00:10:38.700 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T00:10:38.698+0000 7ff822388700 1 -- 172.21.15.191:0/4210803839 wait complete. 2023-10-08T00:10:38.711 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-08T00:10:38.711 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-10-08T00:10:38.713 INFO:tasks.ceph.osd.0:Started 2023-10-08T00:10:38.713 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-08T00:10:38.714 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-10-08T00:10:38.716 INFO:tasks.ceph.osd.1:Started 2023-10-08T00:10:38.716 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-08T00:10:38.716 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-10-08T00:10:38.719 INFO:tasks.ceph.osd.2:Started 2023-10-08T00:10:38.719 INFO:tasks.ceph.osd.3:Restarting daemon 2023-10-08T00:10:38.719 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-10-08T00:10:38.721 INFO:tasks.ceph.osd.3:Started 2023-10-08T00:10:38.721 INFO:tasks.ceph.osd.4:Restarting daemon 2023-10-08T00:10:38.721 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-10-08T00:10:38.724 INFO:tasks.ceph.osd.4:Started 2023-10-08T00:10:38.724 INFO:tasks.ceph.osd.5:Restarting daemon 2023-10-08T00:10:38.724 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-10-08T00:10:38.727 INFO:tasks.ceph.osd.5:Started 2023-10-08T00:10:38.727 INFO:tasks.ceph.osd.6:Restarting daemon 2023-10-08T00:10:38.727 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-10-08T00:10:38.729 INFO:tasks.ceph.osd.6:Started 2023-10-08T00:10:38.729 INFO:tasks.ceph.osd.7:Restarting daemon 2023-10-08T00:10:38.730 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-10-08T00:10:38.732 INFO:tasks.ceph.osd.7:Started 2023-10-08T00:10:38.732 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:10:38.739 INFO:tasks.ceph.mgr.y.smithi099.stderr:2023-10-08T00:10:38.735+0000 7f3119368180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T00:10:38.798 INFO:tasks.ceph.osd.0.smithi099.stderr:2023-10-08T00:10:38.797+0000 7fc82480c540 -1 Falling back to public interface 2023-10-08T00:10:38.801 INFO:tasks.ceph.osd.4.smithi191.stderr:2023-10-08T00:10:38.801+0000 7fc947086540 -1 Falling back to public interface 2023-10-08T00:10:38.804 INFO:tasks.ceph.osd.1.smithi099.stderr:2023-10-08T00:10:38.803+0000 7fa37d4e0540 -1 Falling back to public interface 2023-10-08T00:10:38.808 INFO:tasks.ceph.osd.6.smithi191.stderr:2023-10-08T00:10:38.808+0000 7f0126459540 -1 Falling back to public interface 2023-10-08T00:10:38.813 INFO:tasks.ceph.osd.4.smithi191.stderr:2023-10-08T00:10:38.813+0000 7fc947086540 -1 osd.4 0 log_to_monitors true 2023-10-08T00:10:38.826 INFO:tasks.ceph.osd.6.smithi191.stderr:2023-10-08T00:10:38.826+0000 7f0126459540 -1 osd.6 0 log_to_monitors true 2023-10-08T00:10:38.829 INFO:tasks.ceph.osd.5.smithi191.stderr:2023-10-08T00:10:38.830+0000 7f30eef23540 -1 Falling back to public interface 2023-10-08T00:10:38.830 INFO:tasks.ceph.osd.7.smithi191.stderr:2023-10-08T00:10:38.830+0000 7faef20c7540 -1 Falling back to public interface 2023-10-08T00:10:38.843 INFO:tasks.ceph.osd.5.smithi191.stderr:2023-10-08T00:10:38.843+0000 7f30eef23540 -1 osd.5 0 log_to_monitors true 2023-10-08T00:10:38.844 INFO:tasks.ceph.osd.7.smithi191.stderr:2023-10-08T00:10:38.844+0000 7faef20c7540 -1 osd.7 0 log_to_monitors true 2023-10-08T00:10:38.851 INFO:tasks.ceph.osd.2.smithi099.stderr:2023-10-08T00:10:38.850+0000 7fe19c506540 -1 Falling back to public interface 2023-10-08T00:10:38.851 INFO:tasks.ceph.osd.3.smithi099.stderr:2023-10-08T00:10:38.850+0000 7fec0b902540 -1 Falling back to public interface 2023-10-08T00:10:38.855 INFO:tasks.ceph.osd.1.smithi099.stderr:2023-10-08T00:10:38.855+0000 7fa37d4e0540 -1 osd.1 0 log_to_monitors true 2023-10-08T00:10:38.855 INFO:tasks.ceph.osd.0.smithi099.stderr:2023-10-08T00:10:38.855+0000 7fc82480c540 -1 osd.0 0 log_to_monitors true 2023-10-08T00:10:38.867 INFO:tasks.ceph.osd.2.smithi099.stderr:2023-10-08T00:10:38.866+0000 7fe19c506540 -1 osd.2 0 log_to_monitors true 2023-10-08T00:10:38.867 INFO:tasks.ceph.osd.3.smithi099.stderr:2023-10-08T00:10:38.867+0000 7fec0b902540 -1 osd.3 0 log_to_monitors true 2023-10-08T00:10:38.880 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.879+0000 7facfbf5b700 1 Processor -- start 2023-10-08T00:10:38.880 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.880+0000 7facfbf5b700 1 -- start start 2023-10-08T00:10:38.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.880+0000 7facfbf5b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7facf41572a0 0x7facf4157700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:38.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.880+0000 7facfbf5b700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facf40ae760 con 0x7facf40a7df0 2023-10-08T00:10:38.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.880+0000 7facfbf5b700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facf40aea40 con 0x7facf40a7a20 2023-10-08T00:10:38.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.880+0000 7facfbf5b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facf40aed20 con 0x7facf41572a0 2023-10-08T00:10:38.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.880+0000 7facf94f6700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7facf40a7df0 0x7facf40b29c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:53888/0 (socket says 172.21.15.99:53888) 2023-10-08T00:10:38.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.881+0000 7facf94f6700 1 -- 172.21.15.99:0/627820996 learned_addr learned my addr 172.21.15.99:0/627820996 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:10:38.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.881+0000 7facfa4f8700 1 --2- 172.21.15.99:0/627820996 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7facf41572a0 0x7facf4157700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:38.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.881+0000 7facf8cf5700 1 -- 172.21.15.99:0/627820996 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3340624841 0 0) 0x7facf40aea40 con 0x7facf40a7a20 2023-10-08T00:10:38.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.881+0000 7facf8cf5700 1 -- 172.21.15.99:0/627820996 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7face0003570 con 0x7facf40a7a20 2023-10-08T00:10:38.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.881+0000 7facf8cf5700 1 -- 172.21.15.99:0/627820996 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3958262456 0 0) 0x7facf40ae760 con 0x7facf40a7df0 2023-10-08T00:10:38.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.881+0000 7facf8cf5700 1 -- 172.21.15.99:0/627820996 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facf40aea40 con 0x7facf40a7df0 2023-10-08T00:10:38.884 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.881+0000 7facfa4f8700 1 -- 172.21.15.99:0/627820996 >> v1:172.21.15.191:6789/0 conn(0x7facf40a7a20 legacy=0x7facf40af520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:38.884 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.881+0000 7facfa4f8700 1 -- 172.21.15.99:0/627820996 >> v1:172.21.15.99:6789/0 conn(0x7facf40a7df0 legacy=0x7facf40b29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:38.884 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.881+0000 7facfa4f8700 1 -- 172.21.15.99:0/627820996 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf40af000 con 0x7facf41572a0 2023-10-08T00:10:38.884 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.881+0000 7facfa4f8700 1 --2- 172.21.15.99:0/627820996 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7facf41572a0 0x7facf4157700 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7facec009e90 tx=0x7facec009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bce59c8e197efba5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:38.884 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.881+0000 7facf8cf5700 1 -- 172.21.15.99:0/627820996 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facec00e070 con 0x7facf41572a0 2023-10-08T00:10:38.884 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.881+0000 7facf8cf5700 1 -- 172.21.15.99:0/627820996 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facec00bc10 con 0x7facf41572a0 2023-10-08T00:10:38.884 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.881+0000 7facf8cf5700 1 -- 172.21.15.99:0/627820996 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facec012430 con 0x7facf41572a0 2023-10-08T00:10:38.884 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.882+0000 7facfbf5b700 1 -- 172.21.15.99:0/627820996 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7facf41572a0 msgr2=0x7facf4157700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:38.884 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.882+0000 7facfbf5b700 1 --2- 172.21.15.99:0/627820996 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7facf41572a0 0x7facf4157700 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7facec009e90 tx=0x7facec009780 comp rx=0 tx=0).stop 2023-10-08T00:10:38.884 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.882+0000 7facfbf5b700 1 -- 172.21.15.99:0/627820996 shutdown_connections 2023-10-08T00:10:38.884 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.882+0000 7facfbf5b700 1 --2- 172.21.15.99:0/627820996 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7facf41572a0 0x7facf4157700 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:38.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.882+0000 7facfbf5b700 1 -- 172.21.15.99:0/627820996 >> 172.21.15.99:0/627820996 conn(0x7facf40cd1e0 msgr2=0x7facf40cd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:38.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.882+0000 7facfbf5b700 1 -- 172.21.15.99:0/627820996 shutdown_connections 2023-10-08T00:10:38.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.882+0000 7facfbf5b700 1 -- 172.21.15.99:0/627820996 wait complete. 2023-10-08T00:10:38.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.883+0000 7facfbf5b700 1 Processor -- start 2023-10-08T00:10:38.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.883+0000 7facfbf5b700 1 -- start start 2023-10-08T00:10:38.888 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.884+0000 7facfbf5b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7facf40a7a20 0x7facf416c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:38.888 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.884+0000 7facfbf5b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7facf40a7df0 0x7facf406eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:38.888 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.884+0000 7facfbf5b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7facf41572a0 0x7facf406eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:38.888 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.884+0000 7facfbf5b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf40aafb0 con 0x7facf40a7a20 2023-10-08T00:10:38.888 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.884+0000 7facfbf5b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf416cef0 con 0x7facf41572a0 2023-10-08T00:10:38.888 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.884+0000 7facfbf5b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facf40aa3a0 con 0x7facf40a7df0 2023-10-08T00:10:38.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.884+0000 7facf9cf7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7facf40a7a20 0x7facf416c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:38.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.884+0000 7facf9cf7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7facf40a7a20 0x7facf416c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49134/0 (socket says 172.21.15.99:49134) 2023-10-08T00:10:38.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.884+0000 7facf9cf7700 1 -- 172.21.15.99:0/826592571 learned_addr learned my addr 172.21.15.99:0/826592571 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:10:38.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.884+0000 7facf94f6700 1 --2- 172.21.15.99:0/826592571 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7facf40a7df0 0x7facf406eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:38.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.884+0000 7facfa4f8700 1 --2- 172.21.15.99:0/826592571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7facf41572a0 0x7facf406eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:38.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.884+0000 7facf9cf7700 1 -- 172.21.15.99:0/826592571 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7facf40a7df0 msgr2=0x7facf406eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:38.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.884+0000 7facf9cf7700 1 --2- 172.21.15.99:0/826592571 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7facf40a7df0 0x7facf406eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:38.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.884+0000 7facf9cf7700 1 -- 172.21.15.99:0/826592571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7facf41572a0 msgr2=0x7facf406eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:38.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.884+0000 7facf9cf7700 1 --2- 172.21.15.99:0/826592571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7facf41572a0 0x7facf406eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:38.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.884+0000 7facf9cf7700 1 -- 172.21.15.99:0/826592571 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facec009a20 con 0x7facf40a7a20 2023-10-08T00:10:38.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.884+0000 7facf9cf7700 1 --2- 172.21.15.99:0/826592571 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7facf40a7a20 0x7facf416c9b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7face400c7c0 tx=0x7face4007e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:38.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.884+0000 7face9ffb700 1 -- 172.21.15.99:0/826592571 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face4010510 con 0x7facf40a7a20 2023-10-08T00:10:38.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.884+0000 7facfbf5b700 1 -- 172.21.15.99:0/826592571 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facf40aa680 con 0x7facf40a7a20 2023-10-08T00:10:38.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.884+0000 7face9ffb700 1 -- 172.21.15.99:0/826592571 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face4010670 con 0x7facf40a7a20 2023-10-08T00:10:38.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.884+0000 7face9ffb700 1 -- 172.21.15.99:0/826592571 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face4010a10 con 0x7facf40a7a20 2023-10-08T00:10:38.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.885+0000 7face9ffb700 1 -- 172.21.15.99:0/826592571 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7face40126c0 con 0x7facf40a7a20 2023-10-08T00:10:38.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.885+0000 7facfbf5b700 1 -- 172.21.15.99:0/826592571 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facf40aac40 con 0x7facf40a7a20 2023-10-08T00:10:38.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.885+0000 7face9ffb700 1 -- 172.21.15.99:0/826592571 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7face401ba90 con 0x7facf40a7a20 2023-10-08T00:10:38.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.885+0000 7facfbf5b700 1 -- 172.21.15.99:0/826592571 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facf404c3c0 con 0x7facf40a7a20 2023-10-08T00:10:38.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.887+0000 7face9ffb700 1 -- 172.21.15.99:0/826592571 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7face4021060 con 0x7facf40a7a20 2023-10-08T00:10:38.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.955+0000 7facfbf5b700 1 -- 172.21.15.99:0/826592571 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7facf40ac530 con 0x7facf40a7a20 2023-10-08T00:10:38.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.956+0000 7face9ffb700 1 -- 172.21.15.99:0/826592571 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7face4017070 con 0x7facf40a7a20 2023-10-08T00:10:38.957 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:10:38.957 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":10,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:10:38.690404+0000","last_up_change":"0.000000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:10:38.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.956+0000 7facfbf5b700 1 -- 172.21.15.99:0/826592571 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7facf40a7a20 msgr2=0x7facf416c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:38.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.956+0000 7facfbf5b700 1 --2- 172.21.15.99:0/826592571 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7facf40a7a20 0x7facf416c9b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7face400c7c0 tx=0x7face4007e30 comp rx=0 tx=0).stop 2023-10-08T00:10:38.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.956+0000 7facfbf5b700 1 -- 172.21.15.99:0/826592571 shutdown_connections 2023-10-08T00:10:38.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.956+0000 7facfbf5b700 1 --2- 172.21.15.99:0/826592571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7facf41572a0 0x7facf406eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:38.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.956+0000 7facfbf5b700 1 --2- 172.21.15.99:0/826592571 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7facf40a7a20 0x7facf416c9b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:38.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.956+0000 7facfbf5b700 1 --2- 172.21.15.99:0/826592571 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7facf40a7df0 0x7facf406eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:38.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.956+0000 7facfbf5b700 1 -- 172.21.15.99:0/826592571 >> 172.21.15.99:0/826592571 conn(0x7facf40cd1e0 msgr2=0x7facf4063230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:38.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.957+0000 7facfbf5b700 1 -- 172.21.15.99:0/826592571 shutdown_connections 2023-10-08T00:10:38.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:38.957+0000 7facfbf5b700 1 -- 172.21.15.99:0/826592571 wait complete. 2023-10-08T00:10:38.968 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-10-08T00:10:38.969 INFO:tasks.ceph:Waiting for OSDs to come up 2023-10-08T00:10:38.990 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T00:10:38.990+0000 7fa645af8180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T00:10:39.067 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T00:10:39.067+0000 7fa645af8180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T00:10:39.144 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T00:10:39.144+0000 7fa645af8180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T00:10:39.735 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T00:10:39.735+0000 7fa645af8180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T00:10:39.774 DEBUG:teuthology.orchestra.run.smithi099:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-08T00:10:39.872 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T00:10:39.872+0000 7fa645af8180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T00:10:39.890 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.889+0000 7f7134d11700 1 Processor -- start 2023-10-08T00:10:39.890 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.890+0000 7f7134d11700 1 -- start start 2023-10-08T00:10:39.894 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.890+0000 7f7134d11700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7130157c40 0x7f7130158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:39.894 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.890+0000 7f7134d11700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f713015cbb0 con 0x7f7130157870 2023-10-08T00:10:39.894 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.890+0000 7f7134d11700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f713015bfa0 con 0x7f713015a620 2023-10-08T00:10:39.894 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.890+0000 7f7134d11700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f713015bdb0 con 0x7f7130157c40 2023-10-08T00:10:39.894 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.891+0000 7f712e59c700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f7130157870 0x7f713015b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:53892/0 (socket says 172.21.15.99:53892) 2023-10-08T00:10:39.894 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.891+0000 7f712e59c700 1 -- 172.21.15.99:0/143973019 learned_addr learned my addr 172.21.15.99:0/143973019 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:10:39.895 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.891+0000 7f712ed9d700 1 --2- 172.21.15.99:0/143973019 >> v2:172.21.15.191:3300/0 conn(0x7f7130157c40 0x7f7130158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:39.895 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.891+0000 7f712d59a700 1 -- 172.21.15.99:0/143973019 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1514346373 0 0) 0x7f713015cbb0 con 0x7f7130157870 2023-10-08T00:10:39.895 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.891+0000 7f712d59a700 1 -- 172.21.15.99:0/143973019 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7118003570 con 0x7f7130157870 2023-10-08T00:10:39.895 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.891+0000 7f712d59a700 1 -- 172.21.15.99:0/143973019 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2521096131 0 0) 0x7f713015bfa0 con 0x7f713015a620 2023-10-08T00:10:39.895 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.891+0000 7f712d59a700 1 -- 172.21.15.99:0/143973019 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f713015cbb0 con 0x7f713015a620 2023-10-08T00:10:39.895 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.891+0000 7f712ed9d700 1 -- 172.21.15.99:0/143973019 >> v1:172.21.15.191:6789/0 conn(0x7f713015a620 legacy=0x7f713015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:39.895 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.891+0000 7f712ed9d700 1 -- 172.21.15.99:0/143973019 >> v1:172.21.15.99:6789/0 conn(0x7f7130157870 legacy=0x7f713015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:39.895 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.891+0000 7f712d59a700 1 -- 172.21.15.99:0/143973019 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 610943813 0 0) 0x7f7118003570 con 0x7f7130157870 2023-10-08T00:10:39.895 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.891+0000 7f712ed9d700 1 -- 172.21.15.99:0/143973019 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f713015c280 con 0x7f7130157c40 2023-10-08T00:10:39.895 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.891+0000 7f712ed9d700 1 --2- 172.21.15.99:0/143973019 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7130157c40 0x7f7130158110 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7128009e60 tx=0x7f7128009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa424b2d43a5d67e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:39.897 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.891+0000 7f712d59a700 1 -- 172.21.15.99:0/143973019 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712800e070 con 0x7f7130157c40 2023-10-08T00:10:39.897 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.891+0000 7f712d59a700 1 -- 172.21.15.99:0/143973019 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f712800b7a0 con 0x7f7130157c40 2023-10-08T00:10:39.897 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.891+0000 7f712d59a700 1 -- 172.21.15.99:0/143973019 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7124001470 con 0x7f7130157c40 2023-10-08T00:10:39.897 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.892+0000 7f7134d11700 1 -- 172.21.15.99:0/143973019 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7130157c40 msgr2=0x7f7130158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:39.897 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.892+0000 7f7134d11700 1 --2- 172.21.15.99:0/143973019 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7130157c40 0x7f7130158110 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7128009e60 tx=0x7f7128009780 comp rx=0 tx=0).stop 2023-10-08T00:10:39.897 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.892+0000 7f7134d11700 1 -- 172.21.15.99:0/143973019 shutdown_connections 2023-10-08T00:10:39.897 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.892+0000 7f7134d11700 1 --2- 172.21.15.99:0/143973019 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7130157c40 0x7f7130158110 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:39.897 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.892+0000 7f7134d11700 1 -- 172.21.15.99:0/143973019 >> 172.21.15.99:0/143973019 conn(0x7f7130068ce0 msgr2=0x7f71300690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:39.898 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.892+0000 7f7134d11700 1 -- 172.21.15.99:0/143973019 shutdown_connections 2023-10-08T00:10:39.898 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.892+0000 7f7134d11700 1 -- 172.21.15.99:0/143973019 wait complete. 2023-10-08T00:10:39.898 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.893+0000 7f7134d11700 1 Processor -- start 2023-10-08T00:10:39.898 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.893+0000 7f7134d11700 1 -- start start 2023-10-08T00:10:39.900 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.894+0000 7f7134d11700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7130157870 0x7f713014f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:39.900 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.894+0000 7f7134d11700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7130157c40 0x7f71300b1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:39.900 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.894+0000 7f7134d11700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f713015a620 0x7f713013f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:39.901 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.894+0000 7f7134d11700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71300b2f60 con 0x7f713015a620 2023-10-08T00:10:39.901 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.894+0000 7f7134d11700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71300b1ec0 con 0x7f7130157c40 2023-10-08T00:10:39.901 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.894+0000 7f7134d11700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71300b21a0 con 0x7f7130157870 2023-10-08T00:10:39.901 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.894+0000 7f712ed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f713015a620 0x7f713013f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:39.901 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.894+0000 7f712ed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f713015a620 0x7f713013f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49136/0 (socket says 172.21.15.99:49136) 2023-10-08T00:10:39.901 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.894+0000 7f712ed9d700 1 -- 172.21.15.99:0/3978929475 learned_addr learned my addr 172.21.15.99:0/3978929475 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:10:39.901 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.894+0000 7f712dd9b700 1 --2- 172.21.15.99:0/3978929475 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7130157c40 0x7f71300b1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:39.901 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.894+0000 7f712e59c700 1 --2- 172.21.15.99:0/3978929475 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7130157870 0x7f713014f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:39.901 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.894+0000 7f712ed9d700 1 -- 172.21.15.99:0/3978929475 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7130157870 msgr2=0x7f713014f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:39.901 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.894+0000 7f712ed9d700 1 --2- 172.21.15.99:0/3978929475 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7130157870 0x7f713014f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:39.901 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.894+0000 7f712ed9d700 1 -- 172.21.15.99:0/3978929475 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7130157c40 msgr2=0x7f71300b1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:39.901 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.894+0000 7f712ed9d700 1 --2- 172.21.15.99:0/3978929475 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7130157c40 0x7f71300b1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:39.902 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.894+0000 7f712ed9d700 1 -- 172.21.15.99:0/3978929475 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7128009a20 con 0x7f713015a620 2023-10-08T00:10:39.902 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.894+0000 7f712e59c700 1 --2- 172.21.15.99:0/3978929475 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7130157870 0x7f713014f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:10:39.902 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.894+0000 7f712ed9d700 1 --2- 172.21.15.99:0/3978929475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f713015a620 0x7f713013f6f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7128003000 tx=0x7f7128017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:39.902 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.895+0000 7f711e7fc700 1 -- 172.21.15.99:0/3978929475 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7128012c20 con 0x7f713015a620 2023-10-08T00:10:39.902 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.895+0000 7f711e7fc700 1 -- 172.21.15.99:0/3978929475 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7128012d80 con 0x7f713015a620 2023-10-08T00:10:39.902 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.895+0000 7f7134d11700 1 -- 172.21.15.99:0/3978929475 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71300b2480 con 0x7f713015a620 2023-10-08T00:10:39.902 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.895+0000 7f711e7fc700 1 -- 172.21.15.99:0/3978929475 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712801e660 con 0x7f713015a620 2023-10-08T00:10:39.902 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.895+0000 7f7134d11700 1 -- 172.21.15.99:0/3978929475 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71300b2a40 con 0x7f713015a620 2023-10-08T00:10:39.902 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.895+0000 7f711e7fc700 1 -- 172.21.15.99:0/3978929475 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f712801e9d0 con 0x7f713015a620 2023-10-08T00:10:39.902 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.895+0000 7f711e7fc700 1 -- 172.21.15.99:0/3978929475 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 1879+0+0 (secure 0 0 0) 0x7f7128027e00 con 0x7f713015a620 2023-10-08T00:10:39.902 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.895+0000 7f7134d11700 1 -- 172.21.15.99:0/3978929475 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71300b2db0 con 0x7f713015a620 2023-10-08T00:10:39.903 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.897+0000 7f711e7fc700 1 -- 172.21.15.99:0/3978929475 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f712802c060 con 0x7f713015a620 2023-10-08T00:10:39.958 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T00:10:39.958+0000 7fa645af8180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T00:10:39.967 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.967+0000 7f7134d11700 1 -- 172.21.15.99:0/3978929475 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71300686f0 con 0x7f713015a620 2023-10-08T00:10:39.968 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.967+0000 7f711e7fc700 1 -- 172.21.15.99:0/3978929475 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v11) v1 ==== 74+0+7013 (secure 0 0 0) 0x7f7128022070 con 0x7f713015a620 2023-10-08T00:10:39.968 INFO:teuthology.misc.health.smithi099.stdout: 2023-10-08T00:10:39.968 INFO:teuthology.misc.health.smithi099.stdout:{"epoch":11,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:10:39.693809+0000","last_up_change":"0.000000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:10:39.969 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.968+0000 7f7134d11700 1 -- 172.21.15.99:0/3978929475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f713015a620 msgr2=0x7f713013f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:39.969 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.968+0000 7f7134d11700 1 --2- 172.21.15.99:0/3978929475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f713015a620 0x7f713013f6f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7128003000 tx=0x7f7128017000 comp rx=0 tx=0).stop 2023-10-08T00:10:39.969 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.968+0000 7f7134d11700 1 -- 172.21.15.99:0/3978929475 shutdown_connections 2023-10-08T00:10:39.970 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.968+0000 7f7134d11700 1 --2- 172.21.15.99:0/3978929475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f713015a620 0x7f713013f6f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:39.970 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.968+0000 7f7134d11700 1 --2- 172.21.15.99:0/3978929475 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7130157870 0x7f713014f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:39.970 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.968+0000 7f7134d11700 1 --2- 172.21.15.99:0/3978929475 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7130157c40 0x7f71300b1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:39.970 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.968+0000 7f7134d11700 1 -- 172.21.15.99:0/3978929475 >> 172.21.15.99:0/3978929475 conn(0x7f7130068ce0 msgr2=0x7f71300b46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:39.970 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.968+0000 7f7134d11700 1 -- 172.21.15.99:0/3978929475 shutdown_connections 2023-10-08T00:10:39.970 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:39.968+0000 7f7134d11700 1 -- 172.21.15.99:0/3978929475 wait complete. 2023-10-08T00:10:39.979 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-08T00:10:40.044 INFO:tasks.ceph.mgr.y.smithi099.stderr:2023-10-08T00:10:40.043+0000 7f3119368180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T00:10:40.121 INFO:tasks.ceph.mgr.y.smithi099.stderr:2023-10-08T00:10:40.120+0000 7f3119368180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T00:10:40.198 INFO:tasks.ceph.mgr.y.smithi099.stderr:2023-10-08T00:10:40.197+0000 7f3119368180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T00:10:40.208 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T00:10:40.208+0000 7fa645af8180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T00:10:40.287 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T00:10:40.287+0000 7fa645af8180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T00:10:40.713 INFO:tasks.ceph.osd.2.smithi099.stderr:2023-10-08T00:10:40.712+0000 7fe193929700 -1 osd.2 0 waiting for initial osdmap 2023-10-08T00:10:40.713 INFO:tasks.ceph.osd.4.smithi191.stderr:2023-10-08T00:10:40.713+0000 7fc93e4a9700 -1 osd.4 0 waiting for initial osdmap 2023-10-08T00:10:40.714 INFO:tasks.ceph.osd.6.smithi191.stderr:2023-10-08T00:10:40.714+0000 7f011d87c700 -1 osd.6 0 waiting for initial osdmap 2023-10-08T00:10:40.714 INFO:tasks.ceph.osd.0.smithi099.stderr:2023-10-08T00:10:40.714+0000 7fc81bc2f700 -1 osd.0 0 waiting for initial osdmap 2023-10-08T00:10:40.714 INFO:tasks.ceph.osd.2.smithi099.stderr:2023-10-08T00:10:40.714+0000 7fe18d722700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T00:10:40.715 INFO:tasks.ceph.osd.4.smithi191.stderr:2023-10-08T00:10:40.715+0000 7fc9382a2700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T00:10:40.715 INFO:tasks.ceph.osd.1.smithi099.stderr:2023-10-08T00:10:40.714+0000 7fa374903700 -1 osd.1 0 waiting for initial osdmap 2023-10-08T00:10:40.715 INFO:tasks.ceph.osd.6.smithi191.stderr:2023-10-08T00:10:40.715+0000 7f0117675700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T00:10:40.717 INFO:tasks.ceph.osd.0.smithi099.stderr:2023-10-08T00:10:40.715+0000 7fc815a28700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T00:10:40.717 INFO:tasks.ceph.osd.1.smithi099.stderr:2023-10-08T00:10:40.716+0000 7fa36e6fc700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T00:10:40.720 INFO:tasks.ceph.osd.3.smithi099.stderr:2023-10-08T00:10:40.720+0000 7fec02d25700 -1 osd.3 0 waiting for initial osdmap 2023-10-08T00:10:40.721 INFO:tasks.ceph.osd.7.smithi191.stderr:2023-10-08T00:10:40.721+0000 7faee94ea700 -1 osd.7 0 waiting for initial osdmap 2023-10-08T00:10:40.721 INFO:tasks.ceph.osd.5.smithi191.stderr:2023-10-08T00:10:40.721+0000 7f30e6346700 -1 osd.5 0 waiting for initial osdmap 2023-10-08T00:10:40.721 INFO:tasks.ceph.osd.3.smithi099.stderr:2023-10-08T00:10:40.721+0000 7febfcb1e700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T00:10:40.722 INFO:tasks.ceph.osd.7.smithi191.stderr:2023-10-08T00:10:40.722+0000 7faee32e3700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T00:10:40.722 INFO:tasks.ceph.osd.5.smithi191.stderr:2023-10-08T00:10:40.723+0000 7f30e013f700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T00:10:40.808 INFO:tasks.ceph.mgr.y.smithi099.stderr:2023-10-08T00:10:40.808+0000 7f3119368180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T00:10:40.946 INFO:tasks.ceph.mgr.y.smithi099.stderr:2023-10-08T00:10:40.945+0000 7f3119368180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T00:10:41.044 INFO:tasks.ceph.mgr.y.smithi099.stderr:2023-10-08T00:10:41.044+0000 7f3119368180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T00:10:41.183 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T00:10:41.183+0000 7fa645af8180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T00:10:41.257 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T00:10:41.257+0000 7fa645af8180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T00:10:41.290 INFO:tasks.ceph.mgr.y.smithi099.stderr:2023-10-08T00:10:41.289+0000 7f3119368180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T00:10:41.331 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T00:10:41.331+0000 7fa645af8180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T00:10:41.366 INFO:tasks.ceph.mgr.y.smithi099.stderr:2023-10-08T00:10:41.365+0000 7f3119368180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T00:10:41.463 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T00:10:41.463+0000 7fa645af8180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T00:10:41.612 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T00:10:41.612+0000 7fa645af8180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T00:10:41.770 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T00:10:41.770+0000 7fa645af8180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T00:10:41.846 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T00:10:41.846+0000 7fa645af8180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T00:10:41.924 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T00:10:41.924+0000 7fa645af8180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T00:10:41.999 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T00:10:41.999+0000 7fa645af8180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T00:10:42.197 INFO:tasks.ceph.mgr.y.smithi099.stderr:2023-10-08T00:10:42.196+0000 7f3119368180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T00:10:42.271 INFO:tasks.ceph.mgr.y.smithi099.stderr:2023-10-08T00:10:42.271+0000 7f3119368180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T00:10:42.346 INFO:tasks.ceph.mgr.y.smithi099.stderr:2023-10-08T00:10:42.345+0000 7f3119368180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T00:10:42.479 INFO:tasks.ceph.mgr.y.smithi099.stderr:2023-10-08T00:10:42.478+0000 7f3119368180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T00:10:42.573 INFO:tasks.ceph.mgr.y.smithi099.stderr:2023-10-08T00:10:42.571+0000 7f3119368180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T00:10:42.724 INFO:tasks.ceph.mgr.y.smithi099.stderr:2023-10-08T00:10:42.723+0000 7f3119368180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T00:10:42.803 INFO:tasks.ceph.mgr.y.smithi099.stderr:2023-10-08T00:10:42.802+0000 7f3119368180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T00:10:42.878 INFO:tasks.ceph.mgr.y.smithi099.stderr:2023-10-08T00:10:42.877+0000 7f3119368180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T00:10:42.951 INFO:tasks.ceph.mgr.y.smithi099.stderr:2023-10-08T00:10:42.950+0000 7f3119368180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T00:10:43.033 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T00:10:43.033+0000 7fa5feef9700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-08T00:10:43.033 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T00:10:43.033+0000 7fa5feef9700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-08T00:10:46.786 DEBUG:teuthology.orchestra.run.smithi099:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-08T00:10:46.904 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.903+0000 7fe669ec7700 1 Processor -- start 2023-10-08T00:10:46.904 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.903+0000 7fe669ec7700 1 -- start start 2023-10-08T00:10:46.906 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.904+0000 7fe669ec7700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe6640c9560 0x7fe664157a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:46.907 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.904+0000 7fe669ec7700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fe6640c9930 0x7fe66415c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:46.907 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.904+0000 7fe669ec7700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe664069060 con 0x7fe664069de0 2023-10-08T00:10:46.907 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.904+0000 7fe669ec7700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe664069300 con 0x7fe6640c9560 2023-10-08T00:10:46.907 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.904+0000 7fe669ec7700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6640695e0 con 0x7fe6640c9930 2023-10-08T00:10:46.907 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.904+0000 7fe6637fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe6640c9560 0x7fe664157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:46.907 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.904+0000 7fe6637fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe6640c9560 0x7fe664157a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39010/0 (socket says 172.21.15.99:39010) 2023-10-08T00:10:46.907 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.904+0000 7fe6637fe700 1 -- 172.21.15.99:0/280569874 learned_addr learned my addr 172.21.15.99:0/280569874 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:10:46.908 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.904+0000 7fe6637fe700 1 -- 172.21.15.99:0/280569874 >> v2:172.21.15.191:3300/0 conn(0x7fe6640c9930 msgr2=0x7fe66415c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:46.908 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.904+0000 7fe6637fe700 1 --2- 172.21.15.99:0/280569874 >> v2:172.21.15.191:3300/0 conn(0x7fe6640c9930 0x7fe66415c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:46.908 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.904+0000 7fe6637fe700 1 -- 172.21.15.99:0/280569874 >> v1:172.21.15.191:6789/0 conn(0x7fe664069de0 legacy=0x7fe66415bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:46.908 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.904+0000 7fe6637fe700 1 -- 172.21.15.99:0/280569874 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6640698c0 con 0x7fe6640c9560 2023-10-08T00:10:46.908 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.904+0000 7fe6637fe700 1 --2- 172.21.15.99:0/280569874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6640c9560 0x7fe664157a50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe66415ba40 tx=0x7fe654009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42fbea5845be10be server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:46.908 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.905+0000 7fe6627fc700 1 -- 172.21.15.99:0/280569874 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65400b820 con 0x7fe6640c9560 2023-10-08T00:10:46.908 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.905+0000 7fe6627fc700 1 -- 172.21.15.99:0/280569874 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe65400b980 con 0x7fe6640c9560 2023-10-08T00:10:46.909 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.905+0000 7fe6627fc700 1 -- 172.21.15.99:0/280569874 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65400bd20 con 0x7fe6640c9560 2023-10-08T00:10:46.909 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.905+0000 7fe669ec7700 1 -- 172.21.15.99:0/280569874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6640c9560 msgr2=0x7fe664157a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:46.909 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.905+0000 7fe669ec7700 1 --2- 172.21.15.99:0/280569874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6640c9560 0x7fe664157a50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe66415ba40 tx=0x7fe654009250 comp rx=0 tx=0).stop 2023-10-08T00:10:46.909 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.905+0000 7fe669ec7700 1 -- 172.21.15.99:0/280569874 shutdown_connections 2023-10-08T00:10:46.909 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.905+0000 7fe669ec7700 1 --2- 172.21.15.99:0/280569874 >> v2:172.21.15.191:3300/0 conn(0x7fe6640c9930 0x7fe66415c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:46.909 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.905+0000 7fe669ec7700 1 --2- 172.21.15.99:0/280569874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6640c9560 0x7fe664157a50 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:46.909 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.905+0000 7fe669ec7700 1 -- 172.21.15.99:0/280569874 >> 172.21.15.99:0/280569874 conn(0x7fe664067660 msgr2=0x7fe664067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:46.909 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.905+0000 7fe669ec7700 1 -- 172.21.15.99:0/280569874 shutdown_connections 2023-10-08T00:10:46.910 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.905+0000 7fe669ec7700 1 -- 172.21.15.99:0/280569874 wait complete. 2023-10-08T00:10:46.912 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.906+0000 7fe669ec7700 1 Processor -- start 2023-10-08T00:10:46.913 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.907+0000 7fe669ec7700 1 -- start start 2023-10-08T00:10:46.913 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.907+0000 7fe669ec7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe664069de0 0x7fe6640c2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:46.913 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.907+0000 7fe669ec7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6640c9560 0x7fe6640c6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:46.913 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.907+0000 7fe669ec7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6640c9930 0x7fe6640aa880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:46.913 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.907+0000 7fe669ec7700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe664153890 con 0x7fe6640c9930 2023-10-08T00:10:46.913 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.907+0000 7fe669ec7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6640ab890 con 0x7fe6640c9560 2023-10-08T00:10:46.913 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.907+0000 7fe669ec7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6640ab2d0 con 0x7fe664069de0 2023-10-08T00:10:46.913 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.907+0000 7fe663fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6640c9930 0x7fe6640aa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:46.913 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.907+0000 7fe663fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6640c9930 0x7fe6640aa880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39022/0 (socket says 172.21.15.99:39022) 2023-10-08T00:10:46.914 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.907+0000 7fe663fff700 1 -- 172.21.15.99:0/1043541028 learned_addr learned my addr 172.21.15.99:0/1043541028 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:10:46.914 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.907+0000 7fe662ffd700 1 --2- 172.21.15.99:0/1043541028 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6640c9560 0x7fe6640c6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:46.914 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.907+0000 7fe6637fe700 1 --2- 172.21.15.99:0/1043541028 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe664069de0 0x7fe6640c2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:46.914 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.908+0000 7fe663fff700 1 -- 172.21.15.99:0/1043541028 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe664069de0 msgr2=0x7fe6640c2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:46.914 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.908+0000 7fe663fff700 1 --2- 172.21.15.99:0/1043541028 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe664069de0 0x7fe6640c2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:46.914 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.908+0000 7fe663fff700 1 -- 172.21.15.99:0/1043541028 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6640c9560 msgr2=0x7fe6640c6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:46.914 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.908+0000 7fe663fff700 1 --2- 172.21.15.99:0/1043541028 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6640c9560 0x7fe6640c6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:46.914 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.908+0000 7fe663fff700 1 -- 172.21.15.99:0/1043541028 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6540094f0 con 0x7fe6640c9930 2023-10-08T00:10:46.915 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.908+0000 7fe663fff700 1 --2- 172.21.15.99:0/1043541028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6640c9930 0x7fe6640aa880 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe65c00bc30 tx=0x7fe65c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:46.915 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.908+0000 7fe64b7fe700 1 -- 172.21.15.99:0/1043541028 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65c0092e0 con 0x7fe6640c9930 2023-10-08T00:10:46.915 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.908+0000 7fe64b7fe700 1 -- 172.21.15.99:0/1043541028 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe65c00c630 con 0x7fe6640c9930 2023-10-08T00:10:46.915 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.908+0000 7fe669ec7700 1 -- 172.21.15.99:0/1043541028 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6640ab5b0 con 0x7fe6640c9930 2023-10-08T00:10:46.915 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.908+0000 7fe64b7fe700 1 -- 172.21.15.99:0/1043541028 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65c00ca10 con 0x7fe6640c9930 2023-10-08T00:10:46.915 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.908+0000 7fe669ec7700 1 -- 172.21.15.99:0/1043541028 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6640c3040 con 0x7fe6640c9930 2023-10-08T00:10:46.915 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.909+0000 7fe669ec7700 1 -- 172.21.15.99:0/1043541028 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6640c3e60 con 0x7fe6640c9930 2023-10-08T00:10:46.916 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.909+0000 7fe64b7fe700 1 -- 172.21.15.99:0/1043541028 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe65c012070 con 0x7fe6640c9930 2023-10-08T00:10:46.916 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.910+0000 7fe64b7fe700 1 --2- 172.21.15.99:0/1043541028 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe650066f30 0x7fe6500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:46.916 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.910+0000 7fe64b7fe700 1 -- 172.21.15.99:0/1043541028 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fe65c083cc0 con 0x7fe6640c9930 2023-10-08T00:10:46.916 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.910+0000 7fe6637fe700 1 --2- 172.21.15.99:0/1043541028 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe650066f30 0x7fe6500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:46.916 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.910+0000 7fe6637fe700 1 --2- 172.21.15.99:0/1043541028 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe650066f30 0x7fe6500693e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe66415ba40 tx=0x7fe654016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:46.917 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:46.914+0000 7fe64b7fe700 1 -- 172.21.15.99:0/1043541028 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe65c051280 con 0x7fe6640c9930 2023-10-08T00:10:47.100 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:47.100+0000 7fe669ec7700 1 -- 172.21.15.99:0/1043541028 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe66415c9b0 con 0x7fe6640c9930 2023-10-08T00:10:47.101 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:47.101+0000 7fe64b7fe700 1 -- 172.21.15.99:0/1043541028 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12127 (secure 0 0 0) 0x7fe65c01a090 con 0x7fe6640c9930 2023-10-08T00:10:47.101 INFO:teuthology.misc.health.smithi099.stdout: 2023-10-08T00:10:47.102 INFO:teuthology.misc.health.smithi099.stdout:{"epoch":16,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:10:44.727227+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:10:47.104 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:47.103+0000 7fe669ec7700 1 -- 172.21.15.99:0/1043541028 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe650066f30 msgr2=0x7fe6500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:47.104 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:47.103+0000 7fe669ec7700 1 --2- 172.21.15.99:0/1043541028 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe650066f30 0x7fe6500693e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe66415ba40 tx=0x7fe654016000 comp rx=0 tx=0).stop 2023-10-08T00:10:47.104 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:47.103+0000 7fe669ec7700 1 -- 172.21.15.99:0/1043541028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6640c9930 msgr2=0x7fe6640aa880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:47.104 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:47.103+0000 7fe669ec7700 1 --2- 172.21.15.99:0/1043541028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6640c9930 0x7fe6640aa880 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe65c00bc30 tx=0x7fe65c009840 comp rx=0 tx=0).stop 2023-10-08T00:10:47.104 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:47.103+0000 7fe669ec7700 1 -- 172.21.15.99:0/1043541028 shutdown_connections 2023-10-08T00:10:47.104 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:47.103+0000 7fe669ec7700 1 --2- 172.21.15.99:0/1043541028 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe650066f30 0x7fe6500693e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:47.104 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:47.103+0000 7fe669ec7700 1 --2- 172.21.15.99:0/1043541028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6640c9930 0x7fe6640aa880 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:47.105 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:47.103+0000 7fe669ec7700 1 --2- 172.21.15.99:0/1043541028 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe664069de0 0x7fe6640c2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:47.105 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:47.103+0000 7fe669ec7700 1 --2- 172.21.15.99:0/1043541028 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6640c9560 0x7fe6640c6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:47.105 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:47.103+0000 7fe669ec7700 1 -- 172.21.15.99:0/1043541028 >> 172.21.15.99:0/1043541028 conn(0x7fe664067660 msgr2=0x7fe6641549b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:47.105 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:47.103+0000 7fe669ec7700 1 -- 172.21.15.99:0/1043541028 shutdown_connections 2023-10-08T00:10:47.105 INFO:teuthology.misc.health.smithi099.stderr:2023-10-08T00:10:47.103+0000 7fe669ec7700 1 -- 172.21.15.99:0/1043541028 wait complete. 2023-10-08T00:10:47.115 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-10-08T00:10:47.115 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-10-08T00:10:47.115 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T00:10:47.115 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T00:10:47.118 INFO:tasks.ceph.mds.a:Started 2023-10-08T00:10:47.119 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T00:10:47.119 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T00:10:47.122 INFO:tasks.ceph.mds.c:Started 2023-10-08T00:10:47.122 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T00:10:47.122 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T00:10:47.125 INFO:tasks.ceph.mds.b:Started 2023-10-08T00:10:47.125 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T00:10:47.125 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T00:10:47.128 INFO:tasks.ceph.mds.d:Started 2023-10-08T00:10:47.128 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-10-08T00:10:47.128 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:10:47.191 INFO:tasks.ceph.mds.a.smithi099.stdout:starting mds.a at 2023-10-08T00:10:47.192 INFO:tasks.ceph.mds.c.smithi099.stdout:starting mds.c at 2023-10-08T00:10:47.209 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2023-10-08T00:10:47.209 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2023-10-08T00:10:47.209 INFO:tasks.ceph.mds.b.smithi191.stderr:2023-10-08T00:10:47.209+0000 7f29851cd700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-10-08T00:10:47.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.246+0000 7f4ac11ea700 1 Processor -- start 2023-10-08T00:10:47.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.246+0000 7f4ac11ea700 1 -- start start 2023-10-08T00:10:47.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.247+0000 7f4ac11ea700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4abc15a6e0 0x7f4abc068950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:47.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.247+0000 7f4ac11ea700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4abc06f630 0x7f4abc15aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:47.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.247+0000 7f4ac11ea700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4abc15aff0 0x7f4abc070700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:47.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.247+0000 7f4ac11ea700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4abc15b470 con 0x7f4abc15aff0 2023-10-08T00:10:47.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.247+0000 7f4ac11ea700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4abc067660 con 0x7f4abc06f630 2023-10-08T00:10:47.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.247+0000 7f4ac11ea700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4abc0684c0 con 0x7f4abc15a6e0 2023-10-08T00:10:47.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.247+0000 7f4abb59e700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4abc15aff0 0x7f4abc070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:47.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.247+0000 7f4abb59e700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4abc15aff0 0x7f4abc070700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39046/0 (socket says 172.21.15.99:39046) 2023-10-08T00:10:47.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.247+0000 7f4abb59e700 1 -- 172.21.15.99:0/1737117683 learned_addr learned my addr 172.21.15.99:0/1737117683 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:10:47.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.247+0000 7f4aba59c700 1 --2- 172.21.15.99:0/1737117683 >> v2:172.21.15.191:3300/0 conn(0x7f4abc06f630 0x7f4abc15aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:47.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.247+0000 7f4abad9d700 1 --2- 172.21.15.99:0/1737117683 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4abc15a6e0 0x7f4abc068950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:47.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.247+0000 7f4abb59e700 1 -- 172.21.15.99:0/1737117683 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4abc15a6e0 msgr2=0x7f4abc068950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:47.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.247+0000 7f4abb59e700 1 --2- 172.21.15.99:0/1737117683 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4abc15a6e0 0x7f4abc068950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:47.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.247+0000 7f4abb59e700 1 -- 172.21.15.99:0/1737117683 >> v2:172.21.15.191:3300/0 conn(0x7f4abc06f630 msgr2=0x7f4abc15aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:47.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.248+0000 7f4abb59e700 1 --2- 172.21.15.99:0/1737117683 >> v2:172.21.15.191:3300/0 conn(0x7f4abc06f630 0x7f4abc15aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:47.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.248+0000 7f4abb59e700 1 -- 172.21.15.99:0/1737117683 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4abc0678b0 con 0x7f4abc15aff0 2023-10-08T00:10:47.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.248+0000 7f4aba59c700 1 --2- 172.21.15.99:0/1737117683 >> v2:172.21.15.191:3300/0 conn(0x7f4abc06f630 0x7f4abc15aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:10:47.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.248+0000 7f4abb59e700 1 --2- 172.21.15.99:0/1737117683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4abc15aff0 0x7f4abc070700 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4abc0cb5b0 tx=0x7f4ab4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e967791d6b0b46b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:47.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.248+0000 7f4ab9d9b700 1 -- 172.21.15.99:0/1737117683 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab400e070 con 0x7f4abc15aff0 2023-10-08T00:10:47.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.248+0000 7f4ab9d9b700 1 -- 172.21.15.99:0/1737117683 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ab400ba50 con 0x7f4abc15aff0 2023-10-08T00:10:47.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.248+0000 7f4ab9d9b700 1 -- 172.21.15.99:0/1737117683 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab40092e0 con 0x7f4abc15aff0 2023-10-08T00:10:47.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.248+0000 7f4ac11ea700 1 -- 172.21.15.99:0/1737117683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4abc15aff0 msgr2=0x7f4abc070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:47.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.248+0000 7f4ac11ea700 1 --2- 172.21.15.99:0/1737117683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4abc15aff0 0x7f4abc070700 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4abc0cb5b0 tx=0x7f4ab4009840 comp rx=0 tx=0).stop 2023-10-08T00:10:47.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.248+0000 7f4ac11ea700 1 -- 172.21.15.99:0/1737117683 shutdown_connections 2023-10-08T00:10:47.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.248+0000 7f4ac11ea700 1 --2- 172.21.15.99:0/1737117683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4abc15aff0 0x7f4abc070700 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:47.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.248+0000 7f4ac11ea700 1 --2- 172.21.15.99:0/1737117683 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4abc15a6e0 0x7f4abc068950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:47.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.248+0000 7f4ac11ea700 1 --2- 172.21.15.99:0/1737117683 >> v2:172.21.15.191:3300/0 conn(0x7f4abc06f630 0x7f4abc15aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:47.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.248+0000 7f4ac11ea700 1 -- 172.21.15.99:0/1737117683 >> 172.21.15.99:0/1737117683 conn(0x7f4abc06a250 msgr2=0x7f4abc06a650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:47.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.248+0000 7f4ac11ea700 1 -- 172.21.15.99:0/1737117683 shutdown_connections 2023-10-08T00:10:47.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.248+0000 7f4ac11ea700 1 -- 172.21.15.99:0/1737117683 wait complete. 2023-10-08T00:10:47.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.250+0000 7f4ac11ea700 1 Processor -- start 2023-10-08T00:10:47.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.250+0000 7f4ac11ea700 1 -- start start 2023-10-08T00:10:47.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.250+0000 7f4ac11ea700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4abc06f630 0x7f4abc0af5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:47.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.250+0000 7f4ac11ea700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4abc15a6e0 0x7f4abc15fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:47.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.250+0000 7f4ac11ea700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4abc15aff0 0x7f4abc15fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:47.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.250+0000 7f4ac11ea700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4abc0b0070 con 0x7f4abc15a6e0 2023-10-08T00:10:47.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.250+0000 7f4ac11ea700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4abc0ad5a0 con 0x7f4abc15aff0 2023-10-08T00:10:47.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.250+0000 7f4ac11ea700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4abc0ad880 con 0x7f4abc06f630 2023-10-08T00:10:47.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.250+0000 7f4aba59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4abc15a6e0 0x7f4abc15fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:47.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.251+0000 7f4aba59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4abc15a6e0 0x7f4abc15fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39060/0 (socket says 172.21.15.99:39060) 2023-10-08T00:10:47.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.251+0000 7f4aba59c700 1 -- 172.21.15.99:0/3659453751 learned_addr learned my addr 172.21.15.99:0/3659453751 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:10:47.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.251+0000 7f4abb59e700 1 --2- 172.21.15.99:0/3659453751 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4abc15aff0 0x7f4abc15fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:47.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.251+0000 7f4abad9d700 1 --2- 172.21.15.99:0/3659453751 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4abc06f630 0x7f4abc0af5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:47.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.251+0000 7f4aba59c700 1 -- 172.21.15.99:0/3659453751 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4abc06f630 msgr2=0x7f4abc0af5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:47.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.251+0000 7f4aba59c700 1 --2- 172.21.15.99:0/3659453751 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4abc06f630 0x7f4abc0af5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:47.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.251+0000 7f4aba59c700 1 -- 172.21.15.99:0/3659453751 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4abc15aff0 msgr2=0x7f4abc15fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:47.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.251+0000 7f4aba59c700 1 --2- 172.21.15.99:0/3659453751 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4abc15aff0 0x7f4abc15fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:47.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.251+0000 7f4aba59c700 1 -- 172.21.15.99:0/3659453751 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab4009ae0 con 0x7f4abc15a6e0 2023-10-08T00:10:47.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.251+0000 7f4abad9d700 1 --2- 172.21.15.99:0/3659453751 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4abc06f630 0x7f4abc0af5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:10:47.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.251+0000 7f4aba59c700 1 --2- 172.21.15.99:0/3659453751 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4abc15a6e0 0x7f4abc15fab0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4ab000c800 tx=0x7f4ab0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:47.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.251+0000 7f4aa2ffd700 1 -- 172.21.15.99:0/3659453751 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab0011600 con 0x7f4abc15a6e0 2023-10-08T00:10:47.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.251+0000 7f4aa2ffd700 1 -- 172.21.15.99:0/3659453751 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ab0011760 con 0x7f4abc15a6e0 2023-10-08T00:10:47.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.251+0000 7f4ac11ea700 1 -- 172.21.15.99:0/3659453751 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4abc0adb60 con 0x7f4abc15a6e0 2023-10-08T00:10:47.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.251+0000 7f4aa2ffd700 1 -- 172.21.15.99:0/3659453751 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab0011ae0 con 0x7f4abc15a6e0 2023-10-08T00:10:47.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.251+0000 7f4ac11ea700 1 -- 172.21.15.99:0/3659453751 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4abc0ae080 con 0x7f4abc15a6e0 2023-10-08T00:10:47.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.252+0000 7f4ac11ea700 1 -- 172.21.15.99:0/3659453751 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4abc0add50 con 0x7f4abc15a6e0 2023-10-08T00:10:47.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.252+0000 7f4aa2ffd700 1 -- 172.21.15.99:0/3659453751 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4ab0021350 con 0x7f4abc15a6e0 2023-10-08T00:10:47.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.253+0000 7f4aa2ffd700 1 --2- 172.21.15.99:0/3659453751 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4aa8066f30 0x7f4aa80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:47.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.253+0000 7f4aa2ffd700 1 -- 172.21.15.99:0/3659453751 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f4ab0084050 con 0x7f4abc15a6e0 2023-10-08T00:10:47.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.253+0000 7f4abad9d700 1 --2- 172.21.15.99:0/3659453751 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4aa8066f30 0x7f4aa80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:47.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.253+0000 7f4abad9d700 1 --2- 172.21.15.99:0/3659453751 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4aa8066f30 0x7f4aa80693e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4aac00ab50 tx=0x7f4aac008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:47.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.257+0000 7f4aa2ffd700 1 -- 172.21.15.99:0/3659453751 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4ab00550a0 con 0x7f4abc15a6e0 2023-10-08T00:10:47.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.441+0000 7f4ac11ea700 1 -- 172.21.15.99:0/3659453751 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4abc04c3c0 con 0x7f4abc15a6e0 2023-10-08T00:10:47.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.442+0000 7f4aa2ffd700 1 -- 172.21.15.99:0/3659453751 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12127 (secure 0 0 0) 0x7f4ab0054c30 con 0x7f4abc15a6e0 2023-10-08T00:10:47.443 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:10:47.443 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":16,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:10:44.727227+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:10:47.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.444+0000 7f4ac11ea700 1 -- 172.21.15.99:0/3659453751 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4aa8066f30 msgr2=0x7f4aa80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:47.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.444+0000 7f4ac11ea700 1 --2- 172.21.15.99:0/3659453751 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4aa8066f30 0x7f4aa80693e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4aac00ab50 tx=0x7f4aac008040 comp rx=0 tx=0).stop 2023-10-08T00:10:47.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.444+0000 7f4ac11ea700 1 -- 172.21.15.99:0/3659453751 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4abc15a6e0 msgr2=0x7f4abc15fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:47.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.444+0000 7f4ac11ea700 1 --2- 172.21.15.99:0/3659453751 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4abc15a6e0 0x7f4abc15fab0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4ab000c800 tx=0x7f4ab0008040 comp rx=0 tx=0).stop 2023-10-08T00:10:47.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.444+0000 7f4ac11ea700 1 -- 172.21.15.99:0/3659453751 shutdown_connections 2023-10-08T00:10:47.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.444+0000 7f4ac11ea700 1 --2- 172.21.15.99:0/3659453751 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4aa8066f30 0x7f4aa80693e0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:47.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.444+0000 7f4ac11ea700 1 --2- 172.21.15.99:0/3659453751 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4abc15aff0 0x7f4abc15fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:47.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.444+0000 7f4ac11ea700 1 --2- 172.21.15.99:0/3659453751 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4abc06f630 0x7f4abc0af5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:47.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.444+0000 7f4ac11ea700 1 --2- 172.21.15.99:0/3659453751 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4abc15a6e0 0x7f4abc15fab0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:47.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.444+0000 7f4ac11ea700 1 -- 172.21.15.99:0/3659453751 >> 172.21.15.99:0/3659453751 conn(0x7f4abc06a250 msgr2=0x7f4abc0cbde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:47.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.444+0000 7f4ac11ea700 1 -- 172.21.15.99:0/3659453751 shutdown_connections 2023-10-08T00:10:47.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.444+0000 7f4ac11ea700 1 -- 172.21.15.99:0/3659453751 wait complete. 2023-10-08T00:10:47.457 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:10:42.063344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-08T00:10:47.457 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T00:10:47.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.573+0000 7fb4ebcd3700 1 Processor -- start 2023-10-08T00:10:47.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.573+0000 7fb4ebcd3700 1 -- start start 2023-10-08T00:10:47.577 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.574+0000 7fb4ebcd3700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb4e4159ec0 0x7fb4e40af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:47.577 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.574+0000 7fb4ebcd3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb4e415a290 0x7fb4e40afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:47.577 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.574+0000 7fb4ebcd3700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb4e406a5a0 0x7fb4e406aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:47.578 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.574+0000 7fb4ebcd3700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4e406af60 con 0x7fb4e406a5a0 2023-10-08T00:10:47.578 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.574+0000 7fb4ebcd3700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4e40b0790 con 0x7fb4e4159ec0 2023-10-08T00:10:47.578 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.574+0000 7fb4ebcd3700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e40ce1f0 con 0x7fb4e415a290 2023-10-08T00:10:47.578 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.574+0000 7fb4ea270700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb4e406a5a0 0x7fb4e406aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:47.578 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.574+0000 7fb4ea270700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb4e406a5a0 0x7fb4e406aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39070/0 (socket says 172.21.15.99:39070) 2023-10-08T00:10:47.578 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.574+0000 7fb4ea270700 1 -- 172.21.15.99:0/2487945466 learned_addr learned my addr 172.21.15.99:0/2487945466 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:10:47.578 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.574+0000 7fb4e9a6f700 1 --2- 172.21.15.99:0/2487945466 >> v2:172.21.15.191:3300/0 conn(0x7fb4e4159ec0 0x7fb4e40af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:47.578 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.574+0000 7fb4e926e700 1 --2- 172.21.15.99:0/2487945466 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb4e415a290 0x7fb4e40afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:47.578 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.574+0000 7fb4ea270700 1 -- 172.21.15.99:0/2487945466 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb4e415a290 msgr2=0x7fb4e40afe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:47.578 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.574+0000 7fb4ea270700 1 --2- 172.21.15.99:0/2487945466 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb4e415a290 0x7fb4e40afe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:47.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.574+0000 7fb4ea270700 1 -- 172.21.15.99:0/2487945466 >> v2:172.21.15.191:3300/0 conn(0x7fb4e4159ec0 msgr2=0x7fb4e40af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:47.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.574+0000 7fb4ea270700 1 --2- 172.21.15.99:0/2487945466 >> v2:172.21.15.191:3300/0 conn(0x7fb4e4159ec0 0x7fb4e40af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:47.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.574+0000 7fb4ea270700 1 -- 172.21.15.99:0/2487945466 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e40b0b20 con 0x7fb4e406a5a0 2023-10-08T00:10:47.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.574+0000 7fb4e9a6f700 1 --2- 172.21.15.99:0/2487945466 >> v2:172.21.15.191:3300/0 conn(0x7fb4e4159ec0 0x7fb4e40af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:10:47.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.574+0000 7fb4e926e700 1 --2- 172.21.15.99:0/2487945466 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb4e415a290 0x7fb4e40afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:10:47.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.575+0000 7fb4ea270700 1 --2- 172.21.15.99:0/2487945466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4e406a5a0 0x7fb4e406aa20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb4e40898e0 tx=0x7fb4dc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dc017c505b266c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:47.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.575+0000 7fb4e8a6d700 1 -- 172.21.15.99:0/2487945466 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc00b860 con 0x7fb4e406a5a0 2023-10-08T00:10:47.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.575+0000 7fb4e8a6d700 1 -- 172.21.15.99:0/2487945466 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4dc00b9c0 con 0x7fb4e406a5a0 2023-10-08T00:10:47.580 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.575+0000 7fb4e8a6d700 1 -- 172.21.15.99:0/2487945466 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc00bd60 con 0x7fb4e406a5a0 2023-10-08T00:10:47.580 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.575+0000 7fb4ebcd3700 1 -- 172.21.15.99:0/2487945466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4e406a5a0 msgr2=0x7fb4e406aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:47.580 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.575+0000 7fb4ebcd3700 1 --2- 172.21.15.99:0/2487945466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4e406a5a0 0x7fb4e406aa20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb4e40898e0 tx=0x7fb4dc009250 comp rx=0 tx=0).stop 2023-10-08T00:10:47.580 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.575+0000 7fb4ebcd3700 1 -- 172.21.15.99:0/2487945466 shutdown_connections 2023-10-08T00:10:47.581 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.575+0000 7fb4ebcd3700 1 --2- 172.21.15.99:0/2487945466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4e406a5a0 0x7fb4e406aa20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:47.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.575+0000 7fb4ebcd3700 1 --2- 172.21.15.99:0/2487945466 >> v2:172.21.15.191:3300/0 conn(0x7fb4e4159ec0 0x7fb4e40af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:47.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.575+0000 7fb4ebcd3700 1 --2- 172.21.15.99:0/2487945466 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb4e415a290 0x7fb4e40afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:47.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.575+0000 7fb4ebcd3700 1 -- 172.21.15.99:0/2487945466 >> 172.21.15.99:0/2487945466 conn(0x7fb4e40d1880 msgr2=0x7fb4e40cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:47.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.575+0000 7fb4ebcd3700 1 -- 172.21.15.99:0/2487945466 shutdown_connections 2023-10-08T00:10:47.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.575+0000 7fb4ebcd3700 1 -- 172.21.15.99:0/2487945466 wait complete. 2023-10-08T00:10:47.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.576+0000 7fb4ebcd3700 1 Processor -- start 2023-10-08T00:10:47.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.576+0000 7fb4ebcd3700 1 -- start start 2023-10-08T00:10:47.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.577+0000 7fb4ebcd3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb4e406a5a0 0x7fb4e415a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:47.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.577+0000 7fb4ebcd3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4e4159ec0 0x7fb4e415b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:47.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.577+0000 7fb4ebcd3700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb4e415a290 0x7fb4e4150600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:47.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.577+0000 7fb4ebcd3700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e415add0 con 0x7fb4e4159ec0 2023-10-08T00:10:47.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.577+0000 7fb4ebcd3700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e40b24b0 con 0x7fb4e415a290 2023-10-08T00:10:47.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.577+0000 7fb4ebcd3700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e40b18a0 con 0x7fb4e406a5a0 2023-10-08T00:10:47.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.577+0000 7fb4e926e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4e4159ec0 0x7fb4e415b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:47.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.577+0000 7fb4e926e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4e4159ec0 0x7fb4e415b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39074/0 (socket says 172.21.15.99:39074) 2023-10-08T00:10:47.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.577+0000 7fb4e926e700 1 -- 172.21.15.99:0/1995852924 learned_addr learned my addr 172.21.15.99:0/1995852924 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:10:47.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.577+0000 7fb4e9a6f700 1 --2- 172.21.15.99:0/1995852924 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb4e406a5a0 0x7fb4e415a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:47.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.577+0000 7fb4ea270700 1 --2- 172.21.15.99:0/1995852924 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb4e415a290 0x7fb4e4150600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:47.586 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.578+0000 7fb4e926e700 1 -- 172.21.15.99:0/1995852924 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb4e406a5a0 msgr2=0x7fb4e415a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:47.586 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.578+0000 7fb4e926e700 1 --2- 172.21.15.99:0/1995852924 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb4e406a5a0 0x7fb4e415a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:47.586 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.578+0000 7fb4e926e700 1 -- 172.21.15.99:0/1995852924 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb4e415a290 msgr2=0x7fb4e4150600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:47.586 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.578+0000 7fb4e926e700 1 --2- 172.21.15.99:0/1995852924 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb4e415a290 0x7fb4e4150600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:47.586 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.578+0000 7fb4e926e700 1 -- 172.21.15.99:0/1995852924 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4dc0094f0 con 0x7fb4e4159ec0 2023-10-08T00:10:47.586 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.578+0000 7fb4e926e700 1 --2- 172.21.15.99:0/1995852924 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4e4159ec0 0x7fb4e415b620 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb4d400c850 tx=0x7fb4d4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:47.586 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.578+0000 7fb4d9ffb700 1 -- 172.21.15.99:0/1995852924 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d400a2e0 con 0x7fb4e4159ec0 2023-10-08T00:10:47.586 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.578+0000 7fb4d9ffb700 1 -- 172.21.15.99:0/1995852924 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4d4011600 con 0x7fb4e4159ec0 2023-10-08T00:10:47.586 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.578+0000 7fb4ebcd3700 1 -- 172.21.15.99:0/1995852924 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4e40b1b80 con 0x7fb4e4159ec0 2023-10-08T00:10:47.586 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.578+0000 7fb4d9ffb700 1 -- 172.21.15.99:0/1995852924 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d40119a0 con 0x7fb4e4159ec0 2023-10-08T00:10:47.586 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.578+0000 7fb4ebcd3700 1 -- 172.21.15.99:0/1995852924 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4e40b2140 con 0x7fb4e4159ec0 2023-10-08T00:10:47.586 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.579+0000 7fb4ebcd3700 1 -- 172.21.15.99:0/1995852924 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4e40b2620 con 0x7fb4e4159ec0 2023-10-08T00:10:47.587 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.579+0000 7fb4d9ffb700 1 -- 172.21.15.99:0/1995852924 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb4d4018350 con 0x7fb4e4159ec0 2023-10-08T00:10:47.587 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.580+0000 7fb4d9ffb700 1 --2- 172.21.15.99:0/1995852924 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb4c8066f60 0x7fb4c8069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:47.587 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.580+0000 7fb4d9ffb700 1 -- 172.21.15.99:0/1995852924 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fb4d4083f40 con 0x7fb4e4159ec0 2023-10-08T00:10:47.587 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.580+0000 7fb4e9a6f700 1 --2- 172.21.15.99:0/1995852924 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb4c8066f60 0x7fb4c8069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:47.587 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.580+0000 7fb4e9a6f700 1 --2- 172.21.15.99:0/1995852924 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb4c8066f60 0x7fb4c8069410 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0007740 tx=0x7fb4e00099d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:47.587 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.584+0000 7fb4d9ffb700 1 -- 172.21.15.99:0/1995852924 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb4d4051500 con 0x7fb4e4159ec0 2023-10-08T00:10:47.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.766+0000 7fb4ebcd3700 1 -- 172.21.15.99:0/1995852924 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb4e404c3c0 con 0x7fb4e4159ec0 2023-10-08T00:10:47.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.766+0000 7fb4d9ffb700 1 -- 172.21.15.99:0/1995852924 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb4d4054b20 con 0x7fb4e4159ec0 2023-10-08T00:10:47.767 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2023-10-08T00:10:47.769 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.768+0000 7fb4ebcd3700 1 -- 172.21.15.99:0/1995852924 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb4c8066f60 msgr2=0x7fb4c8069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:47.769 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.768+0000 7fb4ebcd3700 1 --2- 172.21.15.99:0/1995852924 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb4c8066f60 0x7fb4c8069410 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0007740 tx=0x7fb4e00099d0 comp rx=0 tx=0).stop 2023-10-08T00:10:47.769 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.768+0000 7fb4ebcd3700 1 -- 172.21.15.99:0/1995852924 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4e4159ec0 msgr2=0x7fb4e415b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:47.769 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.768+0000 7fb4ebcd3700 1 --2- 172.21.15.99:0/1995852924 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4e4159ec0 0x7fb4e415b620 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb4d400c850 tx=0x7fb4d4008040 comp rx=0 tx=0).stop 2023-10-08T00:10:47.769 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.768+0000 7fb4ebcd3700 1 -- 172.21.15.99:0/1995852924 shutdown_connections 2023-10-08T00:10:47.769 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.768+0000 7fb4ebcd3700 1 --2- 172.21.15.99:0/1995852924 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb4c8066f60 0x7fb4c8069410 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:47.769 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.768+0000 7fb4ebcd3700 1 --2- 172.21.15.99:0/1995852924 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb4e415a290 0x7fb4e4150600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:47.769 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.768+0000 7fb4ebcd3700 1 --2- 172.21.15.99:0/1995852924 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb4e406a5a0 0x7fb4e415a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:47.769 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.768+0000 7fb4ebcd3700 1 --2- 172.21.15.99:0/1995852924 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4e4159ec0 0x7fb4e415b620 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:47.770 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.768+0000 7fb4ebcd3700 1 -- 172.21.15.99:0/1995852924 >> 172.21.15.99:0/1995852924 conn(0x7fb4e40d1880 msgr2=0x7fb4e40c7030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:47.770 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.768+0000 7fb4ebcd3700 1 -- 172.21.15.99:0/1995852924 shutdown_connections 2023-10-08T00:10:47.770 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.768+0000 7fb4ebcd3700 1 -- 172.21.15.99:0/1995852924 wait complete. 2023-10-08T00:10:47.780 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:10:47.894 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.893+0000 7ff0da1be700 1 Processor -- start 2023-10-08T00:10:47.894 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.893+0000 7ff0da1be700 1 -- start start 2023-10-08T00:10:47.898 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.894+0000 7ff0da1be700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0d4067d70 0x7ff0d40cc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:47.898 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.894+0000 7ff0da1be700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0d40cb8b0 con 0x7ff0d40b0f30 2023-10-08T00:10:47.898 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.894+0000 7ff0da1be700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0d4068f80 con 0x7ff0d40b1300 2023-10-08T00:10:47.898 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.894+0000 7ff0da1be700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d4069260 con 0x7ff0d4067d70 2023-10-08T00:10:47.898 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.894+0000 7ff0d37fe700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7ff0d40b0f30 0x7ff0d40cbd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:38408/0 (socket says 172.21.15.99:38408) 2023-10-08T00:10:47.898 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.894+0000 7ff0d37fe700 1 -- 172.21.15.99:0/2264039162 learned_addr learned my addr 172.21.15.99:0/2264039162 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:10:47.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.894+0000 7ff0d27fc700 1 -- 172.21.15.99:0/2264039162 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1712013672 0 0) 0x7ff0d40cb8b0 con 0x7ff0d40b0f30 2023-10-08T00:10:47.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.894+0000 7ff0d27fc700 1 -- 172.21.15.99:0/2264039162 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0bc003570 con 0x7ff0d40b0f30 2023-10-08T00:10:47.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.894+0000 7ff0d3fff700 1 --2- 172.21.15.99:0/2264039162 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0d4067d70 0x7ff0d40cc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:47.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.895+0000 7ff0d27fc700 1 -- 172.21.15.99:0/2264039162 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 666274054 0 0) 0x7ff0bc003570 con 0x7ff0d40b0f30 2023-10-08T00:10:47.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.895+0000 7ff0d27fc700 1 -- 172.21.15.99:0/2264039162 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0d4067d70 msgr2=0x7ff0d40cc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:47.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.895+0000 7ff0d27fc700 1 --2- 172.21.15.99:0/2264039162 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0d4067d70 0x7ff0d40cc750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:47.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.895+0000 7ff0d27fc700 1 -- 172.21.15.99:0/2264039162 >> v1:172.21.15.191:6789/0 conn(0x7ff0d40b1300 legacy=0x7ff0d4067660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:47.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.895+0000 7ff0d27fc700 1 -- 172.21.15.99:0/2264039162 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d4069540 con 0x7ff0d40b0f30 2023-10-08T00:10:47.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.895+0000 7ff0d27fc700 1 -- 172.21.15.99:0/2264039162 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7ff0c40017f0 con 0x7ff0d40b0f30 2023-10-08T00:10:47.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.895+0000 7ff0d27fc700 1 -- 172.21.15.99:0/2264039162 >> v1:172.21.15.99:6789/0 conn(0x7ff0d40b0f30 legacy=0x7ff0d40cbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:47.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.895+0000 7ff0d27fc700 1 --2- 172.21.15.99:0/2264039162 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0d4067d70 0x7ff0bc003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:47.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.895+0000 7ff0d27fc700 1 --2- 172.21.15.99:0/2264039162 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0bc0040c0 0x7ff0bc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:47.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.895+0000 7ff0d27fc700 1 --2- 172.21.15.99:0/2264039162 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0bc004da0 0x7ff0bc0051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:47.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.895+0000 7ff0d27fc700 1 -- 172.21.15.99:0/2264039162 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d4069260 con 0x7ff0bc0040c0 2023-10-08T00:10:47.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.895+0000 7ff0d27fc700 1 -- 172.21.15.99:0/2264039162 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0bc00bc70 con 0x7ff0bc004da0 2023-10-08T00:10:47.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.895+0000 7ff0d27fc700 1 -- 172.21.15.99:0/2264039162 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0bc00be30 con 0x7ff0d4067d70 2023-10-08T00:10:47.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.895+0000 7ff0d2ffd700 1 --2- 172.21.15.99:0/2264039162 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0bc0040c0 0x7ff0bc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:47.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.895+0000 7ff0d37fe700 1 --2- 172.21.15.99:0/2264039162 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0d4067d70 0x7ff0bc003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:47.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.895+0000 7ff0d3fff700 1 --2- 172.21.15.99:0/2264039162 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0bc004da0 0x7ff0bc0051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:47.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.895+0000 7ff0d2ffd700 1 -- 172.21.15.99:0/2264039162 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0d4067d70 msgr2=0x7ff0bc003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:47.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.895+0000 7ff0d2ffd700 1 --2- 172.21.15.99:0/2264039162 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0d4067d70 0x7ff0bc003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:47.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.895+0000 7ff0d2ffd700 1 -- 172.21.15.99:0/2264039162 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0bc004da0 msgr2=0x7ff0bc0051f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:47.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.895+0000 7ff0d2ffd700 1 --2- 172.21.15.99:0/2264039162 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0bc004da0 0x7ff0bc0051f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:47.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.895+0000 7ff0d2ffd700 1 -- 172.21.15.99:0/2264039162 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0bc003570 con 0x7ff0bc0040c0 2023-10-08T00:10:47.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.895+0000 7ff0d2ffd700 1 --2- 172.21.15.99:0/2264039162 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0bc0040c0 0x7ff0bc004810 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8001880 tx=0x7ff0c8007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b7607aa760c2d42 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:47.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.895+0000 7ff0d27fc700 1 -- 172.21.15.99:0/2264039162 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c8012420 con 0x7ff0bc0040c0 2023-10-08T00:10:47.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.895+0000 7ff0d27fc700 1 -- 172.21.15.99:0/2264039162 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c8012580 con 0x7ff0bc0040c0 2023-10-08T00:10:47.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.895+0000 7ff0d27fc700 1 -- 172.21.15.99:0/2264039162 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c8012920 con 0x7ff0bc0040c0 2023-10-08T00:10:47.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.896+0000 7ff0da1be700 1 -- 172.21.15.99:0/2264039162 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0bc0040c0 msgr2=0x7ff0bc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:47.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.896+0000 7ff0da1be700 1 --2- 172.21.15.99:0/2264039162 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0bc0040c0 0x7ff0bc004810 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8001880 tx=0x7ff0c8007d30 comp rx=0 tx=0).stop 2023-10-08T00:10:47.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.896+0000 7ff0d3fff700 1 -- 172.21.15.99:0/2264039162 reap_dead start 2023-10-08T00:10:47.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.896+0000 7ff0da1be700 1 -- 172.21.15.99:0/2264039162 shutdown_connections 2023-10-08T00:10:47.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.896+0000 7ff0da1be700 1 -- 172.21.15.99:0/2264039162 >> 172.21.15.99:0/2264039162 conn(0x7ff0d406fdc0 msgr2=0x7ff0d406bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:47.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.896+0000 7ff0da1be700 1 -- 172.21.15.99:0/2264039162 shutdown_connections 2023-10-08T00:10:47.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.896+0000 7ff0da1be700 1 -- 172.21.15.99:0/2264039162 wait complete. 2023-10-08T00:10:47.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.897+0000 7ff0da1be700 1 Processor -- start 2023-10-08T00:10:47.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.897+0000 7ff0da1be700 1 -- start start 2023-10-08T00:10:47.904 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.898+0000 7ff0da1be700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0bc0040c0 0x7ff0d40abef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:47.904 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.898+0000 7ff0da1be700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0d40ca040 0x7ff0d40adc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:47.904 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.898+0000 7ff0da1be700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0d40ae1d0 0x7ff0d41501e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:47.904 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.898+0000 7ff0da1be700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d40caa20 con 0x7ff0bc0040c0 2023-10-08T00:10:47.904 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.898+0000 7ff0da1be700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d40c7f20 con 0x7ff0d40ae1d0 2023-10-08T00:10:47.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.898+0000 7ff0da1be700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d40c8200 con 0x7ff0d40ca040 2023-10-08T00:10:47.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.898+0000 7ff0d37fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0bc0040c0 0x7ff0d40abef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:47.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.898+0000 7ff0d37fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0bc0040c0 0x7ff0d40abef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39098/0 (socket says 172.21.15.99:39098) 2023-10-08T00:10:47.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.898+0000 7ff0d37fe700 1 -- 172.21.15.99:0/867648422 learned_addr learned my addr 172.21.15.99:0/867648422 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:10:47.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.898+0000 7ff0d2ffd700 1 --2- 172.21.15.99:0/867648422 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0d40ca040 0x7ff0d40adc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:47.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.898+0000 7ff0d3fff700 1 --2- 172.21.15.99:0/867648422 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0d40ae1d0 0x7ff0d41501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:47.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.898+0000 7ff0d37fe700 1 -- 172.21.15.99:0/867648422 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0d40ca040 msgr2=0x7ff0d40adc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:47.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.898+0000 7ff0d37fe700 1 --2- 172.21.15.99:0/867648422 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0d40ca040 0x7ff0d40adc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:47.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.898+0000 7ff0d37fe700 1 -- 172.21.15.99:0/867648422 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0d40ae1d0 msgr2=0x7ff0d41501e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:47.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.898+0000 7ff0d37fe700 1 --2- 172.21.15.99:0/867648422 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0d40ae1d0 0x7ff0d41501e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:47.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.898+0000 7ff0d37fe700 1 -- 172.21.15.99:0/867648422 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d40c84e0 con 0x7ff0bc0040c0 2023-10-08T00:10:47.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.898+0000 7ff0d37fe700 1 --2- 172.21.15.99:0/867648422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0bc0040c0 0x7ff0d40abef0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4007ac0 tx=0x7ff0c40021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:47.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.899+0000 7ff0bb7fe700 1 -- 172.21.15.99:0/867648422 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c400acc0 con 0x7ff0bc0040c0 2023-10-08T00:10:47.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.899+0000 7ff0bb7fe700 1 -- 172.21.15.99:0/867648422 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c400ae60 con 0x7ff0bc0040c0 2023-10-08T00:10:47.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.899+0000 7ff0bb7fe700 1 -- 172.21.15.99:0/867648422 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4013640 con 0x7ff0bc0040c0 2023-10-08T00:10:47.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.899+0000 7ff0da1be700 1 -- 172.21.15.99:0/867648422 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0d40c87c0 con 0x7ff0bc0040c0 2023-10-08T00:10:47.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.899+0000 7ff0da1be700 1 -- 172.21.15.99:0/867648422 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0d40ac430 con 0x7ff0bc0040c0 2023-10-08T00:10:47.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.899+0000 7ff0da1be700 1 -- 172.21.15.99:0/867648422 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0d40ac620 con 0x7ff0bc0040c0 2023-10-08T00:10:47.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.900+0000 7ff0bb7fe700 1 -- 172.21.15.99:0/867648422 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff0c4013a30 con 0x7ff0bc0040c0 2023-10-08T00:10:47.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.900+0000 7ff0bb7fe700 1 --2- 172.21.15.99:0/867648422 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff0c0066f30 0x7ff0c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:47.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.900+0000 7ff0bb7fe700 1 -- 172.21.15.99:0/867648422 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7ff0c4084e00 con 0x7ff0bc0040c0 2023-10-08T00:10:47.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.900+0000 7ff0d2ffd700 1 --2- 172.21.15.99:0/867648422 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff0c0066f30 0x7ff0c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:47.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.901+0000 7ff0d2ffd700 1 --2- 172.21.15.99:0/867648422 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff0c0066f30 0x7ff0c00693e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8008fd0 tx=0x7ff0c8015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:47.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:47.904+0000 7ff0bb7fe700 1 -- 172.21.15.99:0/867648422 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff0c40523c0 con 0x7ff0bc0040c0 2023-10-08T00:10:48.131 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:48.131+0000 7ff0da1be700 1 -- 172.21.15.99:0/867648422 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff0d415b230 con 0x7ff0bc0040c0 2023-10-08T00:10:48.132 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:48.131+0000 7ff0bb7fe700 1 -- 172.21.15.99:0/867648422 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1256 (secure 0 0 0) 0x7ff0c40559e0 con 0x7ff0bc0040c0 2023-10-08T00:10:48.132 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:10:48.132 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4239,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2194110899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2194110899},{"type":"v1","addr":"172.21.15.99:6835","nonce":2194110899}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-10-08T00:10:48.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:48.133+0000 7ff0da1be700 1 -- 172.21.15.99:0/867648422 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff0c0066f30 msgr2=0x7ff0c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:48.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:48.133+0000 7ff0da1be700 1 --2- 172.21.15.99:0/867648422 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff0c0066f30 0x7ff0c00693e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8008fd0 tx=0x7ff0c8015000 comp rx=0 tx=0).stop 2023-10-08T00:10:48.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:48.133+0000 7ff0da1be700 1 -- 172.21.15.99:0/867648422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0bc0040c0 msgr2=0x7ff0d40abef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:48.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:48.133+0000 7ff0da1be700 1 --2- 172.21.15.99:0/867648422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0bc0040c0 0x7ff0d40abef0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4007ac0 tx=0x7ff0c40021d0 comp rx=0 tx=0).stop 2023-10-08T00:10:48.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:48.133+0000 7ff0da1be700 1 -- 172.21.15.99:0/867648422 shutdown_connections 2023-10-08T00:10:48.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:48.133+0000 7ff0da1be700 1 --2- 172.21.15.99:0/867648422 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff0c0066f30 0x7ff0c00693e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:48.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:48.133+0000 7ff0da1be700 1 --2- 172.21.15.99:0/867648422 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0d40ae1d0 0x7ff0d41501e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:48.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:48.133+0000 7ff0da1be700 1 --2- 172.21.15.99:0/867648422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0bc0040c0 0x7ff0d40abef0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:48.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:48.133+0000 7ff0da1be700 1 --2- 172.21.15.99:0/867648422 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0d40ca040 0x7ff0d40adc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:48.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:48.133+0000 7ff0da1be700 1 -- 172.21.15.99:0/867648422 >> 172.21.15.99:0/867648422 conn(0x7ff0d406fdc0 msgr2=0x7ff0d40cbd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:48.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:48.133+0000 7ff0da1be700 1 -- 172.21.15.99:0/867648422 shutdown_connections 2023-10-08T00:10:48.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:48.133+0000 7ff0da1be700 1 -- 172.21.15.99:0/867648422 wait complete. 2023-10-08T00:10:48.135 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 2 2023-10-08T00:10:50.149 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:10:50.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.274+0000 7fd64359e700 1 Processor -- start 2023-10-08T00:10:50.275 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.274+0000 7fd64359e700 1 -- start start 2023-10-08T00:10:50.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.275+0000 7fd64359e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd644143520 0x7fd6440cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:50.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.275+0000 7fd64359e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd6441438f0 0x7fd64406a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:50.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.275+0000 7fd64359e700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6440d0870 con 0x7fd6440ccf00 2023-10-08T00:10:50.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.275+0000 7fd64359e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6440cdfb0 con 0x7fd6441438f0 2023-10-08T00:10:50.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.275+0000 7fd64359e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6440cfc60 con 0x7fd644143520 2023-10-08T00:10:50.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.275+0000 7fd642d9d700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fd6440ccf00 0x7fd64415da60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:38410/0 (socket says 172.21.15.99:38410) 2023-10-08T00:10:50.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.275+0000 7fd642d9d700 1 -- 172.21.15.99:0/1478021759 learned_addr learned my addr 172.21.15.99:0/1478021759 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:10:50.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.275+0000 7fd64259c700 1 --2- 172.21.15.99:0/1478021759 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd644143520 0x7fd6440cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:50.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.275+0000 7fd641d9b700 1 --2- 172.21.15.99:0/1478021759 >> v2:172.21.15.191:3300/0 conn(0x7fd6441438f0 0x7fd64406a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:50.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.275+0000 7fd64159a700 1 -- 172.21.15.99:0/1478021759 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 79207729 0 0) 0x7fd6440d0870 con 0x7fd6440ccf00 2023-10-08T00:10:50.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.275+0000 7fd64159a700 1 -- 172.21.15.99:0/1478021759 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd630003570 con 0x7fd6440ccf00 2023-10-08T00:10:50.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.275+0000 7fd64159a700 1 -- 172.21.15.99:0/1478021759 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3893070423 0 0) 0x7fd630003570 con 0x7fd6440ccf00 2023-10-08T00:10:50.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.275+0000 7fd64159a700 1 -- 172.21.15.99:0/1478021759 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd644143520 msgr2=0x7fd6440cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:50.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.275+0000 7fd64159a700 1 --2- 172.21.15.99:0/1478021759 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd644143520 0x7fd6440cf580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:50.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.275+0000 7fd64159a700 1 -- 172.21.15.99:0/1478021759 >> v2:172.21.15.191:3300/0 conn(0x7fd6441438f0 msgr2=0x7fd64406a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:50.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.275+0000 7fd64159a700 1 --2- 172.21.15.99:0/1478021759 >> v2:172.21.15.191:3300/0 conn(0x7fd6441438f0 0x7fd64406a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:50.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.275+0000 7fd64159a700 1 -- 172.21.15.99:0/1478021759 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6440cff40 con 0x7fd6440ccf00 2023-10-08T00:10:50.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.275+0000 7fd64159a700 1 -- 172.21.15.99:0/1478021759 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7fd6340017f0 con 0x7fd6440ccf00 2023-10-08T00:10:50.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.275+0000 7fd641d9b700 1 --2- 172.21.15.99:0/1478021759 >> v2:172.21.15.191:3300/0 conn(0x7fd6441438f0 0x7fd64406a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:10:50.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.275+0000 7fd64159a700 1 -- 172.21.15.99:0/1478021759 >> v1:172.21.15.99:6789/0 conn(0x7fd6440ccf00 legacy=0x7fd64415da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:50.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.276+0000 7fd64159a700 1 --2- 172.21.15.99:0/1478021759 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd630003a60 0x7fd630003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:50.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.276+0000 7fd64159a700 1 --2- 172.21.15.99:0/1478021759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd630004810 0x7fd630004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:50.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.276+0000 7fd64159a700 1 --2- 172.21.15.99:0/1478021759 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd644143520 0x7fd6300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:50.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.276+0000 7fd64159a700 1 -- 172.21.15.99:0/1478021759 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6440cdfb0 con 0x7fd630004810 2023-10-08T00:10:50.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.276+0000 7fd64159a700 1 -- 172.21.15.99:0/1478021759 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6440cfc60 con 0x7fd630003a60 2023-10-08T00:10:50.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.276+0000 7fd64159a700 1 -- 172.21.15.99:0/1478021759 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd63000ba60 con 0x7fd644143520 2023-10-08T00:10:50.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.276+0000 7fd641d9b700 1 --2- 172.21.15.99:0/1478021759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd630004810 0x7fd630004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:50.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.276+0000 7fd64259c700 1 --2- 172.21.15.99:0/1478021759 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd630003a60 0x7fd630003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:50.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.276+0000 7fd642d9d700 1 --2- 172.21.15.99:0/1478021759 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd644143520 0x7fd6300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:50.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.276+0000 7fd641d9b700 1 -- 172.21.15.99:0/1478021759 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd644143520 msgr2=0x7fd6300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:50.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.276+0000 7fd641d9b700 1 --2- 172.21.15.99:0/1478021759 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd644143520 0x7fd6300051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:50.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.276+0000 7fd641d9b700 1 -- 172.21.15.99:0/1478021759 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd630003a60 msgr2=0x7fd630003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:50.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.276+0000 7fd641d9b700 1 --2- 172.21.15.99:0/1478021759 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd630003a60 0x7fd630003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:50.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.276+0000 7fd641d9b700 1 -- 172.21.15.99:0/1478021759 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd630003570 con 0x7fd630004810 2023-10-08T00:10:50.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.276+0000 7fd641d9b700 1 --2- 172.21.15.99:0/1478021759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd630004810 0x7fd630004c80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd62c009e20 tx=0x7fd62c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6b8dc228a18a5cd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:50.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.276+0000 7fd64159a700 1 -- 172.21.15.99:0/1478021759 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62c00e920 con 0x7fd630004810 2023-10-08T00:10:50.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.276+0000 7fd64159a700 1 -- 172.21.15.99:0/1478021759 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd62c00ea80 con 0x7fd630004810 2023-10-08T00:10:50.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.276+0000 7fd64159a700 1 -- 172.21.15.99:0/1478021759 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62c00ee20 con 0x7fd630004810 2023-10-08T00:10:50.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.277+0000 7fd64359e700 1 -- 172.21.15.99:0/1478021759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd630004810 msgr2=0x7fd630004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:50.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.277+0000 7fd64359e700 1 --2- 172.21.15.99:0/1478021759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd630004810 0x7fd630004c80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd62c009e20 tx=0x7fd62c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:10:50.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.277+0000 7fd642d9d700 1 -- 172.21.15.99:0/1478021759 reap_dead start 2023-10-08T00:10:50.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.277+0000 7fd64359e700 1 -- 172.21.15.99:0/1478021759 shutdown_connections 2023-10-08T00:10:50.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.277+0000 7fd64359e700 1 -- 172.21.15.99:0/1478021759 >> 172.21.15.99:0/1478021759 conn(0x7fd644069750 msgr2=0x7fd6440b25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:50.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.277+0000 7fd64359e700 1 -- 172.21.15.99:0/1478021759 shutdown_connections 2023-10-08T00:10:50.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.277+0000 7fd64359e700 1 -- 172.21.15.99:0/1478021759 wait complete. 2023-10-08T00:10:50.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.278+0000 7fd64359e700 1 Processor -- start 2023-10-08T00:10:50.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.278+0000 7fd64359e700 1 -- start start 2023-10-08T00:10:50.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.279+0000 7fd64359e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd630004810 0x7fd64413a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:50.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.279+0000 7fd64359e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd644142490 0x7fd64415a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:50.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.279+0000 7fd64359e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd64415cd60 0x7fd64415d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:50.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.279+0000 7fd64359e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd644142a60 con 0x7fd64415cd60 2023-10-08T00:10:50.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.279+0000 7fd64359e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64415c280 con 0x7fd644142490 2023-10-08T00:10:50.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.279+0000 7fd64359e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd64415c560 con 0x7fd630004810 2023-10-08T00:10:50.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.279+0000 7fd642d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd64415cd60 0x7fd64415d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:50.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.279+0000 7fd642d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd64415cd60 0x7fd64415d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39116/0 (socket says 172.21.15.99:39116) 2023-10-08T00:10:50.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.279+0000 7fd642d9d700 1 -- 172.21.15.99:0/3955643931 learned_addr learned my addr 172.21.15.99:0/3955643931 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:10:50.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.279+0000 7fd641d9b700 1 --2- 172.21.15.99:0/3955643931 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd644142490 0x7fd64415a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:50.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.279+0000 7fd64259c700 1 --2- 172.21.15.99:0/3955643931 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd630004810 0x7fd64413a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:50.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.279+0000 7fd642d9d700 1 -- 172.21.15.99:0/3955643931 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd630004810 msgr2=0x7fd64413a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:50.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.279+0000 7fd642d9d700 1 --2- 172.21.15.99:0/3955643931 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd630004810 0x7fd64413a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:50.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.279+0000 7fd642d9d700 1 -- 172.21.15.99:0/3955643931 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd644142490 msgr2=0x7fd64415a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:50.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.279+0000 7fd642d9d700 1 --2- 172.21.15.99:0/3955643931 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd644142490 0x7fd64415a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:50.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.279+0000 7fd642d9d700 1 -- 172.21.15.99:0/3955643931 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd64415c840 con 0x7fd64415cd60 2023-10-08T00:10:50.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.279+0000 7fd64259c700 1 --2- 172.21.15.99:0/3955643931 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd630004810 0x7fd64413a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:10:50.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.279+0000 7fd642d9d700 1 --2- 172.21.15.99:0/3955643931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd64415cd60 0x7fd64415d1e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd644071bb0 tx=0x7fd634002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:50.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.279+0000 7fd63a7fc700 1 -- 172.21.15.99:0/3955643931 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63400ad80 con 0x7fd64415cd60 2023-10-08T00:10:50.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.279+0000 7fd63a7fc700 1 -- 172.21.15.99:0/3955643931 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6340133f0 con 0x7fd64415cd60 2023-10-08T00:10:50.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.279+0000 7fd63a7fc700 1 -- 172.21.15.99:0/3955643931 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd634013790 con 0x7fd64415cd60 2023-10-08T00:10:50.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.279+0000 7fd64359e700 1 -- 172.21.15.99:0/3955643931 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd64413aa80 con 0x7fd64415cd60 2023-10-08T00:10:50.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.280+0000 7fd64359e700 1 -- 172.21.15.99:0/3955643931 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd64415a930 con 0x7fd64415cd60 2023-10-08T00:10:50.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.280+0000 7fd64359e700 1 -- 172.21.15.99:0/3955643931 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd644089030 con 0x7fd64415cd60 2023-10-08T00:10:50.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.281+0000 7fd63a7fc700 1 -- 172.21.15.99:0/3955643931 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd634013b80 con 0x7fd64415cd60 2023-10-08T00:10:50.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.281+0000 7fd63a7fc700 1 --2- 172.21.15.99:0/3955643931 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd624066f30 0x7fd6240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:50.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.281+0000 7fd63a7fc700 1 -- 172.21.15.99:0/3955643931 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fd634084b10 con 0x7fd64415cd60 2023-10-08T00:10:50.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.281+0000 7fd64259c700 1 --2- 172.21.15.99:0/3955643931 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd624066f30 0x7fd6240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:50.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.282+0000 7fd64259c700 1 --2- 172.21.15.99:0/3955643931 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd624066f30 0x7fd6240693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd63c006dc0 tx=0x7fd63c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:50.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.285+0000 7fd63a7fc700 1 -- 172.21.15.99:0/3955643931 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd6340520d0 con 0x7fd64415cd60 2023-10-08T00:10:50.508 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.508+0000 7fd64359e700 1 -- 172.21.15.99:0/3955643931 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd64415c0a0 con 0x7fd64415cd60 2023-10-08T00:10:50.509 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.508+0000 7fd63a7fc700 1 -- 172.21.15.99:0/3955643931 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3527 (secure 0 0 0) 0x7fd6340556f0 con 0x7fd64415cd60 2023-10-08T00:10:50.509 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:10:50.509 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/649537180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":649537180},{"type":"v1","addr":"172.21.15.99:6833","nonce":649537180}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4239,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2194110899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2194110899},{"type":"v1","addr":"172.21.15.99:6835","nonce":2194110899}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4251,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/1732443753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":1732443753},{"type":"v1","addr":"172.21.15.191:6837","nonce":1732443753}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/1988845948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1988845948},{"type":"v1","addr":"172.21.15.191:6836","nonce":1988845948}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-10-08T00:10:50.511 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.510+0000 7fd64359e700 1 -- 172.21.15.99:0/3955643931 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd624066f30 msgr2=0x7fd6240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:50.511 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.510+0000 7fd64359e700 1 --2- 172.21.15.99:0/3955643931 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd624066f30 0x7fd6240693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd63c006dc0 tx=0x7fd63c008040 comp rx=0 tx=0).stop 2023-10-08T00:10:50.511 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.510+0000 7fd64359e700 1 -- 172.21.15.99:0/3955643931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd64415cd60 msgr2=0x7fd64415d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:50.512 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.510+0000 7fd64359e700 1 --2- 172.21.15.99:0/3955643931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd64415cd60 0x7fd64415d1e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd644071bb0 tx=0x7fd634002210 comp rx=0 tx=0).stop 2023-10-08T00:10:50.512 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.511+0000 7fd64359e700 1 -- 172.21.15.99:0/3955643931 shutdown_connections 2023-10-08T00:10:50.512 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.511+0000 7fd64359e700 1 --2- 172.21.15.99:0/3955643931 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd624066f30 0x7fd6240693e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:50.512 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.511+0000 7fd64359e700 1 --2- 172.21.15.99:0/3955643931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd64415cd60 0x7fd64415d1e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:50.512 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.511+0000 7fd64359e700 1 --2- 172.21.15.99:0/3955643931 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd630004810 0x7fd64413a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:50.512 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.511+0000 7fd64359e700 1 --2- 172.21.15.99:0/3955643931 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd644142490 0x7fd64415a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:50.512 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.511+0000 7fd64359e700 1 -- 172.21.15.99:0/3955643931 >> 172.21.15.99:0/3955643931 conn(0x7fd644069750 msgr2=0x7fd6440b26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:50.512 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.511+0000 7fd64359e700 1 -- 172.21.15.99:0/3955643931 shutdown_connections 2023-10-08T00:10:50.512 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.511+0000 7fd64359e700 1 -- 172.21.15.99:0/3955643931 wait complete. 2023-10-08T00:10:50.513 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 3 2023-10-08T00:10:50.523 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:10:50.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.635+0000 7fdcd759e700 1 Processor -- start 2023-10-08T00:10:50.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.635+0000 7fdcd759e700 1 -- start start 2023-10-08T00:10:50.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.636+0000 7fdcd759e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fdcd80cc370 0x7fdcd806a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:50.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.636+0000 7fdcd759e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdcd806a5a0 0x7fdcd8156350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:50.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.636+0000 7fdcd759e700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcd8068340 con 0x7fdcd80cbfa0 2023-10-08T00:10:50.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.636+0000 7fdcd759e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcd806abd0 con 0x7fdcd80cc370 2023-10-08T00:10:50.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.636+0000 7fdcd759e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd8068ee0 con 0x7fdcd806a5a0 2023-10-08T00:10:50.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.636+0000 7fdcd659c700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fdcd80cbfa0 0x7fdcd80687d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:38416/0 (socket says 172.21.15.99:38416) 2023-10-08T00:10:50.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.636+0000 7fdcd659c700 1 -- 172.21.15.99:0/1275940779 learned_addr learned my addr 172.21.15.99:0/1275940779 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:10:50.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.636+0000 7fdcd5d9b700 1 -- 172.21.15.99:0/1275940779 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4139783862 0 0) 0x7fdcd8068340 con 0x7fdcd80cbfa0 2023-10-08T00:10:50.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.636+0000 7fdccdd9b700 1 --2- 172.21.15.99:0/1275940779 >> v2:172.21.15.191:3300/0 conn(0x7fdcd80cc370 0x7fdcd806a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:50.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.636+0000 7fdcd6d9d700 1 --2- 172.21.15.99:0/1275940779 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdcd806a5a0 0x7fdcd8156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:50.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.636+0000 7fdcd5d9b700 1 -- 172.21.15.99:0/1275940779 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcc4003570 con 0x7fdcd80cbfa0 2023-10-08T00:10:50.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.636+0000 7fdcd5d9b700 1 -- 172.21.15.99:0/1275940779 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3292258800 0 0) 0x7fdcc4003570 con 0x7fdcd80cbfa0 2023-10-08T00:10:50.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.636+0000 7fdcd5d9b700 1 -- 172.21.15.99:0/1275940779 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdcd806a5a0 msgr2=0x7fdcd8156350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:50.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.636+0000 7fdcd5d9b700 1 --2- 172.21.15.99:0/1275940779 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdcd806a5a0 0x7fdcd8156350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:50.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.636+0000 7fdcd5d9b700 1 -- 172.21.15.99:0/1275940779 >> v2:172.21.15.191:3300/0 conn(0x7fdcd80cc370 msgr2=0x7fdcd806a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:50.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.636+0000 7fdcd5d9b700 1 --2- 172.21.15.99:0/1275940779 >> v2:172.21.15.191:3300/0 conn(0x7fdcd80cc370 0x7fdcd806a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:50.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.636+0000 7fdcd5d9b700 1 -- 172.21.15.99:0/1275940779 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd806f040 con 0x7fdcd80cbfa0 2023-10-08T00:10:50.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.636+0000 7fdcd5d9b700 1 -- 172.21.15.99:0/1275940779 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7fdcc8001470 con 0x7fdcd80cbfa0 2023-10-08T00:10:50.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.636+0000 7fdcd5d9b700 1 -- 172.21.15.99:0/1275940779 >> v1:172.21.15.99:6789/0 conn(0x7fdcd80cbfa0 legacy=0x7fdcd80687d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:50.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.636+0000 7fdcd5d9b700 1 --2- 172.21.15.99:0/1275940779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdcc4003ae0 0x7fdcc4003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:50.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.637+0000 7fdcd5d9b700 1 --2- 172.21.15.99:0/1275940779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdcc4004810 0x7fdcc4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:50.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.637+0000 7fdcd5d9b700 1 --2- 172.21.15.99:0/1275940779 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdcd806a5a0 0x7fdcc40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:50.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.637+0000 7fdcd5d9b700 1 -- 172.21.15.99:0/1275940779 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd806abd0 con 0x7fdcc4003ae0 2023-10-08T00:10:50.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.637+0000 7fdcd5d9b700 1 -- 172.21.15.99:0/1275940779 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd8068ee0 con 0x7fdcc4004810 2023-10-08T00:10:50.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.637+0000 7fdcd5d9b700 1 -- 172.21.15.99:0/1275940779 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc400ddd0 con 0x7fdcd806a5a0 2023-10-08T00:10:50.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.637+0000 7fdcd659c700 1 --2- 172.21.15.99:0/1275940779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdcc4003ae0 0x7fdcc4003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:50.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.637+0000 7fdccdd9b700 1 --2- 172.21.15.99:0/1275940779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdcc4004810 0x7fdcc4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:50.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.637+0000 7fdcd6d9d700 1 --2- 172.21.15.99:0/1275940779 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdcd806a5a0 0x7fdcc40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:50.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.637+0000 7fdcd659c700 1 -- 172.21.15.99:0/1275940779 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdcd806a5a0 msgr2=0x7fdcc40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:50.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.637+0000 7fdcd659c700 1 --2- 172.21.15.99:0/1275940779 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdcd806a5a0 0x7fdcc40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:50.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.637+0000 7fdcd659c700 1 -- 172.21.15.99:0/1275940779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdcc4004810 msgr2=0x7fdcc4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:50.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.637+0000 7fdcd659c700 1 --2- 172.21.15.99:0/1275940779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdcc4004810 0x7fdcc4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:50.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.637+0000 7fdcd659c700 1 -- 172.21.15.99:0/1275940779 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc4003570 con 0x7fdcc4003ae0 2023-10-08T00:10:50.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.637+0000 7fdcd6d9d700 1 --2- 172.21.15.99:0/1275940779 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdcd806a5a0 0x7fdcc40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:10:50.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.637+0000 7fdcd659c700 1 --2- 172.21.15.99:0/1275940779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdcc4003ae0 0x7fdcc4003ef0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdcc800c2a0 tx=0x7fdcd80cd5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a766a029e7c45cb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:50.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.637+0000 7fdcd5d9b700 1 -- 172.21.15.99:0/1275940779 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc800cbd0 con 0x7fdcc4003ae0 2023-10-08T00:10:50.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.637+0000 7fdcd5d9b700 1 -- 172.21.15.99:0/1275940779 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcc800cd50 con 0x7fdcc4003ae0 2023-10-08T00:10:50.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.637+0000 7fdcd5d9b700 1 -- 172.21.15.99:0/1275940779 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc8005b70 con 0x7fdcc4003ae0 2023-10-08T00:10:50.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.638+0000 7fdcd759e700 1 -- 172.21.15.99:0/1275940779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdcc4003ae0 msgr2=0x7fdcc4003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:50.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.638+0000 7fdcd759e700 1 --2- 172.21.15.99:0/1275940779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdcc4003ae0 0x7fdcc4003ef0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdcc800c2a0 tx=0x7fdcd80cd5d0 comp rx=0 tx=0).stop 2023-10-08T00:10:50.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.638+0000 7fdcd6d9d700 1 -- 172.21.15.99:0/1275940779 reap_dead start 2023-10-08T00:10:50.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.638+0000 7fdcd759e700 1 -- 172.21.15.99:0/1275940779 shutdown_connections 2023-10-08T00:10:50.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.638+0000 7fdcd759e700 1 -- 172.21.15.99:0/1275940779 >> 172.21.15.99:0/1275940779 conn(0x7fdcd80cd8f0 msgr2=0x7fdcd80cdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:50.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.638+0000 7fdcd759e700 1 -- 172.21.15.99:0/1275940779 shutdown_connections 2023-10-08T00:10:50.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.638+0000 7fdcd759e700 1 -- 172.21.15.99:0/1275940779 wait complete. 2023-10-08T00:10:50.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.639+0000 7fdcd759e700 1 Processor -- start 2023-10-08T00:10:50.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.639+0000 7fdcd759e700 1 -- start start 2023-10-08T00:10:50.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.640+0000 7fdcd759e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdcc4003ae0 0x7fdcd80c5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:50.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.640+0000 7fdcd759e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdcd8155600 0x7fdcd80c60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:50.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.640+0000 7fdcd759e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdcd8155a70 0x7fdcd80c6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:50.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.640+0000 7fdcd759e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd81586a0 con 0x7fdcc4003ae0 2023-10-08T00:10:50.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.640+0000 7fdcd759e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd8156120 con 0x7fdcd8155a70 2023-10-08T00:10:50.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.640+0000 7fdcd759e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd8156400 con 0x7fdcd8155600 2023-10-08T00:10:50.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.640+0000 7fdcd659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdcc4003ae0 0x7fdcd80c5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:50.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.640+0000 7fdcd659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdcc4003ae0 0x7fdcd80c5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39134/0 (socket says 172.21.15.99:39134) 2023-10-08T00:10:50.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.640+0000 7fdcd659c700 1 -- 172.21.15.99:0/2576959930 learned_addr learned my addr 172.21.15.99:0/2576959930 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:10:50.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.640+0000 7fdcd6d9d700 1 --2- 172.21.15.99:0/2576959930 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdcd8155a70 0x7fdcd80c6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:50.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.640+0000 7fdccdd9b700 1 --2- 172.21.15.99:0/2576959930 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdcd8155600 0x7fdcd80c60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:50.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.640+0000 7fdcd659c700 1 -- 172.21.15.99:0/2576959930 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdcd8155600 msgr2=0x7fdcd80c60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:50.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.640+0000 7fdcd659c700 1 --2- 172.21.15.99:0/2576959930 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdcd8155600 0x7fdcd80c60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:50.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.640+0000 7fdcd659c700 1 -- 172.21.15.99:0/2576959930 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdcd8155a70 msgr2=0x7fdcd80c6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:50.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.640+0000 7fdcd659c700 1 --2- 172.21.15.99:0/2576959930 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdcd8155a70 0x7fdcd80c6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:50.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.640+0000 7fdcd659c700 1 -- 172.21.15.99:0/2576959930 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd8158940 con 0x7fdcc4003ae0 2023-10-08T00:10:50.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.640+0000 7fdcd659c700 1 --2- 172.21.15.99:0/2576959930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdcc4003ae0 0x7fdcd80c5900 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdcc400dd20 tx=0x7fdcc8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:50.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.640+0000 7fdcceffd700 1 -- 172.21.15.99:0/2576959930 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc80070d0 con 0x7fdcc4003ae0 2023-10-08T00:10:50.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.640+0000 7fdcceffd700 1 -- 172.21.15.99:0/2576959930 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcc8007230 con 0x7fdcc4003ae0 2023-10-08T00:10:50.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.640+0000 7fdcd759e700 1 -- 172.21.15.99:0/2576959930 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcd8158c20 con 0x7fdcc4003ae0 2023-10-08T00:10:50.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.640+0000 7fdcceffd700 1 -- 172.21.15.99:0/2576959930 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc80075f0 con 0x7fdcc4003ae0 2023-10-08T00:10:50.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.640+0000 7fdcd759e700 1 -- 172.21.15.99:0/2576959930 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcd81566e0 con 0x7fdcc4003ae0 2023-10-08T00:10:50.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.641+0000 7fdcd759e700 1 -- 172.21.15.99:0/2576959930 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcd806d5d0 con 0x7fdcc4003ae0 2023-10-08T00:10:50.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.642+0000 7fdcceffd700 1 -- 172.21.15.99:0/2576959930 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdcc8019030 con 0x7fdcc4003ae0 2023-10-08T00:10:50.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.642+0000 7fdcceffd700 1 --2- 172.21.15.99:0/2576959930 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdcb8066f30 0x7fdcb80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:50.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.642+0000 7fdcceffd700 1 -- 172.21.15.99:0/2576959930 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fdcc8085980 con 0x7fdcc4003ae0 2023-10-08T00:10:50.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.642+0000 7fdccdd9b700 1 --2- 172.21.15.99:0/2576959930 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdcb8066f30 0x7fdcb80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:50.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.643+0000 7fdccdd9b700 1 --2- 172.21.15.99:0/2576959930 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdcb8066f30 0x7fdcb80693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0006dc0 tx=0x7fdcc0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:50.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.646+0000 7fdcceffd700 1 -- 172.21.15.99:0/2576959930 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdcc8052ff0 con 0x7fdcc4003ae0 2023-10-08T00:10:50.829 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.828+0000 7fdcd759e700 1 -- 172.21.15.99:0/2576959930 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdcd80b3740 con 0x7fdcc4003ae0 2023-10-08T00:10:50.829 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.829+0000 7fdcceffd700 1 -- 172.21.15.99:0/2576959930 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12127 (secure 0 0 0) 0x7fdcc8056610 con 0x7fdcc4003ae0 2023-10-08T00:10:50.830 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:10:50.830 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":16,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:10:44.727227+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:10:50.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.831+0000 7fdcd759e700 1 -- 172.21.15.99:0/2576959930 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdcb8066f30 msgr2=0x7fdcb80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:50.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.831+0000 7fdcd759e700 1 --2- 172.21.15.99:0/2576959930 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdcb8066f30 0x7fdcb80693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0006dc0 tx=0x7fdcc0008040 comp rx=0 tx=0).stop 2023-10-08T00:10:50.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.831+0000 7fdcd759e700 1 -- 172.21.15.99:0/2576959930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdcc4003ae0 msgr2=0x7fdcd80c5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:50.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.831+0000 7fdcd759e700 1 --2- 172.21.15.99:0/2576959930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdcc4003ae0 0x7fdcd80c5900 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdcc400dd20 tx=0x7fdcc8015000 comp rx=0 tx=0).stop 2023-10-08T00:10:50.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.831+0000 7fdcd759e700 1 -- 172.21.15.99:0/2576959930 shutdown_connections 2023-10-08T00:10:50.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.831+0000 7fdcd759e700 1 --2- 172.21.15.99:0/2576959930 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdcb8066f30 0x7fdcb80693e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:50.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.831+0000 7fdcd759e700 1 --2- 172.21.15.99:0/2576959930 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdcd8155a70 0x7fdcd80c6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:50.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.831+0000 7fdcd759e700 1 --2- 172.21.15.99:0/2576959930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdcc4003ae0 0x7fdcd80c5900 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:50.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.831+0000 7fdcd759e700 1 --2- 172.21.15.99:0/2576959930 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdcd8155600 0x7fdcd80c60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:50.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.831+0000 7fdcd759e700 1 -- 172.21.15.99:0/2576959930 >> 172.21.15.99:0/2576959930 conn(0x7fdcd80cd8f0 msgr2=0x7fdcd81524b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:50.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.831+0000 7fdcd759e700 1 -- 172.21.15.99:0/2576959930 shutdown_connections 2023-10-08T00:10:50.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.831+0000 7fdcd759e700 1 -- 172.21.15.99:0/2576959930 wait complete. 2023-10-08T00:10:50.843 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:10:42.063344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-08T00:10:50.844 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T00:10:50.968 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.959+0000 7f6d525ab700 1 Processor -- start 2023-10-08T00:10:50.968 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.959+0000 7f6d525ab700 1 -- start start 2023-10-08T00:10:50.968 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.960+0000 7f6d525ab700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f6d4c159ec0 0x7f6d4c0af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:50.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.960+0000 7f6d525ab700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d4c0b0790 con 0x7f6d4c06acb0 2023-10-08T00:10:50.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.960+0000 7f6d525ab700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d4c0ce1f0 con 0x7f6d4c15a290 2023-10-08T00:10:50.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.960+0000 7f6d525ab700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d4c0b0b20 con 0x7f6d4c159ec0 2023-10-08T00:10:50.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.960+0000 7f6d50b48700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f6d4c06acb0 0x7f6d4c0cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:38432/0 (socket says 172.21.15.99:38432) 2023-10-08T00:10:50.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.960+0000 7f6d50b48700 1 -- 172.21.15.99:0/3216548208 learned_addr learned my addr 172.21.15.99:0/3216548208 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:10:50.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.960+0000 7f6d4bfff700 1 --2- 172.21.15.99:0/3216548208 >> v2:172.21.15.191:3300/0 conn(0x7f6d4c159ec0 0x7f6d4c0af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:50.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.960+0000 7f6d4affd700 1 -- 172.21.15.99:0/3216548208 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2454884952 0 0) 0x7f6d4c0b0790 con 0x7f6d4c06acb0 2023-10-08T00:10:50.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.960+0000 7f6d4affd700 1 -- 172.21.15.99:0/3216548208 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d34003570 con 0x7f6d4c06acb0 2023-10-08T00:10:50.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.961+0000 7f6d4affd700 1 -- 172.21.15.99:0/3216548208 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2608996959 0 0) 0x7f6d4c0ce1f0 con 0x7f6d4c15a290 2023-10-08T00:10:50.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.961+0000 7f6d4affd700 1 -- 172.21.15.99:0/3216548208 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d4c0b0790 con 0x7f6d4c15a290 2023-10-08T00:10:50.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.961+0000 7f6d4bfff700 1 -- 172.21.15.99:0/3216548208 >> v1:172.21.15.191:6789/0 conn(0x7f6d4c15a290 legacy=0x7f6d4c06a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:50.970 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.961+0000 7f6d4bfff700 1 -- 172.21.15.99:0/3216548208 >> v1:172.21.15.99:6789/0 conn(0x7f6d4c06acb0 legacy=0x7f6d4c0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:50.970 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.961+0000 7f6d4affd700 1 -- 172.21.15.99:0/3216548208 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2879831398 0 0) 0x7f6d34003570 con 0x7f6d4c06acb0 2023-10-08T00:10:50.970 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.961+0000 7f6d4bfff700 1 -- 172.21.15.99:0/3216548208 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d4c0b04b0 con 0x7f6d4c159ec0 2023-10-08T00:10:50.970 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.961+0000 7f6d4bfff700 1 --2- 172.21.15.99:0/3216548208 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d4c159ec0 0x7f6d4c0af940 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c0098e0 tx=0x7f6d3c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2158bfcd8dce3f68 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:50.970 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.961+0000 7f6d4affd700 1 -- 172.21.15.99:0/3216548208 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d440017f0 con 0x7f6d4c159ec0 2023-10-08T00:10:50.970 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.961+0000 7f6d4affd700 1 -- 172.21.15.99:0/3216548208 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d3c011420 con 0x7f6d4c159ec0 2023-10-08T00:10:50.970 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.961+0000 7f6d4affd700 1 -- 172.21.15.99:0/3216548208 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c0117c0 con 0x7f6d4c159ec0 2023-10-08T00:10:50.970 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.961+0000 7f6d525ab700 1 -- 172.21.15.99:0/3216548208 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d4c159ec0 msgr2=0x7f6d4c0af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:50.971 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.961+0000 7f6d525ab700 1 --2- 172.21.15.99:0/3216548208 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d4c159ec0 0x7f6d4c0af940 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c0098e0 tx=0x7f6d3c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:10:50.971 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.961+0000 7f6d525ab700 1 -- 172.21.15.99:0/3216548208 shutdown_connections 2023-10-08T00:10:50.971 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.961+0000 7f6d525ab700 1 --2- 172.21.15.99:0/3216548208 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d4c159ec0 0x7f6d4c0af940 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:50.971 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.961+0000 7f6d525ab700 1 -- 172.21.15.99:0/3216548208 >> 172.21.15.99:0/3216548208 conn(0x7f6d4c0d1880 msgr2=0x7f6d4c0cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:50.971 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.961+0000 7f6d525ab700 1 -- 172.21.15.99:0/3216548208 shutdown_connections 2023-10-08T00:10:50.971 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.962+0000 7f6d525ab700 1 -- 172.21.15.99:0/3216548208 wait complete. 2023-10-08T00:10:50.971 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.963+0000 7f6d525ab700 1 Processor -- start 2023-10-08T00:10:50.971 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.963+0000 7f6d525ab700 1 -- start start 2023-10-08T00:10:50.971 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.963+0000 7f6d525ab700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d4c06acb0 0x7f6d4c0c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:50.971 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.963+0000 7f6d525ab700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d4c159ec0 0x7f6d4c154a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:50.972 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.963+0000 7f6d525ab700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d4c15a290 0x7f6d4c0c94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:50.972 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.963+0000 7f6d525ab700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4c0c9050 con 0x7f6d4c06acb0 2023-10-08T00:10:50.972 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.963+0000 7f6d525ab700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4c0c8440 con 0x7f6d4c159ec0 2023-10-08T00:10:50.972 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.963+0000 7f6d525ab700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4c0c8720 con 0x7f6d4c15a290 2023-10-08T00:10:50.972 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.964+0000 7f6d4bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d4c06acb0 0x7f6d4c0c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:50.972 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.964+0000 7f6d4bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d4c06acb0 0x7f6d4c0c7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39140/0 (socket says 172.21.15.99:39140) 2023-10-08T00:10:50.972 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.964+0000 7f6d4bfff700 1 -- 172.21.15.99:0/3854477154 learned_addr learned my addr 172.21.15.99:0/3854477154 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:10:50.972 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.964+0000 7f6d50b48700 1 --2- 172.21.15.99:0/3854477154 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d4c15a290 0x7f6d4c0c94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:50.972 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.964+0000 7f6d4b7fe700 1 --2- 172.21.15.99:0/3854477154 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d4c159ec0 0x7f6d4c154a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:50.972 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.964+0000 7f6d4bfff700 1 -- 172.21.15.99:0/3854477154 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d4c15a290 msgr2=0x7f6d4c0c94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:50.972 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.964+0000 7f6d4bfff700 1 --2- 172.21.15.99:0/3854477154 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d4c15a290 0x7f6d4c0c94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:50.972 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.964+0000 7f6d4bfff700 1 -- 172.21.15.99:0/3854477154 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d4c159ec0 msgr2=0x7f6d4c154a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:50.973 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.964+0000 7f6d4bfff700 1 --2- 172.21.15.99:0/3854477154 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d4c159ec0 0x7f6d4c154a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:50.973 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.964+0000 7f6d4bfff700 1 -- 172.21.15.99:0/3854477154 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d3c009420 con 0x7f6d4c06acb0 2023-10-08T00:10:50.973 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.964+0000 7f6d4b7fe700 1 --2- 172.21.15.99:0/3854477154 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d4c159ec0 0x7f6d4c154a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:10:50.973 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.964+0000 7f6d4bfff700 1 --2- 172.21.15.99:0/3854477154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d4c06acb0 0x7f6d4c0c7e60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c0ce930 tx=0x7f6d3c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:50.973 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.964+0000 7f6d33fff700 1 -- 172.21.15.99:0/3854477154 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c01cbe0 con 0x7f6d4c06acb0 2023-10-08T00:10:50.973 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.964+0000 7f6d525ab700 1 -- 172.21.15.99:0/3854477154 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d4c0c8a00 con 0x7f6d4c06acb0 2023-10-08T00:10:50.973 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.964+0000 7f6d33fff700 1 -- 172.21.15.99:0/3854477154 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d3c025430 con 0x7f6d4c06acb0 2023-10-08T00:10:50.973 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.964+0000 7f6d525ab700 1 -- 172.21.15.99:0/3854477154 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d4c0af910 con 0x7f6d4c06acb0 2023-10-08T00:10:50.973 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.964+0000 7f6d33fff700 1 -- 172.21.15.99:0/3854477154 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c0255a0 con 0x7f6d4c06acb0 2023-10-08T00:10:50.973 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.965+0000 7f6d525ab700 1 -- 172.21.15.99:0/3854477154 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d4c0c9330 con 0x7f6d4c06acb0 2023-10-08T00:10:50.973 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.965+0000 7f6d33fff700 1 -- 172.21.15.99:0/3854477154 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6d3c0259d0 con 0x7f6d4c06acb0 2023-10-08T00:10:50.973 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.966+0000 7f6d33fff700 1 --2- 172.21.15.99:0/3854477154 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6d38066f30 0x7f6d380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:50.974 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.966+0000 7f6d33fff700 1 -- 172.21.15.99:0/3854477154 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f6d3c085800 con 0x7f6d4c06acb0 2023-10-08T00:10:50.974 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.966+0000 7f6d4b7fe700 1 --2- 172.21.15.99:0/3854477154 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6d38066f30 0x7f6d380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:50.974 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.966+0000 7f6d4b7fe700 1 --2- 172.21.15.99:0/3854477154 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6d38066f30 0x7f6d380693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c0b5bc0 tx=0x7f6d40009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:50.974 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:50.970+0000 7f6d33fff700 1 -- 172.21.15.99:0/3854477154 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6d3c052dc0 con 0x7f6d4c06acb0 2023-10-08T00:10:51.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.155+0000 7f6d525ab700 1 -- 172.21.15.99:0/3854477154 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6d4c04c3c0 con 0x7f6d4c06acb0 2023-10-08T00:10:51.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.155+0000 7f6d33fff700 1 -- 172.21.15.99:0/3854477154 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6d3c0563e0 con 0x7f6d4c06acb0 2023-10-08T00:10:51.156 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2023-10-08T00:10:51.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.157+0000 7f6d525ab700 1 -- 172.21.15.99:0/3854477154 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6d38066f30 msgr2=0x7f6d380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:51.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.157+0000 7f6d525ab700 1 --2- 172.21.15.99:0/3854477154 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6d38066f30 0x7f6d380693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c0b5bc0 tx=0x7f6d40009040 comp rx=0 tx=0).stop 2023-10-08T00:10:51.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.157+0000 7f6d525ab700 1 -- 172.21.15.99:0/3854477154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d4c06acb0 msgr2=0x7f6d4c0c7e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:51.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.157+0000 7f6d525ab700 1 --2- 172.21.15.99:0/3854477154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d4c06acb0 0x7f6d4c0c7e60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c0ce930 tx=0x7f6d3c016000 comp rx=0 tx=0).stop 2023-10-08T00:10:51.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.157+0000 7f6d525ab700 1 -- 172.21.15.99:0/3854477154 shutdown_connections 2023-10-08T00:10:51.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.157+0000 7f6d525ab700 1 --2- 172.21.15.99:0/3854477154 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6d38066f30 0x7f6d380693e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:51.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.157+0000 7f6d525ab700 1 --2- 172.21.15.99:0/3854477154 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d4c15a290 0x7f6d4c0c94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:51.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.157+0000 7f6d525ab700 1 --2- 172.21.15.99:0/3854477154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d4c06acb0 0x7f6d4c0c7e60 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:51.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.157+0000 7f6d525ab700 1 --2- 172.21.15.99:0/3854477154 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d4c159ec0 0x7f6d4c154a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:51.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.157+0000 7f6d525ab700 1 -- 172.21.15.99:0/3854477154 >> 172.21.15.99:0/3854477154 conn(0x7f6d4c0d1880 msgr2=0x7f6d4c1558d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:51.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.157+0000 7f6d525ab700 1 -- 172.21.15.99:0/3854477154 shutdown_connections 2023-10-08T00:10:51.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.157+0000 7f6d525ab700 1 -- 172.21.15.99:0/3854477154 wait complete. 2023-10-08T00:10:51.170 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T00:10:51.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.283+0000 7f67bdbce700 1 Processor -- start 2023-10-08T00:10:51.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.283+0000 7f67bdbce700 1 -- start start 2023-10-08T00:10:51.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.284+0000 7f67bdbce700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67b8157c40 0x7f67b8158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:51.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.284+0000 7f67bdbce700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67b815cbb0 con 0x7f67b815a620 2023-10-08T00:10:51.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.284+0000 7f67bdbce700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67b815bfa0 con 0x7f67b8157870 2023-10-08T00:10:51.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.284+0000 7f67bdbce700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67b815bdb0 con 0x7f67b8157c40 2023-10-08T00:10:51.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.284+0000 7f67affff700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f67b815a620 0x7f67b815d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:38436/0 (socket says 172.21.15.99:38436) 2023-10-08T00:10:51.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.284+0000 7f67affff700 1 -- 172.21.15.99:0/2400025378 learned_addr learned my addr 172.21.15.99:0/2400025378 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:10:51.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.284+0000 7f67b7fff700 1 --2- 172.21.15.99:0/2400025378 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67b8157c40 0x7f67b8158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:51.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.284+0000 7f67b6ffd700 1 -- 172.21.15.99:0/2400025378 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2156573740 0 0) 0x7f67b815cbb0 con 0x7f67b815a620 2023-10-08T00:10:51.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.284+0000 7f67b6ffd700 1 -- 172.21.15.99:0/2400025378 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67a4003570 con 0x7f67b815a620 2023-10-08T00:10:51.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.284+0000 7f67b6ffd700 1 -- 172.21.15.99:0/2400025378 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2189557285 0 0) 0x7f67b815bfa0 con 0x7f67b8157870 2023-10-08T00:10:51.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.284+0000 7f67b6ffd700 1 -- 172.21.15.99:0/2400025378 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67b815cbb0 con 0x7f67b8157870 2023-10-08T00:10:51.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.284+0000 7f67b6ffd700 1 -- 172.21.15.99:0/2400025378 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4028537599 0 0) 0x7f67a4003570 con 0x7f67b815a620 2023-10-08T00:10:51.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.284+0000 7f67b6ffd700 1 -- 172.21.15.99:0/2400025378 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67b8157c40 msgr2=0x7f67b8158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:51.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.284+0000 7f67b6ffd700 1 --2- 172.21.15.99:0/2400025378 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67b8157c40 0x7f67b8158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:51.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.284+0000 7f67b6ffd700 1 -- 172.21.15.99:0/2400025378 >> v1:172.21.15.191:6789/0 conn(0x7f67b8157870 legacy=0x7f67b815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:51.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.284+0000 7f67b6ffd700 1 -- 172.21.15.99:0/2400025378 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b815c280 con 0x7f67b815a620 2023-10-08T00:10:51.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.284+0000 7f67b6ffd700 1 -- 172.21.15.99:0/2400025378 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f67a0001470 con 0x7f67b815a620 2023-10-08T00:10:51.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.284+0000 7f67b6ffd700 1 -- 172.21.15.99:0/2400025378 >> v1:172.21.15.99:6789/0 conn(0x7f67b815a620 legacy=0x7f67b815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:51.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.284+0000 7f67b6ffd700 1 --2- 172.21.15.99:0/2400025378 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67b8157c40 0x7f67a4003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:51.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.284+0000 7f67b6ffd700 1 --2- 172.21.15.99:0/2400025378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67a4004180 0x7f67a4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:51.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.285+0000 7f67b6ffd700 1 --2- 172.21.15.99:0/2400025378 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67a4004d50 0x7f67a40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:51.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.285+0000 7f67b6ffd700 1 -- 172.21.15.99:0/2400025378 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a400bb70 con 0x7f67a4004180 2023-10-08T00:10:51.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.285+0000 7f67b6ffd700 1 -- 172.21.15.99:0/2400025378 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a400bce0 con 0x7f67a4004d50 2023-10-08T00:10:51.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.285+0000 7f67b6ffd700 1 -- 172.21.15.99:0/2400025378 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67a400bea0 con 0x7f67b8157c40 2023-10-08T00:10:51.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.285+0000 7f67affff700 1 --2- 172.21.15.99:0/2400025378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67a4004180 0x7f67a4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:51.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.285+0000 7f67b77fe700 1 --2- 172.21.15.99:0/2400025378 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67b8157c40 0x7f67a4003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:51.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.285+0000 7f67b7fff700 1 --2- 172.21.15.99:0/2400025378 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67a4004d50 0x7f67a40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:51.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.285+0000 7f67affff700 1 -- 172.21.15.99:0/2400025378 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67b8157c40 msgr2=0x7f67a4003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:51.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.285+0000 7f67affff700 1 --2- 172.21.15.99:0/2400025378 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67b8157c40 0x7f67a4003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:51.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.285+0000 7f67affff700 1 -- 172.21.15.99:0/2400025378 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67a4004d50 msgr2=0x7f67a40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:51.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.285+0000 7f67affff700 1 --2- 172.21.15.99:0/2400025378 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67a4004d50 0x7f67a40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:51.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.285+0000 7f67affff700 1 -- 172.21.15.99:0/2400025378 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a4003570 con 0x7f67a4004180 2023-10-08T00:10:51.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.285+0000 7f67affff700 1 --2- 172.21.15.99:0/2400025378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67a4004180 0x7f67a4004810 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f67a000c2a0 tx=0x7f67a0001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d63f0c693196ff40 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:51.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.285+0000 7f67b6ffd700 1 -- 172.21.15.99:0/2400025378 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a000cb90 con 0x7f67a4004180 2023-10-08T00:10:51.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.285+0000 7f67b6ffd700 1 -- 172.21.15.99:0/2400025378 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67a000ccf0 con 0x7f67a4004180 2023-10-08T00:10:51.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.285+0000 7f67b6ffd700 1 -- 172.21.15.99:0/2400025378 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a0005b70 con 0x7f67a4004180 2023-10-08T00:10:51.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.286+0000 7f67bdbce700 1 -- 172.21.15.99:0/2400025378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67a4004180 msgr2=0x7f67a4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:51.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.286+0000 7f67bdbce700 1 --2- 172.21.15.99:0/2400025378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67a4004180 0x7f67a4004810 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f67a000c2a0 tx=0x7f67a0001e10 comp rx=0 tx=0).stop 2023-10-08T00:10:51.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.286+0000 7f67b7fff700 1 -- 172.21.15.99:0/2400025378 reap_dead start 2023-10-08T00:10:51.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.286+0000 7f67bdbce700 1 -- 172.21.15.99:0/2400025378 shutdown_connections 2023-10-08T00:10:51.294 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.286+0000 7f67bdbce700 1 -- 172.21.15.99:0/2400025378 >> 172.21.15.99:0/2400025378 conn(0x7f67b8068ce0 msgr2=0x7f67b80690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:51.294 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.286+0000 7f67bdbce700 1 -- 172.21.15.99:0/2400025378 shutdown_connections 2023-10-08T00:10:51.294 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.286+0000 7f67bdbce700 1 -- 172.21.15.99:0/2400025378 wait complete. 2023-10-08T00:10:51.294 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.287+0000 7f67bdbce700 1 Processor -- start 2023-10-08T00:10:51.294 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.287+0000 7f67bdbce700 1 -- start start 2023-10-08T00:10:51.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.288+0000 7f67bdbce700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67a4004180 0x7f67b814cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:51.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.288+0000 7f67bdbce700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67b814e7f0 0x7f67b80b2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:51.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.288+0000 7f67bdbce700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67b814ff10 0x7f67b81503b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:51.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.288+0000 7f67bdbce700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b816a720 con 0x7f67a4004180 2023-10-08T00:10:51.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.288+0000 7f67bdbce700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b8152410 con 0x7f67b814e7f0 2023-10-08T00:10:51.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.288+0000 7f67bdbce700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67b80b2e70 con 0x7f67b814ff10 2023-10-08T00:10:51.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.288+0000 7f67b77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67a4004180 0x7f67b814cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:51.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.288+0000 7f67b77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67a4004180 0x7f67b814cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39166/0 (socket says 172.21.15.99:39166) 2023-10-08T00:10:51.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.288+0000 7f67b77fe700 1 -- 172.21.15.99:0/1617335315 learned_addr learned my addr 172.21.15.99:0/1617335315 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:10:51.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.288+0000 7f67affff700 1 --2- 172.21.15.99:0/1617335315 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67b814e7f0 0x7f67b80b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:51.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.288+0000 7f67b7fff700 1 --2- 172.21.15.99:0/1617335315 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67b814ff10 0x7f67b81503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:51.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.288+0000 7f67b77fe700 1 -- 172.21.15.99:0/1617335315 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67b814ff10 msgr2=0x7f67b81503b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:51.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.288+0000 7f67b77fe700 1 --2- 172.21.15.99:0/1617335315 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67b814ff10 0x7f67b81503b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:51.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.288+0000 7f67b77fe700 1 -- 172.21.15.99:0/1617335315 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67b814e7f0 msgr2=0x7f67b80b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:51.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.288+0000 7f67b77fe700 1 --2- 172.21.15.99:0/1617335315 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67b814e7f0 0x7f67b80b2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:51.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.288+0000 7f67b77fe700 1 -- 172.21.15.99:0/1617335315 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b80b3150 con 0x7f67a4004180 2023-10-08T00:10:51.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.288+0000 7f67b77fe700 1 --2- 172.21.15.99:0/1617335315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67a4004180 0x7f67b814cf80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f67a8002b50 tx=0x7f67a8007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:51.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.288+0000 7f67af7fe700 1 -- 172.21.15.99:0/1617335315 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a800f9a0 con 0x7f67a4004180 2023-10-08T00:10:51.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.288+0000 7f67af7fe700 1 -- 172.21.15.99:0/1617335315 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67a800fb20 con 0x7f67a4004180 2023-10-08T00:10:51.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.288+0000 7f67bdbce700 1 -- 172.21.15.99:0/1617335315 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67b80b3430 con 0x7f67a4004180 2023-10-08T00:10:51.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.288+0000 7f67af7fe700 1 -- 172.21.15.99:0/1617335315 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a800fee0 con 0x7f67a4004180 2023-10-08T00:10:51.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.288+0000 7f67bdbce700 1 -- 172.21.15.99:0/1617335315 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67b81508f0 con 0x7f67a4004180 2023-10-08T00:10:51.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.289+0000 7f67bdbce700 1 -- 172.21.15.99:0/1617335315 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67b815aa50 con 0x7f67a4004180 2023-10-08T00:10:51.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.290+0000 7f67af7fe700 1 -- 172.21.15.99:0/1617335315 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f67a8013600 con 0x7f67a4004180 2023-10-08T00:10:51.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.290+0000 7f67af7fe700 1 --2- 172.21.15.99:0/1617335315 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6798066f30 0x7f67980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:51.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.290+0000 7f67af7fe700 1 -- 172.21.15.99:0/1617335315 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f67a80848f0 con 0x7f67a4004180 2023-10-08T00:10:51.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.290+0000 7f67affff700 1 --2- 172.21.15.99:0/1617335315 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6798066f30 0x7f67980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:51.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.291+0000 7f67affff700 1 --2- 172.21.15.99:0/1617335315 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6798066f30 0x7f67980693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f67a0008fd0 tx=0x7f67a0015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:51.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.294+0000 7f67af7fe700 1 -- 172.21.15.99:0/1617335315 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f67a8051eb0 con 0x7f67a4004180 2023-10-08T00:10:51.480 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.479+0000 7f67bdbce700 1 -- 172.21.15.99:0/1617335315 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f67b804c3c0 con 0x7f67a4004180 2023-10-08T00:10:51.480 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.480+0000 7f67af7fe700 1 -- 172.21.15.99:0/1617335315 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7f67a8017090 con 0x7f67a4004180 2023-10-08T00:10:51.480 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:10:51.481 INFO:teuthology.orchestra.run.smithi099.stdout:[ 2023-10-08T00:10:51.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:10:51.481 INFO:teuthology.orchestra.run.smithi099.stdout: "poolnum": 1, 2023-10-08T00:10:51.481 INFO:teuthology.orchestra.run.smithi099.stdout: "poolname": ".mgr" 2023-10-08T00:10:51.481 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:10:51.481 INFO:teuthology.orchestra.run.smithi099.stdout:] 2023-10-08T00:10:51.482 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.481+0000 7f67bdbce700 1 -- 172.21.15.99:0/1617335315 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6798066f30 msgr2=0x7f67980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:51.483 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.481+0000 7f67bdbce700 1 --2- 172.21.15.99:0/1617335315 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6798066f30 0x7f67980693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f67a0008fd0 tx=0x7f67a0015000 comp rx=0 tx=0).stop 2023-10-08T00:10:51.483 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.481+0000 7f67bdbce700 1 -- 172.21.15.99:0/1617335315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67a4004180 msgr2=0x7f67b814cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:51.483 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.481+0000 7f67bdbce700 1 --2- 172.21.15.99:0/1617335315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67a4004180 0x7f67b814cf80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f67a8002b50 tx=0x7f67a8007d30 comp rx=0 tx=0).stop 2023-10-08T00:10:51.483 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.481+0000 7f67bdbce700 1 -- 172.21.15.99:0/1617335315 shutdown_connections 2023-10-08T00:10:51.483 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.481+0000 7f67bdbce700 1 --2- 172.21.15.99:0/1617335315 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6798066f30 0x7f67980693e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:51.483 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.481+0000 7f67bdbce700 1 --2- 172.21.15.99:0/1617335315 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67b814ff10 0x7f67b81503b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:51.483 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.481+0000 7f67bdbce700 1 --2- 172.21.15.99:0/1617335315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67a4004180 0x7f67b814cf80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:51.483 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.481+0000 7f67bdbce700 1 --2- 172.21.15.99:0/1617335315 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67b814e7f0 0x7f67b80b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:51.483 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.481+0000 7f67bdbce700 1 -- 172.21.15.99:0/1617335315 >> 172.21.15.99:0/1617335315 conn(0x7f67b8068ce0 msgr2=0x7f67b80690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:51.483 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.481+0000 7f67bdbce700 1 -- 172.21.15.99:0/1617335315 shutdown_connections 2023-10-08T00:10:51.483 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.481+0000 7f67bdbce700 1 -- 172.21.15.99:0/1617335315 wait complete. 2023-10-08T00:10:51.494 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T00:10:51.494 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T00:10:51.612 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.611+0000 7fbe72ba3700 1 Processor -- start 2023-10-08T00:10:51.612 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.611+0000 7fbe72ba3700 1 -- start start 2023-10-08T00:10:51.614 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.612+0000 7fbe72ba3700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbe6c15d240 0x7fbe6c0b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:51.615 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.612+0000 7fbe72ba3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbe6c0cc940 0x7fbe6c0ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:51.615 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.612+0000 7fbe72ba3700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fbe6c0c59e0 0x7fbe6c0c5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:51.615 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.612+0000 7fbe72ba3700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe6c0b3f40 con 0x7fbe6c15d240 2023-10-08T00:10:51.615 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.612+0000 7fbe72ba3700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe6c0cbb80 con 0x7fbe6c0c59e0 2023-10-08T00:10:51.615 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.612+0000 7fbe72ba3700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6c0cbe60 con 0x7fbe6c0cc940 2023-10-08T00:10:51.615 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.612+0000 7fbe71ba1700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbe6c15d240 0x7fbe6c0b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:51.615 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.612+0000 7fbe71ba1700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbe6c15d240 0x7fbe6c0b3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39174/0 (socket says 172.21.15.99:39174) 2023-10-08T00:10:51.615 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.612+0000 7fbe71ba1700 1 -- 172.21.15.99:0/1436706008 learned_addr learned my addr 172.21.15.99:0/1436706008 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:10:51.615 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.612+0000 7fbe713a0700 1 --2- 172.21.15.99:0/1436706008 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbe6c0cc940 0x7fbe6c0ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:51.615 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.612+0000 7fbe723a2700 1 --2- 172.21.15.99:0/1436706008 >> v2:172.21.15.191:3300/0 conn(0x7fbe6c0c59e0 0x7fbe6c0c5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:51.616 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.612+0000 7fbe71ba1700 1 -- 172.21.15.99:0/1436706008 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbe6c0cc940 msgr2=0x7fbe6c0ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:51.616 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.612+0000 7fbe71ba1700 1 --2- 172.21.15.99:0/1436706008 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbe6c0cc940 0x7fbe6c0ccdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:51.616 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.612+0000 7fbe71ba1700 1 -- 172.21.15.99:0/1436706008 >> v2:172.21.15.191:3300/0 conn(0x7fbe6c0c59e0 msgr2=0x7fbe6c0c5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:51.616 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.612+0000 7fbe71ba1700 1 --2- 172.21.15.99:0/1436706008 >> v2:172.21.15.191:3300/0 conn(0x7fbe6c0c59e0 0x7fbe6c0c5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:51.616 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.612+0000 7fbe71ba1700 1 -- 172.21.15.99:0/1436706008 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe6c0cc140 con 0x7fbe6c15d240 2023-10-08T00:10:51.616 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.612+0000 7fbe723a2700 1 --2- 172.21.15.99:0/1436706008 >> v2:172.21.15.191:3300/0 conn(0x7fbe6c0c59e0 0x7fbe6c0c5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:10:51.616 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.613+0000 7fbe71ba1700 1 --2- 172.21.15.99:0/1436706008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe6c15d240 0x7fbe6c0b3a00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c08a4b0 tx=0x7fbe68009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d84b70012022934a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:51.616 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.613+0000 7fbe70b9f700 1 -- 172.21.15.99:0/1436706008 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe68005c10 con 0x7fbe6c15d240 2023-10-08T00:10:51.616 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.613+0000 7fbe70b9f700 1 -- 172.21.15.99:0/1436706008 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe68005d70 con 0x7fbe6c15d240 2023-10-08T00:10:51.616 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.613+0000 7fbe70b9f700 1 -- 172.21.15.99:0/1436706008 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6800b880 con 0x7fbe6c15d240 2023-10-08T00:10:51.616 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.613+0000 7fbe72ba3700 1 -- 172.21.15.99:0/1436706008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe6c15d240 msgr2=0x7fbe6c0b3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:51.616 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.613+0000 7fbe72ba3700 1 --2- 172.21.15.99:0/1436706008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe6c15d240 0x7fbe6c0b3a00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c08a4b0 tx=0x7fbe68009220 comp rx=0 tx=0).stop 2023-10-08T00:10:51.617 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.613+0000 7fbe72ba3700 1 -- 172.21.15.99:0/1436706008 shutdown_connections 2023-10-08T00:10:51.617 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.613+0000 7fbe72ba3700 1 --2- 172.21.15.99:0/1436706008 >> v2:172.21.15.191:3300/0 conn(0x7fbe6c0c59e0 0x7fbe6c0c5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:51.617 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.613+0000 7fbe72ba3700 1 --2- 172.21.15.99:0/1436706008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe6c15d240 0x7fbe6c0b3a00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:51.617 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.613+0000 7fbe72ba3700 1 --2- 172.21.15.99:0/1436706008 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbe6c0cc940 0x7fbe6c0ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:51.617 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.613+0000 7fbe72ba3700 1 -- 172.21.15.99:0/1436706008 >> 172.21.15.99:0/1436706008 conn(0x7fbe6c067e40 msgr2=0x7fbe6c0d0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:51.617 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.613+0000 7fbe72ba3700 1 -- 172.21.15.99:0/1436706008 shutdown_connections 2023-10-08T00:10:51.617 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.613+0000 7fbe72ba3700 1 -- 172.21.15.99:0/1436706008 wait complete. 2023-10-08T00:10:51.620 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.614+0000 7fbe72ba3700 1 Processor -- start 2023-10-08T00:10:51.620 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.614+0000 7fbe72ba3700 1 -- start start 2023-10-08T00:10:51.620 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.615+0000 7fbe72ba3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe6c0c59e0 0x7fbe6c09b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:51.620 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.615+0000 7fbe72ba3700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe6c0cc940 0x7fbe6c158b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:51.620 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.615+0000 7fbe72ba3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbe6c15d240 0x7fbe6c09c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:51.620 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.615+0000 7fbe72ba3700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6c16e330 con 0x7fbe6c0c59e0 2023-10-08T00:10:51.620 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.615+0000 7fbe72ba3700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6c16d720 con 0x7fbe6c0cc940 2023-10-08T00:10:51.621 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.615+0000 7fbe72ba3700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6c16da00 con 0x7fbe6c15d240 2023-10-08T00:10:51.621 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.615+0000 7fbe71ba1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe6c0c59e0 0x7fbe6c09b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:51.621 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.615+0000 7fbe71ba1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe6c0c59e0 0x7fbe6c09b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39176/0 (socket says 172.21.15.99:39176) 2023-10-08T00:10:51.621 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.615+0000 7fbe71ba1700 1 -- 172.21.15.99:0/1734230531 learned_addr learned my addr 172.21.15.99:0/1734230531 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:10:51.621 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.615+0000 7fbe713a0700 1 --2- 172.21.15.99:0/1734230531 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe6c0cc940 0x7fbe6c158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:51.621 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.615+0000 7fbe723a2700 1 --2- 172.21.15.99:0/1734230531 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbe6c15d240 0x7fbe6c09c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:51.621 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.615+0000 7fbe71ba1700 1 -- 172.21.15.99:0/1734230531 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbe6c15d240 msgr2=0x7fbe6c09c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:51.621 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.615+0000 7fbe71ba1700 1 --2- 172.21.15.99:0/1734230531 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbe6c15d240 0x7fbe6c09c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:51.621 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.615+0000 7fbe71ba1700 1 -- 172.21.15.99:0/1734230531 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe6c0cc940 msgr2=0x7fbe6c158b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:51.621 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.615+0000 7fbe71ba1700 1 --2- 172.21.15.99:0/1734230531 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe6c0cc940 0x7fbe6c158b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:51.621 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.615+0000 7fbe71ba1700 1 -- 172.21.15.99:0/1734230531 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe68009480 con 0x7fbe6c0c59e0 2023-10-08T00:10:51.621 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.616+0000 7fbe723a2700 1 --2- 172.21.15.99:0/1734230531 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbe6c15d240 0x7fbe6c09c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:10:51.622 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.616+0000 7fbe71ba1700 1 --2- 172.21.15.99:0/1734230531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe6c0c59e0 0x7fbe6c09b8b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbe6800b3c0 tx=0x7fbe68016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:51.622 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.616+0000 7fbe61ffb700 1 -- 172.21.15.99:0/1734230531 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe68011a80 con 0x7fbe6c0c59e0 2023-10-08T00:10:51.622 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.616+0000 7fbe61ffb700 1 -- 172.21.15.99:0/1734230531 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe68011be0 con 0x7fbe6c0c59e0 2023-10-08T00:10:51.622 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.616+0000 7fbe72ba3700 1 -- 172.21.15.99:0/1734230531 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe6c16dce0 con 0x7fbe6c0c59e0 2023-10-08T00:10:51.622 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.616+0000 7fbe61ffb700 1 -- 172.21.15.99:0/1734230531 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6801b430 con 0x7fbe6c0c59e0 2023-10-08T00:10:51.622 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.616+0000 7fbe72ba3700 1 -- 172.21.15.99:0/1734230531 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe6c09bdf0 con 0x7fbe6c0c59e0 2023-10-08T00:10:51.622 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.617+0000 7fbe72ba3700 1 -- 172.21.15.99:0/1734230531 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe6c09c010 con 0x7fbe6c0c59e0 2023-10-08T00:10:51.622 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.617+0000 7fbe61ffb700 1 -- 172.21.15.99:0/1734230531 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbe6801b860 con 0x7fbe6c0c59e0 2023-10-08T00:10:51.623 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.618+0000 7fbe61ffb700 1 --2- 172.21.15.99:0/1734230531 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbe50066f30 0x7fbe500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:51.623 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.618+0000 7fbe61ffb700 1 -- 172.21.15.99:0/1734230531 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fbe68085a80 con 0x7fbe6c0c59e0 2023-10-08T00:10:51.623 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.618+0000 7fbe713a0700 1 --2- 172.21.15.99:0/1734230531 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbe50066f30 0x7fbe500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:51.623 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.618+0000 7fbe713a0700 1 --2- 172.21.15.99:0/1734230531 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbe50066f30 0x7fbe500693e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c089740 tx=0x7fbe5c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:51.623 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.622+0000 7fbe61ffb700 1 -- 172.21.15.99:0/1734230531 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbe68053040 con 0x7fbe6c0c59e0 2023-10-08T00:10:51.805 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:51.804+0000 7fbe72ba3700 1 -- 172.21.15.99:0/1734230531 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fbe6c06ac60 con 0x7fbe6c0c59e0 2023-10-08T00:10:52.068 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.067+0000 7fbe61ffb700 1 -- 172.21.15.99:0/1734230531 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7fbe6802a0b0 con 0x7fbe6c0c59e0 2023-10-08T00:10:52.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.071+0000 7fbe72ba3700 1 -- 172.21.15.99:0/1734230531 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbe50066f30 msgr2=0x7fbe500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:52.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.071+0000 7fbe72ba3700 1 --2- 172.21.15.99:0/1734230531 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbe50066f30 0x7fbe500693e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c089740 tx=0x7fbe5c008040 comp rx=0 tx=0).stop 2023-10-08T00:10:52.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.071+0000 7fbe72ba3700 1 -- 172.21.15.99:0/1734230531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe6c0c59e0 msgr2=0x7fbe6c09b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:52.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.071+0000 7fbe72ba3700 1 --2- 172.21.15.99:0/1734230531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe6c0c59e0 0x7fbe6c09b8b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbe6800b3c0 tx=0x7fbe68016000 comp rx=0 tx=0).stop 2023-10-08T00:10:52.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.071+0000 7fbe72ba3700 1 -- 172.21.15.99:0/1734230531 shutdown_connections 2023-10-08T00:10:52.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.071+0000 7fbe72ba3700 1 --2- 172.21.15.99:0/1734230531 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbe50066f30 0x7fbe500693e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:52.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.071+0000 7fbe72ba3700 1 --2- 172.21.15.99:0/1734230531 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbe6c15d240 0x7fbe6c09c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:52.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.071+0000 7fbe72ba3700 1 --2- 172.21.15.99:0/1734230531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe6c0c59e0 0x7fbe6c09b8b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:52.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.071+0000 7fbe72ba3700 1 --2- 172.21.15.99:0/1734230531 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe6c0cc940 0x7fbe6c158b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:52.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.071+0000 7fbe72ba3700 1 -- 172.21.15.99:0/1734230531 >> 172.21.15.99:0/1734230531 conn(0x7fbe6c067e40 msgr2=0x7fbe6c15b140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:52.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.071+0000 7fbe72ba3700 1 -- 172.21.15.99:0/1734230531 shutdown_connections 2023-10-08T00:10:52.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.072+0000 7fbe72ba3700 1 -- 172.21.15.99:0/1734230531 wait complete. 2023-10-08T00:10:52.073 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' created 2023-10-08T00:10:52.085 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T00:10:52.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.204+0000 7f3b0c9da700 1 Processor -- start 2023-10-08T00:10:52.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.204+0000 7f3b0c9da700 1 -- start start 2023-10-08T00:10:52.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.205+0000 7f3b0c9da700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b08159e60 0x7f3b0815e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:52.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.205+0000 7f3b0c9da700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f3b0815a230 0x7f3b080690b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:52.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.205+0000 7f3b0c9da700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3b080695f0 0x7f3b080af730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:52.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.205+0000 7f3b0c9da700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b080d1cf0 con 0x7f3b080695f0 2023-10-08T00:10:52.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.205+0000 7f3b0c9da700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b08069a90 con 0x7f3b0815a230 2023-10-08T00:10:52.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.205+0000 7f3b0c9da700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0815ea90 con 0x7f3b08159e60 2023-10-08T00:10:52.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.205+0000 7f3b06d9d700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3b080695f0 0x7f3b080af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:52.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.205+0000 7f3b06d9d700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3b080695f0 0x7f3b080af730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39182/0 (socket says 172.21.15.99:39182) 2023-10-08T00:10:52.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.205+0000 7f3b06d9d700 1 -- 172.21.15.99:0/1483595869 learned_addr learned my addr 172.21.15.99:0/1483595869 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:10:52.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.205+0000 7f3b05d9b700 1 --2- 172.21.15.99:0/1483595869 >> v2:172.21.15.191:3300/0 conn(0x7f3b0815a230 0x7f3b080690b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:52.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.205+0000 7f3b0659c700 1 --2- 172.21.15.99:0/1483595869 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b08159e60 0x7f3b0815e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:52.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.205+0000 7f3b06d9d700 1 -- 172.21.15.99:0/1483595869 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b08159e60 msgr2=0x7f3b0815e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:52.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.205+0000 7f3b06d9d700 1 --2- 172.21.15.99:0/1483595869 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b08159e60 0x7f3b0815e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:52.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.205+0000 7f3b06d9d700 1 -- 172.21.15.99:0/1483595869 >> v2:172.21.15.191:3300/0 conn(0x7f3b0815a230 msgr2=0x7f3b080690b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:52.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.205+0000 7f3b06d9d700 1 --2- 172.21.15.99:0/1483595869 >> v2:172.21.15.191:3300/0 conn(0x7f3b0815a230 0x7f3b080690b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:52.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.206+0000 7f3b06d9d700 1 -- 172.21.15.99:0/1483595869 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b080b3100 con 0x7f3b080695f0 2023-10-08T00:10:52.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.206+0000 7f3b06d9d700 1 --2- 172.21.15.99:0/1483595869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b080695f0 0x7f3b080af730 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3b080d0ca0 tx=0x7f3af8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a7bd91e27e8d224 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:52.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.206+0000 7f3b0559a700 1 -- 172.21.15.99:0/1483595869 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af800e070 con 0x7f3b080695f0 2023-10-08T00:10:52.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.206+0000 7f3b0559a700 1 -- 172.21.15.99:0/1483595869 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3af800b970 con 0x7f3b080695f0 2023-10-08T00:10:52.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.206+0000 7f3b0559a700 1 -- 172.21.15.99:0/1483595869 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af80092e0 con 0x7f3b080695f0 2023-10-08T00:10:52.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.206+0000 7f3b0c9da700 1 -- 172.21.15.99:0/1483595869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b080695f0 msgr2=0x7f3b080af730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:52.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.206+0000 7f3b0c9da700 1 --2- 172.21.15.99:0/1483595869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b080695f0 0x7f3b080af730 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3b080d0ca0 tx=0x7f3af8009840 comp rx=0 tx=0).stop 2023-10-08T00:10:52.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.206+0000 7f3b0c9da700 1 -- 172.21.15.99:0/1483595869 shutdown_connections 2023-10-08T00:10:52.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.206+0000 7f3b0c9da700 1 --2- 172.21.15.99:0/1483595869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b080695f0 0x7f3b080af730 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:52.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.206+0000 7f3b0c9da700 1 --2- 172.21.15.99:0/1483595869 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b08159e60 0x7f3b0815e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:52.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.206+0000 7f3b0c9da700 1 --2- 172.21.15.99:0/1483595869 >> v2:172.21.15.191:3300/0 conn(0x7f3b0815a230 0x7f3b080690b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:52.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.206+0000 7f3b0c9da700 1 -- 172.21.15.99:0/1483595869 >> 172.21.15.99:0/1483595869 conn(0x7f3b08070700 msgr2=0x7f3b08070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:52.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.206+0000 7f3b0c9da700 1 -- 172.21.15.99:0/1483595869 shutdown_connections 2023-10-08T00:10:52.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.207+0000 7f3b0c9da700 1 -- 172.21.15.99:0/1483595869 wait complete. 2023-10-08T00:10:52.212 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.208+0000 7f3b0c9da700 1 Processor -- start 2023-10-08T00:10:52.212 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.208+0000 7f3b0c9da700 1 -- start start 2023-10-08T00:10:52.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.208+0000 7f3b0c9da700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b080695f0 0x7f3b080cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:52.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.208+0000 7f3b0c9da700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b08159e60 0x7f3b080c7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:52.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.208+0000 7f3b0c9da700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b0815a230 0x7f3b080c77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:52.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.208+0000 7f3b0c9da700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b080c9b10 con 0x7f3b08159e60 2023-10-08T00:10:52.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.208+0000 7f3b0c9da700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b080c9df0 con 0x7f3b0815a230 2023-10-08T00:10:52.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.208+0000 7f3b0c9da700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b080ca0d0 con 0x7f3b080695f0 2023-10-08T00:10:52.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.208+0000 7f3b05d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b08159e60 0x7f3b080c7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:52.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.208+0000 7f3b05d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b08159e60 0x7f3b080c7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39194/0 (socket says 172.21.15.99:39194) 2023-10-08T00:10:52.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.208+0000 7f3b05d9b700 1 -- 172.21.15.99:0/2344525293 learned_addr learned my addr 172.21.15.99:0/2344525293 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:10:52.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.208+0000 7f3b0659c700 1 --2- 172.21.15.99:0/2344525293 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b080695f0 0x7f3b080cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:52.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.208+0000 7f3b06d9d700 1 --2- 172.21.15.99:0/2344525293 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b0815a230 0x7f3b080c77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:52.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.209+0000 7f3b05d9b700 1 -- 172.21.15.99:0/2344525293 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b080695f0 msgr2=0x7f3b080cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:52.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.209+0000 7f3b05d9b700 1 --2- 172.21.15.99:0/2344525293 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b080695f0 0x7f3b080cabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:52.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.209+0000 7f3b05d9b700 1 -- 172.21.15.99:0/2344525293 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b0815a230 msgr2=0x7f3b080c77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:52.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.209+0000 7f3b05d9b700 1 --2- 172.21.15.99:0/2344525293 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b0815a230 0x7f3b080c77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:52.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.209+0000 7f3b05d9b700 1 -- 172.21.15.99:0/2344525293 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af8009ae0 con 0x7f3b08159e60 2023-10-08T00:10:52.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.209+0000 7f3b06d9d700 1 --2- 172.21.15.99:0/2344525293 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b0815a230 0x7f3b080c77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:10:52.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.209+0000 7f3b05d9b700 1 --2- 172.21.15.99:0/2344525293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b08159e60 0x7f3b080c7280 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3afc00ec50 tx=0x7f3afc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:52.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.209+0000 7f3af67fc700 1 -- 172.21.15.99:0/2344525293 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3afc00c850 con 0x7f3b08159e60 2023-10-08T00:10:52.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.209+0000 7f3af67fc700 1 -- 172.21.15.99:0/2344525293 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3afc00c9b0 con 0x7f3b08159e60 2023-10-08T00:10:52.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.209+0000 7f3b0c9da700 1 -- 172.21.15.99:0/2344525293 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b080ca3b0 con 0x7f3b08159e60 2023-10-08T00:10:52.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.209+0000 7f3af67fc700 1 -- 172.21.15.99:0/2344525293 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3afc00cd70 con 0x7f3b08159e60 2023-10-08T00:10:52.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.209+0000 7f3b0c9da700 1 -- 172.21.15.99:0/2344525293 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b080cb0f0 con 0x7f3b08159e60 2023-10-08T00:10:52.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.210+0000 7f3b0c9da700 1 -- 172.21.15.99:0/2344525293 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b080ca8d0 con 0x7f3b08159e60 2023-10-08T00:10:52.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.210+0000 7f3af67fc700 1 -- 172.21.15.99:0/2344525293 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3afc013680 con 0x7f3b08159e60 2023-10-08T00:10:52.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.211+0000 7f3af67fc700 1 --2- 172.21.15.99:0/2344525293 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3ae8066f30 0x7f3ae80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:52.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.211+0000 7f3af67fc700 1 -- 172.21.15.99:0/2344525293 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f3afc084680 con 0x7f3b08159e60 2023-10-08T00:10:52.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.211+0000 7f3b0659c700 1 --2- 172.21.15.99:0/2344525293 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3ae8066f30 0x7f3ae80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:52.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.212+0000 7f3b0659c700 1 --2- 172.21.15.99:0/2344525293 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3ae8066f30 0x7f3ae80693e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3b00009db0 tx=0x7f3b000091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:52.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.215+0000 7f3af67fc700 1 -- 172.21.15.99:0/2344525293 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3b080ca8d0 con 0x7f3b08159e60 2023-10-08T00:10:52.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:52.397+0000 7f3b0c9da700 1 -- 172.21.15.99:0/2344525293 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f3b080ca8d0 con 0x7f3b08159e60 2023-10-08T00:10:52.483 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:10:52.486 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:10:53.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.071+0000 7f3af67fc700 1 -- 172.21.15.99:0/2344525293 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) v1 ==== 170+0+0 (secure 0 0 0) 0x7f3b080ca8d0 con 0x7f3b08159e60 2023-10-08T00:10:53.077 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.076+0000 7f3b0c9da700 1 -- 172.21.15.99:0/2344525293 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3ae8066f30 msgr2=0x7f3ae80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:53.078 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.076+0000 7f3b0c9da700 1 --2- 172.21.15.99:0/2344525293 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3ae8066f30 0x7f3ae80693e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3b00009db0 tx=0x7f3b000091a0 comp rx=0 tx=0).stop 2023-10-08T00:10:53.078 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.076+0000 7f3b0c9da700 1 -- 172.21.15.99:0/2344525293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b08159e60 msgr2=0x7f3b080c7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:53.078 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.076+0000 7f3b0c9da700 1 --2- 172.21.15.99:0/2344525293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b08159e60 0x7f3b080c7280 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3afc00ec50 tx=0x7f3afc0091a0 comp rx=0 tx=0).stop 2023-10-08T00:10:53.078 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.077+0000 7f3b0c9da700 1 -- 172.21.15.99:0/2344525293 shutdown_connections 2023-10-08T00:10:53.078 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.077+0000 7f3b0c9da700 1 --2- 172.21.15.99:0/2344525293 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3ae8066f30 0x7f3ae80693e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:53.078 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.077+0000 7f3b0c9da700 1 --2- 172.21.15.99:0/2344525293 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b0815a230 0x7f3b080c77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:53.078 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.077+0000 7f3b0c9da700 1 --2- 172.21.15.99:0/2344525293 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b080695f0 0x7f3b080cabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:53.078 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.077+0000 7f3b0c9da700 1 --2- 172.21.15.99:0/2344525293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b08159e60 0x7f3b080c7280 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:53.078 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.077+0000 7f3b0c9da700 1 -- 172.21.15.99:0/2344525293 >> 172.21.15.99:0/2344525293 conn(0x7f3b08070700 msgr2=0x7f3b08070ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:53.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.079+0000 7f3b0c9da700 1 -- 172.21.15.99:0/2344525293 shutdown_connections 2023-10-08T00:10:53.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.080+0000 7f3b0c9da700 1 -- 172.21.15.99:0/2344525293 wait complete. 2023-10-08T00:10:53.085 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' created 2023-10-08T00:10:53.098 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T00:10:53.218 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.217+0000 7fb2cb59e700 1 Processor -- start 2023-10-08T00:10:53.218 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.217+0000 7fb2cb59e700 1 -- start start 2023-10-08T00:10:53.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.218+0000 7fb2cb59e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb2cc0b6580 0x7fb2cc06a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:53.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.218+0000 7fb2cb59e700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2cc06ac90 con 0x7fb2cc0cce30 2023-10-08T00:10:53.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.218+0000 7fb2cb59e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2cc0b4d70 con 0x7fb2cc06a830 2023-10-08T00:10:53.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.218+0000 7fb2cb59e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2cc15b3b0 con 0x7fb2cc0b6580 2023-10-08T00:10:53.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.218+0000 7fb2ca59c700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fb2cc0cce30 0x7fb2cc15aca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:38450/0 (socket says 172.21.15.99:38450) 2023-10-08T00:10:53.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.218+0000 7fb2ca59c700 1 -- 172.21.15.99:0/2600913654 learned_addr learned my addr 172.21.15.99:0/2600913654 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:10:53.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.218+0000 7fb2c9d9b700 1 -- 172.21.15.99:0/2600913654 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1173882547 0 0) 0x7fb2bc001110 con 0x7fb2cc0cce30 2023-10-08T00:10:53.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.218+0000 7fb2c1d9b700 1 --2- 172.21.15.99:0/2600913654 >> v2:172.21.15.191:3300/0 conn(0x7fb2cc0b6580 0x7fb2cc06a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:53.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.218+0000 7fb2c9d9b700 1 -- 172.21.15.99:0/2600913654 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2b8003570 con 0x7fb2cc0cce30 2023-10-08T00:10:53.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.218+0000 7fb2c9d9b700 1 -- 172.21.15.99:0/2600913654 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3610468021 0 0) 0x7fb2b8003570 con 0x7fb2cc0cce30 2023-10-08T00:10:53.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.218+0000 7fb2c9d9b700 1 -- 172.21.15.99:0/2600913654 >> v2:172.21.15.191:3300/0 conn(0x7fb2cc0b6580 msgr2=0x7fb2cc06a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:53.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.218+0000 7fb2c9d9b700 1 --2- 172.21.15.99:0/2600913654 >> v2:172.21.15.191:3300/0 conn(0x7fb2cc0b6580 0x7fb2cc06a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:53.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.218+0000 7fb2c9d9b700 1 -- 172.21.15.99:0/2600913654 >> v1:172.21.15.191:6789/0 conn(0x7fb2cc06a830 legacy=0x7fb2cc0b5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:53.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.218+0000 7fb2c9d9b700 1 -- 172.21.15.99:0/2600913654 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2cc0cd8d0 con 0x7fb2cc0cce30 2023-10-08T00:10:53.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.218+0000 7fb2c9d9b700 1 -- 172.21.15.99:0/2600913654 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7fb2bc001600 con 0x7fb2cc0cce30 2023-10-08T00:10:53.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.219+0000 7fb2c9d9b700 1 -- 172.21.15.99:0/2600913654 >> v1:172.21.15.99:6789/0 conn(0x7fb2cc0cce30 legacy=0x7fb2cc15aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:53.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.219+0000 7fb2c9d9b700 1 --2- 172.21.15.99:0/2600913654 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2b8003a60 0x7fb2b8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:53.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.219+0000 7fb2c9d9b700 1 --2- 172.21.15.99:0/2600913654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2b8004810 0x7fb2b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:53.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.219+0000 7fb2c9d9b700 1 --2- 172.21.15.99:0/2600913654 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2b80051c0 0x7fb2b800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:53.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.219+0000 7fb2c9d9b700 1 -- 172.21.15.99:0/2600913654 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2cc15b3b0 con 0x7fb2b8004810 2023-10-08T00:10:53.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.219+0000 7fb2c9d9b700 1 -- 172.21.15.99:0/2600913654 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b800c0e0 con 0x7fb2b80051c0 2023-10-08T00:10:53.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.219+0000 7fb2c9d9b700 1 -- 172.21.15.99:0/2600913654 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b800c2a0 con 0x7fb2b8003a60 2023-10-08T00:10:53.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.219+0000 7fb2c1d9b700 1 --2- 172.21.15.99:0/2600913654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2b8004810 0x7fb2b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:53.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.219+0000 7fb2ca59c700 1 --2- 172.21.15.99:0/2600913654 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2b8003a60 0x7fb2b8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:53.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.219+0000 7fb2cad9d700 1 --2- 172.21.15.99:0/2600913654 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2b80051c0 0x7fb2b800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:53.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.219+0000 7fb2c1d9b700 1 -- 172.21.15.99:0/2600913654 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2b8003a60 msgr2=0x7fb2b8003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:53.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.219+0000 7fb2c1d9b700 1 --2- 172.21.15.99:0/2600913654 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2b8003a60 0x7fb2b8003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:53.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.219+0000 7fb2c1d9b700 1 -- 172.21.15.99:0/2600913654 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2b80051c0 msgr2=0x7fb2b800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:53.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.219+0000 7fb2c1d9b700 1 --2- 172.21.15.99:0/2600913654 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2b80051c0 0x7fb2b800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:53.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.219+0000 7fb2cad9d700 1 -- 172.21.15.99:0/2600913654 reap_dead start 2023-10-08T00:10:53.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.219+0000 7fb2c1d9b700 1 -- 172.21.15.99:0/2600913654 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b8003570 con 0x7fb2b8004810 2023-10-08T00:10:53.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.219+0000 7fb2c1d9b700 1 --2- 172.21.15.99:0/2600913654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2b8004810 0x7fb2b8004c80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4009c00 tx=0x7fb2b40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1063395c83093cb3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:53.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.219+0000 7fb2c9d9b700 1 -- 172.21.15.99:0/2600913654 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b400e710 con 0x7fb2b8004810 2023-10-08T00:10:53.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.219+0000 7fb2c9d9b700 1 -- 172.21.15.99:0/2600913654 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2b400e870 con 0x7fb2b8004810 2023-10-08T00:10:53.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.219+0000 7fb2c9d9b700 1 -- 172.21.15.99:0/2600913654 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b400ec30 con 0x7fb2b8004810 2023-10-08T00:10:53.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.220+0000 7fb2cb59e700 1 -- 172.21.15.99:0/2600913654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2b8004810 msgr2=0x7fb2b8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:53.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.220+0000 7fb2cb59e700 1 --2- 172.21.15.99:0/2600913654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2b8004810 0x7fb2b8004c80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4009c00 tx=0x7fb2b40091a0 comp rx=0 tx=0).stop 2023-10-08T00:10:53.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.220+0000 7fb2cb59e700 1 -- 172.21.15.99:0/2600913654 shutdown_connections 2023-10-08T00:10:53.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.220+0000 7fb2cb59e700 1 --2- 172.21.15.99:0/2600913654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2b8004810 0x7fb2b8004c80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:53.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.220+0000 7fb2cb59e700 1 -- 172.21.15.99:0/2600913654 >> 172.21.15.99:0/2600913654 conn(0x7fb2cc068cb0 msgr2=0x7fb2cc0690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:53.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.220+0000 7fb2cb59e700 1 -- 172.21.15.99:0/2600913654 shutdown_connections 2023-10-08T00:10:53.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.220+0000 7fb2cb59e700 1 -- 172.21.15.99:0/2600913654 wait complete. 2023-10-08T00:10:53.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.221+0000 7fb2cb59e700 1 Processor -- start 2023-10-08T00:10:53.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.221+0000 7fb2cb59e700 1 -- start start 2023-10-08T00:10:53.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.222+0000 7fb2cb59e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2b8004810 0x7fb2cc14e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:53.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.222+0000 7fb2cb59e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2cc155c70 0x7fb2cc1560e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:53.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.222+0000 7fb2cb59e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2cc0b29c0 0x7fb2cc0b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:53.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.222+0000 7fb2cb59e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2cc157080 con 0x7fb2cc155c70 2023-10-08T00:10:53.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.222+0000 7fb2cb59e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2cc0b0640 con 0x7fb2cc0b29c0 2023-10-08T00:10:53.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.222+0000 7fb2cb59e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2cc0b3350 con 0x7fb2b8004810 2023-10-08T00:10:53.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.222+0000 7fb2c1d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2cc155c70 0x7fb2cc1560e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:53.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.222+0000 7fb2c1d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2cc155c70 0x7fb2cc1560e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39222/0 (socket says 172.21.15.99:39222) 2023-10-08T00:10:53.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.222+0000 7fb2c1d9b700 1 -- 172.21.15.99:0/3575334259 learned_addr learned my addr 172.21.15.99:0/3575334259 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:10:53.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.222+0000 7fb2cad9d700 1 --2- 172.21.15.99:0/3575334259 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2cc0b29c0 0x7fb2cc0b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:53.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.222+0000 7fb2ca59c700 1 --2- 172.21.15.99:0/3575334259 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2b8004810 0x7fb2cc14e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:53.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.222+0000 7fb2c1d9b700 1 -- 172.21.15.99:0/3575334259 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2b8004810 msgr2=0x7fb2cc14e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:53.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.222+0000 7fb2c1d9b700 1 --2- 172.21.15.99:0/3575334259 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2b8004810 0x7fb2cc14e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:53.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.222+0000 7fb2c1d9b700 1 -- 172.21.15.99:0/3575334259 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2cc0b29c0 msgr2=0x7fb2cc0b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:53.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.222+0000 7fb2c1d9b700 1 --2- 172.21.15.99:0/3575334259 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2cc0b29c0 0x7fb2cc0b2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:53.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.222+0000 7fb2c1d9b700 1 -- 172.21.15.99:0/3575334259 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2cc154b40 con 0x7fb2cc155c70 2023-10-08T00:10:53.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.222+0000 7fb2c1d9b700 1 --2- 172.21.15.99:0/3575334259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2cc155c70 0x7fb2cc1560e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb2b400e4b0 tx=0x7fb2b401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:53.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.222+0000 7fb2c2ffd700 1 -- 172.21.15.99:0/3575334259 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b4022070 con 0x7fb2cc155c70 2023-10-08T00:10:53.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.222+0000 7fb2c2ffd700 1 -- 172.21.15.99:0/3575334259 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2b400e9e0 con 0x7fb2cc155c70 2023-10-08T00:10:53.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.222+0000 7fb2c2ffd700 1 -- 172.21.15.99:0/3575334259 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b401c450 con 0x7fb2cc155c70 2023-10-08T00:10:53.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.222+0000 7fb2cb59e700 1 -- 172.21.15.99:0/3575334259 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2cc1567e0 con 0x7fb2cc155c70 2023-10-08T00:10:53.231 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.223+0000 7fb2cb59e700 1 -- 172.21.15.99:0/3575334259 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2cc156da0 con 0x7fb2cc155c70 2023-10-08T00:10:53.231 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.223+0000 7fb2cb59e700 1 -- 172.21.15.99:0/3575334259 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2cc089550 con 0x7fb2cc155c70 2023-10-08T00:10:53.231 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.224+0000 7fb2c2ffd700 1 -- 172.21.15.99:0/3575334259 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb2b401c8a0 con 0x7fb2cc155c70 2023-10-08T00:10:53.231 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.224+0000 7fb2c2ffd700 1 --2- 172.21.15.99:0/3575334259 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb2ac066f30 0x7fb2ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:53.231 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.224+0000 7fb2c2ffd700 1 -- 172.21.15.99:0/3575334259 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5605+0+0 (secure 0 0 0) 0x7fb2b4085ac0 con 0x7fb2cc155c70 2023-10-08T00:10:53.231 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.224+0000 7fb2ca59c700 1 --2- 172.21.15.99:0/3575334259 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb2ac066f30 0x7fb2ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:53.232 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.225+0000 7fb2ca59c700 1 --2- 172.21.15.99:0/3575334259 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb2ac066f30 0x7fb2ac0693e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc007fd0 tx=0x7fb2cc0ce540 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:53.232 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.229+0000 7fb2c2ffd700 1 -- 172.21.15.99:0/3575334259 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb2b4052f00 con 0x7fb2cc155c70 2023-10-08T00:10:53.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:53.452+0000 7fb2cb59e700 1 -- 172.21.15.99:0/3575334259 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb2cc0b0780 con 0x7fb2cc155c70 2023-10-08T00:10:54.083 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.082+0000 7fb2c2ffd700 1 -- 172.21.15.99:0/3575334259 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fb2b402b0c0 con 0x7fb2cc155c70 2023-10-08T00:10:54.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.091+0000 7fb2cb59e700 1 -- 172.21.15.99:0/3575334259 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb2ac066f30 msgr2=0x7fb2ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:54.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.091+0000 7fb2cb59e700 1 --2- 172.21.15.99:0/3575334259 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb2ac066f30 0x7fb2ac0693e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc007fd0 tx=0x7fb2cc0ce540 comp rx=0 tx=0).stop 2023-10-08T00:10:54.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.091+0000 7fb2cb59e700 1 -- 172.21.15.99:0/3575334259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2cc155c70 msgr2=0x7fb2cc1560e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:54.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.091+0000 7fb2cb59e700 1 --2- 172.21.15.99:0/3575334259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2cc155c70 0x7fb2cc1560e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb2b400e4b0 tx=0x7fb2b401a000 comp rx=0 tx=0).stop 2023-10-08T00:10:54.098 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.098+0000 7fb2cb59e700 1 -- 172.21.15.99:0/3575334259 shutdown_connections 2023-10-08T00:10:54.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.098+0000 7fb2cb59e700 1 --2- 172.21.15.99:0/3575334259 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb2ac066f30 0x7fb2ac0693e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:54.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.098+0000 7fb2cb59e700 1 --2- 172.21.15.99:0/3575334259 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2cc0b29c0 0x7fb2cc0b2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:54.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.098+0000 7fb2cb59e700 1 --2- 172.21.15.99:0/3575334259 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2b8004810 0x7fb2cc14e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:54.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.098+0000 7fb2cb59e700 1 --2- 172.21.15.99:0/3575334259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2cc155c70 0x7fb2cc1560e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:54.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.098+0000 7fb2cb59e700 1 -- 172.21.15.99:0/3575334259 >> 172.21.15.99:0/3575334259 conn(0x7fb2cc068cb0 msgr2=0x7fb2cc150670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:54.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.099+0000 7fb2cb59e700 1 -- 172.21.15.99:0/3575334259 shutdown_connections 2023-10-08T00:10:54.100 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.100+0000 7fb2cb59e700 1 -- 172.21.15.99:0/3575334259 wait complete. 2023-10-08T00:10:54.101 INFO:teuthology.orchestra.run.smithi099.stderr:new fs with metadata pool 2 and data pool 3 2023-10-08T00:10:54.113 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:10:54.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.226+0000 7f6891de7700 1 Processor -- start 2023-10-08T00:10:54.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.226+0000 7f6891de7700 1 -- start start 2023-10-08T00:10:54.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.227+0000 7f6891de7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f688c157ba0 0x7f688c06a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:54.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.227+0000 7f6891de7700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f688c15b0d0 0x7f688c06aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:54.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.227+0000 7f6891de7700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f688c15c360 con 0x7f688c1577d0 2023-10-08T00:10:54.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.227+0000 7f6891de7700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f688c15b790 con 0x7f688c15b0d0 2023-10-08T00:10:54.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.227+0000 7f6891de7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f688c15d800 con 0x7f688c157ba0 2023-10-08T00:10:54.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.227+0000 7f688bfff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f688c15b0d0 0x7f688c06aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:54.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.227+0000 7f688bfff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f688c15b0d0 0x7f688c06aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49380/0 (socket says 172.21.15.99:49380) 2023-10-08T00:10:54.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.227+0000 7f688bfff700 1 -- 172.21.15.99:0/3855387422 learned_addr learned my addr 172.21.15.99:0/3855387422 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:10:54.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.227+0000 7f688affd700 1 --2- 172.21.15.99:0/3855387422 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f688c157ba0 0x7f688c06a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:54.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.228+0000 7f688bfff700 1 -- 172.21.15.99:0/3855387422 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f688c157ba0 msgr2=0x7f688c06a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:54.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.228+0000 7f688bfff700 1 --2- 172.21.15.99:0/3855387422 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f688c157ba0 0x7f688c06a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:54.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.228+0000 7f688bfff700 1 -- 172.21.15.99:0/3855387422 >> v1:172.21.15.191:6789/0 conn(0x7f688c1577d0 legacy=0x7f688c0cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:54.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.228+0000 7f688bfff700 1 -- 172.21.15.99:0/3855387422 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f688c15ba30 con 0x7f688c15b0d0 2023-10-08T00:10:54.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.228+0000 7f688bfff700 1 --2- 172.21.15.99:0/3855387422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f688c15b0d0 0x7f688c06aa60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f687c0099b0 tx=0x7f687c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3884d83c5140b91 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:54.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.228+0000 7f688a7fc700 1 -- 172.21.15.99:0/3855387422 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687c00b860 con 0x7f688c15b0d0 2023-10-08T00:10:54.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.228+0000 7f688a7fc700 1 -- 172.21.15.99:0/3855387422 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f687c00b9c0 con 0x7f688c15b0d0 2023-10-08T00:10:54.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.228+0000 7f688a7fc700 1 -- 172.21.15.99:0/3855387422 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687c00bd60 con 0x7f688c15b0d0 2023-10-08T00:10:54.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.228+0000 7f6891de7700 1 -- 172.21.15.99:0/3855387422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f688c15b0d0 msgr2=0x7f688c06aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:54.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.228+0000 7f6891de7700 1 --2- 172.21.15.99:0/3855387422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f688c15b0d0 0x7f688c06aa60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f687c0099b0 tx=0x7f687c009250 comp rx=0 tx=0).stop 2023-10-08T00:10:54.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.228+0000 7f6891de7700 1 -- 172.21.15.99:0/3855387422 shutdown_connections 2023-10-08T00:10:54.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.228+0000 7f6891de7700 1 --2- 172.21.15.99:0/3855387422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f688c15b0d0 0x7f688c06aa60 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:54.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.228+0000 7f6891de7700 1 --2- 172.21.15.99:0/3855387422 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f688c157ba0 0x7f688c06a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:54.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.228+0000 7f6891de7700 1 -- 172.21.15.99:0/3855387422 >> 172.21.15.99:0/3855387422 conn(0x7f688c068ce0 msgr2=0x7f688c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:54.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.228+0000 7f6891de7700 1 -- 172.21.15.99:0/3855387422 shutdown_connections 2023-10-08T00:10:54.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.228+0000 7f6891de7700 1 -- 172.21.15.99:0/3855387422 wait complete. 2023-10-08T00:10:54.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.230+0000 7f6891de7700 1 Processor -- start 2023-10-08T00:10:54.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.230+0000 7f6891de7700 1 -- start start 2023-10-08T00:10:54.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.230+0000 7f6891de7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f688c1577d0 0x7f688c149920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:54.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.230+0000 7f6891de7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f688c157ba0 0x7f688c0c7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:54.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.230+0000 7f6891de7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f688c15b0d0 0x7f688c1596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:54.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.230+0000 7f6891de7700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f688c0c68a0 con 0x7f688c15b0d0 2023-10-08T00:10:54.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.230+0000 7f6891de7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f688c149e60 con 0x7f688c1577d0 2023-10-08T00:10:54.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.230+0000 7f6891de7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f688c0c5c90 con 0x7f688c157ba0 2023-10-08T00:10:54.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.230+0000 7f688bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f688c15b0d0 0x7f688c1596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:54.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.230+0000 7f688bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f688c15b0d0 0x7f688c1596f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49392/0 (socket says 172.21.15.99:49392) 2023-10-08T00:10:54.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.230+0000 7f688bfff700 1 -- 172.21.15.99:0/4123539197 learned_addr learned my addr 172.21.15.99:0/4123539197 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:10:54.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.231+0000 7f688affd700 1 --2- 172.21.15.99:0/4123539197 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f688c157ba0 0x7f688c0c7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:54.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.231+0000 7f688b7fe700 1 --2- 172.21.15.99:0/4123539197 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f688c1577d0 0x7f688c149920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:54.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.231+0000 7f688bfff700 1 -- 172.21.15.99:0/4123539197 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f688c157ba0 msgr2=0x7f688c0c7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:54.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.231+0000 7f688bfff700 1 --2- 172.21.15.99:0/4123539197 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f688c157ba0 0x7f688c0c7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:54.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.231+0000 7f688bfff700 1 -- 172.21.15.99:0/4123539197 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f688c1577d0 msgr2=0x7f688c149920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:54.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.231+0000 7f688bfff700 1 --2- 172.21.15.99:0/4123539197 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f688c1577d0 0x7f688c149920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:54.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.231+0000 7f688bfff700 1 -- 172.21.15.99:0/4123539197 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f687c0094f0 con 0x7f688c15b0d0 2023-10-08T00:10:54.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.231+0000 7f688bfff700 1 --2- 172.21.15.99:0/4123539197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f688c15b0d0 0x7f688c1596f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f688c06b580 tx=0x7f687c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:54.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.231+0000 7f68837fe700 1 -- 172.21.15.99:0/4123539197 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687c01a660 con 0x7f688c15b0d0 2023-10-08T00:10:54.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.231+0000 7f68837fe700 1 -- 172.21.15.99:0/4123539197 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f687c01a7c0 con 0x7f688c15b0d0 2023-10-08T00:10:54.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.231+0000 7f68837fe700 1 -- 172.21.15.99:0/4123539197 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687c01ab80 con 0x7f688c15b0d0 2023-10-08T00:10:54.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.231+0000 7f6891de7700 1 -- 172.21.15.99:0/4123539197 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f688c0c5f70 con 0x7f688c15b0d0 2023-10-08T00:10:54.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.231+0000 7f6891de7700 1 -- 172.21.15.99:0/4123539197 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f688c0c6530 con 0x7f688c15b0d0 2023-10-08T00:10:54.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.232+0000 7f6891de7700 1 -- 172.21.15.99:0/4123539197 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f688c0b0030 con 0x7f688c15b0d0 2023-10-08T00:10:54.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.232+0000 7f68837fe700 1 -- 172.21.15.99:0/4123539197 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f687c02e440 con 0x7f688c15b0d0 2023-10-08T00:10:54.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.233+0000 7f68837fe700 1 --2- 172.21.15.99:0/4123539197 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f686c066f30 0x7f686c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:54.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.233+0000 7f68837fe700 1 -- 172.21.15.99:0/4123539197 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f687c086650 con 0x7f688c15b0d0 2023-10-08T00:10:54.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.233+0000 7f688b7fe700 1 --2- 172.21.15.99:0/4123539197 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f686c066f30 0x7f686c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:54.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.233+0000 7f688b7fe700 1 --2- 172.21.15.99:0/4123539197 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f686c066f30 0x7f686c0693e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6884007fd0 tx=0x7f6884009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:54.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.237+0000 7f68837fe700 1 -- 172.21.15.99:0/4123539197 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f687c053940 con 0x7f688c15b0d0 2023-10-08T00:10:54.421 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.420+0000 7f6891de7700 1 -- 172.21.15.99:0/4123539197 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f688c04c3c0 con 0x7f688c15b0d0 2023-10-08T00:10:54.422 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.421+0000 7f68837fe700 1 -- 172.21.15.99:0/4123539197 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+15363 (secure 0 0 0) 0x7f687c056f60 con 0x7f688c15b0d0 2023-10-08T00:10:54.422 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:10:54.422 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":19,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:10:53.454011+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:10:51.822157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T00:10:52.415130+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:10:54.424 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.423+0000 7f6891de7700 1 -- 172.21.15.99:0/4123539197 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f686c066f30 msgr2=0x7f686c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:54.424 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.423+0000 7f6891de7700 1 --2- 172.21.15.99:0/4123539197 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f686c066f30 0x7f686c0693e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6884007fd0 tx=0x7f6884009040 comp rx=0 tx=0).stop 2023-10-08T00:10:54.424 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.423+0000 7f6891de7700 1 -- 172.21.15.99:0/4123539197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f688c15b0d0 msgr2=0x7f688c1596f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:54.424 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.423+0000 7f6891de7700 1 --2- 172.21.15.99:0/4123539197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f688c15b0d0 0x7f688c1596f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f688c06b580 tx=0x7f687c017000 comp rx=0 tx=0).stop 2023-10-08T00:10:54.425 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.423+0000 7f6891de7700 1 -- 172.21.15.99:0/4123539197 shutdown_connections 2023-10-08T00:10:54.425 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.423+0000 7f6891de7700 1 --2- 172.21.15.99:0/4123539197 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f686c066f30 0x7f686c0693e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:54.425 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.423+0000 7f6891de7700 1 --2- 172.21.15.99:0/4123539197 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f688c1577d0 0x7f688c149920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:54.425 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.423+0000 7f6891de7700 1 --2- 172.21.15.99:0/4123539197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f688c15b0d0 0x7f688c1596f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:54.425 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.423+0000 7f6891de7700 1 --2- 172.21.15.99:0/4123539197 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f688c157ba0 0x7f688c0c7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:54.425 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.423+0000 7f6891de7700 1 -- 172.21.15.99:0/4123539197 >> 172.21.15.99:0/4123539197 conn(0x7f688c068ce0 msgr2=0x7f688c0b4da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:54.425 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.423+0000 7f6891de7700 1 -- 172.21.15.99:0/4123539197 shutdown_connections 2023-10-08T00:10:54.425 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.423+0000 7f6891de7700 1 -- 172.21.15.99:0/4123539197 wait complete. 2023-10-08T00:10:54.437 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:10:54.552 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.551+0000 7f2dc8fa3700 1 Processor -- start 2023-10-08T00:10:54.552 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.551+0000 7f2dc8fa3700 1 -- start start 2023-10-08T00:10:54.555 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.552+0000 7f2dc8fa3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2dc41523e0 0x7f2dc406c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:54.555 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.552+0000 7f2dc8fa3700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dc406c010 con 0x7f2dc4158e50 2023-10-08T00:10:54.555 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.552+0000 7f2dc8fa3700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dc406bd10 con 0x7f2dc4152010 2023-10-08T00:10:54.555 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.552+0000 7f2dc8fa3700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc415c060 con 0x7f2dc41523e0 2023-10-08T00:10:54.555 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.552+0000 7f2dc2d9d700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f2dc4158e50 0x7f2dc40cdf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50704/0 (socket says 172.21.15.99:50704) 2023-10-08T00:10:54.555 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.552+0000 7f2dc2d9d700 1 -- 172.21.15.99:0/814015315 learned_addr learned my addr 172.21.15.99:0/814015315 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:10:54.555 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.553+0000 7f2dc1d9b700 1 --2- 172.21.15.99:0/814015315 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2dc41523e0 0x7f2dc406c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:54.556 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.553+0000 7f2dc159a700 1 -- 172.21.15.99:0/814015315 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1640077664 0 0) 0x7f2dc406c010 con 0x7f2dc4158e50 2023-10-08T00:10:54.556 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.553+0000 7f2dc159a700 1 -- 172.21.15.99:0/814015315 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2dac003570 con 0x7f2dc4158e50 2023-10-08T00:10:54.556 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.553+0000 7f2dc159a700 1 -- 172.21.15.99:0/814015315 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3994722717 0 0) 0x7f2dac003570 con 0x7f2dc4158e50 2023-10-08T00:10:54.556 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.553+0000 7f2dc159a700 1 -- 172.21.15.99:0/814015315 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2dc41523e0 msgr2=0x7f2dc406c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:54.556 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.553+0000 7f2dc159a700 1 --2- 172.21.15.99:0/814015315 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2dc41523e0 0x7f2dc406c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:54.556 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.553+0000 7f2dc159a700 1 -- 172.21.15.99:0/814015315 >> v1:172.21.15.191:6789/0 conn(0x7f2dc4152010 legacy=0x7f2dc415b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:54.556 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.553+0000 7f2dc159a700 1 -- 172.21.15.99:0/814015315 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc415c340 con 0x7f2dc4158e50 2023-10-08T00:10:54.556 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.553+0000 7f2dc159a700 1 -- 172.21.15.99:0/814015315 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f2dbc001470 con 0x7f2dc4158e50 2023-10-08T00:10:54.556 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.553+0000 7f2dc159a700 1 -- 172.21.15.99:0/814015315 >> v1:172.21.15.99:6789/0 conn(0x7f2dc4158e50 legacy=0x7f2dc40cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:54.556 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.553+0000 7f2dc159a700 1 --2- 172.21.15.99:0/814015315 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2dc41523e0 0x7f2dac003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:54.556 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.553+0000 7f2dc159a700 1 --2- 172.21.15.99:0/814015315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2dac003fd0 0x7f2dac004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:54.557 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.553+0000 7f2dc159a700 1 --2- 172.21.15.99:0/814015315 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2dac004d50 0x7f2dac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:54.557 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.553+0000 7f2dc159a700 1 -- 172.21.15.99:0/814015315 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc415c060 con 0x7f2dac003fd0 2023-10-08T00:10:54.557 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.553+0000 7f2dc159a700 1 -- 172.21.15.99:0/814015315 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dac00ff50 con 0x7f2dac004d50 2023-10-08T00:10:54.557 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.553+0000 7f2dc159a700 1 -- 172.21.15.99:0/814015315 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dac010110 con 0x7f2dc41523e0 2023-10-08T00:10:54.557 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.553+0000 7f2dc1d9b700 1 --2- 172.21.15.99:0/814015315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2dac003fd0 0x7f2dac004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:54.557 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.553+0000 7f2dc259c700 1 --2- 172.21.15.99:0/814015315 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2dc41523e0 0x7f2dac003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:54.557 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.553+0000 7f2dc2d9d700 1 --2- 172.21.15.99:0/814015315 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2dac004d50 0x7f2dac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:54.557 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.553+0000 7f2dc1d9b700 1 -- 172.21.15.99:0/814015315 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2dc41523e0 msgr2=0x7f2dac003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:54.557 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.553+0000 7f2dc1d9b700 1 --2- 172.21.15.99:0/814015315 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2dc41523e0 0x7f2dac003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:54.557 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.553+0000 7f2dc1d9b700 1 -- 172.21.15.99:0/814015315 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2dac004d50 msgr2=0x7f2dac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:54.557 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.553+0000 7f2dc1d9b700 1 --2- 172.21.15.99:0/814015315 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2dac004d50 0x7f2dac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:54.557 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.553+0000 7f2dc1d9b700 1 -- 172.21.15.99:0/814015315 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dac003570 con 0x7f2dac003fd0 2023-10-08T00:10:54.558 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.553+0000 7f2dc259c700 1 --2- 172.21.15.99:0/814015315 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2dc41523e0 0x7f2dac003a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:10:54.558 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.553+0000 7f2dc1d9b700 1 --2- 172.21.15.99:0/814015315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2dac003fd0 0x7f2dac004810 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2dc41527f0 tx=0x7f2db80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bdc03800364d769 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:54.558 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.554+0000 7f2dc159a700 1 -- 172.21.15.99:0/814015315 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db800e710 con 0x7f2dac003fd0 2023-10-08T00:10:54.558 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.554+0000 7f2dc159a700 1 -- 172.21.15.99:0/814015315 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2db800e870 con 0x7f2dac003fd0 2023-10-08T00:10:54.558 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.554+0000 7f2dc159a700 1 -- 172.21.15.99:0/814015315 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db800ec30 con 0x7f2dac003fd0 2023-10-08T00:10:54.558 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.554+0000 7f2dc8fa3700 1 -- 172.21.15.99:0/814015315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2dac003fd0 msgr2=0x7f2dac004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:54.558 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.554+0000 7f2dc8fa3700 1 --2- 172.21.15.99:0/814015315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2dac003fd0 0x7f2dac004810 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2dc41527f0 tx=0x7f2db80091a0 comp rx=0 tx=0).stop 2023-10-08T00:10:54.558 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.554+0000 7f2dc2d9d700 1 -- 172.21.15.99:0/814015315 reap_dead start 2023-10-08T00:10:54.558 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.554+0000 7f2dc8fa3700 1 -- 172.21.15.99:0/814015315 shutdown_connections 2023-10-08T00:10:54.558 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.554+0000 7f2dc8fa3700 1 -- 172.21.15.99:0/814015315 >> 172.21.15.99:0/814015315 conn(0x7f2dc4069870 msgr2=0x7f2dc4069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:54.558 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.554+0000 7f2dc8fa3700 1 -- 172.21.15.99:0/814015315 shutdown_connections 2023-10-08T00:10:54.558 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.554+0000 7f2dc8fa3700 1 -- 172.21.15.99:0/814015315 wait complete. 2023-10-08T00:10:54.561 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.555+0000 7f2dc8fa3700 1 Processor -- start 2023-10-08T00:10:54.561 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.555+0000 7f2dc8fa3700 1 -- start start 2023-10-08T00:10:54.561 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.556+0000 7f2dc8fa3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2dac003fd0 0x7f2dc414db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:54.561 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.556+0000 7f2dc8fa3700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2dc414e130 0x7f2dc40c7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:54.561 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.556+0000 7f2dc8fa3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2dc4067660 0x7f2dc4159440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:54.562 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.556+0000 7f2dc8fa3700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc40c38e0 con 0x7f2dc4067660 2023-10-08T00:10:54.562 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.556+0000 7f2dc8fa3700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc40c4140 con 0x7f2dc414e130 2023-10-08T00:10:54.562 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.556+0000 7f2dc8fa3700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc40c8770 con 0x7f2dac003fd0 2023-10-08T00:10:54.562 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.556+0000 7f2dc2d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2dc4067660 0x7f2dc4159440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:54.562 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.556+0000 7f2dc2d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2dc4067660 0x7f2dc4159440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49408/0 (socket says 172.21.15.99:49408) 2023-10-08T00:10:54.562 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.556+0000 7f2dc2d9d700 1 -- 172.21.15.99:0/2717228288 learned_addr learned my addr 172.21.15.99:0/2717228288 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:10:54.562 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.556+0000 7f2dc259c700 1 --2- 172.21.15.99:0/2717228288 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2dac003fd0 0x7f2dc414db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:54.562 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.556+0000 7f2dc1d9b700 1 --2- 172.21.15.99:0/2717228288 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2dc414e130 0x7f2dc40c7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:54.562 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.556+0000 7f2dc2d9d700 1 -- 172.21.15.99:0/2717228288 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2dac003fd0 msgr2=0x7f2dc414db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:54.562 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.556+0000 7f2dc2d9d700 1 --2- 172.21.15.99:0/2717228288 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2dac003fd0 0x7f2dc414db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:54.562 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.556+0000 7f2dc2d9d700 1 -- 172.21.15.99:0/2717228288 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2dc414e130 msgr2=0x7f2dc40c7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:54.563 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.556+0000 7f2dc2d9d700 1 --2- 172.21.15.99:0/2717228288 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2dc414e130 0x7f2dc40c7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:54.563 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.556+0000 7f2dc2d9d700 1 -- 172.21.15.99:0/2717228288 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc40c3af0 con 0x7f2dc4067660 2023-10-08T00:10:54.563 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.556+0000 7f2dc2d9d700 1 --2- 172.21.15.99:0/2717228288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2dc4067660 0x7f2dc4159440 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc000c00 tx=0x7f2dbc000f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:54.563 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.557+0000 7f2db27fc700 1 -- 172.21.15.99:0/2717228288 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc012590 con 0x7f2dc4067660 2023-10-08T00:10:54.563 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.557+0000 7f2db27fc700 1 -- 172.21.15.99:0/2717228288 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dbc012710 con 0x7f2dc4067660 2023-10-08T00:10:54.563 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.557+0000 7f2dc8fa3700 1 -- 172.21.15.99:0/2717228288 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dc40c3dd0 con 0x7f2dc4067660 2023-10-08T00:10:54.563 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.557+0000 7f2db27fc700 1 -- 172.21.15.99:0/2717228288 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc012ad0 con 0x7f2dc4067660 2023-10-08T00:10:54.563 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.557+0000 7f2dc8fa3700 1 -- 172.21.15.99:0/2717228288 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dc4067ab0 con 0x7f2dc4067660 2023-10-08T00:10:54.563 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.557+0000 7f2dc8fa3700 1 -- 172.21.15.99:0/2717228288 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dc404c3c0 con 0x7f2dc4067660 2023-10-08T00:10:54.563 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.558+0000 7f2db27fc700 1 -- 172.21.15.99:0/2717228288 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2dbc01d460 con 0x7f2dc4067660 2023-10-08T00:10:54.563 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.558+0000 7f2db27fc700 1 --2- 172.21.15.99:0/2717228288 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2da4066f60 0x7f2da4069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:54.563 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.558+0000 7f2db27fc700 1 -- 172.21.15.99:0/2717228288 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2dbc0841a0 con 0x7f2dc4067660 2023-10-08T00:10:54.563 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.558+0000 7f2dc259c700 1 --2- 172.21.15.99:0/2717228288 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2da4066f60 0x7f2da4069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:54.564 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.559+0000 7f2dc259c700 1 --2- 172.21.15.99:0/2717228288 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2da4066f60 0x7f2da4069410 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2db4001460 tx=0x7f2db4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:54.564 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.562+0000 7f2db27fc700 1 -- 172.21.15.99:0/2717228288 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2dbc0514d0 con 0x7f2dc4067660 2023-10-08T00:10:54.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.747+0000 7f2dc8fa3700 1 -- 172.21.15.99:0/2717228288 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2dc406d1b0 con 0x7f2dc4067660 2023-10-08T00:10:54.748 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.748+0000 7f2db27fc700 1 -- 172.21.15.99:0/2717228288 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+15363 (secure 0 0 0) 0x7f2dbc054af0 con 0x7f2dc4067660 2023-10-08T00:10:54.749 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:10:54.749 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":19,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:10:53.454011+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:10:51.822157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T00:10:52.415130+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:10:54.751 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.750+0000 7f2dc8fa3700 1 -- 172.21.15.99:0/2717228288 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2da4066f60 msgr2=0x7f2da4069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:54.751 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.750+0000 7f2dc8fa3700 1 --2- 172.21.15.99:0/2717228288 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2da4066f60 0x7f2da4069410 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2db4001460 tx=0x7f2db4009040 comp rx=0 tx=0).stop 2023-10-08T00:10:54.751 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.750+0000 7f2dc8fa3700 1 -- 172.21.15.99:0/2717228288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2dc4067660 msgr2=0x7f2dc4159440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:54.751 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.750+0000 7f2dc8fa3700 1 --2- 172.21.15.99:0/2717228288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2dc4067660 0x7f2dc4159440 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc000c00 tx=0x7f2dbc000f80 comp rx=0 tx=0).stop 2023-10-08T00:10:54.751 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.750+0000 7f2dc8fa3700 1 -- 172.21.15.99:0/2717228288 shutdown_connections 2023-10-08T00:10:54.751 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.750+0000 7f2dc8fa3700 1 --2- 172.21.15.99:0/2717228288 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2da4066f60 0x7f2da4069410 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:54.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.750+0000 7f2dc8fa3700 1 --2- 172.21.15.99:0/2717228288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2dc4067660 0x7f2dc4159440 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:54.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.750+0000 7f2dc8fa3700 1 --2- 172.21.15.99:0/2717228288 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2dac003fd0 0x7f2dc414db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:54.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.750+0000 7f2dc8fa3700 1 --2- 172.21.15.99:0/2717228288 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2dc414e130 0x7f2dc40c7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:54.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.750+0000 7f2dc8fa3700 1 -- 172.21.15.99:0/2717228288 >> 172.21.15.99:0/2717228288 conn(0x7f2dc4069870 msgr2=0x7f2dc40cb3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:54.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.750+0000 7f2dc8fa3700 1 -- 172.21.15.99:0/2717228288 shutdown_connections 2023-10-08T00:10:54.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.750+0000 7f2dc8fa3700 1 -- 172.21.15.99:0/2717228288 wait complete. 2023-10-08T00:10:54.763 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T00:10:54.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.878+0000 7f83b0679700 1 Processor -- start 2023-10-08T00:10:54.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.878+0000 7f83b0679700 1 -- start start 2023-10-08T00:10:54.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.879+0000 7f83b0679700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f83a8159ec0 0x7f83a80af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:54.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.879+0000 7f83b0679700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83a806acb0 0x7f83a80afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:54.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.879+0000 7f83b0679700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83a80b0790 con 0x7f83a815a290 2023-10-08T00:10:54.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.879+0000 7f83b0679700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83a80ce1f0 con 0x7f83a8159ec0 2023-10-08T00:10:54.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.879+0000 7f83b0679700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83a80b0b20 con 0x7f83a806acb0 2023-10-08T00:10:54.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.879+0000 7f83ae415700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f83a8159ec0 0x7f83a80af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:54.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.879+0000 7f83ae415700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f83a8159ec0 0x7f83a80af940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49416/0 (socket says 172.21.15.99:49416) 2023-10-08T00:10:54.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.879+0000 7f83adc14700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f83a815a290 0x7f83a806a5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50710/0 (socket says 172.21.15.99:50710) 2023-10-08T00:10:54.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.879+0000 7f83ae415700 1 -- 172.21.15.99:0/36804654 learned_addr learned my addr 172.21.15.99:0/36804654 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:10:54.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.879+0000 7f83aec16700 1 --2- 172.21.15.99:0/36804654 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83a806acb0 0x7f83a80afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:54.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.879+0000 7f83ae415700 1 -- 172.21.15.99:0/36804654 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83a806acb0 msgr2=0x7f83a80afe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:54.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.879+0000 7f83ae415700 1 --2- 172.21.15.99:0/36804654 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83a806acb0 0x7f83a80afe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:54.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.879+0000 7f83ae415700 1 -- 172.21.15.99:0/36804654 >> v1:172.21.15.99:6789/0 conn(0x7f83a815a290 legacy=0x7f83a806a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:54.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.880+0000 7f83ae415700 1 -- 172.21.15.99:0/36804654 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a80b04b0 con 0x7f83a8159ec0 2023-10-08T00:10:54.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.879+0000 7f83ad413700 1 -- 172.21.15.99:0/36804654 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3207167218 0 0) 0x7f83a80b0790 con 0x7f83a815a290 2023-10-08T00:10:54.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.880+0000 7f83ae415700 1 --2- 172.21.15.99:0/36804654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83a8159ec0 0x7f83a80af940 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f83a40099b0 tx=0x7f83a4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e80250e0c0120283 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:54.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.880+0000 7f83ad413700 1 -- 172.21.15.99:0/36804654 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a400b860 con 0x7f83a8159ec0 2023-10-08T00:10:54.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.880+0000 7f83ad413700 1 -- 172.21.15.99:0/36804654 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83a400b9c0 con 0x7f83a8159ec0 2023-10-08T00:10:54.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.880+0000 7f83ad413700 1 -- 172.21.15.99:0/36804654 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a400bd60 con 0x7f83a8159ec0 2023-10-08T00:10:54.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.880+0000 7f83b0679700 1 -- 172.21.15.99:0/36804654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83a8159ec0 msgr2=0x7f83a80af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:54.884 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.880+0000 7f83b0679700 1 --2- 172.21.15.99:0/36804654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83a8159ec0 0x7f83a80af940 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f83a40099b0 tx=0x7f83a4009250 comp rx=0 tx=0).stop 2023-10-08T00:10:54.884 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.880+0000 7f83b0679700 1 -- 172.21.15.99:0/36804654 shutdown_connections 2023-10-08T00:10:54.884 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.880+0000 7f83b0679700 1 --2- 172.21.15.99:0/36804654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83a8159ec0 0x7f83a80af940 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:54.884 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.880+0000 7f83b0679700 1 --2- 172.21.15.99:0/36804654 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83a806acb0 0x7f83a80afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:54.884 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.880+0000 7f83b0679700 1 -- 172.21.15.99:0/36804654 >> 172.21.15.99:0/36804654 conn(0x7f83a80d1880 msgr2=0x7f83a80cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:54.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.880+0000 7f83b0679700 1 -- 172.21.15.99:0/36804654 shutdown_connections 2023-10-08T00:10:54.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.880+0000 7f83b0679700 1 -- 172.21.15.99:0/36804654 wait complete. 2023-10-08T00:10:54.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.881+0000 7f83b0679700 1 Processor -- start 2023-10-08T00:10:54.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.882+0000 7f83b0679700 1 -- start start 2023-10-08T00:10:54.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.882+0000 7f83b0679700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83a806acb0 0x7f83a815a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:54.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.882+0000 7f83b0679700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83a8159ec0 0x7f83a8155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:54.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.882+0000 7f83b0679700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83a815a290 0x7f83a815ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:54.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.882+0000 7f83b0679700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a815d8c0 con 0x7f83a8159ec0 2023-10-08T00:10:54.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.882+0000 7f83b0679700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a815d340 con 0x7f83a815a290 2023-10-08T00:10:54.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.882+0000 7f83b0679700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83a815d5e0 con 0x7f83a806acb0 2023-10-08T00:10:54.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.882+0000 7f83adc14700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83a8159ec0 0x7f83a8155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:54.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.882+0000 7f83adc14700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83a8159ec0 0x7f83a8155aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49428/0 (socket says 172.21.15.99:49428) 2023-10-08T00:10:54.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.882+0000 7f83adc14700 1 -- 172.21.15.99:0/4150519117 learned_addr learned my addr 172.21.15.99:0/4150519117 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:10:54.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.882+0000 7f83ae415700 1 --2- 172.21.15.99:0/4150519117 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83a806acb0 0x7f83a815a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:54.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.882+0000 7f83aec16700 1 --2- 172.21.15.99:0/4150519117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83a815a290 0x7f83a815ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:54.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.883+0000 7f83adc14700 1 -- 172.21.15.99:0/4150519117 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83a806acb0 msgr2=0x7f83a815a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:54.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.883+0000 7f83adc14700 1 --2- 172.21.15.99:0/4150519117 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83a806acb0 0x7f83a815a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:54.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.883+0000 7f83adc14700 1 -- 172.21.15.99:0/4150519117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83a815a290 msgr2=0x7f83a815ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:54.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.883+0000 7f83adc14700 1 --2- 172.21.15.99:0/4150519117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83a815a290 0x7f83a815ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:54.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.883+0000 7f83adc14700 1 -- 172.21.15.99:0/4150519117 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a40094f0 con 0x7f83a8159ec0 2023-10-08T00:10:54.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.883+0000 7f83adc14700 1 --2- 172.21.15.99:0/4150519117 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83a8159ec0 0x7f83a8155aa0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f839800c5b0 tx=0x7f8398001040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:54.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.883+0000 7f839e7fc700 1 -- 172.21.15.99:0/4150519117 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8398005a70 con 0x7f83a8159ec0 2023-10-08T00:10:54.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.883+0000 7f839e7fc700 1 -- 172.21.15.99:0/4150519117 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8398005bd0 con 0x7f83a8159ec0 2023-10-08T00:10:54.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.883+0000 7f83b0679700 1 -- 172.21.15.99:0/4150519117 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83a8169fc0 con 0x7f83a8159ec0 2023-10-08T00:10:54.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.883+0000 7f839e7fc700 1 -- 172.21.15.99:0/4150519117 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83980070d0 con 0x7f83a8159ec0 2023-10-08T00:10:54.893 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.883+0000 7f83b0679700 1 -- 172.21.15.99:0/4150519117 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83a816a580 con 0x7f83a8159ec0 2023-10-08T00:10:54.893 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.884+0000 7f83b0679700 1 -- 172.21.15.99:0/4150519117 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83a816a7a0 con 0x7f83a8159ec0 2023-10-08T00:10:54.893 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.884+0000 7f839e7fc700 1 -- 172.21.15.99:0/4150519117 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8398007480 con 0x7f83a8159ec0 2023-10-08T00:10:54.893 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.884+0000 7f839e7fc700 1 --2- 172.21.15.99:0/4150519117 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f838c066f30 0x7f838c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:54.893 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.885+0000 7f839e7fc700 1 -- 172.21.15.99:0/4150519117 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8398083a50 con 0x7f83a8159ec0 2023-10-08T00:10:54.893 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.885+0000 7f83ae415700 1 --2- 172.21.15.99:0/4150519117 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f838c066f30 0x7f838c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:54.893 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.885+0000 7f83ae415700 1 --2- 172.21.15.99:0/4150519117 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f838c066f30 0x7f838c0693e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f83a40099b0 tx=0x7f83a4017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:54.893 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:54.888+0000 7f839e7fc700 1 -- 172.21.15.99:0/4150519117 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8398050cc0 con 0x7f83a8159ec0 2023-10-08T00:10:55.116 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:55.115+0000 7f83b0679700 1 -- 172.21.15.99:0/4150519117 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f83a804c3c0 con 0x7f83a8159ec0 2023-10-08T00:10:56.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.098+0000 7f839e7fc700 1 -- 172.21.15.99:0/4150519117 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8398017020 con 0x7f83a8159ec0 2023-10-08T00:10:56.102 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.101+0000 7f83b0679700 1 -- 172.21.15.99:0/4150519117 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f838c066f30 msgr2=0x7f838c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:56.102 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.101+0000 7f83b0679700 1 --2- 172.21.15.99:0/4150519117 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f838c066f30 0x7f838c0693e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f83a40099b0 tx=0x7f83a4017000 comp rx=0 tx=0).stop 2023-10-08T00:10:56.102 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.101+0000 7f83b0679700 1 -- 172.21.15.99:0/4150519117 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83a8159ec0 msgr2=0x7f83a8155aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:56.102 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.101+0000 7f83b0679700 1 --2- 172.21.15.99:0/4150519117 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83a8159ec0 0x7f83a8155aa0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f839800c5b0 tx=0x7f8398001040 comp rx=0 tx=0).stop 2023-10-08T00:10:56.102 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.101+0000 7f83b0679700 1 -- 172.21.15.99:0/4150519117 shutdown_connections 2023-10-08T00:10:56.102 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.101+0000 7f83b0679700 1 --2- 172.21.15.99:0/4150519117 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f838c066f30 0x7f838c0693e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:56.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.101+0000 7f83b0679700 1 --2- 172.21.15.99:0/4150519117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83a815a290 0x7f83a815ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:56.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.101+0000 7f83b0679700 1 --2- 172.21.15.99:0/4150519117 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83a806acb0 0x7f83a815a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:56.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.101+0000 7f83b0679700 1 --2- 172.21.15.99:0/4150519117 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83a8159ec0 0x7f83a8155aa0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:56.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.101+0000 7f83b0679700 1 -- 172.21.15.99:0/4150519117 >> 172.21.15.99:0/4150519117 conn(0x7f83a80d1880 msgr2=0x7f83a80c7290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:56.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.101+0000 7f83b0679700 1 -- 172.21.15.99:0/4150519117 shutdown_connections 2023-10-08T00:10:56.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.101+0000 7f83b0679700 1 -- 172.21.15.99:0/4150519117 wait complete. 2023-10-08T00:10:56.114 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T00:10:56.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.232+0000 7fac6df6e700 1 Processor -- start 2023-10-08T00:10:56.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.232+0000 7fac6df6e700 1 -- start start 2023-10-08T00:10:56.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.233+0000 7fac6df6e700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fac6815a290 0x7fac6806a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:56.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.233+0000 7fac6df6e700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac680b0790 con 0x7fac6806aae0 2023-10-08T00:10:56.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.233+0000 7fac6df6e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac6806af20 con 0x7fac68159ec0 2023-10-08T00:10:56.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.233+0000 7fac6df6e700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac680ce1f0 con 0x7fac6815a290 2023-10-08T00:10:56.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.233+0000 7fac66ffd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fac6815a290 0x7fac6806a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:56.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.233+0000 7fac66ffd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fac6815a290 0x7fac6806a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49430/0 (socket says 172.21.15.99:49430) 2023-10-08T00:10:56.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.233+0000 7fac67fff700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fac6806aae0 0x7fac680cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50722/0 (socket says 172.21.15.99:50722) 2023-10-08T00:10:56.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.233+0000 7fac66ffd700 1 -- 172.21.15.99:0/2473378888 learned_addr learned my addr 172.21.15.99:0/2473378888 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:10:56.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.233+0000 7fac66ffd700 1 -- 172.21.15.99:0/2473378888 >> v1:172.21.15.191:6789/0 conn(0x7fac68159ec0 legacy=0x7fac680af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:56.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.233+0000 7fac66ffd700 1 -- 172.21.15.99:0/2473378888 >> v1:172.21.15.99:6789/0 conn(0x7fac6806aae0 legacy=0x7fac680cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:56.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.234+0000 7fac66ffd700 1 -- 172.21.15.99:0/2473378888 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac680b0b20 con 0x7fac6815a290 2023-10-08T00:10:56.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.234+0000 7fac66ffd700 1 --2- 172.21.15.99:0/2473378888 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac6815a290 0x7fac6806a5a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fac5c0099b0 tx=0x7fac5c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edf36c7cd8e51959 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:56.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.234+0000 7fac667fc700 1 -- 172.21.15.99:0/2473378888 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5c00b760 con 0x7fac6815a290 2023-10-08T00:10:56.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.234+0000 7fac667fc700 1 -- 172.21.15.99:0/2473378888 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac5c00b8c0 con 0x7fac6815a290 2023-10-08T00:10:56.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.234+0000 7fac667fc700 1 -- 172.21.15.99:0/2473378888 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5c00bc60 con 0x7fac6815a290 2023-10-08T00:10:56.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.234+0000 7fac6df6e700 1 -- 172.21.15.99:0/2473378888 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac6815a290 msgr2=0x7fac6806a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:56.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.234+0000 7fac6df6e700 1 --2- 172.21.15.99:0/2473378888 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac6815a290 0x7fac6806a5a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fac5c0099b0 tx=0x7fac5c009250 comp rx=0 tx=0).stop 2023-10-08T00:10:56.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.234+0000 7fac6df6e700 1 -- 172.21.15.99:0/2473378888 shutdown_connections 2023-10-08T00:10:56.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.234+0000 7fac6df6e700 1 --2- 172.21.15.99:0/2473378888 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac6815a290 0x7fac6806a5a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:56.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.234+0000 7fac6df6e700 1 -- 172.21.15.99:0/2473378888 >> 172.21.15.99:0/2473378888 conn(0x7fac680d1880 msgr2=0x7fac680cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:56.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.234+0000 7fac6df6e700 1 -- 172.21.15.99:0/2473378888 shutdown_connections 2023-10-08T00:10:56.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.234+0000 7fac6df6e700 1 -- 172.21.15.99:0/2473378888 wait complete. 2023-10-08T00:10:56.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.236+0000 7fac6df6e700 1 Processor -- start 2023-10-08T00:10:56.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.236+0000 7fac6df6e700 1 -- start start 2023-10-08T00:10:56.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.236+0000 7fac6df6e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac6806aae0 0x7fac6815f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:56.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.236+0000 7fac6df6e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fac68159ec0 0x7fac6815f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:56.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.236+0000 7fac6df6e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac6815a290 0x7fac680cb170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:56.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.236+0000 7fac6df6e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac68155ec0 con 0x7fac6815a290 2023-10-08T00:10:56.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.236+0000 7fac6df6e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac6815cb60 con 0x7fac6806aae0 2023-10-08T00:10:56.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.236+0000 7fac6df6e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac680b4b00 con 0x7fac68159ec0 2023-10-08T00:10:56.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.236+0000 7fac67fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac6815a290 0x7fac680cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:56.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.237+0000 7fac67fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac6815a290 0x7fac680cb170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49440/0 (socket says 172.21.15.99:49440) 2023-10-08T00:10:56.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.237+0000 7fac67fff700 1 -- 172.21.15.99:0/619325160 learned_addr learned my addr 172.21.15.99:0/619325160 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:10:56.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.237+0000 7fac677fe700 1 --2- 172.21.15.99:0/619325160 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac6806aae0 0x7fac6815f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:56.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.237+0000 7fac66ffd700 1 --2- 172.21.15.99:0/619325160 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fac68159ec0 0x7fac6815f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:56.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.237+0000 7fac67fff700 1 -- 172.21.15.99:0/619325160 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fac68159ec0 msgr2=0x7fac6815f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:56.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.237+0000 7fac67fff700 1 --2- 172.21.15.99:0/619325160 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fac68159ec0 0x7fac6815f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:56.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.237+0000 7fac67fff700 1 -- 172.21.15.99:0/619325160 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac6806aae0 msgr2=0x7fac6815f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:56.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.237+0000 7fac67fff700 1 --2- 172.21.15.99:0/619325160 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac6806aae0 0x7fac6815f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:56.242 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.237+0000 7fac67fff700 1 -- 172.21.15.99:0/619325160 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac5c0094f0 con 0x7fac6815a290 2023-10-08T00:10:56.242 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.237+0000 7fac677fe700 1 --2- 172.21.15.99:0/619325160 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac6806aae0 0x7fac6815f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:10:56.242 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.237+0000 7fac67fff700 1 --2- 172.21.15.99:0/619325160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac6815a290 0x7fac680cb170 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fac6000c810 tx=0x7fac60007e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:56.242 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.237+0000 7fac4f7fe700 1 -- 172.21.15.99:0/619325160 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac60007300 con 0x7fac6815a290 2023-10-08T00:10:56.242 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.237+0000 7fac4f7fe700 1 -- 172.21.15.99:0/619325160 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac600074a0 con 0x7fac6815a290 2023-10-08T00:10:56.242 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.237+0000 7fac4f7fe700 1 -- 172.21.15.99:0/619325160 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac60004830 con 0x7fac6815a290 2023-10-08T00:10:56.242 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.237+0000 7fac6df6e700 1 -- 172.21.15.99:0/619325160 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac680b4de0 con 0x7fac6815a290 2023-10-08T00:10:56.242 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.237+0000 7fac6df6e700 1 -- 172.21.15.99:0/619325160 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac680b53a0 con 0x7fac6815a290 2023-10-08T00:10:56.242 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.238+0000 7fac6df6e700 1 -- 172.21.15.99:0/619325160 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac6804c3c0 con 0x7fac6815a290 2023-10-08T00:10:56.242 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.238+0000 7fac4f7fe700 1 -- 172.21.15.99:0/619325160 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fac60004ba0 con 0x7fac6815a290 2023-10-08T00:10:56.242 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.239+0000 7fac4f7fe700 1 --2- 172.21.15.99:0/619325160 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fac54066f30 0x7fac540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:56.243 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.239+0000 7fac4f7fe700 1 -- 172.21.15.99:0/619325160 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fac60083700 con 0x7fac6815a290 2023-10-08T00:10:56.243 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.239+0000 7fac677fe700 1 --2- 172.21.15.99:0/619325160 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fac54066f30 0x7fac540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:56.243 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.239+0000 7fac677fe700 1 --2- 172.21.15.99:0/619325160 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fac54066f30 0x7fac540693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fac58007fd0 tx=0x7fac58009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:56.243 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.243+0000 7fac4f7fe700 1 -- 172.21.15.99:0/619325160 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fac600509f0 con 0x7fac6815a290 2023-10-08T00:10:56.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:56.468+0000 7fac6df6e700 1 -- 172.21.15.99:0/619325160 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fac680b04b0 con 0x7fac6815a290 2023-10-08T00:10:57.100 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.099+0000 7fac4f7fe700 1 -- 172.21.15.99:0/619325160 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7fac600180c0 con 0x7fac6815a290 2023-10-08T00:10:57.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.102+0000 7fac6df6e700 1 -- 172.21.15.99:0/619325160 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fac54066f30 msgr2=0x7fac540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:57.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.102+0000 7fac6df6e700 1 --2- 172.21.15.99:0/619325160 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fac54066f30 0x7fac540693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fac58007fd0 tx=0x7fac58009040 comp rx=0 tx=0).stop 2023-10-08T00:10:57.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.102+0000 7fac6df6e700 1 -- 172.21.15.99:0/619325160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac6815a290 msgr2=0x7fac680cb170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:57.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.102+0000 7fac6df6e700 1 --2- 172.21.15.99:0/619325160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac6815a290 0x7fac680cb170 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fac6000c810 tx=0x7fac60007e70 comp rx=0 tx=0).stop 2023-10-08T00:10:57.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.102+0000 7fac6df6e700 1 -- 172.21.15.99:0/619325160 shutdown_connections 2023-10-08T00:10:57.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.102+0000 7fac6df6e700 1 --2- 172.21.15.99:0/619325160 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fac54066f30 0x7fac540693e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:57.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.102+0000 7fac6df6e700 1 --2- 172.21.15.99:0/619325160 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac6806aae0 0x7fac6815f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:57.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.102+0000 7fac6df6e700 1 --2- 172.21.15.99:0/619325160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac6815a290 0x7fac680cb170 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:57.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.102+0000 7fac6df6e700 1 --2- 172.21.15.99:0/619325160 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fac68159ec0 0x7fac6815f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:57.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.102+0000 7fac6df6e700 1 -- 172.21.15.99:0/619325160 >> 172.21.15.99:0/619325160 conn(0x7fac680d1880 msgr2=0x7fac68158b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:57.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.102+0000 7fac6df6e700 1 -- 172.21.15.99:0/619325160 shutdown_connections 2023-10-08T00:10:57.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.102+0000 7fac6df6e700 1 -- 172.21.15.99:0/619325160 wait complete. 2023-10-08T00:10:57.114 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:10:57.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.227+0000 7fe74fd4f700 1 Processor -- start 2023-10-08T00:10:57.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.228+0000 7fe74fd4f700 1 -- start start 2023-10-08T00:10:57.231 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.228+0000 7fe74fd4f700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe748157870 0x7fe74815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:57.231 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.228+0000 7fe74fd4f700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fe74815a620 0x7fe74815d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:57.231 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.228+0000 7fe74fd4f700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe74815cbb0 con 0x7fe74815d580 2023-10-08T00:10:57.231 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.228+0000 7fe74fd4f700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe74815bfa0 con 0x7fe748157870 2023-10-08T00:10:57.231 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.228+0000 7fe74fd4f700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe74815c280 con 0x7fe74815a620 2023-10-08T00:10:57.231 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.229+0000 7fe74daeb700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe748157870 0x7fe74815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:57.231 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.229+0000 7fe74daeb700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe748157870 0x7fe74815b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49456/0 (socket says 172.21.15.99:49456) 2023-10-08T00:10:57.231 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.229+0000 7fe74daeb700 1 -- 172.21.15.99:0/4162866705 learned_addr learned my addr 172.21.15.99:0/4162866705 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:10:57.231 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.229+0000 7fe74d2ea700 1 --2- 172.21.15.99:0/4162866705 >> v2:172.21.15.191:3300/0 conn(0x7fe74815a620 0x7fe74815d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:57.231 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.229+0000 7fe74daeb700 1 -- 172.21.15.99:0/4162866705 >> v2:172.21.15.191:3300/0 conn(0x7fe74815a620 msgr2=0x7fe74815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:57.232 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.229+0000 7fe74daeb700 1 --2- 172.21.15.99:0/4162866705 >> v2:172.21.15.191:3300/0 conn(0x7fe74815a620 0x7fe74815d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:57.232 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.229+0000 7fe74daeb700 1 -- 172.21.15.99:0/4162866705 >> v1:172.21.15.191:6789/0 conn(0x7fe74815d580 legacy=0x7fe748158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:57.232 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.229+0000 7fe74daeb700 1 -- 172.21.15.99:0/4162866705 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe74815c560 con 0x7fe748157870 2023-10-08T00:10:57.232 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.229+0000 7fe74daeb700 1 --2- 172.21.15.99:0/4162866705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe748157870 0x7fe74815b6a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe74815cb20 tx=0x7fe744009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6aab4f287f42457 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:57.232 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.229+0000 7fe74cae9700 1 -- 172.21.15.99:0/4162866705 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74400e070 con 0x7fe748157870 2023-10-08T00:10:57.232 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.229+0000 7fe74cae9700 1 -- 172.21.15.99:0/4162866705 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe74400ba10 con 0x7fe748157870 2023-10-08T00:10:57.232 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.229+0000 7fe74cae9700 1 -- 172.21.15.99:0/4162866705 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7440092e0 con 0x7fe748157870 2023-10-08T00:10:57.232 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.230+0000 7fe74fd4f700 1 -- 172.21.15.99:0/4162866705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe748157870 msgr2=0x7fe74815b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:57.232 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.230+0000 7fe74fd4f700 1 --2- 172.21.15.99:0/4162866705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe748157870 0x7fe74815b6a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe74815cb20 tx=0x7fe744009840 comp rx=0 tx=0).stop 2023-10-08T00:10:57.232 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.230+0000 7fe74fd4f700 1 -- 172.21.15.99:0/4162866705 shutdown_connections 2023-10-08T00:10:57.232 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.230+0000 7fe74fd4f700 1 --2- 172.21.15.99:0/4162866705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe748157870 0x7fe74815b6a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:57.232 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.230+0000 7fe74fd4f700 1 --2- 172.21.15.99:0/4162866705 >> v2:172.21.15.191:3300/0 conn(0x7fe74815a620 0x7fe74815d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:57.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.230+0000 7fe74fd4f700 1 -- 172.21.15.99:0/4162866705 >> 172.21.15.99:0/4162866705 conn(0x7fe748068ce0 msgr2=0x7fe7480690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:57.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.230+0000 7fe74fd4f700 1 -- 172.21.15.99:0/4162866705 shutdown_connections 2023-10-08T00:10:57.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.230+0000 7fe74fd4f700 1 -- 172.21.15.99:0/4162866705 wait complete. 2023-10-08T00:10:57.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.231+0000 7fe74fd4f700 1 Processor -- start 2023-10-08T00:10:57.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.231+0000 7fe74fd4f700 1 -- start start 2023-10-08T00:10:57.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.232+0000 7fe74fd4f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe748157870 0x7fe748145770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:57.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.232+0000 7fe74fd4f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe74815a620 0x7fe748154280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:57.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.232+0000 7fe74fd4f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe74815d580 0x7fe7480b1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:57.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.232+0000 7fe74fd4f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7480b1b10 con 0x7fe748157870 2023-10-08T00:10:57.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.232+0000 7fe74fd4f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe748152b10 con 0x7fe74815d580 2023-10-08T00:10:57.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.232+0000 7fe74fd4f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7480b1020 con 0x7fe74815a620 2023-10-08T00:10:57.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.232+0000 7fe74daeb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe748157870 0x7fe748145770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:57.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.232+0000 7fe74daeb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe748157870 0x7fe748145770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49458/0 (socket says 172.21.15.99:49458) 2023-10-08T00:10:57.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.232+0000 7fe74daeb700 1 -- 172.21.15.99:0/1234465513 learned_addr learned my addr 172.21.15.99:0/1234465513 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:10:57.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.232+0000 7fe74d2ea700 1 --2- 172.21.15.99:0/1234465513 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe74815a620 0x7fe748154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:57.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.232+0000 7fe74e2ec700 1 --2- 172.21.15.99:0/1234465513 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe74815d580 0x7fe7480b1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:57.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.232+0000 7fe74daeb700 1 -- 172.21.15.99:0/1234465513 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe74815a620 msgr2=0x7fe748154280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:57.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.232+0000 7fe74daeb700 1 --2- 172.21.15.99:0/1234465513 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe74815a620 0x7fe748154280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:57.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.232+0000 7fe74daeb700 1 -- 172.21.15.99:0/1234465513 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe74815d580 msgr2=0x7fe7480b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:57.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.232+0000 7fe74daeb700 1 --2- 172.21.15.99:0/1234465513 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe74815d580 0x7fe7480b1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:57.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.232+0000 7fe74daeb700 1 -- 172.21.15.99:0/1234465513 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe744009ae0 con 0x7fe748157870 2023-10-08T00:10:57.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.232+0000 7fe74d2ea700 1 --2- 172.21.15.99:0/1234465513 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe74815a620 0x7fe748154280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:10:57.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.232+0000 7fe74daeb700 1 --2- 172.21.15.99:0/1234465513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe748157870 0x7fe748145770 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe74806b530 tx=0x7fe744017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:57.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.232+0000 7fe73dffb700 1 -- 172.21.15.99:0/1234465513 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7440094e0 con 0x7fe748157870 2023-10-08T00:10:57.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.232+0000 7fe73dffb700 1 -- 172.21.15.99:0/1234465513 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7440128d0 con 0x7fe748157870 2023-10-08T00:10:57.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.232+0000 7fe73dffb700 1 -- 172.21.15.99:0/1234465513 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe744012c90 con 0x7fe748157870 2023-10-08T00:10:57.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.232+0000 7fe74fd4f700 1 -- 172.21.15.99:0/1234465513 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7480cfc40 con 0x7fe748157870 2023-10-08T00:10:57.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.233+0000 7fe74fd4f700 1 -- 172.21.15.99:0/1234465513 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7480b1550 con 0x7fe748157870 2023-10-08T00:10:57.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.233+0000 7fe74fd4f700 1 -- 172.21.15.99:0/1234465513 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7480cfa10 con 0x7fe748157870 2023-10-08T00:10:57.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.234+0000 7fe73dffb700 1 -- 172.21.15.99:0/1234465513 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe74401d630 con 0x7fe748157870 2023-10-08T00:10:57.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.234+0000 7fe73dffb700 1 --2- 172.21.15.99:0/1234465513 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe72c066f30 0x7fe72c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:57.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.234+0000 7fe73dffb700 1 -- 172.21.15.99:0/1234465513 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe744085aa0 con 0x7fe748157870 2023-10-08T00:10:57.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.234+0000 7fe74d2ea700 1 --2- 172.21.15.99:0/1234465513 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe72c066f30 0x7fe72c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:57.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.235+0000 7fe74d2ea700 1 --2- 172.21.15.99:0/1234465513 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe72c066f30 0x7fe72c0693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe7481531a0 tx=0x7fe738008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:57.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.238+0000 7fe73dffb700 1 -- 172.21.15.99:0/1234465513 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe7480cfa10 con 0x7fe748157870 2023-10-08T00:10:57.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.462+0000 7fe74fd4f700 1 -- 172.21.15.99:0/1234465513 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe7480cfa10 con 0x7fe748157870 2023-10-08T00:10:57.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.462+0000 7fe73dffb700 1 -- 172.21.15.99:0/1234465513 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4594 (secure 0 0 0) 0x7fe7480cfa10 con 0x7fe748157870 2023-10-08T00:10:57.463 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:10:57.463 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/649537180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":649537180},{"type":"v1","addr":"172.21.15.99:6833","nonce":649537180}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4239,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2194110899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2194110899},{"type":"v1","addr":"172.21.15.99:6835","nonce":2194110899}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4251,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/1732443753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":1732443753},{"type":"v1","addr":"172.21.15.191:6837","nonce":1732443753}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:10:53.454321+0000","modified":"2023-10-08T00:10:57.098731+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6836/1988845948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1988845948},{"type":"v1","addr":"172.21.15.191:6836","nonce":1988845948}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-08T00:10:57.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.464+0000 7fe74fd4f700 1 -- 172.21.15.99:0/1234465513 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe72c066f30 msgr2=0x7fe72c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:57.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.464+0000 7fe74fd4f700 1 --2- 172.21.15.99:0/1234465513 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe72c066f30 0x7fe72c0693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe7481531a0 tx=0x7fe738008040 comp rx=0 tx=0).stop 2023-10-08T00:10:57.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.464+0000 7fe74fd4f700 1 -- 172.21.15.99:0/1234465513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe748157870 msgr2=0x7fe748145770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:57.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.464+0000 7fe74fd4f700 1 --2- 172.21.15.99:0/1234465513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe748157870 0x7fe748145770 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe74806b530 tx=0x7fe744017000 comp rx=0 tx=0).stop 2023-10-08T00:10:57.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.464+0000 7fe74fd4f700 1 -- 172.21.15.99:0/1234465513 shutdown_connections 2023-10-08T00:10:57.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.464+0000 7fe74fd4f700 1 --2- 172.21.15.99:0/1234465513 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe72c066f30 0x7fe72c0693e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:57.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.464+0000 7fe74fd4f700 1 --2- 172.21.15.99:0/1234465513 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe74815d580 0x7fe7480b1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:57.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.464+0000 7fe74fd4f700 1 --2- 172.21.15.99:0/1234465513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe748157870 0x7fe748145770 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:57.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.464+0000 7fe74fd4f700 1 --2- 172.21.15.99:0/1234465513 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe74815a620 0x7fe748154280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:57.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.464+0000 7fe74fd4f700 1 -- 172.21.15.99:0/1234465513 >> 172.21.15.99:0/1234465513 conn(0x7fe748068ce0 msgr2=0x7fe748155710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:57.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.464+0000 7fe74fd4f700 1 -- 172.21.15.99:0/1234465513 shutdown_connections 2023-10-08T00:10:57.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.464+0000 7fe74fd4f700 1 -- 172.21.15.99:0/1234465513 wait complete. 2023-10-08T00:10:57.466 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 8 2023-10-08T00:10:57.477 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:10:57.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.591+0000 7fe9650f7700 1 Processor -- start 2023-10-08T00:10:57.592 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.591+0000 7fe9650f7700 1 -- start start 2023-10-08T00:10:57.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.592+0000 7fe9650f7700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe96015a620 0x7fe96015d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:57.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.592+0000 7fe9650f7700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe96015cbb0 con 0x7fe96015d580 2023-10-08T00:10:57.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.592+0000 7fe9650f7700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe96015bfa0 con 0x7fe960157870 2023-10-08T00:10:57.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.592+0000 7fe9650f7700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe96015bdb0 con 0x7fe96015a620 2023-10-08T00:10:57.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.592+0000 7fe957fff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe96015a620 0x7fe96015d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:57.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.592+0000 7fe95f59e700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fe96015d580 0x7fe960158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50734/0 (socket says 172.21.15.99:50734) 2023-10-08T00:10:57.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.592+0000 7fe95f59e700 1 -- 172.21.15.99:0/1827980422 learned_addr learned my addr 172.21.15.99:0/1827980422 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:10:57.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.592+0000 7fe957fff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe96015a620 0x7fe96015d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49466/0 (socket says 172.21.15.99:49466) 2023-10-08T00:10:57.595 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.592+0000 7fe957fff700 1 -- 172.21.15.99:0/1827980422 >> v1:172.21.15.191:6789/0 conn(0x7fe960157870 legacy=0x7fe96015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:57.595 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.592+0000 7fe957fff700 1 -- 172.21.15.99:0/1827980422 >> v1:172.21.15.99:6789/0 conn(0x7fe96015d580 legacy=0x7fe960158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:57.595 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.592+0000 7fe957fff700 1 -- 172.21.15.99:0/1827980422 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe96015c280 con 0x7fe96015a620 2023-10-08T00:10:57.595 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.592+0000 7fe95e59c700 1 -- 172.21.15.99:0/1827980422 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1859471230 0 0) 0x7fe96015cbb0 con 0x7fe96015d580 2023-10-08T00:10:57.595 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.592+0000 7fe957fff700 1 --2- 172.21.15.99:0/1827980422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe96015a620 0x7fe96015d040 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe9480098e0 tx=0x7fe9480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db5f2c391d286ba server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:57.595 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.592+0000 7fe95e59c700 1 -- 172.21.15.99:0/1827980422 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe948009e80 con 0x7fe96015a620 2023-10-08T00:10:57.595 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.592+0000 7fe95e59c700 1 -- 172.21.15.99:0/1827980422 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe94800b860 con 0x7fe96015a620 2023-10-08T00:10:57.595 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.592+0000 7fe95e59c700 1 -- 172.21.15.99:0/1827980422 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94800bbe0 con 0x7fe96015a620 2023-10-08T00:10:57.595 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.593+0000 7fe9650f7700 1 -- 172.21.15.99:0/1827980422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe96015a620 msgr2=0x7fe96015d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:57.595 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.593+0000 7fe9650f7700 1 --2- 172.21.15.99:0/1827980422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe96015a620 0x7fe96015d040 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe9480098e0 tx=0x7fe9480091a0 comp rx=0 tx=0).stop 2023-10-08T00:10:57.595 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.593+0000 7fe9650f7700 1 -- 172.21.15.99:0/1827980422 shutdown_connections 2023-10-08T00:10:57.595 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.593+0000 7fe9650f7700 1 --2- 172.21.15.99:0/1827980422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe96015a620 0x7fe96015d040 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:57.596 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.593+0000 7fe9650f7700 1 -- 172.21.15.99:0/1827980422 >> 172.21.15.99:0/1827980422 conn(0x7fe960068ce0 msgr2=0x7fe9600690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:57.596 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.593+0000 7fe9650f7700 1 -- 172.21.15.99:0/1827980422 shutdown_connections 2023-10-08T00:10:57.596 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.593+0000 7fe9650f7700 1 -- 172.21.15.99:0/1827980422 wait complete. 2023-10-08T00:10:57.598 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.594+0000 7fe9650f7700 1 Processor -- start 2023-10-08T00:10:57.598 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.594+0000 7fe9650f7700 1 -- start start 2023-10-08T00:10:57.598 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.595+0000 7fe9650f7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe960157870 0x7fe96014ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:57.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.595+0000 7fe9650f7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe96015a620 0x7fe9600d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:57.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.595+0000 7fe9650f7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe96015d580 0x7fe96014b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:57.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.595+0000 7fe9650f7700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9600d09c0 con 0x7fe960157870 2023-10-08T00:10:57.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.595+0000 7fe9650f7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9600cfdf0 con 0x7fe96015d580 2023-10-08T00:10:57.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.595+0000 7fe9650f7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9600d0090 con 0x7fe96015a620 2023-10-08T00:10:57.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.595+0000 7fe95ed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe960157870 0x7fe96014ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:57.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.595+0000 7fe95ed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe960157870 0x7fe96014ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49476/0 (socket says 172.21.15.99:49476) 2023-10-08T00:10:57.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.595+0000 7fe95ed9d700 1 -- 172.21.15.99:0/473116776 learned_addr learned my addr 172.21.15.99:0/473116776 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:10:57.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.595+0000 7fe95f59e700 1 --2- 172.21.15.99:0/473116776 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe96015d580 0x7fe96014b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:57.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.595+0000 7fe957fff700 1 --2- 172.21.15.99:0/473116776 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe96015a620 0x7fe9600d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:57.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.595+0000 7fe95ed9d700 1 -- 172.21.15.99:0/473116776 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe96015a620 msgr2=0x7fe9600d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:57.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.595+0000 7fe95ed9d700 1 --2- 172.21.15.99:0/473116776 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe96015a620 0x7fe9600d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:57.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.595+0000 7fe95ed9d700 1 -- 172.21.15.99:0/473116776 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe96015d580 msgr2=0x7fe96014b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:57.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.595+0000 7fe95ed9d700 1 --2- 172.21.15.99:0/473116776 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe96015d580 0x7fe96014b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:57.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.595+0000 7fe95ed9d700 1 -- 172.21.15.99:0/473116776 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe948009420 con 0x7fe960157870 2023-10-08T00:10:57.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.595+0000 7fe95ed9d700 1 --2- 172.21.15.99:0/473116776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe960157870 0x7fe96014ac60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe95000c590 tx=0x7fe95000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:57.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.595+0000 7fe956ffd700 1 -- 172.21.15.99:0/473116776 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe950011590 con 0x7fe960157870 2023-10-08T00:10:57.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.595+0000 7fe956ffd700 1 -- 172.21.15.99:0/473116776 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe950011710 con 0x7fe960157870 2023-10-08T00:10:57.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.595+0000 7fe9650f7700 1 -- 172.21.15.99:0/473116776 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9600d0370 con 0x7fe960157870 2023-10-08T00:10:57.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.595+0000 7fe956ffd700 1 -- 172.21.15.99:0/473116776 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe950011ad0 con 0x7fe960157870 2023-10-08T00:10:57.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.595+0000 7fe9650f7700 1 -- 172.21.15.99:0/473116776 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9600b2f10 con 0x7fe960157870 2023-10-08T00:10:57.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.596+0000 7fe9650f7700 1 -- 172.21.15.99:0/473116776 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe96004c3c0 con 0x7fe960157870 2023-10-08T00:10:57.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.596+0000 7fe956ffd700 1 -- 172.21.15.99:0/473116776 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe95001b3b0 con 0x7fe960157870 2023-10-08T00:10:57.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.597+0000 7fe956ffd700 1 --2- 172.21.15.99:0/473116776 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe940066f30 0x7fe9400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:57.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.597+0000 7fe956ffd700 1 -- 172.21.15.99:0/473116776 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe950083a20 con 0x7fe960157870 2023-10-08T00:10:57.601 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.597+0000 7fe957fff700 1 --2- 172.21.15.99:0/473116776 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe940066f30 0x7fe9400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:57.601 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.597+0000 7fe957fff700 1 --2- 172.21.15.99:0/473116776 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe940066f30 0x7fe9400693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe96015aa20 tx=0x7fe948017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:57.601 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.601+0000 7fe956ffd700 1 -- 172.21.15.99:0/473116776 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe950050c90 con 0x7fe960157870 2023-10-08T00:10:57.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.782+0000 7fe9650f7700 1 -- 172.21.15.99:0/473116776 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe96015c560 con 0x7fe960157870 2023-10-08T00:10:57.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.783+0000 7fe956ffd700 1 -- 172.21.15.99:0/473116776 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15350 (secure 0 0 0) 0x7fe950017070 con 0x7fe960157870 2023-10-08T00:10:57.783 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:10:57.784 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":20,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:10:55.080854+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:10:51.822157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T00:10:52.415130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:10:57.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.785+0000 7fe9650f7700 1 -- 172.21.15.99:0/473116776 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe940066f30 msgr2=0x7fe9400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:57.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.785+0000 7fe9650f7700 1 --2- 172.21.15.99:0/473116776 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe940066f30 0x7fe9400693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe96015aa20 tx=0x7fe948017000 comp rx=0 tx=0).stop 2023-10-08T00:10:57.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.785+0000 7fe9650f7700 1 -- 172.21.15.99:0/473116776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe960157870 msgr2=0x7fe96014ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:57.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.785+0000 7fe9650f7700 1 --2- 172.21.15.99:0/473116776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe960157870 0x7fe96014ac60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe95000c590 tx=0x7fe95000c8a0 comp rx=0 tx=0).stop 2023-10-08T00:10:57.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.785+0000 7fe9650f7700 1 -- 172.21.15.99:0/473116776 shutdown_connections 2023-10-08T00:10:57.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.785+0000 7fe9650f7700 1 --2- 172.21.15.99:0/473116776 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe940066f30 0x7fe9400693e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:57.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.785+0000 7fe9650f7700 1 --2- 172.21.15.99:0/473116776 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe96015d580 0x7fe96014b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:57.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.785+0000 7fe9650f7700 1 --2- 172.21.15.99:0/473116776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe960157870 0x7fe96014ac60 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:57.788 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.785+0000 7fe9650f7700 1 --2- 172.21.15.99:0/473116776 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe96015a620 0x7fe9600d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:57.788 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.785+0000 7fe9650f7700 1 -- 172.21.15.99:0/473116776 >> 172.21.15.99:0/473116776 conn(0x7fe960068ce0 msgr2=0x7fe96014c800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:57.788 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.785+0000 7fe9650f7700 1 -- 172.21.15.99:0/473116776 shutdown_connections 2023-10-08T00:10:57.788 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.785+0000 7fe9650f7700 1 -- 172.21.15.99:0/473116776 wait complete. 2023-10-08T00:10:57.798 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T00:10:57.798 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T00:10:57.912 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.911+0000 7fda4815e700 1 Processor -- start 2023-10-08T00:10:57.912 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.912+0000 7fda4815e700 1 -- start start 2023-10-08T00:10:57.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.912+0000 7fda4815e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fda401577d0 0x7fda400cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:57.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.912+0000 7fda4815e700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda4015c360 con 0x7fda4006ac30 2023-10-08T00:10:57.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.912+0000 7fda4815e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda4015b790 con 0x7fda40157ba0 2023-10-08T00:10:57.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.912+0000 7fda4815e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda4015ba30 con 0x7fda401577d0 2023-10-08T00:10:57.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.912+0000 7fda466fb700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fda4006ac30 0x7fda4015c7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50746/0 (socket says 172.21.15.99:50746) 2023-10-08T00:10:57.917 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.912+0000 7fda466fb700 1 -- 172.21.15.99:0/3504526764 learned_addr learned my addr 172.21.15.99:0/3504526764 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:10:57.917 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.913+0000 7fda44ef8700 1 -- 172.21.15.99:0/3504526764 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 941982587 0 0) 0x7fda4015c360 con 0x7fda4006ac30 2023-10-08T00:10:57.917 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.913+0000 7fda45efa700 1 --2- 172.21.15.99:0/3504526764 >> v2:172.21.15.191:3300/0 conn(0x7fda401577d0 0x7fda400cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:57.917 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.913+0000 7fda44ef8700 1 -- 172.21.15.99:0/3504526764 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda2c003570 con 0x7fda4006ac30 2023-10-08T00:10:57.917 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.913+0000 7fda44ef8700 1 -- 172.21.15.99:0/3504526764 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1614193307 0 0) 0x7fda2c003570 con 0x7fda4006ac30 2023-10-08T00:10:57.917 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.913+0000 7fda44ef8700 1 -- 172.21.15.99:0/3504526764 >> v2:172.21.15.191:3300/0 conn(0x7fda401577d0 msgr2=0x7fda400cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:57.917 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.913+0000 7fda44ef8700 1 --2- 172.21.15.99:0/3504526764 >> v2:172.21.15.191:3300/0 conn(0x7fda401577d0 0x7fda400cd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:57.917 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.913+0000 7fda44ef8700 1 -- 172.21.15.99:0/3504526764 >> v1:172.21.15.191:6789/0 conn(0x7fda40157ba0 legacy=0x7fda4006a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:57.917 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.913+0000 7fda44ef8700 1 -- 172.21.15.99:0/3504526764 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda4015bd10 con 0x7fda4006ac30 2023-10-08T00:10:57.917 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.913+0000 7fda44ef8700 1 -- 172.21.15.99:0/3504526764 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7fda380017f0 con 0x7fda4006ac30 2023-10-08T00:10:57.917 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.913+0000 7fda44ef8700 1 -- 172.21.15.99:0/3504526764 >> v1:172.21.15.99:6789/0 conn(0x7fda4006ac30 legacy=0x7fda4015c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:57.918 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.913+0000 7fda44ef8700 1 --2- 172.21.15.99:0/3504526764 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda2c003a60 0x7fda2c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:57.918 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.913+0000 7fda44ef8700 1 --2- 172.21.15.99:0/3504526764 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda2c004810 0x7fda2c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:57.918 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.913+0000 7fda44ef8700 1 --2- 172.21.15.99:0/3504526764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda2c0051c0 0x7fda2c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:57.918 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.913+0000 7fda44ef8700 1 -- 172.21.15.99:0/3504526764 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4015ba30 con 0x7fda2c0051c0 2023-10-08T00:10:57.918 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.913+0000 7fda44ef8700 1 -- 172.21.15.99:0/3504526764 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c00c160 con 0x7fda2c003a60 2023-10-08T00:10:57.918 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.913+0000 7fda44ef8700 1 -- 172.21.15.99:0/3504526764 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c00c320 con 0x7fda2c004810 2023-10-08T00:10:57.918 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.913+0000 7fda466fb700 1 --2- 172.21.15.99:0/3504526764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda2c0051c0 0x7fda2c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:57.918 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.913+0000 7fda45efa700 1 --2- 172.21.15.99:0/3504526764 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda2c003a60 0x7fda2c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:57.918 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.913+0000 7fda456f9700 1 --2- 172.21.15.99:0/3504526764 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda2c004810 0x7fda2c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:57.918 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.913+0000 7fda466fb700 1 -- 172.21.15.99:0/3504526764 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda2c004810 msgr2=0x7fda2c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:57.918 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.913+0000 7fda466fb700 1 --2- 172.21.15.99:0/3504526764 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda2c004810 0x7fda2c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:57.918 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.913+0000 7fda466fb700 1 -- 172.21.15.99:0/3504526764 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda2c003a60 msgr2=0x7fda2c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:57.919 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.913+0000 7fda466fb700 1 --2- 172.21.15.99:0/3504526764 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda2c003a60 0x7fda2c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:57.919 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.913+0000 7fda466fb700 1 -- 172.21.15.99:0/3504526764 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda2c003570 con 0x7fda2c0051c0 2023-10-08T00:10:57.919 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.913+0000 7fda466fb700 1 -- 172.21.15.99:0/3504526764 reap_dead start 2023-10-08T00:10:57.919 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.914+0000 7fda466fb700 1 --2- 172.21.15.99:0/3504526764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda2c0051c0 0x7fda2c00ba60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fda4006b030 tx=0x7fda380021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67fc4374a571d015 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:57.919 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.914+0000 7fda44ef8700 1 -- 172.21.15.99:0/3504526764 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda380070d0 con 0x7fda2c0051c0 2023-10-08T00:10:57.919 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.914+0000 7fda44ef8700 1 -- 172.21.15.99:0/3504526764 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda38007230 con 0x7fda2c0051c0 2023-10-08T00:10:57.919 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.914+0000 7fda44ef8700 1 -- 172.21.15.99:0/3504526764 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda380075d0 con 0x7fda2c0051c0 2023-10-08T00:10:57.919 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.914+0000 7fda4815e700 1 -- 172.21.15.99:0/3504526764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda2c0051c0 msgr2=0x7fda2c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:57.919 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.914+0000 7fda4815e700 1 --2- 172.21.15.99:0/3504526764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda2c0051c0 0x7fda2c00ba60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fda4006b030 tx=0x7fda380021d0 comp rx=0 tx=0).stop 2023-10-08T00:10:57.919 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.914+0000 7fda4815e700 1 -- 172.21.15.99:0/3504526764 shutdown_connections 2023-10-08T00:10:57.919 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.914+0000 7fda4815e700 1 --2- 172.21.15.99:0/3504526764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda2c0051c0 0x7fda2c00ba60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:57.920 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.914+0000 7fda4815e700 1 -- 172.21.15.99:0/3504526764 >> 172.21.15.99:0/3504526764 conn(0x7fda40068ce0 msgr2=0x7fda400690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:57.920 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.914+0000 7fda4815e700 1 -- 172.21.15.99:0/3504526764 shutdown_connections 2023-10-08T00:10:57.920 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.914+0000 7fda4815e700 1 -- 172.21.15.99:0/3504526764 wait complete. 2023-10-08T00:10:57.920 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.915+0000 7fda4815e700 1 Processor -- start 2023-10-08T00:10:57.920 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.915+0000 7fda4815e700 1 -- start start 2023-10-08T00:10:57.922 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.916+0000 7fda4815e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda2c0051c0 0x7fda4014c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:57.922 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.916+0000 7fda4815e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda400ac950 0x7fda400acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:57.923 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.916+0000 7fda4815e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda40159170 0x7fda40159610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:57.923 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.916+0000 7fda4815e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4014ce20 con 0x7fda40159170 2023-10-08T00:10:57.923 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.916+0000 7fda4815e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda40159b50 con 0x7fda400ac950 2023-10-08T00:10:57.923 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.916+0000 7fda4815e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda400ad410 con 0x7fda2c0051c0 2023-10-08T00:10:57.923 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.916+0000 7fda466fb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda40159170 0x7fda40159610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:57.923 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.916+0000 7fda466fb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda40159170 0x7fda40159610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49496/0 (socket says 172.21.15.99:49496) 2023-10-08T00:10:57.923 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.916+0000 7fda466fb700 1 -- 172.21.15.99:0/2362408722 learned_addr learned my addr 172.21.15.99:0/2362408722 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:10:57.923 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.916+0000 7fda45efa700 1 --2- 172.21.15.99:0/2362408722 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda2c0051c0 0x7fda4014c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:57.923 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.916+0000 7fda456f9700 1 --2- 172.21.15.99:0/2362408722 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda400ac950 0x7fda400acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:57.923 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.916+0000 7fda466fb700 1 -- 172.21.15.99:0/2362408722 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda2c0051c0 msgr2=0x7fda4014c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:57.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.916+0000 7fda466fb700 1 --2- 172.21.15.99:0/2362408722 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda2c0051c0 0x7fda4014c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:57.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.916+0000 7fda466fb700 1 -- 172.21.15.99:0/2362408722 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda400ac950 msgr2=0x7fda400acdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:57.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.916+0000 7fda466fb700 1 --2- 172.21.15.99:0/2362408722 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda400ac950 0x7fda400acdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:57.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.917+0000 7fda466fb700 1 -- 172.21.15.99:0/2362408722 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda4016a7b0 con 0x7fda40159170 2023-10-08T00:10:57.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.917+0000 7fda466fb700 1 --2- 172.21.15.99:0/2362408722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda40159170 0x7fda40159610 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fda40071de0 tx=0x7fda38016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:57.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.917+0000 7fda35ffb700 1 -- 172.21.15.99:0/2362408722 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda380070d0 con 0x7fda40159170 2023-10-08T00:10:57.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.917+0000 7fda35ffb700 1 -- 172.21.15.99:0/2362408722 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda380017b0 con 0x7fda40159170 2023-10-08T00:10:57.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.917+0000 7fda35ffb700 1 -- 172.21.15.99:0/2362408722 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda38001920 con 0x7fda40159170 2023-10-08T00:10:57.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.917+0000 7fda4815e700 1 -- 172.21.15.99:0/2362408722 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda4016aa90 con 0x7fda40159170 2023-10-08T00:10:57.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.917+0000 7fda4815e700 1 -- 172.21.15.99:0/2362408722 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda4016b050 con 0x7fda40159170 2023-10-08T00:10:57.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.918+0000 7fda4815e700 1 -- 172.21.15.99:0/2362408722 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda400ab9c0 con 0x7fda40159170 2023-10-08T00:10:57.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.918+0000 7fda35ffb700 1 -- 172.21.15.99:0/2362408722 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fda3801d420 con 0x7fda40159170 2023-10-08T00:10:57.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.919+0000 7fda35ffb700 1 --2- 172.21.15.99:0/2362408722 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fda24066f30 0x7fda240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:57.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.919+0000 7fda35ffb700 1 -- 172.21.15.99:0/2362408722 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fda38085fb0 con 0x7fda40159170 2023-10-08T00:10:57.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.919+0000 7fda45efa700 1 --2- 172.21.15.99:0/2362408722 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fda24066f30 0x7fda240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:57.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.919+0000 7fda45efa700 1 --2- 172.21.15.99:0/2362408722 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fda24066f30 0x7fda240693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fda400ae770 tx=0x7fda3c0091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:57.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:57.923+0000 7fda35ffb700 1 -- 172.21.15.99:0/2362408722 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fda400ab9c0 con 0x7fda40159170 2023-10-08T00:10:58.102 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.102+0000 7fda4815e700 1 -- 172.21.15.99:0/2362408722 --> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fda4016b240 con 0x7fda24066f30 2023-10-08T00:10:58.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.110+0000 7fda35ffb700 1 -- 172.21.15.99:0/2362408722 <== mgr.4099 v2:172.21.15.191:6832/85985 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251329 (secure 0 0 0) 0x7fda4016b240 con 0x7fda24066f30 2023-10-08T00:10:58.114 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:10:58.116 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.113+0000 7fda4815e700 1 -- 172.21.15.99:0/2362408722 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fda24066f30 msgr2=0x7fda240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:58.116 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.113+0000 7fda4815e700 1 --2- 172.21.15.99:0/2362408722 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fda24066f30 0x7fda240693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fda400ae770 tx=0x7fda3c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:10:58.116 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.113+0000 7fda4815e700 1 -- 172.21.15.99:0/2362408722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda40159170 msgr2=0x7fda40159610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:58.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.113+0000 7fda4815e700 1 --2- 172.21.15.99:0/2362408722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda40159170 0x7fda40159610 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fda40071de0 tx=0x7fda38016000 comp rx=0 tx=0).stop 2023-10-08T00:10:58.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.113+0000 7fda4815e700 1 -- 172.21.15.99:0/2362408722 shutdown_connections 2023-10-08T00:10:58.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.113+0000 7fda4815e700 1 --2- 172.21.15.99:0/2362408722 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fda24066f30 0x7fda240693e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:58.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.113+0000 7fda4815e700 1 --2- 172.21.15.99:0/2362408722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda40159170 0x7fda40159610 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:58.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.113+0000 7fda4815e700 1 --2- 172.21.15.99:0/2362408722 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda2c0051c0 0x7fda4014c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:58.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.113+0000 7fda4815e700 1 --2- 172.21.15.99:0/2362408722 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda400ac950 0x7fda400acdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:58.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.113+0000 7fda4815e700 1 -- 172.21.15.99:0/2362408722 >> 172.21.15.99:0/2362408722 conn(0x7fda40068ce0 msgr2=0x7fda400d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:58.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.113+0000 7fda4815e700 1 -- 172.21.15.99:0/2362408722 shutdown_connections 2023-10-08T00:10:58.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.113+0000 7fda4815e700 1 -- 172.21.15.99:0/2362408722 wait complete. 2023-10-08T00:10:58.117 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2023-10-08T00:10:58.125 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2023-10-08T00:10:58.024843+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":7,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1552,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560947,"statfs":{"total":1600000000,"available":1598409824,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.952830"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.071916+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:00:50.648234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.068998+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:48:47.624288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.070061+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:23:55.581278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.070506+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:16:40.636514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.069315+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:08:39.785742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.072673+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:16:40.636514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.071429+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:08:39.785742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.072393+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:00:50.648234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.070809+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:48:47.624288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.070483+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:08:39.785742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.069507+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:00:50.648234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.069553+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:48:47.624288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.069081+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:48:47.624288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.072884+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:08:25.404396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.070119+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:48:58.606951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.071176+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:23:55.581278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.070171+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:00:50.648234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.069019+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:00:50.648234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.072572+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:08:39.785742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.069029+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:48:58.606951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.072150+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:48:47.624288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":160,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-08T00:10:44.729923+0000","last_change":"2023-10-08T00:10:43.728490+0000","last_active":"2023-10-08T00:10:44.729923+0000","last_peered":"2023-10-08T00:10:44.729923+0000","last_clean":"2023-10-08T00:10:44.729923+0000","last_became_active":"2023-10-08T00:10:43.728341+0000","last_became_peered":"2023-10-08T00:10:43.728341+0000","last_unstale":"2023-10-08T00:10:44.729923+0000","last_undegraded":"2023-10-08T00:10:44.729923+0000","last_fullsized":"2023-10-08T00:10:44.729923+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_clean_scrub_stamp":"2023-10-08T00:10:42.716096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:06:31.035293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.072171+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:23:55.581278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.073149+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:10:42.328030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.071633+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:48:47.624288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.072117+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:48:58.606951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.070144+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:48:47.624288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.070658+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:16:07.345714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.071086+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:00:50.648234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.072116+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:16:07.345714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.070888+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:48:47.624288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.069442+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:16:40.636514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.072128+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:48:58.606951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948820,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948820,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":626,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194685,"statfs":{"total":200000000,"available":199358452,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948820,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948820,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948820,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.318}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948820,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":626,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194685,"statfs":{"total":200000000,"available":199358452,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]}]}],"pool_statfs":[]}} 2023-10-08T00:10:58.132 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T00:10:58.214 INFO:tasks.ceph.mds.d.smithi191.stderr:2023-10-08T00:10:58.213+0000 7f0f0262b700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T00:10:58.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.279+0000 7ff70b363700 1 Processor -- start 2023-10-08T00:10:58.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.280+0000 7ff70b363700 1 -- start start 2023-10-08T00:10:58.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.280+0000 7ff70b363700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff70415a2b0 0x7ff7040b3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:58.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.280+0000 7ff70b363700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff7040d1c30 0x7ff7040cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:58.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.280+0000 7ff70b363700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7040c5d80 con 0x7ff70415a680 2023-10-08T00:10:58.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.280+0000 7ff70b363700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff704068470 con 0x7ff70415a2b0 2023-10-08T00:10:58.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.280+0000 7ff70b363700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7040c6060 con 0x7ff7040d1c30 2023-10-08T00:10:58.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.280+0000 7ff7090ff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff70415a2b0 0x7ff7040b3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:58.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.280+0000 7ff709900700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7ff70415a680 0x7ff7040c7100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50752/0 (socket says 172.21.15.99:50752) 2023-10-08T00:10:58.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.280+0000 7ff709900700 1 -- 172.21.15.99:0/1984941391 learned_addr learned my addr 172.21.15.99:0/1984941391 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:10:58.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.281+0000 7ff7088fe700 1 --2- 172.21.15.99:0/1984941391 >> v2:172.21.15.191:3300/0 conn(0x7ff7040d1c30 0x7ff7040cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:58.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.281+0000 7ff7090ff700 1 -- 172.21.15.99:0/1984941391 >> v2:172.21.15.191:3300/0 conn(0x7ff7040d1c30 msgr2=0x7ff7040cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:58.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.281+0000 7ff7090ff700 1 --2- 172.21.15.99:0/1984941391 >> v2:172.21.15.191:3300/0 conn(0x7ff7040d1c30 0x7ff7040cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:58.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.281+0000 7ff7090ff700 1 -- 172.21.15.99:0/1984941391 >> v1:172.21.15.99:6789/0 conn(0x7ff70415a680 legacy=0x7ff7040c7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:58.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.281+0000 7ff7090ff700 1 -- 172.21.15.99:0/1984941391 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7040c6340 con 0x7ff70415a2b0 2023-10-08T00:10:58.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.281+0000 7ff6fbfff700 1 -- 172.21.15.99:0/1984941391 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2186017725 0 0) 0x7ff7040c5d80 con 0x7ff70415a680 2023-10-08T00:10:58.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.281+0000 7ff7090ff700 1 --2- 172.21.15.99:0/1984941391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff70415a2b0 0x7ff7040b3da0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff704089190 tx=0x7ff700009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a61a1c115ffe818 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:58.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.281+0000 7ff6fbfff700 1 -- 172.21.15.99:0/1984941391 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff700005c10 con 0x7ff70415a2b0 2023-10-08T00:10:58.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.281+0000 7ff6fbfff700 1 -- 172.21.15.99:0/1984941391 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff700005d70 con 0x7ff70415a2b0 2023-10-08T00:10:58.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.281+0000 7ff6fbfff700 1 -- 172.21.15.99:0/1984941391 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff700006260 con 0x7ff70415a2b0 2023-10-08T00:10:58.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.281+0000 7ff70b363700 1 -- 172.21.15.99:0/1984941391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff70415a2b0 msgr2=0x7ff7040b3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:58.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.281+0000 7ff70b363700 1 --2- 172.21.15.99:0/1984941391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff70415a2b0 0x7ff7040b3da0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff704089190 tx=0x7ff700009120 comp rx=0 tx=0).stop 2023-10-08T00:10:58.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.282+0000 7ff70b363700 1 -- 172.21.15.99:0/1984941391 shutdown_connections 2023-10-08T00:10:58.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.282+0000 7ff70b363700 1 --2- 172.21.15.99:0/1984941391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff70415a2b0 0x7ff7040b3da0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:58.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.282+0000 7ff70b363700 1 --2- 172.21.15.99:0/1984941391 >> v2:172.21.15.191:3300/0 conn(0x7ff7040d1c30 0x7ff7040cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:58.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.282+0000 7ff70b363700 1 -- 172.21.15.99:0/1984941391 >> 172.21.15.99:0/1984941391 conn(0x7ff704067e40 msgr2=0x7ff7040d0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:58.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.282+0000 7ff70b363700 1 -- 172.21.15.99:0/1984941391 shutdown_connections 2023-10-08T00:10:58.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.282+0000 7ff70b363700 1 -- 172.21.15.99:0/1984941391 wait complete. 2023-10-08T00:10:58.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.283+0000 7ff70b363700 1 Processor -- start 2023-10-08T00:10:58.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.283+0000 7ff70b363700 1 -- start start 2023-10-08T00:10:58.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.284+0000 7ff70b363700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7040d1c30 0x7ff70415d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:58.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.284+0000 7ff70b363700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff70415a2b0 0x7ff70415b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:58.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.284+0000 7ff70b363700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff70415a680 0x7ff704157b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:58.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.284+0000 7ff70b363700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7040c9ca0 con 0x7ff7040d1c30 2023-10-08T00:10:58.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.284+0000 7ff70b363700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70415bb00 con 0x7ff70415a680 2023-10-08T00:10:58.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.284+0000 7ff70b363700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff70416a280 con 0x7ff70415a2b0 2023-10-08T00:10:58.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.284+0000 7ff7090ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7040d1c30 0x7ff70415d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:58.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.284+0000 7ff7090ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7040d1c30 0x7ff70415d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49508/0 (socket says 172.21.15.99:49508) 2023-10-08T00:10:58.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.284+0000 7ff7090ff700 1 -- 172.21.15.99:0/2089832061 learned_addr learned my addr 172.21.15.99:0/2089832061 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:10:58.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.284+0000 7ff7090ff700 1 -- 172.21.15.99:0/2089832061 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff70415a2b0 msgr2=0x7ff70415b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:58.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.284+0000 7ff7090ff700 1 --2- 172.21.15.99:0/2089832061 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff70415a2b0 0x7ff70415b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:58.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.284+0000 7ff7090ff700 1 -- 172.21.15.99:0/2089832061 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff70415a680 msgr2=0x7ff704157b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:58.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.284+0000 7ff7090ff700 1 --2- 172.21.15.99:0/2089832061 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff70415a680 0x7ff704157b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:58.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.284+0000 7ff7090ff700 1 -- 172.21.15.99:0/2089832061 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7000093c0 con 0x7ff7040d1c30 2023-10-08T00:10:58.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.284+0000 7ff7090ff700 1 --2- 172.21.15.99:0/2089832061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7040d1c30 0x7ff70415d500 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff700009f40 tx=0x7ff700016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:58.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.284+0000 7ff6f97fa700 1 -- 172.21.15.99:0/2089832061 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70001d880 con 0x7ff7040d1c30 2023-10-08T00:10:58.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.284+0000 7ff6f97fa700 1 -- 172.21.15.99:0/2089832061 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff70001da00 con 0x7ff7040d1c30 2023-10-08T00:10:58.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.284+0000 7ff6f97fa700 1 -- 172.21.15.99:0/2089832061 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70001ddc0 con 0x7ff7040d1c30 2023-10-08T00:10:58.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.284+0000 7ff70b363700 1 -- 172.21.15.99:0/2089832061 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff70415bd60 con 0x7ff7040d1c30 2023-10-08T00:10:58.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.284+0000 7ff70b363700 1 -- 172.21.15.99:0/2089832061 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7040c9650 con 0x7ff7040d1c30 2023-10-08T00:10:58.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.285+0000 7ff70b363700 1 -- 172.21.15.99:0/2089832061 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7040c94b0 con 0x7ff7040d1c30 2023-10-08T00:10:58.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.285+0000 7ff6f97fa700 1 -- 172.21.15.99:0/2089832061 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff70001b690 con 0x7ff7040d1c30 2023-10-08T00:10:58.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.286+0000 7ff6f97fa700 1 --2- 172.21.15.99:0/2089832061 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff6e8066f60 0x7ff6e8069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:58.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.286+0000 7ff6f97fa700 1 -- 172.21.15.99:0/2089832061 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff7000862c0 con 0x7ff7040d1c30 2023-10-08T00:10:58.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.286+0000 7ff7088fe700 1 --2- 172.21.15.99:0/2089832061 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff6e8066f60 0x7ff6e8069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:58.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.287+0000 7ff7088fe700 1 --2- 172.21.15.99:0/2089832061 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff6e8066f60 0x7ff6e8069410 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff7040c8140 tx=0x7ff6f40091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:58.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.290+0000 7ff6f97fa700 1 -- 172.21.15.99:0/2089832061 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff7040c94b0 con 0x7ff7040d1c30 2023-10-08T00:10:58.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.466+0000 7ff70b363700 1 -- 172.21.15.99:0/2089832061 --> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff7040c94b0 con 0x7ff6e8066f60 2023-10-08T00:10:58.474 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.474+0000 7ff6f97fa700 1 -- 172.21.15.99:0/2089832061 <== mgr.4099 v2:172.21.15.191:6832/85985 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251329 (secure 0 0 0) 0x7ff7040c94b0 con 0x7ff6e8066f60 2023-10-08T00:10:58.477 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:10:58.479 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.476+0000 7ff70b363700 1 -- 172.21.15.99:0/2089832061 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff6e8066f60 msgr2=0x7ff6e8069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:58.479 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.476+0000 7ff70b363700 1 --2- 172.21.15.99:0/2089832061 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff6e8066f60 0x7ff6e8069410 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff7040c8140 tx=0x7ff6f40091a0 comp rx=0 tx=0).stop 2023-10-08T00:10:58.479 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.476+0000 7ff70b363700 1 -- 172.21.15.99:0/2089832061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7040d1c30 msgr2=0x7ff70415d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:58.479 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.476+0000 7ff70b363700 1 --2- 172.21.15.99:0/2089832061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7040d1c30 0x7ff70415d500 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff700009f40 tx=0x7ff700016000 comp rx=0 tx=0).stop 2023-10-08T00:10:58.480 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.476+0000 7ff70b363700 1 -- 172.21.15.99:0/2089832061 shutdown_connections 2023-10-08T00:10:58.480 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.476+0000 7ff70b363700 1 --2- 172.21.15.99:0/2089832061 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff6e8066f60 0x7ff6e8069410 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:58.480 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.476+0000 7ff70b363700 1 --2- 172.21.15.99:0/2089832061 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff70415a680 0x7ff704157b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:58.480 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.476+0000 7ff70b363700 1 --2- 172.21.15.99:0/2089832061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7040d1c30 0x7ff70415d500 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:58.480 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.476+0000 7ff70b363700 1 --2- 172.21.15.99:0/2089832061 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff70415a2b0 0x7ff70415b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:58.480 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.476+0000 7ff70b363700 1 -- 172.21.15.99:0/2089832061 >> 172.21.15.99:0/2089832061 conn(0x7ff704067e40 msgr2=0x7ff7041595e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:58.480 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.476+0000 7ff70b363700 1 -- 172.21.15.99:0/2089832061 shutdown_connections 2023-10-08T00:10:58.480 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.476+0000 7ff70b363700 1 -- 172.21.15.99:0/2089832061 wait complete. 2023-10-08T00:10:58.480 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2023-10-08T00:10:58.488 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2023-10-08T00:10:58.024843+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":7,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1552,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560947,"statfs":{"total":1600000000,"available":1598409824,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.952830"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.071916+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:00:50.648234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.068998+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:48:47.624288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.070061+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:23:55.581278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.070506+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:16:40.636514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.069315+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:08:39.785742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.072673+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:16:40.636514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.071429+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:08:39.785742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.072393+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:00:50.648234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.070809+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:48:47.624288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.070483+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:08:39.785742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.069507+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:00:50.648234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.069553+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:48:47.624288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.069081+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:48:47.624288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.072884+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:08:25.404396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.070119+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:48:58.606951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.071176+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:23:55.581278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.070171+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:00:50.648234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.069019+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:00:50.648234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.072572+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:08:39.785742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.069029+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:48:58.606951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.072150+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:48:47.624288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":160,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-08T00:10:44.729923+0000","last_change":"2023-10-08T00:10:43.728490+0000","last_active":"2023-10-08T00:10:44.729923+0000","last_peered":"2023-10-08T00:10:44.729923+0000","last_clean":"2023-10-08T00:10:44.729923+0000","last_became_active":"2023-10-08T00:10:43.728341+0000","last_became_peered":"2023-10-08T00:10:43.728341+0000","last_unstale":"2023-10-08T00:10:44.729923+0000","last_undegraded":"2023-10-08T00:10:44.729923+0000","last_fullsized":"2023-10-08T00:10:44.729923+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_clean_scrub_stamp":"2023-10-08T00:10:42.716096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:06:31.035293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.072171+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:23:55.581278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.073149+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:10:42.328030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.071633+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:48:47.624288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.072117+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:48:58.606951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.070144+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:48:47.624288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.070658+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:16:07.345714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.071086+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:00:50.648234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.072116+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:16:07.345714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.070888+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:48:47.624288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.069442+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:16:40.636514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.072128+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:48:58.606951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948820,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948820,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":626,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194685,"statfs":{"total":200000000,"available":199358452,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948820,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948820,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948820,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.318}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948820,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":626,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194685,"statfs":{"total":200000000,"available":199358452,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]}]}],"pool_statfs":[]}} 2023-10-08T00:10:58.494 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T00:10:58.641 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.641+0000 7feaf0ddf700 1 Processor -- start 2023-10-08T00:10:58.641 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.641+0000 7feaf0ddf700 1 -- start start 2023-10-08T00:10:58.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.642+0000 7feaf0ddf700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7feaec157c40 0x7feaec158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:58.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.642+0000 7feaf0ddf700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaec15cbb0 con 0x7feaec157870 2023-10-08T00:10:58.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.642+0000 7feaf0ddf700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaec15bfa0 con 0x7feaec15a620 2023-10-08T00:10:58.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.642+0000 7feaf0ddf700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaec15bdb0 con 0x7feaec157c40 2023-10-08T00:10:58.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.642+0000 7feaea59c700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7feaec157870 0x7feaec15b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50762/0 (socket says 172.21.15.99:50762) 2023-10-08T00:10:58.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.642+0000 7feaea59c700 1 -- 172.21.15.99:0/998702921 learned_addr learned my addr 172.21.15.99:0/998702921 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:10:58.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.642+0000 7feaead9d700 1 --2- 172.21.15.99:0/998702921 >> v2:172.21.15.191:3300/0 conn(0x7feaec157c40 0x7feaec158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:58.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.642+0000 7feae959a700 1 -- 172.21.15.99:0/998702921 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2825339582 0 0) 0x7feaec15cbb0 con 0x7feaec157870 2023-10-08T00:10:58.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.642+0000 7feae959a700 1 -- 172.21.15.99:0/998702921 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fead8003570 con 0x7feaec157870 2023-10-08T00:10:58.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.642+0000 7feae959a700 1 -- 172.21.15.99:0/998702921 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1964870010 0 0) 0x7fead8003570 con 0x7feaec157870 2023-10-08T00:10:58.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.642+0000 7feae959a700 1 -- 172.21.15.99:0/998702921 >> v2:172.21.15.191:3300/0 conn(0x7feaec157c40 msgr2=0x7feaec158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:58.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.642+0000 7feae959a700 1 --2- 172.21.15.99:0/998702921 >> v2:172.21.15.191:3300/0 conn(0x7feaec157c40 0x7feaec158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:58.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.642+0000 7feae959a700 1 -- 172.21.15.99:0/998702921 >> v1:172.21.15.191:6789/0 conn(0x7feaec15a620 legacy=0x7feaec15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:58.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.642+0000 7feae959a700 1 -- 172.21.15.99:0/998702921 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaec15c280 con 0x7feaec157870 2023-10-08T00:10:58.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.642+0000 7feae959a700 1 -- 172.21.15.99:0/998702921 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7feae40017f0 con 0x7feaec157870 2023-10-08T00:10:58.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.642+0000 7feae959a700 1 -- 172.21.15.99:0/998702921 >> v1:172.21.15.99:6789/0 conn(0x7feaec157870 legacy=0x7feaec15b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:58.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.642+0000 7feae959a700 1 --2- 172.21.15.99:0/998702921 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fead8003a60 0x7fead8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:58.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.642+0000 7feae959a700 1 --2- 172.21.15.99:0/998702921 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fead8004810 0x7fead8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:58.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.642+0000 7feae959a700 1 --2- 172.21.15.99:0/998702921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fead80052e0 0x7fead800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:58.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.642+0000 7feae959a700 1 -- 172.21.15.99:0/998702921 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead800c200 con 0x7fead80052e0 2023-10-08T00:10:58.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.642+0000 7feae959a700 1 -- 172.21.15.99:0/998702921 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead800c370 con 0x7fead8003a60 2023-10-08T00:10:58.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.642+0000 7feae959a700 1 -- 172.21.15.99:0/998702921 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fead800c530 con 0x7fead8004810 2023-10-08T00:10:58.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.642+0000 7feaead9d700 1 --2- 172.21.15.99:0/998702921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fead80052e0 0x7fead800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:58.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.642+0000 7feaea59c700 1 --2- 172.21.15.99:0/998702921 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fead8003a60 0x7fead8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:58.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.643+0000 7feae9d9b700 1 --2- 172.21.15.99:0/998702921 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fead8004810 0x7fead8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:58.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.643+0000 7feaead9d700 1 -- 172.21.15.99:0/998702921 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fead8004810 msgr2=0x7fead8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:58.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.643+0000 7feaead9d700 1 --2- 172.21.15.99:0/998702921 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fead8004810 0x7fead8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:58.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.643+0000 7feaead9d700 1 -- 172.21.15.99:0/998702921 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fead8003a60 msgr2=0x7fead8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:58.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.643+0000 7feaead9d700 1 --2- 172.21.15.99:0/998702921 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fead8003a60 0x7fead8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:58.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.643+0000 7feaead9d700 1 -- 172.21.15.99:0/998702921 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead8003570 con 0x7fead80052e0 2023-10-08T00:10:58.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.643+0000 7feaead9d700 1 -- 172.21.15.99:0/998702921 reap_dead start 2023-10-08T00:10:58.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.643+0000 7feaead9d700 1 --2- 172.21.15.99:0/998702921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fead80052e0 0x7fead800ba60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7feadc00bb70 tx=0x7feadc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acfd0e85ff97835e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:58.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.643+0000 7feae959a700 1 -- 172.21.15.99:0/998702921 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc00b4e0 con 0x7fead80052e0 2023-10-08T00:10:58.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.643+0000 7feae959a700 1 -- 172.21.15.99:0/998702921 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feadc00d5d0 con 0x7fead80052e0 2023-10-08T00:10:58.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.643+0000 7feae959a700 1 -- 172.21.15.99:0/998702921 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc00d990 con 0x7fead80052e0 2023-10-08T00:10:58.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.643+0000 7feaf0ddf700 1 -- 172.21.15.99:0/998702921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fead80052e0 msgr2=0x7fead800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:58.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.643+0000 7feaf0ddf700 1 --2- 172.21.15.99:0/998702921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fead80052e0 0x7fead800ba60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7feadc00bb70 tx=0x7feadc008040 comp rx=0 tx=0).stop 2023-10-08T00:10:58.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.644+0000 7feaf0ddf700 1 -- 172.21.15.99:0/998702921 shutdown_connections 2023-10-08T00:10:58.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.644+0000 7feaf0ddf700 1 --2- 172.21.15.99:0/998702921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fead80052e0 0x7fead800ba60 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:58.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.644+0000 7feaf0ddf700 1 -- 172.21.15.99:0/998702921 >> 172.21.15.99:0/998702921 conn(0x7feaec068ce0 msgr2=0x7feaec0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:58.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.644+0000 7feaf0ddf700 1 -- 172.21.15.99:0/998702921 shutdown_connections 2023-10-08T00:10:58.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.644+0000 7feaf0ddf700 1 -- 172.21.15.99:0/998702921 wait complete. 2023-10-08T00:10:58.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.645+0000 7feaf0ddf700 1 Processor -- start 2023-10-08T00:10:58.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.645+0000 7feaf0ddf700 1 -- start start 2023-10-08T00:10:58.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.645+0000 7feaf0ddf700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fead80052e0 0x7feaec13f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:58.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.645+0000 7feaf0ddf700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feaec16aec0 0x7feaec151420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:58.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.645+0000 7feaf0ddf700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feaec151960 0x7feaec0b2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:58.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.645+0000 7feaf0ddf700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaec16bfe0 con 0x7fead80052e0 2023-10-08T00:10:58.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.645+0000 7feaf0ddf700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaec16b3d0 con 0x7feaec151960 2023-10-08T00:10:58.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.645+0000 7feaf0ddf700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaec16b6b0 con 0x7feaec16aec0 2023-10-08T00:10:58.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.645+0000 7feaea59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fead80052e0 0x7feaec13f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:58.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.645+0000 7feaea59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fead80052e0 0x7feaec13f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49518/0 (socket says 172.21.15.99:49518) 2023-10-08T00:10:58.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.645+0000 7feaea59c700 1 -- 172.21.15.99:0/2287598335 learned_addr learned my addr 172.21.15.99:0/2287598335 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:10:58.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.646+0000 7feaead9d700 1 --2- 172.21.15.99:0/2287598335 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feaec151960 0x7feaec0b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:58.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.646+0000 7feae9d9b700 1 --2- 172.21.15.99:0/2287598335 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feaec16aec0 0x7feaec151420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:58.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.646+0000 7feaea59c700 1 -- 172.21.15.99:0/2287598335 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feaec16aec0 msgr2=0x7feaec151420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:58.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.646+0000 7feaea59c700 1 --2- 172.21.15.99:0/2287598335 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feaec16aec0 0x7feaec151420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:58.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.646+0000 7feaea59c700 1 -- 172.21.15.99:0/2287598335 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feaec151960 msgr2=0x7feaec0b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:58.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.646+0000 7feaea59c700 1 --2- 172.21.15.99:0/2287598335 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feaec151960 0x7feaec0b2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:58.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.646+0000 7feaea59c700 1 -- 172.21.15.99:0/2287598335 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaec16b990 con 0x7fead80052e0 2023-10-08T00:10:58.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.646+0000 7feaead9d700 1 --2- 172.21.15.99:0/2287598335 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feaec151960 0x7feaec0b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:10:58.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.646+0000 7feaea59c700 1 --2- 172.21.15.99:0/2287598335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fead80052e0 0x7feaec13f650 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7feae400a0c0 tx=0x7feae40021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:58.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.646+0000 7feae27fc700 1 -- 172.21.15.99:0/2287598335 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae400ada0 con 0x7fead80052e0 2023-10-08T00:10:58.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.646+0000 7feae27fc700 1 -- 172.21.15.99:0/2287598335 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae40133f0 con 0x7fead80052e0 2023-10-08T00:10:58.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.646+0000 7feaf0ddf700 1 -- 172.21.15.99:0/2287598335 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaec16bc70 con 0x7fead80052e0 2023-10-08T00:10:58.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.646+0000 7feae27fc700 1 -- 172.21.15.99:0/2287598335 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae40137b0 con 0x7fead80052e0 2023-10-08T00:10:58.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.646+0000 7feaf0ddf700 1 -- 172.21.15.99:0/2287598335 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaec0b2580 con 0x7fead80052e0 2023-10-08T00:10:58.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.647+0000 7feaf0ddf700 1 -- 172.21.15.99:0/2287598335 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaec04c3c0 con 0x7fead80052e0 2023-10-08T00:10:58.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.647+0000 7feae27fc700 1 -- 172.21.15.99:0/2287598335 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7feae4013b60 con 0x7fead80052e0 2023-10-08T00:10:58.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.648+0000 7feae27fc700 1 --2- 172.21.15.99:0/2287598335 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7feacc066f60 0x7feacc069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:58.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.648+0000 7feae27fc700 1 -- 172.21.15.99:0/2287598335 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7feae4085320 con 0x7fead80052e0 2023-10-08T00:10:58.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.648+0000 7feae9d9b700 1 --2- 172.21.15.99:0/2287598335 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7feacc066f60 0x7feacc069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:58.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.648+0000 7feae9d9b700 1 --2- 172.21.15.99:0/2287598335 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7feacc066f60 0x7feacc069410 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fead4007e40 tx=0x7fead4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:58.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.652+0000 7feae27fc700 1 -- 172.21.15.99:0/2287598335 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feae4052590 con 0x7fead80052e0 2023-10-08T00:10:58.829 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.828+0000 7feaf0ddf700 1 -- 172.21.15.99:0/2287598335 --> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7feaec16c120 con 0x7feacc066f60 2023-10-08T00:10:58.836 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.836+0000 7feae27fc700 1 -- 172.21.15.99:0/2287598335 <== mgr.4099 v2:172.21.15.191:6832/85985 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251329 (secure 0 0 0) 0x7fead4002640 con 0x7feacc066f60 2023-10-08T00:10:58.839 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:10:58.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.838+0000 7feaf0ddf700 1 -- 172.21.15.99:0/2287598335 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7feacc066f60 msgr2=0x7feacc069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:58.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.838+0000 7feaf0ddf700 1 --2- 172.21.15.99:0/2287598335 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7feacc066f60 0x7feacc069410 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fead4007e40 tx=0x7fead4009040 comp rx=0 tx=0).stop 2023-10-08T00:10:58.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.839+0000 7feaf0ddf700 1 -- 172.21.15.99:0/2287598335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fead80052e0 msgr2=0x7feaec13f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:58.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.839+0000 7feaf0ddf700 1 --2- 172.21.15.99:0/2287598335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fead80052e0 0x7feaec13f650 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7feae400a0c0 tx=0x7feae40021d0 comp rx=0 tx=0).stop 2023-10-08T00:10:58.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.839+0000 7feaf0ddf700 1 -- 172.21.15.99:0/2287598335 shutdown_connections 2023-10-08T00:10:58.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.839+0000 7feaf0ddf700 1 --2- 172.21.15.99:0/2287598335 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7feacc066f60 0x7feacc069410 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:58.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.839+0000 7feaf0ddf700 1 --2- 172.21.15.99:0/2287598335 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feaec151960 0x7feaec0b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:58.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.839+0000 7feaf0ddf700 1 --2- 172.21.15.99:0/2287598335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fead80052e0 0x7feaec13f650 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:58.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.839+0000 7feaf0ddf700 1 --2- 172.21.15.99:0/2287598335 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feaec16aec0 0x7feaec151420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:58.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.839+0000 7feaf0ddf700 1 -- 172.21.15.99:0/2287598335 >> 172.21.15.99:0/2287598335 conn(0x7feaec068ce0 msgr2=0x7feaec0b5350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:58.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.839+0000 7feaf0ddf700 1 -- 172.21.15.99:0/2287598335 shutdown_connections 2023-10-08T00:10:58.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:58.839+0000 7feaf0ddf700 1 -- 172.21.15.99:0/2287598335 wait complete. 2023-10-08T00:10:58.842 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2023-10-08T00:10:58.851 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2023-10-08T00:10:58.024843+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":7,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1552,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560947,"statfs":{"total":1600000000,"available":1598409824,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.952830"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.071916+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:00:50.648234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.068998+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:48:47.624288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.070061+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:23:55.581278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.070506+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:16:40.636514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.069315+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:08:39.785742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.072673+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:16:40.636514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.071429+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:08:39.785742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.072393+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:00:50.648234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.070809+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:48:47.624288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.070483+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:08:39.785742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.069507+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:00:50.648234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.069553+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:48:47.624288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.069081+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:48:47.624288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.072884+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:08:25.404396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.070119+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:48:58.606951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.071176+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:23:55.581278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.070171+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:00:50.648234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.069019+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:00:50.648234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.072572+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:08:39.785742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.069029+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:48:58.606951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.072150+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:48:47.624288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":160,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-08T00:10:44.729923+0000","last_change":"2023-10-08T00:10:43.728490+0000","last_active":"2023-10-08T00:10:44.729923+0000","last_peered":"2023-10-08T00:10:44.729923+0000","last_clean":"2023-10-08T00:10:44.729923+0000","last_became_active":"2023-10-08T00:10:43.728341+0000","last_became_peered":"2023-10-08T00:10:43.728341+0000","last_unstale":"2023-10-08T00:10:44.729923+0000","last_undegraded":"2023-10-08T00:10:44.729923+0000","last_fullsized":"2023-10-08T00:10:44.729923+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_clean_scrub_stamp":"2023-10-08T00:10:42.716096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:06:31.035293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.072171+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:23:55.581278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.073149+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:10:42.328030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.071633+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:48:47.624288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.072117+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:48:58.606951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.070144+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:48:47.624288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.070658+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:16:07.345714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.071086+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:00:50.648234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.072116+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:16:07.345714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.070888+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:48:47.624288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.069442+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:16:40.636514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.072128+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:48:58.606951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948820,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948820,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":626,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194685,"statfs":{"total":200000000,"available":199358452,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948820,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948820,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948820,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.318}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948820,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":626,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194685,"statfs":{"total":200000000,"available":199358452,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]}]}],"pool_statfs":[]}} 2023-10-08T00:10:58.860 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3f is not active+clean 2023-10-08T00:10:58.860 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.860 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3e is not active+clean 2023-10-08T00:10:58.860 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.860 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3d is not active+clean 2023-10-08T00:10:58.860 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.861 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3c is not active+clean 2023-10-08T00:10:58.861 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.861 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3b is not active+clean 2023-10-08T00:10:58.861 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.861 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3a is not active+clean 2023-10-08T00:10:58.861 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.861 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.39 is not active+clean 2023-10-08T00:10:58.861 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.861 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.38 is not active+clean 2023-10-08T00:10:58.861 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.862 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.37 is not active+clean 2023-10-08T00:10:58.862 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.862 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.36 is not active+clean 2023-10-08T00:10:58.862 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.862 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.35 is not active+clean 2023-10-08T00:10:58.862 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.862 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.34 is not active+clean 2023-10-08T00:10:58.862 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.863 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.33 is not active+clean 2023-10-08T00:10:58.863 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.863 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.32 is not active+clean 2023-10-08T00:10:58.863 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.863 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.31 is not active+clean 2023-10-08T00:10:58.863 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.863 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.30 is not active+clean 2023-10-08T00:10:58.863 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.863 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2f is not active+clean 2023-10-08T00:10:58.863 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.864 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2e is not active+clean 2023-10-08T00:10:58.864 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.864 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2d is not active+clean 2023-10-08T00:10:58.864 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.864 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1f is not active+clean 2023-10-08T00:10:58.864 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:10:52.061775+0000', 'last_change': '2023-10-08T00:10:52.071916+0000', 'last_active': '2023-10-08T00:10:52.061775+0000', 'last_peered': '2023-10-08T00:10:52.061775+0000', 'last_clean': '2023-10-08T00:10:52.061775+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:52.061775+0000', 'last_undegraded': '2023-10-08T00:10:52.061775+0000', 'last_fullsized': '2023-10-08T00:10:52.061775+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T09:00:50.648234+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T00:10:58.864 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1e is not active+clean 2023-10-08T00:10:58.864 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.864 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1e is not active+clean 2023-10-08T00:10:58.864 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:10:52.061775+0000', 'last_change': '2023-10-08T00:10:52.068998+0000', 'last_active': '2023-10-08T00:10:52.061775+0000', 'last_peered': '2023-10-08T00:10:52.061775+0000', 'last_clean': '2023-10-08T00:10:52.061775+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:52.061775+0000', 'last_undegraded': '2023-10-08T00:10:52.061775+0000', 'last_fullsized': '2023-10-08T00:10:52.061775+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T00:48:47.624288+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T00:10:58.865 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1f is not active+clean 2023-10-08T00:10:58.865 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.865 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1d is not active+clean 2023-10-08T00:10:58.865 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:10:52.061775+0000', 'last_change': '2023-10-08T00:10:52.070061+0000', 'last_active': '2023-10-08T00:10:52.061775+0000', 'last_peered': '2023-10-08T00:10:52.061775+0000', 'last_clean': '2023-10-08T00:10:52.061775+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:52.061775+0000', 'last_undegraded': '2023-10-08T00:10:52.061775+0000', 'last_fullsized': '2023-10-08T00:10:52.061775+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T09:23:55.581278+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T00:10:58.865 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1c is not active+clean 2023-10-08T00:10:58.865 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.865 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1c is not active+clean 2023-10-08T00:10:58.865 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:10:52.061775+0000', 'last_change': '2023-10-08T00:10:52.070506+0000', 'last_active': '2023-10-08T00:10:52.061775+0000', 'last_peered': '2023-10-08T00:10:52.061775+0000', 'last_clean': '2023-10-08T00:10:52.061775+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:52.061775+0000', 'last_undegraded': '2023-10-08T00:10:52.061775+0000', 'last_fullsized': '2023-10-08T00:10:52.061775+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T03:16:40.636514+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T00:10:58.865 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1d is not active+clean 2023-10-08T00:10:58.866 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.866 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1b is not active+clean 2023-10-08T00:10:58.866 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:10:52.061775+0000', 'last_change': '2023-10-08T00:10:52.069315+0000', 'last_active': '2023-10-08T00:10:52.061775+0000', 'last_peered': '2023-10-08T00:10:52.061775+0000', 'last_clean': '2023-10-08T00:10:52.061775+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:52.061775+0000', 'last_undegraded': '2023-10-08T00:10:52.061775+0000', 'last_fullsized': '2023-10-08T00:10:52.061775+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T09:08:39.785742+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T00:10:58.866 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1a is not active+clean 2023-10-08T00:10:58.866 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.866 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1a is not active+clean 2023-10-08T00:10:58.866 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:10:52.061775+0000', 'last_change': '2023-10-08T00:10:52.072673+0000', 'last_active': '2023-10-08T00:10:52.061775+0000', 'last_peered': '2023-10-08T00:10:52.061775+0000', 'last_clean': '2023-10-08T00:10:52.061775+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:52.061775+0000', 'last_undegraded': '2023-10-08T00:10:52.061775+0000', 'last_fullsized': '2023-10-08T00:10:52.061775+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T03:16:40.636514+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T00:10:58.866 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1b is not active+clean 2023-10-08T00:10:58.867 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.867 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.19 is not active+clean 2023-10-08T00:10:58.867 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:10:52.061775+0000', 'last_change': '2023-10-08T00:10:52.071429+0000', 'last_active': '2023-10-08T00:10:52.061775+0000', 'last_peered': '2023-10-08T00:10:52.061775+0000', 'last_clean': '2023-10-08T00:10:52.061775+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:52.061775+0000', 'last_undegraded': '2023-10-08T00:10:52.061775+0000', 'last_fullsized': '2023-10-08T00:10:52.061775+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T09:08:39.785742+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T00:10:58.867 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.18 is not active+clean 2023-10-08T00:10:58.867 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.867 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.18 is not active+clean 2023-10-08T00:10:58.867 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:10:52.061775+0000', 'last_change': '2023-10-08T00:10:52.072393+0000', 'last_active': '2023-10-08T00:10:52.061775+0000', 'last_peered': '2023-10-08T00:10:52.061775+0000', 'last_clean': '2023-10-08T00:10:52.061775+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:52.061775+0000', 'last_undegraded': '2023-10-08T00:10:52.061775+0000', 'last_fullsized': '2023-10-08T00:10:52.061775+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T09:00:50.648234+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T00:10:58.867 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.19 is not active+clean 2023-10-08T00:10:58.867 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.868 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.17 is not active+clean 2023-10-08T00:10:58.868 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:10:52.061775+0000', 'last_change': '2023-10-08T00:10:52.070809+0000', 'last_active': '2023-10-08T00:10:52.061775+0000', 'last_peered': '2023-10-08T00:10:52.061775+0000', 'last_clean': '2023-10-08T00:10:52.061775+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:52.061775+0000', 'last_undegraded': '2023-10-08T00:10:52.061775+0000', 'last_fullsized': '2023-10-08T00:10:52.061775+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T00:48:47.624288+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T00:10:58.868 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.16 is not active+clean 2023-10-08T00:10:58.868 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.868 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.16 is not active+clean 2023-10-08T00:10:58.868 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:10:52.061775+0000', 'last_change': '2023-10-08T00:10:52.070483+0000', 'last_active': '2023-10-08T00:10:52.061775+0000', 'last_peered': '2023-10-08T00:10:52.061775+0000', 'last_clean': '2023-10-08T00:10:52.061775+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:52.061775+0000', 'last_undegraded': '2023-10-08T00:10:52.061775+0000', 'last_fullsized': '2023-10-08T00:10:52.061775+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T09:08:39.785742+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T00:10:58.868 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.17 is not active+clean 2023-10-08T00:10:58.868 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.868 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.15 is not active+clean 2023-10-08T00:10:58.868 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:10:52.061775+0000', 'last_change': '2023-10-08T00:10:52.069507+0000', 'last_active': '2023-10-08T00:10:52.061775+0000', 'last_peered': '2023-10-08T00:10:52.061775+0000', 'last_clean': '2023-10-08T00:10:52.061775+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:52.061775+0000', 'last_undegraded': '2023-10-08T00:10:52.061775+0000', 'last_fullsized': '2023-10-08T00:10:52.061775+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T09:00:50.648234+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T00:10:58.869 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.14 is not active+clean 2023-10-08T00:10:58.869 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.869 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.14 is not active+clean 2023-10-08T00:10:58.869 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:10:52.061775+0000', 'last_change': '2023-10-08T00:10:52.069553+0000', 'last_active': '2023-10-08T00:10:52.061775+0000', 'last_peered': '2023-10-08T00:10:52.061775+0000', 'last_clean': '2023-10-08T00:10:52.061775+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:52.061775+0000', 'last_undegraded': '2023-10-08T00:10:52.061775+0000', 'last_fullsized': '2023-10-08T00:10:52.061775+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T00:48:47.624288+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T00:10:58.869 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.15 is not active+clean 2023-10-08T00:10:58.869 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.869 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.13 is not active+clean 2023-10-08T00:10:58.869 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:10:52.061775+0000', 'last_change': '2023-10-08T00:10:52.069081+0000', 'last_active': '2023-10-08T00:10:52.061775+0000', 'last_peered': '2023-10-08T00:10:52.061775+0000', 'last_clean': '2023-10-08T00:10:52.061775+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:52.061775+0000', 'last_undegraded': '2023-10-08T00:10:52.061775+0000', 'last_fullsized': '2023-10-08T00:10:52.061775+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T00:48:47.624288+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T00:10:58.869 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.12 is not active+clean 2023-10-08T00:10:58.870 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.870 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.12 is not active+clean 2023-10-08T00:10:58.870 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:10:52.061775+0000', 'last_change': '2023-10-08T00:10:52.072884+0000', 'last_active': '2023-10-08T00:10:52.061775+0000', 'last_peered': '2023-10-08T00:10:52.061775+0000', 'last_clean': '2023-10-08T00:10:52.061775+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:52.061775+0000', 'last_undegraded': '2023-10-08T00:10:52.061775+0000', 'last_fullsized': '2023-10-08T00:10:52.061775+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T10:08:25.404396+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T00:10:58.870 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.13 is not active+clean 2023-10-08T00:10:58.870 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.870 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.11 is not active+clean 2023-10-08T00:10:58.870 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:10:52.061775+0000', 'last_change': '2023-10-08T00:10:52.070119+0000', 'last_active': '2023-10-08T00:10:52.061775+0000', 'last_peered': '2023-10-08T00:10:52.061775+0000', 'last_clean': '2023-10-08T00:10:52.061775+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:52.061775+0000', 'last_undegraded': '2023-10-08T00:10:52.061775+0000', 'last_fullsized': '2023-10-08T00:10:52.061775+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T10:48:58.606951+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T00:10:58.870 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.10 is not active+clean 2023-10-08T00:10:58.871 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.871 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.10 is not active+clean 2023-10-08T00:10:58.871 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:10:52.061775+0000', 'last_change': '2023-10-08T00:10:52.071176+0000', 'last_active': '2023-10-08T00:10:52.061775+0000', 'last_peered': '2023-10-08T00:10:52.061775+0000', 'last_clean': '2023-10-08T00:10:52.061775+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:52.061775+0000', 'last_undegraded': '2023-10-08T00:10:52.061775+0000', 'last_fullsized': '2023-10-08T00:10:52.061775+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T09:23:55.581278+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T00:10:58.871 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.11 is not active+clean 2023-10-08T00:10:58.871 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.871 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.f is not active+clean 2023-10-08T00:10:58.871 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:10:52.061775+0000', 'last_change': '2023-10-08T00:10:52.070171+0000', 'last_active': '2023-10-08T00:10:52.061775+0000', 'last_peered': '2023-10-08T00:10:52.061775+0000', 'last_clean': '2023-10-08T00:10:52.061775+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:52.061775+0000', 'last_undegraded': '2023-10-08T00:10:52.061775+0000', 'last_fullsized': '2023-10-08T00:10:52.061775+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T09:00:50.648234+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T00:10:58.871 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.e is not active+clean 2023-10-08T00:10:58.871 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.872 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.4 is not active+clean 2023-10-08T00:10:58.872 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:10:52.061775+0000', 'last_change': '2023-10-08T00:10:52.069019+0000', 'last_active': '2023-10-08T00:10:52.061775+0000', 'last_peered': '2023-10-08T00:10:52.061775+0000', 'last_clean': '2023-10-08T00:10:52.061775+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:52.061775+0000', 'last_undegraded': '2023-10-08T00:10:52.061775+0000', 'last_fullsized': '2023-10-08T00:10:52.061775+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T09:00:50.648234+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T00:10:58.872 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.5 is not active+clean 2023-10-08T00:10:58.872 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.872 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.2 is not active+clean 2023-10-08T00:10:58.872 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:10:52.061775+0000', 'last_change': '2023-10-08T00:10:52.072572+0000', 'last_active': '2023-10-08T00:10:52.061775+0000', 'last_peered': '2023-10-08T00:10:52.061775+0000', 'last_clean': '2023-10-08T00:10:52.061775+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:52.061775+0000', 'last_undegraded': '2023-10-08T00:10:52.061775+0000', 'last_fullsized': '2023-10-08T00:10:52.061775+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T09:08:39.785742+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T00:10:58.872 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3 is not active+clean 2023-10-08T00:10:58.872 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.872 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.0 is not active+clean 2023-10-08T00:10:58.872 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.873 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1 is not active+clean 2023-10-08T00:10:58.873 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:10:52.061775+0000', 'last_change': '2023-10-08T00:10:52.069029+0000', 'last_active': '2023-10-08T00:10:52.061775+0000', 'last_peered': '2023-10-08T00:10:52.061775+0000', 'last_clean': '2023-10-08T00:10:52.061775+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:52.061775+0000', 'last_undegraded': '2023-10-08T00:10:52.061775+0000', 'last_fullsized': '2023-10-08T00:10:52.061775+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T10:48:58.606951+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T00:10:58.873 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2 is not active+clean 2023-10-08T00:10:58.873 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.873 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.3 is not active+clean 2023-10-08T00:10:58.873 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:10:52.061775+0000', 'last_change': '2023-10-08T00:10:52.072150+0000', 'last_active': '2023-10-08T00:10:52.061775+0000', 'last_peered': '2023-10-08T00:10:52.061775+0000', 'last_clean': '2023-10-08T00:10:52.061775+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:52.061775+0000', 'last_undegraded': '2023-10-08T00:10:52.061775+0000', 'last_fullsized': '2023-10-08T00:10:52.061775+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T00:48:47.624288+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T00:10:58.873 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1 is not active+clean 2023-10-08T00:10:58.873 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.873 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.0 is not active+clean 2023-10-08T00:10:58.874 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:10:52.061775+0000', 'last_change': '2023-10-08T00:10:52.072171+0000', 'last_active': '2023-10-08T00:10:52.061775+0000', 'last_peered': '2023-10-08T00:10:52.061775+0000', 'last_clean': '2023-10-08T00:10:52.061775+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:52.061775+0000', 'last_undegraded': '2023-10-08T00:10:52.061775+0000', 'last_fullsized': '2023-10-08T00:10:52.061775+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T09:23:55.581278+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T00:10:58.874 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.5 is not active+clean 2023-10-08T00:10:58.874 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:10:52.061775+0000', 'last_change': '2023-10-08T00:10:52.073149+0000', 'last_active': '2023-10-08T00:10:52.061775+0000', 'last_peered': '2023-10-08T00:10:52.061775+0000', 'last_clean': '2023-10-08T00:10:52.061775+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:52.061775+0000', 'last_undegraded': '2023-10-08T00:10:52.061775+0000', 'last_fullsized': '2023-10-08T00:10:52.061775+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T07:10:42.328030+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T00:10:58.874 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.4 is not active+clean 2023-10-08T00:10:58.874 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.874 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.6 is not active+clean 2023-10-08T00:10:58.874 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:10:52.061775+0000', 'last_change': '2023-10-08T00:10:52.071633+0000', 'last_active': '2023-10-08T00:10:52.061775+0000', 'last_peered': '2023-10-08T00:10:52.061775+0000', 'last_clean': '2023-10-08T00:10:52.061775+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:52.061775+0000', 'last_undegraded': '2023-10-08T00:10:52.061775+0000', 'last_fullsized': '2023-10-08T00:10:52.061775+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T00:48:47.624288+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T00:10:58.874 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.7 is not active+clean 2023-10-08T00:10:58.874 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.875 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.7 is not active+clean 2023-10-08T00:10:58.875 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:10:52.061775+0000', 'last_change': '2023-10-08T00:10:52.072117+0000', 'last_active': '2023-10-08T00:10:52.061775+0000', 'last_peered': '2023-10-08T00:10:52.061775+0000', 'last_clean': '2023-10-08T00:10:52.061775+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:52.061775+0000', 'last_undegraded': '2023-10-08T00:10:52.061775+0000', 'last_fullsized': '2023-10-08T00:10:52.061775+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T10:48:58.606951+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T00:10:58.875 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.6 is not active+clean 2023-10-08T00:10:58.875 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.875 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.8 is not active+clean 2023-10-08T00:10:58.875 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:10:52.061775+0000', 'last_change': '2023-10-08T00:10:52.070144+0000', 'last_active': '2023-10-08T00:10:52.061775+0000', 'last_peered': '2023-10-08T00:10:52.061775+0000', 'last_clean': '2023-10-08T00:10:52.061775+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:52.061775+0000', 'last_undegraded': '2023-10-08T00:10:52.061775+0000', 'last_fullsized': '2023-10-08T00:10:52.061775+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T00:48:47.624288+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T00:10:58.875 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.9 is not active+clean 2023-10-08T00:10:58.875 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.875 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.9 is not active+clean 2023-10-08T00:10:58.875 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:10:52.061775+0000', 'last_change': '2023-10-08T00:10:52.070658+0000', 'last_active': '2023-10-08T00:10:52.061775+0000', 'last_peered': '2023-10-08T00:10:52.061775+0000', 'last_clean': '2023-10-08T00:10:52.061775+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:52.061775+0000', 'last_undegraded': '2023-10-08T00:10:52.061775+0000', 'last_fullsized': '2023-10-08T00:10:52.061775+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T11:16:07.345714+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T00:10:58.876 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.8 is not active+clean 2023-10-08T00:10:58.876 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.876 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.a is not active+clean 2023-10-08T00:10:58.876 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:10:52.061775+0000', 'last_change': '2023-10-08T00:10:52.071086+0000', 'last_active': '2023-10-08T00:10:52.061775+0000', 'last_peered': '2023-10-08T00:10:52.061775+0000', 'last_clean': '2023-10-08T00:10:52.061775+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:52.061775+0000', 'last_undegraded': '2023-10-08T00:10:52.061775+0000', 'last_fullsized': '2023-10-08T00:10:52.061775+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T09:00:50.648234+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T00:10:58.876 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.b is not active+clean 2023-10-08T00:10:58.876 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.876 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.b is not active+clean 2023-10-08T00:10:58.876 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:10:52.061775+0000', 'last_change': '2023-10-08T00:10:52.072116+0000', 'last_active': '2023-10-08T00:10:52.061775+0000', 'last_peered': '2023-10-08T00:10:52.061775+0000', 'last_clean': '2023-10-08T00:10:52.061775+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:52.061775+0000', 'last_undegraded': '2023-10-08T00:10:52.061775+0000', 'last_fullsized': '2023-10-08T00:10:52.061775+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T11:16:07.345714+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T00:10:58.877 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.a is not active+clean 2023-10-08T00:10:58.877 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.877 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.c is not active+clean 2023-10-08T00:10:58.877 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:10:52.061775+0000', 'last_change': '2023-10-08T00:10:52.070888+0000', 'last_active': '2023-10-08T00:10:52.061775+0000', 'last_peered': '2023-10-08T00:10:52.061775+0000', 'last_clean': '2023-10-08T00:10:52.061775+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:52.061775+0000', 'last_undegraded': '2023-10-08T00:10:52.061775+0000', 'last_fullsized': '2023-10-08T00:10:52.061775+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T00:48:47.624288+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T00:10:58.877 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.d is not active+clean 2023-10-08T00:10:58.877 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.877 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.d is not active+clean 2023-10-08T00:10:58.877 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:10:52.061775+0000', 'last_change': '2023-10-08T00:10:52.069442+0000', 'last_active': '2023-10-08T00:10:52.061775+0000', 'last_peered': '2023-10-08T00:10:52.061775+0000', 'last_clean': '2023-10-08T00:10:52.061775+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:52.061775+0000', 'last_undegraded': '2023-10-08T00:10:52.061775+0000', 'last_fullsized': '2023-10-08T00:10:52.061775+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T03:16:40.636514+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T00:10:58.877 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.c is not active+clean 2023-10-08T00:10:58.877 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.878 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.e is not active+clean 2023-10-08T00:10:58.878 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:10:52.061775+0000', 'last_change': '2023-10-08T00:10:52.072128+0000', 'last_active': '2023-10-08T00:10:52.061775+0000', 'last_peered': '2023-10-08T00:10:52.061775+0000', 'last_clean': '2023-10-08T00:10:52.061775+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:52.061775+0000', 'last_undegraded': '2023-10-08T00:10:52.061775+0000', 'last_fullsized': '2023-10-08T00:10:52.061775+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:52.061775+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T10:48:58.606951+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T00:10:58.878 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.f is not active+clean 2023-10-08T00:10:58.878 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.878 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.20 is not active+clean 2023-10-08T00:10:58.878 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.878 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.21 is not active+clean 2023-10-08T00:10:58.878 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.879 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.22 is not active+clean 2023-10-08T00:10:58.879 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.879 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.23 is not active+clean 2023-10-08T00:10:58.879 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.879 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.24 is not active+clean 2023-10-08T00:10:58.879 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.879 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.25 is not active+clean 2023-10-08T00:10:58.879 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.879 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.26 is not active+clean 2023-10-08T00:10:58.879 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.880 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.27 is not active+clean 2023-10-08T00:10:58.880 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.880 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.28 is not active+clean 2023-10-08T00:10:58.880 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.880 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.29 is not active+clean 2023-10-08T00:10:58.880 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.880 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2a is not active+clean 2023-10-08T00:10:58.880 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.880 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2b is not active+clean 2023-10-08T00:10:58.881 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.881 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2c is not active+clean 2023-10-08T00:10:58.881 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T00:10:53.067272+0000', 'last_change': '2023-10-08T00:10:53.067272+0000', 'last_active': '2023-10-08T00:10:53.067272+0000', 'last_peered': '2023-10-08T00:10:53.067272+0000', 'last_clean': '2023-10-08T00:10:53.067272+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:10:53.067272+0000', 'last_undegraded': '2023-10-08T00:10:53.067272+0000', 'last_fullsized': '2023-10-08T00:10:53.067272+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'last_clean_scrub_stamp': '2023-10-08T00:10:53.067272+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T00:10:58.881 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-08T00:10:59.004 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.003+0000 7f0794480700 1 Processor -- start 2023-10-08T00:10:59.004 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.003+0000 7f0794480700 1 -- start start 2023-10-08T00:10:59.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.004+0000 7f0794480700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f078c15a620 0x7f078c15d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:59.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.004+0000 7f0794480700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f078c15d580 0x7f078c158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:59.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.004+0000 7f0794480700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f078c15cbb0 con 0x7f078c157870 2023-10-08T00:10:59.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.004+0000 7f0794480700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f078c15bdb0 con 0x7f078c15d580 2023-10-08T00:10:59.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.004+0000 7f0794480700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f078c15bfa0 con 0x7f078c15a620 2023-10-08T00:10:59.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.004+0000 7f0792a1d700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f078c15d580 0x7f078c158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:59.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.004+0000 7f0792a1d700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f078c15d580 0x7f078c158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49530/0 (socket says 172.21.15.99:49530) 2023-10-08T00:10:59.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.004+0000 7f0792a1d700 1 -- 172.21.15.99:0/3765407178 learned_addr learned my addr 172.21.15.99:0/3765407178 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:10:59.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.004+0000 7f0791a1b700 1 --2- 172.21.15.99:0/3765407178 >> v2:172.21.15.191:3300/0 conn(0x7f078c15a620 0x7f078c15d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:59.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.004+0000 7f0792a1d700 1 -- 172.21.15.99:0/3765407178 >> v2:172.21.15.191:3300/0 conn(0x7f078c15a620 msgr2=0x7f078c15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:59.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.004+0000 7f0792a1d700 1 --2- 172.21.15.99:0/3765407178 >> v2:172.21.15.191:3300/0 conn(0x7f078c15a620 0x7f078c15d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:59.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.004+0000 7f0792a1d700 1 -- 172.21.15.99:0/3765407178 >> v1:172.21.15.191:6789/0 conn(0x7f078c157870 legacy=0x7f078c15b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:59.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.004+0000 7f0792a1d700 1 -- 172.21.15.99:0/3765407178 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f078c15c280 con 0x7f078c15d580 2023-10-08T00:10:59.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.004+0000 7f0791a1b700 1 --2- 172.21.15.99:0/3765407178 >> v2:172.21.15.191:3300/0 conn(0x7f078c15a620 0x7f078c15d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:10:59.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.005+0000 7f0792a1d700 1 --2- 172.21.15.99:0/3765407178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f078c15d580 0x7f078c158110 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f078c15cb20 tx=0x7f0784009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e67f328bb7b082a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:59.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.005+0000 7f079121a700 1 -- 172.21.15.99:0/3765407178 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078400b820 con 0x7f078c15d580 2023-10-08T00:10:59.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.005+0000 7f079121a700 1 -- 172.21.15.99:0/3765407178 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f078400b980 con 0x7f078c15d580 2023-10-08T00:10:59.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.005+0000 7f079121a700 1 -- 172.21.15.99:0/3765407178 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078400bd20 con 0x7f078c15d580 2023-10-08T00:10:59.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.005+0000 7f0794480700 1 -- 172.21.15.99:0/3765407178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f078c15d580 msgr2=0x7f078c158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:59.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.005+0000 7f0794480700 1 --2- 172.21.15.99:0/3765407178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f078c15d580 0x7f078c158110 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f078c15cb20 tx=0x7f0784009250 comp rx=0 tx=0).stop 2023-10-08T00:10:59.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.005+0000 7f0794480700 1 -- 172.21.15.99:0/3765407178 shutdown_connections 2023-10-08T00:10:59.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.005+0000 7f0794480700 1 --2- 172.21.15.99:0/3765407178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f078c15d580 0x7f078c158110 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:59.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.005+0000 7f0794480700 1 --2- 172.21.15.99:0/3765407178 >> v2:172.21.15.191:3300/0 conn(0x7f078c15a620 0x7f078c15d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:59.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.005+0000 7f0794480700 1 -- 172.21.15.99:0/3765407178 >> 172.21.15.99:0/3765407178 conn(0x7f078c068ce0 msgr2=0x7f078c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:59.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.005+0000 7f0794480700 1 -- 172.21.15.99:0/3765407178 shutdown_connections 2023-10-08T00:10:59.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.005+0000 7f0794480700 1 -- 172.21.15.99:0/3765407178 wait complete. 2023-10-08T00:10:59.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.006+0000 7f0794480700 1 Processor -- start 2023-10-08T00:10:59.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.006+0000 7f0794480700 1 -- start start 2023-10-08T00:10:59.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.007+0000 7f0794480700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f078c157870 0x7f078c14b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:59.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.007+0000 7f0794480700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f078c15a620 0x7f078c0d01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:59.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.007+0000 7f0794480700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f078c15d580 0x7f078c0d06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:59.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.007+0000 7f0794480700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078c0b0760 con 0x7f078c15d580 2023-10-08T00:10:59.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.007+0000 7f0794480700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078c0b0a40 con 0x7f078c157870 2023-10-08T00:10:59.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.007+0000 7f0794480700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f078c0b0d20 con 0x7f078c15a620 2023-10-08T00:10:59.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.007+0000 7f0792a1d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f078c15d580 0x7f078c0d06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:59.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.007+0000 7f0792a1d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f078c15d580 0x7f078c0d06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49540/0 (socket says 172.21.15.99:49540) 2023-10-08T00:10:59.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.007+0000 7f0792a1d700 1 -- 172.21.15.99:0/1275822176 learned_addr learned my addr 172.21.15.99:0/1275822176 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:10:59.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.007+0000 7f079221c700 1 --2- 172.21.15.99:0/1275822176 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f078c157870 0x7f078c14b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:59.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.007+0000 7f0791a1b700 1 --2- 172.21.15.99:0/1275822176 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f078c15a620 0x7f078c0d01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:59.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.007+0000 7f0792a1d700 1 -- 172.21.15.99:0/1275822176 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f078c15a620 msgr2=0x7f078c0d01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:59.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.007+0000 7f0792a1d700 1 --2- 172.21.15.99:0/1275822176 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f078c15a620 0x7f078c0d01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:59.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.007+0000 7f0792a1d700 1 -- 172.21.15.99:0/1275822176 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f078c157870 msgr2=0x7f078c14b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:59.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.007+0000 7f0792a1d700 1 --2- 172.21.15.99:0/1275822176 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f078c157870 0x7f078c14b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:59.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.007+0000 7f0792a1d700 1 -- 172.21.15.99:0/1275822176 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07840094f0 con 0x7f078c15d580 2023-10-08T00:10:59.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.008+0000 7f0792a1d700 1 --2- 172.21.15.99:0/1275822176 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f078c15d580 0x7f078c0d06e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0784009290 tx=0x7f0784016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:59.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.008+0000 7f07827fc700 1 -- 172.21.15.99:0/1275822176 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078401b680 con 0x7f078c15d580 2023-10-08T00:10:59.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.008+0000 7f07827fc700 1 -- 172.21.15.99:0/1275822176 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f078401b7e0 con 0x7f078c15d580 2023-10-08T00:10:59.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.008+0000 7f0794480700 1 -- 172.21.15.99:0/1275822176 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f078c0b1000 con 0x7f078c15d580 2023-10-08T00:10:59.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.008+0000 7f07827fc700 1 -- 172.21.15.99:0/1275822176 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078401bba0 con 0x7f078c15d580 2023-10-08T00:10:59.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.008+0000 7f0794480700 1 -- 172.21.15.99:0/1275822176 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f078c14bac0 con 0x7f078c15d580 2023-10-08T00:10:59.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.009+0000 7f0794480700 1 -- 172.21.15.99:0/1275822176 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f078c0b08a0 con 0x7f078c15d580 2023-10-08T00:10:59.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.009+0000 7f07827fc700 1 -- 172.21.15.99:0/1275822176 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f078401c450 con 0x7f078c15d580 2023-10-08T00:10:59.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.010+0000 7f07827fc700 1 --2- 172.21.15.99:0/1275822176 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0770066f30 0x7f07700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:59.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.010+0000 7f07827fc700 1 -- 172.21.15.99:0/1275822176 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0784086280 con 0x7f078c15d580 2023-10-08T00:10:59.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.010+0000 7f079221c700 1 --2- 172.21.15.99:0/1275822176 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0770066f30 0x7f07700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:59.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.010+0000 7f079221c700 1 --2- 172.21.15.99:0/1275822176 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0770066f30 0x7f07700693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0788007fd0 tx=0x7f07880026a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:59.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.014+0000 7f07827fc700 1 -- 172.21.15.99:0/1275822176 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f078c0b08a0 con 0x7f078c15d580 2023-10-08T00:10:59.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.272+0000 7f0794480700 1 -- 172.21.15.99:0/1275822176 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f078c0b08a0 con 0x7f078c15d580 2023-10-08T00:10:59.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.273+0000 7f07827fc700 1 -- 172.21.15.99:0/1275822176 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1102 (secure 0 0 0) 0x7f078402a460 con 0x7f078c15d580 2023-10-08T00:10:59.274 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:10:59.274 INFO:teuthology.orchestra.run.smithi099.stdout:{"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":22,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":20,"num_osds":8,"num_up_osds":8,"osd_up_since":1696723841,"num_in_osds":8,"osd_in_since":1696723838,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":1590176,"bytes_avail":1598409824,"bytes_total":1600000000,"unknown_pgs_ratio":0.65979379415512085,"inactive_pgs_ratio":0.32989689707756042},"fsmap":{"epoch":8,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"d","status":"up:active","gid":4282}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2023-10-08T00:10:36.304677+0000","services":{}},"progress_events":{}} 2023-10-08T00:10:59.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.275+0000 7f0794480700 1 -- 172.21.15.99:0/1275822176 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0770066f30 msgr2=0x7f07700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:59.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.275+0000 7f0794480700 1 --2- 172.21.15.99:0/1275822176 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0770066f30 0x7f07700693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0788007fd0 tx=0x7f07880026a0 comp rx=0 tx=0).stop 2023-10-08T00:10:59.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.275+0000 7f0794480700 1 -- 172.21.15.99:0/1275822176 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f078c15d580 msgr2=0x7f078c0d06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:59.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.275+0000 7f0794480700 1 --2- 172.21.15.99:0/1275822176 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f078c15d580 0x7f078c0d06e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0784009290 tx=0x7f0784016000 comp rx=0 tx=0).stop 2023-10-08T00:10:59.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.275+0000 7f0794480700 1 -- 172.21.15.99:0/1275822176 shutdown_connections 2023-10-08T00:10:59.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.275+0000 7f0794480700 1 --2- 172.21.15.99:0/1275822176 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0770066f30 0x7f07700693e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:59.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.275+0000 7f0794480700 1 --2- 172.21.15.99:0/1275822176 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f078c157870 0x7f078c14b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:59.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.275+0000 7f0794480700 1 --2- 172.21.15.99:0/1275822176 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f078c15d580 0x7f078c0d06e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:59.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.275+0000 7f0794480700 1 --2- 172.21.15.99:0/1275822176 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f078c15a620 0x7f078c0d01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:59.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.275+0000 7f0794480700 1 -- 172.21.15.99:0/1275822176 >> 172.21.15.99:0/1275822176 conn(0x7f078c068ce0 msgr2=0x7f078c0b5410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:59.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.275+0000 7f0794480700 1 -- 172.21.15.99:0/1275822176 shutdown_connections 2023-10-08T00:10:59.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.275+0000 7f0794480700 1 -- 172.21.15.99:0/1275822176 wait complete. 2023-10-08T00:10:59.288 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-08T00:10:59.288 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T00:10:59.402 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.402+0000 7f72e28be700 1 Processor -- start 2023-10-08T00:10:59.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.402+0000 7f72e28be700 1 -- start start 2023-10-08T00:10:59.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.402+0000 7f72e28be700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72dc0a7b20 0x7f72dc0b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:59.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.402+0000 7f72e28be700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f72dc06f1c0 0x7f72dc099530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:59.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.402+0000 7f72e28be700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72dc0990a0 con 0x7f72dc063cf0 2023-10-08T00:10:59.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.402+0000 7f72e28be700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72dc064120 con 0x7f72dc06f1c0 2023-10-08T00:10:59.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.402+0000 7f72e28be700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc0984b0 con 0x7f72dc0a7b20 2023-10-08T00:10:59.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.403+0000 7f72db7fe700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f72dc063cf0 0x7f72dc06eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50770/0 (socket says 172.21.15.99:50770) 2023-10-08T00:10:59.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.403+0000 7f72db7fe700 1 -- 172.21.15.99:0/1209065154 learned_addr learned my addr 172.21.15.99:0/1209065154 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:10:59.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.403+0000 7f72e0e5b700 1 --2- 172.21.15.99:0/1209065154 >> v2:172.21.15.191:3300/0 conn(0x7f72dc06f1c0 0x7f72dc099530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:59.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.403+0000 7f72dbfff700 1 --2- 172.21.15.99:0/1209065154 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72dc0a7b20 0x7f72dc0b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:59.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.403+0000 7f72daffd700 1 -- 172.21.15.99:0/1209065154 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2954372911 0 0) 0x7f72dc0990a0 con 0x7f72dc063cf0 2023-10-08T00:10:59.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.403+0000 7f72daffd700 1 -- 172.21.15.99:0/1209065154 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72c4003570 con 0x7f72dc063cf0 2023-10-08T00:10:59.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.403+0000 7f72daffd700 1 -- 172.21.15.99:0/1209065154 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1355221222 0 0) 0x7f72c4003570 con 0x7f72dc063cf0 2023-10-08T00:10:59.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.403+0000 7f72daffd700 1 -- 172.21.15.99:0/1209065154 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72dc0a7b20 msgr2=0x7f72dc0b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:59.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.403+0000 7f72daffd700 1 --2- 172.21.15.99:0/1209065154 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72dc0a7b20 0x7f72dc0b2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:59.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.403+0000 7f72daffd700 1 -- 172.21.15.99:0/1209065154 >> v2:172.21.15.191:3300/0 conn(0x7f72dc06f1c0 msgr2=0x7f72dc099530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:59.412 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.403+0000 7f72daffd700 1 --2- 172.21.15.99:0/1209065154 >> v2:172.21.15.191:3300/0 conn(0x7f72dc06f1c0 0x7f72dc099530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:59.412 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.403+0000 7f72daffd700 1 -- 172.21.15.99:0/1209065154 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72dc098770 con 0x7f72dc063cf0 2023-10-08T00:10:59.412 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.403+0000 7f72daffd700 1 -- 172.21.15.99:0/1209065154 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f72d0001470 con 0x7f72dc063cf0 2023-10-08T00:10:59.412 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.403+0000 7f72e0e5b700 1 --2- 172.21.15.99:0/1209065154 >> v2:172.21.15.191:3300/0 conn(0x7f72dc06f1c0 0x7f72dc099530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:10:59.412 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.403+0000 7f72daffd700 1 -- 172.21.15.99:0/1209065154 >> v1:172.21.15.99:6789/0 conn(0x7f72dc063cf0 legacy=0x7f72dc06eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:59.413 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.403+0000 7f72daffd700 1 --2- 172.21.15.99:0/1209065154 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72dc0a7b20 0x7f72c4003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:59.413 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.403+0000 7f72daffd700 1 --2- 172.21.15.99:0/1209065154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72c4003fe0 0x7f72c4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:59.413 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.403+0000 7f72daffd700 1 --2- 172.21.15.99:0/1209065154 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72c4004d50 0x7f72c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:59.413 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.403+0000 7f72daffd700 1 -- 172.21.15.99:0/1209065154 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc064120 con 0x7f72c4003fe0 2023-10-08T00:10:59.413 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.403+0000 7f72daffd700 1 -- 172.21.15.99:0/1209065154 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc0984b0 con 0x7f72c4004d50 2023-10-08T00:10:59.413 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.403+0000 7f72daffd700 1 -- 172.21.15.99:0/1209065154 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72c400ba90 con 0x7f72dc0a7b20 2023-10-08T00:10:59.413 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.403+0000 7f72db7fe700 1 --2- 172.21.15.99:0/1209065154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72c4003fe0 0x7f72c4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:59.414 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.403+0000 7f72dbfff700 1 --2- 172.21.15.99:0/1209065154 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72dc0a7b20 0x7f72c4003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:59.414 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.403+0000 7f72e0e5b700 1 --2- 172.21.15.99:0/1209065154 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72c4004d50 0x7f72c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:59.414 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.403+0000 7f72db7fe700 1 -- 172.21.15.99:0/1209065154 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72dc0a7b20 msgr2=0x7f72c4003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:59.414 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.403+0000 7f72db7fe700 1 --2- 172.21.15.99:0/1209065154 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72dc0a7b20 0x7f72c4003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:59.414 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.404+0000 7f72db7fe700 1 -- 172.21.15.99:0/1209065154 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72c4004d50 msgr2=0x7f72c40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:10:59.414 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.404+0000 7f72db7fe700 1 --2- 172.21.15.99:0/1209065154 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72c4004d50 0x7f72c40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:59.414 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.404+0000 7f72db7fe700 1 -- 172.21.15.99:0/1209065154 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c4003570 con 0x7f72c4003fe0 2023-10-08T00:10:59.414 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.404+0000 7f72db7fe700 1 --2- 172.21.15.99:0/1209065154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72c4003fe0 0x7f72c4004810 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f72d000c3c0 tx=0x7f72d0001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b50162e78805d446 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:59.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.404+0000 7f72daffd700 1 -- 172.21.15.99:0/1209065154 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d000cc30 con 0x7f72c4003fe0 2023-10-08T00:10:59.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.404+0000 7f72daffd700 1 -- 172.21.15.99:0/1209065154 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72d000cd90 con 0x7f72c4003fe0 2023-10-08T00:10:59.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.404+0000 7f72daffd700 1 -- 172.21.15.99:0/1209065154 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d0005b70 con 0x7f72c4003fe0 2023-10-08T00:10:59.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.404+0000 7f72e28be700 1 -- 172.21.15.99:0/1209065154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72c4003fe0 msgr2=0x7f72c4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:59.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.404+0000 7f72e28be700 1 --2- 172.21.15.99:0/1209065154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72c4003fe0 0x7f72c4004810 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f72d000c3c0 tx=0x7f72d0001e90 comp rx=0 tx=0).stop 2023-10-08T00:10:59.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.404+0000 7f72e0e5b700 1 -- 172.21.15.99:0/1209065154 reap_dead start 2023-10-08T00:10:59.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.404+0000 7f72e28be700 1 -- 172.21.15.99:0/1209065154 shutdown_connections 2023-10-08T00:10:59.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.404+0000 7f72e28be700 1 -- 172.21.15.99:0/1209065154 >> 172.21.15.99:0/1209065154 conn(0x7f72dc0cd1e0 msgr2=0x7f72dc0cd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:59.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.404+0000 7f72e28be700 1 -- 172.21.15.99:0/1209065154 shutdown_connections 2023-10-08T00:10:59.416 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.404+0000 7f72e28be700 1 -- 172.21.15.99:0/1209065154 wait complete. 2023-10-08T00:10:59.416 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.405+0000 7f72e28be700 1 Processor -- start 2023-10-08T00:10:59.416 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.406+0000 7f72e28be700 1 -- start start 2023-10-08T00:10:59.416 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.406+0000 7f72e28be700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72c4003fe0 0x7f72dc15d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:59.416 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.406+0000 7f72e28be700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72dc157470 0x7f72dc0c2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:59.416 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.406+0000 7f72e28be700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72dc0aeab0 0x7f72dc0aef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:59.416 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.406+0000 7f72e28be700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc0af490 con 0x7f72c4003fe0 2023-10-08T00:10:59.416 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.406+0000 7f72e28be700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc15d760 con 0x7f72dc0aeab0 2023-10-08T00:10:59.416 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.406+0000 7f72e28be700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc0cc2e0 con 0x7f72dc157470 2023-10-08T00:10:59.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.406+0000 7f72dbfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72c4003fe0 0x7f72dc15d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:59.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.406+0000 7f72dbfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72c4003fe0 0x7f72dc15d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49554/0 (socket says 172.21.15.99:49554) 2023-10-08T00:10:59.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.406+0000 7f72dbfff700 1 -- 172.21.15.99:0/3718897997 learned_addr learned my addr 172.21.15.99:0/3718897997 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:10:59.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.407+0000 7f72e0e5b700 1 --2- 172.21.15.99:0/3718897997 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72dc0aeab0 0x7f72dc0aef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:59.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.407+0000 7f72dbfff700 1 -- 172.21.15.99:0/3718897997 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72dc157470 msgr2=0x7f72dc0c2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:59.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.407+0000 7f72dbfff700 1 --2- 172.21.15.99:0/3718897997 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72dc157470 0x7f72dc0c2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:59.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.407+0000 7f72dbfff700 1 -- 172.21.15.99:0/3718897997 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72dc0aeab0 msgr2=0x7f72dc0aef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:59.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.407+0000 7f72dbfff700 1 --2- 172.21.15.99:0/3718897997 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72dc0aeab0 0x7f72dc0aef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:59.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.407+0000 7f72dbfff700 1 -- 172.21.15.99:0/3718897997 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72dc15df30 con 0x7f72c4003fe0 2023-10-08T00:10:59.418 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.407+0000 7f72e0e5b700 1 --2- 172.21.15.99:0/3718897997 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72dc0aeab0 0x7f72dc0aef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:10:59.418 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.407+0000 7f72dbfff700 1 --2- 172.21.15.99:0/3718897997 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72c4003fe0 0x7f72dc15d9f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f72cc011da0 tx=0x7f72cc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:59.418 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.407+0000 7f72c3fff700 1 -- 172.21.15.99:0/3718897997 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72cc009e20 con 0x7f72c4003fe0 2023-10-08T00:10:59.418 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.407+0000 7f72c3fff700 1 -- 172.21.15.99:0/3718897997 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72cc00fcd0 con 0x7f72c4003fe0 2023-10-08T00:10:59.418 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.407+0000 7f72e28be700 1 -- 172.21.15.99:0/3718897997 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72dc0cbd60 con 0x7f72c4003fe0 2023-10-08T00:10:59.418 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.407+0000 7f72c3fff700 1 -- 172.21.15.99:0/3718897997 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72cc014650 con 0x7f72c4003fe0 2023-10-08T00:10:59.418 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.407+0000 7f72e28be700 1 -- 172.21.15.99:0/3718897997 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72dc0cc000 con 0x7f72c4003fe0 2023-10-08T00:10:59.418 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.408+0000 7f72e28be700 1 -- 172.21.15.99:0/3718897997 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72dc04c3c0 con 0x7f72c4003fe0 2023-10-08T00:10:59.418 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.408+0000 7f72c3fff700 1 -- 172.21.15.99:0/3718897997 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f72cc014a80 con 0x7f72c4003fe0 2023-10-08T00:10:59.419 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.409+0000 7f72c3fff700 1 --2- 172.21.15.99:0/3718897997 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f72c8066f30 0x7f72c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:10:59.419 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.409+0000 7f72c3fff700 1 -- 172.21.15.99:0/3718897997 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f72cc018030 con 0x7f72c4003fe0 2023-10-08T00:10:59.419 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.409+0000 7f72db7fe700 1 --2- 172.21.15.99:0/3718897997 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f72c8066f30 0x7f72c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:10:59.419 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.409+0000 7f72db7fe700 1 --2- 172.21.15.99:0/3718897997 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f72c8066f30 0x7f72c80693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f72dc159cb0 tx=0x7f72d0015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:10:59.420 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.413+0000 7f72c3fff700 1 -- 172.21.15.99:0/3718897997 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f72cc052160 con 0x7f72c4003fe0 2023-10-08T00:10:59.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.588+0000 7f72e28be700 1 -- 172.21.15.99:0/3718897997 --> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f72dc15b580 con 0x7f72c8066f30 2023-10-08T00:10:59.596 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.596+0000 7f72c3fff700 1 -- 172.21.15.99:0/3718897997 <== mgr.4099 v2:172.21.15.191:6832/85985 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251329 (secure 0 0 0) 0x7f72dc15b580 con 0x7f72c8066f30 2023-10-08T00:10:59.600 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:10:59.602 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.599+0000 7f72e28be700 1 -- 172.21.15.99:0/3718897997 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f72c8066f30 msgr2=0x7f72c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:59.602 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.599+0000 7f72e28be700 1 --2- 172.21.15.99:0/3718897997 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f72c8066f30 0x7f72c80693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f72dc159cb0 tx=0x7f72d0015000 comp rx=0 tx=0).stop 2023-10-08T00:10:59.602 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.599+0000 7f72e28be700 1 -- 172.21.15.99:0/3718897997 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72c4003fe0 msgr2=0x7f72dc15d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:10:59.602 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.599+0000 7f72e28be700 1 --2- 172.21.15.99:0/3718897997 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72c4003fe0 0x7f72dc15d9f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f72cc011da0 tx=0x7f72cc0091a0 comp rx=0 tx=0).stop 2023-10-08T00:10:59.602 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.599+0000 7f72e28be700 1 -- 172.21.15.99:0/3718897997 shutdown_connections 2023-10-08T00:10:59.603 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.599+0000 7f72e28be700 1 --2- 172.21.15.99:0/3718897997 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f72c8066f30 0x7f72c80693e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:59.603 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.599+0000 7f72e28be700 1 --2- 172.21.15.99:0/3718897997 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72dc0aeab0 0x7f72dc0aef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:59.603 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.599+0000 7f72e28be700 1 --2- 172.21.15.99:0/3718897997 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72c4003fe0 0x7f72dc15d9f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:59.603 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.599+0000 7f72e28be700 1 --2- 172.21.15.99:0/3718897997 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72dc157470 0x7f72dc0c2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:10:59.603 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.599+0000 7f72e28be700 1 -- 172.21.15.99:0/3718897997 >> 172.21.15.99:0/3718897997 conn(0x7f72dc0cd1e0 msgr2=0x7f72dc0695a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:10:59.603 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.599+0000 7f72e28be700 1 -- 172.21.15.99:0/3718897997 shutdown_connections 2023-10-08T00:10:59.603 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:10:59.599+0000 7f72e28be700 1 -- 172.21.15.99:0/3718897997 wait complete. 2023-10-08T00:10:59.603 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2023-10-08T00:10:59.612 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2023-10-08T00:10:58.024843+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":7,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1552,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560947,"statfs":{"total":1600000000,"available":1598409824,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.952830"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.071916+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:00:50.648234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.068998+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:48:47.624288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.070061+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:23:55.581278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.070506+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:16:40.636514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.069315+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:08:39.785742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.072673+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:16:40.636514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.071429+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:08:39.785742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.072393+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:00:50.648234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.070809+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:48:47.624288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.070483+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:08:39.785742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.069507+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:00:50.648234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.069553+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:48:47.624288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.069081+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:48:47.624288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.072884+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:08:25.404396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.070119+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:48:58.606951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.071176+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:23:55.581278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.070171+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:00:50.648234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.069019+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:00:50.648234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.072572+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:08:39.785742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.069029+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:48:58.606951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.072150+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:48:47.624288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":160,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-08T00:10:44.729923+0000","last_change":"2023-10-08T00:10:43.728490+0000","last_active":"2023-10-08T00:10:44.729923+0000","last_peered":"2023-10-08T00:10:44.729923+0000","last_clean":"2023-10-08T00:10:44.729923+0000","last_became_active":"2023-10-08T00:10:43.728341+0000","last_became_peered":"2023-10-08T00:10:43.728341+0000","last_unstale":"2023-10-08T00:10:44.729923+0000","last_undegraded":"2023-10-08T00:10:44.729923+0000","last_fullsized":"2023-10-08T00:10:44.729923+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_clean_scrub_stamp":"2023-10-08T00:10:42.716096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:06:31.035293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.072171+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:23:55.581278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.073149+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:10:42.328030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.071633+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:48:47.624288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.072117+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:48:58.606951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.070144+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:48:47.624288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.070658+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:16:07.345714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.071086+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:00:50.648234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.072116+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:16:07.345714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.070888+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:48:47.624288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.069442+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:16:40.636514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:10:52.061775+0000","last_change":"2023-10-08T00:10:52.072128+0000","last_active":"2023-10-08T00:10:52.061775+0000","last_peered":"2023-10-08T00:10:52.061775+0000","last_clean":"2023-10-08T00:10:52.061775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:52.061775+0000","last_undegraded":"2023-10-08T00:10:52.061775+0000","last_fullsized":"2023-10-08T00:10:52.061775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:48:58.606951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:10:53.067272+0000","last_change":"2023-10-08T00:10:53.067272+0000","last_active":"2023-10-08T00:10:53.067272+0000","last_peered":"2023-10-08T00:10:53.067272+0000","last_clean":"2023-10-08T00:10:53.067272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:10:53.067272+0000","last_undegraded":"2023-10-08T00:10:53.067272+0000","last_fullsized":"2023-10-08T00:10:53.067272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948820,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948820,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":626,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194685,"statfs":{"total":200000000,"available":199358452,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948820,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948820,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948820,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.318}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948820,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":626,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194685,"statfs":{"total":200000000,"available":199358452,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]}]}],"pool_statfs":[]}} 2023-10-08T00:11:02.619 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T00:11:02.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.745+0000 7f81b4417700 1 Processor -- start 2023-10-08T00:11:02.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.745+0000 7f81b4417700 1 -- start start 2023-10-08T00:11:02.748 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.746+0000 7f81b4417700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f81ac15a620 0x7f81ac15d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:02.748 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.746+0000 7f81b4417700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81ac15d580 0x7f81ac158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:02.748 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.746+0000 7f81b4417700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81ac15cbb0 con 0x7f81ac157870 2023-10-08T00:11:02.748 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.746+0000 7f81b4417700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81ac15bdb0 con 0x7f81ac15a620 2023-10-08T00:11:02.748 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.746+0000 7f81b4417700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac15bfa0 con 0x7f81ac15d580 2023-10-08T00:11:02.749 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.746+0000 7f81b29b4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81ac15d580 0x7f81ac158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:02.749 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.746+0000 7f81b29b4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81ac15d580 0x7f81ac158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.99:36524/0 (socket says 172.21.15.99:36524) 2023-10-08T00:11:02.749 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.746+0000 7f81b29b4700 1 -- 172.21.15.99:0/3842251784 learned_addr learned my addr 172.21.15.99:0/3842251784 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:02.749 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.746+0000 7f81b19b2700 1 --2- 172.21.15.99:0/3842251784 >> v2:172.21.15.191:3300/0 conn(0x7f81ac15a620 0x7f81ac15d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:02.749 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.746+0000 7f81b29b4700 1 -- 172.21.15.99:0/3842251784 >> v2:172.21.15.191:3300/0 conn(0x7f81ac15a620 msgr2=0x7f81ac15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:02.749 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.746+0000 7f81b29b4700 1 --2- 172.21.15.99:0/3842251784 >> v2:172.21.15.191:3300/0 conn(0x7f81ac15a620 0x7f81ac15d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:02.749 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.746+0000 7f81b29b4700 1 -- 172.21.15.99:0/3842251784 >> v1:172.21.15.191:6789/0 conn(0x7f81ac157870 legacy=0x7f81ac15b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:02.749 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.746+0000 7f81b29b4700 1 -- 172.21.15.99:0/3842251784 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81ac15c280 con 0x7f81ac15d580 2023-10-08T00:11:02.749 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.746+0000 7f81b19b2700 1 --2- 172.21.15.99:0/3842251784 >> v2:172.21.15.191:3300/0 conn(0x7f81ac15a620 0x7f81ac15d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:11:02.749 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.746+0000 7f81b29b4700 1 --2- 172.21.15.99:0/3842251784 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81ac15d580 0x7f81ac158110 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f81a4009f20 tx=0x7f81a4009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21bb9d9641dff542 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:02.749 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.747+0000 7f81b11b1700 1 -- 172.21.15.99:0/3842251784 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a400e070 con 0x7f81ac15d580 2023-10-08T00:11:02.750 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.747+0000 7f81b11b1700 1 -- 172.21.15.99:0/3842251784 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a400b9d0 con 0x7f81ac15d580 2023-10-08T00:11:02.750 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.747+0000 7f81b11b1700 1 -- 172.21.15.99:0/3842251784 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a40092e0 con 0x7f81ac15d580 2023-10-08T00:11:02.750 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.747+0000 7f81b4417700 1 -- 172.21.15.99:0/3842251784 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81ac15d580 msgr2=0x7f81ac158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:02.750 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.747+0000 7f81b4417700 1 --2- 172.21.15.99:0/3842251784 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81ac15d580 0x7f81ac158110 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f81a4009f20 tx=0x7f81a4009840 comp rx=0 tx=0).stop 2023-10-08T00:11:02.750 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.747+0000 7f81b4417700 1 -- 172.21.15.99:0/3842251784 shutdown_connections 2023-10-08T00:11:02.750 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.747+0000 7f81b4417700 1 --2- 172.21.15.99:0/3842251784 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81ac15d580 0x7f81ac158110 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:02.750 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.747+0000 7f81b4417700 1 --2- 172.21.15.99:0/3842251784 >> v2:172.21.15.191:3300/0 conn(0x7f81ac15a620 0x7f81ac15d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:02.750 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.747+0000 7f81b4417700 1 -- 172.21.15.99:0/3842251784 >> 172.21.15.99:0/3842251784 conn(0x7f81ac068ce0 msgr2=0x7f81ac0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:02.750 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.747+0000 7f81b4417700 1 -- 172.21.15.99:0/3842251784 shutdown_connections 2023-10-08T00:11:02.750 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.747+0000 7f81b4417700 1 -- 172.21.15.99:0/3842251784 wait complete. 2023-10-08T00:11:02.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.748+0000 7f81b4417700 1 Processor -- start 2023-10-08T00:11:02.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.748+0000 7f81b4417700 1 -- start start 2023-10-08T00:11:02.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.749+0000 7f81b4417700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81ac157870 0x7f81ac0d04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:02.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.749+0000 7f81b4417700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81ac15a620 0x7f81ac14abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:02.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.749+0000 7f81b4417700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81ac15d580 0x7f81ac1497b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:02.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.749+0000 7f81b4417700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac0b1140 con 0x7f81ac157870 2023-10-08T00:11:02.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.749+0000 7f81b4417700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac0b13e0 con 0x7f81ac15d580 2023-10-08T00:11:02.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.749+0000 7f81b4417700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac14bcb0 con 0x7f81ac15a620 2023-10-08T00:11:02.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.749+0000 7f81b21b3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81ac157870 0x7f81ac0d04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:02.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.749+0000 7f81b21b3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81ac157870 0x7f81ac0d04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49564/0 (socket says 172.21.15.99:49564) 2023-10-08T00:11:02.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.749+0000 7f81b21b3700 1 -- 172.21.15.99:0/1963925254 learned_addr learned my addr 172.21.15.99:0/1963925254 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:02.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.749+0000 7f81b29b4700 1 --2- 172.21.15.99:0/1963925254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81ac15d580 0x7f81ac1497b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:02.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.749+0000 7f81b19b2700 1 --2- 172.21.15.99:0/1963925254 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81ac15a620 0x7f81ac14abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:02.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.749+0000 7f81b21b3700 1 -- 172.21.15.99:0/1963925254 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81ac15a620 msgr2=0x7f81ac14abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:02.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.749+0000 7f81b21b3700 1 --2- 172.21.15.99:0/1963925254 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81ac15a620 0x7f81ac14abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:02.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.749+0000 7f81b21b3700 1 -- 172.21.15.99:0/1963925254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81ac15d580 msgr2=0x7f81ac1497b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:02.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.749+0000 7f81b21b3700 1 --2- 172.21.15.99:0/1963925254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81ac15d580 0x7f81ac1497b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:02.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.749+0000 7f81b21b3700 1 -- 172.21.15.99:0/1963925254 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f819c009420 con 0x7f81ac157870 2023-10-08T00:11:02.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.750+0000 7f81b21b3700 1 --2- 172.21.15.99:0/1963925254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81ac157870 0x7f81ac0d04c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f81a800c5c0 tx=0x7f81a800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:02.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.750+0000 7f81a27fc700 1 -- 172.21.15.99:0/1963925254 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a8011590 con 0x7f81ac157870 2023-10-08T00:11:02.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.750+0000 7f81a27fc700 1 -- 172.21.15.99:0/1963925254 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a8011710 con 0x7f81ac157870 2023-10-08T00:11:02.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.750+0000 7f81b4417700 1 -- 172.21.15.99:0/1963925254 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81a4009ae0 con 0x7f81ac157870 2023-10-08T00:11:02.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.750+0000 7f81a27fc700 1 -- 172.21.15.99:0/1963925254 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a8011ad0 con 0x7f81ac157870 2023-10-08T00:11:02.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.750+0000 7f81b4417700 1 -- 172.21.15.99:0/1963925254 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81ac14b4c0 con 0x7f81ac157870 2023-10-08T00:11:02.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.751+0000 7f81b4417700 1 -- 172.21.15.99:0/1963925254 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81ac04c3c0 con 0x7f81ac157870 2023-10-08T00:11:02.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.751+0000 7f81a27fc700 1 -- 172.21.15.99:0/1963925254 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f81a801b3b0 con 0x7f81ac157870 2023-10-08T00:11:02.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.751+0000 7f81a27fc700 1 --2- 172.21.15.99:0/1963925254 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8190066f30 0x7f81900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:02.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.752+0000 7f81a27fc700 1 -- 172.21.15.99:0/1963925254 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f81a8083a10 con 0x7f81ac157870 2023-10-08T00:11:02.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.752+0000 7f81b19b2700 1 --2- 172.21.15.99:0/1963925254 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8190066f30 0x7f81900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:02.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.752+0000 7f81b19b2700 1 --2- 172.21.15.99:0/1963925254 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8190066f30 0x7f81900693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f81ac14cf20 tx=0x7f819c0091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:02.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.756+0000 7f81a27fc700 1 -- 172.21.15.99:0/1963925254 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f81a8050c80 con 0x7f81ac157870 2023-10-08T00:11:02.933 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.933+0000 7f81b4417700 1 -- 172.21.15.99:0/1963925254 --> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f81ac14ba90 con 0x7f8190066f30 2023-10-08T00:11:02.941 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.941+0000 7f81a27fc700 1 -- 172.21.15.99:0/1963925254 <== mgr.4099 v2:172.21.15.191:6832/85985 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+260062 (secure 0 0 0) 0x7f81ac14ba90 con 0x7f8190066f30 2023-10-08T00:11:02.944 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:11:02.947 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.944+0000 7f81b4417700 1 -- 172.21.15.99:0/1963925254 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8190066f30 msgr2=0x7f81900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:02.947 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.944+0000 7f81b4417700 1 --2- 172.21.15.99:0/1963925254 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8190066f30 0x7f81900693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f81ac14cf20 tx=0x7f819c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:02.947 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.944+0000 7f81b4417700 1 -- 172.21.15.99:0/1963925254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81ac157870 msgr2=0x7f81ac0d04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:02.947 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.944+0000 7f81b4417700 1 --2- 172.21.15.99:0/1963925254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81ac157870 0x7f81ac0d04c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f81a800c5c0 tx=0x7f81a800c8d0 comp rx=0 tx=0).stop 2023-10-08T00:11:02.947 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.944+0000 7f81b4417700 1 -- 172.21.15.99:0/1963925254 shutdown_connections 2023-10-08T00:11:02.947 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.944+0000 7f81b4417700 1 --2- 172.21.15.99:0/1963925254 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8190066f30 0x7f81900693e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:02.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.944+0000 7f81b4417700 1 --2- 172.21.15.99:0/1963925254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81ac15d580 0x7f81ac1497b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:02.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.944+0000 7f81b4417700 1 --2- 172.21.15.99:0/1963925254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81ac157870 0x7f81ac0d04c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:02.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.944+0000 7f81b4417700 1 --2- 172.21.15.99:0/1963925254 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81ac15a620 0x7f81ac14abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:02.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.944+0000 7f81b4417700 1 -- 172.21.15.99:0/1963925254 >> 172.21.15.99:0/1963925254 conn(0x7f81ac068ce0 msgr2=0x7f81ac14c670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:02.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.944+0000 7f81b4417700 1 -- 172.21.15.99:0/1963925254 shutdown_connections 2023-10-08T00:11:02.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:02.944+0000 7f81b4417700 1 -- 172.21.15.99:0/1963925254 wait complete. 2023-10-08T00:11:02.948 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2023-10-08T00:11:02.956 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-10-08T00:11:02.025782+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1708,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560791,"statfs":{"total":1600000000,"available":1598250700,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002077"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.095544+0000","last_change":"2023-10-08T00:10:54.094801+0000","last_active":"2023-10-08T00:10:54.095544+0000","last_peered":"2023-10-08T00:10:54.095544+0000","last_clean":"2023-10-08T00:10:54.095544+0000","last_became_active":"2023-10-08T00:10:54.094221+0000","last_became_peered":"2023-10-08T00:10:54.094221+0000","last_unstale":"2023-10-08T00:10:54.095544+0000","last_undegraded":"2023-10-08T00:10:54.095544+0000","last_fullsized":"2023-10-08T00:10:54.095544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:30:37.085534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.102469+0000","last_change":"2023-10-08T00:10:54.101252+0000","last_active":"2023-10-08T00:10:54.102469+0000","last_peered":"2023-10-08T00:10:54.102469+0000","last_clean":"2023-10-08T00:10:54.102469+0000","last_became_active":"2023-10-08T00:10:54.101016+0000","last_became_peered":"2023-10-08T00:10:54.101016+0000","last_unstale":"2023-10-08T00:10:54.102469+0000","last_undegraded":"2023-10-08T00:10:54.102469+0000","last_fullsized":"2023-10-08T00:10:54.102469+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:30:15.548694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.104771+0000","last_change":"2023-10-08T00:10:54.104705+0000","last_active":"2023-10-08T00:10:54.104771+0000","last_peered":"2023-10-08T00:10:54.104771+0000","last_clean":"2023-10-08T00:10:54.104771+0000","last_became_active":"2023-10-08T00:10:54.104469+0000","last_became_peered":"2023-10-08T00:10:54.104469+0000","last_unstale":"2023-10-08T00:10:54.104771+0000","last_undegraded":"2023-10-08T00:10:54.104771+0000","last_fullsized":"2023-10-08T00:10:54.104771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:27:57.768735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.090488+0000","last_change":"2023-10-08T00:10:54.089800+0000","last_active":"2023-10-08T00:10:54.090488+0000","last_peered":"2023-10-08T00:10:54.090488+0000","last_clean":"2023-10-08T00:10:54.090488+0000","last_became_active":"2023-10-08T00:10:54.089200+0000","last_became_peered":"2023-10-08T00:10:54.089200+0000","last_unstale":"2023-10-08T00:10:54.090488+0000","last_undegraded":"2023-10-08T00:10:54.090488+0000","last_fullsized":"2023-10-08T00:10:54.090488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:44:19.725645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.099476+0000","last_change":"2023-10-08T00:10:54.099417+0000","last_active":"2023-10-08T00:10:54.099476+0000","last_peered":"2023-10-08T00:10:54.099476+0000","last_clean":"2023-10-08T00:10:54.099476+0000","last_became_active":"2023-10-08T00:10:54.099060+0000","last_became_peered":"2023-10-08T00:10:54.099060+0000","last_unstale":"2023-10-08T00:10:54.099476+0000","last_undegraded":"2023-10-08T00:10:54.099476+0000","last_fullsized":"2023-10-08T00:10:54.099476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:20:35.671993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.088556+0000","last_change":"2023-10-08T00:10:54.088499+0000","last_active":"2023-10-08T00:10:54.088556+0000","last_peered":"2023-10-08T00:10:54.088556+0000","last_clean":"2023-10-08T00:10:54.088556+0000","last_became_active":"2023-10-08T00:10:54.088247+0000","last_became_peered":"2023-10-08T00:10:54.088247+0000","last_unstale":"2023-10-08T00:10:54.088556+0000","last_undegraded":"2023-10-08T00:10:54.088556+0000","last_fullsized":"2023-10-08T00:10:54.088556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:49:51.549111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.098443+0000","last_change":"2023-10-08T00:10:54.098398+0000","last_active":"2023-10-08T00:10:54.098443+0000","last_peered":"2023-10-08T00:10:54.098443+0000","last_clean":"2023-10-08T00:10:54.098443+0000","last_became_active":"2023-10-08T00:10:54.098065+0000","last_became_peered":"2023-10-08T00:10:54.098065+0000","last_unstale":"2023-10-08T00:10:54.098443+0000","last_undegraded":"2023-10-08T00:10:54.098443+0000","last_fullsized":"2023-10-08T00:10:54.098443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:05:53.331248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.099579+0000","last_change":"2023-10-08T00:10:54.098754+0000","last_active":"2023-10-08T00:10:54.099579+0000","last_peered":"2023-10-08T00:10:54.099579+0000","last_clean":"2023-10-08T00:10:54.099579+0000","last_became_active":"2023-10-08T00:10:54.098505+0000","last_became_peered":"2023-10-08T00:10:54.098505+0000","last_unstale":"2023-10-08T00:10:54.099579+0000","last_undegraded":"2023-10-08T00:10:54.099579+0000","last_fullsized":"2023-10-08T00:10:54.099579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:20:35.671993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.096851+0000","last_change":"2023-10-08T00:10:54.096792+0000","last_active":"2023-10-08T00:10:54.096851+0000","last_peered":"2023-10-08T00:10:54.096851+0000","last_clean":"2023-10-08T00:10:54.096851+0000","last_became_active":"2023-10-08T00:10:54.096569+0000","last_became_peered":"2023-10-08T00:10:54.096569+0000","last_unstale":"2023-10-08T00:10:54.096851+0000","last_undegraded":"2023-10-08T00:10:54.096851+0000","last_fullsized":"2023-10-08T00:10:54.096851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:40:12.384542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.093509+0000","last_change":"2023-10-08T00:10:54.092144+0000","last_active":"2023-10-08T00:10:54.093509+0000","last_peered":"2023-10-08T00:10:54.093509+0000","last_clean":"2023-10-08T00:10:54.093509+0000","last_became_active":"2023-10-08T00:10:54.091938+0000","last_became_peered":"2023-10-08T00:10:54.091938+0000","last_unstale":"2023-10-08T00:10:54.093509+0000","last_undegraded":"2023-10-08T00:10:54.093509+0000","last_fullsized":"2023-10-08T00:10:54.093509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:13:19.313381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.095525+0000","last_change":"2023-10-08T00:10:54.094651+0000","last_active":"2023-10-08T00:10:54.095525+0000","last_peered":"2023-10-08T00:10:54.095525+0000","last_clean":"2023-10-08T00:10:54.095525+0000","last_became_active":"2023-10-08T00:10:54.093973+0000","last_became_peered":"2023-10-08T00:10:54.093973+0000","last_unstale":"2023-10-08T00:10:54.095525+0000","last_undegraded":"2023-10-08T00:10:54.095525+0000","last_fullsized":"2023-10-08T00:10:54.095525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:27:57.768735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.093543+0000","last_change":"2023-10-08T00:10:54.092456+0000","last_active":"2023-10-08T00:10:54.093543+0000","last_peered":"2023-10-08T00:10:54.093543+0000","last_clean":"2023-10-08T00:10:54.093543+0000","last_became_active":"2023-10-08T00:10:54.092121+0000","last_became_peered":"2023-10-08T00:10:54.092121+0000","last_unstale":"2023-10-08T00:10:54.093543+0000","last_undegraded":"2023-10-08T00:10:54.093543+0000","last_fullsized":"2023-10-08T00:10:54.093543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:17:33.566050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.102806+0000","last_change":"2023-10-08T00:10:54.102717+0000","last_active":"2023-10-08T00:10:54.102806+0000","last_peered":"2023-10-08T00:10:54.102806+0000","last_clean":"2023-10-08T00:10:54.102806+0000","last_became_active":"2023-10-08T00:10:54.102373+0000","last_became_peered":"2023-10-08T00:10:54.102373+0000","last_unstale":"2023-10-08T00:10:54.102806+0000","last_undegraded":"2023-10-08T00:10:54.102806+0000","last_fullsized":"2023-10-08T00:10:54.102806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:20:26.989181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.103024+0000","last_change":"2023-10-08T00:10:54.102984+0000","last_active":"2023-10-08T00:10:54.103024+0000","last_peered":"2023-10-08T00:10:54.103024+0000","last_clean":"2023-10-08T00:10:54.103024+0000","last_became_active":"2023-10-08T00:10:54.102645+0000","last_became_peered":"2023-10-08T00:10:54.102645+0000","last_unstale":"2023-10-08T00:10:54.103024+0000","last_undegraded":"2023-10-08T00:10:54.103024+0000","last_fullsized":"2023-10-08T00:10:54.103024+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:44:19.725645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.100866+0000","last_change":"2023-10-08T00:10:54.100800+0000","last_active":"2023-10-08T00:10:54.100866+0000","last_peered":"2023-10-08T00:10:54.100866+0000","last_clean":"2023-10-08T00:10:54.100866+0000","last_became_active":"2023-10-08T00:10:54.100384+0000","last_became_peered":"2023-10-08T00:10:54.100384+0000","last_unstale":"2023-10-08T00:10:54.100866+0000","last_undegraded":"2023-10-08T00:10:54.100866+0000","last_fullsized":"2023-10-08T00:10:54.100866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:05:53.331248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.098456+0000","last_change":"2023-10-08T00:10:54.097988+0000","last_active":"2023-10-08T00:10:54.098456+0000","last_peered":"2023-10-08T00:10:54.098456+0000","last_clean":"2023-10-08T00:10:54.098456+0000","last_became_active":"2023-10-08T00:10:54.097155+0000","last_became_peered":"2023-10-08T00:10:54.097155+0000","last_unstale":"2023-10-08T00:10:54.098456+0000","last_undegraded":"2023-10-08T00:10:54.098456+0000","last_fullsized":"2023-10-08T00:10:54.098456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:03:51.083770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.093657+0000","last_change":"2023-10-08T00:10:54.093109+0000","last_active":"2023-10-08T00:10:54.093657+0000","last_peered":"2023-10-08T00:10:54.093657+0000","last_clean":"2023-10-08T00:10:54.093657+0000","last_became_active":"2023-10-08T00:10:54.092788+0000","last_became_peered":"2023-10-08T00:10:54.092788+0000","last_unstale":"2023-10-08T00:10:54.093657+0000","last_undegraded":"2023-10-08T00:10:54.093657+0000","last_fullsized":"2023-10-08T00:10:54.093657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:37:23.781764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.085610+0000","last_change":"2023-10-08T00:10:54.085533+0000","last_active":"2023-10-08T00:10:54.085610+0000","last_peered":"2023-10-08T00:10:54.085610+0000","last_clean":"2023-10-08T00:10:54.085610+0000","last_became_active":"2023-10-08T00:10:54.085297+0000","last_became_peered":"2023-10-08T00:10:54.085297+0000","last_unstale":"2023-10-08T00:10:54.085610+0000","last_undegraded":"2023-10-08T00:10:54.085610+0000","last_fullsized":"2023-10-08T00:10:54.085610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:42:35.332320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.093626+0000","last_change":"2023-10-08T00:10:54.092802+0000","last_active":"2023-10-08T00:10:54.093626+0000","last_peered":"2023-10-08T00:10:54.093626+0000","last_clean":"2023-10-08T00:10:54.093626+0000","last_became_active":"2023-10-08T00:10:54.092217+0000","last_became_peered":"2023-10-08T00:10:54.092217+0000","last_unstale":"2023-10-08T00:10:54.093626+0000","last_undegraded":"2023-10-08T00:10:54.093626+0000","last_fullsized":"2023-10-08T00:10:54.093626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:20:35.671993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.103423+0000","last_change":"2023-10-08T00:10:53.083367+0000","last_active":"2023-10-08T00:10:54.103423+0000","last_peered":"2023-10-08T00:10:54.103423+0000","last_clean":"2023-10-08T00:10:54.103423+0000","last_became_active":"2023-10-08T00:10:53.083127+0000","last_became_peered":"2023-10-08T00:10:53.083127+0000","last_unstale":"2023-10-08T00:10:54.103423+0000","last_undegraded":"2023-10-08T00:10:54.103423+0000","last_fullsized":"2023-10-08T00:10:54.103423+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:10:42.328030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.099821+0000","last_change":"2023-10-08T00:10:54.099725+0000","last_active":"2023-10-08T00:10:54.099821+0000","last_peered":"2023-10-08T00:10:54.099821+0000","last_clean":"2023-10-08T00:10:54.099821+0000","last_became_active":"2023-10-08T00:10:54.099236+0000","last_became_peered":"2023-10-08T00:10:54.099236+0000","last_unstale":"2023-10-08T00:10:54.099821+0000","last_undegraded":"2023-10-08T00:10:54.099821+0000","last_fullsized":"2023-10-08T00:10:54.099821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:13:19.313381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.091381+0000","last_change":"2023-10-08T00:10:53.078523+0000","last_active":"2023-10-08T00:10:54.091381+0000","last_peered":"2023-10-08T00:10:54.091381+0000","last_clean":"2023-10-08T00:10:54.091381+0000","last_became_active":"2023-10-08T00:10:53.078068+0000","last_became_peered":"2023-10-08T00:10:53.078068+0000","last_unstale":"2023-10-08T00:10:54.091381+0000","last_undegraded":"2023-10-08T00:10:54.091381+0000","last_fullsized":"2023-10-08T00:10:54.091381+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:30:06.828965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.094722+0000","last_change":"2023-10-08T00:10:54.094485+0000","last_active":"2023-10-08T00:10:54.094722+0000","last_peered":"2023-10-08T00:10:54.094722+0000","last_clean":"2023-10-08T00:10:54.094722+0000","last_became_active":"2023-10-08T00:10:54.094246+0000","last_became_peered":"2023-10-08T00:10:54.094246+0000","last_unstale":"2023-10-08T00:10:54.094722+0000","last_undegraded":"2023-10-08T00:10:54.094722+0000","last_fullsized":"2023-10-08T00:10:54.094722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:52:39.600888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.099727+0000","last_change":"2023-10-08T00:10:53.086334+0000","last_active":"2023-10-08T00:10:54.099727+0000","last_peered":"2023-10-08T00:10:54.099727+0000","last_clean":"2023-10-08T00:10:54.099727+0000","last_became_active":"2023-10-08T00:10:53.084152+0000","last_became_peered":"2023-10-08T00:10:53.084152+0000","last_unstale":"2023-10-08T00:10:54.099727+0000","last_undegraded":"2023-10-08T00:10:54.099727+0000","last_fullsized":"2023-10-08T00:10:54.099727+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:08:25.404396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.095590+0000","last_change":"2023-10-08T00:10:54.095225+0000","last_active":"2023-10-08T00:10:54.095590+0000","last_peered":"2023-10-08T00:10:54.095590+0000","last_clean":"2023-10-08T00:10:54.095590+0000","last_became_active":"2023-10-08T00:10:54.095005+0000","last_became_peered":"2023-10-08T00:10:54.095005+0000","last_unstale":"2023-10-08T00:10:54.095590+0000","last_undegraded":"2023-10-08T00:10:54.095590+0000","last_fullsized":"2023-10-08T00:10:54.095590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:49:51.549111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.099165+0000","last_change":"2023-10-08T00:10:53.082935+0000","last_active":"2023-10-08T00:10:54.099165+0000","last_peered":"2023-10-08T00:10:54.099165+0000","last_clean":"2023-10-08T00:10:54.099165+0000","last_became_active":"2023-10-08T00:10:53.082451+0000","last_became_peered":"2023-10-08T00:10:53.082451+0000","last_unstale":"2023-10-08T00:10:54.099165+0000","last_undegraded":"2023-10-08T00:10:54.099165+0000","last_fullsized":"2023-10-08T00:10:54.099165+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:30:06.828965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.093599+0000","last_change":"2023-10-08T00:10:54.092264+0000","last_active":"2023-10-08T00:10:54.093599+0000","last_peered":"2023-10-08T00:10:54.093599+0000","last_clean":"2023-10-08T00:10:54.093599+0000","last_became_active":"2023-10-08T00:10:54.092043+0000","last_became_peered":"2023-10-08T00:10:54.092043+0000","last_unstale":"2023-10-08T00:10:54.093599+0000","last_undegraded":"2023-10-08T00:10:54.093599+0000","last_fullsized":"2023-10-08T00:10:54.093599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:39:51.147291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.101350+0000","last_change":"2023-10-08T00:10:53.083563+0000","last_active":"2023-10-08T00:10:54.101350+0000","last_peered":"2023-10-08T00:10:54.101350+0000","last_clean":"2023-10-08T00:10:54.101350+0000","last_became_active":"2023-10-08T00:10:53.079738+0000","last_became_peered":"2023-10-08T00:10:53.079738+0000","last_unstale":"2023-10-08T00:10:54.101350+0000","last_undegraded":"2023-10-08T00:10:54.101350+0000","last_fullsized":"2023-10-08T00:10:54.101350+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:46:39.872883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.095572+0000","last_change":"2023-10-08T00:10:54.095150+0000","last_active":"2023-10-08T00:10:54.095572+0000","last_peered":"2023-10-08T00:10:54.095572+0000","last_clean":"2023-10-08T00:10:54.095572+0000","last_became_active":"2023-10-08T00:10:54.094621+0000","last_became_peered":"2023-10-08T00:10:54.094621+0000","last_unstale":"2023-10-08T00:10:54.095572+0000","last_undegraded":"2023-10-08T00:10:54.095572+0000","last_fullsized":"2023-10-08T00:10:54.095572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:42:35.332320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.098615+0000","last_change":"2023-10-08T00:10:53.085123+0000","last_active":"2023-10-08T00:10:54.098615+0000","last_peered":"2023-10-08T00:10:54.098615+0000","last_clean":"2023-10-08T00:10:54.098615+0000","last_became_active":"2023-10-08T00:10:53.084901+0000","last_became_peered":"2023-10-08T00:10:53.084901+0000","last_unstale":"2023-10-08T00:10:54.098615+0000","last_undegraded":"2023-10-08T00:10:54.098615+0000","last_fullsized":"2023-10-08T00:10:54.098615+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:53:05.715993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.098231+0000","last_change":"2023-10-08T00:10:54.098150+0000","last_active":"2023-10-08T00:10:54.098231+0000","last_peered":"2023-10-08T00:10:54.098231+0000","last_clean":"2023-10-08T00:10:54.098231+0000","last_became_active":"2023-10-08T00:10:54.097887+0000","last_became_peered":"2023-10-08T00:10:54.097887+0000","last_unstale":"2023-10-08T00:10:54.098231+0000","last_undegraded":"2023-10-08T00:10:54.098231+0000","last_fullsized":"2023-10-08T00:10:54.098231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:06:38.007522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.096544+0000","last_change":"2023-10-08T00:10:53.082817+0000","last_active":"2023-10-08T00:10:54.096544+0000","last_peered":"2023-10-08T00:10:54.096544+0000","last_clean":"2023-10-08T00:10:54.096544+0000","last_became_active":"2023-10-08T00:10:53.082576+0000","last_became_peered":"2023-10-08T00:10:53.082576+0000","last_unstale":"2023-10-08T00:10:54.096544+0000","last_undegraded":"2023-10-08T00:10:54.096544+0000","last_fullsized":"2023-10-08T00:10:54.096544+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:46:39.872883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.100057+0000","last_change":"2023-10-08T00:10:54.099958+0000","last_active":"2023-10-08T00:10:54.100057+0000","last_peered":"2023-10-08T00:10:54.100057+0000","last_clean":"2023-10-08T00:10:54.100057+0000","last_became_active":"2023-10-08T00:10:54.099664+0000","last_became_peered":"2023-10-08T00:10:54.099664+0000","last_unstale":"2023-10-08T00:10:54.100057+0000","last_undegraded":"2023-10-08T00:10:54.100057+0000","last_fullsized":"2023-10-08T00:10:54.100057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:17:33.566050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.090051+0000","last_change":"2023-10-08T00:10:53.077494+0000","last_active":"2023-10-08T00:10:54.090051+0000","last_peered":"2023-10-08T00:10:54.090051+0000","last_clean":"2023-10-08T00:10:54.090051+0000","last_became_active":"2023-10-08T00:10:53.077248+0000","last_became_peered":"2023-10-08T00:10:53.077248+0000","last_unstale":"2023-10-08T00:10:54.090051+0000","last_undegraded":"2023-10-08T00:10:54.090051+0000","last_fullsized":"2023-10-08T00:10:54.090051+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:46:39.872883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.097553+0000","last_change":"2023-10-08T00:10:54.095175+0000","last_active":"2023-10-08T00:10:54.097553+0000","last_peered":"2023-10-08T00:10:54.097553+0000","last_clean":"2023-10-08T00:10:54.097553+0000","last_became_active":"2023-10-08T00:10:54.093363+0000","last_became_peered":"2023-10-08T00:10:54.093363+0000","last_unstale":"2023-10-08T00:10:54.097553+0000","last_undegraded":"2023-10-08T00:10:54.097553+0000","last_fullsized":"2023-10-08T00:10:54.097553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:03:42.084791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.087857+0000","last_change":"2023-10-08T00:10:53.079999+0000","last_active":"2023-10-08T00:10:54.087857+0000","last_peered":"2023-10-08T00:10:54.087857+0000","last_clean":"2023-10-08T00:10:54.087857+0000","last_became_active":"2023-10-08T00:10:53.079750+0000","last_became_peered":"2023-10-08T00:10:53.079750+0000","last_unstale":"2023-10-08T00:10:54.087857+0000","last_undegraded":"2023-10-08T00:10:54.087857+0000","last_fullsized":"2023-10-08T00:10:54.087857+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:00:50.648234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.087312+0000","last_change":"2023-10-08T00:10:54.087253+0000","last_active":"2023-10-08T00:10:54.087312+0000","last_peered":"2023-10-08T00:10:54.087312+0000","last_clean":"2023-10-08T00:10:54.087312+0000","last_became_active":"2023-10-08T00:10:54.087034+0000","last_became_peered":"2023-10-08T00:10:54.087034+0000","last_unstale":"2023-10-08T00:10:54.087312+0000","last_undegraded":"2023-10-08T00:10:54.087312+0000","last_fullsized":"2023-10-08T00:10:54.087312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:21:24.202186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.090769+0000","last_change":"2023-10-08T00:10:53.088156+0000","last_active":"2023-10-08T00:10:54.090769+0000","last_peered":"2023-10-08T00:10:54.090769+0000","last_clean":"2023-10-08T00:10:54.090769+0000","last_became_active":"2023-10-08T00:10:53.084715+0000","last_became_peered":"2023-10-08T00:10:53.084715+0000","last_unstale":"2023-10-08T00:10:54.090769+0000","last_undegraded":"2023-10-08T00:10:54.090769+0000","last_fullsized":"2023-10-08T00:10:54.090769+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:10:42.328030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.100622+0000","last_change":"2023-10-08T00:10:54.100149+0000","last_active":"2023-10-08T00:10:54.100622+0000","last_peered":"2023-10-08T00:10:54.100622+0000","last_clean":"2023-10-08T00:10:54.100622+0000","last_became_active":"2023-10-08T00:10:54.099890+0000","last_became_peered":"2023-10-08T00:10:54.099890+0000","last_unstale":"2023-10-08T00:10:54.100622+0000","last_undegraded":"2023-10-08T00:10:54.100622+0000","last_fullsized":"2023-10-08T00:10:54.100622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:30:37.085534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.092369+0000","last_change":"2023-10-08T00:10:53.077948+0000","last_active":"2023-10-08T00:10:54.092369+0000","last_peered":"2023-10-08T00:10:54.092369+0000","last_clean":"2023-10-08T00:10:54.092369+0000","last_became_active":"2023-10-08T00:10:53.077716+0000","last_became_peered":"2023-10-08T00:10:53.077716+0000","last_unstale":"2023-10-08T00:10:54.092369+0000","last_undegraded":"2023-10-08T00:10:54.092369+0000","last_fullsized":"2023-10-08T00:10:54.092369+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:08:25.404396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.101262+0000","last_change":"2023-10-08T00:10:54.101209+0000","last_active":"2023-10-08T00:10:54.101262+0000","last_peered":"2023-10-08T00:10:54.101262+0000","last_clean":"2023-10-08T00:10:54.101262+0000","last_became_active":"2023-10-08T00:10:54.100833+0000","last_became_peered":"2023-10-08T00:10:54.100833+0000","last_unstale":"2023-10-08T00:10:54.101262+0000","last_undegraded":"2023-10-08T00:10:54.101262+0000","last_fullsized":"2023-10-08T00:10:54.101262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:28:03.272641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.105007+0000","last_change":"2023-10-08T00:10:53.078194+0000","last_active":"2023-10-08T00:10:54.105007+0000","last_peered":"2023-10-08T00:10:54.105007+0000","last_clean":"2023-10-08T00:10:54.105007+0000","last_became_active":"2023-10-08T00:10:53.077936+0000","last_became_peered":"2023-10-08T00:10:53.077936+0000","last_unstale":"2023-10-08T00:10:54.105007+0000","last_undegraded":"2023-10-08T00:10:54.105007+0000","last_fullsized":"2023-10-08T00:10:54.105007+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:00:50.648234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.100556+0000","last_change":"2023-10-08T00:10:54.100459+0000","last_active":"2023-10-08T00:10:54.100556+0000","last_peered":"2023-10-08T00:10:54.100556+0000","last_clean":"2023-10-08T00:10:54.100556+0000","last_became_active":"2023-10-08T00:10:54.100194+0000","last_became_peered":"2023-10-08T00:10:54.100194+0000","last_unstale":"2023-10-08T00:10:54.100556+0000","last_undegraded":"2023-10-08T00:10:54.100556+0000","last_fullsized":"2023-10-08T00:10:54.100556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:22:42.484817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.096915+0000","last_change":"2023-10-08T00:10:53.084910+0000","last_active":"2023-10-08T00:10:54.096915+0000","last_peered":"2023-10-08T00:10:54.096915+0000","last_clean":"2023-10-08T00:10:54.096915+0000","last_became_active":"2023-10-08T00:10:53.084674+0000","last_became_peered":"2023-10-08T00:10:53.084674+0000","last_unstale":"2023-10-08T00:10:54.096915+0000","last_undegraded":"2023-10-08T00:10:54.096915+0000","last_fullsized":"2023-10-08T00:10:54.096915+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:08:39.785742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.102086+0000","last_change":"2023-10-08T00:10:54.102036+0000","last_active":"2023-10-08T00:10:54.102086+0000","last_peered":"2023-10-08T00:10:54.102086+0000","last_clean":"2023-10-08T00:10:54.102086+0000","last_became_active":"2023-10-08T00:10:54.101877+0000","last_became_peered":"2023-10-08T00:10:54.101877+0000","last_unstale":"2023-10-08T00:10:54.102086+0000","last_undegraded":"2023-10-08T00:10:54.102086+0000","last_fullsized":"2023-10-08T00:10:54.102086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:21:24.202186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.084370+0000","last_change":"2023-10-08T00:10:53.082679+0000","last_active":"2023-10-08T00:10:54.084370+0000","last_peered":"2023-10-08T00:10:54.084370+0000","last_clean":"2023-10-08T00:10:54.084370+0000","last_became_active":"2023-10-08T00:10:53.082281+0000","last_became_peered":"2023-10-08T00:10:53.082281+0000","last_unstale":"2023-10-08T00:10:54.084370+0000","last_undegraded":"2023-10-08T00:10:54.084370+0000","last_fullsized":"2023-10-08T00:10:54.084370+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:11:04.856043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.099339+0000","last_change":"2023-10-08T00:10:54.099282+0000","last_active":"2023-10-08T00:10:54.099339+0000","last_peered":"2023-10-08T00:10:54.099339+0000","last_clean":"2023-10-08T00:10:54.099339+0000","last_became_active":"2023-10-08T00:10:54.099043+0000","last_became_peered":"2023-10-08T00:10:54.099043+0000","last_unstale":"2023-10-08T00:10:54.099339+0000","last_undegraded":"2023-10-08T00:10:54.099339+0000","last_fullsized":"2023-10-08T00:10:54.099339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:11:32.172262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.090870+0000","last_change":"2023-10-08T00:10:53.088502+0000","last_active":"2023-10-08T00:10:54.090870+0000","last_peered":"2023-10-08T00:10:54.090870+0000","last_clean":"2023-10-08T00:10:54.090870+0000","last_became_active":"2023-10-08T00:10:53.086396+0000","last_became_peered":"2023-10-08T00:10:53.086396+0000","last_unstale":"2023-10-08T00:10:54.090870+0000","last_undegraded":"2023-10-08T00:10:54.090870+0000","last_fullsized":"2023-10-08T00:10:54.090870+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:42:34.326823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.095605+0000","last_change":"2023-10-08T00:10:54.095487+0000","last_active":"2023-10-08T00:10:54.095605+0000","last_peered":"2023-10-08T00:10:54.095605+0000","last_clean":"2023-10-08T00:10:54.095605+0000","last_became_active":"2023-10-08T00:10:54.094918+0000","last_became_peered":"2023-10-08T00:10:54.094918+0000","last_unstale":"2023-10-08T00:10:54.095605+0000","last_undegraded":"2023-10-08T00:10:54.095605+0000","last_fullsized":"2023-10-08T00:10:54.095605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:46:53.130793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.104070+0000","last_change":"2023-10-08T00:10:53.078081+0000","last_active":"2023-10-08T00:10:54.104070+0000","last_peered":"2023-10-08T00:10:54.104070+0000","last_clean":"2023-10-08T00:10:54.104070+0000","last_became_active":"2023-10-08T00:10:53.077802+0000","last_became_peered":"2023-10-08T00:10:53.077802+0000","last_unstale":"2023-10-08T00:10:54.104070+0000","last_undegraded":"2023-10-08T00:10:54.104070+0000","last_fullsized":"2023-10-08T00:10:54.104070+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:16:40.636514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.093642+0000","last_change":"2023-10-08T00:10:54.092992+0000","last_active":"2023-10-08T00:10:54.093642+0000","last_peered":"2023-10-08T00:10:54.093642+0000","last_clean":"2023-10-08T00:10:54.093642+0000","last_became_active":"2023-10-08T00:10:54.092509+0000","last_became_peered":"2023-10-08T00:10:54.092509+0000","last_unstale":"2023-10-08T00:10:54.093642+0000","last_undegraded":"2023-10-08T00:10:54.093642+0000","last_fullsized":"2023-10-08T00:10:54.093642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:52:39.600888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.100708+0000","last_change":"2023-10-08T00:10:53.084676+0000","last_active":"2023-10-08T00:10:54.100708+0000","last_peered":"2023-10-08T00:10:54.100708+0000","last_clean":"2023-10-08T00:10:54.100708+0000","last_became_active":"2023-10-08T00:10:53.084468+0000","last_became_peered":"2023-10-08T00:10:53.084468+0000","last_unstale":"2023-10-08T00:10:54.100708+0000","last_undegraded":"2023-10-08T00:10:54.100708+0000","last_fullsized":"2023-10-08T00:10:54.100708+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:46:39.872883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.099482+0000","last_change":"2023-10-08T00:10:54.093441+0000","last_active":"2023-10-08T00:10:54.099482+0000","last_peered":"2023-10-08T00:10:54.099482+0000","last_clean":"2023-10-08T00:10:54.099482+0000","last_became_active":"2023-10-08T00:10:54.093093+0000","last_became_peered":"2023-10-08T00:10:54.093093+0000","last_unstale":"2023-10-08T00:10:54.099482+0000","last_undegraded":"2023-10-08T00:10:54.099482+0000","last_fullsized":"2023-10-08T00:10:54.099482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:25:03.816735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.093686+0000","last_change":"2023-10-08T00:10:53.080035+0000","last_active":"2023-10-08T00:10:54.093686+0000","last_peered":"2023-10-08T00:10:54.093686+0000","last_clean":"2023-10-08T00:10:54.093686+0000","last_became_active":"2023-10-08T00:10:53.079674+0000","last_became_peered":"2023-10-08T00:10:53.079674+0000","last_unstale":"2023-10-08T00:10:54.093686+0000","last_undegraded":"2023-10-08T00:10:54.093686+0000","last_fullsized":"2023-10-08T00:10:54.093686+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:08:25.404396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.100705+0000","last_change":"2023-10-08T00:10:54.100588+0000","last_active":"2023-10-08T00:10:54.100705+0000","last_peered":"2023-10-08T00:10:54.100705+0000","last_clean":"2023-10-08T00:10:54.100705+0000","last_became_active":"2023-10-08T00:10:54.100259+0000","last_became_peered":"2023-10-08T00:10:54.100259+0000","last_unstale":"2023-10-08T00:10:54.100705+0000","last_undegraded":"2023-10-08T00:10:54.100705+0000","last_fullsized":"2023-10-08T00:10:54.100705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:40:12.384542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.084858+0000","last_change":"2023-10-08T00:10:53.079356+0000","last_active":"2023-10-08T00:10:54.084858+0000","last_peered":"2023-10-08T00:10:54.084858+0000","last_clean":"2023-10-08T00:10:54.084858+0000","last_became_active":"2023-10-08T00:10:53.079134+0000","last_became_peered":"2023-10-08T00:10:53.079134+0000","last_unstale":"2023-10-08T00:10:54.084858+0000","last_undegraded":"2023-10-08T00:10:54.084858+0000","last_fullsized":"2023-10-08T00:10:54.084858+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:30:06.828965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.094617+0000","last_change":"2023-10-08T00:10:54.094512+0000","last_active":"2023-10-08T00:10:54.094617+0000","last_peered":"2023-10-08T00:10:54.094617+0000","last_clean":"2023-10-08T00:10:54.094617+0000","last_became_active":"2023-10-08T00:10:54.093703+0000","last_became_peered":"2023-10-08T00:10:54.093703+0000","last_unstale":"2023-10-08T00:10:54.094617+0000","last_undegraded":"2023-10-08T00:10:54.094617+0000","last_fullsized":"2023-10-08T00:10:54.094617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:03:42.084791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.093726+0000","last_change":"2023-10-08T00:10:54.091651+0000","last_active":"2023-10-08T00:10:54.093726+0000","last_peered":"2023-10-08T00:10:54.093726+0000","last_clean":"2023-10-08T00:10:54.093726+0000","last_became_active":"2023-10-08T00:10:54.091234+0000","last_became_peered":"2023-10-08T00:10:54.091234+0000","last_unstale":"2023-10-08T00:10:54.093726+0000","last_undegraded":"2023-10-08T00:10:54.093726+0000","last_fullsized":"2023-10-08T00:10:54.093726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:06:38.007522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.083596+0000","last_change":"2023-10-08T00:10:53.080285+0000","last_active":"2023-10-08T00:10:54.083596+0000","last_peered":"2023-10-08T00:10:54.083596+0000","last_clean":"2023-10-08T00:10:54.083596+0000","last_became_active":"2023-10-08T00:10:53.079998+0000","last_became_peered":"2023-10-08T00:10:53.079998+0000","last_unstale":"2023-10-08T00:10:54.083596+0000","last_undegraded":"2023-10-08T00:10:54.083596+0000","last_fullsized":"2023-10-08T00:10:54.083596+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:16:40.636514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.097519+0000","last_change":"2023-10-08T00:10:54.094819+0000","last_active":"2023-10-08T00:10:54.097519+0000","last_peered":"2023-10-08T00:10:54.097519+0000","last_clean":"2023-10-08T00:10:54.097519+0000","last_became_active":"2023-10-08T00:10:54.094572+0000","last_became_peered":"2023-10-08T00:10:54.094572+0000","last_unstale":"2023-10-08T00:10:54.097519+0000","last_undegraded":"2023-10-08T00:10:54.097519+0000","last_fullsized":"2023-10-08T00:10:54.097519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:21:24.202186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.084469+0000","last_change":"2023-10-08T00:10:53.083645+0000","last_active":"2023-10-08T00:10:54.084469+0000","last_peered":"2023-10-08T00:10:54.084469+0000","last_clean":"2023-10-08T00:10:54.084469+0000","last_became_active":"2023-10-08T00:10:53.083477+0000","last_became_peered":"2023-10-08T00:10:53.083477+0000","last_unstale":"2023-10-08T00:10:54.084469+0000","last_undegraded":"2023-10-08T00:10:54.084469+0000","last_fullsized":"2023-10-08T00:10:54.084469+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:42:34.326823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":163,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.086096+0000","last_change":"2023-10-08T00:10:43.728490+0000","last_active":"2023-10-08T00:10:54.086096+0000","last_peered":"2023-10-08T00:10:54.086096+0000","last_clean":"2023-10-08T00:10:54.086096+0000","last_became_active":"2023-10-08T00:10:43.728341+0000","last_became_peered":"2023-10-08T00:10:43.728341+0000","last_unstale":"2023-10-08T00:10:54.086096+0000","last_undegraded":"2023-10-08T00:10:54.086096+0000","last_fullsized":"2023-10-08T00:10:54.086096+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_clean_scrub_stamp":"2023-10-08T00:10:42.716096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:06:31.035293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.101392+0000","last_change":"2023-10-08T00:10:54.100890+0000","last_active":"2023-10-08T00:10:54.101392+0000","last_peered":"2023-10-08T00:10:54.101392+0000","last_clean":"2023-10-08T00:10:54.101392+0000","last_became_active":"2023-10-08T00:10:54.100530+0000","last_became_peered":"2023-10-08T00:10:54.100530+0000","last_unstale":"2023-10-08T00:10:54.101392+0000","last_undegraded":"2023-10-08T00:10:54.101392+0000","last_fullsized":"2023-10-08T00:10:54.101392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:20:52.135952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.086851+0000","last_change":"2023-10-08T00:10:53.079813+0000","last_active":"2023-10-08T00:10:54.086851+0000","last_peered":"2023-10-08T00:10:54.086851+0000","last_clean":"2023-10-08T00:10:54.086851+0000","last_became_active":"2023-10-08T00:10:53.079642+0000","last_became_peered":"2023-10-08T00:10:53.079642+0000","last_unstale":"2023-10-08T00:10:54.086851+0000","last_undegraded":"2023-10-08T00:10:54.086851+0000","last_fullsized":"2023-10-08T00:10:54.086851+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:40:11.379045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.086769+0000","last_change":"2023-10-08T00:10:53.082674+0000","last_active":"2023-10-08T00:10:54.086769+0000","last_peered":"2023-10-08T00:10:54.086769+0000","last_clean":"2023-10-08T00:10:54.086769+0000","last_became_active":"2023-10-08T00:10:53.082215+0000","last_became_peered":"2023-10-08T00:10:53.082215+0000","last_unstale":"2023-10-08T00:10:54.086769+0000","last_undegraded":"2023-10-08T00:10:54.086769+0000","last_fullsized":"2023-10-08T00:10:54.086769+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:25:02.811238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.101340+0000","last_change":"2023-10-08T00:10:54.100477+0000","last_active":"2023-10-08T00:10:54.101340+0000","last_peered":"2023-10-08T00:10:54.101340+0000","last_clean":"2023-10-08T00:10:54.101340+0000","last_became_active":"2023-10-08T00:10:54.100039+0000","last_became_peered":"2023-10-08T00:10:54.100039+0000","last_unstale":"2023-10-08T00:10:54.101340+0000","last_undegraded":"2023-10-08T00:10:54.101340+0000","last_fullsized":"2023-10-08T00:10:54.101340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:54:13.221147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T00:10:55.085092+0000","last_change":"2023-10-08T00:10:53.083853+0000","last_active":"2023-10-08T00:10:55.085092+0000","last_peered":"2023-10-08T00:10:55.085092+0000","last_clean":"2023-10-08T00:10:55.085092+0000","last_became_active":"2023-10-08T00:10:53.083635+0000","last_became_peered":"2023-10-08T00:10:53.083635+0000","last_unstale":"2023-10-08T00:10:55.085092+0000","last_undegraded":"2023-10-08T00:10:55.085092+0000","last_fullsized":"2023-10-08T00:10:55.085092+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:46:39.872883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.103289+0000","last_change":"2023-10-08T00:10:54.101690+0000","last_active":"2023-10-08T00:10:54.103289+0000","last_peered":"2023-10-08T00:10:54.103289+0000","last_clean":"2023-10-08T00:10:54.103289+0000","last_became_active":"2023-10-08T00:10:54.101460+0000","last_became_peered":"2023-10-08T00:10:54.101460+0000","last_unstale":"2023-10-08T00:10:54.103289+0000","last_undegraded":"2023-10-08T00:10:54.103289+0000","last_fullsized":"2023-10-08T00:10:54.103289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:20:26.989181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.099336+0000","last_change":"2023-10-08T00:10:53.089306+0000","last_active":"2023-10-08T00:10:54.099336+0000","last_peered":"2023-10-08T00:10:54.099336+0000","last_clean":"2023-10-08T00:10:54.099336+0000","last_became_active":"2023-10-08T00:10:53.089108+0000","last_became_peered":"2023-10-08T00:10:53.089108+0000","last_unstale":"2023-10-08T00:10:54.099336+0000","last_undegraded":"2023-10-08T00:10:54.099336+0000","last_fullsized":"2023-10-08T00:10:54.099336+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:49:50.543614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.091606+0000","last_change":"2023-10-08T00:10:54.091498+0000","last_active":"2023-10-08T00:10:54.091606+0000","last_peered":"2023-10-08T00:10:54.091606+0000","last_clean":"2023-10-08T00:10:54.091606+0000","last_became_active":"2023-10-08T00:10:54.090751+0000","last_became_peered":"2023-10-08T00:10:54.090751+0000","last_unstale":"2023-10-08T00:10:54.091606+0000","last_undegraded":"2023-10-08T00:10:54.091606+0000","last_fullsized":"2023-10-08T00:10:54.091606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:52:39.600888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.088854+0000","last_change":"2023-10-08T00:10:53.083078+0000","last_active":"2023-10-08T00:10:54.088854+0000","last_peered":"2023-10-08T00:10:54.088854+0000","last_clean":"2023-10-08T00:10:54.088854+0000","last_became_active":"2023-10-08T00:10:53.082687+0000","last_became_peered":"2023-10-08T00:10:53.082687+0000","last_unstale":"2023-10-08T00:10:54.088854+0000","last_undegraded":"2023-10-08T00:10:54.088854+0000","last_fullsized":"2023-10-08T00:10:54.088854+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:46:52.125296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.090972+0000","last_change":"2023-10-08T00:10:54.090572+0000","last_active":"2023-10-08T00:10:54.090972+0000","last_peered":"2023-10-08T00:10:54.090972+0000","last_clean":"2023-10-08T00:10:54.090972+0000","last_became_active":"2023-10-08T00:10:54.090352+0000","last_became_peered":"2023-10-08T00:10:54.090352+0000","last_unstale":"2023-10-08T00:10:54.090972+0000","last_undegraded":"2023-10-08T00:10:54.090972+0000","last_fullsized":"2023-10-08T00:10:54.090972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:52:09.003538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.085955+0000","last_change":"2023-10-08T00:10:53.084909+0000","last_active":"2023-10-08T00:10:54.085955+0000","last_peered":"2023-10-08T00:10:54.085955+0000","last_clean":"2023-10-08T00:10:54.085955+0000","last_became_active":"2023-10-08T00:10:53.084676+0000","last_became_peered":"2023-10-08T00:10:53.084676+0000","last_unstale":"2023-10-08T00:10:54.085955+0000","last_undegraded":"2023-10-08T00:10:54.085955+0000","last_fullsized":"2023-10-08T00:10:54.085955+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:08:39.785742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.103202+0000","last_change":"2023-10-08T00:10:54.098822+0000","last_active":"2023-10-08T00:10:54.103202+0000","last_peered":"2023-10-08T00:10:54.103202+0000","last_clean":"2023-10-08T00:10:54.103202+0000","last_became_active":"2023-10-08T00:10:54.098426+0000","last_became_peered":"2023-10-08T00:10:54.098426+0000","last_unstale":"2023-10-08T00:10:54.103202+0000","last_undegraded":"2023-10-08T00:10:54.103202+0000","last_fullsized":"2023-10-08T00:10:54.103202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:44:19.725645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.096443+0000","last_change":"2023-10-08T00:10:53.081675+0000","last_active":"2023-10-08T00:10:54.096443+0000","last_peered":"2023-10-08T00:10:54.096443+0000","last_clean":"2023-10-08T00:10:54.096443+0000","last_became_active":"2023-10-08T00:10:53.081494+0000","last_became_peered":"2023-10-08T00:10:53.081494+0000","last_unstale":"2023-10-08T00:10:54.096443+0000","last_undegraded":"2023-10-08T00:10:54.096443+0000","last_fullsized":"2023-10-08T00:10:54.096443+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:08:25.404396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.103000+0000","last_change":"2023-10-08T00:10:54.102951+0000","last_active":"2023-10-08T00:10:54.103000+0000","last_peered":"2023-10-08T00:10:54.103000+0000","last_clean":"2023-10-08T00:10:54.103000+0000","last_became_active":"2023-10-08T00:10:54.102742+0000","last_became_peered":"2023-10-08T00:10:54.102742+0000","last_unstale":"2023-10-08T00:10:54.103000+0000","last_undegraded":"2023-10-08T00:10:54.103000+0000","last_fullsized":"2023-10-08T00:10:54.103000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:40:12.384542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.100607+0000","last_change":"2023-10-08T00:10:53.074449+0000","last_active":"2023-10-08T00:10:54.100607+0000","last_peered":"2023-10-08T00:10:54.100607+0000","last_clean":"2023-10-08T00:10:54.100607+0000","last_became_active":"2023-10-08T00:10:53.074207+0000","last_became_peered":"2023-10-08T00:10:53.074207+0000","last_unstale":"2023-10-08T00:10:54.100607+0000","last_undegraded":"2023-10-08T00:10:54.100607+0000","last_fullsized":"2023-10-08T00:10:54.100607+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:46:52.125296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.093672+0000","last_change":"2023-10-08T00:10:54.093203+0000","last_active":"2023-10-08T00:10:54.093672+0000","last_peered":"2023-10-08T00:10:54.093672+0000","last_clean":"2023-10-08T00:10:54.093672+0000","last_became_active":"2023-10-08T00:10:54.093007+0000","last_became_peered":"2023-10-08T00:10:54.093007+0000","last_unstale":"2023-10-08T00:10:54.093672+0000","last_undegraded":"2023-10-08T00:10:54.093672+0000","last_fullsized":"2023-10-08T00:10:54.093672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:30:15.548694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.099362+0000","last_change":"2023-10-08T00:10:53.085980+0000","last_active":"2023-10-08T00:10:54.099362+0000","last_peered":"2023-10-08T00:10:54.099362+0000","last_clean":"2023-10-08T00:10:54.099362+0000","last_became_active":"2023-10-08T00:10:53.085796+0000","last_became_peered":"2023-10-08T00:10:53.085796+0000","last_unstale":"2023-10-08T00:10:54.099362+0000","last_undegraded":"2023-10-08T00:10:54.099362+0000","last_fullsized":"2023-10-08T00:10:54.099362+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:16:40.636514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.097850+0000","last_change":"2023-10-08T00:10:54.097790+0000","last_active":"2023-10-08T00:10:54.097850+0000","last_peered":"2023-10-08T00:10:54.097850+0000","last_clean":"2023-10-08T00:10:54.097850+0000","last_became_active":"2023-10-08T00:10:54.096497+0000","last_became_peered":"2023-10-08T00:10:54.096497+0000","last_unstale":"2023-10-08T00:10:54.097850+0000","last_undegraded":"2023-10-08T00:10:54.097850+0000","last_fullsized":"2023-10-08T00:10:54.097850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:05:53.331248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.097812+0000","last_change":"2023-10-08T00:10:53.078098+0000","last_active":"2023-10-08T00:10:54.097812+0000","last_peered":"2023-10-08T00:10:54.097812+0000","last_clean":"2023-10-08T00:10:54.097812+0000","last_became_active":"2023-10-08T00:10:53.077855+0000","last_became_peered":"2023-10-08T00:10:53.077855+0000","last_unstale":"2023-10-08T00:10:54.097812+0000","last_undegraded":"2023-10-08T00:10:54.097812+0000","last_fullsized":"2023-10-08T00:10:54.097812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:10:42.328030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.093527+0000","last_change":"2023-10-08T00:10:54.092335+0000","last_active":"2023-10-08T00:10:54.093527+0000","last_peered":"2023-10-08T00:10:54.093527+0000","last_clean":"2023-10-08T00:10:54.093527+0000","last_became_active":"2023-10-08T00:10:54.091475+0000","last_became_peered":"2023-10-08T00:10:54.091475+0000","last_unstale":"2023-10-08T00:10:54.093527+0000","last_undegraded":"2023-10-08T00:10:54.093527+0000","last_fullsized":"2023-10-08T00:10:54.093527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:30:15.548694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.098946+0000","last_change":"2023-10-08T00:10:53.079388+0000","last_active":"2023-10-08T00:10:54.098946+0000","last_peered":"2023-10-08T00:10:54.098946+0000","last_clean":"2023-10-08T00:10:54.098946+0000","last_became_active":"2023-10-08T00:10:53.079152+0000","last_became_peered":"2023-10-08T00:10:53.079152+0000","last_unstale":"2023-10-08T00:10:54.098946+0000","last_undegraded":"2023-10-08T00:10:54.098946+0000","last_fullsized":"2023-10-08T00:10:54.098946+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:08:25.404396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.090252+0000","last_change":"2023-10-08T00:10:54.088605+0000","last_active":"2023-10-08T00:10:54.090252+0000","last_peered":"2023-10-08T00:10:54.090252+0000","last_clean":"2023-10-08T00:10:54.090252+0000","last_became_active":"2023-10-08T00:10:54.088396+0000","last_became_peered":"2023-10-08T00:10:54.088396+0000","last_unstale":"2023-10-08T00:10:54.090252+0000","last_undegraded":"2023-10-08T00:10:54.090252+0000","last_fullsized":"2023-10-08T00:10:54.090252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:54:13.221147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.100688+0000","last_change":"2023-10-08T00:10:54.100120+0000","last_active":"2023-10-08T00:10:54.100688+0000","last_peered":"2023-10-08T00:10:54.100688+0000","last_clean":"2023-10-08T00:10:54.100688+0000","last_became_active":"2023-10-08T00:10:54.099910+0000","last_became_peered":"2023-10-08T00:10:54.099910+0000","last_unstale":"2023-10-08T00:10:54.100688+0000","last_undegraded":"2023-10-08T00:10:54.100688+0000","last_fullsized":"2023-10-08T00:10:54.100688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:11:05.861540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.094221+0000","last_change":"2023-10-08T00:10:54.093990+0000","last_active":"2023-10-08T00:10:54.094221+0000","last_peered":"2023-10-08T00:10:54.094221+0000","last_clean":"2023-10-08T00:10:54.094221+0000","last_became_active":"2023-10-08T00:10:54.093268+0000","last_became_peered":"2023-10-08T00:10:54.093268+0000","last_unstale":"2023-10-08T00:10:54.094221+0000","last_undegraded":"2023-10-08T00:10:54.094221+0000","last_fullsized":"2023-10-08T00:10:54.094221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:49:51.549111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.096510+0000","last_change":"2023-10-08T00:10:54.096449+0000","last_active":"2023-10-08T00:10:54.096510+0000","last_peered":"2023-10-08T00:10:54.096510+0000","last_clean":"2023-10-08T00:10:54.096510+0000","last_became_active":"2023-10-08T00:10:54.096216+0000","last_became_peered":"2023-10-08T00:10:54.096216+0000","last_unstale":"2023-10-08T00:10:54.096510+0000","last_undegraded":"2023-10-08T00:10:54.096510+0000","last_fullsized":"2023-10-08T00:10:54.096510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:11:41.906007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.093706+0000","last_change":"2023-10-08T00:10:54.091529+0000","last_active":"2023-10-08T00:10:54.093706+0000","last_peered":"2023-10-08T00:10:54.093706+0000","last_clean":"2023-10-08T00:10:54.093706+0000","last_became_active":"2023-10-08T00:10:54.090949+0000","last_became_peered":"2023-10-08T00:10:54.090949+0000","last_unstale":"2023-10-08T00:10:54.093706+0000","last_undegraded":"2023-10-08T00:10:54.093706+0000","last_fullsized":"2023-10-08T00:10:54.093706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:52:44.075366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.101359+0000","last_change":"2023-10-08T00:10:54.100755+0000","last_active":"2023-10-08T00:10:54.101359+0000","last_peered":"2023-10-08T00:10:54.101359+0000","last_clean":"2023-10-08T00:10:54.101359+0000","last_became_active":"2023-10-08T00:10:54.100206+0000","last_became_peered":"2023-10-08T00:10:54.100206+0000","last_unstale":"2023-10-08T00:10:54.101359+0000","last_undegraded":"2023-10-08T00:10:54.101359+0000","last_fullsized":"2023-10-08T00:10:54.101359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:03:42.084791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.101376+0000","last_change":"2023-10-08T00:10:54.100759+0000","last_active":"2023-10-08T00:10:54.101376+0000","last_peered":"2023-10-08T00:10:54.101376+0000","last_clean":"2023-10-08T00:10:54.101376+0000","last_became_active":"2023-10-08T00:10:54.100365+0000","last_became_peered":"2023-10-08T00:10:54.100365+0000","last_unstale":"2023-10-08T00:10:54.101376+0000","last_undegraded":"2023-10-08T00:10:54.101376+0000","last_fullsized":"2023-10-08T00:10:54.101376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:40:12.384542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.087188+0000","last_change":"2023-10-08T00:10:54.085760+0000","last_active":"2023-10-08T00:10:54.087188+0000","last_peered":"2023-10-08T00:10:54.087188+0000","last_clean":"2023-10-08T00:10:54.087188+0000","last_became_active":"2023-10-08T00:10:54.085544+0000","last_became_peered":"2023-10-08T00:10:54.085544+0000","last_unstale":"2023-10-08T00:10:54.087188+0000","last_undegraded":"2023-10-08T00:10:54.087188+0000","last_fullsized":"2023-10-08T00:10:54.087188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:53:06.721490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.099558+0000","last_change":"2023-10-08T00:10:54.095621+0000","last_active":"2023-10-08T00:10:54.099558+0000","last_peered":"2023-10-08T00:10:54.099558+0000","last_clean":"2023-10-08T00:10:54.099558+0000","last_became_active":"2023-10-08T00:10:54.095240+0000","last_became_peered":"2023-10-08T00:10:54.095240+0000","last_unstale":"2023-10-08T00:10:54.099558+0000","last_undegraded":"2023-10-08T00:10:54.099558+0000","last_fullsized":"2023-10-08T00:10:54.099558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:22:42.484817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.101130+0000","last_change":"2023-10-08T00:10:54.101076+0000","last_active":"2023-10-08T00:10:54.101130+0000","last_peered":"2023-10-08T00:10:54.101130+0000","last_clean":"2023-10-08T00:10:54.101130+0000","last_became_active":"2023-10-08T00:10:54.100508+0000","last_became_peered":"2023-10-08T00:10:54.100508+0000","last_unstale":"2023-10-08T00:10:54.101130+0000","last_undegraded":"2023-10-08T00:10:54.101130+0000","last_fullsized":"2023-10-08T00:10:54.101130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:18:09.014594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.100641+0000","last_change":"2023-10-08T00:10:54.100508+0000","last_active":"2023-10-08T00:10:54.100641+0000","last_peered":"2023-10-08T00:10:54.100641+0000","last_clean":"2023-10-08T00:10:54.100641+0000","last_became_active":"2023-10-08T00:10:54.100272+0000","last_became_peered":"2023-10-08T00:10:54.100272+0000","last_unstale":"2023-10-08T00:10:54.100641+0000","last_undegraded":"2023-10-08T00:10:54.100641+0000","last_fullsized":"2023-10-08T00:10:54.100641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:52:09.003538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.095807+0000","last_change":"2023-10-08T00:10:54.095683+0000","last_active":"2023-10-08T00:10:54.095807+0000","last_peered":"2023-10-08T00:10:54.095807+0000","last_clean":"2023-10-08T00:10:54.095807+0000","last_became_active":"2023-10-08T00:10:54.095474+0000","last_became_peered":"2023-10-08T00:10:54.095474+0000","last_unstale":"2023-10-08T00:10:54.095807+0000","last_undegraded":"2023-10-08T00:10:54.095807+0000","last_fullsized":"2023-10-08T00:10:54.095807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:11:05.861540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.092456+0000","last_change":"2023-10-08T00:10:54.092383+0000","last_active":"2023-10-08T00:10:54.092456+0000","last_peered":"2023-10-08T00:10:54.092456+0000","last_clean":"2023-10-08T00:10:54.092456+0000","last_became_active":"2023-10-08T00:10:54.092098+0000","last_became_peered":"2023-10-08T00:10:54.092098+0000","last_unstale":"2023-10-08T00:10:54.092456+0000","last_undegraded":"2023-10-08T00:10:54.092456+0000","last_fullsized":"2023-10-08T00:10:54.092456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:54:13.221147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.098417+0000","last_change":"2023-10-08T00:10:54.097569+0000","last_active":"2023-10-08T00:10:54.098417+0000","last_peered":"2023-10-08T00:10:54.098417+0000","last_clean":"2023-10-08T00:10:54.098417+0000","last_became_active":"2023-10-08T00:10:54.096963+0000","last_became_peered":"2023-10-08T00:10:54.096963+0000","last_unstale":"2023-10-08T00:10:54.098417+0000","last_undegraded":"2023-10-08T00:10:54.098417+0000","last_fullsized":"2023-10-08T00:10:54.098417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:52:09.003538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":68,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929382,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":68,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929506,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":645,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194667,"statfs":{"total":200000000,"available":199339068,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.20499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.318}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.375}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":35,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":68,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929450,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.193}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.251}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":70,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195242,"statfs":{"total":200000000,"available":199927970,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.255}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199928952,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.255}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":70,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195241,"statfs":{"total":200000000,"available":199927326,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":645,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194667,"statfs":{"total":200000000,"available":199339046,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.375}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]}]}],"pool_statfs":[]}} 2023-10-08T00:11:02.963 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T00:11:02.963 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-08T00:11:02.963 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-08T00:11:02.963 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-08T00:11:02.965 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-10-08T00:11:03.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.110+0000 7f8aed94a700 1 Processor -- start 2023-10-08T00:11:03.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.110+0000 7f8aed94a700 1 -- start start 2023-10-08T00:11:03.114 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.111+0000 7f8aed94a700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8ae8152010 0x7f8ae815b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:03.114 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.111+0000 7f8aed94a700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8ae8158e50 0x7f8ae80cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:03.114 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.111+0000 7f8aed94a700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ae806c010 con 0x7f8ae81523e0 2023-10-08T00:11:03.114 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.111+0000 7f8aed94a700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ae806bd10 con 0x7f8ae8158e50 2023-10-08T00:11:03.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.111+0000 7f8aed94a700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ae815c060 con 0x7f8ae8152010 2023-10-08T00:11:03.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.111+0000 7f8ae77fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8ae8158e50 0x7f8ae80cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:03.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.111+0000 7f8ae77fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8ae8158e50 0x7f8ae80cdf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49572/0 (socket says 172.21.15.99:49572) 2023-10-08T00:11:03.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.111+0000 7f8ae77fe700 1 -- 172.21.15.99:0/4045037928 learned_addr learned my addr 172.21.15.99:0/4045037928 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:03.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.111+0000 7f8ade7fc700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f8ae81523e0 0x7f8ae806c4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50772/0 (socket says 172.21.15.99:50772) 2023-10-08T00:11:03.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.111+0000 7f8ae6ffd700 1 --2- 172.21.15.99:0/4045037928 >> v2:172.21.15.191:3300/0 conn(0x7f8ae8152010 0x7f8ae815b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:03.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.112+0000 7f8ae77fe700 1 -- 172.21.15.99:0/4045037928 >> v2:172.21.15.191:3300/0 conn(0x7f8ae8152010 msgr2=0x7f8ae815b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:03.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.112+0000 7f8ae77fe700 1 --2- 172.21.15.99:0/4045037928 >> v2:172.21.15.191:3300/0 conn(0x7f8ae8152010 0x7f8ae815b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:03.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.112+0000 7f8ae77fe700 1 -- 172.21.15.99:0/4045037928 >> v1:172.21.15.99:6789/0 conn(0x7f8ae81523e0 legacy=0x7f8ae806c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:03.116 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.112+0000 7f8ae77fe700 1 -- 172.21.15.99:0/4045037928 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae815c340 con 0x7f8ae8158e50 2023-10-08T00:11:03.116 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.112+0000 7f8ae67fc700 1 -- 172.21.15.99:0/4045037928 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 607915214 0 0) 0x7f8ae806c010 con 0x7f8ae81523e0 2023-10-08T00:11:03.116 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.112+0000 7f8ae77fe700 1 --2- 172.21.15.99:0/4045037928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ae8158e50 0x7f8ae80cdf30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8ae80cc680 tx=0x7f8ae0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86cd3b71fcc0dac5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:03.116 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.112+0000 7f8ae67fc700 1 -- 172.21.15.99:0/4045037928 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae000e070 con 0x7f8ae8158e50 2023-10-08T00:11:03.116 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.112+0000 7f8ae67fc700 1 -- 172.21.15.99:0/4045037928 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ae000ba20 con 0x7f8ae8158e50 2023-10-08T00:11:03.116 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.112+0000 7f8ae67fc700 1 -- 172.21.15.99:0/4045037928 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae00092e0 con 0x7f8ae8158e50 2023-10-08T00:11:03.116 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.112+0000 7f8aed94a700 1 -- 172.21.15.99:0/4045037928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ae8158e50 msgr2=0x7f8ae80cdf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:03.116 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.112+0000 7f8aed94a700 1 --2- 172.21.15.99:0/4045037928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ae8158e50 0x7f8ae80cdf30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8ae80cc680 tx=0x7f8ae0009840 comp rx=0 tx=0).stop 2023-10-08T00:11:03.116 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.112+0000 7f8aed94a700 1 -- 172.21.15.99:0/4045037928 shutdown_connections 2023-10-08T00:11:03.116 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.112+0000 7f8aed94a700 1 --2- 172.21.15.99:0/4045037928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ae8158e50 0x7f8ae80cdf30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:03.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.112+0000 7f8aed94a700 1 --2- 172.21.15.99:0/4045037928 >> v2:172.21.15.191:3300/0 conn(0x7f8ae8152010 0x7f8ae815b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:03.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.112+0000 7f8aed94a700 1 -- 172.21.15.99:0/4045037928 >> 172.21.15.99:0/4045037928 conn(0x7f8ae8069870 msgr2=0x7f8ae8069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:03.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.112+0000 7f8aed94a700 1 -- 172.21.15.99:0/4045037928 shutdown_connections 2023-10-08T00:11:03.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.112+0000 7f8aed94a700 1 -- 172.21.15.99:0/4045037928 wait complete. 2023-10-08T00:11:03.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.114+0000 7f8aed94a700 1 Processor -- start 2023-10-08T00:11:03.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.114+0000 7f8aed94a700 1 -- start start 2023-10-08T00:11:03.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.114+0000 7f8aed94a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ae8152010 0x7f8ae814d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:03.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.114+0000 7f8aed94a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ae81523e0 0x7f8ae8068fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:03.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.114+0000 7f8aed94a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ae8158e50 0x7f8ae81493b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:03.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.114+0000 7f8aed94a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae814dc70 con 0x7f8ae8152010 2023-10-08T00:11:03.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.114+0000 7f8aed94a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae8148ba0 con 0x7f8ae81523e0 2023-10-08T00:11:03.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.114+0000 7f8aed94a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae8159c80 con 0x7f8ae8158e50 2023-10-08T00:11:03.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.114+0000 7f8ae6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ae8152010 0x7f8ae814d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:03.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.114+0000 7f8ae6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ae8152010 0x7f8ae814d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49586/0 (socket says 172.21.15.99:49586) 2023-10-08T00:11:03.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.114+0000 7f8ae6ffd700 1 -- 172.21.15.99:0/1215284788 learned_addr learned my addr 172.21.15.99:0/1215284788 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:03.121 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.115+0000 7f8ade7fc700 1 --2- 172.21.15.99:0/1215284788 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ae81523e0 0x7f8ae8068fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:03.121 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.115+0000 7f8ae77fe700 1 --2- 172.21.15.99:0/1215284788 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ae8158e50 0x7f8ae81493b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:03.121 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.115+0000 7f8ae6ffd700 1 -- 172.21.15.99:0/1215284788 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ae8158e50 msgr2=0x7f8ae81493b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:03.121 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.115+0000 7f8ae6ffd700 1 --2- 172.21.15.99:0/1215284788 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ae8158e50 0x7f8ae81493b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:03.121 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.115+0000 7f8ae6ffd700 1 -- 172.21.15.99:0/1215284788 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ae81523e0 msgr2=0x7f8ae8068fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:03.121 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.115+0000 7f8ae6ffd700 1 --2- 172.21.15.99:0/1215284788 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ae81523e0 0x7f8ae8068fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:03.121 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.115+0000 7f8ae6ffd700 1 -- 172.21.15.99:0/1215284788 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae0009ae0 con 0x7f8ae8152010 2023-10-08T00:11:03.121 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.115+0000 7f8ade7fc700 1 --2- 172.21.15.99:0/1215284788 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ae81523e0 0x7f8ae8068fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:11:03.121 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.115+0000 7f8ae6ffd700 1 --2- 172.21.15.99:0/1215284788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ae8152010 0x7f8ae814d570 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8ad800e930 tx=0x7f8ad80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:03.122 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.115+0000 7f8adf7fe700 1 -- 172.21.15.99:0/1215284788 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad800eed0 con 0x7f8ae8152010 2023-10-08T00:11:03.122 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.115+0000 7f8adf7fe700 1 -- 172.21.15.99:0/1215284788 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad800c850 con 0x7f8ae8152010 2023-10-08T00:11:03.122 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.115+0000 7f8adf7fe700 1 -- 172.21.15.99:0/1215284788 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad800cbf0 con 0x7f8ae8152010 2023-10-08T00:11:03.122 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.115+0000 7f8aed94a700 1 -- 172.21.15.99:0/1215284788 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ae8159350 con 0x7f8ae8152010 2023-10-08T00:11:03.122 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.115+0000 7f8aed94a700 1 -- 172.21.15.99:0/1215284788 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ae8159910 con 0x7f8ae8152010 2023-10-08T00:11:03.122 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.116+0000 7f8aed94a700 1 -- 172.21.15.99:0/1215284788 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ae8159f60 con 0x7f8ae8152010 2023-10-08T00:11:03.122 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.116+0000 7f8adf7fe700 1 -- 172.21.15.99:0/1215284788 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8ad8013470 con 0x7f8ae8152010 2023-10-08T00:11:03.122 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.117+0000 7f8adf7fe700 1 --2- 172.21.15.99:0/1215284788 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8ac8066f30 0x7f8ac80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:03.123 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.117+0000 7f8ade7fc700 1 --2- 172.21.15.99:0/1215284788 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8ac8066f30 0x7f8ac80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:03.123 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.117+0000 7f8adf7fe700 1 -- 172.21.15.99:0/1215284788 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8ad8055d50 con 0x7f8ae8152010 2023-10-08T00:11:03.123 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.117+0000 7f8ade7fc700 1 --2- 172.21.15.99:0/1215284788 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8ac8066f30 0x7f8ac80693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8ae806daa0 tx=0x7f8ad0007c90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:03.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.122+0000 7f8adf7fe700 1 -- 172.21.15.99:0/1215284788 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8ad8051990 con 0x7f8ae8152010 2023-10-08T00:11:03.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.356+0000 7f8aed94a700 1 -- 172.21.15.99:0/1215284788 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f8ae804c3c0 con 0x7f8ae8152010 2023-10-08T00:11:03.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.358+0000 7f8adf7fe700 1 -- 172.21.15.99:0/1215284788 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v4) v1 ==== 74+0+163704 (secure 0 0 0) 0x7f8ad8054fb0 con 0x7f8ae8152010 2023-10-08T00:11:03.361 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:11:03.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.361+0000 7f8aed94a700 1 -- 172.21.15.99:0/1215284788 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8ac8066f30 msgr2=0x7f8ac80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:03.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.361+0000 7f8aed94a700 1 --2- 172.21.15.99:0/1215284788 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8ac8066f30 0x7f8ac80693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8ae806daa0 tx=0x7f8ad0007c90 comp rx=0 tx=0).stop 2023-10-08T00:11:03.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.361+0000 7f8aed94a700 1 -- 172.21.15.99:0/1215284788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ae8152010 msgr2=0x7f8ae814d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:03.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.361+0000 7f8aed94a700 1 --2- 172.21.15.99:0/1215284788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ae8152010 0x7f8ae814d570 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8ad800e930 tx=0x7f8ad80091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:03.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.361+0000 7f8aed94a700 1 -- 172.21.15.99:0/1215284788 shutdown_connections 2023-10-08T00:11:03.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.361+0000 7f8aed94a700 1 --2- 172.21.15.99:0/1215284788 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8ac8066f30 0x7f8ac80693e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:03.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.361+0000 7f8aed94a700 1 --2- 172.21.15.99:0/1215284788 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ae8158e50 0x7f8ae81493b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:03.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.361+0000 7f8aed94a700 1 --2- 172.21.15.99:0/1215284788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ae8152010 0x7f8ae814d570 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:03.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.361+0000 7f8aed94a700 1 --2- 172.21.15.99:0/1215284788 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ae81523e0 0x7f8ae8068fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:03.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.361+0000 7f8aed94a700 1 -- 172.21.15.99:0/1215284788 >> 172.21.15.99:0/1215284788 conn(0x7f8ae8069870 msgr2=0x7f8ae8069c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:03.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.361+0000 7f8aed94a700 1 -- 172.21.15.99:0/1215284788 shutdown_connections 2023-10-08T00:11:03.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.361+0000 7f8aed94a700 1 -- 172.21.15.99:0/1215284788 wait complete. 2023-10-08T00:11:03.373 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":4,"active_gid":4099,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":85985},{"type":"v1","addr":"172.21.15.191:6833","nonce":85985}]},"active_addr":"172.21.15.191:6833/85985","active_change":"2023-10-08T00:10:42.013107+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4113,"name":"y","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.191:0","nonce":3979556806}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.191:0","nonce":3685998324}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.191:0","nonce":637868015}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.191:0","nonce":568495476}]}]} 2023-10-08T00:11:03.381 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-08T00:11:03.381 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-08T00:11:03.381 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:11:03.496 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.495+0000 7f7179662700 1 Processor -- start 2023-10-08T00:11:03.496 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.496+0000 7f7179662700 1 -- start start 2023-10-08T00:11:03.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.496+0000 7f7179662700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7174157870 0x7f717415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:03.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.496+0000 7f7179662700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f717415cbb0 con 0x7f717415a620 2023-10-08T00:11:03.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.496+0000 7f7179662700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f717415bfa0 con 0x7f7174157c40 2023-10-08T00:11:03.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.496+0000 7f7179662700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f717415c280 con 0x7f7174157870 2023-10-08T00:11:03.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.496+0000 7f7172ffd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7174157870 0x7f717415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:03.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.497+0000 7f7172ffd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7174157870 0x7f717415b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38564/0 (socket says 172.21.15.99:38564) 2023-10-08T00:11:03.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.497+0000 7f716a7fc700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f717415a620 0x7f717415d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:44994/0 (socket says 172.21.15.99:44994) 2023-10-08T00:11:03.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.497+0000 7f7172ffd700 1 -- 172.21.15.99:0/3758069691 learned_addr learned my addr 172.21.15.99:0/3758069691 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:03.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.497+0000 7f7172ffd700 1 -- 172.21.15.99:0/3758069691 >> v1:172.21.15.191:6789/0 conn(0x7f7174157c40 legacy=0x7f7174158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:03.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.497+0000 7f7172ffd700 1 -- 172.21.15.99:0/3758069691 >> v1:172.21.15.99:6789/0 conn(0x7f717415a620 legacy=0x7f717415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:03.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.497+0000 7f7172ffd700 1 -- 172.21.15.99:0/3758069691 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f717415c560 con 0x7f7174157870 2023-10-08T00:11:03.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.497+0000 7f71727fc700 1 -- 172.21.15.99:0/3758069691 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2375196522 0 0) 0x7f717415cbb0 con 0x7f717415a620 2023-10-08T00:11:03.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.497+0000 7f7172ffd700 1 --2- 172.21.15.99:0/3758069691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7174157870 0x7f717415b6a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f71640099b0 tx=0x7f7164009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a71e0822acdaba15 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:03.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.497+0000 7f71727fc700 1 -- 172.21.15.99:0/3758069691 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716400b860 con 0x7f7174157870 2023-10-08T00:11:03.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.497+0000 7f71727fc700 1 -- 172.21.15.99:0/3758069691 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f716400b9c0 con 0x7f7174157870 2023-10-08T00:11:03.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.497+0000 7f71727fc700 1 -- 172.21.15.99:0/3758069691 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716400bd60 con 0x7f7174157870 2023-10-08T00:11:03.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.497+0000 7f7179662700 1 -- 172.21.15.99:0/3758069691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7174157870 msgr2=0x7f717415b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:03.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.497+0000 7f7179662700 1 --2- 172.21.15.99:0/3758069691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7174157870 0x7f717415b6a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f71640099b0 tx=0x7f7164009250 comp rx=0 tx=0).stop 2023-10-08T00:11:03.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.498+0000 7f7179662700 1 -- 172.21.15.99:0/3758069691 shutdown_connections 2023-10-08T00:11:03.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.498+0000 7f7179662700 1 --2- 172.21.15.99:0/3758069691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7174157870 0x7f717415b6a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:03.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.498+0000 7f7179662700 1 -- 172.21.15.99:0/3758069691 >> 172.21.15.99:0/3758069691 conn(0x7f7174068ce0 msgr2=0x7f71740690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:03.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.498+0000 7f7179662700 1 -- 172.21.15.99:0/3758069691 shutdown_connections 2023-10-08T00:11:03.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.498+0000 7f7179662700 1 -- 172.21.15.99:0/3758069691 wait complete. 2023-10-08T00:11:03.504 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.499+0000 7f7179662700 1 Processor -- start 2023-10-08T00:11:03.504 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.499+0000 7f7179662700 1 -- start start 2023-10-08T00:11:03.504 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.500+0000 7f7179662700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7174157870 0x7f717414bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:03.504 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.500+0000 7f7179662700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7174157c40 0x7f71740b0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:03.504 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.500+0000 7f7179662700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f717415a620 0x7f71740b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:03.504 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.500+0000 7f7179662700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71740b18e0 con 0x7f7174157c40 2023-10-08T00:11:03.504 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.500+0000 7f7179662700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71741514c0 con 0x7f7174157870 2023-10-08T00:11:03.504 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.500+0000 7f7179662700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f717414c1b0 con 0x7f717415a620 2023-10-08T00:11:03.504 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.500+0000 7f716a7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7174157c40 0x7f71740b0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:03.505 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.500+0000 7f716a7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7174157c40 0x7f71740b0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38574/0 (socket says 172.21.15.99:38574) 2023-10-08T00:11:03.505 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.500+0000 7f716a7fc700 1 -- 172.21.15.99:0/832762853 learned_addr learned my addr 172.21.15.99:0/832762853 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:03.505 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.500+0000 7f7172ffd700 1 --2- 172.21.15.99:0/832762853 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7174157870 0x7f717414bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:03.505 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.500+0000 7f71737fe700 1 --2- 172.21.15.99:0/832762853 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f717415a620 0x7f71740b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:03.505 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.500+0000 7f716a7fc700 1 -- 172.21.15.99:0/832762853 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f717415a620 msgr2=0x7f71740b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:03.505 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.500+0000 7f716a7fc700 1 --2- 172.21.15.99:0/832762853 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f717415a620 0x7f71740b5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:03.505 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.500+0000 7f716a7fc700 1 -- 172.21.15.99:0/832762853 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7174157870 msgr2=0x7f717414bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:03.505 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.500+0000 7f716a7fc700 1 --2- 172.21.15.99:0/832762853 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7174157870 0x7f717414bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:03.505 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.500+0000 7f716a7fc700 1 -- 172.21.15.99:0/832762853 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71640094f0 con 0x7f7174157c40 2023-10-08T00:11:03.505 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.500+0000 7f716a7fc700 1 --2- 172.21.15.99:0/832762853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7174157c40 0x7f71740b0610 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f715c00c530 tx=0x7f715c007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:03.505 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.500+0000 7f716b7fe700 1 -- 172.21.15.99:0/832762853 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715c005a70 con 0x7f7174157c40 2023-10-08T00:11:03.505 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.500+0000 7f716b7fe700 1 -- 172.21.15.99:0/832762853 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f715c005bd0 con 0x7f7174157c40 2023-10-08T00:11:03.506 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.500+0000 7f7179662700 1 -- 172.21.15.99:0/832762853 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7174151780 con 0x7f7174157c40 2023-10-08T00:11:03.506 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.500+0000 7f716b7fe700 1 -- 172.21.15.99:0/832762853 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715c0070d0 con 0x7f7174157c40 2023-10-08T00:11:03.506 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.500+0000 7f7179662700 1 -- 172.21.15.99:0/832762853 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71740b10f0 con 0x7f7174157c40 2023-10-08T00:11:03.506 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.501+0000 7f7179662700 1 -- 172.21.15.99:0/832762853 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7174151910 con 0x7f7174157c40 2023-10-08T00:11:03.506 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.502+0000 7f716b7fe700 1 -- 172.21.15.99:0/832762853 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f715c007480 con 0x7f7174157c40 2023-10-08T00:11:03.506 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.502+0000 7f716b7fe700 1 --2- 172.21.15.99:0/832762853 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7154066f30 0x7f71540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:03.506 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.502+0000 7f716b7fe700 1 -- 172.21.15.99:0/832762853 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f715c083950 con 0x7f7174157c40 2023-10-08T00:11:03.506 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.502+0000 7f7172ffd700 1 --2- 172.21.15.99:0/832762853 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7154066f30 0x7f71540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:03.506 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.502+0000 7f7172ffd700 1 --2- 172.21.15.99:0/832762853 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7154066f30 0x7f71540693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f71640099b0 tx=0x7f7164017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:03.507 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.506+0000 7f716b7fe700 1 -- 172.21.15.99:0/832762853 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f715c050bc0 con 0x7f7174157c40 2023-10-08T00:11:03.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.688+0000 7f7179662700 1 -- 172.21.15.99:0/832762853 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f717404c3c0 con 0x7f7174157c40 2023-10-08T00:11:03.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.689+0000 7f716b7fe700 1 -- 172.21.15.99:0/832762853 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f715c017020 con 0x7f7174157c40 2023-10-08T00:11:03.691 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:11:03.691 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":20,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:10:55.080854+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:10:51.822157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T00:10:52.415130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:11:03.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.691+0000 7f7179662700 1 -- 172.21.15.99:0/832762853 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7154066f30 msgr2=0x7f71540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:03.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.691+0000 7f7179662700 1 --2- 172.21.15.99:0/832762853 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7154066f30 0x7f71540693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f71640099b0 tx=0x7f7164017000 comp rx=0 tx=0).stop 2023-10-08T00:11:03.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.691+0000 7f7179662700 1 -- 172.21.15.99:0/832762853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7174157c40 msgr2=0x7f71740b0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:03.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.691+0000 7f7179662700 1 --2- 172.21.15.99:0/832762853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7174157c40 0x7f71740b0610 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f715c00c530 tx=0x7f715c007f30 comp rx=0 tx=0).stop 2023-10-08T00:11:03.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.691+0000 7f7179662700 1 -- 172.21.15.99:0/832762853 shutdown_connections 2023-10-08T00:11:03.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.691+0000 7f7179662700 1 --2- 172.21.15.99:0/832762853 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7154066f30 0x7f71540693e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:03.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.691+0000 7f7179662700 1 --2- 172.21.15.99:0/832762853 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7174157870 0x7f717414bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:03.694 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.691+0000 7f7179662700 1 --2- 172.21.15.99:0/832762853 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f717415a620 0x7f71740b5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:03.694 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.691+0000 7f7179662700 1 --2- 172.21.15.99:0/832762853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7174157c40 0x7f71740b0610 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:03.694 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.691+0000 7f7179662700 1 -- 172.21.15.99:0/832762853 >> 172.21.15.99:0/832762853 conn(0x7f7174068ce0 msgr2=0x7f71740cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:03.694 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.691+0000 7f7179662700 1 -- 172.21.15.99:0/832762853 shutdown_connections 2023-10-08T00:11:03.694 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.691+0000 7f7179662700 1 -- 172.21.15.99:0/832762853 wait complete. 2023-10-08T00:11:03.704 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-08T00:11:03.705 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:11:03.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.816+0000 7fae48db8700 1 Processor -- start 2023-10-08T00:11:03.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.816+0000 7fae48db8700 1 -- start start 2023-10-08T00:11:03.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.817+0000 7fae48db8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae440afff0 0x7fae4415e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:03.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.817+0000 7fae48db8700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fae44070e10 0x7fae4406a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:03.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.817+0000 7fae48db8700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae440b03c0 con 0x7fae44069e20 2023-10-08T00:11:03.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.817+0000 7fae48db8700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae4406a870 con 0x7fae44070e10 2023-10-08T00:11:03.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.817+0000 7fae48db8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae4406ab50 con 0x7fae440afff0 2023-10-08T00:11:03.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.817+0000 7fae4259c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae440afff0 0x7fae4415e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:03.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.818+0000 7fae4259c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae440afff0 0x7fae4415e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.99:45496/0 (socket says 172.21.15.99:45496) 2023-10-08T00:11:03.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.818+0000 7fae4259c700 1 -- 172.21.15.99:0/1502857833 learned_addr learned my addr 172.21.15.99:0/1502857833 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:03.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.818+0000 7fae42d9d700 1 --2- 172.21.15.99:0/1502857833 >> v2:172.21.15.191:3300/0 conn(0x7fae44070e10 0x7fae4406a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:03.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.818+0000 7fae4259c700 1 -- 172.21.15.99:0/1502857833 >> v2:172.21.15.191:3300/0 conn(0x7fae44070e10 msgr2=0x7fae4406a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:03.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.818+0000 7fae4259c700 1 --2- 172.21.15.99:0/1502857833 >> v2:172.21.15.191:3300/0 conn(0x7fae44070e10 0x7fae4406a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:03.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.818+0000 7fae4259c700 1 -- 172.21.15.99:0/1502857833 >> v1:172.21.15.191:6789/0 conn(0x7fae44069e20 legacy=0x7fae44070700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:03.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.818+0000 7fae4259c700 1 -- 172.21.15.99:0/1502857833 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae4415da60 con 0x7fae440afff0 2023-10-08T00:11:03.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.818+0000 7fae4259c700 1 --2- 172.21.15.99:0/1502857833 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae440afff0 0x7fae4415e540 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fae340099b0 tx=0x7fae34009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5f4be48f621b7a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:03.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.818+0000 7fae41d9b700 1 -- 172.21.15.99:0/1502857833 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae34009ea0 con 0x7fae440afff0 2023-10-08T00:11:03.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.818+0000 7fae41d9b700 1 -- 172.21.15.99:0/1502857833 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae3400b860 con 0x7fae440afff0 2023-10-08T00:11:03.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.818+0000 7fae41d9b700 1 -- 172.21.15.99:0/1502857833 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3400bc00 con 0x7fae440afff0 2023-10-08T00:11:03.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.819+0000 7fae48db8700 1 -- 172.21.15.99:0/1502857833 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae440afff0 msgr2=0x7fae4415e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:03.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.819+0000 7fae48db8700 1 --2- 172.21.15.99:0/1502857833 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae440afff0 0x7fae4415e540 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fae340099b0 tx=0x7fae34009250 comp rx=0 tx=0).stop 2023-10-08T00:11:03.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.819+0000 7fae48db8700 1 -- 172.21.15.99:0/1502857833 shutdown_connections 2023-10-08T00:11:03.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.819+0000 7fae48db8700 1 --2- 172.21.15.99:0/1502857833 >> v2:172.21.15.191:3300/0 conn(0x7fae44070e10 0x7fae4406a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:03.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.819+0000 7fae48db8700 1 --2- 172.21.15.99:0/1502857833 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae440afff0 0x7fae4415e540 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:03.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.819+0000 7fae48db8700 1 -- 172.21.15.99:0/1502857833 >> 172.21.15.99:0/1502857833 conn(0x7fae440cda50 msgr2=0x7fae440cde50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:03.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.819+0000 7fae48db8700 1 -- 172.21.15.99:0/1502857833 shutdown_connections 2023-10-08T00:11:03.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.819+0000 7fae48db8700 1 -- 172.21.15.99:0/1502857833 wait complete. 2023-10-08T00:11:03.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.820+0000 7fae48db8700 1 Processor -- start 2023-10-08T00:11:03.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.820+0000 7fae48db8700 1 -- start start 2023-10-08T00:11:03.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.821+0000 7fae48db8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae44069e20 0x7fae440c3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:03.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.821+0000 7fae48db8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae44070e10 0x7fae440c6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:03.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.821+0000 7fae48db8700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fae440afff0 0x7fae440c7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:03.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.821+0000 7fae48db8700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae440b0900 con 0x7fae44069e20 2023-10-08T00:11:03.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.821+0000 7fae48db8700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae440b1480 con 0x7fae440afff0 2023-10-08T00:11:03.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.821+0000 7fae48db8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae440c3db0 con 0x7fae44070e10 2023-10-08T00:11:03.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.821+0000 7fae4259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae44069e20 0x7fae440c3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:03.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.821+0000 7fae4259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae44069e20 0x7fae440c3870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38584/0 (socket says 172.21.15.99:38584) 2023-10-08T00:11:03.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.821+0000 7fae4259c700 1 -- 172.21.15.99:0/3973872262 learned_addr learned my addr 172.21.15.99:0/3973872262 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:03.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.821+0000 7fae39d9b700 1 --2- 172.21.15.99:0/3973872262 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae44070e10 0x7fae440c6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:03.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.821+0000 7fae42d9d700 1 --2- 172.21.15.99:0/3973872262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fae440afff0 0x7fae440c7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:03.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.821+0000 7fae4259c700 1 -- 172.21.15.99:0/3973872262 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae44070e10 msgr2=0x7fae440c6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:03.829 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.821+0000 7fae4259c700 1 --2- 172.21.15.99:0/3973872262 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae44070e10 0x7fae440c6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:03.829 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.821+0000 7fae4259c700 1 -- 172.21.15.99:0/3973872262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fae440afff0 msgr2=0x7fae440c7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:03.829 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.821+0000 7fae4259c700 1 --2- 172.21.15.99:0/3973872262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fae440afff0 0x7fae440c7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:03.829 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.821+0000 7fae4259c700 1 -- 172.21.15.99:0/3973872262 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae340094f0 con 0x7fae44069e20 2023-10-08T00:11:03.829 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.821+0000 7fae39d9b700 1 --2- 172.21.15.99:0/3973872262 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae44070e10 0x7fae440c6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:11:03.829 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.821+0000 7fae42d9d700 1 --2- 172.21.15.99:0/3973872262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fae440afff0 0x7fae440c7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:11:03.829 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.821+0000 7fae4259c700 1 --2- 172.21.15.99:0/3973872262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae44069e20 0x7fae440c3870 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fae34006fd0 tx=0x7fae34017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:03.829 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.821+0000 7fae3affd700 1 -- 172.21.15.99:0/3973872262 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3400bbe0 con 0x7fae44069e20 2023-10-08T00:11:03.829 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.821+0000 7fae3affd700 1 -- 172.21.15.99:0/3973872262 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae34011420 con 0x7fae44069e20 2023-10-08T00:11:03.829 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.821+0000 7fae48db8700 1 -- 172.21.15.99:0/3973872262 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae440b0b50 con 0x7fae44069e20 2023-10-08T00:11:03.829 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.821+0000 7fae3affd700 1 -- 172.21.15.99:0/3973872262 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae340117c0 con 0x7fae44069e20 2023-10-08T00:11:03.829 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.822+0000 7fae48db8700 1 -- 172.21.15.99:0/3973872262 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae440b1110 con 0x7fae44069e20 2023-10-08T00:11:03.830 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.822+0000 7fae48db8700 1 -- 172.21.15.99:0/3973872262 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae4404c3c0 con 0x7fae44069e20 2023-10-08T00:11:03.830 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.823+0000 7fae3affd700 1 -- 172.21.15.99:0/3973872262 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fae34011bf0 con 0x7fae44069e20 2023-10-08T00:11:03.830 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.823+0000 7fae3affd700 1 --2- 172.21.15.99:0/3973872262 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fae24066f30 0x7fae240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:03.830 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.823+0000 7fae3affd700 1 -- 172.21.15.99:0/3973872262 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fae34085a90 con 0x7fae44069e20 2023-10-08T00:11:03.830 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.823+0000 7fae39d9b700 1 --2- 172.21.15.99:0/3973872262 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fae24066f30 0x7fae240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:03.830 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.824+0000 7fae39d9b700 1 --2- 172.21.15.99:0/3973872262 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fae24066f30 0x7fae240693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fae2c007fd0 tx=0x7fae2c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:03.830 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:03.827+0000 7fae3affd700 1 -- 172.21.15.99:0/3973872262 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fae34052d80 con 0x7fae44069e20 2023-10-08T00:11:04.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.012+0000 7fae48db8700 1 -- 172.21.15.99:0/3973872262 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fae4406e870 con 0x7fae44069e20 2023-10-08T00:11:04.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.013+0000 7fae3affd700 1 -- 172.21.15.99:0/3973872262 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15350 (secure 0 0 0) 0x7fae34015090 con 0x7fae44069e20 2023-10-08T00:11:04.015 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:11:04.015 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":20,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:10:55.080854+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:10:51.822157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T00:10:52.415130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:11:04.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.015+0000 7fae48db8700 1 -- 172.21.15.99:0/3973872262 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fae24066f30 msgr2=0x7fae240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.015+0000 7fae48db8700 1 --2- 172.21.15.99:0/3973872262 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fae24066f30 0x7fae240693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fae2c007fd0 tx=0x7fae2c009040 comp rx=0 tx=0).stop 2023-10-08T00:11:04.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.015+0000 7fae48db8700 1 -- 172.21.15.99:0/3973872262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae44069e20 msgr2=0x7fae440c3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.015+0000 7fae48db8700 1 --2- 172.21.15.99:0/3973872262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae44069e20 0x7fae440c3870 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fae34006fd0 tx=0x7fae34017000 comp rx=0 tx=0).stop 2023-10-08T00:11:04.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.015+0000 7fae48db8700 1 -- 172.21.15.99:0/3973872262 shutdown_connections 2023-10-08T00:11:04.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.015+0000 7fae48db8700 1 --2- 172.21.15.99:0/3973872262 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fae24066f30 0x7fae240693e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.015+0000 7fae48db8700 1 --2- 172.21.15.99:0/3973872262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fae440afff0 0x7fae440c7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.015+0000 7fae48db8700 1 --2- 172.21.15.99:0/3973872262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae44069e20 0x7fae440c3870 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.015+0000 7fae48db8700 1 --2- 172.21.15.99:0/3973872262 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae44070e10 0x7fae440c6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.015+0000 7fae48db8700 1 -- 172.21.15.99:0/3973872262 >> 172.21.15.99:0/3973872262 conn(0x7fae440cda50 msgr2=0x7fae441590c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:04.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.015+0000 7fae48db8700 1 -- 172.21.15.99:0/3973872262 shutdown_connections 2023-10-08T00:11:04.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.015+0000 7fae48db8700 1 -- 172.21.15.99:0/3973872262 wait complete. 2023-10-08T00:11:04.030 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-08T00:11:04.031 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-08T00:11:04.031 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-08T00:11:04.031 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-10-08T00:11:04.032 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-10-08T00:11:04.032 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-10-08T00:11:04.032 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-10-08T00:11:04.033 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-10-08T00:11:04.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.220+0000 7f3a51aa2700 1 Processor -- start 2023-10-08T00:11:04.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.220+0000 7f3a51aa2700 1 -- start start 2023-10-08T00:11:04.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.222+0000 7f3a51aa2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a4c0b10a0 0x7f3a4c0cb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.222+0000 7f3a51aa2700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3a4c15b820 0x7f3a4c0ca930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.222+0000 7f3a51aa2700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a4c0c9450 con 0x7f3a4c0b0cd0 2023-10-08T00:11:04.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.222+0000 7f3a51aa2700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a4c0ce960 con 0x7f3a4c15b820 2023-10-08T00:11:04.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.222+0000 7f3a51aa2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4c0cc520 con 0x7f3a4c0b10a0 2023-10-08T00:11:04.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.222+0000 7f3a4affd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a4c0b10a0 0x7f3a4c0cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.222+0000 7f3a4affd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a4c0b10a0 0x7f3a4c0cb580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.99:45508/0 (socket says 172.21.15.99:45508) 2023-10-08T00:11:04.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.222+0000 7f3a4affd700 1 -- 172.21.15.99:0/1472936412 learned_addr learned my addr 172.21.15.99:0/1472936412 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:04.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.222+0000 7f3a4affd700 1 -- 172.21.15.99:0/1472936412 >> v2:172.21.15.99:3300/0 conn(0x7f3a4c15b820 msgr2=0x7f3a4c0ca930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:04.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.222+0000 7f3a4affd700 1 --2- 172.21.15.99:0/1472936412 >> v2:172.21.15.99:3300/0 conn(0x7f3a4c15b820 0x7f3a4c0ca930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.222+0000 7f3a4affd700 1 -- 172.21.15.99:0/1472936412 >> v1:172.21.15.191:6789/0 conn(0x7f3a4c0b0cd0 legacy=0x7f3a4c0ce250 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T00:11:04.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.223+0000 7f3a4affd700 1 -- 172.21.15.99:0/1472936412 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a4c0cc910 con 0x7f3a4c0b10a0 2023-10-08T00:11:04.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.223+0000 7f3a4affd700 1 --2- 172.21.15.99:0/1472936412 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a4c0b10a0 0x7f3a4c0cb580 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c0099b0 tx=0x7f3a3c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e84d0e6894a96420 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.223+0000 7f3a4a7fc700 1 -- 172.21.15.99:0/1472936412 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c00b820 con 0x7f3a4c0b10a0 2023-10-08T00:11:04.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.223+0000 7f3a4a7fc700 1 -- 172.21.15.99:0/1472936412 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a3c00b980 con 0x7f3a4c0b10a0 2023-10-08T00:11:04.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.223+0000 7f3a4a7fc700 1 -- 172.21.15.99:0/1472936412 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c00bd20 con 0x7f3a4c0b10a0 2023-10-08T00:11:04.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.224+0000 7f3a51aa2700 1 -- 172.21.15.99:0/1472936412 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a4c0b10a0 msgr2=0x7f3a4c0cb580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.224+0000 7f3a51aa2700 1 --2- 172.21.15.99:0/1472936412 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a4c0b10a0 0x7f3a4c0cb580 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c0099b0 tx=0x7f3a3c009250 comp rx=0 tx=0).stop 2023-10-08T00:11:04.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.225+0000 7f3a51aa2700 1 -- 172.21.15.99:0/1472936412 shutdown_connections 2023-10-08T00:11:04.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.225+0000 7f3a51aa2700 1 --2- 172.21.15.99:0/1472936412 >> v2:172.21.15.99:3300/0 conn(0x7f3a4c15b820 0x7f3a4c0ca930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.225+0000 7f3a51aa2700 1 --2- 172.21.15.99:0/1472936412 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a4c0b10a0 0x7f3a4c0cb580 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.225+0000 7f3a51aa2700 1 -- 172.21.15.99:0/1472936412 >> 172.21.15.99:0/1472936412 conn(0x7f3a4c06fdc0 msgr2=0x7f3a4c06bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:04.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.226+0000 7f3a51aa2700 1 -- 172.21.15.99:0/1472936412 shutdown_connections 2023-10-08T00:11:04.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.227+0000 7f3a51aa2700 1 -- 172.21.15.99:0/1472936412 wait complete. 2023-10-08T00:11:04.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.228+0000 7f3a51aa2700 1 Processor -- start 2023-10-08T00:11:04.231 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.228+0000 7f3a51aa2700 1 -- start start 2023-10-08T00:11:04.231 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.229+0000 7f3a51aa2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a4c0b0cd0 0x7f3a4c14c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.231 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.229+0000 7f3a51aa2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a4c0b10a0 0x7f3a4c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.231 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.229+0000 7f3a51aa2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a4c15b820 0x7f3a4c171d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.231 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.229+0000 7f3a51aa2700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4c0b1920 con 0x7f3a4c15b820 2023-10-08T00:11:04.231 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.229+0000 7f3a51aa2700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4c14ccc0 con 0x7f3a4c0b0cd0 2023-10-08T00:11:04.231 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.229+0000 7f3a51aa2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4c068850 con 0x7f3a4c0b10a0 2023-10-08T00:11:04.231 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.229+0000 7f3a4bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a4c15b820 0x7f3a4c171d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.231 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.229+0000 7f3a4bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a4c15b820 0x7f3a4c171d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38606/0 (socket says 172.21.15.99:38606) 2023-10-08T00:11:04.231 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.229+0000 7f3a4bfff700 1 -- 172.21.15.99:0/1922513776 learned_addr learned my addr 172.21.15.99:0/1922513776 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:04.232 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.229+0000 7f3a4b7fe700 1 --2- 172.21.15.99:0/1922513776 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a4c0b0cd0 0x7f3a4c14c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.232 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.229+0000 7f3a4affd700 1 --2- 172.21.15.99:0/1922513776 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a4c0b10a0 0x7f3a4c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.232 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.230+0000 7f3a4bfff700 1 -- 172.21.15.99:0/1922513776 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a4c0b10a0 msgr2=0x7f3a4c067f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.232 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.230+0000 7f3a4bfff700 1 --2- 172.21.15.99:0/1922513776 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a4c0b10a0 0x7f3a4c067f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.230+0000 7f3a4bfff700 1 -- 172.21.15.99:0/1922513776 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a4c0b0cd0 msgr2=0x7f3a4c14c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.230+0000 7f3a4bfff700 1 --2- 172.21.15.99:0/1922513776 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a4c0b0cd0 0x7f3a4c14c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.230+0000 7f3a4bfff700 1 -- 172.21.15.99:0/1922513776 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a3c0094f0 con 0x7f3a4c15b820 2023-10-08T00:11:04.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.230+0000 7f3a4affd700 1 --2- 172.21.15.99:0/1922513776 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a4c0b10a0 0x7f3a4c067f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:11:04.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.230+0000 7f3a4bfff700 1 --2- 172.21.15.99:0/1922513776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a4c15b820 0x7f3a4c171d60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3a4400bc30 tx=0x7f3a44009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.230+0000 7f3a337fe700 1 -- 172.21.15.99:0/1922513776 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a440092e0 con 0x7f3a4c15b820 2023-10-08T00:11:04.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.230+0000 7f3a337fe700 1 -- 172.21.15.99:0/1922513776 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a4400c7a0 con 0x7f3a4c15b820 2023-10-08T00:11:04.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.230+0000 7f3a337fe700 1 -- 172.21.15.99:0/1922513776 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4400cb80 con 0x7f3a4c15b820 2023-10-08T00:11:04.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.230+0000 7f7737949700 1 Processor -- start 2023-10-08T00:11:04.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.231+0000 7f7737949700 1 -- start start 2023-10-08T00:11:04.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.231+0000 7f3a51aa2700 1 -- 172.21.15.99:0/1922513776 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a4c068b30 con 0x7f3a4c15b820 2023-10-08T00:11:04.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.231+0000 7fa43da6d700 1 Processor -- start 2023-10-08T00:11:04.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.231+0000 7fa43da6d700 1 -- start start 2023-10-08T00:11:04.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.232+0000 7f7737949700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77300a7df0 0x7f77300b29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.232+0000 7f7737949700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77300ae760 con 0x7f77301572a0 2023-10-08T00:11:04.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.232+0000 7f7737949700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77300aea40 con 0x7f77300a7a20 2023-10-08T00:11:04.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.232+0000 7f7737949700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77300aed20 con 0x7f77300a7df0 2023-10-08T00:11:04.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.232+0000 7f7734ee4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77300a7df0 0x7f77300b29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.232+0000 7f7734ee4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77300a7df0 0x7f77300b29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.99:45516/0 (socket says 172.21.15.99:45516) 2023-10-08T00:11:04.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.232+0000 7f7734ee4700 1 -- 172.21.15.99:0/3437167810 learned_addr learned my addr 172.21.15.99:0/3437167810 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:04.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.232+0000 7fa43da6d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa43815e0e0 0x7fa4380c6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.232+0000 7fa43da6d700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa4380c85b0 0x7fa4380cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.232+0000 7fa43da6d700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4380c8120 con 0x7fa43806d330 2023-10-08T00:11:04.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.232+0000 7fa43da6d700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4380cbb10 con 0x7fa4380c85b0 2023-10-08T00:11:04.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.232+0000 7fa43da6d700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4380c7510 con 0x7fa43815e0e0 2023-10-08T00:11:04.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.231+0000 7f3a51aa2700 1 -- 172.21.15.99:0/1922513776 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a4c0690f0 con 0x7f3a4c15b820 2023-10-08T00:11:04.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.233+0000 7fa436ffd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa4380c85b0 0x7fa4380cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.233+0000 7fa436ffd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa4380c85b0 0x7fa4380cb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38616/0 (socket says 172.21.15.99:38616) 2023-10-08T00:11:04.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.233+0000 7fa436ffd700 1 -- 172.21.15.99:0/1435441779 learned_addr learned my addr 172.21.15.99:0/1435441779 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:04.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.233+0000 7f7734ee4700 1 -- 172.21.15.99:0/3437167810 >> v1:172.21.15.191:6789/0 conn(0x7f77300a7a20 legacy=0x7f77300af520 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T00:11:04.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.233+0000 7f7734ee4700 1 -- 172.21.15.99:0/3437167810 >> v1:172.21.15.99:6789/0 conn(0x7f77301572a0 legacy=0x7f7730157740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.233+0000 7f7734ee4700 1 -- 172.21.15.99:0/3437167810 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77300af000 con 0x7f77300a7df0 2023-10-08T00:11:04.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.233+0000 7fa4377fe700 1 --2- 172.21.15.99:0/1435441779 >> v2:172.21.15.191:3300/0 conn(0x7fa43815e0e0 0x7fa4380c6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.233+0000 7fa436ffd700 1 -- 172.21.15.99:0/1435441779 >> v2:172.21.15.191:3300/0 conn(0x7fa43815e0e0 msgr2=0x7fa4380c6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:04.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.233+0000 7fa436ffd700 1 --2- 172.21.15.99:0/1435441779 >> v2:172.21.15.191:3300/0 conn(0x7fa43815e0e0 0x7fa4380c6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.233+0000 7fa436ffd700 1 -- 172.21.15.99:0/1435441779 >> v1:172.21.15.99:6789/0 conn(0x7fa43806d330 legacy=0x7fa43806d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:04.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.233+0000 7f7734ee4700 1 --2- 172.21.15.99:0/3437167810 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77300a7df0 0x7f77300b29c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f77200099e0 tx=0x7f7720009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d72ba3c78778d51 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.233+0000 7f3a51aa2700 1 -- 172.21.15.99:0/1922513776 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3a4c04c3c0 con 0x7f3a4c15b820 2023-10-08T00:11:04.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.233+0000 7fa436ffd700 1 -- 172.21.15.99:0/1435441779 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4380c77f0 con 0x7fa4380c85b0 2023-10-08T00:11:04.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.233+0000 7fa436ffd700 1 --2- 172.21.15.99:0/1435441779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4380c85b0 0x7fa4380cb5d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa4380b2a50 tx=0x7fa42c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdda3975ac7f201b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.233+0000 7f7727fff700 1 -- 172.21.15.99:0/3437167810 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772000b8b0 con 0x7f77300a7df0 2023-10-08T00:11:04.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.233+0000 7f18158c4700 1 Processor -- start 2023-10-08T00:11:04.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.233+0000 7f18158c4700 1 -- start start 2023-10-08T00:11:04.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.234+0000 7f3a337fe700 1 -- 172.21.15.99:0/1922513776 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3a44012070 con 0x7f3a4c15b820 2023-10-08T00:11:04.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.234+0000 7f3a337fe700 1 --2- 172.21.15.99:0/1922513776 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3a38066fb0 0x7f3a38069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.234+0000 7f3a337fe700 1 -- 172.21.15.99:0/1922513776 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3a44084290 con 0x7f3a4c15b820 2023-10-08T00:11:04.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.234+0000 7f3a337fe700 1 --2- 172.21.15.99:0/1922513776 >> [v2:172.21.15.191:6800/86570,v1:172.21.15.191:6801/86570] conn(0x7f3a3806f140 0x7f3a38071550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.234+0000 7f3a337fe700 1 -- 172.21.15.99:0/1922513776 --> [v2:172.21.15.191:6800/86570,v1:172.21.15.191:6801/86570] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3a38071c00 con 0x7f3a3806f140 2023-10-08T00:11:04.242 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.235+0000 7f3a4b7fe700 1 --2- 172.21.15.99:0/1922513776 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3a38066fb0 0x7f3a38069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.242 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.235+0000 7f3a4affd700 1 --2- 172.21.15.99:0/1922513776 >> [v2:172.21.15.191:6800/86570,v1:172.21.15.191:6801/86570] conn(0x7f3a3806f140 0x7f3a38071550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.242 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.235+0000 7f18158c4700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f181015e080 0x7f18100b6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.242 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.235+0000 7f18158c4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f18100afd90 0x7f181006a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.242 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.235+0000 7f18158c4700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18100b6cc0 con 0x7f181015e450 2023-10-08T00:11:04.242 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.235+0000 7f18158c4700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18100b56d0 con 0x7f181015e080 2023-10-08T00:11:04.242 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.235+0000 7f18158c4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18100b0230 con 0x7f18100afd90 2023-10-08T00:11:04.243 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.233+0000 7f7727fff700 1 -- 172.21.15.99:0/3437167810 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f772000ba10 con 0x7f77300a7df0 2023-10-08T00:11:04.243 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.233+0000 7f7727fff700 1 -- 172.21.15.99:0/3437167810 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772000bdb0 con 0x7f77300a7df0 2023-10-08T00:11:04.243 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.234+0000 7f7737949700 1 -- 172.21.15.99:0/3437167810 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77300a7df0 msgr2=0x7f77300b29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.243 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.234+0000 7f7737949700 1 --2- 172.21.15.99:0/3437167810 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77300a7df0 0x7f77300b29c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f77200099e0 tx=0x7f7720009250 comp rx=0 tx=0).stop 2023-10-08T00:11:04.243 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.234+0000 7f7737949700 1 -- 172.21.15.99:0/3437167810 shutdown_connections 2023-10-08T00:11:04.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.234+0000 7f7737949700 1 --2- 172.21.15.99:0/3437167810 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77300a7df0 0x7f77300b29c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.234+0000 7f7737949700 1 -- 172.21.15.99:0/3437167810 >> 172.21.15.99:0/3437167810 conn(0x7f77300cd1e0 msgr2=0x7f77300cd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:04.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.235+0000 7f7737949700 1 -- 172.21.15.99:0/3437167810 shutdown_connections 2023-10-08T00:11:04.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.235+0000 7f7737949700 1 -- 172.21.15.99:0/3437167810 wait complete. 2023-10-08T00:11:04.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.235+0000 7f180effd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f181015e080 0x7f18100b6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.235+0000 7f180effd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f181015e080 0x7f18100b6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38626/0 (socket says 172.21.15.99:38626) 2023-10-08T00:11:04.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.235+0000 7f180e7fc700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f181015e450 0x7f18100b5b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:56046/0 (socket says 172.21.15.99:56046) 2023-10-08T00:11:04.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.235+0000 7f180effd700 1 -- 172.21.15.99:0/1279480632 learned_addr learned my addr 172.21.15.99:0/1279480632 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:04.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.235+0000 7f22e52ef700 1 Processor -- start 2023-10-08T00:11:04.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.235+0000 7fe76691e700 1 Processor -- start 2023-10-08T00:11:04.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.235+0000 7f3a4b7fe700 1 --2- 172.21.15.99:0/1922513776 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3a38066fb0 0x7f3a38069460 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3a40006fd0 tx=0x7f3a40008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.235+0000 7fe76691e700 1 -- start start 2023-10-08T00:11:04.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.236+0000 7fe76691e700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe76014ed60 0x7fe7600cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.236+0000 7fe76691e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe76014f130 0x7fe76006d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.236+0000 7fe76691e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7600c9b30 con 0x7fe7600c9fc0 2023-10-08T00:11:04.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.236+0000 7fe76691e700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7600c9240 con 0x7fe76014ed60 2023-10-08T00:11:04.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.236+0000 7fe76691e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe76006dea0 con 0x7fe76014f130 2023-10-08T00:11:04.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.237+0000 7fe75ffff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe76014ed60 0x7fe7600cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.235+0000 7f180effd700 1 -- 172.21.15.99:0/1279480632 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f18100afd90 msgr2=0x7f181006a520 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T00:11:04.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.235+0000 7f180effd700 1 --2- 172.21.15.99:0/1279480632 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f18100afd90 0x7f181006a520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.235+0000 7f180effd700 1 -- 172.21.15.99:0/1279480632 >> v1:172.21.15.191:6789/0 conn(0x7f181015e450 legacy=0x7f18100b5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.235+0000 7f180effd700 1 -- 172.21.15.99:0/1279480632 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18100b4da0 con 0x7f181015e080 2023-10-08T00:11:04.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.235+0000 7f180dffb700 1 -- 172.21.15.99:0/1279480632 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1478409216 0 0) 0x7f18100b6cc0 con 0x7f181015e450 2023-10-08T00:11:04.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.235+0000 7f180effd700 1 --2- 172.21.15.99:0/1279480632 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f181015e080 0x7f18100b6630 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1808009f20 tx=0x7f1808009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab15d775a1942525 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.237+0000 7f7737949700 1 Processor -- start 2023-10-08T00:11:04.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.237+0000 7fe75ffff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe76014ed60 0x7fe7600cccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38640/0 (socket says 172.21.15.99:38640) 2023-10-08T00:11:04.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.237+0000 7fe75ffff700 1 -- 172.21.15.99:0/4033257147 learned_addr learned my addr 172.21.15.99:0/4033257147 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:04.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.237+0000 7f180dffb700 1 -- 172.21.15.99:0/1279480632 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180800e070 con 0x7f181015e080 2023-10-08T00:11:04.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.237+0000 7fe75ffff700 1 -- 172.21.15.99:0/4033257147 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe76014f130 msgr2=0x7fe76006d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:04.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.235+0000 7f22e52ef700 1 -- start start 2023-10-08T00:11:04.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.237+0000 7f22e52ef700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f22e00aee00 0x7f22e00ac730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.237+0000 7f22e52ef700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22e00acd50 con 0x7f22e00ccf90 2023-10-08T00:11:04.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.237+0000 7f22e52ef700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22e00ad030 con 0x7f22e00bf970 2023-10-08T00:11:04.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.237+0000 7f22e52ef700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e015ec10 con 0x7f22e00aee00 2023-10-08T00:11:04.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.237+0000 7f22df7fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f22e00aee00 0x7f22e00ac730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.237+0000 7f22df7fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f22e00aee00 0x7f22e00ac730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.99:45540/0 (socket says 172.21.15.99:45540) 2023-10-08T00:11:04.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.237+0000 7f22df7fe700 1 -- 172.21.15.99:0/899777324 learned_addr learned my addr 172.21.15.99:0/899777324 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:04.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.237+0000 7f22df7fe700 1 -- 172.21.15.99:0/899777324 >> v1:172.21.15.191:6789/0 conn(0x7f22e00bf970 legacy=0x7f22e00cc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:04.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.237+0000 7f22df7fe700 1 -- 172.21.15.99:0/899777324 >> v1:172.21.15.99:6789/0 conn(0x7f22e00ccf90 legacy=0x7f22e00ac020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:04.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.237+0000 7f22df7fe700 1 -- 172.21.15.99:0/899777324 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e00ad310 con 0x7f22e00aee00 2023-10-08T00:11:04.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.237+0000 7fa4367fc700 1 -- 172.21.15.99:0/1435441779 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa42c00b820 con 0x7fa4380c85b0 2023-10-08T00:11:04.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.237+0000 7fe764ebb700 1 --2- 172.21.15.99:0/4033257147 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe76014f130 0x7fe76006d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.237+0000 7fe75effd700 1 -- 172.21.15.99:0/4033257147 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3177404270 0 0) 0x7fe7600c9b30 con 0x7fe7600c9fc0 2023-10-08T00:11:04.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.237+0000 7fe75ffff700 1 --2- 172.21.15.99:0/4033257147 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe76014f130 0x7fe76006d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.237+0000 7fa4367fc700 1 -- 172.21.15.99:0/1435441779 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa42c00b980 con 0x7fa4380c85b0 2023-10-08T00:11:04.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.237+0000 7fe75ffff700 1 -- 172.21.15.99:0/4033257147 >> v1:172.21.15.191:6789/0 conn(0x7fe7600c9fc0 legacy=0x7fe76006d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.237+0000 7fe75ffff700 1 -- 172.21.15.99:0/4033257147 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7600c94e0 con 0x7fe76014ed60 2023-10-08T00:11:04.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.237+0000 7fe75ffff700 1 --2- 172.21.15.99:0/4033257147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe76014ed60 0x7fe7600cccc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe7500099b0 tx=0x7fe750009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49c7feca2b3256ab server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.237+0000 7fa4367fc700 1 -- 172.21.15.99:0/1435441779 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa42c00bd20 con 0x7fa4380c85b0 2023-10-08T00:11:04.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.237+0000 7f180dffb700 1 -- 172.21.15.99:0/1279480632 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f180800ba80 con 0x7f181015e080 2023-10-08T00:11:04.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.237+0000 7f180dffb700 1 -- 172.21.15.99:0/1279480632 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18080092e0 con 0x7f181015e080 2023-10-08T00:11:04.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.238+0000 7fe75effd700 1 -- 172.21.15.99:0/4033257147 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75000b860 con 0x7fe76014ed60 2023-10-08T00:11:04.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.238+0000 7fe75effd700 1 -- 172.21.15.99:0/4033257147 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe75000b9c0 con 0x7fe76014ed60 2023-10-08T00:11:04.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.237+0000 7f7737949700 1 -- start start 2023-10-08T00:11:04.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.238+0000 7f7737949700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77300a7a20 0x7f773014b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.238+0000 7f7737949700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77300a7df0 0x7f77301473f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.238+0000 7f7737949700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77301572a0 0x7f7730147930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.238+0000 7f7737949700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7730148b70 con 0x7f77300a7df0 2023-10-08T00:11:04.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.238+0000 7f7737949700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7730146770 con 0x7f77301572a0 2023-10-08T00:11:04.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.238+0000 7f7737949700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7730147f60 con 0x7f77300a7a20 2023-10-08T00:11:04.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.238+0000 7f7734ee4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77300a7df0 0x7f77301473f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.238+0000 7f7734ee4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77300a7df0 0x7f77301473f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38642/0 (socket says 172.21.15.99:38642) 2023-10-08T00:11:04.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.238+0000 7f7734ee4700 1 -- 172.21.15.99:0/2677119224 learned_addr learned my addr 172.21.15.99:0/2677119224 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:04.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.238+0000 7f3a337fe700 1 -- 172.21.15.99:0/1922513776 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a44051460 con 0x7f3a4c15b820 2023-10-08T00:11:04.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.238+0000 7f3a4affd700 1 --2- 172.21.15.99:0/1922513776 >> [v2:172.21.15.191:6800/86570,v1:172.21.15.191:6801/86570] conn(0x7f3a3806f140 0x7f3a38071550 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.238+0000 7f18158c4700 1 -- 172.21.15.99:0/1279480632 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f181015e080 msgr2=0x7f18100b6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.238+0000 7f18158c4700 1 --2- 172.21.15.99:0/1279480632 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f181015e080 0x7f18100b6630 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1808009f20 tx=0x7f1808009840 comp rx=0 tx=0).stop 2023-10-08T00:11:04.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.237+0000 7f22df7fe700 1 --2- 172.21.15.99:0/899777324 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f22e00aee00 0x7f22e00ac730 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f22d801af50 tx=0x7f22d801a840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a770048838b5a040 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.238+0000 7fe75effd700 1 -- 172.21.15.99:0/4033257147 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75000bd60 con 0x7fe76014ed60 2023-10-08T00:11:04.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.238+0000 7f18158c4700 1 -- 172.21.15.99:0/1279480632 shutdown_connections 2023-10-08T00:11:04.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.238+0000 7f18158c4700 1 --2- 172.21.15.99:0/1279480632 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f18100afd90 0x7f181006a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.238+0000 7f18158c4700 1 --2- 172.21.15.99:0/1279480632 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f181015e080 0x7f18100b6630 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.238+0000 7f18158c4700 1 -- 172.21.15.99:0/1279480632 >> 172.21.15.99:0/1279480632 conn(0x7f1810068ce0 msgr2=0x7f18100690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:04.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.238+0000 7fe76691e700 1 -- 172.21.15.99:0/4033257147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe76014ed60 msgr2=0x7fe7600cccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.238+0000 7fe76691e700 1 --2- 172.21.15.99:0/4033257147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe76014ed60 0x7fe7600cccc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe7500099b0 tx=0x7fe750009250 comp rx=0 tx=0).stop 2023-10-08T00:11:04.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.239+0000 7f7734ee4700 1 -- 172.21.15.99:0/2677119224 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77300a7a20 msgr2=0x7f773014b280 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T00:11:04.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.239+0000 7f7734ee4700 1 --2- 172.21.15.99:0/2677119224 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77300a7a20 0x7f773014b280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.239+0000 7f7734ee4700 1 -- 172.21.15.99:0/2677119224 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77301572a0 msgr2=0x7f7730147930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T00:11:04.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.239+0000 7f7734ee4700 1 --2- 172.21.15.99:0/2677119224 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77301572a0 0x7f7730147930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.239+0000 7f7734ee4700 1 -- 172.21.15.99:0/2677119224 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77200094f0 con 0x7f77300a7df0 2023-10-08T00:11:04.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.238+0000 7fa43da6d700 1 -- 172.21.15.99:0/1435441779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4380c85b0 msgr2=0x7fa4380cb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.238+0000 7fa43da6d700 1 --2- 172.21.15.99:0/1435441779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4380c85b0 0x7fa4380cb5d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa4380b2a50 tx=0x7fa42c009250 comp rx=0 tx=0).stop 2023-10-08T00:11:04.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.239+0000 7f3a337fe700 1 -- 172.21.15.99:0/1922513776 <== osd.4 v2:172.21.15.191:6800/86570 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22340 (crc 0 0 0) 0x7f3a38071c00 con 0x7f3a3806f140 2023-10-08T00:11:04.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.239+0000 7f7734ee4700 1 --2- 172.21.15.99:0/2677119224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77300a7df0 0x7f77301473f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7720009290 tx=0x7f7720018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.239+0000 7fa43da6d700 1 -- 172.21.15.99:0/1435441779 shutdown_connections 2023-10-08T00:11:04.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.239+0000 7fa43da6d700 1 --2- 172.21.15.99:0/1435441779 >> v2:172.21.15.191:3300/0 conn(0x7fa43815e0e0 0x7fa4380c6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.239+0000 7fa43da6d700 1 --2- 172.21.15.99:0/1435441779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4380c85b0 0x7fa4380cb5d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.239+0000 7fa43da6d700 1 -- 172.21.15.99:0/1435441779 >> 172.21.15.99:0/1435441779 conn(0x7fa4380cc8f0 msgr2=0x7fa4380cccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:04.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.239+0000 7fa43da6d700 1 -- 172.21.15.99:0/1435441779 shutdown_connections 2023-10-08T00:11:04.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.239+0000 7fa43da6d700 1 -- 172.21.15.99:0/1435441779 wait complete. 2023-10-08T00:11:04.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.241+0000 7fa43da6d700 1 Processor -- start 2023-10-08T00:11:04.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.241+0000 7fa43da6d700 1 -- start start 2023-10-08T00:11:04.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.242+0000 7fa43da6d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa43806d330 0x7fa4380c4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.242+0000 7fa43da6d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4380c85b0 0x7fa438145d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.242+0000 7fa43da6d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa43815e0e0 0x7fa43815ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.242+0000 7fa43da6d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43815a000 con 0x7fa43815e0e0 2023-10-08T00:11:04.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.242+0000 7fa43da6d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4380c5360 con 0x7fa4380c85b0 2023-10-08T00:11:04.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.242+0000 7fa43da6d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4380c4990 con 0x7fa43806d330 2023-10-08T00:11:04.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.242+0000 7fa4377fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa43806d330 0x7fa4380c4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.242+0000 7fa436ffd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4380c85b0 0x7fa438145d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.242+0000 7fa4377fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa43806d330 0x7fa4380c4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.99:45556/0 (socket says 172.21.15.99:45556) 2023-10-08T00:11:04.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.242+0000 7fa4377fe700 1 -- 172.21.15.99:0/1244309695 learned_addr learned my addr 172.21.15.99:0/1244309695 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:04.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.242+0000 7fa437fff700 1 --2- 172.21.15.99:0/1244309695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa43815e0e0 0x7fa43815ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.243+0000 7fa4377fe700 1 -- 172.21.15.99:0/1244309695 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4380c85b0 msgr2=0x7fa438145d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.243+0000 7fa4377fe700 1 --2- 172.21.15.99:0/1244309695 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4380c85b0 0x7fa438145d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.243+0000 7fa4377fe700 1 -- 172.21.15.99:0/1244309695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa43815e0e0 msgr2=0x7fa43815ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.243+0000 7fa4377fe700 1 --2- 172.21.15.99:0/1244309695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa43815e0e0 0x7fa43815ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.243+0000 7fa4377fe700 1 -- 172.21.15.99:0/1244309695 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa42c0094f0 con 0x7fa43806d330 2023-10-08T00:11:04.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.243+0000 7fa436ffd700 1 --2- 172.21.15.99:0/1244309695 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4380c85b0 0x7fa438145d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:11:04.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.243+0000 7fa4377fe700 1 --2- 172.21.15.99:0/1244309695 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa43806d330 0x7fa4380c4e20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa42800d860 tx=0x7fa428008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.243+0000 7fa41f7fe700 1 -- 172.21.15.99:0/1244309695 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4280125f0 con 0x7fa43806d330 2023-10-08T00:11:04.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.243+0000 7fa43da6d700 1 -- 172.21.15.99:0/1244309695 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa438146280 con 0x7fa43806d330 2023-10-08T00:11:04.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.243+0000 7fa43da6d700 1 -- 172.21.15.99:0/1244309695 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4380c4060 con 0x7fa43806d330 2023-10-08T00:11:04.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.243+0000 7fa41f7fe700 1 -- 172.21.15.99:0/1244309695 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa428012750 con 0x7fa43806d330 2023-10-08T00:11:04.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.243+0000 7fa41f7fe700 1 -- 172.21.15.99:0/1244309695 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa428012b10 con 0x7fa43806d330 2023-10-08T00:11:04.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.244+0000 7fa43da6d700 1 -- 172.21.15.99:0/1244309695 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa418000fd0 con 0x7fa43806d330 2023-10-08T00:11:04.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.245+0000 7fa41f7fe700 1 -- 172.21.15.99:0/1244309695 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa42801e460 con 0x7fa43806d330 2023-10-08T00:11:04.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.245+0000 7fa41f7fe700 1 --2- 172.21.15.99:0/1244309695 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa424066f30 0x7fa4240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.245+0000 7fa41f7fe700 1 -- 172.21.15.99:0/1244309695 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa428084dc0 con 0x7fa43806d330 2023-10-08T00:11:04.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.238+0000 7f22ddffb700 1 -- 172.21.15.99:0/899777324 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d801f070 con 0x7f22e00aee00 2023-10-08T00:11:04.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.238+0000 7f22ddffb700 1 -- 172.21.15.99:0/899777324 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d801cac0 con 0x7f22e00aee00 2023-10-08T00:11:04.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.238+0000 7f22ddffb700 1 -- 172.21.15.99:0/899777324 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d801a2e0 con 0x7f22e00aee00 2023-10-08T00:11:04.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.239+0000 7f22e52ef700 1 -- 172.21.15.99:0/899777324 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f22e00aee00 msgr2=0x7f22e00ac730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.239+0000 7f22e52ef700 1 --2- 172.21.15.99:0/899777324 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f22e00aee00 0x7f22e00ac730 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f22d801af50 tx=0x7f22d801a840 comp rx=0 tx=0).stop 2023-10-08T00:11:04.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.239+0000 7f22e52ef700 1 -- 172.21.15.99:0/899777324 shutdown_connections 2023-10-08T00:11:04.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.239+0000 7f22e52ef700 1 --2- 172.21.15.99:0/899777324 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f22e00aee00 0x7f22e00ac730 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.239+0000 7f22e52ef700 1 -- 172.21.15.99:0/899777324 >> 172.21.15.99:0/899777324 conn(0x7f22e0070700 msgr2=0x7f22e0070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:04.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.240+0000 7f22e52ef700 1 -- 172.21.15.99:0/899777324 shutdown_connections 2023-10-08T00:11:04.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.240+0000 7f22e52ef700 1 -- 172.21.15.99:0/899777324 wait complete. 2023-10-08T00:11:04.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.241+0000 7f22e52ef700 1 Processor -- start 2023-10-08T00:11:04.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.241+0000 7f22e52ef700 1 -- start start 2023-10-08T00:11:04.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.242+0000 7f22e52ef700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22e00aee00 0x7f22e00a84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.242+0000 7f22e52ef700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f22e00bf970 0x7f22e00a6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.242+0000 7f22e52ef700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22e00ccf90 0x7f22e00a69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.242+0000 7f22e52ef700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e0161c30 con 0x7f22e00aee00 2023-10-08T00:11:04.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.242+0000 7f22e52ef700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e0160410 con 0x7f22e00ccf90 2023-10-08T00:11:04.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.242+0000 7f22e52ef700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e00ab3d0 con 0x7f22e00bf970 2023-10-08T00:11:04.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.243+0000 7f22df7fe700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22e00ccf90 0x7f22e00a69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.243+0000 7f22df7fe700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22e00ccf90 0x7f22e00a69b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.99:54568/0 (socket says 172.21.15.99:54568) 2023-10-08T00:11:04.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.243+0000 7f22df7fe700 1 -- 172.21.15.99:0/630488656 learned_addr learned my addr 172.21.15.99:0/630488656 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:04.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.243+0000 7f22deffd700 1 --2- 172.21.15.99:0/630488656 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22e00aee00 0x7f22e00a84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.243+0000 7f22df7fe700 1 -- 172.21.15.99:0/630488656 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f22e00bf970 msgr2=0x7f22e00a6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.243+0000 7f22df7fe700 1 --2- 172.21.15.99:0/630488656 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f22e00bf970 0x7f22e00a6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.243+0000 7f22df7fe700 1 -- 172.21.15.99:0/630488656 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22e00aee00 msgr2=0x7f22e00a84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.243+0000 7f22df7fe700 1 --2- 172.21.15.99:0/630488656 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22e00aee00 0x7f22e00a84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.243+0000 7f22df7fe700 1 -- 172.21.15.99:0/630488656 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d801aae0 con 0x7f22e00ccf90 2023-10-08T00:11:04.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.244+0000 7f22df7fe700 1 --2- 172.21.15.99:0/630488656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22e00ccf90 0x7f22e00a69b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f22d801aab0 tx=0x7f22d8028000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.245+0000 7f2fafdce700 1 Processor -- start 2023-10-08T00:11:04.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.245+0000 7f2fafdce700 1 -- start start 2023-10-08T00:11:04.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.239+0000 7fe76691e700 1 -- 172.21.15.99:0/4033257147 shutdown_connections 2023-10-08T00:11:04.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.239+0000 7fe76691e700 1 --2- 172.21.15.99:0/4033257147 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe76014f130 0x7fe76006d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.239+0000 7fe76691e700 1 --2- 172.21.15.99:0/4033257147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe76014ed60 0x7fe7600cccc0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.239+0000 7fe76691e700 1 -- 172.21.15.99:0/4033257147 >> 172.21.15.99:0/4033257147 conn(0x7fe760070700 msgr2=0x7fe760070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:04.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.239+0000 7fe76691e700 1 -- 172.21.15.99:0/4033257147 shutdown_connections 2023-10-08T00:11:04.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.239+0000 7fe76691e700 1 -- 172.21.15.99:0/4033257147 wait complete. 2023-10-08T00:11:04.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.240+0000 7fe76691e700 1 Processor -- start 2023-10-08T00:11:04.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.241+0000 7fe76691e700 1 -- start start 2023-10-08T00:11:04.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.242+0000 7fe76691e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe7600c9fc0 0x7fe760152490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.242+0000 7fe76691e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe76014ed60 0x7fe76014ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.242+0000 7fe76691e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe76014f130 0x7fe7600c2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.242+0000 7fe76691e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe760158f10 con 0x7fe76014ed60 2023-10-08T00:11:04.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.242+0000 7fe76691e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7600c2770 con 0x7fe76014f130 2023-10-08T00:11:04.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.242+0000 7fe76691e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe76014d360 con 0x7fe7600c9fc0 2023-10-08T00:11:04.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.242+0000 7fe75f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe76014ed60 0x7fe76014ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.242+0000 7fe75f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe76014ed60 0x7fe76014ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38652/0 (socket says 172.21.15.99:38652) 2023-10-08T00:11:04.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.242+0000 7fe75f7fe700 1 -- 172.21.15.99:0/3265728988 learned_addr learned my addr 172.21.15.99:0/3265728988 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:04.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.242+0000 7fe764ebb700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe76014f130 0x7fe7600c2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.242+0000 7fe75ffff700 1 --2- 172.21.15.99:0/3265728988 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe7600c9fc0 0x7fe760152490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.242+0000 7fe75f7fe700 1 -- 172.21.15.99:0/3265728988 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe7600c9fc0 msgr2=0x7fe760152490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.242+0000 7fe75f7fe700 1 --2- 172.21.15.99:0/3265728988 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe7600c9fc0 0x7fe760152490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.242+0000 7fe75f7fe700 1 -- 172.21.15.99:0/3265728988 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe76014f130 msgr2=0x7fe7600c2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.242+0000 7fe75f7fe700 1 --2- 172.21.15.99:0/3265728988 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe76014f130 0x7fe7600c2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.242+0000 7fe75f7fe700 1 -- 172.21.15.99:0/3265728988 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7500094f0 con 0x7fe76014ed60 2023-10-08T00:11:04.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.243+0000 7fe75f7fe700 1 --2- 172.21.15.99:0/3265728988 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe76014ed60 0x7fe76014ce20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe75400c7e0 tx=0x7fe754007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.245+0000 7fe747fff700 1 -- 172.21.15.99:0/3265728988 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe754011590 con 0x7fe76014ed60 2023-10-08T00:11:04.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.245+0000 7fe747fff700 1 -- 172.21.15.99:0/3265728988 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe754011710 con 0x7fe76014ed60 2023-10-08T00:11:04.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.245+0000 7fe76691e700 1 -- 172.21.15.99:0/3265728988 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7600c1e40 con 0x7fe76014ed60 2023-10-08T00:11:04.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.245+0000 7fe747fff700 1 -- 172.21.15.99:0/3265728988 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe754011ad0 con 0x7fe76014ed60 2023-10-08T00:11:04.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.245+0000 7fe76691e700 1 -- 172.21.15.99:0/3265728988 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7600c2400 con 0x7fe76014ed60 2023-10-08T00:11:04.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.241+0000 7f77257fa700 1 -- 172.21.15.99:0/2677119224 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7720018630 con 0x7f77300a7df0 2023-10-08T00:11:04.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.241+0000 7f77257fa700 1 -- 172.21.15.99:0/2677119224 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7720018790 con 0x7f77300a7df0 2023-10-08T00:11:04.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.241+0000 7f77257fa700 1 -- 172.21.15.99:0/2677119224 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7720018b10 con 0x7f77300a7df0 2023-10-08T00:11:04.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.241+0000 7f7737949700 1 -- 172.21.15.99:0/2677119224 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7730148240 con 0x7f77300a7df0 2023-10-08T00:11:04.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.241+0000 7f7737949700 1 -- 172.21.15.99:0/2677119224 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7730148800 con 0x7f77300a7df0 2023-10-08T00:11:04.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.242+0000 7f7737949700 1 -- 172.21.15.99:0/2677119224 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f773004c3c0 con 0x7f77300a7df0 2023-10-08T00:11:04.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.244+0000 7f77257fa700 1 -- 172.21.15.99:0/2677119224 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f772002b440 con 0x7f77300a7df0 2023-10-08T00:11:04.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.244+0000 7f77257fa700 1 --2- 172.21.15.99:0/2677119224 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7714067000 0x7f77140694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.245+0000 7f77257fa700 1 -- 172.21.15.99:0/2677119224 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7720014070 con 0x7f77300a7df0 2023-10-08T00:11:04.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.245+0000 7f77356e5700 1 --2- 172.21.15.99:0/2677119224 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7714067000 0x7f77140694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.245+0000 7f77257fa700 1 --2- 172.21.15.99:0/2677119224 >> [v2:172.21.15.191:6808/86571,v1:172.21.15.191:6809/86571] conn(0x7f771406f210 0x7f7714071620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.245+0000 7f77257fa700 1 -- 172.21.15.99:0/2677119224 --> [v2:172.21.15.191:6808/86571,v1:172.21.15.191:6809/86571] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7714071cd0 con 0x7f771406f210 2023-10-08T00:11:04.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.245+0000 7f77257fa700 1 -- 172.21.15.99:0/2677119224 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7720086730 con 0x7f77300a7df0 2023-10-08T00:11:04.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.245+0000 7f77356e5700 1 --2- 172.21.15.99:0/2677119224 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7714067000 0x7f77140694b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f772c009740 tx=0x7f772c006c60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.245+0000 7f18158c4700 1 -- 172.21.15.99:0/1279480632 shutdown_connections 2023-10-08T00:11:04.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.246+0000 7fa41f7fe700 1 --2- 172.21.15.99:0/1244309695 >> [v2:172.21.15.99:6808/86366,v1:172.21.15.99:6809/86366] conn(0x7fa42406f0c0 0x7fa4240714d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.246+0000 7fa41f7fe700 1 -- 172.21.15.99:0/1244309695 --> [v2:172.21.15.99:6808/86366,v1:172.21.15.99:6809/86366] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa424071b80 con 0x7fa42406f0c0 2023-10-08T00:11:04.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.246+0000 7fa41f7fe700 1 -- 172.21.15.99:0/1244309695 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa428085190 con 0x7fa43806d330 2023-10-08T00:11:04.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.246+0000 7fe76691e700 1 -- 172.21.15.99:0/3265728988 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe740000fd0 con 0x7fe76014ed60 2023-10-08T00:11:04.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.246+0000 7f22c6ffd700 1 -- 172.21.15.99:0/630488656 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d8023df0 con 0x7f22e00ccf90 2023-10-08T00:11:04.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.246+0000 7f22c6ffd700 1 -- 172.21.15.99:0/630488656 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d802f430 con 0x7f22e00ccf90 2023-10-08T00:11:04.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.246+0000 7f22c6ffd700 1 -- 172.21.15.99:0/630488656 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d802f7f0 con 0x7f22e00ccf90 2023-10-08T00:11:04.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.246+0000 7f2fafdce700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fa8159ef0 0x7f2fa80c7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.246+0000 7f2fafdce700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2fa815a2c0 0x7f2fa80cc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.246+0000 7f2fafdce700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2fa80ccdc0 0x7f2fa80c9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.246+0000 7f2fafdce700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fa80cd240 con 0x7f2fa815a2c0 2023-10-08T00:11:04.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.246+0000 7f2fafdce700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fa815ead0 con 0x7f2fa80ccdc0 2023-10-08T00:11:04.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.246+0000 7f2fafdce700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa815dc70 con 0x7f2fa8159ef0 2023-10-08T00:11:04.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.246+0000 7f2fad369700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2fa815a2c0 0x7f2fa80cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.246+0000 7f2fad369700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2fa815a2c0 0x7f2fa80cc880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38682/0 (socket says 172.21.15.99:38682) 2023-10-08T00:11:04.275 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.246+0000 7f2fad369700 1 -- 172.21.15.99:0/17035906 learned_addr learned my addr 172.21.15.99:0/17035906 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:04.275 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.246+0000 7f22e52ef700 1 -- 172.21.15.99:0/630488656 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e00ab0f0 con 0x7f22e00ccf90 2023-10-08T00:11:04.275 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.246+0000 7f22e52ef700 1 -- 172.21.15.99:0/630488656 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e01600a0 con 0x7f22e00ccf90 2023-10-08T00:11:04.275 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.246+0000 7f2fad369700 1 -- 172.21.15.99:0/17035906 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fa8159ef0 msgr2=0x7f2fa80c7860 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T00:11:04.275 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.246+0000 7f2fad369700 1 --2- 172.21.15.99:0/17035906 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fa8159ef0 0x7f2fa80c7860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.246+0000 7f2fad369700 1 -- 172.21.15.99:0/17035906 >> v2:172.21.15.191:3300/0 conn(0x7f2fa80ccdc0 msgr2=0x7f2fa80c9310 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T00:11:04.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.246+0000 7f2fad369700 1 --2- 172.21.15.99:0/17035906 >> v2:172.21.15.191:3300/0 conn(0x7f2fa80ccdc0 0x7f2fa80c9310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.246+0000 7f2fad369700 1 -- 172.21.15.99:0/17035906 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa815dec0 con 0x7f2fa815a2c0 2023-10-08T00:11:04.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.246+0000 7f2fad369700 1 --2- 172.21.15.99:0/17035906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2fa815a2c0 0x7f2fa80cc880 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8069850 tx=0x7f2fa0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b72c0bee7cf3b15 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.247+0000 7f2facb68700 1 -- 172.21.15.99:0/17035906 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa000e070 con 0x7f2fa815a2c0 2023-10-08T00:11:04.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.247+0000 7f2facb68700 1 -- 172.21.15.99:0/17035906 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa000bae0 con 0x7f2fa815a2c0 2023-10-08T00:11:04.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.247+0000 7f2facb68700 1 -- 172.21.15.99:0/17035906 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa00092e0 con 0x7f2fa815a2c0 2023-10-08T00:11:04.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.247+0000 7f22e52ef700 1 -- 172.21.15.99:0/630488656 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f22c0000fd0 con 0x7f22e00ccf90 2023-10-08T00:11:04.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.247+0000 7f2fafdce700 1 -- 172.21.15.99:0/17035906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2fa815a2c0 msgr2=0x7f2fa80cc880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.247+0000 7f2fafdce700 1 --2- 172.21.15.99:0/17035906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2fa815a2c0 0x7f2fa80cc880 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8069850 tx=0x7f2fa0009840 comp rx=0 tx=0).stop 2023-10-08T00:11:04.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.248+0000 7fe747fff700 1 -- 172.21.15.99:0/3265728988 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe75401c460 con 0x7fe76014ed60 2023-10-08T00:11:04.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.248+0000 7f2fafdce700 1 -- 172.21.15.99:0/17035906 shutdown_connections 2023-10-08T00:11:04.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.248+0000 7f2fafdce700 1 --2- 172.21.15.99:0/17035906 >> v2:172.21.15.191:3300/0 conn(0x7f2fa80ccdc0 0x7f2fa80c9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.248+0000 7f2fafdce700 1 --2- 172.21.15.99:0/17035906 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fa8159ef0 0x7f2fa80c7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.248+0000 7f2fafdce700 1 --2- 172.21.15.99:0/17035906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2fa815a2c0 0x7f2fa80cc880 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.248+0000 7f2fafdce700 1 -- 172.21.15.99:0/17035906 >> 172.21.15.99:0/17035906 conn(0x7f2fa8070700 msgr2=0x7f2fa8070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:04.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.248+0000 7f22c6ffd700 1 -- 172.21.15.99:0/630488656 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f22d802fbe0 con 0x7f22e00ccf90 2023-10-08T00:11:04.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.248+0000 7f2fafdce700 1 -- 172.21.15.99:0/17035906 shutdown_connections 2023-10-08T00:11:04.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.248+0000 7f2fafdce700 1 -- 172.21.15.99:0/17035906 wait complete. 2023-10-08T00:11:04.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.248+0000 7fe747fff700 1 --2- 172.21.15.99:0/3265728988 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe74c067000 0x7fe74c0694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.248+0000 7f22c6ffd700 1 --2- 172.21.15.99:0/630488656 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f22cc067080 0x7f22cc069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.249+0000 7f22c6ffd700 1 -- 172.21.15.99:0/630488656 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f22d80976b0 con 0x7f22e00ccf90 2023-10-08T00:11:04.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.249+0000 7fe747fff700 1 -- 172.21.15.99:0/3265728988 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe754084b70 con 0x7fe76014ed60 2023-10-08T00:11:04.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.249+0000 7f22c6ffd700 1 --2- 172.21.15.99:0/630488656 >> [v2:172.21.15.99:6816/86382,v1:172.21.15.99:6817/86382] conn(0x7f22cc06f290 0x7f22cc0716a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.249+0000 7f22c6ffd700 1 -- 172.21.15.99:0/630488656 --> [v2:172.21.15.99:6816/86382,v1:172.21.15.99:6817/86382] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f22cc071d50 con 0x7f22cc06f290 2023-10-08T00:11:04.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.249+0000 7f22c6ffd700 1 -- 172.21.15.99:0/630488656 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22d80979e0 con 0x7f22e00ccf90 2023-10-08T00:11:04.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.249+0000 7fe747fff700 1 --2- 172.21.15.99:0/3265728988 >> [v2:172.21.15.191:6816/86608,v1:172.21.15.191:6818/86608] conn(0x7fe74c06f210 0x7fe74c071620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.249+0000 7fe75ffff700 1 --2- 172.21.15.99:0/3265728988 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe74c067000 0x7fe74c0694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.249+0000 7fe747fff700 1 -- 172.21.15.99:0/3265728988 --> [v2:172.21.15.191:6816/86608,v1:172.21.15.191:6818/86608] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe74c071cd0 con 0x7fe74c06f210 2023-10-08T00:11:04.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.249+0000 7fe747fff700 1 -- 172.21.15.99:0/3265728988 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe754084f40 con 0x7fe76014ed60 2023-10-08T00:11:04.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.249+0000 7f22deffd700 1 --2- 172.21.15.99:0/630488656 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f22cc067080 0x7f22cc069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.249+0000 7fe764ebb700 1 --2- 172.21.15.99:0/3265728988 >> [v2:172.21.15.191:6816/86608,v1:172.21.15.191:6818/86608] conn(0x7fe74c06f210 0x7fe74c071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.249+0000 7fe75ffff700 1 --2- 172.21.15.99:0/3265728988 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe74c067000 0x7fe74c0694b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe7500099b0 tx=0x7fe750017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.249+0000 7f22deffd700 1 --2- 172.21.15.99:0/630488656 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f22cc067080 0x7f22cc069530 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f22d4006fd0 tx=0x7f22d40090f0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.249+0000 7fe764ebb700 1 --2- 172.21.15.99:0/3265728988 >> [v2:172.21.15.191:6816/86608,v1:172.21.15.191:6818/86608] conn(0x7fe74c06f210 0x7fe74c071620 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.250+0000 7fa437fff700 1 --2- 172.21.15.99:0/1244309695 >> [v2:172.21.15.99:6808/86366,v1:172.21.15.99:6809/86366] conn(0x7fa42406f0c0 0x7fa4240714d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.250+0000 7f18158c4700 1 -- 172.21.15.99:0/1279480632 wait complete. 2023-10-08T00:11:04.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.250+0000 7fa436ffd700 1 --2- 172.21.15.99:0/1244309695 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa424066f30 0x7fa4240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.250+0000 7fa437fff700 1 --2- 172.21.15.99:0/1244309695 >> [v2:172.21.15.99:6808/86366,v1:172.21.15.99:6809/86366] conn(0x7fa42406f0c0 0x7fa4240714d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.250+0000 7f2fafdce700 1 Processor -- start 2023-10-08T00:11:04.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.250+0000 7fe747fff700 1 -- 172.21.15.99:0/3265728988 <== osd.5 v2:172.21.15.191:6816/86608 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22340 (crc 0 0 0) 0x7fe74c071cd0 con 0x7fe74c06f210 2023-10-08T00:11:04.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.250+0000 7fa436ffd700 1 --2- 172.21.15.99:0/1244309695 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa424066f30 0x7fa4240693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa4380c8a50 tx=0x7fa42c016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.250+0000 7f22de7fc700 1 --2- 172.21.15.99:0/630488656 >> [v2:172.21.15.99:6816/86382,v1:172.21.15.99:6817/86382] conn(0x7f22cc06f290 0x7f22cc0716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.251+0000 7f22de7fc700 1 --2- 172.21.15.99:0/630488656 >> [v2:172.21.15.99:6816/86382,v1:172.21.15.99:6817/86382] conn(0x7f22cc06f290 0x7f22cc0716a0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.250+0000 7f7735ee6700 1 --2- 172.21.15.99:0/2677119224 >> [v2:172.21.15.191:6808/86571,v1:172.21.15.191:6809/86571] conn(0x7f771406f210 0x7f7714071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.250+0000 7f7735ee6700 1 --2- 172.21.15.99:0/2677119224 >> [v2:172.21.15.191:6808/86571,v1:172.21.15.191:6809/86571] conn(0x7f771406f210 0x7f7714071620 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.251+0000 7f77257fa700 1 -- 172.21.15.99:0/2677119224 <== osd.6 v2:172.21.15.191:6808/86571 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22340 (crc 0 0 0) 0x7f7714071cd0 con 0x7f771406f210 2023-10-08T00:11:04.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.251+0000 7f18158c4700 1 Processor -- start 2023-10-08T00:11:04.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.251+0000 7f18158c4700 1 -- start start 2023-10-08T00:11:04.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.250+0000 7f2fafdce700 1 -- start start 2023-10-08T00:11:04.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.251+0000 7f2fafdce700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2fa80ccdc0 0x7f2fa80c5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.251+0000 7f2fafdce700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fa8159ef0 0x7f2fa816ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.251+0000 7f2fafdce700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fa815a2c0 0x7f2fa816f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.251+0000 7f2fafdce700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa816fa30 con 0x7f2fa80ccdc0 2023-10-08T00:11:04.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.251+0000 7f2fafdce700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa816f780 con 0x7f2fa8159ef0 2023-10-08T00:11:04.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.251+0000 7f2fafdce700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa80c1bd0 con 0x7f2fa815a2c0 2023-10-08T00:11:04.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.252+0000 7f2fadb6a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2fa80ccdc0 0x7f2fa80c5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.252+0000 7f2fadb6a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2fa80ccdc0 0x7f2fa80c5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38686/0 (socket says 172.21.15.99:38686) 2023-10-08T00:11:04.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.252+0000 7f2fadb6a700 1 -- 172.21.15.99:0/93772379 learned_addr learned my addr 172.21.15.99:0/93772379 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:04.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.252+0000 7f2fad369700 1 --2- 172.21.15.99:0/93772379 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fa8159ef0 0x7f2fa816ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.252+0000 7f2fae36b700 1 --2- 172.21.15.99:0/93772379 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fa815a2c0 0x7f2fa816f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.252+0000 7f2fadb6a700 1 -- 172.21.15.99:0/93772379 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fa815a2c0 msgr2=0x7f2fa816f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.252+0000 7f2fadb6a700 1 --2- 172.21.15.99:0/93772379 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fa815a2c0 0x7f2fa816f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.252+0000 7f2fadb6a700 1 -- 172.21.15.99:0/93772379 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fa8159ef0 msgr2=0x7f2fa816ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.252+0000 7f2fadb6a700 1 --2- 172.21.15.99:0/93772379 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fa8159ef0 0x7f2fa816ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.252+0000 7f2fadb6a700 1 -- 172.21.15.99:0/93772379 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa0009ae0 con 0x7f2fa80ccdc0 2023-10-08T00:11:04.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.252+0000 7f2fae36b700 1 --2- 172.21.15.99:0/93772379 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fa815a2c0 0x7f2fa816f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:11:04.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.252+0000 7f2fadb6a700 1 --2- 172.21.15.99:0/93772379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2fa80ccdc0 0x7f2fa80c5eb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2fa400b600 tx=0x7f2fa40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.252+0000 7f2f9dffb700 1 -- 172.21.15.99:0/93772379 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa400f510 con 0x7f2fa80ccdc0 2023-10-08T00:11:04.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.252+0000 7f18158c4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18100afd90 0x7f181006e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.252+0000 7f18158c4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f181015e080 0x7f1810156c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.253+0000 7f18158c4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f181015e450 0x7f1810157180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.253+0000 7f18158c4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1810159fc0 con 0x7f18100afd90 2023-10-08T00:11:04.294 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.253+0000 7f18158c4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f181006c470 con 0x7f181015e080 2023-10-08T00:11:04.294 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.253+0000 7f18158c4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18100d0600 con 0x7f181015e450 2023-10-08T00:11:04.294 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.253+0000 7f7e6aac0700 1 Processor -- start 2023-10-08T00:11:04.294 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.253+0000 7f7e6aac0700 1 -- start start 2023-10-08T00:11:04.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.253+0000 7f2fafdce700 1 -- 172.21.15.99:0/93772379 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fa80c1eb0 con 0x7f2fa80ccdc0 2023-10-08T00:11:04.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.253+0000 7f2fafdce700 1 -- 172.21.15.99:0/93772379 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fa80c2470 con 0x7f2fa80ccdc0 2023-10-08T00:11:04.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.254+0000 7f2fafdce700 1 -- 172.21.15.99:0/93772379 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2fa804c3c0 con 0x7f2fa80ccdc0 2023-10-08T00:11:04.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.254+0000 7f2f9dffb700 1 -- 172.21.15.99:0/93772379 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa400f880 con 0x7f2fa80ccdc0 2023-10-08T00:11:04.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.254+0000 7f2f9dffb700 1 -- 172.21.15.99:0/93772379 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa400f9f0 con 0x7f2fa80ccdc0 2023-10-08T00:11:04.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.254+0000 7f7e6aac0700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7e640a7a20 0x7f7e640af520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.254+0000 7f7e6aac0700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7e640a7df0 0x7f7e640b29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.254+0000 7f7e6aac0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e641572a0 0x7f7e64157700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.253+0000 7f180effd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18100afd90 0x7f181006e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.253+0000 7f180effd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18100afd90 0x7f181006e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38692/0 (socket says 172.21.15.99:38692) 2023-10-08T00:11:04.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.254+0000 7f7e6aac0700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e640ae760 con 0x7f7e640a7df0 2023-10-08T00:11:04.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.254+0000 7f7e6aac0700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e640aea40 con 0x7f7e640a7a20 2023-10-08T00:11:04.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.254+0000 7f7e6aac0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e640aed20 con 0x7f7e641572a0 2023-10-08T00:11:04.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.253+0000 7f180effd700 1 -- 172.21.15.99:0/3545565961 learned_addr learned my addr 172.21.15.99:0/3545565961 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:04.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.253+0000 7f180effd700 1 -- 172.21.15.99:0/3545565961 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f181015e450 msgr2=0x7f1810157180 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T00:11:04.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.253+0000 7f180effd700 1 --2- 172.21.15.99:0/3545565961 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f181015e450 0x7f1810157180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.253+0000 7f180effd700 1 -- 172.21.15.99:0/3545565961 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f181015e080 msgr2=0x7f1810156c40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T00:11:04.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.253+0000 7f180effd700 1 --2- 172.21.15.99:0/3545565961 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f181015e080 0x7f1810156c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.253+0000 7f180effd700 1 -- 172.21.15.99:0/3545565961 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1808009ae0 con 0x7f18100afd90 2023-10-08T00:11:04.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.253+0000 7f180effd700 1 --2- 172.21.15.99:0/3545565961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18100afd90 0x7f181006e860 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f18100899e0 tx=0x7f1808018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.254+0000 7f7e6885c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7e640a7a20 0x7f7e640af520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.254+0000 7f7e6905d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e641572a0 0x7f7e64157700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.254+0000 7f7e6885c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7e640a7a20 0x7f7e640af520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.99:54582/0 (socket says 172.21.15.99:54582) 2023-10-08T00:11:04.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.254+0000 7f7e6905d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e641572a0 0x7f7e64157700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.99:45570/0 (socket says 172.21.15.99:45570) 2023-10-08T00:11:04.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.254+0000 7f7e6885c700 1 -- 172.21.15.99:0/1353325404 learned_addr learned my addr 172.21.15.99:0/1353325404 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:04.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.254+0000 7f7e63fff700 1 --2- 172.21.15.99:0/1353325404 >> v2:172.21.15.99:3300/0 conn(0x7f7e640a7df0 0x7f7e640b29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.255+0000 7f7e6885c700 1 -- 172.21.15.99:0/1353325404 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e641572a0 msgr2=0x7f7e64157700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:04.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.255+0000 7f7e6885c700 1 --2- 172.21.15.99:0/1353325404 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e641572a0 0x7f7e64157700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.255+0000 7f7e6885c700 1 -- 172.21.15.99:0/1353325404 >> v2:172.21.15.99:3300/0 conn(0x7f7e640a7df0 msgr2=0x7f7e640b29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:04.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.255+0000 7f7e6885c700 1 --2- 172.21.15.99:0/1353325404 >> v2:172.21.15.99:3300/0 conn(0x7f7e640a7df0 0x7f7e640b29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.255+0000 7f7e6885c700 1 -- 172.21.15.99:0/1353325404 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e640af000 con 0x7f7e640a7a20 2023-10-08T00:11:04.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.255+0000 7f7e6885c700 1 --2- 172.21.15.99:0/1353325404 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e640a7a20 0x7f7e640af520 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7e6406fce0 tx=0x7f7e50009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbe7be6ae77099dd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.255+0000 7f7e637fe700 1 -- 172.21.15.99:0/1353325404 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5000b6d0 con 0x7f7e640a7a20 2023-10-08T00:11:04.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.255+0000 7f7e637fe700 1 -- 172.21.15.99:0/1353325404 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5000b830 con 0x7f7e640a7a20 2023-10-08T00:11:04.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.255+0000 7f7e637fe700 1 -- 172.21.15.99:0/1353325404 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5000bbf0 con 0x7f7e640a7a20 2023-10-08T00:11:04.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.256+0000 7f7e6aac0700 1 -- 172.21.15.99:0/1353325404 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e640a7a20 msgr2=0x7f7e640af520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.256+0000 7f7e6aac0700 1 --2- 172.21.15.99:0/1353325404 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e640a7a20 0x7f7e640af520 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7e6406fce0 tx=0x7f7e50009250 comp rx=0 tx=0).stop 2023-10-08T00:11:04.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.256+0000 7f7e6aac0700 1 -- 172.21.15.99:0/1353325404 shutdown_connections 2023-10-08T00:11:04.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.256+0000 7f7e6aac0700 1 --2- 172.21.15.99:0/1353325404 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e641572a0 0x7f7e64157700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.256+0000 7f7e6aac0700 1 --2- 172.21.15.99:0/1353325404 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e640a7a20 0x7f7e640af520 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.256+0000 7f7e6aac0700 1 --2- 172.21.15.99:0/1353325404 >> v2:172.21.15.99:3300/0 conn(0x7f7e640a7df0 0x7f7e640b29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.256+0000 7f7e6aac0700 1 -- 172.21.15.99:0/1353325404 >> 172.21.15.99:0/1353325404 conn(0x7f7e640cd1e0 msgr2=0x7f7e640cd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:04.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.256+0000 7f7e6aac0700 1 -- 172.21.15.99:0/1353325404 shutdown_connections 2023-10-08T00:11:04.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.256+0000 7f7e6aac0700 1 -- 172.21.15.99:0/1353325404 wait complete. 2023-10-08T00:11:04.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.258+0000 7f7e6aac0700 1 Processor -- start 2023-10-08T00:11:04.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.258+0000 7f7e6aac0700 1 -- start start 2023-10-08T00:11:04.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.260+0000 7f7e6aac0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e640a7a20 0x7f7e64099e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.260+0000 7f7e6aac0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e640a7df0 0x7f7e64098c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.260+0000 7f7e6aac0700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e641572a0 0x7f7e64099140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.260+0000 7f7e6aac0700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e64098770 con 0x7f7e640a7df0 2023-10-08T00:11:04.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.260+0000 7f7e6aac0700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6414d690 con 0x7f7e641572a0 2023-10-08T00:11:04.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.260+0000 7f7e6aac0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e64097ba0 con 0x7f7e640a7a20 2023-10-08T00:11:04.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.260+0000 7f7e63fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e640a7df0 0x7f7e64098c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.260+0000 7f7e63fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e640a7df0 0x7f7e64098c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38708/0 (socket says 172.21.15.99:38708) 2023-10-08T00:11:04.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.260+0000 7f7e63fff700 1 -- 172.21.15.99:0/4043311721 learned_addr learned my addr 172.21.15.99:0/4043311721 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:04.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.260+0000 7f7e6885c700 1 --2- 172.21.15.99:0/4043311721 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e640a7a20 0x7f7e64099e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.260+0000 7f7e6905d700 1 --2- 172.21.15.99:0/4043311721 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e641572a0 0x7f7e64099140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.260+0000 7f7e63fff700 1 -- 172.21.15.99:0/4043311721 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e640a7a20 msgr2=0x7f7e64099e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.260+0000 7f7e63fff700 1 --2- 172.21.15.99:0/4043311721 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e640a7a20 0x7f7e64099e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.260+0000 7f7e63fff700 1 -- 172.21.15.99:0/4043311721 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e641572a0 msgr2=0x7f7e64099140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.260+0000 7f7e63fff700 1 --2- 172.21.15.99:0/4043311721 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e641572a0 0x7f7e64099140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.260+0000 7f7e63fff700 1 -- 172.21.15.99:0/4043311721 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e500094f0 con 0x7f7e640a7df0 2023-10-08T00:11:04.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.260+0000 7f7e63fff700 1 --2- 172.21.15.99:0/4043311721 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e640a7df0 0x7f7e64098c00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7e5800ec50 tx=0x7f7e580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.261+0000 7f7e60ff9700 1 -- 172.21.15.99:0/4043311721 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5800c850 con 0x7f7e640a7df0 2023-10-08T00:11:04.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.261+0000 7f7e60ff9700 1 -- 172.21.15.99:0/4043311721 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5800c9b0 con 0x7f7e640a7df0 2023-10-08T00:11:04.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.261+0000 7f7e6aac0700 1 -- 172.21.15.99:0/4043311721 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e64097e40 con 0x7f7e640a7df0 2023-10-08T00:11:04.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.261+0000 7f7e60ff9700 1 -- 172.21.15.99:0/4043311721 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5800cd30 con 0x7f7e640a7df0 2023-10-08T00:11:04.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.261+0000 7f7e6aac0700 1 -- 172.21.15.99:0/4043311721 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e64098400 con 0x7f7e640a7df0 2023-10-08T00:11:04.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.263+0000 7f7e60ff9700 1 -- 172.21.15.99:0/4043311721 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7e58013680 con 0x7f7e640a7df0 2023-10-08T00:11:04.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.263+0000 7f7e60ff9700 1 --2- 172.21.15.99:0/4043311721 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7e48066f30 0x7f7e480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.263+0000 7f7e60ff9700 1 -- 172.21.15.99:0/4043311721 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7e58084b80 con 0x7f7e640a7df0 2023-10-08T00:11:04.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.264+0000 7f7e6885c700 1 --2- 172.21.15.99:0/4043311721 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7e48066f30 0x7f7e480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.264+0000 7f7e6885c700 1 --2- 172.21.15.99:0/4043311721 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7e48066f30 0x7f7e480693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7e6406fce0 tx=0x7f7e50017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.266+0000 7f7e6aac0700 1 --2- 172.21.15.99:0/4043311721 >> [v2:172.21.15.99:6800/86365,v1:172.21.15.99:6801/86365] conn(0x7f7e6414cee0 0x7f7e64150e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.266+0000 7f7e6aac0700 1 -- 172.21.15.99:0/4043311721 --> [v2:172.21.15.99:6800/86365,v1:172.21.15.99:6801/86365] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7e640ac530 con 0x7f7e6414cee0 2023-10-08T00:11:04.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.256+0000 7fa41f7fe700 1 -- 172.21.15.99:0/1244309695 <== osd.1 v2:172.21.15.99:6808/86366 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22340 (crc 0 0 0) 0x7fa424071b80 con 0x7fa42406f0c0 2023-10-08T00:11:04.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.256+0000 7f22c6ffd700 1 -- 172.21.15.99:0/630488656 <== osd.2 v2:172.21.15.99:6816/86382 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22340 (crc 0 0 0) 0x7f22cc071d50 con 0x7f22cc06f290 2023-10-08T00:11:04.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.255+0000 7f2f9dffb700 1 -- 172.21.15.99:0/93772379 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2fa4024430 con 0x7f2fa80ccdc0 2023-10-08T00:11:04.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.256+0000 7f2f9dffb700 1 --2- 172.21.15.99:0/93772379 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2f8c066f00 0x7f2f8c0693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.256+0000 7f2f9dffb700 1 -- 172.21.15.99:0/93772379 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2fa4084aa0 con 0x7f2fa80ccdc0 2023-10-08T00:11:04.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.256+0000 7f2fad369700 1 --2- 172.21.15.99:0/93772379 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2f8c066f00 0x7f2f8c0693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.256+0000 7f2f9dffb700 1 --2- 172.21.15.99:0/93772379 >> [v2:172.21.15.99:6818/86393,v1:172.21.15.99:6820/86393] conn(0x7f2f8c06f090 0x7f2f8c0714a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.256+0000 7f2f9dffb700 1 -- 172.21.15.99:0/93772379 --> [v2:172.21.15.99:6818/86393,v1:172.21.15.99:6820/86393] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2f8c071b50 con 0x7f2f8c06f090 2023-10-08T00:11:04.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.256+0000 7f2f9dffb700 1 -- 172.21.15.99:0/93772379 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fa4084e70 con 0x7f2fa80ccdc0 2023-10-08T00:11:04.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.256+0000 7f2fae36b700 1 --2- 172.21.15.99:0/93772379 >> [v2:172.21.15.99:6818/86393,v1:172.21.15.99:6820/86393] conn(0x7f2f8c06f090 0x7f2f8c0714a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.256+0000 7f2fad369700 1 --2- 172.21.15.99:0/93772379 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2f8c066f00 0x7f2f8c0693b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2fa815a6b0 tx=0x7f2fa0018000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.256+0000 7f2fae36b700 1 --2- 172.21.15.99:0/93772379 >> [v2:172.21.15.99:6818/86393,v1:172.21.15.99:6820/86393] conn(0x7f2f8c06f090 0x7f2f8c0714a0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.257+0000 7f2f9dffb700 1 -- 172.21.15.99:0/93772379 <== osd.3 v2:172.21.15.99:6818/86393 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22340 (crc 0 0 0) 0x7f2f8c071b50 con 0x7f2f8c06f090 2023-10-08T00:11:04.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.266+0000 7f3a51aa2700 1 -- 172.21.15.99:0/1922513776 --> [v2:172.21.15.191:6800/86570,v1:172.21.15.191:6801/86570] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3a4c0cf940 con 0x7f3a3806f140 2023-10-08T00:11:04.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.262+0000 7f17f6ffd700 1 -- 172.21.15.99:0/3545565961 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1808008b90 con 0x7f18100afd90 2023-10-08T00:11:04.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.262+0000 7f17f6ffd700 1 -- 172.21.15.99:0/3545565961 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1808008cf0 con 0x7f18100afd90 2023-10-08T00:11:04.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.262+0000 7f17f6ffd700 1 -- 172.21.15.99:0/3545565961 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18080266b0 con 0x7f18100afd90 2023-10-08T00:11:04.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.262+0000 7f18158c4700 1 -- 172.21.15.99:0/3545565961 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f181006c710 con 0x7f18100afd90 2023-10-08T00:11:04.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.262+0000 7f18158c4700 1 -- 172.21.15.99:0/3545565961 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18100d08e0 con 0x7f18100afd90 2023-10-08T00:11:04.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.263+0000 7f18158c4700 1 -- 172.21.15.99:0/3545565961 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f181004c3c0 con 0x7f18100afd90 2023-10-08T00:11:04.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.267+0000 7f7e6905d700 1 --2- 172.21.15.99:0/4043311721 >> [v2:172.21.15.99:6800/86365,v1:172.21.15.99:6801/86365] conn(0x7f7e6414cee0 0x7f7e64150e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.267+0000 7f3a337fe700 1 -- 172.21.15.99:0/1922513776 <== osd.4 v2:172.21.15.191:6800/86570 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3a4c0cf940 con 0x7f3a3806f140 2023-10-08T00:11:04.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.267+0000 7f7e6905d700 1 --2- 172.21.15.99:0/4043311721 >> [v2:172.21.15.99:6800/86365,v1:172.21.15.99:6801/86365] conn(0x7f7e6414cee0 0x7f7e64150e80 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.267+0000 7f3a51aa2700 1 -- 172.21.15.99:0/1922513776 >> [v2:172.21.15.191:6800/86570,v1:172.21.15.191:6801/86570] conn(0x7f3a3806f140 msgr2=0x7f3a38071550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.267+0000 7f3a51aa2700 1 --2- 172.21.15.99:0/1922513776 >> [v2:172.21.15.191:6800/86570,v1:172.21.15.191:6801/86570] conn(0x7f3a3806f140 0x7f3a38071550 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.267+0000 7f3a51aa2700 1 -- 172.21.15.99:0/1922513776 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3a38066fb0 msgr2=0x7f3a38069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.267+0000 7f3a51aa2700 1 --2- 172.21.15.99:0/1922513776 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3a38066fb0 0x7f3a38069460 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3a40006fd0 tx=0x7f3a40008040 comp rx=0 tx=0).stop 2023-10-08T00:11:04.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.267+0000 7f3a51aa2700 1 -- 172.21.15.99:0/1922513776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a4c15b820 msgr2=0x7f3a4c171d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.267+0000 7f3a51aa2700 1 --2- 172.21.15.99:0/1922513776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a4c15b820 0x7f3a4c171d60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3a4400bc30 tx=0x7f3a44009840 comp rx=0 tx=0).stop 2023-10-08T00:11:04.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.267+0000 7f3a4bfff700 1 -- 172.21.15.99:0/1922513776 reap_dead start 2023-10-08T00:11:04.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.268+0000 7f3a51aa2700 1 -- 172.21.15.99:0/1922513776 shutdown_connections 2023-10-08T00:11:04.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.268+0000 7f3a51aa2700 1 -- 172.21.15.99:0/1922513776 >> 172.21.15.99:0/1922513776 conn(0x7f3a4c06fdc0 msgr2=0x7f3a4c0c40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:04.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.268+0000 7f3a51aa2700 1 -- 172.21.15.99:0/1922513776 shutdown_connections 2023-10-08T00:11:04.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.268+0000 7f3a51aa2700 1 -- 172.21.15.99:0/1922513776 wait complete. 2023-10-08T00:11:04.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.268+0000 7f7e60ff9700 1 -- 172.21.15.99:0/4043311721 <== osd.0 v2:172.21.15.99:6800/86365 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22340 (crc 0 0 0) 0x7f7e640ac530 con 0x7f7e6414cee0 2023-10-08T00:11:04.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.270+0000 7f17f6ffd700 1 -- 172.21.15.99:0/3545565961 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1808003040 con 0x7f18100afd90 2023-10-08T00:11:04.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.271+0000 7f17f6ffd700 1 --2- 172.21.15.99:0/3545565961 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f17fc067000 0x7f17fc0694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.271+0000 7f17f6ffd700 1 -- 172.21.15.99:0/3545565961 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1808086530 con 0x7f18100afd90 2023-10-08T00:11:04.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.271+0000 7f17f6ffd700 1 --2- 172.21.15.99:0/3545565961 >> [v2:172.21.15.191:6817/86612,v1:172.21.15.191:6819/86612] conn(0x7f17fc06f210 0x7f17fc071620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.271+0000 7f17f6ffd700 1 -- 172.21.15.99:0/3545565961 --> [v2:172.21.15.191:6817/86612,v1:172.21.15.191:6819/86612] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f17fc071cd0 con 0x7f17fc06f210 2023-10-08T00:11:04.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.271+0000 7f17f6ffd700 1 -- 172.21.15.99:0/3545565961 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1808086900 con 0x7f18100afd90 2023-10-08T00:11:04.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.271+0000 7f180e7fc700 1 --2- 172.21.15.99:0/3545565961 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f17fc067000 0x7f17fc0694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.272+0000 7f180f7fe700 1 --2- 172.21.15.99:0/3545565961 >> [v2:172.21.15.191:6817/86612,v1:172.21.15.191:6819/86612] conn(0x7f17fc06f210 0x7f17fc071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.272+0000 7f180e7fc700 1 --2- 172.21.15.99:0/3545565961 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f17fc067000 0x7f17fc0694b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1800007fd0 tx=0x7f1800007930 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.272+0000 7f180f7fe700 1 --2- 172.21.15.99:0/3545565961 >> [v2:172.21.15.191:6817/86612,v1:172.21.15.191:6819/86612] conn(0x7f17fc06f210 0x7f17fc071620 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.272+0000 7fe76691e700 1 -- 172.21.15.99:0/3265728988 --> [v2:172.21.15.191:6816/86608,v1:172.21.15.191:6818/86608] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe740002d30 con 0x7fe74c06f210 2023-10-08T00:11:04.313 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.273+0000 7f17f6ffd700 1 -- 172.21.15.99:0/3545565961 <== osd.7 v2:172.21.15.191:6817/86612 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22340 (crc 0 0 0) 0x7f17fc071cd0 con 0x7f17fc06f210 2023-10-08T00:11:04.313 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.279+0000 7fe747fff700 1 -- 172.21.15.99:0/3265728988 <== osd.5 v2:172.21.15.191:6816/86608 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe740002d30 con 0x7fe74c06f210 2023-10-08T00:11:04.313 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.279+0000 7fe76691e700 1 -- 172.21.15.99:0/3265728988 >> [v2:172.21.15.191:6816/86608,v1:172.21.15.191:6818/86608] conn(0x7fe74c06f210 msgr2=0x7fe74c071620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.313 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.279+0000 7fe76691e700 1 --2- 172.21.15.99:0/3265728988 >> [v2:172.21.15.191:6816/86608,v1:172.21.15.191:6818/86608] conn(0x7fe74c06f210 0x7fe74c071620 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.314 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.279+0000 7f7737949700 1 -- 172.21.15.99:0/2677119224 --> [v2:172.21.15.191:6808/86571,v1:172.21.15.191:6809/86571] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f773006f260 con 0x7f771406f210 2023-10-08T00:11:04.314 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.279+0000 7fe76691e700 1 -- 172.21.15.99:0/3265728988 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe74c067000 msgr2=0x7fe74c0694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.314 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.279+0000 7fe76691e700 1 --2- 172.21.15.99:0/3265728988 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe74c067000 0x7fe74c0694b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe7500099b0 tx=0x7fe750017000 comp rx=0 tx=0).stop 2023-10-08T00:11:04.314 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.279+0000 7fe76691e700 1 -- 172.21.15.99:0/3265728988 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe76014ed60 msgr2=0x7fe76014ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.279+0000 7fe76691e700 1 --2- 172.21.15.99:0/3265728988 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe76014ed60 0x7fe76014ce20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe75400c7e0 tx=0x7fe754007df0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.279+0000 7fe764ebb700 1 -- 172.21.15.99:0/3265728988 reap_dead start 2023-10-08T00:11:04.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.279+0000 7fe76691e700 1 -- 172.21.15.99:0/3265728988 shutdown_connections 2023-10-08T00:11:04.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.279+0000 7fe76691e700 1 -- 172.21.15.99:0/3265728988 >> 172.21.15.99:0/3265728988 conn(0x7fe760070700 msgr2=0x7fe76014dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:04.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.279+0000 7f77257fa700 1 -- 172.21.15.99:0/2677119224 <== osd.6 v2:172.21.15.191:6808/86571 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f773006f260 con 0x7f771406f210 2023-10-08T00:11:04.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.280+0000 7f7737949700 1 -- 172.21.15.99:0/2677119224 >> [v2:172.21.15.191:6808/86571,v1:172.21.15.191:6809/86571] conn(0x7f771406f210 msgr2=0x7f7714071620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.280+0000 7f7737949700 1 --2- 172.21.15.99:0/2677119224 >> [v2:172.21.15.191:6808/86571,v1:172.21.15.191:6809/86571] conn(0x7f771406f210 0x7f7714071620 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.280+0000 7f7737949700 1 -- 172.21.15.99:0/2677119224 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7714067000 msgr2=0x7f77140694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.280+0000 7f7737949700 1 --2- 172.21.15.99:0/2677119224 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7714067000 0x7f77140694b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f772c009740 tx=0x7f772c006c60 comp rx=0 tx=0).stop 2023-10-08T00:11:04.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.280+0000 7f7737949700 1 -- 172.21.15.99:0/2677119224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77300a7df0 msgr2=0x7f77301473f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.280+0000 7f7737949700 1 --2- 172.21.15.99:0/2677119224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77300a7df0 0x7f77301473f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7720009290 tx=0x7f7720018000 comp rx=0 tx=0).stop 2023-10-08T00:11:04.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.280+0000 7f7735ee6700 1 -- 172.21.15.99:0/2677119224 reap_dead start 2023-10-08T00:11:04.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.280+0000 7f7737949700 1 -- 172.21.15.99:0/2677119224 shutdown_connections 2023-10-08T00:11:04.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.280+0000 7f7737949700 1 -- 172.21.15.99:0/2677119224 >> 172.21.15.99:0/2677119224 conn(0x7f77300cd1e0 msgr2=0x7f77300aff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:04.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.280+0000 7f7737949700 1 -- 172.21.15.99:0/2677119224 shutdown_connections 2023-10-08T00:11:04.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.280+0000 7f7737949700 1 -- 172.21.15.99:0/2677119224 wait complete. 2023-10-08T00:11:04.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.283+0000 7f22e52ef700 1 -- 172.21.15.99:0/630488656 --> [v2:172.21.15.99:6816/86382,v1:172.21.15.99:6817/86382] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f22c0002d30 con 0x7f22cc06f290 2023-10-08T00:11:04.318 INFO:teuthology.orchestra.run.smithi099.stdout:55834574854 2023-10-08T00:11:04.318 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-08T00:11:04.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.285+0000 7f22c6ffd700 1 -- 172.21.15.99:0/630488656 <== osd.2 v2:172.21.15.99:6816/86382 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f22c0002d30 con 0x7f22cc06f290 2023-10-08T00:11:04.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.286+0000 7f22e52ef700 1 -- 172.21.15.99:0/630488656 >> [v2:172.21.15.99:6816/86382,v1:172.21.15.99:6817/86382] conn(0x7f22cc06f290 msgr2=0x7f22cc0716a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.286+0000 7f22e52ef700 1 --2- 172.21.15.99:0/630488656 >> [v2:172.21.15.99:6816/86382,v1:172.21.15.99:6817/86382] conn(0x7f22cc06f290 0x7f22cc0716a0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.286+0000 7f22e52ef700 1 -- 172.21.15.99:0/630488656 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f22cc067080 msgr2=0x7f22cc069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.286+0000 7f22e52ef700 1 --2- 172.21.15.99:0/630488656 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f22cc067080 0x7f22cc069530 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f22d4006fd0 tx=0x7f22d40090f0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.286+0000 7f22e52ef700 1 -- 172.21.15.99:0/630488656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22e00ccf90 msgr2=0x7f22e00a69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.286+0000 7f22e52ef700 1 --2- 172.21.15.99:0/630488656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22e00ccf90 0x7f22e00a69b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f22d801aab0 tx=0x7f22d8028000 comp rx=0 tx=0).stop 2023-10-08T00:11:04.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.290+0000 7f22df7fe700 1 -- 172.21.15.99:0/630488656 reap_dead start 2023-10-08T00:11:04.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.290+0000 7f22e52ef700 1 -- 172.21.15.99:0/630488656 shutdown_connections 2023-10-08T00:11:04.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.290+0000 7f22e52ef700 1 -- 172.21.15.99:0/630488656 >> 172.21.15.99:0/630488656 conn(0x7f22e0070700 msgr2=0x7f22e0067660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:04.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.290+0000 7f22e52ef700 1 -- 172.21.15.99:0/630488656 shutdown_connections 2023-10-08T00:11:04.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.290+0000 7f22e52ef700 1 -- 172.21.15.99:0/630488656 wait complete. 2023-10-08T00:11:04.321 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.291+0000 7fe76691e700 1 -- 172.21.15.99:0/3265728988 shutdown_connections 2023-10-08T00:11:04.321 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.291+0000 7fe76691e700 1 -- 172.21.15.99:0/3265728988 wait complete. 2023-10-08T00:11:04.321 INFO:teuthology.orchestra.run.smithi099.stdout:55834574854 2023-10-08T00:11:04.322 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-08T00:11:04.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.293+0000 7f7e6aac0700 1 -- 172.21.15.99:0/4043311721 --> [v2:172.21.15.99:6800/86365,v1:172.21.15.99:6801/86365] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7e640008d0 con 0x7f7e6414cee0 2023-10-08T00:11:04.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.293+0000 7f7e60ff9700 1 -- 172.21.15.99:0/4043311721 <== osd.0 v2:172.21.15.99:6800/86365 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7e640008d0 con 0x7f7e6414cee0 2023-10-08T00:11:04.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.294+0000 7f7e6aac0700 1 -- 172.21.15.99:0/4043311721 >> [v2:172.21.15.99:6800/86365,v1:172.21.15.99:6801/86365] conn(0x7f7e6414cee0 msgr2=0x7f7e64150e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.294+0000 7f7e6aac0700 1 --2- 172.21.15.99:0/4043311721 >> [v2:172.21.15.99:6800/86365,v1:172.21.15.99:6801/86365] conn(0x7f7e6414cee0 0x7f7e64150e80 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.294+0000 7f7e6aac0700 1 -- 172.21.15.99:0/4043311721 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7e48066f30 msgr2=0x7f7e480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.294+0000 7f7e6aac0700 1 --2- 172.21.15.99:0/4043311721 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7e48066f30 0x7f7e480693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7e6406fce0 tx=0x7f7e50017000 comp rx=0 tx=0).stop 2023-10-08T00:11:04.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.294+0000 7f7e6aac0700 1 -- 172.21.15.99:0/4043311721 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e640a7df0 msgr2=0x7f7e64098c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.294+0000 7f7e6aac0700 1 --2- 172.21.15.99:0/4043311721 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e640a7df0 0x7f7e64098c00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7e5800ec50 tx=0x7f7e580091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.294+0000 7f7e6905d700 1 -- 172.21.15.99:0/4043311721 reap_dead start 2023-10-08T00:11:04.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.295+0000 7f7e6aac0700 1 -- 172.21.15.99:0/4043311721 shutdown_connections 2023-10-08T00:11:04.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.295+0000 7f7e6aac0700 1 -- 172.21.15.99:0/4043311721 >> 172.21.15.99:0/4043311721 conn(0x7f7e640cd1e0 msgr2=0x7f7e640acaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:04.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.295+0000 7f7e6aac0700 1 -- 172.21.15.99:0/4043311721 shutdown_connections 2023-10-08T00:11:04.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.295+0000 7f7e6aac0700 1 -- 172.21.15.99:0/4043311721 wait complete. 2023-10-08T00:11:04.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.295+0000 7fa43da6d700 1 -- 172.21.15.99:0/1244309695 --> [v2:172.21.15.99:6808/86366,v1:172.21.15.99:6809/86366] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa418002d30 con 0x7fa42406f0c0 2023-10-08T00:11:04.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.295+0000 7fa41f7fe700 1 -- 172.21.15.99:0/1244309695 <== osd.1 v2:172.21.15.99:6808/86366 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa418002d30 con 0x7fa42406f0c0 2023-10-08T00:11:04.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.296+0000 7fa43da6d700 1 -- 172.21.15.99:0/1244309695 >> [v2:172.21.15.99:6808/86366,v1:172.21.15.99:6809/86366] conn(0x7fa42406f0c0 msgr2=0x7fa4240714d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.296+0000 7fa43da6d700 1 --2- 172.21.15.99:0/1244309695 >> [v2:172.21.15.99:6808/86366,v1:172.21.15.99:6809/86366] conn(0x7fa42406f0c0 0x7fa4240714d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.296+0000 7fa43da6d700 1 -- 172.21.15.99:0/1244309695 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa424066f30 msgr2=0x7fa4240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.296+0000 7fa43da6d700 1 --2- 172.21.15.99:0/1244309695 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa424066f30 0x7fa4240693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa4380c8a50 tx=0x7fa42c016000 comp rx=0 tx=0).stop 2023-10-08T00:11:04.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.296+0000 7fa43da6d700 1 -- 172.21.15.99:0/1244309695 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa43806d330 msgr2=0x7fa4380c4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.296+0000 7fa43da6d700 1 --2- 172.21.15.99:0/1244309695 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa43806d330 0x7fa4380c4e20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa42800d860 tx=0x7fa428008040 comp rx=0 tx=0).stop 2023-10-08T00:11:04.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.297+0000 7fa437fff700 1 -- 172.21.15.99:0/1244309695 reap_dead start 2023-10-08T00:11:04.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.297+0000 7fa43da6d700 1 -- 172.21.15.99:0/1244309695 shutdown_connections 2023-10-08T00:11:04.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.297+0000 7fa43da6d700 1 -- 172.21.15.99:0/1244309695 >> 172.21.15.99:0/1244309695 conn(0x7fa4380cc8f0 msgr2=0x7fa4380c9170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:04.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.297+0000 7fa43da6d700 1 -- 172.21.15.99:0/1244309695 shutdown_connections 2023-10-08T00:11:04.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.297+0000 7fa43da6d700 1 -- 172.21.15.99:0/1244309695 wait complete. 2023-10-08T00:11:04.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.297+0000 7f2fafdce700 1 -- 172.21.15.99:0/93772379 --> [v2:172.21.15.99:6818/86393,v1:172.21.15.99:6820/86393] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2fa815e1a0 con 0x7f2f8c06f090 2023-10-08T00:11:04.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.298+0000 7f2f9dffb700 1 -- 172.21.15.99:0/93772379 <== osd.3 v2:172.21.15.99:6818/86393 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2fa815e1a0 con 0x7f2f8c06f090 2023-10-08T00:11:04.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.298+0000 7f2fafdce700 1 -- 172.21.15.99:0/93772379 >> [v2:172.21.15.99:6818/86393,v1:172.21.15.99:6820/86393] conn(0x7f2f8c06f090 msgr2=0x7f2f8c0714a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.298+0000 7f2fafdce700 1 --2- 172.21.15.99:0/93772379 >> [v2:172.21.15.99:6818/86393,v1:172.21.15.99:6820/86393] conn(0x7f2f8c06f090 0x7f2f8c0714a0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.298+0000 7f18158c4700 1 -- 172.21.15.99:0/3545565961 --> [v2:172.21.15.191:6817/86612,v1:172.21.15.191:6819/86612] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1810063ac0 con 0x7f17fc06f210 2023-10-08T00:11:04.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.299+0000 7f2fafdce700 1 -- 172.21.15.99:0/93772379 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2f8c066f00 msgr2=0x7f2f8c0693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.299+0000 7f2fafdce700 1 --2- 172.21.15.99:0/93772379 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2f8c066f00 0x7f2f8c0693b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2fa815a6b0 tx=0x7f2fa0018000 comp rx=0 tx=0).stop 2023-10-08T00:11:04.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.299+0000 7f2fafdce700 1 -- 172.21.15.99:0/93772379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2fa80ccdc0 msgr2=0x7f2fa80c5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.299+0000 7f2fafdce700 1 --2- 172.21.15.99:0/93772379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2fa80ccdc0 0x7f2fa80c5eb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2fa400b600 tx=0x7f2fa40091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.299+0000 7f17f6ffd700 1 -- 172.21.15.99:0/3545565961 <== osd.7 v2:172.21.15.191:6817/86612 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1810063ac0 con 0x7f17fc06f210 2023-10-08T00:11:04.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.299+0000 7f18158c4700 1 -- 172.21.15.99:0/3545565961 >> [v2:172.21.15.191:6817/86612,v1:172.21.15.191:6819/86612] conn(0x7f17fc06f210 msgr2=0x7f17fc071620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.299+0000 7f18158c4700 1 --2- 172.21.15.99:0/3545565961 >> [v2:172.21.15.191:6817/86612,v1:172.21.15.191:6819/86612] conn(0x7f17fc06f210 0x7f17fc071620 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.299+0000 7f18158c4700 1 -- 172.21.15.99:0/3545565961 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f17fc067000 msgr2=0x7f17fc0694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.299+0000 7f18158c4700 1 --2- 172.21.15.99:0/3545565961 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f17fc067000 0x7f17fc0694b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1800007fd0 tx=0x7f1800007930 comp rx=0 tx=0).stop 2023-10-08T00:11:04.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.299+0000 7f18158c4700 1 -- 172.21.15.99:0/3545565961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18100afd90 msgr2=0x7f181006e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.299+0000 7f18158c4700 1 --2- 172.21.15.99:0/3545565961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18100afd90 0x7f181006e860 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f18100899e0 tx=0x7f1808018000 comp rx=0 tx=0).stop 2023-10-08T00:11:04.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.299+0000 7f180f7fe700 1 -- 172.21.15.99:0/3545565961 reap_dead start 2023-10-08T00:11:04.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.304+0000 7f18158c4700 1 -- 172.21.15.99:0/3545565961 shutdown_connections 2023-10-08T00:11:04.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.304+0000 7f18158c4700 1 -- 172.21.15.99:0/3545565961 >> 172.21.15.99:0/3545565961 conn(0x7f1810068ce0 msgr2=0x7f18100690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:04.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.304+0000 7f2fae36b700 1 -- 172.21.15.99:0/93772379 reap_dead start 2023-10-08T00:11:04.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.304+0000 7f2fafdce700 1 -- 172.21.15.99:0/93772379 shutdown_connections 2023-10-08T00:11:04.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.304+0000 7f2fafdce700 1 -- 172.21.15.99:0/93772379 >> 172.21.15.99:0/93772379 conn(0x7f2fa8070700 msgr2=0x7f2fa80c92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:04.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.304+0000 7f18158c4700 1 -- 172.21.15.99:0/3545565961 shutdown_connections 2023-10-08T00:11:04.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.304+0000 7f18158c4700 1 -- 172.21.15.99:0/3545565961 wait complete. 2023-10-08T00:11:04.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.304+0000 7f2fafdce700 1 -- 172.21.15.99:0/93772379 shutdown_connections 2023-10-08T00:11:04.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.304+0000 7f2fafdce700 1 -- 172.21.15.99:0/93772379 wait complete. 2023-10-08T00:11:04.333 INFO:teuthology.orchestra.run.smithi099.stdout:55834574854 2023-10-08T00:11:04.333 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-08T00:11:04.334 INFO:teuthology.orchestra.run.smithi099.stdout:55834574854 2023-10-08T00:11:04.334 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-08T00:11:04.335 INFO:teuthology.orchestra.run.smithi099.stdout:55834574854 2023-10-08T00:11:04.335 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-08T00:11:04.336 INFO:teuthology.orchestra.run.smithi099.stdout:55834574854 2023-10-08T00:11:04.336 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-08T00:11:04.337 INFO:teuthology.orchestra.run.smithi099.stdout:55834574854 2023-10-08T00:11:04.337 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-08T00:11:04.338 INFO:teuthology.orchestra.run.smithi099.stdout:55834574854 2023-10-08T00:11:04.338 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-08T00:11:04.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.569+0000 7f4d109f2700 1 Processor -- start 2023-10-08T00:11:04.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.570+0000 7f8f4c311700 1 Processor -- start 2023-10-08T00:11:04.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.572+0000 7fafc166f700 1 Processor -- start 2023-10-08T00:11:04.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.571+0000 7fe82a08c700 1 Processor -- start 2023-10-08T00:11:04.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.571+0000 7fe82a08c700 1 -- start start 2023-10-08T00:11:04.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.572+0000 7fe82a08c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe8240aff50 0x7fe8240c2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.572+0000 7fe82a08c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe824070e10 0x7fe824162be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.572+0000 7fe82a08c700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe824163120 con 0x7fe8240c9bb0 2023-10-08T00:11:04.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.572+0000 7fe82a08c700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8240ca370 con 0x7fe8240aff50 2023-10-08T00:11:04.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.572+0000 7fe82a08c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe824069030 con 0x7fe824070e10 2023-10-08T00:11:04.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.572+0000 7fe8237fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe8240aff50 0x7fe8240c2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.572+0000 7fe8237fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe8240aff50 0x7fe8240c2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38720/0 (socket says 172.21.15.99:38720) 2023-10-08T00:11:04.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.572+0000 7fe823fff700 1 --2- 172.21.15.99:0/3628064203 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe824070e10 0x7fe824162be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.572+0000 7fe8237fe700 1 -- 172.21.15.99:0/3628064203 learned_addr learned my addr 172.21.15.99:0/3628064203 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:04.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.570+0000 7f8f4c311700 1 -- start start 2023-10-08T00:11:04.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.572+0000 7f8f4c311700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f44157ed0 0x7f8f4415b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.572+0000 7f8f4c311700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8f440c7820 0x7f8f440c24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.572+0000 7f8f4c311700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f440c2010 con 0x7f8f44157b00 2023-10-08T00:11:04.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.572+0000 7f8f4c311700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f4415bed0 con 0x7f8f440c7820 2023-10-08T00:11:04.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.572+0000 7f8f4c311700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4415c1b0 con 0x7f8f44157ed0 2023-10-08T00:11:04.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.570+0000 7fa7e8aec700 1 Processor -- start 2023-10-08T00:11:04.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.570+0000 7fa7e8aec700 1 -- start start 2023-10-08T00:11:04.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.571+0000 7fa7e8aec700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7e40bfbe0 0x7fa7e40ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.571+0000 7fa7e8aec700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7e40cd620 con 0x7fa7e40cd1d0 2023-10-08T00:11:04.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.571+0000 7fa7e8aec700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7e40b0380 con 0x7fa7e40afdf0 2023-10-08T00:11:04.577 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.571+0000 7fa7e8aec700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e415e9c0 con 0x7fa7e40bfbe0 2023-10-08T00:11:04.577 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.571+0000 7fa7e1d9b700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fa7e40cd1d0 0x7fa7e40af6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45028/0 (socket says 172.21.15.99:45028) 2023-10-08T00:11:04.577 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.571+0000 7fa7e1d9b700 1 -- 172.21.15.99:0/1517128756 learned_addr learned my addr 172.21.15.99:0/1517128756 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:04.577 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.572+0000 7fa7e259c700 1 --2- 172.21.15.99:0/1517128756 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7e40bfbe0 0x7fa7e40ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.577 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.572+0000 7fa7e159a700 1 -- 172.21.15.99:0/1517128756 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 150005785 0 0) 0x7fa7e40cd620 con 0x7fa7e40cd1d0 2023-10-08T00:11:04.577 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.572+0000 7fa7e159a700 1 -- 172.21.15.99:0/1517128756 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7cc003570 con 0x7fa7e40cd1d0 2023-10-08T00:11:04.577 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.570+0000 7f4d109f2700 1 -- start start 2023-10-08T00:11:04.577 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.571+0000 7f4d109f2700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4d0c0ca630 0x7f4d0c0caa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.577 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.571+0000 7f4d109f2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d0c0c89d0 0x7f4d0c1566d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.577 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.571+0000 7f4d109f2700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d0c06e650 con 0x7f4d0c0adb20 2023-10-08T00:11:04.578 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.571+0000 7f4d109f2700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d0c067cd0 con 0x7f4d0c0ca630 2023-10-08T00:11:04.578 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.571+0000 7f4d109f2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0c0679a0 con 0x7f4d0c0c89d0 2023-10-08T00:11:04.578 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.571+0000 7f4d09d9b700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f4d0c0adb20 0x7f4d0c0c82c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:56072/0 (socket says 172.21.15.99:56072) 2023-10-08T00:11:04.578 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.571+0000 7f4d09d9b700 1 -- 172.21.15.99:0/1117371816 learned_addr learned my addr 172.21.15.99:0/1117371816 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:04.578 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.571+0000 7f4d0a59c700 1 --2- 172.21.15.99:0/1117371816 >> v2:172.21.15.191:3300/0 conn(0x7f4d0c0ca630 0x7f4d0c0caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.578 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.572+0000 7f4d0ad9d700 1 --2- 172.21.15.99:0/1117371816 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d0c0c89d0 0x7f4d0c1566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.578 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.572+0000 7f4d0959a700 1 -- 172.21.15.99:0/1117371816 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2564468049 0 0) 0x7f4d0c06e650 con 0x7f4d0c0adb20 2023-10-08T00:11:04.578 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.572+0000 7f4d0a59c700 1 -- 172.21.15.99:0/1117371816 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d0c0c89d0 msgr2=0x7f4d0c1566d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:04.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.572+0000 7f4d0a59c700 1 --2- 172.21.15.99:0/1117371816 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d0c0c89d0 0x7f4d0c1566d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.572+0000 7f4d0a59c700 1 -- 172.21.15.99:0/1117371816 >> v1:172.21.15.191:6789/0 conn(0x7f4d0c0adb20 legacy=0x7f4d0c0c82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.572+0000 7f4d0a59c700 1 -- 172.21.15.99:0/1117371816 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d0c0c8ea0 con 0x7f4d0c0ca630 2023-10-08T00:11:04.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.572+0000 7f4d0ad9d700 1 --2- 172.21.15.99:0/1117371816 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d0c0c89d0 0x7f4d0c1566d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:11:04.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.572+0000 7fa7e259c700 1 -- 172.21.15.99:0/1517128756 >> v1:172.21.15.191:6789/0 conn(0x7fa7e40afdf0 legacy=0x7fa7e40b1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.572+0000 7fa7e259c700 1 -- 172.21.15.99:0/1517128756 >> v1:172.21.15.99:6789/0 conn(0x7fa7e40cd1d0 legacy=0x7fa7e40af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.572+0000 7fa7e259c700 1 -- 172.21.15.99:0/1517128756 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e40b0660 con 0x7fa7e40bfbe0 2023-10-08T00:11:04.580 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.572+0000 7f4d0a59c700 1 --2- 172.21.15.99:0/1117371816 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d0c0ca630 0x7f4d0c0caa00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc0098e0 tx=0x7f4cfc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9b9bd655b9f0f40 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.580 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.572+0000 7f8f498ac700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f44157ed0 0x7f8f4415b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.580 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.572+0000 7f8f498ac700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f44157ed0 0x7f8f4415b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.99:45622/0 (socket says 172.21.15.99:45622) 2023-10-08T00:11:04.580 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.572+0000 7f8f498ac700 1 -- 172.21.15.99:0/3373737130 learned_addr learned my addr 172.21.15.99:0/3373737130 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:04.580 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.572+0000 7f4d0959a700 1 -- 172.21.15.99:0/1117371816 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc009e80 con 0x7f4d0c0ca630 2023-10-08T00:11:04.581 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.573+0000 7f4d0959a700 1 -- 172.21.15.99:0/1117371816 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cfc00b860 con 0x7f4d0c0ca630 2023-10-08T00:11:04.581 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.573+0000 7f4d0959a700 1 -- 172.21.15.99:0/1117371816 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc00bbe0 con 0x7f4d0c0ca630 2023-10-08T00:11:04.581 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.573+0000 7fa7e259c700 1 --2- 172.21.15.99:0/1517128756 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7e40bfbe0 0x7fa7e40ccc90 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4009910 tx=0x7fa7d40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4dd0d8845d2bba38 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.581 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.573+0000 7fe8227fc700 1 -- 172.21.15.99:0/3628064203 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3236867783 0 0) 0x7fe824163120 con 0x7fe8240c9bb0 2023-10-08T00:11:04.581 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.573+0000 7fe8237fe700 1 -- 172.21.15.99:0/3628064203 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe824070e10 msgr2=0x7fe824162be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:04.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.573+0000 7fe8237fe700 1 --2- 172.21.15.99:0/3628064203 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe824070e10 0x7fe824162be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.573+0000 7fe8237fe700 1 -- 172.21.15.99:0/3628064203 >> v1:172.21.15.191:6789/0 conn(0x7fe8240c9bb0 legacy=0x7fe824070700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.573+0000 7fe8237fe700 1 -- 172.21.15.99:0/3628064203 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe82415eaf0 con 0x7fe8240aff50 2023-10-08T00:11:04.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.573+0000 7f8f498ac700 1 -- 172.21.15.99:0/3373737130 >> v2:172.21.15.99:3300/0 conn(0x7f8f440c7820 msgr2=0x7f8f440c24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:04.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.573+0000 7f8f498ac700 1 --2- 172.21.15.99:0/3373737130 >> v2:172.21.15.99:3300/0 conn(0x7f8f440c7820 0x7f8f440c24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.573+0000 7f8f498ac700 1 -- 172.21.15.99:0/3373737130 >> v1:172.21.15.99:6789/0 conn(0x7f8f44157b00 legacy=0x7f8f4406ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:04.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.573+0000 7f8f498ac700 1 -- 172.21.15.99:0/3373737130 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f4415c490 con 0x7f8f44157ed0 2023-10-08T00:11:04.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.573+0000 7fa7e159a700 1 -- 172.21.15.99:0/1517128756 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d400b680 con 0x7fa7e40bfbe0 2023-10-08T00:11:04.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.573+0000 7fe8237fe700 1 --2- 172.21.15.99:0/3628064203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8240aff50 0x7fe8240c2820 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe8180099e0 tx=0x7fe818009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a8de6e78a2df4c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.573+0000 7fe8227fc700 1 -- 172.21.15.99:0/3628064203 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81800b820 con 0x7fe8240aff50 2023-10-08T00:11:04.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.573+0000 7fe8227fc700 1 -- 172.21.15.99:0/3628064203 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe81800b980 con 0x7fe8240aff50 2023-10-08T00:11:04.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.573+0000 7fe8227fc700 1 -- 172.21.15.99:0/3628064203 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81800bd40 con 0x7fe8240aff50 2023-10-08T00:11:04.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.572+0000 7fafc166f700 1 -- start start 2023-10-08T00:11:04.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.573+0000 7fafc166f700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fafbc0a7da0 0x7fafbc162600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.573+0000 7fafc166f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fafbc15e5a0 0x7fafbc097bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.573+0000 7fafc166f700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafbc0b0420 con 0x7fafbc15e1d0 2023-10-08T00:11:04.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.573+0000 7fafc166f700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafbc0b2bf0 con 0x7fafbc0a7da0 2023-10-08T00:11:04.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.573+0000 7fafc166f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafbc0b0140 con 0x7fafbc15e5a0 2023-10-08T00:11:04.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.574+0000 7fafba7fc700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fafbc0a7da0 0x7fafbc162600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.574+0000 7fafbb7fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fafbc15e5a0 0x7fafbc097bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.573+0000 7f4d109f2700 1 -- 172.21.15.99:0/1117371816 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d0c0ca630 msgr2=0x7f4d0c0caa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.574+0000 7f4d109f2700 1 --2- 172.21.15.99:0/1117371816 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d0c0ca630 0x7f4d0c0caa00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc0098e0 tx=0x7f4cfc0091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.574+0000 7fafba7fc700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fafbc0a7da0 0x7fafbc162600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38726/0 (socket says 172.21.15.99:38726) 2023-10-08T00:11:04.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.574+0000 7fafba7fc700 1 -- 172.21.15.99:0/3611199131 learned_addr learned my addr 172.21.15.99:0/3611199131 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:04.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.574+0000 7f867c778700 1 Processor -- start 2023-10-08T00:11:04.586 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.573+0000 7f8f498ac700 1 --2- 172.21.15.99:0/3373737130 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f44157ed0 0x7f8f4415b8f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8f340099b0 tx=0x7f8f34009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84e3c27df65f0647 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.586 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.573+0000 7f8f490ab700 1 -- 172.21.15.99:0/3373737130 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3400b820 con 0x7f8f44157ed0 2023-10-08T00:11:04.586 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.573+0000 7f8f490ab700 1 -- 172.21.15.99:0/3373737130 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f3400b980 con 0x7f8f44157ed0 2023-10-08T00:11:04.586 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.573+0000 7f8f490ab700 1 -- 172.21.15.99:0/3373737130 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3400bd20 con 0x7f8f44157ed0 2023-10-08T00:11:04.586 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.573+0000 7fa7e159a700 1 -- 172.21.15.99:0/1517128756 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d400b7e0 con 0x7fa7e40bfbe0 2023-10-08T00:11:04.586 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.573+0000 7fa7e159a700 1 -- 172.21.15.99:0/1517128756 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d400bb60 con 0x7fa7e40bfbe0 2023-10-08T00:11:04.586 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.574+0000 7fa7e8aec700 1 -- 172.21.15.99:0/1517128756 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7e40bfbe0 msgr2=0x7fa7e40ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.587 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.574+0000 7fa7e8aec700 1 --2- 172.21.15.99:0/1517128756 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7e40bfbe0 0x7fa7e40ccc90 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4009910 tx=0x7fa7d40091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.587 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.574+0000 7fafba7fc700 1 -- 172.21.15.99:0/3611199131 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fafbc15e5a0 msgr2=0x7fafbc097bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:04.587 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.574+0000 7fafba7fc700 1 --2- 172.21.15.99:0/3611199131 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fafbc15e5a0 0x7fafbc097bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.587 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.574+0000 7fafba7fc700 1 -- 172.21.15.99:0/3611199131 >> v1:172.21.15.99:6789/0 conn(0x7fafbc15e1d0 legacy=0x7fafbc161ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.587 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.574+0000 7fafba7fc700 1 -- 172.21.15.99:0/3611199131 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafbc0b2530 con 0x7fafbc0a7da0 2023-10-08T00:11:04.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.574+0000 7f8f4c311700 1 -- 172.21.15.99:0/3373737130 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f44157ed0 msgr2=0x7f8f4415b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.574+0000 7f8f4c311700 1 --2- 172.21.15.99:0/3373737130 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f44157ed0 0x7f8f4415b8f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8f340099b0 tx=0x7f8f34009250 comp rx=0 tx=0).stop 2023-10-08T00:11:04.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.574+0000 7fafba7fc700 1 --2- 172.21.15.99:0/3611199131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafbc0a7da0 0x7fafbc162600 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fafb4009f20 tx=0x7fafb4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55be354c9e42ec29 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.574+0000 7fe82a08c700 1 -- 172.21.15.99:0/3628064203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8240aff50 msgr2=0x7fe8240c2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.574+0000 7fe82a08c700 1 --2- 172.21.15.99:0/3628064203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8240aff50 0x7fe8240c2820 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe8180099e0 tx=0x7fe818009250 comp rx=0 tx=0).stop 2023-10-08T00:11:04.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.574+0000 7fafb9ffb700 1 -- 172.21.15.99:0/3611199131 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb400e070 con 0x7fafbc0a7da0 2023-10-08T00:11:04.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.574+0000 7fa7e8aec700 1 -- 172.21.15.99:0/1517128756 shutdown_connections 2023-10-08T00:11:04.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.574+0000 7fa7e8aec700 1 --2- 172.21.15.99:0/1517128756 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7e40bfbe0 0x7fa7e40ccc90 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.574+0000 7fa7e8aec700 1 -- 172.21.15.99:0/1517128756 >> 172.21.15.99:0/1517128756 conn(0x7fa7e4070700 msgr2=0x7fa7e4070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:04.590 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.574+0000 7f4d109f2700 1 -- 172.21.15.99:0/1117371816 shutdown_connections 2023-10-08T00:11:04.590 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.574+0000 7f4d109f2700 1 --2- 172.21.15.99:0/1117371816 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d0c0c89d0 0x7f4d0c1566d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.590 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.574+0000 7f4d109f2700 1 --2- 172.21.15.99:0/1117371816 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d0c0ca630 0x7f4d0c0caa00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.590 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.574+0000 7f4d109f2700 1 -- 172.21.15.99:0/1117371816 >> 172.21.15.99:0/1117371816 conn(0x7f4d0c06a250 msgr2=0x7f4d0c06a650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:04.590 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.575+0000 7fa7e8aec700 1 -- 172.21.15.99:0/1517128756 shutdown_connections 2023-10-08T00:11:04.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.575+0000 7f4d109f2700 1 -- 172.21.15.99:0/1117371816 shutdown_connections 2023-10-08T00:11:04.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.575+0000 7fa7e8aec700 1 -- 172.21.15.99:0/1517128756 wait complete. 2023-10-08T00:11:04.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.575+0000 7f4d109f2700 1 -- 172.21.15.99:0/1117371816 wait complete. 2023-10-08T00:11:04.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.574+0000 7fafb9ffb700 1 -- 172.21.15.99:0/3611199131 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb400b9d0 con 0x7fafbc0a7da0 2023-10-08T00:11:04.592 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.574+0000 7fafb9ffb700 1 -- 172.21.15.99:0/3611199131 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb40092e0 con 0x7fafbc0a7da0 2023-10-08T00:11:04.592 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.574+0000 7f867c778700 1 -- start start 2023-10-08T00:11:04.592 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.575+0000 7f867c778700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f867415e450 0x7f86740b6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.592 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.575+0000 7f867c778700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86740b6cc0 con 0x7f867415e080 2023-10-08T00:11:04.592 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.575+0000 7f867c778700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86740b56d0 con 0x7f86740afd90 2023-10-08T00:11:04.592 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.575+0000 7f867c778700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86740b0230 con 0x7f867415e450 2023-10-08T00:11:04.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.575+0000 7fafc166f700 1 -- 172.21.15.99:0/3611199131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafbc0a7da0 msgr2=0x7fafbc162600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.575+0000 7fafc166f700 1 --2- 172.21.15.99:0/3611199131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafbc0a7da0 0x7fafbc162600 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fafb4009f20 tx=0x7fafb4009840 comp rx=0 tx=0).stop 2023-10-08T00:11:04.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.575+0000 7f867a514700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f867415e080 0x7f86740b5b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45042/0 (socket says 172.21.15.99:45042) 2023-10-08T00:11:04.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.575+0000 7f867a514700 1 -- 172.21.15.99:0/3533110163 learned_addr learned my addr 172.21.15.99:0/3533110163 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:04.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.576+0000 7f8679512700 1 -- 172.21.15.99:0/3533110163 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3211403900 0 0) 0x7f86740b6cc0 con 0x7f867415e080 2023-10-08T00:11:04.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.576+0000 7f8679512700 1 -- 172.21.15.99:0/3533110163 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8660003570 con 0x7f867415e080 2023-10-08T00:11:04.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.575+0000 7fe82a08c700 1 -- 172.21.15.99:0/3628064203 shutdown_connections 2023-10-08T00:11:04.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.575+0000 7fe82a08c700 1 --2- 172.21.15.99:0/3628064203 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe824070e10 0x7fe824162be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.575+0000 7fe82a08c700 1 --2- 172.21.15.99:0/3628064203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8240aff50 0x7fe8240c2820 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.576+0000 7fe82a08c700 1 -- 172.21.15.99:0/3628064203 >> 172.21.15.99:0/3628064203 conn(0x7fe8240ca4f0 msgr2=0x7fe8240ca8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:04.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.576+0000 7f4d109f2700 1 Processor -- start 2023-10-08T00:11:04.595 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.576+0000 7f8679512700 1 -- 172.21.15.99:0/3533110163 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3931804337 0 0) 0x7f86740b56d0 con 0x7f86740afd90 2023-10-08T00:11:04.595 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.576+0000 7f8679512700 1 -- 172.21.15.99:0/3533110163 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86740b6cc0 con 0x7f86740afd90 2023-10-08T00:11:04.595 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.576+0000 7f4d109f2700 1 -- start start 2023-10-08T00:11:04.595 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.576+0000 7f8679512700 1 -- 172.21.15.99:0/3533110163 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2373158880 0 0) 0x7f8660003570 con 0x7f867415e080 2023-10-08T00:11:04.596 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.576+0000 7f8679512700 1 -- 172.21.15.99:0/3533110163 >> v2:172.21.15.191:3300/0 conn(0x7f867415e450 msgr2=0x7f86740b6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:04.596 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.576+0000 7f8679512700 1 --2- 172.21.15.99:0/3533110163 >> v2:172.21.15.191:3300/0 conn(0x7f867415e450 0x7f86740b6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.596 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.576+0000 7f8679512700 1 -- 172.21.15.99:0/3533110163 >> v1:172.21.15.191:6789/0 conn(0x7f86740afd90 legacy=0x7f867406a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.596 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.576+0000 7f8679512700 1 -- 172.21.15.99:0/3533110163 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86740b4da0 con 0x7f867415e080 2023-10-08T00:11:04.596 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.576+0000 7fa7e8aec700 1 Processor -- start 2023-10-08T00:11:04.596 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.576+0000 7f8679512700 1 -- 172.21.15.99:0/3533110163 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f8670001470 con 0x7f867415e080 2023-10-08T00:11:04.597 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.575+0000 7f8f4c311700 1 -- 172.21.15.99:0/3373737130 shutdown_connections 2023-10-08T00:11:04.597 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.575+0000 7f8f4c311700 1 --2- 172.21.15.99:0/3373737130 >> v2:172.21.15.99:3300/0 conn(0x7f8f440c7820 0x7f8f440c24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.597 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.575+0000 7f8f4c311700 1 --2- 172.21.15.99:0/3373737130 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f44157ed0 0x7f8f4415b8f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.597 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.575+0000 7f8f4c311700 1 -- 172.21.15.99:0/3373737130 >> 172.21.15.99:0/3373737130 conn(0x7f8f440cd1e0 msgr2=0x7f8f440cd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:04.597 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.576+0000 7f8f4c311700 1 -- 172.21.15.99:0/3373737130 shutdown_connections 2023-10-08T00:11:04.597 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.576+0000 7f8679512700 1 -- 172.21.15.99:0/3533110163 >> v1:172.21.15.99:6789/0 conn(0x7f867415e080 legacy=0x7f86740b5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.598 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.576+0000 7f8679512700 1 --2- 172.21.15.99:0/3533110163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8660003bf0 0x7f8660003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.598 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.576+0000 7fa7e8aec700 1 -- start start 2023-10-08T00:11:04.598 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.576+0000 7f8679512700 1 --2- 172.21.15.99:0/3533110163 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8660004810 0x7f8660004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.598 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.576+0000 7f8679512700 1 --2- 172.21.15.99:0/3533110163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8660005310 0x7f866000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.576+0000 7f8679512700 1 -- 172.21.15.99:0/3533110163 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86740b0230 con 0x7f8660005310 2023-10-08T00:11:04.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.576+0000 7f8679512700 1 -- 172.21.15.99:0/3533110163 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866000c2d0 con 0x7f8660003bf0 2023-10-08T00:11:04.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.576+0000 7f8679512700 1 -- 172.21.15.99:0/3533110163 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f866000c490 con 0x7f8660004810 2023-10-08T00:11:04.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.576+0000 7f867ad15700 1 --2- 172.21.15.99:0/3533110163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8660005310 0x7f866000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.577+0000 7fafc166f700 1 -- 172.21.15.99:0/3611199131 shutdown_connections 2023-10-08T00:11:04.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.577+0000 7fafc166f700 1 --2- 172.21.15.99:0/3611199131 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fafbc15e5a0 0x7fafbc097bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.577+0000 7fafc166f700 1 --2- 172.21.15.99:0/3611199131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafbc0a7da0 0x7fafbc162600 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.577+0000 7fafc166f700 1 -- 172.21.15.99:0/3611199131 >> 172.21.15.99:0/3611199131 conn(0x7fafbc0cd1e0 msgr2=0x7fafbc0cd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:04.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.577+0000 7f867ad15700 1 -- 172.21.15.99:0/3533110163 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8660004810 msgr2=0x7f8660004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:04.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.577+0000 7f867ad15700 1 --2- 172.21.15.99:0/3533110163 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8660004810 0x7f8660004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.577+0000 7f867ad15700 1 -- 172.21.15.99:0/3533110163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8660003bf0 msgr2=0x7f8660003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:04.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.577+0000 7f867ad15700 1 --2- 172.21.15.99:0/3533110163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8660003bf0 0x7f8660003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.577+0000 7f867ad15700 1 -- 172.21.15.99:0/3533110163 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8660003570 con 0x7f8660005310 2023-10-08T00:11:04.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.577+0000 7f867ad15700 1 -- 172.21.15.99:0/3533110163 reap_dead start 2023-10-08T00:11:04.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.577+0000 7f867ad15700 1 --2- 172.21.15.99:0/3533110163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8660005310 0x7f866000ba60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f866c002140 tx=0x7f866c0017c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23bc24241ae1865c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.601 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.577+0000 7f4d109f2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d0c0adb20 0x7f4d0c14fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.601 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.577+0000 7f4d109f2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d0c0c89d0 0x7f4d0c0c4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.601 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.577+0000 7f4d109f2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d0c0ca630 0x7f4d0c0cca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.601 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.577+0000 7f4d109f2700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0c150010 con 0x7f4d0c0adb20 2023-10-08T00:11:04.601 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.577+0000 7f4d109f2700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0c0c5060 con 0x7f4d0c0c89d0 2023-10-08T00:11:04.601 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.577+0000 7f4d109f2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0c166c30 con 0x7f4d0c0ca630 2023-10-08T00:11:04.601 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.576+0000 7fe82a08c700 1 -- 172.21.15.99:0/3628064203 shutdown_connections 2023-10-08T00:11:04.602 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.576+0000 7fe82a08c700 1 -- 172.21.15.99:0/3628064203 wait complete. 2023-10-08T00:11:04.602 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.577+0000 7fa7e8aec700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7e40afdf0 0x7fa7e40c39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.602 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.577+0000 7fa7e8aec700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7e40bfbe0 0x7fa7e416f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.602 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.577+0000 7fa7e8aec700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7e40cd1d0 0x7fa7e40c4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.602 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.577+0000 7f4d0a59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d0c0adb20 0x7f4d0c14fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.603 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.577+0000 7f8679512700 1 -- 172.21.15.99:0/3533110163 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866c00c980 con 0x7f8660005310 2023-10-08T00:11:04.603 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.577+0000 7f8679512700 1 -- 172.21.15.99:0/3533110163 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f866c00cb20 con 0x7f8660005310 2023-10-08T00:11:04.603 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.577+0000 7f8679512700 1 -- 172.21.15.99:0/3533110163 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866c012420 con 0x7f8660005310 2023-10-08T00:11:04.603 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.577+0000 7fa7e8aec700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e40c58f0 con 0x7fa7e40afdf0 2023-10-08T00:11:04.603 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.577+0000 7fa7e8aec700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e40c6470 con 0x7fa7e40cd1d0 2023-10-08T00:11:04.603 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.577+0000 7fa7e8aec700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e40c3f10 con 0x7fa7e40bfbe0 2023-10-08T00:11:04.604 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.577+0000 7f4d09d9b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d0c0c89d0 0x7f4d0c0c4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.604 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.577+0000 7f4d0a59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d0c0adb20 0x7f4d0c14fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38738/0 (socket says 172.21.15.99:38738) 2023-10-08T00:11:04.604 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.577+0000 7f4d0a59c700 1 -- 172.21.15.99:0/515141564 learned_addr learned my addr 172.21.15.99:0/515141564 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:04.604 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.578+0000 7fafc166f700 1 -- 172.21.15.99:0/3611199131 shutdown_connections 2023-10-08T00:11:04.604 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.577+0000 7fa7e1d9b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7e40bfbe0 0x7fa7e416f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.604 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.577+0000 7fa7e2d9d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7e40cd1d0 0x7fa7e40c4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.604 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.577+0000 7fa7e259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7e40afdf0 0x7fa7e40c39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.605 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.577+0000 7fa7e259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7e40afdf0 0x7fa7e40c39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38744/0 (socket says 172.21.15.99:38744) 2023-10-08T00:11:04.605 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.577+0000 7f4d0ad9d700 1 --2- 172.21.15.99:0/515141564 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d0c0ca630 0x7f4d0c0cca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.605 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.577+0000 7fa7e259c700 1 -- 172.21.15.99:0/4246690376 learned_addr learned my addr 172.21.15.99:0/4246690376 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:04.605 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.578+0000 7f8f4c311700 1 -- 172.21.15.99:0/3373737130 wait complete. 2023-10-08T00:11:04.606 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.578+0000 7f4d09d9b700 1 -- 172.21.15.99:0/515141564 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d0c0ca630 msgr2=0x7f4d0c0cca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.606 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.578+0000 7f4d09d9b700 1 --2- 172.21.15.99:0/515141564 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d0c0ca630 0x7f4d0c0cca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.606 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.578+0000 7f4d09d9b700 1 -- 172.21.15.99:0/515141564 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d0c0adb20 msgr2=0x7f4d0c14fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.606 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.578+0000 7f4d09d9b700 1 --2- 172.21.15.99:0/515141564 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d0c0adb20 0x7f4d0c14fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.606 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.578+0000 7f4d09d9b700 1 -- 172.21.15.99:0/515141564 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cfc009420 con 0x7f4d0c0c89d0 2023-10-08T00:11:04.606 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.577+0000 7ff91df78700 1 Processor -- start 2023-10-08T00:11:04.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.577+0000 7ff91df78700 1 -- start start 2023-10-08T00:11:04.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.578+0000 7ff91df78700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff9180bfbe0 0x7ff9180ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.578+0000 7ff91df78700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9180cd1d0 0x7ff9180af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.578+0000 7ff91df78700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff9180afc20 0x7ff918162050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.578+0000 7ff91df78700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff91815e9c0 con 0x7ff9180bfbe0 2023-10-08T00:11:04.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.578+0000 7ff91df78700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9180cd620 con 0x7ff9180afc20 2023-10-08T00:11:04.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.578+0000 7ff91df78700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9180b0380 con 0x7ff9180cd1d0 2023-10-08T00:11:04.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.578+0000 7fa7e259c700 1 -- 172.21.15.99:0/4246690376 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7e40bfbe0 msgr2=0x7fa7e416f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.608 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.578+0000 7fa7e259c700 1 --2- 172.21.15.99:0/4246690376 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7e40bfbe0 0x7fa7e416f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.608 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.578+0000 7fa7e259c700 1 -- 172.21.15.99:0/4246690376 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7e40cd1d0 msgr2=0x7fa7e40c4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.608 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.578+0000 7ff9177fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff9180bfbe0 0x7ff9180ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.608 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.578+0000 7fa7e259c700 1 --2- 172.21.15.99:0/4246690376 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7e40cd1d0 0x7fa7e40c4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.608 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.578+0000 7fa7e259c700 1 -- 172.21.15.99:0/4246690376 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d4009420 con 0x7fa7e40afdf0 2023-10-08T00:11:04.609 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.578+0000 7f867c778700 1 -- 172.21.15.99:0/3533110163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8660005310 msgr2=0x7f866000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.609 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.578+0000 7f867c778700 1 --2- 172.21.15.99:0/3533110163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8660005310 0x7f866000ba60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f866c002140 tx=0x7f866c0017c0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.609 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.578+0000 7ff9177fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff9180bfbe0 0x7ff9180ccc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38750/0 (socket says 172.21.15.99:38750) 2023-10-08T00:11:04.609 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.578+0000 7ff9177fe700 1 -- 172.21.15.99:0/2599693473 learned_addr learned my addr 172.21.15.99:0/2599693473 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:04.609 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.578+0000 7ff917fff700 1 --2- 172.21.15.99:0/2599693473 >> v2:172.21.15.191:3300/0 conn(0x7ff9180afc20 0x7ff918162050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.609 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.578+0000 7ff916ffd700 1 --2- 172.21.15.99:0/2599693473 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9180cd1d0 0x7ff9180af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.610 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.578+0000 7f4d09d9b700 1 --2- 172.21.15.99:0/515141564 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d0c0c89d0 0x7f4d0c0c4b20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4d0000c5d0 tx=0x7f4d00007d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.610 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.578+0000 7ff9177fe700 1 -- 172.21.15.99:0/2599693473 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9180cd1d0 msgr2=0x7ff9180af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:04.610 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.578+0000 7ff9177fe700 1 --2- 172.21.15.99:0/2599693473 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9180cd1d0 0x7ff9180af6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.610 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.578+0000 7ff9177fe700 1 -- 172.21.15.99:0/2599693473 >> v2:172.21.15.191:3300/0 conn(0x7ff9180afc20 msgr2=0x7ff918162050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:04.610 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.579+0000 7ff9177fe700 1 --2- 172.21.15.99:0/2599693473 >> v2:172.21.15.191:3300/0 conn(0x7ff9180afc20 0x7ff918162050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.610 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.579+0000 7ff9177fe700 1 -- 172.21.15.99:0/2599693473 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9180b0660 con 0x7ff9180bfbe0 2023-10-08T00:11:04.611 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.579+0000 7fafc166f700 1 -- 172.21.15.99:0/3611199131 wait complete. 2023-10-08T00:11:04.611 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.579+0000 7fa7e259c700 1 --2- 172.21.15.99:0/4246690376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7e40afdf0 0x7fa7e40c39d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa7e40cc4a0 tx=0x7fa7d4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.611 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.579+0000 7fa7d27fc700 1 -- 172.21.15.99:0/4246690376 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d400bb60 con 0x7fa7e40afdf0 2023-10-08T00:11:04.611 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.579+0000 7fa7e8aec700 1 -- 172.21.15.99:0/4246690376 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7e40c5b40 con 0x7fa7e40afdf0 2023-10-08T00:11:04.611 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.579+0000 7fa7d27fc700 1 -- 172.21.15.99:0/4246690376 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d4000c90 con 0x7fa7e40afdf0 2023-10-08T00:11:04.611 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.579+0000 7fa7d27fc700 1 -- 172.21.15.99:0/4246690376 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d401a660 con 0x7fa7e40afdf0 2023-10-08T00:11:04.612 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.579+0000 7fa7e8aec700 1 -- 172.21.15.99:0/4246690376 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7e40c6100 con 0x7fa7e40afdf0 2023-10-08T00:11:04.612 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.579+0000 7f8f4c311700 1 Processor -- start 2023-10-08T00:11:04.612 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.579+0000 7f8f4c311700 1 -- start start 2023-10-08T00:11:04.612 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.580+0000 7f867c778700 1 -- 172.21.15.99:0/3533110163 shutdown_connections 2023-10-08T00:11:04.612 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.580+0000 7f867c778700 1 --2- 172.21.15.99:0/3533110163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8660005310 0x7f866000ba60 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.612 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.580+0000 7f867c778700 1 -- 172.21.15.99:0/3533110163 >> 172.21.15.99:0/3533110163 conn(0x7f8674068ce0 msgr2=0x7f86740690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:04.613 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.580+0000 7f4cfa7fc700 1 -- 172.21.15.99:0/515141564 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d00005b30 con 0x7f4d0c0c89d0 2023-10-08T00:11:04.613 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.580+0000 7f4cfa7fc700 1 -- 172.21.15.99:0/515141564 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d00005c90 con 0x7f4d0c0c89d0 2023-10-08T00:11:04.613 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.580+0000 7f4d109f2700 1 -- 172.21.15.99:0/515141564 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d0c0b1c00 con 0x7f4d0c0c89d0 2023-10-08T00:11:04.613 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.580+0000 7f8f4c311700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f440c7820 0x7f8f4415b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.613 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.580+0000 7f8f4c311700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f44157b00 0x7f8f44098040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.614 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.580+0000 7f8f4c311700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f44157ed0 0x7f8f4416bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.614 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.580+0000 7f8f4c311700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4415bce0 con 0x7f8f44157b00 2023-10-08T00:11:04.614 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.580+0000 7f8f4c311700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f440b2720 con 0x7f8f440c7820 2023-10-08T00:11:04.614 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.580+0000 7f8f4c311700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f440b1df0 con 0x7f8f44157ed0 2023-10-08T00:11:04.614 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.581+0000 7f8f4a0ad700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f440c7820 0x7f8f4415b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.614 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.581+0000 7f8f4a0ad700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f440c7820 0x7f8f4415b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.99:54654/0 (socket says 172.21.15.99:54654) 2023-10-08T00:11:04.614 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.581+0000 7f8f4a0ad700 1 -- 172.21.15.99:0/3395954984 learned_addr learned my addr 172.21.15.99:0/3395954984 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:04.614 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.580+0000 7f4cfa7fc700 1 -- 172.21.15.99:0/515141564 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d00007300 con 0x7f4d0c0c89d0 2023-10-08T00:11:04.614 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.580+0000 7f4d109f2700 1 -- 172.21.15.99:0/515141564 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d0c0b21c0 con 0x7f4d0c0c89d0 2023-10-08T00:11:04.615 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.579+0000 7ff9177fe700 1 --2- 172.21.15.99:0/2599693473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9180bfbe0 0x7ff9180ccc90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff91806e3a0 tx=0x7ff910009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=677745adf123e0ac server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.615 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.579+0000 7ff9167fc700 1 -- 172.21.15.99:0/2599693473 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff910009ea0 con 0x7ff9180bfbe0 2023-10-08T00:11:04.615 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.579+0000 7ff9167fc700 1 -- 172.21.15.99:0/2599693473 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff91000b820 con 0x7ff9180bfbe0 2023-10-08T00:11:04.615 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.579+0000 7ff9167fc700 1 -- 172.21.15.99:0/2599693473 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91000bbc0 con 0x7ff9180bfbe0 2023-10-08T00:11:04.615 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.581+0000 7f867c778700 1 -- 172.21.15.99:0/3533110163 shutdown_connections 2023-10-08T00:11:04.615 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.581+0000 7f867c778700 1 -- 172.21.15.99:0/3533110163 wait complete. 2023-10-08T00:11:04.616 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.580+0000 7fafc166f700 1 Processor -- start 2023-10-08T00:11:04.616 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.578+0000 7fe82a08c700 1 Processor -- start 2023-10-08T00:11:04.616 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.578+0000 7fe82a08c700 1 -- start start 2023-10-08T00:11:04.616 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.579+0000 7fe82a08c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe824070e10 0x7fe8241600f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.616 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.579+0000 7fe82a08c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8240aff50 0x7fe8241613c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.616 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.579+0000 7fe82a08c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8240c9bb0 0x7fe824171a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.616 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.579+0000 7fe82a08c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8240b2a30 con 0x7fe8240c9bb0 2023-10-08T00:11:04.616 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.579+0000 7fe82a08c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8240b3840 con 0x7fe8240aff50 2023-10-08T00:11:04.617 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.579+0000 7fe82a08c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8240b2c30 con 0x7fe824070e10 2023-10-08T00:11:04.617 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.581+0000 7f8f4a8ae700 1 --2- 172.21.15.99:0/3395954984 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f44157ed0 0x7f8f4416bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.617 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.581+0000 7f8f498ac700 1 --2- 172.21.15.99:0/3395954984 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f44157b00 0x7f8f44098040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.617 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.581+0000 7ff91df78700 1 -- 172.21.15.99:0/2599693473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9180bfbe0 msgr2=0x7ff9180ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.617 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.581+0000 7ff91df78700 1 --2- 172.21.15.99:0/2599693473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9180bfbe0 0x7ff9180ccc90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff91806e3a0 tx=0x7ff910009250 comp rx=0 tx=0).stop 2023-10-08T00:11:04.618 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.580+0000 7fe822ffd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8240aff50 0x7fe8241613c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.618 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.580+0000 7fe822ffd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8240aff50 0x7fe8241613c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.99:54646/0 (socket says 172.21.15.99:54646) 2023-10-08T00:11:04.618 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.580+0000 7fe822ffd700 1 -- 172.21.15.99:0/1686344760 learned_addr learned my addr 172.21.15.99:0/1686344760 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:04.618 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.580+0000 7fe823fff700 1 --2- 172.21.15.99:0/1686344760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8240c9bb0 0x7fe824171a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.618 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.580+0000 7fe8237fe700 1 --2- 172.21.15.99:0/1686344760 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe824070e10 0x7fe8241600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.618 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.580+0000 7fe823fff700 1 -- 172.21.15.99:0/1686344760 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe824070e10 msgr2=0x7fe8241600f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.618 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.580+0000 7fe823fff700 1 --2- 172.21.15.99:0/1686344760 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe824070e10 0x7fe8241600f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.618 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.580+0000 7fe823fff700 1 -- 172.21.15.99:0/1686344760 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8240aff50 msgr2=0x7fe8241613c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.618 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.580+0000 7fe823fff700 1 --2- 172.21.15.99:0/1686344760 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8240aff50 0x7fe8241613c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.618 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.580+0000 7fe823fff700 1 -- 172.21.15.99:0/1686344760 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8180094f0 con 0x7fe8240c9bb0 2023-10-08T00:11:04.619 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.580+0000 7fe8237fe700 1 --2- 172.21.15.99:0/1686344760 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe824070e10 0x7fe8241600f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:11:04.619 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.580+0000 7fe823fff700 1 --2- 172.21.15.99:0/1686344760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8240c9bb0 0x7fe824171a90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe81c007e70 tx=0x7fe81c0097b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.619 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.581+0000 7fe80b7fe700 1 -- 172.21.15.99:0/1686344760 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81c03a070 con 0x7fe8240c9bb0 2023-10-08T00:11:04.619 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.581+0000 7fe80b7fe700 1 -- 172.21.15.99:0/1686344760 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe81c030ad0 con 0x7fe8240c9bb0 2023-10-08T00:11:04.619 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.581+0000 7fe80b7fe700 1 -- 172.21.15.99:0/1686344760 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81c030e90 con 0x7fe8240c9bb0 2023-10-08T00:11:04.619 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.581+0000 7fe82a08c700 1 -- 172.21.15.99:0/1686344760 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8240b2f10 con 0x7fe8240c9bb0 2023-10-08T00:11:04.619 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.581+0000 7f8f4a0ad700 1 -- 172.21.15.99:0/3395954984 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f44157ed0 msgr2=0x7f8f4416bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.619 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.581+0000 7f8f4a0ad700 1 --2- 172.21.15.99:0/3395954984 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f44157ed0 0x7f8f4416bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.619 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.581+0000 7f8f4a0ad700 1 -- 172.21.15.99:0/3395954984 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f44157b00 msgr2=0x7f8f44098040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.620 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.581+0000 7f8f4a0ad700 1 --2- 172.21.15.99:0/3395954984 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f44157b00 0x7f8f44098040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.620 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.581+0000 7f8f4a0ad700 1 -- 172.21.15.99:0/3395954984 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f340094f0 con 0x7f8f440c7820 2023-10-08T00:11:04.620 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.581+0000 7f8f4a8ae700 1 --2- 172.21.15.99:0/3395954984 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f44157ed0 0x7f8f4416bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:11:04.620 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.582+0000 7fa7d27fc700 1 -- 172.21.15.99:0/4246690376 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa7d401aa10 con 0x7fa7e40afdf0 2023-10-08T00:11:04.620 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.582+0000 7f8f4a0ad700 1 --2- 172.21.15.99:0/3395954984 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f440c7820 0x7f8f4415b7a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8f4000b600 tx=0x7f8f400091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.621 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.582+0000 7f8f3a7fc700 1 -- 172.21.15.99:0/3395954984 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4000f510 con 0x7f8f440c7820 2023-10-08T00:11:04.621 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.582+0000 7f8f3a7fc700 1 -- 172.21.15.99:0/3395954984 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f4000f670 con 0x7f8f440c7820 2023-10-08T00:11:04.621 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.582+0000 7f8f3a7fc700 1 -- 172.21.15.99:0/3395954984 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4000fa30 con 0x7f8f440c7820 2023-10-08T00:11:04.621 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.582+0000 7fa7d27fc700 1 --2- 172.21.15.99:0/4246690376 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa7c4066fb0 0x7fa7c4069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.621 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.582+0000 7f8f4c311700 1 -- 172.21.15.99:0/3395954984 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f440b20d0 con 0x7f8f440c7820 2023-10-08T00:11:04.621 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.582+0000 7f8f4c311700 1 -- 172.21.15.99:0/3395954984 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f440b1ac0 con 0x7f8f440c7820 2023-10-08T00:11:04.622 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.582+0000 7fe82a08c700 1 -- 172.21.15.99:0/1686344760 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8240b34d0 con 0x7fe8240c9bb0 2023-10-08T00:11:04.622 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.582+0000 7fa7d27fc700 1 -- 172.21.15.99:0/4246690376 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa7d40861f0 con 0x7fa7e40afdf0 2023-10-08T00:11:04.622 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.582+0000 7fafc166f700 1 -- start start 2023-10-08T00:11:04.623 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.583+0000 7f867c778700 1 Processor -- start 2023-10-08T00:11:04.623 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.582+0000 7fafc166f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fafbc0a7da0 0x7fafbc1427f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.623 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.582+0000 7fafc166f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fafbc15e1d0 0x7fafbc0acae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.623 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.583+0000 7fafc166f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafbc15e5a0 0x7fafbc15cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.623 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.583+0000 7fafc166f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafbc146d00 con 0x7fafbc15e5a0 2023-10-08T00:11:04.624 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.583+0000 7fafc166f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafbc161b50 con 0x7fafbc0a7da0 2023-10-08T00:11:04.624 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.583+0000 7fafc166f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafbc1463d0 con 0x7fafbc15e1d0 2023-10-08T00:11:04.624 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.583+0000 7ff91df78700 1 -- 172.21.15.99:0/2599693473 shutdown_connections 2023-10-08T00:11:04.624 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.583+0000 7ff91df78700 1 --2- 172.21.15.99:0/2599693473 >> v2:172.21.15.191:3300/0 conn(0x7ff9180afc20 0x7ff918162050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.624 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.583+0000 7ff91df78700 1 --2- 172.21.15.99:0/2599693473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9180bfbe0 0x7ff9180ccc90 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.624 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.583+0000 7ff91df78700 1 --2- 172.21.15.99:0/2599693473 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9180cd1d0 0x7ff9180af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.624 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.583+0000 7ff91df78700 1 -- 172.21.15.99:0/2599693473 >> 172.21.15.99:0/2599693473 conn(0x7ff918070700 msgr2=0x7ff918070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:04.625 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.583+0000 7fa7e1d9b700 1 --2- 172.21.15.99:0/4246690376 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa7c4066fb0 0x7fa7c4069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.625 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.583+0000 7fafbaffd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fafbc0a7da0 0x7fafbc1427f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.625 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.583+0000 7ff91df78700 1 -- 172.21.15.99:0/2599693473 shutdown_connections 2023-10-08T00:11:04.626 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.583+0000 7fafbaffd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fafbc0a7da0 0x7fafbc1427f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.99:54662/0 (socket says 172.21.15.99:54662) 2023-10-08T00:11:04.626 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.583+0000 7fafbaffd700 1 -- 172.21.15.99:0/1337588190 learned_addr learned my addr 172.21.15.99:0/1337588190 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:04.626 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.583+0000 7fafbb7fe700 1 --2- 172.21.15.99:0/1337588190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafbc15e5a0 0x7fafbc15cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.626 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.583+0000 7fafba7fc700 1 --2- 172.21.15.99:0/1337588190 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fafbc15e1d0 0x7fafbc0acae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.626 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.583+0000 7fe82a08c700 1 -- 172.21.15.99:0/1686344760 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe82415bec0 con 0x7fe8240c9bb0 2023-10-08T00:11:04.626 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.583+0000 7fa7e1d9b700 1 --2- 172.21.15.99:0/4246690376 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa7c4066fb0 0x7fa7c4069460 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa7e40afbb0 tx=0x7fa7dc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.627 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.583+0000 7ff91df78700 1 -- 172.21.15.99:0/2599693473 wait complete. 2023-10-08T00:11:04.627 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.583+0000 7f4cfa7fc700 1 -- 172.21.15.99:0/515141564 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4d0001b3b0 con 0x7f4d0c0c89d0 2023-10-08T00:11:04.627 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.584+0000 7f4cfa7fc700 1 --2- 172.21.15.99:0/515141564 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4cec067080 0x7f4cec069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.627 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.584+0000 7f4cfa7fc700 1 -- 172.21.15.99:0/515141564 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4d00083960 con 0x7f4d0c0c89d0 2023-10-08T00:11:04.628 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.583+0000 7f867c778700 1 -- start start 2023-10-08T00:11:04.628 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.584+0000 7f867c778700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8660005310 0x7f867414d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.628 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.584+0000 7f867c778700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8674160200 0x7f867416f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.628 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.584+0000 7f867c778700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f867406ec50 0x7f867406f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.628 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.584+0000 7f867c778700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867406f630 con 0x7f8674160200 2023-10-08T00:11:04.628 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.584+0000 7f867c778700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8674170870 con 0x7f8660005310 2023-10-08T00:11:04.628 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.584+0000 7f867c778700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f867416f720 con 0x7f867406ec50 2023-10-08T00:11:04.628 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.584+0000 7f867a514700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8660005310 0x7f867414d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.628 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.584+0000 7f867a514700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8660005310 0x7f867414d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.99:54668/0 (socket says 172.21.15.99:54668) 2023-10-08T00:11:04.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.584+0000 7f867a514700 1 -- 172.21.15.99:0/3811620499 learned_addr learned my addr 172.21.15.99:0/3811620499 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:04.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.584+0000 7ff91df78700 1 Processor -- start 2023-10-08T00:11:04.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.585+0000 7f867a514700 1 -- 172.21.15.99:0/3811620499 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f867406ec50 msgr2=0x7f867406f0f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T00:11:04.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.585+0000 7f867a514700 1 --2- 172.21.15.99:0/3811620499 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f867406ec50 0x7f867406f0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.585+0000 7f867a514700 1 -- 172.21.15.99:0/3811620499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8674160200 msgr2=0x7f867416f1e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T00:11:04.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.585+0000 7f867a514700 1 --2- 172.21.15.99:0/3811620499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8674160200 0x7f867416f1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.585+0000 7f867a514700 1 -- 172.21.15.99:0/3811620499 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f867416f980 con 0x7f8660005310 2023-10-08T00:11:04.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.585+0000 7f867a514700 1 --2- 172.21.15.99:0/3811620499 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8660005310 0x7f867414d2f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f867000c740 tx=0x7f86740cdd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.584+0000 7fe80b7fe700 1 -- 172.21.15.99:0/1686344760 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe81c003020 con 0x7fe8240c9bb0 2023-10-08T00:11:04.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.585+0000 7fe80b7fe700 1 --2- 172.21.15.99:0/1686344760 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe810066fb0 0x7fe810069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.585+0000 7fe80b7fe700 1 -- 172.21.15.99:0/1686344760 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe81c0a66d0 con 0x7fe8240c9bb0 2023-10-08T00:11:04.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.585+0000 7fe8237fe700 1 --2- 172.21.15.99:0/1686344760 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe810066fb0 0x7fe810069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.585+0000 7f8f3a7fc700 1 -- 172.21.15.99:0/3395954984 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8f4001a400 con 0x7f8f440c7820 2023-10-08T00:11:04.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.584+0000 7f4d0a59c700 1 --2- 172.21.15.99:0/515141564 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4cec067080 0x7f4cec069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.585+0000 7f866a7fc700 1 -- 172.21.15.99:0/3811620499 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8670007360 con 0x7f8660005310 2023-10-08T00:11:04.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.583+0000 7fafbaffd700 1 -- 172.21.15.99:0/1337588190 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fafbc15e1d0 msgr2=0x7fafbc0acae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.583+0000 7fafbaffd700 1 --2- 172.21.15.99:0/1337588190 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fafbc15e1d0 0x7fafbc0acae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.583+0000 7fafbaffd700 1 -- 172.21.15.99:0/1337588190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafbc15e5a0 msgr2=0x7fafbc15cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.583+0000 7fafbaffd700 1 --2- 172.21.15.99:0/1337588190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafbc15e5a0 0x7fafbc15cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.583+0000 7fafbaffd700 1 -- 172.21.15.99:0/1337588190 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb4009ae0 con 0x7fafbc0a7da0 2023-10-08T00:11:04.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.583+0000 7fafbaffd700 1 --2- 172.21.15.99:0/1337588190 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fafbc0a7da0 0x7fafbc1427f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fafac00c590 tx=0x7fafac00c8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.584+0000 7fafaaffd700 1 -- 172.21.15.99:0/1337588190 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafac011590 con 0x7fafbc0a7da0 2023-10-08T00:11:04.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.584+0000 7fafaaffd700 1 -- 172.21.15.99:0/1337588190 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafac011710 con 0x7fafbc0a7da0 2023-10-08T00:11:04.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.584+0000 7fafc166f700 1 -- 172.21.15.99:0/1337588190 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafbc1466b0 con 0x7fafbc0a7da0 2023-10-08T00:11:04.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.584+0000 7fafaaffd700 1 -- 172.21.15.99:0/1337588190 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafac011ad0 con 0x7fafbc0a7da0 2023-10-08T00:11:04.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.584+0000 7fafc166f700 1 -- 172.21.15.99:0/1337588190 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafbc0ad020 con 0x7fafbc0a7da0 2023-10-08T00:11:04.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.585+0000 7fafba7fc700 1 --2- 172.21.15.99:0/1337588190 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fafbc15e1d0 0x7fafbc0acae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:11:04.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.585+0000 7f867c778700 1 -- 172.21.15.99:0/3811620499 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f867416fc60 con 0x7f8660005310 2023-10-08T00:11:04.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.585+0000 7f867c778700 1 -- 172.21.15.99:0/3811620499 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8674170220 con 0x7f8660005310 2023-10-08T00:11:04.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.585+0000 7f866a7fc700 1 -- 172.21.15.99:0/3811620499 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86700074c0 con 0x7f8660005310 2023-10-08T00:11:04.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.585+0000 7f866a7fc700 1 -- 172.21.15.99:0/3811620499 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8670004830 con 0x7f8660005310 2023-10-08T00:11:04.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.585+0000 7ff91df78700 1 -- start start 2023-10-08T00:11:04.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.586+0000 7ff91df78700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9180afc20 0x7ff9180c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.586+0000 7ff91df78700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9180bfbe0 0x7ff9180c5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.586+0000 7ff91df78700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9180cd1d0 0x7ff91815b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.586+0000 7ff91df78700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91815e510 con 0x7ff9180cd1d0 2023-10-08T00:11:04.635 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.586+0000 7ff91df78700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9180c89a0 con 0x7ff9180bfbe0 2023-10-08T00:11:04.635 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.586+0000 7ff91df78700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9180c91f0 con 0x7ff9180afc20 2023-10-08T00:11:04.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.586+0000 7ff9177fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9180afc20 0x7ff9180c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.586+0000 7ff9177fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9180afc20 0x7ff9180c5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.99:45714/0 (socket says 172.21.15.99:45714) 2023-10-08T00:11:04.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.586+0000 7ff9177fe700 1 -- 172.21.15.99:0/4166998920 learned_addr learned my addr 172.21.15.99:0/4166998920 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:04.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.586+0000 7ff916ffd700 1 --2- 172.21.15.99:0/4166998920 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9180bfbe0 0x7ff9180c5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.586+0000 7ff917fff700 1 --2- 172.21.15.99:0/4166998920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9180cd1d0 0x7ff91815b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.586+0000 7ff9177fe700 1 -- 172.21.15.99:0/4166998920 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9180bfbe0 msgr2=0x7ff9180c5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.586+0000 7ff9177fe700 1 --2- 172.21.15.99:0/4166998920 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9180bfbe0 0x7ff9180c5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.586+0000 7ff9177fe700 1 -- 172.21.15.99:0/4166998920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9180cd1d0 msgr2=0x7ff91815b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.586+0000 7ff9177fe700 1 --2- 172.21.15.99:0/4166998920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9180cd1d0 0x7ff91815b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.586+0000 7f8f3a7fc700 1 --2- 172.21.15.99:0/3395954984 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8f28066f30 0x7f8f280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.586+0000 7f8f3a7fc700 1 -- 172.21.15.99:0/3395954984 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8f40084020 con 0x7f8f440c7820 2023-10-08T00:11:04.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.586+0000 7f8f498ac700 1 --2- 172.21.15.99:0/3395954984 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8f28066f30 0x7f8f280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.640 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.586+0000 7ff9177fe700 1 -- 172.21.15.99:0/4166998920 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9100094f0 con 0x7ff9180afc20 2023-10-08T00:11:04.640 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.586+0000 7ff917fff700 1 --2- 172.21.15.99:0/4166998920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9180cd1d0 0x7ff91815b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:11:04.640 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.586+0000 7fe8237fe700 1 --2- 172.21.15.99:0/1686344760 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe810066fb0 0x7fe810069460 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe8180099e0 tx=0x7fe818017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.640 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.586+0000 7f8f498ac700 1 --2- 172.21.15.99:0/3395954984 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8f28066f30 0x7f8f280693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8f340099b0 tx=0x7f8f34016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.641 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.587+0000 7fafaaffd700 1 -- 172.21.15.99:0/1337588190 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fafac01b3b0 con 0x7fafbc0a7da0 2023-10-08T00:11:04.641 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.587+0000 7f867c778700 1 -- 172.21.15.99:0/3811620499 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f865c0052b0 con 0x7f8660005310 2023-10-08T00:11:04.641 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.587+0000 7ff9177fe700 1 --2- 172.21.15.99:0/4166998920 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9180afc20 0x7ff9180c5a10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff9100094c0 tx=0x7ff910017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.641 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.587+0000 7f4d0a59c700 1 --2- 172.21.15.99:0/515141564 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4cec067080 0x7f4cec069530 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc006dc0 tx=0x7f4cfc017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.642 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.587+0000 7ff90f7fe700 1 -- 172.21.15.99:0/4166998920 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91000bba0 con 0x7ff9180afc20 2023-10-08T00:11:04.642 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.587+0000 7ff90f7fe700 1 -- 172.21.15.99:0/4166998920 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff910011420 con 0x7ff9180afc20 2023-10-08T00:11:04.642 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.587+0000 7ff90f7fe700 1 -- 172.21.15.99:0/4166998920 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9100117c0 con 0x7ff9180afc20 2023-10-08T00:11:04.642 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.587+0000 7fafaaffd700 1 --2- 172.21.15.99:0/1337588190 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fafa0066f00 0x7fafa00693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.587+0000 7ff91df78700 1 -- 172.21.15.99:0/4166998920 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9180c67f0 con 0x7ff9180afc20 2023-10-08T00:11:04.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.587+0000 7ff91df78700 1 -- 172.21.15.99:0/4166998920 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9180c8ba0 con 0x7ff9180afc20 2023-10-08T00:11:04.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.587+0000 7fafaaffd700 1 -- 172.21.15.99:0/1337588190 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fafac083a30 con 0x7fafbc0a7da0 2023-10-08T00:11:04.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.588+0000 7fafba7fc700 1 --2- 172.21.15.99:0/1337588190 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fafa0066f00 0x7fafa00693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.588+0000 7fafc166f700 1 -- 172.21.15.99:0/1337588190 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafbc04c3c0 con 0x7fafbc0a7da0 2023-10-08T00:11:04.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.588+0000 7f4d109f2700 1 -- 172.21.15.99:0/515141564 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d0c04c3c0 con 0x7f4d0c0c89d0 2023-10-08T00:11:04.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.588+0000 7fa7e8aec700 1 -- 172.21.15.99:0/4246690376 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7e404c3c0 con 0x7fa7e40afdf0 2023-10-08T00:11:04.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.588+0000 7ff91df78700 1 -- 172.21.15.99:0/4166998920 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff91804c3c0 con 0x7ff9180afc20 2023-10-08T00:11:04.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.588+0000 7f866a7fc700 1 -- 172.21.15.99:0/3811620499 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8670004c20 con 0x7f8660005310 2023-10-08T00:11:04.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.588+0000 7f866a7fc700 1 --2- 172.21.15.99:0/3811620499 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8658066f30 0x7f86580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.588+0000 7f866a7fc700 1 -- 172.21.15.99:0/3811620499 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8670083b90 con 0x7f8660005310 2023-10-08T00:11:04.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.588+0000 7f8679d13700 1 --2- 172.21.15.99:0/3811620499 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8658066f30 0x7f86580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.588+0000 7f8f4c311700 1 -- 172.21.15.99:0/3395954984 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f440b2a00 con 0x7f8f440c7820 2023-10-08T00:11:04.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.589+0000 7f8679d13700 1 --2- 172.21.15.99:0/3811620499 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8658066f30 0x7f86580693e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f86740c74c0 tx=0x7f8664006cb0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.589+0000 7ff90f7fe700 1 -- 172.21.15.99:0/4166998920 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff910011bb0 con 0x7ff9180afc20 2023-10-08T00:11:04.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.589+0000 7ff90f7fe700 1 --2- 172.21.15.99:0/4166998920 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff8f8066f30 0x7ff8f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.589+0000 7ff90f7fe700 1 -- 172.21.15.99:0/4166998920 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff910085b30 con 0x7ff9180afc20 2023-10-08T00:11:04.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.589+0000 7ff916ffd700 1 --2- 172.21.15.99:0/4166998920 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff8f8066f30 0x7ff8f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.590+0000 7ff916ffd700 1 --2- 172.21.15.99:0/4166998920 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff8f8066f30 0x7ff8f80693e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff900006fd0 tx=0x7ff900008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.590+0000 7fafba7fc700 1 --2- 172.21.15.99:0/1337588190 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fafa0066f00 0x7fafa00693b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fafb4009ab0 tx=0x7fafb4018000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.591+0000 7fe80b7fe700 1 -- 172.21.15.99:0/1686344760 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe82415bec0 con 0x7fe8240c9bb0 2023-10-08T00:11:04.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.592+0000 7fce5fb79700 1 Processor -- start 2023-10-08T00:11:04.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.593+0000 7fce5fb79700 1 -- start start 2023-10-08T00:11:04.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.593+0000 7f4cfa7fc700 1 -- 172.21.15.99:0/515141564 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4d00050bb0 con 0x7f4d0c0c89d0 2023-10-08T00:11:04.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.593+0000 7fce5fb79700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fce5815a620 0x7fce5815d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.593+0000 7fce5fb79700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fce5815d580 0x7fce58158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.593+0000 7fce5fb79700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce5815cbb0 con 0x7fce58157870 2023-10-08T00:11:04.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.593+0000 7fce5fb79700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce5815bdb0 con 0x7fce5815a620 2023-10-08T00:11:04.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.593+0000 7fce5fb79700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce5815bfa0 con 0x7fce5815d580 2023-10-08T00:11:04.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.594+0000 7fce5d915700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fce58157870 0x7fce5815b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45048/0 (socket says 172.21.15.99:45048) 2023-10-08T00:11:04.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.594+0000 7fce5d114700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fce5815a620 0x7fce5815d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.594+0000 7fce5d915700 1 -- 172.21.15.99:0/851815459 learned_addr learned my addr 172.21.15.99:0/851815459 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:04.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.594+0000 7fce5e116700 1 --2- 172.21.15.99:0/851815459 >> v2:172.21.15.191:3300/0 conn(0x7fce5815d580 0x7fce58158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.594+0000 7ff90f7fe700 1 -- 172.21.15.99:0/4166998920 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff910052da0 con 0x7ff9180afc20 2023-10-08T00:11:04.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.594+0000 7fce5d114700 1 -- 172.21.15.99:0/851815459 >> v2:172.21.15.191:3300/0 conn(0x7fce5815d580 msgr2=0x7fce58158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:04.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.594+0000 7fce5d114700 1 --2- 172.21.15.99:0/851815459 >> v2:172.21.15.191:3300/0 conn(0x7fce5815d580 0x7fce58158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.594+0000 7fce5d114700 1 -- 172.21.15.99:0/851815459 >> v1:172.21.15.99:6789/0 conn(0x7fce58157870 legacy=0x7fce5815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.594+0000 7fce5d114700 1 -- 172.21.15.99:0/851815459 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce5815c280 con 0x7fce5815a620 2023-10-08T00:11:04.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.594+0000 7fce5d114700 1 --2- 172.21.15.99:0/851815459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce5815a620 0x7fce5815d040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fce5806b4e0 tx=0x7fce480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d92b6005f8b042fc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.595+0000 7fce5c913700 1 -- 172.21.15.99:0/851815459 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce48011630 con 0x7fce5815a620 2023-10-08T00:11:04.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.595+0000 7fce5c913700 1 -- 172.21.15.99:0/851815459 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce48011790 con 0x7fce5815a620 2023-10-08T00:11:04.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.595+0000 7fce5c913700 1 -- 172.21.15.99:0/851815459 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce48011b50 con 0x7fce5815a620 2023-10-08T00:11:04.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.595+0000 7fa7d27fc700 1 -- 172.21.15.99:0/4246690376 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa7d40533c0 con 0x7fa7e40afdf0 2023-10-08T00:11:04.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.596+0000 7fce5fb79700 1 -- 172.21.15.99:0/851815459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce5815a620 msgr2=0x7fce5815d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.596+0000 7fce5fb79700 1 --2- 172.21.15.99:0/851815459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce5815a620 0x7fce5815d040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fce5806b4e0 tx=0x7fce480091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.596+0000 7fce5fb79700 1 -- 172.21.15.99:0/851815459 shutdown_connections 2023-10-08T00:11:04.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.596+0000 7fce5fb79700 1 --2- 172.21.15.99:0/851815459 >> v2:172.21.15.191:3300/0 conn(0x7fce5815d580 0x7fce58158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.596+0000 7fce5fb79700 1 --2- 172.21.15.99:0/851815459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce5815a620 0x7fce5815d040 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.596+0000 7fce5fb79700 1 -- 172.21.15.99:0/851815459 >> 172.21.15.99:0/851815459 conn(0x7fce58068ce0 msgr2=0x7fce580690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:04.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.596+0000 7fce5fb79700 1 -- 172.21.15.99:0/851815459 shutdown_connections 2023-10-08T00:11:04.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.596+0000 7fce5fb79700 1 -- 172.21.15.99:0/851815459 wait complete. 2023-10-08T00:11:04.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.597+0000 7f866a7fc700 1 -- 172.21.15.99:0/3811620499 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f867004c5a0 con 0x7f8660005310 2023-10-08T00:11:04.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.598+0000 7fce5fb79700 1 Processor -- start 2023-10-08T00:11:04.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.598+0000 7fce5fb79700 1 -- start start 2023-10-08T00:11:04.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.598+0000 7fce5fb79700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce58157870 0x7fce5816fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.598+0000 7fce5fb79700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce5815a620 0x7fce5814c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.598+0000 7fce5fb79700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce5815d580 0x7fce581521f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.598+0000 7fce5fb79700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce581538c0 con 0x7fce58157870 2023-10-08T00:11:04.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.598+0000 7fce5fb79700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5814ce90 con 0x7fce5815d580 2023-10-08T00:11:04.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.598+0000 7fce5fb79700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce58152c40 con 0x7fce5815a620 2023-10-08T00:11:04.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.598+0000 7fce5d915700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce58157870 0x7fce5816fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.598+0000 7fce5d915700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce58157870 0x7fce5816fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38790/0 (socket says 172.21.15.99:38790) 2023-10-08T00:11:04.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.598+0000 7fce5d915700 1 -- 172.21.15.99:0/1749712096 learned_addr learned my addr 172.21.15.99:0/1749712096 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:04.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.599+0000 7fce5e116700 1 --2- 172.21.15.99:0/1749712096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce5815d580 0x7fce581521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.599+0000 7fce5d915700 1 -- 172.21.15.99:0/1749712096 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce5815a620 msgr2=0x7fce5814c950 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T00:11:04.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.599+0000 7fce5d915700 1 --2- 172.21.15.99:0/1749712096 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce5815a620 0x7fce5814c950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.599+0000 7fce5d915700 1 -- 172.21.15.99:0/1749712096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce5815d580 msgr2=0x7fce581521f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.599+0000 7fce5d915700 1 --2- 172.21.15.99:0/1749712096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce5815d580 0x7fce581521f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.599+0000 7fce5d915700 1 -- 172.21.15.99:0/1749712096 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce48009420 con 0x7fce58157870 2023-10-08T00:11:04.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.599+0000 7fce5e116700 1 --2- 172.21.15.99:0/1749712096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce5815d580 0x7fce581521f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:11:04.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.599+0000 7fce5d915700 1 --2- 172.21.15.99:0/1749712096 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce58157870 0x7fce5816fa40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fce5400c7c0 tx=0x7fce54007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.599+0000 7fce4dffb700 1 -- 172.21.15.99:0/1749712096 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce54005b30 con 0x7fce58157870 2023-10-08T00:11:04.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.599+0000 7fce4dffb700 1 -- 172.21.15.99:0/1749712096 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce54005c90 con 0x7fce58157870 2023-10-08T00:11:04.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.599+0000 7fce4dffb700 1 -- 172.21.15.99:0/1749712096 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce54004a80 con 0x7fce58157870 2023-10-08T00:11:04.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.599+0000 7fce5fb79700 1 -- 172.21.15.99:0/1749712096 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce58152f20 con 0x7fce58157870 2023-10-08T00:11:04.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.600+0000 7fce5fb79700 1 -- 172.21.15.99:0/1749712096 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce581534e0 con 0x7fce58157870 2023-10-08T00:11:04.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.601+0000 7fce4dffb700 1 -- 172.21.15.99:0/1749712096 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fce540070d0 con 0x7fce58157870 2023-10-08T00:11:04.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.601+0000 7fce4dffb700 1 --2- 172.21.15.99:0/1749712096 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fce3c066f30 0x7fce3c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:04.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.601+0000 7fce4dffb700 1 -- 172.21.15.99:0/1749712096 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fce54083910 con 0x7fce58157870 2023-10-08T00:11:04.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.605+0000 7f8f3a7fc700 1 -- 172.21.15.99:0/3395954984 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8f40051310 con 0x7f8f440c7820 2023-10-08T00:11:04.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.606+0000 7fafaaffd700 1 -- 172.21.15.99:0/1337588190 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fafac017020 con 0x7fafbc0a7da0 2023-10-08T00:11:04.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.606+0000 7fce5d114700 1 --2- 172.21.15.99:0/1749712096 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fce3c066f30 0x7fce3c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:04.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.606+0000 7fce5d114700 1 --2- 172.21.15.99:0/1749712096 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fce3c066f30 0x7fce3c0693e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fce5815aa20 tx=0x7fce48017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:04.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.608+0000 7fce5fb79700 1 -- 172.21.15.99:0/1749712096 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce380052b0 con 0x7fce58157870 2023-10-08T00:11:04.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.617+0000 7fce4dffb700 1 -- 172.21.15.99:0/1749712096 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fce5404b5a0 con 0x7fce58157870 2023-10-08T00:11:04.971 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.969+0000 7f867c778700 1 -- 172.21.15.99:0/3811620499 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f865c005560 con 0x7f8660005310 2023-10-08T00:11:04.975 INFO:teuthology.orchestra.run.smithi099.stdout:55834574853 2023-10-08T00:11:04.976 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.971+0000 7f866a7fc700 1 -- 172.21.15.99:0/3811620499 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8670018020 con 0x7f8660005310 2023-10-08T00:11:04.976 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.975+0000 7f867c778700 1 -- 172.21.15.99:0/3811620499 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8658066f30 msgr2=0x7f86580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.977 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.975+0000 7f867c778700 1 --2- 172.21.15.99:0/3811620499 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8658066f30 0x7f86580693e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f86740c74c0 tx=0x7f8664006cb0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.977 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.975+0000 7f867c778700 1 -- 172.21.15.99:0/3811620499 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8660005310 msgr2=0x7f867414d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.977 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.975+0000 7f867c778700 1 --2- 172.21.15.99:0/3811620499 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8660005310 0x7f867414d2f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f867000c740 tx=0x7f86740cdd20 comp rx=0 tx=0).stop 2023-10-08T00:11:04.982 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.981+0000 7f4d109f2700 1 -- 172.21.15.99:0/515141564 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f4d0c0c9180 con 0x7f4d0c0c89d0 2023-10-08T00:11:04.983 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.981+0000 7f867c778700 1 -- 172.21.15.99:0/3811620499 shutdown_connections 2023-10-08T00:11:04.983 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.981+0000 7f867c778700 1 --2- 172.21.15.99:0/3811620499 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8658066f30 0x7f86580693e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.983 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.981+0000 7f867c778700 1 --2- 172.21.15.99:0/3811620499 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8660005310 0x7f867414d2f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.983 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.981+0000 7f867c778700 1 --2- 172.21.15.99:0/3811620499 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f867406ec50 0x7f867406f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.983 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.981+0000 7f867c778700 1 --2- 172.21.15.99:0/3811620499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8674160200 0x7f867416f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.983 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.981+0000 7f867c778700 1 -- 172.21.15.99:0/3811620499 >> 172.21.15.99:0/3811620499 conn(0x7f8674068ce0 msgr2=0x7f867415c610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:04.984 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.982+0000 7f4cfa7fc700 1 -- 172.21.15.99:0/515141564 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4d00017020 con 0x7f4d0c0c89d0 2023-10-08T00:11:04.987 INFO:teuthology.orchestra.run.smithi099.stdout:55834574853 2023-10-08T00:11:04.987 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.984+0000 7fe82a08c700 1 -- 172.21.15.99:0/1686344760 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fe82415bec0 con 0x7fe8240c9bb0 2023-10-08T00:11:04.987 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.984+0000 7fe80b7fe700 1 -- 172.21.15.99:0/1686344760 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe82415bec0 con 0x7fe8240c9bb0 2023-10-08T00:11:04.988 INFO:teuthology.orchestra.run.smithi099.stdout:55834574853 2023-10-08T00:11:04.988 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.984+0000 7ff91df78700 1 -- 172.21.15.99:0/4166998920 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff91815eb40 con 0x7ff9180afc20 2023-10-08T00:11:04.989 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.982+0000 7f867c778700 1 -- 172.21.15.99:0/3811620499 shutdown_connections 2023-10-08T00:11:04.989 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.982+0000 7f867c778700 1 -- 172.21.15.99:0/3811620499 wait complete. 2023-10-08T00:11:04.990 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.986+0000 7ff90f7fe700 1 -- 172.21.15.99:0/4166998920 <== mon.2 v2:172.21.15.191:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff910015020 con 0x7ff9180afc20 2023-10-08T00:11:04.990 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.986+0000 7fa7e8aec700 1 -- 172.21.15.99:0/4246690376 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa7e4063230 con 0x7fa7e40afdf0 2023-10-08T00:11:04.991 INFO:teuthology.orchestra.run.smithi099.stdout:55834574853 2023-10-08T00:11:04.991 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.987+0000 7f4d109f2700 1 -- 172.21.15.99:0/515141564 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4cec067080 msgr2=0x7f4cec069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.992 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.987+0000 7f4d109f2700 1 --2- 172.21.15.99:0/515141564 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4cec067080 0x7f4cec069530 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc006dc0 tx=0x7f4cfc017000 comp rx=0 tx=0).stop 2023-10-08T00:11:04.992 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.987+0000 7f4d109f2700 1 -- 172.21.15.99:0/515141564 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d0c0c89d0 msgr2=0x7f4d0c0c4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.992 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.987+0000 7f4d109f2700 1 --2- 172.21.15.99:0/515141564 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d0c0c89d0 0x7f4d0c0c4b20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4d0000c5d0 tx=0x7f4d00007d30 comp rx=0 tx=0).stop 2023-10-08T00:11:04.993 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.987+0000 7f4d109f2700 1 -- 172.21.15.99:0/515141564 shutdown_connections 2023-10-08T00:11:04.993 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.987+0000 7f4d109f2700 1 --2- 172.21.15.99:0/515141564 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4cec067080 0x7f4cec069530 secure :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc006dc0 tx=0x7f4cfc017000 comp rx=0 tx=0).stop 2023-10-08T00:11:04.993 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.987+0000 7f4d109f2700 1 --2- 172.21.15.99:0/515141564 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d0c0ca630 0x7f4d0c0cca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.993 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.987+0000 7f4d109f2700 1 --2- 172.21.15.99:0/515141564 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d0c0adb20 0x7f4d0c14fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.994 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.987+0000 7f4d109f2700 1 --2- 172.21.15.99:0/515141564 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d0c0c89d0 0x7f4d0c0c4b20 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.994 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.987+0000 7f4d109f2700 1 -- 172.21.15.99:0/515141564 >> 172.21.15.99:0/515141564 conn(0x7f4d0c06a250 msgr2=0x7f4d0c0c3580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:04.995 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.987+0000 7fe82a08c700 1 -- 172.21.15.99:0/1686344760 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe810066fb0 msgr2=0x7fe810069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.995 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.987+0000 7fe82a08c700 1 --2- 172.21.15.99:0/1686344760 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe810066fb0 0x7fe810069460 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe8180099e0 tx=0x7fe818017000 comp rx=0 tx=0).stop 2023-10-08T00:11:04.995 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.987+0000 7fe82a08c700 1 -- 172.21.15.99:0/1686344760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8240c9bb0 msgr2=0x7fe824171a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.995 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.987+0000 7fe82a08c700 1 --2- 172.21.15.99:0/1686344760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8240c9bb0 0x7fe824171a90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe81c007e70 tx=0x7fe81c0097b0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.997 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.988+0000 7ff91df78700 1 -- 172.21.15.99:0/4166998920 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff8f8066f30 msgr2=0x7ff8f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.997 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.988+0000 7ff91df78700 1 --2- 172.21.15.99:0/4166998920 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff8f8066f30 0x7ff8f80693e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff900006fd0 tx=0x7ff900008040 comp rx=0 tx=0).stop 2023-10-08T00:11:04.997 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.988+0000 7ff91df78700 1 -- 172.21.15.99:0/4166998920 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9180afc20 msgr2=0x7ff9180c5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:04.997 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.988+0000 7ff91df78700 1 --2- 172.21.15.99:0/4166998920 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9180afc20 0x7ff9180c5a10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff9100094c0 tx=0x7ff910017000 comp rx=0 tx=0).stop 2023-10-08T00:11:04.998 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.988+0000 7ff91df78700 1 -- 172.21.15.99:0/4166998920 shutdown_connections 2023-10-08T00:11:04.998 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.988+0000 7ff91df78700 1 --2- 172.21.15.99:0/4166998920 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff8f8066f30 0x7ff8f80693e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.998 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.988+0000 7ff91df78700 1 --2- 172.21.15.99:0/4166998920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9180cd1d0 0x7ff91815b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.998 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.988+0000 7ff91df78700 1 --2- 172.21.15.99:0/4166998920 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9180afc20 0x7ff9180c5a10 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.998 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.988+0000 7ff91df78700 1 --2- 172.21.15.99:0/4166998920 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9180bfbe0 0x7ff9180c5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:04.998 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.988+0000 7ff91df78700 1 -- 172.21.15.99:0/4166998920 >> 172.21.15.99:0/4166998920 conn(0x7ff918070700 msgr2=0x7ff91815f280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:04.999 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.988+0000 7ff91df78700 1 -- 172.21.15.99:0/4166998920 shutdown_connections 2023-10-08T00:11:04.999 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.989+0000 7ff91df78700 1 -- 172.21.15.99:0/4166998920 wait complete. 2023-10-08T00:11:04.999 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.992+0000 7fe82a08c700 1 -- 172.21.15.99:0/1686344760 shutdown_connections 2023-10-08T00:11:04.999 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.992+0000 7fe82a08c700 1 --2- 172.21.15.99:0/1686344760 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe810066fb0 0x7fe810069460 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:05.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.992+0000 7fe82a08c700 1 --2- 172.21.15.99:0/1686344760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8240c9bb0 0x7fe824171a90 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:05.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.992+0000 7fe82a08c700 1 --2- 172.21.15.99:0/1686344760 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe824070e10 0x7fe8241600f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:05.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.992+0000 7fe82a08c700 1 --2- 172.21.15.99:0/1686344760 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8240aff50 0x7fe8241613c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:05.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.992+0000 7fe82a08c700 1 -- 172.21.15.99:0/1686344760 >> 172.21.15.99:0/1686344760 conn(0x7fe8240ca4f0 msgr2=0x7fe824162020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:05.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.995+0000 7fa7d27fc700 1 -- 172.21.15.99:0/4246690376 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa7d4015020 con 0x7fa7e40afdf0 2023-10-08T00:11:05.001 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.995+0000 7fafc166f700 1 -- 172.21.15.99:0/1337588190 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fafbc0b2810 con 0x7fafbc0a7da0 2023-10-08T00:11:05.001 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.995+0000 7f4d109f2700 1 -- 172.21.15.99:0/515141564 shutdown_connections 2023-10-08T00:11:05.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.995+0000 7fe82a08c700 1 -- 172.21.15.99:0/1686344760 shutdown_connections 2023-10-08T00:11:05.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.995+0000 7fe82a08c700 1 -- 172.21.15.99:0/1686344760 wait complete. 2023-10-08T00:11:05.002 INFO:teuthology.orchestra.run.smithi099.stdout:55834574853 2023-10-08T00:11:05.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.995+0000 7f4d109f2700 1 -- 172.21.15.99:0/515141564 wait complete. 2023-10-08T00:11:05.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.995+0000 7fce5fb79700 1 -- 172.21.15.99:0/1749712096 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fce38005830 con 0x7fce58157870 2023-10-08T00:11:05.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.996+0000 7fce4dffb700 1 -- 172.21.15.99:0/1749712096 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fce54050b80 con 0x7fce58157870 2023-10-08T00:11:05.004 INFO:teuthology.orchestra.run.smithi099.stdout:55834574853 2023-10-08T00:11:05.004 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.997+0000 7fafaaffd700 1 -- 172.21.15.99:0/1337588190 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fafac050ca0 con 0x7fafbc0a7da0 2023-10-08T00:11:05.005 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.998+0000 7fa7e8aec700 1 -- 172.21.15.99:0/4246690376 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa7c4066fb0 msgr2=0x7fa7c4069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:05.005 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.998+0000 7fa7e8aec700 1 --2- 172.21.15.99:0/4246690376 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa7c4066fb0 0x7fa7c4069460 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa7e40afbb0 tx=0x7fa7dc009040 comp rx=0 tx=0).stop 2023-10-08T00:11:05.005 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.998+0000 7fa7e8aec700 1 -- 172.21.15.99:0/4246690376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7e40afdf0 msgr2=0x7fa7e40c39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:05.005 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.998+0000 7fa7e8aec700 1 --2- 172.21.15.99:0/4246690376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7e40afdf0 0x7fa7e40c39d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa7e40cc4a0 tx=0x7fa7d4017000 comp rx=0 tx=0).stop 2023-10-08T00:11:05.005 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.998+0000 7fa7e8aec700 1 -- 172.21.15.99:0/4246690376 shutdown_connections 2023-10-08T00:11:05.005 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.998+0000 7fa7e8aec700 1 --2- 172.21.15.99:0/4246690376 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa7c4066fb0 0x7fa7c4069460 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:05.005 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.998+0000 7fa7e8aec700 1 --2- 172.21.15.99:0/4246690376 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7e40cd1d0 0x7fa7e40c4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:05.005 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.998+0000 7fa7e8aec700 1 --2- 172.21.15.99:0/4246690376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7e40afdf0 0x7fa7e40c39d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:05.005 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.998+0000 7fa7e8aec700 1 --2- 172.21.15.99:0/4246690376 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7e40bfbe0 0x7fa7e416f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:05.005 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.998+0000 7fa7e8aec700 1 -- 172.21.15.99:0/4246690376 >> 172.21.15.99:0/4246690376 conn(0x7fa7e4070700 msgr2=0x7fa7e40c9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:05.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.998+0000 7fa7e8aec700 1 -- 172.21.15.99:0/4246690376 shutdown_connections 2023-10-08T00:11:05.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:04.998+0000 7fa7e8aec700 1 -- 172.21.15.99:0/4246690376 wait complete. 2023-10-08T00:11:05.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:05.000+0000 7fce5fb79700 1 -- 172.21.15.99:0/1749712096 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fce3c066f30 msgr2=0x7fce3c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:05.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:05.000+0000 7fce5fb79700 1 --2- 172.21.15.99:0/1749712096 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fce3c066f30 0x7fce3c0693e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fce5815aa20 tx=0x7fce48017000 comp rx=0 tx=0).stop 2023-10-08T00:11:05.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:05.000+0000 7fce5fb79700 1 -- 172.21.15.99:0/1749712096 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce58157870 msgr2=0x7fce5816fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:05.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:05.000+0000 7fce5fb79700 1 --2- 172.21.15.99:0/1749712096 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce58157870 0x7fce5816fa40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fce5400c7c0 tx=0x7fce54007f70 comp rx=0 tx=0).stop 2023-10-08T00:11:05.007 INFO:teuthology.orchestra.run.smithi099.stdout:55834574853 2023-10-08T00:11:05.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:05.001+0000 7fce5fb79700 1 -- 172.21.15.99:0/1749712096 shutdown_connections 2023-10-08T00:11:05.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:05.001+0000 7fce5fb79700 1 --2- 172.21.15.99:0/1749712096 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fce3c066f30 0x7fce3c0693e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:05.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:05.001+0000 7fce5fb79700 1 --2- 172.21.15.99:0/1749712096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce5815d580 0x7fce581521f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:05.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:05.001+0000 7fce5fb79700 1 --2- 172.21.15.99:0/1749712096 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce58157870 0x7fce5816fa40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:05.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:05.001+0000 7fce5fb79700 1 --2- 172.21.15.99:0/1749712096 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce5815a620 0x7fce5814c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:05.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:05.001+0000 7fce5fb79700 1 -- 172.21.15.99:0/1749712096 >> 172.21.15.99:0/1749712096 conn(0x7fce58068ce0 msgr2=0x7fce580690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:05.008 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.3 2023-10-08T00:11:05.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:05.003+0000 7fafc166f700 1 -- 172.21.15.99:0/1337588190 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fafa0066f00 msgr2=0x7fafa00693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:05.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:05.003+0000 7fafc166f700 1 --2- 172.21.15.99:0/1337588190 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fafa0066f00 0x7fafa00693b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fafb4009ab0 tx=0x7fafb4018000 comp rx=0 tx=0).stop 2023-10-08T00:11:05.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:05.003+0000 7fafc166f700 1 -- 172.21.15.99:0/1337588190 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fafbc0a7da0 msgr2=0x7fafbc1427f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:05.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:05.003+0000 7fafc166f700 1 --2- 172.21.15.99:0/1337588190 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fafbc0a7da0 0x7fafbc1427f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fafac00c590 tx=0x7fafac00c8a0 comp rx=0 tx=0).stop 2023-10-08T00:11:05.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:05.003+0000 7fafc166f700 1 -- 172.21.15.99:0/1337588190 shutdown_connections 2023-10-08T00:11:05.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:05.003+0000 7fafc166f700 1 --2- 172.21.15.99:0/1337588190 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fafa0066f00 0x7fafa00693b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:05.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:05.003+0000 7fafc166f700 1 --2- 172.21.15.99:0/1337588190 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fafbc0a7da0 0x7fafbc1427f0 secure :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fafac00c590 tx=0x7fafac00c8a0 comp rx=0 tx=0).stop 2023-10-08T00:11:05.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:05.003+0000 7fafc166f700 1 --2- 172.21.15.99:0/1337588190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafbc15e5a0 0x7fafbc15cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:05.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:05.003+0000 7fafc166f700 1 --2- 172.21.15.99:0/1337588190 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fafbc15e1d0 0x7fafbc0acae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:05.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:05.003+0000 7fafc166f700 1 -- 172.21.15.99:0/1337588190 >> 172.21.15.99:0/1337588190 conn(0x7fafbc0cd1e0 msgr2=0x7fafbc0981a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:05.011 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.0 2023-10-08T00:11:05.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:05.009+0000 7f8f4c311700 1 -- 172.21.15.99:0/3395954984 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f8f4406d1c0 con 0x7f8f440c7820 2023-10-08T00:11:05.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:05.009+0000 7fce5fb79700 1 -- 172.21.15.99:0/1749712096 shutdown_connections 2023-10-08T00:11:05.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:05.009+0000 7fafc166f700 1 -- 172.21.15.99:0/1337588190 shutdown_connections 2023-10-08T00:11:05.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:05.009+0000 7fafc166f700 1 -- 172.21.15.99:0/1337588190 wait complete. 2023-10-08T00:11:05.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:05.009+0000 7f8f3a7fc700 1 -- 172.21.15.99:0/3395954984 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8f40054930 con 0x7f8f440c7820 2023-10-08T00:11:05.012 INFO:teuthology.orchestra.run.smithi099.stdout:55834574853 2023-10-08T00:11:05.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:05.013+0000 7fce5fb79700 1 -- 172.21.15.99:0/1749712096 wait complete. 2023-10-08T00:11:05.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:05.013+0000 7f8f4c311700 1 -- 172.21.15.99:0/3395954984 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8f28066f30 msgr2=0x7f8f280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:05.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:05.013+0000 7f8f4c311700 1 --2- 172.21.15.99:0/3395954984 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8f28066f30 0x7f8f280693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8f340099b0 tx=0x7f8f34016000 comp rx=0 tx=0).stop 2023-10-08T00:11:05.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:05.013+0000 7f8f4c311700 1 -- 172.21.15.99:0/3395954984 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f440c7820 msgr2=0x7f8f4415b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:05.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:05.013+0000 7f8f4c311700 1 --2- 172.21.15.99:0/3395954984 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f440c7820 0x7f8f4415b7a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8f4000b600 tx=0x7f8f400091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:05.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:05.013+0000 7f8f4c311700 1 -- 172.21.15.99:0/3395954984 shutdown_connections 2023-10-08T00:11:05.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:05.013+0000 7f8f4c311700 1 --2- 172.21.15.99:0/3395954984 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8f28066f30 0x7f8f280693e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:05.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:05.013+0000 7f8f4c311700 1 --2- 172.21.15.99:0/3395954984 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f440c7820 0x7f8f4415b7a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:05.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:05.013+0000 7f8f4c311700 1 --2- 172.21.15.99:0/3395954984 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f44157ed0 0x7f8f4416bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:05.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:05.013+0000 7f8f4c311700 1 --2- 172.21.15.99:0/3395954984 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f44157b00 0x7f8f44098040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:05.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:05.013+0000 7f8f4c311700 1 -- 172.21.15.99:0/3395954984 >> 172.21.15.99:0/3395954984 conn(0x7f8f440cd1e0 msgr2=0x7f8f4406aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:05.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:05.013+0000 7f8f4c311700 1 -- 172.21.15.99:0/3395954984 shutdown_connections 2023-10-08T00:11:05.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:05.013+0000 7f8f4c311700 1 -- 172.21.15.99:0/3395954984 wait complete. 2023-10-08T00:11:05.020 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.5 2023-10-08T00:11:05.021 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.1 2023-10-08T00:11:05.022 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.4 2023-10-08T00:11:05.031 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.2 2023-10-08T00:11:05.034 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.7 2023-10-08T00:11:05.034 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.6 2023-10-08T00:11:06.009 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-08T00:11:06.011 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-08T00:11:06.021 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-08T00:11:06.022 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-08T00:11:06.023 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-08T00:11:06.032 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-08T00:11:06.035 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-08T00:11:06.035 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-08T00:11:06.182 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.181+0000 7ff668afb700 1 Processor -- start 2023-10-08T00:11:06.183 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.181+0000 7ff668afb700 1 -- start start 2023-10-08T00:11:06.183 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.182+0000 7ff668afb700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff6640cc450 0x7ff6640c6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.183 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.182+0000 7ff668afb700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff6640c73f0 0x7ff664155b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.183 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.182+0000 7f4bb4763700 1 Processor -- start 2023-10-08T00:11:06.184 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.182+0000 7ff668afb700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6640c8760 con 0x7ff6641560a0 2023-10-08T00:11:06.184 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.182+0000 7ff668afb700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6640c9d00 con 0x7ff6640cc450 2023-10-08T00:11:06.184 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.182+0000 7ff668afb700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff66406d9f0 con 0x7ff6640c73f0 2023-10-08T00:11:06.184 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.183+0000 7ff661d9b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff6640c73f0 0x7ff664155b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:06.184 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.183+0000 7ff661d9b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff6640c73f0 0x7ff664155b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.99:54712/0 (socket says 172.21.15.99:54712) 2023-10-08T00:11:06.184 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.183+0000 7ff661d9b700 1 -- 172.21.15.99:0/644746191 learned_addr learned my addr 172.21.15.99:0/644746191 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:06.184 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.183+0000 7ff66259c700 1 --2- 172.21.15.99:0/644746191 >> v2:172.21.15.99:3300/0 conn(0x7ff6640cc450 0x7ff6640c6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:06.185 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.182+0000 7f4bb4763700 1 -- start start 2023-10-08T00:11:06.185 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.183+0000 7ff661d9b700 1 -- 172.21.15.99:0/644746191 >> v2:172.21.15.99:3300/0 conn(0x7ff6640cc450 msgr2=0x7ff6640c6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:06.185 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.183+0000 7ff661d9b700 1 --2- 172.21.15.99:0/644746191 >> v2:172.21.15.99:3300/0 conn(0x7ff6640cc450 0x7ff6640c6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.185 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.183+0000 7ff661d9b700 1 -- 172.21.15.99:0/644746191 >> v1:172.21.15.191:6789/0 conn(0x7ff6641560a0 legacy=0x7ff66406d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:06.185 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.183+0000 7ff661d9b700 1 -- 172.21.15.99:0/644746191 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6640af6e0 con 0x7ff6640c73f0 2023-10-08T00:11:06.186 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.183+0000 7ff661d9b700 1 --2- 172.21.15.99:0/644746191 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6640c73f0 0x7ff664155b60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff6640c3890 tx=0x7ff65c009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cafb12bc272cff0a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:06.186 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.184+0000 7ff66159a700 1 -- 172.21.15.99:0/644746191 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65c00e070 con 0x7ff6640c73f0 2023-10-08T00:11:06.186 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.184+0000 7ff66159a700 1 -- 172.21.15.99:0/644746191 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff65c00b9e0 con 0x7ff6640c73f0 2023-10-08T00:11:06.186 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.184+0000 7ff66159a700 1 -- 172.21.15.99:0/644746191 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65c0092e0 con 0x7ff6640c73f0 2023-10-08T00:11:06.186 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.184+0000 7f4bb4763700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4bac0c75c0 0x7f4bac155b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.186 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.184+0000 7f4bb4763700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4bac06d6b0 0x7f4bac1560a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.186 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.184+0000 7f4bb4763700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bac1565e0 con 0x7f4bac06d2e0 2023-10-08T00:11:06.186 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.184+0000 7f4bb4763700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bac0c8760 con 0x7f4bac06d6b0 2023-10-08T00:11:06.186 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.184+0000 7f4bb4763700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bac0af6e0 con 0x7f4bac0c75c0 2023-10-08T00:11:06.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.184+0000 7f4bb24ff700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f4bac06d2e0 0x7f4bac0c6eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45056/0 (socket says 172.21.15.99:45056) 2023-10-08T00:11:06.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.184+0000 7f4bb24ff700 1 -- 172.21.15.99:0/1089652800 learned_addr learned my addr 172.21.15.99:0/1089652800 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:06.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.185+0000 7f4bb1cfe700 1 --2- 172.21.15.99:0/1089652800 >> v2:172.21.15.191:3300/0 conn(0x7f4bac0c75c0 0x7f4bac155b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:06.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.185+0000 7ff668afb700 1 -- 172.21.15.99:0/644746191 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6640c73f0 msgr2=0x7ff664155b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:06.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.185+0000 7ff668afb700 1 --2- 172.21.15.99:0/644746191 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6640c73f0 0x7ff664155b60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff6640c3890 tx=0x7ff65c009840 comp rx=0 tx=0).stop 2023-10-08T00:11:06.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.185+0000 7f4bb14fd700 1 -- 172.21.15.99:0/1089652800 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2221538034 0 0) 0x7f4ba8001110 con 0x7f4bac06d2e0 2023-10-08T00:11:06.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.185+0000 7f4bb14fd700 1 -- 172.21.15.99:0/1089652800 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b98003570 con 0x7f4bac06d2e0 2023-10-08T00:11:06.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.185+0000 7ff668afb700 1 -- 172.21.15.99:0/644746191 shutdown_connections 2023-10-08T00:11:06.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.185+0000 7ff668afb700 1 --2- 172.21.15.99:0/644746191 >> v2:172.21.15.99:3300/0 conn(0x7ff6640cc450 0x7ff6640c6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.185+0000 7ff668afb700 1 --2- 172.21.15.99:0/644746191 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6640c73f0 0x7ff664155b60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.185+0000 7ff668afb700 1 -- 172.21.15.99:0/644746191 >> 172.21.15.99:0/644746191 conn(0x7ff664070700 msgr2=0x7ff664070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:06.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.186+0000 7ff668afb700 1 -- 172.21.15.99:0/644746191 shutdown_connections 2023-10-08T00:11:06.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.186+0000 7ff668afb700 1 -- 172.21.15.99:0/644746191 wait complete. 2023-10-08T00:11:06.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.187+0000 7ff668afb700 1 Processor -- start 2023-10-08T00:11:06.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.185+0000 7f4bb1cfe700 1 -- 172.21.15.99:0/1089652800 >> v2:172.21.15.99:3300/0 conn(0x7f4bac06d6b0 msgr2=0x7f4bac1560a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:06.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.185+0000 7f4bb1cfe700 1 --2- 172.21.15.99:0/1089652800 >> v2:172.21.15.99:3300/0 conn(0x7f4bac06d6b0 0x7f4bac1560a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.185+0000 7f4bb1cfe700 1 -- 172.21.15.99:0/1089652800 >> v1:172.21.15.99:6789/0 conn(0x7f4bac06d2e0 legacy=0x7f4bac0c6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:06.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.185+0000 7f4bb1cfe700 1 -- 172.21.15.99:0/1089652800 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bac0c7b50 con 0x7f4bac0c75c0 2023-10-08T00:11:06.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.186+0000 7f4bb1cfe700 1 --2- 172.21.15.99:0/1089652800 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4bac0c75c0 0x7f4bac155b60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4bac0c3890 tx=0x7f4ba4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5279c3faa67d311b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:06.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.186+0000 7f4bb14fd700 1 -- 172.21.15.99:0/1089652800 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba400e070 con 0x7f4bac0c75c0 2023-10-08T00:11:06.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.186+0000 7f4bb14fd700 1 -- 172.21.15.99:0/1089652800 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ba400bb60 con 0x7f4bac0c75c0 2023-10-08T00:11:06.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.187+0000 7f4bb4763700 1 -- 172.21.15.99:0/1089652800 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4bac0c75c0 msgr2=0x7f4bac155b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:06.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.187+0000 7f4bb4763700 1 --2- 172.21.15.99:0/1089652800 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4bac0c75c0 0x7f4bac155b60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4bac0c3890 tx=0x7f4ba4009780 comp rx=0 tx=0).stop 2023-10-08T00:11:06.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.188+0000 7ff668afb700 1 -- start start 2023-10-08T00:11:06.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.189+0000 7ff668afb700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6640c73f0 0x7ff6640c5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.189+0000 7ff668afb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6640cc450 0x7ff66414c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.189+0000 7ff668afb700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6641560a0 0x7ff66414cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.189+0000 7ff668afb700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6641556b0 con 0x7ff6640cc450 2023-10-08T00:11:06.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.189+0000 7ff668afb700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff664157e60 con 0x7ff6641560a0 2023-10-08T00:11:06.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.189+0000 7ff668afb700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff664158990 con 0x7ff6640c73f0 2023-10-08T00:11:06.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.189+0000 7ff66259c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6640c73f0 0x7ff6640c5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:06.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.189+0000 7ff66259c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6640c73f0 0x7ff6640c5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.99:45730/0 (socket says 172.21.15.99:45730) 2023-10-08T00:11:06.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.189+0000 7ff66259c700 1 -- 172.21.15.99:0/1875536605 learned_addr learned my addr 172.21.15.99:0/1875536605 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:06.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.189+0000 7ff662d9d700 1 --2- 172.21.15.99:0/1875536605 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6641560a0 0x7ff66414cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:06.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.189+0000 7f4bb4763700 1 -- 172.21.15.99:0/1089652800 shutdown_connections 2023-10-08T00:11:06.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.189+0000 7f4bb4763700 1 --2- 172.21.15.99:0/1089652800 >> v2:172.21.15.99:3300/0 conn(0x7f4bac06d6b0 0x7f4bac1560a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.189+0000 7f4bb4763700 1 --2- 172.21.15.99:0/1089652800 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4bac0c75c0 0x7f4bac155b60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.189+0000 7f4bb4763700 1 -- 172.21.15.99:0/1089652800 >> 172.21.15.99:0/1089652800 conn(0x7f4bac070700 msgr2=0x7f4bac070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:06.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.190+0000 7ff66259c700 1 -- 172.21.15.99:0/1875536605 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6641560a0 msgr2=0x7ff66414cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:06.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.190+0000 7ff66259c700 1 --2- 172.21.15.99:0/1875536605 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6641560a0 0x7ff66414cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.190+0000 7ff66259c700 1 -- 172.21.15.99:0/1875536605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6640cc450 msgr2=0x7ff66414c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:06.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.190+0000 7ff66259c700 1 --2- 172.21.15.99:0/1875536605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6640cc450 0x7ff66414c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.190+0000 7ff66259c700 1 -- 172.21.15.99:0/1875536605 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff65c009ae0 con 0x7ff6640c73f0 2023-10-08T00:11:06.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.190+0000 7f4bb4763700 1 -- 172.21.15.99:0/1089652800 shutdown_connections 2023-10-08T00:11:06.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.190+0000 7f4bb4763700 1 -- 172.21.15.99:0/1089652800 wait complete. 2023-10-08T00:11:06.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.192+0000 7ff66259c700 1 --2- 172.21.15.99:0/1875536605 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6640c73f0 0x7ff6640c5f40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff65800ec80 tx=0x7ff6580091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:06.194 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.192+0000 7f4bb4763700 1 Processor -- start 2023-10-08T00:11:06.194 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.192+0000 7f4bb4763700 1 -- start start 2023-10-08T00:11:06.194 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.193+0000 7f4bb4763700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4bac06d2e0 0x7f4bac0ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.194 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.193+0000 7f4bb4763700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4bac06d6b0 0x7f4bac177cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.194 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.193+0000 7f4bb4763700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4bac0c75c0 0x7f4bac0ae690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.194 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.193+0000 7f4bb4763700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bac178230 con 0x7f4bac0c75c0 2023-10-08T00:11:06.194 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.193+0000 7f4bb4763700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bac0ad8d0 con 0x7f4bac06d6b0 2023-10-08T00:11:06.194 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.193+0000 7f4bb4763700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bac0adbb0 con 0x7f4bac06d2e0 2023-10-08T00:11:06.195 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.193+0000 7ff6527fc700 1 -- 172.21.15.99:0/1875536605 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65800c9d0 con 0x7ff6640c73f0 2023-10-08T00:11:06.195 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.193+0000 7ff6527fc700 1 -- 172.21.15.99:0/1875536605 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff65800cb30 con 0x7ff6640c73f0 2023-10-08T00:11:06.195 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.193+0000 7ff6527fc700 1 -- 172.21.15.99:0/1875536605 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff658013470 con 0x7ff6640c73f0 2023-10-08T00:11:06.195 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.193+0000 7ff668afb700 1 -- 172.21.15.99:0/1875536605 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff66414d180 con 0x7ff6640c73f0 2023-10-08T00:11:06.195 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.194+0000 7ff668afb700 1 -- 172.21.15.99:0/1875536605 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff664158060 con 0x7ff6640c73f0 2023-10-08T00:11:06.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.194+0000 7f4bb1cfe700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4bac06d6b0 0x7f4bac177cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:06.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.194+0000 7f4bb24ff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4bac06d2e0 0x7f4bac0ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:06.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.195+0000 7f4bb24ff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4bac06d2e0 0x7f4bac0ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.99:45744/0 (socket says 172.21.15.99:45744) 2023-10-08T00:11:06.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.195+0000 7f4bb24ff700 1 -- 172.21.15.99:0/1006454474 learned_addr learned my addr 172.21.15.99:0/1006454474 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:06.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.195+0000 7f4bb1cfe700 1 -- 172.21.15.99:0/1006454474 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4bac06d2e0 msgr2=0x7f4bac0ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:06.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.195+0000 7f4bb1cfe700 1 --2- 172.21.15.99:0/1006454474 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4bac06d2e0 0x7f4bac0ac890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.195+0000 7f4bb1cfe700 1 -- 172.21.15.99:0/1006454474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4bac0c75c0 msgr2=0x7f4bac0ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:06.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.195+0000 7f4bb1cfe700 1 --2- 172.21.15.99:0/1006454474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4bac0c75c0 0x7f4bac0ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.195+0000 7f4bb1cfe700 1 -- 172.21.15.99:0/1006454474 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba4009a20 con 0x7f4bac06d6b0 2023-10-08T00:11:06.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.196+0000 7f4bb1cfe700 1 --2- 172.21.15.99:0/1006454474 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4bac06d6b0 0x7f4bac177cf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4016000 tx=0x7f4ba4019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:06.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.195+0000 7ff668afb700 1 -- 172.21.15.99:0/1875536605 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6440052e0 con 0x7ff6640c73f0 2023-10-08T00:11:06.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.197+0000 7f4ba27fc700 1 -- 172.21.15.99:0/1006454474 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba4012c70 con 0x7f4bac06d6b0 2023-10-08T00:11:06.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.197+0000 7f4ba27fc700 1 -- 172.21.15.99:0/1006454474 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ba4012dd0 con 0x7f4bac06d6b0 2023-10-08T00:11:06.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.197+0000 7f4ba27fc700 1 -- 172.21.15.99:0/1006454474 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba4026620 con 0x7f4bac06d6b0 2023-10-08T00:11:06.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.196+0000 7ff6527fc700 1 -- 172.21.15.99:0/1875536605 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff658005a00 con 0x7ff6640c73f0 2023-10-08T00:11:06.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.197+0000 7ff6527fc700 1 --2- 172.21.15.99:0/1875536605 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff64c0671e0 0x7ff64c069690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.197+0000 7ff6527fc700 1 -- 172.21.15.99:0/1875536605 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff658084a90 con 0x7ff6640c73f0 2023-10-08T00:11:06.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.197+0000 7ff661d9b700 1 --2- 172.21.15.99:0/1875536605 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff64c0671e0 0x7ff64c069690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:06.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.197+0000 7f4bb4763700 1 -- 172.21.15.99:0/1006454474 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bac0ade90 con 0x7f4bac06d6b0 2023-10-08T00:11:06.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.197+0000 7f4bb4763700 1 -- 172.21.15.99:0/1006454474 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bac0aebd0 con 0x7f4bac06d6b0 2023-10-08T00:11:06.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.198+0000 7f4bb4763700 1 -- 172.21.15.99:0/1006454474 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bac0ada10 con 0x7f4bac06d6b0 2023-10-08T00:11:06.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.199+0000 7f4ba27fc700 1 -- 172.21.15.99:0/1006454474 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4ba402f450 con 0x7f4bac06d6b0 2023-10-08T00:11:06.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.200+0000 7f4ba27fc700 1 --2- 172.21.15.99:0/1006454474 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4b98067510 0x7f4b980699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.200+0000 7f4ba27fc700 1 -- 172.21.15.99:0/1006454474 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4ba4085f50 con 0x7f4bac06d6b0 2023-10-08T00:11:06.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.200+0000 7ff661d9b700 1 --2- 172.21.15.99:0/1875536605 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff64c0671e0 0x7ff64c069690 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff6640aee50 tx=0x7ff65c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:06.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.201+0000 7f4bb24ff700 1 --2- 172.21.15.99:0/1006454474 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4b98067510 0x7f4b980699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:06.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.201+0000 7f4bb24ff700 1 --2- 172.21.15.99:0/1006454474 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4b98067510 0x7f4b980699c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8008fd0 tx=0x7f4ba8005cc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:06.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.202+0000 7ff6527fc700 1 -- 172.21.15.99:0/1875536605 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff658051d60 con 0x7ff6640c73f0 2023-10-08T00:11:06.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.207+0000 7f4ba27fc700 1 -- 172.21.15.99:0/1006454474 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4bac0ada10 con 0x7f4bac06d6b0 2023-10-08T00:11:06.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.252+0000 7f2d91b4f700 1 Processor -- start 2023-10-08T00:11:06.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.252+0000 7f2d91b4f700 1 -- start start 2023-10-08T00:11:06.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.253+0000 7f87c2106700 1 Processor -- start 2023-10-08T00:11:06.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.253+0000 7f87c2106700 1 -- start start 2023-10-08T00:11:06.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.254+0000 7f2d91b4f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2d8c0cc450 0x7f2d8c0c6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.254+0000 7f2d91b4f700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d8c0c8760 con 0x7f2d8c0c73f0 2023-10-08T00:11:06.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.254+0000 7f2d91b4f700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d8c0af6e0 con 0x7f2d8c156270 2023-10-08T00:11:06.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.254+0000 7f2d91b4f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c0c7b50 con 0x7f2d8c0cc450 2023-10-08T00:11:06.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.254+0000 7f87c2106700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f87bc0bf970 0x7f87bc0bfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.254+0000 7f87c2106700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87bc0ac1a0 0x7f87bc0ac600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.254+0000 7f87c2106700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87bc0acb40 con 0x7f87bc06d4c0 2023-10-08T00:11:06.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.254+0000 7f87c2106700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87bc0c7960 con 0x7f87bc0bf970 2023-10-08T00:11:06.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.254+0000 7f87c2106700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87bc0c7c40 con 0x7f87bc0ac1a0 2023-10-08T00:11:06.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.254+0000 7f2d8affd700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f2d8c0c73f0 0x7f2d8c155b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45070/0 (socket says 172.21.15.99:45070) 2023-10-08T00:11:06.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.254+0000 7f2d8affd700 1 -- 172.21.15.99:0/3125051260 learned_addr learned my addr 172.21.15.99:0/3125051260 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:06.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.254+0000 7f2d8b7fe700 1 --2- 172.21.15.99:0/3125051260 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2d8c0cc450 0x7f2d8c0c6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:06.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.254+0000 7f87baffd700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f87bc06d4c0 0x7f87bc06ec60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45076/0 (socket says 172.21.15.99:45076) 2023-10-08T00:11:06.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.254+0000 7f87baffd700 1 -- 172.21.15.99:0/3934608606 learned_addr learned my addr 172.21.15.99:0/3934608606 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:06.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.254+0000 7f87bbfff700 1 --2- 172.21.15.99:0/3934608606 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87bc0ac1a0 0x7f87bc0ac600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:06.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.254+0000 7f87bb7fe700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f87bc0bf970 0x7f87bc0bfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:06.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.255+0000 7fb6ec84c700 1 Processor -- start 2023-10-08T00:11:06.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.255+0000 7f2d8b7fe700 1 -- 172.21.15.99:0/3125051260 >> v1:172.21.15.191:6789/0 conn(0x7f2d8c156270 legacy=0x7f2d8c0c8bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T00:11:06.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.255+0000 7f2d8b7fe700 1 -- 172.21.15.99:0/3125051260 >> v1:172.21.15.99:6789/0 conn(0x7f2d8c0c73f0 legacy=0x7f2d8c155b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:06.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.255+0000 7f2d8b7fe700 1 -- 172.21.15.99:0/3125051260 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d8c0c7e30 con 0x7f2d8c0cc450 2023-10-08T00:11:06.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.255+0000 7f87bbfff700 1 -- 172.21.15.99:0/3934608606 >> v2:172.21.15.191:3300/0 conn(0x7f87bc0bf970 msgr2=0x7f87bc0bfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:06.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.255+0000 7f87bbfff700 1 --2- 172.21.15.99:0/3934608606 >> v2:172.21.15.191:3300/0 conn(0x7f87bc0bf970 0x7f87bc0bfd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.255+0000 7f87bbfff700 1 -- 172.21.15.99:0/3934608606 >> v1:172.21.15.99:6789/0 conn(0x7f87bc06d4c0 legacy=0x7f87bc06ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:06.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.255+0000 7f87bbfff700 1 -- 172.21.15.99:0/3934608606 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87bc0c7f20 con 0x7f87bc0ac1a0 2023-10-08T00:11:06.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.255+0000 7fb6ec84c700 1 -- start start 2023-10-08T00:11:06.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.255+0000 7f87bbfff700 1 --2- 172.21.15.99:0/3934608606 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87bc0ac1a0 0x7f87bc0ac600 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f87b4009e90 tx=0x7f87b4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64c550a5194f6e4e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:06.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.255+0000 7f87ba7fc700 1 -- 172.21.15.99:0/3934608606 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b400e070 con 0x7f87bc0ac1a0 2023-10-08T00:11:06.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.255+0000 7f87ba7fc700 1 -- 172.21.15.99:0/3934608606 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b400bc80 con 0x7f87bc0ac1a0 2023-10-08T00:11:06.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.255+0000 7f87ba7fc700 1 -- 172.21.15.99:0/3934608606 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b4012430 con 0x7f87bc0ac1a0 2023-10-08T00:11:06.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.256+0000 7fb6ec84c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb6e806d2e0 0x7fb6e80c6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.256+0000 7fb6ec84c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb6e806d6b0 0x7fb6e80c73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.256+0000 7fb6ec84c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb6e8155b60 0x7fb6e8155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.256+0000 7fb6ec84c700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6e80c7930 con 0x7fb6e806d6b0 2023-10-08T00:11:06.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.256+0000 7fb6ec84c700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6e80c8760 con 0x7fb6e8155b60 2023-10-08T00:11:06.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.256+0000 7fb6ec84c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e80af6e0 con 0x7fb6e806d2e0 2023-10-08T00:11:06.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.256+0000 7f87c2106700 1 -- 172.21.15.99:0/3934608606 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87bc0ac1a0 msgr2=0x7f87bc0ac600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:06.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.256+0000 7f87c2106700 1 --2- 172.21.15.99:0/3934608606 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87bc0ac1a0 0x7f87bc0ac600 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f87b4009e90 tx=0x7f87b4009780 comp rx=0 tx=0).stop 2023-10-08T00:11:06.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.256+0000 7f87c2106700 1 -- 172.21.15.99:0/3934608606 shutdown_connections 2023-10-08T00:11:06.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.256+0000 7f87c2106700 1 --2- 172.21.15.99:0/3934608606 >> v2:172.21.15.191:3300/0 conn(0x7f87bc0bf970 0x7f87bc0bfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.256+0000 7f87c2106700 1 --2- 172.21.15.99:0/3934608606 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87bc0ac1a0 0x7f87bc0ac600 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.256+0000 7f87c2106700 1 -- 172.21.15.99:0/3934608606 >> 172.21.15.99:0/3934608606 conn(0x7f87bc067660 msgr2=0x7f87bc067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:06.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.256+0000 7fb6e6d9d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb6e8155b60 0x7fb6e8155fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:06.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.256+0000 7fb6e6d9d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb6e8155b60 0x7fb6e8155fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.99:54742/0 (socket says 172.21.15.99:54742) 2023-10-08T00:11:06.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.256+0000 7fb6e6d9d700 1 -- 172.21.15.99:0/4068859746 learned_addr learned my addr 172.21.15.99:0/4068859746 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:06.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.256+0000 7f87c2106700 1 -- 172.21.15.99:0/3934608606 shutdown_connections 2023-10-08T00:11:06.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.257+0000 7f87c2106700 1 -- 172.21.15.99:0/3934608606 wait complete. 2023-10-08T00:11:06.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.257+0000 7f2d8b7fe700 1 --2- 172.21.15.99:0/3125051260 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2d8c0cc450 0x7f2d8c0c6eb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2d84009e90 tx=0x7f2d84009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=899ee05114b8a31f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:06.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.257+0000 7fb6e5d9b700 1 --2- 172.21.15.99:0/4068859746 >> v2:172.21.15.99:3300/0 conn(0x7fb6e806d6b0 0x7fb6e80c73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:06.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.257+0000 7fb6e6d9d700 1 -- 172.21.15.99:0/4068859746 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb6e806d2e0 msgr2=0x7fb6e80c6eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T00:11:06.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.257+0000 7fb6e6d9d700 1 --2- 172.21.15.99:0/4068859746 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb6e806d2e0 0x7fb6e80c6eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.257+0000 7fb6e6d9d700 1 -- 172.21.15.99:0/4068859746 >> v2:172.21.15.99:3300/0 conn(0x7fb6e806d6b0 msgr2=0x7fb6e80c73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:06.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.257+0000 7fb6e6d9d700 1 --2- 172.21.15.99:0/4068859746 >> v2:172.21.15.99:3300/0 conn(0x7fb6e806d6b0 0x7fb6e80c73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.257+0000 7fb6e6d9d700 1 -- 172.21.15.99:0/4068859746 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e80c7b50 con 0x7fb6e8155b60 2023-10-08T00:11:06.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.256+0000 7f64f6ebe700 1 Processor -- start 2023-10-08T00:11:06.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.256+0000 7f64f6ebe700 1 -- start start 2023-10-08T00:11:06.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.257+0000 7fb6e6d9d700 1 --2- 172.21.15.99:0/4068859746 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb6e8155b60 0x7fb6e8155fb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb6e806a620 tx=0x7fb6dc00ac80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b9af5560656ad4c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:06.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.257+0000 7fb6e559a700 1 -- 172.21.15.99:0/4068859746 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc00f070 con 0x7fb6e8155b60 2023-10-08T00:11:06.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.258+0000 7fb6e559a700 1 -- 172.21.15.99:0/4068859746 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6dc008bf0 con 0x7fb6e8155b60 2023-10-08T00:11:06.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.258+0000 7fb6e559a700 1 -- 172.21.15.99:0/4068859746 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc00a0d0 con 0x7fb6e8155b60 2023-10-08T00:11:06.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.258+0000 7f2d8a7fc700 1 -- 172.21.15.99:0/3125051260 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8400e070 con 0x7f2d8c0cc450 2023-10-08T00:11:06.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.258+0000 7f2d8a7fc700 1 -- 172.21.15.99:0/3125051260 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d8400bbf0 con 0x7f2d8c0cc450 2023-10-08T00:11:06.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.258+0000 7f2d8a7fc700 1 -- 172.21.15.99:0/3125051260 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d84012430 con 0x7f2d8c0cc450 2023-10-08T00:11:06.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.258+0000 7f87c2106700 1 Processor -- start 2023-10-08T00:11:06.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.258+0000 7fb6ec84c700 1 -- 172.21.15.99:0/4068859746 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb6e8155b60 msgr2=0x7fb6e8155fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:06.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.258+0000 7fb6ec84c700 1 --2- 172.21.15.99:0/4068859746 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb6e8155b60 0x7fb6e8155fb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb6e806a620 tx=0x7fb6dc00ac80 comp rx=0 tx=0).stop 2023-10-08T00:11:06.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.259+0000 7fb6ec84c700 1 -- 172.21.15.99:0/4068859746 shutdown_connections 2023-10-08T00:11:06.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.259+0000 7fb6ec84c700 1 --2- 172.21.15.99:0/4068859746 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb6e8155b60 0x7fb6e8155fb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.259+0000 7fb6ec84c700 1 --2- 172.21.15.99:0/4068859746 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb6e806d2e0 0x7fb6e80c6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.259+0000 7fb6ec84c700 1 --2- 172.21.15.99:0/4068859746 >> v2:172.21.15.99:3300/0 conn(0x7fb6e806d6b0 0x7fb6e80c73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.259+0000 7fb6ec84c700 1 -- 172.21.15.99:0/4068859746 >> 172.21.15.99:0/4068859746 conn(0x7fb6e8070700 msgr2=0x7fb6e8070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:06.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.258+0000 7f87c2106700 1 -- start start 2023-10-08T00:11:06.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.258+0000 7f64f6ebe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f64f00bfcc0 0x7f64f00c9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.259+0000 7f64f4c5a700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f64f00bfcc0 0x7f64f00c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:06.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.259+0000 7f64f4c5a700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f64f00bfcc0 0x7f64f00c9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38848/0 (socket says 172.21.15.99:38848) 2023-10-08T00:11:06.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.259+0000 7f64f4c5a700 1 -- 172.21.15.99:0/1637295102 learned_addr learned my addr 172.21.15.99:0/1637295102 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:06.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.259+0000 7f64f6ebe700 1 -- 172.21.15.99:0/1637295102 --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64f015ea50 con 0x7f64f006d3a0 2023-10-08T00:11:06.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.259+0000 7f64f6ebe700 1 -- 172.21.15.99:0/1637295102 --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64f00c87c0 con 0x7f64f00696d0 2023-10-08T00:11:06.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.259+0000 7f64f6ebe700 1 -- 172.21.15.99:0/1637295102 --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64f00c85d0 con 0x7f64f00bfcc0 2023-10-08T00:11:06.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.259+0000 7f64f4c5a700 1 -- 172.21.15.99:0/1637295102 >> v1:172.21.15.191:6789/0 conn(0x7f64f00696d0 legacy=0x7f64f00c7ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T00:11:06.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.259+0000 7f64f4c5a700 1 -- 172.21.15.99:0/1637295102 >> v1:172.21.15.99:6789/0 conn(0x7f64f006d3a0 legacy=0x7f64f0068fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:06.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.259+0000 7f64ef7fe700 1 -- 172.21.15.99:0/1637295102 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1755628003 0 0) 0x7f64f015ea50 con 0x7f64f006d3a0 2023-10-08T00:11:06.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.259+0000 7f87c2106700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87bc06d4c0 0x7f87bc0b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.259+0000 7f87c2106700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87bc0ac1a0 0x7f87bc0c3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.259+0000 7f87c2106700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87bc0bf970 0x7f87bc0c08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.259+0000 7f87c2106700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87bc0b1640 con 0x7f87bc06d4c0 2023-10-08T00:11:06.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.259+0000 7f87c2106700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87bc0b2010 con 0x7f87bc0bf970 2023-10-08T00:11:06.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.259+0000 7f87c2106700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87bc0c3e60 con 0x7f87bc0ac1a0 2023-10-08T00:11:06.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.259+0000 7f87bb7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87bc06d4c0 0x7f87bc0b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:06.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.259+0000 7f87bb7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87bc06d4c0 0x7f87bc0b1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38856/0 (socket says 172.21.15.99:38856) 2023-10-08T00:11:06.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.259+0000 7f87bb7fe700 1 -- 172.21.15.99:0/2488382271 learned_addr learned my addr 172.21.15.99:0/2488382271 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:06.275 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.259+0000 7f64f4c5a700 1 -- 172.21.15.99:0/1637295102 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f00c8aa0 con 0x7f64f00bfcc0 2023-10-08T00:11:06.275 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.260+0000 7f87bbfff700 1 --2- 172.21.15.99:0/2488382271 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87bc0bf970 0x7f87bc0c08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:06.275 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.260+0000 7f64f4c5a700 1 --2- 172.21.15.99:0/1637295102 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64f00bfcc0 0x7f64f00c9860 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f64e40099b0 tx=0x7f64e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c51d901d1b4dda10 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:06.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.260+0000 7f87baffd700 1 --2- 172.21.15.99:0/2488382271 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87bc0ac1a0 0x7f87bc0c3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:06.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.260+0000 7f87bb7fe700 1 -- 172.21.15.99:0/2488382271 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87bc0ac1a0 msgr2=0x7f87bc0c3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:06.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.260+0000 7f87bb7fe700 1 --2- 172.21.15.99:0/2488382271 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87bc0ac1a0 0x7f87bc0c3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.260+0000 7f87bb7fe700 1 -- 172.21.15.99:0/2488382271 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87bc0bf970 msgr2=0x7f87bc0c08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:06.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.260+0000 7f87bb7fe700 1 --2- 172.21.15.99:0/2488382271 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87bc0bf970 0x7f87bc0c08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.260+0000 7f87bb7fe700 1 -- 172.21.15.99:0/2488382271 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b4009a20 con 0x7f87bc06d4c0 2023-10-08T00:11:06.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.260+0000 7f87bb7fe700 1 --2- 172.21.15.99:0/2488382271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87bc06d4c0 0x7f87bc0b1ad0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f87b000e930 tx=0x7f87b00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:06.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.260+0000 7f64ef7fe700 1 -- 172.21.15.99:0/1637295102 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e400b820 con 0x7f64f00bfcc0 2023-10-08T00:11:06.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.260+0000 7f64ef7fe700 1 -- 172.21.15.99:0/1637295102 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e400b980 con 0x7f64f00bfcc0 2023-10-08T00:11:06.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.260+0000 7f64ef7fe700 1 -- 172.21.15.99:0/1637295102 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e400bd20 con 0x7f64f00bfcc0 2023-10-08T00:11:06.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.260+0000 7f87a37fe700 1 -- 172.21.15.99:0/2488382271 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b0013680 con 0x7f87bc06d4c0 2023-10-08T00:11:06.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.261+0000 7f64f6ebe700 1 -- 172.21.15.99:0/1637295102 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64f00bfcc0 msgr2=0x7f64f00c9860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:06.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.261+0000 7f64f6ebe700 1 --2- 172.21.15.99:0/1637295102 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64f00bfcc0 0x7f64f00c9860 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f64e40099b0 tx=0x7f64e4009250 comp rx=0 tx=0).stop 2023-10-08T00:11:06.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.260+0000 7f87a37fe700 1 -- 172.21.15.99:0/2488382271 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b00137e0 con 0x7f87bc06d4c0 2023-10-08T00:11:06.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.260+0000 7f87a37fe700 1 -- 172.21.15.99:0/2488382271 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b0013ba0 con 0x7f87bc06d4c0 2023-10-08T00:11:06.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.261+0000 7f87c2106700 1 -- 172.21.15.99:0/2488382271 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87bc0b0a30 con 0x7f87bc06d4c0 2023-10-08T00:11:06.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.261+0000 7f87c2106700 1 -- 172.21.15.99:0/2488382271 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87bc0b0ff0 con 0x7f87bc06d4c0 2023-10-08T00:11:06.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.262+0000 7f87c2106700 1 -- 172.21.15.99:0/2488382271 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87bc0b1920 con 0x7f87bc06d4c0 2023-10-08T00:11:06.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.262+0000 7fb6ec84c700 1 -- 172.21.15.99:0/4068859746 shutdown_connections 2023-10-08T00:11:06.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.259+0000 7f2d91b4f700 1 -- 172.21.15.99:0/3125051260 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2d8c0cc450 msgr2=0x7f2d8c0c6eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:06.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.259+0000 7f2d91b4f700 1 --2- 172.21.15.99:0/3125051260 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2d8c0cc450 0x7f2d8c0c6eb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2d84009e90 tx=0x7f2d84009780 comp rx=0 tx=0).stop 2023-10-08T00:11:06.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.259+0000 7f2d91b4f700 1 -- 172.21.15.99:0/3125051260 shutdown_connections 2023-10-08T00:11:06.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.259+0000 7f2d91b4f700 1 --2- 172.21.15.99:0/3125051260 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2d8c0cc450 0x7f2d8c0c6eb0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.259+0000 7f2d91b4f700 1 -- 172.21.15.99:0/3125051260 >> 172.21.15.99:0/3125051260 conn(0x7f2d8c070700 msgr2=0x7f2d8c070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:06.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.260+0000 7f2d91b4f700 1 -- 172.21.15.99:0/3125051260 shutdown_connections 2023-10-08T00:11:06.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.260+0000 7f2d91b4f700 1 -- 172.21.15.99:0/3125051260 wait complete. 2023-10-08T00:11:06.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.261+0000 7f2d91b4f700 1 Processor -- start 2023-10-08T00:11:06.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.261+0000 7f2d91b4f700 1 -- start start 2023-10-08T00:11:06.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.262+0000 7f2d91b4f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d8c0c73f0 0x7f2d8c154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.262+0000 7f2d91b4f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d8c0cc450 0x7f2d8c155070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.262+0000 7f2d91b4f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2d8c156270 0x7f2d8c0c2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.262+0000 7f2d91b4f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c0c0d30 con 0x7f2d8c0c73f0 2023-10-08T00:11:06.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.262+0000 7f2d91b4f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c13e5a0 con 0x7f2d8c0cc450 2023-10-08T00:11:06.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.262+0000 7f2d91b4f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c0c46e0 con 0x7f2d8c156270 2023-10-08T00:11:06.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.263+0000 7f2d8bfff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2d8c156270 0x7f2d8c0c2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:06.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.263+0000 7fb6ec84c700 1 -- 172.21.15.99:0/4068859746 wait complete. 2023-10-08T00:11:06.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.264+0000 7fb6ec84c700 1 Processor -- start 2023-10-08T00:11:06.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.264+0000 7fb6ec84c700 1 -- start start 2023-10-08T00:11:06.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.264+0000 7f87a37fe700 1 -- 172.21.15.99:0/2488382271 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f87b000c460 con 0x7f87bc06d4c0 2023-10-08T00:11:06.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.265+0000 7f87a37fe700 1 --2- 172.21.15.99:0/2488382271 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f87a8066fb0 0x7f87a8069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.265+0000 7f87a37fe700 1 -- 172.21.15.99:0/2488382271 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f87b0084e80 con 0x7f87bc06d4c0 2023-10-08T00:11:06.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.265+0000 7f87baffd700 1 --2- 172.21.15.99:0/2488382271 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f87a8066fb0 0x7f87a8069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:06.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.265+0000 7fb6ec84c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6e806d2e0 0x7fb6e80c56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.265+0000 7fb6ec84c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb6e806d6b0 0x7fb6e8177780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.265+0000 7fb6ec84c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb6e8155b60 0x7fb6e8155f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.265+0000 7fb6ec84c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e81712c0 con 0x7fb6e806d2e0 2023-10-08T00:11:06.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.265+0000 7fb6ec84c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e81574a0 con 0x7fb6e8155b60 2023-10-08T00:11:06.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.265+0000 7fb6ec84c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e8157780 con 0x7fb6e806d6b0 2023-10-08T00:11:06.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.265+0000 7f2d8b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d8c0c73f0 0x7f2d8c154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:06.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.265+0000 7f2d8b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d8c0c73f0 0x7f2d8c154b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38868/0 (socket says 172.21.15.99:38868) 2023-10-08T00:11:06.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.265+0000 7f2d8b7fe700 1 -- 172.21.15.99:0/1454026629 learned_addr learned my addr 172.21.15.99:0/1454026629 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:06.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.265+0000 7f2d8b7fe700 1 -- 172.21.15.99:0/1454026629 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2d8c156270 msgr2=0x7f2d8c0c2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:06.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.265+0000 7f2d8b7fe700 1 --2- 172.21.15.99:0/1454026629 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2d8c156270 0x7f2d8c0c2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.265+0000 7f2d8b7fe700 1 -- 172.21.15.99:0/1454026629 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d8c0cc450 msgr2=0x7f2d8c155070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:06.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.265+0000 7f2d8b7fe700 1 --2- 172.21.15.99:0/1454026629 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d8c0cc450 0x7f2d8c155070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.265+0000 7f2d8b7fe700 1 -- 172.21.15.99:0/1454026629 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d84009a20 con 0x7f2d8c0c73f0 2023-10-08T00:11:06.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.265+0000 7f2d8b7fe700 1 --2- 172.21.15.99:0/1454026629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d8c0c73f0 0x7f2d8c154b30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2d840097c0 tx=0x7f2d84018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:06.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.266+0000 7fb6e659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6e806d2e0 0x7fb6e80c56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:06.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.266+0000 7fb6e659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6e806d2e0 0x7fb6e80c56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38872/0 (socket says 172.21.15.99:38872) 2023-10-08T00:11:06.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.266+0000 7fb6e659c700 1 -- 172.21.15.99:0/2214211372 learned_addr learned my addr 172.21.15.99:0/2214211372 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:06.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.266+0000 7fb6e6d9d700 1 --2- 172.21.15.99:0/2214211372 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb6e8155b60 0x7fb6e8155f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:06.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.263+0000 7fe86cdb2700 1 Processor -- start 2023-10-08T00:11:06.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.263+0000 7fe86cdb2700 1 -- start start 2023-10-08T00:11:06.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.264+0000 7fe86cdb2700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fe8680696d0 0x7fe8680c9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.264+0000 7fe86cdb2700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe86815ea50 con 0x7fe86806d3a0 2023-10-08T00:11:06.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.264+0000 7fe86cdb2700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8680c87c0 con 0x7fe8680bfcc0 2023-10-08T00:11:06.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.264+0000 7fe86cdb2700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8680c85d0 con 0x7fe8680696d0 2023-10-08T00:11:06.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.264+0000 7fe86659c700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7fe8680bfcc0 0x7fe868068fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:56132/0 (socket says 172.21.15.99:56132) 2023-10-08T00:11:06.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.264+0000 7fe86659c700 1 -- 172.21.15.99:0/1583092313 learned_addr learned my addr 172.21.15.99:0/1583092313 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:06.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.266+0000 7fe86559a700 1 -- 172.21.15.99:0/1583092313 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4267828055 0 0) 0x7fe8680c87c0 con 0x7fe8680bfcc0 2023-10-08T00:11:06.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.266+0000 7fe86559a700 1 -- 172.21.15.99:0/1583092313 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe850003570 con 0x7fe8680bfcc0 2023-10-08T00:11:06.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.266+0000 7fb6e5d9b700 1 --2- 172.21.15.99:0/2214211372 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb6e806d6b0 0x7fb6e8177780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:06.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.266+0000 7fb6e659c700 1 -- 172.21.15.99:0/2214211372 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb6e806d6b0 msgr2=0x7fb6e8177780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:06.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.266+0000 7fb6e659c700 1 --2- 172.21.15.99:0/2214211372 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb6e806d6b0 0x7fb6e8177780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.266+0000 7fb6e659c700 1 -- 172.21.15.99:0/2214211372 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb6e8155b60 msgr2=0x7fb6e8155f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:06.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.266+0000 7fb6e659c700 1 --2- 172.21.15.99:0/2214211372 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb6e8155b60 0x7fb6e8155f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.266+0000 7fb6e659c700 1 -- 172.21.15.99:0/2214211372 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6dc007040 con 0x7fb6e806d2e0 2023-10-08T00:11:06.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.266+0000 7fe866d9d700 1 --2- 172.21.15.99:0/1583092313 >> v2:172.21.15.191:3300/0 conn(0x7fe8680696d0 0x7fe8680c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:06.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.266+0000 7f87baffd700 1 --2- 172.21.15.99:0/2488382271 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f87a8066fb0 0x7f87a8069460 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f87bc0cb5f0 tx=0x7f87ac009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:06.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.266+0000 7f64f6ebe700 1 -- 172.21.15.99:0/1637295102 shutdown_connections 2023-10-08T00:11:06.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.266+0000 7f64f6ebe700 1 --2- 172.21.15.99:0/1637295102 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64f00bfcc0 0x7f64f00c9860 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.266+0000 7f64f6ebe700 1 -- 172.21.15.99:0/1637295102 >> 172.21.15.99:0/1637295102 conn(0x7f64f0067660 msgr2=0x7f64f0067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:06.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.266+0000 7fb6e659c700 1 --2- 172.21.15.99:0/2214211372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6e806d2e0 0x7fb6e80c56a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb6e000bbb0 tx=0x7fb6e0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:06.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.266+0000 7f64f6ebe700 1 -- 172.21.15.99:0/1637295102 shutdown_connections 2023-10-08T00:11:06.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.266+0000 7f64f6ebe700 1 -- 172.21.15.99:0/1637295102 wait complete. 2023-10-08T00:11:06.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.266+0000 7fe866d9d700 1 -- 172.21.15.99:0/1583092313 >> v1:172.21.15.191:6789/0 conn(0x7fe8680bfcc0 legacy=0x7fe868068fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:06.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.266+0000 7fe866d9d700 1 -- 172.21.15.99:0/1583092313 >> v1:172.21.15.99:6789/0 conn(0x7fe86806d3a0 legacy=0x7fe8680c7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:06.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.266+0000 7fe866d9d700 1 -- 172.21.15.99:0/1583092313 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8680c8aa0 con 0x7fe8680696d0 2023-10-08T00:11:06.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.267+0000 7fe866d9d700 1 --2- 172.21.15.99:0/1583092313 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8680696d0 0x7fe8680c9860 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe860009e60 tx=0x7fe860009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a8ea2c2aa873f73 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:06.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.267+0000 7fe86559a700 1 -- 172.21.15.99:0/1583092313 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86000e070 con 0x7fe8680696d0 2023-10-08T00:11:06.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.267+0000 7fe86559a700 1 -- 172.21.15.99:0/1583092313 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe86000bb60 con 0x7fe8680696d0 2023-10-08T00:11:06.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.267+0000 7fe86559a700 1 -- 172.21.15.99:0/1583092313 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe860012430 con 0x7fe8680696d0 2023-10-08T00:11:06.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.268+0000 7f64f6ebe700 1 Processor -- start 2023-10-08T00:11:06.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.268+0000 7fe86cdb2700 1 -- 172.21.15.99:0/1583092313 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8680696d0 msgr2=0x7fe8680c9860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:06.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.268+0000 7fe86cdb2700 1 --2- 172.21.15.99:0/1583092313 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8680696d0 0x7fe8680c9860 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe860009e60 tx=0x7fe860009780 comp rx=0 tx=0).stop 2023-10-08T00:11:06.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.268+0000 7f64f6ebe700 1 -- start start 2023-10-08T00:11:06.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.269+0000 7f64f6ebe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f64f00696d0 0x7f64f00c0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.269+0000 7f64f6ebe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64f006d3a0 0x7f64f0172320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.269+0000 7fe86cdb2700 1 -- 172.21.15.99:0/1583092313 shutdown_connections 2023-10-08T00:11:06.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.269+0000 7fe86cdb2700 1 --2- 172.21.15.99:0/1583092313 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8680696d0 0x7fe8680c9860 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.269+0000 7fe86cdb2700 1 -- 172.21.15.99:0/1583092313 >> 172.21.15.99:0/1583092313 conn(0x7fe868067660 msgr2=0x7fe868067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:06.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.269+0000 7f64f6ebe700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64f00bfcc0 0x7f64f00bcda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.269+0000 7f64f6ebe700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f0172860 con 0x7f64f006d3a0 2023-10-08T00:11:06.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.269+0000 7f64f6ebe700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f00bd2e0 con 0x7f64f00bfcc0 2023-10-08T00:11:06.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.269+0000 7f64f6ebe700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64f00bd580 con 0x7f64f00696d0 2023-10-08T00:11:06.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.269+0000 7fe86cdb2700 1 -- 172.21.15.99:0/1583092313 shutdown_connections 2023-10-08T00:11:06.294 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.269+0000 7f64f4c5a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f64f00696d0 0x7f64f00c0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:06.294 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.269+0000 7f64f4c5a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f64f00696d0 0x7f64f00c0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.99:45802/0 (socket says 172.21.15.99:45802) 2023-10-08T00:11:06.294 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.269+0000 7f64f4c5a700 1 -- 172.21.15.99:0/1735690408 learned_addr learned my addr 172.21.15.99:0/1735690408 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:06.294 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.269+0000 7f64effff700 1 --2- 172.21.15.99:0/1735690408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64f006d3a0 0x7f64f0172320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:06.294 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.269+0000 7f2d737fe700 1 -- 172.21.15.99:0/1454026629 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d84001090 con 0x7f2d8c0c73f0 2023-10-08T00:11:06.295 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.269+0000 7f2d737fe700 1 -- 172.21.15.99:0/1454026629 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d84027470 con 0x7f2d8c0c73f0 2023-10-08T00:11:06.295 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.269+0000 7f2d737fe700 1 -- 172.21.15.99:0/1454026629 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d840277f0 con 0x7f2d8c0c73f0 2023-10-08T00:11:06.295 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.269+0000 7f87a37fe700 1 -- 172.21.15.99:0/2488382271 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f87b00520d0 con 0x7f87bc06d4c0 2023-10-08T00:11:06.295 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.269+0000 7f64f4c5a700 1 -- 172.21.15.99:0/1735690408 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64f00bfcc0 msgr2=0x7f64f00bcda0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T00:11:06.295 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.269+0000 7f64f4c5a700 1 --2- 172.21.15.99:0/1735690408 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64f00bfcc0 0x7f64f00bcda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.295 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.269+0000 7f64f4c5a700 1 -- 172.21.15.99:0/1735690408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64f006d3a0 msgr2=0x7f64f0172320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:06.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.269+0000 7f64f4c5a700 1 --2- 172.21.15.99:0/1735690408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64f006d3a0 0x7f64f0172320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.269+0000 7f64f4c5a700 1 -- 172.21.15.99:0/1735690408 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e40094f0 con 0x7f64f00696d0 2023-10-08T00:11:06.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.270+0000 7fb6d67fc700 1 -- 172.21.15.99:0/2214211372 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e00092e0 con 0x7fb6e806d2e0 2023-10-08T00:11:06.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.270+0000 7fb6d67fc700 1 -- 172.21.15.99:0/2214211372 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6e000c7a0 con 0x7fb6e806d2e0 2023-10-08T00:11:06.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.270+0000 7fb6d67fc700 1 -- 172.21.15.99:0/2214211372 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e000cb20 con 0x7fb6e806d2e0 2023-10-08T00:11:06.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.269+0000 7f2d91b4f700 1 -- 172.21.15.99:0/1454026629 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d8c13df50 con 0x7f2d8c0c73f0 2023-10-08T00:11:06.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.269+0000 7f2d91b4f700 1 -- 172.21.15.99:0/1454026629 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d8c0c1300 con 0x7f2d8c0c73f0 2023-10-08T00:11:06.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.270+0000 7fb6ec84c700 1 -- 172.21.15.99:0/2214211372 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6e8157a60 con 0x7fb6e806d2e0 2023-10-08T00:11:06.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.270+0000 7fb6ec84c700 1 -- 172.21.15.99:0/2214211372 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6e8158020 con 0x7fb6e806d2e0 2023-10-08T00:11:06.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.270+0000 7f64f4c5a700 1 --2- 172.21.15.99:0/1735690408 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f64f00696d0 0x7f64f00c0810 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f64e4009290 tx=0x7f64e4016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:06.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.269+0000 7fe86cdb2700 1 -- 172.21.15.99:0/1583092313 wait complete. 2023-10-08T00:11:06.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.271+0000 7fe86cdb2700 1 Processor -- start 2023-10-08T00:11:06.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.271+0000 7fe86cdb2700 1 -- start start 2023-10-08T00:11:06.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.271+0000 7fb6ec84c700 1 -- 172.21.15.99:0/2214211372 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6cc0052b0 con 0x7fb6e806d2e0 2023-10-08T00:11:06.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.271+0000 7fe86cdb2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8680696d0 0x7fe868157570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.271+0000 7fe86cdb2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe86806d3a0 0x7fe868157ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.271+0000 7fe86cdb2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8680bfcc0 0x7fe86815b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.271+0000 7fe86cdb2700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8680c33b0 con 0x7fe8680bfcc0 2023-10-08T00:11:06.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.271+0000 7fe86cdb2700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe86815ce80 con 0x7fe86806d3a0 2023-10-08T00:11:06.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.271+0000 7fe86cdb2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8680c30d0 con 0x7fe8680696d0 2023-10-08T00:11:06.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.272+0000 7fe86659c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8680696d0 0x7fe868157570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:06.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.272+0000 7fe86659c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8680696d0 0x7fe868157570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.99:45804/0 (socket says 172.21.15.99:45804) 2023-10-08T00:11:06.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.272+0000 7fe86659c700 1 -- 172.21.15.99:0/3544144483 learned_addr learned my addr 172.21.15.99:0/3544144483 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:06.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.272+0000 7fe866d9d700 1 --2- 172.21.15.99:0/3544144483 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8680bfcc0 0x7fe86815b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:06.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.272+0000 7fe86659c700 1 -- 172.21.15.99:0/3544144483 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe86806d3a0 msgr2=0x7fe868157ab0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T00:11:06.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.272+0000 7fe86659c700 1 --2- 172.21.15.99:0/3544144483 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe86806d3a0 0x7fe868157ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.272+0000 7fe86659c700 1 -- 172.21.15.99:0/3544144483 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8680bfcc0 msgr2=0x7fe86815b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:06.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.272+0000 7fe86659c700 1 --2- 172.21.15.99:0/3544144483 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8680bfcc0 0x7fe86815b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.272+0000 7fe86659c700 1 -- 172.21.15.99:0/3544144483 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe860009a20 con 0x7fe8680696d0 2023-10-08T00:11:06.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.272+0000 7fe86659c700 1 --2- 172.21.15.99:0/3544144483 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8680696d0 0x7fe868157570 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe85800c7f0 tx=0x7fe858007e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:06.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.272+0000 7fe8567fc700 1 -- 172.21.15.99:0/3544144483 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe858005b30 con 0x7fe8680696d0 2023-10-08T00:11:06.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.272+0000 7fe8567fc700 1 -- 172.21.15.99:0/3544144483 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe858005cb0 con 0x7fe8680696d0 2023-10-08T00:11:06.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.272+0000 7fe86cdb2700 1 -- 172.21.15.99:0/3544144483 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe86815c830 con 0x7fe8680696d0 2023-10-08T00:11:06.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.272+0000 7fe8567fc700 1 -- 172.21.15.99:0/3544144483 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe858007320 con 0x7fe8680696d0 2023-10-08T00:11:06.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.272+0000 7fe86cdb2700 1 -- 172.21.15.99:0/3544144483 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8680c2ba0 con 0x7fe8680696d0 2023-10-08T00:11:06.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.273+0000 7f2d737fe700 1 -- 172.21.15.99:0/1454026629 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2d84024070 con 0x7f2d8c0c73f0 2023-10-08T00:11:06.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.273+0000 7f2d737fe700 1 --2- 172.21.15.99:0/1454026629 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2d78067000 0x7f2d780694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.273+0000 7f2d737fe700 1 -- 172.21.15.99:0/1454026629 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2d84086e90 con 0x7f2d8c0c73f0 2023-10-08T00:11:06.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.274+0000 7f2d8affd700 1 --2- 172.21.15.99:0/1454026629 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2d78067000 0x7f2d780694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:06.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.274+0000 7f2d8affd700 1 --2- 172.21.15.99:0/1454026629 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2d78067000 0x7f2d780694b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c007fd0 tx=0x7f2d7c007b90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:06.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.275+0000 7fe8567fc700 1 -- 172.21.15.99:0/3544144483 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe85801b3b0 con 0x7fe8680696d0 2023-10-08T00:11:06.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.275+0000 7fe8567fc700 1 --2- 172.21.15.99:0/3544144483 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe850067340 0x7fe8500697f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.275+0000 7fe8567fc700 1 -- 172.21.15.99:0/3544144483 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe858083b10 con 0x7fe8680696d0 2023-10-08T00:11:06.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.275+0000 7fb6d67fc700 1 -- 172.21.15.99:0/2214211372 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb6e0012070 con 0x7fb6e806d2e0 2023-10-08T00:11:06.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.275+0000 7fb6d67fc700 1 --2- 172.21.15.99:0/2214211372 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb6c8066f30 0x7fb6c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.275+0000 7fb6d67fc700 1 -- 172.21.15.99:0/2214211372 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb6e0084610 con 0x7fb6e806d2e0 2023-10-08T00:11:06.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.275+0000 7fb6e5d9b700 1 --2- 172.21.15.99:0/2214211372 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb6c8066f30 0x7fb6c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:06.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.276+0000 7fb6e5d9b700 1 --2- 172.21.15.99:0/2214211372 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb6c8066f30 0x7fb6c80693e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb6e80af480 tx=0x7fb6d8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:06.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.277+0000 7f64ecff9700 1 -- 172.21.15.99:0/1735690408 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e4011ca0 con 0x7f64f00696d0 2023-10-08T00:11:06.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.277+0000 7f64ecff9700 1 -- 172.21.15.99:0/1735690408 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e4020070 con 0x7f64f00696d0 2023-10-08T00:11:06.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.277+0000 7f64ecff9700 1 -- 172.21.15.99:0/1735690408 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e401b6a0 con 0x7f64f00696d0 2023-10-08T00:11:06.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.277+0000 7f64f6ebe700 1 -- 172.21.15.99:0/1735690408 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64f00bd860 con 0x7f64f00696d0 2023-10-08T00:11:06.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.277+0000 7f64f6ebe700 1 -- 172.21.15.99:0/1735690408 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64f00bde20 con 0x7f64f00696d0 2023-10-08T00:11:06.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.277+0000 7fe86cdb2700 1 -- 172.21.15.99:0/3544144483 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8680c2e80 con 0x7fe8680696d0 2023-10-08T00:11:06.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.277+0000 7f2d91b4f700 1 -- 172.21.15.99:0/1454026629 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d6c0052b0 con 0x7f2d8c0c73f0 2023-10-08T00:11:06.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.278+0000 7fe865d9b700 1 --2- 172.21.15.99:0/3544144483 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe850067340 0x7fe8500697f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:06.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.278+0000 7f64f6ebe700 1 -- 172.21.15.99:0/1735690408 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64f004c3c0 con 0x7f64f00696d0 2023-10-08T00:11:06.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.278+0000 7fe865d9b700 1 --2- 172.21.15.99:0/3544144483 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe850067340 0x7fe8500697f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe85c001050 tx=0x7fe85c007ae0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:06.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.278+0000 7f176f59e700 1 Processor -- start 2023-10-08T00:11:06.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.278+0000 7f176f59e700 1 -- start start 2023-10-08T00:11:06.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.280+0000 7f176f59e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f17700aed30 0x7f17700c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.280+0000 7f176f59e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f17700682c0 0x7f17700c8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.280+0000 7f176f59e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17700c7ef0 con 0x7f17700c88c0 2023-10-08T00:11:06.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.280+0000 7f176f59e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1770156560 con 0x7f17700682c0 2023-10-08T00:11:06.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.280+0000 7f176f59e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17700c72e0 con 0x7f17700aed30 2023-10-08T00:11:06.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.280+0000 7f176e59c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f17700aed30 0x7f17700c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:06.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.280+0000 7f176ed9d700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f17700c88c0 0x7f17700af5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:56148/0 (socket says 172.21.15.99:56148) 2023-10-08T00:11:06.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.280+0000 7f176e59c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f17700aed30 0x7f17700c6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.99:45810/0 (socket says 172.21.15.99:45810) 2023-10-08T00:11:06.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.280+0000 7f176ed9d700 1 -- 172.21.15.99:0/3331121127 learned_addr learned my addr 172.21.15.99:0/3331121127 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:06.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.280+0000 7f176dd9b700 1 --2- 172.21.15.99:0/3331121127 >> v2:172.21.15.191:3300/0 conn(0x7f17700682c0 0x7f17700c8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:06.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.280+0000 7f176e59c700 1 -- 172.21.15.99:0/3331121127 >> v2:172.21.15.191:3300/0 conn(0x7f17700682c0 msgr2=0x7f17700c8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:06.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.280+0000 7f176e59c700 1 --2- 172.21.15.99:0/3331121127 >> v2:172.21.15.191:3300/0 conn(0x7f17700682c0 0x7f17700c8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.280+0000 7f176e59c700 1 -- 172.21.15.99:0/3331121127 >> v1:172.21.15.191:6789/0 conn(0x7f17700c88c0 legacy=0x7f17700af5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:06.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.280+0000 7f176e59c700 1 -- 172.21.15.99:0/3331121127 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17700c75c0 con 0x7f17700aed30 2023-10-08T00:11:06.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.281+0000 7f176e59c700 1 --2- 172.21.15.99:0/3331121127 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f17700aed30 0x7f17700c6d00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f17600098e0 tx=0x7f17600091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=522bf1debd7a6259 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:06.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.282+0000 7fe8567fc700 1 -- 172.21.15.99:0/3544144483 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe858017080 con 0x7fe8680696d0 2023-10-08T00:11:06.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.282+0000 7fb6d67fc700 1 -- 172.21.15.99:0/2214211372 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb6e0051900 con 0x7fb6e806d2e0 2023-10-08T00:11:06.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.283+0000 7f64ecff9700 1 -- 172.21.15.99:0/1735690408 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f64e4022030 con 0x7f64f00696d0 2023-10-08T00:11:06.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.284+0000 7f64ecff9700 1 --2- 172.21.15.99:0/1735690408 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f64d4067000 0x7f64d40694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.313 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.284+0000 7f64ecff9700 1 -- 172.21.15.99:0/1735690408 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f64e4086640 con 0x7f64f00696d0 2023-10-08T00:11:06.313 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.284+0000 7f64effff700 1 --2- 172.21.15.99:0/1735690408 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f64d4067000 0x7f64d40694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:06.313 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.284+0000 7f64effff700 1 --2- 172.21.15.99:0/1735690408 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f64d4067000 0x7f64d40694b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f64dc007fd0 tx=0x7f64dc007d30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:06.313 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.286+0000 7f64ecff9700 1 -- 172.21.15.99:0/1735690408 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f64e4053a10 con 0x7f64f00696d0 2023-10-08T00:11:06.313 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.287+0000 7f176d59a700 1 -- 172.21.15.99:0/3331121127 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176000b640 con 0x7f17700aed30 2023-10-08T00:11:06.313 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.287+0000 7f176d59a700 1 -- 172.21.15.99:0/3331121127 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f176000b7a0 con 0x7f17700aed30 2023-10-08T00:11:06.313 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.287+0000 7f176d59a700 1 -- 172.21.15.99:0/3331121127 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176000bb40 con 0x7f17700aed30 2023-10-08T00:11:06.314 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.291+0000 7f2d737fe700 1 -- 172.21.15.99:0/1454026629 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2d840540e0 con 0x7f2d8c0c73f0 2023-10-08T00:11:06.314 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.292+0000 7f176f59e700 1 -- 172.21.15.99:0/3331121127 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f17700aed30 msgr2=0x7f17700c6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:06.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.292+0000 7f176f59e700 1 --2- 172.21.15.99:0/3331121127 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f17700aed30 0x7f17700c6d00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f17600098e0 tx=0x7f17600091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.303+0000 7f176f59e700 1 -- 172.21.15.99:0/3331121127 shutdown_connections 2023-10-08T00:11:06.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.303+0000 7f176f59e700 1 --2- 172.21.15.99:0/3331121127 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f17700aed30 0x7f17700c6d00 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.303+0000 7f176f59e700 1 --2- 172.21.15.99:0/3331121127 >> v2:172.21.15.191:3300/0 conn(0x7f17700682c0 0x7f17700c8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.303+0000 7f176f59e700 1 -- 172.21.15.99:0/3331121127 >> 172.21.15.99:0/3331121127 conn(0x7f17700cc760 msgr2=0x7f17700ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:06.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.312+0000 7f176f59e700 1 -- 172.21.15.99:0/3331121127 shutdown_connections 2023-10-08T00:11:06.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.313+0000 7f176f59e700 1 -- 172.21.15.99:0/3331121127 wait complete. 2023-10-08T00:11:06.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.314+0000 7f176f59e700 1 Processor -- start 2023-10-08T00:11:06.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.315+0000 7f176f59e700 1 -- start start 2023-10-08T00:11:06.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.316+0000 7f176f59e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f17700682c0 0x7f177015aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.316+0000 7f176f59e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17700aed30 0x7f177015b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.316+0000 7f176f59e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f17700c88c0 0x7f17700c17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.316+0000 7f176f59e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17700c1320 con 0x7f17700aed30 2023-10-08T00:11:06.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.316+0000 7f176f59e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17700c0750 con 0x7f17700c88c0 2023-10-08T00:11:06.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.316+0000 7f176f59e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17700c09f0 con 0x7f17700682c0 2023-10-08T00:11:06.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.316+0000 7f176e59c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f17700682c0 0x7f177015aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:06.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.316+0000 7f176e59c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f17700682c0 0x7f177015aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.99:45822/0 (socket says 172.21.15.99:45822) 2023-10-08T00:11:06.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.316+0000 7f176e59c700 1 -- 172.21.15.99:0/222130063 learned_addr learned my addr 172.21.15.99:0/222130063 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:06.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.316+0000 7f176ed9d700 1 --2- 172.21.15.99:0/222130063 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f17700c88c0 0x7f17700c17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:06.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.317+0000 7f176ed9d700 1 -- 172.21.15.99:0/222130063 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f17700682c0 msgr2=0x7f177015aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:06.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.317+0000 7f176ed9d700 1 --2- 172.21.15.99:0/222130063 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f17700682c0 0x7f177015aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.317+0000 7f176ed9d700 1 -- 172.21.15.99:0/222130063 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17700aed30 msgr2=0x7f177015b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:06.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.317+0000 7f176ed9d700 1 --2- 172.21.15.99:0/222130063 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17700aed30 0x7f177015b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.317+0000 7f176ed9d700 1 -- 172.21.15.99:0/222130063 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1760009420 con 0x7f17700c88c0 2023-10-08T00:11:06.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.317+0000 7f176ed9d700 1 --2- 172.21.15.99:0/222130063 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f17700c88c0 0x7f17700c17b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f176800cb40 tx=0x7f1768001900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:06.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.317+0000 7f175e7fc700 1 -- 172.21.15.99:0/222130063 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176800a8b0 con 0x7f17700c88c0 2023-10-08T00:11:06.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.317+0000 7f175e7fc700 1 -- 172.21.15.99:0/222130063 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f176800aa10 con 0x7f17700c88c0 2023-10-08T00:11:06.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.317+0000 7f175e7fc700 1 -- 172.21.15.99:0/222130063 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176800adb0 con 0x7f17700c88c0 2023-10-08T00:11:06.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.317+0000 7f176f59e700 1 -- 172.21.15.99:0/222130063 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17700c0cd0 con 0x7f17700c88c0 2023-10-08T00:11:06.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.317+0000 7f176f59e700 1 -- 172.21.15.99:0/222130063 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f177015b420 con 0x7f17700c88c0 2023-10-08T00:11:06.321 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.321+0000 7f176f59e700 1 -- 172.21.15.99:0/222130063 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17540052b0 con 0x7f17700c88c0 2023-10-08T00:11:06.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.325+0000 7f175e7fc700 1 -- 172.21.15.99:0/222130063 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1768007300 con 0x7f17700c88c0 2023-10-08T00:11:06.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.325+0000 7f175e7fc700 1 --2- 172.21.15.99:0/222130063 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1758067210 0x7f17580696c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.325+0000 7f175e7fc700 1 -- 172.21.15.99:0/222130063 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f17680847e0 con 0x7f17700c88c0 2023-10-08T00:11:06.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.326+0000 7f176e59c700 1 --2- 172.21.15.99:0/222130063 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1758067210 0x7f17580696c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:06.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.326+0000 7f176e59c700 1 --2- 172.21.15.99:0/222130063 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1758067210 0x7f17580696c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f17600093a0 tx=0x7f1760017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:06.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.330+0000 7f175e7fc700 1 -- 172.21.15.99:0/222130063 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1768051a20 con 0x7f17700c88c0 2023-10-08T00:11:06.577 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.576+0000 7ff668afb700 1 -- 172.21.15.99:0/1875536605 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff644005860 con 0x7ff6640c73f0 2023-10-08T00:11:06.578 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.577+0000 7ff6527fc700 1 -- 172.21.15.99:0/1875536605 <== mon.2 v2:172.21.15.191:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff658021090 con 0x7ff6640c73f0 2023-10-08T00:11:06.581 INFO:teuthology.orchestra.run.smithi099.stdout:55834574854 2023-10-08T00:11:06.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.581+0000 7ff668afb700 1 -- 172.21.15.99:0/1875536605 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff64c0671e0 msgr2=0x7ff64c069690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:06.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.581+0000 7ff668afb700 1 --2- 172.21.15.99:0/1875536605 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff64c0671e0 0x7ff64c069690 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff6640aee50 tx=0x7ff65c017000 comp rx=0 tx=0).stop 2023-10-08T00:11:06.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.581+0000 7ff668afb700 1 -- 172.21.15.99:0/1875536605 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6640c73f0 msgr2=0x7ff6640c5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:06.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.581+0000 7ff668afb700 1 --2- 172.21.15.99:0/1875536605 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6640c73f0 0x7ff6640c5f40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff65800ec80 tx=0x7ff6580091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.581+0000 7ff668afb700 1 -- 172.21.15.99:0/1875536605 shutdown_connections 2023-10-08T00:11:06.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.581+0000 7ff668afb700 1 --2- 172.21.15.99:0/1875536605 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff64c0671e0 0x7ff64c069690 secure :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff6640aee50 tx=0x7ff65c017000 comp rx=0 tx=0).stop 2023-10-08T00:11:06.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.581+0000 7ff668afb700 1 --2- 172.21.15.99:0/1875536605 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6641560a0 0x7ff66414cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.581+0000 7ff668afb700 1 --2- 172.21.15.99:0/1875536605 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6640c73f0 0x7ff6640c5f40 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.581+0000 7ff668afb700 1 --2- 172.21.15.99:0/1875536605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6640cc450 0x7ff66414c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.581+0000 7ff668afb700 1 -- 172.21.15.99:0/1875536605 >> 172.21.15.99:0/1875536605 conn(0x7ff664070700 msgr2=0x7ff6640c8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:06.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.581+0000 7ff668afb700 1 -- 172.21.15.99:0/1875536605 shutdown_connections 2023-10-08T00:11:06.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.581+0000 7ff668afb700 1 -- 172.21.15.99:0/1875536605 wait complete. 2023-10-08T00:11:06.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.592+0000 7f4bb4763700 1 -- 172.21.15.99:0/1006454474 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4bac0ada10 con 0x7f4bac06d6b0 2023-10-08T00:11:06.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.593+0000 7f4ba27fc700 1 -- 172.21.15.99:0/1006454474 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4bac0ada10 con 0x7f4bac06d6b0 2023-10-08T00:11:06.596 INFO:teuthology.orchestra.run.smithi099.stdout:55834574854 2023-10-08T00:11:06.598 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.597+0000 7f4bb4763700 1 -- 172.21.15.99:0/1006454474 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4b98067510 msgr2=0x7f4b980699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:06.598 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.597+0000 7f4bb4763700 1 --2- 172.21.15.99:0/1006454474 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4b98067510 0x7f4b980699c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8008fd0 tx=0x7f4ba8005cc0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.598 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.597+0000 7f4bb4763700 1 -- 172.21.15.99:0/1006454474 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4bac06d6b0 msgr2=0x7f4bac177cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:06.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.597+0000 7f4bb4763700 1 --2- 172.21.15.99:0/1006454474 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4bac06d6b0 0x7f4bac177cf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4016000 tx=0x7f4ba4019000 comp rx=0 tx=0).stop 2023-10-08T00:11:06.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.597+0000 7f4bb4763700 1 -- 172.21.15.99:0/1006454474 shutdown_connections 2023-10-08T00:11:06.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.597+0000 7f4bb4763700 1 --2- 172.21.15.99:0/1006454474 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4b98067510 0x7f4b980699c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.597+0000 7f4bb4763700 1 --2- 172.21.15.99:0/1006454474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4bac0c75c0 0x7f4bac0ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.597+0000 7f4bb4763700 1 --2- 172.21.15.99:0/1006454474 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4bac06d2e0 0x7f4bac0ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.597+0000 7f4bb4763700 1 --2- 172.21.15.99:0/1006454474 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4bac06d6b0 0x7f4bac177cf0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.597+0000 7f4bb4763700 1 -- 172.21.15.99:0/1006454474 >> 172.21.15.99:0/1006454474 conn(0x7f4bac070700 msgr2=0x7f4bac158f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:06.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.597+0000 7f4bb4763700 1 -- 172.21.15.99:0/1006454474 shutdown_connections 2023-10-08T00:11:06.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.597+0000 7f4bb4763700 1 -- 172.21.15.99:0/1006454474 wait complete. 2023-10-08T00:11:06.602 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2023-10-08T00:11:06.602 DEBUG:teuthology.parallel:result is None 2023-10-08T00:11:06.615 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2023-10-08T00:11:06.616 DEBUG:teuthology.parallel:result is None 2023-10-08T00:11:06.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.642+0000 7f64f6ebe700 1 -- 172.21.15.99:0/1735690408 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f64f004c3c0 con 0x7f64f00696d0 2023-10-08T00:11:06.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.643+0000 7f64ecff9700 1 -- 172.21.15.99:0/1735690408 <== mon.2 v2:172.21.15.191:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f64e4057030 con 0x7f64f00696d0 2023-10-08T00:11:06.644 INFO:teuthology.orchestra.run.smithi099.stdout:55834574854 2023-10-08T00:11:06.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.645+0000 7f64f6ebe700 1 -- 172.21.15.99:0/1735690408 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f64d4067000 msgr2=0x7f64d40694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:06.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.645+0000 7f64f6ebe700 1 --2- 172.21.15.99:0/1735690408 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f64d4067000 0x7f64d40694b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f64dc007fd0 tx=0x7f64dc007d30 comp rx=0 tx=0).stop 2023-10-08T00:11:06.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.645+0000 7f64f6ebe700 1 -- 172.21.15.99:0/1735690408 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f64f00696d0 msgr2=0x7f64f00c0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:06.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.645+0000 7f64f6ebe700 1 --2- 172.21.15.99:0/1735690408 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f64f00696d0 0x7f64f00c0810 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f64e4009290 tx=0x7f64e4016000 comp rx=0 tx=0).stop 2023-10-08T00:11:06.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.646+0000 7f64f6ebe700 1 -- 172.21.15.99:0/1735690408 shutdown_connections 2023-10-08T00:11:06.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.646+0000 7f64f6ebe700 1 --2- 172.21.15.99:0/1735690408 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f64d4067000 0x7f64d40694b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.646+0000 7f64f6ebe700 1 --2- 172.21.15.99:0/1735690408 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64f00bfcc0 0x7f64f00bcda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.646+0000 7f64f6ebe700 1 --2- 172.21.15.99:0/1735690408 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f64f00696d0 0x7f64f00c0810 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.646+0000 7f64f6ebe700 1 --2- 172.21.15.99:0/1735690408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64f006d3a0 0x7f64f0172320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.646+0000 7f64f6ebe700 1 -- 172.21.15.99:0/1735690408 >> 172.21.15.99:0/1735690408 conn(0x7f64f0067660 msgr2=0x7f64f00b1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:06.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.646+0000 7f64f6ebe700 1 -- 172.21.15.99:0/1735690408 shutdown_connections 2023-10-08T00:11:06.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.646+0000 7f64f6ebe700 1 -- 172.21.15.99:0/1735690408 wait complete. 2023-10-08T00:11:06.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.650+0000 7f176f59e700 1 -- 172.21.15.99:0/222130063 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f1754005560 con 0x7f17700c88c0 2023-10-08T00:11:06.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.651+0000 7f175e7fc700 1 -- 172.21.15.99:0/222130063 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1768055040 con 0x7f17700c88c0 2023-10-08T00:11:06.651 INFO:teuthology.orchestra.run.smithi099.stdout:55834574854 2023-10-08T00:11:06.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.653+0000 7f176f59e700 1 -- 172.21.15.99:0/222130063 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1758067210 msgr2=0x7f17580696c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:06.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.653+0000 7f176f59e700 1 --2- 172.21.15.99:0/222130063 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1758067210 0x7f17580696c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f17600093a0 tx=0x7f1760017000 comp rx=0 tx=0).stop 2023-10-08T00:11:06.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.653+0000 7f176f59e700 1 -- 172.21.15.99:0/222130063 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f17700c88c0 msgr2=0x7f17700c17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:06.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.653+0000 7f176f59e700 1 --2- 172.21.15.99:0/222130063 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f17700c88c0 0x7f17700c17b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f176800cb40 tx=0x7f1768001900 comp rx=0 tx=0).stop 2023-10-08T00:11:06.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.653+0000 7f176f59e700 1 -- 172.21.15.99:0/222130063 shutdown_connections 2023-10-08T00:11:06.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.653+0000 7f176f59e700 1 --2- 172.21.15.99:0/222130063 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1758067210 0x7f17580696c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.653+0000 7f176f59e700 1 --2- 172.21.15.99:0/222130063 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f17700c88c0 0x7f17700c17b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.653+0000 7f176f59e700 1 --2- 172.21.15.99:0/222130063 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f17700682c0 0x7f177015aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.653+0000 7f176f59e700 1 --2- 172.21.15.99:0/222130063 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17700aed30 0x7f177015b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.653+0000 7f176f59e700 1 -- 172.21.15.99:0/222130063 >> 172.21.15.99:0/222130063 conn(0x7f17700cc760 msgr2=0x7f17700af5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:06.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.653+0000 7f176f59e700 1 -- 172.21.15.99:0/222130063 shutdown_connections 2023-10-08T00:11:06.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.653+0000 7f176f59e700 1 -- 172.21.15.99:0/222130063 wait complete. 2023-10-08T00:11:06.660 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.2 2023-10-08T00:11:06.660 DEBUG:teuthology.parallel:result is None 2023-10-08T00:11:06.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.661+0000 7f87c2106700 1 -- 172.21.15.99:0/2488382271 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f87bc04c3c0 con 0x7f87bc06d4c0 2023-10-08T00:11:06.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.661+0000 7f87a37fe700 1 -- 172.21.15.99:0/2488382271 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f87b00556f0 con 0x7f87bc06d4c0 2023-10-08T00:11:06.662 INFO:teuthology.orchestra.run.smithi099.stdout:55834574854 2023-10-08T00:11:06.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.663+0000 7fe86cdb2700 1 -- 172.21.15.99:0/3544144483 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fe86804c3c0 con 0x7fe8680696d0 2023-10-08T00:11:06.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.664+0000 7fe8567fc700 1 -- 172.21.15.99:0/3544144483 <== mon.2 v2:172.21.15.191:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe858050d80 con 0x7fe8680696d0 2023-10-08T00:11:06.665 INFO:teuthology.orchestra.run.smithi099.stdout:55834574854 2023-10-08T00:11:06.667 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.665+0000 7f87c2106700 1 -- 172.21.15.99:0/2488382271 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f87a8066fb0 msgr2=0x7f87a8069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:06.667 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.665+0000 7f87c2106700 1 --2- 172.21.15.99:0/2488382271 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f87a8066fb0 0x7f87a8069460 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f87bc0cb5f0 tx=0x7f87ac009040 comp rx=0 tx=0).stop 2023-10-08T00:11:06.667 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.665+0000 7f87c2106700 1 -- 172.21.15.99:0/2488382271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87bc06d4c0 msgr2=0x7f87bc0b1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:06.667 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.665+0000 7f87c2106700 1 --2- 172.21.15.99:0/2488382271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87bc06d4c0 0x7f87bc0b1ad0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f87b000e930 tx=0x7f87b00091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.665+0000 7f87c2106700 1 -- 172.21.15.99:0/2488382271 shutdown_connections 2023-10-08T00:11:06.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.665+0000 7f87c2106700 1 --2- 172.21.15.99:0/2488382271 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f87a8066fb0 0x7f87a8069460 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.665+0000 7f87c2106700 1 --2- 172.21.15.99:0/2488382271 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87bc0bf970 0x7f87bc0c08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.665+0000 7f87c2106700 1 --2- 172.21.15.99:0/2488382271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87bc06d4c0 0x7f87bc0b1ad0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.665+0000 7f87c2106700 1 --2- 172.21.15.99:0/2488382271 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87bc0ac1a0 0x7f87bc0c3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.665+0000 7f87c2106700 1 -- 172.21.15.99:0/2488382271 >> 172.21.15.99:0/2488382271 conn(0x7f87bc067660 msgr2=0x7f87bc06ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:06.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.665+0000 7f87c2106700 1 -- 172.21.15.99:0/2488382271 shutdown_connections 2023-10-08T00:11:06.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.665+0000 7f87c2106700 1 -- 172.21.15.99:0/2488382271 wait complete. 2023-10-08T00:11:06.669 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2023-10-08T00:11:06.670 DEBUG:teuthology.parallel:result is None 2023-10-08T00:11:06.670 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.666+0000 7fe86cdb2700 1 -- 172.21.15.99:0/3544144483 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe850067340 msgr2=0x7fe8500697f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:06.671 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.667+0000 7fe86cdb2700 1 --2- 172.21.15.99:0/3544144483 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe850067340 0x7fe8500697f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe85c001050 tx=0x7fe85c007ae0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.671 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.667+0000 7fe86cdb2700 1 -- 172.21.15.99:0/3544144483 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8680696d0 msgr2=0x7fe868157570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:06.671 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.667+0000 7fe86cdb2700 1 --2- 172.21.15.99:0/3544144483 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8680696d0 0x7fe868157570 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe85800c7f0 tx=0x7fe858007e00 comp rx=0 tx=0).stop 2023-10-08T00:11:06.671 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.667+0000 7fe86cdb2700 1 -- 172.21.15.99:0/3544144483 shutdown_connections 2023-10-08T00:11:06.671 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.667+0000 7fe86cdb2700 1 --2- 172.21.15.99:0/3544144483 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe850067340 0x7fe8500697f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.671 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.667+0000 7fe86cdb2700 1 --2- 172.21.15.99:0/3544144483 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8680bfcc0 0x7fe86815b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.667+0000 7fe86cdb2700 1 --2- 172.21.15.99:0/3544144483 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8680696d0 0x7fe868157570 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.667+0000 7fe86cdb2700 1 --2- 172.21.15.99:0/3544144483 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe86806d3a0 0x7fe868157ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.667+0000 7fe86cdb2700 1 -- 172.21.15.99:0/3544144483 >> 172.21.15.99:0/3544144483 conn(0x7fe868067660 msgr2=0x7fe868160a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:06.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.667+0000 7fe86cdb2700 1 -- 172.21.15.99:0/3544144483 shutdown_connections 2023-10-08T00:11:06.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.667+0000 7fe86cdb2700 1 -- 172.21.15.99:0/3544144483 wait complete. 2023-10-08T00:11:06.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.670+0000 7f2d91b4f700 1 -- 172.21.15.99:0/1454026629 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f2d6c005560 con 0x7f2d8c0c73f0 2023-10-08T00:11:06.673 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.670+0000 7f2d737fe700 1 -- 172.21.15.99:0/1454026629 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2d84057700 con 0x7f2d8c0c73f0 2023-10-08T00:11:06.674 INFO:teuthology.orchestra.run.smithi099.stdout:55834574854 2023-10-08T00:11:06.674 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.671+0000 7fb6ec84c700 1 -- 172.21.15.99:0/2214211372 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fb6cc005830 con 0x7fb6e806d2e0 2023-10-08T00:11:06.674 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.671+0000 7fb6d67fc700 1 -- 172.21.15.99:0/2214211372 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb6e0054f20 con 0x7fb6e806d2e0 2023-10-08T00:11:06.674 INFO:teuthology.orchestra.run.smithi099.stdout:55834574854 2023-10-08T00:11:06.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.673+0000 7f2d91b4f700 1 -- 172.21.15.99:0/1454026629 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2d78067000 msgr2=0x7f2d780694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:06.676 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.673+0000 7f2d91b4f700 1 --2- 172.21.15.99:0/1454026629 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2d78067000 0x7f2d780694b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c007fd0 tx=0x7f2d7c007b90 comp rx=0 tx=0).stop 2023-10-08T00:11:06.676 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.673+0000 7f2d91b4f700 1 -- 172.21.15.99:0/1454026629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d8c0c73f0 msgr2=0x7f2d8c154b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:06.676 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.673+0000 7f2d91b4f700 1 --2- 172.21.15.99:0/1454026629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d8c0c73f0 0x7f2d8c154b30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2d840097c0 tx=0x7f2d84018000 comp rx=0 tx=0).stop 2023-10-08T00:11:06.676 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.673+0000 7f2d91b4f700 1 -- 172.21.15.99:0/1454026629 shutdown_connections 2023-10-08T00:11:06.676 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.673+0000 7f2d91b4f700 1 --2- 172.21.15.99:0/1454026629 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2d78067000 0x7f2d780694b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.677 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.673+0000 7f2d91b4f700 1 --2- 172.21.15.99:0/1454026629 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2d8c156270 0x7f2d8c0c2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.677 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.673+0000 7f2d91b4f700 1 --2- 172.21.15.99:0/1454026629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d8c0c73f0 0x7f2d8c154b30 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.677 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.673+0000 7f2d91b4f700 1 --2- 172.21.15.99:0/1454026629 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d8c0cc450 0x7f2d8c155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.677 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.673+0000 7f2d91b4f700 1 -- 172.21.15.99:0/1454026629 >> 172.21.15.99:0/1454026629 conn(0x7f2d8c070700 msgr2=0x7f2d8c158800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:06.677 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.673+0000 7f2d91b4f700 1 -- 172.21.15.99:0/1454026629 shutdown_connections 2023-10-08T00:11:06.677 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.673+0000 7f2d91b4f700 1 -- 172.21.15.99:0/1454026629 wait complete. 2023-10-08T00:11:06.678 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.674+0000 7fb6ec84c700 1 -- 172.21.15.99:0/2214211372 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb6c8066f30 msgr2=0x7fb6c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:06.678 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.674+0000 7fb6ec84c700 1 --2- 172.21.15.99:0/2214211372 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb6c8066f30 0x7fb6c80693e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb6e80af480 tx=0x7fb6d8008040 comp rx=0 tx=0).stop 2023-10-08T00:11:06.678 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.674+0000 7fb6ec84c700 1 -- 172.21.15.99:0/2214211372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6e806d2e0 msgr2=0x7fb6e80c56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:06.678 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.674+0000 7fb6ec84c700 1 --2- 172.21.15.99:0/2214211372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6e806d2e0 0x7fb6e80c56a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb6e000bbb0 tx=0x7fb6e0009840 comp rx=0 tx=0).stop 2023-10-08T00:11:06.679 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.674+0000 7fb6ec84c700 1 -- 172.21.15.99:0/2214211372 shutdown_connections 2023-10-08T00:11:06.679 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.674+0000 7fb6ec84c700 1 --2- 172.21.15.99:0/2214211372 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb6c8066f30 0x7fb6c80693e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.679 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.674+0000 7fb6ec84c700 1 --2- 172.21.15.99:0/2214211372 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb6e8155b60 0x7fb6e8155f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.679 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.674+0000 7fb6ec84c700 1 --2- 172.21.15.99:0/2214211372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6e806d2e0 0x7fb6e80c56a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.679 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.674+0000 7fb6ec84c700 1 --2- 172.21.15.99:0/2214211372 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb6e806d6b0 0x7fb6e8177780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.679 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.674+0000 7fb6ec84c700 1 -- 172.21.15.99:0/2214211372 >> 172.21.15.99:0/2214211372 conn(0x7fb6e8070700 msgr2=0x7fb6e80c8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:06.679 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.674+0000 7fb6ec84c700 1 -- 172.21.15.99:0/2214211372 shutdown_connections 2023-10-08T00:11:06.679 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.674+0000 7fb6ec84c700 1 -- 172.21.15.99:0/2214211372 wait complete. 2023-10-08T00:11:06.682 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.4 2023-10-08T00:11:06.683 DEBUG:teuthology.parallel:result is None 2023-10-08T00:11:06.684 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.6 2023-10-08T00:11:06.684 DEBUG:teuthology.parallel:result is None 2023-10-08T00:11:06.690 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2023-10-08T00:11:06.691 DEBUG:teuthology.parallel:result is None 2023-10-08T00:11:06.692 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.5 2023-10-08T00:11:06.692 DEBUG:teuthology.parallel:result is None 2023-10-08T00:11:06.693 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-08T00:11:06.693 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T00:11:06.845 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.844+0000 7f42f5c0a700 1 Processor -- start 2023-10-08T00:11:06.845 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.844+0000 7f42f5c0a700 1 -- start start 2023-10-08T00:11:06.851 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.845+0000 7f42f5c0a700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f42f0143520 0x7f42f00cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.851 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.845+0000 7f42f5c0a700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f42f01438f0 0x7f42f006a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.851 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.845+0000 7f42f5c0a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f42f00ccf00 0x7f42f006a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.845+0000 7f42f5c0a700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42f00d0870 con 0x7f42f0143520 2023-10-08T00:11:06.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.845+0000 7f42f5c0a700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42f00cdfb0 con 0x7f42f01438f0 2023-10-08T00:11:06.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.845+0000 7f42f5c0a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42f00cfc60 con 0x7f42f00ccf00 2023-10-08T00:11:06.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.845+0000 7f42f4c08700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f42f0143520 0x7f42f00cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:06.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.845+0000 7f42f4c08700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f42f0143520 0x7f42f00cf580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38898/0 (socket says 172.21.15.99:38898) 2023-10-08T00:11:06.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.845+0000 7f42f4c08700 1 -- 172.21.15.99:0/1693961801 learned_addr learned my addr 172.21.15.99:0/1693961801 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:06.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.845+0000 7f42f5409700 1 --2- 172.21.15.99:0/1693961801 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f42f00ccf00 0x7f42f006a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:06.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.845+0000 7f42effff700 1 --2- 172.21.15.99:0/1693961801 >> v2:172.21.15.191:3300/0 conn(0x7f42f01438f0 0x7f42f006a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:06.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.845+0000 7f42f4c08700 1 -- 172.21.15.99:0/1693961801 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f42f00ccf00 msgr2=0x7f42f006a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:06.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.845+0000 7f42f4c08700 1 --2- 172.21.15.99:0/1693961801 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f42f00ccf00 0x7f42f006a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.845+0000 7f42f4c08700 1 -- 172.21.15.99:0/1693961801 >> v2:172.21.15.191:3300/0 conn(0x7f42f01438f0 msgr2=0x7f42f006a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:06.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.845+0000 7f42f4c08700 1 --2- 172.21.15.99:0/1693961801 >> v2:172.21.15.191:3300/0 conn(0x7f42f01438f0 0x7f42f006a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.845+0000 7f42f4c08700 1 -- 172.21.15.99:0/1693961801 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f00cff40 con 0x7f42f0143520 2023-10-08T00:11:06.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.845+0000 7f42effff700 1 --2- 172.21.15.99:0/1693961801 >> v2:172.21.15.191:3300/0 conn(0x7f42f01438f0 0x7f42f006a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:11:06.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.846+0000 7f42f4c08700 1 --2- 172.21.15.99:0/1693961801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42f0143520 0x7f42f00cf580 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f42f015d9c0 tx=0x7f42e8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78512476cd1f330f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:06.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.846+0000 7f42ef7fe700 1 -- 172.21.15.99:0/1693961801 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e800e070 con 0x7f42f0143520 2023-10-08T00:11:06.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.846+0000 7f42ef7fe700 1 -- 172.21.15.99:0/1693961801 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42e800ba20 con 0x7f42f0143520 2023-10-08T00:11:06.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.846+0000 7f42ef7fe700 1 -- 172.21.15.99:0/1693961801 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e80092e0 con 0x7f42f0143520 2023-10-08T00:11:06.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.846+0000 7f42f5c0a700 1 -- 172.21.15.99:0/1693961801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42f0143520 msgr2=0x7f42f00cf580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:06.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.846+0000 7f42f5c0a700 1 --2- 172.21.15.99:0/1693961801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42f0143520 0x7f42f00cf580 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f42f015d9c0 tx=0x7f42e8009840 comp rx=0 tx=0).stop 2023-10-08T00:11:06.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.846+0000 7f42f5c0a700 1 -- 172.21.15.99:0/1693961801 shutdown_connections 2023-10-08T00:11:06.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.846+0000 7f42f5c0a700 1 --2- 172.21.15.99:0/1693961801 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f42f00ccf00 0x7f42f006a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.846+0000 7f42f5c0a700 1 --2- 172.21.15.99:0/1693961801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42f0143520 0x7f42f00cf580 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.846+0000 7f42f5c0a700 1 --2- 172.21.15.99:0/1693961801 >> v2:172.21.15.191:3300/0 conn(0x7f42f01438f0 0x7f42f006a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.846+0000 7f42f5c0a700 1 -- 172.21.15.99:0/1693961801 >> 172.21.15.99:0/1693961801 conn(0x7f42f0069750 msgr2=0x7f42f00b25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:06.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.846+0000 7f42f5c0a700 1 -- 172.21.15.99:0/1693961801 shutdown_connections 2023-10-08T00:11:06.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.846+0000 7f42f5c0a700 1 -- 172.21.15.99:0/1693961801 wait complete. 2023-10-08T00:11:06.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.847+0000 7f42f5c0a700 1 Processor -- start 2023-10-08T00:11:06.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.848+0000 7f42f5c0a700 1 -- start start 2023-10-08T00:11:06.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.848+0000 7f42f5c0a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42f00ccf00 0x7f42f0139f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.848+0000 7f42f5c0a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42f0143520 0x7f42f00b5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.848+0000 7f42f5c0a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f42f01438f0 0x7f42f00b21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.848+0000 7f42f5c0a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f013d1e0 con 0x7f42f00ccf00 2023-10-08T00:11:06.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.848+0000 7f42f5c0a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f013d4c0 con 0x7f42f0143520 2023-10-08T00:11:06.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.848+0000 7f42f5c0a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42f013d7a0 con 0x7f42f01438f0 2023-10-08T00:11:06.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.848+0000 7f42f4c08700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42f00ccf00 0x7f42f0139f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:06.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.848+0000 7f42f4c08700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42f00ccf00 0x7f42f0139f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38904/0 (socket says 172.21.15.99:38904) 2023-10-08T00:11:06.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.848+0000 7f42f4c08700 1 -- 172.21.15.99:0/1974553487 learned_addr learned my addr 172.21.15.99:0/1974553487 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:06.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.848+0000 7f42effff700 1 --2- 172.21.15.99:0/1974553487 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42f0143520 0x7f42f00b5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:06.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.848+0000 7f42f5409700 1 --2- 172.21.15.99:0/1974553487 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f42f01438f0 0x7f42f00b21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:06.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.849+0000 7f42f4c08700 1 -- 172.21.15.99:0/1974553487 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f42f01438f0 msgr2=0x7f42f00b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:06.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.849+0000 7f42f4c08700 1 --2- 172.21.15.99:0/1974553487 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f42f01438f0 0x7f42f00b21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.849+0000 7f42f4c08700 1 -- 172.21.15.99:0/1974553487 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42f0143520 msgr2=0x7f42f00b5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:06.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.849+0000 7f42f4c08700 1 --2- 172.21.15.99:0/1974553487 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42f0143520 0x7f42f00b5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:06.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.849+0000 7f42f4c08700 1 -- 172.21.15.99:0/1974553487 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e8009ae0 con 0x7f42f00ccf00 2023-10-08T00:11:06.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.849+0000 7f42effff700 1 --2- 172.21.15.99:0/1974553487 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42f0143520 0x7f42f00b5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:11:06.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.849+0000 7f42f4c08700 1 --2- 172.21.15.99:0/1974553487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42f00ccf00 0x7f42f0139f30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f42f0071eb0 tx=0x7f42e8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:06.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.849+0000 7f42ecff9700 1 -- 172.21.15.99:0/1974553487 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e8008c30 con 0x7f42f00ccf00 2023-10-08T00:11:06.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.849+0000 7f42ecff9700 1 -- 172.21.15.99:0/1974553487 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42e8008db0 con 0x7f42f00ccf00 2023-10-08T00:11:06.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.849+0000 7f42f5c0a700 1 -- 172.21.15.99:0/1974553487 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42f013da80 con 0x7f42f00ccf00 2023-10-08T00:11:06.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.849+0000 7f42ecff9700 1 -- 172.21.15.99:0/1974553487 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e80266b0 con 0x7f42f00ccf00 2023-10-08T00:11:06.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.849+0000 7f42f5c0a700 1 -- 172.21.15.99:0/1974553487 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42f013e280 con 0x7f42f00ccf00 2023-10-08T00:11:06.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.850+0000 7f42f5c0a700 1 -- 172.21.15.99:0/1974553487 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42f0089030 con 0x7f42f00ccf00 2023-10-08T00:11:06.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.850+0000 7f42ecff9700 1 -- 172.21.15.99:0/1974553487 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f42e8003040 con 0x7f42f00ccf00 2023-10-08T00:11:06.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.851+0000 7f42ecff9700 1 --2- 172.21.15.99:0/1974553487 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f42d0066f30 0x7f42d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:06.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.851+0000 7f42ecff9700 1 -- 172.21.15.99:0/1974553487 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f42e80864f0 con 0x7f42f00ccf00 2023-10-08T00:11:06.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.851+0000 7f42effff700 1 --2- 172.21.15.99:0/1974553487 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f42d0066f30 0x7f42d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:06.859 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.851+0000 7f42effff700 1 --2- 172.21.15.99:0/1974553487 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f42d0066f30 0x7f42d00693e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f42e0006f00 tx=0x7f42e0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:06.859 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:06.855+0000 7f42ecff9700 1 -- 172.21.15.99:0/1974553487 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f42e8053760 con 0x7f42f00ccf00 2023-10-08T00:11:07.032 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.031+0000 7f42f5c0a700 1 -- 172.21.15.99:0/1974553487 --> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f42f013dfa0 con 0x7f42d0066f30 2023-10-08T00:11:07.039 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.039+0000 7f42ecff9700 1 -- 172.21.15.99:0/1974553487 <== mgr.4099 v2:172.21.15.191:6832/85985 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+260191 (secure 0 0 0) 0x7f42f013dfa0 con 0x7f42d0066f30 2023-10-08T00:11:07.043 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:11:07.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.042+0000 7f42f5c0a700 1 -- 172.21.15.99:0/1974553487 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f42d0066f30 msgr2=0x7f42d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:07.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.042+0000 7f42f5c0a700 1 --2- 172.21.15.99:0/1974553487 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f42d0066f30 0x7f42d00693e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f42e0006f00 tx=0x7f42e0008040 comp rx=0 tx=0).stop 2023-10-08T00:11:07.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.042+0000 7f42f5c0a700 1 -- 172.21.15.99:0/1974553487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42f00ccf00 msgr2=0x7f42f0139f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:07.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.042+0000 7f42f5c0a700 1 --2- 172.21.15.99:0/1974553487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42f00ccf00 0x7f42f0139f30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f42f0071eb0 tx=0x7f42e8018000 comp rx=0 tx=0).stop 2023-10-08T00:11:07.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.042+0000 7f42f5c0a700 1 -- 172.21.15.99:0/1974553487 shutdown_connections 2023-10-08T00:11:07.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.042+0000 7f42f5c0a700 1 --2- 172.21.15.99:0/1974553487 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f42d0066f30 0x7f42d00693e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:07.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.042+0000 7f42f5c0a700 1 --2- 172.21.15.99:0/1974553487 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f42f01438f0 0x7f42f00b21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:07.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.042+0000 7f42f5c0a700 1 --2- 172.21.15.99:0/1974553487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42f00ccf00 0x7f42f0139f30 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:07.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.042+0000 7f42f5c0a700 1 --2- 172.21.15.99:0/1974553487 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42f0143520 0x7f42f00b5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:07.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.042+0000 7f42f5c0a700 1 -- 172.21.15.99:0/1974553487 >> 172.21.15.99:0/1974553487 conn(0x7f42f0069750 msgr2=0x7f42f00b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:07.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.042+0000 7f42f5c0a700 1 -- 172.21.15.99:0/1974553487 shutdown_connections 2023-10-08T00:11:07.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.042+0000 7f42f5c0a700 1 -- 172.21.15.99:0/1974553487 wait complete. 2023-10-08T00:11:07.047 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2023-10-08T00:11:07.055 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-08T00:11:06.026880+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1708,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560791,"statfs":{"total":1600000000,"available":1598250700,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.943944"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.095544+0000","last_change":"2023-10-08T00:10:54.094801+0000","last_active":"2023-10-08T00:10:54.095544+0000","last_peered":"2023-10-08T00:10:54.095544+0000","last_clean":"2023-10-08T00:10:54.095544+0000","last_became_active":"2023-10-08T00:10:54.094221+0000","last_became_peered":"2023-10-08T00:10:54.094221+0000","last_unstale":"2023-10-08T00:10:54.095544+0000","last_undegraded":"2023-10-08T00:10:54.095544+0000","last_fullsized":"2023-10-08T00:10:54.095544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:30:37.085534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.102469+0000","last_change":"2023-10-08T00:10:54.101252+0000","last_active":"2023-10-08T00:10:54.102469+0000","last_peered":"2023-10-08T00:10:54.102469+0000","last_clean":"2023-10-08T00:10:54.102469+0000","last_became_active":"2023-10-08T00:10:54.101016+0000","last_became_peered":"2023-10-08T00:10:54.101016+0000","last_unstale":"2023-10-08T00:10:54.102469+0000","last_undegraded":"2023-10-08T00:10:54.102469+0000","last_fullsized":"2023-10-08T00:10:54.102469+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:30:15.548694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.104771+0000","last_change":"2023-10-08T00:10:54.104705+0000","last_active":"2023-10-08T00:10:54.104771+0000","last_peered":"2023-10-08T00:10:54.104771+0000","last_clean":"2023-10-08T00:10:54.104771+0000","last_became_active":"2023-10-08T00:10:54.104469+0000","last_became_peered":"2023-10-08T00:10:54.104469+0000","last_unstale":"2023-10-08T00:10:54.104771+0000","last_undegraded":"2023-10-08T00:10:54.104771+0000","last_fullsized":"2023-10-08T00:10:54.104771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:27:57.768735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.090488+0000","last_change":"2023-10-08T00:10:54.089800+0000","last_active":"2023-10-08T00:10:54.090488+0000","last_peered":"2023-10-08T00:10:54.090488+0000","last_clean":"2023-10-08T00:10:54.090488+0000","last_became_active":"2023-10-08T00:10:54.089200+0000","last_became_peered":"2023-10-08T00:10:54.089200+0000","last_unstale":"2023-10-08T00:10:54.090488+0000","last_undegraded":"2023-10-08T00:10:54.090488+0000","last_fullsized":"2023-10-08T00:10:54.090488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:44:19.725645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.099476+0000","last_change":"2023-10-08T00:10:54.099417+0000","last_active":"2023-10-08T00:10:54.099476+0000","last_peered":"2023-10-08T00:10:54.099476+0000","last_clean":"2023-10-08T00:10:54.099476+0000","last_became_active":"2023-10-08T00:10:54.099060+0000","last_became_peered":"2023-10-08T00:10:54.099060+0000","last_unstale":"2023-10-08T00:10:54.099476+0000","last_undegraded":"2023-10-08T00:10:54.099476+0000","last_fullsized":"2023-10-08T00:10:54.099476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:20:35.671993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.088556+0000","last_change":"2023-10-08T00:10:54.088499+0000","last_active":"2023-10-08T00:10:54.088556+0000","last_peered":"2023-10-08T00:10:54.088556+0000","last_clean":"2023-10-08T00:10:54.088556+0000","last_became_active":"2023-10-08T00:10:54.088247+0000","last_became_peered":"2023-10-08T00:10:54.088247+0000","last_unstale":"2023-10-08T00:10:54.088556+0000","last_undegraded":"2023-10-08T00:10:54.088556+0000","last_fullsized":"2023-10-08T00:10:54.088556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:49:51.549111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.098443+0000","last_change":"2023-10-08T00:10:54.098398+0000","last_active":"2023-10-08T00:10:54.098443+0000","last_peered":"2023-10-08T00:10:54.098443+0000","last_clean":"2023-10-08T00:10:54.098443+0000","last_became_active":"2023-10-08T00:10:54.098065+0000","last_became_peered":"2023-10-08T00:10:54.098065+0000","last_unstale":"2023-10-08T00:10:54.098443+0000","last_undegraded":"2023-10-08T00:10:54.098443+0000","last_fullsized":"2023-10-08T00:10:54.098443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:05:53.331248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.099579+0000","last_change":"2023-10-08T00:10:54.098754+0000","last_active":"2023-10-08T00:10:54.099579+0000","last_peered":"2023-10-08T00:10:54.099579+0000","last_clean":"2023-10-08T00:10:54.099579+0000","last_became_active":"2023-10-08T00:10:54.098505+0000","last_became_peered":"2023-10-08T00:10:54.098505+0000","last_unstale":"2023-10-08T00:10:54.099579+0000","last_undegraded":"2023-10-08T00:10:54.099579+0000","last_fullsized":"2023-10-08T00:10:54.099579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:20:35.671993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.096851+0000","last_change":"2023-10-08T00:10:54.096792+0000","last_active":"2023-10-08T00:10:54.096851+0000","last_peered":"2023-10-08T00:10:54.096851+0000","last_clean":"2023-10-08T00:10:54.096851+0000","last_became_active":"2023-10-08T00:10:54.096569+0000","last_became_peered":"2023-10-08T00:10:54.096569+0000","last_unstale":"2023-10-08T00:10:54.096851+0000","last_undegraded":"2023-10-08T00:10:54.096851+0000","last_fullsized":"2023-10-08T00:10:54.096851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:40:12.384542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.093509+0000","last_change":"2023-10-08T00:10:54.092144+0000","last_active":"2023-10-08T00:10:54.093509+0000","last_peered":"2023-10-08T00:10:54.093509+0000","last_clean":"2023-10-08T00:10:54.093509+0000","last_became_active":"2023-10-08T00:10:54.091938+0000","last_became_peered":"2023-10-08T00:10:54.091938+0000","last_unstale":"2023-10-08T00:10:54.093509+0000","last_undegraded":"2023-10-08T00:10:54.093509+0000","last_fullsized":"2023-10-08T00:10:54.093509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:13:19.313381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.095525+0000","last_change":"2023-10-08T00:10:54.094651+0000","last_active":"2023-10-08T00:10:54.095525+0000","last_peered":"2023-10-08T00:10:54.095525+0000","last_clean":"2023-10-08T00:10:54.095525+0000","last_became_active":"2023-10-08T00:10:54.093973+0000","last_became_peered":"2023-10-08T00:10:54.093973+0000","last_unstale":"2023-10-08T00:10:54.095525+0000","last_undegraded":"2023-10-08T00:10:54.095525+0000","last_fullsized":"2023-10-08T00:10:54.095525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:27:57.768735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.093543+0000","last_change":"2023-10-08T00:10:54.092456+0000","last_active":"2023-10-08T00:10:54.093543+0000","last_peered":"2023-10-08T00:10:54.093543+0000","last_clean":"2023-10-08T00:10:54.093543+0000","last_became_active":"2023-10-08T00:10:54.092121+0000","last_became_peered":"2023-10-08T00:10:54.092121+0000","last_unstale":"2023-10-08T00:10:54.093543+0000","last_undegraded":"2023-10-08T00:10:54.093543+0000","last_fullsized":"2023-10-08T00:10:54.093543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:17:33.566050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.102806+0000","last_change":"2023-10-08T00:10:54.102717+0000","last_active":"2023-10-08T00:10:54.102806+0000","last_peered":"2023-10-08T00:10:54.102806+0000","last_clean":"2023-10-08T00:10:54.102806+0000","last_became_active":"2023-10-08T00:10:54.102373+0000","last_became_peered":"2023-10-08T00:10:54.102373+0000","last_unstale":"2023-10-08T00:10:54.102806+0000","last_undegraded":"2023-10-08T00:10:54.102806+0000","last_fullsized":"2023-10-08T00:10:54.102806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:20:26.989181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.103024+0000","last_change":"2023-10-08T00:10:54.102984+0000","last_active":"2023-10-08T00:10:54.103024+0000","last_peered":"2023-10-08T00:10:54.103024+0000","last_clean":"2023-10-08T00:10:54.103024+0000","last_became_active":"2023-10-08T00:10:54.102645+0000","last_became_peered":"2023-10-08T00:10:54.102645+0000","last_unstale":"2023-10-08T00:10:54.103024+0000","last_undegraded":"2023-10-08T00:10:54.103024+0000","last_fullsized":"2023-10-08T00:10:54.103024+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:44:19.725645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.100866+0000","last_change":"2023-10-08T00:10:54.100800+0000","last_active":"2023-10-08T00:10:54.100866+0000","last_peered":"2023-10-08T00:10:54.100866+0000","last_clean":"2023-10-08T00:10:54.100866+0000","last_became_active":"2023-10-08T00:10:54.100384+0000","last_became_peered":"2023-10-08T00:10:54.100384+0000","last_unstale":"2023-10-08T00:10:54.100866+0000","last_undegraded":"2023-10-08T00:10:54.100866+0000","last_fullsized":"2023-10-08T00:10:54.100866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:05:53.331248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.098456+0000","last_change":"2023-10-08T00:10:54.097988+0000","last_active":"2023-10-08T00:10:54.098456+0000","last_peered":"2023-10-08T00:10:54.098456+0000","last_clean":"2023-10-08T00:10:54.098456+0000","last_became_active":"2023-10-08T00:10:54.097155+0000","last_became_peered":"2023-10-08T00:10:54.097155+0000","last_unstale":"2023-10-08T00:10:54.098456+0000","last_undegraded":"2023-10-08T00:10:54.098456+0000","last_fullsized":"2023-10-08T00:10:54.098456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:03:51.083770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.093657+0000","last_change":"2023-10-08T00:10:54.093109+0000","last_active":"2023-10-08T00:10:54.093657+0000","last_peered":"2023-10-08T00:10:54.093657+0000","last_clean":"2023-10-08T00:10:54.093657+0000","last_became_active":"2023-10-08T00:10:54.092788+0000","last_became_peered":"2023-10-08T00:10:54.092788+0000","last_unstale":"2023-10-08T00:10:54.093657+0000","last_undegraded":"2023-10-08T00:10:54.093657+0000","last_fullsized":"2023-10-08T00:10:54.093657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:37:23.781764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.085610+0000","last_change":"2023-10-08T00:10:54.085533+0000","last_active":"2023-10-08T00:10:54.085610+0000","last_peered":"2023-10-08T00:10:54.085610+0000","last_clean":"2023-10-08T00:10:54.085610+0000","last_became_active":"2023-10-08T00:10:54.085297+0000","last_became_peered":"2023-10-08T00:10:54.085297+0000","last_unstale":"2023-10-08T00:10:54.085610+0000","last_undegraded":"2023-10-08T00:10:54.085610+0000","last_fullsized":"2023-10-08T00:10:54.085610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:42:35.332320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.093626+0000","last_change":"2023-10-08T00:10:54.092802+0000","last_active":"2023-10-08T00:10:54.093626+0000","last_peered":"2023-10-08T00:10:54.093626+0000","last_clean":"2023-10-08T00:10:54.093626+0000","last_became_active":"2023-10-08T00:10:54.092217+0000","last_became_peered":"2023-10-08T00:10:54.092217+0000","last_unstale":"2023-10-08T00:10:54.093626+0000","last_undegraded":"2023-10-08T00:10:54.093626+0000","last_fullsized":"2023-10-08T00:10:54.093626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:20:35.671993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.103423+0000","last_change":"2023-10-08T00:10:53.083367+0000","last_active":"2023-10-08T00:10:54.103423+0000","last_peered":"2023-10-08T00:10:54.103423+0000","last_clean":"2023-10-08T00:10:54.103423+0000","last_became_active":"2023-10-08T00:10:53.083127+0000","last_became_peered":"2023-10-08T00:10:53.083127+0000","last_unstale":"2023-10-08T00:10:54.103423+0000","last_undegraded":"2023-10-08T00:10:54.103423+0000","last_fullsized":"2023-10-08T00:10:54.103423+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:10:42.328030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.099821+0000","last_change":"2023-10-08T00:10:54.099725+0000","last_active":"2023-10-08T00:10:54.099821+0000","last_peered":"2023-10-08T00:10:54.099821+0000","last_clean":"2023-10-08T00:10:54.099821+0000","last_became_active":"2023-10-08T00:10:54.099236+0000","last_became_peered":"2023-10-08T00:10:54.099236+0000","last_unstale":"2023-10-08T00:10:54.099821+0000","last_undegraded":"2023-10-08T00:10:54.099821+0000","last_fullsized":"2023-10-08T00:10:54.099821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:13:19.313381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.091381+0000","last_change":"2023-10-08T00:10:53.078523+0000","last_active":"2023-10-08T00:10:54.091381+0000","last_peered":"2023-10-08T00:10:54.091381+0000","last_clean":"2023-10-08T00:10:54.091381+0000","last_became_active":"2023-10-08T00:10:53.078068+0000","last_became_peered":"2023-10-08T00:10:53.078068+0000","last_unstale":"2023-10-08T00:10:54.091381+0000","last_undegraded":"2023-10-08T00:10:54.091381+0000","last_fullsized":"2023-10-08T00:10:54.091381+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:30:06.828965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.094722+0000","last_change":"2023-10-08T00:10:54.094485+0000","last_active":"2023-10-08T00:10:54.094722+0000","last_peered":"2023-10-08T00:10:54.094722+0000","last_clean":"2023-10-08T00:10:54.094722+0000","last_became_active":"2023-10-08T00:10:54.094246+0000","last_became_peered":"2023-10-08T00:10:54.094246+0000","last_unstale":"2023-10-08T00:10:54.094722+0000","last_undegraded":"2023-10-08T00:10:54.094722+0000","last_fullsized":"2023-10-08T00:10:54.094722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:52:39.600888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.099727+0000","last_change":"2023-10-08T00:10:53.086334+0000","last_active":"2023-10-08T00:10:54.099727+0000","last_peered":"2023-10-08T00:10:54.099727+0000","last_clean":"2023-10-08T00:10:54.099727+0000","last_became_active":"2023-10-08T00:10:53.084152+0000","last_became_peered":"2023-10-08T00:10:53.084152+0000","last_unstale":"2023-10-08T00:10:54.099727+0000","last_undegraded":"2023-10-08T00:10:54.099727+0000","last_fullsized":"2023-10-08T00:10:54.099727+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:08:25.404396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.095590+0000","last_change":"2023-10-08T00:10:54.095225+0000","last_active":"2023-10-08T00:10:54.095590+0000","last_peered":"2023-10-08T00:10:54.095590+0000","last_clean":"2023-10-08T00:10:54.095590+0000","last_became_active":"2023-10-08T00:10:54.095005+0000","last_became_peered":"2023-10-08T00:10:54.095005+0000","last_unstale":"2023-10-08T00:10:54.095590+0000","last_undegraded":"2023-10-08T00:10:54.095590+0000","last_fullsized":"2023-10-08T00:10:54.095590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:49:51.549111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.099165+0000","last_change":"2023-10-08T00:10:53.082935+0000","last_active":"2023-10-08T00:10:54.099165+0000","last_peered":"2023-10-08T00:10:54.099165+0000","last_clean":"2023-10-08T00:10:54.099165+0000","last_became_active":"2023-10-08T00:10:53.082451+0000","last_became_peered":"2023-10-08T00:10:53.082451+0000","last_unstale":"2023-10-08T00:10:54.099165+0000","last_undegraded":"2023-10-08T00:10:54.099165+0000","last_fullsized":"2023-10-08T00:10:54.099165+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:30:06.828965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.093599+0000","last_change":"2023-10-08T00:10:54.092264+0000","last_active":"2023-10-08T00:10:54.093599+0000","last_peered":"2023-10-08T00:10:54.093599+0000","last_clean":"2023-10-08T00:10:54.093599+0000","last_became_active":"2023-10-08T00:10:54.092043+0000","last_became_peered":"2023-10-08T00:10:54.092043+0000","last_unstale":"2023-10-08T00:10:54.093599+0000","last_undegraded":"2023-10-08T00:10:54.093599+0000","last_fullsized":"2023-10-08T00:10:54.093599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:39:51.147291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.101350+0000","last_change":"2023-10-08T00:10:53.083563+0000","last_active":"2023-10-08T00:10:54.101350+0000","last_peered":"2023-10-08T00:10:54.101350+0000","last_clean":"2023-10-08T00:10:54.101350+0000","last_became_active":"2023-10-08T00:10:53.079738+0000","last_became_peered":"2023-10-08T00:10:53.079738+0000","last_unstale":"2023-10-08T00:10:54.101350+0000","last_undegraded":"2023-10-08T00:10:54.101350+0000","last_fullsized":"2023-10-08T00:10:54.101350+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:46:39.872883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.095572+0000","last_change":"2023-10-08T00:10:54.095150+0000","last_active":"2023-10-08T00:10:54.095572+0000","last_peered":"2023-10-08T00:10:54.095572+0000","last_clean":"2023-10-08T00:10:54.095572+0000","last_became_active":"2023-10-08T00:10:54.094621+0000","last_became_peered":"2023-10-08T00:10:54.094621+0000","last_unstale":"2023-10-08T00:10:54.095572+0000","last_undegraded":"2023-10-08T00:10:54.095572+0000","last_fullsized":"2023-10-08T00:10:54.095572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:42:35.332320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.098615+0000","last_change":"2023-10-08T00:10:53.085123+0000","last_active":"2023-10-08T00:10:54.098615+0000","last_peered":"2023-10-08T00:10:54.098615+0000","last_clean":"2023-10-08T00:10:54.098615+0000","last_became_active":"2023-10-08T00:10:53.084901+0000","last_became_peered":"2023-10-08T00:10:53.084901+0000","last_unstale":"2023-10-08T00:10:54.098615+0000","last_undegraded":"2023-10-08T00:10:54.098615+0000","last_fullsized":"2023-10-08T00:10:54.098615+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:53:05.715993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.098231+0000","last_change":"2023-10-08T00:10:54.098150+0000","last_active":"2023-10-08T00:10:54.098231+0000","last_peered":"2023-10-08T00:10:54.098231+0000","last_clean":"2023-10-08T00:10:54.098231+0000","last_became_active":"2023-10-08T00:10:54.097887+0000","last_became_peered":"2023-10-08T00:10:54.097887+0000","last_unstale":"2023-10-08T00:10:54.098231+0000","last_undegraded":"2023-10-08T00:10:54.098231+0000","last_fullsized":"2023-10-08T00:10:54.098231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:06:38.007522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.096544+0000","last_change":"2023-10-08T00:10:53.082817+0000","last_active":"2023-10-08T00:10:54.096544+0000","last_peered":"2023-10-08T00:10:54.096544+0000","last_clean":"2023-10-08T00:10:54.096544+0000","last_became_active":"2023-10-08T00:10:53.082576+0000","last_became_peered":"2023-10-08T00:10:53.082576+0000","last_unstale":"2023-10-08T00:10:54.096544+0000","last_undegraded":"2023-10-08T00:10:54.096544+0000","last_fullsized":"2023-10-08T00:10:54.096544+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:46:39.872883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.100057+0000","last_change":"2023-10-08T00:10:54.099958+0000","last_active":"2023-10-08T00:10:54.100057+0000","last_peered":"2023-10-08T00:10:54.100057+0000","last_clean":"2023-10-08T00:10:54.100057+0000","last_became_active":"2023-10-08T00:10:54.099664+0000","last_became_peered":"2023-10-08T00:10:54.099664+0000","last_unstale":"2023-10-08T00:10:54.100057+0000","last_undegraded":"2023-10-08T00:10:54.100057+0000","last_fullsized":"2023-10-08T00:10:54.100057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:17:33.566050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.090051+0000","last_change":"2023-10-08T00:10:53.077494+0000","last_active":"2023-10-08T00:10:54.090051+0000","last_peered":"2023-10-08T00:10:54.090051+0000","last_clean":"2023-10-08T00:10:54.090051+0000","last_became_active":"2023-10-08T00:10:53.077248+0000","last_became_peered":"2023-10-08T00:10:53.077248+0000","last_unstale":"2023-10-08T00:10:54.090051+0000","last_undegraded":"2023-10-08T00:10:54.090051+0000","last_fullsized":"2023-10-08T00:10:54.090051+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:46:39.872883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.097553+0000","last_change":"2023-10-08T00:10:54.095175+0000","last_active":"2023-10-08T00:10:54.097553+0000","last_peered":"2023-10-08T00:10:54.097553+0000","last_clean":"2023-10-08T00:10:54.097553+0000","last_became_active":"2023-10-08T00:10:54.093363+0000","last_became_peered":"2023-10-08T00:10:54.093363+0000","last_unstale":"2023-10-08T00:10:54.097553+0000","last_undegraded":"2023-10-08T00:10:54.097553+0000","last_fullsized":"2023-10-08T00:10:54.097553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:03:42.084791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.087857+0000","last_change":"2023-10-08T00:10:53.079999+0000","last_active":"2023-10-08T00:10:54.087857+0000","last_peered":"2023-10-08T00:10:54.087857+0000","last_clean":"2023-10-08T00:10:54.087857+0000","last_became_active":"2023-10-08T00:10:53.079750+0000","last_became_peered":"2023-10-08T00:10:53.079750+0000","last_unstale":"2023-10-08T00:10:54.087857+0000","last_undegraded":"2023-10-08T00:10:54.087857+0000","last_fullsized":"2023-10-08T00:10:54.087857+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:00:50.648234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.087312+0000","last_change":"2023-10-08T00:10:54.087253+0000","last_active":"2023-10-08T00:10:54.087312+0000","last_peered":"2023-10-08T00:10:54.087312+0000","last_clean":"2023-10-08T00:10:54.087312+0000","last_became_active":"2023-10-08T00:10:54.087034+0000","last_became_peered":"2023-10-08T00:10:54.087034+0000","last_unstale":"2023-10-08T00:10:54.087312+0000","last_undegraded":"2023-10-08T00:10:54.087312+0000","last_fullsized":"2023-10-08T00:10:54.087312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:21:24.202186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.090769+0000","last_change":"2023-10-08T00:10:53.088156+0000","last_active":"2023-10-08T00:10:54.090769+0000","last_peered":"2023-10-08T00:10:54.090769+0000","last_clean":"2023-10-08T00:10:54.090769+0000","last_became_active":"2023-10-08T00:10:53.084715+0000","last_became_peered":"2023-10-08T00:10:53.084715+0000","last_unstale":"2023-10-08T00:10:54.090769+0000","last_undegraded":"2023-10-08T00:10:54.090769+0000","last_fullsized":"2023-10-08T00:10:54.090769+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:10:42.328030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.100622+0000","last_change":"2023-10-08T00:10:54.100149+0000","last_active":"2023-10-08T00:10:54.100622+0000","last_peered":"2023-10-08T00:10:54.100622+0000","last_clean":"2023-10-08T00:10:54.100622+0000","last_became_active":"2023-10-08T00:10:54.099890+0000","last_became_peered":"2023-10-08T00:10:54.099890+0000","last_unstale":"2023-10-08T00:10:54.100622+0000","last_undegraded":"2023-10-08T00:10:54.100622+0000","last_fullsized":"2023-10-08T00:10:54.100622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:30:37.085534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.092369+0000","last_change":"2023-10-08T00:10:53.077948+0000","last_active":"2023-10-08T00:10:54.092369+0000","last_peered":"2023-10-08T00:10:54.092369+0000","last_clean":"2023-10-08T00:10:54.092369+0000","last_became_active":"2023-10-08T00:10:53.077716+0000","last_became_peered":"2023-10-08T00:10:53.077716+0000","last_unstale":"2023-10-08T00:10:54.092369+0000","last_undegraded":"2023-10-08T00:10:54.092369+0000","last_fullsized":"2023-10-08T00:10:54.092369+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:08:25.404396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.101262+0000","last_change":"2023-10-08T00:10:54.101209+0000","last_active":"2023-10-08T00:10:54.101262+0000","last_peered":"2023-10-08T00:10:54.101262+0000","last_clean":"2023-10-08T00:10:54.101262+0000","last_became_active":"2023-10-08T00:10:54.100833+0000","last_became_peered":"2023-10-08T00:10:54.100833+0000","last_unstale":"2023-10-08T00:10:54.101262+0000","last_undegraded":"2023-10-08T00:10:54.101262+0000","last_fullsized":"2023-10-08T00:10:54.101262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:28:03.272641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.105007+0000","last_change":"2023-10-08T00:10:53.078194+0000","last_active":"2023-10-08T00:10:54.105007+0000","last_peered":"2023-10-08T00:10:54.105007+0000","last_clean":"2023-10-08T00:10:54.105007+0000","last_became_active":"2023-10-08T00:10:53.077936+0000","last_became_peered":"2023-10-08T00:10:53.077936+0000","last_unstale":"2023-10-08T00:10:54.105007+0000","last_undegraded":"2023-10-08T00:10:54.105007+0000","last_fullsized":"2023-10-08T00:10:54.105007+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:00:50.648234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.100556+0000","last_change":"2023-10-08T00:10:54.100459+0000","last_active":"2023-10-08T00:10:54.100556+0000","last_peered":"2023-10-08T00:10:54.100556+0000","last_clean":"2023-10-08T00:10:54.100556+0000","last_became_active":"2023-10-08T00:10:54.100194+0000","last_became_peered":"2023-10-08T00:10:54.100194+0000","last_unstale":"2023-10-08T00:10:54.100556+0000","last_undegraded":"2023-10-08T00:10:54.100556+0000","last_fullsized":"2023-10-08T00:10:54.100556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:22:42.484817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.096915+0000","last_change":"2023-10-08T00:10:53.084910+0000","last_active":"2023-10-08T00:10:54.096915+0000","last_peered":"2023-10-08T00:10:54.096915+0000","last_clean":"2023-10-08T00:10:54.096915+0000","last_became_active":"2023-10-08T00:10:53.084674+0000","last_became_peered":"2023-10-08T00:10:53.084674+0000","last_unstale":"2023-10-08T00:10:54.096915+0000","last_undegraded":"2023-10-08T00:10:54.096915+0000","last_fullsized":"2023-10-08T00:10:54.096915+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:08:39.785742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.102086+0000","last_change":"2023-10-08T00:10:54.102036+0000","last_active":"2023-10-08T00:10:54.102086+0000","last_peered":"2023-10-08T00:10:54.102086+0000","last_clean":"2023-10-08T00:10:54.102086+0000","last_became_active":"2023-10-08T00:10:54.101877+0000","last_became_peered":"2023-10-08T00:10:54.101877+0000","last_unstale":"2023-10-08T00:10:54.102086+0000","last_undegraded":"2023-10-08T00:10:54.102086+0000","last_fullsized":"2023-10-08T00:10:54.102086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:21:24.202186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.084370+0000","last_change":"2023-10-08T00:10:53.082679+0000","last_active":"2023-10-08T00:10:54.084370+0000","last_peered":"2023-10-08T00:10:54.084370+0000","last_clean":"2023-10-08T00:10:54.084370+0000","last_became_active":"2023-10-08T00:10:53.082281+0000","last_became_peered":"2023-10-08T00:10:53.082281+0000","last_unstale":"2023-10-08T00:10:54.084370+0000","last_undegraded":"2023-10-08T00:10:54.084370+0000","last_fullsized":"2023-10-08T00:10:54.084370+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:11:04.856043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.099339+0000","last_change":"2023-10-08T00:10:54.099282+0000","last_active":"2023-10-08T00:10:54.099339+0000","last_peered":"2023-10-08T00:10:54.099339+0000","last_clean":"2023-10-08T00:10:54.099339+0000","last_became_active":"2023-10-08T00:10:54.099043+0000","last_became_peered":"2023-10-08T00:10:54.099043+0000","last_unstale":"2023-10-08T00:10:54.099339+0000","last_undegraded":"2023-10-08T00:10:54.099339+0000","last_fullsized":"2023-10-08T00:10:54.099339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:11:32.172262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.090870+0000","last_change":"2023-10-08T00:10:53.088502+0000","last_active":"2023-10-08T00:10:54.090870+0000","last_peered":"2023-10-08T00:10:54.090870+0000","last_clean":"2023-10-08T00:10:54.090870+0000","last_became_active":"2023-10-08T00:10:53.086396+0000","last_became_peered":"2023-10-08T00:10:53.086396+0000","last_unstale":"2023-10-08T00:10:54.090870+0000","last_undegraded":"2023-10-08T00:10:54.090870+0000","last_fullsized":"2023-10-08T00:10:54.090870+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:42:34.326823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.095605+0000","last_change":"2023-10-08T00:10:54.095487+0000","last_active":"2023-10-08T00:10:54.095605+0000","last_peered":"2023-10-08T00:10:54.095605+0000","last_clean":"2023-10-08T00:10:54.095605+0000","last_became_active":"2023-10-08T00:10:54.094918+0000","last_became_peered":"2023-10-08T00:10:54.094918+0000","last_unstale":"2023-10-08T00:10:54.095605+0000","last_undegraded":"2023-10-08T00:10:54.095605+0000","last_fullsized":"2023-10-08T00:10:54.095605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:46:53.130793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.104070+0000","last_change":"2023-10-08T00:10:53.078081+0000","last_active":"2023-10-08T00:10:54.104070+0000","last_peered":"2023-10-08T00:10:54.104070+0000","last_clean":"2023-10-08T00:10:54.104070+0000","last_became_active":"2023-10-08T00:10:53.077802+0000","last_became_peered":"2023-10-08T00:10:53.077802+0000","last_unstale":"2023-10-08T00:10:54.104070+0000","last_undegraded":"2023-10-08T00:10:54.104070+0000","last_fullsized":"2023-10-08T00:10:54.104070+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:16:40.636514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.093642+0000","last_change":"2023-10-08T00:10:54.092992+0000","last_active":"2023-10-08T00:10:54.093642+0000","last_peered":"2023-10-08T00:10:54.093642+0000","last_clean":"2023-10-08T00:10:54.093642+0000","last_became_active":"2023-10-08T00:10:54.092509+0000","last_became_peered":"2023-10-08T00:10:54.092509+0000","last_unstale":"2023-10-08T00:10:54.093642+0000","last_undegraded":"2023-10-08T00:10:54.093642+0000","last_fullsized":"2023-10-08T00:10:54.093642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:52:39.600888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.100708+0000","last_change":"2023-10-08T00:10:53.084676+0000","last_active":"2023-10-08T00:10:54.100708+0000","last_peered":"2023-10-08T00:10:54.100708+0000","last_clean":"2023-10-08T00:10:54.100708+0000","last_became_active":"2023-10-08T00:10:53.084468+0000","last_became_peered":"2023-10-08T00:10:53.084468+0000","last_unstale":"2023-10-08T00:10:54.100708+0000","last_undegraded":"2023-10-08T00:10:54.100708+0000","last_fullsized":"2023-10-08T00:10:54.100708+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:46:39.872883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.099482+0000","last_change":"2023-10-08T00:10:54.093441+0000","last_active":"2023-10-08T00:10:54.099482+0000","last_peered":"2023-10-08T00:10:54.099482+0000","last_clean":"2023-10-08T00:10:54.099482+0000","last_became_active":"2023-10-08T00:10:54.093093+0000","last_became_peered":"2023-10-08T00:10:54.093093+0000","last_unstale":"2023-10-08T00:10:54.099482+0000","last_undegraded":"2023-10-08T00:10:54.099482+0000","last_fullsized":"2023-10-08T00:10:54.099482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:25:03.816735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.093686+0000","last_change":"2023-10-08T00:10:53.080035+0000","last_active":"2023-10-08T00:10:54.093686+0000","last_peered":"2023-10-08T00:10:54.093686+0000","last_clean":"2023-10-08T00:10:54.093686+0000","last_became_active":"2023-10-08T00:10:53.079674+0000","last_became_peered":"2023-10-08T00:10:53.079674+0000","last_unstale":"2023-10-08T00:10:54.093686+0000","last_undegraded":"2023-10-08T00:10:54.093686+0000","last_fullsized":"2023-10-08T00:10:54.093686+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:08:25.404396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.100705+0000","last_change":"2023-10-08T00:10:54.100588+0000","last_active":"2023-10-08T00:10:54.100705+0000","last_peered":"2023-10-08T00:10:54.100705+0000","last_clean":"2023-10-08T00:10:54.100705+0000","last_became_active":"2023-10-08T00:10:54.100259+0000","last_became_peered":"2023-10-08T00:10:54.100259+0000","last_unstale":"2023-10-08T00:10:54.100705+0000","last_undegraded":"2023-10-08T00:10:54.100705+0000","last_fullsized":"2023-10-08T00:10:54.100705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:40:12.384542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.084858+0000","last_change":"2023-10-08T00:10:53.079356+0000","last_active":"2023-10-08T00:10:54.084858+0000","last_peered":"2023-10-08T00:10:54.084858+0000","last_clean":"2023-10-08T00:10:54.084858+0000","last_became_active":"2023-10-08T00:10:53.079134+0000","last_became_peered":"2023-10-08T00:10:53.079134+0000","last_unstale":"2023-10-08T00:10:54.084858+0000","last_undegraded":"2023-10-08T00:10:54.084858+0000","last_fullsized":"2023-10-08T00:10:54.084858+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:30:06.828965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.094617+0000","last_change":"2023-10-08T00:10:54.094512+0000","last_active":"2023-10-08T00:10:54.094617+0000","last_peered":"2023-10-08T00:10:54.094617+0000","last_clean":"2023-10-08T00:10:54.094617+0000","last_became_active":"2023-10-08T00:10:54.093703+0000","last_became_peered":"2023-10-08T00:10:54.093703+0000","last_unstale":"2023-10-08T00:10:54.094617+0000","last_undegraded":"2023-10-08T00:10:54.094617+0000","last_fullsized":"2023-10-08T00:10:54.094617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:03:42.084791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.093726+0000","last_change":"2023-10-08T00:10:54.091651+0000","last_active":"2023-10-08T00:10:54.093726+0000","last_peered":"2023-10-08T00:10:54.093726+0000","last_clean":"2023-10-08T00:10:54.093726+0000","last_became_active":"2023-10-08T00:10:54.091234+0000","last_became_peered":"2023-10-08T00:10:54.091234+0000","last_unstale":"2023-10-08T00:10:54.093726+0000","last_undegraded":"2023-10-08T00:10:54.093726+0000","last_fullsized":"2023-10-08T00:10:54.093726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:06:38.007522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.083596+0000","last_change":"2023-10-08T00:10:53.080285+0000","last_active":"2023-10-08T00:10:54.083596+0000","last_peered":"2023-10-08T00:10:54.083596+0000","last_clean":"2023-10-08T00:10:54.083596+0000","last_became_active":"2023-10-08T00:10:53.079998+0000","last_became_peered":"2023-10-08T00:10:53.079998+0000","last_unstale":"2023-10-08T00:10:54.083596+0000","last_undegraded":"2023-10-08T00:10:54.083596+0000","last_fullsized":"2023-10-08T00:10:54.083596+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:16:40.636514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.097519+0000","last_change":"2023-10-08T00:10:54.094819+0000","last_active":"2023-10-08T00:10:54.097519+0000","last_peered":"2023-10-08T00:10:54.097519+0000","last_clean":"2023-10-08T00:10:54.097519+0000","last_became_active":"2023-10-08T00:10:54.094572+0000","last_became_peered":"2023-10-08T00:10:54.094572+0000","last_unstale":"2023-10-08T00:10:54.097519+0000","last_undegraded":"2023-10-08T00:10:54.097519+0000","last_fullsized":"2023-10-08T00:10:54.097519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:21:24.202186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.084469+0000","last_change":"2023-10-08T00:10:53.083645+0000","last_active":"2023-10-08T00:10:54.084469+0000","last_peered":"2023-10-08T00:10:54.084469+0000","last_clean":"2023-10-08T00:10:54.084469+0000","last_became_active":"2023-10-08T00:10:53.083477+0000","last_became_peered":"2023-10-08T00:10:53.083477+0000","last_unstale":"2023-10-08T00:10:54.084469+0000","last_undegraded":"2023-10-08T00:10:54.084469+0000","last_fullsized":"2023-10-08T00:10:54.084469+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:42:34.326823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":163,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.086096+0000","last_change":"2023-10-08T00:10:43.728490+0000","last_active":"2023-10-08T00:10:54.086096+0000","last_peered":"2023-10-08T00:10:54.086096+0000","last_clean":"2023-10-08T00:10:54.086096+0000","last_became_active":"2023-10-08T00:10:43.728341+0000","last_became_peered":"2023-10-08T00:10:43.728341+0000","last_unstale":"2023-10-08T00:10:54.086096+0000","last_undegraded":"2023-10-08T00:10:54.086096+0000","last_fullsized":"2023-10-08T00:10:54.086096+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_clean_scrub_stamp":"2023-10-08T00:10:42.716096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:06:31.035293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.101392+0000","last_change":"2023-10-08T00:10:54.100890+0000","last_active":"2023-10-08T00:10:54.101392+0000","last_peered":"2023-10-08T00:10:54.101392+0000","last_clean":"2023-10-08T00:10:54.101392+0000","last_became_active":"2023-10-08T00:10:54.100530+0000","last_became_peered":"2023-10-08T00:10:54.100530+0000","last_unstale":"2023-10-08T00:10:54.101392+0000","last_undegraded":"2023-10-08T00:10:54.101392+0000","last_fullsized":"2023-10-08T00:10:54.101392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:20:52.135952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.086851+0000","last_change":"2023-10-08T00:10:53.079813+0000","last_active":"2023-10-08T00:10:54.086851+0000","last_peered":"2023-10-08T00:10:54.086851+0000","last_clean":"2023-10-08T00:10:54.086851+0000","last_became_active":"2023-10-08T00:10:53.079642+0000","last_became_peered":"2023-10-08T00:10:53.079642+0000","last_unstale":"2023-10-08T00:10:54.086851+0000","last_undegraded":"2023-10-08T00:10:54.086851+0000","last_fullsized":"2023-10-08T00:10:54.086851+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:40:11.379045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.086769+0000","last_change":"2023-10-08T00:10:53.082674+0000","last_active":"2023-10-08T00:10:54.086769+0000","last_peered":"2023-10-08T00:10:54.086769+0000","last_clean":"2023-10-08T00:10:54.086769+0000","last_became_active":"2023-10-08T00:10:53.082215+0000","last_became_peered":"2023-10-08T00:10:53.082215+0000","last_unstale":"2023-10-08T00:10:54.086769+0000","last_undegraded":"2023-10-08T00:10:54.086769+0000","last_fullsized":"2023-10-08T00:10:54.086769+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:25:02.811238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.101340+0000","last_change":"2023-10-08T00:10:54.100477+0000","last_active":"2023-10-08T00:10:54.101340+0000","last_peered":"2023-10-08T00:10:54.101340+0000","last_clean":"2023-10-08T00:10:54.101340+0000","last_became_active":"2023-10-08T00:10:54.100039+0000","last_became_peered":"2023-10-08T00:10:54.100039+0000","last_unstale":"2023-10-08T00:10:54.101340+0000","last_undegraded":"2023-10-08T00:10:54.101340+0000","last_fullsized":"2023-10-08T00:10:54.101340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:54:13.221147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T00:10:55.085092+0000","last_change":"2023-10-08T00:10:53.083853+0000","last_active":"2023-10-08T00:10:55.085092+0000","last_peered":"2023-10-08T00:10:55.085092+0000","last_clean":"2023-10-08T00:10:55.085092+0000","last_became_active":"2023-10-08T00:10:53.083635+0000","last_became_peered":"2023-10-08T00:10:53.083635+0000","last_unstale":"2023-10-08T00:10:55.085092+0000","last_undegraded":"2023-10-08T00:10:55.085092+0000","last_fullsized":"2023-10-08T00:10:55.085092+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:46:39.872883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.103289+0000","last_change":"2023-10-08T00:10:54.101690+0000","last_active":"2023-10-08T00:10:54.103289+0000","last_peered":"2023-10-08T00:10:54.103289+0000","last_clean":"2023-10-08T00:10:54.103289+0000","last_became_active":"2023-10-08T00:10:54.101460+0000","last_became_peered":"2023-10-08T00:10:54.101460+0000","last_unstale":"2023-10-08T00:10:54.103289+0000","last_undegraded":"2023-10-08T00:10:54.103289+0000","last_fullsized":"2023-10-08T00:10:54.103289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:20:26.989181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.099336+0000","last_change":"2023-10-08T00:10:53.089306+0000","last_active":"2023-10-08T00:10:54.099336+0000","last_peered":"2023-10-08T00:10:54.099336+0000","last_clean":"2023-10-08T00:10:54.099336+0000","last_became_active":"2023-10-08T00:10:53.089108+0000","last_became_peered":"2023-10-08T00:10:53.089108+0000","last_unstale":"2023-10-08T00:10:54.099336+0000","last_undegraded":"2023-10-08T00:10:54.099336+0000","last_fullsized":"2023-10-08T00:10:54.099336+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:49:50.543614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.091606+0000","last_change":"2023-10-08T00:10:54.091498+0000","last_active":"2023-10-08T00:10:54.091606+0000","last_peered":"2023-10-08T00:10:54.091606+0000","last_clean":"2023-10-08T00:10:54.091606+0000","last_became_active":"2023-10-08T00:10:54.090751+0000","last_became_peered":"2023-10-08T00:10:54.090751+0000","last_unstale":"2023-10-08T00:10:54.091606+0000","last_undegraded":"2023-10-08T00:10:54.091606+0000","last_fullsized":"2023-10-08T00:10:54.091606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:52:39.600888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.088854+0000","last_change":"2023-10-08T00:10:53.083078+0000","last_active":"2023-10-08T00:10:54.088854+0000","last_peered":"2023-10-08T00:10:54.088854+0000","last_clean":"2023-10-08T00:10:54.088854+0000","last_became_active":"2023-10-08T00:10:53.082687+0000","last_became_peered":"2023-10-08T00:10:53.082687+0000","last_unstale":"2023-10-08T00:10:54.088854+0000","last_undegraded":"2023-10-08T00:10:54.088854+0000","last_fullsized":"2023-10-08T00:10:54.088854+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:46:52.125296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.090972+0000","last_change":"2023-10-08T00:10:54.090572+0000","last_active":"2023-10-08T00:10:54.090972+0000","last_peered":"2023-10-08T00:10:54.090972+0000","last_clean":"2023-10-08T00:10:54.090972+0000","last_became_active":"2023-10-08T00:10:54.090352+0000","last_became_peered":"2023-10-08T00:10:54.090352+0000","last_unstale":"2023-10-08T00:10:54.090972+0000","last_undegraded":"2023-10-08T00:10:54.090972+0000","last_fullsized":"2023-10-08T00:10:54.090972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:52:09.003538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.085955+0000","last_change":"2023-10-08T00:10:53.084909+0000","last_active":"2023-10-08T00:10:54.085955+0000","last_peered":"2023-10-08T00:10:54.085955+0000","last_clean":"2023-10-08T00:10:54.085955+0000","last_became_active":"2023-10-08T00:10:53.084676+0000","last_became_peered":"2023-10-08T00:10:53.084676+0000","last_unstale":"2023-10-08T00:10:54.085955+0000","last_undegraded":"2023-10-08T00:10:54.085955+0000","last_fullsized":"2023-10-08T00:10:54.085955+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:08:39.785742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.103202+0000","last_change":"2023-10-08T00:10:54.098822+0000","last_active":"2023-10-08T00:10:54.103202+0000","last_peered":"2023-10-08T00:10:54.103202+0000","last_clean":"2023-10-08T00:10:54.103202+0000","last_became_active":"2023-10-08T00:10:54.098426+0000","last_became_peered":"2023-10-08T00:10:54.098426+0000","last_unstale":"2023-10-08T00:10:54.103202+0000","last_undegraded":"2023-10-08T00:10:54.103202+0000","last_fullsized":"2023-10-08T00:10:54.103202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:44:19.725645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.096443+0000","last_change":"2023-10-08T00:10:53.081675+0000","last_active":"2023-10-08T00:10:54.096443+0000","last_peered":"2023-10-08T00:10:54.096443+0000","last_clean":"2023-10-08T00:10:54.096443+0000","last_became_active":"2023-10-08T00:10:53.081494+0000","last_became_peered":"2023-10-08T00:10:53.081494+0000","last_unstale":"2023-10-08T00:10:54.096443+0000","last_undegraded":"2023-10-08T00:10:54.096443+0000","last_fullsized":"2023-10-08T00:10:54.096443+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:08:25.404396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.103000+0000","last_change":"2023-10-08T00:10:54.102951+0000","last_active":"2023-10-08T00:10:54.103000+0000","last_peered":"2023-10-08T00:10:54.103000+0000","last_clean":"2023-10-08T00:10:54.103000+0000","last_became_active":"2023-10-08T00:10:54.102742+0000","last_became_peered":"2023-10-08T00:10:54.102742+0000","last_unstale":"2023-10-08T00:10:54.103000+0000","last_undegraded":"2023-10-08T00:10:54.103000+0000","last_fullsized":"2023-10-08T00:10:54.103000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:40:12.384542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.100607+0000","last_change":"2023-10-08T00:10:53.074449+0000","last_active":"2023-10-08T00:10:54.100607+0000","last_peered":"2023-10-08T00:10:54.100607+0000","last_clean":"2023-10-08T00:10:54.100607+0000","last_became_active":"2023-10-08T00:10:53.074207+0000","last_became_peered":"2023-10-08T00:10:53.074207+0000","last_unstale":"2023-10-08T00:10:54.100607+0000","last_undegraded":"2023-10-08T00:10:54.100607+0000","last_fullsized":"2023-10-08T00:10:54.100607+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:46:52.125296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.093672+0000","last_change":"2023-10-08T00:10:54.093203+0000","last_active":"2023-10-08T00:10:54.093672+0000","last_peered":"2023-10-08T00:10:54.093672+0000","last_clean":"2023-10-08T00:10:54.093672+0000","last_became_active":"2023-10-08T00:10:54.093007+0000","last_became_peered":"2023-10-08T00:10:54.093007+0000","last_unstale":"2023-10-08T00:10:54.093672+0000","last_undegraded":"2023-10-08T00:10:54.093672+0000","last_fullsized":"2023-10-08T00:10:54.093672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:30:15.548694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.099362+0000","last_change":"2023-10-08T00:10:53.085980+0000","last_active":"2023-10-08T00:10:54.099362+0000","last_peered":"2023-10-08T00:10:54.099362+0000","last_clean":"2023-10-08T00:10:54.099362+0000","last_became_active":"2023-10-08T00:10:53.085796+0000","last_became_peered":"2023-10-08T00:10:53.085796+0000","last_unstale":"2023-10-08T00:10:54.099362+0000","last_undegraded":"2023-10-08T00:10:54.099362+0000","last_fullsized":"2023-10-08T00:10:54.099362+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:16:40.636514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.097850+0000","last_change":"2023-10-08T00:10:54.097790+0000","last_active":"2023-10-08T00:10:54.097850+0000","last_peered":"2023-10-08T00:10:54.097850+0000","last_clean":"2023-10-08T00:10:54.097850+0000","last_became_active":"2023-10-08T00:10:54.096497+0000","last_became_peered":"2023-10-08T00:10:54.096497+0000","last_unstale":"2023-10-08T00:10:54.097850+0000","last_undegraded":"2023-10-08T00:10:54.097850+0000","last_fullsized":"2023-10-08T00:10:54.097850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:05:53.331248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.097812+0000","last_change":"2023-10-08T00:10:53.078098+0000","last_active":"2023-10-08T00:10:54.097812+0000","last_peered":"2023-10-08T00:10:54.097812+0000","last_clean":"2023-10-08T00:10:54.097812+0000","last_became_active":"2023-10-08T00:10:53.077855+0000","last_became_peered":"2023-10-08T00:10:53.077855+0000","last_unstale":"2023-10-08T00:10:54.097812+0000","last_undegraded":"2023-10-08T00:10:54.097812+0000","last_fullsized":"2023-10-08T00:10:54.097812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:10:42.328030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.093527+0000","last_change":"2023-10-08T00:10:54.092335+0000","last_active":"2023-10-08T00:10:54.093527+0000","last_peered":"2023-10-08T00:10:54.093527+0000","last_clean":"2023-10-08T00:10:54.093527+0000","last_became_active":"2023-10-08T00:10:54.091475+0000","last_became_peered":"2023-10-08T00:10:54.091475+0000","last_unstale":"2023-10-08T00:10:54.093527+0000","last_undegraded":"2023-10-08T00:10:54.093527+0000","last_fullsized":"2023-10-08T00:10:54.093527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:30:15.548694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.098946+0000","last_change":"2023-10-08T00:10:53.079388+0000","last_active":"2023-10-08T00:10:54.098946+0000","last_peered":"2023-10-08T00:10:54.098946+0000","last_clean":"2023-10-08T00:10:54.098946+0000","last_became_active":"2023-10-08T00:10:53.079152+0000","last_became_peered":"2023-10-08T00:10:53.079152+0000","last_unstale":"2023-10-08T00:10:54.098946+0000","last_undegraded":"2023-10-08T00:10:54.098946+0000","last_fullsized":"2023-10-08T00:10:54.098946+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:08:25.404396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.090252+0000","last_change":"2023-10-08T00:10:54.088605+0000","last_active":"2023-10-08T00:10:54.090252+0000","last_peered":"2023-10-08T00:10:54.090252+0000","last_clean":"2023-10-08T00:10:54.090252+0000","last_became_active":"2023-10-08T00:10:54.088396+0000","last_became_peered":"2023-10-08T00:10:54.088396+0000","last_unstale":"2023-10-08T00:10:54.090252+0000","last_undegraded":"2023-10-08T00:10:54.090252+0000","last_fullsized":"2023-10-08T00:10:54.090252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:54:13.221147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.100688+0000","last_change":"2023-10-08T00:10:54.100120+0000","last_active":"2023-10-08T00:10:54.100688+0000","last_peered":"2023-10-08T00:10:54.100688+0000","last_clean":"2023-10-08T00:10:54.100688+0000","last_became_active":"2023-10-08T00:10:54.099910+0000","last_became_peered":"2023-10-08T00:10:54.099910+0000","last_unstale":"2023-10-08T00:10:54.100688+0000","last_undegraded":"2023-10-08T00:10:54.100688+0000","last_fullsized":"2023-10-08T00:10:54.100688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:11:05.861540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.094221+0000","last_change":"2023-10-08T00:10:54.093990+0000","last_active":"2023-10-08T00:10:54.094221+0000","last_peered":"2023-10-08T00:10:54.094221+0000","last_clean":"2023-10-08T00:10:54.094221+0000","last_became_active":"2023-10-08T00:10:54.093268+0000","last_became_peered":"2023-10-08T00:10:54.093268+0000","last_unstale":"2023-10-08T00:10:54.094221+0000","last_undegraded":"2023-10-08T00:10:54.094221+0000","last_fullsized":"2023-10-08T00:10:54.094221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:49:51.549111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.096510+0000","last_change":"2023-10-08T00:10:54.096449+0000","last_active":"2023-10-08T00:10:54.096510+0000","last_peered":"2023-10-08T00:10:54.096510+0000","last_clean":"2023-10-08T00:10:54.096510+0000","last_became_active":"2023-10-08T00:10:54.096216+0000","last_became_peered":"2023-10-08T00:10:54.096216+0000","last_unstale":"2023-10-08T00:10:54.096510+0000","last_undegraded":"2023-10-08T00:10:54.096510+0000","last_fullsized":"2023-10-08T00:10:54.096510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:11:41.906007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.093706+0000","last_change":"2023-10-08T00:10:54.091529+0000","last_active":"2023-10-08T00:10:54.093706+0000","last_peered":"2023-10-08T00:10:54.093706+0000","last_clean":"2023-10-08T00:10:54.093706+0000","last_became_active":"2023-10-08T00:10:54.090949+0000","last_became_peered":"2023-10-08T00:10:54.090949+0000","last_unstale":"2023-10-08T00:10:54.093706+0000","last_undegraded":"2023-10-08T00:10:54.093706+0000","last_fullsized":"2023-10-08T00:10:54.093706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:52:44.075366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.101359+0000","last_change":"2023-10-08T00:10:54.100755+0000","last_active":"2023-10-08T00:10:54.101359+0000","last_peered":"2023-10-08T00:10:54.101359+0000","last_clean":"2023-10-08T00:10:54.101359+0000","last_became_active":"2023-10-08T00:10:54.100206+0000","last_became_peered":"2023-10-08T00:10:54.100206+0000","last_unstale":"2023-10-08T00:10:54.101359+0000","last_undegraded":"2023-10-08T00:10:54.101359+0000","last_fullsized":"2023-10-08T00:10:54.101359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:03:42.084791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.101376+0000","last_change":"2023-10-08T00:10:54.100759+0000","last_active":"2023-10-08T00:10:54.101376+0000","last_peered":"2023-10-08T00:10:54.101376+0000","last_clean":"2023-10-08T00:10:54.101376+0000","last_became_active":"2023-10-08T00:10:54.100365+0000","last_became_peered":"2023-10-08T00:10:54.100365+0000","last_unstale":"2023-10-08T00:10:54.101376+0000","last_undegraded":"2023-10-08T00:10:54.101376+0000","last_fullsized":"2023-10-08T00:10:54.101376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:40:12.384542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.087188+0000","last_change":"2023-10-08T00:10:54.085760+0000","last_active":"2023-10-08T00:10:54.087188+0000","last_peered":"2023-10-08T00:10:54.087188+0000","last_clean":"2023-10-08T00:10:54.087188+0000","last_became_active":"2023-10-08T00:10:54.085544+0000","last_became_peered":"2023-10-08T00:10:54.085544+0000","last_unstale":"2023-10-08T00:10:54.087188+0000","last_undegraded":"2023-10-08T00:10:54.087188+0000","last_fullsized":"2023-10-08T00:10:54.087188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:53:06.721490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.099558+0000","last_change":"2023-10-08T00:10:54.095621+0000","last_active":"2023-10-08T00:10:54.099558+0000","last_peered":"2023-10-08T00:10:54.099558+0000","last_clean":"2023-10-08T00:10:54.099558+0000","last_became_active":"2023-10-08T00:10:54.095240+0000","last_became_peered":"2023-10-08T00:10:54.095240+0000","last_unstale":"2023-10-08T00:10:54.099558+0000","last_undegraded":"2023-10-08T00:10:54.099558+0000","last_fullsized":"2023-10-08T00:10:54.099558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:22:42.484817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.101130+0000","last_change":"2023-10-08T00:10:54.101076+0000","last_active":"2023-10-08T00:10:54.101130+0000","last_peered":"2023-10-08T00:10:54.101130+0000","last_clean":"2023-10-08T00:10:54.101130+0000","last_became_active":"2023-10-08T00:10:54.100508+0000","last_became_peered":"2023-10-08T00:10:54.100508+0000","last_unstale":"2023-10-08T00:10:54.101130+0000","last_undegraded":"2023-10-08T00:10:54.101130+0000","last_fullsized":"2023-10-08T00:10:54.101130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:18:09.014594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.100641+0000","last_change":"2023-10-08T00:10:54.100508+0000","last_active":"2023-10-08T00:10:54.100641+0000","last_peered":"2023-10-08T00:10:54.100641+0000","last_clean":"2023-10-08T00:10:54.100641+0000","last_became_active":"2023-10-08T00:10:54.100272+0000","last_became_peered":"2023-10-08T00:10:54.100272+0000","last_unstale":"2023-10-08T00:10:54.100641+0000","last_undegraded":"2023-10-08T00:10:54.100641+0000","last_fullsized":"2023-10-08T00:10:54.100641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:52:09.003538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.095807+0000","last_change":"2023-10-08T00:10:54.095683+0000","last_active":"2023-10-08T00:10:54.095807+0000","last_peered":"2023-10-08T00:10:54.095807+0000","last_clean":"2023-10-08T00:10:54.095807+0000","last_became_active":"2023-10-08T00:10:54.095474+0000","last_became_peered":"2023-10-08T00:10:54.095474+0000","last_unstale":"2023-10-08T00:10:54.095807+0000","last_undegraded":"2023-10-08T00:10:54.095807+0000","last_fullsized":"2023-10-08T00:10:54.095807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:11:05.861540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.092456+0000","last_change":"2023-10-08T00:10:54.092383+0000","last_active":"2023-10-08T00:10:54.092456+0000","last_peered":"2023-10-08T00:10:54.092456+0000","last_clean":"2023-10-08T00:10:54.092456+0000","last_became_active":"2023-10-08T00:10:54.092098+0000","last_became_peered":"2023-10-08T00:10:54.092098+0000","last_unstale":"2023-10-08T00:10:54.092456+0000","last_undegraded":"2023-10-08T00:10:54.092456+0000","last_fullsized":"2023-10-08T00:10:54.092456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:54:13.221147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.098417+0000","last_change":"2023-10-08T00:10:54.097569+0000","last_active":"2023-10-08T00:10:54.098417+0000","last_peered":"2023-10-08T00:10:54.098417+0000","last_clean":"2023-10-08T00:10:54.098417+0000","last_became_active":"2023-10-08T00:10:54.096963+0000","last_became_peered":"2023-10-08T00:10:54.096963+0000","last_unstale":"2023-10-08T00:10:54.098417+0000","last_undegraded":"2023-10-08T00:10:54.098417+0000","last_fullsized":"2023-10-08T00:10:54.098417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:52:09.003538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":68,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929382,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":68,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929506,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":645,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194667,"statfs":{"total":200000000,"available":199339068,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.315}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":35,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":68,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929450,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.221}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28100000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":70,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195242,"statfs":{"total":200000000,"available":199927970,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199928952,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":70,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195241,"statfs":{"total":200000000,"available":199927326,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":645,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194667,"statfs":{"total":200000000,"available":199339046,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]}]}],"pool_statfs":[]}} 2023-10-08T00:11:07.062 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T00:11:07.212 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.211+0000 7f6f0c06f700 1 Processor -- start 2023-10-08T00:11:07.212 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.212+0000 7f6f0c06f700 1 -- start start 2023-10-08T00:11:07.219 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.212+0000 7f6f0c06f700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f6f040b0e50 0x7f6f040c9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:07.219 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.212+0000 7f6f0c06f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f040b1220 0x7f6f0406d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:07.219 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.212+0000 7f6f0c06f700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f0415de60 con 0x7f6f0406dad0 2023-10-08T00:11:07.219 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.212+0000 7f6f0c06f700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f0415e140 con 0x7f6f040b0e50 2023-10-08T00:11:07.219 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.212+0000 7f6f0c06f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0415e420 con 0x7f6f040b1220 2023-10-08T00:11:07.219 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.212+0000 7f6f0a60c700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f6f0406dad0 0x7f6f040cb4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45112/0 (socket says 172.21.15.99:45112) 2023-10-08T00:11:07.219 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.212+0000 7f6f0a60c700 1 -- 172.21.15.99:0/546502233 learned_addr learned my addr 172.21.15.99:0/546502233 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:07.219 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.213+0000 7f6f09e0b700 1 --2- 172.21.15.99:0/546502233 >> v2:172.21.15.191:3300/0 conn(0x7f6f040b0e50 0x7f6f040c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:07.220 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.213+0000 7f6f0960a700 1 --2- 172.21.15.99:0/546502233 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f040b1220 0x7f6f0406d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:07.220 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.213+0000 7f6f08e09700 1 -- 172.21.15.99:0/546502233 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 440529085 0 0) 0x7f6f0415de60 con 0x7f6f0406dad0 2023-10-08T00:11:07.220 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.213+0000 7f6f08e09700 1 -- 172.21.15.99:0/546502233 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ef0003570 con 0x7f6f0406dad0 2023-10-08T00:11:07.220 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.213+0000 7f6f08e09700 1 -- 172.21.15.99:0/546502233 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3372635832 0 0) 0x7f6ef0003570 con 0x7f6f0406dad0 2023-10-08T00:11:07.220 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.213+0000 7f6f08e09700 1 -- 172.21.15.99:0/546502233 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f040b1220 msgr2=0x7f6f0406d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:07.220 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.213+0000 7f6f08e09700 1 --2- 172.21.15.99:0/546502233 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f040b1220 0x7f6f0406d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:07.220 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.213+0000 7f6f08e09700 1 -- 172.21.15.99:0/546502233 >> v2:172.21.15.191:3300/0 conn(0x7f6f040b0e50 msgr2=0x7f6f040c9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:07.220 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.213+0000 7f6f08e09700 1 --2- 172.21.15.99:0/546502233 >> v2:172.21.15.191:3300/0 conn(0x7f6f040b0e50 0x7f6f040c9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:07.220 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.213+0000 7f6f08e09700 1 -- 172.21.15.99:0/546502233 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f0415e700 con 0x7f6f0406dad0 2023-10-08T00:11:07.220 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.213+0000 7f6f08e09700 1 -- 172.21.15.99:0/546502233 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f6efc0017f0 con 0x7f6f0406dad0 2023-10-08T00:11:07.220 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.213+0000 7f6f08e09700 1 -- 172.21.15.99:0/546502233 >> v1:172.21.15.99:6789/0 conn(0x7f6f0406dad0 legacy=0x7f6f040cb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:07.221 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.213+0000 7f6f08e09700 1 --2- 172.21.15.99:0/546502233 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ef0003a60 0x7f6ef0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:07.221 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.213+0000 7f6f08e09700 1 --2- 172.21.15.99:0/546502233 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6ef0004810 0x7f6ef0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:07.221 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.213+0000 7f6f08e09700 1 --2- 172.21.15.99:0/546502233 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f040b1220 0x7f6ef00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:07.221 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.213+0000 7f6f08e09700 1 -- 172.21.15.99:0/546502233 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0415e140 con 0x7f6ef0003a60 2023-10-08T00:11:07.221 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.213+0000 7f6f08e09700 1 -- 172.21.15.99:0/546502233 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0415e420 con 0x7f6ef0004810 2023-10-08T00:11:07.221 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.213+0000 7f6f08e09700 1 -- 172.21.15.99:0/546502233 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef000bb10 con 0x7f6f040b1220 2023-10-08T00:11:07.221 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.213+0000 7f6f09e0b700 1 --2- 172.21.15.99:0/546502233 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ef0003a60 0x7f6ef0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:07.221 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.213+0000 7f6f0960a700 1 --2- 172.21.15.99:0/546502233 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6ef0004810 0x7f6ef0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:07.221 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.213+0000 7f6f0a60c700 1 --2- 172.21.15.99:0/546502233 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f040b1220 0x7f6ef00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:07.221 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.213+0000 7f6f09e0b700 1 -- 172.21.15.99:0/546502233 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f040b1220 msgr2=0x7f6ef00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:07.221 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.213+0000 7f6f09e0b700 1 --2- 172.21.15.99:0/546502233 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f040b1220 0x7f6ef00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:07.221 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.213+0000 7f6f09e0b700 1 -- 172.21.15.99:0/546502233 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6ef0004810 msgr2=0x7f6ef0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:07.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.213+0000 7f6f09e0b700 1 --2- 172.21.15.99:0/546502233 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6ef0004810 0x7f6ef0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:07.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.213+0000 7f6f09e0b700 1 -- 172.21.15.99:0/546502233 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef0003570 con 0x7f6ef0003a60 2023-10-08T00:11:07.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.213+0000 7f6f0a60c700 1 --2- 172.21.15.99:0/546502233 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f040b1220 0x7f6ef00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:11:07.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.214+0000 7f6f09e0b700 1 --2- 172.21.15.99:0/546502233 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ef0003a60 0x7f6ef0003e70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6f00009c00 tx=0x7f6f000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55105d94f69fa9ec server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:07.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.214+0000 7f6f08e09700 1 -- 172.21.15.99:0/546502233 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0000e710 con 0x7f6ef0003a60 2023-10-08T00:11:07.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.214+0000 7f6f08e09700 1 -- 172.21.15.99:0/546502233 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f0000e870 con 0x7f6ef0003a60 2023-10-08T00:11:07.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.214+0000 7f6f08e09700 1 -- 172.21.15.99:0/546502233 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0000ec30 con 0x7f6ef0003a60 2023-10-08T00:11:07.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.214+0000 7f6f0c06f700 1 -- 172.21.15.99:0/546502233 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ef0003a60 msgr2=0x7f6ef0003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:07.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.214+0000 7f6f0c06f700 1 --2- 172.21.15.99:0/546502233 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ef0003a60 0x7f6ef0003e70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6f00009c00 tx=0x7f6f000091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:07.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.214+0000 7f6f0a60c700 1 -- 172.21.15.99:0/546502233 reap_dead start 2023-10-08T00:11:07.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.214+0000 7f6f0c06f700 1 -- 172.21.15.99:0/546502233 shutdown_connections 2023-10-08T00:11:07.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.214+0000 7f6f0c06f700 1 -- 172.21.15.99:0/546502233 >> 172.21.15.99:0/546502233 conn(0x7f6f04069870 msgr2=0x7f6f04069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:07.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.214+0000 7f6f0c06f700 1 -- 172.21.15.99:0/546502233 shutdown_connections 2023-10-08T00:11:07.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.214+0000 7f6f0c06f700 1 -- 172.21.15.99:0/546502233 wait complete. 2023-10-08T00:11:07.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.215+0000 7f6f0c06f700 1 Processor -- start 2023-10-08T00:11:07.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.215+0000 7f6f0c06f700 1 -- start start 2023-10-08T00:11:07.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.216+0000 7f6f0c06f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6ef0003a60 0x7f6f040b3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:07.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.216+0000 7f6f0c06f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f04154510 0x7f6f04157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:07.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.216+0000 7f6f0c06f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f04157c50 0x7f6f040b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:07.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.216+0000 7f6f0c06f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f041580f0 con 0x7f6f04157c50 2023-10-08T00:11:07.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.216+0000 7f6f0c06f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0415c0b0 con 0x7f6f04154510 2023-10-08T00:11:07.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.216+0000 7f6f0c06f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0415b010 con 0x7f6ef0003a60 2023-10-08T00:11:07.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.216+0000 7f6f0a60c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f04157c50 0x7f6f040b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:07.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.216+0000 7f6f0a60c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f04157c50 0x7f6f040b18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38908/0 (socket says 172.21.15.99:38908) 2023-10-08T00:11:07.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.216+0000 7f6f0a60c700 1 -- 172.21.15.99:0/2294035475 learned_addr learned my addr 172.21.15.99:0/2294035475 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:07.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.216+0000 7f6f0960a700 1 --2- 172.21.15.99:0/2294035475 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f04154510 0x7f6f04157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:07.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.216+0000 7f6f09e0b700 1 --2- 172.21.15.99:0/2294035475 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6ef0003a60 0x7f6f040b3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:07.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.216+0000 7f6f0a60c700 1 -- 172.21.15.99:0/2294035475 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6ef0003a60 msgr2=0x7f6f040b3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:07.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.216+0000 7f6f0a60c700 1 --2- 172.21.15.99:0/2294035475 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6ef0003a60 0x7f6f040b3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:07.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.216+0000 7f6f0a60c700 1 -- 172.21.15.99:0/2294035475 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f04154510 msgr2=0x7f6f04157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:07.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.216+0000 7f6f0a60c700 1 --2- 172.21.15.99:0/2294035475 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f04154510 0x7f6f04157710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:07.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.216+0000 7f6f0a60c700 1 -- 172.21.15.99:0/2294035475 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f0415b2f0 con 0x7f6f04157c50 2023-10-08T00:11:07.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.217+0000 7f6f0a60c700 1 --2- 172.21.15.99:0/2294035475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f04157c50 0x7f6f040b18e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6efc000f80 tx=0x7f6f040cd4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:07.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.217+0000 7f6ef9ffb700 1 -- 172.21.15.99:0/2294035475 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6efc00eab0 con 0x7f6f04157c50 2023-10-08T00:11:07.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.217+0000 7f6ef9ffb700 1 -- 172.21.15.99:0/2294035475 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6efc00ec30 con 0x7f6f04157c50 2023-10-08T00:11:07.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.217+0000 7f6f0c06f700 1 -- 172.21.15.99:0/2294035475 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f0415b5d0 con 0x7f6f04157c50 2023-10-08T00:11:07.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.217+0000 7f6ef9ffb700 1 -- 172.21.15.99:0/2294035475 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6efc013420 con 0x7f6f04157c50 2023-10-08T00:11:07.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.217+0000 7f6f0c06f700 1 -- 172.21.15.99:0/2294035475 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f0415bb90 con 0x7f6f04157c50 2023-10-08T00:11:07.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.218+0000 7f6f0c06f700 1 -- 172.21.15.99:0/2294035475 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f0415bd80 con 0x7f6f04157c50 2023-10-08T00:11:07.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.218+0000 7f6ef9ffb700 1 -- 172.21.15.99:0/2294035475 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6efc0137d0 con 0x7f6f04157c50 2023-10-08T00:11:07.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.218+0000 7f6ef9ffb700 1 --2- 172.21.15.99:0/2294035475 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6ee8066f30 0x7f6ee80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:07.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.218+0000 7f6ef9ffb700 1 -- 172.21.15.99:0/2294035475 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6efc084970 con 0x7f6f04157c50 2023-10-08T00:11:07.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.219+0000 7f6f09e0b700 1 --2- 172.21.15.99:0/2294035475 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6ee8066f30 0x7f6ee80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:07.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.219+0000 7f6f09e0b700 1 --2- 172.21.15.99:0/2294035475 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6ee8066f30 0x7f6ee80693e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6f000091e0 tx=0x7f6f0001a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:07.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.223+0000 7f6ef9ffb700 1 -- 172.21.15.99:0/2294035475 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6efc051be0 con 0x7f6f04157c50 2023-10-08T00:11:07.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.403+0000 7f6f0c06f700 1 -- 172.21.15.99:0/2294035475 --> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6f0404c3c0 con 0x7f6ee8066f30 2023-10-08T00:11:07.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.410+0000 7f6ef9ffb700 1 -- 172.21.15.99:0/2294035475 <== mgr.4099 v2:172.21.15.191:6832/85985 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+260191 (secure 0 0 0) 0x7f6f0404c3c0 con 0x7f6ee8066f30 2023-10-08T00:11:07.415 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:11:07.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.413+0000 7f6f0c06f700 1 -- 172.21.15.99:0/2294035475 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6ee8066f30 msgr2=0x7f6ee80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:07.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.413+0000 7f6f0c06f700 1 --2- 172.21.15.99:0/2294035475 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6ee8066f30 0x7f6ee80693e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6f000091e0 tx=0x7f6f0001a000 comp rx=0 tx=0).stop 2023-10-08T00:11:07.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.413+0000 7f6f0c06f700 1 -- 172.21.15.99:0/2294035475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f04157c50 msgr2=0x7f6f040b18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:07.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.413+0000 7f6f0c06f700 1 --2- 172.21.15.99:0/2294035475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f04157c50 0x7f6f040b18e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6efc000f80 tx=0x7f6f040cd4a0 comp rx=0 tx=0).stop 2023-10-08T00:11:07.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.414+0000 7f6f0c06f700 1 -- 172.21.15.99:0/2294035475 shutdown_connections 2023-10-08T00:11:07.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.414+0000 7f6f0c06f700 1 --2- 172.21.15.99:0/2294035475 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6ee8066f30 0x7f6ee80693e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:07.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.414+0000 7f6f0c06f700 1 --2- 172.21.15.99:0/2294035475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f04157c50 0x7f6f040b18e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:07.418 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.414+0000 7f6f0c06f700 1 --2- 172.21.15.99:0/2294035475 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6ef0003a60 0x7f6f040b3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:07.418 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.414+0000 7f6f0c06f700 1 --2- 172.21.15.99:0/2294035475 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f04154510 0x7f6f04157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:07.418 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.414+0000 7f6f0c06f700 1 -- 172.21.15.99:0/2294035475 >> 172.21.15.99:0/2294035475 conn(0x7f6f04069870 msgr2=0x7f6f04068270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:07.418 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.414+0000 7f6f0c06f700 1 -- 172.21.15.99:0/2294035475 shutdown_connections 2023-10-08T00:11:07.418 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.414+0000 7f6f0c06f700 1 -- 172.21.15.99:0/2294035475 wait complete. 2023-10-08T00:11:07.418 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2023-10-08T00:11:07.426 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-08T00:11:06.026880+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1708,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560791,"statfs":{"total":1600000000,"available":1598250700,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.943944"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.095544+0000","last_change":"2023-10-08T00:10:54.094801+0000","last_active":"2023-10-08T00:10:54.095544+0000","last_peered":"2023-10-08T00:10:54.095544+0000","last_clean":"2023-10-08T00:10:54.095544+0000","last_became_active":"2023-10-08T00:10:54.094221+0000","last_became_peered":"2023-10-08T00:10:54.094221+0000","last_unstale":"2023-10-08T00:10:54.095544+0000","last_undegraded":"2023-10-08T00:10:54.095544+0000","last_fullsized":"2023-10-08T00:10:54.095544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:30:37.085534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.102469+0000","last_change":"2023-10-08T00:10:54.101252+0000","last_active":"2023-10-08T00:10:54.102469+0000","last_peered":"2023-10-08T00:10:54.102469+0000","last_clean":"2023-10-08T00:10:54.102469+0000","last_became_active":"2023-10-08T00:10:54.101016+0000","last_became_peered":"2023-10-08T00:10:54.101016+0000","last_unstale":"2023-10-08T00:10:54.102469+0000","last_undegraded":"2023-10-08T00:10:54.102469+0000","last_fullsized":"2023-10-08T00:10:54.102469+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:30:15.548694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.104771+0000","last_change":"2023-10-08T00:10:54.104705+0000","last_active":"2023-10-08T00:10:54.104771+0000","last_peered":"2023-10-08T00:10:54.104771+0000","last_clean":"2023-10-08T00:10:54.104771+0000","last_became_active":"2023-10-08T00:10:54.104469+0000","last_became_peered":"2023-10-08T00:10:54.104469+0000","last_unstale":"2023-10-08T00:10:54.104771+0000","last_undegraded":"2023-10-08T00:10:54.104771+0000","last_fullsized":"2023-10-08T00:10:54.104771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:27:57.768735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.090488+0000","last_change":"2023-10-08T00:10:54.089800+0000","last_active":"2023-10-08T00:10:54.090488+0000","last_peered":"2023-10-08T00:10:54.090488+0000","last_clean":"2023-10-08T00:10:54.090488+0000","last_became_active":"2023-10-08T00:10:54.089200+0000","last_became_peered":"2023-10-08T00:10:54.089200+0000","last_unstale":"2023-10-08T00:10:54.090488+0000","last_undegraded":"2023-10-08T00:10:54.090488+0000","last_fullsized":"2023-10-08T00:10:54.090488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:44:19.725645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.099476+0000","last_change":"2023-10-08T00:10:54.099417+0000","last_active":"2023-10-08T00:10:54.099476+0000","last_peered":"2023-10-08T00:10:54.099476+0000","last_clean":"2023-10-08T00:10:54.099476+0000","last_became_active":"2023-10-08T00:10:54.099060+0000","last_became_peered":"2023-10-08T00:10:54.099060+0000","last_unstale":"2023-10-08T00:10:54.099476+0000","last_undegraded":"2023-10-08T00:10:54.099476+0000","last_fullsized":"2023-10-08T00:10:54.099476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:20:35.671993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.088556+0000","last_change":"2023-10-08T00:10:54.088499+0000","last_active":"2023-10-08T00:10:54.088556+0000","last_peered":"2023-10-08T00:10:54.088556+0000","last_clean":"2023-10-08T00:10:54.088556+0000","last_became_active":"2023-10-08T00:10:54.088247+0000","last_became_peered":"2023-10-08T00:10:54.088247+0000","last_unstale":"2023-10-08T00:10:54.088556+0000","last_undegraded":"2023-10-08T00:10:54.088556+0000","last_fullsized":"2023-10-08T00:10:54.088556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:49:51.549111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.098443+0000","last_change":"2023-10-08T00:10:54.098398+0000","last_active":"2023-10-08T00:10:54.098443+0000","last_peered":"2023-10-08T00:10:54.098443+0000","last_clean":"2023-10-08T00:10:54.098443+0000","last_became_active":"2023-10-08T00:10:54.098065+0000","last_became_peered":"2023-10-08T00:10:54.098065+0000","last_unstale":"2023-10-08T00:10:54.098443+0000","last_undegraded":"2023-10-08T00:10:54.098443+0000","last_fullsized":"2023-10-08T00:10:54.098443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:05:53.331248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.099579+0000","last_change":"2023-10-08T00:10:54.098754+0000","last_active":"2023-10-08T00:10:54.099579+0000","last_peered":"2023-10-08T00:10:54.099579+0000","last_clean":"2023-10-08T00:10:54.099579+0000","last_became_active":"2023-10-08T00:10:54.098505+0000","last_became_peered":"2023-10-08T00:10:54.098505+0000","last_unstale":"2023-10-08T00:10:54.099579+0000","last_undegraded":"2023-10-08T00:10:54.099579+0000","last_fullsized":"2023-10-08T00:10:54.099579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:20:35.671993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.096851+0000","last_change":"2023-10-08T00:10:54.096792+0000","last_active":"2023-10-08T00:10:54.096851+0000","last_peered":"2023-10-08T00:10:54.096851+0000","last_clean":"2023-10-08T00:10:54.096851+0000","last_became_active":"2023-10-08T00:10:54.096569+0000","last_became_peered":"2023-10-08T00:10:54.096569+0000","last_unstale":"2023-10-08T00:10:54.096851+0000","last_undegraded":"2023-10-08T00:10:54.096851+0000","last_fullsized":"2023-10-08T00:10:54.096851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:40:12.384542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.093509+0000","last_change":"2023-10-08T00:10:54.092144+0000","last_active":"2023-10-08T00:10:54.093509+0000","last_peered":"2023-10-08T00:10:54.093509+0000","last_clean":"2023-10-08T00:10:54.093509+0000","last_became_active":"2023-10-08T00:10:54.091938+0000","last_became_peered":"2023-10-08T00:10:54.091938+0000","last_unstale":"2023-10-08T00:10:54.093509+0000","last_undegraded":"2023-10-08T00:10:54.093509+0000","last_fullsized":"2023-10-08T00:10:54.093509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:13:19.313381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.095525+0000","last_change":"2023-10-08T00:10:54.094651+0000","last_active":"2023-10-08T00:10:54.095525+0000","last_peered":"2023-10-08T00:10:54.095525+0000","last_clean":"2023-10-08T00:10:54.095525+0000","last_became_active":"2023-10-08T00:10:54.093973+0000","last_became_peered":"2023-10-08T00:10:54.093973+0000","last_unstale":"2023-10-08T00:10:54.095525+0000","last_undegraded":"2023-10-08T00:10:54.095525+0000","last_fullsized":"2023-10-08T00:10:54.095525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:27:57.768735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.093543+0000","last_change":"2023-10-08T00:10:54.092456+0000","last_active":"2023-10-08T00:10:54.093543+0000","last_peered":"2023-10-08T00:10:54.093543+0000","last_clean":"2023-10-08T00:10:54.093543+0000","last_became_active":"2023-10-08T00:10:54.092121+0000","last_became_peered":"2023-10-08T00:10:54.092121+0000","last_unstale":"2023-10-08T00:10:54.093543+0000","last_undegraded":"2023-10-08T00:10:54.093543+0000","last_fullsized":"2023-10-08T00:10:54.093543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:17:33.566050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.102806+0000","last_change":"2023-10-08T00:10:54.102717+0000","last_active":"2023-10-08T00:10:54.102806+0000","last_peered":"2023-10-08T00:10:54.102806+0000","last_clean":"2023-10-08T00:10:54.102806+0000","last_became_active":"2023-10-08T00:10:54.102373+0000","last_became_peered":"2023-10-08T00:10:54.102373+0000","last_unstale":"2023-10-08T00:10:54.102806+0000","last_undegraded":"2023-10-08T00:10:54.102806+0000","last_fullsized":"2023-10-08T00:10:54.102806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:20:26.989181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.103024+0000","last_change":"2023-10-08T00:10:54.102984+0000","last_active":"2023-10-08T00:10:54.103024+0000","last_peered":"2023-10-08T00:10:54.103024+0000","last_clean":"2023-10-08T00:10:54.103024+0000","last_became_active":"2023-10-08T00:10:54.102645+0000","last_became_peered":"2023-10-08T00:10:54.102645+0000","last_unstale":"2023-10-08T00:10:54.103024+0000","last_undegraded":"2023-10-08T00:10:54.103024+0000","last_fullsized":"2023-10-08T00:10:54.103024+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:44:19.725645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.100866+0000","last_change":"2023-10-08T00:10:54.100800+0000","last_active":"2023-10-08T00:10:54.100866+0000","last_peered":"2023-10-08T00:10:54.100866+0000","last_clean":"2023-10-08T00:10:54.100866+0000","last_became_active":"2023-10-08T00:10:54.100384+0000","last_became_peered":"2023-10-08T00:10:54.100384+0000","last_unstale":"2023-10-08T00:10:54.100866+0000","last_undegraded":"2023-10-08T00:10:54.100866+0000","last_fullsized":"2023-10-08T00:10:54.100866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:05:53.331248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.098456+0000","last_change":"2023-10-08T00:10:54.097988+0000","last_active":"2023-10-08T00:10:54.098456+0000","last_peered":"2023-10-08T00:10:54.098456+0000","last_clean":"2023-10-08T00:10:54.098456+0000","last_became_active":"2023-10-08T00:10:54.097155+0000","last_became_peered":"2023-10-08T00:10:54.097155+0000","last_unstale":"2023-10-08T00:10:54.098456+0000","last_undegraded":"2023-10-08T00:10:54.098456+0000","last_fullsized":"2023-10-08T00:10:54.098456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:03:51.083770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.093657+0000","last_change":"2023-10-08T00:10:54.093109+0000","last_active":"2023-10-08T00:10:54.093657+0000","last_peered":"2023-10-08T00:10:54.093657+0000","last_clean":"2023-10-08T00:10:54.093657+0000","last_became_active":"2023-10-08T00:10:54.092788+0000","last_became_peered":"2023-10-08T00:10:54.092788+0000","last_unstale":"2023-10-08T00:10:54.093657+0000","last_undegraded":"2023-10-08T00:10:54.093657+0000","last_fullsized":"2023-10-08T00:10:54.093657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:37:23.781764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.085610+0000","last_change":"2023-10-08T00:10:54.085533+0000","last_active":"2023-10-08T00:10:54.085610+0000","last_peered":"2023-10-08T00:10:54.085610+0000","last_clean":"2023-10-08T00:10:54.085610+0000","last_became_active":"2023-10-08T00:10:54.085297+0000","last_became_peered":"2023-10-08T00:10:54.085297+0000","last_unstale":"2023-10-08T00:10:54.085610+0000","last_undegraded":"2023-10-08T00:10:54.085610+0000","last_fullsized":"2023-10-08T00:10:54.085610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:42:35.332320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.093626+0000","last_change":"2023-10-08T00:10:54.092802+0000","last_active":"2023-10-08T00:10:54.093626+0000","last_peered":"2023-10-08T00:10:54.093626+0000","last_clean":"2023-10-08T00:10:54.093626+0000","last_became_active":"2023-10-08T00:10:54.092217+0000","last_became_peered":"2023-10-08T00:10:54.092217+0000","last_unstale":"2023-10-08T00:10:54.093626+0000","last_undegraded":"2023-10-08T00:10:54.093626+0000","last_fullsized":"2023-10-08T00:10:54.093626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:20:35.671993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.103423+0000","last_change":"2023-10-08T00:10:53.083367+0000","last_active":"2023-10-08T00:10:54.103423+0000","last_peered":"2023-10-08T00:10:54.103423+0000","last_clean":"2023-10-08T00:10:54.103423+0000","last_became_active":"2023-10-08T00:10:53.083127+0000","last_became_peered":"2023-10-08T00:10:53.083127+0000","last_unstale":"2023-10-08T00:10:54.103423+0000","last_undegraded":"2023-10-08T00:10:54.103423+0000","last_fullsized":"2023-10-08T00:10:54.103423+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:10:42.328030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.099821+0000","last_change":"2023-10-08T00:10:54.099725+0000","last_active":"2023-10-08T00:10:54.099821+0000","last_peered":"2023-10-08T00:10:54.099821+0000","last_clean":"2023-10-08T00:10:54.099821+0000","last_became_active":"2023-10-08T00:10:54.099236+0000","last_became_peered":"2023-10-08T00:10:54.099236+0000","last_unstale":"2023-10-08T00:10:54.099821+0000","last_undegraded":"2023-10-08T00:10:54.099821+0000","last_fullsized":"2023-10-08T00:10:54.099821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:13:19.313381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.091381+0000","last_change":"2023-10-08T00:10:53.078523+0000","last_active":"2023-10-08T00:10:54.091381+0000","last_peered":"2023-10-08T00:10:54.091381+0000","last_clean":"2023-10-08T00:10:54.091381+0000","last_became_active":"2023-10-08T00:10:53.078068+0000","last_became_peered":"2023-10-08T00:10:53.078068+0000","last_unstale":"2023-10-08T00:10:54.091381+0000","last_undegraded":"2023-10-08T00:10:54.091381+0000","last_fullsized":"2023-10-08T00:10:54.091381+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:30:06.828965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.094722+0000","last_change":"2023-10-08T00:10:54.094485+0000","last_active":"2023-10-08T00:10:54.094722+0000","last_peered":"2023-10-08T00:10:54.094722+0000","last_clean":"2023-10-08T00:10:54.094722+0000","last_became_active":"2023-10-08T00:10:54.094246+0000","last_became_peered":"2023-10-08T00:10:54.094246+0000","last_unstale":"2023-10-08T00:10:54.094722+0000","last_undegraded":"2023-10-08T00:10:54.094722+0000","last_fullsized":"2023-10-08T00:10:54.094722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:52:39.600888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.099727+0000","last_change":"2023-10-08T00:10:53.086334+0000","last_active":"2023-10-08T00:10:54.099727+0000","last_peered":"2023-10-08T00:10:54.099727+0000","last_clean":"2023-10-08T00:10:54.099727+0000","last_became_active":"2023-10-08T00:10:53.084152+0000","last_became_peered":"2023-10-08T00:10:53.084152+0000","last_unstale":"2023-10-08T00:10:54.099727+0000","last_undegraded":"2023-10-08T00:10:54.099727+0000","last_fullsized":"2023-10-08T00:10:54.099727+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:08:25.404396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.095590+0000","last_change":"2023-10-08T00:10:54.095225+0000","last_active":"2023-10-08T00:10:54.095590+0000","last_peered":"2023-10-08T00:10:54.095590+0000","last_clean":"2023-10-08T00:10:54.095590+0000","last_became_active":"2023-10-08T00:10:54.095005+0000","last_became_peered":"2023-10-08T00:10:54.095005+0000","last_unstale":"2023-10-08T00:10:54.095590+0000","last_undegraded":"2023-10-08T00:10:54.095590+0000","last_fullsized":"2023-10-08T00:10:54.095590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:49:51.549111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.099165+0000","last_change":"2023-10-08T00:10:53.082935+0000","last_active":"2023-10-08T00:10:54.099165+0000","last_peered":"2023-10-08T00:10:54.099165+0000","last_clean":"2023-10-08T00:10:54.099165+0000","last_became_active":"2023-10-08T00:10:53.082451+0000","last_became_peered":"2023-10-08T00:10:53.082451+0000","last_unstale":"2023-10-08T00:10:54.099165+0000","last_undegraded":"2023-10-08T00:10:54.099165+0000","last_fullsized":"2023-10-08T00:10:54.099165+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:30:06.828965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.093599+0000","last_change":"2023-10-08T00:10:54.092264+0000","last_active":"2023-10-08T00:10:54.093599+0000","last_peered":"2023-10-08T00:10:54.093599+0000","last_clean":"2023-10-08T00:10:54.093599+0000","last_became_active":"2023-10-08T00:10:54.092043+0000","last_became_peered":"2023-10-08T00:10:54.092043+0000","last_unstale":"2023-10-08T00:10:54.093599+0000","last_undegraded":"2023-10-08T00:10:54.093599+0000","last_fullsized":"2023-10-08T00:10:54.093599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:39:51.147291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.101350+0000","last_change":"2023-10-08T00:10:53.083563+0000","last_active":"2023-10-08T00:10:54.101350+0000","last_peered":"2023-10-08T00:10:54.101350+0000","last_clean":"2023-10-08T00:10:54.101350+0000","last_became_active":"2023-10-08T00:10:53.079738+0000","last_became_peered":"2023-10-08T00:10:53.079738+0000","last_unstale":"2023-10-08T00:10:54.101350+0000","last_undegraded":"2023-10-08T00:10:54.101350+0000","last_fullsized":"2023-10-08T00:10:54.101350+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:46:39.872883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.095572+0000","last_change":"2023-10-08T00:10:54.095150+0000","last_active":"2023-10-08T00:10:54.095572+0000","last_peered":"2023-10-08T00:10:54.095572+0000","last_clean":"2023-10-08T00:10:54.095572+0000","last_became_active":"2023-10-08T00:10:54.094621+0000","last_became_peered":"2023-10-08T00:10:54.094621+0000","last_unstale":"2023-10-08T00:10:54.095572+0000","last_undegraded":"2023-10-08T00:10:54.095572+0000","last_fullsized":"2023-10-08T00:10:54.095572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:42:35.332320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.098615+0000","last_change":"2023-10-08T00:10:53.085123+0000","last_active":"2023-10-08T00:10:54.098615+0000","last_peered":"2023-10-08T00:10:54.098615+0000","last_clean":"2023-10-08T00:10:54.098615+0000","last_became_active":"2023-10-08T00:10:53.084901+0000","last_became_peered":"2023-10-08T00:10:53.084901+0000","last_unstale":"2023-10-08T00:10:54.098615+0000","last_undegraded":"2023-10-08T00:10:54.098615+0000","last_fullsized":"2023-10-08T00:10:54.098615+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:53:05.715993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.098231+0000","last_change":"2023-10-08T00:10:54.098150+0000","last_active":"2023-10-08T00:10:54.098231+0000","last_peered":"2023-10-08T00:10:54.098231+0000","last_clean":"2023-10-08T00:10:54.098231+0000","last_became_active":"2023-10-08T00:10:54.097887+0000","last_became_peered":"2023-10-08T00:10:54.097887+0000","last_unstale":"2023-10-08T00:10:54.098231+0000","last_undegraded":"2023-10-08T00:10:54.098231+0000","last_fullsized":"2023-10-08T00:10:54.098231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:06:38.007522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.096544+0000","last_change":"2023-10-08T00:10:53.082817+0000","last_active":"2023-10-08T00:10:54.096544+0000","last_peered":"2023-10-08T00:10:54.096544+0000","last_clean":"2023-10-08T00:10:54.096544+0000","last_became_active":"2023-10-08T00:10:53.082576+0000","last_became_peered":"2023-10-08T00:10:53.082576+0000","last_unstale":"2023-10-08T00:10:54.096544+0000","last_undegraded":"2023-10-08T00:10:54.096544+0000","last_fullsized":"2023-10-08T00:10:54.096544+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:46:39.872883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.100057+0000","last_change":"2023-10-08T00:10:54.099958+0000","last_active":"2023-10-08T00:10:54.100057+0000","last_peered":"2023-10-08T00:10:54.100057+0000","last_clean":"2023-10-08T00:10:54.100057+0000","last_became_active":"2023-10-08T00:10:54.099664+0000","last_became_peered":"2023-10-08T00:10:54.099664+0000","last_unstale":"2023-10-08T00:10:54.100057+0000","last_undegraded":"2023-10-08T00:10:54.100057+0000","last_fullsized":"2023-10-08T00:10:54.100057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:17:33.566050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.090051+0000","last_change":"2023-10-08T00:10:53.077494+0000","last_active":"2023-10-08T00:10:54.090051+0000","last_peered":"2023-10-08T00:10:54.090051+0000","last_clean":"2023-10-08T00:10:54.090051+0000","last_became_active":"2023-10-08T00:10:53.077248+0000","last_became_peered":"2023-10-08T00:10:53.077248+0000","last_unstale":"2023-10-08T00:10:54.090051+0000","last_undegraded":"2023-10-08T00:10:54.090051+0000","last_fullsized":"2023-10-08T00:10:54.090051+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:46:39.872883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.097553+0000","last_change":"2023-10-08T00:10:54.095175+0000","last_active":"2023-10-08T00:10:54.097553+0000","last_peered":"2023-10-08T00:10:54.097553+0000","last_clean":"2023-10-08T00:10:54.097553+0000","last_became_active":"2023-10-08T00:10:54.093363+0000","last_became_peered":"2023-10-08T00:10:54.093363+0000","last_unstale":"2023-10-08T00:10:54.097553+0000","last_undegraded":"2023-10-08T00:10:54.097553+0000","last_fullsized":"2023-10-08T00:10:54.097553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:03:42.084791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.087857+0000","last_change":"2023-10-08T00:10:53.079999+0000","last_active":"2023-10-08T00:10:54.087857+0000","last_peered":"2023-10-08T00:10:54.087857+0000","last_clean":"2023-10-08T00:10:54.087857+0000","last_became_active":"2023-10-08T00:10:53.079750+0000","last_became_peered":"2023-10-08T00:10:53.079750+0000","last_unstale":"2023-10-08T00:10:54.087857+0000","last_undegraded":"2023-10-08T00:10:54.087857+0000","last_fullsized":"2023-10-08T00:10:54.087857+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:00:50.648234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.087312+0000","last_change":"2023-10-08T00:10:54.087253+0000","last_active":"2023-10-08T00:10:54.087312+0000","last_peered":"2023-10-08T00:10:54.087312+0000","last_clean":"2023-10-08T00:10:54.087312+0000","last_became_active":"2023-10-08T00:10:54.087034+0000","last_became_peered":"2023-10-08T00:10:54.087034+0000","last_unstale":"2023-10-08T00:10:54.087312+0000","last_undegraded":"2023-10-08T00:10:54.087312+0000","last_fullsized":"2023-10-08T00:10:54.087312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:21:24.202186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.090769+0000","last_change":"2023-10-08T00:10:53.088156+0000","last_active":"2023-10-08T00:10:54.090769+0000","last_peered":"2023-10-08T00:10:54.090769+0000","last_clean":"2023-10-08T00:10:54.090769+0000","last_became_active":"2023-10-08T00:10:53.084715+0000","last_became_peered":"2023-10-08T00:10:53.084715+0000","last_unstale":"2023-10-08T00:10:54.090769+0000","last_undegraded":"2023-10-08T00:10:54.090769+0000","last_fullsized":"2023-10-08T00:10:54.090769+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:10:42.328030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.100622+0000","last_change":"2023-10-08T00:10:54.100149+0000","last_active":"2023-10-08T00:10:54.100622+0000","last_peered":"2023-10-08T00:10:54.100622+0000","last_clean":"2023-10-08T00:10:54.100622+0000","last_became_active":"2023-10-08T00:10:54.099890+0000","last_became_peered":"2023-10-08T00:10:54.099890+0000","last_unstale":"2023-10-08T00:10:54.100622+0000","last_undegraded":"2023-10-08T00:10:54.100622+0000","last_fullsized":"2023-10-08T00:10:54.100622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:30:37.085534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.092369+0000","last_change":"2023-10-08T00:10:53.077948+0000","last_active":"2023-10-08T00:10:54.092369+0000","last_peered":"2023-10-08T00:10:54.092369+0000","last_clean":"2023-10-08T00:10:54.092369+0000","last_became_active":"2023-10-08T00:10:53.077716+0000","last_became_peered":"2023-10-08T00:10:53.077716+0000","last_unstale":"2023-10-08T00:10:54.092369+0000","last_undegraded":"2023-10-08T00:10:54.092369+0000","last_fullsized":"2023-10-08T00:10:54.092369+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:08:25.404396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.101262+0000","last_change":"2023-10-08T00:10:54.101209+0000","last_active":"2023-10-08T00:10:54.101262+0000","last_peered":"2023-10-08T00:10:54.101262+0000","last_clean":"2023-10-08T00:10:54.101262+0000","last_became_active":"2023-10-08T00:10:54.100833+0000","last_became_peered":"2023-10-08T00:10:54.100833+0000","last_unstale":"2023-10-08T00:10:54.101262+0000","last_undegraded":"2023-10-08T00:10:54.101262+0000","last_fullsized":"2023-10-08T00:10:54.101262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:28:03.272641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.105007+0000","last_change":"2023-10-08T00:10:53.078194+0000","last_active":"2023-10-08T00:10:54.105007+0000","last_peered":"2023-10-08T00:10:54.105007+0000","last_clean":"2023-10-08T00:10:54.105007+0000","last_became_active":"2023-10-08T00:10:53.077936+0000","last_became_peered":"2023-10-08T00:10:53.077936+0000","last_unstale":"2023-10-08T00:10:54.105007+0000","last_undegraded":"2023-10-08T00:10:54.105007+0000","last_fullsized":"2023-10-08T00:10:54.105007+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:00:50.648234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.100556+0000","last_change":"2023-10-08T00:10:54.100459+0000","last_active":"2023-10-08T00:10:54.100556+0000","last_peered":"2023-10-08T00:10:54.100556+0000","last_clean":"2023-10-08T00:10:54.100556+0000","last_became_active":"2023-10-08T00:10:54.100194+0000","last_became_peered":"2023-10-08T00:10:54.100194+0000","last_unstale":"2023-10-08T00:10:54.100556+0000","last_undegraded":"2023-10-08T00:10:54.100556+0000","last_fullsized":"2023-10-08T00:10:54.100556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:22:42.484817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.096915+0000","last_change":"2023-10-08T00:10:53.084910+0000","last_active":"2023-10-08T00:10:54.096915+0000","last_peered":"2023-10-08T00:10:54.096915+0000","last_clean":"2023-10-08T00:10:54.096915+0000","last_became_active":"2023-10-08T00:10:53.084674+0000","last_became_peered":"2023-10-08T00:10:53.084674+0000","last_unstale":"2023-10-08T00:10:54.096915+0000","last_undegraded":"2023-10-08T00:10:54.096915+0000","last_fullsized":"2023-10-08T00:10:54.096915+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:08:39.785742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.102086+0000","last_change":"2023-10-08T00:10:54.102036+0000","last_active":"2023-10-08T00:10:54.102086+0000","last_peered":"2023-10-08T00:10:54.102086+0000","last_clean":"2023-10-08T00:10:54.102086+0000","last_became_active":"2023-10-08T00:10:54.101877+0000","last_became_peered":"2023-10-08T00:10:54.101877+0000","last_unstale":"2023-10-08T00:10:54.102086+0000","last_undegraded":"2023-10-08T00:10:54.102086+0000","last_fullsized":"2023-10-08T00:10:54.102086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:21:24.202186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.084370+0000","last_change":"2023-10-08T00:10:53.082679+0000","last_active":"2023-10-08T00:10:54.084370+0000","last_peered":"2023-10-08T00:10:54.084370+0000","last_clean":"2023-10-08T00:10:54.084370+0000","last_became_active":"2023-10-08T00:10:53.082281+0000","last_became_peered":"2023-10-08T00:10:53.082281+0000","last_unstale":"2023-10-08T00:10:54.084370+0000","last_undegraded":"2023-10-08T00:10:54.084370+0000","last_fullsized":"2023-10-08T00:10:54.084370+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:11:04.856043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.099339+0000","last_change":"2023-10-08T00:10:54.099282+0000","last_active":"2023-10-08T00:10:54.099339+0000","last_peered":"2023-10-08T00:10:54.099339+0000","last_clean":"2023-10-08T00:10:54.099339+0000","last_became_active":"2023-10-08T00:10:54.099043+0000","last_became_peered":"2023-10-08T00:10:54.099043+0000","last_unstale":"2023-10-08T00:10:54.099339+0000","last_undegraded":"2023-10-08T00:10:54.099339+0000","last_fullsized":"2023-10-08T00:10:54.099339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:11:32.172262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.090870+0000","last_change":"2023-10-08T00:10:53.088502+0000","last_active":"2023-10-08T00:10:54.090870+0000","last_peered":"2023-10-08T00:10:54.090870+0000","last_clean":"2023-10-08T00:10:54.090870+0000","last_became_active":"2023-10-08T00:10:53.086396+0000","last_became_peered":"2023-10-08T00:10:53.086396+0000","last_unstale":"2023-10-08T00:10:54.090870+0000","last_undegraded":"2023-10-08T00:10:54.090870+0000","last_fullsized":"2023-10-08T00:10:54.090870+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:42:34.326823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.095605+0000","last_change":"2023-10-08T00:10:54.095487+0000","last_active":"2023-10-08T00:10:54.095605+0000","last_peered":"2023-10-08T00:10:54.095605+0000","last_clean":"2023-10-08T00:10:54.095605+0000","last_became_active":"2023-10-08T00:10:54.094918+0000","last_became_peered":"2023-10-08T00:10:54.094918+0000","last_unstale":"2023-10-08T00:10:54.095605+0000","last_undegraded":"2023-10-08T00:10:54.095605+0000","last_fullsized":"2023-10-08T00:10:54.095605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:46:53.130793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.104070+0000","last_change":"2023-10-08T00:10:53.078081+0000","last_active":"2023-10-08T00:10:54.104070+0000","last_peered":"2023-10-08T00:10:54.104070+0000","last_clean":"2023-10-08T00:10:54.104070+0000","last_became_active":"2023-10-08T00:10:53.077802+0000","last_became_peered":"2023-10-08T00:10:53.077802+0000","last_unstale":"2023-10-08T00:10:54.104070+0000","last_undegraded":"2023-10-08T00:10:54.104070+0000","last_fullsized":"2023-10-08T00:10:54.104070+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:16:40.636514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.093642+0000","last_change":"2023-10-08T00:10:54.092992+0000","last_active":"2023-10-08T00:10:54.093642+0000","last_peered":"2023-10-08T00:10:54.093642+0000","last_clean":"2023-10-08T00:10:54.093642+0000","last_became_active":"2023-10-08T00:10:54.092509+0000","last_became_peered":"2023-10-08T00:10:54.092509+0000","last_unstale":"2023-10-08T00:10:54.093642+0000","last_undegraded":"2023-10-08T00:10:54.093642+0000","last_fullsized":"2023-10-08T00:10:54.093642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:52:39.600888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.100708+0000","last_change":"2023-10-08T00:10:53.084676+0000","last_active":"2023-10-08T00:10:54.100708+0000","last_peered":"2023-10-08T00:10:54.100708+0000","last_clean":"2023-10-08T00:10:54.100708+0000","last_became_active":"2023-10-08T00:10:53.084468+0000","last_became_peered":"2023-10-08T00:10:53.084468+0000","last_unstale":"2023-10-08T00:10:54.100708+0000","last_undegraded":"2023-10-08T00:10:54.100708+0000","last_fullsized":"2023-10-08T00:10:54.100708+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:46:39.872883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.099482+0000","last_change":"2023-10-08T00:10:54.093441+0000","last_active":"2023-10-08T00:10:54.099482+0000","last_peered":"2023-10-08T00:10:54.099482+0000","last_clean":"2023-10-08T00:10:54.099482+0000","last_became_active":"2023-10-08T00:10:54.093093+0000","last_became_peered":"2023-10-08T00:10:54.093093+0000","last_unstale":"2023-10-08T00:10:54.099482+0000","last_undegraded":"2023-10-08T00:10:54.099482+0000","last_fullsized":"2023-10-08T00:10:54.099482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:25:03.816735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.093686+0000","last_change":"2023-10-08T00:10:53.080035+0000","last_active":"2023-10-08T00:10:54.093686+0000","last_peered":"2023-10-08T00:10:54.093686+0000","last_clean":"2023-10-08T00:10:54.093686+0000","last_became_active":"2023-10-08T00:10:53.079674+0000","last_became_peered":"2023-10-08T00:10:53.079674+0000","last_unstale":"2023-10-08T00:10:54.093686+0000","last_undegraded":"2023-10-08T00:10:54.093686+0000","last_fullsized":"2023-10-08T00:10:54.093686+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:08:25.404396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.100705+0000","last_change":"2023-10-08T00:10:54.100588+0000","last_active":"2023-10-08T00:10:54.100705+0000","last_peered":"2023-10-08T00:10:54.100705+0000","last_clean":"2023-10-08T00:10:54.100705+0000","last_became_active":"2023-10-08T00:10:54.100259+0000","last_became_peered":"2023-10-08T00:10:54.100259+0000","last_unstale":"2023-10-08T00:10:54.100705+0000","last_undegraded":"2023-10-08T00:10:54.100705+0000","last_fullsized":"2023-10-08T00:10:54.100705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:40:12.384542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.084858+0000","last_change":"2023-10-08T00:10:53.079356+0000","last_active":"2023-10-08T00:10:54.084858+0000","last_peered":"2023-10-08T00:10:54.084858+0000","last_clean":"2023-10-08T00:10:54.084858+0000","last_became_active":"2023-10-08T00:10:53.079134+0000","last_became_peered":"2023-10-08T00:10:53.079134+0000","last_unstale":"2023-10-08T00:10:54.084858+0000","last_undegraded":"2023-10-08T00:10:54.084858+0000","last_fullsized":"2023-10-08T00:10:54.084858+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:30:06.828965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.094617+0000","last_change":"2023-10-08T00:10:54.094512+0000","last_active":"2023-10-08T00:10:54.094617+0000","last_peered":"2023-10-08T00:10:54.094617+0000","last_clean":"2023-10-08T00:10:54.094617+0000","last_became_active":"2023-10-08T00:10:54.093703+0000","last_became_peered":"2023-10-08T00:10:54.093703+0000","last_unstale":"2023-10-08T00:10:54.094617+0000","last_undegraded":"2023-10-08T00:10:54.094617+0000","last_fullsized":"2023-10-08T00:10:54.094617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:03:42.084791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.093726+0000","last_change":"2023-10-08T00:10:54.091651+0000","last_active":"2023-10-08T00:10:54.093726+0000","last_peered":"2023-10-08T00:10:54.093726+0000","last_clean":"2023-10-08T00:10:54.093726+0000","last_became_active":"2023-10-08T00:10:54.091234+0000","last_became_peered":"2023-10-08T00:10:54.091234+0000","last_unstale":"2023-10-08T00:10:54.093726+0000","last_undegraded":"2023-10-08T00:10:54.093726+0000","last_fullsized":"2023-10-08T00:10:54.093726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:06:38.007522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.083596+0000","last_change":"2023-10-08T00:10:53.080285+0000","last_active":"2023-10-08T00:10:54.083596+0000","last_peered":"2023-10-08T00:10:54.083596+0000","last_clean":"2023-10-08T00:10:54.083596+0000","last_became_active":"2023-10-08T00:10:53.079998+0000","last_became_peered":"2023-10-08T00:10:53.079998+0000","last_unstale":"2023-10-08T00:10:54.083596+0000","last_undegraded":"2023-10-08T00:10:54.083596+0000","last_fullsized":"2023-10-08T00:10:54.083596+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:16:40.636514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.097519+0000","last_change":"2023-10-08T00:10:54.094819+0000","last_active":"2023-10-08T00:10:54.097519+0000","last_peered":"2023-10-08T00:10:54.097519+0000","last_clean":"2023-10-08T00:10:54.097519+0000","last_became_active":"2023-10-08T00:10:54.094572+0000","last_became_peered":"2023-10-08T00:10:54.094572+0000","last_unstale":"2023-10-08T00:10:54.097519+0000","last_undegraded":"2023-10-08T00:10:54.097519+0000","last_fullsized":"2023-10-08T00:10:54.097519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:21:24.202186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.084469+0000","last_change":"2023-10-08T00:10:53.083645+0000","last_active":"2023-10-08T00:10:54.084469+0000","last_peered":"2023-10-08T00:10:54.084469+0000","last_clean":"2023-10-08T00:10:54.084469+0000","last_became_active":"2023-10-08T00:10:53.083477+0000","last_became_peered":"2023-10-08T00:10:53.083477+0000","last_unstale":"2023-10-08T00:10:54.084469+0000","last_undegraded":"2023-10-08T00:10:54.084469+0000","last_fullsized":"2023-10-08T00:10:54.084469+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:42:34.326823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":163,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.086096+0000","last_change":"2023-10-08T00:10:43.728490+0000","last_active":"2023-10-08T00:10:54.086096+0000","last_peered":"2023-10-08T00:10:54.086096+0000","last_clean":"2023-10-08T00:10:54.086096+0000","last_became_active":"2023-10-08T00:10:43.728341+0000","last_became_peered":"2023-10-08T00:10:43.728341+0000","last_unstale":"2023-10-08T00:10:54.086096+0000","last_undegraded":"2023-10-08T00:10:54.086096+0000","last_fullsized":"2023-10-08T00:10:54.086096+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_clean_scrub_stamp":"2023-10-08T00:10:42.716096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:06:31.035293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.101392+0000","last_change":"2023-10-08T00:10:54.100890+0000","last_active":"2023-10-08T00:10:54.101392+0000","last_peered":"2023-10-08T00:10:54.101392+0000","last_clean":"2023-10-08T00:10:54.101392+0000","last_became_active":"2023-10-08T00:10:54.100530+0000","last_became_peered":"2023-10-08T00:10:54.100530+0000","last_unstale":"2023-10-08T00:10:54.101392+0000","last_undegraded":"2023-10-08T00:10:54.101392+0000","last_fullsized":"2023-10-08T00:10:54.101392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:20:52.135952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.086851+0000","last_change":"2023-10-08T00:10:53.079813+0000","last_active":"2023-10-08T00:10:54.086851+0000","last_peered":"2023-10-08T00:10:54.086851+0000","last_clean":"2023-10-08T00:10:54.086851+0000","last_became_active":"2023-10-08T00:10:53.079642+0000","last_became_peered":"2023-10-08T00:10:53.079642+0000","last_unstale":"2023-10-08T00:10:54.086851+0000","last_undegraded":"2023-10-08T00:10:54.086851+0000","last_fullsized":"2023-10-08T00:10:54.086851+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:40:11.379045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.086769+0000","last_change":"2023-10-08T00:10:53.082674+0000","last_active":"2023-10-08T00:10:54.086769+0000","last_peered":"2023-10-08T00:10:54.086769+0000","last_clean":"2023-10-08T00:10:54.086769+0000","last_became_active":"2023-10-08T00:10:53.082215+0000","last_became_peered":"2023-10-08T00:10:53.082215+0000","last_unstale":"2023-10-08T00:10:54.086769+0000","last_undegraded":"2023-10-08T00:10:54.086769+0000","last_fullsized":"2023-10-08T00:10:54.086769+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:25:02.811238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.101340+0000","last_change":"2023-10-08T00:10:54.100477+0000","last_active":"2023-10-08T00:10:54.101340+0000","last_peered":"2023-10-08T00:10:54.101340+0000","last_clean":"2023-10-08T00:10:54.101340+0000","last_became_active":"2023-10-08T00:10:54.100039+0000","last_became_peered":"2023-10-08T00:10:54.100039+0000","last_unstale":"2023-10-08T00:10:54.101340+0000","last_undegraded":"2023-10-08T00:10:54.101340+0000","last_fullsized":"2023-10-08T00:10:54.101340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:54:13.221147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T00:10:55.085092+0000","last_change":"2023-10-08T00:10:53.083853+0000","last_active":"2023-10-08T00:10:55.085092+0000","last_peered":"2023-10-08T00:10:55.085092+0000","last_clean":"2023-10-08T00:10:55.085092+0000","last_became_active":"2023-10-08T00:10:53.083635+0000","last_became_peered":"2023-10-08T00:10:53.083635+0000","last_unstale":"2023-10-08T00:10:55.085092+0000","last_undegraded":"2023-10-08T00:10:55.085092+0000","last_fullsized":"2023-10-08T00:10:55.085092+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:46:39.872883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.103289+0000","last_change":"2023-10-08T00:10:54.101690+0000","last_active":"2023-10-08T00:10:54.103289+0000","last_peered":"2023-10-08T00:10:54.103289+0000","last_clean":"2023-10-08T00:10:54.103289+0000","last_became_active":"2023-10-08T00:10:54.101460+0000","last_became_peered":"2023-10-08T00:10:54.101460+0000","last_unstale":"2023-10-08T00:10:54.103289+0000","last_undegraded":"2023-10-08T00:10:54.103289+0000","last_fullsized":"2023-10-08T00:10:54.103289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:20:26.989181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.099336+0000","last_change":"2023-10-08T00:10:53.089306+0000","last_active":"2023-10-08T00:10:54.099336+0000","last_peered":"2023-10-08T00:10:54.099336+0000","last_clean":"2023-10-08T00:10:54.099336+0000","last_became_active":"2023-10-08T00:10:53.089108+0000","last_became_peered":"2023-10-08T00:10:53.089108+0000","last_unstale":"2023-10-08T00:10:54.099336+0000","last_undegraded":"2023-10-08T00:10:54.099336+0000","last_fullsized":"2023-10-08T00:10:54.099336+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:49:50.543614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.091606+0000","last_change":"2023-10-08T00:10:54.091498+0000","last_active":"2023-10-08T00:10:54.091606+0000","last_peered":"2023-10-08T00:10:54.091606+0000","last_clean":"2023-10-08T00:10:54.091606+0000","last_became_active":"2023-10-08T00:10:54.090751+0000","last_became_peered":"2023-10-08T00:10:54.090751+0000","last_unstale":"2023-10-08T00:10:54.091606+0000","last_undegraded":"2023-10-08T00:10:54.091606+0000","last_fullsized":"2023-10-08T00:10:54.091606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:52:39.600888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.088854+0000","last_change":"2023-10-08T00:10:53.083078+0000","last_active":"2023-10-08T00:10:54.088854+0000","last_peered":"2023-10-08T00:10:54.088854+0000","last_clean":"2023-10-08T00:10:54.088854+0000","last_became_active":"2023-10-08T00:10:53.082687+0000","last_became_peered":"2023-10-08T00:10:53.082687+0000","last_unstale":"2023-10-08T00:10:54.088854+0000","last_undegraded":"2023-10-08T00:10:54.088854+0000","last_fullsized":"2023-10-08T00:10:54.088854+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:46:52.125296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.090972+0000","last_change":"2023-10-08T00:10:54.090572+0000","last_active":"2023-10-08T00:10:54.090972+0000","last_peered":"2023-10-08T00:10:54.090972+0000","last_clean":"2023-10-08T00:10:54.090972+0000","last_became_active":"2023-10-08T00:10:54.090352+0000","last_became_peered":"2023-10-08T00:10:54.090352+0000","last_unstale":"2023-10-08T00:10:54.090972+0000","last_undegraded":"2023-10-08T00:10:54.090972+0000","last_fullsized":"2023-10-08T00:10:54.090972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:52:09.003538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.085955+0000","last_change":"2023-10-08T00:10:53.084909+0000","last_active":"2023-10-08T00:10:54.085955+0000","last_peered":"2023-10-08T00:10:54.085955+0000","last_clean":"2023-10-08T00:10:54.085955+0000","last_became_active":"2023-10-08T00:10:53.084676+0000","last_became_peered":"2023-10-08T00:10:53.084676+0000","last_unstale":"2023-10-08T00:10:54.085955+0000","last_undegraded":"2023-10-08T00:10:54.085955+0000","last_fullsized":"2023-10-08T00:10:54.085955+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:08:39.785742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.103202+0000","last_change":"2023-10-08T00:10:54.098822+0000","last_active":"2023-10-08T00:10:54.103202+0000","last_peered":"2023-10-08T00:10:54.103202+0000","last_clean":"2023-10-08T00:10:54.103202+0000","last_became_active":"2023-10-08T00:10:54.098426+0000","last_became_peered":"2023-10-08T00:10:54.098426+0000","last_unstale":"2023-10-08T00:10:54.103202+0000","last_undegraded":"2023-10-08T00:10:54.103202+0000","last_fullsized":"2023-10-08T00:10:54.103202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:44:19.725645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.096443+0000","last_change":"2023-10-08T00:10:53.081675+0000","last_active":"2023-10-08T00:10:54.096443+0000","last_peered":"2023-10-08T00:10:54.096443+0000","last_clean":"2023-10-08T00:10:54.096443+0000","last_became_active":"2023-10-08T00:10:53.081494+0000","last_became_peered":"2023-10-08T00:10:53.081494+0000","last_unstale":"2023-10-08T00:10:54.096443+0000","last_undegraded":"2023-10-08T00:10:54.096443+0000","last_fullsized":"2023-10-08T00:10:54.096443+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:08:25.404396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.103000+0000","last_change":"2023-10-08T00:10:54.102951+0000","last_active":"2023-10-08T00:10:54.103000+0000","last_peered":"2023-10-08T00:10:54.103000+0000","last_clean":"2023-10-08T00:10:54.103000+0000","last_became_active":"2023-10-08T00:10:54.102742+0000","last_became_peered":"2023-10-08T00:10:54.102742+0000","last_unstale":"2023-10-08T00:10:54.103000+0000","last_undegraded":"2023-10-08T00:10:54.103000+0000","last_fullsized":"2023-10-08T00:10:54.103000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:40:12.384542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.100607+0000","last_change":"2023-10-08T00:10:53.074449+0000","last_active":"2023-10-08T00:10:54.100607+0000","last_peered":"2023-10-08T00:10:54.100607+0000","last_clean":"2023-10-08T00:10:54.100607+0000","last_became_active":"2023-10-08T00:10:53.074207+0000","last_became_peered":"2023-10-08T00:10:53.074207+0000","last_unstale":"2023-10-08T00:10:54.100607+0000","last_undegraded":"2023-10-08T00:10:54.100607+0000","last_fullsized":"2023-10-08T00:10:54.100607+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:46:52.125296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.093672+0000","last_change":"2023-10-08T00:10:54.093203+0000","last_active":"2023-10-08T00:10:54.093672+0000","last_peered":"2023-10-08T00:10:54.093672+0000","last_clean":"2023-10-08T00:10:54.093672+0000","last_became_active":"2023-10-08T00:10:54.093007+0000","last_became_peered":"2023-10-08T00:10:54.093007+0000","last_unstale":"2023-10-08T00:10:54.093672+0000","last_undegraded":"2023-10-08T00:10:54.093672+0000","last_fullsized":"2023-10-08T00:10:54.093672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:30:15.548694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.099362+0000","last_change":"2023-10-08T00:10:53.085980+0000","last_active":"2023-10-08T00:10:54.099362+0000","last_peered":"2023-10-08T00:10:54.099362+0000","last_clean":"2023-10-08T00:10:54.099362+0000","last_became_active":"2023-10-08T00:10:53.085796+0000","last_became_peered":"2023-10-08T00:10:53.085796+0000","last_unstale":"2023-10-08T00:10:54.099362+0000","last_undegraded":"2023-10-08T00:10:54.099362+0000","last_fullsized":"2023-10-08T00:10:54.099362+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:16:40.636514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.097850+0000","last_change":"2023-10-08T00:10:54.097790+0000","last_active":"2023-10-08T00:10:54.097850+0000","last_peered":"2023-10-08T00:10:54.097850+0000","last_clean":"2023-10-08T00:10:54.097850+0000","last_became_active":"2023-10-08T00:10:54.096497+0000","last_became_peered":"2023-10-08T00:10:54.096497+0000","last_unstale":"2023-10-08T00:10:54.097850+0000","last_undegraded":"2023-10-08T00:10:54.097850+0000","last_fullsized":"2023-10-08T00:10:54.097850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:05:53.331248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.097812+0000","last_change":"2023-10-08T00:10:53.078098+0000","last_active":"2023-10-08T00:10:54.097812+0000","last_peered":"2023-10-08T00:10:54.097812+0000","last_clean":"2023-10-08T00:10:54.097812+0000","last_became_active":"2023-10-08T00:10:53.077855+0000","last_became_peered":"2023-10-08T00:10:53.077855+0000","last_unstale":"2023-10-08T00:10:54.097812+0000","last_undegraded":"2023-10-08T00:10:54.097812+0000","last_fullsized":"2023-10-08T00:10:54.097812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:10:42.328030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.093527+0000","last_change":"2023-10-08T00:10:54.092335+0000","last_active":"2023-10-08T00:10:54.093527+0000","last_peered":"2023-10-08T00:10:54.093527+0000","last_clean":"2023-10-08T00:10:54.093527+0000","last_became_active":"2023-10-08T00:10:54.091475+0000","last_became_peered":"2023-10-08T00:10:54.091475+0000","last_unstale":"2023-10-08T00:10:54.093527+0000","last_undegraded":"2023-10-08T00:10:54.093527+0000","last_fullsized":"2023-10-08T00:10:54.093527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:30:15.548694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.098946+0000","last_change":"2023-10-08T00:10:53.079388+0000","last_active":"2023-10-08T00:10:54.098946+0000","last_peered":"2023-10-08T00:10:54.098946+0000","last_clean":"2023-10-08T00:10:54.098946+0000","last_became_active":"2023-10-08T00:10:53.079152+0000","last_became_peered":"2023-10-08T00:10:53.079152+0000","last_unstale":"2023-10-08T00:10:54.098946+0000","last_undegraded":"2023-10-08T00:10:54.098946+0000","last_fullsized":"2023-10-08T00:10:54.098946+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:52.061775+0000","last_clean_scrub_stamp":"2023-10-08T00:10:52.061775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:08:25.404396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.090252+0000","last_change":"2023-10-08T00:10:54.088605+0000","last_active":"2023-10-08T00:10:54.090252+0000","last_peered":"2023-10-08T00:10:54.090252+0000","last_clean":"2023-10-08T00:10:54.090252+0000","last_became_active":"2023-10-08T00:10:54.088396+0000","last_became_peered":"2023-10-08T00:10:54.088396+0000","last_unstale":"2023-10-08T00:10:54.090252+0000","last_undegraded":"2023-10-08T00:10:54.090252+0000","last_fullsized":"2023-10-08T00:10:54.090252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:54:13.221147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.100688+0000","last_change":"2023-10-08T00:10:54.100120+0000","last_active":"2023-10-08T00:10:54.100688+0000","last_peered":"2023-10-08T00:10:54.100688+0000","last_clean":"2023-10-08T00:10:54.100688+0000","last_became_active":"2023-10-08T00:10:54.099910+0000","last_became_peered":"2023-10-08T00:10:54.099910+0000","last_unstale":"2023-10-08T00:10:54.100688+0000","last_undegraded":"2023-10-08T00:10:54.100688+0000","last_fullsized":"2023-10-08T00:10:54.100688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:11:05.861540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.094221+0000","last_change":"2023-10-08T00:10:54.093990+0000","last_active":"2023-10-08T00:10:54.094221+0000","last_peered":"2023-10-08T00:10:54.094221+0000","last_clean":"2023-10-08T00:10:54.094221+0000","last_became_active":"2023-10-08T00:10:54.093268+0000","last_became_peered":"2023-10-08T00:10:54.093268+0000","last_unstale":"2023-10-08T00:10:54.094221+0000","last_undegraded":"2023-10-08T00:10:54.094221+0000","last_fullsized":"2023-10-08T00:10:54.094221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:49:51.549111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.096510+0000","last_change":"2023-10-08T00:10:54.096449+0000","last_active":"2023-10-08T00:10:54.096510+0000","last_peered":"2023-10-08T00:10:54.096510+0000","last_clean":"2023-10-08T00:10:54.096510+0000","last_became_active":"2023-10-08T00:10:54.096216+0000","last_became_peered":"2023-10-08T00:10:54.096216+0000","last_unstale":"2023-10-08T00:10:54.096510+0000","last_undegraded":"2023-10-08T00:10:54.096510+0000","last_fullsized":"2023-10-08T00:10:54.096510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:11:41.906007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.093706+0000","last_change":"2023-10-08T00:10:54.091529+0000","last_active":"2023-10-08T00:10:54.093706+0000","last_peered":"2023-10-08T00:10:54.093706+0000","last_clean":"2023-10-08T00:10:54.093706+0000","last_became_active":"2023-10-08T00:10:54.090949+0000","last_became_peered":"2023-10-08T00:10:54.090949+0000","last_unstale":"2023-10-08T00:10:54.093706+0000","last_undegraded":"2023-10-08T00:10:54.093706+0000","last_fullsized":"2023-10-08T00:10:54.093706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:52:44.075366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.101359+0000","last_change":"2023-10-08T00:10:54.100755+0000","last_active":"2023-10-08T00:10:54.101359+0000","last_peered":"2023-10-08T00:10:54.101359+0000","last_clean":"2023-10-08T00:10:54.101359+0000","last_became_active":"2023-10-08T00:10:54.100206+0000","last_became_peered":"2023-10-08T00:10:54.100206+0000","last_unstale":"2023-10-08T00:10:54.101359+0000","last_undegraded":"2023-10-08T00:10:54.101359+0000","last_fullsized":"2023-10-08T00:10:54.101359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:03:42.084791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.101376+0000","last_change":"2023-10-08T00:10:54.100759+0000","last_active":"2023-10-08T00:10:54.101376+0000","last_peered":"2023-10-08T00:10:54.101376+0000","last_clean":"2023-10-08T00:10:54.101376+0000","last_became_active":"2023-10-08T00:10:54.100365+0000","last_became_peered":"2023-10-08T00:10:54.100365+0000","last_unstale":"2023-10-08T00:10:54.101376+0000","last_undegraded":"2023-10-08T00:10:54.101376+0000","last_fullsized":"2023-10-08T00:10:54.101376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:40:12.384542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.087188+0000","last_change":"2023-10-08T00:10:54.085760+0000","last_active":"2023-10-08T00:10:54.087188+0000","last_peered":"2023-10-08T00:10:54.087188+0000","last_clean":"2023-10-08T00:10:54.087188+0000","last_became_active":"2023-10-08T00:10:54.085544+0000","last_became_peered":"2023-10-08T00:10:54.085544+0000","last_unstale":"2023-10-08T00:10:54.087188+0000","last_undegraded":"2023-10-08T00:10:54.087188+0000","last_fullsized":"2023-10-08T00:10:54.087188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:53:06.721490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.099558+0000","last_change":"2023-10-08T00:10:54.095621+0000","last_active":"2023-10-08T00:10:54.099558+0000","last_peered":"2023-10-08T00:10:54.099558+0000","last_clean":"2023-10-08T00:10:54.099558+0000","last_became_active":"2023-10-08T00:10:54.095240+0000","last_became_peered":"2023-10-08T00:10:54.095240+0000","last_unstale":"2023-10-08T00:10:54.099558+0000","last_undegraded":"2023-10-08T00:10:54.099558+0000","last_fullsized":"2023-10-08T00:10:54.099558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:22:42.484817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.101130+0000","last_change":"2023-10-08T00:10:54.101076+0000","last_active":"2023-10-08T00:10:54.101130+0000","last_peered":"2023-10-08T00:10:54.101130+0000","last_clean":"2023-10-08T00:10:54.101130+0000","last_became_active":"2023-10-08T00:10:54.100508+0000","last_became_peered":"2023-10-08T00:10:54.100508+0000","last_unstale":"2023-10-08T00:10:54.101130+0000","last_undegraded":"2023-10-08T00:10:54.101130+0000","last_fullsized":"2023-10-08T00:10:54.101130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:18:09.014594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.100641+0000","last_change":"2023-10-08T00:10:54.100508+0000","last_active":"2023-10-08T00:10:54.100641+0000","last_peered":"2023-10-08T00:10:54.100641+0000","last_clean":"2023-10-08T00:10:54.100641+0000","last_became_active":"2023-10-08T00:10:54.100272+0000","last_became_peered":"2023-10-08T00:10:54.100272+0000","last_unstale":"2023-10-08T00:10:54.100641+0000","last_undegraded":"2023-10-08T00:10:54.100641+0000","last_fullsized":"2023-10-08T00:10:54.100641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:52:09.003538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.095807+0000","last_change":"2023-10-08T00:10:54.095683+0000","last_active":"2023-10-08T00:10:54.095807+0000","last_peered":"2023-10-08T00:10:54.095807+0000","last_clean":"2023-10-08T00:10:54.095807+0000","last_became_active":"2023-10-08T00:10:54.095474+0000","last_became_peered":"2023-10-08T00:10:54.095474+0000","last_unstale":"2023-10-08T00:10:54.095807+0000","last_undegraded":"2023-10-08T00:10:54.095807+0000","last_fullsized":"2023-10-08T00:10:54.095807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:11:05.861540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.092456+0000","last_change":"2023-10-08T00:10:54.092383+0000","last_active":"2023-10-08T00:10:54.092456+0000","last_peered":"2023-10-08T00:10:54.092456+0000","last_clean":"2023-10-08T00:10:54.092456+0000","last_became_active":"2023-10-08T00:10:54.092098+0000","last_became_peered":"2023-10-08T00:10:54.092098+0000","last_unstale":"2023-10-08T00:10:54.092456+0000","last_undegraded":"2023-10-08T00:10:54.092456+0000","last_fullsized":"2023-10-08T00:10:54.092456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:54:13.221147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:10:54.098417+0000","last_change":"2023-10-08T00:10:54.097569+0000","last_active":"2023-10-08T00:10:54.098417+0000","last_peered":"2023-10-08T00:10:54.098417+0000","last_clean":"2023-10-08T00:10:54.098417+0000","last_became_active":"2023-10-08T00:10:54.096963+0000","last_became_peered":"2023-10-08T00:10:54.096963+0000","last_unstale":"2023-10-08T00:10:54.098417+0000","last_undegraded":"2023-10-08T00:10:54.098417+0000","last_fullsized":"2023-10-08T00:10:54.098417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:53.067272+0000","last_clean_scrub_stamp":"2023-10-08T00:10:53.067272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:52:09.003538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":68,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929382,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":68,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929506,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":645,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194667,"statfs":{"total":200000000,"available":199339068,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.315}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":35,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":68,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929450,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.221}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28100000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":70,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195242,"statfs":{"total":200000000,"available":199927970,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199928952,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":70,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195241,"statfs":{"total":200000000,"available":199927326,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":645,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194667,"statfs":{"total":200000000,"available":199339046,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]}]}],"pool_statfs":[]}} 2023-10-08T00:11:07.432 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-08T00:11:07.433 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-08T00:11:07.433 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-08T00:11:07.433 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-08T00:11:07.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.578+0000 7ff626308700 1 Processor -- start 2023-10-08T00:11:07.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.578+0000 7ff626308700 1 -- start start 2023-10-08T00:11:07.581 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.579+0000 7ff626308700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff62015cbd0 0x7ff62015d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:07.581 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.579+0000 7ff626308700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6200cb7c0 con 0x7ff6200cd6e0 2023-10-08T00:11:07.581 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.579+0000 7ff626308700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff62015e290 con 0x7ff6200cdab0 2023-10-08T00:11:07.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.579+0000 7ff626308700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff62015e850 con 0x7ff62015cbd0 2023-10-08T00:11:07.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.579+0000 7ff625306700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7ff6200cd6e0 0x7ff6200cbc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45122/0 (socket says 172.21.15.99:45122) 2023-10-08T00:11:07.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.579+0000 7ff625306700 1 -- 172.21.15.99:0/1690158242 learned_addr learned my addr 172.21.15.99:0/1690158242 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:07.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.579+0000 7ff625b07700 1 --2- 172.21.15.99:0/1690158242 >> v2:172.21.15.191:3300/0 conn(0x7ff62015cbd0 0x7ff62015d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:07.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.579+0000 7ff617fff700 1 -- 172.21.15.99:0/1690158242 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3287439111 0 0) 0x7ff6200cb7c0 con 0x7ff6200cd6e0 2023-10-08T00:11:07.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.579+0000 7ff617fff700 1 -- 172.21.15.99:0/1690158242 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff60c003570 con 0x7ff6200cd6e0 2023-10-08T00:11:07.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.580+0000 7ff625b07700 1 -- 172.21.15.99:0/1690158242 >> v1:172.21.15.191:6789/0 conn(0x7ff6200cdab0 legacy=0x7ff62015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:07.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.580+0000 7ff617fff700 1 -- 172.21.15.99:0/1690158242 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3993615837 0 0) 0x7ff62015e290 con 0x7ff6200cdab0 2023-10-08T00:11:07.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.580+0000 7ff625b07700 1 -- 172.21.15.99:0/1690158242 >> v1:172.21.15.99:6789/0 conn(0x7ff6200cd6e0 legacy=0x7ff6200cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:07.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.580+0000 7ff625b07700 1 -- 172.21.15.99:0/1690158242 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff62015e570 con 0x7ff62015cbd0 2023-10-08T00:11:07.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.580+0000 7ff625b07700 1 --2- 172.21.15.99:0/1690158242 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff62015cbd0 0x7ff62015d050 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff618009e60 tx=0x7ff618009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe5bfb01180c29fb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:07.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.580+0000 7ff617fff700 1 -- 172.21.15.99:0/1690158242 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61800e070 con 0x7ff62015cbd0 2023-10-08T00:11:07.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.580+0000 7ff617fff700 1 -- 172.21.15.99:0/1690158242 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff61800b850 con 0x7ff62015cbd0 2023-10-08T00:11:07.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.580+0000 7ff617fff700 1 -- 172.21.15.99:0/1690158242 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61c001470 con 0x7ff62015cbd0 2023-10-08T00:11:07.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.580+0000 7ff626308700 1 -- 172.21.15.99:0/1690158242 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff62015cbd0 msgr2=0x7ff62015d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:07.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.580+0000 7ff626308700 1 --2- 172.21.15.99:0/1690158242 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff62015cbd0 0x7ff62015d050 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff618009e60 tx=0x7ff618009780 comp rx=0 tx=0).stop 2023-10-08T00:11:07.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.580+0000 7ff626308700 1 -- 172.21.15.99:0/1690158242 shutdown_connections 2023-10-08T00:11:07.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.580+0000 7ff626308700 1 --2- 172.21.15.99:0/1690158242 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff62015cbd0 0x7ff62015d050 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:07.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.580+0000 7ff626308700 1 -- 172.21.15.99:0/1690158242 >> 172.21.15.99:0/1690158242 conn(0x7ff620067660 msgr2=0x7ff620067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:07.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.581+0000 7ff626308700 1 -- 172.21.15.99:0/1690158242 shutdown_connections 2023-10-08T00:11:07.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.581+0000 7ff626308700 1 -- 172.21.15.99:0/1690158242 wait complete. 2023-10-08T00:11:07.586 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.582+0000 7ff626308700 1 Processor -- start 2023-10-08T00:11:07.587 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.582+0000 7ff626308700 1 -- start start 2023-10-08T00:11:07.587 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.582+0000 7ff626308700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6200cd6e0 0x7ff6200b5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:07.587 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.582+0000 7ff626308700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6200cdab0 0x7ff6200c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:07.587 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.582+0000 7ff626308700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff62015cbd0 0x7ff620153130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:07.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.582+0000 7ff626308700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6200cc360 con 0x7ff62015cbd0 2023-10-08T00:11:07.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.582+0000 7ff626308700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6200c8cc0 con 0x7ff6200cdab0 2023-10-08T00:11:07.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.582+0000 7ff626308700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6200c8420 con 0x7ff6200cd6e0 2023-10-08T00:11:07.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.583+0000 7ff625b07700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff62015cbd0 0x7ff620153130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:07.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.583+0000 7ff625b07700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff62015cbd0 0x7ff620153130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38918/0 (socket says 172.21.15.99:38918) 2023-10-08T00:11:07.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.583+0000 7ff625b07700 1 -- 172.21.15.99:0/1924748547 learned_addr learned my addr 172.21.15.99:0/1924748547 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:07.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.583+0000 7ff625306700 1 --2- 172.21.15.99:0/1924748547 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6200cd6e0 0x7ff6200b5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:07.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.583+0000 7ff624b05700 1 --2- 172.21.15.99:0/1924748547 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6200cdab0 0x7ff6200c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:07.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.583+0000 7ff625b07700 1 -- 172.21.15.99:0/1924748547 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6200cd6e0 msgr2=0x7ff6200b5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:07.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.583+0000 7ff625b07700 1 --2- 172.21.15.99:0/1924748547 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6200cd6e0 0x7ff6200b5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:07.590 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.583+0000 7ff625b07700 1 -- 172.21.15.99:0/1924748547 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6200cdab0 msgr2=0x7ff6200c7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:07.590 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.583+0000 7ff625b07700 1 --2- 172.21.15.99:0/1924748547 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6200cdab0 0x7ff6200c7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:07.590 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.583+0000 7ff625b07700 1 -- 172.21.15.99:0/1924748547 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff618009a20 con 0x7ff62015cbd0 2023-10-08T00:11:07.590 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.583+0000 7ff625b07700 1 --2- 172.21.15.99:0/1924748547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff62015cbd0 0x7ff620153130 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff6180097c0 tx=0x7ff618017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:07.590 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.583+0000 7ff6157fa700 1 -- 172.21.15.99:0/1924748547 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61801f410 con 0x7ff62015cbd0 2023-10-08T00:11:07.590 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.583+0000 7ff6157fa700 1 -- 172.21.15.99:0/1924748547 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff61801f5b0 con 0x7ff62015cbd0 2023-10-08T00:11:07.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.583+0000 7ff6157fa700 1 -- 172.21.15.99:0/1924748547 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61801f970 con 0x7ff62015cbd0 2023-10-08T00:11:07.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.583+0000 7ff626308700 1 -- 172.21.15.99:0/1924748547 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6200c8700 con 0x7ff62015cbd0 2023-10-08T00:11:07.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.583+0000 7ff626308700 1 -- 172.21.15.99:0/1924748547 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff62016a440 con 0x7ff62015cbd0 2023-10-08T00:11:07.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.584+0000 7ff626308700 1 -- 172.21.15.99:0/1924748547 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff62016a270 con 0x7ff62015cbd0 2023-10-08T00:11:07.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.584+0000 7ff6157fa700 1 -- 172.21.15.99:0/1924748547 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff6180113b0 con 0x7ff62015cbd0 2023-10-08T00:11:07.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.585+0000 7ff6157fa700 1 --2- 172.21.15.99:0/1924748547 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff604066f30 0x7ff6040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:07.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.585+0000 7ff6157fa700 1 -- 172.21.15.99:0/1924748547 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff618086840 con 0x7ff62015cbd0 2023-10-08T00:11:07.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.585+0000 7ff625306700 1 --2- 172.21.15.99:0/1924748547 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff604066f30 0x7ff6040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:07.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.585+0000 7ff625306700 1 --2- 172.21.15.99:0/1924748547 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff604066f30 0x7ff6040693e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff61c001200 tx=0x7ff61c000bc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:07.592 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.589+0000 7ff6157fa700 1 -- 172.21.15.99:0/1924748547 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff618053ab0 con 0x7ff62015cbd0 2023-10-08T00:11:07.814 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.813+0000 7ff626308700 1 -- 172.21.15.99:0/1924748547 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7ff62016a630 con 0x7ff62015cbd0 2023-10-08T00:11:07.814 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.814+0000 7ff6157fa700 1 -- 172.21.15.99:0/1924748547 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7ff618023080 con 0x7ff62015cbd0 2023-10-08T00:11:07.814 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:11:07.815 INFO:teuthology.orchestra.run.smithi099.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-08T00:11:07.816 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.815+0000 7ff626308700 1 -- 172.21.15.99:0/1924748547 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff604066f30 msgr2=0x7ff6040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:07.816 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.815+0000 7ff626308700 1 --2- 172.21.15.99:0/1924748547 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff604066f30 0x7ff6040693e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff61c001200 tx=0x7ff61c000bc0 comp rx=0 tx=0).stop 2023-10-08T00:11:07.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.815+0000 7ff626308700 1 -- 172.21.15.99:0/1924748547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff62015cbd0 msgr2=0x7ff620153130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:07.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.815+0000 7ff626308700 1 --2- 172.21.15.99:0/1924748547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff62015cbd0 0x7ff620153130 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff6180097c0 tx=0x7ff618017000 comp rx=0 tx=0).stop 2023-10-08T00:11:07.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.816+0000 7ff626308700 1 -- 172.21.15.99:0/1924748547 shutdown_connections 2023-10-08T00:11:07.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.816+0000 7ff626308700 1 --2- 172.21.15.99:0/1924748547 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff604066f30 0x7ff6040693e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:07.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.816+0000 7ff626308700 1 --2- 172.21.15.99:0/1924748547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff62015cbd0 0x7ff620153130 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:07.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.816+0000 7ff626308700 1 --2- 172.21.15.99:0/1924748547 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6200cd6e0 0x7ff6200b5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:07.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.816+0000 7ff626308700 1 --2- 172.21.15.99:0/1924748547 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6200cdab0 0x7ff6200c7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:07.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.816+0000 7ff626308700 1 -- 172.21.15.99:0/1924748547 >> 172.21.15.99:0/1924748547 conn(0x7ff620067660 msgr2=0x7ff620154240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:07.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.816+0000 7ff626308700 1 -- 172.21.15.99:0/1924748547 shutdown_connections 2023-10-08T00:11:07.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.816+0000 7ff626308700 1 -- 172.21.15.99:0/1924748547 wait complete. 2023-10-08T00:11:07.828 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-10-08T00:11:07.829 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:11:07.942 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.941+0000 7f4ce2554700 1 Processor -- start 2023-10-08T00:11:07.942 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.941+0000 7f4ce2554700 1 -- start start 2023-10-08T00:11:07.945 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.942+0000 7f4ce2554700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4cdc157a90 0x7f4cdc0b1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:07.945 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.942+0000 7f4ce2554700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4cdc157e60 0x7f4cdc0b0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:07.945 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.942+0000 7f4ce2554700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cdc0cb0b0 con 0x7f4cdc0b0660 2023-10-08T00:11:07.945 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.942+0000 7f4ce2554700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cdc0b0ad0 con 0x7f4cdc157e60 2023-10-08T00:11:07.945 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.942+0000 7f4ce2554700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cdc0b4970 con 0x7f4cdc157a90 2023-10-08T00:11:07.946 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.942+0000 7f4ce0d51700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4cdc157e60 0x7f4cdc0b0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:07.946 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.942+0000 7f4ce1d53700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f4cdc0b0660 0x7f4cdc0ce950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45128/0 (socket says 172.21.15.99:45128) 2023-10-08T00:11:07.946 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.942+0000 7f4ce1d53700 1 -- 172.21.15.99:0/3343710462 learned_addr learned my addr 172.21.15.99:0/3343710462 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:07.946 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.942+0000 7f4ce1552700 1 --2- 172.21.15.99:0/3343710462 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4cdc157a90 0x7f4cdc0b1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:07.946 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.942+0000 7f4ce0d51700 1 -- 172.21.15.99:0/3343710462 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4cdc157a90 msgr2=0x7f4cdc0b1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:07.946 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.942+0000 7f4ce0d51700 1 --2- 172.21.15.99:0/3343710462 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4cdc157a90 0x7f4cdc0b1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:07.946 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.942+0000 7f4ce0d51700 1 -- 172.21.15.99:0/3343710462 >> v1:172.21.15.99:6789/0 conn(0x7f4cdc0b0660 legacy=0x7f4cdc0ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:07.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.942+0000 7f4ce0d51700 1 -- 172.21.15.99:0/3343710462 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cdc0ca780 con 0x7f4cdc157e60 2023-10-08T00:11:07.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.942+0000 7f4cd3fff700 1 -- 172.21.15.99:0/3343710462 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2101370764 0 0) 0x7f4cdc0cb0b0 con 0x7f4cdc0b0660 2023-10-08T00:11:07.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.943+0000 7f4ce0d51700 1 --2- 172.21.15.99:0/3343710462 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cdc157e60 0x7f4cdc0b0120 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc009910 tx=0x7f4ccc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd67592142b66dd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:07.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.943+0000 7f4cd3fff700 1 -- 172.21.15.99:0/3343710462 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc00b820 con 0x7f4cdc157e60 2023-10-08T00:11:07.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.943+0000 7f4cd3fff700 1 -- 172.21.15.99:0/3343710462 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ccc00b980 con 0x7f4cdc157e60 2023-10-08T00:11:07.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.943+0000 7f4cd3fff700 1 -- 172.21.15.99:0/3343710462 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc00bd20 con 0x7f4cdc157e60 2023-10-08T00:11:07.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.943+0000 7f4ce2554700 1 -- 172.21.15.99:0/3343710462 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cdc157e60 msgr2=0x7f4cdc0b0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:07.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.943+0000 7f4ce2554700 1 --2- 172.21.15.99:0/3343710462 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cdc157e60 0x7f4cdc0b0120 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc009910 tx=0x7f4ccc0091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:07.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.943+0000 7f4ce2554700 1 -- 172.21.15.99:0/3343710462 shutdown_connections 2023-10-08T00:11:07.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.943+0000 7f4ce2554700 1 --2- 172.21.15.99:0/3343710462 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4cdc157a90 0x7f4cdc0b1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:07.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.943+0000 7f4ce2554700 1 --2- 172.21.15.99:0/3343710462 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cdc157e60 0x7f4cdc0b0120 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:07.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.943+0000 7f4ce2554700 1 -- 172.21.15.99:0/3343710462 >> 172.21.15.99:0/3343710462 conn(0x7f4cdc06bc70 msgr2=0x7f4cdc06c070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:07.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.943+0000 7f4ce2554700 1 -- 172.21.15.99:0/3343710462 shutdown_connections 2023-10-08T00:11:07.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.943+0000 7f4ce2554700 1 -- 172.21.15.99:0/3343710462 wait complete. 2023-10-08T00:11:07.950 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.944+0000 7f4ce2554700 1 Processor -- start 2023-10-08T00:11:07.950 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.944+0000 7f4ce2554700 1 -- start start 2023-10-08T00:11:07.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.945+0000 7f4ce2554700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4cdc0b0660 0x7f4cdc1683a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:07.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.945+0000 7f4ce2554700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cdc157a90 0x7f4cdc0c47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:07.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.945+0000 7f4ce2554700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4cdc157e60 0x7f4cdc15c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:07.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.945+0000 7f4ce2554700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cdc15c380 con 0x7f4cdc157a90 2023-10-08T00:11:07.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.945+0000 7f4ce2554700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cdc1688e0 con 0x7f4cdc0b0660 2023-10-08T00:11:07.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.945+0000 7f4ce2554700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cdc0c4d10 con 0x7f4cdc157e60 2023-10-08T00:11:07.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.945+0000 7f4ce0d51700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cdc157a90 0x7f4cdc0c47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:07.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.945+0000 7f4ce0d51700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cdc157a90 0x7f4cdc0c47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38928/0 (socket says 172.21.15.99:38928) 2023-10-08T00:11:07.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.945+0000 7f4ce0d51700 1 -- 172.21.15.99:0/1454224712 learned_addr learned my addr 172.21.15.99:0/1454224712 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:07.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.946+0000 7f4ce1552700 1 --2- 172.21.15.99:0/1454224712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4cdc0b0660 0x7f4cdc1683a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:07.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.946+0000 7f4ce1d53700 1 --2- 172.21.15.99:0/1454224712 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4cdc157e60 0x7f4cdc15c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:07.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.946+0000 7f4ce0d51700 1 -- 172.21.15.99:0/1454224712 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4cdc157e60 msgr2=0x7f4cdc15c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:07.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.946+0000 7f4ce0d51700 1 --2- 172.21.15.99:0/1454224712 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4cdc157e60 0x7f4cdc15c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:07.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.946+0000 7f4ce0d51700 1 -- 172.21.15.99:0/1454224712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4cdc0b0660 msgr2=0x7f4cdc1683a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:07.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.946+0000 7f4ce0d51700 1 --2- 172.21.15.99:0/1454224712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4cdc0b0660 0x7f4cdc1683a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:07.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.946+0000 7f4ce0d51700 1 -- 172.21.15.99:0/1454224712 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ccc009420 con 0x7f4cdc157a90 2023-10-08T00:11:07.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.946+0000 7f4ce0d51700 1 --2- 172.21.15.99:0/1454224712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cdc157a90 0x7f4cdc0c47d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc00b3c0 tx=0x7f4ccc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:07.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.946+0000 7f4cd17fa700 1 -- 172.21.15.99:0/1454224712 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc024440 con 0x7f4cdc157a90 2023-10-08T00:11:07.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.946+0000 7f4cd17fa700 1 -- 172.21.15.99:0/1454224712 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ccc0245e0 con 0x7f4cdc157a90 2023-10-08T00:11:07.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.946+0000 7f4ce2554700 1 -- 172.21.15.99:0/1454224712 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cdc15b770 con 0x7f4cdc157a90 2023-10-08T00:11:07.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.946+0000 7f4cd17fa700 1 -- 172.21.15.99:0/1454224712 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc0249a0 con 0x7f4cdc157a90 2023-10-08T00:11:07.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.946+0000 7f4ce2554700 1 -- 172.21.15.99:0/1454224712 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cdc15bd30 con 0x7f4cdc157a90 2023-10-08T00:11:07.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.947+0000 7f4ce2554700 1 -- 172.21.15.99:0/1454224712 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cdc0b4270 con 0x7f4cdc157a90 2023-10-08T00:11:07.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.948+0000 7f4cd17fa700 1 -- 172.21.15.99:0/1454224712 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4ccc01a420 con 0x7f4cdc157a90 2023-10-08T00:11:07.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.948+0000 7f4cd17fa700 1 --2- 172.21.15.99:0/1454224712 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4cc0066f30 0x7f4cc00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:07.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.948+0000 7f4cd17fa700 1 -- 172.21.15.99:0/1454224712 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4ccc086a40 con 0x7f4cdc157a90 2023-10-08T00:11:07.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.949+0000 7f4ce1552700 1 --2- 172.21.15.99:0/1454224712 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4cc0066f30 0x7f4cc00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:07.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.949+0000 7f4ce1552700 1 --2- 172.21.15.99:0/1454224712 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4cc0066f30 0x7f4cc00693e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8006fd0 tx=0x7f4cd8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:07.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:07.952+0000 7f4cd17fa700 1 -- 172.21.15.99:0/1454224712 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4ccc053de0 con 0x7f4cdc157a90 2023-10-08T00:11:08.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.134+0000 7f4ce2554700 1 -- 172.21.15.99:0/1454224712 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4cdc15c4c0 con 0x7f4cdc157a90 2023-10-08T00:11:08.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.135+0000 7f4cd17fa700 1 -- 172.21.15.99:0/1454224712 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f4ccc057400 con 0x7f4cdc157a90 2023-10-08T00:11:08.136 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:11:08.136 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":20,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:10:55.080854+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:10:51.822157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T00:10:52.415130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:11:08.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.137+0000 7f4ce2554700 1 -- 172.21.15.99:0/1454224712 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4cc0066f30 msgr2=0x7f4cc00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:08.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.137+0000 7f4ce2554700 1 --2- 172.21.15.99:0/1454224712 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4cc0066f30 0x7f4cc00693e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8006fd0 tx=0x7f4cd8008040 comp rx=0 tx=0).stop 2023-10-08T00:11:08.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.137+0000 7f4ce2554700 1 -- 172.21.15.99:0/1454224712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cdc157a90 msgr2=0x7f4cdc0c47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:08.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.137+0000 7f4ce2554700 1 --2- 172.21.15.99:0/1454224712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cdc157a90 0x7f4cdc0c47d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc00b3c0 tx=0x7f4ccc017000 comp rx=0 tx=0).stop 2023-10-08T00:11:08.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.137+0000 7f4ce2554700 1 -- 172.21.15.99:0/1454224712 shutdown_connections 2023-10-08T00:11:08.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.137+0000 7f4ce2554700 1 --2- 172.21.15.99:0/1454224712 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4cc0066f30 0x7f4cc00693e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:08.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.137+0000 7f4ce2554700 1 --2- 172.21.15.99:0/1454224712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4cdc0b0660 0x7f4cdc1683a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:08.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.137+0000 7f4ce2554700 1 --2- 172.21.15.99:0/1454224712 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4cdc157e60 0x7f4cdc15c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:08.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.137+0000 7f4ce2554700 1 --2- 172.21.15.99:0/1454224712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cdc157a90 0x7f4cdc0c47d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:08.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.137+0000 7f4ce2554700 1 -- 172.21.15.99:0/1454224712 >> 172.21.15.99:0/1454224712 conn(0x7f4cdc06bc70 msgr2=0x7f4cdc0cc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:08.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.137+0000 7f4ce2554700 1 -- 172.21.15.99:0/1454224712 shutdown_connections 2023-10-08T00:11:08.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.137+0000 7f4ce2554700 1 -- 172.21.15.99:0/1454224712 wait complete. 2023-10-08T00:11:08.150 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:10:42.063344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T00:10:51.822157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T00:10:52.415130+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T00:11:08.150 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T00:11:08.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.267+0000 7fca11c20700 1 Processor -- start 2023-10-08T00:11:08.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.267+0000 7fca11c20700 1 -- start start 2023-10-08T00:11:08.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.268+0000 7fca11c20700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fca0c06d370 0x7fca0c06d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:08.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.268+0000 7fca11c20700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca0c15c8d0 con 0x7fca0c152300 2023-10-08T00:11:08.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.268+0000 7fca11c20700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca0c0c9370 con 0x7fca0c15b9a0 2023-10-08T00:11:08.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.268+0000 7fca11c20700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca0c0b1400 con 0x7fca0c06d370 2023-10-08T00:11:08.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.268+0000 7fca0b7fe700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fca0c152300 0x7fca0c158df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45140/0 (socket says 172.21.15.99:45140) 2023-10-08T00:11:08.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.268+0000 7fca0b7fe700 1 -- 172.21.15.99:0/163199436 learned_addr learned my addr 172.21.15.99:0/163199436 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:08.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.268+0000 7fca0bfff700 1 --2- 172.21.15.99:0/163199436 >> v2:172.21.15.191:3300/0 conn(0x7fca0c06d370 0x7fca0c06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:08.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.268+0000 7fca0a7fc700 1 -- 172.21.15.99:0/163199436 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1770058715 0 0) 0x7fca0c15c8d0 con 0x7fca0c152300 2023-10-08T00:11:08.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.268+0000 7fca0a7fc700 1 -- 172.21.15.99:0/163199436 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9f4003570 con 0x7fca0c152300 2023-10-08T00:11:08.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.268+0000 7fca0a7fc700 1 -- 172.21.15.99:0/163199436 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2434477248 0 0) 0x7fca0c0c9370 con 0x7fca0c15b9a0 2023-10-08T00:11:08.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.268+0000 7fca0a7fc700 1 -- 172.21.15.99:0/163199436 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca0c15c8d0 con 0x7fca0c15b9a0 2023-10-08T00:11:08.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.268+0000 7fca0a7fc700 1 -- 172.21.15.99:0/163199436 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2566186456 0 0) 0x7fc9f4003570 con 0x7fca0c152300 2023-10-08T00:11:08.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.268+0000 7fca0bfff700 1 -- 172.21.15.99:0/163199436 >> v1:172.21.15.191:6789/0 conn(0x7fca0c15b9a0 legacy=0x7fca0c0caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:08.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.268+0000 7fca0bfff700 1 -- 172.21.15.99:0/163199436 >> v1:172.21.15.99:6789/0 conn(0x7fca0c152300 legacy=0x7fca0c158df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:08.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.268+0000 7fca0bfff700 1 -- 172.21.15.99:0/163199436 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca0c15c030 con 0x7fca0c06d370 2023-10-08T00:11:08.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.268+0000 7fca0bfff700 1 --2- 172.21.15.99:0/163199436 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca0c06d370 0x7fca0c06d790 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fca04009e60 tx=0x7fca04009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b1bdba2377318f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:08.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.269+0000 7fca0a7fc700 1 -- 172.21.15.99:0/163199436 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0400e070 con 0x7fca0c06d370 2023-10-08T00:11:08.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.269+0000 7fca0a7fc700 1 -- 172.21.15.99:0/163199436 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca0400b7a0 con 0x7fca0c06d370 2023-10-08T00:11:08.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.269+0000 7fca0a7fc700 1 -- 172.21.15.99:0/163199436 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9fc001470 con 0x7fca0c06d370 2023-10-08T00:11:08.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.269+0000 7fca11c20700 1 -- 172.21.15.99:0/163199436 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca0c06d370 msgr2=0x7fca0c06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:08.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.269+0000 7fca11c20700 1 --2- 172.21.15.99:0/163199436 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca0c06d370 0x7fca0c06d790 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fca04009e60 tx=0x7fca04009780 comp rx=0 tx=0).stop 2023-10-08T00:11:08.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.269+0000 7fca11c20700 1 -- 172.21.15.99:0/163199436 shutdown_connections 2023-10-08T00:11:08.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.269+0000 7fca11c20700 1 --2- 172.21.15.99:0/163199436 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca0c06d370 0x7fca0c06d790 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:08.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.269+0000 7fca11c20700 1 -- 172.21.15.99:0/163199436 >> 172.21.15.99:0/163199436 conn(0x7fca0c069870 msgr2=0x7fca0c069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:08.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.269+0000 7fca11c20700 1 -- 172.21.15.99:0/163199436 shutdown_connections 2023-10-08T00:11:08.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.269+0000 7fca11c20700 1 -- 172.21.15.99:0/163199436 wait complete. 2023-10-08T00:11:08.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.270+0000 7fca11c20700 1 Processor -- start 2023-10-08T00:11:08.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.270+0000 7fca11c20700 1 -- start start 2023-10-08T00:11:08.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.271+0000 7fca11c20700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca0c06d370 0x7fca0c140750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:08.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.271+0000 7fca11c20700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca0c152300 0x7fca0c0c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:08.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.271+0000 7fca11c20700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca0c15b9a0 0x7fca0c150c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:08.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.271+0000 7fca11c20700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0c16a620 con 0x7fca0c152300 2023-10-08T00:11:08.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.271+0000 7fca11c20700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0c169a10 con 0x7fca0c06d370 2023-10-08T00:11:08.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.271+0000 7fca11c20700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca0c169cf0 con 0x7fca0c15b9a0 2023-10-08T00:11:08.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.271+0000 7fca0affd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca0c152300 0x7fca0c0c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:08.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.271+0000 7fca0affd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca0c152300 0x7fca0c0c80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38940/0 (socket says 172.21.15.99:38940) 2023-10-08T00:11:08.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.271+0000 7fca0affd700 1 -- 172.21.15.99:0/3479739137 learned_addr learned my addr 172.21.15.99:0/3479739137 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:08.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.271+0000 7fca0b7fe700 1 --2- 172.21.15.99:0/3479739137 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca0c06d370 0x7fca0c140750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:08.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.271+0000 7fca0bfff700 1 --2- 172.21.15.99:0/3479739137 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca0c15b9a0 0x7fca0c150c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:08.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.272+0000 7fca0affd700 1 -- 172.21.15.99:0/3479739137 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca0c15b9a0 msgr2=0x7fca0c150c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:08.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.272+0000 7fca0affd700 1 --2- 172.21.15.99:0/3479739137 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca0c15b9a0 0x7fca0c150c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:08.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.272+0000 7fca0affd700 1 -- 172.21.15.99:0/3479739137 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca0c06d370 msgr2=0x7fca0c140750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:08.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.272+0000 7fca0affd700 1 --2- 172.21.15.99:0/3479739137 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca0c06d370 0x7fca0c140750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:08.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.272+0000 7fca0affd700 1 -- 172.21.15.99:0/3479739137 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca04009a20 con 0x7fca0c152300 2023-10-08T00:11:08.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.272+0000 7fca0bfff700 1 --2- 172.21.15.99:0/3479739137 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca0c15b9a0 0x7fca0c150c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:11:08.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.272+0000 7fca0affd700 1 --2- 172.21.15.99:0/3479739137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca0c152300 0x7fca0c0c80f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fca0000c580 tx=0x7fca00007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:08.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.272+0000 7fc9f37fe700 1 -- 172.21.15.99:0/3479739137 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca00011840 con 0x7fca0c152300 2023-10-08T00:11:08.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.272+0000 7fc9f37fe700 1 -- 172.21.15.99:0/3479739137 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca000119a0 con 0x7fca0c152300 2023-10-08T00:11:08.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.272+0000 7fca11c20700 1 -- 172.21.15.99:0/3479739137 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca0c169fd0 con 0x7fca0c152300 2023-10-08T00:11:08.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.272+0000 7fc9f37fe700 1 -- 172.21.15.99:0/3479739137 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca00011d60 con 0x7fca0c152300 2023-10-08T00:11:08.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.272+0000 7fca11c20700 1 -- 172.21.15.99:0/3479739137 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca0c1526d0 con 0x7fca0c152300 2023-10-08T00:11:08.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.273+0000 7fca11c20700 1 -- 172.21.15.99:0/3479739137 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca0c16a790 con 0x7fca0c152300 2023-10-08T00:11:08.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.273+0000 7fc9f37fe700 1 -- 172.21.15.99:0/3479739137 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fca00017070 con 0x7fca0c152300 2023-10-08T00:11:08.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.274+0000 7fc9f37fe700 1 --2- 172.21.15.99:0/3479739137 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc9f8066f30 0x7fc9f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:08.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.274+0000 7fc9f37fe700 1 -- 172.21.15.99:0/3479739137 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fca00088030 con 0x7fca0c152300 2023-10-08T00:11:08.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.274+0000 7fca0b7fe700 1 --2- 172.21.15.99:0/3479739137 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc9f8066f30 0x7fc9f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:08.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.274+0000 7fca0b7fe700 1 --2- 172.21.15.99:0/3479739137 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc9f8066f30 0x7fc9f80693e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc005c20 tx=0x7fc9fc000bc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:08.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.278+0000 7fc9f37fe700 1 -- 172.21.15.99:0/3479739137 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fca000552a0 con 0x7fca0c152300 2023-10-08T00:11:08.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.458+0000 7fca11c20700 1 -- 172.21.15.99:0/3479739137 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fca0c04c3c0 con 0x7fca0c152300 2023-10-08T00:11:08.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.459+0000 7fc9f37fe700 1 -- 172.21.15.99:0/3479739137 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fca000588c0 con 0x7fca0c152300 2023-10-08T00:11:08.459 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2023-10-08T00:11:08.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.460+0000 7fca11c20700 1 -- 172.21.15.99:0/3479739137 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc9f8066f30 msgr2=0x7fc9f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:08.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.460+0000 7fca11c20700 1 --2- 172.21.15.99:0/3479739137 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc9f8066f30 0x7fc9f80693e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc005c20 tx=0x7fc9fc000bc0 comp rx=0 tx=0).stop 2023-10-08T00:11:08.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.460+0000 7fca11c20700 1 -- 172.21.15.99:0/3479739137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca0c152300 msgr2=0x7fca0c0c80f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:08.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.460+0000 7fca11c20700 1 --2- 172.21.15.99:0/3479739137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca0c152300 0x7fca0c0c80f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fca0000c580 tx=0x7fca00007e10 comp rx=0 tx=0).stop 2023-10-08T00:11:08.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.460+0000 7fca11c20700 1 -- 172.21.15.99:0/3479739137 shutdown_connections 2023-10-08T00:11:08.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.460+0000 7fca11c20700 1 --2- 172.21.15.99:0/3479739137 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc9f8066f30 0x7fc9f80693e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:08.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.460+0000 7fca11c20700 1 --2- 172.21.15.99:0/3479739137 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca0c06d370 0x7fca0c140750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:08.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.460+0000 7fca11c20700 1 --2- 172.21.15.99:0/3479739137 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca0c15b9a0 0x7fca0c150c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:08.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.461+0000 7fca11c20700 1 --2- 172.21.15.99:0/3479739137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca0c152300 0x7fca0c0c80f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:08.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.461+0000 7fca11c20700 1 -- 172.21.15.99:0/3479739137 >> 172.21.15.99:0/3479739137 conn(0x7fca0c069870 msgr2=0x7fca0c0cf490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:08.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.461+0000 7fca11c20700 1 -- 172.21.15.99:0/3479739137 shutdown_connections 2023-10-08T00:11:08.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.461+0000 7fca11c20700 1 -- 172.21.15.99:0/3479739137 wait complete. 2023-10-08T00:11:08.473 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T00:11:08.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.589+0000 7f1fec328700 1 Processor -- start 2023-10-08T00:11:08.590 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.589+0000 7f1fec328700 1 -- start start 2023-10-08T00:11:08.592 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.590+0000 7f1fec328700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1fe4159e60 0x7f1fe415e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:08.592 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.590+0000 7f1fec328700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1fe40697c0 0x7f1fe40af730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:08.592 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.590+0000 7f1fec328700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fe415ea90 con 0x7f1fe415a230 2023-10-08T00:11:08.592 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.590+0000 7f1fec328700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fe40d1cf0 con 0x7f1fe40697c0 2023-10-08T00:11:08.592 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.590+0000 7f1fec328700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe40b3100 con 0x7f1fe4159e60 2023-10-08T00:11:08.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.590+0000 7f1fea0c4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1fe4159e60 0x7f1fe415e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:08.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.590+0000 7f1fea8c5700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1fe40697c0 0x7f1fe40af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:08.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.590+0000 7f1fea0c4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1fe4159e60 0x7f1fe415e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.99:45924/0 (socket says 172.21.15.99:45924) 2023-10-08T00:11:08.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.590+0000 7f1fea0c4700 1 -- 172.21.15.99:0/3531608082 learned_addr learned my addr 172.21.15.99:0/3531608082 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:08.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.590+0000 7f1fea8c5700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1fe40697c0 0x7f1fe40af730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.99:54956/0 (socket says 172.21.15.99:54956) 2023-10-08T00:11:08.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.590+0000 7f1fea0c4700 1 -- 172.21.15.99:0/3531608082 >> v2:172.21.15.191:3300/0 conn(0x7f1fe40697c0 msgr2=0x7f1fe40af730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:08.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.590+0000 7f1fea0c4700 1 --2- 172.21.15.99:0/3531608082 >> v2:172.21.15.191:3300/0 conn(0x7f1fe40697c0 0x7f1fe40af730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:08.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.590+0000 7f1fea0c4700 1 -- 172.21.15.99:0/3531608082 >> v1:172.21.15.191:6789/0 conn(0x7f1fe415a230 legacy=0x7f1fe40690b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:08.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.590+0000 7f1fea0c4700 1 -- 172.21.15.99:0/3531608082 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe40d10e0 con 0x7f1fe4159e60 2023-10-08T00:11:08.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.591+0000 7f1fea0c4700 1 --2- 172.21.15.99:0/3531608082 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1fe4159e60 0x7f1fe415e550 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1fe00099b0 tx=0x7f1fe0009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90d06634008c871e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:08.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.591+0000 7f1fe90c2700 1 -- 172.21.15.99:0/3531608082 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe0009ea0 con 0x7f1fe4159e60 2023-10-08T00:11:08.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.591+0000 7f1fe90c2700 1 -- 172.21.15.99:0/3531608082 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fe000b860 con 0x7f1fe4159e60 2023-10-08T00:11:08.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.591+0000 7f1fe90c2700 1 -- 172.21.15.99:0/3531608082 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe000bc00 con 0x7f1fe4159e60 2023-10-08T00:11:08.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.591+0000 7f1fec328700 1 -- 172.21.15.99:0/3531608082 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1fe4159e60 msgr2=0x7f1fe415e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:08.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.591+0000 7f1fec328700 1 --2- 172.21.15.99:0/3531608082 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1fe4159e60 0x7f1fe415e550 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1fe00099b0 tx=0x7f1fe0009250 comp rx=0 tx=0).stop 2023-10-08T00:11:08.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.591+0000 7f1fec328700 1 -- 172.21.15.99:0/3531608082 shutdown_connections 2023-10-08T00:11:08.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.591+0000 7f1fec328700 1 --2- 172.21.15.99:0/3531608082 >> v2:172.21.15.191:3300/0 conn(0x7f1fe40697c0 0x7f1fe40af730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:08.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.591+0000 7f1fec328700 1 --2- 172.21.15.99:0/3531608082 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1fe4159e60 0x7f1fe415e550 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:08.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.591+0000 7f1fec328700 1 -- 172.21.15.99:0/3531608082 >> 172.21.15.99:0/3531608082 conn(0x7f1fe4070700 msgr2=0x7f1fe4070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:08.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.591+0000 7f1fec328700 1 -- 172.21.15.99:0/3531608082 shutdown_connections 2023-10-08T00:11:08.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.591+0000 7f1fec328700 1 -- 172.21.15.99:0/3531608082 wait complete. 2023-10-08T00:11:08.597 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.592+0000 7f1fec328700 1 Processor -- start 2023-10-08T00:11:08.597 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.592+0000 7f1fec328700 1 -- start start 2023-10-08T00:11:08.597 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.593+0000 7f1fec328700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1fe40697c0 0x7f1fe40caa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:08.597 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.593+0000 7f1fec328700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fe4159e60 0x7f1fe40c6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:08.597 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.593+0000 7f1fec328700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1fe415a230 0x7f1fe40c70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:08.597 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.593+0000 7f1fec328700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe40c8990 con 0x7f1fe4159e60 2023-10-08T00:11:08.598 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.593+0000 7f1fec328700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe40c7930 con 0x7f1fe415a230 2023-10-08T00:11:08.598 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.593+0000 7f1fec328700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe40c8340 con 0x7f1fe40697c0 2023-10-08T00:11:08.598 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.593+0000 7f1fea8c5700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1fe415a230 0x7f1fe40c70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:08.598 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.593+0000 7f1fea0c4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1fe40697c0 0x7f1fe40caa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:08.598 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.593+0000 7f1fe98c3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fe4159e60 0x7f1fe40c6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:08.598 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.593+0000 7f1fea8c5700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1fe415a230 0x7f1fe40c70b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.99:54972/0 (socket says 172.21.15.99:54972) 2023-10-08T00:11:08.598 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.593+0000 7f1fea8c5700 1 -- 172.21.15.99:0/3385525531 learned_addr learned my addr 172.21.15.99:0/3385525531 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:08.598 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.594+0000 7f1fea8c5700 1 -- 172.21.15.99:0/3385525531 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1fe40697c0 msgr2=0x7f1fe40caa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:08.598 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.594+0000 7f1fea8c5700 1 --2- 172.21.15.99:0/3385525531 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1fe40697c0 0x7f1fe40caa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:08.598 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.594+0000 7f1fea8c5700 1 -- 172.21.15.99:0/3385525531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fe4159e60 msgr2=0x7f1fe40c6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:08.598 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.594+0000 7f1fea8c5700 1 --2- 172.21.15.99:0/3385525531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fe4159e60 0x7f1fe40c6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:08.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.594+0000 7f1fea8c5700 1 -- 172.21.15.99:0/3385525531 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe00094f0 con 0x7f1fe415a230 2023-10-08T00:11:08.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.594+0000 7f1fe98c3700 1 --2- 172.21.15.99:0/3385525531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fe4159e60 0x7f1fe40c6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:11:08.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.594+0000 7f1fea8c5700 1 --2- 172.21.15.99:0/3385525531 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1fe415a230 0x7f1fe40c70b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc009d40 tx=0x7f1fdc00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:08.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.594+0000 7f1fda7fc700 1 -- 172.21.15.99:0/3385525531 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc009ea0 con 0x7f1fe415a230 2023-10-08T00:11:08.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.594+0000 7f1fda7fc700 1 -- 172.21.15.99:0/3385525531 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fdc00f040 con 0x7f1fe415a230 2023-10-08T00:11:08.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.594+0000 7f1fec328700 1 -- 172.21.15.99:0/3385525531 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fe40c8620 con 0x7f1fe415a230 2023-10-08T00:11:08.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.594+0000 7f1fda7fc700 1 -- 172.21.15.99:0/3385525531 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc00ccf0 con 0x7f1fe415a230 2023-10-08T00:11:08.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.594+0000 7f1fec328700 1 -- 172.21.15.99:0/3385525531 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fe40c75f0 con 0x7f1fe415a230 2023-10-08T00:11:08.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.595+0000 7f1fec328700 1 -- 172.21.15.99:0/3385525531 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fe4064650 con 0x7f1fe415a230 2023-10-08T00:11:08.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.595+0000 7f1fda7fc700 1 -- 172.21.15.99:0/3385525531 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1fdc014690 con 0x7f1fe415a230 2023-10-08T00:11:08.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.596+0000 7f1fda7fc700 1 --2- 172.21.15.99:0/3385525531 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1fc8066f30 0x7f1fc80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:08.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.596+0000 7f1fda7fc700 1 -- 172.21.15.99:0/3385525531 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1fdc085510 con 0x7f1fe415a230 2023-10-08T00:11:08.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.596+0000 7f1fea0c4700 1 --2- 172.21.15.99:0/3385525531 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1fc8066f30 0x7f1fc80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:08.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.596+0000 7f1fea0c4700 1 --2- 172.21.15.99:0/3385525531 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1fc8066f30 0x7f1fc80693e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1fe00099b0 tx=0x7f1fe0017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:08.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.600+0000 7f1fda7fc700 1 -- 172.21.15.99:0/3385525531 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1fdc0526e0 con 0x7f1fe415a230 2023-10-08T00:11:08.781 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.781+0000 7f1fec328700 1 -- 172.21.15.99:0/3385525531 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1fe404c3c0 con 0x7f1fe415a230 2023-10-08T00:11:08.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.781+0000 7f1fda7fc700 1 -- 172.21.15.99:0/3385525531 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1fdc055d00 con 0x7f1fe415a230 2023-10-08T00:11:08.782 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2023-10-08T00:11:08.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.783+0000 7f1fec328700 1 -- 172.21.15.99:0/3385525531 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1fc8066f30 msgr2=0x7f1fc80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:08.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.783+0000 7f1fec328700 1 --2- 172.21.15.99:0/3385525531 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1fc8066f30 0x7f1fc80693e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1fe00099b0 tx=0x7f1fe0017000 comp rx=0 tx=0).stop 2023-10-08T00:11:08.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.783+0000 7f1fec328700 1 -- 172.21.15.99:0/3385525531 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1fe415a230 msgr2=0x7f1fe40c70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:08.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.783+0000 7f1fec328700 1 --2- 172.21.15.99:0/3385525531 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1fe415a230 0x7f1fe40c70b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc009d40 tx=0x7f1fdc00b040 comp rx=0 tx=0).stop 2023-10-08T00:11:08.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.783+0000 7f1fec328700 1 -- 172.21.15.99:0/3385525531 shutdown_connections 2023-10-08T00:11:08.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.783+0000 7f1fec328700 1 --2- 172.21.15.99:0/3385525531 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1fc8066f30 0x7f1fc80693e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:08.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.783+0000 7f1fec328700 1 --2- 172.21.15.99:0/3385525531 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1fe415a230 0x7f1fe40c70b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:08.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.783+0000 7f1fec328700 1 --2- 172.21.15.99:0/3385525531 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1fe40697c0 0x7f1fe40caa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:08.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.783+0000 7f1fec328700 1 --2- 172.21.15.99:0/3385525531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fe4159e60 0x7f1fe40c6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:08.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.783+0000 7f1fec328700 1 -- 172.21.15.99:0/3385525531 >> 172.21.15.99:0/3385525531 conn(0x7f1fe4070700 msgr2=0x7f1fe40ccbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:08.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.783+0000 7f1fec328700 1 -- 172.21.15.99:0/3385525531 shutdown_connections 2023-10-08T00:11:08.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.783+0000 7f1fec328700 1 -- 172.21.15.99:0/3385525531 wait complete. 2023-10-08T00:11:08.796 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T00:11:08.913 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.912+0000 7f5bf4dda700 1 Processor -- start 2023-10-08T00:11:08.913 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.912+0000 7f5bf4dda700 1 -- start start 2023-10-08T00:11:08.915 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.913+0000 7f5bf4dda700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5bf0159ec0 0x7f5bf00af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:08.915 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.913+0000 7f5bf4dda700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bf00b0790 con 0x7f5bf015a290 2023-10-08T00:11:08.915 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.913+0000 7f5bf4dda700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bf00ce1f0 con 0x7f5bf006acb0 2023-10-08T00:11:08.915 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.913+0000 7f5bf4dda700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bf00b0b20 con 0x7f5bf0159ec0 2023-10-08T00:11:08.915 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.913+0000 7f5bee59c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5bf0159ec0 0x7f5bf00af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:08.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.913+0000 7f5be5d9b700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f5bf015a290 0x7f5bf006a5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45156/0 (socket says 172.21.15.99:45156) 2023-10-08T00:11:08.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.913+0000 7f5be5d9b700 1 -- 172.21.15.99:0/2783315377 learned_addr learned my addr 172.21.15.99:0/2783315377 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:08.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.913+0000 7f5bee59c700 1 -- 172.21.15.99:0/2783315377 >> v1:172.21.15.191:6789/0 conn(0x7f5bf006acb0 legacy=0x7f5bf00cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:08.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.913+0000 7f5bee59c700 1 -- 172.21.15.99:0/2783315377 >> v1:172.21.15.99:6789/0 conn(0x7f5bf015a290 legacy=0x7f5bf006a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:08.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.913+0000 7f5bee59c700 1 -- 172.21.15.99:0/2783315377 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bf00b04b0 con 0x7f5bf0159ec0 2023-10-08T00:11:08.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.913+0000 7f5bedd9b700 1 -- 172.21.15.99:0/2783315377 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 908691089 0 0) 0x7f5bf00b0790 con 0x7f5bf015a290 2023-10-08T00:11:08.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.914+0000 7f5bee59c700 1 --2- 172.21.15.99:0/2783315377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5bf0159ec0 0x7f5bf00af940 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5be00098e0 tx=0x7f5be00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fafc38169d9fcf3c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:08.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.914+0000 7f5bedd9b700 1 -- 172.21.15.99:0/2783315377 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be0009e80 con 0x7f5bf0159ec0 2023-10-08T00:11:08.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.914+0000 7f5bedd9b700 1 -- 172.21.15.99:0/2783315377 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5be000b820 con 0x7f5bf0159ec0 2023-10-08T00:11:08.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.914+0000 7f5bedd9b700 1 -- 172.21.15.99:0/2783315377 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be000bba0 con 0x7f5bf0159ec0 2023-10-08T00:11:08.917 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.914+0000 7f5bf4dda700 1 -- 172.21.15.99:0/2783315377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5bf0159ec0 msgr2=0x7f5bf00af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:08.917 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.914+0000 7f5bf4dda700 1 --2- 172.21.15.99:0/2783315377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5bf0159ec0 0x7f5bf00af940 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5be00098e0 tx=0x7f5be00091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:08.917 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.914+0000 7f5bf4dda700 1 -- 172.21.15.99:0/2783315377 shutdown_connections 2023-10-08T00:11:08.917 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.914+0000 7f5bf4dda700 1 --2- 172.21.15.99:0/2783315377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5bf0159ec0 0x7f5bf00af940 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:08.917 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.914+0000 7f5bf4dda700 1 -- 172.21.15.99:0/2783315377 >> 172.21.15.99:0/2783315377 conn(0x7f5bf00d1880 msgr2=0x7f5bf00cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:08.917 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.914+0000 7f5bf4dda700 1 -- 172.21.15.99:0/2783315377 shutdown_connections 2023-10-08T00:11:08.917 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.914+0000 7f5bf4dda700 1 -- 172.21.15.99:0/2783315377 wait complete. 2023-10-08T00:11:08.920 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.915+0000 7f5bf4dda700 1 Processor -- start 2023-10-08T00:11:08.920 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.916+0000 7f5bf4dda700 1 -- start start 2023-10-08T00:11:08.920 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.916+0000 7f5bf4dda700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5bf006acb0 0x7f5bf0143f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:08.920 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.916+0000 7f5bf4dda700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5bf0159ec0 0x7f5bf00ca300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:08.920 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.916+0000 7f5bf4dda700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5bf015a290 0x7f5bf00cac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:08.920 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.916+0000 7f5bf4dda700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf00c9e70 con 0x7f5bf015a290 2023-10-08T00:11:08.920 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.916+0000 7f5bf4dda700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf00c9260 con 0x7f5bf006acb0 2023-10-08T00:11:08.921 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.916+0000 7f5bf4dda700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf00c9540 con 0x7f5bf0159ec0 2023-10-08T00:11:08.921 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.916+0000 7f5beed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5bf015a290 0x7f5bf00cac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:08.921 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.916+0000 7f5beed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5bf015a290 0x7f5bf00cac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38974/0 (socket says 172.21.15.99:38974) 2023-10-08T00:11:08.921 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.916+0000 7f5beed9d700 1 -- 172.21.15.99:0/2402583707 learned_addr learned my addr 172.21.15.99:0/2402583707 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:08.921 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.916+0000 7f5bee59c700 1 --2- 172.21.15.99:0/2402583707 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5bf006acb0 0x7f5bf0143f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:08.921 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.916+0000 7f5be5d9b700 1 --2- 172.21.15.99:0/2402583707 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5bf0159ec0 0x7f5bf00ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:08.921 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.917+0000 7f5beed9d700 1 -- 172.21.15.99:0/2402583707 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5bf0159ec0 msgr2=0x7f5bf00ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:08.922 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.917+0000 7f5beed9d700 1 --2- 172.21.15.99:0/2402583707 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5bf0159ec0 0x7f5bf00ca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:08.922 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.917+0000 7f5beed9d700 1 -- 172.21.15.99:0/2402583707 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5bf006acb0 msgr2=0x7f5bf0143f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:08.922 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.917+0000 7f5beed9d700 1 --2- 172.21.15.99:0/2402583707 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5bf006acb0 0x7f5bf0143f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:08.922 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.917+0000 7f5beed9d700 1 -- 172.21.15.99:0/2402583707 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be0009420 con 0x7f5bf015a290 2023-10-08T00:11:08.922 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.917+0000 7f5beed9d700 1 --2- 172.21.15.99:0/2402583707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5bf015a290 0x7f5bf00cac40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5be800c860 tx=0x7f5be8002680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:08.923 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.917+0000 7f5be6ffd700 1 -- 172.21.15.99:0/2402583707 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be8012630 con 0x7f5bf015a290 2023-10-08T00:11:08.923 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.917+0000 7f5be6ffd700 1 -- 172.21.15.99:0/2402583707 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5be8012790 con 0x7f5bf015a290 2023-10-08T00:11:08.923 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.917+0000 7f5be6ffd700 1 -- 172.21.15.99:0/2402583707 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be8012b50 con 0x7f5bf015a290 2023-10-08T00:11:08.923 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.917+0000 7f5bf4dda700 1 -- 172.21.15.99:0/2402583707 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bf00c9820 con 0x7f5bf015a290 2023-10-08T00:11:08.923 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.917+0000 7f5bf4dda700 1 -- 172.21.15.99:0/2402583707 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bf0144470 con 0x7f5bf015a290 2023-10-08T00:11:08.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.918+0000 7f5bf4dda700 1 -- 172.21.15.99:0/2402583707 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bf004c3c0 con 0x7f5bf015a290 2023-10-08T00:11:08.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.918+0000 7f5be6ffd700 1 -- 172.21.15.99:0/2402583707 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5be801b400 con 0x7f5bf015a290 2023-10-08T00:11:08.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.918+0000 7f5be6ffd700 1 --2- 172.21.15.99:0/2402583707 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5bd0066f30 0x7f5bd00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:08.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.919+0000 7f5be6ffd700 1 -- 172.21.15.99:0/2402583707 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5be8084810 con 0x7f5bf015a290 2023-10-08T00:11:08.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.919+0000 7f5bee59c700 1 --2- 172.21.15.99:0/2402583707 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5bd0066f30 0x7f5bd00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:08.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.919+0000 7f5bee59c700 1 --2- 172.21.15.99:0/2402583707 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5bd0066f30 0x7f5bd00693e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5be00030c0 tx=0x7f5be0016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:08.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:08.923+0000 7f5be6ffd700 1 -- 172.21.15.99:0/2402583707 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5be8051a80 con 0x7f5bf015a290 2023-10-08T00:11:09.106 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.105+0000 7f5bf4dda700 1 -- 172.21.15.99:0/2402583707 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5bf00c9fb0 con 0x7f5bf015a290 2023-10-08T00:11:09.106 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.106+0000 7f5be6ffd700 1 -- 172.21.15.99:0/2402583707 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5be80550a0 con 0x7f5bf015a290 2023-10-08T00:11:09.107 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2023-10-08T00:11:09.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.108+0000 7f5bf4dda700 1 -- 172.21.15.99:0/2402583707 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5bd0066f30 msgr2=0x7f5bd00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:09.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.108+0000 7f5bf4dda700 1 --2- 172.21.15.99:0/2402583707 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5bd0066f30 0x7f5bd00693e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5be00030c0 tx=0x7f5be0016000 comp rx=0 tx=0).stop 2023-10-08T00:11:09.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.108+0000 7f5bf4dda700 1 -- 172.21.15.99:0/2402583707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5bf015a290 msgr2=0x7f5bf00cac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:09.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.108+0000 7f5bf4dda700 1 --2- 172.21.15.99:0/2402583707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5bf015a290 0x7f5bf00cac40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5be800c860 tx=0x7f5be8002680 comp rx=0 tx=0).stop 2023-10-08T00:11:09.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.108+0000 7f5bf4dda700 1 -- 172.21.15.99:0/2402583707 shutdown_connections 2023-10-08T00:11:09.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.108+0000 7f5bf4dda700 1 --2- 172.21.15.99:0/2402583707 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5bd0066f30 0x7f5bd00693e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:09.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.108+0000 7f5bf4dda700 1 --2- 172.21.15.99:0/2402583707 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5bf006acb0 0x7f5bf0143f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:09.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.108+0000 7f5bf4dda700 1 --2- 172.21.15.99:0/2402583707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5bf015a290 0x7f5bf00cac40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:09.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.108+0000 7f5bf4dda700 1 --2- 172.21.15.99:0/2402583707 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5bf0159ec0 0x7f5bf00ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:09.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.108+0000 7f5bf4dda700 1 -- 172.21.15.99:0/2402583707 >> 172.21.15.99:0/2402583707 conn(0x7f5bf00d1880 msgr2=0x7f5bf00b5b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:09.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.108+0000 7f5bf4dda700 1 -- 172.21.15.99:0/2402583707 shutdown_connections 2023-10-08T00:11:09.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.108+0000 7f5bf4dda700 1 -- 172.21.15.99:0/2402583707 wait complete. 2023-10-08T00:11:09.120 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:11:09.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.234+0000 7facd493f700 1 Processor -- start 2023-10-08T00:11:09.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.234+0000 7facd493f700 1 -- start start 2023-10-08T00:11:09.242 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.235+0000 7facd493f700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7facd00a79f0 0x7facd006eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:09.242 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.235+0000 7facd493f700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7facd00a7dc0 0x7facd006eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:09.242 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.235+0000 7facd493f700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facd00b26d0 con 0x7facd00b15e0 2023-10-08T00:11:09.242 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.235+0000 7facd493f700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facd00aeab0 con 0x7facd00a79f0 2023-10-08T00:11:09.243 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.235+0000 7facd493f700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facd00b2b60 con 0x7facd00a7dc0 2023-10-08T00:11:09.243 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.235+0000 7facce59c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7facd00a79f0 0x7facd006eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:09.243 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.235+0000 7facce59c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7facd00a79f0 0x7facd006eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38976/0 (socket says 172.21.15.99:38976) 2023-10-08T00:11:09.243 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.235+0000 7facce59c700 1 -- 172.21.15.99:0/2050295282 learned_addr learned my addr 172.21.15.99:0/2050295282 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:09.243 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.236+0000 7facce59c700 1 -- 172.21.15.99:0/2050295282 >> v2:172.21.15.191:3300/0 conn(0x7facd00a7dc0 msgr2=0x7facd006eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:09.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.236+0000 7facce59c700 1 --2- 172.21.15.99:0/2050295282 >> v2:172.21.15.191:3300/0 conn(0x7facd00a7dc0 0x7facd006eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:09.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.236+0000 7facce59c700 1 -- 172.21.15.99:0/2050295282 >> v1:172.21.15.191:6789/0 conn(0x7facd00b15e0 legacy=0x7facd015dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:09.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.236+0000 7facce59c700 1 -- 172.21.15.99:0/2050295282 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facd00b1ac0 con 0x7facd00a79f0 2023-10-08T00:11:09.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.236+0000 7facce59c700 1 --2- 172.21.15.99:0/2050295282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7facd00a79f0 0x7facd006eab0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7facd006b8a0 tx=0x7facc0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36244578b609bad0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:09.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.236+0000 7faccdd9b700 1 -- 172.21.15.99:0/2050295282 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc000b820 con 0x7facd00a79f0 2023-10-08T00:11:09.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.236+0000 7faccdd9b700 1 -- 172.21.15.99:0/2050295282 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facc000b980 con 0x7facd00a79f0 2023-10-08T00:11:09.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.236+0000 7faccdd9b700 1 -- 172.21.15.99:0/2050295282 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc000bd20 con 0x7facd00a79f0 2023-10-08T00:11:09.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.236+0000 7facd493f700 1 -- 172.21.15.99:0/2050295282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7facd00a79f0 msgr2=0x7facd006eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:09.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.236+0000 7facd493f700 1 --2- 172.21.15.99:0/2050295282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7facd00a79f0 0x7facd006eab0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7facd006b8a0 tx=0x7facc0009250 comp rx=0 tx=0).stop 2023-10-08T00:11:09.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.236+0000 7facd493f700 1 -- 172.21.15.99:0/2050295282 shutdown_connections 2023-10-08T00:11:09.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.236+0000 7facd493f700 1 --2- 172.21.15.99:0/2050295282 >> v2:172.21.15.191:3300/0 conn(0x7facd00a7dc0 0x7facd006eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:09.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.236+0000 7facd493f700 1 --2- 172.21.15.99:0/2050295282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7facd00a79f0 0x7facd006eab0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:09.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.236+0000 7facd493f700 1 -- 172.21.15.99:0/2050295282 >> 172.21.15.99:0/2050295282 conn(0x7facd00cd1e0 msgr2=0x7facd00cd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:09.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.236+0000 7facd493f700 1 -- 172.21.15.99:0/2050295282 shutdown_connections 2023-10-08T00:11:09.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.237+0000 7facd493f700 1 -- 172.21.15.99:0/2050295282 wait complete. 2023-10-08T00:11:09.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.238+0000 7facd493f700 1 Processor -- start 2023-10-08T00:11:09.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.238+0000 7facd493f700 1 -- start start 2023-10-08T00:11:09.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.238+0000 7facd493f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7facd00a79f0 0x7facd00c38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:09.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.238+0000 7facd493f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7facd00a7dc0 0x7facd006ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:09.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.238+0000 7facd493f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7facd00b15e0 0x7facd00c2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:09.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.238+0000 7facd493f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd00c46e0 con 0x7facd00a7dc0 2023-10-08T00:11:09.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.238+0000 7facd493f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd00c1f80 con 0x7facd00a79f0 2023-10-08T00:11:09.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.238+0000 7facd493f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facd00c2500 con 0x7facd00b15e0 2023-10-08T00:11:09.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.238+0000 7facc7fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7facd00a7dc0 0x7facd006ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:09.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.238+0000 7facc7fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7facd00a7dc0 0x7facd006ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38992/0 (socket says 172.21.15.99:38992) 2023-10-08T00:11:09.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.238+0000 7facc7fff700 1 -- 172.21.15.99:0/1276707570 learned_addr learned my addr 172.21.15.99:0/1276707570 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:09.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.239+0000 7facce59c700 1 --2- 172.21.15.99:0/1276707570 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7facd00a79f0 0x7facd00c38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:09.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.239+0000 7facced9d700 1 --2- 172.21.15.99:0/1276707570 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7facd00b15e0 0x7facd00c2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:09.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.239+0000 7facc7fff700 1 -- 172.21.15.99:0/1276707570 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7facd00b15e0 msgr2=0x7facd00c2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:09.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.239+0000 7facc7fff700 1 --2- 172.21.15.99:0/1276707570 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7facd00b15e0 0x7facd00c2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:09.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.239+0000 7facc7fff700 1 -- 172.21.15.99:0/1276707570 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7facd00a79f0 msgr2=0x7facd00c38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:09.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.239+0000 7facc7fff700 1 --2- 172.21.15.99:0/1276707570 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7facd00a79f0 0x7facd00c38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:09.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.239+0000 7facc7fff700 1 -- 172.21.15.99:0/1276707570 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc00094f0 con 0x7facd00a7dc0 2023-10-08T00:11:09.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.239+0000 7facce59c700 1 --2- 172.21.15.99:0/1276707570 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7facd00a79f0 0x7facd00c38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:11:09.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.239+0000 7facc7fff700 1 --2- 172.21.15.99:0/1276707570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7facd00a7dc0 0x7facd006ea70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7facb800b5e0 tx=0x7facb80090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:09.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.239+0000 7facc67fc700 1 -- 172.21.15.99:0/1276707570 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb800f510 con 0x7facd00a7dc0 2023-10-08T00:11:09.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.239+0000 7facc67fc700 1 -- 172.21.15.99:0/1276707570 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facb800f690 con 0x7facd00a7dc0 2023-10-08T00:11:09.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.239+0000 7facd493f700 1 -- 172.21.15.99:0/1276707570 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facd00c4120 con 0x7facd00a7dc0 2023-10-08T00:11:09.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.239+0000 7facc67fc700 1 -- 172.21.15.99:0/1276707570 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb800fa50 con 0x7facd00a7dc0 2023-10-08T00:11:09.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.239+0000 7facd493f700 1 -- 172.21.15.99:0/1276707570 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facd00c44b0 con 0x7facd00a7dc0 2023-10-08T00:11:09.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.240+0000 7facd493f700 1 -- 172.21.15.99:0/1276707570 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facd004c3c0 con 0x7facd00a7dc0 2023-10-08T00:11:09.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.240+0000 7facc67fc700 1 -- 172.21.15.99:0/1276707570 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7facb801a400 con 0x7facd00a7dc0 2023-10-08T00:11:09.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.240+0000 7facc67fc700 1 --2- 172.21.15.99:0/1276707570 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7facb0066f30 0x7facb00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:09.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.241+0000 7facc67fc700 1 -- 172.21.15.99:0/1276707570 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7facb8084080 con 0x7facd00a7dc0 2023-10-08T00:11:09.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.241+0000 7facce59c700 1 --2- 172.21.15.99:0/1276707570 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7facb0066f30 0x7facb00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:09.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.241+0000 7facce59c700 1 --2- 172.21.15.99:0/1276707570 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7facb0066f30 0x7facb00693e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7facd006b8a0 tx=0x7facc0016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:09.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.245+0000 7facc67fc700 1 -- 172.21.15.99:0/1276707570 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7facb8051370 con 0x7facd00a7dc0 2023-10-08T00:11:09.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.468+0000 7facd493f700 1 -- 172.21.15.99:0/1276707570 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7facd00c2640 con 0x7facd00a7dc0 2023-10-08T00:11:09.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.469+0000 7facc67fc700 1 -- 172.21.15.99:0/1276707570 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4594 (secure 0 0 0) 0x7facb8054990 con 0x7facd00a7dc0 2023-10-08T00:11:09.470 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:11:09.470 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/649537180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":649537180},{"type":"v1","addr":"172.21.15.99:6833","nonce":649537180}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4239,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2194110899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2194110899},{"type":"v1","addr":"172.21.15.99:6835","nonce":2194110899}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4251,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/1732443753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":1732443753},{"type":"v1","addr":"172.21.15.191:6837","nonce":1732443753}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:10:53.454321+0000","modified":"2023-10-08T00:10:57.098731+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6836/1988845948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1988845948},{"type":"v1","addr":"172.21.15.191:6836","nonce":1988845948}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-08T00:11:09.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.471+0000 7facd493f700 1 -- 172.21.15.99:0/1276707570 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7facb0066f30 msgr2=0x7facb00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:09.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.471+0000 7facd493f700 1 --2- 172.21.15.99:0/1276707570 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7facb0066f30 0x7facb00693e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7facd006b8a0 tx=0x7facc0016000 comp rx=0 tx=0).stop 2023-10-08T00:11:09.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.471+0000 7facd493f700 1 -- 172.21.15.99:0/1276707570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7facd00a7dc0 msgr2=0x7facd006ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:09.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.471+0000 7facd493f700 1 --2- 172.21.15.99:0/1276707570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7facd00a7dc0 0x7facd006ea70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7facb800b5e0 tx=0x7facb80090f0 comp rx=0 tx=0).stop 2023-10-08T00:11:09.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.471+0000 7facd493f700 1 -- 172.21.15.99:0/1276707570 shutdown_connections 2023-10-08T00:11:09.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.471+0000 7facd493f700 1 --2- 172.21.15.99:0/1276707570 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7facb0066f30 0x7facb00693e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:09.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.471+0000 7facd493f700 1 --2- 172.21.15.99:0/1276707570 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7facd00a79f0 0x7facd00c38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:09.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.471+0000 7facd493f700 1 --2- 172.21.15.99:0/1276707570 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7facd00b15e0 0x7facd00c2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:09.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.471+0000 7facd493f700 1 --2- 172.21.15.99:0/1276707570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7facd00a7dc0 0x7facd006ea70 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:09.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.471+0000 7facd493f700 1 -- 172.21.15.99:0/1276707570 >> 172.21.15.99:0/1276707570 conn(0x7facd00cd1e0 msgr2=0x7facd006daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:09.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.471+0000 7facd493f700 1 -- 172.21.15.99:0/1276707570 shutdown_connections 2023-10-08T00:11:09.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.471+0000 7facd493f700 1 -- 172.21.15.99:0/1276707570 wait complete. 2023-10-08T00:11:09.473 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 8 2023-10-08T00:11:09.484 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:11:09.597 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.596+0000 7f104728c700 1 Processor -- start 2023-10-08T00:11:09.597 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.596+0000 7f104728c700 1 -- start start 2023-10-08T00:11:09.597 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.597+0000 7f104728c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f10400c75c0 0x7f10400c8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:09.598 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.597+0000 7f104728c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f10400abeb0 0x7f10400c8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:09.598 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.597+0000 7f104728c700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10400c9cb0 con 0x7f10400abae0 2023-10-08T00:11:09.598 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.597+0000 7f104728c700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10400c8e80 con 0x7f10400abeb0 2023-10-08T00:11:09.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.597+0000 7f104728c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10400c90a0 con 0x7f10400c75c0 2023-10-08T00:11:09.601 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.597+0000 7f1044827700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f10400c75c0 0x7f10400c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:09.601 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.597+0000 7f1045829700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f10400abeb0 0x7f10400c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:09.601 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.598+0000 7f1045028700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f10400abae0 0x7f10400c6eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:56200/0 (socket says 172.21.15.99:56200) 2023-10-08T00:11:09.601 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.598+0000 7f1045829700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f10400abeb0 0x7f10400c8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.99:54994/0 (socket says 172.21.15.99:54994) 2023-10-08T00:11:09.601 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.598+0000 7f1044827700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f10400c75c0 0x7f10400c8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.99:45954/0 (socket says 172.21.15.99:45954) 2023-10-08T00:11:09.601 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.598+0000 7f1045028700 1 -- 172.21.15.99:0/1317814383 learned_addr learned my addr 172.21.15.99:0/1317814383 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:09.601 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.598+0000 7f1044827700 1 -- 172.21.15.99:0/1317814383 >> v2:172.21.15.191:3300/0 conn(0x7f10400abeb0 msgr2=0x7f10400c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:09.601 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.598+0000 7f1044827700 1 --2- 172.21.15.99:0/1317814383 >> v2:172.21.15.191:3300/0 conn(0x7f10400abeb0 0x7f10400c8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:09.601 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.598+0000 7f1044827700 1 -- 172.21.15.99:0/1317814383 >> v1:172.21.15.191:6789/0 conn(0x7f10400abae0 legacy=0x7f10400c6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:09.602 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.598+0000 7f1044827700 1 -- 172.21.15.99:0/1317814383 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10400c9380 con 0x7f10400c75c0 2023-10-08T00:11:09.602 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.598+0000 7f1045829700 1 --2- 172.21.15.99:0/1317814383 >> v2:172.21.15.191:3300/0 conn(0x7f10400abeb0 0x7f10400c8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:11:09.602 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.598+0000 7f1044827700 1 --2- 172.21.15.99:0/1317814383 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f10400c75c0 0x7f10400c8400 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f10300098e0 tx=0x7f10300091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb7266b739a46502 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:09.602 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.598+0000 7f1037fff700 1 -- 172.21.15.99:0/1317814383 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103000b640 con 0x7f10400c75c0 2023-10-08T00:11:09.602 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.598+0000 7f1037fff700 1 -- 172.21.15.99:0/1317814383 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f103000b7a0 con 0x7f10400c75c0 2023-10-08T00:11:09.602 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.598+0000 7f1037fff700 1 -- 172.21.15.99:0/1317814383 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103000bb40 con 0x7f10400c75c0 2023-10-08T00:11:09.602 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.599+0000 7f104728c700 1 -- 172.21.15.99:0/1317814383 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f10400c75c0 msgr2=0x7f10400c8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:09.602 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.599+0000 7f104728c700 1 --2- 172.21.15.99:0/1317814383 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f10400c75c0 0x7f10400c8400 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f10300098e0 tx=0x7f10300091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:09.602 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.599+0000 7f104728c700 1 -- 172.21.15.99:0/1317814383 shutdown_connections 2023-10-08T00:11:09.602 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.599+0000 7f104728c700 1 --2- 172.21.15.99:0/1317814383 >> v2:172.21.15.191:3300/0 conn(0x7f10400abeb0 0x7f10400c8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:09.602 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.599+0000 7f104728c700 1 --2- 172.21.15.99:0/1317814383 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f10400c75c0 0x7f10400c8400 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:09.602 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.599+0000 7f104728c700 1 -- 172.21.15.99:0/1317814383 >> 172.21.15.99:0/1317814383 conn(0x7f1040067660 msgr2=0x7f1040067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:09.603 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.599+0000 7f104728c700 1 -- 172.21.15.99:0/1317814383 shutdown_connections 2023-10-08T00:11:09.603 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.599+0000 7f104728c700 1 -- 172.21.15.99:0/1317814383 wait complete. 2023-10-08T00:11:09.603 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.600+0000 7f104728c700 1 Processor -- start 2023-10-08T00:11:09.603 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.600+0000 7f104728c700 1 -- start start 2023-10-08T00:11:09.605 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.601+0000 7f104728c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10400abae0 0x7f10401587f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:09.605 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.601+0000 7f104728c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f10400abeb0 0x7f10400c14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:09.605 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.601+0000 7f104728c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f10400c75c0 0x7f10400c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:09.605 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.601+0000 7f104728c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10400c1010 con 0x7f10400abae0 2023-10-08T00:11:09.605 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.601+0000 7f104728c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1040158d30 con 0x7f10400c75c0 2023-10-08T00:11:09.605 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.601+0000 7f104728c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10400c0400 con 0x7f10400abeb0 2023-10-08T00:11:09.606 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.601+0000 7f1045028700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10400abae0 0x7f10401587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:09.606 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.601+0000 7f1045028700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10400abae0 0x7f10401587f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39002/0 (socket says 172.21.15.99:39002) 2023-10-08T00:11:09.606 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.601+0000 7f1045028700 1 -- 172.21.15.99:0/1989282795 learned_addr learned my addr 172.21.15.99:0/1989282795 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:09.606 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.601+0000 7f1044827700 1 --2- 172.21.15.99:0/1989282795 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f10400abeb0 0x7f10400c14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:09.606 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.601+0000 7f1045829700 1 --2- 172.21.15.99:0/1989282795 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f10400c75c0 0x7f10400c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:09.606 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.601+0000 7f1045028700 1 -- 172.21.15.99:0/1989282795 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f10400abeb0 msgr2=0x7f10400c14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:09.606 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.601+0000 7f1045028700 1 --2- 172.21.15.99:0/1989282795 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f10400abeb0 0x7f10400c14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:09.606 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.601+0000 7f1045028700 1 -- 172.21.15.99:0/1989282795 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f10400c75c0 msgr2=0x7f10400c5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:09.606 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.601+0000 7f1045028700 1 --2- 172.21.15.99:0/1989282795 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f10400c75c0 0x7f10400c5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:09.606 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.601+0000 7f1045028700 1 -- 172.21.15.99:0/1989282795 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1038009420 con 0x7f10400abae0 2023-10-08T00:11:09.606 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.601+0000 7f1045028700 1 --2- 172.21.15.99:0/1989282795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10400abae0 0x7f10401587f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f103c00ef20 tx=0x7f103c001b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:09.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.602+0000 7f10357fa700 1 -- 172.21.15.99:0/1989282795 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103c00ce00 con 0x7f10400abae0 2023-10-08T00:11:09.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.602+0000 7f10357fa700 1 -- 172.21.15.99:0/1989282795 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f103c011470 con 0x7f10400abae0 2023-10-08T00:11:09.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.602+0000 7f10357fa700 1 -- 172.21.15.99:0/1989282795 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103c011830 con 0x7f10400abae0 2023-10-08T00:11:09.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.602+0000 7f104728c700 1 -- 172.21.15.99:0/1989282795 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1030009420 con 0x7f10400abae0 2023-10-08T00:11:09.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.602+0000 7f104728c700 1 -- 172.21.15.99:0/1989282795 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10400c09c0 con 0x7f10400abae0 2023-10-08T00:11:09.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.603+0000 7f104728c700 1 -- 172.21.15.99:0/1989282795 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10400c12f0 con 0x7f10400abae0 2023-10-08T00:11:09.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.603+0000 7f10357fa700 1 -- 172.21.15.99:0/1989282795 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f103c011ba0 con 0x7f10400abae0 2023-10-08T00:11:09.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.603+0000 7f10357fa700 1 --2- 172.21.15.99:0/1989282795 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1024066f30 0x7f10240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:09.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.603+0000 7f10357fa700 1 -- 172.21.15.99:0/1989282795 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f103c087090 con 0x7f10400abae0 2023-10-08T00:11:09.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.604+0000 7f1044827700 1 --2- 172.21.15.99:0/1989282795 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1024066f30 0x7f10240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:09.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.604+0000 7f1044827700 1 --2- 172.21.15.99:0/1989282795 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1024066f30 0x7f10240693e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f10300091e0 tx=0x7f1030017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:09.608 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.608+0000 7f10357fa700 1 -- 172.21.15.99:0/1989282795 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f103c054440 con 0x7f10400abae0 2023-10-08T00:11:09.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.790+0000 7f104728c700 1 -- 172.21.15.99:0/1989282795 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f104004bb80 con 0x7f10400abae0 2023-10-08T00:11:09.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.791+0000 7f10357fa700 1 -- 172.21.15.99:0/1989282795 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f103c057a60 con 0x7f10400abae0 2023-10-08T00:11:09.792 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:11:09.792 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":20,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:10:55.080854+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:10:51.822157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T00:10:52.415130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:11:09.794 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.793+0000 7f104728c700 1 -- 172.21.15.99:0/1989282795 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1024066f30 msgr2=0x7f10240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:09.795 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.793+0000 7f104728c700 1 --2- 172.21.15.99:0/1989282795 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1024066f30 0x7f10240693e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f10300091e0 tx=0x7f1030017000 comp rx=0 tx=0).stop 2023-10-08T00:11:09.795 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.793+0000 7f104728c700 1 -- 172.21.15.99:0/1989282795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10400abae0 msgr2=0x7f10401587f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:09.795 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.793+0000 7f104728c700 1 --2- 172.21.15.99:0/1989282795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10400abae0 0x7f10401587f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f103c00ef20 tx=0x7f103c001b20 comp rx=0 tx=0).stop 2023-10-08T00:11:09.795 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.793+0000 7f104728c700 1 -- 172.21.15.99:0/1989282795 shutdown_connections 2023-10-08T00:11:09.795 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.793+0000 7f104728c700 1 --2- 172.21.15.99:0/1989282795 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1024066f30 0x7f10240693e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:09.795 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.793+0000 7f104728c700 1 --2- 172.21.15.99:0/1989282795 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f10400c75c0 0x7f10400c5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:09.795 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.793+0000 7f104728c700 1 --2- 172.21.15.99:0/1989282795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10400abae0 0x7f10401587f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:09.796 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.793+0000 7f104728c700 1 --2- 172.21.15.99:0/1989282795 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f10400abeb0 0x7f10400c14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:09.796 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.793+0000 7f104728c700 1 -- 172.21.15.99:0/1989282795 >> 172.21.15.99:0/1989282795 conn(0x7f1040067660 msgr2=0x7f10400cbf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:09.796 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.793+0000 7f104728c700 1 -- 172.21.15.99:0/1989282795 shutdown_connections 2023-10-08T00:11:09.796 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.793+0000 7f104728c700 1 -- 172.21.15.99:0/1989282795 wait complete. 2023-10-08T00:11:09.806 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:10:42.063344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T00:10:51.822157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T00:10:52.415130+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T00:11:09.807 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T00:11:09.923 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.922+0000 7f1e8c8d8700 1 Processor -- start 2023-10-08T00:11:09.923 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.922+0000 7f1e8c8d8700 1 -- start start 2023-10-08T00:11:09.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.923+0000 7f1e8c8d8700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1e8815a290 0x7f1e8806a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:09.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.923+0000 7f1e8c8d8700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e880b0790 con 0x7f1e88159ec0 2023-10-08T00:11:09.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.923+0000 7f1e8c8d8700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e8806af20 con 0x7f1e8806aae0 2023-10-08T00:11:09.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.923+0000 7f1e8c8d8700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e880ce1f0 con 0x7f1e8815a290 2023-10-08T00:11:09.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.923+0000 7f1e7dd9b700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1e8815a290 0x7f1e8806a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:09.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.923+0000 7f1e7dd9b700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1e8815a290 0x7f1e8806a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39014/0 (socket says 172.21.15.99:39014) 2023-10-08T00:11:09.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.923+0000 7f1e8659c700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f1e88159ec0 0x7f1e880af940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45162/0 (socket says 172.21.15.99:45162) 2023-10-08T00:11:09.926 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.923+0000 7f1e7dd9b700 1 -- 172.21.15.99:0/2182315247 learned_addr learned my addr 172.21.15.99:0/2182315247 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:09.926 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.923+0000 7f1e7dd9b700 1 -- 172.21.15.99:0/2182315247 >> v1:172.21.15.191:6789/0 conn(0x7f1e8806aae0 legacy=0x7f1e880cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:09.926 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.923+0000 7f1e7dd9b700 1 -- 172.21.15.99:0/2182315247 >> v1:172.21.15.99:6789/0 conn(0x7f1e88159ec0 legacy=0x7f1e880af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:09.926 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.923+0000 7f1e7dd9b700 1 -- 172.21.15.99:0/2182315247 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e880b0b20 con 0x7f1e8815a290 2023-10-08T00:11:09.926 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.923+0000 7f1e85d9b700 1 -- 172.21.15.99:0/2182315247 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3718654331 0 0) 0x7f1e880b0790 con 0x7f1e88159ec0 2023-10-08T00:11:09.926 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.923+0000 7f1e7dd9b700 1 --2- 172.21.15.99:0/2182315247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e8815a290 0x7f1e8806a5a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1e700099b0 tx=0x7f1e70009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aff7f2af3ede7546 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:09.926 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.924+0000 7f1e85d9b700 1 -- 172.21.15.99:0/2182315247 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e7000b820 con 0x7f1e8815a290 2023-10-08T00:11:09.926 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.924+0000 7f1e85d9b700 1 -- 172.21.15.99:0/2182315247 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e7000b980 con 0x7f1e8815a290 2023-10-08T00:11:09.927 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.924+0000 7f1e85d9b700 1 -- 172.21.15.99:0/2182315247 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e7000bd20 con 0x7f1e8815a290 2023-10-08T00:11:09.927 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.924+0000 7f1e8c8d8700 1 -- 172.21.15.99:0/2182315247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e8815a290 msgr2=0x7f1e8806a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:09.927 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.924+0000 7f1e8c8d8700 1 --2- 172.21.15.99:0/2182315247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e8815a290 0x7f1e8806a5a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1e700099b0 tx=0x7f1e70009250 comp rx=0 tx=0).stop 2023-10-08T00:11:09.927 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.924+0000 7f1e8c8d8700 1 -- 172.21.15.99:0/2182315247 shutdown_connections 2023-10-08T00:11:09.927 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.924+0000 7f1e8c8d8700 1 --2- 172.21.15.99:0/2182315247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e8815a290 0x7f1e8806a5a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:09.927 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.924+0000 7f1e8c8d8700 1 -- 172.21.15.99:0/2182315247 >> 172.21.15.99:0/2182315247 conn(0x7f1e880d1880 msgr2=0x7f1e880cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:09.927 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.924+0000 7f1e8c8d8700 1 -- 172.21.15.99:0/2182315247 shutdown_connections 2023-10-08T00:11:09.927 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.924+0000 7f1e8c8d8700 1 -- 172.21.15.99:0/2182315247 wait complete. 2023-10-08T00:11:09.930 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.925+0000 7f1e8c8d8700 1 Processor -- start 2023-10-08T00:11:09.930 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.925+0000 7f1e8c8d8700 1 -- start start 2023-10-08T00:11:09.930 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.926+0000 7f1e8c8d8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e8806aae0 0x7f1e8816b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:09.930 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.926+0000 7f1e8c8d8700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e88159ec0 0x7f1e88155040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:09.930 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.926+0000 7f1e8c8d8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e8815a290 0x7f1e880b4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:09.930 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.926+0000 7f1e8c8d8700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e880cab80 con 0x7f1e8815a290 2023-10-08T00:11:09.931 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.926+0000 7f1e8c8d8700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8816b2a0 con 0x7f1e88159ec0 2023-10-08T00:11:09.931 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.926+0000 7f1e8c8d8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8816b9c0 con 0x7f1e8806aae0 2023-10-08T00:11:09.931 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.926+0000 7f1e86d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e8815a290 0x7f1e880b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:09.931 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.926+0000 7f1e86d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e8815a290 0x7f1e880b4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39024/0 (socket says 172.21.15.99:39024) 2023-10-08T00:11:09.931 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.926+0000 7f1e86d9d700 1 -- 172.21.15.99:0/110233060 learned_addr learned my addr 172.21.15.99:0/110233060 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:09.931 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.926+0000 7f1e8659c700 1 --2- 172.21.15.99:0/110233060 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e8806aae0 0x7f1e8816b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:09.931 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.926+0000 7f1e7dd9b700 1 --2- 172.21.15.99:0/110233060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e88159ec0 0x7f1e88155040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:09.931 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.926+0000 7f1e86d9d700 1 -- 172.21.15.99:0/110233060 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e8806aae0 msgr2=0x7f1e8816b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:09.931 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.926+0000 7f1e86d9d700 1 --2- 172.21.15.99:0/110233060 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e8806aae0 0x7f1e8816b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:09.931 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.926+0000 7f1e86d9d700 1 -- 172.21.15.99:0/110233060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e88159ec0 msgr2=0x7f1e88155040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:09.931 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.926+0000 7f1e86d9d700 1 --2- 172.21.15.99:0/110233060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e88159ec0 0x7f1e88155040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:09.931 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.926+0000 7f1e86d9d700 1 -- 172.21.15.99:0/110233060 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e700094f0 con 0x7f1e8815a290 2023-10-08T00:11:09.932 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.927+0000 7f1e86d9d700 1 --2- 172.21.15.99:0/110233060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e8815a290 0x7f1e880b4760 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1e8000c930 tx=0x7f1e800026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:09.932 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.927+0000 7f1e7effd700 1 -- 172.21.15.99:0/110233060 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e80012630 con 0x7f1e8815a290 2023-10-08T00:11:09.932 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.927+0000 7f1e7effd700 1 -- 172.21.15.99:0/110233060 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e800127b0 con 0x7f1e8815a290 2023-10-08T00:11:09.932 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.927+0000 7f1e8c8d8700 1 -- 172.21.15.99:0/110233060 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e8816bbb0 con 0x7f1e8815a290 2023-10-08T00:11:09.932 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.927+0000 7f1e7effd700 1 -- 172.21.15.99:0/110233060 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e80012b70 con 0x7f1e8815a290 2023-10-08T00:11:09.932 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.927+0000 7f1e8c8d8700 1 -- 172.21.15.99:0/110233060 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e8816b070 con 0x7f1e8815a290 2023-10-08T00:11:09.932 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.928+0000 7f1e8c8d8700 1 -- 172.21.15.99:0/110233060 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e8804c3c0 con 0x7f1e8815a290 2023-10-08T00:11:09.932 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.928+0000 7f1e7effd700 1 -- 172.21.15.99:0/110233060 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1e8001a3b0 con 0x7f1e8815a290 2023-10-08T00:11:09.932 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.928+0000 7f1e7effd700 1 --2- 172.21.15.99:0/110233060 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1e68066f30 0x7f1e680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:09.932 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.929+0000 7f1e7effd700 1 -- 172.21.15.99:0/110233060 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1e80084090 con 0x7f1e8815a290 2023-10-08T00:11:09.932 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.929+0000 7f1e8659c700 1 --2- 172.21.15.99:0/110233060 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1e68066f30 0x7f1e680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:09.933 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.929+0000 7f1e8659c700 1 --2- 172.21.15.99:0/110233060 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1e68066f30 0x7f1e680693e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1e78007fd0 tx=0x7f1e78007d50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:09.933 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:09.933+0000 7f1e7effd700 1 -- 172.21.15.99:0/110233060 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1e80051300 con 0x7f1e8815a290 2023-10-08T00:11:10.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.119+0000 7f1e8c8d8700 1 -- 172.21.15.99:0/110233060 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1e880b04b0 con 0x7f1e8815a290 2023-10-08T00:11:10.121 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.120+0000 7f1e7effd700 1 -- 172.21.15.99:0/110233060 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1e80054920 con 0x7f1e8815a290 2023-10-08T00:11:10.121 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2023-10-08T00:11:10.123 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.122+0000 7f1e8c8d8700 1 -- 172.21.15.99:0/110233060 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1e68066f30 msgr2=0x7f1e680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:10.123 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.122+0000 7f1e8c8d8700 1 --2- 172.21.15.99:0/110233060 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1e68066f30 0x7f1e680693e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1e78007fd0 tx=0x7f1e78007d50 comp rx=0 tx=0).stop 2023-10-08T00:11:10.123 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.122+0000 7f1e8c8d8700 1 -- 172.21.15.99:0/110233060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e8815a290 msgr2=0x7f1e880b4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:10.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.122+0000 7f1e8c8d8700 1 --2- 172.21.15.99:0/110233060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e8815a290 0x7f1e880b4760 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1e8000c930 tx=0x7f1e800026e0 comp rx=0 tx=0).stop 2023-10-08T00:11:10.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.122+0000 7f1e8c8d8700 1 -- 172.21.15.99:0/110233060 shutdown_connections 2023-10-08T00:11:10.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.122+0000 7f1e8c8d8700 1 --2- 172.21.15.99:0/110233060 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1e68066f30 0x7f1e680693e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:10.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.122+0000 7f1e8c8d8700 1 --2- 172.21.15.99:0/110233060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e8815a290 0x7f1e880b4760 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:10.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.122+0000 7f1e8c8d8700 1 --2- 172.21.15.99:0/110233060 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e8806aae0 0x7f1e8816b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:10.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.122+0000 7f1e8c8d8700 1 --2- 172.21.15.99:0/110233060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e88159ec0 0x7f1e88155040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:10.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.122+0000 7f1e8c8d8700 1 -- 172.21.15.99:0/110233060 >> 172.21.15.99:0/110233060 conn(0x7f1e880d1880 msgr2=0x7f1e881574c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:10.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.122+0000 7f1e8c8d8700 1 -- 172.21.15.99:0/110233060 shutdown_connections 2023-10-08T00:11:10.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.122+0000 7f1e8c8d8700 1 -- 172.21.15.99:0/110233060 wait complete. 2023-10-08T00:11:10.135 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T00:11:10.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.252+0000 7f2de84b7700 1 Processor -- start 2023-10-08T00:11:10.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.252+0000 7f2de84b7700 1 -- start start 2023-10-08T00:11:10.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.253+0000 7f2de84b7700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2de015dcc0 0x7f2de015e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:10.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.253+0000 7f2de84b7700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2de015b260 con 0x7f2de015e5d0 2023-10-08T00:11:10.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.253+0000 7f2de84b7700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2de00c7830 con 0x7f2de00cd900 2023-10-08T00:11:10.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.253+0000 7f2de84b7700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2de009af70 con 0x7f2de015dcc0 2023-10-08T00:11:10.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.253+0000 7f2de5a52700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f2de015e5d0 0x7f2de015b4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45178/0 (socket says 172.21.15.99:45178) 2023-10-08T00:11:10.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.253+0000 7f2de5a52700 1 -- 172.21.15.99:0/494577717 learned_addr learned my addr 172.21.15.99:0/494577717 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:10.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.254+0000 7f2de6253700 1 --2- 172.21.15.99:0/494577717 >> v2:172.21.15.191:3300/0 conn(0x7f2de015dcc0 0x7f2de015e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:10.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.254+0000 7f2de5251700 1 -- 172.21.15.99:0/494577717 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3056907723 0 0) 0x7f2de015b260 con 0x7f2de015e5d0 2023-10-08T00:11:10.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.254+0000 7f2de5251700 1 -- 172.21.15.99:0/494577717 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2dcc003570 con 0x7f2de015e5d0 2023-10-08T00:11:10.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.254+0000 7f2de5251700 1 -- 172.21.15.99:0/494577717 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1925056877 0 0) 0x7f2dcc003570 con 0x7f2de015e5d0 2023-10-08T00:11:10.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.254+0000 7f2de5251700 1 -- 172.21.15.99:0/494577717 >> v2:172.21.15.191:3300/0 conn(0x7f2de015dcc0 msgr2=0x7f2de015e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:10.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.254+0000 7f2de5251700 1 --2- 172.21.15.99:0/494577717 >> v2:172.21.15.191:3300/0 conn(0x7f2de015dcc0 0x7f2de015e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:10.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.254+0000 7f2de5251700 1 -- 172.21.15.99:0/494577717 >> v1:172.21.15.191:6789/0 conn(0x7f2de00cd900 legacy=0x7f2de00c88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:10.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.254+0000 7f2de5251700 1 -- 172.21.15.99:0/494577717 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de00c7b10 con 0x7f2de015e5d0 2023-10-08T00:11:10.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.254+0000 7f2de5251700 1 -- 172.21.15.99:0/494577717 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f2dd0001470 con 0x7f2de015e5d0 2023-10-08T00:11:10.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.254+0000 7f2de5251700 1 -- 172.21.15.99:0/494577717 >> v1:172.21.15.99:6789/0 conn(0x7f2de015e5d0 legacy=0x7f2de015b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:10.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.254+0000 7f2de5251700 1 --2- 172.21.15.99:0/494577717 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2dcc003a60 0x7f2dcc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:10.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.254+0000 7f2de5251700 1 --2- 172.21.15.99:0/494577717 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2dcc004810 0x7f2dcc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:10.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.254+0000 7f2de5251700 1 --2- 172.21.15.99:0/494577717 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2dcc0051c0 0x7f2dcc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:10.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.254+0000 7f2de5251700 1 -- 172.21.15.99:0/494577717 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de009af70 con 0x7f2dcc004810 2023-10-08T00:11:10.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.254+0000 7f2de5251700 1 -- 172.21.15.99:0/494577717 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc00c0d0 con 0x7f2dcc003a60 2023-10-08T00:11:10.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.254+0000 7f2de5251700 1 -- 172.21.15.99:0/494577717 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc00c290 con 0x7f2dcc0051c0 2023-10-08T00:11:10.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.254+0000 7f2de5a52700 1 --2- 172.21.15.99:0/494577717 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2dcc004810 0x7f2dcc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:10.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.254+0000 7f2de6253700 1 --2- 172.21.15.99:0/494577717 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2dcc003a60 0x7f2dcc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:10.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.254+0000 7f2de6a54700 1 --2- 172.21.15.99:0/494577717 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2dcc0051c0 0x7f2dcc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:10.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.254+0000 7f2de5a52700 1 -- 172.21.15.99:0/494577717 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2dcc0051c0 msgr2=0x7f2dcc00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:10.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.254+0000 7f2de5a52700 1 --2- 172.21.15.99:0/494577717 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2dcc0051c0 0x7f2dcc00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:10.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.254+0000 7f2de5a52700 1 -- 172.21.15.99:0/494577717 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2dcc003a60 msgr2=0x7f2dcc003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:10.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.254+0000 7f2de5a52700 1 --2- 172.21.15.99:0/494577717 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2dcc003a60 0x7f2dcc003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:10.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.254+0000 7f2de6a54700 1 -- 172.21.15.99:0/494577717 reap_dead start 2023-10-08T00:11:10.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.254+0000 7f2de5a52700 1 -- 172.21.15.99:0/494577717 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dcc003570 con 0x7f2dcc004810 2023-10-08T00:11:10.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.254+0000 7f2de6253700 1 --2- 172.21.15.99:0/494577717 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2dcc003a60 0x7f2dcc003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:11:10.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.254+0000 7f2de5a52700 1 --2- 172.21.15.99:0/494577717 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2dcc004810 0x7f2dcc004c80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2de015ea70 tx=0x7f2de006e880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f06bbb81a01b40ae server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:10.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.255+0000 7f2de5251700 1 -- 172.21.15.99:0/494577717 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd000cbf0 con 0x7f2dcc004810 2023-10-08T00:11:10.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.255+0000 7f2de5251700 1 -- 172.21.15.99:0/494577717 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dd000cd50 con 0x7f2dcc004810 2023-10-08T00:11:10.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.255+0000 7f2de5251700 1 -- 172.21.15.99:0/494577717 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd0005b70 con 0x7f2dcc004810 2023-10-08T00:11:10.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.255+0000 7f2de84b7700 1 -- 172.21.15.99:0/494577717 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2dcc004810 msgr2=0x7f2dcc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:10.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.255+0000 7f2de84b7700 1 --2- 172.21.15.99:0/494577717 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2dcc004810 0x7f2dcc004c80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2de015ea70 tx=0x7f2de006e880 comp rx=0 tx=0).stop 2023-10-08T00:11:10.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.255+0000 7f2de84b7700 1 -- 172.21.15.99:0/494577717 shutdown_connections 2023-10-08T00:11:10.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.255+0000 7f2de84b7700 1 --2- 172.21.15.99:0/494577717 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2dcc004810 0x7f2dcc004c80 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:10.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.255+0000 7f2de84b7700 1 -- 172.21.15.99:0/494577717 >> 172.21.15.99:0/494577717 conn(0x7f2de0068170 msgr2=0x7f2de00cd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:10.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.255+0000 7f2de84b7700 1 -- 172.21.15.99:0/494577717 shutdown_connections 2023-10-08T00:11:10.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.255+0000 7f2de84b7700 1 -- 172.21.15.99:0/494577717 wait complete. 2023-10-08T00:11:10.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.256+0000 7f2de84b7700 1 Processor -- start 2023-10-08T00:11:10.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.256+0000 7f2de84b7700 1 -- start start 2023-10-08T00:11:10.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.257+0000 7f2de84b7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2dcc004810 0x7f2de0159760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:10.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.257+0000 7f2de84b7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2de00b0250 0x7f2de00b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:10.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.257+0000 7f2de84b7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2de0063230 0x7f2de00636a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:10.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.257+0000 7f2de84b7700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de015a1d0 con 0x7f2dcc004810 2023-10-08T00:11:10.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.257+0000 7f2de84b7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de00b0c00 con 0x7f2de00b0250 2023-10-08T00:11:10.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.257+0000 7f2de84b7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2de0063be0 con 0x7f2de0063230 2023-10-08T00:11:10.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.257+0000 7f2de6253700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2dcc004810 0x7f2de0159760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:10.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.257+0000 7f2de6253700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2dcc004810 0x7f2de0159760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39048/0 (socket says 172.21.15.99:39048) 2023-10-08T00:11:10.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.257+0000 7f2de6253700 1 -- 172.21.15.99:0/520414713 learned_addr learned my addr 172.21.15.99:0/520414713 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:10.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.257+0000 7f2de5a52700 1 --2- 172.21.15.99:0/520414713 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2de00b0250 0x7f2de00b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:10.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.257+0000 7f2de6a54700 1 --2- 172.21.15.99:0/520414713 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2de0063230 0x7f2de00636a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:10.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.257+0000 7f2de6253700 1 -- 172.21.15.99:0/520414713 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2de0063230 msgr2=0x7f2de00636a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:10.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.257+0000 7f2de6253700 1 --2- 172.21.15.99:0/520414713 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2de0063230 0x7f2de00636a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:10.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.257+0000 7f2de6253700 1 -- 172.21.15.99:0/520414713 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2de00b0250 msgr2=0x7f2de00b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:10.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.257+0000 7f2de6253700 1 --2- 172.21.15.99:0/520414713 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2de00b0250 0x7f2de00b06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:10.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.258+0000 7f2de6253700 1 -- 172.21.15.99:0/520414713 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de0159ca0 con 0x7f2dcc004810 2023-10-08T00:11:10.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.258+0000 7f2de6253700 1 --2- 172.21.15.99:0/520414713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2dcc004810 0x7f2de0159760 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc011d50 tx=0x7f2ddc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:10.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.258+0000 7f2dd67fc700 1 -- 172.21.15.99:0/520414713 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc011e50 con 0x7f2dcc004810 2023-10-08T00:11:10.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.258+0000 7f2dd67fc700 1 -- 172.21.15.99:0/520414713 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ddc00fc20 con 0x7f2dcc004810 2023-10-08T00:11:10.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.258+0000 7f2de84b7700 1 -- 172.21.15.99:0/520414713 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2de00b0e30 con 0x7f2dcc004810 2023-10-08T00:11:10.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.258+0000 7f2dd67fc700 1 -- 172.21.15.99:0/520414713 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc01d480 con 0x7f2dcc004810 2023-10-08T00:11:10.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.258+0000 7f2de84b7700 1 -- 172.21.15.99:0/520414713 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2de00b13f0 con 0x7f2dcc004810 2023-10-08T00:11:10.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.259+0000 7f2de84b7700 1 -- 172.21.15.99:0/520414713 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2de004c3c0 con 0x7f2dcc004810 2023-10-08T00:11:10.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.259+0000 7f2dd67fc700 1 -- 172.21.15.99:0/520414713 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2ddc01d830 con 0x7f2dcc004810 2023-10-08T00:11:10.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.259+0000 7f2dd67fc700 1 --2- 172.21.15.99:0/520414713 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2dc4066f30 0x7f2dc40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:10.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.260+0000 7f2dd67fc700 1 -- 172.21.15.99:0/520414713 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2ddc018070 con 0x7f2dcc004810 2023-10-08T00:11:10.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.260+0000 7f2de5a52700 1 --2- 172.21.15.99:0/520414713 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2dc4066f30 0x7f2dc40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:10.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.260+0000 7f2de5a52700 1 --2- 172.21.15.99:0/520414713 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2dc4066f30 0x7f2dc40693e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0007cd0 tx=0x7f2dd0015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:10.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.264+0000 7f2dd67fc700 1 -- 172.21.15.99:0/520414713 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2ddc052500 con 0x7f2dcc004810 2023-10-08T00:11:10.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.447+0000 7f2de84b7700 1 -- 172.21.15.99:0/520414713 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2de00c7df0 con 0x7f2dcc004810 2023-10-08T00:11:10.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.447+0000 7f2dd67fc700 1 -- 172.21.15.99:0/520414713 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2ddc055b20 con 0x7f2dcc004810 2023-10-08T00:11:10.448 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2023-10-08T00:11:10.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.449+0000 7f2de84b7700 1 -- 172.21.15.99:0/520414713 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2dc4066f30 msgr2=0x7f2dc40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:10.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.449+0000 7f2de84b7700 1 --2- 172.21.15.99:0/520414713 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2dc4066f30 0x7f2dc40693e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0007cd0 tx=0x7f2dd0015000 comp rx=0 tx=0).stop 2023-10-08T00:11:10.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.449+0000 7f2de84b7700 1 -- 172.21.15.99:0/520414713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2dcc004810 msgr2=0x7f2de0159760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:10.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.449+0000 7f2de84b7700 1 --2- 172.21.15.99:0/520414713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2dcc004810 0x7f2de0159760 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc011d50 tx=0x7f2ddc0091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:10.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.450+0000 7f2de84b7700 1 -- 172.21.15.99:0/520414713 shutdown_connections 2023-10-08T00:11:10.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.450+0000 7f2de84b7700 1 --2- 172.21.15.99:0/520414713 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2dc4066f30 0x7f2dc40693e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:10.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.450+0000 7f2de84b7700 1 --2- 172.21.15.99:0/520414713 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2de0063230 0x7f2de00636a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:10.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.450+0000 7f2de84b7700 1 --2- 172.21.15.99:0/520414713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2dcc004810 0x7f2de0159760 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:10.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.450+0000 7f2de84b7700 1 --2- 172.21.15.99:0/520414713 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2de00b0250 0x7f2de00b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:10.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.450+0000 7f2de84b7700 1 -- 172.21.15.99:0/520414713 >> 172.21.15.99:0/520414713 conn(0x7f2de0068170 msgr2=0x7f2de00c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:10.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.450+0000 7f2de84b7700 1 -- 172.21.15.99:0/520414713 shutdown_connections 2023-10-08T00:11:10.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.450+0000 7f2de84b7700 1 -- 172.21.15.99:0/520414713 wait complete. 2023-10-08T00:11:10.462 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T00:11:10.580 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.579+0000 7f7605bcd700 1 Processor -- start 2023-10-08T00:11:10.580 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.579+0000 7f7605bcd700 1 -- start start 2023-10-08T00:11:10.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.580+0000 7f7605bcd700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7600159ec0 0x7f76000af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:10.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.580+0000 7f7605bcd700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76000b0790 con 0x7f760006acb0 2023-10-08T00:11:10.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.580+0000 7f7605bcd700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76000ce1f0 con 0x7f760015a290 2023-10-08T00:11:10.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.580+0000 7f7605bcd700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76000b0b20 con 0x7f7600159ec0 2023-10-08T00:11:10.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.580+0000 7f75fffff700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f760006acb0 0x7f76000cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45192/0 (socket says 172.21.15.99:45192) 2023-10-08T00:11:10.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.580+0000 7f75fffff700 1 -- 172.21.15.99:0/3006055339 learned_addr learned my addr 172.21.15.99:0/3006055339 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:10.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.580+0000 7f75ff7fe700 1 --2- 172.21.15.99:0/3006055339 >> v2:172.21.15.191:3300/0 conn(0x7f7600159ec0 0x7f76000af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:10.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.580+0000 7f75fe7fc700 1 -- 172.21.15.99:0/3006055339 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 936615463 0 0) 0x7f76000b0790 con 0x7f760006acb0 2023-10-08T00:11:10.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.580+0000 7f75fe7fc700 1 -- 172.21.15.99:0/3006055339 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75e8003570 con 0x7f760006acb0 2023-10-08T00:11:10.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.581+0000 7f75ff7fe700 1 -- 172.21.15.99:0/3006055339 >> v1:172.21.15.191:6789/0 conn(0x7f760015a290 legacy=0x7f760006a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:10.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.581+0000 7f75ff7fe700 1 -- 172.21.15.99:0/3006055339 >> v1:172.21.15.99:6789/0 conn(0x7f760006acb0 legacy=0x7f76000cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:10.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.581+0000 7f75ff7fe700 1 -- 172.21.15.99:0/3006055339 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76000b04b0 con 0x7f7600159ec0 2023-10-08T00:11:10.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.581+0000 7f75ff7fe700 1 --2- 172.21.15.99:0/3006055339 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7600159ec0 0x7f76000af940 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f75f0009e60 tx=0x7f75f0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52ca90b9969e1da4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:10.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.581+0000 7f75fe7fc700 1 -- 172.21.15.99:0/3006055339 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f000e070 con 0x7f7600159ec0 2023-10-08T00:11:10.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.581+0000 7f75fe7fc700 1 -- 172.21.15.99:0/3006055339 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75f000bb60 con 0x7f7600159ec0 2023-10-08T00:11:10.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.581+0000 7f75fe7fc700 1 -- 172.21.15.99:0/3006055339 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f0012430 con 0x7f7600159ec0 2023-10-08T00:11:10.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.581+0000 7f7605bcd700 1 -- 172.21.15.99:0/3006055339 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7600159ec0 msgr2=0x7f76000af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:10.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.581+0000 7f7605bcd700 1 --2- 172.21.15.99:0/3006055339 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7600159ec0 0x7f76000af940 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f75f0009e60 tx=0x7f75f0009780 comp rx=0 tx=0).stop 2023-10-08T00:11:10.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.581+0000 7f7605bcd700 1 -- 172.21.15.99:0/3006055339 shutdown_connections 2023-10-08T00:11:10.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.581+0000 7f7605bcd700 1 --2- 172.21.15.99:0/3006055339 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7600159ec0 0x7f76000af940 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:10.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.581+0000 7f7605bcd700 1 -- 172.21.15.99:0/3006055339 >> 172.21.15.99:0/3006055339 conn(0x7f76000d1880 msgr2=0x7f76000cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:10.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.581+0000 7f7605bcd700 1 -- 172.21.15.99:0/3006055339 shutdown_connections 2023-10-08T00:11:10.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.581+0000 7f7605bcd700 1 -- 172.21.15.99:0/3006055339 wait complete. 2023-10-08T00:11:10.587 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.583+0000 7f7605bcd700 1 Processor -- start 2023-10-08T00:11:10.587 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.583+0000 7f7605bcd700 1 -- start start 2023-10-08T00:11:10.587 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.583+0000 7f7605bcd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f760006acb0 0x7f760016b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:10.587 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.583+0000 7f7605bcd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7600159ec0 0x7f760015d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:10.587 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.583+0000 7f7605bcd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f760015a290 0x7f7600155560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:10.587 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.583+0000 7f7605bcd700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f760016b690 con 0x7f7600159ec0 2023-10-08T00:11:10.587 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.583+0000 7f7605bcd700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f760015d980 con 0x7f760006acb0 2023-10-08T00:11:10.587 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.583+0000 7f7605bcd700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76000b5a70 con 0x7f760015a290 2023-10-08T00:11:10.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.584+0000 7f75feffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7600159ec0 0x7f760015d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:10.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.584+0000 7f75feffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7600159ec0 0x7f760015d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39050/0 (socket says 172.21.15.99:39050) 2023-10-08T00:11:10.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.584+0000 7f75feffd700 1 -- 172.21.15.99:0/918184338 learned_addr learned my addr 172.21.15.99:0/918184338 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:10.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.584+0000 7f75ff7fe700 1 --2- 172.21.15.99:0/918184338 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f760006acb0 0x7f760016b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:10.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.584+0000 7f75fffff700 1 --2- 172.21.15.99:0/918184338 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f760015a290 0x7f7600155560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:10.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.584+0000 7f75feffd700 1 -- 172.21.15.99:0/918184338 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f760015a290 msgr2=0x7f7600155560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:10.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.584+0000 7f75feffd700 1 --2- 172.21.15.99:0/918184338 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f760015a290 0x7f7600155560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:10.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.584+0000 7f75feffd700 1 -- 172.21.15.99:0/918184338 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f760006acb0 msgr2=0x7f760016b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:10.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.584+0000 7f75feffd700 1 --2- 172.21.15.99:0/918184338 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f760006acb0 0x7f760016b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:10.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.584+0000 7f75feffd700 1 -- 172.21.15.99:0/918184338 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75f0009a20 con 0x7f7600159ec0 2023-10-08T00:11:10.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.584+0000 7f75feffd700 1 --2- 172.21.15.99:0/918184338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7600159ec0 0x7f760015d1a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f75f400c4e0 tx=0x7f75f400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:10.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.584+0000 7f75e77fe700 1 -- 172.21.15.99:0/918184338 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f4011590 con 0x7f7600159ec0 2023-10-08T00:11:10.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.584+0000 7f75e77fe700 1 -- 172.21.15.99:0/918184338 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75f4011710 con 0x7f7600159ec0 2023-10-08T00:11:10.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.584+0000 7f7605bcd700 1 -- 172.21.15.99:0/918184338 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f760015d6e0 con 0x7f7600159ec0 2023-10-08T00:11:10.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.584+0000 7f75e77fe700 1 -- 172.21.15.99:0/918184338 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f4011ad0 con 0x7f7600159ec0 2023-10-08T00:11:10.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.584+0000 7f7605bcd700 1 -- 172.21.15.99:0/918184338 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76000b51c0 con 0x7f7600159ec0 2023-10-08T00:11:10.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.585+0000 7f7605bcd700 1 -- 172.21.15.99:0/918184338 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f760004c3c0 con 0x7f7600159ec0 2023-10-08T00:11:10.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.585+0000 7f75e77fe700 1 -- 172.21.15.99:0/918184338 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f75f401b3b0 con 0x7f7600159ec0 2023-10-08T00:11:10.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.586+0000 7f75e77fe700 1 --2- 172.21.15.99:0/918184338 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f75ec066fb0 0x7f75ec069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:10.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.586+0000 7f75e77fe700 1 -- 172.21.15.99:0/918184338 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f75f4083b10 con 0x7f7600159ec0 2023-10-08T00:11:10.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.586+0000 7f75ff7fe700 1 --2- 172.21.15.99:0/918184338 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f75ec066fb0 0x7f75ec069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:10.590 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.587+0000 7f75ff7fe700 1 --2- 172.21.15.99:0/918184338 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f75ec066fb0 0x7f75ec069460 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f75f00097c0 tx=0x7f75f0019000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:10.590 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.590+0000 7f75e77fe700 1 -- 172.21.15.99:0/918184338 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f75f4050ce0 con 0x7f7600159ec0 2023-10-08T00:11:10.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.771+0000 7f7605bcd700 1 -- 172.21.15.99:0/918184338 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f76000ceab0 con 0x7f7600159ec0 2023-10-08T00:11:10.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.772+0000 7f75e77fe700 1 -- 172.21.15.99:0/918184338 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f75f4017070 con 0x7f7600159ec0 2023-10-08T00:11:10.773 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2023-10-08T00:11:10.774 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.773+0000 7f7605bcd700 1 -- 172.21.15.99:0/918184338 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f75ec066fb0 msgr2=0x7f75ec069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:10.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.773+0000 7f7605bcd700 1 --2- 172.21.15.99:0/918184338 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f75ec066fb0 0x7f75ec069460 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f75f00097c0 tx=0x7f75f0019000 comp rx=0 tx=0).stop 2023-10-08T00:11:10.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.773+0000 7f7605bcd700 1 -- 172.21.15.99:0/918184338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7600159ec0 msgr2=0x7f760015d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:10.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.773+0000 7f7605bcd700 1 --2- 172.21.15.99:0/918184338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7600159ec0 0x7f760015d1a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f75f400c4e0 tx=0x7f75f400c8a0 comp rx=0 tx=0).stop 2023-10-08T00:11:10.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.773+0000 7f7605bcd700 1 -- 172.21.15.99:0/918184338 shutdown_connections 2023-10-08T00:11:10.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.773+0000 7f7605bcd700 1 --2- 172.21.15.99:0/918184338 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f75ec066fb0 0x7f75ec069460 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:10.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.773+0000 7f7605bcd700 1 --2- 172.21.15.99:0/918184338 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f760006acb0 0x7f760016b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:10.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.773+0000 7f7605bcd700 1 --2- 172.21.15.99:0/918184338 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f760015a290 0x7f7600155560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:10.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.773+0000 7f7605bcd700 1 --2- 172.21.15.99:0/918184338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7600159ec0 0x7f760015d1a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:10.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.774+0000 7f7605bcd700 1 -- 172.21.15.99:0/918184338 >> 172.21.15.99:0/918184338 conn(0x7f76000d1880 msgr2=0x7f76000b0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:10.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.774+0000 7f7605bcd700 1 -- 172.21.15.99:0/918184338 shutdown_connections 2023-10-08T00:11:10.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.774+0000 7f7605bcd700 1 -- 172.21.15.99:0/918184338 wait complete. 2023-10-08T00:11:10.786 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:11:10.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.899+0000 7f9c8a1c2700 1 Processor -- start 2023-10-08T00:11:10.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.899+0000 7f9c8a1c2700 1 -- start start 2023-10-08T00:11:10.903 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.900+0000 7f9c8a1c2700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9c84159be0 0x7f9c84068db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:10.903 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.900+0000 7f9c8a1c2700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f9c84159fb0 0x7f9c840ce950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:10.903 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.900+0000 7f9c8a1c2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c840692f0 0x7f9c840cee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:10.903 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.900+0000 7f9c8a1c2700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c840701f0 con 0x7f9c84159be0 2023-10-08T00:11:10.903 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.900+0000 7f9c8a1c2700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c84067800 con 0x7f9c84159fb0 2023-10-08T00:11:10.903 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.900+0000 7f9c8a1c2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c84067ae0 con 0x7f9c840692f0 2023-10-08T00:11:10.904 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.900+0000 7f9c891c0700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9c84159be0 0x7f9c84068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:10.904 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.900+0000 7f9c891c0700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9c84159be0 0x7f9c84068db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39062/0 (socket says 172.21.15.99:39062) 2023-10-08T00:11:10.904 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.900+0000 7f9c891c0700 1 -- 172.21.15.99:0/1162278765 learned_addr learned my addr 172.21.15.99:0/1162278765 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:10.904 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.900+0000 7f9c889bf700 1 --2- 172.21.15.99:0/1162278765 >> v2:172.21.15.191:3300/0 conn(0x7f9c84159fb0 0x7f9c840ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:10.904 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.900+0000 7f9c899c1700 1 --2- 172.21.15.99:0/1162278765 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c840692f0 0x7f9c840cee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:10.904 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.900+0000 7f9c891c0700 1 -- 172.21.15.99:0/1162278765 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c840692f0 msgr2=0x7f9c840cee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:10.904 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.900+0000 7f9c891c0700 1 --2- 172.21.15.99:0/1162278765 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c840692f0 0x7f9c840cee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:10.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.900+0000 7f9c891c0700 1 -- 172.21.15.99:0/1162278765 >> v2:172.21.15.191:3300/0 conn(0x7f9c84159fb0 msgr2=0x7f9c840ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:10.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.900+0000 7f9c891c0700 1 --2- 172.21.15.99:0/1162278765 >> v2:172.21.15.191:3300/0 conn(0x7f9c84159fb0 0x7f9c840ce950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:10.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.900+0000 7f9c891c0700 1 -- 172.21.15.99:0/1162278765 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c84067dc0 con 0x7f9c84159be0 2023-10-08T00:11:10.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.900+0000 7f9c891c0700 1 --2- 172.21.15.99:0/1162278765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c84159be0 0x7f9c84068db0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9c8406ec40 tx=0x7f9c80009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd3f32c38b42ce90 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:10.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.901+0000 7f9c7bfff700 1 -- 172.21.15.99:0/1162278765 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8000b860 con 0x7f9c84159be0 2023-10-08T00:11:10.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.901+0000 7f9c7bfff700 1 -- 172.21.15.99:0/1162278765 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c8000b9c0 con 0x7f9c84159be0 2023-10-08T00:11:10.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.901+0000 7f9c7bfff700 1 -- 172.21.15.99:0/1162278765 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8000bd60 con 0x7f9c84159be0 2023-10-08T00:11:10.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.901+0000 7f9c8a1c2700 1 -- 172.21.15.99:0/1162278765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c84159be0 msgr2=0x7f9c84068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:10.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.901+0000 7f9c8a1c2700 1 --2- 172.21.15.99:0/1162278765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c84159be0 0x7f9c84068db0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9c8406ec40 tx=0x7f9c80009250 comp rx=0 tx=0).stop 2023-10-08T00:11:10.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.901+0000 7f9c8a1c2700 1 -- 172.21.15.99:0/1162278765 shutdown_connections 2023-10-08T00:11:10.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.901+0000 7f9c8a1c2700 1 --2- 172.21.15.99:0/1162278765 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c840692f0 0x7f9c840cee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:10.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.901+0000 7f9c8a1c2700 1 --2- 172.21.15.99:0/1162278765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c84159be0 0x7f9c84068db0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:10.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.901+0000 7f9c8a1c2700 1 --2- 172.21.15.99:0/1162278765 >> v2:172.21.15.191:3300/0 conn(0x7f9c84159fb0 0x7f9c840ce950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:10.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.901+0000 7f9c8a1c2700 1 -- 172.21.15.99:0/1162278765 >> 172.21.15.99:0/1162278765 conn(0x7f9c8406bc70 msgr2=0x7f9c8406c070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:10.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.901+0000 7f9c8a1c2700 1 -- 172.21.15.99:0/1162278765 shutdown_connections 2023-10-08T00:11:10.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.901+0000 7f9c8a1c2700 1 -- 172.21.15.99:0/1162278765 wait complete. 2023-10-08T00:11:10.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.902+0000 7f9c8a1c2700 1 Processor -- start 2023-10-08T00:11:10.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.902+0000 7f9c8a1c2700 1 -- start start 2023-10-08T00:11:10.909 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.903+0000 7f9c8a1c2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c840692f0 0x7f9c84158090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:10.910 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.903+0000 7f9c8a1c2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c84159be0 0x7f9c840b0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:10.910 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.903+0000 7f9c8a1c2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c84159fb0 0x7f9c8415c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:10.910 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.903+0000 7f9c8a1c2700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c840c7040 con 0x7f9c84159fb0 2023-10-08T00:11:10.910 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.903+0000 7f9c8a1c2700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c840c5ed0 con 0x7f9c84159be0 2023-10-08T00:11:10.910 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.903+0000 7f9c8a1c2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c840c61b0 con 0x7f9c840692f0 2023-10-08T00:11:10.910 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.903+0000 7f9c899c1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c84159fb0 0x7f9c8415c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:10.911 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.903+0000 7f9c899c1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c84159fb0 0x7f9c8415c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39072/0 (socket says 172.21.15.99:39072) 2023-10-08T00:11:10.911 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.903+0000 7f9c899c1700 1 -- 172.21.15.99:0/899866643 learned_addr learned my addr 172.21.15.99:0/899866643 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:10.911 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.903+0000 7f9c891c0700 1 --2- 172.21.15.99:0/899866643 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c840692f0 0x7f9c84158090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:10.911 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.904+0000 7f9c889bf700 1 --2- 172.21.15.99:0/899866643 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c84159be0 0x7f9c840b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:10.911 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.904+0000 7f9c899c1700 1 -- 172.21.15.99:0/899866643 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c840692f0 msgr2=0x7f9c84158090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:10.911 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.904+0000 7f9c899c1700 1 --2- 172.21.15.99:0/899866643 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c840692f0 0x7f9c84158090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:10.912 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.904+0000 7f9c899c1700 1 -- 172.21.15.99:0/899866643 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c84159be0 msgr2=0x7f9c840b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:10.912 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.904+0000 7f9c899c1700 1 --2- 172.21.15.99:0/899866643 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c84159be0 0x7f9c840b0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:10.912 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.904+0000 7f9c899c1700 1 -- 172.21.15.99:0/899866643 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c800094f0 con 0x7f9c84159fb0 2023-10-08T00:11:10.912 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.904+0000 7f9c899c1700 1 --2- 172.21.15.99:0/899866643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c84159fb0 0x7f9c8415c6f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c009fa0 tx=0x7f9c7c009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:10.912 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.904+0000 7f9c797fa700 1 -- 172.21.15.99:0/899866643 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c00ebc0 con 0x7f9c84159fb0 2023-10-08T00:11:10.913 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.904+0000 7f9c797fa700 1 -- 172.21.15.99:0/899866643 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c7c00ed20 con 0x7f9c84159fb0 2023-10-08T00:11:10.913 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.904+0000 7f9c8a1c2700 1 -- 172.21.15.99:0/899866643 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c840c6490 con 0x7f9c84159fb0 2023-10-08T00:11:10.913 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.904+0000 7f9c797fa700 1 -- 172.21.15.99:0/899866643 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c014670 con 0x7f9c84159fb0 2023-10-08T00:11:10.913 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.904+0000 7f9c8a1c2700 1 -- 172.21.15.99:0/899866643 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c841585d0 con 0x7f9c84159fb0 2023-10-08T00:11:10.913 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.905+0000 7f9c8a1c2700 1 -- 172.21.15.99:0/899866643 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c8406e3e0 con 0x7f9c84159fb0 2023-10-08T00:11:10.913 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.905+0000 7f9c797fa700 1 -- 172.21.15.99:0/899866643 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9c7c0149e0 con 0x7f9c84159fb0 2023-10-08T00:11:10.913 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.906+0000 7f9c797fa700 1 --2- 172.21.15.99:0/899866643 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9c68066f30 0x7f9c680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:10.914 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.906+0000 7f9c797fa700 1 -- 172.21.15.99:0/899866643 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9c7c084b90 con 0x7f9c84159fb0 2023-10-08T00:11:10.914 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.906+0000 7f9c891c0700 1 --2- 172.21.15.99:0/899866643 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9c68066f30 0x7f9c680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:10.914 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.906+0000 7f9c891c0700 1 --2- 172.21.15.99:0/899866643 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9c68066f30 0x7f9c680693e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9c8406ec40 tx=0x7f9c80017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:10.914 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:10.910+0000 7f9c797fa700 1 -- 172.21.15.99:0/899866643 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9c7c051e00 con 0x7f9c84159fb0 2023-10-08T00:11:11.133 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.132+0000 7f9c8a1c2700 1 -- 172.21.15.99:0/899866643 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9c840c7180 con 0x7f9c84159fb0 2023-10-08T00:11:11.133 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.133+0000 7f9c797fa700 1 -- 172.21.15.99:0/899866643 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4594 (secure 0 0 0) 0x7f9c7c055420 con 0x7f9c84159fb0 2023-10-08T00:11:11.133 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:11:11.134 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/649537180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":649537180},{"type":"v1","addr":"172.21.15.99:6833","nonce":649537180}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4239,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2194110899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2194110899},{"type":"v1","addr":"172.21.15.99:6835","nonce":2194110899}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4251,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/1732443753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":1732443753},{"type":"v1","addr":"172.21.15.191:6837","nonce":1732443753}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:10:53.454321+0000","modified":"2023-10-08T00:10:57.098731+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6836/1988845948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1988845948},{"type":"v1","addr":"172.21.15.191:6836","nonce":1988845948}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-08T00:11:11.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.135+0000 7f9c8a1c2700 1 -- 172.21.15.99:0/899866643 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9c68066f30 msgr2=0x7f9c680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:11.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.135+0000 7f9c8a1c2700 1 --2- 172.21.15.99:0/899866643 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9c68066f30 0x7f9c680693e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9c8406ec40 tx=0x7f9c80017000 comp rx=0 tx=0).stop 2023-10-08T00:11:11.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.135+0000 7f9c8a1c2700 1 -- 172.21.15.99:0/899866643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c84159fb0 msgr2=0x7f9c8415c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:11.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.135+0000 7f9c8a1c2700 1 --2- 172.21.15.99:0/899866643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c84159fb0 0x7f9c8415c6f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c009fa0 tx=0x7f9c7c009900 comp rx=0 tx=0).stop 2023-10-08T00:11:11.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.135+0000 7f9c8a1c2700 1 -- 172.21.15.99:0/899866643 shutdown_connections 2023-10-08T00:11:11.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.135+0000 7f9c8a1c2700 1 --2- 172.21.15.99:0/899866643 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9c68066f30 0x7f9c680693e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:11.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.135+0000 7f9c8a1c2700 1 --2- 172.21.15.99:0/899866643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c84159fb0 0x7f9c8415c6f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:11.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.135+0000 7f9c8a1c2700 1 --2- 172.21.15.99:0/899866643 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c840692f0 0x7f9c84158090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:11.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.135+0000 7f9c8a1c2700 1 --2- 172.21.15.99:0/899866643 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c84159be0 0x7f9c840b0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:11.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.135+0000 7f9c8a1c2700 1 -- 172.21.15.99:0/899866643 >> 172.21.15.99:0/899866643 conn(0x7f9c8406bc70 msgr2=0x7f9c84158f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:11.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.135+0000 7f9c8a1c2700 1 -- 172.21.15.99:0/899866643 shutdown_connections 2023-10-08T00:11:11.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.135+0000 7f9c8a1c2700 1 -- 172.21.15.99:0/899866643 wait complete. 2023-10-08T00:11:11.137 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 8 2023-10-08T00:11:11.148 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:11:11.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.262+0000 7f6436668700 1 Processor -- start 2023-10-08T00:11:11.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.262+0000 7f6436668700 1 -- start start 2023-10-08T00:11:11.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.263+0000 7f6436668700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6430157870 0x7f643015b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:11.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.263+0000 7f6436668700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f643015a620 0x7f643015d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:11.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.263+0000 7f6436668700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f643015d580 0x7f6430158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:11.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.263+0000 7f6436668700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f643015cbb0 con 0x7f643015a620 2023-10-08T00:11:11.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.263+0000 7f6436668700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f643015bfa0 con 0x7f643015d580 2023-10-08T00:11:11.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.263+0000 7f6436668700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f643015c280 con 0x7f6430157870 2023-10-08T00:11:11.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.263+0000 7f642f7fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f643015a620 0x7f643015d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:11.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.263+0000 7f642f7fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f643015a620 0x7f643015d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39084/0 (socket says 172.21.15.99:39084) 2023-10-08T00:11:11.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.263+0000 7f642f7fe700 1 -- 172.21.15.99:0/3360595167 learned_addr learned my addr 172.21.15.99:0/3360595167 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:11.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.263+0000 7f642ffff700 1 --2- 172.21.15.99:0/3360595167 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6430157870 0x7f643015b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:11.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.263+0000 7f642f7fe700 1 -- 172.21.15.99:0/3360595167 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6430157870 msgr2=0x7f643015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:11.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.263+0000 7f642f7fe700 1 --2- 172.21.15.99:0/3360595167 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6430157870 0x7f643015b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:11.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.263+0000 7f642f7fe700 1 -- 172.21.15.99:0/3360595167 >> v2:172.21.15.191:3300/0 conn(0x7f643015d580 msgr2=0x7f6430158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:11.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.263+0000 7f642f7fe700 1 --2- 172.21.15.99:0/3360595167 >> v2:172.21.15.191:3300/0 conn(0x7f643015d580 0x7f6430158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:11.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.263+0000 7f642f7fe700 1 -- 172.21.15.99:0/3360595167 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f643015c560 con 0x7f643015a620 2023-10-08T00:11:11.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.263+0000 7f642f7fe700 1 --2- 172.21.15.99:0/3360595167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f643015a620 0x7f643015d040 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f64300b03b0 tx=0x7f6420009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4786935e0a510323 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:11.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.263+0000 7f642effd700 1 -- 172.21.15.99:0/3360595167 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642000b860 con 0x7f643015a620 2023-10-08T00:11:11.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.263+0000 7f642effd700 1 -- 172.21.15.99:0/3360595167 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f642000b9c0 con 0x7f643015a620 2023-10-08T00:11:11.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.263+0000 7f642effd700 1 -- 172.21.15.99:0/3360595167 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642000bd60 con 0x7f643015a620 2023-10-08T00:11:11.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.264+0000 7f6436668700 1 -- 172.21.15.99:0/3360595167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f643015a620 msgr2=0x7f643015d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:11.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.264+0000 7f6436668700 1 --2- 172.21.15.99:0/3360595167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f643015a620 0x7f643015d040 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f64300b03b0 tx=0x7f6420009250 comp rx=0 tx=0).stop 2023-10-08T00:11:11.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.264+0000 7f6436668700 1 -- 172.21.15.99:0/3360595167 shutdown_connections 2023-10-08T00:11:11.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.264+0000 7f6436668700 1 --2- 172.21.15.99:0/3360595167 >> v2:172.21.15.191:3300/0 conn(0x7f643015d580 0x7f6430158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:11.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.264+0000 7f6436668700 1 --2- 172.21.15.99:0/3360595167 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6430157870 0x7f643015b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:11.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.264+0000 7f6436668700 1 --2- 172.21.15.99:0/3360595167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f643015a620 0x7f643015d040 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:11.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.264+0000 7f6436668700 1 -- 172.21.15.99:0/3360595167 >> 172.21.15.99:0/3360595167 conn(0x7f6430068ce0 msgr2=0x7f64300690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:11.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.264+0000 7f6436668700 1 -- 172.21.15.99:0/3360595167 shutdown_connections 2023-10-08T00:11:11.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.264+0000 7f6436668700 1 -- 172.21.15.99:0/3360595167 wait complete. 2023-10-08T00:11:11.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.265+0000 7f6436668700 1 Processor -- start 2023-10-08T00:11:11.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.265+0000 7f6436668700 1 -- start start 2023-10-08T00:11:11.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.266+0000 7f6436668700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6430157870 0x7f64300b2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:11.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.266+0000 7f6436668700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f643015a620 0x7f643014b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:11.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.266+0000 7f6436668700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f643015d580 0x7f64301550a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:11.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.266+0000 7f6436668700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64300cf990 con 0x7f643015a620 2023-10-08T00:11:11.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.266+0000 7f6436668700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643014bd80 con 0x7f6430157870 2023-10-08T00:11:11.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.266+0000 7f6436668700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64300b2fd0 con 0x7f643015d580 2023-10-08T00:11:11.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.266+0000 7f642f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f643015a620 0x7f643014b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:11.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.266+0000 7f642f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f643015a620 0x7f643014b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39088/0 (socket says 172.21.15.99:39088) 2023-10-08T00:11:11.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.266+0000 7f642f7fe700 1 -- 172.21.15.99:0/3843314291 learned_addr learned my addr 172.21.15.99:0/3843314291 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:11.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.266+0000 7f642ffff700 1 --2- 172.21.15.99:0/3843314291 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6430157870 0x7f64300b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:11.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.266+0000 7f6434c05700 1 --2- 172.21.15.99:0/3843314291 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f643015d580 0x7f64301550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:11.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.266+0000 7f642f7fe700 1 -- 172.21.15.99:0/3843314291 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f643015d580 msgr2=0x7f64301550a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:11.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.266+0000 7f642f7fe700 1 --2- 172.21.15.99:0/3843314291 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f643015d580 0x7f64301550a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:11.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.266+0000 7f642f7fe700 1 -- 172.21.15.99:0/3843314291 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6430157870 msgr2=0x7f64300b2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:11.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.266+0000 7f642f7fe700 1 --2- 172.21.15.99:0/3843314291 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6430157870 0x7f64300b2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:11.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.266+0000 7f642f7fe700 1 -- 172.21.15.99:0/3843314291 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64200094f0 con 0x7f643015a620 2023-10-08T00:11:11.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.267+0000 7f642f7fe700 1 --2- 172.21.15.99:0/3843314291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f643015a620 0x7f643014b840 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6420009290 tx=0x7f6420017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:11.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.267+0000 7f6417fff700 1 -- 172.21.15.99:0/3843314291 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6420026480 con 0x7f643015a620 2023-10-08T00:11:11.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.267+0000 7f6417fff700 1 -- 172.21.15.99:0/3843314291 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f642001f070 con 0x7f643015a620 2023-10-08T00:11:11.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.267+0000 7f6436668700 1 -- 172.21.15.99:0/3843314291 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64300b2cf0 con 0x7f643015a620 2023-10-08T00:11:11.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.267+0000 7f6417fff700 1 -- 172.21.15.99:0/3843314291 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6420026850 con 0x7f643015a620 2023-10-08T00:11:11.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.267+0000 7f6436668700 1 -- 172.21.15.99:0/3843314291 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64300cf620 con 0x7f643015a620 2023-10-08T00:11:11.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.268+0000 7f6436668700 1 -- 172.21.15.99:0/3843314291 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64300cfb00 con 0x7f643015a620 2023-10-08T00:11:11.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.268+0000 7f6417fff700 1 -- 172.21.15.99:0/3843314291 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f642001a430 con 0x7f643015a620 2023-10-08T00:11:11.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.268+0000 7f6417fff700 1 --2- 172.21.15.99:0/3843314291 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f641c066f30 0x7f641c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:11.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.268+0000 7f6417fff700 1 -- 172.21.15.99:0/3843314291 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6420021070 con 0x7f643015a620 2023-10-08T00:11:11.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.269+0000 7f642ffff700 1 --2- 172.21.15.99:0/3843314291 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f641c066f30 0x7f641c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:11.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.269+0000 7f642ffff700 1 --2- 172.21.15.99:0/3843314291 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f641c066f30 0x7f641c0693e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f642400acb0 tx=0x7f6424008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:11.275 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.273+0000 7f6417fff700 1 -- 172.21.15.99:0/3843314291 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f64200536d0 con 0x7f643015a620 2023-10-08T00:11:11.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.456+0000 7f6436668700 1 -- 172.21.15.99:0/3843314291 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f643004c3c0 con 0x7f643015a620 2023-10-08T00:11:11.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.457+0000 7f6417fff700 1 -- 172.21.15.99:0/3843314291 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f6420056cf0 con 0x7f643015a620 2023-10-08T00:11:11.458 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:11:11.458 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":20,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:10:55.080854+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:10:51.822157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T00:10:52.415130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:11:11.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.459+0000 7f6436668700 1 -- 172.21.15.99:0/3843314291 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f641c066f30 msgr2=0x7f641c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:11.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.459+0000 7f6436668700 1 --2- 172.21.15.99:0/3843314291 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f641c066f30 0x7f641c0693e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f642400acb0 tx=0x7f6424008040 comp rx=0 tx=0).stop 2023-10-08T00:11:11.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.459+0000 7f6436668700 1 -- 172.21.15.99:0/3843314291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f643015a620 msgr2=0x7f643014b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:11.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.459+0000 7f6436668700 1 --2- 172.21.15.99:0/3843314291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f643015a620 0x7f643014b840 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6420009290 tx=0x7f6420017000 comp rx=0 tx=0).stop 2023-10-08T00:11:11.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.459+0000 7f6436668700 1 -- 172.21.15.99:0/3843314291 shutdown_connections 2023-10-08T00:11:11.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.459+0000 7f6436668700 1 --2- 172.21.15.99:0/3843314291 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f641c066f30 0x7f641c0693e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:11.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.459+0000 7f6436668700 1 --2- 172.21.15.99:0/3843314291 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6430157870 0x7f64300b2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:11.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.459+0000 7f6436668700 1 --2- 172.21.15.99:0/3843314291 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f643015d580 0x7f64301550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:11.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.459+0000 7f6436668700 1 --2- 172.21.15.99:0/3843314291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f643015a620 0x7f643014b840 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:11.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.459+0000 7f6436668700 1 -- 172.21.15.99:0/3843314291 >> 172.21.15.99:0/3843314291 conn(0x7f6430068ce0 msgr2=0x7f64301597a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:11.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.459+0000 7f6436668700 1 -- 172.21.15.99:0/3843314291 shutdown_connections 2023-10-08T00:11:11.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.459+0000 7f6436668700 1 -- 172.21.15.99:0/3843314291 wait complete. 2023-10-08T00:11:11.473 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:11:11.586 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.585+0000 7fc5c035d700 1 Processor -- start 2023-10-08T00:11:11.586 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.586+0000 7fc5c035d700 1 -- start start 2023-10-08T00:11:11.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.586+0000 7fc5c035d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5b8157870 0x7fc5b815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:11.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.586+0000 7fc5c035d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc5b8157c40 0x7fc5b8158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:11.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.586+0000 7fc5c035d700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5b815cbb0 con 0x7fc5b815a620 2023-10-08T00:11:11.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.586+0000 7fc5c035d700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5b815bfa0 con 0x7fc5b8157c40 2023-10-08T00:11:11.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.586+0000 7fc5c035d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b815c280 con 0x7fc5b8157870 2023-10-08T00:11:11.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.587+0000 7fc5be8fa700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc5b8157c40 0x7fc5b8158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:11.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.587+0000 7fc5be0f9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5b8157870 0x7fc5b815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:11.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.587+0000 7fc5bd8f8700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7fc5b815a620 0x7fc5b815d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:56226/0 (socket says 172.21.15.99:56226) 2023-10-08T00:11:11.590 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.587+0000 7fc5be8fa700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc5b8157c40 0x7fc5b8158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.99:55104/0 (socket says 172.21.15.99:55104) 2023-10-08T00:11:11.590 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.587+0000 7fc5be0f9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5b8157870 0x7fc5b815b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.99:46034/0 (socket says 172.21.15.99:46034) 2023-10-08T00:11:11.590 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.587+0000 7fc5bd8f8700 1 -- 172.21.15.99:0/2812204710 learned_addr learned my addr 172.21.15.99:0/2812204710 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:11.590 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.587+0000 7fc5be0f9700 1 -- 172.21.15.99:0/2812204710 >> v2:172.21.15.191:3300/0 conn(0x7fc5b8157c40 msgr2=0x7fc5b8158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:11.590 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.587+0000 7fc5be0f9700 1 --2- 172.21.15.99:0/2812204710 >> v2:172.21.15.191:3300/0 conn(0x7fc5b8157c40 0x7fc5b8158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:11.590 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.587+0000 7fc5be0f9700 1 -- 172.21.15.99:0/2812204710 >> v1:172.21.15.191:6789/0 conn(0x7fc5b815a620 legacy=0x7fc5b815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:11.590 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.587+0000 7fc5be0f9700 1 -- 172.21.15.99:0/2812204710 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5b815c560 con 0x7fc5b8157870 2023-10-08T00:11:11.590 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.587+0000 7fc5be8fa700 1 --2- 172.21.15.99:0/2812204710 >> v2:172.21.15.191:3300/0 conn(0x7fc5b8157c40 0x7fc5b8158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:11:11.590 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.587+0000 7fc5be0f9700 1 --2- 172.21.15.99:0/2812204710 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5b8157870 0x7fc5b815b6a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc5b40098e0 tx=0x7fc5b40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6596053b220c6480 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:11.590 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.587+0000 7fc5bd0f7700 1 -- 172.21.15.99:0/2812204710 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b400b640 con 0x7fc5b8157870 2023-10-08T00:11:11.590 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.588+0000 7fc5bd0f7700 1 -- 172.21.15.99:0/2812204710 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5b400b7a0 con 0x7fc5b8157870 2023-10-08T00:11:11.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.588+0000 7fc5bd0f7700 1 -- 172.21.15.99:0/2812204710 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b400bb40 con 0x7fc5b8157870 2023-10-08T00:11:11.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.588+0000 7fc5c035d700 1 -- 172.21.15.99:0/2812204710 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5b8157870 msgr2=0x7fc5b815b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:11.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.588+0000 7fc5c035d700 1 --2- 172.21.15.99:0/2812204710 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5b8157870 0x7fc5b815b6a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc5b40098e0 tx=0x7fc5b40091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:11.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.588+0000 7fc5c035d700 1 -- 172.21.15.99:0/2812204710 shutdown_connections 2023-10-08T00:11:11.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.588+0000 7fc5c035d700 1 --2- 172.21.15.99:0/2812204710 >> v2:172.21.15.191:3300/0 conn(0x7fc5b8157c40 0x7fc5b8158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:11.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.588+0000 7fc5c035d700 1 --2- 172.21.15.99:0/2812204710 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5b8157870 0x7fc5b815b6a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:11.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.588+0000 7fc5c035d700 1 -- 172.21.15.99:0/2812204710 >> 172.21.15.99:0/2812204710 conn(0x7fc5b8068ce0 msgr2=0x7fc5b80690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:11.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.588+0000 7fc5c035d700 1 -- 172.21.15.99:0/2812204710 shutdown_connections 2023-10-08T00:11:11.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.588+0000 7fc5c035d700 1 -- 172.21.15.99:0/2812204710 wait complete. 2023-10-08T00:11:11.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.589+0000 7fc5c035d700 1 Processor -- start 2023-10-08T00:11:11.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.589+0000 7fc5c035d700 1 -- start start 2023-10-08T00:11:11.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.590+0000 7fc5c035d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5b8157870 0x7fc5b815ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:11.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.590+0000 7fc5c035d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5b8157c40 0x7fc5b80cd550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:11.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.590+0000 7fc5c035d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5b815a620 0x7fc5b8154d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:11.595 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.590+0000 7fc5c035d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b815d0d0 con 0x7fc5b815a620 2023-10-08T00:11:11.595 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.590+0000 7fc5c035d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b80cda90 con 0x7fc5b8157c40 2023-10-08T00:11:11.595 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.590+0000 7fc5c035d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b81605e0 con 0x7fc5b8157870 2023-10-08T00:11:11.595 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.590+0000 7fc5be8fa700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5b815a620 0x7fc5b8154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:11.595 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.590+0000 7fc5be8fa700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5b815a620 0x7fc5b8154d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39100/0 (socket says 172.21.15.99:39100) 2023-10-08T00:11:11.595 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.590+0000 7fc5be8fa700 1 -- 172.21.15.99:0/3626461502 learned_addr learned my addr 172.21.15.99:0/3626461502 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:11.595 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.590+0000 7fc5be0f9700 1 --2- 172.21.15.99:0/3626461502 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5b8157870 0x7fc5b815ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:11.595 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.590+0000 7fc5bd8f8700 1 --2- 172.21.15.99:0/3626461502 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5b8157c40 0x7fc5b80cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:11.596 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.590+0000 7fc5be8fa700 1 -- 172.21.15.99:0/3626461502 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5b8157870 msgr2=0x7fc5b815ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:11.596 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.590+0000 7fc5be8fa700 1 --2- 172.21.15.99:0/3626461502 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5b8157870 0x7fc5b815ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:11.596 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.590+0000 7fc5be8fa700 1 -- 172.21.15.99:0/3626461502 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5b8157c40 msgr2=0x7fc5b80cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:11.596 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.590+0000 7fc5be8fa700 1 --2- 172.21.15.99:0/3626461502 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5b8157c40 0x7fc5b80cd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:11.596 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.590+0000 7fc5be8fa700 1 -- 172.21.15.99:0/3626461502 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5b0009a20 con 0x7fc5b815a620 2023-10-08T00:11:11.596 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.590+0000 7fc5be8fa700 1 --2- 172.21.15.99:0/3626461502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5b815a620 0x7fc5b8154d00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0009fc0 tx=0x7fc5b000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:11.596 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.591+0000 7fc5ae7fc700 1 -- 172.21.15.99:0/3626461502 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b000cbd0 con 0x7fc5b815a620 2023-10-08T00:11:11.596 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.591+0000 7fc5ae7fc700 1 -- 172.21.15.99:0/3626461502 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5b000f040 con 0x7fc5b815a620 2023-10-08T00:11:11.597 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.591+0000 7fc5c035d700 1 -- 172.21.15.99:0/3626461502 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5b4009420 con 0x7fc5b815a620 2023-10-08T00:11:11.597 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.591+0000 7fc5ae7fc700 1 -- 172.21.15.99:0/3626461502 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b0014480 con 0x7fc5b815a620 2023-10-08T00:11:11.597 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.591+0000 7fc5c035d700 1 -- 172.21.15.99:0/3626461502 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5b8160ba0 con 0x7fc5b815a620 2023-10-08T00:11:11.597 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.591+0000 7fc5c035d700 1 -- 172.21.15.99:0/3626461502 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5b804c3c0 con 0x7fc5b815a620 2023-10-08T00:11:11.597 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.592+0000 7fc5ae7fc700 1 -- 172.21.15.99:0/3626461502 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc5b0003020 con 0x7fc5b815a620 2023-10-08T00:11:11.597 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.592+0000 7fc5ae7fc700 1 --2- 172.21.15.99:0/3626461502 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc59c066f30 0x7fc59c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:11.597 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.592+0000 7fc5ae7fc700 1 -- 172.21.15.99:0/3626461502 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc5b0084e50 con 0x7fc5b815a620 2023-10-08T00:11:11.597 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.593+0000 7fc5be0f9700 1 --2- 172.21.15.99:0/3626461502 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc59c066f30 0x7fc59c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:11.597 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.593+0000 7fc5be0f9700 1 --2- 172.21.15.99:0/3626461502 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc59c066f30 0x7fc59c0693e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc5b40091e0 tx=0x7fc5b4017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:11.598 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.596+0000 7fc5ae7fc700 1 -- 172.21.15.99:0/3626461502 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc5b00520c0 con 0x7fc5b815a620 2023-10-08T00:11:11.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.820+0000 7fc5c035d700 1 -- 172.21.15.99:0/3626461502 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc5b815d210 con 0x7fc5b815a620 2023-10-08T00:11:11.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.820+0000 7fc5ae7fc700 1 -- 172.21.15.99:0/3626461502 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4594 (secure 0 0 0) 0x7fc5b00556e0 con 0x7fc5b815a620 2023-10-08T00:11:11.821 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:11:11.821 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/649537180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":649537180},{"type":"v1","addr":"172.21.15.99:6833","nonce":649537180}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4239,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2194110899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2194110899},{"type":"v1","addr":"172.21.15.99:6835","nonce":2194110899}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4251,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/1732443753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":1732443753},{"type":"v1","addr":"172.21.15.191:6837","nonce":1732443753}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:10:53.454321+0000","modified":"2023-10-08T00:10:57.098731+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6836/1988845948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1988845948},{"type":"v1","addr":"172.21.15.191:6836","nonce":1988845948}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-08T00:11:11.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.822+0000 7fc5c035d700 1 -- 172.21.15.99:0/3626461502 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc59c066f30 msgr2=0x7fc59c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:11.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.822+0000 7fc5c035d700 1 --2- 172.21.15.99:0/3626461502 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc59c066f30 0x7fc59c0693e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc5b40091e0 tx=0x7fc5b4017000 comp rx=0 tx=0).stop 2023-10-08T00:11:11.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.822+0000 7fc5c035d700 1 -- 172.21.15.99:0/3626461502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5b815a620 msgr2=0x7fc5b8154d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:11.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.822+0000 7fc5c035d700 1 --2- 172.21.15.99:0/3626461502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5b815a620 0x7fc5b8154d00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0009fc0 tx=0x7fc5b000b040 comp rx=0 tx=0).stop 2023-10-08T00:11:11.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.823+0000 7fc5c035d700 1 -- 172.21.15.99:0/3626461502 shutdown_connections 2023-10-08T00:11:11.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.823+0000 7fc5c035d700 1 --2- 172.21.15.99:0/3626461502 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc59c066f30 0x7fc59c0693e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:11.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.823+0000 7fc5c035d700 1 --2- 172.21.15.99:0/3626461502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5b815a620 0x7fc5b8154d00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:11.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.823+0000 7fc5c035d700 1 --2- 172.21.15.99:0/3626461502 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5b8157870 0x7fc5b815ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:11.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.823+0000 7fc5c035d700 1 --2- 172.21.15.99:0/3626461502 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5b8157c40 0x7fc5b80cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:11.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.823+0000 7fc5c035d700 1 -- 172.21.15.99:0/3626461502 >> 172.21.15.99:0/3626461502 conn(0x7fc5b8068ce0 msgr2=0x7fc5b80b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:11.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.823+0000 7fc5c035d700 1 -- 172.21.15.99:0/3626461502 shutdown_connections 2023-10-08T00:11:11.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:11.823+0000 7fc5c035d700 1 -- 172.21.15.99:0/3626461502 wait complete. 2023-10-08T00:11:11.825 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 8 2023-10-08T00:11:12.237 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T00:10:53.454321+0000', 'modified': '2023-10-08T00:10:57.098731+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4282}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4282': {'gid': 4282, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.191:6836/1988845948', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6834', 'nonce': 1988845948}, {'type': 'v1', 'addr': '172.21.15.191:6836', 'nonce': 1988845948}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-08T00:11:12.237 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T00:11:12.238 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-08T00:11:12.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.356+0000 7fa373aca700 1 Processor -- start 2023-10-08T00:11:12.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.356+0000 7fa373aca700 1 -- start start 2023-10-08T00:11:12.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.357+0000 7fa373aca700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa36c15a620 0x7fa36c15d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:12.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.357+0000 7fa373aca700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa36c15d580 0x7fa36c158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:12.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.357+0000 7fa373aca700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa36c15cbb0 con 0x7fa36c157870 2023-10-08T00:11:12.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.357+0000 7fa373aca700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa36c15bdb0 con 0x7fa36c15a620 2023-10-08T00:11:12.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.357+0000 7fa373aca700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa36c15bfa0 con 0x7fa36c15d580 2023-10-08T00:11:12.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.357+0000 7fa371866700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fa36c157870 0x7fa36c15b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45208/0 (socket says 172.21.15.99:45208) 2023-10-08T00:11:12.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.357+0000 7fa371866700 1 -- 172.21.15.99:0/3634450271 learned_addr learned my addr 172.21.15.99:0/3634450271 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:12.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.357+0000 7fa372067700 1 --2- 172.21.15.99:0/3634450271 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa36c15d580 0x7fa36c158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:12.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.357+0000 7fa371065700 1 --2- 172.21.15.99:0/3634450271 >> v2:172.21.15.191:3300/0 conn(0x7fa36c15a620 0x7fa36c15d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:12.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.357+0000 7fa370864700 1 -- 172.21.15.99:0/3634450271 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3077453950 0 0) 0x7fa36c15cbb0 con 0x7fa36c157870 2023-10-08T00:11:12.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.357+0000 7fa370864700 1 -- 172.21.15.99:0/3634450271 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa358003570 con 0x7fa36c157870 2023-10-08T00:11:12.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.357+0000 7fa370864700 1 -- 172.21.15.99:0/3634450271 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2739868608 0 0) 0x7fa358003570 con 0x7fa36c157870 2023-10-08T00:11:12.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.357+0000 7fa370864700 1 -- 172.21.15.99:0/3634450271 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa36c15d580 msgr2=0x7fa36c158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:12.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.357+0000 7fa370864700 1 --2- 172.21.15.99:0/3634450271 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa36c15d580 0x7fa36c158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:12.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.357+0000 7fa370864700 1 -- 172.21.15.99:0/3634450271 >> v2:172.21.15.191:3300/0 conn(0x7fa36c15a620 msgr2=0x7fa36c15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:12.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.357+0000 7fa370864700 1 --2- 172.21.15.99:0/3634450271 >> v2:172.21.15.191:3300/0 conn(0x7fa36c15a620 0x7fa36c15d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:12.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.357+0000 7fa370864700 1 -- 172.21.15.99:0/3634450271 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa36c15c280 con 0x7fa36c157870 2023-10-08T00:11:12.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.357+0000 7fa370864700 1 -- 172.21.15.99:0/3634450271 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7fa368001470 con 0x7fa36c157870 2023-10-08T00:11:12.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.357+0000 7fa370864700 1 -- 172.21.15.99:0/3634450271 >> v1:172.21.15.99:6789/0 conn(0x7fa36c157870 legacy=0x7fa36c15b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:12.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.357+0000 7fa370864700 1 --2- 172.21.15.99:0/3634450271 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa358003a60 0x7fa358003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:12.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.358+0000 7fa370864700 1 --2- 172.21.15.99:0/3634450271 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa36c15d580 0x7fa358004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:12.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.358+0000 7fa370864700 1 --2- 172.21.15.99:0/3634450271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa358004d50 0x7fa3580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:12.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.358+0000 7fa370864700 1 -- 172.21.15.99:0/3634450271 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36c15bfa0 con 0x7fa358004d50 2023-10-08T00:11:12.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.358+0000 7fa370864700 1 -- 172.21.15.99:0/3634450271 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35800ba60 con 0x7fa358003a60 2023-10-08T00:11:12.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.358+0000 7fa370864700 1 -- 172.21.15.99:0/3634450271 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa35800bba0 con 0x7fa36c15d580 2023-10-08T00:11:12.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.358+0000 7fa372067700 1 --2- 172.21.15.99:0/3634450271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa358004d50 0x7fa3580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:12.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.358+0000 7fa371866700 1 --2- 172.21.15.99:0/3634450271 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa358003a60 0x7fa358003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:12.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.358+0000 7fa371065700 1 --2- 172.21.15.99:0/3634450271 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa36c15d580 0x7fa358004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:12.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.358+0000 7fa372067700 1 -- 172.21.15.99:0/3634450271 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa36c15d580 msgr2=0x7fa358004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:12.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.358+0000 7fa372067700 1 --2- 172.21.15.99:0/3634450271 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa36c15d580 0x7fa358004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:12.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.358+0000 7fa372067700 1 -- 172.21.15.99:0/3634450271 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa358003a60 msgr2=0x7fa358003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:12.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.358+0000 7fa372067700 1 --2- 172.21.15.99:0/3634450271 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa358003a60 0x7fa358003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:12.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.358+0000 7fa372067700 1 -- 172.21.15.99:0/3634450271 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa358003570 con 0x7fa358004d50 2023-10-08T00:11:12.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.358+0000 7fa372067700 1 --2- 172.21.15.99:0/3634450271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa358004d50 0x7fa3580051c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa36c15d9e0 tx=0x7fa36400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69ef40e30d5e9f86 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:12.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.358+0000 7fa370864700 1 -- 172.21.15.99:0/3634450271 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36400ea30 con 0x7fa358004d50 2023-10-08T00:11:12.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.358+0000 7fa370864700 1 -- 172.21.15.99:0/3634450271 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa36400f040 con 0x7fa358004d50 2023-10-08T00:11:12.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.358+0000 7fa370864700 1 -- 172.21.15.99:0/3634450271 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36400ede0 con 0x7fa358004d50 2023-10-08T00:11:12.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.359+0000 7fa373aca700 1 -- 172.21.15.99:0/3634450271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa358004d50 msgr2=0x7fa3580051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:12.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.359+0000 7fa373aca700 1 --2- 172.21.15.99:0/3634450271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa358004d50 0x7fa3580051c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa36c15d9e0 tx=0x7fa36400b040 comp rx=0 tx=0).stop 2023-10-08T00:11:12.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.359+0000 7fa372067700 1 -- 172.21.15.99:0/3634450271 reap_dead start 2023-10-08T00:11:12.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.359+0000 7fa373aca700 1 -- 172.21.15.99:0/3634450271 shutdown_connections 2023-10-08T00:11:12.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.359+0000 7fa373aca700 1 -- 172.21.15.99:0/3634450271 >> 172.21.15.99:0/3634450271 conn(0x7fa36c068ce0 msgr2=0x7fa36c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:12.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.359+0000 7fa373aca700 1 -- 172.21.15.99:0/3634450271 shutdown_connections 2023-10-08T00:11:12.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.359+0000 7fa373aca700 1 -- 172.21.15.99:0/3634450271 wait complete. 2023-10-08T00:11:12.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.360+0000 7fa373aca700 1 Processor -- start 2023-10-08T00:11:12.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.360+0000 7fa373aca700 1 -- start start 2023-10-08T00:11:12.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.361+0000 7fa373aca700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa358004d50 0x7fa36c0b0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:12.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.361+0000 7fa373aca700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa36c0b2290 0x7fa36c0b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:12.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.361+0000 7fa373aca700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa36c0cf7b0 0x7fa36c0cfc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:12.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.361+0000 7fa373aca700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36c159200 con 0x7fa358004d50 2023-10-08T00:11:12.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.361+0000 7fa373aca700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36c0b1e00 con 0x7fa36c0b2290 2023-10-08T00:11:12.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.361+0000 7fa373aca700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa36c0b11f0 con 0x7fa36c0cf7b0 2023-10-08T00:11:12.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.361+0000 7fa371866700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa358004d50 0x7fa36c0b0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:12.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.361+0000 7fa371866700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa358004d50 0x7fa36c0b0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39122/0 (socket says 172.21.15.99:39122) 2023-10-08T00:11:12.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.361+0000 7fa371866700 1 -- 172.21.15.99:0/3718461512 learned_addr learned my addr 172.21.15.99:0/3718461512 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:12.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.361+0000 7fa372067700 1 --2- 172.21.15.99:0/3718461512 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa36c0cf7b0 0x7fa36c0cfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:12.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.361+0000 7fa371065700 1 --2- 172.21.15.99:0/3718461512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa36c0b2290 0x7fa36c0b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:12.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.361+0000 7fa371866700 1 -- 172.21.15.99:0/3718461512 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa36c0cf7b0 msgr2=0x7fa36c0cfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:12.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.361+0000 7fa371866700 1 --2- 172.21.15.99:0/3718461512 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa36c0cf7b0 0x7fa36c0cfc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:12.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.361+0000 7fa371866700 1 -- 172.21.15.99:0/3718461512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa36c0b2290 msgr2=0x7fa36c0b0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:12.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.361+0000 7fa371866700 1 --2- 172.21.15.99:0/3718461512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa36c0b2290 0x7fa36c0b0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:12.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.361+0000 7fa371866700 1 -- 172.21.15.99:0/3718461512 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa36c0b14d0 con 0x7fa358004d50 2023-10-08T00:11:12.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.361+0000 7fa371866700 1 --2- 172.21.15.99:0/3718461512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa358004d50 0x7fa36c0b0610 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa368001d20 tx=0x7fa368001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:12.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.361+0000 7fa361ffb700 1 -- 172.21.15.99:0/3718461512 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa368018030 con 0x7fa358004d50 2023-10-08T00:11:12.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.361+0000 7fa361ffb700 1 -- 172.21.15.99:0/3718461512 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa368007740 con 0x7fa358004d50 2023-10-08T00:11:12.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.361+0000 7fa361ffb700 1 -- 172.21.15.99:0/3718461512 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36800faf0 con 0x7fa358004d50 2023-10-08T00:11:12.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.361+0000 7fa373aca700 1 -- 172.21.15.99:0/3718461512 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa36c0b17b0 con 0x7fa358004d50 2023-10-08T00:11:12.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.361+0000 7fa373aca700 1 -- 172.21.15.99:0/3718461512 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa36c0d0170 con 0x7fa358004d50 2023-10-08T00:11:12.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.363+0000 7fa361ffb700 1 -- 172.21.15.99:0/3718461512 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa36801b070 con 0x7fa358004d50 2023-10-08T00:11:12.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.363+0000 7fa361ffb700 1 --2- 172.21.15.99:0/3718461512 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa350066f30 0x7fa3500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:12.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.363+0000 7fa361ffb700 1 -- 172.21.15.99:0/3718461512 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa368019070 con 0x7fa358004d50 2023-10-08T00:11:12.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.363+0000 7fa371065700 1 --2- 172.21.15.99:0/3718461512 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa350066f30 0x7fa3500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:12.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.364+0000 7fa371065700 1 --2- 172.21.15.99:0/3718461512 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa350066f30 0x7fa3500693e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa36c0b2680 tx=0x7fa35c0091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:12.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.365+0000 7fa37cdc2b80 1 Processor -- start 2023-10-08T00:11:12.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.365+0000 7fa37cdc2b80 1 -- start start 2023-10-08T00:11:12.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.366+0000 7fa37cdc2b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5573216bda50 0x5573216bde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:12.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.366+0000 7fa37cdc2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557321707d00 0x5573216e6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:12.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.366+0000 7fa37cdc2b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5573216e6f60 0x5573216b9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:12.378 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.366+0000 7fa37cdc2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557321614a70 con 0x557321707d00 2023-10-08T00:11:12.378 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.366+0000 7fa37cdc2b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x557321612270 con 0x5573216e6f60 2023-10-08T00:11:12.378 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.366+0000 7fa37cdc2b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x55732160e060 con 0x5573216bda50 2023-10-08T00:11:12.378 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.366+0000 7fa371065700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557321707d00 0x5573216e6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:12.378 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.366+0000 7fa371065700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557321707d00 0x5573216e6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39128/0 (socket says 172.21.15.99:39128) 2023-10-08T00:11:12.379 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.366+0000 7fa371065700 1 -- 172.21.15.99:0/497025803 learned_addr learned my addr 172.21.15.99:0/497025803 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:12.379 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.366+0000 7fa371866700 1 --2- 172.21.15.99:0/497025803 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5573216bda50 0x5573216bde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:12.379 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.366+0000 7fa372067700 1 --2- 172.21.15.99:0/497025803 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5573216e6f60 0x5573216b9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:12.379 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.366+0000 7fa371065700 1 -- 172.21.15.99:0/497025803 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5573216bda50 msgr2=0x5573216bde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:12.379 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.366+0000 7fa371065700 1 --2- 172.21.15.99:0/497025803 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5573216bda50 0x5573216bde20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:12.379 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.366+0000 7fa371065700 1 -- 172.21.15.99:0/497025803 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5573216e6f60 msgr2=0x5573216b9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:12.379 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.366+0000 7fa371065700 1 --2- 172.21.15.99:0/497025803 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5573216e6f60 0x5573216b9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:12.379 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.366+0000 7fa371065700 1 -- 172.21.15.99:0/497025803 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573214a8270 con 0x557321707d00 2023-10-08T00:11:12.379 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.366+0000 7fa372067700 1 --2- 172.21.15.99:0/497025803 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5573216e6f60 0x5573216b9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:11:12.380 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.366+0000 7fa371065700 1 --2- 172.21.15.99:0/497025803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557321707d00 0x5573216e6a20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa35c014ce0 tx=0x7fa35c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e84d48b2b28e1b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:12.380 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.366+0000 7fa34effd700 1 -- 172.21.15.99:0/497025803 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c015b70 con 0x557321707d00 2023-10-08T00:11:12.380 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.366+0000 7fa34effd700 1 -- 172.21.15.99:0/497025803 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa35c019070 con 0x557321707d00 2023-10-08T00:11:12.380 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.366+0000 7fa34effd700 1 -- 172.21.15.99:0/497025803 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c022430 con 0x557321707d00 2023-10-08T00:11:12.380 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.367+0000 7fa37cdc2b80 1 -- 172.21.15.99:0/497025803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557321707d00 msgr2=0x5573216e6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:12.380 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.367+0000 7fa37cdc2b80 1 --2- 172.21.15.99:0/497025803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557321707d00 0x5573216e6a20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa35c014ce0 tx=0x7fa35c01c000 comp rx=0 tx=0).stop 2023-10-08T00:11:12.380 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.367+0000 7fa37cdc2b80 1 -- 172.21.15.99:0/497025803 shutdown_connections 2023-10-08T00:11:12.380 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.367+0000 7fa37cdc2b80 1 --2- 172.21.15.99:0/497025803 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5573216e6f60 0x5573216b9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:12.380 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.367+0000 7fa37cdc2b80 1 --2- 172.21.15.99:0/497025803 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5573216bda50 0x5573216bde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:12.380 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.367+0000 7fa37cdc2b80 1 --2- 172.21.15.99:0/497025803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557321707d00 0x5573216e6a20 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:12.380 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.367+0000 7fa37cdc2b80 1 -- 172.21.15.99:0/497025803 >> 172.21.15.99:0/497025803 conn(0x5573216bf370 msgr2=0x5573216bf740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:12.380 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.367+0000 7fa37cdc2b80 1 -- 172.21.15.99:0/497025803 shutdown_connections 2023-10-08T00:11:12.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.367+0000 7fa37cdc2b80 1 -- 172.21.15.99:0/497025803 wait complete. 2023-10-08T00:11:12.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.367+0000 7fa37cdc2b80 1 -- start start 2023-10-08T00:11:12.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.368+0000 7fa37cdc2b80 1 Processor -- start 2023-10-08T00:11:12.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.368+0000 7fa37cdc2b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5573216bda50 0x557321716cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:12.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.368+0000 7fa37cdc2b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5573216e6f60 0x557321707170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:12.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.368+0000 7fa37cdc2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557321707d00 0x5573216fde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:12.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.368+0000 7fa37cdc2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573215b9890 con 0x557321707d00 2023-10-08T00:11:12.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.368+0000 7fa37cdc2b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573215d4ea0 con 0x5573216bda50 2023-10-08T00:11:12.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.368+0000 7fa37cdc2b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x557321490e30 con 0x5573216e6f60 2023-10-08T00:11:12.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.369+0000 7fa37cdc2b80 10 client.0 _ll_register_callbacks cb 0x557321681110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T00:11:12.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.369+0000 7fa372067700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557321707d00 0x5573216fde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:12.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.369+0000 7fa372067700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557321707d00 0x5573216fde70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39136/0 (socket says 172.21.15.99:39136) 2023-10-08T00:11:12.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.369+0000 7fa371065700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5573216e6f60 0x557321707170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:12.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.369+0000 7fa372067700 1 -- 172.21.15.99:0/3229492156 learned_addr learned my addr 172.21.15.99:0/3229492156 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:12.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.369+0000 7fa371866700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5573216bda50 0x557321716cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:12.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.369+0000 7fa372067700 1 -- 172.21.15.99:0/3229492156 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5573216e6f60 msgr2=0x557321707170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:12.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.369+0000 7fa372067700 1 --2- 172.21.15.99:0/3229492156 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5573216e6f60 0x557321707170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:12.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.369+0000 7fa372067700 1 -- 172.21.15.99:0/3229492156 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5573216bda50 msgr2=0x557321716cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:12.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.369+0000 7fa372067700 1 --2- 172.21.15.99:0/3229492156 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5573216bda50 0x557321716cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:12.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.369+0000 7fa372067700 1 -- 172.21.15.99:0/3229492156 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa35c0148c0 con 0x557321707d00 2023-10-08T00:11:12.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.369+0000 7fa372067700 1 --2- 172.21.15.99:0/3229492156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557321707d00 0x5573216fde70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa358004760 tx=0x7fa36401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e156a176f7ae0907 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:12.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.369+0000 7fa347fff700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T00:11:12.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.369+0000 7fa347fff700 1 -- 172.21.15.99:0/3229492156 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa364023070 con 0x557321707d00 2023-10-08T00:11:12.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.369+0000 7fa347fff700 1 -- 172.21.15.99:0/3229492156 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa3640148a0 con 0x557321707d00 2023-10-08T00:11:12.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.369+0000 7fa37cdc2b80 1 -- 172.21.15.99:0/3229492156 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5573216b2b70 con 0x557321707d00 2023-10-08T00:11:12.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.369+0000 7fa347fff700 1 -- 172.21.15.99:0/3229492156 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa364014c70 con 0x557321707d00 2023-10-08T00:11:12.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.369+0000 7fa37cdc2b80 10 client.4632 fetch_fsmap learned FSMap version 8 2023-10-08T00:11:12.384 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.369+0000 7fa37cdc2b80 1 -- 172.21.15.99:0/3229492156 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55732144afe0 con 0x557321707d00 2023-10-08T00:11:12.384 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.369+0000 7fa347fff700 1 -- 172.21.15.99:0/3229492156 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5573216b2b70 con 0x557321707d00 2023-10-08T00:11:12.384 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.369+0000 7fa347fff700 1 -- 172.21.15.99:0/3229492156 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 8) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fa364028460 con 0x557321707d00 2023-10-08T00:11:12.384 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.370+0000 7fa37cdc2b80 10 client.4632 fetch_fsmap finished waiting for FSMap version 8 2023-10-08T00:11:12.384 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.370+0000 7fa37cdc2b80 10 client.4632 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-08T00:11:12.384 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.370+0000 7fa37cdc2b80 20 client.4632 populate_metadata read hostname 'smithi099' 2023-10-08T00:11:12.384 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.370+0000 7fa37cdc2b80 1 --2- 172.21.15.99:0/3229492156 >> [v2:172.21.15.191:6834/1988845948,v1:172.21.15.191:6836/1988845948] conn(0x557321715610 0x5573217159e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:12.384 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.370+0000 7fa37cdc2b80 4 client.4632 mds_command: new command op to 4282 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-08T00:11:12.384 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.370+0000 7fa37cdc2b80 1 -- 172.21.15.99:0/3229492156 --> [v2:172.21.15.191:6834/1988845948,v1:172.21.15.191:6836/1988845948] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55732168faa0 con 0x557321715610 2023-10-08T00:11:12.384 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.370+0000 7fa371866700 1 --2- 172.21.15.99:0/3229492156 >> [v2:172.21.15.191:6834/1988845948,v1:172.21.15.191:6836/1988845948] conn(0x557321715610 0x5573217159e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:12.384 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.370+0000 7fa347fff700 10 client.4632 ms_handle_connect on v2:172.21.15.191:6834/1988845948 2023-10-08T00:11:12.384 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.370+0000 7fa371866700 1 --2- 172.21.15.99:0/3229492156 >> [v2:172.21.15.191:6834/1988845948,v1:172.21.15.191:6836/1988845948] conn(0x557321715610 0x5573217159e0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d57b6ca25e47f9f1 server_cookie=a980a6c6548a424a in_seq=0 out_seq=0 2023-10-08T00:11:12.385 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.371+0000 7fa347fff700 1 -- 172.21.15.99:0/3229492156 <== mds.0 v2:172.21.15.191:6834/1988845948 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55732168faa0 con 0x557321715610 2023-10-08T00:11:12.385 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.371+0000 7fa347fff700 10 client.4632 handle_command_reply: tid=0 2023-10-08T00:11:12.385 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.371+0000 7fa37cdc2b80 1 client.4632 shutdown 2023-10-08T00:11:12.385 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.371+0000 7fa37cdc2b80 1 -- 172.21.15.99:0/3229492156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557321707d00 msgr2=0x5573216fde70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:12.385 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.371+0000 7fa37cdc2b80 1 --2- 172.21.15.99:0/3229492156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557321707d00 0x5573216fde70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa358004760 tx=0x7fa36401a000 comp rx=0 tx=0).stop 2023-10-08T00:11:12.385 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.371+0000 7fa37cdc2b80 1 -- 172.21.15.99:0/3229492156 shutdown_connections 2023-10-08T00:11:12.385 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.371+0000 7fa37cdc2b80 1 --2- 172.21.15.99:0/3229492156 >> [v2:172.21.15.191:6834/1988845948,v1:172.21.15.191:6836/1988845948] conn(0x557321715610 0x5573217159e0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:12.385 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.371+0000 7fa37cdc2b80 1 --2- 172.21.15.99:0/3229492156 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5573216bda50 0x557321716cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:12.385 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.371+0000 7fa347fff700 0 client.4632 ms_handle_reset on v2:172.21.15.191:6834/1988845948 2023-10-08T00:11:12.385 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.371+0000 7fa37cdc2b80 1 --2- 172.21.15.99:0/3229492156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557321707d00 0x5573216fde70 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:12.385 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.371+0000 7fa37cdc2b80 1 --2- 172.21.15.99:0/3229492156 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5573216e6f60 0x557321707170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:12.385 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.371+0000 7fa37cdc2b80 1 -- 172.21.15.99:0/3229492156 >> 172.21.15.99:0/3229492156 conn(0x5573216bf370 msgr2=0x5573216b9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:12.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.371+0000 7fa37cdc2b80 1 -- 172.21.15.99:0/3229492156 shutdown_connections 2023-10-08T00:11:12.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.371+0000 7fa37cdc2b80 1 -- 172.21.15.99:0/3229492156 wait complete. 2023-10-08T00:11:12.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.371+0000 7fa37cdc2b80 20 client.4632 trim_cache size 0 max 16384 2023-10-08T00:11:12.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.388+0000 7fa37cdc2b80 1 Processor -- start 2023-10-08T00:11:12.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.388+0000 7fa37cdc2b80 1 -- start start 2023-10-08T00:11:12.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.388+0000 7fa37cdc2b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5573216bda50 0x5573216fe070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:12.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.388+0000 7fa37cdc2b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5573216e6f60 0x557321713710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:12.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.388+0000 7fa37cdc2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557321707d00 0x5573217159e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:12.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.388+0000 7fa37cdc2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573216b65e0 con 0x557321707d00 2023-10-08T00:11:12.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.388+0000 7fa37cdc2b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x557321714bf0 con 0x5573216bda50 2023-10-08T00:11:12.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.388+0000 7fa37cdc2b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x557321696410 con 0x5573216e6f60 2023-10-08T00:11:12.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.388+0000 7fa372067700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557321707d00 0x5573217159e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:12.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.388+0000 7fa372067700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557321707d00 0x5573217159e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39152/0 (socket says 172.21.15.99:39152) 2023-10-08T00:11:12.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.388+0000 7fa372067700 1 -- 172.21.15.99:0/4232749014 learned_addr learned my addr 172.21.15.99:0/4232749014 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:12.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.389+0000 7fa371866700 1 --2- 172.21.15.99:0/4232749014 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5573216bda50 0x5573216fe070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:12.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.389+0000 7fa372067700 1 -- 172.21.15.99:0/4232749014 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5573216e6f60 msgr2=0x557321713710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:12.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.389+0000 7fa372067700 1 --2- 172.21.15.99:0/4232749014 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5573216e6f60 0x557321713710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:12.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.389+0000 7fa372067700 1 -- 172.21.15.99:0/4232749014 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5573216bda50 msgr2=0x5573216fe070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:12.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.389+0000 7fa372067700 1 --2- 172.21.15.99:0/4232749014 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5573216bda50 0x5573216fe070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:12.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.389+0000 7fa372067700 1 -- 172.21.15.99:0/4232749014 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55732171cff0 con 0x557321707d00 2023-10-08T00:11:12.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.389+0000 7fa372067700 1 --2- 172.21.15.99:0/4232749014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557321707d00 0x5573217159e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa36400e7d0 tx=0x7fa364023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db70c563a39eac7d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:12.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.389+0000 7fa34f7fe700 1 -- 172.21.15.99:0/4232749014 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa364038070 con 0x557321707d00 2023-10-08T00:11:12.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.389+0000 7fa34f7fe700 1 -- 172.21.15.99:0/4232749014 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa364028970 con 0x557321707d00 2023-10-08T00:11:12.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.389+0000 7fa34f7fe700 1 -- 172.21.15.99:0/4232749014 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa364028460 con 0x557321707d00 2023-10-08T00:11:12.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.389+0000 7fa37cdc2b80 1 -- 172.21.15.99:0/4232749014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557321707d00 msgr2=0x5573217159e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:12.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.389+0000 7fa37cdc2b80 1 --2- 172.21.15.99:0/4232749014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557321707d00 0x5573217159e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa36400e7d0 tx=0x7fa364023000 comp rx=0 tx=0).stop 2023-10-08T00:11:12.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.389+0000 7fa37cdc2b80 1 -- 172.21.15.99:0/4232749014 shutdown_connections 2023-10-08T00:11:12.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.389+0000 7fa37cdc2b80 1 --2- 172.21.15.99:0/4232749014 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5573216bda50 0x5573216fe070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:12.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.389+0000 7fa37cdc2b80 1 --2- 172.21.15.99:0/4232749014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557321707d00 0x5573217159e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:12.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.389+0000 7fa37cdc2b80 1 --2- 172.21.15.99:0/4232749014 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5573216e6f60 0x557321713710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:12.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.389+0000 7fa37cdc2b80 1 -- 172.21.15.99:0/4232749014 >> 172.21.15.99:0/4232749014 conn(0x5573216bf370 msgr2=0x5573217037e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:12.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.389+0000 7fa37cdc2b80 1 -- 172.21.15.99:0/4232749014 shutdown_connections 2023-10-08T00:11:12.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.390+0000 7fa37cdc2b80 1 -- 172.21.15.99:0/4232749014 wait complete. 2023-10-08T00:11:12.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.390+0000 7fa37cdc2b80 1 -- start start 2023-10-08T00:11:12.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.390+0000 7fa37cdc2b80 1 Processor -- start 2023-10-08T00:11:12.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.391+0000 7fa37cdc2b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5573216bda50 0x5573216e5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:12.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.391+0000 7fa37cdc2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573216e6f60 0x557321766990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:12.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.391+0000 7fa37cdc2b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557321707d00 0x5573217159e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:12.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.391+0000 7fa37cdc2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55732176a590 con 0x5573216e6f60 2023-10-08T00:11:12.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.391+0000 7fa37cdc2b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55732176a9b0 con 0x557321707d00 2023-10-08T00:11:12.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.391+0000 7fa37cdc2b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x557321700bf0 con 0x5573216bda50 2023-10-08T00:11:12.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.391+0000 7fa37cdc2b80 10 client.0 _ll_register_callbacks cb 0x557321681110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T00:11:12.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.391+0000 7fa371065700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573216e6f60 0x557321766990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:12.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.391+0000 7fa371065700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573216e6f60 0x557321766990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39154/0 (socket says 172.21.15.99:39154) 2023-10-08T00:11:12.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.391+0000 7fa371065700 1 -- 172.21.15.99:0/488258856 learned_addr learned my addr 172.21.15.99:0/488258856 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:12.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.391+0000 7fa371866700 1 --2- 172.21.15.99:0/488258856 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5573216bda50 0x5573216e5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:12.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.391+0000 7fa372067700 1 --2- 172.21.15.99:0/488258856 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557321707d00 0x5573217159e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:12.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.391+0000 7fa371065700 1 -- 172.21.15.99:0/488258856 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5573216bda50 msgr2=0x5573216e5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:12.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.391+0000 7fa371065700 1 --2- 172.21.15.99:0/488258856 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5573216bda50 0x5573216e5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:12.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.391+0000 7fa371065700 1 -- 172.21.15.99:0/488258856 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557321707d00 msgr2=0x5573217159e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:12.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.391+0000 7fa371065700 1 --2- 172.21.15.99:0/488258856 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557321707d00 0x5573217159e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:12.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.391+0000 7fa371065700 1 -- 172.21.15.99:0/488258856 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55732144afe0 con 0x5573216e6f60 2023-10-08T00:11:12.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.391+0000 7fa371866700 1 --2- 172.21.15.99:0/488258856 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5573216bda50 0x5573216e5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:11:12.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.391+0000 7fa347fff700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T00:11:12.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.391+0000 7fa371065700 1 --2- 172.21.15.99:0/488258856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573216e6f60 0x557321766990 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa35c016000 tx=0x7fa35c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97109153252a2323 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:12.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.392+0000 7fa347fff700 1 -- 172.21.15.99:0/488258856 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c022430 con 0x5573216e6f60 2023-10-08T00:11:12.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.392+0000 7fa347fff700 1 -- 172.21.15.99:0/488258856 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa35c0363f0 con 0x5573216e6f60 2023-10-08T00:11:12.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.392+0000 7fa37cdc2b80 1 -- 172.21.15.99:0/488258856 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa35c019070 con 0x5573216e6f60 2023-10-08T00:11:12.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.392+0000 7fa347fff700 1 -- 172.21.15.99:0/488258856 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa35c0367c0 con 0x5573216e6f60 2023-10-08T00:11:12.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.392+0000 7fa37cdc2b80 10 client.4638 fetch_fsmap learned FSMap version 8 2023-10-08T00:11:12.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.392+0000 7fa37cdc2b80 1 -- 172.21.15.99:0/488258856 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x5573215dc780 con 0x5573216e6f60 2023-10-08T00:11:12.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.392+0000 7fa347fff700 1 -- 172.21.15.99:0/488258856 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa35c019070 con 0x5573216e6f60 2023-10-08T00:11:12.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.392+0000 7fa347fff700 1 -- 172.21.15.99:0/488258856 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 8) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fa35c035070 con 0x5573216e6f60 2023-10-08T00:11:12.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.392+0000 7fa37cdc2b80 10 client.4638 fetch_fsmap finished waiting for FSMap version 8 2023-10-08T00:11:12.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.392+0000 7fa37cdc2b80 10 client.4638 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-08T00:11:12.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.392+0000 7fa37cdc2b80 20 client.4638 populate_metadata read hostname 'smithi099' 2023-10-08T00:11:12.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.392+0000 7fa37cdc2b80 1 --2- 172.21.15.99:0/488258856 >> [v2:172.21.15.191:6834/1988845948,v1:172.21.15.191:6836/1988845948] conn(0x5573217014a0 0x557321701870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:12.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.392+0000 7fa37cdc2b80 4 client.4638 mds_command: new command op to 4282 tid=0[{"prefix": "status"}] 2023-10-08T00:11:12.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.392+0000 7fa37cdc2b80 1 -- 172.21.15.99:0/488258856 --> [v2:172.21.15.191:6834/1988845948,v1:172.21.15.191:6836/1988845948] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5573215b0e00 con 0x5573217014a0 2023-10-08T00:11:12.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.392+0000 7fa371866700 1 --2- 172.21.15.99:0/488258856 >> [v2:172.21.15.191:6834/1988845948,v1:172.21.15.191:6836/1988845948] conn(0x5573217014a0 0x557321701870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:12.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.392+0000 7fa347fff700 10 client.4638 ms_handle_connect on v2:172.21.15.191:6834/1988845948 2023-10-08T00:11:12.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.392+0000 7fa371866700 1 --2- 172.21.15.99:0/488258856 >> [v2:172.21.15.191:6834/1988845948,v1:172.21.15.191:6836/1988845948] conn(0x5573217014a0 0x557321701870 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fab0c33eeb1ee3b server_cookie=9896d120489e7cf5 in_seq=0 out_seq=0 2023-10-08T00:11:12.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.393+0000 7fa347fff700 1 -- 172.21.15.99:0/488258856 <== mds.0 v2:172.21.15.191:6834/1988845948 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x5573215b0e00 con 0x5573217014a0 2023-10-08T00:11:12.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.393+0000 7fa347fff700 10 client.4638 handle_command_reply: tid=0 2023-10-08T00:11:12.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.393+0000 7fa37cdc2b80 1 client.4638 shutdown 2023-10-08T00:11:12.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.393+0000 7fa37cdc2b80 1 -- 172.21.15.99:0/488258856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573216e6f60 msgr2=0x557321766990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:12.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.393+0000 7fa37cdc2b80 1 --2- 172.21.15.99:0/488258856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573216e6f60 0x557321766990 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa35c016000 tx=0x7fa35c01e000 comp rx=0 tx=0).stop 2023-10-08T00:11:12.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.393+0000 7fa37cdc2b80 1 -- 172.21.15.99:0/488258856 shutdown_connections 2023-10-08T00:11:12.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.393+0000 7fa37cdc2b80 1 --2- 172.21.15.99:0/488258856 >> [v2:172.21.15.191:6834/1988845948,v1:172.21.15.191:6836/1988845948] conn(0x5573217014a0 0x557321701870 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:12.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.393+0000 7fa37cdc2b80 1 --2- 172.21.15.99:0/488258856 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557321707d00 0x5573217159e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:12.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.393+0000 7fa37cdc2b80 1 --2- 172.21.15.99:0/488258856 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5573216bda50 0x5573216e5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:12.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.393+0000 7fa37cdc2b80 1 --2- 172.21.15.99:0/488258856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573216e6f60 0x557321766990 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:12.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.393+0000 7fa347fff700 0 client.4638 ms_handle_reset on v2:172.21.15.191:6834/1988845948 2023-10-08T00:11:12.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.393+0000 7fa37cdc2b80 1 -- 172.21.15.99:0/488258856 >> 172.21.15.99:0/488258856 conn(0x5573216bf370 msgr2=0x557321713710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:12.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.393+0000 7fa37cdc2b80 1 -- 172.21.15.99:0/488258856 shutdown_connections 2023-10-08T00:11:12.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.393+0000 7fa37cdc2b80 1 -- 172.21.15.99:0/488258856 wait complete. 2023-10-08T00:11:12.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.393+0000 7fa37cdc2b80 20 client.4638 trim_cache size 0 max 16384 2023-10-08T00:11:12.401 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:11:12.402 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", 2023-10-08T00:11:12.402 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 0, 2023-10-08T00:11:12.402 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 4282, 2023-10-08T00:11:12.402 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2023-10-08T00:11:12.402 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2023-10-08T00:11:12.402 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2023-10-08T00:11:12.402 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 18.312322676000001, 2023-10-08T00:11:12.402 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 8, 2023-10-08T00:11:12.402 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 20, 2023-10-08T00:11:12.402 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 20, 2023-10-08T00:11:12.402 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 25.183533668999999 2023-10-08T00:11:12.403 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:11:12.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.393+0000 7fa373aca700 1 -- 172.21.15.99:0/3718461512 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa350066f30 msgr2=0x7fa3500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:12.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.393+0000 7fa373aca700 1 --2- 172.21.15.99:0/3718461512 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa350066f30 0x7fa3500693e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa36c0b2680 tx=0x7fa35c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:12.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.394+0000 7fa373aca700 1 -- 172.21.15.99:0/3718461512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa358004d50 msgr2=0x7fa36c0b0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:12.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.394+0000 7fa373aca700 1 --2- 172.21.15.99:0/3718461512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa358004d50 0x7fa36c0b0610 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa368001d20 tx=0x7fa368001e90 comp rx=0 tx=0).stop 2023-10-08T00:11:12.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.394+0000 7fa373aca700 1 -- 172.21.15.99:0/3718461512 shutdown_connections 2023-10-08T00:11:12.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.394+0000 7fa373aca700 1 --2- 172.21.15.99:0/3718461512 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa350066f30 0x7fa3500693e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:12.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.394+0000 7fa373aca700 1 --2- 172.21.15.99:0/3718461512 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa36c0cf7b0 0x7fa36c0cfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:12.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.394+0000 7fa373aca700 1 --2- 172.21.15.99:0/3718461512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa358004d50 0x7fa36c0b0610 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:12.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.394+0000 7fa373aca700 1 --2- 172.21.15.99:0/3718461512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa36c0b2290 0x7fa36c0b0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:12.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.394+0000 7fa373aca700 1 -- 172.21.15.99:0/3718461512 >> 172.21.15.99:0/3718461512 conn(0x7fa36c068ce0 msgr2=0x7fa36c15da10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:12.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.394+0000 7fa373aca700 1 -- 172.21.15.99:0/3718461512 shutdown_connections 2023-10-08T00:11:12.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:12.394+0000 7fa373aca700 1 -- 172.21.15.99:0/3718461512 wait complete. 2023-10-08T00:11:12.406 INFO:teuthology.run_tasks:Running task kclient... 2023-10-08T00:11:12.487 INFO:tasks.kclient:Mounting kernel clients... 2023-10-08T00:11:12.487 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-10-08T00:11:12.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:12.487 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2023-10-08T00:11:12.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:12.503 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link delete ceph-brx 2023-10-08T00:11:12.569 INFO:teuthology.orchestra.run.smithi099.stderr:Cannot find device "ceph-brx" 2023-10-08T00:11:12.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T00:11:12.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:12.570 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2023-10-08T00:11:12.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:12.625 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link delete ceph-brx 2023-10-08T00:11:12.690 INFO:teuthology.orchestra.run.smithi099.stderr:Cannot find device "ceph-brx" 2023-10-08T00:11:12.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T00:11:12.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:12.692 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2023-10-08T00:11:12.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:12.710 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link delete ceph-brx 2023-10-08T00:11:12.781 INFO:teuthology.orchestra.run.smithi191.stderr:Cannot find device "ceph-brx" 2023-10-08T00:11:12.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T00:11:12.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:12.783 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2023-10-08T00:11:12.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:12.841 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link delete ceph-brx 2023-10-08T00:11:12.909 INFO:teuthology.orchestra.run.smithi191.stderr:Cannot find device "ceph-brx" 2023-10-08T00:11:12.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T00:11:12.911 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2023-10-08T00:11:12.911 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T00:11:13.026 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.025+0000 7f34f32cc700 1 Processor -- start 2023-10-08T00:11:13.026 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.026+0000 7f34f32cc700 1 -- start start 2023-10-08T00:11:13.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.026+0000 7f34f32cc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34ec156ea0 0x7f34ec0c30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:13.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.026+0000 7f34f32cc700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f34ec06e2b0 0x7f34ec06e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:13.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.026+0000 7f34f32cc700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34ec0c42b0 con 0x7f34ec157270 2023-10-08T00:11:13.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.026+0000 7f34f32cc700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34ec06fac0 con 0x7f34ec06e2b0 2023-10-08T00:11:13.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.026+0000 7f34f32cc700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34ec06ec70 con 0x7f34ec156ea0 2023-10-08T00:11:13.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.027+0000 7f34f1869700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f34ec06e2b0 0x7f34ec06e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:13.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.027+0000 7f34f0867700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f34ec157270 0x7f34ec06f3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45222/0 (socket says 172.21.15.99:45222) 2023-10-08T00:11:13.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.027+0000 7f34f0867700 1 -- 172.21.15.99:0/143023927 learned_addr learned my addr 172.21.15.99:0/143023927 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:13.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.027+0000 7f34f1068700 1 --2- 172.21.15.99:0/143023927 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34ec156ea0 0x7f34ec0c30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:13.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.027+0000 7f34f1869700 1 -- 172.21.15.99:0/143023927 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34ec156ea0 msgr2=0x7f34ec0c30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:13.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.027+0000 7f34f1869700 1 --2- 172.21.15.99:0/143023927 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34ec156ea0 0x7f34ec0c30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:13.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.027+0000 7f34f1869700 1 -- 172.21.15.99:0/143023927 >> v1:172.21.15.99:6789/0 conn(0x7f34ec157270 legacy=0x7f34ec06f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:13.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.027+0000 7f34e3fff700 1 -- 172.21.15.99:0/143023927 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3857918387 0 0) 0x7f34ec0c42b0 con 0x7f34ec157270 2023-10-08T00:11:13.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.027+0000 7f34f1869700 1 -- 172.21.15.99:0/143023927 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34ec0c36a0 con 0x7f34ec06e2b0 2023-10-08T00:11:13.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.027+0000 7f34f1068700 1 --2- 172.21.15.99:0/143023927 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34ec156ea0 0x7f34ec0c30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:11:13.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.027+0000 7f34f1869700 1 --2- 172.21.15.99:0/143023927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34ec06e2b0 0x7f34ec06e730 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f34e40098e0 tx=0x7f34e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e5dd979ad6b609a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:13.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.027+0000 7f34e3fff700 1 -- 172.21.15.99:0/143023927 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e4009e80 con 0x7f34ec06e2b0 2023-10-08T00:11:13.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.027+0000 7f34e3fff700 1 -- 172.21.15.99:0/143023927 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e400b860 con 0x7f34ec06e2b0 2023-10-08T00:11:13.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.027+0000 7f34e3fff700 1 -- 172.21.15.99:0/143023927 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e400bbe0 con 0x7f34ec06e2b0 2023-10-08T00:11:13.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.028+0000 7f34f32cc700 1 -- 172.21.15.99:0/143023927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34ec06e2b0 msgr2=0x7f34ec06e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:13.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.028+0000 7f34f32cc700 1 --2- 172.21.15.99:0/143023927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34ec06e2b0 0x7f34ec06e730 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f34e40098e0 tx=0x7f34e40091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:13.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.028+0000 7f34f32cc700 1 -- 172.21.15.99:0/143023927 shutdown_connections 2023-10-08T00:11:13.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.028+0000 7f34f32cc700 1 --2- 172.21.15.99:0/143023927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34ec06e2b0 0x7f34ec06e730 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:13.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.028+0000 7f34f32cc700 1 --2- 172.21.15.99:0/143023927 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34ec156ea0 0x7f34ec0c30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:13.039 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.028+0000 7f34f32cc700 1 -- 172.21.15.99:0/143023927 >> 172.21.15.99:0/143023927 conn(0x7f34ec0ca4f0 msgr2=0x7f34ec0ca8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:13.039 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.028+0000 7f34f32cc700 1 -- 172.21.15.99:0/143023927 shutdown_connections 2023-10-08T00:11:13.039 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.028+0000 7f34f32cc700 1 -- 172.21.15.99:0/143023927 wait complete. 2023-10-08T00:11:13.039 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.029+0000 7f34f32cc700 1 Processor -- start 2023-10-08T00:11:13.039 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.029+0000 7f34f32cc700 1 -- start start 2023-10-08T00:11:13.039 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.030+0000 7f34f32cc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34ec06e2b0 0x7f34ec15a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:13.039 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.030+0000 7f34f32cc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34ec156ea0 0x7f34ec158680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:13.039 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.030+0000 7f34f32cc700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34ec157270 0x7f34ec158bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:13.039 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.030+0000 7f34f32cc700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ec15cc20 con 0x7f34ec156ea0 2023-10-08T00:11:13.039 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.030+0000 7f34f32cc700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ec0b0d90 con 0x7f34ec157270 2023-10-08T00:11:13.039 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.030+0000 7f34f32cc700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34ec15c660 con 0x7f34ec06e2b0 2023-10-08T00:11:13.039 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.030+0000 7f34f0867700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34ec156ea0 0x7f34ec158680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:13.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.030+0000 7f34f0867700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34ec156ea0 0x7f34ec158680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39168/0 (socket says 172.21.15.99:39168) 2023-10-08T00:11:13.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.030+0000 7f34f0867700 1 -- 172.21.15.99:0/3622436819 learned_addr learned my addr 172.21.15.99:0/3622436819 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:13.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.030+0000 7f34f1068700 1 --2- 172.21.15.99:0/3622436819 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34ec06e2b0 0x7f34ec15a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:13.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.030+0000 7f34f1869700 1 --2- 172.21.15.99:0/3622436819 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34ec157270 0x7f34ec158bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:13.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.030+0000 7f34f0867700 1 -- 172.21.15.99:0/3622436819 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34ec06e2b0 msgr2=0x7f34ec15a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:13.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.030+0000 7f34f0867700 1 --2- 172.21.15.99:0/3622436819 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34ec06e2b0 0x7f34ec15a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:13.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.030+0000 7f34f0867700 1 -- 172.21.15.99:0/3622436819 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34ec157270 msgr2=0x7f34ec158bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:13.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.030+0000 7f34f0867700 1 --2- 172.21.15.99:0/3622436819 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34ec157270 0x7f34ec158bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:13.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.030+0000 7f34f0867700 1 -- 172.21.15.99:0/3622436819 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e4009420 con 0x7f34ec156ea0 2023-10-08T00:11:13.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.030+0000 7f34f0867700 1 --2- 172.21.15.99:0/3622436819 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34ec156ea0 0x7f34ec158680 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f34dc00c610 tx=0x7f34dc000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:13.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.030+0000 7f34e17fa700 1 -- 172.21.15.99:0/3622436819 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34dc005b30 con 0x7f34ec156ea0 2023-10-08T00:11:13.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.030+0000 7f34e17fa700 1 -- 172.21.15.99:0/3622436819 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34dc005cb0 con 0x7f34ec156ea0 2023-10-08T00:11:13.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.030+0000 7f34e17fa700 1 -- 172.21.15.99:0/3622436819 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34dc007320 con 0x7f34ec156ea0 2023-10-08T00:11:13.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.030+0000 7f34f32cc700 1 -- 172.21.15.99:0/3622436819 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34ec15c940 con 0x7f34ec156ea0 2023-10-08T00:11:13.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.030+0000 7f34f32cc700 1 -- 172.21.15.99:0/3622436819 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34ec15ad60 con 0x7f34ec156ea0 2023-10-08T00:11:13.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.031+0000 7f34f32cc700 1 -- 172.21.15.99:0/3622436819 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34ec0b0880 con 0x7f34ec156ea0 2023-10-08T00:11:13.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.032+0000 7f34e17fa700 1 -- 172.21.15.99:0/3622436819 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f34dc01b3b0 con 0x7f34ec156ea0 2023-10-08T00:11:13.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.032+0000 7f34e17fa700 1 --2- 172.21.15.99:0/3622436819 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f34d0066f30 0x7f34d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:13.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.032+0000 7f34e17fa700 1 -- 172.21.15.99:0/3622436819 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f34dc083890 con 0x7f34ec156ea0 2023-10-08T00:11:13.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.032+0000 7f34f1068700 1 --2- 172.21.15.99:0/3622436819 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f34d0066f30 0x7f34d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:13.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.033+0000 7f34f1068700 1 --2- 172.21.15.99:0/3622436819 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f34d0066f30 0x7f34d00693e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f34ec068d40 tx=0x7f34e8007850 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:13.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.036+0000 7f34e17fa700 1 -- 172.21.15.99:0/3622436819 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f34ec0b0880 con 0x7f34ec156ea0 2023-10-08T00:11:13.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.265+0000 7f34f32cc700 1 -- 172.21.15.99:0/3622436819 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f34ec0b0880 con 0x7f34ec156ea0 2023-10-08T00:11:13.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.265+0000 7f34e17fa700 1 -- 172.21.15.99:0/3622436819 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f34ec0b0880 con 0x7f34ec156ea0 2023-10-08T00:11:13.266 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T00:11:13.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.267+0000 7f34f32cc700 1 -- 172.21.15.99:0/3622436819 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f34d0066f30 msgr2=0x7f34d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:13.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.267+0000 7f34f32cc700 1 --2- 172.21.15.99:0/3622436819 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f34d0066f30 0x7f34d00693e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f34ec068d40 tx=0x7f34e8007850 comp rx=0 tx=0).stop 2023-10-08T00:11:13.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.267+0000 7f34f32cc700 1 -- 172.21.15.99:0/3622436819 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34ec156ea0 msgr2=0x7f34ec158680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:13.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.267+0000 7f34f32cc700 1 --2- 172.21.15.99:0/3622436819 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34ec156ea0 0x7f34ec158680 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f34dc00c610 tx=0x7f34dc000fe0 comp rx=0 tx=0).stop 2023-10-08T00:11:13.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.267+0000 7f34f32cc700 1 -- 172.21.15.99:0/3622436819 shutdown_connections 2023-10-08T00:11:13.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.267+0000 7f34f32cc700 1 --2- 172.21.15.99:0/3622436819 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f34d0066f30 0x7f34d00693e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:13.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.267+0000 7f34f32cc700 1 --2- 172.21.15.99:0/3622436819 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34ec157270 0x7f34ec158bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:13.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.267+0000 7f34f32cc700 1 --2- 172.21.15.99:0/3622436819 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34ec06e2b0 0x7f34ec15a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:13.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.267+0000 7f34f32cc700 1 --2- 172.21.15.99:0/3622436819 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34ec156ea0 0x7f34ec158680 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:13.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.267+0000 7f34f32cc700 1 -- 172.21.15.99:0/3622436819 >> 172.21.15.99:0/3622436819 conn(0x7f34ec0ca4f0 msgr2=0x7f34ec1532e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:13.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.267+0000 7f34f32cc700 1 -- 172.21.15.99:0/3622436819 shutdown_connections 2023-10-08T00:11:13.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:13.268+0000 7f34f32cc700 1 -- 172.21.15.99:0/3622436819 wait complete. 2023-10-08T00:11:13.280 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-08T00:11:13.280 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T00:11:13.280 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi099.front.sepia.ceph.com 2023-10-08T00:11:13.280 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T00:11:13.280 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T00:11:13.280 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-08T00:11:13.280 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-08T00:11:13.280 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T00:11:13.280 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T00:11:13.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:13.281 DEBUG:teuthology.orchestra.run.smithi099:> ip addr 2023-10-08T00:11:13.296 INFO:teuthology.orchestra.run.smithi099.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T00:11:13.296 INFO:teuthology.orchestra.run.smithi099.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T00:11:13.296 INFO:teuthology.orchestra.run.smithi099.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T00:11:13.296 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:11:13.296 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 ::1/128 scope host 2023-10-08T00:11:13.296 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:11:13.296 INFO:teuthology.orchestra.run.smithi099.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:11:13.296 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:10 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:11:13.296 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f0 2023-10-08T00:11:13.297 INFO:teuthology.orchestra.run.smithi099.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:11:13.297 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:11 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:11:13.297 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f1 2023-10-08T00:11:13.297 INFO:teuthology.orchestra.run.smithi099.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:11:13.297 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:02 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:11:13.297 INFO:teuthology.orchestra.run.smithi099.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T00:11:13.297 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:03 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:11:13.297 INFO:teuthology.orchestra.run.smithi099.stdout: inet 172.21.15.99/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T00:11:13.297 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft 42517sec preferred_lft 42517sec 2023-10-08T00:11:13.297 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::ec4:7aff:fe8f:d803/64 scope link 2023-10-08T00:11:13.297 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:11:13.298 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-08T00:11:13.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:13.298 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2023-10-08T00:11:13.298 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2023-10-08T00:11:13.298 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link add name ceph-brx type bridge 2023-10-08T00:11:13.298 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip addr flush dev ceph-brx 2023-10-08T00:11:13.298 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set ceph-brx up 2023-10-08T00:11:13.298 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-08T00:11:13.298 DEBUG:teuthology.orchestra.run.smithi099:> ') 2023-10-08T00:11:13.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:13.508 DEBUG:teuthology.orchestra.run.smithi099:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T00:11:13.549 INFO:teuthology.orchestra.run.smithi099.stdout:1 2023-10-08T00:11:13.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:13.551 DEBUG:teuthology.orchestra.run.smithi099:> route 2023-10-08T00:11:13.624 INFO:teuthology.orchestra.run.smithi099.stdout:Kernel IP routing table 2023-10-08T00:11:13.624 INFO:teuthology.orchestra.run.smithi099.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T00:11:13.625 INFO:teuthology.orchestra.run.smithi099.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T00:11:13.625 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T00:11:13.625 INFO:teuthology.orchestra.run.smithi099.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T00:11:13.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:13.625 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2023-10-08T00:11:13.625 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2023-10-08T00:11:13.625 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T00:11:13.625 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T00:11:13.626 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T00:11:13.626 DEBUG:teuthology.orchestra.run.smithi099:> ') 2023-10-08T00:11:13.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:13.900 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2023-10-08T00:11:13.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:13.916 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list-id 2023-10-08T00:11:13.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:13.970 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2023-10-08T00:11:13.970 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2023-10-08T00:11:13.970 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T00:11:13.970 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-10-08T00:11:13.971 DEBUG:teuthology.orchestra.run.smithi099:> ') 2023-10-08T00:11:14.059 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-10-08T00:11:14.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:14.059 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2023-10-08T00:11:14.059 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2023-10-08T00:11:14.060 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-10-08T00:11:14.060 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-08T00:11:14.060 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-10-08T00:11:14.060 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-10-08T00:11:14.060 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-10-08T00:11:14.060 DEBUG:teuthology.orchestra.run.smithi099:> ') 2023-10-08T00:11:14.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:14.241 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2023-10-08T00:11:14.242 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2023-10-08T00:11:14.242 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set brx.0 up 2023-10-08T00:11:14.242 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set dev brx.0 master ceph-brx 2023-10-08T00:11:14.242 DEBUG:teuthology.orchestra.run.smithi099:> ') 2023-10-08T00:11:14.334 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:11:14.335 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T00:11:14.349 INFO:teuthology.orchestra.run.smithi099.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T00:11:14.349 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:11:14.349 DEBUG:teuthology.orchestra.run.smithi099:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T00:11:14.404 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-08T00:11:14.404 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T00:11:14.404 DEBUG:teuthology.orchestra.run.smithi099:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:11:14.489 INFO:teuthology.orchestra.run.smithi099.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:11:14.489 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:11:14.572 INFO:teuthology.orchestra.run.smithi099.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:11:14.572 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:11:14.572 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: resolved to: "172.21.15.99:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T00:11:14.572 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: trying mount with new device syntax: 0@06962cdb-9a6f-417f-82fd-9e4e3dedbccf.cephfs=/ 2023-10-08T00:11:14.572 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.99:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T00:11:14.572 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T00:11:14.573 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T00:11:14.573 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T00:11:14.594 INFO:teuthology.orchestra.run.smithi099.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:11:14.594 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:11:14.594 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: resolved to: "172.21.15.99:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T00:11:14.594 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: trying mount with new device syntax: 0@06962cdb-9a6f-417f-82fd-9e4e3dedbccf.cephfs=/ 2023-10-08T00:11:14.594 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.99:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T00:11:14.594 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T00:11:14.595 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T00:11:14.595 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T00:11:14.595 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: resolved to: "172.21.15.99:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T00:11:14.595 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.99:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T00:11:14.595 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=06962cdb-9a6f-417f-82fd-9e4e3dedbccf" will pass to kernel 2023-10-08T00:11:14.596 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T00:11:14.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:14.597 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T00:11:14.621 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2023-10-08T00:11:14.621 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T00:11:14.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.774+0000 7fc777fff700 1 Processor -- start 2023-10-08T00:11:14.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.774+0000 7fc777fff700 1 -- start start 2023-10-08T00:11:14.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.775+0000 7fc777fff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc7780cdab0 0x7fc77815d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:14.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.775+0000 7fc777fff700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7780cb7c0 con 0x7fc77815dda0 2023-10-08T00:11:14.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.775+0000 7fc777fff700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc77815e290 con 0x7fc7780cd6e0 2023-10-08T00:11:14.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.775+0000 7fc777fff700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc77815e850 con 0x7fc7780cdab0 2023-10-08T00:11:14.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.775+0000 7fc7777fe700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fc77815dda0 0x7fc77815cbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:55974/0 (socket says 172.21.15.99:55974) 2023-10-08T00:11:14.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.775+0000 7fc7777fe700 1 -- 172.21.15.99:0/4279739810 learned_addr learned my addr 172.21.15.99:0/4279739810 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:14.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.775+0000 7fc775ffb700 1 -- 172.21.15.99:0/4279739810 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 772475923 0 0) 0x7fc7780cb7c0 con 0x7fc77815dda0 2023-10-08T00:11:14.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.775+0000 7fc7767fc700 1 --2- 172.21.15.99:0/4279739810 >> v2:172.21.15.191:3300/0 conn(0x7fc7780cdab0 0x7fc77815d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:14.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.775+0000 7fc775ffb700 1 -- 172.21.15.99:0/4279739810 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc764003570 con 0x7fc77815dda0 2023-10-08T00:11:14.781 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.776+0000 7fc775ffb700 1 -- 172.21.15.99:0/4279739810 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 344139912 0 0) 0x7fc764003570 con 0x7fc77815dda0 2023-10-08T00:11:14.781 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.776+0000 7fc775ffb700 1 -- 172.21.15.99:0/4279739810 >> v2:172.21.15.191:3300/0 conn(0x7fc7780cdab0 msgr2=0x7fc77815d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:14.781 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.776+0000 7fc775ffb700 1 --2- 172.21.15.99:0/4279739810 >> v2:172.21.15.191:3300/0 conn(0x7fc7780cdab0 0x7fc77815d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:14.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.776+0000 7fc775ffb700 1 -- 172.21.15.99:0/4279739810 >> v1:172.21.15.191:6789/0 conn(0x7fc7780cd6e0 legacy=0x7fc7780cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:14.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.776+0000 7fc775ffb700 1 -- 172.21.15.99:0/4279739810 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc77815e570 con 0x7fc77815dda0 2023-10-08T00:11:14.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.776+0000 7fc775ffb700 1 -- 172.21.15.99:0/4279739810 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7fc768001470 con 0x7fc77815dda0 2023-10-08T00:11:14.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.776+0000 7fc775ffb700 1 -- 172.21.15.99:0/4279739810 >> v1:172.21.15.99:6789/0 conn(0x7fc77815dda0 legacy=0x7fc77815cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:14.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.776+0000 7fc775ffb700 1 --2- 172.21.15.99:0/4279739810 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc764003a60 0x7fc764003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:14.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.776+0000 7fc775ffb700 1 --2- 172.21.15.99:0/4279739810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc764004810 0x7fc764004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:14.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.776+0000 7fc775ffb700 1 --2- 172.21.15.99:0/4279739810 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7640051c0 0x7fc76400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:14.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.776+0000 7fc775ffb700 1 -- 172.21.15.99:0/4279739810 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77815e850 con 0x7fc764004810 2023-10-08T00:11:14.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.776+0000 7fc775ffb700 1 -- 172.21.15.99:0/4279739810 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc76400c0d0 con 0x7fc764003a60 2023-10-08T00:11:14.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.776+0000 7fc775ffb700 1 -- 172.21.15.99:0/4279739810 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc76400c290 con 0x7fc7640051c0 2023-10-08T00:11:14.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.776+0000 7fc7767fc700 1 --2- 172.21.15.99:0/4279739810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc764004810 0x7fc764004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:14.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.776+0000 7fc776ffd700 1 --2- 172.21.15.99:0/4279739810 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc764003a60 0x7fc764003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:14.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.776+0000 7fc7777fe700 1 --2- 172.21.15.99:0/4279739810 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7640051c0 0x7fc76400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:14.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.776+0000 7fc7767fc700 1 -- 172.21.15.99:0/4279739810 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7640051c0 msgr2=0x7fc76400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:14.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.776+0000 7fc7767fc700 1 --2- 172.21.15.99:0/4279739810 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7640051c0 0x7fc76400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:14.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.776+0000 7fc7767fc700 1 -- 172.21.15.99:0/4279739810 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc764003a60 msgr2=0x7fc764003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:14.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.776+0000 7fc7767fc700 1 --2- 172.21.15.99:0/4279739810 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc764003a60 0x7fc764003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:14.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.776+0000 7fc7767fc700 1 -- 172.21.15.99:0/4279739810 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc764003570 con 0x7fc764004810 2023-10-08T00:11:14.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.776+0000 7fc7777fe700 1 --2- 172.21.15.99:0/4279739810 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7640051c0 0x7fc76400ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:11:14.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.776+0000 7fc7777fe700 1 -- 172.21.15.99:0/4279739810 reap_dead start 2023-10-08T00:11:14.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.776+0000 7fc7767fc700 1 --2- 172.21.15.99:0/4279739810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc764004810 0x7fc764004c80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc760009c00 tx=0x7fc7600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4fe2341bd836bdd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:14.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.776+0000 7fc775ffb700 1 -- 172.21.15.99:0/4279739810 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76000e710 con 0x7fc764004810 2023-10-08T00:11:14.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.776+0000 7fc775ffb700 1 -- 172.21.15.99:0/4279739810 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc76000e870 con 0x7fc764004810 2023-10-08T00:11:14.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.776+0000 7fc775ffb700 1 -- 172.21.15.99:0/4279739810 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76000ec30 con 0x7fc764004810 2023-10-08T00:11:14.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.777+0000 7fc777fff700 1 -- 172.21.15.99:0/4279739810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc764004810 msgr2=0x7fc764004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:14.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.777+0000 7fc777fff700 1 --2- 172.21.15.99:0/4279739810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc764004810 0x7fc764004c80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc760009c00 tx=0x7fc7600091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:14.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.777+0000 7fc777fff700 1 -- 172.21.15.99:0/4279739810 shutdown_connections 2023-10-08T00:11:14.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.777+0000 7fc777fff700 1 --2- 172.21.15.99:0/4279739810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc764004810 0x7fc764004c80 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:14.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.777+0000 7fc777fff700 1 -- 172.21.15.99:0/4279739810 >> 172.21.15.99:0/4279739810 conn(0x7fc778067660 msgr2=0x7fc778067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:14.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.777+0000 7fc777fff700 1 -- 172.21.15.99:0/4279739810 shutdown_connections 2023-10-08T00:11:14.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.777+0000 7fc777fff700 1 -- 172.21.15.99:0/4279739810 wait complete. 2023-10-08T00:11:14.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.778+0000 7fc777fff700 1 Processor -- start 2023-10-08T00:11:14.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.778+0000 7fc777fff700 1 -- start start 2023-10-08T00:11:14.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.779+0000 7fc777fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc764004810 0x7fc77816b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:14.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.779+0000 7fc777fff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc77814ce30 0x7fc7780c96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:14.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.779+0000 7fc777fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7780c9c10 0x7fc77816a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:14.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.779+0000 7fc777fff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7780cc360 con 0x7fc7780c9c10 2023-10-08T00:11:14.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.779+0000 7fc777fff700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77816b350 con 0x7fc77814ce30 2023-10-08T00:11:14.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.779+0000 7fc777fff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7780ca090 con 0x7fc764004810 2023-10-08T00:11:14.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.779+0000 7fc7777fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7780c9c10 0x7fc77816a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:14.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.779+0000 7fc7777fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7780c9c10 0x7fc77816a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55792/0 (socket says 172.21.15.99:55792) 2023-10-08T00:11:14.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.779+0000 7fc7777fe700 1 -- 172.21.15.99:0/2101734572 learned_addr learned my addr 172.21.15.99:0/2101734572 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:14.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.779+0000 7fc7767fc700 1 --2- 172.21.15.99:0/2101734572 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc77814ce30 0x7fc7780c96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:14.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.779+0000 7fc776ffd700 1 --2- 172.21.15.99:0/2101734572 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc764004810 0x7fc77816b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:14.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.779+0000 7fc7777fe700 1 -- 172.21.15.99:0/2101734572 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc764004810 msgr2=0x7fc77816b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:14.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.779+0000 7fc7777fe700 1 --2- 172.21.15.99:0/2101734572 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc764004810 0x7fc77816b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:14.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.779+0000 7fc7777fe700 1 -- 172.21.15.99:0/2101734572 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc77814ce30 msgr2=0x7fc7780c96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:14.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.779+0000 7fc7777fe700 1 --2- 172.21.15.99:0/2101734572 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc77814ce30 0x7fc7780c96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:14.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.779+0000 7fc7777fe700 1 -- 172.21.15.99:0/2101734572 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc77816b070 con 0x7fc7780c9c10 2023-10-08T00:11:14.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.779+0000 7fc7767fc700 1 --2- 172.21.15.99:0/2101734572 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc77814ce30 0x7fc7780c96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:11:14.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.780+0000 7fc7777fe700 1 --2- 172.21.15.99:0/2101734572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7780c9c10 0x7fc77816a300 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc778089ac0 tx=0x7fc768001470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:14.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.780+0000 7fc76effd700 1 -- 172.21.15.99:0/2101734572 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76800ee50 con 0x7fc7780c9c10 2023-10-08T00:11:14.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.780+0000 7fc76effd700 1 -- 172.21.15.99:0/2101734572 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc76800c750 con 0x7fc7780c9c10 2023-10-08T00:11:14.788 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.780+0000 7fc777fff700 1 -- 172.21.15.99:0/2101734572 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7780b3b40 con 0x7fc7780c9c10 2023-10-08T00:11:14.788 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.780+0000 7fc76effd700 1 -- 172.21.15.99:0/2101734572 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76800cb10 con 0x7fc7780c9c10 2023-10-08T00:11:14.788 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.780+0000 7fc777fff700 1 -- 172.21.15.99:0/2101734572 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7780b4100 con 0x7fc7780c9c10 2023-10-08T00:11:14.788 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.781+0000 7fc777fff700 1 -- 172.21.15.99:0/2101734572 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7780b4320 con 0x7fc7780c9c10 2023-10-08T00:11:14.788 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.781+0000 7fc76effd700 1 -- 172.21.15.99:0/2101734572 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc7680133e0 con 0x7fc7780c9c10 2023-10-08T00:11:14.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.781+0000 7fc76effd700 1 --2- 172.21.15.99:0/2101734572 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc758066fb0 0x7fc758069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:14.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.781+0000 7fc76effd700 1 -- 172.21.15.99:0/2101734572 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc768083ea0 con 0x7fc7780c9c10 2023-10-08T00:11:14.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.782+0000 7fc776ffd700 1 --2- 172.21.15.99:0/2101734572 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc758066fb0 0x7fc758069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:14.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.782+0000 7fc776ffd700 1 --2- 172.21.15.99:0/2101734572 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc758066fb0 0x7fc758069460 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc7780cbfa0 tx=0x7fc770001640 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:14.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:14.785+0000 7fc76effd700 1 -- 172.21.15.99:0/2101734572 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc7680510f0 con 0x7fc7780c9c10 2023-10-08T00:11:15.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:15.012+0000 7fc777fff700 1 -- 172.21.15.99:0/2101734572 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc7780ccf70 con 0x7fc7780c9c10 2023-10-08T00:11:15.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:15.012+0000 7fc76effd700 1 -- 172.21.15.99:0/2101734572 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc768019060 con 0x7fc7780c9c10 2023-10-08T00:11:15.013 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T00:11:15.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:15.014+0000 7fc777fff700 1 -- 172.21.15.99:0/2101734572 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc758066fb0 msgr2=0x7fc758069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:15.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:15.014+0000 7fc777fff700 1 --2- 172.21.15.99:0/2101734572 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc758066fb0 0x7fc758069460 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc7780cbfa0 tx=0x7fc770001640 comp rx=0 tx=0).stop 2023-10-08T00:11:15.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:15.014+0000 7fc777fff700 1 -- 172.21.15.99:0/2101734572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7780c9c10 msgr2=0x7fc77816a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:15.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:15.014+0000 7fc777fff700 1 --2- 172.21.15.99:0/2101734572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7780c9c10 0x7fc77816a300 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc778089ac0 tx=0x7fc768001470 comp rx=0 tx=0).stop 2023-10-08T00:11:15.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:15.014+0000 7fc777fff700 1 -- 172.21.15.99:0/2101734572 shutdown_connections 2023-10-08T00:11:15.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:15.014+0000 7fc777fff700 1 --2- 172.21.15.99:0/2101734572 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc758066fb0 0x7fc758069460 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:15.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:15.014+0000 7fc777fff700 1 --2- 172.21.15.99:0/2101734572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7780c9c10 0x7fc77816a300 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:15.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:15.014+0000 7fc777fff700 1 --2- 172.21.15.99:0/2101734572 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc764004810 0x7fc77816b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:15.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:15.014+0000 7fc777fff700 1 --2- 172.21.15.99:0/2101734572 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc77814ce30 0x7fc7780c96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:15.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:15.014+0000 7fc777fff700 1 -- 172.21.15.99:0/2101734572 >> 172.21.15.99:0/2101734572 conn(0x7fc778067660 msgr2=0x7fc7780b07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:15.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:15.014+0000 7fc777fff700 1 -- 172.21.15.99:0/2101734572 shutdown_connections 2023-10-08T00:11:15.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:15.014+0000 7fc777fff700 1 -- 172.21.15.99:0/2101734572 wait complete. 2023-10-08T00:11:15.026 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-08T00:11:15.027 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T00:11:15.027 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi099.front.sepia.ceph.com 2023-10-08T00:11:15.027 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-10-08T00:11:15.027 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-08T00:11:15.027 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-08T00:11:15.027 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-08T00:11:15.027 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T00:11:15.027 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-10-08T00:11:15.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:15.027 DEBUG:teuthology.orchestra.run.smithi099:> ip addr 2023-10-08T00:11:15.043 INFO:teuthology.orchestra.run.smithi099.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T00:11:15.043 INFO:teuthology.orchestra.run.smithi099.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T00:11:15.043 INFO:teuthology.orchestra.run.smithi099.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T00:11:15.043 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:11:15.043 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 ::1/128 scope host 2023-10-08T00:11:15.043 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:11:15.043 INFO:teuthology.orchestra.run.smithi099.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:11:15.043 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:10 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:11:15.044 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f0 2023-10-08T00:11:15.044 INFO:teuthology.orchestra.run.smithi099.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:11:15.044 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:11 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:11:15.044 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f1 2023-10-08T00:11:15.044 INFO:teuthology.orchestra.run.smithi099.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:11:15.044 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:02 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:11:15.044 INFO:teuthology.orchestra.run.smithi099.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T00:11:15.044 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:03 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:11:15.044 INFO:teuthology.orchestra.run.smithi099.stdout: inet 172.21.15.99/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T00:11:15.044 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft 42515sec preferred_lft 42515sec 2023-10-08T00:11:15.044 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::ec4:7aff:fe8f:d803/64 scope link 2023-10-08T00:11:15.044 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:11:15.044 INFO:teuthology.orchestra.run.smithi099.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T00:11:15.045 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether ae:2f:b5:94:69:ba brd ff:ff:ff:ff:ff:ff 2023-10-08T00:11:15.045 INFO:teuthology.orchestra.run.smithi099.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T00:11:15.045 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:11:15.045 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::600d:26ff:fede:5eec/64 scope link 2023-10-08T00:11:15.045 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:11:15.045 INFO:teuthology.orchestra.run.smithi099.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T00:11:15.045 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether ae:2f:b5:94:69:ba brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T00:11:15.045 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::ac2f:b5ff:fe94:69ba/64 scope link tentative 2023-10-08T00:11:15.045 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:11:15.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:15.046 DEBUG:teuthology.orchestra.run.smithi099:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T00:11:15.106 INFO:teuthology.orchestra.run.smithi099.stdout:1 2023-10-08T00:11:15.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:15.108 DEBUG:teuthology.orchestra.run.smithi099:> route 2023-10-08T00:11:15.164 INFO:teuthology.orchestra.run.smithi099.stdout:Kernel IP routing table 2023-10-08T00:11:15.164 INFO:teuthology.orchestra.run.smithi099.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T00:11:15.165 INFO:teuthology.orchestra.run.smithi099.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T00:11:15.165 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T00:11:15.165 INFO:teuthology.orchestra.run.smithi099.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T00:11:15.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:15.165 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2023-10-08T00:11:15.165 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2023-10-08T00:11:15.165 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T00:11:15.165 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T00:11:15.165 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T00:11:15.166 DEBUG:teuthology.orchestra.run.smithi099:> ') 2023-10-08T00:11:15.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:15.264 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2023-10-08T00:11:15.279 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T00:11:15.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:15.279 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list-id 2023-10-08T00:11:15.332 INFO:teuthology.orchestra.run.smithi099.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-10-08T00:11:15.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:15.333 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2023-10-08T00:11:15.333 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2023-10-08T00:11:15.333 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T00:11:15.333 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-10-08T00:11:15.333 DEBUG:teuthology.orchestra.run.smithi099:> ') 2023-10-08T00:11:15.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:15.421 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-08T00:11:15.454 INFO:teuthology.orchestra.run.smithi099.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T00:11:15.454 INFO:teuthology.orchestra.run.smithi099.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T00:11:15.454 INFO:teuthology.orchestra.run.smithi099.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T00:11:15.455 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:11:15.455 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 ::1/128 scope host 2023-10-08T00:11:15.455 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:11:15.455 INFO:teuthology.orchestra.run.smithi099.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T00:11:15.455 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 1a:ef:c7:8b:f4:0c brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-08T00:11:15.455 INFO:teuthology.orchestra.run.smithi099.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-08T00:11:15.455 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:11:15.455 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::18ef:c7ff:fe8b:f40c/64 scope link tentative 2023-10-08T00:11:15.455 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:11:15.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:15.474 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-08T00:11:15.552 INFO:teuthology.orchestra.run.smithi099.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T00:11:15.552 INFO:teuthology.orchestra.run.smithi099.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T00:11:15.552 INFO:teuthology.orchestra.run.smithi099.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T00:11:15.553 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:11:15.553 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 ::1/128 scope host 2023-10-08T00:11:15.553 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:11:15.553 INFO:teuthology.orchestra.run.smithi099.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T00:11:15.553 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 1a:ef:c7:8b:f4:0c brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-08T00:11:15.553 INFO:teuthology.orchestra.run.smithi099.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-08T00:11:15.553 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:11:15.553 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::18ef:c7ff:fe8b:f40c/64 scope link tentative 2023-10-08T00:11:15.553 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:11:15.564 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-10-08T00:11:15.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:15.564 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2023-10-08T00:11:15.564 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2023-10-08T00:11:15.564 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-10-08T00:11:15.564 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-08T00:11:15.564 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-10-08T00:11:15.564 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-10-08T00:11:15.565 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-10-08T00:11:15.565 DEBUG:teuthology.orchestra.run.smithi099:> ') 2023-10-08T00:11:15.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:15.780 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2023-10-08T00:11:15.780 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2023-10-08T00:11:15.780 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set brx.1 up 2023-10-08T00:11:15.780 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set dev brx.1 master ceph-brx 2023-10-08T00:11:15.780 DEBUG:teuthology.orchestra.run.smithi099:> ') 2023-10-08T00:11:15.860 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:11:15.860 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-10-08T00:11:15.874 INFO:teuthology.orchestra.run.smithi099.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-10-08T00:11:15.875 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:11:15.875 DEBUG:teuthology.orchestra.run.smithi099:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-10-08T00:11:15.929 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2023-10-08T00:11:15.930 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T00:11:15.930 DEBUG:teuthology.orchestra.run.smithi099:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:11:16.009 INFO:teuthology.orchestra.run.smithi099.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:11:16.009 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:11:16.027 INFO:teuthology.orchestra.run.smithi099.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:11:16.027 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:11:16.028 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: resolved to: "172.21.15.99:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T00:11:16.028 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: trying mount with new device syntax: 2@06962cdb-9a6f-417f-82fd-9e4e3dedbccf.cephfs=/ 2023-10-08T00:11:16.028 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.99:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T00:11:16.028 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T00:11:16.028 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T00:11:16.028 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T00:11:16.051 INFO:teuthology.orchestra.run.smithi099.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:11:16.051 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:11:16.051 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: resolved to: "172.21.15.99:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T00:11:16.051 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: trying mount with new device syntax: 2@06962cdb-9a6f-417f-82fd-9e4e3dedbccf.cephfs=/ 2023-10-08T00:11:16.052 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.99:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T00:11:16.052 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T00:11:16.052 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T00:11:16.052 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T00:11:16.052 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: resolved to: "172.21.15.99:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T00:11:16.052 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.99:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T00:11:16.052 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,key=2,mds_namespace=cephfs,fsid=06962cdb-9a6f-417f-82fd-9e4e3dedbccf" will pass to kernel 2023-10-08T00:11:16.053 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T00:11:16.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:16.054 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-10-08T00:11:16.118 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2023-10-08T00:11:16.119 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T00:11:16.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.272+0000 7f65a32da700 1 Processor -- start 2023-10-08T00:11:16.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.272+0000 7f65a32da700 1 -- start start 2023-10-08T00:11:16.275 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.273+0000 7f65a32da700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f659c157870 0x7f659c15b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:16.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.273+0000 7f65a32da700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f659c15a620 0x7f659c15d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:16.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.273+0000 7f65a32da700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f659c15d580 0x7f659c158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:16.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.273+0000 7f65a32da700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f659c15cbb0 con 0x7f659c157870 2023-10-08T00:11:16.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.273+0000 7f65a32da700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f659c15bfa0 con 0x7f659c15d580 2023-10-08T00:11:16.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.273+0000 7f65a32da700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f659c15c280 con 0x7f659c15a620 2023-10-08T00:11:16.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.273+0000 7f65a1076700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f659c157870 0x7f659c15b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:16.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.273+0000 7f65a1076700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f659c157870 0x7f659c15b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55808/0 (socket says 172.21.15.99:55808) 2023-10-08T00:11:16.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.273+0000 7f65a1076700 1 -- 172.21.15.99:0/2336892001 learned_addr learned my addr 172.21.15.99:0/2336892001 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:16.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.273+0000 7f65a0875700 1 --2- 172.21.15.99:0/2336892001 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f659c15a620 0x7f659c15d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:16.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.273+0000 7f65a1877700 1 --2- 172.21.15.99:0/2336892001 >> v2:172.21.15.191:3300/0 conn(0x7f659c15d580 0x7f659c158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:16.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.274+0000 7f65a1076700 1 -- 172.21.15.99:0/2336892001 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f659c15a620 msgr2=0x7f659c15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:16.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.274+0000 7f65a1076700 1 --2- 172.21.15.99:0/2336892001 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f659c15a620 0x7f659c15d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:16.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.274+0000 7f65a1076700 1 -- 172.21.15.99:0/2336892001 >> v2:172.21.15.191:3300/0 conn(0x7f659c15d580 msgr2=0x7f659c158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:16.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.274+0000 7f65a1076700 1 --2- 172.21.15.99:0/2336892001 >> v2:172.21.15.191:3300/0 conn(0x7f659c15d580 0x7f659c158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:16.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.274+0000 7f65a1076700 1 -- 172.21.15.99:0/2336892001 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f659c15c560 con 0x7f659c157870 2023-10-08T00:11:16.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.274+0000 7f65a0875700 1 --2- 172.21.15.99:0/2336892001 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f659c15a620 0x7f659c15d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:11:16.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.274+0000 7f65a1877700 1 --2- 172.21.15.99:0/2336892001 >> v2:172.21.15.191:3300/0 conn(0x7f659c15d580 0x7f659c158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:11:16.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.274+0000 7f65a1076700 1 --2- 172.21.15.99:0/2336892001 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f659c157870 0x7f659c15b6a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f659c0b03b0 tx=0x7f6598009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4fceb1a5d7448f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:16.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.274+0000 7f6593fff700 1 -- 172.21.15.99:0/2336892001 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659800e070 con 0x7f659c157870 2023-10-08T00:11:16.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.274+0000 7f6593fff700 1 -- 172.21.15.99:0/2336892001 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659800ba50 con 0x7f659c157870 2023-10-08T00:11:16.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.274+0000 7f6593fff700 1 -- 172.21.15.99:0/2336892001 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65980092e0 con 0x7f659c157870 2023-10-08T00:11:16.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.274+0000 7f65a32da700 1 -- 172.21.15.99:0/2336892001 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f659c157870 msgr2=0x7f659c15b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:16.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.274+0000 7f65a32da700 1 --2- 172.21.15.99:0/2336892001 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f659c157870 0x7f659c15b6a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f659c0b03b0 tx=0x7f6598009840 comp rx=0 tx=0).stop 2023-10-08T00:11:16.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.274+0000 7f65a32da700 1 -- 172.21.15.99:0/2336892001 shutdown_connections 2023-10-08T00:11:16.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.274+0000 7f65a32da700 1 --2- 172.21.15.99:0/2336892001 >> v2:172.21.15.191:3300/0 conn(0x7f659c15d580 0x7f659c158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:16.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.274+0000 7f65a32da700 1 --2- 172.21.15.99:0/2336892001 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f659c157870 0x7f659c15b6a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:16.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.274+0000 7f65a32da700 1 --2- 172.21.15.99:0/2336892001 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f659c15a620 0x7f659c15d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:16.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.274+0000 7f65a32da700 1 -- 172.21.15.99:0/2336892001 >> 172.21.15.99:0/2336892001 conn(0x7f659c068ce0 msgr2=0x7f659c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:16.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.274+0000 7f65a32da700 1 -- 172.21.15.99:0/2336892001 shutdown_connections 2023-10-08T00:11:16.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.275+0000 7f65a32da700 1 -- 172.21.15.99:0/2336892001 wait complete. 2023-10-08T00:11:16.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.276+0000 7f65a32da700 1 Processor -- start 2023-10-08T00:11:16.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.276+0000 7f65a32da700 1 -- start start 2023-10-08T00:11:16.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.276+0000 7f65a32da700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f659c157870 0x7f659c0b49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:16.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.276+0000 7f65a32da700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f659c15a620 0x7f659c0b6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:16.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.277+0000 7f65a32da700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f659c15d580 0x7f659c0b2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:16.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.277+0000 7f65a32da700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659c159610 con 0x7f659c15d580 2023-10-08T00:11:16.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.277+0000 7f65a32da700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659c14fb10 con 0x7f659c157870 2023-10-08T00:11:16.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.277+0000 7f65a32da700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f659c0b58c0 con 0x7f659c15a620 2023-10-08T00:11:16.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.277+0000 7f65a1877700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f659c15d580 0x7f659c0b2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:16.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.277+0000 7f65a1877700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f659c15d580 0x7f659c0b2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55822/0 (socket says 172.21.15.99:55822) 2023-10-08T00:11:16.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.277+0000 7f65a1877700 1 -- 172.21.15.99:0/1706278182 learned_addr learned my addr 172.21.15.99:0/1706278182 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:16.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.277+0000 7f65a1076700 1 --2- 172.21.15.99:0/1706278182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f659c157870 0x7f659c0b49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:16.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.277+0000 7f65a0875700 1 --2- 172.21.15.99:0/1706278182 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f659c15a620 0x7f659c0b6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:16.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.277+0000 7f65a1877700 1 -- 172.21.15.99:0/1706278182 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f659c15a620 msgr2=0x7f659c0b6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:16.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.277+0000 7f65a1877700 1 --2- 172.21.15.99:0/1706278182 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f659c15a620 0x7f659c0b6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:16.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.277+0000 7f65a1877700 1 -- 172.21.15.99:0/1706278182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f659c157870 msgr2=0x7f659c0b49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:16.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.277+0000 7f65a1877700 1 --2- 172.21.15.99:0/1706278182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f659c157870 0x7f659c0b49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:16.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.277+0000 7f65a1877700 1 -- 172.21.15.99:0/1706278182 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6598009ae0 con 0x7f659c15d580 2023-10-08T00:11:16.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.277+0000 7f65a1076700 1 --2- 172.21.15.99:0/1706278182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f659c157870 0x7f659c0b49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:11:16.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.277+0000 7f65a0875700 1 --2- 172.21.15.99:0/1706278182 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f659c15a620 0x7f659c0b6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:11:16.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.277+0000 7f65a1877700 1 --2- 172.21.15.99:0/1706278182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f659c15d580 0x7f659c0b2ab0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f659400d890 tx=0x7f6594008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:16.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.277+0000 7f65917fa700 1 -- 172.21.15.99:0/1706278182 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65940125f0 con 0x7f659c15d580 2023-10-08T00:11:16.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.277+0000 7f65917fa700 1 -- 172.21.15.99:0/1706278182 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6594012750 con 0x7f659c15d580 2023-10-08T00:11:16.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.277+0000 7f65a32da700 1 -- 172.21.15.99:0/1706278182 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f659c0b6b50 con 0x7f659c15d580 2023-10-08T00:11:16.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.277+0000 7f65917fa700 1 -- 172.21.15.99:0/1706278182 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6594012af0 con 0x7f659c15d580 2023-10-08T00:11:16.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.277+0000 7f65a32da700 1 -- 172.21.15.99:0/1706278182 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f659c0b5300 con 0x7f659c15d580 2023-10-08T00:11:16.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.278+0000 7f65a32da700 1 -- 172.21.15.99:0/1706278182 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f659c14fc50 con 0x7f659c15d580 2023-10-08T00:11:16.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.279+0000 7f65917fa700 1 -- 172.21.15.99:0/1706278182 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f659401e460 con 0x7f659c15d580 2023-10-08T00:11:16.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.279+0000 7f65917fa700 1 --2- 172.21.15.99:0/1706278182 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6580066f30 0x7f65800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:16.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.279+0000 7f65917fa700 1 -- 172.21.15.99:0/1706278182 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6594084a30 con 0x7f659c15d580 2023-10-08T00:11:16.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.279+0000 7f65a1076700 1 --2- 172.21.15.99:0/1706278182 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6580066f30 0x7f65800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:16.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.280+0000 7f65a1076700 1 --2- 172.21.15.99:0/1706278182 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6580066f30 0x7f65800693e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6598009ab0 tx=0x7f6598018000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:16.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.283+0000 7f65917fa700 1 -- 172.21.15.99:0/1706278182 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f659c14fc50 con 0x7f659c15d580 2023-10-08T00:11:16.505 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.504+0000 7f65a32da700 1 -- 172.21.15.99:0/1706278182 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f659c14fc50 con 0x7f659c15d580 2023-10-08T00:11:16.505 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.505+0000 7f65917fa700 1 -- 172.21.15.99:0/1706278182 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f659c14fc50 con 0x7f659c15d580 2023-10-08T00:11:16.506 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T00:11:16.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.507+0000 7f65a32da700 1 -- 172.21.15.99:0/1706278182 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6580066f30 msgr2=0x7f65800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:16.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.507+0000 7f65a32da700 1 --2- 172.21.15.99:0/1706278182 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6580066f30 0x7f65800693e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6598009ab0 tx=0x7f6598018000 comp rx=0 tx=0).stop 2023-10-08T00:11:16.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.507+0000 7f65a32da700 1 -- 172.21.15.99:0/1706278182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f659c15d580 msgr2=0x7f659c0b2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:16.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.507+0000 7f65a32da700 1 --2- 172.21.15.99:0/1706278182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f659c15d580 0x7f659c0b2ab0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f659400d890 tx=0x7f6594008040 comp rx=0 tx=0).stop 2023-10-08T00:11:16.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.507+0000 7f65a32da700 1 -- 172.21.15.99:0/1706278182 shutdown_connections 2023-10-08T00:11:16.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.507+0000 7f65a32da700 1 --2- 172.21.15.99:0/1706278182 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6580066f30 0x7f65800693e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:16.511 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.507+0000 7f65a32da700 1 --2- 172.21.15.99:0/1706278182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f659c157870 0x7f659c0b49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:16.511 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.507+0000 7f65a32da700 1 --2- 172.21.15.99:0/1706278182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f659c15d580 0x7f659c0b2ab0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:16.511 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.507+0000 7f65a32da700 1 --2- 172.21.15.99:0/1706278182 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f659c15a620 0x7f659c0b6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:16.511 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.507+0000 7f65a32da700 1 -- 172.21.15.99:0/1706278182 >> 172.21.15.99:0/1706278182 conn(0x7f659c068ce0 msgr2=0x7f659c1581e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:16.511 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.507+0000 7f65a32da700 1 -- 172.21.15.99:0/1706278182 shutdown_connections 2023-10-08T00:11:16.511 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:16.507+0000 7f65a32da700 1 -- 172.21.15.99:0/1706278182 wait complete. 2023-10-08T00:11:16.519 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-08T00:11:16.519 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T00:11:16.519 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi191.front.sepia.ceph.com 2023-10-08T00:11:16.519 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-08T00:11:16.520 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-08T00:11:16.520 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-08T00:11:16.520 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-08T00:11:16.520 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T00:11:16.520 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T00:11:16.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:16.520 DEBUG:teuthology.orchestra.run.smithi191:> ip addr 2023-10-08T00:11:16.539 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T00:11:16.539 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T00:11:16.539 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T00:11:16.539 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:11:16.539 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2023-10-08T00:11:16.539 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:11:16.539 INFO:teuthology.orchestra.run.smithi191.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:11:16.540 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f6 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:11:16.540 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f0 2023-10-08T00:11:16.540 INFO:teuthology.orchestra.run.smithi191.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:11:16.540 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f7 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:11:16.540 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f1 2023-10-08T00:11:16.540 INFO:teuthology.orchestra.run.smithi191.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:11:16.540 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:10 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:11:16.540 INFO:teuthology.orchestra.run.smithi191.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T00:11:16.540 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:11 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:11:16.540 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.21.15.191/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T00:11:16.541 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft 42515sec preferred_lft 42515sec 2023-10-08T00:11:16.541 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec4:7aff:fe88:7211/64 scope link 2023-10-08T00:11:16.541 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:11:16.541 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-08T00:11:16.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:16.541 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2023-10-08T00:11:16.541 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2023-10-08T00:11:16.541 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link add name ceph-brx type bridge 2023-10-08T00:11:16.541 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip addr flush dev ceph-brx 2023-10-08T00:11:16.542 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set ceph-brx up 2023-10-08T00:11:16.542 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-08T00:11:16.542 DEBUG:teuthology.orchestra.run.smithi191:> ') 2023-10-08T00:11:16.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:16.708 DEBUG:teuthology.orchestra.run.smithi191:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T00:11:16.749 INFO:teuthology.orchestra.run.smithi191.stdout:1 2023-10-08T00:11:16.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:16.751 DEBUG:teuthology.orchestra.run.smithi191:> route 2023-10-08T00:11:16.821 INFO:teuthology.orchestra.run.smithi191.stdout:Kernel IP routing table 2023-10-08T00:11:16.821 INFO:teuthology.orchestra.run.smithi191.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T00:11:16.821 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T00:11:16.821 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T00:11:16.821 INFO:teuthology.orchestra.run.smithi191.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T00:11:16.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:16.822 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2023-10-08T00:11:16.822 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2023-10-08T00:11:16.822 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T00:11:16.822 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T00:11:16.822 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T00:11:16.822 DEBUG:teuthology.orchestra.run.smithi191:> ') 2023-10-08T00:11:17.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:17.068 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2023-10-08T00:11:17.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:17.087 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list-id 2023-10-08T00:11:17.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:17.145 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2023-10-08T00:11:17.145 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2023-10-08T00:11:17.145 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-08T00:11:17.145 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-10-08T00:11:17.145 DEBUG:teuthology.orchestra.run.smithi191:> ') 2023-10-08T00:11:17.236 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-10-08T00:11:17.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:17.237 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2023-10-08T00:11:17.237 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2023-10-08T00:11:17.237 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-10-08T00:11:17.237 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-08T00:11:17.237 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-10-08T00:11:17.237 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-10-08T00:11:17.237 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-10-08T00:11:17.237 DEBUG:teuthology.orchestra.run.smithi191:> ') 2023-10-08T00:11:17.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:17.426 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2023-10-08T00:11:17.426 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2023-10-08T00:11:17.426 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set brx.0 up 2023-10-08T00:11:17.426 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set dev brx.0 master ceph-brx 2023-10-08T00:11:17.426 DEBUG:teuthology.orchestra.run.smithi191:> ') 2023-10-08T00:11:17.517 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:11:17.517 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-08T00:11:17.534 INFO:teuthology.orchestra.run.smithi191.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-08T00:11:17.534 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:11:17.534 DEBUG:teuthology.orchestra.run.smithi191:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-08T00:11:17.588 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-08T00:11:17.588 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T00:11:17.588 DEBUG:teuthology.orchestra.run.smithi191:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:11:17.674 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:11:17.674 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:11:17.772 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:11:17.772 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:11:17.772 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.99:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T00:11:17.772 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@06962cdb-9a6f-417f-82fd-9e4e3dedbccf.cephfs=/ 2023-10-08T00:11:17.772 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.99:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T00:11:17.772 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T00:11:17.772 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T00:11:17.773 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T00:11:17.793 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:11:17.793 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:11:17.794 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.99:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T00:11:17.794 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@06962cdb-9a6f-417f-82fd-9e4e3dedbccf.cephfs=/ 2023-10-08T00:11:17.794 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.99:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T00:11:17.794 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T00:11:17.794 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T00:11:17.794 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T00:11:17.794 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.99:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T00:11:17.794 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.99:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T00:11:17.794 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=06962cdb-9a6f-417f-82fd-9e4e3dedbccf" will pass to kernel 2023-10-08T00:11:17.795 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T00:11:17.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:17.796 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-08T00:11:17.819 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2023-10-08T00:11:17.819 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T00:11:17.945 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.944+0000 7f3a2bcec700 1 Processor -- start 2023-10-08T00:11:17.946 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.945+0000 7f3a2bcec700 1 -- start start 2023-10-08T00:11:17.952 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.945+0000 7f3a2bcec700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3a2406f1c0 0x7f3a240b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:17.952 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.945+0000 7f3a2bcec700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a24063cf0 0x7f3a24099530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:17.952 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.945+0000 7f3a2bcec700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a24064170 con 0x7f3a240a7b20 2023-10-08T00:11:17.952 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.945+0000 7f3a2bcec700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a240990a0 con 0x7f3a2406f1c0 2023-10-08T00:11:17.952 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.945+0000 7f3a2bcec700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a240984b0 con 0x7f3a24063cf0 2023-10-08T00:11:17.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.945+0000 7f3a29287700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3a2406f1c0 0x7f3a240b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:17.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.946+0000 7f3a29a88700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f3a240a7b20 0x7f3a2406eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:55986/0 (socket says 172.21.15.99:55986) 2023-10-08T00:11:17.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.946+0000 7f3a29a88700 1 -- 172.21.15.99:0/1858310572 learned_addr learned my addr 172.21.15.99:0/1858310572 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:17.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.946+0000 7f3a2a289700 1 --2- 172.21.15.99:0/1858310572 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a24063cf0 0x7f3a24099530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:17.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.946+0000 7f3a29287700 1 -- 172.21.15.99:0/1858310572 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a24063cf0 msgr2=0x7f3a24099530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:17.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.946+0000 7f3a29287700 1 --2- 172.21.15.99:0/1858310572 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a24063cf0 0x7f3a24099530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:17.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.946+0000 7f3a29287700 1 -- 172.21.15.99:0/1858310572 >> v1:172.21.15.99:6789/0 conn(0x7f3a240a7b20 legacy=0x7f3a2406eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:17.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.946+0000 7f3a28a86700 1 -- 172.21.15.99:0/1858310572 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3330749946 0 0) 0x7f3a20001490 con 0x7f3a240a7b20 2023-10-08T00:11:17.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.946+0000 7f3a29287700 1 -- 172.21.15.99:0/1858310572 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a24098770 con 0x7f3a2406f1c0 2023-10-08T00:11:17.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.946+0000 7f3a29287700 1 --2- 172.21.15.99:0/1858310572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a2406f1c0 0x7f3a240b2ce0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3a140098e0 tx=0x7f3a140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d837f269218d4c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:17.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.946+0000 7f3a28a86700 1 -- 172.21.15.99:0/1858310572 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1400b640 con 0x7f3a2406f1c0 2023-10-08T00:11:17.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.946+0000 7f3a28a86700 1 -- 172.21.15.99:0/1858310572 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a1400b7a0 con 0x7f3a2406f1c0 2023-10-08T00:11:17.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.946+0000 7f3a28a86700 1 -- 172.21.15.99:0/1858310572 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1400bb40 con 0x7f3a2406f1c0 2023-10-08T00:11:17.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.947+0000 7f3a2bcec700 1 -- 172.21.15.99:0/1858310572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a2406f1c0 msgr2=0x7f3a240b2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:17.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.947+0000 7f3a2bcec700 1 --2- 172.21.15.99:0/1858310572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a2406f1c0 0x7f3a240b2ce0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3a140098e0 tx=0x7f3a140091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:17.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.947+0000 7f3a2bcec700 1 -- 172.21.15.99:0/1858310572 shutdown_connections 2023-10-08T00:11:17.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.947+0000 7f3a2bcec700 1 --2- 172.21.15.99:0/1858310572 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a24063cf0 0x7f3a24099530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:17.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.947+0000 7f3a2bcec700 1 --2- 172.21.15.99:0/1858310572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a2406f1c0 0x7f3a240b2ce0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:17.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.947+0000 7f3a2bcec700 1 -- 172.21.15.99:0/1858310572 >> 172.21.15.99:0/1858310572 conn(0x7f3a240cd1e0 msgr2=0x7f3a240cd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:17.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.947+0000 7f3a2bcec700 1 -- 172.21.15.99:0/1858310572 shutdown_connections 2023-10-08T00:11:17.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.947+0000 7f3a2bcec700 1 -- 172.21.15.99:0/1858310572 wait complete. 2023-10-08T00:11:17.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.948+0000 7f3a2bcec700 1 Processor -- start 2023-10-08T00:11:17.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.948+0000 7f3a2bcec700 1 -- start start 2023-10-08T00:11:17.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.949+0000 7f3a2bcec700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a24063cf0 0x7f3a24158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:17.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.949+0000 7f3a2bcec700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a2406f1c0 0x7f3a2415ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:17.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.949+0000 7f3a2bcec700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a240a7b20 0x7f3a2415f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:17.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.949+0000 7f3a2bcec700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2416aad0 con 0x7f3a24063cf0 2023-10-08T00:11:17.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.949+0000 7f3a2bcec700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2416afa0 con 0x7f3a2406f1c0 2023-10-08T00:11:17.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.949+0000 7f3a2bcec700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a24169f00 con 0x7f3a240a7b20 2023-10-08T00:11:17.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.949+0000 7f3a29a88700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a24063cf0 0x7f3a24158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:17.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.949+0000 7f3a29a88700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a24063cf0 0x7f3a24158150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55848/0 (socket says 172.21.15.99:55848) 2023-10-08T00:11:17.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.949+0000 7f3a29a88700 1 -- 172.21.15.99:0/2362159587 learned_addr learned my addr 172.21.15.99:0/2362159587 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:17.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.949+0000 7f3a2a289700 1 --2- 172.21.15.99:0/2362159587 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a240a7b20 0x7f3a2415f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:17.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.949+0000 7f3a29287700 1 --2- 172.21.15.99:0/2362159587 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a2406f1c0 0x7f3a2415ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:17.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.949+0000 7f3a29a88700 1 -- 172.21.15.99:0/2362159587 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a240a7b20 msgr2=0x7f3a2415f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:17.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.949+0000 7f3a29a88700 1 --2- 172.21.15.99:0/2362159587 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a240a7b20 0x7f3a2415f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:17.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.949+0000 7f3a29a88700 1 -- 172.21.15.99:0/2362159587 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a2406f1c0 msgr2=0x7f3a2415ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:17.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.949+0000 7f3a29a88700 1 --2- 172.21.15.99:0/2362159587 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a2406f1c0 0x7f3a2415ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:17.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.949+0000 7f3a29a88700 1 -- 172.21.15.99:0/2362159587 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a14009420 con 0x7f3a24063cf0 2023-10-08T00:11:17.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.949+0000 7f3a2a289700 1 --2- 172.21.15.99:0/2362159587 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a240a7b20 0x7f3a2415f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:11:17.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.949+0000 7f3a29a88700 1 --2- 172.21.15.99:0/2362159587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a24063cf0 0x7f3a24158150 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3a2000cc70 tx=0x7f3a20001a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:17.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.949+0000 7f3a19ffb700 1 -- 172.21.15.99:0/2362159587 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2000ce80 con 0x7f3a24063cf0 2023-10-08T00:11:17.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.949+0000 7f3a19ffb700 1 -- 172.21.15.99:0/2362159587 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a2000ab70 con 0x7f3a24063cf0 2023-10-08T00:11:17.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.949+0000 7f3a2bcec700 1 -- 172.21.15.99:0/2362159587 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a2416a1a0 con 0x7f3a24063cf0 2023-10-08T00:11:17.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.949+0000 7f3a19ffb700 1 -- 172.21.15.99:0/2362159587 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a200070f0 con 0x7f3a24063cf0 2023-10-08T00:11:17.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.950+0000 7f3a2bcec700 1 -- 172.21.15.99:0/2362159587 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a2416a760 con 0x7f3a24063cf0 2023-10-08T00:11:17.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.951+0000 7f3a2bcec700 1 -- 172.21.15.99:0/2362159587 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a24098150 con 0x7f3a24063cf0 2023-10-08T00:11:17.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.951+0000 7f3a19ffb700 1 -- 172.21.15.99:0/2362159587 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3a200074e0 con 0x7f3a24063cf0 2023-10-08T00:11:17.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.951+0000 7f3a19ffb700 1 --2- 172.21.15.99:0/2362159587 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3a08066f30 0x7f3a080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:17.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.951+0000 7f3a19ffb700 1 -- 172.21.15.99:0/2362159587 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3a20084840 con 0x7f3a24063cf0 2023-10-08T00:11:17.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.951+0000 7f3a29287700 1 --2- 172.21.15.99:0/2362159587 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3a08066f30 0x7f3a080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:17.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.952+0000 7f3a29287700 1 --2- 172.21.15.99:0/2362159587 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3a08066f30 0x7f3a080693e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3a140093f0 tx=0x7f3a14017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:17.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:17.955+0000 7f3a19ffb700 1 -- 172.21.15.99:0/2362159587 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3a20051b30 con 0x7f3a24063cf0 2023-10-08T00:11:18.184 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:18.184+0000 7f3a2bcec700 1 -- 172.21.15.99:0/2362159587 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3a24064170 con 0x7f3a24063cf0 2023-10-08T00:11:18.185 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:18.184+0000 7f3a19ffb700 1 -- 172.21.15.99:0/2362159587 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3a24064170 con 0x7f3a24063cf0 2023-10-08T00:11:18.185 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T00:11:18.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:18.186+0000 7f3a2bcec700 1 -- 172.21.15.99:0/2362159587 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3a08066f30 msgr2=0x7f3a080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:18.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:18.186+0000 7f3a2bcec700 1 --2- 172.21.15.99:0/2362159587 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3a08066f30 0x7f3a080693e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3a140093f0 tx=0x7f3a14017000 comp rx=0 tx=0).stop 2023-10-08T00:11:18.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:18.186+0000 7f3a2bcec700 1 -- 172.21.15.99:0/2362159587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a24063cf0 msgr2=0x7f3a24158150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:18.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:18.186+0000 7f3a2bcec700 1 --2- 172.21.15.99:0/2362159587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a24063cf0 0x7f3a24158150 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3a2000cc70 tx=0x7f3a20001a30 comp rx=0 tx=0).stop 2023-10-08T00:11:18.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:18.186+0000 7f3a2bcec700 1 -- 172.21.15.99:0/2362159587 shutdown_connections 2023-10-08T00:11:18.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:18.186+0000 7f3a2bcec700 1 --2- 172.21.15.99:0/2362159587 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3a08066f30 0x7f3a080693e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:18.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:18.186+0000 7f3a2bcec700 1 --2- 172.21.15.99:0/2362159587 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a240a7b20 0x7f3a2415f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:18.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:18.186+0000 7f3a2bcec700 1 --2- 172.21.15.99:0/2362159587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a24063cf0 0x7f3a24158150 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:18.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:18.186+0000 7f3a2bcec700 1 --2- 172.21.15.99:0/2362159587 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a2406f1c0 0x7f3a2415ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:18.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:18.186+0000 7f3a2bcec700 1 -- 172.21.15.99:0/2362159587 >> 172.21.15.99:0/2362159587 conn(0x7f3a240cd1e0 msgr2=0x7f3a2406df90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:18.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:18.186+0000 7f3a2bcec700 1 -- 172.21.15.99:0/2362159587 shutdown_connections 2023-10-08T00:11:18.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:18.186+0000 7f3a2bcec700 1 -- 172.21.15.99:0/2362159587 wait complete. 2023-10-08T00:11:18.199 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-08T00:11:18.199 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T00:11:18.199 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi191.front.sepia.ceph.com 2023-10-08T00:11:18.199 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-10-08T00:11:18.199 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-08T00:11:18.199 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-08T00:11:18.199 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-08T00:11:18.200 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T00:11:18.200 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-10-08T00:11:18.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:18.200 DEBUG:teuthology.orchestra.run.smithi191:> ip addr 2023-10-08T00:11:18.215 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T00:11:18.215 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T00:11:18.216 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T00:11:18.216 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:11:18.216 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2023-10-08T00:11:18.216 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:11:18.216 INFO:teuthology.orchestra.run.smithi191.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:11:18.216 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f6 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:11:18.216 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f0 2023-10-08T00:11:18.216 INFO:teuthology.orchestra.run.smithi191.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:11:18.217 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f7 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:11:18.217 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f1 2023-10-08T00:11:18.217 INFO:teuthology.orchestra.run.smithi191.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:11:18.217 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:10 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:11:18.217 INFO:teuthology.orchestra.run.smithi191.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T00:11:18.217 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:11 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:11:18.217 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.21.15.191/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T00:11:18.217 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft 42513sec preferred_lft 42513sec 2023-10-08T00:11:18.218 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec4:7aff:fe88:7211/64 scope link 2023-10-08T00:11:18.218 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:11:18.218 INFO:teuthology.orchestra.run.smithi191.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T00:11:18.218 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether ea:1c:d9:46:05:b3 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:11:18.218 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T00:11:18.218 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:11:18.218 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec77:a7ff:fea4:fbb/64 scope link 2023-10-08T00:11:18.218 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:11:18.218 INFO:teuthology.orchestra.run.smithi191.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T00:11:18.219 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether ea:1c:d9:46:05:b3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-08T00:11:18.219 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::e81c:d9ff:fe46:5b3/64 scope link tentative 2023-10-08T00:11:18.219 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:11:18.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:18.219 DEBUG:teuthology.orchestra.run.smithi191:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T00:11:18.280 INFO:teuthology.orchestra.run.smithi191.stdout:1 2023-10-08T00:11:18.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:18.281 DEBUG:teuthology.orchestra.run.smithi191:> route 2023-10-08T00:11:18.338 INFO:teuthology.orchestra.run.smithi191.stdout:Kernel IP routing table 2023-10-08T00:11:18.338 INFO:teuthology.orchestra.run.smithi191.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T00:11:18.338 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T00:11:18.338 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T00:11:18.339 INFO:teuthology.orchestra.run.smithi191.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T00:11:18.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:18.339 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2023-10-08T00:11:18.339 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2023-10-08T00:11:18.339 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T00:11:18.339 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T00:11:18.339 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T00:11:18.339 DEBUG:teuthology.orchestra.run.smithi191:> ') 2023-10-08T00:11:18.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:18.437 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2023-10-08T00:11:18.452 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-08T00:11:18.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:18.452 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list-id 2023-10-08T00:11:18.507 INFO:teuthology.orchestra.run.smithi191.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-10-08T00:11:18.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:18.508 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2023-10-08T00:11:18.509 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2023-10-08T00:11:18.509 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-08T00:11:18.509 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-10-08T00:11:18.510 DEBUG:teuthology.orchestra.run.smithi191:> ') 2023-10-08T00:11:18.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:18.595 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-08T00:11:18.628 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T00:11:18.629 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T00:11:18.629 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T00:11:18.629 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:11:18.629 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2023-10-08T00:11:18.629 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:11:18.629 INFO:teuthology.orchestra.run.smithi191.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T00:11:18.629 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 4e:7c:47:21:e5:ae brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-08T00:11:18.629 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-08T00:11:18.629 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:11:18.629 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::4c7c:47ff:fe21:e5ae/64 scope link tentative 2023-10-08T00:11:18.630 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:11:18.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:18.637 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-08T00:11:18.707 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T00:11:18.707 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T00:11:18.707 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T00:11:18.707 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:11:18.707 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2023-10-08T00:11:18.708 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:11:18.708 INFO:teuthology.orchestra.run.smithi191.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T00:11:18.708 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 4e:7c:47:21:e5:ae brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-08T00:11:18.708 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-08T00:11:18.708 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:11:18.708 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::4c7c:47ff:fe21:e5ae/64 scope link tentative 2023-10-08T00:11:18.708 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:11:18.716 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-10-08T00:11:18.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:18.717 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2023-10-08T00:11:18.717 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2023-10-08T00:11:18.717 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-10-08T00:11:18.717 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-08T00:11:18.717 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-10-08T00:11:18.717 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-10-08T00:11:18.717 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-10-08T00:11:18.717 DEBUG:teuthology.orchestra.run.smithi191:> ') 2023-10-08T00:11:18.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:18.933 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2023-10-08T00:11:18.934 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2023-10-08T00:11:18.934 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set brx.1 up 2023-10-08T00:11:18.934 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set dev brx.1 master ceph-brx 2023-10-08T00:11:18.934 DEBUG:teuthology.orchestra.run.smithi191:> ') 2023-10-08T00:11:19.013 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:11:19.014 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-10-08T00:11:19.029 INFO:teuthology.orchestra.run.smithi191.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-10-08T00:11:19.029 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:11:19.030 DEBUG:teuthology.orchestra.run.smithi191:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-10-08T00:11:19.083 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2023-10-08T00:11:19.084 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T00:11:19.084 DEBUG:teuthology.orchestra.run.smithi191:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:11:19.164 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:11:19.164 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:11:19.183 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:11:19.183 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:11:19.183 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.99:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T00:11:19.183 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 3@06962cdb-9a6f-417f-82fd-9e4e3dedbccf.cephfs=/ 2023-10-08T00:11:19.183 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.99:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T00:11:19.183 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T00:11:19.183 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T00:11:19.184 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T00:11:19.204 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:11:19.204 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:11:19.204 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.99:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T00:11:19.204 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 3@06962cdb-9a6f-417f-82fd-9e4e3dedbccf.cephfs=/ 2023-10-08T00:11:19.205 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.99:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T00:11:19.205 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T00:11:19.205 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T00:11:19.205 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T00:11:19.205 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.99:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T00:11:19.205 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.99:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T00:11:19.205 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,key=3,mds_namespace=cephfs,fsid=06962cdb-9a6f-417f-82fd-9e4e3dedbccf" will pass to kernel 2023-10-08T00:11:19.206 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T00:11:19.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:19.207 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-10-08T00:11:19.271 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-10-08T00:11:19.299 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:11:19.416 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.415+0000 7f2176325700 1 Processor -- start 2023-10-08T00:11:19.416 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.415+0000 7f2176325700 1 -- start start 2023-10-08T00:11:19.419 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.416+0000 7f2176325700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2170159e60 0x7f21700b6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:19.419 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.416+0000 7f2176325700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f21700cffa0 0x7f217006a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:19.419 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.416+0000 7f2176325700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21700b3d40 con 0x7f217015a230 2023-10-08T00:11:19.419 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.416+0000 7f2176325700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2170069d70 con 0x7f2170159e60 2023-10-08T00:11:19.419 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.416+0000 7f2176325700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21700b4300 con 0x7f21700cffa0 2023-10-08T00:11:19.419 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.416+0000 7f216ffff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2170159e60 0x7f21700b6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:19.419 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.416+0000 7f21748c2700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f217015a230 0x7f21700b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:55996/0 (socket says 172.21.15.99:55996) 2023-10-08T00:11:19.419 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.416+0000 7f21748c2700 1 -- 172.21.15.99:0/2592866599 learned_addr learned my addr 172.21.15.99:0/2592866599 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:19.420 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.416+0000 7f216ffff700 1 -- 172.21.15.99:0/2592866599 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f21700cffa0 msgr2=0x7f217006a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:19.420 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.416+0000 7f2167fff700 1 --2- 172.21.15.99:0/2592866599 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f21700cffa0 0x7f217006a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:19.420 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.416+0000 7f216f7fe700 1 -- 172.21.15.99:0/2592866599 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1220722559 0 0) 0x7f21700b3d40 con 0x7f217015a230 2023-10-08T00:11:19.420 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.416+0000 7f216ffff700 1 --2- 172.21.15.99:0/2592866599 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f21700cffa0 0x7f217006a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:19.420 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.416+0000 7f216ffff700 1 -- 172.21.15.99:0/2592866599 >> v1:172.21.15.99:6789/0 conn(0x7f217015a230 legacy=0x7f21700b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:19.420 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.416+0000 7f216ffff700 1 -- 172.21.15.99:0/2592866599 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21700d0480 con 0x7f2170159e60 2023-10-08T00:11:19.420 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.416+0000 7f2167fff700 1 --2- 172.21.15.99:0/2592866599 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f21700cffa0 0x7f217006a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:11:19.420 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.417+0000 7f216ffff700 1 --2- 172.21.15.99:0/2592866599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2170159e60 0x7f21700b6890 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f21600098e0 tx=0x7f21600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfd4db507fc4fc71 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:19.420 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.417+0000 7f216f7fe700 1 -- 172.21.15.99:0/2592866599 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2160009e80 con 0x7f2170159e60 2023-10-08T00:11:19.420 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.417+0000 7f216f7fe700 1 -- 172.21.15.99:0/2592866599 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f216000b860 con 0x7f2170159e60 2023-10-08T00:11:19.420 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.417+0000 7f216f7fe700 1 -- 172.21.15.99:0/2592866599 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216000bbe0 con 0x7f2170159e60 2023-10-08T00:11:19.420 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.417+0000 7f2176325700 1 -- 172.21.15.99:0/2592866599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2170159e60 msgr2=0x7f21700b6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:19.421 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.417+0000 7f2176325700 1 --2- 172.21.15.99:0/2592866599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2170159e60 0x7f21700b6890 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f21600098e0 tx=0x7f21600091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:19.421 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.417+0000 7f2176325700 1 -- 172.21.15.99:0/2592866599 shutdown_connections 2023-10-08T00:11:19.421 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.417+0000 7f2176325700 1 --2- 172.21.15.99:0/2592866599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2170159e60 0x7f21700b6890 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:19.421 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.417+0000 7f2176325700 1 --2- 172.21.15.99:0/2592866599 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f21700cffa0 0x7f217006a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:19.421 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.417+0000 7f2176325700 1 -- 172.21.15.99:0/2592866599 >> 172.21.15.99:0/2592866599 conn(0x7f21700d1a50 msgr2=0x7f2170068f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:19.421 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.417+0000 7f2176325700 1 -- 172.21.15.99:0/2592866599 shutdown_connections 2023-10-08T00:11:19.421 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.417+0000 7f2176325700 1 -- 172.21.15.99:0/2592866599 wait complete. 2023-10-08T00:11:19.421 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.418+0000 7f2176325700 1 Processor -- start 2023-10-08T00:11:19.421 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.418+0000 7f2176325700 1 -- start start 2023-10-08T00:11:19.424 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.419+0000 7f2176325700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21700cffa0 0x7f21700cda50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:19.424 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.419+0000 7f2176325700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2170159e60 0x7f21700c5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:19.424 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.419+0000 7f2176325700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f217015a230 0x7f2170154da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:19.424 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.419+0000 7f2176325700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21700c7110 con 0x7f21700cffa0 2023-10-08T00:11:19.424 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.419+0000 7f2176325700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21700c6260 con 0x7f217015a230 2023-10-08T00:11:19.424 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.419+0000 7f2176325700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21700c6500 con 0x7f2170159e60 2023-10-08T00:11:19.424 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.419+0000 7f216ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21700cffa0 0x7f21700cda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:19.424 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.419+0000 7f216ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21700cffa0 0x7f21700cda50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55878/0 (socket says 172.21.15.99:55878) 2023-10-08T00:11:19.425 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.419+0000 7f216ffff700 1 -- 172.21.15.99:0/837694293 learned_addr learned my addr 172.21.15.99:0/837694293 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:19.425 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.419+0000 7f2167fff700 1 --2- 172.21.15.99:0/837694293 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2170159e60 0x7f21700c5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:19.425 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.419+0000 7f21748c2700 1 --2- 172.21.15.99:0/837694293 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f217015a230 0x7f2170154da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:19.425 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.420+0000 7f216ffff700 1 -- 172.21.15.99:0/837694293 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2170159e60 msgr2=0x7f21700c5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:19.425 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.420+0000 7f216ffff700 1 --2- 172.21.15.99:0/837694293 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2170159e60 0x7f21700c5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:19.425 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.420+0000 7f216ffff700 1 -- 172.21.15.99:0/837694293 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f217015a230 msgr2=0x7f2170154da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:19.425 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.420+0000 7f216ffff700 1 --2- 172.21.15.99:0/837694293 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f217015a230 0x7f2170154da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:19.425 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.420+0000 7f216ffff700 1 -- 172.21.15.99:0/837694293 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2160009420 con 0x7f21700cffa0 2023-10-08T00:11:19.425 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.420+0000 7f216ffff700 1 --2- 172.21.15.99:0/837694293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21700cffa0 0x7f21700cda50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2170089730 tx=0x7f2160017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:19.425 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.420+0000 7f216cff9700 1 -- 172.21.15.99:0/837694293 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216000bbe0 con 0x7f21700cffa0 2023-10-08T00:11:19.425 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.420+0000 7f216cff9700 1 -- 172.21.15.99:0/837694293 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2160011420 con 0x7f21700cffa0 2023-10-08T00:11:19.426 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.420+0000 7f2176325700 1 -- 172.21.15.99:0/837694293 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21700c67e0 con 0x7f21700cffa0 2023-10-08T00:11:19.426 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.420+0000 7f216cff9700 1 -- 172.21.15.99:0/837694293 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2160011800 con 0x7f21700cffa0 2023-10-08T00:11:19.426 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.420+0000 7f2176325700 1 -- 172.21.15.99:0/837694293 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21700c6da0 con 0x7f21700cffa0 2023-10-08T00:11:19.426 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.421+0000 7f2176325700 1 -- 172.21.15.99:0/837694293 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f217004c3c0 con 0x7f21700cffa0 2023-10-08T00:11:19.426 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.421+0000 7f216cff9700 1 -- 172.21.15.99:0/837694293 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2160011bf0 con 0x7f21700cffa0 2023-10-08T00:11:19.426 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.422+0000 7f216cff9700 1 --2- 172.21.15.99:0/837694293 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2150066f30 0x7f21500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:19.426 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.422+0000 7f216cff9700 1 -- 172.21.15.99:0/837694293 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2160085f00 con 0x7f21700cffa0 2023-10-08T00:11:19.426 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.422+0000 7f2167fff700 1 --2- 172.21.15.99:0/837694293 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2150066f30 0x7f21500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:19.426 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.422+0000 7f2167fff700 1 --2- 172.21.15.99:0/837694293 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2150066f30 0x7f21500693e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f21700c74f0 tx=0x7f2158006f90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:19.427 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.426+0000 7f216cff9700 1 -- 172.21.15.99:0/837694293 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2160053170 con 0x7f21700cffa0 2023-10-08T00:11:19.608 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.608+0000 7f2176325700 1 -- 172.21.15.99:0/837694293 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f21700cdfa0 con 0x7f21700cffa0 2023-10-08T00:11:19.609 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.609+0000 7f216cff9700 1 -- 172.21.15.99:0/837694293 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f2160056790 con 0x7f21700cffa0 2023-10-08T00:11:19.610 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:11:19.610 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":20,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:10:55.080854+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:10:51.822157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T00:10:52.415130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:11:19.612 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.611+0000 7f2176325700 1 -- 172.21.15.99:0/837694293 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2150066f30 msgr2=0x7f21500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:19.612 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.611+0000 7f2176325700 1 --2- 172.21.15.99:0/837694293 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2150066f30 0x7f21500693e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f21700c74f0 tx=0x7f2158006f90 comp rx=0 tx=0).stop 2023-10-08T00:11:19.612 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.611+0000 7f2176325700 1 -- 172.21.15.99:0/837694293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21700cffa0 msgr2=0x7f21700cda50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:19.612 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.611+0000 7f2176325700 1 --2- 172.21.15.99:0/837694293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21700cffa0 0x7f21700cda50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2170089730 tx=0x7f2160017000 comp rx=0 tx=0).stop 2023-10-08T00:11:19.612 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.611+0000 7f2176325700 1 -- 172.21.15.99:0/837694293 shutdown_connections 2023-10-08T00:11:19.613 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.611+0000 7f2176325700 1 --2- 172.21.15.99:0/837694293 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2150066f30 0x7f21500693e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:19.613 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.611+0000 7f2176325700 1 --2- 172.21.15.99:0/837694293 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f217015a230 0x7f2170154da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:19.613 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.611+0000 7f2176325700 1 --2- 172.21.15.99:0/837694293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21700cffa0 0x7f21700cda50 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:19.613 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.611+0000 7f2176325700 1 --2- 172.21.15.99:0/837694293 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2170159e60 0x7f21700c5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:19.613 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.611+0000 7f2176325700 1 -- 172.21.15.99:0/837694293 >> 172.21.15.99:0/837694293 conn(0x7f21700d1a50 msgr2=0x7f217006a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:19.613 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.611+0000 7f2176325700 1 -- 172.21.15.99:0/837694293 shutdown_connections 2023-10-08T00:11:19.613 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.611+0000 7f2176325700 1 -- 172.21.15.99:0/837694293 wait complete. 2023-10-08T00:11:19.624 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:10:42.063344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T00:10:51.822157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T00:10:52.415130+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T00:11:19.624 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T00:11:19.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.741+0000 7f57164f4700 1 Processor -- start 2023-10-08T00:11:19.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.741+0000 7f57164f4700 1 -- start start 2023-10-08T00:11:19.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.742+0000 7f57164f4700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f57100b4860 0x7f57100b6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:19.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.742+0000 7f57164f4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57100b4c30 0x7f5710159a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:19.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.742+0000 7f57164f4700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57100b14f0 con 0x7f5710159fd0 2023-10-08T00:11:19.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.742+0000 7f57164f4700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57100b17d0 con 0x7f57100b4860 2023-10-08T00:11:19.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.742+0000 7f57164f4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57100b1ab0 con 0x7f57100b4c30 2023-10-08T00:11:19.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.742+0000 7f570ffff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f57100b4860 0x7f57100b6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:19.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.742+0000 7f5714a91700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f5710159fd0 0x7f57100cfe90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:55998/0 (socket says 172.21.15.99:55998) 2023-10-08T00:11:19.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.742+0000 7f5714a91700 1 -- 172.21.15.99:0/1651895951 learned_addr learned my addr 172.21.15.99:0/1651895951 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:19.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.743+0000 7f570f7fe700 1 --2- 172.21.15.99:0/1651895951 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57100b4c30 0x7f5710159a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:19.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.743+0000 7f570ffff700 1 -- 172.21.15.99:0/1651895951 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57100b4c30 msgr2=0x7f5710159a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:19.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.743+0000 7f570ffff700 1 --2- 172.21.15.99:0/1651895951 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57100b4c30 0x7f5710159a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:19.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.743+0000 7f570ffff700 1 -- 172.21.15.99:0/1651895951 >> v1:172.21.15.99:6789/0 conn(0x7f5710159fd0 legacy=0x7f57100cfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:19.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.743+0000 7f570ffff700 1 -- 172.21.15.99:0/1651895951 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57100b1d90 con 0x7f57100b4860 2023-10-08T00:11:19.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.743+0000 7f570effd700 1 -- 172.21.15.99:0/1651895951 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1332289988 0 0) 0x7f57100b14f0 con 0x7f5710159fd0 2023-10-08T00:11:19.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.743+0000 7f570ffff700 1 --2- 172.21.15.99:0/1651895951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57100b4860 0x7f57100b6700 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f57000098d0 tx=0x7f5700009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f428c9c57317eed server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:19.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.743+0000 7f570effd700 1 -- 172.21.15.99:0/1651895951 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5700009e70 con 0x7f57100b4860 2023-10-08T00:11:19.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.743+0000 7f570effd700 1 -- 172.21.15.99:0/1651895951 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5700005c10 con 0x7f57100b4860 2023-10-08T00:11:19.748 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.743+0000 7f570effd700 1 -- 172.21.15.99:0/1651895951 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570000b6e0 con 0x7f57100b4860 2023-10-08T00:11:19.748 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.743+0000 7f57164f4700 1 -- 172.21.15.99:0/1651895951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57100b4860 msgr2=0x7f57100b6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:19.748 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.743+0000 7f57164f4700 1 --2- 172.21.15.99:0/1651895951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57100b4860 0x7f57100b6700 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f57000098d0 tx=0x7f5700009120 comp rx=0 tx=0).stop 2023-10-08T00:11:19.748 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.744+0000 7f57164f4700 1 -- 172.21.15.99:0/1651895951 shutdown_connections 2023-10-08T00:11:19.748 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.744+0000 7f57164f4700 1 --2- 172.21.15.99:0/1651895951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57100b4860 0x7f57100b6700 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:19.748 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.744+0000 7f57164f4700 1 --2- 172.21.15.99:0/1651895951 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57100b4c30 0x7f5710159a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:19.748 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.744+0000 7f57164f4700 1 -- 172.21.15.99:0/1651895951 >> 172.21.15.99:0/1651895951 conn(0x7f571009a0f0 msgr2=0x7f5710067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:19.749 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.744+0000 7f57164f4700 1 -- 172.21.15.99:0/1651895951 shutdown_connections 2023-10-08T00:11:19.749 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.744+0000 7f57164f4700 1 -- 172.21.15.99:0/1651895951 wait complete. 2023-10-08T00:11:19.749 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.745+0000 7f57164f4700 1 Processor -- start 2023-10-08T00:11:19.749 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.745+0000 7f57164f4700 1 -- start start 2023-10-08T00:11:19.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.745+0000 7f57164f4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57100b4860 0x7f571009c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:19.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.746+0000 7f57164f4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57100b4c30 0x7f57101590a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:19.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.746+0000 7f57164f4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5710159fd0 0x7f57100abe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:19.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.746+0000 7f57164f4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57100adb70 con 0x7f57100b4c30 2023-10-08T00:11:19.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.746+0000 7f57164f4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57100ac3c0 con 0x7f57100b4860 2023-10-08T00:11:19.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.746+0000 7f57164f4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f571009cb20 con 0x7f5710159fd0 2023-10-08T00:11:19.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.746+0000 7f570f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57100b4c30 0x7f57101590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:19.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.746+0000 7f570f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57100b4c30 0x7f57101590a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55910/0 (socket says 172.21.15.99:55910) 2023-10-08T00:11:19.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.746+0000 7f570f7fe700 1 -- 172.21.15.99:0/2048583414 learned_addr learned my addr 172.21.15.99:0/2048583414 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:19.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.746+0000 7f570ffff700 1 --2- 172.21.15.99:0/2048583414 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57100b4860 0x7f571009c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:19.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.746+0000 7f5714a91700 1 --2- 172.21.15.99:0/2048583414 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5710159fd0 0x7f57100abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:19.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.746+0000 7f570f7fe700 1 -- 172.21.15.99:0/2048583414 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5710159fd0 msgr2=0x7f57100abe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:19.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.746+0000 7f570f7fe700 1 --2- 172.21.15.99:0/2048583414 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5710159fd0 0x7f57100abe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:19.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.746+0000 7f570f7fe700 1 -- 172.21.15.99:0/2048583414 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57100b4860 msgr2=0x7f571009c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:19.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.746+0000 7f570f7fe700 1 --2- 172.21.15.99:0/2048583414 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57100b4860 0x7f571009c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:19.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.746+0000 7f570f7fe700 1 -- 172.21.15.99:0/2048583414 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57000093c0 con 0x7f57100b4c30 2023-10-08T00:11:19.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.746+0000 7f570f7fe700 1 --2- 172.21.15.99:0/2048583414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57100b4c30 0x7f57101590a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f570400d890 tx=0x7f5704008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:19.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.746+0000 7f56f7fff700 1 -- 172.21.15.99:0/2048583414 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57040125f0 con 0x7f57100b4c30 2023-10-08T00:11:19.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.746+0000 7f56f7fff700 1 -- 172.21.15.99:0/2048583414 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5704012750 con 0x7f57100b4c30 2023-10-08T00:11:19.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.746+0000 7f57164f4700 1 -- 172.21.15.99:0/2048583414 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f571009ce00 con 0x7f57100b4c30 2023-10-08T00:11:19.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.746+0000 7f56f7fff700 1 -- 172.21.15.99:0/2048583414 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5704012af0 con 0x7f57100b4c30 2023-10-08T00:11:19.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.746+0000 7f57164f4700 1 -- 172.21.15.99:0/2048583414 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f571009d3c0 con 0x7f57100b4c30 2023-10-08T00:11:19.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.747+0000 7f57164f4700 1 -- 172.21.15.99:0/2048583414 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f571004c3c0 con 0x7f57100b4c30 2023-10-08T00:11:19.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.748+0000 7f56f7fff700 1 -- 172.21.15.99:0/2048583414 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f570401e460 con 0x7f57100b4c30 2023-10-08T00:11:19.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.748+0000 7f56f7fff700 1 --2- 172.21.15.99:0/2048583414 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f56fc066f30 0x7f56fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:19.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.748+0000 7f56f7fff700 1 -- 172.21.15.99:0/2048583414 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5704084a20 con 0x7f57100b4c30 2023-10-08T00:11:19.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.748+0000 7f570ffff700 1 --2- 172.21.15.99:0/2048583414 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f56fc066f30 0x7f56fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:19.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.749+0000 7f570ffff700 1 --2- 172.21.15.99:0/2048583414 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f56fc066f30 0x7f56fc0693e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5700009160 tx=0x7f5700016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:19.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.752+0000 7f56f7fff700 1 -- 172.21.15.99:0/2048583414 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5704051c90 con 0x7f57100b4c30 2023-10-08T00:11:19.935 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.934+0000 7f57164f4700 1 -- 172.21.15.99:0/2048583414 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f57100b2070 con 0x7f57100b4c30 2023-10-08T00:11:19.935 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.935+0000 7f56f7fff700 1 -- 172.21.15.99:0/2048583414 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f57040552b0 con 0x7f57100b4c30 2023-10-08T00:11:19.935 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2023-10-08T00:11:19.937 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.936+0000 7f57164f4700 1 -- 172.21.15.99:0/2048583414 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f56fc066f30 msgr2=0x7f56fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:19.937 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.936+0000 7f57164f4700 1 --2- 172.21.15.99:0/2048583414 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f56fc066f30 0x7f56fc0693e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5700009160 tx=0x7f5700016000 comp rx=0 tx=0).stop 2023-10-08T00:11:19.938 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.936+0000 7f57164f4700 1 -- 172.21.15.99:0/2048583414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57100b4c30 msgr2=0x7f57101590a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:19.938 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.937+0000 7f57164f4700 1 --2- 172.21.15.99:0/2048583414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57100b4c30 0x7f57101590a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f570400d890 tx=0x7f5704008040 comp rx=0 tx=0).stop 2023-10-08T00:11:19.938 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.937+0000 7f57164f4700 1 -- 172.21.15.99:0/2048583414 shutdown_connections 2023-10-08T00:11:19.938 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.937+0000 7f57164f4700 1 --2- 172.21.15.99:0/2048583414 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f56fc066f30 0x7f56fc0693e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:19.938 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.937+0000 7f57164f4700 1 --2- 172.21.15.99:0/2048583414 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57100b4860 0x7f571009c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:19.938 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.937+0000 7f57164f4700 1 --2- 172.21.15.99:0/2048583414 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5710159fd0 0x7f57100abe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:19.938 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.937+0000 7f57164f4700 1 --2- 172.21.15.99:0/2048583414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57100b4c30 0x7f57101590a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:19.938 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.937+0000 7f57164f4700 1 -- 172.21.15.99:0/2048583414 >> 172.21.15.99:0/2048583414 conn(0x7f571009a0f0 msgr2=0x7f571015a880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:19.938 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.937+0000 7f57164f4700 1 -- 172.21.15.99:0/2048583414 shutdown_connections 2023-10-08T00:11:19.938 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:19.937+0000 7f57164f4700 1 -- 172.21.15.99:0/2048583414 wait complete. 2023-10-08T00:11:19.949 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T00:11:20.066 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.066+0000 7f282e3c0700 1 Processor -- start 2023-10-08T00:11:20.067 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.066+0000 7f282e3c0700 1 -- start start 2023-10-08T00:11:20.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.066+0000 7f282e3c0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f282815b9a0 0x7f28280caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:20.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.067+0000 7f282e3c0700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f282815bdf0 con 0x7f282806d370 2023-10-08T00:11:20.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.067+0000 7f282e3c0700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f282815c8d0 con 0x7f2828152300 2023-10-08T00:11:20.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.067+0000 7f282e3c0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28280b1400 con 0x7f282815b9a0 2023-10-08T00:11:20.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.067+0000 7f282c95d700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f282806d370 0x7f282806d790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56006/0 (socket says 172.21.15.99:56006) 2023-10-08T00:11:20.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.067+0000 7f282c95d700 1 -- 172.21.15.99:0/2587813554 learned_addr learned my addr 172.21.15.99:0/2587813554 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:20.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.067+0000 7f28277fe700 1 --2- 172.21.15.99:0/2587813554 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f282815b9a0 0x7f28280caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:20.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.067+0000 7f2826ffd700 1 -- 172.21.15.99:0/2587813554 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1202253761 0 0) 0x7f2820001490 con 0x7f282806d370 2023-10-08T00:11:20.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.067+0000 7f2826ffd700 1 -- 172.21.15.99:0/2587813554 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2810003570 con 0x7f282806d370 2023-10-08T00:11:20.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.067+0000 7f2826ffd700 1 -- 172.21.15.99:0/2587813554 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2819202283 0 0) 0x7f282815c8d0 con 0x7f2828152300 2023-10-08T00:11:20.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.067+0000 7f2826ffd700 1 -- 172.21.15.99:0/2587813554 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2820001490 con 0x7f2828152300 2023-10-08T00:11:20.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.067+0000 7f2826ffd700 1 -- 172.21.15.99:0/2587813554 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1126210452 0 0) 0x7f2810003570 con 0x7f282806d370 2023-10-08T00:11:20.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.067+0000 7f28277fe700 1 -- 172.21.15.99:0/2587813554 >> v1:172.21.15.191:6789/0 conn(0x7f2828152300 legacy=0x7f2828158df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:20.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.067+0000 7f28277fe700 1 -- 172.21.15.99:0/2587813554 >> v1:172.21.15.99:6789/0 conn(0x7f282806d370 legacy=0x7f282806d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:20.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.067+0000 7f28277fe700 1 -- 172.21.15.99:0/2587813554 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28280c9370 con 0x7f282815b9a0 2023-10-08T00:11:20.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.067+0000 7f28277fe700 1 --2- 172.21.15.99:0/2587813554 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f282815b9a0 0x7f28280caaf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2818009910 tx=0x7f28180091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79d7b9fe1c8ac44b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:20.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.067+0000 7f2826ffd700 1 -- 172.21.15.99:0/2587813554 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2820001980 con 0x7f282815b9a0 2023-10-08T00:11:20.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.068+0000 7f2826ffd700 1 -- 172.21.15.99:0/2587813554 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2818011420 con 0x7f282815b9a0 2023-10-08T00:11:20.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.068+0000 7f2826ffd700 1 -- 172.21.15.99:0/2587813554 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28180117c0 con 0x7f282815b9a0 2023-10-08T00:11:20.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.068+0000 7f282e3c0700 1 -- 172.21.15.99:0/2587813554 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f282815b9a0 msgr2=0x7f28280caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:20.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.068+0000 7f282e3c0700 1 --2- 172.21.15.99:0/2587813554 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f282815b9a0 0x7f28280caaf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2818009910 tx=0x7f28180091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:20.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.068+0000 7f282e3c0700 1 -- 172.21.15.99:0/2587813554 shutdown_connections 2023-10-08T00:11:20.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.068+0000 7f282e3c0700 1 --2- 172.21.15.99:0/2587813554 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f282815b9a0 0x7f28280caaf0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:20.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.068+0000 7f282e3c0700 1 -- 172.21.15.99:0/2587813554 >> 172.21.15.99:0/2587813554 conn(0x7f2828069870 msgr2=0x7f2828069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:20.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.068+0000 7f282e3c0700 1 -- 172.21.15.99:0/2587813554 shutdown_connections 2023-10-08T00:11:20.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.068+0000 7f282e3c0700 1 -- 172.21.15.99:0/2587813554 wait complete. 2023-10-08T00:11:20.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.069+0000 7f282e3c0700 1 Processor -- start 2023-10-08T00:11:20.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.069+0000 7f282e3c0700 1 -- start start 2023-10-08T00:11:20.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.070+0000 7f282e3c0700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f282806d370 0x7f28281694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:20.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.070+0000 7f282e3c0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2828152300 0x7f282816b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:20.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.070+0000 7f282e3c0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f282815b9a0 0x7f28280c7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:20.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.070+0000 7f282e3c0700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282816b710 con 0x7f2828152300 2023-10-08T00:11:20.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.070+0000 7f282e3c0700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28281595f0 con 0x7f282806d370 2023-10-08T00:11:20.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.070+0000 7f282e3c0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28281535c0 con 0x7f282815b9a0 2023-10-08T00:11:20.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.070+0000 7f28277fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2828152300 0x7f282816b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:20.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.070+0000 7f28277fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2828152300 0x7f282816b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55920/0 (socket says 172.21.15.99:55920) 2023-10-08T00:11:20.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.070+0000 7f28277fe700 1 -- 172.21.15.99:0/4260667805 learned_addr learned my addr 172.21.15.99:0/4260667805 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:20.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.070+0000 7f2827fff700 1 --2- 172.21.15.99:0/4260667805 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f282806d370 0x7f28281694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:20.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.070+0000 7f282c95d700 1 --2- 172.21.15.99:0/4260667805 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f282815b9a0 0x7f28280c7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:20.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.070+0000 7f28277fe700 1 -- 172.21.15.99:0/4260667805 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f282815b9a0 msgr2=0x7f28280c7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:20.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.070+0000 7f28277fe700 1 --2- 172.21.15.99:0/4260667805 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f282815b9a0 0x7f28280c7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:20.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.070+0000 7f28277fe700 1 -- 172.21.15.99:0/4260667805 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f282806d370 msgr2=0x7f28281694b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:20.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.070+0000 7f28277fe700 1 --2- 172.21.15.99:0/4260667805 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f282806d370 0x7f28281694b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:20.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.070+0000 7f28277fe700 1 -- 172.21.15.99:0/4260667805 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2818009420 con 0x7f2828152300 2023-10-08T00:11:20.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.071+0000 7f28277fe700 1 --2- 172.21.15.99:0/4260667805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2828152300 0x7f282816b1d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f282806a380 tx=0x7f2818016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:20.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.071+0000 7f280ffff700 1 -- 172.21.15.99:0/4260667805 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281801caf0 con 0x7f2828152300 2023-10-08T00:11:20.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.071+0000 7f280ffff700 1 -- 172.21.15.99:0/4260667805 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f281801cc70 con 0x7f2828152300 2023-10-08T00:11:20.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.071+0000 7f282e3c0700 1 -- 172.21.15.99:0/4260667805 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2828158cc0 con 0x7f2828152300 2023-10-08T00:11:20.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.071+0000 7f280ffff700 1 -- 172.21.15.99:0/4260667805 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281801a450 con 0x7f2828152300 2023-10-08T00:11:20.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.071+0000 7f282e3c0700 1 -- 172.21.15.99:0/4260667805 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2828159280 con 0x7f2828152300 2023-10-08T00:11:20.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.072+0000 7f282e3c0700 1 -- 172.21.15.99:0/4260667805 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2828159780 con 0x7f2828152300 2023-10-08T00:11:20.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.072+0000 7f280ffff700 1 -- 172.21.15.99:0/4260667805 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f281801a840 con 0x7f2828152300 2023-10-08T00:11:20.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.072+0000 7f280ffff700 1 --2- 172.21.15.99:0/4260667805 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2814066f30 0x7f28140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:20.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.072+0000 7f280ffff700 1 -- 172.21.15.99:0/4260667805 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2818085dc0 con 0x7f2828152300 2023-10-08T00:11:20.077 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.073+0000 7f2827fff700 1 --2- 172.21.15.99:0/4260667805 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2814066f30 0x7f28140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:20.077 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.073+0000 7f2827fff700 1 --2- 172.21.15.99:0/4260667805 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2814066f30 0x7f28140693e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f281c007fd0 tx=0x7f281c007d10 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:20.077 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.077+0000 7f280ffff700 1 -- 172.21.15.99:0/4260667805 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2818053030 con 0x7f2828152300 2023-10-08T00:11:20.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.259+0000 7f282e3c0700 1 -- 172.21.15.99:0/4260667805 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f282804c3c0 con 0x7f2828152300 2023-10-08T00:11:20.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.260+0000 7f280ffff700 1 -- 172.21.15.99:0/4260667805 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2818056650 con 0x7f2828152300 2023-10-08T00:11:20.261 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2023-10-08T00:11:20.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.262+0000 7f282e3c0700 1 -- 172.21.15.99:0/4260667805 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2814066f30 msgr2=0x7f28140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:20.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.262+0000 7f282e3c0700 1 --2- 172.21.15.99:0/4260667805 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2814066f30 0x7f28140693e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f281c007fd0 tx=0x7f281c007d10 comp rx=0 tx=0).stop 2023-10-08T00:11:20.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.262+0000 7f282e3c0700 1 -- 172.21.15.99:0/4260667805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2828152300 msgr2=0x7f282816b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:20.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.262+0000 7f282e3c0700 1 --2- 172.21.15.99:0/4260667805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2828152300 0x7f282816b1d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f282806a380 tx=0x7f2818016000 comp rx=0 tx=0).stop 2023-10-08T00:11:20.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.262+0000 7f282e3c0700 1 -- 172.21.15.99:0/4260667805 shutdown_connections 2023-10-08T00:11:20.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.262+0000 7f282e3c0700 1 --2- 172.21.15.99:0/4260667805 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2814066f30 0x7f28140693e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:20.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.262+0000 7f282e3c0700 1 --2- 172.21.15.99:0/4260667805 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f282806d370 0x7f28281694b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:20.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.262+0000 7f282e3c0700 1 --2- 172.21.15.99:0/4260667805 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f282815b9a0 0x7f28280c7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:20.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.262+0000 7f282e3c0700 1 --2- 172.21.15.99:0/4260667805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2828152300 0x7f282816b1d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:20.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.262+0000 7f282e3c0700 1 -- 172.21.15.99:0/4260667805 >> 172.21.15.99:0/4260667805 conn(0x7f2828069870 msgr2=0x7f28280c4230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:20.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.262+0000 7f282e3c0700 1 -- 172.21.15.99:0/4260667805 shutdown_connections 2023-10-08T00:11:20.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.262+0000 7f282e3c0700 1 -- 172.21.15.99:0/4260667805 wait complete. 2023-10-08T00:11:20.274 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T00:11:20.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.391+0000 7f399a7ec700 1 Processor -- start 2023-10-08T00:11:20.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.391+0000 7f399a7ec700 1 -- start start 2023-10-08T00:11:20.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.392+0000 7f399a7ec700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f39940cd300 0x7f399415afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:20.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.392+0000 7f399a7ec700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f39940b6760 0x7f39940d0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:20.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.392+0000 7f399a7ec700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f39940d12a0 0x7f399406a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:20.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.392+0000 7f399a7ec700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f399415e7a0 con 0x7f39940d12a0 2023-10-08T00:11:20.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.392+0000 7f399a7ec700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39941513f0 con 0x7f39940cd300 2023-10-08T00:11:20.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.392+0000 7f399a7ec700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39940d1700 con 0x7f39940b6760 2023-10-08T00:11:20.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.392+0000 7f3998d89700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f39940d12a0 0x7f399406a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:20.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.392+0000 7f3998d89700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f39940d12a0 0x7f399406a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55930/0 (socket says 172.21.15.99:55930) 2023-10-08T00:11:20.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.392+0000 7f3998d89700 1 -- 172.21.15.99:0/493621538 learned_addr learned my addr 172.21.15.99:0/493621538 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:20.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.392+0000 7f39937fe700 1 --2- 172.21.15.99:0/493621538 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f39940b6760 0x7f39940d0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:20.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.392+0000 7f3993fff700 1 --2- 172.21.15.99:0/493621538 >> v2:172.21.15.191:3300/0 conn(0x7f39940cd300 0x7f399415afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:20.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.393+0000 7f3998d89700 1 -- 172.21.15.99:0/493621538 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f39940b6760 msgr2=0x7f39940d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:20.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.393+0000 7f3998d89700 1 --2- 172.21.15.99:0/493621538 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f39940b6760 0x7f39940d0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:20.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.393+0000 7f3998d89700 1 -- 172.21.15.99:0/493621538 >> v2:172.21.15.191:3300/0 conn(0x7f39940cd300 msgr2=0x7f399415afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:20.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.393+0000 7f3998d89700 1 --2- 172.21.15.99:0/493621538 >> v2:172.21.15.191:3300/0 conn(0x7f39940cd300 0x7f399415afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:20.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.393+0000 7f3998d89700 1 -- 172.21.15.99:0/493621538 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f399415db20 con 0x7f39940d12a0 2023-10-08T00:11:20.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.393+0000 7f3993fff700 1 --2- 172.21.15.99:0/493621538 >> v2:172.21.15.191:3300/0 conn(0x7f39940cd300 0x7f399415afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:11:20.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.393+0000 7f3998d89700 1 --2- 172.21.15.99:0/493621538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39940d12a0 0x7f399406a510 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3994089ea0 tx=0x7f398c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7e30f7d989320b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:20.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.393+0000 7f3992ffd700 1 -- 172.21.15.99:0/493621538 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398c00e070 con 0x7f39940d12a0 2023-10-08T00:11:20.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.393+0000 7f3992ffd700 1 -- 172.21.15.99:0/493621538 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f398c00ba50 con 0x7f39940d12a0 2023-10-08T00:11:20.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.393+0000 7f3992ffd700 1 -- 172.21.15.99:0/493621538 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398c0092e0 con 0x7f39940d12a0 2023-10-08T00:11:20.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.393+0000 7f399a7ec700 1 -- 172.21.15.99:0/493621538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39940d12a0 msgr2=0x7f399406a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:20.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.393+0000 7f399a7ec700 1 --2- 172.21.15.99:0/493621538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39940d12a0 0x7f399406a510 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3994089ea0 tx=0x7f398c009840 comp rx=0 tx=0).stop 2023-10-08T00:11:20.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.393+0000 7f399a7ec700 1 -- 172.21.15.99:0/493621538 shutdown_connections 2023-10-08T00:11:20.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.393+0000 7f399a7ec700 1 --2- 172.21.15.99:0/493621538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39940d12a0 0x7f399406a510 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:20.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.393+0000 7f399a7ec700 1 --2- 172.21.15.99:0/493621538 >> v2:172.21.15.191:3300/0 conn(0x7f39940cd300 0x7f399415afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:20.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.393+0000 7f399a7ec700 1 --2- 172.21.15.99:0/493621538 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f39940b6760 0x7f39940d0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:20.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.393+0000 7f399a7ec700 1 -- 172.21.15.99:0/493621538 >> 172.21.15.99:0/493621538 conn(0x7f3994068d90 msgr2=0x7f3994069190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:20.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.393+0000 7f399a7ec700 1 -- 172.21.15.99:0/493621538 shutdown_connections 2023-10-08T00:11:20.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.393+0000 7f399a7ec700 1 -- 172.21.15.99:0/493621538 wait complete. 2023-10-08T00:11:20.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.394+0000 7f399a7ec700 1 Processor -- start 2023-10-08T00:11:20.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.395+0000 7f399a7ec700 1 -- start start 2023-10-08T00:11:20.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.395+0000 7f399a7ec700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f39940b6760 0x7f3994169f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:20.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.395+0000 7f399a7ec700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f39940cd300 0x7f39940b0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:20.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.395+0000 7f399a7ec700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39940d12a0 0x7f399414a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:20.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.395+0000 7f399a7ec700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399414ea20 con 0x7f39940d12a0 2023-10-08T00:11:20.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.395+0000 7f399a7ec700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39940b0bb0 con 0x7f39940cd300 2023-10-08T00:11:20.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.395+0000 7f399a7ec700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39940b0e30 con 0x7f39940b6760 2023-10-08T00:11:20.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.395+0000 7f3998d89700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39940d12a0 0x7f399414a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:20.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.396+0000 7f3998d89700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39940d12a0 0x7f399414a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55936/0 (socket says 172.21.15.99:55936) 2023-10-08T00:11:20.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.396+0000 7f3998d89700 1 -- 172.21.15.99:0/2090920188 learned_addr learned my addr 172.21.15.99:0/2090920188 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:20.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.396+0000 7f3993fff700 1 --2- 172.21.15.99:0/2090920188 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f39940b6760 0x7f3994169f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:20.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.396+0000 7f39937fe700 1 --2- 172.21.15.99:0/2090920188 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f39940cd300 0x7f39940b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:20.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.396+0000 7f3998d89700 1 -- 172.21.15.99:0/2090920188 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f39940b6760 msgr2=0x7f3994169f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:20.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.396+0000 7f3998d89700 1 --2- 172.21.15.99:0/2090920188 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f39940b6760 0x7f3994169f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:20.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.396+0000 7f3998d89700 1 -- 172.21.15.99:0/2090920188 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f39940cd300 msgr2=0x7f39940b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:20.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.396+0000 7f3998d89700 1 --2- 172.21.15.99:0/2090920188 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f39940cd300 0x7f39940b0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:20.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.396+0000 7f3998d89700 1 -- 172.21.15.99:0/2090920188 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f398c009ae0 con 0x7f39940d12a0 2023-10-08T00:11:20.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.396+0000 7f39937fe700 1 --2- 172.21.15.99:0/2090920188 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f39940cd300 0x7f39940b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:11:20.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.396+0000 7f3998d89700 1 --2- 172.21.15.99:0/2090920188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39940d12a0 0x7f399414a920 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f399406b520 tx=0x7f398c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:20.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.396+0000 7f397bfff700 1 -- 172.21.15.99:0/2090920188 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398c0032f0 con 0x7f39940d12a0 2023-10-08T00:11:20.402 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.396+0000 7f397bfff700 1 -- 172.21.15.99:0/2090920188 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f398c008e40 con 0x7f39940d12a0 2023-10-08T00:11:20.402 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.396+0000 7f397bfff700 1 -- 172.21.15.99:0/2090920188 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398c0266b0 con 0x7f39940d12a0 2023-10-08T00:11:20.402 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.396+0000 7f399a7ec700 1 -- 172.21.15.99:0/2090920188 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39940b1110 con 0x7f39940d12a0 2023-10-08T00:11:20.402 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.396+0000 7f399a7ec700 1 -- 172.21.15.99:0/2090920188 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f399414ec30 con 0x7f39940d12a0 2023-10-08T00:11:20.402 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.397+0000 7f399a7ec700 1 -- 172.21.15.99:0/2090920188 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f399404c3c0 con 0x7f39940d12a0 2023-10-08T00:11:20.402 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.397+0000 7f397bfff700 1 -- 172.21.15.99:0/2090920188 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f398c003040 con 0x7f39940d12a0 2023-10-08T00:11:20.402 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.398+0000 7f397bfff700 1 --2- 172.21.15.99:0/2090920188 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3980066f30 0x7f39800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:20.402 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.398+0000 7f397bfff700 1 -- 172.21.15.99:0/2090920188 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f398c086020 con 0x7f39940d12a0 2023-10-08T00:11:20.402 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.398+0000 7f3993fff700 1 --2- 172.21.15.99:0/2090920188 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3980066f30 0x7f39800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:20.402 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.398+0000 7f3993fff700 1 --2- 172.21.15.99:0/2090920188 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3980066f30 0x7f39800693e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3984006f00 tx=0x7f3984008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:20.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.402+0000 7f397bfff700 1 -- 172.21.15.99:0/2090920188 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f398c053340 con 0x7f39940d12a0 2023-10-08T00:11:20.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.584+0000 7f399a7ec700 1 -- 172.21.15.99:0/2090920188 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f39940b4010 con 0x7f39940d12a0 2023-10-08T00:11:20.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.584+0000 7f397bfff700 1 -- 172.21.15.99:0/2090920188 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f398c056960 con 0x7f39940d12a0 2023-10-08T00:11:20.585 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2023-10-08T00:11:20.587 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.586+0000 7f399a7ec700 1 -- 172.21.15.99:0/2090920188 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3980066f30 msgr2=0x7f39800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:20.587 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.586+0000 7f399a7ec700 1 --2- 172.21.15.99:0/2090920188 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3980066f30 0x7f39800693e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3984006f00 tx=0x7f3984008040 comp rx=0 tx=0).stop 2023-10-08T00:11:20.587 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.586+0000 7f399a7ec700 1 -- 172.21.15.99:0/2090920188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39940d12a0 msgr2=0x7f399414a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:20.587 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.586+0000 7f399a7ec700 1 --2- 172.21.15.99:0/2090920188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39940d12a0 0x7f399414a920 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f399406b520 tx=0x7f398c018000 comp rx=0 tx=0).stop 2023-10-08T00:11:20.587 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.586+0000 7f399a7ec700 1 -- 172.21.15.99:0/2090920188 shutdown_connections 2023-10-08T00:11:20.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.586+0000 7f399a7ec700 1 --2- 172.21.15.99:0/2090920188 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3980066f30 0x7f39800693e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:20.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.586+0000 7f399a7ec700 1 --2- 172.21.15.99:0/2090920188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39940d12a0 0x7f399414a920 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:20.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.586+0000 7f399a7ec700 1 --2- 172.21.15.99:0/2090920188 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f39940b6760 0x7f3994169f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:20.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.586+0000 7f399a7ec700 1 --2- 172.21.15.99:0/2090920188 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f39940cd300 0x7f39940b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:20.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.586+0000 7f399a7ec700 1 -- 172.21.15.99:0/2090920188 >> 172.21.15.99:0/2090920188 conn(0x7f3994068d90 msgr2=0x7f3994069160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:20.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.586+0000 7f399a7ec700 1 -- 172.21.15.99:0/2090920188 shutdown_connections 2023-10-08T00:11:20.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.586+0000 7f399a7ec700 1 -- 172.21.15.99:0/2090920188 wait complete. 2023-10-08T00:11:20.599 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:11:20.713 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.712+0000 7ff6ba75b700 1 Processor -- start 2023-10-08T00:11:20.714 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.712+0000 7ff6ba75b700 1 -- start start 2023-10-08T00:11:20.721 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.713+0000 7ff6ba75b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff6b406c630 0x7ff6b4067660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:20.721 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.713+0000 7ff6ba75b700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6b40cb370 con 0x7ff6b40b0890 2023-10-08T00:11:20.721 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.713+0000 7ff6ba75b700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6b4067fa0 con 0x7ff6b4067ba0 2023-10-08T00:11:20.721 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.713+0000 7ff6ba75b700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6b40b0c60 con 0x7ff6b406c630 2023-10-08T00:11:20.721 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.713+0000 7ff6b3fff700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7ff6b40b0890 0x7ff6b40cb800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56012/0 (socket says 172.21.15.99:56012) 2023-10-08T00:11:20.721 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.713+0000 7ff6b3fff700 1 -- 172.21.15.99:0/2725867181 learned_addr learned my addr 172.21.15.99:0/2725867181 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:20.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.714+0000 7ff6b37fe700 1 --2- 172.21.15.99:0/2725867181 >> v2:172.21.15.191:3300/0 conn(0x7ff6b406c630 0x7ff6b4067660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:20.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.714+0000 7ff6b2ffd700 1 -- 172.21.15.99:0/2725867181 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2431477289 0 0) 0x7ff6b40cb370 con 0x7ff6b40b0890 2023-10-08T00:11:20.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.714+0000 7ff6b2ffd700 1 -- 172.21.15.99:0/2725867181 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff69c003570 con 0x7ff6b40b0890 2023-10-08T00:11:20.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.714+0000 7ff6b37fe700 1 -- 172.21.15.99:0/2725867181 >> v1:172.21.15.191:6789/0 conn(0x7ff6b4067ba0 legacy=0x7ff6b4159790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:20.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.714+0000 7ff6b37fe700 1 -- 172.21.15.99:0/2725867181 >> v1:172.21.15.99:6789/0 conn(0x7ff6b40b0890 legacy=0x7ff6b40cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:20.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.714+0000 7ff6b37fe700 1 -- 172.21.15.99:0/2725867181 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b406a1e0 con 0x7ff6b406c630 2023-10-08T00:11:20.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.714+0000 7ff6b37fe700 1 --2- 172.21.15.99:0/2725867181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6b406c630 0x7ff6b4067660 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff6a80098e0 tx=0x7ff6a80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d70ad00131945201 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:20.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.714+0000 7ff6b2ffd700 1 -- 172.21.15.99:0/2725867181 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a800b640 con 0x7ff6b406c630 2023-10-08T00:11:20.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.714+0000 7ff6b2ffd700 1 -- 172.21.15.99:0/2725867181 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6a800b7a0 con 0x7ff6b406c630 2023-10-08T00:11:20.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.714+0000 7ff6b2ffd700 1 -- 172.21.15.99:0/2725867181 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a800bb40 con 0x7ff6b406c630 2023-10-08T00:11:20.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.715+0000 7ff6ba75b700 1 -- 172.21.15.99:0/2725867181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6b406c630 msgr2=0x7ff6b4067660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:20.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.715+0000 7ff6ba75b700 1 --2- 172.21.15.99:0/2725867181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6b406c630 0x7ff6b4067660 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff6a80098e0 tx=0x7ff6a80091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:20.724 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.715+0000 7ff6ba75b700 1 -- 172.21.15.99:0/2725867181 shutdown_connections 2023-10-08T00:11:20.724 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.715+0000 7ff6ba75b700 1 --2- 172.21.15.99:0/2725867181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6b406c630 0x7ff6b4067660 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:20.724 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.715+0000 7ff6ba75b700 1 -- 172.21.15.99:0/2725867181 >> 172.21.15.99:0/2725867181 conn(0x7ff6b406fe50 msgr2=0x7ff6b4069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:20.724 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.715+0000 7ff6ba75b700 1 -- 172.21.15.99:0/2725867181 shutdown_connections 2023-10-08T00:11:20.724 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.715+0000 7ff6ba75b700 1 -- 172.21.15.99:0/2725867181 wait complete. 2023-10-08T00:11:20.724 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.716+0000 7ff6ba75b700 1 Processor -- start 2023-10-08T00:11:20.725 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.716+0000 7ff6ba75b700 1 -- start start 2023-10-08T00:11:20.725 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.717+0000 7ff6ba75b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6b4067ba0 0x7ff6b40af090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:20.725 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.717+0000 7ff6ba75b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6b406c630 0x7ff6b4158f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:20.725 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.717+0000 7ff6ba75b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6b40b0890 0x7ff6b4159470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:20.725 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.717+0000 7ff6ba75b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b41599b0 con 0x7ff6b40b0890 2023-10-08T00:11:20.725 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.717+0000 7ff6ba75b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b416d7e0 con 0x7ff6b406c630 2023-10-08T00:11:20.725 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.717+0000 7ff6ba75b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b416e360 con 0x7ff6b4067ba0 2023-10-08T00:11:20.726 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.717+0000 7ff6b8cf8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6b40b0890 0x7ff6b4159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:20.726 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.717+0000 7ff6b8cf8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6b40b0890 0x7ff6b4159470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55948/0 (socket says 172.21.15.99:55948) 2023-10-08T00:11:20.726 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.717+0000 7ff6b8cf8700 1 -- 172.21.15.99:0/3398089837 learned_addr learned my addr 172.21.15.99:0/3398089837 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:20.726 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.717+0000 7ff6b37fe700 1 --2- 172.21.15.99:0/3398089837 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6b406c630 0x7ff6b4158f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:20.726 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.717+0000 7ff6b3fff700 1 --2- 172.21.15.99:0/3398089837 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6b4067ba0 0x7ff6b40af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:20.726 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.717+0000 7ff6b8cf8700 1 -- 172.21.15.99:0/3398089837 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6b4067ba0 msgr2=0x7ff6b40af090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:20.726 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.717+0000 7ff6b8cf8700 1 --2- 172.21.15.99:0/3398089837 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6b4067ba0 0x7ff6b40af090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:20.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.717+0000 7ff6b8cf8700 1 -- 172.21.15.99:0/3398089837 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6b406c630 msgr2=0x7ff6b4158f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:20.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.717+0000 7ff6b8cf8700 1 --2- 172.21.15.99:0/3398089837 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6b406c630 0x7ff6b4158f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:20.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.717+0000 7ff6b8cf8700 1 -- 172.21.15.99:0/3398089837 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a8009420 con 0x7ff6b40b0890 2023-10-08T00:11:20.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.717+0000 7ff6b8cf8700 1 --2- 172.21.15.99:0/3398089837 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6b40b0890 0x7ff6b4159470 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac00c840 tx=0x7ff6ac001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:20.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.718+0000 7ff69bfff700 1 -- 172.21.15.99:0/3398089837 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac012630 con 0x7ff6b40b0890 2023-10-08T00:11:20.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.718+0000 7ff69bfff700 1 -- 172.21.15.99:0/3398089837 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6ac012790 con 0x7ff6b40b0890 2023-10-08T00:11:20.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.718+0000 7ff69bfff700 1 -- 172.21.15.99:0/3398089837 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac012b50 con 0x7ff6b40b0890 2023-10-08T00:11:20.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.718+0000 7ff6ba75b700 1 -- 172.21.15.99:0/3398089837 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6b416da30 con 0x7ff6b40b0890 2023-10-08T00:11:20.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.718+0000 7ff6ba75b700 1 -- 172.21.15.99:0/3398089837 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6b416dff0 con 0x7ff6b40b0890 2023-10-08T00:11:20.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.718+0000 7ff6ba75b700 1 -- 172.21.15.99:0/3398089837 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6b416e4d0 con 0x7ff6b40b0890 2023-10-08T00:11:20.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.719+0000 7ff69bfff700 1 -- 172.21.15.99:0/3398089837 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff6ac003030 con 0x7ff6b40b0890 2023-10-08T00:11:20.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.719+0000 7ff69bfff700 1 --2- 172.21.15.99:0/3398089837 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff6a0066f30 0x7ff6a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:20.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.719+0000 7ff69bfff700 1 -- 172.21.15.99:0/3398089837 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff6ac084320 con 0x7ff6b40b0890 2023-10-08T00:11:20.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.720+0000 7ff6b3fff700 1 --2- 172.21.15.99:0/3398089837 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff6a0066f30 0x7ff6a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:20.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.720+0000 7ff6b3fff700 1 --2- 172.21.15.99:0/3398089837 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff6a0066f30 0x7ff6a00693e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4007fd0 tx=0x7ff6a4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:20.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.723+0000 7ff69bfff700 1 -- 172.21.15.99:0/3398089837 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff6ac051590 con 0x7ff6b40b0890 2023-10-08T00:11:20.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.905+0000 7ff6ba75b700 1 -- 172.21.15.99:0/3398089837 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6b404c3c0 con 0x7ff6b40b0890 2023-10-08T00:11:20.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.906+0000 7ff69bfff700 1 -- 172.21.15.99:0/3398089837 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15350 (secure 0 0 0) 0x7ff6ac054bb0 con 0x7ff6b40b0890 2023-10-08T00:11:20.907 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:11:20.907 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":20,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:10:55.080854+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:10:51.822157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T00:10:52.415130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:11:20.909 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.908+0000 7ff6ba75b700 1 -- 172.21.15.99:0/3398089837 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff6a0066f30 msgr2=0x7ff6a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:20.909 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.908+0000 7ff6ba75b700 1 --2- 172.21.15.99:0/3398089837 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff6a0066f30 0x7ff6a00693e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4007fd0 tx=0x7ff6a4009040 comp rx=0 tx=0).stop 2023-10-08T00:11:20.909 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.908+0000 7ff6ba75b700 1 -- 172.21.15.99:0/3398089837 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6b40b0890 msgr2=0x7ff6b4159470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:20.909 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.908+0000 7ff6ba75b700 1 --2- 172.21.15.99:0/3398089837 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6b40b0890 0x7ff6b4159470 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac00c840 tx=0x7ff6ac001670 comp rx=0 tx=0).stop 2023-10-08T00:11:20.909 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.908+0000 7ff6ba75b700 1 -- 172.21.15.99:0/3398089837 shutdown_connections 2023-10-08T00:11:20.909 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.908+0000 7ff6ba75b700 1 --2- 172.21.15.99:0/3398089837 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff6a0066f30 0x7ff6a00693e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:20.909 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.908+0000 7ff6ba75b700 1 --2- 172.21.15.99:0/3398089837 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6b40b0890 0x7ff6b4159470 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:20.910 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.908+0000 7ff6ba75b700 1 --2- 172.21.15.99:0/3398089837 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6b4067ba0 0x7ff6b40af090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:20.910 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.908+0000 7ff6ba75b700 1 --2- 172.21.15.99:0/3398089837 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6b406c630 0x7ff6b4158f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:20.910 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.908+0000 7ff6ba75b700 1 -- 172.21.15.99:0/3398089837 >> 172.21.15.99:0/3398089837 conn(0x7ff6b406fe50 msgr2=0x7ff6b40cdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:20.910 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.908+0000 7ff6ba75b700 1 -- 172.21.15.99:0/3398089837 shutdown_connections 2023-10-08T00:11:20.910 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:20.908+0000 7ff6ba75b700 1 -- 172.21.15.99:0/3398089837 wait complete. 2023-10-08T00:11:20.921 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:10:42.063344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T00:10:51.822157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T00:10:52.415130+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T00:11:20.921 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T00:11:21.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.037+0000 7f1e62a8b700 1 Processor -- start 2023-10-08T00:11:21.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.037+0000 7f1e62a8b700 1 -- start start 2023-10-08T00:11:21.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.038+0000 7f1e62a8b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e5c15a290 0x7f1e5c06a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:21.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.038+0000 7f1e62a8b700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e5c0b0790 con 0x7f1e5c06aae0 2023-10-08T00:11:21.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.038+0000 7f1e62a8b700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e5c0ce1f0 con 0x7f1e5c159ec0 2023-10-08T00:11:21.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.038+0000 7f1e62a8b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c06af60 con 0x7f1e5c15a290 2023-10-08T00:11:21.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.038+0000 7f1e61028700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f1e5c06aae0 0x7f1e5c0cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56022/0 (socket says 172.21.15.99:56022) 2023-10-08T00:11:21.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.038+0000 7f1e61028700 1 -- 172.21.15.99:0/1745755727 learned_addr learned my addr 172.21.15.99:0/1745755727 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:21.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.038+0000 7f1e5bfff700 1 --2- 172.21.15.99:0/1745755727 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e5c15a290 0x7f1e5c06a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:21.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.039+0000 7f1e5b7fe700 1 -- 172.21.15.99:0/1745755727 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 430877482 0 0) 0x7f1e5c0b0790 con 0x7f1e5c06aae0 2023-10-08T00:11:21.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.039+0000 7f1e5b7fe700 1 -- 172.21.15.99:0/1745755727 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e4c003570 con 0x7f1e5c06aae0 2023-10-08T00:11:21.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.039+0000 7f1e5b7fe700 1 -- 172.21.15.99:0/1745755727 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 198085568 0 0) 0x7f1e5c0ce1f0 con 0x7f1e5c159ec0 2023-10-08T00:11:21.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.039+0000 7f1e5b7fe700 1 -- 172.21.15.99:0/1745755727 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e5c0b0790 con 0x7f1e5c159ec0 2023-10-08T00:11:21.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.039+0000 7f1e5bfff700 1 -- 172.21.15.99:0/1745755727 >> v1:172.21.15.191:6789/0 conn(0x7f1e5c159ec0 legacy=0x7f1e5c0af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:21.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.039+0000 7f1e5bfff700 1 -- 172.21.15.99:0/1745755727 >> v1:172.21.15.99:6789/0 conn(0x7f1e5c06aae0 legacy=0x7f1e5c0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:21.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.039+0000 7f1e5bfff700 1 -- 172.21.15.99:0/1745755727 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e5c0b0b20 con 0x7f1e5c15a290 2023-10-08T00:11:21.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.039+0000 7f1e5bfff700 1 --2- 172.21.15.99:0/1745755727 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e5c15a290 0x7f1e5c06a5a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1e48009910 tx=0x7f1e480091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d41b7279db86ef53 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:21.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.039+0000 7f1e5b7fe700 1 -- 172.21.15.99:0/1745755727 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e48011630 con 0x7f1e5c15a290 2023-10-08T00:11:21.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.039+0000 7f1e5b7fe700 1 -- 172.21.15.99:0/1745755727 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e48011790 con 0x7f1e5c15a290 2023-10-08T00:11:21.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.039+0000 7f1e5b7fe700 1 -- 172.21.15.99:0/1745755727 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e48011b30 con 0x7f1e5c15a290 2023-10-08T00:11:21.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.040+0000 7f1e62a8b700 1 -- 172.21.15.99:0/1745755727 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e5c15a290 msgr2=0x7f1e5c06a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:21.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.040+0000 7f1e62a8b700 1 --2- 172.21.15.99:0/1745755727 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e5c15a290 0x7f1e5c06a5a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1e48009910 tx=0x7f1e480091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:21.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.040+0000 7f1e62a8b700 1 -- 172.21.15.99:0/1745755727 shutdown_connections 2023-10-08T00:11:21.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.040+0000 7f1e62a8b700 1 --2- 172.21.15.99:0/1745755727 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e5c15a290 0x7f1e5c06a5a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:21.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.040+0000 7f1e62a8b700 1 -- 172.21.15.99:0/1745755727 >> 172.21.15.99:0/1745755727 conn(0x7f1e5c0d1880 msgr2=0x7f1e5c0cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:21.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.040+0000 7f1e62a8b700 1 -- 172.21.15.99:0/1745755727 shutdown_connections 2023-10-08T00:11:21.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.040+0000 7f1e62a8b700 1 -- 172.21.15.99:0/1745755727 wait complete. 2023-10-08T00:11:21.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.041+0000 7f1e62a8b700 1 Processor -- start 2023-10-08T00:11:21.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.041+0000 7f1e62a8b700 1 -- start start 2023-10-08T00:11:21.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.042+0000 7f1e62a8b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e5c06aae0 0x7f1e5c15dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:21.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.042+0000 7f1e62a8b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e5c159ec0 0x7f1e5c15b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:21.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.042+0000 7f1e62a8b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e5c15a290 0x7f1e5c15bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:21.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.042+0000 7f1e62a8b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c159050 con 0x7f1e5c159ec0 2023-10-08T00:11:21.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.042+0000 7f1e62a8b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c15c210 con 0x7f1e5c15a290 2023-10-08T00:11:21.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.042+0000 7f1e62a8b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c15c4f0 con 0x7f1e5c06aae0 2023-10-08T00:11:21.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.042+0000 7f1e5bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e5c159ec0 0x7f1e5c15b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:21.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.042+0000 7f1e5bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e5c159ec0 0x7f1e5c15b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55964/0 (socket says 172.21.15.99:55964) 2023-10-08T00:11:21.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.042+0000 7f1e5bfff700 1 -- 172.21.15.99:0/2233442129 learned_addr learned my addr 172.21.15.99:0/2233442129 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:21.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.042+0000 7f1e61028700 1 --2- 172.21.15.99:0/2233442129 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e5c15a290 0x7f1e5c15bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:21.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.042+0000 7f1e5bfff700 1 -- 172.21.15.99:0/2233442129 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e5c06aae0 msgr2=0x7f1e5c15dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:21.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.042+0000 7f1e60827700 1 --2- 172.21.15.99:0/2233442129 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e5c06aae0 0x7f1e5c15dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:21.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.042+0000 7f1e5bfff700 1 --2- 172.21.15.99:0/2233442129 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e5c06aae0 0x7f1e5c15dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:21.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.042+0000 7f1e5bfff700 1 -- 172.21.15.99:0/2233442129 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e5c15a290 msgr2=0x7f1e5c15bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:21.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.042+0000 7f1e5bfff700 1 --2- 172.21.15.99:0/2233442129 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e5c15a290 0x7f1e5c15bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:21.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.042+0000 7f1e5bfff700 1 -- 172.21.15.99:0/2233442129 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e48009420 con 0x7f1e5c159ec0 2023-10-08T00:11:21.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.042+0000 7f1e61028700 1 --2- 172.21.15.99:0/2233442129 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e5c15a290 0x7f1e5c15bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:11:21.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.042+0000 7f1e5bfff700 1 --2- 172.21.15.99:0/2233442129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e5c159ec0 0x7f1e5c15b650 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c071eb0 tx=0x7f1e48017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:21.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.042+0000 7f1e58ff9700 1 -- 172.21.15.99:0/2233442129 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4801f070 con 0x7f1e5c159ec0 2023-10-08T00:11:21.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.042+0000 7f1e58ff9700 1 -- 172.21.15.99:0/2233442129 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e48011900 con 0x7f1e5c159ec0 2023-10-08T00:11:21.048 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.042+0000 7f1e62a8b700 1 -- 172.21.15.99:0/2233442129 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e5c15c7d0 con 0x7f1e5c159ec0 2023-10-08T00:11:21.048 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.042+0000 7f1e58ff9700 1 -- 172.21.15.99:0/2233442129 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e48023410 con 0x7f1e5c159ec0 2023-10-08T00:11:21.048 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.043+0000 7f1e62a8b700 1 -- 172.21.15.99:0/2233442129 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e5c15cd90 con 0x7f1e5c159ec0 2023-10-08T00:11:21.048 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.043+0000 7f1e62a8b700 1 -- 172.21.15.99:0/2233442129 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e5c15cf80 con 0x7f1e5c159ec0 2023-10-08T00:11:21.048 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.044+0000 7f1e58ff9700 1 -- 172.21.15.99:0/2233442129 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1e480193b0 con 0x7f1e5c159ec0 2023-10-08T00:11:21.048 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.044+0000 7f1e58ff9700 1 --2- 172.21.15.99:0/2233442129 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1e40066f30 0x7f1e400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:21.048 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.044+0000 7f1e58ff9700 1 -- 172.21.15.99:0/2233442129 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1e480858c0 con 0x7f1e5c159ec0 2023-10-08T00:11:21.048 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.044+0000 7f1e60827700 1 --2- 172.21.15.99:0/2233442129 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1e40066f30 0x7f1e400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:21.048 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.045+0000 7f1e60827700 1 --2- 172.21.15.99:0/2233442129 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1e40066f30 0x7f1e400693e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1e500027d0 tx=0x7f1e50009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:21.049 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.049+0000 7f1e58ff9700 1 -- 172.21.15.99:0/2233442129 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1e48052b30 con 0x7f1e5c159ec0 2023-10-08T00:11:21.232 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.232+0000 7f1e62a8b700 1 -- 172.21.15.99:0/2233442129 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1e5c04c3c0 con 0x7f1e5c159ec0 2023-10-08T00:11:21.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.232+0000 7f1e58ff9700 1 -- 172.21.15.99:0/2233442129 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1e480280a0 con 0x7f1e5c159ec0 2023-10-08T00:11:21.233 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2023-10-08T00:11:21.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.234+0000 7f1e62a8b700 1 -- 172.21.15.99:0/2233442129 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1e40066f30 msgr2=0x7f1e400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:21.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.234+0000 7f1e62a8b700 1 --2- 172.21.15.99:0/2233442129 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1e40066f30 0x7f1e400693e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1e500027d0 tx=0x7f1e50009040 comp rx=0 tx=0).stop 2023-10-08T00:11:21.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.234+0000 7f1e62a8b700 1 -- 172.21.15.99:0/2233442129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e5c159ec0 msgr2=0x7f1e5c15b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:21.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.234+0000 7f1e62a8b700 1 --2- 172.21.15.99:0/2233442129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e5c159ec0 0x7f1e5c15b650 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c071eb0 tx=0x7f1e48017000 comp rx=0 tx=0).stop 2023-10-08T00:11:21.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.234+0000 7f1e62a8b700 1 -- 172.21.15.99:0/2233442129 shutdown_connections 2023-10-08T00:11:21.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.234+0000 7f1e62a8b700 1 --2- 172.21.15.99:0/2233442129 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1e40066f30 0x7f1e400693e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:21.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.234+0000 7f1e62a8b700 1 --2- 172.21.15.99:0/2233442129 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e5c15a290 0x7f1e5c15bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:21.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.234+0000 7f1e62a8b700 1 --2- 172.21.15.99:0/2233442129 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e5c06aae0 0x7f1e5c15dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:21.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.234+0000 7f1e62a8b700 1 --2- 172.21.15.99:0/2233442129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e5c159ec0 0x7f1e5c15b650 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:21.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.234+0000 7f1e62a8b700 1 -- 172.21.15.99:0/2233442129 >> 172.21.15.99:0/2233442129 conn(0x7f1e5c0d1880 msgr2=0x7f1e5c0b0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:21.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.235+0000 7f1e62a8b700 1 -- 172.21.15.99:0/2233442129 shutdown_connections 2023-10-08T00:11:21.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.235+0000 7f1e62a8b700 1 -- 172.21.15.99:0/2233442129 wait complete. 2023-10-08T00:11:21.247 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T00:11:21.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.362+0000 7f019d0a5700 1 Processor -- start 2023-10-08T00:11:21.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.362+0000 7f019d0a5700 1 -- start start 2023-10-08T00:11:21.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.363+0000 7f019d0a5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0198159ec0 0x7f01980af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:21.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.363+0000 7f019d0a5700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01980b0790 con 0x7f019815a290 2023-10-08T00:11:21.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.363+0000 7f019d0a5700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01980ce1f0 con 0x7f019806acb0 2023-10-08T00:11:21.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.363+0000 7f019d0a5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01980b0b20 con 0x7f0198159ec0 2023-10-08T00:11:21.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.363+0000 7f018e59c700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f019815a290 0x7f019806a5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56032/0 (socket says 172.21.15.99:56032) 2023-10-08T00:11:21.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.363+0000 7f018e59c700 1 -- 172.21.15.99:0/1561569156 learned_addr learned my addr 172.21.15.99:0/1561569156 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:21.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.363+0000 7f019659c700 1 -- 172.21.15.99:0/1561569156 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3435955449 0 0) 0x7f01980b0790 con 0x7f019815a290 2023-10-08T00:11:21.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.363+0000 7f0196d9d700 1 --2- 172.21.15.99:0/1561569156 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0198159ec0 0x7f01980af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:21.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.363+0000 7f019659c700 1 -- 172.21.15.99:0/1561569156 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0184003570 con 0x7f019815a290 2023-10-08T00:11:21.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.364+0000 7f019659c700 1 -- 172.21.15.99:0/1561569156 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 923547973 0 0) 0x7f0184003570 con 0x7f019815a290 2023-10-08T00:11:21.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.364+0000 7f019659c700 1 -- 172.21.15.99:0/1561569156 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0198159ec0 msgr2=0x7f01980af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:21.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.364+0000 7f019659c700 1 --2- 172.21.15.99:0/1561569156 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0198159ec0 0x7f01980af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:21.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.364+0000 7f019659c700 1 -- 172.21.15.99:0/1561569156 >> v1:172.21.15.191:6789/0 conn(0x7f019806acb0 legacy=0x7f01980cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:21.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.364+0000 7f019659c700 1 -- 172.21.15.99:0/1561569156 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01980b04b0 con 0x7f019815a290 2023-10-08T00:11:21.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.364+0000 7f019659c700 1 -- 172.21.15.99:0/1561569156 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f0180001470 con 0x7f019815a290 2023-10-08T00:11:21.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.364+0000 7f019659c700 1 -- 172.21.15.99:0/1561569156 >> v1:172.21.15.99:6789/0 conn(0x7f019815a290 legacy=0x7f019806a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:21.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.364+0000 7f019659c700 1 --2- 172.21.15.99:0/1561569156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0184003a90 0x7f0184003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:21.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.364+0000 7f019659c700 1 --2- 172.21.15.99:0/1561569156 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0198159ec0 0x7f0184004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:21.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.364+0000 7f019659c700 1 --2- 172.21.15.99:0/1561569156 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0184004d50 0x7f01840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:21.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.364+0000 7f019659c700 1 -- 172.21.15.99:0/1561569156 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01980b0b20 con 0x7f0184003a90 2023-10-08T00:11:21.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.364+0000 7f019659c700 1 -- 172.21.15.99:0/1561569156 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f018400bb70 con 0x7f0184004d50 2023-10-08T00:11:21.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.364+0000 7f019659c700 1 -- 172.21.15.99:0/1561569156 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f018400bd30 con 0x7f0198159ec0 2023-10-08T00:11:21.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.364+0000 7f0196d9d700 1 --2- 172.21.15.99:0/1561569156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0184003a90 0x7f0184003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:21.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.364+0000 7f018e59c700 1 --2- 172.21.15.99:0/1561569156 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0198159ec0 0x7f0184004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:21.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.364+0000 7f019759e700 1 --2- 172.21.15.99:0/1561569156 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0184004d50 0x7f01840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:21.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.364+0000 7f0196d9d700 1 -- 172.21.15.99:0/1561569156 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0198159ec0 msgr2=0x7f0184004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:21.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.364+0000 7f0196d9d700 1 --2- 172.21.15.99:0/1561569156 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0198159ec0 0x7f0184004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:21.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.364+0000 7f0196d9d700 1 -- 172.21.15.99:0/1561569156 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0184004d50 msgr2=0x7f01840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:21.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.364+0000 7f0196d9d700 1 --2- 172.21.15.99:0/1561569156 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0184004d50 0x7f01840051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:21.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.364+0000 7f0196d9d700 1 -- 172.21.15.99:0/1561569156 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0184003570 con 0x7f0184003a90 2023-10-08T00:11:21.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.364+0000 7f018e59c700 1 --2- 172.21.15.99:0/1561569156 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0198159ec0 0x7f0184004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:11:21.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.364+0000 7f0196d9d700 1 --2- 172.21.15.99:0/1561569156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0184003a90 0x7f0184003ea0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0188009c00 tx=0x7f01880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41fbdbc074420fc7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:21.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.364+0000 7f019659c700 1 -- 172.21.15.99:0/1561569156 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018800e710 con 0x7f0184003a90 2023-10-08T00:11:21.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.364+0000 7f019659c700 1 -- 172.21.15.99:0/1561569156 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f018800e870 con 0x7f0184003a90 2023-10-08T00:11:21.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.364+0000 7f019659c700 1 -- 172.21.15.99:0/1561569156 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018800ec30 con 0x7f0184003a90 2023-10-08T00:11:21.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.365+0000 7f019d0a5700 1 -- 172.21.15.99:0/1561569156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0184003a90 msgr2=0x7f0184003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:21.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.365+0000 7f019d0a5700 1 --2- 172.21.15.99:0/1561569156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0184003a90 0x7f0184003ea0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0188009c00 tx=0x7f01880091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:21.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.365+0000 7f019759e700 1 -- 172.21.15.99:0/1561569156 reap_dead start 2023-10-08T00:11:21.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.365+0000 7f019d0a5700 1 -- 172.21.15.99:0/1561569156 shutdown_connections 2023-10-08T00:11:21.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.365+0000 7f019d0a5700 1 -- 172.21.15.99:0/1561569156 >> 172.21.15.99:0/1561569156 conn(0x7f01980d1880 msgr2=0x7f01980cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:21.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.365+0000 7f019d0a5700 1 -- 172.21.15.99:0/1561569156 shutdown_connections 2023-10-08T00:11:21.371 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.365+0000 7f019d0a5700 1 -- 172.21.15.99:0/1561569156 wait complete. 2023-10-08T00:11:21.371 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.366+0000 7f019d0a5700 1 Processor -- start 2023-10-08T00:11:21.371 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.366+0000 7f019d0a5700 1 -- start start 2023-10-08T00:11:21.373 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.367+0000 7f019d0a5700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0184003a90 0x7f01980c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:21.373 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.367+0000 7f019d0a5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01980cacd0 0x7f01980b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:21.373 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.367+0000 7f019d0a5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01980c8c70 0x7f01980c9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:21.373 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.367+0000 7f019d0a5700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01980c84f0 con 0x7f01980c8c70 2023-10-08T00:11:21.373 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.367+0000 7f019d0a5700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01980cb140 con 0x7f0184003a90 2023-10-08T00:11:21.373 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.367+0000 7f019d0a5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01980c9650 con 0x7f01980cacd0 2023-10-08T00:11:21.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.367+0000 7f019759e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01980c8c70 0x7f01980c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:21.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.367+0000 7f019759e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01980c8c70 0x7f01980c9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55976/0 (socket says 172.21.15.99:55976) 2023-10-08T00:11:21.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.367+0000 7f019759e700 1 -- 172.21.15.99:0/3587868778 learned_addr learned my addr 172.21.15.99:0/3587868778 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:21.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.367+0000 7f018e59c700 1 --2- 172.21.15.99:0/3587868778 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01980cacd0 0x7f01980b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:21.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.367+0000 7f0196d9d700 1 --2- 172.21.15.99:0/3587868778 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0184003a90 0x7f01980c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:21.375 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.367+0000 7f019759e700 1 -- 172.21.15.99:0/3587868778 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01980cacd0 msgr2=0x7f01980b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:21.375 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.367+0000 7f019759e700 1 --2- 172.21.15.99:0/3587868778 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01980cacd0 0x7f01980b2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:21.375 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.367+0000 7f019759e700 1 -- 172.21.15.99:0/3587868778 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0184003a90 msgr2=0x7f01980c7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:21.375 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.367+0000 7f019759e700 1 --2- 172.21.15.99:0/3587868778 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0184003a90 0x7f01980c7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:21.375 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.367+0000 7f019759e700 1 -- 172.21.15.99:0/3587868778 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f019816aa80 con 0x7f01980c8c70 2023-10-08T00:11:21.375 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.368+0000 7f019759e700 1 --2- 172.21.15.99:0/3587868778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01980c8c70 0x7f01980c9110 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f01980d1e60 tx=0x7f0190009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:21.375 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.368+0000 7f018f7fe700 1 -- 172.21.15.99:0/3587868778 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01900032e0 con 0x7f01980c8c70 2023-10-08T00:11:21.375 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.368+0000 7f018f7fe700 1 -- 172.21.15.99:0/3587868778 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0190001580 con 0x7f01980c8c70 2023-10-08T00:11:21.375 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.368+0000 7f019d0a5700 1 -- 172.21.15.99:0/3587868778 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f019816ad60 con 0x7f01980c8c70 2023-10-08T00:11:21.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.368+0000 7f018f7fe700 1 -- 172.21.15.99:0/3587868778 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0190003440 con 0x7f01980c8c70 2023-10-08T00:11:21.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.368+0000 7f019d0a5700 1 -- 172.21.15.99:0/3587868778 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f019816b320 con 0x7f01980c8c70 2023-10-08T00:11:21.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.369+0000 7f019d0a5700 1 -- 172.21.15.99:0/3587868778 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01980cd3a0 con 0x7f01980c8c70 2023-10-08T00:11:21.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.369+0000 7f018f7fe700 1 -- 172.21.15.99:0/3587868778 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0190001bd0 con 0x7f01980c8c70 2023-10-08T00:11:21.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.369+0000 7f018f7fe700 1 --2- 172.21.15.99:0/3587868778 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0178066f30 0x7f01780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:21.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.369+0000 7f018f7fe700 1 -- 172.21.15.99:0/3587868778 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0190083d70 con 0x7f01980c8c70 2023-10-08T00:11:21.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.370+0000 7f0196d9d700 1 --2- 172.21.15.99:0/3587868778 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0178066f30 0x7f01780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:21.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.370+0000 7f0196d9d700 1 --2- 172.21.15.99:0/3587868778 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0178066f30 0x7f01780693e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f01980b3790 tx=0x7f018801a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:21.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.374+0000 7f018f7fe700 1 -- 172.21.15.99:0/3587868778 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0190050fe0 con 0x7f01980c8c70 2023-10-08T00:11:21.552 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.551+0000 7f019d0a5700 1 -- 172.21.15.99:0/3587868778 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f019804c3c0 con 0x7f01980c8c70 2023-10-08T00:11:21.553 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.552+0000 7f018f7fe700 1 -- 172.21.15.99:0/3587868778 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0190054600 con 0x7f01980c8c70 2023-10-08T00:11:21.553 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2023-10-08T00:11:21.554 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.554+0000 7f019d0a5700 1 -- 172.21.15.99:0/3587868778 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0178066f30 msgr2=0x7f01780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:21.555 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.554+0000 7f019d0a5700 1 --2- 172.21.15.99:0/3587868778 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0178066f30 0x7f01780693e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f01980b3790 tx=0x7f018801a000 comp rx=0 tx=0).stop 2023-10-08T00:11:21.555 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.554+0000 7f019d0a5700 1 -- 172.21.15.99:0/3587868778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01980c8c70 msgr2=0x7f01980c9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:21.555 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.554+0000 7f019d0a5700 1 --2- 172.21.15.99:0/3587868778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01980c8c70 0x7f01980c9110 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f01980d1e60 tx=0x7f0190009040 comp rx=0 tx=0).stop 2023-10-08T00:11:21.555 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.554+0000 7f019d0a5700 1 -- 172.21.15.99:0/3587868778 shutdown_connections 2023-10-08T00:11:21.555 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.554+0000 7f019d0a5700 1 --2- 172.21.15.99:0/3587868778 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0178066f30 0x7f01780693e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:21.555 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.554+0000 7f019d0a5700 1 --2- 172.21.15.99:0/3587868778 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0184003a90 0x7f01980c7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:21.555 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.554+0000 7f019d0a5700 1 --2- 172.21.15.99:0/3587868778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01980c8c70 0x7f01980c9110 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:21.555 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.554+0000 7f019d0a5700 1 --2- 172.21.15.99:0/3587868778 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01980cacd0 0x7f01980b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:21.555 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.554+0000 7f019d0a5700 1 -- 172.21.15.99:0/3587868778 >> 172.21.15.99:0/3587868778 conn(0x7f01980d1880 msgr2=0x7f01980b5b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:21.555 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.554+0000 7f019d0a5700 1 -- 172.21.15.99:0/3587868778 shutdown_connections 2023-10-08T00:11:21.555 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.554+0000 7f019d0a5700 1 -- 172.21.15.99:0/3587868778 wait complete. 2023-10-08T00:11:21.566 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T00:11:21.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.684+0000 7f7845560700 1 Processor -- start 2023-10-08T00:11:21.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.684+0000 7f7845560700 1 -- start start 2023-10-08T00:11:21.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.685+0000 7f7845560700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f78400b6580 0x7f784006a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:21.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.685+0000 7f7845560700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f784006a830 0x7f78400b5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:21.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.685+0000 7f7845560700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f784006ac90 con 0x7f78400cce30 2023-10-08T00:11:21.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.685+0000 7f7845560700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78400b4d70 con 0x7f78400b6580 2023-10-08T00:11:21.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.685+0000 7f7845560700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f784015b3b0 con 0x7f784006a830 2023-10-08T00:11:21.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.685+0000 7f783f7fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f78400b6580 0x7f784006a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:21.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.685+0000 7f783ffff700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f78400cce30 0x7f784015aca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56046/0 (socket says 172.21.15.99:56046) 2023-10-08T00:11:21.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.685+0000 7f783ffff700 1 -- 172.21.15.99:0/3555975510 learned_addr learned my addr 172.21.15.99:0/3555975510 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:21.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.685+0000 7f7844d5f700 1 --2- 172.21.15.99:0/3555975510 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f784006a830 0x7f78400b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:21.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.685+0000 7f783f7fe700 1 -- 172.21.15.99:0/3555975510 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f784006a830 msgr2=0x7f78400b5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:21.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.685+0000 7f783f7fe700 1 --2- 172.21.15.99:0/3555975510 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f784006a830 0x7f78400b5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:21.694 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.685+0000 7f783f7fe700 1 -- 172.21.15.99:0/3555975510 >> v1:172.21.15.99:6789/0 conn(0x7f78400cce30 legacy=0x7f784015aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:21.694 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.686+0000 7f783f7fe700 1 -- 172.21.15.99:0/3555975510 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78400cd8d0 con 0x7f78400b6580 2023-10-08T00:11:21.694 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.686+0000 7f783effd700 1 -- 172.21.15.99:0/3555975510 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 677510735 0 0) 0x7f7830001110 con 0x7f78400cce30 2023-10-08T00:11:21.694 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.686+0000 7f7844d5f700 1 --2- 172.21.15.99:0/3555975510 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f784006a830 0x7f78400b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:11:21.694 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.686+0000 7f783f7fe700 1 --2- 172.21.15.99:0/3555975510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78400b6580 0x7f784006a2f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f78340098e0 tx=0x7f78340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11692915d86d3ae server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:21.694 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.686+0000 7f783effd700 1 -- 172.21.15.99:0/3555975510 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7834009e80 con 0x7f78400b6580 2023-10-08T00:11:21.694 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.686+0000 7f783effd700 1 -- 172.21.15.99:0/3555975510 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f783400b860 con 0x7f78400b6580 2023-10-08T00:11:21.694 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.686+0000 7f783effd700 1 -- 172.21.15.99:0/3555975510 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f783400bbe0 con 0x7f78400b6580 2023-10-08T00:11:21.694 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.686+0000 7f7845560700 1 -- 172.21.15.99:0/3555975510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78400b6580 msgr2=0x7f784006a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:21.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.686+0000 7f7845560700 1 --2- 172.21.15.99:0/3555975510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78400b6580 0x7f784006a2f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f78340098e0 tx=0x7f78340091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:21.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.686+0000 7f7845560700 1 -- 172.21.15.99:0/3555975510 shutdown_connections 2023-10-08T00:11:21.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.686+0000 7f7845560700 1 --2- 172.21.15.99:0/3555975510 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f784006a830 0x7f78400b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:21.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.686+0000 7f7845560700 1 --2- 172.21.15.99:0/3555975510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78400b6580 0x7f784006a2f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:21.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.686+0000 7f7845560700 1 -- 172.21.15.99:0/3555975510 >> 172.21.15.99:0/3555975510 conn(0x7f7840068cb0 msgr2=0x7f78400690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:21.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.686+0000 7f7845560700 1 -- 172.21.15.99:0/3555975510 shutdown_connections 2023-10-08T00:11:21.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.686+0000 7f7845560700 1 -- 172.21.15.99:0/3555975510 wait complete. 2023-10-08T00:11:21.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.687+0000 7f7845560700 1 Processor -- start 2023-10-08T00:11:21.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.688+0000 7f7845560700 1 -- start start 2023-10-08T00:11:21.696 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.688+0000 7f7845560700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f784006a830 0x7f7840157390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:21.696 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.688+0000 7f7845560700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78400b6580 0x7f784014e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:21.696 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.688+0000 7f7845560700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f78400cce30 0x7f784015e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:21.696 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.688+0000 7f7845560700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7840156f00 con 0x7f78400b6580 2023-10-08T00:11:21.696 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.688+0000 7f7845560700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78401562f0 con 0x7f78400cce30 2023-10-08T00:11:21.696 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.688+0000 7f7845560700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78401565d0 con 0x7f784006a830 2023-10-08T00:11:21.696 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.688+0000 7f783f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78400b6580 0x7f784014e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:21.696 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.688+0000 7f783f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78400b6580 0x7f784014e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56000/0 (socket says 172.21.15.99:56000) 2023-10-08T00:11:21.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.688+0000 7f783f7fe700 1 -- 172.21.15.99:0/1325250314 learned_addr learned my addr 172.21.15.99:0/1325250314 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:21.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.689+0000 7f783f7fe700 1 -- 172.21.15.99:0/1325250314 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f784006a830 msgr2=0x7f7840157390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:21.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.689+0000 7f783f7fe700 1 --2- 172.21.15.99:0/1325250314 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f784006a830 0x7f7840157390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:21.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.689+0000 7f783f7fe700 1 -- 172.21.15.99:0/1325250314 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f78400cce30 msgr2=0x7f784015e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:21.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.689+0000 7f783f7fe700 1 --2- 172.21.15.99:0/1325250314 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f78400cce30 0x7f784015e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:21.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.689+0000 7f783f7fe700 1 -- 172.21.15.99:0/1325250314 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7834009420 con 0x7f78400b6580 2023-10-08T00:11:21.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.689+0000 7f783f7fe700 1 --2- 172.21.15.99:0/1325250314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78400b6580 0x7f784014e370 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f784006b300 tx=0x7f7834017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:21.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.689+0000 7f7827fff700 1 -- 172.21.15.99:0/1325250314 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f783400bbe0 con 0x7f78400b6580 2023-10-08T00:11:21.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.689+0000 7f7827fff700 1 -- 172.21.15.99:0/1325250314 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7834011420 con 0x7f78400b6580 2023-10-08T00:11:21.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.689+0000 7f7845560700 1 -- 172.21.15.99:0/1325250314 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78401568b0 con 0x7f78400b6580 2023-10-08T00:11:21.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.689+0000 7f7827fff700 1 -- 172.21.15.99:0/1325250314 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78340117c0 con 0x7f78400b6580 2023-10-08T00:11:21.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.689+0000 7f7845560700 1 -- 172.21.15.99:0/1325250314 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f784014e8b0 con 0x7f78400b6580 2023-10-08T00:11:21.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.690+0000 7f7845560700 1 -- 172.21.15.99:0/1325250314 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7840089550 con 0x7f78400b6580 2023-10-08T00:11:21.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.690+0000 7f7827fff700 1 -- 172.21.15.99:0/1325250314 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7834011bb0 con 0x7f78400b6580 2023-10-08T00:11:21.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.691+0000 7f7827fff700 1 --2- 172.21.15.99:0/1325250314 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f782c066f30 0x7f782c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:21.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.691+0000 7f7827fff700 1 -- 172.21.15.99:0/1325250314 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7834085d30 con 0x7f78400b6580 2023-10-08T00:11:21.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.691+0000 7f783ffff700 1 --2- 172.21.15.99:0/1325250314 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f782c066f30 0x7f782c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:21.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.691+0000 7f783ffff700 1 --2- 172.21.15.99:0/1325250314 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f782c066f30 0x7f782c0693e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7830001640 tx=0x7f7830007b60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:21.699 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.695+0000 7f7827fff700 1 -- 172.21.15.99:0/1325250314 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7834052f00 con 0x7f78400b6580 2023-10-08T00:11:21.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.875+0000 7f7845560700 1 -- 172.21.15.99:0/1325250314 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7840157040 con 0x7f78400b6580 2023-10-08T00:11:21.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.876+0000 7f7827fff700 1 -- 172.21.15.99:0/1325250314 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7834011e60 con 0x7f78400b6580 2023-10-08T00:11:21.877 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2023-10-08T00:11:21.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.878+0000 7f7845560700 1 -- 172.21.15.99:0/1325250314 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f782c066f30 msgr2=0x7f782c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:21.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.878+0000 7f7845560700 1 --2- 172.21.15.99:0/1325250314 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f782c066f30 0x7f782c0693e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7830001640 tx=0x7f7830007b60 comp rx=0 tx=0).stop 2023-10-08T00:11:21.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.878+0000 7f7845560700 1 -- 172.21.15.99:0/1325250314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78400b6580 msgr2=0x7f784014e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:21.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.878+0000 7f7845560700 1 --2- 172.21.15.99:0/1325250314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78400b6580 0x7f784014e370 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f784006b300 tx=0x7f7834017000 comp rx=0 tx=0).stop 2023-10-08T00:11:21.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.878+0000 7f7845560700 1 -- 172.21.15.99:0/1325250314 shutdown_connections 2023-10-08T00:11:21.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.878+0000 7f7845560700 1 --2- 172.21.15.99:0/1325250314 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f782c066f30 0x7f782c0693e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:21.880 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.878+0000 7f7845560700 1 --2- 172.21.15.99:0/1325250314 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f78400cce30 0x7f784015e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:21.880 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.878+0000 7f7845560700 1 --2- 172.21.15.99:0/1325250314 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f784006a830 0x7f7840157390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:21.880 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.878+0000 7f7845560700 1 --2- 172.21.15.99:0/1325250314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78400b6580 0x7f784014e370 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:21.880 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.878+0000 7f7845560700 1 -- 172.21.15.99:0/1325250314 >> 172.21.15.99:0/1325250314 conn(0x7f7840068cb0 msgr2=0x7f784015ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:21.880 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.878+0000 7f7845560700 1 -- 172.21.15.99:0/1325250314 shutdown_connections 2023-10-08T00:11:21.880 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:21.878+0000 7f7845560700 1 -- 172.21.15.99:0/1325250314 wait complete. 2023-10-08T00:11:21.891 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:11:22.005 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.004+0000 7ff155621700 1 Processor -- start 2023-10-08T00:11:22.005 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.004+0000 7ff155621700 1 -- start start 2023-10-08T00:11:22.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.005+0000 7ff155621700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff15015a620 0x7ff15015d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:22.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.005+0000 7ff155621700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff15015cbb0 con 0x7ff150157870 2023-10-08T00:11:22.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.005+0000 7ff155621700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff15015bfa0 con 0x7ff15015d580 2023-10-08T00:11:22.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.005+0000 7ff155621700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff15015bdb0 con 0x7ff15015a620 2023-10-08T00:11:22.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.005+0000 7ff14effd700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7ff150157870 0x7ff15015b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56048/0 (socket says 172.21.15.99:56048) 2023-10-08T00:11:22.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.005+0000 7ff14effd700 1 -- 172.21.15.99:0/2384000635 learned_addr learned my addr 172.21.15.99:0/2384000635 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:22.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.006+0000 7ff14e7fc700 1 -- 172.21.15.99:0/2384000635 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 534297120 0 0) 0x7ff15015cbb0 con 0x7ff150157870 2023-10-08T00:11:22.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.006+0000 7ff147fff700 1 --2- 172.21.15.99:0/2384000635 >> v2:172.21.15.191:3300/0 conn(0x7ff15015a620 0x7ff15015d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:22.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.006+0000 7ff14e7fc700 1 -- 172.21.15.99:0/2384000635 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff13c003570 con 0x7ff150157870 2023-10-08T00:11:22.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.006+0000 7ff14e7fc700 1 -- 172.21.15.99:0/2384000635 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3322622645 0 0) 0x7ff13c003570 con 0x7ff150157870 2023-10-08T00:11:22.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.006+0000 7ff14e7fc700 1 -- 172.21.15.99:0/2384000635 >> v2:172.21.15.191:3300/0 conn(0x7ff15015a620 msgr2=0x7ff15015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:22.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.006+0000 7ff14e7fc700 1 --2- 172.21.15.99:0/2384000635 >> v2:172.21.15.191:3300/0 conn(0x7ff15015a620 0x7ff15015d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:22.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.006+0000 7ff14e7fc700 1 -- 172.21.15.99:0/2384000635 >> v1:172.21.15.191:6789/0 conn(0x7ff15015d580 legacy=0x7ff150158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:22.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.006+0000 7ff14e7fc700 1 -- 172.21.15.99:0/2384000635 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff15015c280 con 0x7ff150157870 2023-10-08T00:11:22.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.006+0000 7ff14e7fc700 1 -- 172.21.15.99:0/2384000635 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7ff140001470 con 0x7ff150157870 2023-10-08T00:11:22.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.006+0000 7ff14e7fc700 1 -- 172.21.15.99:0/2384000635 >> v1:172.21.15.99:6789/0 conn(0x7ff150157870 legacy=0x7ff15015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:22.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.006+0000 7ff14e7fc700 1 --2- 172.21.15.99:0/2384000635 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff13c003a60 0x7ff13c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:22.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.006+0000 7ff14e7fc700 1 --2- 172.21.15.99:0/2384000635 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff13c004810 0x7ff13c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:22.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.006+0000 7ff14e7fc700 1 --2- 172.21.15.99:0/2384000635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff13c0051c0 0x7ff13c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:22.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.006+0000 7ff14e7fc700 1 -- 172.21.15.99:0/2384000635 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13c00c130 con 0x7ff13c0051c0 2023-10-08T00:11:22.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.006+0000 7ff14e7fc700 1 -- 172.21.15.99:0/2384000635 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13c00c2a0 con 0x7ff13c003a60 2023-10-08T00:11:22.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.006+0000 7ff14e7fc700 1 -- 172.21.15.99:0/2384000635 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff13c00c460 con 0x7ff13c004810 2023-10-08T00:11:22.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.006+0000 7ff14f7fe700 1 --2- 172.21.15.99:0/2384000635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff13c0051c0 0x7ff13c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:22.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.006+0000 7ff14effd700 1 --2- 172.21.15.99:0/2384000635 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff13c003a60 0x7ff13c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:22.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.006+0000 7ff147fff700 1 --2- 172.21.15.99:0/2384000635 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff13c004810 0x7ff13c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:22.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.006+0000 7ff14f7fe700 1 -- 172.21.15.99:0/2384000635 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff13c004810 msgr2=0x7ff13c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:22.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.006+0000 7ff14f7fe700 1 --2- 172.21.15.99:0/2384000635 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff13c004810 0x7ff13c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:22.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.006+0000 7ff14f7fe700 1 -- 172.21.15.99:0/2384000635 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff13c003a60 msgr2=0x7ff13c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:22.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.006+0000 7ff14f7fe700 1 --2- 172.21.15.99:0/2384000635 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff13c003a60 0x7ff13c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:22.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.006+0000 7ff14f7fe700 1 -- 172.21.15.99:0/2384000635 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff13c003570 con 0x7ff13c0051c0 2023-10-08T00:11:22.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.006+0000 7ff14f7fe700 1 -- 172.21.15.99:0/2384000635 reap_dead start 2023-10-08T00:11:22.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.007+0000 7ff14f7fe700 1 --2- 172.21.15.99:0/2384000635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff13c0051c0 0x7ff13c00ba60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff148001f90 tx=0x7ff148001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d34e370e1e9244 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:22.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.007+0000 7ff14e7fc700 1 -- 172.21.15.99:0/2384000635 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff148001950 con 0x7ff13c0051c0 2023-10-08T00:11:22.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.007+0000 7ff14e7fc700 1 -- 172.21.15.99:0/2384000635 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff14800c8b0 con 0x7ff13c0051c0 2023-10-08T00:11:22.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.007+0000 7ff14e7fc700 1 -- 172.21.15.99:0/2384000635 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14800cc50 con 0x7ff13c0051c0 2023-10-08T00:11:22.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.007+0000 7ff155621700 1 -- 172.21.15.99:0/2384000635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff13c0051c0 msgr2=0x7ff13c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:22.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.007+0000 7ff155621700 1 --2- 172.21.15.99:0/2384000635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff13c0051c0 0x7ff13c00ba60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff148001f90 tx=0x7ff148001640 comp rx=0 tx=0).stop 2023-10-08T00:11:22.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.007+0000 7ff155621700 1 -- 172.21.15.99:0/2384000635 shutdown_connections 2023-10-08T00:11:22.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.007+0000 7ff155621700 1 --2- 172.21.15.99:0/2384000635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff13c0051c0 0x7ff13c00ba60 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:22.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.007+0000 7ff155621700 1 -- 172.21.15.99:0/2384000635 >> 172.21.15.99:0/2384000635 conn(0x7ff150068ce0 msgr2=0x7ff1500690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:22.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.007+0000 7ff155621700 1 -- 172.21.15.99:0/2384000635 shutdown_connections 2023-10-08T00:11:22.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.007+0000 7ff155621700 1 -- 172.21.15.99:0/2384000635 wait complete. 2023-10-08T00:11:22.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.008+0000 7ff155621700 1 Processor -- start 2023-10-08T00:11:22.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.009+0000 7ff155621700 1 -- start start 2023-10-08T00:11:22.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.009+0000 7ff155621700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff13c0051c0 0x7ff1500d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:22.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.009+0000 7ff155621700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff15014b1b0 0x7ff15014b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:22.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.009+0000 7ff155621700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff15016acd0 0x7ff150169ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:22.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.009+0000 7ff155621700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1500b4da0 con 0x7ff15016acd0 2023-10-08T00:11:22.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.009+0000 7ff155621700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1500b5360 con 0x7ff15014b1b0 2023-10-08T00:11:22.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.009+0000 7ff155621700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1500d11e0 con 0x7ff13c0051c0 2023-10-08T00:11:22.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.009+0000 7ff14f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff15016acd0 0x7ff150169ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:22.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.009+0000 7ff14f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff15016acd0 0x7ff150169ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56024/0 (socket says 172.21.15.99:56024) 2023-10-08T00:11:22.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.009+0000 7ff14f7fe700 1 -- 172.21.15.99:0/2982225310 learned_addr learned my addr 172.21.15.99:0/2982225310 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:22.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.009+0000 7ff14effd700 1 --2- 172.21.15.99:0/2982225310 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff13c0051c0 0x7ff1500d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:22.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.010+0000 7ff147fff700 1 --2- 172.21.15.99:0/2982225310 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff15014b1b0 0x7ff15014b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:22.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.010+0000 7ff14f7fe700 1 -- 172.21.15.99:0/2982225310 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff13c0051c0 msgr2=0x7ff1500d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:22.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.010+0000 7ff14f7fe700 1 --2- 172.21.15.99:0/2982225310 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff13c0051c0 0x7ff1500d0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:22.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.010+0000 7ff14f7fe700 1 -- 172.21.15.99:0/2982225310 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff15014b1b0 msgr2=0x7ff15014b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:22.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.010+0000 7ff14f7fe700 1 --2- 172.21.15.99:0/2982225310 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff15014b1b0 0x7ff15014b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:22.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.010+0000 7ff14f7fe700 1 -- 172.21.15.99:0/2982225310 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1500d14c0 con 0x7ff15016acd0 2023-10-08T00:11:22.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.010+0000 7ff14effd700 1 --2- 172.21.15.99:0/2982225310 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff13c0051c0 0x7ff1500d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:11:22.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.010+0000 7ff14f7fe700 1 --2- 172.21.15.99:0/2982225310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff15016acd0 0x7ff150169ff0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff1500b03b0 tx=0x7ff148015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:22.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.010+0000 7ff146ffd700 1 -- 172.21.15.99:0/2982225310 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14800c6d0 con 0x7ff15016acd0 2023-10-08T00:11:22.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.010+0000 7ff146ffd700 1 -- 172.21.15.99:0/2982225310 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff14801d420 con 0x7ff15016acd0 2023-10-08T00:11:22.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.010+0000 7ff155621700 1 -- 172.21.15.99:0/2982225310 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1500b4ff0 con 0x7ff15016acd0 2023-10-08T00:11:22.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.010+0000 7ff146ffd700 1 -- 172.21.15.99:0/2982225310 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14801d800 con 0x7ff15016acd0 2023-10-08T00:11:22.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.010+0000 7ff155621700 1 -- 172.21.15.99:0/2982225310 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff15016a530 con 0x7ff15016acd0 2023-10-08T00:11:22.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.011+0000 7ff155621700 1 -- 172.21.15.99:0/2982225310 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1500b5510 con 0x7ff15016acd0 2023-10-08T00:11:22.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.011+0000 7ff146ffd700 1 -- 172.21.15.99:0/2982225310 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff14801dc30 con 0x7ff15016acd0 2023-10-08T00:11:22.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.011+0000 7ff146ffd700 1 --2- 172.21.15.99:0/2982225310 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff130066f30 0x7ff1300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:22.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.012+0000 7ff146ffd700 1 -- 172.21.15.99:0/2982225310 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff148085b30 con 0x7ff15016acd0 2023-10-08T00:11:22.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.012+0000 7ff14effd700 1 --2- 172.21.15.99:0/2982225310 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff130066f30 0x7ff1300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:22.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.012+0000 7ff14effd700 1 --2- 172.21.15.99:0/2982225310 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff130066f30 0x7ff1300693e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff1400015a0 tx=0x7ff140009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:22.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.016+0000 7ff146ffd700 1 -- 172.21.15.99:0/2982225310 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff1500b5510 con 0x7ff15016acd0 2023-10-08T00:11:22.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.197+0000 7ff155621700 1 -- 172.21.15.99:0/2982225310 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff1500b54d0 con 0x7ff15016acd0 2023-10-08T00:11:22.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.197+0000 7ff146ffd700 1 -- 172.21.15.99:0/2982225310 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15350 (secure 0 0 0) 0x7ff148017020 con 0x7ff15016acd0 2023-10-08T00:11:22.198 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:11:22.199 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":20,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:10:55.080854+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:10:51.822157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T00:10:52.415130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:11:22.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.199+0000 7ff155621700 1 -- 172.21.15.99:0/2982225310 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff130066f30 msgr2=0x7ff1300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:22.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.199+0000 7ff155621700 1 --2- 172.21.15.99:0/2982225310 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff130066f30 0x7ff1300693e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff1400015a0 tx=0x7ff140009040 comp rx=0 tx=0).stop 2023-10-08T00:11:22.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.200+0000 7ff155621700 1 -- 172.21.15.99:0/2982225310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff15016acd0 msgr2=0x7ff150169ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:22.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.200+0000 7ff155621700 1 --2- 172.21.15.99:0/2982225310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff15016acd0 0x7ff150169ff0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff1500b03b0 tx=0x7ff148015000 comp rx=0 tx=0).stop 2023-10-08T00:11:22.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.200+0000 7ff155621700 1 -- 172.21.15.99:0/2982225310 shutdown_connections 2023-10-08T00:11:22.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.200+0000 7ff155621700 1 --2- 172.21.15.99:0/2982225310 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff130066f30 0x7ff1300693e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:22.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.200+0000 7ff155621700 1 --2- 172.21.15.99:0/2982225310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff15016acd0 0x7ff150169ff0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:22.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.200+0000 7ff155621700 1 --2- 172.21.15.99:0/2982225310 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff13c0051c0 0x7ff1500d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:22.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.200+0000 7ff155621700 1 --2- 172.21.15.99:0/2982225310 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff15014b1b0 0x7ff15014b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:22.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.200+0000 7ff155621700 1 -- 172.21.15.99:0/2982225310 >> 172.21.15.99:0/2982225310 conn(0x7ff150068ce0 msgr2=0x7ff150158840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:22.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.200+0000 7ff155621700 1 -- 172.21.15.99:0/2982225310 shutdown_connections 2023-10-08T00:11:22.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.200+0000 7ff155621700 1 -- 172.21.15.99:0/2982225310 wait complete. 2023-10-08T00:11:22.213 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:10:42.063344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T00:10:51.822157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T00:10:52.415130+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T00:11:22.214 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T00:11:22.333 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.333+0000 7fdc8198d700 1 Processor -- start 2023-10-08T00:11:22.333 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.333+0000 7fdc8198d700 1 -- start start 2023-10-08T00:11:22.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.333+0000 7fdc8198d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fdc7c070700 0x7fdc7c15c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:22.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.333+0000 7fdc8198d700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdc7c070ad0 0x7fdc7c157f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:22.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.333+0000 7fdc8198d700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc7c067700 con 0x7fdc7c1584d0 2023-10-08T00:11:22.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.333+0000 7fdc8198d700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc7c15c770 con 0x7fdc7c070ad0 2023-10-08T00:11:22.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.333+0000 7fdc8198d700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc7c158940 con 0x7fdc7c070700 2023-10-08T00:11:22.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.334+0000 7fdc727fc700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdc7c070ad0 0x7fdc7c157f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:22.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.334+0000 7fdc727fc700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdc7c070ad0 0x7fdc7c157f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56040/0 (socket says 172.21.15.99:56040) 2023-10-08T00:11:22.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.334+0000 7fdc727fc700 1 -- 172.21.15.99:0/3047693080 learned_addr learned my addr 172.21.15.99:0/3047693080 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:22.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.334+0000 7fdc7affd700 1 --2- 172.21.15.99:0/3047693080 >> v2:172.21.15.191:3300/0 conn(0x7fdc7c070700 0x7fdc7c15c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:22.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.334+0000 7fdc727fc700 1 -- 172.21.15.99:0/3047693080 >> v2:172.21.15.191:3300/0 conn(0x7fdc7c070700 msgr2=0x7fdc7c15c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:22.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.334+0000 7fdc727fc700 1 --2- 172.21.15.99:0/3047693080 >> v2:172.21.15.191:3300/0 conn(0x7fdc7c070700 0x7fdc7c15c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:22.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.334+0000 7fdc727fc700 1 -- 172.21.15.99:0/3047693080 >> v1:172.21.15.191:6789/0 conn(0x7fdc7c1584d0 legacy=0x7fdc7c06e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:22.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.334+0000 7fdc727fc700 1 -- 172.21.15.99:0/3047693080 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc7c158c20 con 0x7fdc7c070ad0 2023-10-08T00:11:22.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.334+0000 7fdc7affd700 1 --2- 172.21.15.99:0/3047693080 >> v2:172.21.15.191:3300/0 conn(0x7fdc7c070700 0x7fdc7c15c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:11:22.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.334+0000 7fdc727fc700 1 --2- 172.21.15.99:0/3047693080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc7c070ad0 0x7fdc7c157f90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c069af0 tx=0x7fdc64009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e77a4099f8693a17 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:22.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.334+0000 7fdc7a7fc700 1 -- 172.21.15.99:0/3047693080 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6400b860 con 0x7fdc7c070ad0 2023-10-08T00:11:22.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.334+0000 7fdc7a7fc700 1 -- 172.21.15.99:0/3047693080 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc6400b9c0 con 0x7fdc7c070ad0 2023-10-08T00:11:22.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.334+0000 7fdc7a7fc700 1 -- 172.21.15.99:0/3047693080 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6400bd60 con 0x7fdc7c070ad0 2023-10-08T00:11:22.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.335+0000 7fdc8198d700 1 -- 172.21.15.99:0/3047693080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc7c070ad0 msgr2=0x7fdc7c157f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:22.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.335+0000 7fdc8198d700 1 --2- 172.21.15.99:0/3047693080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc7c070ad0 0x7fdc7c157f90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c069af0 tx=0x7fdc64009250 comp rx=0 tx=0).stop 2023-10-08T00:11:22.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.335+0000 7fdc8198d700 1 -- 172.21.15.99:0/3047693080 shutdown_connections 2023-10-08T00:11:22.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.335+0000 7fdc8198d700 1 --2- 172.21.15.99:0/3047693080 >> v2:172.21.15.191:3300/0 conn(0x7fdc7c070700 0x7fdc7c15c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:22.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.335+0000 7fdc8198d700 1 --2- 172.21.15.99:0/3047693080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc7c070ad0 0x7fdc7c157f90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:22.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.335+0000 7fdc8198d700 1 -- 172.21.15.99:0/3047693080 >> 172.21.15.99:0/3047693080 conn(0x7fdc7c0ce4d0 msgr2=0x7fdc7c0ca4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:22.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.335+0000 7fdc8198d700 1 -- 172.21.15.99:0/3047693080 shutdown_connections 2023-10-08T00:11:22.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.335+0000 7fdc8198d700 1 -- 172.21.15.99:0/3047693080 wait complete. 2023-10-08T00:11:22.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.336+0000 7fdc8198d700 1 Processor -- start 2023-10-08T00:11:22.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.336+0000 7fdc8198d700 1 -- start start 2023-10-08T00:11:22.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.337+0000 7fdc8198d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc7c070700 0x7fdc7c1691f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:22.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.337+0000 7fdc8198d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdc7c070ad0 0x7fdc7c0b2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:22.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.337+0000 7fdc8198d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc7c1584d0 0x7fdc7c0c68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:22.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.337+0000 7fdc8198d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7c0b1850 con 0x7fdc7c1584d0 2023-10-08T00:11:22.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.337+0000 7fdc8198d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7c168bc0 con 0x7fdc7c070700 2023-10-08T00:11:22.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.337+0000 7fdc8198d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7c0c7420 con 0x7fdc7c070ad0 2023-10-08T00:11:22.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.337+0000 7fdc7b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc7c1584d0 0x7fdc7c0c68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:22.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.337+0000 7fdc7b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc7c1584d0 0x7fdc7c0c68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56044/0 (socket says 172.21.15.99:56044) 2023-10-08T00:11:22.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.337+0000 7fdc7b7fe700 1 -- 172.21.15.99:0/1387812968 learned_addr learned my addr 172.21.15.99:0/1387812968 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:22.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.337+0000 7fdc727fc700 1 --2- 172.21.15.99:0/1387812968 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdc7c070ad0 0x7fdc7c0b2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:22.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.337+0000 7fdc7affd700 1 --2- 172.21.15.99:0/1387812968 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc7c070700 0x7fdc7c1691f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:22.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.337+0000 7fdc7b7fe700 1 -- 172.21.15.99:0/1387812968 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdc7c070ad0 msgr2=0x7fdc7c0b2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:22.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.337+0000 7fdc7b7fe700 1 --2- 172.21.15.99:0/1387812968 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdc7c070ad0 0x7fdc7c0b2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:22.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.337+0000 7fdc7b7fe700 1 -- 172.21.15.99:0/1387812968 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc7c070700 msgr2=0x7fdc7c1691f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:22.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.337+0000 7fdc7b7fe700 1 --2- 172.21.15.99:0/1387812968 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc7c070700 0x7fdc7c1691f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:22.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.337+0000 7fdc7b7fe700 1 -- 172.21.15.99:0/1387812968 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc640094f0 con 0x7fdc7c1584d0 2023-10-08T00:11:22.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.337+0000 7fdc7b7fe700 1 --2- 172.21.15.99:0/1387812968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc7c1584d0 0x7fdc7c0c68b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fdc7400c840 tx=0x7fdc74001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:22.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.337+0000 7fdc737fe700 1 -- 172.21.15.99:0/1387812968 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc74012630 con 0x7fdc7c1584d0 2023-10-08T00:11:22.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.337+0000 7fdc737fe700 1 -- 172.21.15.99:0/1387812968 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc740127b0 con 0x7fdc7c1584d0 2023-10-08T00:11:22.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.337+0000 7fdc8198d700 1 -- 172.21.15.99:0/1387812968 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc7c0c7700 con 0x7fdc7c1584d0 2023-10-08T00:11:22.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.337+0000 7fdc737fe700 1 -- 172.21.15.99:0/1387812968 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc74012b70 con 0x7fdc7c1584d0 2023-10-08T00:11:22.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.338+0000 7fdc8198d700 1 -- 172.21.15.99:0/1387812968 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc7c0b1d40 con 0x7fdc7c1584d0 2023-10-08T00:11:22.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.338+0000 7fdc8198d700 1 -- 172.21.15.99:0/1387812968 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc7c04c3c0 con 0x7fdc7c1584d0 2023-10-08T00:11:22.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.339+0000 7fdc737fe700 1 -- 172.21.15.99:0/1387812968 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdc7401a3a0 con 0x7fdc7c1584d0 2023-10-08T00:11:22.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.339+0000 7fdc737fe700 1 --2- 172.21.15.99:0/1387812968 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdc5c066f30 0x7fdc5c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:22.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.339+0000 7fdc737fe700 1 -- 172.21.15.99:0/1387812968 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdc74084660 con 0x7fdc7c1584d0 2023-10-08T00:11:22.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.339+0000 7fdc7affd700 1 --2- 172.21.15.99:0/1387812968 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdc5c066f30 0x7fdc5c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:22.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.340+0000 7fdc7affd700 1 --2- 172.21.15.99:0/1387812968 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdc5c066f30 0x7fdc5c0693e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c006f00 tx=0x7fdc6c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:22.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.343+0000 7fdc737fe700 1 -- 172.21.15.99:0/1387812968 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdc740518d0 con 0x7fdc7c1584d0 2023-10-08T00:11:22.523 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.522+0000 7fdc8198d700 1 -- 172.21.15.99:0/1387812968 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdc7c0b1f30 con 0x7fdc7c1584d0 2023-10-08T00:11:22.523 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.523+0000 7fdc737fe700 1 -- 172.21.15.99:0/1387812968 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdc74054ef0 con 0x7fdc7c1584d0 2023-10-08T00:11:22.524 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2023-10-08T00:11:22.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.524+0000 7fdc8198d700 1 -- 172.21.15.99:0/1387812968 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdc5c066f30 msgr2=0x7fdc5c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:22.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.524+0000 7fdc8198d700 1 --2- 172.21.15.99:0/1387812968 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdc5c066f30 0x7fdc5c0693e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c006f00 tx=0x7fdc6c008040 comp rx=0 tx=0).stop 2023-10-08T00:11:22.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.524+0000 7fdc8198d700 1 -- 172.21.15.99:0/1387812968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc7c1584d0 msgr2=0x7fdc7c0c68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:22.526 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.524+0000 7fdc8198d700 1 --2- 172.21.15.99:0/1387812968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc7c1584d0 0x7fdc7c0c68b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fdc7400c840 tx=0x7fdc74001670 comp rx=0 tx=0).stop 2023-10-08T00:11:22.526 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.524+0000 7fdc8198d700 1 -- 172.21.15.99:0/1387812968 shutdown_connections 2023-10-08T00:11:22.526 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.524+0000 7fdc8198d700 1 --2- 172.21.15.99:0/1387812968 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdc5c066f30 0x7fdc5c0693e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:22.526 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.524+0000 7fdc8198d700 1 --2- 172.21.15.99:0/1387812968 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc7c070700 0x7fdc7c1691f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:22.526 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.524+0000 7fdc8198d700 1 --2- 172.21.15.99:0/1387812968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc7c1584d0 0x7fdc7c0c68b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:22.526 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.524+0000 7fdc8198d700 1 --2- 172.21.15.99:0/1387812968 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdc7c070ad0 0x7fdc7c0b2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:22.526 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.524+0000 7fdc8198d700 1 -- 172.21.15.99:0/1387812968 >> 172.21.15.99:0/1387812968 conn(0x7fdc7c0ce4d0 msgr2=0x7fdc7c068980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:22.526 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.524+0000 7fdc8198d700 1 -- 172.21.15.99:0/1387812968 shutdown_connections 2023-10-08T00:11:22.526 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.524+0000 7fdc8198d700 1 -- 172.21.15.99:0/1387812968 wait complete. 2023-10-08T00:11:22.537 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T00:11:22.624 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:11:22.626 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:11:22.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.655+0000 7f1a94e61700 1 Processor -- start 2023-10-08T00:11:22.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.655+0000 7f1a94e61700 1 -- start start 2023-10-08T00:11:22.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.656+0000 7f1a94e61700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1a900c9e60 0x7f1a9015afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:22.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.656+0000 7f1a94e61700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a900ca230 0x7f1a9015da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:22.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.656+0000 7f1a94e61700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a9015ab40 con 0x7f1a9015dfc0 2023-10-08T00:11:22.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.656+0000 7f1a94e61700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a9015e500 con 0x7f1a900c9e60 2023-10-08T00:11:22.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.656+0000 7f1a94e61700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9015e7e0 con 0x7f1a900ca230 2023-10-08T00:11:22.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.656+0000 7f1a8e59c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1a900c9e60 0x7f1a9015afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:22.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.656+0000 7f1a8ed9d700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f1a9015dfc0 0x7f1a90153e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56064/0 (socket says 172.21.15.99:56064) 2023-10-08T00:11:22.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.656+0000 7f1a8ed9d700 1 -- 172.21.15.99:0/2721659584 learned_addr learned my addr 172.21.15.99:0/2721659584 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:22.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.656+0000 7f1a85d9b700 1 --2- 172.21.15.99:0/2721659584 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a900ca230 0x7f1a9015da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:22.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.656+0000 7f1a8e59c700 1 -- 172.21.15.99:0/2721659584 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a900ca230 msgr2=0x7f1a9015da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:22.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.656+0000 7f1a8e59c700 1 --2- 172.21.15.99:0/2721659584 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a900ca230 0x7f1a9015da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:22.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.656+0000 7f1a8e59c700 1 -- 172.21.15.99:0/2721659584 >> v1:172.21.15.99:6789/0 conn(0x7f1a9015dfc0 legacy=0x7f1a90153e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:22.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.656+0000 7f1a8e59c700 1 -- 172.21.15.99:0/2721659584 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a9015a7d0 con 0x7f1a900c9e60 2023-10-08T00:11:22.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.656+0000 7f1a8dd9b700 1 -- 172.21.15.99:0/2721659584 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2680856549 0 0) 0x7f1a9015ab40 con 0x7f1a9015dfc0 2023-10-08T00:11:22.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.656+0000 7f1a85d9b700 1 --2- 172.21.15.99:0/2721659584 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a900ca230 0x7f1a9015da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:11:22.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.656+0000 7f1a8e59c700 1 --2- 172.21.15.99:0/2721659584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a900c9e60 0x7f1a9015afd0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1a800098e0 tx=0x7f1a800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f687918ea2c2447 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:22.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.657+0000 7f1a8dd9b700 1 -- 172.21.15.99:0/2721659584 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a80009e80 con 0x7f1a900c9e60 2023-10-08T00:11:22.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.657+0000 7f1a8dd9b700 1 -- 172.21.15.99:0/2721659584 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a8000b860 con 0x7f1a900c9e60 2023-10-08T00:11:22.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.657+0000 7f1a8dd9b700 1 -- 172.21.15.99:0/2721659584 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8000bbe0 con 0x7f1a900c9e60 2023-10-08T00:11:22.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.657+0000 7f1a94e61700 1 -- 172.21.15.99:0/2721659584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a900c9e60 msgr2=0x7f1a9015afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:22.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.657+0000 7f1a94e61700 1 --2- 172.21.15.99:0/2721659584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a900c9e60 0x7f1a9015afd0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1a800098e0 tx=0x7f1a800091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:22.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.657+0000 7f1a94e61700 1 -- 172.21.15.99:0/2721659584 shutdown_connections 2023-10-08T00:11:22.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.657+0000 7f1a94e61700 1 --2- 172.21.15.99:0/2721659584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a900c9e60 0x7f1a9015afd0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:22.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.657+0000 7f1a94e61700 1 --2- 172.21.15.99:0/2721659584 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a900ca230 0x7f1a9015da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:22.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.657+0000 7f1a94e61700 1 -- 172.21.15.99:0/2721659584 >> 172.21.15.99:0/2721659584 conn(0x7f1a90067660 msgr2=0x7f1a90067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:22.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.657+0000 7f1a94e61700 1 -- 172.21.15.99:0/2721659584 shutdown_connections 2023-10-08T00:11:22.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.657+0000 7f1a94e61700 1 -- 172.21.15.99:0/2721659584 wait complete. 2023-10-08T00:11:22.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.658+0000 7f1a94e61700 1 Processor -- start 2023-10-08T00:11:22.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.658+0000 7f1a94e61700 1 -- start start 2023-10-08T00:11:22.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.659+0000 7f1a94e61700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a900c9e60 0x7f1a90159520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:22.669 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.659+0000 7f1a94e61700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a900ca230 0x7f1a900b1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:22.669 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.659+0000 7f1a94e61700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a9015dfc0 0x7f1a900c30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:22.669 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.659+0000 7f1a94e61700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a900b2eb0 con 0x7f1a900c9e60 2023-10-08T00:11:22.669 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.659+0000 7f1a94e61700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a900c5280 con 0x7f1a900ca230 2023-10-08T00:11:22.669 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.659+0000 7f1a94e61700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a900b2610 con 0x7f1a9015dfc0 2023-10-08T00:11:22.669 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.659+0000 7f1a8e59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a900c9e60 0x7f1a90159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:22.669 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.659+0000 7f1a8e59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a900c9e60 0x7f1a90159520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56064/0 (socket says 172.21.15.99:56064) 2023-10-08T00:11:22.669 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.659+0000 7f1a8e59c700 1 -- 172.21.15.99:0/3402221503 learned_addr learned my addr 172.21.15.99:0/3402221503 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:22.669 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.659+0000 7f1a8ed9d700 1 --2- 172.21.15.99:0/3402221503 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a9015dfc0 0x7f1a900c30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:22.670 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.659+0000 7f1a85d9b700 1 --2- 172.21.15.99:0/3402221503 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a900ca230 0x7f1a900b1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:22.670 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.659+0000 7f1a8e59c700 1 -- 172.21.15.99:0/3402221503 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a9015dfc0 msgr2=0x7f1a900c30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:22.670 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.659+0000 7f1a8e59c700 1 --2- 172.21.15.99:0/3402221503 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a9015dfc0 0x7f1a900c30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:22.670 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.659+0000 7f1a8e59c700 1 -- 172.21.15.99:0/3402221503 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a900ca230 msgr2=0x7f1a900b1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:22.670 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.659+0000 7f1a8e59c700 1 --2- 172.21.15.99:0/3402221503 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a900ca230 0x7f1a900b1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:22.670 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.659+0000 7f1a8e59c700 1 -- 172.21.15.99:0/3402221503 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a80009420 con 0x7f1a900c9e60 2023-10-08T00:11:22.670 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.660+0000 7f1a8e59c700 1 --2- 172.21.15.99:0/3402221503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a900c9e60 0x7f1a90159520 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1a90068640 tx=0x7f1a80017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:22.670 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.660+0000 7f1a86ffd700 1 -- 172.21.15.99:0/3402221503 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8000b9d0 con 0x7f1a900c9e60 2023-10-08T00:11:22.671 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.660+0000 7f1a86ffd700 1 -- 172.21.15.99:0/3402221503 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a8000bb30 con 0x7f1a900c9e60 2023-10-08T00:11:22.671 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.660+0000 7f1a94e61700 1 -- 172.21.15.99:0/3402221503 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a900b28f0 con 0x7f1a900c9e60 2023-10-08T00:11:22.671 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.660+0000 7f1a86ffd700 1 -- 172.21.15.99:0/3402221503 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a80011650 con 0x7f1a900c9e60 2023-10-08T00:11:22.671 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.660+0000 7f1a94e61700 1 -- 172.21.15.99:0/3402221503 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a90159a60 con 0x7f1a900c9e60 2023-10-08T00:11:22.671 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.661+0000 7f1a94e61700 1 -- 172.21.15.99:0/3402221503 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a900c5560 con 0x7f1a900c9e60 2023-10-08T00:11:22.671 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.661+0000 7f1a86ffd700 1 -- 172.21.15.99:0/3402221503 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1a80011a80 con 0x7f1a900c9e60 2023-10-08T00:11:22.671 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.661+0000 7f1a86ffd700 1 --2- 172.21.15.99:0/3402221503 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1a70066f30 0x7f1a700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:22.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.662+0000 7f1a86ffd700 1 -- 172.21.15.99:0/3402221503 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1a80085df0 con 0x7f1a900c9e60 2023-10-08T00:11:22.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.662+0000 7f1a85d9b700 1 --2- 172.21.15.99:0/3402221503 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1a70066f30 0x7f1a700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:22.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.662+0000 7f1a85d9b700 1 --2- 172.21.15.99:0/3402221503 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1a70066f30 0x7f1a700693e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1a900af490 tx=0x7f1a78008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:22.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.666+0000 7f1a86ffd700 1 -- 172.21.15.99:0/3402221503 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1a80053060 con 0x7f1a900c9e60 2023-10-08T00:11:22.850 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.849+0000 7f1a94e61700 1 -- 172.21.15.99:0/3402221503 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1a900c5070 con 0x7f1a900c9e60 2023-10-08T00:11:22.850 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.849+0000 7f1a86ffd700 1 -- 172.21.15.99:0/3402221503 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1a900c5070 con 0x7f1a900c9e60 2023-10-08T00:11:22.850 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2023-10-08T00:11:22.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.851+0000 7f1a94e61700 1 -- 172.21.15.99:0/3402221503 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1a70066f30 msgr2=0x7f1a700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:22.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.851+0000 7f1a94e61700 1 --2- 172.21.15.99:0/3402221503 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1a70066f30 0x7f1a700693e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1a900af490 tx=0x7f1a78008040 comp rx=0 tx=0).stop 2023-10-08T00:11:22.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.851+0000 7f1a94e61700 1 -- 172.21.15.99:0/3402221503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a900c9e60 msgr2=0x7f1a90159520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:22.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.851+0000 7f1a94e61700 1 --2- 172.21.15.99:0/3402221503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a900c9e60 0x7f1a90159520 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1a90068640 tx=0x7f1a80017000 comp rx=0 tx=0).stop 2023-10-08T00:11:22.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.852+0000 7f1a94e61700 1 -- 172.21.15.99:0/3402221503 shutdown_connections 2023-10-08T00:11:22.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.852+0000 7f1a94e61700 1 --2- 172.21.15.99:0/3402221503 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1a70066f30 0x7f1a700693e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:22.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.852+0000 7f1a94e61700 1 --2- 172.21.15.99:0/3402221503 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a9015dfc0 0x7f1a900c30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:22.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.852+0000 7f1a94e61700 1 --2- 172.21.15.99:0/3402221503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a900c9e60 0x7f1a90159520 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:22.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.852+0000 7f1a94e61700 1 --2- 172.21.15.99:0/3402221503 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a900ca230 0x7f1a900b1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:22.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.852+0000 7f1a94e61700 1 -- 172.21.15.99:0/3402221503 >> 172.21.15.99:0/3402221503 conn(0x7f1a90067660 msgr2=0x7f1a900ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:22.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.852+0000 7f1a94e61700 1 -- 172.21.15.99:0/3402221503 shutdown_connections 2023-10-08T00:11:22.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.852+0000 7f1a94e61700 1 -- 172.21.15.99:0/3402221503 wait complete. 2023-10-08T00:11:22.864 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T00:11:22.981 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.980+0000 7fc15ebdf700 1 Processor -- start 2023-10-08T00:11:22.981 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.980+0000 7fc15ebdf700 1 -- start start 2023-10-08T00:11:22.983 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.981+0000 7fc15ebdf700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc1580b6580 0x7fc15806a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:22.983 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.981+0000 7fc15ebdf700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc15806a830 0x7fc1580b5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:22.983 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.981+0000 7fc15ebdf700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc15806ac90 con 0x7fc1580cce30 2023-10-08T00:11:22.984 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.981+0000 7fc15ebdf700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1580b4d70 con 0x7fc1580b6580 2023-10-08T00:11:22.984 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.981+0000 7fc15ebdf700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc15815b3b0 con 0x7fc15806a830 2023-10-08T00:11:22.984 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.981+0000 7fc15d3dc700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc1580b6580 0x7fc15806a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:22.984 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.981+0000 7fc15dbdd700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fc1580cce30 0x7fc15815aca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56078/0 (socket says 172.21.15.99:56078) 2023-10-08T00:11:22.984 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.981+0000 7fc15dbdd700 1 -- 172.21.15.99:0/2797271862 learned_addr learned my addr 172.21.15.99:0/2797271862 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:22.984 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.981+0000 7fc15e3de700 1 --2- 172.21.15.99:0/2797271862 >> v2:172.21.15.191:3300/0 conn(0x7fc15806a830 0x7fc1580b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:22.985 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.981+0000 7fc15d3dc700 1 -- 172.21.15.99:0/2797271862 >> v2:172.21.15.191:3300/0 conn(0x7fc15806a830 msgr2=0x7fc1580b5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:22.985 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.981+0000 7fc15d3dc700 1 --2- 172.21.15.99:0/2797271862 >> v2:172.21.15.191:3300/0 conn(0x7fc15806a830 0x7fc1580b5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:22.985 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.981+0000 7fc15d3dc700 1 -- 172.21.15.99:0/2797271862 >> v1:172.21.15.99:6789/0 conn(0x7fc1580cce30 legacy=0x7fc15815aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:22.985 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.981+0000 7fc15d3dc700 1 -- 172.21.15.99:0/2797271862 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1580cd8d0 con 0x7fc1580b6580 2023-10-08T00:11:22.985 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.981+0000 7fc15cbdb700 1 -- 172.21.15.99:0/2797271862 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4021246228 0 0) 0x7fc150001490 con 0x7fc1580cce30 2023-10-08T00:11:22.985 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.981+0000 7fc15e3de700 1 --2- 172.21.15.99:0/2797271862 >> v2:172.21.15.191:3300/0 conn(0x7fc15806a830 0x7fc1580b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:11:22.985 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.981+0000 7fc15d3dc700 1 --2- 172.21.15.99:0/2797271862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1580b6580 0x7fc15806a2f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc1580b4ce0 tx=0x7fc1480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a74327bfb6c3349d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:22.985 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.982+0000 7fc15cbdb700 1 -- 172.21.15.99:0/2797271862 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14800b6b0 con 0x7fc1580b6580 2023-10-08T00:11:22.986 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.982+0000 7fc15cbdb700 1 -- 172.21.15.99:0/2797271862 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc14800b810 con 0x7fc1580b6580 2023-10-08T00:11:22.986 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.982+0000 7fc15cbdb700 1 -- 172.21.15.99:0/2797271862 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14800bbb0 con 0x7fc1580b6580 2023-10-08T00:11:22.986 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.982+0000 7fc15ebdf700 1 -- 172.21.15.99:0/2797271862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1580b6580 msgr2=0x7fc15806a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:22.986 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.982+0000 7fc15ebdf700 1 --2- 172.21.15.99:0/2797271862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1580b6580 0x7fc15806a2f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc1580b4ce0 tx=0x7fc1480091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:22.986 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.982+0000 7fc15ebdf700 1 -- 172.21.15.99:0/2797271862 shutdown_connections 2023-10-08T00:11:22.986 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.982+0000 7fc15ebdf700 1 --2- 172.21.15.99:0/2797271862 >> v2:172.21.15.191:3300/0 conn(0x7fc15806a830 0x7fc1580b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:22.986 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.982+0000 7fc15ebdf700 1 --2- 172.21.15.99:0/2797271862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1580b6580 0x7fc15806a2f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:22.986 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.982+0000 7fc15ebdf700 1 -- 172.21.15.99:0/2797271862 >> 172.21.15.99:0/2797271862 conn(0x7fc158068cb0 msgr2=0x7fc1580690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:22.987 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.982+0000 7fc15ebdf700 1 -- 172.21.15.99:0/2797271862 shutdown_connections 2023-10-08T00:11:22.987 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.982+0000 7fc15ebdf700 1 -- 172.21.15.99:0/2797271862 wait complete. 2023-10-08T00:11:22.990 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.983+0000 7fc15ebdf700 1 Processor -- start 2023-10-08T00:11:22.990 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.983+0000 7fc15ebdf700 1 -- start start 2023-10-08T00:11:22.990 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.984+0000 7fc15ebdf700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc15806a830 0x7fc15815cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:22.990 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.984+0000 7fc15ebdf700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1580b6580 0x7fc158153150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:22.990 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.984+0000 7fc15ebdf700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1580cce30 0x7fc15813dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:22.990 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.984+0000 7fc15ebdf700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15814ed60 con 0x7fc1580b6580 2023-10-08T00:11:22.990 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.984+0000 7fc15ebdf700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc158153690 con 0x7fc1580cce30 2023-10-08T00:11:22.991 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.984+0000 7fc15ebdf700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1581544d0 con 0x7fc15806a830 2023-10-08T00:11:22.991 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.984+0000 7fc15d3dc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1580b6580 0x7fc158153150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:22.991 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.984+0000 7fc15d3dc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1580b6580 0x7fc158153150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56080/0 (socket says 172.21.15.99:56080) 2023-10-08T00:11:22.991 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.984+0000 7fc15d3dc700 1 -- 172.21.15.99:0/2190387352 learned_addr learned my addr 172.21.15.99:0/2190387352 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:22.991 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.984+0000 7fc15e3de700 1 --2- 172.21.15.99:0/2190387352 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1580cce30 0x7fc15813dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:22.991 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.984+0000 7fc15dbdd700 1 --2- 172.21.15.99:0/2190387352 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc15806a830 0x7fc15815cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:22.991 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.984+0000 7fc15d3dc700 1 -- 172.21.15.99:0/2190387352 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc15806a830 msgr2=0x7fc15815cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:22.991 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.985+0000 7fc15d3dc700 1 --2- 172.21.15.99:0/2190387352 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc15806a830 0x7fc15815cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:22.991 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.985+0000 7fc15d3dc700 1 -- 172.21.15.99:0/2190387352 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1580cce30 msgr2=0x7fc15813dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:22.992 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.985+0000 7fc15d3dc700 1 --2- 172.21.15.99:0/2190387352 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1580cce30 0x7fc15813dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:22.992 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.985+0000 7fc15d3dc700 1 -- 172.21.15.99:0/2190387352 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc148009420 con 0x7fc1580b6580 2023-10-08T00:11:22.992 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.985+0000 7fc15dbdd700 1 --2- 172.21.15.99:0/2190387352 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc15806a830 0x7fc15815cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:11:22.992 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.985+0000 7fc15d3dc700 1 --2- 172.21.15.99:0/2190387352 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1580b6580 0x7fc158153150 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc15806b300 tx=0x7fc148017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:22.992 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.985+0000 7fc14dffb700 1 -- 172.21.15.99:0/2190387352 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc148011420 con 0x7fc1580b6580 2023-10-08T00:11:22.992 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.985+0000 7fc14dffb700 1 -- 172.21.15.99:0/2190387352 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc148011580 con 0x7fc1580b6580 2023-10-08T00:11:22.992 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.985+0000 7fc15ebdf700 1 -- 172.21.15.99:0/2190387352 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc158153920 con 0x7fc1580b6580 2023-10-08T00:11:22.993 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.985+0000 7fc14dffb700 1 -- 172.21.15.99:0/2190387352 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc148011940 con 0x7fc1580b6580 2023-10-08T00:11:22.993 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.985+0000 7fc15ebdf700 1 -- 172.21.15.99:0/2190387352 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc158154160 con 0x7fc1580b6580 2023-10-08T00:11:22.993 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.986+0000 7fc15ebdf700 1 -- 172.21.15.99:0/2190387352 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc158154660 con 0x7fc1580b6580 2023-10-08T00:11:22.993 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.986+0000 7fc14dffb700 1 -- 172.21.15.99:0/2190387352 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc148011cf0 con 0x7fc1580b6580 2023-10-08T00:11:22.993 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.987+0000 7fc14dffb700 1 --2- 172.21.15.99:0/2190387352 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc13c066f30 0x7fc13c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:22.993 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.987+0000 7fc14dffb700 1 -- 172.21.15.99:0/2190387352 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc148085b80 con 0x7fc1580b6580 2023-10-08T00:11:22.993 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.987+0000 7fc15dbdd700 1 --2- 172.21.15.99:0/2190387352 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc13c066f30 0x7fc13c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:22.993 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.987+0000 7fc15dbdd700 1 --2- 172.21.15.99:0/2190387352 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc13c066f30 0x7fc13c0693e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc1580b3310 tx=0x7fc150009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:22.994 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:22.991+0000 7fc14dffb700 1 -- 172.21.15.99:0/2190387352 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc148052e70 con 0x7fc1580b6580 2023-10-08T00:11:23.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.170+0000 7fc15ebdf700 1 -- 172.21.15.99:0/2190387352 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc1580895b0 con 0x7fc1580b6580 2023-10-08T00:11:23.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.171+0000 7fc14dffb700 1 -- 172.21.15.99:0/2190387352 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc148056490 con 0x7fc1580b6580 2023-10-08T00:11:23.172 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2023-10-08T00:11:23.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.173+0000 7fc15ebdf700 1 -- 172.21.15.99:0/2190387352 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc13c066f30 msgr2=0x7fc13c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:23.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.173+0000 7fc15ebdf700 1 --2- 172.21.15.99:0/2190387352 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc13c066f30 0x7fc13c0693e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc1580b3310 tx=0x7fc150009040 comp rx=0 tx=0).stop 2023-10-08T00:11:23.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.173+0000 7fc15ebdf700 1 -- 172.21.15.99:0/2190387352 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1580b6580 msgr2=0x7fc158153150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:23.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.173+0000 7fc15ebdf700 1 --2- 172.21.15.99:0/2190387352 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1580b6580 0x7fc158153150 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc15806b300 tx=0x7fc148017000 comp rx=0 tx=0).stop 2023-10-08T00:11:23.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.173+0000 7fc15ebdf700 1 -- 172.21.15.99:0/2190387352 shutdown_connections 2023-10-08T00:11:23.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.173+0000 7fc15ebdf700 1 --2- 172.21.15.99:0/2190387352 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc13c066f30 0x7fc13c0693e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:23.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.173+0000 7fc15ebdf700 1 --2- 172.21.15.99:0/2190387352 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1580cce30 0x7fc15813dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:23.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.173+0000 7fc15ebdf700 1 --2- 172.21.15.99:0/2190387352 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc15806a830 0x7fc15815cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:23.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.173+0000 7fc15ebdf700 1 --2- 172.21.15.99:0/2190387352 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1580b6580 0x7fc158153150 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:23.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.173+0000 7fc15ebdf700 1 -- 172.21.15.99:0/2190387352 >> 172.21.15.99:0/2190387352 conn(0x7fc158068cb0 msgr2=0x7fc1580d1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:23.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.173+0000 7fc15ebdf700 1 -- 172.21.15.99:0/2190387352 shutdown_connections 2023-10-08T00:11:23.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.173+0000 7fc15ebdf700 1 -- 172.21.15.99:0/2190387352 wait complete. 2023-10-08T00:11:23.186 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:11:23.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.301+0000 7fbcad63e700 1 Processor -- start 2023-10-08T00:11:23.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.301+0000 7fbcad63e700 1 -- start start 2023-10-08T00:11:23.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.302+0000 7fbcad63e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fbca8157870 0x7fbca815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:23.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.302+0000 7fbcad63e700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbca815cbb0 con 0x7fbca815a620 2023-10-08T00:11:23.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.302+0000 7fbcad63e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbca815bfa0 con 0x7fbca8157c40 2023-10-08T00:11:23.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.302+0000 7fbcad63e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbca815c280 con 0x7fbca8157870 2023-10-08T00:11:23.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.302+0000 7fbca67fc700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fbca815a620 0x7fbca815d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56090/0 (socket says 172.21.15.99:56090) 2023-10-08T00:11:23.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.302+0000 7fbca67fc700 1 -- 172.21.15.99:0/356547576 learned_addr learned my addr 172.21.15.99:0/356547576 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:23.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.302+0000 7fbca6ffd700 1 --2- 172.21.15.99:0/356547576 >> v2:172.21.15.191:3300/0 conn(0x7fbca8157870 0x7fbca815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:23.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.302+0000 7fbca5ffb700 1 -- 172.21.15.99:0/356547576 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 70677592 0 0) 0x7fbca815cbb0 con 0x7fbca815a620 2023-10-08T00:11:23.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.302+0000 7fbca5ffb700 1 -- 172.21.15.99:0/356547576 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc90003570 con 0x7fbca815a620 2023-10-08T00:11:23.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.302+0000 7fbca5ffb700 1 -- 172.21.15.99:0/356547576 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1646261838 0 0) 0x7fbc90003570 con 0x7fbca815a620 2023-10-08T00:11:23.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.302+0000 7fbca5ffb700 1 -- 172.21.15.99:0/356547576 >> v2:172.21.15.191:3300/0 conn(0x7fbca8157870 msgr2=0x7fbca815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:23.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.302+0000 7fbca5ffb700 1 --2- 172.21.15.99:0/356547576 >> v2:172.21.15.191:3300/0 conn(0x7fbca8157870 0x7fbca815b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:23.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.302+0000 7fbca5ffb700 1 -- 172.21.15.99:0/356547576 >> v1:172.21.15.191:6789/0 conn(0x7fbca8157c40 legacy=0x7fbca8158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:23.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.302+0000 7fbca5ffb700 1 -- 172.21.15.99:0/356547576 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca815c560 con 0x7fbca815a620 2023-10-08T00:11:23.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.302+0000 7fbca5ffb700 1 -- 172.21.15.99:0/356547576 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7fbc9c001470 con 0x7fbca815a620 2023-10-08T00:11:23.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.302+0000 7fbca5ffb700 1 -- 172.21.15.99:0/356547576 >> v1:172.21.15.99:6789/0 conn(0x7fbca815a620 legacy=0x7fbca815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:23.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.302+0000 7fbca5ffb700 1 --2- 172.21.15.99:0/356547576 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc90003a60 0x7fbc90003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:23.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.302+0000 7fbca5ffb700 1 --2- 172.21.15.99:0/356547576 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbc90004810 0x7fbc90004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:23.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.302+0000 7fbca5ffb700 1 --2- 172.21.15.99:0/356547576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc900051c0 0x7fbc9000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:23.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.302+0000 7fbca5ffb700 1 -- 172.21.15.99:0/356547576 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca815c280 con 0x7fbc900051c0 2023-10-08T00:11:23.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.302+0000 7fbca5ffb700 1 -- 172.21.15.99:0/356547576 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9000c160 con 0x7fbc90003a60 2023-10-08T00:11:23.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.302+0000 7fbca5ffb700 1 -- 172.21.15.99:0/356547576 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9000c320 con 0x7fbc90004810 2023-10-08T00:11:23.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.303+0000 7fbca6ffd700 1 --2- 172.21.15.99:0/356547576 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc90003a60 0x7fbc90003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:23.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.303+0000 7fbca67fc700 1 --2- 172.21.15.99:0/356547576 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbc90004810 0x7fbc90004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:23.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.303+0000 7fbca77fe700 1 --2- 172.21.15.99:0/356547576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc900051c0 0x7fbc9000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:23.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.303+0000 7fbca77fe700 1 -- 172.21.15.99:0/356547576 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbc90004810 msgr2=0x7fbc90004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:23.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.303+0000 7fbca77fe700 1 --2- 172.21.15.99:0/356547576 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbc90004810 0x7fbc90004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:23.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.303+0000 7fbca77fe700 1 -- 172.21.15.99:0/356547576 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc90003a60 msgr2=0x7fbc90003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:23.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.303+0000 7fbca77fe700 1 --2- 172.21.15.99:0/356547576 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc90003a60 0x7fbc90003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:23.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.303+0000 7fbca77fe700 1 -- 172.21.15.99:0/356547576 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc90003570 con 0x7fbc900051c0 2023-10-08T00:11:23.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.303+0000 7fbca77fe700 1 -- 172.21.15.99:0/356547576 reap_dead start 2023-10-08T00:11:23.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.303+0000 7fbca77fe700 1 --2- 172.21.15.99:0/356547576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc900051c0 0x7fbc9000ba60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbc98001f90 tx=0x7fbc98001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e18273cff0fc61ee server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:23.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.303+0000 7fbca5ffb700 1 -- 172.21.15.99:0/356547576 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc98001950 con 0x7fbc900051c0 2023-10-08T00:11:23.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.303+0000 7fbca5ffb700 1 -- 172.21.15.99:0/356547576 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc9800c8b0 con 0x7fbc900051c0 2023-10-08T00:11:23.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.303+0000 7fbca5ffb700 1 -- 172.21.15.99:0/356547576 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc9800cc50 con 0x7fbc900051c0 2023-10-08T00:11:23.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.304+0000 7fbcad63e700 1 -- 172.21.15.99:0/356547576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc900051c0 msgr2=0x7fbc9000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:23.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.304+0000 7fbcad63e700 1 --2- 172.21.15.99:0/356547576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc900051c0 0x7fbc9000ba60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbc98001f90 tx=0x7fbc98001640 comp rx=0 tx=0).stop 2023-10-08T00:11:23.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.304+0000 7fbcad63e700 1 -- 172.21.15.99:0/356547576 shutdown_connections 2023-10-08T00:11:23.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.304+0000 7fbcad63e700 1 --2- 172.21.15.99:0/356547576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc900051c0 0x7fbc9000ba60 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:23.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.304+0000 7fbcad63e700 1 -- 172.21.15.99:0/356547576 >> 172.21.15.99:0/356547576 conn(0x7fbca8068ce0 msgr2=0x7fbca80690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:23.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.304+0000 7fbcad63e700 1 -- 172.21.15.99:0/356547576 shutdown_connections 2023-10-08T00:11:23.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.304+0000 7fbcad63e700 1 -- 172.21.15.99:0/356547576 wait complete. 2023-10-08T00:11:23.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.305+0000 7fbcad63e700 1 Processor -- start 2023-10-08T00:11:23.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.305+0000 7fbcad63e700 1 -- start start 2023-10-08T00:11:23.314 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.306+0000 7fbcad63e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc900051c0 0x7fbca8150840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:23.314 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.306+0000 7fbcad63e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbca8151040 0x7fbca81514b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:23.314 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.306+0000 7fbcad63e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbca816a610 0x7fbca816aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:23.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.306+0000 7fbcad63e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca8154400 con 0x7fbc900051c0 2023-10-08T00:11:23.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.306+0000 7fbcad63e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca8150d80 con 0x7fbca8151040 2023-10-08T00:11:23.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.306+0000 7fbcad63e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbca8153b60 con 0x7fbca816a610 2023-10-08T00:11:23.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.306+0000 7fbca6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc900051c0 0x7fbca8150840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:23.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.306+0000 7fbca6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc900051c0 0x7fbca8150840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56088/0 (socket says 172.21.15.99:56088) 2023-10-08T00:11:23.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.306+0000 7fbca6ffd700 1 -- 172.21.15.99:0/565457118 learned_addr learned my addr 172.21.15.99:0/565457118 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:23.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.306+0000 7fbca67fc700 1 --2- 172.21.15.99:0/565457118 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbca8151040 0x7fbca81514b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:23.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.306+0000 7fbca77fe700 1 --2- 172.21.15.99:0/565457118 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbca816a610 0x7fbca816aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:23.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.306+0000 7fbca6ffd700 1 -- 172.21.15.99:0/565457118 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbca816a610 msgr2=0x7fbca816aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:23.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.306+0000 7fbca6ffd700 1 --2- 172.21.15.99:0/565457118 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbca816a610 0x7fbca816aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:23.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.306+0000 7fbca6ffd700 1 -- 172.21.15.99:0/565457118 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbca8151040 msgr2=0x7fbca81514b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:23.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.306+0000 7fbca6ffd700 1 --2- 172.21.15.99:0/565457118 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbca8151040 0x7fbca81514b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:23.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.306+0000 7fbca6ffd700 1 -- 172.21.15.99:0/565457118 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca8153e40 con 0x7fbc900051c0 2023-10-08T00:11:23.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.306+0000 7fbca6ffd700 1 --2- 172.21.15.99:0/565457118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc900051c0 0x7fbca8150840 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbca0006ac0 tx=0x7fbca00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:23.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.306+0000 7fbc8effd700 1 -- 172.21.15.99:0/565457118 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca000fa10 con 0x7fbc900051c0 2023-10-08T00:11:23.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.306+0000 7fbc8effd700 1 -- 172.21.15.99:0/565457118 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca000fb70 con 0x7fbc900051c0 2023-10-08T00:11:23.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.306+0000 7fbcad63e700 1 -- 172.21.15.99:0/565457118 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbca8154120 con 0x7fbc900051c0 2023-10-08T00:11:23.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.306+0000 7fbc8effd700 1 -- 172.21.15.99:0/565457118 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca0014440 con 0x7fbc900051c0 2023-10-08T00:11:23.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.306+0000 7fbcad63e700 1 -- 172.21.15.99:0/565457118 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbca816afc0 con 0x7fbc900051c0 2023-10-08T00:11:23.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.307+0000 7fbcad63e700 1 -- 172.21.15.99:0/565457118 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbca80b15e0 con 0x7fbc900051c0 2023-10-08T00:11:23.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.308+0000 7fbc8effd700 1 -- 172.21.15.99:0/565457118 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbca0014830 con 0x7fbc900051c0 2023-10-08T00:11:23.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.308+0000 7fbc8effd700 1 --2- 172.21.15.99:0/565457118 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbc94066f60 0x7fbc94069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:23.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.308+0000 7fbc8effd700 1 -- 172.21.15.99:0/565457118 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbca0018030 con 0x7fbc900051c0 2023-10-08T00:11:23.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.309+0000 7fbca67fc700 1 --2- 172.21.15.99:0/565457118 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbc94066f60 0x7fbc94069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:23.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.309+0000 7fbca67fc700 1 --2- 172.21.15.99:0/565457118 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbc94066f60 0x7fbc94069410 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c0015a0 tx=0x7fbc9c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:23.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.313+0000 7fbc8effd700 1 -- 172.21.15.99:0/565457118 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbca80b15e0 con 0x7fbc900051c0 2023-10-08T00:11:23.496 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.496+0000 7fbcad63e700 1 -- 172.21.15.99:0/565457118 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbca80b15e0 con 0x7fbc900051c0 2023-10-08T00:11:23.497 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.497+0000 7fbc8effd700 1 -- 172.21.15.99:0/565457118 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15350 (secure 0 0 0) 0x7fbca80b15e0 con 0x7fbc900051c0 2023-10-08T00:11:23.497 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:11:23.498 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":20,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:10:55.080854+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:10:51.822157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T00:10:52.415130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:11:23.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.499+0000 7fbcad63e700 1 -- 172.21.15.99:0/565457118 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbc94066f60 msgr2=0x7fbc94069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:23.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.499+0000 7fbcad63e700 1 --2- 172.21.15.99:0/565457118 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbc94066f60 0x7fbc94069410 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c0015a0 tx=0x7fbc9c009040 comp rx=0 tx=0).stop 2023-10-08T00:11:23.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.499+0000 7fbcad63e700 1 -- 172.21.15.99:0/565457118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc900051c0 msgr2=0x7fbca8150840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:23.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.499+0000 7fbcad63e700 1 --2- 172.21.15.99:0/565457118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc900051c0 0x7fbca8150840 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbca0006ac0 tx=0x7fbca00091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:23.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.499+0000 7fbcad63e700 1 -- 172.21.15.99:0/565457118 shutdown_connections 2023-10-08T00:11:23.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.499+0000 7fbcad63e700 1 --2- 172.21.15.99:0/565457118 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbc94066f60 0x7fbc94069410 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:23.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.499+0000 7fbcad63e700 1 --2- 172.21.15.99:0/565457118 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbca816a610 0x7fbca816aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:23.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.499+0000 7fbcad63e700 1 --2- 172.21.15.99:0/565457118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc900051c0 0x7fbca8150840 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:23.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.499+0000 7fbcad63e700 1 --2- 172.21.15.99:0/565457118 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbca8151040 0x7fbca81514b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:23.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.499+0000 7fbcad63e700 1 -- 172.21.15.99:0/565457118 >> 172.21.15.99:0/565457118 conn(0x7fbca8068ce0 msgr2=0x7fbca80d09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:23.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.499+0000 7fbcad63e700 1 -- 172.21.15.99:0/565457118 shutdown_connections 2023-10-08T00:11:23.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.499+0000 7fbcad63e700 1 -- 172.21.15.99:0/565457118 wait complete. 2023-10-08T00:11:23.512 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:10:42.063344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T00:10:51.822157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T00:10:52.415130+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T00:11:23.512 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T00:11:23.628 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.627+0000 7f3a0b3bd700 1 Processor -- start 2023-10-08T00:11:23.628 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.627+0000 7f3a0b3bd700 1 -- start start 2023-10-08T00:11:23.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.628+0000 7f3a0b3bd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3a040cce30 0x7f3a0415aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:23.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.628+0000 7f3a0b3bd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a040b6580 0x7f3a0406a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:23.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.628+0000 7f3a0b3bd700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f3a0406a830 0x7f3a0415d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:23.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.628+0000 7f3a0b3bd700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a040b4d70 con 0x7f3a040cce30 2023-10-08T00:11:23.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.628+0000 7f3a0b3bd700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a0406ac70 con 0x7f3a0406a830 2023-10-08T00:11:23.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.628+0000 7f3a0b3bd700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a040cd8d0 con 0x7f3a040b6580 2023-10-08T00:11:23.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.628+0000 7f3a0a3bb700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3a040cce30 0x7f3a0415aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:23.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.628+0000 7f3a0a3bb700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3a040cce30 0x7f3a0415aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52074/0 (socket says 172.21.15.99:52074) 2023-10-08T00:11:23.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.628+0000 7f3a0a3bb700 1 -- 172.21.15.99:0/3238274419 learned_addr learned my addr 172.21.15.99:0/3238274419 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:23.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.629+0000 7f3a0abbc700 1 --2- 172.21.15.99:0/3238274419 >> v2:172.21.15.191:3300/0 conn(0x7f3a0406a830 0x7f3a0415d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:23.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.629+0000 7f3a09bba700 1 --2- 172.21.15.99:0/3238274419 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a040b6580 0x7f3a0406a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:23.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.629+0000 7f3a0a3bb700 1 -- 172.21.15.99:0/3238274419 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a040b6580 msgr2=0x7f3a0406a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:23.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.629+0000 7f3a0a3bb700 1 --2- 172.21.15.99:0/3238274419 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a040b6580 0x7f3a0406a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:23.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.629+0000 7f3a0a3bb700 1 -- 172.21.15.99:0/3238274419 >> v2:172.21.15.191:3300/0 conn(0x7f3a0406a830 msgr2=0x7f3a0415d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:23.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.629+0000 7f3a0a3bb700 1 --2- 172.21.15.99:0/3238274419 >> v2:172.21.15.191:3300/0 conn(0x7f3a0406a830 0x7f3a0415d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:23.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.629+0000 7f3a0a3bb700 1 -- 172.21.15.99:0/3238274419 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a040b4440 con 0x7f3a040cce30 2023-10-08T00:11:23.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.629+0000 7f3a0a3bb700 1 --2- 172.21.15.99:0/3238274419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a040cce30 0x7f3a0415aca0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3a0415a280 tx=0x7f39f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2fdccf27d4009fa server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:23.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.629+0000 7f3a093b9700 1 -- 172.21.15.99:0/3238274419 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f400b860 con 0x7f3a040cce30 2023-10-08T00:11:23.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.629+0000 7f3a093b9700 1 -- 172.21.15.99:0/3238274419 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39f400b9c0 con 0x7f3a040cce30 2023-10-08T00:11:23.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.629+0000 7f3a093b9700 1 -- 172.21.15.99:0/3238274419 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f400bd60 con 0x7f3a040cce30 2023-10-08T00:11:23.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.630+0000 7f3a0b3bd700 1 -- 172.21.15.99:0/3238274419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a040cce30 msgr2=0x7f3a0415aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:23.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.630+0000 7f3a0b3bd700 1 --2- 172.21.15.99:0/3238274419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a040cce30 0x7f3a0415aca0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3a0415a280 tx=0x7f39f4009250 comp rx=0 tx=0).stop 2023-10-08T00:11:23.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.630+0000 7f3a0b3bd700 1 -- 172.21.15.99:0/3238274419 shutdown_connections 2023-10-08T00:11:23.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.630+0000 7f3a0b3bd700 1 --2- 172.21.15.99:0/3238274419 >> v2:172.21.15.191:3300/0 conn(0x7f3a0406a830 0x7f3a0415d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:23.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.630+0000 7f3a0b3bd700 1 --2- 172.21.15.99:0/3238274419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a040cce30 0x7f3a0415aca0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:23.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.630+0000 7f3a0b3bd700 1 --2- 172.21.15.99:0/3238274419 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a040b6580 0x7f3a0406a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:23.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.630+0000 7f3a0b3bd700 1 -- 172.21.15.99:0/3238274419 >> 172.21.15.99:0/3238274419 conn(0x7f3a04068cb0 msgr2=0x7f3a040690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:23.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.630+0000 7f3a0b3bd700 1 -- 172.21.15.99:0/3238274419 shutdown_connections 2023-10-08T00:11:23.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.630+0000 7f3a0b3bd700 1 -- 172.21.15.99:0/3238274419 wait complete. 2023-10-08T00:11:23.635 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.631+0000 7f3a0b3bd700 1 Processor -- start 2023-10-08T00:11:23.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.631+0000 7f3a0b3bd700 1 -- start start 2023-10-08T00:11:23.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.632+0000 7f3a0b3bd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a0406a830 0x7f3a041426e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:23.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.632+0000 7f3a0b3bd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a040b6580 0x7f3a040cf020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:23.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.632+0000 7f3a0b3bd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a040cce30 0x7f3a040cf560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:23.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.632+0000 7f3a0b3bd700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a04152d40 con 0x7f3a040b6580 2023-10-08T00:11:23.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.632+0000 7f3a0b3bd700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a04142c20 con 0x7f3a0406a830 2023-10-08T00:11:23.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.632+0000 7f3a0b3bd700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a041531d0 con 0x7f3a040cce30 2023-10-08T00:11:23.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.632+0000 7f3a09bba700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a040b6580 0x7f3a040cf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:23.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.632+0000 7f3a09bba700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a040b6580 0x7f3a040cf020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52082/0 (socket says 172.21.15.99:52082) 2023-10-08T00:11:23.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.632+0000 7f3a09bba700 1 -- 172.21.15.99:0/1459400795 learned_addr learned my addr 172.21.15.99:0/1459400795 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:23.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.632+0000 7f3a09bba700 1 -- 172.21.15.99:0/1459400795 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a040cce30 msgr2=0x7f3a040cf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:23.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.632+0000 7f3a09bba700 1 --2- 172.21.15.99:0/1459400795 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a040cce30 0x7f3a040cf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:23.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.632+0000 7f3a09bba700 1 -- 172.21.15.99:0/1459400795 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a0406a830 msgr2=0x7f3a041426e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:23.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.632+0000 7f3a09bba700 1 --2- 172.21.15.99:0/1459400795 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a0406a830 0x7f3a041426e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:23.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.632+0000 7f3a09bba700 1 -- 172.21.15.99:0/1459400795 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f40094f0 con 0x7f3a040b6580 2023-10-08T00:11:23.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.632+0000 7f3a09bba700 1 --2- 172.21.15.99:0/1459400795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a040b6580 0x7f3a040cf020 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3a0000d860 tx=0x7f3a00008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:23.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.632+0000 7f39fa7fc700 1 -- 172.21.15.99:0/1459400795 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a000125f0 con 0x7f3a040b6580 2023-10-08T00:11:23.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.632+0000 7f39fa7fc700 1 -- 172.21.15.99:0/1459400795 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a00012750 con 0x7f3a040b6580 2023-10-08T00:11:23.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.632+0000 7f3a0b3bd700 1 -- 172.21.15.99:0/1459400795 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a04152130 con 0x7f3a040b6580 2023-10-08T00:11:23.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.632+0000 7f39fa7fc700 1 -- 172.21.15.99:0/1459400795 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a00012ad0 con 0x7f3a040b6580 2023-10-08T00:11:23.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.632+0000 7f3a0b3bd700 1 -- 172.21.15.99:0/1459400795 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a041526f0 con 0x7f3a040b6580 2023-10-08T00:11:23.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.633+0000 7f3a0b3bd700 1 -- 172.21.15.99:0/1459400795 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a04153020 con 0x7f3a040b6580 2023-10-08T00:11:23.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.633+0000 7f39fa7fc700 1 -- 172.21.15.99:0/1459400795 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3a0001e460 con 0x7f3a040b6580 2023-10-08T00:11:23.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.634+0000 7f39fa7fc700 1 --2- 172.21.15.99:0/1459400795 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f39e8066f30 0x7f39e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:23.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.634+0000 7f39fa7fc700 1 -- 172.21.15.99:0/1459400795 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3a00084c00 con 0x7f3a040b6580 2023-10-08T00:11:23.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.634+0000 7f3a0a3bb700 1 --2- 172.21.15.99:0/1459400795 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f39e8066f30 0x7f39e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:23.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.635+0000 7f3a0a3bb700 1 --2- 172.21.15.99:0/1459400795 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f39e8066f30 0x7f39e80693e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3a0415a280 tx=0x7f39f4017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:23.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.638+0000 7f39fa7fc700 1 -- 172.21.15.99:0/1459400795 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3a0008b080 con 0x7f3a040b6580 2023-10-08T00:11:23.819 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.819+0000 7f3a0b3bd700 1 -- 172.21.15.99:0/1459400795 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3a04152e80 con 0x7f3a040b6580 2023-10-08T00:11:23.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.819+0000 7f39fa7fc700 1 -- 172.21.15.99:0/1459400795 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3a04152e80 con 0x7f3a040b6580 2023-10-08T00:11:23.820 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2023-10-08T00:11:23.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.821+0000 7f3a0b3bd700 1 -- 172.21.15.99:0/1459400795 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f39e8066f30 msgr2=0x7f39e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:23.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.821+0000 7f3a0b3bd700 1 --2- 172.21.15.99:0/1459400795 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f39e8066f30 0x7f39e80693e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3a0415a280 tx=0x7f39f4017000 comp rx=0 tx=0).stop 2023-10-08T00:11:23.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.821+0000 7f3a0b3bd700 1 -- 172.21.15.99:0/1459400795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a040b6580 msgr2=0x7f3a040cf020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:23.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.821+0000 7f3a0b3bd700 1 --2- 172.21.15.99:0/1459400795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a040b6580 0x7f3a040cf020 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3a0000d860 tx=0x7f3a00008040 comp rx=0 tx=0).stop 2023-10-08T00:11:23.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.821+0000 7f3a0b3bd700 1 -- 172.21.15.99:0/1459400795 shutdown_connections 2023-10-08T00:11:23.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.821+0000 7f3a0b3bd700 1 --2- 172.21.15.99:0/1459400795 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f39e8066f30 0x7f39e80693e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:23.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.821+0000 7f3a0b3bd700 1 --2- 172.21.15.99:0/1459400795 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a0406a830 0x7f3a041426e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:23.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.821+0000 7f3a0b3bd700 1 --2- 172.21.15.99:0/1459400795 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a040cce30 0x7f3a040cf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:23.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.821+0000 7f3a0b3bd700 1 --2- 172.21.15.99:0/1459400795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a040b6580 0x7f3a040cf020 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:23.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.821+0000 7f3a0b3bd700 1 -- 172.21.15.99:0/1459400795 >> 172.21.15.99:0/1459400795 conn(0x7f3a04068cb0 msgr2=0x7f3a040d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:23.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.821+0000 7f3a0b3bd700 1 -- 172.21.15.99:0/1459400795 shutdown_connections 2023-10-08T00:11:23.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.821+0000 7f3a0b3bd700 1 -- 172.21.15.99:0/1459400795 wait complete. 2023-10-08T00:11:23.834 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T00:11:23.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.949+0000 7f32e3215700 1 Processor -- start 2023-10-08T00:11:23.950 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.949+0000 7f32e3215700 1 -- start start 2023-10-08T00:11:23.952 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.950+0000 7f32e3215700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f32dc0cce30 0x7f32dc15aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:23.952 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.950+0000 7f32e3215700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f32dc0b6580 0x7f32dc06a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:23.952 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.950+0000 7f32e3215700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f32dc06a830 0x7f32dc15d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:23.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.950+0000 7f32e3215700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32dc06ac90 con 0x7f32dc0cce30 2023-10-08T00:11:23.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.950+0000 7f32e3215700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32dc0b4d70 con 0x7f32dc0b6580 2023-10-08T00:11:23.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.950+0000 7f32e3215700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32dc0cd8d0 con 0x7f32dc06a830 2023-10-08T00:11:23.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.950+0000 7f32e2213700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f32dc0cce30 0x7f32dc15aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:23.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.950+0000 7f32e2213700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f32dc0cce30 0x7f32dc15aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52090/0 (socket says 172.21.15.99:52090) 2023-10-08T00:11:23.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.950+0000 7f32e2213700 1 -- 172.21.15.99:0/852312450 learned_addr learned my addr 172.21.15.99:0/852312450 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:23.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.950+0000 7f32e2a14700 1 --2- 172.21.15.99:0/852312450 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f32dc06a830 0x7f32dc15d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:23.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.950+0000 7f32e1a12700 1 --2- 172.21.15.99:0/852312450 >> v2:172.21.15.191:3300/0 conn(0x7f32dc0b6580 0x7f32dc06a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:23.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.950+0000 7f32e2213700 1 -- 172.21.15.99:0/852312450 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f32dc06a830 msgr2=0x7f32dc15d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:23.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.950+0000 7f32e2213700 1 --2- 172.21.15.99:0/852312450 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f32dc06a830 0x7f32dc15d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:23.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.950+0000 7f32e2213700 1 -- 172.21.15.99:0/852312450 >> v2:172.21.15.191:3300/0 conn(0x7f32dc0b6580 msgr2=0x7f32dc06a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:23.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.950+0000 7f32e2213700 1 --2- 172.21.15.99:0/852312450 >> v2:172.21.15.191:3300/0 conn(0x7f32dc0b6580 0x7f32dc06a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:23.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.950+0000 7f32e2213700 1 -- 172.21.15.99:0/852312450 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32dc0b4440 con 0x7f32dc0cce30 2023-10-08T00:11:23.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.950+0000 7f32e1a12700 1 --2- 172.21.15.99:0/852312450 >> v2:172.21.15.191:3300/0 conn(0x7f32dc0b6580 0x7f32dc06a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:11:23.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.950+0000 7f32e2a14700 1 --2- 172.21.15.99:0/852312450 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f32dc06a830 0x7f32dc15d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:11:23.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.950+0000 7f32e2213700 1 --2- 172.21.15.99:0/852312450 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32dc0cce30 0x7f32dc15aca0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f32dc089bc0 tx=0x7f32cc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9996025f2895f731 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:23.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.951+0000 7f32e1211700 1 -- 172.21.15.99:0/852312450 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32cc00e070 con 0x7f32dc0cce30 2023-10-08T00:11:23.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.951+0000 7f32e1211700 1 -- 172.21.15.99:0/852312450 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32cc00ba50 con 0x7f32dc0cce30 2023-10-08T00:11:23.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.951+0000 7f32e1211700 1 -- 172.21.15.99:0/852312450 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32cc0092e0 con 0x7f32dc0cce30 2023-10-08T00:11:23.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.951+0000 7f32e3215700 1 -- 172.21.15.99:0/852312450 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32dc0cce30 msgr2=0x7f32dc15aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:23.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.951+0000 7f32e3215700 1 --2- 172.21.15.99:0/852312450 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32dc0cce30 0x7f32dc15aca0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f32dc089bc0 tx=0x7f32cc009840 comp rx=0 tx=0).stop 2023-10-08T00:11:23.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.951+0000 7f32e3215700 1 -- 172.21.15.99:0/852312450 shutdown_connections 2023-10-08T00:11:23.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.951+0000 7f32e3215700 1 --2- 172.21.15.99:0/852312450 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f32dc06a830 0x7f32dc15d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:23.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.951+0000 7f32e3215700 1 --2- 172.21.15.99:0/852312450 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32dc0cce30 0x7f32dc15aca0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:23.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.951+0000 7f32e3215700 1 --2- 172.21.15.99:0/852312450 >> v2:172.21.15.191:3300/0 conn(0x7f32dc0b6580 0x7f32dc06a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:23.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.951+0000 7f32e3215700 1 -- 172.21.15.99:0/852312450 >> 172.21.15.99:0/852312450 conn(0x7f32dc068cb0 msgr2=0x7f32dc0690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:23.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.951+0000 7f32e3215700 1 -- 172.21.15.99:0/852312450 shutdown_connections 2023-10-08T00:11:23.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.951+0000 7f32e3215700 1 -- 172.21.15.99:0/852312450 wait complete. 2023-10-08T00:11:23.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.952+0000 7f32e3215700 1 Processor -- start 2023-10-08T00:11:23.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.952+0000 7f32e3215700 1 -- start start 2023-10-08T00:11:23.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.953+0000 7f32e3215700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f32dc06a830 0x7f32dc148fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:23.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.953+0000 7f32e3215700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32dc0b6580 0x7f32dc15ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:23.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.953+0000 7f32e3215700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32dc0cce30 0x7f32dc14e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:23.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.953+0000 7f32e3215700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32dc149510 con 0x7f32dc0cce30 2023-10-08T00:11:23.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.953+0000 7f32e3215700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32dc15c2c0 con 0x7f32dc0b6580 2023-10-08T00:11:23.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.953+0000 7f32e3215700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32dc15c5a0 con 0x7f32dc06a830 2023-10-08T00:11:23.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.953+0000 7f32e2a14700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32dc0cce30 0x7f32dc14e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:23.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.953+0000 7f32e2a14700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32dc0cce30 0x7f32dc14e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52098/0 (socket says 172.21.15.99:52098) 2023-10-08T00:11:23.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.953+0000 7f32e2a14700 1 -- 172.21.15.99:0/1797518829 learned_addr learned my addr 172.21.15.99:0/1797518829 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:23.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.953+0000 7f32e2213700 1 --2- 172.21.15.99:0/1797518829 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f32dc06a830 0x7f32dc148fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:23.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.953+0000 7f32e1a12700 1 --2- 172.21.15.99:0/1797518829 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32dc0b6580 0x7f32dc15ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:23.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.953+0000 7f32e2a14700 1 -- 172.21.15.99:0/1797518829 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f32dc06a830 msgr2=0x7f32dc148fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:23.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.953+0000 7f32e2a14700 1 --2- 172.21.15.99:0/1797518829 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f32dc06a830 0x7f32dc148fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:23.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.953+0000 7f32e2a14700 1 -- 172.21.15.99:0/1797518829 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32dc0b6580 msgr2=0x7f32dc15ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:23.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.953+0000 7f32e2a14700 1 --2- 172.21.15.99:0/1797518829 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32dc0b6580 0x7f32dc15ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:23.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.953+0000 7f32e2a14700 1 -- 172.21.15.99:0/1797518829 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32cc009ae0 con 0x7f32dc0cce30 2023-10-08T00:11:23.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.953+0000 7f32e1a12700 1 --2- 172.21.15.99:0/1797518829 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32dc0b6580 0x7f32dc15ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:11:23.960 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.953+0000 7f32e2a14700 1 --2- 172.21.15.99:0/1797518829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32dc0cce30 0x7f32dc14e740 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f32d400c880 tx=0x7f32d4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:23.960 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.954+0000 7f32da7fc700 1 -- 172.21.15.99:0/1797518829 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d4011600 con 0x7f32dc0cce30 2023-10-08T00:11:23.960 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.954+0000 7f32da7fc700 1 -- 172.21.15.99:0/1797518829 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32d4011780 con 0x7f32dc0cce30 2023-10-08T00:11:23.960 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.954+0000 7f32e3215700 1 -- 172.21.15.99:0/1797518829 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32dc0b0520 con 0x7f32dc0cce30 2023-10-08T00:11:23.960 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.954+0000 7f32da7fc700 1 -- 172.21.15.99:0/1797518829 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d4011b40 con 0x7f32dc0cce30 2023-10-08T00:11:23.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.954+0000 7f32e3215700 1 -- 172.21.15.99:0/1797518829 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32dc0b0ae0 con 0x7f32dc0cce30 2023-10-08T00:11:23.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.955+0000 7f32e3215700 1 -- 172.21.15.99:0/1797518829 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32dc089550 con 0x7f32dc0cce30 2023-10-08T00:11:23.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.955+0000 7f32da7fc700 1 -- 172.21.15.99:0/1797518829 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f32d4021350 con 0x7f32dc0cce30 2023-10-08T00:11:23.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.955+0000 7f32da7fc700 1 --2- 172.21.15.99:0/1797518829 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f32c8066f60 0x7f32c8069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:23.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.956+0000 7f32da7fc700 1 -- 172.21.15.99:0/1797518829 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f32d4084340 con 0x7f32dc0cce30 2023-10-08T00:11:23.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.956+0000 7f32e2213700 1 --2- 172.21.15.99:0/1797518829 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f32c8066f60 0x7f32c8069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:23.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.956+0000 7f32e2213700 1 --2- 172.21.15.99:0/1797518829 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f32c8066f60 0x7f32c8069410 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f32cc009880 tx=0x7f32cc018000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:23.964 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:23.959+0000 7f32da7fc700 1 -- 172.21.15.99:0/1797518829 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f32d4055040 con 0x7f32dc0cce30 2023-10-08T00:11:24.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.141+0000 7f32e3215700 1 -- 172.21.15.99:0/1797518829 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f32dc0b4720 con 0x7f32dc0cce30 2023-10-08T00:11:24.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.141+0000 7f32da7fc700 1 -- 172.21.15.99:0/1797518829 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f32d4054bd0 con 0x7f32dc0cce30 2023-10-08T00:11:24.142 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2023-10-08T00:11:24.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.143+0000 7f32e3215700 1 -- 172.21.15.99:0/1797518829 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f32c8066f60 msgr2=0x7f32c8069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:24.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.143+0000 7f32e3215700 1 --2- 172.21.15.99:0/1797518829 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f32c8066f60 0x7f32c8069410 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f32cc009880 tx=0x7f32cc018000 comp rx=0 tx=0).stop 2023-10-08T00:11:24.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.143+0000 7f32e3215700 1 -- 172.21.15.99:0/1797518829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32dc0cce30 msgr2=0x7f32dc14e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:24.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.143+0000 7f32e3215700 1 --2- 172.21.15.99:0/1797518829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32dc0cce30 0x7f32dc14e740 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f32d400c880 tx=0x7f32d4008040 comp rx=0 tx=0).stop 2023-10-08T00:11:24.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.143+0000 7f32e3215700 1 -- 172.21.15.99:0/1797518829 shutdown_connections 2023-10-08T00:11:24.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.143+0000 7f32e3215700 1 --2- 172.21.15.99:0/1797518829 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f32c8066f60 0x7f32c8069410 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:24.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.143+0000 7f32e3215700 1 --2- 172.21.15.99:0/1797518829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32dc0cce30 0x7f32dc14e740 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:24.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.143+0000 7f32e3215700 1 --2- 172.21.15.99:0/1797518829 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f32dc06a830 0x7f32dc148fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:24.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.143+0000 7f32e3215700 1 --2- 172.21.15.99:0/1797518829 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32dc0b6580 0x7f32dc15ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:24.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.143+0000 7f32e3215700 1 -- 172.21.15.99:0/1797518829 >> 172.21.15.99:0/1797518829 conn(0x7f32dc068cb0 msgr2=0x7f32dc0af8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:24.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.143+0000 7f32e3215700 1 -- 172.21.15.99:0/1797518829 shutdown_connections 2023-10-08T00:11:24.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.143+0000 7f32e3215700 1 -- 172.21.15.99:0/1797518829 wait complete. 2023-10-08T00:11:24.156 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T00:11:24.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.273+0000 7f277e84a700 1 Processor -- start 2023-10-08T00:11:24.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.274+0000 7f277e84a700 1 -- start start 2023-10-08T00:11:24.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.274+0000 7f277e84a700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2778156460 0x7f27780cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:24.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.274+0000 7f277e84a700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f27780cbc20 0x7f277806d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:24.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.275+0000 7f277e84a700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f277806e2b0 con 0x7f277806de70 2023-10-08T00:11:24.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.275+0000 7f277e84a700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2778156830 con 0x7f27780cbc20 2023-10-08T00:11:24.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.275+0000 7f27777fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f27780cbc20 0x7f277806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:24.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.275+0000 7f277e84a700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27780cc6d0 con 0x7f2778156460 2023-10-08T00:11:24.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.275+0000 7f27777fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f27780cbc20 0x7f277806d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52104/0 (socket says 172.21.15.99:52104) 2023-10-08T00:11:24.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.275+0000 7f2777fff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2778156460 0x7f27780cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:24.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.275+0000 7f27777fe700 1 -- 172.21.15.99:0/3350811257 learned_addr learned my addr 172.21.15.99:0/3350811257 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:24.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.275+0000 7f277cde7700 1 --1- 172.21.15.99:0/3350811257 >> v1:172.21.15.191:6789/0 conn(0x7f277806de70 0x7f2778159e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:43958/0 (socket says 172.21.15.99:43958) 2023-10-08T00:11:24.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.275+0000 7f27777fe700 1 -- 172.21.15.99:0/3350811257 >> v2:172.21.15.191:3300/0 conn(0x7f2778156460 msgr2=0x7f27780cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:24.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.275+0000 7f27777fe700 1 --2- 172.21.15.99:0/3350811257 >> v2:172.21.15.191:3300/0 conn(0x7f2778156460 0x7f27780cb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:24.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.275+0000 7f27777fe700 1 -- 172.21.15.99:0/3350811257 >> v1:172.21.15.191:6789/0 conn(0x7f277806de70 legacy=0x7f2778159e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:24.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.275+0000 7f27777fe700 1 -- 172.21.15.99:0/3350811257 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27780cc110 con 0x7f27780cbc20 2023-10-08T00:11:24.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.275+0000 7f27777fe700 1 --2- 172.21.15.99:0/3350811257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27780cbc20 0x7f277806d930 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f27780687f0 tx=0x7f276c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7af20512628fb534 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:24.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.275+0000 7f2776ffd700 1 -- 172.21.15.99:0/3350811257 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c009e70 con 0x7f27780cbc20 2023-10-08T00:11:24.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.275+0000 7f2776ffd700 1 -- 172.21.15.99:0/3350811257 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f276c00b860 con 0x7f27780cbc20 2023-10-08T00:11:24.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.275+0000 7f2776ffd700 1 -- 172.21.15.99:0/3350811257 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c00bc00 con 0x7f27780cbc20 2023-10-08T00:11:24.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.276+0000 7f277e84a700 1 -- 172.21.15.99:0/3350811257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27780cbc20 msgr2=0x7f277806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:24.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.276+0000 7f277e84a700 1 --2- 172.21.15.99:0/3350811257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27780cbc20 0x7f277806d930 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f27780687f0 tx=0x7f276c009250 comp rx=0 tx=0).stop 2023-10-08T00:11:24.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.276+0000 7f277e84a700 1 -- 172.21.15.99:0/3350811257 shutdown_connections 2023-10-08T00:11:24.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.276+0000 7f277e84a700 1 --2- 172.21.15.99:0/3350811257 >> v2:172.21.15.191:3300/0 conn(0x7f2778156460 0x7f27780cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:24.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.276+0000 7f277e84a700 1 --2- 172.21.15.99:0/3350811257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27780cbc20 0x7f277806d930 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:24.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.276+0000 7f277e84a700 1 -- 172.21.15.99:0/3350811257 >> 172.21.15.99:0/3350811257 conn(0x7f2778068170 msgr2=0x7f27780cd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:24.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.276+0000 7f277e84a700 1 -- 172.21.15.99:0/3350811257 shutdown_connections 2023-10-08T00:11:24.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.276+0000 7f277e84a700 1 -- 172.21.15.99:0/3350811257 wait complete. 2023-10-08T00:11:24.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.277+0000 7f277e84a700 1 Processor -- start 2023-10-08T00:11:24.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.277+0000 7f277e84a700 1 -- start start 2023-10-08T00:11:24.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.278+0000 7f277e84a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f277806de70 0x7f277815b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:24.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.278+0000 7f277e84a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27780cbc20 0x7f27780aa820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:24.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.278+0000 7f277e84a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2778156460 0x7f2778153640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:24.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.278+0000 7f277e84a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27780aea00 con 0x7f27780cbc20 2023-10-08T00:11:24.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.278+0000 7f277e84a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27780aece0 con 0x7f277806de70 2023-10-08T00:11:24.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.278+0000 7f277e84a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27780aefc0 con 0x7f2778156460 2023-10-08T00:11:24.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.278+0000 7f27777fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27780cbc20 0x7f27780aa820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:24.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.278+0000 7f27777fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27780cbc20 0x7f27780aa820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52112/0 (socket says 172.21.15.99:52112) 2023-10-08T00:11:24.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.278+0000 7f27777fe700 1 -- 172.21.15.99:0/1893553968 learned_addr learned my addr 172.21.15.99:0/1893553968 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:24.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.278+0000 7f277cde7700 1 --2- 172.21.15.99:0/1893553968 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2778156460 0x7f2778153640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:24.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.278+0000 7f2777fff700 1 --2- 172.21.15.99:0/1893553968 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f277806de70 0x7f277815b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:24.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.278+0000 7f27777fe700 1 -- 172.21.15.99:0/1893553968 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2778156460 msgr2=0x7f2778153640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:24.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.278+0000 7f27777fe700 1 --2- 172.21.15.99:0/1893553968 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2778156460 0x7f2778153640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:24.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.278+0000 7f27777fe700 1 -- 172.21.15.99:0/1893553968 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f277806de70 msgr2=0x7f277815b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:24.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.278+0000 7f27777fe700 1 --2- 172.21.15.99:0/1893553968 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f277806de70 0x7f277815b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:24.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.278+0000 7f27777fe700 1 -- 172.21.15.99:0/1893553968 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f276c0094f0 con 0x7f27780cbc20 2023-10-08T00:11:24.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.278+0000 7f27777fe700 1 --2- 172.21.15.99:0/1893553968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27780cbc20 0x7f27780aa820 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2778068870 tx=0x7f276c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:24.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.278+0000 7f275ffff700 1 -- 172.21.15.99:0/1893553968 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c011420 con 0x7f27780cbc20 2023-10-08T00:11:24.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.278+0000 7f275ffff700 1 -- 172.21.15.99:0/1893553968 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f276c011580 con 0x7f27780cbc20 2023-10-08T00:11:24.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.278+0000 7f277e84a700 1 -- 172.21.15.99:0/1893553968 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27780af2a0 con 0x7f27780cbc20 2023-10-08T00:11:24.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.278+0000 7f275ffff700 1 -- 172.21.15.99:0/1893553968 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c011940 con 0x7f27780cbc20 2023-10-08T00:11:24.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.278+0000 7f277e84a700 1 -- 172.21.15.99:0/1893553968 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f277815bee0 con 0x7f27780cbc20 2023-10-08T00:11:24.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.279+0000 7f277e84a700 1 -- 172.21.15.99:0/1893553968 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f277804c3c0 con 0x7f27780cbc20 2023-10-08T00:11:24.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.280+0000 7f275ffff700 1 -- 172.21.15.99:0/1893553968 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f276c02c3b0 con 0x7f27780cbc20 2023-10-08T00:11:24.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.280+0000 7f275ffff700 1 --2- 172.21.15.99:0/1893553968 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2764066f30 0x7f27640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:24.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.280+0000 7f275ffff700 1 -- 172.21.15.99:0/1893553968 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f276c085be0 con 0x7f27780cbc20 2023-10-08T00:11:24.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.280+0000 7f2777fff700 1 --2- 172.21.15.99:0/1893553968 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2764066f30 0x7f27640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:24.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.281+0000 7f2777fff700 1 --2- 172.21.15.99:0/1893553968 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2764066f30 0x7f27640693e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f276800aef0 tx=0x7f276800a330 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:24.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.284+0000 7f275ffff700 1 -- 172.21.15.99:0/1893553968 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f276c052e50 con 0x7f27780cbc20 2023-10-08T00:11:24.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.467+0000 7f277e84a700 1 -- 172.21.15.99:0/1893553968 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f27780c97a0 con 0x7f27780cbc20 2023-10-08T00:11:24.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.468+0000 7f275ffff700 1 -- 172.21.15.99:0/1893553968 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f276c056470 con 0x7f27780cbc20 2023-10-08T00:11:24.469 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2023-10-08T00:11:24.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.470+0000 7f277e84a700 1 -- 172.21.15.99:0/1893553968 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2764066f30 msgr2=0x7f27640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:24.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.470+0000 7f277e84a700 1 --2- 172.21.15.99:0/1893553968 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2764066f30 0x7f27640693e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f276800aef0 tx=0x7f276800a330 comp rx=0 tx=0).stop 2023-10-08T00:11:24.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.470+0000 7f277e84a700 1 -- 172.21.15.99:0/1893553968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27780cbc20 msgr2=0x7f27780aa820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:24.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.470+0000 7f277e84a700 1 --2- 172.21.15.99:0/1893553968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27780cbc20 0x7f27780aa820 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2778068870 tx=0x7f276c017000 comp rx=0 tx=0).stop 2023-10-08T00:11:24.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.470+0000 7f277e84a700 1 -- 172.21.15.99:0/1893553968 shutdown_connections 2023-10-08T00:11:24.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.470+0000 7f277e84a700 1 --2- 172.21.15.99:0/1893553968 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2764066f30 0x7f27640693e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:24.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.470+0000 7f277e84a700 1 --2- 172.21.15.99:0/1893553968 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f277806de70 0x7f277815b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:24.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.470+0000 7f277e84a700 1 --2- 172.21.15.99:0/1893553968 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2778156460 0x7f2778153640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:24.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.470+0000 7f277e84a700 1 --2- 172.21.15.99:0/1893553968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27780cbc20 0x7f27780aa820 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:24.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.470+0000 7f277e84a700 1 -- 172.21.15.99:0/1893553968 >> 172.21.15.99:0/1893553968 conn(0x7f2778068170 msgr2=0x7f27780c8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:24.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.470+0000 7f277e84a700 1 -- 172.21.15.99:0/1893553968 shutdown_connections 2023-10-08T00:11:24.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.470+0000 7f277e84a700 1 -- 172.21.15.99:0/1893553968 wait complete. 2023-10-08T00:11:24.514 INFO:tasks.cephfs_test_runner:Starting test: test_barrier (tasks.cephfs.test_full.TestClusterFull) That when an OSD epoch barrier is set on an MDS, subsequently 2023-10-08T00:11:24.514 INFO:tasks.cephfs_test_runner:test_barrier (tasks.cephfs.test_full.TestClusterFull) 2023-10-08T00:11:24.515 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_barrier' 2023-10-08T00:11:24.628 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.627+0000 7f2a8683a700 1 Processor -- start 2023-10-08T00:11:24.628 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.628+0000 7f2a8683a700 1 -- start start 2023-10-08T00:11:24.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.628+0000 7f2a8683a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a800ade00 0x7f2a80161b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:24.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.628+0000 7f2a8683a700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a800c9cb0 con 0x7f2a800abae0 2023-10-08T00:11:24.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.628+0000 7f2a8683a700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a800c90a0 con 0x7f2a800c75c0 2023-10-08T00:11:24.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.628+0000 7f2a8683a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a800c9380 con 0x7f2a800ade00 2023-10-08T00:11:24.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.628+0000 7f2a7ffff700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f2a800abae0 0x7f2a800c6eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:44018/0 (socket says 172.21.15.99:44018) 2023-10-08T00:11:24.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.629+0000 7f2a7ffff700 1 -- 172.21.15.99:0/2380653362 learned_addr learned my addr 172.21.15.99:0/2380653362 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:24.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.629+0000 7f2a7effd700 1 -- 172.21.15.99:0/2380653362 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2587120862 0 0) 0x7f2a800c9cb0 con 0x7f2a800abae0 2023-10-08T00:11:24.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.629+0000 7f2a84dd7700 1 --2- 172.21.15.99:0/2380653362 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a800ade00 0x7f2a80161b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:24.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.629+0000 7f2a7effd700 1 -- 172.21.15.99:0/2380653362 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a68003570 con 0x7f2a800abae0 2023-10-08T00:11:24.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.629+0000 7f2a7effd700 1 -- 172.21.15.99:0/2380653362 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2784281985 0 0) 0x7f2a68003570 con 0x7f2a800abae0 2023-10-08T00:11:24.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.629+0000 7f2a7effd700 1 -- 172.21.15.99:0/2380653362 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a800ade00 msgr2=0x7f2a80161b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:24.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.629+0000 7f2a7effd700 1 --2- 172.21.15.99:0/2380653362 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a800ade00 0x7f2a80161b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:24.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.629+0000 7f2a7effd700 1 -- 172.21.15.99:0/2380653362 >> v1:172.21.15.191:6789/0 conn(0x7f2a800c75c0 legacy=0x7f2a800c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:24.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.629+0000 7f2a7effd700 1 -- 172.21.15.99:0/2380653362 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a800c9660 con 0x7f2a800abae0 2023-10-08T00:11:24.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.629+0000 7f2a7effd700 1 -- 172.21.15.99:0/2380653362 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f2a70001470 con 0x7f2a800abae0 2023-10-08T00:11:24.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.629+0000 7f2a7effd700 1 -- 172.21.15.99:0/2380653362 >> v1:172.21.15.99:6789/0 conn(0x7f2a800abae0 legacy=0x7f2a800c6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:24.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.629+0000 7f2a7effd700 1 --2- 172.21.15.99:0/2380653362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a68003a90 0x7f2a68003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:24.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.629+0000 7f2a7effd700 1 --2- 172.21.15.99:0/2380653362 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a800ade00 0x7f2a68004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:24.635 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.629+0000 7f2a7effd700 1 --2- 172.21.15.99:0/2380653362 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a68004d50 0x7f2a680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:24.635 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.629+0000 7f2a7effd700 1 -- 172.21.15.99:0/2380653362 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a800c9380 con 0x7f2a68003a90 2023-10-08T00:11:24.635 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.629+0000 7f2a7effd700 1 -- 172.21.15.99:0/2380653362 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6800bba0 con 0x7f2a68004d50 2023-10-08T00:11:24.635 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.629+0000 7f2a7effd700 1 -- 172.21.15.99:0/2380653362 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6800bd60 con 0x7f2a800ade00 2023-10-08T00:11:24.635 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.629+0000 7f2a7ffff700 1 --2- 172.21.15.99:0/2380653362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a68003a90 0x7f2a68003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:24.635 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.629+0000 7f2a7f7fe700 1 --2- 172.21.15.99:0/2380653362 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a800ade00 0x7f2a68004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:24.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.629+0000 7f2a84dd7700 1 --2- 172.21.15.99:0/2380653362 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a68004d50 0x7f2a680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:24.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.629+0000 7f2a7ffff700 1 -- 172.21.15.99:0/2380653362 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a800ade00 msgr2=0x7f2a68004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:24.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.629+0000 7f2a7ffff700 1 --2- 172.21.15.99:0/2380653362 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a800ade00 0x7f2a68004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:24.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.630+0000 7f2a7ffff700 1 -- 172.21.15.99:0/2380653362 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a68004d50 msgr2=0x7f2a680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:24.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.630+0000 7f2a7ffff700 1 --2- 172.21.15.99:0/2380653362 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a68004d50 0x7f2a680051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:24.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.630+0000 7f2a7ffff700 1 -- 172.21.15.99:0/2380653362 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a68003570 con 0x7f2a68003a90 2023-10-08T00:11:24.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.630+0000 7f2a7f7fe700 1 --2- 172.21.15.99:0/2380653362 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a800ade00 0x7f2a68004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:11:24.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.630+0000 7f2a84dd7700 1 --2- 172.21.15.99:0/2380653362 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a68004d50 0x7f2a680051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:11:24.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.630+0000 7f2a7ffff700 1 --2- 172.21.15.99:0/2380653362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a68003a90 0x7f2a68003ea0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2a70007f60 tx=0x7f2a70001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=198d370aa5fce60b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:24.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.630+0000 7f2a7effd700 1 -- 172.21.15.99:0/2380653362 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7000cbf0 con 0x7f2a68003a90 2023-10-08T00:11:24.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.630+0000 7f2a7effd700 1 -- 172.21.15.99:0/2380653362 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a7000cd50 con 0x7f2a68003a90 2023-10-08T00:11:24.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.630+0000 7f2a7effd700 1 -- 172.21.15.99:0/2380653362 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a70015960 con 0x7f2a68003a90 2023-10-08T00:11:24.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.630+0000 7f2a8683a700 1 -- 172.21.15.99:0/2380653362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a68003a90 msgr2=0x7f2a68003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:24.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.630+0000 7f2a8683a700 1 --2- 172.21.15.99:0/2380653362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a68003a90 0x7f2a68003ea0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2a70007f60 tx=0x7f2a70001e50 comp rx=0 tx=0).stop 2023-10-08T00:11:24.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.630+0000 7f2a84dd7700 1 -- 172.21.15.99:0/2380653362 reap_dead start 2023-10-08T00:11:24.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.630+0000 7f2a8683a700 1 -- 172.21.15.99:0/2380653362 shutdown_connections 2023-10-08T00:11:24.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.630+0000 7f2a8683a700 1 -- 172.21.15.99:0/2380653362 >> 172.21.15.99:0/2380653362 conn(0x7f2a80067660 msgr2=0x7f2a80067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:24.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.630+0000 7f2a8683a700 1 -- 172.21.15.99:0/2380653362 shutdown_connections 2023-10-08T00:11:24.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.630+0000 7f2a8683a700 1 -- 172.21.15.99:0/2380653362 wait complete. 2023-10-08T00:11:24.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.631+0000 7f2a8683a700 1 Processor -- start 2023-10-08T00:11:24.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.632+0000 7f2a8683a700 1 -- start start 2023-10-08T00:11:24.641 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.632+0000 7f2a8683a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a68003a90 0x7f2a801541c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:24.642 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.632+0000 7f2a8683a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a801548b0 0x7f2a80154d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:24.642 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.632+0000 7f2a8683a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a80159750 0x7f2a80159bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:24.642 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.632+0000 7f2a8683a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a80154700 con 0x7f2a801548b0 2023-10-08T00:11:24.642 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.632+0000 7f2a8683a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8015a100 con 0x7f2a68003a90 2023-10-08T00:11:24.642 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.632+0000 7f2a8683a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a800cbc20 con 0x7f2a80159750 2023-10-08T00:11:24.642 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.632+0000 7f2a7f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a801548b0 0x7f2a80154d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:24.642 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.633+0000 7f2a7f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a801548b0 0x7f2a80154d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52122/0 (socket says 172.21.15.99:52122) 2023-10-08T00:11:24.642 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.633+0000 7f2a7f7fe700 1 -- 172.21.15.99:0/156005784 learned_addr learned my addr 172.21.15.99:0/156005784 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:24.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.633+0000 7f2a84dd7700 1 --2- 172.21.15.99:0/156005784 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a80159750 0x7f2a80159bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:24.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.633+0000 7f2a7ffff700 1 --2- 172.21.15.99:0/156005784 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a68003a90 0x7f2a801541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:24.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.633+0000 7f2a7f7fe700 1 -- 172.21.15.99:0/156005784 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a80159750 msgr2=0x7f2a80159bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:24.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.633+0000 7f2a7f7fe700 1 --2- 172.21.15.99:0/156005784 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a80159750 0x7f2a80159bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:24.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.633+0000 7f2a7f7fe700 1 -- 172.21.15.99:0/156005784 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a68003a90 msgr2=0x7f2a801541c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:24.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.633+0000 7f2a7f7fe700 1 --2- 172.21.15.99:0/156005784 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a68003a90 0x7f2a801541c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:24.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.633+0000 7f2a7f7fe700 1 -- 172.21.15.99:0/156005784 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a800b08a0 con 0x7f2a801548b0 2023-10-08T00:11:24.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.633+0000 7f2a7f7fe700 1 --2- 172.21.15.99:0/156005784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a801548b0 0x7f2a80154d20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2a7400e920 tx=0x7f2a74009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:24.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.633+0000 7f2a67fff700 1 -- 172.21.15.99:0/156005784 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7400b070 con 0x7f2a801548b0 2023-10-08T00:11:24.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.633+0000 7f2a67fff700 1 -- 172.21.15.99:0/156005784 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a7400ee60 con 0x7f2a801548b0 2023-10-08T00:11:24.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.633+0000 7f2a8683a700 1 -- 172.21.15.99:0/156005784 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a800cb380 con 0x7f2a801548b0 2023-10-08T00:11:24.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.633+0000 7f2a67fff700 1 -- 172.21.15.99:0/156005784 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7400ca70 con 0x7f2a801548b0 2023-10-08T00:11:24.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.633+0000 7f2a8683a700 1 -- 172.21.15.99:0/156005784 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a800cb940 con 0x7f2a801548b0 2023-10-08T00:11:24.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.634+0000 7f2a8683a700 1 -- 172.21.15.99:0/156005784 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a800b0a30 con 0x7f2a801548b0 2023-10-08T00:11:24.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.634+0000 7f2a67fff700 1 -- 172.21.15.99:0/156005784 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2a74011470 con 0x7f2a801548b0 2023-10-08T00:11:24.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.635+0000 7f2a67fff700 1 --2- 172.21.15.99:0/156005784 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2a6c066f30 0x7f2a6c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:24.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.635+0000 7f2a67fff700 1 -- 172.21.15.99:0/156005784 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2a74084590 con 0x7f2a801548b0 2023-10-08T00:11:24.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.635+0000 7f2a7ffff700 1 --2- 172.21.15.99:0/156005784 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2a6c066f30 0x7f2a6c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:24.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.635+0000 7f2a7ffff700 1 --2- 172.21.15.99:0/156005784 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2a6c066f30 0x7f2a6c0693e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2a70007cd0 tx=0x7f2a70018000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:24.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.639+0000 7f2a67fff700 1 -- 172.21.15.99:0/156005784 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2a74051800 con 0x7f2a801548b0 2023-10-08T00:11:24.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:24.897+0000 7f2a8683a700 1 -- 172.21.15.99:0/156005784 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_barrier"]} v 0) v1 -- 0x7f2a8004c3c0 con 0x7f2a801548b0 2023-10-08T00:11:25.243 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.243+0000 7f2a67fff700 1 -- 172.21.15.99:0/156005784 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_barrier"]}]=0 v46) v1 ==== 133+0+0 (secure 0 0 0) 0x7f2a74054e20 con 0x7f2a801548b0 2023-10-08T00:11:25.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.245+0000 7f2a8683a700 1 -- 172.21.15.99:0/156005784 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2a6c066f30 msgr2=0x7f2a6c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:25.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.245+0000 7f2a8683a700 1 --2- 172.21.15.99:0/156005784 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2a6c066f30 0x7f2a6c0693e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2a70007cd0 tx=0x7f2a70018000 comp rx=0 tx=0).stop 2023-10-08T00:11:25.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.245+0000 7f2a8683a700 1 -- 172.21.15.99:0/156005784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a801548b0 msgr2=0x7f2a80154d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:25.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.245+0000 7f2a8683a700 1 --2- 172.21.15.99:0/156005784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a801548b0 0x7f2a80154d20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2a7400e920 tx=0x7f2a74009040 comp rx=0 tx=0).stop 2023-10-08T00:11:25.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.245+0000 7f2a8683a700 1 -- 172.21.15.99:0/156005784 shutdown_connections 2023-10-08T00:11:25.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.245+0000 7f2a8683a700 1 --2- 172.21.15.99:0/156005784 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2a6c066f30 0x7f2a6c0693e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:25.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.245+0000 7f2a8683a700 1 --2- 172.21.15.99:0/156005784 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a68003a90 0x7f2a801541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:25.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.245+0000 7f2a8683a700 1 --2- 172.21.15.99:0/156005784 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a80159750 0x7f2a80159bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:25.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.245+0000 7f2a8683a700 1 --2- 172.21.15.99:0/156005784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a801548b0 0x7f2a80154d20 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:25.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.245+0000 7f2a8683a700 1 -- 172.21.15.99:0/156005784 >> 172.21.15.99:0/156005784 conn(0x7f2a80067660 msgr2=0x7f2a80068fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:25.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.245+0000 7f2a8683a700 1 -- 172.21.15.99:0/156005784 shutdown_connections 2023-10-08T00:11:25.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.245+0000 7f2a8683a700 1 -- 172.21.15.99:0/156005784 wait complete. 2023-10-08T00:11:25.258 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2023-10-08T00:11:25.387 INFO:teuthology.orchestra.run.smithi099.stdout:{"osd_objectstore":"memstore"} 2023-10-08T00:11:25.387 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2023-10-08T00:11:25.388 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T00:11:25.503 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.503+0000 7f72237de700 1 Processor -- start 2023-10-08T00:11:25.504 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.503+0000 7f72237de700 1 -- start start 2023-10-08T00:11:25.507 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.504+0000 7f72237de700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f721c0affe0 0x7f721c0c80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:25.507 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.504+0000 7f72237de700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f721c068880 0x7f721c06ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:25.508 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.504+0000 7f72237de700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f721c06f980 con 0x7f721c070700 2023-10-08T00:11:25.508 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.504+0000 7f72237de700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f721c06f230 con 0x7f721c068880 2023-10-08T00:11:25.508 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.504+0000 7f72237de700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f721c0cd8c0 con 0x7f721c0affe0 2023-10-08T00:11:25.508 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.504+0000 7f7221d7b700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f721c070700 0x7f721c15dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:44024/0 (socket says 172.21.15.99:44024) 2023-10-08T00:11:25.508 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.504+0000 7f7221d7b700 1 -- 172.21.15.99:0/2966901186 learned_addr learned my addr 172.21.15.99:0/2966901186 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:25.508 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.504+0000 7f722157a700 1 --2- 172.21.15.99:0/2966901186 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f721c0affe0 0x7f721c0c80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:25.508 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.504+0000 7f7220d79700 1 --2- 172.21.15.99:0/2966901186 >> v2:172.21.15.191:3300/0 conn(0x7f721c068880 0x7f721c06ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:25.508 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.504+0000 7f7213fff700 1 -- 172.21.15.99:0/2966901186 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 647726710 0 0) 0x7f721c06f980 con 0x7f721c070700 2023-10-08T00:11:25.508 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.504+0000 7f7213fff700 1 -- 172.21.15.99:0/2966901186 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7208003570 con 0x7f721c070700 2023-10-08T00:11:25.508 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.504+0000 7f7213fff700 1 -- 172.21.15.99:0/2966901186 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1476905839 0 0) 0x7f7208003570 con 0x7f721c070700 2023-10-08T00:11:25.508 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.504+0000 7f7213fff700 1 -- 172.21.15.99:0/2966901186 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f721c0affe0 msgr2=0x7f721c0c80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:25.509 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.504+0000 7f7213fff700 1 --2- 172.21.15.99:0/2966901186 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f721c0affe0 0x7f721c0c80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:25.509 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.504+0000 7f7213fff700 1 -- 172.21.15.99:0/2966901186 >> v2:172.21.15.191:3300/0 conn(0x7f721c068880 msgr2=0x7f721c06ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:25.509 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.504+0000 7f7213fff700 1 --2- 172.21.15.99:0/2966901186 >> v2:172.21.15.191:3300/0 conn(0x7f721c068880 0x7f721c06ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:25.509 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.504+0000 7f7213fff700 1 -- 172.21.15.99:0/2966901186 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f721c0ccfd0 con 0x7f721c070700 2023-10-08T00:11:25.509 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.504+0000 7f7213fff700 1 -- 172.21.15.99:0/2966901186 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f72180017f0 con 0x7f721c070700 2023-10-08T00:11:25.509 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.504+0000 7f7213fff700 1 -- 172.21.15.99:0/2966901186 >> v1:172.21.15.99:6789/0 conn(0x7f721c070700 legacy=0x7f721c15dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:25.509 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.504+0000 7f7213fff700 1 --2- 172.21.15.99:0/2966901186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7208003a60 0x7f7208003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:25.509 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.504+0000 7f7213fff700 1 --2- 172.21.15.99:0/2966901186 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f721c0affe0 0x7f7208004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:25.509 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.505+0000 7f7213fff700 1 --2- 172.21.15.99:0/2966901186 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7208004d50 0x7f72080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:25.509 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.505+0000 7f7213fff700 1 -- 172.21.15.99:0/2966901186 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721c06f230 con 0x7f7208003a60 2023-10-08T00:11:25.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.505+0000 7f7213fff700 1 -- 172.21.15.99:0/2966901186 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721c0cd8c0 con 0x7f7208004d50 2023-10-08T00:11:25.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.505+0000 7f7213fff700 1 -- 172.21.15.99:0/2966901186 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f720800ba60 con 0x7f721c0affe0 2023-10-08T00:11:25.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.505+0000 7f722157a700 1 --2- 172.21.15.99:0/2966901186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7208003a60 0x7f7208003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:25.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.505+0000 7f7220d79700 1 --2- 172.21.15.99:0/2966901186 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f721c0affe0 0x7f7208004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:25.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.505+0000 7f7221d7b700 1 --2- 172.21.15.99:0/2966901186 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7208004d50 0x7f72080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:25.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.505+0000 7f722157a700 1 -- 172.21.15.99:0/2966901186 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f721c0affe0 msgr2=0x7f7208004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:25.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.505+0000 7f722157a700 1 --2- 172.21.15.99:0/2966901186 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f721c0affe0 0x7f7208004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:25.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.505+0000 7f722157a700 1 -- 172.21.15.99:0/2966901186 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7208004d50 msgr2=0x7f72080051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:25.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.505+0000 7f722157a700 1 --2- 172.21.15.99:0/2966901186 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7208004d50 0x7f72080051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:25.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.505+0000 7f722157a700 1 -- 172.21.15.99:0/2966901186 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7208003570 con 0x7f7208003a60 2023-10-08T00:11:25.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.505+0000 7f722157a700 1 --2- 172.21.15.99:0/2966901186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7208003a60 0x7f7208003e70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7214009c00 tx=0x7f72140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b889391d900c949 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:25.511 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.505+0000 7f7213fff700 1 -- 172.21.15.99:0/2966901186 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721400e6b0 con 0x7f7208003a60 2023-10-08T00:11:25.511 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.505+0000 7f7213fff700 1 -- 172.21.15.99:0/2966901186 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f721400e810 con 0x7f7208003a60 2023-10-08T00:11:25.511 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.505+0000 7f7213fff700 1 -- 172.21.15.99:0/2966901186 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721400ebd0 con 0x7f7208003a60 2023-10-08T00:11:25.511 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.506+0000 7f72237de700 1 -- 172.21.15.99:0/2966901186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7208003a60 msgr2=0x7f7208003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:25.511 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.506+0000 7f72237de700 1 --2- 172.21.15.99:0/2966901186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7208003a60 0x7f7208003e70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7214009c00 tx=0x7f72140091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:25.511 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.506+0000 7f7221d7b700 1 -- 172.21.15.99:0/2966901186 reap_dead start 2023-10-08T00:11:25.511 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.506+0000 7f72237de700 1 -- 172.21.15.99:0/2966901186 shutdown_connections 2023-10-08T00:11:25.511 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.506+0000 7f72237de700 1 -- 172.21.15.99:0/2966901186 >> 172.21.15.99:0/2966901186 conn(0x7f721c0ce570 msgr2=0x7f721c0ca510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:25.511 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.506+0000 7f72237de700 1 -- 172.21.15.99:0/2966901186 shutdown_connections 2023-10-08T00:11:25.511 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.506+0000 7f72237de700 1 -- 172.21.15.99:0/2966901186 wait complete. 2023-10-08T00:11:25.512 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.507+0000 7f72237de700 1 Processor -- start 2023-10-08T00:11:25.512 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.507+0000 7f72237de700 1 -- start start 2023-10-08T00:11:25.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.508+0000 7f72237de700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7208003a60 0x7f721c155d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:25.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.508+0000 7f72237de700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f721c0b15f0 0x7f721c0b1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:25.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.508+0000 7f72237de700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f721c0c5f70 0x7f721c0c6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:25.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.508+0000 7f72237de700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721c16abc0 con 0x7f7208003a60 2023-10-08T00:11:25.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.508+0000 7f72237de700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721c0c6950 con 0x7f721c0b15f0 2023-10-08T00:11:25.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.508+0000 7f72237de700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f721c16a8e0 con 0x7f721c0c5f70 2023-10-08T00:11:25.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.508+0000 7f722157a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7208003a60 0x7f721c155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:25.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.508+0000 7f722157a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7208003a60 0x7f721c155d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52142/0 (socket says 172.21.15.99:52142) 2023-10-08T00:11:25.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.508+0000 7f722157a700 1 -- 172.21.15.99:0/2795036389 learned_addr learned my addr 172.21.15.99:0/2795036389 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:25.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.508+0000 7f7221d7b700 1 --2- 172.21.15.99:0/2795036389 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f721c0c5f70 0x7f721c0c6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:25.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.508+0000 7f7220d79700 1 --2- 172.21.15.99:0/2795036389 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f721c0b15f0 0x7f721c0b1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:25.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.508+0000 7f722157a700 1 -- 172.21.15.99:0/2795036389 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f721c0c5f70 msgr2=0x7f721c0c6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:25.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.508+0000 7f722157a700 1 --2- 172.21.15.99:0/2795036389 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f721c0c5f70 0x7f721c0c6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:25.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.508+0000 7f722157a700 1 -- 172.21.15.99:0/2795036389 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f721c0b15f0 msgr2=0x7f721c0b1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:25.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.508+0000 7f722157a700 1 --2- 172.21.15.99:0/2795036389 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f721c0b15f0 0x7f721c0b1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:25.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.508+0000 7f722157a700 1 -- 172.21.15.99:0/2795036389 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f721c0b0b10 con 0x7f7208003a60 2023-10-08T00:11:25.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.508+0000 7f7220d79700 1 --2- 172.21.15.99:0/2795036389 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f721c0b15f0 0x7f721c0b1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:11:25.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.508+0000 7f7221d7b700 1 --2- 172.21.15.99:0/2795036389 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f721c0c5f70 0x7f721c0c6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:11:25.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.508+0000 7f722157a700 1 --2- 172.21.15.99:0/2795036389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7208003a60 0x7f721c155d40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7208004730 tx=0x7f721401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:25.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.509+0000 7f72117fa700 1 -- 172.21.15.99:0/2795036389 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7214022070 con 0x7f7208003a60 2023-10-08T00:11:25.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.509+0000 7f72117fa700 1 -- 172.21.15.99:0/2795036389 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f721400e980 con 0x7f7208003a60 2023-10-08T00:11:25.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.509+0000 7f72237de700 1 -- 172.21.15.99:0/2795036389 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f721c0b0df0 con 0x7f7208003a60 2023-10-08T00:11:25.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.509+0000 7f72117fa700 1 -- 172.21.15.99:0/2795036389 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7214027470 con 0x7f7208003a60 2023-10-08T00:11:25.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.509+0000 7f72237de700 1 -- 172.21.15.99:0/2795036389 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f721c156280 con 0x7f7208003a60 2023-10-08T00:11:25.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.509+0000 7f72237de700 1 -- 172.21.15.99:0/2795036389 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f721c0b0970 con 0x7f7208003a60 2023-10-08T00:11:25.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.510+0000 7f72117fa700 1 -- 172.21.15.99:0/2795036389 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f721401c450 con 0x7f7208003a60 2023-10-08T00:11:25.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.510+0000 7f72117fa700 1 --2- 172.21.15.99:0/2795036389 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7200066f30 0x7f72000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:25.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.510+0000 7f72117fa700 1 -- 172.21.15.99:0/2795036389 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7214086180 con 0x7f7208003a60 2023-10-08T00:11:25.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.511+0000 7f7220d79700 1 --2- 172.21.15.99:0/2795036389 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7200066f30 0x7f72000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:25.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.511+0000 7f7220d79700 1 --2- 172.21.15.99:0/2795036389 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7200066f30 0x7f72000693e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f720c006dc0 tx=0x7f720c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:25.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.515+0000 7f72117fa700 1 -- 172.21.15.99:0/2795036389 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f721c0b0970 con 0x7f7208003a60 2023-10-08T00:11:25.696 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.696+0000 7f72237de700 1 -- 172.21.15.99:0/2795036389 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f721c0b0970 con 0x7f7208003a60 2023-10-08T00:11:25.699 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.699+0000 7f72117fa700 1 -- 172.21.15.99:0/2795036389 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v2)=0 v2) v1 ==== 122+0+0 (secure 0 0 0) 0x7f721c0b0970 con 0x7f7208003a60 2023-10-08T00:11:25.702 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.701+0000 7f72237de700 1 -- 172.21.15.99:0/2795036389 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7200066f30 msgr2=0x7f72000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:25.702 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.701+0000 7f72237de700 1 --2- 172.21.15.99:0/2795036389 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7200066f30 0x7f72000693e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f720c006dc0 tx=0x7f720c008040 comp rx=0 tx=0).stop 2023-10-08T00:11:25.703 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.701+0000 7f72237de700 1 -- 172.21.15.99:0/2795036389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7208003a60 msgr2=0x7f721c155d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:25.703 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.701+0000 7f72237de700 1 --2- 172.21.15.99:0/2795036389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7208003a60 0x7f721c155d40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7208004730 tx=0x7f721401a000 comp rx=0 tx=0).stop 2023-10-08T00:11:25.703 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.701+0000 7f72237de700 1 -- 172.21.15.99:0/2795036389 shutdown_connections 2023-10-08T00:11:25.703 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.701+0000 7f72237de700 1 --2- 172.21.15.99:0/2795036389 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7200066f30 0x7f72000693e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:25.703 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.702+0000 7f72237de700 1 --2- 172.21.15.99:0/2795036389 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f721c0c5f70 0x7f721c0c6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:25.703 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.702+0000 7f72237de700 1 --2- 172.21.15.99:0/2795036389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7208003a60 0x7f721c155d40 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:25.703 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.702+0000 7f72237de700 1 --2- 172.21.15.99:0/2795036389 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f721c0b15f0 0x7f721c0b1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:25.703 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.702+0000 7f72237de700 1 -- 172.21.15.99:0/2795036389 >> 172.21.15.99:0/2795036389 conn(0x7f721c0ce570 msgr2=0x7f721c069dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:25.703 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.702+0000 7f72237de700 1 -- 172.21.15.99:0/2795036389 shutdown_connections 2023-10-08T00:11:25.703 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:25.702+0000 7f72237de700 1 -- 172.21.15.99:0/2795036389 wait complete. 2023-10-08T00:11:25.714 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T00:11:25.714 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T00:11:25.751 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T00:11:25.796 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T00:11:25.796 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T00:11:25.796 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:11:25.796 DEBUG:teuthology.orchestra.run.smithi099:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-10-08T00:11:25.832 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2023-10-08T00:11:25.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:25.833 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T00:11:25.887 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-08T00:11:25.887 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:11:25.887 DEBUG:teuthology.orchestra.run.smithi191:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-10-08T00:11:25.942 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2023-10-08T00:11:25.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:25.942 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-08T00:11:25.959 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-10-08T00:11:25.959 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:11:25.959 DEBUG:teuthology.orchestra.run.smithi099:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-10-08T00:11:25.997 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2023-10-08T00:11:25.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:25.997 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-08T00:11:26.052 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-10-08T00:11:26.052 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:11:26.052 DEBUG:teuthology.orchestra.run.smithi191:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-10-08T00:11:26.092 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2023-10-08T00:11:26.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:26.093 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-08T00:11:26.149 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-08T00:11:26.149 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:11:26.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.263+0000 7f9d26ec7700 1 Processor -- start 2023-10-08T00:11:26.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.263+0000 7f9d26ec7700 1 -- start start 2023-10-08T00:11:26.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.264+0000 7f9d26ec7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9d20159e60 0x7f9d200cb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:26.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.264+0000 7f9d26ec7700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f9d200cb8d0 0x7f9d2015d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:26.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.264+0000 7f9d26ec7700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d2015b020 con 0x7f9d20159a90 2023-10-08T00:11:26.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.264+0000 7f9d26ec7700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d200cbd50 con 0x7f9d200cb8d0 2023-10-08T00:11:26.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.264+0000 7f9d26ec7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2015a450 con 0x7f9d20159e60 2023-10-08T00:11:26.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.264+0000 7f9d25ec5700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f9d20159a90 0x7f9d20068e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:44036/0 (socket says 172.21.15.99:44036) 2023-10-08T00:11:26.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.264+0000 7f9d25ec5700 1 -- 172.21.15.99:0/1971511134 learned_addr learned my addr 172.21.15.99:0/1971511134 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:26.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.264+0000 7f9d256c4700 1 --2- 172.21.15.99:0/1971511134 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9d20159e60 0x7f9d200cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:26.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.264+0000 7f9d266c6700 1 --2- 172.21.15.99:0/1971511134 >> v2:172.21.15.191:3300/0 conn(0x7f9d200cb8d0 0x7f9d2015d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:26.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.265+0000 7f9d24ec3700 1 -- 172.21.15.99:0/1971511134 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1840399236 0 0) 0x7f9d2015b020 con 0x7f9d20159a90 2023-10-08T00:11:26.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.265+0000 7f9d24ec3700 1 -- 172.21.15.99:0/1971511134 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d0c003570 con 0x7f9d20159a90 2023-10-08T00:11:26.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.265+0000 7f9d256c4700 1 -- 172.21.15.99:0/1971511134 >> v2:172.21.15.191:3300/0 conn(0x7f9d200cb8d0 msgr2=0x7f9d2015d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:26.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.265+0000 7f9d256c4700 1 --2- 172.21.15.99:0/1971511134 >> v2:172.21.15.191:3300/0 conn(0x7f9d200cb8d0 0x7f9d2015d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:26.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.265+0000 7f9d256c4700 1 -- 172.21.15.99:0/1971511134 >> v1:172.21.15.99:6789/0 conn(0x7f9d20159a90 legacy=0x7f9d20068e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:26.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.265+0000 7f9d256c4700 1 -- 172.21.15.99:0/1971511134 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d20068670 con 0x7f9d20159e60 2023-10-08T00:11:26.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.265+0000 7f9d266c6700 1 --2- 172.21.15.99:0/1971511134 >> v2:172.21.15.191:3300/0 conn(0x7f9d200cb8d0 0x7f9d2015d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:11:26.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.265+0000 7f9d256c4700 1 --2- 172.21.15.99:0/1971511134 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9d20159e60 0x7f9d200cb390 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9d100098e0 tx=0x7f9d100091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb2aa04b8a5c5936 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:26.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.265+0000 7f9d24ec3700 1 -- 172.21.15.99:0/1971511134 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d10011630 con 0x7f9d20159e60 2023-10-08T00:11:26.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.265+0000 7f9d24ec3700 1 -- 172.21.15.99:0/1971511134 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d10011790 con 0x7f9d20159e60 2023-10-08T00:11:26.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.265+0000 7f9d24ec3700 1 -- 172.21.15.99:0/1971511134 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d10011b30 con 0x7f9d20159e60 2023-10-08T00:11:26.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.266+0000 7f9d26ec7700 1 -- 172.21.15.99:0/1971511134 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9d20159e60 msgr2=0x7f9d200cb390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:26.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.266+0000 7f9d26ec7700 1 --2- 172.21.15.99:0/1971511134 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9d20159e60 0x7f9d200cb390 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9d100098e0 tx=0x7f9d100091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:26.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.266+0000 7f9d26ec7700 1 -- 172.21.15.99:0/1971511134 shutdown_connections 2023-10-08T00:11:26.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.266+0000 7f9d26ec7700 1 --2- 172.21.15.99:0/1971511134 >> v2:172.21.15.191:3300/0 conn(0x7f9d200cb8d0 0x7f9d2015d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:26.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.266+0000 7f9d26ec7700 1 --2- 172.21.15.99:0/1971511134 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9d20159e60 0x7f9d200cb390 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:26.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.266+0000 7f9d26ec7700 1 -- 172.21.15.99:0/1971511134 >> 172.21.15.99:0/1971511134 conn(0x7f9d20069870 msgr2=0x7f9d20069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:26.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.266+0000 7f9d26ec7700 1 -- 172.21.15.99:0/1971511134 shutdown_connections 2023-10-08T00:11:26.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.266+0000 7f9d26ec7700 1 -- 172.21.15.99:0/1971511134 wait complete. 2023-10-08T00:11:26.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.267+0000 7f9d26ec7700 1 Processor -- start 2023-10-08T00:11:26.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.267+0000 7f9d26ec7700 1 -- start start 2023-10-08T00:11:26.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.268+0000 7f9d26ec7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d200cb8d0 0x7f9d200c23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:26.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.268+0000 7f9d26ec7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9d20159a90 0x7f9d200c2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:26.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.268+0000 7f9d26ec7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d20159e60 0x7f9d200afe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:26.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.268+0000 7f9d26ec7700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d200c2e70 con 0x7f9d200cb8d0 2023-10-08T00:11:26.275 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.268+0000 7f9d26ec7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d200b2ca0 con 0x7f9d20159e60 2023-10-08T00:11:26.275 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.268+0000 7f9d26ec7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d200b2090 con 0x7f9d20159a90 2023-10-08T00:11:26.275 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.268+0000 7f9d25ec5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d200cb8d0 0x7f9d200c23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:26.275 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.268+0000 7f9d25ec5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d200cb8d0 0x7f9d200c23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52154/0 (socket says 172.21.15.99:52154) 2023-10-08T00:11:26.275 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.268+0000 7f9d25ec5700 1 -- 172.21.15.99:0/2841074688 learned_addr learned my addr 172.21.15.99:0/2841074688 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:26.275 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.268+0000 7f9d266c6700 1 --2- 172.21.15.99:0/2841074688 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d20159e60 0x7f9d200afe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:26.275 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.268+0000 7f9d256c4700 1 --2- 172.21.15.99:0/2841074688 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9d20159a90 0x7f9d200c2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:26.275 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.268+0000 7f9d25ec5700 1 -- 172.21.15.99:0/2841074688 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9d20159a90 msgr2=0x7f9d200c2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:26.275 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.268+0000 7f9d25ec5700 1 --2- 172.21.15.99:0/2841074688 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9d20159a90 0x7f9d200c2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:26.275 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.268+0000 7f9d25ec5700 1 -- 172.21.15.99:0/2841074688 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d20159e60 msgr2=0x7f9d200afe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:26.275 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.268+0000 7f9d25ec5700 1 --2- 172.21.15.99:0/2841074688 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d20159e60 0x7f9d200afe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:26.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.268+0000 7f9d25ec5700 1 -- 172.21.15.99:0/2841074688 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d18009a20 con 0x7f9d200cb8d0 2023-10-08T00:11:26.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.268+0000 7f9d25ec5700 1 --2- 172.21.15.99:0/2841074688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d200cb8d0 0x7f9d200c23f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c00c7c0 tx=0x7f9d1c007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:26.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.268+0000 7f9d15ffb700 1 -- 172.21.15.99:0/2841074688 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c005b30 con 0x7f9d200cb8d0 2023-10-08T00:11:26.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.268+0000 7f9d15ffb700 1 -- 172.21.15.99:0/2841074688 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d1c005c90 con 0x7f9d200cb8d0 2023-10-08T00:11:26.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.268+0000 7f9d26ec7700 1 -- 172.21.15.99:0/2841074688 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d10009420 con 0x7f9d200cb8d0 2023-10-08T00:11:26.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.268+0000 7f9d15ffb700 1 -- 172.21.15.99:0/2841074688 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c007340 con 0x7f9d200cb8d0 2023-10-08T00:11:26.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.269+0000 7f9d26ec7700 1 -- 172.21.15.99:0/2841074688 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d200b2650 con 0x7f9d200cb8d0 2023-10-08T00:11:26.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.270+0000 7f9d26ec7700 1 -- 172.21.15.99:0/2841074688 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d2006e3e0 con 0x7f9d200cb8d0 2023-10-08T00:11:26.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.270+0000 7f9d15ffb700 1 -- 172.21.15.99:0/2841074688 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9d1c01b400 con 0x7f9d200cb8d0 2023-10-08T00:11:26.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.270+0000 7f9d15ffb700 1 --2- 172.21.15.99:0/2841074688 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9d04066f30 0x7f9d040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:26.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.270+0000 7f9d15ffb700 1 -- 172.21.15.99:0/2841074688 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9d1c084040 con 0x7f9d200cb8d0 2023-10-08T00:11:26.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.270+0000 7f9d256c4700 1 --2- 172.21.15.99:0/2841074688 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9d04066f30 0x7f9d040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:26.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.271+0000 7f9d256c4700 1 --2- 172.21.15.99:0/2841074688 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9d04066f30 0x7f9d040693e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9d20154ed0 tx=0x7f9d10017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:26.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.275+0000 7f9d15ffb700 1 -- 172.21.15.99:0/2841074688 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9d1c051330 con 0x7f9d200cb8d0 2023-10-08T00:11:26.497 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.497+0000 7f9d26ec7700 1 -- 172.21.15.99:0/2841074688 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d200b1e70 con 0x7f9d200cb8d0 2023-10-08T00:11:26.498 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.497+0000 7f9d15ffb700 1 -- 172.21.15.99:0/2841074688 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4594 (secure 0 0 0) 0x7f9d1c054950 con 0x7f9d200cb8d0 2023-10-08T00:11:26.498 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:11:26.498 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/649537180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":649537180},{"type":"v1","addr":"172.21.15.99:6833","nonce":649537180}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4239,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2194110899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2194110899},{"type":"v1","addr":"172.21.15.99:6835","nonce":2194110899}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4251,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/1732443753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":1732443753},{"type":"v1","addr":"172.21.15.191:6837","nonce":1732443753}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:10:53.454321+0000","modified":"2023-10-08T00:10:57.098731+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6836/1988845948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1988845948},{"type":"v1","addr":"172.21.15.191:6836","nonce":1988845948}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-08T00:11:26.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.499+0000 7f9d26ec7700 1 -- 172.21.15.99:0/2841074688 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9d04066f30 msgr2=0x7f9d040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:26.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.499+0000 7f9d26ec7700 1 --2- 172.21.15.99:0/2841074688 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9d04066f30 0x7f9d040693e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9d20154ed0 tx=0x7f9d10017000 comp rx=0 tx=0).stop 2023-10-08T00:11:26.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.499+0000 7f9d26ec7700 1 -- 172.21.15.99:0/2841074688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d200cb8d0 msgr2=0x7f9d200c23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:26.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.499+0000 7f9d26ec7700 1 --2- 172.21.15.99:0/2841074688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d200cb8d0 0x7f9d200c23f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c00c7c0 tx=0x7f9d1c007e00 comp rx=0 tx=0).stop 2023-10-08T00:11:26.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.499+0000 7f9d26ec7700 1 -- 172.21.15.99:0/2841074688 shutdown_connections 2023-10-08T00:11:26.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.499+0000 7f9d26ec7700 1 --2- 172.21.15.99:0/2841074688 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9d04066f30 0x7f9d040693e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:26.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.499+0000 7f9d26ec7700 1 --2- 172.21.15.99:0/2841074688 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d20159e60 0x7f9d200afe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:26.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.499+0000 7f9d26ec7700 1 --2- 172.21.15.99:0/2841074688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d200cb8d0 0x7f9d200c23f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:26.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.499+0000 7f9d26ec7700 1 --2- 172.21.15.99:0/2841074688 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9d20159a90 0x7f9d200c2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:26.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.499+0000 7f9d26ec7700 1 -- 172.21.15.99:0/2841074688 >> 172.21.15.99:0/2841074688 conn(0x7f9d20069870 msgr2=0x7f9d20158a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:26.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.499+0000 7f9d26ec7700 1 -- 172.21.15.99:0/2841074688 shutdown_connections 2023-10-08T00:11:26.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.499+0000 7f9d26ec7700 1 -- 172.21.15.99:0/2841074688 wait complete. 2023-10-08T00:11:26.502 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 8 2023-10-08T00:11:26.512 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:11:26.624 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.623+0000 7ff15bfff700 1 Processor -- start 2023-10-08T00:11:26.624 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.623+0000 7ff15bfff700 1 -- start start 2023-10-08T00:11:26.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.624+0000 7ff15bfff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff15c0cdab0 0x7ff15c15d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:26.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.624+0000 7ff15bfff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff15c15dda0 0x7ff15c15cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:26.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.624+0000 7ff15bfff700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff15c0cb7c0 con 0x7ff15c0cd6e0 2023-10-08T00:11:26.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.624+0000 7ff15bfff700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff15c15e850 con 0x7ff15c15dda0 2023-10-08T00:11:26.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.624+0000 7ff15bfff700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff15c0cc360 con 0x7ff15c0cdab0 2023-10-08T00:11:26.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.624+0000 7ff15b7fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff15c15dda0 0x7ff15c15cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:26.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.624+0000 7ff15b7fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff15c15dda0 0x7ff15c15cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52156/0 (socket says 172.21.15.99:52156) 2023-10-08T00:11:26.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.624+0000 7ff15b7fe700 1 -- 172.21.15.99:0/1122590996 learned_addr learned my addr 172.21.15.99:0/1122590996 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:26.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.624+0000 7ff1527fc700 1 --2- 172.21.15.99:0/1122590996 >> v2:172.21.15.191:3300/0 conn(0x7ff15c0cdab0 0x7ff15c15d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:26.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.624+0000 7ff15b7fe700 1 -- 172.21.15.99:0/1122590996 >> v2:172.21.15.191:3300/0 conn(0x7ff15c0cdab0 msgr2=0x7ff15c15d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:26.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.624+0000 7ff15b7fe700 1 --2- 172.21.15.99:0/1122590996 >> v2:172.21.15.191:3300/0 conn(0x7ff15c0cdab0 0x7ff15c15d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:26.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.624+0000 7ff15b7fe700 1 -- 172.21.15.99:0/1122590996 >> v1:172.21.15.191:6789/0 conn(0x7ff15c0cd6e0 legacy=0x7ff15c0cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:26.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.624+0000 7ff15b7fe700 1 -- 172.21.15.99:0/1122590996 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff15c15e290 con 0x7ff15c15dda0 2023-10-08T00:11:26.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.624+0000 7ff1527fc700 1 --2- 172.21.15.99:0/1122590996 >> v2:172.21.15.191:3300/0 conn(0x7ff15c0cdab0 0x7ff15c15d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:11:26.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.624+0000 7ff15b7fe700 1 --2- 172.21.15.99:0/1122590996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff15c15dda0 0x7ff15c15cbd0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff15c0b6c70 tx=0x7ff154009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5adde78bd1a2f646 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:26.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.624+0000 7ff15a7fc700 1 -- 172.21.15.99:0/1122590996 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15400e070 con 0x7ff15c15dda0 2023-10-08T00:11:26.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.625+0000 7ff15a7fc700 1 -- 172.21.15.99:0/1122590996 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff15400ba50 con 0x7ff15c15dda0 2023-10-08T00:11:26.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.625+0000 7ff15a7fc700 1 -- 172.21.15.99:0/1122590996 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1540092e0 con 0x7ff15c15dda0 2023-10-08T00:11:26.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.625+0000 7ff15bfff700 1 -- 172.21.15.99:0/1122590996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff15c15dda0 msgr2=0x7ff15c15cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:26.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.625+0000 7ff15bfff700 1 --2- 172.21.15.99:0/1122590996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff15c15dda0 0x7ff15c15cbd0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff15c0b6c70 tx=0x7ff154009840 comp rx=0 tx=0).stop 2023-10-08T00:11:26.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.625+0000 7ff15bfff700 1 -- 172.21.15.99:0/1122590996 shutdown_connections 2023-10-08T00:11:26.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.625+0000 7ff15bfff700 1 --2- 172.21.15.99:0/1122590996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff15c15dda0 0x7ff15c15cbd0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:26.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.625+0000 7ff15bfff700 1 --2- 172.21.15.99:0/1122590996 >> v2:172.21.15.191:3300/0 conn(0x7ff15c0cdab0 0x7ff15c15d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:26.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.625+0000 7ff15bfff700 1 -- 172.21.15.99:0/1122590996 >> 172.21.15.99:0/1122590996 conn(0x7ff15c067660 msgr2=0x7ff15c067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:26.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.625+0000 7ff15bfff700 1 -- 172.21.15.99:0/1122590996 shutdown_connections 2023-10-08T00:11:26.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.625+0000 7ff15bfff700 1 -- 172.21.15.99:0/1122590996 wait complete. 2023-10-08T00:11:26.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.626+0000 7ff15bfff700 1 Processor -- start 2023-10-08T00:11:26.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.626+0000 7ff15bfff700 1 -- start start 2023-10-08T00:11:26.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.627+0000 7ff15bfff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff15c0cd6e0 0x7ff15c0ca600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:26.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.627+0000 7ff15bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff15c0cdab0 0x7ff15c15b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:26.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.627+0000 7ff15bfff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff15c15dda0 0x7ff15c159580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:26.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.627+0000 7ff15bfff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15c16aad0 con 0x7ff15c0cdab0 2023-10-08T00:11:26.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.627+0000 7ff15bfff700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15c16adb0 con 0x7ff15c15dda0 2023-10-08T00:11:26.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.627+0000 7ff15bfff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff15c16b090 con 0x7ff15c0cd6e0 2023-10-08T00:11:26.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.627+0000 7ff1527fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff15c0cdab0 0x7ff15c15b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:26.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.627+0000 7ff1527fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff15c0cdab0 0x7ff15c15b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52170/0 (socket says 172.21.15.99:52170) 2023-10-08T00:11:26.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.627+0000 7ff1527fc700 1 -- 172.21.15.99:0/3060970278 learned_addr learned my addr 172.21.15.99:0/3060970278 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:26.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.627+0000 7ff15affd700 1 --2- 172.21.15.99:0/3060970278 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff15c0cd6e0 0x7ff15c0ca600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:26.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.627+0000 7ff15b7fe700 1 --2- 172.21.15.99:0/3060970278 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff15c15dda0 0x7ff15c159580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:26.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.627+0000 7ff1527fc700 1 -- 172.21.15.99:0/3060970278 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff15c0cd6e0 msgr2=0x7ff15c0ca600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:26.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.627+0000 7ff1527fc700 1 --2- 172.21.15.99:0/3060970278 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff15c0cd6e0 0x7ff15c0ca600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:26.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.627+0000 7ff1527fc700 1 -- 172.21.15.99:0/3060970278 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff15c15dda0 msgr2=0x7ff15c159580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:26.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.627+0000 7ff1527fc700 1 --2- 172.21.15.99:0/3060970278 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff15c15dda0 0x7ff15c159580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:26.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.627+0000 7ff1527fc700 1 -- 172.21.15.99:0/3060970278 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff154009ae0 con 0x7ff15c0cdab0 2023-10-08T00:11:26.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.628+0000 7ff1527fc700 1 --2- 172.21.15.99:0/3060970278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff15c0cdab0 0x7ff15c15b260 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff14400e8d0 tx=0x7ff144008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:26.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.628+0000 7ff1537fe700 1 -- 172.21.15.99:0/3060970278 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff144013410 con 0x7ff15c0cdab0 2023-10-08T00:11:26.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.628+0000 7ff1537fe700 1 -- 172.21.15.99:0/3060970278 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff144013570 con 0x7ff15c0cdab0 2023-10-08T00:11:26.635 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.628+0000 7ff15bfff700 1 -- 172.21.15.99:0/3060970278 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff15c16b370 con 0x7ff15c0cdab0 2023-10-08T00:11:26.635 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.628+0000 7ff1537fe700 1 -- 172.21.15.99:0/3060970278 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff144013910 con 0x7ff15c0cdab0 2023-10-08T00:11:26.635 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.628+0000 7ff15bfff700 1 -- 172.21.15.99:0/3060970278 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff15c15b7a0 con 0x7ff15c0cdab0 2023-10-08T00:11:26.635 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.629+0000 7ff15bfff700 1 -- 172.21.15.99:0/3060970278 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff15c0c7db0 con 0x7ff15c0cdab0 2023-10-08T00:11:26.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.629+0000 7ff1537fe700 1 -- 172.21.15.99:0/3060970278 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff144013d40 con 0x7ff15c0cdab0 2023-10-08T00:11:26.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.629+0000 7ff1537fe700 1 --2- 172.21.15.99:0/3060970278 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff13c066f30 0x7ff13c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:26.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.630+0000 7ff1537fe700 1 -- 172.21.15.99:0/3060970278 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff144084230 con 0x7ff15c0cdab0 2023-10-08T00:11:26.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.630+0000 7ff15affd700 1 --2- 172.21.15.99:0/3060970278 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff13c066f30 0x7ff13c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:26.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.630+0000 7ff15affd700 1 --2- 172.21.15.99:0/3060970278 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff13c066f30 0x7ff13c0693e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff15c0b0c80 tx=0x7ff14c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:26.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.634+0000 7ff1537fe700 1 -- 172.21.15.99:0/3060970278 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff15c0c7db0 con 0x7ff15c0cdab0 2023-10-08T00:11:26.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.817+0000 7ff15bfff700 1 -- 172.21.15.99:0/3060970278 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff15c0c7db0 con 0x7ff15c0cdab0 2023-10-08T00:11:26.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.817+0000 7ff1537fe700 1 -- 172.21.15.99:0/3060970278 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15350 (secure 0 0 0) 0x7ff15c0c7db0 con 0x7ff15c0cdab0 2023-10-08T00:11:26.818 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:11:26.818 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":20,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:10:55.080854+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:10:51.822157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T00:10:52.415130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:11:26.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.819+0000 7ff15bfff700 1 -- 172.21.15.99:0/3060970278 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff13c066f30 msgr2=0x7ff13c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:26.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.819+0000 7ff15bfff700 1 --2- 172.21.15.99:0/3060970278 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff13c066f30 0x7ff13c0693e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff15c0b0c80 tx=0x7ff14c009040 comp rx=0 tx=0).stop 2023-10-08T00:11:26.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.819+0000 7ff15bfff700 1 -- 172.21.15.99:0/3060970278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff15c0cdab0 msgr2=0x7ff15c15b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:26.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.819+0000 7ff15bfff700 1 --2- 172.21.15.99:0/3060970278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff15c0cdab0 0x7ff15c15b260 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff14400e8d0 tx=0x7ff144008040 comp rx=0 tx=0).stop 2023-10-08T00:11:26.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.819+0000 7ff15bfff700 1 -- 172.21.15.99:0/3060970278 shutdown_connections 2023-10-08T00:11:26.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.819+0000 7ff15bfff700 1 --2- 172.21.15.99:0/3060970278 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff13c066f30 0x7ff13c0693e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:26.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.819+0000 7ff15bfff700 1 --2- 172.21.15.99:0/3060970278 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff15c15dda0 0x7ff15c159580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:26.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.819+0000 7ff15bfff700 1 --2- 172.21.15.99:0/3060970278 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff15c0cd6e0 0x7ff15c0ca600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:26.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.819+0000 7ff15bfff700 1 --2- 172.21.15.99:0/3060970278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff15c0cdab0 0x7ff15c15b260 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:26.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.819+0000 7ff15bfff700 1 -- 172.21.15.99:0/3060970278 >> 172.21.15.99:0/3060970278 conn(0x7ff15c067660 msgr2=0x7ff15c15d230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:26.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.819+0000 7ff15bfff700 1 -- 172.21.15.99:0/3060970278 shutdown_connections 2023-10-08T00:11:26.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.819+0000 7ff15bfff700 1 -- 172.21.15.99:0/3060970278 wait complete. 2023-10-08T00:11:26.833 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:10:42.063344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T00:10:51.822157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T00:10:52.415130+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T00:11:26.833 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T00:11:26.950 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.949+0000 7f6240e4f700 1 Processor -- start 2023-10-08T00:11:26.950 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.949+0000 7f6240e4f700 1 -- start start 2023-10-08T00:11:26.952 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.950+0000 7f6240e4f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f623c0cce30 0x7f623c15aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:26.952 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.950+0000 7f6240e4f700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f623c06aa00 0x7f623c0b5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:26.952 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.950+0000 7f6240e4f700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f623c0b4d70 con 0x7f623c0b6580 2023-10-08T00:11:26.952 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.950+0000 7f6240e4f700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f623c0cd8d0 con 0x7f623c06aa00 2023-10-08T00:11:26.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.950+0000 7f6240e4f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f623c0b4440 con 0x7f623c0cce30 2023-10-08T00:11:26.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.950+0000 7f623bfff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f623c06aa00 0x7f623c0b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:26.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.950+0000 7f623bfff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f623c06aa00 0x7f623c0b5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52182/0 (socket says 172.21.15.99:52182) 2023-10-08T00:11:26.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.950+0000 7f623bfff700 1 -- 172.21.15.99:0/710014779 learned_addr learned my addr 172.21.15.99:0/710014779 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:26.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.951+0000 7f623b7fe700 1 --2- 172.21.15.99:0/710014779 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f623c0cce30 0x7f623c15aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:26.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.951+0000 7f623bfff700 1 -- 172.21.15.99:0/710014779 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f623c0cce30 msgr2=0x7f623c15aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:26.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.951+0000 7f623bfff700 1 --2- 172.21.15.99:0/710014779 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f623c0cce30 0x7f623c15aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:26.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.951+0000 7f623bfff700 1 -- 172.21.15.99:0/710014779 >> v1:172.21.15.191:6789/0 conn(0x7f623c0b6580 legacy=0x7f623c06a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:26.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.951+0000 7f623bfff700 1 -- 172.21.15.99:0/710014779 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f623c0b4720 con 0x7f623c06aa00 2023-10-08T00:11:26.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.951+0000 7f623bfff700 1 --2- 172.21.15.99:0/710014779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f623c06aa00 0x7f623c0b5200 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6234009f20 tx=0x7f6234009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d556b68a8a7d0edb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:26.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.951+0000 7f623a7fc700 1 -- 172.21.15.99:0/710014779 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623400e070 con 0x7f623c06aa00 2023-10-08T00:11:26.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.951+0000 7f623a7fc700 1 -- 172.21.15.99:0/710014779 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f623400ba80 con 0x7f623c06aa00 2023-10-08T00:11:26.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.951+0000 7f623a7fc700 1 -- 172.21.15.99:0/710014779 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62340092e0 con 0x7f623c06aa00 2023-10-08T00:11:26.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.951+0000 7f6240e4f700 1 -- 172.21.15.99:0/710014779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f623c06aa00 msgr2=0x7f623c0b5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:26.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.951+0000 7f6240e4f700 1 --2- 172.21.15.99:0/710014779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f623c06aa00 0x7f623c0b5200 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6234009f20 tx=0x7f6234009840 comp rx=0 tx=0).stop 2023-10-08T00:11:26.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.952+0000 7f6240e4f700 1 -- 172.21.15.99:0/710014779 shutdown_connections 2023-10-08T00:11:26.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.952+0000 7f6240e4f700 1 --2- 172.21.15.99:0/710014779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f623c06aa00 0x7f623c0b5200 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:26.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.952+0000 7f6240e4f700 1 --2- 172.21.15.99:0/710014779 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f623c0cce30 0x7f623c15aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:26.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.952+0000 7f6240e4f700 1 -- 172.21.15.99:0/710014779 >> 172.21.15.99:0/710014779 conn(0x7f623c068cb0 msgr2=0x7f623c0690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:26.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.952+0000 7f6240e4f700 1 -- 172.21.15.99:0/710014779 shutdown_connections 2023-10-08T00:11:26.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.952+0000 7f6240e4f700 1 -- 172.21.15.99:0/710014779 wait complete. 2023-10-08T00:11:26.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.953+0000 7f6240e4f700 1 Processor -- start 2023-10-08T00:11:26.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.953+0000 7f6240e4f700 1 -- start start 2023-10-08T00:11:26.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.953+0000 7f6240e4f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f623c06aa00 0x7f623c14e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:26.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.953+0000 7f6240e4f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f623c0b6580 0x7f623c0b2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:26.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.953+0000 7f6240e4f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f623c0cce30 0x7f623c16d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:26.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.953+0000 7f6240e4f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623c16cfb0 con 0x7f623c0cce30 2023-10-08T00:11:26.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.953+0000 7f6240e4f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623c14e7b0 con 0x7f623c06aa00 2023-10-08T00:11:26.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.953+0000 7f6240e4f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f623c1530a0 con 0x7f623c0b6580 2023-10-08T00:11:26.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.954+0000 7f623bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f623c0cce30 0x7f623c16d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:26.960 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.954+0000 7f623bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f623c0cce30 0x7f623c16d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52184/0 (socket says 172.21.15.99:52184) 2023-10-08T00:11:26.960 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.954+0000 7f623bfff700 1 -- 172.21.15.99:0/1711275229 learned_addr learned my addr 172.21.15.99:0/1711275229 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:26.960 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.954+0000 7f623b7fe700 1 --2- 172.21.15.99:0/1711275229 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f623c06aa00 0x7f623c14e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:26.960 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.954+0000 7f623affd700 1 --2- 172.21.15.99:0/1711275229 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f623c0b6580 0x7f623c0b2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:26.960 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.954+0000 7f623bfff700 1 -- 172.21.15.99:0/1711275229 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f623c0b6580 msgr2=0x7f623c0b2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:26.960 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.954+0000 7f623bfff700 1 --2- 172.21.15.99:0/1711275229 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f623c0b6580 0x7f623c0b2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:26.960 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.954+0000 7f623bfff700 1 -- 172.21.15.99:0/1711275229 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f623c06aa00 msgr2=0x7f623c14e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:26.960 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.954+0000 7f623bfff700 1 --2- 172.21.15.99:0/1711275229 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f623c06aa00 0x7f623c14e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:26.960 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.954+0000 7f623bfff700 1 -- 172.21.15.99:0/1711275229 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6234009ae0 con 0x7f623c0cce30 2023-10-08T00:11:26.960 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.954+0000 7f623bfff700 1 --2- 172.21.15.99:0/1711275229 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f623c0cce30 0x7f623c16d190 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6234012000 tx=0x7f6234018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:26.960 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.954+0000 7f62237fe700 1 -- 172.21.15.99:0/1711275229 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62340032f0 con 0x7f623c0cce30 2023-10-08T00:11:26.960 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.954+0000 7f62237fe700 1 -- 172.21.15.99:0/1711275229 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6234008d20 con 0x7f623c0cce30 2023-10-08T00:11:26.961 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.954+0000 7f6240e4f700 1 -- 172.21.15.99:0/1711275229 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f623c14ebe0 con 0x7f623c0cce30 2023-10-08T00:11:26.961 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.954+0000 7f62237fe700 1 -- 172.21.15.99:0/1711275229 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6234026690 con 0x7f623c0cce30 2023-10-08T00:11:26.961 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.954+0000 7f6240e4f700 1 -- 172.21.15.99:0/1711275229 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f623c152770 con 0x7f623c0cce30 2023-10-08T00:11:26.961 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.955+0000 7f6240e4f700 1 -- 172.21.15.99:0/1711275229 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f623c089550 con 0x7f623c0cce30 2023-10-08T00:11:26.961 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.956+0000 7f62237fe700 1 -- 172.21.15.99:0/1711275229 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6234003040 con 0x7f623c0cce30 2023-10-08T00:11:26.961 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.956+0000 7f62237fe700 1 --2- 172.21.15.99:0/1711275229 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6228066f30 0x7f62280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:26.961 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.956+0000 7f62237fe700 1 -- 172.21.15.99:0/1711275229 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f62340863f0 con 0x7f623c0cce30 2023-10-08T00:11:26.961 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.956+0000 7f623b7fe700 1 --2- 172.21.15.99:0/1711275229 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6228066f30 0x7f62280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:26.961 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.957+0000 7f623b7fe700 1 --2- 172.21.15.99:0/1711275229 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6228066f30 0x7f62280693e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f623c150590 tx=0x7f622c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:26.961 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:26.960+0000 7f62237fe700 1 -- 172.21.15.99:0/1711275229 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6234053660 con 0x7f623c0cce30 2023-10-08T00:11:27.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.141+0000 7f6240e4f700 1 -- 172.21.15.99:0/1711275229 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f623c0b4a00 con 0x7f623c0cce30 2023-10-08T00:11:27.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.141+0000 7f62237fe700 1 -- 172.21.15.99:0/1711275229 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6234056c80 con 0x7f623c0cce30 2023-10-08T00:11:27.142 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2023-10-08T00:11:27.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.143+0000 7f6240e4f700 1 -- 172.21.15.99:0/1711275229 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6228066f30 msgr2=0x7f62280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:27.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.143+0000 7f6240e4f700 1 --2- 172.21.15.99:0/1711275229 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6228066f30 0x7f62280693e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f623c150590 tx=0x7f622c008040 comp rx=0 tx=0).stop 2023-10-08T00:11:27.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.143+0000 7f6240e4f700 1 -- 172.21.15.99:0/1711275229 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f623c0cce30 msgr2=0x7f623c16d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:27.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.143+0000 7f6240e4f700 1 --2- 172.21.15.99:0/1711275229 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f623c0cce30 0x7f623c16d190 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6234012000 tx=0x7f6234018000 comp rx=0 tx=0).stop 2023-10-08T00:11:27.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.143+0000 7f6240e4f700 1 -- 172.21.15.99:0/1711275229 shutdown_connections 2023-10-08T00:11:27.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.143+0000 7f6240e4f700 1 --2- 172.21.15.99:0/1711275229 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6228066f30 0x7f62280693e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:27.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.143+0000 7f6240e4f700 1 --2- 172.21.15.99:0/1711275229 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f623c06aa00 0x7f623c14e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:27.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.143+0000 7f6240e4f700 1 --2- 172.21.15.99:0/1711275229 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f623c0cce30 0x7f623c16d190 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:27.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.143+0000 7f6240e4f700 1 --2- 172.21.15.99:0/1711275229 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f623c0b6580 0x7f623c0b2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:27.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.143+0000 7f6240e4f700 1 -- 172.21.15.99:0/1711275229 >> 172.21.15.99:0/1711275229 conn(0x7f623c068cb0 msgr2=0x7f623c150710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:27.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.143+0000 7f6240e4f700 1 -- 172.21.15.99:0/1711275229 shutdown_connections 2023-10-08T00:11:27.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.143+0000 7f6240e4f700 1 -- 172.21.15.99:0/1711275229 wait complete. 2023-10-08T00:11:27.156 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T00:11:27.275 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.274+0000 7f84bb430700 1 Processor -- start 2023-10-08T00:11:27.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.274+0000 7f84bb430700 1 -- start start 2023-10-08T00:11:27.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.275+0000 7f84bb430700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f84b4157cc0 0x7f84b40b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:27.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.275+0000 7f84bb430700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f84b40ca9e0 0x7f84b40cae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:27.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.275+0000 7f84bb430700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84b40afb50 con 0x7f84b4069870 2023-10-08T00:11:27.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.275+0000 7f84bb430700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84b40cb3c0 con 0x7f84b40ca9e0 2023-10-08T00:11:27.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.275+0000 7f84bb430700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84b40afe30 con 0x7f84b4157cc0 2023-10-08T00:11:27.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.275+0000 7f84b89cb700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f84b40ca9e0 0x7f84b40cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:27.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.275+0000 7f84b99cd700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f84b4069870 0x7f84b4069c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:44038/0 (socket says 172.21.15.99:44038) 2023-10-08T00:11:27.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.275+0000 7f84b89cb700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f84b40ca9e0 0x7f84b40cae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52198/0 (socket says 172.21.15.99:52198) 2023-10-08T00:11:27.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.275+0000 7f84b99cd700 1 -- 172.21.15.99:0/4032361072 learned_addr learned my addr 172.21.15.99:0/4032361072 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:27.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.275+0000 7f84b91cc700 1 --2- 172.21.15.99:0/4032361072 >> v2:172.21.15.191:3300/0 conn(0x7f84b4157cc0 0x7f84b40b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:27.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.275+0000 7f84b89cb700 1 -- 172.21.15.99:0/4032361072 >> v2:172.21.15.191:3300/0 conn(0x7f84b4157cc0 msgr2=0x7f84b40b0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:27.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.275+0000 7f84b89cb700 1 --2- 172.21.15.99:0/4032361072 >> v2:172.21.15.191:3300/0 conn(0x7f84b4157cc0 0x7f84b40b0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:27.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.275+0000 7f84b89cb700 1 -- 172.21.15.99:0/4032361072 >> v1:172.21.15.99:6789/0 conn(0x7f84b4069870 legacy=0x7f84b4069c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:27.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.275+0000 7f84b89cb700 1 -- 172.21.15.99:0/4032361072 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b40b0110 con 0x7f84b40ca9e0 2023-10-08T00:11:27.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.275+0000 7f84abfff700 1 -- 172.21.15.99:0/4032361072 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3924638588 0 0) 0x7f84b40afb50 con 0x7f84b4069870 2023-10-08T00:11:27.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.276+0000 7f84b89cb700 1 --2- 172.21.15.99:0/4032361072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84b40ca9e0 0x7f84b40cae80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f84b40cfbc0 tx=0x7f84a40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e2e10136b8b52ce server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:27.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.276+0000 7f84abfff700 1 -- 172.21.15.99:0/4032361072 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a400b6b0 con 0x7f84b40ca9e0 2023-10-08T00:11:27.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.276+0000 7f84abfff700 1 -- 172.21.15.99:0/4032361072 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84a400b810 con 0x7f84b40ca9e0 2023-10-08T00:11:27.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.276+0000 7f84abfff700 1 -- 172.21.15.99:0/4032361072 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a400bbb0 con 0x7f84b40ca9e0 2023-10-08T00:11:27.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.276+0000 7f84bb430700 1 -- 172.21.15.99:0/4032361072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84b40ca9e0 msgr2=0x7f84b40cae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:27.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.276+0000 7f84bb430700 1 --2- 172.21.15.99:0/4032361072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84b40ca9e0 0x7f84b40cae80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f84b40cfbc0 tx=0x7f84a40091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:27.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.276+0000 7f84bb430700 1 -- 172.21.15.99:0/4032361072 shutdown_connections 2023-10-08T00:11:27.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.276+0000 7f84bb430700 1 --2- 172.21.15.99:0/4032361072 >> v2:172.21.15.191:3300/0 conn(0x7f84b4157cc0 0x7f84b40b0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:27.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.276+0000 7f84bb430700 1 --2- 172.21.15.99:0/4032361072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84b40ca9e0 0x7f84b40cae80 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:27.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.276+0000 7f84bb430700 1 -- 172.21.15.99:0/4032361072 >> 172.21.15.99:0/4032361072 conn(0x7f84b40cb5c0 msgr2=0x7f84b40cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:27.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.276+0000 7f84bb430700 1 -- 172.21.15.99:0/4032361072 shutdown_connections 2023-10-08T00:11:27.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.276+0000 7f84bb430700 1 -- 172.21.15.99:0/4032361072 wait complete. 2023-10-08T00:11:27.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.277+0000 7f84bb430700 1 Processor -- start 2023-10-08T00:11:27.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.277+0000 7f84bb430700 1 -- start start 2023-10-08T00:11:27.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.278+0000 7f84bb430700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84b4069870 0x7f84b40c45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:27.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.278+0000 7f84bb430700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84b40ca9e0 0x7f84b4155f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:27.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.278+0000 7f84bb430700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84b4157cc0 0x7f84b4159940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:27.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.278+0000 7f84bb430700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b40c4b30 con 0x7f84b4069870 2023-10-08T00:11:27.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.278+0000 7f84bb430700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b40b0cc0 con 0x7f84b40ca9e0 2023-10-08T00:11:27.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.278+0000 7f84bb430700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84b40c4d50 con 0x7f84b4157cc0 2023-10-08T00:11:27.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.278+0000 7f84b91cc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84b4069870 0x7f84b40c45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:27.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.278+0000 7f84b91cc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84b4069870 0x7f84b40c45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52202/0 (socket says 172.21.15.99:52202) 2023-10-08T00:11:27.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.278+0000 7f84b91cc700 1 -- 172.21.15.99:0/1510993189 learned_addr learned my addr 172.21.15.99:0/1510993189 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:27.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.278+0000 7f84b89cb700 1 --2- 172.21.15.99:0/1510993189 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84b40ca9e0 0x7f84b4155f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:27.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.279+0000 7f84b99cd700 1 --2- 172.21.15.99:0/1510993189 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84b4157cc0 0x7f84b4159940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:27.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.279+0000 7f84b91cc700 1 -- 172.21.15.99:0/1510993189 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84b4157cc0 msgr2=0x7f84b4159940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:27.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.279+0000 7f84b91cc700 1 --2- 172.21.15.99:0/1510993189 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84b4157cc0 0x7f84b4159940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:27.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.279+0000 7f84b91cc700 1 -- 172.21.15.99:0/1510993189 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84b40ca9e0 msgr2=0x7f84b4155f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:27.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.279+0000 7f84b91cc700 1 --2- 172.21.15.99:0/1510993189 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84b40ca9e0 0x7f84b4155f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:27.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.279+0000 7f84b91cc700 1 -- 172.21.15.99:0/1510993189 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a4009420 con 0x7f84b4069870 2023-10-08T00:11:27.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.279+0000 7f84b89cb700 1 --2- 172.21.15.99:0/1510993189 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84b40ca9e0 0x7f84b4155f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:11:27.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.279+0000 7f84b91cc700 1 --2- 172.21.15.99:0/1510993189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84b4069870 0x7f84b40c45f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f84b000d8c0 tx=0x7f84b0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:27.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.279+0000 7f84a97fa700 1 -- 172.21.15.99:0/1510993189 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b000de60 con 0x7f84b4069870 2023-10-08T00:11:27.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.279+0000 7f84a97fa700 1 -- 172.21.15.99:0/1510993189 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84b000b780 con 0x7f84b4069870 2023-10-08T00:11:27.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.279+0000 7f84bb430700 1 -- 172.21.15.99:0/1510993189 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84b40c5030 con 0x7f84b4069870 2023-10-08T00:11:27.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.279+0000 7f84a97fa700 1 -- 172.21.15.99:0/1510993189 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b000bb20 con 0x7f84b4069870 2023-10-08T00:11:27.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.279+0000 7f84bb430700 1 -- 172.21.15.99:0/1510993189 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84b40c55f0 con 0x7f84b4069870 2023-10-08T00:11:27.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.280+0000 7f84bb430700 1 -- 172.21.15.99:0/1510993189 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84b404c3c0 con 0x7f84b4069870 2023-10-08T00:11:27.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.280+0000 7f84a97fa700 1 -- 172.21.15.99:0/1510993189 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f84b001e460 con 0x7f84b4069870 2023-10-08T00:11:27.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.281+0000 7f84a97fa700 1 --2- 172.21.15.99:0/1510993189 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8498066f30 0x7f84980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:27.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.281+0000 7f84a97fa700 1 -- 172.21.15.99:0/1510993189 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f84b0084c60 con 0x7f84b4069870 2023-10-08T00:11:27.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.281+0000 7f84b89cb700 1 --2- 172.21.15.99:0/1510993189 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8498066f30 0x7f84980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:27.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.282+0000 7f84b89cb700 1 --2- 172.21.15.99:0/1510993189 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8498066f30 0x7f84980693e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f84a40093f0 tx=0x7f84a4017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:27.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.285+0000 7f84a97fa700 1 -- 172.21.15.99:0/1510993189 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f84b0051e30 con 0x7f84b4069870 2023-10-08T00:11:27.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.466+0000 7f84bb430700 1 -- 172.21.15.99:0/1510993189 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f84b4168060 con 0x7f84b4069870 2023-10-08T00:11:27.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.466+0000 7f84a97fa700 1 -- 172.21.15.99:0/1510993189 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f84b0055450 con 0x7f84b4069870 2023-10-08T00:11:27.467 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2023-10-08T00:11:27.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.468+0000 7f84bb430700 1 -- 172.21.15.99:0/1510993189 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8498066f30 msgr2=0x7f84980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:27.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.468+0000 7f84bb430700 1 --2- 172.21.15.99:0/1510993189 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8498066f30 0x7f84980693e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f84a40093f0 tx=0x7f84a4017000 comp rx=0 tx=0).stop 2023-10-08T00:11:27.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.468+0000 7f84bb430700 1 -- 172.21.15.99:0/1510993189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84b4069870 msgr2=0x7f84b40c45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:27.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.468+0000 7f84bb430700 1 --2- 172.21.15.99:0/1510993189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84b4069870 0x7f84b40c45f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f84b000d8c0 tx=0x7f84b0008040 comp rx=0 tx=0).stop 2023-10-08T00:11:27.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.468+0000 7f84bb430700 1 -- 172.21.15.99:0/1510993189 shutdown_connections 2023-10-08T00:11:27.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.468+0000 7f84bb430700 1 --2- 172.21.15.99:0/1510993189 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8498066f30 0x7f84980693e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:27.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.468+0000 7f84bb430700 1 --2- 172.21.15.99:0/1510993189 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84b4157cc0 0x7f84b4159940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:27.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.468+0000 7f84bb430700 1 --2- 172.21.15.99:0/1510993189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84b4069870 0x7f84b40c45f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:27.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.468+0000 7f84bb430700 1 --2- 172.21.15.99:0/1510993189 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84b40ca9e0 0x7f84b4155f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:27.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.468+0000 7f84bb430700 1 -- 172.21.15.99:0/1510993189 >> 172.21.15.99:0/1510993189 conn(0x7f84b40cb5c0 msgr2=0x7f84b4156d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:27.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.468+0000 7f84bb430700 1 -- 172.21.15.99:0/1510993189 shutdown_connections 2023-10-08T00:11:27.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.468+0000 7f84bb430700 1 -- 172.21.15.99:0/1510993189 wait complete. 2023-10-08T00:11:27.480 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T00:11:27.597 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.596+0000 7f5bb146c700 1 Processor -- start 2023-10-08T00:11:27.597 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.596+0000 7f5bb146c700 1 -- start start 2023-10-08T00:11:27.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.597+0000 7f5bb146c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5bac159ec0 0x7f5bac0af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:27.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.597+0000 7f5bb146c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5bac15a290 0x7f5bac0afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:27.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.597+0000 7f5bb146c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f5bac06a5a0 0x7f5bac06aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:27.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.597+0000 7f5bb146c700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bac06af60 con 0x7f5bac159ec0 2023-10-08T00:11:27.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.597+0000 7f5bb146c700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bac0b0790 con 0x7f5bac06a5a0 2023-10-08T00:11:27.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.597+0000 7f5bb146c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bac0ce1f0 con 0x7f5bac15a290 2023-10-08T00:11:27.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.597+0000 7f5baaffd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5bac159ec0 0x7f5bac0af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:27.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.597+0000 7f5baaffd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5bac159ec0 0x7f5bac0af940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52204/0 (socket says 172.21.15.99:52204) 2023-10-08T00:11:27.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.597+0000 7f5baaffd700 1 -- 172.21.15.99:0/107581260 learned_addr learned my addr 172.21.15.99:0/107581260 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:27.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.597+0000 7f5bab7fe700 1 --2- 172.21.15.99:0/107581260 >> v2:172.21.15.191:3300/0 conn(0x7f5bac06a5a0 0x7f5bac06aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:27.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.597+0000 7f5baaffd700 1 -- 172.21.15.99:0/107581260 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5bac15a290 msgr2=0x7f5bac0afe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:27.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.597+0000 7f5baaffd700 1 --2- 172.21.15.99:0/107581260 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5bac15a290 0x7f5bac0afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:27.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.597+0000 7f5baaffd700 1 -- 172.21.15.99:0/107581260 >> v2:172.21.15.191:3300/0 conn(0x7f5bac06a5a0 msgr2=0x7f5bac06aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:27.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.597+0000 7f5baaffd700 1 --2- 172.21.15.99:0/107581260 >> v2:172.21.15.191:3300/0 conn(0x7f5bac06a5a0 0x7f5bac06aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:27.601 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.598+0000 7f5baaffd700 1 -- 172.21.15.99:0/107581260 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bac0b0b20 con 0x7f5bac159ec0 2023-10-08T00:11:27.601 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.598+0000 7f5baaffd700 1 --2- 172.21.15.99:0/107581260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5bac159ec0 0x7f5bac0af940 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5bac0b03c0 tx=0x7f5b9c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f29c259e087a43a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:27.601 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.598+0000 7f5baa7fc700 1 -- 172.21.15.99:0/107581260 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c00b860 con 0x7f5bac159ec0 2023-10-08T00:11:27.601 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.598+0000 7f5baa7fc700 1 -- 172.21.15.99:0/107581260 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b9c00b9c0 con 0x7f5bac159ec0 2023-10-08T00:11:27.601 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.598+0000 7f5baa7fc700 1 -- 172.21.15.99:0/107581260 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c00bd60 con 0x7f5bac159ec0 2023-10-08T00:11:27.601 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.598+0000 7f5bb146c700 1 -- 172.21.15.99:0/107581260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5bac159ec0 msgr2=0x7f5bac0af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:27.601 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.598+0000 7f5bb146c700 1 --2- 172.21.15.99:0/107581260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5bac159ec0 0x7f5bac0af940 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5bac0b03c0 tx=0x7f5b9c009250 comp rx=0 tx=0).stop 2023-10-08T00:11:27.601 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.598+0000 7f5bb146c700 1 -- 172.21.15.99:0/107581260 shutdown_connections 2023-10-08T00:11:27.601 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.598+0000 7f5bb146c700 1 --2- 172.21.15.99:0/107581260 >> v2:172.21.15.191:3300/0 conn(0x7f5bac06a5a0 0x7f5bac06aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:27.601 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.598+0000 7f5bb146c700 1 --2- 172.21.15.99:0/107581260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5bac159ec0 0x7f5bac0af940 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:27.601 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.598+0000 7f5bb146c700 1 --2- 172.21.15.99:0/107581260 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5bac15a290 0x7f5bac0afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:27.601 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.598+0000 7f5bb146c700 1 -- 172.21.15.99:0/107581260 >> 172.21.15.99:0/107581260 conn(0x7f5bac0d1880 msgr2=0x7f5bac0cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:27.602 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.598+0000 7f5bb146c700 1 -- 172.21.15.99:0/107581260 shutdown_connections 2023-10-08T00:11:27.602 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.598+0000 7f5bb146c700 1 -- 172.21.15.99:0/107581260 wait complete. 2023-10-08T00:11:27.604 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.600+0000 7f5bb146c700 1 Processor -- start 2023-10-08T00:11:27.604 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.600+0000 7f5bb146c700 1 -- start start 2023-10-08T00:11:27.604 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.600+0000 7f5bb146c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5bac06a5a0 0x7f5bac152260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:27.604 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.600+0000 7f5bb146c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5bac159ec0 0x7f5bac0c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:27.605 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.600+0000 7f5bb146c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5bac15a290 0x7f5bac0c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:27.605 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.600+0000 7f5bb146c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bac1527a0 con 0x7f5bac06a5a0 2023-10-08T00:11:27.605 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.600+0000 7f5bb146c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bac0caa00 con 0x7f5bac15a290 2023-10-08T00:11:27.605 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.600+0000 7f5bb146c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bac0ca140 con 0x7f5bac159ec0 2023-10-08T00:11:27.605 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.600+0000 7f5baaffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5bac06a5a0 0x7f5bac152260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:27.605 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.600+0000 7f5baaffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5bac06a5a0 0x7f5bac152260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52210/0 (socket says 172.21.15.99:52210) 2023-10-08T00:11:27.605 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.600+0000 7f5baaffd700 1 -- 172.21.15.99:0/3965158969 learned_addr learned my addr 172.21.15.99:0/3965158969 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:27.605 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.601+0000 7f5bab7fe700 1 --2- 172.21.15.99:0/3965158969 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5bac15a290 0x7f5bac0c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:27.605 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.601+0000 7f5ba27fc700 1 --2- 172.21.15.99:0/3965158969 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5bac159ec0 0x7f5bac0c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:27.605 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.601+0000 7f5baaffd700 1 -- 172.21.15.99:0/3965158969 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5bac159ec0 msgr2=0x7f5bac0c7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:27.605 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.601+0000 7f5baaffd700 1 --2- 172.21.15.99:0/3965158969 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5bac159ec0 0x7f5bac0c7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:27.605 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.601+0000 7f5baaffd700 1 -- 172.21.15.99:0/3965158969 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5bac15a290 msgr2=0x7f5bac0c83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:27.606 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.601+0000 7f5baaffd700 1 --2- 172.21.15.99:0/3965158969 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5bac15a290 0x7f5bac0c83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:27.606 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.601+0000 7f5baaffd700 1 -- 172.21.15.99:0/3965158969 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b9c0094f0 con 0x7f5bac06a5a0 2023-10-08T00:11:27.606 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.601+0000 7f5bab7fe700 1 --2- 172.21.15.99:0/3965158969 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5bac15a290 0x7f5bac0c83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:11:27.606 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.601+0000 7f5ba27fc700 1 --2- 172.21.15.99:0/3965158969 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5bac159ec0 0x7f5bac0c7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:11:27.606 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.601+0000 7f5baaffd700 1 --2- 172.21.15.99:0/3965158969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5bac06a5a0 0x7f5bac152260 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c0090b0 tx=0x7f5b9c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:27.606 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.601+0000 7f5ba37fe700 1 -- 172.21.15.99:0/3965158969 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c026480 con 0x7f5bac06a5a0 2023-10-08T00:11:27.606 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.601+0000 7f5ba37fe700 1 -- 172.21.15.99:0/3965158969 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b9c01f070 con 0x7f5bac06a5a0 2023-10-08T00:11:27.606 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.601+0000 7f5bb146c700 1 -- 172.21.15.99:0/3965158969 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bac152c40 con 0x7f5bac06a5a0 2023-10-08T00:11:27.606 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.601+0000 7f5ba37fe700 1 -- 172.21.15.99:0/3965158969 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c026830 con 0x7f5bac06a5a0 2023-10-08T00:11:27.606 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.601+0000 7f5bb146c700 1 -- 172.21.15.99:0/3965158969 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bac0ca690 con 0x7f5bac06a5a0 2023-10-08T00:11:27.606 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.602+0000 7f5bb146c700 1 -- 172.21.15.99:0/3965158969 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bac04c3c0 con 0x7f5bac06a5a0 2023-10-08T00:11:27.606 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.602+0000 7f5ba37fe700 1 -- 172.21.15.99:0/3965158969 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5b9c026ba0 con 0x7f5bac06a5a0 2023-10-08T00:11:27.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.603+0000 7f5ba37fe700 1 --2- 172.21.15.99:0/3965158969 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5b8c066f30 0x7f5b8c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:27.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.603+0000 7f5ba37fe700 1 -- 172.21.15.99:0/3965158969 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5b9c021070 con 0x7f5bac06a5a0 2023-10-08T00:11:27.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.603+0000 7f5ba27fc700 1 --2- 172.21.15.99:0/3965158969 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5b8c066f30 0x7f5b8c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:27.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.604+0000 7f5ba27fc700 1 --2- 172.21.15.99:0/3965158969 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5b8c066f30 0x7f5b8c0693e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5bac0caf50 tx=0x7f5b94008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:27.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.607+0000 7f5ba37fe700 1 -- 172.21.15.99:0/3965158969 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5b9c053630 con 0x7f5bac06a5a0 2023-10-08T00:11:27.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.787+0000 7f5bb146c700 1 -- 172.21.15.99:0/3965158969 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5bac0b04b0 con 0x7f5bac06a5a0 2023-10-08T00:11:27.788 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.787+0000 7f5ba37fe700 1 -- 172.21.15.99:0/3965158969 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5b9c056c50 con 0x7f5bac06a5a0 2023-10-08T00:11:27.788 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2023-10-08T00:11:27.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.789+0000 7f5bb146c700 1 -- 172.21.15.99:0/3965158969 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5b8c066f30 msgr2=0x7f5b8c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:27.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.789+0000 7f5bb146c700 1 --2- 172.21.15.99:0/3965158969 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5b8c066f30 0x7f5b8c0693e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5bac0caf50 tx=0x7f5b94008040 comp rx=0 tx=0).stop 2023-10-08T00:11:27.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.789+0000 7f5bb146c700 1 -- 172.21.15.99:0/3965158969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5bac06a5a0 msgr2=0x7f5bac152260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:27.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.789+0000 7f5bb146c700 1 --2- 172.21.15.99:0/3965158969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5bac06a5a0 0x7f5bac152260 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c0090b0 tx=0x7f5b9c017000 comp rx=0 tx=0).stop 2023-10-08T00:11:27.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.789+0000 7f5bb146c700 1 -- 172.21.15.99:0/3965158969 shutdown_connections 2023-10-08T00:11:27.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.789+0000 7f5bb146c700 1 --2- 172.21.15.99:0/3965158969 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5b8c066f30 0x7f5b8c0693e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:27.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.789+0000 7f5bb146c700 1 --2- 172.21.15.99:0/3965158969 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5bac15a290 0x7f5bac0c83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:27.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.789+0000 7f5bb146c700 1 --2- 172.21.15.99:0/3965158969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5bac06a5a0 0x7f5bac152260 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:27.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.789+0000 7f5bb146c700 1 --2- 172.21.15.99:0/3965158969 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5bac159ec0 0x7f5bac0c7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:27.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.789+0000 7f5bb146c700 1 -- 172.21.15.99:0/3965158969 >> 172.21.15.99:0/3965158969 conn(0x7f5bac0d1880 msgr2=0x7f5bac0d0110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:27.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.789+0000 7f5bb146c700 1 -- 172.21.15.99:0/3965158969 shutdown_connections 2023-10-08T00:11:27.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.789+0000 7f5bb146c700 1 -- 172.21.15.99:0/3965158969 wait complete. 2023-10-08T00:11:27.802 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:11:27.915 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.914+0000 7fbc898a0700 1 Processor -- start 2023-10-08T00:11:27.915 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.914+0000 7fbc898a0700 1 -- start start 2023-10-08T00:11:27.921 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.915+0000 7fbc898a0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbc84156e20 0x7fbc84067b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:27.921 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.915+0000 7fbc898a0700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc840c87d0 con 0x7fbc84156a50 2023-10-08T00:11:27.921 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.915+0000 7fbc898a0700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc84068750 con 0x7fbc840680b0 2023-10-08T00:11:27.921 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.915+0000 7fbc898a0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc84068d10 con 0x7fbc84156e20 2023-10-08T00:11:27.922 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.915+0000 7fbc82ffd700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fbc84156a50 0x7fbc840c8c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:44040/0 (socket says 172.21.15.99:44040) 2023-10-08T00:11:27.922 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.915+0000 7fbc82ffd700 1 -- 172.21.15.99:0/4216916666 learned_addr learned my addr 172.21.15.99:0/4216916666 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:27.922 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.915+0000 7fbc827fc700 1 --2- 172.21.15.99:0/4216916666 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbc84156e20 0x7fbc84067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:27.922 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.916+0000 7fbc81ffb700 1 -- 172.21.15.99:0/4216916666 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2682501318 0 0) 0x7fbc840c87d0 con 0x7fbc84156a50 2023-10-08T00:11:27.922 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.916+0000 7fbc81ffb700 1 -- 172.21.15.99:0/4216916666 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc6c003570 con 0x7fbc84156a50 2023-10-08T00:11:27.922 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.916+0000 7fbc81ffb700 1 -- 172.21.15.99:0/4216916666 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1234573783 0 0) 0x7fbc6c003570 con 0x7fbc84156a50 2023-10-08T00:11:27.923 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.916+0000 7fbc81ffb700 1 -- 172.21.15.99:0/4216916666 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbc84156e20 msgr2=0x7fbc84067b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:27.923 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.916+0000 7fbc81ffb700 1 --2- 172.21.15.99:0/4216916666 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbc84156e20 0x7fbc84067b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:27.923 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.916+0000 7fbc81ffb700 1 -- 172.21.15.99:0/4216916666 >> v1:172.21.15.191:6789/0 conn(0x7fbc840680b0 legacy=0x7fbc840adab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:27.923 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.916+0000 7fbc81ffb700 1 -- 172.21.15.99:0/4216916666 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc84068a30 con 0x7fbc84156a50 2023-10-08T00:11:27.923 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.916+0000 7fbc81ffb700 1 -- 172.21.15.99:0/4216916666 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7fbc740017f0 con 0x7fbc84156a50 2023-10-08T00:11:27.923 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.916+0000 7fbc81ffb700 1 -- 172.21.15.99:0/4216916666 >> v1:172.21.15.99:6789/0 conn(0x7fbc84156a50 legacy=0x7fbc840c8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:27.923 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.916+0000 7fbc81ffb700 1 --2- 172.21.15.99:0/4216916666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc6c003a90 0x7fbc6c003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:27.923 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.916+0000 7fbc81ffb700 1 --2- 172.21.15.99:0/4216916666 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbc84156e20 0x7fbc6c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:27.923 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.916+0000 7fbc81ffb700 1 --2- 172.21.15.99:0/4216916666 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc6c004d50 0x7fbc6c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:27.923 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.916+0000 7fbc81ffb700 1 -- 172.21.15.99:0/4216916666 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc84068d10 con 0x7fbc6c003a90 2023-10-08T00:11:27.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.916+0000 7fbc81ffb700 1 -- 172.21.15.99:0/4216916666 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6c00bb10 con 0x7fbc6c004d50 2023-10-08T00:11:27.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.916+0000 7fbc81ffb700 1 -- 172.21.15.99:0/4216916666 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6c00bcd0 con 0x7fbc84156e20 2023-10-08T00:11:27.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.916+0000 7fbc82ffd700 1 --2- 172.21.15.99:0/4216916666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc6c003a90 0x7fbc6c003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:27.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.916+0000 7fbc827fc700 1 --2- 172.21.15.99:0/4216916666 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbc84156e20 0x7fbc6c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:27.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.916+0000 7fbc837fe700 1 --2- 172.21.15.99:0/4216916666 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc6c004d50 0x7fbc6c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:27.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.916+0000 7fbc82ffd700 1 -- 172.21.15.99:0/4216916666 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbc84156e20 msgr2=0x7fbc6c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:27.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.916+0000 7fbc82ffd700 1 --2- 172.21.15.99:0/4216916666 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbc84156e20 0x7fbc6c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:27.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.916+0000 7fbc82ffd700 1 -- 172.21.15.99:0/4216916666 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc6c004d50 msgr2=0x7fbc6c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:27.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.916+0000 7fbc82ffd700 1 --2- 172.21.15.99:0/4216916666 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc6c004d50 0x7fbc6c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:27.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.916+0000 7fbc82ffd700 1 -- 172.21.15.99:0/4216916666 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc6c003570 con 0x7fbc6c003a90 2023-10-08T00:11:27.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.916+0000 7fbc827fc700 1 --2- 172.21.15.99:0/4216916666 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbc84156e20 0x7fbc6c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:11:27.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.916+0000 7fbc82ffd700 1 --2- 172.21.15.99:0/4216916666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc6c003a90 0x7fbc6c003e60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fbc7400a950 tx=0x7fbc740021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=787d4c9ff26fca16 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:27.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.916+0000 7fbc81ffb700 1 -- 172.21.15.99:0/4216916666 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc740070d0 con 0x7fbc6c003a90 2023-10-08T00:11:27.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.917+0000 7fbc81ffb700 1 -- 172.21.15.99:0/4216916666 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc74007250 con 0x7fbc6c003a90 2023-10-08T00:11:27.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.917+0000 7fbc81ffb700 1 -- 172.21.15.99:0/4216916666 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc74007610 con 0x7fbc6c003a90 2023-10-08T00:11:27.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.917+0000 7fbc898a0700 1 -- 172.21.15.99:0/4216916666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc6c003a90 msgr2=0x7fbc6c003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:27.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.917+0000 7fbc898a0700 1 --2- 172.21.15.99:0/4216916666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc6c003a90 0x7fbc6c003e60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fbc7400a950 tx=0x7fbc740021f0 comp rx=0 tx=0).stop 2023-10-08T00:11:27.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.917+0000 7fbc837fe700 1 -- 172.21.15.99:0/4216916666 reap_dead start 2023-10-08T00:11:27.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.917+0000 7fbc898a0700 1 -- 172.21.15.99:0/4216916666 shutdown_connections 2023-10-08T00:11:27.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.917+0000 7fbc898a0700 1 -- 172.21.15.99:0/4216916666 >> 172.21.15.99:0/4216916666 conn(0x7fbc8406a250 msgr2=0x7fbc8406a650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:27.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.917+0000 7fbc898a0700 1 -- 172.21.15.99:0/4216916666 shutdown_connections 2023-10-08T00:11:27.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.917+0000 7fbc898a0700 1 -- 172.21.15.99:0/4216916666 wait complete. 2023-10-08T00:11:27.926 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.918+0000 7fbc898a0700 1 Processor -- start 2023-10-08T00:11:27.926 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.918+0000 7fbc898a0700 1 -- start start 2023-10-08T00:11:27.926 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.919+0000 7fbc898a0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc6c003a90 0x7fbc84146b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:27.926 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.919+0000 7fbc898a0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbc840c7090 0x7fbc840aec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:27.926 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.919+0000 7fbc898a0700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc840af140 0x7fbc840b0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:27.926 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.919+0000 7fbc898a0700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc840c7500 con 0x7fbc6c003a90 2023-10-08T00:11:27.926 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.919+0000 7fbc898a0700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc840c79e0 con 0x7fbc840af140 2023-10-08T00:11:27.926 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.919+0000 7fbc898a0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc840c40b0 con 0x7fbc840c7090 2023-10-08T00:11:27.926 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.919+0000 7fbc82ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc6c003a90 0x7fbc84146b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:27.926 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.919+0000 7fbc82ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc6c003a90 0x7fbc84146b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52214/0 (socket says 172.21.15.99:52214) 2023-10-08T00:11:27.927 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.919+0000 7fbc82ffd700 1 -- 172.21.15.99:0/32795203 learned_addr learned my addr 172.21.15.99:0/32795203 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:27.927 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.919+0000 7fbc827fc700 1 --2- 172.21.15.99:0/32795203 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbc840c7090 0x7fbc840aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:27.927 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.919+0000 7fbc837fe700 1 --2- 172.21.15.99:0/32795203 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc840af140 0x7fbc840b0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:27.927 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.919+0000 7fbc82ffd700 1 -- 172.21.15.99:0/32795203 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbc840c7090 msgr2=0x7fbc840aec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:27.927 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.919+0000 7fbc82ffd700 1 --2- 172.21.15.99:0/32795203 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbc840c7090 0x7fbc840aec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:27.927 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.919+0000 7fbc82ffd700 1 -- 172.21.15.99:0/32795203 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc840af140 msgr2=0x7fbc840b0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:27.927 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.919+0000 7fbc82ffd700 1 --2- 172.21.15.99:0/32795203 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc840af140 0x7fbc840b0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:27.927 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.919+0000 7fbc82ffd700 1 -- 172.21.15.99:0/32795203 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc840c7700 con 0x7fbc6c003a90 2023-10-08T00:11:27.927 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.919+0000 7fbc827fc700 1 --2- 172.21.15.99:0/32795203 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbc840c7090 0x7fbc840aec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:11:27.927 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.920+0000 7fbc82ffd700 1 --2- 172.21.15.99:0/32795203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc6c003a90 0x7fbc84146b00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbc7400a410 tx=0x7fbc74015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:27.927 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.920+0000 7fbc6affd700 1 -- 172.21.15.99:0/32795203 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc74001250 con 0x7fbc6c003a90 2023-10-08T00:11:27.927 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.920+0000 7fbc6affd700 1 -- 172.21.15.99:0/32795203 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc74023030 con 0x7fbc6c003a90 2023-10-08T00:11:27.928 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.920+0000 7fbc898a0700 1 -- 172.21.15.99:0/32795203 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc840c4350 con 0x7fbc6c003a90 2023-10-08T00:11:27.928 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.920+0000 7fbc6affd700 1 -- 172.21.15.99:0/32795203 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc740070d0 con 0x7fbc6c003a90 2023-10-08T00:11:27.928 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.920+0000 7fbc898a0700 1 -- 172.21.15.99:0/32795203 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc841473a0 con 0x7fbc6c003a90 2023-10-08T00:11:27.929 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.921+0000 7fbc898a0700 1 -- 172.21.15.99:0/32795203 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc8404c3c0 con 0x7fbc6c003a90 2023-10-08T00:11:27.929 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.921+0000 7fbc6affd700 1 -- 172.21.15.99:0/32795203 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbc740073c0 con 0x7fbc6c003a90 2023-10-08T00:11:27.929 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.922+0000 7fbc6affd700 1 --2- 172.21.15.99:0/32795203 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbc70066f30 0x7fbc700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:27.929 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.922+0000 7fbc6affd700 1 -- 172.21.15.99:0/32795203 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbc74086570 con 0x7fbc6c003a90 2023-10-08T00:11:27.929 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.922+0000 7fbc827fc700 1 --2- 172.21.15.99:0/32795203 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbc70066f30 0x7fbc700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:27.929 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.922+0000 7fbc827fc700 1 --2- 172.21.15.99:0/32795203 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbc70066f30 0x7fbc700693e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbc78006dc0 tx=0x7fbc78008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:27.929 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:27.926+0000 7fbc6affd700 1 -- 172.21.15.99:0/32795203 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbc740537e0 con 0x7fbc6c003a90 2023-10-08T00:11:28.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.151+0000 7fbc898a0700 1 -- 172.21.15.99:0/32795203 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbc8406f890 con 0x7fbc6c003a90 2023-10-08T00:11:28.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.152+0000 7fbc6affd700 1 -- 172.21.15.99:0/32795203 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4594 (secure 0 0 0) 0x7fbc74056e00 con 0x7fbc6c003a90 2023-10-08T00:11:28.153 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:11:28.153 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/649537180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":649537180},{"type":"v1","addr":"172.21.15.99:6833","nonce":649537180}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4239,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2194110899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2194110899},{"type":"v1","addr":"172.21.15.99:6835","nonce":2194110899}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4251,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/1732443753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":1732443753},{"type":"v1","addr":"172.21.15.191:6837","nonce":1732443753}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:10:53.454321+0000","modified":"2023-10-08T00:10:57.098731+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6836/1988845948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1988845948},{"type":"v1","addr":"172.21.15.191:6836","nonce":1988845948}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-08T00:11:28.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.153+0000 7fbc898a0700 1 -- 172.21.15.99:0/32795203 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbc70066f30 msgr2=0x7fbc700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:28.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.153+0000 7fbc898a0700 1 --2- 172.21.15.99:0/32795203 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbc70066f30 0x7fbc700693e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbc78006dc0 tx=0x7fbc78008040 comp rx=0 tx=0).stop 2023-10-08T00:11:28.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.153+0000 7fbc898a0700 1 -- 172.21.15.99:0/32795203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc6c003a90 msgr2=0x7fbc84146b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:28.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.153+0000 7fbc898a0700 1 --2- 172.21.15.99:0/32795203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc6c003a90 0x7fbc84146b00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbc7400a410 tx=0x7fbc74015000 comp rx=0 tx=0).stop 2023-10-08T00:11:28.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.154+0000 7fbc898a0700 1 -- 172.21.15.99:0/32795203 shutdown_connections 2023-10-08T00:11:28.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.154+0000 7fbc898a0700 1 --2- 172.21.15.99:0/32795203 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbc70066f30 0x7fbc700693e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:28.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.154+0000 7fbc898a0700 1 --2- 172.21.15.99:0/32795203 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc840af140 0x7fbc840b0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:28.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.154+0000 7fbc898a0700 1 --2- 172.21.15.99:0/32795203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc6c003a90 0x7fbc84146b00 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:28.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.154+0000 7fbc898a0700 1 --2- 172.21.15.99:0/32795203 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbc840c7090 0x7fbc840aec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:28.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.154+0000 7fbc898a0700 1 -- 172.21.15.99:0/32795203 >> 172.21.15.99:0/32795203 conn(0x7fbc8406a250 msgr2=0x7fbc84149250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:28.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.154+0000 7fbc898a0700 1 -- 172.21.15.99:0/32795203 shutdown_connections 2023-10-08T00:11:28.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.154+0000 7fbc898a0700 1 -- 172.21.15.99:0/32795203 wait complete. 2023-10-08T00:11:28.156 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 8 2023-10-08T00:11:28.166 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:11:28.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.280+0000 7f63b4e82700 1 Processor -- start 2023-10-08T00:11:28.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.280+0000 7f63b4e82700 1 -- start start 2023-10-08T00:11:28.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.281+0000 7f63b4e82700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63b0157c40 0x7f63b0158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:28.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.281+0000 7f63b4e82700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63b015cbb0 con 0x7f63b015a620 2023-10-08T00:11:28.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.281+0000 7f63b4e82700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63b015bfa0 con 0x7f63b0157870 2023-10-08T00:11:28.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.281+0000 7f63b4e82700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63b015bdb0 con 0x7f63b0157c40 2023-10-08T00:11:28.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.281+0000 7f63add9b700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f63b015a620 0x7f63b015d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:44050/0 (socket says 172.21.15.99:44050) 2023-10-08T00:11:28.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.281+0000 7f63add9b700 1 -- 172.21.15.99:0/1855966824 learned_addr learned my addr 172.21.15.99:0/1855966824 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:28.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.281+0000 7f63aed9d700 1 --2- 172.21.15.99:0/1855966824 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63b0157c40 0x7f63b0158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:28.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.281+0000 7f63ad59a700 1 -- 172.21.15.99:0/1855966824 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1637207297 0 0) 0x7f63b015cbb0 con 0x7f63b015a620 2023-10-08T00:11:28.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.281+0000 7f63ad59a700 1 -- 172.21.15.99:0/1855966824 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6398003570 con 0x7f63b015a620 2023-10-08T00:11:28.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.281+0000 7f63ad59a700 1 -- 172.21.15.99:0/1855966824 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2887183290 0 0) 0x7f63b015bfa0 con 0x7f63b0157870 2023-10-08T00:11:28.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.281+0000 7f63ad59a700 1 -- 172.21.15.99:0/1855966824 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63b015cbb0 con 0x7f63b0157870 2023-10-08T00:11:28.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.281+0000 7f63aed9d700 1 -- 172.21.15.99:0/1855966824 >> v1:172.21.15.191:6789/0 conn(0x7f63b0157870 legacy=0x7f63b015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:28.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.281+0000 7f63aed9d700 1 -- 172.21.15.99:0/1855966824 >> v1:172.21.15.99:6789/0 conn(0x7f63b015a620 legacy=0x7f63b015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:28.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.281+0000 7f63aed9d700 1 -- 172.21.15.99:0/1855966824 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b015c280 con 0x7f63b0157c40 2023-10-08T00:11:28.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.282+0000 7f63aed9d700 1 --2- 172.21.15.99:0/1855966824 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63b0157c40 0x7f63b0158110 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f63a0009e90 tx=0x7f63a0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8892902cfa270ba4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:28.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.282+0000 7f63ad59a700 1 -- 172.21.15.99:0/1855966824 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a000e070 con 0x7f63b0157c40 2023-10-08T00:11:28.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.282+0000 7f63ad59a700 1 -- 172.21.15.99:0/1855966824 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a000bb80 con 0x7f63b0157c40 2023-10-08T00:11:28.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.282+0000 7f63ad59a700 1 -- 172.21.15.99:0/1855966824 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a0012430 con 0x7f63b0157c40 2023-10-08T00:11:28.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.282+0000 7f63b4e82700 1 -- 172.21.15.99:0/1855966824 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63b0157c40 msgr2=0x7f63b0158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:28.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.282+0000 7f63b4e82700 1 --2- 172.21.15.99:0/1855966824 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63b0157c40 0x7f63b0158110 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f63a0009e90 tx=0x7f63a0009780 comp rx=0 tx=0).stop 2023-10-08T00:11:28.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.282+0000 7f63b4e82700 1 -- 172.21.15.99:0/1855966824 shutdown_connections 2023-10-08T00:11:28.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.282+0000 7f63b4e82700 1 --2- 172.21.15.99:0/1855966824 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63b0157c40 0x7f63b0158110 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:28.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.282+0000 7f63b4e82700 1 -- 172.21.15.99:0/1855966824 >> 172.21.15.99:0/1855966824 conn(0x7f63b0068ce0 msgr2=0x7f63b00690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:28.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.282+0000 7f63b4e82700 1 -- 172.21.15.99:0/1855966824 shutdown_connections 2023-10-08T00:11:28.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.282+0000 7f63b4e82700 1 -- 172.21.15.99:0/1855966824 wait complete. 2023-10-08T00:11:28.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.283+0000 7f63b4e82700 1 Processor -- start 2023-10-08T00:11:28.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.283+0000 7f63b4e82700 1 -- start start 2023-10-08T00:11:28.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.284+0000 7f63b4e82700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63b0157870 0x7f63b014bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:28.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.284+0000 7f63b4e82700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63b0157c40 0x7f63b0154db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:28.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.284+0000 7f63b4e82700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63b015a620 0x7f63b015df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:28.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.284+0000 7f63b4e82700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b014c120 con 0x7f63b015a620 2023-10-08T00:11:28.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.284+0000 7f63b4e82700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b00b25e0 con 0x7f63b0157870 2023-10-08T00:11:28.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.284+0000 7f63b4e82700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63b00b1a10 con 0x7f63b0157c40 2023-10-08T00:11:28.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.284+0000 7f63aed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63b015a620 0x7f63b015df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:28.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.284+0000 7f63aed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63b015a620 0x7f63b015df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52226/0 (socket says 172.21.15.99:52226) 2023-10-08T00:11:28.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.284+0000 7f63aed9d700 1 -- 172.21.15.99:0/3265467691 learned_addr learned my addr 172.21.15.99:0/3265467691 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:28.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.284+0000 7f63add9b700 1 --2- 172.21.15.99:0/3265467691 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63b0157c40 0x7f63b0154db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:28.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.284+0000 7f63ae59c700 1 --2- 172.21.15.99:0/3265467691 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63b0157870 0x7f63b014bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:28.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.284+0000 7f63aed9d700 1 -- 172.21.15.99:0/3265467691 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63b0157c40 msgr2=0x7f63b0154db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:28.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.284+0000 7f63aed9d700 1 --2- 172.21.15.99:0/3265467691 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63b0157c40 0x7f63b0154db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:28.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.284+0000 7f63aed9d700 1 -- 172.21.15.99:0/3265467691 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63b0157870 msgr2=0x7f63b014bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:28.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.284+0000 7f63aed9d700 1 --2- 172.21.15.99:0/3265467691 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63b0157870 0x7f63b014bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:28.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.285+0000 7f63aed9d700 1 -- 172.21.15.99:0/3265467691 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a0009a20 con 0x7f63b015a620 2023-10-08T00:11:28.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.285+0000 7f63aed9d700 1 --2- 172.21.15.99:0/3265467691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63b015a620 0x7f63b015df10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f63a0013000 tx=0x7f63a0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:28.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.285+0000 7f639e7fc700 1 -- 172.21.15.99:0/3265467691 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a001b420 con 0x7f63b015a620 2023-10-08T00:11:28.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.285+0000 7f639e7fc700 1 -- 172.21.15.99:0/3265467691 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a001b5a0 con 0x7f63b015a620 2023-10-08T00:11:28.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.285+0000 7f63b4e82700 1 -- 172.21.15.99:0/3265467691 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63b01552f0 con 0x7f63b015a620 2023-10-08T00:11:28.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.285+0000 7f639e7fc700 1 -- 172.21.15.99:0/3265467691 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a001b960 con 0x7f63b015a620 2023-10-08T00:11:28.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.285+0000 7f63b4e82700 1 -- 172.21.15.99:0/3265467691 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63b00b1f90 con 0x7f63b015a620 2023-10-08T00:11:28.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.286+0000 7f63b4e82700 1 -- 172.21.15.99:0/3265467691 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63b00b2790 con 0x7f63b015a620 2023-10-08T00:11:28.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.286+0000 7f639e7fc700 1 -- 172.21.15.99:0/3265467691 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f63a001c400 con 0x7f63b015a620 2023-10-08T00:11:28.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.286+0000 7f639e7fc700 1 --2- 172.21.15.99:0/3265467691 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6390066f30 0x7f63900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:28.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.287+0000 7f639e7fc700 1 -- 172.21.15.99:0/3265467691 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f63a0085e20 con 0x7f63b015a620 2023-10-08T00:11:28.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.287+0000 7f63ae59c700 1 --2- 172.21.15.99:0/3265467691 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6390066f30 0x7f63900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:28.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.287+0000 7f63ae59c700 1 --2- 172.21.15.99:0/3265467691 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6390066f30 0x7f63900693e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f63a8007fd0 tx=0x7f63a8007d50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:28.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.291+0000 7f639e7fc700 1 -- 172.21.15.99:0/3265467691 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f63b00b2790 con 0x7f63b015a620 2023-10-08T00:11:28.474 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.473+0000 7f63b4e82700 1 -- 172.21.15.99:0/3265467691 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63b015ab00 con 0x7f63b015a620 2023-10-08T00:11:28.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.474+0000 7f639e7fc700 1 -- 172.21.15.99:0/3265467691 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f63a0053090 con 0x7f63b015a620 2023-10-08T00:11:28.475 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:11:28.475 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":20,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:10:55.080854+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:10:51.822157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T00:10:52.415130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:11:28.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.476+0000 7f63b4e82700 1 -- 172.21.15.99:0/3265467691 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6390066f30 msgr2=0x7f63900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:28.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.476+0000 7f63b4e82700 1 --2- 172.21.15.99:0/3265467691 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6390066f30 0x7f63900693e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f63a8007fd0 tx=0x7f63a8007d50 comp rx=0 tx=0).stop 2023-10-08T00:11:28.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.476+0000 7f63b4e82700 1 -- 172.21.15.99:0/3265467691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63b015a620 msgr2=0x7f63b015df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:28.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.476+0000 7f63b4e82700 1 --2- 172.21.15.99:0/3265467691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63b015a620 0x7f63b015df10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f63a0013000 tx=0x7f63a0019000 comp rx=0 tx=0).stop 2023-10-08T00:11:28.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.476+0000 7f63b4e82700 1 -- 172.21.15.99:0/3265467691 shutdown_connections 2023-10-08T00:11:28.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.476+0000 7f63b4e82700 1 --2- 172.21.15.99:0/3265467691 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6390066f30 0x7f63900693e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:28.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.476+0000 7f63b4e82700 1 --2- 172.21.15.99:0/3265467691 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63b0157870 0x7f63b014bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:28.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.476+0000 7f63b4e82700 1 --2- 172.21.15.99:0/3265467691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63b015a620 0x7f63b015df10 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:28.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.476+0000 7f63b4e82700 1 --2- 172.21.15.99:0/3265467691 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63b0157c40 0x7f63b0154db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:28.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.476+0000 7f63b4e82700 1 -- 172.21.15.99:0/3265467691 >> 172.21.15.99:0/3265467691 conn(0x7f63b0068ce0 msgr2=0x7f63b0159820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:28.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.476+0000 7f63b4e82700 1 -- 172.21.15.99:0/3265467691 shutdown_connections 2023-10-08T00:11:28.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.476+0000 7f63b4e82700 1 -- 172.21.15.99:0/3265467691 wait complete. 2023-10-08T00:11:28.490 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T00:11:28.490 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:11:28.604 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.603+0000 7f431edbc700 1 Processor -- start 2023-10-08T00:11:28.604 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.603+0000 7f431edbc700 1 -- start start 2023-10-08T00:11:28.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.604+0000 7f431edbc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4318070950 0x7f4318068890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:28.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.604+0000 7f431edbc700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4318143700 0x7f43180cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:28.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.604+0000 7f431edbc700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43180677f0 con 0x7f4318070d20 2023-10-08T00:11:28.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.604+0000 7f431edbc700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4318067ad0 con 0x7f4318143700 2023-10-08T00:11:28.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.604+0000 7f431edbc700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4318067db0 con 0x7f4318070950 2023-10-08T00:11:28.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.604+0000 7f431d359700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4318143700 0x7f43180cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:28.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.604+0000 7f431d359700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4318143700 0x7f43180cbe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52234/0 (socket says 172.21.15.99:52234) 2023-10-08T00:11:28.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.604+0000 7f431d359700 1 -- 172.21.15.99:0/3320767559 learned_addr learned my addr 172.21.15.99:0/3320767559 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:28.608 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.605+0000 7f431cb58700 1 --2- 172.21.15.99:0/3320767559 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4318070950 0x7f4318068890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:28.608 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.605+0000 7f431d359700 1 -- 172.21.15.99:0/3320767559 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4318070950 msgr2=0x7f4318068890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:28.608 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.605+0000 7f431d359700 1 --2- 172.21.15.99:0/3320767559 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4318070950 0x7f4318068890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:28.608 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.605+0000 7f431d359700 1 -- 172.21.15.99:0/3320767559 >> v1:172.21.15.191:6789/0 conn(0x7f4318070d20 legacy=0x7f4318142ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:28.608 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.605+0000 7f431d359700 1 -- 172.21.15.99:0/3320767559 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4318068090 con 0x7f4318143700 2023-10-08T00:11:28.608 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.605+0000 7f431d359700 1 --2- 172.21.15.99:0/3320767559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4318143700 0x7f43180cbe70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f43100099b0 tx=0x7f4310009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=135aea2e9e1dd272 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:28.608 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.605+0000 7f43177fe700 1 -- 172.21.15.99:0/3320767559 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431000b860 con 0x7f4318143700 2023-10-08T00:11:28.608 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.605+0000 7f43177fe700 1 -- 172.21.15.99:0/3320767559 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f431000b9c0 con 0x7f4318143700 2023-10-08T00:11:28.609 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.605+0000 7f43177fe700 1 -- 172.21.15.99:0/3320767559 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431000bd60 con 0x7f4318143700 2023-10-08T00:11:28.609 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.605+0000 7f431edbc700 1 -- 172.21.15.99:0/3320767559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4318143700 msgr2=0x7f43180cbe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:28.609 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.605+0000 7f431edbc700 1 --2- 172.21.15.99:0/3320767559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4318143700 0x7f43180cbe70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f43100099b0 tx=0x7f4310009250 comp rx=0 tx=0).stop 2023-10-08T00:11:28.609 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.605+0000 7f431edbc700 1 -- 172.21.15.99:0/3320767559 shutdown_connections 2023-10-08T00:11:28.609 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.605+0000 7f431edbc700 1 --2- 172.21.15.99:0/3320767559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4318143700 0x7f43180cbe70 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:28.609 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.606+0000 7f431edbc700 1 --2- 172.21.15.99:0/3320767559 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4318070950 0x7f4318068890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:28.609 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.606+0000 7f431edbc700 1 -- 172.21.15.99:0/3320767559 >> 172.21.15.99:0/3320767559 conn(0x7f431806ad20 msgr2=0x7f43180cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:28.609 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.606+0000 7f431edbc700 1 -- 172.21.15.99:0/3320767559 shutdown_connections 2023-10-08T00:11:28.609 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.606+0000 7f431edbc700 1 -- 172.21.15.99:0/3320767559 wait complete. 2023-10-08T00:11:28.612 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.607+0000 7f431edbc700 1 Processor -- start 2023-10-08T00:11:28.612 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.607+0000 7f431edbc700 1 -- start start 2023-10-08T00:11:28.613 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.607+0000 7f431edbc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4318070950 0x7f4318159700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:28.613 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.607+0000 7f431edbc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4318070d20 0x7f431813f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:28.613 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.607+0000 7f431edbc700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4318143700 0x7f431813a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:28.613 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.607+0000 7f431edbc700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4318159c40 con 0x7f4318070d20 2023-10-08T00:11:28.613 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.607+0000 7f431edbc700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43181383d0 con 0x7f4318143700 2023-10-08T00:11:28.613 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.607+0000 7f431edbc700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43181377c0 con 0x7f4318070950 2023-10-08T00:11:28.613 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.608+0000 7f4317fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4318070d20 0x7f431813f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:28.613 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.608+0000 7f4317fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4318070d20 0x7f431813f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52240/0 (socket says 172.21.15.99:52240) 2023-10-08T00:11:28.613 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.608+0000 7f4317fff700 1 -- 172.21.15.99:0/2857415391 learned_addr learned my addr 172.21.15.99:0/2857415391 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:28.614 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.608+0000 7f431d359700 1 --2- 172.21.15.99:0/2857415391 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4318143700 0x7f431813a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:28.614 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.608+0000 7f431cb58700 1 --2- 172.21.15.99:0/2857415391 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4318070950 0x7f4318159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:28.614 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.608+0000 7f4317fff700 1 -- 172.21.15.99:0/2857415391 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4318070950 msgr2=0x7f4318159700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:28.614 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.608+0000 7f4317fff700 1 --2- 172.21.15.99:0/2857415391 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4318070950 0x7f4318159700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:28.614 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.608+0000 7f4317fff700 1 -- 172.21.15.99:0/2857415391 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4318143700 msgr2=0x7f431813a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:28.614 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.608+0000 7f4317fff700 1 --2- 172.21.15.99:0/2857415391 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4318143700 0x7f431813a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:28.614 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.608+0000 7f4317fff700 1 -- 172.21.15.99:0/2857415391 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43100094f0 con 0x7f4318070d20 2023-10-08T00:11:28.614 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.608+0000 7f4317fff700 1 --2- 172.21.15.99:0/2857415391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4318070d20 0x7f431813f690 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f430c00c590 tx=0x7f430c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:28.614 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.608+0000 7f4314ff9700 1 -- 172.21.15.99:0/2857415391 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c011590 con 0x7f4318070d20 2023-10-08T00:11:28.615 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.608+0000 7f4314ff9700 1 -- 172.21.15.99:0/2857415391 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f430c0116f0 con 0x7f4318070d20 2023-10-08T00:11:28.615 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.608+0000 7f431edbc700 1 -- 172.21.15.99:0/2857415391 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4318137aa0 con 0x7f4318070d20 2023-10-08T00:11:28.615 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.608+0000 7f4314ff9700 1 -- 172.21.15.99:0/2857415391 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c011ab0 con 0x7f4318070d20 2023-10-08T00:11:28.615 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.608+0000 7f431edbc700 1 -- 172.21.15.99:0/2857415391 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4318138060 con 0x7f4318070d20 2023-10-08T00:11:28.615 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.609+0000 7f431edbc700 1 -- 172.21.15.99:0/2857415391 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4318138540 con 0x7f4318070d20 2023-10-08T00:11:28.615 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.609+0000 7f4314ff9700 1 -- 172.21.15.99:0/2857415391 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f430c01b3b0 con 0x7f4318070d20 2023-10-08T00:11:28.615 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.610+0000 7f4314ff9700 1 --2- 172.21.15.99:0/2857415391 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4304066f30 0x7f43040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:28.615 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.610+0000 7f4314ff9700 1 -- 172.21.15.99:0/2857415391 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f430c083ae0 con 0x7f4318070d20 2023-10-08T00:11:28.615 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.610+0000 7f431cb58700 1 --2- 172.21.15.99:0/2857415391 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4304066f30 0x7f43040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:28.615 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.611+0000 7f431cb58700 1 --2- 172.21.15.99:0/2857415391 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4304066f30 0x7f43040693e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f43080075a0 tx=0x7f43080097a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:28.616 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.614+0000 7f4314ff9700 1 -- 172.21.15.99:0/2857415391 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f430c050d50 con 0x7f4318070d20 2023-10-08T00:11:28.836 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.836+0000 7f431edbc700 1 -- 172.21.15.99:0/2857415391 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f431804c3c0 con 0x7f4318070d20 2023-10-08T00:11:28.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.836+0000 7f4314ff9700 1 -- 172.21.15.99:0/2857415391 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4594 (secure 0 0 0) 0x7f430c017070 con 0x7f4318070d20 2023-10-08T00:11:28.837 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:11:28.837 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/649537180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":649537180},{"type":"v1","addr":"172.21.15.99:6833","nonce":649537180}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4239,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2194110899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2194110899},{"type":"v1","addr":"172.21.15.99:6835","nonce":2194110899}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4251,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/1732443753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":1732443753},{"type":"v1","addr":"172.21.15.191:6837","nonce":1732443753}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:10:53.454321+0000","modified":"2023-10-08T00:10:57.098731+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6836/1988845948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1988845948},{"type":"v1","addr":"172.21.15.191:6836","nonce":1988845948}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-08T00:11:28.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.838+0000 7f431edbc700 1 -- 172.21.15.99:0/2857415391 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4304066f30 msgr2=0x7f43040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:28.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.838+0000 7f431edbc700 1 --2- 172.21.15.99:0/2857415391 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4304066f30 0x7f43040693e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f43080075a0 tx=0x7f43080097a0 comp rx=0 tx=0).stop 2023-10-08T00:11:28.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.838+0000 7f431edbc700 1 -- 172.21.15.99:0/2857415391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4318070d20 msgr2=0x7f431813f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:28.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.838+0000 7f431edbc700 1 --2- 172.21.15.99:0/2857415391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4318070d20 0x7f431813f690 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f430c00c590 tx=0x7f430c00c950 comp rx=0 tx=0).stop 2023-10-08T00:11:28.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.839+0000 7f431edbc700 1 -- 172.21.15.99:0/2857415391 shutdown_connections 2023-10-08T00:11:28.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.839+0000 7f431edbc700 1 --2- 172.21.15.99:0/2857415391 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4304066f30 0x7f43040693e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:28.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.839+0000 7f431edbc700 1 --2- 172.21.15.99:0/2857415391 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4318143700 0x7f431813a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:28.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.839+0000 7f431edbc700 1 --2- 172.21.15.99:0/2857415391 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4318070950 0x7f4318159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:28.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.839+0000 7f431edbc700 1 --2- 172.21.15.99:0/2857415391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4318070d20 0x7f431813f690 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:28.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.839+0000 7f431edbc700 1 -- 172.21.15.99:0/2857415391 >> 172.21.15.99:0/2857415391 conn(0x7f431806ad20 msgr2=0x7f431813ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:28.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.839+0000 7f431edbc700 1 -- 172.21.15.99:0/2857415391 shutdown_connections 2023-10-08T00:11:28.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.839+0000 7f431edbc700 1 -- 172.21.15.99:0/2857415391 wait complete. 2023-10-08T00:11:28.841 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 8 2023-10-08T00:11:28.851 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:11:28.966 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.965+0000 7f5e36424700 1 Processor -- start 2023-10-08T00:11:28.966 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.965+0000 7f5e36424700 1 -- start start 2023-10-08T00:11:28.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.966+0000 7f5e36424700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e30143520 0x7f5e300cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:28.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.966+0000 7f5e36424700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5e3006ab50 0x7f5e300b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:28.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.966+0000 7f5e36424700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e300d0870 con 0x7f5e301438f0 2023-10-08T00:11:28.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.966+0000 7f5e36424700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e300cdfb0 con 0x7f5e3006ab50 2023-10-08T00:11:28.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.966+0000 7f5e36424700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e300cfc60 con 0x7f5e30143520 2023-10-08T00:11:28.970 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.966+0000 7f5e35c23700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5e3006ab50 0x7f5e300b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:28.970 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.966+0000 7f5e35c23700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5e3006ab50 0x7f5e300b3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52242/0 (socket says 172.21.15.99:52242) 2023-10-08T00:11:28.970 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.966+0000 7f5e35c23700 1 -- 172.21.15.99:0/1425974069 learned_addr learned my addr 172.21.15.99:0/1425974069 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:28.970 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.966+0000 7f5e35422700 1 --2- 172.21.15.99:0/1425974069 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e30143520 0x7f5e300cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:28.970 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.966+0000 7f5e35c23700 1 -- 172.21.15.99:0/1425974069 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e30143520 msgr2=0x7f5e300cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:28.970 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.966+0000 7f5e35c23700 1 --2- 172.21.15.99:0/1425974069 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e30143520 0x7f5e300cf580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:28.970 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.966+0000 7f5e35c23700 1 -- 172.21.15.99:0/1425974069 >> v1:172.21.15.191:6789/0 conn(0x7f5e301438f0 legacy=0x7f5e3006a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:28.971 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.966+0000 7f5e35c23700 1 -- 172.21.15.99:0/1425974069 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e300cff40 con 0x7f5e3006ab50 2023-10-08T00:11:28.971 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.967+0000 7f5e35c23700 1 --2- 172.21.15.99:0/1425974069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e3006ab50 0x7f5e300b3ac0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5e28009f20 tx=0x7f5e28009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d488b10762400bc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:28.971 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.967+0000 7f5e27fff700 1 -- 172.21.15.99:0/1425974069 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2800e070 con 0x7f5e3006ab50 2023-10-08T00:11:28.971 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.967+0000 7f5e27fff700 1 -- 172.21.15.99:0/1425974069 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e2800b9d0 con 0x7f5e3006ab50 2023-10-08T00:11:28.971 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.967+0000 7f5e27fff700 1 -- 172.21.15.99:0/1425974069 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e280092e0 con 0x7f5e3006ab50 2023-10-08T00:11:28.971 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.967+0000 7f5e36424700 1 -- 172.21.15.99:0/1425974069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e3006ab50 msgr2=0x7f5e300b3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:28.971 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.967+0000 7f5e36424700 1 --2- 172.21.15.99:0/1425974069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e3006ab50 0x7f5e300b3ac0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5e28009f20 tx=0x7f5e28009840 comp rx=0 tx=0).stop 2023-10-08T00:11:28.972 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.967+0000 7f5e36424700 1 -- 172.21.15.99:0/1425974069 shutdown_connections 2023-10-08T00:11:28.972 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.967+0000 7f5e36424700 1 --2- 172.21.15.99:0/1425974069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e3006ab50 0x7f5e300b3ac0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:28.972 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.967+0000 7f5e36424700 1 --2- 172.21.15.99:0/1425974069 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e30143520 0x7f5e300cf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:28.972 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.967+0000 7f5e36424700 1 -- 172.21.15.99:0/1425974069 >> 172.21.15.99:0/1425974069 conn(0x7f5e30069750 msgr2=0x7f5e300b25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:28.972 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.967+0000 7f5e36424700 1 -- 172.21.15.99:0/1425974069 shutdown_connections 2023-10-08T00:11:28.972 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.967+0000 7f5e36424700 1 -- 172.21.15.99:0/1425974069 wait complete. 2023-10-08T00:11:28.972 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.968+0000 7f5e36424700 1 Processor -- start 2023-10-08T00:11:28.972 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.968+0000 7f5e36424700 1 -- start start 2023-10-08T00:11:28.975 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.969+0000 7f5e36424700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e3006ab50 0x7f5e3016d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:28.975 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.969+0000 7f5e36424700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e30143520 0x7f5e3015a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:28.975 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.969+0000 7f5e36424700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e301438f0 0x7f5e30140a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:28.975 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.969+0000 7f5e36424700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3015d410 con 0x7f5e301438f0 2023-10-08T00:11:28.975 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.969+0000 7f5e36424700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3015aec0 con 0x7f5e30143520 2023-10-08T00:11:28.975 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.969+0000 7f5e36424700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3015c800 con 0x7f5e3006ab50 2023-10-08T00:11:28.975 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.969+0000 7f5e35c23700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e301438f0 0x7f5e30140a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:28.976 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.969+0000 7f5e35c23700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e301438f0 0x7f5e30140a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52246/0 (socket says 172.21.15.99:52246) 2023-10-08T00:11:28.976 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.969+0000 7f5e35c23700 1 -- 172.21.15.99:0/2503786725 learned_addr learned my addr 172.21.15.99:0/2503786725 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:28.976 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.970+0000 7f5e35422700 1 --2- 172.21.15.99:0/2503786725 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e3006ab50 0x7f5e3016d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:28.976 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.970+0000 7f5e34c21700 1 --2- 172.21.15.99:0/2503786725 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e30143520 0x7f5e3015a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:28.976 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.970+0000 7f5e35c23700 1 -- 172.21.15.99:0/2503786725 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e3006ab50 msgr2=0x7f5e3016d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:28.976 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.970+0000 7f5e35c23700 1 --2- 172.21.15.99:0/2503786725 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e3006ab50 0x7f5e3016d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:28.976 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.970+0000 7f5e35c23700 1 -- 172.21.15.99:0/2503786725 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e30143520 msgr2=0x7f5e3015a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:28.976 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.970+0000 7f5e35c23700 1 --2- 172.21.15.99:0/2503786725 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e30143520 0x7f5e3015a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:28.976 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.970+0000 7f5e35c23700 1 -- 172.21.15.99:0/2503786725 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e28009ae0 con 0x7f5e301438f0 2023-10-08T00:11:28.976 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.970+0000 7f5e35c23700 1 --2- 172.21.15.99:0/2503786725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e301438f0 0x7f5e30140a00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5e30071eb0 tx=0x7f5e28018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:28.976 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.970+0000 7f5e257fa700 1 -- 172.21.15.99:0/2503786725 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e28008ad0 con 0x7f5e301438f0 2023-10-08T00:11:28.976 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.970+0000 7f5e257fa700 1 -- 172.21.15.99:0/2503786725 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e28008c30 con 0x7f5e301438f0 2023-10-08T00:11:28.977 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.970+0000 7f5e36424700 1 -- 172.21.15.99:0/2503786725 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e3015cae0 con 0x7f5e301438f0 2023-10-08T00:11:28.977 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.970+0000 7f5e257fa700 1 -- 172.21.15.99:0/2503786725 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e28026460 con 0x7f5e301438f0 2023-10-08T00:11:28.977 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.970+0000 7f5e36424700 1 -- 172.21.15.99:0/2503786725 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e3015d0a0 con 0x7f5e301438f0 2023-10-08T00:11:28.977 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.971+0000 7f5e36424700 1 -- 172.21.15.99:0/2503786725 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e300d0a00 con 0x7f5e301438f0 2023-10-08T00:11:28.977 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.971+0000 7f5e257fa700 1 -- 172.21.15.99:0/2503786725 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5e28003040 con 0x7f5e301438f0 2023-10-08T00:11:28.977 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.972+0000 7f5e257fa700 1 --2- 172.21.15.99:0/2503786725 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5e14066f30 0x7f5e140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:28.977 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.972+0000 7f5e257fa700 1 -- 172.21.15.99:0/2503786725 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5e28086350 con 0x7f5e301438f0 2023-10-08T00:11:28.977 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.972+0000 7f5e35422700 1 --2- 172.21.15.99:0/2503786725 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5e14066f30 0x7f5e140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:28.977 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.973+0000 7f5e35422700 1 --2- 172.21.15.99:0/2503786725 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5e14066f30 0x7f5e140693e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5e3013e560 tx=0x7f5e2c0091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:28.978 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:28.976+0000 7f5e257fa700 1 -- 172.21.15.99:0/2503786725 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5e280535c0 con 0x7f5e301438f0 2023-10-08T00:11:29.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.199+0000 7f5e36424700 1 -- 172.21.15.99:0/2503786725 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e30089030 con 0x7f5e301438f0 2023-10-08T00:11:29.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.199+0000 7f5e257fa700 1 -- 172.21.15.99:0/2503786725 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4594 (secure 0 0 0) 0x7f5e28056be0 con 0x7f5e301438f0 2023-10-08T00:11:29.200 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:11:29.200 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/649537180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":649537180},{"type":"v1","addr":"172.21.15.99:6833","nonce":649537180}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4239,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2194110899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2194110899},{"type":"v1","addr":"172.21.15.99:6835","nonce":2194110899}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4251,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/1732443753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":1732443753},{"type":"v1","addr":"172.21.15.191:6837","nonce":1732443753}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:10:53.454321+0000","modified":"2023-10-08T00:10:57.098731+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6836/1988845948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1988845948},{"type":"v1","addr":"172.21.15.191:6836","nonce":1988845948}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-08T00:11:29.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.201+0000 7f5e36424700 1 -- 172.21.15.99:0/2503786725 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5e14066f30 msgr2=0x7f5e140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:29.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.201+0000 7f5e36424700 1 --2- 172.21.15.99:0/2503786725 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5e14066f30 0x7f5e140693e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5e3013e560 tx=0x7f5e2c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:29.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.201+0000 7f5e36424700 1 -- 172.21.15.99:0/2503786725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e301438f0 msgr2=0x7f5e30140a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:29.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.201+0000 7f5e36424700 1 --2- 172.21.15.99:0/2503786725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e301438f0 0x7f5e30140a00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5e30071eb0 tx=0x7f5e28018000 comp rx=0 tx=0).stop 2023-10-08T00:11:29.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.201+0000 7f5e36424700 1 -- 172.21.15.99:0/2503786725 shutdown_connections 2023-10-08T00:11:29.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.201+0000 7f5e36424700 1 --2- 172.21.15.99:0/2503786725 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5e14066f30 0x7f5e140693e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:29.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.201+0000 7f5e36424700 1 --2- 172.21.15.99:0/2503786725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e301438f0 0x7f5e30140a00 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:29.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.201+0000 7f5e36424700 1 --2- 172.21.15.99:0/2503786725 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e3006ab50 0x7f5e3016d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:29.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.201+0000 7f5e36424700 1 --2- 172.21.15.99:0/2503786725 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e30143520 0x7f5e3015a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:29.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.201+0000 7f5e36424700 1 -- 172.21.15.99:0/2503786725 >> 172.21.15.99:0/2503786725 conn(0x7f5e30069750 msgr2=0x7f5e300b5f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:29.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.201+0000 7f5e36424700 1 -- 172.21.15.99:0/2503786725 shutdown_connections 2023-10-08T00:11:29.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.201+0000 7f5e36424700 1 -- 172.21.15.99:0/2503786725 wait complete. 2023-10-08T00:11:29.203 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 8 2023-10-08T00:11:29.213 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:11:29.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.327+0000 7faac9afa700 1 Processor -- start 2023-10-08T00:11:29.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.327+0000 7faac9afa700 1 -- start start 2023-10-08T00:11:29.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.328+0000 7faac9afa700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faac40cd420 0x7faac40cb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:29.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.328+0000 7faac9afa700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7faac4156d50 0x7faac40cbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:29.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.328+0000 7faac9afa700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faac40ad5c0 con 0x7faac4156980 2023-10-08T00:11:29.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.328+0000 7faac9afa700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faac40ac9b0 con 0x7faac4156d50 2023-10-08T00:11:29.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.328+0000 7faac9afa700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faac40acc90 con 0x7faac40cd420 2023-10-08T00:11:29.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.328+0000 7faac3fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faac40cd420 0x7faac40cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:29.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.328+0000 7faac92f9700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7faac4156d50 0x7faac40cbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:29.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.328+0000 7faac3fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faac40cd420 0x7faac40cb800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.99:33710/0 (socket says 172.21.15.99:33710) 2023-10-08T00:11:29.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.328+0000 7faac3fff700 1 -- 172.21.15.99:0/130155565 learned_addr learned my addr 172.21.15.99:0/130155565 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:29.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.328+0000 7faac8af8700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7faac4156980 0x7faac40ac0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:44036/0 (socket says 172.21.15.99:44036) 2023-10-08T00:11:29.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.328+0000 7faac37fe700 1 -- 172.21.15.99:0/130155565 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2906727881 0 0) 0x7faac40ad5c0 con 0x7faac4156980 2023-10-08T00:11:29.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.328+0000 7faac37fe700 1 -- 172.21.15.99:0/130155565 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faab0003570 con 0x7faac4156980 2023-10-08T00:11:29.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.328+0000 7faac3fff700 1 -- 172.21.15.99:0/130155565 >> v2:172.21.15.191:3300/0 conn(0x7faac4156d50 msgr2=0x7faac40cbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:29.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.328+0000 7faac3fff700 1 --2- 172.21.15.99:0/130155565 >> v2:172.21.15.191:3300/0 conn(0x7faac4156d50 0x7faac40cbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:29.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.328+0000 7faac3fff700 1 -- 172.21.15.99:0/130155565 >> v1:172.21.15.191:6789/0 conn(0x7faac4156980 legacy=0x7faac40ac0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:29.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.329+0000 7faac3fff700 1 -- 172.21.15.99:0/130155565 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac40acf70 con 0x7faac40cd420 2023-10-08T00:11:29.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.329+0000 7faac92f9700 1 --2- 172.21.15.99:0/130155565 >> v2:172.21.15.191:3300/0 conn(0x7faac4156d50 0x7faac40cbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:11:29.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.329+0000 7faac3fff700 1 --2- 172.21.15.99:0/130155565 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faac40cd420 0x7faac40cb800 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7faab400bc50 tx=0x7faab400b4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81c5c49eb6ca41f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:29.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.329+0000 7faac37fe700 1 -- 172.21.15.99:0/130155565 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab400db40 con 0x7faac40cd420 2023-10-08T00:11:29.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.329+0000 7faac37fe700 1 -- 172.21.15.99:0/130155565 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faab400dca0 con 0x7faac40cd420 2023-10-08T00:11:29.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.329+0000 7faac37fe700 1 -- 172.21.15.99:0/130155565 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab4019690 con 0x7faac40cd420 2023-10-08T00:11:29.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.329+0000 7faac9afa700 1 -- 172.21.15.99:0/130155565 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faac40cd420 msgr2=0x7faac40cb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:29.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.329+0000 7faac9afa700 1 --2- 172.21.15.99:0/130155565 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faac40cd420 0x7faac40cb800 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7faab400bc50 tx=0x7faab400b4f0 comp rx=0 tx=0).stop 2023-10-08T00:11:29.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.329+0000 7faac9afa700 1 -- 172.21.15.99:0/130155565 shutdown_connections 2023-10-08T00:11:29.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.329+0000 7faac9afa700 1 --2- 172.21.15.99:0/130155565 >> v2:172.21.15.191:3300/0 conn(0x7faac4156d50 0x7faac40cbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:29.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.329+0000 7faac9afa700 1 --2- 172.21.15.99:0/130155565 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faac40cd420 0x7faac40cb800 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:29.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.329+0000 7faac9afa700 1 -- 172.21.15.99:0/130155565 >> 172.21.15.99:0/130155565 conn(0x7faac40680e0 msgr2=0x7faac40cc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:29.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.329+0000 7faac9afa700 1 -- 172.21.15.99:0/130155565 shutdown_connections 2023-10-08T00:11:29.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.329+0000 7faac9afa700 1 -- 172.21.15.99:0/130155565 wait complete. 2023-10-08T00:11:29.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.330+0000 7faac9afa700 1 Processor -- start 2023-10-08T00:11:29.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.331+0000 7faac9afa700 1 -- start start 2023-10-08T00:11:29.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.331+0000 7faac9afa700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faac40cd420 0x7faac414d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:29.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.331+0000 7faac9afa700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faac4156980 0x7faac40b0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:29.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.331+0000 7faac9afa700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faac4156d50 0x7faac4150f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:29.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.331+0000 7faac9afa700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac4157670 con 0x7faac40cd420 2023-10-08T00:11:29.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.331+0000 7faac9afa700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac40ab020 con 0x7faac4156980 2023-10-08T00:11:29.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.331+0000 7faac9afa700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faac40b0b30 con 0x7faac4156d50 2023-10-08T00:11:29.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.331+0000 7faac8af8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faac40cd420 0x7faac414d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:29.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.331+0000 7faac8af8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faac40cd420 0x7faac414d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52252/0 (socket says 172.21.15.99:52252) 2023-10-08T00:11:29.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.331+0000 7faac8af8700 1 -- 172.21.15.99:0/3971287783 learned_addr learned my addr 172.21.15.99:0/3971287783 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:29.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.332+0000 7faac3fff700 1 --2- 172.21.15.99:0/3971287783 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faac4156980 0x7faac40b0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:29.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.332+0000 7faac92f9700 1 --2- 172.21.15.99:0/3971287783 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faac4156d50 0x7faac4150f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:29.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.332+0000 7faac8af8700 1 -- 172.21.15.99:0/3971287783 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faac4156d50 msgr2=0x7faac4150f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:29.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.332+0000 7faac8af8700 1 --2- 172.21.15.99:0/3971287783 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faac4156d50 0x7faac4150f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:29.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.332+0000 7faac8af8700 1 -- 172.21.15.99:0/3971287783 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faac4156980 msgr2=0x7faac40b0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:29.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.332+0000 7faac8af8700 1 --2- 172.21.15.99:0/3971287783 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faac4156980 0x7faac40b0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:29.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.332+0000 7faac8af8700 1 -- 172.21.15.99:0/3971287783 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaac009420 con 0x7faac40cd420 2023-10-08T00:11:29.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.332+0000 7faac8af8700 1 --2- 172.21.15.99:0/3971287783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faac40cd420 0x7faac414d6c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7faabc00cdb0 tx=0x7faabc001b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:29.347 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.332+0000 7faac0ff9700 1 -- 172.21.15.99:0/3971287783 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faabc005df0 con 0x7faac40cd420 2023-10-08T00:11:29.347 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.332+0000 7faac9afa700 1 -- 172.21.15.99:0/3971287783 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faab400b790 con 0x7faac40cd420 2023-10-08T00:11:29.347 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.332+0000 7faac0ff9700 1 -- 172.21.15.99:0/3971287783 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faabc00a960 con 0x7faac40cd420 2023-10-08T00:11:29.347 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.332+0000 7faac0ff9700 1 -- 172.21.15.99:0/3971287783 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faabc00ad00 con 0x7faac40cd420 2023-10-08T00:11:29.347 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.332+0000 7faac9afa700 1 -- 172.21.15.99:0/3971287783 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faac40b10f0 con 0x7faac40cd420 2023-10-08T00:11:29.347 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.333+0000 7faac9afa700 1 -- 172.21.15.99:0/3971287783 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faac406bc70 con 0x7faac40cd420 2023-10-08T00:11:29.347 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.334+0000 7faac0ff9700 1 -- 172.21.15.99:0/3971287783 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faabc007300 con 0x7faac40cd420 2023-10-08T00:11:29.348 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.334+0000 7faac0ff9700 1 --2- 172.21.15.99:0/3971287783 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7faaa4066f30 0x7faaa40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:29.348 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.334+0000 7faac0ff9700 1 -- 172.21.15.99:0/3971287783 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7faabc084650 con 0x7faac40cd420 2023-10-08T00:11:29.348 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.334+0000 7faac3fff700 1 --2- 172.21.15.99:0/3971287783 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7faaa4066f30 0x7faaa40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:29.348 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.335+0000 7faac3fff700 1 --2- 172.21.15.99:0/3971287783 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7faaa4066f30 0x7faaa40693e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7faab400bc50 tx=0x7faab401b000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:29.348 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.338+0000 7faac0ff9700 1 -- 172.21.15.99:0/3971287783 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faabc051980 con 0x7faac40cd420 2023-10-08T00:11:29.521 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.520+0000 7faac9afa700 1 -- 172.21.15.99:0/3971287783 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faac4153ea0 con 0x7faac40cd420 2023-10-08T00:11:29.522 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.521+0000 7faac0ff9700 1 -- 172.21.15.99:0/3971287783 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15350 (secure 0 0 0) 0x7faabc00ae60 con 0x7faac40cd420 2023-10-08T00:11:29.523 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:11:29.523 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":20,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:10:55.080854+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:10:51.822157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T00:10:52.415130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:11:29.524 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.523+0000 7faac9afa700 1 -- 172.21.15.99:0/3971287783 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7faaa4066f30 msgr2=0x7faaa40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:29.524 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.523+0000 7faac9afa700 1 --2- 172.21.15.99:0/3971287783 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7faaa4066f30 0x7faaa40693e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7faab400bc50 tx=0x7faab401b000 comp rx=0 tx=0).stop 2023-10-08T00:11:29.524 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.523+0000 7faac9afa700 1 -- 172.21.15.99:0/3971287783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faac40cd420 msgr2=0x7faac414d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:29.524 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.523+0000 7faac9afa700 1 --2- 172.21.15.99:0/3971287783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faac40cd420 0x7faac414d6c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7faabc00cdb0 tx=0x7faabc001b90 comp rx=0 tx=0).stop 2023-10-08T00:11:29.524 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.523+0000 7faac9afa700 1 -- 172.21.15.99:0/3971287783 shutdown_connections 2023-10-08T00:11:29.524 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.523+0000 7faac9afa700 1 --2- 172.21.15.99:0/3971287783 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7faaa4066f30 0x7faaa40693e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:29.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.523+0000 7faac9afa700 1 --2- 172.21.15.99:0/3971287783 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faac4156d50 0x7faac4150f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:29.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.523+0000 7faac9afa700 1 --2- 172.21.15.99:0/3971287783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faac40cd420 0x7faac414d6c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:29.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.523+0000 7faac9afa700 1 --2- 172.21.15.99:0/3971287783 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faac4156980 0x7faac40b0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:29.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.523+0000 7faac9afa700 1 -- 172.21.15.99:0/3971287783 >> 172.21.15.99:0/3971287783 conn(0x7faac40680e0 msgr2=0x7faac4154800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:29.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.523+0000 7faac9afa700 1 -- 172.21.15.99:0/3971287783 shutdown_connections 2023-10-08T00:11:29.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.523+0000 7faac9afa700 1 -- 172.21.15.99:0/3971287783 wait complete. 2023-10-08T00:11:29.536 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T00:11:29.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.649+0000 7f2a82312700 1 Processor -- start 2023-10-08T00:11:29.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.649+0000 7f2a82312700 1 -- start start 2023-10-08T00:11:29.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.650+0000 7f2a82312700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a7c1577d0 0x7f2a7c0cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:29.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.650+0000 7f2a82312700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2a7c06ac30 0x7f2a7c15c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:29.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.650+0000 7f2a82312700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a7c15c360 con 0x7f2a7c157ba0 2023-10-08T00:11:29.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.650+0000 7f2a82312700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a7c15b790 con 0x7f2a7c06ac30 2023-10-08T00:11:29.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.650+0000 7f2a82312700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7c15ba30 con 0x7f2a7c1577d0 2023-10-08T00:11:29.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.650+0000 7f2a7b7fe700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f2a7c157ba0 0x7f2a7c06a520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:44058/0 (socket says 172.21.15.99:44058) 2023-10-08T00:11:29.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.650+0000 7f2a7b7fe700 1 -- 172.21.15.99:0/1650472229 learned_addr learned my addr 172.21.15.99:0/1650472229 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:29.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.651+0000 7f2a7bfff700 1 --2- 172.21.15.99:0/1650472229 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a7c1577d0 0x7f2a7c0cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:29.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.651+0000 7f2a808af700 1 --2- 172.21.15.99:0/1650472229 >> v2:172.21.15.191:3300/0 conn(0x7f2a7c06ac30 0x7f2a7c15c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:29.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.651+0000 7f2a7affd700 1 -- 172.21.15.99:0/1650472229 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3154318274 0 0) 0x7f2a7c15c360 con 0x7f2a7c157ba0 2023-10-08T00:11:29.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.651+0000 7f2a7affd700 1 -- 172.21.15.99:0/1650472229 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a64003570 con 0x7f2a7c157ba0 2023-10-08T00:11:29.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.651+0000 7f2a7affd700 1 -- 172.21.15.99:0/1650472229 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 530170669 0 0) 0x7f2a64003570 con 0x7f2a7c157ba0 2023-10-08T00:11:29.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.651+0000 7f2a7affd700 1 -- 172.21.15.99:0/1650472229 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a7c1577d0 msgr2=0x7f2a7c0cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:29.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.651+0000 7f2a7affd700 1 --2- 172.21.15.99:0/1650472229 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a7c1577d0 0x7f2a7c0cd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:29.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.651+0000 7f2a7affd700 1 -- 172.21.15.99:0/1650472229 >> v2:172.21.15.191:3300/0 conn(0x7f2a7c06ac30 msgr2=0x7f2a7c15c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:29.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.651+0000 7f2a7affd700 1 --2- 172.21.15.99:0/1650472229 >> v2:172.21.15.191:3300/0 conn(0x7f2a7c06ac30 0x7f2a7c15c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:29.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.651+0000 7f2a7affd700 1 -- 172.21.15.99:0/1650472229 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a7c15bd10 con 0x7f2a7c157ba0 2023-10-08T00:11:29.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.651+0000 7f2a7affd700 1 -- 172.21.15.99:0/1650472229 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f2a70001470 con 0x7f2a7c157ba0 2023-10-08T00:11:29.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.651+0000 7f2a7affd700 1 -- 172.21.15.99:0/1650472229 >> v1:172.21.15.99:6789/0 conn(0x7f2a7c157ba0 legacy=0x7f2a7c06a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:29.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.651+0000 7f2a7affd700 1 --2- 172.21.15.99:0/1650472229 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a64003a60 0x7f2a64003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:29.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.651+0000 7f2a7affd700 1 --2- 172.21.15.99:0/1650472229 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a64004810 0x7f2a64004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:29.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.651+0000 7f2a7affd700 1 --2- 172.21.15.99:0/1650472229 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a640051c0 0x7f2a6400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:29.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.651+0000 7f2a7affd700 1 -- 172.21.15.99:0/1650472229 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7c15b790 con 0x7f2a640051c0 2023-10-08T00:11:29.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.651+0000 7f2a7affd700 1 -- 172.21.15.99:0/1650472229 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7c15ba30 con 0x7f2a64004810 2023-10-08T00:11:29.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.651+0000 7f2a7affd700 1 -- 172.21.15.99:0/1650472229 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6400bfa0 con 0x7f2a64003a60 2023-10-08T00:11:29.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.651+0000 7f2a7bfff700 1 --2- 172.21.15.99:0/1650472229 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a7c1577d0 0x7f2a7c0cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:11:29.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.651+0000 7f2a808af700 1 --2- 172.21.15.99:0/1650472229 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a640051c0 0x7f2a6400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:29.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.651+0000 7f2a7b7fe700 1 --2- 172.21.15.99:0/1650472229 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a64004810 0x7f2a64004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:29.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.651+0000 7f2a7bfff700 1 --2- 172.21.15.99:0/1650472229 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a64003a60 0x7f2a64003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:29.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.651+0000 7f2a808af700 1 -- 172.21.15.99:0/1650472229 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a64003a60 msgr2=0x7f2a64003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:29.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.651+0000 7f2a808af700 1 --2- 172.21.15.99:0/1650472229 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a64003a60 0x7f2a64003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:29.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.651+0000 7f2a808af700 1 -- 172.21.15.99:0/1650472229 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a64004810 msgr2=0x7f2a64004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:29.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.651+0000 7f2a808af700 1 --2- 172.21.15.99:0/1650472229 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a64004810 0x7f2a64004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:29.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.651+0000 7f2a808af700 1 -- 172.21.15.99:0/1650472229 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a64003570 con 0x7f2a640051c0 2023-10-08T00:11:29.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.651+0000 7f2a7bfff700 1 --2- 172.21.15.99:0/1650472229 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a64003a60 0x7f2a64003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:11:29.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.652+0000 7f2a808af700 1 --2- 172.21.15.99:0/1650472229 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a640051c0 0x7f2a6400ba60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2a7400e210 tx=0x7f2a7400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f4120d2fca029c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:29.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.652+0000 7f2a7affd700 1 -- 172.21.15.99:0/1650472229 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7400ea60 con 0x7f2a640051c0 2023-10-08T00:11:29.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.652+0000 7f2a7affd700 1 -- 172.21.15.99:0/1650472229 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a7400f040 con 0x7f2a640051c0 2023-10-08T00:11:29.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.652+0000 7f2a7affd700 1 -- 172.21.15.99:0/1650472229 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7400ee10 con 0x7f2a640051c0 2023-10-08T00:11:29.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.652+0000 7f2a82312700 1 -- 172.21.15.99:0/1650472229 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a640051c0 msgr2=0x7f2a6400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:29.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.652+0000 7f2a82312700 1 --2- 172.21.15.99:0/1650472229 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a640051c0 0x7f2a6400ba60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2a7400e210 tx=0x7f2a7400b040 comp rx=0 tx=0).stop 2023-10-08T00:11:29.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.652+0000 7f2a808af700 1 -- 172.21.15.99:0/1650472229 reap_dead start 2023-10-08T00:11:29.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.652+0000 7f2a82312700 1 -- 172.21.15.99:0/1650472229 shutdown_connections 2023-10-08T00:11:29.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.652+0000 7f2a82312700 1 -- 172.21.15.99:0/1650472229 >> 172.21.15.99:0/1650472229 conn(0x7f2a7c068ce0 msgr2=0x7f2a7c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:29.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.652+0000 7f2a82312700 1 -- 172.21.15.99:0/1650472229 shutdown_connections 2023-10-08T00:11:29.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.652+0000 7f2a82312700 1 -- 172.21.15.99:0/1650472229 wait complete. 2023-10-08T00:11:29.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.653+0000 7f2a82312700 1 Processor -- start 2023-10-08T00:11:29.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.654+0000 7f2a82312700 1 -- start start 2023-10-08T00:11:29.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.654+0000 7f2a82312700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a7c1549d0 0x7f2a7c15ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:29.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.654+0000 7f2a82312700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a7c0a7610 0x7f2a7c0a90d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:29.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.654+0000 7f2a82312700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a7c0a9610 0x7f2a7c16a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:29.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.654+0000 7f2a82312700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7c0a9a80 con 0x7f2a7c0a7610 2023-10-08T00:11:29.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.654+0000 7f2a82312700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7c15f230 con 0x7f2a7c1549d0 2023-10-08T00:11:29.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.654+0000 7f2a82312700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7c0a7b10 con 0x7f2a7c0a9610 2023-10-08T00:11:29.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.654+0000 7f2a7b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a7c0a7610 0x7f2a7c0a90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:29.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.654+0000 7f2a7b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a7c0a7610 0x7f2a7c0a90d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52256/0 (socket says 172.21.15.99:52256) 2023-10-08T00:11:29.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.654+0000 7f2a7b7fe700 1 -- 172.21.15.99:0/3938200968 learned_addr learned my addr 172.21.15.99:0/3938200968 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:29.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.654+0000 7f2a7bfff700 1 --2- 172.21.15.99:0/3938200968 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a7c1549d0 0x7f2a7c15ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:29.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.655+0000 7f2a808af700 1 --2- 172.21.15.99:0/3938200968 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a7c0a9610 0x7f2a7c16a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:29.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.655+0000 7f2a7b7fe700 1 -- 172.21.15.99:0/3938200968 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a7c0a9610 msgr2=0x7f2a7c16a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:29.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.655+0000 7f2a7b7fe700 1 --2- 172.21.15.99:0/3938200968 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a7c0a9610 0x7f2a7c16a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:29.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.655+0000 7f2a7b7fe700 1 -- 172.21.15.99:0/3938200968 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a7c1549d0 msgr2=0x7f2a7c15ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:29.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.655+0000 7f2a7b7fe700 1 --2- 172.21.15.99:0/3938200968 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a7c1549d0 0x7f2a7c15ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:29.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.655+0000 7f2a7b7fe700 1 -- 172.21.15.99:0/3938200968 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a7c0a7df0 con 0x7f2a7c0a7610 2023-10-08T00:11:29.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.655+0000 7f2a7b7fe700 1 --2- 172.21.15.99:0/3938200968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a7c0a7610 0x7f2a7c0a90d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2a70007390 tx=0x7f2a70001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:29.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.655+0000 7f2a63fff700 1 -- 172.21.15.99:0/3938200968 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a70018030 con 0x7f2a7c0a7610 2023-10-08T00:11:29.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.655+0000 7f2a63fff700 1 -- 172.21.15.99:0/3938200968 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a7000f8d0 con 0x7f2a7c0a7610 2023-10-08T00:11:29.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.655+0000 7f2a63fff700 1 -- 172.21.15.99:0/3938200968 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7000fc90 con 0x7f2a7c0a7610 2023-10-08T00:11:29.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.655+0000 7f2a82312700 1 -- 172.21.15.99:0/3938200968 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a7c0a80d0 con 0x7f2a7c0a7610 2023-10-08T00:11:29.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.655+0000 7f2a82312700 1 -- 172.21.15.99:0/3938200968 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a7c0a8690 con 0x7f2a7c0a7610 2023-10-08T00:11:29.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.656+0000 7f2a82312700 1 -- 172.21.15.99:0/3938200968 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a7c04c3c0 con 0x7f2a7c0a7610 2023-10-08T00:11:29.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.656+0000 7f2a63fff700 1 -- 172.21.15.99:0/3938200968 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2a7001b070 con 0x7f2a7c0a7610 2023-10-08T00:11:29.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.657+0000 7f2a63fff700 1 --2- 172.21.15.99:0/3938200968 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2a68066f30 0x7f2a680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:29.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.657+0000 7f2a63fff700 1 -- 172.21.15.99:0/3938200968 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2a70019070 con 0x7f2a7c0a7610 2023-10-08T00:11:29.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.657+0000 7f2a7bfff700 1 --2- 172.21.15.99:0/3938200968 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2a68066f30 0x7f2a680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:29.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.657+0000 7f2a7bfff700 1 --2- 172.21.15.99:0/3938200968 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2a68066f30 0x7f2a680693e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c0b6dc0 tx=0x7f2a6c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:29.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.661+0000 7f2a63fff700 1 -- 172.21.15.99:0/3938200968 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2a70051700 con 0x7f2a7c0a7610 2023-10-08T00:11:29.886 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.885+0000 7f2a82312700 1 -- 172.21.15.99:0/3938200968 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f2a7c15bff0 con 0x7f2a7c0a7610 2023-10-08T00:11:29.886 INFO:tasks.ceph.mon.a.smithi099.stderr:2023-10-08T00:11:29.886+0000 7f6c60fa2700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T00:11:29.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.889+0000 7f2a63fff700 1 -- 172.21.15.99:0/3938200968 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f2a70054d20 con 0x7f2a7c0a7610 2023-10-08T00:11:29.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.891+0000 7f2a82312700 1 -- 172.21.15.99:0/3938200968 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2a68066f30 msgr2=0x7f2a680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:29.893 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.891+0000 7f2a82312700 1 --2- 172.21.15.99:0/3938200968 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2a68066f30 0x7f2a680693e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c0b6dc0 tx=0x7f2a6c008040 comp rx=0 tx=0).stop 2023-10-08T00:11:29.893 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.891+0000 7f2a82312700 1 -- 172.21.15.99:0/3938200968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a7c0a7610 msgr2=0x7f2a7c0a90d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:29.893 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.891+0000 7f2a82312700 1 --2- 172.21.15.99:0/3938200968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a7c0a7610 0x7f2a7c0a90d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2a70007390 tx=0x7f2a70001e90 comp rx=0 tx=0).stop 2023-10-08T00:11:29.893 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.891+0000 7f2a82312700 1 -- 172.21.15.99:0/3938200968 shutdown_connections 2023-10-08T00:11:29.893 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.891+0000 7f2a82312700 1 --2- 172.21.15.99:0/3938200968 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2a68066f30 0x7f2a680693e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:29.893 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.891+0000 7f2a82312700 1 --2- 172.21.15.99:0/3938200968 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a7c1549d0 0x7f2a7c15ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:29.893 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.891+0000 7f2a82312700 1 --2- 172.21.15.99:0/3938200968 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a7c0a9610 0x7f2a7c16a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:29.893 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.891+0000 7f2a82312700 1 --2- 172.21.15.99:0/3938200968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a7c0a7610 0x7f2a7c0a90d0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:29.893 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.891+0000 7f2a82312700 1 -- 172.21.15.99:0/3938200968 >> 172.21.15.99:0/3938200968 conn(0x7f2a7c068ce0 msgr2=0x7f2a7c156720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:29.894 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.891+0000 7f2a82312700 1 -- 172.21.15.99:0/3938200968 shutdown_connections 2023-10-08T00:11:29.894 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:29.891+0000 7f2a82312700 1 -- 172.21.15.99:0/3938200968 wait complete. 2023-10-08T00:11:29.894 INFO:tasks.ceph.mds.d.smithi191.stderr: -1631> 2023-10-08T00:10:58.213+0000 7f0f0262b700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T00:11:29.895 INFO:teuthology.orchestra.run.smithi099.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T00:11:29.904 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T00:11:29.915 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2023-10-08T00:11:30.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.018+0000 7fb6fbfff700 1 Processor -- start 2023-10-08T00:11:30.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.018+0000 7fb6fbfff700 1 -- start start 2023-10-08T00:11:30.025 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.019+0000 7fb6fbfff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb6fc0b06b0 0x7fb6fc0b0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:30.025 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.019+0000 7fb6fbfff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb6fc159ec0 0x7fb6fc0b1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:30.025 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.019+0000 7fb6fbfff700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6fc0ce7e0 con 0x7fb6fc0cf5a0 2023-10-08T00:11:30.026 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.019+0000 7fb6fbfff700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6fc0ceac0 con 0x7fb6fc159ec0 2023-10-08T00:11:30.026 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.019+0000 7fb6fbfff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6fc0ceda0 con 0x7fb6fc0b06b0 2023-10-08T00:11:30.026 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.019+0000 7fb6fb7fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb6fc159ec0 0x7fb6fc0b1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:30.026 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.019+0000 7fb6fa7fc700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fb6fc0cf5a0 0x7fb6fc1597b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:44070/0 (socket says 172.21.15.99:44070) 2023-10-08T00:11:30.026 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.019+0000 7fb6fb7fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb6fc159ec0 0x7fb6fc0b1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52262/0 (socket says 172.21.15.99:52262) 2023-10-08T00:11:30.026 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.019+0000 7fb6fa7fc700 1 -- 172.21.15.99:0/1971209350 learned_addr learned my addr 172.21.15.99:0/1971209350 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:30.026 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.019+0000 7fb6faffd700 1 --2- 172.21.15.99:0/1971209350 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb6fc0b06b0 0x7fb6fc0b0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:30.026 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.019+0000 7fb6f9ffb700 1 -- 172.21.15.99:0/1971209350 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2608453819 0 0) 0x7fb6fc0ce7e0 con 0x7fb6fc0cf5a0 2023-10-08T00:11:30.027 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.019+0000 7fb6f9ffb700 1 -- 172.21.15.99:0/1971209350 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6e8003570 con 0x7fb6fc0cf5a0 2023-10-08T00:11:30.027 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.019+0000 7fb6fb7fe700 1 -- 172.21.15.99:0/1971209350 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb6fc0b06b0 msgr2=0x7fb6fc0b0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:30.027 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.019+0000 7fb6fb7fe700 1 --2- 172.21.15.99:0/1971209350 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb6fc0b06b0 0x7fb6fc0b0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:30.027 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.019+0000 7fb6fb7fe700 1 -- 172.21.15.99:0/1971209350 >> v1:172.21.15.99:6789/0 conn(0x7fb6fc0cf5a0 legacy=0x7fb6fc1597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:30.027 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.019+0000 7fb6fb7fe700 1 -- 172.21.15.99:0/1971209350 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6fc0cf080 con 0x7fb6fc159ec0 2023-10-08T00:11:30.027 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.019+0000 7fb6faffd700 1 --2- 172.21.15.99:0/1971209350 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb6fc0b06b0 0x7fb6fc0b0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:11:30.027 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.020+0000 7fb6fb7fe700 1 --2- 172.21.15.99:0/1971209350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6fc159ec0 0x7fb6fc0b1850 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb6f40098e0 tx=0x7fb6f40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3660624d8a2acdfa server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:30.027 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.020+0000 7fb6f9ffb700 1 -- 172.21.15.99:0/1971209350 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f400b640 con 0x7fb6fc159ec0 2023-10-08T00:11:30.028 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.020+0000 7fb6f9ffb700 1 -- 172.21.15.99:0/1971209350 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6f400b7a0 con 0x7fb6fc159ec0 2023-10-08T00:11:30.028 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.020+0000 7fb6f9ffb700 1 -- 172.21.15.99:0/1971209350 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f400bb40 con 0x7fb6fc159ec0 2023-10-08T00:11:30.028 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.020+0000 7fb6fbfff700 1 -- 172.21.15.99:0/1971209350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6fc159ec0 msgr2=0x7fb6fc0b1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:30.028 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.020+0000 7fb6fbfff700 1 --2- 172.21.15.99:0/1971209350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6fc159ec0 0x7fb6fc0b1850 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb6f40098e0 tx=0x7fb6f40091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:30.028 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.020+0000 7fb6fbfff700 1 -- 172.21.15.99:0/1971209350 shutdown_connections 2023-10-08T00:11:30.028 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.020+0000 7fb6fbfff700 1 --2- 172.21.15.99:0/1971209350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6fc159ec0 0x7fb6fc0b1850 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:30.028 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.020+0000 7fb6fbfff700 1 --2- 172.21.15.99:0/1971209350 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb6fc0b06b0 0x7fb6fc0b0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:30.028 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.020+0000 7fb6fbfff700 1 -- 172.21.15.99:0/1971209350 >> 172.21.15.99:0/1971209350 conn(0x7fb6fc06c700 msgr2=0x7fb6fc069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:30.029 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.020+0000 7fb6fbfff700 1 -- 172.21.15.99:0/1971209350 shutdown_connections 2023-10-08T00:11:30.029 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.020+0000 7fb6fbfff700 1 -- 172.21.15.99:0/1971209350 wait complete. 2023-10-08T00:11:30.029 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.021+0000 7fb6fbfff700 1 Processor -- start 2023-10-08T00:11:30.029 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.022+0000 7fb6fbfff700 1 -- start start 2023-10-08T00:11:30.029 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.022+0000 7fb6fbfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6fc0b06b0 0x7fb6fc158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:30.029 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.022+0000 7fb6fbfff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb6fc0cf5a0 0x7fb6fc0ae660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:30.029 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.022+0000 7fb6fbfff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb6fc159ec0 0x7fb6fc150530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:30.029 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.022+0000 7fb6fbfff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6fc16e360 con 0x7fb6fc0b06b0 2023-10-08T00:11:30.030 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.022+0000 7fb6fbfff700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6fc16d750 con 0x7fb6fc0cf5a0 2023-10-08T00:11:30.030 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.022+0000 7fb6fbfff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6fc16da30 con 0x7fb6fc159ec0 2023-10-08T00:11:30.030 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.022+0000 7fb6faffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6fc0b06b0 0x7fb6fc158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:30.030 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.022+0000 7fb6faffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6fc0b06b0 0x7fb6fc158bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52278/0 (socket says 172.21.15.99:52278) 2023-10-08T00:11:30.030 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.022+0000 7fb6faffd700 1 -- 172.21.15.99:0/3766248354 learned_addr learned my addr 172.21.15.99:0/3766248354 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:30.030 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.022+0000 7fb6fb7fe700 1 --2- 172.21.15.99:0/3766248354 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb6fc159ec0 0x7fb6fc150530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:30.030 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.022+0000 7fb6fa7fc700 1 --2- 172.21.15.99:0/3766248354 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb6fc0cf5a0 0x7fb6fc0ae660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:30.030 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.023+0000 7fb6faffd700 1 -- 172.21.15.99:0/3766248354 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb6fc159ec0 msgr2=0x7fb6fc150530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:30.030 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.023+0000 7fb6faffd700 1 --2- 172.21.15.99:0/3766248354 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb6fc159ec0 0x7fb6fc150530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:30.031 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.023+0000 7fb6faffd700 1 -- 172.21.15.99:0/3766248354 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb6fc0cf5a0 msgr2=0x7fb6fc0ae660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:30.031 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.023+0000 7fb6faffd700 1 --2- 172.21.15.99:0/3766248354 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb6fc0cf5a0 0x7fb6fc0ae660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:30.031 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.023+0000 7fb6faffd700 1 -- 172.21.15.99:0/3766248354 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f4009420 con 0x7fb6fc0b06b0 2023-10-08T00:11:30.031 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.023+0000 7fb6fa7fc700 1 --2- 172.21.15.99:0/3766248354 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb6fc0cf5a0 0x7fb6fc0ae660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:11:30.031 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.023+0000 7fb6faffd700 1 --2- 172.21.15.99:0/3766248354 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6fc0b06b0 0x7fb6fc158bf0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec00d1d0 tx=0x7fb6ec009a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:30.031 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.023+0000 7fb6f2ffd700 1 -- 172.21.15.99:0/3766248354 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec00ddc0 con 0x7fb6fc0b06b0 2023-10-08T00:11:30.031 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.023+0000 7fb6f2ffd700 1 -- 172.21.15.99:0/3766248354 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6ec00f040 con 0x7fb6fc0b06b0 2023-10-08T00:11:30.031 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.023+0000 7fb6fbfff700 1 -- 172.21.15.99:0/3766248354 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6fc16dd10 con 0x7fb6fc0b06b0 2023-10-08T00:11:30.031 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.023+0000 7fb6f2ffd700 1 -- 172.21.15.99:0/3766248354 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec015670 con 0x7fb6fc0b06b0 2023-10-08T00:11:30.031 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.023+0000 7fb6fbfff700 1 -- 172.21.15.99:0/3766248354 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6fc159130 con 0x7fb6fc0b06b0 2023-10-08T00:11:30.031 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.024+0000 7fb6fbfff700 1 -- 172.21.15.99:0/3766248354 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6fc04c3c0 con 0x7fb6fc0b06b0 2023-10-08T00:11:30.032 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.024+0000 7fb6f2ffd700 1 -- 172.21.15.99:0/3766248354 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb6ec015a60 con 0x7fb6fc0b06b0 2023-10-08T00:11:30.032 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.025+0000 7fb6f2ffd700 1 --2- 172.21.15.99:0/3766248354 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb6dc066f30 0x7fb6dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:30.032 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.025+0000 7fb6f2ffd700 1 -- 172.21.15.99:0/3766248354 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5791+0+0 (secure 0 0 0) 0x7fb6ec0849b0 con 0x7fb6fc0b06b0 2023-10-08T00:11:30.032 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.025+0000 7fb6fa7fc700 1 --2- 172.21.15.99:0/3766248354 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb6dc066f30 0x7fb6dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:30.032 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.025+0000 7fb6fa7fc700 1 --2- 172.21.15.99:0/3766248354 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb6dc066f30 0x7fb6dc0693e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc158270 tx=0x7fb6e4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:30.032 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.029+0000 7fb6f2ffd700 1 -- 172.21.15.99:0/3766248354 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb6ec04e5a0 con 0x7fb6fc0b06b0 2023-10-08T00:11:30.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.253+0000 7fb6fbfff700 1 -- 172.21.15.99:0/3766248354 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb6fc0b1ec0 con 0x7fb6fc0b06b0 2023-10-08T00:11:30.894 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.894+0000 7fb6f2ffd700 1 -- 172.21.15.99:0/3766248354 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb6ec051c80 con 0x7fb6fc0b06b0 2023-10-08T00:11:30.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.896+0000 7fb6fbfff700 1 -- 172.21.15.99:0/3766248354 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb6dc066f30 msgr2=0x7fb6dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:30.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.896+0000 7fb6fbfff700 1 --2- 172.21.15.99:0/3766248354 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb6dc066f30 0x7fb6dc0693e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc158270 tx=0x7fb6e4009040 comp rx=0 tx=0).stop 2023-10-08T00:11:30.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.896+0000 7fb6fbfff700 1 -- 172.21.15.99:0/3766248354 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6fc0b06b0 msgr2=0x7fb6fc158bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:30.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.896+0000 7fb6fbfff700 1 --2- 172.21.15.99:0/3766248354 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6fc0b06b0 0x7fb6fc158bf0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec00d1d0 tx=0x7fb6ec009a90 comp rx=0 tx=0).stop 2023-10-08T00:11:30.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.896+0000 7fb6fbfff700 1 -- 172.21.15.99:0/3766248354 shutdown_connections 2023-10-08T00:11:30.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.896+0000 7fb6fbfff700 1 --2- 172.21.15.99:0/3766248354 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb6dc066f30 0x7fb6dc0693e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:30.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.896+0000 7fb6fbfff700 1 --2- 172.21.15.99:0/3766248354 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb6fc159ec0 0x7fb6fc150530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:30.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.896+0000 7fb6fbfff700 1 --2- 172.21.15.99:0/3766248354 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6fc0b06b0 0x7fb6fc158bf0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:30.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.896+0000 7fb6fbfff700 1 --2- 172.21.15.99:0/3766248354 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb6fc0cf5a0 0x7fb6fc0ae660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:30.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.896+0000 7fb6fbfff700 1 -- 172.21.15.99:0/3766248354 >> 172.21.15.99:0/3766248354 conn(0x7fb6fc06c700 msgr2=0x7fb6fc1567b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:30.898 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.896+0000 7fb6fbfff700 1 -- 172.21.15.99:0/3766248354 shutdown_connections 2023-10-08T00:11:30.898 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:30.896+0000 7fb6fbfff700 1 -- 172.21.15.99:0/3766248354 wait complete. 2023-10-08T00:11:30.909 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T00:11:31.027 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.026+0000 7fad2c960700 1 Processor -- start 2023-10-08T00:11:31.027 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.026+0000 7fad2c960700 1 -- start start 2023-10-08T00:11:31.029 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.027+0000 7fad2c960700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fad2806f0b0 0x7fad2806bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:31.029 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.027+0000 7fad2c960700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad280687f0 con 0x7fad28156a30 2023-10-08T00:11:31.029 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.027+0000 7fad2c960700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad28067be0 con 0x7fad28156e00 2023-10-08T00:11:31.030 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.027+0000 7fad2c960700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad28067ec0 con 0x7fad2806f0b0 2023-10-08T00:11:31.030 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.027+0000 7fad2659c700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fad28156a30 0x7fad280cc540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:44080/0 (socket says 172.21.15.99:44080) 2023-10-08T00:11:31.030 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.027+0000 7fad2659c700 1 -- 172.21.15.99:0/1664890719 learned_addr learned my addr 172.21.15.99:0/1664890719 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:31.030 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.027+0000 7fad1dd9b700 1 --2- 172.21.15.99:0/1664890719 >> v2:172.21.15.191:3300/0 conn(0x7fad2806f0b0 0x7fad2806bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:31.030 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.027+0000 7fad25d9b700 1 -- 172.21.15.99:0/1664890719 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 233452788 0 0) 0x7fad280687f0 con 0x7fad28156a30 2023-10-08T00:11:31.030 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.027+0000 7fad25d9b700 1 -- 172.21.15.99:0/1664890719 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad14003570 con 0x7fad28156a30 2023-10-08T00:11:31.030 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.028+0000 7fad25d9b700 1 -- 172.21.15.99:0/1664890719 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2176465037 0 0) 0x7fad28067be0 con 0x7fad28156e00 2023-10-08T00:11:31.030 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.028+0000 7fad25d9b700 1 -- 172.21.15.99:0/1664890719 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad280687f0 con 0x7fad28156e00 2023-10-08T00:11:31.030 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.028+0000 7fad1dd9b700 1 -- 172.21.15.99:0/1664890719 >> v1:172.21.15.191:6789/0 conn(0x7fad28156e00 legacy=0x7fad280c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:31.030 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.028+0000 7fad25d9b700 1 -- 172.21.15.99:0/1664890719 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 849229793 0 0) 0x7fad14003570 con 0x7fad28156a30 2023-10-08T00:11:31.030 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.028+0000 7fad1dd9b700 1 -- 172.21.15.99:0/1664890719 >> v1:172.21.15.99:6789/0 conn(0x7fad28156a30 legacy=0x7fad280cc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:31.030 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.028+0000 7fad1dd9b700 1 -- 172.21.15.99:0/1664890719 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad280681a0 con 0x7fad2806f0b0 2023-10-08T00:11:31.031 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.028+0000 7fad1dd9b700 1 --2- 172.21.15.99:0/1664890719 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad2806f0b0 0x7fad2806bc70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fad100098e0 tx=0x7fad100091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2470d61a462dda31 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:31.031 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.028+0000 7fad25d9b700 1 -- 172.21.15.99:0/1664890719 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad18001470 con 0x7fad2806f0b0 2023-10-08T00:11:31.031 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.028+0000 7fad25d9b700 1 -- 172.21.15.99:0/1664890719 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad10011420 con 0x7fad2806f0b0 2023-10-08T00:11:31.031 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.028+0000 7fad25d9b700 1 -- 172.21.15.99:0/1664890719 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad100117a0 con 0x7fad2806f0b0 2023-10-08T00:11:31.031 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.028+0000 7fad2c960700 1 -- 172.21.15.99:0/1664890719 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad2806f0b0 msgr2=0x7fad2806bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:31.031 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.028+0000 7fad2c960700 1 --2- 172.21.15.99:0/1664890719 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad2806f0b0 0x7fad2806bc70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fad100098e0 tx=0x7fad100091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:31.031 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.029+0000 7fad2c960700 1 -- 172.21.15.99:0/1664890719 shutdown_connections 2023-10-08T00:11:31.031 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.029+0000 7fad2c960700 1 --2- 172.21.15.99:0/1664890719 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad2806f0b0 0x7fad2806bc70 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:31.031 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.029+0000 7fad2c960700 1 -- 172.21.15.99:0/1664890719 >> 172.21.15.99:0/1664890719 conn(0x7fad280cdae0 msgr2=0x7fad280cdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:31.031 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.029+0000 7fad2c960700 1 -- 172.21.15.99:0/1664890719 shutdown_connections 2023-10-08T00:11:31.031 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.029+0000 7fad2c960700 1 -- 172.21.15.99:0/1664890719 wait complete. 2023-10-08T00:11:31.034 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.030+0000 7fad2c960700 1 Processor -- start 2023-10-08T00:11:31.034 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.030+0000 7fad2c960700 1 -- start start 2023-10-08T00:11:31.034 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.031+0000 7fad2c960700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad2806f0b0 0x7fad280b3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:31.034 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.031+0000 7fad2c960700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad28156a30 0x7fad28148570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:31.034 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.031+0000 7fad2c960700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad28156e00 0x7fad28141c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:31.034 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.031+0000 7fad2c960700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad280b2690 con 0x7fad2806f0b0 2023-10-08T00:11:31.034 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.031+0000 7fad2c960700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad280c5560 con 0x7fad28156a30 2023-10-08T00:11:31.034 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.031+0000 7fad2c960700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad280b17c0 con 0x7fad28156e00 2023-10-08T00:11:31.034 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.031+0000 7fad2659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad2806f0b0 0x7fad280b3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:31.034 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.031+0000 7fad2659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad2806f0b0 0x7fad280b3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52290/0 (socket says 172.21.15.99:52290) 2023-10-08T00:11:31.035 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.031+0000 7fad2659c700 1 -- 172.21.15.99:0/3518232952 learned_addr learned my addr 172.21.15.99:0/3518232952 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:31.035 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.031+0000 7fad2659c700 1 -- 172.21.15.99:0/3518232952 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad28156e00 msgr2=0x7fad28141c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:31.035 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.031+0000 7fad2659c700 1 --2- 172.21.15.99:0/3518232952 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad28156e00 0x7fad28141c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:31.035 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.031+0000 7fad2659c700 1 -- 172.21.15.99:0/3518232952 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad28156a30 msgr2=0x7fad28148570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:31.035 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.031+0000 7fad2659c700 1 --2- 172.21.15.99:0/3518232952 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad28156a30 0x7fad28148570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:31.035 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.031+0000 7fad2659c700 1 -- 172.21.15.99:0/3518232952 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad10009420 con 0x7fad2806f0b0 2023-10-08T00:11:31.035 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.031+0000 7fad2659c700 1 --2- 172.21.15.99:0/3518232952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad2806f0b0 0x7fad280b3280 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fad1800cb60 tx=0x7fad18001ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:31.035 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.031+0000 7fad1effd700 1 -- 172.21.15.99:0/3518232952 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad18001d60 con 0x7fad2806f0b0 2023-10-08T00:11:31.035 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.031+0000 7fad1effd700 1 -- 172.21.15.99:0/3518232952 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad18007360 con 0x7fad2806f0b0 2023-10-08T00:11:31.035 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.031+0000 7fad2c960700 1 -- 172.21.15.99:0/3518232952 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad280c4c30 con 0x7fad2806f0b0 2023-10-08T00:11:31.035 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.031+0000 7fad1effd700 1 -- 172.21.15.99:0/3518232952 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad18007700 con 0x7fad2806f0b0 2023-10-08T00:11:31.035 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.031+0000 7fad2c960700 1 -- 172.21.15.99:0/3518232952 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad280c51f0 con 0x7fad2806f0b0 2023-10-08T00:11:31.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.032+0000 7fad2c960700 1 -- 172.21.15.99:0/3518232952 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad28071630 con 0x7fad2806f0b0 2023-10-08T00:11:31.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.033+0000 7fad1effd700 1 -- 172.21.15.99:0/3518232952 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fad18004a40 con 0x7fad2806f0b0 2023-10-08T00:11:31.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.033+0000 7fad1effd700 1 --2- 172.21.15.99:0/3518232952 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fad08066f30 0x7fad080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:31.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.033+0000 7fad1effd700 1 -- 172.21.15.99:0/3518232952 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5791+0+0 (secure 0 0 0) 0x7fad18083d00 con 0x7fad2806f0b0 2023-10-08T00:11:31.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.033+0000 7fad1dd9b700 1 --2- 172.21.15.99:0/3518232952 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fad08066f30 0x7fad080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:31.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.034+0000 7fad1dd9b700 1 --2- 172.21.15.99:0/3518232952 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fad08066f30 0x7fad080693e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fad28143630 tx=0x7fad10016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:31.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.037+0000 7fad1effd700 1 -- 172.21.15.99:0/3518232952 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fad18054a30 con 0x7fad2806f0b0 2023-10-08T00:11:31.220 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:31.220+0000 7fad2c960700 1 -- 172.21.15.99:0/3518232952 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fad2804c3c0 con 0x7fad2806f0b0 2023-10-08T00:11:32.059 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.058+0000 7fad1effd700 1 -- 172.21.15.99:0/3518232952 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v22) v1 ==== 181+0+0 (secure 0 0 0) 0x7fad180180d0 con 0x7fad2806f0b0 2023-10-08T00:11:32.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.061+0000 7fad2c960700 1 -- 172.21.15.99:0/3518232952 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fad08066f30 msgr2=0x7fad080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:32.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.061+0000 7fad2c960700 1 --2- 172.21.15.99:0/3518232952 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fad08066f30 0x7fad080693e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fad28143630 tx=0x7fad10016000 comp rx=0 tx=0).stop 2023-10-08T00:11:32.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.061+0000 7fad2c960700 1 -- 172.21.15.99:0/3518232952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad2806f0b0 msgr2=0x7fad280b3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:32.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.061+0000 7fad2c960700 1 --2- 172.21.15.99:0/3518232952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad2806f0b0 0x7fad280b3280 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fad1800cb60 tx=0x7fad18001ad0 comp rx=0 tx=0).stop 2023-10-08T00:11:32.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.061+0000 7fad2c960700 1 -- 172.21.15.99:0/3518232952 shutdown_connections 2023-10-08T00:11:32.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.061+0000 7fad2c960700 1 --2- 172.21.15.99:0/3518232952 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fad08066f30 0x7fad080693e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:32.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.061+0000 7fad2c960700 1 --2- 172.21.15.99:0/3518232952 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad28156e00 0x7fad28141c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:32.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.061+0000 7fad2c960700 1 --2- 172.21.15.99:0/3518232952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad2806f0b0 0x7fad280b3280 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:32.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.061+0000 7fad2c960700 1 --2- 172.21.15.99:0/3518232952 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad28156a30 0x7fad28148570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:32.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.061+0000 7fad2c960700 1 -- 172.21.15.99:0/3518232952 >> 172.21.15.99:0/3518232952 conn(0x7fad280cdae0 msgr2=0x7fad2814c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:32.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.061+0000 7fad2c960700 1 -- 172.21.15.99:0/3518232952 shutdown_connections 2023-10-08T00:11:32.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.061+0000 7fad2c960700 1 -- 172.21.15.99:0/3518232952 wait complete. 2023-10-08T00:11:32.063 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' removed 2023-10-08T00:11:32.078 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T00:11:32.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.199+0000 7f8b25aa2700 1 Processor -- start 2023-10-08T00:11:32.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.199+0000 7f8b25aa2700 1 -- start start 2023-10-08T00:11:32.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.200+0000 7f8b25aa2700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8b200b6a10 0x7f8b2015a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:32.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.200+0000 7f8b25aa2700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8b200b46f0 0x7f8b200b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:32.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.200+0000 7f8b25aa2700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b200b17b0 con 0x7f8b20159ef0 2023-10-08T00:11:32.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.200+0000 7f8b25aa2700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b200b5020 con 0x7f8b200b46f0 2023-10-08T00:11:32.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.200+0000 7f8b25aa2700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b200b1a60 con 0x7f8b200b6a10 2023-10-08T00:11:32.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.200+0000 7f8b1ffff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8b200b46f0 0x7f8b200b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:32.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.200+0000 7f8b1ffff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8b200b46f0 0x7f8b200b4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52296/0 (socket says 172.21.15.99:52296) 2023-10-08T00:11:32.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.200+0000 7f8b1ffff700 1 -- 172.21.15.99:0/3402800899 learned_addr learned my addr 172.21.15.99:0/3402800899 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:32.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.201+0000 7f8b1ffff700 1 -- 172.21.15.99:0/3402800899 >> v2:172.21.15.191:3300/0 conn(0x7f8b200b6a10 msgr2=0x7f8b2015a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:32.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.201+0000 7f8b1ffff700 1 --2- 172.21.15.99:0/3402800899 >> v2:172.21.15.191:3300/0 conn(0x7f8b200b6a10 0x7f8b2015a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:32.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.201+0000 7f8b1ffff700 1 -- 172.21.15.99:0/3402800899 >> v1:172.21.15.191:6789/0 conn(0x7f8b20159ef0 legacy=0x7f8b200b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:32.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.201+0000 7f8b1ffff700 1 -- 172.21.15.99:0/3402800899 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b200b1d40 con 0x7f8b200b46f0 2023-10-08T00:11:32.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.201+0000 7f8b1ffff700 1 --2- 172.21.15.99:0/3402800899 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b200b46f0 0x7f8b200b4ae0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8b20089550 tx=0x7f8b18009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e70a45690a87770 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:32.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.201+0000 7f8b1e7fc700 1 -- 172.21.15.99:0/3402800899 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b1800b820 con 0x7f8b200b46f0 2023-10-08T00:11:32.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.201+0000 7f8b1e7fc700 1 -- 172.21.15.99:0/3402800899 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b1800b980 con 0x7f8b200b46f0 2023-10-08T00:11:32.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.201+0000 7f8b1e7fc700 1 -- 172.21.15.99:0/3402800899 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b1800bd20 con 0x7f8b200b46f0 2023-10-08T00:11:32.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.201+0000 7f8b25aa2700 1 -- 172.21.15.99:0/3402800899 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b200b46f0 msgr2=0x7f8b200b4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:32.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.201+0000 7f8b25aa2700 1 --2- 172.21.15.99:0/3402800899 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b200b46f0 0x7f8b200b4ae0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8b20089550 tx=0x7f8b18009250 comp rx=0 tx=0).stop 2023-10-08T00:11:32.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.201+0000 7f8b25aa2700 1 -- 172.21.15.99:0/3402800899 shutdown_connections 2023-10-08T00:11:32.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.201+0000 7f8b25aa2700 1 --2- 172.21.15.99:0/3402800899 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b200b46f0 0x7f8b200b4ae0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:32.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.201+0000 7f8b25aa2700 1 --2- 172.21.15.99:0/3402800899 >> v2:172.21.15.191:3300/0 conn(0x7f8b200b6a10 0x7f8b2015a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:32.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.201+0000 7f8b25aa2700 1 -- 172.21.15.99:0/3402800899 >> 172.21.15.99:0/3402800899 conn(0x7f8b2009a0a0 msgr2=0x7f8b20067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:32.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.201+0000 7f8b25aa2700 1 -- 172.21.15.99:0/3402800899 shutdown_connections 2023-10-08T00:11:32.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.201+0000 7f8b25aa2700 1 -- 172.21.15.99:0/3402800899 wait complete. 2023-10-08T00:11:32.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.203+0000 7f8b25aa2700 1 Processor -- start 2023-10-08T00:11:32.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.203+0000 7f8b25aa2700 1 -- start start 2023-10-08T00:11:32.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.203+0000 7f8b25aa2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b200b46f0 0x7f8b201559d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:32.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.203+0000 7f8b25aa2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b200b6a10 0x7f8b2009caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:32.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.203+0000 7f8b25aa2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b20159ef0 0x7f8b200ac0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:32.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.203+0000 7f8b25aa2700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2009ab10 con 0x7f8b200b6a10 2023-10-08T00:11:32.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.203+0000 7f8b25aa2700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2009b930 con 0x7f8b200b46f0 2023-10-08T00:11:32.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.203+0000 7f8b25aa2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2009ad20 con 0x7f8b20159ef0 2023-10-08T00:11:32.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.203+0000 7f8b1effd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b200b6a10 0x7f8b2009caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:32.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.203+0000 7f8b1effd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b200b6a10 0x7f8b2009caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52304/0 (socket says 172.21.15.99:52304) 2023-10-08T00:11:32.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.204+0000 7f8b1effd700 1 -- 172.21.15.99:0/1209970133 learned_addr learned my addr 172.21.15.99:0/1209970133 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:32.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.204+0000 7f8b1f7fe700 1 --2- 172.21.15.99:0/1209970133 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b200b46f0 0x7f8b201559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:32.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.204+0000 7f8b1ffff700 1 --2- 172.21.15.99:0/1209970133 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b20159ef0 0x7f8b200ac0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:32.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.204+0000 7f8b1effd700 1 -- 172.21.15.99:0/1209970133 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b20159ef0 msgr2=0x7f8b200ac0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:32.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.204+0000 7f8b1effd700 1 --2- 172.21.15.99:0/1209970133 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b20159ef0 0x7f8b200ac0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:32.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.204+0000 7f8b1effd700 1 -- 172.21.15.99:0/1209970133 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b200b46f0 msgr2=0x7f8b201559d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:32.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.204+0000 7f8b1effd700 1 --2- 172.21.15.99:0/1209970133 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b200b46f0 0x7f8b201559d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:32.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.204+0000 7f8b1effd700 1 -- 172.21.15.99:0/1209970133 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b180094f0 con 0x7f8b200b6a10 2023-10-08T00:11:32.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.204+0000 7f8b1ffff700 1 --2- 172.21.15.99:0/1209970133 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b20159ef0 0x7f8b200ac0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:11:32.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.204+0000 7f8b1effd700 1 --2- 172.21.15.99:0/1209970133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b200b6a10 0x7f8b2009caf0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8b1400b810 tx=0x7f8b14009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:32.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.204+0000 7f8b077fe700 1 -- 172.21.15.99:0/1209970133 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b1400f510 con 0x7f8b200b6a10 2023-10-08T00:11:32.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.204+0000 7f8b077fe700 1 -- 172.21.15.99:0/1209970133 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b1400f690 con 0x7f8b200b6a10 2023-10-08T00:11:32.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.204+0000 7f8b25aa2700 1 -- 172.21.15.99:0/1209970133 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b2009b000 con 0x7f8b200b6a10 2023-10-08T00:11:32.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.204+0000 7f8b077fe700 1 -- 172.21.15.99:0/1209970133 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b1400fa50 con 0x7f8b200b6a10 2023-10-08T00:11:32.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.204+0000 7f8b25aa2700 1 -- 172.21.15.99:0/1209970133 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b2009b5c0 con 0x7f8b200b6a10 2023-10-08T00:11:32.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.205+0000 7f8b25aa2700 1 -- 172.21.15.99:0/1209970133 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b2009bac0 con 0x7f8b200b6a10 2023-10-08T00:11:32.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.205+0000 7f8b077fe700 1 -- 172.21.15.99:0/1209970133 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8b1401a400 con 0x7f8b200b6a10 2023-10-08T00:11:32.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.206+0000 7f8b077fe700 1 --2- 172.21.15.99:0/1209970133 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8b0c066f30 0x7f8b0c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:32.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.206+0000 7f8b077fe700 1 -- 172.21.15.99:0/1209970133 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5353+0+0 (secure 0 0 0) 0x7f8b14083f00 con 0x7f8b200b6a10 2023-10-08T00:11:32.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.206+0000 7f8b1f7fe700 1 --2- 172.21.15.99:0/1209970133 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8b0c066f30 0x7f8b0c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:32.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.207+0000 7f8b1f7fe700 1 --2- 172.21.15.99:0/1209970133 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8b0c066f30 0x7f8b0c0693e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8b100075c0 tx=0x7f8b10009780 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:32.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.210+0000 7f8b077fe700 1 -- 172.21.15.99:0/1209970133 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8b14051350 con 0x7f8b200b6a10 2023-10-08T00:11:32.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:32.395+0000 7f8b25aa2700 1 -- 172.21.15.99:0/1209970133 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8b2004c3c0 con 0x7f8b200b6a10 2023-10-08T00:11:33.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.261+0000 7f8b077fe700 1 -- 172.21.15.99:0/1209970133 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v23) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8b14054970 con 0x7f8b200b6a10 2023-10-08T00:11:33.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.264+0000 7f8b25aa2700 1 -- 172.21.15.99:0/1209970133 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8b0c066f30 msgr2=0x7f8b0c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:33.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.264+0000 7f8b25aa2700 1 --2- 172.21.15.99:0/1209970133 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8b0c066f30 0x7f8b0c0693e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8b100075c0 tx=0x7f8b10009780 comp rx=0 tx=0).stop 2023-10-08T00:11:33.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.264+0000 7f8b25aa2700 1 -- 172.21.15.99:0/1209970133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b200b6a10 msgr2=0x7f8b2009caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:33.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.264+0000 7f8b25aa2700 1 --2- 172.21.15.99:0/1209970133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b200b6a10 0x7f8b2009caf0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8b1400b810 tx=0x7f8b14009220 comp rx=0 tx=0).stop 2023-10-08T00:11:33.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.264+0000 7f8b25aa2700 1 -- 172.21.15.99:0/1209970133 shutdown_connections 2023-10-08T00:11:33.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.264+0000 7f8b25aa2700 1 --2- 172.21.15.99:0/1209970133 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8b0c066f30 0x7f8b0c0693e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:33.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.264+0000 7f8b25aa2700 1 --2- 172.21.15.99:0/1209970133 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b200b46f0 0x7f8b201559d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:33.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.264+0000 7f8b25aa2700 1 --2- 172.21.15.99:0/1209970133 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b20159ef0 0x7f8b200ac0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:33.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.264+0000 7f8b25aa2700 1 --2- 172.21.15.99:0/1209970133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b200b6a10 0x7f8b2009caf0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:33.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.264+0000 7f8b25aa2700 1 -- 172.21.15.99:0/1209970133 >> 172.21.15.99:0/1209970133 conn(0x7f8b2009a0a0 msgr2=0x7f8b2015bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:33.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.264+0000 7f8b25aa2700 1 -- 172.21.15.99:0/1209970133 shutdown_connections 2023-10-08T00:11:33.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.264+0000 7f8b25aa2700 1 -- 172.21.15.99:0/1209970133 wait complete. 2023-10-08T00:11:33.266 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' removed 2023-10-08T00:11:33.279 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T00:11:33.279 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-08T00:11:33.279 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T00:11:33.279 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:11:33.279 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T00:11:33.279 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-08T00:11:33.280 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T00:11:33.280 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:11:33.280 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T00:11:33.280 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-08T00:11:33.280 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T00:11:33.281 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:11:33.281 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T00:11:33.281 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-08T00:11:33.281 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T00:11:33.281 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:11:33.290 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T00:11:33.291 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T00:11:33.291 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T00:11:33.292 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T00:11:33.293 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T00:11:33.293 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T00:11:33.293 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T00:11:33.294 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T00:11:33.333 INFO:tasks.ceph.mds.a:Started 2023-10-08T00:11:33.333 DEBUG:teuthology.parallel:result is None 2023-10-08T00:11:33.334 INFO:tasks.ceph.mds.c:Started 2023-10-08T00:11:33.334 DEBUG:teuthology.parallel:result is None 2023-10-08T00:11:33.335 INFO:tasks.ceph.mds.b:Started 2023-10-08T00:11:33.335 DEBUG:teuthology.parallel:result is None 2023-10-08T00:11:33.336 INFO:tasks.ceph.mds.d:Started 2023-10-08T00:11:33.336 DEBUG:teuthology.parallel:result is None 2023-10-08T00:11:33.336 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-08T00:11:33.410 INFO:tasks.ceph.mds.c.smithi099.stdout:starting mds.c at 2023-10-08T00:11:33.410 INFO:tasks.ceph.mds.a.smithi099.stdout:starting mds.a at 2023-10-08T00:11:33.417 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2023-10-08T00:11:33.418 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2023-10-08T00:11:33.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.459+0000 7fd8fcb5d700 1 Processor -- start 2023-10-08T00:11:33.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.459+0000 7fd8fcb5d700 1 -- start start 2023-10-08T00:11:33.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.460+0000 7fd8fcb5d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8f8143630 0x7fd8f80cfee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:33.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.460+0000 7fd8fcb5d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd8f8143a00 0x7fd8f80d0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:33.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.460+0000 7fd8fcb5d700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8f80b3b70 con 0x7fd8f806a4e0 2023-10-08T00:11:33.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.460+0000 7fd8fcb5d700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8f80b3380 con 0x7fd8f8143a00 2023-10-08T00:11:33.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.460+0000 7fd8fcb5d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f80b3e50 con 0x7fd8f8143630 2023-10-08T00:11:33.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.460+0000 7fd8f659c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8f8143630 0x7fd8f80cfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:33.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.460+0000 7fd8edd9b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd8f8143a00 0x7fd8f80d0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:33.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.460+0000 7fd8f659c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8f8143630 0x7fd8f80cfee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.99:35144/0 (socket says 172.21.15.99:35144) 2023-10-08T00:11:33.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.460+0000 7fd8f659c700 1 -- 172.21.15.99:0/975276547 learned_addr learned my addr 172.21.15.99:0/975276547 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:33.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.460+0000 7fd8f6d9d700 1 --1- 172.21.15.99:0/975276547 >> v1:172.21.15.191:6789/0 conn(0x7fd8f806a4e0 0x7fd8f815d9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:36958/0 (socket says 172.21.15.99:36958) 2023-10-08T00:11:33.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.460+0000 7fd8f659c700 1 -- 172.21.15.99:0/975276547 >> v2:172.21.15.191:3300/0 conn(0x7fd8f8143a00 msgr2=0x7fd8f80d0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:33.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.460+0000 7fd8f659c700 1 --2- 172.21.15.99:0/975276547 >> v2:172.21.15.191:3300/0 conn(0x7fd8f8143a00 0x7fd8f80d0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:33.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.460+0000 7fd8f659c700 1 -- 172.21.15.99:0/975276547 >> v1:172.21.15.191:6789/0 conn(0x7fd8f806a4e0 legacy=0x7fd8f815d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:33.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.460+0000 7fd8f659c700 1 -- 172.21.15.99:0/975276547 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f80b4130 con 0x7fd8f8143630 2023-10-08T00:11:33.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.461+0000 7fd8f659c700 1 --2- 172.21.15.99:0/975276547 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8f8143630 0x7fd8f80cfee0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd8e80099b0 tx=0x7fd8e8009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a6da2ab1defd95f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:33.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.461+0000 7fd8f5d9b700 1 -- 172.21.15.99:0/975276547 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e8009ea0 con 0x7fd8f8143630 2023-10-08T00:11:33.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.461+0000 7fd8f5d9b700 1 -- 172.21.15.99:0/975276547 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e800b860 con 0x7fd8f8143630 2023-10-08T00:11:33.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.461+0000 7fd8f5d9b700 1 -- 172.21.15.99:0/975276547 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e800bc00 con 0x7fd8f8143630 2023-10-08T00:11:33.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.461+0000 7fd8fcb5d700 1 -- 172.21.15.99:0/975276547 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8f8143630 msgr2=0x7fd8f80cfee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:33.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.461+0000 7fd8fcb5d700 1 --2- 172.21.15.99:0/975276547 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8f8143630 0x7fd8f80cfee0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd8e80099b0 tx=0x7fd8e8009250 comp rx=0 tx=0).stop 2023-10-08T00:11:33.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.461+0000 7fd8fcb5d700 1 -- 172.21.15.99:0/975276547 shutdown_connections 2023-10-08T00:11:33.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.461+0000 7fd8fcb5d700 1 --2- 172.21.15.99:0/975276547 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8f8143630 0x7fd8f80cfee0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:33.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.461+0000 7fd8fcb5d700 1 --2- 172.21.15.99:0/975276547 >> v2:172.21.15.191:3300/0 conn(0x7fd8f8143a00 0x7fd8f80d0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:33.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.461+0000 7fd8fcb5d700 1 -- 172.21.15.99:0/975276547 >> 172.21.15.99:0/975276547 conn(0x7fd8f8069760 msgr2=0x7fd8f80b26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:33.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.461+0000 7fd8fcb5d700 1 -- 172.21.15.99:0/975276547 shutdown_connections 2023-10-08T00:11:33.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.461+0000 7fd8fcb5d700 1 -- 172.21.15.99:0/975276547 wait complete. 2023-10-08T00:11:33.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.463+0000 7fd8fcb5d700 1 Processor -- start 2023-10-08T00:11:33.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.463+0000 7fd8fcb5d700 1 -- start start 2023-10-08T00:11:33.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.463+0000 7fd8fcb5d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8f806a4e0 0x7fd8f80c86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:33.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.463+0000 7fd8fcb5d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8f8143630 0x7fd8f81427d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:33.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.463+0000 7fd8fcb5d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8f8143a00 0x7fd8f815a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:33.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.463+0000 7fd8fcb5d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f80c8cb0 con 0x7fd8f8143630 2023-10-08T00:11:33.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.463+0000 7fd8fcb5d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f80cbfe0 con 0x7fd8f806a4e0 2023-10-08T00:11:33.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.463+0000 7fd8fcb5d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f80cc2c0 con 0x7fd8f8143a00 2023-10-08T00:11:33.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.463+0000 7fd8edd9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8f8143630 0x7fd8f81427d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:33.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.463+0000 7fd8edd9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8f8143630 0x7fd8f81427d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39716/0 (socket says 172.21.15.99:39716) 2023-10-08T00:11:33.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.463+0000 7fd8edd9b700 1 -- 172.21.15.99:0/4276847980 learned_addr learned my addr 172.21.15.99:0/4276847980 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:33.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.463+0000 7fd8f659c700 1 --2- 172.21.15.99:0/4276847980 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8f806a4e0 0x7fd8f80c86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:33.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.463+0000 7fd8f6d9d700 1 --2- 172.21.15.99:0/4276847980 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8f8143a00 0x7fd8f815a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:33.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.464+0000 7fd8edd9b700 1 -- 172.21.15.99:0/4276847980 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8f8143a00 msgr2=0x7fd8f815a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:33.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.464+0000 7fd8edd9b700 1 --2- 172.21.15.99:0/4276847980 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8f8143a00 0x7fd8f815a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:33.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.464+0000 7fd8edd9b700 1 -- 172.21.15.99:0/4276847980 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8f806a4e0 msgr2=0x7fd8f80c86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:33.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.464+0000 7fd8edd9b700 1 --2- 172.21.15.99:0/4276847980 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8f806a4e0 0x7fd8f80c86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:33.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.464+0000 7fd8edd9b700 1 -- 172.21.15.99:0/4276847980 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e80094f0 con 0x7fd8f8143630 2023-10-08T00:11:33.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.464+0000 7fd8edd9b700 1 --2- 172.21.15.99:0/4276847980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8f8143630 0x7fd8f81427d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd8e000e900 tx=0x7fd8e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:33.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.464+0000 7fd8eeffd700 1 -- 172.21.15.99:0/4276847980 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e0013680 con 0x7fd8f8143630 2023-10-08T00:11:33.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.464+0000 7fd8eeffd700 1 -- 172.21.15.99:0/4276847980 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e00137e0 con 0x7fd8f8143630 2023-10-08T00:11:33.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.464+0000 7fd8fcb5d700 1 -- 172.21.15.99:0/4276847980 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8f80cc5a0 con 0x7fd8f8143630 2023-10-08T00:11:33.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.464+0000 7fd8eeffd700 1 -- 172.21.15.99:0/4276847980 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e0013b80 con 0x7fd8f8143630 2023-10-08T00:11:33.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.464+0000 7fd8fcb5d700 1 -- 172.21.15.99:0/4276847980 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8f80cbcd0 con 0x7fd8f8143630 2023-10-08T00:11:33.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.465+0000 7fd8fcb5d700 1 -- 172.21.15.99:0/4276847980 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8f8069b60 con 0x7fd8f8143630 2023-10-08T00:11:33.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.465+0000 7fd8eeffd700 1 -- 172.21.15.99:0/4276847980 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd8e00123d0 con 0x7fd8f8143630 2023-10-08T00:11:33.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.466+0000 7fd8eeffd700 1 --2- 172.21.15.99:0/4276847980 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd8d8066f30 0x7fd8d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:33.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.466+0000 7fd8eeffd700 1 -- 172.21.15.99:0/4276847980 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fd8e0084300 con 0x7fd8f8143630 2023-10-08T00:11:33.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.466+0000 7fd8f659c700 1 --2- 172.21.15.99:0/4276847980 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd8d8066f30 0x7fd8d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:33.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.466+0000 7fd8f659c700 1 --2- 172.21.15.99:0/4276847980 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd8d8066f30 0x7fd8d80693e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd8e80099b0 tx=0x7fd8e8017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:33.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.470+0000 7fd8eeffd700 1 -- 172.21.15.99:0/4276847980 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd8e00517d0 con 0x7fd8f8143630 2023-10-08T00:11:33.666 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:33.666+0000 7fd8fcb5d700 1 -- 172.21.15.99:0/4276847980 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fd8f80cc730 con 0x7fd8f8143630 2023-10-08T00:11:34.275 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.275+0000 7fd8eeffd700 1 -- 172.21.15.99:0/4276847980 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v25) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd8e0054df0 con 0x7fd8f8143630 2023-10-08T00:11:34.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.277+0000 7fd8fcb5d700 1 -- 172.21.15.99:0/4276847980 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd8d8066f30 msgr2=0x7fd8d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:34.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.277+0000 7fd8fcb5d700 1 --2- 172.21.15.99:0/4276847980 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd8d8066f30 0x7fd8d80693e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd8e80099b0 tx=0x7fd8e8017000 comp rx=0 tx=0).stop 2023-10-08T00:11:34.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.277+0000 7fd8fcb5d700 1 -- 172.21.15.99:0/4276847980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8f8143630 msgr2=0x7fd8f81427d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:34.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.277+0000 7fd8fcb5d700 1 --2- 172.21.15.99:0/4276847980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8f8143630 0x7fd8f81427d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd8e000e900 tx=0x7fd8e00091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:34.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.277+0000 7fd8fcb5d700 1 -- 172.21.15.99:0/4276847980 shutdown_connections 2023-10-08T00:11:34.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.277+0000 7fd8fcb5d700 1 --2- 172.21.15.99:0/4276847980 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd8d8066f30 0x7fd8d80693e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:34.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.277+0000 7fd8fcb5d700 1 --2- 172.21.15.99:0/4276847980 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8f806a4e0 0x7fd8f80c86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:34.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.277+0000 7fd8fcb5d700 1 --2- 172.21.15.99:0/4276847980 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8f8143a00 0x7fd8f815a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:34.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.277+0000 7fd8fcb5d700 1 --2- 172.21.15.99:0/4276847980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8f8143630 0x7fd8f81427d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:34.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.277+0000 7fd8fcb5d700 1 -- 172.21.15.99:0/4276847980 >> 172.21.15.99:0/4276847980 conn(0x7fd8f8069760 msgr2=0x7fd8f80afac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:34.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.277+0000 7fd8fcb5d700 1 -- 172.21.15.99:0/4276847980 shutdown_connections 2023-10-08T00:11:34.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.277+0000 7fd8fcb5d700 1 -- 172.21.15.99:0/4276847980 wait complete. 2023-10-08T00:11:34.279 INFO:teuthology.orchestra.run.smithi099.stderr: removed all blocklist entries 2023-10-08T00:11:34.293 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-08T00:11:34.407 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.406+0000 7f5167fff700 1 Processor -- start 2023-10-08T00:11:34.407 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.406+0000 7f5167fff700 1 -- start start 2023-10-08T00:11:34.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.407+0000 7f5167fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f51680cdab0 0x7f516815d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:34.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.407+0000 7f5167fff700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51680cb7c0 con 0x7f51680cd6e0 2023-10-08T00:11:34.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.407+0000 7f5167fff700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f516815e290 con 0x7f516815dda0 2023-10-08T00:11:34.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.407+0000 7f5167fff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f516815e850 con 0x7f51680cdab0 2023-10-08T00:11:34.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.407+0000 7f5166ffd700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f51680cd6e0 0x7f51680cbc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:48500/0 (socket says 172.21.15.99:48500) 2023-10-08T00:11:34.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.407+0000 7f5166ffd700 1 -- 172.21.15.99:0/3505013937 learned_addr learned my addr 172.21.15.99:0/3505013937 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:34.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.407+0000 7f515e7fc700 1 --2- 172.21.15.99:0/3505013937 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f51680cdab0 0x7f516815d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:34.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.407+0000 7f51667fc700 1 -- 172.21.15.99:0/3505013937 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2022658094 0 0) 0x7f51680cb7c0 con 0x7f51680cd6e0 2023-10-08T00:11:34.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.407+0000 7f51667fc700 1 -- 172.21.15.99:0/3505013937 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5154003570 con 0x7f51680cd6e0 2023-10-08T00:11:34.412 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.407+0000 7f51667fc700 1 -- 172.21.15.99:0/3505013937 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2550986507 0 0) 0x7f516815e290 con 0x7f516815dda0 2023-10-08T00:11:34.412 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.407+0000 7f51667fc700 1 -- 172.21.15.99:0/3505013937 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51680cb7c0 con 0x7f516815dda0 2023-10-08T00:11:34.412 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.407+0000 7f515e7fc700 1 -- 172.21.15.99:0/3505013937 >> v1:172.21.15.191:6789/0 conn(0x7f516815dda0 legacy=0x7f516815cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:34.412 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.407+0000 7f515e7fc700 1 -- 172.21.15.99:0/3505013937 >> v1:172.21.15.99:6789/0 conn(0x7f51680cd6e0 legacy=0x7f51680cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:34.412 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.407+0000 7f515e7fc700 1 -- 172.21.15.99:0/3505013937 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f516815e570 con 0x7f51680cdab0 2023-10-08T00:11:34.412 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.408+0000 7f515e7fc700 1 --2- 172.21.15.99:0/3505013937 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f51680cdab0 0x7f516815d860 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5150009910 tx=0x7f51500091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f77a46ec9ba755d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:34.412 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.408+0000 7f51667fc700 1 -- 172.21.15.99:0/3505013937 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5150011630 con 0x7f51680cdab0 2023-10-08T00:11:34.413 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.408+0000 7f51667fc700 1 -- 172.21.15.99:0/3505013937 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5150011790 con 0x7f51680cdab0 2023-10-08T00:11:34.413 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.408+0000 7f51667fc700 1 -- 172.21.15.99:0/3505013937 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5150011b30 con 0x7f51680cdab0 2023-10-08T00:11:34.413 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.408+0000 7f5167fff700 1 -- 172.21.15.99:0/3505013937 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f51680cdab0 msgr2=0x7f516815d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:34.413 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.408+0000 7f5167fff700 1 --2- 172.21.15.99:0/3505013937 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f51680cdab0 0x7f516815d860 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5150009910 tx=0x7f51500091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:34.413 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.408+0000 7f5167fff700 1 -- 172.21.15.99:0/3505013937 shutdown_connections 2023-10-08T00:11:34.413 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.408+0000 7f5167fff700 1 --2- 172.21.15.99:0/3505013937 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f51680cdab0 0x7f516815d860 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:34.413 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.408+0000 7f5167fff700 1 -- 172.21.15.99:0/3505013937 >> 172.21.15.99:0/3505013937 conn(0x7f5168067660 msgr2=0x7f5168067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:34.413 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.408+0000 7f5167fff700 1 -- 172.21.15.99:0/3505013937 shutdown_connections 2023-10-08T00:11:34.414 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.408+0000 7f5167fff700 1 -- 172.21.15.99:0/3505013937 wait complete. 2023-10-08T00:11:34.414 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.410+0000 7f5167fff700 1 Processor -- start 2023-10-08T00:11:34.414 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.410+0000 7f5167fff700 1 -- start start 2023-10-08T00:11:34.419 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.410+0000 7f5167fff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f51680cd6e0 0x7f5168139a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:34.419 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.410+0000 7f5167fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f51680cdab0 0x7f516816e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:34.419 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.410+0000 7f5167fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f516815dda0 0x7f516816e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:34.419 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.410+0000 7f5167fff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51680cc360 con 0x7f516815dda0 2023-10-08T00:11:34.419 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.410+0000 7f5167fff700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51680b2a40 con 0x7f51680cd6e0 2023-10-08T00:11:34.420 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.410+0000 7f5167fff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51680b1e30 con 0x7f51680cdab0 2023-10-08T00:11:34.420 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.410+0000 7f51677fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f516815dda0 0x7f516816e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:34.420 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.410+0000 7f51677fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f516815dda0 0x7f516816e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39732/0 (socket says 172.21.15.99:39732) 2023-10-08T00:11:34.420 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.410+0000 7f51677fe700 1 -- 172.21.15.99:0/1259482807 learned_addr learned my addr 172.21.15.99:0/1259482807 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:34.420 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.411+0000 7f5166ffd700 1 --2- 172.21.15.99:0/1259482807 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f51680cd6e0 0x7f5168139a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:34.420 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.411+0000 7f515e7fc700 1 --2- 172.21.15.99:0/1259482807 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f51680cdab0 0x7f516816e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:34.421 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.411+0000 7f51677fe700 1 -- 172.21.15.99:0/1259482807 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f51680cdab0 msgr2=0x7f516816e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:34.421 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.411+0000 7f51677fe700 1 --2- 172.21.15.99:0/1259482807 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f51680cdab0 0x7f516816e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:34.421 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.411+0000 7f51677fe700 1 -- 172.21.15.99:0/1259482807 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f51680cd6e0 msgr2=0x7f5168139a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:34.421 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.411+0000 7f51677fe700 1 --2- 172.21.15.99:0/1259482807 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f51680cd6e0 0x7f5168139a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:34.422 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.411+0000 7f51677fe700 1 -- 172.21.15.99:0/1259482807 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5150009420 con 0x7f516815dda0 2023-10-08T00:11:34.422 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.411+0000 7f51677fe700 1 --2- 172.21.15.99:0/1259482807 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f516815dda0 0x7f516816e5d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f516000c860 tx=0x7f5160002700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:34.422 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.411+0000 7f515f7fe700 1 -- 172.21.15.99:0/1259482807 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5160002030 con 0x7f516815dda0 2023-10-08T00:11:34.422 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.411+0000 7f515f7fe700 1 -- 172.21.15.99:0/1259482807 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51600021b0 con 0x7f516815dda0 2023-10-08T00:11:34.422 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.411+0000 7f5167fff700 1 -- 172.21.15.99:0/1259482807 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51680b2110 con 0x7f516815dda0 2023-10-08T00:11:34.422 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.411+0000 7f515f7fe700 1 -- 172.21.15.99:0/1259482807 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5160012630 con 0x7f516815dda0 2023-10-08T00:11:34.422 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.411+0000 7f5167fff700 1 -- 172.21.15.99:0/1259482807 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51680b26d0 con 0x7f516815dda0 2023-10-08T00:11:34.423 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.412+0000 7f5167fff700 1 -- 172.21.15.99:0/1259482807 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5168089280 con 0x7f516815dda0 2023-10-08T00:11:34.423 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.412+0000 7f515f7fe700 1 -- 172.21.15.99:0/1259482807 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5160012a20 con 0x7f516815dda0 2023-10-08T00:11:34.423 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.413+0000 7f515f7fe700 1 --2- 172.21.15.99:0/1259482807 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5148066f30 0x7f51480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:34.423 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.413+0000 7f515f7fe700 1 -- 172.21.15.99:0/1259482807 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f51600840f0 con 0x7f516815dda0 2023-10-08T00:11:34.423 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.413+0000 7f5166ffd700 1 --2- 172.21.15.99:0/1259482807 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5148066f30 0x7f51480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:34.423 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.413+0000 7f5166ffd700 1 --2- 172.21.15.99:0/1259482807 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5148066f30 0x7f51480693e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5158007fd0 tx=0x7f5158009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:34.423 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.417+0000 7f515f7fe700 1 -- 172.21.15.99:0/1259482807 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f51600516b0 con 0x7f516815dda0 2023-10-08T00:11:34.673 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.673+0000 7f5167fff700 1 -- 172.21.15.99:0/1259482807 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f516804bb80 con 0x7f516815dda0 2023-10-08T00:11:34.674 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.673+0000 7f515f7fe700 1 -- 172.21.15.99:0/1259482807 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f5160054cd0 con 0x7f516815dda0 2023-10-08T00:11:34.674 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:11:34.674 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:11:34.674 INFO:teuthology.orchestra.run.smithi099.stdout: "auth_dump": [ 2023-10-08T00:11:34.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:11:34.674 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.a", 2023-10-08T00:11:34.674 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlFlN9AhAAKdbeH0XznSOuyUnReD6RsA==", 2023-10-08T00:11:34.675 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:11:34.675 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:11:34.675 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2023-10-08T00:11:34.675 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2023-10-08T00:11:34.675 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:11:34.675 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:11:34.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:11:34.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:11:34.675 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.b", 2023-10-08T00:11:34.675 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlh2G0DRAA1mYGrUrELHsS9abUxCCG2g==", 2023-10-08T00:11:34.675 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:11:34.675 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:11:34.676 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2023-10-08T00:11:34.676 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2023-10-08T00:11:34.676 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:11:34.676 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:11:34.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:11:34.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:11:34.676 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.c", 2023-10-08T00:11:34.676 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFl/M1DCRAANrfHJGZO1s0KOyCdUzZWIA==", 2023-10-08T00:11:34.676 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:11:34.676 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:11:34.676 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2023-10-08T00:11:34.676 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2023-10-08T00:11:34.676 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:11:34.677 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:11:34.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:11:34.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:11:34.677 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.d", 2023-10-08T00:11:34.677 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlr5F2FBAAYYeG6EM0/N0FiEAwSbYRyA==", 2023-10-08T00:11:34.677 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:11:34.677 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:11:34.677 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2023-10-08T00:11:34.677 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2023-10-08T00:11:34.677 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:11:34.677 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:11:34.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:11:34.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:11:34.678 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.0", 2023-10-08T00:11:34.678 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB08yFla7rpKxAAWTOutLcrjDeR69JbPZMoLg==", 2023-10-08T00:11:34.678 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:11:34.678 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:11:34.678 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:11:34.678 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:11:34.678 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:11:34.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:11:34.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:11:34.678 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.1", 2023-10-08T00:11:34.679 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB08yFlw+y2MhAAAvLDIB/tb49IlaireTqTFg==", 2023-10-08T00:11:34.679 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:11:34.679 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:11:34.679 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:11:34.679 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:11:34.679 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:11:34.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:11:34.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:11:34.679 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.2", 2023-10-08T00:11:34.679 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB18yFlh4wvABAALXOtJZoEfGmt/pOBxReooQ==", 2023-10-08T00:11:34.679 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:11:34.680 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:11:34.680 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:11:34.680 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:11:34.680 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:11:34.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:11:34.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:11:34.680 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.3", 2023-10-08T00:11:34.680 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB18yFl8cVfCRAAoGXTACMT+f5Avll5mxG9OA==", 2023-10-08T00:11:34.680 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:11:34.680 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:11:34.681 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:11:34.681 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:11:34.681 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:11:34.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:11:34.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:11:34.681 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.4", 2023-10-08T00:11:34.681 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB48yFlft9WGhAA8JZN4MjSX9wbXiMKJnfCYg==", 2023-10-08T00:11:34.681 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:11:34.681 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:11:34.681 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:11:34.681 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:11:34.682 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:11:34.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:11:34.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:11:34.682 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.5", 2023-10-08T00:11:34.682 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB48yFlUf1BIRAA2FMBt1jAcm4ia/vYBDyojw==", 2023-10-08T00:11:34.682 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:11:34.682 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:11:34.682 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:11:34.682 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:11:34.682 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:11:34.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:11:34.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:11:34.683 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.6", 2023-10-08T00:11:34.683 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB48yFl0SI6KhAAI6G+Sqfm7OQmGf9nK59x9Q==", 2023-10-08T00:11:34.683 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:11:34.683 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:11:34.683 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:11:34.683 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:11:34.683 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:11:34.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:11:34.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:11:34.684 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.7", 2023-10-08T00:11:34.684 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB48yFlo5xWMxAA4p9OSLK0JpWhEExsifX+3A==", 2023-10-08T00:11:34.684 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:11:34.684 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:11:34.684 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:11:34.684 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:11:34.684 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:11:34.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:11:34.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:11:34.686 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.0", 2023-10-08T00:11:34.687 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlmzhQGBAAReslU/Pvcsr0dwE6l1N3/Q==", 2023-10-08T00:11:34.687 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:11:34.687 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:11:34.687 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2023-10-08T00:11:34.687 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2023-10-08T00:11:34.687 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2023-10-08T00:11:34.687 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:11:34.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:11:34.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:11:34.687 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.1", 2023-10-08T00:11:34.687 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFl2NKsIBAAUg6rXynl1J7qB6BkUizmpA==", 2023-10-08T00:11:34.687 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:11:34.688 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:11:34.688 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2023-10-08T00:11:34.688 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2023-10-08T00:11:34.688 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2023-10-08T00:11:34.688 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:11:34.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:11:34.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:11:34.688 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.2", 2023-10-08T00:11:34.688 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlMp6rHRAArVgJfA3mY4n78hOMeVlw1Q==", 2023-10-08T00:11:34.688 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:11:34.688 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:11:34.688 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2023-10-08T00:11:34.688 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2023-10-08T00:11:34.689 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2023-10-08T00:11:34.689 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:11:34.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:11:34.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:11:34.689 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.3", 2023-10-08T00:11:34.689 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlH0gNJhAAEgGVGiabVIWknpWUohOnHA==", 2023-10-08T00:11:34.689 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:11:34.689 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:11:34.689 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2023-10-08T00:11:34.689 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2023-10-08T00:11:34.689 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2023-10-08T00:11:34.689 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:11:34.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:11:34.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:11:34.690 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.admin", 2023-10-08T00:11:34.690 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBw8yFlidUfIxAAIvpvkHbyCx5JFvHo/g5A9g==", 2023-10-08T00:11:34.690 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:11:34.690 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2023-10-08T00:11:34.690 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2023-10-08T00:11:34.690 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2023-10-08T00:11:34.690 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:11:34.690 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:11:34.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:11:34.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:11:34.691 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.bootstrap-mds", 2023-10-08T00:11:34.691 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB88yFl1kcfEhAAk1PsyCsw6xOM22XGrMTEDQ==", 2023-10-08T00:11:34.691 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:11:34.691 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile bootstrap-mds" 2023-10-08T00:11:34.691 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:11:34.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:11:34.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:11:34.691 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.bootstrap-mgr", 2023-10-08T00:11:34.691 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB88yFlDFAfEhAANPYFRSI8feQQDmQZ3u7+nA==", 2023-10-08T00:11:34.691 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:11:34.692 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile bootstrap-mgr" 2023-10-08T00:11:34.692 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:11:34.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:11:34.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:11:34.692 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.bootstrap-osd", 2023-10-08T00:11:34.692 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB88yFlt1ofEhAAiDHw3HFxSrfsL83Ol9hFew==", 2023-10-08T00:11:34.692 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:11:34.692 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile bootstrap-osd" 2023-10-08T00:11:34.692 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:11:34.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:11:34.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:11:34.693 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.bootstrap-rbd", 2023-10-08T00:11:34.693 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB88yFl1WEfEhAAh59/oWfuvRBx8+Wl/Eptyg==", 2023-10-08T00:11:34.693 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:11:34.693 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile bootstrap-rbd" 2023-10-08T00:11:34.693 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:11:34.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:11:34.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:11:34.693 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-10-08T00:11:34.693 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB88yFl4mofEhAAJrDyM9ZSJPdFE98mlmnWwg==", 2023-10-08T00:11:34.694 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:11:34.694 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-10-08T00:11:34.694 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:11:34.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:11:34.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:11:34.694 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.bootstrap-rgw", 2023-10-08T00:11:34.694 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB88yFlOHIfEhAAUraD1w99fWpFUXwiXWJTSg==", 2023-10-08T00:11:34.694 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:11:34.694 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile bootstrap-rgw" 2023-10-08T00:11:34.694 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:11:34.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:11:34.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:11:34.695 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mgr.x", 2023-10-08T00:11:34.695 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBw8yFl2kMLOxAAsEyuLBLsjQCUS0ERDOi75A==", 2023-10-08T00:11:34.695 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:11:34.695 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2023-10-08T00:11:34.695 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile mgr", 2023-10-08T00:11:34.695 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:11:34.695 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:11:34.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:11:34.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:11:34.695 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mgr.y", 2023-10-08T00:11:34.695 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBw8yFl6ht5NxAAFzXnLDjklCePD6hw/pSfFg==", 2023-10-08T00:11:34.695 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:11:34.695 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2023-10-08T00:11:34.696 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile mgr", 2023-10-08T00:11:34.696 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:11:34.696 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:11:34.696 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:11:34.696 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:11:34.696 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:11:34.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.675+0000 7f5167fff700 1 -- 172.21.15.99:0/1259482807 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5148066f30 msgr2=0x7f51480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:34.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.675+0000 7f5167fff700 1 --2- 172.21.15.99:0/1259482807 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5148066f30 0x7f51480693e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5158007fd0 tx=0x7f5158009040 comp rx=0 tx=0).stop 2023-10-08T00:11:34.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.675+0000 7f5167fff700 1 -- 172.21.15.99:0/1259482807 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f516815dda0 msgr2=0x7f516816e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:34.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.675+0000 7f5167fff700 1 --2- 172.21.15.99:0/1259482807 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f516815dda0 0x7f516816e5d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f516000c860 tx=0x7f5160002700 comp rx=0 tx=0).stop 2023-10-08T00:11:34.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.675+0000 7f5167fff700 1 -- 172.21.15.99:0/1259482807 shutdown_connections 2023-10-08T00:11:34.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.675+0000 7f5167fff700 1 --2- 172.21.15.99:0/1259482807 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5148066f30 0x7f51480693e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:34.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.675+0000 7f5167fff700 1 --2- 172.21.15.99:0/1259482807 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f51680cd6e0 0x7f5168139a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:34.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.675+0000 7f5167fff700 1 --2- 172.21.15.99:0/1259482807 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f516815dda0 0x7f516816e5d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:34.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.675+0000 7f5167fff700 1 --2- 172.21.15.99:0/1259482807 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f51680cdab0 0x7f516816e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:34.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.675+0000 7f5167fff700 1 -- 172.21.15.99:0/1259482807 >> 172.21.15.99:0/1259482807 conn(0x7f5168067660 msgr2=0x7f51680b07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:34.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.675+0000 7f5167fff700 1 -- 172.21.15.99:0/1259482807 shutdown_connections 2023-10-08T00:11:34.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.675+0000 7f5167fff700 1 -- 172.21.15.99:0/1259482807 wait complete. 2023-10-08T00:11:34.699 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-10-08T00:11:34.814 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.813+0000 7f09faea6700 1 Processor -- start 2023-10-08T00:11:34.814 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.813+0000 7f09faea6700 1 -- start start 2023-10-08T00:11:34.816 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.814+0000 7f09faea6700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f09f4068880 0x7f09f4067d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:34.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.814+0000 7f09faea6700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09f40ad060 con 0x7f09f40ad540 2023-10-08T00:11:34.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.814+0000 7f09faea6700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09f406f030 con 0x7f09f4156b80 2023-10-08T00:11:34.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.814+0000 7f09faea6700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09f406f310 con 0x7f09f4068880 2023-10-08T00:11:34.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.814+0000 7f09f9443700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f09f40ad540 0x7f09f40ad940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:48502/0 (socket says 172.21.15.99:48502) 2023-10-08T00:11:34.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.814+0000 7f09f9443700 1 -- 172.21.15.99:0/1438259859 learned_addr learned my addr 172.21.15.99:0/1438259859 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:34.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.814+0000 7f09ebfff700 1 --2- 172.21.15.99:0/1438259859 >> v2:172.21.15.191:3300/0 conn(0x7f09f4068880 0x7f09f4067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:34.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.815+0000 7f09eb7fe700 1 -- 172.21.15.99:0/1438259859 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2521017186 0 0) 0x7f09f40ad060 con 0x7f09f40ad540 2023-10-08T00:11:34.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.815+0000 7f09eb7fe700 1 -- 172.21.15.99:0/1438259859 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09dc003570 con 0x7f09f40ad540 2023-10-08T00:11:34.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.815+0000 7f09ebfff700 1 -- 172.21.15.99:0/1438259859 >> v1:172.21.15.191:6789/0 conn(0x7f09f4156b80 legacy=0x7f09f4067660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:34.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.815+0000 7f09ebfff700 1 -- 172.21.15.99:0/1438259859 >> v1:172.21.15.99:6789/0 conn(0x7f09f40ad540 legacy=0x7f09f40ad940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:34.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.815+0000 7f09ebfff700 1 -- 172.21.15.99:0/1438259859 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f406f5f0 con 0x7f09f4068880 2023-10-08T00:11:34.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.815+0000 7f09ebfff700 1 --2- 172.21.15.99:0/1438259859 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09f4068880 0x7f09f4067d70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f09e40098d0 tx=0x7f09e4009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aecb8127a9ed4c67 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:34.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.815+0000 7f09eb7fe700 1 -- 172.21.15.99:0/1438259859 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e4005c10 con 0x7f09f4068880 2023-10-08T00:11:34.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.815+0000 7f09eb7fe700 1 -- 172.21.15.99:0/1438259859 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09e4005d70 con 0x7f09f4068880 2023-10-08T00:11:34.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.815+0000 7f09eb7fe700 1 -- 172.21.15.99:0/1438259859 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e4006280 con 0x7f09f4068880 2023-10-08T00:11:34.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.816+0000 7f09faea6700 1 -- 172.21.15.99:0/1438259859 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09f4068880 msgr2=0x7f09f4067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:34.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.816+0000 7f09faea6700 1 --2- 172.21.15.99:0/1438259859 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09f4068880 0x7f09f4067d70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f09e40098d0 tx=0x7f09e4009120 comp rx=0 tx=0).stop 2023-10-08T00:11:34.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.816+0000 7f09faea6700 1 -- 172.21.15.99:0/1438259859 shutdown_connections 2023-10-08T00:11:34.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.816+0000 7f09faea6700 1 --2- 172.21.15.99:0/1438259859 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09f4068880 0x7f09f4067d70 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:34.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.816+0000 7f09faea6700 1 -- 172.21.15.99:0/1438259859 >> 172.21.15.99:0/1438259859 conn(0x7f09f406ad40 msgr2=0x7f09f40cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:34.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.816+0000 7f09faea6700 1 -- 172.21.15.99:0/1438259859 shutdown_connections 2023-10-08T00:11:34.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.816+0000 7f09faea6700 1 -- 172.21.15.99:0/1438259859 wait complete. 2023-10-08T00:11:34.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.817+0000 7f09faea6700 1 Processor -- start 2023-10-08T00:11:34.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.817+0000 7f09faea6700 1 -- start start 2023-10-08T00:11:34.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.818+0000 7f09faea6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09f4068880 0x7f09f4155a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:34.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.818+0000 7f09faea6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09f40ad540 0x7f09f40ca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:34.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.818+0000 7f09faea6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09f4156b80 0x7f09f40aba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:34.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.818+0000 7f09faea6700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f4155fa0 con 0x7f09f4068880 2023-10-08T00:11:34.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.818+0000 7f09faea6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f41549e0 con 0x7f09f40ad540 2023-10-08T00:11:34.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.818+0000 7f09faea6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09f4154cc0 con 0x7f09f4156b80 2023-10-08T00:11:34.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.818+0000 7f09f8c42700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09f4068880 0x7f09f4155a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:34.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.818+0000 7f09f8c42700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09f4068880 0x7f09f4155a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39744/0 (socket says 172.21.15.99:39744) 2023-10-08T00:11:34.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.818+0000 7f09f8c42700 1 -- 172.21.15.99:0/3444187623 learned_addr learned my addr 172.21.15.99:0/3444187623 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:34.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.818+0000 7f09f8c42700 1 -- 172.21.15.99:0/3444187623 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09f4156b80 msgr2=0x7f09f40aba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:34.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.818+0000 7f09f8c42700 1 --2- 172.21.15.99:0/3444187623 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09f4156b80 0x7f09f40aba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:34.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.818+0000 7f09ebfff700 1 --2- 172.21.15.99:0/3444187623 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09f40ad540 0x7f09f40ca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:34.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.818+0000 7f09f8c42700 1 -- 172.21.15.99:0/3444187623 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09f40ad540 msgr2=0x7f09f40ca7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:34.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.818+0000 7f09f8c42700 1 --2- 172.21.15.99:0/3444187623 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09f40ad540 0x7f09f40ca7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:34.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.818+0000 7f09f8c42700 1 -- 172.21.15.99:0/3444187623 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e40093c0 con 0x7f09f4068880 2023-10-08T00:11:34.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.818+0000 7f09f8c42700 1 --2- 172.21.15.99:0/3444187623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09f4068880 0x7f09f4155a60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f09f000c590 tx=0x7f09f000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:34.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.818+0000 7f09e8ff9700 1 -- 172.21.15.99:0/3444187623 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f0011590 con 0x7f09f4068880 2023-10-08T00:11:34.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.818+0000 7f09e8ff9700 1 -- 172.21.15.99:0/3444187623 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09f00116f0 con 0x7f09f4068880 2023-10-08T00:11:34.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.818+0000 7f09faea6700 1 -- 172.21.15.99:0/3444187623 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09f4154fa0 con 0x7f09f4068880 2023-10-08T00:11:34.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.818+0000 7f09e8ff9700 1 -- 172.21.15.99:0/3444187623 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f0011ab0 con 0x7f09f4068880 2023-10-08T00:11:34.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.818+0000 7f09faea6700 1 -- 172.21.15.99:0/3444187623 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09f4155560 con 0x7f09f4068880 2023-10-08T00:11:34.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.819+0000 7f09faea6700 1 -- 172.21.15.99:0/3444187623 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09f41558d0 con 0x7f09f4068880 2023-10-08T00:11:34.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.820+0000 7f09e8ff9700 1 -- 172.21.15.99:0/3444187623 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f09f001b3b0 con 0x7f09f4068880 2023-10-08T00:11:34.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.820+0000 7f09e8ff9700 1 --2- 172.21.15.99:0/3444187623 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f09e0066f30 0x7f09e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:34.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.820+0000 7f09e8ff9700 1 -- 172.21.15.99:0/3444187623 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f09f0083770 con 0x7f09f4068880 2023-10-08T00:11:34.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.820+0000 7f09ebfff700 1 --2- 172.21.15.99:0/3444187623 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f09e0066f30 0x7f09e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:34.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.821+0000 7f09ebfff700 1 --2- 172.21.15.99:0/3444187623 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f09e0066f30 0x7f09e00693e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f09e4009160 tx=0x7f09e4016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:34.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:34.824+0000 7f09e8ff9700 1 -- 172.21.15.99:0/3444187623 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f09f0050d30 con 0x7f09f4068880 2023-10-08T00:11:35.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.085+0000 7f09faea6700 1 -- 172.21.15.99:0/3444187623 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f09f404c3c0 con 0x7f09f4068880 2023-10-08T00:11:35.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.088+0000 7f09e8ff9700 1 -- 172.21.15.99:0/3444187623 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7f09f0017020 con 0x7f09f4068880 2023-10-08T00:11:35.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.090+0000 7f09faea6700 1 -- 172.21.15.99:0/3444187623 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f09e0066f30 msgr2=0x7f09e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:35.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.090+0000 7f09faea6700 1 --2- 172.21.15.99:0/3444187623 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f09e0066f30 0x7f09e00693e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f09e4009160 tx=0x7f09e4016000 comp rx=0 tx=0).stop 2023-10-08T00:11:35.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.090+0000 7f09faea6700 1 -- 172.21.15.99:0/3444187623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09f4068880 msgr2=0x7f09f4155a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:35.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.090+0000 7f09faea6700 1 --2- 172.21.15.99:0/3444187623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09f4068880 0x7f09f4155a60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f09f000c590 tx=0x7f09f000c950 comp rx=0 tx=0).stop 2023-10-08T00:11:35.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.090+0000 7f09faea6700 1 -- 172.21.15.99:0/3444187623 shutdown_connections 2023-10-08T00:11:35.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.090+0000 7f09faea6700 1 --2- 172.21.15.99:0/3444187623 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f09e0066f30 0x7f09e00693e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:35.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.090+0000 7f09faea6700 1 --2- 172.21.15.99:0/3444187623 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09f4156b80 0x7f09f40aba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:35.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.090+0000 7f09faea6700 1 --2- 172.21.15.99:0/3444187623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09f4068880 0x7f09f4155a60 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:35.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.090+0000 7f09faea6700 1 --2- 172.21.15.99:0/3444187623 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09f40ad540 0x7f09f40ca7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:35.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.090+0000 7f09faea6700 1 -- 172.21.15.99:0/3444187623 >> 172.21.15.99:0/3444187623 conn(0x7f09f406ad40 msgr2=0x7f09f40b24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:35.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.090+0000 7f09faea6700 1 -- 172.21.15.99:0/3444187623 shutdown_connections 2023-10-08T00:11:35.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.090+0000 7f09faea6700 1 -- 172.21.15.99:0/3444187623 wait complete. 2023-10-08T00:11:35.103 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-10-08T00:11:35.217 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.216+0000 7f38c7373700 1 Processor -- start 2023-10-08T00:11:35.217 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.216+0000 7f38c7373700 1 -- start start 2023-10-08T00:11:35.220 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.217+0000 7f38c7373700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f38c01523e0 0x7f38c006c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:35.220 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.217+0000 7f38c7373700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f38c0158e50 0x7f38c00cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:35.220 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.217+0000 7f38c7373700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38c006c010 con 0x7f38c0152010 2023-10-08T00:11:35.220 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.217+0000 7f38c7373700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38c006bd10 con 0x7f38c01523e0 2023-10-08T00:11:35.220 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.217+0000 7f38c7373700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38c015c060 con 0x7f38c0158e50 2023-10-08T00:11:35.220 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.218+0000 7f38c490e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f38c01523e0 0x7f38c006c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:35.220 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.218+0000 7f38c5910700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f38c0158e50 0x7f38c00cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:35.220 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.218+0000 7f38c5910700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f38c0158e50 0x7f38c00cdf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.99:35204/0 (socket says 172.21.15.99:35204) 2023-10-08T00:11:35.220 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.218+0000 7f38c5910700 1 -- 172.21.15.99:0/1220849508 learned_addr learned my addr 172.21.15.99:0/1220849508 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:35.221 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.218+0000 7f38c510f700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f38c0152010 0x7f38c015b810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:37000/0 (socket says 172.21.15.99:37000) 2023-10-08T00:11:35.221 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.218+0000 7f38c5910700 1 -- 172.21.15.99:0/1220849508 >> v2:172.21.15.191:3300/0 conn(0x7f38c01523e0 msgr2=0x7f38c006c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:35.221 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.218+0000 7f38c5910700 1 --2- 172.21.15.99:0/1220849508 >> v2:172.21.15.191:3300/0 conn(0x7f38c01523e0 0x7f38c006c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:35.221 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.218+0000 7f38c5910700 1 -- 172.21.15.99:0/1220849508 >> v1:172.21.15.191:6789/0 conn(0x7f38c0152010 legacy=0x7f38c015b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:35.221 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.218+0000 7f38c5910700 1 -- 172.21.15.99:0/1220849508 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c015c340 con 0x7f38c0158e50 2023-10-08T00:11:35.221 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.218+0000 7f38c5910700 1 --2- 172.21.15.99:0/1220849508 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f38c0158e50 0x7f38c00cdf30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f38b8009f20 tx=0x7f38b8009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88f42243041c8864 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:35.221 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.218+0000 7f38b7fff700 1 -- 172.21.15.99:0/1220849508 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b800e070 con 0x7f38c0158e50 2023-10-08T00:11:35.221 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.218+0000 7f38b7fff700 1 -- 172.21.15.99:0/1220849508 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38b800b9d0 con 0x7f38c0158e50 2023-10-08T00:11:35.221 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.218+0000 7f38b7fff700 1 -- 172.21.15.99:0/1220849508 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b80092e0 con 0x7f38c0158e50 2023-10-08T00:11:35.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.219+0000 7f38c7373700 1 -- 172.21.15.99:0/1220849508 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f38c0158e50 msgr2=0x7f38c00cdf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:35.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.219+0000 7f38c7373700 1 --2- 172.21.15.99:0/1220849508 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f38c0158e50 0x7f38c00cdf30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f38b8009f20 tx=0x7f38b8009840 comp rx=0 tx=0).stop 2023-10-08T00:11:35.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.219+0000 7f38c7373700 1 -- 172.21.15.99:0/1220849508 shutdown_connections 2023-10-08T00:11:35.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.219+0000 7f38c7373700 1 --2- 172.21.15.99:0/1220849508 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f38c0158e50 0x7f38c00cdf30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:35.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.219+0000 7f38c7373700 1 --2- 172.21.15.99:0/1220849508 >> v2:172.21.15.191:3300/0 conn(0x7f38c01523e0 0x7f38c006c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:35.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.219+0000 7f38c7373700 1 -- 172.21.15.99:0/1220849508 >> 172.21.15.99:0/1220849508 conn(0x7f38c0069870 msgr2=0x7f38c0069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:35.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.219+0000 7f38c7373700 1 -- 172.21.15.99:0/1220849508 shutdown_connections 2023-10-08T00:11:35.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.219+0000 7f38c7373700 1 -- 172.21.15.99:0/1220849508 wait complete. 2023-10-08T00:11:35.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.220+0000 7f38c7373700 1 Processor -- start 2023-10-08T00:11:35.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.220+0000 7f38c7373700 1 -- start start 2023-10-08T00:11:35.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.221+0000 7f38c7373700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38c0152010 0x7f38c0159c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:35.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.221+0000 7f38c7373700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f38c01523e0 0x7f38c014ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:35.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.221+0000 7f38c7373700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38c0158e50 0x7f38c014b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:35.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.221+0000 7f38c7373700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c0168c60 con 0x7f38c0158e50 2023-10-08T00:11:35.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.221+0000 7f38c7373700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c0167dd0 con 0x7f38c0152010 2023-10-08T00:11:35.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.221+0000 7f38c7373700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38c0168050 con 0x7f38c01523e0 2023-10-08T00:11:35.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.221+0000 7f38c5910700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38c0158e50 0x7f38c014b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:35.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.221+0000 7f38c5910700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38c0158e50 0x7f38c014b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39758/0 (socket says 172.21.15.99:39758) 2023-10-08T00:11:35.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.221+0000 7f38c5910700 1 -- 172.21.15.99:0/3290652012 learned_addr learned my addr 172.21.15.99:0/3290652012 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:35.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.221+0000 7f38c490e700 1 --2- 172.21.15.99:0/3290652012 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f38c01523e0 0x7f38c014ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:35.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.221+0000 7f38c510f700 1 --2- 172.21.15.99:0/3290652012 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38c0152010 0x7f38c0159c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:35.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.221+0000 7f38c5910700 1 -- 172.21.15.99:0/3290652012 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f38c01523e0 msgr2=0x7f38c014ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:35.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.221+0000 7f38c5910700 1 --2- 172.21.15.99:0/3290652012 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f38c01523e0 0x7f38c014ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:35.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.221+0000 7f38c5910700 1 -- 172.21.15.99:0/3290652012 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38c0152010 msgr2=0x7f38c0159c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:35.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.221+0000 7f38c5910700 1 --2- 172.21.15.99:0/3290652012 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38c0152010 0x7f38c0159c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:35.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.221+0000 7f38c5910700 1 -- 172.21.15.99:0/3290652012 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b8009ae0 con 0x7f38c0158e50 2023-10-08T00:11:35.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.222+0000 7f38c5910700 1 --2- 172.21.15.99:0/3290652012 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38c0158e50 0x7f38c014b1b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f38b800b610 tx=0x7f38b8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:35.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.222+0000 7f38b57fa700 1 -- 172.21.15.99:0/3290652012 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b8007ac0 con 0x7f38c0158e50 2023-10-08T00:11:35.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.222+0000 7f38b57fa700 1 -- 172.21.15.99:0/3290652012 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38b8007c40 con 0x7f38c0158e50 2023-10-08T00:11:35.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.222+0000 7f38c7373700 1 -- 172.21.15.99:0/3290652012 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38c0168330 con 0x7f38c0158e50 2023-10-08T00:11:35.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.222+0000 7f38b57fa700 1 -- 172.21.15.99:0/3290652012 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b801e430 con 0x7f38c0158e50 2023-10-08T00:11:35.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.222+0000 7f38c7373700 1 -- 172.21.15.99:0/3290652012 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38c01688f0 con 0x7f38c0158e50 2023-10-08T00:11:35.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.223+0000 7f38c7373700 1 -- 172.21.15.99:0/3290652012 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38c004c3c0 con 0x7f38c0158e50 2023-10-08T00:11:35.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.223+0000 7f38b57fa700 1 -- 172.21.15.99:0/3290652012 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f38b8003040 con 0x7f38c0158e50 2023-10-08T00:11:35.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.223+0000 7f38b57fa700 1 --2- 172.21.15.99:0/3290652012 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f38a4066f30 0x7f38a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:35.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.223+0000 7f38b57fa700 1 -- 172.21.15.99:0/3290652012 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f38b80862e0 con 0x7f38c0158e50 2023-10-08T00:11:35.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.224+0000 7f38c510f700 1 --2- 172.21.15.99:0/3290652012 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f38a4066f30 0x7f38a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:35.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.224+0000 7f38c510f700 1 --2- 172.21.15.99:0/3290652012 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f38a4066f30 0x7f38a40693e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f38c0071c90 tx=0x7f38bc007f40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:35.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.228+0000 7f38b57fa700 1 -- 172.21.15.99:0/3290652012 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f38b80538a0 con 0x7f38c0158e50 2023-10-08T00:11:35.489 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.488+0000 7f38c7373700 1 -- 172.21.15.99:0/3290652012 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f38c0168da0 con 0x7f38c0158e50 2023-10-08T00:11:35.494 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.494+0000 7f38b57fa700 1 -- 172.21.15.99:0/3290652012 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f38c0168da0 con 0x7f38c0158e50 2023-10-08T00:11:35.496 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.496+0000 7f38c7373700 1 -- 172.21.15.99:0/3290652012 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f38a4066f30 msgr2=0x7f38a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:35.497 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.496+0000 7f38c7373700 1 --2- 172.21.15.99:0/3290652012 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f38a4066f30 0x7f38a40693e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f38c0071c90 tx=0x7f38bc007f40 comp rx=0 tx=0).stop 2023-10-08T00:11:35.497 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.496+0000 7f38c7373700 1 -- 172.21.15.99:0/3290652012 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38c0158e50 msgr2=0x7f38c014b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:35.497 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.496+0000 7f38c7373700 1 --2- 172.21.15.99:0/3290652012 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38c0158e50 0x7f38c014b1b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f38b800b610 tx=0x7f38b8018000 comp rx=0 tx=0).stop 2023-10-08T00:11:35.497 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.496+0000 7f38c7373700 1 -- 172.21.15.99:0/3290652012 shutdown_connections 2023-10-08T00:11:35.497 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.496+0000 7f38c7373700 1 --2- 172.21.15.99:0/3290652012 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f38a4066f30 0x7f38a40693e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:35.497 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.496+0000 7f38c7373700 1 --2- 172.21.15.99:0/3290652012 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38c0152010 0x7f38c0159c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:35.497 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.496+0000 7f38c7373700 1 --2- 172.21.15.99:0/3290652012 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38c0158e50 0x7f38c014b1b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:35.497 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.496+0000 7f38c7373700 1 --2- 172.21.15.99:0/3290652012 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f38c01523e0 0x7f38c014ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:35.497 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.496+0000 7f38c7373700 1 -- 172.21.15.99:0/3290652012 >> 172.21.15.99:0/3290652012 conn(0x7f38c0069870 msgr2=0x7f38c015b740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:35.497 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.496+0000 7f38c7373700 1 -- 172.21.15.99:0/3290652012 shutdown_connections 2023-10-08T00:11:35.498 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.496+0000 7f38c7373700 1 -- 172.21.15.99:0/3290652012 wait complete. 2023-10-08T00:11:35.508 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-10-08T00:11:35.623 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.622+0000 7fd7068d6700 1 Processor -- start 2023-10-08T00:11:35.623 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.622+0000 7fd7068d6700 1 -- start start 2023-10-08T00:11:35.625 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.623+0000 7fd7068d6700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd700157c40 0x7fd700158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:35.625 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.623+0000 7fd7068d6700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd70015cbb0 con 0x7fd70015a620 2023-10-08T00:11:35.625 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.623+0000 7fd7068d6700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd70015bfa0 con 0x7fd700157870 2023-10-08T00:11:35.625 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.623+0000 7fd7068d6700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd70015bdb0 con 0x7fd700157c40 2023-10-08T00:11:35.626 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.623+0000 7fd704e73700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd700157c40 0x7fd700158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:35.626 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.623+0000 7fd6ff7fe700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fd70015a620 0x7fd70015d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:48518/0 (socket says 172.21.15.99:48518) 2023-10-08T00:11:35.626 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.623+0000 7fd6ff7fe700 1 -- 172.21.15.99:0/3275621212 learned_addr learned my addr 172.21.15.99:0/3275621212 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:35.626 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.623+0000 7fd704e73700 1 -- 172.21.15.99:0/3275621212 >> v1:172.21.15.191:6789/0 conn(0x7fd700157870 legacy=0x7fd70015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:35.626 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.623+0000 7fd704e73700 1 -- 172.21.15.99:0/3275621212 >> v1:172.21.15.99:6789/0 conn(0x7fd70015a620 legacy=0x7fd70015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:35.626 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.623+0000 7fd704e73700 1 -- 172.21.15.99:0/3275621212 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd70015c280 con 0x7fd700157c40 2023-10-08T00:11:35.626 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.623+0000 7fd6feffd700 1 -- 172.21.15.99:0/3275621212 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1301483083 0 0) 0x7fd70015cbb0 con 0x7fd70015a620 2023-10-08T00:11:35.626 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.623+0000 7fd704e73700 1 --2- 172.21.15.99:0/3275621212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd700157c40 0x7fd700158110 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8009e60 tx=0x7fd6f8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5140f05537c25183 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:35.626 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.624+0000 7fd6feffd700 1 -- 172.21.15.99:0/3275621212 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f800e070 con 0x7fd700157c40 2023-10-08T00:11:35.627 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.624+0000 7fd6feffd700 1 -- 172.21.15.99:0/3275621212 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f800bbf0 con 0x7fd700157c40 2023-10-08T00:11:35.627 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.624+0000 7fd6feffd700 1 -- 172.21.15.99:0/3275621212 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f8012430 con 0x7fd700157c40 2023-10-08T00:11:35.627 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.624+0000 7fd7068d6700 1 -- 172.21.15.99:0/3275621212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd700157c40 msgr2=0x7fd700158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:35.627 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.624+0000 7fd7068d6700 1 --2- 172.21.15.99:0/3275621212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd700157c40 0x7fd700158110 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8009e60 tx=0x7fd6f8009780 comp rx=0 tx=0).stop 2023-10-08T00:11:35.627 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.624+0000 7fd7068d6700 1 -- 172.21.15.99:0/3275621212 shutdown_connections 2023-10-08T00:11:35.627 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.624+0000 7fd7068d6700 1 --2- 172.21.15.99:0/3275621212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd700157c40 0x7fd700158110 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:35.627 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.624+0000 7fd7068d6700 1 -- 172.21.15.99:0/3275621212 >> 172.21.15.99:0/3275621212 conn(0x7fd700068ce0 msgr2=0x7fd7000690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:35.627 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.624+0000 7fd7068d6700 1 -- 172.21.15.99:0/3275621212 shutdown_connections 2023-10-08T00:11:35.627 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.624+0000 7fd7068d6700 1 -- 172.21.15.99:0/3275621212 wait complete. 2023-10-08T00:11:35.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.625+0000 7fd7068d6700 1 Processor -- start 2023-10-08T00:11:35.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.625+0000 7fd7068d6700 1 -- start start 2023-10-08T00:11:35.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.626+0000 7fd7068d6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd700157870 0x7fd7000b20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:35.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.626+0000 7fd7068d6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd700157c40 0x7fd70016d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:35.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.626+0000 7fd7068d6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd70015a620 0x7fd70016dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:35.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.626+0000 7fd7068d6700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd700150d90 con 0x7fd70015a620 2023-10-08T00:11:35.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.626+0000 7fd7068d6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd700129df0 con 0x7fd700157c40 2023-10-08T00:11:35.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.626+0000 7fd7068d6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7000b2a10 con 0x7fd700157870 2023-10-08T00:11:35.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.626+0000 7fd704e73700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd70015a620 0x7fd70016dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:35.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.626+0000 7fd704e73700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd70015a620 0x7fd70016dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39766/0 (socket says 172.21.15.99:39766) 2023-10-08T00:11:35.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.626+0000 7fd704e73700 1 -- 172.21.15.99:0/2608937886 learned_addr learned my addr 172.21.15.99:0/2608937886 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:35.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.626+0000 7fd6fffff700 1 --2- 172.21.15.99:0/2608937886 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd700157870 0x7fd7000b20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:35.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.626+0000 7fd6ff7fe700 1 --2- 172.21.15.99:0/2608937886 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd700157c40 0x7fd70016d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:35.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.626+0000 7fd704e73700 1 -- 172.21.15.99:0/2608937886 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd700157870 msgr2=0x7fd7000b20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:35.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.626+0000 7fd704e73700 1 --2- 172.21.15.99:0/2608937886 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd700157870 0x7fd7000b20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:35.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.626+0000 7fd704e73700 1 -- 172.21.15.99:0/2608937886 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd700157c40 msgr2=0x7fd70016d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:35.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.626+0000 7fd704e73700 1 --2- 172.21.15.99:0/2608937886 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd700157c40 0x7fd70016d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:35.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.626+0000 7fd704e73700 1 -- 172.21.15.99:0/2608937886 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f8009a20 con 0x7fd70015a620 2023-10-08T00:11:35.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.627+0000 7fd704e73700 1 --2- 172.21.15.99:0/2608937886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd70015a620 0x7fd70016dc20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd70006b530 tx=0x7fd6f8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:35.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.627+0000 7fd6e7fff700 1 -- 172.21.15.99:0/2608937886 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f800bde0 con 0x7fd70015a620 2023-10-08T00:11:35.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.627+0000 7fd6e7fff700 1 -- 172.21.15.99:0/2608937886 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f801e640 con 0x7fd70015a620 2023-10-08T00:11:35.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.627+0000 7fd7068d6700 1 -- 172.21.15.99:0/2608937886 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7000b2cf0 con 0x7fd70015a620 2023-10-08T00:11:35.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.627+0000 7fd6e7fff700 1 -- 172.21.15.99:0/2608937886 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f801ea00 con 0x7fd70015a620 2023-10-08T00:11:35.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.627+0000 7fd7068d6700 1 -- 172.21.15.99:0/2608937886 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd70012a110 con 0x7fd70015a620 2023-10-08T00:11:35.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.628+0000 7fd7068d6700 1 -- 172.21.15.99:0/2608937886 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd70012a2a0 con 0x7fd70015a620 2023-10-08T00:11:35.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.628+0000 7fd6e7fff700 1 -- 172.21.15.99:0/2608937886 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd6f8024070 con 0x7fd70015a620 2023-10-08T00:11:35.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.628+0000 7fd6e7fff700 1 --2- 172.21.15.99:0/2608937886 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd6ec066f30 0x7fd6ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:35.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.629+0000 7fd6e7fff700 1 -- 172.21.15.99:0/2608937886 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fd6f8086b40 con 0x7fd70015a620 2023-10-08T00:11:35.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.629+0000 7fd6fffff700 1 --2- 172.21.15.99:0/2608937886 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd6ec066f30 0x7fd6ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:35.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.629+0000 7fd6fffff700 1 --2- 172.21.15.99:0/2608937886 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd6ec066f30 0x7fd6ec0693e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0003a30 tx=0x7fd6f0007ea0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:35.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.633+0000 7fd6e7fff700 1 -- 172.21.15.99:0/2608937886 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd70012a2a0 con 0x7fd70015a620 2023-10-08T00:11:35.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.888+0000 7fd7068d6700 1 -- 172.21.15.99:0/2608937886 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fd70012a2a0 con 0x7fd70015a620 2023-10-08T00:11:35.894 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.893+0000 7fd6e7fff700 1 -- 172.21.15.99:0/2608937886 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7fd70012a2a0 con 0x7fd70015a620 2023-10-08T00:11:35.896 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.895+0000 7fd7068d6700 1 -- 172.21.15.99:0/2608937886 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd6ec066f30 msgr2=0x7fd6ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:35.896 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.895+0000 7fd7068d6700 1 --2- 172.21.15.99:0/2608937886 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd6ec066f30 0x7fd6ec0693e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0003a30 tx=0x7fd6f0007ea0 comp rx=0 tx=0).stop 2023-10-08T00:11:35.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.895+0000 7fd7068d6700 1 -- 172.21.15.99:0/2608937886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd70015a620 msgr2=0x7fd70016dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:35.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.895+0000 7fd7068d6700 1 --2- 172.21.15.99:0/2608937886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd70015a620 0x7fd70016dc20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd70006b530 tx=0x7fd6f8018000 comp rx=0 tx=0).stop 2023-10-08T00:11:35.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.895+0000 7fd7068d6700 1 -- 172.21.15.99:0/2608937886 shutdown_connections 2023-10-08T00:11:35.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.895+0000 7fd7068d6700 1 --2- 172.21.15.99:0/2608937886 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd6ec066f30 0x7fd6ec0693e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:35.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.895+0000 7fd7068d6700 1 --2- 172.21.15.99:0/2608937886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd70015a620 0x7fd70016dc20 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:35.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.895+0000 7fd7068d6700 1 --2- 172.21.15.99:0/2608937886 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd700157870 0x7fd7000b20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:35.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.895+0000 7fd7068d6700 1 --2- 172.21.15.99:0/2608937886 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd700157c40 0x7fd70016d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:35.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.895+0000 7fd7068d6700 1 -- 172.21.15.99:0/2608937886 >> 172.21.15.99:0/2608937886 conn(0x7fd700068ce0 msgr2=0x7fd7000d0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:35.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.896+0000 7fd7068d6700 1 -- 172.21.15.99:0/2608937886 shutdown_connections 2023-10-08T00:11:35.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:35.896+0000 7fd7068d6700 1 -- 172.21.15.99:0/2608937886 wait complete. 2023-10-08T00:11:35.908 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-10-08T00:11:36.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.022+0000 7faffa2ea700 1 Processor -- start 2023-10-08T00:11:36.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.022+0000 7faffa2ea700 1 -- start start 2023-10-08T00:11:36.025 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.023+0000 7faffa2ea700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faff415a170 0x7faff4068b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:36.026 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.023+0000 7faffa2ea700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7faff4069040 0x7faff40af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:36.026 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.023+0000 7faffa2ea700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faff415ea70 con 0x7faff4159da0 2023-10-08T00:11:36.026 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.023+0000 7faffa2ea700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faff40b34b0 con 0x7faff4069040 2023-10-08T00:11:36.026 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.023+0000 7faffa2ea700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faff40694e0 con 0x7faff415a170 2023-10-08T00:11:36.026 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.023+0000 7faff9ae9700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7faff4069040 0x7faff40af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:36.026 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.023+0000 7faff92e8700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7faff4159da0 0x7faff40b2da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:37020/0 (socket says 172.21.15.99:37020) 2023-10-08T00:11:36.026 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.023+0000 7faff8ae7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faff415a170 0x7faff4068b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:36.026 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.023+0000 7faff92e8700 1 -- 172.21.15.99:0/3108337346 learned_addr learned my addr 172.21.15.99:0/3108337346 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:36.027 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.024+0000 7fafebfff700 1 -- 172.21.15.99:0/3108337346 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2903889010 0 0) 0x7faff415ea70 con 0x7faff4159da0 2023-10-08T00:11:36.027 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.024+0000 7fafebfff700 1 -- 172.21.15.99:0/3108337346 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafe0003570 con 0x7faff4159da0 2023-10-08T00:11:36.027 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.024+0000 7faff8ae7700 1 -- 172.21.15.99:0/3108337346 >> v2:172.21.15.191:3300/0 conn(0x7faff4069040 msgr2=0x7faff40af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:36.027 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.024+0000 7faff8ae7700 1 --2- 172.21.15.99:0/3108337346 >> v2:172.21.15.191:3300/0 conn(0x7faff4069040 0x7faff40af4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:36.027 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.024+0000 7faff8ae7700 1 -- 172.21.15.99:0/3108337346 >> v1:172.21.15.191:6789/0 conn(0x7faff4159da0 legacy=0x7faff40b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:36.027 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.024+0000 7faff8ae7700 1 -- 172.21.15.99:0/3108337346 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff40b3990 con 0x7faff415a170 2023-10-08T00:11:36.028 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.024+0000 7faff9ae9700 1 --2- 172.21.15.99:0/3108337346 >> v2:172.21.15.191:3300/0 conn(0x7faff4069040 0x7faff40af4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:11:36.028 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.024+0000 7faff8ae7700 1 --2- 172.21.15.99:0/3108337346 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faff415a170 0x7faff4068b00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fafe40098e0 tx=0x7fafe40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf7f8f0141103e9e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:36.028 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.024+0000 7fafebfff700 1 -- 172.21.15.99:0/3108337346 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe4011630 con 0x7faff415a170 2023-10-08T00:11:36.028 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.024+0000 7fafebfff700 1 -- 172.21.15.99:0/3108337346 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafe4011790 con 0x7faff415a170 2023-10-08T00:11:36.028 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.024+0000 7fafebfff700 1 -- 172.21.15.99:0/3108337346 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe4011b30 con 0x7faff415a170 2023-10-08T00:11:36.028 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.025+0000 7faffa2ea700 1 -- 172.21.15.99:0/3108337346 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faff415a170 msgr2=0x7faff4068b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:36.028 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.025+0000 7faffa2ea700 1 --2- 172.21.15.99:0/3108337346 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faff415a170 0x7faff4068b00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fafe40098e0 tx=0x7fafe40091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:36.028 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.025+0000 7faffa2ea700 1 -- 172.21.15.99:0/3108337346 shutdown_connections 2023-10-08T00:11:36.028 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.025+0000 7faffa2ea700 1 --2- 172.21.15.99:0/3108337346 >> v2:172.21.15.191:3300/0 conn(0x7faff4069040 0x7faff40af4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:36.029 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.025+0000 7faffa2ea700 1 --2- 172.21.15.99:0/3108337346 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faff415a170 0x7faff4068b00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:36.029 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.025+0000 7faffa2ea700 1 -- 172.21.15.99:0/3108337346 >> 172.21.15.99:0/3108337346 conn(0x7faff40d0bc0 msgr2=0x7faff40d0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:36.029 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.025+0000 7faffa2ea700 1 -- 172.21.15.99:0/3108337346 shutdown_connections 2023-10-08T00:11:36.029 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.025+0000 7faffa2ea700 1 -- 172.21.15.99:0/3108337346 wait complete. 2023-10-08T00:11:36.032 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.026+0000 7faffa2ea700 1 Processor -- start 2023-10-08T00:11:36.033 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.026+0000 7faffa2ea700 1 -- start start 2023-10-08T00:11:36.033 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.027+0000 7faffa2ea700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faff4069040 0x7faff41570b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:36.033 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.027+0000 7faffa2ea700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faff4159da0 0x7faff415c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:36.033 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.027+0000 7faffa2ea700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faff415a170 0x7faff40cef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:36.033 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.027+0000 7faffa2ea700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff40c8980 con 0x7faff415a170 2023-10-08T00:11:36.033 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.027+0000 7faffa2ea700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff41575f0 con 0x7faff4159da0 2023-10-08T00:11:36.033 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.027+0000 7faffa2ea700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faff415cbd0 con 0x7faff4069040 2023-10-08T00:11:36.034 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.027+0000 7faff9ae9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faff415a170 0x7faff40cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:36.034 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.027+0000 7faff9ae9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faff415a170 0x7faff40cef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39780/0 (socket says 172.21.15.99:39780) 2023-10-08T00:11:36.034 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.027+0000 7faff9ae9700 1 -- 172.21.15.99:0/1129463062 learned_addr learned my addr 172.21.15.99:0/1129463062 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:36.034 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.027+0000 7faff8ae7700 1 --2- 172.21.15.99:0/1129463062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faff4159da0 0x7faff415c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:36.034 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.027+0000 7faff92e8700 1 --2- 172.21.15.99:0/1129463062 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faff4069040 0x7faff41570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:36.035 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.027+0000 7faff9ae9700 1 -- 172.21.15.99:0/1129463062 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faff4069040 msgr2=0x7faff41570b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:36.035 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.027+0000 7faff9ae9700 1 --2- 172.21.15.99:0/1129463062 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faff4069040 0x7faff41570b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:36.035 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.027+0000 7faff9ae9700 1 -- 172.21.15.99:0/1129463062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faff4159da0 msgr2=0x7faff415c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:36.035 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.027+0000 7faff9ae9700 1 --2- 172.21.15.99:0/1129463062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faff4159da0 0x7faff415c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:36.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.027+0000 7faff9ae9700 1 -- 172.21.15.99:0/1129463062 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafec009a20 con 0x7faff415a170 2023-10-08T00:11:36.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.027+0000 7faff9ae9700 1 --2- 172.21.15.99:0/1129463062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faff415a170 0x7faff40cef70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fafec009fc0 tx=0x7fafec00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:36.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.027+0000 7fafe97fa700 1 -- 172.21.15.99:0/1129463062 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafec00cbd0 con 0x7faff415a170 2023-10-08T00:11:36.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.027+0000 7fafe97fa700 1 -- 172.21.15.99:0/1129463062 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafec00f040 con 0x7faff415a170 2023-10-08T00:11:36.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.027+0000 7faffa2ea700 1 -- 172.21.15.99:0/1129463062 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafe4009420 con 0x7faff415a170 2023-10-08T00:11:36.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.027+0000 7fafe97fa700 1 -- 172.21.15.99:0/1129463062 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafec014480 con 0x7faff415a170 2023-10-08T00:11:36.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.027+0000 7faffa2ea700 1 -- 172.21.15.99:0/1129463062 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faff40c8050 con 0x7faff415a170 2023-10-08T00:11:36.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.028+0000 7faffa2ea700 1 -- 172.21.15.99:0/1129463062 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faff4089480 con 0x7faff415a170 2023-10-08T00:11:36.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.029+0000 7fafe97fa700 1 -- 172.21.15.99:0/1129463062 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fafec003020 con 0x7faff415a170 2023-10-08T00:11:36.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.029+0000 7fafe97fa700 1 --2- 172.21.15.99:0/1129463062 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fafd8066f30 0x7fafd80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:36.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.029+0000 7fafe97fa700 1 -- 172.21.15.99:0/1129463062 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fafec084b00 con 0x7faff415a170 2023-10-08T00:11:36.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.029+0000 7faff92e8700 1 --2- 172.21.15.99:0/1129463062 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fafd8066f30 0x7fafd80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:36.039 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.030+0000 7faff92e8700 1 --2- 172.21.15.99:0/1129463062 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fafd8066f30 0x7fafd80693e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7faff0007fd0 tx=0x7faff0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:36.039 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.033+0000 7fafe97fa700 1 -- 172.21.15.99:0/1129463062 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fafec0520c0 con 0x7faff415a170 2023-10-08T00:11:36.294 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.293+0000 7faffa2ea700 1 -- 172.21.15.99:0/1129463062 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7faff40b0b70 con 0x7faff415a170 2023-10-08T00:11:36.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.301+0000 7fafe97fa700 1 -- 172.21.15.99:0/1129463062 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7fafec0556e0 con 0x7faff415a170 2023-10-08T00:11:36.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.303+0000 7faffa2ea700 1 -- 172.21.15.99:0/1129463062 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fafd8066f30 msgr2=0x7fafd80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:36.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.303+0000 7faffa2ea700 1 --2- 172.21.15.99:0/1129463062 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fafd8066f30 0x7fafd80693e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7faff0007fd0 tx=0x7faff0009040 comp rx=0 tx=0).stop 2023-10-08T00:11:36.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.303+0000 7faffa2ea700 1 -- 172.21.15.99:0/1129463062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faff415a170 msgr2=0x7faff40cef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:36.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.303+0000 7faffa2ea700 1 --2- 172.21.15.99:0/1129463062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faff415a170 0x7faff40cef70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fafec009fc0 tx=0x7fafec00b040 comp rx=0 tx=0).stop 2023-10-08T00:11:36.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.303+0000 7faffa2ea700 1 -- 172.21.15.99:0/1129463062 shutdown_connections 2023-10-08T00:11:36.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.303+0000 7faffa2ea700 1 --2- 172.21.15.99:0/1129463062 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fafd8066f30 0x7fafd80693e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:36.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.303+0000 7faffa2ea700 1 --2- 172.21.15.99:0/1129463062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faff415a170 0x7faff40cef70 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:36.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.303+0000 7faffa2ea700 1 --2- 172.21.15.99:0/1129463062 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faff4069040 0x7faff41570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:36.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.303+0000 7faffa2ea700 1 --2- 172.21.15.99:0/1129463062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faff4159da0 0x7faff415c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:36.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.303+0000 7faffa2ea700 1 -- 172.21.15.99:0/1129463062 >> 172.21.15.99:0/1129463062 conn(0x7faff40d0bc0 msgr2=0x7faff4158f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:36.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.303+0000 7faffa2ea700 1 -- 172.21.15.99:0/1129463062 shutdown_connections 2023-10-08T00:11:36.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.303+0000 7faffa2ea700 1 -- 172.21.15.99:0/1129463062 wait complete. 2023-10-08T00:11:36.316 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-10-08T00:11:36.430 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.429+0000 7f6d818af700 1 Processor -- start 2023-10-08T00:11:36.430 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.430+0000 7f6d818af700 1 -- start start 2023-10-08T00:11:36.434 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.430+0000 7f6d818af700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d7c15a620 0x7f6d7c15d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:36.434 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.430+0000 7f6d818af700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d7c15cbb0 con 0x7f6d7c15d580 2023-10-08T00:11:36.434 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.430+0000 7f6d818af700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d7c15bfa0 con 0x7f6d7c157870 2023-10-08T00:11:36.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.430+0000 7f6d818af700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7c15bdb0 con 0x7f6d7c15a620 2023-10-08T00:11:36.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.431+0000 7f6d7b7fe700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f6d7c15d580 0x7f6d7c158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:48526/0 (socket says 172.21.15.99:48526) 2023-10-08T00:11:36.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.431+0000 7f6d7b7fe700 1 -- 172.21.15.99:0/2851288711 learned_addr learned my addr 172.21.15.99:0/2851288711 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:36.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.431+0000 7f6d7a7fc700 1 -- 172.21.15.99:0/2851288711 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1733919304 0 0) 0x7f6d7c15cbb0 con 0x7f6d7c15d580 2023-10-08T00:11:36.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.431+0000 7f6d727fc700 1 --2- 172.21.15.99:0/2851288711 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d7c15a620 0x7f6d7c15d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:36.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.431+0000 7f6d7a7fc700 1 -- 172.21.15.99:0/2851288711 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d68003570 con 0x7f6d7c15d580 2023-10-08T00:11:36.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.431+0000 7f6d7a7fc700 1 -- 172.21.15.99:0/2851288711 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3955967950 0 0) 0x7f6d68003570 con 0x7f6d7c15d580 2023-10-08T00:11:36.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.431+0000 7f6d7a7fc700 1 -- 172.21.15.99:0/2851288711 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d7c15a620 msgr2=0x7f6d7c15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:36.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.431+0000 7f6d7a7fc700 1 --2- 172.21.15.99:0/2851288711 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d7c15a620 0x7f6d7c15d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:36.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.431+0000 7f6d7a7fc700 1 -- 172.21.15.99:0/2851288711 >> v1:172.21.15.191:6789/0 conn(0x7f6d7c157870 legacy=0x7f6d7c15b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:36.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.431+0000 7f6d7a7fc700 1 -- 172.21.15.99:0/2851288711 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d7c15c280 con 0x7f6d7c15d580 2023-10-08T00:11:36.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.431+0000 7f6d7a7fc700 1 -- 172.21.15.99:0/2851288711 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f6d740017f0 con 0x7f6d7c15d580 2023-10-08T00:11:36.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.431+0000 7f6d7a7fc700 1 -- 172.21.15.99:0/2851288711 >> v1:172.21.15.99:6789/0 conn(0x7f6d7c15d580 legacy=0x7f6d7c158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:36.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.431+0000 7f6d7a7fc700 1 --2- 172.21.15.99:0/2851288711 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d68003a90 0x7f6d68003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:36.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.431+0000 7f6d7a7fc700 1 --2- 172.21.15.99:0/2851288711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d68004810 0x7f6d68004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:36.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.431+0000 7f6d7a7fc700 1 --2- 172.21.15.99:0/2851288711 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d7c15a620 0x7f6d680051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:36.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.431+0000 7f6d7a7fc700 1 -- 172.21.15.99:0/2851288711 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6800bb60 con 0x7f6d68004810 2023-10-08T00:11:36.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.431+0000 7f6d7a7fc700 1 -- 172.21.15.99:0/2851288711 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6800bcd0 con 0x7f6d68003a90 2023-10-08T00:11:36.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.431+0000 7f6d7a7fc700 1 -- 172.21.15.99:0/2851288711 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6800be90 con 0x7f6d7c15a620 2023-10-08T00:11:36.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.431+0000 7f6d727fc700 1 --2- 172.21.15.99:0/2851288711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d68004810 0x7f6d68004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:36.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.431+0000 7f6d7affd700 1 --2- 172.21.15.99:0/2851288711 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d68003a90 0x7f6d68003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:36.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.431+0000 7f6d7b7fe700 1 --2- 172.21.15.99:0/2851288711 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d7c15a620 0x7f6d680051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:36.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.431+0000 7f6d727fc700 1 -- 172.21.15.99:0/2851288711 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d7c15a620 msgr2=0x7f6d680051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:36.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.431+0000 7f6d727fc700 1 --2- 172.21.15.99:0/2851288711 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d7c15a620 0x7f6d680051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:36.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.432+0000 7f6d727fc700 1 -- 172.21.15.99:0/2851288711 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d68003a90 msgr2=0x7f6d68003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:36.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.432+0000 7f6d727fc700 1 --2- 172.21.15.99:0/2851288711 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d68003a90 0x7f6d68003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:36.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.432+0000 7f6d727fc700 1 -- 172.21.15.99:0/2851288711 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d68003570 con 0x7f6d68004810 2023-10-08T00:11:36.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.432+0000 7f6d727fc700 1 --2- 172.21.15.99:0/2851288711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d68004810 0x7f6d68004c80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c15aa20 tx=0x7f6d640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4215f43ed655dff3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:36.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.432+0000 7f6d7a7fc700 1 -- 172.21.15.99:0/2851288711 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6400e710 con 0x7f6d68004810 2023-10-08T00:11:36.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.432+0000 7f6d7a7fc700 1 -- 172.21.15.99:0/2851288711 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d6400e870 con 0x7f6d68004810 2023-10-08T00:11:36.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.432+0000 7f6d7a7fc700 1 -- 172.21.15.99:0/2851288711 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6400ec30 con 0x7f6d68004810 2023-10-08T00:11:36.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.432+0000 7f6d818af700 1 -- 172.21.15.99:0/2851288711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d68004810 msgr2=0x7f6d68004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:36.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.432+0000 7f6d818af700 1 --2- 172.21.15.99:0/2851288711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d68004810 0x7f6d68004c80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c15aa20 tx=0x7f6d640091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:36.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.432+0000 7f6d7b7fe700 1 -- 172.21.15.99:0/2851288711 reap_dead start 2023-10-08T00:11:36.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.432+0000 7f6d818af700 1 -- 172.21.15.99:0/2851288711 shutdown_connections 2023-10-08T00:11:36.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.432+0000 7f6d818af700 1 -- 172.21.15.99:0/2851288711 >> 172.21.15.99:0/2851288711 conn(0x7f6d7c068ce0 msgr2=0x7f6d7c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:36.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.432+0000 7f6d818af700 1 -- 172.21.15.99:0/2851288711 shutdown_connections 2023-10-08T00:11:36.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.432+0000 7f6d818af700 1 -- 172.21.15.99:0/2851288711 wait complete. 2023-10-08T00:11:36.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.434+0000 7f6d818af700 1 Processor -- start 2023-10-08T00:11:36.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.434+0000 7f6d818af700 1 -- start start 2023-10-08T00:11:36.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.434+0000 7f6d818af700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d68004810 0x7f6d7c153110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:36.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.434+0000 7f6d818af700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d7c0b1480 0x7f6d7c0b2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:36.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.434+0000 7f6d818af700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d7c14fdb0 0x7f6d7c0b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:36.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.434+0000 7f6d818af700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7c16ece0 con 0x7f6d7c0b1480 2023-10-08T00:11:36.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.434+0000 7f6d818af700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7c16ddf0 con 0x7f6d68004810 2023-10-08T00:11:36.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.434+0000 7f6d818af700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7c16e0d0 con 0x7f6d7c14fdb0 2023-10-08T00:11:36.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.434+0000 7f6d727fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d7c0b1480 0x7f6d7c0b2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:36.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.435+0000 7f6d727fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d7c0b1480 0x7f6d7c0b2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39804/0 (socket says 172.21.15.99:39804) 2023-10-08T00:11:36.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.435+0000 7f6d727fc700 1 -- 172.21.15.99:0/1888497159 learned_addr learned my addr 172.21.15.99:0/1888497159 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:36.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.435+0000 7f6d7b7fe700 1 --2- 172.21.15.99:0/1888497159 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d7c14fdb0 0x7f6d7c0b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:36.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.435+0000 7f6d7affd700 1 --2- 172.21.15.99:0/1888497159 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d68004810 0x7f6d7c153110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:36.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.435+0000 7f6d727fc700 1 -- 172.21.15.99:0/1888497159 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d7c14fdb0 msgr2=0x7f6d7c0b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:36.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.435+0000 7f6d727fc700 1 --2- 172.21.15.99:0/1888497159 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d7c14fdb0 0x7f6d7c0b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:36.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.435+0000 7f6d727fc700 1 -- 172.21.15.99:0/1888497159 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d68004810 msgr2=0x7f6d7c153110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:36.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.435+0000 7f6d727fc700 1 --2- 172.21.15.99:0/1888497159 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d68004810 0x7f6d7c153110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:36.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.435+0000 7f6d727fc700 1 -- 172.21.15.99:0/1888497159 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d7c16e3b0 con 0x7f6d7c0b1480 2023-10-08T00:11:36.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.435+0000 7f6d727fc700 1 --2- 172.21.15.99:0/1888497159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d7c0b1480 0x7f6d7c0b2c60 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6d6400e4b0 tx=0x7f6d6401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:36.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.435+0000 7f6d737fe700 1 -- 172.21.15.99:0/1888497159 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d64023070 con 0x7f6d7c0b1480 2023-10-08T00:11:36.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.435+0000 7f6d737fe700 1 -- 172.21.15.99:0/1888497159 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d6400e9e0 con 0x7f6d7c0b1480 2023-10-08T00:11:36.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.435+0000 7f6d818af700 1 -- 172.21.15.99:0/1888497159 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d7c16e690 con 0x7f6d7c0b1480 2023-10-08T00:11:36.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.435+0000 7f6d737fe700 1 -- 172.21.15.99:0/1888497159 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6401e030 con 0x7f6d7c0b1480 2023-10-08T00:11:36.443 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.435+0000 7f6d818af700 1 -- 172.21.15.99:0/1888497159 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d7c153650 con 0x7f6d7c0b1480 2023-10-08T00:11:36.443 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.436+0000 7f6d818af700 1 -- 172.21.15.99:0/1888497159 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d7c04c3c0 con 0x7f6d7c0b1480 2023-10-08T00:11:36.443 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.436+0000 7f6d737fe700 1 -- 172.21.15.99:0/1888497159 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6d64013470 con 0x7f6d7c0b1480 2023-10-08T00:11:36.443 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.437+0000 7f6d737fe700 1 --2- 172.21.15.99:0/1888497159 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6d5c066f30 0x7f6d5c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:36.443 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.437+0000 7f6d737fe700 1 -- 172.21.15.99:0/1888497159 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f6d640859d0 con 0x7f6d7c0b1480 2023-10-08T00:11:36.443 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.437+0000 7f6d7affd700 1 --2- 172.21.15.99:0/1888497159 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6d5c066f30 0x7f6d5c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:36.443 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.438+0000 7f6d7affd700 1 --2- 172.21.15.99:0/1888497159 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6d5c066f30 0x7f6d5c0693e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c000f60 tx=0x7f6d6c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:36.444 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.441+0000 7f6d737fe700 1 -- 172.21.15.99:0/1888497159 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6d64052f90 con 0x7f6d7c0b1480 2023-10-08T00:11:36.699 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.698+0000 7f6d818af700 1 -- 172.21.15.99:0/1888497159 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f6d7c15c560 con 0x7f6d7c0b1480 2023-10-08T00:11:36.702 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.701+0000 7f6d737fe700 1 -- 172.21.15.99:0/1888497159 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6d640170a0 con 0x7f6d7c0b1480 2023-10-08T00:11:36.704 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.703+0000 7f6d818af700 1 -- 172.21.15.99:0/1888497159 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6d5c066f30 msgr2=0x7f6d5c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:36.704 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.703+0000 7f6d818af700 1 --2- 172.21.15.99:0/1888497159 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6d5c066f30 0x7f6d5c0693e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c000f60 tx=0x7f6d6c009040 comp rx=0 tx=0).stop 2023-10-08T00:11:36.705 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.703+0000 7f6d818af700 1 -- 172.21.15.99:0/1888497159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d7c0b1480 msgr2=0x7f6d7c0b2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:36.705 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.703+0000 7f6d818af700 1 --2- 172.21.15.99:0/1888497159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d7c0b1480 0x7f6d7c0b2c60 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6d6400e4b0 tx=0x7f6d6401a000 comp rx=0 tx=0).stop 2023-10-08T00:11:36.705 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.703+0000 7f6d818af700 1 -- 172.21.15.99:0/1888497159 shutdown_connections 2023-10-08T00:11:36.705 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.703+0000 7f6d818af700 1 --2- 172.21.15.99:0/1888497159 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6d5c066f30 0x7f6d5c0693e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:36.705 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.703+0000 7f6d818af700 1 --2- 172.21.15.99:0/1888497159 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d68004810 0x7f6d7c153110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:36.705 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.703+0000 7f6d818af700 1 --2- 172.21.15.99:0/1888497159 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d7c14fdb0 0x7f6d7c0b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:36.705 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.703+0000 7f6d818af700 1 --2- 172.21.15.99:0/1888497159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d7c0b1480 0x7f6d7c0b2c60 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:36.705 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.703+0000 7f6d818af700 1 -- 172.21.15.99:0/1888497159 >> 172.21.15.99:0/1888497159 conn(0x7f6d7c068ce0 msgr2=0x7f6d7c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:36.705 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.703+0000 7f6d818af700 1 -- 172.21.15.99:0/1888497159 shutdown_connections 2023-10-08T00:11:36.706 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.703+0000 7f6d818af700 1 -- 172.21.15.99:0/1888497159 wait complete. 2023-10-08T00:11:36.716 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-10-08T00:11:36.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.830+0000 7f007f87a700 1 Processor -- start 2023-10-08T00:11:36.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.831+0000 7f007f87a700 1 -- start start 2023-10-08T00:11:36.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.831+0000 7f007f87a700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f0078157870 0x7f007815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:36.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.831+0000 7f007f87a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f007815a620 0x7f007815d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:36.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.831+0000 7f007f87a700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f007815d580 0x7f0078158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:36.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.831+0000 7f007f87a700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f007815cbb0 con 0x7f007815d580 2023-10-08T00:11:36.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.831+0000 7f007f87a700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f007815bfa0 con 0x7f0078157870 2023-10-08T00:11:36.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.831+0000 7f007f87a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f007815c280 con 0x7f007815a620 2023-10-08T00:11:36.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.831+0000 7f007de17700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f007815d580 0x7f0078158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:36.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.832+0000 7f007de17700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f007815d580 0x7f0078158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39812/0 (socket says 172.21.15.99:39812) 2023-10-08T00:11:36.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.832+0000 7f007de17700 1 -- 172.21.15.99:0/3239857273 learned_addr learned my addr 172.21.15.99:0/3239857273 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:36.835 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.832+0000 7f007ce15700 1 --2- 172.21.15.99:0/3239857273 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f007815a620 0x7f007815d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:36.835 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.832+0000 7f007d616700 1 --2- 172.21.15.99:0/3239857273 >> v2:172.21.15.191:3300/0 conn(0x7f0078157870 0x7f007815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:36.835 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.832+0000 7f007de17700 1 -- 172.21.15.99:0/3239857273 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f007815a620 msgr2=0x7f007815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:36.835 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.832+0000 7f007de17700 1 --2- 172.21.15.99:0/3239857273 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f007815a620 0x7f007815d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:36.835 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.832+0000 7f007de17700 1 -- 172.21.15.99:0/3239857273 >> v2:172.21.15.191:3300/0 conn(0x7f0078157870 msgr2=0x7f007815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:36.835 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.832+0000 7f007de17700 1 --2- 172.21.15.99:0/3239857273 >> v2:172.21.15.191:3300/0 conn(0x7f0078157870 0x7f007815b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:36.835 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.832+0000 7f007de17700 1 -- 172.21.15.99:0/3239857273 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f007815c560 con 0x7f007815d580 2023-10-08T00:11:36.835 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.832+0000 7f007ce15700 1 --2- 172.21.15.99:0/3239857273 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f007815a620 0x7f007815d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:11:36.835 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.832+0000 7f007d616700 1 --2- 172.21.15.99:0/3239857273 >> v2:172.21.15.191:3300/0 conn(0x7f0078157870 0x7f007815b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:11:36.835 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.832+0000 7f007de17700 1 --2- 172.21.15.99:0/3239857273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f007815d580 0x7f0078158110 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0078069d80 tx=0x7f0070009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da100ba33e9b77ee server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:36.835 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.832+0000 7f006ffff700 1 -- 172.21.15.99:0/3239857273 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007000b860 con 0x7f007815d580 2023-10-08T00:11:36.835 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.832+0000 7f006ffff700 1 -- 172.21.15.99:0/3239857273 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f007000b9c0 con 0x7f007815d580 2023-10-08T00:11:36.836 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.832+0000 7f006ffff700 1 -- 172.21.15.99:0/3239857273 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007000bd60 con 0x7f007815d580 2023-10-08T00:11:36.836 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.833+0000 7f007f87a700 1 -- 172.21.15.99:0/3239857273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f007815d580 msgr2=0x7f0078158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:36.836 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.833+0000 7f007f87a700 1 --2- 172.21.15.99:0/3239857273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f007815d580 0x7f0078158110 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0078069d80 tx=0x7f0070009250 comp rx=0 tx=0).stop 2023-10-08T00:11:36.836 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.833+0000 7f007f87a700 1 -- 172.21.15.99:0/3239857273 shutdown_connections 2023-10-08T00:11:36.836 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.833+0000 7f007f87a700 1 --2- 172.21.15.99:0/3239857273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f007815d580 0x7f0078158110 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:36.836 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.833+0000 7f007f87a700 1 --2- 172.21.15.99:0/3239857273 >> v2:172.21.15.191:3300/0 conn(0x7f0078157870 0x7f007815b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:36.836 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.833+0000 7f007f87a700 1 --2- 172.21.15.99:0/3239857273 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f007815a620 0x7f007815d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:36.836 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.833+0000 7f007f87a700 1 -- 172.21.15.99:0/3239857273 >> 172.21.15.99:0/3239857273 conn(0x7f0078068ce0 msgr2=0x7f00780690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:36.836 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.833+0000 7f007f87a700 1 -- 172.21.15.99:0/3239857273 shutdown_connections 2023-10-08T00:11:36.836 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.833+0000 7f007f87a700 1 -- 172.21.15.99:0/3239857273 wait complete. 2023-10-08T00:11:36.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.834+0000 7f007f87a700 1 Processor -- start 2023-10-08T00:11:36.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.834+0000 7f007f87a700 1 -- start start 2023-10-08T00:11:36.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.835+0000 7f007f87a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0078157870 0x7f00781568c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:36.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.835+0000 7f007f87a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f007815a620 0x7f00780b10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:36.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.835+0000 7f007f87a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f007815d580 0x7f007814b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:36.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.835+0000 7f007f87a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00780b24c0 con 0x7f007815a620 2023-10-08T00:11:36.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.835+0000 7f007f87a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00780b1610 con 0x7f0078157870 2023-10-08T00:11:36.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.835+0000 7f007f87a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f007814bdd0 con 0x7f007815d580 2023-10-08T00:11:36.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.835+0000 7f007ce15700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f007815a620 0x7f00780b10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:36.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.835+0000 7f007ce15700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f007815a620 0x7f00780b10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39816/0 (socket says 172.21.15.99:39816) 2023-10-08T00:11:36.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.835+0000 7f007ce15700 1 -- 172.21.15.99:0/2833450451 learned_addr learned my addr 172.21.15.99:0/2833450451 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:36.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.835+0000 7f007d616700 1 --2- 172.21.15.99:0/2833450451 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0078157870 0x7f00781568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:36.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.835+0000 7f007de17700 1 --2- 172.21.15.99:0/2833450451 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f007815d580 0x7f007814b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:36.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.835+0000 7f007ce15700 1 -- 172.21.15.99:0/2833450451 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f007815d580 msgr2=0x7f007814b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:36.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.835+0000 7f007ce15700 1 --2- 172.21.15.99:0/2833450451 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f007815d580 0x7f007814b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:36.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.835+0000 7f007ce15700 1 -- 172.21.15.99:0/2833450451 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0078157870 msgr2=0x7f00781568c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:36.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.835+0000 7f007ce15700 1 --2- 172.21.15.99:0/2833450451 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0078157870 0x7f00781568c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:36.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.835+0000 7f007ce15700 1 -- 172.21.15.99:0/2833450451 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00700094f0 con 0x7f007815a620 2023-10-08T00:11:36.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.835+0000 7f007d616700 1 --2- 172.21.15.99:0/2833450451 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0078157870 0x7f00781568c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:11:36.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.835+0000 7f007de17700 1 --2- 172.21.15.99:0/2833450451 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f007815d580 0x7f007814b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:11:36.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.835+0000 7f007ce15700 1 --2- 172.21.15.99:0/2833450451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f007815a620 0x7f00780b10d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f006800c880 tx=0x7f0068008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:36.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.835+0000 7f006d7fa700 1 -- 172.21.15.99:0/2833450451 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0068011600 con 0x7f007815a620 2023-10-08T00:11:36.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.835+0000 7f006d7fa700 1 -- 172.21.15.99:0/2833450451 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0068011760 con 0x7f007815a620 2023-10-08T00:11:36.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.835+0000 7f006d7fa700 1 -- 172.21.15.99:0/2833450451 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0068011ae0 con 0x7f007815a620 2023-10-08T00:11:36.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.835+0000 7f007f87a700 1 -- 172.21.15.99:0/2833450451 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00780b18b0 con 0x7f007815a620 2023-10-08T00:11:36.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.836+0000 7f007f87a700 1 -- 172.21.15.99:0/2833450451 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00780b1e70 con 0x7f007815a620 2023-10-08T00:11:36.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.836+0000 7f007f87a700 1 -- 172.21.15.99:0/2833450451 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f007804c3c0 con 0x7f007815a620 2023-10-08T00:11:36.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.837+0000 7f006d7fa700 1 -- 172.21.15.99:0/2833450451 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f00680073d0 con 0x7f007815a620 2023-10-08T00:11:36.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.837+0000 7f006d7fa700 1 --2- 172.21.15.99:0/2833450451 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f005c066f30 0x7f005c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:36.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.837+0000 7f006d7fa700 1 -- 172.21.15.99:0/2833450451 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f00680842a0 con 0x7f007815a620 2023-10-08T00:11:36.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.837+0000 7f007d616700 1 --2- 172.21.15.99:0/2833450451 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f005c066f30 0x7f005c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:36.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.838+0000 7f007d616700 1 --2- 172.21.15.99:0/2833450451 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f005c066f30 0x7f005c0693e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f007400d510 tx=0x7f00740075e0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:36.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:36.841+0000 7f006d7fa700 1 -- 172.21.15.99:0/2833450451 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0068051860 con 0x7f007815a620 2023-10-08T00:11:37.098 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.098+0000 7f007f87a700 1 -- 172.21.15.99:0/2833450451 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f00780b3c30 con 0x7f007815a620 2023-10-08T00:11:37.102 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.101+0000 7f006d7fa700 1 -- 172.21.15.99:0/2833450451 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f0068054e80 con 0x7f007815a620 2023-10-08T00:11:37.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.103+0000 7f007f87a700 1 -- 172.21.15.99:0/2833450451 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f005c066f30 msgr2=0x7f005c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:37.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.103+0000 7f007f87a700 1 --2- 172.21.15.99:0/2833450451 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f005c066f30 0x7f005c0693e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f007400d510 tx=0x7f00740075e0 comp rx=0 tx=0).stop 2023-10-08T00:11:37.105 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.103+0000 7f007f87a700 1 -- 172.21.15.99:0/2833450451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f007815a620 msgr2=0x7f00780b10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:37.105 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.103+0000 7f007f87a700 1 --2- 172.21.15.99:0/2833450451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f007815a620 0x7f00780b10d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f006800c880 tx=0x7f0068008040 comp rx=0 tx=0).stop 2023-10-08T00:11:37.105 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.103+0000 7f007f87a700 1 -- 172.21.15.99:0/2833450451 shutdown_connections 2023-10-08T00:11:37.105 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.103+0000 7f007f87a700 1 --2- 172.21.15.99:0/2833450451 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f005c066f30 0x7f005c0693e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:37.105 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.104+0000 7f007f87a700 1 --2- 172.21.15.99:0/2833450451 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0078157870 0x7f00781568c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:37.105 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.104+0000 7f007f87a700 1 --2- 172.21.15.99:0/2833450451 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f007815d580 0x7f007814b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:37.105 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.104+0000 7f007f87a700 1 --2- 172.21.15.99:0/2833450451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f007815a620 0x7f00780b10d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:37.105 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.104+0000 7f007f87a700 1 -- 172.21.15.99:0/2833450451 >> 172.21.15.99:0/2833450451 conn(0x7f0078068ce0 msgr2=0x7f00780b4f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:37.106 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.104+0000 7f007f87a700 1 -- 172.21.15.99:0/2833450451 shutdown_connections 2023-10-08T00:11:37.106 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.104+0000 7f007f87a700 1 -- 172.21.15.99:0/2833450451 wait complete. 2023-10-08T00:11:37.116 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:11:37.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.229+0000 7f92c9093700 1 Processor -- start 2023-10-08T00:11:37.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.229+0000 7f92c9093700 1 -- start start 2023-10-08T00:11:37.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.230+0000 7f92c9093700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f92c415a620 0x7f92c415d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:37.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.230+0000 7f92c9093700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92c415cbb0 con 0x7f92c415d580 2023-10-08T00:11:37.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.230+0000 7f92c9093700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92c415bfa0 con 0x7f92c4157870 2023-10-08T00:11:37.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.230+0000 7f92c9093700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92c415bdb0 con 0x7f92c415a620 2023-10-08T00:11:37.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.230+0000 7f92c359e700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f92c415d580 0x7f92c4158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:48542/0 (socket says 172.21.15.99:48542) 2023-10-08T00:11:37.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.230+0000 7f92c359e700 1 -- 172.21.15.99:0/328096780 learned_addr learned my addr 172.21.15.99:0/328096780 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:37.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.230+0000 7f92c259c700 1 --2- 172.21.15.99:0/328096780 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f92c415a620 0x7f92c415d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:37.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.230+0000 7f92c1d9b700 1 -- 172.21.15.99:0/328096780 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2141970278 0 0) 0x7f92c415cbb0 con 0x7f92c415d580 2023-10-08T00:11:37.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.230+0000 7f92c1d9b700 1 -- 172.21.15.99:0/328096780 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92ac003570 con 0x7f92c415d580 2023-10-08T00:11:37.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.230+0000 7f92c1d9b700 1 -- 172.21.15.99:0/328096780 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 313247515 0 0) 0x7f92c415bfa0 con 0x7f92c4157870 2023-10-08T00:11:37.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.230+0000 7f92c1d9b700 1 -- 172.21.15.99:0/328096780 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92c415cbb0 con 0x7f92c4157870 2023-10-08T00:11:37.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.230+0000 7f92c1d9b700 1 -- 172.21.15.99:0/328096780 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1948766298 0 0) 0x7f92ac003570 con 0x7f92c415d580 2023-10-08T00:11:37.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.230+0000 7f92c1d9b700 1 -- 172.21.15.99:0/328096780 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f92c415a620 msgr2=0x7f92c415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:37.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.230+0000 7f92c1d9b700 1 --2- 172.21.15.99:0/328096780 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f92c415a620 0x7f92c415d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:37.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.230+0000 7f92c1d9b700 1 -- 172.21.15.99:0/328096780 >> v1:172.21.15.191:6789/0 conn(0x7f92c4157870 legacy=0x7f92c415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:37.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.230+0000 7f92c1d9b700 1 -- 172.21.15.99:0/328096780 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c415c280 con 0x7f92c415d580 2023-10-08T00:11:37.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.230+0000 7f92c1d9b700 1 -- 172.21.15.99:0/328096780 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f92bc0017f0 con 0x7f92c415d580 2023-10-08T00:11:37.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.230+0000 7f92c1d9b700 1 -- 172.21.15.99:0/328096780 >> v1:172.21.15.99:6789/0 conn(0x7f92c415d580 legacy=0x7f92c4158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:37.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.230+0000 7f92c1d9b700 1 --2- 172.21.15.99:0/328096780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92ac003c20 0x7f92ac004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:37.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.230+0000 7f92c1d9b700 1 --2- 172.21.15.99:0/328096780 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f92c415a620 0x7f92ac004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:37.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.230+0000 7f92c1d9b700 1 --2- 172.21.15.99:0/328096780 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92ac004d50 0x7f92ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:37.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.230+0000 7f92c1d9b700 1 -- 172.21.15.99:0/328096780 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ac00bb40 con 0x7f92ac003c20 2023-10-08T00:11:37.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.230+0000 7f92c1d9b700 1 -- 172.21.15.99:0/328096780 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ac00bcb0 con 0x7f92ac004d50 2023-10-08T00:11:37.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.230+0000 7f92c1d9b700 1 -- 172.21.15.99:0/328096780 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92ac00be70 con 0x7f92c415a620 2023-10-08T00:11:37.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.231+0000 7f92c2d9d700 1 --2- 172.21.15.99:0/328096780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92ac003c20 0x7f92ac004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:37.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.231+0000 7f92c259c700 1 --2- 172.21.15.99:0/328096780 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f92c415a620 0x7f92ac004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:37.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.231+0000 7f92c359e700 1 --2- 172.21.15.99:0/328096780 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92ac004d50 0x7f92ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:37.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.231+0000 7f92c2d9d700 1 -- 172.21.15.99:0/328096780 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f92c415a620 msgr2=0x7f92ac004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:37.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.231+0000 7f92c2d9d700 1 --2- 172.21.15.99:0/328096780 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f92c415a620 0x7f92ac004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:37.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.231+0000 7f92c2d9d700 1 -- 172.21.15.99:0/328096780 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92ac004d50 msgr2=0x7f92ac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:37.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.231+0000 7f92c2d9d700 1 --2- 172.21.15.99:0/328096780 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92ac004d50 0x7f92ac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:37.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.231+0000 7f92c2d9d700 1 -- 172.21.15.99:0/328096780 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92ac003570 con 0x7f92ac003c20 2023-10-08T00:11:37.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.231+0000 7f92c359e700 1 --2- 172.21.15.99:0/328096780 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92ac004d50 0x7f92ac0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:11:37.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.231+0000 7f92c2d9d700 1 --2- 172.21.15.99:0/328096780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92ac003c20 0x7f92ac004010 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f92b4001a10 tx=0x7f92b4007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e9e70428d73bfd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:37.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.231+0000 7f92c1d9b700 1 -- 172.21.15.99:0/328096780 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b400c5d0 con 0x7f92ac003c20 2023-10-08T00:11:37.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.231+0000 7f92c1d9b700 1 -- 172.21.15.99:0/328096780 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b400c730 con 0x7f92ac003c20 2023-10-08T00:11:37.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.231+0000 7f92c1d9b700 1 -- 172.21.15.99:0/328096780 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b400cad0 con 0x7f92ac003c20 2023-10-08T00:11:37.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.232+0000 7f92c9093700 1 -- 172.21.15.99:0/328096780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92ac003c20 msgr2=0x7f92ac004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:37.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.232+0000 7f92c9093700 1 --2- 172.21.15.99:0/328096780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92ac003c20 0x7f92ac004010 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f92b4001a10 tx=0x7f92b4007d30 comp rx=0 tx=0).stop 2023-10-08T00:11:37.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.232+0000 7f92c359e700 1 -- 172.21.15.99:0/328096780 reap_dead start 2023-10-08T00:11:37.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.232+0000 7f92c9093700 1 -- 172.21.15.99:0/328096780 shutdown_connections 2023-10-08T00:11:37.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.232+0000 7f92c9093700 1 -- 172.21.15.99:0/328096780 >> 172.21.15.99:0/328096780 conn(0x7f92c4068ce0 msgr2=0x7f92c40690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:37.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.232+0000 7f92c9093700 1 -- 172.21.15.99:0/328096780 shutdown_connections 2023-10-08T00:11:37.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.232+0000 7f92c9093700 1 -- 172.21.15.99:0/328096780 wait complete. 2023-10-08T00:11:37.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.233+0000 7f92c9093700 1 Processor -- start 2023-10-08T00:11:37.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.233+0000 7f92c9093700 1 -- start start 2023-10-08T00:11:37.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.234+0000 7f92c9093700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f92ac003c20 0x7f92c4153470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:37.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.234+0000 7f92c9093700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92c416b6e0 0x7f92c40a7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:37.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.234+0000 7f92c9093700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92c40a8150 0x7f92c40b2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:37.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.234+0000 7f92c9093700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c41539b0 con 0x7f92c40a8150 2023-10-08T00:11:37.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.234+0000 7f92c9093700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c40b0f80 con 0x7f92c416b6e0 2023-10-08T00:11:37.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.234+0000 7f92c9093700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92c40b1260 con 0x7f92ac003c20 2023-10-08T00:11:37.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.234+0000 7f92c359e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92c40a8150 0x7f92c40b2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:37.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.234+0000 7f92c359e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92c40a8150 0x7f92c40b2020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39830/0 (socket says 172.21.15.99:39830) 2023-10-08T00:11:37.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.234+0000 7f92c359e700 1 -- 172.21.15.99:0/3547304315 learned_addr learned my addr 172.21.15.99:0/3547304315 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:37.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.234+0000 7f92c2d9d700 1 --2- 172.21.15.99:0/3547304315 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f92ac003c20 0x7f92c4153470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:37.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.234+0000 7f92c259c700 1 --2- 172.21.15.99:0/3547304315 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92c416b6e0 0x7f92c40a7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:37.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.234+0000 7f92c359e700 1 -- 172.21.15.99:0/3547304315 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f92ac003c20 msgr2=0x7f92c4153470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:37.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.234+0000 7f92c359e700 1 --2- 172.21.15.99:0/3547304315 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f92ac003c20 0x7f92c4153470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:37.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.234+0000 7f92c359e700 1 -- 172.21.15.99:0/3547304315 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92c416b6e0 msgr2=0x7f92c40a7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:37.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.234+0000 7f92c359e700 1 --2- 172.21.15.99:0/3547304315 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92c416b6e0 0x7f92c40a7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:37.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.234+0000 7f92c359e700 1 -- 172.21.15.99:0/3547304315 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c40b1540 con 0x7f92c40a8150 2023-10-08T00:11:37.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.234+0000 7f92c359e700 1 --2- 172.21.15.99:0/3547304315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92c40a8150 0x7f92c40b2020 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f92c40b3830 tx=0x7f92bc0020e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:37.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.234+0000 7f92aaffd700 1 -- 172.21.15.99:0/3547304315 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92bc00e900 con 0x7f92c40a8150 2023-10-08T00:11:37.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.234+0000 7f92aaffd700 1 -- 172.21.15.99:0/3547304315 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92bc00ea60 con 0x7f92c40a8150 2023-10-08T00:11:37.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.234+0000 7f92c9093700 1 -- 172.21.15.99:0/3547304315 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92c40b1820 con 0x7f92c40a8150 2023-10-08T00:11:37.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.234+0000 7f92aaffd700 1 -- 172.21.15.99:0/3547304315 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92bc00ee20 con 0x7f92c40a8150 2023-10-08T00:11:37.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.234+0000 7f92c9093700 1 -- 172.21.15.99:0/3547304315 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92c40a7760 con 0x7f92c40a8150 2023-10-08T00:11:37.242 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.235+0000 7f92c9093700 1 -- 172.21.15.99:0/3547304315 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92c40b1d40 con 0x7f92c40a8150 2023-10-08T00:11:37.242 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.236+0000 7f92aaffd700 1 -- 172.21.15.99:0/3547304315 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f92bc013630 con 0x7f92c40a8150 2023-10-08T00:11:37.242 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.236+0000 7f92aaffd700 1 --2- 172.21.15.99:0/3547304315 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f92b0066f30 0x7f92b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:37.242 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.236+0000 7f92aaffd700 1 -- 172.21.15.99:0/3547304315 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f92bc084490 con 0x7f92c40a8150 2023-10-08T00:11:37.242 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.236+0000 7f92c2d9d700 1 --2- 172.21.15.99:0/3547304315 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f92b0066f30 0x7f92b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:37.242 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.237+0000 7f92c2d9d700 1 --2- 172.21.15.99:0/3547304315 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f92b0066f30 0x7f92b00693e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f92b4000c00 tx=0x7f92b4015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:37.242 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.240+0000 7f92aaffd700 1 -- 172.21.15.99:0/3547304315 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f92c40b1d40 con 0x7f92c40a8150 2023-10-08T00:11:37.427 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.425+0000 7f92c9093700 1 -- 172.21.15.99:0/3547304315 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f92c40b1d40 con 0x7f92c40a8150 2023-10-08T00:11:37.428 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.426+0000 7f92aaffd700 1 -- 172.21.15.99:0/3547304315 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+12134 (secure 0 0 0) 0x7f92c40b1d40 con 0x7f92c40a8150 2023-10-08T00:11:37.428 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:11:37.429 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":27,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:11:34.283830+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:11:37.429 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.428+0000 7f92c9093700 1 -- 172.21.15.99:0/3547304315 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f92b0066f30 msgr2=0x7f92b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:37.429 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.428+0000 7f92c9093700 1 --2- 172.21.15.99:0/3547304315 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f92b0066f30 0x7f92b00693e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f92b4000c00 tx=0x7f92b4015000 comp rx=0 tx=0).stop 2023-10-08T00:11:37.430 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.429+0000 7f92c9093700 1 -- 172.21.15.99:0/3547304315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92c40a8150 msgr2=0x7f92c40b2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:37.430 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.429+0000 7f92c9093700 1 --2- 172.21.15.99:0/3547304315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92c40a8150 0x7f92c40b2020 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f92c40b3830 tx=0x7f92bc0020e0 comp rx=0 tx=0).stop 2023-10-08T00:11:37.430 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.429+0000 7f92c9093700 1 -- 172.21.15.99:0/3547304315 shutdown_connections 2023-10-08T00:11:37.430 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.429+0000 7f92c9093700 1 --2- 172.21.15.99:0/3547304315 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f92b0066f30 0x7f92b00693e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:37.430 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.429+0000 7f92c9093700 1 --2- 172.21.15.99:0/3547304315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92c40a8150 0x7f92c40b2020 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:37.430 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.429+0000 7f92c9093700 1 --2- 172.21.15.99:0/3547304315 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f92ac003c20 0x7f92c4153470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:37.430 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.429+0000 7f92c9093700 1 --2- 172.21.15.99:0/3547304315 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92c416b6e0 0x7f92c40a7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:37.430 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.429+0000 7f92c9093700 1 -- 172.21.15.99:0/3547304315 >> 172.21.15.99:0/3547304315 conn(0x7f92c4068ce0 msgr2=0x7f92c415aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:37.430 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.429+0000 7f92c9093700 1 -- 172.21.15.99:0/3547304315 shutdown_connections 2023-10-08T00:11:37.430 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.429+0000 7f92c9093700 1 -- 172.21.15.99:0/3547304315 wait complete. 2023-10-08T00:11:37.441 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:10:42.063344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-08T00:11:37.442 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T00:11:37.559 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.558+0000 7fbb4275f700 1 Processor -- start 2023-10-08T00:11:37.559 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.558+0000 7fbb4275f700 1 -- start start 2023-10-08T00:11:37.561 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.559+0000 7fbb4275f700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fbb3c0cce30 0x7fbb3c15aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:37.562 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.559+0000 7fbb4275f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb3c0b6580 0x7fbb3c06a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:37.562 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.559+0000 7fbb4275f700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb3c0b4d70 con 0x7fbb3c06a830 2023-10-08T00:11:37.562 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.559+0000 7fbb4275f700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb3c06ac70 con 0x7fbb3c0cce30 2023-10-08T00:11:37.562 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.559+0000 7fbb4275f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3c0cd8d0 con 0x7fbb3c0b6580 2023-10-08T00:11:37.562 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.559+0000 7fbb4175d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fbb3c0cce30 0x7fbb3c15aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:37.562 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.559+0000 7fbb41f5e700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7fbb3c06a830 0x7fbb3c15d880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:37038/0 (socket says 172.21.15.99:37038) 2023-10-08T00:11:37.562 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.559+0000 7fbb41f5e700 1 -- 172.21.15.99:0/1364376354 learned_addr learned my addr 172.21.15.99:0/1364376354 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:37.562 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.559+0000 7fbb40f5c700 1 --2- 172.21.15.99:0/1364376354 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb3c0b6580 0x7fbb3c06a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:37.563 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.560+0000 7fbb4175d700 1 -- 172.21.15.99:0/1364376354 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb3c0b6580 msgr2=0x7fbb3c06a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:37.563 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.560+0000 7fbb4175d700 1 --2- 172.21.15.99:0/1364376354 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb3c0b6580 0x7fbb3c06a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:37.563 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.560+0000 7fbb4175d700 1 -- 172.21.15.99:0/1364376354 >> v1:172.21.15.191:6789/0 conn(0x7fbb3c06a830 legacy=0x7fbb3c15d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:37.563 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.560+0000 7fbb4175d700 1 -- 172.21.15.99:0/1364376354 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb3c0b4440 con 0x7fbb3c0cce30 2023-10-08T00:11:37.563 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.560+0000 7fbb40f5c700 1 --2- 172.21.15.99:0/1364376354 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb3c0b6580 0x7fbb3c06a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:11:37.563 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.560+0000 7fbb4175d700 1 --2- 172.21.15.99:0/1364376354 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb3c0cce30 0x7fbb3c15aca0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbb38009e60 tx=0x7fbb38009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59cf0a7e683926c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:37.563 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.560+0000 7fbb33fff700 1 -- 172.21.15.99:0/1364376354 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3800e070 con 0x7fbb3c0cce30 2023-10-08T00:11:37.563 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.560+0000 7fbb33fff700 1 -- 172.21.15.99:0/1364376354 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb3800bb60 con 0x7fbb3c0cce30 2023-10-08T00:11:37.563 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.560+0000 7fbb33fff700 1 -- 172.21.15.99:0/1364376354 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb38012430 con 0x7fbb3c0cce30 2023-10-08T00:11:37.564 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.561+0000 7fbb4275f700 1 -- 172.21.15.99:0/1364376354 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb3c0cce30 msgr2=0x7fbb3c15aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:37.564 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.561+0000 7fbb4275f700 1 --2- 172.21.15.99:0/1364376354 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb3c0cce30 0x7fbb3c15aca0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbb38009e60 tx=0x7fbb38009780 comp rx=0 tx=0).stop 2023-10-08T00:11:37.564 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.561+0000 7fbb4275f700 1 -- 172.21.15.99:0/1364376354 shutdown_connections 2023-10-08T00:11:37.564 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.561+0000 7fbb4275f700 1 --2- 172.21.15.99:0/1364376354 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb3c0cce30 0x7fbb3c15aca0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:37.564 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.561+0000 7fbb4275f700 1 --2- 172.21.15.99:0/1364376354 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb3c0b6580 0x7fbb3c06a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:37.564 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.561+0000 7fbb4275f700 1 -- 172.21.15.99:0/1364376354 >> 172.21.15.99:0/1364376354 conn(0x7fbb3c068cb0 msgr2=0x7fbb3c0690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:37.564 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.561+0000 7fbb4275f700 1 -- 172.21.15.99:0/1364376354 shutdown_connections 2023-10-08T00:11:37.564 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.561+0000 7fbb4275f700 1 -- 172.21.15.99:0/1364376354 wait complete. 2023-10-08T00:11:37.567 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.562+0000 7fbb4275f700 1 Processor -- start 2023-10-08T00:11:37.567 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.562+0000 7fbb4275f700 1 -- start start 2023-10-08T00:11:37.567 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.563+0000 7fbb4275f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb3c06a830 0x7fbb3c16d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:37.567 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.563+0000 7fbb4275f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb3c0b6580 0x7fbb3c16df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:37.567 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.563+0000 7fbb4275f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb3c0cce30 0x7fbb3c0b1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:37.568 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.563+0000 7fbb4275f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3c0b2d20 con 0x7fbb3c0b6580 2023-10-08T00:11:37.568 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.563+0000 7fbb4275f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3c16d740 con 0x7fbb3c06a830 2023-10-08T00:11:37.568 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.563+0000 7fbb4275f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3c0b2110 con 0x7fbb3c0cce30 2023-10-08T00:11:37.568 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.563+0000 7fbb40f5c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb3c0b6580 0x7fbb3c16df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:37.568 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.563+0000 7fbb40f5c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb3c0b6580 0x7fbb3c16df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39834/0 (socket says 172.21.15.99:39834) 2023-10-08T00:11:37.568 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.563+0000 7fbb40f5c700 1 -- 172.21.15.99:0/628436924 learned_addr learned my addr 172.21.15.99:0/628436924 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:37.568 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.563+0000 7fbb4175d700 1 --2- 172.21.15.99:0/628436924 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb3c06a830 0x7fbb3c16d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:37.568 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.563+0000 7fbb41f5e700 1 --2- 172.21.15.99:0/628436924 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb3c0cce30 0x7fbb3c0b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:37.568 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.563+0000 7fbb40f5c700 1 -- 172.21.15.99:0/628436924 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb3c0cce30 msgr2=0x7fbb3c0b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:37.568 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.563+0000 7fbb40f5c700 1 --2- 172.21.15.99:0/628436924 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb3c0cce30 0x7fbb3c0b1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:37.568 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.563+0000 7fbb40f5c700 1 -- 172.21.15.99:0/628436924 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb3c06a830 msgr2=0x7fbb3c16d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:37.568 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.563+0000 7fbb40f5c700 1 --2- 172.21.15.99:0/628436924 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb3c06a830 0x7fbb3c16d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:37.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.563+0000 7fbb40f5c700 1 -- 172.21.15.99:0/628436924 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb2c009420 con 0x7fbb3c0b6580 2023-10-08T00:11:37.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.563+0000 7fbb40f5c700 1 --2- 172.21.15.99:0/628436924 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb3c0b6580 0x7fbb3c16df00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c00ec80 tx=0x7fbb2c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:37.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.563+0000 7fbb317fa700 1 -- 172.21.15.99:0/628436924 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c00c9d0 con 0x7fbb3c0b6580 2023-10-08T00:11:37.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.563+0000 7fbb317fa700 1 -- 172.21.15.99:0/628436924 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb2c00cb30 con 0x7fbb3c0b6580 2023-10-08T00:11:37.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.563+0000 7fbb4275f700 1 -- 172.21.15.99:0/628436924 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb38009a20 con 0x7fbb3c0b6580 2023-10-08T00:11:37.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.563+0000 7fbb317fa700 1 -- 172.21.15.99:0/628436924 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c013470 con 0x7fbb3c0b6580 2023-10-08T00:11:37.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.563+0000 7fbb4275f700 1 -- 172.21.15.99:0/628436924 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb3c0b26d0 con 0x7fbb3c0b6580 2023-10-08T00:11:37.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.564+0000 7fbb4275f700 1 -- 172.21.15.99:0/628436924 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb3c089550 con 0x7fbb3c0b6580 2023-10-08T00:11:37.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.565+0000 7fbb317fa700 1 -- 172.21.15.99:0/628436924 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbb2c013820 con 0x7fbb3c0b6580 2023-10-08T00:11:37.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.565+0000 7fbb317fa700 1 --2- 172.21.15.99:0/628436924 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbb20066f30 0x7fbb200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:37.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.565+0000 7fbb317fa700 1 -- 172.21.15.99:0/628436924 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fbb2c0843d0 con 0x7fbb3c0b6580 2023-10-08T00:11:37.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.565+0000 7fbb4175d700 1 --2- 172.21.15.99:0/628436924 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbb20066f30 0x7fbb200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:37.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.566+0000 7fbb4175d700 1 --2- 172.21.15.99:0/628436924 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbb20066f30 0x7fbb200693e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbb380097c0 tx=0x7fbb38019000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:37.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.569+0000 7fbb317fa700 1 -- 172.21.15.99:0/628436924 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbb2c051990 con 0x7fbb3c0b6580 2023-10-08T00:11:37.749 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.749+0000 7fbb4275f700 1 -- 172.21.15.99:0/628436924 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbb3c0b4720 con 0x7fbb3c0b6580 2023-10-08T00:11:37.750 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.749+0000 7fbb317fa700 1 -- 172.21.15.99:0/628436924 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbb2c054fb0 con 0x7fbb3c0b6580 2023-10-08T00:11:37.750 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2023-10-08T00:11:37.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.751+0000 7fbb4275f700 1 -- 172.21.15.99:0/628436924 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbb20066f30 msgr2=0x7fbb200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:37.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.751+0000 7fbb4275f700 1 --2- 172.21.15.99:0/628436924 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbb20066f30 0x7fbb200693e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbb380097c0 tx=0x7fbb38019000 comp rx=0 tx=0).stop 2023-10-08T00:11:37.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.751+0000 7fbb4275f700 1 -- 172.21.15.99:0/628436924 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb3c0b6580 msgr2=0x7fbb3c16df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:37.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.751+0000 7fbb4275f700 1 --2- 172.21.15.99:0/628436924 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb3c0b6580 0x7fbb3c16df00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c00ec80 tx=0x7fbb2c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:37.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.751+0000 7fbb4275f700 1 -- 172.21.15.99:0/628436924 shutdown_connections 2023-10-08T00:11:37.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.751+0000 7fbb4275f700 1 --2- 172.21.15.99:0/628436924 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbb20066f30 0x7fbb200693e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:37.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.751+0000 7fbb4275f700 1 --2- 172.21.15.99:0/628436924 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb3c06a830 0x7fbb3c16d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:37.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.751+0000 7fbb4275f700 1 --2- 172.21.15.99:0/628436924 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb3c0cce30 0x7fbb3c0b1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:37.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.751+0000 7fbb4275f700 1 --2- 172.21.15.99:0/628436924 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb3c0b6580 0x7fbb3c16df00 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:37.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.751+0000 7fbb4275f700 1 -- 172.21.15.99:0/628436924 >> 172.21.15.99:0/628436924 conn(0x7fbb3c068cb0 msgr2=0x7fbb3c15bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:37.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.751+0000 7fbb4275f700 1 -- 172.21.15.99:0/628436924 shutdown_connections 2023-10-08T00:11:37.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.751+0000 7fbb4275f700 1 -- 172.21.15.99:0/628436924 wait complete. 2023-10-08T00:11:37.764 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T00:11:37.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.877+0000 7fbd4cb32700 1 Processor -- start 2023-10-08T00:11:37.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.877+0000 7fbd4cb32700 1 -- start start 2023-10-08T00:11:37.880 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.878+0000 7fbd4cb32700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd481577d0 0x7fbd480cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:37.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.878+0000 7fbd4cb32700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd4815c360 con 0x7fbd4806ac30 2023-10-08T00:11:37.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.878+0000 7fbd4cb32700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd4815b790 con 0x7fbd48157ba0 2023-10-08T00:11:37.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.878+0000 7fbd4cb32700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4815ba30 con 0x7fbd481577d0 2023-10-08T00:11:37.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.878+0000 7fbd46d9d700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fbd4806ac30 0x7fbd4815c7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:48556/0 (socket says 172.21.15.99:48556) 2023-10-08T00:11:37.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.878+0000 7fbd46d9d700 1 -- 172.21.15.99:0/1119285953 learned_addr learned my addr 172.21.15.99:0/1119285953 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:37.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.878+0000 7fbd4659c700 1 --2- 172.21.15.99:0/1119285953 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd481577d0 0x7fbd480cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:37.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.879+0000 7fbd4559a700 1 -- 172.21.15.99:0/1119285953 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 629578757 0 0) 0x7fbd4815c360 con 0x7fbd4806ac30 2023-10-08T00:11:37.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.879+0000 7fbd4559a700 1 -- 172.21.15.99:0/1119285953 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd30003570 con 0x7fbd4806ac30 2023-10-08T00:11:37.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.879+0000 7fbd4559a700 1 -- 172.21.15.99:0/1119285953 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4140690556 0 0) 0x7fbd4815b790 con 0x7fbd48157ba0 2023-10-08T00:11:37.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.879+0000 7fbd4559a700 1 -- 172.21.15.99:0/1119285953 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd4815c360 con 0x7fbd48157ba0 2023-10-08T00:11:37.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.879+0000 7fbd4659c700 1 -- 172.21.15.99:0/1119285953 >> v1:172.21.15.191:6789/0 conn(0x7fbd48157ba0 legacy=0x7fbd4806a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:37.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.879+0000 7fbd4659c700 1 -- 172.21.15.99:0/1119285953 >> v1:172.21.15.99:6789/0 conn(0x7fbd4806ac30 legacy=0x7fbd4815c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:37.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.879+0000 7fbd4659c700 1 -- 172.21.15.99:0/1119285953 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd4815bd10 con 0x7fbd481577d0 2023-10-08T00:11:37.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.879+0000 7fbd4659c700 1 --2- 172.21.15.99:0/1119285953 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd481577d0 0x7fbd480cd770 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbd38009910 tx=0x7fbd380091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9408f1ad5e79e24 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:37.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.879+0000 7fbd4559a700 1 -- 172.21.15.99:0/1119285953 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd38011630 con 0x7fbd481577d0 2023-10-08T00:11:37.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.879+0000 7fbd4559a700 1 -- 172.21.15.99:0/1119285953 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd38011790 con 0x7fbd481577d0 2023-10-08T00:11:37.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.879+0000 7fbd4559a700 1 -- 172.21.15.99:0/1119285953 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd38011b30 con 0x7fbd481577d0 2023-10-08T00:11:37.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.880+0000 7fbd4cb32700 1 -- 172.21.15.99:0/1119285953 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd481577d0 msgr2=0x7fbd480cd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:37.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.880+0000 7fbd4cb32700 1 --2- 172.21.15.99:0/1119285953 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd481577d0 0x7fbd480cd770 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbd38009910 tx=0x7fbd380091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:37.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.880+0000 7fbd4cb32700 1 -- 172.21.15.99:0/1119285953 shutdown_connections 2023-10-08T00:11:37.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.880+0000 7fbd4cb32700 1 --2- 172.21.15.99:0/1119285953 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd481577d0 0x7fbd480cd770 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:37.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.880+0000 7fbd4cb32700 1 -- 172.21.15.99:0/1119285953 >> 172.21.15.99:0/1119285953 conn(0x7fbd48068ce0 msgr2=0x7fbd480690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:37.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.880+0000 7fbd4cb32700 1 -- 172.21.15.99:0/1119285953 shutdown_connections 2023-10-08T00:11:37.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.880+0000 7fbd4cb32700 1 -- 172.21.15.99:0/1119285953 wait complete. 2023-10-08T00:11:37.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.881+0000 7fbd4cb32700 1 Processor -- start 2023-10-08T00:11:37.886 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.881+0000 7fbd4cb32700 1 -- start start 2023-10-08T00:11:37.886 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.882+0000 7fbd4cb32700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd4806ac30 0x7fbd48155740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:37.886 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.882+0000 7fbd4cb32700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd481577d0 0x7fbd4815e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:37.886 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.882+0000 7fbd4cb32700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd48157ba0 0x7fbd4815e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:37.886 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.882+0000 7fbd4cb32700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd481602e0 con 0x7fbd4806ac30 2023-10-08T00:11:37.886 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.882+0000 7fbd4cb32700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4816a920 con 0x7fbd48157ba0 2023-10-08T00:11:37.886 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.882+0000 7fbd4cb32700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4815edb0 con 0x7fbd481577d0 2023-10-08T00:11:37.886 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.882+0000 7fbd4659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd4806ac30 0x7fbd48155740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:37.886 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.882+0000 7fbd4659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd4806ac30 0x7fbd48155740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39838/0 (socket says 172.21.15.99:39838) 2023-10-08T00:11:37.886 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.882+0000 7fbd4659c700 1 -- 172.21.15.99:0/1789457915 learned_addr learned my addr 172.21.15.99:0/1789457915 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:37.886 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.882+0000 7fbd45d9b700 1 --2- 172.21.15.99:0/1789457915 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd481577d0 0x7fbd4815e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:37.886 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.882+0000 7fbd46d9d700 1 --2- 172.21.15.99:0/1789457915 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd48157ba0 0x7fbd4815e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:37.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.882+0000 7fbd4659c700 1 -- 172.21.15.99:0/1789457915 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd481577d0 msgr2=0x7fbd4815e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:37.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.882+0000 7fbd4659c700 1 --2- 172.21.15.99:0/1789457915 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd481577d0 0x7fbd4815e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:37.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.882+0000 7fbd4659c700 1 -- 172.21.15.99:0/1789457915 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd48157ba0 msgr2=0x7fbd4815e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:37.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.882+0000 7fbd4659c700 1 --2- 172.21.15.99:0/1789457915 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd48157ba0 0x7fbd4815e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:37.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.882+0000 7fbd4659c700 1 -- 172.21.15.99:0/1789457915 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd38009420 con 0x7fbd4806ac30 2023-10-08T00:11:37.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.882+0000 7fbd4659c700 1 --2- 172.21.15.99:0/1789457915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd4806ac30 0x7fbd48155740 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbd380091e0 tx=0x7fbd38017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:37.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.882+0000 7fbd367fc700 1 -- 172.21.15.99:0/1789457915 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd3801f070 con 0x7fbd4806ac30 2023-10-08T00:11:37.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.882+0000 7fbd367fc700 1 -- 172.21.15.99:0/1789457915 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd38000bc0 con 0x7fbd4806ac30 2023-10-08T00:11:37.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.882+0000 7fbd4cb32700 1 -- 172.21.15.99:0/1789457915 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd4816abc0 con 0x7fbd4806ac30 2023-10-08T00:11:37.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.882+0000 7fbd367fc700 1 -- 172.21.15.99:0/1789457915 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd38011b30 con 0x7fbd4806ac30 2023-10-08T00:11:37.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.882+0000 7fbd4cb32700 1 -- 172.21.15.99:0/1789457915 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd4815f090 con 0x7fbd4806ac30 2023-10-08T00:11:37.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.883+0000 7fbd4cb32700 1 -- 172.21.15.99:0/1789457915 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd4816b0e0 con 0x7fbd4806ac30 2023-10-08T00:11:37.888 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.884+0000 7fbd367fc700 1 -- 172.21.15.99:0/1789457915 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbd38023070 con 0x7fbd4806ac30 2023-10-08T00:11:37.888 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.884+0000 7fbd367fc700 1 --2- 172.21.15.99:0/1789457915 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbd28066f30 0x7fbd280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:37.888 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.884+0000 7fbd367fc700 1 -- 172.21.15.99:0/1789457915 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fbd38085580 con 0x7fbd4806ac30 2023-10-08T00:11:37.888 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.884+0000 7fbd45d9b700 1 --2- 172.21.15.99:0/1789457915 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbd28066f30 0x7fbd280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:37.888 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.885+0000 7fbd45d9b700 1 --2- 172.21.15.99:0/1789457915 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbd28066f30 0x7fbd280693e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbd4815fd50 tx=0x7fbd3c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:37.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:37.888+0000 7fbd367fc700 1 -- 172.21.15.99:0/1789457915 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbd4816b0e0 con 0x7fbd4806ac30 2023-10-08T00:11:38.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.072+0000 7fbd4cb32700 1 -- 172.21.15.99:0/1789457915 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fbd4816b0e0 con 0x7fbd4806ac30 2023-10-08T00:11:38.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.073+0000 7fbd367fc700 1 -- 172.21.15.99:0/1789457915 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v27) v1 ==== 84+0+65 (secure 0 0 0) 0x7fbd3802a060 con 0x7fbd4806ac30 2023-10-08T00:11:38.074 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:11:38.074 INFO:teuthology.orchestra.run.smithi099.stdout:[ 2023-10-08T00:11:38.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:11:38.074 INFO:teuthology.orchestra.run.smithi099.stdout: "poolnum": 1, 2023-10-08T00:11:38.074 INFO:teuthology.orchestra.run.smithi099.stdout: "poolname": ".mgr" 2023-10-08T00:11:38.074 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:11:38.074 INFO:teuthology.orchestra.run.smithi099.stdout:] 2023-10-08T00:11:38.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.075+0000 7fbd4cb32700 1 -- 172.21.15.99:0/1789457915 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbd28066f30 msgr2=0x7fbd280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:38.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.075+0000 7fbd4cb32700 1 --2- 172.21.15.99:0/1789457915 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbd28066f30 0x7fbd280693e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbd4815fd50 tx=0x7fbd3c009040 comp rx=0 tx=0).stop 2023-10-08T00:11:38.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.075+0000 7fbd4cb32700 1 -- 172.21.15.99:0/1789457915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd4806ac30 msgr2=0x7fbd48155740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:38.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.075+0000 7fbd4cb32700 1 --2- 172.21.15.99:0/1789457915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd4806ac30 0x7fbd48155740 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbd380091e0 tx=0x7fbd38017000 comp rx=0 tx=0).stop 2023-10-08T00:11:38.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.075+0000 7fbd4cb32700 1 -- 172.21.15.99:0/1789457915 shutdown_connections 2023-10-08T00:11:38.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.075+0000 7fbd4cb32700 1 --2- 172.21.15.99:0/1789457915 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbd28066f30 0x7fbd280693e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:38.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.075+0000 7fbd4cb32700 1 --2- 172.21.15.99:0/1789457915 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd48157ba0 0x7fbd4815e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:38.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.075+0000 7fbd4cb32700 1 --2- 172.21.15.99:0/1789457915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd4806ac30 0x7fbd48155740 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:38.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.075+0000 7fbd4cb32700 1 --2- 172.21.15.99:0/1789457915 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd481577d0 0x7fbd4815e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:38.077 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.075+0000 7fbd4cb32700 1 -- 172.21.15.99:0/1789457915 >> 172.21.15.99:0/1789457915 conn(0x7fbd48068ce0 msgr2=0x7fbd480cfb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:38.077 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.075+0000 7fbd4cb32700 1 -- 172.21.15.99:0/1789457915 shutdown_connections 2023-10-08T00:11:38.077 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.075+0000 7fbd4cb32700 1 -- 172.21.15.99:0/1789457915 wait complete. 2023-10-08T00:11:38.087 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T00:11:38.087 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T00:11:38.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.204+0000 7f41088dd700 1 Processor -- start 2023-10-08T00:11:38.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.204+0000 7f41088dd700 1 -- start start 2023-10-08T00:11:38.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.205+0000 7f41088dd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f41040cc940 0x7f41040ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:38.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.205+0000 7f41088dd700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41040cbb80 con 0x7f410415d240 2023-10-08T00:11:38.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.205+0000 7f41088dd700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41040cbe60 con 0x7f41040ce810 2023-10-08T00:11:38.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.205+0000 7f41088dd700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41040cc140 con 0x7f41040cc940 2023-10-08T00:11:38.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.205+0000 7f41037fe700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f410415d240 0x7f41040b3a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:48558/0 (socket says 172.21.15.99:48558) 2023-10-08T00:11:38.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.205+0000 7f4102ffd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f41040cc940 0x7f41040ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:38.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.205+0000 7f41037fe700 1 -- 172.21.15.99:0/2913298905 learned_addr learned my addr 172.21.15.99:0/2913298905 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:38.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.205+0000 7f41027fc700 1 -- 172.21.15.99:0/2913298905 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3234123207 0 0) 0x7f41040cbb80 con 0x7f410415d240 2023-10-08T00:11:38.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.205+0000 7f41027fc700 1 -- 172.21.15.99:0/2913298905 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40ec003570 con 0x7f410415d240 2023-10-08T00:11:38.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.205+0000 7f4102ffd700 1 -- 172.21.15.99:0/2913298905 >> v1:172.21.15.191:6789/0 conn(0x7f41040ce810 legacy=0x7f41040cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:38.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.205+0000 7f4102ffd700 1 -- 172.21.15.99:0/2913298905 >> v1:172.21.15.99:6789/0 conn(0x7f410415d240 legacy=0x7f41040b3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:38.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.205+0000 7f4102ffd700 1 -- 172.21.15.99:0/2913298905 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41040cc420 con 0x7f41040cc940 2023-10-08T00:11:38.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.206+0000 7f4102ffd700 1 --2- 172.21.15.99:0/2913298905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41040cc940 0x7f41040ccde0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f40f80098e0 tx=0x7f40f80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88574e0c162f4c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:38.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.206+0000 7f41027fc700 1 -- 172.21.15.99:0/2913298905 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f8011630 con 0x7f41040cc940 2023-10-08T00:11:38.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.206+0000 7f41027fc700 1 -- 172.21.15.99:0/2913298905 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40f8011790 con 0x7f41040cc940 2023-10-08T00:11:38.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.206+0000 7f41027fc700 1 -- 172.21.15.99:0/2913298905 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f8011b30 con 0x7f41040cc940 2023-10-08T00:11:38.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.206+0000 7f41088dd700 1 -- 172.21.15.99:0/2913298905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41040cc940 msgr2=0x7f41040ccde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:38.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.206+0000 7f41088dd700 1 --2- 172.21.15.99:0/2913298905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41040cc940 0x7f41040ccde0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f40f80098e0 tx=0x7f40f80091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:38.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.206+0000 7f41088dd700 1 -- 172.21.15.99:0/2913298905 shutdown_connections 2023-10-08T00:11:38.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.206+0000 7f41088dd700 1 --2- 172.21.15.99:0/2913298905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41040cc940 0x7f41040ccde0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:38.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.206+0000 7f41088dd700 1 -- 172.21.15.99:0/2913298905 >> 172.21.15.99:0/2913298905 conn(0x7f4104067e40 msgr2=0x7f41040d0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:38.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.206+0000 7f41088dd700 1 -- 172.21.15.99:0/2913298905 shutdown_connections 2023-10-08T00:11:38.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.206+0000 7f41088dd700 1 -- 172.21.15.99:0/2913298905 wait complete. 2023-10-08T00:11:38.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.207+0000 7f41088dd700 1 Processor -- start 2023-10-08T00:11:38.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.207+0000 7f41088dd700 1 -- start start 2023-10-08T00:11:38.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.208+0000 7f41088dd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41040cc940 0x7f41040ca450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:38.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.208+0000 7f41088dd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41040ce810 0x7f4104151430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:38.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.208+0000 7f41088dd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f410415d240 0x7f4104152230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:38.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.208+0000 7f41088dd700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41040b69d0 con 0x7f41040cc940 2023-10-08T00:11:38.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.208+0000 7f41088dd700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41040c96b0 con 0x7f410415d240 2023-10-08T00:11:38.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.208+0000 7f41088dd700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41040c8aa0 con 0x7f41040ce810 2023-10-08T00:11:38.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.208+0000 7f41037fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41040cc940 0x7f41040ca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:38.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.208+0000 7f41037fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41040cc940 0x7f41040ca450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39862/0 (socket says 172.21.15.99:39862) 2023-10-08T00:11:38.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.208+0000 7f41037fe700 1 -- 172.21.15.99:0/1542150790 learned_addr learned my addr 172.21.15.99:0/1542150790 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:38.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.208+0000 7f4103fff700 1 --2- 172.21.15.99:0/1542150790 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f410415d240 0x7f4104152230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:38.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.208+0000 7f4102ffd700 1 --2- 172.21.15.99:0/1542150790 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41040ce810 0x7f4104151430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:38.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.209+0000 7f41037fe700 1 -- 172.21.15.99:0/1542150790 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41040ce810 msgr2=0x7f4104151430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:38.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.209+0000 7f41037fe700 1 --2- 172.21.15.99:0/1542150790 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41040ce810 0x7f4104151430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:38.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.209+0000 7f41037fe700 1 -- 172.21.15.99:0/1542150790 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f410415d240 msgr2=0x7f4104152230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:38.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.209+0000 7f41037fe700 1 --2- 172.21.15.99:0/1542150790 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f410415d240 0x7f4104152230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:38.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.209+0000 7f41037fe700 1 -- 172.21.15.99:0/1542150790 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40f8009420 con 0x7f41040cc940 2023-10-08T00:11:38.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.209+0000 7f4103fff700 1 --2- 172.21.15.99:0/1542150790 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f410415d240 0x7f4104152230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:11:38.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.209+0000 7f41037fe700 1 --2- 172.21.15.99:0/1542150790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41040cc940 0x7f41040ca450 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f40f400c900 tx=0x7f40f4002740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:38.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.209+0000 7f40eb7fe700 1 -- 172.21.15.99:0/1542150790 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f400cea0 con 0x7f41040cc940 2023-10-08T00:11:38.216 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.209+0000 7f40eb7fe700 1 -- 172.21.15.99:0/1542150790 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40f4005d80 con 0x7f41040cc940 2023-10-08T00:11:38.216 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.209+0000 7f40eb7fe700 1 -- 172.21.15.99:0/1542150790 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f400a7e0 con 0x7f41040cc940 2023-10-08T00:11:38.216 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.209+0000 7f41088dd700 1 -- 172.21.15.99:0/1542150790 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41040c8d80 con 0x7f41040cc940 2023-10-08T00:11:38.216 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.209+0000 7f41088dd700 1 -- 172.21.15.99:0/1542150790 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41040c93c0 con 0x7f41040cc940 2023-10-08T00:11:38.216 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.210+0000 7f41088dd700 1 -- 172.21.15.99:0/1542150790 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41040c9990 con 0x7f41040cc940 2023-10-08T00:11:38.216 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.210+0000 7f40eb7fe700 1 -- 172.21.15.99:0/1542150790 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f40f400ac10 con 0x7f41040cc940 2023-10-08T00:11:38.216 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.211+0000 7f40eb7fe700 1 --2- 172.21.15.99:0/1542150790 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f40f0066f30 0x7f40f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:38.216 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.211+0000 7f40eb7fe700 1 -- 172.21.15.99:0/1542150790 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f40f4083f70 con 0x7f41040cc940 2023-10-08T00:11:38.216 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.211+0000 7f4102ffd700 1 --2- 172.21.15.99:0/1542150790 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f40f0066f30 0x7f40f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:38.216 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.211+0000 7f4102ffd700 1 --2- 172.21.15.99:0/1542150790 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f40f0066f30 0x7f40f00693e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f40f80091e0 tx=0x7f40f8016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:38.217 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.215+0000 7f40eb7fe700 1 -- 172.21.15.99:0/1542150790 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f40f40515b0 con 0x7f41040cc940 2023-10-08T00:11:38.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:38.397+0000 7f41088dd700 1 -- 172.21.15.99:0/1542150790 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f410406ac60 con 0x7f41040cc940 2023-10-08T00:11:39.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.288+0000 7f40eb7fe700 1 -- 172.21.15.99:0/1542150790 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v28) v1 ==== 138+0+0 (secure 0 0 0) 0x7f40f4054bd0 con 0x7f41040cc940 2023-10-08T00:11:39.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.291+0000 7f41088dd700 1 -- 172.21.15.99:0/1542150790 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f40f0066f30 msgr2=0x7f40f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:39.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.291+0000 7f41088dd700 1 --2- 172.21.15.99:0/1542150790 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f40f0066f30 0x7f40f00693e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f40f80091e0 tx=0x7f40f8016000 comp rx=0 tx=0).stop 2023-10-08T00:11:39.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.291+0000 7f41088dd700 1 -- 172.21.15.99:0/1542150790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41040cc940 msgr2=0x7f41040ca450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:39.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.291+0000 7f41088dd700 1 --2- 172.21.15.99:0/1542150790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41040cc940 0x7f41040ca450 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f40f400c900 tx=0x7f40f4002740 comp rx=0 tx=0).stop 2023-10-08T00:11:39.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.291+0000 7f41088dd700 1 -- 172.21.15.99:0/1542150790 shutdown_connections 2023-10-08T00:11:39.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.291+0000 7f41088dd700 1 --2- 172.21.15.99:0/1542150790 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f40f0066f30 0x7f40f00693e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:39.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.291+0000 7f41088dd700 1 --2- 172.21.15.99:0/1542150790 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f410415d240 0x7f4104152230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:39.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.291+0000 7f41088dd700 1 --2- 172.21.15.99:0/1542150790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41040cc940 0x7f41040ca450 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:39.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.291+0000 7f41088dd700 1 --2- 172.21.15.99:0/1542150790 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41040ce810 0x7f4104151430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:39.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.291+0000 7f41088dd700 1 -- 172.21.15.99:0/1542150790 >> 172.21.15.99:0/1542150790 conn(0x7f4104067e40 msgr2=0x7f41040c6430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:39.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.292+0000 7f41088dd700 1 -- 172.21.15.99:0/1542150790 shutdown_connections 2023-10-08T00:11:39.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.292+0000 7f41088dd700 1 -- 172.21.15.99:0/1542150790 wait complete. 2023-10-08T00:11:39.294 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' created 2023-10-08T00:11:39.305 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T00:11:39.426 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.425+0000 7fd7fdc77700 1 Processor -- start 2023-10-08T00:11:39.426 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.425+0000 7fd7fdc77700 1 -- start start 2023-10-08T00:11:39.429 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.426+0000 7fd7fdc77700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd7f8159ec0 0x7fd7f80af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:39.429 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.426+0000 7fd7fdc77700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7f80b0790 con 0x7fd7f806acb0 2023-10-08T00:11:39.429 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.426+0000 7fd7fdc77700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7f80ce1f0 con 0x7fd7f815a290 2023-10-08T00:11:39.429 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.426+0000 7fd7fdc77700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7f80b0b20 con 0x7fd7f8159ec0 2023-10-08T00:11:39.429 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.426+0000 7fd7f7fff700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fd7f806acb0 0x7fd7f80cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:48574/0 (socket says 172.21.15.99:48574) 2023-10-08T00:11:39.429 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.426+0000 7fd7f7fff700 1 -- 172.21.15.99:0/1864259628 learned_addr learned my addr 172.21.15.99:0/1864259628 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:39.429 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.426+0000 7fd7f77fe700 1 --2- 172.21.15.99:0/1864259628 >> v2:172.21.15.191:3300/0 conn(0x7fd7f8159ec0 0x7fd7f80af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:39.430 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.427+0000 7fd7f67fc700 1 -- 172.21.15.99:0/1864259628 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3981573913 0 0) 0x7fd7f80b0790 con 0x7fd7f806acb0 2023-10-08T00:11:39.430 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.427+0000 7fd7f67fc700 1 -- 172.21.15.99:0/1864259628 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7e0003570 con 0x7fd7f806acb0 2023-10-08T00:11:39.430 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.427+0000 7fd7f67fc700 1 -- 172.21.15.99:0/1864259628 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1225556746 0 0) 0x7fd7f80ce1f0 con 0x7fd7f815a290 2023-10-08T00:11:39.430 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.427+0000 7fd7f67fc700 1 -- 172.21.15.99:0/1864259628 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7f80b0790 con 0x7fd7f815a290 2023-10-08T00:11:39.430 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.427+0000 7fd7f67fc700 1 -- 172.21.15.99:0/1864259628 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2693375915 0 0) 0x7fd7e0003570 con 0x7fd7f806acb0 2023-10-08T00:11:39.430 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.427+0000 7fd7f77fe700 1 -- 172.21.15.99:0/1864259628 >> v1:172.21.15.191:6789/0 conn(0x7fd7f815a290 legacy=0x7fd7f806a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:39.430 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.427+0000 7fd7f77fe700 1 -- 172.21.15.99:0/1864259628 >> v1:172.21.15.99:6789/0 conn(0x7fd7f806acb0 legacy=0x7fd7f80cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:39.430 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.427+0000 7fd7f77fe700 1 -- 172.21.15.99:0/1864259628 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f80b04b0 con 0x7fd7f8159ec0 2023-10-08T00:11:39.430 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.427+0000 7fd7f77fe700 1 --2- 172.21.15.99:0/1864259628 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7f8159ec0 0x7fd7f80af940 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8009e60 tx=0x7fd7e8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b97adc577c47433d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:39.431 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.427+0000 7fd7f67fc700 1 -- 172.21.15.99:0/1864259628 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e800e070 con 0x7fd7f8159ec0 2023-10-08T00:11:39.431 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.427+0000 7fd7f67fc700 1 -- 172.21.15.99:0/1864259628 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7e800b7a0 con 0x7fd7f8159ec0 2023-10-08T00:11:39.431 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.427+0000 7fd7f67fc700 1 -- 172.21.15.99:0/1864259628 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f0001470 con 0x7fd7f8159ec0 2023-10-08T00:11:39.431 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.428+0000 7fd7fdc77700 1 -- 172.21.15.99:0/1864259628 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7f8159ec0 msgr2=0x7fd7f80af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:39.431 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.428+0000 7fd7fdc77700 1 --2- 172.21.15.99:0/1864259628 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7f8159ec0 0x7fd7f80af940 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8009e60 tx=0x7fd7e8009780 comp rx=0 tx=0).stop 2023-10-08T00:11:39.431 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.428+0000 7fd7fdc77700 1 -- 172.21.15.99:0/1864259628 shutdown_connections 2023-10-08T00:11:39.431 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.428+0000 7fd7fdc77700 1 --2- 172.21.15.99:0/1864259628 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7f8159ec0 0x7fd7f80af940 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:39.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.428+0000 7fd7fdc77700 1 -- 172.21.15.99:0/1864259628 >> 172.21.15.99:0/1864259628 conn(0x7fd7f80d1880 msgr2=0x7fd7f80cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:39.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.428+0000 7fd7fdc77700 1 -- 172.21.15.99:0/1864259628 shutdown_connections 2023-10-08T00:11:39.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.428+0000 7fd7fdc77700 1 -- 172.21.15.99:0/1864259628 wait complete. 2023-10-08T00:11:39.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.429+0000 7fd7fdc77700 1 Processor -- start 2023-10-08T00:11:39.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.429+0000 7fd7fdc77700 1 -- start start 2023-10-08T00:11:39.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.430+0000 7fd7fdc77700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7f806acb0 0x7fd7f80cab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:39.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.430+0000 7fd7fdc77700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7f8159ec0 0x7fd7f8158e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:39.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.430+0000 7fd7fdc77700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7f815a290 0x7fd7f816dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:39.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.430+0000 7fd7fdc77700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f80c9470 con 0x7fd7f815a290 2023-10-08T00:11:39.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.430+0000 7fd7fdc77700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f80c83d0 con 0x7fd7f806acb0 2023-10-08T00:11:39.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.430+0000 7fd7fdc77700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f80c86b0 con 0x7fd7f8159ec0 2023-10-08T00:11:39.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.430+0000 7fd7f7fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7f815a290 0x7fd7f816dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:39.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.430+0000 7fd7f7fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7f815a290 0x7fd7f816dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39876/0 (socket says 172.21.15.99:39876) 2023-10-08T00:11:39.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.430+0000 7fd7f7fff700 1 -- 172.21.15.99:0/1199314315 learned_addr learned my addr 172.21.15.99:0/1199314315 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:39.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.430+0000 7fd7f77fe700 1 --2- 172.21.15.99:0/1199314315 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7f806acb0 0x7fd7f80cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:39.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.430+0000 7fd7f6ffd700 1 --2- 172.21.15.99:0/1199314315 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7f8159ec0 0x7fd7f8158e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:39.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.430+0000 7fd7f7fff700 1 -- 172.21.15.99:0/1199314315 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7f8159ec0 msgr2=0x7fd7f8158e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:39.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.430+0000 7fd7f7fff700 1 --2- 172.21.15.99:0/1199314315 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7f8159ec0 0x7fd7f8158e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:39.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.430+0000 7fd7f7fff700 1 -- 172.21.15.99:0/1199314315 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7f806acb0 msgr2=0x7fd7f80cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:39.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.430+0000 7fd7f7fff700 1 --2- 172.21.15.99:0/1199314315 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7f806acb0 0x7fd7f80cab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:39.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.430+0000 7fd7f7fff700 1 -- 172.21.15.99:0/1199314315 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e8009a20 con 0x7fd7f815a290 2023-10-08T00:11:39.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.430+0000 7fd7f77fe700 1 --2- 172.21.15.99:0/1199314315 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7f806acb0 0x7fd7f80cab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:11:39.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.430+0000 7fd7f6ffd700 1 --2- 172.21.15.99:0/1199314315 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7f8159ec0 0x7fd7f8158e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:11:39.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.430+0000 7fd7f7fff700 1 --2- 172.21.15.99:0/1199314315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7f815a290 0x7fd7f816dde0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd7f000c7c0 tx=0x7fd7f0001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:39.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.430+0000 7fd7df7fe700 1 -- 172.21.15.99:0/1199314315 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f0007360 con 0x7fd7f815a290 2023-10-08T00:11:39.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.430+0000 7fd7df7fe700 1 -- 172.21.15.99:0/1199314315 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7f00074e0 con 0x7fd7f815a290 2023-10-08T00:11:39.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.430+0000 7fd7fdc77700 1 -- 172.21.15.99:0/1199314315 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7f80c8990 con 0x7fd7f815a290 2023-10-08T00:11:39.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.430+0000 7fd7df7fe700 1 -- 172.21.15.99:0/1199314315 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f0004830 con 0x7fd7f815a290 2023-10-08T00:11:39.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.431+0000 7fd7fdc77700 1 -- 172.21.15.99:0/1199314315 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7f80c8f50 con 0x7fd7f815a290 2023-10-08T00:11:39.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.431+0000 7fd7fdc77700 1 -- 172.21.15.99:0/1199314315 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7f8064650 con 0x7fd7f815a290 2023-10-08T00:11:39.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.432+0000 7fd7df7fe700 1 -- 172.21.15.99:0/1199314315 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd7f001c3b0 con 0x7fd7f815a290 2023-10-08T00:11:39.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.432+0000 7fd7df7fe700 1 --2- 172.21.15.99:0/1199314315 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd7e4066f30 0x7fd7e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:39.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.433+0000 7fd7df7fe700 1 -- 172.21.15.99:0/1199314315 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5223+0+0 (secure 0 0 0) 0x7fd7f0083cd0 con 0x7fd7f815a290 2023-10-08T00:11:39.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.433+0000 7fd7f77fe700 1 --2- 172.21.15.99:0/1199314315 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd7e4066f30 0x7fd7e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:39.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.433+0000 7fd7f77fe700 1 --2- 172.21.15.99:0/1199314315 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd7e4066f30 0x7fd7e40693e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd7e80099a0 tx=0x7fd7e8017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:39.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.436+0000 7fd7df7fe700 1 -- 172.21.15.99:0/1199314315 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd7f0054d10 con 0x7fd7f815a290 2023-10-08T00:11:39.617 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:39.616+0000 7fd7fdc77700 1 -- 172.21.15.99:0/1199314315 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd7f80c9170 con 0x7fd7f815a290 2023-10-08T00:11:40.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.293+0000 7fd7df7fe700 1 -- 172.21.15.99:0/1199314315 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v29) v1 ==== 170+0+0 (secure 0 0 0) 0x7fd7f00180d0 con 0x7fd7f815a290 2023-10-08T00:11:40.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.297+0000 7fd7fdc77700 1 -- 172.21.15.99:0/1199314315 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd7e4066f30 msgr2=0x7fd7e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:40.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.297+0000 7fd7fdc77700 1 --2- 172.21.15.99:0/1199314315 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd7e4066f30 0x7fd7e40693e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd7e80099a0 tx=0x7fd7e8017000 comp rx=0 tx=0).stop 2023-10-08T00:11:40.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.297+0000 7fd7fdc77700 1 -- 172.21.15.99:0/1199314315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7f815a290 msgr2=0x7fd7f816dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:40.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.297+0000 7fd7fdc77700 1 --2- 172.21.15.99:0/1199314315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7f815a290 0x7fd7f816dde0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd7f000c7c0 tx=0x7fd7f0001a20 comp rx=0 tx=0).stop 2023-10-08T00:11:40.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.301+0000 7fd7fdc77700 1 -- 172.21.15.99:0/1199314315 shutdown_connections 2023-10-08T00:11:40.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.301+0000 7fd7fdc77700 1 --2- 172.21.15.99:0/1199314315 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd7e4066f30 0x7fd7e40693e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:40.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.301+0000 7fd7fdc77700 1 --2- 172.21.15.99:0/1199314315 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7f806acb0 0x7fd7f80cab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:40.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.301+0000 7fd7fdc77700 1 --2- 172.21.15.99:0/1199314315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7f815a290 0x7fd7f816dde0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:40.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.301+0000 7fd7fdc77700 1 --2- 172.21.15.99:0/1199314315 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7f8159ec0 0x7fd7f8158e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:40.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.301+0000 7fd7fdc77700 1 -- 172.21.15.99:0/1199314315 >> 172.21.15.99:0/1199314315 conn(0x7fd7f80d1880 msgr2=0x7fd7f80b0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:40.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.303+0000 7fd7fdc77700 1 -- 172.21.15.99:0/1199314315 shutdown_connections 2023-10-08T00:11:40.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.304+0000 7fd7fdc77700 1 -- 172.21.15.99:0/1199314315 wait complete. 2023-10-08T00:11:40.306 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' created 2023-10-08T00:11:40.318 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T00:11:40.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.435+0000 7f1715f5b700 1 Processor -- start 2023-10-08T00:11:40.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.436+0000 7f1715f5b700 1 -- start start 2023-10-08T00:11:40.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.436+0000 7f1715f5b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1710159ec0 0x7f17100af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:40.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.436+0000 7f1715f5b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f171015a290 0x7f17100afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:40.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.436+0000 7f1715f5b700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17100b0790 con 0x7f171006a5a0 2023-10-08T00:11:40.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.436+0000 7f1715f5b700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17100ce1f0 con 0x7f1710159ec0 2023-10-08T00:11:40.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.436+0000 7f1715f5b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17100b0b20 con 0x7f171015a290 2023-10-08T00:11:40.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.437+0000 7f170f7fe700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1710159ec0 0x7f17100af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:40.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.437+0000 7f170effd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f171015a290 0x7f17100afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:40.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.437+0000 7f170ffff700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f171006a5a0 0x7f171015db90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:37068/0 (socket says 172.21.15.99:37068) 2023-10-08T00:11:40.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.437+0000 7f170ffff700 1 -- 172.21.15.99:0/805898636 learned_addr learned my addr 172.21.15.99:0/805898636 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:40.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.437+0000 7f170f7fe700 1 -- 172.21.15.99:0/805898636 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f171015a290 msgr2=0x7f17100afe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:40.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.437+0000 7f170f7fe700 1 --2- 172.21.15.99:0/805898636 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f171015a290 0x7f17100afe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:40.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.437+0000 7f170f7fe700 1 -- 172.21.15.99:0/805898636 >> v1:172.21.15.191:6789/0 conn(0x7f171006a5a0 legacy=0x7f171015db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:40.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.437+0000 7f170f7fe700 1 -- 172.21.15.99:0/805898636 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17100b04b0 con 0x7f1710159ec0 2023-10-08T00:11:40.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.437+0000 7f170effd700 1 --2- 172.21.15.99:0/805898636 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f171015a290 0x7f17100afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:11:40.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.437+0000 7f170f7fe700 1 --2- 172.21.15.99:0/805898636 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1710159ec0 0x7f17100af940 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1700009e60 tx=0x7f1700009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76d544fb7bf46556 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:40.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.437+0000 7f170e7fc700 1 -- 172.21.15.99:0/805898636 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170000e070 con 0x7f1710159ec0 2023-10-08T00:11:40.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.437+0000 7f170e7fc700 1 -- 172.21.15.99:0/805898636 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f170000bb60 con 0x7f1710159ec0 2023-10-08T00:11:40.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.437+0000 7f170e7fc700 1 -- 172.21.15.99:0/805898636 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1700012430 con 0x7f1710159ec0 2023-10-08T00:11:40.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.438+0000 7f1715f5b700 1 -- 172.21.15.99:0/805898636 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1710159ec0 msgr2=0x7f17100af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:40.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.438+0000 7f1715f5b700 1 --2- 172.21.15.99:0/805898636 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1710159ec0 0x7f17100af940 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1700009e60 tx=0x7f1700009780 comp rx=0 tx=0).stop 2023-10-08T00:11:40.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.438+0000 7f1715f5b700 1 -- 172.21.15.99:0/805898636 shutdown_connections 2023-10-08T00:11:40.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.438+0000 7f1715f5b700 1 --2- 172.21.15.99:0/805898636 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1710159ec0 0x7f17100af940 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:40.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.438+0000 7f1715f5b700 1 --2- 172.21.15.99:0/805898636 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f171015a290 0x7f17100afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:40.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.438+0000 7f1715f5b700 1 -- 172.21.15.99:0/805898636 >> 172.21.15.99:0/805898636 conn(0x7f17100d1880 msgr2=0x7f17100cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:40.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.438+0000 7f1715f5b700 1 -- 172.21.15.99:0/805898636 shutdown_connections 2023-10-08T00:11:40.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.438+0000 7f1715f5b700 1 -- 172.21.15.99:0/805898636 wait complete. 2023-10-08T00:11:40.444 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.439+0000 7f1715f5b700 1 Processor -- start 2023-10-08T00:11:40.444 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.439+0000 7f1715f5b700 1 -- start start 2023-10-08T00:11:40.444 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.440+0000 7f1715f5b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f171006a5a0 0x7f1710156c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:40.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.440+0000 7f1715f5b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1710159ec0 0x7f17100b4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:40.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.440+0000 7f1715f5b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f171015a290 0x7f17100c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:40.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.440+0000 7f1715f5b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17100cb120 con 0x7f1710159ec0 2023-10-08T00:11:40.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.440+0000 7f1715f5b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171015b310 con 0x7f171006a5a0 2023-10-08T00:11:40.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.440+0000 7f1715f5b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17101571a0 con 0x7f171015a290 2023-10-08T00:11:40.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.440+0000 7f170effd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1710159ec0 0x7f17100b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:40.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.440+0000 7f170effd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1710159ec0 0x7f17100b4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39890/0 (socket says 172.21.15.99:39890) 2023-10-08T00:11:40.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.440+0000 7f170effd700 1 -- 172.21.15.99:0/230133 learned_addr learned my addr 172.21.15.99:0/230133 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:40.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.440+0000 7f170f7fe700 1 --2- 172.21.15.99:0/230133 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f171006a5a0 0x7f1710156c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:40.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.440+0000 7f170ffff700 1 --2- 172.21.15.99:0/230133 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f171015a290 0x7f17100c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:40.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.441+0000 7f170effd700 1 -- 172.21.15.99:0/230133 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f171015a290 msgr2=0x7f17100c8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:40.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.441+0000 7f170effd700 1 --2- 172.21.15.99:0/230133 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f171015a290 0x7f17100c8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:40.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.441+0000 7f170effd700 1 -- 172.21.15.99:0/230133 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f171006a5a0 msgr2=0x7f1710156c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:40.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.441+0000 7f170effd700 1 --2- 172.21.15.99:0/230133 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f171006a5a0 0x7f1710156c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:40.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.441+0000 7f170effd700 1 -- 172.21.15.99:0/230133 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1704009420 con 0x7f1710159ec0 2023-10-08T00:11:40.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.441+0000 7f170effd700 1 --2- 172.21.15.99:0/230133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1710159ec0 0x7f17100b4560 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f170400ec80 tx=0x7f17040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:40.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.441+0000 7f16f77fe700 1 -- 172.21.15.99:0/230133 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170400c920 con 0x7f1710159ec0 2023-10-08T00:11:40.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.441+0000 7f16f77fe700 1 -- 172.21.15.99:0/230133 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f170400ca80 con 0x7f1710159ec0 2023-10-08T00:11:40.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.441+0000 7f16f77fe700 1 -- 172.21.15.99:0/230133 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170400ce00 con 0x7f1710159ec0 2023-10-08T00:11:40.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.441+0000 7f1715f5b700 1 -- 172.21.15.99:0/230133 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1700009a20 con 0x7f1710159ec0 2023-10-08T00:11:40.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.441+0000 7f1715f5b700 1 -- 172.21.15.99:0/230133 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f171015a7b0 con 0x7f1710159ec0 2023-10-08T00:11:40.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.442+0000 7f1715f5b700 1 -- 172.21.15.99:0/230133 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f171015b480 con 0x7f1710159ec0 2023-10-08T00:11:40.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.442+0000 7f16f77fe700 1 -- 172.21.15.99:0/230133 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1704013680 con 0x7f1710159ec0 2023-10-08T00:11:40.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.443+0000 7f16f77fe700 1 --2- 172.21.15.99:0/230133 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f16fc066f30 0x7f16fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:40.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.443+0000 7f16f77fe700 1 -- 172.21.15.99:0/230133 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f17040848b0 con 0x7f1710159ec0 2023-10-08T00:11:40.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.443+0000 7f170f7fe700 1 --2- 172.21.15.99:0/230133 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f16fc066f30 0x7f16fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:40.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.443+0000 7f170f7fe700 1 --2- 172.21.15.99:0/230133 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f16fc066f30 0x7f16fc0693e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f17000097c0 tx=0x7f1700019000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:40.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.447+0000 7f16f77fe700 1 -- 172.21.15.99:0/230133 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1704051b80 con 0x7f1710159ec0 2023-10-08T00:11:40.671 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:40.671+0000 7f1715f5b700 1 -- 172.21.15.99:0/230133 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f17100ceab0 con 0x7f1710159ec0 2023-10-08T00:11:41.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.294+0000 7f16f77fe700 1 -- 172.21.15.99:0/230133 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7f17040551a0 con 0x7f1710159ec0 2023-10-08T00:11:41.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.298+0000 7f1715f5b700 1 -- 172.21.15.99:0/230133 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f16fc066f30 msgr2=0x7f16fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:41.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.298+0000 7f1715f5b700 1 --2- 172.21.15.99:0/230133 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f16fc066f30 0x7f16fc0693e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f17000097c0 tx=0x7f1700019000 comp rx=0 tx=0).stop 2023-10-08T00:11:41.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.298+0000 7f1715f5b700 1 -- 172.21.15.99:0/230133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1710159ec0 msgr2=0x7f17100b4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:41.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.298+0000 7f1715f5b700 1 --2- 172.21.15.99:0/230133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1710159ec0 0x7f17100b4560 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f170400ec80 tx=0x7f17040091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:41.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.298+0000 7f1715f5b700 1 -- 172.21.15.99:0/230133 shutdown_connections 2023-10-08T00:11:41.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.298+0000 7f1715f5b700 1 --2- 172.21.15.99:0/230133 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f16fc066f30 0x7f16fc0693e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:41.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.298+0000 7f1715f5b700 1 --2- 172.21.15.99:0/230133 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f171006a5a0 0x7f1710156c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:41.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.298+0000 7f1715f5b700 1 --2- 172.21.15.99:0/230133 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f171015a290 0x7f17100c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:41.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.298+0000 7f1715f5b700 1 --2- 172.21.15.99:0/230133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1710159ec0 0x7f17100b4560 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:41.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.298+0000 7f1715f5b700 1 -- 172.21.15.99:0/230133 >> 172.21.15.99:0/230133 conn(0x7f17100d1880 msgr2=0x7f17100b10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:41.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.298+0000 7f1715f5b700 1 -- 172.21.15.99:0/230133 shutdown_connections 2023-10-08T00:11:41.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.298+0000 7f1715f5b700 1 -- 172.21.15.99:0/230133 wait complete. 2023-10-08T00:11:41.300 INFO:teuthology.orchestra.run.smithi099.stderr:new fs with metadata pool 4 and data pool 5 2023-10-08T00:11:41.313 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:11:41.428 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.427+0000 7f02745e6700 1 Processor -- start 2023-10-08T00:11:41.428 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.427+0000 7f02745e6700 1 -- start start 2023-10-08T00:11:41.430 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.428+0000 7f02745e6700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f026c0afff0 0x7f026c15e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:41.430 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.428+0000 7f02745e6700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f026c0b03c0 con 0x7f026c069e20 2023-10-08T00:11:41.430 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.428+0000 7f02745e6700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f026c06a870 con 0x7f026c070e10 2023-10-08T00:11:41.430 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.428+0000 7f02745e6700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f026c06ab50 con 0x7f026c0afff0 2023-10-08T00:11:41.431 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.428+0000 7f0272382700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f026c0afff0 0x7f026c15e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:41.431 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.428+0000 7f0271b81700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f026c069e20 0x7f026c070700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:48582/0 (socket says 172.21.15.99:48582) 2023-10-08T00:11:41.431 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.428+0000 7f0271b81700 1 -- 172.21.15.99:0/781431566 learned_addr learned my addr 172.21.15.99:0/781431566 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:41.431 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.429+0000 7f0272382700 1 -- 172.21.15.99:0/781431566 >> v1:172.21.15.191:6789/0 conn(0x7f026c070e10 legacy=0x7f026c06ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:41.431 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.429+0000 7f0272382700 1 -- 172.21.15.99:0/781431566 >> v1:172.21.15.99:6789/0 conn(0x7f026c069e20 legacy=0x7f026c070700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:41.431 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.429+0000 7f0272382700 1 -- 172.21.15.99:0/781431566 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f026c15da60 con 0x7f026c0afff0 2023-10-08T00:11:41.431 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.429+0000 7f0271380700 1 -- 172.21.15.99:0/781431566 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3617071668 0 0) 0x7f026c0b03c0 con 0x7f026c069e20 2023-10-08T00:11:41.431 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.429+0000 7f0272382700 1 --2- 172.21.15.99:0/781431566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f026c0afff0 0x7f026c15e540 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0268009e60 tx=0x7f0268009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efb890033d0ef1e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:41.431 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.429+0000 7f0271380700 1 -- 172.21.15.99:0/781431566 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026800e070 con 0x7f026c0afff0 2023-10-08T00:11:41.431 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.429+0000 7f0271380700 1 -- 172.21.15.99:0/781431566 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f026800bbf0 con 0x7f026c0afff0 2023-10-08T00:11:41.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.429+0000 7f0271380700 1 -- 172.21.15.99:0/781431566 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0268012430 con 0x7f026c0afff0 2023-10-08T00:11:41.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.429+0000 7f02745e6700 1 -- 172.21.15.99:0/781431566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f026c0afff0 msgr2=0x7f026c15e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:41.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.429+0000 7f02745e6700 1 --2- 172.21.15.99:0/781431566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f026c0afff0 0x7f026c15e540 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0268009e60 tx=0x7f0268009780 comp rx=0 tx=0).stop 2023-10-08T00:11:41.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.429+0000 7f02745e6700 1 -- 172.21.15.99:0/781431566 shutdown_connections 2023-10-08T00:11:41.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.429+0000 7f02745e6700 1 --2- 172.21.15.99:0/781431566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f026c0afff0 0x7f026c15e540 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:41.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.429+0000 7f02745e6700 1 -- 172.21.15.99:0/781431566 >> 172.21.15.99:0/781431566 conn(0x7f026c0cda50 msgr2=0x7f026c0cde50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:41.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.429+0000 7f02745e6700 1 -- 172.21.15.99:0/781431566 shutdown_connections 2023-10-08T00:11:41.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.430+0000 7f02745e6700 1 -- 172.21.15.99:0/781431566 wait complete. 2023-10-08T00:11:41.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.431+0000 7f02745e6700 1 Processor -- start 2023-10-08T00:11:41.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.431+0000 7f02745e6700 1 -- start start 2023-10-08T00:11:41.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.431+0000 7f02745e6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f026c069e20 0x7f026c16db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:41.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.431+0000 7f02745e6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f026c070e10 0x7f026c0b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:41.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.431+0000 7f02745e6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f026c0afff0 0x7f026c0abbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:41.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.431+0000 7f02745e6700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026c0b2990 con 0x7f026c069e20 2023-10-08T00:11:41.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.431+0000 7f02745e6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026c16e0b0 con 0x7f026c0afff0 2023-10-08T00:11:41.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.431+0000 7f02745e6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f026c16e380 con 0x7f026c070e10 2023-10-08T00:11:41.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.431+0000 7f0272382700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f026c069e20 0x7f026c16db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:41.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.431+0000 7f0272382700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f026c069e20 0x7f026c16db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39908/0 (socket says 172.21.15.99:39908) 2023-10-08T00:11:41.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.431+0000 7f0272382700 1 -- 172.21.15.99:0/3580214819 learned_addr learned my addr 172.21.15.99:0/3580214819 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:41.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.432+0000 7f0271b81700 1 --2- 172.21.15.99:0/3580214819 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f026c070e10 0x7f026c0b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:41.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.432+0000 7f0272382700 1 -- 172.21.15.99:0/3580214819 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f026c070e10 msgr2=0x7f026c0b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:41.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.432+0000 7f0272382700 1 --2- 172.21.15.99:0/3580214819 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f026c070e10 0x7f026c0b30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:41.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.432+0000 7f0272b83700 1 --2- 172.21.15.99:0/3580214819 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f026c0afff0 0x7f026c0abbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:41.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.432+0000 7f0272382700 1 -- 172.21.15.99:0/3580214819 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f026c0afff0 msgr2=0x7f026c0abbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:41.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.432+0000 7f0272382700 1 --2- 172.21.15.99:0/3580214819 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f026c0afff0 0x7f026c0abbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:41.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.432+0000 7f0272382700 1 -- 172.21.15.99:0/3580214819 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0268009a20 con 0x7f026c069e20 2023-10-08T00:11:41.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.432+0000 7f0272382700 1 --2- 172.21.15.99:0/3580214819 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f026c069e20 0x7f026c16db70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f026c06b150 tx=0x7f0268018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:41.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.432+0000 7f02627fc700 1 -- 172.21.15.99:0/3580214819 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0268027470 con 0x7f026c069e20 2023-10-08T00:11:41.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.432+0000 7f02627fc700 1 -- 172.21.15.99:0/3580214819 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02680275f0 con 0x7f026c069e20 2023-10-08T00:11:41.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.432+0000 7f02745e6700 1 -- 172.21.15.99:0/3580214819 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f026c16e660 con 0x7f026c069e20 2023-10-08T00:11:41.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.432+0000 7f02627fc700 1 -- 172.21.15.99:0/3580214819 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02680279b0 con 0x7f026c069e20 2023-10-08T00:11:41.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.432+0000 7f02745e6700 1 -- 172.21.15.99:0/3580214819 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f026c0b2be0 con 0x7f026c069e20 2023-10-08T00:11:41.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.433+0000 7f02745e6700 1 -- 172.21.15.99:0/3580214819 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f026c0b2800 con 0x7f026c069e20 2023-10-08T00:11:41.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.433+0000 7f02627fc700 1 -- 172.21.15.99:0/3580214819 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0268024070 con 0x7f026c069e20 2023-10-08T00:11:41.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.434+0000 7f02627fc700 1 --2- 172.21.15.99:0/3580214819 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0250066f30 0x7f02500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:41.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.434+0000 7f02627fc700 1 -- 172.21.15.99:0/3580214819 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0268086df0 con 0x7f026c069e20 2023-10-08T00:11:41.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.434+0000 7f0271b81700 1 --2- 172.21.15.99:0/3580214819 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0250066f30 0x7f02500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:41.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.434+0000 7f0271b81700 1 --2- 172.21.15.99:0/3580214819 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0250066f30 0x7f02500693e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f025c001750 tx=0x7f025c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:41.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.438+0000 7f02627fc700 1 -- 172.21.15.99:0/3580214819 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0268054060 con 0x7f026c069e20 2023-10-08T00:11:41.622 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.622+0000 7f02745e6700 1 -- 172.21.15.99:0/3580214819 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f026c04c3c0 con 0x7f026c069e20 2023-10-08T00:11:41.623 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.623+0000 7f02627fc700 1 -- 172.21.15.99:0/3580214819 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+15363 (secure 0 0 0) 0x7f0268057680 con 0x7f026c069e20 2023-10-08T00:11:41.624 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:11:41.624 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":30,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:11:40.672572+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:11:38.415553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-08T00:11:39.634576+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:11:41.626 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.625+0000 7f02745e6700 1 -- 172.21.15.99:0/3580214819 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0250066f30 msgr2=0x7f02500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:41.626 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.625+0000 7f02745e6700 1 --2- 172.21.15.99:0/3580214819 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0250066f30 0x7f02500693e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f025c001750 tx=0x7f025c009040 comp rx=0 tx=0).stop 2023-10-08T00:11:41.626 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.625+0000 7f02745e6700 1 -- 172.21.15.99:0/3580214819 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f026c069e20 msgr2=0x7f026c16db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:41.626 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.625+0000 7f02745e6700 1 --2- 172.21.15.99:0/3580214819 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f026c069e20 0x7f026c16db70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f026c06b150 tx=0x7f0268018000 comp rx=0 tx=0).stop 2023-10-08T00:11:41.626 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.625+0000 7f02745e6700 1 -- 172.21.15.99:0/3580214819 shutdown_connections 2023-10-08T00:11:41.626 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.625+0000 7f02745e6700 1 --2- 172.21.15.99:0/3580214819 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0250066f30 0x7f02500693e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:41.626 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.625+0000 7f02745e6700 1 --2- 172.21.15.99:0/3580214819 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f026c0afff0 0x7f026c0abbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:41.626 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.625+0000 7f02745e6700 1 --2- 172.21.15.99:0/3580214819 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f026c069e20 0x7f026c16db70 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:41.626 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.625+0000 7f02745e6700 1 --2- 172.21.15.99:0/3580214819 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f026c070e10 0x7f026c0b30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:41.627 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.625+0000 7f02745e6700 1 -- 172.21.15.99:0/3580214819 >> 172.21.15.99:0/3580214819 conn(0x7f026c0cda50 msgr2=0x7f026c159520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:41.627 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.625+0000 7f02745e6700 1 -- 172.21.15.99:0/3580214819 shutdown_connections 2023-10-08T00:11:41.627 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.625+0000 7f02745e6700 1 -- 172.21.15.99:0/3580214819 wait complete. 2023-10-08T00:11:41.637 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:11:41.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.751+0000 7fa8b9534700 1 Processor -- start 2023-10-08T00:11:41.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.751+0000 7fa8b9534700 1 -- start start 2023-10-08T00:11:41.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.752+0000 7fa8b9534700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa8b40cd6e0 0x7fa8b40cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:41.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.752+0000 7fa8b9534700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa8b40cdab0 0x7fa8b415d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:41.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.752+0000 7fa8b9534700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8b40cb7c0 con 0x7fa8b415dda0 2023-10-08T00:11:41.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.752+0000 7fa8b9534700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8b415e850 con 0x7fa8b40cdab0 2023-10-08T00:11:41.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.752+0000 7fa8b9534700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8b40cd030 con 0x7fa8b40cd6e0 2023-10-08T00:11:41.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.752+0000 7fa8b37fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa8b40cdab0 0x7fa8b415d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:41.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.752+0000 7fa8b8d33700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fa8b415dda0 0x7fa8b415cbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:48590/0 (socket says 172.21.15.99:48590) 2023-10-08T00:11:41.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.752+0000 7fa8b8d33700 1 -- 172.21.15.99:0/1387163864 learned_addr learned my addr 172.21.15.99:0/1387163864 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:41.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.752+0000 7fa8b3fff700 1 --2- 172.21.15.99:0/1387163864 >> v2:172.21.15.191:3300/0 conn(0x7fa8b40cd6e0 0x7fa8b40cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:41.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.752+0000 7fa8b37fe700 1 -- 172.21.15.99:0/1387163864 >> v2:172.21.15.191:3300/0 conn(0x7fa8b40cd6e0 msgr2=0x7fa8b40cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:41.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.752+0000 7fa8b37fe700 1 --2- 172.21.15.99:0/1387163864 >> v2:172.21.15.191:3300/0 conn(0x7fa8b40cd6e0 0x7fa8b40cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:41.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.752+0000 7fa8b37fe700 1 -- 172.21.15.99:0/1387163864 >> v1:172.21.15.99:6789/0 conn(0x7fa8b415dda0 legacy=0x7fa8b415cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:41.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.752+0000 7fa8b37fe700 1 -- 172.21.15.99:0/1387163864 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8b415e290 con 0x7fa8b40cdab0 2023-10-08T00:11:41.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.753+0000 7fa8b37fe700 1 --2- 172.21.15.99:0/1387163864 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8b40cdab0 0x7fa8b415d860 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa8b406b730 tx=0x7fa8a80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90e98a666556d551 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:41.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.753+0000 7fa8b2ffd700 1 -- 172.21.15.99:0/1387163864 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a800b6b0 con 0x7fa8b40cdab0 2023-10-08T00:11:41.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.753+0000 7fa8b2ffd700 1 -- 172.21.15.99:0/1387163864 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8a800b810 con 0x7fa8b40cdab0 2023-10-08T00:11:41.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.753+0000 7fa8b2ffd700 1 -- 172.21.15.99:0/1387163864 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a800bbb0 con 0x7fa8b40cdab0 2023-10-08T00:11:41.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.753+0000 7fa8b9534700 1 -- 172.21.15.99:0/1387163864 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8b40cdab0 msgr2=0x7fa8b415d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:41.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.753+0000 7fa8b9534700 1 --2- 172.21.15.99:0/1387163864 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8b40cdab0 0x7fa8b415d860 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa8b406b730 tx=0x7fa8a80091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:41.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.753+0000 7fa8b9534700 1 -- 172.21.15.99:0/1387163864 shutdown_connections 2023-10-08T00:11:41.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.753+0000 7fa8b9534700 1 --2- 172.21.15.99:0/1387163864 >> v2:172.21.15.191:3300/0 conn(0x7fa8b40cd6e0 0x7fa8b40cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:41.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.753+0000 7fa8b9534700 1 --2- 172.21.15.99:0/1387163864 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8b40cdab0 0x7fa8b415d860 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:41.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.753+0000 7fa8b9534700 1 -- 172.21.15.99:0/1387163864 >> 172.21.15.99:0/1387163864 conn(0x7fa8b4067660 msgr2=0x7fa8b4067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:41.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.753+0000 7fa8b9534700 1 -- 172.21.15.99:0/1387163864 shutdown_connections 2023-10-08T00:11:41.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.753+0000 7fa8b9534700 1 -- 172.21.15.99:0/1387163864 wait complete. 2023-10-08T00:11:41.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.754+0000 7fa8b9534700 1 Processor -- start 2023-10-08T00:11:41.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.755+0000 7fa8b9534700 1 -- start start 2023-10-08T00:11:41.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.755+0000 7fa8b9534700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8b40cd6e0 0x7fa8b414d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:41.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.755+0000 7fa8b9534700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa8b40cdab0 0x7fa8b40c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:41.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.755+0000 7fa8b9534700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa8b415dda0 0x7fa8b41573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:41.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.755+0000 7fa8b9534700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b414e840 con 0x7fa8b40cd6e0 2023-10-08T00:11:41.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.755+0000 7fa8b9534700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b4151220 con 0x7fa8b415dda0 2023-10-08T00:11:41.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.755+0000 7fa8b9534700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b414eac0 con 0x7fa8b40cdab0 2023-10-08T00:11:41.759 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.755+0000 7fa8b3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8b40cd6e0 0x7fa8b414d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:41.759 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.755+0000 7fa8b3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8b40cd6e0 0x7fa8b414d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39928/0 (socket says 172.21.15.99:39928) 2023-10-08T00:11:41.759 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.755+0000 7fa8b3fff700 1 -- 172.21.15.99:0/2039806676 learned_addr learned my addr 172.21.15.99:0/2039806676 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:41.759 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.756+0000 7fa8b3fff700 1 -- 172.21.15.99:0/2039806676 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa8b40cdab0 msgr2=0x7fa8b40c7d20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T00:11:41.759 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.756+0000 7fa8b3fff700 1 --2- 172.21.15.99:0/2039806676 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa8b40cdab0 0x7fa8b40c7d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:41.759 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.756+0000 7fa8b3fff700 1 -- 172.21.15.99:0/2039806676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa8b415dda0 msgr2=0x7fa8b41573e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T00:11:41.759 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.756+0000 7fa8b3fff700 1 --2- 172.21.15.99:0/2039806676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa8b415dda0 0x7fa8b41573e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:41.759 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.756+0000 7fa8b3fff700 1 -- 172.21.15.99:0/2039806676 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a8009420 con 0x7fa8b40cd6e0 2023-10-08T00:11:41.759 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.756+0000 7fa8b3fff700 1 --2- 172.21.15.99:0/2039806676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8b40cd6e0 0x7fa8b414d0d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa8a400d890 tx=0x7fa8a4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:41.760 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.756+0000 7fa89bfff700 1 -- 172.21.15.99:0/2039806676 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a400b600 con 0x7fa8b40cd6e0 2023-10-08T00:11:41.760 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.756+0000 7fa89bfff700 1 -- 172.21.15.99:0/2039806676 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8a400b760 con 0x7fa8b40cd6e0 2023-10-08T00:11:41.760 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.756+0000 7fa8b9534700 1 -- 172.21.15.99:0/2039806676 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8b40c8ec0 con 0x7fa8b40cd6e0 2023-10-08T00:11:41.760 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.756+0000 7fa89bfff700 1 -- 172.21.15.99:0/2039806676 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a400bae0 con 0x7fa8b40cd6e0 2023-10-08T00:11:41.760 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.756+0000 7fa8b9534700 1 -- 172.21.15.99:0/2039806676 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8b40c8980 con 0x7fa8b40cd6e0 2023-10-08T00:11:41.760 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.757+0000 7fa8b9534700 1 -- 172.21.15.99:0/2039806676 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8b4089280 con 0x7fa8b40cd6e0 2023-10-08T00:11:41.761 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.757+0000 7fa89bfff700 1 -- 172.21.15.99:0/2039806676 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa8a401e460 con 0x7fa8b40cd6e0 2023-10-08T00:11:41.761 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.758+0000 7fa89bfff700 1 --2- 172.21.15.99:0/2039806676 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa8a0066f30 0x7fa8a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:41.762 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.758+0000 7fa89bfff700 1 -- 172.21.15.99:0/2039806676 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa8a4084bf0 con 0x7fa8b40cd6e0 2023-10-08T00:11:41.762 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.758+0000 7fa8b37fe700 1 --2- 172.21.15.99:0/2039806676 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa8a0066f30 0x7fa8a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:41.762 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.758+0000 7fa8b37fe700 1 --2- 172.21.15.99:0/2039806676 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa8a0066f30 0x7fa8a00693e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa8b414da10 tx=0x7fa8a8017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:41.762 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.762+0000 7fa89bfff700 1 -- 172.21.15.99:0/2039806676 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa8a4051e60 con 0x7fa8b40cd6e0 2023-10-08T00:11:41.945 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.944+0000 7fa8b9534700 1 -- 172.21.15.99:0/2039806676 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa8b415e570 con 0x7fa8b40cd6e0 2023-10-08T00:11:41.946 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.945+0000 7fa89bfff700 1 -- 172.21.15.99:0/2039806676 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+15363 (secure 0 0 0) 0x7fa8a4055480 con 0x7fa8b40cd6e0 2023-10-08T00:11:41.946 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:11:41.946 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":30,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:11:40.672572+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:11:38.415553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-08T00:11:39.634576+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:11:41.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.947+0000 7fa8b9534700 1 -- 172.21.15.99:0/2039806676 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa8a0066f30 msgr2=0x7fa8a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:41.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.947+0000 7fa8b9534700 1 --2- 172.21.15.99:0/2039806676 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa8a0066f30 0x7fa8a00693e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa8b414da10 tx=0x7fa8a8017000 comp rx=0 tx=0).stop 2023-10-08T00:11:41.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.947+0000 7fa8b9534700 1 -- 172.21.15.99:0/2039806676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8b40cd6e0 msgr2=0x7fa8b414d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:41.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.947+0000 7fa8b9534700 1 --2- 172.21.15.99:0/2039806676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8b40cd6e0 0x7fa8b414d0d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa8a400d890 tx=0x7fa8a4008040 comp rx=0 tx=0).stop 2023-10-08T00:11:41.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.947+0000 7fa8b9534700 1 -- 172.21.15.99:0/2039806676 shutdown_connections 2023-10-08T00:11:41.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.947+0000 7fa8b9534700 1 --2- 172.21.15.99:0/2039806676 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa8a0066f30 0x7fa8a00693e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:41.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.947+0000 7fa8b9534700 1 --2- 172.21.15.99:0/2039806676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa8b415dda0 0x7fa8b41573e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:41.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.947+0000 7fa8b9534700 1 --2- 172.21.15.99:0/2039806676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8b40cd6e0 0x7fa8b414d0d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:41.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.947+0000 7fa8b9534700 1 --2- 172.21.15.99:0/2039806676 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa8b40cdab0 0x7fa8b40c7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:41.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.947+0000 7fa8b9534700 1 -- 172.21.15.99:0/2039806676 >> 172.21.15.99:0/2039806676 conn(0x7fa8b4067660 msgr2=0x7fa8b40b07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:41.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.947+0000 7fa8b9534700 1 -- 172.21.15.99:0/2039806676 shutdown_connections 2023-10-08T00:11:41.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:41.947+0000 7fa8b9534700 1 -- 172.21.15.99:0/2039806676 wait complete. 2023-10-08T00:11:41.960 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T00:11:42.079 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.078+0000 7ff9abfff700 1 Processor -- start 2023-10-08T00:11:42.079 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.078+0000 7ff9abfff700 1 -- start start 2023-10-08T00:11:42.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.079+0000 7ff9abfff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff9ac0b6580 0x7ff9ac06a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:42.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.079+0000 7ff9abfff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff9ac06a830 0x7ff9ac0b5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:42.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.079+0000 7ff9abfff700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9ac06ac90 con 0x7ff9ac0cce30 2023-10-08T00:11:42.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.079+0000 7ff9abfff700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9ac0b4d70 con 0x7ff9ac06a830 2023-10-08T00:11:42.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.079+0000 7ff9abfff700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9ac15b3b0 con 0x7ff9ac0b6580 2023-10-08T00:11:42.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.079+0000 7ff9ab7fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff9ac06a830 0x7ff9ac0b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:42.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.079+0000 7ff9aaffd700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7ff9ac0cce30 0x7ff9ac15aca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:48592/0 (socket says 172.21.15.99:48592) 2023-10-08T00:11:42.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.079+0000 7ff9aaffd700 1 -- 172.21.15.99:0/2882894687 learned_addr learned my addr 172.21.15.99:0/2882894687 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:42.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.079+0000 7ff9aa7fc700 1 -- 172.21.15.99:0/2882894687 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 569654936 0 0) 0x7ff99c001110 con 0x7ff9ac0cce30 2023-10-08T00:11:42.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.079+0000 7ff9a27fc700 1 --2- 172.21.15.99:0/2882894687 >> v2:172.21.15.191:3300/0 conn(0x7ff9ac0b6580 0x7ff9ac06a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:42.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.079+0000 7ff9aa7fc700 1 -- 172.21.15.99:0/2882894687 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff998003570 con 0x7ff9ac0cce30 2023-10-08T00:11:42.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.079+0000 7ff9ab7fe700 1 -- 172.21.15.99:0/2882894687 >> v2:172.21.15.191:3300/0 conn(0x7ff9ac0b6580 msgr2=0x7ff9ac06a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:42.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.079+0000 7ff9ab7fe700 1 --2- 172.21.15.99:0/2882894687 >> v2:172.21.15.191:3300/0 conn(0x7ff9ac0b6580 0x7ff9ac06a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:42.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.079+0000 7ff9ab7fe700 1 -- 172.21.15.99:0/2882894687 >> v1:172.21.15.99:6789/0 conn(0x7ff9ac0cce30 legacy=0x7ff9ac15aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:42.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.079+0000 7ff9ab7fe700 1 -- 172.21.15.99:0/2882894687 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9ac0cd8d0 con 0x7ff9ac06a830 2023-10-08T00:11:42.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.080+0000 7ff9ab7fe700 1 --2- 172.21.15.99:0/2882894687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9ac06a830 0x7ff9ac0b5200 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac089bc0 tx=0x7ff9a4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a42b61d1a79c78f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:42.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.080+0000 7ff9aa7fc700 1 -- 172.21.15.99:0/2882894687 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a400e070 con 0x7ff9ac06a830 2023-10-08T00:11:42.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.080+0000 7ff9aa7fc700 1 -- 172.21.15.99:0/2882894687 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a400bb60 con 0x7ff9ac06a830 2023-10-08T00:11:42.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.080+0000 7ff9aa7fc700 1 -- 172.21.15.99:0/2882894687 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4012430 con 0x7ff9ac06a830 2023-10-08T00:11:42.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.080+0000 7ff9abfff700 1 -- 172.21.15.99:0/2882894687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9ac06a830 msgr2=0x7ff9ac0b5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:42.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.080+0000 7ff9abfff700 1 --2- 172.21.15.99:0/2882894687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9ac06a830 0x7ff9ac0b5200 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac089bc0 tx=0x7ff9a4009780 comp rx=0 tx=0).stop 2023-10-08T00:11:42.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.080+0000 7ff9abfff700 1 -- 172.21.15.99:0/2882894687 shutdown_connections 2023-10-08T00:11:42.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.080+0000 7ff9abfff700 1 --2- 172.21.15.99:0/2882894687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9ac06a830 0x7ff9ac0b5200 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:42.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.080+0000 7ff9abfff700 1 --2- 172.21.15.99:0/2882894687 >> v2:172.21.15.191:3300/0 conn(0x7ff9ac0b6580 0x7ff9ac06a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:42.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.080+0000 7ff9abfff700 1 -- 172.21.15.99:0/2882894687 >> 172.21.15.99:0/2882894687 conn(0x7ff9ac068cb0 msgr2=0x7ff9ac0690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:42.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.080+0000 7ff9abfff700 1 -- 172.21.15.99:0/2882894687 shutdown_connections 2023-10-08T00:11:42.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.080+0000 7ff9abfff700 1 -- 172.21.15.99:0/2882894687 wait complete. 2023-10-08T00:11:42.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.082+0000 7ff9abfff700 1 Processor -- start 2023-10-08T00:11:42.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.082+0000 7ff9abfff700 1 -- start start 2023-10-08T00:11:42.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.082+0000 7ff9abfff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9ac06a830 0x7ff9ac169ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:42.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.082+0000 7ff9abfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9ac0b6580 0x7ff9ac14e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:42.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.082+0000 7ff9abfff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9ac0cce30 0x7ff9ac14e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:42.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.082+0000 7ff9abfff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac154f90 con 0x7ff9ac0b6580 2023-10-08T00:11:42.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.082+0000 7ff9abfff700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac154cb0 con 0x7ff9ac0cce30 2023-10-08T00:11:42.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.082+0000 7ff9abfff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac153070 con 0x7ff9ac06a830 2023-10-08T00:11:42.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.082+0000 7ff9a27fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9ac0b6580 0x7ff9ac14e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:42.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.082+0000 7ff9a27fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9ac0b6580 0x7ff9ac14e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39940/0 (socket says 172.21.15.99:39940) 2023-10-08T00:11:42.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.083+0000 7ff9a27fc700 1 -- 172.21.15.99:0/1102359042 learned_addr learned my addr 172.21.15.99:0/1102359042 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:42.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.083+0000 7ff9ab7fe700 1 --2- 172.21.15.99:0/1102359042 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9ac0cce30 0x7ff9ac14e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:42.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.083+0000 7ff9aaffd700 1 --2- 172.21.15.99:0/1102359042 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9ac06a830 0x7ff9ac169ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:42.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.083+0000 7ff9a27fc700 1 -- 172.21.15.99:0/1102359042 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9ac06a830 msgr2=0x7ff9ac169ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:42.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.083+0000 7ff9a27fc700 1 --2- 172.21.15.99:0/1102359042 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9ac06a830 0x7ff9ac169ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:42.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.083+0000 7ff9a27fc700 1 -- 172.21.15.99:0/1102359042 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9ac0cce30 msgr2=0x7ff9ac14e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:42.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.083+0000 7ff9a27fc700 1 --2- 172.21.15.99:0/1102359042 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9ac0cce30 0x7ff9ac14e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:42.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.083+0000 7ff9a27fc700 1 -- 172.21.15.99:0/1102359042 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a4009a20 con 0x7ff9ac0b6580 2023-10-08T00:11:42.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.083+0000 7ff9ab7fe700 1 --2- 172.21.15.99:0/1102359042 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9ac0cce30 0x7ff9ac14e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:11:42.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.083+0000 7ff9a27fc700 1 --2- 172.21.15.99:0/1102359042 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9ac0b6580 0x7ff9ac14e450 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff99400d890 tx=0x7ff994008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:42.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.083+0000 7ff9a37fe700 1 -- 172.21.15.99:0/1102359042 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99400b600 con 0x7ff9ac0b6580 2023-10-08T00:11:42.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.083+0000 7ff9a37fe700 1 -- 172.21.15.99:0/1102359042 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff99400b760 con 0x7ff9ac0b6580 2023-10-08T00:11:42.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.083+0000 7ff9abfff700 1 -- 172.21.15.99:0/1102359042 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9ac153350 con 0x7ff9ac0b6580 2023-10-08T00:11:42.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.083+0000 7ff9a37fe700 1 -- 172.21.15.99:0/1102359042 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99400bae0 con 0x7ff9ac0b6580 2023-10-08T00:11:42.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.083+0000 7ff9abfff700 1 -- 172.21.15.99:0/1102359042 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9ac154400 con 0x7ff9ac0b6580 2023-10-08T00:11:42.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.084+0000 7ff9abfff700 1 -- 172.21.15.99:0/1102359042 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9ac089550 con 0x7ff9ac0b6580 2023-10-08T00:11:42.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.084+0000 7ff9a37fe700 1 -- 172.21.15.99:0/1102359042 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff99401e460 con 0x7ff9ac0b6580 2023-10-08T00:11:42.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.085+0000 7ff9a37fe700 1 --2- 172.21.15.99:0/1102359042 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff98c066f30 0x7ff98c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:42.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.085+0000 7ff9a37fe700 1 -- 172.21.15.99:0/1102359042 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff994084be0 con 0x7ff9ac0b6580 2023-10-08T00:11:42.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.085+0000 7ff9aaffd700 1 --2- 172.21.15.99:0/1102359042 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff98c066f30 0x7ff98c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:42.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.086+0000 7ff9aaffd700 1 --2- 172.21.15.99:0/1102359042 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff98c066f30 0x7ff98c0693e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff99c001270 tx=0x7ff99c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:42.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.089+0000 7ff9a37fe700 1 -- 172.21.15.99:0/1102359042 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff994051e50 con 0x7ff9ac0b6580 2023-10-08T00:11:42.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:42.316+0000 7ff9abfff700 1 -- 172.21.15.99:0/1102359042 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff9ac0b4440 con 0x7ff9ac0b6580 2023-10-08T00:11:43.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.314+0000 7ff9a37fe700 1 -- 172.21.15.99:0/1102359042 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff994055470 con 0x7ff9ac0b6580 2023-10-08T00:11:43.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.317+0000 7ff9abfff700 1 -- 172.21.15.99:0/1102359042 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff98c066f30 msgr2=0x7ff98c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:43.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.317+0000 7ff9abfff700 1 --2- 172.21.15.99:0/1102359042 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff98c066f30 0x7ff98c0693e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff99c001270 tx=0x7ff99c009040 comp rx=0 tx=0).stop 2023-10-08T00:11:43.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.317+0000 7ff9abfff700 1 -- 172.21.15.99:0/1102359042 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9ac0b6580 msgr2=0x7ff9ac14e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:43.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.317+0000 7ff9abfff700 1 --2- 172.21.15.99:0/1102359042 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9ac0b6580 0x7ff9ac14e450 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff99400d890 tx=0x7ff994008040 comp rx=0 tx=0).stop 2023-10-08T00:11:43.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.317+0000 7ff9abfff700 1 -- 172.21.15.99:0/1102359042 shutdown_connections 2023-10-08T00:11:43.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.317+0000 7ff9abfff700 1 --2- 172.21.15.99:0/1102359042 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff98c066f30 0x7ff98c0693e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:43.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.317+0000 7ff9abfff700 1 --2- 172.21.15.99:0/1102359042 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9ac0cce30 0x7ff9ac14e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:43.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.317+0000 7ff9abfff700 1 --2- 172.21.15.99:0/1102359042 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9ac06a830 0x7ff9ac169ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:43.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.317+0000 7ff9abfff700 1 --2- 172.21.15.99:0/1102359042 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9ac0b6580 0x7ff9ac14e450 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:43.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.317+0000 7ff9abfff700 1 -- 172.21.15.99:0/1102359042 >> 172.21.15.99:0/1102359042 conn(0x7ff9ac068cb0 msgr2=0x7ff9ac069320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:43.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.317+0000 7ff9abfff700 1 -- 172.21.15.99:0/1102359042 shutdown_connections 2023-10-08T00:11:43.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.317+0000 7ff9abfff700 1 -- 172.21.15.99:0/1102359042 wait complete. 2023-10-08T00:11:43.332 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T00:11:43.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.453+0000 7fa29f5b7700 1 Processor -- start 2023-10-08T00:11:43.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.454+0000 7fa29f5b7700 1 -- start start 2023-10-08T00:11:43.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.454+0000 7fa29f5b7700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa29815a290 0x7fa29806a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:43.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.454+0000 7fa29f5b7700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa29806aae0 0x7fa2980cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:43.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.454+0000 7fa29f5b7700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2980b0790 con 0x7fa298159ec0 2023-10-08T00:11:43.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.454+0000 7fa29f5b7700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa29806af20 con 0x7fa29806aae0 2023-10-08T00:11:43.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.454+0000 7fa29f5b7700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2980ce1f0 con 0x7fa29815a290 2023-10-08T00:11:43.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.454+0000 7fa29db54700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa29806aae0 0x7fa2980cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:43.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.455+0000 7fa29db54700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa29806aae0 0x7fa2980cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50498/0 (socket says 172.21.15.99:50498) 2023-10-08T00:11:43.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.455+0000 7fa29db54700 1 -- 172.21.15.99:0/3469216577 learned_addr learned my addr 172.21.15.99:0/3469216577 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:43.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.455+0000 7fa29cb52700 1 --2- 172.21.15.99:0/3469216577 >> v2:172.21.15.191:3300/0 conn(0x7fa29815a290 0x7fa29806a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:43.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.455+0000 7fa29db54700 1 -- 172.21.15.99:0/3469216577 >> v2:172.21.15.191:3300/0 conn(0x7fa29815a290 msgr2=0x7fa29806a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:43.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.455+0000 7fa29db54700 1 --2- 172.21.15.99:0/3469216577 >> v2:172.21.15.191:3300/0 conn(0x7fa29815a290 0x7fa29806a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:43.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.455+0000 7fa29db54700 1 -- 172.21.15.99:0/3469216577 >> v1:172.21.15.191:6789/0 conn(0x7fa298159ec0 legacy=0x7fa2980af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:43.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.455+0000 7fa29db54700 1 -- 172.21.15.99:0/3469216577 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2980b0b20 con 0x7fa29806aae0 2023-10-08T00:11:43.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.455+0000 7fa29db54700 1 --2- 172.21.15.99:0/3469216577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa29806aae0 0x7fa2980cf590 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa298089910 tx=0x7fa290009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa9ea555a07aa012 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:43.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.455+0000 7fa28ffff700 1 -- 172.21.15.99:0/3469216577 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa290009e70 con 0x7fa29806aae0 2023-10-08T00:11:43.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.455+0000 7fa28ffff700 1 -- 172.21.15.99:0/3469216577 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa29000b860 con 0x7fa29806aae0 2023-10-08T00:11:43.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.455+0000 7fa28ffff700 1 -- 172.21.15.99:0/3469216577 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29000bc00 con 0x7fa29806aae0 2023-10-08T00:11:43.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.456+0000 7fa29f5b7700 1 -- 172.21.15.99:0/3469216577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa29806aae0 msgr2=0x7fa2980cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:43.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.456+0000 7fa29f5b7700 1 --2- 172.21.15.99:0/3469216577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa29806aae0 0x7fa2980cf590 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa298089910 tx=0x7fa290009250 comp rx=0 tx=0).stop 2023-10-08T00:11:43.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.456+0000 7fa29f5b7700 1 -- 172.21.15.99:0/3469216577 shutdown_connections 2023-10-08T00:11:43.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.456+0000 7fa29f5b7700 1 --2- 172.21.15.99:0/3469216577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa29806aae0 0x7fa2980cf590 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:43.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.456+0000 7fa29f5b7700 1 --2- 172.21.15.99:0/3469216577 >> v2:172.21.15.191:3300/0 conn(0x7fa29815a290 0x7fa29806a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:43.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.456+0000 7fa29f5b7700 1 -- 172.21.15.99:0/3469216577 >> 172.21.15.99:0/3469216577 conn(0x7fa2980d1880 msgr2=0x7fa2980cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:43.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.456+0000 7fa29f5b7700 1 -- 172.21.15.99:0/3469216577 shutdown_connections 2023-10-08T00:11:43.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.456+0000 7fa29f5b7700 1 -- 172.21.15.99:0/3469216577 wait complete. 2023-10-08T00:11:43.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.457+0000 7fa29f5b7700 1 Processor -- start 2023-10-08T00:11:43.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.457+0000 7fa29f5b7700 1 -- start start 2023-10-08T00:11:43.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.458+0000 7fa29f5b7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa29806aae0 0x7fa2980c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:43.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.458+0000 7fa29f5b7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa298159ec0 0x7fa2980ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:43.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.458+0000 7fa29f5b7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa29815a290 0x7fa298156cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:43.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.458+0000 7fa29f5b7700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa298156820 con 0x7fa29806aae0 2023-10-08T00:11:43.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.458+0000 7fa29f5b7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2980cb240 con 0x7fa298159ec0 2023-10-08T00:11:43.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.458+0000 7fa29f5b7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2980cb520 con 0x7fa29815a290 2023-10-08T00:11:43.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.458+0000 7fa29d353700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa29806aae0 0x7fa2980c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:43.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.458+0000 7fa29d353700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa29806aae0 0x7fa2980c8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50500/0 (socket says 172.21.15.99:50500) 2023-10-08T00:11:43.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.458+0000 7fa29d353700 1 -- 172.21.15.99:0/2874944056 learned_addr learned my addr 172.21.15.99:0/2874944056 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:43.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.458+0000 7fa29cb52700 1 --2- 172.21.15.99:0/2874944056 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa298159ec0 0x7fa2980ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:43.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.458+0000 7fa29db54700 1 --2- 172.21.15.99:0/2874944056 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa29815a290 0x7fa298156cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:43.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.458+0000 7fa29d353700 1 -- 172.21.15.99:0/2874944056 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa29815a290 msgr2=0x7fa298156cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:43.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.458+0000 7fa29d353700 1 --2- 172.21.15.99:0/2874944056 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa29815a290 0x7fa298156cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:43.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.458+0000 7fa29d353700 1 -- 172.21.15.99:0/2874944056 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa298159ec0 msgr2=0x7fa2980ca9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:43.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.458+0000 7fa29d353700 1 --2- 172.21.15.99:0/2874944056 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa298159ec0 0x7fa2980ca9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:43.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.458+0000 7fa29d353700 1 -- 172.21.15.99:0/2874944056 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2900094f0 con 0x7fa29806aae0 2023-10-08T00:11:43.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.458+0000 7fa29cb52700 1 --2- 172.21.15.99:0/2874944056 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa298159ec0 0x7fa2980ca9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:11:43.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.458+0000 7fa29db54700 1 --2- 172.21.15.99:0/2874944056 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa29815a290 0x7fa298156cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:11:43.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.458+0000 7fa29d353700 1 --2- 172.21.15.99:0/2874944056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa29806aae0 0x7fa2980c8090 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa29400cb10 tx=0x7fa2940017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:43.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.458+0000 7fa28d7fa700 1 -- 172.21.15.99:0/2874944056 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29400a760 con 0x7fa29806aae0 2023-10-08T00:11:43.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.458+0000 7fa28d7fa700 1 -- 172.21.15.99:0/2874944056 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa29400a8e0 con 0x7fa29806aae0 2023-10-08T00:11:43.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.458+0000 7fa29f5b7700 1 -- 172.21.15.99:0/2874944056 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2981564b0 con 0x7fa29806aae0 2023-10-08T00:11:43.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.458+0000 7fa28d7fa700 1 -- 172.21.15.99:0/2874944056 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29400aca0 con 0x7fa29806aae0 2023-10-08T00:11:43.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.458+0000 7fa29f5b7700 1 -- 172.21.15.99:0/2874944056 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2981571f0 con 0x7fa29806aae0 2023-10-08T00:11:43.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.459+0000 7fa29f5b7700 1 -- 172.21.15.99:0/2874944056 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa298156b00 con 0x7fa29806aae0 2023-10-08T00:11:43.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.460+0000 7fa28d7fa700 1 -- 172.21.15.99:0/2874944056 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa294003030 con 0x7fa29806aae0 2023-10-08T00:11:43.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.460+0000 7fa28d7fa700 1 --2- 172.21.15.99:0/2874944056 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa27c066f30 0x7fa27c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:43.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.460+0000 7fa28d7fa700 1 -- 172.21.15.99:0/2874944056 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa2940845f0 con 0x7fa29806aae0 2023-10-08T00:11:43.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.460+0000 7fa29cb52700 1 --2- 172.21.15.99:0/2874944056 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa27c066f30 0x7fa27c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:43.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.461+0000 7fa29cb52700 1 --2- 172.21.15.99:0/2874944056 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa27c066f30 0x7fa27c0693e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa2980af7e0 tx=0x7fa288008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:43.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.464+0000 7fa28d7fa700 1 -- 172.21.15.99:0/2874944056 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa294051860 con 0x7fa29806aae0 2023-10-08T00:11:43.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:43.689+0000 7fa29f5b7700 1 -- 172.21.15.99:0/2874944056 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa29804c3c0 con 0x7fa29806aae0 2023-10-08T00:11:44.272 INFO:tasks.ceph.mds.c.smithi099.stderr:2023-10-08T00:11:44.272+0000 7fea12398700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T00:11:44.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.319+0000 7fa28d7fa700 1 -- 172.21.15.99:0/2874944056 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa294054e80 con 0x7fa29806aae0 2023-10-08T00:11:44.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.321+0000 7fa29f5b7700 1 -- 172.21.15.99:0/2874944056 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa27c066f30 msgr2=0x7fa27c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:44.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.321+0000 7fa29f5b7700 1 --2- 172.21.15.99:0/2874944056 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa27c066f30 0x7fa27c0693e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa2980af7e0 tx=0x7fa288008040 comp rx=0 tx=0).stop 2023-10-08T00:11:44.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.321+0000 7fa29f5b7700 1 -- 172.21.15.99:0/2874944056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa29806aae0 msgr2=0x7fa2980c8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:44.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.321+0000 7fa29f5b7700 1 --2- 172.21.15.99:0/2874944056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa29806aae0 0x7fa2980c8090 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa29400cb10 tx=0x7fa2940017b0 comp rx=0 tx=0).stop 2023-10-08T00:11:44.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.321+0000 7fa29f5b7700 1 -- 172.21.15.99:0/2874944056 shutdown_connections 2023-10-08T00:11:44.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.321+0000 7fa29f5b7700 1 --2- 172.21.15.99:0/2874944056 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa27c066f30 0x7fa27c0693e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:44.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.321+0000 7fa29f5b7700 1 --2- 172.21.15.99:0/2874944056 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa29815a290 0x7fa298156cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:44.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.321+0000 7fa29f5b7700 1 --2- 172.21.15.99:0/2874944056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa29806aae0 0x7fa2980c8090 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:44.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.321+0000 7fa29f5b7700 1 --2- 172.21.15.99:0/2874944056 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa298159ec0 0x7fa2980ca9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:44.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.321+0000 7fa29f5b7700 1 -- 172.21.15.99:0/2874944056 >> 172.21.15.99:0/2874944056 conn(0x7fa2980d1880 msgr2=0x7fa2980b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:44.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.321+0000 7fa29f5b7700 1 -- 172.21.15.99:0/2874944056 shutdown_connections 2023-10-08T00:11:44.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.321+0000 7fa29f5b7700 1 -- 172.21.15.99:0/2874944056 wait complete. 2023-10-08T00:11:44.334 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:11:44.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.450+0000 7f1f85d74700 1 Processor -- start 2023-10-08T00:11:44.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.450+0000 7f1f85d74700 1 -- start start 2023-10-08T00:11:44.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.451+0000 7f1f85d74700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1f800b50c0 0x7f1f8015a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:44.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.451+0000 7f1f85d74700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1f800cf810 0x7f1f800cb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:44.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.451+0000 7f1f85d74700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f800d1960 con 0x7f1f8015a030 2023-10-08T00:11:44.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.451+0000 7f1f85d74700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f800d0610 con 0x7f1f800b50c0 2023-10-08T00:11:44.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.451+0000 7f1f85d74700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f8015a940 con 0x7f1f800cf810 2023-10-08T00:11:44.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.451+0000 7f1f7ffff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1f800b50c0 0x7f1f8015a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:44.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.451+0000 7f1f7ffff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1f800b50c0 0x7f1f8015a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50514/0 (socket says 172.21.15.99:50514) 2023-10-08T00:11:44.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.451+0000 7f1f84d72700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f1f8015a030 0x7f1f800cf100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:54166/0 (socket says 172.21.15.99:54166) 2023-10-08T00:11:44.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.451+0000 7f1f7ffff700 1 -- 172.21.15.99:0/1208421128 learned_addr learned my addr 172.21.15.99:0/1208421128 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:44.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.451+0000 7f1f7ffff700 1 -- 172.21.15.99:0/1208421128 >> v2:172.21.15.191:3300/0 conn(0x7f1f800cf810 msgr2=0x7f1f800cb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:44.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.451+0000 7f1f7ffff700 1 --2- 172.21.15.99:0/1208421128 >> v2:172.21.15.191:3300/0 conn(0x7f1f800cf810 0x7f1f800cb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:44.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.451+0000 7f1f7ffff700 1 -- 172.21.15.99:0/1208421128 >> v1:172.21.15.99:6789/0 conn(0x7f1f8015a030 legacy=0x7f1f800cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:44.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.451+0000 7f1f7ffff700 1 -- 172.21.15.99:0/1208421128 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f800cfd70 con 0x7f1f800b50c0 2023-10-08T00:11:44.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.451+0000 7f1f7ffff700 1 --2- 172.21.15.99:0/1208421128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f800b50c0 0x7f1f8015a400 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1f800cd640 tx=0x7f1f6c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58db4eb2e8abba7c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:44.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.451+0000 7f1f7f7fe700 1 -- 172.21.15.99:0/1208421128 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c00b860 con 0x7f1f800b50c0 2023-10-08T00:11:44.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.452+0000 7f1f7f7fe700 1 -- 172.21.15.99:0/1208421128 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f6c00b9c0 con 0x7f1f800b50c0 2023-10-08T00:11:44.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.452+0000 7f1f7f7fe700 1 -- 172.21.15.99:0/1208421128 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c00bd60 con 0x7f1f800b50c0 2023-10-08T00:11:44.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.452+0000 7f1f85d74700 1 -- 172.21.15.99:0/1208421128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f800b50c0 msgr2=0x7f1f8015a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:44.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.452+0000 7f1f85d74700 1 --2- 172.21.15.99:0/1208421128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f800b50c0 0x7f1f8015a400 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1f800cd640 tx=0x7f1f6c009250 comp rx=0 tx=0).stop 2023-10-08T00:11:44.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.452+0000 7f1f85d74700 1 -- 172.21.15.99:0/1208421128 shutdown_connections 2023-10-08T00:11:44.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.452+0000 7f1f85d74700 1 --2- 172.21.15.99:0/1208421128 >> v2:172.21.15.191:3300/0 conn(0x7f1f800cf810 0x7f1f800cb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:44.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.452+0000 7f1f85d74700 1 --2- 172.21.15.99:0/1208421128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f800b50c0 0x7f1f8015a400 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:44.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.452+0000 7f1f85d74700 1 -- 172.21.15.99:0/1208421128 >> 172.21.15.99:0/1208421128 conn(0x7f1f80067e40 msgr2=0x7f1f800d0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:44.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.452+0000 7f1f85d74700 1 -- 172.21.15.99:0/1208421128 shutdown_connections 2023-10-08T00:11:44.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.452+0000 7f1f85d74700 1 -- 172.21.15.99:0/1208421128 wait complete. 2023-10-08T00:11:44.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.453+0000 7f1f85d74700 1 Processor -- start 2023-10-08T00:11:44.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.453+0000 7f1f85d74700 1 -- start start 2023-10-08T00:11:44.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.454+0000 7f1f85d74700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f800b50c0 0x7f1f80158230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:44.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.454+0000 7f1f85d74700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f800cf810 0x7f1f8009a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:44.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.454+0000 7f1f85d74700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f8015a030 0x7f1f8009ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:44.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.454+0000 7f1f85d74700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8015b4a0 con 0x7f1f8015a030 2023-10-08T00:11:44.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.454+0000 7f1f85d74700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8015bce0 con 0x7f1f800b50c0 2023-10-08T00:11:44.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.454+0000 7f1f85d74700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8009cfb0 con 0x7f1f800cf810 2023-10-08T00:11:44.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.454+0000 7f1f85573700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f8015a030 0x7f1f8009ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:44.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.454+0000 7f1f85573700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f8015a030 0x7f1f8009ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50530/0 (socket says 172.21.15.99:50530) 2023-10-08T00:11:44.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.454+0000 7f1f85573700 1 -- 172.21.15.99:0/410708349 learned_addr learned my addr 172.21.15.99:0/410708349 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:44.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.454+0000 7f1f84d72700 1 --2- 172.21.15.99:0/410708349 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f800b50c0 0x7f1f80158230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:44.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.454+0000 7f1f7ffff700 1 --2- 172.21.15.99:0/410708349 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f800cf810 0x7f1f8009a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:44.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.454+0000 7f1f85573700 1 -- 172.21.15.99:0/410708349 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f800cf810 msgr2=0x7f1f8009a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:44.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.454+0000 7f1f85573700 1 --2- 172.21.15.99:0/410708349 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f800cf810 0x7f1f8009a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:44.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.454+0000 7f1f85573700 1 -- 172.21.15.99:0/410708349 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f800b50c0 msgr2=0x7f1f80158230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:44.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.454+0000 7f1f85573700 1 --2- 172.21.15.99:0/410708349 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f800b50c0 0x7f1f80158230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:44.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.454+0000 7f1f85573700 1 -- 172.21.15.99:0/410708349 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f6c0094f0 con 0x7f1f8015a030 2023-10-08T00:11:44.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.455+0000 7f1f85573700 1 --2- 172.21.15.99:0/410708349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f8015a030 0x7f1f8009ad50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1f7800bc30 tx=0x7f1f78009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:44.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.455+0000 7f1f7cff9700 1 -- 172.21.15.99:0/410708349 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f780092e0 con 0x7f1f8015a030 2023-10-08T00:11:44.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.455+0000 7f1f7cff9700 1 -- 172.21.15.99:0/410708349 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f7800c7a0 con 0x7f1f8015a030 2023-10-08T00:11:44.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.455+0000 7f1f85d74700 1 -- 172.21.15.99:0/410708349 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f8015b690 con 0x7f1f8015a030 2023-10-08T00:11:44.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.455+0000 7f1f7cff9700 1 -- 172.21.15.99:0/410708349 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7800cb80 con 0x7f1f8015a030 2023-10-08T00:11:44.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.455+0000 7f1f85d74700 1 -- 172.21.15.99:0/410708349 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f80158770 con 0x7f1f8015a030 2023-10-08T00:11:44.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.456+0000 7f1f85d74700 1 -- 172.21.15.99:0/410708349 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f80089300 con 0x7f1f8015a030 2023-10-08T00:11:44.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.456+0000 7f1f7cff9700 1 -- 172.21.15.99:0/410708349 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1f78012070 con 0x7f1f8015a030 2023-10-08T00:11:44.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.457+0000 7f1f7cff9700 1 --2- 172.21.15.99:0/410708349 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1f64066fb0 0x7f1f64069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:44.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.457+0000 7f1f7cff9700 1 -- 172.21.15.99:0/410708349 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1f78084260 con 0x7f1f8015a030 2023-10-08T00:11:44.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.457+0000 7f1f84d72700 1 --2- 172.21.15.99:0/410708349 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1f64066fb0 0x7f1f64069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:44.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.457+0000 7f1f84d72700 1 --2- 172.21.15.99:0/410708349 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1f64066fb0 0x7f1f64069460 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1f74002790 tx=0x7f1f74009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:44.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.461+0000 7f1f7cff9700 1 -- 172.21.15.99:0/410708349 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1f78051430 con 0x7f1f8015a030 2023-10-08T00:11:44.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.684+0000 7f1f85d74700 1 -- 172.21.15.99:0/410708349 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f800890a0 con 0x7f1f8015a030 2023-10-08T00:11:44.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.684+0000 7f1f7cff9700 1 -- 172.21.15.99:0/410708349 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4600 (secure 0 0 0) 0x7f1f78054a50 con 0x7f1f8015a030 2023-10-08T00:11:44.685 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:11:44.685 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4852,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/824084995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":824084995},{"type":"v1","addr":"172.21.15.191:6837","nonce":824084995}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4855,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/3349126920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3349126920},{"type":"v1","addr":"172.21.15.191:6836","nonce":3349126920}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4899,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/1004180438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1004180438},{"type":"v1","addr":"172.21.15.99:6834","nonce":1004180438}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:11:40.672879+0000","modified":"2023-10-08T00:11:44.314526+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4902},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4902":{"gid":4902,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/4189095717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":4189095717},{"type":"v1","addr":"172.21.15.99:6835","nonce":4189095717}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-08T00:11:44.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.686+0000 7f1f85d74700 1 -- 172.21.15.99:0/410708349 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1f64066fb0 msgr2=0x7f1f64069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:44.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.686+0000 7f1f85d74700 1 --2- 172.21.15.99:0/410708349 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1f64066fb0 0x7f1f64069460 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1f74002790 tx=0x7f1f74009040 comp rx=0 tx=0).stop 2023-10-08T00:11:44.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.686+0000 7f1f85d74700 1 -- 172.21.15.99:0/410708349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f8015a030 msgr2=0x7f1f8009ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:44.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.686+0000 7f1f85d74700 1 --2- 172.21.15.99:0/410708349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f8015a030 0x7f1f8009ad50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1f7800bc30 tx=0x7f1f78009840 comp rx=0 tx=0).stop 2023-10-08T00:11:44.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.686+0000 7f1f85d74700 1 -- 172.21.15.99:0/410708349 shutdown_connections 2023-10-08T00:11:44.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.686+0000 7f1f85d74700 1 --2- 172.21.15.99:0/410708349 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1f64066fb0 0x7f1f64069460 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:44.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.686+0000 7f1f85d74700 1 --2- 172.21.15.99:0/410708349 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f800b50c0 0x7f1f80158230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:44.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.686+0000 7f1f85d74700 1 --2- 172.21.15.99:0/410708349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f8015a030 0x7f1f8009ad50 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:44.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.686+0000 7f1f85d74700 1 --2- 172.21.15.99:0/410708349 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f800cf810 0x7f1f8009a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:44.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.686+0000 7f1f85d74700 1 -- 172.21.15.99:0/410708349 >> 172.21.15.99:0/410708349 conn(0x7f1f80067e40 msgr2=0x7f1f801590f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:44.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.687+0000 7f1f85d74700 1 -- 172.21.15.99:0/410708349 shutdown_connections 2023-10-08T00:11:44.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.687+0000 7f1f85d74700 1 -- 172.21.15.99:0/410708349 wait complete. 2023-10-08T00:11:44.688 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 17 2023-10-08T00:11:44.699 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:11:44.813 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.812+0000 7f41315ec700 1 Processor -- start 2023-10-08T00:11:44.813 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.812+0000 7f41315ec700 1 -- start start 2023-10-08T00:11:44.816 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.813+0000 7f41315ec700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f412c157870 0x7f412c15b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:44.816 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.813+0000 7f41315ec700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f412c15cbb0 con 0x7f412c157c40 2023-10-08T00:11:44.816 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.813+0000 7f41315ec700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f412c15bfa0 con 0x7f412c15a620 2023-10-08T00:11:44.816 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.813+0000 7f41315ec700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f412c15c280 con 0x7f412c157870 2023-10-08T00:11:44.816 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.813+0000 7f412b7fe700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f412c157c40 0x7f412c158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:54182/0 (socket says 172.21.15.99:54182) 2023-10-08T00:11:44.816 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.813+0000 7f412b7fe700 1 -- 172.21.15.99:0/2407330436 learned_addr learned my addr 172.21.15.99:0/2407330436 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:44.816 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.813+0000 7f412affd700 1 --2- 172.21.15.99:0/2407330436 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f412c157870 0x7f412c15b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:44.816 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.814+0000 7f412a7fc700 1 -- 172.21.15.99:0/2407330436 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2762780233 0 0) 0x7f412c15cbb0 con 0x7f412c157c40 2023-10-08T00:11:44.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.814+0000 7f412a7fc700 1 -- 172.21.15.99:0/2407330436 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4118003570 con 0x7f412c157c40 2023-10-08T00:11:44.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.814+0000 7f412a7fc700 1 -- 172.21.15.99:0/2407330436 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1039346075 0 0) 0x7f412c15bfa0 con 0x7f412c15a620 2023-10-08T00:11:44.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.814+0000 7f412a7fc700 1 -- 172.21.15.99:0/2407330436 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f412c15cbb0 con 0x7f412c15a620 2023-10-08T00:11:44.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.814+0000 7f412affd700 1 -- 172.21.15.99:0/2407330436 >> v1:172.21.15.191:6789/0 conn(0x7f412c15a620 legacy=0x7f412c15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:44.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.814+0000 7f412affd700 1 -- 172.21.15.99:0/2407330436 >> v1:172.21.15.99:6789/0 conn(0x7f412c157c40 legacy=0x7f412c158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:44.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.814+0000 7f412a7fc700 1 -- 172.21.15.99:0/2407330436 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3169577709 0 0) 0x7f4118003570 con 0x7f412c157c40 2023-10-08T00:11:44.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.814+0000 7f412affd700 1 -- 172.21.15.99:0/2407330436 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f412c15c560 con 0x7f412c157870 2023-10-08T00:11:44.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.814+0000 7f412affd700 1 --2- 172.21.15.99:0/2407330436 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f412c157870 0x7f412c15b6a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f411c009910 tx=0x7f411c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5790baf65f838f2c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:44.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.814+0000 7f412a7fc700 1 -- 172.21.15.99:0/2407330436 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41240017f0 con 0x7f412c157870 2023-10-08T00:11:44.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.814+0000 7f412a7fc700 1 -- 172.21.15.99:0/2407330436 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f411c011420 con 0x7f412c157870 2023-10-08T00:11:44.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.814+0000 7f412a7fc700 1 -- 172.21.15.99:0/2407330436 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411c0117c0 con 0x7f412c157870 2023-10-08T00:11:44.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.815+0000 7f41315ec700 1 -- 172.21.15.99:0/2407330436 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f412c157870 msgr2=0x7f412c15b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:44.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.815+0000 7f41315ec700 1 --2- 172.21.15.99:0/2407330436 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f412c157870 0x7f412c15b6a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f411c009910 tx=0x7f411c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:44.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.815+0000 7f41315ec700 1 -- 172.21.15.99:0/2407330436 shutdown_connections 2023-10-08T00:11:44.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.815+0000 7f41315ec700 1 --2- 172.21.15.99:0/2407330436 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f412c157870 0x7f412c15b6a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:44.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.815+0000 7f41315ec700 1 -- 172.21.15.99:0/2407330436 >> 172.21.15.99:0/2407330436 conn(0x7f412c068ce0 msgr2=0x7f412c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:44.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.815+0000 7f41315ec700 1 -- 172.21.15.99:0/2407330436 shutdown_connections 2023-10-08T00:11:44.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.815+0000 7f41315ec700 1 -- 172.21.15.99:0/2407330436 wait complete. 2023-10-08T00:11:44.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.816+0000 7f41315ec700 1 Processor -- start 2023-10-08T00:11:44.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.816+0000 7f41315ec700 1 -- start start 2023-10-08T00:11:44.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.817+0000 7f41315ec700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f412c157870 0x7f412c14ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:44.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.817+0000 7f41315ec700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f412c157c40 0x7f412c14d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:44.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.817+0000 7f41315ec700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f412c15a620 0x7f412c16a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:44.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.817+0000 7f41315ec700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412c14fe70 con 0x7f412c157c40 2023-10-08T00:11:44.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.817+0000 7f41315ec700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412c1543c0 con 0x7f412c15a620 2023-10-08T00:11:44.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.817+0000 7f41315ec700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f412c14f260 con 0x7f412c157870 2023-10-08T00:11:44.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.817+0000 7f41227fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f412c157c40 0x7f412c14d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:44.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.817+0000 7f41227fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f412c157c40 0x7f412c14d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50538/0 (socket says 172.21.15.99:50538) 2023-10-08T00:11:44.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.817+0000 7f41227fc700 1 -- 172.21.15.99:0/3032314076 learned_addr learned my addr 172.21.15.99:0/3032314076 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:44.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.817+0000 7f412b7fe700 1 --2- 172.21.15.99:0/3032314076 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f412c15a620 0x7f412c16a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:44.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.817+0000 7f412affd700 1 --2- 172.21.15.99:0/3032314076 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f412c157870 0x7f412c14ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:44.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.817+0000 7f41227fc700 1 -- 172.21.15.99:0/3032314076 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f412c157870 msgr2=0x7f412c14ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:44.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.817+0000 7f41227fc700 1 --2- 172.21.15.99:0/3032314076 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f412c157870 0x7f412c14ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:44.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.817+0000 7f41227fc700 1 -- 172.21.15.99:0/3032314076 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f412c15a620 msgr2=0x7f412c16a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:44.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.817+0000 7f41227fc700 1 --2- 172.21.15.99:0/3032314076 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f412c15a620 0x7f412c16a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:44.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.817+0000 7f41227fc700 1 -- 172.21.15.99:0/3032314076 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f411c009420 con 0x7f412c157c40 2023-10-08T00:11:44.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.817+0000 7f412affd700 1 --2- 172.21.15.99:0/3032314076 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f412c157870 0x7f412c14ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:11:44.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.817+0000 7f412b7fe700 1 --2- 172.21.15.99:0/3032314076 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f412c15a620 0x7f412c16a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:11:44.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.817+0000 7f41227fc700 1 --2- 172.21.15.99:0/3032314076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f412c157c40 0x7f412c14d1d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f411400c580 tx=0x7f4114007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:44.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.818+0000 7f41237fe700 1 -- 172.21.15.99:0/3032314076 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4114010510 con 0x7f412c157c40 2023-10-08T00:11:44.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.818+0000 7f41237fe700 1 -- 172.21.15.99:0/3032314076 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4114010670 con 0x7f412c157c40 2023-10-08T00:11:44.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.818+0000 7f41315ec700 1 -- 172.21.15.99:0/3032314076 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f412c14f540 con 0x7f412c157c40 2023-10-08T00:11:44.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.818+0000 7f41237fe700 1 -- 172.21.15.99:0/3032314076 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4114010a30 con 0x7f412c157c40 2023-10-08T00:11:44.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.818+0000 7f41315ec700 1 -- 172.21.15.99:0/3032314076 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f412c14fb00 con 0x7f412c157c40 2023-10-08T00:11:44.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.819+0000 7f41315ec700 1 -- 172.21.15.99:0/3032314076 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f412c14ffe0 con 0x7f412c157c40 2023-10-08T00:11:44.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.819+0000 7f41237fe700 1 -- 172.21.15.99:0/3032314076 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f411401b400 con 0x7f412c157c40 2023-10-08T00:11:44.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.819+0000 7f41237fe700 1 --2- 172.21.15.99:0/3032314076 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f410c066f30 0x7f410c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:44.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.819+0000 7f41237fe700 1 -- 172.21.15.99:0/3032314076 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4114084170 con 0x7f412c157c40 2023-10-08T00:11:44.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.820+0000 7f412affd700 1 --2- 172.21.15.99:0/3032314076 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f410c066f30 0x7f410c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:44.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.820+0000 7f412affd700 1 --2- 172.21.15.99:0/3032314076 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f410c066f30 0x7f410c0693e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f411c00a010 tx=0x7f411c016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:44.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:44.824+0000 7f41237fe700 1 -- 172.21.15.99:0/3032314076 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f41140513e0 con 0x7f412c157c40 2023-10-08T00:11:45.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.006+0000 7f41315ec700 1 -- 172.21.15.99:0/3032314076 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f412c04c3c0 con 0x7f412c157c40 2023-10-08T00:11:45.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.007+0000 7f41237fe700 1 -- 172.21.15.99:0/3032314076 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f4114054a00 con 0x7f412c157c40 2023-10-08T00:11:45.008 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:11:45.008 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":31,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:11:42.057476+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:11:38.415553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-08T00:11:39.634576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:11:45.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.009+0000 7f41315ec700 1 -- 172.21.15.99:0/3032314076 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f410c066f30 msgr2=0x7f410c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:45.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.009+0000 7f41315ec700 1 --2- 172.21.15.99:0/3032314076 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f410c066f30 0x7f410c0693e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f411c00a010 tx=0x7f411c016000 comp rx=0 tx=0).stop 2023-10-08T00:11:45.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.009+0000 7f41315ec700 1 -- 172.21.15.99:0/3032314076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f412c157c40 msgr2=0x7f412c14d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:45.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.009+0000 7f41315ec700 1 --2- 172.21.15.99:0/3032314076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f412c157c40 0x7f412c14d1d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f411400c580 tx=0x7f4114007e10 comp rx=0 tx=0).stop 2023-10-08T00:11:45.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.009+0000 7f41315ec700 1 -- 172.21.15.99:0/3032314076 shutdown_connections 2023-10-08T00:11:45.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.009+0000 7f41315ec700 1 --2- 172.21.15.99:0/3032314076 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f410c066f30 0x7f410c0693e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:45.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.009+0000 7f41315ec700 1 --2- 172.21.15.99:0/3032314076 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f412c15a620 0x7f412c16a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:45.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.009+0000 7f41315ec700 1 --2- 172.21.15.99:0/3032314076 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f412c157870 0x7f412c14ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:45.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.009+0000 7f41315ec700 1 --2- 172.21.15.99:0/3032314076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f412c157c40 0x7f412c14d1d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:45.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.009+0000 7f41315ec700 1 -- 172.21.15.99:0/3032314076 >> 172.21.15.99:0/3032314076 conn(0x7f412c068ce0 msgr2=0x7f412c0b5860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:45.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.009+0000 7f41315ec700 1 -- 172.21.15.99:0/3032314076 shutdown_connections 2023-10-08T00:11:45.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.009+0000 7f41315ec700 1 -- 172.21.15.99:0/3032314076 wait complete. 2023-10-08T00:11:45.021 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T00:11:45.022 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T00:11:45.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.134+0000 7f1adac77700 1 Processor -- start 2023-10-08T00:11:45.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.134+0000 7f1adac77700 1 -- start start 2023-10-08T00:11:45.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.135+0000 7f1adac77700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1ad41510f0 0x7f1ad406ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:45.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.135+0000 7f1adac77700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ad4070700 0x7f1ad4070b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:45.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.135+0000 7f1adac77700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ad4071080 con 0x7f1ad40c9af0 2023-10-08T00:11:45.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.135+0000 7f1adac77700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ad40ca0b0 con 0x7f1ad41510f0 2023-10-08T00:11:45.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.135+0000 7f1adac77700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad4068a10 con 0x7f1ad4070700 2023-10-08T00:11:45.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.135+0000 7f1ad8a13700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1ad41510f0 0x7f1ad406ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:45.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.135+0000 7f1acbfff700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f1ad40c9af0 0x7f1ad415dd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:54188/0 (socket says 172.21.15.99:54188) 2023-10-08T00:11:45.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.135+0000 7f1acbfff700 1 -- 172.21.15.99:0/3098064594 learned_addr learned my addr 172.21.15.99:0/3098064594 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:45.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.136+0000 7f1ad9214700 1 --2- 172.21.15.99:0/3098064594 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ad4070700 0x7f1ad4070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:45.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.136+0000 7f1ad8a13700 1 -- 172.21.15.99:0/3098064594 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ad4070700 msgr2=0x7f1ad4070b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:45.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.136+0000 7f1ad8a13700 1 --2- 172.21.15.99:0/3098064594 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ad4070700 0x7f1ad4070b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:45.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.136+0000 7f1ad8a13700 1 -- 172.21.15.99:0/3098064594 >> v1:172.21.15.99:6789/0 conn(0x7f1ad40c9af0 legacy=0x7f1ad415dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:45.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.136+0000 7f1acb7fe700 1 -- 172.21.15.99:0/3098064594 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3885058662 0 0) 0x7f1ad4071080 con 0x7f1ad40c9af0 2023-10-08T00:11:45.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.136+0000 7f1ad8a13700 1 -- 172.21.15.99:0/3098064594 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad4068cf0 con 0x7f1ad41510f0 2023-10-08T00:11:45.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.136+0000 7f1ad8a13700 1 --2- 172.21.15.99:0/3098064594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ad41510f0 0x7f1ad406ecc0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0009e60 tx=0x7f1ad0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86c0c702999223fd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:45.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.136+0000 7f1acb7fe700 1 -- 172.21.15.99:0/3098064594 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad000e070 con 0x7f1ad41510f0 2023-10-08T00:11:45.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.136+0000 7f1acb7fe700 1 -- 172.21.15.99:0/3098064594 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ad000bc30 con 0x7f1ad41510f0 2023-10-08T00:11:45.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.136+0000 7f1acb7fe700 1 -- 172.21.15.99:0/3098064594 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad0012430 con 0x7f1ad41510f0 2023-10-08T00:11:45.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.137+0000 7f1adac77700 1 -- 172.21.15.99:0/3098064594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ad41510f0 msgr2=0x7f1ad406ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:45.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.137+0000 7f1adac77700 1 --2- 172.21.15.99:0/3098064594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ad41510f0 0x7f1ad406ecc0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0009e60 tx=0x7f1ad0009780 comp rx=0 tx=0).stop 2023-10-08T00:11:45.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.137+0000 7f1adac77700 1 -- 172.21.15.99:0/3098064594 shutdown_connections 2023-10-08T00:11:45.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.137+0000 7f1adac77700 1 --2- 172.21.15.99:0/3098064594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ad41510f0 0x7f1ad406ecc0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:45.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.137+0000 7f1adac77700 1 --2- 172.21.15.99:0/3098064594 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ad4070700 0x7f1ad4070b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:45.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.137+0000 7f1adac77700 1 -- 172.21.15.99:0/3098064594 >> 172.21.15.99:0/3098064594 conn(0x7f1ad40ca4f0 msgr2=0x7f1ad40ca8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:45.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.137+0000 7f1adac77700 1 -- 172.21.15.99:0/3098064594 shutdown_connections 2023-10-08T00:11:45.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.137+0000 7f1adac77700 1 -- 172.21.15.99:0/3098064594 wait complete. 2023-10-08T00:11:45.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.138+0000 7f1adac77700 1 Processor -- start 2023-10-08T00:11:45.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.138+0000 7f1adac77700 1 -- start start 2023-10-08T00:11:45.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.139+0000 7f1adac77700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ad4070700 0x7f1ad415ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:45.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.139+0000 7f1adac77700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ad40c9af0 0x7f1ad40b17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:45.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.139+0000 7f1adac77700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ad41510f0 0x7f1ad40b2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:45.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.139+0000 7f1adac77700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad415da40 con 0x7f1ad4070700 2023-10-08T00:11:45.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.139+0000 7f1adac77700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad40b0b40 con 0x7f1ad41510f0 2023-10-08T00:11:45.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.139+0000 7f1adac77700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad415ce30 con 0x7f1ad40c9af0 2023-10-08T00:11:45.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.139+0000 7f1ad8a13700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ad4070700 0x7f1ad415ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:45.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.139+0000 7f1ad8a13700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ad4070700 0x7f1ad415ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50564/0 (socket says 172.21.15.99:50564) 2023-10-08T00:11:45.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.139+0000 7f1ad8a13700 1 -- 172.21.15.99:0/3143011633 learned_addr learned my addr 172.21.15.99:0/3143011633 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:45.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.139+0000 7f1acbfff700 1 --2- 172.21.15.99:0/3143011633 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ad40c9af0 0x7f1ad40b17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:45.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.139+0000 7f1ad9214700 1 --2- 172.21.15.99:0/3143011633 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ad41510f0 0x7f1ad40b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:45.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.139+0000 7f1ad8a13700 1 -- 172.21.15.99:0/3143011633 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ad40c9af0 msgr2=0x7f1ad40b17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:45.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.139+0000 7f1ad8a13700 1 --2- 172.21.15.99:0/3143011633 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ad40c9af0 0x7f1ad40b17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:45.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.139+0000 7f1ad8a13700 1 -- 172.21.15.99:0/3143011633 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ad41510f0 msgr2=0x7f1ad40b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:45.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.139+0000 7f1ad8a13700 1 --2- 172.21.15.99:0/3143011633 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ad41510f0 0x7f1ad40b2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:45.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.139+0000 7f1ad8a13700 1 -- 172.21.15.99:0/3143011633 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad0009a20 con 0x7f1ad4070700 2023-10-08T00:11:45.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.139+0000 7f1ad9214700 1 --2- 172.21.15.99:0/3143011633 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ad41510f0 0x7f1ad40b2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:11:45.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.139+0000 7f1ad8a13700 1 --2- 172.21.15.99:0/3143011633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ad4070700 0x7f1ad415ded0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1ad000b6c0 tx=0x7f1ad0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:45.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.139+0000 7f1ac8ff9700 1 -- 172.21.15.99:0/3143011633 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad0012e40 con 0x7f1ad4070700 2023-10-08T00:11:45.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.139+0000 7f1ac8ff9700 1 -- 172.21.15.99:0/3143011633 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ad0027470 con 0x7f1ad4070700 2023-10-08T00:11:45.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.139+0000 7f1ac8ff9700 1 -- 172.21.15.99:0/3143011633 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad0027830 con 0x7f1ad4070700 2023-10-08T00:11:45.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.139+0000 7f1adac77700 1 -- 172.21.15.99:0/3143011633 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ad415d110 con 0x7f1ad4070700 2023-10-08T00:11:45.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.139+0000 7f1adac77700 1 -- 172.21.15.99:0/3143011633 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ad415d6d0 con 0x7f1ad4070700 2023-10-08T00:11:45.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.140+0000 7f1adac77700 1 -- 172.21.15.99:0/3143011633 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ad415dd20 con 0x7f1ad4070700 2023-10-08T00:11:45.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.141+0000 7f1ac8ff9700 1 -- 172.21.15.99:0/3143011633 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1ad0024070 con 0x7f1ad4070700 2023-10-08T00:11:45.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.141+0000 7f1ac8ff9700 1 --2- 172.21.15.99:0/3143011633 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1ac0066f30 0x7f1ac00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:45.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.141+0000 7f1ac8ff9700 1 -- 172.21.15.99:0/3143011633 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1ad0087060 con 0x7f1ad4070700 2023-10-08T00:11:45.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.141+0000 7f1acbfff700 1 --2- 172.21.15.99:0/3143011633 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1ac0066f30 0x7f1ac00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:45.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.142+0000 7f1acbfff700 1 --2- 172.21.15.99:0/3143011633 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1ac0066f30 0x7f1ac00693e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1ad415a2b0 tx=0x7f1ac4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:45.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.145+0000 7f1ac8ff9700 1 -- 172.21.15.99:0/3143011633 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1ad00542d0 con 0x7f1ad4070700 2023-10-08T00:11:45.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.327+0000 7f1adac77700 1 -- 172.21.15.99:0/3143011633 --> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1ad415db80 con 0x7f1ac0066f30 2023-10-08T00:11:45.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.337+0000 7f1ac8ff9700 1 -- 172.21.15.99:0/3143011633 <== mgr.4099 v2:172.21.15.191:6832/85985 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+272712 (secure 0 0 0) 0x7f1ac4001cc0 con 0x7f1ac0066f30 2023-10-08T00:11:45.341 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:11:45.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.340+0000 7f1adac77700 1 -- 172.21.15.99:0/3143011633 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1ac0066f30 msgr2=0x7f1ac00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:45.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.340+0000 7f1adac77700 1 --2- 172.21.15.99:0/3143011633 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1ac0066f30 0x7f1ac00693e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1ad415a2b0 tx=0x7f1ac4009040 comp rx=0 tx=0).stop 2023-10-08T00:11:45.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.340+0000 7f1adac77700 1 -- 172.21.15.99:0/3143011633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ad4070700 msgr2=0x7f1ad415ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:45.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.340+0000 7f1adac77700 1 --2- 172.21.15.99:0/3143011633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ad4070700 0x7f1ad415ded0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1ad000b6c0 tx=0x7f1ad0019000 comp rx=0 tx=0).stop 2023-10-08T00:11:45.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.340+0000 7f1adac77700 1 -- 172.21.15.99:0/3143011633 shutdown_connections 2023-10-08T00:11:45.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.340+0000 7f1adac77700 1 --2- 172.21.15.99:0/3143011633 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1ac0066f30 0x7f1ac00693e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:45.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.340+0000 7f1adac77700 1 --2- 172.21.15.99:0/3143011633 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ad41510f0 0x7f1ad40b2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:45.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.340+0000 7f1adac77700 1 --2- 172.21.15.99:0/3143011633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ad4070700 0x7f1ad415ded0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:45.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.340+0000 7f1adac77700 1 --2- 172.21.15.99:0/3143011633 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ad40c9af0 0x7f1ad40b17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:45.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.340+0000 7f1adac77700 1 -- 172.21.15.99:0/3143011633 >> 172.21.15.99:0/3143011633 conn(0x7f1ad40ca4f0 msgr2=0x7f1ad40ccff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:45.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.340+0000 7f1adac77700 1 -- 172.21.15.99:0/3143011633 shutdown_connections 2023-10-08T00:11:45.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.340+0000 7f1adac77700 1 -- 172.21.15.99:0/3143011633 wait complete. 2023-10-08T00:11:45.344 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2023-10-08T00:11:45.353 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":52,"stamp":"2023-10-08T00:11:44.035908+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":114,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2188,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560311,"statfs":{"total":1600000000,"available":1597758732,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.746534"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.069656+0000","last_change":"2023-10-08T00:11:41.069601+0000","last_active":"2023-10-08T00:11:41.069656+0000","last_peered":"2023-10-08T00:11:41.069656+0000","last_clean":"2023-10-08T00:11:41.069656+0000","last_became_active":"2023-10-08T00:11:41.069388+0000","last_became_peered":"2023-10-08T00:11:41.069388+0000","last_unstale":"2023-10-08T00:11:41.069656+0000","last_undegraded":"2023-10-08T00:11:41.069656+0000","last_fullsized":"2023-10-08T00:11:41.069656+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:16:52.644456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.068405+0000","last_change":"2023-10-08T00:11:41.068179+0000","last_active":"2023-10-08T00:11:41.068405+0000","last_peered":"2023-10-08T00:11:41.068405+0000","last_clean":"2023-10-08T00:11:41.068405+0000","last_became_active":"2023-10-08T00:11:41.067614+0000","last_became_peered":"2023-10-08T00:11:41.067614+0000","last_unstale":"2023-10-08T00:11:41.068405+0000","last_undegraded":"2023-10-08T00:11:41.068405+0000","last_fullsized":"2023-10-08T00:11:41.068405+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:07:25.684529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.068085+0000","last_change":"2023-10-08T00:11:41.067952+0000","last_active":"2023-10-08T00:11:41.068085+0000","last_peered":"2023-10-08T00:11:41.068085+0000","last_clean":"2023-10-08T00:11:41.068085+0000","last_became_active":"2023-10-08T00:11:41.067727+0000","last_became_peered":"2023-10-08T00:11:41.067727+0000","last_unstale":"2023-10-08T00:11:41.068085+0000","last_undegraded":"2023-10-08T00:11:41.068085+0000","last_fullsized":"2023-10-08T00:11:41.068085+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:09:24.354082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.071107+0000","last_change":"2023-10-08T00:11:41.070804+0000","last_active":"2023-10-08T00:11:41.071107+0000","last_peered":"2023-10-08T00:11:41.071107+0000","last_clean":"2023-10-08T00:11:41.071107+0000","last_became_active":"2023-10-08T00:11:41.069994+0000","last_became_peered":"2023-10-08T00:11:41.069994+0000","last_unstale":"2023-10-08T00:11:41.071107+0000","last_undegraded":"2023-10-08T00:11:41.071107+0000","last_fullsized":"2023-10-08T00:11:41.071107+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:54:41.831036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.070097+0000","last_change":"2023-10-08T00:11:41.070030+0000","last_active":"2023-10-08T00:11:41.070097+0000","last_peered":"2023-10-08T00:11:41.070097+0000","last_clean":"2023-10-08T00:11:41.070097+0000","last_became_active":"2023-10-08T00:11:41.069799+0000","last_became_peered":"2023-10-08T00:11:41.069799+0000","last_unstale":"2023-10-08T00:11:41.070097+0000","last_undegraded":"2023-10-08T00:11:41.070097+0000","last_fullsized":"2023-10-08T00:11:41.070097+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:16:52.644456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.062305+0000","last_change":"2023-10-08T00:11:41.062237+0000","last_active":"2023-10-08T00:11:41.062305+0000","last_peered":"2023-10-08T00:11:41.062305+0000","last_clean":"2023-10-08T00:11:41.062305+0000","last_became_active":"2023-10-08T00:11:41.061947+0000","last_became_peered":"2023-10-08T00:11:41.061947+0000","last_unstale":"2023-10-08T00:11:41.062305+0000","last_undegraded":"2023-10-08T00:11:41.062305+0000","last_fullsized":"2023-10-08T00:11:41.062305+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:41:41.926109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.067283+0000","last_change":"2023-10-08T00:11:41.065080+0000","last_active":"2023-10-08T00:11:41.067283+0000","last_peered":"2023-10-08T00:11:41.067283+0000","last_clean":"2023-10-08T00:11:41.067283+0000","last_became_active":"2023-10-08T00:11:41.064579+0000","last_became_peered":"2023-10-08T00:11:41.064579+0000","last_unstale":"2023-10-08T00:11:41.067283+0000","last_undegraded":"2023-10-08T00:11:41.067283+0000","last_fullsized":"2023-10-08T00:11:41.067283+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:53:16.932728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.072042+0000","last_change":"2023-10-08T00:11:41.071983+0000","last_active":"2023-10-08T00:11:41.072042+0000","last_peered":"2023-10-08T00:11:41.072042+0000","last_clean":"2023-10-08T00:11:41.072042+0000","last_became_active":"2023-10-08T00:11:41.071608+0000","last_became_peered":"2023-10-08T00:11:41.071608+0000","last_unstale":"2023-10-08T00:11:41.072042+0000","last_undegraded":"2023-10-08T00:11:41.072042+0000","last_fullsized":"2023-10-08T00:11:41.072042+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:12:29.493504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.070961+0000","last_change":"2023-10-08T00:11:41.070911+0000","last_active":"2023-10-08T00:11:41.070961+0000","last_peered":"2023-10-08T00:11:41.070961+0000","last_clean":"2023-10-08T00:11:41.070961+0000","last_became_active":"2023-10-08T00:11:41.070696+0000","last_became_peered":"2023-10-08T00:11:41.070696+0000","last_unstale":"2023-10-08T00:11:41.070961+0000","last_undegraded":"2023-10-08T00:11:41.070961+0000","last_fullsized":"2023-10-08T00:11:41.070961+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:13:40.235186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.065444+0000","last_change":"2023-10-08T00:11:41.065297+0000","last_active":"2023-10-08T00:11:41.065444+0000","last_peered":"2023-10-08T00:11:41.065444+0000","last_clean":"2023-10-08T00:11:41.065444+0000","last_became_active":"2023-10-08T00:11:41.065062+0000","last_became_peered":"2023-10-08T00:11:41.065062+0000","last_unstale":"2023-10-08T00:11:41.065444+0000","last_undegraded":"2023-10-08T00:11:41.065444+0000","last_fullsized":"2023-10-08T00:11:41.065444+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:02:14.124345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.067108+0000","last_change":"2023-10-08T00:11:41.065279+0000","last_active":"2023-10-08T00:11:41.067108+0000","last_peered":"2023-10-08T00:11:41.067108+0000","last_clean":"2023-10-08T00:11:41.067108+0000","last_became_active":"2023-10-08T00:11:41.065066+0000","last_became_peered":"2023-10-08T00:11:41.065066+0000","last_unstale":"2023-10-08T00:11:41.067108+0000","last_undegraded":"2023-10-08T00:11:41.067108+0000","last_fullsized":"2023-10-08T00:11:41.067108+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:02:35.995739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.066233+0000","last_change":"2023-10-08T00:11:41.066145+0000","last_active":"2023-10-08T00:11:41.066233+0000","last_peered":"2023-10-08T00:11:41.066233+0000","last_clean":"2023-10-08T00:11:41.066233+0000","last_became_active":"2023-10-08T00:11:41.065766+0000","last_became_peered":"2023-10-08T00:11:41.065766+0000","last_unstale":"2023-10-08T00:11:41.066233+0000","last_undegraded":"2023-10-08T00:11:41.066233+0000","last_fullsized":"2023-10-08T00:11:41.066233+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:41:41.926109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.074433+0000","last_change":"2023-10-08T00:11:41.074355+0000","last_active":"2023-10-08T00:11:41.074433+0000","last_peered":"2023-10-08T00:11:41.074433+0000","last_clean":"2023-10-08T00:11:41.074433+0000","last_became_active":"2023-10-08T00:11:41.072388+0000","last_became_peered":"2023-10-08T00:11:41.072388+0000","last_unstale":"2023-10-08T00:11:41.074433+0000","last_undegraded":"2023-10-08T00:11:41.074433+0000","last_fullsized":"2023-10-08T00:11:41.074433+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:04:17.729964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.068488+0000","last_change":"2023-10-08T00:11:41.067908+0000","last_active":"2023-10-08T00:11:41.068488+0000","last_peered":"2023-10-08T00:11:41.068488+0000","last_clean":"2023-10-08T00:11:41.068488+0000","last_became_active":"2023-10-08T00:11:41.067520+0000","last_became_peered":"2023-10-08T00:11:41.067520+0000","last_unstale":"2023-10-08T00:11:41.068488+0000","last_undegraded":"2023-10-08T00:11:41.068488+0000","last_fullsized":"2023-10-08T00:11:41.068488+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:12:29.493504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.069923+0000","last_change":"2023-10-08T00:11:41.069811+0000","last_active":"2023-10-08T00:11:41.069923+0000","last_peered":"2023-10-08T00:11:41.069923+0000","last_clean":"2023-10-08T00:11:41.069923+0000","last_became_active":"2023-10-08T00:11:41.069334+0000","last_became_peered":"2023-10-08T00:11:41.069334+0000","last_unstale":"2023-10-08T00:11:41.069923+0000","last_undegraded":"2023-10-08T00:11:41.069923+0000","last_fullsized":"2023-10-08T00:11:41.069923+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:53:16.932728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.069265+0000","last_change":"2023-10-08T00:11:41.069072+0000","last_active":"2023-10-08T00:11:41.069265+0000","last_peered":"2023-10-08T00:11:41.069265+0000","last_clean":"2023-10-08T00:11:41.069265+0000","last_became_active":"2023-10-08T00:11:41.068814+0000","last_became_peered":"2023-10-08T00:11:41.068814+0000","last_unstale":"2023-10-08T00:11:41.069265+0000","last_undegraded":"2023-10-08T00:11:41.069265+0000","last_fullsized":"2023-10-08T00:11:41.069265+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:16:52.644456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.068930+0000","last_change":"2023-10-08T00:11:41.068867+0000","last_active":"2023-10-08T00:11:41.068930+0000","last_peered":"2023-10-08T00:11:41.068930+0000","last_clean":"2023-10-08T00:11:41.068930+0000","last_became_active":"2023-10-08T00:11:41.068030+0000","last_became_peered":"2023-10-08T00:11:41.068030+0000","last_unstale":"2023-10-08T00:11:41.068930+0000","last_undegraded":"2023-10-08T00:11:41.068930+0000","last_fullsized":"2023-10-08T00:11:41.068930+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:47:29.945735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.071539+0000","last_change":"2023-10-08T00:11:41.071391+0000","last_active":"2023-10-08T00:11:41.071539+0000","last_peered":"2023-10-08T00:11:41.071539+0000","last_clean":"2023-10-08T00:11:41.071539+0000","last_became_active":"2023-10-08T00:11:41.070979+0000","last_became_peered":"2023-10-08T00:11:41.070979+0000","last_unstale":"2023-10-08T00:11:41.071539+0000","last_undegraded":"2023-10-08T00:11:41.071539+0000","last_fullsized":"2023-10-08T00:11:41.071539+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:13:40.235186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.067542+0000","last_change":"2023-10-08T00:11:41.066614+0000","last_active":"2023-10-08T00:11:41.067542+0000","last_peered":"2023-10-08T00:11:41.067542+0000","last_clean":"2023-10-08T00:11:41.067542+0000","last_became_active":"2023-10-08T00:11:41.066339+0000","last_became_peered":"2023-10-08T00:11:41.066339+0000","last_unstale":"2023-10-08T00:11:41.067542+0000","last_undegraded":"2023-10-08T00:11:41.067542+0000","last_fullsized":"2023-10-08T00:11:41.067542+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:21:58.588524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.073159+0000","last_change":"2023-10-08T00:11:41.073093+0000","last_active":"2023-10-08T00:11:41.073159+0000","last_peered":"2023-10-08T00:11:41.073159+0000","last_clean":"2023-10-08T00:11:41.073159+0000","last_became_active":"2023-10-08T00:11:41.072306+0000","last_became_peered":"2023-10-08T00:11:41.072306+0000","last_unstale":"2023-10-08T00:11:41.073159+0000","last_undegraded":"2023-10-08T00:11:41.073159+0000","last_fullsized":"2023-10-08T00:11:41.073159+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:03:28.086788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.068677+0000","last_change":"2023-10-08T00:11:41.068583+0000","last_active":"2023-10-08T00:11:41.068677+0000","last_peered":"2023-10-08T00:11:41.068677+0000","last_clean":"2023-10-08T00:11:41.068677+0000","last_became_active":"2023-10-08T00:11:41.067298+0000","last_became_peered":"2023-10-08T00:11:41.067298+0000","last_unstale":"2023-10-08T00:11:41.068677+0000","last_undegraded":"2023-10-08T00:11:41.068677+0000","last_fullsized":"2023-10-08T00:11:41.068677+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:19:12.100172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.067304+0000","last_change":"2023-10-08T00:11:41.065215+0000","last_active":"2023-10-08T00:11:41.067304+0000","last_peered":"2023-10-08T00:11:41.067304+0000","last_clean":"2023-10-08T00:11:41.067304+0000","last_became_active":"2023-10-08T00:11:41.064860+0000","last_became_peered":"2023-10-08T00:11:41.064860+0000","last_unstale":"2023-10-08T00:11:41.067304+0000","last_undegraded":"2023-10-08T00:11:41.067304+0000","last_fullsized":"2023-10-08T00:11:41.067304+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:54:41.831036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.074595+0000","last_change":"2023-10-08T00:11:41.074524+0000","last_active":"2023-10-08T00:11:41.074595+0000","last_peered":"2023-10-08T00:11:41.074595+0000","last_clean":"2023-10-08T00:11:41.074595+0000","last_became_active":"2023-10-08T00:11:41.073858+0000","last_became_peered":"2023-10-08T00:11:41.073858+0000","last_unstale":"2023-10-08T00:11:41.074595+0000","last_undegraded":"2023-10-08T00:11:41.074595+0000","last_fullsized":"2023-10-08T00:11:41.074595+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:02:14.124345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.070387+0000","last_change":"2023-10-08T00:11:41.070325+0000","last_active":"2023-10-08T00:11:41.070387+0000","last_peered":"2023-10-08T00:11:41.070387+0000","last_clean":"2023-10-08T00:11:41.070387+0000","last_became_active":"2023-10-08T00:11:41.069924+0000","last_became_peered":"2023-10-08T00:11:41.069924+0000","last_unstale":"2023-10-08T00:11:41.070387+0000","last_undegraded":"2023-10-08T00:11:41.070387+0000","last_fullsized":"2023-10-08T00:11:41.070387+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:21:58.487927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.071685+0000","last_change":"2023-10-08T00:11:41.071473+0000","last_active":"2023-10-08T00:11:41.071685+0000","last_peered":"2023-10-08T00:11:41.071685+0000","last_clean":"2023-10-08T00:11:41.071685+0000","last_became_active":"2023-10-08T00:11:41.069876+0000","last_became_peered":"2023-10-08T00:11:41.069876+0000","last_unstale":"2023-10-08T00:11:41.071685+0000","last_undegraded":"2023-10-08T00:11:41.071685+0000","last_fullsized":"2023-10-08T00:11:41.071685+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:57:19.584921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.070270+0000","last_change":"2023-10-08T00:11:41.069110+0000","last_active":"2023-10-08T00:11:41.070270+0000","last_peered":"2023-10-08T00:11:41.070270+0000","last_clean":"2023-10-08T00:11:41.070270+0000","last_became_active":"2023-10-08T00:11:41.068098+0000","last_became_peered":"2023-10-08T00:11:41.068098+0000","last_unstale":"2023-10-08T00:11:41.070270+0000","last_undegraded":"2023-10-08T00:11:41.070270+0000","last_fullsized":"2023-10-08T00:11:41.070270+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:04:17.729964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.069387+0000","last_change":"2023-10-08T00:11:41.069322+0000","last_active":"2023-10-08T00:11:41.069387+0000","last_peered":"2023-10-08T00:11:41.069387+0000","last_clean":"2023-10-08T00:11:41.069387+0000","last_became_active":"2023-10-08T00:11:41.069092+0000","last_became_peered":"2023-10-08T00:11:41.069092+0000","last_unstale":"2023-10-08T00:11:41.069387+0000","last_undegraded":"2023-10-08T00:11:41.069387+0000","last_fullsized":"2023-10-08T00:11:41.069387+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:56:57.764426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.070642+0000","last_change":"2023-10-08T00:11:41.070585+0000","last_active":"2023-10-08T00:11:41.070642+0000","last_peered":"2023-10-08T00:11:41.070642+0000","last_clean":"2023-10-08T00:11:41.070642+0000","last_became_active":"2023-10-08T00:11:41.070379+0000","last_became_peered":"2023-10-08T00:11:41.070379+0000","last_unstale":"2023-10-08T00:11:41.070642+0000","last_undegraded":"2023-10-08T00:11:41.070642+0000","last_fullsized":"2023-10-08T00:11:41.070642+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:49:25.596070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.068760+0000","last_change":"2023-10-08T00:11:41.068702+0000","last_active":"2023-10-08T00:11:41.068760+0000","last_peered":"2023-10-08T00:11:41.068760+0000","last_clean":"2023-10-08T00:11:41.068760+0000","last_became_active":"2023-10-08T00:11:41.068455+0000","last_became_peered":"2023-10-08T00:11:41.068455+0000","last_unstale":"2023-10-08T00:11:41.068760+0000","last_undegraded":"2023-10-08T00:11:41.068760+0000","last_fullsized":"2023-10-08T00:11:41.068760+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:56:57.764426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.069907+0000","last_change":"2023-10-08T00:11:41.069678+0000","last_active":"2023-10-08T00:11:41.069907+0000","last_peered":"2023-10-08T00:11:41.069907+0000","last_clean":"2023-10-08T00:11:41.069907+0000","last_became_active":"2023-10-08T00:11:41.069048+0000","last_became_peered":"2023-10-08T00:11:41.069048+0000","last_unstale":"2023-10-08T00:11:41.069907+0000","last_undegraded":"2023-10-08T00:11:41.069907+0000","last_fullsized":"2023-10-08T00:11:41.069907+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:49:25.596070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.061458+0000","last_change":"2023-10-08T00:11:41.061399+0000","last_active":"2023-10-08T00:11:41.061458+0000","last_peered":"2023-10-08T00:11:41.061458+0000","last_clean":"2023-10-08T00:11:41.061458+0000","last_became_active":"2023-10-08T00:11:41.061202+0000","last_became_peered":"2023-10-08T00:11:41.061202+0000","last_unstale":"2023-10-08T00:11:41.061458+0000","last_undegraded":"2023-10-08T00:11:41.061458+0000","last_fullsized":"2023-10-08T00:11:41.061458+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:56:57.764426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.069662+0000","last_change":"2023-10-08T00:11:41.069189+0000","last_active":"2023-10-08T00:11:41.069662+0000","last_peered":"2023-10-08T00:11:41.069662+0000","last_clean":"2023-10-08T00:11:41.069662+0000","last_became_active":"2023-10-08T00:11:41.068628+0000","last_became_peered":"2023-10-08T00:11:41.068628+0000","last_unstale":"2023-10-08T00:11:41.069662+0000","last_undegraded":"2023-10-08T00:11:41.069662+0000","last_fullsized":"2023-10-08T00:11:41.069662+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:56:18.539290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.070155+0000","last_change":"2023-10-08T00:11:41.069413+0000","last_active":"2023-10-08T00:11:41.070155+0000","last_peered":"2023-10-08T00:11:41.070155+0000","last_clean":"2023-10-08T00:11:41.070155+0000","last_became_active":"2023-10-08T00:11:41.068721+0000","last_became_peered":"2023-10-08T00:11:41.068721+0000","last_unstale":"2023-10-08T00:11:41.070155+0000","last_undegraded":"2023-10-08T00:11:41.070155+0000","last_fullsized":"2023-10-08T00:11:41.070155+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:32:58.999045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"30'4","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.301875+0000","last_change":"2023-10-08T00:11:40.300462+0000","last_active":"2023-10-08T00:11:41.301875+0000","last_peered":"2023-10-08T00:11:41.301875+0000","last_clean":"2023-10-08T00:11:41.301875+0000","last_became_active":"2023-10-08T00:11:40.298899+0000","last_became_peered":"2023-10-08T00:11:40.298899+0000","last_unstale":"2023-10-08T00:11:41.301875+0000","last_undegraded":"2023-10-08T00:11:41.301875+0000","last_fullsized":"2023-10-08T00:11:41.301875+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:16:51.639625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.071926+0000","last_change":"2023-10-08T00:11:41.071838+0000","last_active":"2023-10-08T00:11:41.071926+0000","last_peered":"2023-10-08T00:11:41.071926+0000","last_clean":"2023-10-08T00:11:41.071926+0000","last_became_active":"2023-10-08T00:11:41.071274+0000","last_became_peered":"2023-10-08T00:11:41.071274+0000","last_unstale":"2023-10-08T00:11:41.071926+0000","last_undegraded":"2023-10-08T00:11:41.071926+0000","last_fullsized":"2023-10-08T00:11:41.071926+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:07:25.684529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"30'1","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.298504+0000","last_change":"2023-10-08T00:11:40.307137+0000","last_active":"2023-10-08T00:11:41.298504+0000","last_peered":"2023-10-08T00:11:41.298504+0000","last_clean":"2023-10-08T00:11:41.298504+0000","last_became_active":"2023-10-08T00:11:40.306100+0000","last_became_peered":"2023-10-08T00:11:40.306100+0000","last_unstale":"2023-10-08T00:11:41.298504+0000","last_undegraded":"2023-10-08T00:11:41.298504+0000","last_fullsized":"2023-10-08T00:11:41.298504+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:41:40.921278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.067343+0000","last_change":"2023-10-08T00:11:41.066055+0000","last_active":"2023-10-08T00:11:41.067343+0000","last_peered":"2023-10-08T00:11:41.067343+0000","last_clean":"2023-10-08T00:11:41.067343+0000","last_became_active":"2023-10-08T00:11:41.065834+0000","last_became_peered":"2023-10-08T00:11:41.065834+0000","last_unstale":"2023-10-08T00:11:41.067343+0000","last_undegraded":"2023-10-08T00:11:41.067343+0000","last_fullsized":"2023-10-08T00:11:41.067343+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:51:15.073889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"30'2","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.298538+0000","last_change":"2023-10-08T00:11:40.299193+0000","last_active":"2023-10-08T00:11:41.298538+0000","last_peered":"2023-10-08T00:11:41.298538+0000","last_clean":"2023-10-08T00:11:41.298538+0000","last_became_active":"2023-10-08T00:11:40.298932+0000","last_became_peered":"2023-10-08T00:11:40.298932+0000","last_unstale":"2023-10-08T00:11:41.298538+0000","last_undegraded":"2023-10-08T00:11:41.298538+0000","last_fullsized":"2023-10-08T00:11:41.298538+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:12:47.778998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.071844+0000","last_change":"2023-10-08T00:11:41.071781+0000","last_active":"2023-10-08T00:11:41.071844+0000","last_peered":"2023-10-08T00:11:41.071844+0000","last_clean":"2023-10-08T00:11:41.071844+0000","last_became_active":"2023-10-08T00:11:41.071234+0000","last_became_peered":"2023-10-08T00:11:41.071234+0000","last_unstale":"2023-10-08T00:11:41.071844+0000","last_undegraded":"2023-10-08T00:11:41.071844+0000","last_fullsized":"2023-10-08T00:11:41.071844+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:56:50.566568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"30'2","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.301986+0000","last_change":"2023-10-08T00:11:40.302878+0000","last_active":"2023-10-08T00:11:41.301986+0000","last_peered":"2023-10-08T00:11:41.301986+0000","last_clean":"2023-10-08T00:11:41.301986+0000","last_became_active":"2023-10-08T00:11:40.301405+0000","last_became_peered":"2023-10-08T00:11:40.301405+0000","last_unstale":"2023-10-08T00:11:41.301986+0000","last_undegraded":"2023-10-08T00:11:41.301986+0000","last_fullsized":"2023-10-08T00:11:41.301986+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:11:55.751058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.065414+0000","last_change":"2023-10-08T00:11:41.065340+0000","last_active":"2023-10-08T00:11:41.065414+0000","last_peered":"2023-10-08T00:11:41.065414+0000","last_clean":"2023-10-08T00:11:41.065414+0000","last_became_active":"2023-10-08T00:11:41.065113+0000","last_became_peered":"2023-10-08T00:11:41.065113+0000","last_unstale":"2023-10-08T00:11:41.065414+0000","last_undegraded":"2023-10-08T00:11:41.065414+0000","last_fullsized":"2023-10-08T00:11:41.065414+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:03:46.298642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"30'1","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.298630+0000","last_change":"2023-10-08T00:11:40.294854+0000","last_active":"2023-10-08T00:11:41.298630+0000","last_peered":"2023-10-08T00:11:41.298630+0000","last_clean":"2023-10-08T00:11:41.298630+0000","last_became_active":"2023-10-08T00:11:40.294631+0000","last_became_peered":"2023-10-08T00:11:40.294631+0000","last_unstale":"2023-10-08T00:11:41.298630+0000","last_undegraded":"2023-10-08T00:11:41.298630+0000","last_fullsized":"2023-10-08T00:11:41.298630+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:20:01.582031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.067220+0000","last_change":"2023-10-08T00:11:41.062485+0000","last_active":"2023-10-08T00:11:41.067220+0000","last_peered":"2023-10-08T00:11:41.067220+0000","last_clean":"2023-10-08T00:11:41.067220+0000","last_became_active":"2023-10-08T00:11:41.062269+0000","last_became_peered":"2023-10-08T00:11:41.062269+0000","last_unstale":"2023-10-08T00:11:41.067220+0000","last_undegraded":"2023-10-08T00:11:41.067220+0000","last_fullsized":"2023-10-08T00:11:41.067220+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:57:19.584921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"30'1","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.299563+0000","last_change":"2023-10-08T00:11:40.302118+0000","last_active":"2023-10-08T00:11:41.299563+0000","last_peered":"2023-10-08T00:11:41.299563+0000","last_clean":"2023-10-08T00:11:41.299563+0000","last_became_active":"2023-10-08T00:11:40.300756+0000","last_became_peered":"2023-10-08T00:11:40.300756+0000","last_unstale":"2023-10-08T00:11:41.299563+0000","last_undegraded":"2023-10-08T00:11:41.299563+0000","last_fullsized":"2023-10-08T00:11:41.299563+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:13:39.230355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.071434+0000","last_change":"2023-10-08T00:11:41.071377+0000","last_active":"2023-10-08T00:11:41.071434+0000","last_peered":"2023-10-08T00:11:41.071434+0000","last_clean":"2023-10-08T00:11:41.071434+0000","last_became_active":"2023-10-08T00:11:41.071149+0000","last_became_peered":"2023-10-08T00:11:41.071149+0000","last_unstale":"2023-10-08T00:11:41.071434+0000","last_undegraded":"2023-10-08T00:11:41.071434+0000","last_fullsized":"2023-10-08T00:11:41.071434+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:20:02.586863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.060428+0000","last_change":"2023-10-08T00:11:40.302427+0000","last_active":"2023-10-08T00:11:41.060428+0000","last_peered":"2023-10-08T00:11:41.060428+0000","last_clean":"2023-10-08T00:11:41.060428+0000","last_became_active":"2023-10-08T00:11:40.302196+0000","last_became_peered":"2023-10-08T00:11:40.302196+0000","last_unstale":"2023-10-08T00:11:41.060428+0000","last_undegraded":"2023-10-08T00:11:41.060428+0000","last_fullsized":"2023-10-08T00:11:41.060428+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:49:46.767411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.073874+0000","last_change":"2023-10-08T00:11:41.073809+0000","last_active":"2023-10-08T00:11:41.073874+0000","last_peered":"2023-10-08T00:11:41.073874+0000","last_clean":"2023-10-08T00:11:41.073874+0000","last_became_active":"2023-10-08T00:11:41.073353+0000","last_became_peered":"2023-10-08T00:11:41.073353+0000","last_unstale":"2023-10-08T00:11:41.073874+0000","last_undegraded":"2023-10-08T00:11:41.073874+0000","last_fullsized":"2023-10-08T00:11:41.073874+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:49:25.596070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.058298+0000","last_change":"2023-10-08T00:11:40.299748+0000","last_active":"2023-10-08T00:11:41.058298+0000","last_peered":"2023-10-08T00:11:41.058298+0000","last_clean":"2023-10-08T00:11:41.058298+0000","last_became_active":"2023-10-08T00:11:40.298913+0000","last_became_peered":"2023-10-08T00:11:40.298913+0000","last_unstale":"2023-10-08T00:11:41.058298+0000","last_undegraded":"2023-10-08T00:11:41.058298+0000","last_fullsized":"2023-10-08T00:11:41.058298+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:23:12.401507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.067092+0000","last_change":"2023-10-08T00:11:41.065209+0000","last_active":"2023-10-08T00:11:41.067092+0000","last_peered":"2023-10-08T00:11:41.067092+0000","last_clean":"2023-10-08T00:11:41.067092+0000","last_became_active":"2023-10-08T00:11:41.065002+0000","last_became_peered":"2023-10-08T00:11:41.065002+0000","last_unstale":"2023-10-08T00:11:41.067092+0000","last_undegraded":"2023-10-08T00:11:41.067092+0000","last_fullsized":"2023-10-08T00:11:41.067092+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:14:53.717627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.062113+0000","last_change":"2023-10-08T00:11:40.299963+0000","last_active":"2023-10-08T00:11:41.062113+0000","last_peered":"2023-10-08T00:11:41.062113+0000","last_clean":"2023-10-08T00:11:41.062113+0000","last_became_active":"2023-10-08T00:11:40.299725+0000","last_became_peered":"2023-10-08T00:11:40.299725+0000","last_unstale":"2023-10-08T00:11:41.062113+0000","last_undegraded":"2023-10-08T00:11:41.062113+0000","last_fullsized":"2023-10-08T00:11:41.062113+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:32:57.994214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.071127+0000","last_change":"2023-10-08T00:11:41.071031+0000","last_active":"2023-10-08T00:11:41.071127+0000","last_peered":"2023-10-08T00:11:41.071127+0000","last_clean":"2023-10-08T00:11:41.071127+0000","last_became_active":"2023-10-08T00:11:41.070498+0000","last_became_peered":"2023-10-08T00:11:41.070498+0000","last_unstale":"2023-10-08T00:11:41.071127+0000","last_undegraded":"2023-10-08T00:11:41.071127+0000","last_fullsized":"2023-10-08T00:11:41.071127+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:52:08.436392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.063154+0000","last_change":"2023-10-08T00:11:40.299048+0000","last_active":"2023-10-08T00:11:41.063154+0000","last_peered":"2023-10-08T00:11:41.063154+0000","last_clean":"2023-10-08T00:11:41.063154+0000","last_became_active":"2023-10-08T00:11:40.298886+0000","last_became_peered":"2023-10-08T00:11:40.298886+0000","last_unstale":"2023-10-08T00:11:41.063154+0000","last_undegraded":"2023-10-08T00:11:41.063154+0000","last_fullsized":"2023-10-08T00:11:41.063154+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:09:12.065285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.074846+0000","last_change":"2023-10-08T00:11:41.074787+0000","last_active":"2023-10-08T00:11:41.074846+0000","last_peered":"2023-10-08T00:11:41.074846+0000","last_clean":"2023-10-08T00:11:41.074846+0000","last_became_active":"2023-10-08T00:11:41.074028+0000","last_became_peered":"2023-10-08T00:11:41.074028+0000","last_unstale":"2023-10-08T00:11:41.074846+0000","last_undegraded":"2023-10-08T00:11:41.074846+0000","last_fullsized":"2023-10-08T00:11:41.074846+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:56:57.764426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"30'3","reported_seq":13,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.300937+0000","last_change":"2023-10-08T00:11:40.301508+0000","last_active":"2023-10-08T00:11:41.300937+0000","last_peered":"2023-10-08T00:11:41.300937+0000","last_clean":"2023-10-08T00:11:41.300937+0000","last_became_active":"2023-10-08T00:11:40.301290+0000","last_became_peered":"2023-10-08T00:11:40.301290+0000","last_unstale":"2023-10-08T00:11:41.300937+0000","last_undegraded":"2023-10-08T00:11:41.300937+0000","last_fullsized":"2023-10-08T00:11:41.300937+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:56:56.759595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.067123+0000","last_change":"2023-10-08T00:11:41.065594+0000","last_active":"2023-10-08T00:11:41.067123+0000","last_peered":"2023-10-08T00:11:41.067123+0000","last_clean":"2023-10-08T00:11:41.067123+0000","last_became_active":"2023-10-08T00:11:41.065385+0000","last_became_peered":"2023-10-08T00:11:41.065385+0000","last_unstale":"2023-10-08T00:11:41.067123+0000","last_undegraded":"2023-10-08T00:11:41.067123+0000","last_fullsized":"2023-10-08T00:11:41.067123+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:54:53.692310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"30'6","reported_seq":16,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.301998+0000","last_change":"2023-10-08T00:11:40.307488+0000","last_active":"2023-10-08T00:11:41.301998+0000","last_peered":"2023-10-08T00:11:41.301998+0000","last_clean":"2023-10-08T00:11:41.301998+0000","last_became_active":"2023-10-08T00:11:40.307258+0000","last_became_peered":"2023-10-08T00:11:40.307258+0000","last_unstale":"2023-10-08T00:11:41.301998+0000","last_undegraded":"2023-10-08T00:11:41.301998+0000","last_fullsized":"2023-10-08T00:11:41.301998+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:09:23.349251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.073179+0000","last_change":"2023-10-08T00:11:41.073139+0000","last_active":"2023-10-08T00:11:41.073179+0000","last_peered":"2023-10-08T00:11:41.073179+0000","last_clean":"2023-10-08T00:11:41.073179+0000","last_became_active":"2023-10-08T00:11:41.072647+0000","last_became_peered":"2023-10-08T00:11:41.072647+0000","last_unstale":"2023-10-08T00:11:41.073179+0000","last_undegraded":"2023-10-08T00:11:41.073179+0000","last_fullsized":"2023-10-08T00:11:41.073179+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:13:05.134473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"30'2","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.302322+0000","last_change":"2023-10-08T00:11:40.299984+0000","last_active":"2023-10-08T00:11:41.302322+0000","last_peered":"2023-10-08T00:11:41.302322+0000","last_clean":"2023-10-08T00:11:41.302322+0000","last_became_active":"2023-10-08T00:11:40.299532+0000","last_became_peered":"2023-10-08T00:11:40.299532+0000","last_unstale":"2023-10-08T00:11:41.302322+0000","last_undegraded":"2023-10-08T00:11:41.302322+0000","last_fullsized":"2023-10-08T00:11:41.302322+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:11:55.751058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.071707+0000","last_change":"2023-10-08T00:11:41.071633+0000","last_active":"2023-10-08T00:11:41.071707+0000","last_peered":"2023-10-08T00:11:41.071707+0000","last_clean":"2023-10-08T00:11:41.071707+0000","last_became_active":"2023-10-08T00:11:41.070388+0000","last_became_peered":"2023-10-08T00:11:41.070388+0000","last_unstale":"2023-10-08T00:11:41.071707+0000","last_undegraded":"2023-10-08T00:11:41.071707+0000","last_fullsized":"2023-10-08T00:11:41.071707+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:09:13.070116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.064146+0000","last_change":"2023-10-08T00:11:40.303633+0000","last_active":"2023-10-08T00:11:41.064146+0000","last_peered":"2023-10-08T00:11:41.064146+0000","last_clean":"2023-10-08T00:11:41.064146+0000","last_became_active":"2023-10-08T00:11:40.303137+0000","last_became_peered":"2023-10-08T00:11:40.303137+0000","last_unstale":"2023-10-08T00:11:41.064146+0000","last_undegraded":"2023-10-08T00:11:41.064146+0000","last_fullsized":"2023-10-08T00:11:41.064146+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:21:08.277105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.067321+0000","last_change":"2023-10-08T00:11:41.065872+0000","last_active":"2023-10-08T00:11:41.067321+0000","last_peered":"2023-10-08T00:11:41.067321+0000","last_clean":"2023-10-08T00:11:41.067321+0000","last_became_active":"2023-10-08T00:11:41.065573+0000","last_became_peered":"2023-10-08T00:11:41.065573+0000","last_unstale":"2023-10-08T00:11:41.067321+0000","last_undegraded":"2023-10-08T00:11:41.067321+0000","last_fullsized":"2023-10-08T00:11:41.067321+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:02:14.124345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.062601+0000","last_change":"2023-10-08T00:11:40.301793+0000","last_active":"2023-10-08T00:11:41.062601+0000","last_peered":"2023-10-08T00:11:41.062601+0000","last_clean":"2023-10-08T00:11:41.062601+0000","last_became_active":"2023-10-08T00:11:40.301229+0000","last_became_peered":"2023-10-08T00:11:40.301229+0000","last_unstale":"2023-10-08T00:11:41.062601+0000","last_undegraded":"2023-10-08T00:11:41.062601+0000","last_fullsized":"2023-10-08T00:11:41.062601+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:35:01.644676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.069197+0000","last_change":"2023-10-08T00:11:41.069063+0000","last_active":"2023-10-08T00:11:41.069197+0000","last_peered":"2023-10-08T00:11:41.069197+0000","last_clean":"2023-10-08T00:11:41.069197+0000","last_became_active":"2023-10-08T00:11:41.068310+0000","last_became_peered":"2023-10-08T00:11:41.068310+0000","last_unstale":"2023-10-08T00:11:41.069197+0000","last_undegraded":"2023-10-08T00:11:41.069197+0000","last_fullsized":"2023-10-08T00:11:41.069197+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:00:08.778342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"30'8","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.301851+0000","last_change":"2023-10-08T00:11:40.301528+0000","last_active":"2023-10-08T00:11:41.301851+0000","last_peered":"2023-10-08T00:11:41.301851+0000","last_clean":"2023-10-08T00:11:41.301851+0000","last_became_active":"2023-10-08T00:11:40.301268+0000","last_became_peered":"2023-10-08T00:11:40.301268+0000","last_unstale":"2023-10-08T00:11:41.301851+0000","last_undegraded":"2023-10-08T00:11:41.301851+0000","last_fullsized":"2023-10-08T00:11:41.301851+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:34:45.909544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.067138+0000","last_change":"2023-10-08T00:11:41.065696+0000","last_active":"2023-10-08T00:11:41.067138+0000","last_peered":"2023-10-08T00:11:41.067138+0000","last_clean":"2023-10-08T00:11:41.067138+0000","last_became_active":"2023-10-08T00:11:41.065493+0000","last_became_peered":"2023-10-08T00:11:41.065493+0000","last_unstale":"2023-10-08T00:11:41.067138+0000","last_undegraded":"2023-10-08T00:11:41.067138+0000","last_fullsized":"2023-10-08T00:11:41.067138+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:02:19.666640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"30'1","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.299124+0000","last_change":"2023-10-08T00:11:40.307608+0000","last_active":"2023-10-08T00:11:41.299124+0000","last_peered":"2023-10-08T00:11:41.299124+0000","last_clean":"2023-10-08T00:11:41.299124+0000","last_became_active":"2023-10-08T00:11:40.307366+0000","last_became_peered":"2023-10-08T00:11:40.307366+0000","last_unstale":"2023-10-08T00:11:41.299124+0000","last_undegraded":"2023-10-08T00:11:41.299124+0000","last_fullsized":"2023-10-08T00:11:41.299124+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:28:37.258489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.067908+0000","last_change":"2023-10-08T00:11:41.067849+0000","last_active":"2023-10-08T00:11:41.067908+0000","last_peered":"2023-10-08T00:11:41.067908+0000","last_clean":"2023-10-08T00:11:41.067908+0000","last_became_active":"2023-10-08T00:11:41.067612+0000","last_became_peered":"2023-10-08T00:11:41.067612+0000","last_unstale":"2023-10-08T00:11:41.067908+0000","last_undegraded":"2023-10-08T00:11:41.067908+0000","last_fullsized":"2023-10-08T00:11:41.067908+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:11:56.755889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"30'2","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.302108+0000","last_change":"2023-10-08T00:11:40.299858+0000","last_active":"2023-10-08T00:11:41.302108+0000","last_peered":"2023-10-08T00:11:41.302108+0000","last_clean":"2023-10-08T00:11:41.302108+0000","last_became_active":"2023-10-08T00:11:40.299250+0000","last_became_peered":"2023-10-08T00:11:40.299250+0000","last_unstale":"2023-10-08T00:11:41.302108+0000","last_undegraded":"2023-10-08T00:11:41.302108+0000","last_fullsized":"2023-10-08T00:11:41.302108+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:35:01.644676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.068434+0000","last_change":"2023-10-08T00:11:41.068324+0000","last_active":"2023-10-08T00:11:41.068434+0000","last_peered":"2023-10-08T00:11:41.068434+0000","last_clean":"2023-10-08T00:11:41.068434+0000","last_became_active":"2023-10-08T00:11:41.067954+0000","last_became_peered":"2023-10-08T00:11:41.067954+0000","last_unstale":"2023-10-08T00:11:41.068434+0000","last_undegraded":"2023-10-08T00:11:41.068434+0000","last_fullsized":"2023-10-08T00:11:41.068434+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:53:16.932728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"30'1","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.298766+0000","last_change":"2023-10-08T00:11:40.303265+0000","last_active":"2023-10-08T00:11:41.298766+0000","last_peered":"2023-10-08T00:11:41.298766+0000","last_clean":"2023-10-08T00:11:41.298766+0000","last_became_active":"2023-10-08T00:11:40.302748+0000","last_became_peered":"2023-10-08T00:11:40.302748+0000","last_unstale":"2023-10-08T00:11:41.298766+0000","last_undegraded":"2023-10-08T00:11:41.298766+0000","last_fullsized":"2023-10-08T00:11:41.298766+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:07:24.679697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.070186+0000","last_change":"2023-10-08T00:11:41.070079+0000","last_active":"2023-10-08T00:11:41.070186+0000","last_peered":"2023-10-08T00:11:41.070186+0000","last_clean":"2023-10-08T00:11:41.070186+0000","last_became_active":"2023-10-08T00:11:41.069697+0000","last_became_peered":"2023-10-08T00:11:41.069697+0000","last_unstale":"2023-10-08T00:11:41.070186+0000","last_undegraded":"2023-10-08T00:11:41.070186+0000","last_fullsized":"2023-10-08T00:11:41.070186+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:35:02.649508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.066906+0000","last_change":"2023-10-08T00:11:40.301506+0000","last_active":"2023-10-08T00:11:41.066906+0000","last_peered":"2023-10-08T00:11:41.066906+0000","last_clean":"2023-10-08T00:11:41.066906+0000","last_became_active":"2023-10-08T00:11:40.301264+0000","last_became_peered":"2023-10-08T00:11:40.301264+0000","last_unstale":"2023-10-08T00:11:41.066906+0000","last_undegraded":"2023-10-08T00:11:41.066906+0000","last_fullsized":"2023-10-08T00:11:41.066906+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:21:43.768378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.068864+0000","last_change":"2023-10-08T00:11:41.068481+0000","last_active":"2023-10-08T00:11:41.068864+0000","last_peered":"2023-10-08T00:11:41.068864+0000","last_clean":"2023-10-08T00:11:41.068864+0000","last_became_active":"2023-10-08T00:11:41.068243+0000","last_became_peered":"2023-10-08T00:11:41.068243+0000","last_unstale":"2023-10-08T00:11:41.068864+0000","last_undegraded":"2023-10-08T00:11:41.068864+0000","last_fullsized":"2023-10-08T00:11:41.068864+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:56:50.566568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"30'2","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.301583+0000","last_change":"2023-10-08T00:11:40.297307+0000","last_active":"2023-10-08T00:11:41.301583+0000","last_peered":"2023-10-08T00:11:41.301583+0000","last_clean":"2023-10-08T00:11:41.301583+0000","last_became_active":"2023-10-08T00:11:40.297047+0000","last_became_peered":"2023-10-08T00:11:40.297047+0000","last_unstale":"2023-10-08T00:11:41.301583+0000","last_undegraded":"2023-10-08T00:11:41.301583+0000","last_fullsized":"2023-10-08T00:11:41.301583+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:56:49.561736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.072594+0000","last_change":"2023-10-08T00:11:41.072520+0000","last_active":"2023-10-08T00:11:41.072594+0000","last_peered":"2023-10-08T00:11:41.072594+0000","last_clean":"2023-10-08T00:11:41.072594+0000","last_became_active":"2023-10-08T00:11:41.072257+0000","last_became_peered":"2023-10-08T00:11:41.072257+0000","last_unstale":"2023-10-08T00:11:41.072594+0000","last_undegraded":"2023-10-08T00:11:41.072594+0000","last_fullsized":"2023-10-08T00:11:41.072594+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:57:19.584921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.058539+0000","last_change":"2023-10-08T00:11:40.306139+0000","last_active":"2023-10-08T00:11:41.058539+0000","last_peered":"2023-10-08T00:11:41.058539+0000","last_clean":"2023-10-08T00:11:41.058539+0000","last_became_active":"2023-10-08T00:11:40.305874+0000","last_became_peered":"2023-10-08T00:11:40.305874+0000","last_unstale":"2023-10-08T00:11:41.058539+0000","last_undegraded":"2023-10-08T00:11:41.058539+0000","last_fullsized":"2023-10-08T00:11:41.058539+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:21:57.583692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.073315+0000","last_change":"2023-10-08T00:11:41.073262+0000","last_active":"2023-10-08T00:11:41.073315+0000","last_peered":"2023-10-08T00:11:41.073315+0000","last_clean":"2023-10-08T00:11:41.073315+0000","last_became_active":"2023-10-08T00:11:41.072951+0000","last_became_peered":"2023-10-08T00:11:41.072951+0000","last_unstale":"2023-10-08T00:11:41.073315+0000","last_undegraded":"2023-10-08T00:11:41.073315+0000","last_fullsized":"2023-10-08T00:11:41.073315+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:51:40.262177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.063297+0000","last_change":"2023-10-08T00:11:40.301402+0000","last_active":"2023-10-08T00:11:41.063297+0000","last_peered":"2023-10-08T00:11:41.063297+0000","last_clean":"2023-10-08T00:11:41.063297+0000","last_became_active":"2023-10-08T00:11:40.301183+0000","last_became_peered":"2023-10-08T00:11:40.301183+0000","last_unstale":"2023-10-08T00:11:41.063297+0000","last_undegraded":"2023-10-08T00:11:41.063297+0000","last_fullsized":"2023-10-08T00:11:41.063297+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:51:40.610344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.069558+0000","last_change":"2023-10-08T00:11:41.069312+0000","last_active":"2023-10-08T00:11:41.069558+0000","last_peered":"2023-10-08T00:11:41.069558+0000","last_clean":"2023-10-08T00:11:41.069558+0000","last_became_active":"2023-10-08T00:11:41.069078+0000","last_became_peered":"2023-10-08T00:11:41.069078+0000","last_unstale":"2023-10-08T00:11:41.069558+0000","last_undegraded":"2023-10-08T00:11:41.069558+0000","last_fullsized":"2023-10-08T00:11:41.069558+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:21:09.281936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.060804+0000","last_change":"2023-10-08T00:11:40.303399+0000","last_active":"2023-10-08T00:11:41.060804+0000","last_peered":"2023-10-08T00:11:41.060804+0000","last_clean":"2023-10-08T00:11:41.060804+0000","last_became_active":"2023-10-08T00:11:40.303051+0000","last_became_peered":"2023-10-08T00:11:40.303051+0000","last_unstale":"2023-10-08T00:11:41.060804+0000","last_undegraded":"2023-10-08T00:11:41.060804+0000","last_fullsized":"2023-10-08T00:11:41.060804+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:53:15.927897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.071521+0000","last_change":"2023-10-08T00:11:41.071206+0000","last_active":"2023-10-08T00:11:41.071521+0000","last_peered":"2023-10-08T00:11:41.071521+0000","last_clean":"2023-10-08T00:11:41.071521+0000","last_became_active":"2023-10-08T00:11:41.070680+0000","last_became_peered":"2023-10-08T00:11:41.070680+0000","last_unstale":"2023-10-08T00:11:41.071521+0000","last_undegraded":"2023-10-08T00:11:41.071521+0000","last_fullsized":"2023-10-08T00:11:41.071521+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:11:56.755889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"30'2","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.301386+0000","last_change":"2023-10-08T00:11:40.305058+0000","last_active":"2023-10-08T00:11:41.301386+0000","last_peered":"2023-10-08T00:11:41.301386+0000","last_clean":"2023-10-08T00:11:41.301386+0000","last_became_active":"2023-10-08T00:11:40.304528+0000","last_became_peered":"2023-10-08T00:11:40.304528+0000","last_unstale":"2023-10-08T00:11:41.301386+0000","last_undegraded":"2023-10-08T00:11:41.301386+0000","last_fullsized":"2023-10-08T00:11:41.301386+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:35:01.644676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.067156+0000","last_change":"2023-10-08T00:11:41.066345+0000","last_active":"2023-10-08T00:11:41.067156+0000","last_peered":"2023-10-08T00:11:41.067156+0000","last_clean":"2023-10-08T00:11:41.067156+0000","last_became_active":"2023-10-08T00:11:41.066159+0000","last_became_peered":"2023-10-08T00:11:41.066159+0000","last_unstale":"2023-10-08T00:11:41.067156+0000","last_undegraded":"2023-10-08T00:11:41.067156+0000","last_fullsized":"2023-10-08T00:11:41.067156+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:16:37.383597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.070040+0000","last_change":"2023-10-08T00:11:40.299694+0000","last_active":"2023-10-08T00:11:41.070040+0000","last_peered":"2023-10-08T00:11:41.070040+0000","last_clean":"2023-10-08T00:11:41.070040+0000","last_became_active":"2023-10-08T00:11:40.299462+0000","last_became_peered":"2023-10-08T00:11:40.299462+0000","last_unstale":"2023-10-08T00:11:41.070040+0000","last_undegraded":"2023-10-08T00:11:41.070040+0000","last_fullsized":"2023-10-08T00:11:41.070040+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:13:04.129642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.069801+0000","last_change":"2023-10-08T00:11:41.069734+0000","last_active":"2023-10-08T00:11:41.069801+0000","last_peered":"2023-10-08T00:11:41.069801+0000","last_clean":"2023-10-08T00:11:41.069801+0000","last_became_active":"2023-10-08T00:11:41.069509+0000","last_became_peered":"2023-10-08T00:11:41.069509+0000","last_unstale":"2023-10-08T00:11:41.069801+0000","last_undegraded":"2023-10-08T00:11:41.069801+0000","last_fullsized":"2023-10-08T00:11:41.069801+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:23:13.406338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.065608+0000","last_change":"2023-10-08T00:11:40.302152+0000","last_active":"2023-10-08T00:11:41.065608+0000","last_peered":"2023-10-08T00:11:41.065608+0000","last_clean":"2023-10-08T00:11:41.065608+0000","last_became_active":"2023-10-08T00:11:40.301896+0000","last_became_peered":"2023-10-08T00:11:40.301896+0000","last_unstale":"2023-10-08T00:11:41.065608+0000","last_undegraded":"2023-10-08T00:11:41.065608+0000","last_fullsized":"2023-10-08T00:11:41.065608+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:13:28.760481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.068824+0000","last_change":"2023-10-08T00:11:41.068732+0000","last_active":"2023-10-08T00:11:41.068824+0000","last_peered":"2023-10-08T00:11:41.068824+0000","last_clean":"2023-10-08T00:11:41.068824+0000","last_became_active":"2023-10-08T00:11:41.068509+0000","last_became_peered":"2023-10-08T00:11:41.068509+0000","last_unstale":"2023-10-08T00:11:41.068824+0000","last_undegraded":"2023-10-08T00:11:41.068824+0000","last_fullsized":"2023-10-08T00:11:41.068824+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:57:19.584921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.064989+0000","last_change":"2023-10-08T00:11:40.301226+0000","last_active":"2023-10-08T00:11:41.064989+0000","last_peered":"2023-10-08T00:11:41.064989+0000","last_clean":"2023-10-08T00:11:41.064989+0000","last_became_active":"2023-10-08T00:11:40.300970+0000","last_became_peered":"2023-10-08T00:11:40.300970+0000","last_unstale":"2023-10-08T00:11:41.064989+0000","last_undegraded":"2023-10-08T00:11:41.064989+0000","last_fullsized":"2023-10-08T00:11:41.064989+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:20:01.582031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.074928+0000","last_change":"2023-10-08T00:11:41.074877+0000","last_active":"2023-10-08T00:11:41.074928+0000","last_peered":"2023-10-08T00:11:41.074928+0000","last_clean":"2023-10-08T00:11:41.074928+0000","last_became_active":"2023-10-08T00:11:41.074388+0000","last_became_peered":"2023-10-08T00:11:41.074388+0000","last_unstale":"2023-10-08T00:11:41.074928+0000","last_undegraded":"2023-10-08T00:11:41.074928+0000","last_fullsized":"2023-10-08T00:11:41.074928+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:32:58.999045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.064119+0000","last_change":"2023-10-08T00:11:40.305228+0000","last_active":"2023-10-08T00:11:41.064119+0000","last_peered":"2023-10-08T00:11:41.064119+0000","last_clean":"2023-10-08T00:11:41.064119+0000","last_became_active":"2023-10-08T00:11:40.304830+0000","last_became_peered":"2023-10-08T00:11:40.304830+0000","last_unstale":"2023-10-08T00:11:41.064119+0000","last_undegraded":"2023-10-08T00:11:41.064119+0000","last_fullsized":"2023-10-08T00:11:41.064119+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:21:08.277105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.071453+0000","last_change":"2023-10-08T00:11:41.071382+0000","last_active":"2023-10-08T00:11:41.071453+0000","last_peered":"2023-10-08T00:11:41.071453+0000","last_clean":"2023-10-08T00:11:41.071453+0000","last_became_active":"2023-10-08T00:11:41.070341+0000","last_became_peered":"2023-10-08T00:11:41.070341+0000","last_unstale":"2023-10-08T00:11:41.071453+0000","last_undegraded":"2023-10-08T00:11:41.071453+0000","last_fullsized":"2023-10-08T00:11:41.071453+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:16:52.644456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.059251+0000","last_change":"2023-10-08T00:11:40.299183+0000","last_active":"2023-10-08T00:11:41.059251+0000","last_peered":"2023-10-08T00:11:41.059251+0000","last_clean":"2023-10-08T00:11:41.059251+0000","last_became_active":"2023-10-08T00:11:40.298749+0000","last_became_peered":"2023-10-08T00:11:40.298749+0000","last_unstale":"2023-10-08T00:11:41.059251+0000","last_undegraded":"2023-10-08T00:11:41.059251+0000","last_fullsized":"2023-10-08T00:11:41.059251+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:41:40.921278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.067055+0000","last_change":"2023-10-08T00:11:41.063259+0000","last_active":"2023-10-08T00:11:41.067055+0000","last_peered":"2023-10-08T00:11:41.067055+0000","last_clean":"2023-10-08T00:11:41.067055+0000","last_became_active":"2023-10-08T00:11:41.063024+0000","last_became_peered":"2023-10-08T00:11:41.063024+0000","last_unstale":"2023-10-08T00:11:41.067055+0000","last_undegraded":"2023-10-08T00:11:41.067055+0000","last_fullsized":"2023-10-08T00:11:41.067055+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:52:15.180944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.065759+0000","last_change":"2023-10-08T00:11:40.307268+0000","last_active":"2023-10-08T00:11:41.065759+0000","last_peered":"2023-10-08T00:11:41.065759+0000","last_clean":"2023-10-08T00:11:41.065759+0000","last_became_active":"2023-10-08T00:11:40.306560+0000","last_became_peered":"2023-10-08T00:11:40.306560+0000","last_unstale":"2023-10-08T00:11:41.065759+0000","last_undegraded":"2023-10-08T00:11:41.065759+0000","last_fullsized":"2023-10-08T00:11:41.065759+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:20:01.582031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.069595+0000","last_change":"2023-10-08T00:11:41.069539+0000","last_active":"2023-10-08T00:11:41.069595+0000","last_peered":"2023-10-08T00:11:41.069595+0000","last_clean":"2023-10-08T00:11:41.069595+0000","last_became_active":"2023-10-08T00:11:41.068352+0000","last_became_peered":"2023-10-08T00:11:41.068352+0000","last_unstale":"2023-10-08T00:11:41.069595+0000","last_undegraded":"2023-10-08T00:11:41.069595+0000","last_fullsized":"2023-10-08T00:11:41.069595+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:07:25.684529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.060277+0000","last_change":"2023-10-08T00:11:40.302566+0000","last_active":"2023-10-08T00:11:41.060277+0000","last_peered":"2023-10-08T00:11:41.060277+0000","last_clean":"2023-10-08T00:11:41.060277+0000","last_became_active":"2023-10-08T00:11:40.302357+0000","last_became_peered":"2023-10-08T00:11:40.302357+0000","last_unstale":"2023-10-08T00:11:41.060277+0000","last_undegraded":"2023-10-08T00:11:41.060277+0000","last_fullsized":"2023-10-08T00:11:41.060277+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:57:18.580090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":171,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.064491+0000","last_change":"2023-10-08T00:10:43.728490+0000","last_active":"2023-10-08T00:11:41.064491+0000","last_peered":"2023-10-08T00:11:41.064491+0000","last_clean":"2023-10-08T00:11:41.064491+0000","last_became_active":"2023-10-08T00:10:43.728341+0000","last_became_peered":"2023-10-08T00:10:43.728341+0000","last_unstale":"2023-10-08T00:11:41.064491+0000","last_undegraded":"2023-10-08T00:11:41.064491+0000","last_fullsized":"2023-10-08T00:11:41.064491+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_clean_scrub_stamp":"2023-10-08T00:10:42.716096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:06:31.035293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574862,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":124,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195188,"statfs":{"total":200000000,"available":199872704,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.65100000000000002}]},{"osd":1,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.746}]},{"osd":2,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.53000000000000003}]},{"osd":3,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.63200000000000001}]},{"osd":4,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.78900000000000003}]},{"osd":5,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.72999999999999998}]},{"osd":6,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.81699999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574862,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":131,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195180,"statfs":{"total":200000000,"available":199864902,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]}]},{"osd":1,"up_from":13,"seq":55834574862,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":707,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194605,"statfs":{"total":200000000,"available":199276000,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.19500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574862,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":130,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195181,"statfs":{"total":200000000,"available":199866348,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.20300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574862,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":130,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195182,"statfs":{"total":200000000,"available":199866404,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]}]},{"osd":3,"up_from":13,"seq":55834574862,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":131,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195181,"statfs":{"total":200000000,"available":199865690,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.255}]},{"osd":1,"last update":"Sun Oct 8 00:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":12.351000000000001,"5min":12.351000000000001,"15min":12.351000000000001},"last":0.309},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.22600000000000001}]},{"osd":2,"last update":"Sun Oct 8 00:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.28100000000000003}]},{"osd":4,"last update":"Sun Oct 8 00:11:41 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":11.333,"5min":11.333,"15min":11.333},"last":0.54200000000000004},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":12.321999999999999,"5min":12.321999999999999,"15min":12.321999999999999},"last":0.55500000000000005}]},{"osd":5,"last update":"Sun Oct 8 00:11:41 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.181,"15min":1.181},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":12.291,"5min":12.291,"15min":12.291},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.52100000000000002}]},{"osd":6,"last update":"Sun Oct 8 00:11:41 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":11.288,"5min":11.288,"15min":11.288},"last":0.45000000000000001},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":12.253,"5min":12.253,"15min":12.253},"last":0.495}]},{"osd":7,"last update":"Sun Oct 8 00:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.44},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":11.363,"5min":11.363,"15min":11.363},"last":0.505}]}]},{"osd":4,"up_from":13,"seq":55834574862,"num_pgs":35,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":125,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195186,"statfs":{"total":200000000,"available":199871168,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.63400000000000001}]},{"osd":1,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.59099999999999997}]},{"osd":2,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.56399999999999995}]},{"osd":3,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.40400000000000003}]},{"osd":5,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.54500000000000004}]},{"osd":6,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.67100000000000004}]},{"osd":7,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.60899999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574862,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":707,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194604,"statfs":{"total":200000000,"available":199275516,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.67700000000000005}]},{"osd":1,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.61899999999999999}]},{"osd":2,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.435},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.57699999999999996}]},{"osd":3,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.64700000000000002}]},{"osd":4,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.84499999999999997}]},{"osd":6,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.745},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.70599999999999996}]},{"osd":7,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.82399999999999995}]}]}],"pool_statfs":[]}} 2023-10-08T00:11:45.359 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T00:11:45.507 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.506+0000 7f809c017700 1 Processor -- start 2023-10-08T00:11:45.507 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.507+0000 7f809c017700 1 -- start start 2023-10-08T00:11:45.509 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.507+0000 7f809c017700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f809406f1c0 0x7f8094161e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:45.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.507+0000 7f809c017700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f80940b15e0 0x7f8094164360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:45.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.507+0000 7f809c017700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80940b26d0 con 0x7f80940a79f0 2023-10-08T00:11:45.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.507+0000 7f809c017700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80940aeab0 con 0x7f80940b15e0 2023-10-08T00:11:45.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.507+0000 7f809c017700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80940b2b60 con 0x7f809406f1c0 2023-10-08T00:11:45.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.507+0000 7f8099db3700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f80940a79f0 0x7f809406eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:54202/0 (socket says 172.21.15.99:54202) 2023-10-08T00:11:45.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.507+0000 7f809a5b4700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f80940b15e0 0x7f8094164360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:45.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.507+0000 7f8099db3700 1 -- 172.21.15.99:0/343166736 learned_addr learned my addr 172.21.15.99:0/343166736 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:45.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.508+0000 7f80995b2700 1 --2- 172.21.15.99:0/343166736 >> v2:172.21.15.191:3300/0 conn(0x7f809406f1c0 0x7f8094161e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:45.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.508+0000 7f809a5b4700 1 -- 172.21.15.99:0/343166736 >> v2:172.21.15.191:3300/0 conn(0x7f809406f1c0 msgr2=0x7f8094161e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:45.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.508+0000 7f809a5b4700 1 --2- 172.21.15.99:0/343166736 >> v2:172.21.15.191:3300/0 conn(0x7f809406f1c0 0x7f8094161e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:45.511 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.508+0000 7f809a5b4700 1 -- 172.21.15.99:0/343166736 >> v1:172.21.15.99:6789/0 conn(0x7f80940a79f0 legacy=0x7f809406eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:45.511 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.508+0000 7f8098db1700 1 -- 172.21.15.99:0/343166736 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1286561834 0 0) 0x7f80940b26d0 con 0x7f80940a79f0 2023-10-08T00:11:45.511 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.508+0000 7f809a5b4700 1 -- 172.21.15.99:0/343166736 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80940b1ac0 con 0x7f80940b15e0 2023-10-08T00:11:45.511 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.508+0000 7f809a5b4700 1 --2- 172.21.15.99:0/343166736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80940b15e0 0x7f8094164360 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8094064320 tx=0x7f808c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f7a26cefb2d838d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:45.511 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.508+0000 7f8098db1700 1 -- 172.21.15.99:0/343166736 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808c00b6b0 con 0x7f80940b15e0 2023-10-08T00:11:45.511 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.508+0000 7f8098db1700 1 -- 172.21.15.99:0/343166736 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f808c00b810 con 0x7f80940b15e0 2023-10-08T00:11:45.511 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.508+0000 7f8098db1700 1 -- 172.21.15.99:0/343166736 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808c00bbb0 con 0x7f80940b15e0 2023-10-08T00:11:45.511 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.509+0000 7f809c017700 1 -- 172.21.15.99:0/343166736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80940b15e0 msgr2=0x7f8094164360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:45.511 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.509+0000 7f809c017700 1 --2- 172.21.15.99:0/343166736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80940b15e0 0x7f8094164360 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8094064320 tx=0x7f808c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:45.511 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.509+0000 7f809c017700 1 -- 172.21.15.99:0/343166736 shutdown_connections 2023-10-08T00:11:45.511 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.509+0000 7f809c017700 1 --2- 172.21.15.99:0/343166736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80940b15e0 0x7f8094164360 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:45.511 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.509+0000 7f809c017700 1 --2- 172.21.15.99:0/343166736 >> v2:172.21.15.191:3300/0 conn(0x7f809406f1c0 0x7f8094161e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:45.511 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.509+0000 7f809c017700 1 -- 172.21.15.99:0/343166736 >> 172.21.15.99:0/343166736 conn(0x7f80940cd1e0 msgr2=0x7f80940cd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:45.512 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.509+0000 7f809c017700 1 -- 172.21.15.99:0/343166736 shutdown_connections 2023-10-08T00:11:45.512 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.509+0000 7f809c017700 1 -- 172.21.15.99:0/343166736 wait complete. 2023-10-08T00:11:45.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.510+0000 7f809c017700 1 Processor -- start 2023-10-08T00:11:45.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.510+0000 7f809c017700 1 -- start start 2023-10-08T00:11:45.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.510+0000 7f809c017700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f809406f1c0 0x7f80940c3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:45.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.510+0000 7f809c017700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80940a79f0 0x7f8094099b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:45.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.510+0000 7f809c017700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80940b15e0 0x7f80941596d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:45.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.510+0000 7f809c017700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809416a100 con 0x7f80940a79f0 2023-10-08T00:11:45.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.511+0000 7f809c017700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809416a3a0 con 0x7f809406f1c0 2023-10-08T00:11:45.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.511+0000 7f809c017700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f809409ab90 con 0x7f80940b15e0 2023-10-08T00:11:45.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.511+0000 7f80995b2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80940a79f0 0x7f8094099b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:45.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.511+0000 7f80995b2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80940a79f0 0x7f8094099b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50578/0 (socket says 172.21.15.99:50578) 2023-10-08T00:11:45.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.511+0000 7f80995b2700 1 -- 172.21.15.99:0/4261675099 learned_addr learned my addr 172.21.15.99:0/4261675099 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:45.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.511+0000 7f80995b2700 1 -- 172.21.15.99:0/4261675099 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80940b15e0 msgr2=0x7f80941596d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:45.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.511+0000 7f80995b2700 1 --2- 172.21.15.99:0/4261675099 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80940b15e0 0x7f80941596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:45.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.511+0000 7f80995b2700 1 -- 172.21.15.99:0/4261675099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f809406f1c0 msgr2=0x7f80940c3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:45.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.511+0000 7f80995b2700 1 --2- 172.21.15.99:0/4261675099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f809406f1c0 0x7f80940c3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:45.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.511+0000 7f80995b2700 1 -- 172.21.15.99:0/4261675099 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f808c009420 con 0x7f80940a79f0 2023-10-08T00:11:45.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.511+0000 7f80995b2700 1 --2- 172.21.15.99:0/4261675099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80940a79f0 0x7f8094099b10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f808400d8c0 tx=0x7f8084008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:45.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.511+0000 7f8089ffb700 1 -- 172.21.15.99:0/4261675099 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808400b5d0 con 0x7f80940a79f0 2023-10-08T00:11:45.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.511+0000 7f8089ffb700 1 -- 172.21.15.99:0/4261675099 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f808400b770 con 0x7f80940a79f0 2023-10-08T00:11:45.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.511+0000 7f809c017700 1 -- 172.21.15.99:0/4261675099 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f809409a050 con 0x7f80940a79f0 2023-10-08T00:11:45.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.511+0000 7f8089ffb700 1 -- 172.21.15.99:0/4261675099 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808400bb30 con 0x7f80940a79f0 2023-10-08T00:11:45.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.511+0000 7f809c017700 1 -- 172.21.15.99:0/4261675099 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f809409a5d0 con 0x7f80940a79f0 2023-10-08T00:11:45.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.512+0000 7f809c017700 1 -- 172.21.15.99:0/4261675099 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f809404c3c0 con 0x7f80940a79f0 2023-10-08T00:11:45.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.512+0000 7f8089ffb700 1 -- 172.21.15.99:0/4261675099 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f808401e460 con 0x7f80940a79f0 2023-10-08T00:11:45.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.513+0000 7f8089ffb700 1 --2- 172.21.15.99:0/4261675099 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8078066f30 0x7f80780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:45.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.513+0000 7f8089ffb700 1 -- 172.21.15.99:0/4261675099 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8084084bc0 con 0x7f80940a79f0 2023-10-08T00:11:45.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.513+0000 7f8099db3700 1 --2- 172.21.15.99:0/4261675099 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8078066f30 0x7f80780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:45.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.513+0000 7f8099db3700 1 --2- 172.21.15.99:0/4261675099 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8078066f30 0x7f80780693e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8090001560 tx=0x7f80900016c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:45.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.517+0000 7f8089ffb700 1 -- 172.21.15.99:0/4261675099 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8084051e30 con 0x7f80940a79f0 2023-10-08T00:11:45.694 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.693+0000 7f809c017700 1 -- 172.21.15.99:0/4261675099 --> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f80940b1da0 con 0x7f8078066f30 2023-10-08T00:11:45.701 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.701+0000 7f8089ffb700 1 -- 172.21.15.99:0/4261675099 <== mgr.4099 v2:172.21.15.191:6832/85985 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+272712 (secure 0 0 0) 0x7f80940b1da0 con 0x7f8078066f30 2023-10-08T00:11:45.705 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:11:45.707 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.704+0000 7f809c017700 1 -- 172.21.15.99:0/4261675099 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8078066f30 msgr2=0x7f80780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:45.707 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.704+0000 7f809c017700 1 --2- 172.21.15.99:0/4261675099 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8078066f30 0x7f80780693e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8090001560 tx=0x7f80900016c0 comp rx=0 tx=0).stop 2023-10-08T00:11:45.708 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.704+0000 7f809c017700 1 -- 172.21.15.99:0/4261675099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80940a79f0 msgr2=0x7f8094099b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:45.708 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.704+0000 7f809c017700 1 --2- 172.21.15.99:0/4261675099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80940a79f0 0x7f8094099b10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f808400d8c0 tx=0x7f8084008040 comp rx=0 tx=0).stop 2023-10-08T00:11:45.708 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.704+0000 7f809c017700 1 -- 172.21.15.99:0/4261675099 shutdown_connections 2023-10-08T00:11:45.708 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.704+0000 7f809c017700 1 --2- 172.21.15.99:0/4261675099 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8078066f30 0x7f80780693e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:45.708 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.704+0000 7f809c017700 1 --2- 172.21.15.99:0/4261675099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f809406f1c0 0x7f80940c3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:45.708 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.704+0000 7f809c017700 1 --2- 172.21.15.99:0/4261675099 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80940b15e0 0x7f80941596d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:45.708 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.704+0000 7f809c017700 1 --2- 172.21.15.99:0/4261675099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80940a79f0 0x7f8094099b10 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:45.708 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.704+0000 7f809c017700 1 -- 172.21.15.99:0/4261675099 >> 172.21.15.99:0/4261675099 conn(0x7f80940cd1e0 msgr2=0x7f80940ad330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:45.708 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.704+0000 7f809c017700 1 -- 172.21.15.99:0/4261675099 shutdown_connections 2023-10-08T00:11:45.708 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.704+0000 7f809c017700 1 -- 172.21.15.99:0/4261675099 wait complete. 2023-10-08T00:11:45.708 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2023-10-08T00:11:45.717 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":52,"stamp":"2023-10-08T00:11:44.035908+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":114,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2188,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560311,"statfs":{"total":1600000000,"available":1597758732,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.746534"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.069656+0000","last_change":"2023-10-08T00:11:41.069601+0000","last_active":"2023-10-08T00:11:41.069656+0000","last_peered":"2023-10-08T00:11:41.069656+0000","last_clean":"2023-10-08T00:11:41.069656+0000","last_became_active":"2023-10-08T00:11:41.069388+0000","last_became_peered":"2023-10-08T00:11:41.069388+0000","last_unstale":"2023-10-08T00:11:41.069656+0000","last_undegraded":"2023-10-08T00:11:41.069656+0000","last_fullsized":"2023-10-08T00:11:41.069656+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:16:52.644456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.068405+0000","last_change":"2023-10-08T00:11:41.068179+0000","last_active":"2023-10-08T00:11:41.068405+0000","last_peered":"2023-10-08T00:11:41.068405+0000","last_clean":"2023-10-08T00:11:41.068405+0000","last_became_active":"2023-10-08T00:11:41.067614+0000","last_became_peered":"2023-10-08T00:11:41.067614+0000","last_unstale":"2023-10-08T00:11:41.068405+0000","last_undegraded":"2023-10-08T00:11:41.068405+0000","last_fullsized":"2023-10-08T00:11:41.068405+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:07:25.684529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.068085+0000","last_change":"2023-10-08T00:11:41.067952+0000","last_active":"2023-10-08T00:11:41.068085+0000","last_peered":"2023-10-08T00:11:41.068085+0000","last_clean":"2023-10-08T00:11:41.068085+0000","last_became_active":"2023-10-08T00:11:41.067727+0000","last_became_peered":"2023-10-08T00:11:41.067727+0000","last_unstale":"2023-10-08T00:11:41.068085+0000","last_undegraded":"2023-10-08T00:11:41.068085+0000","last_fullsized":"2023-10-08T00:11:41.068085+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:09:24.354082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.071107+0000","last_change":"2023-10-08T00:11:41.070804+0000","last_active":"2023-10-08T00:11:41.071107+0000","last_peered":"2023-10-08T00:11:41.071107+0000","last_clean":"2023-10-08T00:11:41.071107+0000","last_became_active":"2023-10-08T00:11:41.069994+0000","last_became_peered":"2023-10-08T00:11:41.069994+0000","last_unstale":"2023-10-08T00:11:41.071107+0000","last_undegraded":"2023-10-08T00:11:41.071107+0000","last_fullsized":"2023-10-08T00:11:41.071107+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:54:41.831036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.070097+0000","last_change":"2023-10-08T00:11:41.070030+0000","last_active":"2023-10-08T00:11:41.070097+0000","last_peered":"2023-10-08T00:11:41.070097+0000","last_clean":"2023-10-08T00:11:41.070097+0000","last_became_active":"2023-10-08T00:11:41.069799+0000","last_became_peered":"2023-10-08T00:11:41.069799+0000","last_unstale":"2023-10-08T00:11:41.070097+0000","last_undegraded":"2023-10-08T00:11:41.070097+0000","last_fullsized":"2023-10-08T00:11:41.070097+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:16:52.644456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.062305+0000","last_change":"2023-10-08T00:11:41.062237+0000","last_active":"2023-10-08T00:11:41.062305+0000","last_peered":"2023-10-08T00:11:41.062305+0000","last_clean":"2023-10-08T00:11:41.062305+0000","last_became_active":"2023-10-08T00:11:41.061947+0000","last_became_peered":"2023-10-08T00:11:41.061947+0000","last_unstale":"2023-10-08T00:11:41.062305+0000","last_undegraded":"2023-10-08T00:11:41.062305+0000","last_fullsized":"2023-10-08T00:11:41.062305+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:41:41.926109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.067283+0000","last_change":"2023-10-08T00:11:41.065080+0000","last_active":"2023-10-08T00:11:41.067283+0000","last_peered":"2023-10-08T00:11:41.067283+0000","last_clean":"2023-10-08T00:11:41.067283+0000","last_became_active":"2023-10-08T00:11:41.064579+0000","last_became_peered":"2023-10-08T00:11:41.064579+0000","last_unstale":"2023-10-08T00:11:41.067283+0000","last_undegraded":"2023-10-08T00:11:41.067283+0000","last_fullsized":"2023-10-08T00:11:41.067283+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:53:16.932728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.072042+0000","last_change":"2023-10-08T00:11:41.071983+0000","last_active":"2023-10-08T00:11:41.072042+0000","last_peered":"2023-10-08T00:11:41.072042+0000","last_clean":"2023-10-08T00:11:41.072042+0000","last_became_active":"2023-10-08T00:11:41.071608+0000","last_became_peered":"2023-10-08T00:11:41.071608+0000","last_unstale":"2023-10-08T00:11:41.072042+0000","last_undegraded":"2023-10-08T00:11:41.072042+0000","last_fullsized":"2023-10-08T00:11:41.072042+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:12:29.493504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.070961+0000","last_change":"2023-10-08T00:11:41.070911+0000","last_active":"2023-10-08T00:11:41.070961+0000","last_peered":"2023-10-08T00:11:41.070961+0000","last_clean":"2023-10-08T00:11:41.070961+0000","last_became_active":"2023-10-08T00:11:41.070696+0000","last_became_peered":"2023-10-08T00:11:41.070696+0000","last_unstale":"2023-10-08T00:11:41.070961+0000","last_undegraded":"2023-10-08T00:11:41.070961+0000","last_fullsized":"2023-10-08T00:11:41.070961+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:13:40.235186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.065444+0000","last_change":"2023-10-08T00:11:41.065297+0000","last_active":"2023-10-08T00:11:41.065444+0000","last_peered":"2023-10-08T00:11:41.065444+0000","last_clean":"2023-10-08T00:11:41.065444+0000","last_became_active":"2023-10-08T00:11:41.065062+0000","last_became_peered":"2023-10-08T00:11:41.065062+0000","last_unstale":"2023-10-08T00:11:41.065444+0000","last_undegraded":"2023-10-08T00:11:41.065444+0000","last_fullsized":"2023-10-08T00:11:41.065444+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:02:14.124345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.067108+0000","last_change":"2023-10-08T00:11:41.065279+0000","last_active":"2023-10-08T00:11:41.067108+0000","last_peered":"2023-10-08T00:11:41.067108+0000","last_clean":"2023-10-08T00:11:41.067108+0000","last_became_active":"2023-10-08T00:11:41.065066+0000","last_became_peered":"2023-10-08T00:11:41.065066+0000","last_unstale":"2023-10-08T00:11:41.067108+0000","last_undegraded":"2023-10-08T00:11:41.067108+0000","last_fullsized":"2023-10-08T00:11:41.067108+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:02:35.995739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.066233+0000","last_change":"2023-10-08T00:11:41.066145+0000","last_active":"2023-10-08T00:11:41.066233+0000","last_peered":"2023-10-08T00:11:41.066233+0000","last_clean":"2023-10-08T00:11:41.066233+0000","last_became_active":"2023-10-08T00:11:41.065766+0000","last_became_peered":"2023-10-08T00:11:41.065766+0000","last_unstale":"2023-10-08T00:11:41.066233+0000","last_undegraded":"2023-10-08T00:11:41.066233+0000","last_fullsized":"2023-10-08T00:11:41.066233+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:41:41.926109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.074433+0000","last_change":"2023-10-08T00:11:41.074355+0000","last_active":"2023-10-08T00:11:41.074433+0000","last_peered":"2023-10-08T00:11:41.074433+0000","last_clean":"2023-10-08T00:11:41.074433+0000","last_became_active":"2023-10-08T00:11:41.072388+0000","last_became_peered":"2023-10-08T00:11:41.072388+0000","last_unstale":"2023-10-08T00:11:41.074433+0000","last_undegraded":"2023-10-08T00:11:41.074433+0000","last_fullsized":"2023-10-08T00:11:41.074433+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:04:17.729964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.068488+0000","last_change":"2023-10-08T00:11:41.067908+0000","last_active":"2023-10-08T00:11:41.068488+0000","last_peered":"2023-10-08T00:11:41.068488+0000","last_clean":"2023-10-08T00:11:41.068488+0000","last_became_active":"2023-10-08T00:11:41.067520+0000","last_became_peered":"2023-10-08T00:11:41.067520+0000","last_unstale":"2023-10-08T00:11:41.068488+0000","last_undegraded":"2023-10-08T00:11:41.068488+0000","last_fullsized":"2023-10-08T00:11:41.068488+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:12:29.493504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.069923+0000","last_change":"2023-10-08T00:11:41.069811+0000","last_active":"2023-10-08T00:11:41.069923+0000","last_peered":"2023-10-08T00:11:41.069923+0000","last_clean":"2023-10-08T00:11:41.069923+0000","last_became_active":"2023-10-08T00:11:41.069334+0000","last_became_peered":"2023-10-08T00:11:41.069334+0000","last_unstale":"2023-10-08T00:11:41.069923+0000","last_undegraded":"2023-10-08T00:11:41.069923+0000","last_fullsized":"2023-10-08T00:11:41.069923+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:53:16.932728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.069265+0000","last_change":"2023-10-08T00:11:41.069072+0000","last_active":"2023-10-08T00:11:41.069265+0000","last_peered":"2023-10-08T00:11:41.069265+0000","last_clean":"2023-10-08T00:11:41.069265+0000","last_became_active":"2023-10-08T00:11:41.068814+0000","last_became_peered":"2023-10-08T00:11:41.068814+0000","last_unstale":"2023-10-08T00:11:41.069265+0000","last_undegraded":"2023-10-08T00:11:41.069265+0000","last_fullsized":"2023-10-08T00:11:41.069265+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:16:52.644456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.068930+0000","last_change":"2023-10-08T00:11:41.068867+0000","last_active":"2023-10-08T00:11:41.068930+0000","last_peered":"2023-10-08T00:11:41.068930+0000","last_clean":"2023-10-08T00:11:41.068930+0000","last_became_active":"2023-10-08T00:11:41.068030+0000","last_became_peered":"2023-10-08T00:11:41.068030+0000","last_unstale":"2023-10-08T00:11:41.068930+0000","last_undegraded":"2023-10-08T00:11:41.068930+0000","last_fullsized":"2023-10-08T00:11:41.068930+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:47:29.945735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.071539+0000","last_change":"2023-10-08T00:11:41.071391+0000","last_active":"2023-10-08T00:11:41.071539+0000","last_peered":"2023-10-08T00:11:41.071539+0000","last_clean":"2023-10-08T00:11:41.071539+0000","last_became_active":"2023-10-08T00:11:41.070979+0000","last_became_peered":"2023-10-08T00:11:41.070979+0000","last_unstale":"2023-10-08T00:11:41.071539+0000","last_undegraded":"2023-10-08T00:11:41.071539+0000","last_fullsized":"2023-10-08T00:11:41.071539+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:13:40.235186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.067542+0000","last_change":"2023-10-08T00:11:41.066614+0000","last_active":"2023-10-08T00:11:41.067542+0000","last_peered":"2023-10-08T00:11:41.067542+0000","last_clean":"2023-10-08T00:11:41.067542+0000","last_became_active":"2023-10-08T00:11:41.066339+0000","last_became_peered":"2023-10-08T00:11:41.066339+0000","last_unstale":"2023-10-08T00:11:41.067542+0000","last_undegraded":"2023-10-08T00:11:41.067542+0000","last_fullsized":"2023-10-08T00:11:41.067542+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:21:58.588524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.073159+0000","last_change":"2023-10-08T00:11:41.073093+0000","last_active":"2023-10-08T00:11:41.073159+0000","last_peered":"2023-10-08T00:11:41.073159+0000","last_clean":"2023-10-08T00:11:41.073159+0000","last_became_active":"2023-10-08T00:11:41.072306+0000","last_became_peered":"2023-10-08T00:11:41.072306+0000","last_unstale":"2023-10-08T00:11:41.073159+0000","last_undegraded":"2023-10-08T00:11:41.073159+0000","last_fullsized":"2023-10-08T00:11:41.073159+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:03:28.086788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.068677+0000","last_change":"2023-10-08T00:11:41.068583+0000","last_active":"2023-10-08T00:11:41.068677+0000","last_peered":"2023-10-08T00:11:41.068677+0000","last_clean":"2023-10-08T00:11:41.068677+0000","last_became_active":"2023-10-08T00:11:41.067298+0000","last_became_peered":"2023-10-08T00:11:41.067298+0000","last_unstale":"2023-10-08T00:11:41.068677+0000","last_undegraded":"2023-10-08T00:11:41.068677+0000","last_fullsized":"2023-10-08T00:11:41.068677+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:19:12.100172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.067304+0000","last_change":"2023-10-08T00:11:41.065215+0000","last_active":"2023-10-08T00:11:41.067304+0000","last_peered":"2023-10-08T00:11:41.067304+0000","last_clean":"2023-10-08T00:11:41.067304+0000","last_became_active":"2023-10-08T00:11:41.064860+0000","last_became_peered":"2023-10-08T00:11:41.064860+0000","last_unstale":"2023-10-08T00:11:41.067304+0000","last_undegraded":"2023-10-08T00:11:41.067304+0000","last_fullsized":"2023-10-08T00:11:41.067304+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:54:41.831036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.074595+0000","last_change":"2023-10-08T00:11:41.074524+0000","last_active":"2023-10-08T00:11:41.074595+0000","last_peered":"2023-10-08T00:11:41.074595+0000","last_clean":"2023-10-08T00:11:41.074595+0000","last_became_active":"2023-10-08T00:11:41.073858+0000","last_became_peered":"2023-10-08T00:11:41.073858+0000","last_unstale":"2023-10-08T00:11:41.074595+0000","last_undegraded":"2023-10-08T00:11:41.074595+0000","last_fullsized":"2023-10-08T00:11:41.074595+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:02:14.124345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.070387+0000","last_change":"2023-10-08T00:11:41.070325+0000","last_active":"2023-10-08T00:11:41.070387+0000","last_peered":"2023-10-08T00:11:41.070387+0000","last_clean":"2023-10-08T00:11:41.070387+0000","last_became_active":"2023-10-08T00:11:41.069924+0000","last_became_peered":"2023-10-08T00:11:41.069924+0000","last_unstale":"2023-10-08T00:11:41.070387+0000","last_undegraded":"2023-10-08T00:11:41.070387+0000","last_fullsized":"2023-10-08T00:11:41.070387+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:21:58.487927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.071685+0000","last_change":"2023-10-08T00:11:41.071473+0000","last_active":"2023-10-08T00:11:41.071685+0000","last_peered":"2023-10-08T00:11:41.071685+0000","last_clean":"2023-10-08T00:11:41.071685+0000","last_became_active":"2023-10-08T00:11:41.069876+0000","last_became_peered":"2023-10-08T00:11:41.069876+0000","last_unstale":"2023-10-08T00:11:41.071685+0000","last_undegraded":"2023-10-08T00:11:41.071685+0000","last_fullsized":"2023-10-08T00:11:41.071685+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:57:19.584921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.070270+0000","last_change":"2023-10-08T00:11:41.069110+0000","last_active":"2023-10-08T00:11:41.070270+0000","last_peered":"2023-10-08T00:11:41.070270+0000","last_clean":"2023-10-08T00:11:41.070270+0000","last_became_active":"2023-10-08T00:11:41.068098+0000","last_became_peered":"2023-10-08T00:11:41.068098+0000","last_unstale":"2023-10-08T00:11:41.070270+0000","last_undegraded":"2023-10-08T00:11:41.070270+0000","last_fullsized":"2023-10-08T00:11:41.070270+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:04:17.729964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.069387+0000","last_change":"2023-10-08T00:11:41.069322+0000","last_active":"2023-10-08T00:11:41.069387+0000","last_peered":"2023-10-08T00:11:41.069387+0000","last_clean":"2023-10-08T00:11:41.069387+0000","last_became_active":"2023-10-08T00:11:41.069092+0000","last_became_peered":"2023-10-08T00:11:41.069092+0000","last_unstale":"2023-10-08T00:11:41.069387+0000","last_undegraded":"2023-10-08T00:11:41.069387+0000","last_fullsized":"2023-10-08T00:11:41.069387+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:56:57.764426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.070642+0000","last_change":"2023-10-08T00:11:41.070585+0000","last_active":"2023-10-08T00:11:41.070642+0000","last_peered":"2023-10-08T00:11:41.070642+0000","last_clean":"2023-10-08T00:11:41.070642+0000","last_became_active":"2023-10-08T00:11:41.070379+0000","last_became_peered":"2023-10-08T00:11:41.070379+0000","last_unstale":"2023-10-08T00:11:41.070642+0000","last_undegraded":"2023-10-08T00:11:41.070642+0000","last_fullsized":"2023-10-08T00:11:41.070642+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:49:25.596070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.068760+0000","last_change":"2023-10-08T00:11:41.068702+0000","last_active":"2023-10-08T00:11:41.068760+0000","last_peered":"2023-10-08T00:11:41.068760+0000","last_clean":"2023-10-08T00:11:41.068760+0000","last_became_active":"2023-10-08T00:11:41.068455+0000","last_became_peered":"2023-10-08T00:11:41.068455+0000","last_unstale":"2023-10-08T00:11:41.068760+0000","last_undegraded":"2023-10-08T00:11:41.068760+0000","last_fullsized":"2023-10-08T00:11:41.068760+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:56:57.764426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.069907+0000","last_change":"2023-10-08T00:11:41.069678+0000","last_active":"2023-10-08T00:11:41.069907+0000","last_peered":"2023-10-08T00:11:41.069907+0000","last_clean":"2023-10-08T00:11:41.069907+0000","last_became_active":"2023-10-08T00:11:41.069048+0000","last_became_peered":"2023-10-08T00:11:41.069048+0000","last_unstale":"2023-10-08T00:11:41.069907+0000","last_undegraded":"2023-10-08T00:11:41.069907+0000","last_fullsized":"2023-10-08T00:11:41.069907+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:49:25.596070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.061458+0000","last_change":"2023-10-08T00:11:41.061399+0000","last_active":"2023-10-08T00:11:41.061458+0000","last_peered":"2023-10-08T00:11:41.061458+0000","last_clean":"2023-10-08T00:11:41.061458+0000","last_became_active":"2023-10-08T00:11:41.061202+0000","last_became_peered":"2023-10-08T00:11:41.061202+0000","last_unstale":"2023-10-08T00:11:41.061458+0000","last_undegraded":"2023-10-08T00:11:41.061458+0000","last_fullsized":"2023-10-08T00:11:41.061458+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:56:57.764426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.069662+0000","last_change":"2023-10-08T00:11:41.069189+0000","last_active":"2023-10-08T00:11:41.069662+0000","last_peered":"2023-10-08T00:11:41.069662+0000","last_clean":"2023-10-08T00:11:41.069662+0000","last_became_active":"2023-10-08T00:11:41.068628+0000","last_became_peered":"2023-10-08T00:11:41.068628+0000","last_unstale":"2023-10-08T00:11:41.069662+0000","last_undegraded":"2023-10-08T00:11:41.069662+0000","last_fullsized":"2023-10-08T00:11:41.069662+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:56:18.539290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.070155+0000","last_change":"2023-10-08T00:11:41.069413+0000","last_active":"2023-10-08T00:11:41.070155+0000","last_peered":"2023-10-08T00:11:41.070155+0000","last_clean":"2023-10-08T00:11:41.070155+0000","last_became_active":"2023-10-08T00:11:41.068721+0000","last_became_peered":"2023-10-08T00:11:41.068721+0000","last_unstale":"2023-10-08T00:11:41.070155+0000","last_undegraded":"2023-10-08T00:11:41.070155+0000","last_fullsized":"2023-10-08T00:11:41.070155+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:32:58.999045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"30'4","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.301875+0000","last_change":"2023-10-08T00:11:40.300462+0000","last_active":"2023-10-08T00:11:41.301875+0000","last_peered":"2023-10-08T00:11:41.301875+0000","last_clean":"2023-10-08T00:11:41.301875+0000","last_became_active":"2023-10-08T00:11:40.298899+0000","last_became_peered":"2023-10-08T00:11:40.298899+0000","last_unstale":"2023-10-08T00:11:41.301875+0000","last_undegraded":"2023-10-08T00:11:41.301875+0000","last_fullsized":"2023-10-08T00:11:41.301875+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:16:51.639625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.071926+0000","last_change":"2023-10-08T00:11:41.071838+0000","last_active":"2023-10-08T00:11:41.071926+0000","last_peered":"2023-10-08T00:11:41.071926+0000","last_clean":"2023-10-08T00:11:41.071926+0000","last_became_active":"2023-10-08T00:11:41.071274+0000","last_became_peered":"2023-10-08T00:11:41.071274+0000","last_unstale":"2023-10-08T00:11:41.071926+0000","last_undegraded":"2023-10-08T00:11:41.071926+0000","last_fullsized":"2023-10-08T00:11:41.071926+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:07:25.684529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"30'1","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.298504+0000","last_change":"2023-10-08T00:11:40.307137+0000","last_active":"2023-10-08T00:11:41.298504+0000","last_peered":"2023-10-08T00:11:41.298504+0000","last_clean":"2023-10-08T00:11:41.298504+0000","last_became_active":"2023-10-08T00:11:40.306100+0000","last_became_peered":"2023-10-08T00:11:40.306100+0000","last_unstale":"2023-10-08T00:11:41.298504+0000","last_undegraded":"2023-10-08T00:11:41.298504+0000","last_fullsized":"2023-10-08T00:11:41.298504+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:41:40.921278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.067343+0000","last_change":"2023-10-08T00:11:41.066055+0000","last_active":"2023-10-08T00:11:41.067343+0000","last_peered":"2023-10-08T00:11:41.067343+0000","last_clean":"2023-10-08T00:11:41.067343+0000","last_became_active":"2023-10-08T00:11:41.065834+0000","last_became_peered":"2023-10-08T00:11:41.065834+0000","last_unstale":"2023-10-08T00:11:41.067343+0000","last_undegraded":"2023-10-08T00:11:41.067343+0000","last_fullsized":"2023-10-08T00:11:41.067343+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:51:15.073889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"30'2","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.298538+0000","last_change":"2023-10-08T00:11:40.299193+0000","last_active":"2023-10-08T00:11:41.298538+0000","last_peered":"2023-10-08T00:11:41.298538+0000","last_clean":"2023-10-08T00:11:41.298538+0000","last_became_active":"2023-10-08T00:11:40.298932+0000","last_became_peered":"2023-10-08T00:11:40.298932+0000","last_unstale":"2023-10-08T00:11:41.298538+0000","last_undegraded":"2023-10-08T00:11:41.298538+0000","last_fullsized":"2023-10-08T00:11:41.298538+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:12:47.778998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.071844+0000","last_change":"2023-10-08T00:11:41.071781+0000","last_active":"2023-10-08T00:11:41.071844+0000","last_peered":"2023-10-08T00:11:41.071844+0000","last_clean":"2023-10-08T00:11:41.071844+0000","last_became_active":"2023-10-08T00:11:41.071234+0000","last_became_peered":"2023-10-08T00:11:41.071234+0000","last_unstale":"2023-10-08T00:11:41.071844+0000","last_undegraded":"2023-10-08T00:11:41.071844+0000","last_fullsized":"2023-10-08T00:11:41.071844+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:56:50.566568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"30'2","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.301986+0000","last_change":"2023-10-08T00:11:40.302878+0000","last_active":"2023-10-08T00:11:41.301986+0000","last_peered":"2023-10-08T00:11:41.301986+0000","last_clean":"2023-10-08T00:11:41.301986+0000","last_became_active":"2023-10-08T00:11:40.301405+0000","last_became_peered":"2023-10-08T00:11:40.301405+0000","last_unstale":"2023-10-08T00:11:41.301986+0000","last_undegraded":"2023-10-08T00:11:41.301986+0000","last_fullsized":"2023-10-08T00:11:41.301986+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:11:55.751058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.065414+0000","last_change":"2023-10-08T00:11:41.065340+0000","last_active":"2023-10-08T00:11:41.065414+0000","last_peered":"2023-10-08T00:11:41.065414+0000","last_clean":"2023-10-08T00:11:41.065414+0000","last_became_active":"2023-10-08T00:11:41.065113+0000","last_became_peered":"2023-10-08T00:11:41.065113+0000","last_unstale":"2023-10-08T00:11:41.065414+0000","last_undegraded":"2023-10-08T00:11:41.065414+0000","last_fullsized":"2023-10-08T00:11:41.065414+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:03:46.298642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"30'1","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.298630+0000","last_change":"2023-10-08T00:11:40.294854+0000","last_active":"2023-10-08T00:11:41.298630+0000","last_peered":"2023-10-08T00:11:41.298630+0000","last_clean":"2023-10-08T00:11:41.298630+0000","last_became_active":"2023-10-08T00:11:40.294631+0000","last_became_peered":"2023-10-08T00:11:40.294631+0000","last_unstale":"2023-10-08T00:11:41.298630+0000","last_undegraded":"2023-10-08T00:11:41.298630+0000","last_fullsized":"2023-10-08T00:11:41.298630+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:20:01.582031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.067220+0000","last_change":"2023-10-08T00:11:41.062485+0000","last_active":"2023-10-08T00:11:41.067220+0000","last_peered":"2023-10-08T00:11:41.067220+0000","last_clean":"2023-10-08T00:11:41.067220+0000","last_became_active":"2023-10-08T00:11:41.062269+0000","last_became_peered":"2023-10-08T00:11:41.062269+0000","last_unstale":"2023-10-08T00:11:41.067220+0000","last_undegraded":"2023-10-08T00:11:41.067220+0000","last_fullsized":"2023-10-08T00:11:41.067220+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:57:19.584921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"30'1","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.299563+0000","last_change":"2023-10-08T00:11:40.302118+0000","last_active":"2023-10-08T00:11:41.299563+0000","last_peered":"2023-10-08T00:11:41.299563+0000","last_clean":"2023-10-08T00:11:41.299563+0000","last_became_active":"2023-10-08T00:11:40.300756+0000","last_became_peered":"2023-10-08T00:11:40.300756+0000","last_unstale":"2023-10-08T00:11:41.299563+0000","last_undegraded":"2023-10-08T00:11:41.299563+0000","last_fullsized":"2023-10-08T00:11:41.299563+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:13:39.230355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.071434+0000","last_change":"2023-10-08T00:11:41.071377+0000","last_active":"2023-10-08T00:11:41.071434+0000","last_peered":"2023-10-08T00:11:41.071434+0000","last_clean":"2023-10-08T00:11:41.071434+0000","last_became_active":"2023-10-08T00:11:41.071149+0000","last_became_peered":"2023-10-08T00:11:41.071149+0000","last_unstale":"2023-10-08T00:11:41.071434+0000","last_undegraded":"2023-10-08T00:11:41.071434+0000","last_fullsized":"2023-10-08T00:11:41.071434+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:20:02.586863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.060428+0000","last_change":"2023-10-08T00:11:40.302427+0000","last_active":"2023-10-08T00:11:41.060428+0000","last_peered":"2023-10-08T00:11:41.060428+0000","last_clean":"2023-10-08T00:11:41.060428+0000","last_became_active":"2023-10-08T00:11:40.302196+0000","last_became_peered":"2023-10-08T00:11:40.302196+0000","last_unstale":"2023-10-08T00:11:41.060428+0000","last_undegraded":"2023-10-08T00:11:41.060428+0000","last_fullsized":"2023-10-08T00:11:41.060428+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:49:46.767411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.073874+0000","last_change":"2023-10-08T00:11:41.073809+0000","last_active":"2023-10-08T00:11:41.073874+0000","last_peered":"2023-10-08T00:11:41.073874+0000","last_clean":"2023-10-08T00:11:41.073874+0000","last_became_active":"2023-10-08T00:11:41.073353+0000","last_became_peered":"2023-10-08T00:11:41.073353+0000","last_unstale":"2023-10-08T00:11:41.073874+0000","last_undegraded":"2023-10-08T00:11:41.073874+0000","last_fullsized":"2023-10-08T00:11:41.073874+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:49:25.596070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.058298+0000","last_change":"2023-10-08T00:11:40.299748+0000","last_active":"2023-10-08T00:11:41.058298+0000","last_peered":"2023-10-08T00:11:41.058298+0000","last_clean":"2023-10-08T00:11:41.058298+0000","last_became_active":"2023-10-08T00:11:40.298913+0000","last_became_peered":"2023-10-08T00:11:40.298913+0000","last_unstale":"2023-10-08T00:11:41.058298+0000","last_undegraded":"2023-10-08T00:11:41.058298+0000","last_fullsized":"2023-10-08T00:11:41.058298+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:23:12.401507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.067092+0000","last_change":"2023-10-08T00:11:41.065209+0000","last_active":"2023-10-08T00:11:41.067092+0000","last_peered":"2023-10-08T00:11:41.067092+0000","last_clean":"2023-10-08T00:11:41.067092+0000","last_became_active":"2023-10-08T00:11:41.065002+0000","last_became_peered":"2023-10-08T00:11:41.065002+0000","last_unstale":"2023-10-08T00:11:41.067092+0000","last_undegraded":"2023-10-08T00:11:41.067092+0000","last_fullsized":"2023-10-08T00:11:41.067092+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:14:53.717627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.062113+0000","last_change":"2023-10-08T00:11:40.299963+0000","last_active":"2023-10-08T00:11:41.062113+0000","last_peered":"2023-10-08T00:11:41.062113+0000","last_clean":"2023-10-08T00:11:41.062113+0000","last_became_active":"2023-10-08T00:11:40.299725+0000","last_became_peered":"2023-10-08T00:11:40.299725+0000","last_unstale":"2023-10-08T00:11:41.062113+0000","last_undegraded":"2023-10-08T00:11:41.062113+0000","last_fullsized":"2023-10-08T00:11:41.062113+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:32:57.994214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.071127+0000","last_change":"2023-10-08T00:11:41.071031+0000","last_active":"2023-10-08T00:11:41.071127+0000","last_peered":"2023-10-08T00:11:41.071127+0000","last_clean":"2023-10-08T00:11:41.071127+0000","last_became_active":"2023-10-08T00:11:41.070498+0000","last_became_peered":"2023-10-08T00:11:41.070498+0000","last_unstale":"2023-10-08T00:11:41.071127+0000","last_undegraded":"2023-10-08T00:11:41.071127+0000","last_fullsized":"2023-10-08T00:11:41.071127+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:52:08.436392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.063154+0000","last_change":"2023-10-08T00:11:40.299048+0000","last_active":"2023-10-08T00:11:41.063154+0000","last_peered":"2023-10-08T00:11:41.063154+0000","last_clean":"2023-10-08T00:11:41.063154+0000","last_became_active":"2023-10-08T00:11:40.298886+0000","last_became_peered":"2023-10-08T00:11:40.298886+0000","last_unstale":"2023-10-08T00:11:41.063154+0000","last_undegraded":"2023-10-08T00:11:41.063154+0000","last_fullsized":"2023-10-08T00:11:41.063154+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:09:12.065285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.074846+0000","last_change":"2023-10-08T00:11:41.074787+0000","last_active":"2023-10-08T00:11:41.074846+0000","last_peered":"2023-10-08T00:11:41.074846+0000","last_clean":"2023-10-08T00:11:41.074846+0000","last_became_active":"2023-10-08T00:11:41.074028+0000","last_became_peered":"2023-10-08T00:11:41.074028+0000","last_unstale":"2023-10-08T00:11:41.074846+0000","last_undegraded":"2023-10-08T00:11:41.074846+0000","last_fullsized":"2023-10-08T00:11:41.074846+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:56:57.764426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"30'3","reported_seq":13,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.300937+0000","last_change":"2023-10-08T00:11:40.301508+0000","last_active":"2023-10-08T00:11:41.300937+0000","last_peered":"2023-10-08T00:11:41.300937+0000","last_clean":"2023-10-08T00:11:41.300937+0000","last_became_active":"2023-10-08T00:11:40.301290+0000","last_became_peered":"2023-10-08T00:11:40.301290+0000","last_unstale":"2023-10-08T00:11:41.300937+0000","last_undegraded":"2023-10-08T00:11:41.300937+0000","last_fullsized":"2023-10-08T00:11:41.300937+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:56:56.759595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.067123+0000","last_change":"2023-10-08T00:11:41.065594+0000","last_active":"2023-10-08T00:11:41.067123+0000","last_peered":"2023-10-08T00:11:41.067123+0000","last_clean":"2023-10-08T00:11:41.067123+0000","last_became_active":"2023-10-08T00:11:41.065385+0000","last_became_peered":"2023-10-08T00:11:41.065385+0000","last_unstale":"2023-10-08T00:11:41.067123+0000","last_undegraded":"2023-10-08T00:11:41.067123+0000","last_fullsized":"2023-10-08T00:11:41.067123+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:54:53.692310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"30'6","reported_seq":16,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.301998+0000","last_change":"2023-10-08T00:11:40.307488+0000","last_active":"2023-10-08T00:11:41.301998+0000","last_peered":"2023-10-08T00:11:41.301998+0000","last_clean":"2023-10-08T00:11:41.301998+0000","last_became_active":"2023-10-08T00:11:40.307258+0000","last_became_peered":"2023-10-08T00:11:40.307258+0000","last_unstale":"2023-10-08T00:11:41.301998+0000","last_undegraded":"2023-10-08T00:11:41.301998+0000","last_fullsized":"2023-10-08T00:11:41.301998+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:09:23.349251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.073179+0000","last_change":"2023-10-08T00:11:41.073139+0000","last_active":"2023-10-08T00:11:41.073179+0000","last_peered":"2023-10-08T00:11:41.073179+0000","last_clean":"2023-10-08T00:11:41.073179+0000","last_became_active":"2023-10-08T00:11:41.072647+0000","last_became_peered":"2023-10-08T00:11:41.072647+0000","last_unstale":"2023-10-08T00:11:41.073179+0000","last_undegraded":"2023-10-08T00:11:41.073179+0000","last_fullsized":"2023-10-08T00:11:41.073179+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:13:05.134473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"30'2","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.302322+0000","last_change":"2023-10-08T00:11:40.299984+0000","last_active":"2023-10-08T00:11:41.302322+0000","last_peered":"2023-10-08T00:11:41.302322+0000","last_clean":"2023-10-08T00:11:41.302322+0000","last_became_active":"2023-10-08T00:11:40.299532+0000","last_became_peered":"2023-10-08T00:11:40.299532+0000","last_unstale":"2023-10-08T00:11:41.302322+0000","last_undegraded":"2023-10-08T00:11:41.302322+0000","last_fullsized":"2023-10-08T00:11:41.302322+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:11:55.751058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.071707+0000","last_change":"2023-10-08T00:11:41.071633+0000","last_active":"2023-10-08T00:11:41.071707+0000","last_peered":"2023-10-08T00:11:41.071707+0000","last_clean":"2023-10-08T00:11:41.071707+0000","last_became_active":"2023-10-08T00:11:41.070388+0000","last_became_peered":"2023-10-08T00:11:41.070388+0000","last_unstale":"2023-10-08T00:11:41.071707+0000","last_undegraded":"2023-10-08T00:11:41.071707+0000","last_fullsized":"2023-10-08T00:11:41.071707+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:09:13.070116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.064146+0000","last_change":"2023-10-08T00:11:40.303633+0000","last_active":"2023-10-08T00:11:41.064146+0000","last_peered":"2023-10-08T00:11:41.064146+0000","last_clean":"2023-10-08T00:11:41.064146+0000","last_became_active":"2023-10-08T00:11:40.303137+0000","last_became_peered":"2023-10-08T00:11:40.303137+0000","last_unstale":"2023-10-08T00:11:41.064146+0000","last_undegraded":"2023-10-08T00:11:41.064146+0000","last_fullsized":"2023-10-08T00:11:41.064146+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:21:08.277105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.067321+0000","last_change":"2023-10-08T00:11:41.065872+0000","last_active":"2023-10-08T00:11:41.067321+0000","last_peered":"2023-10-08T00:11:41.067321+0000","last_clean":"2023-10-08T00:11:41.067321+0000","last_became_active":"2023-10-08T00:11:41.065573+0000","last_became_peered":"2023-10-08T00:11:41.065573+0000","last_unstale":"2023-10-08T00:11:41.067321+0000","last_undegraded":"2023-10-08T00:11:41.067321+0000","last_fullsized":"2023-10-08T00:11:41.067321+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:02:14.124345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.062601+0000","last_change":"2023-10-08T00:11:40.301793+0000","last_active":"2023-10-08T00:11:41.062601+0000","last_peered":"2023-10-08T00:11:41.062601+0000","last_clean":"2023-10-08T00:11:41.062601+0000","last_became_active":"2023-10-08T00:11:40.301229+0000","last_became_peered":"2023-10-08T00:11:40.301229+0000","last_unstale":"2023-10-08T00:11:41.062601+0000","last_undegraded":"2023-10-08T00:11:41.062601+0000","last_fullsized":"2023-10-08T00:11:41.062601+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:35:01.644676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.069197+0000","last_change":"2023-10-08T00:11:41.069063+0000","last_active":"2023-10-08T00:11:41.069197+0000","last_peered":"2023-10-08T00:11:41.069197+0000","last_clean":"2023-10-08T00:11:41.069197+0000","last_became_active":"2023-10-08T00:11:41.068310+0000","last_became_peered":"2023-10-08T00:11:41.068310+0000","last_unstale":"2023-10-08T00:11:41.069197+0000","last_undegraded":"2023-10-08T00:11:41.069197+0000","last_fullsized":"2023-10-08T00:11:41.069197+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:00:08.778342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"30'8","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.301851+0000","last_change":"2023-10-08T00:11:40.301528+0000","last_active":"2023-10-08T00:11:41.301851+0000","last_peered":"2023-10-08T00:11:41.301851+0000","last_clean":"2023-10-08T00:11:41.301851+0000","last_became_active":"2023-10-08T00:11:40.301268+0000","last_became_peered":"2023-10-08T00:11:40.301268+0000","last_unstale":"2023-10-08T00:11:41.301851+0000","last_undegraded":"2023-10-08T00:11:41.301851+0000","last_fullsized":"2023-10-08T00:11:41.301851+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:34:45.909544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.067138+0000","last_change":"2023-10-08T00:11:41.065696+0000","last_active":"2023-10-08T00:11:41.067138+0000","last_peered":"2023-10-08T00:11:41.067138+0000","last_clean":"2023-10-08T00:11:41.067138+0000","last_became_active":"2023-10-08T00:11:41.065493+0000","last_became_peered":"2023-10-08T00:11:41.065493+0000","last_unstale":"2023-10-08T00:11:41.067138+0000","last_undegraded":"2023-10-08T00:11:41.067138+0000","last_fullsized":"2023-10-08T00:11:41.067138+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:02:19.666640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"30'1","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.299124+0000","last_change":"2023-10-08T00:11:40.307608+0000","last_active":"2023-10-08T00:11:41.299124+0000","last_peered":"2023-10-08T00:11:41.299124+0000","last_clean":"2023-10-08T00:11:41.299124+0000","last_became_active":"2023-10-08T00:11:40.307366+0000","last_became_peered":"2023-10-08T00:11:40.307366+0000","last_unstale":"2023-10-08T00:11:41.299124+0000","last_undegraded":"2023-10-08T00:11:41.299124+0000","last_fullsized":"2023-10-08T00:11:41.299124+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:28:37.258489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.067908+0000","last_change":"2023-10-08T00:11:41.067849+0000","last_active":"2023-10-08T00:11:41.067908+0000","last_peered":"2023-10-08T00:11:41.067908+0000","last_clean":"2023-10-08T00:11:41.067908+0000","last_became_active":"2023-10-08T00:11:41.067612+0000","last_became_peered":"2023-10-08T00:11:41.067612+0000","last_unstale":"2023-10-08T00:11:41.067908+0000","last_undegraded":"2023-10-08T00:11:41.067908+0000","last_fullsized":"2023-10-08T00:11:41.067908+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:11:56.755889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"30'2","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.302108+0000","last_change":"2023-10-08T00:11:40.299858+0000","last_active":"2023-10-08T00:11:41.302108+0000","last_peered":"2023-10-08T00:11:41.302108+0000","last_clean":"2023-10-08T00:11:41.302108+0000","last_became_active":"2023-10-08T00:11:40.299250+0000","last_became_peered":"2023-10-08T00:11:40.299250+0000","last_unstale":"2023-10-08T00:11:41.302108+0000","last_undegraded":"2023-10-08T00:11:41.302108+0000","last_fullsized":"2023-10-08T00:11:41.302108+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:35:01.644676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.068434+0000","last_change":"2023-10-08T00:11:41.068324+0000","last_active":"2023-10-08T00:11:41.068434+0000","last_peered":"2023-10-08T00:11:41.068434+0000","last_clean":"2023-10-08T00:11:41.068434+0000","last_became_active":"2023-10-08T00:11:41.067954+0000","last_became_peered":"2023-10-08T00:11:41.067954+0000","last_unstale":"2023-10-08T00:11:41.068434+0000","last_undegraded":"2023-10-08T00:11:41.068434+0000","last_fullsized":"2023-10-08T00:11:41.068434+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:53:16.932728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"30'1","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.298766+0000","last_change":"2023-10-08T00:11:40.303265+0000","last_active":"2023-10-08T00:11:41.298766+0000","last_peered":"2023-10-08T00:11:41.298766+0000","last_clean":"2023-10-08T00:11:41.298766+0000","last_became_active":"2023-10-08T00:11:40.302748+0000","last_became_peered":"2023-10-08T00:11:40.302748+0000","last_unstale":"2023-10-08T00:11:41.298766+0000","last_undegraded":"2023-10-08T00:11:41.298766+0000","last_fullsized":"2023-10-08T00:11:41.298766+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:07:24.679697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.070186+0000","last_change":"2023-10-08T00:11:41.070079+0000","last_active":"2023-10-08T00:11:41.070186+0000","last_peered":"2023-10-08T00:11:41.070186+0000","last_clean":"2023-10-08T00:11:41.070186+0000","last_became_active":"2023-10-08T00:11:41.069697+0000","last_became_peered":"2023-10-08T00:11:41.069697+0000","last_unstale":"2023-10-08T00:11:41.070186+0000","last_undegraded":"2023-10-08T00:11:41.070186+0000","last_fullsized":"2023-10-08T00:11:41.070186+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:35:02.649508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.066906+0000","last_change":"2023-10-08T00:11:40.301506+0000","last_active":"2023-10-08T00:11:41.066906+0000","last_peered":"2023-10-08T00:11:41.066906+0000","last_clean":"2023-10-08T00:11:41.066906+0000","last_became_active":"2023-10-08T00:11:40.301264+0000","last_became_peered":"2023-10-08T00:11:40.301264+0000","last_unstale":"2023-10-08T00:11:41.066906+0000","last_undegraded":"2023-10-08T00:11:41.066906+0000","last_fullsized":"2023-10-08T00:11:41.066906+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:21:43.768378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.068864+0000","last_change":"2023-10-08T00:11:41.068481+0000","last_active":"2023-10-08T00:11:41.068864+0000","last_peered":"2023-10-08T00:11:41.068864+0000","last_clean":"2023-10-08T00:11:41.068864+0000","last_became_active":"2023-10-08T00:11:41.068243+0000","last_became_peered":"2023-10-08T00:11:41.068243+0000","last_unstale":"2023-10-08T00:11:41.068864+0000","last_undegraded":"2023-10-08T00:11:41.068864+0000","last_fullsized":"2023-10-08T00:11:41.068864+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:56:50.566568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"30'2","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.301583+0000","last_change":"2023-10-08T00:11:40.297307+0000","last_active":"2023-10-08T00:11:41.301583+0000","last_peered":"2023-10-08T00:11:41.301583+0000","last_clean":"2023-10-08T00:11:41.301583+0000","last_became_active":"2023-10-08T00:11:40.297047+0000","last_became_peered":"2023-10-08T00:11:40.297047+0000","last_unstale":"2023-10-08T00:11:41.301583+0000","last_undegraded":"2023-10-08T00:11:41.301583+0000","last_fullsized":"2023-10-08T00:11:41.301583+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:56:49.561736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.072594+0000","last_change":"2023-10-08T00:11:41.072520+0000","last_active":"2023-10-08T00:11:41.072594+0000","last_peered":"2023-10-08T00:11:41.072594+0000","last_clean":"2023-10-08T00:11:41.072594+0000","last_became_active":"2023-10-08T00:11:41.072257+0000","last_became_peered":"2023-10-08T00:11:41.072257+0000","last_unstale":"2023-10-08T00:11:41.072594+0000","last_undegraded":"2023-10-08T00:11:41.072594+0000","last_fullsized":"2023-10-08T00:11:41.072594+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:57:19.584921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.058539+0000","last_change":"2023-10-08T00:11:40.306139+0000","last_active":"2023-10-08T00:11:41.058539+0000","last_peered":"2023-10-08T00:11:41.058539+0000","last_clean":"2023-10-08T00:11:41.058539+0000","last_became_active":"2023-10-08T00:11:40.305874+0000","last_became_peered":"2023-10-08T00:11:40.305874+0000","last_unstale":"2023-10-08T00:11:41.058539+0000","last_undegraded":"2023-10-08T00:11:41.058539+0000","last_fullsized":"2023-10-08T00:11:41.058539+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:21:57.583692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.073315+0000","last_change":"2023-10-08T00:11:41.073262+0000","last_active":"2023-10-08T00:11:41.073315+0000","last_peered":"2023-10-08T00:11:41.073315+0000","last_clean":"2023-10-08T00:11:41.073315+0000","last_became_active":"2023-10-08T00:11:41.072951+0000","last_became_peered":"2023-10-08T00:11:41.072951+0000","last_unstale":"2023-10-08T00:11:41.073315+0000","last_undegraded":"2023-10-08T00:11:41.073315+0000","last_fullsized":"2023-10-08T00:11:41.073315+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:51:40.262177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.063297+0000","last_change":"2023-10-08T00:11:40.301402+0000","last_active":"2023-10-08T00:11:41.063297+0000","last_peered":"2023-10-08T00:11:41.063297+0000","last_clean":"2023-10-08T00:11:41.063297+0000","last_became_active":"2023-10-08T00:11:40.301183+0000","last_became_peered":"2023-10-08T00:11:40.301183+0000","last_unstale":"2023-10-08T00:11:41.063297+0000","last_undegraded":"2023-10-08T00:11:41.063297+0000","last_fullsized":"2023-10-08T00:11:41.063297+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:51:40.610344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.069558+0000","last_change":"2023-10-08T00:11:41.069312+0000","last_active":"2023-10-08T00:11:41.069558+0000","last_peered":"2023-10-08T00:11:41.069558+0000","last_clean":"2023-10-08T00:11:41.069558+0000","last_became_active":"2023-10-08T00:11:41.069078+0000","last_became_peered":"2023-10-08T00:11:41.069078+0000","last_unstale":"2023-10-08T00:11:41.069558+0000","last_undegraded":"2023-10-08T00:11:41.069558+0000","last_fullsized":"2023-10-08T00:11:41.069558+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:21:09.281936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.060804+0000","last_change":"2023-10-08T00:11:40.303399+0000","last_active":"2023-10-08T00:11:41.060804+0000","last_peered":"2023-10-08T00:11:41.060804+0000","last_clean":"2023-10-08T00:11:41.060804+0000","last_became_active":"2023-10-08T00:11:40.303051+0000","last_became_peered":"2023-10-08T00:11:40.303051+0000","last_unstale":"2023-10-08T00:11:41.060804+0000","last_undegraded":"2023-10-08T00:11:41.060804+0000","last_fullsized":"2023-10-08T00:11:41.060804+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:53:15.927897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.071521+0000","last_change":"2023-10-08T00:11:41.071206+0000","last_active":"2023-10-08T00:11:41.071521+0000","last_peered":"2023-10-08T00:11:41.071521+0000","last_clean":"2023-10-08T00:11:41.071521+0000","last_became_active":"2023-10-08T00:11:41.070680+0000","last_became_peered":"2023-10-08T00:11:41.070680+0000","last_unstale":"2023-10-08T00:11:41.071521+0000","last_undegraded":"2023-10-08T00:11:41.071521+0000","last_fullsized":"2023-10-08T00:11:41.071521+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:11:56.755889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"30'2","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.301386+0000","last_change":"2023-10-08T00:11:40.305058+0000","last_active":"2023-10-08T00:11:41.301386+0000","last_peered":"2023-10-08T00:11:41.301386+0000","last_clean":"2023-10-08T00:11:41.301386+0000","last_became_active":"2023-10-08T00:11:40.304528+0000","last_became_peered":"2023-10-08T00:11:40.304528+0000","last_unstale":"2023-10-08T00:11:41.301386+0000","last_undegraded":"2023-10-08T00:11:41.301386+0000","last_fullsized":"2023-10-08T00:11:41.301386+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:35:01.644676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.067156+0000","last_change":"2023-10-08T00:11:41.066345+0000","last_active":"2023-10-08T00:11:41.067156+0000","last_peered":"2023-10-08T00:11:41.067156+0000","last_clean":"2023-10-08T00:11:41.067156+0000","last_became_active":"2023-10-08T00:11:41.066159+0000","last_became_peered":"2023-10-08T00:11:41.066159+0000","last_unstale":"2023-10-08T00:11:41.067156+0000","last_undegraded":"2023-10-08T00:11:41.067156+0000","last_fullsized":"2023-10-08T00:11:41.067156+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:16:37.383597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.070040+0000","last_change":"2023-10-08T00:11:40.299694+0000","last_active":"2023-10-08T00:11:41.070040+0000","last_peered":"2023-10-08T00:11:41.070040+0000","last_clean":"2023-10-08T00:11:41.070040+0000","last_became_active":"2023-10-08T00:11:40.299462+0000","last_became_peered":"2023-10-08T00:11:40.299462+0000","last_unstale":"2023-10-08T00:11:41.070040+0000","last_undegraded":"2023-10-08T00:11:41.070040+0000","last_fullsized":"2023-10-08T00:11:41.070040+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:13:04.129642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.069801+0000","last_change":"2023-10-08T00:11:41.069734+0000","last_active":"2023-10-08T00:11:41.069801+0000","last_peered":"2023-10-08T00:11:41.069801+0000","last_clean":"2023-10-08T00:11:41.069801+0000","last_became_active":"2023-10-08T00:11:41.069509+0000","last_became_peered":"2023-10-08T00:11:41.069509+0000","last_unstale":"2023-10-08T00:11:41.069801+0000","last_undegraded":"2023-10-08T00:11:41.069801+0000","last_fullsized":"2023-10-08T00:11:41.069801+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:23:13.406338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.065608+0000","last_change":"2023-10-08T00:11:40.302152+0000","last_active":"2023-10-08T00:11:41.065608+0000","last_peered":"2023-10-08T00:11:41.065608+0000","last_clean":"2023-10-08T00:11:41.065608+0000","last_became_active":"2023-10-08T00:11:40.301896+0000","last_became_peered":"2023-10-08T00:11:40.301896+0000","last_unstale":"2023-10-08T00:11:41.065608+0000","last_undegraded":"2023-10-08T00:11:41.065608+0000","last_fullsized":"2023-10-08T00:11:41.065608+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:13:28.760481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.068824+0000","last_change":"2023-10-08T00:11:41.068732+0000","last_active":"2023-10-08T00:11:41.068824+0000","last_peered":"2023-10-08T00:11:41.068824+0000","last_clean":"2023-10-08T00:11:41.068824+0000","last_became_active":"2023-10-08T00:11:41.068509+0000","last_became_peered":"2023-10-08T00:11:41.068509+0000","last_unstale":"2023-10-08T00:11:41.068824+0000","last_undegraded":"2023-10-08T00:11:41.068824+0000","last_fullsized":"2023-10-08T00:11:41.068824+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:57:19.584921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.064989+0000","last_change":"2023-10-08T00:11:40.301226+0000","last_active":"2023-10-08T00:11:41.064989+0000","last_peered":"2023-10-08T00:11:41.064989+0000","last_clean":"2023-10-08T00:11:41.064989+0000","last_became_active":"2023-10-08T00:11:40.300970+0000","last_became_peered":"2023-10-08T00:11:40.300970+0000","last_unstale":"2023-10-08T00:11:41.064989+0000","last_undegraded":"2023-10-08T00:11:41.064989+0000","last_fullsized":"2023-10-08T00:11:41.064989+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:20:01.582031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.074928+0000","last_change":"2023-10-08T00:11:41.074877+0000","last_active":"2023-10-08T00:11:41.074928+0000","last_peered":"2023-10-08T00:11:41.074928+0000","last_clean":"2023-10-08T00:11:41.074928+0000","last_became_active":"2023-10-08T00:11:41.074388+0000","last_became_peered":"2023-10-08T00:11:41.074388+0000","last_unstale":"2023-10-08T00:11:41.074928+0000","last_undegraded":"2023-10-08T00:11:41.074928+0000","last_fullsized":"2023-10-08T00:11:41.074928+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:32:58.999045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.064119+0000","last_change":"2023-10-08T00:11:40.305228+0000","last_active":"2023-10-08T00:11:41.064119+0000","last_peered":"2023-10-08T00:11:41.064119+0000","last_clean":"2023-10-08T00:11:41.064119+0000","last_became_active":"2023-10-08T00:11:40.304830+0000","last_became_peered":"2023-10-08T00:11:40.304830+0000","last_unstale":"2023-10-08T00:11:41.064119+0000","last_undegraded":"2023-10-08T00:11:41.064119+0000","last_fullsized":"2023-10-08T00:11:41.064119+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:21:08.277105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.071453+0000","last_change":"2023-10-08T00:11:41.071382+0000","last_active":"2023-10-08T00:11:41.071453+0000","last_peered":"2023-10-08T00:11:41.071453+0000","last_clean":"2023-10-08T00:11:41.071453+0000","last_became_active":"2023-10-08T00:11:41.070341+0000","last_became_peered":"2023-10-08T00:11:41.070341+0000","last_unstale":"2023-10-08T00:11:41.071453+0000","last_undegraded":"2023-10-08T00:11:41.071453+0000","last_fullsized":"2023-10-08T00:11:41.071453+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:16:52.644456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.059251+0000","last_change":"2023-10-08T00:11:40.299183+0000","last_active":"2023-10-08T00:11:41.059251+0000","last_peered":"2023-10-08T00:11:41.059251+0000","last_clean":"2023-10-08T00:11:41.059251+0000","last_became_active":"2023-10-08T00:11:40.298749+0000","last_became_peered":"2023-10-08T00:11:40.298749+0000","last_unstale":"2023-10-08T00:11:41.059251+0000","last_undegraded":"2023-10-08T00:11:41.059251+0000","last_fullsized":"2023-10-08T00:11:41.059251+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:41:40.921278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.067055+0000","last_change":"2023-10-08T00:11:41.063259+0000","last_active":"2023-10-08T00:11:41.067055+0000","last_peered":"2023-10-08T00:11:41.067055+0000","last_clean":"2023-10-08T00:11:41.067055+0000","last_became_active":"2023-10-08T00:11:41.063024+0000","last_became_peered":"2023-10-08T00:11:41.063024+0000","last_unstale":"2023-10-08T00:11:41.067055+0000","last_undegraded":"2023-10-08T00:11:41.067055+0000","last_fullsized":"2023-10-08T00:11:41.067055+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:52:15.180944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.065759+0000","last_change":"2023-10-08T00:11:40.307268+0000","last_active":"2023-10-08T00:11:41.065759+0000","last_peered":"2023-10-08T00:11:41.065759+0000","last_clean":"2023-10-08T00:11:41.065759+0000","last_became_active":"2023-10-08T00:11:40.306560+0000","last_became_peered":"2023-10-08T00:11:40.306560+0000","last_unstale":"2023-10-08T00:11:41.065759+0000","last_undegraded":"2023-10-08T00:11:41.065759+0000","last_fullsized":"2023-10-08T00:11:41.065759+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:20:01.582031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.069595+0000","last_change":"2023-10-08T00:11:41.069539+0000","last_active":"2023-10-08T00:11:41.069595+0000","last_peered":"2023-10-08T00:11:41.069595+0000","last_clean":"2023-10-08T00:11:41.069595+0000","last_became_active":"2023-10-08T00:11:41.068352+0000","last_became_peered":"2023-10-08T00:11:41.068352+0000","last_unstale":"2023-10-08T00:11:41.069595+0000","last_undegraded":"2023-10-08T00:11:41.069595+0000","last_fullsized":"2023-10-08T00:11:41.069595+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:40.287493+0000","last_clean_scrub_stamp":"2023-10-08T00:11:40.287493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:07:25.684529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.060277+0000","last_change":"2023-10-08T00:11:40.302566+0000","last_active":"2023-10-08T00:11:41.060277+0000","last_peered":"2023-10-08T00:11:41.060277+0000","last_clean":"2023-10-08T00:11:41.060277+0000","last_became_active":"2023-10-08T00:11:40.302357+0000","last_became_peered":"2023-10-08T00:11:40.302357+0000","last_unstale":"2023-10-08T00:11:41.060277+0000","last_undegraded":"2023-10-08T00:11:41.060277+0000","last_fullsized":"2023-10-08T00:11:41.060277+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:11:39.282662+0000","last_clean_scrub_stamp":"2023-10-08T00:11:39.282662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:57:18.580090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":171,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:11:41.064491+0000","last_change":"2023-10-08T00:10:43.728490+0000","last_active":"2023-10-08T00:11:41.064491+0000","last_peered":"2023-10-08T00:11:41.064491+0000","last_clean":"2023-10-08T00:11:41.064491+0000","last_became_active":"2023-10-08T00:10:43.728341+0000","last_became_peered":"2023-10-08T00:10:43.728341+0000","last_unstale":"2023-10-08T00:11:41.064491+0000","last_undegraded":"2023-10-08T00:11:41.064491+0000","last_fullsized":"2023-10-08T00:11:41.064491+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_clean_scrub_stamp":"2023-10-08T00:10:42.716096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:06:31.035293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574862,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":124,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195188,"statfs":{"total":200000000,"available":199872704,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.65100000000000002}]},{"osd":1,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.746}]},{"osd":2,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.53000000000000003}]},{"osd":3,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.63200000000000001}]},{"osd":4,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.78900000000000003}]},{"osd":5,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.72999999999999998}]},{"osd":6,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.81699999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574862,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":131,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195180,"statfs":{"total":200000000,"available":199864902,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]}]},{"osd":1,"up_from":13,"seq":55834574862,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":707,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194605,"statfs":{"total":200000000,"available":199276000,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.19500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574862,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":130,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195181,"statfs":{"total":200000000,"available":199866348,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.20300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574862,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":130,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195182,"statfs":{"total":200000000,"available":199866404,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]}]},{"osd":3,"up_from":13,"seq":55834574862,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":131,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195181,"statfs":{"total":200000000,"available":199865690,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.255}]},{"osd":1,"last update":"Sun Oct 8 00:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":12.351000000000001,"5min":12.351000000000001,"15min":12.351000000000001},"last":0.309},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.22600000000000001}]},{"osd":2,"last update":"Sun Oct 8 00:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.28100000000000003}]},{"osd":4,"last update":"Sun Oct 8 00:11:41 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":11.333,"5min":11.333,"15min":11.333},"last":0.54200000000000004},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":12.321999999999999,"5min":12.321999999999999,"15min":12.321999999999999},"last":0.55500000000000005}]},{"osd":5,"last update":"Sun Oct 8 00:11:41 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.181,"15min":1.181},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":12.291,"5min":12.291,"15min":12.291},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.52100000000000002}]},{"osd":6,"last update":"Sun Oct 8 00:11:41 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":11.288,"5min":11.288,"15min":11.288},"last":0.45000000000000001},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":12.253,"5min":12.253,"15min":12.253},"last":0.495}]},{"osd":7,"last update":"Sun Oct 8 00:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.44},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":11.363,"5min":11.363,"15min":11.363},"last":0.505}]}]},{"osd":4,"up_from":13,"seq":55834574862,"num_pgs":35,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":125,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195186,"statfs":{"total":200000000,"available":199871168,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.63400000000000001}]},{"osd":1,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.59099999999999997}]},{"osd":2,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.56399999999999995}]},{"osd":3,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.40400000000000003}]},{"osd":5,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.54500000000000004}]},{"osd":6,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.67100000000000004}]},{"osd":7,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.60899999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574862,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":707,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194604,"statfs":{"total":200000000,"available":199275516,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.67700000000000005}]},{"osd":1,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.61899999999999999}]},{"osd":2,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.435},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.57699999999999996}]},{"osd":3,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.64700000000000002}]},{"osd":4,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.84499999999999997}]},{"osd":6,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.745},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.70599999999999996}]},{"osd":7,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.82399999999999995}]}]}],"pool_statfs":[]}} 2023-10-08T00:11:45.830 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T00:11:45.830 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-08T00:11:45.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.952+0000 7f9a9efcd700 1 Processor -- start 2023-10-08T00:11:45.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.952+0000 7f9a9efcd700 1 -- start start 2023-10-08T00:11:45.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.953+0000 7f9a9efcd700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f9a981542a0 0x7f9a980b65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:45.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.953+0000 7f9a9efcd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9a980cd630 0x7f9a9806a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:45.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.953+0000 7f9a9efcd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a9806a830 0x7f9a9815d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:45.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.953+0000 7f9a9efcd700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a9815b2e0 con 0x7f9a980cd630 2023-10-08T00:11:45.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.953+0000 7f9a9efcd700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a9806b0a0 con 0x7f9a981542a0 2023-10-08T00:11:45.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.953+0000 7f9a9efcd700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9806ac30 con 0x7f9a9806a830 2023-10-08T00:11:45.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.953+0000 7f9a9d7ca700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9a980cd630 0x7f9a9806a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:45.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.953+0000 7f9a9d7ca700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9a980cd630 0x7f9a9806a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50590/0 (socket says 172.21.15.99:50590) 2023-10-08T00:11:45.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.953+0000 7f9a9d7ca700 1 -- 172.21.15.99:0/2308874307 learned_addr learned my addr 172.21.15.99:0/2308874307 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:45.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.953+0000 7f9a9e7cc700 1 --2- 172.21.15.99:0/2308874307 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a9806a830 0x7f9a9815d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:45.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.953+0000 7f9a9dfcb700 1 --2- 172.21.15.99:0/2308874307 >> v2:172.21.15.191:3300/0 conn(0x7f9a981542a0 0x7f9a980b65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:45.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.953+0000 7f9a9d7ca700 1 -- 172.21.15.99:0/2308874307 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a9806a830 msgr2=0x7f9a9815d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:45.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.953+0000 7f9a9d7ca700 1 --2- 172.21.15.99:0/2308874307 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a9806a830 0x7f9a9815d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:45.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.953+0000 7f9a9d7ca700 1 -- 172.21.15.99:0/2308874307 >> v2:172.21.15.191:3300/0 conn(0x7f9a981542a0 msgr2=0x7f9a980b65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:45.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.953+0000 7f9a9d7ca700 1 --2- 172.21.15.99:0/2308874307 >> v2:172.21.15.191:3300/0 conn(0x7f9a981542a0 0x7f9a980b65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:45.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.953+0000 7f9a9d7ca700 1 -- 172.21.15.99:0/2308874307 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a9815ad20 con 0x7f9a980cd630 2023-10-08T00:11:45.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.953+0000 7f9a9dfcb700 1 --2- 172.21.15.99:0/2308874307 >> v2:172.21.15.191:3300/0 conn(0x7f9a981542a0 0x7f9a980b65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:11:45.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.954+0000 7f9a9d7ca700 1 --2- 172.21.15.99:0/2308874307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a980cd630 0x7f9a9806a2f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9a980d1c50 tx=0x7f9a88009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b27e1c2c2107610 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:45.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.954+0000 7f9a9cfc9700 1 -- 172.21.15.99:0/2308874307 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8800b860 con 0x7f9a980cd630 2023-10-08T00:11:45.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.954+0000 7f9a9cfc9700 1 -- 172.21.15.99:0/2308874307 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a8800b9c0 con 0x7f9a980cd630 2023-10-08T00:11:45.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.954+0000 7f9a9cfc9700 1 -- 172.21.15.99:0/2308874307 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8800bd60 con 0x7f9a980cd630 2023-10-08T00:11:45.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.954+0000 7f9a9efcd700 1 -- 172.21.15.99:0/2308874307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a980cd630 msgr2=0x7f9a9806a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:45.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.954+0000 7f9a9efcd700 1 --2- 172.21.15.99:0/2308874307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a980cd630 0x7f9a9806a2f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9a980d1c50 tx=0x7f9a88009250 comp rx=0 tx=0).stop 2023-10-08T00:11:45.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.954+0000 7f9a9efcd700 1 -- 172.21.15.99:0/2308874307 shutdown_connections 2023-10-08T00:11:45.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.954+0000 7f9a9efcd700 1 --2- 172.21.15.99:0/2308874307 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a9806a830 0x7f9a9815d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:45.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.954+0000 7f9a9efcd700 1 --2- 172.21.15.99:0/2308874307 >> v2:172.21.15.191:3300/0 conn(0x7f9a981542a0 0x7f9a980b65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:45.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.954+0000 7f9a9efcd700 1 --2- 172.21.15.99:0/2308874307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a980cd630 0x7f9a9806a2f0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:45.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.954+0000 7f9a9efcd700 1 -- 172.21.15.99:0/2308874307 >> 172.21.15.99:0/2308874307 conn(0x7f9a98068cb0 msgr2=0x7f9a980690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:45.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.954+0000 7f9a9efcd700 1 -- 172.21.15.99:0/2308874307 shutdown_connections 2023-10-08T00:11:45.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.954+0000 7f9a9efcd700 1 -- 172.21.15.99:0/2308874307 wait complete. 2023-10-08T00:11:45.961 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.955+0000 7f9a9efcd700 1 Processor -- start 2023-10-08T00:11:45.961 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.955+0000 7f9a9efcd700 1 -- start start 2023-10-08T00:11:45.961 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.956+0000 7f9a9efcd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a9806a830 0x7f9a9816dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:45.962 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.956+0000 7f9a9efcd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a980cd630 0x7f9a980c6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:45.962 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.956+0000 7f9a9efcd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a981542a0 0x7f9a98152400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:45.962 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.956+0000 7f9a9efcd700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a98157a40 con 0x7f9a9806a830 2023-10-08T00:11:45.962 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.956+0000 7f9a9efcd700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9816e2e0 con 0x7f9a980cd630 2023-10-08T00:11:45.962 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.956+0000 7f9a9efcd700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a98157c90 con 0x7f9a981542a0 2023-10-08T00:11:45.962 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.956+0000 7f9a9dfcb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a9806a830 0x7f9a9816dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:45.962 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.956+0000 7f9a9dfcb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a9806a830 0x7f9a9816dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50598/0 (socket says 172.21.15.99:50598) 2023-10-08T00:11:45.962 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.956+0000 7f9a9dfcb700 1 -- 172.21.15.99:0/1599132964 learned_addr learned my addr 172.21.15.99:0/1599132964 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:45.962 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.956+0000 7f9a9d7ca700 1 --2- 172.21.15.99:0/1599132964 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a980cd630 0x7f9a980c6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:45.962 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.956+0000 7f9a9e7cc700 1 --2- 172.21.15.99:0/1599132964 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a981542a0 0x7f9a98152400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:45.962 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.957+0000 7f9a9dfcb700 1 -- 172.21.15.99:0/1599132964 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a981542a0 msgr2=0x7f9a98152400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:45.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.957+0000 7f9a9dfcb700 1 --2- 172.21.15.99:0/1599132964 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a981542a0 0x7f9a98152400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:45.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.957+0000 7f9a9dfcb700 1 -- 172.21.15.99:0/1599132964 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a980cd630 msgr2=0x7f9a980c6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:45.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.957+0000 7f9a9dfcb700 1 --2- 172.21.15.99:0/1599132964 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a980cd630 0x7f9a980c6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:45.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.957+0000 7f9a9dfcb700 1 -- 172.21.15.99:0/1599132964 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a880094f0 con 0x7f9a9806a830 2023-10-08T00:11:45.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.957+0000 7f9a9dfcb700 1 --2- 172.21.15.99:0/1599132964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a9806a830 0x7f9a9816dda0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9a9400a300 tx=0x7f9a940078c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:45.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.957+0000 7f9a8dffb700 1 -- 172.21.15.99:0/1599132964 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9400ae80 con 0x7f9a9806a830 2023-10-08T00:11:45.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.957+0000 7f9a8dffb700 1 -- 172.21.15.99:0/1599132964 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a94015440 con 0x7f9a9806a830 2023-10-08T00:11:45.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.957+0000 7f9a9efcd700 1 -- 172.21.15.99:0/1599132964 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a98157f70 con 0x7f9a9806a830 2023-10-08T00:11:45.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.957+0000 7f9a8dffb700 1 -- 172.21.15.99:0/1599132964 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a940157c0 con 0x7f9a9806a830 2023-10-08T00:11:45.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.957+0000 7f9a9efcd700 1 -- 172.21.15.99:0/1599132964 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a98158300 con 0x7f9a9806a830 2023-10-08T00:11:45.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.958+0000 7f9a9efcd700 1 -- 172.21.15.99:0/1599132964 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a98089550 con 0x7f9a9806a830 2023-10-08T00:11:45.964 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.958+0000 7f9a8dffb700 1 -- 172.21.15.99:0/1599132964 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9a94015b70 con 0x7f9a9806a830 2023-10-08T00:11:45.964 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.959+0000 7f9a8dffb700 1 --2- 172.21.15.99:0/1599132964 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9a7c066f30 0x7f9a7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:45.964 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.959+0000 7f9a8dffb700 1 -- 172.21.15.99:0/1599132964 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9a94084aa0 con 0x7f9a9806a830 2023-10-08T00:11:45.964 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.959+0000 7f9a9d7ca700 1 --2- 172.21.15.99:0/1599132964 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9a7c066f30 0x7f9a7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:45.964 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.960+0000 7f9a9d7ca700 1 --2- 172.21.15.99:0/1599132964 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9a7c066f30 0x7f9a7c0693e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9a980d1c50 tx=0x7f9a88017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:45.964 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:45.963+0000 7f9a8dffb700 1 -- 172.21.15.99:0/1599132964 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9a9404e5a0 con 0x7f9a9806a830 2023-10-08T00:11:46.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.222+0000 7f9a9efcd700 1 -- 172.21.15.99:0/1599132964 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f9a9815d610 con 0x7f9a9806a830 2023-10-08T00:11:46.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.226+0000 7f9a8dffb700 1 -- 172.21.15.99:0/1599132964 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+0 (secure 0 0 0) 0x7f9a9401a020 con 0x7f9a9806a830 2023-10-08T00:11:46.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.228+0000 7f9a9efcd700 1 -- 172.21.15.99:0/1599132964 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9a7c066f30 msgr2=0x7f9a7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:46.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.228+0000 7f9a9efcd700 1 --2- 172.21.15.99:0/1599132964 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9a7c066f30 0x7f9a7c0693e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9a980d1c50 tx=0x7f9a88017000 comp rx=0 tx=0).stop 2023-10-08T00:11:46.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.228+0000 7f9a9efcd700 1 -- 172.21.15.99:0/1599132964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a9806a830 msgr2=0x7f9a9816dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:46.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.228+0000 7f9a9efcd700 1 --2- 172.21.15.99:0/1599132964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a9806a830 0x7f9a9816dda0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9a9400a300 tx=0x7f9a940078c0 comp rx=0 tx=0).stop 2023-10-08T00:11:46.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.228+0000 7f9a9efcd700 1 -- 172.21.15.99:0/1599132964 shutdown_connections 2023-10-08T00:11:46.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.228+0000 7f9a9efcd700 1 --2- 172.21.15.99:0/1599132964 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9a7c066f30 0x7f9a7c0693e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:46.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.228+0000 7f9a9efcd700 1 --2- 172.21.15.99:0/1599132964 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a981542a0 0x7f9a98152400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:46.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.228+0000 7f9a9efcd700 1 --2- 172.21.15.99:0/1599132964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a9806a830 0x7f9a9816dda0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:46.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.228+0000 7f9a9efcd700 1 --2- 172.21.15.99:0/1599132964 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a980cd630 0x7f9a980c6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:46.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.228+0000 7f9a9efcd700 1 -- 172.21.15.99:0/1599132964 >> 172.21.15.99:0/1599132964 conn(0x7f9a98068cb0 msgr2=0x7f9a98153860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:46.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.228+0000 7f9a9efcd700 1 -- 172.21.15.99:0/1599132964 shutdown_connections 2023-10-08T00:11:46.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.228+0000 7f9a9efcd700 1 -- 172.21.15.99:0/1599132964 wait complete. 2023-10-08T00:11:46.230 INFO:teuthology.orchestra.run.smithi099.stderr:updated caps for client.0 2023-10-08T00:11:46.240 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:11:46.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.355+0000 7fb546758700 1 Processor -- start 2023-10-08T00:11:46.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.355+0000 7fb546758700 1 -- start start 2023-10-08T00:11:46.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.356+0000 7fb546758700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb540152020 0x7fb5400ca760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:46.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.356+0000 7fb546758700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb5401523f0 0x7fb54006c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:46.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.356+0000 7fb546758700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb54015b7f0 0x7fb54015bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:46.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.356+0000 7fb546758700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb54015c190 con 0x7fb54015b7f0 2023-10-08T00:11:46.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.356+0000 7fb546758700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5400cbb10 con 0x7fb540152020 2023-10-08T00:11:46.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.356+0000 7fb546758700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5400cbfa0 con 0x7fb5401523f0 2023-10-08T00:11:46.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.356+0000 7fb544cf5700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb54015b7f0 0x7fb54015bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:46.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.356+0000 7fb544cf5700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb54015b7f0 0x7fb54015bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50612/0 (socket says 172.21.15.99:50612) 2023-10-08T00:11:46.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.356+0000 7fb544cf5700 1 -- 172.21.15.99:0/3084279468 learned_addr learned my addr 172.21.15.99:0/3084279468 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:46.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.357+0000 7fb53f7fe700 1 --2- 172.21.15.99:0/3084279468 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb5401523f0 0x7fb54006c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:46.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.357+0000 7fb53ffff700 1 --2- 172.21.15.99:0/3084279468 >> v2:172.21.15.191:3300/0 conn(0x7fb540152020 0x7fb5400ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:46.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.357+0000 7fb544cf5700 1 -- 172.21.15.99:0/3084279468 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb5401523f0 msgr2=0x7fb54006c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:46.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.357+0000 7fb544cf5700 1 --2- 172.21.15.99:0/3084279468 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb5401523f0 0x7fb54006c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:46.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.357+0000 7fb544cf5700 1 -- 172.21.15.99:0/3084279468 >> v2:172.21.15.191:3300/0 conn(0x7fb540152020 msgr2=0x7fb5400ca760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:46.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.357+0000 7fb544cf5700 1 --2- 172.21.15.99:0/3084279468 >> v2:172.21.15.191:3300/0 conn(0x7fb540152020 0x7fb5400ca760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:46.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.357+0000 7fb544cf5700 1 -- 172.21.15.99:0/3084279468 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5400caca0 con 0x7fb54015b7f0 2023-10-08T00:11:46.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.357+0000 7fb544cf5700 1 --2- 172.21.15.99:0/3084279468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb54015b7f0 0x7fb54015bc50 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb5400cfc10 tx=0x7fb538009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fa1412728f46e04 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:46.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.357+0000 7fb53effd700 1 -- 172.21.15.99:0/3084279468 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53800e070 con 0x7fb54015b7f0 2023-10-08T00:11:46.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.357+0000 7fb53effd700 1 -- 172.21.15.99:0/3084279468 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb53800ba50 con 0x7fb54015b7f0 2023-10-08T00:11:46.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.357+0000 7fb53effd700 1 -- 172.21.15.99:0/3084279468 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5380092e0 con 0x7fb54015b7f0 2023-10-08T00:11:46.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.358+0000 7fb546758700 1 -- 172.21.15.99:0/3084279468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb54015b7f0 msgr2=0x7fb54015bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:46.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.358+0000 7fb546758700 1 --2- 172.21.15.99:0/3084279468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb54015b7f0 0x7fb54015bc50 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb5400cfc10 tx=0x7fb538009840 comp rx=0 tx=0).stop 2023-10-08T00:11:46.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.358+0000 7fb546758700 1 -- 172.21.15.99:0/3084279468 shutdown_connections 2023-10-08T00:11:46.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.358+0000 7fb546758700 1 --2- 172.21.15.99:0/3084279468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb54015b7f0 0x7fb54015bc50 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:46.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.358+0000 7fb546758700 1 --2- 172.21.15.99:0/3084279468 >> v2:172.21.15.191:3300/0 conn(0x7fb540152020 0x7fb5400ca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:46.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.358+0000 7fb546758700 1 --2- 172.21.15.99:0/3084279468 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb5401523f0 0x7fb54006c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:46.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.358+0000 7fb546758700 1 -- 172.21.15.99:0/3084279468 >> 172.21.15.99:0/3084279468 conn(0x7fb540069870 msgr2=0x7fb540069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:46.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.358+0000 7fb546758700 1 -- 172.21.15.99:0/3084279468 shutdown_connections 2023-10-08T00:11:46.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.358+0000 7fb546758700 1 -- 172.21.15.99:0/3084279468 wait complete. 2023-10-08T00:11:46.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.359+0000 7fb546758700 1 Processor -- start 2023-10-08T00:11:46.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.359+0000 7fb546758700 1 -- start start 2023-10-08T00:11:46.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.360+0000 7fb546758700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb540152020 0x7fb54014a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:46.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.360+0000 7fb546758700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5401523f0 0x7fb540069040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:46.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.360+0000 7fb546758700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb54015b7f0 0x7fb54014b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:46.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.360+0000 7fb546758700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb540068d70 con 0x7fb540152020 2023-10-08T00:11:46.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.360+0000 7fb546758700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb540067de0 con 0x7fb5401523f0 2023-10-08T00:11:46.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.360+0000 7fb546758700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb54014ad00 con 0x7fb54015b7f0 2023-10-08T00:11:46.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.360+0000 7fb53ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb540152020 0x7fb54014a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:46.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.360+0000 7fb53ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb540152020 0x7fb54014a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50622/0 (socket says 172.21.15.99:50622) 2023-10-08T00:11:46.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.360+0000 7fb53ffff700 1 -- 172.21.15.99:0/3964270127 learned_addr learned my addr 172.21.15.99:0/3964270127 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:46.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.360+0000 7fb544cf5700 1 --2- 172.21.15.99:0/3964270127 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb54015b7f0 0x7fb54014b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:46.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.360+0000 7fb53f7fe700 1 --2- 172.21.15.99:0/3964270127 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5401523f0 0x7fb540069040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:46.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.360+0000 7fb53ffff700 1 -- 172.21.15.99:0/3964270127 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb54015b7f0 msgr2=0x7fb54014b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:46.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.360+0000 7fb53ffff700 1 --2- 172.21.15.99:0/3964270127 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb54015b7f0 0x7fb54014b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:46.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.360+0000 7fb53ffff700 1 -- 172.21.15.99:0/3964270127 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5401523f0 msgr2=0x7fb540069040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:46.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.360+0000 7fb53ffff700 1 --2- 172.21.15.99:0/3964270127 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5401523f0 0x7fb540069040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:46.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.360+0000 7fb53ffff700 1 -- 172.21.15.99:0/3964270127 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb538009ae0 con 0x7fb540152020 2023-10-08T00:11:46.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.360+0000 7fb53ffff700 1 --2- 172.21.15.99:0/3964270127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb540152020 0x7fb54014a1c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb53000d860 tx=0x7fb530008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:46.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.360+0000 7fb527fff700 1 -- 172.21.15.99:0/3964270127 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5300125f0 con 0x7fb540152020 2023-10-08T00:11:46.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.360+0000 7fb527fff700 1 -- 172.21.15.99:0/3964270127 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb530012750 con 0x7fb540152020 2023-10-08T00:11:46.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.360+0000 7fb546758700 1 -- 172.21.15.99:0/3964270127 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb540068a90 con 0x7fb540152020 2023-10-08T00:11:46.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.360+0000 7fb527fff700 1 -- 172.21.15.99:0/3964270127 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb530012b10 con 0x7fb540152020 2023-10-08T00:11:46.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.360+0000 7fb546758700 1 -- 172.21.15.99:0/3964270127 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb540067a70 con 0x7fb540152020 2023-10-08T00:11:46.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.361+0000 7fb546758700 1 -- 172.21.15.99:0/3964270127 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5400680c0 con 0x7fb540152020 2023-10-08T00:11:46.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.362+0000 7fb527fff700 1 -- 172.21.15.99:0/3964270127 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb53001e460 con 0x7fb540152020 2023-10-08T00:11:46.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.362+0000 7fb527fff700 1 --2- 172.21.15.99:0/3964270127 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb52c066f30 0x7fb52c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:46.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.362+0000 7fb527fff700 1 -- 172.21.15.99:0/3964270127 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb530084a60 con 0x7fb540152020 2023-10-08T00:11:46.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.362+0000 7fb53f7fe700 1 --2- 172.21.15.99:0/3964270127 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb52c066f30 0x7fb52c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:46.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.363+0000 7fb53f7fe700 1 --2- 172.21.15.99:0/3964270127 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb52c066f30 0x7fb52c0693e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb534006fd0 tx=0x7fb534008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:46.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.366+0000 7fb527fff700 1 -- 172.21.15.99:0/3964270127 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb530051cd0 con 0x7fb540152020 2023-10-08T00:11:46.590 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.589+0000 7fb546758700 1 -- 172.21.15.99:0/3964270127 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb54004c3c0 con 0x7fb540152020 2023-10-08T00:11:46.590 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.590+0000 7fb527fff700 1 -- 172.21.15.99:0/3964270127 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4600 (secure 0 0 0) 0x7fb5300552f0 con 0x7fb540152020 2023-10-08T00:11:46.590 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:11:46.591 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4852,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/824084995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":824084995},{"type":"v1","addr":"172.21.15.191:6837","nonce":824084995}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4855,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/3349126920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3349126920},{"type":"v1","addr":"172.21.15.191:6836","nonce":3349126920}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4899,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/1004180438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1004180438},{"type":"v1","addr":"172.21.15.99:6834","nonce":1004180438}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:11:40.672879+0000","modified":"2023-10-08T00:11:44.314526+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4902},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4902":{"gid":4902,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/4189095717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":4189095717},{"type":"v1","addr":"172.21.15.99:6835","nonce":4189095717}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-08T00:11:46.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.592+0000 7fb546758700 1 -- 172.21.15.99:0/3964270127 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb52c066f30 msgr2=0x7fb52c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:46.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.592+0000 7fb546758700 1 --2- 172.21.15.99:0/3964270127 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb52c066f30 0x7fb52c0693e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb534006fd0 tx=0x7fb534008040 comp rx=0 tx=0).stop 2023-10-08T00:11:46.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.592+0000 7fb546758700 1 -- 172.21.15.99:0/3964270127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb540152020 msgr2=0x7fb54014a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:46.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.592+0000 7fb546758700 1 --2- 172.21.15.99:0/3964270127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb540152020 0x7fb54014a1c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb53000d860 tx=0x7fb530008040 comp rx=0 tx=0).stop 2023-10-08T00:11:46.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.592+0000 7fb546758700 1 -- 172.21.15.99:0/3964270127 shutdown_connections 2023-10-08T00:11:46.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.592+0000 7fb546758700 1 --2- 172.21.15.99:0/3964270127 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb52c066f30 0x7fb52c0693e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:46.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.592+0000 7fb546758700 1 --2- 172.21.15.99:0/3964270127 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb54015b7f0 0x7fb54014b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:46.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.592+0000 7fb546758700 1 --2- 172.21.15.99:0/3964270127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb540152020 0x7fb54014a1c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:46.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.592+0000 7fb546758700 1 --2- 172.21.15.99:0/3964270127 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5401523f0 0x7fb540069040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:46.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.592+0000 7fb546758700 1 -- 172.21.15.99:0/3964270127 >> 172.21.15.99:0/3964270127 conn(0x7fb540069870 msgr2=0x7fb540151050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:46.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.592+0000 7fb546758700 1 -- 172.21.15.99:0/3964270127 shutdown_connections 2023-10-08T00:11:46.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:46.592+0000 7fb546758700 1 -- 172.21.15.99:0/3964270127 wait complete. 2023-10-08T00:11:46.594 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 17 2023-10-08T00:11:47.006 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T00:11:40.672879+0000', 'modified': '2023-10-08T00:11:44.314526+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4902}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4902': {'gid': 4902, 'name': 'c', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.99:6835/4189095717', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6833', 'nonce': 4189095717}, {'type': 'v1', 'addr': '172.21.15.99:6835', 'nonce': 4189095717}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-08T00:11:47.007 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T00:11:47.007 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-08T00:11:47.122 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.122+0000 7fdf753af700 1 Processor -- start 2023-10-08T00:11:47.123 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.122+0000 7fdf753af700 1 -- start start 2023-10-08T00:11:47.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.123+0000 7fdf753af700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdf70159da0 0x7fdf700b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:47.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.123+0000 7fdf753af700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf7015ea70 con 0x7fdf7015a170 2023-10-08T00:11:47.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.123+0000 7fdf753af700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf700b3990 con 0x7fdf70069210 2023-10-08T00:11:47.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.123+0000 7fdf753af700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf700b0b70 con 0x7fdf70159da0 2023-10-08T00:11:47.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.123+0000 7fdf6ffff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdf70159da0 0x7fdf700b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:47.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.123+0000 7fdf6f7fe700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fdf7015a170 0x7fdf70068b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:54218/0 (socket says 172.21.15.99:54218) 2023-10-08T00:11:47.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.123+0000 7fdf6f7fe700 1 -- 172.21.15.99:0/2638611231 learned_addr learned my addr 172.21.15.99:0/2638611231 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:47.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.123+0000 7fdf6ffff700 1 -- 172.21.15.99:0/2638611231 >> v1:172.21.15.191:6789/0 conn(0x7fdf70069210 legacy=0x7fdf700af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:47.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.123+0000 7fdf6ffff700 1 -- 172.21.15.99:0/2638611231 >> v1:172.21.15.99:6789/0 conn(0x7fdf7015a170 legacy=0x7fdf70068b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:47.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.123+0000 7fdf6ffff700 1 -- 172.21.15.99:0/2638611231 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf7015def0 con 0x7fdf70159da0 2023-10-08T00:11:47.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.123+0000 7fdf6effd700 1 -- 172.21.15.99:0/2638611231 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2505237262 0 0) 0x7fdf7015ea70 con 0x7fdf7015a170 2023-10-08T00:11:47.126 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.123+0000 7fdf6ffff700 1 --2- 172.21.15.99:0/2638611231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf70159da0 0x7fdf700b2da0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fdf640098e0 tx=0x7fdf640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3028fea8e655e18 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:47.126 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.123+0000 7fdf6effd700 1 -- 172.21.15.99:0/2638611231 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf64009e80 con 0x7fdf70159da0 2023-10-08T00:11:47.126 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.123+0000 7fdf6effd700 1 -- 172.21.15.99:0/2638611231 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf6400b860 con 0x7fdf70159da0 2023-10-08T00:11:47.126 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.123+0000 7fdf6effd700 1 -- 172.21.15.99:0/2638611231 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6400bbe0 con 0x7fdf70159da0 2023-10-08T00:11:47.126 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.124+0000 7fdf753af700 1 -- 172.21.15.99:0/2638611231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf70159da0 msgr2=0x7fdf700b2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:47.126 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.124+0000 7fdf753af700 1 --2- 172.21.15.99:0/2638611231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf70159da0 0x7fdf700b2da0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fdf640098e0 tx=0x7fdf640091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:47.126 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.124+0000 7fdf753af700 1 -- 172.21.15.99:0/2638611231 shutdown_connections 2023-10-08T00:11:47.126 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.124+0000 7fdf753af700 1 --2- 172.21.15.99:0/2638611231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf70159da0 0x7fdf700b2da0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:47.126 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.124+0000 7fdf753af700 1 -- 172.21.15.99:0/2638611231 >> 172.21.15.99:0/2638611231 conn(0x7fdf700d0bc0 msgr2=0x7fdf700d0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:47.126 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.124+0000 7fdf753af700 1 -- 172.21.15.99:0/2638611231 shutdown_connections 2023-10-08T00:11:47.126 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.124+0000 7fdf753af700 1 -- 172.21.15.99:0/2638611231 wait complete. 2023-10-08T00:11:47.129 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.125+0000 7fdf753af700 1 Processor -- start 2023-10-08T00:11:47.129 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.125+0000 7fdf753af700 1 -- start start 2023-10-08T00:11:47.129 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.126+0000 7fdf753af700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf70069210 0x7fdf700c6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:47.129 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.126+0000 7fdf753af700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf70159da0 0x7fdf7015af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:47.129 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.126+0000 7fdf753af700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf7015a170 0x7fdf700b4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:47.129 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.126+0000 7fdf753af700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7015b9e0 con 0x7fdf70159da0 2023-10-08T00:11:47.129 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.126+0000 7fdf753af700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7016bd30 con 0x7fdf70069210 2023-10-08T00:11:47.129 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.126+0000 7fdf753af700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7016b400 con 0x7fdf7015a170 2023-10-08T00:11:47.129 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.126+0000 7fdf6f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf70159da0 0x7fdf7015af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:47.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.126+0000 7fdf6f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf70159da0 0x7fdf7015af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50646/0 (socket says 172.21.15.99:50646) 2023-10-08T00:11:47.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.126+0000 7fdf6f7fe700 1 -- 172.21.15.99:0/3942731887 learned_addr learned my addr 172.21.15.99:0/3942731887 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:47.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.126+0000 7fdf6ffff700 1 --2- 172.21.15.99:0/3942731887 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf70069210 0x7fdf700c6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:47.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.126+0000 7fdf74bae700 1 --2- 172.21.15.99:0/3942731887 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf7015a170 0x7fdf700b4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:47.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.126+0000 7fdf6f7fe700 1 -- 172.21.15.99:0/3942731887 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf7015a170 msgr2=0x7fdf700b4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:47.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.126+0000 7fdf6f7fe700 1 --2- 172.21.15.99:0/3942731887 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf7015a170 0x7fdf700b4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:47.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.126+0000 7fdf6f7fe700 1 -- 172.21.15.99:0/3942731887 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf70069210 msgr2=0x7fdf700c6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:47.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.126+0000 7fdf6f7fe700 1 --2- 172.21.15.99:0/3942731887 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf70069210 0x7fdf700c6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:47.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.126+0000 7fdf6f7fe700 1 -- 172.21.15.99:0/3942731887 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf64009420 con 0x7fdf70159da0 2023-10-08T00:11:47.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.127+0000 7fdf6f7fe700 1 --2- 172.21.15.99:0/3942731887 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf70159da0 0x7fdf7015af40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fdf6000c560 tx=0x7fdf60000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:47.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.127+0000 7fdf57fff700 1 -- 172.21.15.99:0/3942731887 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf60005b30 con 0x7fdf70159da0 2023-10-08T00:11:47.131 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.127+0000 7fdf57fff700 1 -- 172.21.15.99:0/3942731887 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf60005c90 con 0x7fdf70159da0 2023-10-08T00:11:47.131 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.127+0000 7fdf753af700 1 -- 172.21.15.99:0/3942731887 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf7016b6e0 con 0x7fdf70159da0 2023-10-08T00:11:47.131 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.127+0000 7fdf57fff700 1 -- 172.21.15.99:0/3942731887 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf60007320 con 0x7fdf70159da0 2023-10-08T00:11:47.131 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.127+0000 7fdf753af700 1 -- 172.21.15.99:0/3942731887 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf7015d090 con 0x7fdf70159da0 2023-10-08T00:11:47.131 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.128+0000 7fdf57fff700 1 -- 172.21.15.99:0/3942731887 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdf6001b3b0 con 0x7fdf70159da0 2023-10-08T00:11:47.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.129+0000 7fdf57fff700 1 --2- 172.21.15.99:0/3942731887 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdf5c066f30 0x7fdf5c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:47.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.129+0000 7fdf57fff700 1 -- 172.21.15.99:0/3942731887 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdf60083930 con 0x7fdf70159da0 2023-10-08T00:11:47.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.129+0000 7fdf6ffff700 1 --2- 172.21.15.99:0/3942731887 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdf5c066f30 0x7fdf5c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:47.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.129+0000 7fdf6ffff700 1 --2- 172.21.15.99:0/3942731887 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdf5c066f30 0x7fdf5c0693e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdf640093f0 tx=0x7fdf64017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:47.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.130+0000 7fdf7f909b80 1 Processor -- start 2023-10-08T00:11:47.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.131+0000 7fdf7f909b80 1 -- start start 2023-10-08T00:11:47.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.131+0000 7fdf7f909b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55ab03a48480 0x55ab03a277d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:47.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.131+0000 7fdf7f909b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab039fb5f0 0x55ab03a48850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:47.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.131+0000 7fdf7f909b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab039fdb80 0x55ab039fdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:47.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.131+0000 7fdf7f909b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab03954a70 con 0x55ab039fb5f0 2023-10-08T00:11:47.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.131+0000 7fdf7f909b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab03952270 con 0x55ab039fdb80 2023-10-08T00:11:47.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.131+0000 7fdf7f909b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ab0394e060 con 0x55ab03a48480 2023-10-08T00:11:47.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.132+0000 7fdf6f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab039fb5f0 0x55ab03a48850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:47.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.132+0000 7fdf6f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab039fb5f0 0x55ab03a48850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50648/0 (socket says 172.21.15.99:50648) 2023-10-08T00:11:47.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.132+0000 7fdf6f7fe700 1 -- 172.21.15.99:0/1788190850 learned_addr learned my addr 172.21.15.99:0/1788190850 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:47.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.132+0000 7fdf6ffff700 1 --2- 172.21.15.99:0/1788190850 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55ab03a48480 0x55ab03a277d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:47.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.132+0000 7fdf74bae700 1 --2- 172.21.15.99:0/1788190850 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab039fdb80 0x55ab039fdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:47.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.132+0000 7fdf6f7fe700 1 -- 172.21.15.99:0/1788190850 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55ab03a48480 msgr2=0x55ab03a277d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:47.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.132+0000 7fdf6f7fe700 1 --2- 172.21.15.99:0/1788190850 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55ab03a48480 0x55ab03a277d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:47.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.132+0000 7fdf6f7fe700 1 -- 172.21.15.99:0/1788190850 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab039fdb80 msgr2=0x55ab039fdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:47.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.132+0000 7fdf6f7fe700 1 --2- 172.21.15.99:0/1788190850 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab039fdb80 0x55ab039fdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:47.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.132+0000 7fdf6f7fe700 1 -- 172.21.15.99:0/1788190850 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab037e8270 con 0x55ab039fb5f0 2023-10-08T00:11:47.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.132+0000 7fdf74bae700 1 --2- 172.21.15.99:0/1788190850 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab039fdb80 0x55ab039fdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:11:47.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.132+0000 7fdf6f7fe700 1 --2- 172.21.15.99:0/1788190850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab039fb5f0 0x55ab03a48850 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fdf600070a0 tx=0x7fdf60081db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3aa36de494b216c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:47.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.132+0000 7fdf557fa700 1 -- 172.21.15.99:0/1788190850 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf60050ba0 con 0x55ab039fb5f0 2023-10-08T00:11:47.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.132+0000 7fdf557fa700 1 -- 172.21.15.99:0/1788190850 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf60054860 con 0x55ab039fb5f0 2023-10-08T00:11:47.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.132+0000 7fdf557fa700 1 -- 172.21.15.99:0/1788190850 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf60057ab0 con 0x55ab039fb5f0 2023-10-08T00:11:47.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.133+0000 7fdf7f909b80 1 -- 172.21.15.99:0/1788190850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab039fb5f0 msgr2=0x55ab03a48850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:47.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.133+0000 7fdf7f909b80 1 --2- 172.21.15.99:0/1788190850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab039fb5f0 0x55ab03a48850 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fdf600070a0 tx=0x7fdf60081db0 comp rx=0 tx=0).stop 2023-10-08T00:11:47.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.133+0000 7fdf7f909b80 1 -- 172.21.15.99:0/1788190850 shutdown_connections 2023-10-08T00:11:47.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.133+0000 7fdf7f909b80 1 --2- 172.21.15.99:0/1788190850 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab039fdb80 0x55ab039fdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:47.137 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.133+0000 7fdf7f909b80 1 --2- 172.21.15.99:0/1788190850 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55ab03a48480 0x55ab03a277d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:47.137 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.133+0000 7fdf7f909b80 1 --2- 172.21.15.99:0/1788190850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab039fb5f0 0x55ab03a48850 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:47.137 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.133+0000 7fdf7f909b80 1 -- 172.21.15.99:0/1788190850 >> 172.21.15.99:0/1788190850 conn(0x55ab039ff0c0 msgr2=0x55ab039ff8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:47.137 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.133+0000 7fdf7f909b80 1 -- 172.21.15.99:0/1788190850 shutdown_connections 2023-10-08T00:11:47.137 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.133+0000 7fdf7f909b80 1 -- 172.21.15.99:0/1788190850 wait complete. 2023-10-08T00:11:47.137 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.133+0000 7fdf7f909b80 1 -- start start 2023-10-08T00:11:47.137 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.134+0000 7fdf7f909b80 1 Processor -- start 2023-10-08T00:11:47.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.134+0000 7fdf7f909b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab039fb5f0 0x55ab03a44290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:47.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.134+0000 7fdf7f909b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab039fdb80 0x55ab03a21ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:47.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.134+0000 7fdf7f909b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55ab03a48480 0x55ab03a25700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:47.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.134+0000 7fdf7f909b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab038f9890 con 0x55ab039fb5f0 2023-10-08T00:11:47.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.134+0000 7fdf7f909b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab03914ea0 con 0x55ab039fdb80 2023-10-08T00:11:47.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.134+0000 7fdf7f909b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ab037d0e30 con 0x55ab03a48480 2023-10-08T00:11:47.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.134+0000 7fdf7f909b80 10 client.0 _ll_register_callbacks cb 0x55ab039c12c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T00:11:47.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.134+0000 7fdf6ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab039fb5f0 0x55ab03a44290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:47.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.134+0000 7fdf6ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab039fb5f0 0x55ab03a44290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50656/0 (socket says 172.21.15.99:50656) 2023-10-08T00:11:47.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.134+0000 7fdf6ffff700 1 -- 172.21.15.99:0/2300434050 learned_addr learned my addr 172.21.15.99:0/2300434050 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:47.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.134+0000 7fdf74bae700 1 --2- 172.21.15.99:0/2300434050 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55ab03a48480 0x55ab03a25700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:47.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.135+0000 7fdf6f7fe700 1 --2- 172.21.15.99:0/2300434050 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab039fdb80 0x55ab03a21ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:47.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.135+0000 7fdf6ffff700 1 -- 172.21.15.99:0/2300434050 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55ab03a48480 msgr2=0x55ab03a25700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:47.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.135+0000 7fdf6ffff700 1 --2- 172.21.15.99:0/2300434050 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55ab03a48480 0x55ab03a25700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:47.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.135+0000 7fdf6ffff700 1 -- 172.21.15.99:0/2300434050 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab039fdb80 msgr2=0x55ab03a21ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:47.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.135+0000 7fdf6ffff700 1 --2- 172.21.15.99:0/2300434050 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab039fdb80 0x55ab03a21ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:47.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.135+0000 7fdf6ffff700 1 -- 172.21.15.99:0/2300434050 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf7015d090 con 0x55ab039fb5f0 2023-10-08T00:11:47.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.135+0000 7fdf6ffff700 1 --2- 172.21.15.99:0/2300434050 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab039fb5f0 0x55ab03a44290 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fdf6400b3c0 tx=0x7fdf64022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b16c140f97983d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:47.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.135+0000 7fdf4e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T00:11:47.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.135+0000 7fdf4e7fc700 1 -- 172.21.15.99:0/2300434050 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6402c070 con 0x55ab039fb5f0 2023-10-08T00:11:47.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.135+0000 7fdf4e7fc700 1 -- 172.21.15.99:0/2300434050 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdf64033ad0 con 0x55ab039fb5f0 2023-10-08T00:11:47.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.135+0000 7fdf7f909b80 1 -- 172.21.15.99:0/2300434050 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ab039f2cd0 con 0x55ab039fb5f0 2023-10-08T00:11:47.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.135+0000 7fdf4e7fc700 1 -- 172.21.15.99:0/2300434050 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf64033ea0 con 0x55ab039fb5f0 2023-10-08T00:11:47.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.135+0000 7fdf7f909b80 10 client.5076 fetch_fsmap learned FSMap version 17 2023-10-08T00:11:47.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.135+0000 7fdf7f909b80 1 -- 172.21.15.99:0/2300434050 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55ab0378afe0 con 0x55ab039fb5f0 2023-10-08T00:11:47.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.135+0000 7fdf4e7fc700 1 -- 172.21.15.99:0/2300434050 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ab039f2cd0 con 0x55ab039fb5f0 2023-10-08T00:11:47.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.135+0000 7fdf4e7fc700 1 -- 172.21.15.99:0/2300434050 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 17) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fdf64011630 con 0x55ab039fb5f0 2023-10-08T00:11:47.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.135+0000 7fdf7f909b80 10 client.5076 fetch_fsmap finished waiting for FSMap version 17 2023-10-08T00:11:47.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.135+0000 7fdf7f909b80 10 client.5076 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-08T00:11:47.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.135+0000 7fdf7f909b80 20 client.5076 populate_metadata read hostname 'smithi099' 2023-10-08T00:11:47.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.135+0000 7fdf7f909b80 1 --2- 172.21.15.99:0/2300434050 >> [v2:172.21.15.99:6833/4189095717,v1:172.21.15.99:6835/4189095717] conn(0x55ab03a27d60 0x55ab03a28130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:47.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.135+0000 7fdf7f909b80 4 client.5076 mds_command: new command op to 4902 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-08T00:11:47.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.135+0000 7fdf7f909b80 1 -- 172.21.15.99:0/2300434050 --> [v2:172.21.15.99:6833/4189095717,v1:172.21.15.99:6835/4189095717] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ab039cfaa0 con 0x55ab03a27d60 2023-10-08T00:11:47.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.136+0000 7fdf6f7fe700 1 --2- 172.21.15.99:0/2300434050 >> [v2:172.21.15.99:6833/4189095717,v1:172.21.15.99:6835/4189095717] conn(0x55ab03a27d60 0x55ab03a28130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:47.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.136+0000 7fdf4e7fc700 10 client.5076 ms_handle_connect on v2:172.21.15.99:6833/4189095717 2023-10-08T00:11:47.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.136+0000 7fdf6f7fe700 1 --2- 172.21.15.99:0/2300434050 >> [v2:172.21.15.99:6833/4189095717,v1:172.21.15.99:6835/4189095717] conn(0x55ab03a27d60 0x55ab03a28130 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29662e8041559ec3 server_cookie=c4e803ac24f56e2f in_seq=0 out_seq=0 2023-10-08T00:11:47.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.136+0000 7fdf4e7fc700 1 -- 172.21.15.99:0/2300434050 <== mds.0 v2:172.21.15.99:6833/4189095717 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55ab039cfaa0 con 0x55ab03a27d60 2023-10-08T00:11:47.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.136+0000 7fdf4e7fc700 10 client.5076 handle_command_reply: tid=0 2023-10-08T00:11:47.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.136+0000 7fdf7f909b80 1 client.5076 shutdown 2023-10-08T00:11:47.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.136+0000 7fdf7f909b80 1 -- 172.21.15.99:0/2300434050 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab039fb5f0 msgr2=0x55ab03a44290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:47.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.136+0000 7fdf7f909b80 1 --2- 172.21.15.99:0/2300434050 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab039fb5f0 0x55ab03a44290 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fdf6400b3c0 tx=0x7fdf64022000 comp rx=0 tx=0).stop 2023-10-08T00:11:47.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.137+0000 7fdf7f909b80 1 -- 172.21.15.99:0/2300434050 shutdown_connections 2023-10-08T00:11:47.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.137+0000 7fdf7f909b80 1 --2- 172.21.15.99:0/2300434050 >> [v2:172.21.15.99:6833/4189095717,v1:172.21.15.99:6835/4189095717] conn(0x55ab03a27d60 0x55ab03a28130 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:47.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.137+0000 7fdf7f909b80 1 --2- 172.21.15.99:0/2300434050 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55ab03a48480 0x55ab03a25700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:47.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.137+0000 7fdf4e7fc700 0 client.5076 ms_handle_reset on v2:172.21.15.99:6833/4189095717 2023-10-08T00:11:47.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.137+0000 7fdf7f909b80 1 --2- 172.21.15.99:0/2300434050 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab039fb5f0 0x55ab03a44290 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:47.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.137+0000 7fdf7f909b80 1 --2- 172.21.15.99:0/2300434050 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab039fdb80 0x55ab03a21ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:47.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.137+0000 7fdf7f909b80 1 -- 172.21.15.99:0/2300434050 >> 172.21.15.99:0/2300434050 conn(0x55ab039ff0c0 msgr2=0x55ab039f4bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:47.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.137+0000 7fdf7f909b80 1 -- 172.21.15.99:0/2300434050 shutdown_connections 2023-10-08T00:11:47.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.137+0000 7fdf7f909b80 1 -- 172.21.15.99:0/2300434050 wait complete. 2023-10-08T00:11:47.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.137+0000 7fdf7f909b80 20 client.5076 trim_cache size 0 max 16384 2023-10-08T00:11:47.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.153+0000 7fdf7f909b80 1 Processor -- start 2023-10-08T00:11:47.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.153+0000 7fdf7f909b80 1 -- start start 2023-10-08T00:11:47.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.154+0000 7fdf7f909b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab039fb5f0 0x55ab03a23570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:47.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.154+0000 7fdf7f909b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab039fdb80 0x55ab039faa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:47.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.154+0000 7fdf7f909b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55ab03a48480 0x55ab039fafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:47.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.154+0000 7fdf7f909b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab03a26fe0 con 0x55ab039fdb80 2023-10-08T00:11:47.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.154+0000 7fdf7f909b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab03a2b8a0 con 0x55ab039fb5f0 2023-10-08T00:11:47.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.154+0000 7fdf7f909b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ab03a25c50 con 0x55ab03a48480 2023-10-08T00:11:47.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.154+0000 7fdf6f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab039fdb80 0x55ab039faa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:47.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.154+0000 7fdf6f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab039fdb80 0x55ab039faa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50662/0 (socket says 172.21.15.99:50662) 2023-10-08T00:11:47.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.154+0000 7fdf6f7fe700 1 -- 172.21.15.99:0/2110164589 learned_addr learned my addr 172.21.15.99:0/2110164589 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:47.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.154+0000 7fdf74bae700 1 --2- 172.21.15.99:0/2110164589 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55ab03a48480 0x55ab039fafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:47.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.154+0000 7fdf6ffff700 1 --2- 172.21.15.99:0/2110164589 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab039fb5f0 0x55ab03a23570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:47.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.154+0000 7fdf6f7fe700 1 -- 172.21.15.99:0/2110164589 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55ab03a48480 msgr2=0x55ab039fafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:47.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.154+0000 7fdf6f7fe700 1 --2- 172.21.15.99:0/2110164589 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55ab03a48480 0x55ab039fafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:47.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.154+0000 7fdf6f7fe700 1 -- 172.21.15.99:0/2110164589 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab039fb5f0 msgr2=0x55ab03a23570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:47.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.154+0000 7fdf6f7fe700 1 --2- 172.21.15.99:0/2110164589 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab039fb5f0 0x55ab03a23570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:47.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.154+0000 7fdf6f7fe700 1 -- 172.21.15.99:0/2110164589 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab0391c780 con 0x55ab039fdb80 2023-10-08T00:11:47.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.154+0000 7fdf6f7fe700 1 --2- 172.21.15.99:0/2110164589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab039fdb80 0x55ab039faa70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fdf60052bb0 tx=0x7fdf60069000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5784c07e3ae1c070 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:47.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.155+0000 7fdf55ffb700 1 -- 172.21.15.99:0/2110164589 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf60081040 con 0x55ab039fdb80 2023-10-08T00:11:47.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.155+0000 7fdf55ffb700 1 -- 172.21.15.99:0/2110164589 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf60057ab0 con 0x55ab039fdb80 2023-10-08T00:11:47.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.155+0000 7fdf55ffb700 1 -- 172.21.15.99:0/2110164589 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf60005e00 con 0x55ab039fdb80 2023-10-08T00:11:47.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.155+0000 7fdf7f909b80 1 -- 172.21.15.99:0/2110164589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab039fdb80 msgr2=0x55ab039faa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:47.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.155+0000 7fdf7f909b80 1 --2- 172.21.15.99:0/2110164589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab039fdb80 0x55ab039faa70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fdf60052bb0 tx=0x7fdf60069000 comp rx=0 tx=0).stop 2023-10-08T00:11:47.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.155+0000 7fdf7f909b80 1 -- 172.21.15.99:0/2110164589 shutdown_connections 2023-10-08T00:11:47.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.155+0000 7fdf7f909b80 1 --2- 172.21.15.99:0/2110164589 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab039fb5f0 0x55ab03a23570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:47.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.155+0000 7fdf7f909b80 1 --2- 172.21.15.99:0/2110164589 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55ab03a48480 0x55ab039fafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:47.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.155+0000 7fdf7f909b80 1 --2- 172.21.15.99:0/2110164589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab039fdb80 0x55ab039faa70 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:47.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.155+0000 7fdf7f909b80 1 -- 172.21.15.99:0/2110164589 >> 172.21.15.99:0/2110164589 conn(0x55ab039ff0c0 msgr2=0x55ab03a25540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:47.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.155+0000 7fdf7f909b80 1 -- 172.21.15.99:0/2110164589 shutdown_connections 2023-10-08T00:11:47.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.155+0000 7fdf7f909b80 1 -- 172.21.15.99:0/2110164589 wait complete. 2023-10-08T00:11:47.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.156+0000 7fdf7f909b80 1 -- start start 2023-10-08T00:11:47.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.156+0000 7fdf7f909b80 1 Processor -- start 2023-10-08T00:11:47.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.157+0000 7fdf7f909b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab039fb5f0 0x55ab03a29590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:47.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.157+0000 7fdf7f909b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab039fdb80 0x55ab03a23b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:47.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.157+0000 7fdf7f909b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55ab03a48480 0x55ab03a240a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:47.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.157+0000 7fdf7f909b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab03a5c5b0 con 0x55ab039fb5f0 2023-10-08T00:11:47.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.157+0000 7fdf7f909b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab03a599c0 con 0x55ab039fdb80 2023-10-08T00:11:47.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.157+0000 7fdf7f909b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ab03a58df0 con 0x55ab03a48480 2023-10-08T00:11:47.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.157+0000 7fdf7f909b80 10 client.0 _ll_register_callbacks cb 0x55ab039c12c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T00:11:47.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.157+0000 7fdf6ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab039fb5f0 0x55ab03a29590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:47.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.157+0000 7fdf6ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab039fb5f0 0x55ab03a29590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50670/0 (socket says 172.21.15.99:50670) 2023-10-08T00:11:47.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.157+0000 7fdf6ffff700 1 -- 172.21.15.99:0/1288150134 learned_addr learned my addr 172.21.15.99:0/1288150134 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:47.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.157+0000 7fdf74bae700 1 --2- 172.21.15.99:0/1288150134 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55ab03a48480 0x55ab03a240a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:47.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.157+0000 7fdf6f7fe700 1 --2- 172.21.15.99:0/1288150134 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab039fdb80 0x55ab03a23b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:47.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.157+0000 7fdf6ffff700 1 -- 172.21.15.99:0/1288150134 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55ab03a48480 msgr2=0x55ab03a240a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:47.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.157+0000 7fdf6ffff700 1 --2- 172.21.15.99:0/1288150134 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55ab03a48480 0x55ab03a240a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:47.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.157+0000 7fdf6ffff700 1 -- 172.21.15.99:0/1288150134 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab039fdb80 msgr2=0x55ab03a23b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:47.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.157+0000 7fdf6ffff700 1 --2- 172.21.15.99:0/1288150134 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab039fdb80 0x55ab03a23b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:47.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.157+0000 7fdf6ffff700 1 -- 172.21.15.99:0/1288150134 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ab037e8270 con 0x55ab039fb5f0 2023-10-08T00:11:47.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.157+0000 7fdf6f7fe700 1 --2- 172.21.15.99:0/1288150134 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab039fdb80 0x55ab03a23b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:11:47.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.157+0000 7fdf6ffff700 1 --2- 172.21.15.99:0/1288150134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab039fb5f0 0x55ab03a29590 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fdf64011bc0 tx=0x7fdf6400b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39dde55f4684eac3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:47.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.157+0000 7fdf4e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T00:11:47.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.157+0000 7fdf4e7fc700 1 -- 172.21.15.99:0/1288150134 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6402c040 con 0x55ab039fb5f0 2023-10-08T00:11:47.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.157+0000 7fdf4e7fc700 1 -- 172.21.15.99:0/1288150134 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdf64011630 con 0x55ab039fb5f0 2023-10-08T00:11:47.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.157+0000 7fdf7f909b80 1 -- 172.21.15.99:0/1288150134 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdf60054860 con 0x55ab039fb5f0 2023-10-08T00:11:47.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.157+0000 7fdf4e7fc700 1 -- 172.21.15.99:0/1288150134 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf64033ea0 con 0x55ab039fb5f0 2023-10-08T00:11:47.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.157+0000 7fdf7f909b80 10 client.5082 fetch_fsmap learned FSMap version 17 2023-10-08T00:11:47.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.157+0000 7fdf7f909b80 1 -- 172.21.15.99:0/1288150134 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55ab03903470 con 0x55ab039fb5f0 2023-10-08T00:11:47.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.158+0000 7fdf4e7fc700 1 -- 172.21.15.99:0/1288150134 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf60054860 con 0x55ab039fb5f0 2023-10-08T00:11:47.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.158+0000 7fdf4e7fc700 1 -- 172.21.15.99:0/1288150134 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 17) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fdf6403c680 con 0x55ab039fb5f0 2023-10-08T00:11:47.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.158+0000 7fdf7f909b80 10 client.5082 fetch_fsmap finished waiting for FSMap version 17 2023-10-08T00:11:47.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.158+0000 7fdf7f909b80 10 client.5082 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-08T00:11:47.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.158+0000 7fdf7f909b80 20 client.5082 populate_metadata read hostname 'smithi099' 2023-10-08T00:11:47.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.158+0000 7fdf7f909b80 1 --2- 172.21.15.99:0/1288150134 >> [v2:172.21.15.99:6833/4189095717,v1:172.21.15.99:6835/4189095717] conn(0x55ab03a221a0 0x55ab03a22570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:47.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.158+0000 7fdf7f909b80 4 client.5082 mds_command: new command op to 4902 tid=0[{"prefix": "status"}] 2023-10-08T00:11:47.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.158+0000 7fdf7f909b80 1 -- 172.21.15.99:0/1288150134 --> [v2:172.21.15.99:6833/4189095717,v1:172.21.15.99:6835/4189095717] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ab038f0e00 con 0x55ab03a221a0 2023-10-08T00:11:47.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.158+0000 7fdf6f7fe700 1 --2- 172.21.15.99:0/1288150134 >> [v2:172.21.15.99:6833/4189095717,v1:172.21.15.99:6835/4189095717] conn(0x55ab03a221a0 0x55ab03a22570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:47.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.158+0000 7fdf4e7fc700 10 client.5082 ms_handle_connect on v2:172.21.15.99:6833/4189095717 2023-10-08T00:11:47.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.158+0000 7fdf6f7fe700 1 --2- 172.21.15.99:0/1288150134 >> [v2:172.21.15.99:6833/4189095717,v1:172.21.15.99:6835/4189095717] conn(0x55ab03a221a0 0x55ab03a22570 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=446a394a2206ba2a server_cookie=e3759d62ef8ca16c in_seq=0 out_seq=0 2023-10-08T00:11:47.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.158+0000 7fdf4e7fc700 1 -- 172.21.15.99:0/1288150134 <== mds.0 v2:172.21.15.99:6833/4189095717 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55ab038f0e00 con 0x55ab03a221a0 2023-10-08T00:11:47.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.158+0000 7fdf4e7fc700 10 client.5082 handle_command_reply: tid=0 2023-10-08T00:11:47.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.158+0000 7fdf7f909b80 1 client.5082 shutdown 2023-10-08T00:11:47.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.158+0000 7fdf7f909b80 1 -- 172.21.15.99:0/1288150134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab039fb5f0 msgr2=0x55ab03a29590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:47.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.158+0000 7fdf7f909b80 1 --2- 172.21.15.99:0/1288150134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab039fb5f0 0x55ab03a29590 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fdf64011bc0 tx=0x7fdf6400b050 comp rx=0 tx=0).stop 2023-10-08T00:11:47.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.158+0000 7fdf7f909b80 1 -- 172.21.15.99:0/1288150134 shutdown_connections 2023-10-08T00:11:47.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.158+0000 7fdf7f909b80 1 --2- 172.21.15.99:0/1288150134 >> [v2:172.21.15.99:6833/4189095717,v1:172.21.15.99:6835/4189095717] conn(0x55ab03a221a0 0x55ab03a22570 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:47.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.158+0000 7fdf7f909b80 1 --2- 172.21.15.99:0/1288150134 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55ab03a48480 0x55ab03a240a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:47.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.158+0000 7fdf4e7fc700 0 client.5082 ms_handle_reset on v2:172.21.15.99:6833/4189095717 2023-10-08T00:11:47.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.158+0000 7fdf7f909b80 1 --2- 172.21.15.99:0/1288150134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab039fb5f0 0x55ab03a29590 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:47.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.158+0000 7fdf7f909b80 1 --2- 172.21.15.99:0/1288150134 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab039fdb80 0x55ab03a23b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:47.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.158+0000 7fdf7f909b80 1 -- 172.21.15.99:0/1288150134 >> 172.21.15.99:0/1288150134 conn(0x55ab039ff0c0 msgr2=0x55ab03a29c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:47.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.158+0000 7fdf7f909b80 1 -- 172.21.15.99:0/1288150134 shutdown_connections 2023-10-08T00:11:47.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.158+0000 7fdf7f909b80 1 -- 172.21.15.99:0/1288150134 wait complete. 2023-10-08T00:11:47.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.159+0000 7fdf7f909b80 20 client.5082 trim_cache size 0 max 16384 2023-10-08T00:11:47.167 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:11:47.167 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", 2023-10-08T00:11:47.167 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 0, 2023-10-08T00:11:47.168 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 4902, 2023-10-08T00:11:47.168 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2023-10-08T00:11:47.168 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2023-10-08T00:11:47.168 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2023-10-08T00:11:47.168 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 5.8661531450000002, 2023-10-08T00:11:47.168 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 17, 2023-10-08T00:11:47.168 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 31, 2023-10-08T00:11:47.168 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 31, 2023-10-08T00:11:47.168 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 13.747594851000001 2023-10-08T00:11:47.168 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:11:47.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.159+0000 7fdf753af700 1 -- 172.21.15.99:0/3942731887 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdf5c066f30 msgr2=0x7fdf5c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:47.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.159+0000 7fdf753af700 1 --2- 172.21.15.99:0/3942731887 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdf5c066f30 0x7fdf5c0693e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdf640093f0 tx=0x7fdf64017000 comp rx=0 tx=0).stop 2023-10-08T00:11:47.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.159+0000 7fdf753af700 1 -- 172.21.15.99:0/3942731887 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf70159da0 msgr2=0x7fdf7015af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:47.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.159+0000 7fdf753af700 1 --2- 172.21.15.99:0/3942731887 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf70159da0 0x7fdf7015af40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fdf6000c560 tx=0x7fdf60000fe0 comp rx=0 tx=0).stop 2023-10-08T00:11:47.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.159+0000 7fdf753af700 1 -- 172.21.15.99:0/3942731887 shutdown_connections 2023-10-08T00:11:47.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.159+0000 7fdf753af700 1 --2- 172.21.15.99:0/3942731887 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdf5c066f30 0x7fdf5c0693e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:47.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.159+0000 7fdf753af700 1 --2- 172.21.15.99:0/3942731887 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf70069210 0x7fdf700c6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:47.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.159+0000 7fdf753af700 1 --2- 172.21.15.99:0/3942731887 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf7015a170 0x7fdf700b4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:47.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.159+0000 7fdf753af700 1 --2- 172.21.15.99:0/3942731887 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf70159da0 0x7fdf7015af40 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:47.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.159+0000 7fdf753af700 1 -- 172.21.15.99:0/3942731887 >> 172.21.15.99:0/3942731887 conn(0x7fdf700d0bc0 msgr2=0x7fdf700d0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:47.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.159+0000 7fdf753af700 1 -- 172.21.15.99:0/3942731887 shutdown_connections 2023-10-08T00:11:47.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.159+0000 7fdf753af700 1 -- 172.21.15.99:0/3942731887 wait complete. 2023-10-08T00:11:47.171 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T00:11:47.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.284+0000 7fd0ba589700 1 Processor -- start 2023-10-08T00:11:47.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.284+0000 7fd0ba589700 1 -- start start 2023-10-08T00:11:47.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.284+0000 7fd0ba589700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd0b4157870 0x7fd0b415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:47.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.285+0000 7fd0ba589700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd0b4157c40 0x7fd0b4158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:47.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.285+0000 7fd0ba589700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0b415cbb0 con 0x7fd0b415a620 2023-10-08T00:11:47.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.285+0000 7fd0ba589700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0b415bfa0 con 0x7fd0b4157870 2023-10-08T00:11:47.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.285+0000 7fd0ba589700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b415c280 con 0x7fd0b4157c40 2023-10-08T00:11:47.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.285+0000 7fd0b37fe700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fd0b415a620 0x7fd0b415d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:54220/0 (socket says 172.21.15.99:54220) 2023-10-08T00:11:47.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.285+0000 7fd0b37fe700 1 -- 172.21.15.99:0/3168383689 learned_addr learned my addr 172.21.15.99:0/3168383689 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:47.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.285+0000 7fd0b8b26700 1 --2- 172.21.15.99:0/3168383689 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd0b4157c40 0x7fd0b4158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:47.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.285+0000 7fd0b2ffd700 1 -- 172.21.15.99:0/3168383689 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1412074253 0 0) 0x7fd0b415cbb0 con 0x7fd0b415a620 2023-10-08T00:11:47.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.285+0000 7fd0b3fff700 1 --2- 172.21.15.99:0/3168383689 >> v2:172.21.15.191:3300/0 conn(0x7fd0b4157870 0x7fd0b415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:47.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.285+0000 7fd0b2ffd700 1 -- 172.21.15.99:0/3168383689 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd09c003570 con 0x7fd0b415a620 2023-10-08T00:11:47.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.285+0000 7fd0b2ffd700 1 -- 172.21.15.99:0/3168383689 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3630194217 0 0) 0x7fd09c003570 con 0x7fd0b415a620 2023-10-08T00:11:47.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.285+0000 7fd0b2ffd700 1 -- 172.21.15.99:0/3168383689 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd0b4157c40 msgr2=0x7fd0b4158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:47.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.285+0000 7fd0b2ffd700 1 --2- 172.21.15.99:0/3168383689 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd0b4157c40 0x7fd0b4158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:47.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.285+0000 7fd0b2ffd700 1 -- 172.21.15.99:0/3168383689 >> v2:172.21.15.191:3300/0 conn(0x7fd0b4157870 msgr2=0x7fd0b415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:47.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.285+0000 7fd0b2ffd700 1 --2- 172.21.15.99:0/3168383689 >> v2:172.21.15.191:3300/0 conn(0x7fd0b4157870 0x7fd0b415b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:47.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.285+0000 7fd0b2ffd700 1 -- 172.21.15.99:0/3168383689 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b415c560 con 0x7fd0b415a620 2023-10-08T00:11:47.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.285+0000 7fd0b2ffd700 1 -- 172.21.15.99:0/3168383689 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7fd0a8001470 con 0x7fd0b415a620 2023-10-08T00:11:47.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.285+0000 7fd0b2ffd700 1 -- 172.21.15.99:0/3168383689 >> v1:172.21.15.99:6789/0 conn(0x7fd0b415a620 legacy=0x7fd0b415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:47.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.285+0000 7fd0b2ffd700 1 --2- 172.21.15.99:0/3168383689 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd0b4157c40 0x7fd09c003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:47.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.285+0000 7fd0b2ffd700 1 --2- 172.21.15.99:0/3168383689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd09c003fe0 0x7fd09c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:47.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.285+0000 7fd0b2ffd700 1 --2- 172.21.15.99:0/3168383689 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd09c004d50 0x7fd09c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:47.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.285+0000 7fd0b2ffd700 1 -- 172.21.15.99:0/3168383689 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b415bfa0 con 0x7fd09c004d50 2023-10-08T00:11:47.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.285+0000 7fd0b2ffd700 1 -- 172.21.15.99:0/3168383689 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b415c280 con 0x7fd09c003fe0 2023-10-08T00:11:47.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.285+0000 7fd0b2ffd700 1 -- 172.21.15.99:0/3168383689 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd09c00ba90 con 0x7fd0b4157c40 2023-10-08T00:11:47.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.285+0000 7fd0b8b26700 1 --2- 172.21.15.99:0/3168383689 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd09c004d50 0x7fd09c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:47.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.285+0000 7fd0b3fff700 1 --2- 172.21.15.99:0/3168383689 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd0b4157c40 0x7fd09c003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:47.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.286+0000 7fd0b37fe700 1 --2- 172.21.15.99:0/3168383689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd09c003fe0 0x7fd09c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:47.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.286+0000 7fd0b8b26700 1 -- 172.21.15.99:0/3168383689 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd0b4157c40 msgr2=0x7fd09c003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:47.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.286+0000 7fd0b8b26700 1 --2- 172.21.15.99:0/3168383689 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd0b4157c40 0x7fd09c003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:47.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.286+0000 7fd0b8b26700 1 -- 172.21.15.99:0/3168383689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd09c003fe0 msgr2=0x7fd09c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:47.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.286+0000 7fd0b8b26700 1 --2- 172.21.15.99:0/3168383689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd09c003fe0 0x7fd09c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:47.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.286+0000 7fd0b8b26700 1 -- 172.21.15.99:0/3168383689 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd09c003570 con 0x7fd09c004d50 2023-10-08T00:11:47.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.286+0000 7fd0b8b26700 1 --2- 172.21.15.99:0/3168383689 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd09c004d50 0x7fd09c0051c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd0b41580c0 tx=0x7fd0ac00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45eede6072d8f7ba server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:47.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.286+0000 7fd0b2ffd700 1 -- 172.21.15.99:0/3168383689 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac00ea30 con 0x7fd09c004d50 2023-10-08T00:11:47.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.286+0000 7fd0b2ffd700 1 -- 172.21.15.99:0/3168383689 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0ac00f040 con 0x7fd09c004d50 2023-10-08T00:11:47.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.286+0000 7fd0b2ffd700 1 -- 172.21.15.99:0/3168383689 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac00ede0 con 0x7fd09c004d50 2023-10-08T00:11:47.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.286+0000 7fd0ba589700 1 -- 172.21.15.99:0/3168383689 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd09c004d50 msgr2=0x7fd09c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:47.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.286+0000 7fd0ba589700 1 --2- 172.21.15.99:0/3168383689 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd09c004d50 0x7fd09c0051c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd0b41580c0 tx=0x7fd0ac00b040 comp rx=0 tx=0).stop 2023-10-08T00:11:47.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.286+0000 7fd0b8b26700 1 -- 172.21.15.99:0/3168383689 reap_dead start 2023-10-08T00:11:47.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.286+0000 7fd0ba589700 1 -- 172.21.15.99:0/3168383689 shutdown_connections 2023-10-08T00:11:47.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.287+0000 7fd0ba589700 1 -- 172.21.15.99:0/3168383689 >> 172.21.15.99:0/3168383689 conn(0x7fd0b4068ce0 msgr2=0x7fd0b40690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:47.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.287+0000 7fd0ba589700 1 -- 172.21.15.99:0/3168383689 shutdown_connections 2023-10-08T00:11:47.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.287+0000 7fd0ba589700 1 -- 172.21.15.99:0/3168383689 wait complete. 2023-10-08T00:11:47.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.288+0000 7fd0ba589700 1 Processor -- start 2023-10-08T00:11:47.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.288+0000 7fd0ba589700 1 -- start start 2023-10-08T00:11:47.295 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.288+0000 7fd0ba589700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0b4148810 0x7fd0b40a7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:47.295 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.288+0000 7fd0ba589700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd0b4155a00 0x7fd0b40a9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:47.295 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.288+0000 7fd0ba589700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0b40a9560 0x7fd0b40b2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:47.295 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.289+0000 7fd0ba589700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b40a7f80 con 0x7fd0b4148810 2023-10-08T00:11:47.295 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.289+0000 7fd0ba589700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b40a8260 con 0x7fd0b40a9560 2023-10-08T00:11:47.295 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.289+0000 7fd0ba589700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b40a8540 con 0x7fd0b4155a00 2023-10-08T00:11:47.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.289+0000 7fd0b3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0b4148810 0x7fd0b40a7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:47.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.289+0000 7fd0b3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0b4148810 0x7fd0b40a7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50692/0 (socket says 172.21.15.99:50692) 2023-10-08T00:11:47.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.289+0000 7fd0b3fff700 1 -- 172.21.15.99:0/160304941 learned_addr learned my addr 172.21.15.99:0/160304941 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:47.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.289+0000 7fd0b3fff700 1 -- 172.21.15.99:0/160304941 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd0b4155a00 msgr2=0x7fd0b40a9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:47.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.289+0000 7fd0b3fff700 1 --2- 172.21.15.99:0/160304941 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd0b4155a00 0x7fd0b40a9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:47.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.289+0000 7fd0b3fff700 1 -- 172.21.15.99:0/160304941 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0b40a9560 msgr2=0x7fd0b40b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:47.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.289+0000 7fd0b8b26700 1 --2- 172.21.15.99:0/160304941 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0b40a9560 0x7fd0b40b2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:47.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.289+0000 7fd0b3fff700 1 --2- 172.21.15.99:0/160304941 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0b40a9560 0x7fd0b40b2b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:47.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.289+0000 7fd0b3fff700 1 -- 172.21.15.99:0/160304941 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b40a8820 con 0x7fd0b4148810 2023-10-08T00:11:47.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.289+0000 7fd0b3fff700 1 --2- 172.21.15.99:0/160304941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0b4148810 0x7fd0b40a7610 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd0a4006ac0 tx=0x7fd0a40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:47.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.289+0000 7fd09bfff700 1 -- 172.21.15.99:0/160304941 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a400fa10 con 0x7fd0b4148810 2023-10-08T00:11:47.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.289+0000 7fd09bfff700 1 -- 172.21.15.99:0/160304941 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0a400fb70 con 0x7fd0b4148810 2023-10-08T00:11:47.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.289+0000 7fd09bfff700 1 -- 172.21.15.99:0/160304941 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a4014440 con 0x7fd0b4148810 2023-10-08T00:11:47.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.289+0000 7fd0ba589700 1 -- 172.21.15.99:0/160304941 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0b40a8b00 con 0x7fd0b4148810 2023-10-08T00:11:47.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.289+0000 7fd0ba589700 1 -- 172.21.15.99:0/160304941 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0b40a7b50 con 0x7fd0b4148810 2023-10-08T00:11:47.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.290+0000 7fd0ba589700 1 -- 172.21.15.99:0/160304941 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0b40a8e70 con 0x7fd0b4148810 2023-10-08T00:11:47.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.290+0000 7fd09bfff700 1 -- 172.21.15.99:0/160304941 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd0a4014830 con 0x7fd0b4148810 2023-10-08T00:11:47.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.291+0000 7fd09bfff700 1 --2- 172.21.15.99:0/160304941 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd0a0066f30 0x7fd0a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:47.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.291+0000 7fd09bfff700 1 -- 172.21.15.99:0/160304941 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd0a4018030 con 0x7fd0b4148810 2023-10-08T00:11:47.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.291+0000 7fd0b37fe700 1 --2- 172.21.15.99:0/160304941 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd0a0066f30 0x7fd0a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:47.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.292+0000 7fd0b37fe700 1 --2- 172.21.15.99:0/160304941 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd0a0066f30 0x7fd0a00693e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8000e10 tx=0x7fd0a8001e90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:47.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.295+0000 7fd09bfff700 1 -- 172.21.15.99:0/160304941 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd0a4052210 con 0x7fd0b4148810 2023-10-08T00:11:47.522 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.521+0000 7fd0ba589700 1 -- 172.21.15.99:0/160304941 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd0b404c3c0 con 0x7fd0b4148810 2023-10-08T00:11:47.522 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.522+0000 7fd09bfff700 1 -- 172.21.15.99:0/160304941 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd0a4022070 con 0x7fd0b4148810 2023-10-08T00:11:47.523 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T00:11:47.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.524+0000 7fd0ba589700 1 -- 172.21.15.99:0/160304941 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd0a0066f30 msgr2=0x7fd0a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:47.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.524+0000 7fd0ba589700 1 --2- 172.21.15.99:0/160304941 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd0a0066f30 0x7fd0a00693e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8000e10 tx=0x7fd0a8001e90 comp rx=0 tx=0).stop 2023-10-08T00:11:47.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.524+0000 7fd0ba589700 1 -- 172.21.15.99:0/160304941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0b4148810 msgr2=0x7fd0b40a7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:47.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.524+0000 7fd0ba589700 1 --2- 172.21.15.99:0/160304941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0b4148810 0x7fd0b40a7610 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd0a4006ac0 tx=0x7fd0a40091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:47.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.524+0000 7fd0ba589700 1 -- 172.21.15.99:0/160304941 shutdown_connections 2023-10-08T00:11:47.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.524+0000 7fd0ba589700 1 --2- 172.21.15.99:0/160304941 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd0a0066f30 0x7fd0a00693e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:47.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.524+0000 7fd0ba589700 1 --2- 172.21.15.99:0/160304941 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0b40a9560 0x7fd0b40b2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:47.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.524+0000 7fd0ba589700 1 --2- 172.21.15.99:0/160304941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0b4148810 0x7fd0b40a7610 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:47.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.524+0000 7fd0ba589700 1 --2- 172.21.15.99:0/160304941 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd0b4155a00 0x7fd0b40a9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:47.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.524+0000 7fd0ba589700 1 -- 172.21.15.99:0/160304941 >> 172.21.15.99:0/160304941 conn(0x7fd0b4068ce0 msgr2=0x7fd0b40d03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:47.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.524+0000 7fd0ba589700 1 -- 172.21.15.99:0/160304941 shutdown_connections 2023-10-08T00:11:47.526 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:47.524+0000 7fd0ba589700 1 -- 172.21.15.99:0/160304941 wait complete. 2023-10-08T00:11:47.536 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T00:11:47.536 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T00:11:47.536 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi099.front.sepia.ceph.com 2023-10-08T00:11:47.536 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T00:11:47.537 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T00:11:47.537 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T00:11:47.537 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T00:11:47.537 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T00:11:47.537 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T00:11:47.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:47.537 DEBUG:teuthology.orchestra.run.smithi099:> ip addr 2023-10-08T00:11:47.551 INFO:teuthology.orchestra.run.smithi099.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T00:11:47.552 INFO:teuthology.orchestra.run.smithi099.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T00:11:47.552 INFO:teuthology.orchestra.run.smithi099.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T00:11:47.552 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:11:47.552 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 ::1/128 scope host 2023-10-08T00:11:47.552 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:11:47.552 INFO:teuthology.orchestra.run.smithi099.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:11:47.552 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:10 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:11:47.552 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f0 2023-10-08T00:11:47.552 INFO:teuthology.orchestra.run.smithi099.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:11:47.552 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:11 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:11:47.553 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f1 2023-10-08T00:11:47.553 INFO:teuthology.orchestra.run.smithi099.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:11:47.553 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:02 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:11:47.553 INFO:teuthology.orchestra.run.smithi099.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T00:11:47.553 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:03 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:11:47.553 INFO:teuthology.orchestra.run.smithi099.stdout: inet 172.21.15.99/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T00:11:47.553 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft 42482sec preferred_lft 42482sec 2023-10-08T00:11:47.553 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::ec4:7aff:fe8f:d803/64 scope link 2023-10-08T00:11:47.553 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:11:47.553 INFO:teuthology.orchestra.run.smithi099.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T00:11:47.553 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 22:2c:1c:f6:42:e1 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:11:47.553 INFO:teuthology.orchestra.run.smithi099.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T00:11:47.554 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:11:47.554 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::600d:26ff:fede:5eec/64 scope link 2023-10-08T00:11:47.554 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:11:47.554 INFO:teuthology.orchestra.run.smithi099.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T00:11:47.554 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether ae:2f:b5:94:69:ba brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T00:11:47.554 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::ac2f:b5ff:fe94:69ba/64 scope link 2023-10-08T00:11:47.554 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:11:47.554 INFO:teuthology.orchestra.run.smithi099.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T00:11:47.554 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 22:2c:1c:f6:42:e1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T00:11:47.554 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::202c:1cff:fef6:42e1/64 scope link 2023-10-08T00:11:47.554 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:11:47.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:47.555 DEBUG:teuthology.orchestra.run.smithi099:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T00:11:47.614 INFO:teuthology.orchestra.run.smithi099.stdout:1 2023-10-08T00:11:47.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:47.615 DEBUG:teuthology.orchestra.run.smithi099:> route 2023-10-08T00:11:47.671 INFO:teuthology.orchestra.run.smithi099.stdout:Kernel IP routing table 2023-10-08T00:11:47.672 INFO:teuthology.orchestra.run.smithi099.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T00:11:47.672 INFO:teuthology.orchestra.run.smithi099.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T00:11:47.672 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T00:11:47.672 INFO:teuthology.orchestra.run.smithi099.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T00:11:47.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:47.672 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2023-10-08T00:11:47.672 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2023-10-08T00:11:47.672 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T00:11:47.673 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T00:11:47.673 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T00:11:47.673 DEBUG:teuthology.orchestra.run.smithi099:> ') 2023-10-08T00:11:47.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:47.771 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2023-10-08T00:11:47.785 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T00:11:47.786 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T00:11:47.786 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T00:11:47.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:47.786 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set brx.0 up 2023-10-08T00:11:47.850 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:11:47.850 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T00:11:47.904 INFO:teuthology.orchestra.run.smithi099.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T00:11:47.904 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:11:47.905 DEBUG:teuthology.orchestra.run.smithi099:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T00:11:47.958 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-08T00:11:47.958 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T00:11:47.959 DEBUG:teuthology.orchestra.run.smithi099:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:11:48.039 INFO:teuthology.orchestra.run.smithi099.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:11:48.039 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:11:48.058 INFO:teuthology.orchestra.run.smithi099.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:11:48.058 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:11:48.059 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: resolved to: "172.21.15.99:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T00:11:48.059 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: trying mount with new device syntax: 0@06962cdb-9a6f-417f-82fd-9e4e3dedbccf.cephfs=/ 2023-10-08T00:11:48.059 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.99:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T00:11:48.059 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T00:11:48.059 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T00:11:48.059 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T00:11:48.081 INFO:teuthology.orchestra.run.smithi099.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:11:48.081 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:11:48.081 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: resolved to: "172.21.15.99:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T00:11:48.081 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: trying mount with new device syntax: 0@06962cdb-9a6f-417f-82fd-9e4e3dedbccf.cephfs=/ 2023-10-08T00:11:48.081 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.99:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T00:11:48.082 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T00:11:48.082 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T00:11:48.082 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T00:11:48.082 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: resolved to: "172.21.15.99:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T00:11:48.082 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.99:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T00:11:48.082 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=06962cdb-9a6f-417f-82fd-9e4e3dedbccf" will pass to kernel 2023-10-08T00:11:48.084 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T00:11:48.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:48.085 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T00:11:48.150 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T00:11:48.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.306+0000 7fa4c3fff700 1 Processor -- start 2023-10-08T00:11:48.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.306+0000 7fa4c3fff700 1 -- start start 2023-10-08T00:11:48.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.307+0000 7fa4c3fff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa4c40cd6e0 0x7fa4c40cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:48.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.307+0000 7fa4c3fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4c40cdab0 0x7fa4c415d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:48.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.307+0000 7fa4c3fff700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4c40cb7c0 con 0x7fa4c415dda0 2023-10-08T00:11:48.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.307+0000 7fa4c3fff700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4c415e850 con 0x7fa4c40cd6e0 2023-10-08T00:11:48.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.307+0000 7fa4c3fff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c40cd030 con 0x7fa4c40cdab0 2023-10-08T00:11:48.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.307+0000 7fa4c2ffd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa4c40cd6e0 0x7fa4c40cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:48.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.307+0000 7fa4c2ffd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa4c40cd6e0 0x7fa4c40cbc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50706/0 (socket says 172.21.15.99:50706) 2023-10-08T00:11:48.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.307+0000 7fa4c2ffd700 1 -- 172.21.15.99:0/2841989802 learned_addr learned my addr 172.21.15.99:0/2841989802 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:48.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.307+0000 7fa4ba7fc700 1 --2- 172.21.15.99:0/2841989802 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4c40cdab0 0x7fa4c415d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:48.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.308+0000 7fa4c2ffd700 1 -- 172.21.15.99:0/2841989802 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4c40cdab0 msgr2=0x7fa4c415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:48.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.308+0000 7fa4c2ffd700 1 --2- 172.21.15.99:0/2841989802 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4c40cdab0 0x7fa4c415d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:48.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.308+0000 7fa4c2ffd700 1 -- 172.21.15.99:0/2841989802 >> v1:172.21.15.191:6789/0 conn(0x7fa4c415dda0 legacy=0x7fa4c415cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:48.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.308+0000 7fa4c2ffd700 1 -- 172.21.15.99:0/2841989802 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4c415e290 con 0x7fa4c40cd6e0 2023-10-08T00:11:48.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.308+0000 7fa4c2ffd700 1 --2- 172.21.15.99:0/2841989802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4c40cd6e0 0x7fa4c40cbc50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa4b40099b0 tx=0x7fa4b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=400cbd28887baf81 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:48.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.308+0000 7fa4c27fc700 1 -- 172.21.15.99:0/2841989802 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b4009ea0 con 0x7fa4c40cd6e0 2023-10-08T00:11:48.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.308+0000 7fa4c27fc700 1 -- 172.21.15.99:0/2841989802 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4b400b860 con 0x7fa4c40cd6e0 2023-10-08T00:11:48.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.308+0000 7fa4c27fc700 1 -- 172.21.15.99:0/2841989802 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b400bc00 con 0x7fa4c40cd6e0 2023-10-08T00:11:48.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.308+0000 7fa4c3fff700 1 -- 172.21.15.99:0/2841989802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4c40cd6e0 msgr2=0x7fa4c40cbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:48.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.308+0000 7fa4c3fff700 1 --2- 172.21.15.99:0/2841989802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4c40cd6e0 0x7fa4c40cbc50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa4b40099b0 tx=0x7fa4b4009250 comp rx=0 tx=0).stop 2023-10-08T00:11:48.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.308+0000 7fa4c3fff700 1 -- 172.21.15.99:0/2841989802 shutdown_connections 2023-10-08T00:11:48.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.308+0000 7fa4c3fff700 1 --2- 172.21.15.99:0/2841989802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4c40cd6e0 0x7fa4c40cbc50 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:48.313 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.308+0000 7fa4c3fff700 1 --2- 172.21.15.99:0/2841989802 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4c40cdab0 0x7fa4c415d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:48.313 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.308+0000 7fa4c3fff700 1 -- 172.21.15.99:0/2841989802 >> 172.21.15.99:0/2841989802 conn(0x7fa4c4067660 msgr2=0x7fa4c4067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:48.313 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.308+0000 7fa4c3fff700 1 -- 172.21.15.99:0/2841989802 shutdown_connections 2023-10-08T00:11:48.313 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.308+0000 7fa4c3fff700 1 -- 172.21.15.99:0/2841989802 wait complete. 2023-10-08T00:11:48.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.310+0000 7fa4c3fff700 1 Processor -- start 2023-10-08T00:11:48.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.310+0000 7fa4c3fff700 1 -- start start 2023-10-08T00:11:48.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.310+0000 7fa4c3fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4c40cd6e0 0x7fa4c414eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:48.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.310+0000 7fa4c3fff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4c40cdab0 0x7fa4c40ca060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:48.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.310+0000 7fa4c3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4c415dda0 0x7fa4c40b22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:48.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.310+0000 7fa4c3fff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c40c8000 con 0x7fa4c415dda0 2023-10-08T00:11:48.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.310+0000 7fa4c3fff700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c414f380 con 0x7fa4c40cdab0 2023-10-08T00:11:48.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.310+0000 7fa4c3fff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c414f660 con 0x7fa4c40cd6e0 2023-10-08T00:11:48.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.310+0000 7fa4c37fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4c415dda0 0x7fa4c40b22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:48.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.310+0000 7fa4c37fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4c415dda0 0x7fa4c40b22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50722/0 (socket says 172.21.15.99:50722) 2023-10-08T00:11:48.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.310+0000 7fa4c37fe700 1 -- 172.21.15.99:0/3085363155 learned_addr learned my addr 172.21.15.99:0/3085363155 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:48.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.311+0000 7fa4c2ffd700 1 --2- 172.21.15.99:0/3085363155 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4c40cd6e0 0x7fa4c414eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:48.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.311+0000 7fa4ba7fc700 1 --2- 172.21.15.99:0/3085363155 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4c40cdab0 0x7fa4c40ca060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:48.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.311+0000 7fa4c37fe700 1 -- 172.21.15.99:0/3085363155 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4c40cd6e0 msgr2=0x7fa4c414eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:48.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.311+0000 7fa4c37fe700 1 --2- 172.21.15.99:0/3085363155 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4c40cd6e0 0x7fa4c414eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:48.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.311+0000 7fa4c37fe700 1 -- 172.21.15.99:0/3085363155 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4c40cdab0 msgr2=0x7fa4c40ca060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:48.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.311+0000 7fa4c37fe700 1 --2- 172.21.15.99:0/3085363155 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4c40cdab0 0x7fa4c40ca060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:48.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.311+0000 7fa4c37fe700 1 -- 172.21.15.99:0/3085363155 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4b40094f0 con 0x7fa4c415dda0 2023-10-08T00:11:48.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.311+0000 7fa4ba7fc700 1 --2- 172.21.15.99:0/3085363155 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4c40cdab0 0x7fa4c40ca060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:11:48.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.311+0000 7fa4c37fe700 1 --2- 172.21.15.99:0/3085363155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4c415dda0 0x7fa4c40b22f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc00c840 tx=0x7fa4bc001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:48.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.311+0000 7fa4bb7fe700 1 -- 172.21.15.99:0/3085363155 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc012630 con 0x7fa4c415dda0 2023-10-08T00:11:48.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.311+0000 7fa4bb7fe700 1 -- 172.21.15.99:0/3085363155 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4bc0127b0 con 0x7fa4c415dda0 2023-10-08T00:11:48.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.311+0000 7fa4c3fff700 1 -- 172.21.15.99:0/3085363155 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4c414f940 con 0x7fa4c415dda0 2023-10-08T00:11:48.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.311+0000 7fa4bb7fe700 1 -- 172.21.15.99:0/3085363155 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc012b70 con 0x7fa4c415dda0 2023-10-08T00:11:48.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.311+0000 7fa4c3fff700 1 -- 172.21.15.99:0/3085363155 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4c414f070 con 0x7fa4c415dda0 2023-10-08T00:11:48.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.312+0000 7fa4c3fff700 1 -- 172.21.15.99:0/3085363155 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4c40c8170 con 0x7fa4c415dda0 2023-10-08T00:11:48.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.312+0000 7fa4bb7fe700 1 -- 172.21.15.99:0/3085363155 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa4bc003030 con 0x7fa4c415dda0 2023-10-08T00:11:48.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.313+0000 7fa4bb7fe700 1 --2- 172.21.15.99:0/3085363155 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa4a4066f30 0x7fa4a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:48.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.313+0000 7fa4bb7fe700 1 -- 172.21.15.99:0/3085363155 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa4bc0844f0 con 0x7fa4c415dda0 2023-10-08T00:11:48.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.313+0000 7fa4c2ffd700 1 --2- 172.21.15.99:0/3085363155 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa4a4066f30 0x7fa4a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:48.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.313+0000 7fa4c2ffd700 1 --2- 172.21.15.99:0/3085363155 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa4a4066f30 0x7fa4a40693e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa4b40099b0 tx=0x7fa4b4017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:48.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.317+0000 7fa4bb7fe700 1 -- 172.21.15.99:0/3085363155 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa4bc051760 con 0x7fa4c415dda0 2023-10-08T00:11:48.546 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.544+0000 7fa4c3fff700 1 -- 172.21.15.99:0/3085363155 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa4c415e570 con 0x7fa4c415dda0 2023-10-08T00:11:48.546 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.544+0000 7fa4bb7fe700 1 -- 172.21.15.99:0/3085363155 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa4bc054d80 con 0x7fa4c415dda0 2023-10-08T00:11:48.546 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T00:11:48.549 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.546+0000 7fa4c3fff700 1 -- 172.21.15.99:0/3085363155 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa4a4066f30 msgr2=0x7fa4a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:48.549 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.546+0000 7fa4c3fff700 1 --2- 172.21.15.99:0/3085363155 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa4a4066f30 0x7fa4a40693e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa4b40099b0 tx=0x7fa4b4017000 comp rx=0 tx=0).stop 2023-10-08T00:11:48.549 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.546+0000 7fa4c3fff700 1 -- 172.21.15.99:0/3085363155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4c415dda0 msgr2=0x7fa4c40b22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:48.549 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.546+0000 7fa4c3fff700 1 --2- 172.21.15.99:0/3085363155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4c415dda0 0x7fa4c40b22f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc00c840 tx=0x7fa4bc001670 comp rx=0 tx=0).stop 2023-10-08T00:11:48.550 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.546+0000 7fa4c3fff700 1 -- 172.21.15.99:0/3085363155 shutdown_connections 2023-10-08T00:11:48.550 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.546+0000 7fa4c3fff700 1 --2- 172.21.15.99:0/3085363155 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa4a4066f30 0x7fa4a40693e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:48.550 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.546+0000 7fa4c3fff700 1 --2- 172.21.15.99:0/3085363155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4c415dda0 0x7fa4c40b22f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:48.550 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.546+0000 7fa4c3fff700 1 --2- 172.21.15.99:0/3085363155 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4c40cd6e0 0x7fa4c414eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:48.550 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.546+0000 7fa4c3fff700 1 --2- 172.21.15.99:0/3085363155 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4c40cdab0 0x7fa4c40ca060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:48.550 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.546+0000 7fa4c3fff700 1 -- 172.21.15.99:0/3085363155 >> 172.21.15.99:0/3085363155 conn(0x7fa4c4067660 msgr2=0x7fa4c40d1340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:48.551 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.546+0000 7fa4c3fff700 1 -- 172.21.15.99:0/3085363155 shutdown_connections 2023-10-08T00:11:48.551 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:48.546+0000 7fa4c3fff700 1 -- 172.21.15.99:0/3085363155 wait complete. 2023-10-08T00:11:48.560 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T00:11:48.560 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T00:11:48.560 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi191.front.sepia.ceph.com 2023-10-08T00:11:48.560 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-08T00:11:48.560 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-08T00:11:48.560 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T00:11:48.561 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T00:11:48.561 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T00:11:48.561 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T00:11:48.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:48.561 DEBUG:teuthology.orchestra.run.smithi191:> ip addr 2023-10-08T00:11:48.581 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T00:11:48.581 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T00:11:48.581 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T00:11:48.581 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:11:48.581 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2023-10-08T00:11:48.581 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:11:48.581 INFO:teuthology.orchestra.run.smithi191.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:11:48.582 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f6 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:11:48.582 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f0 2023-10-08T00:11:48.582 INFO:teuthology.orchestra.run.smithi191.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:11:48.582 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f7 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:11:48.582 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f1 2023-10-08T00:11:48.582 INFO:teuthology.orchestra.run.smithi191.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:11:48.582 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:10 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:11:48.582 INFO:teuthology.orchestra.run.smithi191.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T00:11:48.582 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:11 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:11:48.582 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.21.15.191/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T00:11:48.582 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft 42483sec preferred_lft 42483sec 2023-10-08T00:11:48.583 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec4:7aff:fe88:7211/64 scope link 2023-10-08T00:11:48.583 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:11:48.583 INFO:teuthology.orchestra.run.smithi191.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T00:11:48.583 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 9a:bc:39:72:87:3d brd ff:ff:ff:ff:ff:ff 2023-10-08T00:11:48.583 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T00:11:48.583 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:11:48.583 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec77:a7ff:fea4:fbb/64 scope link 2023-10-08T00:11:48.583 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:11:48.583 INFO:teuthology.orchestra.run.smithi191.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T00:11:48.583 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether ea:1c:d9:46:05:b3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-08T00:11:48.583 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::e81c:d9ff:fe46:5b3/64 scope link 2023-10-08T00:11:48.584 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:11:48.584 INFO:teuthology.orchestra.run.smithi191.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T00:11:48.584 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 9a:bc:39:72:87:3d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-08T00:11:48.584 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::98bc:39ff:fe72:873d/64 scope link 2023-10-08T00:11:48.584 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:11:48.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:48.584 DEBUG:teuthology.orchestra.run.smithi191:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T00:11:48.650 INFO:teuthology.orchestra.run.smithi191.stdout:1 2023-10-08T00:11:48.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:48.652 DEBUG:teuthology.orchestra.run.smithi191:> route 2023-10-08T00:11:48.712 INFO:teuthology.orchestra.run.smithi191.stdout:Kernel IP routing table 2023-10-08T00:11:48.712 INFO:teuthology.orchestra.run.smithi191.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T00:11:48.712 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T00:11:48.712 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T00:11:48.712 INFO:teuthology.orchestra.run.smithi191.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T00:11:48.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:48.713 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2023-10-08T00:11:48.713 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2023-10-08T00:11:48.713 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T00:11:48.713 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T00:11:48.713 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T00:11:48.713 DEBUG:teuthology.orchestra.run.smithi191:> ') 2023-10-08T00:11:48.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:48.820 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2023-10-08T00:11:48.835 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-08T00:11:48.836 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-08T00:11:48.836 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T00:11:48.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:48.836 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set brx.0 up 2023-10-08T00:11:48.902 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:11:48.903 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-08T00:11:48.958 INFO:teuthology.orchestra.run.smithi191.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-08T00:11:48.959 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:11:48.959 DEBUG:teuthology.orchestra.run.smithi191:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-08T00:11:49.014 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-08T00:11:49.014 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T00:11:49.014 DEBUG:teuthology.orchestra.run.smithi191:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:11:49.095 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:11:49.095 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:11:49.114 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:11:49.114 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:11:49.114 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.99:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T00:11:49.114 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@06962cdb-9a6f-417f-82fd-9e4e3dedbccf.cephfs=/ 2023-10-08T00:11:49.115 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.99:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T00:11:49.115 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T00:11:49.115 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T00:11:49.115 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T00:11:49.136 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:11:49.136 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:11:49.136 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.99:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T00:11:49.136 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@06962cdb-9a6f-417f-82fd-9e4e3dedbccf.cephfs=/ 2023-10-08T00:11:49.136 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.99:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T00:11:49.136 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T00:11:49.136 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T00:11:49.136 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T00:11:49.136 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.99:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T00:11:49.136 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.99:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T00:11:49.137 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=06962cdb-9a6f-417f-82fd-9e4e3dedbccf" will pass to kernel 2023-10-08T00:11:49.138 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T00:11:49.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:49.138 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-08T00:11:49.203 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:11:49.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.318+0000 7f0133d7e700 1 Processor -- start 2023-10-08T00:11:49.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.318+0000 7f0133d7e700 1 -- start start 2023-10-08T00:11:49.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.319+0000 7f0133d7e700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f012c157870 0x7f012c15b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:49.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.319+0000 7f0133d7e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f012c15a620 0x7f012c15d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:49.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.319+0000 7f0133d7e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f012c15cbb0 con 0x7f012c15d580 2023-10-08T00:11:49.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.319+0000 7f0133d7e700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f012c15bfa0 con 0x7f012c157870 2023-10-08T00:11:49.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.319+0000 7f0133d7e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f012c15c280 con 0x7f012c15a620 2023-10-08T00:11:49.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.319+0000 7f0131b1a700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f012c157870 0x7f012c15b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:49.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.319+0000 7f0131b1a700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f012c157870 0x7f012c15b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50734/0 (socket says 172.21.15.99:50734) 2023-10-08T00:11:49.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.319+0000 7f0131b1a700 1 -- 172.21.15.99:0/3498145937 learned_addr learned my addr 172.21.15.99:0/3498145937 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:49.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.319+0000 7f0131319700 1 --2- 172.21.15.99:0/3498145937 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f012c15a620 0x7f012c15d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:49.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.320+0000 7f0131b1a700 1 -- 172.21.15.99:0/3498145937 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f012c15a620 msgr2=0x7f012c15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:49.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.320+0000 7f0131b1a700 1 --2- 172.21.15.99:0/3498145937 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f012c15a620 0x7f012c15d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:49.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.320+0000 7f0131b1a700 1 -- 172.21.15.99:0/3498145937 >> v1:172.21.15.191:6789/0 conn(0x7f012c15d580 legacy=0x7f012c158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:49.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.320+0000 7f0130b18700 1 -- 172.21.15.99:0/3498145937 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1144838037 0 0) 0x7f012c15cbb0 con 0x7f012c15d580 2023-10-08T00:11:49.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.320+0000 7f0131b1a700 1 -- 172.21.15.99:0/3498145937 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f012c15c560 con 0x7f012c157870 2023-10-08T00:11:49.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.320+0000 7f0131b1a700 1 --2- 172.21.15.99:0/3498145937 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f012c157870 0x7f012c15b6a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f01280099b0 tx=0x7f0128009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6c723a7bcd315d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:49.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.320+0000 7f0130b18700 1 -- 172.21.15.99:0/3498145937 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0128009ea0 con 0x7f012c157870 2023-10-08T00:11:49.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.320+0000 7f0130b18700 1 -- 172.21.15.99:0/3498145937 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f012800b860 con 0x7f012c157870 2023-10-08T00:11:49.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.320+0000 7f0130b18700 1 -- 172.21.15.99:0/3498145937 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012800bc00 con 0x7f012c157870 2023-10-08T00:11:49.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.320+0000 7f0133d7e700 1 -- 172.21.15.99:0/3498145937 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f012c157870 msgr2=0x7f012c15b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:49.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.320+0000 7f0133d7e700 1 --2- 172.21.15.99:0/3498145937 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f012c157870 0x7f012c15b6a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f01280099b0 tx=0x7f0128009250 comp rx=0 tx=0).stop 2023-10-08T00:11:49.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.320+0000 7f0133d7e700 1 -- 172.21.15.99:0/3498145937 shutdown_connections 2023-10-08T00:11:49.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.320+0000 7f0133d7e700 1 --2- 172.21.15.99:0/3498145937 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f012c157870 0x7f012c15b6a0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:49.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.320+0000 7f0133d7e700 1 --2- 172.21.15.99:0/3498145937 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f012c15a620 0x7f012c15d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:49.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.320+0000 7f0133d7e700 1 -- 172.21.15.99:0/3498145937 >> 172.21.15.99:0/3498145937 conn(0x7f012c068ce0 msgr2=0x7f012c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:49.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.321+0000 7f0133d7e700 1 -- 172.21.15.99:0/3498145937 shutdown_connections 2023-10-08T00:11:49.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.321+0000 7f0133d7e700 1 -- 172.21.15.99:0/3498145937 wait complete. 2023-10-08T00:11:49.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.322+0000 7f0133d7e700 1 Processor -- start 2023-10-08T00:11:49.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.322+0000 7f0133d7e700 1 -- start start 2023-10-08T00:11:49.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.322+0000 7f0133d7e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f012c157870 0x7f012c139e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:49.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.322+0000 7f0133d7e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f012c15a620 0x7f012c1505b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:49.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.322+0000 7f0133d7e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f012c15d580 0x7f012c150af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:49.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.322+0000 7f0133d7e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012c151030 con 0x7f012c15a620 2023-10-08T00:11:49.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.322+0000 7f0133d7e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012c0b1000 con 0x7f012c15d580 2023-10-08T00:11:49.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.322+0000 7f0133d7e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f012c0b12a0 con 0x7f012c157870 2023-10-08T00:11:49.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.322+0000 7f0131319700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f012c15a620 0x7f012c1505b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:49.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.322+0000 7f0131319700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f012c15a620 0x7f012c1505b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50750/0 (socket says 172.21.15.99:50750) 2023-10-08T00:11:49.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.322+0000 7f0131319700 1 -- 172.21.15.99:0/2793706072 learned_addr learned my addr 172.21.15.99:0/2793706072 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:49.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.322+0000 7f0131b1a700 1 --2- 172.21.15.99:0/2793706072 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f012c157870 0x7f012c139e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:49.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.322+0000 7f013231b700 1 --2- 172.21.15.99:0/2793706072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f012c15d580 0x7f012c150af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:49.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.323+0000 7f0131319700 1 -- 172.21.15.99:0/2793706072 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f012c157870 msgr2=0x7f012c139e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:49.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.323+0000 7f0131319700 1 --2- 172.21.15.99:0/2793706072 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f012c157870 0x7f012c139e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:49.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.323+0000 7f0131319700 1 -- 172.21.15.99:0/2793706072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f012c15d580 msgr2=0x7f012c150af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:49.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.323+0000 7f0131319700 1 --2- 172.21.15.99:0/2793706072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f012c15d580 0x7f012c150af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:49.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.323+0000 7f0131319700 1 -- 172.21.15.99:0/2793706072 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01280094f0 con 0x7f012c15a620 2023-10-08T00:11:49.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.323+0000 7f0131319700 1 --2- 172.21.15.99:0/2793706072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f012c15a620 0x7f012c1505b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f011c00ec50 tx=0x7f011c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:49.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.323+0000 7f0121ffb700 1 -- 172.21.15.99:0/2793706072 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011c009690 con 0x7f012c15a620 2023-10-08T00:11:49.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.323+0000 7f0121ffb700 1 -- 172.21.15.99:0/2793706072 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f011c00c850 con 0x7f012c15a620 2023-10-08T00:11:49.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.323+0000 7f0133d7e700 1 -- 172.21.15.99:0/2793706072 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f012c0b1580 con 0x7f012c15a620 2023-10-08T00:11:49.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.323+0000 7f0121ffb700 1 -- 172.21.15.99:0/2793706072 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011c00cc10 con 0x7f012c15a620 2023-10-08T00:11:49.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.323+0000 7f0133d7e700 1 -- 172.21.15.99:0/2793706072 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f012c0b1b40 con 0x7f012c15a620 2023-10-08T00:11:49.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.324+0000 7f0133d7e700 1 -- 172.21.15.99:0/2793706072 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f012c04c3c0 con 0x7f012c15a620 2023-10-08T00:11:49.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.325+0000 7f0121ffb700 1 -- 172.21.15.99:0/2793706072 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f011c013470 con 0x7f012c15a620 2023-10-08T00:11:49.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.325+0000 7f0121ffb700 1 --2- 172.21.15.99:0/2793706072 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0110066f30 0x7f01100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:49.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.325+0000 7f0121ffb700 1 -- 172.21.15.99:0/2793706072 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f011c0846f0 con 0x7f012c15a620 2023-10-08T00:11:49.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.325+0000 7f0131b1a700 1 --2- 172.21.15.99:0/2793706072 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0110066f30 0x7f01100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:49.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.326+0000 7f0131b1a700 1 --2- 172.21.15.99:0/2793706072 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0110066f30 0x7f01100693e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f01280099b0 tx=0x7f0128017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:49.333 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.330+0000 7f0121ffb700 1 -- 172.21.15.99:0/2793706072 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f011c051960 con 0x7f012c15a620 2023-10-08T00:11:49.552 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.550+0000 7f0133d7e700 1 -- 172.21.15.99:0/2793706072 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f012c15c840 con 0x7f012c15a620 2023-10-08T00:11:49.553 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.551+0000 7f0121ffb700 1 -- 172.21.15.99:0/2793706072 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4600 (secure 0 0 0) 0x7f011c054f80 con 0x7f012c15a620 2023-10-08T00:11:49.553 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:11:49.553 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4852,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/824084995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":824084995},{"type":"v1","addr":"172.21.15.191:6837","nonce":824084995}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4855,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/3349126920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3349126920},{"type":"v1","addr":"172.21.15.191:6836","nonce":3349126920}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4899,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/1004180438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1004180438},{"type":"v1","addr":"172.21.15.99:6834","nonce":1004180438}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:11:40.672879+0000","modified":"2023-10-08T00:11:44.314526+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4902},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4902":{"gid":4902,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/4189095717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":4189095717},{"type":"v1","addr":"172.21.15.99:6835","nonce":4189095717}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-08T00:11:49.555 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.553+0000 7f0133d7e700 1 -- 172.21.15.99:0/2793706072 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0110066f30 msgr2=0x7f01100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:49.555 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.553+0000 7f0133d7e700 1 --2- 172.21.15.99:0/2793706072 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0110066f30 0x7f01100693e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f01280099b0 tx=0x7f0128017000 comp rx=0 tx=0).stop 2023-10-08T00:11:49.555 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.553+0000 7f0133d7e700 1 -- 172.21.15.99:0/2793706072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f012c15a620 msgr2=0x7f012c1505b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:49.555 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.553+0000 7f0133d7e700 1 --2- 172.21.15.99:0/2793706072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f012c15a620 0x7f012c1505b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f011c00ec50 tx=0x7f011c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:49.555 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.553+0000 7f0133d7e700 1 -- 172.21.15.99:0/2793706072 shutdown_connections 2023-10-08T00:11:49.556 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.553+0000 7f0133d7e700 1 --2- 172.21.15.99:0/2793706072 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0110066f30 0x7f01100693e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:49.556 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.553+0000 7f0133d7e700 1 --2- 172.21.15.99:0/2793706072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f012c15d580 0x7f012c150af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:49.556 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.553+0000 7f0133d7e700 1 --2- 172.21.15.99:0/2793706072 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f012c157870 0x7f012c139e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:49.556 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.553+0000 7f0133d7e700 1 --2- 172.21.15.99:0/2793706072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f012c15a620 0x7f012c1505b0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:49.556 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.553+0000 7f0133d7e700 1 -- 172.21.15.99:0/2793706072 >> 172.21.15.99:0/2793706072 conn(0x7f012c068ce0 msgr2=0x7f012c15e330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:49.556 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.553+0000 7f0133d7e700 1 -- 172.21.15.99:0/2793706072 shutdown_connections 2023-10-08T00:11:49.556 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.553+0000 7f0133d7e700 1 -- 172.21.15.99:0/2793706072 wait complete. 2023-10-08T00:11:49.556 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 17 2023-10-08T00:11:49.567 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-10-08T00:11:49.699 INFO:teuthology.orchestra.run.smithi099.stdout:{"cluster_fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","whoami":0,"id":4902,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":8.3966180930000007,"mdsmap_epoch":17,"osdmap_epoch":31,"osdmap_epoch_barrier":31,"uptime":16.278060391} 2023-10-08T00:11:49.699 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", "fs_name": "cephfs", "id": 4902, "mdsmap_epoch": 17, "osdmap_epoch": 31, "osdmap_epoch_barrier": 31, "rank_uptime": 8.396618093, "state": "up:active", "uptime": 16.278060391, "want_state": "up:active", "whoami": 0 } 2023-10-08T00:11:49.700 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph df --format=json-pretty 2023-10-08T00:11:49.813 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.811+0000 7f89ea632700 1 Processor -- start 2023-10-08T00:11:49.813 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.811+0000 7f89ea632700 1 -- start start 2023-10-08T00:11:49.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.812+0000 7f89ea632700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f89e40cdab0 0x7f89e415d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:49.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.812+0000 7f89ea632700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89e40cb7c0 con 0x7f89e40cd6e0 2023-10-08T00:11:49.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.812+0000 7f89ea632700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89e415e290 con 0x7f89e415dda0 2023-10-08T00:11:49.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.812+0000 7f89ea632700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89e415e850 con 0x7f89e40cdab0 2023-10-08T00:11:49.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.812+0000 7f89e8e2f700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f89e40cdab0 0x7f89e415d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:49.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.812+0000 7f89e9630700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f89e40cd6e0 0x7f89e40cbc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:54230/0 (socket says 172.21.15.99:54230) 2023-10-08T00:11:49.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.812+0000 7f89e9630700 1 -- 172.21.15.99:0/189706691 learned_addr learned my addr 172.21.15.99:0/189706691 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:49.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.812+0000 7f89e8e2f700 1 -- 172.21.15.99:0/189706691 >> v1:172.21.15.191:6789/0 conn(0x7f89e415dda0 legacy=0x7f89e415cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:49.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.812+0000 7f89e8e2f700 1 -- 172.21.15.99:0/189706691 >> v1:172.21.15.99:6789/0 conn(0x7f89e40cd6e0 legacy=0x7f89e40cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:49.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.812+0000 7f89e8e2f700 1 -- 172.21.15.99:0/189706691 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e415e570 con 0x7f89e40cdab0 2023-10-08T00:11:49.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.812+0000 7f89dbfff700 1 -- 172.21.15.99:0/189706691 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1067368221 0 0) 0x7f89e40cb7c0 con 0x7f89e40cd6e0 2023-10-08T00:11:49.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.813+0000 7f89e8e2f700 1 --2- 172.21.15.99:0/189706691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89e40cdab0 0x7f89e415d860 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f89d40098e0 tx=0x7f89d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0cdb0053c8b292c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:49.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.813+0000 7f89dbfff700 1 -- 172.21.15.99:0/189706691 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d4009e80 con 0x7f89e40cdab0 2023-10-08T00:11:49.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.813+0000 7f89dbfff700 1 -- 172.21.15.99:0/189706691 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89d400b860 con 0x7f89e40cdab0 2023-10-08T00:11:49.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.813+0000 7f89dbfff700 1 -- 172.21.15.99:0/189706691 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d400bbe0 con 0x7f89e40cdab0 2023-10-08T00:11:49.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.813+0000 7f89ea632700 1 -- 172.21.15.99:0/189706691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89e40cdab0 msgr2=0x7f89e415d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:49.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.813+0000 7f89ea632700 1 --2- 172.21.15.99:0/189706691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89e40cdab0 0x7f89e415d860 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f89d40098e0 tx=0x7f89d40091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:49.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.813+0000 7f89ea632700 1 -- 172.21.15.99:0/189706691 shutdown_connections 2023-10-08T00:11:49.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.813+0000 7f89ea632700 1 --2- 172.21.15.99:0/189706691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89e40cdab0 0x7f89e415d860 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:49.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.813+0000 7f89ea632700 1 -- 172.21.15.99:0/189706691 >> 172.21.15.99:0/189706691 conn(0x7f89e4067660 msgr2=0x7f89e4067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:49.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.813+0000 7f89ea632700 1 -- 172.21.15.99:0/189706691 shutdown_connections 2023-10-08T00:11:49.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.813+0000 7f89ea632700 1 -- 172.21.15.99:0/189706691 wait complete. 2023-10-08T00:11:49.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.814+0000 7f89ea632700 1 Processor -- start 2023-10-08T00:11:49.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.815+0000 7f89ea632700 1 -- start start 2023-10-08T00:11:49.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.815+0000 7f89ea632700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89e40cd6e0 0x7f89e40b26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:49.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.815+0000 7f89ea632700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89e40cdab0 0x7f89e416ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:49.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.815+0000 7f89ea632700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89e415dda0 0x7f89e416e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:49.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.815+0000 7f89ea632700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e40cc360 con 0x7f89e415dda0 2023-10-08T00:11:49.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.815+0000 7f89ea632700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e4153de0 con 0x7f89e40cdab0 2023-10-08T00:11:49.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.815+0000 7f89ea632700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e416e960 con 0x7f89e40cd6e0 2023-10-08T00:11:49.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.815+0000 7f89e9e31700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89e415dda0 0x7f89e416e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:49.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.816+0000 7f89e9e31700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89e415dda0 0x7f89e416e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50756/0 (socket says 172.21.15.99:50756) 2023-10-08T00:11:49.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.816+0000 7f89e9e31700 1 -- 172.21.15.99:0/3955614799 learned_addr learned my addr 172.21.15.99:0/3955614799 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:49.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.816+0000 7f89e9630700 1 --2- 172.21.15.99:0/3955614799 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89e40cd6e0 0x7f89e40b26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:49.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.816+0000 7f89e8e2f700 1 --2- 172.21.15.99:0/3955614799 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89e40cdab0 0x7f89e416ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:49.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.816+0000 7f89e9e31700 1 -- 172.21.15.99:0/3955614799 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89e40cd6e0 msgr2=0x7f89e40b26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:49.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.816+0000 7f89e9e31700 1 --2- 172.21.15.99:0/3955614799 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89e40cd6e0 0x7f89e40b26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:49.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.816+0000 7f89e9e31700 1 -- 172.21.15.99:0/3955614799 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89e40cdab0 msgr2=0x7f89e416ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:49.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.816+0000 7f89e9e31700 1 --2- 172.21.15.99:0/3955614799 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89e40cdab0 0x7f89e416ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:49.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.816+0000 7f89e9e31700 1 -- 172.21.15.99:0/3955614799 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d4009420 con 0x7f89e415dda0 2023-10-08T00:11:49.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.816+0000 7f89e9e31700 1 --2- 172.21.15.99:0/3955614799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89e415dda0 0x7f89e416e300 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f89dc00c810 tx=0x7f89dc001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:49.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.816+0000 7f89d97fa700 1 -- 172.21.15.99:0/3955614799 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89dc00a650 con 0x7f89e415dda0 2023-10-08T00:11:49.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.816+0000 7f89d97fa700 1 -- 172.21.15.99:0/3955614799 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89dc00a7d0 con 0x7f89e415dda0 2023-10-08T00:11:49.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.816+0000 7f89ea632700 1 -- 172.21.15.99:0/3955614799 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89e416ec40 con 0x7f89e415dda0 2023-10-08T00:11:49.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.816+0000 7f89d97fa700 1 -- 172.21.15.99:0/3955614799 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89dc00ab90 con 0x7f89e415dda0 2023-10-08T00:11:49.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.816+0000 7f89ea632700 1 -- 172.21.15.99:0/3955614799 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89e40c80a0 con 0x7f89e415dda0 2023-10-08T00:11:49.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.817+0000 7f89ea632700 1 -- 172.21.15.99:0/3955614799 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89e4089280 con 0x7f89e415dda0 2023-10-08T00:11:49.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.817+0000 7f89d97fa700 1 -- 172.21.15.99:0/3955614799 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f89dc003030 con 0x7f89e415dda0 2023-10-08T00:11:49.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.818+0000 7f89d97fa700 1 --2- 172.21.15.99:0/3955614799 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f89c8066f30 0x7f89c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:49.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.818+0000 7f89d97fa700 1 -- 172.21.15.99:0/3955614799 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f89dc084540 con 0x7f89e415dda0 2023-10-08T00:11:49.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.818+0000 7f89e9630700 1 --2- 172.21.15.99:0/3955614799 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f89c8066f30 0x7f89c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:49.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.818+0000 7f89e9630700 1 --2- 172.21.15.99:0/3955614799 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f89c8066f30 0x7f89c80693e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f89e0000c00 tx=0x7f89e0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:49.829 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:49.823+0000 7f89d97fa700 1 -- 172.21.15.99:0/3955614799 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f89dc0517b0 con 0x7f89e415dda0 2023-10-08T00:11:50.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.085+0000 7f89ea632700 1 -- 172.21.15.99:0/3955614799 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "df", "format": "json-pretty"} v 0) v1 -- 0x7f89e406b400 con 0x7f89e415dda0 2023-10-08T00:11:50.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.086+0000 7f89d97fa700 1 -- 172.21.15.99:0/3955614799 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "df", "format": "json-pretty"}]=0 v0) v1 ==== 75+0+1620 (secure 0 0 0) 0x7f89dc054dd0 con 0x7f89e415dda0 2023-10-08T00:11:50.087 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:11:50.087 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:11:50.088 INFO:teuthology.orchestra.run.smithi099.stdout: "stats": { 2023-10-08T00:11:50.088 INFO:teuthology.orchestra.run.smithi099.stdout: "total_bytes": 1600000000, 2023-10-08T00:11:50.088 INFO:teuthology.orchestra.run.smithi099.stdout: "total_avail_bytes": 1597746156, 2023-10-08T00:11:50.088 INFO:teuthology.orchestra.run.smithi099.stdout: "total_used_bytes": 2253844, 2023-10-08T00:11:50.088 INFO:teuthology.orchestra.run.smithi099.stdout: "total_used_raw_bytes": 2253844, 2023-10-08T00:11:50.088 INFO:teuthology.orchestra.run.smithi099.stdout: "total_used_raw_ratio": 0.0014086525188758969, 2023-10-08T00:11:50.088 INFO:teuthology.orchestra.run.smithi099.stdout: "num_osds": 8, 2023-10-08T00:11:50.089 INFO:teuthology.orchestra.run.smithi099.stdout: "num_per_pool_osds": 0, 2023-10-08T00:11:50.089 INFO:teuthology.orchestra.run.smithi099.stdout: "num_per_pool_omap_osds": 0 2023-10-08T00:11:50.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:11:50.089 INFO:teuthology.orchestra.run.smithi099.stdout: "stats_by_class": { 2023-10-08T00:11:50.089 INFO:teuthology.orchestra.run.smithi099.stdout: "hdd": { 2023-10-08T00:11:50.089 INFO:teuthology.orchestra.run.smithi099.stdout: "total_bytes": 1600000000, 2023-10-08T00:11:50.089 INFO:teuthology.orchestra.run.smithi099.stdout: "total_avail_bytes": 1597746156, 2023-10-08T00:11:50.090 INFO:teuthology.orchestra.run.smithi099.stdout: "total_used_bytes": 2253844, 2023-10-08T00:11:50.090 INFO:teuthology.orchestra.run.smithi099.stdout: "total_used_raw_bytes": 2253844, 2023-10-08T00:11:50.090 INFO:teuthology.orchestra.run.smithi099.stdout: "total_used_raw_ratio": 0.0014086525188758969 2023-10-08T00:11:50.090 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:11:50.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:11:50.090 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2023-10-08T00:11:50.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:11:50.091 INFO:teuthology.orchestra.run.smithi099.stdout: "name": ".mgr", 2023-10-08T00:11:50.091 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 1, 2023-10-08T00:11:50.091 INFO:teuthology.orchestra.run.smithi099.stdout: "stats": { 2023-10-08T00:11:50.091 INFO:teuthology.orchestra.run.smithi099.stdout: "stored": 590368, 2023-10-08T00:11:50.091 INFO:teuthology.orchestra.run.smithi099.stdout: "objects": 2, 2023-10-08T00:11:50.091 INFO:teuthology.orchestra.run.smithi099.stdout: "kb_used": 577, 2023-10-08T00:11:50.091 INFO:teuthology.orchestra.run.smithi099.stdout: "bytes_used": 590368, 2023-10-08T00:11:50.092 INFO:teuthology.orchestra.run.smithi099.stdout: "percent_used": 0.00052957900334149599, 2023-10-08T00:11:50.092 INFO:teuthology.orchestra.run.smithi099.stdout: "max_avail": 557098560 2023-10-08T00:11:50.092 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:11:50.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:11:50.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:11:50.092 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cephfs_metadata", 2023-10-08T00:11:50.092 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 4, 2023-10-08T00:11:50.093 INFO:teuthology.orchestra.run.smithi099.stdout: "stats": { 2023-10-08T00:11:50.093 INFO:teuthology.orchestra.run.smithi099.stdout: "stored": 2306, 2023-10-08T00:11:50.093 INFO:teuthology.orchestra.run.smithi099.stdout: "objects": 21, 2023-10-08T00:11:50.093 INFO:teuthology.orchestra.run.smithi099.stdout: "kb_used": 3, 2023-10-08T00:11:50.093 INFO:teuthology.orchestra.run.smithi099.stdout: "bytes_used": 2306, 2023-10-08T00:11:50.093 INFO:teuthology.orchestra.run.smithi099.stdout: "percent_used": 2.0696475075965282e-06, 2023-10-08T00:11:50.093 INFO:teuthology.orchestra.run.smithi099.stdout: "max_avail": 557098560 2023-10-08T00:11:50.094 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:11:50.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:11:50.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:11:50.094 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cephfs_data", 2023-10-08T00:11:50.094 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 5, 2023-10-08T00:11:50.094 INFO:teuthology.orchestra.run.smithi099.stdout: "stats": { 2023-10-08T00:11:50.094 INFO:teuthology.orchestra.run.smithi099.stdout: "stored": 0, 2023-10-08T00:11:50.095 INFO:teuthology.orchestra.run.smithi099.stdout: "objects": 0, 2023-10-08T00:11:50.095 INFO:teuthology.orchestra.run.smithi099.stdout: "kb_used": 0, 2023-10-08T00:11:50.095 INFO:teuthology.orchestra.run.smithi099.stdout: "bytes_used": 0, 2023-10-08T00:11:50.095 INFO:teuthology.orchestra.run.smithi099.stdout: "percent_used": 0, 2023-10-08T00:11:50.095 INFO:teuthology.orchestra.run.smithi099.stdout: "max_avail": 557098560 2023-10-08T00:11:50.095 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:11:50.096 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:11:50.096 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:11:50.096 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:11:50.097 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.087+0000 7f89ea632700 1 -- 172.21.15.99:0/3955614799 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f89c8066f30 msgr2=0x7f89c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:50.098 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.087+0000 7f89ea632700 1 --2- 172.21.15.99:0/3955614799 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f89c8066f30 0x7f89c80693e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f89e0000c00 tx=0x7f89e0009040 comp rx=0 tx=0).stop 2023-10-08T00:11:50.098 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.087+0000 7f89ea632700 1 -- 172.21.15.99:0/3955614799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89e415dda0 msgr2=0x7f89e416e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:50.098 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.087+0000 7f89ea632700 1 --2- 172.21.15.99:0/3955614799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89e415dda0 0x7f89e416e300 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f89dc00c810 tx=0x7f89dc001650 comp rx=0 tx=0).stop 2023-10-08T00:11:50.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.088+0000 7f89ea632700 1 -- 172.21.15.99:0/3955614799 shutdown_connections 2023-10-08T00:11:50.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.088+0000 7f89ea632700 1 --2- 172.21.15.99:0/3955614799 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f89c8066f30 0x7f89c80693e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:50.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.088+0000 7f89ea632700 1 --2- 172.21.15.99:0/3955614799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89e415dda0 0x7f89e416e300 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:50.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.088+0000 7f89ea632700 1 --2- 172.21.15.99:0/3955614799 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89e40cd6e0 0x7f89e40b26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:50.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.088+0000 7f89ea632700 1 --2- 172.21.15.99:0/3955614799 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89e40cdab0 0x7f89e416ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:50.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.088+0000 7f89ea632700 1 -- 172.21.15.99:0/3955614799 >> 172.21.15.99:0/3955614799 conn(0x7f89e4067660 msgr2=0x7f89e40c6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:50.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.088+0000 7f89ea632700 1 -- 172.21.15.99:0/3955614799 shutdown_connections 2023-10-08T00:11:50.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.088+0000 7f89ea632700 1 -- 172.21.15.99:0/3955614799 wait complete. 2023-10-08T00:11:50.101 DEBUG:teuthology.orchestra.run.smithi099:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-08T00:11:50.101 DEBUG:teuthology.orchestra.run.smithi099:> import os 2023-10-08T00:11:50.101 DEBUG:teuthology.orchestra.run.smithi099:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2023-10-08T00:11:50.101 DEBUG:teuthology.orchestra.run.smithi099:> os.fchmod(fd, 0O666) 2023-10-08T00:11:50.101 DEBUG:teuthology.orchestra.run.smithi099:> os.fsync(fd) 2023-10-08T00:11:50.101 DEBUG:teuthology.orchestra.run.smithi099:> os.close(fd) 2023-10-08T00:11:50.101 DEBUG:teuthology.orchestra.run.smithi099:> ' 2023-10-08T00:11:50.354 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:11:50.354 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-08T00:11:50.423 INFO:teuthology.orchestra.run.smithi099.stdout:client4996 2023-10-08T00:11:50.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:50.424 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/06962cdb-9a6f-417f-82fd-9e4e3dedbccf.client4996/osdmap') 2023-10-08T00:11:50.499 INFO:teuthology.orchestra.run.smithi099.stdout:epoch 31 barrier 31 flags 0x588000 2023-10-08T00:11:50.499 INFO:teuthology.orchestra.run.smithi099.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-08T00:11:50.499 INFO:teuthology.orchestra.run.smithi099.stdout:pool 4 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-08T00:11:50.499 INFO:teuthology.orchestra.run.smithi099.stdout:pool 5 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-08T00:11:50.499 INFO:teuthology.orchestra.run.smithi099.stdout:osd0 (1)172.21.15.99:6801 100% (exists, up) 100% -1 2023-10-08T00:11:50.499 INFO:teuthology.orchestra.run.smithi099.stdout:osd1 (1)172.21.15.99:6809 100% (exists, up) 100% -1 2023-10-08T00:11:50.499 INFO:teuthology.orchestra.run.smithi099.stdout:osd2 (1)172.21.15.99:6817 100% (exists, up) 100% -1 2023-10-08T00:11:50.499 INFO:teuthology.orchestra.run.smithi099.stdout:osd3 (1)172.21.15.99:6820 100% (exists, up) 100% -1 2023-10-08T00:11:50.500 INFO:teuthology.orchestra.run.smithi099.stdout:osd4 (1)172.21.15.191:6801 100% (exists, up) 100% -1 2023-10-08T00:11:50.500 INFO:teuthology.orchestra.run.smithi099.stdout:osd5 (1)172.21.15.191:6818 100% (exists, up) 100% -1 2023-10-08T00:11:50.500 INFO:teuthology.orchestra.run.smithi099.stdout:osd6 (1)172.21.15.191:6809 100% (exists, up) 100% -1 2023-10-08T00:11:50.500 INFO:teuthology.orchestra.run.smithi099.stdout:osd7 (1)172.21.15.191:6819 100% (exists, up) 100% -1 2023-10-08T00:11:50.501 INFO:teuthology.orchestra.run.smithi099.stderr:1+1 records in 2023-10-08T00:11:50.501 INFO:teuthology.orchestra.run.smithi099.stderr:1+1 records out 2023-10-08T00:11:50.501 INFO:teuthology.orchestra.run.smithi099.stderr:795 bytes copied, 3.543e-05 s, 22.4 MB/s 2023-10-08T00:11:50.501 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set pause 2023-10-08T00:11:50.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.655+0000 7f9fee5e7700 1 Processor -- start 2023-10-08T00:11:50.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.655+0000 7f9fee5e7700 1 -- start start 2023-10-08T00:11:50.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.656+0000 7f9fee5e7700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9fe8157870 0x7f9fe815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:50.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.656+0000 7f9fee5e7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9fe8157c40 0x7f9fe8158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:50.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.656+0000 7f9fee5e7700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fe815cbb0 con 0x7f9fe815a620 2023-10-08T00:11:50.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.656+0000 7f9fee5e7700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fe815bfa0 con 0x7f9fe8157870 2023-10-08T00:11:50.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.656+0000 7f9fee5e7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe815c280 con 0x7f9fe8157c40 2023-10-08T00:11:50.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.656+0000 7f9fe7fff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9fe8157870 0x7f9fe815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:50.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.656+0000 7f9fe77fe700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f9fe815a620 0x7f9fe815d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:54246/0 (socket says 172.21.15.99:54246) 2023-10-08T00:11:50.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.656+0000 7f9fe77fe700 1 -- 172.21.15.99:0/4027005499 learned_addr learned my addr 172.21.15.99:0/4027005499 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:50.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.656+0000 7f9fe7fff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9fe8157870 0x7f9fe815b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50766/0 (socket says 172.21.15.99:50766) 2023-10-08T00:11:50.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.656+0000 7f9fe7fff700 1 -- 172.21.15.99:0/4027005499 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9fe8157c40 msgr2=0x7f9fe8158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:50.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.656+0000 7f9fe7fff700 1 --2- 172.21.15.99:0/4027005499 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9fe8157c40 0x7f9fe8158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:50.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.656+0000 7f9fe7fff700 1 -- 172.21.15.99:0/4027005499 >> v1:172.21.15.99:6789/0 conn(0x7f9fe815a620 legacy=0x7f9fe815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:50.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.656+0000 7f9fe7fff700 1 -- 172.21.15.99:0/4027005499 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe815c560 con 0x7f9fe8157870 2023-10-08T00:11:50.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.656+0000 7f9fe6ffd700 1 -- 172.21.15.99:0/4027005499 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 522070896 0 0) 0x7f9fe815cbb0 con 0x7f9fe815a620 2023-10-08T00:11:50.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.657+0000 7f9fe7fff700 1 --2- 172.21.15.99:0/4027005499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9fe8157870 0x7f9fe815b6a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9fd80098e0 tx=0x7f9fd80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=854f1cd1849fdd58 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:50.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.657+0000 7f9fe6ffd700 1 -- 172.21.15.99:0/4027005499 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd8009e80 con 0x7f9fe8157870 2023-10-08T00:11:50.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.657+0000 7f9fe6ffd700 1 -- 172.21.15.99:0/4027005499 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd800b860 con 0x7f9fe8157870 2023-10-08T00:11:50.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.657+0000 7f9fe6ffd700 1 -- 172.21.15.99:0/4027005499 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd800bbe0 con 0x7f9fe8157870 2023-10-08T00:11:50.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.657+0000 7f9fee5e7700 1 -- 172.21.15.99:0/4027005499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9fe8157870 msgr2=0x7f9fe815b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:50.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.657+0000 7f9fee5e7700 1 --2- 172.21.15.99:0/4027005499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9fe8157870 0x7f9fe815b6a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9fd80098e0 tx=0x7f9fd80091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:50.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.657+0000 7f9fee5e7700 1 -- 172.21.15.99:0/4027005499 shutdown_connections 2023-10-08T00:11:50.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.657+0000 7f9fee5e7700 1 --2- 172.21.15.99:0/4027005499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9fe8157870 0x7f9fe815b6a0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:50.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.657+0000 7f9fee5e7700 1 --2- 172.21.15.99:0/4027005499 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9fe8157c40 0x7f9fe8158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:50.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.657+0000 7f9fee5e7700 1 -- 172.21.15.99:0/4027005499 >> 172.21.15.99:0/4027005499 conn(0x7f9fe8068ce0 msgr2=0x7f9fe80690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:50.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.657+0000 7f9fee5e7700 1 -- 172.21.15.99:0/4027005499 shutdown_connections 2023-10-08T00:11:50.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.657+0000 7f9fee5e7700 1 -- 172.21.15.99:0/4027005499 wait complete. 2023-10-08T00:11:50.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.658+0000 7f9fee5e7700 1 Processor -- start 2023-10-08T00:11:50.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.659+0000 7f9fee5e7700 1 -- start start 2023-10-08T00:11:50.669 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.659+0000 7f9fee5e7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fe8157870 0x7f9fe8150530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:50.669 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.659+0000 7f9fee5e7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9fe8157c40 0x7f9fe814cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:50.669 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.659+0000 7f9fee5e7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9fe815a620 0x7f9fe814d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:50.669 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.659+0000 7f9fee5e7700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe8153010 con 0x7f9fe815a620 2023-10-08T00:11:50.669 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.659+0000 7f9fee5e7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe8152440 con 0x7f9fe8157870 2023-10-08T00:11:50.670 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.659+0000 7f9fee5e7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe81526e0 con 0x7f9fe8157c40 2023-10-08T00:11:50.670 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.659+0000 7f9fecb84700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9fe815a620 0x7f9fe814d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:50.670 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.659+0000 7f9fecb84700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9fe815a620 0x7f9fe814d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50770/0 (socket says 172.21.15.99:50770) 2023-10-08T00:11:50.670 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.659+0000 7f9fecb84700 1 -- 172.21.15.99:0/3142585491 learned_addr learned my addr 172.21.15.99:0/3142585491 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:50.671 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.660+0000 7f9fe77fe700 1 --2- 172.21.15.99:0/3142585491 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9fe8157c40 0x7f9fe814cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:50.671 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.660+0000 7f9fe7fff700 1 --2- 172.21.15.99:0/3142585491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fe8157870 0x7f9fe8150530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:50.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.660+0000 7f9fecb84700 1 -- 172.21.15.99:0/3142585491 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9fe8157c40 msgr2=0x7f9fe814cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:50.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.660+0000 7f9fecb84700 1 --2- 172.21.15.99:0/3142585491 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9fe8157c40 0x7f9fe814cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:50.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.660+0000 7f9fecb84700 1 -- 172.21.15.99:0/3142585491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fe8157870 msgr2=0x7f9fe8150530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:50.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.660+0000 7f9fecb84700 1 --2- 172.21.15.99:0/3142585491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fe8157870 0x7f9fe8150530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:50.673 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.660+0000 7f9fecb84700 1 -- 172.21.15.99:0/3142585491 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd8009420 con 0x7f9fe815a620 2023-10-08T00:11:50.673 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.660+0000 7f9fecb84700 1 --2- 172.21.15.99:0/3142585491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9fe815a620 0x7f9fe814d1a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9fe000bb60 tx=0x7f9fe0009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:50.673 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.660+0000 7f9fcffff700 1 -- 172.21.15.99:0/3142585491 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe000c940 con 0x7f9fe815a620 2023-10-08T00:11:50.673 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.660+0000 7f9fcffff700 1 -- 172.21.15.99:0/3142585491 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fe000cac0 con 0x7f9fe815a620 2023-10-08T00:11:50.674 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.660+0000 7f9fee5e7700 1 -- 172.21.15.99:0/3142585491 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fe81529c0 con 0x7f9fe815a620 2023-10-08T00:11:50.674 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.660+0000 7f9fcffff700 1 -- 172.21.15.99:0/3142585491 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe000ce80 con 0x7f9fe815a620 2023-10-08T00:11:50.674 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.660+0000 7f9fee5e7700 1 -- 172.21.15.99:0/3142585491 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fe8150a70 con 0x7f9fe815a620 2023-10-08T00:11:50.674 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.661+0000 7f9fee5e7700 1 -- 172.21.15.99:0/3142585491 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fe804c3c0 con 0x7f9fe815a620 2023-10-08T00:11:50.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.661+0000 7f9fcffff700 1 -- 172.21.15.99:0/3142585491 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9fe0014620 con 0x7f9fe815a620 2023-10-08T00:11:50.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.662+0000 7f9fcffff700 1 --2- 172.21.15.99:0/3142585491 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9fd4066f30 0x7f9fd40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:50.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.662+0000 7f9fcffff700 1 -- 172.21.15.99:0/3142585491 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9fe0084570 con 0x7f9fe815a620 2023-10-08T00:11:50.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.662+0000 7f9fe7fff700 1 --2- 172.21.15.99:0/3142585491 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9fd4066f30 0x7f9fd40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:50.676 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.663+0000 7f9fe7fff700 1 --2- 172.21.15.99:0/3142585491 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9fd4066f30 0x7f9fd40693e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9fd80091e0 tx=0x7f9fd8017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:50.676 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.666+0000 7f9fcffff700 1 -- 172.21.15.99:0/3142585491 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9fe004c5a0 con 0x7f9fe815a620 2023-10-08T00:11:50.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:50.859+0000 7f9fee5e7700 1 -- 172.21.15.99:0/3142585491 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd set", "key": "pause"} v 0) v1 -- 0x7f9fe815c840 con 0x7f9fe815a620 2023-10-08T00:11:51.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.339+0000 7f9fcffff700 1 -- 172.21.15.99:0/3142585491 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "pause"}]=0 pauserd,pausewr is set v32) v1 ==== 93+0+0 (secure 0 0 0) 0x7f9fe00517e0 con 0x7f9fe815a620 2023-10-08T00:11:51.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.341+0000 7f9fee5e7700 1 -- 172.21.15.99:0/3142585491 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9fd4066f30 msgr2=0x7f9fd40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:51.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.341+0000 7f9fee5e7700 1 --2- 172.21.15.99:0/3142585491 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9fd4066f30 0x7f9fd40693e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9fd80091e0 tx=0x7f9fd8017000 comp rx=0 tx=0).stop 2023-10-08T00:11:51.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.341+0000 7f9fee5e7700 1 -- 172.21.15.99:0/3142585491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9fe815a620 msgr2=0x7f9fe814d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:51.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.341+0000 7f9fee5e7700 1 --2- 172.21.15.99:0/3142585491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9fe815a620 0x7f9fe814d1a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9fe000bb60 tx=0x7f9fe0009810 comp rx=0 tx=0).stop 2023-10-08T00:11:51.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.341+0000 7f9fee5e7700 1 -- 172.21.15.99:0/3142585491 shutdown_connections 2023-10-08T00:11:51.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.341+0000 7f9fee5e7700 1 --2- 172.21.15.99:0/3142585491 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9fd4066f30 0x7f9fd40693e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:51.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.341+0000 7f9fee5e7700 1 --2- 172.21.15.99:0/3142585491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fe8157870 0x7f9fe8150530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:51.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.341+0000 7f9fee5e7700 1 --2- 172.21.15.99:0/3142585491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9fe815a620 0x7f9fe814d1a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:51.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.341+0000 7f9fee5e7700 1 --2- 172.21.15.99:0/3142585491 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9fe8157c40 0x7f9fe814cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:51.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.341+0000 7f9fee5e7700 1 -- 172.21.15.99:0/3142585491 >> 172.21.15.99:0/3142585491 conn(0x7f9fe8068ce0 msgr2=0x7f9fe80b47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:51.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.341+0000 7f9fee5e7700 1 -- 172.21.15.99:0/3142585491 shutdown_connections 2023-10-08T00:11:51.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.341+0000 7f9fee5e7700 1 -- 172.21.15.99:0/3142585491 wait complete. 2023-10-08T00:11:51.343 INFO:teuthology.orchestra.run.smithi099.stderr:pauserd,pausewr is set 2023-10-08T00:11:51.354 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset pause 2023-10-08T00:11:51.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.468+0000 7fd0f79c6700 1 Processor -- start 2023-10-08T00:11:51.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.469+0000 7fd0f79c6700 1 -- start start 2023-10-08T00:11:51.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.469+0000 7fd0f79c6700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd0f015a330 0x7fd0f0156940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:51.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.469+0000 7fd0f79c6700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd0f006da90 0x7fd0f0156e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:51.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.469+0000 7fd0f79c6700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0f0157c30 con 0x7fd0f0156570 2023-10-08T00:11:51.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.469+0000 7fd0f79c6700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0f01584f0 con 0x7fd0f006da90 2023-10-08T00:11:51.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.469+0000 7fd0f79c6700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0f01573c0 con 0x7fd0f015a330 2023-10-08T00:11:51.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.470+0000 7fd0f5f63700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd0f006da90 0x7fd0f0156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:51.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.470+0000 7fd0f5f63700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd0f006da90 0x7fd0f0156e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50774/0 (socket says 172.21.15.99:50774) 2023-10-08T00:11:51.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.470+0000 7fd0f5f63700 1 -- 172.21.15.99:0/1775179691 learned_addr learned my addr 172.21.15.99:0/1775179691 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:51.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.470+0000 7fd0f4f61700 1 --2- 172.21.15.99:0/1775179691 >> v2:172.21.15.191:3300/0 conn(0x7fd0f015a330 0x7fd0f0156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:51.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.470+0000 7fd0f5f63700 1 -- 172.21.15.99:0/1775179691 >> v2:172.21.15.191:3300/0 conn(0x7fd0f015a330 msgr2=0x7fd0f0156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:51.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.470+0000 7fd0f5f63700 1 --2- 172.21.15.99:0/1775179691 >> v2:172.21.15.191:3300/0 conn(0x7fd0f015a330 0x7fd0f0156940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:51.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.470+0000 7fd0f5f63700 1 -- 172.21.15.99:0/1775179691 >> v1:172.21.15.191:6789/0 conn(0x7fd0f0156570 legacy=0x7fd0f006d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:51.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.470+0000 7fd0f5f63700 1 -- 172.21.15.99:0/1775179691 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f0157670 con 0x7fd0f006da90 2023-10-08T00:11:51.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.470+0000 7fd0f5f63700 1 --2- 172.21.15.99:0/1775179691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0f006da90 0x7fd0f0156e80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd0f006cd60 tx=0x7fd0e8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69fe59dc4b83428f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:51.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.470+0000 7fd0e7fff700 1 -- 172.21.15.99:0/1775179691 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e800e070 con 0x7fd0f006da90 2023-10-08T00:11:51.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.470+0000 7fd0e7fff700 1 -- 172.21.15.99:0/1775179691 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0e800ba10 con 0x7fd0f006da90 2023-10-08T00:11:51.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.470+0000 7fd0e7fff700 1 -- 172.21.15.99:0/1775179691 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e80092e0 con 0x7fd0f006da90 2023-10-08T00:11:51.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.471+0000 7fd0f79c6700 1 -- 172.21.15.99:0/1775179691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0f006da90 msgr2=0x7fd0f0156e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:51.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.471+0000 7fd0f79c6700 1 --2- 172.21.15.99:0/1775179691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0f006da90 0x7fd0f0156e80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd0f006cd60 tx=0x7fd0e8009840 comp rx=0 tx=0).stop 2023-10-08T00:11:51.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.471+0000 7fd0f79c6700 1 -- 172.21.15.99:0/1775179691 shutdown_connections 2023-10-08T00:11:51.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.471+0000 7fd0f79c6700 1 --2- 172.21.15.99:0/1775179691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0f006da90 0x7fd0f0156e80 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:51.474 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.471+0000 7fd0f79c6700 1 --2- 172.21.15.99:0/1775179691 >> v2:172.21.15.191:3300/0 conn(0x7fd0f015a330 0x7fd0f0156940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:51.474 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.471+0000 7fd0f79c6700 1 -- 172.21.15.99:0/1775179691 >> 172.21.15.99:0/1775179691 conn(0x7fd0f0067660 msgr2=0x7fd0f0067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:51.474 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.471+0000 7fd0f79c6700 1 -- 172.21.15.99:0/1775179691 shutdown_connections 2023-10-08T00:11:51.474 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.471+0000 7fd0f79c6700 1 -- 172.21.15.99:0/1775179691 wait complete. 2023-10-08T00:11:51.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.472+0000 7fd0f79c6700 1 Processor -- start 2023-10-08T00:11:51.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.472+0000 7fd0f79c6700 1 -- start start 2023-10-08T00:11:51.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.473+0000 7fd0f79c6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0f006da90 0x7fd0f014f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:51.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.473+0000 7fd0f79c6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd0f0156570 0x7fd0f014fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:51.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.473+0000 7fd0f79c6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0f015a330 0x7fd0f00c5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:51.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.473+0000 7fd0f79c6700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f00c0e00 con 0x7fd0f015a330 2023-10-08T00:11:51.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.473+0000 7fd0f79c6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f014fba0 con 0x7fd0f006da90 2023-10-08T00:11:51.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.473+0000 7fd0f79c6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f00b1d10 con 0x7fd0f0156570 2023-10-08T00:11:51.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.473+0000 7fd0f5f63700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0f015a330 0x7fd0f00c5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:51.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.473+0000 7fd0f5f63700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0f015a330 0x7fd0f00c5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50782/0 (socket says 172.21.15.99:50782) 2023-10-08T00:11:51.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.473+0000 7fd0f5f63700 1 -- 172.21.15.99:0/3900004917 learned_addr learned my addr 172.21.15.99:0/3900004917 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:51.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.473+0000 7fd0f5762700 1 --2- 172.21.15.99:0/3900004917 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0f006da90 0x7fd0f014f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:51.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.473+0000 7fd0f4f61700 1 --2- 172.21.15.99:0/3900004917 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd0f0156570 0x7fd0f014fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:51.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.473+0000 7fd0f5f63700 1 -- 172.21.15.99:0/3900004917 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd0f0156570 msgr2=0x7fd0f014fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:51.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.473+0000 7fd0f5f63700 1 --2- 172.21.15.99:0/3900004917 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd0f0156570 0x7fd0f014fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:51.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.473+0000 7fd0f5f63700 1 -- 172.21.15.99:0/3900004917 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0f006da90 msgr2=0x7fd0f014f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:51.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.473+0000 7fd0f5f63700 1 --2- 172.21.15.99:0/3900004917 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0f006da90 0x7fd0f014f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:51.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.473+0000 7fd0f5f63700 1 -- 172.21.15.99:0/3900004917 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e8009ae0 con 0x7fd0f015a330 2023-10-08T00:11:51.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.473+0000 7fd0f5f63700 1 --2- 172.21.15.99:0/3900004917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0f015a330 0x7fd0f00c5d00 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd0f006ba00 tx=0x7fd0e8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:51.479 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.474+0000 7fd0e57fa700 1 -- 172.21.15.99:0/3900004917 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e8009840 con 0x7fd0f015a330 2023-10-08T00:11:51.479 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.474+0000 7fd0e57fa700 1 -- 172.21.15.99:0/3900004917 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0e80092e0 con 0x7fd0f015a330 2023-10-08T00:11:51.479 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.474+0000 7fd0f79c6700 1 -- 172.21.15.99:0/3900004917 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0f00b1ff0 con 0x7fd0f015a330 2023-10-08T00:11:51.479 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.474+0000 7fd0e57fa700 1 -- 172.21.15.99:0/3900004917 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e801d650 con 0x7fd0f015a330 2023-10-08T00:11:51.479 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.474+0000 7fd0f79c6700 1 -- 172.21.15.99:0/3900004917 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0f00c1330 con 0x7fd0f015a330 2023-10-08T00:11:51.479 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.474+0000 7fd0f79c6700 1 -- 172.21.15.99:0/3900004917 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0f00c1550 con 0x7fd0f015a330 2023-10-08T00:11:51.479 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.475+0000 7fd0e57fa700 1 -- 172.21.15.99:0/3900004917 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd0e801da40 con 0x7fd0f015a330 2023-10-08T00:11:51.479 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.475+0000 7fd0e57fa700 1 --2- 172.21.15.99:0/3900004917 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd0d4066f30 0x7fd0d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:51.479 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.475+0000 7fd0e57fa700 1 -- 172.21.15.99:0/3900004917 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd0e8085be0 con 0x7fd0f015a330 2023-10-08T00:11:51.479 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.475+0000 7fd0f5762700 1 --2- 172.21.15.99:0/3900004917 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd0d4066f30 0x7fd0d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:51.479 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.475+0000 7fd0e57fa700 1 -- 172.21.15.99:0/3900004917 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=33+}) v3 -- 0x7fd0d406ea50 con 0x7fd0f015a330 2023-10-08T00:11:51.480 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.476+0000 7fd0f5762700 1 --2- 172.21.15.99:0/3900004917 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd0d4066f30 0x7fd0d40693e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec003870 tx=0x7fd0ec009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:51.480 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.479+0000 7fd0e57fa700 1 -- 172.21.15.99:0/3900004917 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd0e8052e50 con 0x7fd0f015a330 2023-10-08T00:11:51.676 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:51.676+0000 7fd0f79c6700 1 -- 172.21.15.99:0/3900004917 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd unset", "key": "pause"} v 0) v1 -- 0x7fd0f004c3c0 con 0x7fd0f015a330 2023-10-08T00:11:52.352 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.351+0000 7fd0e57fa700 1 -- 172.21.15.99:0/3900004917 <== mon.0 v2:172.21.15.99:3300/0 7 ==== osd_map(33..33 src has 1..33) v4 ==== 280+0+0 (secure 0 0 0) 0x7fd0e8057210 con 0x7fd0f015a330 2023-10-08T00:11:52.352 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.351+0000 7fd0e57fa700 1 -- 172.21.15.99:0/3900004917 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=34}) v3 -- 0x7fd0d406f330 con 0x7fd0f015a330 2023-10-08T00:11:52.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.354+0000 7fd0e57fa700 1 -- 172.21.15.99:0/3900004917 <== mon.0 v2:172.21.15.99:3300/0 8 ==== mon_command_ack([{"prefix": "osd unset", "key": "pause"}]=0 pauserd,pausewr is unset v33) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd0e802a030 con 0x7fd0f015a330 2023-10-08T00:11:52.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.356+0000 7fd0f79c6700 1 -- 172.21.15.99:0/3900004917 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd0d4066f30 msgr2=0x7fd0d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:52.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.356+0000 7fd0f79c6700 1 --2- 172.21.15.99:0/3900004917 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd0d4066f30 0x7fd0d40693e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec003870 tx=0x7fd0ec009040 comp rx=0 tx=0).stop 2023-10-08T00:11:52.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.356+0000 7fd0f79c6700 1 -- 172.21.15.99:0/3900004917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0f015a330 msgr2=0x7fd0f00c5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:52.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.356+0000 7fd0f79c6700 1 --2- 172.21.15.99:0/3900004917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0f015a330 0x7fd0f00c5d00 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd0f006ba00 tx=0x7fd0e8017000 comp rx=0 tx=0).stop 2023-10-08T00:11:52.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.356+0000 7fd0f79c6700 1 -- 172.21.15.99:0/3900004917 shutdown_connections 2023-10-08T00:11:52.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.356+0000 7fd0f79c6700 1 --2- 172.21.15.99:0/3900004917 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd0d4066f30 0x7fd0d40693e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:52.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.356+0000 7fd0f79c6700 1 --2- 172.21.15.99:0/3900004917 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0f006da90 0x7fd0f014f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:52.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.356+0000 7fd0f79c6700 1 --2- 172.21.15.99:0/3900004917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0f015a330 0x7fd0f00c5d00 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:52.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.356+0000 7fd0f79c6700 1 --2- 172.21.15.99:0/3900004917 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd0f0156570 0x7fd0f014fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:52.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.356+0000 7fd0f79c6700 1 -- 172.21.15.99:0/3900004917 >> 172.21.15.99:0/3900004917 conn(0x7fd0f0067660 msgr2=0x7fd0f00c92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:52.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.357+0000 7fd0f79c6700 1 -- 172.21.15.99:0/3900004917 shutdown_connections 2023-10-08T00:11:52.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.357+0000 7fd0f79c6700 1 -- 172.21.15.99:0/3900004917 wait complete. 2023-10-08T00:11:52.358 INFO:teuthology.orchestra.run.smithi099.stderr:pauserd,pausewr is unset 2023-10-08T00:11:52.370 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:11:52.485 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.484+0000 7f13190f7700 1 Processor -- start 2023-10-08T00:11:52.485 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.484+0000 7f13190f7700 1 -- start start 2023-10-08T00:11:52.487 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.485+0000 7f13190f7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13141510f0 0x7f131406ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:52.487 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.485+0000 7f13190f7700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1314070700 0x7f1314070b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:52.488 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.485+0000 7f13190f7700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1314071080 con 0x7f13140c9af0 2023-10-08T00:11:52.488 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.485+0000 7f13190f7700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13140ca0b0 con 0x7f1314070700 2023-10-08T00:11:52.488 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.485+0000 7f13190f7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1314068a10 con 0x7f13141510f0 2023-10-08T00:11:52.488 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.485+0000 7f131359e700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1314070700 0x7f1314070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:52.488 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.485+0000 7f131359e700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1314070700 0x7f1314070b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50798/0 (socket says 172.21.15.99:50798) 2023-10-08T00:11:52.488 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.485+0000 7f131359e700 1 -- 172.21.15.99:0/2420789821 learned_addr learned my addr 172.21.15.99:0/2420789821 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:52.488 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.485+0000 7f130a59c700 1 --1- 172.21.15.99:0/2420789821 >> v1:172.21.15.99:6789/0 conn(0x7f13140c9af0 0x7f131415dd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:54262/0 (socket says 172.21.15.99:54262) 2023-10-08T00:11:52.488 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.486+0000 7f1312d9d700 1 --2- 172.21.15.99:0/2420789821 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13141510f0 0x7f131406ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:52.488 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.486+0000 7f131359e700 1 -- 172.21.15.99:0/2420789821 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13141510f0 msgr2=0x7f131406ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:52.488 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.486+0000 7f131359e700 1 --2- 172.21.15.99:0/2420789821 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13141510f0 0x7f131406ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:52.488 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.486+0000 7f131359e700 1 -- 172.21.15.99:0/2420789821 >> v1:172.21.15.99:6789/0 conn(0x7f13140c9af0 legacy=0x7f131415dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:52.489 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.486+0000 7f131359e700 1 -- 172.21.15.99:0/2420789821 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1314068cf0 con 0x7f1314070700 2023-10-08T00:11:52.489 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.486+0000 7f131259c700 1 -- 172.21.15.99:0/2420789821 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3954591994 0 0) 0x7f1314071080 con 0x7f13140c9af0 2023-10-08T00:11:52.489 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.486+0000 7f131359e700 1 --2- 172.21.15.99:0/2420789821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1314070700 0x7f1314070b40 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f130c009f20 tx=0x7f130c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e42ed73fedc66d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:52.489 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.486+0000 7f131259c700 1 -- 172.21.15.99:0/2420789821 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130c00e070 con 0x7f1314070700 2023-10-08T00:11:52.489 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.486+0000 7f131259c700 1 -- 172.21.15.99:0/2420789821 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f130c00ba80 con 0x7f1314070700 2023-10-08T00:11:52.489 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.486+0000 7f131259c700 1 -- 172.21.15.99:0/2420789821 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130c0092e0 con 0x7f1314070700 2023-10-08T00:11:52.489 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.486+0000 7f13190f7700 1 -- 172.21.15.99:0/2420789821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1314070700 msgr2=0x7f1314070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:52.489 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.486+0000 7f13190f7700 1 --2- 172.21.15.99:0/2420789821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1314070700 0x7f1314070b40 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f130c009f20 tx=0x7f130c009840 comp rx=0 tx=0).stop 2023-10-08T00:11:52.489 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.486+0000 7f13190f7700 1 -- 172.21.15.99:0/2420789821 shutdown_connections 2023-10-08T00:11:52.489 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.486+0000 7f13190f7700 1 --2- 172.21.15.99:0/2420789821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1314070700 0x7f1314070b40 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:52.489 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.486+0000 7f13190f7700 1 --2- 172.21.15.99:0/2420789821 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13141510f0 0x7f131406ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:52.489 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.486+0000 7f13190f7700 1 -- 172.21.15.99:0/2420789821 >> 172.21.15.99:0/2420789821 conn(0x7f13140ca4f0 msgr2=0x7f13140ca8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:52.490 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.486+0000 7f13190f7700 1 -- 172.21.15.99:0/2420789821 shutdown_connections 2023-10-08T00:11:52.490 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.487+0000 7f13190f7700 1 -- 172.21.15.99:0/2420789821 wait complete. 2023-10-08T00:11:52.492 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.488+0000 7f13190f7700 1 Processor -- start 2023-10-08T00:11:52.492 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.488+0000 7f13190f7700 1 -- start start 2023-10-08T00:11:52.492 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.488+0000 7f13190f7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1314070700 0x7f131406e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:52.492 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.488+0000 7f13190f7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13140c9af0 0x7f131416a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:52.493 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.488+0000 7f13190f7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13141510f0 0x7f131414fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:52.493 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.488+0000 7f13190f7700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131416b1c0 con 0x7f13141510f0 2023-10-08T00:11:52.493 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.488+0000 7f13190f7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131416b4a0 con 0x7f1314070700 2023-10-08T00:11:52.493 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.488+0000 7f13190f7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f131416b780 con 0x7f13140c9af0 2023-10-08T00:11:52.493 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.488+0000 7f131359e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13141510f0 0x7f131414fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:52.493 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.488+0000 7f131359e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13141510f0 0x7f131414fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50800/0 (socket says 172.21.15.99:50800) 2023-10-08T00:11:52.493 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.488+0000 7f131359e700 1 -- 172.21.15.99:0/3931888606 learned_addr learned my addr 172.21.15.99:0/3931888606 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:52.493 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.489+0000 7f130a59c700 1 --2- 172.21.15.99:0/3931888606 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13140c9af0 0x7f131416a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:52.494 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.489+0000 7f131359e700 1 -- 172.21.15.99:0/3931888606 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13140c9af0 msgr2=0x7f131416a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:52.494 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.489+0000 7f1312d9d700 1 --2- 172.21.15.99:0/3931888606 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1314070700 0x7f131406e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:52.494 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.489+0000 7f131359e700 1 --2- 172.21.15.99:0/3931888606 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13140c9af0 0x7f131416a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:52.494 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.489+0000 7f131359e700 1 -- 172.21.15.99:0/3931888606 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1314070700 msgr2=0x7f131406e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:52.494 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.489+0000 7f131359e700 1 --2- 172.21.15.99:0/3931888606 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1314070700 0x7f131406e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:52.494 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.489+0000 7f131359e700 1 -- 172.21.15.99:0/3931888606 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f130c009ae0 con 0x7f13141510f0 2023-10-08T00:11:52.494 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.489+0000 7f130a59c700 1 --2- 172.21.15.99:0/3931888606 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13140c9af0 0x7f131416a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:11:52.494 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.489+0000 7f131359e700 1 --2- 172.21.15.99:0/3931888606 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13141510f0 0x7f131414fce0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f131406d0c0 tx=0x7f130c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:52.494 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.489+0000 7f130b7fe700 1 -- 172.21.15.99:0/3931888606 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130c0032f0 con 0x7f13141510f0 2023-10-08T00:11:52.494 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.489+0000 7f130b7fe700 1 -- 172.21.15.99:0/3931888606 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f130c026460 con 0x7f13141510f0 2023-10-08T00:11:52.494 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.489+0000 7f13190f7700 1 -- 172.21.15.99:0/3931888606 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13140c5c20 con 0x7f13141510f0 2023-10-08T00:11:52.494 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.489+0000 7f130b7fe700 1 -- 172.21.15.99:0/3931888606 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130c026800 con 0x7f13141510f0 2023-10-08T00:11:52.495 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.489+0000 7f13190f7700 1 -- 172.21.15.99:0/3931888606 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1314154180 con 0x7f13141510f0 2023-10-08T00:11:52.495 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.490+0000 7f13190f7700 1 -- 172.21.15.99:0/3931888606 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13140cb210 con 0x7f13141510f0 2023-10-08T00:11:52.495 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.490+0000 7f130b7fe700 1 -- 172.21.15.99:0/3931888606 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f130c003040 con 0x7f13141510f0 2023-10-08T00:11:52.495 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.491+0000 7f130b7fe700 1 --2- 172.21.15.99:0/3931888606 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f12f4066f30 0x7f12f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:52.495 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.491+0000 7f130b7fe700 1 -- 172.21.15.99:0/3931888606 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f130c086950 con 0x7f13141510f0 2023-10-08T00:11:52.495 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.491+0000 7f1312d9d700 1 --2- 172.21.15.99:0/3931888606 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f12f4066f30 0x7f12f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:52.495 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.491+0000 7f1312d9d700 1 --2- 172.21.15.99:0/3931888606 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f12f4066f30 0x7f12f40693e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f13141582e0 tx=0x7f1304008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:52.495 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.495+0000 7f130b7fe700 1 -- 172.21.15.99:0/3931888606 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f130c053bc0 con 0x7f13141510f0 2023-10-08T00:11:52.678 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.677+0000 7f13190f7700 1 -- 172.21.15.99:0/3931888606 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f131404c3c0 con 0x7f13141510f0 2023-10-08T00:11:52.678 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.678+0000 7f130b7fe700 1 -- 172.21.15.99:0/3931888606 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f130c0571e0 con 0x7f13141510f0 2023-10-08T00:11:52.678 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:11:52.679 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":33,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:11:52.347370+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:11:38.415553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-08T00:11:39.634576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:11:52.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.680+0000 7f13190f7700 1 -- 172.21.15.99:0/3931888606 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f12f4066f30 msgr2=0x7f12f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:52.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.680+0000 7f13190f7700 1 --2- 172.21.15.99:0/3931888606 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f12f4066f30 0x7f12f40693e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f13141582e0 tx=0x7f1304008040 comp rx=0 tx=0).stop 2023-10-08T00:11:52.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.680+0000 7f13190f7700 1 -- 172.21.15.99:0/3931888606 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13141510f0 msgr2=0x7f131414fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:52.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.680+0000 7f13190f7700 1 --2- 172.21.15.99:0/3931888606 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13141510f0 0x7f131414fce0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f131406d0c0 tx=0x7f130c018000 comp rx=0 tx=0).stop 2023-10-08T00:11:52.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.680+0000 7f13190f7700 1 -- 172.21.15.99:0/3931888606 shutdown_connections 2023-10-08T00:11:52.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.680+0000 7f13190f7700 1 --2- 172.21.15.99:0/3931888606 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f12f4066f30 0x7f12f40693e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:52.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.680+0000 7f13190f7700 1 --2- 172.21.15.99:0/3931888606 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1314070700 0x7f131406e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:52.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.680+0000 7f13190f7700 1 --2- 172.21.15.99:0/3931888606 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13141510f0 0x7f131414fce0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:52.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.680+0000 7f13190f7700 1 --2- 172.21.15.99:0/3931888606 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13140c9af0 0x7f131416a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:52.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.680+0000 7f13190f7700 1 -- 172.21.15.99:0/3931888606 >> 172.21.15.99:0/3931888606 conn(0x7f13140ca4f0 msgr2=0x7f13140c2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:52.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.680+0000 7f13190f7700 1 -- 172.21.15.99:0/3931888606 shutdown_connections 2023-10-08T00:11:52.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:52.680+0000 7f13190f7700 1 -- 172.21.15.99:0/3931888606 wait complete. 2023-10-08T00:11:52.685 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:11:52.687 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:11:52.693 DEBUG:teuthology.orchestra.run.smithi099:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-08T00:11:52.693 DEBUG:teuthology.orchestra.run.smithi099:> import os 2023-10-08T00:11:52.693 DEBUG:teuthology.orchestra.run.smithi099:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2023-10-08T00:11:52.693 DEBUG:teuthology.orchestra.run.smithi099:> os.fchmod(fd, 0O666) 2023-10-08T00:11:52.693 DEBUG:teuthology.orchestra.run.smithi099:> os.fsync(fd) 2023-10-08T00:11:52.693 DEBUG:teuthology.orchestra.run.smithi099:> os.close(fd) 2023-10-08T00:11:52.694 DEBUG:teuthology.orchestra.run.smithi099:> ' 2023-10-08T00:11:52.985 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:11:52.986 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-08T00:11:53.013 INFO:teuthology.orchestra.run.smithi099.stdout:client4996 2023-10-08T00:11:53.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:53.014 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/06962cdb-9a6f-417f-82fd-9e4e3dedbccf.client4996/osdmap') 2023-10-08T00:11:53.089 INFO:teuthology.orchestra.run.smithi099.stdout:epoch 31 barrier 31 flags 0x588000 2023-10-08T00:11:53.089 INFO:teuthology.orchestra.run.smithi099.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-08T00:11:53.090 INFO:teuthology.orchestra.run.smithi099.stdout:pool 4 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-08T00:11:53.090 INFO:teuthology.orchestra.run.smithi099.stdout:pool 5 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-08T00:11:53.090 INFO:teuthology.orchestra.run.smithi099.stdout:osd0 (1)172.21.15.99:6801 100% (exists, up) 100% -1 2023-10-08T00:11:53.090 INFO:teuthology.orchestra.run.smithi099.stdout:osd1 (1)172.21.15.99:6809 100% (exists, up) 100% -1 2023-10-08T00:11:53.090 INFO:teuthology.orchestra.run.smithi099.stdout:osd2 (1)172.21.15.99:6817 100% (exists, up) 100% -1 2023-10-08T00:11:53.090 INFO:teuthology.orchestra.run.smithi099.stdout:osd3 (1)172.21.15.99:6820 100% (exists, up) 100% -1 2023-10-08T00:11:53.090 INFO:teuthology.orchestra.run.smithi099.stdout:osd4 (1)172.21.15.191:6801 100% (exists, up) 100% -1 2023-10-08T00:11:53.091 INFO:teuthology.orchestra.run.smithi099.stdout:osd5 (1)172.21.15.191:6818 100% (exists, up) 100% -1 2023-10-08T00:11:53.091 INFO:teuthology.orchestra.run.smithi099.stdout:osd6 (1)172.21.15.191:6809 100% (exists, up) 100% -1 2023-10-08T00:11:53.091 INFO:teuthology.orchestra.run.smithi099.stdout:osd7 (1)172.21.15.191:6819 100% (exists, up) 100% -1 2023-10-08T00:11:53.092 INFO:teuthology.orchestra.run.smithi099.stderr:1+1 records in 2023-10-08T00:11:53.092 INFO:teuthology.orchestra.run.smithi099.stderr:1+1 records out 2023-10-08T00:11:53.092 INFO:teuthology.orchestra.run.smithi099.stderr:795 bytes copied, 3.5691e-05 s, 22.3 MB/s 2023-10-08T00:11:53.092 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:11:53.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.247+0000 7f58135c3700 1 Processor -- start 2023-10-08T00:11:53.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.247+0000 7f58135c3700 1 -- start start 2023-10-08T00:11:53.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.248+0000 7f58135c3700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f580c15a620 0x7f580c15d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:53.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.248+0000 7f58135c3700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f580c15cbb0 con 0x7f580c15d580 2023-10-08T00:11:53.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.248+0000 7f58135c3700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f580c15bfa0 con 0x7f580c157870 2023-10-08T00:11:53.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.248+0000 7f58135c3700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f580c15bdb0 con 0x7f580c15a620 2023-10-08T00:11:53.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.248+0000 7f5811b60700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f580c15d580 0x7f580c158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:54266/0 (socket says 172.21.15.99:54266) 2023-10-08T00:11:53.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.248+0000 7f5811b60700 1 -- 172.21.15.99:0/4262956693 learned_addr learned my addr 172.21.15.99:0/4262956693 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:53.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.248+0000 7f5810b5e700 1 --2- 172.21.15.99:0/4262956693 >> v2:172.21.15.191:3300/0 conn(0x7f580c15a620 0x7f580c15d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:53.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.248+0000 7f5803fff700 1 -- 172.21.15.99:0/4262956693 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3586504524 0 0) 0x7f580c15cbb0 con 0x7f580c15d580 2023-10-08T00:11:53.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.248+0000 7f5803fff700 1 -- 172.21.15.99:0/4262956693 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57f8003570 con 0x7f580c15d580 2023-10-08T00:11:53.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.248+0000 7f5803fff700 1 -- 172.21.15.99:0/4262956693 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2731173481 0 0) 0x7f57f8003570 con 0x7f580c15d580 2023-10-08T00:11:53.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.248+0000 7f5803fff700 1 -- 172.21.15.99:0/4262956693 >> v2:172.21.15.191:3300/0 conn(0x7f580c15a620 msgr2=0x7f580c15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:53.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.248+0000 7f5803fff700 1 --2- 172.21.15.99:0/4262956693 >> v2:172.21.15.191:3300/0 conn(0x7f580c15a620 0x7f580c15d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:53.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.248+0000 7f5803fff700 1 -- 172.21.15.99:0/4262956693 >> v1:172.21.15.191:6789/0 conn(0x7f580c157870 legacy=0x7f580c15b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:53.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.248+0000 7f5803fff700 1 -- 172.21.15.99:0/4262956693 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f580c15c280 con 0x7f580c15d580 2023-10-08T00:11:53.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.248+0000 7f5803fff700 1 -- 172.21.15.99:0/4262956693 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f5804001470 con 0x7f580c15d580 2023-10-08T00:11:53.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.248+0000 7f5803fff700 1 -- 172.21.15.99:0/4262956693 >> v1:172.21.15.99:6789/0 conn(0x7f580c15d580 legacy=0x7f580c158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:53.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.248+0000 7f5803fff700 1 --2- 172.21.15.99:0/4262956693 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57f8003a60 0x7f57f8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:53.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.248+0000 7f5803fff700 1 --2- 172.21.15.99:0/4262956693 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57f8004810 0x7f57f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:53.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.248+0000 7f5803fff700 1 --2- 172.21.15.99:0/4262956693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57f80051c0 0x7f57f800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:53.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.248+0000 7f5803fff700 1 -- 172.21.15.99:0/4262956693 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f800c130 con 0x7f57f80051c0 2023-10-08T00:11:53.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.248+0000 7f5803fff700 1 -- 172.21.15.99:0/4262956693 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f800c2a0 con 0x7f57f8003a60 2023-10-08T00:11:53.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.248+0000 7f5803fff700 1 -- 172.21.15.99:0/4262956693 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57f800c460 con 0x7f57f8004810 2023-10-08T00:11:53.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.249+0000 7f5811b60700 1 --2- 172.21.15.99:0/4262956693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57f80051c0 0x7f57f800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:53.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.249+0000 7f581135f700 1 --2- 172.21.15.99:0/4262956693 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57f8003a60 0x7f57f8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:53.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.249+0000 7f5810b5e700 1 --2- 172.21.15.99:0/4262956693 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57f8004810 0x7f57f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:53.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.249+0000 7f581135f700 1 -- 172.21.15.99:0/4262956693 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57f8004810 msgr2=0x7f57f8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:53.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.249+0000 7f581135f700 1 --2- 172.21.15.99:0/4262956693 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57f8004810 0x7f57f8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:53.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.249+0000 7f581135f700 1 -- 172.21.15.99:0/4262956693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57f80051c0 msgr2=0x7f57f800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:53.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.249+0000 7f581135f700 1 --2- 172.21.15.99:0/4262956693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57f80051c0 0x7f57f800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:53.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.249+0000 7f581135f700 1 -- 172.21.15.99:0/4262956693 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f8003570 con 0x7f57f8003a60 2023-10-08T00:11:53.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.249+0000 7f5811b60700 1 --2- 172.21.15.99:0/4262956693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57f80051c0 0x7f57f800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:11:53.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.249+0000 7f5811b60700 1 -- 172.21.15.99:0/4262956693 reap_dead start 2023-10-08T00:11:53.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.249+0000 7f581135f700 1 --2- 172.21.15.99:0/4262956693 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57f8003a60 0x7f57f8003e70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5808001fc0 tx=0x7f5808001640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea003c8bc310a4fc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:53.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.249+0000 7f5803fff700 1 -- 172.21.15.99:0/4262956693 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5808001950 con 0x7f57f8003a60 2023-10-08T00:11:53.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.249+0000 7f5803fff700 1 -- 172.21.15.99:0/4262956693 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f580800c8b0 con 0x7f57f8003a60 2023-10-08T00:11:53.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.249+0000 7f5803fff700 1 -- 172.21.15.99:0/4262956693 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580800cc50 con 0x7f57f8003a60 2023-10-08T00:11:53.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.250+0000 7f58135c3700 1 -- 172.21.15.99:0/4262956693 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57f8003a60 msgr2=0x7f57f8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:53.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.250+0000 7f58135c3700 1 --2- 172.21.15.99:0/4262956693 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57f8003a60 0x7f57f8003e70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5808001fc0 tx=0x7f5808001640 comp rx=0 tx=0).stop 2023-10-08T00:11:53.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.250+0000 7f58135c3700 1 -- 172.21.15.99:0/4262956693 shutdown_connections 2023-10-08T00:11:53.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.250+0000 7f58135c3700 1 --2- 172.21.15.99:0/4262956693 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57f8003a60 0x7f57f8003e70 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:53.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.250+0000 7f58135c3700 1 -- 172.21.15.99:0/4262956693 >> 172.21.15.99:0/4262956693 conn(0x7f580c068ce0 msgr2=0x7f580c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:53.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.250+0000 7f58135c3700 1 -- 172.21.15.99:0/4262956693 shutdown_connections 2023-10-08T00:11:53.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.250+0000 7f58135c3700 1 -- 172.21.15.99:0/4262956693 wait complete. 2023-10-08T00:11:53.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.251+0000 7f58135c3700 1 Processor -- start 2023-10-08T00:11:53.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.251+0000 7f58135c3700 1 -- start start 2023-10-08T00:11:53.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.252+0000 7f58135c3700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57f8003a60 0x7f580c0b3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:53.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.252+0000 7f58135c3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f580c158bf0 0x7f580c159060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:53.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.252+0000 7f58135c3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f580c1595a0 0x7f580c0b12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:53.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.252+0000 7f58135c3700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f580c16a3e0 con 0x7f580c1595a0 2023-10-08T00:11:53.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.252+0000 7f58135c3700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f580c0b2920 con 0x7f57f8003a60 2023-10-08T00:11:53.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.252+0000 7f58135c3700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f580c0c6c40 con 0x7f580c158bf0 2023-10-08T00:11:53.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.252+0000 7f5811b60700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f580c1595a0 0x7f580c0b12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:53.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.252+0000 7f5811b60700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f580c1595a0 0x7f580c0b12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50806/0 (socket says 172.21.15.99:50806) 2023-10-08T00:11:53.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.252+0000 7f5811b60700 1 -- 172.21.15.99:0/2638186021 learned_addr learned my addr 172.21.15.99:0/2638186021 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:53.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.252+0000 7f581135f700 1 --2- 172.21.15.99:0/2638186021 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57f8003a60 0x7f580c0b3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:53.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.252+0000 7f5810b5e700 1 --2- 172.21.15.99:0/2638186021 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f580c158bf0 0x7f580c159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:53.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.252+0000 7f5811b60700 1 -- 172.21.15.99:0/2638186021 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f580c158bf0 msgr2=0x7f580c159060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:53.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.252+0000 7f5811b60700 1 --2- 172.21.15.99:0/2638186021 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f580c158bf0 0x7f580c159060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:53.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.252+0000 7f5811b60700 1 -- 172.21.15.99:0/2638186021 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57f8003a60 msgr2=0x7f580c0b3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:53.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.252+0000 7f5811b60700 1 --2- 172.21.15.99:0/2638186021 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57f8003a60 0x7f580c0b3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:53.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.252+0000 7f5811b60700 1 -- 172.21.15.99:0/2638186021 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f580c147cf0 con 0x7f580c1595a0 2023-10-08T00:11:53.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.252+0000 7f5811b60700 1 --2- 172.21.15.99:0/2638186021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f580c1595a0 0x7f580c0b12a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f580c0b3830 tx=0x7f5804001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:53.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.252+0000 7f58017fa700 1 -- 172.21.15.99:0/2638186021 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5804001ae0 con 0x7f580c1595a0 2023-10-08T00:11:53.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.252+0000 7f58135c3700 1 -- 172.21.15.99:0/2638186021 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f580c0c5c90 con 0x7f580c1595a0 2023-10-08T00:11:53.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.252+0000 7f58017fa700 1 -- 172.21.15.99:0/2638186021 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f580400f970 con 0x7f580c1595a0 2023-10-08T00:11:53.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.252+0000 7f58017fa700 1 -- 172.21.15.99:0/2638186021 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580400fd10 con 0x7f580c1595a0 2023-10-08T00:11:53.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.252+0000 7f58135c3700 1 -- 172.21.15.99:0/2638186021 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f580c153b20 con 0x7f580c1595a0 2023-10-08T00:11:53.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.253+0000 7f58135c3700 1 -- 172.21.15.99:0/2638186021 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f580c0c5eb0 con 0x7f580c1595a0 2023-10-08T00:11:53.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.254+0000 7f58017fa700 1 -- 172.21.15.99:0/2638186021 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5804013600 con 0x7f580c1595a0 2023-10-08T00:11:53.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.254+0000 7f58017fa700 1 --2- 172.21.15.99:0/2638186021 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f57f0066f30 0x7f57f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:53.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.254+0000 7f58017fa700 1 -- 172.21.15.99:0/2638186021 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f58040846b0 con 0x7f580c1595a0 2023-10-08T00:11:53.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.254+0000 7f581135f700 1 --2- 172.21.15.99:0/2638186021 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f57f0066f30 0x7f57f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:53.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.255+0000 7f581135f700 1 --2- 172.21.15.99:0/2638186021 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f57f0066f30 0x7f57f00693e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f580c0cdc70 tx=0x7f5808015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:53.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.258+0000 7f58017fa700 1 -- 172.21.15.99:0/2638186021 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f580c0c5eb0 con 0x7f580c1595a0 2023-10-08T00:11:53.485 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.484+0000 7f58135c3700 1 -- 172.21.15.99:0/2638186021 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f580c0c5e80 con 0x7f580c1595a0 2023-10-08T00:11:53.485 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.485+0000 7f58017fa700 1 -- 172.21.15.99:0/2638186021 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4600 (secure 0 0 0) 0x7f5804051920 con 0x7f580c1595a0 2023-10-08T00:11:53.485 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:11:53.486 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4852,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/824084995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":824084995},{"type":"v1","addr":"172.21.15.191:6837","nonce":824084995}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4855,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/3349126920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3349126920},{"type":"v1","addr":"172.21.15.191:6836","nonce":3349126920}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4899,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/1004180438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1004180438},{"type":"v1","addr":"172.21.15.99:6834","nonce":1004180438}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:11:40.672879+0000","modified":"2023-10-08T00:11:44.314526+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4902},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4902":{"gid":4902,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/4189095717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":4189095717},{"type":"v1","addr":"172.21.15.99:6835","nonce":4189095717}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-08T00:11:53.487 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.486+0000 7f58135c3700 1 -- 172.21.15.99:0/2638186021 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f57f0066f30 msgr2=0x7f57f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:53.488 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.487+0000 7f58135c3700 1 --2- 172.21.15.99:0/2638186021 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f57f0066f30 0x7f57f00693e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f580c0cdc70 tx=0x7f5808015000 comp rx=0 tx=0).stop 2023-10-08T00:11:53.488 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.487+0000 7f58135c3700 1 -- 172.21.15.99:0/2638186021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f580c1595a0 msgr2=0x7f580c0b12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:53.488 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.487+0000 7f58135c3700 1 --2- 172.21.15.99:0/2638186021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f580c1595a0 0x7f580c0b12a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f580c0b3830 tx=0x7f5804001e50 comp rx=0 tx=0).stop 2023-10-08T00:11:53.488 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.487+0000 7f58135c3700 1 -- 172.21.15.99:0/2638186021 shutdown_connections 2023-10-08T00:11:53.488 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.487+0000 7f58135c3700 1 --2- 172.21.15.99:0/2638186021 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f57f0066f30 0x7f57f00693e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:53.488 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.487+0000 7f58135c3700 1 --2- 172.21.15.99:0/2638186021 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57f8003a60 0x7f580c0b3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:53.488 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.487+0000 7f58135c3700 1 --2- 172.21.15.99:0/2638186021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f580c1595a0 0x7f580c0b12a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:53.488 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.487+0000 7f58135c3700 1 --2- 172.21.15.99:0/2638186021 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f580c158bf0 0x7f580c159060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:53.488 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.487+0000 7f58135c3700 1 -- 172.21.15.99:0/2638186021 >> 172.21.15.99:0/2638186021 conn(0x7f580c068ce0 msgr2=0x7f580c155610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:53.488 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.487+0000 7f58135c3700 1 -- 172.21.15.99:0/2638186021 shutdown_connections 2023-10-08T00:11:53.488 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:53.487+0000 7f58135c3700 1 -- 172.21.15.99:0/2638186021 wait complete. 2023-10-08T00:11:53.489 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 17 2023-10-08T00:11:53.499 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json osdmap barrier 33 2023-10-08T00:11:53.629 DEBUG:tasks.cephfs.filesystem:_json_asok output empty 2023-10-08T00:11:53.630 DEBUG:teuthology.orchestra.run.smithi099:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-08T00:11:53.630 DEBUG:teuthology.orchestra.run.smithi099:> import os 2023-10-08T00:11:53.630 DEBUG:teuthology.orchestra.run.smithi099:> fd = os.open("/home/ubuntu/cephtest/mnt.0/baz", os.O_CREAT | os.O_RDWR, 0O600) 2023-10-08T00:11:53.630 DEBUG:teuthology.orchestra.run.smithi099:> os.fchmod(fd, 0O666) 2023-10-08T00:11:53.630 DEBUG:teuthology.orchestra.run.smithi099:> os.fsync(fd) 2023-10-08T00:11:53.630 DEBUG:teuthology.orchestra.run.smithi099:> os.close(fd) 2023-10-08T00:11:53.630 DEBUG:teuthology.orchestra.run.smithi099:> ' 2023-10-08T00:11:53.882 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:11:53.883 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-08T00:11:53.951 INFO:teuthology.orchestra.run.smithi099.stdout:client4996 2023-10-08T00:11:53.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:53.952 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/06962cdb-9a6f-417f-82fd-9e4e3dedbccf.client4996/osdmap') 2023-10-08T00:11:54.027 INFO:teuthology.orchestra.run.smithi099.stdout:epoch 33 barrier 33 flags 0x588000 2023-10-08T00:11:54.028 INFO:teuthology.orchestra.run.smithi099.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-08T00:11:54.028 INFO:teuthology.orchestra.run.smithi099.stdout:pool 4 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-08T00:11:54.028 INFO:teuthology.orchestra.run.smithi099.stdout:pool 5 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-08T00:11:54.028 INFO:teuthology.orchestra.run.smithi099.stdout:osd0 (1)172.21.15.99:6801 100% (exists, up) 100% -1 2023-10-08T00:11:54.028 INFO:teuthology.orchestra.run.smithi099.stdout:osd1 (1)172.21.15.99:6809 100% (exists, up) 100% -1 2023-10-08T00:11:54.028 INFO:teuthology.orchestra.run.smithi099.stdout:osd2 (1)172.21.15.99:6817 100% (exists, up) 100% -1 2023-10-08T00:11:54.028 INFO:teuthology.orchestra.run.smithi099.stdout:osd3 (1)172.21.15.99:6820 100% (exists, up) 100% -1 2023-10-08T00:11:54.028 INFO:teuthology.orchestra.run.smithi099.stdout:osd4 (1)172.21.15.191:6801 100% (exists, up) 100% -1 2023-10-08T00:11:54.028 INFO:teuthology.orchestra.run.smithi099.stdout:osd5 (1)172.21.15.191:6818 100% (exists, up) 100% -1 2023-10-08T00:11:54.028 INFO:teuthology.orchestra.run.smithi099.stdout:osd6 (1)172.21.15.191:6809 100% (exists, up) 100% -1 2023-10-08T00:11:54.029 INFO:teuthology.orchestra.run.smithi099.stdout:osd7 (1)172.21.15.191:6819 100% (exists, up) 100% -1 2023-10-08T00:11:54.029 INFO:teuthology.orchestra.run.smithi099.stderr:1+1 records in 2023-10-08T00:11:54.030 INFO:teuthology.orchestra.run.smithi099.stderr:1+1 records out 2023-10-08T00:11:54.030 INFO:teuthology.orchestra.run.smithi099.stderr:795 bytes copied, 3.6539e-05 s, 21.8 MB/s 2023-10-08T00:11:54.030 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:11:54.030 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-08T00:11:54.098 INFO:teuthology.orchestra.run.smithi099.stdout:client4996 2023-10-08T00:11:54.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:54.099 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/06962cdb-9a6f-417f-82fd-9e4e3dedbccf.client4996/osdmap') 2023-10-08T00:11:54.174 INFO:teuthology.orchestra.run.smithi099.stdout:epoch 33 barrier 33 flags 0x588000 2023-10-08T00:11:54.174 INFO:teuthology.orchestra.run.smithi099.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-08T00:11:54.174 INFO:teuthology.orchestra.run.smithi099.stdout:pool 4 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-08T00:11:54.174 INFO:teuthology.orchestra.run.smithi099.stdout:pool 5 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-08T00:11:54.174 INFO:teuthology.orchestra.run.smithi099.stdout:osd0 (1)172.21.15.99:6801 100% (exists, up) 100% -1 2023-10-08T00:11:54.174 INFO:teuthology.orchestra.run.smithi099.stdout:osd1 (1)172.21.15.99:6809 100% (exists, up) 100% -1 2023-10-08T00:11:54.174 INFO:teuthology.orchestra.run.smithi099.stdout:osd2 (1)172.21.15.99:6817 100% (exists, up) 100% -1 2023-10-08T00:11:54.175 INFO:teuthology.orchestra.run.smithi099.stdout:osd3 (1)172.21.15.99:6820 100% (exists, up) 100% -1 2023-10-08T00:11:54.175 INFO:teuthology.orchestra.run.smithi099.stdout:osd4 (1)172.21.15.191:6801 100% (exists, up) 100% -1 2023-10-08T00:11:54.175 INFO:teuthology.orchestra.run.smithi099.stdout:osd5 (1)172.21.15.191:6818 100% (exists, up) 100% -1 2023-10-08T00:11:54.175 INFO:teuthology.orchestra.run.smithi099.stdout:osd6 (1)172.21.15.191:6809 100% (exists, up) 100% -1 2023-10-08T00:11:54.175 INFO:teuthology.orchestra.run.smithi099.stdout:osd7 (1)172.21.15.191:6819 100% (exists, up) 100% -1 2023-10-08T00:11:54.176 INFO:teuthology.orchestra.run.smithi099.stderr:1+1 records in 2023-10-08T00:11:54.176 INFO:teuthology.orchestra.run.smithi099.stderr:1+1 records out 2023-10-08T00:11:54.176 INFO:teuthology.orchestra.run.smithi099.stderr:795 bytes copied, 3.6528e-05 s, 21.8 MB/s 2023-10-08T00:11:54.177 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-08T00:11:54.177 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T00:11:54.177 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T00:11:54.253 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T00:11:54.296 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T00:11:54.296 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T00:11:54.296 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:11:54.296 DEBUG:teuthology.orchestra.run.smithi099:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-08T00:11:54.353 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2023-10-08T00:11:54.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:54.353 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T00:11:54.372 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-08T00:11:54.372 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:11:54.372 DEBUG:teuthology.orchestra.run.smithi191:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-08T00:11:54.426 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2023-10-08T00:11:54.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:11:54.426 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-08T00:11:54.442 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:11:54.554 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.554+0000 7f474d82d700 1 Processor -- start 2023-10-08T00:11:54.555 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.554+0000 7f474d82d700 1 -- start start 2023-10-08T00:11:54.558 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.555+0000 7f474d82d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f47480cffa0 0x7f474806a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:54.558 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.555+0000 7f474d82d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f474815a230 0x7f474806abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:54.559 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.555+0000 7f474d82d700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47480b3d40 con 0x7f4748159e60 2023-10-08T00:11:54.559 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.555+0000 7f474d82d700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4748069d70 con 0x7f47480cffa0 2023-10-08T00:11:54.559 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.555+0000 7f474d82d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47480b4300 con 0x7f474815a230 2023-10-08T00:11:54.559 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.555+0000 7f4746ffd700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f4748159e60 0x7f47480b6890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56344/0 (socket says 172.21.15.99:56344) 2023-10-08T00:11:54.559 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.555+0000 7f4746ffd700 1 -- 172.21.15.99:0/401155673 learned_addr learned my addr 172.21.15.99:0/401155673 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:54.559 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.555+0000 7f47477fe700 1 --2- 172.21.15.99:0/401155673 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f474815a230 0x7f474806abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:54.559 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.555+0000 7f47467fc700 1 --2- 172.21.15.99:0/401155673 >> v2:172.21.15.191:3300/0 conn(0x7f47480cffa0 0x7f474806a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:54.559 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.555+0000 7f4745ffb700 1 -- 172.21.15.99:0/401155673 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3007300501 0 0) 0x7f47480b3d40 con 0x7f4748159e60 2023-10-08T00:11:54.559 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.555+0000 7f4745ffb700 1 -- 172.21.15.99:0/401155673 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4734003570 con 0x7f4748159e60 2023-10-08T00:11:54.559 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.555+0000 7f4745ffb700 1 -- 172.21.15.99:0/401155673 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2615686380 0 0) 0x7f4734003570 con 0x7f4748159e60 2023-10-08T00:11:54.559 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.555+0000 7f4745ffb700 1 -- 172.21.15.99:0/401155673 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f474815a230 msgr2=0x7f474806abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:54.560 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.555+0000 7f4745ffb700 1 --2- 172.21.15.99:0/401155673 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f474815a230 0x7f474806abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:54.560 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.555+0000 7f4745ffb700 1 -- 172.21.15.99:0/401155673 >> v2:172.21.15.191:3300/0 conn(0x7f47480cffa0 msgr2=0x7f474806a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:54.560 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.555+0000 7f4745ffb700 1 --2- 172.21.15.99:0/401155673 >> v2:172.21.15.191:3300/0 conn(0x7f47480cffa0 0x7f474806a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:54.560 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.555+0000 7f4745ffb700 1 -- 172.21.15.99:0/401155673 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47480d0480 con 0x7f4748159e60 2023-10-08T00:11:54.560 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.555+0000 7f4745ffb700 1 -- 172.21.15.99:0/401155673 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f47400017f0 con 0x7f4748159e60 2023-10-08T00:11:54.560 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.555+0000 7f4745ffb700 1 -- 172.21.15.99:0/401155673 >> v1:172.21.15.99:6789/0 conn(0x7f4748159e60 legacy=0x7f47480b6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:54.560 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.555+0000 7f4745ffb700 1 --2- 172.21.15.99:0/401155673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4734003a60 0x7f4734003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:54.560 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.555+0000 7f4745ffb700 1 --2- 172.21.15.99:0/401155673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4734004810 0x7f4734004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:54.560 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.555+0000 7f4745ffb700 1 --2- 172.21.15.99:0/401155673 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f474815a230 0x7f47340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:54.560 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.555+0000 7f4745ffb700 1 -- 172.21.15.99:0/401155673 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4748069d70 con 0x7f4734004810 2023-10-08T00:11:54.560 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.555+0000 7f4745ffb700 1 -- 172.21.15.99:0/401155673 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47480b4300 con 0x7f4734003a60 2023-10-08T00:11:54.560 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.555+0000 7f4745ffb700 1 -- 172.21.15.99:0/401155673 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f473400ba60 con 0x7f474815a230 2023-10-08T00:11:54.561 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.555+0000 7f47467fc700 1 --2- 172.21.15.99:0/401155673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4734004810 0x7f4734004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:54.561 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.556+0000 7f4746ffd700 1 --2- 172.21.15.99:0/401155673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4734003a60 0x7f4734003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:54.561 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.556+0000 7f47477fe700 1 --2- 172.21.15.99:0/401155673 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f474815a230 0x7f47340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:54.561 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.556+0000 7f47467fc700 1 -- 172.21.15.99:0/401155673 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f474815a230 msgr2=0x7f47340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:54.561 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.556+0000 7f47467fc700 1 --2- 172.21.15.99:0/401155673 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f474815a230 0x7f47340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:54.561 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.556+0000 7f47467fc700 1 -- 172.21.15.99:0/401155673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4734003a60 msgr2=0x7f4734003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:54.561 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.556+0000 7f47467fc700 1 --2- 172.21.15.99:0/401155673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4734003a60 0x7f4734003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:54.561 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.556+0000 7f47467fc700 1 -- 172.21.15.99:0/401155673 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4734003570 con 0x7f4734004810 2023-10-08T00:11:54.561 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.556+0000 7f47467fc700 1 --2- 172.21.15.99:0/401155673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4734004810 0x7f4734004c80 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4730009c00 tx=0x7f47300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14b315cc9fec497f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:54.561 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.556+0000 7f4745ffb700 1 -- 172.21.15.99:0/401155673 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473000e710 con 0x7f4734004810 2023-10-08T00:11:54.561 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.556+0000 7f4745ffb700 1 -- 172.21.15.99:0/401155673 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f473000e870 con 0x7f4734004810 2023-10-08T00:11:54.561 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.556+0000 7f4745ffb700 1 -- 172.21.15.99:0/401155673 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473000ec30 con 0x7f4734004810 2023-10-08T00:11:54.562 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.557+0000 7f474d82d700 1 -- 172.21.15.99:0/401155673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4734004810 msgr2=0x7f4734004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:54.562 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.557+0000 7f474d82d700 1 --2- 172.21.15.99:0/401155673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4734004810 0x7f4734004c80 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4730009c00 tx=0x7f47300091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:54.562 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.557+0000 7f47477fe700 1 -- 172.21.15.99:0/401155673 reap_dead start 2023-10-08T00:11:54.562 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.557+0000 7f474d82d700 1 -- 172.21.15.99:0/401155673 shutdown_connections 2023-10-08T00:11:54.562 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.557+0000 7f474d82d700 1 -- 172.21.15.99:0/401155673 >> 172.21.15.99:0/401155673 conn(0x7f47480d1a50 msgr2=0x7f4748068f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:54.562 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.557+0000 7f474d82d700 1 -- 172.21.15.99:0/401155673 shutdown_connections 2023-10-08T00:11:54.562 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.557+0000 7f474d82d700 1 -- 172.21.15.99:0/401155673 wait complete. 2023-10-08T00:11:54.562 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.558+0000 7f474d82d700 1 Processor -- start 2023-10-08T00:11:54.562 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.558+0000 7f474d82d700 1 -- start start 2023-10-08T00:11:54.565 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.559+0000 7f474d82d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4734004810 0x7f47480c7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:54.565 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.559+0000 7f474d82d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f474816b2c0 0x7f4748157280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:54.565 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.559+0000 7f474d82d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4748154a70 0x7f47480b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:54.565 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.559+0000 7f474d82d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47480c5d20 con 0x7f4748154a70 2023-10-08T00:11:54.565 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.559+0000 7f474d82d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47480c6000 con 0x7f4734004810 2023-10-08T00:11:54.565 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.559+0000 7f474d82d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4748157ad0 con 0x7f474816b2c0 2023-10-08T00:11:54.565 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.559+0000 7f47477fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4748154a70 0x7f47480b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:54.565 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.559+0000 7f47477fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4748154a70 0x7f47480b24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58126/0 (socket says 172.21.15.99:58126) 2023-10-08T00:11:54.565 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.559+0000 7f47477fe700 1 -- 172.21.15.99:0/3703681745 learned_addr learned my addr 172.21.15.99:0/3703681745 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:54.566 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.559+0000 7f4746ffd700 1 --2- 172.21.15.99:0/3703681745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4734004810 0x7f47480c7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:54.566 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.559+0000 7f47467fc700 1 --2- 172.21.15.99:0/3703681745 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f474816b2c0 0x7f4748157280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:54.566 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.559+0000 7f47477fe700 1 -- 172.21.15.99:0/3703681745 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f474816b2c0 msgr2=0x7f4748157280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:54.566 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.559+0000 7f47477fe700 1 --2- 172.21.15.99:0/3703681745 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f474816b2c0 0x7f4748157280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:54.566 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.559+0000 7f47477fe700 1 -- 172.21.15.99:0/3703681745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4734004810 msgr2=0x7f47480c7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:54.566 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.559+0000 7f47477fe700 1 --2- 172.21.15.99:0/3703681745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4734004810 0x7f47480c7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:54.566 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.559+0000 7f47477fe700 1 -- 172.21.15.99:0/3703681745 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4748157db0 con 0x7f4748154a70 2023-10-08T00:11:54.566 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.559+0000 7f47477fe700 1 --2- 172.21.15.99:0/3703681745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4748154a70 0x7f47480b24a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4738006f00 tx=0x7f47380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:54.566 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.559+0000 7f473effd700 1 -- 172.21.15.99:0/3703681745 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4738013750 con 0x7f4748154a70 2023-10-08T00:11:54.566 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.559+0000 7f473effd700 1 -- 172.21.15.99:0/3703681745 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47380138b0 con 0x7f4748154a70 2023-10-08T00:11:54.566 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.559+0000 7f474d82d700 1 -- 172.21.15.99:0/3703681745 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4748158090 con 0x7f4748154a70 2023-10-08T00:11:54.567 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.559+0000 7f473effd700 1 -- 172.21.15.99:0/3703681745 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4738013c50 con 0x7f4748154a70 2023-10-08T00:11:54.567 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.559+0000 7f474d82d700 1 -- 172.21.15.99:0/3703681745 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47481577c0 con 0x7f4748154a70 2023-10-08T00:11:54.567 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.560+0000 7f474d82d700 1 -- 172.21.15.99:0/3703681745 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47480c6190 con 0x7f4748154a70 2023-10-08T00:11:54.567 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.561+0000 7f473effd700 1 -- 172.21.15.99:0/3703681745 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4738014650 con 0x7f4748154a70 2023-10-08T00:11:54.567 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.561+0000 7f473effd700 1 --2- 172.21.15.99:0/3703681745 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4728066f30 0x7f47280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:54.567 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.561+0000 7f473effd700 1 -- 172.21.15.99:0/3703681745 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f47380849e0 con 0x7f4748154a70 2023-10-08T00:11:54.567 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.561+0000 7f4746ffd700 1 --2- 172.21.15.99:0/3703681745 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4728066f30 0x7f47280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:54.567 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.562+0000 7f4746ffd700 1 --2- 172.21.15.99:0/3703681745 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4728066f30 0x7f47280693e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f47400019a0 tx=0x7f4740006040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:54.567 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.565+0000 7f473effd700 1 -- 172.21.15.99:0/3703681745 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4738051cd0 con 0x7f4748154a70 2023-10-08T00:11:54.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.787+0000 7f474d82d700 1 -- 172.21.15.99:0/3703681745 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f474804c3c0 con 0x7f4748154a70 2023-10-08T00:11:54.788 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.788+0000 7f473effd700 1 -- 172.21.15.99:0/3703681745 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4600 (secure 0 0 0) 0x7f4738022090 con 0x7f4748154a70 2023-10-08T00:11:54.789 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:11:54.789 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4852,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/824084995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":824084995},{"type":"v1","addr":"172.21.15.191:6837","nonce":824084995}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4855,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/3349126920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3349126920},{"type":"v1","addr":"172.21.15.191:6836","nonce":3349126920}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4899,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/1004180438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1004180438},{"type":"v1","addr":"172.21.15.99:6834","nonce":1004180438}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:11:40.672879+0000","modified":"2023-10-08T00:11:44.314526+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4902},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4902":{"gid":4902,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/4189095717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":4189095717},{"type":"v1","addr":"172.21.15.99:6835","nonce":4189095717}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-08T00:11:54.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.790+0000 7f474d82d700 1 -- 172.21.15.99:0/3703681745 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4728066f30 msgr2=0x7f47280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:54.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.790+0000 7f474d82d700 1 --2- 172.21.15.99:0/3703681745 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4728066f30 0x7f47280693e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f47400019a0 tx=0x7f4740006040 comp rx=0 tx=0).stop 2023-10-08T00:11:54.792 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.790+0000 7f474d82d700 1 -- 172.21.15.99:0/3703681745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4748154a70 msgr2=0x7f47480b24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:54.792 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.790+0000 7f474d82d700 1 --2- 172.21.15.99:0/3703681745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4748154a70 0x7f47480b24a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4738006f00 tx=0x7f47380091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:54.792 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.790+0000 7f474d82d700 1 -- 172.21.15.99:0/3703681745 shutdown_connections 2023-10-08T00:11:54.792 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.790+0000 7f474d82d700 1 --2- 172.21.15.99:0/3703681745 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4728066f30 0x7f47280693e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:54.792 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.790+0000 7f474d82d700 1 --2- 172.21.15.99:0/3703681745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4734004810 0x7f47480c7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:54.793 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.790+0000 7f474d82d700 1 --2- 172.21.15.99:0/3703681745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4748154a70 0x7f47480b24a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:54.793 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.790+0000 7f474d82d700 1 --2- 172.21.15.99:0/3703681745 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f474816b2c0 0x7f4748157280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:54.793 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.790+0000 7f474d82d700 1 -- 172.21.15.99:0/3703681745 >> 172.21.15.99:0/3703681745 conn(0x7f47480d1a50 msgr2=0x7f474815a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:54.793 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.790+0000 7f474d82d700 1 -- 172.21.15.99:0/3703681745 shutdown_connections 2023-10-08T00:11:54.793 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.790+0000 7f474d82d700 1 -- 172.21.15.99:0/3703681745 wait complete. 2023-10-08T00:11:54.793 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 17 2023-10-08T00:11:54.803 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:11:54.917 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.916+0000 7f45bdca3700 1 Processor -- start 2023-10-08T00:11:54.917 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.916+0000 7f45bdca3700 1 -- start start 2023-10-08T00:11:54.918 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.917+0000 7f45bdca3700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f45b80ae2f0 0x7f45b80ae790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:54.918 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.917+0000 7f45bdca3700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45b80b1ff0 con 0x7f45b80b2460 2023-10-08T00:11:54.918 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.917+0000 7f45bdca3700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45b80aed70 con 0x7f45b80afc00 2023-10-08T00:11:54.919 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.917+0000 7f45bdca3700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45b81580e0 con 0x7f45b80ae2f0 2023-10-08T00:11:54.919 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.917+0000 7f45affff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f45b80ae2f0 0x7f45b80ae790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:54.919 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.917+0000 7f45bd4a2700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f45b80b2460 0x7f45b80b2890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56358/0 (socket says 172.21.15.99:56358) 2023-10-08T00:11:54.919 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.917+0000 7f45bd4a2700 1 -- 172.21.15.99:0/1094160618 learned_addr learned my addr 172.21.15.99:0/1094160618 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:54.919 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.917+0000 7f45affff700 1 -- 172.21.15.99:0/1094160618 >> v1:172.21.15.191:6789/0 conn(0x7f45b80afc00 legacy=0x7f45b80affd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:54.919 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.917+0000 7f45affff700 1 -- 172.21.15.99:0/1094160618 >> v1:172.21.15.99:6789/0 conn(0x7f45b80b2460 legacy=0x7f45b80b2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:54.919 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.917+0000 7f45affff700 1 -- 172.21.15.99:0/1094160618 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b80af050 con 0x7f45b80ae2f0 2023-10-08T00:11:54.919 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.918+0000 7f45affff700 1 --2- 172.21.15.99:0/1094160618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45b80ae2f0 0x7f45b80ae790 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f45a80098e0 tx=0x7f45a80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=126af0fb8c3aafbe server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:54.919 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.918+0000 7f45af7fe700 1 -- 172.21.15.99:0/1094160618 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a8009e80 con 0x7f45b80ae2f0 2023-10-08T00:11:54.919 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.918+0000 7f45af7fe700 1 -- 172.21.15.99:0/1094160618 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45a800b860 con 0x7f45b80ae2f0 2023-10-08T00:11:54.919 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.918+0000 7f45af7fe700 1 -- 172.21.15.99:0/1094160618 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a800bbe0 con 0x7f45b80ae2f0 2023-10-08T00:11:54.922 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.918+0000 7f45bdca3700 1 -- 172.21.15.99:0/1094160618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45b80ae2f0 msgr2=0x7f45b80ae790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:54.922 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.918+0000 7f45bdca3700 1 --2- 172.21.15.99:0/1094160618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45b80ae2f0 0x7f45b80ae790 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f45a80098e0 tx=0x7f45a80091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:54.922 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.918+0000 7f45bdca3700 1 -- 172.21.15.99:0/1094160618 shutdown_connections 2023-10-08T00:11:54.922 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.918+0000 7f45bdca3700 1 --2- 172.21.15.99:0/1094160618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45b80ae2f0 0x7f45b80ae790 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:54.922 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.918+0000 7f45bdca3700 1 -- 172.21.15.99:0/1094160618 >> 172.21.15.99:0/1094160618 conn(0x7f45b806c700 msgr2=0x7f45b8069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:54.923 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.918+0000 7f45bdca3700 1 -- 172.21.15.99:0/1094160618 shutdown_connections 2023-10-08T00:11:54.923 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.918+0000 7f45bdca3700 1 -- 172.21.15.99:0/1094160618 wait complete. 2023-10-08T00:11:54.923 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.919+0000 7f45bdca3700 1 Processor -- start 2023-10-08T00:11:54.923 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.920+0000 7f45bdca3700 1 -- start start 2023-10-08T00:11:54.923 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.920+0000 7f45bdca3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f45b80ae2f0 0x7f45b8169540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:54.923 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.920+0000 7f45bdca3700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f45b80afc00 0x7f45b816b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:54.923 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.920+0000 7f45bdca3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45b80b2460 0x7f45b80a9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:54.923 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.920+0000 7f45bdca3700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b81540a0 con 0x7f45b80b2460 2023-10-08T00:11:54.923 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.920+0000 7f45bdca3700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b8168720 con 0x7f45b80afc00 2023-10-08T00:11:54.923 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.920+0000 7f45bdca3700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45b80abb20 con 0x7f45b80ae2f0 2023-10-08T00:11:54.923 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.920+0000 7f45bd4a2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45b80b2460 0x7f45b80a9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:54.923 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.920+0000 7f45bd4a2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45b80b2460 0x7f45b80a9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58148/0 (socket says 172.21.15.99:58148) 2023-10-08T00:11:54.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.920+0000 7f45bd4a2700 1 -- 172.21.15.99:0/2212317696 learned_addr learned my addr 172.21.15.99:0/2212317696 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:54.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.921+0000 7f45affff700 1 --2- 172.21.15.99:0/2212317696 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f45b80afc00 0x7f45b816b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:54.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.921+0000 7f45bcca1700 1 --2- 172.21.15.99:0/2212317696 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f45b80ae2f0 0x7f45b8169540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:54.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.921+0000 7f45bd4a2700 1 -- 172.21.15.99:0/2212317696 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f45b80ae2f0 msgr2=0x7f45b8169540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:54.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.921+0000 7f45bd4a2700 1 --2- 172.21.15.99:0/2212317696 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f45b80ae2f0 0x7f45b8169540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:54.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.921+0000 7f45bd4a2700 1 -- 172.21.15.99:0/2212317696 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f45b80afc00 msgr2=0x7f45b816b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:54.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.921+0000 7f45bd4a2700 1 --2- 172.21.15.99:0/2212317696 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f45b80afc00 0x7f45b816b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:54.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.921+0000 7f45bd4a2700 1 -- 172.21.15.99:0/2212317696 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a8009420 con 0x7f45b80b2460 2023-10-08T00:11:54.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.921+0000 7f45bd4a2700 1 --2- 172.21.15.99:0/2212317696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45b80b2460 0x7f45b80a9500 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f45b000c560 tx=0x7f45b0007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:54.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.921+0000 7f45acff9700 1 -- 172.21.15.99:0/2212317696 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b0005b30 con 0x7f45b80b2460 2023-10-08T00:11:54.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.921+0000 7f45acff9700 1 -- 172.21.15.99:0/2212317696 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45b0005c90 con 0x7f45b80b2460 2023-10-08T00:11:54.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.921+0000 7f45bdca3700 1 -- 172.21.15.99:0/2212317696 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45b80aa0e0 con 0x7f45b80b2460 2023-10-08T00:11:54.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.921+0000 7f45acff9700 1 -- 172.21.15.99:0/2212317696 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b00072e0 con 0x7f45b80b2460 2023-10-08T00:11:54.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.921+0000 7f45bdca3700 1 -- 172.21.15.99:0/2212317696 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45b80abda0 con 0x7f45b80b2460 2023-10-08T00:11:54.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.922+0000 7f45bdca3700 1 -- 172.21.15.99:0/2212317696 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45b80aa5e0 con 0x7f45b80b2460 2023-10-08T00:11:54.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.922+0000 7f45acff9700 1 -- 172.21.15.99:0/2212317696 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f45b0007690 con 0x7f45b80b2460 2023-10-08T00:11:54.926 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.923+0000 7f45acff9700 1 --2- 172.21.15.99:0/2212317696 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f45a4066f30 0x7f45a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:54.926 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.923+0000 7f45acff9700 1 -- 172.21.15.99:0/2212317696 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f45b00838f0 con 0x7f45b80b2460 2023-10-08T00:11:54.926 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.923+0000 7f45bcca1700 1 --2- 172.21.15.99:0/2212317696 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f45a4066f30 0x7f45a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:54.926 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.923+0000 7f45bcca1700 1 --2- 172.21.15.99:0/2212317696 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f45a4066f30 0x7f45a40693e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f45b4007fd0 tx=0x7f45b4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:54.927 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:54.927+0000 7f45acff9700 1 -- 172.21.15.99:0/2212317696 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f45b004b5a0 con 0x7f45b80b2460 2023-10-08T00:11:55.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.108+0000 7f45bdca3700 1 -- 172.21.15.99:0/2212317696 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f45b80ce410 con 0x7f45b80b2460 2023-10-08T00:11:55.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.109+0000 7f45acff9700 1 -- 172.21.15.99:0/2212317696 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f45b0017020 con 0x7f45b80b2460 2023-10-08T00:11:55.110 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:11:55.110 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":33,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:11:52.347370+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:11:38.415553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-08T00:11:39.634576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:11:55.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.111+0000 7f45bdca3700 1 -- 172.21.15.99:0/2212317696 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f45a4066f30 msgr2=0x7f45a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:55.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.111+0000 7f45bdca3700 1 --2- 172.21.15.99:0/2212317696 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f45a4066f30 0x7f45a40693e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f45b4007fd0 tx=0x7f45b4009040 comp rx=0 tx=0).stop 2023-10-08T00:11:55.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.111+0000 7f45bdca3700 1 -- 172.21.15.99:0/2212317696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45b80b2460 msgr2=0x7f45b80a9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:55.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.111+0000 7f45bdca3700 1 --2- 172.21.15.99:0/2212317696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45b80b2460 0x7f45b80a9500 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f45b000c560 tx=0x7f45b0007f60 comp rx=0 tx=0).stop 2023-10-08T00:11:55.113 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.111+0000 7f45bdca3700 1 -- 172.21.15.99:0/2212317696 shutdown_connections 2023-10-08T00:11:55.113 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.111+0000 7f45bdca3700 1 --2- 172.21.15.99:0/2212317696 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f45a4066f30 0x7f45a40693e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:55.113 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.111+0000 7f45bdca3700 1 --2- 172.21.15.99:0/2212317696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45b80b2460 0x7f45b80a9500 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:55.113 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.111+0000 7f45bdca3700 1 --2- 172.21.15.99:0/2212317696 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f45b80ae2f0 0x7f45b8169540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:55.113 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.111+0000 7f45bdca3700 1 --2- 172.21.15.99:0/2212317696 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f45b80afc00 0x7f45b816b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:55.113 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.111+0000 7f45bdca3700 1 -- 172.21.15.99:0/2212317696 >> 172.21.15.99:0/2212317696 conn(0x7f45b806c700 msgr2=0x7f45b806fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:55.113 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.111+0000 7f45bdca3700 1 -- 172.21.15.99:0/2212317696 shutdown_connections 2023-10-08T00:11:55.113 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.111+0000 7f45bdca3700 1 -- 172.21.15.99:0/2212317696 wait complete. 2023-10-08T00:11:55.124 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:10:42.063344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T00:11:38.415553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '30', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T00:11:39.634576+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '31', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T00:11:55.125 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T00:11:55.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.241+0000 7f1046780700 1 Processor -- start 2023-10-08T00:11:55.242 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.241+0000 7f1046780700 1 -- start start 2023-10-08T00:11:55.243 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.242+0000 7f1046780700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f10400b6580 0x7f104006a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:55.243 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.242+0000 7f1046780700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f104006a830 0x7f10400b5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:55.243 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.242+0000 7f1046780700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f104006ac90 con 0x7f10400cce30 2023-10-08T00:11:55.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.242+0000 7f1046780700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10400b4d70 con 0x7f10400b6580 2023-10-08T00:11:55.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.242+0000 7f1046780700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f104015b3b0 con 0x7f104006a830 2023-10-08T00:11:55.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.242+0000 7f1044f7d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f10400b6580 0x7f104006a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:55.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.242+0000 7f1045f7f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f104006a830 0x7f10400b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:55.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.242+0000 7f1045f7f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f104006a830 0x7f10400b5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.99:56518/0 (socket says 172.21.15.99:56518) 2023-10-08T00:11:55.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.242+0000 7f1044f7d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f10400b6580 0x7f104006a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.99:56600/0 (socket says 172.21.15.99:56600) 2023-10-08T00:11:55.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.242+0000 7f1045f7f700 1 -- 172.21.15.99:0/3285360753 learned_addr learned my addr 172.21.15.99:0/3285360753 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:55.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.242+0000 7f104577e700 1 --1- 172.21.15.99:0/3285360753 >> v1:172.21.15.191:6789/0 conn(0x7f10400cce30 0x7f104015aca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:36242/0 (socket says 172.21.15.99:36242) 2023-10-08T00:11:55.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.242+0000 7f1045f7f700 1 -- 172.21.15.99:0/3285360753 >> v2:172.21.15.191:3300/0 conn(0x7f10400b6580 msgr2=0x7f104006a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:55.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.242+0000 7f1045f7f700 1 --2- 172.21.15.99:0/3285360753 >> v2:172.21.15.191:3300/0 conn(0x7f10400b6580 0x7f104006a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:55.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.242+0000 7f1045f7f700 1 -- 172.21.15.99:0/3285360753 >> v1:172.21.15.191:6789/0 conn(0x7f10400cce30 legacy=0x7f104015aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:55.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.242+0000 7f1045f7f700 1 -- 172.21.15.99:0/3285360753 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10400cd8d0 con 0x7f104006a830 2023-10-08T00:11:55.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.243+0000 7f1045f7f700 1 --2- 172.21.15.99:0/3285360753 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f104006a830 0x7f10400b5200 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f103c009f20 tx=0x7f103c009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=695ad11c6d43bc0d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:55.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.243+0000 7f1037fff700 1 -- 172.21.15.99:0/3285360753 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103c00e070 con 0x7f104006a830 2023-10-08T00:11:55.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.243+0000 7f1037fff700 1 -- 172.21.15.99:0/3285360753 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f103c00b9d0 con 0x7f104006a830 2023-10-08T00:11:55.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.243+0000 7f1037fff700 1 -- 172.21.15.99:0/3285360753 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103c0092e0 con 0x7f104006a830 2023-10-08T00:11:55.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.243+0000 7f1046780700 1 -- 172.21.15.99:0/3285360753 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f104006a830 msgr2=0x7f10400b5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:55.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.243+0000 7f1046780700 1 --2- 172.21.15.99:0/3285360753 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f104006a830 0x7f10400b5200 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f103c009f20 tx=0x7f103c009840 comp rx=0 tx=0).stop 2023-10-08T00:11:55.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.243+0000 7f1046780700 1 -- 172.21.15.99:0/3285360753 shutdown_connections 2023-10-08T00:11:55.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.243+0000 7f1046780700 1 --2- 172.21.15.99:0/3285360753 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f104006a830 0x7f10400b5200 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:55.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.243+0000 7f1046780700 1 --2- 172.21.15.99:0/3285360753 >> v2:172.21.15.191:3300/0 conn(0x7f10400b6580 0x7f104006a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:55.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.243+0000 7f1046780700 1 -- 172.21.15.99:0/3285360753 >> 172.21.15.99:0/3285360753 conn(0x7f1040068cb0 msgr2=0x7f10400690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:55.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.243+0000 7f1046780700 1 -- 172.21.15.99:0/3285360753 shutdown_connections 2023-10-08T00:11:55.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.243+0000 7f1046780700 1 -- 172.21.15.99:0/3285360753 wait complete. 2023-10-08T00:11:55.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.244+0000 7f1046780700 1 Processor -- start 2023-10-08T00:11:55.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.244+0000 7f1046780700 1 -- start start 2023-10-08T00:11:55.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.245+0000 7f1046780700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f104006a830 0x7f10401435f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:55.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.245+0000 7f1046780700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f10400b6580 0x7f104015c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:55.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.245+0000 7f1046780700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10400cce30 0x7f104015ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:55.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.245+0000 7f1046780700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f104016b140 con 0x7f10400cce30 2023-10-08T00:11:55.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.245+0000 7f1046780700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f104014fac0 con 0x7f10400b6580 2023-10-08T00:11:55.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.245+0000 7f1046780700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1040151ad0 con 0x7f104006a830 2023-10-08T00:11:55.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.245+0000 7f1045f7f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10400cce30 0x7f104015ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:55.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.245+0000 7f1045f7f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10400cce30 0x7f104015ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58152/0 (socket says 172.21.15.99:58152) 2023-10-08T00:11:55.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.245+0000 7f1045f7f700 1 -- 172.21.15.99:0/897077864 learned_addr learned my addr 172.21.15.99:0/897077864 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:55.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.246+0000 7f104577e700 1 --2- 172.21.15.99:0/897077864 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f104006a830 0x7f10401435f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:55.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.246+0000 7f1045f7f700 1 -- 172.21.15.99:0/897077864 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f104006a830 msgr2=0x7f10401435f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:55.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.246+0000 7f1045f7f700 1 --2- 172.21.15.99:0/897077864 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f104006a830 0x7f10401435f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:55.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.246+0000 7f1045f7f700 1 -- 172.21.15.99:0/897077864 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f10400b6580 msgr2=0x7f104015c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:55.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.246+0000 7f1045f7f700 1 --2- 172.21.15.99:0/897077864 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f10400b6580 0x7f104015c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:55.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.246+0000 7f1045f7f700 1 -- 172.21.15.99:0/897077864 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f103c009ae0 con 0x7f10400cce30 2023-10-08T00:11:55.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.246+0000 7f1045f7f700 1 --2- 172.21.15.99:0/897077864 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10400cce30 0x7f104015ce80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f103c00b610 tx=0x7f103c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:55.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.246+0000 7f10357fa700 1 -- 172.21.15.99:0/897077864 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103c008c30 con 0x7f10400cce30 2023-10-08T00:11:55.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.246+0000 7f10357fa700 1 -- 172.21.15.99:0/897077864 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f103c008dd0 con 0x7f10400cce30 2023-10-08T00:11:55.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.246+0000 7f1046780700 1 -- 172.21.15.99:0/897077864 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f104016ae60 con 0x7f10400cce30 2023-10-08T00:11:55.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.246+0000 7f10357fa700 1 -- 172.21.15.99:0/897077864 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103c0266b0 con 0x7f10400cce30 2023-10-08T00:11:55.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.246+0000 7f1046780700 1 -- 172.21.15.99:0/897077864 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1040151760 con 0x7f10400cce30 2023-10-08T00:11:55.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.247+0000 7f1046780700 1 -- 172.21.15.99:0/897077864 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10401512e0 con 0x7f10400cce30 2023-10-08T00:11:55.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.247+0000 7f10357fa700 1 -- 172.21.15.99:0/897077864 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f103c003040 con 0x7f10400cce30 2023-10-08T00:11:55.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.248+0000 7f10357fa700 1 --2- 172.21.15.99:0/897077864 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1024066f30 0x7f10240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:55.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.248+0000 7f10357fa700 1 -- 172.21.15.99:0/897077864 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f103c086570 con 0x7f10400cce30 2023-10-08T00:11:55.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.248+0000 7f104577e700 1 --2- 172.21.15.99:0/897077864 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1024066f30 0x7f10240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:55.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.248+0000 7f104577e700 1 --2- 172.21.15.99:0/897077864 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1024066f30 0x7f10240693e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1038007120 tx=0x7f1038000bc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:55.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.252+0000 7f10357fa700 1 -- 172.21.15.99:0/897077864 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f10401512e0 con 0x7f10400cce30 2023-10-08T00:11:55.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.436+0000 7f1046780700 1 -- 172.21.15.99:0/897077864 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f10401512e0 con 0x7f10400cce30 2023-10-08T00:11:55.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.436+0000 7f10357fa700 1 -- 172.21.15.99:0/897077864 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) v1 ==== 93+0+10 (secure 0 0 0) 0x7f10401512e0 con 0x7f10400cce30 2023-10-08T00:11:55.437 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2023-10-08T00:11:55.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.438+0000 7f1046780700 1 -- 172.21.15.99:0/897077864 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1024066f30 msgr2=0x7f10240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:55.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.438+0000 7f1046780700 1 --2- 172.21.15.99:0/897077864 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1024066f30 0x7f10240693e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1038007120 tx=0x7f1038000bc0 comp rx=0 tx=0).stop 2023-10-08T00:11:55.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.438+0000 7f1046780700 1 -- 172.21.15.99:0/897077864 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10400cce30 msgr2=0x7f104015ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:55.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.438+0000 7f1046780700 1 --2- 172.21.15.99:0/897077864 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10400cce30 0x7f104015ce80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f103c00b610 tx=0x7f103c018000 comp rx=0 tx=0).stop 2023-10-08T00:11:55.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.438+0000 7f1046780700 1 -- 172.21.15.99:0/897077864 shutdown_connections 2023-10-08T00:11:55.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.438+0000 7f1046780700 1 --2- 172.21.15.99:0/897077864 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1024066f30 0x7f10240693e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:55.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.438+0000 7f1046780700 1 --2- 172.21.15.99:0/897077864 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10400cce30 0x7f104015ce80 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:55.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.438+0000 7f1046780700 1 --2- 172.21.15.99:0/897077864 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f104006a830 0x7f10401435f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:55.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.438+0000 7f1046780700 1 --2- 172.21.15.99:0/897077864 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f10400b6580 0x7f104015c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:55.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.438+0000 7f1046780700 1 -- 172.21.15.99:0/897077864 >> 172.21.15.99:0/897077864 conn(0x7f1040068cb0 msgr2=0x7f1040150500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:55.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.438+0000 7f1046780700 1 -- 172.21.15.99:0/897077864 shutdown_connections 2023-10-08T00:11:55.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.438+0000 7f1046780700 1 -- 172.21.15.99:0/897077864 wait complete. 2023-10-08T00:11:55.452 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T00:11:55.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.568+0000 7f35b5b6e700 1 Processor -- start 2023-10-08T00:11:55.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.568+0000 7f35b5b6e700 1 -- start start 2023-10-08T00:11:55.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.569+0000 7f35b5b6e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f35b00cce30 0x7f35b015aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:55.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.569+0000 7f35b5b6e700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f35b00b6580 0x7f35b006a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:55.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.569+0000 7f35b5b6e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35b006ac90 con 0x7f35b006a830 2023-10-08T00:11:55.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.569+0000 7f35b5b6e700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35b00b4d70 con 0x7f35b00b6580 2023-10-08T00:11:55.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.569+0000 7f35b5b6e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35b00cd8d0 con 0x7f35b00cce30 2023-10-08T00:11:55.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.569+0000 7f35affff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f35b00b6580 0x7f35b006a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:55.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.569+0000 7f35affff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f35b00b6580 0x7f35b006a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58156/0 (socket says 172.21.15.99:58156) 2023-10-08T00:11:55.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.569+0000 7f35affff700 1 -- 172.21.15.99:0/395902750 learned_addr learned my addr 172.21.15.99:0/395902750 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:55.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.569+0000 7f35b4b6c700 1 --2- 172.21.15.99:0/395902750 >> v2:172.21.15.191:3300/0 conn(0x7f35b00cce30 0x7f35b015aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:55.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.569+0000 7f35affff700 1 -- 172.21.15.99:0/395902750 >> v2:172.21.15.191:3300/0 conn(0x7f35b00cce30 msgr2=0x7f35b015aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:55.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.569+0000 7f35affff700 1 --2- 172.21.15.99:0/395902750 >> v2:172.21.15.191:3300/0 conn(0x7f35b00cce30 0x7f35b015aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:55.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.569+0000 7f35affff700 1 -- 172.21.15.99:0/395902750 >> v1:172.21.15.191:6789/0 conn(0x7f35b006a830 legacy=0x7f35b015d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:55.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.569+0000 7f35affff700 1 -- 172.21.15.99:0/395902750 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b00b4440 con 0x7f35b00b6580 2023-10-08T00:11:55.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.569+0000 7f35affff700 1 --2- 172.21.15.99:0/395902750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35b00b6580 0x7f35b006a2f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f35b00b3770 tx=0x7f359c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10f5348160bfc4fb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:55.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.569+0000 7f35af7fe700 1 -- 172.21.15.99:0/395902750 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359c00b860 con 0x7f35b00b6580 2023-10-08T00:11:55.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.570+0000 7f35af7fe700 1 -- 172.21.15.99:0/395902750 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f359c00b9c0 con 0x7f35b00b6580 2023-10-08T00:11:55.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.570+0000 7f35af7fe700 1 -- 172.21.15.99:0/395902750 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359c00bd60 con 0x7f35b00b6580 2023-10-08T00:11:55.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.570+0000 7f35b5b6e700 1 -- 172.21.15.99:0/395902750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35b00b6580 msgr2=0x7f35b006a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:55.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.570+0000 7f35b5b6e700 1 --2- 172.21.15.99:0/395902750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35b00b6580 0x7f35b006a2f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f35b00b3770 tx=0x7f359c009250 comp rx=0 tx=0).stop 2023-10-08T00:11:55.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.570+0000 7f35b5b6e700 1 -- 172.21.15.99:0/395902750 shutdown_connections 2023-10-08T00:11:55.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.570+0000 7f35b5b6e700 1 --2- 172.21.15.99:0/395902750 >> v2:172.21.15.191:3300/0 conn(0x7f35b00cce30 0x7f35b015aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:55.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.570+0000 7f35b5b6e700 1 --2- 172.21.15.99:0/395902750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35b00b6580 0x7f35b006a2f0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:55.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.570+0000 7f35b5b6e700 1 -- 172.21.15.99:0/395902750 >> 172.21.15.99:0/395902750 conn(0x7f35b0068cb0 msgr2=0x7f35b00690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:55.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.570+0000 7f35b5b6e700 1 -- 172.21.15.99:0/395902750 shutdown_connections 2023-10-08T00:11:55.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.570+0000 7f35b5b6e700 1 -- 172.21.15.99:0/395902750 wait complete. 2023-10-08T00:11:55.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.571+0000 7f35b5b6e700 1 Processor -- start 2023-10-08T00:11:55.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.571+0000 7f35b5b6e700 1 -- start start 2023-10-08T00:11:55.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.572+0000 7f35b5b6e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35b006a830 0x7f35b0143710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:55.577 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.572+0000 7f35b5b6e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f35b00b6580 0x7f35b0152480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:55.577 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.572+0000 7f35b5b6e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35b00cce30 0x7f35b00b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:55.577 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.572+0000 7f35b5b6e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b0156060 con 0x7f35b00cce30 2023-10-08T00:11:55.577 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.572+0000 7f35b5b6e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b0143c50 con 0x7f35b006a830 2023-10-08T00:11:55.577 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.572+0000 7f35b5b6e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35b016d550 con 0x7f35b00b6580 2023-10-08T00:11:55.577 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.572+0000 7f35b536d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35b00cce30 0x7f35b00b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:55.577 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.572+0000 7f35b536d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35b00cce30 0x7f35b00b25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58160/0 (socket says 172.21.15.99:58160) 2023-10-08T00:11:55.577 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.572+0000 7f35b536d700 1 -- 172.21.15.99:0/2611435410 learned_addr learned my addr 172.21.15.99:0/2611435410 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:55.577 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.572+0000 7f35b4b6c700 1 --2- 172.21.15.99:0/2611435410 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35b006a830 0x7f35b0143710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:55.577 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.572+0000 7f35affff700 1 --2- 172.21.15.99:0/2611435410 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f35b00b6580 0x7f35b0152480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:55.577 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.572+0000 7f35b536d700 1 -- 172.21.15.99:0/2611435410 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f35b00b6580 msgr2=0x7f35b0152480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:55.578 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.572+0000 7f35b536d700 1 --2- 172.21.15.99:0/2611435410 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f35b00b6580 0x7f35b0152480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:55.578 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.572+0000 7f35b536d700 1 -- 172.21.15.99:0/2611435410 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35b006a830 msgr2=0x7f35b0143710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:55.578 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.572+0000 7f35b536d700 1 --2- 172.21.15.99:0/2611435410 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35b006a830 0x7f35b0143710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:55.578 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.572+0000 7f35b536d700 1 -- 172.21.15.99:0/2611435410 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f359c0094f0 con 0x7f35b00cce30 2023-10-08T00:11:55.578 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.573+0000 7f35b536d700 1 --2- 172.21.15.99:0/2611435410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35b00cce30 0x7f35b00b25d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f35a800c8e0 tx=0x7f35a8001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:55.578 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.573+0000 7f35acff9700 1 -- 172.21.15.99:0/2611435410 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a8012630 con 0x7f35b00cce30 2023-10-08T00:11:55.578 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.573+0000 7f35acff9700 1 -- 172.21.15.99:0/2611435410 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a8012790 con 0x7f35b00cce30 2023-10-08T00:11:55.578 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.573+0000 7f35acff9700 1 -- 172.21.15.99:0/2611435410 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a8012b50 con 0x7f35b00cce30 2023-10-08T00:11:55.578 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.573+0000 7f35b5b6e700 1 -- 172.21.15.99:0/2611435410 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35b01518a0 con 0x7f35b00cce30 2023-10-08T00:11:55.578 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.573+0000 7f35b5b6e700 1 -- 172.21.15.99:0/2611435410 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35b0151e60 con 0x7f35b00cce30 2023-10-08T00:11:55.578 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.574+0000 7f35b5b6e700 1 -- 172.21.15.99:0/2611435410 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35b0089550 con 0x7f35b00cce30 2023-10-08T00:11:55.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.574+0000 7f35acff9700 1 -- 172.21.15.99:0/2611435410 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f35a801a3a0 con 0x7f35b00cce30 2023-10-08T00:11:55.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.574+0000 7f35acff9700 1 --2- 172.21.15.99:0/2611435410 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3594066f30 0x7f35940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:55.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.574+0000 7f35acff9700 1 -- 172.21.15.99:0/2611435410 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f35a8084580 con 0x7f35b00cce30 2023-10-08T00:11:55.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.575+0000 7f35b4b6c700 1 --2- 172.21.15.99:0/2611435410 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3594066f30 0x7f35940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:55.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.575+0000 7f35b4b6c700 1 --2- 172.21.15.99:0/2611435410 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3594066f30 0x7f35940693e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f35a4006fd0 tx=0x7f35a4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:55.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.578+0000 7f35acff9700 1 -- 172.21.15.99:0/2611435410 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f35a80517f0 con 0x7f35b00cce30 2023-10-08T00:11:55.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.765+0000 7f35b5b6e700 1 -- 172.21.15.99:0/2611435410 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f35b006ac90 con 0x7f35b00cce30 2023-10-08T00:11:55.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.766+0000 7f35acff9700 1 -- 172.21.15.99:0/2611435410 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) v1 ==== 104+0+11 (secure 0 0 0) 0x7f35b006ac90 con 0x7f35b00cce30 2023-10-08T00:11:55.767 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2023-10-08T00:11:55.769 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.768+0000 7f35b5b6e700 1 -- 172.21.15.99:0/2611435410 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3594066f30 msgr2=0x7f35940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:55.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.768+0000 7f35b5b6e700 1 --2- 172.21.15.99:0/2611435410 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3594066f30 0x7f35940693e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f35a4006fd0 tx=0x7f35a4008040 comp rx=0 tx=0).stop 2023-10-08T00:11:55.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.768+0000 7f35b5b6e700 1 -- 172.21.15.99:0/2611435410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35b00cce30 msgr2=0x7f35b00b25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:55.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.768+0000 7f35b5b6e700 1 --2- 172.21.15.99:0/2611435410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35b00cce30 0x7f35b00b25d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f35a800c8e0 tx=0x7f35a8001710 comp rx=0 tx=0).stop 2023-10-08T00:11:55.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.768+0000 7f35b5b6e700 1 -- 172.21.15.99:0/2611435410 shutdown_connections 2023-10-08T00:11:55.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.768+0000 7f35b5b6e700 1 --2- 172.21.15.99:0/2611435410 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3594066f30 0x7f35940693e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:55.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.768+0000 7f35b5b6e700 1 --2- 172.21.15.99:0/2611435410 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35b006a830 0x7f35b0143710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:55.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.768+0000 7f35b5b6e700 1 --2- 172.21.15.99:0/2611435410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35b00cce30 0x7f35b00b25d0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:55.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.768+0000 7f35b5b6e700 1 --2- 172.21.15.99:0/2611435410 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f35b00b6580 0x7f35b0152480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:55.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.768+0000 7f35b5b6e700 1 -- 172.21.15.99:0/2611435410 >> 172.21.15.99:0/2611435410 conn(0x7f35b0068cb0 msgr2=0x7f35b0150500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:55.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.768+0000 7f35b5b6e700 1 -- 172.21.15.99:0/2611435410 shutdown_connections 2023-10-08T00:11:55.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.768+0000 7f35b5b6e700 1 -- 172.21.15.99:0/2611435410 wait complete. 2023-10-08T00:11:55.780 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T00:11:55.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.896+0000 7fb9506b6700 1 Processor -- start 2023-10-08T00:11:55.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.896+0000 7fb9506b6700 1 -- start start 2023-10-08T00:11:55.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.897+0000 7fb9506b6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb948159ec0 0x7fb9480af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:55.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.897+0000 7fb9506b6700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb94815a290 0x7fb9480afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:55.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.897+0000 7fb9506b6700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9480b0790 con 0x7fb94806a5a0 2023-10-08T00:11:55.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.897+0000 7fb9506b6700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9480ce1f0 con 0x7fb94815a290 2023-10-08T00:11:55.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.897+0000 7fb9506b6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9480b0b20 con 0x7fb948159ec0 2023-10-08T00:11:55.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.897+0000 7fb94ec53700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fb94806a5a0 0x7fb94815db90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56364/0 (socket says 172.21.15.99:56364) 2023-10-08T00:11:55.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.897+0000 7fb94ec53700 1 -- 172.21.15.99:0/1708170571 learned_addr learned my addr 172.21.15.99:0/1708170571 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:55.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.897+0000 7fb94dc51700 1 --2- 172.21.15.99:0/1708170571 >> v2:172.21.15.191:3300/0 conn(0x7fb94815a290 0x7fb9480afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:55.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.897+0000 7fb94e452700 1 --2- 172.21.15.99:0/1708170571 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb948159ec0 0x7fb9480af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:55.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.898+0000 7fb94d450700 1 -- 172.21.15.99:0/1708170571 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3281716673 0 0) 0x7fb9480b0790 con 0x7fb94806a5a0 2023-10-08T00:11:55.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.898+0000 7fb94d450700 1 -- 172.21.15.99:0/1708170571 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb934003570 con 0x7fb94806a5a0 2023-10-08T00:11:55.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.898+0000 7fb94dc51700 1 -- 172.21.15.99:0/1708170571 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb948159ec0 msgr2=0x7fb9480af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:55.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.898+0000 7fb94dc51700 1 --2- 172.21.15.99:0/1708170571 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb948159ec0 0x7fb9480af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:55.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.898+0000 7fb94dc51700 1 -- 172.21.15.99:0/1708170571 >> v1:172.21.15.99:6789/0 conn(0x7fb94806a5a0 legacy=0x7fb94815db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:55.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.898+0000 7fb94dc51700 1 -- 172.21.15.99:0/1708170571 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9480b04b0 con 0x7fb94815a290 2023-10-08T00:11:55.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.898+0000 7fb94dc51700 1 --2- 172.21.15.99:0/1708170571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb94815a290 0x7fb9480afe80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb9380098e0 tx=0x7fb9380091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d88589deec43045 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:55.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.898+0000 7fb94d450700 1 -- 172.21.15.99:0/1708170571 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb938011630 con 0x7fb94815a290 2023-10-08T00:11:55.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.898+0000 7fb94d450700 1 -- 172.21.15.99:0/1708170571 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb938011790 con 0x7fb94815a290 2023-10-08T00:11:55.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.898+0000 7fb94d450700 1 -- 172.21.15.99:0/1708170571 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb938011b10 con 0x7fb94815a290 2023-10-08T00:11:55.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.899+0000 7fb9506b6700 1 -- 172.21.15.99:0/1708170571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb94815a290 msgr2=0x7fb9480afe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:55.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.899+0000 7fb9506b6700 1 --2- 172.21.15.99:0/1708170571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb94815a290 0x7fb9480afe80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb9380098e0 tx=0x7fb9380091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:55.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.899+0000 7fb9506b6700 1 -- 172.21.15.99:0/1708170571 shutdown_connections 2023-10-08T00:11:55.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.899+0000 7fb9506b6700 1 --2- 172.21.15.99:0/1708170571 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb948159ec0 0x7fb9480af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:55.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.899+0000 7fb9506b6700 1 --2- 172.21.15.99:0/1708170571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb94815a290 0x7fb9480afe80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:55.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.899+0000 7fb9506b6700 1 -- 172.21.15.99:0/1708170571 >> 172.21.15.99:0/1708170571 conn(0x7fb9480d1880 msgr2=0x7fb9480cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:55.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.899+0000 7fb9506b6700 1 -- 172.21.15.99:0/1708170571 shutdown_connections 2023-10-08T00:11:55.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.899+0000 7fb9506b6700 1 -- 172.21.15.99:0/1708170571 wait complete. 2023-10-08T00:11:55.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.900+0000 7fb9506b6700 1 Processor -- start 2023-10-08T00:11:55.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.900+0000 7fb9506b6700 1 -- start start 2023-10-08T00:11:55.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.900+0000 7fb9506b6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb94806a5a0 0x7fb9480ca090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:55.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.900+0000 7fb9506b6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb948159ec0 0x7fb94814f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:55.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.900+0000 7fb9506b6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb94815a290 0x7fb94814ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:55.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.900+0000 7fb9506b6700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94814f5d0 con 0x7fb948159ec0 2023-10-08T00:11:55.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.900+0000 7fb9506b6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94814f8a0 con 0x7fb94815a290 2023-10-08T00:11:55.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.900+0000 7fb9506b6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb94814fb80 con 0x7fb94806a5a0 2023-10-08T00:11:55.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.901+0000 7fb94dc51700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb948159ec0 0x7fb94814f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:55.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.901+0000 7fb94dc51700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb948159ec0 0x7fb94814f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58170/0 (socket says 172.21.15.99:58170) 2023-10-08T00:11:55.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.901+0000 7fb94dc51700 1 -- 172.21.15.99:0/184800362 learned_addr learned my addr 172.21.15.99:0/184800362 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:55.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.901+0000 7fb94ec53700 1 --2- 172.21.15.99:0/184800362 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb94815a290 0x7fb94814ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:55.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.901+0000 7fb94e452700 1 --2- 172.21.15.99:0/184800362 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb94806a5a0 0x7fb9480ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:55.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.901+0000 7fb94dc51700 1 -- 172.21.15.99:0/184800362 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb94806a5a0 msgr2=0x7fb9480ca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:55.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.901+0000 7fb94dc51700 1 --2- 172.21.15.99:0/184800362 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb94806a5a0 0x7fb9480ca090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:55.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.901+0000 7fb94dc51700 1 -- 172.21.15.99:0/184800362 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb94815a290 msgr2=0x7fb94814ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:55.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.901+0000 7fb94dc51700 1 --2- 172.21.15.99:0/184800362 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb94815a290 0x7fb94814ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:55.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.901+0000 7fb94dc51700 1 -- 172.21.15.99:0/184800362 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb938009420 con 0x7fb948159ec0 2023-10-08T00:11:55.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.901+0000 7fb94ec53700 1 --2- 172.21.15.99:0/184800362 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb94815a290 0x7fb94814ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:11:55.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.901+0000 7fb94dc51700 1 --2- 172.21.15.99:0/184800362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb948159ec0 0x7fb94814f090 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb948071eb0 tx=0x7fb938017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:55.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.901+0000 7fb93e7fc700 1 -- 172.21.15.99:0/184800362 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb938011b10 con 0x7fb948159ec0 2023-10-08T00:11:55.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.901+0000 7fb93e7fc700 1 -- 172.21.15.99:0/184800362 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb938000cc0 con 0x7fb948159ec0 2023-10-08T00:11:55.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.901+0000 7fb9506b6700 1 -- 172.21.15.99:0/184800362 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb94814fe60 con 0x7fb948159ec0 2023-10-08T00:11:55.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.901+0000 7fb93e7fc700 1 -- 172.21.15.99:0/184800362 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93801a680 con 0x7fb948159ec0 2023-10-08T00:11:55.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.901+0000 7fb9506b6700 1 -- 172.21.15.99:0/184800362 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb94814d200 con 0x7fb948159ec0 2023-10-08T00:11:55.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.902+0000 7fb9506b6700 1 -- 172.21.15.99:0/184800362 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb94804c3c0 con 0x7fb948159ec0 2023-10-08T00:11:55.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.903+0000 7fb93e7fc700 1 -- 172.21.15.99:0/184800362 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb93801aab0 con 0x7fb948159ec0 2023-10-08T00:11:55.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.903+0000 7fb93e7fc700 1 --2- 172.21.15.99:0/184800362 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb92c066f30 0x7fb92c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:55.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.903+0000 7fb93e7fc700 1 -- 172.21.15.99:0/184800362 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb938086150 con 0x7fb948159ec0 2023-10-08T00:11:55.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.903+0000 7fb94e452700 1 --2- 172.21.15.99:0/184800362 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb92c066f30 0x7fb92c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:55.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.904+0000 7fb94e452700 1 --2- 172.21.15.99:0/184800362 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb92c066f30 0x7fb92c0693e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb944009d40 tx=0x7fb944008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:55.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:55.907+0000 7fb93e7fc700 1 -- 172.21.15.99:0/184800362 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb9380533c0 con 0x7fb948159ec0 2023-10-08T00:11:56.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.089+0000 7fb9506b6700 1 -- 172.21.15.99:0/184800362 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb9480ceab0 con 0x7fb948159ec0 2023-10-08T00:11:56.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.090+0000 7fb93e7fc700 1 -- 172.21.15.99:0/184800362 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb9380569e0 con 0x7fb948159ec0 2023-10-08T00:11:56.090 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2023-10-08T00:11:56.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.092+0000 7fb9506b6700 1 -- 172.21.15.99:0/184800362 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb92c066f30 msgr2=0x7fb92c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:56.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.092+0000 7fb9506b6700 1 --2- 172.21.15.99:0/184800362 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb92c066f30 0x7fb92c0693e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb944009d40 tx=0x7fb944008040 comp rx=0 tx=0).stop 2023-10-08T00:11:56.093 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.092+0000 7fb9506b6700 1 -- 172.21.15.99:0/184800362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb948159ec0 msgr2=0x7fb94814f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:56.093 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.092+0000 7fb9506b6700 1 --2- 172.21.15.99:0/184800362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb948159ec0 0x7fb94814f090 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb948071eb0 tx=0x7fb938017000 comp rx=0 tx=0).stop 2023-10-08T00:11:56.093 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.092+0000 7fb9506b6700 1 -- 172.21.15.99:0/184800362 shutdown_connections 2023-10-08T00:11:56.093 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.092+0000 7fb9506b6700 1 --2- 172.21.15.99:0/184800362 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb92c066f30 0x7fb92c0693e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:56.093 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.092+0000 7fb9506b6700 1 --2- 172.21.15.99:0/184800362 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb94815a290 0x7fb94814ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:56.093 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.092+0000 7fb9506b6700 1 --2- 172.21.15.99:0/184800362 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb94806a5a0 0x7fb9480ca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:56.093 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.092+0000 7fb9506b6700 1 --2- 172.21.15.99:0/184800362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb948159ec0 0x7fb94814f090 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:56.093 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.092+0000 7fb9506b6700 1 -- 172.21.15.99:0/184800362 >> 172.21.15.99:0/184800362 conn(0x7fb9480d1880 msgr2=0x7fb9480b1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:56.093 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.092+0000 7fb9506b6700 1 -- 172.21.15.99:0/184800362 shutdown_connections 2023-10-08T00:11:56.093 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.092+0000 7fb9506b6700 1 -- 172.21.15.99:0/184800362 wait complete. 2023-10-08T00:11:56.104 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:11:56.220 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.219+0000 7f44740fa700 1 Processor -- start 2023-10-08T00:11:56.220 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.219+0000 7f44740fa700 1 -- start start 2023-10-08T00:11:56.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.220+0000 7f44740fa700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f446c157c40 0x7f446c158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:56.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.220+0000 7f44740fa700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f446c15cbb0 con 0x7f446c15a620 2023-10-08T00:11:56.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.220+0000 7f44740fa700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f446c15bfa0 con 0x7f446c157870 2023-10-08T00:11:56.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.220+0000 7f44740fa700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f446c15bdb0 con 0x7f446c157c40 2023-10-08T00:11:56.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.220+0000 7f4471695700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f446c15a620 0x7f446c15d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56368/0 (socket says 172.21.15.99:56368) 2023-10-08T00:11:56.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.220+0000 7f4471695700 1 -- 172.21.15.99:0/2808552436 learned_addr learned my addr 172.21.15.99:0/2808552436 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:56.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.220+0000 7f4472697700 1 --2- 172.21.15.99:0/2808552436 >> v2:172.21.15.191:3300/0 conn(0x7f446c157c40 0x7f446c158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:56.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.220+0000 7f4470e94700 1 -- 172.21.15.99:0/2808552436 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 39648144 0 0) 0x7f446c15cbb0 con 0x7f446c15a620 2023-10-08T00:11:56.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.220+0000 7f4470e94700 1 -- 172.21.15.99:0/2808552436 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4458003570 con 0x7f446c15a620 2023-10-08T00:11:56.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.221+0000 7f4470e94700 1 -- 172.21.15.99:0/2808552436 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 540211214 0 0) 0x7f4458003570 con 0x7f446c15a620 2023-10-08T00:11:56.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.221+0000 7f4470e94700 1 -- 172.21.15.99:0/2808552436 >> v2:172.21.15.191:3300/0 conn(0x7f446c157c40 msgr2=0x7f446c158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:56.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.221+0000 7f4470e94700 1 --2- 172.21.15.99:0/2808552436 >> v2:172.21.15.191:3300/0 conn(0x7f446c157c40 0x7f446c158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:56.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.221+0000 7f4470e94700 1 -- 172.21.15.99:0/2808552436 >> v1:172.21.15.191:6789/0 conn(0x7f446c157870 legacy=0x7f446c15b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:56.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.221+0000 7f4470e94700 1 -- 172.21.15.99:0/2808552436 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f446c15c280 con 0x7f446c15a620 2023-10-08T00:11:56.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.221+0000 7f4470e94700 1 -- 172.21.15.99:0/2808552436 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f445c001470 con 0x7f446c15a620 2023-10-08T00:11:56.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.221+0000 7f4470e94700 1 -- 172.21.15.99:0/2808552436 >> v1:172.21.15.99:6789/0 conn(0x7f446c15a620 legacy=0x7f446c15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:56.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.221+0000 7f4470e94700 1 --2- 172.21.15.99:0/2808552436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4458003a60 0x7f4458003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:56.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.221+0000 7f4470e94700 1 --2- 172.21.15.99:0/2808552436 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4458004810 0x7f4458004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:56.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.221+0000 7f4470e94700 1 --2- 172.21.15.99:0/2808552436 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f44580051c0 0x7f445800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:56.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.221+0000 7f4470e94700 1 -- 172.21.15.99:0/2808552436 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445800c130 con 0x7f4458003a60 2023-10-08T00:11:56.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.221+0000 7f4470e94700 1 -- 172.21.15.99:0/2808552436 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445800c2a0 con 0x7f44580051c0 2023-10-08T00:11:56.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.221+0000 7f4470e94700 1 -- 172.21.15.99:0/2808552436 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f445800c460 con 0x7f4458004810 2023-10-08T00:11:56.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.221+0000 7f4471e96700 1 --2- 172.21.15.99:0/2808552436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4458003a60 0x7f4458003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:56.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.221+0000 7f4471695700 1 --2- 172.21.15.99:0/2808552436 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4458004810 0x7f4458004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:56.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.221+0000 7f4472697700 1 --2- 172.21.15.99:0/2808552436 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f44580051c0 0x7f445800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:56.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.221+0000 7f4471e96700 1 -- 172.21.15.99:0/2808552436 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4458004810 msgr2=0x7f4458004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:56.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.221+0000 7f4471e96700 1 --2- 172.21.15.99:0/2808552436 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4458004810 0x7f4458004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:56.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.221+0000 7f4471e96700 1 -- 172.21.15.99:0/2808552436 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f44580051c0 msgr2=0x7f445800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:56.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.221+0000 7f4471e96700 1 --2- 172.21.15.99:0/2808552436 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f44580051c0 0x7f445800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:56.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.221+0000 7f4471e96700 1 -- 172.21.15.99:0/2808552436 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4458003570 con 0x7f4458003a60 2023-10-08T00:11:56.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.221+0000 7f4472697700 1 --2- 172.21.15.99:0/2808552436 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f44580051c0 0x7f445800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:11:56.231 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.221+0000 7f4472697700 1 -- 172.21.15.99:0/2808552436 reap_dead start 2023-10-08T00:11:56.231 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.221+0000 7f4471e96700 1 --2- 172.21.15.99:0/2808552436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4458003a60 0x7f4458003e70 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4468001fc0 tx=0x7f4468001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33f761d29c91b6f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:56.231 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.221+0000 7f4470e94700 1 -- 172.21.15.99:0/2808552436 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4468001950 con 0x7f4458003a60 2023-10-08T00:11:56.231 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.221+0000 7f4470e94700 1 -- 172.21.15.99:0/2808552436 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f446800c8b0 con 0x7f4458003a60 2023-10-08T00:11:56.231 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.221+0000 7f4470e94700 1 -- 172.21.15.99:0/2808552436 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446800cc50 con 0x7f4458003a60 2023-10-08T00:11:56.232 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.222+0000 7f44740fa700 1 -- 172.21.15.99:0/2808552436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4458003a60 msgr2=0x7f4458003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:56.232 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.222+0000 7f44740fa700 1 --2- 172.21.15.99:0/2808552436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4458003a60 0x7f4458003e70 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4468001fc0 tx=0x7f4468001640 comp rx=0 tx=0).stop 2023-10-08T00:11:56.232 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.222+0000 7f44740fa700 1 -- 172.21.15.99:0/2808552436 shutdown_connections 2023-10-08T00:11:56.232 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.222+0000 7f44740fa700 1 --2- 172.21.15.99:0/2808552436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4458003a60 0x7f4458003e70 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:56.232 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.222+0000 7f44740fa700 1 -- 172.21.15.99:0/2808552436 >> 172.21.15.99:0/2808552436 conn(0x7f446c068ce0 msgr2=0x7f446c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:56.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.222+0000 7f44740fa700 1 -- 172.21.15.99:0/2808552436 shutdown_connections 2023-10-08T00:11:56.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.222+0000 7f44740fa700 1 -- 172.21.15.99:0/2808552436 wait complete. 2023-10-08T00:11:56.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.223+0000 7f44740fa700 1 Processor -- start 2023-10-08T00:11:56.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.223+0000 7f44740fa700 1 -- start start 2023-10-08T00:11:56.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.224+0000 7f44740fa700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4458003a60 0x7f446c14fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:56.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.224+0000 7f44740fa700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f446c150430 0x7f446c0b0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:56.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.224+0000 7f44740fa700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f446c0b0b50 0x7f446c1537d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:56.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.224+0000 7f44740fa700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f446c14fa60 con 0x7f446c0b0b50 2023-10-08T00:11:56.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.224+0000 7f44740fa700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f446c14ebf0 con 0x7f4458003a60 2023-10-08T00:11:56.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.224+0000 7f44740fa700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f446c14ee90 con 0x7f446c150430 2023-10-08T00:11:56.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.224+0000 7f4472697700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f446c0b0b50 0x7f446c1537d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:56.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.224+0000 7f4472697700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f446c0b0b50 0x7f446c1537d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58194/0 (socket says 172.21.15.99:58194) 2023-10-08T00:11:56.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.224+0000 7f4472697700 1 -- 172.21.15.99:0/3804897583 learned_addr learned my addr 172.21.15.99:0/3804897583 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:56.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.224+0000 7f4471e96700 1 --2- 172.21.15.99:0/3804897583 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4458003a60 0x7f446c14fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:56.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.224+0000 7f4471695700 1 --2- 172.21.15.99:0/3804897583 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f446c150430 0x7f446c0b0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:56.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.224+0000 7f4472697700 1 -- 172.21.15.99:0/3804897583 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f446c150430 msgr2=0x7f446c0b0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:56.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.224+0000 7f4472697700 1 --2- 172.21.15.99:0/3804897583 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f446c150430 0x7f446c0b0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:56.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.224+0000 7f4472697700 1 -- 172.21.15.99:0/3804897583 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4458003a60 msgr2=0x7f446c14fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:56.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.224+0000 7f4472697700 1 --2- 172.21.15.99:0/3804897583 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4458003a60 0x7f446c14fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:56.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.224+0000 7f4472697700 1 -- 172.21.15.99:0/3804897583 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f446c14f130 con 0x7f446c0b0b50 2023-10-08T00:11:56.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.224+0000 7f4471695700 1 --2- 172.21.15.99:0/3804897583 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f446c150430 0x7f446c0b0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:11:56.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.224+0000 7f4472697700 1 --2- 172.21.15.99:0/3804897583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f446c0b0b50 0x7f446c1537d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f446c0b3830 tx=0x7f4464008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:56.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.225+0000 7f4461ffb700 1 -- 172.21.15.99:0/3804897583 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4464015670 con 0x7f446c0b0b50 2023-10-08T00:11:56.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.225+0000 7f4461ffb700 1 -- 172.21.15.99:0/3804897583 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44640157f0 con 0x7f446c0b0b50 2023-10-08T00:11:56.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.225+0000 7f44740fa700 1 -- 172.21.15.99:0/3804897583 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f446c14f410 con 0x7f446c0b0b50 2023-10-08T00:11:56.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.225+0000 7f4461ffb700 1 -- 172.21.15.99:0/3804897583 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4464015bb0 con 0x7f446c0b0b50 2023-10-08T00:11:56.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.225+0000 7f44740fa700 1 -- 172.21.15.99:0/3804897583 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f446c153d10 con 0x7f446c0b0b50 2023-10-08T00:11:56.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.226+0000 7f44740fa700 1 -- 172.21.15.99:0/3804897583 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f446c14fbd0 con 0x7f446c0b0b50 2023-10-08T00:11:56.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.226+0000 7f4461ffb700 1 -- 172.21.15.99:0/3804897583 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f446401f410 con 0x7f446c0b0b50 2023-10-08T00:11:56.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.227+0000 7f4461ffb700 1 --2- 172.21.15.99:0/3804897583 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4450066f30 0x7f44500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:56.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.227+0000 7f4461ffb700 1 -- 172.21.15.99:0/3804897583 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4464084290 con 0x7f446c0b0b50 2023-10-08T00:11:56.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.227+0000 7f4471e96700 1 --2- 172.21.15.99:0/3804897583 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4450066f30 0x7f44500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:56.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.227+0000 7f4471e96700 1 --2- 172.21.15.99:0/3804897583 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4450066f30 0x7f44500693e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4468001390 tx=0x7f4468015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:56.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.231+0000 7f4461ffb700 1 -- 172.21.15.99:0/3804897583 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4464051500 con 0x7f446c0b0b50 2023-10-08T00:11:56.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.457+0000 7f44740fa700 1 -- 172.21.15.99:0/3804897583 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f446c04c3c0 con 0x7f446c0b0b50 2023-10-08T00:11:56.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.458+0000 7f4461ffb700 1 -- 172.21.15.99:0/3804897583 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4600 (secure 0 0 0) 0x7f4464054b20 con 0x7f446c0b0b50 2023-10-08T00:11:56.459 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:11:56.459 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4852,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/824084995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":824084995},{"type":"v1","addr":"172.21.15.191:6837","nonce":824084995}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4855,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/3349126920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3349126920},{"type":"v1","addr":"172.21.15.191:6836","nonce":3349126920}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4899,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/1004180438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1004180438},{"type":"v1","addr":"172.21.15.99:6834","nonce":1004180438}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:11:40.672879+0000","modified":"2023-10-08T00:11:44.314526+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4902},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4902":{"gid":4902,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/4189095717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":4189095717},{"type":"v1","addr":"172.21.15.99:6835","nonce":4189095717}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-08T00:11:56.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.460+0000 7f44740fa700 1 -- 172.21.15.99:0/3804897583 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4450066f30 msgr2=0x7f44500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:56.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.460+0000 7f44740fa700 1 --2- 172.21.15.99:0/3804897583 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4450066f30 0x7f44500693e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4468001390 tx=0x7f4468015000 comp rx=0 tx=0).stop 2023-10-08T00:11:56.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.460+0000 7f44740fa700 1 -- 172.21.15.99:0/3804897583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f446c0b0b50 msgr2=0x7f446c1537d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:56.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.460+0000 7f44740fa700 1 --2- 172.21.15.99:0/3804897583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f446c0b0b50 0x7f446c1537d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f446c0b3830 tx=0x7f4464008040 comp rx=0 tx=0).stop 2023-10-08T00:11:56.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.460+0000 7f44740fa700 1 -- 172.21.15.99:0/3804897583 shutdown_connections 2023-10-08T00:11:56.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.460+0000 7f44740fa700 1 --2- 172.21.15.99:0/3804897583 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4450066f30 0x7f44500693e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:56.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.460+0000 7f44740fa700 1 --2- 172.21.15.99:0/3804897583 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4458003a60 0x7f446c14fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:56.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.460+0000 7f44740fa700 1 --2- 172.21.15.99:0/3804897583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f446c0b0b50 0x7f446c1537d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:56.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.460+0000 7f44740fa700 1 --2- 172.21.15.99:0/3804897583 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f446c150430 0x7f446c0b0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:56.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.460+0000 7f44740fa700 1 -- 172.21.15.99:0/3804897583 >> 172.21.15.99:0/3804897583 conn(0x7f446c068ce0 msgr2=0x7f446c0b46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:56.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.460+0000 7f44740fa700 1 -- 172.21.15.99:0/3804897583 shutdown_connections 2023-10-08T00:11:56.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.460+0000 7f44740fa700 1 -- 172.21.15.99:0/3804897583 wait complete. 2023-10-08T00:11:56.462 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 17 2023-10-08T00:11:56.474 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:11:56.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.587+0000 7f8a42287700 1 Processor -- start 2023-10-08T00:11:56.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.587+0000 7f8a42287700 1 -- start start 2023-10-08T00:11:56.590 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.588+0000 7f8a42287700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a3c0cd6e0 0x7f8a3c0cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:56.590 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.588+0000 7f8a42287700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8a3c0cdab0 0x7f8a3c15d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:56.590 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.588+0000 7f8a42287700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8a3c15dda0 0x7f8a3c15cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:56.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.588+0000 7f8a42287700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a3c15e850 con 0x7f8a3c15dda0 2023-10-08T00:11:56.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.588+0000 7f8a42287700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a3c0cd030 con 0x7f8a3c0cdab0 2023-10-08T00:11:56.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.588+0000 7f8a42287700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3c0cb7c0 con 0x7f8a3c0cd6e0 2023-10-08T00:11:56.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.588+0000 7f8a41a86700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8a3c15dda0 0x7f8a3c15cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:56.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.588+0000 7f8a41a86700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8a3c15dda0 0x7f8a3c15cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58204/0 (socket says 172.21.15.99:58204) 2023-10-08T00:11:56.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.588+0000 7f8a41a86700 1 -- 172.21.15.99:0/3434833626 learned_addr learned my addr 172.21.15.99:0/3434833626 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:56.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.588+0000 7f8a41285700 1 --2- 172.21.15.99:0/3434833626 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a3c0cd6e0 0x7f8a3c0cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:56.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.588+0000 7f8a40a84700 1 --2- 172.21.15.99:0/3434833626 >> v2:172.21.15.191:3300/0 conn(0x7f8a3c0cdab0 0x7f8a3c15d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:56.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.588+0000 7f8a41a86700 1 -- 172.21.15.99:0/3434833626 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a3c0cd6e0 msgr2=0x7f8a3c0cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:56.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.588+0000 7f8a41a86700 1 --2- 172.21.15.99:0/3434833626 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a3c0cd6e0 0x7f8a3c0cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:56.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.588+0000 7f8a41a86700 1 -- 172.21.15.99:0/3434833626 >> v2:172.21.15.191:3300/0 conn(0x7f8a3c0cdab0 msgr2=0x7f8a3c15d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:56.592 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.588+0000 7f8a41a86700 1 --2- 172.21.15.99:0/3434833626 >> v2:172.21.15.191:3300/0 conn(0x7f8a3c0cdab0 0x7f8a3c15d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:56.592 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.588+0000 7f8a41a86700 1 -- 172.21.15.99:0/3434833626 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a3c15e290 con 0x7f8a3c15dda0 2023-10-08T00:11:56.592 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.588+0000 7f8a40a84700 1 --2- 172.21.15.99:0/3434833626 >> v2:172.21.15.191:3300/0 conn(0x7f8a3c0cdab0 0x7f8a3c15d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:11:56.592 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.589+0000 7f8a41a86700 1 --2- 172.21.15.99:0/3434833626 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a3c15dda0 0x7f8a3c15cbd0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c089b50 tx=0x7f8a34009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95d25642f0f39d97 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:56.592 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.589+0000 7f8a33fff700 1 -- 172.21.15.99:0/3434833626 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3400b860 con 0x7f8a3c15dda0 2023-10-08T00:11:56.592 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.589+0000 7f8a33fff700 1 -- 172.21.15.99:0/3434833626 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a3400b9c0 con 0x7f8a3c15dda0 2023-10-08T00:11:56.592 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.589+0000 7f8a33fff700 1 -- 172.21.15.99:0/3434833626 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3400bd60 con 0x7f8a3c15dda0 2023-10-08T00:11:56.592 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.589+0000 7f8a42287700 1 -- 172.21.15.99:0/3434833626 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a3c15dda0 msgr2=0x7f8a3c15cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:56.592 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.589+0000 7f8a42287700 1 --2- 172.21.15.99:0/3434833626 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a3c15dda0 0x7f8a3c15cbd0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c089b50 tx=0x7f8a34009250 comp rx=0 tx=0).stop 2023-10-08T00:11:56.592 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.589+0000 7f8a42287700 1 -- 172.21.15.99:0/3434833626 shutdown_connections 2023-10-08T00:11:56.592 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.589+0000 7f8a42287700 1 --2- 172.21.15.99:0/3434833626 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a3c15dda0 0x7f8a3c15cbd0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:56.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.589+0000 7f8a42287700 1 --2- 172.21.15.99:0/3434833626 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a3c0cd6e0 0x7f8a3c0cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:56.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.589+0000 7f8a42287700 1 --2- 172.21.15.99:0/3434833626 >> v2:172.21.15.191:3300/0 conn(0x7f8a3c0cdab0 0x7f8a3c15d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:56.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.589+0000 7f8a42287700 1 -- 172.21.15.99:0/3434833626 >> 172.21.15.99:0/3434833626 conn(0x7f8a3c067660 msgr2=0x7f8a3c067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:56.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.589+0000 7f8a42287700 1 -- 172.21.15.99:0/3434833626 shutdown_connections 2023-10-08T00:11:56.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.589+0000 7f8a42287700 1 -- 172.21.15.99:0/3434833626 wait complete. 2023-10-08T00:11:56.595 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.590+0000 7f8a42287700 1 Processor -- start 2023-10-08T00:11:56.595 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.591+0000 7f8a42287700 1 -- start start 2023-10-08T00:11:56.595 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.591+0000 7f8a42287700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a3c0cd6e0 0x7f8a3c0c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:56.596 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.591+0000 7f8a42287700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a3c0cdab0 0x7f8a3c0b63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:56.596 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.591+0000 7f8a42287700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a3c15dda0 0x7f8a3c141840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:56.596 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.591+0000 7f8a42287700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3c0c8ff0 con 0x7f8a3c0cdab0 2023-10-08T00:11:56.596 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.591+0000 7f8a42287700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3c0c83e0 con 0x7f8a3c15dda0 2023-10-08T00:11:56.596 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.591+0000 7f8a42287700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3c0c86c0 con 0x7f8a3c0cd6e0 2023-10-08T00:11:56.596 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.591+0000 7f8a40a84700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a3c0cdab0 0x7f8a3c0b63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:56.596 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.591+0000 7f8a40a84700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a3c0cdab0 0x7f8a3c0b63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58210/0 (socket says 172.21.15.99:58210) 2023-10-08T00:11:56.596 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.591+0000 7f8a40a84700 1 -- 172.21.15.99:0/700046225 learned_addr learned my addr 172.21.15.99:0/700046225 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:56.596 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.592+0000 7f8a40a84700 1 -- 172.21.15.99:0/700046225 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a3c0cd6e0 msgr2=0x7f8a3c0c7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:56.596 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.592+0000 7f8a40a84700 1 --2- 172.21.15.99:0/700046225 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a3c0cd6e0 0x7f8a3c0c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:56.596 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.592+0000 7f8a40a84700 1 -- 172.21.15.99:0/700046225 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a3c15dda0 msgr2=0x7f8a3c141840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:56.597 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.592+0000 7f8a40a84700 1 --2- 172.21.15.99:0/700046225 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a3c15dda0 0x7f8a3c141840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:56.597 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.592+0000 7f8a40a84700 1 -- 172.21.15.99:0/700046225 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a340094f0 con 0x7f8a3c0cdab0 2023-10-08T00:11:56.597 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.592+0000 7f8a40a84700 1 --2- 172.21.15.99:0/700046225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a3c0cdab0 0x7f8a3c0b63d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c00e8d0 tx=0x7f8a2c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:56.597 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.592+0000 7f8a317fa700 1 -- 172.21.15.99:0/700046225 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c00ee70 con 0x7f8a3c0cdab0 2023-10-08T00:11:56.597 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.592+0000 7f8a317fa700 1 -- 172.21.15.99:0/700046225 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a2c00c5e0 con 0x7f8a3c0cdab0 2023-10-08T00:11:56.597 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.592+0000 7f8a42287700 1 -- 172.21.15.99:0/700046225 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a3c0c89a0 con 0x7f8a3c0cdab0 2023-10-08T00:11:56.597 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.592+0000 7f8a317fa700 1 -- 172.21.15.99:0/700046225 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c00c980 con 0x7f8a3c0cdab0 2023-10-08T00:11:56.597 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.592+0000 7f8a42287700 1 -- 172.21.15.99:0/700046225 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a3c141d80 con 0x7f8a3c0cdab0 2023-10-08T00:11:56.597 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.593+0000 7f8a42287700 1 -- 172.21.15.99:0/700046225 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a3c089280 con 0x7f8a3c0cdab0 2023-10-08T00:11:56.597 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.593+0000 7f8a317fa700 1 -- 172.21.15.99:0/700046225 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8a2c013410 con 0x7f8a3c0cdab0 2023-10-08T00:11:56.597 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.594+0000 7f8a317fa700 1 --2- 172.21.15.99:0/700046225 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8a20066f30 0x7f8a200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:56.597 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.594+0000 7f8a317fa700 1 -- 172.21.15.99:0/700046225 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8a2c084340 con 0x7f8a3c0cdab0 2023-10-08T00:11:56.598 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.594+0000 7f8a41285700 1 --2- 172.21.15.99:0/700046225 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8a20066f30 0x7f8a200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:56.598 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.595+0000 7f8a41285700 1 --2- 172.21.15.99:0/700046225 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8a20066f30 0x7f8a200693e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8a38009fd0 tx=0x7f8a38009970 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:56.598 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.598+0000 7f8a317fa700 1 -- 172.21.15.99:0/700046225 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8a2c0515b0 con 0x7f8a3c0cdab0 2023-10-08T00:11:56.781 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.780+0000 7f8a42287700 1 -- 172.21.15.99:0/700046225 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8a3c0c9130 con 0x7f8a3c0cdab0 2023-10-08T00:11:56.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.781+0000 7f8a317fa700 1 -- 172.21.15.99:0/700046225 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f8a2c054bd0 con 0x7f8a3c0cdab0 2023-10-08T00:11:56.782 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:11:56.782 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":33,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:11:52.347370+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:11:38.415553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-08T00:11:39.634576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:11:56.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.783+0000 7f8a42287700 1 -- 172.21.15.99:0/700046225 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8a20066f30 msgr2=0x7f8a200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:56.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.783+0000 7f8a42287700 1 --2- 172.21.15.99:0/700046225 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8a20066f30 0x7f8a200693e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8a38009fd0 tx=0x7f8a38009970 comp rx=0 tx=0).stop 2023-10-08T00:11:56.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.783+0000 7f8a42287700 1 -- 172.21.15.99:0/700046225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a3c0cdab0 msgr2=0x7f8a3c0b63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:56.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.783+0000 7f8a42287700 1 --2- 172.21.15.99:0/700046225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a3c0cdab0 0x7f8a3c0b63d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c00e8d0 tx=0x7f8a2c008040 comp rx=0 tx=0).stop 2023-10-08T00:11:56.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.783+0000 7f8a42287700 1 -- 172.21.15.99:0/700046225 shutdown_connections 2023-10-08T00:11:56.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.783+0000 7f8a42287700 1 --2- 172.21.15.99:0/700046225 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8a20066f30 0x7f8a200693e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:56.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.783+0000 7f8a42287700 1 --2- 172.21.15.99:0/700046225 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a3c15dda0 0x7f8a3c141840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:56.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.783+0000 7f8a42287700 1 --2- 172.21.15.99:0/700046225 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a3c0cd6e0 0x7f8a3c0c7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:56.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.783+0000 7f8a42287700 1 --2- 172.21.15.99:0/700046225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a3c0cdab0 0x7f8a3c0b63d0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:56.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.783+0000 7f8a42287700 1 -- 172.21.15.99:0/700046225 >> 172.21.15.99:0/700046225 conn(0x7f8a3c067660 msgr2=0x7f8a3c15d670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:56.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.783+0000 7f8a42287700 1 -- 172.21.15.99:0/700046225 shutdown_connections 2023-10-08T00:11:56.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.783+0000 7f8a42287700 1 -- 172.21.15.99:0/700046225 wait complete. 2023-10-08T00:11:56.796 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T00:11:56.796 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:11:56.910 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.909+0000 7fd66dc08700 1 Processor -- start 2023-10-08T00:11:56.910 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.909+0000 7fd66dc08700 1 -- start start 2023-10-08T00:11:56.912 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.910+0000 7fd66dc08700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd6680c9560 0x7fd668157a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:56.913 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.910+0000 7fd66dc08700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd668069de0 0x7fd668157f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:56.913 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.910+0000 7fd66dc08700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6680c9930 0x7fd66815bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:56.913 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.910+0000 7fd66dc08700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd668069060 con 0x7fd668069de0 2023-10-08T00:11:56.913 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.910+0000 7fd66dc08700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd668069300 con 0x7fd6680c9560 2023-10-08T00:11:56.913 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.910+0000 7fd66dc08700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6680695e0 con 0x7fd6680c9930 2023-10-08T00:11:56.913 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.910+0000 7fd65effd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd668069de0 0x7fd668157f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:56.913 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.910+0000 7fd65effd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd668069de0 0x7fd668157f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58212/0 (socket says 172.21.15.99:58212) 2023-10-08T00:11:56.913 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.910+0000 7fd65effd700 1 -- 172.21.15.99:0/345583471 learned_addr learned my addr 172.21.15.99:0/345583471 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:56.913 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.910+0000 7fd667fff700 1 --2- 172.21.15.99:0/345583471 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6680c9930 0x7fd66815bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:56.914 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.910+0000 7fd6677fe700 1 --2- 172.21.15.99:0/345583471 >> v2:172.21.15.191:3300/0 conn(0x7fd6680c9560 0x7fd668157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:56.914 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.911+0000 7fd65effd700 1 -- 172.21.15.99:0/345583471 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6680c9930 msgr2=0x7fd66815bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:56.914 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.911+0000 7fd65effd700 1 --2- 172.21.15.99:0/345583471 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6680c9930 0x7fd66815bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:56.914 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.911+0000 7fd65effd700 1 -- 172.21.15.99:0/345583471 >> v2:172.21.15.191:3300/0 conn(0x7fd6680c9560 msgr2=0x7fd668157a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:56.914 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.911+0000 7fd65effd700 1 --2- 172.21.15.99:0/345583471 >> v2:172.21.15.191:3300/0 conn(0x7fd6680c9560 0x7fd668157a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:56.914 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.911+0000 7fd65effd700 1 -- 172.21.15.99:0/345583471 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6680698c0 con 0x7fd668069de0 2023-10-08T00:11:56.914 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.911+0000 7fd6677fe700 1 --2- 172.21.15.99:0/345583471 >> v2:172.21.15.191:3300/0 conn(0x7fd6680c9560 0x7fd668157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:11:56.914 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.911+0000 7fd65effd700 1 --2- 172.21.15.99:0/345583471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd668069de0 0x7fd668157f90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd6680ca3f0 tx=0x7fd650009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cee4dee2ec4a64e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:56.914 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.911+0000 7fd666ffd700 1 -- 172.21.15.99:0/345583471 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd650009e70 con 0x7fd668069de0 2023-10-08T00:11:56.914 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.911+0000 7fd666ffd700 1 -- 172.21.15.99:0/345583471 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd65000b860 con 0x7fd668069de0 2023-10-08T00:11:56.915 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.911+0000 7fd666ffd700 1 -- 172.21.15.99:0/345583471 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65000bc00 con 0x7fd668069de0 2023-10-08T00:11:56.915 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.911+0000 7fd66dc08700 1 -- 172.21.15.99:0/345583471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd668069de0 msgr2=0x7fd668157f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:56.915 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.911+0000 7fd66dc08700 1 --2- 172.21.15.99:0/345583471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd668069de0 0x7fd668157f90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd6680ca3f0 tx=0x7fd650009250 comp rx=0 tx=0).stop 2023-10-08T00:11:56.915 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.911+0000 7fd66dc08700 1 -- 172.21.15.99:0/345583471 shutdown_connections 2023-10-08T00:11:56.915 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.911+0000 7fd66dc08700 1 --2- 172.21.15.99:0/345583471 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6680c9930 0x7fd66815bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:56.915 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.911+0000 7fd66dc08700 1 --2- 172.21.15.99:0/345583471 >> v2:172.21.15.191:3300/0 conn(0x7fd6680c9560 0x7fd668157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:56.915 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.911+0000 7fd66dc08700 1 --2- 172.21.15.99:0/345583471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd668069de0 0x7fd668157f90 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:56.915 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.911+0000 7fd66dc08700 1 -- 172.21.15.99:0/345583471 >> 172.21.15.99:0/345583471 conn(0x7fd668067660 msgr2=0x7fd668067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:56.915 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.912+0000 7fd66dc08700 1 -- 172.21.15.99:0/345583471 shutdown_connections 2023-10-08T00:11:56.915 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.912+0000 7fd66dc08700 1 -- 172.21.15.99:0/345583471 wait complete. 2023-10-08T00:11:56.918 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.913+0000 7fd66dc08700 1 Processor -- start 2023-10-08T00:11:56.918 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.913+0000 7fd66dc08700 1 -- start start 2023-10-08T00:11:56.918 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.913+0000 7fd66dc08700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd668069de0 0x7fd6680c5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:56.918 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.913+0000 7fd66dc08700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6680c9560 0x7fd66816a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:56.919 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.914+0000 7fd66dc08700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6680c9930 0x7fd66816aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:56.919 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.914+0000 7fd66dc08700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6680c4330 con 0x7fd668069de0 2023-10-08T00:11:56.919 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.914+0000 7fd66dc08700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6680c4580 con 0x7fd6680c9930 2023-10-08T00:11:56.919 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.914+0000 7fd66dc08700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6680c4860 con 0x7fd6680c9560 2023-10-08T00:11:56.919 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.914+0000 7fd6677fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd668069de0 0x7fd6680c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:56.919 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.914+0000 7fd6677fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd668069de0 0x7fd6680c5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58224/0 (socket says 172.21.15.99:58224) 2023-10-08T00:11:56.919 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.914+0000 7fd6677fe700 1 -- 172.21.15.99:0/417959588 learned_addr learned my addr 172.21.15.99:0/417959588 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:56.919 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.914+0000 7fd65effd700 1 --2- 172.21.15.99:0/417959588 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6680c9560 0x7fd66816a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:56.919 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.914+0000 7fd667fff700 1 --2- 172.21.15.99:0/417959588 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6680c9930 0x7fd66816aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:56.920 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.914+0000 7fd6677fe700 1 -- 172.21.15.99:0/417959588 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6680c9560 msgr2=0x7fd66816a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:56.920 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.914+0000 7fd6677fe700 1 --2- 172.21.15.99:0/417959588 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6680c9560 0x7fd66816a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:56.920 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.914+0000 7fd6677fe700 1 -- 172.21.15.99:0/417959588 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6680c9930 msgr2=0x7fd66816aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:56.920 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.914+0000 7fd6677fe700 1 --2- 172.21.15.99:0/417959588 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6680c9930 0x7fd66816aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:56.920 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.914+0000 7fd6677fe700 1 -- 172.21.15.99:0/417959588 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6500094f0 con 0x7fd668069de0 2023-10-08T00:11:56.920 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.914+0000 7fd6677fe700 1 --2- 172.21.15.99:0/417959588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd668069de0 0x7fd6680c5340 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd65800e8d0 tx=0x7fd658008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:56.920 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.914+0000 7fd65ffff700 1 -- 172.21.15.99:0/417959588 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65800ee70 con 0x7fd668069de0 2023-10-08T00:11:56.920 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.914+0000 7fd65ffff700 1 -- 172.21.15.99:0/417959588 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd65800c5e0 con 0x7fd668069de0 2023-10-08T00:11:56.920 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.914+0000 7fd66dc08700 1 -- 172.21.15.99:0/417959588 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6680c4b40 con 0x7fd668069de0 2023-10-08T00:11:56.920 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.914+0000 7fd65ffff700 1 -- 172.21.15.99:0/417959588 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65800c980 con 0x7fd668069de0 2023-10-08T00:11:56.921 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.914+0000 7fd66dc08700 1 -- 172.21.15.99:0/417959588 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6680c6f20 con 0x7fd668069de0 2023-10-08T00:11:56.921 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.915+0000 7fd66dc08700 1 -- 172.21.15.99:0/417959588 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6680c5190 con 0x7fd668069de0 2023-10-08T00:11:56.921 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.916+0000 7fd65ffff700 1 -- 172.21.15.99:0/417959588 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd658013410 con 0x7fd668069de0 2023-10-08T00:11:56.921 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.916+0000 7fd65ffff700 1 --2- 172.21.15.99:0/417959588 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd648066f30 0x7fd6480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:56.921 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.916+0000 7fd65ffff700 1 -- 172.21.15.99:0/417959588 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd6580845c0 con 0x7fd668069de0 2023-10-08T00:11:56.921 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.916+0000 7fd65effd700 1 --2- 172.21.15.99:0/417959588 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd648066f30 0x7fd6480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:56.921 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.916+0000 7fd65effd700 1 --2- 172.21.15.99:0/417959588 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd648066f30 0x7fd6480693e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd6680ca3f0 tx=0x7fd650017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:56.922 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:56.920+0000 7fd65ffff700 1 -- 172.21.15.99:0/417959588 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd658051830 con 0x7fd668069de0 2023-10-08T00:11:57.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.147+0000 7fd66dc08700 1 -- 172.21.15.99:0/417959588 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd66804c3c0 con 0x7fd668069de0 2023-10-08T00:11:57.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.147+0000 7fd65ffff700 1 -- 172.21.15.99:0/417959588 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4600 (secure 0 0 0) 0x7fd658054e50 con 0x7fd668069de0 2023-10-08T00:11:57.148 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:11:57.148 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4852,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/824084995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":824084995},{"type":"v1","addr":"172.21.15.191:6837","nonce":824084995}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4855,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/3349126920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3349126920},{"type":"v1","addr":"172.21.15.191:6836","nonce":3349126920}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4899,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/1004180438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1004180438},{"type":"v1","addr":"172.21.15.99:6834","nonce":1004180438}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:11:40.672879+0000","modified":"2023-10-08T00:11:44.314526+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4902},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4902":{"gid":4902,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/4189095717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":4189095717},{"type":"v1","addr":"172.21.15.99:6835","nonce":4189095717}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-08T00:11:57.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.149+0000 7fd66dc08700 1 -- 172.21.15.99:0/417959588 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd648066f30 msgr2=0x7fd6480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:57.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.149+0000 7fd66dc08700 1 --2- 172.21.15.99:0/417959588 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd648066f30 0x7fd6480693e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd6680ca3f0 tx=0x7fd650017000 comp rx=0 tx=0).stop 2023-10-08T00:11:57.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.149+0000 7fd66dc08700 1 -- 172.21.15.99:0/417959588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd668069de0 msgr2=0x7fd6680c5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:57.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.149+0000 7fd66dc08700 1 --2- 172.21.15.99:0/417959588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd668069de0 0x7fd6680c5340 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd65800e8d0 tx=0x7fd658008040 comp rx=0 tx=0).stop 2023-10-08T00:11:57.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.149+0000 7fd66dc08700 1 -- 172.21.15.99:0/417959588 shutdown_connections 2023-10-08T00:11:57.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.149+0000 7fd66dc08700 1 --2- 172.21.15.99:0/417959588 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd648066f30 0x7fd6480693e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:57.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.149+0000 7fd66dc08700 1 --2- 172.21.15.99:0/417959588 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6680c9930 0x7fd66816aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:57.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.149+0000 7fd66dc08700 1 --2- 172.21.15.99:0/417959588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd668069de0 0x7fd6680c5340 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:57.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.149+0000 7fd66dc08700 1 --2- 172.21.15.99:0/417959588 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6680c9560 0x7fd66816a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:57.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.149+0000 7fd66dc08700 1 -- 172.21.15.99:0/417959588 >> 172.21.15.99:0/417959588 conn(0x7fd668067660 msgr2=0x7fd6680cb6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:57.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.149+0000 7fd66dc08700 1 -- 172.21.15.99:0/417959588 shutdown_connections 2023-10-08T00:11:57.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.149+0000 7fd66dc08700 1 -- 172.21.15.99:0/417959588 wait complete. 2023-10-08T00:11:57.151 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 17 2023-10-08T00:11:57.162 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:11:57.275 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.275+0000 7f440f59e700 1 Processor -- start 2023-10-08T00:11:57.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.275+0000 7f440f59e700 1 -- start start 2023-10-08T00:11:57.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.276+0000 7f440f59e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f44101579d0 0x7f441015b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:57.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.276+0000 7f440f59e700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4410070700 0x7f4410070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:57.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.276+0000 7f440f59e700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f441006e8a0 con 0x7f4410067660 2023-10-08T00:11:57.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.276+0000 7f440f59e700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4410071060 con 0x7f4410070700 2023-10-08T00:11:57.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.276+0000 7f440f59e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f441015bc40 con 0x7f44101579d0 2023-10-08T00:11:57.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.276+0000 7f440ed9d700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4410070700 0x7f4410070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:57.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.276+0000 7f440dd9b700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f4410067660 0x7f4410158ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56378/0 (socket says 172.21.15.99:56378) 2023-10-08T00:11:57.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.276+0000 7f440dd9b700 1 -- 172.21.15.99:0/800070189 learned_addr learned my addr 172.21.15.99:0/800070189 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:57.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.276+0000 7f440e59c700 1 --2- 172.21.15.99:0/800070189 >> v2:172.21.15.191:3300/0 conn(0x7f44101579d0 0x7f441015b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:57.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.276+0000 7f440ed9d700 1 -- 172.21.15.99:0/800070189 >> v2:172.21.15.191:3300/0 conn(0x7f44101579d0 msgr2=0x7f441015b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:57.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.276+0000 7f440ed9d700 1 --2- 172.21.15.99:0/800070189 >> v2:172.21.15.191:3300/0 conn(0x7f44101579d0 0x7f441015b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:57.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.276+0000 7f440ed9d700 1 -- 172.21.15.99:0/800070189 >> v1:172.21.15.99:6789/0 conn(0x7f4410067660 legacy=0x7f4410158ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:57.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.276+0000 7f440ed9d700 1 -- 172.21.15.99:0/800070189 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f441015bf20 con 0x7f4410070700 2023-10-08T00:11:57.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.276+0000 7f440d59a700 1 -- 172.21.15.99:0/800070189 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1414597894 0 0) 0x7f441006e8a0 con 0x7f4410067660 2023-10-08T00:11:57.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.276+0000 7f440ed9d700 1 --2- 172.21.15.99:0/800070189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4410070700 0x7f4410070b20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f441006b1a0 tx=0x7f4408009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17513cd14f9b22a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:57.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.277+0000 7f440d59a700 1 -- 172.21.15.99:0/800070189 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440800e070 con 0x7f4410070700 2023-10-08T00:11:57.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.277+0000 7f440d59a700 1 -- 172.21.15.99:0/800070189 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f440800bb60 con 0x7f4410070700 2023-10-08T00:11:57.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.277+0000 7f440d59a700 1 -- 172.21.15.99:0/800070189 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4408012430 con 0x7f4410070700 2023-10-08T00:11:57.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.277+0000 7f440f59e700 1 -- 172.21.15.99:0/800070189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4410070700 msgr2=0x7f4410070b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:57.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.277+0000 7f440f59e700 1 --2- 172.21.15.99:0/800070189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4410070700 0x7f4410070b20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f441006b1a0 tx=0x7f4408009780 comp rx=0 tx=0).stop 2023-10-08T00:11:57.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.277+0000 7f440f59e700 1 -- 172.21.15.99:0/800070189 shutdown_connections 2023-10-08T00:11:57.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.277+0000 7f440f59e700 1 --2- 172.21.15.99:0/800070189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4410070700 0x7f4410070b20 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:57.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.277+0000 7f440f59e700 1 --2- 172.21.15.99:0/800070189 >> v2:172.21.15.191:3300/0 conn(0x7f44101579d0 0x7f441015b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:57.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.277+0000 7f440f59e700 1 -- 172.21.15.99:0/800070189 >> 172.21.15.99:0/800070189 conn(0x7f44100cd8f0 msgr2=0x7f44100cdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:57.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.277+0000 7f440f59e700 1 -- 172.21.15.99:0/800070189 shutdown_connections 2023-10-08T00:11:57.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.277+0000 7f440f59e700 1 -- 172.21.15.99:0/800070189 wait complete. 2023-10-08T00:11:57.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.278+0000 7f440f59e700 1 Processor -- start 2023-10-08T00:11:57.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.278+0000 7f440f59e700 1 -- start start 2023-10-08T00:11:57.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.279+0000 7f440f59e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4410067660 0x7f441015a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:57.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.279+0000 7f440f59e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4410070700 0x7f44100c9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:57.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.279+0000 7f440f59e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44101579d0 0x7f44100ab190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:57.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.279+0000 7f440f59e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4410159830 con 0x7f44101579d0 2023-10-08T00:11:57.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.279+0000 7f440f59e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4410159b10 con 0x7f4410067660 2023-10-08T00:11:57.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.279+0000 7f440f59e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4410159df0 con 0x7f4410070700 2023-10-08T00:11:57.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.279+0000 7f440ed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44101579d0 0x7f44100ab190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:57.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.279+0000 7f440ed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44101579d0 0x7f44100ab190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58236/0 (socket says 172.21.15.99:58236) 2023-10-08T00:11:57.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.279+0000 7f440ed9d700 1 -- 172.21.15.99:0/772045323 learned_addr learned my addr 172.21.15.99:0/772045323 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:57.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.279+0000 7f440e59c700 1 --2- 172.21.15.99:0/772045323 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4410067660 0x7f441015a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:57.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.279+0000 7f440dd9b700 1 --2- 172.21.15.99:0/772045323 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4410070700 0x7f44100c9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:57.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.279+0000 7f440ed9d700 1 -- 172.21.15.99:0/772045323 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4410070700 msgr2=0x7f44100c9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:57.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.279+0000 7f440ed9d700 1 --2- 172.21.15.99:0/772045323 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4410070700 0x7f44100c9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:57.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.279+0000 7f440ed9d700 1 -- 172.21.15.99:0/772045323 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4410067660 msgr2=0x7f441015a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:57.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.279+0000 7f440ed9d700 1 --2- 172.21.15.99:0/772045323 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4410067660 0x7f441015a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:57.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.279+0000 7f440ed9d700 1 -- 172.21.15.99:0/772045323 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4408009a20 con 0x7f44101579d0 2023-10-08T00:11:57.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.280+0000 7f440ed9d700 1 --2- 172.21.15.99:0/772045323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44101579d0 0x7f44100ab190 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f441006b1d0 tx=0x7f4408019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:57.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.280+0000 7f43fe7fc700 1 -- 172.21.15.99:0/772045323 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4408027470 con 0x7f44101579d0 2023-10-08T00:11:57.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.280+0000 7f43fe7fc700 1 -- 172.21.15.99:0/772045323 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44080275d0 con 0x7f44101579d0 2023-10-08T00:11:57.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.280+0000 7f43fe7fc700 1 -- 172.21.15.99:0/772045323 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4408027990 con 0x7f44101579d0 2023-10-08T00:11:57.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.280+0000 7f440f59e700 1 -- 172.21.15.99:0/772045323 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f441015a0d0 con 0x7f44101579d0 2023-10-08T00:11:57.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.280+0000 7f440f59e700 1 -- 172.21.15.99:0/772045323 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44100c9e80 con 0x7f44101579d0 2023-10-08T00:11:57.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.281+0000 7f440f59e700 1 -- 172.21.15.99:0/772045323 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f441015a5f0 con 0x7f44101579d0 2023-10-08T00:11:57.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.281+0000 7f43fe7fc700 1 -- 172.21.15.99:0/772045323 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4408024070 con 0x7f44101579d0 2023-10-08T00:11:57.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.281+0000 7f43fe7fc700 1 --2- 172.21.15.99:0/772045323 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f43f0066f30 0x7f43f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:57.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.282+0000 7f43fe7fc700 1 -- 172.21.15.99:0/772045323 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4408086eb0 con 0x7f44101579d0 2023-10-08T00:11:57.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.282+0000 7f440e59c700 1 --2- 172.21.15.99:0/772045323 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f43f0066f30 0x7f43f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:57.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.282+0000 7f440e59c700 1 --2- 172.21.15.99:0/772045323 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f43f0066f30 0x7f43f00693e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4400009d10 tx=0x7f44000091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:57.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.286+0000 7f43fe7fc700 1 -- 172.21.15.99:0/772045323 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f441015a5f0 con 0x7f44101579d0 2023-10-08T00:11:57.512 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.511+0000 7f440f59e700 1 -- 172.21.15.99:0/772045323 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f441015a5f0 con 0x7f44101579d0 2023-10-08T00:11:57.513 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.512+0000 7f43fe7fc700 1 -- 172.21.15.99:0/772045323 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4600 (secure 0 0 0) 0x7f441015a5f0 con 0x7f44101579d0 2023-10-08T00:11:57.513 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:11:57.513 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4852,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/824084995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":824084995},{"type":"v1","addr":"172.21.15.191:6837","nonce":824084995}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4855,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/3349126920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3349126920},{"type":"v1","addr":"172.21.15.191:6836","nonce":3349126920}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4899,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/1004180438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1004180438},{"type":"v1","addr":"172.21.15.99:6834","nonce":1004180438}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:11:40.672879+0000","modified":"2023-10-08T00:11:44.314526+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4902},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4902":{"gid":4902,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/4189095717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":4189095717},{"type":"v1","addr":"172.21.15.99:6835","nonce":4189095717}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-08T00:11:57.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.514+0000 7f440f59e700 1 -- 172.21.15.99:0/772045323 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f43f0066f30 msgr2=0x7f43f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:57.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.514+0000 7f440f59e700 1 --2- 172.21.15.99:0/772045323 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f43f0066f30 0x7f43f00693e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4400009d10 tx=0x7f44000091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:57.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.514+0000 7f440f59e700 1 -- 172.21.15.99:0/772045323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44101579d0 msgr2=0x7f44100ab190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:57.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.514+0000 7f440f59e700 1 --2- 172.21.15.99:0/772045323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44101579d0 0x7f44100ab190 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f441006b1d0 tx=0x7f4408019000 comp rx=0 tx=0).stop 2023-10-08T00:11:57.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.514+0000 7f440f59e700 1 -- 172.21.15.99:0/772045323 shutdown_connections 2023-10-08T00:11:57.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.514+0000 7f440f59e700 1 --2- 172.21.15.99:0/772045323 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f43f0066f30 0x7f43f00693e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:57.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.514+0000 7f440f59e700 1 --2- 172.21.15.99:0/772045323 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4410067660 0x7f441015a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:57.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.514+0000 7f440f59e700 1 --2- 172.21.15.99:0/772045323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44101579d0 0x7f44100ab190 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:57.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.514+0000 7f440f59e700 1 --2- 172.21.15.99:0/772045323 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4410070700 0x7f44100c9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:57.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.514+0000 7f440f59e700 1 -- 172.21.15.99:0/772045323 >> 172.21.15.99:0/772045323 conn(0x7f44100cd8f0 msgr2=0x7f44100cb4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:57.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.514+0000 7f440f59e700 1 -- 172.21.15.99:0/772045323 shutdown_connections 2023-10-08T00:11:57.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.514+0000 7f440f59e700 1 -- 172.21.15.99:0/772045323 wait complete. 2023-10-08T00:11:57.516 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 17 2023-10-08T00:11:57.526 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:11:57.641 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.641+0000 7f9cfb59e700 1 Processor -- start 2023-10-08T00:11:57.642 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.641+0000 7f9cfb59e700 1 -- start start 2023-10-08T00:11:57.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.642+0000 7f9cfb59e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9cfc0cd6e0 0x7f9cfc0cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:57.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.642+0000 7f9cfb59e700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cfc0cb7c0 con 0x7f9cfc15cbd0 2023-10-08T00:11:57.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.642+0000 7f9cfb59e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cfc15e290 con 0x7f9cfc0cdab0 2023-10-08T00:11:57.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.642+0000 7f9cfb59e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cfc15e850 con 0x7f9cfc0cd6e0 2023-10-08T00:11:57.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.642+0000 7f9cfad9d700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f9cfc15cbd0 0x7f9cfc15d020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56394/0 (socket says 172.21.15.99:56394) 2023-10-08T00:11:57.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.642+0000 7f9cfad9d700 1 -- 172.21.15.99:0/915675578 learned_addr learned my addr 172.21.15.99:0/915675578 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:57.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.642+0000 7f9cfa59c700 1 --2- 172.21.15.99:0/915675578 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9cfc0cd6e0 0x7f9cfc0cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:57.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.642+0000 7f9cf959a700 1 -- 172.21.15.99:0/915675578 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4078776541 0 0) 0x7f9cfc0cb7c0 con 0x7f9cfc15cbd0 2023-10-08T00:11:57.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.642+0000 7f9cf959a700 1 -- 172.21.15.99:0/915675578 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ce8003570 con 0x7f9cfc15cbd0 2023-10-08T00:11:57.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.642+0000 7f9cf959a700 1 -- 172.21.15.99:0/915675578 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 58610211 0 0) 0x7f9ce8003570 con 0x7f9cfc15cbd0 2023-10-08T00:11:57.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.642+0000 7f9cf959a700 1 -- 172.21.15.99:0/915675578 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9cfc0cd6e0 msgr2=0x7f9cfc0cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:57.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.642+0000 7f9cf959a700 1 --2- 172.21.15.99:0/915675578 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9cfc0cd6e0 0x7f9cfc0cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:57.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.642+0000 7f9cf959a700 1 -- 172.21.15.99:0/915675578 >> v1:172.21.15.191:6789/0 conn(0x7f9cfc0cdab0 legacy=0x7f9cfc15d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:57.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.642+0000 7f9cf959a700 1 -- 172.21.15.99:0/915675578 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cfc15e570 con 0x7f9cfc15cbd0 2023-10-08T00:11:57.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.642+0000 7f9cf959a700 1 -- 172.21.15.99:0/915675578 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f9cec001470 con 0x7f9cfc15cbd0 2023-10-08T00:11:57.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.642+0000 7f9cf959a700 1 -- 172.21.15.99:0/915675578 >> v1:172.21.15.99:6789/0 conn(0x7f9cfc15cbd0 legacy=0x7f9cfc15d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:57.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.643+0000 7f9cf959a700 1 --2- 172.21.15.99:0/915675578 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ce8003a90 0x7f9ce8003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:57.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.643+0000 7f9cf959a700 1 --2- 172.21.15.99:0/915675578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ce8004810 0x7f9ce8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:57.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.643+0000 7f9cf959a700 1 --2- 172.21.15.99:0/915675578 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9cfc0cd6e0 0x7f9ce80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:57.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.643+0000 7f9cf959a700 1 -- 172.21.15.99:0/915675578 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cfc15e850 con 0x7f9ce8003a90 2023-10-08T00:11:57.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.643+0000 7f9cf959a700 1 -- 172.21.15.99:0/915675578 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce800bb10 con 0x7f9ce8004810 2023-10-08T00:11:57.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.643+0000 7f9cf959a700 1 -- 172.21.15.99:0/915675578 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce800bcd0 con 0x7f9cfc0cd6e0 2023-10-08T00:11:57.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.643+0000 7f9cfa59c700 1 --2- 172.21.15.99:0/915675578 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ce8003a90 0x7f9ce8003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:57.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.643+0000 7f9cfa59c700 1 -- 172.21.15.99:0/915675578 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9cfc0cd6e0 msgr2=0x7f9ce80051c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T00:11:57.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.643+0000 7f9cfa59c700 1 --2- 172.21.15.99:0/915675578 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9cfc0cd6e0 0x7f9ce80051c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:57.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.643+0000 7f9cfa59c700 1 -- 172.21.15.99:0/915675578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ce8004810 msgr2=0x7f9ce8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:57.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.643+0000 7f9cfa59c700 1 --2- 172.21.15.99:0/915675578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ce8004810 0x7f9ce8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:57.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.643+0000 7f9cfa59c700 1 -- 172.21.15.99:0/915675578 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce8003570 con 0x7f9ce8003a90 2023-10-08T00:11:57.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.643+0000 7f9cfa59c700 1 --2- 172.21.15.99:0/915675578 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ce8003a90 0x7f9ce8003e60 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400e210 tx=0x7f9cf400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7aeb60c9fac626d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:57.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.643+0000 7f9cf959a700 1 -- 172.21.15.99:0/915675578 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf400eac0 con 0x7f9ce8003a90 2023-10-08T00:11:57.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.643+0000 7f9cf959a700 1 -- 172.21.15.99:0/915675578 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf400f040 con 0x7f9ce8003a90 2023-10-08T00:11:57.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.643+0000 7f9cf959a700 1 -- 172.21.15.99:0/915675578 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf400ee70 con 0x7f9ce8003a90 2023-10-08T00:11:57.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.644+0000 7f9cfb59e700 1 -- 172.21.15.99:0/915675578 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ce8003a90 msgr2=0x7f9ce8003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:57.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.644+0000 7f9cfb59e700 1 --2- 172.21.15.99:0/915675578 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ce8003a90 0x7f9ce8003e60 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400e210 tx=0x7f9cf400b040 comp rx=0 tx=0).stop 2023-10-08T00:11:57.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.644+0000 7f9cfad9d700 1 -- 172.21.15.99:0/915675578 reap_dead start 2023-10-08T00:11:57.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.644+0000 7f9cfb59e700 1 -- 172.21.15.99:0/915675578 shutdown_connections 2023-10-08T00:11:57.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.644+0000 7f9cfb59e700 1 -- 172.21.15.99:0/915675578 >> 172.21.15.99:0/915675578 conn(0x7f9cfc067660 msgr2=0x7f9cfc067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:57.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.644+0000 7f9cfb59e700 1 -- 172.21.15.99:0/915675578 shutdown_connections 2023-10-08T00:11:57.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.644+0000 7f9cfb59e700 1 -- 172.21.15.99:0/915675578 wait complete. 2023-10-08T00:11:57.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.645+0000 7f9cfb59e700 1 Processor -- start 2023-10-08T00:11:57.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.645+0000 7f9cfb59e700 1 -- start start 2023-10-08T00:11:57.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.646+0000 7f9cfb59e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ce8003a90 0x7f9cfc147770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:57.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.646+0000 7f9cfb59e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9cfc0c8c40 0x7f9cfc0c90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:57.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.646+0000 7f9cfb59e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cfc0b5d60 0x7f9cfc0b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:57.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.646+0000 7f9cfb59e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cfc147cb0 con 0x7f9cfc0b5d60 2023-10-08T00:11:57.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.646+0000 7f9cfb59e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cfc0b58d0 con 0x7f9ce8003a90 2023-10-08T00:11:57.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.646+0000 7f9cfb59e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cfc0b4cc0 con 0x7f9cfc0c8c40 2023-10-08T00:11:57.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.646+0000 7f9cfad9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cfc0b5d60 0x7f9cfc0b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:57.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.646+0000 7f9cfad9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cfc0b5d60 0x7f9cfc0b61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58248/0 (socket says 172.21.15.99:58248) 2023-10-08T00:11:57.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.646+0000 7f9cfad9d700 1 -- 172.21.15.99:0/1352061364 learned_addr learned my addr 172.21.15.99:0/1352061364 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:57.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.646+0000 7f9cfa59c700 1 --2- 172.21.15.99:0/1352061364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ce8003a90 0x7f9cfc147770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:57.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.646+0000 7f9cf9d9b700 1 --2- 172.21.15.99:0/1352061364 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9cfc0c8c40 0x7f9cfc0c90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:57.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.646+0000 7f9cfad9d700 1 -- 172.21.15.99:0/1352061364 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9cfc0c8c40 msgr2=0x7f9cfc0c90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:57.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.646+0000 7f9cfad9d700 1 --2- 172.21.15.99:0/1352061364 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9cfc0c8c40 0x7f9cfc0c90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:57.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.646+0000 7f9cfad9d700 1 -- 172.21.15.99:0/1352061364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ce8003a90 msgr2=0x7f9cfc147770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:57.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.646+0000 7f9cfad9d700 1 --2- 172.21.15.99:0/1352061364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ce8003a90 0x7f9cfc147770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:57.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.646+0000 7f9cfad9d700 1 -- 172.21.15.99:0/1352061364 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cfc0b4fa0 con 0x7f9cfc0b5d60 2023-10-08T00:11:57.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.646+0000 7f9cfa59c700 1 --2- 172.21.15.99:0/1352061364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ce8003a90 0x7f9cfc147770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:11:57.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.646+0000 7f9cf9d9b700 1 --2- 172.21.15.99:0/1352061364 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9cfc0c8c40 0x7f9cfc0c90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:11:57.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.646+0000 7f9cfad9d700 1 --2- 172.21.15.99:0/1352061364 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cfc0b5d60 0x7f9cfc0b61d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc0cc500 tx=0x7f9cec001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:57.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.646+0000 7f9cf27fc700 1 -- 172.21.15.99:0/1352061364 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec007e30 con 0x7f9cfc0b5d60 2023-10-08T00:11:57.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.646+0000 7f9cf27fc700 1 -- 172.21.15.99:0/1352061364 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cec007360 con 0x7f9cfc0b5d60 2023-10-08T00:11:57.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.646+0000 7f9cfb59e700 1 -- 172.21.15.99:0/1352061364 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cfc0b5280 con 0x7f9cfc0b5d60 2023-10-08T00:11:57.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.646+0000 7f9cf27fc700 1 -- 172.21.15.99:0/1352061364 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec004830 con 0x7f9cfc0b5d60 2023-10-08T00:11:57.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.646+0000 7f9cfb59e700 1 -- 172.21.15.99:0/1352061364 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cfc158e00 con 0x7f9cfc0b5d60 2023-10-08T00:11:57.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.647+0000 7f9cfb59e700 1 -- 172.21.15.99:0/1352061364 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cfc089280 con 0x7f9cfc0b5d60 2023-10-08T00:11:57.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.648+0000 7f9cf27fc700 1 -- 172.21.15.99:0/1352061364 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9cec004c60 con 0x7f9cfc0b5d60 2023-10-08T00:11:57.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.648+0000 7f9cf27fc700 1 --2- 172.21.15.99:0/1352061364 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9cdc066f30 0x7f9cdc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:57.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.648+0000 7f9cf27fc700 1 -- 172.21.15.99:0/1352061364 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9cec0841a0 con 0x7f9cfc0b5d60 2023-10-08T00:11:57.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.648+0000 7f9cfa59c700 1 --2- 172.21.15.99:0/1352061364 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9cdc066f30 0x7f9cdc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:57.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.649+0000 7f9cfa59c700 1 --2- 172.21.15.99:0/1352061364 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9cdc066f30 0x7f9cdc0693e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400afd0 tx=0x7f9cf401a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:57.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.652+0000 7f9cf27fc700 1 -- 172.21.15.99:0/1352061364 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9cec051550 con 0x7f9cfc0b5d60 2023-10-08T00:11:57.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.833+0000 7f9cfb59e700 1 -- 172.21.15.99:0/1352061364 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9cfc04bb80 con 0x7f9cfc0b5d60 2023-10-08T00:11:57.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.834+0000 7f9cf27fc700 1 -- 172.21.15.99:0/1352061364 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f9cec054b70 con 0x7f9cfc0b5d60 2023-10-08T00:11:57.834 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:11:57.834 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":33,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:11:52.347370+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:11:38.415553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-08T00:11:39.634576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:11:57.836 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.835+0000 7f9cfb59e700 1 -- 172.21.15.99:0/1352061364 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9cdc066f30 msgr2=0x7f9cdc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:57.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.836+0000 7f9cfb59e700 1 --2- 172.21.15.99:0/1352061364 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9cdc066f30 0x7f9cdc0693e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400afd0 tx=0x7f9cf401a000 comp rx=0 tx=0).stop 2023-10-08T00:11:57.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.836+0000 7f9cfb59e700 1 -- 172.21.15.99:0/1352061364 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cfc0b5d60 msgr2=0x7f9cfc0b61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:57.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.836+0000 7f9cfb59e700 1 --2- 172.21.15.99:0/1352061364 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cfc0b5d60 0x7f9cfc0b61d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc0cc500 tx=0x7f9cec001b50 comp rx=0 tx=0).stop 2023-10-08T00:11:57.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.836+0000 7f9cfb59e700 1 -- 172.21.15.99:0/1352061364 shutdown_connections 2023-10-08T00:11:57.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.836+0000 7f9cfb59e700 1 --2- 172.21.15.99:0/1352061364 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9cdc066f30 0x7f9cdc0693e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:57.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.836+0000 7f9cfb59e700 1 --2- 172.21.15.99:0/1352061364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ce8003a90 0x7f9cfc147770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:57.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.836+0000 7f9cfb59e700 1 --2- 172.21.15.99:0/1352061364 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cfc0b5d60 0x7f9cfc0b61d0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:57.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.836+0000 7f9cfb59e700 1 --2- 172.21.15.99:0/1352061364 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9cfc0c8c40 0x7f9cfc0c90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:57.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.836+0000 7f9cfb59e700 1 -- 172.21.15.99:0/1352061364 >> 172.21.15.99:0/1352061364 conn(0x7f9cfc067660 msgr2=0x7f9cfc1566b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:57.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.836+0000 7f9cfb59e700 1 -- 172.21.15.99:0/1352061364 shutdown_connections 2023-10-08T00:11:57.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.836+0000 7f9cfb59e700 1 -- 172.21.15.99:0/1352061364 wait complete. 2023-10-08T00:11:57.848 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T00:11:57.965 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.964+0000 7fe5de3f8700 1 Processor -- start 2023-10-08T00:11:57.965 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.964+0000 7fe5de3f8700 1 -- start start 2023-10-08T00:11:57.968 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.965+0000 7fe5de3f8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5d8157c40 0x7fe5d8158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:57.968 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.965+0000 7fe5de3f8700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5d815cbb0 con 0x7fe5d815a620 2023-10-08T00:11:57.968 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.965+0000 7fe5de3f8700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5d815bfa0 con 0x7fe5d8157870 2023-10-08T00:11:57.968 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.965+0000 7fe5de3f8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d815bdb0 con 0x7fe5d8157c40 2023-10-08T00:11:57.968 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.965+0000 7fe5d77fe700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fe5d815a620 0x7fe5d815d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56398/0 (socket says 172.21.15.99:56398) 2023-10-08T00:11:57.968 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.965+0000 7fe5d77fe700 1 -- 172.21.15.99:0/1142150262 learned_addr learned my addr 172.21.15.99:0/1142150262 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:57.968 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.965+0000 7fe5d6ffd700 1 -- 172.21.15.99:0/1142150262 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 22559158 0 0) 0x7fe5d815cbb0 con 0x7fe5d815a620 2023-10-08T00:11:57.968 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.965+0000 7fe5dc995700 1 --2- 172.21.15.99:0/1142150262 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5d8157c40 0x7fe5d8158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:57.968 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.965+0000 7fe5d6ffd700 1 -- 172.21.15.99:0/1142150262 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5c4003570 con 0x7fe5d815a620 2023-10-08T00:11:57.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.965+0000 7fe5d6ffd700 1 -- 172.21.15.99:0/1142150262 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2359247335 0 0) 0x7fe5c4003570 con 0x7fe5d815a620 2023-10-08T00:11:57.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.965+0000 7fe5d6ffd700 1 -- 172.21.15.99:0/1142150262 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5d8157c40 msgr2=0x7fe5d8158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:57.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.965+0000 7fe5d6ffd700 1 --2- 172.21.15.99:0/1142150262 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5d8157c40 0x7fe5d8158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:57.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.965+0000 7fe5d6ffd700 1 -- 172.21.15.99:0/1142150262 >> v1:172.21.15.191:6789/0 conn(0x7fe5d8157870 legacy=0x7fe5d815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:57.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.965+0000 7fe5d6ffd700 1 -- 172.21.15.99:0/1142150262 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d815c280 con 0x7fe5d815a620 2023-10-08T00:11:57.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.965+0000 7fe5d6ffd700 1 -- 172.21.15.99:0/1142150262 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7fe5c0001470 con 0x7fe5d815a620 2023-10-08T00:11:57.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.965+0000 7fe5d6ffd700 1 -- 172.21.15.99:0/1142150262 >> v1:172.21.15.99:6789/0 conn(0x7fe5d815a620 legacy=0x7fe5d815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:57.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.965+0000 7fe5d6ffd700 1 --2- 172.21.15.99:0/1142150262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5c4003a90 0x7fe5c4003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:57.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.966+0000 7fe5d6ffd700 1 --2- 172.21.15.99:0/1142150262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5c4004810 0x7fe5c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:57.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.966+0000 7fe5d6ffd700 1 --2- 172.21.15.99:0/1142150262 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5d8157c40 0x7fe5c40051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:57.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.966+0000 7fe5d6ffd700 1 -- 172.21.15.99:0/1142150262 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c400bb30 con 0x7fe5c4003a90 2023-10-08T00:11:57.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.966+0000 7fe5d6ffd700 1 -- 172.21.15.99:0/1142150262 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c400bca0 con 0x7fe5c4004810 2023-10-08T00:11:57.970 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.966+0000 7fe5d6ffd700 1 -- 172.21.15.99:0/1142150262 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c400be60 con 0x7fe5d8157c40 2023-10-08T00:11:57.970 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.966+0000 7fe5d7fff700 1 --2- 172.21.15.99:0/1142150262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5c4003a90 0x7fe5c4003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:57.970 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.966+0000 7fe5d77fe700 1 --2- 172.21.15.99:0/1142150262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5c4004810 0x7fe5c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:57.970 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.966+0000 7fe5dc995700 1 --2- 172.21.15.99:0/1142150262 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5d8157c40 0x7fe5c40051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:57.970 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.966+0000 7fe5d7fff700 1 -- 172.21.15.99:0/1142150262 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5d8157c40 msgr2=0x7fe5c40051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:57.970 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.966+0000 7fe5d7fff700 1 --2- 172.21.15.99:0/1142150262 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5d8157c40 0x7fe5c40051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:57.970 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.966+0000 7fe5d7fff700 1 -- 172.21.15.99:0/1142150262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5c4004810 msgr2=0x7fe5c4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:57.970 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.966+0000 7fe5d7fff700 1 --2- 172.21.15.99:0/1142150262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5c4004810 0x7fe5c4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:57.970 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.966+0000 7fe5d7fff700 1 -- 172.21.15.99:0/1142150262 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c4003570 con 0x7fe5c4003a90 2023-10-08T00:11:57.970 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.966+0000 7fe5dc995700 1 --2- 172.21.15.99:0/1142150262 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5d8157c40 0x7fe5c40051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:11:57.970 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.966+0000 7fe5d7fff700 1 --2- 172.21.15.99:0/1142150262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5c4003a90 0x7fe5c4003ea0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe5c80018b0 tx=0x7fe5c8007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8425f2b71e2e20fe server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:57.971 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.966+0000 7fe5d6ffd700 1 -- 172.21.15.99:0/1142150262 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c8012420 con 0x7fe5c4003a90 2023-10-08T00:11:57.971 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.966+0000 7fe5d6ffd700 1 -- 172.21.15.99:0/1142150262 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c8012580 con 0x7fe5c4003a90 2023-10-08T00:11:57.971 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.966+0000 7fe5d6ffd700 1 -- 172.21.15.99:0/1142150262 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c8012920 con 0x7fe5c4003a90 2023-10-08T00:11:57.971 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.967+0000 7fe5de3f8700 1 -- 172.21.15.99:0/1142150262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5c4003a90 msgr2=0x7fe5c4003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:57.971 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.967+0000 7fe5de3f8700 1 --2- 172.21.15.99:0/1142150262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5c4003a90 0x7fe5c4003ea0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe5c80018b0 tx=0x7fe5c8007d30 comp rx=0 tx=0).stop 2023-10-08T00:11:57.971 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.967+0000 7fe5dc995700 1 -- 172.21.15.99:0/1142150262 reap_dead start 2023-10-08T00:11:57.971 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.967+0000 7fe5de3f8700 1 -- 172.21.15.99:0/1142150262 shutdown_connections 2023-10-08T00:11:57.971 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.967+0000 7fe5de3f8700 1 -- 172.21.15.99:0/1142150262 >> 172.21.15.99:0/1142150262 conn(0x7fe5d8068ce0 msgr2=0x7fe5d80690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:57.971 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.967+0000 7fe5de3f8700 1 -- 172.21.15.99:0/1142150262 shutdown_connections 2023-10-08T00:11:57.971 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.967+0000 7fe5de3f8700 1 -- 172.21.15.99:0/1142150262 wait complete. 2023-10-08T00:11:57.974 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.968+0000 7fe5de3f8700 1 Processor -- start 2023-10-08T00:11:57.974 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.968+0000 7fe5de3f8700 1 -- start start 2023-10-08T00:11:57.974 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.969+0000 7fe5de3f8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5c4003a90 0x7fe5d814e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:57.974 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.969+0000 7fe5de3f8700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5d814cbc0 0x7fe5d8153c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:57.974 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.969+0000 7fe5de3f8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5d8151350 0x7fe5d813c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:57.974 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.969+0000 7fe5de3f8700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d80b1f60 con 0x7fe5d8151350 2023-10-08T00:11:57.975 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.969+0000 7fe5de3f8700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d80b1630 con 0x7fe5d814cbc0 2023-10-08T00:11:57.975 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.969+0000 7fe5de3f8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d80b1910 con 0x7fe5c4003a90 2023-10-08T00:11:57.975 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.969+0000 7fe5dc995700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5d8151350 0x7fe5d813c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:57.975 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.969+0000 7fe5dc995700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5d8151350 0x7fe5d813c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58278/0 (socket says 172.21.15.99:58278) 2023-10-08T00:11:57.975 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.969+0000 7fe5dc995700 1 -- 172.21.15.99:0/338439979 learned_addr learned my addr 172.21.15.99:0/338439979 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:57.975 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.969+0000 7fe5d7fff700 1 --2- 172.21.15.99:0/338439979 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5c4003a90 0x7fe5d814e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:57.975 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.969+0000 7fe5d77fe700 1 --2- 172.21.15.99:0/338439979 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5d814cbc0 0x7fe5d8153c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:57.975 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.969+0000 7fe5dc995700 1 -- 172.21.15.99:0/338439979 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5c4003a90 msgr2=0x7fe5d814e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:57.975 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.969+0000 7fe5dc995700 1 --2- 172.21.15.99:0/338439979 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5c4003a90 0x7fe5d814e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:57.975 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.969+0000 7fe5dc995700 1 -- 172.21.15.99:0/338439979 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5d814cbc0 msgr2=0x7fe5d8153c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:57.975 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.969+0000 7fe5dc995700 1 --2- 172.21.15.99:0/338439979 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5d814cbc0 0x7fe5d8153c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:57.975 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.969+0000 7fe5dc995700 1 -- 172.21.15.99:0/338439979 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d80b1bf0 con 0x7fe5d8151350 2023-10-08T00:11:57.976 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.969+0000 7fe5d7fff700 1 --2- 172.21.15.99:0/338439979 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5c4003a90 0x7fe5d814e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:11:57.976 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.969+0000 7fe5d77fe700 1 --2- 172.21.15.99:0/338439979 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5d814cbc0 0x7fe5d8153c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:11:57.976 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.969+0000 7fe5dc995700 1 --2- 172.21.15.99:0/338439979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5d8151350 0x7fe5d813c340 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe5d80b3830 tx=0x7fe5d000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:57.976 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.969+0000 7fe5cffff700 1 -- 172.21.15.99:0/338439979 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d001b070 con 0x7fe5d8151350 2023-10-08T00:11:57.976 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.969+0000 7fe5cffff700 1 -- 172.21.15.99:0/338439979 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d000f040 con 0x7fe5d8151350 2023-10-08T00:11:57.976 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.969+0000 7fe5cffff700 1 -- 172.21.15.99:0/338439979 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d000e050 con 0x7fe5d8151350 2023-10-08T00:11:57.976 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.969+0000 7fe5de3f8700 1 -- 172.21.15.99:0/338439979 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5d814ef10 con 0x7fe5d8151350 2023-10-08T00:11:57.976 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.969+0000 7fe5de3f8700 1 -- 172.21.15.99:0/338439979 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5d81517d0 con 0x7fe5d8151350 2023-10-08T00:11:57.976 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.970+0000 7fe5de3f8700 1 -- 172.21.15.99:0/338439979 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5d804c3c0 con 0x7fe5d8151350 2023-10-08T00:11:57.976 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.971+0000 7fe5cffff700 1 -- 172.21.15.99:0/338439979 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe5d0003020 con 0x7fe5d8151350 2023-10-08T00:11:57.976 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.971+0000 7fe5cffff700 1 --2- 172.21.15.99:0/338439979 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe5b8066f30 0x7fe5b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:57.976 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.971+0000 7fe5cffff700 1 -- 172.21.15.99:0/338439979 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe5d0084990 con 0x7fe5d8151350 2023-10-08T00:11:57.977 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.971+0000 7fe5d7fff700 1 --2- 172.21.15.99:0/338439979 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe5b8066f30 0x7fe5b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:57.977 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.972+0000 7fe5d7fff700 1 --2- 172.21.15.99:0/338439979 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe5b8066f30 0x7fe5b80693e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8001010 tx=0x7fe5c8015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:57.977 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:57.975+0000 7fe5cffff700 1 -- 172.21.15.99:0/338439979 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe5d0051c00 con 0x7fe5d8151350 2023-10-08T00:11:58.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.200+0000 7fe5de3f8700 1 -- 172.21.15.99:0/338439979 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe5d815c560 con 0x7fe5d8151350 2023-10-08T00:11:58.201 INFO:tasks.ceph.mon.a.smithi099.stderr:2023-10-08T00:11:58.201+0000 7f6c60fa2700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T00:11:58.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.205+0000 7fe5cffff700 1 -- 172.21.15.99:0/338439979 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe5d0010ea0 con 0x7fe5d8151350 2023-10-08T00:11:58.207 INFO:tasks.ceph.mds.c.smithi099.stderr: -3373> 2023-10-08T00:11:44.272+0000 7fea12398700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T00:11:58.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.208+0000 7fe5de3f8700 1 -- 172.21.15.99:0/338439979 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe5b8066f30 msgr2=0x7fe5b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:58.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.208+0000 7fe5de3f8700 1 --2- 172.21.15.99:0/338439979 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe5b8066f30 0x7fe5b80693e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8001010 tx=0x7fe5c8015000 comp rx=0 tx=0).stop 2023-10-08T00:11:58.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.208+0000 7fe5de3f8700 1 -- 172.21.15.99:0/338439979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5d8151350 msgr2=0x7fe5d813c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:58.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.208+0000 7fe5de3f8700 1 --2- 172.21.15.99:0/338439979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5d8151350 0x7fe5d813c340 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe5d80b3830 tx=0x7fe5d000b040 comp rx=0 tx=0).stop 2023-10-08T00:11:58.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.208+0000 7fe5de3f8700 1 -- 172.21.15.99:0/338439979 shutdown_connections 2023-10-08T00:11:58.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.208+0000 7fe5de3f8700 1 --2- 172.21.15.99:0/338439979 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe5b8066f30 0x7fe5b80693e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:58.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.208+0000 7fe5de3f8700 1 --2- 172.21.15.99:0/338439979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5d8151350 0x7fe5d813c340 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:58.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.208+0000 7fe5de3f8700 1 --2- 172.21.15.99:0/338439979 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5c4003a90 0x7fe5d814e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:58.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.208+0000 7fe5de3f8700 1 --2- 172.21.15.99:0/338439979 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5d814cbc0 0x7fe5d8153c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:58.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.208+0000 7fe5de3f8700 1 -- 172.21.15.99:0/338439979 >> 172.21.15.99:0/338439979 conn(0x7fe5d8068ce0 msgr2=0x7fe5d80d10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:58.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.208+0000 7fe5de3f8700 1 -- 172.21.15.99:0/338439979 shutdown_connections 2023-10-08T00:11:58.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.208+0000 7fe5de3f8700 1 -- 172.21.15.99:0/338439979 wait complete. 2023-10-08T00:11:58.211 INFO:teuthology.orchestra.run.smithi099.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T00:11:58.222 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T00:11:58.232 INFO:tasks.ceph.mds.c.smithi099.stdout:starting mds.c at 2023-10-08T00:11:58.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.339+0000 7f4db82f5700 1 Processor -- start 2023-10-08T00:11:58.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.339+0000 7f4db82f5700 1 -- start start 2023-10-08T00:11:58.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.340+0000 7f4db82f5700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4db0159e60 0x7f4db015e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:58.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.340+0000 7f4db82f5700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4db015a230 0x7f4db00690b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:58.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.340+0000 7f4db82f5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4db00695f0 0x7f4db00af730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:58.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.340+0000 7f4db82f5700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4db00d1cf0 con 0x7f4db015a230 2023-10-08T00:11:58.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.340+0000 7f4db82f5700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4db0069a70 con 0x7f4db0159e60 2023-10-08T00:11:58.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.340+0000 7f4db82f5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4db015ea90 con 0x7f4db00695f0 2023-10-08T00:11:58.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.340+0000 7f4db5890700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4db015a230 0x7f4db00690b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:58.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.340+0000 7f4db5890700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4db015a230 0x7f4db00690b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58298/0 (socket says 172.21.15.99:58298) 2023-10-08T00:11:58.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.340+0000 7f4db5890700 1 -- 172.21.15.99:0/2895414589 learned_addr learned my addr 172.21.15.99:0/2895414589 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:58.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.340+0000 7f4db6091700 1 --2- 172.21.15.99:0/2895414589 >> v2:172.21.15.191:3300/0 conn(0x7f4db0159e60 0x7f4db015e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:58.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.340+0000 7f4db6892700 1 --2- 172.21.15.99:0/2895414589 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4db00695f0 0x7f4db00af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:58.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.340+0000 7f4db5890700 1 -- 172.21.15.99:0/2895414589 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4db00695f0 msgr2=0x7f4db00af730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:58.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.341+0000 7f4db5890700 1 --2- 172.21.15.99:0/2895414589 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4db00695f0 0x7f4db00af730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:58.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.341+0000 7f4db5890700 1 -- 172.21.15.99:0/2895414589 >> v2:172.21.15.191:3300/0 conn(0x7f4db0159e60 msgr2=0x7f4db015e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:58.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.341+0000 7f4db5890700 1 --2- 172.21.15.99:0/2895414589 >> v2:172.21.15.191:3300/0 conn(0x7f4db0159e60 0x7f4db015e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:58.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.341+0000 7f4db5890700 1 -- 172.21.15.99:0/2895414589 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db00b3100 con 0x7f4db015a230 2023-10-08T00:11:58.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.341+0000 7f4db6091700 1 --2- 172.21.15.99:0/2895414589 >> v2:172.21.15.191:3300/0 conn(0x7f4db0159e60 0x7f4db015e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:11:58.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.341+0000 7f4db5890700 1 --2- 172.21.15.99:0/2895414589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4db015a230 0x7f4db00690b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4db00cdf30 tx=0x7f4da0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcc9f115a8c16553 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:58.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.341+0000 7f4db508f700 1 -- 172.21.15.99:0/2895414589 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da000b820 con 0x7f4db015a230 2023-10-08T00:11:58.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.341+0000 7f4db508f700 1 -- 172.21.15.99:0/2895414589 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4da000b980 con 0x7f4db015a230 2023-10-08T00:11:58.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.341+0000 7f4db508f700 1 -- 172.21.15.99:0/2895414589 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da000bd20 con 0x7f4db015a230 2023-10-08T00:11:58.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.341+0000 7f4db82f5700 1 -- 172.21.15.99:0/2895414589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4db015a230 msgr2=0x7f4db00690b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:58.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.341+0000 7f4db82f5700 1 --2- 172.21.15.99:0/2895414589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4db015a230 0x7f4db00690b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4db00cdf30 tx=0x7f4da0009250 comp rx=0 tx=0).stop 2023-10-08T00:11:58.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.341+0000 7f4db82f5700 1 -- 172.21.15.99:0/2895414589 shutdown_connections 2023-10-08T00:11:58.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.341+0000 7f4db82f5700 1 --2- 172.21.15.99:0/2895414589 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4db00695f0 0x7f4db00af730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:58.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.341+0000 7f4db82f5700 1 --2- 172.21.15.99:0/2895414589 >> v2:172.21.15.191:3300/0 conn(0x7f4db0159e60 0x7f4db015e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:58.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.341+0000 7f4db82f5700 1 --2- 172.21.15.99:0/2895414589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4db015a230 0x7f4db00690b0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:58.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.341+0000 7f4db82f5700 1 -- 172.21.15.99:0/2895414589 >> 172.21.15.99:0/2895414589 conn(0x7f4db0070700 msgr2=0x7f4db0070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:58.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.341+0000 7f4db82f5700 1 -- 172.21.15.99:0/2895414589 shutdown_connections 2023-10-08T00:11:58.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.342+0000 7f4db82f5700 1 -- 172.21.15.99:0/2895414589 wait complete. 2023-10-08T00:11:58.348 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.343+0000 7f4db82f5700 1 Processor -- start 2023-10-08T00:11:58.348 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.343+0000 7f4db82f5700 1 -- start start 2023-10-08T00:11:58.348 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.343+0000 7f4db82f5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4db00695f0 0x7f4db014d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:58.348 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.343+0000 7f4db82f5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4db0159e60 0x7f4db015b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:58.348 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.343+0000 7f4db82f5700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4db015a230 0x7f4db015df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:58.348 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.343+0000 7f4db82f5700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db015e490 con 0x7f4db00695f0 2023-10-08T00:11:58.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.343+0000 7f4db82f5700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db00c9d50 con 0x7f4db015a230 2023-10-08T00:11:58.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.343+0000 7f4db82f5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4db00c9770 con 0x7f4db0159e60 2023-10-08T00:11:58.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.343+0000 7f4db6091700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4db00695f0 0x7f4db014d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:58.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.343+0000 7f4db6091700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4db00695f0 0x7f4db014d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58308/0 (socket says 172.21.15.99:58308) 2023-10-08T00:11:58.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.343+0000 7f4db6091700 1 -- 172.21.15.99:0/1616867941 learned_addr learned my addr 172.21.15.99:0/1616867941 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:58.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.344+0000 7f4db6892700 1 --2- 172.21.15.99:0/1616867941 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4db015a230 0x7f4db015df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:58.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.344+0000 7f4db5890700 1 --2- 172.21.15.99:0/1616867941 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4db0159e60 0x7f4db015b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:58.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.344+0000 7f4db6091700 1 -- 172.21.15.99:0/1616867941 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4db0159e60 msgr2=0x7f4db015b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:58.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.344+0000 7f4db6091700 1 --2- 172.21.15.99:0/1616867941 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4db0159e60 0x7f4db015b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:58.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.344+0000 7f4db6091700 1 -- 172.21.15.99:0/1616867941 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4db015a230 msgr2=0x7f4db015df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:58.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.344+0000 7f4db6091700 1 --2- 172.21.15.99:0/1616867941 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4db015a230 0x7f4db015df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:58.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.344+0000 7f4db6091700 1 -- 172.21.15.99:0/1616867941 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da00094f0 con 0x7f4db00695f0 2023-10-08T00:11:58.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.344+0000 7f4db5890700 1 --2- 172.21.15.99:0/1616867941 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4db0159e60 0x7f4db015b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:11:58.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.344+0000 7f4db6091700 1 --2- 172.21.15.99:0/1616867941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4db00695f0 0x7f4db014d4b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4dac00c860 tx=0x7f4dac008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:58.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.344+0000 7f4da67fc700 1 -- 172.21.15.99:0/1616867941 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dac011600 con 0x7f4db00695f0 2023-10-08T00:11:58.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.344+0000 7f4da67fc700 1 -- 172.21.15.99:0/1616867941 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dac011760 con 0x7f4db00695f0 2023-10-08T00:11:58.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.344+0000 7f4db82f5700 1 -- 172.21.15.99:0/1616867941 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4db015b690 con 0x7f4db00695f0 2023-10-08T00:11:58.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.344+0000 7f4da67fc700 1 -- 172.21.15.99:0/1616867941 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dac011b20 con 0x7f4db00695f0 2023-10-08T00:11:58.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.344+0000 7f4db82f5700 1 -- 172.21.15.99:0/1616867941 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4db016a320 con 0x7f4db00695f0 2023-10-08T00:11:58.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.345+0000 7f4db82f5700 1 -- 172.21.15.99:0/1616867941 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4db004c3c0 con 0x7f4db00695f0 2023-10-08T00:11:58.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.346+0000 7f4da67fc700 1 -- 172.21.15.99:0/1616867941 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4dac0193b0 con 0x7f4db00695f0 2023-10-08T00:11:58.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.346+0000 7f4da67fc700 1 --2- 172.21.15.99:0/1616867941 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4d94066f30 0x7f4d940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:58.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.346+0000 7f4da67fc700 1 -- 172.21.15.99:0/1616867941 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f4dac084610 con 0x7f4db00695f0 2023-10-08T00:11:58.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.346+0000 7f4db5890700 1 --2- 172.21.15.99:0/1616867941 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4d94066f30 0x7f4d940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:58.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.347+0000 7f4db5890700 1 --2- 172.21.15.99:0/1616867941 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4d94066f30 0x7f4d940693e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4db00cdf30 tx=0x7f4da0017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:58.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.350+0000 7f4da67fc700 1 -- 172.21.15.99:0/1616867941 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4dac051830 con 0x7f4db00695f0 2023-10-08T00:11:58.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:58.574+0000 7f4db82f5700 1 -- 172.21.15.99:0/1616867941 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4db01589e0 con 0x7f4db00695f0 2023-10-08T00:11:59.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.205+0000 7f4da67fc700 1 -- 172.21.15.99:0/1616867941 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4dac054e50 con 0x7f4db00695f0 2023-10-08T00:11:59.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.208+0000 7f4db82f5700 1 -- 172.21.15.99:0/1616867941 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4d94066f30 msgr2=0x7f4d940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:59.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.208+0000 7f4db82f5700 1 --2- 172.21.15.99:0/1616867941 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4d94066f30 0x7f4d940693e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4db00cdf30 tx=0x7f4da0017000 comp rx=0 tx=0).stop 2023-10-08T00:11:59.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.208+0000 7f4db82f5700 1 -- 172.21.15.99:0/1616867941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4db00695f0 msgr2=0x7f4db014d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:59.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.208+0000 7f4db82f5700 1 --2- 172.21.15.99:0/1616867941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4db00695f0 0x7f4db014d4b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4dac00c860 tx=0x7f4dac008040 comp rx=0 tx=0).stop 2023-10-08T00:11:59.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.208+0000 7f4db82f5700 1 -- 172.21.15.99:0/1616867941 shutdown_connections 2023-10-08T00:11:59.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.208+0000 7f4db82f5700 1 --2- 172.21.15.99:0/1616867941 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4d94066f30 0x7f4d940693e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:59.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.208+0000 7f4db82f5700 1 --2- 172.21.15.99:0/1616867941 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4db015a230 0x7f4db015df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:59.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.208+0000 7f4db82f5700 1 --2- 172.21.15.99:0/1616867941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4db00695f0 0x7f4db014d4b0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:59.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.208+0000 7f4db82f5700 1 --2- 172.21.15.99:0/1616867941 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4db0159e60 0x7f4db015b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:59.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.208+0000 7f4db82f5700 1 -- 172.21.15.99:0/1616867941 >> 172.21.15.99:0/1616867941 conn(0x7f4db0070700 msgr2=0x7f4db0070ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:59.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.208+0000 7f4db82f5700 1 -- 172.21.15.99:0/1616867941 shutdown_connections 2023-10-08T00:11:59.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.208+0000 7f4db82f5700 1 -- 172.21.15.99:0/1616867941 wait complete. 2023-10-08T00:11:59.222 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T00:11:59.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.386+0000 7f3ac9f72700 1 Processor -- start 2023-10-08T00:11:59.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.386+0000 7f3ac9f72700 1 -- start start 2023-10-08T00:11:59.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.387+0000 7f3ac9f72700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3ac40ccff0 0x7f3ac415e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:59.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.387+0000 7f3ac9f72700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f3ac406c9a0 0x7f3ac415d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:59.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.387+0000 7f3ac9f72700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ac40cd890 con 0x7f3ac406c5d0 2023-10-08T00:11:59.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.387+0000 7f3ac9f72700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ac40cd5b0 con 0x7f3ac40ccff0 2023-10-08T00:11:59.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.387+0000 7f3ac9f72700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ac415dd80 con 0x7f3ac406c9a0 2023-10-08T00:11:59.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.387+0000 7f3abbfff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3ac40ccff0 0x7f3ac415e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:59.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.387+0000 7f3ac37fe700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f3ac406c5d0 0x7f3ac415cf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56406/0 (socket says 172.21.15.99:56406) 2023-10-08T00:11:59.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.388+0000 7f3ac37fe700 1 -- 172.21.15.99:0/1201281211 learned_addr learned my addr 172.21.15.99:0/1201281211 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:11:59.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.388+0000 7f3ac3fff700 1 --2- 172.21.15.99:0/1201281211 >> v2:172.21.15.191:3300/0 conn(0x7f3ac406c9a0 0x7f3ac415d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:59.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.388+0000 7f3abbfff700 1 -- 172.21.15.99:0/1201281211 >> v2:172.21.15.191:3300/0 conn(0x7f3ac406c9a0 msgr2=0x7f3ac415d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:11:59.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.388+0000 7f3abbfff700 1 --2- 172.21.15.99:0/1201281211 >> v2:172.21.15.191:3300/0 conn(0x7f3ac406c9a0 0x7f3ac415d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:59.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.388+0000 7f3abbfff700 1 -- 172.21.15.99:0/1201281211 >> v1:172.21.15.99:6789/0 conn(0x7f3ac406c5d0 legacy=0x7f3ac415cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:59.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.388+0000 7f3abbfff700 1 -- 172.21.15.99:0/1201281211 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac415e060 con 0x7f3ac40ccff0 2023-10-08T00:11:59.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.388+0000 7f3ac2ffd700 1 -- 172.21.15.99:0/1201281211 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 69100114 0 0) 0x7f3ac40cd890 con 0x7f3ac406c5d0 2023-10-08T00:11:59.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.388+0000 7f3abbfff700 1 --2- 172.21.15.99:0/1201281211 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ac40ccff0 0x7f3ac415e860 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3ac415dc10 tx=0x7f3aac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fce7009850180b22 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:59.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.388+0000 7f3ac2ffd700 1 -- 172.21.15.99:0/1201281211 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aac011630 con 0x7f3ac40ccff0 2023-10-08T00:11:59.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.388+0000 7f3ac2ffd700 1 -- 172.21.15.99:0/1201281211 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aac011790 con 0x7f3ac40ccff0 2023-10-08T00:11:59.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.388+0000 7f3ac2ffd700 1 -- 172.21.15.99:0/1201281211 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aac011b30 con 0x7f3ac40ccff0 2023-10-08T00:11:59.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.389+0000 7f3ac9f72700 1 -- 172.21.15.99:0/1201281211 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ac40ccff0 msgr2=0x7f3ac415e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:59.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.389+0000 7f3ac9f72700 1 --2- 172.21.15.99:0/1201281211 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ac40ccff0 0x7f3ac415e860 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3ac415dc10 tx=0x7f3aac0091a0 comp rx=0 tx=0).stop 2023-10-08T00:11:59.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.389+0000 7f3ac9f72700 1 -- 172.21.15.99:0/1201281211 shutdown_connections 2023-10-08T00:11:59.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.389+0000 7f3ac9f72700 1 --2- 172.21.15.99:0/1201281211 >> v2:172.21.15.191:3300/0 conn(0x7f3ac406c9a0 0x7f3ac415d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:59.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.389+0000 7f3ac9f72700 1 --2- 172.21.15.99:0/1201281211 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ac40ccff0 0x7f3ac415e860 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:59.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.389+0000 7f3ac9f72700 1 -- 172.21.15.99:0/1201281211 >> 172.21.15.99:0/1201281211 conn(0x7f3ac40cafd0 msgr2=0x7f3ac406bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:11:59.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.389+0000 7f3ac9f72700 1 -- 172.21.15.99:0/1201281211 shutdown_connections 2023-10-08T00:11:59.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.389+0000 7f3ac9f72700 1 -- 172.21.15.99:0/1201281211 wait complete. 2023-10-08T00:11:59.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.390+0000 7f3ac9f72700 1 Processor -- start 2023-10-08T00:11:59.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.390+0000 7f3ac9f72700 1 -- start start 2023-10-08T00:11:59.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.391+0000 7f3ac9f72700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ac406c5d0 0x7f3ac4154220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:59.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.391+0000 7f3ac9f72700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ac406c9a0 0x7f3ac4153a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:59.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.391+0000 7f3ac9f72700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3ac40ccff0 0x7f3ac415f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:59.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.391+0000 7f3ac9f72700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac41614e0 con 0x7f3ac406c9a0 2023-10-08T00:11:59.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.391+0000 7f3ac9f72700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac4159510 con 0x7f3ac406c5d0 2023-10-08T00:11:59.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.391+0000 7f3ac9f72700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac41597f0 con 0x7f3ac40ccff0 2023-10-08T00:11:59.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.391+0000 7f3abbfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ac406c9a0 0x7f3ac4153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:59.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.391+0000 7f3abbfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ac406c9a0 0x7f3ac4153a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58328/0 (socket says 172.21.15.99:58328) 2023-10-08T00:11:59.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.391+0000 7f3abbfff700 1 -- 172.21.15.99:0/419312160 learned_addr learned my addr 172.21.15.99:0/419312160 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:11:59.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.391+0000 7f3ac3fff700 1 --2- 172.21.15.99:0/419312160 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3ac40ccff0 0x7f3ac415f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:59.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.391+0000 7f3ac37fe700 1 --2- 172.21.15.99:0/419312160 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ac406c5d0 0x7f3ac4154220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:59.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.391+0000 7f3abbfff700 1 -- 172.21.15.99:0/419312160 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3ac40ccff0 msgr2=0x7f3ac415f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:59.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.391+0000 7f3abbfff700 1 --2- 172.21.15.99:0/419312160 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3ac40ccff0 0x7f3ac415f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:59.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.391+0000 7f3abbfff700 1 -- 172.21.15.99:0/419312160 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ac406c5d0 msgr2=0x7f3ac4154220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:11:59.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.391+0000 7f3abbfff700 1 --2- 172.21.15.99:0/419312160 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ac406c5d0 0x7f3ac4154220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:11:59.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.391+0000 7f3abbfff700 1 -- 172.21.15.99:0/419312160 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aac009420 con 0x7f3ac406c9a0 2023-10-08T00:11:59.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.391+0000 7f3abbfff700 1 --2- 172.21.15.99:0/419312160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ac406c9a0 0x7f3ac4153a80 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4069ad0 tx=0x7f3aac017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:59.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.391+0000 7f3abb7fe700 1 -- 172.21.15.99:0/419312160 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aac000d10 con 0x7f3ac406c9a0 2023-10-08T00:11:59.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.391+0000 7f3abb7fe700 1 -- 172.21.15.99:0/419312160 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aac011b10 con 0x7f3ac406c9a0 2023-10-08T00:11:59.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.391+0000 7f3ac9f72700 1 -- 172.21.15.99:0/419312160 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ac4160e20 con 0x7f3ac406c9a0 2023-10-08T00:11:59.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.391+0000 7f3abb7fe700 1 -- 172.21.15.99:0/419312160 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aac025640 con 0x7f3ac406c9a0 2023-10-08T00:11:59.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.391+0000 7f3ac9f72700 1 -- 172.21.15.99:0/419312160 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ac4153fc0 con 0x7f3ac406c9a0 2023-10-08T00:11:59.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.392+0000 7f3ac9f72700 1 -- 172.21.15.99:0/419312160 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ac404c3c0 con 0x7f3ac406c9a0 2023-10-08T00:11:59.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.393+0000 7f3abb7fe700 1 -- 172.21.15.99:0/419312160 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3aac025a30 con 0x7f3ac406c9a0 2023-10-08T00:11:59.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.393+0000 7f3abb7fe700 1 --2- 172.21.15.99:0/419312160 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3aa4066f30 0x7f3aa40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:11:59.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.393+0000 7f3abb7fe700 1 -- 172.21.15.99:0/419312160 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f3aac085db0 con 0x7f3ac406c9a0 2023-10-08T00:11:59.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.393+0000 7f3ac37fe700 1 --2- 172.21.15.99:0/419312160 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3aa4066f30 0x7f3aa40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:11:59.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.394+0000 7f3ac37fe700 1 --2- 172.21.15.99:0/419312160 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3aa4066f30 0x7f3aa40693e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4007160 tx=0x7f3ab4007f50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:11:59.402 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.397+0000 7f3abb7fe700 1 -- 172.21.15.99:0/419312160 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3aac052fd0 con 0x7f3ac406c9a0 2023-10-08T00:11:59.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:11:59.579+0000 7f3ac9f72700 1 -- 172.21.15.99:0/419312160 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3ac40cb920 con 0x7f3ac406c9a0 2023-10-08T00:12:00.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.390+0000 7f3abb7fe700 1 -- 172.21.15.99:0/419312160 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v35) v1 ==== 181+0+0 (secure 0 0 0) 0x7f3aac0150d0 con 0x7f3ac406c9a0 2023-10-08T00:12:00.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.392+0000 7f3ac9f72700 1 -- 172.21.15.99:0/419312160 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3aa4066f30 msgr2=0x7f3aa40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:00.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.392+0000 7f3ac9f72700 1 --2- 172.21.15.99:0/419312160 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3aa4066f30 0x7f3aa40693e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4007160 tx=0x7f3ab4007f50 comp rx=0 tx=0).stop 2023-10-08T00:12:00.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.392+0000 7f3ac9f72700 1 -- 172.21.15.99:0/419312160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ac406c9a0 msgr2=0x7f3ac4153a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:00.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.392+0000 7f3ac9f72700 1 --2- 172.21.15.99:0/419312160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ac406c9a0 0x7f3ac4153a80 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4069ad0 tx=0x7f3aac017000 comp rx=0 tx=0).stop 2023-10-08T00:12:00.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.392+0000 7f3ac9f72700 1 -- 172.21.15.99:0/419312160 shutdown_connections 2023-10-08T00:12:00.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.392+0000 7f3ac9f72700 1 --2- 172.21.15.99:0/419312160 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3aa4066f30 0x7f3aa40693e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:00.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.392+0000 7f3ac9f72700 1 --2- 172.21.15.99:0/419312160 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ac406c5d0 0x7f3ac4154220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:00.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.392+0000 7f3ac9f72700 1 --2- 172.21.15.99:0/419312160 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3ac40ccff0 0x7f3ac415f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:00.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.392+0000 7f3ac9f72700 1 --2- 172.21.15.99:0/419312160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ac406c9a0 0x7f3ac4153a80 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:00.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.392+0000 7f3ac9f72700 1 -- 172.21.15.99:0/419312160 >> 172.21.15.99:0/419312160 conn(0x7f3ac40cafd0 msgr2=0x7f3ac406a920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:00.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.392+0000 7f3ac9f72700 1 -- 172.21.15.99:0/419312160 shutdown_connections 2023-10-08T00:12:00.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.392+0000 7f3ac9f72700 1 -- 172.21.15.99:0/419312160 wait complete. 2023-10-08T00:12:00.395 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' removed 2023-10-08T00:12:00.406 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T00:12:00.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.524+0000 7fe3a4e3c700 1 Processor -- start 2023-10-08T00:12:00.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.524+0000 7fe3a4e3c700 1 -- start start 2023-10-08T00:12:00.527 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.525+0000 7fe3a4e3c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe3a0159e50 0x7fe3a00b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:00.527 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.525+0000 7fe3a4e3c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fe3a0068cb0 0x7fe3a00690e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:00.527 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.525+0000 7fe3a4e3c700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3a00b0d60 con 0x7fe3a00b3370 2023-10-08T00:12:00.527 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.525+0000 7fe3a4e3c700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3a0069620 con 0x7fe3a0159e50 2023-10-08T00:12:00.527 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.525+0000 7fe3a4e3c700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3a00d15b0 con 0x7fe3a0068cb0 2023-10-08T00:12:00.527 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.525+0000 7fe39e59c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe3a0159e50 0x7fe3a00b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:00.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.525+0000 7fe39e59c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe3a0159e50 0x7fe3a00b2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58334/0 (socket says 172.21.15.99:58334) 2023-10-08T00:12:00.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.525+0000 7fe39e59c700 1 -- 172.21.15.99:0/654387360 learned_addr learned my addr 172.21.15.99:0/654387360 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:00.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.525+0000 7fe39e59c700 1 -- 172.21.15.99:0/654387360 >> v2:172.21.15.191:3300/0 conn(0x7fe3a0068cb0 msgr2=0x7fe3a00690e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:00.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.525+0000 7fe39ed9d700 1 --2- 172.21.15.99:0/654387360 >> v2:172.21.15.191:3300/0 conn(0x7fe3a0068cb0 0x7fe3a00690e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:00.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.525+0000 7fe39e59c700 1 --2- 172.21.15.99:0/654387360 >> v2:172.21.15.191:3300/0 conn(0x7fe3a0068cb0 0x7fe3a00690e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:00.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.525+0000 7fe39e59c700 1 -- 172.21.15.99:0/654387360 >> v1:172.21.15.191:6789/0 conn(0x7fe3a00b3370 legacy=0x7fe3a00b11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:00.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.525+0000 7fe39e59c700 1 -- 172.21.15.99:0/654387360 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a00d1890 con 0x7fe3a0159e50 2023-10-08T00:12:00.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.525+0000 7fe39ed9d700 1 --2- 172.21.15.99:0/654387360 >> v2:172.21.15.191:3300/0 conn(0x7fe3a0068cb0 0x7fe3a00690e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:12:00.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.526+0000 7fe39e59c700 1 --2- 172.21.15.99:0/654387360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3a0159e50 0x7fe3a00b2e30 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0064410 tx=0x7fe390009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9a016a713e0ccde server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:00.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.526+0000 7fe39dd9b700 1 -- 172.21.15.99:0/654387360 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39000b820 con 0x7fe3a0159e50 2023-10-08T00:12:00.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.526+0000 7fe39dd9b700 1 -- 172.21.15.99:0/654387360 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe39000b980 con 0x7fe3a0159e50 2023-10-08T00:12:00.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.526+0000 7fe39dd9b700 1 -- 172.21.15.99:0/654387360 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39000bd20 con 0x7fe3a0159e50 2023-10-08T00:12:00.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.526+0000 7fe3a4e3c700 1 -- 172.21.15.99:0/654387360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3a0159e50 msgr2=0x7fe3a00b2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:00.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.526+0000 7fe3a4e3c700 1 --2- 172.21.15.99:0/654387360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3a0159e50 0x7fe3a00b2e30 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0064410 tx=0x7fe390009250 comp rx=0 tx=0).stop 2023-10-08T00:12:00.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.526+0000 7fe3a4e3c700 1 -- 172.21.15.99:0/654387360 shutdown_connections 2023-10-08T00:12:00.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.526+0000 7fe3a4e3c700 1 --2- 172.21.15.99:0/654387360 >> v2:172.21.15.191:3300/0 conn(0x7fe3a0068cb0 0x7fe3a00690e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:00.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.526+0000 7fe3a4e3c700 1 --2- 172.21.15.99:0/654387360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3a0159e50 0x7fe3a00b2e30 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:00.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.526+0000 7fe3a4e3c700 1 -- 172.21.15.99:0/654387360 >> 172.21.15.99:0/654387360 conn(0x7fe3a0068130 msgr2=0x7fe3a00d0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:00.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.526+0000 7fe3a4e3c700 1 -- 172.21.15.99:0/654387360 shutdown_connections 2023-10-08T00:12:00.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.526+0000 7fe3a4e3c700 1 -- 172.21.15.99:0/654387360 wait complete. 2023-10-08T00:12:00.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.527+0000 7fe3a4e3c700 1 Processor -- start 2023-10-08T00:12:00.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.527+0000 7fe3a4e3c700 1 -- start start 2023-10-08T00:12:00.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.528+0000 7fe3a4e3c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe3a0068cb0 0x7fe3a00ab5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:00.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.528+0000 7fe3a4e3c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe3a00b3370 0x7fe3a00c9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:00.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.528+0000 7fe3a4e3c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3a0159e50 0x7fe3a00c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:00.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.528+0000 7fe3a4e3c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a016a3d0 con 0x7fe3a0159e50 2023-10-08T00:12:00.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.528+0000 7fe3a4e3c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a016a530 con 0x7fe3a00b3370 2023-10-08T00:12:00.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.528+0000 7fe3a4e3c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a00a9700 con 0x7fe3a0068cb0 2023-10-08T00:12:00.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.528+0000 7fe39ed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3a0159e50 0x7fe3a00c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:00.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.528+0000 7fe39ed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3a0159e50 0x7fe3a00c9d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58344/0 (socket says 172.21.15.99:58344) 2023-10-08T00:12:00.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.528+0000 7fe39ed9d700 1 -- 172.21.15.99:0/938673653 learned_addr learned my addr 172.21.15.99:0/938673653 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:00.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.528+0000 7fe39e59c700 1 --2- 172.21.15.99:0/938673653 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe3a0068cb0 0x7fe3a00ab5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:00.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.528+0000 7fe395d9b700 1 --2- 172.21.15.99:0/938673653 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe3a00b3370 0x7fe3a00c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:00.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.529+0000 7fe39ed9d700 1 -- 172.21.15.99:0/938673653 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe3a0068cb0 msgr2=0x7fe3a00ab5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:00.534 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.529+0000 7fe39ed9d700 1 --2- 172.21.15.99:0/938673653 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe3a0068cb0 0x7fe3a00ab5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:00.534 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.529+0000 7fe39ed9d700 1 -- 172.21.15.99:0/938673653 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe3a00b3370 msgr2=0x7fe3a00c9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:00.534 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.529+0000 7fe39ed9d700 1 --2- 172.21.15.99:0/938673653 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe3a00b3370 0x7fe3a00c9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:00.534 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.529+0000 7fe39ed9d700 1 -- 172.21.15.99:0/938673653 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3900094f0 con 0x7fe3a0159e50 2023-10-08T00:12:00.534 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.529+0000 7fe39ed9d700 1 --2- 172.21.15.99:0/938673653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3a0159e50 0x7fe3a00c9d40 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe39800cef0 tx=0x7fe398007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:00.534 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.529+0000 7fe396ffd700 1 -- 172.21.15.99:0/938673653 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39800ad20 con 0x7fe3a0159e50 2023-10-08T00:12:00.534 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.529+0000 7fe396ffd700 1 -- 172.21.15.99:0/938673653 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe398016420 con 0x7fe3a0159e50 2023-10-08T00:12:00.534 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.529+0000 7fe3a4e3c700 1 -- 172.21.15.99:0/938673653 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3a00a99e0 con 0x7fe3a0159e50 2023-10-08T00:12:00.534 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.529+0000 7fe396ffd700 1 -- 172.21.15.99:0/938673653 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39800aea0 con 0x7fe3a0159e50 2023-10-08T00:12:00.534 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.529+0000 7fe3a4e3c700 1 -- 172.21.15.99:0/938673653 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3a00a9fa0 con 0x7fe3a0159e50 2023-10-08T00:12:00.535 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.530+0000 7fe3a4e3c700 1 -- 172.21.15.99:0/938673653 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3a016a6e0 con 0x7fe3a0159e50 2023-10-08T00:12:00.535 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.530+0000 7fe396ffd700 1 -- 172.21.15.99:0/938673653 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe398016a70 con 0x7fe3a0159e50 2023-10-08T00:12:00.535 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.531+0000 7fe396ffd700 1 --2- 172.21.15.99:0/938673653 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe380066f30 0x7fe3800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:00.535 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.531+0000 7fe396ffd700 1 -- 172.21.15.99:0/938673653 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5353+0+0 (secure 0 0 0) 0x7fe398012070 con 0x7fe3a0159e50 2023-10-08T00:12:00.535 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.531+0000 7fe39e59c700 1 --2- 172.21.15.99:0/938673653 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe380066f30 0x7fe3800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:00.535 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.531+0000 7fe39e59c700 1 --2- 172.21.15.99:0/938673653 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe380066f30 0x7fe3800693e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0064410 tx=0x7fe390016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:00.535 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.535+0000 7fe396ffd700 1 -- 172.21.15.99:0/938673653 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe39804e5a0 con 0x7fe3a0159e50 2023-10-08T00:12:00.718 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:00.717+0000 7fe3a4e3c700 1 -- 172.21.15.99:0/938673653 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe3a004c3c0 con 0x7fe3a0159e50 2023-10-08T00:12:01.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.396+0000 7fe396ffd700 1 -- 172.21.15.99:0/938673653 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v36) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe3a016a6e0 con 0x7fe3a0159e50 2023-10-08T00:12:01.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.398+0000 7fe3a4e3c700 1 -- 172.21.15.99:0/938673653 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe380066f30 msgr2=0x7fe3800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:01.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.398+0000 7fe3a4e3c700 1 --2- 172.21.15.99:0/938673653 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe380066f30 0x7fe3800693e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0064410 tx=0x7fe390016000 comp rx=0 tx=0).stop 2023-10-08T00:12:01.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.398+0000 7fe3a4e3c700 1 -- 172.21.15.99:0/938673653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3a0159e50 msgr2=0x7fe3a00c9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:01.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.398+0000 7fe3a4e3c700 1 --2- 172.21.15.99:0/938673653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3a0159e50 0x7fe3a00c9d40 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe39800cef0 tx=0x7fe398007860 comp rx=0 tx=0).stop 2023-10-08T00:12:01.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.398+0000 7fe3a4e3c700 1 -- 172.21.15.99:0/938673653 shutdown_connections 2023-10-08T00:12:01.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.398+0000 7fe3a4e3c700 1 --2- 172.21.15.99:0/938673653 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe380066f30 0x7fe3800693e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:01.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.398+0000 7fe3a4e3c700 1 --2- 172.21.15.99:0/938673653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3a0159e50 0x7fe3a00c9d40 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:01.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.398+0000 7fe3a4e3c700 1 --2- 172.21.15.99:0/938673653 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe3a0068cb0 0x7fe3a00ab5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:01.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.398+0000 7fe3a4e3c700 1 --2- 172.21.15.99:0/938673653 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe3a00b3370 0x7fe3a00c9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:01.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.398+0000 7fe3a4e3c700 1 -- 172.21.15.99:0/938673653 >> 172.21.15.99:0/938673653 conn(0x7fe3a0068130 msgr2=0x7fe3a00b5b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:01.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.399+0000 7fe3a4e3c700 1 -- 172.21.15.99:0/938673653 shutdown_connections 2023-10-08T00:12:01.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.399+0000 7fe3a4e3c700 1 -- 172.21.15.99:0/938673653 wait complete. 2023-10-08T00:12:01.401 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' removed 2023-10-08T00:12:01.412 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-08T00:12:01.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.528+0000 7ff2767dd700 1 Processor -- start 2023-10-08T00:12:01.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.528+0000 7ff2767dd700 1 -- start start 2023-10-08T00:12:01.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.529+0000 7ff2767dd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff2700cce30 0x7ff27015aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:01.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.529+0000 7ff2767dd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff27006aa00 0x7ff2700b5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:01.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.529+0000 7ff2767dd700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2700b4d70 con 0x7ff2700b6580 2023-10-08T00:12:01.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.529+0000 7ff2767dd700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2700cd8d0 con 0x7ff2700cce30 2023-10-08T00:12:01.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.529+0000 7ff2767dd700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2700b4440 con 0x7ff27006aa00 2023-10-08T00:12:01.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.529+0000 7ff2757db700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff2700cce30 0x7ff27015aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:01.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.529+0000 7ff2757db700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff2700cce30 0x7ff27015aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58354/0 (socket says 172.21.15.99:58354) 2023-10-08T00:12:01.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.529+0000 7ff2757db700 1 -- 172.21.15.99:0/1008422639 learned_addr learned my addr 172.21.15.99:0/1008422639 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:01.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.529+0000 7ff275fdc700 1 --2- 172.21.15.99:0/1008422639 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff27006aa00 0x7ff2700b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:01.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.529+0000 7ff2757db700 1 -- 172.21.15.99:0/1008422639 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff27006aa00 msgr2=0x7ff2700b5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:01.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.529+0000 7ff2757db700 1 --2- 172.21.15.99:0/1008422639 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff27006aa00 0x7ff2700b5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:01.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.529+0000 7ff2757db700 1 -- 172.21.15.99:0/1008422639 >> v1:172.21.15.191:6789/0 conn(0x7ff2700b6580 legacy=0x7ff27006a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:01.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.529+0000 7ff2757db700 1 -- 172.21.15.99:0/1008422639 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2700b4720 con 0x7ff2700cce30 2023-10-08T00:12:01.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.529+0000 7ff275fdc700 1 --2- 172.21.15.99:0/1008422639 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff27006aa00 0x7ff2700b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:12:01.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.530+0000 7ff2757db700 1 --2- 172.21.15.99:0/1008422639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2700cce30 0x7ff27015aca0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff26c009f20 tx=0x7ff26c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9435494c9bebb92e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:01.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.530+0000 7ff267fff700 1 -- 172.21.15.99:0/1008422639 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26c00e070 con 0x7ff2700cce30 2023-10-08T00:12:01.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.530+0000 7ff267fff700 1 -- 172.21.15.99:0/1008422639 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff26c00ba40 con 0x7ff2700cce30 2023-10-08T00:12:01.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.530+0000 7ff267fff700 1 -- 172.21.15.99:0/1008422639 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26c0092e0 con 0x7ff2700cce30 2023-10-08T00:12:01.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.530+0000 7ff2767dd700 1 -- 172.21.15.99:0/1008422639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2700cce30 msgr2=0x7ff27015aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:01.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.530+0000 7ff2767dd700 1 --2- 172.21.15.99:0/1008422639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2700cce30 0x7ff27015aca0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff26c009f20 tx=0x7ff26c009840 comp rx=0 tx=0).stop 2023-10-08T00:12:01.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.530+0000 7ff2767dd700 1 -- 172.21.15.99:0/1008422639 shutdown_connections 2023-10-08T00:12:01.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.530+0000 7ff2767dd700 1 --2- 172.21.15.99:0/1008422639 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff27006aa00 0x7ff2700b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:01.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.530+0000 7ff2767dd700 1 --2- 172.21.15.99:0/1008422639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2700cce30 0x7ff27015aca0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:01.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.530+0000 7ff2767dd700 1 -- 172.21.15.99:0/1008422639 >> 172.21.15.99:0/1008422639 conn(0x7ff270068cb0 msgr2=0x7ff2700690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:01.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.530+0000 7ff2767dd700 1 -- 172.21.15.99:0/1008422639 shutdown_connections 2023-10-08T00:12:01.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.530+0000 7ff2767dd700 1 -- 172.21.15.99:0/1008422639 wait complete. 2023-10-08T00:12:01.536 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.531+0000 7ff2767dd700 1 Processor -- start 2023-10-08T00:12:01.536 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.531+0000 7ff2767dd700 1 -- start start 2023-10-08T00:12:01.536 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.532+0000 7ff2767dd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff27006aa00 0x7ff27014ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:01.536 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.532+0000 7ff2767dd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff2700b6580 0x7ff270154f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:01.536 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.532+0000 7ff2767dd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2700cce30 0x7ff2701554d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:01.536 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.532+0000 7ff2767dd700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27014f140 con 0x7ff2700cce30 2023-10-08T00:12:01.536 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.532+0000 7ff2767dd700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2700b1d00 con 0x7ff2700b6580 2023-10-08T00:12:01.537 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.532+0000 7ff2767dd700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2700b1fe0 con 0x7ff27006aa00 2023-10-08T00:12:01.537 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.532+0000 7ff275fdc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2700cce30 0x7ff2701554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:01.537 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.532+0000 7ff275fdc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2700cce30 0x7ff2701554d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58356/0 (socket says 172.21.15.99:58356) 2023-10-08T00:12:01.537 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.532+0000 7ff275fdc700 1 -- 172.21.15.99:0/536683614 learned_addr learned my addr 172.21.15.99:0/536683614 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:01.537 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.532+0000 7ff2757db700 1 --2- 172.21.15.99:0/536683614 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff27006aa00 0x7ff27014ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:01.537 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.533+0000 7ff274fda700 1 --2- 172.21.15.99:0/536683614 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff2700b6580 0x7ff270154f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:01.537 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.533+0000 7ff275fdc700 1 -- 172.21.15.99:0/536683614 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff27006aa00 msgr2=0x7ff27014ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:01.537 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.533+0000 7ff275fdc700 1 --2- 172.21.15.99:0/536683614 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff27006aa00 0x7ff27014ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:01.537 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.533+0000 7ff275fdc700 1 -- 172.21.15.99:0/536683614 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff2700b6580 msgr2=0x7ff270154f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:01.537 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.533+0000 7ff275fdc700 1 --2- 172.21.15.99:0/536683614 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff2700b6580 0x7ff270154f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:01.537 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.533+0000 7ff275fdc700 1 -- 172.21.15.99:0/536683614 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff26c009ae0 con 0x7ff2700cce30 2023-10-08T00:12:01.537 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.533+0000 7ff275fdc700 1 --2- 172.21.15.99:0/536683614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2700cce30 0x7ff2701554d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff26800c8b0 tx=0x7ff268008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:01.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.533+0000 7ff2657fa700 1 -- 172.21.15.99:0/536683614 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff268011600 con 0x7ff2700cce30 2023-10-08T00:12:01.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.533+0000 7ff2657fa700 1 -- 172.21.15.99:0/536683614 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff268011760 con 0x7ff2700cce30 2023-10-08T00:12:01.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.533+0000 7ff2767dd700 1 -- 172.21.15.99:0/536683614 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2700b0720 con 0x7ff2700cce30 2023-10-08T00:12:01.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.533+0000 7ff2657fa700 1 -- 172.21.15.99:0/536683614 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff268011b20 con 0x7ff2700cce30 2023-10-08T00:12:01.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.533+0000 7ff2767dd700 1 -- 172.21.15.99:0/536683614 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff270155de0 con 0x7ff2700cce30 2023-10-08T00:12:01.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.534+0000 7ff2767dd700 1 -- 172.21.15.99:0/536683614 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff270089550 con 0x7ff2700cce30 2023-10-08T00:12:01.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.534+0000 7ff2657fa700 1 -- 172.21.15.99:0/536683614 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff268018350 con 0x7ff2700cce30 2023-10-08T00:12:01.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.535+0000 7ff2657fa700 1 --2- 172.21.15.99:0/536683614 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff254066f30 0x7ff2540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:01.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.535+0000 7ff2657fa700 1 -- 172.21.15.99:0/536683614 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4939+0+0 (secure 0 0 0) 0x7ff268084120 con 0x7ff2700cce30 2023-10-08T00:12:01.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.535+0000 7ff2757db700 1 --2- 172.21.15.99:0/536683614 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff254066f30 0x7ff2540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:01.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.535+0000 7ff2757db700 1 --2- 172.21.15.99:0/536683614 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff254066f30 0x7ff2540693e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff26c009880 tx=0x7ff26c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:01.539 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.539+0000 7ff2657fa700 1 -- 172.21.15.99:0/536683614 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff268051690 con 0x7ff2700cce30 2023-10-08T00:12:01.724 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.724+0000 7ff2767dd700 1 -- 172.21.15.99:0/536683614 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7ff2700b4a00 con 0x7ff2700cce30 2023-10-08T00:12:01.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.727+0000 7ff2657fa700 1 -- 172.21.15.99:0/536683614 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v3) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff268054cb0 con 0x7ff2700cce30 2023-10-08T00:12:01.730 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.729+0000 7ff2767dd700 1 -- 172.21.15.99:0/536683614 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff254066f30 msgr2=0x7ff2540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:01.730 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.729+0000 7ff2767dd700 1 --2- 172.21.15.99:0/536683614 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff254066f30 0x7ff2540693e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff26c009880 tx=0x7ff26c017000 comp rx=0 tx=0).stop 2023-10-08T00:12:01.730 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.729+0000 7ff2767dd700 1 -- 172.21.15.99:0/536683614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2700cce30 msgr2=0x7ff2701554d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:01.730 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.729+0000 7ff2767dd700 1 --2- 172.21.15.99:0/536683614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2700cce30 0x7ff2701554d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff26800c8b0 tx=0x7ff268008040 comp rx=0 tx=0).stop 2023-10-08T00:12:01.730 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.729+0000 7ff2767dd700 1 -- 172.21.15.99:0/536683614 shutdown_connections 2023-10-08T00:12:01.730 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.729+0000 7ff2767dd700 1 --2- 172.21.15.99:0/536683614 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff254066f30 0x7ff2540693e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:01.731 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.729+0000 7ff2767dd700 1 --2- 172.21.15.99:0/536683614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2700cce30 0x7ff2701554d0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:01.731 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.729+0000 7ff2767dd700 1 --2- 172.21.15.99:0/536683614 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff27006aa00 0x7ff27014ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:01.731 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.729+0000 7ff2767dd700 1 --2- 172.21.15.99:0/536683614 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff2700b6580 0x7ff270154f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:01.731 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.729+0000 7ff2767dd700 1 -- 172.21.15.99:0/536683614 >> 172.21.15.99:0/536683614 conn(0x7ff270068cb0 msgr2=0x7ff270159540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:01.731 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.729+0000 7ff2767dd700 1 -- 172.21.15.99:0/536683614 shutdown_connections 2023-10-08T00:12:01.731 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.729+0000 7ff2767dd700 1 -- 172.21.15.99:0/536683614 wait complete. 2023-10-08T00:12:01.742 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_barrier' 2023-10-08T00:12:01.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.855+0000 7fc8bb0ea700 1 Processor -- start 2023-10-08T00:12:01.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.855+0000 7fc8bb0ea700 1 -- start start 2023-10-08T00:12:01.859 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.856+0000 7fc8bb0ea700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc8b41577d0 0x7fc8b40cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:01.859 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.856+0000 7fc8bb0ea700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8b4157ba0 0x7fc8b406a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:01.859 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.856+0000 7fc8bb0ea700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8b415c360 con 0x7fc8b415b0d0 2023-10-08T00:12:01.859 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.856+0000 7fc8bb0ea700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8b415b790 con 0x7fc8b41577d0 2023-10-08T00:12:01.859 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.856+0000 7fc8bb0ea700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b415d800 con 0x7fc8b4157ba0 2023-10-08T00:12:01.859 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.856+0000 7fc8b8e86700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc8b41577d0 0x7fc8b40cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:01.859 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.856+0000 7fc8b8e86700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc8b41577d0 0x7fc8b40cd770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58366/0 (socket says 172.21.15.99:58366) 2023-10-08T00:12:01.859 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.856+0000 7fc8b8e86700 1 -- 172.21.15.99:0/2378081832 learned_addr learned my addr 172.21.15.99:0/2378081832 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:01.859 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.857+0000 7fc8abfff700 1 --2- 172.21.15.99:0/2378081832 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8b4157ba0 0x7fc8b406a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:01.859 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.857+0000 7fc8b8e86700 1 -- 172.21.15.99:0/2378081832 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8b4157ba0 msgr2=0x7fc8b406a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:01.859 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.857+0000 7fc8b8e86700 1 --2- 172.21.15.99:0/2378081832 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8b4157ba0 0x7fc8b406a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:01.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.857+0000 7fc8b8e86700 1 -- 172.21.15.99:0/2378081832 >> v1:172.21.15.191:6789/0 conn(0x7fc8b415b0d0 legacy=0x7fc8b415da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:01.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.857+0000 7fc8b8e86700 1 -- 172.21.15.99:0/2378081832 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b415ba30 con 0x7fc8b41577d0 2023-10-08T00:12:01.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.857+0000 7fc8abfff700 1 --2- 172.21.15.99:0/2378081832 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8b4157ba0 0x7fc8b406a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:12:01.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.857+0000 7fc8b8e86700 1 --2- 172.21.15.99:0/2378081832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8b41577d0 0x7fc8b40cd770 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc8b00099b0 tx=0x7fc8b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e347184413fb8aec server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:01.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.857+0000 7fc8ab7fe700 1 -- 172.21.15.99:0/2378081832 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b000b820 con 0x7fc8b41577d0 2023-10-08T00:12:01.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.857+0000 7fc8ab7fe700 1 -- 172.21.15.99:0/2378081832 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8b000b980 con 0x7fc8b41577d0 2023-10-08T00:12:01.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.857+0000 7fc8ab7fe700 1 -- 172.21.15.99:0/2378081832 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b000bd20 con 0x7fc8b41577d0 2023-10-08T00:12:01.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.857+0000 7fc8bb0ea700 1 -- 172.21.15.99:0/2378081832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8b41577d0 msgr2=0x7fc8b40cd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:01.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.857+0000 7fc8bb0ea700 1 --2- 172.21.15.99:0/2378081832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8b41577d0 0x7fc8b40cd770 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc8b00099b0 tx=0x7fc8b0009250 comp rx=0 tx=0).stop 2023-10-08T00:12:01.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.858+0000 7fc8bb0ea700 1 -- 172.21.15.99:0/2378081832 shutdown_connections 2023-10-08T00:12:01.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.858+0000 7fc8bb0ea700 1 --2- 172.21.15.99:0/2378081832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8b41577d0 0x7fc8b40cd770 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:01.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.858+0000 7fc8bb0ea700 1 --2- 172.21.15.99:0/2378081832 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8b4157ba0 0x7fc8b406a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:01.861 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.858+0000 7fc8bb0ea700 1 -- 172.21.15.99:0/2378081832 >> 172.21.15.99:0/2378081832 conn(0x7fc8b4068ce0 msgr2=0x7fc8b40690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:01.861 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.858+0000 7fc8bb0ea700 1 -- 172.21.15.99:0/2378081832 shutdown_connections 2023-10-08T00:12:01.861 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.858+0000 7fc8bb0ea700 1 -- 172.21.15.99:0/2378081832 wait complete. 2023-10-08T00:12:01.863 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.859+0000 7fc8bb0ea700 1 Processor -- start 2023-10-08T00:12:01.863 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.859+0000 7fc8bb0ea700 1 -- start start 2023-10-08T00:12:01.863 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.860+0000 7fc8bb0ea700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8b41577d0 0x7fc8b414bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:01.863 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.860+0000 7fc8bb0ea700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8b4157ba0 0x7fc8b40b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:01.864 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.860+0000 7fc8bb0ea700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8b415b0d0 0x7fc8b4154af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:01.864 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.860+0000 7fc8bb0ea700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b414d7f0 con 0x7fc8b41577d0 2023-10-08T00:12:01.864 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.860+0000 7fc8bb0ea700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b414d4c0 con 0x7fc8b415b0d0 2023-10-08T00:12:01.864 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.860+0000 7fc8bb0ea700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b40b1010 con 0x7fc8b4157ba0 2023-10-08T00:12:01.864 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.860+0000 7fc8b8e86700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8b41577d0 0x7fc8b414bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:01.864 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.860+0000 7fc8b8e86700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8b41577d0 0x7fc8b414bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58372/0 (socket says 172.21.15.99:58372) 2023-10-08T00:12:01.864 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.860+0000 7fc8b8e86700 1 -- 172.21.15.99:0/2831374857 learned_addr learned my addr 172.21.15.99:0/2831374857 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:01.864 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.860+0000 7fc8b8e86700 1 -- 172.21.15.99:0/2831374857 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8b4157ba0 msgr2=0x7fc8b40b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:01.864 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.860+0000 7fc8b8e86700 1 --2- 172.21.15.99:0/2831374857 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8b4157ba0 0x7fc8b40b05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:01.864 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.860+0000 7fc8b8e86700 1 -- 172.21.15.99:0/2831374857 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8b415b0d0 msgr2=0x7fc8b4154af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:01.865 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.860+0000 7fc8b8e86700 1 --2- 172.21.15.99:0/2831374857 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8b415b0d0 0x7fc8b4154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:01.865 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.860+0000 7fc8b8e86700 1 -- 172.21.15.99:0/2831374857 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b00094f0 con 0x7fc8b41577d0 2023-10-08T00:12:01.865 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.860+0000 7fc8b8e86700 1 --2- 172.21.15.99:0/2831374857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8b41577d0 0x7fc8b414bbe0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0009290 tx=0x7fc8b0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:01.865 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.860+0000 7fc8a8ff9700 1 -- 172.21.15.99:0/2831374857 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b001b470 con 0x7fc8b41577d0 2023-10-08T00:12:01.865 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.860+0000 7fc8a8ff9700 1 -- 172.21.15.99:0/2831374857 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8b001b5d0 con 0x7fc8b41577d0 2023-10-08T00:12:01.865 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.860+0000 7fc8bb0ea700 1 -- 172.21.15.99:0/2831374857 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8b40b12f0 con 0x7fc8b41577d0 2023-10-08T00:12:01.865 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.860+0000 7fc8a8ff9700 1 -- 172.21.15.99:0/2831374857 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b001b990 con 0x7fc8b41577d0 2023-10-08T00:12:01.865 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.860+0000 7fc8bb0ea700 1 -- 172.21.15.99:0/2831374857 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8b40d0900 con 0x7fc8b41577d0 2023-10-08T00:12:01.865 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.861+0000 7fc8bb0ea700 1 -- 172.21.15.99:0/2831374857 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8b404c3c0 con 0x7fc8b41577d0 2023-10-08T00:12:01.865 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.862+0000 7fc8a8ff9700 1 -- 172.21.15.99:0/2831374857 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc8b001c450 con 0x7fc8b41577d0 2023-10-08T00:12:01.865 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.862+0000 7fc8a8ff9700 1 --2- 172.21.15.99:0/2831374857 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc8a0066f30 0x7fc8a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:01.865 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.862+0000 7fc8a8ff9700 1 -- 172.21.15.99:0/2831374857 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fc8b0085ee0 con 0x7fc8b41577d0 2023-10-08T00:12:01.866 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.862+0000 7fc8abfff700 1 --2- 172.21.15.99:0/2831374857 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc8a0066f30 0x7fc8a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:01.866 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.863+0000 7fc8abfff700 1 --2- 172.21.15.99:0/2831374857 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc8a0066f30 0x7fc8a00693e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc8b40d0ff0 tx=0x7fc8a4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:01.867 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:01.866+0000 7fc8a8ff9700 1 -- 172.21.15.99:0/2831374857 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc8b0053450 con 0x7fc8b41577d0 2023-10-08T00:12:02.129 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.128+0000 7fc8bb0ea700 1 -- 172.21.15.99:0/2831374857 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_barrier"]} v 0) v1 -- 0x7fc8b415bd10 con 0x7fc8b41577d0 2023-10-08T00:12:02.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.396+0000 7fc8a8ff9700 1 -- 172.21.15.99:0/2831374857 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_barrier"]}]=0 v82) v1 ==== 130+0+0 (secure 0 0 0) 0x7fc8b002a0c0 con 0x7fc8b41577d0 2023-10-08T00:12:02.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.398+0000 7fc8bb0ea700 1 -- 172.21.15.99:0/2831374857 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc8a0066f30 msgr2=0x7fc8a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:02.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.398+0000 7fc8bb0ea700 1 --2- 172.21.15.99:0/2831374857 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc8a0066f30 0x7fc8a00693e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc8b40d0ff0 tx=0x7fc8a4008040 comp rx=0 tx=0).stop 2023-10-08T00:12:02.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.398+0000 7fc8bb0ea700 1 -- 172.21.15.99:0/2831374857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8b41577d0 msgr2=0x7fc8b414bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:02.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.398+0000 7fc8bb0ea700 1 --2- 172.21.15.99:0/2831374857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8b41577d0 0x7fc8b414bbe0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0009290 tx=0x7fc8b0016000 comp rx=0 tx=0).stop 2023-10-08T00:12:02.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.398+0000 7fc8bb0ea700 1 -- 172.21.15.99:0/2831374857 shutdown_connections 2023-10-08T00:12:02.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.398+0000 7fc8bb0ea700 1 --2- 172.21.15.99:0/2831374857 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc8a0066f30 0x7fc8a00693e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:02.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.398+0000 7fc8bb0ea700 1 --2- 172.21.15.99:0/2831374857 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8b415b0d0 0x7fc8b4154af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:02.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.398+0000 7fc8bb0ea700 1 --2- 172.21.15.99:0/2831374857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8b41577d0 0x7fc8b414bbe0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:02.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.398+0000 7fc8bb0ea700 1 --2- 172.21.15.99:0/2831374857 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8b4157ba0 0x7fc8b40b05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:02.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.398+0000 7fc8bb0ea700 1 -- 172.21.15.99:0/2831374857 >> 172.21.15.99:0/2831374857 conn(0x7fc8b4068ce0 msgr2=0x7fc8b414c570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:02.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.399+0000 7fc8bb0ea700 1 -- 172.21.15.99:0/2831374857 shutdown_connections 2023-10-08T00:12:02.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.399+0000 7fc8bb0ea700 1 -- 172.21.15.99:0/2831374857 wait complete. 2023-10-08T00:12:02.411 INFO:tasks.cephfs_test_runner:That when an OSD epoch barrier is set on an MDS, subsequently ... ok 2023-10-08T00:12:02.411 INFO:tasks.cephfs_test_runner:Starting test: test_full_different_file (tasks.cephfs.test_full.TestClusterFull) 2023-10-08T00:12:02.412 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_different_file' 2023-10-08T00:12:02.526 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.525+0000 7f97957ec700 1 Processor -- start 2023-10-08T00:12:02.526 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.526+0000 7f97957ec700 1 -- start start 2023-10-08T00:12:02.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.526+0000 7f97957ec700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f979006ab90 0x7f97900cb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:02.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.526+0000 7f97957ec700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f97900cc710 0x7f979006af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:02.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.526+0000 7f97957ec700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97900cac20 con 0x7f979015abb0 2023-10-08T00:12:02.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.526+0000 7f97957ec700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97900640c0 con 0x7f979006ab90 2023-10-08T00:12:02.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.526+0000 7f97957ec700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97900cb4c0 con 0x7f97900cc710 2023-10-08T00:12:02.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.527+0000 7f978ffff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f979006ab90 0x7f97900cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:02.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.527+0000 7f978ffff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f979006ab90 0x7f97900cb950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58382/0 (socket says 172.21.15.99:58382) 2023-10-08T00:12:02.530 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.527+0000 7f978ffff700 1 -- 172.21.15.99:0/4060624554 learned_addr learned my addr 172.21.15.99:0/4060624554 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:02.530 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.527+0000 7f978f7fe700 1 --2- 172.21.15.99:0/4060624554 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f97900cc710 0x7f979006af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:02.530 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.527+0000 7f978ffff700 1 -- 172.21.15.99:0/4060624554 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f97900cc710 msgr2=0x7f979006af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:02.530 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.527+0000 7f978ffff700 1 --2- 172.21.15.99:0/4060624554 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f97900cc710 0x7f979006af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:02.530 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.527+0000 7f978ffff700 1 -- 172.21.15.99:0/4060624554 >> v1:172.21.15.191:6789/0 conn(0x7f979015abb0 legacy=0x7f97900c7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:02.530 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.527+0000 7f978ffff700 1 -- 172.21.15.99:0/4060624554 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f979006b4a0 con 0x7f979006ab90 2023-10-08T00:12:02.530 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.527+0000 7f978f7fe700 1 --2- 172.21.15.99:0/4060624554 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f97900cc710 0x7f979006af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:12:02.530 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.527+0000 7f978ffff700 1 --2- 172.21.15.99:0/4060624554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f979006ab90 0x7f97900cb950 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f97800099b0 tx=0x7f9780009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eb06a5d856e723 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:02.530 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.527+0000 7f978effd700 1 -- 172.21.15.99:0/4060624554 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978000b860 con 0x7f979006ab90 2023-10-08T00:12:02.530 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.527+0000 7f978effd700 1 -- 172.21.15.99:0/4060624554 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f978000b9c0 con 0x7f979006ab90 2023-10-08T00:12:02.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.527+0000 7f978effd700 1 -- 172.21.15.99:0/4060624554 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978000bd60 con 0x7f979006ab90 2023-10-08T00:12:02.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.528+0000 7f97957ec700 1 -- 172.21.15.99:0/4060624554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f979006ab90 msgr2=0x7f97900cb950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:02.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.528+0000 7f97957ec700 1 --2- 172.21.15.99:0/4060624554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f979006ab90 0x7f97900cb950 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f97800099b0 tx=0x7f9780009250 comp rx=0 tx=0).stop 2023-10-08T00:12:02.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.528+0000 7f97957ec700 1 -- 172.21.15.99:0/4060624554 shutdown_connections 2023-10-08T00:12:02.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.528+0000 7f97957ec700 1 --2- 172.21.15.99:0/4060624554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f979006ab90 0x7f97900cb950 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:02.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.528+0000 7f97957ec700 1 --2- 172.21.15.99:0/4060624554 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f97900cc710 0x7f979006af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:02.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.528+0000 7f97957ec700 1 -- 172.21.15.99:0/4060624554 >> 172.21.15.99:0/4060624554 conn(0x7f97900cd050 msgr2=0x7f97900cd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:02.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.528+0000 7f97957ec700 1 -- 172.21.15.99:0/4060624554 shutdown_connections 2023-10-08T00:12:02.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.528+0000 7f97957ec700 1 -- 172.21.15.99:0/4060624554 wait complete. 2023-10-08T00:12:02.534 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.529+0000 7f97957ec700 1 Processor -- start 2023-10-08T00:12:02.535 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.529+0000 7f97957ec700 1 -- start start 2023-10-08T00:12:02.535 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.530+0000 7f97957ec700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f979006ab90 0x7f979016b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:02.535 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.530+0000 7f97957ec700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97900cc710 0x7f979016ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:02.535 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.530+0000 7f97957ec700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f979015abb0 0x7f9790159420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:02.535 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.530+0000 7f97957ec700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97900c89a0 con 0x7f97900cc710 2023-10-08T00:12:02.535 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.530+0000 7f97957ec700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97900c6a40 con 0x7f979015abb0 2023-10-08T00:12:02.535 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.530+0000 7f97957ec700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97900c6710 con 0x7f979006ab90 2023-10-08T00:12:02.535 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.530+0000 7f978f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97900cc710 0x7f979016ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:02.535 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.530+0000 7f978f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97900cc710 0x7f979016ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58394/0 (socket says 172.21.15.99:58394) 2023-10-08T00:12:02.536 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.530+0000 7f978f7fe700 1 -- 172.21.15.99:0/2304746238 learned_addr learned my addr 172.21.15.99:0/2304746238 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:02.536 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.530+0000 7f978ffff700 1 --2- 172.21.15.99:0/2304746238 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f979006ab90 0x7f979016b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:02.536 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.530+0000 7f9794feb700 1 --2- 172.21.15.99:0/2304746238 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f979015abb0 0x7f9790159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:02.536 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.530+0000 7f978f7fe700 1 -- 172.21.15.99:0/2304746238 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f979006ab90 msgr2=0x7f979016b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:02.536 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.530+0000 7f978f7fe700 1 --2- 172.21.15.99:0/2304746238 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f979006ab90 0x7f979016b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:02.536 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.530+0000 7f978f7fe700 1 -- 172.21.15.99:0/2304746238 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f979015abb0 msgr2=0x7f9790159420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:02.536 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.530+0000 7f978f7fe700 1 --2- 172.21.15.99:0/2304746238 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f979015abb0 0x7f9790159420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:02.536 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.530+0000 7f978f7fe700 1 -- 172.21.15.99:0/2304746238 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97800094f0 con 0x7f97900cc710 2023-10-08T00:12:02.536 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.530+0000 7f978f7fe700 1 --2- 172.21.15.99:0/2304746238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97900cc710 0x7f979016ade0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f978400e8a0 tx=0x7f9784008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:02.536 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.530+0000 7f9777fff700 1 -- 172.21.15.99:0/2304746238 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9784013410 con 0x7f97900cc710 2023-10-08T00:12:02.537 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.531+0000 7f9777fff700 1 -- 172.21.15.99:0/2304746238 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9784013570 con 0x7f97900cc710 2023-10-08T00:12:02.537 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.531+0000 7f97957ec700 1 -- 172.21.15.99:0/2304746238 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f979016b320 con 0x7f97900cc710 2023-10-08T00:12:02.537 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.531+0000 7f9777fff700 1 -- 172.21.15.99:0/2304746238 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9784013930 con 0x7f97900cc710 2023-10-08T00:12:02.537 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.531+0000 7f97957ec700 1 -- 172.21.15.99:0/2304746238 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97900c4cb0 con 0x7f97900cc710 2023-10-08T00:12:02.537 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.532+0000 7f97957ec700 1 -- 172.21.15.99:0/2304746238 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97900c6bb0 con 0x7f97900cc710 2023-10-08T00:12:02.537 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.532+0000 7f9777fff700 1 -- 172.21.15.99:0/2304746238 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9784013ca0 con 0x7f97900cc710 2023-10-08T00:12:02.537 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.532+0000 7f9777fff700 1 --2- 172.21.15.99:0/2304746238 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f977c066f30 0x7f977c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:02.537 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.532+0000 7f9777fff700 1 -- 172.21.15.99:0/2304746238 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f9784083ee0 con 0x7f97900cc710 2023-10-08T00:12:02.537 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.533+0000 7f978ffff700 1 --2- 172.21.15.99:0/2304746238 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f977c066f30 0x7f977c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:02.537 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.533+0000 7f978ffff700 1 --2- 172.21.15.99:0/2304746238 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f977c066f30 0x7f977c0693e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f97800099b0 tx=0x7f9780017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:02.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.536+0000 7f9777fff700 1 -- 172.21.15.99:0/2304746238 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f97900c6bb0 con 0x7f97900cc710 2023-10-08T00:12:02.797 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:02.796+0000 7f97957ec700 1 -- 172.21.15.99:0/2304746238 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]} v 0) v1 -- 0x7f97900c6bb0 con 0x7f97900cc710 2023-10-08T00:12:03.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.398+0000 7f9777fff700 1 -- 172.21.15.99:0/2304746238 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]}]=0 v83) v1 ==== 145+0+0 (secure 0 0 0) 0x7f97900c6bb0 con 0x7f97900cc710 2023-10-08T00:12:03.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.400+0000 7f97957ec700 1 -- 172.21.15.99:0/2304746238 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f977c066f30 msgr2=0x7f977c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:03.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.400+0000 7f97957ec700 1 --2- 172.21.15.99:0/2304746238 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f977c066f30 0x7f977c0693e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f97800099b0 tx=0x7f9780017000 comp rx=0 tx=0).stop 2023-10-08T00:12:03.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.400+0000 7f97957ec700 1 -- 172.21.15.99:0/2304746238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97900cc710 msgr2=0x7f979016ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:03.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.400+0000 7f97957ec700 1 --2- 172.21.15.99:0/2304746238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97900cc710 0x7f979016ade0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f978400e8a0 tx=0x7f9784008040 comp rx=0 tx=0).stop 2023-10-08T00:12:03.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.401+0000 7f97957ec700 1 -- 172.21.15.99:0/2304746238 shutdown_connections 2023-10-08T00:12:03.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.401+0000 7f97957ec700 1 --2- 172.21.15.99:0/2304746238 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f977c066f30 0x7f977c0693e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:03.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.401+0000 7f97957ec700 1 --2- 172.21.15.99:0/2304746238 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f979015abb0 0x7f9790159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:03.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.401+0000 7f97957ec700 1 --2- 172.21.15.99:0/2304746238 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f979006ab90 0x7f979016b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:03.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.401+0000 7f97957ec700 1 --2- 172.21.15.99:0/2304746238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97900cc710 0x7f979016ade0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:03.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.401+0000 7f97957ec700 1 -- 172.21.15.99:0/2304746238 >> 172.21.15.99:0/2304746238 conn(0x7f97900cd050 msgr2=0x7f97900b2210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:03.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.401+0000 7f97957ec700 1 -- 172.21.15.99:0/2304746238 shutdown_connections 2023-10-08T00:12:03.406 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.401+0000 7f97957ec700 1 -- 172.21.15.99:0/2304746238 wait complete. 2023-10-08T00:12:03.414 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2023-10-08T00:12:03.546 INFO:teuthology.orchestra.run.smithi099.stdout:{"osd_objectstore":"memstore"} 2023-10-08T00:12:03.546 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2023-10-08T00:12:03.546 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T00:12:03.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.663+0000 7fd0730fc700 1 Processor -- start 2023-10-08T00:12:03.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.663+0000 7fd0730fc700 1 -- start start 2023-10-08T00:12:03.667 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.664+0000 7fd0730fc700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd06c159ec0 0x7fd06c0af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:03.667 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.664+0000 7fd0730fc700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd06c15a290 0x7fd06c0afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:03.667 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.664+0000 7fd0730fc700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd06c0b0790 con 0x7fd06c06a5a0 2023-10-08T00:12:03.667 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.664+0000 7fd0730fc700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd06c0ce1f0 con 0x7fd06c159ec0 2023-10-08T00:12:03.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.664+0000 7fd0730fc700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd06c0b0b20 con 0x7fd06c15a290 2023-10-08T00:12:03.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.664+0000 7fd070e98700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd06c159ec0 0x7fd06c0af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:03.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.664+0000 7fd071699700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fd06c06a5a0 0x7fd06c15db90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56502/0 (socket says 172.21.15.99:56502) 2023-10-08T00:12:03.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.664+0000 7fd070e98700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd06c159ec0 0x7fd06c0af940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41042/0 (socket says 172.21.15.99:41042) 2023-10-08T00:12:03.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.664+0000 7fd071699700 1 -- 172.21.15.99:0/3717264367 learned_addr learned my addr 172.21.15.99:0/3717264367 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:12:03.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.664+0000 7fd06bfff700 1 --2- 172.21.15.99:0/3717264367 >> v2:172.21.15.191:3300/0 conn(0x7fd06c15a290 0x7fd06c0afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:03.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.665+0000 7fd070e98700 1 -- 172.21.15.99:0/3717264367 >> v2:172.21.15.191:3300/0 conn(0x7fd06c15a290 msgr2=0x7fd06c0afe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:03.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.665+0000 7fd070e98700 1 --2- 172.21.15.99:0/3717264367 >> v2:172.21.15.191:3300/0 conn(0x7fd06c15a290 0x7fd06c0afe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:03.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.665+0000 7fd070e98700 1 -- 172.21.15.99:0/3717264367 >> v1:172.21.15.99:6789/0 conn(0x7fd06c06a5a0 legacy=0x7fd06c15db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:03.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.665+0000 7fd070e98700 1 -- 172.21.15.99:0/3717264367 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd06c0b04b0 con 0x7fd06c159ec0 2023-10-08T00:12:03.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.665+0000 7fd06b7fe700 1 -- 172.21.15.99:0/3717264367 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2283439861 0 0) 0x7fd06c0b0790 con 0x7fd06c06a5a0 2023-10-08T00:12:03.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.665+0000 7fd070e98700 1 --2- 172.21.15.99:0/3717264367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd06c159ec0 0x7fd06c0af940 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd06c089910 tx=0x7fd0600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32e4922eace0c471 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:03.669 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.665+0000 7fd06b7fe700 1 -- 172.21.15.99:0/3717264367 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06000b6b0 con 0x7fd06c159ec0 2023-10-08T00:12:03.669 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.665+0000 7fd06b7fe700 1 -- 172.21.15.99:0/3717264367 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd06000b810 con 0x7fd06c159ec0 2023-10-08T00:12:03.669 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.665+0000 7fd06b7fe700 1 -- 172.21.15.99:0/3717264367 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06000bbb0 con 0x7fd06c159ec0 2023-10-08T00:12:03.669 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.665+0000 7fd0730fc700 1 -- 172.21.15.99:0/3717264367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd06c159ec0 msgr2=0x7fd06c0af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:03.669 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.665+0000 7fd0730fc700 1 --2- 172.21.15.99:0/3717264367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd06c159ec0 0x7fd06c0af940 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd06c089910 tx=0x7fd0600091a0 comp rx=0 tx=0).stop 2023-10-08T00:12:03.669 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.665+0000 7fd0730fc700 1 -- 172.21.15.99:0/3717264367 shutdown_connections 2023-10-08T00:12:03.669 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.665+0000 7fd0730fc700 1 --2- 172.21.15.99:0/3717264367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd06c159ec0 0x7fd06c0af940 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:03.669 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.665+0000 7fd0730fc700 1 --2- 172.21.15.99:0/3717264367 >> v2:172.21.15.191:3300/0 conn(0x7fd06c15a290 0x7fd06c0afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:03.669 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.665+0000 7fd0730fc700 1 -- 172.21.15.99:0/3717264367 >> 172.21.15.99:0/3717264367 conn(0x7fd06c0d1880 msgr2=0x7fd06c0cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:03.669 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.665+0000 7fd0730fc700 1 -- 172.21.15.99:0/3717264367 shutdown_connections 2023-10-08T00:12:03.669 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.665+0000 7fd0730fc700 1 -- 172.21.15.99:0/3717264367 wait complete. 2023-10-08T00:12:03.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.666+0000 7fd0730fc700 1 Processor -- start 2023-10-08T00:12:03.673 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.667+0000 7fd0730fc700 1 -- start start 2023-10-08T00:12:03.673 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.667+0000 7fd0730fc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd06c06a5a0 0x7fd06c0cb940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:03.673 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.667+0000 7fd0730fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd06c159ec0 0x7fd06c1586b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:03.673 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.667+0000 7fd0730fc700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd06c15a290 0x7fd06c156220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:03.673 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.667+0000 7fd0730fc700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06c158ec0 con 0x7fd06c159ec0 2023-10-08T00:12:03.673 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.667+0000 7fd0730fc700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06c0c88c0 con 0x7fd06c15a290 2023-10-08T00:12:03.673 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.667+0000 7fd0730fc700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd06c158bf0 con 0x7fd06c06a5a0 2023-10-08T00:12:03.673 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.667+0000 7fd06bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd06c159ec0 0x7fd06c1586b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:03.673 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.667+0000 7fd06bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd06c159ec0 0x7fd06c1586b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41044/0 (socket says 172.21.15.99:41044) 2023-10-08T00:12:03.673 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.667+0000 7fd06bfff700 1 -- 172.21.15.99:0/1334588257 learned_addr learned my addr 172.21.15.99:0/1334588257 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:03.674 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.668+0000 7fd070e98700 1 --2- 172.21.15.99:0/1334588257 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd06c06a5a0 0x7fd06c0cb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:03.674 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.668+0000 7fd071699700 1 --2- 172.21.15.99:0/1334588257 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd06c15a290 0x7fd06c156220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:03.674 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.668+0000 7fd06bfff700 1 -- 172.21.15.99:0/1334588257 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd06c06a5a0 msgr2=0x7fd06c0cb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:03.674 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.668+0000 7fd06bfff700 1 --2- 172.21.15.99:0/1334588257 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd06c06a5a0 0x7fd06c0cb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:03.674 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.668+0000 7fd06bfff700 1 -- 172.21.15.99:0/1334588257 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd06c15a290 msgr2=0x7fd06c156220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:03.674 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.668+0000 7fd06bfff700 1 --2- 172.21.15.99:0/1334588257 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd06c15a290 0x7fd06c156220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:03.674 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.668+0000 7fd06bfff700 1 -- 172.21.15.99:0/1334588257 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd060009420 con 0x7fd06c159ec0 2023-10-08T00:12:03.674 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.668+0000 7fd071699700 1 --2- 172.21.15.99:0/1334588257 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd06c15a290 0x7fd06c156220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:12:03.674 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.668+0000 7fd070e98700 1 --2- 172.21.15.99:0/1334588257 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd06c06a5a0 0x7fd06c0cb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:12:03.674 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.668+0000 7fd06bfff700 1 --2- 172.21.15.99:0/1334588257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd06c159ec0 0x7fd06c1586b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd05800c880 tx=0x7fd058008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:03.674 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.668+0000 7fd068ff9700 1 -- 172.21.15.99:0/1334588257 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd058011600 con 0x7fd06c159ec0 2023-10-08T00:12:03.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.668+0000 7fd068ff9700 1 -- 172.21.15.99:0/1334588257 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd058011780 con 0x7fd06c159ec0 2023-10-08T00:12:03.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.668+0000 7fd0730fc700 1 -- 172.21.15.99:0/1334588257 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd06c0c7f90 con 0x7fd06c159ec0 2023-10-08T00:12:03.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.668+0000 7fd068ff9700 1 -- 172.21.15.99:0/1334588257 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd058011b40 con 0x7fd06c159ec0 2023-10-08T00:12:03.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.668+0000 7fd0730fc700 1 -- 172.21.15.99:0/1334588257 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd06c0c8550 con 0x7fd06c159ec0 2023-10-08T00:12:03.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.669+0000 7fd0730fc700 1 -- 172.21.15.99:0/1334588257 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd06c0c8a70 con 0x7fd06c159ec0 2023-10-08T00:12:03.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.669+0000 7fd068ff9700 1 -- 172.21.15.99:0/1334588257 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd058018360 con 0x7fd06c159ec0 2023-10-08T00:12:03.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.670+0000 7fd068ff9700 1 --2- 172.21.15.99:0/1334588257 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd050066f30 0x7fd0500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:03.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.670+0000 7fd068ff9700 1 -- 172.21.15.99:0/1334588257 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fd058083fb0 con 0x7fd06c159ec0 2023-10-08T00:12:03.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.670+0000 7fd070e98700 1 --2- 172.21.15.99:0/1334588257 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd050066f30 0x7fd0500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:03.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.670+0000 7fd070e98700 1 --2- 172.21.15.99:0/1334588257 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd050066f30 0x7fd0500693e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd0600093f0 tx=0x7fd060017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:03.676 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.674+0000 7fd068ff9700 1 -- 172.21.15.99:0/1334588257 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd06c0c8a70 con 0x7fd06c159ec0 2023-10-08T00:12:03.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.855+0000 7fd0730fc700 1 -- 172.21.15.99:0/1334588257 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd06c0c8a30 con 0x7fd06c159ec0 2023-10-08T00:12:03.859 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.858+0000 7fd068ff9700 1 -- 172.21.15.99:0/1334588257 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v4)=0 v4) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd0580150c0 con 0x7fd06c159ec0 2023-10-08T00:12:03.862 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.861+0000 7fd0730fc700 1 -- 172.21.15.99:0/1334588257 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd050066f30 msgr2=0x7fd0500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:03.862 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.861+0000 7fd0730fc700 1 --2- 172.21.15.99:0/1334588257 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd050066f30 0x7fd0500693e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd0600093f0 tx=0x7fd060017000 comp rx=0 tx=0).stop 2023-10-08T00:12:03.862 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.861+0000 7fd0730fc700 1 -- 172.21.15.99:0/1334588257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd06c159ec0 msgr2=0x7fd06c1586b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:03.862 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.861+0000 7fd0730fc700 1 --2- 172.21.15.99:0/1334588257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd06c159ec0 0x7fd06c1586b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd05800c880 tx=0x7fd058008040 comp rx=0 tx=0).stop 2023-10-08T00:12:03.862 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.861+0000 7fd0730fc700 1 -- 172.21.15.99:0/1334588257 shutdown_connections 2023-10-08T00:12:03.862 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.861+0000 7fd0730fc700 1 --2- 172.21.15.99:0/1334588257 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd050066f30 0x7fd0500693e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:03.862 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.861+0000 7fd0730fc700 1 --2- 172.21.15.99:0/1334588257 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd06c15a290 0x7fd06c156220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:03.862 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.861+0000 7fd0730fc700 1 --2- 172.21.15.99:0/1334588257 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd06c06a5a0 0x7fd06c0cb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:03.862 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.861+0000 7fd0730fc700 1 --2- 172.21.15.99:0/1334588257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd06c159ec0 0x7fd06c1586b0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:03.862 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.861+0000 7fd0730fc700 1 -- 172.21.15.99:0/1334588257 >> 172.21.15.99:0/1334588257 conn(0x7fd06c0d1880 msgr2=0x7fd06c0b1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:03.862 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.861+0000 7fd0730fc700 1 -- 172.21.15.99:0/1334588257 shutdown_connections 2023-10-08T00:12:03.863 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:03.861+0000 7fd0730fc700 1 -- 172.21.15.99:0/1334588257 wait complete. 2023-10-08T00:12:03.873 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T00:12:03.873 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T00:12:03.911 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T00:12:03.951 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T00:12:03.951 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-08T00:12:03.951 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:12:04.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.064+0000 7fbfce60c700 1 Processor -- start 2023-10-08T00:12:04.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.064+0000 7fbfce60c700 1 -- start start 2023-10-08T00:12:04.068 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.065+0000 7fbfce60c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbfc8157870 0x7fbfc815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:04.068 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.065+0000 7fbfce60c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fbfc815a620 0x7fbfc815d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:04.068 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.065+0000 7fbfce60c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfc815d580 0x7fbfc8158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:04.068 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.065+0000 7fbfce60c700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfc815cbb0 con 0x7fbfc8157870 2023-10-08T00:12:04.068 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.065+0000 7fbfce60c700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfc815bfa0 con 0x7fbfc815a620 2023-10-08T00:12:04.068 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.065+0000 7fbfce60c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc815c280 con 0x7fbfc815d580 2023-10-08T00:12:04.068 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.065+0000 7fbfc7fff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbfc8157870 0x7fbfc815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:04.068 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.065+0000 7fbfc7fff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbfc8157870 0x7fbfc815b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41050/0 (socket says 172.21.15.99:41050) 2023-10-08T00:12:04.068 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.065+0000 7fbfc7fff700 1 -- 172.21.15.99:0/1233363381 learned_addr learned my addr 172.21.15.99:0/1233363381 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:04.068 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.066+0000 7fbfc77fe700 1 --2- 172.21.15.99:0/1233363381 >> v2:172.21.15.191:3300/0 conn(0x7fbfc815a620 0x7fbfc815d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:04.068 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.066+0000 7fbfccba9700 1 --2- 172.21.15.99:0/1233363381 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfc815d580 0x7fbfc8158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:04.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.066+0000 7fbfc7fff700 1 -- 172.21.15.99:0/1233363381 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfc815d580 msgr2=0x7fbfc8158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:04.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.066+0000 7fbfc7fff700 1 --2- 172.21.15.99:0/1233363381 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfc815d580 0x7fbfc8158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:04.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.066+0000 7fbfc7fff700 1 -- 172.21.15.99:0/1233363381 >> v2:172.21.15.191:3300/0 conn(0x7fbfc815a620 msgr2=0x7fbfc815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:04.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.066+0000 7fbfc7fff700 1 --2- 172.21.15.99:0/1233363381 >> v2:172.21.15.191:3300/0 conn(0x7fbfc815a620 0x7fbfc815d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:04.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.066+0000 7fbfc7fff700 1 -- 172.21.15.99:0/1233363381 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc815c560 con 0x7fbfc8157870 2023-10-08T00:12:04.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.066+0000 7fbfc7fff700 1 --2- 172.21.15.99:0/1233363381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfc8157870 0x7fbfc815b6a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fbfc815cb20 tx=0x7fbfb8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b8fbb95048b5ff server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:04.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.066+0000 7fbfc6ffd700 1 -- 172.21.15.99:0/1233363381 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb800b860 con 0x7fbfc8157870 2023-10-08T00:12:04.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.066+0000 7fbfc6ffd700 1 -- 172.21.15.99:0/1233363381 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb800b9c0 con 0x7fbfc8157870 2023-10-08T00:12:04.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.066+0000 7fbfc6ffd700 1 -- 172.21.15.99:0/1233363381 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb800bd60 con 0x7fbfc8157870 2023-10-08T00:12:04.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.066+0000 7fbfce60c700 1 -- 172.21.15.99:0/1233363381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfc8157870 msgr2=0x7fbfc815b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:04.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.066+0000 7fbfce60c700 1 --2- 172.21.15.99:0/1233363381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfc8157870 0x7fbfc815b6a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fbfc815cb20 tx=0x7fbfb8009250 comp rx=0 tx=0).stop 2023-10-08T00:12:04.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.066+0000 7fbfce60c700 1 -- 172.21.15.99:0/1233363381 shutdown_connections 2023-10-08T00:12:04.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.066+0000 7fbfce60c700 1 --2- 172.21.15.99:0/1233363381 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfc815d580 0x7fbfc8158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:04.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.066+0000 7fbfce60c700 1 --2- 172.21.15.99:0/1233363381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfc8157870 0x7fbfc815b6a0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:04.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.066+0000 7fbfce60c700 1 --2- 172.21.15.99:0/1233363381 >> v2:172.21.15.191:3300/0 conn(0x7fbfc815a620 0x7fbfc815d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:04.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.066+0000 7fbfce60c700 1 -- 172.21.15.99:0/1233363381 >> 172.21.15.99:0/1233363381 conn(0x7fbfc8068ce0 msgr2=0x7fbfc80690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:04.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.066+0000 7fbfce60c700 1 -- 172.21.15.99:0/1233363381 shutdown_connections 2023-10-08T00:12:04.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.066+0000 7fbfce60c700 1 -- 172.21.15.99:0/1233363381 wait complete. 2023-10-08T00:12:04.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.068+0000 7fbfce60c700 1 Processor -- start 2023-10-08T00:12:04.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.068+0000 7fbfce60c700 1 -- start start 2023-10-08T00:12:04.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.068+0000 7fbfce60c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfc8157870 0x7fbfc814a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:04.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.068+0000 7fbfce60c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbfc815a620 0x7fbfc81553d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:04.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.068+0000 7fbfce60c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfc815d580 0x7fbfc8155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:04.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.068+0000 7fbfce60c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc8154730 con 0x7fbfc815d580 2023-10-08T00:12:04.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.068+0000 7fbfce60c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc816efe0 con 0x7fbfc815a620 2023-10-08T00:12:04.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.068+0000 7fbfce60c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc8153e00 con 0x7fbfc8157870 2023-10-08T00:12:04.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.069+0000 7fbfccba9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfc815d580 0x7fbfc8155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:04.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.069+0000 7fbfccba9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfc815d580 0x7fbfc8155910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41058/0 (socket says 172.21.15.99:41058) 2023-10-08T00:12:04.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.069+0000 7fbfccba9700 1 -- 172.21.15.99:0/704390319 learned_addr learned my addr 172.21.15.99:0/704390319 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:04.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.069+0000 7fbfc7fff700 1 --2- 172.21.15.99:0/704390319 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfc8157870 0x7fbfc814a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:04.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.069+0000 7fbfc77fe700 1 --2- 172.21.15.99:0/704390319 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbfc815a620 0x7fbfc81553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:04.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.069+0000 7fbfccba9700 1 -- 172.21.15.99:0/704390319 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfc8157870 msgr2=0x7fbfc814a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:04.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.069+0000 7fbfccba9700 1 --2- 172.21.15.99:0/704390319 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfc8157870 0x7fbfc814a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:04.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.069+0000 7fbfccba9700 1 -- 172.21.15.99:0/704390319 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbfc815a620 msgr2=0x7fbfc81553d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:04.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.069+0000 7fbfccba9700 1 --2- 172.21.15.99:0/704390319 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbfc815a620 0x7fbfc81553d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:04.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.069+0000 7fbfccba9700 1 -- 172.21.15.99:0/704390319 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb80094f0 con 0x7fbfc815d580 2023-10-08T00:12:04.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.069+0000 7fbfccba9700 1 --2- 172.21.15.99:0/704390319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfc815d580 0x7fbfc8155910 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fbfc000cdb0 tx=0x7fbfc00077a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:04.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.069+0000 7fbfaffff700 1 -- 172.21.15.99:0/704390319 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc000aa10 con 0x7fbfc815d580 2023-10-08T00:12:04.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.069+0000 7fbfaffff700 1 -- 172.21.15.99:0/704390319 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfc000ab70 con 0x7fbfc815d580 2023-10-08T00:12:04.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.069+0000 7fbfce60c700 1 -- 172.21.15.99:0/704390319 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfc81540e0 con 0x7fbfc815d580 2023-10-08T00:12:04.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.069+0000 7fbfaffff700 1 -- 172.21.15.99:0/704390319 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0017430 con 0x7fbfc815d580 2023-10-08T00:12:04.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.069+0000 7fbfce60c700 1 -- 172.21.15.99:0/704390319 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfc814a9b0 con 0x7fbfc815d580 2023-10-08T00:12:04.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.070+0000 7fbfce60c700 1 -- 172.21.15.99:0/704390319 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfc804c3c0 con 0x7fbfc815d580 2023-10-08T00:12:04.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.071+0000 7fbfaffff700 1 -- 172.21.15.99:0/704390319 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbfc0003020 con 0x7fbfc815d580 2023-10-08T00:12:04.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.071+0000 7fbfaffff700 1 --2- 172.21.15.99:0/704390319 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbfb4066f30 0x7fbfb40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:04.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.071+0000 7fbfaffff700 1 -- 172.21.15.99:0/704390319 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fbfc0013070 con 0x7fbfc815d580 2023-10-08T00:12:04.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.071+0000 7fbfc7fff700 1 --2- 172.21.15.99:0/704390319 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbfb4066f30 0x7fbfb40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:04.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.072+0000 7fbfc7fff700 1 --2- 172.21.15.99:0/704390319 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbfb4066f30 0x7fbfb40693e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbfc815cb20 tx=0x7fbfb8017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:04.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.075+0000 7fbfaffff700 1 -- 172.21.15.99:0/704390319 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbfc0051a80 con 0x7fbfc815d580 2023-10-08T00:12:04.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.297+0000 7fbfce60c700 1 -- 172.21.15.99:0/704390319 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbfc8154870 con 0x7fbfc815d580 2023-10-08T00:12:04.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.298+0000 7fbfaffff700 1 -- 172.21.15.99:0/704390319 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+3529 (secure 0 0 0) 0x7fbfc001c020 con 0x7fbfc815d580 2023-10-08T00:12:04.299 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:12:04.299 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4852,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/824084995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":824084995},{"type":"v1","addr":"172.21.15.191:6837","nonce":824084995}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4855,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/3349126920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3349126920},{"type":"v1","addr":"172.21.15.191:6836","nonce":3349126920}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4899,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/1004180438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1004180438},{"type":"v1","addr":"172.21.15.99:6834","nonce":1004180438}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5214,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/112856822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":112856822},{"type":"v1","addr":"172.21.15.99:6835","nonce":112856822}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[]} 2023-10-08T00:12:04.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.300+0000 7fbfce60c700 1 -- 172.21.15.99:0/704390319 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbfb4066f30 msgr2=0x7fbfb40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:04.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.300+0000 7fbfce60c700 1 --2- 172.21.15.99:0/704390319 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbfb4066f30 0x7fbfb40693e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbfc815cb20 tx=0x7fbfb8017000 comp rx=0 tx=0).stop 2023-10-08T00:12:04.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.300+0000 7fbfce60c700 1 -- 172.21.15.99:0/704390319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfc815d580 msgr2=0x7fbfc8155910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:04.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.300+0000 7fbfce60c700 1 --2- 172.21.15.99:0/704390319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfc815d580 0x7fbfc8155910 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fbfc000cdb0 tx=0x7fbfc00077a0 comp rx=0 tx=0).stop 2023-10-08T00:12:04.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.300+0000 7fbfce60c700 1 -- 172.21.15.99:0/704390319 shutdown_connections 2023-10-08T00:12:04.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.300+0000 7fbfce60c700 1 --2- 172.21.15.99:0/704390319 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbfb4066f30 0x7fbfb40693e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:04.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.300+0000 7fbfce60c700 1 --2- 172.21.15.99:0/704390319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfc815d580 0x7fbfc8155910 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:04.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.300+0000 7fbfce60c700 1 --2- 172.21.15.99:0/704390319 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfc8157870 0x7fbfc814a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:04.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.300+0000 7fbfce60c700 1 --2- 172.21.15.99:0/704390319 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbfc815a620 0x7fbfc81553d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:04.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.300+0000 7fbfce60c700 1 -- 172.21.15.99:0/704390319 >> 172.21.15.99:0/704390319 conn(0x7fbfc8068ce0 msgr2=0x7fbfc80afb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:04.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.300+0000 7fbfce60c700 1 -- 172.21.15.99:0/704390319 shutdown_connections 2023-10-08T00:12:04.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.300+0000 7fbfce60c700 1 -- 172.21.15.99:0/704390319 wait complete. 2023-10-08T00:12:04.303 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 19 2023-10-08T00:12:04.313 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T00:12:04.313 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-08T00:12:04.314 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T00:12:04.314 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:12:04.314 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T00:12:04.314 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-08T00:12:04.314 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T00:12:04.314 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:12:04.314 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T00:12:04.315 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-08T00:12:04.315 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T00:12:04.315 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:12:04.315 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T00:12:04.315 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-08T00:12:04.315 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T00:12:04.315 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:12:04.322 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T00:12:04.322 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T00:12:04.323 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T00:12:04.323 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T00:12:04.325 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T00:12:04.325 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T00:12:04.326 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T00:12:04.326 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T00:12:04.365 INFO:tasks.ceph.mds.a:Started 2023-10-08T00:12:04.365 DEBUG:teuthology.parallel:result is None 2023-10-08T00:12:04.366 INFO:tasks.ceph.mds.c:Started 2023-10-08T00:12:04.366 DEBUG:teuthology.parallel:result is None 2023-10-08T00:12:04.367 INFO:tasks.ceph.mds.b:Started 2023-10-08T00:12:04.367 DEBUG:teuthology.parallel:result is None 2023-10-08T00:12:04.368 INFO:tasks.ceph.mds.d:Started 2023-10-08T00:12:04.368 DEBUG:teuthology.parallel:result is None 2023-10-08T00:12:04.368 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-08T00:12:04.437 INFO:tasks.ceph.mds.c.smithi099.stdout:starting mds.c at 2023-10-08T00:12:04.438 INFO:tasks.ceph.mds.a.smithi099.stdout:starting mds.a at 2023-10-08T00:12:04.449 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2023-10-08T00:12:04.449 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2023-10-08T00:12:04.489 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.488+0000 7fd01f055700 1 Processor -- start 2023-10-08T00:12:04.489 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.488+0000 7fd01f055700 1 -- start start 2023-10-08T00:12:04.491 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.489+0000 7fd01f055700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd018157870 0x7fd01815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:04.491 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.489+0000 7fd01f055700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd01815a620 0x7fd01815d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:04.491 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.489+0000 7fd01f055700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd01815d580 0x7fd018158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:04.492 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.489+0000 7fd01f055700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd01815cbb0 con 0x7fd018157870 2023-10-08T00:12:04.492 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.489+0000 7fd01f055700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd01815bfa0 con 0x7fd01815d580 2023-10-08T00:12:04.492 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.489+0000 7fd01f055700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd01815c280 con 0x7fd01815a620 2023-10-08T00:12:04.492 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.489+0000 7fd01cdf1700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd018157870 0x7fd01815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:04.492 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.489+0000 7fd01cdf1700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd018157870 0x7fd01815b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41112/0 (socket says 172.21.15.99:41112) 2023-10-08T00:12:04.492 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.489+0000 7fd01cdf1700 1 -- 172.21.15.99:0/1589059202 learned_addr learned my addr 172.21.15.99:0/1589059202 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:04.492 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.490+0000 7fd017fff700 1 --2- 172.21.15.99:0/1589059202 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd01815a620 0x7fd01815d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:04.492 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.490+0000 7fd01d5f2700 1 --2- 172.21.15.99:0/1589059202 >> v2:172.21.15.191:3300/0 conn(0x7fd01815d580 0x7fd018158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:04.492 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.490+0000 7fd01cdf1700 1 -- 172.21.15.99:0/1589059202 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd01815a620 msgr2=0x7fd01815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:04.492 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.490+0000 7fd01cdf1700 1 --2- 172.21.15.99:0/1589059202 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd01815a620 0x7fd01815d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:04.492 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.490+0000 7fd01cdf1700 1 -- 172.21.15.99:0/1589059202 >> v2:172.21.15.191:3300/0 conn(0x7fd01815d580 msgr2=0x7fd018158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:04.492 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.490+0000 7fd01cdf1700 1 --2- 172.21.15.99:0/1589059202 >> v2:172.21.15.191:3300/0 conn(0x7fd01815d580 0x7fd018158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:04.493 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.490+0000 7fd01cdf1700 1 -- 172.21.15.99:0/1589059202 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd01815c560 con 0x7fd018157870 2023-10-08T00:12:04.493 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.490+0000 7fd017fff700 1 --2- 172.21.15.99:0/1589059202 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd01815a620 0x7fd01815d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:12:04.493 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.490+0000 7fd01cdf1700 1 --2- 172.21.15.99:0/1589059202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd018157870 0x7fd01815b6a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd0180b03b0 tx=0x7fd010009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0f622aa3ae6598f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:04.493 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.490+0000 7fd0177fe700 1 -- 172.21.15.99:0/1589059202 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01000e070 con 0x7fd018157870 2023-10-08T00:12:04.493 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.490+0000 7fd0177fe700 1 -- 172.21.15.99:0/1589059202 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd01000ba50 con 0x7fd018157870 2023-10-08T00:12:04.493 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.490+0000 7fd0177fe700 1 -- 172.21.15.99:0/1589059202 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0100092e0 con 0x7fd018157870 2023-10-08T00:12:04.493 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.490+0000 7fd01f055700 1 -- 172.21.15.99:0/1589059202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd018157870 msgr2=0x7fd01815b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:04.493 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.491+0000 7fd01f055700 1 --2- 172.21.15.99:0/1589059202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd018157870 0x7fd01815b6a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd0180b03b0 tx=0x7fd010009840 comp rx=0 tx=0).stop 2023-10-08T00:12:04.493 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.491+0000 7fd01f055700 1 -- 172.21.15.99:0/1589059202 shutdown_connections 2023-10-08T00:12:04.493 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.491+0000 7fd01f055700 1 --2- 172.21.15.99:0/1589059202 >> v2:172.21.15.191:3300/0 conn(0x7fd01815d580 0x7fd018158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:04.493 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.491+0000 7fd01f055700 1 --2- 172.21.15.99:0/1589059202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd018157870 0x7fd01815b6a0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:04.493 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.491+0000 7fd01f055700 1 --2- 172.21.15.99:0/1589059202 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd01815a620 0x7fd01815d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:04.494 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.491+0000 7fd01f055700 1 -- 172.21.15.99:0/1589059202 >> 172.21.15.99:0/1589059202 conn(0x7fd018068ce0 msgr2=0x7fd0180690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:04.494 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.491+0000 7fd01f055700 1 -- 172.21.15.99:0/1589059202 shutdown_connections 2023-10-08T00:12:04.494 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.491+0000 7fd01f055700 1 -- 172.21.15.99:0/1589059202 wait complete. 2023-10-08T00:12:04.496 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.492+0000 7fd01f055700 1 Processor -- start 2023-10-08T00:12:04.496 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.492+0000 7fd01f055700 1 -- start start 2023-10-08T00:12:04.496 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.493+0000 7fd01f055700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd018157870 0x7fd01814a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:04.496 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.493+0000 7fd01f055700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd01815a620 0x7fd01815eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:04.497 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.493+0000 7fd01f055700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd01815d580 0x7fd01815f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:04.497 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.493+0000 7fd01f055700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd01814a910 con 0x7fd01815d580 2023-10-08T00:12:04.497 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.493+0000 7fd01f055700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd01816dd10 con 0x7fd018157870 2023-10-08T00:12:04.497 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.493+0000 7fd01f055700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd01816e5b0 con 0x7fd01815a620 2023-10-08T00:12:04.497 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.493+0000 7fd01d5f2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd01815d580 0x7fd01815f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:04.497 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.493+0000 7fd01d5f2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd01815d580 0x7fd01815f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41120/0 (socket says 172.21.15.99:41120) 2023-10-08T00:12:04.497 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.493+0000 7fd01d5f2700 1 -- 172.21.15.99:0/4252259344 learned_addr learned my addr 172.21.15.99:0/4252259344 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:04.497 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.493+0000 7fd017fff700 1 --2- 172.21.15.99:0/4252259344 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd01815a620 0x7fd01815eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:04.497 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.493+0000 7fd01cdf1700 1 --2- 172.21.15.99:0/4252259344 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd018157870 0x7fd01814a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:04.497 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.493+0000 7fd01d5f2700 1 -- 172.21.15.99:0/4252259344 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd01815a620 msgr2=0x7fd01815eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:04.497 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.493+0000 7fd01d5f2700 1 --2- 172.21.15.99:0/4252259344 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd01815a620 0x7fd01815eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:04.497 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.493+0000 7fd01d5f2700 1 -- 172.21.15.99:0/4252259344 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd018157870 msgr2=0x7fd01814a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:04.498 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.493+0000 7fd01d5f2700 1 --2- 172.21.15.99:0/4252259344 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd018157870 0x7fd01814a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:04.498 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.493+0000 7fd01d5f2700 1 -- 172.21.15.99:0/4252259344 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd010009ae0 con 0x7fd01815d580 2023-10-08T00:12:04.498 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.493+0000 7fd01d5f2700 1 --2- 172.21.15.99:0/4252259344 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd01815d580 0x7fd01815f0d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd00800d860 tx=0x7fd008008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:04.498 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.493+0000 7fd014ff9700 1 -- 172.21.15.99:0/4252259344 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0080125f0 con 0x7fd01815d580 2023-10-08T00:12:04.498 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.493+0000 7fd014ff9700 1 -- 172.21.15.99:0/4252259344 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd008012750 con 0x7fd01815d580 2023-10-08T00:12:04.498 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.493+0000 7fd014ff9700 1 -- 172.21.15.99:0/4252259344 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd008012af0 con 0x7fd01815d580 2023-10-08T00:12:04.498 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.493+0000 7fd01f055700 1 -- 172.21.15.99:0/4252259344 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd01816b4b0 con 0x7fd01815d580 2023-10-08T00:12:04.498 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.493+0000 7fd01f055700 1 -- 172.21.15.99:0/4252259344 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd01816e240 con 0x7fd01815d580 2023-10-08T00:12:04.498 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.494+0000 7fd01f055700 1 -- 172.21.15.99:0/4252259344 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd01816e720 con 0x7fd01815d580 2023-10-08T00:12:04.498 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.495+0000 7fd014ff9700 1 -- 172.21.15.99:0/4252259344 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd008013480 con 0x7fd01815d580 2023-10-08T00:12:04.498 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.495+0000 7fd014ff9700 1 --2- 172.21.15.99:0/4252259344 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd004066f30 0x7fd0040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:04.498 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.495+0000 7fd014ff9700 1 -- 172.21.15.99:0/4252259344 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fd0080844b0 con 0x7fd01815d580 2023-10-08T00:12:04.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.495+0000 7fd01cdf1700 1 --2- 172.21.15.99:0/4252259344 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd004066f30 0x7fd0040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:04.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.495+0000 7fd01cdf1700 1 --2- 172.21.15.99:0/4252259344 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd004066f30 0x7fd0040693e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd010009880 tx=0x7fd010018000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:04.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.499+0000 7fd014ff9700 1 -- 172.21.15.99:0/4252259344 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd008051a20 con 0x7fd01815d580 2023-10-08T00:12:04.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:04.696+0000 7fd01f055700 1 -- 172.21.15.99:0/4252259344 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fd01804c3c0 con 0x7fd01815d580 2023-10-08T00:12:05.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:05.875+0000 7fd014ff9700 1 -- 172.21.15.99:0/4252259344 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v41) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd008055040 con 0x7fd01815d580 2023-10-08T00:12:05.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:05.877+0000 7fd01f055700 1 -- 172.21.15.99:0/4252259344 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd004066f30 msgr2=0x7fd0040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:05.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:05.877+0000 7fd01f055700 1 --2- 172.21.15.99:0/4252259344 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd004066f30 0x7fd0040693e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd010009880 tx=0x7fd010018000 comp rx=0 tx=0).stop 2023-10-08T00:12:05.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:05.877+0000 7fd01f055700 1 -- 172.21.15.99:0/4252259344 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd01815d580 msgr2=0x7fd01815f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:05.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:05.877+0000 7fd01f055700 1 --2- 172.21.15.99:0/4252259344 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd01815d580 0x7fd01815f0d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd00800d860 tx=0x7fd008008040 comp rx=0 tx=0).stop 2023-10-08T00:12:05.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:05.877+0000 7fd01f055700 1 -- 172.21.15.99:0/4252259344 shutdown_connections 2023-10-08T00:12:05.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:05.877+0000 7fd01f055700 1 --2- 172.21.15.99:0/4252259344 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd004066f30 0x7fd0040693e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:05.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:05.877+0000 7fd01f055700 1 --2- 172.21.15.99:0/4252259344 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd018157870 0x7fd01814a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:05.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:05.877+0000 7fd01f055700 1 --2- 172.21.15.99:0/4252259344 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd01815d580 0x7fd01815f0d0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:05.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:05.877+0000 7fd01f055700 1 --2- 172.21.15.99:0/4252259344 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd01815a620 0x7fd01815eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:05.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:05.877+0000 7fd01f055700 1 -- 172.21.15.99:0/4252259344 >> 172.21.15.99:0/4252259344 conn(0x7fd018068ce0 msgr2=0x7fd0180b51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:05.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:05.877+0000 7fd01f055700 1 -- 172.21.15.99:0/4252259344 shutdown_connections 2023-10-08T00:12:05.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:05.878+0000 7fd01f055700 1 -- 172.21.15.99:0/4252259344 wait complete. 2023-10-08T00:12:05.880 INFO:teuthology.orchestra.run.smithi099.stderr: removed all blocklist entries 2023-10-08T00:12:05.891 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-08T00:12:06.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.006+0000 7fe34a9d1700 1 Processor -- start 2023-10-08T00:12:06.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.006+0000 7fe34a9d1700 1 -- start start 2023-10-08T00:12:06.409 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.007+0000 7fe34a9d1700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe344157870 0x7fe34415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:06.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.007+0000 7fe34a9d1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe344157c40 0x7fe344158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:06.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.007+0000 7fe34a9d1700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe34415cbb0 con 0x7fe34415a620 2023-10-08T00:12:06.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.007+0000 7fe34a9d1700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe34415bfa0 con 0x7fe344157870 2023-10-08T00:12:06.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.007+0000 7fe34a9d1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe34415c280 con 0x7fe344157c40 2023-10-08T00:12:06.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.007+0000 7fe343fff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe344157870 0x7fe34415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:06.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.007+0000 7fe3437fe700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fe34415a620 0x7fe34415d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56524/0 (socket says 172.21.15.99:56524) 2023-10-08T00:12:06.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.007+0000 7fe343fff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe344157870 0x7fe34415b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41132/0 (socket says 172.21.15.99:41132) 2023-10-08T00:12:06.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.007+0000 7fe3437fe700 1 -- 172.21.15.99:0/3217362750 learned_addr learned my addr 172.21.15.99:0/3217362750 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:12:06.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.007+0000 7fe348f6e700 1 --2- 172.21.15.99:0/3217362750 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe344157c40 0x7fe344158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:06.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.007+0000 7fe343fff700 1 -- 172.21.15.99:0/3217362750 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe344157c40 msgr2=0x7fe344158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:06.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.007+0000 7fe343fff700 1 --2- 172.21.15.99:0/3217362750 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe344157c40 0x7fe344158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:06.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.007+0000 7fe343fff700 1 -- 172.21.15.99:0/3217362750 >> v1:172.21.15.99:6789/0 conn(0x7fe34415a620 legacy=0x7fe34415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:06.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.007+0000 7fe342ffd700 1 -- 172.21.15.99:0/3217362750 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2549169858 0 0) 0x7fe34415cbb0 con 0x7fe34415a620 2023-10-08T00:12:06.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.007+0000 7fe343fff700 1 -- 172.21.15.99:0/3217362750 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe34415c560 con 0x7fe344157870 2023-10-08T00:12:06.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.008+0000 7fe343fff700 1 --2- 172.21.15.99:0/3217362750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe344157870 0x7fe34415b6a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe3340098e0 tx=0x7fe3340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f84a1325ad6d58 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:06.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.008+0000 7fe342ffd700 1 -- 172.21.15.99:0/3217362750 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe334009e80 con 0x7fe344157870 2023-10-08T00:12:06.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.008+0000 7fe342ffd700 1 -- 172.21.15.99:0/3217362750 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe33400b860 con 0x7fe344157870 2023-10-08T00:12:06.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.008+0000 7fe342ffd700 1 -- 172.21.15.99:0/3217362750 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33400bbe0 con 0x7fe344157870 2023-10-08T00:12:06.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.008+0000 7fe34a9d1700 1 -- 172.21.15.99:0/3217362750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe344157870 msgr2=0x7fe34415b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:06.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.008+0000 7fe34a9d1700 1 --2- 172.21.15.99:0/3217362750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe344157870 0x7fe34415b6a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe3340098e0 tx=0x7fe3340091a0 comp rx=0 tx=0).stop 2023-10-08T00:12:06.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.008+0000 7fe34a9d1700 1 -- 172.21.15.99:0/3217362750 shutdown_connections 2023-10-08T00:12:06.412 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.008+0000 7fe34a9d1700 1 --2- 172.21.15.99:0/3217362750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe344157870 0x7fe34415b6a0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:06.412 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.008+0000 7fe34a9d1700 1 --2- 172.21.15.99:0/3217362750 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe344157c40 0x7fe344158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:06.412 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.008+0000 7fe34a9d1700 1 -- 172.21.15.99:0/3217362750 >> 172.21.15.99:0/3217362750 conn(0x7fe344068ce0 msgr2=0x7fe3440690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:06.412 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.008+0000 7fe34a9d1700 1 -- 172.21.15.99:0/3217362750 shutdown_connections 2023-10-08T00:12:06.412 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.008+0000 7fe34a9d1700 1 -- 172.21.15.99:0/3217362750 wait complete. 2023-10-08T00:12:06.412 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.009+0000 7fe34a9d1700 1 Processor -- start 2023-10-08T00:12:06.412 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.010+0000 7fe34a9d1700 1 -- start start 2023-10-08T00:12:06.412 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.010+0000 7fe34a9d1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe344157870 0x7fe344152180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:06.412 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.010+0000 7fe34a9d1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe344157c40 0x7fe34414e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:06.414 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.010+0000 7fe34a9d1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe34415a620 0x7fe344150050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:06.414 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.010+0000 7fe34a9d1700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34414d630 con 0x7fe344157870 2023-10-08T00:12:06.414 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.010+0000 7fe34a9d1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3441526c0 con 0x7fe344157c40 2023-10-08T00:12:06.414 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.010+0000 7fe34a9d1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3441534e0 con 0x7fe34415a620 2023-10-08T00:12:06.414 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.010+0000 7fe343fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe344157870 0x7fe344152180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:06.414 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.010+0000 7fe343fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe344157870 0x7fe344152180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41142/0 (socket says 172.21.15.99:41142) 2023-10-08T00:12:06.414 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.010+0000 7fe343fff700 1 -- 172.21.15.99:0/2241585993 learned_addr learned my addr 172.21.15.99:0/2241585993 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:06.414 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.010+0000 7fe3437fe700 1 --2- 172.21.15.99:0/2241585993 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe344157c40 0x7fe34414e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:06.414 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.010+0000 7fe348f6e700 1 --2- 172.21.15.99:0/2241585993 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe34415a620 0x7fe344150050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:06.414 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.011+0000 7fe343fff700 1 -- 172.21.15.99:0/2241585993 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe34415a620 msgr2=0x7fe344150050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:06.414 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.011+0000 7fe343fff700 1 --2- 172.21.15.99:0/2241585993 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe34415a620 0x7fe344150050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:06.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.011+0000 7fe343fff700 1 -- 172.21.15.99:0/2241585993 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe344157c40 msgr2=0x7fe34414e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:06.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.011+0000 7fe343fff700 1 --2- 172.21.15.99:0/2241585993 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe344157c40 0x7fe34414e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:06.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.011+0000 7fe343fff700 1 -- 172.21.15.99:0/2241585993 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe334009420 con 0x7fe344157870 2023-10-08T00:12:06.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.011+0000 7fe343fff700 1 --2- 172.21.15.99:0/2241585993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe344157870 0x7fe344152180 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe34406b530 tx=0x7fe334017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:06.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.011+0000 7fe32bfff700 1 -- 172.21.15.99:0/2241585993 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe334011420 con 0x7fe344157870 2023-10-08T00:12:06.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.011+0000 7fe32bfff700 1 -- 172.21.15.99:0/2241585993 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3340115c0 con 0x7fe344157870 2023-10-08T00:12:06.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.011+0000 7fe34a9d1700 1 -- 172.21.15.99:0/2241585993 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3441528d0 con 0x7fe344157870 2023-10-08T00:12:06.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.011+0000 7fe32bfff700 1 -- 172.21.15.99:0/2241585993 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe334011980 con 0x7fe344157870 2023-10-08T00:12:06.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.011+0000 7fe34a9d1700 1 -- 172.21.15.99:0/2241585993 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe344152e90 con 0x7fe344157870 2023-10-08T00:12:06.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.012+0000 7fe34a9d1700 1 -- 172.21.15.99:0/2241585993 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe34404c3c0 con 0x7fe344157870 2023-10-08T00:12:06.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.012+0000 7fe32bfff700 1 -- 172.21.15.99:0/2241585993 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe3340253f0 con 0x7fe344157870 2023-10-08T00:12:06.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.013+0000 7fe32bfff700 1 --2- 172.21.15.99:0/2241585993 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe330066f30 0x7fe3300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:06.416 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.013+0000 7fe32bfff700 1 -- 172.21.15.99:0/2241585993 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fe334085c50 con 0x7fe344157870 2023-10-08T00:12:06.416 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.013+0000 7fe3437fe700 1 --2- 172.21.15.99:0/2241585993 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe330066f30 0x7fe3300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:06.416 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.014+0000 7fe3437fe700 1 --2- 172.21.15.99:0/2241585993 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe330066f30 0x7fe3300693e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe34414de10 tx=0x7fe338009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:06.416 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.017+0000 7fe32bfff700 1 -- 172.21.15.99:0/2241585993 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe33408b020 con 0x7fe344157870 2023-10-08T00:12:06.416 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.271+0000 7fe34a9d1700 1 -- 172.21.15.99:0/2241585993 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fe34415c840 con 0x7fe344157870 2023-10-08T00:12:06.416 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.272+0000 7fe32bfff700 1 -- 172.21.15.99:0/2241585993 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fe33405ae60 con 0x7fe344157870 2023-10-08T00:12:06.416 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:12:06.416 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:12:06.417 INFO:teuthology.orchestra.run.smithi099.stdout: "auth_dump": [ 2023-10-08T00:12:06.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:06.417 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.a", 2023-10-08T00:12:06.417 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlFlN9AhAAKdbeH0XznSOuyUnReD6RsA==", 2023-10-08T00:12:06.417 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:12:06.417 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:12:06.417 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2023-10-08T00:12:06.417 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2023-10-08T00:12:06.417 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:12:06.417 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:06.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:06.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:06.418 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.b", 2023-10-08T00:12:06.418 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlh2G0DRAA1mYGrUrELHsS9abUxCCG2g==", 2023-10-08T00:12:06.418 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:12:06.418 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:12:06.418 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2023-10-08T00:12:06.418 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2023-10-08T00:12:06.418 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:12:06.418 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:06.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:06.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:06.418 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.c", 2023-10-08T00:12:06.418 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFl/M1DCRAANrfHJGZO1s0KOyCdUzZWIA==", 2023-10-08T00:12:06.419 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:12:06.419 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:12:06.419 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2023-10-08T00:12:06.419 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2023-10-08T00:12:06.419 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:12:06.419 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:06.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:06.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:06.419 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.d", 2023-10-08T00:12:06.419 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlr5F2FBAAYYeG6EM0/N0FiEAwSbYRyA==", 2023-10-08T00:12:06.419 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:12:06.419 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:12:06.420 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2023-10-08T00:12:06.420 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2023-10-08T00:12:06.420 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:12:06.420 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:06.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:06.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:06.420 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.0", 2023-10-08T00:12:06.420 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB08yFla7rpKxAAWTOutLcrjDeR69JbPZMoLg==", 2023-10-08T00:12:06.420 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:12:06.420 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:12:06.420 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:12:06.420 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:12:06.420 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:06.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:06.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:06.421 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.1", 2023-10-08T00:12:06.421 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB08yFlw+y2MhAAAvLDIB/tb49IlaireTqTFg==", 2023-10-08T00:12:06.421 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:12:06.421 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:12:06.421 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:12:06.421 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:12:06.421 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:06.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:06.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:06.421 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.2", 2023-10-08T00:12:06.422 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB18yFlh4wvABAALXOtJZoEfGmt/pOBxReooQ==", 2023-10-08T00:12:06.422 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:12:06.422 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:12:06.422 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:12:06.422 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:12:06.422 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:06.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:06.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:06.422 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.3", 2023-10-08T00:12:06.422 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB18yFl8cVfCRAAoGXTACMT+f5Avll5mxG9OA==", 2023-10-08T00:12:06.422 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:12:06.423 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:12:06.423 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:12:06.423 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:12:06.423 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:06.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:06.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:06.423 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.4", 2023-10-08T00:12:06.423 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB48yFlft9WGhAA8JZN4MjSX9wbXiMKJnfCYg==", 2023-10-08T00:12:06.423 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:12:06.423 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:12:06.423 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:12:06.423 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:12:06.424 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:06.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:06.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:06.424 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.5", 2023-10-08T00:12:06.424 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB48yFlUf1BIRAA2FMBt1jAcm4ia/vYBDyojw==", 2023-10-08T00:12:06.424 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:12:06.424 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:12:06.424 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:12:06.424 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:12:06.424 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:06.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:06.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:06.424 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.6", 2023-10-08T00:12:06.425 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB48yFl0SI6KhAAI6G+Sqfm7OQmGf9nK59x9Q==", 2023-10-08T00:12:06.425 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:12:06.425 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:12:06.425 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:12:06.425 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:12:06.425 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:06.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:06.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:06.425 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.7", 2023-10-08T00:12:06.425 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB48yFlo5xWMxAA4p9OSLK0JpWhEExsifX+3A==", 2023-10-08T00:12:06.425 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:12:06.425 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:12:06.426 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:12:06.426 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:12:06.426 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:06.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:06.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:06.426 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.0", 2023-10-08T00:12:06.426 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlmzhQGBAAReslU/Pvcsr0dwE6l1N3/Q==", 2023-10-08T00:12:06.426 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:12:06.426 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:12:06.426 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow r", 2023-10-08T00:12:06.426 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-08T00:12:06.426 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:06.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:06.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:06.427 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.1", 2023-10-08T00:12:06.427 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFl2NKsIBAAUg6rXynl1J7qB6BkUizmpA==", 2023-10-08T00:12:06.427 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:12:06.427 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:12:06.427 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2023-10-08T00:12:06.427 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2023-10-08T00:12:06.427 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2023-10-08T00:12:06.427 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:06.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:06.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:06.428 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.2", 2023-10-08T00:12:06.428 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlMp6rHRAArVgJfA3mY4n78hOMeVlw1Q==", 2023-10-08T00:12:06.428 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:12:06.428 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:12:06.428 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2023-10-08T00:12:06.428 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2023-10-08T00:12:06.428 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2023-10-08T00:12:06.428 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:06.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:06.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:06.428 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.3", 2023-10-08T00:12:06.428 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlH0gNJhAAEgGVGiabVIWknpWUohOnHA==", 2023-10-08T00:12:06.429 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:12:06.429 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:12:06.429 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2023-10-08T00:12:06.429 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2023-10-08T00:12:06.429 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2023-10-08T00:12:06.429 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:06.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:06.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:06.429 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.admin", 2023-10-08T00:12:06.429 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBw8yFlidUfIxAAIvpvkHbyCx5JFvHo/g5A9g==", 2023-10-08T00:12:06.429 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:12:06.429 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2023-10-08T00:12:06.430 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2023-10-08T00:12:06.430 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2023-10-08T00:12:06.430 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:12:06.430 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:06.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:06.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:06.430 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mgr.x", 2023-10-08T00:12:06.430 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBw8yFl2kMLOxAAsEyuLBLsjQCUS0ERDOi75A==", 2023-10-08T00:12:06.430 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:12:06.430 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2023-10-08T00:12:06.430 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile mgr", 2023-10-08T00:12:06.430 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:12:06.431 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:06.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:06.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:06.431 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mgr.y", 2023-10-08T00:12:06.431 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBw8yFl6ht5NxAAFzXnLDjklCePD6hw/pSfFg==", 2023-10-08T00:12:06.431 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:12:06.431 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2023-10-08T00:12:06.431 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile mgr", 2023-10-08T00:12:06.431 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:12:06.431 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:06.431 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:06.431 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:06.431 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:12:06.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.274+0000 7fe34a9d1700 1 -- 172.21.15.99:0/2241585993 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe330066f30 msgr2=0x7fe3300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:06.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.274+0000 7fe34a9d1700 1 --2- 172.21.15.99:0/2241585993 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe330066f30 0x7fe3300693e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe34414de10 tx=0x7fe338009040 comp rx=0 tx=0).stop 2023-10-08T00:12:06.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.274+0000 7fe34a9d1700 1 -- 172.21.15.99:0/2241585993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe344157870 msgr2=0x7fe344152180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:06.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.274+0000 7fe34a9d1700 1 --2- 172.21.15.99:0/2241585993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe344157870 0x7fe344152180 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe34406b530 tx=0x7fe334017000 comp rx=0 tx=0).stop 2023-10-08T00:12:06.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.274+0000 7fe34a9d1700 1 -- 172.21.15.99:0/2241585993 shutdown_connections 2023-10-08T00:12:06.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.274+0000 7fe34a9d1700 1 --2- 172.21.15.99:0/2241585993 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe330066f30 0x7fe3300693e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:06.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.274+0000 7fe34a9d1700 1 --2- 172.21.15.99:0/2241585993 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe34415a620 0x7fe344150050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:06.434 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.274+0000 7fe34a9d1700 1 --2- 172.21.15.99:0/2241585993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe344157870 0x7fe344152180 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:06.434 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.274+0000 7fe34a9d1700 1 --2- 172.21.15.99:0/2241585993 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe344157c40 0x7fe34414e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:06.434 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.274+0000 7fe34a9d1700 1 -- 172.21.15.99:0/2241585993 >> 172.21.15.99:0/2241585993 conn(0x7fe344068ce0 msgr2=0x7fe3440690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:06.434 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.274+0000 7fe34a9d1700 1 -- 172.21.15.99:0/2241585993 shutdown_connections 2023-10-08T00:12:06.434 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.274+0000 7fe34a9d1700 1 -- 172.21.15.99:0/2241585993 wait complete. 2023-10-08T00:12:06.434 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:12:06.549 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.549+0000 7fdf4b11b700 1 Processor -- start 2023-10-08T00:12:06.550 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.549+0000 7fdf4b11b700 1 -- start start 2023-10-08T00:12:06.552 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.550+0000 7fdf4b11b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fdf440cd6e0 0x7fdf440cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:06.552 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.550+0000 7fdf4b11b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf4415cbd0 0x7fdf4415d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:06.552 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.550+0000 7fdf4b11b700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf440cb7c0 con 0x7fdf440cdab0 2023-10-08T00:12:06.552 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.550+0000 7fdf4b11b700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf4415e850 con 0x7fdf440cd6e0 2023-10-08T00:12:06.552 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.550+0000 7fdf4b11b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf440cd030 con 0x7fdf4415cbd0 2023-10-08T00:12:06.552 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.550+0000 7fdf4a119700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fdf440cd6e0 0x7fdf440cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:06.552 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.550+0000 7fdf4a91a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf4415cbd0 0x7fdf4415d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:06.553 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.550+0000 7fdf4a119700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fdf440cd6e0 0x7fdf440cbc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.99:46394/0 (socket says 172.21.15.99:46394) 2023-10-08T00:12:06.553 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.550+0000 7fdf4a91a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf4415cbd0 0x7fdf4415d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.99:54084/0 (socket says 172.21.15.99:54084) 2023-10-08T00:12:06.553 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.550+0000 7fdf49918700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7fdf440cdab0 0x7fdf4415d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:38360/0 (socket says 172.21.15.99:38360) 2023-10-08T00:12:06.553 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.550+0000 7fdf4a119700 1 -- 172.21.15.99:0/699664726 learned_addr learned my addr 172.21.15.99:0/699664726 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:06.553 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.550+0000 7fdf4a91a700 1 -- 172.21.15.99:0/699664726 >> v2:172.21.15.191:3300/0 conn(0x7fdf440cd6e0 msgr2=0x7fdf440cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:06.553 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.550+0000 7fdf4a91a700 1 --2- 172.21.15.99:0/699664726 >> v2:172.21.15.191:3300/0 conn(0x7fdf440cd6e0 0x7fdf440cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:06.553 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.550+0000 7fdf4a91a700 1 -- 172.21.15.99:0/699664726 >> v1:172.21.15.191:6789/0 conn(0x7fdf440cdab0 legacy=0x7fdf4415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:06.553 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.550+0000 7fdf4a91a700 1 -- 172.21.15.99:0/699664726 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf4415e290 con 0x7fdf4415cbd0 2023-10-08T00:12:06.553 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.550+0000 7fdf4a119700 1 --2- 172.21.15.99:0/699664726 >> v2:172.21.15.191:3300/0 conn(0x7fdf440cd6e0 0x7fdf440cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:12:06.553 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.551+0000 7fdf4a91a700 1 --2- 172.21.15.99:0/699664726 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf4415cbd0 0x7fdf4415d020 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c009e60 tx=0x7fdf3c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=211a73889ac4e436 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:06.553 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.551+0000 7fdf49117700 1 -- 172.21.15.99:0/699664726 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c00e070 con 0x7fdf4415cbd0 2023-10-08T00:12:06.553 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.551+0000 7fdf49117700 1 -- 172.21.15.99:0/699664726 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf3c00bb60 con 0x7fdf4415cbd0 2023-10-08T00:12:06.554 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.551+0000 7fdf49117700 1 -- 172.21.15.99:0/699664726 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c012430 con 0x7fdf4415cbd0 2023-10-08T00:12:06.554 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.551+0000 7fdf4b11b700 1 -- 172.21.15.99:0/699664726 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf4415cbd0 msgr2=0x7fdf4415d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:06.554 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.551+0000 7fdf4b11b700 1 --2- 172.21.15.99:0/699664726 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf4415cbd0 0x7fdf4415d020 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c009e60 tx=0x7fdf3c009780 comp rx=0 tx=0).stop 2023-10-08T00:12:06.554 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.551+0000 7fdf4b11b700 1 -- 172.21.15.99:0/699664726 shutdown_connections 2023-10-08T00:12:06.554 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.551+0000 7fdf4b11b700 1 --2- 172.21.15.99:0/699664726 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf4415cbd0 0x7fdf4415d020 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:06.554 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.551+0000 7fdf4b11b700 1 --2- 172.21.15.99:0/699664726 >> v2:172.21.15.191:3300/0 conn(0x7fdf440cd6e0 0x7fdf440cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:06.554 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.551+0000 7fdf4b11b700 1 -- 172.21.15.99:0/699664726 >> 172.21.15.99:0/699664726 conn(0x7fdf44067660 msgr2=0x7fdf44067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:06.554 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.551+0000 7fdf4b11b700 1 -- 172.21.15.99:0/699664726 shutdown_connections 2023-10-08T00:12:06.554 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.551+0000 7fdf4b11b700 1 -- 172.21.15.99:0/699664726 wait complete. 2023-10-08T00:12:06.557 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.552+0000 7fdf4b11b700 1 Processor -- start 2023-10-08T00:12:06.557 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.553+0000 7fdf4b11b700 1 -- start start 2023-10-08T00:12:06.557 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.553+0000 7fdf4b11b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf440cd6e0 0x7fdf4414c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:06.557 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.553+0000 7fdf4b11b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf440cdab0 0x7fdf440b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:06.557 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.553+0000 7fdf4b11b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf4415cbd0 0x7fdf440b6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:06.557 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.553+0000 7fdf4b11b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4414ce30 con 0x7fdf4415cbd0 2023-10-08T00:12:06.557 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.553+0000 7fdf4b11b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf440b4a40 con 0x7fdf440cd6e0 2023-10-08T00:12:06.557 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.553+0000 7fdf4b11b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf440b4d20 con 0x7fdf440cdab0 2023-10-08T00:12:06.557 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.553+0000 7fdf4a91a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf4415cbd0 0x7fdf440b6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:06.558 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.553+0000 7fdf4a91a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf4415cbd0 0x7fdf440b6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41146/0 (socket says 172.21.15.99:41146) 2023-10-08T00:12:06.558 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.553+0000 7fdf4a91a700 1 -- 172.21.15.99:0/2446531767 learned_addr learned my addr 172.21.15.99:0/2446531767 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:06.558 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.553+0000 7fdf4a119700 1 --2- 172.21.15.99:0/2446531767 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf440cd6e0 0x7fdf4414c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:06.558 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.553+0000 7fdf49918700 1 --2- 172.21.15.99:0/2446531767 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf440cdab0 0x7fdf440b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:06.558 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.554+0000 7fdf4a91a700 1 -- 172.21.15.99:0/2446531767 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf440cdab0 msgr2=0x7fdf440b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:06.558 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.554+0000 7fdf4a91a700 1 --2- 172.21.15.99:0/2446531767 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf440cdab0 0x7fdf440b5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:06.558 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.554+0000 7fdf4a91a700 1 -- 172.21.15.99:0/2446531767 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf440cd6e0 msgr2=0x7fdf4414c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:06.558 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.554+0000 7fdf4a91a700 1 --2- 172.21.15.99:0/2446531767 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf440cd6e0 0x7fdf4414c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:06.558 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.554+0000 7fdf4a91a700 1 -- 172.21.15.99:0/2446531767 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf400094f0 con 0x7fdf4415cbd0 2023-10-08T00:12:06.558 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.554+0000 7fdf4a91a700 1 --2- 172.21.15.99:0/2446531767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf4415cbd0 0x7fdf440b6020 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fdf440ccdd0 tx=0x7fdf3c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:06.558 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.554+0000 7fdf3a7fc700 1 -- 172.21.15.99:0/2446531767 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c012ea0 con 0x7fdf4415cbd0 2023-10-08T00:12:06.559 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.554+0000 7fdf3a7fc700 1 -- 172.21.15.99:0/2446531767 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf3c027470 con 0x7fdf4415cbd0 2023-10-08T00:12:06.559 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.554+0000 7fdf4b11b700 1 -- 172.21.15.99:0/2446531767 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf3c009a20 con 0x7fdf4415cbd0 2023-10-08T00:12:06.559 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.554+0000 7fdf3a7fc700 1 -- 172.21.15.99:0/2446531767 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c027850 con 0x7fdf4415cbd0 2023-10-08T00:12:06.559 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.554+0000 7fdf4b11b700 1 -- 172.21.15.99:0/2446531767 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf440b52e0 con 0x7fdf4415cbd0 2023-10-08T00:12:06.559 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.555+0000 7fdf4b11b700 1 -- 172.21.15.99:0/2446531767 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf44089280 con 0x7fdf4415cbd0 2023-10-08T00:12:06.559 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.555+0000 7fdf3a7fc700 1 -- 172.21.15.99:0/2446531767 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdf3c024070 con 0x7fdf4415cbd0 2023-10-08T00:12:06.559 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.556+0000 7fdf3a7fc700 1 --2- 172.21.15.99:0/2446531767 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdf28066f30 0x7fdf280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:06.559 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.556+0000 7fdf3a7fc700 1 -- 172.21.15.99:0/2446531767 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fdf3c086ca0 con 0x7fdf4415cbd0 2023-10-08T00:12:06.559 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.556+0000 7fdf4a119700 1 --2- 172.21.15.99:0/2446531767 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdf28066f30 0x7fdf280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:06.559 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.556+0000 7fdf4a119700 1 --2- 172.21.15.99:0/2446531767 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdf28066f30 0x7fdf280693e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdf440c7870 tx=0x7fdf40009250 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:06.560 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.560+0000 7fdf3a7fc700 1 -- 172.21.15.99:0/2446531767 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdf3c054260 con 0x7fdf4415cbd0 2023-10-08T00:12:06.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.742+0000 7fdf4b11b700 1 -- 172.21.15.99:0/2446531767 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf4415e570 con 0x7fdf4415cbd0 2023-10-08T00:12:06.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.743+0000 7fdf3a7fc700 1 -- 172.21.15.99:0/2446531767 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+12134 (secure 0 0 0) 0x7fdf3c02c090 con 0x7fdf4415cbd0 2023-10-08T00:12:06.743 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:12:06.744 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":41,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:12:05.873133+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:12:06.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.745+0000 7fdf4b11b700 1 -- 172.21.15.99:0/2446531767 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdf28066f30 msgr2=0x7fdf280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:06.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.745+0000 7fdf4b11b700 1 --2- 172.21.15.99:0/2446531767 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdf28066f30 0x7fdf280693e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdf440c7870 tx=0x7fdf40009250 comp rx=0 tx=0).stop 2023-10-08T00:12:06.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.745+0000 7fdf4b11b700 1 -- 172.21.15.99:0/2446531767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf4415cbd0 msgr2=0x7fdf440b6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:06.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.745+0000 7fdf4b11b700 1 --2- 172.21.15.99:0/2446531767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf4415cbd0 0x7fdf440b6020 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fdf440ccdd0 tx=0x7fdf3c019000 comp rx=0 tx=0).stop 2023-10-08T00:12:06.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.745+0000 7fdf4b11b700 1 -- 172.21.15.99:0/2446531767 shutdown_connections 2023-10-08T00:12:06.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.745+0000 7fdf4b11b700 1 --2- 172.21.15.99:0/2446531767 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdf28066f30 0x7fdf280693e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:06.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.745+0000 7fdf4b11b700 1 --2- 172.21.15.99:0/2446531767 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf440cd6e0 0x7fdf4414c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:06.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.745+0000 7fdf4b11b700 1 --2- 172.21.15.99:0/2446531767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf4415cbd0 0x7fdf440b6020 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:06.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.745+0000 7fdf4b11b700 1 --2- 172.21.15.99:0/2446531767 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf440cdab0 0x7fdf440b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:06.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.745+0000 7fdf4b11b700 1 -- 172.21.15.99:0/2446531767 >> 172.21.15.99:0/2446531767 conn(0x7fdf44067660 msgr2=0x7fdf441576d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:06.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.745+0000 7fdf4b11b700 1 -- 172.21.15.99:0/2446531767 shutdown_connections 2023-10-08T00:12:06.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.745+0000 7fdf4b11b700 1 -- 172.21.15.99:0/2446531767 wait complete. 2023-10-08T00:12:06.758 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:10:42.063344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-08T00:12:06.758 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T00:12:06.874 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.874+0000 7f4ca3795700 1 Processor -- start 2023-10-08T00:12:06.875 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.874+0000 7f4ca3795700 1 -- start start 2023-10-08T00:12:06.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.875+0000 7f4ca3795700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4c9c159dd0 0x7f4c9c067660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:06.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.875+0000 7f4ca3795700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4c9c0ce470 0x7f4c9c0cc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:06.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.875+0000 7f4ca3795700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c9c15a5c0 con 0x7f4c9c0cb520 2023-10-08T00:12:06.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.875+0000 7f4ca3795700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c9c0ccc90 con 0x7f4c9c159dd0 2023-10-08T00:12:06.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.875+0000 7f4ca3795700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c067cf0 con 0x7f4c9c0ce470 2023-10-08T00:12:06.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.875+0000 7f4ca1d32700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f4c9c0cb520 0x7f4c9c0cb9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56526/0 (socket says 172.21.15.99:56526) 2023-10-08T00:12:06.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.875+0000 7f4ca1d32700 1 -- 172.21.15.99:0/3068896765 learned_addr learned my addr 172.21.15.99:0/3068896765 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:12:06.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.875+0000 7f4ca1531700 1 --2- 172.21.15.99:0/3068896765 >> v2:172.21.15.191:3300/0 conn(0x7f4c9c159dd0 0x7f4c9c067660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:06.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.875+0000 7f4ca0d30700 1 --2- 172.21.15.99:0/3068896765 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4c9c0ce470 0x7f4c9c0cc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:06.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.875+0000 7f4ca1531700 1 -- 172.21.15.99:0/3068896765 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4c9c0ce470 msgr2=0x7f4c9c0cc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:06.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.875+0000 7f4ca1531700 1 --2- 172.21.15.99:0/3068896765 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4c9c0ce470 0x7f4c9c0cc750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:06.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.875+0000 7f4ca1531700 1 -- 172.21.15.99:0/3068896765 >> v1:172.21.15.99:6789/0 conn(0x7f4c9c0cb520 legacy=0x7f4c9c0cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:06.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.875+0000 7f4ca1531700 1 -- 172.21.15.99:0/3068896765 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c9c067fd0 con 0x7f4c9c159dd0 2023-10-08T00:12:06.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.876+0000 7f4ca1531700 1 --2- 172.21.15.99:0/3068896765 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c9c159dd0 0x7f4c9c067660 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4c980098e0 tx=0x7f4c980091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=266653cef677d186 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:06.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.877+0000 7f4c93fff700 1 -- 172.21.15.99:0/3068896765 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c9800b640 con 0x7f4c9c159dd0 2023-10-08T00:12:06.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.877+0000 7f4c93fff700 1 -- 172.21.15.99:0/3068896765 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c9800b7a0 con 0x7f4c9c159dd0 2023-10-08T00:12:06.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.877+0000 7f4c93fff700 1 -- 172.21.15.99:0/3068896765 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c9800bb40 con 0x7f4c9c159dd0 2023-10-08T00:12:06.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.877+0000 7f4ca3795700 1 -- 172.21.15.99:0/3068896765 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c9c159dd0 msgr2=0x7f4c9c067660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:06.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.877+0000 7f4ca3795700 1 --2- 172.21.15.99:0/3068896765 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c9c159dd0 0x7f4c9c067660 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4c980098e0 tx=0x7f4c980091a0 comp rx=0 tx=0).stop 2023-10-08T00:12:06.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.877+0000 7f4ca3795700 1 -- 172.21.15.99:0/3068896765 shutdown_connections 2023-10-08T00:12:06.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.877+0000 7f4ca3795700 1 --2- 172.21.15.99:0/3068896765 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c9c159dd0 0x7f4c9c067660 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:06.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.877+0000 7f4ca3795700 1 --2- 172.21.15.99:0/3068896765 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4c9c0ce470 0x7f4c9c0cc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:06.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.877+0000 7f4ca3795700 1 -- 172.21.15.99:0/3068896765 >> 172.21.15.99:0/3068896765 conn(0x7f4c9c069870 msgr2=0x7f4c9c069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:06.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.878+0000 7f4ca3795700 1 -- 172.21.15.99:0/3068896765 shutdown_connections 2023-10-08T00:12:06.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.878+0000 7f4ca3795700 1 -- 172.21.15.99:0/3068896765 wait complete. 2023-10-08T00:12:06.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.879+0000 7f4ca3795700 1 Processor -- start 2023-10-08T00:12:06.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.879+0000 7f4ca3795700 1 -- start start 2023-10-08T00:12:06.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.879+0000 7f4ca3795700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c9c0cb520 0x7f4c9c0b2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:06.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.879+0000 7f4ca3795700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c9c0ce470 0x7f4c9c154b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:06.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.879+0000 7f4ca3795700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4c9c159dd0 0x7f4c9c1554c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:06.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.879+0000 7f4ca3795700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c0b4140 con 0x7f4c9c0cb520 2023-10-08T00:12:06.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.879+0000 7f4ca3795700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c0b3530 con 0x7f4c9c0ce470 2023-10-08T00:12:06.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.879+0000 7f4ca3795700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c0b3810 con 0x7f4c9c159dd0 2023-10-08T00:12:06.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.880+0000 7f4ca1531700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c9c0cb520 0x7f4c9c0b2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:06.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.880+0000 7f4ca1531700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c9c0cb520 0x7f4c9c0b2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41150/0 (socket says 172.21.15.99:41150) 2023-10-08T00:12:06.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.880+0000 7f4ca1531700 1 -- 172.21.15.99:0/4034475128 learned_addr learned my addr 172.21.15.99:0/4034475128 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:06.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.880+0000 7f4ca0d30700 1 --2- 172.21.15.99:0/4034475128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c9c0ce470 0x7f4c9c154b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:06.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.880+0000 7f4ca1d32700 1 --2- 172.21.15.99:0/4034475128 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4c9c159dd0 0x7f4c9c1554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:06.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.880+0000 7f4ca1531700 1 -- 172.21.15.99:0/4034475128 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4c9c159dd0 msgr2=0x7f4c9c1554c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:06.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.880+0000 7f4ca1531700 1 --2- 172.21.15.99:0/4034475128 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4c9c159dd0 0x7f4c9c1554c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:06.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.880+0000 7f4ca1531700 1 -- 172.21.15.99:0/4034475128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c9c0ce470 msgr2=0x7f4c9c154b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:06.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.880+0000 7f4ca1531700 1 --2- 172.21.15.99:0/4034475128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c9c0ce470 0x7f4c9c154b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:06.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.880+0000 7f4ca1531700 1 -- 172.21.15.99:0/4034475128 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c98009420 con 0x7f4c9c0cb520 2023-10-08T00:12:06.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.880+0000 7f4ca1d32700 1 --2- 172.21.15.99:0/4034475128 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4c9c159dd0 0x7f4c9c1554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:12:06.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.880+0000 7f4ca0d30700 1 --2- 172.21.15.99:0/4034475128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c9c0ce470 0x7f4c9c154b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:12:06.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.880+0000 7f4ca1531700 1 --2- 172.21.15.99:0/4034475128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c9c0cb520 0x7f4c9c0b2580 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4c9800a010 tx=0x7f4c98017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:06.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.880+0000 7f4c917fa700 1 -- 172.21.15.99:0/4034475128 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c9801f070 con 0x7f4c9c0cb520 2023-10-08T00:12:06.884 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.880+0000 7f4c917fa700 1 -- 172.21.15.99:0/4034475128 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c9801aab0 con 0x7f4c9c0cb520 2023-10-08T00:12:06.884 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.880+0000 7f4ca3795700 1 -- 172.21.15.99:0/4034475128 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c9c0b3af0 con 0x7f4c9c0cb520 2023-10-08T00:12:06.884 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.880+0000 7f4c917fa700 1 -- 172.21.15.99:0/4034475128 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c9801ae50 con 0x7f4c9c0cb520 2023-10-08T00:12:06.884 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.880+0000 7f4ca3795700 1 -- 172.21.15.99:0/4034475128 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c9c0b45d0 con 0x7f4c9c0cb520 2023-10-08T00:12:06.884 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.881+0000 7f4ca3795700 1 -- 172.21.15.99:0/4034475128 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c9c04c3c0 con 0x7f4c9c0cb520 2023-10-08T00:12:06.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.882+0000 7f4c917fa700 1 -- 172.21.15.99:0/4034475128 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4c98023070 con 0x7f4c9c0cb520 2023-10-08T00:12:06.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.882+0000 7f4c917fa700 1 --2- 172.21.15.99:0/4034475128 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4c80066f30 0x7f4c800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:06.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.882+0000 7f4c917fa700 1 -- 172.21.15.99:0/4034475128 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f4c98085510 con 0x7f4c9c0cb520 2023-10-08T00:12:06.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.882+0000 7f4ca0d30700 1 --2- 172.21.15.99:0/4034475128 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4c80066f30 0x7f4c800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:06.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.883+0000 7f4ca0d30700 1 --2- 172.21.15.99:0/4034475128 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4c80066f30 0x7f4c800693e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c16d4b0 tx=0x7f4c8c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:06.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:06.886+0000 7f4c917fa700 1 -- 172.21.15.99:0/4034475128 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4c98052ad0 con 0x7f4c9c0cb520 2023-10-08T00:12:07.067 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.066+0000 7f4ca3795700 1 -- 172.21.15.99:0/4034475128 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4c9c068ca0 con 0x7f4c9c0cb520 2023-10-08T00:12:07.067 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.067+0000 7f4c917fa700 1 -- 172.21.15.99:0/4034475128 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4c980560f0 con 0x7f4c9c0cb520 2023-10-08T00:12:07.068 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2023-10-08T00:12:07.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.069+0000 7f4ca3795700 1 -- 172.21.15.99:0/4034475128 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4c80066f30 msgr2=0x7f4c800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:07.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.069+0000 7f4ca3795700 1 --2- 172.21.15.99:0/4034475128 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4c80066f30 0x7f4c800693e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c16d4b0 tx=0x7f4c8c008040 comp rx=0 tx=0).stop 2023-10-08T00:12:07.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.069+0000 7f4ca3795700 1 -- 172.21.15.99:0/4034475128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c9c0cb520 msgr2=0x7f4c9c0b2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:07.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.069+0000 7f4ca3795700 1 --2- 172.21.15.99:0/4034475128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c9c0cb520 0x7f4c9c0b2580 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4c9800a010 tx=0x7f4c98017000 comp rx=0 tx=0).stop 2023-10-08T00:12:07.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.069+0000 7f4ca3795700 1 -- 172.21.15.99:0/4034475128 shutdown_connections 2023-10-08T00:12:07.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.069+0000 7f4ca3795700 1 --2- 172.21.15.99:0/4034475128 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4c80066f30 0x7f4c800693e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:07.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.069+0000 7f4ca3795700 1 --2- 172.21.15.99:0/4034475128 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4c9c159dd0 0x7f4c9c1554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:07.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.069+0000 7f4ca3795700 1 --2- 172.21.15.99:0/4034475128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c9c0cb520 0x7f4c9c0b2580 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:07.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.069+0000 7f4ca3795700 1 --2- 172.21.15.99:0/4034475128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c9c0ce470 0x7f4c9c154b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:07.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.069+0000 7f4ca3795700 1 -- 172.21.15.99:0/4034475128 >> 172.21.15.99:0/4034475128 conn(0x7f4c9c069870 msgr2=0x7f4c9c0cce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:07.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.069+0000 7f4ca3795700 1 -- 172.21.15.99:0/4034475128 shutdown_connections 2023-10-08T00:12:07.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.069+0000 7f4ca3795700 1 -- 172.21.15.99:0/4034475128 wait complete. 2023-10-08T00:12:07.081 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T00:12:07.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.195+0000 7feec2b48700 1 Processor -- start 2023-10-08T00:12:07.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.195+0000 7feec2b48700 1 -- start start 2023-10-08T00:12:07.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.196+0000 7feec2b48700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7feebc157870 0x7feebc15b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:07.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.196+0000 7feec2b48700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feebc157c40 0x7feebc158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:07.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.196+0000 7feec2b48700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feebc15cbb0 con 0x7feebc15a620 2023-10-08T00:12:07.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.196+0000 7feec2b48700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feebc15bfa0 con 0x7feebc157870 2023-10-08T00:12:07.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.196+0000 7feec2b48700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feebc15c280 con 0x7feebc157c40 2023-10-08T00:12:07.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.196+0000 7feec08e4700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7feebc157870 0x7feebc15b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:07.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.196+0000 7feec08e4700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7feebc157870 0x7feebc15b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41152/0 (socket says 172.21.15.99:41152) 2023-10-08T00:12:07.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.196+0000 7feec08e4700 1 -- 172.21.15.99:0/255342354 learned_addr learned my addr 172.21.15.99:0/255342354 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:07.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.196+0000 7feec10e5700 1 --2- 172.21.15.99:0/255342354 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feebc157c40 0x7feebc158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:07.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.196+0000 7feec08e4700 1 -- 172.21.15.99:0/255342354 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feebc157c40 msgr2=0x7feebc158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:07.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.196+0000 7feec08e4700 1 --2- 172.21.15.99:0/255342354 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feebc157c40 0x7feebc158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:07.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.196+0000 7feec08e4700 1 -- 172.21.15.99:0/255342354 >> v1:172.21.15.191:6789/0 conn(0x7feebc15a620 legacy=0x7feebc15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:07.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.196+0000 7feec08e4700 1 -- 172.21.15.99:0/255342354 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feebc15c560 con 0x7feebc157870 2023-10-08T00:12:07.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.196+0000 7feec08e4700 1 --2- 172.21.15.99:0/255342354 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feebc157870 0x7feebc15b6a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7feeb80099b0 tx=0x7feeb8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7850e211ab1252dd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:07.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.196+0000 7feeb37fe700 1 -- 172.21.15.99:0/255342354 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb800b860 con 0x7feebc157870 2023-10-08T00:12:07.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.196+0000 7feeb37fe700 1 -- 172.21.15.99:0/255342354 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeb800b9c0 con 0x7feebc157870 2023-10-08T00:12:07.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.196+0000 7feeb37fe700 1 -- 172.21.15.99:0/255342354 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb800bd60 con 0x7feebc157870 2023-10-08T00:12:07.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.197+0000 7feec2b48700 1 -- 172.21.15.99:0/255342354 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feebc157870 msgr2=0x7feebc15b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:07.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.197+0000 7feec2b48700 1 --2- 172.21.15.99:0/255342354 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feebc157870 0x7feebc15b6a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7feeb80099b0 tx=0x7feeb8009250 comp rx=0 tx=0).stop 2023-10-08T00:12:07.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.197+0000 7feec2b48700 1 -- 172.21.15.99:0/255342354 shutdown_connections 2023-10-08T00:12:07.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.197+0000 7feec2b48700 1 --2- 172.21.15.99:0/255342354 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feebc157c40 0x7feebc158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:07.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.197+0000 7feec2b48700 1 --2- 172.21.15.99:0/255342354 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feebc157870 0x7feebc15b6a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:07.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.197+0000 7feec2b48700 1 -- 172.21.15.99:0/255342354 >> 172.21.15.99:0/255342354 conn(0x7feebc068ce0 msgr2=0x7feebc0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:07.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.197+0000 7feec2b48700 1 -- 172.21.15.99:0/255342354 shutdown_connections 2023-10-08T00:12:07.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.197+0000 7feec2b48700 1 -- 172.21.15.99:0/255342354 wait complete. 2023-10-08T00:12:07.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.198+0000 7feec2b48700 1 Processor -- start 2023-10-08T00:12:07.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.198+0000 7feec2b48700 1 -- start start 2023-10-08T00:12:07.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.199+0000 7feec2b48700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feebc157870 0x7feebc1458e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:07.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.199+0000 7feec2b48700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feebc157c40 0x7feebc0c5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:07.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.199+0000 7feec2b48700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feebc15a620 0x7feebc0c74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:07.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.199+0000 7feec2b48700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feebc15d5f0 con 0x7feebc15a620 2023-10-08T00:12:07.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.199+0000 7feec2b48700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feebc0c65d0 con 0x7feebc157870 2023-10-08T00:12:07.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.199+0000 7feec2b48700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feebc0c68b0 con 0x7feebc157c40 2023-10-08T00:12:07.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.199+0000 7feec10e5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feebc15a620 0x7feebc0c74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:07.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.199+0000 7feec10e5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feebc15a620 0x7feebc0c74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41156/0 (socket says 172.21.15.99:41156) 2023-10-08T00:12:07.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.199+0000 7feec08e4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feebc157870 0x7feebc1458e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:07.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.199+0000 7feec10e5700 1 -- 172.21.15.99:0/4050544192 learned_addr learned my addr 172.21.15.99:0/4050544192 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:07.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.199+0000 7feeb3fff700 1 --2- 172.21.15.99:0/4050544192 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feebc157c40 0x7feebc0c5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:07.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.199+0000 7feec10e5700 1 -- 172.21.15.99:0/4050544192 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feebc157c40 msgr2=0x7feebc0c5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:07.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.199+0000 7feec10e5700 1 --2- 172.21.15.99:0/4050544192 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feebc157c40 0x7feebc0c5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:07.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.199+0000 7feec10e5700 1 -- 172.21.15.99:0/4050544192 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feebc157870 msgr2=0x7feebc1458e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:07.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.199+0000 7feec10e5700 1 --2- 172.21.15.99:0/4050544192 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feebc157870 0x7feebc1458e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:07.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.199+0000 7feec10e5700 1 -- 172.21.15.99:0/4050544192 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb80094f0 con 0x7feebc15a620 2023-10-08T00:12:07.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.200+0000 7feec10e5700 1 --2- 172.21.15.99:0/4050544192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feebc15a620 0x7feebc0c74d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7feeb4007f30 tx=0x7feeb40098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:07.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.200+0000 7feeb0ff9700 1 -- 172.21.15.99:0/4050544192 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb400eb90 con 0x7feebc15a620 2023-10-08T00:12:07.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.200+0000 7feeb0ff9700 1 -- 172.21.15.99:0/4050544192 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeb400ecf0 con 0x7feebc15a620 2023-10-08T00:12:07.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.200+0000 7feec2b48700 1 -- 172.21.15.99:0/4050544192 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feebc0c6b90 con 0x7feebc15a620 2023-10-08T00:12:07.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.200+0000 7feeb0ff9700 1 -- 172.21.15.99:0/4050544192 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb4014670 con 0x7feebc15a620 2023-10-08T00:12:07.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.200+0000 7feec2b48700 1 -- 172.21.15.99:0/4050544192 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feebc156010 con 0x7feebc15a620 2023-10-08T00:12:07.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.201+0000 7feec2b48700 1 -- 172.21.15.99:0/4050544192 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feebc15d760 con 0x7feebc15a620 2023-10-08T00:12:07.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.201+0000 7feeb0ff9700 1 -- 172.21.15.99:0/4050544192 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7feeb4014a20 con 0x7feebc15a620 2023-10-08T00:12:07.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.201+0000 7feeb0ff9700 1 --2- 172.21.15.99:0/4050544192 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7feea8066f30 0x7feea80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:07.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.201+0000 7feeb0ff9700 1 -- 172.21.15.99:0/4050544192 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4853+0+0 (secure 0 0 0) 0x7feeb4084820 con 0x7feebc15a620 2023-10-08T00:12:07.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.202+0000 7feec08e4700 1 --2- 172.21.15.99:0/4050544192 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7feea8066f30 0x7feea80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:07.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.202+0000 7feec08e4700 1 --2- 172.21.15.99:0/4050544192 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7feea8066f30 0x7feea80693e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7feeb80099b0 tx=0x7feeb8017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:07.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.205+0000 7feeb0ff9700 1 -- 172.21.15.99:0/4050544192 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feeb4051de0 con 0x7feebc15a620 2023-10-08T00:12:07.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.392+0000 7feec2b48700 1 -- 172.21.15.99:0/4050544192 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7feebc04c3c0 con 0x7feebc15a620 2023-10-08T00:12:07.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.392+0000 7feeb0ff9700 1 -- 172.21.15.99:0/4050544192 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v41) v1 ==== 84+0+65 (secure 0 0 0) 0x7feeb400ee60 con 0x7feebc15a620 2023-10-08T00:12:07.393 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:12:07.393 INFO:teuthology.orchestra.run.smithi099.stdout:[ 2023-10-08T00:12:07.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:07.394 INFO:teuthology.orchestra.run.smithi099.stdout: "poolnum": 1, 2023-10-08T00:12:07.394 INFO:teuthology.orchestra.run.smithi099.stdout: "poolname": ".mgr" 2023-10-08T00:12:07.394 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:07.394 INFO:teuthology.orchestra.run.smithi099.stdout:] 2023-10-08T00:12:07.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.394+0000 7feec2b48700 1 -- 172.21.15.99:0/4050544192 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7feea8066f30 msgr2=0x7feea80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:07.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.394+0000 7feec2b48700 1 --2- 172.21.15.99:0/4050544192 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7feea8066f30 0x7feea80693e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7feeb80099b0 tx=0x7feeb8017000 comp rx=0 tx=0).stop 2023-10-08T00:12:07.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.394+0000 7feec2b48700 1 -- 172.21.15.99:0/4050544192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feebc15a620 msgr2=0x7feebc0c74d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:07.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.394+0000 7feec2b48700 1 --2- 172.21.15.99:0/4050544192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feebc15a620 0x7feebc0c74d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7feeb4007f30 tx=0x7feeb40098e0 comp rx=0 tx=0).stop 2023-10-08T00:12:07.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.394+0000 7feec2b48700 1 -- 172.21.15.99:0/4050544192 shutdown_connections 2023-10-08T00:12:07.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.394+0000 7feec2b48700 1 --2- 172.21.15.99:0/4050544192 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7feea8066f30 0x7feea80693e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:07.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.394+0000 7feec2b48700 1 --2- 172.21.15.99:0/4050544192 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feebc157870 0x7feebc1458e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:07.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.394+0000 7feec2b48700 1 --2- 172.21.15.99:0/4050544192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feebc15a620 0x7feebc0c74d0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:07.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.394+0000 7feec2b48700 1 --2- 172.21.15.99:0/4050544192 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feebc157c40 0x7feebc0c5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:07.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.394+0000 7feec2b48700 1 -- 172.21.15.99:0/4050544192 >> 172.21.15.99:0/4050544192 conn(0x7feebc068ce0 msgr2=0x7feebc0690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:07.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.394+0000 7feec2b48700 1 -- 172.21.15.99:0/4050544192 shutdown_connections 2023-10-08T00:12:07.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.394+0000 7feec2b48700 1 -- 172.21.15.99:0/4050544192 wait complete. 2023-10-08T00:12:07.407 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T00:12:07.407 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T00:12:07.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.524+0000 7f83163fb700 1 Processor -- start 2023-10-08T00:12:07.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.524+0000 7f83163fb700 1 -- start start 2023-10-08T00:12:07.527 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.525+0000 7f83163fb700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8310159ef0 0x7f831015a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:07.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.525+0000 7f83163fb700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f83100b46f0 0x7f83100b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:07.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.525+0000 7f83163fb700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83100b17b0 con 0x7f83100b6a10 2023-10-08T00:12:07.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.525+0000 7f83163fb700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83100b5020 con 0x7f83100b46f0 2023-10-08T00:12:07.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.525+0000 7f83163fb700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83100b1a60 con 0x7f8310159ef0 2023-10-08T00:12:07.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.525+0000 7f8314998700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f83100b46f0 0x7f83100b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:07.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.525+0000 7f8314998700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f83100b46f0 0x7f83100b4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41168/0 (socket says 172.21.15.99:41168) 2023-10-08T00:12:07.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.525+0000 7f8314998700 1 -- 172.21.15.99:0/3783721002 learned_addr learned my addr 172.21.15.99:0/3783721002 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:07.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.526+0000 7f830ffff700 1 --2- 172.21.15.99:0/3783721002 >> v2:172.21.15.191:3300/0 conn(0x7f8310159ef0 0x7f831015a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:07.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.526+0000 7f8314998700 1 -- 172.21.15.99:0/3783721002 >> v2:172.21.15.191:3300/0 conn(0x7f8310159ef0 msgr2=0x7f831015a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:07.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.526+0000 7f8314998700 1 --2- 172.21.15.99:0/3783721002 >> v2:172.21.15.191:3300/0 conn(0x7f8310159ef0 0x7f831015a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:07.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.526+0000 7f8314998700 1 -- 172.21.15.99:0/3783721002 >> v1:172.21.15.191:6789/0 conn(0x7f83100b6a10 legacy=0x7f83100b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:07.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.526+0000 7f8314998700 1 -- 172.21.15.99:0/3783721002 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83100b1d40 con 0x7f83100b46f0 2023-10-08T00:12:07.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.526+0000 7f8314998700 1 --2- 172.21.15.99:0/3783721002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83100b46f0 0x7f83100b4ae0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f831008a710 tx=0x7f8308009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f297d7c8d3540614 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:07.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.526+0000 7f830effd700 1 -- 172.21.15.99:0/3783721002 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830800e070 con 0x7f83100b46f0 2023-10-08T00:12:07.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.526+0000 7f830effd700 1 -- 172.21.15.99:0/3783721002 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f830800ba20 con 0x7f83100b46f0 2023-10-08T00:12:07.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.526+0000 7f830effd700 1 -- 172.21.15.99:0/3783721002 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83080092e0 con 0x7f83100b46f0 2023-10-08T00:12:07.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.526+0000 7f83163fb700 1 -- 172.21.15.99:0/3783721002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83100b46f0 msgr2=0x7f83100b4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:07.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.526+0000 7f83163fb700 1 --2- 172.21.15.99:0/3783721002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83100b46f0 0x7f83100b4ae0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f831008a710 tx=0x7f8308009840 comp rx=0 tx=0).stop 2023-10-08T00:12:07.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.527+0000 7f83163fb700 1 -- 172.21.15.99:0/3783721002 shutdown_connections 2023-10-08T00:12:07.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.527+0000 7f83163fb700 1 --2- 172.21.15.99:0/3783721002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83100b46f0 0x7f83100b4ae0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:07.530 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.527+0000 7f83163fb700 1 --2- 172.21.15.99:0/3783721002 >> v2:172.21.15.191:3300/0 conn(0x7f8310159ef0 0x7f831015a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:07.530 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.527+0000 7f83163fb700 1 -- 172.21.15.99:0/3783721002 >> 172.21.15.99:0/3783721002 conn(0x7f831009a0a0 msgr2=0x7f8310067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:07.530 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.527+0000 7f83163fb700 1 -- 172.21.15.99:0/3783721002 shutdown_connections 2023-10-08T00:12:07.530 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.527+0000 7f83163fb700 1 -- 172.21.15.99:0/3783721002 wait complete. 2023-10-08T00:12:07.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.528+0000 7f83163fb700 1 Processor -- start 2023-10-08T00:12:07.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.528+0000 7f83163fb700 1 -- start start 2023-10-08T00:12:07.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.528+0000 7f83163fb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83100b46f0 0x7f831014a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:07.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.529+0000 7f83163fb700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83100b6a10 0x7f831009b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:07.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.529+0000 7f83163fb700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8310159ef0 0x7f8310069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:07.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.529+0000 7f83163fb700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83100ae520 con 0x7f83100b46f0 2023-10-08T00:12:07.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.529+0000 7f83163fb700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f831009e0d0 con 0x7f83100b6a10 2023-10-08T00:12:07.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.529+0000 7f83163fb700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f831009e3b0 con 0x7f8310159ef0 2023-10-08T00:12:07.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.529+0000 7f830ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83100b46f0 0x7f831014a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:07.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.529+0000 7f830ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83100b46f0 0x7f831014a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41172/0 (socket says 172.21.15.99:41172) 2023-10-08T00:12:07.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.529+0000 7f830ffff700 1 -- 172.21.15.99:0/1027021451 learned_addr learned my addr 172.21.15.99:0/1027021451 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:07.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.529+0000 7f830f7fe700 1 --2- 172.21.15.99:0/1027021451 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83100b6a10 0x7f831009b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:07.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.529+0000 7f8314998700 1 --2- 172.21.15.99:0/1027021451 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8310159ef0 0x7f8310069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:07.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.529+0000 7f830ffff700 1 -- 172.21.15.99:0/1027021451 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8310159ef0 msgr2=0x7f8310069dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:07.534 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.529+0000 7f830ffff700 1 --2- 172.21.15.99:0/1027021451 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8310159ef0 0x7f8310069dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:07.534 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.529+0000 7f830ffff700 1 -- 172.21.15.99:0/1027021451 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83100b6a10 msgr2=0x7f831009b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:07.534 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.529+0000 7f830ffff700 1 --2- 172.21.15.99:0/1027021451 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83100b6a10 0x7f831009b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:07.534 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.529+0000 7f830ffff700 1 -- 172.21.15.99:0/1027021451 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8308009ae0 con 0x7f83100b46f0 2023-10-08T00:12:07.534 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.529+0000 7f830f7fe700 1 --2- 172.21.15.99:0/1027021451 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83100b6a10 0x7f831009b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:12:07.534 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.529+0000 7f830ffff700 1 --2- 172.21.15.99:0/1027021451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83100b46f0 0x7f831014a3e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f830000ea20 tx=0x7f8300009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:07.534 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.529+0000 7f82f7fff700 1 -- 172.21.15.99:0/1027021451 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830000c850 con 0x7f83100b46f0 2023-10-08T00:12:07.534 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.529+0000 7f82f7fff700 1 -- 172.21.15.99:0/1027021451 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f830000c9b0 con 0x7f83100b46f0 2023-10-08T00:12:07.534 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.529+0000 7f82f7fff700 1 -- 172.21.15.99:0/1027021451 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830000cd70 con 0x7f83100b46f0 2023-10-08T00:12:07.534 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.529+0000 7f83163fb700 1 -- 172.21.15.99:0/1027021451 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f831009e690 con 0x7f83100b46f0 2023-10-08T00:12:07.534 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.529+0000 7f83163fb700 1 -- 172.21.15.99:0/1027021451 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f831009bab0 con 0x7f83100b46f0 2023-10-08T00:12:07.535 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.530+0000 7f83163fb700 1 -- 172.21.15.99:0/1027021451 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f831009bcd0 con 0x7f83100b46f0 2023-10-08T00:12:07.535 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.530+0000 7f82f7fff700 1 -- 172.21.15.99:0/1027021451 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8300013680 con 0x7f83100b46f0 2023-10-08T00:12:07.535 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.531+0000 7f82f7fff700 1 --2- 172.21.15.99:0/1027021451 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f82fc066f30 0x7f82fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:07.535 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.531+0000 7f82f7fff700 1 -- 172.21.15.99:0/1027021451 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f8300084840 con 0x7f83100b46f0 2023-10-08T00:12:07.535 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.531+0000 7f830f7fe700 1 --2- 172.21.15.99:0/1027021451 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f82fc066f30 0x7f82fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:07.535 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.531+0000 7f830f7fe700 1 --2- 172.21.15.99:0/1027021451 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f82fc066f30 0x7f82fc0693e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8304007fd0 tx=0x7f8304009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:07.535 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.535+0000 7f82f7fff700 1 -- 172.21.15.99:0/1027021451 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8300051e00 con 0x7f83100b46f0 2023-10-08T00:12:07.716 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:07.715+0000 7f83163fb700 1 -- 172.21.15.99:0/1027021451 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f831004c3c0 con 0x7f83100b46f0 2023-10-08T00:12:08.412 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.412+0000 7f82f7fff700 1 -- 172.21.15.99:0/1027021451 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v42) v1 ==== 138+0+0 (secure 0 0 0) 0x7f83000210b0 con 0x7f83100b46f0 2023-10-08T00:12:08.416 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.415+0000 7f83163fb700 1 -- 172.21.15.99:0/1027021451 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f82fc066f30 msgr2=0x7f82fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:08.416 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.415+0000 7f83163fb700 1 --2- 172.21.15.99:0/1027021451 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f82fc066f30 0x7f82fc0693e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8304007fd0 tx=0x7f8304009040 comp rx=0 tx=0).stop 2023-10-08T00:12:08.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.415+0000 7f83163fb700 1 -- 172.21.15.99:0/1027021451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83100b46f0 msgr2=0x7f831014a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:08.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.415+0000 7f83163fb700 1 --2- 172.21.15.99:0/1027021451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83100b46f0 0x7f831014a3e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f830000ea20 tx=0x7f8300009120 comp rx=0 tx=0).stop 2023-10-08T00:12:08.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.415+0000 7f83163fb700 1 -- 172.21.15.99:0/1027021451 shutdown_connections 2023-10-08T00:12:08.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.415+0000 7f83163fb700 1 --2- 172.21.15.99:0/1027021451 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f82fc066f30 0x7f82fc0693e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:08.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.415+0000 7f83163fb700 1 --2- 172.21.15.99:0/1027021451 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8310159ef0 0x7f8310069dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:08.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.415+0000 7f83163fb700 1 --2- 172.21.15.99:0/1027021451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83100b46f0 0x7f831014a3e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:08.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.415+0000 7f83163fb700 1 --2- 172.21.15.99:0/1027021451 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83100b6a10 0x7f831009b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:08.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.416+0000 7f83163fb700 1 -- 172.21.15.99:0/1027021451 >> 172.21.15.99:0/1027021451 conn(0x7f831009a0a0 msgr2=0x7f8310157ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:08.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.416+0000 7f83163fb700 1 -- 172.21.15.99:0/1027021451 shutdown_connections 2023-10-08T00:12:08.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.416+0000 7f83163fb700 1 -- 172.21.15.99:0/1027021451 wait complete. 2023-10-08T00:12:08.418 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' created 2023-10-08T00:12:08.430 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T00:12:08.552 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.551+0000 7f2e6a7a4700 1 Processor -- start 2023-10-08T00:12:08.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.551+0000 7f2e6a7a4700 1 -- start start 2023-10-08T00:12:08.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.552+0000 7f2e6a7a4700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2e641542a0 0x7f2e640b65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:08.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.552+0000 7f2e6a7a4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e640cd630 0x7f2e6406a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:08.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.552+0000 7f2e6a7a4700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2e6406a830 0x7f2e6415d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:08.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.552+0000 7f2e6a7a4700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e6415b2e0 con 0x7f2e6406a830 2023-10-08T00:12:08.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.552+0000 7f2e6a7a4700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e6406ac70 con 0x7f2e641542a0 2023-10-08T00:12:08.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.552+0000 7f2e6a7a4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6406b0a0 con 0x7f2e640cd630 2023-10-08T00:12:08.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.552+0000 7f2e69fa3700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2e6406a830 0x7f2e6415d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:08.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.552+0000 7f2e69fa3700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2e6406a830 0x7f2e6415d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41184/0 (socket says 172.21.15.99:41184) 2023-10-08T00:12:08.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.552+0000 7f2e69fa3700 1 -- 172.21.15.99:0/1363778466 learned_addr learned my addr 172.21.15.99:0/1363778466 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:08.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.552+0000 7f2e69fa3700 1 -- 172.21.15.99:0/1363778466 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e640cd630 msgr2=0x7f2e6406a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:08.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.552+0000 7f2e697a2700 1 --2- 172.21.15.99:0/1363778466 >> v2:172.21.15.191:3300/0 conn(0x7f2e641542a0 0x7f2e640b65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:08.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.552+0000 7f2e68fa1700 1 --2- 172.21.15.99:0/1363778466 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e640cd630 0x7f2e6406a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:08.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.553+0000 7f2e69fa3700 1 --2- 172.21.15.99:0/1363778466 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e640cd630 0x7f2e6406a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:08.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.553+0000 7f2e69fa3700 1 -- 172.21.15.99:0/1363778466 >> v2:172.21.15.191:3300/0 conn(0x7f2e641542a0 msgr2=0x7f2e640b65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:08.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.553+0000 7f2e69fa3700 1 --2- 172.21.15.99:0/1363778466 >> v2:172.21.15.191:3300/0 conn(0x7f2e641542a0 0x7f2e640b65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:08.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.553+0000 7f2e69fa3700 1 -- 172.21.15.99:0/1363778466 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e6415ad20 con 0x7f2e6406a830 2023-10-08T00:12:08.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.553+0000 7f2e697a2700 1 --2- 172.21.15.99:0/1363778466 >> v2:172.21.15.191:3300/0 conn(0x7f2e641542a0 0x7f2e640b65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:12:08.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.553+0000 7f2e69fa3700 1 --2- 172.21.15.99:0/1363778466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e6406a830 0x7f2e6415d8a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2e640d1c50 tx=0x7f2e5c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18f034d8a07aacf6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:08.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.553+0000 7f2e5bfff700 1 -- 172.21.15.99:0/1363778466 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c00e070 con 0x7f2e6406a830 2023-10-08T00:12:08.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.553+0000 7f2e5bfff700 1 -- 172.21.15.99:0/1363778466 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e5c00ba50 con 0x7f2e6406a830 2023-10-08T00:12:08.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.553+0000 7f2e5bfff700 1 -- 172.21.15.99:0/1363778466 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c0092e0 con 0x7f2e6406a830 2023-10-08T00:12:08.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.553+0000 7f2e6a7a4700 1 -- 172.21.15.99:0/1363778466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e6406a830 msgr2=0x7f2e6415d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:08.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.553+0000 7f2e6a7a4700 1 --2- 172.21.15.99:0/1363778466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e6406a830 0x7f2e6415d8a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2e640d1c50 tx=0x7f2e5c009840 comp rx=0 tx=0).stop 2023-10-08T00:12:08.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.553+0000 7f2e6a7a4700 1 -- 172.21.15.99:0/1363778466 shutdown_connections 2023-10-08T00:12:08.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.553+0000 7f2e6a7a4700 1 --2- 172.21.15.99:0/1363778466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e6406a830 0x7f2e6415d8a0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:08.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.553+0000 7f2e6a7a4700 1 --2- 172.21.15.99:0/1363778466 >> v2:172.21.15.191:3300/0 conn(0x7f2e641542a0 0x7f2e640b65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:08.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.553+0000 7f2e6a7a4700 1 --2- 172.21.15.99:0/1363778466 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e640cd630 0x7f2e6406a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:08.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.553+0000 7f2e6a7a4700 1 -- 172.21.15.99:0/1363778466 >> 172.21.15.99:0/1363778466 conn(0x7f2e64068cb0 msgr2=0x7f2e640690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:08.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.553+0000 7f2e6a7a4700 1 -- 172.21.15.99:0/1363778466 shutdown_connections 2023-10-08T00:12:08.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.553+0000 7f2e6a7a4700 1 -- 172.21.15.99:0/1363778466 wait complete. 2023-10-08T00:12:08.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.554+0000 7f2e6a7a4700 1 Processor -- start 2023-10-08T00:12:08.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.555+0000 7f2e6a7a4700 1 -- start start 2023-10-08T00:12:08.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.555+0000 7f2e6a7a4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e6406a830 0x7f2e64145630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:08.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.555+0000 7f2e6a7a4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e640cd630 0x7f2e64157dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:08.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.555+0000 7f2e6a7a4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e641542a0 0x7f2e64158310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:08.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.555+0000 7f2e6a7a4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6416ab60 con 0x7f2e640cd630 2023-10-08T00:12:08.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.555+0000 7f2e6a7a4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6416b9c0 con 0x7f2e6406a830 2023-10-08T00:12:08.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.555+0000 7f2e6a7a4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6416adb0 con 0x7f2e641542a0 2023-10-08T00:12:08.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.555+0000 7f2e68fa1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e640cd630 0x7f2e64157dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:08.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.556+0000 7f2e68fa1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e640cd630 0x7f2e64157dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41190/0 (socket says 172.21.15.99:41190) 2023-10-08T00:12:08.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.556+0000 7f2e68fa1700 1 -- 172.21.15.99:0/3636479071 learned_addr learned my addr 172.21.15.99:0/3636479071 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:08.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.556+0000 7f2e697a2700 1 --2- 172.21.15.99:0/3636479071 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e6406a830 0x7f2e64145630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:08.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.556+0000 7f2e69fa3700 1 --2- 172.21.15.99:0/3636479071 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e641542a0 0x7f2e64158310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:08.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.556+0000 7f2e68fa1700 1 -- 172.21.15.99:0/3636479071 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e641542a0 msgr2=0x7f2e64158310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:08.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.556+0000 7f2e68fa1700 1 --2- 172.21.15.99:0/3636479071 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e641542a0 0x7f2e64158310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:08.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.556+0000 7f2e68fa1700 1 -- 172.21.15.99:0/3636479071 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e6406a830 msgr2=0x7f2e64145630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:08.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.556+0000 7f2e68fa1700 1 --2- 172.21.15.99:0/3636479071 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e6406a830 0x7f2e64145630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:08.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.556+0000 7f2e68fa1700 1 -- 172.21.15.99:0/3636479071 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e5c009ae0 con 0x7f2e640cd630 2023-10-08T00:12:08.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.556+0000 7f2e69fa3700 1 --2- 172.21.15.99:0/3636479071 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e641542a0 0x7f2e64158310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:12:08.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.556+0000 7f2e68fa1700 1 --2- 172.21.15.99:0/3636479071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e640cd630 0x7f2e64157dd0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2e5400c7d0 tx=0x7f2e54008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:08.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.556+0000 7f2e597fa700 1 -- 172.21.15.99:0/3636479071 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e54011600 con 0x7f2e640cd630 2023-10-08T00:12:08.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.556+0000 7f2e597fa700 1 -- 172.21.15.99:0/3636479071 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e54011760 con 0x7f2e640cd630 2023-10-08T00:12:08.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.556+0000 7f2e6a7a4700 1 -- 172.21.15.99:0/3636479071 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e6416b090 con 0x7f2e640cd630 2023-10-08T00:12:08.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.556+0000 7f2e597fa700 1 -- 172.21.15.99:0/3636479071 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e54011b20 con 0x7f2e640cd630 2023-10-08T00:12:08.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.556+0000 7f2e6a7a4700 1 -- 172.21.15.99:0/3636479071 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e6416b650 con 0x7f2e640cd630 2023-10-08T00:12:08.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.557+0000 7f2e6a7a4700 1 -- 172.21.15.99:0/3636479071 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e6416bb30 con 0x7f2e640cd630 2023-10-08T00:12:08.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.557+0000 7f2e597fa700 1 -- 172.21.15.99:0/3636479071 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2e54018350 con 0x7f2e640cd630 2023-10-08T00:12:08.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.558+0000 7f2e597fa700 1 --2- 172.21.15.99:0/3636479071 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2e48066f30 0x7f2e480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:08.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.558+0000 7f2e597fa700 1 -- 172.21.15.99:0/3636479071 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f2e54084220 con 0x7f2e640cd630 2023-10-08T00:12:08.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.558+0000 7f2e697a2700 1 --2- 172.21.15.99:0/3636479071 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2e48066f30 0x7f2e480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:08.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.558+0000 7f2e697a2700 1 --2- 172.21.15.99:0/3636479071 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2e48066f30 0x7f2e480693e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2e6000ad40 tx=0x7f2e60008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:08.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.562+0000 7f2e597fa700 1 -- 172.21.15.99:0/3636479071 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2e54051670 con 0x7f2e640cd630 2023-10-08T00:12:08.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:08.744+0000 7f2e6a7a4700 1 -- 172.21.15.99:0/3636479071 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f2e6404bb80 con 0x7f2e640cd630 2023-10-08T00:12:09.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.417+0000 7f2e597fa700 1 -- 172.21.15.99:0/3636479071 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v43) v1 ==== 170+0+0 (secure 0 0 0) 0x7f2e54054c90 con 0x7f2e640cd630 2023-10-08T00:12:09.426 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.425+0000 7f2e6a7a4700 1 -- 172.21.15.99:0/3636479071 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2e48066f30 msgr2=0x7f2e480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:09.426 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.425+0000 7f2e6a7a4700 1 --2- 172.21.15.99:0/3636479071 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2e48066f30 0x7f2e480693e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2e6000ad40 tx=0x7f2e60008040 comp rx=0 tx=0).stop 2023-10-08T00:12:09.426 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.425+0000 7f2e6a7a4700 1 -- 172.21.15.99:0/3636479071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e640cd630 msgr2=0x7f2e64157dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:09.426 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.425+0000 7f2e6a7a4700 1 --2- 172.21.15.99:0/3636479071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e640cd630 0x7f2e64157dd0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2e5400c7d0 tx=0x7f2e54008040 comp rx=0 tx=0).stop 2023-10-08T00:12:09.427 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.426+0000 7f2e6a7a4700 1 -- 172.21.15.99:0/3636479071 shutdown_connections 2023-10-08T00:12:09.427 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.426+0000 7f2e6a7a4700 1 --2- 172.21.15.99:0/3636479071 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2e48066f30 0x7f2e480693e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:09.427 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.426+0000 7f2e6a7a4700 1 --2- 172.21.15.99:0/3636479071 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e6406a830 0x7f2e64145630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:09.427 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.426+0000 7f2e6a7a4700 1 --2- 172.21.15.99:0/3636479071 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e641542a0 0x7f2e64158310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:09.427 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.426+0000 7f2e6a7a4700 1 --2- 172.21.15.99:0/3636479071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e640cd630 0x7f2e64157dd0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:09.427 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.426+0000 7f2e6a7a4700 1 -- 172.21.15.99:0/3636479071 >> 172.21.15.99:0/3636479071 conn(0x7f2e64068cb0 msgr2=0x7f2e6415f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:09.427 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.426+0000 7f2e6a7a4700 1 -- 172.21.15.99:0/3636479071 shutdown_connections 2023-10-08T00:12:09.427 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.427+0000 7f2e6a7a4700 1 -- 172.21.15.99:0/3636479071 wait complete. 2023-10-08T00:12:09.429 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' created 2023-10-08T00:12:09.443 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T00:12:09.563 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.562+0000 7efcc3461700 1 Processor -- start 2023-10-08T00:12:09.563 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.562+0000 7efcc3461700 1 -- start start 2023-10-08T00:12:09.565 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.563+0000 7efcc3461700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7efcbc06ea00 0x7efcbc0a9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:09.566 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.563+0000 7efcc3461700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7efcbc156d90 0x7efcbc0cbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:09.566 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.563+0000 7efcc3461700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcbc0cdb90 con 0x7efcbc1569c0 2023-10-08T00:12:09.566 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.563+0000 7efcc3461700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcbc0c78c0 con 0x7efcbc06ea00 2023-10-08T00:12:09.566 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.563+0000 7efcc3461700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcbc0c7ba0 con 0x7efcbc156d90 2023-10-08T00:12:09.566 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.563+0000 7efcc09fc700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7efcbc06ea00 0x7efcbc0a9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:09.566 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.563+0000 7efcc11fd700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7efcbc1569c0 0x7efcbc06e2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56534/0 (socket says 172.21.15.99:56534) 2023-10-08T00:12:09.566 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.563+0000 7efcc11fd700 1 -- 172.21.15.99:0/2351406472 learned_addr learned my addr 172.21.15.99:0/2351406472 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:12:09.566 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.564+0000 7efcc19fe700 1 --2- 172.21.15.99:0/2351406472 >> v2:172.21.15.191:3300/0 conn(0x7efcbc156d90 0x7efcbc0cbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:09.566 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.564+0000 7efcc09fc700 1 -- 172.21.15.99:0/2351406472 >> v2:172.21.15.191:3300/0 conn(0x7efcbc156d90 msgr2=0x7efcbc0cbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:09.566 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.564+0000 7efcc09fc700 1 --2- 172.21.15.99:0/2351406472 >> v2:172.21.15.191:3300/0 conn(0x7efcbc156d90 0x7efcbc0cbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:09.567 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.564+0000 7efcc09fc700 1 -- 172.21.15.99:0/2351406472 >> v1:172.21.15.99:6789/0 conn(0x7efcbc1569c0 legacy=0x7efcbc06e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:09.567 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.564+0000 7efcc09fc700 1 -- 172.21.15.99:0/2351406472 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcbc0c7e80 con 0x7efcbc06ea00 2023-10-08T00:12:09.567 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.564+0000 7efcb3fff700 1 -- 172.21.15.99:0/2351406472 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1965130202 0 0) 0x7efcbc0cdb90 con 0x7efcbc1569c0 2023-10-08T00:12:09.567 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.564+0000 7efcc09fc700 1 --2- 172.21.15.99:0/2351406472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcbc06ea00 0x7efcbc0a9ff0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7efcac0098e0 tx=0x7efcac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6445ae48a455298e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:09.567 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.564+0000 7efcb3fff700 1 -- 172.21.15.99:0/2351406472 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcac009e80 con 0x7efcbc06ea00 2023-10-08T00:12:09.567 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.564+0000 7efcb3fff700 1 -- 172.21.15.99:0/2351406472 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcac00b820 con 0x7efcbc06ea00 2023-10-08T00:12:09.567 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.564+0000 7efcb3fff700 1 -- 172.21.15.99:0/2351406472 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcac00bba0 con 0x7efcbc06ea00 2023-10-08T00:12:09.567 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.565+0000 7efcc3461700 1 -- 172.21.15.99:0/2351406472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcbc06ea00 msgr2=0x7efcbc0a9ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:09.567 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.565+0000 7efcc3461700 1 --2- 172.21.15.99:0/2351406472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcbc06ea00 0x7efcbc0a9ff0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7efcac0098e0 tx=0x7efcac0091a0 comp rx=0 tx=0).stop 2023-10-08T00:12:09.567 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.565+0000 7efcc3461700 1 -- 172.21.15.99:0/2351406472 shutdown_connections 2023-10-08T00:12:09.567 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.565+0000 7efcc3461700 1 --2- 172.21.15.99:0/2351406472 >> v2:172.21.15.191:3300/0 conn(0x7efcbc156d90 0x7efcbc0cbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:09.567 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.565+0000 7efcc3461700 1 --2- 172.21.15.99:0/2351406472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcbc06ea00 0x7efcbc0a9ff0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:09.567 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.565+0000 7efcc3461700 1 -- 172.21.15.99:0/2351406472 >> 172.21.15.99:0/2351406472 conn(0x7efcbc068170 msgr2=0x7efcbc0cd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:09.568 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.565+0000 7efcc3461700 1 -- 172.21.15.99:0/2351406472 shutdown_connections 2023-10-08T00:12:09.568 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.565+0000 7efcc3461700 1 -- 172.21.15.99:0/2351406472 wait complete. 2023-10-08T00:12:09.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.566+0000 7efcc3461700 1 Processor -- start 2023-10-08T00:12:09.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.566+0000 7efcc3461700 1 -- start start 2023-10-08T00:12:09.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.567+0000 7efcc3461700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efcbc06ea00 0x7efcbc151130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:09.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.567+0000 7efcc3461700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efcbc1569c0 0x7efcbc097e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:09.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.567+0000 7efcc3461700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcbc156d90 0x7efcbc099800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:09.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.567+0000 7efcc3461700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcbc1520e0 con 0x7efcbc156d90 2023-10-08T00:12:09.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.567+0000 7efcc3461700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcbc0b06b0 con 0x7efcbc1569c0 2023-10-08T00:12:09.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.567+0000 7efcc3461700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcbc098370 con 0x7efcbc06ea00 2023-10-08T00:12:09.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.567+0000 7efcc19fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcbc156d90 0x7efcbc099800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:09.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.567+0000 7efcc19fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcbc156d90 0x7efcbc099800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41202/0 (socket says 172.21.15.99:41202) 2023-10-08T00:12:09.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.567+0000 7efcc19fe700 1 -- 172.21.15.99:0/807101091 learned_addr learned my addr 172.21.15.99:0/807101091 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:09.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.567+0000 7efcc11fd700 1 --2- 172.21.15.99:0/807101091 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efcbc06ea00 0x7efcbc151130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:09.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.567+0000 7efcc19fe700 1 -- 172.21.15.99:0/807101091 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efcbc06ea00 msgr2=0x7efcbc151130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:09.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.567+0000 7efcc09fc700 1 --2- 172.21.15.99:0/807101091 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efcbc1569c0 0x7efcbc097e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:09.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.567+0000 7efcc19fe700 1 --2- 172.21.15.99:0/807101091 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efcbc06ea00 0x7efcbc151130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:09.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.567+0000 7efcc19fe700 1 -- 172.21.15.99:0/807101091 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efcbc1569c0 msgr2=0x7efcbc097e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:09.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.567+0000 7efcc19fe700 1 --2- 172.21.15.99:0/807101091 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efcbc1569c0 0x7efcbc097e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:09.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.567+0000 7efcc19fe700 1 -- 172.21.15.99:0/807101091 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcac009420 con 0x7efcbc156d90 2023-10-08T00:12:09.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.567+0000 7efcc11fd700 1 --2- 172.21.15.99:0/807101091 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efcbc06ea00 0x7efcbc151130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:12:09.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.567+0000 7efcc09fc700 1 --2- 172.21.15.99:0/807101091 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efcbc1569c0 0x7efcbc097e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:12:09.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.567+0000 7efcc19fe700 1 --2- 172.21.15.99:0/807101091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcbc156d90 0x7efcbc099800 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7efcb400d890 tx=0x7efcb4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:09.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.567+0000 7efcb17fa700 1 -- 172.21.15.99:0/807101091 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb40125f0 con 0x7efcbc156d90 2023-10-08T00:12:09.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.567+0000 7efcb17fa700 1 -- 172.21.15.99:0/807101091 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcb4012770 con 0x7efcbc156d90 2023-10-08T00:12:09.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.567+0000 7efcc3461700 1 -- 172.21.15.99:0/807101091 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcbc151840 con 0x7efcbc156d90 2023-10-08T00:12:09.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.567+0000 7efcb17fa700 1 -- 172.21.15.99:0/807101091 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb4012b30 con 0x7efcbc156d90 2023-10-08T00:12:09.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.568+0000 7efcc3461700 1 -- 172.21.15.99:0/807101091 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcbc151e00 con 0x7efcbc156d90 2023-10-08T00:12:09.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.568+0000 7efcc3461700 1 -- 172.21.15.99:0/807101091 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcbc04c3c0 con 0x7efcbc156d90 2023-10-08T00:12:09.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.569+0000 7efcb17fa700 1 -- 172.21.15.99:0/807101091 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efcb401e460 con 0x7efcbc156d90 2023-10-08T00:12:09.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.569+0000 7efcb17fa700 1 --2- 172.21.15.99:0/807101091 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efca0066f30 0x7efca00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:09.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.569+0000 7efcb17fa700 1 -- 172.21.15.99:0/807101091 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5605+0+0 (secure 0 0 0) 0x7efcb4084b00 con 0x7efcbc156d90 2023-10-08T00:12:09.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.570+0000 7efcc11fd700 1 --2- 172.21.15.99:0/807101091 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efca0066f30 0x7efca00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:09.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.570+0000 7efcc11fd700 1 --2- 172.21.15.99:0/807101091 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efca0066f30 0x7efca00693e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efcb800d270 tx=0x7efcb8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:09.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.574+0000 7efcb17fa700 1 -- 172.21.15.99:0/807101091 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efcb4051dd0 con 0x7efcbc156d90 2023-10-08T00:12:09.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:09.802+0000 7efcc3461700 1 -- 172.21.15.99:0/807101091 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7efcbc0b07f0 con 0x7efcbc156d90 2023-10-08T00:12:10.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.433+0000 7efcb17fa700 1 -- 172.21.15.99:0/807101091 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v21) v1 ==== 172+0+0 (secure 0 0 0) 0x7efcb40553f0 con 0x7efcbc156d90 2023-10-08T00:12:10.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.435+0000 7efcc3461700 1 -- 172.21.15.99:0/807101091 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efca0066f30 msgr2=0x7efca00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:10.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.435+0000 7efcc3461700 1 --2- 172.21.15.99:0/807101091 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efca0066f30 0x7efca00693e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efcb800d270 tx=0x7efcb8009040 comp rx=0 tx=0).stop 2023-10-08T00:12:10.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.435+0000 7efcc3461700 1 -- 172.21.15.99:0/807101091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcbc156d90 msgr2=0x7efcbc099800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:10.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.435+0000 7efcc3461700 1 --2- 172.21.15.99:0/807101091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcbc156d90 0x7efcbc099800 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7efcb400d890 tx=0x7efcb4008040 comp rx=0 tx=0).stop 2023-10-08T00:12:10.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.436+0000 7efcc3461700 1 -- 172.21.15.99:0/807101091 shutdown_connections 2023-10-08T00:12:10.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.436+0000 7efcc3461700 1 --2- 172.21.15.99:0/807101091 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efca0066f30 0x7efca00693e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:10.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.436+0000 7efcc3461700 1 --2- 172.21.15.99:0/807101091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcbc156d90 0x7efcbc099800 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:10.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.436+0000 7efcc3461700 1 --2- 172.21.15.99:0/807101091 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efcbc06ea00 0x7efcbc151130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:10.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.436+0000 7efcc3461700 1 --2- 172.21.15.99:0/807101091 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efcbc1569c0 0x7efcbc097e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:10.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.436+0000 7efcc3461700 1 -- 172.21.15.99:0/807101091 >> 172.21.15.99:0/807101091 conn(0x7efcbc068170 msgr2=0x7efcbc153820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:10.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.436+0000 7efcc3461700 1 -- 172.21.15.99:0/807101091 shutdown_connections 2023-10-08T00:12:10.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.436+0000 7efcc3461700 1 -- 172.21.15.99:0/807101091 wait complete. 2023-10-08T00:12:10.438 INFO:teuthology.orchestra.run.smithi099.stderr:new fs with metadata pool 6 and data pool 7 2023-10-08T00:12:10.450 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:12:10.564 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.563+0000 7fc6ef6f7700 1 Processor -- start 2023-10-08T00:12:10.564 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.563+0000 7fc6ef6f7700 1 -- start start 2023-10-08T00:12:10.567 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.564+0000 7fc6ef6f7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc6e815a620 0x7fc6e815d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:10.568 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.564+0000 7fc6ef6f7700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6e815cbb0 con 0x7fc6e815d580 2023-10-08T00:12:10.568 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.564+0000 7fc6ef6f7700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6e815bfa0 con 0x7fc6e8157870 2023-10-08T00:12:10.568 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.564+0000 7fc6ef6f7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e815bdb0 con 0x7fc6e815a620 2023-10-08T00:12:10.568 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.564+0000 7fc6edc94700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fc6e815d580 0x7fc6e8158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56544/0 (socket says 172.21.15.99:56544) 2023-10-08T00:12:10.568 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.564+0000 7fc6edc94700 1 -- 172.21.15.99:0/168320289 learned_addr learned my addr 172.21.15.99:0/168320289 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:12:10.568 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.564+0000 7fc6dffff700 1 -- 172.21.15.99:0/168320289 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 255491934 0 0) 0x7fc6e815cbb0 con 0x7fc6e815d580 2023-10-08T00:12:10.568 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.564+0000 7fc6ecc92700 1 --2- 172.21.15.99:0/168320289 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc6e815a620 0x7fc6e815d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:10.568 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.564+0000 7fc6dffff700 1 -- 172.21.15.99:0/168320289 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6d4003570 con 0x7fc6e815d580 2023-10-08T00:12:10.568 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.565+0000 7fc6dffff700 1 -- 172.21.15.99:0/168320289 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1404196007 0 0) 0x7fc6d4003570 con 0x7fc6e815d580 2023-10-08T00:12:10.568 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.565+0000 7fc6dffff700 1 -- 172.21.15.99:0/168320289 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc6e815a620 msgr2=0x7fc6e815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:10.568 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.565+0000 7fc6dffff700 1 --2- 172.21.15.99:0/168320289 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc6e815a620 0x7fc6e815d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:10.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.565+0000 7fc6dffff700 1 -- 172.21.15.99:0/168320289 >> v1:172.21.15.191:6789/0 conn(0x7fc6e8157870 legacy=0x7fc6e815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:10.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.565+0000 7fc6dffff700 1 -- 172.21.15.99:0/168320289 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e815c280 con 0x7fc6e815d580 2023-10-08T00:12:10.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.565+0000 7fc6dffff700 1 -- 172.21.15.99:0/168320289 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7fc6e0001470 con 0x7fc6e815d580 2023-10-08T00:12:10.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.565+0000 7fc6dffff700 1 -- 172.21.15.99:0/168320289 >> v1:172.21.15.99:6789/0 conn(0x7fc6e815d580 legacy=0x7fc6e8158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:10.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.565+0000 7fc6dffff700 1 --2- 172.21.15.99:0/168320289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6d4003a90 0x7fc6d4003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:10.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.565+0000 7fc6dffff700 1 --2- 172.21.15.99:0/168320289 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc6e815a620 0x7fc6d4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:10.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.565+0000 7fc6dffff700 1 --2- 172.21.15.99:0/168320289 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc6d4004d50 0x7fc6d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:10.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.565+0000 7fc6dffff700 1 -- 172.21.15.99:0/168320289 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d400bb70 con 0x7fc6d4003a90 2023-10-08T00:12:10.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.565+0000 7fc6dffff700 1 -- 172.21.15.99:0/168320289 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d400bce0 con 0x7fc6d4004d50 2023-10-08T00:12:10.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.565+0000 7fc6dffff700 1 -- 172.21.15.99:0/168320289 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d400bea0 con 0x7fc6e815a620 2023-10-08T00:12:10.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.565+0000 7fc6ed493700 1 --2- 172.21.15.99:0/168320289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6d4003a90 0x7fc6d4003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:10.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.565+0000 7fc6ecc92700 1 --2- 172.21.15.99:0/168320289 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc6e815a620 0x7fc6d4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:10.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.565+0000 7fc6edc94700 1 --2- 172.21.15.99:0/168320289 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc6d4004d50 0x7fc6d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:10.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.565+0000 7fc6ed493700 1 -- 172.21.15.99:0/168320289 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc6e815a620 msgr2=0x7fc6d4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:10.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.565+0000 7fc6ed493700 1 --2- 172.21.15.99:0/168320289 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc6e815a620 0x7fc6d4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:10.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.565+0000 7fc6ed493700 1 -- 172.21.15.99:0/168320289 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc6d4004d50 msgr2=0x7fc6d40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:10.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.565+0000 7fc6ed493700 1 --2- 172.21.15.99:0/168320289 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc6d4004d50 0x7fc6d40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:10.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.565+0000 7fc6ed493700 1 -- 172.21.15.99:0/168320289 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d4003570 con 0x7fc6d4003a90 2023-10-08T00:12:10.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.565+0000 7fc6ed493700 1 --2- 172.21.15.99:0/168320289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6d4003a90 0x7fc6d4003ea0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4001fc0 tx=0x7fc6e4001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95909df3bca3e97e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:10.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.565+0000 7fc6dffff700 1 -- 172.21.15.99:0/168320289 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4001950 con 0x7fc6d4003a90 2023-10-08T00:12:10.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.566+0000 7fc6dffff700 1 -- 172.21.15.99:0/168320289 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e400c8b0 con 0x7fc6d4003a90 2023-10-08T00:12:10.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.566+0000 7fc6dffff700 1 -- 172.21.15.99:0/168320289 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e400cc50 con 0x7fc6d4003a90 2023-10-08T00:12:10.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.566+0000 7fc6ef6f7700 1 -- 172.21.15.99:0/168320289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6d4003a90 msgr2=0x7fc6d4003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:10.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.566+0000 7fc6ef6f7700 1 --2- 172.21.15.99:0/168320289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6d4003a90 0x7fc6d4003ea0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4001fc0 tx=0x7fc6e4001640 comp rx=0 tx=0).stop 2023-10-08T00:12:10.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.566+0000 7fc6edc94700 1 -- 172.21.15.99:0/168320289 reap_dead start 2023-10-08T00:12:10.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.566+0000 7fc6ef6f7700 1 -- 172.21.15.99:0/168320289 shutdown_connections 2023-10-08T00:12:10.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.566+0000 7fc6ef6f7700 1 -- 172.21.15.99:0/168320289 >> 172.21.15.99:0/168320289 conn(0x7fc6e8068ce0 msgr2=0x7fc6e80690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:10.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.566+0000 7fc6ef6f7700 1 -- 172.21.15.99:0/168320289 shutdown_connections 2023-10-08T00:12:10.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.566+0000 7fc6ef6f7700 1 -- 172.21.15.99:0/168320289 wait complete. 2023-10-08T00:12:10.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.567+0000 7fc6ef6f7700 1 Processor -- start 2023-10-08T00:12:10.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.567+0000 7fc6ef6f7700 1 -- start start 2023-10-08T00:12:10.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.568+0000 7fc6ef6f7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6d4003a90 0x7fc6e8154d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:10.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.568+0000 7fc6ef6f7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc6e80b0610 0x7fc6e813e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:10.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.568+0000 7fc6ef6f7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc6e80b1460 0x7fc6e80b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:10.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.568+0000 7fc6ef6f7700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e80b1e10 con 0x7fc6d4003a90 2023-10-08T00:12:10.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.568+0000 7fc6ef6f7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e81548e0 con 0x7fc6e80b1460 2023-10-08T00:12:10.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.568+0000 7fc6ef6f7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e80b0a80 con 0x7fc6e80b0610 2023-10-08T00:12:10.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.568+0000 7fc6ed493700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6d4003a90 0x7fc6e8154d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:10.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.568+0000 7fc6ed493700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6d4003a90 0x7fc6e8154d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41210/0 (socket says 172.21.15.99:41210) 2023-10-08T00:12:10.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.568+0000 7fc6ed493700 1 -- 172.21.15.99:0/4010911642 learned_addr learned my addr 172.21.15.99:0/4010911642 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:10.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.568+0000 7fc6ecc92700 1 --2- 172.21.15.99:0/4010911642 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc6e80b0610 0x7fc6e813e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:10.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.568+0000 7fc6edc94700 1 --2- 172.21.15.99:0/4010911642 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc6e80b1460 0x7fc6e80b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:10.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.568+0000 7fc6ed493700 1 -- 172.21.15.99:0/4010911642 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc6e80b0610 msgr2=0x7fc6e813e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:10.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.568+0000 7fc6ed493700 1 --2- 172.21.15.99:0/4010911642 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc6e80b0610 0x7fc6e813e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:10.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.568+0000 7fc6ed493700 1 -- 172.21.15.99:0/4010911642 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc6e80b1460 msgr2=0x7fc6e80b18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:10.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.568+0000 7fc6ed493700 1 --2- 172.21.15.99:0/4010911642 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc6e80b1460 0x7fc6e80b18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:10.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.568+0000 7fc6ed493700 1 -- 172.21.15.99:0/4010911642 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e8153cd0 con 0x7fc6d4003a90 2023-10-08T00:12:10.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.568+0000 7fc6ed493700 1 --2- 172.21.15.99:0/4010911642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6d4003a90 0x7fc6e8154d70 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc6e400c6a0 tx=0x7fc6e4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:10.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.569+0000 7fc6dd7fa700 1 -- 172.21.15.99:0/4010911642 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e400c6d0 con 0x7fc6d4003a90 2023-10-08T00:12:10.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.569+0000 7fc6dd7fa700 1 -- 172.21.15.99:0/4010911642 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e400ca20 con 0x7fc6d4003a90 2023-10-08T00:12:10.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.569+0000 7fc6ef6f7700 1 -- 172.21.15.99:0/4010911642 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6e8153fb0 con 0x7fc6d4003a90 2023-10-08T00:12:10.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.569+0000 7fc6dd7fa700 1 -- 172.21.15.99:0/4010911642 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e401f410 con 0x7fc6d4003a90 2023-10-08T00:12:10.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.569+0000 7fc6ef6f7700 1 -- 172.21.15.99:0/4010911642 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6e8154570 con 0x7fc6d4003a90 2023-10-08T00:12:10.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.570+0000 7fc6ef6f7700 1 -- 172.21.15.99:0/4010911642 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6e81552b0 con 0x7fc6d4003a90 2023-10-08T00:12:10.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.570+0000 7fc6dd7fa700 1 -- 172.21.15.99:0/4010911642 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc6e401f7c0 con 0x7fc6d4003a90 2023-10-08T00:12:10.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.570+0000 7fc6dd7fa700 1 --2- 172.21.15.99:0/4010911642 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc6cc066f30 0x7fc6cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:10.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.570+0000 7fc6dd7fa700 1 -- 172.21.15.99:0/4010911642 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc6e4086360 con 0x7fc6d4003a90 2023-10-08T00:12:10.577 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.571+0000 7fc6ecc92700 1 --2- 172.21.15.99:0/4010911642 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc6cc066f30 0x7fc6cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:10.577 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.571+0000 7fc6ecc92700 1 --2- 172.21.15.99:0/4010911642 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc6cc066f30 0x7fc6cc0693e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8155d20 tx=0x7fc6d80091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:10.577 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.575+0000 7fc6dd7fa700 1 -- 172.21.15.99:0/4010911642 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc6e40535d0 con 0x7fc6d4003a90 2023-10-08T00:12:10.761 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.760+0000 7fc6ef6f7700 1 -- 172.21.15.99:0/4010911642 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc6e80b1e10 con 0x7fc6d4003a90 2023-10-08T00:12:10.762 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.761+0000 7fc6dd7fa700 1 -- 172.21.15.99:0/4010911642 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+15363 (secure 0 0 0) 0x7fc6e4056bf0 con 0x7fc6d4003a90 2023-10-08T00:12:10.762 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:12:10.763 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":44,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:12:09.803503+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:12:07.733432+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-08T00:12:08.762879+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:12:10.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.763+0000 7fc6ef6f7700 1 -- 172.21.15.99:0/4010911642 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc6cc066f30 msgr2=0x7fc6cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:10.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.763+0000 7fc6ef6f7700 1 --2- 172.21.15.99:0/4010911642 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc6cc066f30 0x7fc6cc0693e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8155d20 tx=0x7fc6d80091a0 comp rx=0 tx=0).stop 2023-10-08T00:12:10.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.763+0000 7fc6ef6f7700 1 -- 172.21.15.99:0/4010911642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6d4003a90 msgr2=0x7fc6e8154d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:10.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.763+0000 7fc6ef6f7700 1 --2- 172.21.15.99:0/4010911642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6d4003a90 0x7fc6e8154d70 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc6e400c6a0 tx=0x7fc6e4015000 comp rx=0 tx=0).stop 2023-10-08T00:12:10.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.763+0000 7fc6ef6f7700 1 -- 172.21.15.99:0/4010911642 shutdown_connections 2023-10-08T00:12:10.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.763+0000 7fc6ef6f7700 1 --2- 172.21.15.99:0/4010911642 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc6cc066f30 0x7fc6cc0693e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:10.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.763+0000 7fc6ef6f7700 1 --2- 172.21.15.99:0/4010911642 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc6e80b1460 0x7fc6e80b18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:10.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.763+0000 7fc6ef6f7700 1 --2- 172.21.15.99:0/4010911642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6d4003a90 0x7fc6e8154d70 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:10.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.763+0000 7fc6ef6f7700 1 --2- 172.21.15.99:0/4010911642 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc6e80b0610 0x7fc6e813e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:10.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.763+0000 7fc6ef6f7700 1 -- 172.21.15.99:0/4010911642 >> 172.21.15.99:0/4010911642 conn(0x7fc6e8068ce0 msgr2=0x7fc6e814c210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:10.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.763+0000 7fc6ef6f7700 1 -- 172.21.15.99:0/4010911642 shutdown_connections 2023-10-08T00:12:10.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.763+0000 7fc6ef6f7700 1 -- 172.21.15.99:0/4010911642 wait complete. 2023-10-08T00:12:10.776 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:12:10.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.890+0000 7f1bcab16700 1 Processor -- start 2023-10-08T00:12:10.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.890+0000 7f1bcab16700 1 -- start start 2023-10-08T00:12:10.894 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.891+0000 7f1bcab16700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1bc4157870 0x7f1bc415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:10.894 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.891+0000 7f1bcab16700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1bc415a620 0x7f1bc415d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:10.894 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.891+0000 7f1bcab16700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1bc415d580 0x7f1bc4158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:10.894 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.891+0000 7f1bcab16700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bc415cbb0 con 0x7f1bc415d580 2023-10-08T00:12:10.894 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.891+0000 7f1bcab16700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bc415bfa0 con 0x7f1bc415a620 2023-10-08T00:12:10.894 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.891+0000 7f1bcab16700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc415c280 con 0x7f1bc4157870 2023-10-08T00:12:10.895 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.891+0000 7f1bc90b3700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1bc415d580 0x7f1bc4158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:10.895 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.891+0000 7f1bc90b3700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1bc415d580 0x7f1bc4158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41220/0 (socket says 172.21.15.99:41220) 2023-10-08T00:12:10.895 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.891+0000 7f1bc90b3700 1 -- 172.21.15.99:0/294946176 learned_addr learned my addr 172.21.15.99:0/294946176 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:10.895 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.892+0000 7f1bc88b2700 1 --2- 172.21.15.99:0/294946176 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1bc4157870 0x7f1bc415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:10.895 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.892+0000 7f1bc90b3700 1 -- 172.21.15.99:0/294946176 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1bc4157870 msgr2=0x7f1bc415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:10.895 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.892+0000 7f1bc90b3700 1 --2- 172.21.15.99:0/294946176 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1bc4157870 0x7f1bc415b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:10.895 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.892+0000 7f1bc90b3700 1 -- 172.21.15.99:0/294946176 >> v2:172.21.15.191:3300/0 conn(0x7f1bc415a620 msgr2=0x7f1bc415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:10.895 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.892+0000 7f1bc3fff700 1 --2- 172.21.15.99:0/294946176 >> v2:172.21.15.191:3300/0 conn(0x7f1bc415a620 0x7f1bc415d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:10.895 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.892+0000 7f1bc90b3700 1 --2- 172.21.15.99:0/294946176 >> v2:172.21.15.191:3300/0 conn(0x7f1bc415a620 0x7f1bc415d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:10.895 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.892+0000 7f1bc90b3700 1 -- 172.21.15.99:0/294946176 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bc415c560 con 0x7f1bc415d580 2023-10-08T00:12:10.895 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.892+0000 7f1bc90b3700 1 --2- 172.21.15.99:0/294946176 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bc415d580 0x7f1bc4158110 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1bc40897d0 tx=0x7f1bbc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa1fc3f80e7a6063 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:10.896 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.892+0000 7f1bc37fe700 1 -- 172.21.15.99:0/294946176 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc00e070 con 0x7f1bc415d580 2023-10-08T00:12:10.896 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.892+0000 7f1bc37fe700 1 -- 172.21.15.99:0/294946176 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bbc00ba50 con 0x7f1bc415d580 2023-10-08T00:12:10.896 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.892+0000 7f1bc37fe700 1 -- 172.21.15.99:0/294946176 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc0092e0 con 0x7f1bc415d580 2023-10-08T00:12:10.896 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.892+0000 7f1bcab16700 1 -- 172.21.15.99:0/294946176 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bc415d580 msgr2=0x7f1bc4158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:10.896 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.892+0000 7f1bcab16700 1 --2- 172.21.15.99:0/294946176 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bc415d580 0x7f1bc4158110 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1bc40897d0 tx=0x7f1bbc009840 comp rx=0 tx=0).stop 2023-10-08T00:12:10.896 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.892+0000 7f1bcab16700 1 -- 172.21.15.99:0/294946176 shutdown_connections 2023-10-08T00:12:10.896 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.892+0000 7f1bcab16700 1 --2- 172.21.15.99:0/294946176 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bc415d580 0x7f1bc4158110 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:10.896 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.892+0000 7f1bcab16700 1 --2- 172.21.15.99:0/294946176 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1bc4157870 0x7f1bc415b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:10.896 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.892+0000 7f1bcab16700 1 --2- 172.21.15.99:0/294946176 >> v2:172.21.15.191:3300/0 conn(0x7f1bc415a620 0x7f1bc415d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:10.896 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.892+0000 7f1bcab16700 1 -- 172.21.15.99:0/294946176 >> 172.21.15.99:0/294946176 conn(0x7f1bc4068ce0 msgr2=0x7f1bc40690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:10.896 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.892+0000 7f1bcab16700 1 -- 172.21.15.99:0/294946176 shutdown_connections 2023-10-08T00:12:10.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.893+0000 7f1bcab16700 1 -- 172.21.15.99:0/294946176 wait complete. 2023-10-08T00:12:10.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.894+0000 7f1bcab16700 1 Processor -- start 2023-10-08T00:12:10.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.894+0000 7f1bcab16700 1 -- start start 2023-10-08T00:12:10.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.894+0000 7f1bcab16700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1bc4157870 0x7f1bc40d02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:10.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.894+0000 7f1bcab16700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1bc415a620 0x7f1bc416a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:10.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.894+0000 7f1bcab16700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bc415d580 0x7f1bc416a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:10.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.894+0000 7f1bcab16700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc40b06c0 con 0x7f1bc415d580 2023-10-08T00:12:10.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.894+0000 7f1bcab16700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc414ab60 con 0x7f1bc415a620 2023-10-08T00:12:10.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.894+0000 7f1bcab16700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc40b0960 con 0x7f1bc4157870 2023-10-08T00:12:10.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.895+0000 7f1bc90b3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bc415d580 0x7f1bc416a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:10.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.895+0000 7f1bc90b3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bc415d580 0x7f1bc416a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41224/0 (socket says 172.21.15.99:41224) 2023-10-08T00:12:10.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.895+0000 7f1bc90b3700 1 -- 172.21.15.99:0/3797177302 learned_addr learned my addr 172.21.15.99:0/3797177302 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:10.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.895+0000 7f1bc88b2700 1 --2- 172.21.15.99:0/3797177302 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1bc4157870 0x7f1bc40d02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:10.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.895+0000 7f1bc3fff700 1 --2- 172.21.15.99:0/3797177302 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1bc415a620 0x7f1bc416a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:10.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.895+0000 7f1bc90b3700 1 -- 172.21.15.99:0/3797177302 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1bc4157870 msgr2=0x7f1bc40d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:10.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.895+0000 7f1bc90b3700 1 --2- 172.21.15.99:0/3797177302 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1bc4157870 0x7f1bc40d02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:10.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.895+0000 7f1bc90b3700 1 -- 172.21.15.99:0/3797177302 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1bc415a620 msgr2=0x7f1bc416a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:10.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.895+0000 7f1bc90b3700 1 --2- 172.21.15.99:0/3797177302 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1bc415a620 0x7f1bc416a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:10.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.895+0000 7f1bc90b3700 1 -- 172.21.15.99:0/3797177302 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bbc009ae0 con 0x7f1bc415d580 2023-10-08T00:12:10.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.895+0000 7f1bc3fff700 1 --2- 172.21.15.99:0/3797177302 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1bc415a620 0x7f1bc416a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:12:10.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.895+0000 7f1bc90b3700 1 --2- 172.21.15.99:0/3797177302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bc415d580 0x7f1bc416a8d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1bc406b530 tx=0x7f1bbc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:10.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.895+0000 7f1bc0ff9700 1 -- 172.21.15.99:0/3797177302 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc0032f0 con 0x7f1bc415d580 2023-10-08T00:12:10.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.895+0000 7f1bc0ff9700 1 -- 172.21.15.99:0/3797177302 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bbc008d90 con 0x7f1bc415d580 2023-10-08T00:12:10.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.895+0000 7f1bc0ff9700 1 -- 172.21.15.99:0/3797177302 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc025690 con 0x7f1bc415d580 2023-10-08T00:12:10.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.895+0000 7f1bcab16700 1 -- 172.21.15.99:0/3797177302 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bc40d0c10 con 0x7f1bc415d580 2023-10-08T00:12:10.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.895+0000 7f1bcab16700 1 -- 172.21.15.99:0/3797177302 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bc40d11d0 con 0x7f1bc415d580 2023-10-08T00:12:10.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.896+0000 7f1bcab16700 1 -- 172.21.15.99:0/3797177302 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bc404c3c0 con 0x7f1bc415d580 2023-10-08T00:12:10.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.896+0000 7f1bc0ff9700 1 -- 172.21.15.99:0/3797177302 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1bbc003040 con 0x7f1bc415d580 2023-10-08T00:12:10.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.897+0000 7f1bc0ff9700 1 --2- 172.21.15.99:0/3797177302 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1ba8066f30 0x7f1ba80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:10.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.897+0000 7f1bc0ff9700 1 -- 172.21.15.99:0/3797177302 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1bbc085c80 con 0x7f1bc415d580 2023-10-08T00:12:10.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.897+0000 7f1bc88b2700 1 --2- 172.21.15.99:0/3797177302 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1ba8066f30 0x7f1ba80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:10.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.897+0000 7f1bc88b2700 1 --2- 172.21.15.99:0/3797177302 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1ba8066f30 0x7f1ba80693e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1bc415a460 tx=0x7f1bb8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:10.903 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:10.901+0000 7f1bc0ff9700 1 -- 172.21.15.99:0/3797177302 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1bbc052fa0 con 0x7f1bc415d580 2023-10-08T00:12:11.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.085+0000 7f1bcab16700 1 -- 172.21.15.99:0/3797177302 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1bc40b0aa0 con 0x7f1bc415d580 2023-10-08T00:12:11.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.086+0000 7f1bc0ff9700 1 -- 172.21.15.99:0/3797177302 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+15363 (secure 0 0 0) 0x7f1bbc0565c0 con 0x7f1bc415d580 2023-10-08T00:12:11.087 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:12:11.087 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":44,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:12:09.803503+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:12:07.733432+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-08T00:12:08.762879+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:12:11.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.088+0000 7f1bcab16700 1 -- 172.21.15.99:0/3797177302 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1ba8066f30 msgr2=0x7f1ba80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:11.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.088+0000 7f1bcab16700 1 --2- 172.21.15.99:0/3797177302 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1ba8066f30 0x7f1ba80693e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1bc415a460 tx=0x7f1bb8008040 comp rx=0 tx=0).stop 2023-10-08T00:12:11.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.088+0000 7f1bcab16700 1 -- 172.21.15.99:0/3797177302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bc415d580 msgr2=0x7f1bc416a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:11.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.088+0000 7f1bcab16700 1 --2- 172.21.15.99:0/3797177302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bc415d580 0x7f1bc416a8d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1bc406b530 tx=0x7f1bbc018000 comp rx=0 tx=0).stop 2023-10-08T00:12:11.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.088+0000 7f1bcab16700 1 -- 172.21.15.99:0/3797177302 shutdown_connections 2023-10-08T00:12:11.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.088+0000 7f1bcab16700 1 --2- 172.21.15.99:0/3797177302 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1ba8066f30 0x7f1ba80693e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:11.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.088+0000 7f1bcab16700 1 --2- 172.21.15.99:0/3797177302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bc415d580 0x7f1bc416a8d0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:11.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.088+0000 7f1bcab16700 1 --2- 172.21.15.99:0/3797177302 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1bc4157870 0x7f1bc40d02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:11.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.088+0000 7f1bcab16700 1 --2- 172.21.15.99:0/3797177302 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1bc415a620 0x7f1bc416a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:11.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.088+0000 7f1bcab16700 1 -- 172.21.15.99:0/3797177302 >> 172.21.15.99:0/3797177302 conn(0x7f1bc4068ce0 msgr2=0x7f1bc40b4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:11.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.088+0000 7f1bcab16700 1 -- 172.21.15.99:0/3797177302 shutdown_connections 2023-10-08T00:12:11.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.089+0000 7f1bcab16700 1 -- 172.21.15.99:0/3797177302 wait complete. 2023-10-08T00:12:11.101 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T00:12:11.218 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.217+0000 7f170b59e700 1 Processor -- start 2023-10-08T00:12:11.218 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.217+0000 7f170b59e700 1 -- start start 2023-10-08T00:12:11.219 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.218+0000 7f170b59e700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f170c0cba70 0x7f170c15a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:11.219 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.218+0000 7f170b59e700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f170c15e660 con 0x7f170c0cd850 2023-10-08T00:12:11.219 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.218+0000 7f170b59e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f170c15da50 con 0x7f170c0b32f0 2023-10-08T00:12:11.219 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.218+0000 7f170b59e700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f170c15d870 con 0x7f170c0cba70 2023-10-08T00:12:11.220 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.218+0000 7f170a59c700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f170c0cd850 0x7f170c0b2be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56560/0 (socket says 172.21.15.99:56560) 2023-10-08T00:12:11.220 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.218+0000 7f170a59c700 1 -- 172.21.15.99:0/2221114300 learned_addr learned my addr 172.21.15.99:0/2221114300 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:12:11.220 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.218+0000 7f170ad9d700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f170c0cba70 0x7f170c15a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:11.220 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.218+0000 7f170ad9d700 1 -- 172.21.15.99:0/2221114300 >> v1:172.21.15.191:6789/0 conn(0x7f170c0b32f0 legacy=0x7f170c157250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:11.220 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.218+0000 7f170ad9d700 1 -- 172.21.15.99:0/2221114300 >> v1:172.21.15.99:6789/0 conn(0x7f170c0cd850 legacy=0x7f170c0b2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:11.220 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.218+0000 7f170ad9d700 1 -- 172.21.15.99:0/2221114300 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f170c15dd30 con 0x7f170c0cba70 2023-10-08T00:12:11.220 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.218+0000 7f170ad9d700 1 --2- 172.21.15.99:0/2221114300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f170c0cba70 0x7f170c15a420 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1704009e60 tx=0x7f1704009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64af2cf2bea64f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:11.220 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.219+0000 7f1709d9b700 1 -- 172.21.15.99:0/2221114300 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170400e070 con 0x7f170c0cba70 2023-10-08T00:12:11.220 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.219+0000 7f1709d9b700 1 -- 172.21.15.99:0/2221114300 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f170400bb30 con 0x7f170c0cba70 2023-10-08T00:12:11.220 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.219+0000 7f1709d9b700 1 -- 172.21.15.99:0/2221114300 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170400bed0 con 0x7f170c0cba70 2023-10-08T00:12:11.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.219+0000 7f170b59e700 1 -- 172.21.15.99:0/2221114300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f170c0cba70 msgr2=0x7f170c15a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:11.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.219+0000 7f170b59e700 1 --2- 172.21.15.99:0/2221114300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f170c0cba70 0x7f170c15a420 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1704009e60 tx=0x7f1704009780 comp rx=0 tx=0).stop 2023-10-08T00:12:11.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.219+0000 7f170b59e700 1 -- 172.21.15.99:0/2221114300 shutdown_connections 2023-10-08T00:12:11.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.219+0000 7f170b59e700 1 --2- 172.21.15.99:0/2221114300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f170c0cba70 0x7f170c15a420 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:11.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.219+0000 7f170b59e700 1 -- 172.21.15.99:0/2221114300 >> 172.21.15.99:0/2221114300 conn(0x7f170c067660 msgr2=0x7f170c067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:11.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.219+0000 7f170b59e700 1 -- 172.21.15.99:0/2221114300 shutdown_connections 2023-10-08T00:12:11.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.219+0000 7f170b59e700 1 -- 172.21.15.99:0/2221114300 wait complete. 2023-10-08T00:12:11.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.220+0000 7f170b59e700 1 Processor -- start 2023-10-08T00:12:11.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.220+0000 7f170b59e700 1 -- start start 2023-10-08T00:12:11.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.221+0000 7f170b59e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f170c0b32f0 0x7f170c16e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:11.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.221+0000 7f170b59e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f170c0cba70 0x7f170c0b59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:11.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.221+0000 7f170b59e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f170c0cd850 0x7f170c0b5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:11.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.221+0000 7f170b59e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170c16af70 con 0x7f170c0cd850 2023-10-08T00:12:11.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.221+0000 7f170b59e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170c16b250 con 0x7f170c0b32f0 2023-10-08T00:12:11.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.221+0000 7f170b59e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f170c16b530 con 0x7f170c0cba70 2023-10-08T00:12:11.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.221+0000 7f170ad9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f170c0cd850 0x7f170c0b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:11.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.221+0000 7f170ad9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f170c0cd850 0x7f170c0b5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41240/0 (socket says 172.21.15.99:41240) 2023-10-08T00:12:11.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.221+0000 7f170ad9d700 1 -- 172.21.15.99:0/3757803805 learned_addr learned my addr 172.21.15.99:0/3757803805 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:11.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.221+0000 7f1701d9b700 1 --2- 172.21.15.99:0/3757803805 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f170c0cba70 0x7f170c0b59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:11.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.221+0000 7f170a59c700 1 --2- 172.21.15.99:0/3757803805 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f170c0b32f0 0x7f170c16e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:11.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.222+0000 7f170ad9d700 1 -- 172.21.15.99:0/3757803805 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f170c0cba70 msgr2=0x7f170c0b59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:11.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.222+0000 7f170ad9d700 1 --2- 172.21.15.99:0/3757803805 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f170c0cba70 0x7f170c0b59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:11.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.222+0000 7f170ad9d700 1 -- 172.21.15.99:0/3757803805 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f170c0b32f0 msgr2=0x7f170c16e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:11.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.222+0000 7f170ad9d700 1 --2- 172.21.15.99:0/3757803805 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f170c0b32f0 0x7f170c16e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:11.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.222+0000 7f170ad9d700 1 -- 172.21.15.99:0/3757803805 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1704009a20 con 0x7f170c0cd850 2023-10-08T00:12:11.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.222+0000 7f170ad9d700 1 --2- 172.21.15.99:0/3757803805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f170c0cd850 0x7f170c0b5f00 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f170c071080 tx=0x7f1704017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:11.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.222+0000 7f1702ffd700 1 -- 172.21.15.99:0/3757803805 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170401f620 con 0x7f170c0cd850 2023-10-08T00:12:11.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.222+0000 7f1702ffd700 1 -- 172.21.15.99:0/3757803805 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f170401f7a0 con 0x7f170c0cd850 2023-10-08T00:12:11.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.222+0000 7f1702ffd700 1 -- 172.21.15.99:0/3757803805 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170401fb60 con 0x7f170c0cd850 2023-10-08T00:12:11.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.222+0000 7f170b59e700 1 -- 172.21.15.99:0/3757803805 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f170c16b810 con 0x7f170c0cd850 2023-10-08T00:12:11.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.222+0000 7f170b59e700 1 -- 172.21.15.99:0/3757803805 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f170c16ee20 con 0x7f170c0cd850 2023-10-08T00:12:11.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.223+0000 7f170b59e700 1 -- 172.21.15.99:0/3757803805 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f170c0d0220 con 0x7f170c0cd850 2023-10-08T00:12:11.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.223+0000 7f1702ffd700 1 -- 172.21.15.99:0/3757803805 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1704028450 con 0x7f170c0cd850 2023-10-08T00:12:11.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.223+0000 7f1702ffd700 1 --2- 172.21.15.99:0/3757803805 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f16ec066f30 0x7f16ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:11.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.224+0000 7f1702ffd700 1 -- 172.21.15.99:0/3757803805 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1704087060 con 0x7f170c0cd850 2023-10-08T00:12:11.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.224+0000 7f170a59c700 1 --2- 172.21.15.99:0/3757803805 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f16ec066f30 0x7f16ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:11.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.224+0000 7f170a59c700 1 --2- 172.21.15.99:0/3757803805 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f16ec066f30 0x7f16ec0693e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f16fc0058d0 tx=0x7f16fc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:11.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.228+0000 7f1702ffd700 1 -- 172.21.15.99:0/3757803805 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f170c0d0220 con 0x7f170c0cd850 2023-10-08T00:12:11.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:11.457+0000 7f170b59e700 1 -- 172.21.15.99:0/3757803805 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f170c0d0220 con 0x7f170c0cd850 2023-10-08T00:12:12.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.437+0000 7f1702ffd700 1 -- 172.21.15.99:0/3757803805 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v23) v1 ==== 118+0+0 (secure 0 0 0) 0x7f170c0d0220 con 0x7f170c0cd850 2023-10-08T00:12:12.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.439+0000 7f170b59e700 1 -- 172.21.15.99:0/3757803805 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f16ec066f30 msgr2=0x7f16ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:12.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.439+0000 7f170b59e700 1 --2- 172.21.15.99:0/3757803805 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f16ec066f30 0x7f16ec0693e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f16fc0058d0 tx=0x7f16fc009040 comp rx=0 tx=0).stop 2023-10-08T00:12:12.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.440+0000 7f170b59e700 1 -- 172.21.15.99:0/3757803805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f170c0cd850 msgr2=0x7f170c0b5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:12.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.440+0000 7f170b59e700 1 --2- 172.21.15.99:0/3757803805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f170c0cd850 0x7f170c0b5f00 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f170c071080 tx=0x7f1704017000 comp rx=0 tx=0).stop 2023-10-08T00:12:12.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.440+0000 7f170b59e700 1 -- 172.21.15.99:0/3757803805 shutdown_connections 2023-10-08T00:12:12.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.440+0000 7f170b59e700 1 --2- 172.21.15.99:0/3757803805 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f16ec066f30 0x7f16ec0693e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:12.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.440+0000 7f170b59e700 1 --2- 172.21.15.99:0/3757803805 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f170c0b32f0 0x7f170c16e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:12.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.440+0000 7f170b59e700 1 --2- 172.21.15.99:0/3757803805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f170c0cd850 0x7f170c0b5f00 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:12.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.440+0000 7f170b59e700 1 --2- 172.21.15.99:0/3757803805 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f170c0cba70 0x7f170c0b59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:12.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.440+0000 7f170b59e700 1 -- 172.21.15.99:0/3757803805 >> 172.21.15.99:0/3757803805 conn(0x7f170c067660 msgr2=0x7f170c0cbf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:12.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.440+0000 7f170b59e700 1 -- 172.21.15.99:0/3757803805 shutdown_connections 2023-10-08T00:12:12.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.440+0000 7f170b59e700 1 -- 172.21.15.99:0/3757803805 wait complete. 2023-10-08T00:12:12.454 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T00:12:12.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.575+0000 7f6922369700 1 Processor -- start 2023-10-08T00:12:12.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.575+0000 7f6922369700 1 -- start start 2023-10-08T00:12:12.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.576+0000 7f6922369700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f691c159ec0 0x7f691c0af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:12.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.576+0000 7f6922369700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f691c06acb0 0x7f691c0afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:12.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.576+0000 7f6922369700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f691c0b0790 con 0x7f691c15a290 2023-10-08T00:12:12.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.576+0000 7f6922369700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f691c0ce1f0 con 0x7f691c159ec0 2023-10-08T00:12:12.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.576+0000 7f6922369700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f691c0b0b20 con 0x7f691c06acb0 2023-10-08T00:12:12.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.576+0000 7f691bfff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f691c159ec0 0x7f691c0af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:12.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.576+0000 7f691b7fe700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f691c15a290 0x7f691c06a5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56576/0 (socket says 172.21.15.99:56576) 2023-10-08T00:12:12.580 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.576+0000 7f691bfff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f691c159ec0 0x7f691c0af940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41256/0 (socket says 172.21.15.99:41256) 2023-10-08T00:12:12.580 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.576+0000 7f691b7fe700 1 -- 172.21.15.99:0/3524824619 learned_addr learned my addr 172.21.15.99:0/3524824619 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:12:12.580 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.577+0000 7f6920906700 1 --2- 172.21.15.99:0/3524824619 >> v2:172.21.15.191:3300/0 conn(0x7f691c06acb0 0x7f691c0afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:12.580 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.577+0000 7f691bfff700 1 -- 172.21.15.99:0/3524824619 >> v2:172.21.15.191:3300/0 conn(0x7f691c06acb0 msgr2=0x7f691c0afe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:12.580 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.577+0000 7f691bfff700 1 --2- 172.21.15.99:0/3524824619 >> v2:172.21.15.191:3300/0 conn(0x7f691c06acb0 0x7f691c0afe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:12.580 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.577+0000 7f691bfff700 1 -- 172.21.15.99:0/3524824619 >> v1:172.21.15.99:6789/0 conn(0x7f691c15a290 legacy=0x7f691c06a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:12.580 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.577+0000 7f691bfff700 1 -- 172.21.15.99:0/3524824619 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f691c0b04b0 con 0x7f691c159ec0 2023-10-08T00:12:12.580 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.577+0000 7f691affd700 1 -- 172.21.15.99:0/3524824619 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1478136301 0 0) 0x7f691c0b0790 con 0x7f691c15a290 2023-10-08T00:12:12.580 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.577+0000 7f6920906700 1 --2- 172.21.15.99:0/3524824619 >> v2:172.21.15.191:3300/0 conn(0x7f691c06acb0 0x7f691c0afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:12:12.580 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.577+0000 7f691bfff700 1 --2- 172.21.15.99:0/3524824619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f691c159ec0 0x7f691c0af940 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f691c0711f0 tx=0x7f690c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e69de74377717c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:12.580 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.577+0000 7f691affd700 1 -- 172.21.15.99:0/3524824619 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690c00b6b0 con 0x7f691c159ec0 2023-10-08T00:12:12.581 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.577+0000 7f691affd700 1 -- 172.21.15.99:0/3524824619 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f690c00b810 con 0x7f691c159ec0 2023-10-08T00:12:12.581 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.577+0000 7f691affd700 1 -- 172.21.15.99:0/3524824619 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690c00bbb0 con 0x7f691c159ec0 2023-10-08T00:12:12.581 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.578+0000 7f6922369700 1 -- 172.21.15.99:0/3524824619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f691c159ec0 msgr2=0x7f691c0af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:12.581 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.578+0000 7f6922369700 1 --2- 172.21.15.99:0/3524824619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f691c159ec0 0x7f691c0af940 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f691c0711f0 tx=0x7f690c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:12:12.581 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.578+0000 7f6922369700 1 -- 172.21.15.99:0/3524824619 shutdown_connections 2023-10-08T00:12:12.581 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.578+0000 7f6922369700 1 --2- 172.21.15.99:0/3524824619 >> v2:172.21.15.191:3300/0 conn(0x7f691c06acb0 0x7f691c0afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:12.581 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.578+0000 7f6922369700 1 --2- 172.21.15.99:0/3524824619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f691c159ec0 0x7f691c0af940 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:12.581 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.578+0000 7f6922369700 1 -- 172.21.15.99:0/3524824619 >> 172.21.15.99:0/3524824619 conn(0x7f691c0d1880 msgr2=0x7f691c0cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:12.581 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.578+0000 7f6922369700 1 -- 172.21.15.99:0/3524824619 shutdown_connections 2023-10-08T00:12:12.581 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.578+0000 7f6922369700 1 -- 172.21.15.99:0/3524824619 wait complete. 2023-10-08T00:12:12.581 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.579+0000 7f6922369700 1 Processor -- start 2023-10-08T00:12:12.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.579+0000 7f6922369700 1 -- start start 2023-10-08T00:12:12.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.580+0000 7f6922369700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f691c06acb0 0x7f691c155990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:12.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.580+0000 7f6922369700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f691c159ec0 0x7f691c1572e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:12.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.580+0000 7f6922369700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f691c15a290 0x7f691c157820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:12.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.580+0000 7f6922369700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f691c0c8da0 con 0x7f691c06acb0 2023-10-08T00:12:12.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.580+0000 7f6922369700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f691c0c8910 con 0x7f691c159ec0 2023-10-08T00:12:12.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.580+0000 7f6922369700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f691c159720 con 0x7f691c15a290 2023-10-08T00:12:12.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.580+0000 7f691bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f691c06acb0 0x7f691c155990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:12.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.580+0000 7f691bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f691c06acb0 0x7f691c155990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41268/0 (socket says 172.21.15.99:41268) 2023-10-08T00:12:12.586 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.580+0000 7f691bfff700 1 -- 172.21.15.99:0/1307434648 learned_addr learned my addr 172.21.15.99:0/1307434648 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:12.586 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.580+0000 7f691bfff700 1 -- 172.21.15.99:0/1307434648 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f691c15a290 msgr2=0x7f691c157820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:12.586 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.580+0000 7f691bfff700 1 --2- 172.21.15.99:0/1307434648 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f691c15a290 0x7f691c157820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:12.586 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.580+0000 7f691b7fe700 1 --2- 172.21.15.99:0/1307434648 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f691c159ec0 0x7f691c1572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:12.586 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.580+0000 7f691bfff700 1 -- 172.21.15.99:0/1307434648 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f691c159ec0 msgr2=0x7f691c1572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:12.586 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.580+0000 7f691bfff700 1 --2- 172.21.15.99:0/1307434648 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f691c159ec0 0x7f691c1572e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:12.587 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.580+0000 7f691bfff700 1 -- 172.21.15.99:0/1307434648 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f690c009420 con 0x7f691c06acb0 2023-10-08T00:12:12.587 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.580+0000 7f691bfff700 1 --2- 172.21.15.99:0/1307434648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f691c06acb0 0x7f691c155990 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f691c071eb0 tx=0x7f690c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:12.587 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.580+0000 7f6903fff700 1 -- 172.21.15.99:0/1307434648 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690c011420 con 0x7f691c06acb0 2023-10-08T00:12:12.587 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.580+0000 7f6903fff700 1 -- 172.21.15.99:0/1307434648 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f690c011580 con 0x7f691c06acb0 2023-10-08T00:12:12.587 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.580+0000 7f6903fff700 1 -- 172.21.15.99:0/1307434648 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690c011920 con 0x7f691c06acb0 2023-10-08T00:12:12.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.580+0000 7f6922369700 1 -- 172.21.15.99:0/1307434648 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f691c0c7fe0 con 0x7f691c06acb0 2023-10-08T00:12:12.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.580+0000 7f6922369700 1 -- 172.21.15.99:0/1307434648 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f691c0c85a0 con 0x7f691c06acb0 2023-10-08T00:12:12.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.581+0000 7f6922369700 1 -- 172.21.15.99:0/1307434648 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f691c0c8bf0 con 0x7f691c06acb0 2023-10-08T00:12:12.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.582+0000 7f6903fff700 1 -- 172.21.15.99:0/1307434648 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f690c011d10 con 0x7f691c06acb0 2023-10-08T00:12:12.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.582+0000 7f6903fff700 1 --2- 172.21.15.99:0/1307434648 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6908066f30 0x7f69080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:12.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.582+0000 7f6903fff700 1 -- 172.21.15.99:0/1307434648 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f690c085bc0 con 0x7f691c06acb0 2023-10-08T00:12:12.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.582+0000 7f691b7fe700 1 --2- 172.21.15.99:0/1307434648 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6908066f30 0x7f69080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:12.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.583+0000 7f691b7fe700 1 --2- 172.21.15.99:0/1307434648 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6908066f30 0x7f69080693e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f691c0ca100 tx=0x7f6910009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:12.590 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.586+0000 7f6903fff700 1 -- 172.21.15.99:0/1307434648 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f690c052eb0 con 0x7f691c06acb0 2023-10-08T00:12:12.812 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:12.811+0000 7f6922369700 1 -- 172.21.15.99:0/1307434648 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f691c04c3c0 con 0x7f691c06acb0 2023-10-08T00:12:13.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.437+0000 7f6903fff700 1 -- 172.21.15.99:0/1307434648 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7f690c0564d0 con 0x7f691c06acb0 2023-10-08T00:12:13.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.439+0000 7f6922369700 1 -- 172.21.15.99:0/1307434648 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6908066f30 msgr2=0x7f69080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:13.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.439+0000 7f6922369700 1 --2- 172.21.15.99:0/1307434648 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6908066f30 0x7f69080693e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f691c0ca100 tx=0x7f6910009040 comp rx=0 tx=0).stop 2023-10-08T00:12:13.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.439+0000 7f6922369700 1 -- 172.21.15.99:0/1307434648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f691c06acb0 msgr2=0x7f691c155990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:13.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.439+0000 7f6922369700 1 --2- 172.21.15.99:0/1307434648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f691c06acb0 0x7f691c155990 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f691c071eb0 tx=0x7f690c017000 comp rx=0 tx=0).stop 2023-10-08T00:12:13.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.439+0000 7f6922369700 1 -- 172.21.15.99:0/1307434648 shutdown_connections 2023-10-08T00:12:13.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.439+0000 7f6922369700 1 --2- 172.21.15.99:0/1307434648 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6908066f30 0x7f69080693e0 secure :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f691c0ca100 tx=0x7f6910009040 comp rx=0 tx=0).stop 2023-10-08T00:12:13.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.439+0000 7f6922369700 1 --2- 172.21.15.99:0/1307434648 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f691c15a290 0x7f691c157820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:13.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.439+0000 7f6922369700 1 --2- 172.21.15.99:0/1307434648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f691c06acb0 0x7f691c155990 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:13.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.439+0000 7f6922369700 1 --2- 172.21.15.99:0/1307434648 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f691c159ec0 0x7f691c1572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:13.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.439+0000 7f6922369700 1 -- 172.21.15.99:0/1307434648 >> 172.21.15.99:0/1307434648 conn(0x7f691c0d1880 msgr2=0x7f691c0c5b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:13.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.440+0000 7f6922369700 1 -- 172.21.15.99:0/1307434648 shutdown_connections 2023-10-08T00:12:13.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.440+0000 7f6922369700 1 -- 172.21.15.99:0/1307434648 wait complete. 2023-10-08T00:12:13.453 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:12:13.567 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.566+0000 7fdd88233700 1 Processor -- start 2023-10-08T00:12:13.567 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.566+0000 7fdd88233700 1 -- start start 2023-10-08T00:12:13.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.567+0000 7fdd88233700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdd80157870 0x7fdd8015b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:13.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.567+0000 7fdd88233700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd8015cbb0 con 0x7fdd80157c40 2023-10-08T00:12:13.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.567+0000 7fdd88233700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd8015bfa0 con 0x7fdd8015a620 2023-10-08T00:12:13.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.567+0000 7fdd88233700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd8015c280 con 0x7fdd80157870 2023-10-08T00:12:13.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.567+0000 7fdd85fcf700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdd80157870 0x7fdd8015b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:13.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.567+0000 7fdd867d0700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fdd80157c40 0x7fdd80158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45112/0 (socket says 172.21.15.99:45112) 2023-10-08T00:12:13.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.567+0000 7fdd85fcf700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdd80157870 0x7fdd8015b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44538/0 (socket says 172.21.15.99:44538) 2023-10-08T00:12:13.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.567+0000 7fdd867d0700 1 -- 172.21.15.99:0/2643343913 learned_addr learned my addr 172.21.15.99:0/2643343913 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:12:13.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.567+0000 7fdd85fcf700 1 -- 172.21.15.99:0/2643343913 >> v1:172.21.15.191:6789/0 conn(0x7fdd8015a620 legacy=0x7fdd8015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:13.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.568+0000 7fdd85fcf700 1 -- 172.21.15.99:0/2643343913 >> v1:172.21.15.99:6789/0 conn(0x7fdd80157c40 legacy=0x7fdd80158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:13.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.568+0000 7fdd85fcf700 1 -- 172.21.15.99:0/2643343913 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd8015c560 con 0x7fdd80157870 2023-10-08T00:12:13.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.568+0000 7fdd84fcd700 1 -- 172.21.15.99:0/2643343913 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 448611590 0 0) 0x7fdd8015cbb0 con 0x7fdd80157c40 2023-10-08T00:12:13.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.568+0000 7fdd85fcf700 1 --2- 172.21.15.99:0/2643343913 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd80157870 0x7fdd8015b6a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c0098e0 tx=0x7fdd7c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d36b0236e24bf server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:13.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.568+0000 7fdd84fcd700 1 -- 172.21.15.99:0/2643343913 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c009e80 con 0x7fdd80157870 2023-10-08T00:12:13.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.568+0000 7fdd84fcd700 1 -- 172.21.15.99:0/2643343913 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd7c00b820 con 0x7fdd80157870 2023-10-08T00:12:13.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.568+0000 7fdd84fcd700 1 -- 172.21.15.99:0/2643343913 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c00bba0 con 0x7fdd80157870 2023-10-08T00:12:13.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.568+0000 7fdd88233700 1 -- 172.21.15.99:0/2643343913 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd80157870 msgr2=0x7fdd8015b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:13.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.568+0000 7fdd88233700 1 --2- 172.21.15.99:0/2643343913 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd80157870 0x7fdd8015b6a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c0098e0 tx=0x7fdd7c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:12:13.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.568+0000 7fdd88233700 1 -- 172.21.15.99:0/2643343913 shutdown_connections 2023-10-08T00:12:13.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.568+0000 7fdd88233700 1 --2- 172.21.15.99:0/2643343913 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd80157870 0x7fdd8015b6a0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:13.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.568+0000 7fdd88233700 1 -- 172.21.15.99:0/2643343913 >> 172.21.15.99:0/2643343913 conn(0x7fdd80068ce0 msgr2=0x7fdd800690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:13.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.568+0000 7fdd88233700 1 -- 172.21.15.99:0/2643343913 shutdown_connections 2023-10-08T00:12:13.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.568+0000 7fdd88233700 1 -- 172.21.15.99:0/2643343913 wait complete. 2023-10-08T00:12:13.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.570+0000 7fdd88233700 1 Processor -- start 2023-10-08T00:12:13.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.570+0000 7fdd88233700 1 -- start start 2023-10-08T00:12:13.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.570+0000 7fdd88233700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd80157870 0x7fdd8015fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:13.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.570+0000 7fdd88233700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd80157c40 0x7fdd800d06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:13.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.570+0000 7fdd88233700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd8015a620 0x7fdd801390f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:13.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.570+0000 7fdd88233700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd80160360 con 0x7fdd8015a620 2023-10-08T00:12:13.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.570+0000 7fdd88233700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd80160640 con 0x7fdd80157870 2023-10-08T00:12:13.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.570+0000 7fdd88233700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd80160920 con 0x7fdd80157c40 2023-10-08T00:12:13.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.570+0000 7fdd867d0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd8015a620 0x7fdd801390f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:13.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.570+0000 7fdd867d0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd8015a620 0x7fdd801390f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44544/0 (socket says 172.21.15.99:44544) 2023-10-08T00:12:13.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.570+0000 7fdd867d0700 1 -- 172.21.15.99:0/3743034661 learned_addr learned my addr 172.21.15.99:0/3743034661 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:13.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.571+0000 7fdd85fcf700 1 --2- 172.21.15.99:0/3743034661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd80157870 0x7fdd8015fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:13.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.571+0000 7fdd857ce700 1 --2- 172.21.15.99:0/3743034661 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd80157c40 0x7fdd800d06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:13.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.571+0000 7fdd867d0700 1 -- 172.21.15.99:0/3743034661 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd80157c40 msgr2=0x7fdd800d06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:13.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.571+0000 7fdd867d0700 1 --2- 172.21.15.99:0/3743034661 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd80157c40 0x7fdd800d06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:13.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.571+0000 7fdd867d0700 1 -- 172.21.15.99:0/3743034661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd80157870 msgr2=0x7fdd8015fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:13.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.571+0000 7fdd867d0700 1 --2- 172.21.15.99:0/3743034661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd80157870 0x7fdd8015fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:13.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.571+0000 7fdd867d0700 1 -- 172.21.15.99:0/3743034661 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd7c009420 con 0x7fdd8015a620 2023-10-08T00:12:13.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.571+0000 7fdd857ce700 1 --2- 172.21.15.99:0/3743034661 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd80157c40 0x7fdd800d06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:12:13.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.571+0000 7fdd867d0700 1 --2- 172.21.15.99:0/3743034661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd8015a620 0x7fdd801390f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fdd7800ccc0 tx=0x7fdd78001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:13.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.571+0000 7fdd75ffb700 1 -- 172.21.15.99:0/3743034661 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7800a930 con 0x7fdd8015a620 2023-10-08T00:12:13.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.571+0000 7fdd75ffb700 1 -- 172.21.15.99:0/3743034661 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd7800aa90 con 0x7fdd8015a620 2023-10-08T00:12:13.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.571+0000 7fdd75ffb700 1 -- 172.21.15.99:0/3743034661 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7800ae30 con 0x7fdd8015a620 2023-10-08T00:12:13.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.571+0000 7fdd88233700 1 -- 172.21.15.99:0/3743034661 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd80160c00 con 0x7fdd8015a620 2023-10-08T00:12:13.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.571+0000 7fdd88233700 1 -- 172.21.15.99:0/3743034661 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd80160050 con 0x7fdd8015a620 2023-10-08T00:12:13.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.572+0000 7fdd88233700 1 -- 172.21.15.99:0/3743034661 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd80161120 con 0x7fdd8015a620 2023-10-08T00:12:13.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.572+0000 7fdd75ffb700 1 -- 172.21.15.99:0/3743034661 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdd78007300 con 0x7fdd8015a620 2023-10-08T00:12:13.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.573+0000 7fdd75ffb700 1 --2- 172.21.15.99:0/3743034661 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdd64066f30 0x7fdd640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:13.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.573+0000 7fdd75ffb700 1 -- 172.21.15.99:0/3743034661 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdd780848c0 con 0x7fdd8015a620 2023-10-08T00:12:13.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.573+0000 7fdd85fcf700 1 --2- 172.21.15.99:0/3743034661 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdd64066f30 0x7fdd640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:13.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.573+0000 7fdd85fcf700 1 --2- 172.21.15.99:0/3743034661 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdd64066f30 0x7fdd640693e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c0030c0 tx=0x7fdd7c016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:13.577 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.577+0000 7fdd75ffb700 1 -- 172.21.15.99:0/3743034661 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdd78051bb0 con 0x7fdd8015a620 2023-10-08T00:12:13.801 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.800+0000 7fdd88233700 1 -- 172.21.15.99:0/3743034661 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd8004c3c0 con 0x7fdd8015a620 2023-10-08T00:12:13.801 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.801+0000 7fdd75ffb700 1 -- 172.21.15.99:0/3743034661 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4603 (secure 0 0 0) 0x7fdd780551d0 con 0x7fdd8015a620 2023-10-08T00:12:13.801 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:12:13.802 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5146,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/1438931938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1438931938},{"type":"v1","addr":"172.21.15.191:6836","nonce":1438931938}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5280,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/3090674680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3090674680},{"type":"v1","addr":"172.21.15.99:6833","nonce":3090674680}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5283,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/4025282357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":4025282357},{"type":"v1","addr":"172.21.15.99:6835","nonce":4025282357}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:12:09.803824+0000","modified":"2023-10-08T00:12:13.436570+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5289},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5289":{"gid":5289,"name":"d","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/2062242133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":2062242133},{"type":"v1","addr":"172.21.15.191:6837","nonce":2062242133}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-08T00:12:13.803 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.802+0000 7fdd88233700 1 -- 172.21.15.99:0/3743034661 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdd64066f30 msgr2=0x7fdd640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:13.804 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.803+0000 7fdd88233700 1 --2- 172.21.15.99:0/3743034661 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdd64066f30 0x7fdd640693e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c0030c0 tx=0x7fdd7c016000 comp rx=0 tx=0).stop 2023-10-08T00:12:13.804 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.803+0000 7fdd88233700 1 -- 172.21.15.99:0/3743034661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd8015a620 msgr2=0x7fdd801390f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:13.804 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.803+0000 7fdd88233700 1 --2- 172.21.15.99:0/3743034661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd8015a620 0x7fdd801390f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fdd7800ccc0 tx=0x7fdd78001a20 comp rx=0 tx=0).stop 2023-10-08T00:12:13.804 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.803+0000 7fdd88233700 1 -- 172.21.15.99:0/3743034661 shutdown_connections 2023-10-08T00:12:13.804 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.803+0000 7fdd88233700 1 --2- 172.21.15.99:0/3743034661 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdd64066f30 0x7fdd640693e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:13.804 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.803+0000 7fdd88233700 1 --2- 172.21.15.99:0/3743034661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd80157870 0x7fdd8015fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:13.804 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.803+0000 7fdd88233700 1 --2- 172.21.15.99:0/3743034661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd8015a620 0x7fdd801390f0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:13.804 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.803+0000 7fdd88233700 1 --2- 172.21.15.99:0/3743034661 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd80157c40 0x7fdd800d06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:13.804 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.803+0000 7fdd88233700 1 -- 172.21.15.99:0/3743034661 >> 172.21.15.99:0/3743034661 conn(0x7fdd80068ce0 msgr2=0x7fdd800b46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:13.804 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.803+0000 7fdd88233700 1 -- 172.21.15.99:0/3743034661 shutdown_connections 2023-10-08T00:12:13.804 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.803+0000 7fdd88233700 1 -- 172.21.15.99:0/3743034661 wait complete. 2023-10-08T00:12:13.805 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 24 2023-10-08T00:12:13.815 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:12:13.929 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.928+0000 7fc3080c6700 1 Processor -- start 2023-10-08T00:12:13.929 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.928+0000 7fc3080c6700 1 -- start start 2023-10-08T00:12:13.936 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.929+0000 7fc3080c6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc300157870 0x7fc30015b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:13.936 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.929+0000 7fc3080c6700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc300157c40 0x7fc300158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:13.937 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.929+0000 7fc3080c6700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc30015cbb0 con 0x7fc30015a620 2023-10-08T00:12:13.937 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.929+0000 7fc3080c6700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc30015bfa0 con 0x7fc300157c40 2023-10-08T00:12:13.937 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.929+0000 7fc3080c6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc30015c280 con 0x7fc300157870 2023-10-08T00:12:13.937 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.929+0000 7fc306663700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc300157c40 0x7fc300158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:13.937 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.929+0000 7fc306663700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc300157c40 0x7fc300158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44546/0 (socket says 172.21.15.99:44546) 2023-10-08T00:12:13.937 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.929+0000 7fc306663700 1 -- 172.21.15.99:0/823612037 learned_addr learned my addr 172.21.15.99:0/823612037 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:13.937 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.929+0000 7fc305e62700 1 --2- 172.21.15.99:0/823612037 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc300157870 0x7fc30015b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:13.938 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.930+0000 7fc306663700 1 -- 172.21.15.99:0/823612037 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc300157870 msgr2=0x7fc30015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:13.938 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.930+0000 7fc306663700 1 --2- 172.21.15.99:0/823612037 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc300157870 0x7fc30015b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:13.938 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.930+0000 7fc306663700 1 -- 172.21.15.99:0/823612037 >> v1:172.21.15.191:6789/0 conn(0x7fc30015a620 legacy=0x7fc30015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:13.938 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.930+0000 7fc306663700 1 -- 172.21.15.99:0/823612037 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc30015c560 con 0x7fc300157c40 2023-10-08T00:12:13.938 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.930+0000 7fc306663700 1 --2- 172.21.15.99:0/823612037 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc300157c40 0x7fc300158110 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc2f80099b0 tx=0x7fc2f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d15b8e8cde6cc2ff server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:13.938 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.930+0000 7fc304e60700 1 -- 172.21.15.99:0/823612037 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f8009ea0 con 0x7fc300157c40 2023-10-08T00:12:13.939 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.930+0000 7fc304e60700 1 -- 172.21.15.99:0/823612037 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f800b860 con 0x7fc300157c40 2023-10-08T00:12:13.939 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.930+0000 7fc304e60700 1 -- 172.21.15.99:0/823612037 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f800bc00 con 0x7fc300157c40 2023-10-08T00:12:13.939 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.930+0000 7fc3080c6700 1 -- 172.21.15.99:0/823612037 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc300157c40 msgr2=0x7fc300158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:13.939 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.930+0000 7fc3080c6700 1 --2- 172.21.15.99:0/823612037 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc300157c40 0x7fc300158110 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc2f80099b0 tx=0x7fc2f8009250 comp rx=0 tx=0).stop 2023-10-08T00:12:13.939 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.930+0000 7fc3080c6700 1 -- 172.21.15.99:0/823612037 shutdown_connections 2023-10-08T00:12:13.939 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.930+0000 7fc3080c6700 1 --2- 172.21.15.99:0/823612037 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc300157c40 0x7fc300158110 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:13.939 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.930+0000 7fc3080c6700 1 --2- 172.21.15.99:0/823612037 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc300157870 0x7fc30015b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:13.939 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.930+0000 7fc3080c6700 1 -- 172.21.15.99:0/823612037 >> 172.21.15.99:0/823612037 conn(0x7fc300068ce0 msgr2=0x7fc3000690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:13.939 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.931+0000 7fc3080c6700 1 -- 172.21.15.99:0/823612037 shutdown_connections 2023-10-08T00:12:13.940 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.931+0000 7fc3080c6700 1 -- 172.21.15.99:0/823612037 wait complete. 2023-10-08T00:12:13.940 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.932+0000 7fc3080c6700 1 Processor -- start 2023-10-08T00:12:13.940 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.932+0000 7fc3080c6700 1 -- start start 2023-10-08T00:12:13.940 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.932+0000 7fc3080c6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc300157870 0x7fc300139090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:13.940 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.932+0000 7fc3080c6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc300157c40 0x7fc30014d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:13.940 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.932+0000 7fc3080c6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc30015a620 0x7fc30014de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:13.940 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.932+0000 7fc3080c6700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc300151990 con 0x7fc300157870 2023-10-08T00:12:13.940 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.932+0000 7fc3080c6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc300151c00 con 0x7fc30015a620 2023-10-08T00:12:13.940 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.932+0000 7fc3080c6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc300151ee0 con 0x7fc300157c40 2023-10-08T00:12:13.941 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.933+0000 7fc305e62700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc300157870 0x7fc300139090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:13.941 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.933+0000 7fc305e62700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc300157870 0x7fc300139090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44552/0 (socket says 172.21.15.99:44552) 2023-10-08T00:12:13.941 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.933+0000 7fc305e62700 1 -- 172.21.15.99:0/150070418 learned_addr learned my addr 172.21.15.99:0/150070418 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:13.941 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.933+0000 7fc305e62700 1 -- 172.21.15.99:0/150070418 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc300157c40 msgr2=0x7fc30014d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:13.941 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.933+0000 7fc305e62700 1 --2- 172.21.15.99:0/150070418 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc300157c40 0x7fc30014d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:13.941 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.933+0000 7fc305e62700 1 -- 172.21.15.99:0/150070418 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc30015a620 msgr2=0x7fc30014de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:13.941 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.933+0000 7fc305e62700 1 --2- 172.21.15.99:0/150070418 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc30015a620 0x7fc30014de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:13.942 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.933+0000 7fc305e62700 1 -- 172.21.15.99:0/150070418 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f80094f0 con 0x7fc300157870 2023-10-08T00:12:13.942 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.933+0000 7fc305e62700 1 --2- 172.21.15.99:0/150070418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc300157870 0x7fc300139090 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc009ef0 tx=0x7fc2fc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:13.942 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.933+0000 7fc2f5ffb700 1 -- 172.21.15.99:0/150070418 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc00cc50 con 0x7fc300157870 2023-10-08T00:12:13.942 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.933+0000 7fc2f5ffb700 1 -- 172.21.15.99:0/150070418 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2fc00f040 con 0x7fc300157870 2023-10-08T00:12:13.942 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.933+0000 7fc2f5ffb700 1 -- 172.21.15.99:0/150070418 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc014480 con 0x7fc300157870 2023-10-08T00:12:13.942 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.933+0000 7fc3080c6700 1 -- 172.21.15.99:0/150070418 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3001521c0 con 0x7fc300157870 2023-10-08T00:12:13.942 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.933+0000 7fc3080c6700 1 -- 172.21.15.99:0/150070418 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3001395d0 con 0x7fc300157870 2023-10-08T00:12:13.942 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.934+0000 7fc3080c6700 1 -- 172.21.15.99:0/150070418 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3000d1760 con 0x7fc300157870 2023-10-08T00:12:13.943 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.935+0000 7fc2f5ffb700 1 -- 172.21.15.99:0/150070418 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc2fc0147f0 con 0x7fc300157870 2023-10-08T00:12:13.943 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.935+0000 7fc2f5ffb700 1 --2- 172.21.15.99:0/150070418 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc2e4066f30 0x7fc2e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:13.943 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.935+0000 7fc2f5ffb700 1 -- 172.21.15.99:0/150070418 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc2fc084fe0 con 0x7fc300157870 2023-10-08T00:12:13.943 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.935+0000 7fc305661700 1 --2- 172.21.15.99:0/150070418 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc2e4066f30 0x7fc2e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:13.943 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.936+0000 7fc305661700 1 --2- 172.21.15.99:0/150070418 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc2e4066f30 0x7fc2e40693e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0001690 tx=0x7fc2f0007b40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:13.944 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:13.939+0000 7fc2f5ffb700 1 -- 172.21.15.99:0/150070418 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc2fc052250 con 0x7fc300157870 2023-10-08T00:12:14.123 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.123+0000 7fc3080c6700 1 -- 172.21.15.99:0/150070418 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc30004c3c0 con 0x7fc300157870 2023-10-08T00:12:14.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.123+0000 7fc2f5ffb700 1 -- 172.21.15.99:0/150070418 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+15350 (secure 0 0 0) 0x7fc2fc055870 con 0x7fc300157870 2023-10-08T00:12:14.124 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:12:14.125 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":45,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:12:11.417504+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:12:07.733432+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-08T00:12:08.762879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:12:14.126 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.125+0000 7fc3080c6700 1 -- 172.21.15.99:0/150070418 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc2e4066f30 msgr2=0x7fc2e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:14.127 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.125+0000 7fc3080c6700 1 --2- 172.21.15.99:0/150070418 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc2e4066f30 0x7fc2e40693e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0001690 tx=0x7fc2f0007b40 comp rx=0 tx=0).stop 2023-10-08T00:12:14.127 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.125+0000 7fc3080c6700 1 -- 172.21.15.99:0/150070418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc300157870 msgr2=0x7fc300139090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:14.127 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.125+0000 7fc3080c6700 1 --2- 172.21.15.99:0/150070418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc300157870 0x7fc300139090 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc009ef0 tx=0x7fc2fc00b040 comp rx=0 tx=0).stop 2023-10-08T00:12:14.127 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.125+0000 7fc3080c6700 1 -- 172.21.15.99:0/150070418 shutdown_connections 2023-10-08T00:12:14.127 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.125+0000 7fc3080c6700 1 --2- 172.21.15.99:0/150070418 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc2e4066f30 0x7fc2e40693e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:14.127 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.125+0000 7fc3080c6700 1 --2- 172.21.15.99:0/150070418 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc30015a620 0x7fc30014de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:14.127 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.126+0000 7fc3080c6700 1 --2- 172.21.15.99:0/150070418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc300157870 0x7fc300139090 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:14.127 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.126+0000 7fc3080c6700 1 --2- 172.21.15.99:0/150070418 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc300157c40 0x7fc30014d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:14.127 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.126+0000 7fc3080c6700 1 -- 172.21.15.99:0/150070418 >> 172.21.15.99:0/150070418 conn(0x7fc300068ce0 msgr2=0x7fc3000b46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:14.128 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.126+0000 7fc3080c6700 1 -- 172.21.15.99:0/150070418 shutdown_connections 2023-10-08T00:12:14.128 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.126+0000 7fc3080c6700 1 -- 172.21.15.99:0/150070418 wait complete. 2023-10-08T00:12:14.138 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T00:12:14.139 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T00:12:14.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.252+0000 7fe827888700 1 Processor -- start 2023-10-08T00:12:14.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.252+0000 7fe827888700 1 -- start start 2023-10-08T00:12:14.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.253+0000 7fe827888700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fe820157870 0x7fe82015b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:14.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.253+0000 7fe827888700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe820157c40 0x7fe820158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:14.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.253+0000 7fe827888700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe82015cbb0 con 0x7fe82015a620 2023-10-08T00:12:14.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.253+0000 7fe827888700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe82015bfa0 con 0x7fe820157c40 2023-10-08T00:12:14.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.253+0000 7fe827888700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe82015c280 con 0x7fe820157870 2023-10-08T00:12:14.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.253+0000 7fe825e25700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe820157c40 0x7fe820158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:14.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.253+0000 7fe824e23700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fe82015a620 0x7fe82015d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45114/0 (socket says 172.21.15.99:45114) 2023-10-08T00:12:14.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.253+0000 7fe824e23700 1 -- 172.21.15.99:0/3499075142 learned_addr learned my addr 172.21.15.99:0/3499075142 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:12:14.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.253+0000 7fe825e25700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe820157c40 0x7fe820158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44562/0 (socket says 172.21.15.99:44562) 2023-10-08T00:12:14.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.254+0000 7fe825624700 1 --2- 172.21.15.99:0/3499075142 >> v2:172.21.15.191:3300/0 conn(0x7fe820157870 0x7fe82015b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:14.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.254+0000 7fe825e25700 1 -- 172.21.15.99:0/3499075142 >> v2:172.21.15.191:3300/0 conn(0x7fe820157870 msgr2=0x7fe82015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:14.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.254+0000 7fe825e25700 1 --2- 172.21.15.99:0/3499075142 >> v2:172.21.15.191:3300/0 conn(0x7fe820157870 0x7fe82015b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:14.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.254+0000 7fe825e25700 1 -- 172.21.15.99:0/3499075142 >> v1:172.21.15.99:6789/0 conn(0x7fe82015a620 legacy=0x7fe82015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:14.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.254+0000 7fe825e25700 1 -- 172.21.15.99:0/3499075142 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe82015c560 con 0x7fe820157c40 2023-10-08T00:12:14.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.254+0000 7fe825e25700 1 --2- 172.21.15.99:0/3499075142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe820157c40 0x7fe820158110 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe820069d80 tx=0x7fe818009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3321b33c06763b17 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:14.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.254+0000 7fe817fff700 1 -- 172.21.15.99:0/3499075142 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81800e070 con 0x7fe820157c40 2023-10-08T00:12:14.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.254+0000 7fe817fff700 1 -- 172.21.15.99:0/3499075142 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe81800bc30 con 0x7fe820157c40 2023-10-08T00:12:14.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.254+0000 7fe817fff700 1 -- 172.21.15.99:0/3499075142 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe818012430 con 0x7fe820157c40 2023-10-08T00:12:14.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.254+0000 7fe827888700 1 -- 172.21.15.99:0/3499075142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe820157c40 msgr2=0x7fe820158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:14.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.254+0000 7fe827888700 1 --2- 172.21.15.99:0/3499075142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe820157c40 0x7fe820158110 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe820069d80 tx=0x7fe818009780 comp rx=0 tx=0).stop 2023-10-08T00:12:14.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.254+0000 7fe827888700 1 -- 172.21.15.99:0/3499075142 shutdown_connections 2023-10-08T00:12:14.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.254+0000 7fe827888700 1 --2- 172.21.15.99:0/3499075142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe820157c40 0x7fe820158110 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:14.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.254+0000 7fe827888700 1 --2- 172.21.15.99:0/3499075142 >> v2:172.21.15.191:3300/0 conn(0x7fe820157870 0x7fe82015b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:14.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.254+0000 7fe827888700 1 -- 172.21.15.99:0/3499075142 >> 172.21.15.99:0/3499075142 conn(0x7fe820068ce0 msgr2=0x7fe8200690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:14.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.255+0000 7fe827888700 1 -- 172.21.15.99:0/3499075142 shutdown_connections 2023-10-08T00:12:14.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.255+0000 7fe827888700 1 -- 172.21.15.99:0/3499075142 wait complete. 2023-10-08T00:12:14.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.256+0000 7fe827888700 1 Processor -- start 2023-10-08T00:12:14.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.256+0000 7fe827888700 1 -- start start 2023-10-08T00:12:14.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.256+0000 7fe827888700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe820157870 0x7fe82013f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:14.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.256+0000 7fe827888700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe820157c40 0x7fe82014f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:14.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.256+0000 7fe827888700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe82015a620 0x7fe8200b1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:14.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.256+0000 7fe827888700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8200b1400 con 0x7fe820157870 2023-10-08T00:12:14.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.256+0000 7fe827888700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8200b07f0 con 0x7fe82015a620 2023-10-08T00:12:14.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.256+0000 7fe827888700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8200b0ad0 con 0x7fe820157c40 2023-10-08T00:12:14.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.256+0000 7fe825624700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe820157870 0x7fe82013f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:14.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.257+0000 7fe825624700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe820157870 0x7fe82013f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44570/0 (socket says 172.21.15.99:44570) 2023-10-08T00:12:14.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.257+0000 7fe825624700 1 -- 172.21.15.99:0/2074161020 learned_addr learned my addr 172.21.15.99:0/2074161020 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:14.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.257+0000 7fe825e25700 1 --2- 172.21.15.99:0/2074161020 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe82015a620 0x7fe8200b1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:14.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.257+0000 7fe824e23700 1 --2- 172.21.15.99:0/2074161020 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe820157c40 0x7fe82014f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:14.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.257+0000 7fe825624700 1 -- 172.21.15.99:0/2074161020 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe820157c40 msgr2=0x7fe82014f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:14.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.257+0000 7fe825624700 1 --2- 172.21.15.99:0/2074161020 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe820157c40 0x7fe82014f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:14.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.257+0000 7fe825624700 1 -- 172.21.15.99:0/2074161020 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe82015a620 msgr2=0x7fe8200b1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:14.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.257+0000 7fe825624700 1 --2- 172.21.15.99:0/2074161020 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe82015a620 0x7fe8200b1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:14.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.257+0000 7fe825624700 1 -- 172.21.15.99:0/2074161020 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe818009a20 con 0x7fe820157870 2023-10-08T00:12:14.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.257+0000 7fe824e23700 1 --2- 172.21.15.99:0/2074161020 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe820157c40 0x7fe82014f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:12:14.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.257+0000 7fe825624700 1 --2- 172.21.15.99:0/2074161020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe820157870 0x7fe82013f920 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe81c00c7d0 tx=0x7fe81c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:14.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.257+0000 7fe8157fa700 1 -- 172.21.15.99:0/2074161020 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81c011600 con 0x7fe820157870 2023-10-08T00:12:14.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.257+0000 7fe8157fa700 1 -- 172.21.15.99:0/2074161020 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe81c011760 con 0x7fe820157870 2023-10-08T00:12:14.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.257+0000 7fe8157fa700 1 -- 172.21.15.99:0/2074161020 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81c011b20 con 0x7fe820157870 2023-10-08T00:12:14.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.257+0000 7fe827888700 1 -- 172.21.15.99:0/2074161020 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8200b0db0 con 0x7fe820157870 2023-10-08T00:12:14.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.257+0000 7fe827888700 1 -- 172.21.15.99:0/2074161020 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe82014ff20 con 0x7fe820157870 2023-10-08T00:12:14.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.258+0000 7fe827888700 1 -- 172.21.15.99:0/2074161020 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe82004c3c0 con 0x7fe820157870 2023-10-08T00:12:14.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.258+0000 7fe8157fa700 1 -- 172.21.15.99:0/2074161020 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe81c0193f0 con 0x7fe820157870 2023-10-08T00:12:14.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.259+0000 7fe8157fa700 1 --2- 172.21.15.99:0/2074161020 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe804066f30 0x7fe8040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:14.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.259+0000 7fe8157fa700 1 -- 172.21.15.99:0/2074161020 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe81c084930 con 0x7fe820157870 2023-10-08T00:12:14.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.259+0000 7fe824e23700 1 --2- 172.21.15.99:0/2074161020 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe804066f30 0x7fe8040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:14.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.259+0000 7fe824e23700 1 --2- 172.21.15.99:0/2074161020 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe804066f30 0x7fe8040693e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe82015b460 tx=0x7fe810009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:14.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.263+0000 7fe8157fa700 1 -- 172.21.15.99:0/2074161020 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe81c051ba0 con 0x7fe820157870 2023-10-08T00:12:14.443 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.443+0000 7fe827888700 1 -- 172.21.15.99:0/2074161020 --> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe8200b1540 con 0x7fe804066f30 2023-10-08T00:12:14.452 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.452+0000 7fe8157fa700 1 -- 172.21.15.99:0/2074161020 <== mgr.4099 v2:172.21.15.191:6832/85985 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284974 (secure 0 0 0) 0x7fe810005900 con 0x7fe804066f30 2023-10-08T00:12:14.456 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:12:14.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.455+0000 7fe827888700 1 -- 172.21.15.99:0/2074161020 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe804066f30 msgr2=0x7fe8040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:14.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.455+0000 7fe827888700 1 --2- 172.21.15.99:0/2074161020 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe804066f30 0x7fe8040693e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe82015b460 tx=0x7fe810009040 comp rx=0 tx=0).stop 2023-10-08T00:12:14.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.455+0000 7fe827888700 1 -- 172.21.15.99:0/2074161020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe820157870 msgr2=0x7fe82013f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:14.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.455+0000 7fe827888700 1 --2- 172.21.15.99:0/2074161020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe820157870 0x7fe82013f920 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe81c00c7d0 tx=0x7fe81c008040 comp rx=0 tx=0).stop 2023-10-08T00:12:14.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.455+0000 7fe827888700 1 -- 172.21.15.99:0/2074161020 shutdown_connections 2023-10-08T00:12:14.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.455+0000 7fe827888700 1 --2- 172.21.15.99:0/2074161020 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe804066f30 0x7fe8040693e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:14.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.455+0000 7fe827888700 1 --2- 172.21.15.99:0/2074161020 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe82015a620 0x7fe8200b1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:14.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.455+0000 7fe827888700 1 --2- 172.21.15.99:0/2074161020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe820157870 0x7fe82013f920 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:14.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.455+0000 7fe827888700 1 --2- 172.21.15.99:0/2074161020 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe820157c40 0x7fe82014f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:14.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.455+0000 7fe827888700 1 -- 172.21.15.99:0/2074161020 >> 172.21.15.99:0/2074161020 conn(0x7fe820068ce0 msgr2=0x7fe8200b46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:14.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.455+0000 7fe827888700 1 -- 172.21.15.99:0/2074161020 shutdown_connections 2023-10-08T00:12:14.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.455+0000 7fe827888700 1 -- 172.21.15.99:0/2074161020 wait complete. 2023-10-08T00:12:14.460 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2023-10-08T00:12:14.467 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":81,"stamp":"2023-10-08T00:12:14.042081+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":142,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2827,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1559672,"statfs":{"total":1600000000,"available":1597104836,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.630035"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.430352+0000","last_change":"2023-10-08T00:12:10.430219+0000","last_active":"2023-10-08T00:12:10.430352+0000","last_peered":"2023-10-08T00:12:10.430352+0000","last_clean":"2023-10-08T00:12:10.430352+0000","last_became_active":"2023-10-08T00:12:10.429880+0000","last_became_peered":"2023-10-08T00:12:10.429880+0000","last_unstale":"2023-10-08T00:12:10.430352+0000","last_undegraded":"2023-10-08T00:12:10.430352+0000","last_fullsized":"2023-10-08T00:12:10.430352+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:04:24.070701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.424244+0000","last_change":"2023-10-08T00:12:10.424199+0000","last_active":"2023-10-08T00:12:10.424244+0000","last_peered":"2023-10-08T00:12:10.424244+0000","last_clean":"2023-10-08T00:12:10.424244+0000","last_became_active":"2023-10-08T00:12:10.424021+0000","last_became_peered":"2023-10-08T00:12:10.424021+0000","last_unstale":"2023-10-08T00:12:10.424244+0000","last_undegraded":"2023-10-08T00:12:10.424244+0000","last_fullsized":"2023-10-08T00:12:10.424244+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:46:32.815495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.429530+0000","last_change":"2023-10-08T00:12:10.429486+0000","last_active":"2023-10-08T00:12:10.429530+0000","last_peered":"2023-10-08T00:12:10.429530+0000","last_clean":"2023-10-08T00:12:10.429530+0000","last_became_active":"2023-10-08T00:12:10.428910+0000","last_became_peered":"2023-10-08T00:12:10.428910+0000","last_unstale":"2023-10-08T00:12:10.429530+0000","last_undegraded":"2023-10-08T00:12:10.429530+0000","last_fullsized":"2023-10-08T00:12:10.429530+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:16:41.819876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.428735+0000","last_change":"2023-10-08T00:12:10.428507+0000","last_active":"2023-10-08T00:12:10.428735+0000","last_peered":"2023-10-08T00:12:10.428735+0000","last_clean":"2023-10-08T00:12:10.428735+0000","last_became_active":"2023-10-08T00:12:10.428162+0000","last_became_peered":"2023-10-08T00:12:10.428162+0000","last_unstale":"2023-10-08T00:12:10.428735+0000","last_undegraded":"2023-10-08T00:12:10.428735+0000","last_fullsized":"2023-10-08T00:12:10.428735+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:46:32.815495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.431889+0000","last_change":"2023-10-08T00:12:10.431827+0000","last_active":"2023-10-08T00:12:10.431889+0000","last_peered":"2023-10-08T00:12:10.431889+0000","last_clean":"2023-10-08T00:12:10.431889+0000","last_became_active":"2023-10-08T00:12:10.431647+0000","last_became_peered":"2023-10-08T00:12:10.431647+0000","last_unstale":"2023-10-08T00:12:10.431889+0000","last_undegraded":"2023-10-08T00:12:10.431889+0000","last_fullsized":"2023-10-08T00:12:10.431889+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:47:03.239420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.432499+0000","last_change":"2023-10-08T00:12:10.432415+0000","last_active":"2023-10-08T00:12:10.432499+0000","last_peered":"2023-10-08T00:12:10.432499+0000","last_clean":"2023-10-08T00:12:10.432499+0000","last_became_active":"2023-10-08T00:12:10.431854+0000","last_became_peered":"2023-10-08T00:12:10.431854+0000","last_unstale":"2023-10-08T00:12:10.432499+0000","last_undegraded":"2023-10-08T00:12:10.432499+0000","last_fullsized":"2023-10-08T00:12:10.432499+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:37:59.831362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.430311+0000","last_change":"2023-10-08T00:12:10.430180+0000","last_active":"2023-10-08T00:12:10.430311+0000","last_peered":"2023-10-08T00:12:10.430311+0000","last_clean":"2023-10-08T00:12:10.430311+0000","last_became_active":"2023-10-08T00:12:10.429982+0000","last_became_peered":"2023-10-08T00:12:10.429982+0000","last_unstale":"2023-10-08T00:12:10.430311+0000","last_undegraded":"2023-10-08T00:12:10.430311+0000","last_fullsized":"2023-10-08T00:12:10.430311+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:37:45.937118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.433191+0000","last_change":"2023-10-08T00:12:10.432807+0000","last_active":"2023-10-08T00:12:10.433191+0000","last_peered":"2023-10-08T00:12:10.433191+0000","last_clean":"2023-10-08T00:12:10.433191+0000","last_became_active":"2023-10-08T00:12:10.432474+0000","last_became_peered":"2023-10-08T00:12:10.432474+0000","last_unstale":"2023-10-08T00:12:10.433191+0000","last_undegraded":"2023-10-08T00:12:10.433191+0000","last_fullsized":"2023-10-08T00:12:10.433191+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:06:26.256214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.429181+0000","last_change":"2023-10-08T00:12:10.429077+0000","last_active":"2023-10-08T00:12:10.429181+0000","last_peered":"2023-10-08T00:12:10.429181+0000","last_clean":"2023-10-08T00:12:10.429181+0000","last_became_active":"2023-10-08T00:12:10.428597+0000","last_became_peered":"2023-10-08T00:12:10.428597+0000","last_unstale":"2023-10-08T00:12:10.429181+0000","last_undegraded":"2023-10-08T00:12:10.429181+0000","last_fullsized":"2023-10-08T00:12:10.429181+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:02:10.173120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.431215+0000","last_change":"2023-10-08T00:12:10.431148+0000","last_active":"2023-10-08T00:12:10.431215+0000","last_peered":"2023-10-08T00:12:10.431215+0000","last_clean":"2023-10-08T00:12:10.431215+0000","last_became_active":"2023-10-08T00:12:10.430869+0000","last_became_peered":"2023-10-08T00:12:10.430869+0000","last_unstale":"2023-10-08T00:12:10.431215+0000","last_undegraded":"2023-10-08T00:12:10.431215+0000","last_fullsized":"2023-10-08T00:12:10.431215+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:31:05.143687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.433326+0000","last_change":"2023-10-08T00:12:10.433259+0000","last_active":"2023-10-08T00:12:10.433326+0000","last_peered":"2023-10-08T00:12:10.433326+0000","last_clean":"2023-10-08T00:12:10.433326+0000","last_became_active":"2023-10-08T00:12:10.431784+0000","last_became_peered":"2023-10-08T00:12:10.431784+0000","last_unstale":"2023-10-08T00:12:10.433326+0000","last_undegraded":"2023-10-08T00:12:10.433326+0000","last_fullsized":"2023-10-08T00:12:10.433326+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:31:05.143687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.422667+0000","last_change":"2023-10-08T00:12:10.422629+0000","last_active":"2023-10-08T00:12:10.422667+0000","last_peered":"2023-10-08T00:12:10.422667+0000","last_clean":"2023-10-08T00:12:10.422667+0000","last_became_active":"2023-10-08T00:12:10.422438+0000","last_became_peered":"2023-10-08T00:12:10.422438+0000","last_unstale":"2023-10-08T00:12:10.422667+0000","last_undegraded":"2023-10-08T00:12:10.422667+0000","last_fullsized":"2023-10-08T00:12:10.422667+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:20:12.899376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.432207+0000","last_change":"2023-10-08T00:12:10.432082+0000","last_active":"2023-10-08T00:12:10.432207+0000","last_peered":"2023-10-08T00:12:10.432207+0000","last_clean":"2023-10-08T00:12:10.432207+0000","last_became_active":"2023-10-08T00:12:10.430639+0000","last_became_peered":"2023-10-08T00:12:10.430639+0000","last_unstale":"2023-10-08T00:12:10.432207+0000","last_undegraded":"2023-10-08T00:12:10.432207+0000","last_fullsized":"2023-10-08T00:12:10.432207+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:51:28.940941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.431183+0000","last_change":"2023-10-08T00:12:10.430634+0000","last_active":"2023-10-08T00:12:10.431183+0000","last_peered":"2023-10-08T00:12:10.431183+0000","last_clean":"2023-10-08T00:12:10.431183+0000","last_became_active":"2023-10-08T00:12:10.429673+0000","last_became_peered":"2023-10-08T00:12:10.429673+0000","last_unstale":"2023-10-08T00:12:10.431183+0000","last_undegraded":"2023-10-08T00:12:10.431183+0000","last_fullsized":"2023-10-08T00:12:10.431183+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:11:43.103812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.428716+0000","last_change":"2023-10-08T00:12:10.428245+0000","last_active":"2023-10-08T00:12:10.428716+0000","last_peered":"2023-10-08T00:12:10.428716+0000","last_clean":"2023-10-08T00:12:10.428716+0000","last_became_active":"2023-10-08T00:12:10.428018+0000","last_became_peered":"2023-10-08T00:12:10.428018+0000","last_unstale":"2023-10-08T00:12:10.428716+0000","last_undegraded":"2023-10-08T00:12:10.428716+0000","last_fullsized":"2023-10-08T00:12:10.428716+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:27:54.992033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.434095+0000","last_change":"2023-10-08T00:12:10.434048+0000","last_active":"2023-10-08T00:12:10.434095+0000","last_peered":"2023-10-08T00:12:10.434095+0000","last_clean":"2023-10-08T00:12:10.434095+0000","last_became_active":"2023-10-08T00:12:10.433754+0000","last_became_peered":"2023-10-08T00:12:10.433754+0000","last_unstale":"2023-10-08T00:12:10.434095+0000","last_undegraded":"2023-10-08T00:12:10.434095+0000","last_fullsized":"2023-10-08T00:12:10.434095+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:51:35.647600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.432653+0000","last_change":"2023-10-08T00:12:10.432625+0000","last_active":"2023-10-08T00:12:10.432653+0000","last_peered":"2023-10-08T00:12:10.432653+0000","last_clean":"2023-10-08T00:12:10.432653+0000","last_became_active":"2023-10-08T00:12:10.432313+0000","last_became_peered":"2023-10-08T00:12:10.432313+0000","last_unstale":"2023-10-08T00:12:10.432653+0000","last_undegraded":"2023-10-08T00:12:10.432653+0000","last_fullsized":"2023-10-08T00:12:10.432653+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:04:24.070701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.431532+0000","last_change":"2023-10-08T00:12:10.431474+0000","last_active":"2023-10-08T00:12:10.431532+0000","last_peered":"2023-10-08T00:12:10.431532+0000","last_clean":"2023-10-08T00:12:10.431532+0000","last_became_active":"2023-10-08T00:12:10.431234+0000","last_became_peered":"2023-10-08T00:12:10.431234+0000","last_unstale":"2023-10-08T00:12:10.431532+0000","last_undegraded":"2023-10-08T00:12:10.431532+0000","last_fullsized":"2023-10-08T00:12:10.431532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:23:01.656028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.430334+0000","last_change":"2023-10-08T00:12:10.430162+0000","last_active":"2023-10-08T00:12:10.430334+0000","last_peered":"2023-10-08T00:12:10.430334+0000","last_clean":"2023-10-08T00:12:10.430334+0000","last_became_active":"2023-10-08T00:12:10.429904+0000","last_became_peered":"2023-10-08T00:12:10.429904+0000","last_unstale":"2023-10-08T00:12:10.430334+0000","last_undegraded":"2023-10-08T00:12:10.430334+0000","last_fullsized":"2023-10-08T00:12:10.430334+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:48:55.441440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.426963+0000","last_change":"2023-10-08T00:12:10.426836+0000","last_active":"2023-10-08T00:12:10.426963+0000","last_peered":"2023-10-08T00:12:10.426963+0000","last_clean":"2023-10-08T00:12:10.426963+0000","last_became_active":"2023-10-08T00:12:10.426614+0000","last_became_peered":"2023-10-08T00:12:10.426614+0000","last_unstale":"2023-10-08T00:12:10.426963+0000","last_undegraded":"2023-10-08T00:12:10.426963+0000","last_fullsized":"2023-10-08T00:12:10.426963+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:11:43.103812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.433129+0000","last_change":"2023-10-08T00:12:10.432291+0000","last_active":"2023-10-08T00:12:10.433129+0000","last_peered":"2023-10-08T00:12:10.433129+0000","last_clean":"2023-10-08T00:12:10.433129+0000","last_became_active":"2023-10-08T00:12:10.430327+0000","last_became_peered":"2023-10-08T00:12:10.430327+0000","last_unstale":"2023-10-08T00:12:10.433129+0000","last_undegraded":"2023-10-08T00:12:10.433129+0000","last_fullsized":"2023-10-08T00:12:10.433129+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:47:03.239420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.428413+0000","last_change":"2023-10-08T00:12:10.428287+0000","last_active":"2023-10-08T00:12:10.428413+0000","last_peered":"2023-10-08T00:12:10.428413+0000","last_clean":"2023-10-08T00:12:10.428413+0000","last_became_active":"2023-10-08T00:12:10.428026+0000","last_became_peered":"2023-10-08T00:12:10.428026+0000","last_unstale":"2023-10-08T00:12:10.428413+0000","last_undegraded":"2023-10-08T00:12:10.428413+0000","last_fullsized":"2023-10-08T00:12:10.428413+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:43:26.869632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.431176+0000","last_change":"2023-10-08T00:12:10.431070+0000","last_active":"2023-10-08T00:12:10.431176+0000","last_peered":"2023-10-08T00:12:10.431176+0000","last_clean":"2023-10-08T00:12:10.431176+0000","last_became_active":"2023-10-08T00:12:10.430748+0000","last_became_peered":"2023-10-08T00:12:10.430748+0000","last_unstale":"2023-10-08T00:12:10.431176+0000","last_undegraded":"2023-10-08T00:12:10.431176+0000","last_fullsized":"2023-10-08T00:12:10.431176+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:18:53.588623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.427973+0000","last_change":"2023-10-08T00:12:10.427910+0000","last_active":"2023-10-08T00:12:10.427973+0000","last_peered":"2023-10-08T00:12:10.427973+0000","last_clean":"2023-10-08T00:12:10.427973+0000","last_became_active":"2023-10-08T00:12:10.427699+0000","last_became_peered":"2023-10-08T00:12:10.427699+0000","last_unstale":"2023-10-08T00:12:10.427973+0000","last_undegraded":"2023-10-08T00:12:10.427973+0000","last_fullsized":"2023-10-08T00:12:10.427973+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:44:49.192024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.430319+0000","last_change":"2023-10-08T00:12:10.430042+0000","last_active":"2023-10-08T00:12:10.430319+0000","last_peered":"2023-10-08T00:12:10.430319+0000","last_clean":"2023-10-08T00:12:10.430319+0000","last_became_active":"2023-10-08T00:12:10.429759+0000","last_became_peered":"2023-10-08T00:12:10.429759+0000","last_unstale":"2023-10-08T00:12:10.430319+0000","last_undegraded":"2023-10-08T00:12:10.430319+0000","last_fullsized":"2023-10-08T00:12:10.430319+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:43:20.037231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.433415+0000","last_change":"2023-10-08T00:12:10.433367+0000","last_active":"2023-10-08T00:12:10.433415+0000","last_peered":"2023-10-08T00:12:10.433415+0000","last_clean":"2023-10-08T00:12:10.433415+0000","last_became_active":"2023-10-08T00:12:10.432074+0000","last_became_peered":"2023-10-08T00:12:10.432074+0000","last_unstale":"2023-10-08T00:12:10.433415+0000","last_undegraded":"2023-10-08T00:12:10.433415+0000","last_fullsized":"2023-10-08T00:12:10.433415+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:04:24.070701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.427829+0000","last_change":"2023-10-08T00:12:10.427681+0000","last_active":"2023-10-08T00:12:10.427829+0000","last_peered":"2023-10-08T00:12:10.427829+0000","last_clean":"2023-10-08T00:12:10.427829+0000","last_became_active":"2023-10-08T00:12:10.427474+0000","last_became_peered":"2023-10-08T00:12:10.427474+0000","last_unstale":"2023-10-08T00:12:10.427829+0000","last_undegraded":"2023-10-08T00:12:10.427829+0000","last_fullsized":"2023-10-08T00:12:10.427829+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:59:43.392048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.423636+0000","last_change":"2023-10-08T00:12:10.423573+0000","last_active":"2023-10-08T00:12:10.423636+0000","last_peered":"2023-10-08T00:12:10.423636+0000","last_clean":"2023-10-08T00:12:10.423636+0000","last_became_active":"2023-10-08T00:12:10.423393+0000","last_became_peered":"2023-10-08T00:12:10.423393+0000","last_unstale":"2023-10-08T00:12:10.423636+0000","last_undegraded":"2023-10-08T00:12:10.423636+0000","last_fullsized":"2023-10-08T00:12:10.423636+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:10:25.717447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.431152+0000","last_change":"2023-10-08T00:12:10.431079+0000","last_active":"2023-10-08T00:12:10.431152+0000","last_peered":"2023-10-08T00:12:10.431152+0000","last_clean":"2023-10-08T00:12:10.431152+0000","last_became_active":"2023-10-08T00:12:10.430839+0000","last_became_peered":"2023-10-08T00:12:10.430839+0000","last_unstale":"2023-10-08T00:12:10.431152+0000","last_undegraded":"2023-10-08T00:12:10.431152+0000","last_fullsized":"2023-10-08T00:12:10.431152+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:20:12.899376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.423139+0000","last_change":"2023-10-08T00:12:10.423057+0000","last_active":"2023-10-08T00:12:10.423139+0000","last_peered":"2023-10-08T00:12:10.423139+0000","last_clean":"2023-10-08T00:12:10.423139+0000","last_became_active":"2023-10-08T00:12:10.422451+0000","last_became_peered":"2023-10-08T00:12:10.422451+0000","last_unstale":"2023-10-08T00:12:10.423139+0000","last_undegraded":"2023-10-08T00:12:10.423139+0000","last_fullsized":"2023-10-08T00:12:10.423139+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:48:55.441440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.432482+0000","last_change":"2023-10-08T00:12:10.432267+0000","last_active":"2023-10-08T00:12:10.432482+0000","last_peered":"2023-10-08T00:12:10.432482+0000","last_clean":"2023-10-08T00:12:10.432482+0000","last_became_active":"2023-10-08T00:12:10.431258+0000","last_became_peered":"2023-10-08T00:12:10.431258+0000","last_unstale":"2023-10-08T00:12:10.432482+0000","last_undegraded":"2023-10-08T00:12:10.432482+0000","last_fullsized":"2023-10-08T00:12:10.432482+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:37:22.198496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.427632+0000","last_change":"2023-10-08T00:12:10.425882+0000","last_active":"2023-10-08T00:12:10.427632+0000","last_peered":"2023-10-08T00:12:10.427632+0000","last_clean":"2023-10-08T00:12:10.427632+0000","last_became_active":"2023-10-08T00:12:10.425659+0000","last_became_peered":"2023-10-08T00:12:10.425659+0000","last_unstale":"2023-10-08T00:12:10.427632+0000","last_undegraded":"2023-10-08T00:12:10.427632+0000","last_fullsized":"2023-10-08T00:12:10.427632+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:06:26.932034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.428972+0000","last_change":"2023-10-08T00:12:10.428892+0000","last_active":"2023-10-08T00:12:10.428972+0000","last_peered":"2023-10-08T00:12:10.428972+0000","last_clean":"2023-10-08T00:12:10.428972+0000","last_became_active":"2023-10-08T00:12:10.428163+0000","last_became_peered":"2023-10-08T00:12:10.428163+0000","last_unstale":"2023-10-08T00:12:10.428972+0000","last_undegraded":"2023-10-08T00:12:10.428972+0000","last_fullsized":"2023-10-08T00:12:10.428972+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:28:43.542905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"44'4","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.440774+0000","last_change":"2023-10-08T00:12:09.424522+0000","last_active":"2023-10-08T00:12:10.440774+0000","last_peered":"2023-10-08T00:12:10.440774+0000","last_clean":"2023-10-08T00:12:10.440774+0000","last_became_active":"2023-10-08T00:12:09.424281+0000","last_became_peered":"2023-10-08T00:12:09.424281+0000","last_unstale":"2023-10-08T00:12:10.440774+0000","last_undegraded":"2023-10-08T00:12:10.440774+0000","last_fullsized":"2023-10-08T00:12:10.440774+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:07:18.855061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.434021+0000","last_change":"2023-10-08T00:12:10.433976+0000","last_active":"2023-10-08T00:12:10.434021+0000","last_peered":"2023-10-08T00:12:10.434021+0000","last_clean":"2023-10-08T00:12:10.434021+0000","last_became_active":"2023-10-08T00:12:10.433826+0000","last_became_peered":"2023-10-08T00:12:10.433826+0000","last_unstale":"2023-10-08T00:12:10.434021+0000","last_undegraded":"2023-10-08T00:12:10.434021+0000","last_fullsized":"2023-10-08T00:12:10.434021+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:46:32.815495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"44'1","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.438987+0000","last_change":"2023-10-08T00:12:09.421418+0000","last_active":"2023-10-08T00:12:10.438987+0000","last_peered":"2023-10-08T00:12:10.438987+0000","last_clean":"2023-10-08T00:12:10.438987+0000","last_became_active":"2023-10-08T00:12:09.421193+0000","last_became_peered":"2023-10-08T00:12:09.421193+0000","last_unstale":"2023-10-08T00:12:10.438987+0000","last_undegraded":"2023-10-08T00:12:10.438987+0000","last_fullsized":"2023-10-08T00:12:10.438987+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:20:11.895076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.422172+0000","last_change":"2023-10-08T00:12:10.422110+0000","last_active":"2023-10-08T00:12:10.422172+0000","last_peered":"2023-10-08T00:12:10.422172+0000","last_clean":"2023-10-08T00:12:10.422172+0000","last_became_active":"2023-10-08T00:12:10.421644+0000","last_became_peered":"2023-10-08T00:12:10.421644+0000","last_unstale":"2023-10-08T00:12:10.422172+0000","last_undegraded":"2023-10-08T00:12:10.422172+0000","last_fullsized":"2023-10-08T00:12:10.422172+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:55:23.292081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"44'2","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.439928+0000","last_change":"2023-10-08T00:12:09.424964+0000","last_active":"2023-10-08T00:12:10.439928+0000","last_peered":"2023-10-08T00:12:10.439928+0000","last_clean":"2023-10-08T00:12:10.439928+0000","last_became_active":"2023-10-08T00:12:09.424731+0000","last_became_peered":"2023-10-08T00:12:09.424731+0000","last_unstale":"2023-10-08T00:12:10.439928+0000","last_undegraded":"2023-10-08T00:12:10.439928+0000","last_fullsized":"2023-10-08T00:12:10.439928+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:25:57.800139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.427600+0000","last_change":"2023-10-08T00:12:10.425076+0000","last_active":"2023-10-08T00:12:10.427600+0000","last_peered":"2023-10-08T00:12:10.427600+0000","last_clean":"2023-10-08T00:12:10.427600+0000","last_became_active":"2023-10-08T00:12:10.424831+0000","last_became_peered":"2023-10-08T00:12:10.424831+0000","last_unstale":"2023-10-08T00:12:10.427600+0000","last_undegraded":"2023-10-08T00:12:10.427600+0000","last_fullsized":"2023-10-08T00:12:10.427600+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:47:09.400917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"44'2","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.439919+0000","last_change":"2023-10-08T00:12:09.420047+0000","last_active":"2023-10-08T00:12:10.439919+0000","last_peered":"2023-10-08T00:12:10.439919+0000","last_clean":"2023-10-08T00:12:10.439919+0000","last_became_active":"2023-10-08T00:12:09.419818+0000","last_became_peered":"2023-10-08T00:12:09.419818+0000","last_unstale":"2023-10-08T00:12:10.439919+0000","last_undegraded":"2023-10-08T00:12:10.439919+0000","last_fullsized":"2023-10-08T00:12:10.439919+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:51:34.643301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.428252+0000","last_change":"2023-10-08T00:12:10.428197+0000","last_active":"2023-10-08T00:12:10.428252+0000","last_peered":"2023-10-08T00:12:10.428252+0000","last_clean":"2023-10-08T00:12:10.428252+0000","last_became_active":"2023-10-08T00:12:10.427976+0000","last_became_peered":"2023-10-08T00:12:10.427976+0000","last_unstale":"2023-10-08T00:12:10.428252+0000","last_undegraded":"2023-10-08T00:12:10.428252+0000","last_fullsized":"2023-10-08T00:12:10.428252+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:24:22.363575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"44'1","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.440304+0000","last_change":"2023-10-08T00:12:09.428018+0000","last_active":"2023-10-08T00:12:10.440304+0000","last_peered":"2023-10-08T00:12:10.440304+0000","last_clean":"2023-10-08T00:12:10.440304+0000","last_became_active":"2023-10-08T00:12:09.427776+0000","last_became_peered":"2023-10-08T00:12:09.427776+0000","last_unstale":"2023-10-08T00:12:10.440304+0000","last_undegraded":"2023-10-08T00:12:10.440304+0000","last_fullsized":"2023-10-08T00:12:10.440304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:26:59.016031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.435174+0000","last_change":"2023-10-08T00:12:10.434627+0000","last_active":"2023-10-08T00:12:10.435174+0000","last_peered":"2023-10-08T00:12:10.435174+0000","last_clean":"2023-10-08T00:12:10.435174+0000","last_became_active":"2023-10-08T00:12:10.434333+0000","last_became_peered":"2023-10-08T00:12:10.434333+0000","last_unstale":"2023-10-08T00:12:10.435174+0000","last_undegraded":"2023-10-08T00:12:10.435174+0000","last_fullsized":"2023-10-08T00:12:10.435174+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:48:55.441440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"44'1","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.439088+0000","last_change":"2023-10-08T00:12:09.424860+0000","last_active":"2023-10-08T00:12:10.439088+0000","last_peered":"2023-10-08T00:12:10.439088+0000","last_clean":"2023-10-08T00:12:10.439088+0000","last_became_active":"2023-10-08T00:12:09.424472+0000","last_became_peered":"2023-10-08T00:12:09.424472+0000","last_unstale":"2023-10-08T00:12:10.439088+0000","last_undegraded":"2023-10-08T00:12:10.439088+0000","last_fullsized":"2023-10-08T00:12:10.439088+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:11:42.099513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.426925+0000","last_change":"2023-10-08T00:12:10.426264+0000","last_active":"2023-10-08T00:12:10.426925+0000","last_peered":"2023-10-08T00:12:10.426925+0000","last_clean":"2023-10-08T00:12:10.426925+0000","last_became_active":"2023-10-08T00:12:10.425754+0000","last_became_peered":"2023-10-08T00:12:10.425754+0000","last_unstale":"2023-10-08T00:12:10.426925+0000","last_undegraded":"2023-10-08T00:12:10.426925+0000","last_fullsized":"2023-10-08T00:12:10.426925+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:51:35.647600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.426463+0000","last_change":"2023-10-08T00:12:09.426297+0000","last_active":"2023-10-08T00:12:10.426463+0000","last_peered":"2023-10-08T00:12:10.426463+0000","last_clean":"2023-10-08T00:12:10.426463+0000","last_became_active":"2023-10-08T00:12:09.426024+0000","last_became_peered":"2023-10-08T00:12:09.426024+0000","last_unstale":"2023-10-08T00:12:10.426463+0000","last_undegraded":"2023-10-08T00:12:10.426463+0000","last_fullsized":"2023-10-08T00:12:10.426463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:48:52.703347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.426239+0000","last_change":"2023-10-08T00:12:10.426198+0000","last_active":"2023-10-08T00:12:10.426239+0000","last_peered":"2023-10-08T00:12:10.426239+0000","last_clean":"2023-10-08T00:12:10.426239+0000","last_became_active":"2023-10-08T00:12:10.426013+0000","last_became_peered":"2023-10-08T00:12:10.426013+0000","last_unstale":"2023-10-08T00:12:10.426239+0000","last_undegraded":"2023-10-08T00:12:10.426239+0000","last_fullsized":"2023-10-08T00:12:10.426239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:22:00.356801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.421870+0000","last_change":"2023-10-08T00:12:09.425731+0000","last_active":"2023-10-08T00:12:10.421870+0000","last_peered":"2023-10-08T00:12:10.421870+0000","last_clean":"2023-10-08T00:12:10.421870+0000","last_became_active":"2023-10-08T00:12:09.425365+0000","last_became_peered":"2023-10-08T00:12:09.425365+0000","last_unstale":"2023-10-08T00:12:10.421870+0000","last_undegraded":"2023-10-08T00:12:10.421870+0000","last_fullsized":"2023-10-08T00:12:10.421870+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:37:58.827062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.429471+0000","last_change":"2023-10-08T00:12:10.429427+0000","last_active":"2023-10-08T00:12:10.429471+0000","last_peered":"2023-10-08T00:12:10.429471+0000","last_clean":"2023-10-08T00:12:10.429471+0000","last_became_active":"2023-10-08T00:12:10.428112+0000","last_became_peered":"2023-10-08T00:12:10.428112+0000","last_unstale":"2023-10-08T00:12:10.429471+0000","last_undegraded":"2023-10-08T00:12:10.429471+0000","last_fullsized":"2023-10-08T00:12:10.429471+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:43:26.869632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.425748+0000","last_change":"2023-10-08T00:12:09.429625+0000","last_active":"2023-10-08T00:12:10.425748+0000","last_peered":"2023-10-08T00:12:10.425748+0000","last_clean":"2023-10-08T00:12:10.425748+0000","last_became_active":"2023-10-08T00:12:09.429459+0000","last_became_peered":"2023-10-08T00:12:09.429459+0000","last_unstale":"2023-10-08T00:12:10.425748+0000","last_undegraded":"2023-10-08T00:12:10.425748+0000","last_fullsized":"2023-10-08T00:12:10.425748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:48:34.458509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.431766+0000","last_change":"2023-10-08T00:12:10.431709+0000","last_active":"2023-10-08T00:12:10.431766+0000","last_peered":"2023-10-08T00:12:10.431766+0000","last_clean":"2023-10-08T00:12:10.431766+0000","last_became_active":"2023-10-08T00:12:10.431396+0000","last_became_peered":"2023-10-08T00:12:10.431396+0000","last_unstale":"2023-10-08T00:12:10.431766+0000","last_undegraded":"2023-10-08T00:12:10.431766+0000","last_fullsized":"2023-10-08T00:12:10.431766+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:54:18.156490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.422856+0000","last_change":"2023-10-08T00:12:09.423812+0000","last_active":"2023-10-08T00:12:10.422856+0000","last_peered":"2023-10-08T00:12:10.422856+0000","last_clean":"2023-10-08T00:12:10.422856+0000","last_became_active":"2023-10-08T00:12:09.423459+0000","last_became_peered":"2023-10-08T00:12:09.423459+0000","last_unstale":"2023-10-08T00:12:10.422856+0000","last_undegraded":"2023-10-08T00:12:10.422856+0000","last_fullsized":"2023-10-08T00:12:10.422856+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:37:21.194196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.431202+0000","last_change":"2023-10-08T00:12:10.430902+0000","last_active":"2023-10-08T00:12:10.431202+0000","last_peered":"2023-10-08T00:12:10.431202+0000","last_clean":"2023-10-08T00:12:10.431202+0000","last_became_active":"2023-10-08T00:12:10.430057+0000","last_became_peered":"2023-10-08T00:12:10.430057+0000","last_unstale":"2023-10-08T00:12:10.431202+0000","last_undegraded":"2023-10-08T00:12:10.431202+0000","last_fullsized":"2023-10-08T00:12:10.431202+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:20:12.899376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"44'3","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.441420+0000","last_change":"2023-10-08T00:12:09.425278+0000","last_active":"2023-10-08T00:12:10.441420+0000","last_peered":"2023-10-08T00:12:10.441420+0000","last_clean":"2023-10-08T00:12:10.441420+0000","last_became_active":"2023-10-08T00:12:09.425050+0000","last_became_peered":"2023-10-08T00:12:09.425050+0000","last_unstale":"2023-10-08T00:12:10.441420+0000","last_undegraded":"2023-10-08T00:12:10.441420+0000","last_fullsized":"2023-10-08T00:12:10.441420+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:11:29.789694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.431160+0000","last_change":"2023-10-08T00:12:10.430944+0000","last_active":"2023-10-08T00:12:10.431160+0000","last_peered":"2023-10-08T00:12:10.431160+0000","last_clean":"2023-10-08T00:12:10.431160+0000","last_became_active":"2023-10-08T00:12:10.430469+0000","last_became_peered":"2023-10-08T00:12:10.430469+0000","last_unstale":"2023-10-08T00:12:10.431160+0000","last_undegraded":"2023-10-08T00:12:10.431160+0000","last_fullsized":"2023-10-08T00:12:10.431160+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:53:03.168977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"44'6","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.443767+0000","last_change":"2023-10-08T00:12:09.423947+0000","last_active":"2023-10-08T00:12:10.443767+0000","last_peered":"2023-10-08T00:12:10.443767+0000","last_clean":"2023-10-08T00:12:10.443767+0000","last_became_active":"2023-10-08T00:12:09.423698+0000","last_became_peered":"2023-10-08T00:12:09.423698+0000","last_unstale":"2023-10-08T00:12:10.443767+0000","last_undegraded":"2023-10-08T00:12:10.443767+0000","last_fullsized":"2023-10-08T00:12:10.443767+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:06:49.684844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.434951+0000","last_change":"2023-10-08T00:12:10.434905+0000","last_active":"2023-10-08T00:12:10.434951+0000","last_peered":"2023-10-08T00:12:10.434951+0000","last_clean":"2023-10-08T00:12:10.434951+0000","last_became_active":"2023-10-08T00:12:10.434734+0000","last_became_peered":"2023-10-08T00:12:10.434734+0000","last_unstale":"2023-10-08T00:12:10.434951+0000","last_undegraded":"2023-10-08T00:12:10.434951+0000","last_fullsized":"2023-10-08T00:12:10.434951+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:27:00.020330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"44'2","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.440724+0000","last_change":"2023-10-08T00:12:09.433962+0000","last_active":"2023-10-08T00:12:10.440724+0000","last_peered":"2023-10-08T00:12:10.440724+0000","last_clean":"2023-10-08T00:12:10.440724+0000","last_became_active":"2023-10-08T00:12:09.433746+0000","last_became_peered":"2023-10-08T00:12:09.433746+0000","last_unstale":"2023-10-08T00:12:10.440724+0000","last_undegraded":"2023-10-08T00:12:10.440724+0000","last_fullsized":"2023-10-08T00:12:10.440724+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:12:29.446649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.424809+0000","last_change":"2023-10-08T00:12:10.424353+0000","last_active":"2023-10-08T00:12:10.424809+0000","last_peered":"2023-10-08T00:12:10.424809+0000","last_clean":"2023-10-08T00:12:10.424809+0000","last_became_active":"2023-10-08T00:12:10.424123+0000","last_became_peered":"2023-10-08T00:12:10.424123+0000","last_unstale":"2023-10-08T00:12:10.424809+0000","last_undegraded":"2023-10-08T00:12:10.424809+0000","last_fullsized":"2023-10-08T00:12:10.424809+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:23:01.656028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.427640+0000","last_change":"2023-10-08T00:12:09.427413+0000","last_active":"2023-10-08T00:12:10.427640+0000","last_peered":"2023-10-08T00:12:10.427640+0000","last_clean":"2023-10-08T00:12:10.427640+0000","last_became_active":"2023-10-08T00:12:09.427185+0000","last_became_peered":"2023-10-08T00:12:09.427185+0000","last_unstale":"2023-10-08T00:12:10.427640+0000","last_undegraded":"2023-10-08T00:12:10.427640+0000","last_fullsized":"2023-10-08T00:12:10.427640+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:06:49.684844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.427038+0000","last_change":"2023-10-08T00:12:10.426883+0000","last_active":"2023-10-08T00:12:10.427038+0000","last_peered":"2023-10-08T00:12:10.427038+0000","last_clean":"2023-10-08T00:12:10.427038+0000","last_became_active":"2023-10-08T00:12:10.426652+0000","last_became_peered":"2023-10-08T00:12:10.426652+0000","last_unstale":"2023-10-08T00:12:10.427038+0000","last_undegraded":"2023-10-08T00:12:10.427038+0000","last_fullsized":"2023-10-08T00:12:10.427038+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:37:19.700013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.424476+0000","last_change":"2023-10-08T00:12:09.426676+0000","last_active":"2023-10-08T00:12:10.424476+0000","last_peered":"2023-10-08T00:12:10.424476+0000","last_clean":"2023-10-08T00:12:10.424476+0000","last_became_active":"2023-10-08T00:12:09.426118+0000","last_became_peered":"2023-10-08T00:12:09.426118+0000","last_unstale":"2023-10-08T00:12:10.424476+0000","last_undegraded":"2023-10-08T00:12:10.424476+0000","last_fullsized":"2023-10-08T00:12:10.424476+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:28:42.538605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.432571+0000","last_change":"2023-10-08T00:12:10.432523+0000","last_active":"2023-10-08T00:12:10.432571+0000","last_peered":"2023-10-08T00:12:10.432571+0000","last_clean":"2023-10-08T00:12:10.432571+0000","last_became_active":"2023-10-08T00:12:10.432111+0000","last_became_peered":"2023-10-08T00:12:10.432111+0000","last_unstale":"2023-10-08T00:12:10.432571+0000","last_undegraded":"2023-10-08T00:12:10.432571+0000","last_fullsized":"2023-10-08T00:12:10.432571+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:43:20.037231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"44'8","reported_seq":18,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.443017+0000","last_change":"2023-10-08T00:12:09.419934+0000","last_active":"2023-10-08T00:12:10.443017+0000","last_peered":"2023-10-08T00:12:10.443017+0000","last_clean":"2023-10-08T00:12:10.443017+0000","last_became_active":"2023-10-08T00:12:09.419794+0000","last_became_peered":"2023-10-08T00:12:09.419794+0000","last_unstale":"2023-10-08T00:12:10.443017+0000","last_undegraded":"2023-10-08T00:12:10.443017+0000","last_fullsized":"2023-10-08T00:12:10.443017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:11:29.789694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.429153+0000","last_change":"2023-10-08T00:12:10.429095+0000","last_active":"2023-10-08T00:12:10.429153+0000","last_peered":"2023-10-08T00:12:10.429153+0000","last_clean":"2023-10-08T00:12:10.429153+0000","last_became_active":"2023-10-08T00:12:10.428708+0000","last_became_peered":"2023-10-08T00:12:10.428708+0000","last_unstale":"2023-10-08T00:12:10.429153+0000","last_undegraded":"2023-10-08T00:12:10.429153+0000","last_fullsized":"2023-10-08T00:12:10.429153+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:16:41.819876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"44'1","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.441613+0000","last_change":"2023-10-08T00:12:09.424444+0000","last_active":"2023-10-08T00:12:10.441613+0000","last_peered":"2023-10-08T00:12:10.441613+0000","last_clean":"2023-10-08T00:12:10.441613+0000","last_became_active":"2023-10-08T00:12:09.424190+0000","last_became_peered":"2023-10-08T00:12:09.424190+0000","last_unstale":"2023-10-08T00:12:10.441613+0000","last_undegraded":"2023-10-08T00:12:10.441613+0000","last_fullsized":"2023-10-08T00:12:10.441613+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:28:42.538605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.429111+0000","last_change":"2023-10-08T00:12:10.429045+0000","last_active":"2023-10-08T00:12:10.429111+0000","last_peered":"2023-10-08T00:12:10.429111+0000","last_clean":"2023-10-08T00:12:10.429111+0000","last_became_active":"2023-10-08T00:12:10.428460+0000","last_became_peered":"2023-10-08T00:12:10.428460+0000","last_unstale":"2023-10-08T00:12:10.429111+0000","last_undegraded":"2023-10-08T00:12:10.429111+0000","last_fullsized":"2023-10-08T00:12:10.429111+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:27:00.020330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"44'2","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.440653+0000","last_change":"2023-10-08T00:12:09.426927+0000","last_active":"2023-10-08T00:12:10.440653+0000","last_peered":"2023-10-08T00:12:10.440653+0000","last_clean":"2023-10-08T00:12:10.440653+0000","last_became_active":"2023-10-08T00:12:09.425736+0000","last_became_peered":"2023-10-08T00:12:09.425736+0000","last_unstale":"2023-10-08T00:12:10.440653+0000","last_undegraded":"2023-10-08T00:12:10.440653+0000","last_fullsized":"2023-10-08T00:12:10.440653+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:21:55.609798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.428506+0000","last_change":"2023-10-08T00:12:10.428442+0000","last_active":"2023-10-08T00:12:10.428506+0000","last_peered":"2023-10-08T00:12:10.428506+0000","last_clean":"2023-10-08T00:12:10.428506+0000","last_became_active":"2023-10-08T00:12:10.428203+0000","last_became_peered":"2023-10-08T00:12:10.428203+0000","last_unstale":"2023-10-08T00:12:10.428506+0000","last_undegraded":"2023-10-08T00:12:10.428506+0000","last_fullsized":"2023-10-08T00:12:10.428506+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:07:32.802880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"44'1","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.440786+0000","last_change":"2023-10-08T00:12:09.425049+0000","last_active":"2023-10-08T00:12:10.440786+0000","last_peered":"2023-10-08T00:12:10.440786+0000","last_clean":"2023-10-08T00:12:10.440786+0000","last_became_active":"2023-10-08T00:12:09.424829+0000","last_became_peered":"2023-10-08T00:12:09.424829+0000","last_unstale":"2023-10-08T00:12:10.440786+0000","last_undegraded":"2023-10-08T00:12:10.440786+0000","last_fullsized":"2023-10-08T00:12:10.440786+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:47:58.065638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.428802+0000","last_change":"2023-10-08T00:12:10.428717+0000","last_active":"2023-10-08T00:12:10.428802+0000","last_peered":"2023-10-08T00:12:10.428802+0000","last_clean":"2023-10-08T00:12:10.428802+0000","last_became_active":"2023-10-08T00:12:10.428492+0000","last_became_peered":"2023-10-08T00:12:10.428492+0000","last_unstale":"2023-10-08T00:12:10.428802+0000","last_undegraded":"2023-10-08T00:12:10.428802+0000","last_fullsized":"2023-10-08T00:12:10.428802+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:37:19.700013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.424551+0000","last_change":"2023-10-08T00:12:09.426898+0000","last_active":"2023-10-08T00:12:10.424551+0000","last_peered":"2023-10-08T00:12:10.424551+0000","last_clean":"2023-10-08T00:12:10.424551+0000","last_became_active":"2023-10-08T00:12:09.426428+0000","last_became_peered":"2023-10-08T00:12:09.426428+0000","last_unstale":"2023-10-08T00:12:10.424551+0000","last_undegraded":"2023-10-08T00:12:10.424551+0000","last_fullsized":"2023-10-08T00:12:10.424551+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:37:18.695713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.428394+0000","last_change":"2023-10-08T00:12:10.428279+0000","last_active":"2023-10-08T00:12:10.428394+0000","last_peered":"2023-10-08T00:12:10.428394+0000","last_clean":"2023-10-08T00:12:10.428394+0000","last_became_active":"2023-10-08T00:12:10.427639+0000","last_became_peered":"2023-10-08T00:12:10.427639+0000","last_unstale":"2023-10-08T00:12:10.428394+0000","last_undegraded":"2023-10-08T00:12:10.428394+0000","last_fullsized":"2023-10-08T00:12:10.428394+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:04:24.070701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"44'2","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.443407+0000","last_change":"2023-10-08T00:12:09.427750+0000","last_active":"2023-10-08T00:12:10.443407+0000","last_peered":"2023-10-08T00:12:10.443407+0000","last_clean":"2023-10-08T00:12:10.443407+0000","last_became_active":"2023-10-08T00:12:09.427539+0000","last_became_peered":"2023-10-08T00:12:09.427539+0000","last_unstale":"2023-10-08T00:12:10.443407+0000","last_undegraded":"2023-10-08T00:12:10.443407+0000","last_fullsized":"2023-10-08T00:12:10.443407+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:51:27.936642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.430988+0000","last_change":"2023-10-08T00:12:10.430858+0000","last_active":"2023-10-08T00:12:10.430988+0000","last_peered":"2023-10-08T00:12:10.430988+0000","last_clean":"2023-10-08T00:12:10.430988+0000","last_became_active":"2023-10-08T00:12:10.429897+0000","last_became_peered":"2023-10-08T00:12:10.429897+0000","last_unstale":"2023-10-08T00:12:10.430988+0000","last_undegraded":"2023-10-08T00:12:10.430988+0000","last_fullsized":"2023-10-08T00:12:10.430988+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:47:09.400917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.426356+0000","last_change":"2023-10-08T00:12:09.427687+0000","last_active":"2023-10-08T00:12:10.426356+0000","last_peered":"2023-10-08T00:12:10.426356+0000","last_clean":"2023-10-08T00:12:10.426356+0000","last_became_active":"2023-10-08T00:12:09.427146+0000","last_became_peered":"2023-10-08T00:12:09.427146+0000","last_unstale":"2023-10-08T00:12:10.426356+0000","last_undegraded":"2023-10-08T00:12:10.426356+0000","last_fullsized":"2023-10-08T00:12:10.426356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:37:21.194196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.433146+0000","last_change":"2023-10-08T00:12:10.432582+0000","last_active":"2023-10-08T00:12:10.433146+0000","last_peered":"2023-10-08T00:12:10.433146+0000","last_clean":"2023-10-08T00:12:10.433146+0000","last_became_active":"2023-10-08T00:12:10.430758+0000","last_became_peered":"2023-10-08T00:12:10.430758+0000","last_unstale":"2023-10-08T00:12:10.433146+0000","last_undegraded":"2023-10-08T00:12:10.433146+0000","last_fullsized":"2023-10-08T00:12:10.433146+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:55:23.292081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.425055+0000","last_change":"2023-10-08T00:12:09.425588+0000","last_active":"2023-10-08T00:12:10.425055+0000","last_peered":"2023-10-08T00:12:10.425055+0000","last_clean":"2023-10-08T00:12:10.425055+0000","last_became_active":"2023-10-08T00:12:09.424358+0000","last_became_peered":"2023-10-08T00:12:09.424358+0000","last_unstale":"2023-10-08T00:12:10.425055+0000","last_undegraded":"2023-10-08T00:12:10.425055+0000","last_fullsized":"2023-10-08T00:12:10.425055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:02:09.168821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.434447+0000","last_change":"2023-10-08T00:12:10.434372+0000","last_active":"2023-10-08T00:12:10.434447+0000","last_peered":"2023-10-08T00:12:10.434447+0000","last_clean":"2023-10-08T00:12:10.434447+0000","last_became_active":"2023-10-08T00:12:10.434142+0000","last_became_peered":"2023-10-08T00:12:10.434142+0000","last_unstale":"2023-10-08T00:12:10.434447+0000","last_undegraded":"2023-10-08T00:12:10.434447+0000","last_fullsized":"2023-10-08T00:12:10.434447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:51:35.647600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.420542+0000","last_change":"2023-10-08T00:12:09.423746+0000","last_active":"2023-10-08T00:12:10.420542+0000","last_peered":"2023-10-08T00:12:10.420542+0000","last_clean":"2023-10-08T00:12:10.420542+0000","last_became_active":"2023-10-08T00:12:09.423524+0000","last_became_peered":"2023-10-08T00:12:09.423524+0000","last_unstale":"2023-10-08T00:12:10.420542+0000","last_undegraded":"2023-10-08T00:12:10.420542+0000","last_fullsized":"2023-10-08T00:12:10.420542+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:46:31.811196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.426304+0000","last_change":"2023-10-08T00:12:10.426257+0000","last_active":"2023-10-08T00:12:10.426304+0000","last_peered":"2023-10-08T00:12:10.426304+0000","last_clean":"2023-10-08T00:12:10.426304+0000","last_became_active":"2023-10-08T00:12:10.426043+0000","last_became_peered":"2023-10-08T00:12:10.426043+0000","last_unstale":"2023-10-08T00:12:10.426304+0000","last_undegraded":"2023-10-08T00:12:10.426304+0000","last_fullsized":"2023-10-08T00:12:10.426304+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:13:11.108981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.421480+0000","last_change":"2023-10-08T00:12:09.424182+0000","last_active":"2023-10-08T00:12:10.421480+0000","last_peered":"2023-10-08T00:12:10.421480+0000","last_clean":"2023-10-08T00:12:10.421480+0000","last_became_active":"2023-10-08T00:12:09.423960+0000","last_became_peered":"2023-10-08T00:12:09.423960+0000","last_unstale":"2023-10-08T00:12:10.421480+0000","last_undegraded":"2023-10-08T00:12:10.421480+0000","last_fullsized":"2023-10-08T00:12:10.421480+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:25:57.800139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.426945+0000","last_change":"2023-10-08T00:12:10.426402+0000","last_active":"2023-10-08T00:12:10.426945+0000","last_peered":"2023-10-08T00:12:10.426945+0000","last_clean":"2023-10-08T00:12:10.426945+0000","last_became_active":"2023-10-08T00:12:10.426046+0000","last_became_peered":"2023-10-08T00:12:10.426046+0000","last_unstale":"2023-10-08T00:12:10.426945+0000","last_undegraded":"2023-10-08T00:12:10.426945+0000","last_fullsized":"2023-10-08T00:12:10.426945+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:20:12.899376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.426564+0000","last_change":"2023-10-08T00:12:09.428177+0000","last_active":"2023-10-08T00:12:10.426564+0000","last_peered":"2023-10-08T00:12:10.426564+0000","last_clean":"2023-10-08T00:12:10.426564+0000","last_became_active":"2023-10-08T00:12:09.427468+0000","last_became_peered":"2023-10-08T00:12:09.427468+0000","last_unstale":"2023-10-08T00:12:10.426564+0000","last_undegraded":"2023-10-08T00:12:10.426564+0000","last_fullsized":"2023-10-08T00:12:10.426564+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:02:09.168821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.429429+0000","last_change":"2023-10-08T00:12:10.429370+0000","last_active":"2023-10-08T00:12:10.429429+0000","last_peered":"2023-10-08T00:12:10.429429+0000","last_clean":"2023-10-08T00:12:10.429429+0000","last_became_active":"2023-10-08T00:12:10.429161+0000","last_became_peered":"2023-10-08T00:12:10.429161+0000","last_unstale":"2023-10-08T00:12:10.429429+0000","last_undegraded":"2023-10-08T00:12:10.429429+0000","last_fullsized":"2023-10-08T00:12:10.429429+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:49:22.374374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.421009+0000","last_change":"2023-10-08T00:12:09.432022+0000","last_active":"2023-10-08T00:12:10.421009+0000","last_peered":"2023-10-08T00:12:10.421009+0000","last_clean":"2023-10-08T00:12:10.421009+0000","last_became_active":"2023-10-08T00:12:09.431795+0000","last_became_peered":"2023-10-08T00:12:09.431795+0000","last_unstale":"2023-10-08T00:12:10.421009+0000","last_undegraded":"2023-10-08T00:12:10.421009+0000","last_fullsized":"2023-10-08T00:12:10.421009+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:29:02.607388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.427562+0000","last_change":"2023-10-08T00:12:10.427453+0000","last_active":"2023-10-08T00:12:10.427562+0000","last_peered":"2023-10-08T00:12:10.427562+0000","last_clean":"2023-10-08T00:12:10.427562+0000","last_became_active":"2023-10-08T00:12:10.427178+0000","last_became_peered":"2023-10-08T00:12:10.427178+0000","last_unstale":"2023-10-08T00:12:10.427562+0000","last_undegraded":"2023-10-08T00:12:10.427562+0000","last_fullsized":"2023-10-08T00:12:10.427562+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:07:32.802880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.427738+0000","last_change":"2023-10-08T00:12:09.433714+0000","last_active":"2023-10-08T00:12:10.427738+0000","last_peered":"2023-10-08T00:12:10.427738+0000","last_clean":"2023-10-08T00:12:10.427738+0000","last_became_active":"2023-10-08T00:12:09.433557+0000","last_became_peered":"2023-10-08T00:12:09.433557+0000","last_unstale":"2023-10-08T00:12:10.427738+0000","last_undegraded":"2023-10-08T00:12:10.427738+0000","last_fullsized":"2023-10-08T00:12:10.427738+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:23:36.280027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.427577+0000","last_change":"2023-10-08T00:12:10.424847+0000","last_active":"2023-10-08T00:12:10.427577+0000","last_peered":"2023-10-08T00:12:10.427577+0000","last_clean":"2023-10-08T00:12:10.427577+0000","last_became_active":"2023-10-08T00:12:10.424583+0000","last_became_peered":"2023-10-08T00:12:10.424583+0000","last_unstale":"2023-10-08T00:12:10.427577+0000","last_undegraded":"2023-10-08T00:12:10.427577+0000","last_fullsized":"2023-10-08T00:12:10.427577+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:48:55.441440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.421195+0000","last_change":"2023-10-08T00:12:09.421238+0000","last_active":"2023-10-08T00:12:10.421195+0000","last_peered":"2023-10-08T00:12:10.421195+0000","last_clean":"2023-10-08T00:12:10.421195+0000","last_became_active":"2023-10-08T00:12:09.421013+0000","last_became_peered":"2023-10-08T00:12:09.421013+0000","last_unstale":"2023-10-08T00:12:10.421195+0000","last_undegraded":"2023-10-08T00:12:10.421195+0000","last_fullsized":"2023-10-08T00:12:10.421195+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:31:04.139388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.429347+0000","last_change":"2023-10-08T00:12:10.429292+0000","last_active":"2023-10-08T00:12:10.429347+0000","last_peered":"2023-10-08T00:12:10.429347+0000","last_clean":"2023-10-08T00:12:10.429347+0000","last_became_active":"2023-10-08T00:12:10.429115+0000","last_became_peered":"2023-10-08T00:12:10.429115+0000","last_unstale":"2023-10-08T00:12:10.429347+0000","last_undegraded":"2023-10-08T00:12:10.429347+0000","last_fullsized":"2023-10-08T00:12:10.429347+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:28:43.542905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.423984+0000","last_change":"2023-10-08T00:12:09.426097+0000","last_active":"2023-10-08T00:12:10.423984+0000","last_peered":"2023-10-08T00:12:10.423984+0000","last_clean":"2023-10-08T00:12:10.423984+0000","last_became_active":"2023-10-08T00:12:09.425841+0000","last_became_peered":"2023-10-08T00:12:09.425841+0000","last_unstale":"2023-10-08T00:12:10.423984+0000","last_undegraded":"2023-10-08T00:12:10.423984+0000","last_fullsized":"2023-10-08T00:12:10.423984+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:23:00.651728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.428491+0000","last_change":"2023-10-08T00:12:10.428415+0000","last_active":"2023-10-08T00:12:10.428491+0000","last_peered":"2023-10-08T00:12:10.428491+0000","last_clean":"2023-10-08T00:12:10.428491+0000","last_became_active":"2023-10-08T00:12:10.428178+0000","last_became_peered":"2023-10-08T00:12:10.428178+0000","last_unstale":"2023-10-08T00:12:10.428491+0000","last_undegraded":"2023-10-08T00:12:10.428491+0000","last_fullsized":"2023-10-08T00:12:10.428491+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:47:03.239420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.420449+0000","last_change":"2023-10-08T00:12:09.432071+0000","last_active":"2023-10-08T00:12:10.420449+0000","last_peered":"2023-10-08T00:12:10.420449+0000","last_clean":"2023-10-08T00:12:10.420449+0000","last_became_active":"2023-10-08T00:12:09.431833+0000","last_became_peered":"2023-10-08T00:12:09.431833+0000","last_unstale":"2023-10-08T00:12:10.420449+0000","last_undegraded":"2023-10-08T00:12:10.420449+0000","last_fullsized":"2023-10-08T00:12:10.420449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:40:35.508089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.433161+0000","last_change":"2023-10-08T00:12:10.432681+0000","last_active":"2023-10-08T00:12:10.433161+0000","last_peered":"2023-10-08T00:12:10.433161+0000","last_clean":"2023-10-08T00:12:10.433161+0000","last_became_active":"2023-10-08T00:12:10.432127+0000","last_became_peered":"2023-10-08T00:12:10.432127+0000","last_unstale":"2023-10-08T00:12:10.433161+0000","last_undegraded":"2023-10-08T00:12:10.433161+0000","last_fullsized":"2023-10-08T00:12:10.433161+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:07:32.802880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"44'2","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.439343+0000","last_change":"2023-10-08T00:12:09.425632+0000","last_active":"2023-10-08T00:12:10.439343+0000","last_peered":"2023-10-08T00:12:10.439343+0000","last_clean":"2023-10-08T00:12:10.439343+0000","last_became_active":"2023-10-08T00:12:09.425084+0000","last_became_peered":"2023-10-08T00:12:09.425084+0000","last_unstale":"2023-10-08T00:12:10.439343+0000","last_undegraded":"2023-10-08T00:12:10.439343+0000","last_fullsized":"2023-10-08T00:12:10.439343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:47:02.235120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":181,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.424983+0000","last_change":"2023-10-08T00:10:43.728490+0000","last_active":"2023-10-08T00:12:10.424983+0000","last_peered":"2023-10-08T00:12:10.424983+0000","last_clean":"2023-10-08T00:12:10.424983+0000","last_became_active":"2023-10-08T00:10:43.728341+0000","last_became_peered":"2023-10-08T00:10:43.728341+0000","last_unstale":"2023-10-08T00:12:10.424983+0000","last_undegraded":"2023-10-08T00:12:10.424983+0000","last_fullsized":"2023-10-08T00:12:10.424983+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_clean_scrub_stamp":"2023-10-08T00:10:42.716096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:06:31.035293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":208,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195103,"statfs":{"total":200000000,"available":199786239,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.45500000000000002}]},{"osd":1,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.53000000000000003}]},{"osd":2,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.497},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.61899999999999999}]},{"osd":3,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.65500000000000003}]},{"osd":4,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.58199999999999996}]},{"osd":5,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.69399999999999995}]},{"osd":6,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.63300000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":209,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195103,"statfs":{"total":200000000,"available":199785685,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":5.1230000000000002,"5min":5.1230000000000002,"15min":5.1230000000000002},"last":0.54500000000000004}]},{"osd":1,"last update":"Sun Oct 8 00:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.60399999999999998}]},{"osd":2,"last update":"Sun Oct 8 00:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.64200000000000002}]},{"osd":3,"last update":"Sun Oct 8 00:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.65300000000000002}]},{"osd":4,"last update":"Sun Oct 8 00:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.379},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.51500000000000001}]},{"osd":5,"last update":"Sun Oct 8 00:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.433}]},{"osd":7,"last update":"Sun Oct 8 00:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.501}]}]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":785,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194527,"statfs":{"total":200000000,"available":199195835,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.34499999999999997}]},{"osd":2,"last update":"Sun Oct 8 00:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.442,"5min":0.442,"15min":0.442},"last":0.36599999999999999}]},{"osd":3,"last update":"Sun Oct 8 00:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.497,"5min":0.497,"15min":0.497},"last":0.26900000000000002}]},{"osd":4,"last update":"Sun Oct 8 00:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.41899999999999998}]},{"osd":5,"last update":"Sun Oct 8 00:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.42999999999999999}]},{"osd":6,"last update":"Sun Oct 8 00:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.45400000000000001}]},{"osd":7,"last update":"Sun Oct 8 00:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.46500000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":209,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195103,"statfs":{"total":200000000,"available":199785629,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 00:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.247},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.33200000000000002}]},{"osd":2,"last update":"Sun Oct 8 00:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.185,"5min":0.185,"15min":0.185},"max":{"1min":12.019,"5min":12.019,"15min":12.019},"last":0.26200000000000001}]},{"osd":3,"last update":"Sun Oct 8 00:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.35699999999999998}]},{"osd":4,"last update":"Sun Oct 8 00:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.55400000000000005}]},{"osd":5,"last update":"Sun Oct 8 00:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":11.992000000000001,"5min":11.992000000000001,"15min":11.992000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.52600000000000002}]},{"osd":6,"last update":"Sun Oct 8 00:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.45400000000000001},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":11.927,"5min":11.927,"15min":11.927},"last":0.51700000000000002}]},{"osd":7,"last update":"Sun Oct 8 00:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":11.965,"5min":11.965,"15min":11.965},"last":0.5},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.58599999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":208,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195103,"statfs":{"total":200000000,"available":199786183,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":2.5139999999999998,"5min":2.5139999999999998,"15min":2.5139999999999998},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":2.1309999999999998,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":0.26600000000000001}]},{"osd":1,"last update":"Sun Oct 8 00:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":2.2639999999999998,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":0.25},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":2.5379999999999998,"5min":2.5379999999999998,"15min":2.5379999999999998},"last":0.29999999999999999}]},{"osd":3,"last update":"Sun Oct 8 00:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":2.2890000000000001,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":2.5619999999999998,"5min":2.5619999999999998,"15min":2.5619999999999998},"last":0.32500000000000001}]},{"osd":4,"last update":"Sun Oct 8 00:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.488,"5min":2.488,"15min":2.488},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":2.073,"5min":2.073,"15min":2.073},"last":0.57899999999999996}]},{"osd":5,"last update":"Sun Oct 8 00:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":2.2069999999999999,"5min":2.2069999999999999,"15min":2.2069999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.456,"5min":2.456,"15min":2.456},"last":0.71999999999999997}]},{"osd":6,"last update":"Sun Oct 8 00:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":2.0339999999999998,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.53000000000000003}]},{"osd":7,"last update":"Sun Oct 8 00:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":2.1030000000000002,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.2349999999999999,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":0.56399999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":210,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195102,"statfs":{"total":200000000,"available":199784737,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.251}]},{"osd":1,"last update":"Sun Oct 8 00:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":12.351000000000001,"5min":12.351000000000001,"15min":12.351000000000001},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.32600000000000001}]},{"osd":2,"last update":"Sun Oct 8 00:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.307}]},{"osd":4,"last update":"Sun Oct 8 00:11:41 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":11.333,"5min":11.333,"15min":11.333},"last":0.59699999999999998},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":12.321999999999999,"5min":12.321999999999999,"15min":12.321999999999999},"last":0.58899999999999997}]},{"osd":5,"last update":"Sun Oct 8 00:11:41 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.181,"15min":1.181},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":12.291,"5min":12.291,"15min":12.291},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.46100000000000002}]},{"osd":6,"last update":"Sun Oct 8 00:11:41 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":11.288,"5min":11.288,"15min":11.288},"last":0.52800000000000002},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":12.253,"5min":12.253,"15min":12.253},"last":0.40500000000000003}]},{"osd":7,"last update":"Sun Oct 8 00:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.48399999999999999},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":11.363,"5min":11.363,"15min":11.363},"last":0.44900000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":210,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195102,"statfs":{"total":200000000,"available":199784645,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.51200000000000001}]},{"osd":1,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.56399999999999995}]},{"osd":2,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.55000000000000004}]},{"osd":3,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.57299999999999995}]},{"osd":5,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.433},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.46400000000000002}]},{"osd":6,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.377}]},{"osd":7,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.53400000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":33,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":785,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194527,"statfs":{"total":200000000,"available":199195883,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.5},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.433}]},{"osd":1,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.41299999999999998}]},{"osd":2,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.53300000000000003}]},{"osd":3,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.56499999999999995}]},{"osd":4,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.54700000000000004}]},{"osd":6,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.52500000000000002}]},{"osd":7,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.61799999999999999}]}]}],"pool_statfs":[]}} 2023-10-08T00:12:14.475 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T00:12:14.622 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.621+0000 7f2aa4659700 1 Processor -- start 2023-10-08T00:12:14.622 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.621+0000 7f2aa4659700 1 -- start start 2023-10-08T00:12:14.623 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.622+0000 7f2aa4659700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2a9c157870 0x7f2a9c15b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:14.624 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.622+0000 7f2aa4659700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2a9c157c40 0x7f2a9c158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:14.624 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.622+0000 7f2aa4659700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a9c15cbb0 con 0x7f2a9c15a620 2023-10-08T00:12:14.624 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.622+0000 7f2aa4659700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a9c15bfa0 con 0x7f2a9c157c40 2023-10-08T00:12:14.624 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.622+0000 7f2aa4659700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a9c15c280 con 0x7f2a9c157870 2023-10-08T00:12:14.624 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.622+0000 7f2aa2bf6700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2a9c157c40 0x7f2a9c158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:14.624 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.622+0000 7f2aa1bf4700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f2a9c15a620 0x7f2a9c15d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45128/0 (socket says 172.21.15.99:45128) 2023-10-08T00:12:14.624 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.622+0000 7f2aa1bf4700 1 -- 172.21.15.99:0/1639462157 learned_addr learned my addr 172.21.15.99:0/1639462157 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:12:14.624 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.623+0000 7f2aa23f5700 1 --2- 172.21.15.99:0/1639462157 >> v2:172.21.15.191:3300/0 conn(0x7f2a9c157870 0x7f2a9c15b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:14.624 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.623+0000 7f2aa2bf6700 1 -- 172.21.15.99:0/1639462157 >> v2:172.21.15.191:3300/0 conn(0x7f2a9c157870 msgr2=0x7f2a9c15b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:14.624 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.623+0000 7f2aa2bf6700 1 --2- 172.21.15.99:0/1639462157 >> v2:172.21.15.191:3300/0 conn(0x7f2a9c157870 0x7f2a9c15b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:14.624 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.623+0000 7f2aa2bf6700 1 -- 172.21.15.99:0/1639462157 >> v1:172.21.15.99:6789/0 conn(0x7f2a9c15a620 legacy=0x7f2a9c15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:14.625 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.623+0000 7f2aa2bf6700 1 -- 172.21.15.99:0/1639462157 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a9c15c560 con 0x7f2a9c157c40 2023-10-08T00:12:14.625 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.623+0000 7f2aa2bf6700 1 --2- 172.21.15.99:0/1639462157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a9c157c40 0x7f2a9c158110 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c069d80 tx=0x7f2a94009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39bc0c6ceab81d07 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:14.625 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.623+0000 7f2aa13f3700 1 -- 172.21.15.99:0/1639462157 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9400e070 con 0x7f2a9c157c40 2023-10-08T00:12:14.625 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.623+0000 7f2aa13f3700 1 -- 172.21.15.99:0/1639462157 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a9400bc30 con 0x7f2a9c157c40 2023-10-08T00:12:14.625 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.623+0000 7f2aa13f3700 1 -- 172.21.15.99:0/1639462157 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a94012430 con 0x7f2a9c157c40 2023-10-08T00:12:14.628 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.623+0000 7f2aa4659700 1 -- 172.21.15.99:0/1639462157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a9c157c40 msgr2=0x7f2a9c158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:14.628 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.623+0000 7f2aa4659700 1 --2- 172.21.15.99:0/1639462157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a9c157c40 0x7f2a9c158110 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c069d80 tx=0x7f2a94009780 comp rx=0 tx=0).stop 2023-10-08T00:12:14.628 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.624+0000 7f2aa4659700 1 -- 172.21.15.99:0/1639462157 shutdown_connections 2023-10-08T00:12:14.628 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.624+0000 7f2aa4659700 1 --2- 172.21.15.99:0/1639462157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a9c157c40 0x7f2a9c158110 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:14.628 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.624+0000 7f2aa4659700 1 --2- 172.21.15.99:0/1639462157 >> v2:172.21.15.191:3300/0 conn(0x7f2a9c157870 0x7f2a9c15b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:14.628 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.624+0000 7f2aa4659700 1 -- 172.21.15.99:0/1639462157 >> 172.21.15.99:0/1639462157 conn(0x7f2a9c068ce0 msgr2=0x7f2a9c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:14.628 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.624+0000 7f2aa4659700 1 -- 172.21.15.99:0/1639462157 shutdown_connections 2023-10-08T00:12:14.628 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.624+0000 7f2aa4659700 1 -- 172.21.15.99:0/1639462157 wait complete. 2023-10-08T00:12:14.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.625+0000 7f2aa4659700 1 Processor -- start 2023-10-08T00:12:14.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.625+0000 7f2aa4659700 1 -- start start 2023-10-08T00:12:14.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.625+0000 7f2aa4659700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a9c157870 0x7f2a9c13f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:14.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.625+0000 7f2aa4659700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a9c157c40 0x7f2a9c14f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:14.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.625+0000 7f2aa4659700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a9c15a620 0x7f2a9c0b1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:14.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.625+0000 7f2aa4659700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9c0b1400 con 0x7f2a9c157870 2023-10-08T00:12:14.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.625+0000 7f2aa4659700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9c0b07f0 con 0x7f2a9c15a620 2023-10-08T00:12:14.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.625+0000 7f2aa4659700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9c0b0ad0 con 0x7f2a9c157c40 2023-10-08T00:12:14.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.626+0000 7f2aa23f5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a9c157870 0x7f2a9c13f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:14.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.626+0000 7f2aa23f5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a9c157870 0x7f2a9c13f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44578/0 (socket says 172.21.15.99:44578) 2023-10-08T00:12:14.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.626+0000 7f2aa23f5700 1 -- 172.21.15.99:0/214175392 learned_addr learned my addr 172.21.15.99:0/214175392 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:14.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.626+0000 7f2aa1bf4700 1 --2- 172.21.15.99:0/214175392 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a9c157c40 0x7f2a9c14f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:14.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.626+0000 7f2aa2bf6700 1 --2- 172.21.15.99:0/214175392 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a9c15a620 0x7f2a9c0b1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:14.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.626+0000 7f2aa23f5700 1 -- 172.21.15.99:0/214175392 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a9c157c40 msgr2=0x7f2a9c14f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:14.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.626+0000 7f2aa23f5700 1 --2- 172.21.15.99:0/214175392 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a9c157c40 0x7f2a9c14f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:14.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.626+0000 7f2aa23f5700 1 -- 172.21.15.99:0/214175392 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a9c15a620 msgr2=0x7f2a9c0b1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:14.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.626+0000 7f2aa23f5700 1 --2- 172.21.15.99:0/214175392 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a9c15a620 0x7f2a9c0b1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:14.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.626+0000 7f2aa23f5700 1 -- 172.21.15.99:0/214175392 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a94009a20 con 0x7f2a9c157870 2023-10-08T00:12:14.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.626+0000 7f2aa1bf4700 1 --2- 172.21.15.99:0/214175392 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a9c157c40 0x7f2a9c14f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:12:14.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.626+0000 7f2aa2bf6700 1 --2- 172.21.15.99:0/214175392 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a9c15a620 0x7f2a9c0b1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:12:14.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.626+0000 7f2aa23f5700 1 --2- 172.21.15.99:0/214175392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a9c157870 0x7f2a9c13f920 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2a9800c7d0 tx=0x7f2a98008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:14.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.626+0000 7f2a927fc700 1 -- 172.21.15.99:0/214175392 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a98011600 con 0x7f2a9c157870 2023-10-08T00:12:14.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.626+0000 7f2a927fc700 1 -- 172.21.15.99:0/214175392 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a98011760 con 0x7f2a9c157870 2023-10-08T00:12:14.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.626+0000 7f2a927fc700 1 -- 172.21.15.99:0/214175392 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a98011b20 con 0x7f2a9c157870 2023-10-08T00:12:14.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.626+0000 7f2aa4659700 1 -- 172.21.15.99:0/214175392 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a9c0b0db0 con 0x7f2a9c157870 2023-10-08T00:12:14.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.626+0000 7f2aa4659700 1 -- 172.21.15.99:0/214175392 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a9c14ff20 con 0x7f2a9c157870 2023-10-08T00:12:14.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.627+0000 7f2aa4659700 1 -- 172.21.15.99:0/214175392 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a9c04c3c0 con 0x7f2a9c157870 2023-10-08T00:12:14.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.628+0000 7f2a927fc700 1 -- 172.21.15.99:0/214175392 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2a980193f0 con 0x7f2a9c157870 2023-10-08T00:12:14.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.628+0000 7f2a927fc700 1 --2- 172.21.15.99:0/214175392 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2a80066f30 0x7f2a800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:14.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.628+0000 7f2a927fc700 1 -- 172.21.15.99:0/214175392 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2a98084930 con 0x7f2a9c157870 2023-10-08T00:12:14.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.628+0000 7f2aa1bf4700 1 --2- 172.21.15.99:0/214175392 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2a80066f30 0x7f2a800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:14.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.628+0000 7f2aa1bf4700 1 --2- 172.21.15.99:0/214175392 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2a80066f30 0x7f2a800693e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c15b460 tx=0x7f2a8c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:14.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.632+0000 7f2a927fc700 1 -- 172.21.15.99:0/214175392 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2a98051ba0 con 0x7f2a9c157870 2023-10-08T00:12:14.809 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.809+0000 7f2aa4659700 1 -- 172.21.15.99:0/214175392 --> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2a9c0b1540 con 0x7f2a80066f30 2023-10-08T00:12:14.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.817+0000 7f2a927fc700 1 -- 172.21.15.99:0/214175392 <== mgr.4099 v2:172.21.15.191:6832/85985 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284974 (secure 0 0 0) 0x7f2a8c005900 con 0x7f2a80066f30 2023-10-08T00:12:14.821 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:12:14.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.820+0000 7f2aa4659700 1 -- 172.21.15.99:0/214175392 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2a80066f30 msgr2=0x7f2a800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:14.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.820+0000 7f2aa4659700 1 --2- 172.21.15.99:0/214175392 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2a80066f30 0x7f2a800693e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c15b460 tx=0x7f2a8c009040 comp rx=0 tx=0).stop 2023-10-08T00:12:14.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.820+0000 7f2aa4659700 1 -- 172.21.15.99:0/214175392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a9c157870 msgr2=0x7f2a9c13f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:14.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.820+0000 7f2aa4659700 1 --2- 172.21.15.99:0/214175392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a9c157870 0x7f2a9c13f920 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2a9800c7d0 tx=0x7f2a98008040 comp rx=0 tx=0).stop 2023-10-08T00:12:14.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.820+0000 7f2aa4659700 1 -- 172.21.15.99:0/214175392 shutdown_connections 2023-10-08T00:12:14.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.820+0000 7f2aa4659700 1 --2- 172.21.15.99:0/214175392 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2a80066f30 0x7f2a800693e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:14.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.820+0000 7f2aa4659700 1 --2- 172.21.15.99:0/214175392 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a9c15a620 0x7f2a9c0b1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:14.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.820+0000 7f2aa4659700 1 --2- 172.21.15.99:0/214175392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a9c157870 0x7f2a9c13f920 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:14.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.820+0000 7f2aa4659700 1 --2- 172.21.15.99:0/214175392 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a9c157c40 0x7f2a9c14f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:14.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.820+0000 7f2aa4659700 1 -- 172.21.15.99:0/214175392 >> 172.21.15.99:0/214175392 conn(0x7f2a9c068ce0 msgr2=0x7f2a9c0b46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:14.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.820+0000 7f2aa4659700 1 -- 172.21.15.99:0/214175392 shutdown_connections 2023-10-08T00:12:14.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.820+0000 7f2aa4659700 1 -- 172.21.15.99:0/214175392 wait complete. 2023-10-08T00:12:14.825 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2023-10-08T00:12:14.832 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":81,"stamp":"2023-10-08T00:12:14.042081+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":142,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2827,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1559672,"statfs":{"total":1600000000,"available":1597104836,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.630035"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.430352+0000","last_change":"2023-10-08T00:12:10.430219+0000","last_active":"2023-10-08T00:12:10.430352+0000","last_peered":"2023-10-08T00:12:10.430352+0000","last_clean":"2023-10-08T00:12:10.430352+0000","last_became_active":"2023-10-08T00:12:10.429880+0000","last_became_peered":"2023-10-08T00:12:10.429880+0000","last_unstale":"2023-10-08T00:12:10.430352+0000","last_undegraded":"2023-10-08T00:12:10.430352+0000","last_fullsized":"2023-10-08T00:12:10.430352+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:04:24.070701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.424244+0000","last_change":"2023-10-08T00:12:10.424199+0000","last_active":"2023-10-08T00:12:10.424244+0000","last_peered":"2023-10-08T00:12:10.424244+0000","last_clean":"2023-10-08T00:12:10.424244+0000","last_became_active":"2023-10-08T00:12:10.424021+0000","last_became_peered":"2023-10-08T00:12:10.424021+0000","last_unstale":"2023-10-08T00:12:10.424244+0000","last_undegraded":"2023-10-08T00:12:10.424244+0000","last_fullsized":"2023-10-08T00:12:10.424244+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:46:32.815495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.429530+0000","last_change":"2023-10-08T00:12:10.429486+0000","last_active":"2023-10-08T00:12:10.429530+0000","last_peered":"2023-10-08T00:12:10.429530+0000","last_clean":"2023-10-08T00:12:10.429530+0000","last_became_active":"2023-10-08T00:12:10.428910+0000","last_became_peered":"2023-10-08T00:12:10.428910+0000","last_unstale":"2023-10-08T00:12:10.429530+0000","last_undegraded":"2023-10-08T00:12:10.429530+0000","last_fullsized":"2023-10-08T00:12:10.429530+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:16:41.819876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.428735+0000","last_change":"2023-10-08T00:12:10.428507+0000","last_active":"2023-10-08T00:12:10.428735+0000","last_peered":"2023-10-08T00:12:10.428735+0000","last_clean":"2023-10-08T00:12:10.428735+0000","last_became_active":"2023-10-08T00:12:10.428162+0000","last_became_peered":"2023-10-08T00:12:10.428162+0000","last_unstale":"2023-10-08T00:12:10.428735+0000","last_undegraded":"2023-10-08T00:12:10.428735+0000","last_fullsized":"2023-10-08T00:12:10.428735+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:46:32.815495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.431889+0000","last_change":"2023-10-08T00:12:10.431827+0000","last_active":"2023-10-08T00:12:10.431889+0000","last_peered":"2023-10-08T00:12:10.431889+0000","last_clean":"2023-10-08T00:12:10.431889+0000","last_became_active":"2023-10-08T00:12:10.431647+0000","last_became_peered":"2023-10-08T00:12:10.431647+0000","last_unstale":"2023-10-08T00:12:10.431889+0000","last_undegraded":"2023-10-08T00:12:10.431889+0000","last_fullsized":"2023-10-08T00:12:10.431889+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:47:03.239420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.432499+0000","last_change":"2023-10-08T00:12:10.432415+0000","last_active":"2023-10-08T00:12:10.432499+0000","last_peered":"2023-10-08T00:12:10.432499+0000","last_clean":"2023-10-08T00:12:10.432499+0000","last_became_active":"2023-10-08T00:12:10.431854+0000","last_became_peered":"2023-10-08T00:12:10.431854+0000","last_unstale":"2023-10-08T00:12:10.432499+0000","last_undegraded":"2023-10-08T00:12:10.432499+0000","last_fullsized":"2023-10-08T00:12:10.432499+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:37:59.831362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.430311+0000","last_change":"2023-10-08T00:12:10.430180+0000","last_active":"2023-10-08T00:12:10.430311+0000","last_peered":"2023-10-08T00:12:10.430311+0000","last_clean":"2023-10-08T00:12:10.430311+0000","last_became_active":"2023-10-08T00:12:10.429982+0000","last_became_peered":"2023-10-08T00:12:10.429982+0000","last_unstale":"2023-10-08T00:12:10.430311+0000","last_undegraded":"2023-10-08T00:12:10.430311+0000","last_fullsized":"2023-10-08T00:12:10.430311+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:37:45.937118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.433191+0000","last_change":"2023-10-08T00:12:10.432807+0000","last_active":"2023-10-08T00:12:10.433191+0000","last_peered":"2023-10-08T00:12:10.433191+0000","last_clean":"2023-10-08T00:12:10.433191+0000","last_became_active":"2023-10-08T00:12:10.432474+0000","last_became_peered":"2023-10-08T00:12:10.432474+0000","last_unstale":"2023-10-08T00:12:10.433191+0000","last_undegraded":"2023-10-08T00:12:10.433191+0000","last_fullsized":"2023-10-08T00:12:10.433191+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:06:26.256214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.429181+0000","last_change":"2023-10-08T00:12:10.429077+0000","last_active":"2023-10-08T00:12:10.429181+0000","last_peered":"2023-10-08T00:12:10.429181+0000","last_clean":"2023-10-08T00:12:10.429181+0000","last_became_active":"2023-10-08T00:12:10.428597+0000","last_became_peered":"2023-10-08T00:12:10.428597+0000","last_unstale":"2023-10-08T00:12:10.429181+0000","last_undegraded":"2023-10-08T00:12:10.429181+0000","last_fullsized":"2023-10-08T00:12:10.429181+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:02:10.173120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.431215+0000","last_change":"2023-10-08T00:12:10.431148+0000","last_active":"2023-10-08T00:12:10.431215+0000","last_peered":"2023-10-08T00:12:10.431215+0000","last_clean":"2023-10-08T00:12:10.431215+0000","last_became_active":"2023-10-08T00:12:10.430869+0000","last_became_peered":"2023-10-08T00:12:10.430869+0000","last_unstale":"2023-10-08T00:12:10.431215+0000","last_undegraded":"2023-10-08T00:12:10.431215+0000","last_fullsized":"2023-10-08T00:12:10.431215+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:31:05.143687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.433326+0000","last_change":"2023-10-08T00:12:10.433259+0000","last_active":"2023-10-08T00:12:10.433326+0000","last_peered":"2023-10-08T00:12:10.433326+0000","last_clean":"2023-10-08T00:12:10.433326+0000","last_became_active":"2023-10-08T00:12:10.431784+0000","last_became_peered":"2023-10-08T00:12:10.431784+0000","last_unstale":"2023-10-08T00:12:10.433326+0000","last_undegraded":"2023-10-08T00:12:10.433326+0000","last_fullsized":"2023-10-08T00:12:10.433326+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:31:05.143687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.422667+0000","last_change":"2023-10-08T00:12:10.422629+0000","last_active":"2023-10-08T00:12:10.422667+0000","last_peered":"2023-10-08T00:12:10.422667+0000","last_clean":"2023-10-08T00:12:10.422667+0000","last_became_active":"2023-10-08T00:12:10.422438+0000","last_became_peered":"2023-10-08T00:12:10.422438+0000","last_unstale":"2023-10-08T00:12:10.422667+0000","last_undegraded":"2023-10-08T00:12:10.422667+0000","last_fullsized":"2023-10-08T00:12:10.422667+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:20:12.899376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.432207+0000","last_change":"2023-10-08T00:12:10.432082+0000","last_active":"2023-10-08T00:12:10.432207+0000","last_peered":"2023-10-08T00:12:10.432207+0000","last_clean":"2023-10-08T00:12:10.432207+0000","last_became_active":"2023-10-08T00:12:10.430639+0000","last_became_peered":"2023-10-08T00:12:10.430639+0000","last_unstale":"2023-10-08T00:12:10.432207+0000","last_undegraded":"2023-10-08T00:12:10.432207+0000","last_fullsized":"2023-10-08T00:12:10.432207+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:51:28.940941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.431183+0000","last_change":"2023-10-08T00:12:10.430634+0000","last_active":"2023-10-08T00:12:10.431183+0000","last_peered":"2023-10-08T00:12:10.431183+0000","last_clean":"2023-10-08T00:12:10.431183+0000","last_became_active":"2023-10-08T00:12:10.429673+0000","last_became_peered":"2023-10-08T00:12:10.429673+0000","last_unstale":"2023-10-08T00:12:10.431183+0000","last_undegraded":"2023-10-08T00:12:10.431183+0000","last_fullsized":"2023-10-08T00:12:10.431183+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:11:43.103812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.428716+0000","last_change":"2023-10-08T00:12:10.428245+0000","last_active":"2023-10-08T00:12:10.428716+0000","last_peered":"2023-10-08T00:12:10.428716+0000","last_clean":"2023-10-08T00:12:10.428716+0000","last_became_active":"2023-10-08T00:12:10.428018+0000","last_became_peered":"2023-10-08T00:12:10.428018+0000","last_unstale":"2023-10-08T00:12:10.428716+0000","last_undegraded":"2023-10-08T00:12:10.428716+0000","last_fullsized":"2023-10-08T00:12:10.428716+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:27:54.992033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.434095+0000","last_change":"2023-10-08T00:12:10.434048+0000","last_active":"2023-10-08T00:12:10.434095+0000","last_peered":"2023-10-08T00:12:10.434095+0000","last_clean":"2023-10-08T00:12:10.434095+0000","last_became_active":"2023-10-08T00:12:10.433754+0000","last_became_peered":"2023-10-08T00:12:10.433754+0000","last_unstale":"2023-10-08T00:12:10.434095+0000","last_undegraded":"2023-10-08T00:12:10.434095+0000","last_fullsized":"2023-10-08T00:12:10.434095+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:51:35.647600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.432653+0000","last_change":"2023-10-08T00:12:10.432625+0000","last_active":"2023-10-08T00:12:10.432653+0000","last_peered":"2023-10-08T00:12:10.432653+0000","last_clean":"2023-10-08T00:12:10.432653+0000","last_became_active":"2023-10-08T00:12:10.432313+0000","last_became_peered":"2023-10-08T00:12:10.432313+0000","last_unstale":"2023-10-08T00:12:10.432653+0000","last_undegraded":"2023-10-08T00:12:10.432653+0000","last_fullsized":"2023-10-08T00:12:10.432653+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:04:24.070701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.431532+0000","last_change":"2023-10-08T00:12:10.431474+0000","last_active":"2023-10-08T00:12:10.431532+0000","last_peered":"2023-10-08T00:12:10.431532+0000","last_clean":"2023-10-08T00:12:10.431532+0000","last_became_active":"2023-10-08T00:12:10.431234+0000","last_became_peered":"2023-10-08T00:12:10.431234+0000","last_unstale":"2023-10-08T00:12:10.431532+0000","last_undegraded":"2023-10-08T00:12:10.431532+0000","last_fullsized":"2023-10-08T00:12:10.431532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:23:01.656028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.430334+0000","last_change":"2023-10-08T00:12:10.430162+0000","last_active":"2023-10-08T00:12:10.430334+0000","last_peered":"2023-10-08T00:12:10.430334+0000","last_clean":"2023-10-08T00:12:10.430334+0000","last_became_active":"2023-10-08T00:12:10.429904+0000","last_became_peered":"2023-10-08T00:12:10.429904+0000","last_unstale":"2023-10-08T00:12:10.430334+0000","last_undegraded":"2023-10-08T00:12:10.430334+0000","last_fullsized":"2023-10-08T00:12:10.430334+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:48:55.441440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.426963+0000","last_change":"2023-10-08T00:12:10.426836+0000","last_active":"2023-10-08T00:12:10.426963+0000","last_peered":"2023-10-08T00:12:10.426963+0000","last_clean":"2023-10-08T00:12:10.426963+0000","last_became_active":"2023-10-08T00:12:10.426614+0000","last_became_peered":"2023-10-08T00:12:10.426614+0000","last_unstale":"2023-10-08T00:12:10.426963+0000","last_undegraded":"2023-10-08T00:12:10.426963+0000","last_fullsized":"2023-10-08T00:12:10.426963+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:11:43.103812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.433129+0000","last_change":"2023-10-08T00:12:10.432291+0000","last_active":"2023-10-08T00:12:10.433129+0000","last_peered":"2023-10-08T00:12:10.433129+0000","last_clean":"2023-10-08T00:12:10.433129+0000","last_became_active":"2023-10-08T00:12:10.430327+0000","last_became_peered":"2023-10-08T00:12:10.430327+0000","last_unstale":"2023-10-08T00:12:10.433129+0000","last_undegraded":"2023-10-08T00:12:10.433129+0000","last_fullsized":"2023-10-08T00:12:10.433129+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:47:03.239420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.428413+0000","last_change":"2023-10-08T00:12:10.428287+0000","last_active":"2023-10-08T00:12:10.428413+0000","last_peered":"2023-10-08T00:12:10.428413+0000","last_clean":"2023-10-08T00:12:10.428413+0000","last_became_active":"2023-10-08T00:12:10.428026+0000","last_became_peered":"2023-10-08T00:12:10.428026+0000","last_unstale":"2023-10-08T00:12:10.428413+0000","last_undegraded":"2023-10-08T00:12:10.428413+0000","last_fullsized":"2023-10-08T00:12:10.428413+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:43:26.869632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.431176+0000","last_change":"2023-10-08T00:12:10.431070+0000","last_active":"2023-10-08T00:12:10.431176+0000","last_peered":"2023-10-08T00:12:10.431176+0000","last_clean":"2023-10-08T00:12:10.431176+0000","last_became_active":"2023-10-08T00:12:10.430748+0000","last_became_peered":"2023-10-08T00:12:10.430748+0000","last_unstale":"2023-10-08T00:12:10.431176+0000","last_undegraded":"2023-10-08T00:12:10.431176+0000","last_fullsized":"2023-10-08T00:12:10.431176+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:18:53.588623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.427973+0000","last_change":"2023-10-08T00:12:10.427910+0000","last_active":"2023-10-08T00:12:10.427973+0000","last_peered":"2023-10-08T00:12:10.427973+0000","last_clean":"2023-10-08T00:12:10.427973+0000","last_became_active":"2023-10-08T00:12:10.427699+0000","last_became_peered":"2023-10-08T00:12:10.427699+0000","last_unstale":"2023-10-08T00:12:10.427973+0000","last_undegraded":"2023-10-08T00:12:10.427973+0000","last_fullsized":"2023-10-08T00:12:10.427973+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:44:49.192024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.430319+0000","last_change":"2023-10-08T00:12:10.430042+0000","last_active":"2023-10-08T00:12:10.430319+0000","last_peered":"2023-10-08T00:12:10.430319+0000","last_clean":"2023-10-08T00:12:10.430319+0000","last_became_active":"2023-10-08T00:12:10.429759+0000","last_became_peered":"2023-10-08T00:12:10.429759+0000","last_unstale":"2023-10-08T00:12:10.430319+0000","last_undegraded":"2023-10-08T00:12:10.430319+0000","last_fullsized":"2023-10-08T00:12:10.430319+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:43:20.037231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.433415+0000","last_change":"2023-10-08T00:12:10.433367+0000","last_active":"2023-10-08T00:12:10.433415+0000","last_peered":"2023-10-08T00:12:10.433415+0000","last_clean":"2023-10-08T00:12:10.433415+0000","last_became_active":"2023-10-08T00:12:10.432074+0000","last_became_peered":"2023-10-08T00:12:10.432074+0000","last_unstale":"2023-10-08T00:12:10.433415+0000","last_undegraded":"2023-10-08T00:12:10.433415+0000","last_fullsized":"2023-10-08T00:12:10.433415+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:04:24.070701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.427829+0000","last_change":"2023-10-08T00:12:10.427681+0000","last_active":"2023-10-08T00:12:10.427829+0000","last_peered":"2023-10-08T00:12:10.427829+0000","last_clean":"2023-10-08T00:12:10.427829+0000","last_became_active":"2023-10-08T00:12:10.427474+0000","last_became_peered":"2023-10-08T00:12:10.427474+0000","last_unstale":"2023-10-08T00:12:10.427829+0000","last_undegraded":"2023-10-08T00:12:10.427829+0000","last_fullsized":"2023-10-08T00:12:10.427829+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:59:43.392048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.423636+0000","last_change":"2023-10-08T00:12:10.423573+0000","last_active":"2023-10-08T00:12:10.423636+0000","last_peered":"2023-10-08T00:12:10.423636+0000","last_clean":"2023-10-08T00:12:10.423636+0000","last_became_active":"2023-10-08T00:12:10.423393+0000","last_became_peered":"2023-10-08T00:12:10.423393+0000","last_unstale":"2023-10-08T00:12:10.423636+0000","last_undegraded":"2023-10-08T00:12:10.423636+0000","last_fullsized":"2023-10-08T00:12:10.423636+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:10:25.717447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.431152+0000","last_change":"2023-10-08T00:12:10.431079+0000","last_active":"2023-10-08T00:12:10.431152+0000","last_peered":"2023-10-08T00:12:10.431152+0000","last_clean":"2023-10-08T00:12:10.431152+0000","last_became_active":"2023-10-08T00:12:10.430839+0000","last_became_peered":"2023-10-08T00:12:10.430839+0000","last_unstale":"2023-10-08T00:12:10.431152+0000","last_undegraded":"2023-10-08T00:12:10.431152+0000","last_fullsized":"2023-10-08T00:12:10.431152+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:20:12.899376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.423139+0000","last_change":"2023-10-08T00:12:10.423057+0000","last_active":"2023-10-08T00:12:10.423139+0000","last_peered":"2023-10-08T00:12:10.423139+0000","last_clean":"2023-10-08T00:12:10.423139+0000","last_became_active":"2023-10-08T00:12:10.422451+0000","last_became_peered":"2023-10-08T00:12:10.422451+0000","last_unstale":"2023-10-08T00:12:10.423139+0000","last_undegraded":"2023-10-08T00:12:10.423139+0000","last_fullsized":"2023-10-08T00:12:10.423139+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:48:55.441440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.432482+0000","last_change":"2023-10-08T00:12:10.432267+0000","last_active":"2023-10-08T00:12:10.432482+0000","last_peered":"2023-10-08T00:12:10.432482+0000","last_clean":"2023-10-08T00:12:10.432482+0000","last_became_active":"2023-10-08T00:12:10.431258+0000","last_became_peered":"2023-10-08T00:12:10.431258+0000","last_unstale":"2023-10-08T00:12:10.432482+0000","last_undegraded":"2023-10-08T00:12:10.432482+0000","last_fullsized":"2023-10-08T00:12:10.432482+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:37:22.198496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.427632+0000","last_change":"2023-10-08T00:12:10.425882+0000","last_active":"2023-10-08T00:12:10.427632+0000","last_peered":"2023-10-08T00:12:10.427632+0000","last_clean":"2023-10-08T00:12:10.427632+0000","last_became_active":"2023-10-08T00:12:10.425659+0000","last_became_peered":"2023-10-08T00:12:10.425659+0000","last_unstale":"2023-10-08T00:12:10.427632+0000","last_undegraded":"2023-10-08T00:12:10.427632+0000","last_fullsized":"2023-10-08T00:12:10.427632+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:06:26.932034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.428972+0000","last_change":"2023-10-08T00:12:10.428892+0000","last_active":"2023-10-08T00:12:10.428972+0000","last_peered":"2023-10-08T00:12:10.428972+0000","last_clean":"2023-10-08T00:12:10.428972+0000","last_became_active":"2023-10-08T00:12:10.428163+0000","last_became_peered":"2023-10-08T00:12:10.428163+0000","last_unstale":"2023-10-08T00:12:10.428972+0000","last_undegraded":"2023-10-08T00:12:10.428972+0000","last_fullsized":"2023-10-08T00:12:10.428972+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:28:43.542905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"44'4","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.440774+0000","last_change":"2023-10-08T00:12:09.424522+0000","last_active":"2023-10-08T00:12:10.440774+0000","last_peered":"2023-10-08T00:12:10.440774+0000","last_clean":"2023-10-08T00:12:10.440774+0000","last_became_active":"2023-10-08T00:12:09.424281+0000","last_became_peered":"2023-10-08T00:12:09.424281+0000","last_unstale":"2023-10-08T00:12:10.440774+0000","last_undegraded":"2023-10-08T00:12:10.440774+0000","last_fullsized":"2023-10-08T00:12:10.440774+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:07:18.855061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.434021+0000","last_change":"2023-10-08T00:12:10.433976+0000","last_active":"2023-10-08T00:12:10.434021+0000","last_peered":"2023-10-08T00:12:10.434021+0000","last_clean":"2023-10-08T00:12:10.434021+0000","last_became_active":"2023-10-08T00:12:10.433826+0000","last_became_peered":"2023-10-08T00:12:10.433826+0000","last_unstale":"2023-10-08T00:12:10.434021+0000","last_undegraded":"2023-10-08T00:12:10.434021+0000","last_fullsized":"2023-10-08T00:12:10.434021+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:46:32.815495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"44'1","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.438987+0000","last_change":"2023-10-08T00:12:09.421418+0000","last_active":"2023-10-08T00:12:10.438987+0000","last_peered":"2023-10-08T00:12:10.438987+0000","last_clean":"2023-10-08T00:12:10.438987+0000","last_became_active":"2023-10-08T00:12:09.421193+0000","last_became_peered":"2023-10-08T00:12:09.421193+0000","last_unstale":"2023-10-08T00:12:10.438987+0000","last_undegraded":"2023-10-08T00:12:10.438987+0000","last_fullsized":"2023-10-08T00:12:10.438987+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:20:11.895076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.422172+0000","last_change":"2023-10-08T00:12:10.422110+0000","last_active":"2023-10-08T00:12:10.422172+0000","last_peered":"2023-10-08T00:12:10.422172+0000","last_clean":"2023-10-08T00:12:10.422172+0000","last_became_active":"2023-10-08T00:12:10.421644+0000","last_became_peered":"2023-10-08T00:12:10.421644+0000","last_unstale":"2023-10-08T00:12:10.422172+0000","last_undegraded":"2023-10-08T00:12:10.422172+0000","last_fullsized":"2023-10-08T00:12:10.422172+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:55:23.292081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"44'2","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.439928+0000","last_change":"2023-10-08T00:12:09.424964+0000","last_active":"2023-10-08T00:12:10.439928+0000","last_peered":"2023-10-08T00:12:10.439928+0000","last_clean":"2023-10-08T00:12:10.439928+0000","last_became_active":"2023-10-08T00:12:09.424731+0000","last_became_peered":"2023-10-08T00:12:09.424731+0000","last_unstale":"2023-10-08T00:12:10.439928+0000","last_undegraded":"2023-10-08T00:12:10.439928+0000","last_fullsized":"2023-10-08T00:12:10.439928+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:25:57.800139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.427600+0000","last_change":"2023-10-08T00:12:10.425076+0000","last_active":"2023-10-08T00:12:10.427600+0000","last_peered":"2023-10-08T00:12:10.427600+0000","last_clean":"2023-10-08T00:12:10.427600+0000","last_became_active":"2023-10-08T00:12:10.424831+0000","last_became_peered":"2023-10-08T00:12:10.424831+0000","last_unstale":"2023-10-08T00:12:10.427600+0000","last_undegraded":"2023-10-08T00:12:10.427600+0000","last_fullsized":"2023-10-08T00:12:10.427600+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:47:09.400917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"44'2","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.439919+0000","last_change":"2023-10-08T00:12:09.420047+0000","last_active":"2023-10-08T00:12:10.439919+0000","last_peered":"2023-10-08T00:12:10.439919+0000","last_clean":"2023-10-08T00:12:10.439919+0000","last_became_active":"2023-10-08T00:12:09.419818+0000","last_became_peered":"2023-10-08T00:12:09.419818+0000","last_unstale":"2023-10-08T00:12:10.439919+0000","last_undegraded":"2023-10-08T00:12:10.439919+0000","last_fullsized":"2023-10-08T00:12:10.439919+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:51:34.643301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.428252+0000","last_change":"2023-10-08T00:12:10.428197+0000","last_active":"2023-10-08T00:12:10.428252+0000","last_peered":"2023-10-08T00:12:10.428252+0000","last_clean":"2023-10-08T00:12:10.428252+0000","last_became_active":"2023-10-08T00:12:10.427976+0000","last_became_peered":"2023-10-08T00:12:10.427976+0000","last_unstale":"2023-10-08T00:12:10.428252+0000","last_undegraded":"2023-10-08T00:12:10.428252+0000","last_fullsized":"2023-10-08T00:12:10.428252+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:24:22.363575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"44'1","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.440304+0000","last_change":"2023-10-08T00:12:09.428018+0000","last_active":"2023-10-08T00:12:10.440304+0000","last_peered":"2023-10-08T00:12:10.440304+0000","last_clean":"2023-10-08T00:12:10.440304+0000","last_became_active":"2023-10-08T00:12:09.427776+0000","last_became_peered":"2023-10-08T00:12:09.427776+0000","last_unstale":"2023-10-08T00:12:10.440304+0000","last_undegraded":"2023-10-08T00:12:10.440304+0000","last_fullsized":"2023-10-08T00:12:10.440304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:26:59.016031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.435174+0000","last_change":"2023-10-08T00:12:10.434627+0000","last_active":"2023-10-08T00:12:10.435174+0000","last_peered":"2023-10-08T00:12:10.435174+0000","last_clean":"2023-10-08T00:12:10.435174+0000","last_became_active":"2023-10-08T00:12:10.434333+0000","last_became_peered":"2023-10-08T00:12:10.434333+0000","last_unstale":"2023-10-08T00:12:10.435174+0000","last_undegraded":"2023-10-08T00:12:10.435174+0000","last_fullsized":"2023-10-08T00:12:10.435174+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:48:55.441440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"44'1","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.439088+0000","last_change":"2023-10-08T00:12:09.424860+0000","last_active":"2023-10-08T00:12:10.439088+0000","last_peered":"2023-10-08T00:12:10.439088+0000","last_clean":"2023-10-08T00:12:10.439088+0000","last_became_active":"2023-10-08T00:12:09.424472+0000","last_became_peered":"2023-10-08T00:12:09.424472+0000","last_unstale":"2023-10-08T00:12:10.439088+0000","last_undegraded":"2023-10-08T00:12:10.439088+0000","last_fullsized":"2023-10-08T00:12:10.439088+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:11:42.099513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.426925+0000","last_change":"2023-10-08T00:12:10.426264+0000","last_active":"2023-10-08T00:12:10.426925+0000","last_peered":"2023-10-08T00:12:10.426925+0000","last_clean":"2023-10-08T00:12:10.426925+0000","last_became_active":"2023-10-08T00:12:10.425754+0000","last_became_peered":"2023-10-08T00:12:10.425754+0000","last_unstale":"2023-10-08T00:12:10.426925+0000","last_undegraded":"2023-10-08T00:12:10.426925+0000","last_fullsized":"2023-10-08T00:12:10.426925+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:51:35.647600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.426463+0000","last_change":"2023-10-08T00:12:09.426297+0000","last_active":"2023-10-08T00:12:10.426463+0000","last_peered":"2023-10-08T00:12:10.426463+0000","last_clean":"2023-10-08T00:12:10.426463+0000","last_became_active":"2023-10-08T00:12:09.426024+0000","last_became_peered":"2023-10-08T00:12:09.426024+0000","last_unstale":"2023-10-08T00:12:10.426463+0000","last_undegraded":"2023-10-08T00:12:10.426463+0000","last_fullsized":"2023-10-08T00:12:10.426463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:48:52.703347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.426239+0000","last_change":"2023-10-08T00:12:10.426198+0000","last_active":"2023-10-08T00:12:10.426239+0000","last_peered":"2023-10-08T00:12:10.426239+0000","last_clean":"2023-10-08T00:12:10.426239+0000","last_became_active":"2023-10-08T00:12:10.426013+0000","last_became_peered":"2023-10-08T00:12:10.426013+0000","last_unstale":"2023-10-08T00:12:10.426239+0000","last_undegraded":"2023-10-08T00:12:10.426239+0000","last_fullsized":"2023-10-08T00:12:10.426239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:22:00.356801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.421870+0000","last_change":"2023-10-08T00:12:09.425731+0000","last_active":"2023-10-08T00:12:10.421870+0000","last_peered":"2023-10-08T00:12:10.421870+0000","last_clean":"2023-10-08T00:12:10.421870+0000","last_became_active":"2023-10-08T00:12:09.425365+0000","last_became_peered":"2023-10-08T00:12:09.425365+0000","last_unstale":"2023-10-08T00:12:10.421870+0000","last_undegraded":"2023-10-08T00:12:10.421870+0000","last_fullsized":"2023-10-08T00:12:10.421870+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:37:58.827062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.429471+0000","last_change":"2023-10-08T00:12:10.429427+0000","last_active":"2023-10-08T00:12:10.429471+0000","last_peered":"2023-10-08T00:12:10.429471+0000","last_clean":"2023-10-08T00:12:10.429471+0000","last_became_active":"2023-10-08T00:12:10.428112+0000","last_became_peered":"2023-10-08T00:12:10.428112+0000","last_unstale":"2023-10-08T00:12:10.429471+0000","last_undegraded":"2023-10-08T00:12:10.429471+0000","last_fullsized":"2023-10-08T00:12:10.429471+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:43:26.869632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.425748+0000","last_change":"2023-10-08T00:12:09.429625+0000","last_active":"2023-10-08T00:12:10.425748+0000","last_peered":"2023-10-08T00:12:10.425748+0000","last_clean":"2023-10-08T00:12:10.425748+0000","last_became_active":"2023-10-08T00:12:09.429459+0000","last_became_peered":"2023-10-08T00:12:09.429459+0000","last_unstale":"2023-10-08T00:12:10.425748+0000","last_undegraded":"2023-10-08T00:12:10.425748+0000","last_fullsized":"2023-10-08T00:12:10.425748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:48:34.458509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.431766+0000","last_change":"2023-10-08T00:12:10.431709+0000","last_active":"2023-10-08T00:12:10.431766+0000","last_peered":"2023-10-08T00:12:10.431766+0000","last_clean":"2023-10-08T00:12:10.431766+0000","last_became_active":"2023-10-08T00:12:10.431396+0000","last_became_peered":"2023-10-08T00:12:10.431396+0000","last_unstale":"2023-10-08T00:12:10.431766+0000","last_undegraded":"2023-10-08T00:12:10.431766+0000","last_fullsized":"2023-10-08T00:12:10.431766+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:54:18.156490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.422856+0000","last_change":"2023-10-08T00:12:09.423812+0000","last_active":"2023-10-08T00:12:10.422856+0000","last_peered":"2023-10-08T00:12:10.422856+0000","last_clean":"2023-10-08T00:12:10.422856+0000","last_became_active":"2023-10-08T00:12:09.423459+0000","last_became_peered":"2023-10-08T00:12:09.423459+0000","last_unstale":"2023-10-08T00:12:10.422856+0000","last_undegraded":"2023-10-08T00:12:10.422856+0000","last_fullsized":"2023-10-08T00:12:10.422856+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:37:21.194196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.431202+0000","last_change":"2023-10-08T00:12:10.430902+0000","last_active":"2023-10-08T00:12:10.431202+0000","last_peered":"2023-10-08T00:12:10.431202+0000","last_clean":"2023-10-08T00:12:10.431202+0000","last_became_active":"2023-10-08T00:12:10.430057+0000","last_became_peered":"2023-10-08T00:12:10.430057+0000","last_unstale":"2023-10-08T00:12:10.431202+0000","last_undegraded":"2023-10-08T00:12:10.431202+0000","last_fullsized":"2023-10-08T00:12:10.431202+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:20:12.899376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"44'3","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.441420+0000","last_change":"2023-10-08T00:12:09.425278+0000","last_active":"2023-10-08T00:12:10.441420+0000","last_peered":"2023-10-08T00:12:10.441420+0000","last_clean":"2023-10-08T00:12:10.441420+0000","last_became_active":"2023-10-08T00:12:09.425050+0000","last_became_peered":"2023-10-08T00:12:09.425050+0000","last_unstale":"2023-10-08T00:12:10.441420+0000","last_undegraded":"2023-10-08T00:12:10.441420+0000","last_fullsized":"2023-10-08T00:12:10.441420+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:11:29.789694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.431160+0000","last_change":"2023-10-08T00:12:10.430944+0000","last_active":"2023-10-08T00:12:10.431160+0000","last_peered":"2023-10-08T00:12:10.431160+0000","last_clean":"2023-10-08T00:12:10.431160+0000","last_became_active":"2023-10-08T00:12:10.430469+0000","last_became_peered":"2023-10-08T00:12:10.430469+0000","last_unstale":"2023-10-08T00:12:10.431160+0000","last_undegraded":"2023-10-08T00:12:10.431160+0000","last_fullsized":"2023-10-08T00:12:10.431160+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:53:03.168977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"44'6","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.443767+0000","last_change":"2023-10-08T00:12:09.423947+0000","last_active":"2023-10-08T00:12:10.443767+0000","last_peered":"2023-10-08T00:12:10.443767+0000","last_clean":"2023-10-08T00:12:10.443767+0000","last_became_active":"2023-10-08T00:12:09.423698+0000","last_became_peered":"2023-10-08T00:12:09.423698+0000","last_unstale":"2023-10-08T00:12:10.443767+0000","last_undegraded":"2023-10-08T00:12:10.443767+0000","last_fullsized":"2023-10-08T00:12:10.443767+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:06:49.684844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.434951+0000","last_change":"2023-10-08T00:12:10.434905+0000","last_active":"2023-10-08T00:12:10.434951+0000","last_peered":"2023-10-08T00:12:10.434951+0000","last_clean":"2023-10-08T00:12:10.434951+0000","last_became_active":"2023-10-08T00:12:10.434734+0000","last_became_peered":"2023-10-08T00:12:10.434734+0000","last_unstale":"2023-10-08T00:12:10.434951+0000","last_undegraded":"2023-10-08T00:12:10.434951+0000","last_fullsized":"2023-10-08T00:12:10.434951+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:27:00.020330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"44'2","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.440724+0000","last_change":"2023-10-08T00:12:09.433962+0000","last_active":"2023-10-08T00:12:10.440724+0000","last_peered":"2023-10-08T00:12:10.440724+0000","last_clean":"2023-10-08T00:12:10.440724+0000","last_became_active":"2023-10-08T00:12:09.433746+0000","last_became_peered":"2023-10-08T00:12:09.433746+0000","last_unstale":"2023-10-08T00:12:10.440724+0000","last_undegraded":"2023-10-08T00:12:10.440724+0000","last_fullsized":"2023-10-08T00:12:10.440724+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:12:29.446649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.424809+0000","last_change":"2023-10-08T00:12:10.424353+0000","last_active":"2023-10-08T00:12:10.424809+0000","last_peered":"2023-10-08T00:12:10.424809+0000","last_clean":"2023-10-08T00:12:10.424809+0000","last_became_active":"2023-10-08T00:12:10.424123+0000","last_became_peered":"2023-10-08T00:12:10.424123+0000","last_unstale":"2023-10-08T00:12:10.424809+0000","last_undegraded":"2023-10-08T00:12:10.424809+0000","last_fullsized":"2023-10-08T00:12:10.424809+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:23:01.656028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.427640+0000","last_change":"2023-10-08T00:12:09.427413+0000","last_active":"2023-10-08T00:12:10.427640+0000","last_peered":"2023-10-08T00:12:10.427640+0000","last_clean":"2023-10-08T00:12:10.427640+0000","last_became_active":"2023-10-08T00:12:09.427185+0000","last_became_peered":"2023-10-08T00:12:09.427185+0000","last_unstale":"2023-10-08T00:12:10.427640+0000","last_undegraded":"2023-10-08T00:12:10.427640+0000","last_fullsized":"2023-10-08T00:12:10.427640+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:06:49.684844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.427038+0000","last_change":"2023-10-08T00:12:10.426883+0000","last_active":"2023-10-08T00:12:10.427038+0000","last_peered":"2023-10-08T00:12:10.427038+0000","last_clean":"2023-10-08T00:12:10.427038+0000","last_became_active":"2023-10-08T00:12:10.426652+0000","last_became_peered":"2023-10-08T00:12:10.426652+0000","last_unstale":"2023-10-08T00:12:10.427038+0000","last_undegraded":"2023-10-08T00:12:10.427038+0000","last_fullsized":"2023-10-08T00:12:10.427038+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:37:19.700013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.424476+0000","last_change":"2023-10-08T00:12:09.426676+0000","last_active":"2023-10-08T00:12:10.424476+0000","last_peered":"2023-10-08T00:12:10.424476+0000","last_clean":"2023-10-08T00:12:10.424476+0000","last_became_active":"2023-10-08T00:12:09.426118+0000","last_became_peered":"2023-10-08T00:12:09.426118+0000","last_unstale":"2023-10-08T00:12:10.424476+0000","last_undegraded":"2023-10-08T00:12:10.424476+0000","last_fullsized":"2023-10-08T00:12:10.424476+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:28:42.538605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.432571+0000","last_change":"2023-10-08T00:12:10.432523+0000","last_active":"2023-10-08T00:12:10.432571+0000","last_peered":"2023-10-08T00:12:10.432571+0000","last_clean":"2023-10-08T00:12:10.432571+0000","last_became_active":"2023-10-08T00:12:10.432111+0000","last_became_peered":"2023-10-08T00:12:10.432111+0000","last_unstale":"2023-10-08T00:12:10.432571+0000","last_undegraded":"2023-10-08T00:12:10.432571+0000","last_fullsized":"2023-10-08T00:12:10.432571+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:43:20.037231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"44'8","reported_seq":18,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.443017+0000","last_change":"2023-10-08T00:12:09.419934+0000","last_active":"2023-10-08T00:12:10.443017+0000","last_peered":"2023-10-08T00:12:10.443017+0000","last_clean":"2023-10-08T00:12:10.443017+0000","last_became_active":"2023-10-08T00:12:09.419794+0000","last_became_peered":"2023-10-08T00:12:09.419794+0000","last_unstale":"2023-10-08T00:12:10.443017+0000","last_undegraded":"2023-10-08T00:12:10.443017+0000","last_fullsized":"2023-10-08T00:12:10.443017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:11:29.789694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.429153+0000","last_change":"2023-10-08T00:12:10.429095+0000","last_active":"2023-10-08T00:12:10.429153+0000","last_peered":"2023-10-08T00:12:10.429153+0000","last_clean":"2023-10-08T00:12:10.429153+0000","last_became_active":"2023-10-08T00:12:10.428708+0000","last_became_peered":"2023-10-08T00:12:10.428708+0000","last_unstale":"2023-10-08T00:12:10.429153+0000","last_undegraded":"2023-10-08T00:12:10.429153+0000","last_fullsized":"2023-10-08T00:12:10.429153+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:16:41.819876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"44'1","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.441613+0000","last_change":"2023-10-08T00:12:09.424444+0000","last_active":"2023-10-08T00:12:10.441613+0000","last_peered":"2023-10-08T00:12:10.441613+0000","last_clean":"2023-10-08T00:12:10.441613+0000","last_became_active":"2023-10-08T00:12:09.424190+0000","last_became_peered":"2023-10-08T00:12:09.424190+0000","last_unstale":"2023-10-08T00:12:10.441613+0000","last_undegraded":"2023-10-08T00:12:10.441613+0000","last_fullsized":"2023-10-08T00:12:10.441613+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:28:42.538605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.429111+0000","last_change":"2023-10-08T00:12:10.429045+0000","last_active":"2023-10-08T00:12:10.429111+0000","last_peered":"2023-10-08T00:12:10.429111+0000","last_clean":"2023-10-08T00:12:10.429111+0000","last_became_active":"2023-10-08T00:12:10.428460+0000","last_became_peered":"2023-10-08T00:12:10.428460+0000","last_unstale":"2023-10-08T00:12:10.429111+0000","last_undegraded":"2023-10-08T00:12:10.429111+0000","last_fullsized":"2023-10-08T00:12:10.429111+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:27:00.020330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"44'2","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.440653+0000","last_change":"2023-10-08T00:12:09.426927+0000","last_active":"2023-10-08T00:12:10.440653+0000","last_peered":"2023-10-08T00:12:10.440653+0000","last_clean":"2023-10-08T00:12:10.440653+0000","last_became_active":"2023-10-08T00:12:09.425736+0000","last_became_peered":"2023-10-08T00:12:09.425736+0000","last_unstale":"2023-10-08T00:12:10.440653+0000","last_undegraded":"2023-10-08T00:12:10.440653+0000","last_fullsized":"2023-10-08T00:12:10.440653+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:21:55.609798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.428506+0000","last_change":"2023-10-08T00:12:10.428442+0000","last_active":"2023-10-08T00:12:10.428506+0000","last_peered":"2023-10-08T00:12:10.428506+0000","last_clean":"2023-10-08T00:12:10.428506+0000","last_became_active":"2023-10-08T00:12:10.428203+0000","last_became_peered":"2023-10-08T00:12:10.428203+0000","last_unstale":"2023-10-08T00:12:10.428506+0000","last_undegraded":"2023-10-08T00:12:10.428506+0000","last_fullsized":"2023-10-08T00:12:10.428506+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:07:32.802880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"44'1","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.440786+0000","last_change":"2023-10-08T00:12:09.425049+0000","last_active":"2023-10-08T00:12:10.440786+0000","last_peered":"2023-10-08T00:12:10.440786+0000","last_clean":"2023-10-08T00:12:10.440786+0000","last_became_active":"2023-10-08T00:12:09.424829+0000","last_became_peered":"2023-10-08T00:12:09.424829+0000","last_unstale":"2023-10-08T00:12:10.440786+0000","last_undegraded":"2023-10-08T00:12:10.440786+0000","last_fullsized":"2023-10-08T00:12:10.440786+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:47:58.065638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.428802+0000","last_change":"2023-10-08T00:12:10.428717+0000","last_active":"2023-10-08T00:12:10.428802+0000","last_peered":"2023-10-08T00:12:10.428802+0000","last_clean":"2023-10-08T00:12:10.428802+0000","last_became_active":"2023-10-08T00:12:10.428492+0000","last_became_peered":"2023-10-08T00:12:10.428492+0000","last_unstale":"2023-10-08T00:12:10.428802+0000","last_undegraded":"2023-10-08T00:12:10.428802+0000","last_fullsized":"2023-10-08T00:12:10.428802+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:37:19.700013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.424551+0000","last_change":"2023-10-08T00:12:09.426898+0000","last_active":"2023-10-08T00:12:10.424551+0000","last_peered":"2023-10-08T00:12:10.424551+0000","last_clean":"2023-10-08T00:12:10.424551+0000","last_became_active":"2023-10-08T00:12:09.426428+0000","last_became_peered":"2023-10-08T00:12:09.426428+0000","last_unstale":"2023-10-08T00:12:10.424551+0000","last_undegraded":"2023-10-08T00:12:10.424551+0000","last_fullsized":"2023-10-08T00:12:10.424551+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:37:18.695713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.428394+0000","last_change":"2023-10-08T00:12:10.428279+0000","last_active":"2023-10-08T00:12:10.428394+0000","last_peered":"2023-10-08T00:12:10.428394+0000","last_clean":"2023-10-08T00:12:10.428394+0000","last_became_active":"2023-10-08T00:12:10.427639+0000","last_became_peered":"2023-10-08T00:12:10.427639+0000","last_unstale":"2023-10-08T00:12:10.428394+0000","last_undegraded":"2023-10-08T00:12:10.428394+0000","last_fullsized":"2023-10-08T00:12:10.428394+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:04:24.070701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"44'2","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.443407+0000","last_change":"2023-10-08T00:12:09.427750+0000","last_active":"2023-10-08T00:12:10.443407+0000","last_peered":"2023-10-08T00:12:10.443407+0000","last_clean":"2023-10-08T00:12:10.443407+0000","last_became_active":"2023-10-08T00:12:09.427539+0000","last_became_peered":"2023-10-08T00:12:09.427539+0000","last_unstale":"2023-10-08T00:12:10.443407+0000","last_undegraded":"2023-10-08T00:12:10.443407+0000","last_fullsized":"2023-10-08T00:12:10.443407+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:51:27.936642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.430988+0000","last_change":"2023-10-08T00:12:10.430858+0000","last_active":"2023-10-08T00:12:10.430988+0000","last_peered":"2023-10-08T00:12:10.430988+0000","last_clean":"2023-10-08T00:12:10.430988+0000","last_became_active":"2023-10-08T00:12:10.429897+0000","last_became_peered":"2023-10-08T00:12:10.429897+0000","last_unstale":"2023-10-08T00:12:10.430988+0000","last_undegraded":"2023-10-08T00:12:10.430988+0000","last_fullsized":"2023-10-08T00:12:10.430988+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:47:09.400917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.426356+0000","last_change":"2023-10-08T00:12:09.427687+0000","last_active":"2023-10-08T00:12:10.426356+0000","last_peered":"2023-10-08T00:12:10.426356+0000","last_clean":"2023-10-08T00:12:10.426356+0000","last_became_active":"2023-10-08T00:12:09.427146+0000","last_became_peered":"2023-10-08T00:12:09.427146+0000","last_unstale":"2023-10-08T00:12:10.426356+0000","last_undegraded":"2023-10-08T00:12:10.426356+0000","last_fullsized":"2023-10-08T00:12:10.426356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:37:21.194196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.433146+0000","last_change":"2023-10-08T00:12:10.432582+0000","last_active":"2023-10-08T00:12:10.433146+0000","last_peered":"2023-10-08T00:12:10.433146+0000","last_clean":"2023-10-08T00:12:10.433146+0000","last_became_active":"2023-10-08T00:12:10.430758+0000","last_became_peered":"2023-10-08T00:12:10.430758+0000","last_unstale":"2023-10-08T00:12:10.433146+0000","last_undegraded":"2023-10-08T00:12:10.433146+0000","last_fullsized":"2023-10-08T00:12:10.433146+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:55:23.292081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.425055+0000","last_change":"2023-10-08T00:12:09.425588+0000","last_active":"2023-10-08T00:12:10.425055+0000","last_peered":"2023-10-08T00:12:10.425055+0000","last_clean":"2023-10-08T00:12:10.425055+0000","last_became_active":"2023-10-08T00:12:09.424358+0000","last_became_peered":"2023-10-08T00:12:09.424358+0000","last_unstale":"2023-10-08T00:12:10.425055+0000","last_undegraded":"2023-10-08T00:12:10.425055+0000","last_fullsized":"2023-10-08T00:12:10.425055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:02:09.168821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.434447+0000","last_change":"2023-10-08T00:12:10.434372+0000","last_active":"2023-10-08T00:12:10.434447+0000","last_peered":"2023-10-08T00:12:10.434447+0000","last_clean":"2023-10-08T00:12:10.434447+0000","last_became_active":"2023-10-08T00:12:10.434142+0000","last_became_peered":"2023-10-08T00:12:10.434142+0000","last_unstale":"2023-10-08T00:12:10.434447+0000","last_undegraded":"2023-10-08T00:12:10.434447+0000","last_fullsized":"2023-10-08T00:12:10.434447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:51:35.647600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.420542+0000","last_change":"2023-10-08T00:12:09.423746+0000","last_active":"2023-10-08T00:12:10.420542+0000","last_peered":"2023-10-08T00:12:10.420542+0000","last_clean":"2023-10-08T00:12:10.420542+0000","last_became_active":"2023-10-08T00:12:09.423524+0000","last_became_peered":"2023-10-08T00:12:09.423524+0000","last_unstale":"2023-10-08T00:12:10.420542+0000","last_undegraded":"2023-10-08T00:12:10.420542+0000","last_fullsized":"2023-10-08T00:12:10.420542+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:46:31.811196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.426304+0000","last_change":"2023-10-08T00:12:10.426257+0000","last_active":"2023-10-08T00:12:10.426304+0000","last_peered":"2023-10-08T00:12:10.426304+0000","last_clean":"2023-10-08T00:12:10.426304+0000","last_became_active":"2023-10-08T00:12:10.426043+0000","last_became_peered":"2023-10-08T00:12:10.426043+0000","last_unstale":"2023-10-08T00:12:10.426304+0000","last_undegraded":"2023-10-08T00:12:10.426304+0000","last_fullsized":"2023-10-08T00:12:10.426304+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:13:11.108981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.421480+0000","last_change":"2023-10-08T00:12:09.424182+0000","last_active":"2023-10-08T00:12:10.421480+0000","last_peered":"2023-10-08T00:12:10.421480+0000","last_clean":"2023-10-08T00:12:10.421480+0000","last_became_active":"2023-10-08T00:12:09.423960+0000","last_became_peered":"2023-10-08T00:12:09.423960+0000","last_unstale":"2023-10-08T00:12:10.421480+0000","last_undegraded":"2023-10-08T00:12:10.421480+0000","last_fullsized":"2023-10-08T00:12:10.421480+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:25:57.800139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.426945+0000","last_change":"2023-10-08T00:12:10.426402+0000","last_active":"2023-10-08T00:12:10.426945+0000","last_peered":"2023-10-08T00:12:10.426945+0000","last_clean":"2023-10-08T00:12:10.426945+0000","last_became_active":"2023-10-08T00:12:10.426046+0000","last_became_peered":"2023-10-08T00:12:10.426046+0000","last_unstale":"2023-10-08T00:12:10.426945+0000","last_undegraded":"2023-10-08T00:12:10.426945+0000","last_fullsized":"2023-10-08T00:12:10.426945+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:20:12.899376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.426564+0000","last_change":"2023-10-08T00:12:09.428177+0000","last_active":"2023-10-08T00:12:10.426564+0000","last_peered":"2023-10-08T00:12:10.426564+0000","last_clean":"2023-10-08T00:12:10.426564+0000","last_became_active":"2023-10-08T00:12:09.427468+0000","last_became_peered":"2023-10-08T00:12:09.427468+0000","last_unstale":"2023-10-08T00:12:10.426564+0000","last_undegraded":"2023-10-08T00:12:10.426564+0000","last_fullsized":"2023-10-08T00:12:10.426564+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:02:09.168821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.429429+0000","last_change":"2023-10-08T00:12:10.429370+0000","last_active":"2023-10-08T00:12:10.429429+0000","last_peered":"2023-10-08T00:12:10.429429+0000","last_clean":"2023-10-08T00:12:10.429429+0000","last_became_active":"2023-10-08T00:12:10.429161+0000","last_became_peered":"2023-10-08T00:12:10.429161+0000","last_unstale":"2023-10-08T00:12:10.429429+0000","last_undegraded":"2023-10-08T00:12:10.429429+0000","last_fullsized":"2023-10-08T00:12:10.429429+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:49:22.374374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.421009+0000","last_change":"2023-10-08T00:12:09.432022+0000","last_active":"2023-10-08T00:12:10.421009+0000","last_peered":"2023-10-08T00:12:10.421009+0000","last_clean":"2023-10-08T00:12:10.421009+0000","last_became_active":"2023-10-08T00:12:09.431795+0000","last_became_peered":"2023-10-08T00:12:09.431795+0000","last_unstale":"2023-10-08T00:12:10.421009+0000","last_undegraded":"2023-10-08T00:12:10.421009+0000","last_fullsized":"2023-10-08T00:12:10.421009+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:29:02.607388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.427562+0000","last_change":"2023-10-08T00:12:10.427453+0000","last_active":"2023-10-08T00:12:10.427562+0000","last_peered":"2023-10-08T00:12:10.427562+0000","last_clean":"2023-10-08T00:12:10.427562+0000","last_became_active":"2023-10-08T00:12:10.427178+0000","last_became_peered":"2023-10-08T00:12:10.427178+0000","last_unstale":"2023-10-08T00:12:10.427562+0000","last_undegraded":"2023-10-08T00:12:10.427562+0000","last_fullsized":"2023-10-08T00:12:10.427562+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:07:32.802880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.427738+0000","last_change":"2023-10-08T00:12:09.433714+0000","last_active":"2023-10-08T00:12:10.427738+0000","last_peered":"2023-10-08T00:12:10.427738+0000","last_clean":"2023-10-08T00:12:10.427738+0000","last_became_active":"2023-10-08T00:12:09.433557+0000","last_became_peered":"2023-10-08T00:12:09.433557+0000","last_unstale":"2023-10-08T00:12:10.427738+0000","last_undegraded":"2023-10-08T00:12:10.427738+0000","last_fullsized":"2023-10-08T00:12:10.427738+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:23:36.280027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.427577+0000","last_change":"2023-10-08T00:12:10.424847+0000","last_active":"2023-10-08T00:12:10.427577+0000","last_peered":"2023-10-08T00:12:10.427577+0000","last_clean":"2023-10-08T00:12:10.427577+0000","last_became_active":"2023-10-08T00:12:10.424583+0000","last_became_peered":"2023-10-08T00:12:10.424583+0000","last_unstale":"2023-10-08T00:12:10.427577+0000","last_undegraded":"2023-10-08T00:12:10.427577+0000","last_fullsized":"2023-10-08T00:12:10.427577+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:48:55.441440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.421195+0000","last_change":"2023-10-08T00:12:09.421238+0000","last_active":"2023-10-08T00:12:10.421195+0000","last_peered":"2023-10-08T00:12:10.421195+0000","last_clean":"2023-10-08T00:12:10.421195+0000","last_became_active":"2023-10-08T00:12:09.421013+0000","last_became_peered":"2023-10-08T00:12:09.421013+0000","last_unstale":"2023-10-08T00:12:10.421195+0000","last_undegraded":"2023-10-08T00:12:10.421195+0000","last_fullsized":"2023-10-08T00:12:10.421195+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:31:04.139388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.429347+0000","last_change":"2023-10-08T00:12:10.429292+0000","last_active":"2023-10-08T00:12:10.429347+0000","last_peered":"2023-10-08T00:12:10.429347+0000","last_clean":"2023-10-08T00:12:10.429347+0000","last_became_active":"2023-10-08T00:12:10.429115+0000","last_became_peered":"2023-10-08T00:12:10.429115+0000","last_unstale":"2023-10-08T00:12:10.429347+0000","last_undegraded":"2023-10-08T00:12:10.429347+0000","last_fullsized":"2023-10-08T00:12:10.429347+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:28:43.542905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.423984+0000","last_change":"2023-10-08T00:12:09.426097+0000","last_active":"2023-10-08T00:12:10.423984+0000","last_peered":"2023-10-08T00:12:10.423984+0000","last_clean":"2023-10-08T00:12:10.423984+0000","last_became_active":"2023-10-08T00:12:09.425841+0000","last_became_peered":"2023-10-08T00:12:09.425841+0000","last_unstale":"2023-10-08T00:12:10.423984+0000","last_undegraded":"2023-10-08T00:12:10.423984+0000","last_fullsized":"2023-10-08T00:12:10.423984+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:23:00.651728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.428491+0000","last_change":"2023-10-08T00:12:10.428415+0000","last_active":"2023-10-08T00:12:10.428491+0000","last_peered":"2023-10-08T00:12:10.428491+0000","last_clean":"2023-10-08T00:12:10.428491+0000","last_became_active":"2023-10-08T00:12:10.428178+0000","last_became_peered":"2023-10-08T00:12:10.428178+0000","last_unstale":"2023-10-08T00:12:10.428491+0000","last_undegraded":"2023-10-08T00:12:10.428491+0000","last_fullsized":"2023-10-08T00:12:10.428491+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:47:03.239420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.420449+0000","last_change":"2023-10-08T00:12:09.432071+0000","last_active":"2023-10-08T00:12:10.420449+0000","last_peered":"2023-10-08T00:12:10.420449+0000","last_clean":"2023-10-08T00:12:10.420449+0000","last_became_active":"2023-10-08T00:12:09.431833+0000","last_became_peered":"2023-10-08T00:12:09.431833+0000","last_unstale":"2023-10-08T00:12:10.420449+0000","last_undegraded":"2023-10-08T00:12:10.420449+0000","last_fullsized":"2023-10-08T00:12:10.420449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:40:35.508089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.433161+0000","last_change":"2023-10-08T00:12:10.432681+0000","last_active":"2023-10-08T00:12:10.433161+0000","last_peered":"2023-10-08T00:12:10.433161+0000","last_clean":"2023-10-08T00:12:10.433161+0000","last_became_active":"2023-10-08T00:12:10.432127+0000","last_became_peered":"2023-10-08T00:12:10.432127+0000","last_unstale":"2023-10-08T00:12:10.433161+0000","last_undegraded":"2023-10-08T00:12:10.433161+0000","last_fullsized":"2023-10-08T00:12:10.433161+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:09.411696+0000","last_clean_scrub_stamp":"2023-10-08T00:12:09.411696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:07:32.802880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"44'2","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.439343+0000","last_change":"2023-10-08T00:12:09.425632+0000","last_active":"2023-10-08T00:12:10.439343+0000","last_peered":"2023-10-08T00:12:10.439343+0000","last_clean":"2023-10-08T00:12:10.439343+0000","last_became_active":"2023-10-08T00:12:09.425084+0000","last_became_peered":"2023-10-08T00:12:09.425084+0000","last_unstale":"2023-10-08T00:12:10.439343+0000","last_undegraded":"2023-10-08T00:12:10.439343+0000","last_fullsized":"2023-10-08T00:12:10.439343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:08.407397+0000","last_clean_scrub_stamp":"2023-10-08T00:12:08.407397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:47:02.235120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":181,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T00:12:10.424983+0000","last_change":"2023-10-08T00:10:43.728490+0000","last_active":"2023-10-08T00:12:10.424983+0000","last_peered":"2023-10-08T00:12:10.424983+0000","last_clean":"2023-10-08T00:12:10.424983+0000","last_became_active":"2023-10-08T00:10:43.728341+0000","last_became_peered":"2023-10-08T00:10:43.728341+0000","last_unstale":"2023-10-08T00:12:10.424983+0000","last_undegraded":"2023-10-08T00:12:10.424983+0000","last_fullsized":"2023-10-08T00:12:10.424983+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_clean_scrub_stamp":"2023-10-08T00:10:42.716096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:06:31.035293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":208,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195103,"statfs":{"total":200000000,"available":199786239,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.45500000000000002}]},{"osd":1,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.53000000000000003}]},{"osd":2,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.497},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.61899999999999999}]},{"osd":3,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.65500000000000003}]},{"osd":4,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.58199999999999996}]},{"osd":5,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.69399999999999995}]},{"osd":6,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.63300000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":209,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195103,"statfs":{"total":200000000,"available":199785685,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":5.1230000000000002,"5min":5.1230000000000002,"15min":5.1230000000000002},"last":0.54500000000000004}]},{"osd":1,"last update":"Sun Oct 8 00:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.60399999999999998}]},{"osd":2,"last update":"Sun Oct 8 00:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.64200000000000002}]},{"osd":3,"last update":"Sun Oct 8 00:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.65300000000000002}]},{"osd":4,"last update":"Sun Oct 8 00:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.379},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.51500000000000001}]},{"osd":5,"last update":"Sun Oct 8 00:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.433}]},{"osd":7,"last update":"Sun Oct 8 00:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.501}]}]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":785,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194527,"statfs":{"total":200000000,"available":199195835,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.34499999999999997}]},{"osd":2,"last update":"Sun Oct 8 00:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.442,"5min":0.442,"15min":0.442},"last":0.36599999999999999}]},{"osd":3,"last update":"Sun Oct 8 00:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.497,"5min":0.497,"15min":0.497},"last":0.26900000000000002}]},{"osd":4,"last update":"Sun Oct 8 00:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.41899999999999998}]},{"osd":5,"last update":"Sun Oct 8 00:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.42999999999999999}]},{"osd":6,"last update":"Sun Oct 8 00:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.45400000000000001}]},{"osd":7,"last update":"Sun Oct 8 00:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.46500000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":209,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195103,"statfs":{"total":200000000,"available":199785629,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 00:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.247},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.33200000000000002}]},{"osd":2,"last update":"Sun Oct 8 00:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.185,"5min":0.185,"15min":0.185},"max":{"1min":12.019,"5min":12.019,"15min":12.019},"last":0.26200000000000001}]},{"osd":3,"last update":"Sun Oct 8 00:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.35699999999999998}]},{"osd":4,"last update":"Sun Oct 8 00:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.55400000000000005}]},{"osd":5,"last update":"Sun Oct 8 00:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":11.992000000000001,"5min":11.992000000000001,"15min":11.992000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.52600000000000002}]},{"osd":6,"last update":"Sun Oct 8 00:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.45400000000000001},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":11.927,"5min":11.927,"15min":11.927},"last":0.51700000000000002}]},{"osd":7,"last update":"Sun Oct 8 00:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":11.965,"5min":11.965,"15min":11.965},"last":0.5},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.58599999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":208,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195103,"statfs":{"total":200000000,"available":199786183,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":2.5139999999999998,"5min":2.5139999999999998,"15min":2.5139999999999998},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":2.1309999999999998,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":0.26600000000000001}]},{"osd":1,"last update":"Sun Oct 8 00:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":2.2639999999999998,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":0.25},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":2.5379999999999998,"5min":2.5379999999999998,"15min":2.5379999999999998},"last":0.29999999999999999}]},{"osd":3,"last update":"Sun Oct 8 00:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":2.2890000000000001,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":2.5619999999999998,"5min":2.5619999999999998,"15min":2.5619999999999998},"last":0.32500000000000001}]},{"osd":4,"last update":"Sun Oct 8 00:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.488,"5min":2.488,"15min":2.488},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":2.073,"5min":2.073,"15min":2.073},"last":0.57899999999999996}]},{"osd":5,"last update":"Sun Oct 8 00:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":2.2069999999999999,"5min":2.2069999999999999,"15min":2.2069999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.456,"5min":2.456,"15min":2.456},"last":0.71999999999999997}]},{"osd":6,"last update":"Sun Oct 8 00:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":2.0339999999999998,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.53000000000000003}]},{"osd":7,"last update":"Sun Oct 8 00:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":2.1030000000000002,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.2349999999999999,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":0.56399999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":210,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195102,"statfs":{"total":200000000,"available":199784737,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.251}]},{"osd":1,"last update":"Sun Oct 8 00:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":12.351000000000001,"5min":12.351000000000001,"15min":12.351000000000001},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.32600000000000001}]},{"osd":2,"last update":"Sun Oct 8 00:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.307}]},{"osd":4,"last update":"Sun Oct 8 00:11:41 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":11.333,"5min":11.333,"15min":11.333},"last":0.59699999999999998},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":12.321999999999999,"5min":12.321999999999999,"15min":12.321999999999999},"last":0.58899999999999997}]},{"osd":5,"last update":"Sun Oct 8 00:11:41 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.181,"15min":1.181},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":12.291,"5min":12.291,"15min":12.291},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.46100000000000002}]},{"osd":6,"last update":"Sun Oct 8 00:11:41 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":11.288,"5min":11.288,"15min":11.288},"last":0.52800000000000002},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":12.253,"5min":12.253,"15min":12.253},"last":0.40500000000000003}]},{"osd":7,"last update":"Sun Oct 8 00:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.48399999999999999},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":11.363,"5min":11.363,"15min":11.363},"last":0.44900000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":210,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195102,"statfs":{"total":200000000,"available":199784645,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.51200000000000001}]},{"osd":1,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.56399999999999995}]},{"osd":2,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.55000000000000004}]},{"osd":3,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.57299999999999995}]},{"osd":5,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.433},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.46400000000000002}]},{"osd":6,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.377}]},{"osd":7,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.53400000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":33,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":785,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194527,"statfs":{"total":200000000,"available":199195883,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.5},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.433}]},{"osd":1,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.41299999999999998}]},{"osd":2,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.53300000000000003}]},{"osd":3,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.56499999999999995}]},{"osd":4,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.54700000000000004}]},{"osd":6,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.52500000000000002}]},{"osd":7,"last update":"Sun Oct 8 00:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.61799999999999999}]}]}],"pool_statfs":[]}} 2023-10-08T00:12:14.840 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T00:12:14.840 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-08T00:12:14.993 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.993+0000 7f71ad066700 1 Processor -- start 2023-10-08T00:12:14.994 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.993+0000 7f71ad066700 1 -- start start 2023-10-08T00:12:14.996 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.993+0000 7f71ad066700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f71a8159ec0 0x7f71a80af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:14.996 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.994+0000 7f71ad066700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71a80b0790 con 0x7f71a806acb0 2023-10-08T00:12:14.996 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.994+0000 7f71ad066700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71a80ce1f0 con 0x7f71a815a290 2023-10-08T00:12:14.996 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.994+0000 7f71ad066700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71a80b0b20 con 0x7f71a8159ec0 2023-10-08T00:12:14.996 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.994+0000 7f71a6d9d700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f71a8159ec0 0x7f71a80af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:14.996 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.994+0000 7f71a759e700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f71a806acb0 0x7f71a80cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45142/0 (socket says 172.21.15.99:45142) 2023-10-08T00:12:14.996 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.994+0000 7f71a759e700 1 -- 172.21.15.99:0/917656266 learned_addr learned my addr 172.21.15.99:0/917656266 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:12:14.996 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.994+0000 7f71a6d9d700 1 -- 172.21.15.99:0/917656266 >> v1:172.21.15.191:6789/0 conn(0x7f71a815a290 legacy=0x7f71a806a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:14.997 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.994+0000 7f71a6d9d700 1 -- 172.21.15.99:0/917656266 >> v1:172.21.15.99:6789/0 conn(0x7f71a806acb0 legacy=0x7f71a80cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:14.997 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.994+0000 7f71a6d9d700 1 -- 172.21.15.99:0/917656266 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a80b04b0 con 0x7f71a8159ec0 2023-10-08T00:12:14.997 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.994+0000 7f71a5d9b700 1 -- 172.21.15.99:0/917656266 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2455439039 0 0) 0x7f71a80b0790 con 0x7f71a806acb0 2023-10-08T00:12:14.997 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.994+0000 7f71a6d9d700 1 --2- 172.21.15.99:0/917656266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71a8159ec0 0x7f71a80af940 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f71980098e0 tx=0x7f71980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71705a3f48aa5176 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:14.997 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.994+0000 7f71a5d9b700 1 -- 172.21.15.99:0/917656266 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7198009e80 con 0x7f71a8159ec0 2023-10-08T00:12:14.997 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.994+0000 7f71a5d9b700 1 -- 172.21.15.99:0/917656266 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f719800b860 con 0x7f71a8159ec0 2023-10-08T00:12:14.997 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.994+0000 7f71a5d9b700 1 -- 172.21.15.99:0/917656266 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719800bbe0 con 0x7f71a8159ec0 2023-10-08T00:12:14.997 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.995+0000 7f71ad066700 1 -- 172.21.15.99:0/917656266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71a8159ec0 msgr2=0x7f71a80af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:14.997 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.995+0000 7f71ad066700 1 --2- 172.21.15.99:0/917656266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71a8159ec0 0x7f71a80af940 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f71980098e0 tx=0x7f71980091a0 comp rx=0 tx=0).stop 2023-10-08T00:12:14.997 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.995+0000 7f71ad066700 1 -- 172.21.15.99:0/917656266 shutdown_connections 2023-10-08T00:12:14.998 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.995+0000 7f71ad066700 1 --2- 172.21.15.99:0/917656266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71a8159ec0 0x7f71a80af940 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:14.998 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.995+0000 7f71ad066700 1 -- 172.21.15.99:0/917656266 >> 172.21.15.99:0/917656266 conn(0x7f71a80d1880 msgr2=0x7f71a80cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:14.998 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.995+0000 7f71ad066700 1 -- 172.21.15.99:0/917656266 shutdown_connections 2023-10-08T00:12:14.998 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.995+0000 7f71ad066700 1 -- 172.21.15.99:0/917656266 wait complete. 2023-10-08T00:12:15.001 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.996+0000 7f71ad066700 1 Processor -- start 2023-10-08T00:12:15.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.996+0000 7f71ad066700 1 -- start start 2023-10-08T00:12:15.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.997+0000 7f71ad066700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71a806acb0 0x7f71a814dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:15.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.997+0000 7f71ad066700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71a8159ec0 0x7f71a816d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:15.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.997+0000 7f71ad066700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71a815a290 0x7f71a816dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:15.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.997+0000 7f71ad066700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a814faf0 con 0x7f71a815a290 2023-10-08T00:12:15.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.997+0000 7f71ad066700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a814ec40 con 0x7f71a806acb0 2023-10-08T00:12:15.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.997+0000 7f71ad066700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71a814e0f0 con 0x7f71a8159ec0 2023-10-08T00:12:15.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.997+0000 7f71a759e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71a815a290 0x7f71a816dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:15.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.997+0000 7f71a759e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71a815a290 0x7f71a816dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44584/0 (socket says 172.21.15.99:44584) 2023-10-08T00:12:15.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.997+0000 7f71a759e700 1 -- 172.21.15.99:0/2054597693 learned_addr learned my addr 172.21.15.99:0/2054597693 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:15.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.997+0000 7f71a6d9d700 1 --2- 172.21.15.99:0/2054597693 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71a806acb0 0x7f71a814dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:15.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.997+0000 7f71a659c700 1 --2- 172.21.15.99:0/2054597693 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71a8159ec0 0x7f71a816d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:15.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.997+0000 7f71a759e700 1 -- 172.21.15.99:0/2054597693 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71a8159ec0 msgr2=0x7f71a816d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:15.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.997+0000 7f71a759e700 1 --2- 172.21.15.99:0/2054597693 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71a8159ec0 0x7f71a816d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:15.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.997+0000 7f71a759e700 1 -- 172.21.15.99:0/2054597693 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71a806acb0 msgr2=0x7f71a814dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:15.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.997+0000 7f71a759e700 1 --2- 172.21.15.99:0/2054597693 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71a806acb0 0x7f71a814dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:15.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.997+0000 7f71a759e700 1 -- 172.21.15.99:0/2054597693 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7198009420 con 0x7f71a815a290 2023-10-08T00:12:15.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.997+0000 7f71a759e700 1 --2- 172.21.15.99:0/2054597693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71a815a290 0x7f71a816dc30 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f71a000cc90 tx=0x7f71a0001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:15.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.997+0000 7f718effd700 1 -- 172.21.15.99:0/2054597693 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a000a850 con 0x7f71a815a290 2023-10-08T00:12:15.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.998+0000 7f718effd700 1 -- 172.21.15.99:0/2054597693 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71a000a9b0 con 0x7f71a815a290 2023-10-08T00:12:15.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.998+0000 7f71ad066700 1 -- 172.21.15.99:0/2054597693 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71a814eee0 con 0x7f71a815a290 2023-10-08T00:12:15.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.998+0000 7f718effd700 1 -- 172.21.15.99:0/2054597693 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a000ad70 con 0x7f71a815a290 2023-10-08T00:12:15.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.998+0000 7f71ad066700 1 -- 172.21.15.99:0/2054597693 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71a814f4a0 con 0x7f71a815a290 2023-10-08T00:12:15.004 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.998+0000 7f71ad066700 1 -- 172.21.15.99:0/2054597693 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71a814fca0 con 0x7f71a815a290 2023-10-08T00:12:15.004 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.999+0000 7f718effd700 1 -- 172.21.15.99:0/2054597693 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f71a0007300 con 0x7f71a815a290 2023-10-08T00:12:15.004 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.999+0000 7f718effd700 1 --2- 172.21.15.99:0/2054597693 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7194066f30 0x7f71940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:15.004 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.999+0000 7f718effd700 1 -- 172.21.15.99:0/2054597693 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f71a00847d0 con 0x7f71a815a290 2023-10-08T00:12:15.004 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:14.999+0000 7f71a6d9d700 1 --2- 172.21.15.99:0/2054597693 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7194066f30 0x7f71940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:15.004 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.000+0000 7f71a6d9d700 1 --2- 172.21.15.99:0/2054597693 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7194066f30 0x7f71940693e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f71980093f0 tx=0x7f7198017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:15.004 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.003+0000 7f718effd700 1 -- 172.21.15.99:0/2054597693 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f71a814fca0 con 0x7f71a815a290 2023-10-08T00:12:15.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.257+0000 7f71ad066700 1 -- 172.21.15.99:0/2054597693 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f71a804c3c0 con 0x7f71a815a290 2023-10-08T00:12:15.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.261+0000 7f718effd700 1 -- 172.21.15.99:0/2054597693 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+0 (secure 0 0 0) 0x7f71a0051ac0 con 0x7f71a815a290 2023-10-08T00:12:15.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.263+0000 7f71ad066700 1 -- 172.21.15.99:0/2054597693 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7194066f30 msgr2=0x7f71940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:15.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.263+0000 7f71ad066700 1 --2- 172.21.15.99:0/2054597693 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7194066f30 0x7f71940693e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f71980093f0 tx=0x7f7198017000 comp rx=0 tx=0).stop 2023-10-08T00:12:15.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.263+0000 7f71ad066700 1 -- 172.21.15.99:0/2054597693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71a815a290 msgr2=0x7f71a816dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:15.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.263+0000 7f71ad066700 1 --2- 172.21.15.99:0/2054597693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71a815a290 0x7f71a816dc30 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f71a000cc90 tx=0x7f71a0001a00 comp rx=0 tx=0).stop 2023-10-08T00:12:15.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.263+0000 7f71ad066700 1 -- 172.21.15.99:0/2054597693 shutdown_connections 2023-10-08T00:12:15.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.263+0000 7f71ad066700 1 --2- 172.21.15.99:0/2054597693 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7194066f30 0x7f71940693e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:15.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.263+0000 7f71ad066700 1 --2- 172.21.15.99:0/2054597693 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71a806acb0 0x7f71a814dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:15.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.263+0000 7f71ad066700 1 --2- 172.21.15.99:0/2054597693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71a815a290 0x7f71a816dc30 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:15.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.263+0000 7f71ad066700 1 --2- 172.21.15.99:0/2054597693 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71a8159ec0 0x7f71a816d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:15.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.263+0000 7f71ad066700 1 -- 172.21.15.99:0/2054597693 >> 172.21.15.99:0/2054597693 conn(0x7f71a80d1880 msgr2=0x7f71a80b0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:15.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.263+0000 7f71ad066700 1 -- 172.21.15.99:0/2054597693 shutdown_connections 2023-10-08T00:12:15.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.263+0000 7f71ad066700 1 -- 172.21.15.99:0/2054597693 wait complete. 2023-10-08T00:12:15.265 INFO:teuthology.orchestra.run.smithi099.stderr:updated caps for client.0 2023-10-08T00:12:15.275 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:12:15.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.389+0000 7fdec4504700 1 Processor -- start 2023-10-08T00:12:15.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.389+0000 7fdec4504700 1 -- start start 2023-10-08T00:12:15.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.390+0000 7fdec4504700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fdebc157870 0x7fdebc15b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:15.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.390+0000 7fdec4504700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdebc157c40 0x7fdebc158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:15.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.390+0000 7fdec4504700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdebc15cbb0 con 0x7fdebc15a620 2023-10-08T00:12:15.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.390+0000 7fdec4504700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdebc15bfa0 con 0x7fdebc157870 2023-10-08T00:12:15.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.390+0000 7fdec4504700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdebc15c280 con 0x7fdebc157c40 2023-10-08T00:12:15.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.390+0000 7fdec1a9f700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fdebc15a620 0x7fdebc15d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45150/0 (socket says 172.21.15.99:45150) 2023-10-08T00:12:15.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.390+0000 7fdec1a9f700 1 -- 172.21.15.99:0/1144198779 learned_addr learned my addr 172.21.15.99:0/1144198779 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:12:15.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.390+0000 7fdec2aa1700 1 --2- 172.21.15.99:0/1144198779 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdebc157c40 0x7fdebc158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:15.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.390+0000 7fdec22a0700 1 --2- 172.21.15.99:0/1144198779 >> v2:172.21.15.191:3300/0 conn(0x7fdebc157870 0x7fdebc15b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:15.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.391+0000 7fdec129e700 1 -- 172.21.15.99:0/1144198779 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 753104731 0 0) 0x7fdebc15cbb0 con 0x7fdebc15a620 2023-10-08T00:12:15.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.391+0000 7fdec129e700 1 -- 172.21.15.99:0/1144198779 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdea8003570 con 0x7fdebc15a620 2023-10-08T00:12:15.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.391+0000 7fdec129e700 1 -- 172.21.15.99:0/1144198779 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2538534834 0 0) 0x7fdea8003570 con 0x7fdebc15a620 2023-10-08T00:12:15.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.391+0000 7fdec129e700 1 -- 172.21.15.99:0/1144198779 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdebc157c40 msgr2=0x7fdebc158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:15.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.391+0000 7fdec129e700 1 --2- 172.21.15.99:0/1144198779 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdebc157c40 0x7fdebc158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:15.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.391+0000 7fdec129e700 1 -- 172.21.15.99:0/1144198779 >> v2:172.21.15.191:3300/0 conn(0x7fdebc157870 msgr2=0x7fdebc15b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:15.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.391+0000 7fdec129e700 1 --2- 172.21.15.99:0/1144198779 >> v2:172.21.15.191:3300/0 conn(0x7fdebc157870 0x7fdebc15b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:15.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.391+0000 7fdec129e700 1 -- 172.21.15.99:0/1144198779 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdebc15c560 con 0x7fdebc15a620 2023-10-08T00:12:15.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.391+0000 7fdec129e700 1 -- 172.21.15.99:0/1144198779 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7fdeac001470 con 0x7fdebc15a620 2023-10-08T00:12:15.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.391+0000 7fdec129e700 1 -- 172.21.15.99:0/1144198779 >> v1:172.21.15.99:6789/0 conn(0x7fdebc15a620 legacy=0x7fdebc15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:15.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.391+0000 7fdec129e700 1 --2- 172.21.15.99:0/1144198779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdea8003a60 0x7fdea8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:15.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.391+0000 7fdec129e700 1 --2- 172.21.15.99:0/1144198779 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdebc157c40 0x7fdea8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:15.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.391+0000 7fdec129e700 1 --2- 172.21.15.99:0/1144198779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdea8004d50 0x7fdea80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:15.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.391+0000 7fdec129e700 1 -- 172.21.15.99:0/1144198779 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdebc15bfa0 con 0x7fdea8003a60 2023-10-08T00:12:15.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.391+0000 7fdec129e700 1 -- 172.21.15.99:0/1144198779 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdebc15c280 con 0x7fdea8004d50 2023-10-08T00:12:15.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.391+0000 7fdec129e700 1 -- 172.21.15.99:0/1144198779 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdea800ba60 con 0x7fdebc157c40 2023-10-08T00:12:15.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.391+0000 7fdec22a0700 1 --2- 172.21.15.99:0/1144198779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdea8003a60 0x7fdea8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:15.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.391+0000 7fdec2aa1700 1 --2- 172.21.15.99:0/1144198779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdea8004d50 0x7fdea80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:15.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.391+0000 7fdec1a9f700 1 --2- 172.21.15.99:0/1144198779 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdebc157c40 0x7fdea8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:15.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.391+0000 7fdec22a0700 1 -- 172.21.15.99:0/1144198779 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdebc157c40 msgr2=0x7fdea8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:15.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.391+0000 7fdec22a0700 1 --2- 172.21.15.99:0/1144198779 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdebc157c40 0x7fdea8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:15.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.391+0000 7fdec22a0700 1 -- 172.21.15.99:0/1144198779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdea8004d50 msgr2=0x7fdea80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:15.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.391+0000 7fdec22a0700 1 --2- 172.21.15.99:0/1144198779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdea8004d50 0x7fdea80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:15.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.391+0000 7fdec22a0700 1 -- 172.21.15.99:0/1144198779 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea8003570 con 0x7fdea8003a60 2023-10-08T00:12:15.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.391+0000 7fdec1a9f700 1 --2- 172.21.15.99:0/1144198779 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdebc157c40 0x7fdea8004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:12:15.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.391+0000 7fdec22a0700 1 --2- 172.21.15.99:0/1144198779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdea8003a60 0x7fdea8003e70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8009c00 tx=0x7fdeb80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbb2f48754ded364 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:15.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.391+0000 7fdec129e700 1 -- 172.21.15.99:0/1144198779 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb800e6b0 con 0x7fdea8003a60 2023-10-08T00:12:15.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.391+0000 7fdec129e700 1 -- 172.21.15.99:0/1144198779 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeb800e810 con 0x7fdea8003a60 2023-10-08T00:12:15.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.392+0000 7fdec129e700 1 -- 172.21.15.99:0/1144198779 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb800ebd0 con 0x7fdea8003a60 2023-10-08T00:12:15.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.392+0000 7fdec4504700 1 -- 172.21.15.99:0/1144198779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdea8003a60 msgr2=0x7fdea8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:15.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.392+0000 7fdec4504700 1 --2- 172.21.15.99:0/1144198779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdea8003a60 0x7fdea8003e70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8009c00 tx=0x7fdeb80091a0 comp rx=0 tx=0).stop 2023-10-08T00:12:15.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.392+0000 7fdec2aa1700 1 -- 172.21.15.99:0/1144198779 reap_dead start 2023-10-08T00:12:15.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.392+0000 7fdec4504700 1 -- 172.21.15.99:0/1144198779 shutdown_connections 2023-10-08T00:12:15.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.392+0000 7fdec4504700 1 -- 172.21.15.99:0/1144198779 >> 172.21.15.99:0/1144198779 conn(0x7fdebc068ce0 msgr2=0x7fdebc0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:15.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.392+0000 7fdec4504700 1 -- 172.21.15.99:0/1144198779 shutdown_connections 2023-10-08T00:12:15.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.392+0000 7fdec4504700 1 -- 172.21.15.99:0/1144198779 wait complete. 2023-10-08T00:12:15.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.393+0000 7fdec4504700 1 Processor -- start 2023-10-08T00:12:15.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.393+0000 7fdec4504700 1 -- start start 2023-10-08T00:12:15.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.394+0000 7fdec4504700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdea8003a60 0x7fdebc14d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:15.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.394+0000 7fdec4504700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdebc15e410 0x7fdebc15e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:15.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.394+0000 7fdec4504700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdebc0b0880 0x7fdebc0b0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:15.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.394+0000 7fdec4504700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdebc15edc0 con 0x7fdebc0b0880 2023-10-08T00:12:15.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.394+0000 7fdec4504700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdebc14f610 con 0x7fdea8003a60 2023-10-08T00:12:15.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.394+0000 7fdec4504700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdebc14f300 con 0x7fdebc15e410 2023-10-08T00:12:15.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.394+0000 7fdec2aa1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdebc0b0880 0x7fdebc0b0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:15.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.394+0000 7fdec2aa1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdebc0b0880 0x7fdebc0b0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44602/0 (socket says 172.21.15.99:44602) 2023-10-08T00:12:15.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.394+0000 7fdec2aa1700 1 -- 172.21.15.99:0/3184684132 learned_addr learned my addr 172.21.15.99:0/3184684132 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:15.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.394+0000 7fdec1a9f700 1 --2- 172.21.15.99:0/3184684132 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdebc15e410 0x7fdebc15e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:15.402 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.394+0000 7fdec22a0700 1 --2- 172.21.15.99:0/3184684132 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdea8003a60 0x7fdebc14d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:15.402 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.394+0000 7fdec2aa1700 1 -- 172.21.15.99:0/3184684132 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdebc15e410 msgr2=0x7fdebc15e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:15.402 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.394+0000 7fdec2aa1700 1 --2- 172.21.15.99:0/3184684132 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdebc15e410 0x7fdebc15e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:15.402 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.394+0000 7fdec2aa1700 1 -- 172.21.15.99:0/3184684132 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdea8003a60 msgr2=0x7fdebc14d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:15.402 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.394+0000 7fdec2aa1700 1 --2- 172.21.15.99:0/3184684132 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdea8003a60 0x7fdebc14d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:15.402 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.394+0000 7fdec2aa1700 1 -- 172.21.15.99:0/3184684132 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdebc14d8c0 con 0x7fdebc0b0880 2023-10-08T00:12:15.402 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.395+0000 7fdec2aa1700 1 --2- 172.21.15.99:0/3184684132 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdebc0b0880 0x7fdebc0b0d00 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fdebc0b3830 tx=0x7fdeb400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:15.402 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.395+0000 7fdeb27fc700 1 -- 172.21.15.99:0/3184684132 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb400e070 con 0x7fdebc0b0880 2023-10-08T00:12:15.402 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.395+0000 7fdeb27fc700 1 -- 172.21.15.99:0/3184684132 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeb4011b80 con 0x7fdebc0b0880 2023-10-08T00:12:15.402 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.395+0000 7fdeb27fc700 1 -- 172.21.15.99:0/3184684132 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4016470 con 0x7fdebc0b0880 2023-10-08T00:12:15.402 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.395+0000 7fdec4504700 1 -- 172.21.15.99:0/3184684132 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdebc0b15f0 con 0x7fdebc0b0880 2023-10-08T00:12:15.402 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.395+0000 7fdec4504700 1 -- 172.21.15.99:0/3184684132 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdebc0b1bb0 con 0x7fdebc0b0880 2023-10-08T00:12:15.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.396+0000 7fdec4504700 1 -- 172.21.15.99:0/3184684132 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdebc0b1dd0 con 0x7fdebc0b0880 2023-10-08T00:12:15.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.396+0000 7fdeb27fc700 1 -- 172.21.15.99:0/3184684132 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdeb4016860 con 0x7fdebc0b0880 2023-10-08T00:12:15.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.396+0000 7fdeb27fc700 1 --2- 172.21.15.99:0/3184684132 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdea0066f30 0x7fdea00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:15.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.397+0000 7fdeb27fc700 1 -- 172.21.15.99:0/3184684132 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdeb40850e0 con 0x7fdebc0b0880 2023-10-08T00:12:15.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.397+0000 7fdec22a0700 1 --2- 172.21.15.99:0/3184684132 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdea0066f30 0x7fdea00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:15.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.397+0000 7fdec22a0700 1 --2- 172.21.15.99:0/3184684132 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdea0066f30 0x7fdea00693e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdeb80091e0 tx=0x7fdeb801a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:15.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.401+0000 7fdeb27fc700 1 -- 172.21.15.99:0/3184684132 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdeb4052350 con 0x7fdebc0b0880 2023-10-08T00:12:15.410 INFO:tasks.ceph.mds.d.smithi191.stderr:2023-10-08T00:12:15.409+0000 7f5c612ac700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T00:12:15.627 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.627+0000 7fdec4504700 1 -- 172.21.15.99:0/3184684132 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdebc04c3c0 con 0x7fdebc0b0880 2023-10-08T00:12:15.628 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.627+0000 7fdeb27fc700 1 -- 172.21.15.99:0/3184684132 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4603 (secure 0 0 0) 0x7fdeb4055970 con 0x7fdebc0b0880 2023-10-08T00:12:15.628 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:12:15.628 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5146,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/1438931938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1438931938},{"type":"v1","addr":"172.21.15.191:6836","nonce":1438931938}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5280,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/3090674680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3090674680},{"type":"v1","addr":"172.21.15.99:6833","nonce":3090674680}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5283,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/4025282357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":4025282357},{"type":"v1","addr":"172.21.15.99:6835","nonce":4025282357}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:12:09.803824+0000","modified":"2023-10-08T00:12:13.436570+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5289},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5289":{"gid":5289,"name":"d","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/2062242133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":2062242133},{"type":"v1","addr":"172.21.15.191:6837","nonce":2062242133}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-08T00:12:15.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.629+0000 7fdec4504700 1 -- 172.21.15.99:0/3184684132 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdea0066f30 msgr2=0x7fdea00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:15.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.629+0000 7fdec4504700 1 --2- 172.21.15.99:0/3184684132 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdea0066f30 0x7fdea00693e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdeb80091e0 tx=0x7fdeb801a000 comp rx=0 tx=0).stop 2023-10-08T00:12:15.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.629+0000 7fdec4504700 1 -- 172.21.15.99:0/3184684132 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdebc0b0880 msgr2=0x7fdebc0b0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:15.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.629+0000 7fdec4504700 1 --2- 172.21.15.99:0/3184684132 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdebc0b0880 0x7fdebc0b0d00 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fdebc0b3830 tx=0x7fdeb400b040 comp rx=0 tx=0).stop 2023-10-08T00:12:15.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.629+0000 7fdec4504700 1 -- 172.21.15.99:0/3184684132 shutdown_connections 2023-10-08T00:12:15.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.629+0000 7fdec4504700 1 --2- 172.21.15.99:0/3184684132 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdea0066f30 0x7fdea00693e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:15.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.629+0000 7fdec4504700 1 --2- 172.21.15.99:0/3184684132 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdea8003a60 0x7fdebc14d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:15.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.629+0000 7fdec4504700 1 --2- 172.21.15.99:0/3184684132 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdebc0b0880 0x7fdebc0b0d00 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:15.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.629+0000 7fdec4504700 1 --2- 172.21.15.99:0/3184684132 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdebc15e410 0x7fdebc15e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:15.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.629+0000 7fdec4504700 1 -- 172.21.15.99:0/3184684132 >> 172.21.15.99:0/3184684132 conn(0x7fdebc068ce0 msgr2=0x7fdebc0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:15.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.629+0000 7fdec4504700 1 -- 172.21.15.99:0/3184684132 shutdown_connections 2023-10-08T00:12:15.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:15.629+0000 7fdec4504700 1 -- 172.21.15.99:0/3184684132 wait complete. 2023-10-08T00:12:15.632 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 24 2023-10-08T00:12:16.044 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T00:12:09.803824+0000', 'modified': '2023-10-08T00:12:13.436570+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5289}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5289': {'gid': 5289, 'name': 'd', 'rank': 0, 'incarnation': 21, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.191:6837/2062242133', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6835', 'nonce': 2062242133}, {'type': 'v1', 'addr': '172.21.15.191:6837', 'nonce': 2062242133}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-08T00:12:16.045 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T00:12:16.045 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-08T00:12:16.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.160+0000 7f1eabfff700 1 Processor -- start 2023-10-08T00:12:16.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.160+0000 7f1eabfff700 1 -- start start 2023-10-08T00:12:16.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.161+0000 7f1eabfff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1eac15a170 0x7f1eac068b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:16.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.161+0000 7f1eabfff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1eac069040 0x7f1eac0af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:16.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.161+0000 7f1eabfff700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eac15ea70 con 0x7f1eac159da0 2023-10-08T00:12:16.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.161+0000 7f1eabfff700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1eac0b34b0 con 0x7f1eac15a170 2023-10-08T00:12:16.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.161+0000 7f1eabfff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eac0694e0 con 0x7f1eac069040 2023-10-08T00:12:16.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.161+0000 7f1eaa7fc700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1eac15a170 0x7f1eac068b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:16.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.161+0000 7f1eaa7fc700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1eac15a170 0x7f1eac068b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44616/0 (socket says 172.21.15.99:44616) 2023-10-08T00:12:16.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.161+0000 7f1eaa7fc700 1 -- 172.21.15.99:0/833908417 learned_addr learned my addr 172.21.15.99:0/833908417 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:16.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.161+0000 7f1eab7fe700 1 --2- 172.21.15.99:0/833908417 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1eac069040 0x7f1eac0af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:16.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.161+0000 7f1eaa7fc700 1 -- 172.21.15.99:0/833908417 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1eac069040 msgr2=0x7f1eac0af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:16.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.161+0000 7f1eaa7fc700 1 --2- 172.21.15.99:0/833908417 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1eac069040 0x7f1eac0af4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:16.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.161+0000 7f1eaa7fc700 1 -- 172.21.15.99:0/833908417 >> v1:172.21.15.191:6789/0 conn(0x7f1eac159da0 legacy=0x7f1eac0b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:16.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.161+0000 7f1eaa7fc700 1 -- 172.21.15.99:0/833908417 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eac0b3990 con 0x7f1eac15a170 2023-10-08T00:12:16.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.162+0000 7f1eaa7fc700 1 --2- 172.21.15.99:0/833908417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1eac15a170 0x7f1eac068b00 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1e940099b0 tx=0x7f1e94009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee148db156769028 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:16.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.162+0000 7f1ea9ffb700 1 -- 172.21.15.99:0/833908417 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9400b860 con 0x7f1eac15a170 2023-10-08T00:12:16.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.162+0000 7f1ea9ffb700 1 -- 172.21.15.99:0/833908417 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e9400b9c0 con 0x7f1eac15a170 2023-10-08T00:12:16.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.162+0000 7f1ea9ffb700 1 -- 172.21.15.99:0/833908417 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9400bd80 con 0x7f1eac15a170 2023-10-08T00:12:16.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.162+0000 7f1eabfff700 1 -- 172.21.15.99:0/833908417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1eac15a170 msgr2=0x7f1eac068b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:16.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.162+0000 7f1eabfff700 1 --2- 172.21.15.99:0/833908417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1eac15a170 0x7f1eac068b00 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1e940099b0 tx=0x7f1e94009250 comp rx=0 tx=0).stop 2023-10-08T00:12:16.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.162+0000 7f1eabfff700 1 -- 172.21.15.99:0/833908417 shutdown_connections 2023-10-08T00:12:16.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.162+0000 7f1eabfff700 1 --2- 172.21.15.99:0/833908417 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1eac069040 0x7f1eac0af4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:16.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.162+0000 7f1eabfff700 1 --2- 172.21.15.99:0/833908417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1eac15a170 0x7f1eac068b00 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:16.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.162+0000 7f1eabfff700 1 -- 172.21.15.99:0/833908417 >> 172.21.15.99:0/833908417 conn(0x7f1eac0d0bc0 msgr2=0x7f1eac0d0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:16.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.162+0000 7f1eabfff700 1 -- 172.21.15.99:0/833908417 shutdown_connections 2023-10-08T00:12:16.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.162+0000 7f1eabfff700 1 -- 172.21.15.99:0/833908417 wait complete. 2023-10-08T00:12:16.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.164+0000 7f1eabfff700 1 Processor -- start 2023-10-08T00:12:16.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.164+0000 7f1eabfff700 1 -- start start 2023-10-08T00:12:16.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.164+0000 7f1eabfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1eac069040 0x7f1eac0c8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:16.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.164+0000 7f1eabfff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1eac159da0 0x7f1eac0af920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:16.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.164+0000 7f1eabfff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1eac15a170 0x7f1eac0c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:16.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.164+0000 7f1eabfff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eac0cabc0 con 0x7f1eac069040 2023-10-08T00:12:16.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.164+0000 7f1eabfff700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eac0c4830 con 0x7f1eac15a170 2023-10-08T00:12:16.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.164+0000 7f1eabfff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eac0ca320 con 0x7f1eac159da0 2023-10-08T00:12:16.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.164+0000 7f1eaaffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1eac069040 0x7f1eac0c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:16.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.164+0000 7f1eaaffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1eac069040 0x7f1eac0c8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44620/0 (socket says 172.21.15.99:44620) 2023-10-08T00:12:16.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.164+0000 7f1eaaffd700 1 -- 172.21.15.99:0/2808881177 learned_addr learned my addr 172.21.15.99:0/2808881177 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:16.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.164+0000 7f1eaa7fc700 1 --2- 172.21.15.99:0/2808881177 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1eac159da0 0x7f1eac0af920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:16.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.165+0000 7f1eab7fe700 1 --2- 172.21.15.99:0/2808881177 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1eac15a170 0x7f1eac0c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:16.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.165+0000 7f1eaaffd700 1 -- 172.21.15.99:0/2808881177 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1eac159da0 msgr2=0x7f1eac0af920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:16.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.165+0000 7f1eaaffd700 1 --2- 172.21.15.99:0/2808881177 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1eac159da0 0x7f1eac0af920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:16.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.165+0000 7f1eaaffd700 1 -- 172.21.15.99:0/2808881177 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1eac15a170 msgr2=0x7f1eac0c9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:16.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.165+0000 7f1eaaffd700 1 --2- 172.21.15.99:0/2808881177 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1eac15a170 0x7f1eac0c9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:16.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.165+0000 7f1eaaffd700 1 -- 172.21.15.99:0/2808881177 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e940094f0 con 0x7f1eac069040 2023-10-08T00:12:16.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.165+0000 7f1eaaffd700 1 --2- 172.21.15.99:0/2808881177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1eac069040 0x7f1eac0c8150 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1ea400c590 tx=0x7f1ea400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:16.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.165+0000 7f1ea2ffd700 1 -- 172.21.15.99:0/2808881177 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4011590 con 0x7f1eac069040 2023-10-08T00:12:16.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.165+0000 7f1ea2ffd700 1 -- 172.21.15.99:0/2808881177 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ea4011710 con 0x7f1eac069040 2023-10-08T00:12:16.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.165+0000 7f1eabfff700 1 -- 172.21.15.99:0/2808881177 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eac0ca600 con 0x7f1eac069040 2023-10-08T00:12:16.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.165+0000 7f1ea2ffd700 1 -- 172.21.15.99:0/2808881177 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4011ad0 con 0x7f1eac069040 2023-10-08T00:12:16.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.165+0000 7f1eabfff700 1 -- 172.21.15.99:0/2808881177 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eac0afe60 con 0x7f1eac069040 2023-10-08T00:12:16.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.166+0000 7f1ea2ffd700 1 -- 172.21.15.99:0/2808881177 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1ea401b3b0 con 0x7f1eac069040 2023-10-08T00:12:16.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.167+0000 7f1ea2ffd700 1 --2- 172.21.15.99:0/2808881177 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1e8c066f30 0x7f1e8c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:16.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.167+0000 7f1ea2ffd700 1 -- 172.21.15.99:0/2808881177 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1ea4083a60 con 0x7f1eac069040 2023-10-08T00:12:16.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.167+0000 7f1eaa7fc700 1 --2- 172.21.15.99:0/2808881177 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1e8c066f30 0x7f1e8c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:16.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.167+0000 7f1eaa7fc700 1 --2- 172.21.15.99:0/2808881177 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1e8c066f30 0x7f1e8c0693e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1e940099b0 tx=0x7f1e94017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:16.176 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.169+0000 7f1eba730b80 1 Processor -- start 2023-10-08T00:12:16.176 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.169+0000 7f1eba730b80 1 -- start start 2023-10-08T00:12:16.177 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.169+0000 7f1eba730b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a0cd90480 0x560a0cd6f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:16.177 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.170+0000 7f1eba730b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560a0cd435f0 0x560a0cd90850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:16.177 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.170+0000 7f1eba730b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560a0cd45b80 0x560a0cd45ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:16.177 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.170+0000 7f1eba730b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a0cc9ca70 con 0x560a0cd90480 2023-10-08T00:12:16.177 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.170+0000 7f1eba730b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a0cc9a270 con 0x560a0cd435f0 2023-10-08T00:12:16.177 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.170+0000 7f1eba730b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x560a0cc96060 con 0x560a0cd45b80 2023-10-08T00:12:16.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.170+0000 7f1eaaffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a0cd90480 0x560a0cd6f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:16.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.170+0000 7f1eaaffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a0cd90480 0x560a0cd6f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44630/0 (socket says 172.21.15.99:44630) 2023-10-08T00:12:16.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.170+0000 7f1eaaffd700 1 -- 172.21.15.99:0/412728283 learned_addr learned my addr 172.21.15.99:0/412728283 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:16.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.170+0000 7f1eab7fe700 1 --2- 172.21.15.99:0/412728283 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560a0cd45b80 0x560a0cd45ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:16.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.170+0000 7f1eaa7fc700 1 --2- 172.21.15.99:0/412728283 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560a0cd435f0 0x560a0cd90850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:16.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.170+0000 7f1eaaffd700 1 -- 172.21.15.99:0/412728283 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560a0cd45b80 msgr2=0x560a0cd45ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:16.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.170+0000 7f1eaaffd700 1 --2- 172.21.15.99:0/412728283 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560a0cd45b80 0x560a0cd45ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:16.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.170+0000 7f1eaaffd700 1 -- 172.21.15.99:0/412728283 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560a0cd435f0 msgr2=0x560a0cd90850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:16.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.170+0000 7f1eaaffd700 1 --2- 172.21.15.99:0/412728283 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560a0cd435f0 0x560a0cd90850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:16.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.170+0000 7f1eaaffd700 1 -- 172.21.15.99:0/412728283 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a0cb30270 con 0x560a0cd90480 2023-10-08T00:12:16.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.170+0000 7f1eaaffd700 1 --2- 172.21.15.99:0/412728283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a0cd90480 0x560a0cd6f7d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4050df0 tx=0x7f1ea4081e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abde4253ea32e5c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:16.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.170+0000 7f1e8bfff700 1 -- 172.21.15.99:0/412728283 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea40548f0 con 0x560a0cd90480 2023-10-08T00:12:16.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.170+0000 7f1e8bfff700 1 -- 172.21.15.99:0/412728283 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ea4068070 con 0x560a0cd90480 2023-10-08T00:12:16.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.170+0000 7f1e8bfff700 1 -- 172.21.15.99:0/412728283 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4057b40 con 0x560a0cd90480 2023-10-08T00:12:16.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.171+0000 7f1eba730b80 1 -- 172.21.15.99:0/412728283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a0cd90480 msgr2=0x560a0cd6f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:16.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.171+0000 7f1eba730b80 1 --2- 172.21.15.99:0/412728283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a0cd90480 0x560a0cd6f7d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4050df0 tx=0x7f1ea4081e40 comp rx=0 tx=0).stop 2023-10-08T00:12:16.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.171+0000 7f1eba730b80 1 -- 172.21.15.99:0/412728283 shutdown_connections 2023-10-08T00:12:16.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.171+0000 7f1eba730b80 1 --2- 172.21.15.99:0/412728283 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560a0cd45b80 0x560a0cd45ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:16.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.171+0000 7f1eba730b80 1 --2- 172.21.15.99:0/412728283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a0cd90480 0x560a0cd6f7d0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:16.182 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.171+0000 7f1eba730b80 1 --2- 172.21.15.99:0/412728283 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560a0cd435f0 0x560a0cd90850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:16.182 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.171+0000 7f1eba730b80 1 -- 172.21.15.99:0/412728283 >> 172.21.15.99:0/412728283 conn(0x560a0cd470c0 msgr2=0x560a0cd478f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:16.182 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.171+0000 7f1eba730b80 1 -- 172.21.15.99:0/412728283 shutdown_connections 2023-10-08T00:12:16.182 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.171+0000 7f1eba730b80 1 -- 172.21.15.99:0/412728283 wait complete. 2023-10-08T00:12:16.182 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.171+0000 7f1eba730b80 1 -- start start 2023-10-08T00:12:16.182 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.172+0000 7f1eba730b80 1 Processor -- start 2023-10-08T00:12:16.183 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.172+0000 7f1eba730b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560a0cd435f0 0x560a0cd8f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:16.183 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.172+0000 7f1eba730b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a0cd45b80 0x560a0cda00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:16.183 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.172+0000 7f1eba730b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560a0cd90480 0x560a0cd47cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:16.183 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.172+0000 7f1eba730b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a0cc41890 con 0x560a0cd45b80 2023-10-08T00:12:16.184 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.172+0000 7f1eba730b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a0cc5cea0 con 0x560a0cd90480 2023-10-08T00:12:16.184 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.172+0000 7f1eba730b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x560a0cb18e30 con 0x560a0cd435f0 2023-10-08T00:12:16.184 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.172+0000 7f1eba730b80 10 client.0 _ll_register_callbacks cb 0x560a0cd092c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T00:12:16.184 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.173+0000 7f1eaa7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a0cd45b80 0x560a0cda00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:16.185 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.173+0000 7f1eaa7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a0cd45b80 0x560a0cda00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44640/0 (socket says 172.21.15.99:44640) 2023-10-08T00:12:16.185 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.173+0000 7f1eaa7fc700 1 -- 172.21.15.99:0/1501444623 learned_addr learned my addr 172.21.15.99:0/1501444623 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:16.185 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.173+0000 7f1eab7fe700 1 --2- 172.21.15.99:0/1501444623 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560a0cd90480 0x560a0cd47cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:16.185 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.173+0000 7f1eaaffd700 1 --2- 172.21.15.99:0/1501444623 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560a0cd435f0 0x560a0cd8f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:16.185 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.173+0000 7f1eaa7fc700 1 -- 172.21.15.99:0/1501444623 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560a0cd435f0 msgr2=0x560a0cd8f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:16.186 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.173+0000 7f1eaa7fc700 1 --2- 172.21.15.99:0/1501444623 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560a0cd435f0 0x560a0cd8f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:16.186 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.173+0000 7f1eaa7fc700 1 -- 172.21.15.99:0/1501444623 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560a0cd90480 msgr2=0x560a0cd47cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:16.186 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.173+0000 7f1eaa7fc700 1 --2- 172.21.15.99:0/1501444623 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560a0cd90480 0x560a0cd47cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:16.186 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.173+0000 7f1eaa7fc700 1 -- 172.21.15.99:0/1501444623 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eac0afe60 con 0x560a0cd45b80 2023-10-08T00:12:16.186 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.173+0000 7f1eaa7fc700 1 --2- 172.21.15.99:0/1501444623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a0cd45b80 0x560a0cda00a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1e9400b3c0 tx=0x7f1e94022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75d4d206f7000c22 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:16.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.173+0000 7f1e897fa700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T00:12:16.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.173+0000 7f1e897fa700 1 -- 172.21.15.99:0/1501444623 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9402f070 con 0x560a0cd45b80 2023-10-08T00:12:16.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.173+0000 7f1eba730b80 1 -- 172.21.15.99:0/1501444623 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560a0cd3acd0 con 0x560a0cd45b80 2023-10-08T00:12:16.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.173+0000 7f1e897fa700 1 -- 172.21.15.99:0/1501444623 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1e94011d20 con 0x560a0cd45b80 2023-10-08T00:12:16.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.173+0000 7f1e897fa700 1 -- 172.21.15.99:0/1501444623 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e9401bcf0 con 0x560a0cd45b80 2023-10-08T00:12:16.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.173+0000 7f1eba730b80 10 client.5430 fetch_fsmap learned FSMap version 24 2023-10-08T00:12:16.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.173+0000 7f1eba730b80 1 -- 172.21.15.99:0/1501444623 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x560a0cad2fe0 con 0x560a0cd45b80 2023-10-08T00:12:16.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.173+0000 7f1e897fa700 1 -- 172.21.15.99:0/1501444623 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560a0cd3acd0 con 0x560a0cd45b80 2023-10-08T00:12:16.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.174+0000 7f1e897fa700 1 -- 172.21.15.99:0/1501444623 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 24) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f1e9401b020 con 0x560a0cd45b80 2023-10-08T00:12:16.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.174+0000 7f1eba730b80 10 client.5430 fetch_fsmap finished waiting for FSMap version 24 2023-10-08T00:12:16.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.174+0000 7f1eba730b80 10 client.5430 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-08T00:12:16.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.174+0000 7f1eba730b80 20 client.5430 populate_metadata read hostname 'smithi099' 2023-10-08T00:12:16.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.174+0000 7f1eba730b80 1 --2- 172.21.15.99:0/1501444623 >> [v2:172.21.15.191:6835/2062242133,v1:172.21.15.191:6837/2062242133] conn(0x560a0cd69a40 0x560a0cd69e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:16.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.174+0000 7f1eba730b80 4 client.5430 mds_command: new command op to 5289 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-08T00:12:16.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.174+0000 7f1eba730b80 1 -- 172.21.15.99:0/1501444623 --> [v2:172.21.15.191:6835/2062242133,v1:172.21.15.191:6837/2062242133] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560a0cd17aa0 con 0x560a0cd69a40 2023-10-08T00:12:16.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.174+0000 7f1eaaffd700 1 --2- 172.21.15.99:0/1501444623 >> [v2:172.21.15.191:6835/2062242133,v1:172.21.15.191:6837/2062242133] conn(0x560a0cd69a40 0x560a0cd69e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:16.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.174+0000 7f1e897fa700 10 client.5430 ms_handle_connect on v2:172.21.15.191:6835/2062242133 2023-10-08T00:12:16.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.174+0000 7f1eaaffd700 1 --2- 172.21.15.99:0/1501444623 >> [v2:172.21.15.191:6835/2062242133,v1:172.21.15.191:6837/2062242133] conn(0x560a0cd69a40 0x560a0cd69e10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f43fc8e9be439e51 server_cookie=1d138940e6e24f44 in_seq=0 out_seq=0 2023-10-08T00:12:16.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.175+0000 7f1e897fa700 1 -- 172.21.15.99:0/1501444623 <== mds.0 v2:172.21.15.191:6835/2062242133 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x560a0cd17aa0 con 0x560a0cd69a40 2023-10-08T00:12:16.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.175+0000 7f1e897fa700 10 client.5430 handle_command_reply: tid=0 2023-10-08T00:12:16.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.175+0000 7f1eba730b80 1 client.5430 shutdown 2023-10-08T00:12:16.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.175+0000 7f1eba730b80 1 -- 172.21.15.99:0/1501444623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a0cd45b80 msgr2=0x560a0cda00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:16.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.175+0000 7f1eba730b80 1 --2- 172.21.15.99:0/1501444623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a0cd45b80 0x560a0cda00a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1e9400b3c0 tx=0x7f1e94022000 comp rx=0 tx=0).stop 2023-10-08T00:12:16.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.175+0000 7f1eba730b80 1 -- 172.21.15.99:0/1501444623 shutdown_connections 2023-10-08T00:12:16.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.175+0000 7f1eba730b80 1 --2- 172.21.15.99:0/1501444623 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560a0cd90480 0x560a0cd47cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:16.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.175+0000 7f1eba730b80 1 --2- 172.21.15.99:0/1501444623 >> [v2:172.21.15.191:6835/2062242133,v1:172.21.15.191:6837/2062242133] conn(0x560a0cd69a40 0x560a0cd69e10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:16.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.175+0000 7f1eba730b80 1 --2- 172.21.15.99:0/1501444623 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560a0cd435f0 0x560a0cd8f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:16.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.175+0000 7f1eba730b80 1 --2- 172.21.15.99:0/1501444623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a0cd45b80 0x560a0cda00a0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:16.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.175+0000 7f1e897fa700 0 client.5430 ms_handle_reset on v2:172.21.15.191:6835/2062242133 2023-10-08T00:12:16.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.175+0000 7f1eba730b80 1 -- 172.21.15.99:0/1501444623 >> 172.21.15.99:0/1501444623 conn(0x560a0cd470c0 msgr2=0x560a0cd3cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:16.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.175+0000 7f1eba730b80 1 -- 172.21.15.99:0/1501444623 shutdown_connections 2023-10-08T00:12:16.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.175+0000 7f1eba730b80 1 -- 172.21.15.99:0/1501444623 wait complete. 2023-10-08T00:12:16.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.175+0000 7f1eba730b80 20 client.5430 trim_cache size 0 max 16384 2023-10-08T00:12:16.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.192+0000 7f1eba730b80 1 Processor -- start 2023-10-08T00:12:16.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.192+0000 7f1eba730b80 1 -- start start 2023-10-08T00:12:16.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.193+0000 7f1eba730b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a0cd435f0 0x560a0cd724b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:16.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.193+0000 7f1eba730b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560a0cd45b80 0x560a0cd8f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:16.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.193+0000 7f1eba730b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560a0cd90480 0x560a0cd8bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:16.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.193+0000 7f1eba730b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a0cd3aca0 con 0x560a0cd435f0 2023-10-08T00:12:16.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.193+0000 7f1eba730b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a0cd8f9d0 con 0x560a0cd45b80 2023-10-08T00:12:16.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.193+0000 7f1eba730b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x560a0cd45170 con 0x560a0cd90480 2023-10-08T00:12:16.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.193+0000 7f1eaaffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a0cd435f0 0x560a0cd724b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:16.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.193+0000 7f1eaaffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a0cd435f0 0x560a0cd724b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44648/0 (socket says 172.21.15.99:44648) 2023-10-08T00:12:16.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.193+0000 7f1eaaffd700 1 -- 172.21.15.99:0/3931350949 learned_addr learned my addr 172.21.15.99:0/3931350949 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:16.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.193+0000 7f1eaaffd700 1 -- 172.21.15.99:0/3931350949 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560a0cd90480 msgr2=0x560a0cd8bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:16.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.193+0000 7f1eaaffd700 1 --2- 172.21.15.99:0/3931350949 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560a0cd90480 0x560a0cd8bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:16.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.193+0000 7f1eaaffd700 1 -- 172.21.15.99:0/3931350949 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560a0cd45b80 msgr2=0x560a0cd8f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:16.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.193+0000 7f1eaaffd700 1 --2- 172.21.15.99:0/3931350949 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560a0cd45b80 0x560a0cd8f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:16.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.193+0000 7f1eaaffd700 1 -- 172.21.15.99:0/3931350949 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a0cc64780 con 0x560a0cd435f0 2023-10-08T00:12:16.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.193+0000 7f1eaaffd700 1 --2- 172.21.15.99:0/3931350949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a0cd435f0 0x560a0cd724b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f1ea400c560 tx=0x7f1ea4002000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e673150b81125aa server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:16.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.193+0000 7f1ea0ff9700 1 -- 172.21.15.99:0/3931350949 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4081040 con 0x560a0cd435f0 2023-10-08T00:12:16.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.193+0000 7f1ea0ff9700 1 -- 172.21.15.99:0/3931350949 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ea40548f0 con 0x560a0cd435f0 2023-10-08T00:12:16.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.193+0000 7f1ea0ff9700 1 -- 172.21.15.99:0/3931350949 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4052c90 con 0x560a0cd435f0 2023-10-08T00:12:16.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.194+0000 7f1eba730b80 1 -- 172.21.15.99:0/3931350949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a0cd435f0 msgr2=0x560a0cd724b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:16.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.194+0000 7f1eba730b80 1 --2- 172.21.15.99:0/3931350949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a0cd435f0 0x560a0cd724b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f1ea400c560 tx=0x7f1ea4002000 comp rx=0 tx=0).stop 2023-10-08T00:12:16.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.194+0000 7f1eba730b80 1 -- 172.21.15.99:0/3931350949 shutdown_connections 2023-10-08T00:12:16.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.194+0000 7f1eba730b80 1 --2- 172.21.15.99:0/3931350949 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560a0cd90480 0x560a0cd8bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:16.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.194+0000 7f1eba730b80 1 --2- 172.21.15.99:0/3931350949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a0cd435f0 0x560a0cd724b0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:16.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.194+0000 7f1eba730b80 1 --2- 172.21.15.99:0/3931350949 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560a0cd45b80 0x560a0cd8f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:16.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.194+0000 7f1eba730b80 1 -- 172.21.15.99:0/3931350949 >> 172.21.15.99:0/3931350949 conn(0x560a0cd470c0 msgr2=0x560a0cd8c870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:16.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.194+0000 7f1eba730b80 1 -- 172.21.15.99:0/3931350949 shutdown_connections 2023-10-08T00:12:16.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.194+0000 7f1eba730b80 1 -- 172.21.15.99:0/3931350949 wait complete. 2023-10-08T00:12:16.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.194+0000 7f1eba730b80 1 -- start start 2023-10-08T00:12:16.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.194+0000 7f1eba730b80 1 Processor -- start 2023-10-08T00:12:16.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.195+0000 7f1eba730b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a0cd435f0 0x560a0cd42ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:16.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.195+0000 7f1eba730b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560a0cd45b80 0x560a0cd90850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:16.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.195+0000 7f1eba730b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560a0cd90480 0x560a0cdf5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:16.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.195+0000 7f1eba730b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a0cd688b0 con 0x560a0cd435f0 2023-10-08T00:12:16.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.195+0000 7f1eba730b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a0cd433e0 con 0x560a0cd45b80 2023-10-08T00:12:16.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.195+0000 7f1eba730b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x560a0cdf5990 con 0x560a0cd90480 2023-10-08T00:12:16.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.195+0000 7f1eba730b80 10 client.0 _ll_register_callbacks cb 0x560a0cd092c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T00:12:16.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.195+0000 7f1eaaffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a0cd435f0 0x560a0cd42ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:16.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.195+0000 7f1eaaffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a0cd435f0 0x560a0cd42ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44654/0 (socket says 172.21.15.99:44654) 2023-10-08T00:12:16.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.195+0000 7f1eaaffd700 1 -- 172.21.15.99:0/936414681 learned_addr learned my addr 172.21.15.99:0/936414681 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:16.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.196+0000 7f1eab7fe700 1 --2- 172.21.15.99:0/936414681 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560a0cd90480 0x560a0cdf5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:16.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.196+0000 7f1eaa7fc700 1 --2- 172.21.15.99:0/936414681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560a0cd45b80 0x560a0cd90850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:16.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.196+0000 7f1eaaffd700 1 -- 172.21.15.99:0/936414681 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560a0cd90480 msgr2=0x560a0cdf5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:16.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.196+0000 7f1eaaffd700 1 --2- 172.21.15.99:0/936414681 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560a0cd90480 0x560a0cdf5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:16.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.196+0000 7f1eaaffd700 1 -- 172.21.15.99:0/936414681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560a0cd45b80 msgr2=0x560a0cd90850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:16.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.196+0000 7f1eaaffd700 1 --2- 172.21.15.99:0/936414681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560a0cd45b80 0x560a0cd90850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:16.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.196+0000 7f1eaaffd700 1 -- 172.21.15.99:0/936414681 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ea406e040 con 0x560a0cd435f0 2023-10-08T00:12:16.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.196+0000 7f1eaaffd700 1 --2- 172.21.15.99:0/936414681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a0cd435f0 0x560a0cd42ea0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1ea40551a0 tx=0x7f1ea405e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3caffd9a99a6615b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:16.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.196+0000 7f1e897fa700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T00:12:16.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.196+0000 7f1e897fa700 1 -- 172.21.15.99:0/936414681 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4081070 con 0x560a0cd435f0 2023-10-08T00:12:16.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.196+0000 7f1e897fa700 1 -- 172.21.15.99:0/936414681 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1ea4053ff0 con 0x560a0cd435f0 2023-10-08T00:12:16.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.196+0000 7f1eba730b80 1 -- 172.21.15.99:0/936414681 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1ea4068070 con 0x560a0cd435f0 2023-10-08T00:12:16.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.196+0000 7f1e897fa700 1 -- 172.21.15.99:0/936414681 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ea405b400 con 0x560a0cd435f0 2023-10-08T00:12:16.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.196+0000 7f1eba730b80 10 client.5436 fetch_fsmap learned FSMap version 24 2023-10-08T00:12:16.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.196+0000 7f1eba730b80 1 -- 172.21.15.99:0/936414681 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x560a0cc4b470 con 0x560a0cd435f0 2023-10-08T00:12:16.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.196+0000 7f1e897fa700 1 -- 172.21.15.99:0/936414681 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ea4068070 con 0x560a0cd435f0 2023-10-08T00:12:16.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.196+0000 7f1e897fa700 1 -- 172.21.15.99:0/936414681 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 24) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f1ea4011c30 con 0x560a0cd435f0 2023-10-08T00:12:16.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.196+0000 7f1eba730b80 10 client.5436 fetch_fsmap finished waiting for FSMap version 24 2023-10-08T00:12:16.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.196+0000 7f1eba730b80 10 client.5436 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-08T00:12:16.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.196+0000 7f1eba730b80 20 client.5436 populate_metadata read hostname 'smithi099' 2023-10-08T00:12:16.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.196+0000 7f1eba730b80 1 --2- 172.21.15.99:0/936414681 >> [v2:172.21.15.191:6835/2062242133,v1:172.21.15.191:6837/2062242133] conn(0x560a0cd68470 0x560a0cdf4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:16.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.196+0000 7f1eba730b80 4 client.5436 mds_command: new command op to 5289 tid=0[{"prefix": "status"}] 2023-10-08T00:12:16.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.196+0000 7f1eba730b80 1 -- 172.21.15.99:0/936414681 --> [v2:172.21.15.191:6835/2062242133,v1:172.21.15.191:6837/2062242133] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560a0cc38e00 con 0x560a0cd68470 2023-10-08T00:12:16.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.196+0000 7f1eaa7fc700 1 --2- 172.21.15.99:0/936414681 >> [v2:172.21.15.191:6835/2062242133,v1:172.21.15.191:6837/2062242133] conn(0x560a0cd68470 0x560a0cdf4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:16.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.197+0000 7f1e897fa700 10 client.5436 ms_handle_connect on v2:172.21.15.191:6835/2062242133 2023-10-08T00:12:16.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.197+0000 7f1eaa7fc700 1 --2- 172.21.15.99:0/936414681 >> [v2:172.21.15.191:6835/2062242133,v1:172.21.15.191:6837/2062242133] conn(0x560a0cd68470 0x560a0cdf4930 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac579ef227ac85ec server_cookie=89923aa418493810 in_seq=0 out_seq=0 2023-10-08T00:12:16.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.197+0000 7f1e897fa700 1 -- 172.21.15.99:0/936414681 <== mds.0 v2:172.21.15.191:6835/2062242133 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x560a0cc38e00 con 0x560a0cd68470 2023-10-08T00:12:16.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.197+0000 7f1e897fa700 10 client.5436 handle_command_reply: tid=0 2023-10-08T00:12:16.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.197+0000 7f1eba730b80 1 client.5436 shutdown 2023-10-08T00:12:16.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.197+0000 7f1eba730b80 1 -- 172.21.15.99:0/936414681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a0cd435f0 msgr2=0x560a0cd42ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:16.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.197+0000 7f1eba730b80 1 --2- 172.21.15.99:0/936414681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a0cd435f0 0x560a0cd42ea0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1ea40551a0 tx=0x7f1ea405e000 comp rx=0 tx=0).stop 2023-10-08T00:12:16.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.197+0000 7f1eba730b80 1 -- 172.21.15.99:0/936414681 shutdown_connections 2023-10-08T00:12:16.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.197+0000 7f1eba730b80 1 --2- 172.21.15.99:0/936414681 >> [v2:172.21.15.191:6835/2062242133,v1:172.21.15.191:6837/2062242133] conn(0x560a0cd68470 0x560a0cdf4930 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:16.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.197+0000 7f1eba730b80 1 --2- 172.21.15.99:0/936414681 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560a0cd90480 0x560a0cdf5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:16.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.197+0000 7f1eba730b80 1 --2- 172.21.15.99:0/936414681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a0cd435f0 0x560a0cd42ea0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:16.211 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.197+0000 7f1eba730b80 1 --2- 172.21.15.99:0/936414681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560a0cd45b80 0x560a0cd90850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:16.211 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.197+0000 7f1e897fa700 0 client.5436 ms_handle_reset on v2:172.21.15.191:6835/2062242133 2023-10-08T00:12:16.211 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.197+0000 7f1eba730b80 1 -- 172.21.15.99:0/936414681 >> 172.21.15.99:0/936414681 conn(0x560a0cd470c0 msgr2=0x560a0cd3cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:16.211 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.197+0000 7f1eba730b80 1 -- 172.21.15.99:0/936414681 shutdown_connections 2023-10-08T00:12:16.211 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.197+0000 7f1eba730b80 1 -- 172.21.15.99:0/936414681 wait complete. 2023-10-08T00:12:16.211 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.197+0000 7f1eba730b80 20 client.5436 trim_cache size 0 max 16384 2023-10-08T00:12:16.211 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:12:16.211 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", 2023-10-08T00:12:16.211 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 0, 2023-10-08T00:12:16.212 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 5289, 2023-10-08T00:12:16.212 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2023-10-08T00:12:16.212 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2023-10-08T00:12:16.212 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2023-10-08T00:12:16.212 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 5.7661197670000002, 2023-10-08T00:12:16.212 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 24, 2023-10-08T00:12:16.212 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 45, 2023-10-08T00:12:16.212 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 45, 2023-10-08T00:12:16.212 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 11.747123413000001 2023-10-08T00:12:16.212 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:12:16.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.198+0000 7f1eabfff700 1 -- 172.21.15.99:0/2808881177 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1e8c066f30 msgr2=0x7f1e8c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:16.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.198+0000 7f1eabfff700 1 --2- 172.21.15.99:0/2808881177 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1e8c066f30 0x7f1e8c0693e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1e940099b0 tx=0x7f1e94017000 comp rx=0 tx=0).stop 2023-10-08T00:12:16.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.198+0000 7f1eabfff700 1 -- 172.21.15.99:0/2808881177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1eac069040 msgr2=0x7f1eac0c8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:16.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.198+0000 7f1eabfff700 1 --2- 172.21.15.99:0/2808881177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1eac069040 0x7f1eac0c8150 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1ea400c590 tx=0x7f1ea400c8a0 comp rx=0 tx=0).stop 2023-10-08T00:12:16.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.198+0000 7f1eabfff700 1 -- 172.21.15.99:0/2808881177 shutdown_connections 2023-10-08T00:12:16.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.198+0000 7f1eabfff700 1 --2- 172.21.15.99:0/2808881177 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1e8c066f30 0x7f1e8c0693e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:16.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.198+0000 7f1eabfff700 1 --2- 172.21.15.99:0/2808881177 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1eac15a170 0x7f1eac0c9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:16.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.198+0000 7f1eabfff700 1 --2- 172.21.15.99:0/2808881177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1eac069040 0x7f1eac0c8150 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:16.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.198+0000 7f1eabfff700 1 --2- 172.21.15.99:0/2808881177 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1eac159da0 0x7f1eac0af920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:16.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.198+0000 7f1eabfff700 1 -- 172.21.15.99:0/2808881177 >> 172.21.15.99:0/2808881177 conn(0x7f1eac0d0bc0 msgr2=0x7f1eac0b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:16.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.198+0000 7f1eabfff700 1 -- 172.21.15.99:0/2808881177 shutdown_connections 2023-10-08T00:12:16.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.198+0000 7f1eabfff700 1 -- 172.21.15.99:0/2808881177 wait complete. 2023-10-08T00:12:16.215 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T00:12:16.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.329+0000 7fe04b2f9700 1 Processor -- start 2023-10-08T00:12:16.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.329+0000 7fe04b2f9700 1 -- start start 2023-10-08T00:12:16.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.330+0000 7fe04b2f9700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe04415cbd0 0x7fe04415d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:16.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.330+0000 7fe04b2f9700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0440cb7c0 con 0x7fe0440cd6e0 2023-10-08T00:12:16.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.330+0000 7fe04b2f9700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe04415e290 con 0x7fe0440cdab0 2023-10-08T00:12:16.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.330+0000 7fe04b2f9700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe04415e850 con 0x7fe04415cbd0 2023-10-08T00:12:16.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.330+0000 7fe04aaf8700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe04415cbd0 0x7fe04415d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:16.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.330+0000 7fe04a2f7700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fe0440cd6e0 0x7fe0440cbc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45166/0 (socket says 172.21.15.99:45166) 2023-10-08T00:12:16.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.330+0000 7fe04a2f7700 1 -- 172.21.15.99:0/1602746272 learned_addr learned my addr 172.21.15.99:0/1602746272 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:12:16.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.330+0000 7fe04aaf8700 1 -- 172.21.15.99:0/1602746272 >> v1:172.21.15.191:6789/0 conn(0x7fe0440cdab0 legacy=0x7fe04415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:16.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.330+0000 7fe0492f5700 1 -- 172.21.15.99:0/1602746272 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3359302105 0 0) 0x7fe0440cb7c0 con 0x7fe0440cd6e0 2023-10-08T00:12:16.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.330+0000 7fe04aaf8700 1 -- 172.21.15.99:0/1602746272 >> v1:172.21.15.99:6789/0 conn(0x7fe0440cd6e0 legacy=0x7fe0440cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:16.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.330+0000 7fe04aaf8700 1 -- 172.21.15.99:0/1602746272 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe04415e570 con 0x7fe04415cbd0 2023-10-08T00:12:16.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.331+0000 7fe04aaf8700 1 --2- 172.21.15.99:0/1602746272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe04415cbd0 0x7fe04415d050 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe03c009e60 tx=0x7fe03c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=439d0343e40fc7f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:16.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.331+0000 7fe0492f5700 1 -- 172.21.15.99:0/1602746272 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03c00e070 con 0x7fe04415cbd0 2023-10-08T00:12:16.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.331+0000 7fe0492f5700 1 -- 172.21.15.99:0/1602746272 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe03c00bc30 con 0x7fe04415cbd0 2023-10-08T00:12:16.333 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.331+0000 7fe0492f5700 1 -- 172.21.15.99:0/1602746272 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03c012430 con 0x7fe04415cbd0 2023-10-08T00:12:16.335 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.331+0000 7fe04b2f9700 1 -- 172.21.15.99:0/1602746272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe04415cbd0 msgr2=0x7fe04415d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:16.335 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.331+0000 7fe04b2f9700 1 --2- 172.21.15.99:0/1602746272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe04415cbd0 0x7fe04415d050 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe03c009e60 tx=0x7fe03c009780 comp rx=0 tx=0).stop 2023-10-08T00:12:16.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.331+0000 7fe04b2f9700 1 -- 172.21.15.99:0/1602746272 shutdown_connections 2023-10-08T00:12:16.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.331+0000 7fe04b2f9700 1 --2- 172.21.15.99:0/1602746272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe04415cbd0 0x7fe04415d050 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:16.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.331+0000 7fe04b2f9700 1 -- 172.21.15.99:0/1602746272 >> 172.21.15.99:0/1602746272 conn(0x7fe044067660 msgr2=0x7fe044067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:16.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.331+0000 7fe04b2f9700 1 -- 172.21.15.99:0/1602746272 shutdown_connections 2023-10-08T00:12:16.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.331+0000 7fe04b2f9700 1 -- 172.21.15.99:0/1602746272 wait complete. 2023-10-08T00:12:16.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.332+0000 7fe04b2f9700 1 Processor -- start 2023-10-08T00:12:16.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.333+0000 7fe04b2f9700 1 -- start start 2023-10-08T00:12:16.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.333+0000 7fe04b2f9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0440cd6e0 0x7fe0440b61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:16.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.333+0000 7fe04b2f9700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0440cdab0 0x7fe0440cf510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:16.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.333+0000 7fe04b2f9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe04415cbd0 0x7fe0440cfa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:16.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.333+0000 7fe04b2f9700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0440cc360 con 0x7fe0440cd6e0 2023-10-08T00:12:16.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.333+0000 7fe04b2f9700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0440b6720 con 0x7fe0440cdab0 2023-10-08T00:12:16.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.333+0000 7fe04b2f9700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0440c9f90 con 0x7fe04415cbd0 2023-10-08T00:12:16.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.333+0000 7fe04a2f7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0440cd6e0 0x7fe0440b61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:16.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.333+0000 7fe04a2f7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0440cd6e0 0x7fe0440b61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44662/0 (socket says 172.21.15.99:44662) 2023-10-08T00:12:16.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.333+0000 7fe04a2f7700 1 -- 172.21.15.99:0/3026856135 learned_addr learned my addr 172.21.15.99:0/3026856135 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:16.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.333+0000 7fe04aaf8700 1 --2- 172.21.15.99:0/3026856135 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe04415cbd0 0x7fe0440cfa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:16.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.333+0000 7fe049af6700 1 --2- 172.21.15.99:0/3026856135 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0440cdab0 0x7fe0440cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:16.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.334+0000 7fe04a2f7700 1 -- 172.21.15.99:0/3026856135 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe04415cbd0 msgr2=0x7fe0440cfa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:16.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.334+0000 7fe04a2f7700 1 --2- 172.21.15.99:0/3026856135 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe04415cbd0 0x7fe0440cfa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:16.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.334+0000 7fe04a2f7700 1 -- 172.21.15.99:0/3026856135 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0440cdab0 msgr2=0x7fe0440cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:16.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.334+0000 7fe04a2f7700 1 --2- 172.21.15.99:0/3026856135 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0440cdab0 0x7fe0440cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:16.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.334+0000 7fe04a2f7700 1 -- 172.21.15.99:0/3026856135 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe03c009a20 con 0x7fe0440cd6e0 2023-10-08T00:12:16.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.334+0000 7fe049af6700 1 --2- 172.21.15.99:0/3026856135 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0440cdab0 0x7fe0440cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:12:16.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.334+0000 7fe04a2f7700 1 --2- 172.21.15.99:0/3026856135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0440cd6e0 0x7fe0440b61e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe04000c610 tx=0x7fe040001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:16.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.334+0000 7fe03a7fc700 1 -- 172.21.15.99:0/3026856135 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe040005b30 con 0x7fe0440cd6e0 2023-10-08T00:12:16.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.334+0000 7fe03a7fc700 1 -- 172.21.15.99:0/3026856135 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe040005c90 con 0x7fe0440cd6e0 2023-10-08T00:12:16.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.334+0000 7fe04b2f9700 1 -- 172.21.15.99:0/3026856135 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0440c9380 con 0x7fe0440cd6e0 2023-10-08T00:12:16.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.334+0000 7fe03a7fc700 1 -- 172.21.15.99:0/3026856135 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe040007300 con 0x7fe0440cd6e0 2023-10-08T00:12:16.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.334+0000 7fe04b2f9700 1 -- 172.21.15.99:0/3026856135 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0440c9940 con 0x7fe0440cd6e0 2023-10-08T00:12:16.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.335+0000 7fe04b2f9700 1 -- 172.21.15.99:0/3026856135 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe044156a00 con 0x7fe0440cd6e0 2023-10-08T00:12:16.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.335+0000 7fe03a7fc700 1 -- 172.21.15.99:0/3026856135 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe04001b3b0 con 0x7fe0440cd6e0 2023-10-08T00:12:16.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.335+0000 7fe03a7fc700 1 --2- 172.21.15.99:0/3026856135 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe028066fb0 0x7fe028069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:16.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.336+0000 7fe03a7fc700 1 -- 172.21.15.99:0/3026856135 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe040083b70 con 0x7fe0440cd6e0 2023-10-08T00:12:16.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.336+0000 7fe049af6700 1 --2- 172.21.15.99:0/3026856135 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe028066fb0 0x7fe028069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:16.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.336+0000 7fe049af6700 1 --2- 172.21.15.99:0/3026856135 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe028066fb0 0x7fe028069460 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe0440b53a0 tx=0x7fe034009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:16.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.340+0000 7fe03a7fc700 1 -- 172.21.15.99:0/3026856135 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe040050d40 con 0x7fe0440cd6e0 2023-10-08T00:12:16.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.568+0000 7fe04b2f9700 1 -- 172.21.15.99:0/3026856135 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe04404bb80 con 0x7fe0440cd6e0 2023-10-08T00:12:16.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.569+0000 7fe03a7fc700 1 -- 172.21.15.99:0/3026856135 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe040017070 con 0x7fe0440cd6e0 2023-10-08T00:12:16.570 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T00:12:16.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.571+0000 7fe04b2f9700 1 -- 172.21.15.99:0/3026856135 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe028066fb0 msgr2=0x7fe028069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:16.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.571+0000 7fe04b2f9700 1 --2- 172.21.15.99:0/3026856135 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe028066fb0 0x7fe028069460 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe0440b53a0 tx=0x7fe034009040 comp rx=0 tx=0).stop 2023-10-08T00:12:16.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.571+0000 7fe04b2f9700 1 -- 172.21.15.99:0/3026856135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0440cd6e0 msgr2=0x7fe0440b61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:16.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.571+0000 7fe04b2f9700 1 --2- 172.21.15.99:0/3026856135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0440cd6e0 0x7fe0440b61e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe04000c610 tx=0x7fe040001070 comp rx=0 tx=0).stop 2023-10-08T00:12:16.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.571+0000 7fe04b2f9700 1 -- 172.21.15.99:0/3026856135 shutdown_connections 2023-10-08T00:12:16.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.571+0000 7fe04b2f9700 1 --2- 172.21.15.99:0/3026856135 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe028066fb0 0x7fe028069460 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:16.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.571+0000 7fe04b2f9700 1 --2- 172.21.15.99:0/3026856135 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe04415cbd0 0x7fe0440cfa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:16.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.571+0000 7fe04b2f9700 1 --2- 172.21.15.99:0/3026856135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0440cd6e0 0x7fe0440b61e0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:16.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.571+0000 7fe04b2f9700 1 --2- 172.21.15.99:0/3026856135 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0440cdab0 0x7fe0440cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:16.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.571+0000 7fe04b2f9700 1 -- 172.21.15.99:0/3026856135 >> 172.21.15.99:0/3026856135 conn(0x7fe044067660 msgr2=0x7fe0440d1530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:16.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.571+0000 7fe04b2f9700 1 -- 172.21.15.99:0/3026856135 shutdown_connections 2023-10-08T00:12:16.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:16.571+0000 7fe04b2f9700 1 -- 172.21.15.99:0/3026856135 wait complete. 2023-10-08T00:12:16.583 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T00:12:16.583 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T00:12:16.584 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi099.front.sepia.ceph.com 2023-10-08T00:12:16.584 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T00:12:16.584 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T00:12:16.584 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T00:12:16.584 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T00:12:16.584 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T00:12:16.584 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T00:12:16.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:12:16.584 DEBUG:teuthology.orchestra.run.smithi099:> ip addr 2023-10-08T00:12:16.600 INFO:teuthology.orchestra.run.smithi099.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T00:12:16.600 INFO:teuthology.orchestra.run.smithi099.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T00:12:16.600 INFO:teuthology.orchestra.run.smithi099.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T00:12:16.600 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:12:16.600 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 ::1/128 scope host 2023-10-08T00:12:16.601 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:12:16.601 INFO:teuthology.orchestra.run.smithi099.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:12:16.601 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:10 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:12:16.601 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f0 2023-10-08T00:12:16.601 INFO:teuthology.orchestra.run.smithi099.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:12:16.601 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:11 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:12:16.601 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f1 2023-10-08T00:12:16.601 INFO:teuthology.orchestra.run.smithi099.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:12:16.602 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:02 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:12:16.602 INFO:teuthology.orchestra.run.smithi099.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T00:12:16.602 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:03 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:12:16.602 INFO:teuthology.orchestra.run.smithi099.stdout: inet 172.21.15.99/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T00:12:16.602 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft 42453sec preferred_lft 42453sec 2023-10-08T00:12:16.602 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::ec4:7aff:fe8f:d803/64 scope link 2023-10-08T00:12:16.602 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:12:16.602 INFO:teuthology.orchestra.run.smithi099.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T00:12:16.603 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 22:2c:1c:f6:42:e1 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:12:16.603 INFO:teuthology.orchestra.run.smithi099.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T00:12:16.603 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:12:16.603 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::600d:26ff:fede:5eec/64 scope link 2023-10-08T00:12:16.603 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:12:16.603 INFO:teuthology.orchestra.run.smithi099.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T00:12:16.603 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether ae:2f:b5:94:69:ba brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T00:12:16.603 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::ac2f:b5ff:fe94:69ba/64 scope link 2023-10-08T00:12:16.604 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:12:16.604 INFO:teuthology.orchestra.run.smithi099.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T00:12:16.604 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 22:2c:1c:f6:42:e1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T00:12:16.604 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::202c:1cff:fef6:42e1/64 scope link 2023-10-08T00:12:16.604 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:12:16.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:12:16.604 DEBUG:teuthology.orchestra.run.smithi099:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T00:12:16.663 INFO:teuthology.orchestra.run.smithi099.stdout:1 2023-10-08T00:12:16.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:12:16.665 DEBUG:teuthology.orchestra.run.smithi099:> route 2023-10-08T00:12:16.720 INFO:teuthology.orchestra.run.smithi099.stdout:Kernel IP routing table 2023-10-08T00:12:16.720 INFO:teuthology.orchestra.run.smithi099.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T00:12:16.720 INFO:teuthology.orchestra.run.smithi099.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T00:12:16.720 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T00:12:16.720 INFO:teuthology.orchestra.run.smithi099.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T00:12:16.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:12:16.721 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2023-10-08T00:12:16.721 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2023-10-08T00:12:16.721 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T00:12:16.721 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T00:12:16.721 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T00:12:16.721 DEBUG:teuthology.orchestra.run.smithi099:> ') 2023-10-08T00:12:16.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:12:16.820 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2023-10-08T00:12:16.834 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T00:12:16.834 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T00:12:16.835 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T00:12:16.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:12:16.835 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set brx.0 up 2023-10-08T00:12:16.899 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:12:16.899 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T00:12:16.954 INFO:teuthology.orchestra.run.smithi099.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T00:12:16.954 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:12:16.954 DEBUG:teuthology.orchestra.run.smithi099:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T00:12:17.008 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-08T00:12:17.009 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T00:12:17.009 DEBUG:teuthology.orchestra.run.smithi099:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:12:17.088 INFO:teuthology.orchestra.run.smithi099.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:12:17.088 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:12:17.106 INFO:teuthology.orchestra.run.smithi099.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:12:17.107 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:12:17.107 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: resolved to: "172.21.15.99:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T00:12:17.107 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: trying mount with new device syntax: 0@06962cdb-9a6f-417f-82fd-9e4e3dedbccf.cephfs=/ 2023-10-08T00:12:17.107 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.99:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T00:12:17.107 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T00:12:17.107 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T00:12:17.107 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T00:12:17.129 INFO:teuthology.orchestra.run.smithi099.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:12:17.129 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:12:17.129 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: resolved to: "172.21.15.99:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T00:12:17.129 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: trying mount with new device syntax: 0@06962cdb-9a6f-417f-82fd-9e4e3dedbccf.cephfs=/ 2023-10-08T00:12:17.129 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.99:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T00:12:17.129 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T00:12:17.129 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T00:12:17.129 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T00:12:17.129 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: resolved to: "172.21.15.99:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T00:12:17.130 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.99:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T00:12:17.130 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=06962cdb-9a6f-417f-82fd-9e4e3dedbccf" will pass to kernel 2023-10-08T00:12:17.131 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T00:12:17.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:12:17.131 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T00:12:17.197 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T00:12:17.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.350+0000 7fce5a642700 1 Processor -- start 2023-10-08T00:12:17.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.350+0000 7fce5a642700 1 -- start start 2023-10-08T00:12:17.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.351+0000 7fce5a642700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce54157870 0x7fce5415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:17.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.351+0000 7fce5a642700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fce54157c40 0x7fce54158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:17.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.351+0000 7fce5a642700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce5415cbb0 con 0x7fce5415a620 2023-10-08T00:12:17.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.351+0000 7fce5a642700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce5415bfa0 con 0x7fce54157c40 2023-10-08T00:12:17.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.351+0000 7fce5a642700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce5415c280 con 0x7fce54157870 2023-10-08T00:12:17.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.351+0000 7fce58bdf700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fce54157c40 0x7fce54158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:17.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.351+0000 7fce53fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce54157870 0x7fce5415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:17.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.351+0000 7fce53fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce54157870 0x7fce5415b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.99:39428/0 (socket says 172.21.15.99:39428) 2023-10-08T00:12:17.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.351+0000 7fce58bdf700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fce54157c40 0x7fce54158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.99:42662/0 (socket says 172.21.15.99:42662) 2023-10-08T00:12:17.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.351+0000 7fce537fe700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7fce5415a620 0x7fce5415d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:50936/0 (socket says 172.21.15.99:50936) 2023-10-08T00:12:17.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.351+0000 7fce53fff700 1 -- 172.21.15.99:0/3420957109 learned_addr learned my addr 172.21.15.99:0/3420957109 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:17.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.352+0000 7fce53fff700 1 -- 172.21.15.99:0/3420957109 >> v2:172.21.15.191:3300/0 conn(0x7fce54157c40 msgr2=0x7fce54158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:17.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.352+0000 7fce53fff700 1 --2- 172.21.15.99:0/3420957109 >> v2:172.21.15.191:3300/0 conn(0x7fce54157c40 0x7fce54158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:17.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.352+0000 7fce53fff700 1 -- 172.21.15.99:0/3420957109 >> v1:172.21.15.191:6789/0 conn(0x7fce5415a620 legacy=0x7fce5415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:17.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.352+0000 7fce53fff700 1 -- 172.21.15.99:0/3420957109 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce5415c560 con 0x7fce54157870 2023-10-08T00:12:17.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.352+0000 7fce58bdf700 1 --2- 172.21.15.99:0/3420957109 >> v2:172.21.15.191:3300/0 conn(0x7fce54157c40 0x7fce54158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:12:17.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.352+0000 7fce53fff700 1 --2- 172.21.15.99:0/3420957109 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce54157870 0x7fce5415b6a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fce440099b0 tx=0x7fce44009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=991bb4726f5b2931 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:17.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.352+0000 7fce52ffd700 1 -- 172.21.15.99:0/3420957109 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce44009ea0 con 0x7fce54157870 2023-10-08T00:12:17.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.352+0000 7fce52ffd700 1 -- 172.21.15.99:0/3420957109 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce4400b860 con 0x7fce54157870 2023-10-08T00:12:17.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.352+0000 7fce52ffd700 1 -- 172.21.15.99:0/3420957109 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4400bc00 con 0x7fce54157870 2023-10-08T00:12:17.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.353+0000 7fce5a642700 1 -- 172.21.15.99:0/3420957109 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce54157870 msgr2=0x7fce5415b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:17.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.353+0000 7fce5a642700 1 --2- 172.21.15.99:0/3420957109 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce54157870 0x7fce5415b6a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fce440099b0 tx=0x7fce44009250 comp rx=0 tx=0).stop 2023-10-08T00:12:17.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.353+0000 7fce5a642700 1 -- 172.21.15.99:0/3420957109 shutdown_connections 2023-10-08T00:12:17.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.353+0000 7fce5a642700 1 --2- 172.21.15.99:0/3420957109 >> v2:172.21.15.191:3300/0 conn(0x7fce54157c40 0x7fce54158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:17.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.353+0000 7fce5a642700 1 --2- 172.21.15.99:0/3420957109 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce54157870 0x7fce5415b6a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:17.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.353+0000 7fce5a642700 1 -- 172.21.15.99:0/3420957109 >> 172.21.15.99:0/3420957109 conn(0x7fce54068ce0 msgr2=0x7fce540690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:17.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.353+0000 7fce5a642700 1 -- 172.21.15.99:0/3420957109 shutdown_connections 2023-10-08T00:12:17.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.353+0000 7fce5a642700 1 -- 172.21.15.99:0/3420957109 wait complete. 2023-10-08T00:12:17.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.354+0000 7fce5a642700 1 Processor -- start 2023-10-08T00:12:17.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.354+0000 7fce5a642700 1 -- start start 2023-10-08T00:12:17.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.355+0000 7fce5a642700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce54157870 0x7fce5414f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:17.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.355+0000 7fce5a642700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce54157c40 0x7fce541492d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:17.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.355+0000 7fce5a642700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce5415a620 0x7fce54148490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:17.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.355+0000 7fce5a642700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce54149d50 con 0x7fce54157870 2023-10-08T00:12:17.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.355+0000 7fce5a642700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5414a030 con 0x7fce54157c40 2023-10-08T00:12:17.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.355+0000 7fce5a642700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce5414a310 con 0x7fce5415a620 2023-10-08T00:12:17.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.355+0000 7fce53fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce54157870 0x7fce5414f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:17.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.355+0000 7fce53fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce54157870 0x7fce5414f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44664/0 (socket says 172.21.15.99:44664) 2023-10-08T00:12:17.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.355+0000 7fce53fff700 1 -- 172.21.15.99:0/488025487 learned_addr learned my addr 172.21.15.99:0/488025487 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:17.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.355+0000 7fce58bdf700 1 --2- 172.21.15.99:0/488025487 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce5415a620 0x7fce54148490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:17.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.355+0000 7fce537fe700 1 --2- 172.21.15.99:0/488025487 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce54157c40 0x7fce541492d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:17.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.355+0000 7fce53fff700 1 -- 172.21.15.99:0/488025487 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce5415a620 msgr2=0x7fce54148490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:17.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.355+0000 7fce53fff700 1 --2- 172.21.15.99:0/488025487 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce5415a620 0x7fce54148490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:17.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.355+0000 7fce53fff700 1 -- 172.21.15.99:0/488025487 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce54157c40 msgr2=0x7fce541492d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:17.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.355+0000 7fce53fff700 1 --2- 172.21.15.99:0/488025487 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce54157c40 0x7fce541492d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:17.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.355+0000 7fce53fff700 1 -- 172.21.15.99:0/488025487 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce4c009a20 con 0x7fce54157870 2023-10-08T00:12:17.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.355+0000 7fce53fff700 1 --2- 172.21.15.99:0/488025487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce54157870 0x7fce5414f610 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fce5415c3c0 tx=0x7fce44017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:17.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.355+0000 7fce3bfff700 1 -- 172.21.15.99:0/488025487 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4400bbe0 con 0x7fce54157870 2023-10-08T00:12:17.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.355+0000 7fce3bfff700 1 -- 172.21.15.99:0/488025487 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce44011420 con 0x7fce54157870 2023-10-08T00:12:17.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.355+0000 7fce5a642700 1 -- 172.21.15.99:0/488025487 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce440094f0 con 0x7fce54157870 2023-10-08T00:12:17.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.355+0000 7fce3bfff700 1 -- 172.21.15.99:0/488025487 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce440117c0 con 0x7fce54157870 2023-10-08T00:12:17.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.355+0000 7fce5a642700 1 -- 172.21.15.99:0/488025487 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce5414a8d0 con 0x7fce54157870 2023-10-08T00:12:17.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.356+0000 7fce5a642700 1 -- 172.21.15.99:0/488025487 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce5414aaf0 con 0x7fce54157870 2023-10-08T00:12:17.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.357+0000 7fce3bfff700 1 -- 172.21.15.99:0/488025487 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fce44011bf0 con 0x7fce54157870 2023-10-08T00:12:17.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.357+0000 7fce3bfff700 1 --2- 172.21.15.99:0/488025487 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fce40066f30 0x7fce400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:17.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.357+0000 7fce3bfff700 1 -- 172.21.15.99:0/488025487 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fce44085f80 con 0x7fce54157870 2023-10-08T00:12:17.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.357+0000 7fce537fe700 1 --2- 172.21.15.99:0/488025487 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fce40066f30 0x7fce400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:17.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.358+0000 7fce537fe700 1 --2- 172.21.15.99:0/488025487 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fce40066f30 0x7fce400693e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fce48007fd0 tx=0x7fce48009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:17.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.361+0000 7fce3bfff700 1 -- 172.21.15.99:0/488025487 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fce440b75a0 con 0x7fce54157870 2023-10-08T00:12:17.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.588+0000 7fce5a642700 1 -- 172.21.15.99:0/488025487 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fce5404c3c0 con 0x7fce54157870 2023-10-08T00:12:17.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.588+0000 7fce3bfff700 1 -- 172.21.15.99:0/488025487 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+73 (secure 0 0 0) 0x7fce440531f0 con 0x7fce54157870 2023-10-08T00:12:17.589 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T00:12:17.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.590+0000 7fce5a642700 1 -- 172.21.15.99:0/488025487 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fce40066f30 msgr2=0x7fce400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:17.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.590+0000 7fce5a642700 1 --2- 172.21.15.99:0/488025487 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fce40066f30 0x7fce400693e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fce48007fd0 tx=0x7fce48009040 comp rx=0 tx=0).stop 2023-10-08T00:12:17.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.590+0000 7fce5a642700 1 -- 172.21.15.99:0/488025487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce54157870 msgr2=0x7fce5414f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:17.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.590+0000 7fce5a642700 1 --2- 172.21.15.99:0/488025487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce54157870 0x7fce5414f610 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fce5415c3c0 tx=0x7fce44017000 comp rx=0 tx=0).stop 2023-10-08T00:12:17.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.590+0000 7fce5a642700 1 -- 172.21.15.99:0/488025487 shutdown_connections 2023-10-08T00:12:17.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.590+0000 7fce5a642700 1 --2- 172.21.15.99:0/488025487 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fce40066f30 0x7fce400693e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:17.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.590+0000 7fce5a642700 1 --2- 172.21.15.99:0/488025487 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce5415a620 0x7fce54148490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:17.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.590+0000 7fce5a642700 1 --2- 172.21.15.99:0/488025487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce54157870 0x7fce5414f610 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:17.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.590+0000 7fce5a642700 1 --2- 172.21.15.99:0/488025487 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce54157c40 0x7fce541492d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:17.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.590+0000 7fce5a642700 1 -- 172.21.15.99:0/488025487 >> 172.21.15.99:0/488025487 conn(0x7fce54068ce0 msgr2=0x7fce540b4950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:17.592 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.590+0000 7fce5a642700 1 -- 172.21.15.99:0/488025487 shutdown_connections 2023-10-08T00:12:17.592 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:17.590+0000 7fce5a642700 1 -- 172.21.15.99:0/488025487 wait complete. 2023-10-08T00:12:17.602 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T00:12:17.603 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T00:12:17.603 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi191.front.sepia.ceph.com 2023-10-08T00:12:17.603 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-08T00:12:17.603 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-08T00:12:17.603 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T00:12:17.603 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T00:12:17.603 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T00:12:17.603 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T00:12:17.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:12:17.604 DEBUG:teuthology.orchestra.run.smithi191:> ip addr 2023-10-08T00:12:17.623 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T00:12:17.623 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T00:12:17.623 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T00:12:17.623 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:12:17.623 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2023-10-08T00:12:17.623 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:12:17.623 INFO:teuthology.orchestra.run.smithi191.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:12:17.623 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f6 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:12:17.623 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f0 2023-10-08T00:12:17.624 INFO:teuthology.orchestra.run.smithi191.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:12:17.624 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f7 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:12:17.624 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f1 2023-10-08T00:12:17.624 INFO:teuthology.orchestra.run.smithi191.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:12:17.624 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:10 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:12:17.624 INFO:teuthology.orchestra.run.smithi191.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T00:12:17.624 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:11 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:12:17.624 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.21.15.191/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T00:12:17.624 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft 42454sec preferred_lft 42454sec 2023-10-08T00:12:17.625 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec4:7aff:fe88:7211/64 scope link 2023-10-08T00:12:17.625 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:12:17.625 INFO:teuthology.orchestra.run.smithi191.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T00:12:17.625 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 9a:bc:39:72:87:3d brd ff:ff:ff:ff:ff:ff 2023-10-08T00:12:17.625 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T00:12:17.625 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:12:17.625 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec77:a7ff:fea4:fbb/64 scope link 2023-10-08T00:12:17.625 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:12:17.625 INFO:teuthology.orchestra.run.smithi191.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T00:12:17.625 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether ea:1c:d9:46:05:b3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-08T00:12:17.625 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::e81c:d9ff:fe46:5b3/64 scope link 2023-10-08T00:12:17.625 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:12:17.626 INFO:teuthology.orchestra.run.smithi191.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T00:12:17.626 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 9a:bc:39:72:87:3d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-08T00:12:17.626 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::98bc:39ff:fe72:873d/64 scope link 2023-10-08T00:12:17.626 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:12:17.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:12:17.626 DEBUG:teuthology.orchestra.run.smithi191:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T00:12:17.690 INFO:teuthology.orchestra.run.smithi191.stdout:1 2023-10-08T00:12:17.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:12:17.692 DEBUG:teuthology.orchestra.run.smithi191:> route 2023-10-08T00:12:17.749 INFO:teuthology.orchestra.run.smithi191.stdout:Kernel IP routing table 2023-10-08T00:12:17.750 INFO:teuthology.orchestra.run.smithi191.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T00:12:17.750 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T00:12:17.750 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T00:12:17.750 INFO:teuthology.orchestra.run.smithi191.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T00:12:17.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:12:17.750 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2023-10-08T00:12:17.750 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2023-10-08T00:12:17.751 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T00:12:17.751 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T00:12:17.751 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T00:12:17.751 DEBUG:teuthology.orchestra.run.smithi191:> ') 2023-10-08T00:12:17.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:12:17.854 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2023-10-08T00:12:17.869 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-08T00:12:17.869 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-08T00:12:17.869 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T00:12:17.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:12:17.869 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set brx.0 up 2023-10-08T00:12:17.934 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:12:17.934 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-08T00:12:17.990 INFO:teuthology.orchestra.run.smithi191.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-08T00:12:17.990 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:12:17.990 DEBUG:teuthology.orchestra.run.smithi191:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-08T00:12:18.045 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-08T00:12:18.045 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T00:12:18.045 DEBUG:teuthology.orchestra.run.smithi191:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:12:18.125 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:12:18.126 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:12:18.145 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:12:18.145 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:12:18.145 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.99:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T00:12:18.145 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@06962cdb-9a6f-417f-82fd-9e4e3dedbccf.cephfs=/ 2023-10-08T00:12:18.145 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.99:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T00:12:18.145 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T00:12:18.145 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T00:12:18.145 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T00:12:18.165 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:12:18.166 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:12:18.166 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.99:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T00:12:18.166 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@06962cdb-9a6f-417f-82fd-9e4e3dedbccf.cephfs=/ 2023-10-08T00:12:18.166 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.99:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T00:12:18.166 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T00:12:18.166 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T00:12:18.166 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T00:12:18.166 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.99:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T00:12:18.166 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.99:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T00:12:18.167 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=06962cdb-9a6f-417f-82fd-9e4e3dedbccf" will pass to kernel 2023-10-08T00:12:18.168 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T00:12:18.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:12:18.168 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-08T00:12:18.232 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:12:18.352 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.351+0000 7ff9d0b2f700 1 Processor -- start 2023-10-08T00:12:18.352 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.351+0000 7ff9d0b2f700 1 -- start start 2023-10-08T00:12:18.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.352+0000 7ff9d0b2f700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff9cc157c40 0x7ff9cc158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:18.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.352+0000 7ff9d0b2f700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9cc15cbb0 con 0x7ff9cc15a620 2023-10-08T00:12:18.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.352+0000 7ff9d0b2f700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9cc15bfa0 con 0x7ff9cc157870 2023-10-08T00:12:18.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.352+0000 7ff9d0b2f700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9cc15bdb0 con 0x7ff9cc157c40 2023-10-08T00:12:18.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.352+0000 7ff9c9d9b700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7ff9cc15a620 0x7ff9cc15d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45174/0 (socket says 172.21.15.99:45174) 2023-10-08T00:12:18.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.352+0000 7ff9c9d9b700 1 -- 172.21.15.99:0/3874743528 learned_addr learned my addr 172.21.15.99:0/3874743528 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:12:18.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.352+0000 7ff9cad9d700 1 --2- 172.21.15.99:0/3874743528 >> v2:172.21.15.191:3300/0 conn(0x7ff9cc157c40 0x7ff9cc158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:18.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.352+0000 7ff9c959a700 1 -- 172.21.15.99:0/3874743528 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1731532592 0 0) 0x7ff9cc15cbb0 con 0x7ff9cc15a620 2023-10-08T00:12:18.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.353+0000 7ff9c959a700 1 -- 172.21.15.99:0/3874743528 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9b8003570 con 0x7ff9cc15a620 2023-10-08T00:12:18.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.353+0000 7ff9c959a700 1 -- 172.21.15.99:0/3874743528 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1448593440 0 0) 0x7ff9cc15bfa0 con 0x7ff9cc157870 2023-10-08T00:12:18.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.353+0000 7ff9c959a700 1 -- 172.21.15.99:0/3874743528 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9cc15cbb0 con 0x7ff9cc157870 2023-10-08T00:12:18.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.353+0000 7ff9cad9d700 1 -- 172.21.15.99:0/3874743528 >> v1:172.21.15.191:6789/0 conn(0x7ff9cc157870 legacy=0x7ff9cc15b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:18.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.353+0000 7ff9cad9d700 1 -- 172.21.15.99:0/3874743528 >> v1:172.21.15.99:6789/0 conn(0x7ff9cc15a620 legacy=0x7ff9cc15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:18.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.353+0000 7ff9cad9d700 1 -- 172.21.15.99:0/3874743528 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9cc15c280 con 0x7ff9cc157c40 2023-10-08T00:12:18.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.353+0000 7ff9cad9d700 1 --2- 172.21.15.99:0/3874743528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9cc157c40 0x7ff9cc158110 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff9c40098e0 tx=0x7ff9c40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcc4e9e3549dd515 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:18.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.353+0000 7ff9c959a700 1 -- 172.21.15.99:0/3874743528 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c4011630 con 0x7ff9cc157c40 2023-10-08T00:12:18.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.353+0000 7ff9c959a700 1 -- 172.21.15.99:0/3874743528 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9c4011790 con 0x7ff9cc157c40 2023-10-08T00:12:18.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.353+0000 7ff9c959a700 1 -- 172.21.15.99:0/3874743528 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c4011b30 con 0x7ff9cc157c40 2023-10-08T00:12:18.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.354+0000 7ff9d0b2f700 1 -- 172.21.15.99:0/3874743528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9cc157c40 msgr2=0x7ff9cc158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:18.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.354+0000 7ff9d0b2f700 1 --2- 172.21.15.99:0/3874743528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9cc157c40 0x7ff9cc158110 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff9c40098e0 tx=0x7ff9c40091a0 comp rx=0 tx=0).stop 2023-10-08T00:12:18.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.354+0000 7ff9d0b2f700 1 -- 172.21.15.99:0/3874743528 shutdown_connections 2023-10-08T00:12:18.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.354+0000 7ff9d0b2f700 1 --2- 172.21.15.99:0/3874743528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9cc157c40 0x7ff9cc158110 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:18.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.354+0000 7ff9d0b2f700 1 -- 172.21.15.99:0/3874743528 >> 172.21.15.99:0/3874743528 conn(0x7ff9cc068ce0 msgr2=0x7ff9cc0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:18.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.354+0000 7ff9d0b2f700 1 -- 172.21.15.99:0/3874743528 shutdown_connections 2023-10-08T00:12:18.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.354+0000 7ff9d0b2f700 1 -- 172.21.15.99:0/3874743528 wait complete. 2023-10-08T00:12:18.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.355+0000 7ff9d0b2f700 1 Processor -- start 2023-10-08T00:12:18.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.355+0000 7ff9d0b2f700 1 -- start start 2023-10-08T00:12:18.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.355+0000 7ff9d0b2f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9cc157870 0x7ff9cc148780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:18.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.355+0000 7ff9d0b2f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9cc157c40 0x7ff9cc0a8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:18.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.355+0000 7ff9d0b2f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9cc15a620 0x7ff9cc0a9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:18.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.355+0000 7ff9d0b2f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc16e6b0 con 0x7ff9cc157870 2023-10-08T00:12:18.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.355+0000 7ff9d0b2f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc156390 con 0x7ff9cc15a620 2023-10-08T00:12:18.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.355+0000 7ff9d0b2f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc16ddc0 con 0x7ff9cc157c40 2023-10-08T00:12:18.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.356+0000 7ff9ca59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9cc157870 0x7ff9cc148780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:18.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.356+0000 7ff9ca59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9cc157870 0x7ff9cc148780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44680/0 (socket says 172.21.15.99:44680) 2023-10-08T00:12:18.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.356+0000 7ff9ca59c700 1 -- 172.21.15.99:0/1210063918 learned_addr learned my addr 172.21.15.99:0/1210063918 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:18.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.356+0000 7ff9c9d9b700 1 --2- 172.21.15.99:0/1210063918 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9cc157c40 0x7ff9cc0a8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:18.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.356+0000 7ff9cad9d700 1 --2- 172.21.15.99:0/1210063918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9cc15a620 0x7ff9cc0a9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:18.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.356+0000 7ff9ca59c700 1 -- 172.21.15.99:0/1210063918 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9cc157c40 msgr2=0x7ff9cc0a8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:18.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.356+0000 7ff9ca59c700 1 --2- 172.21.15.99:0/1210063918 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9cc157c40 0x7ff9cc0a8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:18.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.356+0000 7ff9ca59c700 1 -- 172.21.15.99:0/1210063918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9cc15a620 msgr2=0x7ff9cc0a9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:18.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.356+0000 7ff9ca59c700 1 --2- 172.21.15.99:0/1210063918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9cc15a620 0x7ff9cc0a9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:18.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.356+0000 7ff9ca59c700 1 -- 172.21.15.99:0/1210063918 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9c4009420 con 0x7ff9cc157870 2023-10-08T00:12:18.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.356+0000 7ff9ca59c700 1 --2- 172.21.15.99:0/1210063918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9cc157870 0x7ff9cc148780 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc00cb10 tx=0x7ff9bc002820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:18.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.356+0000 7ff9c27fc700 1 -- 172.21.15.99:0/1210063918 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc00a760 con 0x7ff9cc157870 2023-10-08T00:12:18.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.356+0000 7ff9c27fc700 1 -- 172.21.15.99:0/1210063918 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9bc00a8c0 con 0x7ff9cc157870 2023-10-08T00:12:18.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.356+0000 7ff9d0b2f700 1 -- 172.21.15.99:0/1210063918 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9cc0a9930 con 0x7ff9cc157870 2023-10-08T00:12:18.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.356+0000 7ff9c27fc700 1 -- 172.21.15.99:0/1210063918 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc00ac60 con 0x7ff9cc157870 2023-10-08T00:12:18.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.356+0000 7ff9d0b2f700 1 -- 172.21.15.99:0/1210063918 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9cc16e340 con 0x7ff9cc157870 2023-10-08T00:12:18.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.357+0000 7ff9d0b2f700 1 -- 172.21.15.99:0/1210063918 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9cc04c3c0 con 0x7ff9cc157870 2023-10-08T00:12:18.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.358+0000 7ff9c27fc700 1 -- 172.21.15.99:0/1210063918 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff9bc012630 con 0x7ff9cc157870 2023-10-08T00:12:18.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.358+0000 7ff9c27fc700 1 --2- 172.21.15.99:0/1210063918 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff9ac066f30 0x7ff9ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:18.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.358+0000 7ff9c27fc700 1 -- 172.21.15.99:0/1210063918 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff9bc084a10 con 0x7ff9cc157870 2023-10-08T00:12:18.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.358+0000 7ff9c9d9b700 1 --2- 172.21.15.99:0/1210063918 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff9ac066f30 0x7ff9ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:18.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.359+0000 7ff9c9d9b700 1 --2- 172.21.15.99:0/1210063918 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff9ac066f30 0x7ff9ac0693e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc0b61c0 tx=0x7ff9b4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:18.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.362+0000 7ff9c27fc700 1 -- 172.21.15.99:0/1210063918 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff9bc051d30 con 0x7ff9cc157870 2023-10-08T00:12:18.590 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.590+0000 7ff9d0b2f700 1 -- 172.21.15.99:0/1210063918 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff9cc15c560 con 0x7ff9cc157870 2023-10-08T00:12:18.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.590+0000 7ff9c27fc700 1 -- 172.21.15.99:0/1210063918 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4603 (secure 0 0 0) 0x7ff9bc055350 con 0x7ff9cc157870 2023-10-08T00:12:18.591 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:12:18.591 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5146,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/1438931938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1438931938},{"type":"v1","addr":"172.21.15.191:6836","nonce":1438931938}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5280,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/3090674680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3090674680},{"type":"v1","addr":"172.21.15.99:6833","nonce":3090674680}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5283,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/4025282357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":4025282357},{"type":"v1","addr":"172.21.15.99:6835","nonce":4025282357}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:12:09.803824+0000","modified":"2023-10-08T00:12:13.436570+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5289},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5289":{"gid":5289,"name":"d","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/2062242133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":2062242133},{"type":"v1","addr":"172.21.15.191:6837","nonce":2062242133}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-08T00:12:18.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.593+0000 7ff9d0b2f700 1 -- 172.21.15.99:0/1210063918 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff9ac066f30 msgr2=0x7ff9ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:18.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.593+0000 7ff9d0b2f700 1 --2- 172.21.15.99:0/1210063918 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff9ac066f30 0x7ff9ac0693e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc0b61c0 tx=0x7ff9b4009040 comp rx=0 tx=0).stop 2023-10-08T00:12:18.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.593+0000 7ff9d0b2f700 1 -- 172.21.15.99:0/1210063918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9cc157870 msgr2=0x7ff9cc148780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:18.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.593+0000 7ff9d0b2f700 1 --2- 172.21.15.99:0/1210063918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9cc157870 0x7ff9cc148780 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc00cb10 tx=0x7ff9bc002820 comp rx=0 tx=0).stop 2023-10-08T00:12:18.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.593+0000 7ff9d0b2f700 1 -- 172.21.15.99:0/1210063918 shutdown_connections 2023-10-08T00:12:18.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.593+0000 7ff9d0b2f700 1 --2- 172.21.15.99:0/1210063918 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff9ac066f30 0x7ff9ac0693e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:18.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.593+0000 7ff9d0b2f700 1 --2- 172.21.15.99:0/1210063918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9cc15a620 0x7ff9cc0a9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:18.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.593+0000 7ff9d0b2f700 1 --2- 172.21.15.99:0/1210063918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9cc157870 0x7ff9cc148780 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:18.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.593+0000 7ff9d0b2f700 1 --2- 172.21.15.99:0/1210063918 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9cc157c40 0x7ff9cc0a8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:18.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.593+0000 7ff9d0b2f700 1 -- 172.21.15.99:0/1210063918 >> 172.21.15.99:0/1210063918 conn(0x7ff9cc068ce0 msgr2=0x7ff9cc0690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:18.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.593+0000 7ff9d0b2f700 1 -- 172.21.15.99:0/1210063918 shutdown_connections 2023-10-08T00:12:18.595 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:18.593+0000 7ff9d0b2f700 1 -- 172.21.15.99:0/1210063918 wait complete. 2023-10-08T00:12:18.595 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 24 2023-10-08T00:12:18.605 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json status 2023-10-08T00:12:18.738 INFO:teuthology.orchestra.run.smithi191.stdout:{"cluster_fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","whoami":0,"id":5289,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":8.2966951739999999,"mdsmap_epoch":24,"osdmap_epoch":45,"osdmap_epoch_barrier":45,"uptime":14.277699361} 2023-10-08T00:12:18.738 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", "fs_name": "cephfs", "id": 5289, "mdsmap_epoch": 24, "osdmap_epoch": 45, "osdmap_epoch_barrier": 45, "rank_uptime": 8.296695174, "state": "up:active", "uptime": 14.277699361, "want_state": "up:active", "whoami": 0 } 2023-10-08T00:12:18.739 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-10-08T00:12:18.867 INFO:teuthology.orchestra.run.smithi099.stdout:{"osd_mon_report_interval":"5"} 2023-10-08T00:12:18.868 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-10-08T00:12:18.868 INFO:tasks.cephfs.test_full:Writing 531MB should fill this cluster 2023-10-08T00:12:18.868 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:12:18.868 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=265 seek=0) 2023-10-08T00:12:20.245 INFO:teuthology.orchestra.run.smithi099.stderr:265+0 records in 2023-10-08T00:12:20.245 INFO:teuthology.orchestra.run.smithi099.stderr:265+0 records out 2023-10-08T00:12:20.245 INFO:teuthology.orchestra.run.smithi099.stderr:277872640 bytes (278 MB, 265 MiB) copied, 1.35916 s, 204 MB/s 2023-10-08T00:12:20.246 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:12:20.246 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=292 seek=0) 2023-10-08T00:12:21.754 INFO:teuthology.orchestra.run.smithi099.stderr:292+0 records in 2023-10-08T00:12:21.755 INFO:teuthology.orchestra.run.smithi099.stderr:292+0 records out 2023-10-08T00:12:21.755 INFO:teuthology.orchestra.run.smithi099.stderr:306184192 bytes (306 MB, 292 MiB) copied, 1.49231 s, 205 MB/s 2023-10-08T00:12:21.755 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2023-10-08T00:12:21.755 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-08T00:12:21.869 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.868+0000 7f73ecc46700 1 Processor -- start 2023-10-08T00:12:21.869 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.868+0000 7f73ecc46700 1 -- start start 2023-10-08T00:12:21.871 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.869+0000 7f73ecc46700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f73e80a79f0 0x7f73e806eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:21.871 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.869+0000 7f73ecc46700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f73e80b15e0 0x7f73e806eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:21.872 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.869+0000 7f73ecc46700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73e80b26d0 con 0x7f73e80a7dc0 2023-10-08T00:12:21.872 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.869+0000 7f73ecc46700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73e80aeab0 con 0x7f73e80a79f0 2023-10-08T00:12:21.872 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.869+0000 7f73ecc46700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73e80b2b60 con 0x7f73e80b15e0 2023-10-08T00:12:21.872 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.869+0000 7f73e659c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f73e80a79f0 0x7f73e806eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:21.872 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.869+0000 7f73e6d9d700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f73e80a7dc0 0x7f73e815dae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45176/0 (socket says 172.21.15.99:45176) 2023-10-08T00:12:21.872 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.869+0000 7f73e659c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f73e80a79f0 0x7f73e806eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44688/0 (socket says 172.21.15.99:44688) 2023-10-08T00:12:21.872 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.869+0000 7f73e6d9d700 1 -- 172.21.15.99:0/1860394460 learned_addr learned my addr 172.21.15.99:0/1860394460 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:12:21.872 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.869+0000 7f73ddd9b700 1 --2- 172.21.15.99:0/1860394460 >> v2:172.21.15.191:3300/0 conn(0x7f73e80b15e0 0x7f73e806eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:21.872 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.870+0000 7f73e659c700 1 -- 172.21.15.99:0/1860394460 >> v2:172.21.15.191:3300/0 conn(0x7f73e80b15e0 msgr2=0x7f73e806eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:21.872 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.870+0000 7f73e659c700 1 --2- 172.21.15.99:0/1860394460 >> v2:172.21.15.191:3300/0 conn(0x7f73e80b15e0 0x7f73e806eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:21.873 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.870+0000 7f73e659c700 1 -- 172.21.15.99:0/1860394460 >> v1:172.21.15.99:6789/0 conn(0x7f73e80a7dc0 legacy=0x7f73e815dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:21.873 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.870+0000 7f73e659c700 1 -- 172.21.15.99:0/1860394460 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e80b1ac0 con 0x7f73e80a79f0 2023-10-08T00:12:21.873 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.870+0000 7f73e5d9b700 1 -- 172.21.15.99:0/1860394460 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1293275545 0 0) 0x7f73e80b26d0 con 0x7f73e80a7dc0 2023-10-08T00:12:21.873 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.870+0000 7f73e659c700 1 --2- 172.21.15.99:0/1860394460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73e80a79f0 0x7f73e806eab0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f73e8064320 tx=0x7f73d80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7822af3879ec47e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:21.873 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.870+0000 7f73e5d9b700 1 -- 172.21.15.99:0/1860394460 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d800b6b0 con 0x7f73e80a79f0 2023-10-08T00:12:21.873 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.870+0000 7f73e5d9b700 1 -- 172.21.15.99:0/1860394460 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73d800b810 con 0x7f73e80a79f0 2023-10-08T00:12:21.873 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.870+0000 7f73e5d9b700 1 -- 172.21.15.99:0/1860394460 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d800bbb0 con 0x7f73e80a79f0 2023-10-08T00:12:21.873 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.870+0000 7f73ecc46700 1 -- 172.21.15.99:0/1860394460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73e80a79f0 msgr2=0x7f73e806eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:21.873 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.870+0000 7f73ecc46700 1 --2- 172.21.15.99:0/1860394460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73e80a79f0 0x7f73e806eab0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f73e8064320 tx=0x7f73d80091a0 comp rx=0 tx=0).stop 2023-10-08T00:12:21.873 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.870+0000 7f73ecc46700 1 -- 172.21.15.99:0/1860394460 shutdown_connections 2023-10-08T00:12:21.873 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.870+0000 7f73ecc46700 1 --2- 172.21.15.99:0/1860394460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73e80a79f0 0x7f73e806eab0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:21.873 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.870+0000 7f73ecc46700 1 --2- 172.21.15.99:0/1860394460 >> v2:172.21.15.191:3300/0 conn(0x7f73e80b15e0 0x7f73e806eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:21.874 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.870+0000 7f73ecc46700 1 -- 172.21.15.99:0/1860394460 >> 172.21.15.99:0/1860394460 conn(0x7f73e80cd1e0 msgr2=0x7f73e80cd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:21.874 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.870+0000 7f73ecc46700 1 -- 172.21.15.99:0/1860394460 shutdown_connections 2023-10-08T00:12:21.874 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.870+0000 7f73ecc46700 1 -- 172.21.15.99:0/1860394460 wait complete. 2023-10-08T00:12:21.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.872+0000 7f73ecc46700 1 Processor -- start 2023-10-08T00:12:21.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.872+0000 7f73ecc46700 1 -- start start 2023-10-08T00:12:21.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.872+0000 7f73ecc46700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73e80a79f0 0x7f73e81571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:21.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.872+0000 7f73ecc46700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f73e80a7dc0 0x7f73e80cb570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:21.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.872+0000 7f73ecc46700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f73e80b15e0 0x7f73e80cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:21.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.872+0000 7f73ecc46700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e80cc130 con 0x7f73e80a79f0 2023-10-08T00:12:21.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.872+0000 7f73ecc46700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e8099ed0 con 0x7f73e80a7dc0 2023-10-08T00:12:21.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.872+0000 7f73ecc46700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73e80995a0 con 0x7f73e80b15e0 2023-10-08T00:12:21.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.872+0000 7f73e659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73e80a79f0 0x7f73e81571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:21.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.872+0000 7f73e659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73e80a79f0 0x7f73e81571a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44700/0 (socket says 172.21.15.99:44700) 2023-10-08T00:12:21.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.872+0000 7f73e659c700 1 -- 172.21.15.99:0/206133668 learned_addr learned my addr 172.21.15.99:0/206133668 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:21.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.873+0000 7f73e659c700 1 -- 172.21.15.99:0/206133668 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f73e80b15e0 msgr2=0x7f73e80cbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:21.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.873+0000 7f73e659c700 1 --2- 172.21.15.99:0/206133668 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f73e80b15e0 0x7f73e80cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:21.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.873+0000 7f73e659c700 1 -- 172.21.15.99:0/206133668 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f73e80a7dc0 msgr2=0x7f73e80cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:21.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.873+0000 7f73e659c700 1 --2- 172.21.15.99:0/206133668 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f73e80a7dc0 0x7f73e80cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:21.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.873+0000 7f73e659c700 1 -- 172.21.15.99:0/206133668 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d8009420 con 0x7f73e80a79f0 2023-10-08T00:12:21.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.873+0000 7f73e659c700 1 --2- 172.21.15.99:0/206133668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73e80a79f0 0x7f73e81571a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f73d800b0e0 tx=0x7f73d8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:21.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.873+0000 7f73deffd700 1 -- 172.21.15.99:0/206133668 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d8011420 con 0x7f73e80a79f0 2023-10-08T00:12:21.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.873+0000 7f73deffd700 1 -- 172.21.15.99:0/206133668 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73d8011580 con 0x7f73e80a79f0 2023-10-08T00:12:21.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.873+0000 7f73ecc46700 1 -- 172.21.15.99:0/206133668 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73e8099880 con 0x7f73e80a79f0 2023-10-08T00:12:21.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.873+0000 7f73deffd700 1 -- 172.21.15.99:0/206133668 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d8011940 con 0x7f73e80a79f0 2023-10-08T00:12:21.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.873+0000 7f73ecc46700 1 -- 172.21.15.99:0/206133668 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73e81576e0 con 0x7f73e80a79f0 2023-10-08T00:12:21.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.874+0000 7f73ecc46700 1 -- 172.21.15.99:0/206133668 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73e8157930 con 0x7f73e80a79f0 2023-10-08T00:12:21.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.874+0000 7f73deffd700 1 -- 172.21.15.99:0/206133668 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f73d8011d30 con 0x7f73e80a79f0 2023-10-08T00:12:21.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.875+0000 7f73deffd700 1 --2- 172.21.15.99:0/206133668 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f73c8066f60 0x7f73c8069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:21.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.875+0000 7f73deffd700 1 -- 172.21.15.99:0/206133668 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f73d8085eb0 con 0x7f73e80a79f0 2023-10-08T00:12:21.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.875+0000 7f73ddd9b700 1 --2- 172.21.15.99:0/206133668 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f73c8066f60 0x7f73c8069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:21.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.875+0000 7f73ddd9b700 1 --2- 172.21.15.99:0/206133668 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f73c8066f60 0x7f73c8069410 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f73e80b0c90 tx=0x7f73d0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:21.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:21.879+0000 7f73deffd700 1 -- 172.21.15.99:0/206133668 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f73d8053120 con 0x7f73e80a79f0 2023-10-08T00:12:22.064 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:22.063+0000 7f73ecc46700 1 -- 172.21.15.99:0/206133668 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f73e804c3c0 con 0x7f73e80a79f0 2023-10-08T00:12:22.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:22.064+0000 7f73deffd700 1 -- 172.21.15.99:0/206133668 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v45) v1 ==== 81+0+32260 (secure 0 0 0) 0x7f73d8056740 con 0x7f73e80a79f0 2023-10-08T00:12:22.065 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:12:22.066 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:12:22.066 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 45, 2023-10-08T00:12:22.066 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", 2023-10-08T00:12:22.066 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2023-10-08T00:10:36.303601+0000", 2023-10-08T00:12:22.066 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2023-10-08T00:12:11.417504+0000", 2023-10-08T00:12:22.066 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2023-10-08T00:10:41.709171+0000", 2023-10-08T00:12:22.066 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2023-10-08T00:10:38.690404+0000", 2023-10-08T00:12:22.066 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-08T00:12:22.067 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2023-10-08T00:12:22.067 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2023-10-08T00:12:22.067 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2023-10-08T00:12:22.067 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2023-10-08T00:12:22.067 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2023-10-08T00:12:22.067 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2023-10-08T00:12:22.067 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:12:22.067 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2023-10-08T00:12:22.068 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2023-10-08T00:12:22.068 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-08T00:12:22.068 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-08T00:12:22.068 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2023-10-08T00:12:22.068 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 7, 2023-10-08T00:12:22.068 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2023-10-08T00:12:22.068 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2023-10-08T00:12:22.068 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "jewel", 2023-10-08T00:12:22.068 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "quincy", 2023-10-08T00:12:22.069 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2023-10-08T00:12:22.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.069 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2023-10-08T00:12:22.069 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2023-10-08T00:12:22.069 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:10:42.063344+0000", 2023-10-08T00:12:22.069 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:12:22.069 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:12:22.069 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:12:22.069 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:12:22.069 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:12:22.070 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:12:22.070 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:12:22.070 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:12:22.070 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:12:22.070 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:12:22.070 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:12:22.070 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:12:22.070 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2023-10-08T00:12:22.070 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2023-10-08T00:12:22.071 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2023-10-08T00:12:22.071 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2023-10-08T00:12:22.071 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2023-10-08T00:12:22.071 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:12:22.071 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:12:22.071 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:12:22.071 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:12:22.071 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:12:22.072 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:12:22.072 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:12:22.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.072 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "16", 2023-10-08T00:12:22.072 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:12:22.072 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:12:22.072 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:12:22.072 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:12:22.073 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:12:22.073 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:12:22.073 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:12:22.073 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:12:22.073 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:12:22.073 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:12:22.073 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:12:22.073 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:12:22.073 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:12:22.074 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:12:22.074 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:12:22.074 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:12:22.074 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:12:22.074 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:12:22.074 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:12:22.074 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:12:22.074 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:12:22.074 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:12:22.074 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:12:22.075 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:12:22.075 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:12:22.075 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:12:22.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.075 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:12:22.075 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:12:22.075 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:12:22.075 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:12:22.076 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:12:22.076 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:12:22.076 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:12:22.076 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:12:22.076 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:12:22.076 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:12:22.076 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:12:22.076 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:12:22.076 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2023-10-08T00:12:22.076 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2023-10-08T00:12:22.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.077 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:12:22.077 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2023-10-08T00:12:22.077 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:22.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.077 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 6, 2023-10-08T00:12:22.077 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2023-10-08T00:12:22.078 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:12:07.733432+0000", 2023-10-08T00:12:22.078 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:12:22.078 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:12:22.078 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:12:22.078 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:12:22.079 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:12:22.079 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:12:22.079 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:12:22.079 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:12:22.079 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:12:22.079 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:12:22.079 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:12:22.079 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:12:22.079 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2023-10-08T00:12:22.080 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2023-10-08T00:12:22.080 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2023-10-08T00:12:22.080 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2023-10-08T00:12:22.080 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2023-10-08T00:12:22.080 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:12:22.080 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:12:22.080 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:12:22.080 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:12:22.080 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:12:22.080 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:12:22.081 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:12:22.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.081 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "44", 2023-10-08T00:12:22.081 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:12:22.081 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:12:22.081 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:12:22.081 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:12:22.081 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:12:22.081 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:12:22.081 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:12:22.082 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:12:22.082 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:12:22.082 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:12:22.082 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:12:22.082 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:12:22.082 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:12:22.082 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:12:22.082 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:12:22.082 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:12:22.083 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:12:22.083 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:12:22.083 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:12:22.083 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:12:22.083 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:12:22.083 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:12:22.083 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:12:22.083 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:12:22.083 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:12:22.083 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:12:22.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.084 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:12:22.084 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:12:22.084 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:12:22.084 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:12:22.084 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:12:22.084 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:12:22.084 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:12:22.084 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:12:22.085 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:12:22.085 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:12:22.085 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:12:22.085 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:12:22.085 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2023-10-08T00:12:22.085 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2023-10-08T00:12:22.085 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2023-10-08T00:12:22.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.085 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:12:22.085 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:12:22.086 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2023-10-08T00:12:22.086 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:22.086 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:22.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.086 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 7, 2023-10-08T00:12:22.086 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2023-10-08T00:12:22.086 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:12:08.762879+0000", 2023-10-08T00:12:22.086 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:12:22.087 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:12:22.087 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:12:22.087 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:12:22.087 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:12:22.087 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:12:22.087 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:12:22.087 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:12:22.087 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:12:22.087 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:12:22.087 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:12:22.087 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:12:22.087 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2023-10-08T00:12:22.088 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2023-10-08T00:12:22.088 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2023-10-08T00:12:22.088 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2023-10-08T00:12:22.088 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2023-10-08T00:12:22.088 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:12:22.088 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:12:22.089 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:12:22.089 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:12:22.089 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:12:22.089 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:12:22.090 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:12:22.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.090 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "45", 2023-10-08T00:12:22.090 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:12:22.090 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:12:22.090 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:12:22.090 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:12:22.090 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:12:22.090 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:12:22.091 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:12:22.091 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:12:22.091 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:12:22.091 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:12:22.091 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:12:22.091 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:12:22.091 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:12:22.091 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:12:22.091 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:12:22.091 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:12:22.092 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:12:22.092 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:12:22.092 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:12:22.092 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:12:22.092 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:12:22.092 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:12:22.092 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:12:22.092 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:12:22.092 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:12:22.093 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:12:22.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.093 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:12:22.093 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:12:22.093 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:12:22.093 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:12:22.093 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:12:22.093 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:12:22.093 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:12:22.094 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:12:22.094 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:12:22.094 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:12:22.094 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:12:22.094 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:12:22.094 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2023-10-08T00:12:22.094 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2023-10-08T00:12:22.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.095 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:12:22.095 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:12:22.095 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2023-10-08T00:12:22.095 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:22.095 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:22.095 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:22.096 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:12:22.096 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2023-10-08T00:12:22.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.096 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:12:22.096 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:22.096 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:22.096 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:22.096 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:22.096 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:22.097 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:22.097 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:22.097 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:22.097 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:22.097 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:22.097 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:22.097 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:22.098 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:22.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.098 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:22.098 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2023-10-08T00:12:22.098 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:22.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.098 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:22.099 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2023-10-08T00:12:22.099 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:22.099 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:22.099 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:22.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.099 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:22.099 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:22.099 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.099 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:22.099 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2023-10-08T00:12:22.100 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:22.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.100 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:22.100 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2023-10-08T00:12:22.100 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:22.100 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:22.100 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:22.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.101 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:22.101 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:22.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.101 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:22.101 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2023-10-08T00:12:22.101 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:22.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.101 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:22.101 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2023-10-08T00:12:22.102 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:22.102 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:22.102 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:22.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.102 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:22.102 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:22.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.102 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:22.102 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2023-10-08T00:12:22.103 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:22.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.103 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:22.103 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2023-10-08T00:12:22.103 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:22.103 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:22.103 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:22.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.104 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/86365", 2023-10-08T00:12:22.104 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/86365", 2023-10-08T00:12:22.104 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/86365", 2023-10-08T00:12:22.104 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/86365", 2023-10-08T00:12:22.104 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:22.104 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:22.104 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:22.105 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:22.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.105 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.105 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:12:22.105 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:22.105 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:22.105 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:22.105 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:22.105 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:22.106 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:22.106 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:22.106 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:22.106 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:22.106 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:22.106 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:22.106 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:22.106 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:22.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.106 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:22.107 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2023-10-08T00:12:22.107 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:22.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.107 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:22.107 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2023-10-08T00:12:22.107 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:22.107 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:22.107 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:22.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.108 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:22.108 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:22.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.108 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:22.108 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2023-10-08T00:12:22.108 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:22.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.108 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:22.109 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2023-10-08T00:12:22.109 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:22.109 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:22.109 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:22.109 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.109 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:22.109 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:22.109 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.109 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:22.109 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2023-10-08T00:12:22.110 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:22.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.110 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:22.110 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2023-10-08T00:12:22.110 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:22.110 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:22.110 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:22.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.110 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:22.111 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:22.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.111 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:22.111 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2023-10-08T00:12:22.111 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:22.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.111 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:22.111 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2023-10-08T00:12:22.111 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:22.112 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:22.112 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:22.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.112 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/86366", 2023-10-08T00:12:22.112 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/86366", 2023-10-08T00:12:22.112 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/86366", 2023-10-08T00:12:22.113 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/86366", 2023-10-08T00:12:22.113 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:22.113 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:22.114 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:22.116 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:22.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.117 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:12:22.117 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:22.117 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:22.117 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:22.117 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:22.117 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:22.117 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:22.118 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:22.118 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:22.118 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:22.118 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:22.118 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:22.118 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:22.118 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:22.118 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.118 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:22.118 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2023-10-08T00:12:22.118 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:22.118 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.119 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.119 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:22.119 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2023-10-08T00:12:22.119 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:22.119 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:22.119 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:22.119 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.119 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:22.119 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:22.119 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.119 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:22.119 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2023-10-08T00:12:22.119 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:22.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.120 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.120 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:22.120 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2023-10-08T00:12:22.120 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:22.120 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:22.120 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:22.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.120 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:22.120 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:22.120 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.121 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:22.121 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2023-10-08T00:12:22.121 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:22.121 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.121 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.121 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:22.121 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2023-10-08T00:12:22.121 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:22.121 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:22.121 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:22.121 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.121 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:22.121 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:22.122 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.122 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:22.122 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2023-10-08T00:12:22.122 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:22.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.122 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.122 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:22.122 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2023-10-08T00:12:22.122 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:22.122 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:22.122 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:22.123 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.123 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6817/86382", 2023-10-08T00:12:22.123 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/86382", 2023-10-08T00:12:22.123 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/86382", 2023-10-08T00:12:22.123 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/86382", 2023-10-08T00:12:22.123 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:22.123 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:22.123 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:22.123 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:22.123 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.123 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.123 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:12:22.124 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:22.124 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:22.124 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:22.124 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:22.124 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:22.124 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:22.124 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:22.124 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:22.124 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:22.124 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:22.124 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:22.124 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:22.124 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:22.125 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.125 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:22.125 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2023-10-08T00:12:22.125 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:22.125 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.125 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.125 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:22.125 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2023-10-08T00:12:22.125 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:22.125 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:22.125 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:22.125 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.125 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:22.126 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:22.126 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.126 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:22.126 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2023-10-08T00:12:22.126 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:22.126 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.126 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.126 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:22.126 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2023-10-08T00:12:22.126 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:22.126 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:22.126 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:22.127 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.127 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:22.127 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:22.127 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.127 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:22.127 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2023-10-08T00:12:22.127 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:22.127 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.127 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.127 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:22.127 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2023-10-08T00:12:22.127 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:22.128 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:22.128 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:22.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.128 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:22.128 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:22.128 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.128 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:22.128 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2023-10-08T00:12:22.128 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:22.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.128 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.128 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:22.128 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2023-10-08T00:12:22.129 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:22.129 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:22.129 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:22.129 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.129 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6820/86393", 2023-10-08T00:12:22.129 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/86393", 2023-10-08T00:12:22.129 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/86393", 2023-10-08T00:12:22.129 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/86393", 2023-10-08T00:12:22.129 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:22.129 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:22.129 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:22.129 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:22.130 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.130 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.130 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:12:22.130 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:22.130 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:22.130 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:22.130 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:22.130 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:22.130 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:22.130 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:22.130 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:22.130 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:22.131 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:22.131 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:22.131 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:22.131 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:22.131 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.131 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:22.131 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6800", 2023-10-08T00:12:22.131 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:22.131 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.131 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.131 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:22.131 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6801", 2023-10-08T00:12:22.132 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:22.132 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:22.132 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:22.132 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.132 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:22.132 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:22.132 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.132 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:22.132 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6802", 2023-10-08T00:12:22.132 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:22.132 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.132 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.133 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:22.133 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6803", 2023-10-08T00:12:22.133 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:22.133 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:22.133 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:22.133 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.133 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:22.133 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:22.133 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.133 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:22.133 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6806", 2023-10-08T00:12:22.133 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:22.134 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.134 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.134 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:22.134 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6807", 2023-10-08T00:12:22.134 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:22.134 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:22.134 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:22.134 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.134 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:22.134 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:22.134 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.134 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:22.135 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6804", 2023-10-08T00:12:22.135 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:22.135 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.135 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.135 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:22.135 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6805", 2023-10-08T00:12:22.135 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:22.135 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:22.135 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:22.135 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.135 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6801/86570", 2023-10-08T00:12:22.135 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6803/86570", 2023-10-08T00:12:22.136 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6807/86570", 2023-10-08T00:12:22.136 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6805/86570", 2023-10-08T00:12:22.136 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:22.136 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:22.136 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:22.136 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:22.136 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.136 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.136 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:12:22.136 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:22.136 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:22.136 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:22.137 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:22.137 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:22.137 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:22.137 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:22.137 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:22.137 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:22.137 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:22.137 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:22.137 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:22.137 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:22.137 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.137 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:22.138 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6816", 2023-10-08T00:12:22.138 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:22.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.138 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:22.138 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6818", 2023-10-08T00:12:22.138 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:22.138 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:22.138 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:22.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.138 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:22.138 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:22.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.139 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:22.139 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6820", 2023-10-08T00:12:22.139 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:22.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.139 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:22.139 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6821", 2023-10-08T00:12:22.139 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:22.139 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:22.139 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:22.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.140 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:22.140 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:22.140 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.140 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:22.140 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6826", 2023-10-08T00:12:22.140 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:22.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.140 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.140 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:22.140 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6828", 2023-10-08T00:12:22.140 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:22.140 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:22.141 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:22.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.141 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:22.141 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:22.141 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.141 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:22.141 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6823", 2023-10-08T00:12:22.141 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:22.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.141 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.141 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:22.141 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6825", 2023-10-08T00:12:22.142 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:22.142 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:22.142 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:22.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.142 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6818/86608", 2023-10-08T00:12:22.142 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6821/86608", 2023-10-08T00:12:22.142 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6828/86608", 2023-10-08T00:12:22.142 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6825/86608", 2023-10-08T00:12:22.142 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:22.142 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:22.142 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:22.142 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:22.143 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.143 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.143 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:12:22.143 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:22.143 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:22.143 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:22.143 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:22.143 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:22.143 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:22.143 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:22.143 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:22.143 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:22.143 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:22.144 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:22.144 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:22.144 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:22.144 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.144 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:22.144 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6808", 2023-10-08T00:12:22.144 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:22.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.144 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.144 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:22.144 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6809", 2023-10-08T00:12:22.144 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:22.145 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:22.145 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:22.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.145 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:22.145 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:22.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.145 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:22.145 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6810", 2023-10-08T00:12:22.145 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:22.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.145 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:22.146 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6811", 2023-10-08T00:12:22.146 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:22.146 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:22.146 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:22.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.146 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:22.146 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:22.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.146 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:22.146 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6814", 2023-10-08T00:12:22.146 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:22.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.147 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.147 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:22.147 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6815", 2023-10-08T00:12:22.147 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:22.147 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:22.147 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:22.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.147 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:22.147 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:22.147 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.147 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:22.148 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6812", 2023-10-08T00:12:22.148 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:22.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.148 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.148 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:22.148 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6813", 2023-10-08T00:12:22.148 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:22.148 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:22.148 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:22.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.148 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6809/86571", 2023-10-08T00:12:22.148 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6811/86571", 2023-10-08T00:12:22.149 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6815/86571", 2023-10-08T00:12:22.149 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6813/86571", 2023-10-08T00:12:22.149 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:22.149 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:22.149 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:22.149 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:22.149 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.149 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.149 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:12:22.149 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:22.149 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:22.149 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:22.150 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:22.150 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:22.150 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:22.150 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:22.150 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:22.150 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:22.150 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:22.150 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:22.150 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:22.150 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:22.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.150 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:22.151 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6817", 2023-10-08T00:12:22.151 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:22.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.151 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:22.151 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6819", 2023-10-08T00:12:22.151 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:22.151 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:22.151 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:22.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.151 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:22.151 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:22.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.152 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:22.152 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6822", 2023-10-08T00:12:22.152 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:22.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.152 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:22.152 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6824", 2023-10-08T00:12:22.152 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:22.152 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:22.152 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:22.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.153 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:22.153 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:22.153 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.153 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:22.153 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6830", 2023-10-08T00:12:22.153 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:22.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.153 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.153 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:22.153 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6831", 2023-10-08T00:12:22.153 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:22.153 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:22.154 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:22.154 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.154 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:22.154 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:22.154 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.154 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:22.154 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6827", 2023-10-08T00:12:22.154 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:22.154 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.154 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.154 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:22.154 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6829", 2023-10-08T00:12:22.155 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:22.155 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:22.155 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:22.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.155 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6819/86612", 2023-10-08T00:12:22.155 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6824/86612", 2023-10-08T00:12:22.155 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6831/86612", 2023-10-08T00:12:22.155 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6829/86612", 2023-10-08T00:12:22.155 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:22.155 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:22.155 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:22.156 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:22.156 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:22.156 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:12:22.156 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2023-10-08T00:12:22.156 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.156 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:12:22.156 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:22.156 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:22.156 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:22.156 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:22.156 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:22.157 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.905615+0000", 2023-10-08T00:12:22.157 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:22.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.157 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.157 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:12:22.157 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:22.157 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:22.157 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:22.157 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:22.157 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:22.157 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.880709+0000", 2023-10-08T00:12:22.157 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:22.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.158 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:12:22.158 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:22.158 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:22.158 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:22.158 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:22.158 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:22.158 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.874704+0000", 2023-10-08T00:12:22.158 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:22.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.159 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:12:22.159 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:22.159 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:22.159 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:22.159 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:22.159 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:22.159 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.899604+0000", 2023-10-08T00:12:22.159 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:22.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.159 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.159 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:12:22.159 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:22.160 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:22.160 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:22.160 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:22.160 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:22.160 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.844371+0000", 2023-10-08T00:12:22.160 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:22.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.160 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:12:22.160 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:22.160 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:22.160 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:22.161 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:22.161 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:22.161 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.794810+0000", 2023-10-08T00:12:22.161 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:22.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.161 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:12:22.161 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:22.161 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:22.161 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:22.161 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:22.161 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:22.162 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.859474+0000", 2023-10-08T00:12:22.162 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:22.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:22.162 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:12:22.162 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:22.162 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:22.162 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:22.162 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:22.162 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:22.162 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.893130+0000", 2023-10-08T00:12:22.162 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:22.163 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:22.163 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:12:22.163 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2023-10-08T00:12:22.163 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [], 2023-10-08T00:12:22.163 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2023-10-08T00:12:22.163 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2023-10-08T00:12:22.163 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2023-10-08T00:12:22.163 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2023-10-08T00:12:22.163 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2023-10-08T00:12:22.163 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2023-10-08T00:12:22.163 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2023-10-08T00:12:22.163 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2023-10-08T00:12:22.163 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2023-10-08T00:12:22.164 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2023-10-08T00:12:22.164 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2023-10-08T00:12:22.164 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:22.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:22.164 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2023-10-08T00:12:22.164 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2023-10-08T00:12:22.164 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2023-10-08T00:12:22.164 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2023-10-08T00:12:22.164 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2023-10-08T00:12:22.164 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2023-10-08T00:12:22.164 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2023-10-08T00:12:22.164 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2023-10-08T00:12:22.165 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2023-10-08T00:12:22.165 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2023-10-08T00:12:22.165 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2023-10-08T00:12:22.165 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:22.165 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:12:22.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:22.066+0000 7f73ecc46700 1 -- 172.21.15.99:0/206133668 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f73c8066f60 msgr2=0x7f73c8069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:22.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:22.066+0000 7f73ecc46700 1 --2- 172.21.15.99:0/206133668 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f73c8066f60 0x7f73c8069410 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f73e80b0c90 tx=0x7f73d0008040 comp rx=0 tx=0).stop 2023-10-08T00:12:22.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:22.066+0000 7f73ecc46700 1 -- 172.21.15.99:0/206133668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73e80a79f0 msgr2=0x7f73e81571a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:22.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:22.066+0000 7f73ecc46700 1 --2- 172.21.15.99:0/206133668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73e80a79f0 0x7f73e81571a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f73d800b0e0 tx=0x7f73d8017000 comp rx=0 tx=0).stop 2023-10-08T00:12:22.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:22.066+0000 7f73ecc46700 1 -- 172.21.15.99:0/206133668 shutdown_connections 2023-10-08T00:12:22.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:22.066+0000 7f73ecc46700 1 --2- 172.21.15.99:0/206133668 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f73c8066f60 0x7f73c8069410 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:22.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:22.066+0000 7f73ecc46700 1 --2- 172.21.15.99:0/206133668 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f73e80b15e0 0x7f73e80cbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:22.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:22.066+0000 7f73ecc46700 1 --2- 172.21.15.99:0/206133668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73e80a79f0 0x7f73e81571a0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:22.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:22.066+0000 7f73ecc46700 1 --2- 172.21.15.99:0/206133668 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f73e80a7dc0 0x7f73e80cb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:22.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:22.066+0000 7f73ecc46700 1 -- 172.21.15.99:0/206133668 >> 172.21.15.99:0/206133668 conn(0x7f73e80cd1e0 msgr2=0x7f73e806daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:22.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:22.066+0000 7f73ecc46700 1 -- 172.21.15.99:0/206133668 shutdown_connections 2023-10-08T00:12:22.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:22.066+0000 7f73ecc46700 1 -- 172.21.15.99:0/206133668 wait complete. 2023-10-08T00:12:22.168 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-08T00:12:22.749 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:12:22.752 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:12:23.065 INFO:tasks.ceph.mon.a.smithi099.stderr:2023-10-08T00:12:23.065+0000 7f6c637a7700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 full osd(s) (OSD_FULL) 2023-10-08T00:12:27.169 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-08T00:12:27.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.295+0000 7fe58e303700 1 Processor -- start 2023-10-08T00:12:27.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.295+0000 7fe58e303700 1 -- start start 2023-10-08T00:12:27.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.296+0000 7fe58e303700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe58815a620 0x7fe58815d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:27.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.296+0000 7fe58e303700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe58815d580 0x7fe588158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:27.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.296+0000 7fe58e303700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe58815cbb0 con 0x7fe588157870 2023-10-08T00:12:27.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.296+0000 7fe58e303700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe58815bdb0 con 0x7fe58815d580 2023-10-08T00:12:27.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.296+0000 7fe58e303700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe58815bfa0 con 0x7fe58815a620 2023-10-08T00:12:27.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.296+0000 7fe587fff700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fe588157870 0x7fe58815b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:43498/0 (socket says 172.21.15.99:43498) 2023-10-08T00:12:27.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.296+0000 7fe587fff700 1 -- 172.21.15.99:0/2826275414 learned_addr learned my addr 172.21.15.99:0/2826275414 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:12:27.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.296+0000 7fe58c8a0700 1 --2- 172.21.15.99:0/2826275414 >> v2:172.21.15.99:3300/0 conn(0x7fe58815d580 0x7fe588158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:27.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.296+0000 7fe5877fe700 1 --2- 172.21.15.99:0/2826275414 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe58815a620 0x7fe58815d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:27.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.296+0000 7fe586ffd700 1 -- 172.21.15.99:0/2826275414 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2313190840 0 0) 0x7fe58815cbb0 con 0x7fe588157870 2023-10-08T00:12:27.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.296+0000 7fe586ffd700 1 -- 172.21.15.99:0/2826275414 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe570003570 con 0x7fe588157870 2023-10-08T00:12:27.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.296+0000 7fe58c8a0700 1 -- 172.21.15.99:0/2826275414 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe58815a620 msgr2=0x7fe58815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:27.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.296+0000 7fe58c8a0700 1 --2- 172.21.15.99:0/2826275414 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe58815a620 0x7fe58815d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:27.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.296+0000 7fe58c8a0700 1 -- 172.21.15.99:0/2826275414 >> v1:172.21.15.99:6789/0 conn(0x7fe588157870 legacy=0x7fe58815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:27.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.296+0000 7fe58c8a0700 1 -- 172.21.15.99:0/2826275414 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe58815c280 con 0x7fe58815d580 2023-10-08T00:12:27.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.296+0000 7fe58c8a0700 1 --2- 172.21.15.99:0/2826275414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe58815d580 0x7fe588158110 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe580009e60 tx=0x7fe580009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7a42ac5f75b6efa server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:27.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.297+0000 7fe586ffd700 1 -- 172.21.15.99:0/2826275414 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58000e070 con 0x7fe58815d580 2023-10-08T00:12:27.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.297+0000 7fe586ffd700 1 -- 172.21.15.99:0/2826275414 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe58000bab0 con 0x7fe58815d580 2023-10-08T00:12:27.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.297+0000 7fe586ffd700 1 -- 172.21.15.99:0/2826275414 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58000be70 con 0x7fe58815d580 2023-10-08T00:12:27.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.297+0000 7fe58e303700 1 -- 172.21.15.99:0/2826275414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe58815d580 msgr2=0x7fe588158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:27.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.297+0000 7fe58e303700 1 --2- 172.21.15.99:0/2826275414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe58815d580 0x7fe588158110 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe580009e60 tx=0x7fe580009780 comp rx=0 tx=0).stop 2023-10-08T00:12:27.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.297+0000 7fe58e303700 1 -- 172.21.15.99:0/2826275414 shutdown_connections 2023-10-08T00:12:27.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.297+0000 7fe58e303700 1 --2- 172.21.15.99:0/2826275414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe58815d580 0x7fe588158110 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:27.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.297+0000 7fe58e303700 1 --2- 172.21.15.99:0/2826275414 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe58815a620 0x7fe58815d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:27.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.297+0000 7fe58e303700 1 -- 172.21.15.99:0/2826275414 >> 172.21.15.99:0/2826275414 conn(0x7fe588068ce0 msgr2=0x7fe5880690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:27.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.297+0000 7fe58e303700 1 -- 172.21.15.99:0/2826275414 shutdown_connections 2023-10-08T00:12:27.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.297+0000 7fe58e303700 1 -- 172.21.15.99:0/2826275414 wait complete. 2023-10-08T00:12:27.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.298+0000 7fe58e303700 1 Processor -- start 2023-10-08T00:12:27.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.299+0000 7fe58e303700 1 -- start start 2023-10-08T00:12:27.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.299+0000 7fe58e303700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe588157870 0x7fe58814aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:27.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.299+0000 7fe58e303700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe58815a620 0x7fe588158160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:27.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.299+0000 7fe58e303700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe58815d580 0x7fe5881586a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:27.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.299+0000 7fe58e303700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58814b020 con 0x7fe58815d580 2023-10-08T00:12:27.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.299+0000 7fe58e303700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5880b31c0 con 0x7fe58815a620 2023-10-08T00:12:27.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.299+0000 7fe58e303700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5881599a0 con 0x7fe588157870 2023-10-08T00:12:27.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.299+0000 7fe58c8a0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe58815d580 0x7fe5881586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:27.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.299+0000 7fe58c8a0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe58815d580 0x7fe5881586a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51848/0 (socket says 172.21.15.99:51848) 2023-10-08T00:12:27.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.299+0000 7fe58c8a0700 1 -- 172.21.15.99:0/491551313 learned_addr learned my addr 172.21.15.99:0/491551313 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:27.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.300+0000 7fe587fff700 1 --2- 172.21.15.99:0/491551313 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe588157870 0x7fe58814aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:27.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.300+0000 7fe5877fe700 1 --2- 172.21.15.99:0/491551313 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe58815a620 0x7fe588158160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:27.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.300+0000 7fe58c8a0700 1 -- 172.21.15.99:0/491551313 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe588157870 msgr2=0x7fe58814aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:27.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.300+0000 7fe58c8a0700 1 --2- 172.21.15.99:0/491551313 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe588157870 0x7fe58814aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:27.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.300+0000 7fe58c8a0700 1 -- 172.21.15.99:0/491551313 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe58815a620 msgr2=0x7fe588158160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:27.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.300+0000 7fe58c8a0700 1 --2- 172.21.15.99:0/491551313 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe58815a620 0x7fe588158160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:27.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.300+0000 7fe58c8a0700 1 -- 172.21.15.99:0/491551313 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe580009a20 con 0x7fe58815d580 2023-10-08T00:12:27.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.300+0000 7fe5877fe700 1 --2- 172.21.15.99:0/491551313 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe58815a620 0x7fe588158160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:12:27.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.300+0000 7fe58c8a0700 1 --2- 172.21.15.99:0/491551313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe58815d580 0x7fe5881586a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe58806b530 tx=0x7fe580019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:27.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.300+0000 7fe56ffff700 1 -- 172.21.15.99:0/491551313 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe580027470 con 0x7fe58815d580 2023-10-08T00:12:27.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.300+0000 7fe56ffff700 1 -- 172.21.15.99:0/491551313 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5800275f0 con 0x7fe58815d580 2023-10-08T00:12:27.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.300+0000 7fe58e303700 1 -- 172.21.15.99:0/491551313 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe588158d90 con 0x7fe58815d580 2023-10-08T00:12:27.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.300+0000 7fe56ffff700 1 -- 172.21.15.99:0/491551313 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5800279b0 con 0x7fe58815d580 2023-10-08T00:12:27.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.300+0000 7fe58e303700 1 -- 172.21.15.99:0/491551313 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe588159350 con 0x7fe58815d580 2023-10-08T00:12:27.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.301+0000 7fe58e303700 1 -- 172.21.15.99:0/491551313 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5880b5010 con 0x7fe58815d580 2023-10-08T00:12:27.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.301+0000 7fe56ffff700 1 -- 172.21.15.99:0/491551313 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe580024070 con 0x7fe58815d580 2023-10-08T00:12:27.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.302+0000 7fe56ffff700 1 --2- 172.21.15.99:0/491551313 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe574066f30 0x7fe5740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:27.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.302+0000 7fe56ffff700 1 -- 172.21.15.99:0/491551313 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe580086ee0 con 0x7fe58815d580 2023-10-08T00:12:27.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.302+0000 7fe56ffff700 1 -- 172.21.15.99:0/491551313 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=50}) v3 -- 0x7fe57406ea50 con 0x7fe58815d580 2023-10-08T00:12:27.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.302+0000 7fe587fff700 1 --2- 172.21.15.99:0/491551313 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe574066f30 0x7fe5740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:27.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.302+0000 7fe587fff700 1 --2- 172.21.15.99:0/491551313 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe574066f30 0x7fe5740693e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe5780058d0 tx=0x7fe578009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:27.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.306+0000 7fe56ffff700 1 -- 172.21.15.99:0/491551313 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe58002c060 con 0x7fe58815d580 2023-10-08T00:12:27.490 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.489+0000 7fe58e303700 1 -- 172.21.15.99:0/491551313 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fe58804c3c0 con 0x7fe58815d580 2023-10-08T00:12:27.491 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.490+0000 7fe56ffff700 1 -- 172.21.15.99:0/491551313 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v49) v1 ==== 81+0+32475 (secure 0 0 0) 0x7fe580054150 con 0x7fe58815d580 2023-10-08T00:12:27.491 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:12:27.491 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:12:27.492 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 49, 2023-10-08T00:12:27.492 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", 2023-10-08T00:12:27.492 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2023-10-08T00:10:36.303601+0000", 2023-10-08T00:12:27.492 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2023-10-08T00:12:26.076074+0000", 2023-10-08T00:12:27.492 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2023-10-08T00:10:41.709171+0000", 2023-10-08T00:12:27.492 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2023-10-08T00:10:38.690404+0000", 2023-10-08T00:12:27.492 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-08T00:12:27.492 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2023-10-08T00:12:27.492 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2023-10-08T00:12:27.492 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2023-10-08T00:12:27.492 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2023-10-08T00:12:27.493 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2023-10-08T00:12:27.493 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2023-10-08T00:12:27.493 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:12:27.493 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2023-10-08T00:12:27.493 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2023-10-08T00:12:27.493 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-08T00:12:27.493 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-08T00:12:27.493 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2023-10-08T00:12:27.493 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 7, 2023-10-08T00:12:27.493 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2023-10-08T00:12:27.493 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2023-10-08T00:12:27.493 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "jewel", 2023-10-08T00:12:27.494 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "quincy", 2023-10-08T00:12:27.494 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2023-10-08T00:12:27.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.494 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2023-10-08T00:12:27.494 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2023-10-08T00:12:27.494 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:10:42.063344+0000", 2023-10-08T00:12:27.494 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 3, 2023-10-08T00:12:27.494 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full", 2023-10-08T00:12:27.494 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:12:27.494 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:12:27.494 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:12:27.495 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:12:27.495 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:12:27.495 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:12:27.495 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:12:27.495 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:12:27.495 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:12:27.495 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:12:27.495 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2023-10-08T00:12:27.495 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2023-10-08T00:12:27.495 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2023-10-08T00:12:27.495 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2023-10-08T00:12:27.495 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2023-10-08T00:12:27.496 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:12:27.496 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:12:27.496 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:12:27.496 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:12:27.496 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:12:27.496 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:12:27.496 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:12:27.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.496 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "46", 2023-10-08T00:12:27.496 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:12:27.496 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:12:27.497 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:12:27.497 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:12:27.497 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:12:27.497 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:12:27.497 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:12:27.497 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:12:27.497 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:12:27.497 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:12:27.497 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:12:27.497 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:12:27.497 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:12:27.497 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:12:27.498 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:12:27.498 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:12:27.498 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:12:27.498 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:12:27.498 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:12:27.498 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:12:27.498 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:12:27.498 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:12:27.498 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:12:27.498 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:12:27.498 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:12:27.498 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:12:27.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.499 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:12:27.499 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:12:27.499 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:12:27.499 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:12:27.499 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:12:27.499 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:12:27.499 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:12:27.499 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:12:27.499 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:12:27.499 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:12:27.499 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:12:27.500 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:12:27.500 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2023-10-08T00:12:27.500 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2023-10-08T00:12:27.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.500 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:12:27.500 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2023-10-08T00:12:27.500 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:27.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.500 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 6, 2023-10-08T00:12:27.500 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2023-10-08T00:12:27.501 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:12:07.733432+0000", 2023-10-08T00:12:27.501 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 3, 2023-10-08T00:12:27.501 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full", 2023-10-08T00:12:27.501 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:12:27.501 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:12:27.501 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:12:27.501 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:12:27.501 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:12:27.501 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:12:27.501 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:12:27.501 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:12:27.502 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:12:27.502 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:12:27.502 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2023-10-08T00:12:27.502 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2023-10-08T00:12:27.502 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2023-10-08T00:12:27.502 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2023-10-08T00:12:27.502 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2023-10-08T00:12:27.502 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:12:27.502 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:12:27.502 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:12:27.503 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:12:27.503 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:12:27.503 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:12:27.503 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:12:27.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.503 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "46", 2023-10-08T00:12:27.503 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:12:27.503 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:12:27.503 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:12:27.503 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:12:27.503 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:12:27.504 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:12:27.504 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:12:27.504 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:12:27.504 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:12:27.504 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:12:27.504 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:12:27.504 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:12:27.504 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:12:27.505 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:12:27.505 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:12:27.505 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:12:27.505 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:12:27.505 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:12:27.505 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:12:27.505 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:12:27.505 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:12:27.505 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:12:27.505 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:12:27.506 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:12:27.506 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:12:27.506 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:12:27.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.506 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:12:27.506 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:12:27.506 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:12:27.506 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:12:27.506 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:12:27.506 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:12:27.507 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:12:27.507 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:12:27.507 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:12:27.507 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:12:27.507 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:12:27.507 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:12:27.507 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2023-10-08T00:12:27.507 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2023-10-08T00:12:27.507 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2023-10-08T00:12:27.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.508 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:12:27.508 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:12:27.508 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2023-10-08T00:12:27.508 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:27.508 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:27.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.508 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 7, 2023-10-08T00:12:27.508 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2023-10-08T00:12:27.508 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:12:08.762879+0000", 2023-10-08T00:12:27.508 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 3, 2023-10-08T00:12:27.509 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full", 2023-10-08T00:12:27.509 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:12:27.509 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:12:27.509 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:12:27.509 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:12:27.509 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:12:27.509 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:12:27.509 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:12:27.509 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:12:27.509 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:12:27.509 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:12:27.510 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2023-10-08T00:12:27.510 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2023-10-08T00:12:27.510 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2023-10-08T00:12:27.510 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2023-10-08T00:12:27.510 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2023-10-08T00:12:27.510 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:12:27.510 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:12:27.510 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:12:27.510 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:12:27.510 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:12:27.510 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:12:27.510 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:12:27.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.511 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "46", 2023-10-08T00:12:27.511 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:12:27.511 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:12:27.511 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:12:27.511 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:12:27.511 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:12:27.511 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:12:27.511 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:12:27.511 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:12:27.511 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:12:27.511 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:12:27.511 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:12:27.512 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:12:27.512 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:12:27.512 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:12:27.512 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:12:27.512 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:12:27.512 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:12:27.512 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:12:27.512 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:12:27.512 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:12:27.512 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:12:27.512 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:12:27.513 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:12:27.513 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:12:27.513 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:12:27.513 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:12:27.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.513 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:12:27.513 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:12:27.513 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:12:27.513 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:12:27.513 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:12:27.513 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:12:27.513 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:12:27.514 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:12:27.514 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:12:27.514 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:12:27.514 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:12:27.514 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:12:27.514 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2023-10-08T00:12:27.514 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2023-10-08T00:12:27.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.514 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:12:27.514 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:12:27.514 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2023-10-08T00:12:27.514 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:27.515 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:27.515 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:27.515 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:12:27.515 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2023-10-08T00:12:27.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.515 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:12:27.515 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:27.515 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:27.515 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:27.515 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:27.515 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:27.515 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:27.516 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:27.516 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:27.516 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:27.516 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:27.516 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:27.516 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:27.516 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:27.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.516 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:27.516 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2023-10-08T00:12:27.516 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:27.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.517 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:27.517 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2023-10-08T00:12:27.517 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:27.517 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:27.517 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:27.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.517 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:27.517 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:27.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.517 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:27.518 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2023-10-08T00:12:27.518 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:27.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.518 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:27.518 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2023-10-08T00:12:27.518 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:27.518 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:27.518 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:27.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.518 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:27.518 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:27.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.519 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:27.519 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2023-10-08T00:12:27.519 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:27.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.519 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:27.519 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2023-10-08T00:12:27.519 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:27.519 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:27.519 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:27.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.520 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:27.520 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:27.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.520 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:27.520 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2023-10-08T00:12:27.520 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:27.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.520 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:27.520 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2023-10-08T00:12:27.520 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:27.520 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:27.521 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:27.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.521 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/86365", 2023-10-08T00:12:27.521 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/86365", 2023-10-08T00:12:27.521 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/86365", 2023-10-08T00:12:27.521 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/86365", 2023-10-08T00:12:27.521 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:27.521 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull", 2023-10-08T00:12:27.521 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:27.521 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:27.521 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:27.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.522 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:12:27.522 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:27.522 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:27.522 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:27.522 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:27.522 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:27.522 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:27.522 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:27.522 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:27.522 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:27.523 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:27.523 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:27.523 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:27.523 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:27.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.523 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:27.523 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2023-10-08T00:12:27.523 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:27.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.523 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:27.524 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2023-10-08T00:12:27.524 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:27.524 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:27.524 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:27.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.524 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:27.524 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:27.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.524 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:27.524 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2023-10-08T00:12:27.524 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:27.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.525 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:27.525 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2023-10-08T00:12:27.525 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:27.525 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:27.525 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:27.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.525 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:27.525 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:27.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.526 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:27.526 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2023-10-08T00:12:27.526 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:27.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.526 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:27.526 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2023-10-08T00:12:27.526 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:27.526 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:27.527 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:27.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.527 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:27.527 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:27.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.527 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:27.527 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2023-10-08T00:12:27.527 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:27.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.528 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:27.528 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2023-10-08T00:12:27.528 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:27.528 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:27.528 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:27.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.528 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/86366", 2023-10-08T00:12:27.528 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/86366", 2023-10-08T00:12:27.528 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/86366", 2023-10-08T00:12:27.528 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/86366", 2023-10-08T00:12:27.528 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:27.529 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull", 2023-10-08T00:12:27.529 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:27.529 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:27.529 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:27.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.529 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:12:27.529 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:27.529 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:27.529 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:27.529 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:27.530 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:27.530 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:27.530 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:27.530 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:27.530 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:27.530 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:27.530 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:27.530 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:27.530 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:27.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.530 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:27.531 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2023-10-08T00:12:27.531 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:27.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.531 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:27.531 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2023-10-08T00:12:27.531 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:27.531 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:27.531 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:27.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.531 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:27.531 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:27.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.532 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:27.532 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2023-10-08T00:12:27.532 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:27.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.532 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:27.532 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2023-10-08T00:12:27.532 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:27.532 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:27.532 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:27.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.533 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:27.533 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:27.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.533 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:27.533 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2023-10-08T00:12:27.533 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:27.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.533 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:27.533 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2023-10-08T00:12:27.533 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:27.533 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:27.534 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:27.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.534 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:27.534 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:27.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.534 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:27.534 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2023-10-08T00:12:27.534 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:27.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.534 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:27.534 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2023-10-08T00:12:27.535 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:27.535 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:27.535 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:27.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.535 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6817/86382", 2023-10-08T00:12:27.535 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/86382", 2023-10-08T00:12:27.535 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/86382", 2023-10-08T00:12:27.535 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/86382", 2023-10-08T00:12:27.535 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:27.535 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:27.535 INFO:teuthology.orchestra.run.smithi099.stdout: "full", 2023-10-08T00:12:27.535 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:27.536 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:27.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.536 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:12:27.536 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:27.536 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:27.536 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:27.536 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:27.536 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:27.536 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:27.536 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:27.537 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:27.537 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:27.537 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:27.537 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:27.537 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:27.537 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:27.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.537 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:27.537 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2023-10-08T00:12:27.537 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:27.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.538 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:27.538 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2023-10-08T00:12:27.538 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:27.538 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:27.538 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:27.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.538 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:27.538 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:27.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.538 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:27.538 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2023-10-08T00:12:27.538 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:27.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.539 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:27.539 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2023-10-08T00:12:27.539 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:27.539 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:27.539 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:27.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.539 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:27.539 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:27.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.539 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:27.540 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2023-10-08T00:12:27.540 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:27.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.540 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:27.540 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2023-10-08T00:12:27.540 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:27.540 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:27.540 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:27.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.540 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:27.540 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:27.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.541 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:27.541 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2023-10-08T00:12:27.541 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:27.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.541 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:27.541 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2023-10-08T00:12:27.541 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:27.541 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:27.541 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:27.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.542 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6820/86393", 2023-10-08T00:12:27.542 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/86393", 2023-10-08T00:12:27.542 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/86393", 2023-10-08T00:12:27.542 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/86393", 2023-10-08T00:12:27.542 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:27.542 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull", 2023-10-08T00:12:27.542 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:27.542 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:27.542 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:27.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.542 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:12:27.543 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:27.543 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:27.543 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:27.543 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:27.543 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:27.543 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:27.543 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:27.543 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:27.543 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:27.543 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:27.543 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:27.543 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:27.544 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:27.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.544 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:27.544 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6800", 2023-10-08T00:12:27.544 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:27.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.544 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:27.544 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6801", 2023-10-08T00:12:27.544 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:27.544 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:27.544 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:27.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.545 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:27.545 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:27.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.545 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:27.545 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6802", 2023-10-08T00:12:27.545 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:27.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.545 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:27.545 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6803", 2023-10-08T00:12:27.546 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:27.546 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:27.546 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:27.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.546 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:27.546 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:27.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.546 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:27.546 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6806", 2023-10-08T00:12:27.546 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:27.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.547 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:27.547 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6807", 2023-10-08T00:12:27.547 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:27.547 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:27.547 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:27.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.547 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:27.547 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:27.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.547 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:27.547 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6804", 2023-10-08T00:12:27.547 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:27.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.548 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:27.548 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6805", 2023-10-08T00:12:27.548 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:27.548 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:27.548 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:27.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.548 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6801/86570", 2023-10-08T00:12:27.548 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6803/86570", 2023-10-08T00:12:27.548 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6807/86570", 2023-10-08T00:12:27.548 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6805/86570", 2023-10-08T00:12:27.549 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:27.549 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:27.549 INFO:teuthology.orchestra.run.smithi099.stdout: "full", 2023-10-08T00:12:27.549 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:27.549 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:27.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.549 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:12:27.549 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:27.549 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:27.549 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:27.549 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:27.550 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:27.550 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:27.550 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:27.550 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:27.550 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:27.550 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:27.550 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:27.550 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:27.550 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:27.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.550 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:27.550 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6816", 2023-10-08T00:12:27.551 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:27.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.551 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:27.551 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6818", 2023-10-08T00:12:27.551 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:27.551 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:27.551 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:27.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.551 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:27.551 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:27.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.552 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:27.552 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6820", 2023-10-08T00:12:27.552 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:27.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.552 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:27.552 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6821", 2023-10-08T00:12:27.552 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:27.552 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:27.552 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:27.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.553 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:27.553 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:27.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.553 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:27.553 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6826", 2023-10-08T00:12:27.553 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:27.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.553 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:27.553 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6828", 2023-10-08T00:12:27.553 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:27.553 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:27.554 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:27.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.554 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:27.554 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:27.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.554 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:27.554 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6823", 2023-10-08T00:12:27.554 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:27.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.554 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:27.554 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6825", 2023-10-08T00:12:27.555 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:27.555 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:27.555 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:27.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.555 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6818/86608", 2023-10-08T00:12:27.555 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6821/86608", 2023-10-08T00:12:27.555 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6828/86608", 2023-10-08T00:12:27.555 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6825/86608", 2023-10-08T00:12:27.555 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:27.555 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:27.555 INFO:teuthology.orchestra.run.smithi099.stdout: "full", 2023-10-08T00:12:27.556 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:27.556 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:27.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.556 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:12:27.556 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:27.556 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:27.556 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:27.556 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:27.557 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:27.557 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:27.557 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:27.557 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:27.557 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:27.557 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:27.557 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:27.557 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:27.557 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:27.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.557 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:27.557 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6808", 2023-10-08T00:12:27.558 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:27.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.558 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:27.558 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6809", 2023-10-08T00:12:27.558 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:27.558 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:27.558 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:27.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.559 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:27.559 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:27.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.559 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:27.559 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6810", 2023-10-08T00:12:27.559 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:27.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.559 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:27.559 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6811", 2023-10-08T00:12:27.559 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:27.559 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:27.560 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:27.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.560 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:27.560 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:27.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.560 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:27.560 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6814", 2023-10-08T00:12:27.560 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:27.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.560 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:27.560 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6815", 2023-10-08T00:12:27.561 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:27.561 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:27.561 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:27.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.561 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:27.561 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:27.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.561 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:27.561 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6812", 2023-10-08T00:12:27.561 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:27.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.562 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:27.562 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6813", 2023-10-08T00:12:27.562 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:27.562 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:27.562 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:27.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.562 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6809/86571", 2023-10-08T00:12:27.562 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6811/86571", 2023-10-08T00:12:27.562 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6815/86571", 2023-10-08T00:12:27.562 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6813/86571", 2023-10-08T00:12:27.563 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:27.563 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:27.563 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:27.563 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:27.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.563 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:12:27.563 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:27.563 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:27.563 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:27.563 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:27.564 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:27.564 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:27.564 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:27.564 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:27.564 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:27.564 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:27.564 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:27.564 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:27.564 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:27.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.564 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:27.565 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6817", 2023-10-08T00:12:27.565 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:27.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.565 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:27.565 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6819", 2023-10-08T00:12:27.565 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:27.565 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:27.565 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:27.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.565 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:27.565 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:27.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.566 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:27.566 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6822", 2023-10-08T00:12:27.566 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:27.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.566 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:27.566 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6824", 2023-10-08T00:12:27.566 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:27.566 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:27.566 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:27.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.567 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:27.567 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:27.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.567 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:27.567 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6830", 2023-10-08T00:12:27.567 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:27.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.567 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:27.567 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6831", 2023-10-08T00:12:27.567 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:27.567 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:27.568 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:27.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.568 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:27.568 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:27.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.568 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:27.568 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6827", 2023-10-08T00:12:27.568 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:27.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.568 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:27.569 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6829", 2023-10-08T00:12:27.569 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:27.569 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:27.569 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:27.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.569 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6819/86612", 2023-10-08T00:12:27.569 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6824/86612", 2023-10-08T00:12:27.569 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6831/86612", 2023-10-08T00:12:27.569 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6829/86612", 2023-10-08T00:12:27.569 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:27.569 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull", 2023-10-08T00:12:27.569 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:27.570 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:27.570 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:27.570 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:27.570 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:12:27.570 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2023-10-08T00:12:27.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.570 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:12:27.570 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:27.570 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:27.570 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:27.570 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:27.570 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:27.571 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.905615+0000", 2023-10-08T00:12:27.571 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:27.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.571 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:12:27.571 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:27.571 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:27.571 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:27.571 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:27.571 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:27.572 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.880709+0000", 2023-10-08T00:12:27.572 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:27.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.572 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:12:27.572 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:27.572 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:27.572 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:27.572 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:27.572 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:27.572 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.874704+0000", 2023-10-08T00:12:27.573 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:27.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.573 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:12:27.573 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:27.573 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:27.573 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:27.573 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:27.573 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:27.573 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.899604+0000", 2023-10-08T00:12:27.573 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:27.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.574 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:12:27.574 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:27.574 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:27.574 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:27.574 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:27.574 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:27.574 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.844371+0000", 2023-10-08T00:12:27.574 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:27.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.574 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:12:27.575 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:27.575 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:27.575 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:27.575 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:27.575 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:27.575 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.794810+0000", 2023-10-08T00:12:27.575 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:27.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.575 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:12:27.575 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:27.575 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:27.576 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:27.576 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:27.576 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:27.576 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.859474+0000", 2023-10-08T00:12:27.576 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:27.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:27.576 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:12:27.576 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:27.576 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:27.576 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:27.576 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:27.577 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:27.577 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.893130+0000", 2023-10-08T00:12:27.577 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:27.577 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:27.577 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:12:27.577 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2023-10-08T00:12:27.577 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [], 2023-10-08T00:12:27.577 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2023-10-08T00:12:27.577 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2023-10-08T00:12:27.577 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2023-10-08T00:12:27.577 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2023-10-08T00:12:27.578 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2023-10-08T00:12:27.578 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2023-10-08T00:12:27.578 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2023-10-08T00:12:27.578 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2023-10-08T00:12:27.578 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2023-10-08T00:12:27.578 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2023-10-08T00:12:27.578 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2023-10-08T00:12:27.578 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:27.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:27.578 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2023-10-08T00:12:27.578 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2023-10-08T00:12:27.578 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2023-10-08T00:12:27.579 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2023-10-08T00:12:27.579 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2023-10-08T00:12:27.579 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2023-10-08T00:12:27.579 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2023-10-08T00:12:27.579 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2023-10-08T00:12:27.579 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2023-10-08T00:12:27.579 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2023-10-08T00:12:27.579 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2023-10-08T00:12:27.579 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:27.579 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:12:27.581 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.493+0000 7fe58e303700 1 -- 172.21.15.99:0/491551313 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe574066f30 msgr2=0x7fe5740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:27.581 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.493+0000 7fe58e303700 1 --2- 172.21.15.99:0/491551313 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe574066f30 0x7fe5740693e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe5780058d0 tx=0x7fe578009040 comp rx=0 tx=0).stop 2023-10-08T00:12:27.581 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.493+0000 7fe58e303700 1 -- 172.21.15.99:0/491551313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe58815d580 msgr2=0x7fe5881586a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:27.581 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.493+0000 7fe58e303700 1 --2- 172.21.15.99:0/491551313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe58815d580 0x7fe5881586a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe58806b530 tx=0x7fe580019000 comp rx=0 tx=0).stop 2023-10-08T00:12:27.581 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.493+0000 7fe58e303700 1 -- 172.21.15.99:0/491551313 shutdown_connections 2023-10-08T00:12:27.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.493+0000 7fe58e303700 1 --2- 172.21.15.99:0/491551313 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe574066f30 0x7fe5740693e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:27.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.493+0000 7fe58e303700 1 --2- 172.21.15.99:0/491551313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe58815d580 0x7fe5881586a0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:27.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.493+0000 7fe58e303700 1 --2- 172.21.15.99:0/491551313 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe588157870 0x7fe58814aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:27.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.493+0000 7fe58e303700 1 --2- 172.21.15.99:0/491551313 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe58815a620 0x7fe588158160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:27.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.493+0000 7fe58e303700 1 -- 172.21.15.99:0/491551313 >> 172.21.15.99:0/491551313 conn(0x7fe588068ce0 msgr2=0x7fe5880b4900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:27.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.493+0000 7fe58e303700 1 -- 172.21.15.99:0/491551313 shutdown_connections 2023-10-08T00:12:27.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.493+0000 7fe58e303700 1 -- 172.21.15.99:0/491551313 wait complete. 2023-10-08T00:12:27.583 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-10-08T00:12:27.583 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:12:27.583 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=265) 2023-10-08T00:12:27.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T00:12:27.832 INFO:teuthology.orchestra.run.smithi099.stderr:dd: fdatasync failed for 'large_file_b': No space left on device 2023-10-08T00:12:27.832 INFO:teuthology.orchestra.run.smithi099.stderr:50+0 records in 2023-10-08T00:12:27.832 INFO:teuthology.orchestra.run.smithi099.stderr:50+0 records out 2023-10-08T00:12:27.832 INFO:teuthology.orchestra.run.smithi099.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.227151 s, 231 MB/s 2023-10-08T00:12:27.832 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-08T00:12:27.946 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.945+0000 7f6d6a2d6700 1 Processor -- start 2023-10-08T00:12:27.946 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.945+0000 7f6d6a2d6700 1 -- start start 2023-10-08T00:12:27.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.946+0000 7f6d6a2d6700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6d640b3430 0x7f6d64164e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:27.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.946+0000 7f6d6a2d6700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d6415db80 con 0x7f6d6415a1e0 2023-10-08T00:12:27.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.946+0000 7f6d6a2d6700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d6415de60 con 0x7f6d64159e10 2023-10-08T00:12:27.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.946+0000 7f6d6a2d6700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d6415e800 con 0x7f6d640b3430 2023-10-08T00:12:27.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.946+0000 7f6d637fe700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f6d6415a1e0 0x7f6d640cd740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:43508/0 (socket says 172.21.15.99:43508) 2023-10-08T00:12:27.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.946+0000 7f6d637fe700 1 -- 172.21.15.99:0/2066210266 learned_addr learned my addr 172.21.15.99:0/2066210266 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:12:27.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.946+0000 7f6d68873700 1 --2- 172.21.15.99:0/2066210266 >> v2:172.21.15.99:3300/0 conn(0x7f6d640b3430 0x7f6d64164e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:27.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.946+0000 7f6d68873700 1 -- 172.21.15.99:0/2066210266 >> v1:172.21.15.191:6789/0 conn(0x7f6d64159e10 legacy=0x7f6d640b2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:27.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.946+0000 7f6d68873700 1 -- 172.21.15.99:0/2066210266 >> v1:172.21.15.99:6789/0 conn(0x7f6d6415a1e0 legacy=0x7f6d640cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:27.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.946+0000 7f6d68873700 1 -- 172.21.15.99:0/2066210266 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d6415e140 con 0x7f6d640b3430 2023-10-08T00:12:27.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.946+0000 7f6d62ffd700 1 -- 172.21.15.99:0/2066210266 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 611129847 0 0) 0x7f6d6415db80 con 0x7f6d6415a1e0 2023-10-08T00:12:27.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.947+0000 7f6d68873700 1 --2- 172.21.15.99:0/2066210266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d640b3430 0x7f6d64164e20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c009e60 tx=0x7f6d5c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f2338f8865851c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:27.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.947+0000 7f6d62ffd700 1 -- 172.21.15.99:0/2066210266 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c00e070 con 0x7f6d640b3430 2023-10-08T00:12:27.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.947+0000 7f6d62ffd700 1 -- 172.21.15.99:0/2066210266 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d5c00bbf0 con 0x7f6d640b3430 2023-10-08T00:12:27.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.947+0000 7f6d62ffd700 1 -- 172.21.15.99:0/2066210266 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c012430 con 0x7f6d640b3430 2023-10-08T00:12:27.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.947+0000 7f6d6a2d6700 1 -- 172.21.15.99:0/2066210266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d640b3430 msgr2=0x7f6d64164e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:27.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.947+0000 7f6d6a2d6700 1 --2- 172.21.15.99:0/2066210266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d640b3430 0x7f6d64164e20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c009e60 tx=0x7f6d5c009780 comp rx=0 tx=0).stop 2023-10-08T00:12:27.950 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.947+0000 7f6d6a2d6700 1 -- 172.21.15.99:0/2066210266 shutdown_connections 2023-10-08T00:12:27.950 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.947+0000 7f6d6a2d6700 1 --2- 172.21.15.99:0/2066210266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d640b3430 0x7f6d64164e20 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:27.950 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.947+0000 7f6d6a2d6700 1 -- 172.21.15.99:0/2066210266 >> 172.21.15.99:0/2066210266 conn(0x7f6d64067660 msgr2=0x7f6d64067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:27.950 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.947+0000 7f6d6a2d6700 1 -- 172.21.15.99:0/2066210266 shutdown_connections 2023-10-08T00:12:27.950 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.947+0000 7f6d6a2d6700 1 -- 172.21.15.99:0/2066210266 wait complete. 2023-10-08T00:12:27.952 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.948+0000 7f6d6a2d6700 1 Processor -- start 2023-10-08T00:12:27.952 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.949+0000 7f6d6a2d6700 1 -- start start 2023-10-08T00:12:27.952 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.949+0000 7f6d6a2d6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d640b3430 0x7f6d64159410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:27.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.949+0000 7f6d6a2d6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d64159e10 0x7f6d640cf000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:27.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.949+0000 7f6d6a2d6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d6415a1e0 0x7f6d640cf540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:27.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.949+0000 7f6d6a2d6700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d640c9840 con 0x7f6d640b3430 2023-10-08T00:12:27.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.949+0000 7f6d6a2d6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d640ceb90 con 0x7f6d64159e10 2023-10-08T00:12:27.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.949+0000 7f6d6a2d6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d640cfbe0 con 0x7f6d6415a1e0 2023-10-08T00:12:27.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.949+0000 7f6d63fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d640b3430 0x7f6d64159410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:27.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.949+0000 7f6d63fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d640b3430 0x7f6d64159410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51864/0 (socket says 172.21.15.99:51864) 2023-10-08T00:12:27.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.949+0000 7f6d63fff700 1 -- 172.21.15.99:0/2689518537 learned_addr learned my addr 172.21.15.99:0/2689518537 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:27.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.950+0000 7f6d637fe700 1 --2- 172.21.15.99:0/2689518537 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d64159e10 0x7f6d640cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:27.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.950+0000 7f6d63fff700 1 -- 172.21.15.99:0/2689518537 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d6415a1e0 msgr2=0x7f6d640cf540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:27.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.950+0000 7f6d63fff700 1 --2- 172.21.15.99:0/2689518537 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d6415a1e0 0x7f6d640cf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:27.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.950+0000 7f6d63fff700 1 -- 172.21.15.99:0/2689518537 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d64159e10 msgr2=0x7f6d640cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:27.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.950+0000 7f6d63fff700 1 --2- 172.21.15.99:0/2689518537 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d64159e10 0x7f6d640cf000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:27.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.950+0000 7f6d63fff700 1 -- 172.21.15.99:0/2689518537 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d5c009a20 con 0x7f6d640b3430 2023-10-08T00:12:27.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.950+0000 7f6d63fff700 1 --2- 172.21.15.99:0/2689518537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d640b3430 0x7f6d64159410 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6d5400b600 tx=0x7f6d540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:27.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.950+0000 7f6d4bfff700 1 -- 172.21.15.99:0/2689518537 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5400f510 con 0x7f6d640b3430 2023-10-08T00:12:27.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.950+0000 7f6d4bfff700 1 -- 172.21.15.99:0/2689518537 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d5400f670 con 0x7f6d640b3430 2023-10-08T00:12:27.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.950+0000 7f6d6a2d6700 1 -- 172.21.15.99:0/2689518537 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d640cfec0 con 0x7f6d640b3430 2023-10-08T00:12:27.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.950+0000 7f6d4bfff700 1 -- 172.21.15.99:0/2689518537 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5400fa30 con 0x7f6d640b3430 2023-10-08T00:12:27.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.950+0000 7f6d6a2d6700 1 -- 172.21.15.99:0/2689518537 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d640d01a0 con 0x7f6d640b3430 2023-10-08T00:12:27.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.951+0000 7f6d6a2d6700 1 -- 172.21.15.99:0/2689518537 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d640c9330 con 0x7f6d640b3430 2023-10-08T00:12:27.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.951+0000 7f6d4bfff700 1 -- 172.21.15.99:0/2689518537 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6d5401a400 con 0x7f6d640b3430 2023-10-08T00:12:27.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.952+0000 7f6d4bfff700 1 --2- 172.21.15.99:0/2689518537 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6d50066f30 0x7f6d500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:27.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.952+0000 7f6d4bfff700 1 -- 172.21.15.99:0/2689518537 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6d540840e0 con 0x7f6d640b3430 2023-10-08T00:12:27.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.952+0000 7f6d4bfff700 1 -- 172.21.15.99:0/2689518537 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=50}) v3 -- 0x7f6d5006ea50 con 0x7f6d640b3430 2023-10-08T00:12:27.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.952+0000 7f6d637fe700 1 --2- 172.21.15.99:0/2689518537 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6d50066f30 0x7f6d500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:27.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.952+0000 7f6d637fe700 1 --2- 172.21.15.99:0/2689518537 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6d50066f30 0x7f6d500693e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6d6415a970 tx=0x7f6d58009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:27.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:27.956+0000 7f6d4bfff700 1 -- 172.21.15.99:0/2689518537 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6d640c9330 con 0x7f6d640b3430 2023-10-08T00:12:28.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.142+0000 7f6d6a2d6700 1 -- 172.21.15.99:0/2689518537 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f6d640c9330 con 0x7f6d640b3430 2023-10-08T00:12:28.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.143+0000 7f6d4bfff700 1 -- 172.21.15.99:0/2689518537 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v49) v1 ==== 81+0+32475 (secure 0 0 0) 0x7f6d640c9330 con 0x7f6d640b3430 2023-10-08T00:12:28.144 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:12:28.144 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:12:28.144 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 49, 2023-10-08T00:12:28.144 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", 2023-10-08T00:12:28.144 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2023-10-08T00:10:36.303601+0000", 2023-10-08T00:12:28.144 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2023-10-08T00:12:26.076074+0000", 2023-10-08T00:12:28.145 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2023-10-08T00:10:41.709171+0000", 2023-10-08T00:12:28.145 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2023-10-08T00:10:38.690404+0000", 2023-10-08T00:12:28.145 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-08T00:12:28.145 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2023-10-08T00:12:28.145 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2023-10-08T00:12:28.145 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2023-10-08T00:12:28.145 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2023-10-08T00:12:28.145 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2023-10-08T00:12:28.145 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2023-10-08T00:12:28.145 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:12:28.145 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2023-10-08T00:12:28.145 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2023-10-08T00:12:28.146 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-08T00:12:28.146 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-08T00:12:28.146 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2023-10-08T00:12:28.146 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 7, 2023-10-08T00:12:28.146 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2023-10-08T00:12:28.146 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2023-10-08T00:12:28.146 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "jewel", 2023-10-08T00:12:28.146 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "quincy", 2023-10-08T00:12:28.146 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2023-10-08T00:12:28.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.146 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2023-10-08T00:12:28.147 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2023-10-08T00:12:28.147 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:10:42.063344+0000", 2023-10-08T00:12:28.147 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 3, 2023-10-08T00:12:28.147 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full", 2023-10-08T00:12:28.147 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:12:28.147 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:12:28.147 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:12:28.147 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:12:28.147 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:12:28.147 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:12:28.147 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:12:28.147 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:12:28.148 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:12:28.148 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:12:28.148 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2023-10-08T00:12:28.148 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2023-10-08T00:12:28.148 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2023-10-08T00:12:28.148 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2023-10-08T00:12:28.148 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2023-10-08T00:12:28.148 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:12:28.148 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:12:28.148 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:12:28.148 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:12:28.148 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:12:28.149 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:12:28.149 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:12:28.149 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.149 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "46", 2023-10-08T00:12:28.149 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:12:28.149 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:12:28.149 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:12:28.149 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:12:28.149 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:12:28.149 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:12:28.149 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:12:28.149 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:12:28.150 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:12:28.150 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:12:28.150 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:12:28.150 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:12:28.150 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:12:28.150 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:12:28.150 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:12:28.150 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:12:28.150 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:12:28.150 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:12:28.150 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:12:28.150 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:12:28.151 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:12:28.151 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:12:28.151 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:12:28.151 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:12:28.151 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:12:28.151 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:12:28.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.151 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:12:28.151 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:12:28.151 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:12:28.151 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:12:28.151 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:12:28.152 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:12:28.152 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:12:28.152 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:12:28.152 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:12:28.152 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:12:28.152 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:12:28.152 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:12:28.152 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2023-10-08T00:12:28.152 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2023-10-08T00:12:28.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.152 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:12:28.152 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2023-10-08T00:12:28.153 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:28.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.153 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.153 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 6, 2023-10-08T00:12:28.153 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2023-10-08T00:12:28.153 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:12:07.733432+0000", 2023-10-08T00:12:28.153 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 3, 2023-10-08T00:12:28.153 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full", 2023-10-08T00:12:28.153 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:12:28.153 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:12:28.153 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:12:28.153 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:12:28.154 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:12:28.154 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:12:28.154 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:12:28.154 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:12:28.154 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:12:28.154 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:12:28.154 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2023-10-08T00:12:28.154 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2023-10-08T00:12:28.154 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2023-10-08T00:12:28.154 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2023-10-08T00:12:28.154 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2023-10-08T00:12:28.154 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:12:28.155 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:12:28.155 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:12:28.155 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:12:28.155 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:12:28.155 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:12:28.155 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:12:28.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.155 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "46", 2023-10-08T00:12:28.155 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:12:28.155 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:12:28.155 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:12:28.155 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:12:28.155 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:12:28.156 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:12:28.156 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:12:28.156 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:12:28.156 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:12:28.156 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:12:28.156 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:12:28.156 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:12:28.156 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:12:28.157 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:12:28.157 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:12:28.157 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:12:28.157 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:12:28.157 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:12:28.157 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:12:28.157 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:12:28.157 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:12:28.157 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:12:28.157 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:12:28.157 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:12:28.157 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:12:28.158 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:12:28.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.158 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:12:28.158 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:12:28.158 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:12:28.158 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:12:28.158 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:12:28.158 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:12:28.158 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:12:28.159 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:12:28.159 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:12:28.159 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:12:28.159 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:12:28.159 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:12:28.159 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2023-10-08T00:12:28.159 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2023-10-08T00:12:28.159 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2023-10-08T00:12:28.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.159 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:12:28.159 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:12:28.159 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2023-10-08T00:12:28.160 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:28.160 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:28.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.160 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 7, 2023-10-08T00:12:28.160 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2023-10-08T00:12:28.160 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:12:08.762879+0000", 2023-10-08T00:12:28.160 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 3, 2023-10-08T00:12:28.160 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full", 2023-10-08T00:12:28.160 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:12:28.160 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:12:28.160 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:12:28.161 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:12:28.161 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:12:28.161 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:12:28.161 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:12:28.161 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:12:28.161 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:12:28.161 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:12:28.161 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2023-10-08T00:12:28.161 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2023-10-08T00:12:28.161 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2023-10-08T00:12:28.161 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2023-10-08T00:12:28.161 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2023-10-08T00:12:28.162 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:12:28.162 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:12:28.162 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:12:28.162 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:12:28.162 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:12:28.162 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:12:28.162 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:12:28.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.162 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "46", 2023-10-08T00:12:28.162 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:12:28.162 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:12:28.162 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:12:28.163 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:12:28.163 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:12:28.163 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:12:28.163 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:12:28.163 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:12:28.163 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:12:28.163 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:12:28.163 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:12:28.163 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:12:28.163 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:12:28.163 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:12:28.163 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:12:28.164 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:12:28.164 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:12:28.164 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:12:28.164 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:12:28.164 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:12:28.164 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:12:28.164 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:12:28.164 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:12:28.164 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:12:28.164 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:12:28.164 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:12:28.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.165 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:12:28.165 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:12:28.165 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:12:28.165 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:12:28.165 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:12:28.165 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:12:28.165 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:12:28.165 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:12:28.165 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:12:28.166 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:12:28.166 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:12:28.166 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:12:28.166 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2023-10-08T00:12:28.166 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2023-10-08T00:12:28.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.166 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:12:28.166 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:12:28.166 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2023-10-08T00:12:28.166 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:28.167 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:28.167 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:28.167 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:12:28.167 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2023-10-08T00:12:28.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.167 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:12:28.167 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:28.167 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:28.167 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:28.167 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:28.167 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:28.168 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:28.168 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:28.168 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:28.168 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:28.168 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:28.168 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:28.168 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:28.168 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:28.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.168 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:28.168 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2023-10-08T00:12:28.168 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:28.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.169 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:28.169 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2023-10-08T00:12:28.169 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:28.169 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:28.169 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:28.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.169 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:28.169 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:28.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.170 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:28.170 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2023-10-08T00:12:28.170 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:28.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.170 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:28.170 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2023-10-08T00:12:28.170 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:28.170 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:28.170 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:28.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.170 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:28.171 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:28.171 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.171 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:28.171 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2023-10-08T00:12:28.171 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:28.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.171 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.171 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:28.171 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2023-10-08T00:12:28.171 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:28.171 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:28.171 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:28.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.172 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:28.172 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:28.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.172 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:28.172 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2023-10-08T00:12:28.172 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:28.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.172 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:28.172 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2023-10-08T00:12:28.172 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:28.173 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:28.173 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:28.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.173 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/86365", 2023-10-08T00:12:28.173 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/86365", 2023-10-08T00:12:28.173 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/86365", 2023-10-08T00:12:28.173 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/86365", 2023-10-08T00:12:28.173 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:28.173 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull", 2023-10-08T00:12:28.173 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:28.173 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:28.173 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:28.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.174 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.174 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:12:28.174 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:28.174 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:28.174 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:28.174 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:28.174 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:28.174 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:28.174 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:28.174 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:28.174 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:28.175 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:28.175 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:28.175 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:28.175 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:28.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.175 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:28.175 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2023-10-08T00:12:28.175 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:28.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.175 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:28.175 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2023-10-08T00:12:28.175 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:28.176 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:28.176 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:28.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.176 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:28.176 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:28.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.176 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:28.176 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2023-10-08T00:12:28.176 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:28.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.176 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:28.177 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2023-10-08T00:12:28.177 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:28.177 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:28.177 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:28.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.177 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:28.177 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:28.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.177 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:28.177 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2023-10-08T00:12:28.177 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:28.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.178 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.178 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:28.178 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2023-10-08T00:12:28.178 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:28.178 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:28.178 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:28.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.178 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:28.178 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:28.178 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.178 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:28.178 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2023-10-08T00:12:28.179 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:28.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.179 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:28.179 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2023-10-08T00:12:28.179 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:28.179 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:28.179 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:28.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.179 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/86366", 2023-10-08T00:12:28.179 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/86366", 2023-10-08T00:12:28.179 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/86366", 2023-10-08T00:12:28.180 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/86366", 2023-10-08T00:12:28.180 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:28.180 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull", 2023-10-08T00:12:28.180 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:28.180 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:28.180 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:28.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.180 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:12:28.180 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:28.180 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:28.180 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:28.181 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:28.181 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:28.181 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:28.181 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:28.181 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:28.181 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:28.181 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:28.181 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:28.181 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:28.181 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:28.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.182 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:28.182 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2023-10-08T00:12:28.182 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:28.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.182 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:28.182 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2023-10-08T00:12:28.182 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:28.182 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:28.182 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:28.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.183 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:28.183 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:28.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.183 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:28.183 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2023-10-08T00:12:28.183 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:28.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.183 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:28.183 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2023-10-08T00:12:28.183 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:28.183 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:28.184 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:28.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.184 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:28.184 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:28.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.184 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:28.184 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2023-10-08T00:12:28.184 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:28.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.184 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:28.184 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2023-10-08T00:12:28.185 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:28.185 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:28.185 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:28.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.185 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:28.185 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:28.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.185 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:28.185 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2023-10-08T00:12:28.185 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:28.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.186 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:28.186 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2023-10-08T00:12:28.186 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:28.186 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:28.186 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:28.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.186 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6817/86382", 2023-10-08T00:12:28.186 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/86382", 2023-10-08T00:12:28.186 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/86382", 2023-10-08T00:12:28.186 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/86382", 2023-10-08T00:12:28.186 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:28.187 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:28.187 INFO:teuthology.orchestra.run.smithi099.stdout: "full", 2023-10-08T00:12:28.187 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:28.187 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:28.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.187 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:12:28.187 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:28.187 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:28.187 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:28.187 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:28.187 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:28.188 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:28.188 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:28.188 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:28.188 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:28.188 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:28.188 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:28.188 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:28.188 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:28.188 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.188 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:28.188 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2023-10-08T00:12:28.188 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:28.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.189 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:28.189 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2023-10-08T00:12:28.189 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:28.189 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:28.189 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:28.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.189 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:28.189 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:28.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.189 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:28.189 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2023-10-08T00:12:28.190 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:28.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.190 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:28.190 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2023-10-08T00:12:28.190 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:28.190 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:28.190 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:28.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.190 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:28.190 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:28.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.191 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:28.191 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2023-10-08T00:12:28.191 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:28.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.191 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:28.191 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2023-10-08T00:12:28.191 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:28.191 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:28.191 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:28.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.192 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:28.192 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:28.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.192 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:28.192 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2023-10-08T00:12:28.192 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:28.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.192 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:28.192 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2023-10-08T00:12:28.192 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:28.192 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:28.193 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:28.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.193 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6820/86393", 2023-10-08T00:12:28.193 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/86393", 2023-10-08T00:12:28.193 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/86393", 2023-10-08T00:12:28.193 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/86393", 2023-10-08T00:12:28.193 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:28.193 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull", 2023-10-08T00:12:28.193 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:28.193 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:28.193 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:28.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.194 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:12:28.194 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:28.194 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:28.194 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:28.194 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:28.194 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:28.194 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:28.194 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:28.194 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:28.194 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:28.195 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:28.195 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:28.195 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:28.195 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:28.195 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.195 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:28.195 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6800", 2023-10-08T00:12:28.195 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:28.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.195 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.195 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:28.195 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6801", 2023-10-08T00:12:28.196 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:28.196 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:28.196 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:28.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.196 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:28.196 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:28.196 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.196 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:28.196 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6802", 2023-10-08T00:12:28.196 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:28.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.197 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:28.197 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6803", 2023-10-08T00:12:28.197 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:28.197 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:28.197 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:28.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.197 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:28.197 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:28.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.197 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:28.197 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6806", 2023-10-08T00:12:28.198 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:28.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.198 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:28.198 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6807", 2023-10-08T00:12:28.198 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:28.198 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:28.198 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:28.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.198 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:28.198 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:28.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.199 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:28.199 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6804", 2023-10-08T00:12:28.199 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:28.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.199 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:28.199 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6805", 2023-10-08T00:12:28.199 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:28.199 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:28.199 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:28.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.200 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6801/86570", 2023-10-08T00:12:28.200 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6803/86570", 2023-10-08T00:12:28.200 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6807/86570", 2023-10-08T00:12:28.200 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6805/86570", 2023-10-08T00:12:28.200 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:28.200 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:28.200 INFO:teuthology.orchestra.run.smithi099.stdout: "full", 2023-10-08T00:12:28.201 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:28.201 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:28.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.201 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:12:28.201 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:28.201 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:28.201 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:28.201 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:28.201 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:28.201 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:28.202 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:28.202 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:28.202 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:28.202 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:28.202 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:28.202 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:28.202 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:28.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.202 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:28.202 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6816", 2023-10-08T00:12:28.202 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:28.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.203 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:28.203 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6818", 2023-10-08T00:12:28.203 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:28.203 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:28.203 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:28.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.203 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:28.203 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:28.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.203 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:28.203 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6820", 2023-10-08T00:12:28.204 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:28.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.204 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:28.204 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6821", 2023-10-08T00:12:28.204 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:28.204 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:28.204 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:28.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.204 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:28.204 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:28.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.204 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:28.205 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6826", 2023-10-08T00:12:28.205 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:28.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.205 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:28.205 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6828", 2023-10-08T00:12:28.205 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:28.205 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:28.205 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:28.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.205 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:28.205 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:28.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.206 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:28.206 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6823", 2023-10-08T00:12:28.206 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:28.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.206 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:28.206 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6825", 2023-10-08T00:12:28.206 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:28.207 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:28.207 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:28.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.207 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6818/86608", 2023-10-08T00:12:28.207 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6821/86608", 2023-10-08T00:12:28.207 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6828/86608", 2023-10-08T00:12:28.207 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6825/86608", 2023-10-08T00:12:28.207 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:28.207 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:28.207 INFO:teuthology.orchestra.run.smithi099.stdout: "full", 2023-10-08T00:12:28.207 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:28.208 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:28.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.208 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:12:28.208 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:28.208 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:28.208 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:28.208 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:28.208 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:28.208 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:28.208 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:28.208 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:28.209 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:28.209 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:28.209 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:28.209 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:28.209 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:28.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.209 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:28.209 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6808", 2023-10-08T00:12:28.209 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:28.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.209 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:28.210 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6809", 2023-10-08T00:12:28.210 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:28.210 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:28.210 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:28.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.210 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:28.210 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:28.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.210 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:28.210 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6810", 2023-10-08T00:12:28.210 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:28.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.211 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:28.211 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6811", 2023-10-08T00:12:28.211 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:28.211 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:28.211 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:28.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.211 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:28.211 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:28.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.211 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:28.212 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6814", 2023-10-08T00:12:28.212 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:28.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.212 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:28.212 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6815", 2023-10-08T00:12:28.212 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:28.212 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:28.212 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:28.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.212 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:28.212 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:28.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.213 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:28.213 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6812", 2023-10-08T00:12:28.213 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:28.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.213 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:28.213 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6813", 2023-10-08T00:12:28.213 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:28.213 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:28.213 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:28.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.214 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6809/86571", 2023-10-08T00:12:28.214 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6811/86571", 2023-10-08T00:12:28.214 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6815/86571", 2023-10-08T00:12:28.214 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6813/86571", 2023-10-08T00:12:28.214 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:28.214 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:28.214 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:28.214 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:28.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.214 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.214 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:12:28.214 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:28.215 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:28.215 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:28.215 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:28.215 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:28.215 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:28.215 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:28.215 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:28.215 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:28.215 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:28.215 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:28.215 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:28.215 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:28.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.216 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:28.216 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6817", 2023-10-08T00:12:28.216 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:28.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.216 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:28.216 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6819", 2023-10-08T00:12:28.216 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:28.216 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:28.216 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:28.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.216 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:28.217 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:28.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.217 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:28.217 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6822", 2023-10-08T00:12:28.217 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:28.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.217 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:28.217 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6824", 2023-10-08T00:12:28.217 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:28.217 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:28.218 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:28.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.218 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:28.218 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:28.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.218 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:28.218 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6830", 2023-10-08T00:12:28.218 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:28.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.218 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:28.218 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6831", 2023-10-08T00:12:28.219 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:28.219 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:28.219 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:28.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.219 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:28.219 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:28.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.219 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:28.219 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6827", 2023-10-08T00:12:28.219 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:28.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.220 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:28.220 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6829", 2023-10-08T00:12:28.220 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:28.220 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:28.220 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:28.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.220 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6819/86612", 2023-10-08T00:12:28.220 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6824/86612", 2023-10-08T00:12:28.220 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6831/86612", 2023-10-08T00:12:28.220 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6829/86612", 2023-10-08T00:12:28.220 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:28.220 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull", 2023-10-08T00:12:28.221 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:28.221 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:28.221 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:28.221 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:28.221 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:12:28.221 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2023-10-08T00:12:28.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.221 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:12:28.221 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:28.221 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:28.221 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:28.221 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:28.221 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:28.222 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.905615+0000", 2023-10-08T00:12:28.222 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:28.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.222 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:12:28.222 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:28.222 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:28.222 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:28.222 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:28.222 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:28.222 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.880709+0000", 2023-10-08T00:12:28.223 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:28.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.223 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:12:28.223 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:28.223 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:28.223 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:28.223 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:28.223 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:28.223 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.874704+0000", 2023-10-08T00:12:28.223 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:28.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.224 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:12:28.224 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:28.224 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:28.224 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:28.224 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:28.224 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:28.224 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.899604+0000", 2023-10-08T00:12:28.224 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:28.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.224 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.224 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:12:28.224 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:28.225 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:28.225 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:28.225 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:28.225 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:28.225 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.844371+0000", 2023-10-08T00:12:28.225 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:28.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.225 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:12:28.225 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:28.225 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:28.225 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:28.226 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:28.226 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:28.226 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.794810+0000", 2023-10-08T00:12:28.226 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:28.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.226 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:12:28.226 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:28.226 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:28.226 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:28.226 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:28.226 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:28.227 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.859474+0000", 2023-10-08T00:12:28.227 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:28.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:28.227 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:12:28.227 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:28.227 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:28.227 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:28.227 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:28.227 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:28.227 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.893130+0000", 2023-10-08T00:12:28.227 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:28.228 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:28.228 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:12:28.228 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2023-10-08T00:12:28.228 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [], 2023-10-08T00:12:28.228 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2023-10-08T00:12:28.228 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2023-10-08T00:12:28.228 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2023-10-08T00:12:28.228 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2023-10-08T00:12:28.228 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2023-10-08T00:12:28.228 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2023-10-08T00:12:28.228 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2023-10-08T00:12:28.228 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2023-10-08T00:12:28.229 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2023-10-08T00:12:28.229 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2023-10-08T00:12:28.229 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2023-10-08T00:12:28.229 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:28.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:28.229 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2023-10-08T00:12:28.229 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2023-10-08T00:12:28.229 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2023-10-08T00:12:28.229 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2023-10-08T00:12:28.229 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2023-10-08T00:12:28.229 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2023-10-08T00:12:28.229 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2023-10-08T00:12:28.230 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2023-10-08T00:12:28.230 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2023-10-08T00:12:28.230 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2023-10-08T00:12:28.230 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2023-10-08T00:12:28.230 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:28.230 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:12:28.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.145+0000 7f6d6a2d6700 1 -- 172.21.15.99:0/2689518537 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6d50066f30 msgr2=0x7f6d500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:28.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.145+0000 7f6d6a2d6700 1 --2- 172.21.15.99:0/2689518537 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6d50066f30 0x7f6d500693e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6d6415a970 tx=0x7f6d58009040 comp rx=0 tx=0).stop 2023-10-08T00:12:28.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.145+0000 7f6d6a2d6700 1 -- 172.21.15.99:0/2689518537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d640b3430 msgr2=0x7f6d64159410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:28.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.145+0000 7f6d6a2d6700 1 --2- 172.21.15.99:0/2689518537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d640b3430 0x7f6d64159410 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6d5400b600 tx=0x7f6d540091a0 comp rx=0 tx=0).stop 2023-10-08T00:12:28.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.145+0000 7f6d6a2d6700 1 -- 172.21.15.99:0/2689518537 shutdown_connections 2023-10-08T00:12:28.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.145+0000 7f6d6a2d6700 1 --2- 172.21.15.99:0/2689518537 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6d50066f30 0x7f6d500693e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:28.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.145+0000 7f6d6a2d6700 1 --2- 172.21.15.99:0/2689518537 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d6415a1e0 0x7f6d640cf540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:28.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.145+0000 7f6d6a2d6700 1 --2- 172.21.15.99:0/2689518537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d640b3430 0x7f6d64159410 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:28.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.145+0000 7f6d6a2d6700 1 --2- 172.21.15.99:0/2689518537 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d64159e10 0x7f6d640cf000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:28.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.145+0000 7f6d6a2d6700 1 -- 172.21.15.99:0/2689518537 >> 172.21.15.99:0/2689518537 conn(0x7f6d64067660 msgr2=0x7f6d640b5020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:28.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.145+0000 7f6d6a2d6700 1 -- 172.21.15.99:0/2689518537 shutdown_connections 2023-10-08T00:12:28.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.145+0000 7f6d6a2d6700 1 -- 172.21.15.99:0/2689518537 wait complete. 2023-10-08T00:12:28.236 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:12:28.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.349+0000 7fce3695b700 1 Processor -- start 2023-10-08T00:12:28.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.349+0000 7fce3695b700 1 -- start start 2023-10-08T00:12:28.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.350+0000 7fce3695b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce301577d0 0x7fce300cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:28.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.350+0000 7fce3695b700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fce30157ba0 0x7fce3006a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:28.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.350+0000 7fce3695b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fce3015b0d0 0x7fce3006aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:28.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.350+0000 7fce3695b700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce3015c360 con 0x7fce30157ba0 2023-10-08T00:12:28.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.350+0000 7fce3695b700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce3015b790 con 0x7fce3015b0d0 2023-10-08T00:12:28.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.350+0000 7fce3695b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce3015ba30 con 0x7fce301577d0 2023-10-08T00:12:28.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.350+0000 7fce2f7fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fce30157ba0 0x7fce3006a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:28.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.350+0000 7fce2f7fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fce30157ba0 0x7fce3006a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51876/0 (socket says 172.21.15.99:51876) 2023-10-08T00:12:28.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.350+0000 7fce2f7fe700 1 -- 172.21.15.99:0/3983620098 learned_addr learned my addr 172.21.15.99:0/3983620098 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:28.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.351+0000 7fce2ffff700 1 --2- 172.21.15.99:0/3983620098 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce301577d0 0x7fce300cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:28.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.351+0000 7fce34ef8700 1 --2- 172.21.15.99:0/3983620098 >> v2:172.21.15.191:3300/0 conn(0x7fce3015b0d0 0x7fce3006aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:28.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.351+0000 7fce2f7fe700 1 -- 172.21.15.99:0/3983620098 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce301577d0 msgr2=0x7fce300cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:28.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.351+0000 7fce2f7fe700 1 --2- 172.21.15.99:0/3983620098 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce301577d0 0x7fce300cd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:28.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.351+0000 7fce2f7fe700 1 -- 172.21.15.99:0/3983620098 >> v2:172.21.15.191:3300/0 conn(0x7fce3015b0d0 msgr2=0x7fce3006aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:28.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.351+0000 7fce2f7fe700 1 --2- 172.21.15.99:0/3983620098 >> v2:172.21.15.191:3300/0 conn(0x7fce3015b0d0 0x7fce3006aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:28.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.351+0000 7fce2f7fe700 1 -- 172.21.15.99:0/3983620098 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce3015bd10 con 0x7fce30157ba0 2023-10-08T00:12:28.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.351+0000 7fce2f7fe700 1 --2- 172.21.15.99:0/3983620098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce30157ba0 0x7fce3006a520 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fce3015c2d0 tx=0x7fce24009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2481ba3113a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:28.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.351+0000 7fce2effd700 1 -- 172.21.15.99:0/3983620098 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2400b860 con 0x7fce30157ba0 2023-10-08T00:12:28.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.351+0000 7fce2effd700 1 -- 172.21.15.99:0/3983620098 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce2400b9c0 con 0x7fce30157ba0 2023-10-08T00:12:28.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.351+0000 7fce2effd700 1 -- 172.21.15.99:0/3983620098 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2400bd60 con 0x7fce30157ba0 2023-10-08T00:12:28.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.351+0000 7fce3695b700 1 -- 172.21.15.99:0/3983620098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce30157ba0 msgr2=0x7fce3006a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:28.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.351+0000 7fce3695b700 1 --2- 172.21.15.99:0/3983620098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce30157ba0 0x7fce3006a520 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fce3015c2d0 tx=0x7fce24009250 comp rx=0 tx=0).stop 2023-10-08T00:12:28.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.351+0000 7fce3695b700 1 -- 172.21.15.99:0/3983620098 shutdown_connections 2023-10-08T00:12:28.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.351+0000 7fce3695b700 1 --2- 172.21.15.99:0/3983620098 >> v2:172.21.15.191:3300/0 conn(0x7fce3015b0d0 0x7fce3006aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:28.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.351+0000 7fce3695b700 1 --2- 172.21.15.99:0/3983620098 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce301577d0 0x7fce300cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:28.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.351+0000 7fce3695b700 1 --2- 172.21.15.99:0/3983620098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce30157ba0 0x7fce3006a520 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:28.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.351+0000 7fce3695b700 1 -- 172.21.15.99:0/3983620098 >> 172.21.15.99:0/3983620098 conn(0x7fce30068ce0 msgr2=0x7fce300690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:28.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.352+0000 7fce3695b700 1 -- 172.21.15.99:0/3983620098 shutdown_connections 2023-10-08T00:12:28.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.352+0000 7fce3695b700 1 -- 172.21.15.99:0/3983620098 wait complete. 2023-10-08T00:12:28.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.353+0000 7fce3695b700 1 Processor -- start 2023-10-08T00:12:28.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.353+0000 7fce3695b700 1 -- start start 2023-10-08T00:12:28.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.353+0000 7fce3695b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce301577d0 0x7fce3014b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:28.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.353+0000 7fce3695b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce30157ba0 0x7fce300b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:28.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.353+0000 7fce3695b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce3015b0d0 0x7fce3016a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:28.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.353+0000 7fce3695b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3015e850 con 0x7fce30157ba0 2023-10-08T00:12:28.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.353+0000 7fce3695b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3014bbc0 con 0x7fce301577d0 2023-10-08T00:12:28.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.353+0000 7fce3695b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce3015dc40 con 0x7fce3015b0d0 2023-10-08T00:12:28.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.354+0000 7fce2f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce30157ba0 0x7fce300b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:28.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.354+0000 7fce2f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce30157ba0 0x7fce300b0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51890/0 (socket says 172.21.15.99:51890) 2023-10-08T00:12:28.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.354+0000 7fce2f7fe700 1 -- 172.21.15.99:0/1207346856 learned_addr learned my addr 172.21.15.99:0/1207346856 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:28.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.354+0000 7fce2ffff700 1 --2- 172.21.15.99:0/1207346856 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce301577d0 0x7fce3014b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:28.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.354+0000 7fce34ef8700 1 --2- 172.21.15.99:0/1207346856 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce3015b0d0 0x7fce3016a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:28.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.354+0000 7fce2f7fe700 1 -- 172.21.15.99:0/1207346856 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce3015b0d0 msgr2=0x7fce3016a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:28.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.354+0000 7fce2f7fe700 1 --2- 172.21.15.99:0/1207346856 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce3015b0d0 0x7fce3016a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:28.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.354+0000 7fce2f7fe700 1 -- 172.21.15.99:0/1207346856 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce301577d0 msgr2=0x7fce3014b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:28.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.354+0000 7fce2f7fe700 1 --2- 172.21.15.99:0/1207346856 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce301577d0 0x7fce3014b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:28.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.354+0000 7fce2f7fe700 1 -- 172.21.15.99:0/1207346856 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce240094f0 con 0x7fce30157ba0 2023-10-08T00:12:28.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.354+0000 7fce2ffff700 1 --2- 172.21.15.99:0/1207346856 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce301577d0 0x7fce3014b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:12:28.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.354+0000 7fce2f7fe700 1 --2- 172.21.15.99:0/1207346856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce30157ba0 0x7fce300b0f60 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fce3006b580 tx=0x7fce24017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:28.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.354+0000 7fce17fff700 1 -- 172.21.15.99:0/1207346856 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce24024440 con 0x7fce30157ba0 2023-10-08T00:12:28.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.354+0000 7fce17fff700 1 -- 172.21.15.99:0/1207346856 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce240245a0 con 0x7fce30157ba0 2023-10-08T00:12:28.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.354+0000 7fce17fff700 1 -- 172.21.15.99:0/1207346856 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce24024960 con 0x7fce30157ba0 2023-10-08T00:12:28.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.354+0000 7fce3695b700 1 -- 172.21.15.99:0/1207346856 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce3015df20 con 0x7fce30157ba0 2023-10-08T00:12:28.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.354+0000 7fce3695b700 1 -- 172.21.15.99:0/1207346856 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce3015e4e0 con 0x7fce30157ba0 2023-10-08T00:12:28.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.355+0000 7fce3695b700 1 -- 172.21.15.99:0/1207346856 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce3015e9c0 con 0x7fce30157ba0 2023-10-08T00:12:28.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.355+0000 7fce17fff700 1 -- 172.21.15.99:0/1207346856 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fce2401a420 con 0x7fce30157ba0 2023-10-08T00:12:28.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.356+0000 7fce17fff700 1 --2- 172.21.15.99:0/1207346856 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fce1c066f30 0x7fce1c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:28.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.356+0000 7fce17fff700 1 -- 172.21.15.99:0/1207346856 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fce24086370 con 0x7fce30157ba0 2023-10-08T00:12:28.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.356+0000 7fce2ffff700 1 --2- 172.21.15.99:0/1207346856 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fce1c066f30 0x7fce1c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:28.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.356+0000 7fce17fff700 1 -- 172.21.15.99:0/1207346856 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=50}) v3 -- 0x7fce1c06ea50 con 0x7fce30157ba0 2023-10-08T00:12:28.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.357+0000 7fce2ffff700 1 --2- 172.21.15.99:0/1207346856 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fce1c066f30 0x7fce1c0693e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fce20006fd0 tx=0x7fce20008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:28.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.360+0000 7fce17fff700 1 -- 172.21.15.99:0/1207346856 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fce24053660 con 0x7fce30157ba0 2023-10-08T00:12:28.586 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.585+0000 7fce3695b700 1 -- 172.21.15.99:0/1207346856 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce3004c3c0 con 0x7fce30157ba0 2023-10-08T00:12:28.586 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.586+0000 7fce17fff700 1 -- 172.21.15.99:0/1207346856 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4603 (secure 0 0 0) 0x7fce24015070 con 0x7fce30157ba0 2023-10-08T00:12:28.587 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:12:28.587 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5146,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/1438931938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1438931938},{"type":"v1","addr":"172.21.15.191:6836","nonce":1438931938}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5280,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/3090674680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3090674680},{"type":"v1","addr":"172.21.15.99:6833","nonce":3090674680}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5283,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/4025282357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":4025282357},{"type":"v1","addr":"172.21.15.99:6835","nonce":4025282357}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:12:09.803824+0000","modified":"2023-10-08T00:12:13.436570+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5289},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5289":{"gid":5289,"name":"d","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/2062242133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":2062242133},{"type":"v1","addr":"172.21.15.191:6837","nonce":2062242133}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-08T00:12:28.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.588+0000 7fce3695b700 1 -- 172.21.15.99:0/1207346856 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fce1c066f30 msgr2=0x7fce1c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:28.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.588+0000 7fce3695b700 1 --2- 172.21.15.99:0/1207346856 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fce1c066f30 0x7fce1c0693e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fce20006fd0 tx=0x7fce20008040 comp rx=0 tx=0).stop 2023-10-08T00:12:28.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.588+0000 7fce3695b700 1 -- 172.21.15.99:0/1207346856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce30157ba0 msgr2=0x7fce300b0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:28.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.588+0000 7fce3695b700 1 --2- 172.21.15.99:0/1207346856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce30157ba0 0x7fce300b0f60 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fce3006b580 tx=0x7fce24017000 comp rx=0 tx=0).stop 2023-10-08T00:12:28.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.588+0000 7fce3695b700 1 -- 172.21.15.99:0/1207346856 shutdown_connections 2023-10-08T00:12:28.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.588+0000 7fce3695b700 1 --2- 172.21.15.99:0/1207346856 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fce1c066f30 0x7fce1c0693e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:28.590 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.588+0000 7fce3695b700 1 --2- 172.21.15.99:0/1207346856 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce301577d0 0x7fce3014b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:28.590 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.588+0000 7fce3695b700 1 --2- 172.21.15.99:0/1207346856 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce3015b0d0 0x7fce3016a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:28.590 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.588+0000 7fce3695b700 1 --2- 172.21.15.99:0/1207346856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce30157ba0 0x7fce300b0f60 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:28.590 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.588+0000 7fce3695b700 1 -- 172.21.15.99:0/1207346856 >> 172.21.15.99:0/1207346856 conn(0x7fce30068ce0 msgr2=0x7fce3014d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:28.590 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.588+0000 7fce3695b700 1 -- 172.21.15.99:0/1207346856 shutdown_connections 2023-10-08T00:12:28.590 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:28.588+0000 7fce3695b700 1 -- 172.21.15.99:0/1207346856 wait complete. 2023-10-08T00:12:28.590 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 24 2023-10-08T00:12:28.601 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json status 2023-10-08T00:12:28.745 INFO:teuthology.orchestra.run.smithi191.stdout:{"cluster_fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","whoami":0,"id":5289,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":18.303859159000002,"mdsmap_epoch":24,"osdmap_epoch":49,"osdmap_epoch_barrier":49,"uptime":24.284863744999999} 2023-10-08T00:12:28.746 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", "fs_name": "cephfs", "id": 5289, "mdsmap_epoch": 24, "osdmap_epoch": 49, "osdmap_epoch_barrier": 49, "rank_uptime": 18.303859159, "state": "up:active", "uptime": 24.284863745, "want_state": "up:active", "whoami": 0 } 2023-10-08T00:12:28.746 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-08T00:12:28.746 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:12:28.746 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=small_file_1 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T00:12:28.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T00:12:28.762 INFO:teuthology.orchestra.run.smithi099.stderr:dd: failed to open 'small_file_1': Permission denied 2023-10-08T00:12:28.762 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:12:28.762 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2023-10-08T00:12:28.819 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:12:28.819 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2023-10-08T00:12:28.878 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-08T00:12:29.032 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.031+0000 7fa7ddea5700 1 Processor -- start 2023-10-08T00:12:29.032 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.031+0000 7fa7ddea5700 1 -- start start 2023-10-08T00:12:29.034 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.032+0000 7fa7ddea5700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa7d80b1220 0x7fa7d80c9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:29.034 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.032+0000 7fa7ddea5700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa7d80cb4f0 0x7fa7d80cb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:29.034 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.032+0000 7fa7ddea5700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7d815de60 con 0x7fa7d80b0e50 2023-10-08T00:12:29.035 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.032+0000 7fa7ddea5700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7d80cbeb0 con 0x7fa7d80b1220 2023-10-08T00:12:29.035 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.032+0000 7fa7ddea5700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7d815e140 con 0x7fa7d80cb4f0 2023-10-08T00:12:29.035 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.032+0000 7fa7d6ffd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa7d80b1220 0x7fa7d80c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:29.035 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.032+0000 7fa7d6ffd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa7d80b1220 0x7fa7d80c9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51902/0 (socket says 172.21.15.99:51902) 2023-10-08T00:12:29.035 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.032+0000 7fa7d6ffd700 1 -- 172.21.15.99:0/4093345756 learned_addr learned my addr 172.21.15.99:0/4093345756 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:29.035 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.032+0000 7fa7d7fff700 1 --2- 172.21.15.99:0/4093345756 >> v2:172.21.15.191:3300/0 conn(0x7fa7d80cb4f0 0x7fa7d80cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:29.035 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.032+0000 7fa7d6ffd700 1 -- 172.21.15.99:0/4093345756 >> v2:172.21.15.191:3300/0 conn(0x7fa7d80cb4f0 msgr2=0x7fa7d80cb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:29.035 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.032+0000 7fa7d6ffd700 1 --2- 172.21.15.99:0/4093345756 >> v2:172.21.15.191:3300/0 conn(0x7fa7d80cb4f0 0x7fa7d80cb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:29.035 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.032+0000 7fa7d6ffd700 1 -- 172.21.15.99:0/4093345756 >> v1:172.21.15.191:6789/0 conn(0x7fa7d80b0e50 legacy=0x7fa7d806d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:29.035 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.033+0000 7fa7d6ffd700 1 -- 172.21.15.99:0/4093345756 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d815e420 con 0x7fa7d80b1220 2023-10-08T00:12:29.035 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.033+0000 7fa7d6ffd700 1 --2- 172.21.15.99:0/4093345756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7d80b1220 0x7fa7d80c9180 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa7d815e9e0 tx=0x7fa7cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efb11509e6f7fae4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:29.035 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.033+0000 7fa7d67fc700 1 -- 172.21.15.99:0/4093345756 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc00b860 con 0x7fa7d80b1220 2023-10-08T00:12:29.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.033+0000 7fa7d67fc700 1 -- 172.21.15.99:0/4093345756 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7cc00b9c0 con 0x7fa7d80b1220 2023-10-08T00:12:29.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.033+0000 7fa7d67fc700 1 -- 172.21.15.99:0/4093345756 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc00bd80 con 0x7fa7d80b1220 2023-10-08T00:12:29.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.033+0000 7fa7ddea5700 1 -- 172.21.15.99:0/4093345756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7d80b1220 msgr2=0x7fa7d80c9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:29.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.033+0000 7fa7ddea5700 1 --2- 172.21.15.99:0/4093345756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7d80b1220 0x7fa7d80c9180 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa7d815e9e0 tx=0x7fa7cc009250 comp rx=0 tx=0).stop 2023-10-08T00:12:29.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.033+0000 7fa7ddea5700 1 -- 172.21.15.99:0/4093345756 shutdown_connections 2023-10-08T00:12:29.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.033+0000 7fa7ddea5700 1 --2- 172.21.15.99:0/4093345756 >> v2:172.21.15.191:3300/0 conn(0x7fa7d80cb4f0 0x7fa7d80cb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:29.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.033+0000 7fa7ddea5700 1 --2- 172.21.15.99:0/4093345756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7d80b1220 0x7fa7d80c9180 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:29.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.033+0000 7fa7ddea5700 1 -- 172.21.15.99:0/4093345756 >> 172.21.15.99:0/4093345756 conn(0x7fa7d8069870 msgr2=0x7fa7d8069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:29.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.033+0000 7fa7ddea5700 1 -- 172.21.15.99:0/4093345756 shutdown_connections 2023-10-08T00:12:29.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.033+0000 7fa7ddea5700 1 -- 172.21.15.99:0/4093345756 wait complete. 2023-10-08T00:12:29.039 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.034+0000 7fa7ddea5700 1 Processor -- start 2023-10-08T00:12:29.039 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.035+0000 7fa7ddea5700 1 -- start start 2023-10-08T00:12:29.039 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.035+0000 7fa7ddea5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7d80b0e50 0x7fa7d8152c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:29.039 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.035+0000 7fa7ddea5700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7d80b1220 0x7fa7d80c65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:29.039 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.035+0000 7fa7ddea5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7d80cb4f0 0x7fa7d81589c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:29.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.035+0000 7fa7ddea5700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d80c77f0 con 0x7fa7d80cb4f0 2023-10-08T00:12:29.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.035+0000 7fa7ddea5700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d80c6be0 con 0x7fa7d80b1220 2023-10-08T00:12:29.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.035+0000 7fa7ddea5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d80c6ec0 con 0x7fa7d80b0e50 2023-10-08T00:12:29.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.035+0000 7fa7d7fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7d80cb4f0 0x7fa7d81589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:29.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.035+0000 7fa7d7fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7d80cb4f0 0x7fa7d81589c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51908/0 (socket says 172.21.15.99:51908) 2023-10-08T00:12:29.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.035+0000 7fa7d7fff700 1 -- 172.21.15.99:0/1564155599 learned_addr learned my addr 172.21.15.99:0/1564155599 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:29.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.036+0000 7fa7d6ffd700 1 --2- 172.21.15.99:0/1564155599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7d80b1220 0x7fa7d80c65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:29.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.036+0000 7fa7d77fe700 1 --2- 172.21.15.99:0/1564155599 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7d80b0e50 0x7fa7d8152c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:29.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.036+0000 7fa7d7fff700 1 -- 172.21.15.99:0/1564155599 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7d80b0e50 msgr2=0x7fa7d8152c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:29.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.036+0000 7fa7d7fff700 1 --2- 172.21.15.99:0/1564155599 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7d80b0e50 0x7fa7d8152c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:29.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.036+0000 7fa7d7fff700 1 -- 172.21.15.99:0/1564155599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7d80b1220 msgr2=0x7fa7d80c65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:29.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.036+0000 7fa7d7fff700 1 --2- 172.21.15.99:0/1564155599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7d80b1220 0x7fa7d80c65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:29.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.036+0000 7fa7d7fff700 1 -- 172.21.15.99:0/1564155599 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7cc0094f0 con 0x7fa7d80cb4f0 2023-10-08T00:12:29.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.036+0000 7fa7d7fff700 1 --2- 172.21.15.99:0/1564155599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7d80cb4f0 0x7fa7d81589c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0007f20 tx=0x7fa7d0009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:29.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.036+0000 7fa7bf7fe700 1 -- 172.21.15.99:0/1564155599 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d000eb90 con 0x7fa7d80cb4f0 2023-10-08T00:12:29.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.036+0000 7fa7bf7fe700 1 -- 172.21.15.99:0/1564155599 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d000ed10 con 0x7fa7d80cb4f0 2023-10-08T00:12:29.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.036+0000 7fa7ddea5700 1 -- 172.21.15.99:0/1564155599 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7d80c71a0 con 0x7fa7d80cb4f0 2023-10-08T00:12:29.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.036+0000 7fa7bf7fe700 1 -- 172.21.15.99:0/1564155599 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d0014670 con 0x7fa7d80cb4f0 2023-10-08T00:12:29.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.036+0000 7fa7ddea5700 1 -- 172.21.15.99:0/1564155599 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7d8158f00 con 0x7fa7d80cb4f0 2023-10-08T00:12:29.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.037+0000 7fa7ddea5700 1 -- 172.21.15.99:0/1564155599 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7d80c7960 con 0x7fa7d80cb4f0 2023-10-08T00:12:29.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.037+0000 7fa7bf7fe700 1 -- 172.21.15.99:0/1564155599 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa7d00149e0 con 0x7fa7d80cb4f0 2023-10-08T00:12:29.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.038+0000 7fa7bf7fe700 1 --2- 172.21.15.99:0/1564155599 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa7c4066f30 0x7fa7c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:29.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.038+0000 7fa7bf7fe700 1 -- 172.21.15.99:0/1564155599 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa7d0084b30 con 0x7fa7d80cb4f0 2023-10-08T00:12:29.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.038+0000 7fa7bf7fe700 1 -- 172.21.15.99:0/1564155599 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=50}) v3 -- 0x7fa7c406ea50 con 0x7fa7d80cb4f0 2023-10-08T00:12:29.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.038+0000 7fa7d77fe700 1 --2- 172.21.15.99:0/1564155599 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa7c4066f30 0x7fa7c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:29.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.038+0000 7fa7d77fe700 1 --2- 172.21.15.99:0/1564155599 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa7c4066f30 0x7fa7c40693e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8003a80 tx=0x7fa7c8007ec0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:29.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.042+0000 7fa7bf7fe700 1 -- 172.21.15.99:0/1564155599 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa7d0051da0 con 0x7fa7d80cb4f0 2023-10-08T00:12:29.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.087+0000 7fa7bf7fe700 1 -- 172.21.15.99:0/1564155599 <== mon.0 v2:172.21.15.99:3300/0 7 ==== osd_map(50..50 src has 1..50) v4 ==== 288+0+0 (secure 0 0 0) 0x7fa7d0056160 con 0x7fa7d80cb4f0 2023-10-08T00:12:29.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.087+0000 7fa7bf7fe700 1 -- 172.21.15.99:0/1564155599 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=51}) v3 -- 0x7fa7c406f3a0 con 0x7fa7d80cb4f0 2023-10-08T00:12:29.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.228+0000 7fa7ddea5700 1 -- 172.21.15.99:0/1564155599 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fa7d804c3c0 con 0x7fa7d80cb4f0 2023-10-08T00:12:29.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.229+0000 7fa7bf7fe700 1 -- 172.21.15.99:0/1564155599 <== mon.0 v2:172.21.15.99:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v50) v1 ==== 81+0+32443 (secure 0 0 0) 0x7fa7d00553c0 con 0x7fa7d80cb4f0 2023-10-08T00:12:29.230 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:12:29.230 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:12:29.231 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 50, 2023-10-08T00:12:29.231 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", 2023-10-08T00:12:29.231 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2023-10-08T00:10:36.303601+0000", 2023-10-08T00:12:29.231 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2023-10-08T00:12:29.084724+0000", 2023-10-08T00:12:29.231 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2023-10-08T00:10:41.709171+0000", 2023-10-08T00:12:29.231 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2023-10-08T00:10:38.690404+0000", 2023-10-08T00:12:29.231 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-08T00:12:29.231 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2023-10-08T00:12:29.231 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2023-10-08T00:12:29.231 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2023-10-08T00:12:29.231 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2023-10-08T00:12:29.232 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2023-10-08T00:12:29.232 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2023-10-08T00:12:29.232 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:12:29.232 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2023-10-08T00:12:29.232 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2023-10-08T00:12:29.232 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-08T00:12:29.232 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-08T00:12:29.232 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2023-10-08T00:12:29.232 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 7, 2023-10-08T00:12:29.232 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2023-10-08T00:12:29.232 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2023-10-08T00:12:29.232 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "jewel", 2023-10-08T00:12:29.233 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "quincy", 2023-10-08T00:12:29.233 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2023-10-08T00:12:29.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.233 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2023-10-08T00:12:29.233 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2023-10-08T00:12:29.233 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:10:42.063344+0000", 2023-10-08T00:12:29.233 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 3, 2023-10-08T00:12:29.233 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full", 2023-10-08T00:12:29.234 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:12:29.234 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:12:29.234 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:12:29.234 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:12:29.234 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:12:29.234 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:12:29.234 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:12:29.235 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:12:29.235 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:12:29.235 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:12:29.235 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2023-10-08T00:12:29.235 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2023-10-08T00:12:29.235 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2023-10-08T00:12:29.235 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2023-10-08T00:12:29.235 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2023-10-08T00:12:29.235 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:12:29.235 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:12:29.236 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:12:29.236 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:12:29.236 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:12:29.236 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:12:29.236 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:12:29.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.236 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "46", 2023-10-08T00:12:29.236 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:12:29.236 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:12:29.236 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:12:29.236 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:12:29.236 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:12:29.237 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:12:29.237 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:12:29.237 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:12:29.237 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:12:29.237 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:12:29.237 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:12:29.237 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:12:29.237 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:12:29.237 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:12:29.237 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:12:29.237 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:12:29.237 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:12:29.238 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:12:29.238 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:12:29.238 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:12:29.238 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:12:29.238 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:12:29.238 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:12:29.238 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:12:29.238 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:12:29.238 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:12:29.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.239 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:12:29.239 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:12:29.239 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:12:29.239 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:12:29.239 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:12:29.239 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:12:29.240 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:12:29.240 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:12:29.240 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:12:29.240 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:12:29.240 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:12:29.240 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:12:29.240 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2023-10-08T00:12:29.240 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2023-10-08T00:12:29.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.241 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:12:29.241 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2023-10-08T00:12:29.241 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:29.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.241 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 6, 2023-10-08T00:12:29.242 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2023-10-08T00:12:29.242 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:12:07.733432+0000", 2023-10-08T00:12:29.242 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 3, 2023-10-08T00:12:29.242 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full", 2023-10-08T00:12:29.242 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:12:29.242 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:12:29.242 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:12:29.243 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:12:29.243 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:12:29.243 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:12:29.243 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:12:29.243 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:12:29.243 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:12:29.243 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:12:29.243 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2023-10-08T00:12:29.244 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2023-10-08T00:12:29.244 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2023-10-08T00:12:29.244 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2023-10-08T00:12:29.244 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2023-10-08T00:12:29.244 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:12:29.244 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:12:29.244 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:12:29.244 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:12:29.245 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:12:29.245 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:12:29.245 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:12:29.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.245 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "46", 2023-10-08T00:12:29.245 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:12:29.245 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:12:29.246 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:12:29.246 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:12:29.246 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:12:29.246 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:12:29.246 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:12:29.247 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:12:29.247 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:12:29.247 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:12:29.247 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:12:29.247 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:12:29.247 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:12:29.248 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:12:29.248 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:12:29.248 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:12:29.248 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:12:29.248 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:12:29.248 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:12:29.249 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:12:29.249 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:12:29.249 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:12:29.249 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:12:29.249 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:12:29.249 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:12:29.249 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:12:29.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.250 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:12:29.250 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:12:29.250 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:12:29.250 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:12:29.250 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:12:29.250 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:12:29.250 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:12:29.251 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:12:29.251 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:12:29.251 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:12:29.251 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:12:29.251 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:12:29.251 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2023-10-08T00:12:29.251 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2023-10-08T00:12:29.252 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2023-10-08T00:12:29.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.252 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:12:29.252 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:12:29.252 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2023-10-08T00:12:29.252 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:29.252 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:29.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.253 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 7, 2023-10-08T00:12:29.253 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2023-10-08T00:12:29.253 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:12:08.762879+0000", 2023-10-08T00:12:29.253 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 3, 2023-10-08T00:12:29.253 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full", 2023-10-08T00:12:29.254 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:12:29.254 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:12:29.254 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:12:29.254 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:12:29.254 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:12:29.254 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:12:29.254 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:12:29.254 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:12:29.254 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:12:29.254 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:12:29.254 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2023-10-08T00:12:29.255 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2023-10-08T00:12:29.255 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2023-10-08T00:12:29.255 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2023-10-08T00:12:29.255 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2023-10-08T00:12:29.255 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:12:29.255 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:12:29.255 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:12:29.255 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:12:29.255 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:12:29.256 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:12:29.256 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:12:29.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.256 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "46", 2023-10-08T00:12:29.256 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:12:29.256 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:12:29.256 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:12:29.256 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:12:29.257 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:12:29.257 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:12:29.257 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:12:29.257 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:12:29.257 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:12:29.257 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:12:29.258 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:12:29.258 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:12:29.258 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:12:29.258 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:12:29.258 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:12:29.259 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:12:29.259 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:12:29.259 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:12:29.259 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:12:29.259 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:12:29.259 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:12:29.259 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:12:29.260 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:12:29.260 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:12:29.260 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:12:29.260 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:12:29.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.261 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:12:29.261 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:12:29.261 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:12:29.261 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:12:29.261 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:12:29.261 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:12:29.261 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:12:29.261 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:12:29.262 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:12:29.262 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:12:29.262 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:12:29.262 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:12:29.262 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2023-10-08T00:12:29.262 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2023-10-08T00:12:29.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.263 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:12:29.263 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:12:29.263 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2023-10-08T00:12:29.263 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:29.263 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:29.263 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:29.263 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:12:29.264 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2023-10-08T00:12:29.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.264 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:12:29.264 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:29.264 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:29.264 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:29.264 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:29.264 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:29.264 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:29.265 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:29.265 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:29.265 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:29.265 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:29.265 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:29.265 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:29.265 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:29.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.266 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:29.266 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2023-10-08T00:12:29.266 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:29.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.266 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:29.266 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2023-10-08T00:12:29.267 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:29.267 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:29.267 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:29.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.267 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:29.267 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:29.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.267 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:29.268 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2023-10-08T00:12:29.268 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:29.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.268 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:29.268 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2023-10-08T00:12:29.268 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:29.268 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:29.268 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:29.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.268 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:29.268 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:29.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.269 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:29.269 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2023-10-08T00:12:29.269 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:29.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.269 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:29.269 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2023-10-08T00:12:29.269 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:29.269 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:29.269 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:29.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.269 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:29.270 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:29.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.270 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:29.270 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2023-10-08T00:12:29.270 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:29.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.270 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:29.270 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2023-10-08T00:12:29.270 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:29.271 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:29.271 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:29.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.271 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/86365", 2023-10-08T00:12:29.271 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/86365", 2023-10-08T00:12:29.271 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/86365", 2023-10-08T00:12:29.271 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/86365", 2023-10-08T00:12:29.271 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:29.271 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull", 2023-10-08T00:12:29.272 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:29.272 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:29.272 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:29.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.272 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:12:29.272 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:29.272 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:29.272 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:29.272 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:29.272 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:29.273 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:29.273 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:29.273 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:29.273 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:29.273 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:29.273 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:29.273 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:29.273 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:29.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.273 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:29.273 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2023-10-08T00:12:29.273 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:29.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.274 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:29.274 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2023-10-08T00:12:29.274 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:29.274 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:29.274 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:29.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.274 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:29.274 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:29.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.274 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:29.275 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2023-10-08T00:12:29.275 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:29.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.275 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:29.275 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2023-10-08T00:12:29.275 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:29.275 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:29.275 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:29.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.275 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:29.276 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:29.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.276 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:29.276 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2023-10-08T00:12:29.276 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:29.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.276 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:29.276 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2023-10-08T00:12:29.276 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:29.276 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:29.276 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:29.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.277 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:29.277 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:29.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.277 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:29.277 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2023-10-08T00:12:29.277 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:29.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.277 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:29.277 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2023-10-08T00:12:29.277 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:29.278 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:29.278 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:29.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.278 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/86366", 2023-10-08T00:12:29.278 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/86366", 2023-10-08T00:12:29.278 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/86366", 2023-10-08T00:12:29.278 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/86366", 2023-10-08T00:12:29.278 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:29.278 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:29.278 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:29.279 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:29.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.279 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:12:29.279 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:29.279 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:29.279 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:29.279 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:29.279 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:29.279 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:29.279 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:29.279 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:29.280 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:29.280 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:29.280 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:29.280 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:29.280 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:29.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.280 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:29.280 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2023-10-08T00:12:29.280 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:29.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.280 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:29.281 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2023-10-08T00:12:29.281 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:29.281 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:29.281 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:29.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.281 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:29.281 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:29.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.281 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:29.281 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2023-10-08T00:12:29.281 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:29.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.282 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:29.282 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2023-10-08T00:12:29.282 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:29.282 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:29.282 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:29.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.282 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:29.282 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:29.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.282 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:29.283 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2023-10-08T00:12:29.283 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:29.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.283 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:29.283 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2023-10-08T00:12:29.283 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:29.283 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:29.283 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:29.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.283 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:29.283 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:29.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.284 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:29.284 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2023-10-08T00:12:29.284 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:29.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.284 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:29.284 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2023-10-08T00:12:29.284 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:29.284 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:29.284 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:29.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.285 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6817/86382", 2023-10-08T00:12:29.285 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/86382", 2023-10-08T00:12:29.285 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/86382", 2023-10-08T00:12:29.285 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/86382", 2023-10-08T00:12:29.285 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:29.285 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:29.285 INFO:teuthology.orchestra.run.smithi099.stdout: "full", 2023-10-08T00:12:29.285 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:29.285 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:29.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.285 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:12:29.286 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:29.286 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:29.286 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:29.286 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:29.286 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:29.286 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:29.286 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:29.286 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:29.286 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:29.286 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:29.286 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:29.287 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:29.287 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:29.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.287 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:29.287 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2023-10-08T00:12:29.287 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:29.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.287 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:29.287 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2023-10-08T00:12:29.287 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:29.287 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:29.288 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:29.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.288 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:29.288 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:29.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.288 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:29.288 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2023-10-08T00:12:29.288 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:29.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.288 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:29.289 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2023-10-08T00:12:29.289 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:29.289 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:29.289 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:29.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.289 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:29.289 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:29.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.289 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:29.289 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2023-10-08T00:12:29.289 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:29.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.290 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:29.290 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2023-10-08T00:12:29.290 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:29.290 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:29.290 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:29.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.290 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:29.290 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:29.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.290 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:29.290 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2023-10-08T00:12:29.291 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:29.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.291 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:29.291 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2023-10-08T00:12:29.291 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:29.291 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:29.291 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:29.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.291 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6820/86393", 2023-10-08T00:12:29.291 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/86393", 2023-10-08T00:12:29.291 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/86393", 2023-10-08T00:12:29.292 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/86393", 2023-10-08T00:12:29.292 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:29.292 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull", 2023-10-08T00:12:29.292 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:29.292 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:29.292 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:29.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.292 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:12:29.292 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:29.292 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:29.292 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:29.292 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:29.293 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:29.293 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:29.293 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:29.293 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:29.293 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:29.293 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:29.293 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:29.293 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:29.293 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:29.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.293 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:29.293 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6800", 2023-10-08T00:12:29.294 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:29.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.294 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:29.294 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6801", 2023-10-08T00:12:29.294 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:29.294 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:29.294 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:29.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.294 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:29.294 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:29.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.295 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:29.295 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6802", 2023-10-08T00:12:29.295 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:29.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.295 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:29.295 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6803", 2023-10-08T00:12:29.295 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:29.295 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:29.295 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:29.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.295 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:29.296 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:29.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.296 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:29.296 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6806", 2023-10-08T00:12:29.296 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:29.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.296 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:29.296 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6807", 2023-10-08T00:12:29.296 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:29.296 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:29.297 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:29.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.297 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:29.297 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:29.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.297 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:29.297 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6804", 2023-10-08T00:12:29.297 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:29.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.297 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:29.297 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6805", 2023-10-08T00:12:29.297 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:29.298 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:29.298 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:29.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.298 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6801/86570", 2023-10-08T00:12:29.298 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6803/86570", 2023-10-08T00:12:29.298 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6807/86570", 2023-10-08T00:12:29.298 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6805/86570", 2023-10-08T00:12:29.298 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:29.298 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:29.298 INFO:teuthology.orchestra.run.smithi099.stdout: "full", 2023-10-08T00:12:29.298 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:29.298 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:29.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.299 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:12:29.299 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:29.299 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:29.299 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:29.299 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:29.299 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:29.299 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:29.299 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:29.299 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:29.299 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:29.300 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:29.300 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:29.300 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:29.300 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:29.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.300 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:29.300 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6816", 2023-10-08T00:12:29.300 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:29.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.300 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:29.300 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6818", 2023-10-08T00:12:29.301 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:29.301 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:29.301 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:29.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.301 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:29.301 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:29.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.301 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:29.301 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6820", 2023-10-08T00:12:29.301 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:29.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.301 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:29.302 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6821", 2023-10-08T00:12:29.302 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:29.302 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:29.302 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:29.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.302 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:29.302 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:29.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.302 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:29.302 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6826", 2023-10-08T00:12:29.302 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:29.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.303 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:29.303 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6828", 2023-10-08T00:12:29.303 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:29.303 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:29.303 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:29.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.303 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:29.303 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:29.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.303 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:29.303 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6823", 2023-10-08T00:12:29.304 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:29.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.304 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:29.304 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6825", 2023-10-08T00:12:29.304 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:29.304 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:29.304 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:29.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.304 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6818/86608", 2023-10-08T00:12:29.304 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6821/86608", 2023-10-08T00:12:29.304 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6828/86608", 2023-10-08T00:12:29.305 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6825/86608", 2023-10-08T00:12:29.305 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:29.305 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:29.305 INFO:teuthology.orchestra.run.smithi099.stdout: "full", 2023-10-08T00:12:29.305 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:29.305 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:29.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.305 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:12:29.305 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:29.305 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:29.305 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:29.306 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:29.306 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:29.306 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:29.306 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:29.306 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:29.306 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:29.306 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:29.306 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:29.306 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:29.306 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:29.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.306 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:29.307 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6808", 2023-10-08T00:12:29.307 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:29.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.307 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:29.307 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6809", 2023-10-08T00:12:29.307 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:29.307 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:29.307 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:29.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.307 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:29.307 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:29.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.308 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:29.308 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6810", 2023-10-08T00:12:29.308 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:29.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.308 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:29.308 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6811", 2023-10-08T00:12:29.308 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:29.308 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:29.308 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:29.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.309 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:29.309 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:29.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.309 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:29.309 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6814", 2023-10-08T00:12:29.309 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:29.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.309 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:29.309 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6815", 2023-10-08T00:12:29.310 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:29.310 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:29.310 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:29.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.310 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:29.310 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:29.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.310 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:29.310 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6812", 2023-10-08T00:12:29.310 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:29.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.311 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:29.311 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6813", 2023-10-08T00:12:29.311 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:29.311 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:29.311 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:29.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.311 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6809/86571", 2023-10-08T00:12:29.311 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6811/86571", 2023-10-08T00:12:29.311 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6815/86571", 2023-10-08T00:12:29.311 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6813/86571", 2023-10-08T00:12:29.311 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:29.311 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:29.312 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:29.312 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:29.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.312 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:12:29.312 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:29.312 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:29.312 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:29.312 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:29.312 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:29.312 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:29.312 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:29.313 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:29.313 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:29.313 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:29.313 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:29.313 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:29.313 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:29.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.313 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:29.313 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6817", 2023-10-08T00:12:29.313 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:29.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.314 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:29.314 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6819", 2023-10-08T00:12:29.314 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:29.314 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:29.314 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:29.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.314 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:29.314 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:29.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.314 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:29.314 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6822", 2023-10-08T00:12:29.315 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:29.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.315 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:29.315 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6824", 2023-10-08T00:12:29.315 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:29.315 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:29.315 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:29.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.315 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:29.315 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:29.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.316 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:29.316 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6830", 2023-10-08T00:12:29.316 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:29.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.316 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:29.316 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6831", 2023-10-08T00:12:29.316 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:29.316 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:29.316 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:29.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.316 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:29.316 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:29.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.317 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:29.317 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6827", 2023-10-08T00:12:29.317 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:29.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.317 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:29.317 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6829", 2023-10-08T00:12:29.317 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:29.317 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:29.317 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:29.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.318 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6819/86612", 2023-10-08T00:12:29.318 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6824/86612", 2023-10-08T00:12:29.318 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6831/86612", 2023-10-08T00:12:29.318 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6829/86612", 2023-10-08T00:12:29.318 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:29.318 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull", 2023-10-08T00:12:29.318 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:29.318 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:29.318 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:29.318 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:29.319 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:12:29.319 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2023-10-08T00:12:29.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.319 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:12:29.319 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:29.319 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:29.319 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:29.319 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:29.319 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:29.319 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.905615+0000", 2023-10-08T00:12:29.320 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:29.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.320 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:12:29.320 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:29.320 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:29.320 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:29.320 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:29.320 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:29.320 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.880709+0000", 2023-10-08T00:12:29.320 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:29.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.321 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:12:29.321 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:29.321 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:29.321 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:29.321 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:29.321 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:29.321 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.874704+0000", 2023-10-08T00:12:29.321 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:29.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.322 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:12:29.322 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:29.322 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:29.322 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:29.322 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:29.322 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:29.322 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.899604+0000", 2023-10-08T00:12:29.322 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:29.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.323 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:12:29.323 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:29.323 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:29.323 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:29.323 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:29.323 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:29.323 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.844371+0000", 2023-10-08T00:12:29.323 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:29.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.323 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:12:29.324 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:29.324 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:29.324 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:29.324 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:29.324 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:29.324 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.794810+0000", 2023-10-08T00:12:29.324 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:29.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.325 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:12:29.325 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:29.325 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:29.325 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:29.325 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:29.325 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:29.325 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.859474+0000", 2023-10-08T00:12:29.325 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:29.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:29.326 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:12:29.326 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:29.326 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:29.326 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:29.326 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:29.326 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:29.326 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.893130+0000", 2023-10-08T00:12:29.326 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:29.326 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:29.326 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:12:29.327 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2023-10-08T00:12:29.327 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [], 2023-10-08T00:12:29.327 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2023-10-08T00:12:29.327 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2023-10-08T00:12:29.327 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2023-10-08T00:12:29.327 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2023-10-08T00:12:29.327 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2023-10-08T00:12:29.327 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2023-10-08T00:12:29.327 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2023-10-08T00:12:29.327 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2023-10-08T00:12:29.328 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2023-10-08T00:12:29.328 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2023-10-08T00:12:29.328 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2023-10-08T00:12:29.328 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:29.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:29.328 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2023-10-08T00:12:29.328 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2023-10-08T00:12:29.328 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2023-10-08T00:12:29.328 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2023-10-08T00:12:29.328 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2023-10-08T00:12:29.329 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2023-10-08T00:12:29.329 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2023-10-08T00:12:29.329 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2023-10-08T00:12:29.329 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2023-10-08T00:12:29.329 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2023-10-08T00:12:29.329 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2023-10-08T00:12:29.329 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:29.329 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:12:29.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.232+0000 7fa7ddea5700 1 -- 172.21.15.99:0/1564155599 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa7c4066f30 msgr2=0x7fa7c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:29.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.232+0000 7fa7ddea5700 1 --2- 172.21.15.99:0/1564155599 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa7c4066f30 0x7fa7c40693e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8003a80 tx=0x7fa7c8007ec0 comp rx=0 tx=0).stop 2023-10-08T00:12:29.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.232+0000 7fa7ddea5700 1 -- 172.21.15.99:0/1564155599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7d80cb4f0 msgr2=0x7fa7d81589c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:29.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.232+0000 7fa7ddea5700 1 --2- 172.21.15.99:0/1564155599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7d80cb4f0 0x7fa7d81589c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0007f20 tx=0x7fa7d0009860 comp rx=0 tx=0).stop 2023-10-08T00:12:29.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.232+0000 7fa7ddea5700 1 -- 172.21.15.99:0/1564155599 shutdown_connections 2023-10-08T00:12:29.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.232+0000 7fa7ddea5700 1 --2- 172.21.15.99:0/1564155599 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa7c4066f30 0x7fa7c40693e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:29.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.232+0000 7fa7ddea5700 1 --2- 172.21.15.99:0/1564155599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7d80cb4f0 0x7fa7d81589c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:29.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.232+0000 7fa7ddea5700 1 --2- 172.21.15.99:0/1564155599 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7d80b0e50 0x7fa7d8152c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:29.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.232+0000 7fa7ddea5700 1 --2- 172.21.15.99:0/1564155599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7d80b1220 0x7fa7d80c65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:29.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.232+0000 7fa7ddea5700 1 -- 172.21.15.99:0/1564155599 >> 172.21.15.99:0/1564155599 conn(0x7fa7d8069870 msgr2=0x7fa7d80c4720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:29.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.232+0000 7fa7ddea5700 1 -- 172.21.15.99:0/1564155599 shutdown_connections 2023-10-08T00:12:29.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:29.232+0000 7fa7ddea5700 1 -- 172.21.15.99:0/1564155599 wait complete. 2023-10-08T00:12:29.332 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-08T00:12:31.063 INFO:tasks.ceph.mon.a.smithi099.stderr:2023-10-08T00:12:31.064+0000 7f6c637a7700 -1 log_channel(cluster) log [ERR] : Health check update: 3 full osd(s) (OSD_FULL) 2023-10-08T00:12:34.333 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-08T00:12:34.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.465+0000 7f8ac71b6700 1 Processor -- start 2023-10-08T00:12:34.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.465+0000 7f8ac71b6700 1 -- start start 2023-10-08T00:12:34.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.466+0000 7f8ac71b6700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8ac015a620 0x7f8ac015d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:34.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.466+0000 7f8ac71b6700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ac015cbb0 con 0x7f8ac0157870 2023-10-08T00:12:34.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.466+0000 7f8ac71b6700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ac015bfa0 con 0x7f8ac015d580 2023-10-08T00:12:34.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.466+0000 7f8ac71b6700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ac015bdb0 con 0x7f8ac015a620 2023-10-08T00:12:34.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.466+0000 7f8ac4f52700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f8ac0157870 0x7f8ac015b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50074/0 (socket says 172.21.15.99:50074) 2023-10-08T00:12:34.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.466+0000 7f8ac4f52700 1 -- 172.21.15.99:0/3035849933 learned_addr learned my addr 172.21.15.99:0/3035849933 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:12:34.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.467+0000 7f8abffff700 1 --2- 172.21.15.99:0/3035849933 >> v2:172.21.15.191:3300/0 conn(0x7f8ac015a620 0x7f8ac015d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:34.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.467+0000 7f8abf7fe700 1 -- 172.21.15.99:0/3035849933 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1362798230 0 0) 0x7f8ac015cbb0 con 0x7f8ac0157870 2023-10-08T00:12:34.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.467+0000 7f8abf7fe700 1 -- 172.21.15.99:0/3035849933 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ab0003570 con 0x7f8ac0157870 2023-10-08T00:12:34.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.467+0000 7f8abffff700 1 -- 172.21.15.99:0/3035849933 >> v1:172.21.15.191:6789/0 conn(0x7f8ac015d580 legacy=0x7f8ac0158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:34.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.467+0000 7f8abf7fe700 1 -- 172.21.15.99:0/3035849933 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3660450665 0 0) 0x7f8ab0003570 con 0x7f8ac0157870 2023-10-08T00:12:34.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.467+0000 7f8abffff700 1 -- 172.21.15.99:0/3035849933 >> v1:172.21.15.99:6789/0 conn(0x7f8ac0157870 legacy=0x7f8ac015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:34.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.467+0000 7f8abffff700 1 -- 172.21.15.99:0/3035849933 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac015c280 con 0x7f8ac015a620 2023-10-08T00:12:34.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.467+0000 7f8abffff700 1 --2- 172.21.15.99:0/3035849933 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ac015a620 0x7f8ac015d040 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8aac0098e0 tx=0x7f8aac0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd15331f414d58df server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:34.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.467+0000 7f8abf7fe700 1 -- 172.21.15.99:0/3035849933 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4001470 con 0x7f8ac015a620 2023-10-08T00:12:34.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.467+0000 7f8abf7fe700 1 -- 172.21.15.99:0/3035849933 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aac011420 con 0x7f8ac015a620 2023-10-08T00:12:34.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.467+0000 7f8abf7fe700 1 -- 172.21.15.99:0/3035849933 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aac0117a0 con 0x7f8ac015a620 2023-10-08T00:12:34.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.468+0000 7f8ac71b6700 1 -- 172.21.15.99:0/3035849933 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ac015a620 msgr2=0x7f8ac015d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:34.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.468+0000 7f8ac71b6700 1 --2- 172.21.15.99:0/3035849933 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ac015a620 0x7f8ac015d040 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8aac0098e0 tx=0x7f8aac0091a0 comp rx=0 tx=0).stop 2023-10-08T00:12:34.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.468+0000 7f8ac71b6700 1 -- 172.21.15.99:0/3035849933 shutdown_connections 2023-10-08T00:12:34.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.468+0000 7f8ac71b6700 1 --2- 172.21.15.99:0/3035849933 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ac015a620 0x7f8ac015d040 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:34.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.468+0000 7f8ac71b6700 1 -- 172.21.15.99:0/3035849933 >> 172.21.15.99:0/3035849933 conn(0x7f8ac0068ce0 msgr2=0x7f8ac00690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:34.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.468+0000 7f8ac71b6700 1 -- 172.21.15.99:0/3035849933 shutdown_connections 2023-10-08T00:12:34.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.468+0000 7f8ac71b6700 1 -- 172.21.15.99:0/3035849933 wait complete. 2023-10-08T00:12:34.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.469+0000 7f8ac71b6700 1 Processor -- start 2023-10-08T00:12:34.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.469+0000 7f8ac71b6700 1 -- start start 2023-10-08T00:12:34.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.470+0000 7f8ac71b6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ac0157870 0x7f8ac014fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:34.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.470+0000 7f8ac71b6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ac015a620 0x7f8ac00b2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:34.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.470+0000 7f8ac71b6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ac015d580 0x7f8ac00b2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:34.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.470+0000 7f8ac71b6700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac00ad8e0 con 0x7f8ac015d580 2023-10-08T00:12:34.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.470+0000 7f8ac71b6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac01502c0 con 0x7f8ac015a620 2023-10-08T00:12:34.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.470+0000 7f8ac71b6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac00a9bb0 con 0x7f8ac0157870 2023-10-08T00:12:34.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.470+0000 7f8ac5753700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ac015d580 0x7f8ac00b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:34.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.470+0000 7f8abffff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ac015a620 0x7f8ac00b2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:34.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.470+0000 7f8ac5753700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ac015d580 0x7f8ac00b2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46110/0 (socket says 172.21.15.99:46110) 2023-10-08T00:12:34.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.470+0000 7f8ac5753700 1 -- 172.21.15.99:0/915161343 learned_addr learned my addr 172.21.15.99:0/915161343 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:34.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.470+0000 7f8ac4f52700 1 --2- 172.21.15.99:0/915161343 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ac0157870 0x7f8ac014fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:34.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.470+0000 7f8ac5753700 1 -- 172.21.15.99:0/915161343 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ac0157870 msgr2=0x7f8ac014fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:34.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.470+0000 7f8ac5753700 1 --2- 172.21.15.99:0/915161343 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ac0157870 0x7f8ac014fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:34.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.470+0000 7f8ac5753700 1 -- 172.21.15.99:0/915161343 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ac015a620 msgr2=0x7f8ac00b2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:34.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.470+0000 7f8ac5753700 1 --2- 172.21.15.99:0/915161343 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ac015a620 0x7f8ac00b2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:34.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.470+0000 7f8ac5753700 1 -- 172.21.15.99:0/915161343 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aac009420 con 0x7f8ac015d580 2023-10-08T00:12:34.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.471+0000 7f8ac5753700 1 --2- 172.21.15.99:0/915161343 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ac015d580 0x7f8ac00b2560 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8ab800c8b0 tx=0x7f8ab8001680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:34.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.471+0000 7f8abcff9700 1 -- 172.21.15.99:0/915161343 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab8012630 con 0x7f8ac015d580 2023-10-08T00:12:34.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.471+0000 7f8abcff9700 1 -- 172.21.15.99:0/915161343 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab80127d0 con 0x7f8ac015d580 2023-10-08T00:12:34.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.471+0000 7f8ac71b6700 1 -- 172.21.15.99:0/915161343 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ac00a9e90 con 0x7f8ac015d580 2023-10-08T00:12:34.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.471+0000 7f8abcff9700 1 -- 172.21.15.99:0/915161343 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab8012b90 con 0x7f8ac015d580 2023-10-08T00:12:34.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.471+0000 7f8ac71b6700 1 -- 172.21.15.99:0/915161343 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ac00aa450 con 0x7f8ac015d580 2023-10-08T00:12:34.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.472+0000 7f8ac71b6700 1 -- 172.21.15.99:0/915161343 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ac00ad6b0 con 0x7f8ac015d580 2023-10-08T00:12:34.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.472+0000 7f8abcff9700 1 -- 172.21.15.99:0/915161343 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8ab8011380 con 0x7f8ac015d580 2023-10-08T00:12:34.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.472+0000 7f8abcff9700 1 --2- 172.21.15.99:0/915161343 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8aa4066f30 0x7f8aa40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:34.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.472+0000 7f8abcff9700 1 -- 172.21.15.99:0/915161343 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8ab80846f0 con 0x7f8ac015d580 2023-10-08T00:12:34.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.473+0000 7f8ac4f52700 1 --2- 172.21.15.99:0/915161343 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8aa4066f30 0x7f8aa40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:34.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.473+0000 7f8abcff9700 1 -- 172.21.15.99:0/915161343 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=53}) v3 -- 0x7f8aa406ea50 con 0x7f8ac015d580 2023-10-08T00:12:34.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.473+0000 7f8ac4f52700 1 --2- 172.21.15.99:0/915161343 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8aa4066f30 0x7f8aa40693e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4001ed0 tx=0x7f8ab4000bc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:34.479 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.477+0000 7f8abcff9700 1 -- 172.21.15.99:0/915161343 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8ac00ad6b0 con 0x7f8ac015d580 2023-10-08T00:12:34.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.662+0000 7f8ac71b6700 1 -- 172.21.15.99:0/915161343 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f8ac00ad6b0 con 0x7f8ac015d580 2023-10-08T00:12:34.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.663+0000 7f8abcff9700 1 -- 172.21.15.99:0/915161343 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v52) v1 ==== 81+0+32379 (secure 0 0 0) 0x7f8ac00ad6b0 con 0x7f8ac015d580 2023-10-08T00:12:34.664 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:12:34.664 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:12:34.664 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 52, 2023-10-08T00:12:34.664 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", 2023-10-08T00:12:34.664 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2023-10-08T00:10:36.303601+0000", 2023-10-08T00:12:34.665 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2023-10-08T00:12:32.105706+0000", 2023-10-08T00:12:34.665 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2023-10-08T00:10:41.709171+0000", 2023-10-08T00:12:34.665 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2023-10-08T00:10:38.690404+0000", 2023-10-08T00:12:34.665 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-08T00:12:34.665 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2023-10-08T00:12:34.665 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2023-10-08T00:12:34.665 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2023-10-08T00:12:34.665 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2023-10-08T00:12:34.665 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2023-10-08T00:12:34.665 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2023-10-08T00:12:34.665 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:12:34.666 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2023-10-08T00:12:34.666 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2023-10-08T00:12:34.666 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-08T00:12:34.666 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-08T00:12:34.666 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2023-10-08T00:12:34.666 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 7, 2023-10-08T00:12:34.666 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2023-10-08T00:12:34.666 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2023-10-08T00:12:34.666 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "jewel", 2023-10-08T00:12:34.666 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "quincy", 2023-10-08T00:12:34.666 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2023-10-08T00:12:34.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.667 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2023-10-08T00:12:34.667 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2023-10-08T00:12:34.667 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:10:42.063344+0000", 2023-10-08T00:12:34.667 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 3, 2023-10-08T00:12:34.667 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full", 2023-10-08T00:12:34.667 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:12:34.667 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:12:34.667 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:12:34.667 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:12:34.667 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:12:34.667 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:12:34.667 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:12:34.668 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:12:34.668 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:12:34.668 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:12:34.668 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2023-10-08T00:12:34.668 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2023-10-08T00:12:34.668 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2023-10-08T00:12:34.668 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2023-10-08T00:12:34.668 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2023-10-08T00:12:34.668 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:12:34.668 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:12:34.669 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:12:34.669 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:12:34.669 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:12:34.669 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:12:34.669 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:12:34.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.669 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "46", 2023-10-08T00:12:34.669 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:12:34.669 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:12:34.669 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:12:34.669 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:12:34.669 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:12:34.669 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:12:34.670 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:12:34.670 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:12:34.670 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:12:34.670 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:12:34.670 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:12:34.670 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:12:34.670 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:12:34.670 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:12:34.671 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:12:34.671 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:12:34.671 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:12:34.671 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:12:34.671 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:12:34.671 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:12:34.671 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:12:34.671 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:12:34.672 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:12:34.672 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:12:34.672 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:12:34.672 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:12:34.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.672 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:12:34.672 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:12:34.672 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:12:34.673 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:12:34.673 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:12:34.673 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:12:34.673 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:12:34.673 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:12:34.673 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:12:34.673 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:12:34.673 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:12:34.674 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:12:34.674 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2023-10-08T00:12:34.674 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2023-10-08T00:12:34.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.674 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:12:34.674 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2023-10-08T00:12:34.675 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:34.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.675 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 6, 2023-10-08T00:12:34.675 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2023-10-08T00:12:34.675 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:12:07.733432+0000", 2023-10-08T00:12:34.676 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 3, 2023-10-08T00:12:34.676 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full", 2023-10-08T00:12:34.676 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:12:34.676 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:12:34.676 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:12:34.676 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:12:34.677 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:12:34.677 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:12:34.677 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:12:34.677 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:12:34.677 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:12:34.677 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:12:34.677 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2023-10-08T00:12:34.678 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2023-10-08T00:12:34.678 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2023-10-08T00:12:34.678 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2023-10-08T00:12:34.678 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2023-10-08T00:12:34.678 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:12:34.678 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:12:34.679 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:12:34.679 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:12:34.679 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:12:34.679 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:12:34.679 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:12:34.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.680 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "46", 2023-10-08T00:12:34.680 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:12:34.680 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:12:34.680 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:12:34.680 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:12:34.681 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:12:34.681 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:12:34.681 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:12:34.681 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:12:34.681 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:12:34.681 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:12:34.681 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:12:34.681 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:12:34.681 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:12:34.681 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:12:34.682 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:12:34.682 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:12:34.682 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:12:34.682 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:12:34.682 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:12:34.682 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:12:34.682 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:12:34.682 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:12:34.682 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:12:34.682 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:12:34.682 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:12:34.683 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:12:34.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.683 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:12:34.683 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:12:34.683 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:12:34.683 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:12:34.683 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:12:34.683 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:12:34.683 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:12:34.683 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:12:34.683 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:12:34.683 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:12:34.683 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:12:34.684 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:12:34.684 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2023-10-08T00:12:34.684 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2023-10-08T00:12:34.684 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2023-10-08T00:12:34.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.684 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:12:34.684 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:12:34.684 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2023-10-08T00:12:34.684 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:34.684 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:34.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.685 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 7, 2023-10-08T00:12:34.685 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2023-10-08T00:12:34.685 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:12:08.762879+0000", 2023-10-08T00:12:34.685 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 3, 2023-10-08T00:12:34.685 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full", 2023-10-08T00:12:34.685 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:12:34.685 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:12:34.685 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:12:34.685 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:12:34.686 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:12:34.686 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:12:34.686 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:12:34.686 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:12:34.686 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:12:34.686 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:12:34.686 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2023-10-08T00:12:34.686 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2023-10-08T00:12:34.686 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2023-10-08T00:12:34.686 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2023-10-08T00:12:34.686 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2023-10-08T00:12:34.686 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:12:34.687 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:12:34.687 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:12:34.687 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:12:34.687 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:12:34.687 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:12:34.687 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:12:34.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.687 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "46", 2023-10-08T00:12:34.687 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:12:34.687 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:12:34.687 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:12:34.688 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:12:34.688 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:12:34.688 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:12:34.688 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:12:34.688 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:12:34.688 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:12:34.688 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:12:34.688 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:12:34.688 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:12:34.688 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:12:34.688 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:12:34.688 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:12:34.689 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:12:34.689 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:12:34.689 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:12:34.689 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:12:34.689 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:12:34.689 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:12:34.689 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:12:34.689 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:12:34.689 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:12:34.689 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:12:34.689 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:12:34.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.690 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:12:34.690 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:12:34.690 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:12:34.690 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:12:34.690 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:12:34.690 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:12:34.690 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:12:34.690 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:12:34.690 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:12:34.691 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:12:34.691 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:12:34.691 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:12:34.691 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2023-10-08T00:12:34.691 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2023-10-08T00:12:34.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.691 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:12:34.691 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:12:34.691 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2023-10-08T00:12:34.691 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:34.691 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:34.691 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:34.692 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:12:34.692 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2023-10-08T00:12:34.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.692 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:12:34.692 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:34.692 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:34.692 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:34.692 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:34.693 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:34.693 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:34.693 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:34.693 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:34.693 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:34.693 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:34.693 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:34.693 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:34.693 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:34.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.694 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:34.694 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2023-10-08T00:12:34.694 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:34.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.694 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:34.694 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2023-10-08T00:12:34.694 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:34.695 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:34.695 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:34.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.695 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:34.695 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:34.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.695 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:34.696 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2023-10-08T00:12:34.696 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:34.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.696 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:34.696 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2023-10-08T00:12:34.696 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:34.696 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:34.696 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:34.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.697 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:34.697 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:34.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.697 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:34.697 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2023-10-08T00:12:34.697 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:34.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.697 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:34.697 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2023-10-08T00:12:34.697 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:34.697 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:34.697 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:34.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.698 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:34.698 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:34.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.698 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:34.698 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2023-10-08T00:12:34.698 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:34.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.698 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:34.698 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2023-10-08T00:12:34.699 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:34.699 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:34.699 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:34.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.699 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/86365", 2023-10-08T00:12:34.699 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/86365", 2023-10-08T00:12:34.699 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/86365", 2023-10-08T00:12:34.699 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/86365", 2023-10-08T00:12:34.699 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:34.699 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:34.699 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:34.699 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:34.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.700 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:12:34.700 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:34.700 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:34.700 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:34.700 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:34.700 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:34.700 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:34.700 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:34.700 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:34.700 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:34.700 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:34.701 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:34.701 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:34.701 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:34.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.701 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:34.701 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2023-10-08T00:12:34.701 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:34.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.701 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:34.702 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2023-10-08T00:12:34.702 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:34.702 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:34.702 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:34.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.702 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:34.703 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:34.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.703 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:34.703 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2023-10-08T00:12:34.703 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:34.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.704 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:34.704 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2023-10-08T00:12:34.704 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:34.704 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:34.704 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:34.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.705 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:34.705 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:34.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.705 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:34.705 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2023-10-08T00:12:34.705 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:34.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.706 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:34.706 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2023-10-08T00:12:34.706 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:34.706 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:34.707 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:34.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.707 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:34.707 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:34.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.707 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:34.708 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2023-10-08T00:12:34.708 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:34.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.708 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:34.708 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2023-10-08T00:12:34.709 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:34.709 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:34.709 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:34.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.709 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/86366", 2023-10-08T00:12:34.709 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/86366", 2023-10-08T00:12:34.710 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/86366", 2023-10-08T00:12:34.710 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/86366", 2023-10-08T00:12:34.710 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:34.710 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:34.710 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:34.710 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:34.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.711 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:12:34.711 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:34.711 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:34.711 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:34.711 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:34.712 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:34.712 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:34.712 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:34.712 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:34.712 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:34.713 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:34.713 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:34.713 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:34.713 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:34.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.713 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:34.714 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2023-10-08T00:12:34.714 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:34.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.714 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:34.714 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2023-10-08T00:12:34.715 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:34.715 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:34.715 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:34.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.715 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:34.715 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:34.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.716 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:34.716 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2023-10-08T00:12:34.716 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:34.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.717 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:34.717 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2023-10-08T00:12:34.717 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:34.717 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:34.717 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:34.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.718 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:34.718 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:34.718 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.718 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:34.718 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2023-10-08T00:12:34.718 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:34.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.719 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:34.719 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2023-10-08T00:12:34.719 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:34.720 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:34.720 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:34.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.720 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:34.720 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:34.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.721 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:34.721 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2023-10-08T00:12:34.721 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:34.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.721 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:34.722 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2023-10-08T00:12:34.722 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:34.722 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:34.722 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:34.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.722 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6817/86382", 2023-10-08T00:12:34.722 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/86382", 2023-10-08T00:12:34.723 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/86382", 2023-10-08T00:12:34.723 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/86382", 2023-10-08T00:12:34.723 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:34.723 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:34.723 INFO:teuthology.orchestra.run.smithi099.stdout: "full", 2023-10-08T00:12:34.723 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:34.723 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:34.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.724 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:12:34.724 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:34.724 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:34.724 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:34.724 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:34.724 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:34.724 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:34.725 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:34.725 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:34.725 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:34.725 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:34.725 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:34.725 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:34.725 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:34.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.725 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:34.725 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2023-10-08T00:12:34.725 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:34.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.726 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:34.726 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2023-10-08T00:12:34.726 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:34.726 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:34.726 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:34.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.726 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:34.726 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:34.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.726 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:34.726 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2023-10-08T00:12:34.727 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:34.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.727 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:34.727 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2023-10-08T00:12:34.727 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:34.727 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:34.727 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:34.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.727 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:34.727 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:34.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.728 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:34.728 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2023-10-08T00:12:34.728 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:34.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.728 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:34.728 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2023-10-08T00:12:34.728 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:34.728 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:34.728 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:34.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.728 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:34.728 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:34.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.729 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:34.729 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2023-10-08T00:12:34.729 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:34.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.729 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:34.729 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2023-10-08T00:12:34.729 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:34.729 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:34.729 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:34.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.730 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6820/86393", 2023-10-08T00:12:34.730 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/86393", 2023-10-08T00:12:34.730 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/86393", 2023-10-08T00:12:34.730 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/86393", 2023-10-08T00:12:34.730 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:34.730 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:34.730 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:34.730 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:34.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.730 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:12:34.730 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:34.731 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:34.731 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:34.731 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:34.731 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:34.731 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:34.731 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:34.731 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:34.731 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:34.731 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:34.731 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:34.731 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:34.731 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:34.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.732 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:34.732 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6800", 2023-10-08T00:12:34.732 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:34.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.732 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.732 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:34.732 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6801", 2023-10-08T00:12:34.732 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:34.732 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:34.732 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:34.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.732 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:34.733 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:34.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.733 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:34.733 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6802", 2023-10-08T00:12:34.733 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:34.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.733 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:34.733 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6803", 2023-10-08T00:12:34.733 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:34.733 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:34.733 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:34.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.734 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:34.734 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:34.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.734 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:34.734 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6806", 2023-10-08T00:12:34.734 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:34.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.734 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:34.734 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6807", 2023-10-08T00:12:34.734 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:34.734 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:34.734 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:34.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.735 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:34.735 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:34.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.735 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:34.735 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6804", 2023-10-08T00:12:34.735 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:34.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.735 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:34.735 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6805", 2023-10-08T00:12:34.735 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:34.736 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:34.736 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:34.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.736 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6801/86570", 2023-10-08T00:12:34.736 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6803/86570", 2023-10-08T00:12:34.736 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6807/86570", 2023-10-08T00:12:34.736 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6805/86570", 2023-10-08T00:12:34.736 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:34.736 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:34.736 INFO:teuthology.orchestra.run.smithi099.stdout: "full", 2023-10-08T00:12:34.736 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:34.736 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:34.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.737 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:12:34.737 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:34.737 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:34.737 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:34.737 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:34.737 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:34.737 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:34.737 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:34.737 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:34.737 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:34.737 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:34.738 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:34.738 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:34.738 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:34.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.738 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:34.738 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6816", 2023-10-08T00:12:34.738 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:34.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.738 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:34.738 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6818", 2023-10-08T00:12:34.738 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:34.738 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:34.739 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:34.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.739 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:34.739 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:34.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.739 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:34.739 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6820", 2023-10-08T00:12:34.739 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:34.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.739 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:34.739 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6821", 2023-10-08T00:12:34.740 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:34.740 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:34.740 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:34.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.740 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:34.740 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:34.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.740 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:34.740 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6826", 2023-10-08T00:12:34.740 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:34.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.740 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:34.741 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6828", 2023-10-08T00:12:34.741 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:34.741 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:34.741 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:34.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.741 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:34.741 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:34.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.741 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:34.741 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6823", 2023-10-08T00:12:34.741 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:34.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.742 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:34.742 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6825", 2023-10-08T00:12:34.742 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:34.742 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:34.742 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:34.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.742 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6818/86608", 2023-10-08T00:12:34.742 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6821/86608", 2023-10-08T00:12:34.742 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6828/86608", 2023-10-08T00:12:34.742 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6825/86608", 2023-10-08T00:12:34.742 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:34.742 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:34.742 INFO:teuthology.orchestra.run.smithi099.stdout: "full", 2023-10-08T00:12:34.743 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:34.743 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:34.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.743 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:12:34.743 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:34.743 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:34.743 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:34.743 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:34.743 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:34.743 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:34.743 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:34.744 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:34.744 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:34.744 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:34.744 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:34.744 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:34.744 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:34.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.744 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:34.744 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6808", 2023-10-08T00:12:34.744 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:34.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.745 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:34.745 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6809", 2023-10-08T00:12:34.745 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:34.745 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:34.745 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:34.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.745 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:34.745 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:34.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.745 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:34.745 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6810", 2023-10-08T00:12:34.745 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:34.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.746 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:34.746 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6811", 2023-10-08T00:12:34.746 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:34.746 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:34.746 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:34.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.746 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:34.746 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:34.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.746 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:34.746 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6814", 2023-10-08T00:12:34.746 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:34.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.747 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:34.747 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6815", 2023-10-08T00:12:34.747 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:34.747 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:34.747 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:34.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.747 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:34.747 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:34.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.747 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:34.748 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6812", 2023-10-08T00:12:34.748 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:34.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.748 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:34.748 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6813", 2023-10-08T00:12:34.748 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:34.748 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:34.748 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:34.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.748 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6809/86571", 2023-10-08T00:12:34.748 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6811/86571", 2023-10-08T00:12:34.748 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6815/86571", 2023-10-08T00:12:34.749 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6813/86571", 2023-10-08T00:12:34.749 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:34.749 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:34.749 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:34.749 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:34.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.749 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:12:34.749 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:34.749 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:34.749 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:34.749 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:34.749 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:34.750 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:34.750 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:34.750 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:34.750 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:34.750 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:34.750 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:34.750 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:34.750 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:34.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.750 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:34.750 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6817", 2023-10-08T00:12:34.750 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:34.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.751 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:34.751 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6819", 2023-10-08T00:12:34.751 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:34.751 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:34.751 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:34.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.751 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:34.751 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:34.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.751 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:34.751 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6822", 2023-10-08T00:12:34.752 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:34.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.752 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:34.752 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6824", 2023-10-08T00:12:34.752 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:34.752 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:34.752 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:34.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.752 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:34.752 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:34.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.753 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:34.753 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6830", 2023-10-08T00:12:34.753 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:34.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.753 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:34.753 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6831", 2023-10-08T00:12:34.753 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:34.753 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:34.753 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:34.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.753 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:34.754 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:34.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.754 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:34.754 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6827", 2023-10-08T00:12:34.754 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:34.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.754 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:34.754 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6829", 2023-10-08T00:12:34.754 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:34.754 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:34.754 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:34.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.755 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6819/86612", 2023-10-08T00:12:34.755 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6824/86612", 2023-10-08T00:12:34.755 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6831/86612", 2023-10-08T00:12:34.755 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6829/86612", 2023-10-08T00:12:34.755 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:34.755 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull", 2023-10-08T00:12:34.755 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:34.755 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:34.755 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:34.755 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:34.755 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:12:34.755 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2023-10-08T00:12:34.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.756 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:12:34.756 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:34.756 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:34.756 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:34.756 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:34.756 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:34.756 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.905615+0000", 2023-10-08T00:12:34.756 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:34.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.756 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:12:34.756 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:34.757 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:34.757 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:34.757 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:34.757 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:34.757 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.880709+0000", 2023-10-08T00:12:34.757 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:34.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.757 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:12:34.757 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:34.757 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:34.757 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:34.758 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:34.758 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:34.758 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.874704+0000", 2023-10-08T00:12:34.758 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:34.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.758 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:12:34.758 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:34.758 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:34.758 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:34.758 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:34.758 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:34.758 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.899604+0000", 2023-10-08T00:12:34.759 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:34.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.759 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:12:34.759 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:34.759 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:34.759 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:34.759 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:34.759 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:34.759 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.844371+0000", 2023-10-08T00:12:34.759 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:34.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.760 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:12:34.760 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:34.760 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:34.760 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:34.760 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:34.760 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:34.760 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.794810+0000", 2023-10-08T00:12:34.760 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:34.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.760 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:12:34.760 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:34.761 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:34.761 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:34.761 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:34.761 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:34.761 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.859474+0000", 2023-10-08T00:12:34.761 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:34.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:34.761 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:12:34.761 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:34.761 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:34.761 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:34.761 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:34.762 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:34.762 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.893130+0000", 2023-10-08T00:12:34.762 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:34.762 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:34.762 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:12:34.762 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2023-10-08T00:12:34.762 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [], 2023-10-08T00:12:34.762 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2023-10-08T00:12:34.762 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2023-10-08T00:12:34.762 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2023-10-08T00:12:34.762 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2023-10-08T00:12:34.762 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2023-10-08T00:12:34.763 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2023-10-08T00:12:34.763 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2023-10-08T00:12:34.763 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2023-10-08T00:12:34.763 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2023-10-08T00:12:34.763 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2023-10-08T00:12:34.763 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2023-10-08T00:12:34.763 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:34.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:34.763 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2023-10-08T00:12:34.763 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2023-10-08T00:12:34.763 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2023-10-08T00:12:34.763 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2023-10-08T00:12:34.764 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2023-10-08T00:12:34.764 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2023-10-08T00:12:34.764 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2023-10-08T00:12:34.764 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2023-10-08T00:12:34.764 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2023-10-08T00:12:34.764 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2023-10-08T00:12:34.764 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2023-10-08T00:12:34.764 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:34.764 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:12:34.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.665+0000 7f8ac71b6700 1 -- 172.21.15.99:0/915161343 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8aa4066f30 msgr2=0x7f8aa40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:34.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.665+0000 7f8ac71b6700 1 --2- 172.21.15.99:0/915161343 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8aa4066f30 0x7f8aa40693e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4001ed0 tx=0x7f8ab4000bc0 comp rx=0 tx=0).stop 2023-10-08T00:12:34.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.665+0000 7f8ac71b6700 1 -- 172.21.15.99:0/915161343 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ac015d580 msgr2=0x7f8ac00b2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:34.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.665+0000 7f8ac71b6700 1 --2- 172.21.15.99:0/915161343 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ac015d580 0x7f8ac00b2560 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8ab800c8b0 tx=0x7f8ab8001680 comp rx=0 tx=0).stop 2023-10-08T00:12:34.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.665+0000 7f8ac71b6700 1 -- 172.21.15.99:0/915161343 shutdown_connections 2023-10-08T00:12:34.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.665+0000 7f8ac71b6700 1 --2- 172.21.15.99:0/915161343 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8aa4066f30 0x7f8aa40693e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:34.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.665+0000 7f8ac71b6700 1 --2- 172.21.15.99:0/915161343 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ac015d580 0x7f8ac00b2560 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:34.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.665+0000 7f8ac71b6700 1 --2- 172.21.15.99:0/915161343 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ac0157870 0x7f8ac014fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:34.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.665+0000 7f8ac71b6700 1 --2- 172.21.15.99:0/915161343 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ac015a620 0x7f8ac00b2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:34.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.665+0000 7f8ac71b6700 1 -- 172.21.15.99:0/915161343 >> 172.21.15.99:0/915161343 conn(0x7f8ac0068ce0 msgr2=0x7f8ac00690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:34.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.666+0000 7f8ac71b6700 1 -- 172.21.15.99:0/915161343 shutdown_connections 2023-10-08T00:12:34.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:34.666+0000 7f8ac71b6700 1 -- 172.21.15.99:0/915161343 wait complete. 2023-10-08T00:12:34.767 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-08T00:12:36.117 INFO:tasks.ceph.mon.a.smithi099.stderr:2023-10-08T00:12:36.116+0000 7f6c637a7700 -1 log_channel(cluster) log [ERR] : Health check update: 1 full osd(s) (OSD_FULL) 2023-10-08T00:12:39.768 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-08T00:12:39.895 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.894+0000 7fb341a34700 1 Processor -- start 2023-10-08T00:12:39.895 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.894+0000 7fb341a34700 1 -- start start 2023-10-08T00:12:39.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.895+0000 7fb341a34700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb33c157870 0x7fb33c15b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:39.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.895+0000 7fb341a34700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb33c157c40 0x7fb33c158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:39.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.895+0000 7fb341a34700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb33c15cbb0 con 0x7fb33c15a620 2023-10-08T00:12:39.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.895+0000 7fb341a34700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb33c15bfa0 con 0x7fb33c157870 2023-10-08T00:12:39.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.895+0000 7fb341a34700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c15c280 con 0x7fb33c157c40 2023-10-08T00:12:39.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.895+0000 7fb3327fc700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fb33c15a620 0x7fb33c15d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50088/0 (socket says 172.21.15.99:50088) 2023-10-08T00:12:39.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.895+0000 7fb3327fc700 1 -- 172.21.15.99:0/3200862021 learned_addr learned my addr 172.21.15.99:0/3200862021 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:12:39.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.895+0000 7fb33b7fe700 1 --2- 172.21.15.99:0/3200862021 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb33c157c40 0x7fb33c158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:39.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.895+0000 7fb33affd700 1 --2- 172.21.15.99:0/3200862021 >> v2:172.21.15.191:3300/0 conn(0x7fb33c157870 0x7fb33c15b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:39.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.895+0000 7fb33a7fc700 1 -- 172.21.15.99:0/3200862021 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2323070640 0 0) 0x7fb33c15cbb0 con 0x7fb33c15a620 2023-10-08T00:12:39.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.895+0000 7fb33a7fc700 1 -- 172.21.15.99:0/3200862021 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb328003570 con 0x7fb33c15a620 2023-10-08T00:12:39.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.896+0000 7fb33a7fc700 1 -- 172.21.15.99:0/3200862021 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 23716150 0 0) 0x7fb328003570 con 0x7fb33c15a620 2023-10-08T00:12:39.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.896+0000 7fb33a7fc700 1 -- 172.21.15.99:0/3200862021 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb33c157c40 msgr2=0x7fb33c158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:39.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.896+0000 7fb33a7fc700 1 --2- 172.21.15.99:0/3200862021 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb33c157c40 0x7fb33c158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:39.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.896+0000 7fb33a7fc700 1 -- 172.21.15.99:0/3200862021 >> v2:172.21.15.191:3300/0 conn(0x7fb33c157870 msgr2=0x7fb33c15b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:39.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.896+0000 7fb33a7fc700 1 --2- 172.21.15.99:0/3200862021 >> v2:172.21.15.191:3300/0 conn(0x7fb33c157870 0x7fb33c15b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:39.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.896+0000 7fb33a7fc700 1 -- 172.21.15.99:0/3200862021 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb33c15c560 con 0x7fb33c15a620 2023-10-08T00:12:39.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.896+0000 7fb33a7fc700 1 -- 172.21.15.99:0/3200862021 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7fb324001470 con 0x7fb33c15a620 2023-10-08T00:12:39.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.896+0000 7fb33a7fc700 1 -- 172.21.15.99:0/3200862021 >> v1:172.21.15.99:6789/0 conn(0x7fb33c15a620 legacy=0x7fb33c15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:39.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.896+0000 7fb33a7fc700 1 --2- 172.21.15.99:0/3200862021 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb328003a60 0x7fb328003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:39.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.896+0000 7fb33a7fc700 1 --2- 172.21.15.99:0/3200862021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb328004810 0x7fb328004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:39.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.896+0000 7fb33a7fc700 1 --2- 172.21.15.99:0/3200862021 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb33c157c40 0x7fb3280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:39.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.896+0000 7fb33a7fc700 1 -- 172.21.15.99:0/3200862021 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c15bfa0 con 0x7fb328004810 2023-10-08T00:12:39.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.896+0000 7fb33a7fc700 1 -- 172.21.15.99:0/3200862021 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c15c280 con 0x7fb328003a60 2023-10-08T00:12:39.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.896+0000 7fb33a7fc700 1 -- 172.21.15.99:0/3200862021 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb32800ba60 con 0x7fb33c157c40 2023-10-08T00:12:39.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.896+0000 7fb3327fc700 1 --2- 172.21.15.99:0/3200862021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb328004810 0x7fb328004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:39.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.896+0000 7fb33affd700 1 --2- 172.21.15.99:0/3200862021 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb328003a60 0x7fb328003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:39.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.896+0000 7fb33b7fe700 1 --2- 172.21.15.99:0/3200862021 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb33c157c40 0x7fb3280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:39.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.896+0000 7fb3327fc700 1 -- 172.21.15.99:0/3200862021 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb33c157c40 msgr2=0x7fb3280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:39.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.896+0000 7fb3327fc700 1 --2- 172.21.15.99:0/3200862021 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb33c157c40 0x7fb3280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:39.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.896+0000 7fb3327fc700 1 -- 172.21.15.99:0/3200862021 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb328003a60 msgr2=0x7fb328003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:39.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.896+0000 7fb3327fc700 1 --2- 172.21.15.99:0/3200862021 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb328003a60 0x7fb328003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:39.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.896+0000 7fb3327fc700 1 -- 172.21.15.99:0/3200862021 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb328003570 con 0x7fb328004810 2023-10-08T00:12:39.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.896+0000 7fb3327fc700 1 --2- 172.21.15.99:0/3200862021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb328004810 0x7fb328004c80 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb32400c3c0 tx=0x7fb324001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70aee8b993a7edf3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:39.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.897+0000 7fb33a7fc700 1 -- 172.21.15.99:0/3200862021 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32400cc50 con 0x7fb328004810 2023-10-08T00:12:39.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.897+0000 7fb33a7fc700 1 -- 172.21.15.99:0/3200862021 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb32400cdd0 con 0x7fb328004810 2023-10-08T00:12:39.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.897+0000 7fb33a7fc700 1 -- 172.21.15.99:0/3200862021 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb324005b70 con 0x7fb328004810 2023-10-08T00:12:39.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.897+0000 7fb341a34700 1 -- 172.21.15.99:0/3200862021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb328004810 msgr2=0x7fb328004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:39.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.897+0000 7fb341a34700 1 --2- 172.21.15.99:0/3200862021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb328004810 0x7fb328004c80 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb32400c3c0 tx=0x7fb324001e90 comp rx=0 tx=0).stop 2023-10-08T00:12:39.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.897+0000 7fb33b7fe700 1 -- 172.21.15.99:0/3200862021 reap_dead start 2023-10-08T00:12:39.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.897+0000 7fb341a34700 1 -- 172.21.15.99:0/3200862021 shutdown_connections 2023-10-08T00:12:39.903 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.897+0000 7fb341a34700 1 -- 172.21.15.99:0/3200862021 >> 172.21.15.99:0/3200862021 conn(0x7fb33c068ce0 msgr2=0x7fb33c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:39.903 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.897+0000 7fb341a34700 1 -- 172.21.15.99:0/3200862021 shutdown_connections 2023-10-08T00:12:39.903 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.897+0000 7fb341a34700 1 -- 172.21.15.99:0/3200862021 wait complete. 2023-10-08T00:12:39.903 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.898+0000 7fb341a34700 1 Processor -- start 2023-10-08T00:12:39.903 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.898+0000 7fb341a34700 1 -- start start 2023-10-08T00:12:39.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.899+0000 7fb341a34700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb328004810 0x7fb33c1563c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:39.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.899+0000 7fb341a34700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb33c14e410 0x7fb33c14bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:39.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.899+0000 7fb341a34700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb33c14c160 0x7fb33c15d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:39.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.899+0000 7fb341a34700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c16ef40 con 0x7fb33c14c160 2023-10-08T00:12:39.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.899+0000 7fb341a34700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c156900 con 0x7fb33c14e410 2023-10-08T00:12:39.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.899+0000 7fb341a34700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c16e3c0 con 0x7fb328004810 2023-10-08T00:12:39.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.899+0000 7fb33b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb33c14c160 0x7fb33c15d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:39.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.899+0000 7fb33b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb33c14c160 0x7fb33c15d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46120/0 (socket says 172.21.15.99:46120) 2023-10-08T00:12:39.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.899+0000 7fb33b7fe700 1 -- 172.21.15.99:0/3033823691 learned_addr learned my addr 172.21.15.99:0/3033823691 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:39.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.899+0000 7fb33affd700 1 --2- 172.21.15.99:0/3033823691 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb328004810 0x7fb33c1563c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:39.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.899+0000 7fb3327fc700 1 --2- 172.21.15.99:0/3033823691 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb33c14e410 0x7fb33c14bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:39.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.900+0000 7fb33b7fe700 1 -- 172.21.15.99:0/3033823691 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb328004810 msgr2=0x7fb33c1563c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:39.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.900+0000 7fb33b7fe700 1 --2- 172.21.15.99:0/3033823691 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb328004810 0x7fb33c1563c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:39.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.900+0000 7fb33b7fe700 1 -- 172.21.15.99:0/3033823691 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb33c14e410 msgr2=0x7fb33c14bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:39.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.900+0000 7fb33b7fe700 1 --2- 172.21.15.99:0/3033823691 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb33c14e410 0x7fb33c14bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:39.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.900+0000 7fb33b7fe700 1 -- 172.21.15.99:0/3033823691 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb33c16e6a0 con 0x7fb33c14c160 2023-10-08T00:12:39.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.900+0000 7fb33b7fe700 1 --2- 172.21.15.99:0/3033823691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb33c14c160 0x7fb33c15d730 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb3340094d0 tx=0x7fb33400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:39.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.900+0000 7fb3337fe700 1 -- 172.21.15.99:0/3033823691 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33400e040 con 0x7fb33c14c160 2023-10-08T00:12:39.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.900+0000 7fb3337fe700 1 -- 172.21.15.99:0/3033823691 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb33400f040 con 0x7fb33c14c160 2023-10-08T00:12:39.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.900+0000 7fb3337fe700 1 -- 172.21.15.99:0/3033823691 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33400fd70 con 0x7fb33c14c160 2023-10-08T00:12:39.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.900+0000 7fb341a34700 1 -- 172.21.15.99:0/3033823691 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb33c16e980 con 0x7fb33c14c160 2023-10-08T00:12:39.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.900+0000 7fb341a34700 1 -- 172.21.15.99:0/3033823691 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb33c155d20 con 0x7fb33c14c160 2023-10-08T00:12:39.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.901+0000 7fb341a34700 1 -- 172.21.15.99:0/3033823691 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb33c04c3c0 con 0x7fb33c14c160 2023-10-08T00:12:39.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.901+0000 7fb3337fe700 1 -- 172.21.15.99:0/3033823691 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb334015640 con 0x7fb33c14c160 2023-10-08T00:12:39.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.902+0000 7fb3337fe700 1 --2- 172.21.15.99:0/3033823691 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb31c066f30 0x7fb31c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:39.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.902+0000 7fb3337fe700 1 -- 172.21.15.99:0/3033823691 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb334084b60 con 0x7fb33c14c160 2023-10-08T00:12:39.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.902+0000 7fb33affd700 1 --2- 172.21.15.99:0/3033823691 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb31c066f30 0x7fb31c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:39.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.902+0000 7fb33affd700 1 --2- 172.21.15.99:0/3033823691 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb31c066f30 0x7fb31c0693e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb32c006f00 tx=0x7fb32c0091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:39.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:39.906+0000 7fb3337fe700 1 -- 172.21.15.99:0/3033823691 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb334051dd0 con 0x7fb33c14c160 2023-10-08T00:12:40.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.090+0000 7fb341a34700 1 -- 172.21.15.99:0/3033823691 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fb33c15c840 con 0x7fb33c14c160 2023-10-08T00:12:40.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.091+0000 7fb3337fe700 1 -- 172.21.15.99:0/3033823691 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v54) v1 ==== 81+0+32340 (secure 0 0 0) 0x7fb3340553f0 con 0x7fb33c14c160 2023-10-08T00:12:40.092 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:12:40.092 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:12:40.093 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 54, 2023-10-08T00:12:40.093 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", 2023-10-08T00:12:40.093 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2023-10-08T00:10:36.303601+0000", 2023-10-08T00:12:40.093 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2023-10-08T00:12:37.124521+0000", 2023-10-08T00:12:40.093 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2023-10-08T00:10:41.709171+0000", 2023-10-08T00:12:40.093 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2023-10-08T00:10:38.690404+0000", 2023-10-08T00:12:40.093 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-08T00:12:40.093 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2023-10-08T00:12:40.093 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2023-10-08T00:12:40.093 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2023-10-08T00:12:40.093 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2023-10-08T00:12:40.093 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2023-10-08T00:12:40.094 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2023-10-08T00:12:40.094 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:12:40.094 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2023-10-08T00:12:40.094 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2023-10-08T00:12:40.094 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-08T00:12:40.094 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-08T00:12:40.094 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2023-10-08T00:12:40.094 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 7, 2023-10-08T00:12:40.094 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2023-10-08T00:12:40.094 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2023-10-08T00:12:40.094 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "jewel", 2023-10-08T00:12:40.094 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "quincy", 2023-10-08T00:12:40.095 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2023-10-08T00:12:40.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.095 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2023-10-08T00:12:40.095 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2023-10-08T00:12:40.095 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:10:42.063344+0000", 2023-10-08T00:12:40.095 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 4097, 2023-10-08T00:12:40.095 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,backfillfull", 2023-10-08T00:12:40.095 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:12:40.095 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:12:40.095 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:12:40.095 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:12:40.096 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:12:40.096 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:12:40.096 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:12:40.096 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:12:40.096 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:12:40.096 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:12:40.096 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2023-10-08T00:12:40.096 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2023-10-08T00:12:40.096 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2023-10-08T00:12:40.096 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2023-10-08T00:12:40.096 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2023-10-08T00:12:40.096 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:12:40.097 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:12:40.097 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:12:40.097 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:12:40.097 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:12:40.097 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:12:40.097 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:12:40.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.097 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "54", 2023-10-08T00:12:40.097 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:12:40.097 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:12:40.097 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:12:40.097 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:12:40.097 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:12:40.098 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:12:40.098 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:12:40.098 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:12:40.098 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:12:40.098 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:12:40.098 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:12:40.098 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:12:40.098 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:12:40.098 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:12:40.098 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:12:40.098 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:12:40.098 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:12:40.099 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:12:40.099 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:12:40.099 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:12:40.099 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:12:40.099 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:12:40.099 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:12:40.099 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:12:40.099 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:12:40.099 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:12:40.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.099 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:12:40.099 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:12:40.100 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:12:40.100 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:12:40.100 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:12:40.100 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:12:40.100 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:12:40.100 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:12:40.100 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:12:40.100 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:12:40.100 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:12:40.100 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:12:40.100 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2023-10-08T00:12:40.100 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2023-10-08T00:12:40.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.101 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:12:40.101 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2023-10-08T00:12:40.101 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.101 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 6, 2023-10-08T00:12:40.101 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2023-10-08T00:12:40.101 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:12:07.733432+0000", 2023-10-08T00:12:40.101 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 4097, 2023-10-08T00:12:40.101 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,backfillfull", 2023-10-08T00:12:40.102 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:12:40.102 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:12:40.102 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:12:40.102 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:12:40.102 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:12:40.102 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:12:40.102 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:12:40.102 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:12:40.102 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:12:40.102 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:12:40.103 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2023-10-08T00:12:40.103 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2023-10-08T00:12:40.103 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2023-10-08T00:12:40.103 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2023-10-08T00:12:40.103 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2023-10-08T00:12:40.103 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:12:40.103 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:12:40.103 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:12:40.103 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:12:40.103 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:12:40.103 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:12:40.103 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:12:40.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.104 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "54", 2023-10-08T00:12:40.104 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:12:40.104 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:12:40.104 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:12:40.104 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:12:40.104 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:12:40.104 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:12:40.104 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:12:40.104 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:12:40.104 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:12:40.104 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:12:40.105 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:12:40.105 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:12:40.105 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:12:40.105 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:12:40.105 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:12:40.105 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:12:40.105 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:12:40.105 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:12:40.105 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:12:40.105 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:12:40.105 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:12:40.105 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:12:40.106 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:12:40.106 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:12:40.106 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:12:40.106 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:12:40.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.106 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:12:40.106 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:12:40.106 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:12:40.106 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:12:40.106 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:12:40.106 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:12:40.106 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:12:40.106 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:12:40.107 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:12:40.107 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:12:40.107 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:12:40.107 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:12:40.107 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2023-10-08T00:12:40.107 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2023-10-08T00:12:40.107 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2023-10-08T00:12:40.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.107 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:12:40.107 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:12:40.107 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2023-10-08T00:12:40.108 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.108 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.108 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 7, 2023-10-08T00:12:40.108 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2023-10-08T00:12:40.108 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:12:08.762879+0000", 2023-10-08T00:12:40.108 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 4097, 2023-10-08T00:12:40.108 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,backfillfull", 2023-10-08T00:12:40.108 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:12:40.108 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:12:40.108 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:12:40.109 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:12:40.109 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:12:40.109 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:12:40.109 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:12:40.109 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:12:40.109 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:12:40.109 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:12:40.109 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2023-10-08T00:12:40.109 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2023-10-08T00:12:40.109 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2023-10-08T00:12:40.109 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2023-10-08T00:12:40.109 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2023-10-08T00:12:40.109 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:12:40.110 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:12:40.110 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:12:40.110 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:12:40.110 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:12:40.110 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:12:40.110 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:12:40.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.110 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "54", 2023-10-08T00:12:40.110 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:12:40.110 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:12:40.111 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:12:40.111 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:12:40.111 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:12:40.111 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:12:40.111 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:12:40.111 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:12:40.111 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:12:40.111 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:12:40.111 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:12:40.111 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:12:40.111 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:12:40.111 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:12:40.112 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:12:40.112 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:12:40.112 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:12:40.112 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:12:40.112 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:12:40.112 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:12:40.112 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:12:40.112 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:12:40.112 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:12:40.112 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:12:40.112 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:12:40.112 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:12:40.113 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.113 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:12:40.113 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:12:40.113 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:12:40.113 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:12:40.113 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:12:40.113 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:12:40.113 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:12:40.113 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:12:40.113 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:12:40.113 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:12:40.113 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:12:40.114 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:12:40.114 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2023-10-08T00:12:40.114 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2023-10-08T00:12:40.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.114 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:12:40.114 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:12:40.114 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2023-10-08T00:12:40.114 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.114 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.114 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.114 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:12:40.114 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2023-10-08T00:12:40.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.115 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:12:40.115 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:40.115 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:40.115 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:40.115 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:40.115 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:40.115 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:40.115 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:40.115 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:40.115 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:40.116 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:40.116 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:40.116 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:40.116 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.116 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.116 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2023-10-08T00:12:40.116 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:40.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.116 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.116 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2023-10-08T00:12:40.117 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:40.117 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.117 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.117 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:40.117 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.117 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.117 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.117 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2023-10-08T00:12:40.117 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:40.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.117 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.117 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.118 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2023-10-08T00:12:40.118 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:40.118 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.118 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.118 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.118 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:40.118 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.118 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.118 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.118 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2023-10-08T00:12:40.118 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:40.118 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.119 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.119 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.119 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2023-10-08T00:12:40.119 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:40.119 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.119 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.119 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.119 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:40.119 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.119 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.119 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.119 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2023-10-08T00:12:40.120 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:40.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.120 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.120 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.120 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2023-10-08T00:12:40.120 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:40.120 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.120 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.120 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/86365", 2023-10-08T00:12:40.120 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/86365", 2023-10-08T00:12:40.120 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/86365", 2023-10-08T00:12:40.121 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/86365", 2023-10-08T00:12:40.121 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:40.121 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:40.121 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:40.121 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.121 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.121 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.121 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:12:40.121 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:40.121 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:40.121 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:40.121 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:40.122 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:40.122 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:40.122 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:40.122 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:40.122 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:40.122 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:40.122 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:40.122 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:40.122 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.122 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.122 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.122 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2023-10-08T00:12:40.123 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:40.123 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.123 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.123 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.123 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2023-10-08T00:12:40.123 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:40.123 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.123 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.123 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.123 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:40.123 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.123 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.124 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.124 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2023-10-08T00:12:40.124 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:40.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.124 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.124 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.124 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2023-10-08T00:12:40.124 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:40.124 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.124 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.124 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:40.125 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.125 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.125 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.125 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2023-10-08T00:12:40.125 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:40.125 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.125 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.125 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.125 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2023-10-08T00:12:40.125 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:40.126 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.126 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.126 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.126 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:40.126 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.126 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.126 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.126 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2023-10-08T00:12:40.126 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:40.126 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.126 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.126 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.127 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2023-10-08T00:12:40.127 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:40.127 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.127 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.127 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.127 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/86366", 2023-10-08T00:12:40.127 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/86366", 2023-10-08T00:12:40.127 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/86366", 2023-10-08T00:12:40.127 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/86366", 2023-10-08T00:12:40.127 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:40.127 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:40.127 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:40.128 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.128 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.128 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:12:40.128 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:40.128 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:40.128 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:40.128 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:40.128 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:40.128 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:40.128 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:40.128 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:40.129 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:40.129 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:40.129 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:40.129 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:40.129 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.129 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.129 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.129 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2023-10-08T00:12:40.129 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:40.129 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.129 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.129 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.130 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2023-10-08T00:12:40.130 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:40.130 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.130 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.130 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.130 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:40.130 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.130 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.130 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.130 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2023-10-08T00:12:40.130 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:40.131 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.131 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.131 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.131 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2023-10-08T00:12:40.131 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:40.131 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.131 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.131 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.131 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:40.131 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.131 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.131 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.132 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2023-10-08T00:12:40.132 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:40.132 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.132 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.132 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.132 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2023-10-08T00:12:40.132 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:40.132 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.132 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.132 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.132 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:40.132 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.133 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.133 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.133 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2023-10-08T00:12:40.133 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:40.133 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.133 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.133 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.133 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2023-10-08T00:12:40.133 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:40.133 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.133 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.133 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.134 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6817/86382", 2023-10-08T00:12:40.134 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/86382", 2023-10-08T00:12:40.134 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/86382", 2023-10-08T00:12:40.134 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/86382", 2023-10-08T00:12:40.134 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:40.134 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:40.134 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:40.134 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.134 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.134 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.134 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:12:40.134 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:40.135 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:40.135 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:40.135 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:40.135 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:40.135 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:40.135 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:40.135 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:40.135 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:40.135 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:40.135 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:40.135 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:40.135 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.136 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.136 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.136 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2023-10-08T00:12:40.136 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:40.136 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.136 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.136 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.136 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2023-10-08T00:12:40.136 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:40.136 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.136 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.136 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.137 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:40.137 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.137 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.137 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.137 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2023-10-08T00:12:40.137 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:40.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.137 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.137 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.137 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2023-10-08T00:12:40.137 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:40.137 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.138 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.138 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:40.138 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.138 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.138 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2023-10-08T00:12:40.138 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:40.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.138 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.138 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2023-10-08T00:12:40.139 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:40.139 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.139 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.139 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:40.139 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.139 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.139 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2023-10-08T00:12:40.139 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:40.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.140 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.140 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2023-10-08T00:12:40.140 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:40.140 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.140 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.140 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6820/86393", 2023-10-08T00:12:40.140 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/86393", 2023-10-08T00:12:40.140 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/86393", 2023-10-08T00:12:40.140 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/86393", 2023-10-08T00:12:40.140 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:40.140 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:40.141 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:40.141 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.141 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.141 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:12:40.141 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:40.141 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:40.141 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:40.141 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:40.141 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:40.141 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:40.141 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:40.142 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:40.142 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:40.142 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:40.142 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:40.142 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:40.142 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.142 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.142 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6800", 2023-10-08T00:12:40.142 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:40.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.143 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.143 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6801", 2023-10-08T00:12:40.143 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:40.143 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.143 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.143 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.143 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:40.143 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.143 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.143 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.143 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6802", 2023-10-08T00:12:40.144 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:40.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.144 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.144 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.144 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6803", 2023-10-08T00:12:40.144 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:40.144 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.144 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.144 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:40.144 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.144 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.145 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.145 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6806", 2023-10-08T00:12:40.145 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:40.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.145 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.145 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6807", 2023-10-08T00:12:40.145 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:40.145 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.145 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.145 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:40.146 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.146 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.146 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6804", 2023-10-08T00:12:40.146 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:40.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.146 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.146 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6805", 2023-10-08T00:12:40.146 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:40.146 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.146 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.147 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6801/86570", 2023-10-08T00:12:40.147 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6803/86570", 2023-10-08T00:12:40.147 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6807/86570", 2023-10-08T00:12:40.147 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6805/86570", 2023-10-08T00:12:40.147 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:40.147 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:40.147 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:40.147 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.147 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.147 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:12:40.148 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:40.148 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:40.148 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:40.148 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:40.148 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:40.148 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:40.148 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:40.148 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:40.148 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:40.148 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:40.148 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:40.148 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:40.149 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.149 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.149 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.149 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6816", 2023-10-08T00:12:40.149 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:40.149 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.149 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.149 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.149 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6818", 2023-10-08T00:12:40.149 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:40.149 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.149 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.150 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:40.150 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.150 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.150 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6820", 2023-10-08T00:12:40.150 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:40.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.150 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.150 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6821", 2023-10-08T00:12:40.150 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:40.151 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.151 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.151 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:40.151 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.151 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.151 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6826", 2023-10-08T00:12:40.151 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:40.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.151 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.152 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6828", 2023-10-08T00:12:40.152 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:40.152 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.152 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.152 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:40.152 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.152 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.152 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6823", 2023-10-08T00:12:40.152 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:40.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.153 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.153 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.153 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6825", 2023-10-08T00:12:40.153 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:40.153 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.153 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.153 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6818/86608", 2023-10-08T00:12:40.153 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6821/86608", 2023-10-08T00:12:40.153 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6828/86608", 2023-10-08T00:12:40.153 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6825/86608", 2023-10-08T00:12:40.153 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:40.154 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:40.154 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:40.154 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.154 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.154 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.154 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:12:40.154 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:40.154 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:40.154 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:40.154 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:40.154 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:40.154 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:40.154 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:40.155 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:40.155 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:40.155 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:40.155 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:40.155 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:40.155 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.155 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.155 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6808", 2023-10-08T00:12:40.155 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:40.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.155 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.156 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6809", 2023-10-08T00:12:40.156 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:40.156 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.156 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.156 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:40.156 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.156 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.156 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.156 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6810", 2023-10-08T00:12:40.156 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:40.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.156 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.156 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.157 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6811", 2023-10-08T00:12:40.157 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:40.157 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.157 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.158 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:40.158 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.158 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.158 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6814", 2023-10-08T00:12:40.158 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:40.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.158 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.158 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6815", 2023-10-08T00:12:40.159 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:40.159 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.159 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.159 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:40.159 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.159 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.159 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.159 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6812", 2023-10-08T00:12:40.159 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:40.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.159 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.160 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.160 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6813", 2023-10-08T00:12:40.160 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:40.160 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.160 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.160 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6809/86571", 2023-10-08T00:12:40.160 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6811/86571", 2023-10-08T00:12:40.160 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6815/86571", 2023-10-08T00:12:40.160 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6813/86571", 2023-10-08T00:12:40.160 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:40.160 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:40.161 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:40.161 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.161 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:12:40.161 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:40.161 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:40.161 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:40.161 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:40.161 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:40.161 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:40.161 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:40.162 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:40.162 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:40.162 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:40.162 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:40.162 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:40.162 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.162 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.162 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6817", 2023-10-08T00:12:40.162 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:40.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.163 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.163 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6819", 2023-10-08T00:12:40.163 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:40.163 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.163 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.163 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:40.163 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.163 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.163 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6822", 2023-10-08T00:12:40.163 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:40.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.164 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.164 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6824", 2023-10-08T00:12:40.164 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:40.164 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.164 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.164 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:40.164 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.164 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.164 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6830", 2023-10-08T00:12:40.165 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:40.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.165 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.165 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6831", 2023-10-08T00:12:40.165 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:40.165 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.165 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.165 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:40.165 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.166 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.166 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6827", 2023-10-08T00:12:40.166 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:40.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.166 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.166 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6829", 2023-10-08T00:12:40.166 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:40.166 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.166 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.166 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6819/86612", 2023-10-08T00:12:40.167 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6824/86612", 2023-10-08T00:12:40.167 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6831/86612", 2023-10-08T00:12:40.167 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6829/86612", 2023-10-08T00:12:40.167 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:40.167 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull", 2023-10-08T00:12:40.167 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:40.167 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:40.167 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.167 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.167 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:12:40.167 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2023-10-08T00:12:40.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.168 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:12:40.168 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:40.168 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:40.168 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:40.168 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:40.168 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:40.168 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.905615+0000", 2023-10-08T00:12:40.168 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:40.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.168 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:12:40.168 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:40.169 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:40.169 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:40.169 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:40.169 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:40.169 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.880709+0000", 2023-10-08T00:12:40.169 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:40.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.169 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:12:40.169 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:40.169 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:40.169 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:40.170 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:40.170 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:40.170 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.874704+0000", 2023-10-08T00:12:40.170 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:40.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.170 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:12:40.170 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:40.170 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:40.170 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:40.170 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:40.171 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:40.171 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.899604+0000", 2023-10-08T00:12:40.171 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:40.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.171 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.171 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:12:40.171 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:40.171 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:40.171 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:40.171 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:40.171 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:40.171 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.844371+0000", 2023-10-08T00:12:40.172 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:40.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.172 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:12:40.172 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:40.172 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:40.172 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:40.172 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:40.172 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:40.172 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.794810+0000", 2023-10-08T00:12:40.172 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:40.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.173 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:12:40.173 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:40.173 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:40.173 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:40.173 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:40.173 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:40.173 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.859474+0000", 2023-10-08T00:12:40.173 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:40.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.173 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:12:40.174 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:40.174 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:40.174 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:40.174 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:40.174 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:40.174 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.893130+0000", 2023-10-08T00:12:40.174 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:40.174 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.174 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:12:40.174 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2023-10-08T00:12:40.174 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [], 2023-10-08T00:12:40.174 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2023-10-08T00:12:40.175 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2023-10-08T00:12:40.175 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2023-10-08T00:12:40.175 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2023-10-08T00:12:40.175 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2023-10-08T00:12:40.175 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2023-10-08T00:12:40.175 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2023-10-08T00:12:40.175 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2023-10-08T00:12:40.175 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2023-10-08T00:12:40.175 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2023-10-08T00:12:40.175 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2023-10-08T00:12:40.175 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.176 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2023-10-08T00:12:40.176 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2023-10-08T00:12:40.176 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2023-10-08T00:12:40.176 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2023-10-08T00:12:40.176 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2023-10-08T00:12:40.176 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2023-10-08T00:12:40.176 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2023-10-08T00:12:40.176 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2023-10-08T00:12:40.176 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2023-10-08T00:12:40.176 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2023-10-08T00:12:40.176 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2023-10-08T00:12:40.176 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.177 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:12:40.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.093+0000 7fb341a34700 1 -- 172.21.15.99:0/3033823691 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb31c066f30 msgr2=0x7fb31c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:40.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.093+0000 7fb341a34700 1 --2- 172.21.15.99:0/3033823691 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb31c066f30 0x7fb31c0693e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb32c006f00 tx=0x7fb32c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:12:40.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.093+0000 7fb341a34700 1 -- 172.21.15.99:0/3033823691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb33c14c160 msgr2=0x7fb33c15d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:40.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.093+0000 7fb341a34700 1 --2- 172.21.15.99:0/3033823691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb33c14c160 0x7fb33c15d730 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb3340094d0 tx=0x7fb33400b040 comp rx=0 tx=0).stop 2023-10-08T00:12:40.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.093+0000 7fb341a34700 1 -- 172.21.15.99:0/3033823691 shutdown_connections 2023-10-08T00:12:40.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.093+0000 7fb341a34700 1 --2- 172.21.15.99:0/3033823691 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb31c066f30 0x7fb31c0693e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:40.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.093+0000 7fb341a34700 1 --2- 172.21.15.99:0/3033823691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb33c14c160 0x7fb33c15d730 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:40.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.093+0000 7fb341a34700 1 --2- 172.21.15.99:0/3033823691 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb328004810 0x7fb33c1563c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:40.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.093+0000 7fb341a34700 1 --2- 172.21.15.99:0/3033823691 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb33c14e410 0x7fb33c14bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:40.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.093+0000 7fb341a34700 1 -- 172.21.15.99:0/3033823691 >> 172.21.15.99:0/3033823691 conn(0x7fb33c068ce0 msgr2=0x7fb33c0b46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:40.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.093+0000 7fb341a34700 1 -- 172.21.15.99:0/3033823691 shutdown_connections 2023-10-08T00:12:40.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.093+0000 7fb341a34700 1 -- 172.21.15.99:0/3033823691 wait complete. 2023-10-08T00:12:40.180 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2023-10-08T00:12:40.180 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-08T00:12:40.294 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.294+0000 7fd76bd06700 1 Processor -- start 2023-10-08T00:12:40.295 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.294+0000 7fd76bd06700 1 -- start start 2023-10-08T00:12:40.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.295+0000 7fd76bd06700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd764157870 0x7fd76415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:40.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.295+0000 7fd76bd06700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd764157c40 0x7fd764158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:40.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.295+0000 7fd76bd06700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd76415cbb0 con 0x7fd76415a620 2023-10-08T00:12:40.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.295+0000 7fd76bd06700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd76415bfa0 con 0x7fd764157870 2023-10-08T00:12:40.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.295+0000 7fd76bd06700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd76415c280 con 0x7fd764157c40 2023-10-08T00:12:40.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.295+0000 7fd76a2a3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd764157c40 0x7fd764158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:40.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.295+0000 7fd769aa2700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd764157870 0x7fd76415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:40.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.295+0000 7fd76a2a3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd764157c40 0x7fd764158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.99:39094/0 (socket says 172.21.15.99:39094) 2023-10-08T00:12:40.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.295+0000 7fd769aa2700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd764157870 0x7fd76415b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.99:56164/0 (socket says 172.21.15.99:56164) 2023-10-08T00:12:40.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.295+0000 7fd76a2a3700 1 -- 172.21.15.99:0/653431812 learned_addr learned my addr 172.21.15.99:0/653431812 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:40.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.295+0000 7fd769aa2700 1 -- 172.21.15.99:0/653431812 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd764157c40 msgr2=0x7fd764158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:40.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.295+0000 7fd769aa2700 1 --2- 172.21.15.99:0/653431812 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd764157c40 0x7fd764158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:40.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.295+0000 7fd769aa2700 1 -- 172.21.15.99:0/653431812 >> v1:172.21.15.191:6789/0 conn(0x7fd76415a620 legacy=0x7fd76415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:40.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.295+0000 7fd769aa2700 1 -- 172.21.15.99:0/653431812 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd76415c560 con 0x7fd764157870 2023-10-08T00:12:40.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.295+0000 7fd76a2a3700 1 --2- 172.21.15.99:0/653431812 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd764157c40 0x7fd764158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:12:40.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.296+0000 7fd769aa2700 1 --2- 172.21.15.99:0/653431812 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd764157870 0x7fd76415b6a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd7600098e0 tx=0x7fd7600091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce57b8d289074f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:40.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.296+0000 7fd768aa0700 1 -- 172.21.15.99:0/653431812 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76000b640 con 0x7fd764157870 2023-10-08T00:12:40.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.296+0000 7fd768aa0700 1 -- 172.21.15.99:0/653431812 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd76000b7a0 con 0x7fd764157870 2023-10-08T00:12:40.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.296+0000 7fd768aa0700 1 -- 172.21.15.99:0/653431812 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76000bb40 con 0x7fd764157870 2023-10-08T00:12:40.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.296+0000 7fd76bd06700 1 -- 172.21.15.99:0/653431812 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd764157870 msgr2=0x7fd76415b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:40.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.296+0000 7fd76bd06700 1 --2- 172.21.15.99:0/653431812 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd764157870 0x7fd76415b6a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd7600098e0 tx=0x7fd7600091a0 comp rx=0 tx=0).stop 2023-10-08T00:12:40.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.296+0000 7fd76bd06700 1 -- 172.21.15.99:0/653431812 shutdown_connections 2023-10-08T00:12:40.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.296+0000 7fd76bd06700 1 --2- 172.21.15.99:0/653431812 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd764157c40 0x7fd764158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:40.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.296+0000 7fd76bd06700 1 --2- 172.21.15.99:0/653431812 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd764157870 0x7fd76415b6a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:40.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.296+0000 7fd76bd06700 1 -- 172.21.15.99:0/653431812 >> 172.21.15.99:0/653431812 conn(0x7fd764068ce0 msgr2=0x7fd7640690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:40.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.296+0000 7fd76bd06700 1 -- 172.21.15.99:0/653431812 shutdown_connections 2023-10-08T00:12:40.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.296+0000 7fd76bd06700 1 -- 172.21.15.99:0/653431812 wait complete. 2023-10-08T00:12:40.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.297+0000 7fd76bd06700 1 Processor -- start 2023-10-08T00:12:40.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.297+0000 7fd76bd06700 1 -- start start 2023-10-08T00:12:40.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.298+0000 7fd76bd06700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd764157870 0x7fd7641556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:40.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.298+0000 7fd76bd06700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd764157c40 0x7fd76416e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:40.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.298+0000 7fd76bd06700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd76415a620 0x7fd76415e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:40.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.298+0000 7fd76bd06700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd76413d6c0 con 0x7fd76415a620 2023-10-08T00:12:40.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.298+0000 7fd76bd06700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd76416eb10 con 0x7fd764157c40 2023-10-08T00:12:40.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.298+0000 7fd76bd06700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7640b2300 con 0x7fd764157870 2023-10-08T00:12:40.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.298+0000 7fd76a2a3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd76415a620 0x7fd76415e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:40.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.298+0000 7fd76a2a3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd76415a620 0x7fd76415e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46122/0 (socket says 172.21.15.99:46122) 2023-10-08T00:12:40.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.298+0000 7fd76a2a3700 1 -- 172.21.15.99:0/2229724717 learned_addr learned my addr 172.21.15.99:0/2229724717 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:40.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.298+0000 7fd769aa2700 1 --2- 172.21.15.99:0/2229724717 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd764157870 0x7fd7641556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:40.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.299+0000 7fd7692a1700 1 --2- 172.21.15.99:0/2229724717 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd764157c40 0x7fd76416e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:40.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.299+0000 7fd76a2a3700 1 -- 172.21.15.99:0/2229724717 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd764157870 msgr2=0x7fd7641556d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:40.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.299+0000 7fd76a2a3700 1 --2- 172.21.15.99:0/2229724717 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd764157870 0x7fd7641556d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:40.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.299+0000 7fd76a2a3700 1 -- 172.21.15.99:0/2229724717 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd764157c40 msgr2=0x7fd76416e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:40.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.299+0000 7fd76a2a3700 1 --2- 172.21.15.99:0/2229724717 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd764157c40 0x7fd76416e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:40.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.299+0000 7fd76a2a3700 1 -- 172.21.15.99:0/2229724717 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd75c009ae0 con 0x7fd76415a620 2023-10-08T00:12:40.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.299+0000 7fd76a2a3700 1 --2- 172.21.15.99:0/2229724717 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd76415a620 0x7fd76415e560 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd75c009fa0 tx=0x7fd75c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:40.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.299+0000 7fd759ffb700 1 -- 172.21.15.99:0/2229724717 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75c018070 con 0x7fd76415a620 2023-10-08T00:12:40.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.299+0000 7fd759ffb700 1 -- 172.21.15.99:0/2229724717 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd75c00f040 con 0x7fd76415a620 2023-10-08T00:12:40.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.299+0000 7fd759ffb700 1 -- 172.21.15.99:0/2229724717 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75c0092e0 con 0x7fd76415a620 2023-10-08T00:12:40.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.299+0000 7fd76bd06700 1 -- 172.21.15.99:0/2229724717 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd760009420 con 0x7fd76415a620 2023-10-08T00:12:40.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.299+0000 7fd76bd06700 1 -- 172.21.15.99:0/2229724717 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7640b25a0 con 0x7fd76415a620 2023-10-08T00:12:40.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.300+0000 7fd76bd06700 1 -- 172.21.15.99:0/2229724717 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd76404c3c0 con 0x7fd76415a620 2023-10-08T00:12:40.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.300+0000 7fd759ffb700 1 -- 172.21.15.99:0/2229724717 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd75c003020 con 0x7fd76415a620 2023-10-08T00:12:40.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.301+0000 7fd759ffb700 1 --2- 172.21.15.99:0/2229724717 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd748066f30 0x7fd7480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:40.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.301+0000 7fd759ffb700 1 -- 172.21.15.99:0/2229724717 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd75c084a40 con 0x7fd76415a620 2023-10-08T00:12:40.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.301+0000 7fd769aa2700 1 --2- 172.21.15.99:0/2229724717 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd748066f30 0x7fd7480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:40.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.301+0000 7fd769aa2700 1 --2- 172.21.15.99:0/2229724717 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd748066f30 0x7fd7480693e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd7600093f0 tx=0x7fd760017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:40.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.305+0000 7fd759ffb700 1 -- 172.21.15.99:0/2229724717 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd75c086e60 con 0x7fd76415a620 2023-10-08T00:12:40.488 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.487+0000 7fd76bd06700 1 -- 172.21.15.99:0/2229724717 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fd76413d800 con 0x7fd76415a620 2023-10-08T00:12:40.489 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.488+0000 7fd759ffb700 1 -- 172.21.15.99:0/2229724717 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v55) v1 ==== 81+0+32260 (secure 0 0 0) 0x7fd75c051cb0 con 0x7fd76415a620 2023-10-08T00:12:40.489 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:12:40.490 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:12:40.490 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 55, 2023-10-08T00:12:40.490 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", 2023-10-08T00:12:40.490 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2023-10-08T00:10:36.303601+0000", 2023-10-08T00:12:40.490 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2023-10-08T00:12:40.131587+0000", 2023-10-08T00:12:40.490 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2023-10-08T00:10:41.709171+0000", 2023-10-08T00:12:40.490 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2023-10-08T00:10:38.690404+0000", 2023-10-08T00:12:40.490 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-08T00:12:40.490 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2023-10-08T00:12:40.490 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2023-10-08T00:12:40.491 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2023-10-08T00:12:40.491 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2023-10-08T00:12:40.491 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2023-10-08T00:12:40.491 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2023-10-08T00:12:40.491 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:12:40.491 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2023-10-08T00:12:40.491 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2023-10-08T00:12:40.492 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-08T00:12:40.492 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-08T00:12:40.492 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2023-10-08T00:12:40.492 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 7, 2023-10-08T00:12:40.492 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2023-10-08T00:12:40.492 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2023-10-08T00:12:40.493 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "jewel", 2023-10-08T00:12:40.493 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "quincy", 2023-10-08T00:12:40.493 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2023-10-08T00:12:40.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.493 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2023-10-08T00:12:40.493 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2023-10-08T00:12:40.494 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:10:42.063344+0000", 2023-10-08T00:12:40.494 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:12:40.494 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:12:40.494 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:12:40.495 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:12:40.495 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:12:40.495 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:12:40.495 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:12:40.495 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:12:40.495 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:12:40.495 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:12:40.495 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:12:40.495 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:12:40.496 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2023-10-08T00:12:40.496 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2023-10-08T00:12:40.496 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2023-10-08T00:12:40.496 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2023-10-08T00:12:40.496 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2023-10-08T00:12:40.496 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:12:40.496 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:12:40.496 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:12:40.497 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:12:40.497 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:12:40.497 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:12:40.497 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:12:40.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.497 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "55", 2023-10-08T00:12:40.497 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:12:40.497 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:12:40.497 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:12:40.497 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:12:40.497 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:12:40.498 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:12:40.498 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:12:40.498 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:12:40.498 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:12:40.498 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:12:40.498 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:12:40.498 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:12:40.498 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:12:40.498 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:12:40.498 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:12:40.499 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:12:40.499 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:12:40.499 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:12:40.499 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:12:40.499 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:12:40.499 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:12:40.499 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:12:40.499 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:12:40.499 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:12:40.499 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:12:40.500 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:12:40.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.500 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:12:40.500 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:12:40.500 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:12:40.500 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:12:40.500 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:12:40.500 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:12:40.500 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:12:40.500 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:12:40.501 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:12:40.501 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:12:40.501 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:12:40.501 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:12:40.501 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2023-10-08T00:12:40.501 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2023-10-08T00:12:40.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.501 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:12:40.501 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2023-10-08T00:12:40.502 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.502 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 6, 2023-10-08T00:12:40.502 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2023-10-08T00:12:40.502 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:12:07.733432+0000", 2023-10-08T00:12:40.502 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:12:40.502 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:12:40.502 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:12:40.503 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:12:40.503 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:12:40.503 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:12:40.503 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:12:40.503 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:12:40.503 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:12:40.503 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:12:40.503 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:12:40.503 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:12:40.503 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2023-10-08T00:12:40.503 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2023-10-08T00:12:40.504 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2023-10-08T00:12:40.504 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2023-10-08T00:12:40.504 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2023-10-08T00:12:40.504 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:12:40.504 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:12:40.504 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:12:40.504 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:12:40.504 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:12:40.504 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:12:40.504 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:12:40.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.505 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "55", 2023-10-08T00:12:40.505 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:12:40.505 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:12:40.505 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:12:40.505 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:12:40.506 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:12:40.506 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:12:40.506 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:12:40.506 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:12:40.506 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:12:40.506 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:12:40.506 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:12:40.506 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:12:40.507 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:12:40.507 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:12:40.507 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:12:40.508 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:12:40.508 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:12:40.508 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:12:40.508 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:12:40.508 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:12:40.508 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:12:40.508 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:12:40.508 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:12:40.508 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:12:40.508 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:12:40.509 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:12:40.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.509 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:12:40.509 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:12:40.509 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:12:40.509 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:12:40.509 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:12:40.509 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:12:40.509 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:12:40.510 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:12:40.510 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:12:40.510 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:12:40.510 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:12:40.510 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:12:40.510 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2023-10-08T00:12:40.510 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2023-10-08T00:12:40.511 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2023-10-08T00:12:40.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.511 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:12:40.511 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:12:40.511 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2023-10-08T00:12:40.512 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.512 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.512 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 7, 2023-10-08T00:12:40.513 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2023-10-08T00:12:40.513 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:12:08.762879+0000", 2023-10-08T00:12:40.513 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:12:40.513 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:12:40.513 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:12:40.513 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:12:40.513 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:12:40.513 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:12:40.513 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:12:40.513 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:12:40.513 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:12:40.513 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:12:40.514 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:12:40.514 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:12:40.514 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2023-10-08T00:12:40.514 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2023-10-08T00:12:40.514 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2023-10-08T00:12:40.514 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2023-10-08T00:12:40.514 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2023-10-08T00:12:40.514 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:12:40.514 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:12:40.514 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:12:40.514 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:12:40.515 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:12:40.515 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:12:40.515 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:12:40.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.515 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "55", 2023-10-08T00:12:40.515 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:12:40.515 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:12:40.515 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:12:40.515 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:12:40.516 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:12:40.516 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:12:40.516 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:12:40.516 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:12:40.516 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:12:40.516 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:12:40.517 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:12:40.517 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:12:40.517 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:12:40.517 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:12:40.517 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:12:40.517 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:12:40.517 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:12:40.517 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:12:40.518 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:12:40.518 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:12:40.518 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:12:40.518 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:12:40.518 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:12:40.518 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:12:40.518 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:12:40.518 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:12:40.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.519 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:12:40.519 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:12:40.519 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:12:40.519 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:12:40.519 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:12:40.519 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:12:40.519 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:12:40.519 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:12:40.519 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:12:40.519 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:12:40.519 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:12:40.519 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:12:40.520 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2023-10-08T00:12:40.520 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2023-10-08T00:12:40.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.520 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:12:40.521 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:12:40.521 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2023-10-08T00:12:40.521 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.521 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.521 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.521 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:12:40.521 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2023-10-08T00:12:40.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.521 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:12:40.521 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:40.521 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:40.521 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:40.522 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:40.522 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:40.522 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:40.522 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:40.522 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:40.522 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:40.522 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:40.522 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:40.522 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:40.522 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.523 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.523 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2023-10-08T00:12:40.523 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:40.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.523 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.523 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2023-10-08T00:12:40.523 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:40.523 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.523 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.524 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:40.524 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.524 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.524 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2023-10-08T00:12:40.524 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:40.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.525 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.525 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2023-10-08T00:12:40.525 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:40.525 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.526 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.526 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:40.526 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.526 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.526 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2023-10-08T00:12:40.526 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:40.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.527 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.527 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2023-10-08T00:12:40.527 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:40.527 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.527 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.527 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:40.527 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.528 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.528 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2023-10-08T00:12:40.528 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:40.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.528 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.528 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2023-10-08T00:12:40.528 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:12:40.528 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.528 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.529 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/86365", 2023-10-08T00:12:40.529 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/86365", 2023-10-08T00:12:40.529 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/86365", 2023-10-08T00:12:40.529 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/86365", 2023-10-08T00:12:40.529 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:40.529 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:40.529 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:40.529 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.529 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:12:40.530 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:40.530 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:40.530 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:40.530 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:40.530 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:40.530 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:40.530 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:40.530 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:40.530 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:40.530 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:40.530 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:40.531 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:40.531 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.531 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.531 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2023-10-08T00:12:40.531 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:40.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.531 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.531 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2023-10-08T00:12:40.531 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:40.532 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.532 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.532 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:40.532 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.532 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.532 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2023-10-08T00:12:40.532 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:40.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.532 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.533 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2023-10-08T00:12:40.533 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:40.533 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.533 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.533 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:40.533 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.533 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.533 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2023-10-08T00:12:40.534 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:40.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.534 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.534 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2023-10-08T00:12:40.534 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:40.534 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.534 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.534 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:40.535 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.535 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.535 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2023-10-08T00:12:40.535 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:40.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.535 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.535 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2023-10-08T00:12:40.535 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:12:40.535 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.536 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.536 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/86366", 2023-10-08T00:12:40.536 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/86366", 2023-10-08T00:12:40.536 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/86366", 2023-10-08T00:12:40.536 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/86366", 2023-10-08T00:12:40.536 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:40.536 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:40.536 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:40.536 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.537 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:12:40.537 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:40.537 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:40.537 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:40.537 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:40.537 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:40.537 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:40.537 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:40.537 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:40.537 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:40.538 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:40.538 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:40.538 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:40.538 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.538 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.538 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2023-10-08T00:12:40.538 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:40.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.538 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.539 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2023-10-08T00:12:40.539 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:40.539 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.539 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.539 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:40.539 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.539 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.539 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2023-10-08T00:12:40.540 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:40.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.540 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.540 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2023-10-08T00:12:40.540 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:40.540 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.540 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.541 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:40.541 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.541 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.541 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2023-10-08T00:12:40.541 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:40.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.541 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.541 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2023-10-08T00:12:40.542 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:40.542 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.542 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.542 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:40.542 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.542 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.542 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2023-10-08T00:12:40.542 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:40.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.543 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.543 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2023-10-08T00:12:40.543 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:12:40.543 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.543 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.543 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6817/86382", 2023-10-08T00:12:40.543 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/86382", 2023-10-08T00:12:40.543 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/86382", 2023-10-08T00:12:40.543 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/86382", 2023-10-08T00:12:40.543 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:40.544 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:40.544 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:40.544 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.544 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:12:40.544 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:40.544 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:40.544 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:40.544 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:40.545 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:40.545 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:40.545 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:40.545 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:40.545 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:40.545 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:40.545 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:40.545 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:40.545 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.545 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.546 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2023-10-08T00:12:40.546 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:40.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.546 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.546 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2023-10-08T00:12:40.546 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:40.546 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.546 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.546 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:40.547 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.547 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.547 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2023-10-08T00:12:40.547 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:40.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.547 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.547 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2023-10-08T00:12:40.547 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:40.547 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.547 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.548 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:40.548 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.548 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.548 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2023-10-08T00:12:40.548 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:40.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.548 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.548 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2023-10-08T00:12:40.548 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:40.548 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.549 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.549 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:40.549 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.549 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.549 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2023-10-08T00:12:40.549 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:40.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.549 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.549 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2023-10-08T00:12:40.550 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:12:40.550 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.550 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.550 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6820/86393", 2023-10-08T00:12:40.550 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/86393", 2023-10-08T00:12:40.550 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/86393", 2023-10-08T00:12:40.550 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/86393", 2023-10-08T00:12:40.550 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:40.550 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:40.550 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:40.550 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.551 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:12:40.551 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:40.551 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:40.551 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:40.551 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:40.551 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:40.551 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:40.551 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:40.551 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:40.551 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:40.551 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:40.552 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:40.552 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:40.552 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.552 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.552 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6800", 2023-10-08T00:12:40.552 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:40.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.552 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.552 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6801", 2023-10-08T00:12:40.552 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:40.553 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.553 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.553 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:40.553 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.553 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.553 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6802", 2023-10-08T00:12:40.553 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:40.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.553 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.553 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6803", 2023-10-08T00:12:40.554 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:40.554 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.554 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.554 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:40.554 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.554 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.554 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6806", 2023-10-08T00:12:40.554 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:40.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.555 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.555 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6807", 2023-10-08T00:12:40.555 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:40.555 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.555 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.555 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:40.555 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.555 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.555 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6804", 2023-10-08T00:12:40.555 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:40.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.556 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.556 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6805", 2023-10-08T00:12:40.556 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:12:40.556 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.556 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.556 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6801/86570", 2023-10-08T00:12:40.556 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6803/86570", 2023-10-08T00:12:40.556 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6807/86570", 2023-10-08T00:12:40.556 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6805/86570", 2023-10-08T00:12:40.557 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:40.557 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:40.557 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:40.557 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.557 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:12:40.557 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:40.557 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:40.557 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:40.557 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:40.557 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:40.557 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:40.558 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:40.558 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:40.558 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:40.558 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:40.558 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:40.558 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:40.558 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.558 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.558 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6816", 2023-10-08T00:12:40.558 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:40.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.559 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.559 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6818", 2023-10-08T00:12:40.559 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:40.559 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.559 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.559 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:40.559 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.560 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.560 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6820", 2023-10-08T00:12:40.560 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:40.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.560 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.560 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6821", 2023-10-08T00:12:40.560 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:40.560 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.560 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.561 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:40.561 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.561 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.561 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6826", 2023-10-08T00:12:40.561 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:40.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.561 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.561 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6828", 2023-10-08T00:12:40.561 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:40.562 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.562 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.562 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:40.562 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.562 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.562 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6823", 2023-10-08T00:12:40.562 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:40.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.563 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.563 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6825", 2023-10-08T00:12:40.563 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:12:40.563 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.563 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.563 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6818/86608", 2023-10-08T00:12:40.564 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6821/86608", 2023-10-08T00:12:40.564 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6828/86608", 2023-10-08T00:12:40.564 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6825/86608", 2023-10-08T00:12:40.564 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:40.564 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:40.564 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:40.564 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.564 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:12:40.565 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:40.565 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:40.565 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:40.565 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:40.565 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:40.565 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:40.565 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:40.565 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:40.565 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:40.565 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:40.566 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:40.566 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:40.566 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.566 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.566 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6808", 2023-10-08T00:12:40.566 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:40.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.566 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.566 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6809", 2023-10-08T00:12:40.566 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:40.567 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.567 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.567 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:40.567 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.567 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.567 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6810", 2023-10-08T00:12:40.567 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:40.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.567 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.567 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6811", 2023-10-08T00:12:40.568 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:40.568 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.568 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.568 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:40.568 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.568 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.568 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6814", 2023-10-08T00:12:40.569 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:40.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.569 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.569 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6815", 2023-10-08T00:12:40.570 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:40.570 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.570 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.570 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:40.570 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.570 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.570 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6812", 2023-10-08T00:12:40.570 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:40.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.571 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.571 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6813", 2023-10-08T00:12:40.571 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:12:40.571 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.571 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.572 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6809/86571", 2023-10-08T00:12:40.572 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6811/86571", 2023-10-08T00:12:40.572 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6815/86571", 2023-10-08T00:12:40.572 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6813/86571", 2023-10-08T00:12:40.572 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:40.572 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:40.572 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:40.572 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.573 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:12:40.573 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:12:40.573 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:12:40.573 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:12:40.573 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:12:40.573 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:12:40.573 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:12:40.573 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:12:40.573 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:12:40.573 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2023-10-08T00:12:40.574 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:12:40.574 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:12:40.574 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:12:40.574 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.574 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.574 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6817", 2023-10-08T00:12:40.574 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:40.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.575 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.575 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6819", 2023-10-08T00:12:40.575 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:40.575 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.575 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.575 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:12:40.575 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.576 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.576 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6822", 2023-10-08T00:12:40.576 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:40.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.576 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.576 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6824", 2023-10-08T00:12:40.576 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:40.576 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.577 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.577 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:12:40.577 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.577 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.577 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6830", 2023-10-08T00:12:40.577 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:40.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.578 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.578 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6831", 2023-10-08T00:12:40.578 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:40.578 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.578 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.578 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:12:40.578 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:12:40.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.579 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:12:40.579 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6827", 2023-10-08T00:12:40.579 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:40.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.579 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:12:40.579 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6829", 2023-10-08T00:12:40.579 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:12:40.579 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.579 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.580 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6819/86612", 2023-10-08T00:12:40.580 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6824/86612", 2023-10-08T00:12:40.580 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6831/86612", 2023-10-08T00:12:40.580 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6829/86612", 2023-10-08T00:12:40.580 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:12:40.580 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:12:40.580 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:12:40.580 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:40.580 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.580 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:12:40.581 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2023-10-08T00:12:40.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.581 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:12:40.581 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:40.581 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:40.581 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:40.581 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:40.581 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:40.581 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.905615+0000", 2023-10-08T00:12:40.581 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:40.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.582 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:12:40.582 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:40.582 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:40.582 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:40.582 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:40.582 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:40.582 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.880709+0000", 2023-10-08T00:12:40.582 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:40.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.583 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:12:40.583 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:40.583 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:40.583 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:40.583 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:40.583 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:40.583 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.874704+0000", 2023-10-08T00:12:40.583 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:40.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.584 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:12:40.584 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:40.584 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:40.584 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:40.584 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:40.584 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:40.584 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.899604+0000", 2023-10-08T00:12:40.584 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:40.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.584 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:12:40.585 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:40.585 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:40.585 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:40.585 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:40.585 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:40.585 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.844371+0000", 2023-10-08T00:12:40.585 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:40.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.585 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:12:40.586 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:40.586 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:40.586 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:40.586 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:40.586 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:40.586 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.794810+0000", 2023-10-08T00:12:40.586 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:40.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.586 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:12:40.586 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:40.587 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:40.587 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:40.587 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:40.587 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:40.587 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.859474+0000", 2023-10-08T00:12:40.587 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:40.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:40.587 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:12:40.587 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:12:40.588 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:12:40.588 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:12:40.588 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:12:40.588 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:12:40.588 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.893130+0000", 2023-10-08T00:12:40.588 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:12:40.588 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.588 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:12:40.588 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2023-10-08T00:12:40.588 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [], 2023-10-08T00:12:40.589 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2023-10-08T00:12:40.589 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2023-10-08T00:12:40.589 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2023-10-08T00:12:40.589 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2023-10-08T00:12:40.589 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2023-10-08T00:12:40.589 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2023-10-08T00:12:40.589 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2023-10-08T00:12:40.589 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2023-10-08T00:12:40.589 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2023-10-08T00:12:40.589 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2023-10-08T00:12:40.590 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2023-10-08T00:12:40.590 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:40.590 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2023-10-08T00:12:40.590 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2023-10-08T00:12:40.590 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2023-10-08T00:12:40.590 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2023-10-08T00:12:40.590 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2023-10-08T00:12:40.590 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2023-10-08T00:12:40.590 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2023-10-08T00:12:40.591 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2023-10-08T00:12:40.591 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2023-10-08T00:12:40.591 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2023-10-08T00:12:40.591 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2023-10-08T00:12:40.591 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:40.591 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:12:40.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.491+0000 7fd76bd06700 1 -- 172.21.15.99:0/2229724717 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd748066f30 msgr2=0x7fd7480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:40.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.491+0000 7fd76bd06700 1 --2- 172.21.15.99:0/2229724717 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd748066f30 0x7fd7480693e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd7600093f0 tx=0x7fd760017000 comp rx=0 tx=0).stop 2023-10-08T00:12:40.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.491+0000 7fd76bd06700 1 -- 172.21.15.99:0/2229724717 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd76415a620 msgr2=0x7fd76415e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:40.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.491+0000 7fd76bd06700 1 --2- 172.21.15.99:0/2229724717 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd76415a620 0x7fd76415e560 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd75c009fa0 tx=0x7fd75c00b040 comp rx=0 tx=0).stop 2023-10-08T00:12:40.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.491+0000 7fd76bd06700 1 -- 172.21.15.99:0/2229724717 shutdown_connections 2023-10-08T00:12:40.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.491+0000 7fd76bd06700 1 --2- 172.21.15.99:0/2229724717 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd748066f30 0x7fd7480693e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:40.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.491+0000 7fd76bd06700 1 --2- 172.21.15.99:0/2229724717 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd76415a620 0x7fd76415e560 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:40.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.491+0000 7fd76bd06700 1 --2- 172.21.15.99:0/2229724717 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd764157870 0x7fd7641556d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:40.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.491+0000 7fd76bd06700 1 --2- 172.21.15.99:0/2229724717 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd764157c40 0x7fd76416e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:40.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.491+0000 7fd76bd06700 1 -- 172.21.15.99:0/2229724717 >> 172.21.15.99:0/2229724717 conn(0x7fd764068ce0 msgr2=0x7fd7640b55e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:40.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.491+0000 7fd76bd06700 1 -- 172.21.15.99:0/2229724717 shutdown_connections 2023-10-08T00:12:40.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.491+0000 7fd76bd06700 1 -- 172.21.15.99:0/2229724717 wait complete. 2023-10-08T00:12:40.594 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:12:40.709 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.709+0000 7f5e55761700 1 Processor -- start 2023-10-08T00:12:40.709 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.709+0000 7f5e55761700 1 -- start start 2023-10-08T00:12:40.713 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.709+0000 7f5e55761700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f5e500cc370 0x7f5e5006a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:40.713 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.709+0000 7f5e55761700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e5006a5a0 0x7f5e50156350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:40.713 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.709+0000 7f5e55761700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e50068340 con 0x7f5e500cbfa0 2023-10-08T00:12:40.713 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.709+0000 7f5e55761700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e5006abd0 con 0x7f5e500cc370 2023-10-08T00:12:40.713 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.709+0000 7f5e55761700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e50068ee0 con 0x7f5e5006a5a0 2023-10-08T00:12:40.714 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.710+0000 7f5e4ffff700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f5e500cbfa0 0x7f5e500687d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50104/0 (socket says 172.21.15.99:50104) 2023-10-08T00:12:40.714 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.710+0000 7f5e4ffff700 1 -- 172.21.15.99:0/4114343933 learned_addr learned my addr 172.21.15.99:0/4114343933 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:12:40.714 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.710+0000 7f5e54f60700 1 --2- 172.21.15.99:0/4114343933 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e5006a5a0 0x7f5e50156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:40.714 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.710+0000 7f5e4f7fe700 1 --2- 172.21.15.99:0/4114343933 >> v2:172.21.15.191:3300/0 conn(0x7f5e500cc370 0x7f5e5006a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:40.714 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.710+0000 7f5e4effd700 1 -- 172.21.15.99:0/4114343933 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3038170050 0 0) 0x7f5e50068340 con 0x7f5e500cbfa0 2023-10-08T00:12:40.714 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.710+0000 7f5e4effd700 1 -- 172.21.15.99:0/4114343933 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e38003570 con 0x7f5e500cbfa0 2023-10-08T00:12:40.714 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.710+0000 7f5e4effd700 1 -- 172.21.15.99:0/4114343933 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3112429554 0 0) 0x7f5e38003570 con 0x7f5e500cbfa0 2023-10-08T00:12:40.714 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.710+0000 7f5e4effd700 1 -- 172.21.15.99:0/4114343933 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e5006a5a0 msgr2=0x7f5e50156350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:40.714 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.710+0000 7f5e4effd700 1 --2- 172.21.15.99:0/4114343933 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e5006a5a0 0x7f5e50156350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:40.714 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.710+0000 7f5e4effd700 1 -- 172.21.15.99:0/4114343933 >> v2:172.21.15.191:3300/0 conn(0x7f5e500cc370 msgr2=0x7f5e5006a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:40.714 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.710+0000 7f5e4effd700 1 --2- 172.21.15.99:0/4114343933 >> v2:172.21.15.191:3300/0 conn(0x7f5e500cc370 0x7f5e5006a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:40.715 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.710+0000 7f5e4effd700 1 -- 172.21.15.99:0/4114343933 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e5006f040 con 0x7f5e500cbfa0 2023-10-08T00:12:40.715 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.710+0000 7f5e4effd700 1 -- 172.21.15.99:0/4114343933 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f5e40001470 con 0x7f5e500cbfa0 2023-10-08T00:12:40.715 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.710+0000 7f5e4effd700 1 -- 172.21.15.99:0/4114343933 >> v1:172.21.15.99:6789/0 conn(0x7f5e500cbfa0 legacy=0x7f5e500687d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:40.715 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.710+0000 7f5e4effd700 1 --2- 172.21.15.99:0/4114343933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e38003a60 0x7f5e38003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:40.715 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.710+0000 7f5e4effd700 1 --2- 172.21.15.99:0/4114343933 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e5006a5a0 0x7f5e38004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:40.715 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.710+0000 7f5e4effd700 1 --2- 172.21.15.99:0/4114343933 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e38004d50 0x7f5e380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:40.715 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.710+0000 7f5e4effd700 1 -- 172.21.15.99:0/4114343933 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5006abd0 con 0x7f5e38003a60 2023-10-08T00:12:40.715 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.710+0000 7f5e4effd700 1 -- 172.21.15.99:0/4114343933 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e50068ee0 con 0x7f5e38004d50 2023-10-08T00:12:40.715 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.710+0000 7f5e4effd700 1 -- 172.21.15.99:0/4114343933 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3800ba60 con 0x7f5e5006a5a0 2023-10-08T00:12:40.715 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.710+0000 7f5e4ffff700 1 --2- 172.21.15.99:0/4114343933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e38003a60 0x7f5e38003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:40.715 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.710+0000 7f5e4f7fe700 1 --2- 172.21.15.99:0/4114343933 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e5006a5a0 0x7f5e38004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:40.715 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.710+0000 7f5e54f60700 1 --2- 172.21.15.99:0/4114343933 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e38004d50 0x7f5e380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:40.716 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.711+0000 7f5e4ffff700 1 -- 172.21.15.99:0/4114343933 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e5006a5a0 msgr2=0x7f5e38004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:40.716 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.711+0000 7f5e4ffff700 1 --2- 172.21.15.99:0/4114343933 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e5006a5a0 0x7f5e38004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:40.716 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.711+0000 7f5e4ffff700 1 -- 172.21.15.99:0/4114343933 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e38004d50 msgr2=0x7f5e380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:40.716 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.711+0000 7f5e4ffff700 1 --2- 172.21.15.99:0/4114343933 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e38004d50 0x7f5e380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:40.716 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.711+0000 7f5e4ffff700 1 -- 172.21.15.99:0/4114343933 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e38003570 con 0x7f5e38003a60 2023-10-08T00:12:40.716 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.711+0000 7f5e4f7fe700 1 --2- 172.21.15.99:0/4114343933 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e5006a5a0 0x7f5e38004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:12:40.716 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.711+0000 7f5e4ffff700 1 --2- 172.21.15.99:0/4114343933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e38003a60 0x7f5e38003e70 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f5e4000c3c0 tx=0x7f5e500cd5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=749fffb6961c0e20 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:40.716 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.711+0000 7f5e4effd700 1 -- 172.21.15.99:0/4114343933 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4000cc30 con 0x7f5e38003a60 2023-10-08T00:12:40.716 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.711+0000 7f5e4effd700 1 -- 172.21.15.99:0/4114343933 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e4000cdd0 con 0x7f5e38003a60 2023-10-08T00:12:40.716 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.711+0000 7f5e4effd700 1 -- 172.21.15.99:0/4114343933 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e40005b70 con 0x7f5e38003a60 2023-10-08T00:12:40.716 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.711+0000 7f5e55761700 1 -- 172.21.15.99:0/4114343933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e38003a60 msgr2=0x7f5e38003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:40.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.711+0000 7f5e55761700 1 --2- 172.21.15.99:0/4114343933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e38003a60 0x7f5e38003e70 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f5e4000c3c0 tx=0x7f5e500cd5d0 comp rx=0 tx=0).stop 2023-10-08T00:12:40.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.711+0000 7f5e54f60700 1 -- 172.21.15.99:0/4114343933 reap_dead start 2023-10-08T00:12:40.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.711+0000 7f5e55761700 1 -- 172.21.15.99:0/4114343933 shutdown_connections 2023-10-08T00:12:40.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.711+0000 7f5e55761700 1 -- 172.21.15.99:0/4114343933 >> 172.21.15.99:0/4114343933 conn(0x7f5e500cd8f0 msgr2=0x7f5e500cdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:40.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.711+0000 7f5e55761700 1 -- 172.21.15.99:0/4114343933 shutdown_connections 2023-10-08T00:12:40.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.711+0000 7f5e55761700 1 -- 172.21.15.99:0/4114343933 wait complete. 2023-10-08T00:12:40.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.712+0000 7f5e55761700 1 Processor -- start 2023-10-08T00:12:40.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.713+0000 7f5e55761700 1 -- start start 2023-10-08T00:12:40.719 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.713+0000 7f5e55761700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e38003a60 0x7f5e50140ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:40.720 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.713+0000 7f5e55761700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e500c6330 0x7f5e5015a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:40.720 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.713+0000 7f5e55761700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e50158840 0x7f5e50158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:40.720 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.713+0000 7f5e55761700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e50152b70 con 0x7f5e500c6330 2023-10-08T00:12:40.720 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.713+0000 7f5e55761700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e50141430 con 0x7f5e50158840 2023-10-08T00:12:40.720 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.713+0000 7f5e55761700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e500c6880 con 0x7f5e38003a60 2023-10-08T00:12:40.720 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.714+0000 7f5e4f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e500c6330 0x7f5e5015a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:40.720 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.714+0000 7f5e4f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e500c6330 0x7f5e5015a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46138/0 (socket says 172.21.15.99:46138) 2023-10-08T00:12:40.720 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.714+0000 7f5e4f7fe700 1 -- 172.21.15.99:0/1298013333 learned_addr learned my addr 172.21.15.99:0/1298013333 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:40.720 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.714+0000 7f5e4f7fe700 1 -- 172.21.15.99:0/1298013333 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e38003a60 msgr2=0x7f5e50140ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:40.720 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.714+0000 7f5e4f7fe700 1 --2- 172.21.15.99:0/1298013333 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e38003a60 0x7f5e50140ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:40.721 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.714+0000 7f5e4f7fe700 1 -- 172.21.15.99:0/1298013333 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e50158840 msgr2=0x7f5e50158ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:40.721 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.714+0000 7f5e54f60700 1 --2- 172.21.15.99:0/1298013333 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e50158840 0x7f5e50158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:40.721 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.714+0000 7f5e4f7fe700 1 --2- 172.21.15.99:0/1298013333 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e50158840 0x7f5e50158ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:40.721 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.714+0000 7f5e4f7fe700 1 -- 172.21.15.99:0/1298013333 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e500c6b60 con 0x7f5e500c6330 2023-10-08T00:12:40.721 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.714+0000 7f5e4f7fe700 1 --2- 172.21.15.99:0/1298013333 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e500c6330 0x7f5e5015a960 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5e44011c70 tx=0x7f5e44008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:40.721 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.714+0000 7f5e37fff700 1 -- 172.21.15.99:0/1298013333 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e44016070 con 0x7f5e500c6330 2023-10-08T00:12:40.721 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.714+0000 7f5e37fff700 1 -- 172.21.15.99:0/1298013333 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e4400cb60 con 0x7f5e500c6330 2023-10-08T00:12:40.721 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.714+0000 7f5e55761700 1 -- 172.21.15.99:0/1298013333 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e500c6e40 con 0x7f5e500c6330 2023-10-08T00:12:40.721 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.714+0000 7f5e37fff700 1 -- 172.21.15.99:0/1298013333 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4401a410 con 0x7f5e500c6330 2023-10-08T00:12:40.721 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.714+0000 7f5e55761700 1 -- 172.21.15.99:0/1298013333 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e50159220 con 0x7f5e500c6330 2023-10-08T00:12:40.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.715+0000 7f5e55761700 1 -- 172.21.15.99:0/1298013333 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e5006d5d0 con 0x7f5e500c6330 2023-10-08T00:12:40.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.715+0000 7f5e37fff700 1 -- 172.21.15.99:0/1298013333 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5e4401a780 con 0x7f5e500c6330 2023-10-08T00:12:40.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.716+0000 7f5e37fff700 1 --2- 172.21.15.99:0/1298013333 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5e3c066f30 0x7f5e3c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:40.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.716+0000 7f5e37fff700 1 -- 172.21.15.99:0/1298013333 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5e440864e0 con 0x7f5e500c6330 2023-10-08T00:12:40.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.716+0000 7f5e4ffff700 1 --2- 172.21.15.99:0/1298013333 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5e3c066f30 0x7f5e3c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:40.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.716+0000 7f5e4ffff700 1 --2- 172.21.15.99:0/1298013333 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5e3c066f30 0x7f5e3c0693e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5e4000c3c0 tx=0x7f5e40015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:40.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.720+0000 7f5e37fff700 1 -- 172.21.15.99:0/1298013333 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5e440537d0 con 0x7f5e500c6330 2023-10-08T00:12:40.944 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.944+0000 7f5e55761700 1 -- 172.21.15.99:0/1298013333 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e5004bb80 con 0x7f5e500c6330 2023-10-08T00:12:40.945 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.944+0000 7f5e37fff700 1 -- 172.21.15.99:0/1298013333 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4603 (secure 0 0 0) 0x7f5e44056df0 con 0x7f5e500c6330 2023-10-08T00:12:40.945 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:12:40.945 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5146,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/1438931938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1438931938},{"type":"v1","addr":"172.21.15.191:6836","nonce":1438931938}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5280,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/3090674680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3090674680},{"type":"v1","addr":"172.21.15.99:6833","nonce":3090674680}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5283,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/4025282357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":4025282357},{"type":"v1","addr":"172.21.15.99:6835","nonce":4025282357}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:12:09.803824+0000","modified":"2023-10-08T00:12:13.436570+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5289},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5289":{"gid":5289,"name":"d","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/2062242133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":2062242133},{"type":"v1","addr":"172.21.15.191:6837","nonce":2062242133}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-08T00:12:40.947 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.946+0000 7f5e55761700 1 -- 172.21.15.99:0/1298013333 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5e3c066f30 msgr2=0x7f5e3c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:40.947 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.946+0000 7f5e55761700 1 --2- 172.21.15.99:0/1298013333 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5e3c066f30 0x7f5e3c0693e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5e4000c3c0 tx=0x7f5e40015000 comp rx=0 tx=0).stop 2023-10-08T00:12:40.947 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.946+0000 7f5e55761700 1 -- 172.21.15.99:0/1298013333 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e500c6330 msgr2=0x7f5e5015a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:40.947 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.946+0000 7f5e55761700 1 --2- 172.21.15.99:0/1298013333 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e500c6330 0x7f5e5015a960 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5e44011c70 tx=0x7f5e44008040 comp rx=0 tx=0).stop 2023-10-08T00:12:40.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.946+0000 7f5e55761700 1 -- 172.21.15.99:0/1298013333 shutdown_connections 2023-10-08T00:12:40.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.946+0000 7f5e55761700 1 --2- 172.21.15.99:0/1298013333 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5e3c066f30 0x7f5e3c0693e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:40.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.946+0000 7f5e55761700 1 --2- 172.21.15.99:0/1298013333 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e50158840 0x7f5e50158ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:40.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.946+0000 7f5e55761700 1 --2- 172.21.15.99:0/1298013333 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e38003a60 0x7f5e50140ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:40.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.946+0000 7f5e55761700 1 --2- 172.21.15.99:0/1298013333 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e500c6330 0x7f5e5015a960 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:40.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.946+0000 7f5e55761700 1 -- 172.21.15.99:0/1298013333 >> 172.21.15.99:0/1298013333 conn(0x7f5e500cd8f0 msgr2=0x7f5e500b0710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:40.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.946+0000 7f5e55761700 1 -- 172.21.15.99:0/1298013333 shutdown_connections 2023-10-08T00:12:40.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:40.946+0000 7f5e55761700 1 -- 172.21.15.99:0/1298013333 wait complete. 2023-10-08T00:12:40.949 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 24 2023-10-08T00:12:40.959 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json status 2023-10-08T00:12:41.107 INFO:teuthology.orchestra.run.smithi191.stdout:{"cluster_fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","whoami":0,"id":5289,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":30.664692050999999,"mdsmap_epoch":24,"osdmap_epoch":55,"osdmap_epoch_barrier":49,"uptime":36.645693397000002} 2023-10-08T00:12:41.107 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", "fs_name": "cephfs", "id": 5289, "mdsmap_epoch": 24, "osdmap_epoch": 55, "osdmap_epoch_barrier": 49, "rank_uptime": 30.664692051, "state": "up:active", "uptime": 36.645693397, "want_state": "up:active", "whoami": 0 } 2023-10-08T00:12:41.107 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-08T00:12:41.107 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:12:41.108 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2023-10-08T00:12:41.407 INFO:teuthology.orchestra.run.smithi099.stderr:50+0 records in 2023-10-08T00:12:41.407 INFO:teuthology.orchestra.run.smithi099.stderr:50+0 records out 2023-10-08T00:12:41.407 INFO:teuthology.orchestra.run.smithi099.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.281758 s, 186 MB/s 2023-10-08T00:12:41.407 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T00:12:41.407 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T00:12:41.445 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T00:12:41.483 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T00:12:41.484 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T00:12:41.484 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:12:41.484 DEBUG:teuthology.orchestra.run.smithi099:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-08T00:12:41.551 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2023-10-08T00:12:41.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:12:41.552 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T00:12:41.567 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-08T00:12:41.567 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:12:41.567 DEBUG:teuthology.orchestra.run.smithi191:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-08T00:12:41.606 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2023-10-08T00:12:41.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:12:41.607 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-08T00:12:41.662 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:12:41.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.775+0000 7f00ac528700 1 Processor -- start 2023-10-08T00:12:41.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.775+0000 7f00ac528700 1 -- start start 2023-10-08T00:12:41.780 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.776+0000 7f00ac528700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00a4157870 0x7f00a415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:41.780 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.776+0000 7f00ac528700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00a415cbb0 con 0x7f00a4157c40 2023-10-08T00:12:41.780 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.776+0000 7f00ac528700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00a415bfa0 con 0x7f00a415a620 2023-10-08T00:12:41.781 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.776+0000 7f00ac528700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00a415c280 con 0x7f00a4157870 2023-10-08T00:12:41.781 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.776+0000 7f00aaac5700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f00a4157c40 0x7f00a4158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50120/0 (socket says 172.21.15.99:50120) 2023-10-08T00:12:41.781 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.776+0000 7f00aaac5700 1 -- 172.21.15.99:0/4008675748 learned_addr learned my addr 172.21.15.99:0/4008675748 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:12:41.781 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.777+0000 7f00aa2c4700 1 --2- 172.21.15.99:0/4008675748 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00a4157870 0x7f00a415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:41.781 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.777+0000 7f00a92c2700 1 -- 172.21.15.99:0/4008675748 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3129147950 0 0) 0x7f00a415cbb0 con 0x7f00a4157c40 2023-10-08T00:12:41.781 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.777+0000 7f00a92c2700 1 -- 172.21.15.99:0/4008675748 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0090003570 con 0x7f00a4157c40 2023-10-08T00:12:41.781 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.777+0000 7f00a92c2700 1 -- 172.21.15.99:0/4008675748 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 464528816 0 0) 0x7f0090003570 con 0x7f00a4157c40 2023-10-08T00:12:41.781 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.777+0000 7f00a92c2700 1 -- 172.21.15.99:0/4008675748 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00a4157870 msgr2=0x7f00a415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:41.781 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.777+0000 7f00a92c2700 1 --2- 172.21.15.99:0/4008675748 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00a4157870 0x7f00a415b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:41.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.777+0000 7f00a92c2700 1 -- 172.21.15.99:0/4008675748 >> v1:172.21.15.191:6789/0 conn(0x7f00a415a620 legacy=0x7f00a415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:41.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.777+0000 7f00a92c2700 1 -- 172.21.15.99:0/4008675748 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a415c560 con 0x7f00a4157c40 2023-10-08T00:12:41.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.777+0000 7f00a92c2700 1 -- 172.21.15.99:0/4008675748 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f009c001470 con 0x7f00a4157c40 2023-10-08T00:12:41.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.777+0000 7f00a92c2700 1 -- 172.21.15.99:0/4008675748 >> v1:172.21.15.99:6789/0 conn(0x7f00a4157c40 legacy=0x7f00a4158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:41.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.777+0000 7f00a92c2700 1 --2- 172.21.15.99:0/4008675748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0090003a90 0x7f0090003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:41.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.777+0000 7f00a92c2700 1 --2- 172.21.15.99:0/4008675748 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0090004810 0x7f0090004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:41.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.777+0000 7f00a92c2700 1 --2- 172.21.15.99:0/4008675748 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00a4157870 0x7f00900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:41.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.777+0000 7f00a92c2700 1 -- 172.21.15.99:0/4008675748 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a415c280 con 0x7f0090003a90 2023-10-08T00:12:41.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.777+0000 7f00a92c2700 1 -- 172.21.15.99:0/4008675748 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f009000bb10 con 0x7f0090004810 2023-10-08T00:12:41.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.777+0000 7f00a92c2700 1 -- 172.21.15.99:0/4008675748 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f009000bcd0 con 0x7f00a4157870 2023-10-08T00:12:41.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.777+0000 7f00aa2c4700 1 --2- 172.21.15.99:0/4008675748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0090003a90 0x7f0090003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:41.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.777+0000 7f00a9ac3700 1 --2- 172.21.15.99:0/4008675748 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0090004810 0x7f0090004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:41.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.777+0000 7f00aaac5700 1 --2- 172.21.15.99:0/4008675748 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00a4157870 0x7f00900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:41.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.777+0000 7f00aa2c4700 1 -- 172.21.15.99:0/4008675748 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00a4157870 msgr2=0x7f00900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:41.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.777+0000 7f00aa2c4700 1 --2- 172.21.15.99:0/4008675748 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00a4157870 0x7f00900051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:41.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.777+0000 7f00aa2c4700 1 -- 172.21.15.99:0/4008675748 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0090004810 msgr2=0x7f0090004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:41.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.777+0000 7f00aa2c4700 1 --2- 172.21.15.99:0/4008675748 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0090004810 0x7f0090004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:41.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.777+0000 7f00aa2c4700 1 -- 172.21.15.99:0/4008675748 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0090003570 con 0x7f0090003a90 2023-10-08T00:12:41.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.778+0000 7f00aa2c4700 1 --2- 172.21.15.99:0/4008675748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0090003a90 0x7f0090003e60 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f00a000e210 tx=0x7f00a000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd09646d331e835f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:41.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.778+0000 7f00a92c2700 1 -- 172.21.15.99:0/4008675748 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a000eac0 con 0x7f0090003a90 2023-10-08T00:12:41.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.778+0000 7f00a92c2700 1 -- 172.21.15.99:0/4008675748 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00a000f040 con 0x7f0090003a90 2023-10-08T00:12:41.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.778+0000 7f00a92c2700 1 -- 172.21.15.99:0/4008675748 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a000ee70 con 0x7f0090003a90 2023-10-08T00:12:41.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.778+0000 7f00ac528700 1 -- 172.21.15.99:0/4008675748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0090003a90 msgr2=0x7f0090003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:41.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.778+0000 7f00ac528700 1 --2- 172.21.15.99:0/4008675748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0090003a90 0x7f0090003e60 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f00a000e210 tx=0x7f00a000b040 comp rx=0 tx=0).stop 2023-10-08T00:12:41.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.778+0000 7f00aaac5700 1 -- 172.21.15.99:0/4008675748 reap_dead start 2023-10-08T00:12:41.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.778+0000 7f00ac528700 1 -- 172.21.15.99:0/4008675748 shutdown_connections 2023-10-08T00:12:41.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.778+0000 7f00ac528700 1 -- 172.21.15.99:0/4008675748 >> 172.21.15.99:0/4008675748 conn(0x7f00a4068ce0 msgr2=0x7f00a40690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:41.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.778+0000 7f00ac528700 1 -- 172.21.15.99:0/4008675748 shutdown_connections 2023-10-08T00:12:41.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.778+0000 7f00ac528700 1 -- 172.21.15.99:0/4008675748 wait complete. 2023-10-08T00:12:41.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.779+0000 7f00ac528700 1 Processor -- start 2023-10-08T00:12:41.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.779+0000 7f00ac528700 1 -- start start 2023-10-08T00:12:41.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.780+0000 7f00ac528700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0090003a90 0x7f00a413f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:41.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.780+0000 7f00ac528700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00a414c820 0x7f00a414cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:41.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.780+0000 7f00ac528700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00a4150ed0 0x7f00a4151350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:41.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.780+0000 7f00ac528700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a413fd30 con 0x7f00a4150ed0 2023-10-08T00:12:41.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.780+0000 7f00ac528700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a4151890 con 0x7f00a414c820 2023-10-08T00:12:41.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.780+0000 7f00ac528700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00a4154210 con 0x7f0090003a90 2023-10-08T00:12:41.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.780+0000 7f00aaac5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00a4150ed0 0x7f00a4151350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:41.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.780+0000 7f00aa2c4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0090003a90 0x7f00a413f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:41.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.780+0000 7f00aaac5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00a4150ed0 0x7f00a4151350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46152/0 (socket says 172.21.15.99:46152) 2023-10-08T00:12:41.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.780+0000 7f00aaac5700 1 -- 172.21.15.99:0/3884531621 learned_addr learned my addr 172.21.15.99:0/3884531621 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:41.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.780+0000 7f00aa2c4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0090003a90 0x7f00a413f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.99:39198/0 (socket says 172.21.15.99:39198) 2023-10-08T00:12:41.792 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.780+0000 7f00a9ac3700 1 --2- 172.21.15.99:0/3884531621 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00a414c820 0x7f00a414cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:41.792 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.781+0000 7f00aaac5700 1 -- 172.21.15.99:0/3884531621 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0090003a90 msgr2=0x7f00a413f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:41.792 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.781+0000 7f00aaac5700 1 --2- 172.21.15.99:0/3884531621 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0090003a90 0x7f00a413f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:41.792 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.781+0000 7f00aaac5700 1 -- 172.21.15.99:0/3884531621 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00a414c820 msgr2=0x7f00a414cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:41.792 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.781+0000 7f00aaac5700 1 --2- 172.21.15.99:0/3884531621 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00a414c820 0x7f00a414cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:41.792 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.781+0000 7f00aaac5700 1 -- 172.21.15.99:0/3884531621 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a40b1860 con 0x7f00a4150ed0 2023-10-08T00:12:41.792 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.781+0000 7f00aaac5700 1 --2- 172.21.15.99:0/3884531621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00a4150ed0 0x7f00a4151350 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f00a40b3830 tx=0x7f009c000fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:41.792 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.781+0000 7f009a7fc700 1 -- 172.21.15.99:0/3884531621 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009c010520 con 0x7f00a4150ed0 2023-10-08T00:12:41.792 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.781+0000 7f009a7fc700 1 -- 172.21.15.99:0/3884531621 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f009c0106a0 con 0x7f00a4150ed0 2023-10-08T00:12:41.792 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.781+0000 7f009a7fc700 1 -- 172.21.15.99:0/3884531621 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009c010a60 con 0x7f00a4150ed0 2023-10-08T00:12:41.793 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.781+0000 7f00ac528700 1 -- 172.21.15.99:0/3884531621 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00a4153980 con 0x7f00a4150ed0 2023-10-08T00:12:41.793 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.781+0000 7f00ac528700 1 -- 172.21.15.99:0/3884531621 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00a4153f30 con 0x7f00a4150ed0 2023-10-08T00:12:41.793 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.782+0000 7f00ac528700 1 -- 172.21.15.99:0/3884531621 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00a40b19f0 con 0x7f00a4150ed0 2023-10-08T00:12:41.793 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.782+0000 7f009a7fc700 1 -- 172.21.15.99:0/3884531621 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f009c0133e0 con 0x7f00a4150ed0 2023-10-08T00:12:41.793 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.783+0000 7f009a7fc700 1 --2- 172.21.15.99:0/3884531621 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0088066f30 0x7f00880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:41.793 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.783+0000 7f009a7fc700 1 -- 172.21.15.99:0/3884531621 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f009c083d20 con 0x7f00a4150ed0 2023-10-08T00:12:41.793 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.783+0000 7f00aa2c4700 1 --2- 172.21.15.99:0/3884531621 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0088066f30 0x7f00880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:41.793 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.783+0000 7f00aa2c4700 1 --2- 172.21.15.99:0/3884531621 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0088066f30 0x7f00880693e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f00a000afd0 tx=0x7f00a001a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:41.793 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:41.787+0000 7f009a7fc700 1 -- 172.21.15.99:0/3884531621 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f009c051010 con 0x7f00a4150ed0 2023-10-08T00:12:42.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.010+0000 7f00ac528700 1 -- 172.21.15.99:0/3884531621 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f00a404c3c0 con 0x7f00a4150ed0 2023-10-08T00:12:42.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.011+0000 7f009a7fc700 1 -- 172.21.15.99:0/3884531621 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4603 (secure 0 0 0) 0x7f009c054630 con 0x7f00a4150ed0 2023-10-08T00:12:42.012 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:12:42.012 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5146,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/1438931938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1438931938},{"type":"v1","addr":"172.21.15.191:6836","nonce":1438931938}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5280,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/3090674680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3090674680},{"type":"v1","addr":"172.21.15.99:6833","nonce":3090674680}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5283,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/4025282357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":4025282357},{"type":"v1","addr":"172.21.15.99:6835","nonce":4025282357}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:12:09.803824+0000","modified":"2023-10-08T00:12:13.436570+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5289},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5289":{"gid":5289,"name":"d","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/2062242133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":2062242133},{"type":"v1","addr":"172.21.15.191:6837","nonce":2062242133}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-08T00:12:42.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.013+0000 7f00ac528700 1 -- 172.21.15.99:0/3884531621 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0088066f30 msgr2=0x7f00880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:42.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.013+0000 7f00ac528700 1 --2- 172.21.15.99:0/3884531621 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0088066f30 0x7f00880693e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f00a000afd0 tx=0x7f00a001a000 comp rx=0 tx=0).stop 2023-10-08T00:12:42.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.013+0000 7f00ac528700 1 -- 172.21.15.99:0/3884531621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00a4150ed0 msgr2=0x7f00a4151350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:42.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.013+0000 7f00ac528700 1 --2- 172.21.15.99:0/3884531621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00a4150ed0 0x7f00a4151350 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f00a40b3830 tx=0x7f009c000fb0 comp rx=0 tx=0).stop 2023-10-08T00:12:42.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.013+0000 7f00ac528700 1 -- 172.21.15.99:0/3884531621 shutdown_connections 2023-10-08T00:12:42.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.013+0000 7f00ac528700 1 --2- 172.21.15.99:0/3884531621 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0088066f30 0x7f00880693e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:42.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.013+0000 7f00ac528700 1 --2- 172.21.15.99:0/3884531621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00a4150ed0 0x7f00a4151350 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:42.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.013+0000 7f00ac528700 1 --2- 172.21.15.99:0/3884531621 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0090003a90 0x7f00a413f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:42.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.013+0000 7f00ac528700 1 --2- 172.21.15.99:0/3884531621 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00a414c820 0x7f00a414cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:42.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.013+0000 7f00ac528700 1 -- 172.21.15.99:0/3884531621 >> 172.21.15.99:0/3884531621 conn(0x7f00a4068ce0 msgr2=0x7f00a40b46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:42.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.013+0000 7f00ac528700 1 -- 172.21.15.99:0/3884531621 shutdown_connections 2023-10-08T00:12:42.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.013+0000 7f00ac528700 1 -- 172.21.15.99:0/3884531621 wait complete. 2023-10-08T00:12:42.015 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 24 2023-10-08T00:12:42.025 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:12:42.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.138+0000 7fd3f06f3700 1 Processor -- start 2023-10-08T00:12:42.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.138+0000 7fd3f06f3700 1 -- start start 2023-10-08T00:12:42.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.139+0000 7fd3f06f3700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd3e8157870 0x7fd3e815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:42.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.139+0000 7fd3f06f3700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3e815cbb0 con 0x7fd3e8157c40 2023-10-08T00:12:42.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.139+0000 7fd3f06f3700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3e815bfa0 con 0x7fd3e815a620 2023-10-08T00:12:42.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.139+0000 7fd3f06f3700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3e815c280 con 0x7fd3e8157870 2023-10-08T00:12:42.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.139+0000 7fd3ee48f700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd3e8157870 0x7fd3e815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:42.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.139+0000 7fd3eec90700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fd3e8157c40 0x7fd3e8158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50126/0 (socket says 172.21.15.99:50126) 2023-10-08T00:12:42.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.139+0000 7fd3eec90700 1 -- 172.21.15.99:0/952805691 learned_addr learned my addr 172.21.15.99:0/952805691 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:12:42.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.140+0000 7fd3ee48f700 1 -- 172.21.15.99:0/952805691 >> v1:172.21.15.191:6789/0 conn(0x7fd3e815a620 legacy=0x7fd3e815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:42.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.140+0000 7fd3ee48f700 1 -- 172.21.15.99:0/952805691 >> v1:172.21.15.99:6789/0 conn(0x7fd3e8157c40 legacy=0x7fd3e8158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:42.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.140+0000 7fd3ee48f700 1 -- 172.21.15.99:0/952805691 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3e815c560 con 0x7fd3e8157870 2023-10-08T00:12:42.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.140+0000 7fd3ee48f700 1 --2- 172.21.15.99:0/952805691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3e8157870 0x7fd3e815b6a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fd3e40098e0 tx=0x7fd3e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f2c8275c9a286ed server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:42.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.140+0000 7fd3ed48d700 1 -- 172.21.15.99:0/952805691 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e4009e80 con 0x7fd3e8157870 2023-10-08T00:12:42.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.140+0000 7fd3ed48d700 1 -- 172.21.15.99:0/952805691 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3e400b860 con 0x7fd3e8157870 2023-10-08T00:12:42.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.140+0000 7fd3ed48d700 1 -- 172.21.15.99:0/952805691 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e400bbe0 con 0x7fd3e8157870 2023-10-08T00:12:42.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.141+0000 7fd3f06f3700 1 -- 172.21.15.99:0/952805691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3e8157870 msgr2=0x7fd3e815b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:42.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.141+0000 7fd3f06f3700 1 --2- 172.21.15.99:0/952805691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3e8157870 0x7fd3e815b6a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fd3e40098e0 tx=0x7fd3e40091a0 comp rx=0 tx=0).stop 2023-10-08T00:12:42.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.141+0000 7fd3f06f3700 1 -- 172.21.15.99:0/952805691 shutdown_connections 2023-10-08T00:12:42.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.141+0000 7fd3f06f3700 1 --2- 172.21.15.99:0/952805691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3e8157870 0x7fd3e815b6a0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:42.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.141+0000 7fd3f06f3700 1 -- 172.21.15.99:0/952805691 >> 172.21.15.99:0/952805691 conn(0x7fd3e8068ce0 msgr2=0x7fd3e80690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:42.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.141+0000 7fd3f06f3700 1 -- 172.21.15.99:0/952805691 shutdown_connections 2023-10-08T00:12:42.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.141+0000 7fd3f06f3700 1 -- 172.21.15.99:0/952805691 wait complete. 2023-10-08T00:12:42.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.142+0000 7fd3f06f3700 1 Processor -- start 2023-10-08T00:12:42.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.142+0000 7fd3f06f3700 1 -- start start 2023-10-08T00:12:42.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.143+0000 7fd3f06f3700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3e8157870 0x7fd3e8150ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:42.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.143+0000 7fd3f06f3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd3e8157c40 0x7fd3e816ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:42.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.143+0000 7fd3f06f3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3e815a620 0x7fd3e80b5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:42.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.143+0000 7fd3f06f3700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e814d140 con 0x7fd3e815a620 2023-10-08T00:12:42.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.143+0000 7fd3f06f3700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e81500a0 con 0x7fd3e8157870 2023-10-08T00:12:42.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.143+0000 7fd3f06f3700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e8150620 con 0x7fd3e8157c40 2023-10-08T00:12:42.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.143+0000 7fd3eec90700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3e815a620 0x7fd3e80b5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:42.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.143+0000 7fd3eec90700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3e815a620 0x7fd3e80b5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46178/0 (socket says 172.21.15.99:46178) 2023-10-08T00:12:42.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.143+0000 7fd3eec90700 1 -- 172.21.15.99:0/3347587199 learned_addr learned my addr 172.21.15.99:0/3347587199 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:42.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.143+0000 7fd3eec90700 1 -- 172.21.15.99:0/3347587199 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd3e8157c40 msgr2=0x7fd3e816ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:42.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.143+0000 7fd3eec90700 1 --2- 172.21.15.99:0/3347587199 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd3e8157c40 0x7fd3e816ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:42.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.143+0000 7fd3eec90700 1 -- 172.21.15.99:0/3347587199 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3e8157870 msgr2=0x7fd3e8150ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:42.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.143+0000 7fd3eec90700 1 --2- 172.21.15.99:0/3347587199 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3e8157870 0x7fd3e8150ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:42.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.143+0000 7fd3eec90700 1 -- 172.21.15.99:0/3347587199 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3e4009420 con 0x7fd3e815a620 2023-10-08T00:12:42.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.144+0000 7fd3eec90700 1 --2- 172.21.15.99:0/3347587199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3e815a620 0x7fd3e80b5630 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd3e000cca0 tx=0x7fd3e0001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:42.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.145+0000 7fd3de7fc700 1 -- 172.21.15.99:0/3347587199 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e000aad0 con 0x7fd3e815a620 2023-10-08T00:12:42.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.145+0000 7fd3f06f3700 1 -- 172.21.15.99:0/3347587199 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3e816f690 con 0x7fd3e815a620 2023-10-08T00:12:42.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.145+0000 7fd3de7fc700 1 -- 172.21.15.99:0/3347587199 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3e000ac30 con 0x7fd3e815a620 2023-10-08T00:12:42.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.145+0000 7fd3de7fc700 1 -- 172.21.15.99:0/3347587199 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e00070f0 con 0x7fd3e815a620 2023-10-08T00:12:42.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.145+0000 7fd3f06f3700 1 -- 172.21.15.99:0/3347587199 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3e81502b0 con 0x7fd3e815a620 2023-10-08T00:12:42.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.146+0000 7fd3f06f3700 1 -- 172.21.15.99:0/3347587199 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3e804c3c0 con 0x7fd3e815a620 2023-10-08T00:12:42.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.148+0000 7fd3de7fc700 1 -- 172.21.15.99:0/3347587199 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd3e0007500 con 0x7fd3e815a620 2023-10-08T00:12:42.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.148+0000 7fd3de7fc700 1 --2- 172.21.15.99:0/3347587199 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd3cc066f30 0x7fd3cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:42.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.149+0000 7fd3de7fc700 1 -- 172.21.15.99:0/3347587199 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5850+0+0 (secure 0 0 0) 0x7fd3e00854d0 con 0x7fd3e815a620 2023-10-08T00:12:42.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.149+0000 7fd3ee48f700 1 --2- 172.21.15.99:0/3347587199 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd3cc066f30 0x7fd3cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:42.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.149+0000 7fd3ee48f700 1 --2- 172.21.15.99:0/3347587199 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd3cc066f30 0x7fd3cc0693e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd3e80b6810 tx=0x7fd3e4017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:42.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.152+0000 7fd3de7fc700 1 -- 172.21.15.99:0/3347587199 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd3e0052730 con 0x7fd3e815a620 2023-10-08T00:12:42.334 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.333+0000 7fd3f06f3700 1 -- 172.21.15.99:0/3347587199 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd3e8150760 con 0x7fd3e815a620 2023-10-08T00:12:42.335 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.334+0000 7fd3de7fc700 1 -- 172.21.15.99:0/3347587199 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+15586 (secure 0 0 0) 0x7fd3e0055d50 con 0x7fd3e815a620 2023-10-08T00:12:42.336 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:12:42.336 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":56,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:12:42.138824+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:12:07.733432+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-08T00:12:08.762879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.8","mappings":[{"from":5,"to":7}]},{"pgid":"7.15","mappings":[{"from":5,"to":0}]},{"pgid":"7.28","mappings":[{"from":5,"to":7}]},{"pgid":"7.2b","mappings":[{"from":5,"to":7}]},{"pgid":"7.33","mappings":[{"from":5,"to":7}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:12:42.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.336+0000 7fd3f06f3700 1 -- 172.21.15.99:0/3347587199 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd3cc066f30 msgr2=0x7fd3cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:42.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.336+0000 7fd3f06f3700 1 --2- 172.21.15.99:0/3347587199 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd3cc066f30 0x7fd3cc0693e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd3e80b6810 tx=0x7fd3e4017000 comp rx=0 tx=0).stop 2023-10-08T00:12:42.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.336+0000 7fd3f06f3700 1 -- 172.21.15.99:0/3347587199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3e815a620 msgr2=0x7fd3e80b5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:42.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.336+0000 7fd3f06f3700 1 --2- 172.21.15.99:0/3347587199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3e815a620 0x7fd3e80b5630 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd3e000cca0 tx=0x7fd3e0001a20 comp rx=0 tx=0).stop 2023-10-08T00:12:42.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.336+0000 7fd3f06f3700 1 -- 172.21.15.99:0/3347587199 shutdown_connections 2023-10-08T00:12:42.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.336+0000 7fd3f06f3700 1 --2- 172.21.15.99:0/3347587199 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd3cc066f30 0x7fd3cc0693e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:42.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.336+0000 7fd3f06f3700 1 --2- 172.21.15.99:0/3347587199 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3e8157870 0x7fd3e8150ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:42.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.336+0000 7fd3f06f3700 1 --2- 172.21.15.99:0/3347587199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3e815a620 0x7fd3e80b5630 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:42.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.336+0000 7fd3f06f3700 1 --2- 172.21.15.99:0/3347587199 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd3e8157c40 0x7fd3e816ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:42.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.336+0000 7fd3f06f3700 1 -- 172.21.15.99:0/3347587199 >> 172.21.15.99:0/3347587199 conn(0x7fd3e8068ce0 msgr2=0x7fd3e814d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:42.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.336+0000 7fd3f06f3700 1 -- 172.21.15.99:0/3347587199 shutdown_connections 2023-10-08T00:12:42.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.336+0000 7fd3f06f3700 1 -- 172.21.15.99:0/3347587199 wait complete. 2023-10-08T00:12:42.349 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:10:42.063344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T00:12:07.733432+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T00:12:08.762879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T00:12:42.350 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T00:12:42.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.467+0000 7f1510b34700 1 Processor -- start 2023-10-08T00:12:42.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.467+0000 7f1510b34700 1 -- start start 2023-10-08T00:12:42.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.468+0000 7f1510b34700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f150c0cb280 0x7f150c156530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:42.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.468+0000 7f1510b34700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f150c0cc440 con 0x7f150c156160 2023-10-08T00:12:42.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.468+0000 7f1510b34700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f150c0cb830 con 0x7f150c0c71d0 2023-10-08T00:12:42.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.468+0000 7f1510b34700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f150c0c31a0 con 0x7f150c0cb280 2023-10-08T00:12:42.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.468+0000 7f150a59c700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f150c156160 0x7f150c0c2a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50140/0 (socket says 172.21.15.99:50140) 2023-10-08T00:12:42.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.468+0000 7f150a59c700 1 -- 172.21.15.99:0/2667083523 learned_addr learned my addr 172.21.15.99:0/2667083523 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:12:42.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.469+0000 7f1509d9b700 1 --2- 172.21.15.99:0/2667083523 >> v2:172.21.15.191:3300/0 conn(0x7f150c0cb280 0x7f150c156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:42.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.469+0000 7f150959a700 1 -- 172.21.15.99:0/2667083523 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2890743942 0 0) 0x7f150c0cc440 con 0x7f150c156160 2023-10-08T00:12:42.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.469+0000 7f150959a700 1 -- 172.21.15.99:0/2667083523 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14f4003570 con 0x7f150c156160 2023-10-08T00:12:42.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.469+0000 7f1509d9b700 1 -- 172.21.15.99:0/2667083523 >> v1:172.21.15.191:6789/0 conn(0x7f150c0c71d0 legacy=0x7f150c06d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:42.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.469+0000 7f1509d9b700 1 -- 172.21.15.99:0/2667083523 >> v1:172.21.15.99:6789/0 conn(0x7f150c156160 legacy=0x7f150c0c2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:42.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.469+0000 7f1509d9b700 1 -- 172.21.15.99:0/2667083523 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f150c0cbb10 con 0x7f150c0cb280 2023-10-08T00:12:42.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.469+0000 7f1509d9b700 1 --2- 172.21.15.99:0/2667083523 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f150c0cb280 0x7f150c156530 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f150000aac0 tx=0x7f150000a330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67dea344f3e9757 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:42.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.469+0000 7f150959a700 1 -- 172.21.15.99:0/2667083523 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150000c930 con 0x7f150c0cb280 2023-10-08T00:12:42.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.469+0000 7f150959a700 1 -- 172.21.15.99:0/2667083523 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f150000ca90 con 0x7f150c0cb280 2023-10-08T00:12:42.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.469+0000 7f150959a700 1 -- 172.21.15.99:0/2667083523 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150000ce10 con 0x7f150c0cb280 2023-10-08T00:12:42.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.470+0000 7f1510b34700 1 -- 172.21.15.99:0/2667083523 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f150c0cb280 msgr2=0x7f150c156530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:42.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.470+0000 7f1510b34700 1 --2- 172.21.15.99:0/2667083523 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f150c0cb280 0x7f150c156530 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f150000aac0 tx=0x7f150000a330 comp rx=0 tx=0).stop 2023-10-08T00:12:42.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.470+0000 7f1510b34700 1 -- 172.21.15.99:0/2667083523 shutdown_connections 2023-10-08T00:12:42.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.470+0000 7f1510b34700 1 --2- 172.21.15.99:0/2667083523 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f150c0cb280 0x7f150c156530 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:42.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.470+0000 7f1510b34700 1 -- 172.21.15.99:0/2667083523 >> 172.21.15.99:0/2667083523 conn(0x7f150c067660 msgr2=0x7f150c067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:42.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.470+0000 7f1510b34700 1 -- 172.21.15.99:0/2667083523 shutdown_connections 2023-10-08T00:12:42.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.470+0000 7f1510b34700 1 -- 172.21.15.99:0/2667083523 wait complete. 2023-10-08T00:12:42.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.471+0000 7f1510b34700 1 Processor -- start 2023-10-08T00:12:42.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.471+0000 7f1510b34700 1 -- start start 2023-10-08T00:12:42.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.471+0000 7f1510b34700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f150c0c71d0 0x7f150c14e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:42.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.472+0000 7f1510b34700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f150c0cb280 0x7f150c0a87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:42.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.472+0000 7f1510b34700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f150c156160 0x7f150c0adb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:42.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.472+0000 7f1510b34700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150c0c5500 con 0x7f150c0cb280 2023-10-08T00:12:42.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.472+0000 7f1510b34700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150c0c48f0 con 0x7f150c156160 2023-10-08T00:12:42.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.472+0000 7f1510b34700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f150c0c4bd0 con 0x7f150c0c71d0 2023-10-08T00:12:42.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.472+0000 7f1509d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f150c0cb280 0x7f150c0a87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:42.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.472+0000 7f1509d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f150c0cb280 0x7f150c0a87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46184/0 (socket says 172.21.15.99:46184) 2023-10-08T00:12:42.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.472+0000 7f1509d9b700 1 -- 172.21.15.99:0/2970827766 learned_addr learned my addr 172.21.15.99:0/2970827766 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:42.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.472+0000 7f150ad9d700 1 --2- 172.21.15.99:0/2970827766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f150c156160 0x7f150c0adb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:42.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.472+0000 7f1509d9b700 1 -- 172.21.15.99:0/2970827766 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f150c0c71d0 msgr2=0x7f150c14e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:42.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.472+0000 7f150a59c700 1 --2- 172.21.15.99:0/2970827766 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f150c0c71d0 0x7f150c14e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:42.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.472+0000 7f1509d9b700 1 --2- 172.21.15.99:0/2970827766 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f150c0c71d0 0x7f150c14e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:42.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.472+0000 7f1509d9b700 1 -- 172.21.15.99:0/2970827766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f150c156160 msgr2=0x7f150c0adb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:42.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.472+0000 7f1509d9b700 1 --2- 172.21.15.99:0/2970827766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f150c156160 0x7f150c0adb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:42.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.472+0000 7f1509d9b700 1 -- 172.21.15.99:0/2970827766 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f150000a5d0 con 0x7f150c0cb280 2023-10-08T00:12:42.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.472+0000 7f150ad9d700 1 --2- 172.21.15.99:0/2970827766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f150c156160 0x7f150c0adb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:12:42.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.472+0000 7f150a59c700 1 --2- 172.21.15.99:0/2970827766 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f150c0c71d0 0x7f150c14e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:12:42.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.472+0000 7f1509d9b700 1 --2- 172.21.15.99:0/2970827766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f150c0cb280 0x7f150c0a87d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f150000aa10 tx=0x7f150001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:42.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.472+0000 7f14fa7fc700 1 -- 172.21.15.99:0/2970827766 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150001c070 con 0x7f150c0cb280 2023-10-08T00:12:42.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.472+0000 7f1510b34700 1 -- 172.21.15.99:0/2970827766 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f150c0c4eb0 con 0x7f150c0cb280 2023-10-08T00:12:42.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.472+0000 7f14fa7fc700 1 -- 172.21.15.99:0/2970827766 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f150001d040 con 0x7f150c0cb280 2023-10-08T00:12:42.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.472+0000 7f14fa7fc700 1 -- 172.21.15.99:0/2970827766 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1500023880 con 0x7f150c0cb280 2023-10-08T00:12:42.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.472+0000 7f1510b34700 1 -- 172.21.15.99:0/2970827766 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f150c14e980 con 0x7f150c0cb280 2023-10-08T00:12:42.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.473+0000 7f1510b34700 1 -- 172.21.15.99:0/2970827766 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f150c0c5670 con 0x7f150c0cb280 2023-10-08T00:12:42.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.474+0000 7f14fa7fc700 1 -- 172.21.15.99:0/2970827766 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1500023bf0 con 0x7f150c0cb280 2023-10-08T00:12:42.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.474+0000 7f14fa7fc700 1 --2- 172.21.15.99:0/2970827766 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f14ec066f30 0x7f14ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:42.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.474+0000 7f14fa7fc700 1 -- 172.21.15.99:0/2970827766 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5850+0+0 (secure 0 0 0) 0x7f15000878c0 con 0x7f150c0cb280 2023-10-08T00:12:42.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.474+0000 7f150a59c700 1 --2- 172.21.15.99:0/2970827766 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f14ec066f30 0x7f14ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:42.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.474+0000 7f150a59c700 1 --2- 172.21.15.99:0/2970827766 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f14ec066f30 0x7f14ec0693e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f14fc007fd0 tx=0x7f14fc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:42.479 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.478+0000 7f14fa7fc700 1 -- 172.21.15.99:0/2970827766 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1500054aa0 con 0x7f150c0cb280 2023-10-08T00:12:42.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.658+0000 7f1510b34700 1 -- 172.21.15.99:0/2970827766 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f150c04c3c0 con 0x7f150c0cb280 2023-10-08T00:12:42.709 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.659+0000 7f14fa7fc700 1 -- 172.21.15.99:0/2970827766 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) v1 ==== 93+0+10 (secure 0 0 0) 0x7f15000870a0 con 0x7f150c0cb280 2023-10-08T00:12:42.709 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2023-10-08T00:12:42.710 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.660+0000 7f1510b34700 1 -- 172.21.15.99:0/2970827766 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f14ec066f30 msgr2=0x7f14ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:42.710 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.660+0000 7f1510b34700 1 --2- 172.21.15.99:0/2970827766 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f14ec066f30 0x7f14ec0693e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f14fc007fd0 tx=0x7f14fc009040 comp rx=0 tx=0).stop 2023-10-08T00:12:42.710 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.660+0000 7f1510b34700 1 -- 172.21.15.99:0/2970827766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f150c0cb280 msgr2=0x7f150c0a87d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:42.710 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.660+0000 7f1510b34700 1 --2- 172.21.15.99:0/2970827766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f150c0cb280 0x7f150c0a87d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f150000aa10 tx=0x7f150001a000 comp rx=0 tx=0).stop 2023-10-08T00:12:42.711 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.660+0000 7f1510b34700 1 -- 172.21.15.99:0/2970827766 shutdown_connections 2023-10-08T00:12:42.711 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.660+0000 7f1510b34700 1 --2- 172.21.15.99:0/2970827766 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f14ec066f30 0x7f14ec0693e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:42.711 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.660+0000 7f1510b34700 1 --2- 172.21.15.99:0/2970827766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f150c156160 0x7f150c0adb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:42.711 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.660+0000 7f1510b34700 1 --2- 172.21.15.99:0/2970827766 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f150c0c71d0 0x7f150c14e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:42.711 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.660+0000 7f1510b34700 1 --2- 172.21.15.99:0/2970827766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f150c0cb280 0x7f150c0a87d0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:42.711 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.660+0000 7f1510b34700 1 -- 172.21.15.99:0/2970827766 >> 172.21.15.99:0/2970827766 conn(0x7f150c067660 msgr2=0x7f150c0b1880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:42.711 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.660+0000 7f1510b34700 1 -- 172.21.15.99:0/2970827766 shutdown_connections 2023-10-08T00:12:42.711 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.660+0000 7f1510b34700 1 -- 172.21.15.99:0/2970827766 wait complete. 2023-10-08T00:12:42.711 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T00:12:42.830 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.830+0000 7f2bbb5b2700 1 Processor -- start 2023-10-08T00:12:42.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.830+0000 7f2bbb5b2700 1 -- start start 2023-10-08T00:12:42.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.831+0000 7f2bbb5b2700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2bb415a290 0x7f2bb406a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:42.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.831+0000 7f2bbb5b2700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2bb406aae0 0x7f2bb40cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:42.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.831+0000 7f2bbb5b2700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bb40b0790 con 0x7f2bb4159ec0 2023-10-08T00:12:42.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.831+0000 7f2bbb5b2700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bb406af20 con 0x7f2bb406aae0 2023-10-08T00:12:42.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.831+0000 7f2bbb5b2700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bb40ce1f0 con 0x7f2bb415a290 2023-10-08T00:12:42.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.831+0000 7f2bb934e700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f2bb4159ec0 0x7f2bb40af940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50152/0 (socket says 172.21.15.99:50152) 2023-10-08T00:12:42.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.831+0000 7f2bb934e700 1 -- 172.21.15.99:0/145467293 learned_addr learned my addr 172.21.15.99:0/145467293 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:12:42.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.831+0000 7f2bb9b4f700 1 --2- 172.21.15.99:0/145467293 >> v2:172.21.15.99:3300/0 conn(0x7f2bb406aae0 0x7f2bb40cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:42.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.831+0000 7f2bb8b4d700 1 --2- 172.21.15.99:0/145467293 >> v2:172.21.15.191:3300/0 conn(0x7f2bb415a290 0x7f2bb406a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:42.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.831+0000 7f2bb9b4f700 1 -- 172.21.15.99:0/145467293 >> v2:172.21.15.191:3300/0 conn(0x7f2bb415a290 msgr2=0x7f2bb406a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:42.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.831+0000 7f2bb9b4f700 1 --2- 172.21.15.99:0/145467293 >> v2:172.21.15.191:3300/0 conn(0x7f2bb415a290 0x7f2bb406a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:42.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.831+0000 7f2bb9b4f700 1 -- 172.21.15.99:0/145467293 >> v1:172.21.15.99:6789/0 conn(0x7f2bb4159ec0 legacy=0x7f2bb40af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:42.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.831+0000 7f2bb9b4f700 1 -- 172.21.15.99:0/145467293 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb40b0b20 con 0x7f2bb406aae0 2023-10-08T00:12:42.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.831+0000 7f2babfff700 1 -- 172.21.15.99:0/145467293 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 941484875 0 0) 0x7f2bb40b0790 con 0x7f2bb4159ec0 2023-10-08T00:12:42.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.831+0000 7f2bb9b4f700 1 --2- 172.21.15.99:0/145467293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bb406aae0 0x7f2bb40cf590 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4089910 tx=0x7f2bac009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f98d46856f969f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:42.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.831+0000 7f2babfff700 1 -- 172.21.15.99:0/145467293 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bac00e070 con 0x7f2bb406aae0 2023-10-08T00:12:42.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.831+0000 7f2babfff700 1 -- 172.21.15.99:0/145467293 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bac00bb60 con 0x7f2bb406aae0 2023-10-08T00:12:42.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.832+0000 7f2babfff700 1 -- 172.21.15.99:0/145467293 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bac012430 con 0x7f2bb406aae0 2023-10-08T00:12:42.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.832+0000 7f2bbb5b2700 1 -- 172.21.15.99:0/145467293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bb406aae0 msgr2=0x7f2bb40cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:42.835 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.832+0000 7f2bbb5b2700 1 --2- 172.21.15.99:0/145467293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bb406aae0 0x7f2bb40cf590 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4089910 tx=0x7f2bac009780 comp rx=0 tx=0).stop 2023-10-08T00:12:42.835 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.832+0000 7f2bbb5b2700 1 -- 172.21.15.99:0/145467293 shutdown_connections 2023-10-08T00:12:42.835 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.832+0000 7f2bbb5b2700 1 --2- 172.21.15.99:0/145467293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bb406aae0 0x7f2bb40cf590 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:42.835 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.832+0000 7f2bbb5b2700 1 --2- 172.21.15.99:0/145467293 >> v2:172.21.15.191:3300/0 conn(0x7f2bb415a290 0x7f2bb406a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:42.835 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.832+0000 7f2bbb5b2700 1 -- 172.21.15.99:0/145467293 >> 172.21.15.99:0/145467293 conn(0x7f2bb40d1880 msgr2=0x7f2bb40cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:42.835 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.832+0000 7f2bbb5b2700 1 -- 172.21.15.99:0/145467293 shutdown_connections 2023-10-08T00:12:42.835 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.832+0000 7f2bbb5b2700 1 -- 172.21.15.99:0/145467293 wait complete. 2023-10-08T00:12:42.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.833+0000 7f2bbb5b2700 1 Processor -- start 2023-10-08T00:12:42.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.833+0000 7f2bbb5b2700 1 -- start start 2023-10-08T00:12:42.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.834+0000 7f2bbb5b2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2bb406aae0 0x7f2bb40b6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:42.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.834+0000 7f2bbb5b2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2bb4159ec0 0x7f2bb40b4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:42.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.834+0000 7f2bbb5b2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bb415a290 0x7f2bb40ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:42.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.834+0000 7f2bbb5b2700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb40b4aa0 con 0x7f2bb415a290 2023-10-08T00:12:42.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.834+0000 7f2bbb5b2700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb40ca250 con 0x7f2bb406aae0 2023-10-08T00:12:42.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.834+0000 7f2bbb5b2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb40b4cd0 con 0x7f2bb4159ec0 2023-10-08T00:12:42.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.834+0000 7f2bb9b4f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bb415a290 0x7f2bb40ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:42.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.834+0000 7f2bb9b4f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bb415a290 0x7f2bb40ca6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46202/0 (socket says 172.21.15.99:46202) 2023-10-08T00:12:42.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.834+0000 7f2bb9b4f700 1 -- 172.21.15.99:0/2213049433 learned_addr learned my addr 172.21.15.99:0/2213049433 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:42.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.834+0000 7f2bb934e700 1 --2- 172.21.15.99:0/2213049433 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2bb406aae0 0x7f2bb40b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:42.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.835+0000 7f2bb8b4d700 1 --2- 172.21.15.99:0/2213049433 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2bb4159ec0 0x7f2bb40b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:42.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.835+0000 7f2bb9b4f700 1 -- 172.21.15.99:0/2213049433 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2bb4159ec0 msgr2=0x7f2bb40b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:42.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.835+0000 7f2bb9b4f700 1 --2- 172.21.15.99:0/2213049433 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2bb4159ec0 0x7f2bb40b4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:42.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.835+0000 7f2bb9b4f700 1 -- 172.21.15.99:0/2213049433 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2bb406aae0 msgr2=0x7f2bb40b6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:42.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.835+0000 7f2bb9b4f700 1 --2- 172.21.15.99:0/2213049433 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2bb406aae0 0x7f2bb40b6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:42.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.835+0000 7f2bb9b4f700 1 -- 172.21.15.99:0/2213049433 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bac009a20 con 0x7f2bb415a290 2023-10-08T00:12:42.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.835+0000 7f2bb8b4d700 1 --2- 172.21.15.99:0/2213049433 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2bb4159ec0 0x7f2bb40b4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:12:42.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.835+0000 7f2bb934e700 1 --2- 172.21.15.99:0/2213049433 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2bb406aae0 0x7f2bb40b6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:12:42.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.835+0000 7f2bb9b4f700 1 --2- 172.21.15.99:0/2213049433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bb415a290 0x7f2bb40ca6e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2bac009fd0 tx=0x7f2bac019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:42.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.835+0000 7f2ba97fa700 1 -- 172.21.15.99:0/2213049433 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bac012e00 con 0x7f2bb415a290 2023-10-08T00:12:42.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.835+0000 7f2ba97fa700 1 -- 172.21.15.99:0/2213049433 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bac01d630 con 0x7f2bb415a290 2023-10-08T00:12:42.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.835+0000 7f2bbb5b2700 1 -- 172.21.15.99:0/2213049433 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bb40c9920 con 0x7f2bb415a290 2023-10-08T00:12:42.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.835+0000 7f2ba97fa700 1 -- 172.21.15.99:0/2213049433 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bac01d9f0 con 0x7f2bb415a290 2023-10-08T00:12:42.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.835+0000 7f2bbb5b2700 1 -- 172.21.15.99:0/2213049433 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bb40c9ee0 con 0x7f2bb415a290 2023-10-08T00:12:42.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.836+0000 7f2bbb5b2700 1 -- 172.21.15.99:0/2213049433 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bb40ca400 con 0x7f2bb415a290 2023-10-08T00:12:42.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.836+0000 7f2ba97fa700 1 -- 172.21.15.99:0/2213049433 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2bac024070 con 0x7f2bb415a290 2023-10-08T00:12:42.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.837+0000 7f2ba97fa700 1 --2- 172.21.15.99:0/2213049433 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2b98066f30 0x7f2b980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:42.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.837+0000 7f2ba97fa700 1 -- 172.21.15.99:0/2213049433 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5850+0+0 (secure 0 0 0) 0x7f2bac086d60 con 0x7f2bb415a290 2023-10-08T00:12:42.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.837+0000 7f2bb934e700 1 --2- 172.21.15.99:0/2213049433 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2b98066f30 0x7f2b980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:42.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.837+0000 7f2bb934e700 1 --2- 172.21.15.99:0/2213049433 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2b98066f30 0x7f2b980693e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2bb00011e0 tx=0x7f2bb0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:42.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:42.841+0000 7f2ba97fa700 1 -- 172.21.15.99:0/2213049433 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2bac053f40 con 0x7f2bb415a290 2023-10-08T00:12:43.020 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.019+0000 7f2bbb5b2700 1 -- 172.21.15.99:0/2213049433 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2bb404c3c0 con 0x7f2bb415a290 2023-10-08T00:12:43.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.020+0000 7f2ba97fa700 1 -- 172.21.15.99:0/2213049433 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v56) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2bac057560 con 0x7f2bb415a290 2023-10-08T00:12:43.304 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2023-10-08T00:12:43.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.022+0000 7f2bbb5b2700 1 -- 172.21.15.99:0/2213049433 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2b98066f30 msgr2=0x7f2b980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:43.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.022+0000 7f2bbb5b2700 1 --2- 172.21.15.99:0/2213049433 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2b98066f30 0x7f2b980693e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2bb00011e0 tx=0x7f2bb0009040 comp rx=0 tx=0).stop 2023-10-08T00:12:43.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.022+0000 7f2bbb5b2700 1 -- 172.21.15.99:0/2213049433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bb415a290 msgr2=0x7f2bb40ca6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:43.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.022+0000 7f2bbb5b2700 1 --2- 172.21.15.99:0/2213049433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bb415a290 0x7f2bb40ca6e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2bac009fd0 tx=0x7f2bac019000 comp rx=0 tx=0).stop 2023-10-08T00:12:43.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.022+0000 7f2bbb5b2700 1 -- 172.21.15.99:0/2213049433 shutdown_connections 2023-10-08T00:12:43.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.022+0000 7f2bbb5b2700 1 --2- 172.21.15.99:0/2213049433 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2b98066f30 0x7f2b980693e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:43.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.022+0000 7f2bbb5b2700 1 --2- 172.21.15.99:0/2213049433 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2bb406aae0 0x7f2bb40b6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:43.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.022+0000 7f2bbb5b2700 1 --2- 172.21.15.99:0/2213049433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bb415a290 0x7f2bb40ca6e0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:43.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.022+0000 7f2bbb5b2700 1 --2- 172.21.15.99:0/2213049433 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2bb4159ec0 0x7f2bb40b4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:43.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.022+0000 7f2bbb5b2700 1 -- 172.21.15.99:0/2213049433 >> 172.21.15.99:0/2213049433 conn(0x7f2bb40d1880 msgr2=0x7f2bb40c6370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:43.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.022+0000 7f2bbb5b2700 1 -- 172.21.15.99:0/2213049433 shutdown_connections 2023-10-08T00:12:43.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.022+0000 7f2bbb5b2700 1 -- 172.21.15.99:0/2213049433 wait complete. 2023-10-08T00:12:43.308 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T00:12:43.425 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.424+0000 7fcb79e61700 1 Processor -- start 2023-10-08T00:12:43.425 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.424+0000 7fcb79e61700 1 -- start start 2023-10-08T00:12:43.428 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.425+0000 7fcb79e61700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fcb74068960 0x7fcb740b1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:43.428 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.425+0000 7fcb79e61700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb7415b410 con 0x7fcb74154480 2023-10-08T00:12:43.429 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.425+0000 7fcb79e61700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb7415a800 con 0x7fcb74154850 2023-10-08T00:12:43.429 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.425+0000 7fcb79e61700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb7415aae0 con 0x7fcb74068960 2023-10-08T00:12:43.429 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.425+0000 7fcb737fe700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fcb74154480 0x7fcb74157b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59474/0 (socket says 172.21.15.99:59474) 2023-10-08T00:12:43.592 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.425+0000 7fcb737fe700 1 -- 172.21.15.99:0/2357943153 learned_addr learned my addr 172.21.15.99:0/2357943153 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:12:43.592 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.426+0000 7fcb727fc700 1 -- 172.21.15.99:0/2357943153 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2619488745 0 0) 0x7fcb7415b410 con 0x7fcb74154480 2023-10-08T00:12:43.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.426+0000 7fcb72ffd700 1 --2- 172.21.15.99:0/2357943153 >> v2:172.21.15.191:3300/0 conn(0x7fcb74068960 0x7fcb740b1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:43.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.426+0000 7fcb727fc700 1 -- 172.21.15.99:0/2357943153 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb5c003570 con 0x7fcb74154480 2023-10-08T00:12:43.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.426+0000 7fcb727fc700 1 -- 172.21.15.99:0/2357943153 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2072998212 0 0) 0x7fcb5c003570 con 0x7fcb74154480 2023-10-08T00:12:43.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.426+0000 7fcb727fc700 1 -- 172.21.15.99:0/2357943153 >> v2:172.21.15.191:3300/0 conn(0x7fcb74068960 msgr2=0x7fcb740b1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:43.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.426+0000 7fcb727fc700 1 --2- 172.21.15.99:0/2357943153 >> v2:172.21.15.191:3300/0 conn(0x7fcb74068960 0x7fcb740b1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:43.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.426+0000 7fcb727fc700 1 -- 172.21.15.99:0/2357943153 >> v1:172.21.15.191:6789/0 conn(0x7fcb74154850 legacy=0x7fcb7406e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:43.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.426+0000 7fcb727fc700 1 -- 172.21.15.99:0/2357943153 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb7415adc0 con 0x7fcb74154480 2023-10-08T00:12:43.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.426+0000 7fcb727fc700 1 -- 172.21.15.99:0/2357943153 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7fcb640017f0 con 0x7fcb74154480 2023-10-08T00:12:43.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.426+0000 7fcb727fc700 1 -- 172.21.15.99:0/2357943153 >> v1:172.21.15.99:6789/0 conn(0x7fcb74154480 legacy=0x7fcb74157b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:43.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.426+0000 7fcb727fc700 1 --2- 172.21.15.99:0/2357943153 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb5c003a60 0x7fcb5c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:43.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.426+0000 7fcb727fc700 1 --2- 172.21.15.99:0/2357943153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb5c004810 0x7fcb5c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:43.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.426+0000 7fcb727fc700 1 --2- 172.21.15.99:0/2357943153 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb5c0051c0 0x7fcb5c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:43.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.426+0000 7fcb727fc700 1 -- 172.21.15.99:0/2357943153 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7415aae0 con 0x7fcb5c004810 2023-10-08T00:12:43.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.426+0000 7fcb727fc700 1 -- 172.21.15.99:0/2357943153 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c00c160 con 0x7fcb5c003a60 2023-10-08T00:12:43.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.426+0000 7fcb727fc700 1 -- 172.21.15.99:0/2357943153 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c00c320 con 0x7fcb5c0051c0 2023-10-08T00:12:43.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.426+0000 7fcb72ffd700 1 --2- 172.21.15.99:0/2357943153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb5c004810 0x7fcb5c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:43.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.426+0000 7fcb737fe700 1 --2- 172.21.15.99:0/2357943153 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb5c003a60 0x7fcb5c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:43.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.426+0000 7fcb73fff700 1 --2- 172.21.15.99:0/2357943153 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb5c0051c0 0x7fcb5c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:43.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.426+0000 7fcb72ffd700 1 -- 172.21.15.99:0/2357943153 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb5c0051c0 msgr2=0x7fcb5c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:43.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.426+0000 7fcb72ffd700 1 --2- 172.21.15.99:0/2357943153 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb5c0051c0 0x7fcb5c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:43.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.426+0000 7fcb72ffd700 1 -- 172.21.15.99:0/2357943153 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb5c003a60 msgr2=0x7fcb5c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:43.595 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.426+0000 7fcb72ffd700 1 --2- 172.21.15.99:0/2357943153 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb5c003a60 0x7fcb5c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:43.595 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.426+0000 7fcb72ffd700 1 -- 172.21.15.99:0/2357943153 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb5c003570 con 0x7fcb5c004810 2023-10-08T00:12:43.595 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.426+0000 7fcb73fff700 1 -- 172.21.15.99:0/2357943153 reap_dead start 2023-10-08T00:12:43.595 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.426+0000 7fcb737fe700 1 --2- 172.21.15.99:0/2357943153 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb5c003a60 0x7fcb5c003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:12:43.595 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.427+0000 7fcb72ffd700 1 --2- 172.21.15.99:0/2357943153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb5c004810 0x7fcb5c004c80 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fcb68009c00 tx=0x7fcb680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47e4043e7616445 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:43.595 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.427+0000 7fcb727fc700 1 -- 172.21.15.99:0/2357943153 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6800e710 con 0x7fcb5c004810 2023-10-08T00:12:43.595 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.427+0000 7fcb727fc700 1 -- 172.21.15.99:0/2357943153 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb6800e870 con 0x7fcb5c004810 2023-10-08T00:12:43.595 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.427+0000 7fcb727fc700 1 -- 172.21.15.99:0/2357943153 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6800ec30 con 0x7fcb5c004810 2023-10-08T00:12:43.595 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.427+0000 7fcb79e61700 1 -- 172.21.15.99:0/2357943153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb5c004810 msgr2=0x7fcb5c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:43.595 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.427+0000 7fcb79e61700 1 --2- 172.21.15.99:0/2357943153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb5c004810 0x7fcb5c004c80 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fcb68009c00 tx=0x7fcb680091a0 comp rx=0 tx=0).stop 2023-10-08T00:12:43.596 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.427+0000 7fcb79e61700 1 -- 172.21.15.99:0/2357943153 shutdown_connections 2023-10-08T00:12:43.596 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.427+0000 7fcb79e61700 1 --2- 172.21.15.99:0/2357943153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb5c004810 0x7fcb5c004c80 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:43.596 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.427+0000 7fcb79e61700 1 -- 172.21.15.99:0/2357943153 >> 172.21.15.99:0/2357943153 conn(0x7fcb740ce580 msgr2=0x7fcb740af650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:43.596 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.427+0000 7fcb79e61700 1 -- 172.21.15.99:0/2357943153 shutdown_connections 2023-10-08T00:12:43.596 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.427+0000 7fcb79e61700 1 -- 172.21.15.99:0/2357943153 wait complete. 2023-10-08T00:12:43.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.428+0000 7fcb79e61700 1 Processor -- start 2023-10-08T00:12:43.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.429+0000 7fcb79e61700 1 -- start start 2023-10-08T00:12:43.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.429+0000 7fcb79e61700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb5c004810 0x7fcb740aa060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:43.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.429+0000 7fcb79e61700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb740c2960 0x7fcb741509a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:43.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.429+0000 7fcb79e61700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb74070700 0x7fcb74070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:43.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.429+0000 7fcb79e61700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb74159380 con 0x7fcb740c2960 2023-10-08T00:12:43.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.429+0000 7fcb79e61700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb740710b0 con 0x7fcb74070700 2023-10-08T00:12:43.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.429+0000 7fcb79e61700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb74159a00 con 0x7fcb5c004810 2023-10-08T00:12:43.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.429+0000 7fcb72ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb740c2960 0x7fcb741509a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:43.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.429+0000 7fcb72ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb740c2960 0x7fcb741509a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55376/0 (socket says 172.21.15.99:55376) 2023-10-08T00:12:43.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.429+0000 7fcb72ffd700 1 -- 172.21.15.99:0/561917058 learned_addr learned my addr 172.21.15.99:0/561917058 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:43.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.429+0000 7fcb737fe700 1 --2- 172.21.15.99:0/561917058 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb5c004810 0x7fcb740aa060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:43.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.429+0000 7fcb73fff700 1 --2- 172.21.15.99:0/561917058 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb74070700 0x7fcb74070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:43.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.430+0000 7fcb72ffd700 1 -- 172.21.15.99:0/561917058 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb5c004810 msgr2=0x7fcb740aa060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:43.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.430+0000 7fcb72ffd700 1 --2- 172.21.15.99:0/561917058 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb5c004810 0x7fcb740aa060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:43.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.430+0000 7fcb72ffd700 1 -- 172.21.15.99:0/561917058 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb74070700 msgr2=0x7fcb74070b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:43.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.430+0000 7fcb72ffd700 1 --2- 172.21.15.99:0/561917058 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb74070700 0x7fcb74070b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:43.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.430+0000 7fcb72ffd700 1 -- 172.21.15.99:0/561917058 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb74158dc0 con 0x7fcb740c2960 2023-10-08T00:12:43.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.430+0000 7fcb73fff700 1 --2- 172.21.15.99:0/561917058 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb74070700 0x7fcb74070b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:12:43.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.430+0000 7fcb72ffd700 1 --2- 172.21.15.99:0/561917058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb740c2960 0x7fcb741509a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c00c130 tx=0x7fcb6801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:43.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.430+0000 7fcb5b7fe700 1 -- 172.21.15.99:0/561917058 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb68022070 con 0x7fcb740c2960 2023-10-08T00:12:43.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.430+0000 7fcb5b7fe700 1 -- 172.21.15.99:0/561917058 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb6800e9e0 con 0x7fcb740c2960 2023-10-08T00:12:43.608 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.430+0000 7fcb5b7fe700 1 -- 172.21.15.99:0/561917058 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb68027470 con 0x7fcb740c2960 2023-10-08T00:12:43.608 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.430+0000 7fcb79e61700 1 -- 172.21.15.99:0/561917058 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb741590a0 con 0x7fcb740c2960 2023-10-08T00:12:43.608 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.430+0000 7fcb79e61700 1 -- 172.21.15.99:0/561917058 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb740c2dd0 con 0x7fcb740c2960 2023-10-08T00:12:43.608 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.431+0000 7fcb79e61700 1 -- 172.21.15.99:0/561917058 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb74159bb0 con 0x7fcb740c2960 2023-10-08T00:12:43.608 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.431+0000 7fcb5b7fe700 1 -- 172.21.15.99:0/561917058 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcb6801c450 con 0x7fcb740c2960 2023-10-08T00:12:43.608 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.432+0000 7fcb5b7fe700 1 --2- 172.21.15.99:0/561917058 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcb60066f30 0x7fcb600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:43.608 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.432+0000 7fcb5b7fe700 1 -- 172.21.15.99:0/561917058 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5850+0+0 (secure 0 0 0) 0x7fcb68086150 con 0x7fcb740c2960 2023-10-08T00:12:43.608 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.432+0000 7fcb737fe700 1 --2- 172.21.15.99:0/561917058 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcb60066f30 0x7fcb600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:43.609 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.432+0000 7fcb737fe700 1 --2- 172.21.15.99:0/561917058 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcb60066f30 0x7fcb600693e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcb64002060 tx=0x7fcb64006040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:43.609 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.436+0000 7fcb5b7fe700 1 -- 172.21.15.99:0/561917058 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcb74159bb0 con 0x7fcb740c2960 2023-10-08T00:12:43.617 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.616+0000 7fcb79e61700 1 -- 172.21.15.99:0/561917058 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcb740b13c0 con 0x7fcb740c2960 2023-10-08T00:12:43.617 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.616+0000 7fcb5b7fe700 1 -- 172.21.15.99:0/561917058 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v57) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcb68053330 con 0x7fcb740c2960 2023-10-08T00:12:43.617 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2023-10-08T00:12:43.619 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.618+0000 7fcb79e61700 1 -- 172.21.15.99:0/561917058 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcb60066f30 msgr2=0x7fcb600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:43.620 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.618+0000 7fcb79e61700 1 --2- 172.21.15.99:0/561917058 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcb60066f30 0x7fcb600693e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcb64002060 tx=0x7fcb64006040 comp rx=0 tx=0).stop 2023-10-08T00:12:43.620 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.618+0000 7fcb79e61700 1 -- 172.21.15.99:0/561917058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb740c2960 msgr2=0x7fcb741509a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:43.620 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.618+0000 7fcb79e61700 1 --2- 172.21.15.99:0/561917058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb740c2960 0x7fcb741509a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c00c130 tx=0x7fcb6801a000 comp rx=0 tx=0).stop 2023-10-08T00:12:43.620 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.618+0000 7fcb79e61700 1 -- 172.21.15.99:0/561917058 shutdown_connections 2023-10-08T00:12:43.620 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.618+0000 7fcb79e61700 1 --2- 172.21.15.99:0/561917058 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcb60066f30 0x7fcb600693e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:43.620 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.618+0000 7fcb79e61700 1 --2- 172.21.15.99:0/561917058 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb74070700 0x7fcb74070b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:43.621 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.618+0000 7fcb79e61700 1 --2- 172.21.15.99:0/561917058 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb5c004810 0x7fcb740aa060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:43.621 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.618+0000 7fcb79e61700 1 --2- 172.21.15.99:0/561917058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb740c2960 0x7fcb741509a0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:43.621 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.618+0000 7fcb79e61700 1 -- 172.21.15.99:0/561917058 >> 172.21.15.99:0/561917058 conn(0x7fcb740ce580 msgr2=0x7fcb740b1920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:43.621 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.618+0000 7fcb79e61700 1 -- 172.21.15.99:0/561917058 shutdown_connections 2023-10-08T00:12:43.621 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.618+0000 7fcb79e61700 1 -- 172.21.15.99:0/561917058 wait complete. 2023-10-08T00:12:43.631 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:12:43.748 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.747+0000 7fe3e5616700 1 Processor -- start 2023-10-08T00:12:43.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.747+0000 7fe3e5616700 1 -- start start 2023-10-08T00:12:43.937 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.748+0000 7fe3e5616700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fe3e015a620 0x7fe3e015d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:43.937 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.748+0000 7fe3e5616700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe3e015d580 0x7fe3e0158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:43.937 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.748+0000 7fe3e5616700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3e015cbb0 con 0x7fe3e0157870 2023-10-08T00:12:43.938 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.748+0000 7fe3e5616700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3e015bdb0 con 0x7fe3e015a620 2023-10-08T00:12:43.938 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.748+0000 7fe3e5616700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e015bfa0 con 0x7fe3e015d580 2023-10-08T00:12:43.938 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.748+0000 7fe3d7fff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fe3e015a620 0x7fe3e015d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:43.938 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.748+0000 7fe3df7fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe3e015d580 0x7fe3e0158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:43.938 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.748+0000 7fe3d7fff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fe3e015a620 0x7fe3e015d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.99:54992/0 (socket says 172.21.15.99:54992) 2023-10-08T00:12:43.939 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.748+0000 7fe3d7fff700 1 -- 172.21.15.99:0/1135167239 learned_addr learned my addr 172.21.15.99:0/1135167239 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:43.939 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.748+0000 7fe3deffd700 1 --1- 172.21.15.99:0/1135167239 >> v1:172.21.15.191:6789/0 conn(0x7fe3e0157870 0x7fe3e015b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:50398/0 (socket says 172.21.15.99:50398) 2023-10-08T00:12:43.939 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.749+0000 7fe3df7fe700 1 -- 172.21.15.99:0/1135167239 >> v2:172.21.15.191:3300/0 conn(0x7fe3e015a620 msgr2=0x7fe3e015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:43.939 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.749+0000 7fe3df7fe700 1 --2- 172.21.15.99:0/1135167239 >> v2:172.21.15.191:3300/0 conn(0x7fe3e015a620 0x7fe3e015d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:43.939 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.749+0000 7fe3df7fe700 1 -- 172.21.15.99:0/1135167239 >> v1:172.21.15.191:6789/0 conn(0x7fe3e0157870 legacy=0x7fe3e015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:43.940 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.749+0000 7fe3df7fe700 1 -- 172.21.15.99:0/1135167239 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e015c280 con 0x7fe3e015d580 2023-10-08T00:12:43.940 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.749+0000 7fe3d7fff700 1 --2- 172.21.15.99:0/1135167239 >> v2:172.21.15.191:3300/0 conn(0x7fe3e015a620 0x7fe3e015d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:12:43.940 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.749+0000 7fe3df7fe700 1 --2- 172.21.15.99:0/1135167239 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe3e015d580 0x7fe3e0158110 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8009e60 tx=0x7fe3d8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0ca91dce335557b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:43.940 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.749+0000 7fe3de7fc700 1 -- 172.21.15.99:0/1135167239 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d800e070 con 0x7fe3e015d580 2023-10-08T00:12:43.941 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.749+0000 7fe3de7fc700 1 -- 172.21.15.99:0/1135167239 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3d800bb60 con 0x7fe3e015d580 2023-10-08T00:12:43.941 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.749+0000 7fe3de7fc700 1 -- 172.21.15.99:0/1135167239 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d8012430 con 0x7fe3e015d580 2023-10-08T00:12:43.941 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.750+0000 7fe3e5616700 1 -- 172.21.15.99:0/1135167239 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe3e015d580 msgr2=0x7fe3e0158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:43.942 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.750+0000 7fe3e5616700 1 --2- 172.21.15.99:0/1135167239 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe3e015d580 0x7fe3e0158110 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8009e60 tx=0x7fe3d8009780 comp rx=0 tx=0).stop 2023-10-08T00:12:43.942 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.750+0000 7fe3e5616700 1 -- 172.21.15.99:0/1135167239 shutdown_connections 2023-10-08T00:12:43.942 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.750+0000 7fe3e5616700 1 --2- 172.21.15.99:0/1135167239 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe3e015d580 0x7fe3e0158110 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:43.942 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.750+0000 7fe3e5616700 1 --2- 172.21.15.99:0/1135167239 >> v2:172.21.15.191:3300/0 conn(0x7fe3e015a620 0x7fe3e015d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:43.942 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.750+0000 7fe3e5616700 1 -- 172.21.15.99:0/1135167239 >> 172.21.15.99:0/1135167239 conn(0x7fe3e0068ce0 msgr2=0x7fe3e00690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:43.942 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.750+0000 7fe3e5616700 1 -- 172.21.15.99:0/1135167239 shutdown_connections 2023-10-08T00:12:43.942 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.750+0000 7fe3e5616700 1 -- 172.21.15.99:0/1135167239 wait complete. 2023-10-08T00:12:43.943 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.751+0000 7fe3e5616700 1 Processor -- start 2023-10-08T00:12:43.943 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.751+0000 7fe3e5616700 1 -- start start 2023-10-08T00:12:43.943 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.752+0000 7fe3e5616700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3e0157870 0x7fe3e014bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:43.943 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.752+0000 7fe3e5616700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe3e015a620 0x7fe3e0157e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:43.943 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.752+0000 7fe3e5616700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe3e015d580 0x7fe3e0150980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:43.943 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.752+0000 7fe3e5616700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e00b0ad0 con 0x7fe3e0157870 2023-10-08T00:12:43.943 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.752+0000 7fe3e5616700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e01587f0 con 0x7fe3e015d580 2023-10-08T00:12:43.943 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.752+0000 7fe3e5616700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e0158ad0 con 0x7fe3e015a620 2023-10-08T00:12:43.943 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.752+0000 7fe3deffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3e0157870 0x7fe3e014bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:43.944 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.752+0000 7fe3deffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3e0157870 0x7fe3e014bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55384/0 (socket says 172.21.15.99:55384) 2023-10-08T00:12:43.944 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.752+0000 7fe3deffd700 1 -- 172.21.15.99:0/1843895281 learned_addr learned my addr 172.21.15.99:0/1843895281 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:43.944 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.752+0000 7fe3df7fe700 1 --2- 172.21.15.99:0/1843895281 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe3e015d580 0x7fe3e0150980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:43.944 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.752+0000 7fe3d7fff700 1 --2- 172.21.15.99:0/1843895281 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe3e015a620 0x7fe3e0157e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:43.944 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.752+0000 7fe3deffd700 1 -- 172.21.15.99:0/1843895281 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe3e015a620 msgr2=0x7fe3e0157e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:43.945 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.752+0000 7fe3deffd700 1 --2- 172.21.15.99:0/1843895281 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe3e015a620 0x7fe3e0157e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:43.945 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.752+0000 7fe3deffd700 1 -- 172.21.15.99:0/1843895281 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe3e015d580 msgr2=0x7fe3e0150980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:43.945 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.752+0000 7fe3deffd700 1 --2- 172.21.15.99:0/1843895281 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe3e015d580 0x7fe3e0150980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:43.946 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.752+0000 7fe3deffd700 1 -- 172.21.15.99:0/1843895281 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c80094f0 con 0x7fe3e0157870 2023-10-08T00:12:43.946 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.752+0000 7fe3deffd700 1 --2- 172.21.15.99:0/1843895281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3e0157870 0x7fe3e014bbe0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fe3d000c7c0 tx=0x7fe3d0007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:43.946 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.752+0000 7fe3d6ffd700 1 -- 172.21.15.99:0/1843895281 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d0005a70 con 0x7fe3e0157870 2023-10-08T00:12:43.946 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.752+0000 7fe3d6ffd700 1 -- 172.21.15.99:0/1843895281 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3d0005bd0 con 0x7fe3e0157870 2023-10-08T00:12:43.947 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.753+0000 7fe3e5616700 1 -- 172.21.15.99:0/1843895281 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3d8009a20 con 0x7fe3e0157870 2023-10-08T00:12:43.947 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.753+0000 7fe3d6ffd700 1 -- 172.21.15.99:0/1843895281 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d00070d0 con 0x7fe3e0157870 2023-10-08T00:12:43.947 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.753+0000 7fe3e5616700 1 -- 172.21.15.99:0/1843895281 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e00b0760 con 0x7fe3e0157870 2023-10-08T00:12:43.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.753+0000 7fe3e5616700 1 -- 172.21.15.99:0/1843895281 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3e004c3c0 con 0x7fe3e0157870 2023-10-08T00:12:43.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.754+0000 7fe3d6ffd700 1 -- 172.21.15.99:0/1843895281 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe3d0007480 con 0x7fe3e0157870 2023-10-08T00:12:43.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.754+0000 7fe3d6ffd700 1 --2- 172.21.15.99:0/1843895281 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe3c0066f30 0x7fe3c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:43.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.754+0000 7fe3d6ffd700 1 -- 172.21.15.99:0/1843895281 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5850+0+0 (secure 0 0 0) 0x7fe3d00839d0 con 0x7fe3e0157870 2023-10-08T00:12:43.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.754+0000 7fe3d7fff700 1 --2- 172.21.15.99:0/1843895281 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe3c0066f30 0x7fe3c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:43.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.755+0000 7fe3d7fff700 1 --2- 172.21.15.99:0/1843895281 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe3c0066f30 0x7fe3c00693e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0157200 tx=0x7fe3c8009250 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:43.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.758+0000 7fe3d6ffd700 1 -- 172.21.15.99:0/1843895281 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe3d0050bb0 con 0x7fe3e0157870 2023-10-08T00:12:43.980 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.980+0000 7fe3e5616700 1 -- 172.21.15.99:0/1843895281 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe3e015c560 con 0x7fe3e0157870 2023-10-08T00:12:43.981 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.980+0000 7fe3d6ffd700 1 -- 172.21.15.99:0/1843895281 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4603 (secure 0 0 0) 0x7fe3d0017020 con 0x7fe3e0157870 2023-10-08T00:12:43.981 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:12:43.981 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5146,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/1438931938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1438931938},{"type":"v1","addr":"172.21.15.191:6836","nonce":1438931938}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5280,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/3090674680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3090674680},{"type":"v1","addr":"172.21.15.99:6833","nonce":3090674680}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5283,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/4025282357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":4025282357},{"type":"v1","addr":"172.21.15.99:6835","nonce":4025282357}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:12:09.803824+0000","modified":"2023-10-08T00:12:13.436570+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5289},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5289":{"gid":5289,"name":"d","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/2062242133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":2062242133},{"type":"v1","addr":"172.21.15.191:6837","nonce":2062242133}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-08T00:12:43.983 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.982+0000 7fe3e5616700 1 -- 172.21.15.99:0/1843895281 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe3c0066f30 msgr2=0x7fe3c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:43.983 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.982+0000 7fe3e5616700 1 --2- 172.21.15.99:0/1843895281 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe3c0066f30 0x7fe3c00693e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0157200 tx=0x7fe3c8009250 comp rx=0 tx=0).stop 2023-10-08T00:12:43.983 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.982+0000 7fe3e5616700 1 -- 172.21.15.99:0/1843895281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3e0157870 msgr2=0x7fe3e014bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:43.983 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.982+0000 7fe3e5616700 1 --2- 172.21.15.99:0/1843895281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3e0157870 0x7fe3e014bbe0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fe3d000c7c0 tx=0x7fe3d0007f70 comp rx=0 tx=0).stop 2023-10-08T00:12:43.984 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.982+0000 7fe3e5616700 1 -- 172.21.15.99:0/1843895281 shutdown_connections 2023-10-08T00:12:43.984 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.982+0000 7fe3e5616700 1 --2- 172.21.15.99:0/1843895281 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe3c0066f30 0x7fe3c00693e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:43.984 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.982+0000 7fe3e5616700 1 --2- 172.21.15.99:0/1843895281 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe3e015d580 0x7fe3e0150980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:43.984 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.982+0000 7fe3e5616700 1 --2- 172.21.15.99:0/1843895281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3e0157870 0x7fe3e014bbe0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:43.984 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.982+0000 7fe3e5616700 1 --2- 172.21.15.99:0/1843895281 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe3e015a620 0x7fe3e0157e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:43.984 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.982+0000 7fe3e5616700 1 -- 172.21.15.99:0/1843895281 >> 172.21.15.99:0/1843895281 conn(0x7fe3e0068ce0 msgr2=0x7fe3e00b5990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:43.984 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.982+0000 7fe3e5616700 1 -- 172.21.15.99:0/1843895281 shutdown_connections 2023-10-08T00:12:43.984 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:43.982+0000 7fe3e5616700 1 -- 172.21.15.99:0/1843895281 wait complete. 2023-10-08T00:12:43.984 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 24 2023-10-08T00:12:43.995 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:12:44.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.108+0000 7fb52cc6f700 1 Processor -- start 2023-10-08T00:12:44.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.108+0000 7fb52cc6f700 1 -- start start 2023-10-08T00:12:44.542 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.109+0000 7fb52cc6f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb528157870 0x7fb52815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:44.543 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.109+0000 7fb52cc6f700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb528157c40 0x7fb528158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:44.544 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.109+0000 7fb52cc6f700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb52815cbb0 con 0x7fb52815a620 2023-10-08T00:12:44.544 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.109+0000 7fb52cc6f700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb52815bfa0 con 0x7fb528157c40 2023-10-08T00:12:44.544 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.109+0000 7fb52cc6f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb52815c280 con 0x7fb528157870 2023-10-08T00:12:44.544 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.109+0000 7fb526d9d700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb528157c40 0x7fb528158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:44.544 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.109+0000 7fb51dd9b700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fb52815a620 0x7fb52815d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59482/0 (socket says 172.21.15.99:59482) 2023-10-08T00:12:44.544 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.109+0000 7fb51dd9b700 1 -- 172.21.15.99:0/1156325624 learned_addr learned my addr 172.21.15.99:0/1156325624 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:12:44.544 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.109+0000 7fb52659c700 1 --2- 172.21.15.99:0/1156325624 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb528157870 0x7fb52815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:44.544 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.109+0000 7fb526d9d700 1 -- 172.21.15.99:0/1156325624 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb528157870 msgr2=0x7fb52815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:44.545 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.109+0000 7fb526d9d700 1 --2- 172.21.15.99:0/1156325624 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb528157870 0x7fb52815b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:44.545 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.109+0000 7fb526d9d700 1 -- 172.21.15.99:0/1156325624 >> v1:172.21.15.99:6789/0 conn(0x7fb52815a620 legacy=0x7fb52815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:44.545 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.109+0000 7fb525d9b700 1 -- 172.21.15.99:0/1156325624 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2535637561 0 0) 0x7fb52815cbb0 con 0x7fb52815a620 2023-10-08T00:12:44.545 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.109+0000 7fb526d9d700 1 -- 172.21.15.99:0/1156325624 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb52815c560 con 0x7fb528157c40 2023-10-08T00:12:44.545 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.109+0000 7fb52659c700 1 --2- 172.21.15.99:0/1156325624 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb528157870 0x7fb52815b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:12:44.545 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.109+0000 7fb526d9d700 1 --2- 172.21.15.99:0/1156325624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb528157c40 0x7fb528158110 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb520009e60 tx=0x7fb520009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=141af63f9bce033 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:44.545 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.110+0000 7fb525d9b700 1 -- 172.21.15.99:0/1156325624 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52000e070 con 0x7fb528157c40 2023-10-08T00:12:44.545 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.110+0000 7fb525d9b700 1 -- 172.21.15.99:0/1156325624 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb52000bc30 con 0x7fb528157c40 2023-10-08T00:12:44.545 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.110+0000 7fb525d9b700 1 -- 172.21.15.99:0/1156325624 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb520012430 con 0x7fb528157c40 2023-10-08T00:12:44.545 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.110+0000 7fb52cc6f700 1 -- 172.21.15.99:0/1156325624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb528157c40 msgr2=0x7fb528158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:44.545 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.110+0000 7fb52cc6f700 1 --2- 172.21.15.99:0/1156325624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb528157c40 0x7fb528158110 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb520009e60 tx=0x7fb520009780 comp rx=0 tx=0).stop 2023-10-08T00:12:44.545 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.110+0000 7fb52cc6f700 1 -- 172.21.15.99:0/1156325624 shutdown_connections 2023-10-08T00:12:44.546 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.110+0000 7fb52cc6f700 1 --2- 172.21.15.99:0/1156325624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb528157c40 0x7fb528158110 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:44.546 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.110+0000 7fb52cc6f700 1 --2- 172.21.15.99:0/1156325624 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb528157870 0x7fb52815b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:44.546 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.110+0000 7fb52cc6f700 1 -- 172.21.15.99:0/1156325624 >> 172.21.15.99:0/1156325624 conn(0x7fb528068ce0 msgr2=0x7fb5280690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:44.546 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.110+0000 7fb52cc6f700 1 -- 172.21.15.99:0/1156325624 shutdown_connections 2023-10-08T00:12:44.546 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.110+0000 7fb52cc6f700 1 -- 172.21.15.99:0/1156325624 wait complete. 2023-10-08T00:12:44.546 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.111+0000 7fb52cc6f700 1 Processor -- start 2023-10-08T00:12:44.546 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.111+0000 7fb52cc6f700 1 -- start start 2023-10-08T00:12:44.547 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.112+0000 7fb52cc6f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb528157870 0x7fb5281433b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:44.547 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.112+0000 7fb52cc6f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb528157c40 0x7fb5280b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:44.547 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.112+0000 7fb52cc6f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb52815a620 0x7fb52815d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:44.547 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.112+0000 7fb52cc6f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52815e400 con 0x7fb52815a620 2023-10-08T00:12:44.547 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.112+0000 7fb52cc6f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52815e6e0 con 0x7fb528157c40 2023-10-08T00:12:44.547 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.112+0000 7fb52cc6f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb52815e9c0 con 0x7fb528157870 2023-10-08T00:12:44.547 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.112+0000 7fb526d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb52815a620 0x7fb52815d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:44.547 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.112+0000 7fb526d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb52815a620 0x7fb52815d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55404/0 (socket says 172.21.15.99:55404) 2023-10-08T00:12:44.547 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.112+0000 7fb526d9d700 1 -- 172.21.15.99:0/324830343 learned_addr learned my addr 172.21.15.99:0/324830343 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:44.547 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.112+0000 7fb51dd9b700 1 --2- 172.21.15.99:0/324830343 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb528157c40 0x7fb5280b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:44.548 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.112+0000 7fb52659c700 1 --2- 172.21.15.99:0/324830343 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb528157870 0x7fb5281433b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:44.548 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.112+0000 7fb526d9d700 1 -- 172.21.15.99:0/324830343 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb528157870 msgr2=0x7fb5281433b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:44.548 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.112+0000 7fb526d9d700 1 --2- 172.21.15.99:0/324830343 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb528157870 0x7fb5281433b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:44.548 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.112+0000 7fb526d9d700 1 -- 172.21.15.99:0/324830343 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb528157c40 msgr2=0x7fb5280b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:44.548 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.112+0000 7fb526d9d700 1 --2- 172.21.15.99:0/324830343 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb528157c40 0x7fb5280b2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:44.548 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.112+0000 7fb526d9d700 1 -- 172.21.15.99:0/324830343 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb520009a20 con 0x7fb52815a620 2023-10-08T00:12:44.548 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.113+0000 7fb526d9d700 1 --2- 172.21.15.99:0/324830343 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb52815a620 0x7fb52815d6f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb52806b530 tx=0x7fb520019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:44.548 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.113+0000 7fb51effd700 1 -- 172.21.15.99:0/324830343 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52001d630 con 0x7fb52815a620 2023-10-08T00:12:44.548 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.113+0000 7fb51effd700 1 -- 172.21.15.99:0/324830343 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb52001d7b0 con 0x7fb52815a620 2023-10-08T00:12:44.548 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.113+0000 7fb52cc6f700 1 -- 172.21.15.99:0/324830343 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb52815eca0 con 0x7fb52815a620 2023-10-08T00:12:44.548 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.113+0000 7fb51effd700 1 -- 172.21.15.99:0/324830343 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52001db70 con 0x7fb52815a620 2023-10-08T00:12:44.548 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.113+0000 7fb52cc6f700 1 -- 172.21.15.99:0/324830343 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5281438f0 con 0x7fb52815a620 2023-10-08T00:12:44.548 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.114+0000 7fb52cc6f700 1 -- 172.21.15.99:0/324830343 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb52804c3c0 con 0x7fb52815a620 2023-10-08T00:12:44.549 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.114+0000 7fb51effd700 1 -- 172.21.15.99:0/324830343 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb520024070 con 0x7fb52815a620 2023-10-08T00:12:44.549 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.114+0000 7fb51effd700 1 --2- 172.21.15.99:0/324830343 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb508066f30 0x7fb5080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:44.549 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.115+0000 7fb51effd700 1 -- 172.21.15.99:0/324830343 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5850+0+0 (secure 0 0 0) 0x7fb520086f50 con 0x7fb52815a620 2023-10-08T00:12:44.549 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.115+0000 7fb52659c700 1 --2- 172.21.15.99:0/324830343 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb508066f30 0x7fb5080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:44.549 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.115+0000 7fb52659c700 1 --2- 172.21.15.99:0/324830343 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb508066f30 0x7fb5080693e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb518008a50 tx=0x7fb518008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:44.549 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.119+0000 7fb51effd700 1 -- 172.21.15.99:0/324830343 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb520054130 con 0x7fb52815a620 2023-10-08T00:12:44.549 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.305+0000 7fb52cc6f700 1 -- 172.21.15.99:0/324830343 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb52815c840 con 0x7fb52815a620 2023-10-08T00:12:44.549 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.306+0000 7fb51effd700 1 -- 172.21.15.99:0/324830343 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+15586 (secure 0 0 0) 0x7fb520057750 con 0x7fb52815a620 2023-10-08T00:12:44.550 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:12:44.550 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":57,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:12:43.142481+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:12:07.733432+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-08T00:12:08.762879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.8","mappings":[{"from":5,"to":7}]},{"pgid":"7.15","mappings":[{"from":5,"to":0}]},{"pgid":"7.28","mappings":[{"from":5,"to":7}]},{"pgid":"7.2b","mappings":[{"from":5,"to":7}]},{"pgid":"7.33","mappings":[{"from":5,"to":7}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:12:44.551 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.308+0000 7fb52cc6f700 1 -- 172.21.15.99:0/324830343 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb508066f30 msgr2=0x7fb5080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:44.551 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.308+0000 7fb52cc6f700 1 --2- 172.21.15.99:0/324830343 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb508066f30 0x7fb5080693e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb518008a50 tx=0x7fb518008040 comp rx=0 tx=0).stop 2023-10-08T00:12:44.551 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.308+0000 7fb52cc6f700 1 -- 172.21.15.99:0/324830343 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb52815a620 msgr2=0x7fb52815d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:44.551 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.308+0000 7fb52cc6f700 1 --2- 172.21.15.99:0/324830343 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb52815a620 0x7fb52815d6f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb52806b530 tx=0x7fb520019000 comp rx=0 tx=0).stop 2023-10-08T00:12:44.551 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.308+0000 7fb52cc6f700 1 -- 172.21.15.99:0/324830343 shutdown_connections 2023-10-08T00:12:44.552 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.308+0000 7fb52cc6f700 1 --2- 172.21.15.99:0/324830343 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb508066f30 0x7fb5080693e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:44.552 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.308+0000 7fb52cc6f700 1 --2- 172.21.15.99:0/324830343 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb52815a620 0x7fb52815d6f0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:44.552 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.308+0000 7fb52cc6f700 1 --2- 172.21.15.99:0/324830343 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb528157870 0x7fb5281433b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:44.552 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.308+0000 7fb52cc6f700 1 --2- 172.21.15.99:0/324830343 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb528157c40 0x7fb5280b2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:44.552 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.308+0000 7fb52cc6f700 1 -- 172.21.15.99:0/324830343 >> 172.21.15.99:0/324830343 conn(0x7fb528068ce0 msgr2=0x7fb5280cfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:44.552 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.308+0000 7fb52cc6f700 1 -- 172.21.15.99:0/324830343 shutdown_connections 2023-10-08T00:12:44.552 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.308+0000 7fb52cc6f700 1 -- 172.21.15.99:0/324830343 wait complete. 2023-10-08T00:12:44.553 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T00:12:44.553 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:12:44.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.667+0000 7fb235ec6700 1 Processor -- start 2023-10-08T00:12:44.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.667+0000 7fb235ec6700 1 -- start start 2023-10-08T00:12:44.671 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.668+0000 7fb235ec6700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb2301568d0 0x7fb230069e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:44.671 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.668+0000 7fb235ec6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2300c87a0 0x7fb23006a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:44.671 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.668+0000 7fb235ec6700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb230067ba0 con 0x7fb2300ad9b0 2023-10-08T00:12:44.671 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.668+0000 7fb235ec6700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb23015aa90 con 0x7fb2301568d0 2023-10-08T00:12:44.671 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.668+0000 7fb235ec6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb230067870 con 0x7fb2300c87a0 2023-10-08T00:12:44.671 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.668+0000 7fb22f7fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb2301568d0 0x7fb230069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:44.671 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.668+0000 7fb22f7fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb2301568d0 0x7fb230069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55420/0 (socket says 172.21.15.99:55420) 2023-10-08T00:12:44.671 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.668+0000 7fb22f7fe700 1 -- 172.21.15.99:0/4143413261 learned_addr learned my addr 172.21.15.99:0/4143413261 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:44.671 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.668+0000 7fb22ffff700 1 --2- 172.21.15.99:0/4143413261 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2300c87a0 0x7fb23006a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:44.671 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.669+0000 7fb22f7fe700 1 -- 172.21.15.99:0/4143413261 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2300c87a0 msgr2=0x7fb23006a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:44.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.669+0000 7fb22f7fe700 1 --2- 172.21.15.99:0/4143413261 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2300c87a0 0x7fb23006a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:44.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.669+0000 7fb22f7fe700 1 -- 172.21.15.99:0/4143413261 >> v1:172.21.15.191:6789/0 conn(0x7fb2300ad9b0 legacy=0x7fb2300c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:44.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.669+0000 7fb22f7fe700 1 -- 172.21.15.99:0/4143413261 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2300c8ca0 con 0x7fb2301568d0 2023-10-08T00:12:44.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.669+0000 7fb22f7fe700 1 --2- 172.21.15.99:0/4143413261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2301568d0 0x7fb230069e20 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb220009f20 tx=0x7fb220009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e52abba51c6369e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:44.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.669+0000 7fb22e7fc700 1 -- 172.21.15.99:0/4143413261 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22000e070 con 0x7fb2301568d0 2023-10-08T00:12:44.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.669+0000 7fb22e7fc700 1 -- 172.21.15.99:0/4143413261 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb22000b9d0 con 0x7fb2301568d0 2023-10-08T00:12:44.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.669+0000 7fb22e7fc700 1 -- 172.21.15.99:0/4143413261 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2200092e0 con 0x7fb2301568d0 2023-10-08T00:12:44.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.669+0000 7fb235ec6700 1 -- 172.21.15.99:0/4143413261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2301568d0 msgr2=0x7fb230069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:44.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.669+0000 7fb235ec6700 1 --2- 172.21.15.99:0/4143413261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2301568d0 0x7fb230069e20 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb220009f20 tx=0x7fb220009840 comp rx=0 tx=0).stop 2023-10-08T00:12:44.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.669+0000 7fb235ec6700 1 -- 172.21.15.99:0/4143413261 shutdown_connections 2023-10-08T00:12:44.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.669+0000 7fb235ec6700 1 --2- 172.21.15.99:0/4143413261 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2300c87a0 0x7fb23006a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:44.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.669+0000 7fb235ec6700 1 --2- 172.21.15.99:0/4143413261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2301568d0 0x7fb230069e20 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:44.673 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.669+0000 7fb235ec6700 1 -- 172.21.15.99:0/4143413261 >> 172.21.15.99:0/4143413261 conn(0x7fb2300cda50 msgr2=0x7fb2300cde50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:44.673 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.669+0000 7fb235ec6700 1 -- 172.21.15.99:0/4143413261 shutdown_connections 2023-10-08T00:12:44.673 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.670+0000 7fb235ec6700 1 -- 172.21.15.99:0/4143413261 wait complete. 2023-10-08T00:12:44.676 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.671+0000 7fb235ec6700 1 Processor -- start 2023-10-08T00:12:44.676 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.671+0000 7fb235ec6700 1 -- start start 2023-10-08T00:12:44.676 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.671+0000 7fb235ec6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2300ad9b0 0x7fb2300c3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:44.676 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.671+0000 7fb235ec6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2300c87a0 0x7fb23016aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:44.676 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.671+0000 7fb235ec6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2301568d0 0x7fb230153ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:44.677 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.671+0000 7fb235ec6700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2300c1550 con 0x7fb2301568d0 2023-10-08T00:12:44.677 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.671+0000 7fb235ec6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb23016b010 con 0x7fb2300ad9b0 2023-10-08T00:12:44.677 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.671+0000 7fb235ec6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb23016b2b0 con 0x7fb2300c87a0 2023-10-08T00:12:44.677 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.672+0000 7fb22ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2301568d0 0x7fb230153ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:44.677 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.672+0000 7fb22ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2301568d0 0x7fb230153ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55426/0 (socket says 172.21.15.99:55426) 2023-10-08T00:12:44.677 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.672+0000 7fb22ffff700 1 -- 172.21.15.99:0/4242124486 learned_addr learned my addr 172.21.15.99:0/4242124486 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:44.677 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.672+0000 7fb22effd700 1 --2- 172.21.15.99:0/4242124486 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2300c87a0 0x7fb23016aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:44.677 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.672+0000 7fb22f7fe700 1 --2- 172.21.15.99:0/4242124486 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2300ad9b0 0x7fb2300c3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:44.677 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.672+0000 7fb22ffff700 1 -- 172.21.15.99:0/4242124486 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2300c87a0 msgr2=0x7fb23016aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:44.677 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.672+0000 7fb22ffff700 1 --2- 172.21.15.99:0/4242124486 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2300c87a0 0x7fb23016aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:44.677 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.672+0000 7fb22ffff700 1 -- 172.21.15.99:0/4242124486 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2300ad9b0 msgr2=0x7fb2300c3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:44.678 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.672+0000 7fb22ffff700 1 --2- 172.21.15.99:0/4242124486 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2300ad9b0 0x7fb2300c3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:44.678 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.672+0000 7fb22ffff700 1 -- 172.21.15.99:0/4242124486 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb220009ae0 con 0x7fb2301568d0 2023-10-08T00:12:44.678 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.672+0000 7fb22effd700 1 --2- 172.21.15.99:0/4242124486 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2300c87a0 0x7fb23016aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:12:44.678 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.672+0000 7fb22ffff700 1 --2- 172.21.15.99:0/4242124486 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2301568d0 0x7fb230153ec0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb22800ea50 tx=0x7fb2280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:44.678 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.672+0000 7fb2177fe700 1 -- 172.21.15.99:0/4242124486 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22800c920 con 0x7fb2301568d0 2023-10-08T00:12:44.678 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.672+0000 7fb2177fe700 1 -- 172.21.15.99:0/4242124486 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb22800caa0 con 0x7fb2301568d0 2023-10-08T00:12:44.678 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.672+0000 7fb235ec6700 1 -- 172.21.15.99:0/4242124486 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb23016b590 con 0x7fb2301568d0 2023-10-08T00:12:44.678 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.672+0000 7fb2177fe700 1 -- 172.21.15.99:0/4242124486 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22800ce60 con 0x7fb2301568d0 2023-10-08T00:12:44.678 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.672+0000 7fb235ec6700 1 -- 172.21.15.99:0/4242124486 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2300c17a0 con 0x7fb2301568d0 2023-10-08T00:12:44.678 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.673+0000 7fb235ec6700 1 -- 172.21.15.99:0/4242124486 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2300c19c0 con 0x7fb2301568d0 2023-10-08T00:12:44.678 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.673+0000 7fb2177fe700 1 -- 172.21.15.99:0/4242124486 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb228011660 con 0x7fb2301568d0 2023-10-08T00:12:44.679 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.674+0000 7fb2177fe700 1 --2- 172.21.15.99:0/4242124486 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb21c066f30 0x7fb21c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:44.679 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.674+0000 7fb2177fe700 1 -- 172.21.15.99:0/4242124486 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5850+0+0 (secure 0 0 0) 0x7fb2280865e0 con 0x7fb2301568d0 2023-10-08T00:12:44.679 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.674+0000 7fb22f7fe700 1 --2- 172.21.15.99:0/4242124486 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb21c066f30 0x7fb21c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:44.679 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.675+0000 7fb22f7fe700 1 --2- 172.21.15.99:0/4242124486 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb21c066f30 0x7fb21c0693e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb220009880 tx=0x7fb220018000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:44.679 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.678+0000 7fb2177fe700 1 -- 172.21.15.99:0/4242124486 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb2280537c0 con 0x7fb2301568d0 2023-10-08T00:12:44.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.901+0000 7fb235ec6700 1 -- 172.21.15.99:0/4242124486 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb23004c3c0 con 0x7fb2301568d0 2023-10-08T00:12:44.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.901+0000 7fb2177fe700 1 -- 172.21.15.99:0/4242124486 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4603 (secure 0 0 0) 0x7fb228056de0 con 0x7fb2301568d0 2023-10-08T00:12:44.902 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:12:44.902 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5146,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/1438931938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1438931938},{"type":"v1","addr":"172.21.15.191:6836","nonce":1438931938}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5280,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/3090674680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3090674680},{"type":"v1","addr":"172.21.15.99:6833","nonce":3090674680}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5283,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/4025282357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":4025282357},{"type":"v1","addr":"172.21.15.99:6835","nonce":4025282357}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:12:09.803824+0000","modified":"2023-10-08T00:12:13.436570+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5289},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5289":{"gid":5289,"name":"d","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/2062242133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":2062242133},{"type":"v1","addr":"172.21.15.191:6837","nonce":2062242133}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-08T00:12:44.904 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.903+0000 7fb235ec6700 1 -- 172.21.15.99:0/4242124486 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb21c066f30 msgr2=0x7fb21c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:44.904 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.903+0000 7fb235ec6700 1 --2- 172.21.15.99:0/4242124486 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb21c066f30 0x7fb21c0693e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb220009880 tx=0x7fb220018000 comp rx=0 tx=0).stop 2023-10-08T00:12:44.904 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.903+0000 7fb235ec6700 1 -- 172.21.15.99:0/4242124486 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2301568d0 msgr2=0x7fb230153ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:44.904 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.903+0000 7fb235ec6700 1 --2- 172.21.15.99:0/4242124486 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2301568d0 0x7fb230153ec0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb22800ea50 tx=0x7fb2280091a0 comp rx=0 tx=0).stop 2023-10-08T00:12:44.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.903+0000 7fb235ec6700 1 -- 172.21.15.99:0/4242124486 shutdown_connections 2023-10-08T00:12:44.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.903+0000 7fb235ec6700 1 --2- 172.21.15.99:0/4242124486 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb21c066f30 0x7fb21c0693e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:44.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.903+0000 7fb235ec6700 1 --2- 172.21.15.99:0/4242124486 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2300ad9b0 0x7fb2300c3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:44.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.903+0000 7fb235ec6700 1 --2- 172.21.15.99:0/4242124486 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2301568d0 0x7fb230153ec0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:44.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.903+0000 7fb235ec6700 1 --2- 172.21.15.99:0/4242124486 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2300c87a0 0x7fb23016aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:44.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.903+0000 7fb235ec6700 1 -- 172.21.15.99:0/4242124486 >> 172.21.15.99:0/4242124486 conn(0x7fb2300cda50 msgr2=0x7fb2300ade30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:44.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.903+0000 7fb235ec6700 1 -- 172.21.15.99:0/4242124486 shutdown_connections 2023-10-08T00:12:44.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:44.903+0000 7fb235ec6700 1 -- 172.21.15.99:0/4242124486 wait complete. 2023-10-08T00:12:44.905 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 24 2023-10-08T00:12:44.916 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:12:45.031 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.030+0000 7fd898c44700 1 Processor -- start 2023-10-08T00:12:45.031 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.030+0000 7fd898c44700 1 -- start start 2023-10-08T00:12:45.035 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.031+0000 7fd898c44700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd894159500 0x7fd8941598d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:45.035 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.031+0000 7fd898c44700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8940c7f50 0x7fd89406a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:45.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.031+0000 7fd898c44700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8940cbd10 con 0x7fd8940a9db0 2023-10-08T00:12:45.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.031+0000 7fd898c44700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8940cb100 con 0x7fd894159500 2023-10-08T00:12:45.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.031+0000 7fd898c44700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8940cb3e0 con 0x7fd8940c7f50 2023-10-08T00:12:45.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.031+0000 7fd889d9b700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fd8940a9db0 0x7fd8940c7840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59490/0 (socket says 172.21.15.99:59490) 2023-10-08T00:12:45.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.031+0000 7fd889d9b700 1 -- 172.21.15.99:0/3676255653 learned_addr learned my addr 172.21.15.99:0/3676255653 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:12:45.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.032+0000 7fd89259c700 1 --2- 172.21.15.99:0/3676255653 >> v2:172.21.15.191:3300/0 conn(0x7fd894159500 0x7fd8941598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:45.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.032+0000 7fd892d9d700 1 --2- 172.21.15.99:0/3676255653 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8940c7f50 0x7fd89406a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:45.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.032+0000 7fd891d9b700 1 -- 172.21.15.99:0/3676255653 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1298669810 0 0) 0x7fd8940cbd10 con 0x7fd8940a9db0 2023-10-08T00:12:45.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.032+0000 7fd891d9b700 1 -- 172.21.15.99:0/3676255653 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd880003570 con 0x7fd8940a9db0 2023-10-08T00:12:45.039 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.032+0000 7fd891d9b700 1 -- 172.21.15.99:0/3676255653 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 875739112 0 0) 0x7fd880003570 con 0x7fd8940a9db0 2023-10-08T00:12:45.039 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.032+0000 7fd891d9b700 1 -- 172.21.15.99:0/3676255653 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8940c7f50 msgr2=0x7fd89406a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:45.039 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.032+0000 7fd891d9b700 1 --2- 172.21.15.99:0/3676255653 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8940c7f50 0x7fd89406a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:45.039 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.032+0000 7fd891d9b700 1 -- 172.21.15.99:0/3676255653 >> v2:172.21.15.191:3300/0 conn(0x7fd894159500 msgr2=0x7fd8941598d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:45.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.032+0000 7fd891d9b700 1 --2- 172.21.15.99:0/3676255653 >> v2:172.21.15.191:3300/0 conn(0x7fd894159500 0x7fd8941598d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:45.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.032+0000 7fd891d9b700 1 -- 172.21.15.99:0/3676255653 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8940cb6c0 con 0x7fd8940a9db0 2023-10-08T00:12:45.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.032+0000 7fd891d9b700 1 -- 172.21.15.99:0/3676255653 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7fd87c001470 con 0x7fd8940a9db0 2023-10-08T00:12:45.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.032+0000 7fd891d9b700 1 -- 172.21.15.99:0/3676255653 >> v1:172.21.15.99:6789/0 conn(0x7fd8940a9db0 legacy=0x7fd8940c7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:45.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.032+0000 7fd891d9b700 1 --2- 172.21.15.99:0/3676255653 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd880003a60 0x7fd880003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:45.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.032+0000 7fd891d9b700 1 --2- 172.21.15.99:0/3676255653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd880004810 0x7fd880004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:45.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.032+0000 7fd891d9b700 1 --2- 172.21.15.99:0/3676255653 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8800051c0 0x7fd88000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:45.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.032+0000 7fd891d9b700 1 -- 172.21.15.99:0/3676255653 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8940cb100 con 0x7fd880004810 2023-10-08T00:12:45.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.032+0000 7fd891d9b700 1 -- 172.21.15.99:0/3676255653 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8940cb3e0 con 0x7fd8800051c0 2023-10-08T00:12:45.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.032+0000 7fd891d9b700 1 -- 172.21.15.99:0/3676255653 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd88000bfa0 con 0x7fd880003a60 2023-10-08T00:12:45.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.032+0000 7fd892d9d700 1 --2- 172.21.15.99:0/3676255653 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8940c7f50 0x7fd89406a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:12:45.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.032+0000 7fd89259c700 1 --2- 172.21.15.99:0/3676255653 >> v2:172.21.15.191:3300/0 conn(0x7fd894159500 0x7fd8941598d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:12:45.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.032+0000 7fd889d9b700 1 --2- 172.21.15.99:0/3676255653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd880004810 0x7fd880004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:45.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.032+0000 7fd89259c700 1 --2- 172.21.15.99:0/3676255653 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd880003a60 0x7fd880003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:45.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.032+0000 7fd892d9d700 1 --2- 172.21.15.99:0/3676255653 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8800051c0 0x7fd88000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:45.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.032+0000 7fd889d9b700 1 -- 172.21.15.99:0/3676255653 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd880003a60 msgr2=0x7fd880003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:45.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.032+0000 7fd889d9b700 1 --2- 172.21.15.99:0/3676255653 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd880003a60 0x7fd880003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:45.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.032+0000 7fd889d9b700 1 -- 172.21.15.99:0/3676255653 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8800051c0 msgr2=0x7fd88000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:45.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.032+0000 7fd889d9b700 1 --2- 172.21.15.99:0/3676255653 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8800051c0 0x7fd88000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:45.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.032+0000 7fd889d9b700 1 -- 172.21.15.99:0/3676255653 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd880003570 con 0x7fd880004810 2023-10-08T00:12:45.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.032+0000 7fd889d9b700 1 --2- 172.21.15.99:0/3676255653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd880004810 0x7fd880004c80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fd87c00c3c0 tx=0x7fd8940c9080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6674eda0f367c4e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:45.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.033+0000 7fd891d9b700 1 -- 172.21.15.99:0/3676255653 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87c00cbf0 con 0x7fd880004810 2023-10-08T00:12:45.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.033+0000 7fd891d9b700 1 -- 172.21.15.99:0/3676255653 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd87c00cd90 con 0x7fd880004810 2023-10-08T00:12:45.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.033+0000 7fd891d9b700 1 -- 172.21.15.99:0/3676255653 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87c005b70 con 0x7fd880004810 2023-10-08T00:12:45.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.033+0000 7fd898c44700 1 -- 172.21.15.99:0/3676255653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd880004810 msgr2=0x7fd880004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:45.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.033+0000 7fd898c44700 1 --2- 172.21.15.99:0/3676255653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd880004810 0x7fd880004c80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fd87c00c3c0 tx=0x7fd8940c9080 comp rx=0 tx=0).stop 2023-10-08T00:12:45.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.033+0000 7fd892d9d700 1 -- 172.21.15.99:0/3676255653 reap_dead start 2023-10-08T00:12:45.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.033+0000 7fd898c44700 1 -- 172.21.15.99:0/3676255653 shutdown_connections 2023-10-08T00:12:45.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.033+0000 7fd898c44700 1 -- 172.21.15.99:0/3676255653 >> 172.21.15.99:0/3676255653 conn(0x7fd8940cd200 msgr2=0x7fd8940cd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:45.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.033+0000 7fd898c44700 1 -- 172.21.15.99:0/3676255653 shutdown_connections 2023-10-08T00:12:45.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.033+0000 7fd898c44700 1 -- 172.21.15.99:0/3676255653 wait complete. 2023-10-08T00:12:45.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.034+0000 7fd898c44700 1 Processor -- start 2023-10-08T00:12:45.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.034+0000 7fd898c44700 1 -- start start 2023-10-08T00:12:45.048 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.035+0000 7fd898c44700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd880004810 0x7fd894165ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:45.048 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.035+0000 7fd898c44700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8941690c0 0x7fd8941584f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:45.048 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.035+0000 7fd898c44700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8940b1930 0x7fd894099f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:45.048 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.035+0000 7fd898c44700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8940b20e0 con 0x7fd8940b1930 2023-10-08T00:12:45.048 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.035+0000 7fd898c44700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8940ae110 con 0x7fd8941690c0 2023-10-08T00:12:45.048 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.035+0000 7fd898c44700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8941695c0 con 0x7fd880004810 2023-10-08T00:12:45.049 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.035+0000 7fd892d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8940b1930 0x7fd894099f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:45.049 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.035+0000 7fd892d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8940b1930 0x7fd894099f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55452/0 (socket says 172.21.15.99:55452) 2023-10-08T00:12:45.049 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.035+0000 7fd892d9d700 1 -- 172.21.15.99:0/3961503309 learned_addr learned my addr 172.21.15.99:0/3961503309 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:45.049 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.035+0000 7fd89259c700 1 --2- 172.21.15.99:0/3961503309 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd880004810 0x7fd894165ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:45.049 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.035+0000 7fd889d9b700 1 --2- 172.21.15.99:0/3961503309 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8941690c0 0x7fd8941584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:45.049 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.036+0000 7fd892d9d700 1 -- 172.21.15.99:0/3961503309 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd880004810 msgr2=0x7fd894165ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:45.049 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.036+0000 7fd892d9d700 1 --2- 172.21.15.99:0/3961503309 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd880004810 0x7fd894165ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:45.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.036+0000 7fd892d9d700 1 -- 172.21.15.99:0/3961503309 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8941690c0 msgr2=0x7fd8941584f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:45.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.036+0000 7fd892d9d700 1 --2- 172.21.15.99:0/3961503309 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8941690c0 0x7fd8941584f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:45.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.036+0000 7fd892d9d700 1 -- 172.21.15.99:0/3961503309 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8941698a0 con 0x7fd8940b1930 2023-10-08T00:12:45.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.036+0000 7fd889d9b700 1 --2- 172.21.15.99:0/3961503309 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8941690c0 0x7fd8941584f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:12:45.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.036+0000 7fd892d9d700 1 --2- 172.21.15.99:0/3961503309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8940b1930 0x7fd894099f20 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd89406ff50 tx=0x7fd88c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:45.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.036+0000 7fd88affd700 1 -- 172.21.15.99:0/3961503309 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88c00e070 con 0x7fd8940b1930 2023-10-08T00:12:45.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.036+0000 7fd88affd700 1 -- 172.21.15.99:0/3961503309 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd88c011db0 con 0x7fd8940b1930 2023-10-08T00:12:45.051 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.036+0000 7fd898c44700 1 -- 172.21.15.99:0/3961503309 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd894169b80 con 0x7fd8940b1930 2023-10-08T00:12:45.051 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.036+0000 7fd88affd700 1 -- 172.21.15.99:0/3961503309 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88c0166e0 con 0x7fd8940b1930 2023-10-08T00:12:45.051 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.036+0000 7fd898c44700 1 -- 172.21.15.99:0/3961503309 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd894166000 con 0x7fd8940b1930 2023-10-08T00:12:45.051 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.037+0000 7fd898c44700 1 -- 172.21.15.99:0/3961503309 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd89404c3c0 con 0x7fd8940b1930 2023-10-08T00:12:45.051 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.037+0000 7fd88affd700 1 -- 172.21.15.99:0/3961503309 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd88c016a90 con 0x7fd8940b1930 2023-10-08T00:12:45.051 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.038+0000 7fd88affd700 1 --2- 172.21.15.99:0/3961503309 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd874066f30 0x7fd8740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:45.052 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.038+0000 7fd88affd700 1 -- 172.21.15.99:0/3961503309 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5850+0+0 (secure 0 0 0) 0x7fd88c085170 con 0x7fd8940b1930 2023-10-08T00:12:45.052 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.038+0000 7fd89259c700 1 --2- 172.21.15.99:0/3961503309 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd874066f30 0x7fd8740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:45.052 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.038+0000 7fd89259c700 1 --2- 172.21.15.99:0/3961503309 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd874066f30 0x7fd8740693e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd884014350 tx=0x7fd884008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:45.052 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.042+0000 7fd88affd700 1 -- 172.21.15.99:0/3961503309 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd88c052410 con 0x7fd8940b1930 2023-10-08T00:12:45.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.267+0000 7fd898c44700 1 -- 172.21.15.99:0/3961503309 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd8940cb9a0 con 0x7fd8940b1930 2023-10-08T00:12:45.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.268+0000 7fd88affd700 1 -- 172.21.15.99:0/3961503309 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4603 (secure 0 0 0) 0x7fd88c055a30 con 0x7fd8940b1930 2023-10-08T00:12:45.269 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:12:45.269 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5146,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/1438931938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1438931938},{"type":"v1","addr":"172.21.15.191:6836","nonce":1438931938}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5280,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/3090674680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3090674680},{"type":"v1","addr":"172.21.15.99:6833","nonce":3090674680}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5283,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/4025282357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":4025282357},{"type":"v1","addr":"172.21.15.99:6835","nonce":4025282357}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:12:09.803824+0000","modified":"2023-10-08T00:12:13.436570+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5289},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5289":{"gid":5289,"name":"d","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/2062242133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":2062242133},{"type":"v1","addr":"172.21.15.191:6837","nonce":2062242133}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-08T00:12:45.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.269+0000 7fd898c44700 1 -- 172.21.15.99:0/3961503309 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd874066f30 msgr2=0x7fd8740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:45.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.269+0000 7fd898c44700 1 --2- 172.21.15.99:0/3961503309 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd874066f30 0x7fd8740693e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd884014350 tx=0x7fd884008040 comp rx=0 tx=0).stop 2023-10-08T00:12:45.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.269+0000 7fd898c44700 1 -- 172.21.15.99:0/3961503309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8940b1930 msgr2=0x7fd894099f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:45.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.269+0000 7fd898c44700 1 --2- 172.21.15.99:0/3961503309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8940b1930 0x7fd894099f20 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd89406ff50 tx=0x7fd88c00b040 comp rx=0 tx=0).stop 2023-10-08T00:12:45.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.270+0000 7fd898c44700 1 -- 172.21.15.99:0/3961503309 shutdown_connections 2023-10-08T00:12:45.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.270+0000 7fd898c44700 1 --2- 172.21.15.99:0/3961503309 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd874066f30 0x7fd8740693e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:45.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.270+0000 7fd898c44700 1 --2- 172.21.15.99:0/3961503309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8940b1930 0x7fd894099f20 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:45.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.270+0000 7fd898c44700 1 --2- 172.21.15.99:0/3961503309 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd880004810 0x7fd894165ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:45.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.270+0000 7fd898c44700 1 --2- 172.21.15.99:0/3961503309 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8941690c0 0x7fd8941584f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:45.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.270+0000 7fd898c44700 1 -- 172.21.15.99:0/3961503309 >> 172.21.15.99:0/3961503309 conn(0x7fd8940cd200 msgr2=0x7fd8940c3940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:45.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.270+0000 7fd898c44700 1 -- 172.21.15.99:0/3961503309 shutdown_connections 2023-10-08T00:12:45.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.270+0000 7fd898c44700 1 -- 172.21.15.99:0/3961503309 wait complete. 2023-10-08T00:12:45.272 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 24 2023-10-08T00:12:45.282 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:12:45.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.396+0000 7f7376be1700 1 Processor -- start 2023-10-08T00:12:45.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.396+0000 7f7376be1700 1 -- start start 2023-10-08T00:12:45.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.397+0000 7f7376be1700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f73700c9200 0x7f73700b18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:45.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.397+0000 7f7376be1700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f73700cba50 0x7f73700cbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:45.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.397+0000 7f7376be1700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73700caa60 con 0x7f73700cc720 2023-10-08T00:12:45.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.397+0000 7f7376be1700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73700cf330 con 0x7f73700c9200 2023-10-08T00:12:45.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.397+0000 7f7376be1700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73700cf570 con 0x7f73700cba50 2023-10-08T00:12:45.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.397+0000 7f7367fff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f73700c9200 0x7f73700b18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:45.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.397+0000 7f7367fff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f73700c9200 0x7f73700b18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55458/0 (socket says 172.21.15.99:55458) 2023-10-08T00:12:45.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.397+0000 7f737497d700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f73700cc720 0x7f73700cec20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59496/0 (socket says 172.21.15.99:59496) 2023-10-08T00:12:45.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.397+0000 7f7367fff700 1 -- 172.21.15.99:0/3082777981 learned_addr learned my addr 172.21.15.99:0/3082777981 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:45.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.397+0000 7f737517e700 1 --2- 172.21.15.99:0/3082777981 >> v2:172.21.15.191:3300/0 conn(0x7f73700cba50 0x7f73700cbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:45.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.398+0000 7f7367fff700 1 -- 172.21.15.99:0/3082777981 >> v2:172.21.15.191:3300/0 conn(0x7f73700cba50 msgr2=0x7f73700cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:45.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.398+0000 7f7367fff700 1 --2- 172.21.15.99:0/3082777981 >> v2:172.21.15.191:3300/0 conn(0x7f73700cba50 0x7f73700cbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:45.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.398+0000 7f7367fff700 1 -- 172.21.15.99:0/3082777981 >> v1:172.21.15.99:6789/0 conn(0x7f73700cc720 legacy=0x7f73700cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:45.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.398+0000 7f73677fe700 1 -- 172.21.15.99:0/3082777981 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 941340884 0 0) 0x7f73700caa60 con 0x7f73700cc720 2023-10-08T00:12:45.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.398+0000 7f7367fff700 1 -- 172.21.15.99:0/3082777981 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73700cb5c0 con 0x7f73700c9200 2023-10-08T00:12:45.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.398+0000 7f7367fff700 1 --2- 172.21.15.99:0/3082777981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73700c9200 0x7f73700b18f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f737006ed80 tx=0x7f7360009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f4355519e86a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:45.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.398+0000 7f73677fe700 1 -- 172.21.15.99:0/3082777981 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736000b860 con 0x7f73700c9200 2023-10-08T00:12:45.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.398+0000 7f73677fe700 1 -- 172.21.15.99:0/3082777981 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f736000b9c0 con 0x7f73700c9200 2023-10-08T00:12:45.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.398+0000 7f73677fe700 1 -- 172.21.15.99:0/3082777981 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736000bd60 con 0x7f73700c9200 2023-10-08T00:12:45.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.398+0000 7f7376be1700 1 -- 172.21.15.99:0/3082777981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73700c9200 msgr2=0x7f73700b18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:45.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.398+0000 7f7376be1700 1 --2- 172.21.15.99:0/3082777981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73700c9200 0x7f73700b18f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f737006ed80 tx=0x7f7360009250 comp rx=0 tx=0).stop 2023-10-08T00:12:45.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.398+0000 7f7376be1700 1 -- 172.21.15.99:0/3082777981 shutdown_connections 2023-10-08T00:12:45.402 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.398+0000 7f7376be1700 1 --2- 172.21.15.99:0/3082777981 >> v2:172.21.15.191:3300/0 conn(0x7f73700cba50 0x7f73700cbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:45.402 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.398+0000 7f7376be1700 1 --2- 172.21.15.99:0/3082777981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73700c9200 0x7f73700b18f0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:45.402 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.398+0000 7f7376be1700 1 -- 172.21.15.99:0/3082777981 >> 172.21.15.99:0/3082777981 conn(0x7f737006c6e0 msgr2=0x7f7370069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:45.402 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.398+0000 7f7376be1700 1 -- 172.21.15.99:0/3082777981 shutdown_connections 2023-10-08T00:12:45.402 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.398+0000 7f7376be1700 1 -- 172.21.15.99:0/3082777981 wait complete. 2023-10-08T00:12:45.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.399+0000 7f7376be1700 1 Processor -- start 2023-10-08T00:12:45.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.400+0000 7f7376be1700 1 -- start start 2023-10-08T00:12:45.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.400+0000 7f7376be1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73700c9200 0x7f7370168610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:45.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.400+0000 7f7376be1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f73700cba50 0x7f7370165000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:45.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.400+0000 7f7376be1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f73700cc720 0x7f7370168d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:45.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.400+0000 7f7376be1700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7370155340 con 0x7f73700c9200 2023-10-08T00:12:45.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.400+0000 7f7376be1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73700c80a0 con 0x7f73700cc720 2023-10-08T00:12:45.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.400+0000 7f7376be1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7370168b50 con 0x7f73700cba50 2023-10-08T00:12:45.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.400+0000 7f737497d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73700c9200 0x7f7370168610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:45.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.400+0000 7f737497d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73700c9200 0x7f7370168610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55462/0 (socket says 172.21.15.99:55462) 2023-10-08T00:12:45.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.400+0000 7f737497d700 1 -- 172.21.15.99:0/2791775496 learned_addr learned my addr 172.21.15.99:0/2791775496 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:45.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.401+0000 7f737497d700 1 -- 172.21.15.99:0/2791775496 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f73700cba50 msgr2=0x7f7370165000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:45.406 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.401+0000 7f737497d700 1 --2- 172.21.15.99:0/2791775496 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f73700cba50 0x7f7370165000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:45.406 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.401+0000 7f737497d700 1 -- 172.21.15.99:0/2791775496 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f73700cc720 msgr2=0x7f7370168d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:45.406 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.401+0000 7f737497d700 1 --2- 172.21.15.99:0/2791775496 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f73700cc720 0x7f7370168d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:45.406 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.401+0000 7f737497d700 1 -- 172.21.15.99:0/2791775496 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73600094f0 con 0x7f73700c9200 2023-10-08T00:12:45.406 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.401+0000 7f737497d700 1 --2- 172.21.15.99:0/2791775496 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73700c9200 0x7f7370168610 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f736c00c610 tx=0x7f736c001040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:45.406 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.401+0000 7f7364ff9700 1 -- 172.21.15.99:0/2791775496 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736c005a70 con 0x7f73700c9200 2023-10-08T00:12:45.406 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.401+0000 7f7364ff9700 1 -- 172.21.15.99:0/2791775496 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f736c005bd0 con 0x7f73700c9200 2023-10-08T00:12:45.406 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.401+0000 7f7376be1700 1 -- 172.21.15.99:0/2791775496 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73701546c0 con 0x7f73700c9200 2023-10-08T00:12:45.406 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.401+0000 7f7364ff9700 1 -- 172.21.15.99:0/2791775496 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736c0070d0 con 0x7f73700c9200 2023-10-08T00:12:45.406 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.401+0000 7f7376be1700 1 -- 172.21.15.99:0/2791775496 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7370154c80 con 0x7f73700c9200 2023-10-08T00:12:45.406 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.402+0000 7f7376be1700 1 -- 172.21.15.99:0/2791775496 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f737004c3c0 con 0x7f73700c9200 2023-10-08T00:12:45.407 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.402+0000 7f7364ff9700 1 -- 172.21.15.99:0/2791775496 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f736c007480 con 0x7f73700c9200 2023-10-08T00:12:45.407 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.403+0000 7f7364ff9700 1 --2- 172.21.15.99:0/2791775496 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f735c066f30 0x7f735c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:45.407 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.403+0000 7f7364ff9700 1 -- 172.21.15.99:0/2791775496 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5850+0+0 (secure 0 0 0) 0x7f736c083bd0 con 0x7f73700c9200 2023-10-08T00:12:45.407 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.403+0000 7f7367fff700 1 --2- 172.21.15.99:0/2791775496 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f735c066f30 0x7f735c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:45.407 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.403+0000 7f7367fff700 1 --2- 172.21.15.99:0/2791775496 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f735c066f30 0x7f735c0693e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f737006ed80 tx=0x7f7360017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:45.407 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.407+0000 7f7364ff9700 1 -- 172.21.15.99:0/2791775496 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f736c050db0 con 0x7f73700c9200 2023-10-08T00:12:45.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.589+0000 7f7376be1700 1 -- 172.21.15.99:0/2791775496 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f73700cac90 con 0x7f73700c9200 2023-10-08T00:12:45.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.590+0000 7f7364ff9700 1 -- 172.21.15.99:0/2791775496 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+15586 (secure 0 0 0) 0x7f736c017020 con 0x7f73700c9200 2023-10-08T00:12:45.592 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:12:45.592 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":57,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:12:43.142481+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:12:07.733432+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-08T00:12:08.762879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.8","mappings":[{"from":5,"to":7}]},{"pgid":"7.15","mappings":[{"from":5,"to":0}]},{"pgid":"7.28","mappings":[{"from":5,"to":7}]},{"pgid":"7.2b","mappings":[{"from":5,"to":7}]},{"pgid":"7.33","mappings":[{"from":5,"to":7}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:12:45.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.591+0000 7f7376be1700 1 -- 172.21.15.99:0/2791775496 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f735c066f30 msgr2=0x7f735c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:45.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.592+0000 7f7376be1700 1 --2- 172.21.15.99:0/2791775496 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f735c066f30 0x7f735c0693e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f737006ed80 tx=0x7f7360017000 comp rx=0 tx=0).stop 2023-10-08T00:12:45.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.592+0000 7f7376be1700 1 -- 172.21.15.99:0/2791775496 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73700c9200 msgr2=0x7f7370168610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:45.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.592+0000 7f7376be1700 1 --2- 172.21.15.99:0/2791775496 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73700c9200 0x7f7370168610 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f736c00c610 tx=0x7f736c001040 comp rx=0 tx=0).stop 2023-10-08T00:12:45.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.592+0000 7f7376be1700 1 -- 172.21.15.99:0/2791775496 shutdown_connections 2023-10-08T00:12:45.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.592+0000 7f7376be1700 1 --2- 172.21.15.99:0/2791775496 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f735c066f30 0x7f735c0693e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:45.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.592+0000 7f7376be1700 1 --2- 172.21.15.99:0/2791775496 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f73700cc720 0x7f7370168d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:45.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.592+0000 7f7376be1700 1 --2- 172.21.15.99:0/2791775496 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73700c9200 0x7f7370168610 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:45.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.592+0000 7f7376be1700 1 --2- 172.21.15.99:0/2791775496 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f73700cba50 0x7f7370165000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:45.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.592+0000 7f7376be1700 1 -- 172.21.15.99:0/2791775496 >> 172.21.15.99:0/2791775496 conn(0x7f737006c6e0 msgr2=0x7f73700ccaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:45.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.592+0000 7f7376be1700 1 -- 172.21.15.99:0/2791775496 shutdown_connections 2023-10-08T00:12:45.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.592+0000 7f7376be1700 1 -- 172.21.15.99:0/2791775496 wait complete. 2023-10-08T00:12:45.605 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T00:12:45.719 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.718+0000 7fdb9979a700 1 Processor -- start 2023-10-08T00:12:45.719 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.719+0000 7fdb9979a700 1 -- start start 2023-10-08T00:12:45.721 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.719+0000 7fdb9979a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdb9415a330 0x7fdb94156940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:45.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.719+0000 7fdb9979a700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdb9406da90 0x7fdb94156e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:45.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.719+0000 7fdb9979a700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb94157c30 con 0x7fdb94156570 2023-10-08T00:12:45.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.719+0000 7fdb9979a700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb941584f0 con 0x7fdb9406da90 2023-10-08T00:12:45.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.719+0000 7fdb9979a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb941573c0 con 0x7fdb9415a330 2023-10-08T00:12:45.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.719+0000 7fdb937fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdb9406da90 0x7fdb94156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:45.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.719+0000 7fdb937fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdb9406da90 0x7fdb94156e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55468/0 (socket says 172.21.15.99:55468) 2023-10-08T00:12:45.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.719+0000 7fdb937fe700 1 -- 172.21.15.99:0/1022970242 learned_addr learned my addr 172.21.15.99:0/1022970242 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:45.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.719+0000 7fdb8a7fc700 1 --2- 172.21.15.99:0/1022970242 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdb9415a330 0x7fdb94156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:45.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.720+0000 7fdb937fe700 1 -- 172.21.15.99:0/1022970242 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdb9415a330 msgr2=0x7fdb94156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:45.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.720+0000 7fdb937fe700 1 --2- 172.21.15.99:0/1022970242 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdb9415a330 0x7fdb94156940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:45.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.720+0000 7fdb937fe700 1 -- 172.21.15.99:0/1022970242 >> v1:172.21.15.191:6789/0 conn(0x7fdb94156570 legacy=0x7fdb9406d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:45.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.720+0000 7fdb937fe700 1 -- 172.21.15.99:0/1022970242 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb94157670 con 0x7fdb9406da90 2023-10-08T00:12:45.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.720+0000 7fdb937fe700 1 --2- 172.21.15.99:0/1022970242 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb9406da90 0x7fdb94156e80 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c009f20 tx=0x7fdb8c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7159a1b7a1d6770 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:45.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.720+0000 7fdb927fc700 1 -- 172.21.15.99:0/1022970242 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c00e070 con 0x7fdb9406da90 2023-10-08T00:12:45.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.720+0000 7fdb927fc700 1 -- 172.21.15.99:0/1022970242 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb8c00b9d0 con 0x7fdb9406da90 2023-10-08T00:12:45.724 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.720+0000 7fdb927fc700 1 -- 172.21.15.99:0/1022970242 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c0092e0 con 0x7fdb9406da90 2023-10-08T00:12:45.724 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.720+0000 7fdb9979a700 1 -- 172.21.15.99:0/1022970242 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb9406da90 msgr2=0x7fdb94156e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:45.724 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.720+0000 7fdb9979a700 1 --2- 172.21.15.99:0/1022970242 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb9406da90 0x7fdb94156e80 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c009f20 tx=0x7fdb8c009840 comp rx=0 tx=0).stop 2023-10-08T00:12:45.724 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.720+0000 7fdb9979a700 1 -- 172.21.15.99:0/1022970242 shutdown_connections 2023-10-08T00:12:45.725 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.720+0000 7fdb9979a700 1 --2- 172.21.15.99:0/1022970242 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb9406da90 0x7fdb94156e80 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:45.725 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.720+0000 7fdb9979a700 1 --2- 172.21.15.99:0/1022970242 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdb9415a330 0x7fdb94156940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:45.725 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.720+0000 7fdb9979a700 1 -- 172.21.15.99:0/1022970242 >> 172.21.15.99:0/1022970242 conn(0x7fdb94067660 msgr2=0x7fdb94067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:45.725 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.721+0000 7fdb9979a700 1 -- 172.21.15.99:0/1022970242 shutdown_connections 2023-10-08T00:12:45.726 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.721+0000 7fdb9979a700 1 -- 172.21.15.99:0/1022970242 wait complete. 2023-10-08T00:12:45.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.722+0000 7fdb9979a700 1 Processor -- start 2023-10-08T00:12:45.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.722+0000 7fdb9979a700 1 -- start start 2023-10-08T00:12:45.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.722+0000 7fdb9979a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdb9406da90 0x7fdb940c61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:45.730 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.722+0000 7fdb9979a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb94156570 0x7fdb940af980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:45.730 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.722+0000 7fdb9979a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb9415a330 0x7fdb940afec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:45.730 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.722+0000 7fdb9979a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb940c30d0 con 0x7fdb9415a330 2023-10-08T00:12:45.730 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.722+0000 7fdb9979a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb940c2810 con 0x7fdb94156570 2023-10-08T00:12:45.730 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.722+0000 7fdb9979a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb940c2a80 con 0x7fdb9406da90 2023-10-08T00:12:45.730 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.723+0000 7fdb937fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb9415a330 0x7fdb940afec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:45.730 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.723+0000 7fdb937fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb9415a330 0x7fdb940afec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55476/0 (socket says 172.21.15.99:55476) 2023-10-08T00:12:45.730 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.723+0000 7fdb937fe700 1 -- 172.21.15.99:0/3846423392 learned_addr learned my addr 172.21.15.99:0/3846423392 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:45.730 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.723+0000 7fdb937fe700 1 -- 172.21.15.99:0/3846423392 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdb9406da90 msgr2=0x7fdb940c61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:45.730 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.723+0000 7fdb937fe700 1 --2- 172.21.15.99:0/3846423392 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdb9406da90 0x7fdb940c61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:45.731 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.723+0000 7fdb8a7fc700 1 --2- 172.21.15.99:0/3846423392 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb94156570 0x7fdb940af980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:45.731 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.723+0000 7fdb937fe700 1 -- 172.21.15.99:0/3846423392 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb94156570 msgr2=0x7fdb940af980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:45.731 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.723+0000 7fdb937fe700 1 --2- 172.21.15.99:0/3846423392 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb94156570 0x7fdb940af980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:45.731 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.723+0000 7fdb937fe700 1 -- 172.21.15.99:0/3846423392 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb8c009ae0 con 0x7fdb9415a330 2023-10-08T00:12:45.731 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.723+0000 7fdb937fe700 1 --2- 172.21.15.99:0/3846423392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb9415a330 0x7fdb940afec0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c00b610 tx=0x7fdb8c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:45.731 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.723+0000 7fdb8b7fe700 1 -- 172.21.15.99:0/3846423392 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c01f070 con 0x7fdb9415a330 2023-10-08T00:12:45.731 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.723+0000 7fdb8b7fe700 1 -- 172.21.15.99:0/3846423392 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb8c0092e0 con 0x7fdb9415a330 2023-10-08T00:12:45.731 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.723+0000 7fdb9979a700 1 -- 172.21.15.99:0/3846423392 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb940c2d60 con 0x7fdb9415a330 2023-10-08T00:12:45.732 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.723+0000 7fdb8b7fe700 1 -- 172.21.15.99:0/3846423392 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c023410 con 0x7fdb9415a330 2023-10-08T00:12:45.732 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.723+0000 7fdb9979a700 1 -- 172.21.15.99:0/3846423392 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb940c56c0 con 0x7fdb9415a330 2023-10-08T00:12:45.732 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.724+0000 7fdb9979a700 1 -- 172.21.15.99:0/3846423392 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb9404c3c0 con 0x7fdb9415a330 2023-10-08T00:12:45.732 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.725+0000 7fdb8b7fe700 1 -- 172.21.15.99:0/3846423392 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdb8c003050 con 0x7fdb9415a330 2023-10-08T00:12:45.732 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.725+0000 7fdb8b7fe700 1 --2- 172.21.15.99:0/3846423392 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdb74066f30 0x7fdb740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:45.732 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.725+0000 7fdb8b7fe700 1 -- 172.21.15.99:0/3846423392 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5850+0+0 (secure 0 0 0) 0x7fdb8c0873d0 con 0x7fdb9415a330 2023-10-08T00:12:45.732 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.725+0000 7fdb92ffd700 1 --2- 172.21.15.99:0/3846423392 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdb74066f30 0x7fdb740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:45.732 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.726+0000 7fdb92ffd700 1 --2- 172.21.15.99:0/3846423392 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdb74066f30 0x7fdb740693e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdb84001730 tx=0x7fdb84007b40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:45.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.729+0000 7fdb8b7fe700 1 -- 172.21.15.99:0/3846423392 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdb8c0545b0 con 0x7fdb9415a330 2023-10-08T00:12:45.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.955+0000 7fdb9979a700 1 -- 172.21.15.99:0/3846423392 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fdb94157950 con 0x7fdb9415a330 2023-10-08T00:12:45.956 INFO:tasks.ceph.mon.a.smithi099.stderr:2023-10-08T00:12:45.955+0000 7f6c60fa2700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T00:12:45.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.959+0000 7fdb8b7fe700 1 -- 172.21.15.99:0/3846423392 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v25) v1 ==== 161+0+0 (secure 0 0 0) 0x7fdb8c057bd0 con 0x7fdb9415a330 2023-10-08T00:12:45.962 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.961+0000 7fdb9979a700 1 -- 172.21.15.99:0/3846423392 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdb74066f30 msgr2=0x7fdb740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:45.962 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.961+0000 7fdb9979a700 1 --2- 172.21.15.99:0/3846423392 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdb74066f30 0x7fdb740693e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdb84001730 tx=0x7fdb84007b40 comp rx=0 tx=0).stop 2023-10-08T00:12:45.962 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.961+0000 7fdb9979a700 1 -- 172.21.15.99:0/3846423392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb9415a330 msgr2=0x7fdb940afec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:45.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.961+0000 7fdb9979a700 1 --2- 172.21.15.99:0/3846423392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb9415a330 0x7fdb940afec0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c00b610 tx=0x7fdb8c018000 comp rx=0 tx=0).stop 2023-10-08T00:12:45.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.961+0000 7fdb9979a700 1 -- 172.21.15.99:0/3846423392 shutdown_connections 2023-10-08T00:12:45.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.961+0000 7fdb9979a700 1 --2- 172.21.15.99:0/3846423392 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdb74066f30 0x7fdb740693e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:45.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.961+0000 7fdb9979a700 1 --2- 172.21.15.99:0/3846423392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb9415a330 0x7fdb940afec0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:45.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.961+0000 7fdb9979a700 1 --2- 172.21.15.99:0/3846423392 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdb9406da90 0x7fdb940c61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:45.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.961+0000 7fdb9979a700 1 --2- 172.21.15.99:0/3846423392 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb94156570 0x7fdb940af980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:45.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.961+0000 7fdb9979a700 1 -- 172.21.15.99:0/3846423392 >> 172.21.15.99:0/3846423392 conn(0x7fdb94067660 msgr2=0x7fdb94156940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:45.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.961+0000 7fdb9979a700 1 -- 172.21.15.99:0/3846423392 shutdown_connections 2023-10-08T00:12:45.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:45.961+0000 7fdb9979a700 1 -- 172.21.15.99:0/3846423392 wait complete. 2023-10-08T00:12:45.964 INFO:teuthology.orchestra.run.smithi099.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T00:12:45.964 INFO:tasks.ceph.mds.d.smithi191.stderr: -9094> 2023-10-08T00:12:15.409+0000 7f5c612ac700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T00:12:45.974 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T00:12:45.995 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2023-10-08T00:12:46.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.091+0000 7fd0593e6700 1 Processor -- start 2023-10-08T00:12:46.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.091+0000 7fd0593e6700 1 -- start start 2023-10-08T00:12:46.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.092+0000 7fd0593e6700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd05415a290 0x7fd05406a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:46.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.092+0000 7fd0593e6700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd05406aae0 0x7fd0540cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:46.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.092+0000 7fd0593e6700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0540b0790 con 0x7fd054159ec0 2023-10-08T00:12:46.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.092+0000 7fd0593e6700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd05406af20 con 0x7fd05406aae0 2023-10-08T00:12:46.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.092+0000 7fd0593e6700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0540ce1f0 con 0x7fd05415a290 2023-10-08T00:12:46.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.092+0000 7fd0537fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd05406aae0 0x7fd0540cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:46.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.092+0000 7fd0537fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd05406aae0 0x7fd0540cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55490/0 (socket says 172.21.15.99:55490) 2023-10-08T00:12:46.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.092+0000 7fd0537fe700 1 -- 172.21.15.99:0/1737112438 learned_addr learned my addr 172.21.15.99:0/1737112438 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:46.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.092+0000 7fd0527fc700 1 --2- 172.21.15.99:0/1737112438 >> v2:172.21.15.191:3300/0 conn(0x7fd05415a290 0x7fd05406a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:46.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.092+0000 7fd0537fe700 1 -- 172.21.15.99:0/1737112438 >> v2:172.21.15.191:3300/0 conn(0x7fd05415a290 msgr2=0x7fd05406a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:46.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.092+0000 7fd0537fe700 1 --2- 172.21.15.99:0/1737112438 >> v2:172.21.15.191:3300/0 conn(0x7fd05415a290 0x7fd05406a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:46.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.092+0000 7fd0537fe700 1 -- 172.21.15.99:0/1737112438 >> v1:172.21.15.191:6789/0 conn(0x7fd054159ec0 legacy=0x7fd0540af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:46.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.092+0000 7fd0537fe700 1 -- 172.21.15.99:0/1737112438 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0540b0b20 con 0x7fd05406aae0 2023-10-08T00:12:46.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.092+0000 7fd0537fe700 1 --2- 172.21.15.99:0/1737112438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd05406aae0 0x7fd0540cf590 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fd054089910 tx=0x7fd04c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78856368186eb7ee server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:46.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.092+0000 7fd051ffb700 1 -- 172.21.15.99:0/1737112438 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04c00e070 con 0x7fd05406aae0 2023-10-08T00:12:46.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.092+0000 7fd051ffb700 1 -- 172.21.15.99:0/1737112438 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd04c00ba20 con 0x7fd05406aae0 2023-10-08T00:12:46.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.092+0000 7fd051ffb700 1 -- 172.21.15.99:0/1737112438 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04c0092e0 con 0x7fd05406aae0 2023-10-08T00:12:46.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.093+0000 7fd0593e6700 1 -- 172.21.15.99:0/1737112438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd05406aae0 msgr2=0x7fd0540cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:46.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.093+0000 7fd0593e6700 1 --2- 172.21.15.99:0/1737112438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd05406aae0 0x7fd0540cf590 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fd054089910 tx=0x7fd04c009840 comp rx=0 tx=0).stop 2023-10-08T00:12:46.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.093+0000 7fd0593e6700 1 -- 172.21.15.99:0/1737112438 shutdown_connections 2023-10-08T00:12:46.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.093+0000 7fd0593e6700 1 --2- 172.21.15.99:0/1737112438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd05406aae0 0x7fd0540cf590 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:46.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.093+0000 7fd0593e6700 1 --2- 172.21.15.99:0/1737112438 >> v2:172.21.15.191:3300/0 conn(0x7fd05415a290 0x7fd05406a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:46.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.093+0000 7fd0593e6700 1 -- 172.21.15.99:0/1737112438 >> 172.21.15.99:0/1737112438 conn(0x7fd0540d1880 msgr2=0x7fd0540cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:46.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.093+0000 7fd0593e6700 1 -- 172.21.15.99:0/1737112438 shutdown_connections 2023-10-08T00:12:46.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.093+0000 7fd0593e6700 1 -- 172.21.15.99:0/1737112438 wait complete. 2023-10-08T00:12:46.098 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.094+0000 7fd0593e6700 1 Processor -- start 2023-10-08T00:12:46.098 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.094+0000 7fd0593e6700 1 -- start start 2023-10-08T00:12:46.098 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.095+0000 7fd0593e6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd05406aae0 0x7fd0540b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:46.098 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.095+0000 7fd0593e6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd054159ec0 0x7fd0540cb430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:46.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.095+0000 7fd0593e6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd05415a290 0x7fd0541558a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:46.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.095+0000 7fd0593e6700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0540cb970 con 0x7fd054159ec0 2023-10-08T00:12:46.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.095+0000 7fd0593e6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0540b4130 con 0x7fd05415a290 2023-10-08T00:12:46.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.095+0000 7fd0593e6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0540b4410 con 0x7fd05406aae0 2023-10-08T00:12:46.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.095+0000 7fd0527fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd054159ec0 0x7fd0540cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:46.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.095+0000 7fd0527fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd054159ec0 0x7fd0540cb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55502/0 (socket says 172.21.15.99:55502) 2023-10-08T00:12:46.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.095+0000 7fd0527fc700 1 -- 172.21.15.99:0/1292878815 learned_addr learned my addr 172.21.15.99:0/1292878815 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:46.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.095+0000 7fd0537fe700 1 --2- 172.21.15.99:0/1292878815 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd05415a290 0x7fd0541558a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:46.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.095+0000 7fd052ffd700 1 --2- 172.21.15.99:0/1292878815 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd05406aae0 0x7fd0540b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:46.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.095+0000 7fd0527fc700 1 -- 172.21.15.99:0/1292878815 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd05406aae0 msgr2=0x7fd0540b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:46.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.095+0000 7fd0527fc700 1 --2- 172.21.15.99:0/1292878815 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd05406aae0 0x7fd0540b3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:46.100 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.095+0000 7fd0527fc700 1 -- 172.21.15.99:0/1292878815 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd05415a290 msgr2=0x7fd0541558a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:46.100 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.095+0000 7fd0527fc700 1 --2- 172.21.15.99:0/1292878815 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd05415a290 0x7fd0541558a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:46.100 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.095+0000 7fd0527fc700 1 -- 172.21.15.99:0/1292878815 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd04c009ae0 con 0x7fd054159ec0 2023-10-08T00:12:46.100 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.096+0000 7fd0527fc700 1 --2- 172.21.15.99:0/1292878815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd054159ec0 0x7fd0540cb430 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd04800e930 tx=0x7fd0480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:46.100 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.096+0000 7fd03affd700 1 -- 172.21.15.99:0/1292878815 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd048013680 con 0x7fd054159ec0 2023-10-08T00:12:46.100 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.096+0000 7fd03affd700 1 -- 172.21.15.99:0/1292878815 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0480137e0 con 0x7fd054159ec0 2023-10-08T00:12:46.100 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.096+0000 7fd0593e6700 1 -- 172.21.15.99:0/1292878815 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0540b46f0 con 0x7fd054159ec0 2023-10-08T00:12:46.102 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.096+0000 7fd03affd700 1 -- 172.21.15.99:0/1292878815 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd048013b80 con 0x7fd054159ec0 2023-10-08T00:12:46.102 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.096+0000 7fd0593e6700 1 -- 172.21.15.99:0/1292878815 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd054155de0 con 0x7fd054159ec0 2023-10-08T00:12:46.102 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.097+0000 7fd0593e6700 1 -- 172.21.15.99:0/1292878815 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0540b51e0 con 0x7fd054159ec0 2023-10-08T00:12:46.102 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.097+0000 7fd03affd700 1 -- 172.21.15.99:0/1292878815 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd04800c3e0 con 0x7fd054159ec0 2023-10-08T00:12:46.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.097+0000 7fd03affd700 1 --2- 172.21.15.99:0/1292878815 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd040066f30 0x7fd0400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:46.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.097+0000 7fd03affd700 1 -- 172.21.15.99:0/1292878815 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5936+0+0 (secure 0 0 0) 0x7fd048084690 con 0x7fd054159ec0 2023-10-08T00:12:46.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.098+0000 7fd052ffd700 1 --2- 172.21.15.99:0/1292878815 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd040066f30 0x7fd0400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:46.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.098+0000 7fd052ffd700 1 --2- 172.21.15.99:0/1292878815 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd040066f30 0x7fd0400693e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd044003a80 tx=0x7fd044007ec0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:46.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.101+0000 7fd03affd700 1 -- 172.21.15.99:0/1292878815 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd048051820 con 0x7fd054159ec0 2023-10-08T00:12:46.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.325+0000 7fd0593e6700 1 -- 172.21.15.99:0/1292878815 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd05404c3c0 con 0x7fd054159ec0 2023-10-08T00:12:46.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.963+0000 7fd03affd700 1 -- 172.21.15.99:0/1292878815 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v26) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd048054e40 con 0x7fd054159ec0 2023-10-08T00:12:46.966 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.965+0000 7fd0593e6700 1 -- 172.21.15.99:0/1292878815 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd040066f30 msgr2=0x7fd0400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:46.966 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.965+0000 7fd0593e6700 1 --2- 172.21.15.99:0/1292878815 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd040066f30 0x7fd0400693e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd044003a80 tx=0x7fd044007ec0 comp rx=0 tx=0).stop 2023-10-08T00:12:46.966 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.965+0000 7fd0593e6700 1 -- 172.21.15.99:0/1292878815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd054159ec0 msgr2=0x7fd0540cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:46.966 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.965+0000 7fd0593e6700 1 --2- 172.21.15.99:0/1292878815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd054159ec0 0x7fd0540cb430 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd04800e930 tx=0x7fd0480091a0 comp rx=0 tx=0).stop 2023-10-08T00:12:46.966 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.965+0000 7fd0593e6700 1 -- 172.21.15.99:0/1292878815 shutdown_connections 2023-10-08T00:12:46.966 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.965+0000 7fd0593e6700 1 --2- 172.21.15.99:0/1292878815 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd040066f30 0x7fd0400693e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:46.967 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.965+0000 7fd0593e6700 1 --2- 172.21.15.99:0/1292878815 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd05415a290 0x7fd0541558a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:46.967 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.965+0000 7fd0593e6700 1 --2- 172.21.15.99:0/1292878815 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd05406aae0 0x7fd0540b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:46.967 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.965+0000 7fd0593e6700 1 --2- 172.21.15.99:0/1292878815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd054159ec0 0x7fd0540cb430 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:46.967 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.965+0000 7fd0593e6700 1 -- 172.21.15.99:0/1292878815 >> 172.21.15.99:0/1292878815 conn(0x7fd0540d1880 msgr2=0x7fd0540cf490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:46.967 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.965+0000 7fd0593e6700 1 -- 172.21.15.99:0/1292878815 shutdown_connections 2023-10-08T00:12:46.967 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:46.965+0000 7fd0593e6700 1 -- 172.21.15.99:0/1292878815 wait complete. 2023-10-08T00:12:46.978 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T00:12:47.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.095+0000 7f33cecfe700 1 Processor -- start 2023-10-08T00:12:47.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.095+0000 7f33cecfe700 1 -- start start 2023-10-08T00:12:47.097 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.095+0000 7f33cecfe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33c80b6a10 0x7f33c815a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:47.097 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.095+0000 7f33cecfe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f33c80b46f0 0x7f33c80b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:47.097 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.095+0000 7f33cecfe700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33c80b5090 con 0x7f33c8159ef0 2023-10-08T00:12:47.098 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.095+0000 7f33cecfe700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33c80b17b0 con 0x7f33c80b46f0 2023-10-08T00:12:47.098 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.095+0000 7f33cecfe700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33c80b1a60 con 0x7f33c80b6a10 2023-10-08T00:12:47.098 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.095+0000 7f33cd29b700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f33c80b46f0 0x7f33c80b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:47.098 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.096+0000 7f33cd29b700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f33c80b46f0 0x7f33c80b4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55510/0 (socket says 172.21.15.99:55510) 2023-10-08T00:12:47.098 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.096+0000 7f33cd29b700 1 -- 172.21.15.99:0/1378758854 learned_addr learned my addr 172.21.15.99:0/1378758854 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:47.098 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.096+0000 7f33bffff700 1 --2- 172.21.15.99:0/1378758854 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33c80b6a10 0x7f33c815a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:47.098 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.096+0000 7f33cd29b700 1 -- 172.21.15.99:0/1378758854 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33c80b6a10 msgr2=0x7f33c815a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:47.098 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.096+0000 7f33cd29b700 1 --2- 172.21.15.99:0/1378758854 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33c80b6a10 0x7f33c815a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:47.098 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.096+0000 7f33cd29b700 1 -- 172.21.15.99:0/1378758854 >> v1:172.21.15.191:6789/0 conn(0x7f33c8159ef0 legacy=0x7f33c80b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:47.098 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.096+0000 7f33cd29b700 1 -- 172.21.15.99:0/1378758854 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c80b1d40 con 0x7f33c80b46f0 2023-10-08T00:12:47.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.096+0000 7f33cd29b700 1 --2- 172.21.15.99:0/1378758854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33c80b46f0 0x7f33c80b4b50 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f33c00099b0 tx=0x7f33c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4c0468403aca457 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:47.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.096+0000 7f33bf7fe700 1 -- 172.21.15.99:0/1378758854 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c000b860 con 0x7f33c80b46f0 2023-10-08T00:12:47.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.096+0000 7f33bf7fe700 1 -- 172.21.15.99:0/1378758854 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33c000b9c0 con 0x7f33c80b46f0 2023-10-08T00:12:47.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.096+0000 7f33bf7fe700 1 -- 172.21.15.99:0/1378758854 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c000bd60 con 0x7f33c80b46f0 2023-10-08T00:12:47.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.097+0000 7f33cecfe700 1 -- 172.21.15.99:0/1378758854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33c80b46f0 msgr2=0x7f33c80b4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:47.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.097+0000 7f33cecfe700 1 --2- 172.21.15.99:0/1378758854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33c80b46f0 0x7f33c80b4b50 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f33c00099b0 tx=0x7f33c0009250 comp rx=0 tx=0).stop 2023-10-08T00:12:47.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.097+0000 7f33cecfe700 1 -- 172.21.15.99:0/1378758854 shutdown_connections 2023-10-08T00:12:47.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.097+0000 7f33cecfe700 1 --2- 172.21.15.99:0/1378758854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33c80b46f0 0x7f33c80b4b50 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:47.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.097+0000 7f33cecfe700 1 --2- 172.21.15.99:0/1378758854 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33c80b6a10 0x7f33c815a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:47.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.097+0000 7f33cecfe700 1 -- 172.21.15.99:0/1378758854 >> 172.21.15.99:0/1378758854 conn(0x7f33c809a0a0 msgr2=0x7f33c8067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:47.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.097+0000 7f33cecfe700 1 -- 172.21.15.99:0/1378758854 shutdown_connections 2023-10-08T00:12:47.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.097+0000 7f33cecfe700 1 -- 172.21.15.99:0/1378758854 wait complete. 2023-10-08T00:12:47.102 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.098+0000 7f33cecfe700 1 Processor -- start 2023-10-08T00:12:47.102 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.098+0000 7f33cecfe700 1 -- start start 2023-10-08T00:12:47.102 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.099+0000 7f33cecfe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33c80b46f0 0x7f33c8155e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:47.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.099+0000 7f33cecfe700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33c80b6a10 0x7f33c809c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:47.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.099+0000 7f33cecfe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33c8159ef0 0x7f33c80af830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:47.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.099+0000 7f33cecfe700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c809c540 con 0x7f33c8159ef0 2023-10-08T00:12:47.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.099+0000 7f33cecfe700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c809bb70 con 0x7f33c80b6a10 2023-10-08T00:12:47.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.099+0000 7f33cecfe700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33c809af60 con 0x7f33c80b46f0 2023-10-08T00:12:47.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.099+0000 7f33cd29b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33c8159ef0 0x7f33c80af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:47.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.099+0000 7f33cd29b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33c8159ef0 0x7f33c80af830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55518/0 (socket says 172.21.15.99:55518) 2023-10-08T00:12:47.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.099+0000 7f33cd29b700 1 -- 172.21.15.99:0/3328323823 learned_addr learned my addr 172.21.15.99:0/3328323823 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:47.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.099+0000 7f33cca9a700 1 --2- 172.21.15.99:0/3328323823 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33c80b46f0 0x7f33c8155e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:47.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.099+0000 7f33bffff700 1 --2- 172.21.15.99:0/3328323823 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33c80b6a10 0x7f33c809c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:47.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.099+0000 7f33cd29b700 1 -- 172.21.15.99:0/3328323823 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33c80b46f0 msgr2=0x7f33c8155e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:47.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.099+0000 7f33cd29b700 1 --2- 172.21.15.99:0/3328323823 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33c80b46f0 0x7f33c8155e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:47.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.099+0000 7f33cd29b700 1 -- 172.21.15.99:0/3328323823 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33c80b6a10 msgr2=0x7f33c809c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:47.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.099+0000 7f33cd29b700 1 --2- 172.21.15.99:0/3328323823 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33c80b6a10 0x7f33c809c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:47.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.099+0000 7f33cd29b700 1 -- 172.21.15.99:0/3328323823 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c00094f0 con 0x7f33c8159ef0 2023-10-08T00:12:47.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.099+0000 7f33cd29b700 1 --2- 172.21.15.99:0/3328323823 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33c8159ef0 0x7f33c80af830 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f33c8071f80 tx=0x7f33c0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:47.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.099+0000 7f33bcff9700 1 -- 172.21.15.99:0/3328323823 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c000b860 con 0x7f33c8159ef0 2023-10-08T00:12:47.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.099+0000 7f33bcff9700 1 -- 172.21.15.99:0/3328323823 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33c001a630 con 0x7f33c8159ef0 2023-10-08T00:12:47.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.100+0000 7f33bcff9700 1 -- 172.21.15.99:0/3328323823 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c001aa10 con 0x7f33c8159ef0 2023-10-08T00:12:47.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.100+0000 7f33cecfe700 1 -- 172.21.15.99:0/3328323823 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33c809b240 con 0x7f33c8159ef0 2023-10-08T00:12:47.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.100+0000 7f33cecfe700 1 -- 172.21.15.99:0/3328323823 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33c809b800 con 0x7f33c8159ef0 2023-10-08T00:12:47.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.100+0000 7f33cecfe700 1 -- 172.21.15.99:0/3328323823 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33c8089b60 con 0x7f33c8159ef0 2023-10-08T00:12:47.105 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.101+0000 7f33bcff9700 1 -- 172.21.15.99:0/3328323823 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f33c002d440 con 0x7f33c8159ef0 2023-10-08T00:12:47.105 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.101+0000 7f33bcff9700 1 --2- 172.21.15.99:0/3328323823 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f33b4066f30 0x7f33b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:47.105 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.101+0000 7f33bcff9700 1 -- 172.21.15.99:0/3328323823 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5936+0+0 (secure 0 0 0) 0x7f33c00861b0 con 0x7f33c8159ef0 2023-10-08T00:12:47.105 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.101+0000 7f33cca9a700 1 --2- 172.21.15.99:0/3328323823 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f33b4066f30 0x7f33b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:47.105 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.102+0000 7f33cca9a700 1 --2- 172.21.15.99:0/3328323823 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f33b4066f30 0x7f33b40693e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f33c815b470 tx=0x7f33c4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:47.106 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.105+0000 7f33bcff9700 1 -- 172.21.15.99:0/3328323823 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f33c0053340 con 0x7f33c8159ef0 2023-10-08T00:12:47.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:47.287+0000 7f33cecfe700 1 -- 172.21.15.99:0/3328323823 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f33c804c3c0 con 0x7f33c8159ef0 2023-10-08T00:12:48.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.163+0000 7f33bcff9700 1 -- 172.21.15.99:0/3328323823 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v59) v1 ==== 181+0+0 (secure 0 0 0) 0x7f33c0056960 con 0x7f33c8159ef0 2023-10-08T00:12:48.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.166+0000 7f33cecfe700 1 -- 172.21.15.99:0/3328323823 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f33b4066f30 msgr2=0x7f33b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:48.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.166+0000 7f33cecfe700 1 --2- 172.21.15.99:0/3328323823 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f33b4066f30 0x7f33b40693e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f33c815b470 tx=0x7f33c4009040 comp rx=0 tx=0).stop 2023-10-08T00:12:48.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.166+0000 7f33cecfe700 1 -- 172.21.15.99:0/3328323823 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33c8159ef0 msgr2=0x7f33c80af830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:48.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.166+0000 7f33cecfe700 1 --2- 172.21.15.99:0/3328323823 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33c8159ef0 0x7f33c80af830 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f33c8071f80 tx=0x7f33c0017000 comp rx=0 tx=0).stop 2023-10-08T00:12:48.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.166+0000 7f33cecfe700 1 -- 172.21.15.99:0/3328323823 shutdown_connections 2023-10-08T00:12:48.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.166+0000 7f33cecfe700 1 --2- 172.21.15.99:0/3328323823 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f33b4066f30 0x7f33b40693e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:48.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.166+0000 7f33cecfe700 1 --2- 172.21.15.99:0/3328323823 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33c8159ef0 0x7f33c80af830 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:48.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.166+0000 7f33cecfe700 1 --2- 172.21.15.99:0/3328323823 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33c80b46f0 0x7f33c8155e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:48.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.166+0000 7f33cecfe700 1 --2- 172.21.15.99:0/3328323823 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33c80b6a10 0x7f33c809c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:48.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.166+0000 7f33cecfe700 1 -- 172.21.15.99:0/3328323823 >> 172.21.15.99:0/3328323823 conn(0x7f33c809a0a0 msgr2=0x7f33c815bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:48.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.166+0000 7f33cecfe700 1 -- 172.21.15.99:0/3328323823 shutdown_connections 2023-10-08T00:12:48.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.166+0000 7f33cecfe700 1 -- 172.21.15.99:0/3328323823 wait complete. 2023-10-08T00:12:48.168 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' removed 2023-10-08T00:12:48.179 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T00:12:48.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.298+0000 7fe89ac27700 1 Processor -- start 2023-10-08T00:12:48.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.298+0000 7fe89ac27700 1 -- start start 2023-10-08T00:12:48.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.298+0000 7fe89ac27700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe894159ef0 0x7fe89415a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:48.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.298+0000 7fe89ac27700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8940b17b0 con 0x7fe8940b6a10 2023-10-08T00:12:48.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.298+0000 7fe89ac27700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8940b1a60 con 0x7fe8940b46f0 2023-10-08T00:12:48.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.298+0000 7fe89ac27700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8940b1d40 con 0x7fe894159ef0 2023-10-08T00:12:48.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.299+0000 7fe8989c3700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe894159ef0 0x7fe89415a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:48.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.299+0000 7fe88bfff700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fe8940b6a10 0x7fe8940b10a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59504/0 (socket says 172.21.15.99:59504) 2023-10-08T00:12:48.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.299+0000 7fe88bfff700 1 -- 172.21.15.99:0/772986080 learned_addr learned my addr 172.21.15.99:0/772986080 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:12:48.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.299+0000 7fe8989c3700 1 -- 172.21.15.99:0/772986080 >> v1:172.21.15.191:6789/0 conn(0x7fe8940b46f0 legacy=0x7fe8940b4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:48.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.299+0000 7fe8989c3700 1 -- 172.21.15.99:0/772986080 >> v1:172.21.15.99:6789/0 conn(0x7fe8940b6a10 legacy=0x7fe8940b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:48.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.299+0000 7fe8989c3700 1 -- 172.21.15.99:0/772986080 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8940b2020 con 0x7fe894159ef0 2023-10-08T00:12:48.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.299+0000 7fe8989c3700 1 --2- 172.21.15.99:0/772986080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe894159ef0 0x7fe89415a2c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fe8900098d0 tx=0x7fe890009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3541597a55ae46d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:48.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.299+0000 7fe88b7fe700 1 -- 172.21.15.99:0/772986080 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe890009e70 con 0x7fe894159ef0 2023-10-08T00:12:48.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.299+0000 7fe88b7fe700 1 -- 172.21.15.99:0/772986080 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe890005c10 con 0x7fe894159ef0 2023-10-08T00:12:48.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.299+0000 7fe88b7fe700 1 -- 172.21.15.99:0/772986080 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89000b6a0 con 0x7fe894159ef0 2023-10-08T00:12:48.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.300+0000 7fe89ac27700 1 -- 172.21.15.99:0/772986080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe894159ef0 msgr2=0x7fe89415a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:48.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.300+0000 7fe89ac27700 1 --2- 172.21.15.99:0/772986080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe894159ef0 0x7fe89415a2c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fe8900098d0 tx=0x7fe890009120 comp rx=0 tx=0).stop 2023-10-08T00:12:48.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.300+0000 7fe89ac27700 1 -- 172.21.15.99:0/772986080 shutdown_connections 2023-10-08T00:12:48.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.300+0000 7fe89ac27700 1 --2- 172.21.15.99:0/772986080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe894159ef0 0x7fe89415a2c0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:48.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.300+0000 7fe89ac27700 1 -- 172.21.15.99:0/772986080 >> 172.21.15.99:0/772986080 conn(0x7fe89409a0a0 msgr2=0x7fe894067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:48.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.300+0000 7fe89ac27700 1 -- 172.21.15.99:0/772986080 shutdown_connections 2023-10-08T00:12:48.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.300+0000 7fe89ac27700 1 -- 172.21.15.99:0/772986080 wait complete. 2023-10-08T00:12:48.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.301+0000 7fe89ac27700 1 Processor -- start 2023-10-08T00:12:48.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.301+0000 7fe89ac27700 1 -- start start 2023-10-08T00:12:48.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.302+0000 7fe89ac27700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8940b46f0 0x7fe8940af0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:48.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.302+0000 7fe89ac27700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8940b6a10 0x7fe8940af630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:48.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.302+0000 7fe89ac27700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe894159ef0 0x7fe89409dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:48.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.302+0000 7fe89ac27700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe89409d1d0 con 0x7fe8940b46f0 2023-10-08T00:12:48.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.302+0000 7fe89ac27700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8940afeb0 con 0x7fe894159ef0 2023-10-08T00:12:48.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.302+0000 7fe89ac27700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe89409d440 con 0x7fe8940b6a10 2023-10-08T00:12:48.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.302+0000 7fe8989c3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8940b46f0 0x7fe8940af0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:48.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.302+0000 7fe8989c3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8940b46f0 0x7fe8940af0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55532/0 (socket says 172.21.15.99:55532) 2023-10-08T00:12:48.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.302+0000 7fe8989c3700 1 -- 172.21.15.99:0/446261151 learned_addr learned my addr 172.21.15.99:0/446261151 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:48.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.302+0000 7fe8989c3700 1 -- 172.21.15.99:0/446261151 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8940b6a10 msgr2=0x7fe8940af630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:48.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.302+0000 7fe8989c3700 1 --2- 172.21.15.99:0/446261151 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8940b6a10 0x7fe8940af630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:48.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.302+0000 7fe8989c3700 1 -- 172.21.15.99:0/446261151 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe894159ef0 msgr2=0x7fe89409dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:48.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.302+0000 7fe8989c3700 1 --2- 172.21.15.99:0/446261151 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe894159ef0 0x7fe89409dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:48.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.302+0000 7fe8989c3700 1 -- 172.21.15.99:0/446261151 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8900093c0 con 0x7fe8940b46f0 2023-10-08T00:12:48.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.302+0000 7fe8989c3700 1 --2- 172.21.15.99:0/446261151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8940b46f0 0x7fe8940af0f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fe890009160 tx=0x7fe890015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:48.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.302+0000 7fe888ff9700 1 -- 172.21.15.99:0/446261151 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe890006740 con 0x7fe8940b46f0 2023-10-08T00:12:48.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.302+0000 7fe888ff9700 1 -- 172.21.15.99:0/446261151 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8900068c0 con 0x7fe8940b46f0 2023-10-08T00:12:48.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.302+0000 7fe888ff9700 1 -- 172.21.15.99:0/446261151 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8900266c0 con 0x7fe8940b46f0 2023-10-08T00:12:48.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.302+0000 7fe89ac27700 1 -- 172.21.15.99:0/446261151 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe89409d720 con 0x7fe8940b46f0 2023-10-08T00:12:48.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.302+0000 7fe89ac27700 1 -- 172.21.15.99:0/446261151 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8940b0190 con 0x7fe8940b46f0 2023-10-08T00:12:48.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.303+0000 7fe89ac27700 1 -- 172.21.15.99:0/446261151 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe89404c3c0 con 0x7fe8940b46f0 2023-10-08T00:12:48.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.303+0000 7fe888ff9700 1 -- 172.21.15.99:0/446261151 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe890023070 con 0x7fe8940b46f0 2023-10-08T00:12:48.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.304+0000 7fe888ff9700 1 --2- 172.21.15.99:0/446261151 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe880066f30 0x7fe8800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:48.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.304+0000 7fe888ff9700 1 -- 172.21.15.99:0/446261151 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5498+0+0 (secure 0 0 0) 0x7fe8900862f0 con 0x7fe8940b46f0 2023-10-08T00:12:48.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.304+0000 7fe88bfff700 1 --2- 172.21.15.99:0/446261151 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe880066f30 0x7fe8800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:48.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.304+0000 7fe88bfff700 1 --2- 172.21.15.99:0/446261151 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe880066f30 0x7fe8800693e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe8940b6e10 tx=0x7fe884007b30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:48.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.308+0000 7fe888ff9700 1 -- 172.21.15.99:0/446261151 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe890053630 con 0x7fe8940b46f0 2023-10-08T00:12:48.492 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:48.492+0000 7fe89ac27700 1 -- 172.21.15.99:0/446261151 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe894068fa0 con 0x7fe8940b46f0 2023-10-08T00:12:49.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.166+0000 7fe888ff9700 1 -- 172.21.15.99:0/446261151 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v60) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe890056c50 con 0x7fe8940b46f0 2023-10-08T00:12:49.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.168+0000 7fe89ac27700 1 -- 172.21.15.99:0/446261151 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe880066f30 msgr2=0x7fe8800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:49.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.168+0000 7fe89ac27700 1 --2- 172.21.15.99:0/446261151 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe880066f30 0x7fe8800693e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe8940b6e10 tx=0x7fe884007b30 comp rx=0 tx=0).stop 2023-10-08T00:12:49.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.168+0000 7fe89ac27700 1 -- 172.21.15.99:0/446261151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8940b46f0 msgr2=0x7fe8940af0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:49.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.168+0000 7fe89ac27700 1 --2- 172.21.15.99:0/446261151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8940b46f0 0x7fe8940af0f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fe890009160 tx=0x7fe890015000 comp rx=0 tx=0).stop 2023-10-08T00:12:49.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.168+0000 7fe89ac27700 1 -- 172.21.15.99:0/446261151 shutdown_connections 2023-10-08T00:12:49.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.168+0000 7fe89ac27700 1 --2- 172.21.15.99:0/446261151 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe880066f30 0x7fe8800693e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:49.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.168+0000 7fe89ac27700 1 --2- 172.21.15.99:0/446261151 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe894159ef0 0x7fe89409dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:49.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.168+0000 7fe89ac27700 1 --2- 172.21.15.99:0/446261151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8940b46f0 0x7fe8940af0f0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:49.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.168+0000 7fe89ac27700 1 --2- 172.21.15.99:0/446261151 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8940b6a10 0x7fe8940af630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:49.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.169+0000 7fe89ac27700 1 -- 172.21.15.99:0/446261151 >> 172.21.15.99:0/446261151 conn(0x7fe89409a0a0 msgr2=0x7fe89415bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:49.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.169+0000 7fe89ac27700 1 -- 172.21.15.99:0/446261151 shutdown_connections 2023-10-08T00:12:49.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.169+0000 7fe89ac27700 1 -- 172.21.15.99:0/446261151 wait complete. 2023-10-08T00:12:49.171 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' removed 2023-10-08T00:12:49.182 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-08T00:12:49.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.297+0000 7f08f10a9700 1 Processor -- start 2023-10-08T00:12:49.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.297+0000 7f08f10a9700 1 -- start start 2023-10-08T00:12:49.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.298+0000 7f08f10a9700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f08ec156830 0x7f08ec0c9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:49.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.298+0000 7f08f10a9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08ec0af800 0x7f08ec15ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:49.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.298+0000 7f08f10a9700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08ec06d790 con 0x7f08ec156c00 2023-10-08T00:12:49.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.298+0000 7f08f10a9700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08ec06e030 con 0x7f08ec156830 2023-10-08T00:12:49.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.298+0000 7f08f10a9700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec15b390 con 0x7f08ec0af800 2023-10-08T00:12:49.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.298+0000 7f08ebfff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f08ec156830 0x7f08ec0c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:49.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.298+0000 7f08eb7fe700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f08ec156c00 0x7f08ec15a620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59506/0 (socket says 172.21.15.99:59506) 2023-10-08T00:12:49.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.298+0000 7f08ebfff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f08ec156830 0x7f08ec0c9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55542/0 (socket says 172.21.15.99:55542) 2023-10-08T00:12:49.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.298+0000 7f08eb7fe700 1 -- 172.21.15.99:0/4156556157 learned_addr learned my addr 172.21.15.99:0/4156556157 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:12:49.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.298+0000 7f08f08a8700 1 --2- 172.21.15.99:0/4156556157 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08ec0af800 0x7f08ec15ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:49.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.298+0000 7f08ebfff700 1 -- 172.21.15.99:0/4156556157 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08ec0af800 msgr2=0x7f08ec15ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:49.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.298+0000 7f08ebfff700 1 --2- 172.21.15.99:0/4156556157 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08ec0af800 0x7f08ec15ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:49.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.298+0000 7f08ebfff700 1 -- 172.21.15.99:0/4156556157 >> v1:172.21.15.99:6789/0 conn(0x7f08ec156c00 legacy=0x7f08ec15a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:49.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.298+0000 7f08ebfff700 1 -- 172.21.15.99:0/4156556157 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08ec06d9e0 con 0x7f08ec156830 2023-10-08T00:12:49.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.298+0000 7f08eaffd700 1 -- 172.21.15.99:0/4156556157 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 997483868 0 0) 0x7f08ec06d790 con 0x7f08ec156c00 2023-10-08T00:12:49.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.299+0000 7f08ebfff700 1 --2- 172.21.15.99:0/4156556157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08ec156830 0x7f08ec0c9090 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f08dc0098d0 tx=0x7f08dc009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50a9676900e3f8b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:49.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.299+0000 7f08eaffd700 1 -- 172.21.15.99:0/4156556157 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc009e70 con 0x7f08ec156830 2023-10-08T00:12:49.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.299+0000 7f08eaffd700 1 -- 172.21.15.99:0/4156556157 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08dc005c10 con 0x7f08ec156830 2023-10-08T00:12:49.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.299+0000 7f08eaffd700 1 -- 172.21.15.99:0/4156556157 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc00b650 con 0x7f08ec156830 2023-10-08T00:12:49.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.299+0000 7f08f10a9700 1 -- 172.21.15.99:0/4156556157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08ec156830 msgr2=0x7f08ec0c9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:49.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.299+0000 7f08f10a9700 1 --2- 172.21.15.99:0/4156556157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08ec156830 0x7f08ec0c9090 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f08dc0098d0 tx=0x7f08dc009120 comp rx=0 tx=0).stop 2023-10-08T00:12:49.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.299+0000 7f08f10a9700 1 -- 172.21.15.99:0/4156556157 shutdown_connections 2023-10-08T00:12:49.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.299+0000 7f08f10a9700 1 --2- 172.21.15.99:0/4156556157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08ec156830 0x7f08ec0c9090 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:49.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.299+0000 7f08f10a9700 1 --2- 172.21.15.99:0/4156556157 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08ec0af800 0x7f08ec15ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:49.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.299+0000 7f08f10a9700 1 -- 172.21.15.99:0/4156556157 >> 172.21.15.99:0/4156556157 conn(0x7f08ec067660 msgr2=0x7f08ec067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:49.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.299+0000 7f08f10a9700 1 -- 172.21.15.99:0/4156556157 shutdown_connections 2023-10-08T00:12:49.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.299+0000 7f08f10a9700 1 -- 172.21.15.99:0/4156556157 wait complete. 2023-10-08T00:12:49.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.300+0000 7f08f10a9700 1 Processor -- start 2023-10-08T00:12:49.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.301+0000 7f08f10a9700 1 -- start start 2023-10-08T00:12:49.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.301+0000 7f08f10a9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08ec0af800 0x7f08ec154480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:49.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.301+0000 7f08f10a9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08ec156830 0x7f08ec0abbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:49.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.301+0000 7f08f10a9700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08ec156c00 0x7f08ec15b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:49.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.301+0000 7f08f10a9700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec0b1dc0 con 0x7f08ec0af800 2023-10-08T00:12:49.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.301+0000 7f08f10a9700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec159e30 con 0x7f08ec156c00 2023-10-08T00:12:49.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.301+0000 7f08f10a9700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec0b11b0 con 0x7f08ec156830 2023-10-08T00:12:49.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.301+0000 7f08ebfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08ec0af800 0x7f08ec154480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:49.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.301+0000 7f08ebfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08ec0af800 0x7f08ec154480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55550/0 (socket says 172.21.15.99:55550) 2023-10-08T00:12:49.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.301+0000 7f08ebfff700 1 -- 172.21.15.99:0/4171480249 learned_addr learned my addr 172.21.15.99:0/4171480249 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:49.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.301+0000 7f08f08a8700 1 --2- 172.21.15.99:0/4171480249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08ec156c00 0x7f08ec15b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:49.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.301+0000 7f08eb7fe700 1 --2- 172.21.15.99:0/4171480249 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08ec156830 0x7f08ec0abbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:49.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.302+0000 7f08ebfff700 1 -- 172.21.15.99:0/4171480249 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08ec156830 msgr2=0x7f08ec0abbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:49.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.302+0000 7f08ebfff700 1 --2- 172.21.15.99:0/4171480249 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08ec156830 0x7f08ec0abbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:49.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.302+0000 7f08ebfff700 1 -- 172.21.15.99:0/4171480249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08ec156c00 msgr2=0x7f08ec15b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:49.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.302+0000 7f08ebfff700 1 --2- 172.21.15.99:0/4171480249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08ec156c00 0x7f08ec15b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:49.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.302+0000 7f08ebfff700 1 -- 172.21.15.99:0/4171480249 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08dc0093c0 con 0x7f08ec0af800 2023-10-08T00:12:49.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.302+0000 7f08ebfff700 1 --2- 172.21.15.99:0/4171480249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08ec0af800 0x7f08ec154480 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00b3c0 tx=0x7f08dc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:49.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.302+0000 7f08d3fff700 1 -- 172.21.15.99:0/4171480249 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc006360 con 0x7f08ec0af800 2023-10-08T00:12:49.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.302+0000 7f08f10a9700 1 -- 172.21.15.99:0/4171480249 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08ec0b1490 con 0x7f08ec0af800 2023-10-08T00:12:49.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.302+0000 7f08f10a9700 1 -- 172.21.15.99:0/4171480249 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08ec0b1a50 con 0x7f08ec0af800 2023-10-08T00:12:49.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.302+0000 7f08d3fff700 1 -- 172.21.15.99:0/4171480249 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08dc0066d0 con 0x7f08ec0af800 2023-10-08T00:12:49.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.302+0000 7f08d3fff700 1 -- 172.21.15.99:0/4171480249 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc006840 con 0x7f08ec0af800 2023-10-08T00:12:49.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.303+0000 7f08f10a9700 1 -- 172.21.15.99:0/4171480249 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08ec06c890 con 0x7f08ec0af800 2023-10-08T00:12:49.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.303+0000 7f08d3fff700 1 -- 172.21.15.99:0/4171480249 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f08dc01b690 con 0x7f08ec0af800 2023-10-08T00:12:49.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.304+0000 7f08d3fff700 1 --2- 172.21.15.99:0/4171480249 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f08d8066f30 0x7f08d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:49.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.304+0000 7f08d3fff700 1 -- 172.21.15.99:0/4171480249 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f08dc0863b0 con 0x7f08ec0af800 2023-10-08T00:12:49.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.304+0000 7f08eb7fe700 1 --2- 172.21.15.99:0/4171480249 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f08d8066f30 0x7f08d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:49.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.304+0000 7f08eb7fe700 1 --2- 172.21.15.99:0/4171480249 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f08d8066f30 0x7f08d80693e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f08ec06af30 tx=0x7f08e0007f50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:49.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.308+0000 7f08d3fff700 1 -- 172.21.15.99:0/4171480249 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f08dc053920 con 0x7f08ec0af800 2023-10-08T00:12:49.493 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.493+0000 7f08f10a9700 1 -- 172.21.15.99:0/4171480249 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f08ec06dcc0 con 0x7f08ec0af800 2023-10-08T00:12:49.497 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.496+0000 7f08d3fff700 1 -- 172.21.15.99:0/4171480249 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v5) v1 ==== 104+0+0 (secure 0 0 0) 0x7f08dc056f40 con 0x7f08ec0af800 2023-10-08T00:12:49.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.498+0000 7f08f10a9700 1 -- 172.21.15.99:0/4171480249 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f08d8066f30 msgr2=0x7f08d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:49.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.498+0000 7f08f10a9700 1 --2- 172.21.15.99:0/4171480249 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f08d8066f30 0x7f08d80693e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f08ec06af30 tx=0x7f08e0007f50 comp rx=0 tx=0).stop 2023-10-08T00:12:49.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.498+0000 7f08f10a9700 1 -- 172.21.15.99:0/4171480249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08ec0af800 msgr2=0x7f08ec154480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:49.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.498+0000 7f08f10a9700 1 --2- 172.21.15.99:0/4171480249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08ec0af800 0x7f08ec154480 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00b3c0 tx=0x7f08dc016000 comp rx=0 tx=0).stop 2023-10-08T00:12:49.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.498+0000 7f08f10a9700 1 -- 172.21.15.99:0/4171480249 shutdown_connections 2023-10-08T00:12:49.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.498+0000 7f08f10a9700 1 --2- 172.21.15.99:0/4171480249 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f08d8066f30 0x7f08d80693e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:49.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.498+0000 7f08f10a9700 1 --2- 172.21.15.99:0/4171480249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08ec156c00 0x7f08ec15b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:49.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.498+0000 7f08f10a9700 1 --2- 172.21.15.99:0/4171480249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08ec0af800 0x7f08ec154480 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:49.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.498+0000 7f08f10a9700 1 --2- 172.21.15.99:0/4171480249 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08ec156830 0x7f08ec0abbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:49.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.498+0000 7f08f10a9700 1 -- 172.21.15.99:0/4171480249 >> 172.21.15.99:0/4171480249 conn(0x7f08ec067660 msgr2=0x7f08ec0cd190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:49.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.498+0000 7f08f10a9700 1 -- 172.21.15.99:0/4171480249 shutdown_connections 2023-10-08T00:12:49.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.498+0000 7f08f10a9700 1 -- 172.21.15.99:0/4171480249 wait complete. 2023-10-08T00:12:49.511 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_different_file' 2023-10-08T00:12:49.624 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.623+0000 7f41b95f7700 1 Processor -- start 2023-10-08T00:12:49.624 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.623+0000 7f41b95f7700 1 -- start start 2023-10-08T00:12:49.627 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.624+0000 7f41b95f7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41b415cbd0 0x7f41b415d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:49.627 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.624+0000 7f41b95f7700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41b40cb7c0 con 0x7f41b40cd6e0 2023-10-08T00:12:49.628 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.624+0000 7f41b95f7700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41b415e290 con 0x7f41b40cdab0 2023-10-08T00:12:49.628 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.624+0000 7f41b95f7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41b415e850 con 0x7f41b415cbd0 2023-10-08T00:12:49.628 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.624+0000 7f41b3fff700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f41b40cd6e0 0x7f41b40cbc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59508/0 (socket says 172.21.15.99:59508) 2023-10-08T00:12:49.628 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.624+0000 7f41b3fff700 1 -- 172.21.15.99:0/1015954512 learned_addr learned my addr 172.21.15.99:0/1015954512 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:12:49.628 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.624+0000 7f41b2ffd700 1 -- 172.21.15.99:0/1015954512 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1770108644 0 0) 0x7f41b40cb7c0 con 0x7f41b40cd6e0 2023-10-08T00:12:49.628 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.624+0000 7f41b8df6700 1 --2- 172.21.15.99:0/1015954512 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41b415cbd0 0x7f41b415d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:49.628 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.625+0000 7f41b2ffd700 1 -- 172.21.15.99:0/1015954512 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f419c003570 con 0x7f41b40cd6e0 2023-10-08T00:12:49.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.625+0000 7f41b2ffd700 1 -- 172.21.15.99:0/1015954512 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3983741874 0 0) 0x7f419c003570 con 0x7f41b40cd6e0 2023-10-08T00:12:49.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.625+0000 7f41b2ffd700 1 -- 172.21.15.99:0/1015954512 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41b415cbd0 msgr2=0x7f41b415d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:49.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.625+0000 7f41b2ffd700 1 --2- 172.21.15.99:0/1015954512 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41b415cbd0 0x7f41b415d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:49.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.625+0000 7f41b2ffd700 1 -- 172.21.15.99:0/1015954512 >> v1:172.21.15.191:6789/0 conn(0x7f41b40cdab0 legacy=0x7f41b415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:49.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.625+0000 7f41b2ffd700 1 -- 172.21.15.99:0/1015954512 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b415e570 con 0x7f41b40cd6e0 2023-10-08T00:12:49.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.625+0000 7f41b2ffd700 1 -- 172.21.15.99:0/1015954512 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f41a4001470 con 0x7f41b40cd6e0 2023-10-08T00:12:49.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.625+0000 7f41b2ffd700 1 -- 172.21.15.99:0/1015954512 >> v1:172.21.15.99:6789/0 conn(0x7f41b40cd6e0 legacy=0x7f41b40cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:49.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.625+0000 7f41b2ffd700 1 --2- 172.21.15.99:0/1015954512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f419c003a90 0x7f419c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:49.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.625+0000 7f41b2ffd700 1 --2- 172.21.15.99:0/1015954512 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41b415cbd0 0x7f419c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:49.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.625+0000 7f41b2ffd700 1 --2- 172.21.15.99:0/1015954512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f419c004d50 0x7f419c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:49.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.625+0000 7f41b2ffd700 1 -- 172.21.15.99:0/1015954512 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b415e850 con 0x7f419c003a90 2023-10-08T00:12:49.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.625+0000 7f41b2ffd700 1 -- 172.21.15.99:0/1015954512 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f419c00bb70 con 0x7f419c004d50 2023-10-08T00:12:49.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.625+0000 7f41b2ffd700 1 -- 172.21.15.99:0/1015954512 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f419c00bd30 con 0x7f41b415cbd0 2023-10-08T00:12:49.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.625+0000 7f41b3fff700 1 --2- 172.21.15.99:0/1015954512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f419c003a90 0x7f419c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:49.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.625+0000 7f41b37fe700 1 --2- 172.21.15.99:0/1015954512 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41b415cbd0 0x7f419c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:49.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.625+0000 7f41b8df6700 1 --2- 172.21.15.99:0/1015954512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f419c004d50 0x7f419c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:49.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.625+0000 7f41b3fff700 1 -- 172.21.15.99:0/1015954512 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41b415cbd0 msgr2=0x7f419c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:49.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.625+0000 7f41b3fff700 1 --2- 172.21.15.99:0/1015954512 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41b415cbd0 0x7f419c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:49.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.625+0000 7f41b3fff700 1 -- 172.21.15.99:0/1015954512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f419c004d50 msgr2=0x7f419c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:49.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.625+0000 7f41b3fff700 1 --2- 172.21.15.99:0/1015954512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f419c004d50 0x7f419c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:49.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.625+0000 7f41b3fff700 1 -- 172.21.15.99:0/1015954512 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f419c003570 con 0x7f419c003a90 2023-10-08T00:12:49.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.625+0000 7f41b3fff700 1 --2- 172.21.15.99:0/1015954512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f419c003a90 0x7f419c003ea0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f41a400c3c0 tx=0x7f41a4001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5fb6103dd523114 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:49.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.625+0000 7f41b2ffd700 1 -- 172.21.15.99:0/1015954512 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a400cc90 con 0x7f419c003a90 2023-10-08T00:12:49.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.625+0000 7f41b2ffd700 1 -- 172.21.15.99:0/1015954512 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41a400cdf0 con 0x7f419c003a90 2023-10-08T00:12:49.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.626+0000 7f41b2ffd700 1 -- 172.21.15.99:0/1015954512 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a4005b50 con 0x7f419c003a90 2023-10-08T00:12:49.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.626+0000 7f41b95f7700 1 -- 172.21.15.99:0/1015954512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f419c003a90 msgr2=0x7f419c003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:49.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.626+0000 7f41b95f7700 1 --2- 172.21.15.99:0/1015954512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f419c003a90 0x7f419c003ea0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f41a400c3c0 tx=0x7f41a4001e50 comp rx=0 tx=0).stop 2023-10-08T00:12:49.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.626+0000 7f41b8df6700 1 -- 172.21.15.99:0/1015954512 reap_dead start 2023-10-08T00:12:49.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.626+0000 7f41b95f7700 1 -- 172.21.15.99:0/1015954512 shutdown_connections 2023-10-08T00:12:49.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.626+0000 7f41b95f7700 1 -- 172.21.15.99:0/1015954512 >> 172.21.15.99:0/1015954512 conn(0x7f41b4067660 msgr2=0x7f41b4067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:49.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.626+0000 7f41b95f7700 1 -- 172.21.15.99:0/1015954512 shutdown_connections 2023-10-08T00:12:49.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.626+0000 7f41b95f7700 1 -- 172.21.15.99:0/1015954512 wait complete. 2023-10-08T00:12:49.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.627+0000 7f41b95f7700 1 Processor -- start 2023-10-08T00:12:49.635 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.627+0000 7f41b95f7700 1 -- start start 2023-10-08T00:12:49.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.628+0000 7f41b95f7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f419c003a90 0x7f41b40b53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:49.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.628+0000 7f41b95f7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41b414d630 0x7f41b4156000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:49.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.628+0000 7f41b95f7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41b414daa0 0x7f41b4156540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:49.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.628+0000 7f41b95f7700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b40cc360 con 0x7f41b414daa0 2023-10-08T00:12:49.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.628+0000 7f41b95f7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b40ca140 con 0x7f419c003a90 2023-10-08T00:12:49.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.628+0000 7f41b95f7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41b40b5eb0 con 0x7f41b414d630 2023-10-08T00:12:49.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.628+0000 7f41b8df6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41b414daa0 0x7f41b4156540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:49.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.628+0000 7f41b8df6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41b414daa0 0x7f41b4156540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55554/0 (socket says 172.21.15.99:55554) 2023-10-08T00:12:49.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.628+0000 7f41b8df6700 1 -- 172.21.15.99:0/3954861335 learned_addr learned my addr 172.21.15.99:0/3954861335 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:49.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.628+0000 7f41b3fff700 1 --2- 172.21.15.99:0/3954861335 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f419c003a90 0x7f41b40b53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:49.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.628+0000 7f41b37fe700 1 --2- 172.21.15.99:0/3954861335 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41b414d630 0x7f41b4156000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:49.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.628+0000 7f41b8df6700 1 -- 172.21.15.99:0/3954861335 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41b414d630 msgr2=0x7f41b4156000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:49.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.628+0000 7f41b8df6700 1 --2- 172.21.15.99:0/3954861335 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41b414d630 0x7f41b4156000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:49.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.628+0000 7f41b8df6700 1 -- 172.21.15.99:0/3954861335 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f419c003a90 msgr2=0x7f41b40b53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:49.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.628+0000 7f41b8df6700 1 --2- 172.21.15.99:0/3954861335 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f419c003a90 0x7f41b40b53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:49.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.629+0000 7f41b8df6700 1 -- 172.21.15.99:0/3954861335 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b40c9530 con 0x7f41b414daa0 2023-10-08T00:12:49.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.629+0000 7f41b8df6700 1 --2- 172.21.15.99:0/3954861335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41b414daa0 0x7f41b4156540 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f41b40c7cd0 tx=0x7f41ac00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:49.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.629+0000 7f419bfff700 1 -- 172.21.15.99:0/3954861335 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ac00e040 con 0x7f41b414daa0 2023-10-08T00:12:49.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.629+0000 7f419bfff700 1 -- 172.21.15.99:0/3954861335 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41ac00f040 con 0x7f41b414daa0 2023-10-08T00:12:49.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.629+0000 7f419bfff700 1 -- 172.21.15.99:0/3954861335 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ac00fde0 con 0x7f41b414daa0 2023-10-08T00:12:49.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.629+0000 7f41b95f7700 1 -- 172.21.15.99:0/3954861335 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41b40c9810 con 0x7f41b414daa0 2023-10-08T00:12:49.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.629+0000 7f41b95f7700 1 -- 172.21.15.99:0/3954861335 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41b40c9dd0 con 0x7f41b414daa0 2023-10-08T00:12:49.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.630+0000 7f41b95f7700 1 -- 172.21.15.99:0/3954861335 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41b4089280 con 0x7f41b414daa0 2023-10-08T00:12:49.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.630+0000 7f419bfff700 1 -- 172.21.15.99:0/3954861335 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f41ac015640 con 0x7f41b414daa0 2023-10-08T00:12:49.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.630+0000 7f419bfff700 1 --2- 172.21.15.99:0/3954861335 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f41a0066f30 0x7f41a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:49.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.630+0000 7f419bfff700 1 -- 172.21.15.99:0/3954861335 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f41ac084bc0 con 0x7f41b414daa0 2023-10-08T00:12:49.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.631+0000 7f41b3fff700 1 --2- 172.21.15.99:0/3954861335 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f41a0066f30 0x7f41a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:49.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.631+0000 7f41b3fff700 1 --2- 172.21.15.99:0/3954861335 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f41a0066f30 0x7f41a00693e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f41a4007cd0 tx=0x7f41a4015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:49.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.635+0000 7f419bfff700 1 -- 172.21.15.99:0/3954861335 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f41ac052130 con 0x7f41b414daa0 2023-10-08T00:12:49.894 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:49.893+0000 7f41b95f7700 1 -- 172.21.15.99:0/3954861335 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]} v 0) v1 -- 0x7f41b404bb80 con 0x7f41b414daa0 2023-10-08T00:12:50.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.174+0000 7f419bfff700 1 -- 172.21.15.99:0/3954861335 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]}]=0 v127) v1 ==== 142+0+0 (secure 0 0 0) 0x7f41ac055750 con 0x7f41b414daa0 2023-10-08T00:12:50.177 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.176+0000 7f41b95f7700 1 -- 172.21.15.99:0/3954861335 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f41a0066f30 msgr2=0x7f41a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:50.177 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.176+0000 7f41b95f7700 1 --2- 172.21.15.99:0/3954861335 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f41a0066f30 0x7f41a00693e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f41a4007cd0 tx=0x7f41a4015000 comp rx=0 tx=0).stop 2023-10-08T00:12:50.177 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.176+0000 7f41b95f7700 1 -- 172.21.15.99:0/3954861335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41b414daa0 msgr2=0x7f41b4156540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:50.177 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.176+0000 7f41b95f7700 1 --2- 172.21.15.99:0/3954861335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41b414daa0 0x7f41b4156540 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f41b40c7cd0 tx=0x7f41ac00b040 comp rx=0 tx=0).stop 2023-10-08T00:12:50.177 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.176+0000 7f41b95f7700 1 -- 172.21.15.99:0/3954861335 shutdown_connections 2023-10-08T00:12:50.177 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.176+0000 7f41b95f7700 1 --2- 172.21.15.99:0/3954861335 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f41a0066f30 0x7f41a00693e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:50.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.176+0000 7f41b95f7700 1 --2- 172.21.15.99:0/3954861335 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f419c003a90 0x7f41b40b53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:50.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.176+0000 7f41b95f7700 1 --2- 172.21.15.99:0/3954861335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41b414daa0 0x7f41b4156540 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:50.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.176+0000 7f41b95f7700 1 --2- 172.21.15.99:0/3954861335 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41b414d630 0x7f41b4156000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:50.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.176+0000 7f41b95f7700 1 -- 172.21.15.99:0/3954861335 >> 172.21.15.99:0/3954861335 conn(0x7f41b4067660 msgr2=0x7f41b40b07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:50.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.176+0000 7f41b95f7700 1 -- 172.21.15.99:0/3954861335 shutdown_connections 2023-10-08T00:12:50.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.176+0000 7f41b95f7700 1 -- 172.21.15.99:0/3954861335 wait complete. 2023-10-08T00:12:50.188 INFO:tasks.cephfs_test_runner:test_full_different_file (tasks.cephfs.test_full.TestClusterFull) ... ok 2023-10-08T00:12:50.189 INFO:tasks.cephfs_test_runner:Starting test: test_full_fclose (tasks.cephfs.test_full.TestClusterFull) 2023-10-08T00:12:50.189 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fclose' 2023-10-08T00:12:50.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.301+0000 7f2ea0846700 1 Processor -- start 2023-10-08T00:12:50.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.301+0000 7f2ea0846700 1 -- start start 2023-10-08T00:12:50.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.302+0000 7f2ea0846700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e9c0cd6e0 0x7f2e9c0cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:50.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.302+0000 7f2ea0846700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2e9c15cbd0 0x7f2e9c15d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:50.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.302+0000 7f2ea0846700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e9c0cb7c0 con 0x7f2e9c0cdab0 2023-10-08T00:12:50.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.302+0000 7f2ea0846700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e9c15e850 con 0x7f2e9c15cbd0 2023-10-08T00:12:50.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.302+0000 7f2ea0846700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c0cd030 con 0x7f2e9c0cd6e0 2023-10-08T00:12:50.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.302+0000 7f2e9bfff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2e9c15cbd0 0x7f2e9c15d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:50.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.302+0000 7f2e9affd700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f2e9c0cdab0 0x7f2e9c15d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59514/0 (socket says 172.21.15.99:59514) 2023-10-08T00:12:50.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.302+0000 7f2e9affd700 1 -- 172.21.15.99:0/86446802 learned_addr learned my addr 172.21.15.99:0/86446802 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:12:50.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.303+0000 7f2e9b7fe700 1 --2- 172.21.15.99:0/86446802 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e9c0cd6e0 0x7f2e9c0cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:50.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.303+0000 7f2e9bfff700 1 -- 172.21.15.99:0/86446802 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e9c0cd6e0 msgr2=0x7f2e9c0cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:50.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.303+0000 7f2e9bfff700 1 --2- 172.21.15.99:0/86446802 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e9c0cd6e0 0x7f2e9c0cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:50.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.303+0000 7f2e9bfff700 1 -- 172.21.15.99:0/86446802 >> v1:172.21.15.99:6789/0 conn(0x7f2e9c0cdab0 legacy=0x7f2e9c15d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:50.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.303+0000 7f2e9bfff700 1 -- 172.21.15.99:0/86446802 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e9c15e290 con 0x7f2e9c15cbd0 2023-10-08T00:12:50.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.303+0000 7f2e9bfff700 1 --2- 172.21.15.99:0/86446802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e9c15cbd0 0x7f2e9c15d020 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f2e940098e0 tx=0x7f2e940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21543a97bc551bc7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:50.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.303+0000 7f2e9a7fc700 1 -- 172.21.15.99:0/86446802 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e94009e80 con 0x7f2e9c15cbd0 2023-10-08T00:12:50.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.303+0000 7f2e9a7fc700 1 -- 172.21.15.99:0/86446802 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e9400b860 con 0x7f2e9c15cbd0 2023-10-08T00:12:50.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.303+0000 7f2e9a7fc700 1 -- 172.21.15.99:0/86446802 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9400bbe0 con 0x7f2e9c15cbd0 2023-10-08T00:12:50.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.303+0000 7f2ea0846700 1 -- 172.21.15.99:0/86446802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e9c15cbd0 msgr2=0x7f2e9c15d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:50.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.303+0000 7f2ea0846700 1 --2- 172.21.15.99:0/86446802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e9c15cbd0 0x7f2e9c15d020 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f2e940098e0 tx=0x7f2e940091a0 comp rx=0 tx=0).stop 2023-10-08T00:12:50.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.303+0000 7f2ea0846700 1 -- 172.21.15.99:0/86446802 shutdown_connections 2023-10-08T00:12:50.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.303+0000 7f2ea0846700 1 --2- 172.21.15.99:0/86446802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e9c15cbd0 0x7f2e9c15d020 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:50.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.303+0000 7f2ea0846700 1 --2- 172.21.15.99:0/86446802 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e9c0cd6e0 0x7f2e9c0cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:50.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.303+0000 7f2ea0846700 1 -- 172.21.15.99:0/86446802 >> 172.21.15.99:0/86446802 conn(0x7f2e9c067660 msgr2=0x7f2e9c067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:50.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.304+0000 7f2ea0846700 1 -- 172.21.15.99:0/86446802 shutdown_connections 2023-10-08T00:12:50.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.304+0000 7f2ea0846700 1 -- 172.21.15.99:0/86446802 wait complete. 2023-10-08T00:12:50.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.305+0000 7f2ea0846700 1 Processor -- start 2023-10-08T00:12:50.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.305+0000 7f2ea0846700 1 -- start start 2023-10-08T00:12:50.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.305+0000 7f2ea0846700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e9c0cd6e0 0x7f2e9c0b2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:50.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.305+0000 7f2ea0846700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e9c0cdab0 0x7f2e9c0c8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:50.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.305+0000 7f2ea0846700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e9c15cbd0 0x7f2e9c15aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:50.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.305+0000 7f2ea0846700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c159290 con 0x7f2e9c0cd6e0 2023-10-08T00:12:50.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.305+0000 7f2ea0846700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c0b1e80 con 0x7f2e9c15cbd0 2023-10-08T00:12:50.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.306+0000 7f2ea0846700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c1558c0 con 0x7f2e9c0cdab0 2023-10-08T00:12:50.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.306+0000 7f2e9b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e9c0cd6e0 0x7f2e9c0b2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:50.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.306+0000 7f2e9b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e9c0cd6e0 0x7f2e9c0b2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55578/0 (socket says 172.21.15.99:55578) 2023-10-08T00:12:50.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.306+0000 7f2e9b7fe700 1 -- 172.21.15.99:0/358049087 learned_addr learned my addr 172.21.15.99:0/358049087 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:50.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.306+0000 7f2e9affd700 1 --2- 172.21.15.99:0/358049087 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e9c0cdab0 0x7f2e9c0c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:50.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.306+0000 7f2e9bfff700 1 --2- 172.21.15.99:0/358049087 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e9c15cbd0 0x7f2e9c15aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:50.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.306+0000 7f2e9b7fe700 1 -- 172.21.15.99:0/358049087 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e9c0cdab0 msgr2=0x7f2e9c0c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:50.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.306+0000 7f2e9b7fe700 1 --2- 172.21.15.99:0/358049087 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e9c0cdab0 0x7f2e9c0c8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:50.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.306+0000 7f2e9b7fe700 1 -- 172.21.15.99:0/358049087 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e9c15cbd0 msgr2=0x7f2e9c15aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:50.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.306+0000 7f2e9b7fe700 1 --2- 172.21.15.99:0/358049087 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e9c15cbd0 0x7f2e9c15aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:50.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.306+0000 7f2e9b7fe700 1 -- 172.21.15.99:0/358049087 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e94009420 con 0x7f2e9c0cd6e0 2023-10-08T00:12:50.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.306+0000 7f2e9b7fe700 1 --2- 172.21.15.99:0/358049087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e9c0cd6e0 0x7f2e9c0b2310 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c00e190 tx=0x7f2e8c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:50.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.306+0000 7f2e837fe700 1 -- 172.21.15.99:0/358049087 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c00ee30 con 0x7f2e9c0cd6e0 2023-10-08T00:12:50.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.306+0000 7f2e837fe700 1 -- 172.21.15.99:0/358049087 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e8c014420 con 0x7f2e9c0cd6e0 2023-10-08T00:12:50.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.306+0000 7f2ea0846700 1 -- 172.21.15.99:0/358049087 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e9c0b2850 con 0x7f2e9c0cd6e0 2023-10-08T00:12:50.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.306+0000 7f2e837fe700 1 -- 172.21.15.99:0/358049087 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c0147e0 con 0x7f2e9c0cd6e0 2023-10-08T00:12:50.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.306+0000 7f2ea0846700 1 -- 172.21.15.99:0/358049087 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e9c0b1320 con 0x7f2e9c0cd6e0 2023-10-08T00:12:50.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.307+0000 7f2ea0846700 1 -- 172.21.15.99:0/358049087 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e9c089280 con 0x7f2e9c0cd6e0 2023-10-08T00:12:50.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.307+0000 7f2e837fe700 1 -- 172.21.15.99:0/358049087 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2e8c014b90 con 0x7f2e9c0cd6e0 2023-10-08T00:12:50.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.308+0000 7f2e837fe700 1 --2- 172.21.15.99:0/358049087 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2e88066f30 0x7f2e880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:50.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.308+0000 7f2e837fe700 1 -- 172.21.15.99:0/358049087 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f2e8c084a40 con 0x7f2e9c0cd6e0 2023-10-08T00:12:50.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.308+0000 7f2e9affd700 1 --2- 172.21.15.99:0/358049087 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2e88066f30 0x7f2e880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:50.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.308+0000 7f2e9affd700 1 --2- 172.21.15.99:0/358049087 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2e88066f30 0x7f2e880693e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c158f10 tx=0x7f2e90009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:50.313 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.312+0000 7f2e837fe700 1 -- 172.21.15.99:0/358049087 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2e8c086e60 con 0x7f2e9c0cd6e0 2023-10-08T00:12:50.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:50.572+0000 7f2ea0846700 1 -- 172.21.15.99:0/358049087 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]} v 0) v1 -- 0x7f2e9c15e570 con 0x7f2e9c0cd6e0 2023-10-08T00:12:51.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.174+0000 7f2e837fe700 1 -- 172.21.15.99:0/358049087 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]}]=0 v128) v1 ==== 137+0+0 (secure 0 0 0) 0x7f2e8c051fb0 con 0x7f2e9c0cd6e0 2023-10-08T00:12:51.177 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.176+0000 7f2ea0846700 1 -- 172.21.15.99:0/358049087 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2e88066f30 msgr2=0x7f2e880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:51.177 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.176+0000 7f2ea0846700 1 --2- 172.21.15.99:0/358049087 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2e88066f30 0x7f2e880693e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c158f10 tx=0x7f2e90009040 comp rx=0 tx=0).stop 2023-10-08T00:12:51.177 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.176+0000 7f2ea0846700 1 -- 172.21.15.99:0/358049087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e9c0cd6e0 msgr2=0x7f2e9c0b2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:51.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.176+0000 7f2ea0846700 1 --2- 172.21.15.99:0/358049087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e9c0cd6e0 0x7f2e9c0b2310 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c00e190 tx=0x7f2e8c009920 comp rx=0 tx=0).stop 2023-10-08T00:12:51.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.176+0000 7f2ea0846700 1 -- 172.21.15.99:0/358049087 shutdown_connections 2023-10-08T00:12:51.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.176+0000 7f2ea0846700 1 --2- 172.21.15.99:0/358049087 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2e88066f30 0x7f2e880693e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:51.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.176+0000 7f2ea0846700 1 --2- 172.21.15.99:0/358049087 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e9c15cbd0 0x7f2e9c15aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:51.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.176+0000 7f2ea0846700 1 --2- 172.21.15.99:0/358049087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e9c0cd6e0 0x7f2e9c0b2310 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:51.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.176+0000 7f2ea0846700 1 --2- 172.21.15.99:0/358049087 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e9c0cdab0 0x7f2e9c0c8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:51.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.176+0000 7f2ea0846700 1 -- 172.21.15.99:0/358049087 >> 172.21.15.99:0/358049087 conn(0x7f2e9c067660 msgr2=0x7f2e9c0b2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:51.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.177+0000 7f2ea0846700 1 -- 172.21.15.99:0/358049087 shutdown_connections 2023-10-08T00:12:51.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.177+0000 7f2ea0846700 1 -- 172.21.15.99:0/358049087 wait complete. 2023-10-08T00:12:51.189 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2023-10-08T00:12:51.316 INFO:teuthology.orchestra.run.smithi099.stdout:{"osd_objectstore":"memstore"} 2023-10-08T00:12:51.317 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2023-10-08T00:12:51.317 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T00:12:51.434 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.433+0000 7f03a7a32700 1 Processor -- start 2023-10-08T00:12:51.434 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.433+0000 7f03a7a32700 1 -- start start 2023-10-08T00:12:51.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.434+0000 7f03a7a32700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f03a015b990 0x7f03a015a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:51.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.434+0000 7f03a7a32700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03a00b14f0 con 0x7f03a00b4860 2023-10-08T00:12:51.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.434+0000 7f03a7a32700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03a00b17d0 con 0x7f03a00b3570 2023-10-08T00:12:51.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.434+0000 7f03a7a32700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03a00b1ab0 con 0x7f03a015b990 2023-10-08T00:12:51.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.434+0000 7f03a5fcf700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f03a015b990 0x7f03a015a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:51.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.434+0000 7f03a57ce700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f03a00b4860 0x7f03a0159a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59522/0 (socket says 172.21.15.99:59522) 2023-10-08T00:12:51.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.434+0000 7f03a57ce700 1 -- 172.21.15.99:0/252421878 learned_addr learned my addr 172.21.15.99:0/252421878 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:12:51.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.434+0000 7f03a5fcf700 1 -- 172.21.15.99:0/252421878 >> v1:172.21.15.191:6789/0 conn(0x7f03a00b3570 legacy=0x7f03a00cfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:51.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.434+0000 7f03a5fcf700 1 -- 172.21.15.99:0/252421878 >> v1:172.21.15.99:6789/0 conn(0x7f03a00b4860 legacy=0x7f03a0159a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:51.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.434+0000 7f03a5fcf700 1 -- 172.21.15.99:0/252421878 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a00b1d90 con 0x7f03a015b990 2023-10-08T00:12:51.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.434+0000 7f0397fff700 1 -- 172.21.15.99:0/252421878 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2600794901 0 0) 0x7f03a00b14f0 con 0x7f03a00b4860 2023-10-08T00:12:51.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.434+0000 7f03a5fcf700 1 --2- 172.21.15.99:0/252421878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03a015b990 0x7f03a015a1a0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f03980098e0 tx=0x7f03980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=813b8fa2d5b9fc93 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:51.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.435+0000 7f0397fff700 1 -- 172.21.15.99:0/252421878 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0398009e80 con 0x7f03a015b990 2023-10-08T00:12:51.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.435+0000 7f0397fff700 1 -- 172.21.15.99:0/252421878 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f039800b8b0 con 0x7f03a015b990 2023-10-08T00:12:51.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.435+0000 7f0397fff700 1 -- 172.21.15.99:0/252421878 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039800bc30 con 0x7f03a015b990 2023-10-08T00:12:51.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.435+0000 7f03a7a32700 1 -- 172.21.15.99:0/252421878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03a015b990 msgr2=0x7f03a015a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:51.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.435+0000 7f03a7a32700 1 --2- 172.21.15.99:0/252421878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03a015b990 0x7f03a015a1a0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f03980098e0 tx=0x7f03980091a0 comp rx=0 tx=0).stop 2023-10-08T00:12:51.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.435+0000 7f03a7a32700 1 -- 172.21.15.99:0/252421878 shutdown_connections 2023-10-08T00:12:51.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.435+0000 7f03a7a32700 1 --2- 172.21.15.99:0/252421878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03a015b990 0x7f03a015a1a0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:51.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.435+0000 7f03a7a32700 1 -- 172.21.15.99:0/252421878 >> 172.21.15.99:0/252421878 conn(0x7f03a009a0f0 msgr2=0x7f03a0067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:51.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.435+0000 7f03a7a32700 1 -- 172.21.15.99:0/252421878 shutdown_connections 2023-10-08T00:12:51.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.435+0000 7f03a7a32700 1 -- 172.21.15.99:0/252421878 wait complete. 2023-10-08T00:12:51.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.436+0000 7f03a7a32700 1 Processor -- start 2023-10-08T00:12:51.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.436+0000 7f03a7a32700 1 -- start start 2023-10-08T00:12:51.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.437+0000 7f03a7a32700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f03a00b3570 0x7f03a0158190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:51.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.437+0000 7f03a7a32700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03a00b4860 0x7f03a00aa700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:51.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.437+0000 7f03a7a32700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03a015b990 0x7f03a00aca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:51.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.437+0000 7f03a7a32700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a0169a70 con 0x7f03a00b4860 2023-10-08T00:12:51.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.437+0000 7f03a7a32700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a016aa20 con 0x7f03a015b990 2023-10-08T00:12:51.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.437+0000 7f03a7a32700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03a016a590 con 0x7f03a00b3570 2023-10-08T00:12:51.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.437+0000 7f03a4fcd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03a00b4860 0x7f03a00aa700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:51.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.437+0000 7f03a4fcd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03a00b4860 0x7f03a00aa700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55608/0 (socket says 172.21.15.99:55608) 2023-10-08T00:12:51.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.437+0000 7f03a4fcd700 1 -- 172.21.15.99:0/4167439912 learned_addr learned my addr 172.21.15.99:0/4167439912 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:51.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.438+0000 7f03a57ce700 1 --2- 172.21.15.99:0/4167439912 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f03a00b3570 0x7f03a0158190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:51.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.438+0000 7f03a5fcf700 1 --2- 172.21.15.99:0/4167439912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03a015b990 0x7f03a00aca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:51.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.438+0000 7f03a4fcd700 1 -- 172.21.15.99:0/4167439912 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f03a00b3570 msgr2=0x7f03a0158190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:51.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.438+0000 7f03a4fcd700 1 --2- 172.21.15.99:0/4167439912 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f03a00b3570 0x7f03a0158190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:51.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.438+0000 7f03a4fcd700 1 -- 172.21.15.99:0/4167439912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03a015b990 msgr2=0x7f03a00aca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:51.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.438+0000 7f03a4fcd700 1 --2- 172.21.15.99:0/4167439912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03a015b990 0x7f03a00aca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:51.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.438+0000 7f03a4fcd700 1 -- 172.21.15.99:0/4167439912 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0398009420 con 0x7f03a00b4860 2023-10-08T00:12:51.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.438+0000 7f03a57ce700 1 --2- 172.21.15.99:0/4167439912 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f03a00b3570 0x7f03a0158190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:12:51.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.438+0000 7f03a5fcf700 1 --2- 172.21.15.99:0/4167439912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03a015b990 0x7f03a00aca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:12:51.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.438+0000 7f03a4fcd700 1 --2- 172.21.15.99:0/4167439912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03a00b4860 0x7f03a00aa700 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f039000c600 tx=0x7f0390007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:51.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.438+0000 7f03957fa700 1 -- 172.21.15.99:0/4167439912 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0390011590 con 0x7f03a00b4860 2023-10-08T00:12:51.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.438+0000 7f03957fa700 1 -- 172.21.15.99:0/4167439912 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0390011710 con 0x7f03a00b4860 2023-10-08T00:12:51.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.438+0000 7f03a7a32700 1 -- 172.21.15.99:0/4167439912 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03a0169c60 con 0x7f03a00b4860 2023-10-08T00:12:51.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.438+0000 7f03957fa700 1 -- 172.21.15.99:0/4167439912 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0390011ad0 con 0x7f03a00b4860 2023-10-08T00:12:51.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.438+0000 7f03a7a32700 1 -- 172.21.15.99:0/4167439912 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03a016a220 con 0x7f03a00b4860 2023-10-08T00:12:51.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.439+0000 7f03a7a32700 1 -- 172.21.15.99:0/4167439912 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03a016a3b0 con 0x7f03a00b4860 2023-10-08T00:12:51.443 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.439+0000 7f03957fa700 1 -- 172.21.15.99:0/4167439912 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f039001c460 con 0x7f03a00b4860 2023-10-08T00:12:51.443 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.440+0000 7f03957fa700 1 --2- 172.21.15.99:0/4167439912 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0384066f30 0x7f03840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:51.443 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.440+0000 7f03957fa700 1 -- 172.21.15.99:0/4167439912 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f0390084010 con 0x7f03a00b4860 2023-10-08T00:12:51.443 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.440+0000 7f03a57ce700 1 --2- 172.21.15.99:0/4167439912 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0384066f30 0x7f03840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:51.443 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.440+0000 7f03a57ce700 1 --2- 172.21.15.99:0/4167439912 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0384066f30 0x7f03840693e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f039c001f50 tx=0x7f039c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:51.444 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.444+0000 7f03957fa700 1 -- 172.21.15.99:0/4167439912 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0390051580 con 0x7f03a00b4860 2023-10-08T00:12:51.624 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.624+0000 7f03a7a32700 1 -- 172.21.15.99:0/4167439912 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f03a004c3c0 con 0x7f03a00b4860 2023-10-08T00:12:51.628 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.627+0000 7f03957fa700 1 -- 172.21.15.99:0/4167439912 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v6)=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0390054ba0 con 0x7f03a00b4860 2023-10-08T00:12:51.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.629+0000 7f03a7a32700 1 -- 172.21.15.99:0/4167439912 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0384066f30 msgr2=0x7f03840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:51.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.629+0000 7f03a7a32700 1 --2- 172.21.15.99:0/4167439912 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0384066f30 0x7f03840693e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f039c001f50 tx=0x7f039c009040 comp rx=0 tx=0).stop 2023-10-08T00:12:51.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.629+0000 7f03a7a32700 1 -- 172.21.15.99:0/4167439912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03a00b4860 msgr2=0x7f03a00aa700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:51.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.629+0000 7f03a7a32700 1 --2- 172.21.15.99:0/4167439912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03a00b4860 0x7f03a00aa700 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f039000c600 tx=0x7f0390007f60 comp rx=0 tx=0).stop 2023-10-08T00:12:51.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.629+0000 7f03a7a32700 1 -- 172.21.15.99:0/4167439912 shutdown_connections 2023-10-08T00:12:51.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.629+0000 7f03a7a32700 1 --2- 172.21.15.99:0/4167439912 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0384066f30 0x7f03840693e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:51.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.629+0000 7f03a7a32700 1 --2- 172.21.15.99:0/4167439912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03a015b990 0x7f03a00aca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:51.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.629+0000 7f03a7a32700 1 --2- 172.21.15.99:0/4167439912 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f03a00b3570 0x7f03a0158190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:51.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.629+0000 7f03a7a32700 1 --2- 172.21.15.99:0/4167439912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03a00b4860 0x7f03a00aa700 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:51.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.629+0000 7f03a7a32700 1 -- 172.21.15.99:0/4167439912 >> 172.21.15.99:0/4167439912 conn(0x7f03a009a0f0 msgr2=0x7f03a00b4c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:51.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.629+0000 7f03a7a32700 1 -- 172.21.15.99:0/4167439912 shutdown_connections 2023-10-08T00:12:51.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.629+0000 7f03a7a32700 1 -- 172.21.15.99:0/4167439912 wait complete. 2023-10-08T00:12:51.642 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T00:12:51.642 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T00:12:51.688 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T00:12:51.733 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T00:12:51.733 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-08T00:12:51.733 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:12:51.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.848+0000 7fadbd450700 1 Processor -- start 2023-10-08T00:12:51.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.848+0000 7fadbd450700 1 -- start start 2023-10-08T00:12:51.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.849+0000 7fadbd450700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fadb8157870 0x7fadb815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:51.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.849+0000 7fadbd450700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fadb8157c40 0x7fadb8158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:51.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.849+0000 7fadbd450700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadb815cbb0 con 0x7fadb815a620 2023-10-08T00:12:51.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.849+0000 7fadbd450700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadb815bfa0 con 0x7fadb8157c40 2023-10-08T00:12:51.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.849+0000 7fadbd450700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadb815c280 con 0x7fadb8157870 2023-10-08T00:12:51.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.849+0000 7fadb77fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fadb8157c40 0x7fadb8158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:51.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.849+0000 7fadaffff700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fadb815a620 0x7fadb815d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59528/0 (socket says 172.21.15.99:59528) 2023-10-08T00:12:51.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.849+0000 7fadaffff700 1 -- 172.21.15.99:0/2207983736 learned_addr learned my addr 172.21.15.99:0/2207983736 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:12:51.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.849+0000 7fadb6ffd700 1 --2- 172.21.15.99:0/2207983736 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fadb8157870 0x7fadb815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:51.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.849+0000 7fadb77fe700 1 -- 172.21.15.99:0/2207983736 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fadb8157870 msgr2=0x7fadb815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:51.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.849+0000 7fadb77fe700 1 --2- 172.21.15.99:0/2207983736 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fadb8157870 0x7fadb815b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:51.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.849+0000 7fadb77fe700 1 -- 172.21.15.99:0/2207983736 >> v1:172.21.15.99:6789/0 conn(0x7fadb815a620 legacy=0x7fadb815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:51.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.849+0000 7fadb77fe700 1 -- 172.21.15.99:0/2207983736 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb815c560 con 0x7fadb8157c40 2023-10-08T00:12:51.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.849+0000 7fadb67fc700 1 -- 172.21.15.99:0/2207983736 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 204486408 0 0) 0x7fadb815cbb0 con 0x7fadb815a620 2023-10-08T00:12:51.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.849+0000 7fadb6ffd700 1 --2- 172.21.15.99:0/2207983736 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fadb8157870 0x7fadb815b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:12:51.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.849+0000 7fadb77fe700 1 --2- 172.21.15.99:0/2207983736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fadb8157c40 0x7fadb8158110 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fadb0009e60 tx=0x7fadb0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d82d71d83e682051 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:51.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.849+0000 7fadb67fc700 1 -- 172.21.15.99:0/2207983736 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb000e070 con 0x7fadb8157c40 2023-10-08T00:12:51.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.849+0000 7fadb67fc700 1 -- 172.21.15.99:0/2207983736 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb000bc30 con 0x7fadb8157c40 2023-10-08T00:12:51.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.849+0000 7fadb67fc700 1 -- 172.21.15.99:0/2207983736 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb0012430 con 0x7fadb8157c40 2023-10-08T00:12:51.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.850+0000 7fadbd450700 1 -- 172.21.15.99:0/2207983736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fadb8157c40 msgr2=0x7fadb8158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:51.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.850+0000 7fadbd450700 1 --2- 172.21.15.99:0/2207983736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fadb8157c40 0x7fadb8158110 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fadb0009e60 tx=0x7fadb0009780 comp rx=0 tx=0).stop 2023-10-08T00:12:51.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.850+0000 7fadbd450700 1 -- 172.21.15.99:0/2207983736 shutdown_connections 2023-10-08T00:12:51.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.850+0000 7fadbd450700 1 --2- 172.21.15.99:0/2207983736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fadb8157c40 0x7fadb8158110 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:51.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.850+0000 7fadbd450700 1 --2- 172.21.15.99:0/2207983736 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fadb8157870 0x7fadb815b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:51.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.850+0000 7fadbd450700 1 -- 172.21.15.99:0/2207983736 >> 172.21.15.99:0/2207983736 conn(0x7fadb8068ce0 msgr2=0x7fadb80690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:51.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.850+0000 7fadbd450700 1 -- 172.21.15.99:0/2207983736 shutdown_connections 2023-10-08T00:12:51.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.850+0000 7fadbd450700 1 -- 172.21.15.99:0/2207983736 wait complete. 2023-10-08T00:12:51.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.851+0000 7fadbd450700 1 Processor -- start 2023-10-08T00:12:51.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.851+0000 7fadbd450700 1 -- start start 2023-10-08T00:12:51.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.852+0000 7fadbd450700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fadb8157870 0x7fadb8153760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:51.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.852+0000 7fadbd450700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fadb8157c40 0x7fadb80b0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:51.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.852+0000 7fadbd450700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fadb815a620 0x7fadb814fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:51.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.852+0000 7fadbd450700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb8153ca0 con 0x7fadb8157c40 2023-10-08T00:12:51.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.852+0000 7fadbd450700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb80b0ec0 con 0x7fadb8157870 2023-10-08T00:12:51.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.852+0000 7fadbd450700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadb80b1160 con 0x7fadb815a620 2023-10-08T00:12:51.859 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.852+0000 7fadaffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fadb8157c40 0x7fadb80b0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:51.859 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.852+0000 7fadaffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fadb8157c40 0x7fadb80b0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55640/0 (socket says 172.21.15.99:55640) 2023-10-08T00:12:51.859 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.852+0000 7fadaffff700 1 -- 172.21.15.99:0/1472357670 learned_addr learned my addr 172.21.15.99:0/1472357670 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:51.859 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.852+0000 7fadb6ffd700 1 --2- 172.21.15.99:0/1472357670 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fadb8157870 0x7fadb8153760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:51.859 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.852+0000 7fadaffff700 1 -- 172.21.15.99:0/1472357670 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fadb815a620 msgr2=0x7fadb814fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:51.859 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.852+0000 7fadaffff700 1 --2- 172.21.15.99:0/1472357670 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fadb815a620 0x7fadb814fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:51.859 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.852+0000 7fadaffff700 1 -- 172.21.15.99:0/1472357670 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fadb8157870 msgr2=0x7fadb8153760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:51.859 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.852+0000 7fadaffff700 1 --2- 172.21.15.99:0/1472357670 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fadb8157870 0x7fadb8153760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:51.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.852+0000 7fadaffff700 1 -- 172.21.15.99:0/1472357670 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb0009a20 con 0x7fadb8157c40 2023-10-08T00:12:51.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.852+0000 7fadaffff700 1 --2- 172.21.15.99:0/1472357670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fadb8157c40 0x7fadb80b0980 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fada000c5b0 tx=0x7fada0001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:51.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.852+0000 7fadaeffd700 1 -- 172.21.15.99:0/1472357670 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada0005b30 con 0x7fadb8157c40 2023-10-08T00:12:51.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.853+0000 7fadaeffd700 1 -- 172.21.15.99:0/1472357670 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fada0005c90 con 0x7fadb8157c40 2023-10-08T00:12:51.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.853+0000 7fadaeffd700 1 -- 172.21.15.99:0/1472357670 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada0007300 con 0x7fadb8157c40 2023-10-08T00:12:51.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.853+0000 7fadbd450700 1 -- 172.21.15.99:0/1472357670 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadb80b1440 con 0x7fadb8157c40 2023-10-08T00:12:51.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.853+0000 7fadbd450700 1 -- 172.21.15.99:0/1472357670 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadb80b1a00 con 0x7fadb8157c40 2023-10-08T00:12:51.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.854+0000 7fadbd450700 1 -- 172.21.15.99:0/1472357670 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadb804c3c0 con 0x7fadb8157c40 2023-10-08T00:12:51.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.854+0000 7fadaeffd700 1 -- 172.21.15.99:0/1472357670 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fada001b3b0 con 0x7fadb8157c40 2023-10-08T00:12:51.861 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.854+0000 7fadaeffd700 1 --2- 172.21.15.99:0/1472357670 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fad98066f30 0x7fad980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:51.861 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.854+0000 7fadaeffd700 1 -- 172.21.15.99:0/1472357670 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fada0083760 con 0x7fadb8157c40 2023-10-08T00:12:51.861 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.855+0000 7fadb6ffd700 1 --2- 172.21.15.99:0/1472357670 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fad98066f30 0x7fad980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:51.861 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.855+0000 7fadb6ffd700 1 --2- 172.21.15.99:0/1472357670 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fad98066f30 0x7fad980693e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fada8006f00 tx=0x7fada8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:51.861 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:51.858+0000 7fadaeffd700 1 -- 172.21.15.99:0/1472357670 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fada0050cd0 con 0x7fadb8157c40 2023-10-08T00:12:52.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.082+0000 7fadbd450700 1 -- 172.21.15.99:0/1472357670 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fadb815c840 con 0x7fadb8157c40 2023-10-08T00:12:52.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.082+0000 7fadaeffd700 1 -- 172.21.15.99:0/1472357670 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+3535 (secure 0 0 0) 0x7fada0017de0 con 0x7fadb8157c40 2023-10-08T00:12:52.083 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:12:52.083 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":26,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5146,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/1438931938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1438931938},{"type":"v1","addr":"172.21.15.191:6836","nonce":1438931938}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5280,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/3090674680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3090674680},{"type":"v1","addr":"172.21.15.99:6833","nonce":3090674680}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5283,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/4025282357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":4025282357},{"type":"v1","addr":"172.21.15.99:6835","nonce":4025282357}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/2511662336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":2511662336},{"type":"v1","addr":"172.21.15.191:6837","nonce":2511662336}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[]} 2023-10-08T00:12:52.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.084+0000 7fadbd450700 1 -- 172.21.15.99:0/1472357670 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fad98066f30 msgr2=0x7fad980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:52.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.084+0000 7fadbd450700 1 --2- 172.21.15.99:0/1472357670 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fad98066f30 0x7fad980693e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fada8006f00 tx=0x7fada8008040 comp rx=0 tx=0).stop 2023-10-08T00:12:52.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.084+0000 7fadbd450700 1 -- 172.21.15.99:0/1472357670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fadb8157c40 msgr2=0x7fadb80b0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:52.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.084+0000 7fadbd450700 1 --2- 172.21.15.99:0/1472357670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fadb8157c40 0x7fadb80b0980 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fada000c5b0 tx=0x7fada0001070 comp rx=0 tx=0).stop 2023-10-08T00:12:52.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.084+0000 7fadbd450700 1 -- 172.21.15.99:0/1472357670 shutdown_connections 2023-10-08T00:12:52.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.084+0000 7fadbd450700 1 --2- 172.21.15.99:0/1472357670 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fad98066f30 0x7fad980693e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:52.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.084+0000 7fadbd450700 1 --2- 172.21.15.99:0/1472357670 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fadb8157870 0x7fadb8153760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:52.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.084+0000 7fadbd450700 1 --2- 172.21.15.99:0/1472357670 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fadb815a620 0x7fadb814fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:52.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.084+0000 7fadbd450700 1 --2- 172.21.15.99:0/1472357670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fadb8157c40 0x7fadb80b0980 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:52.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.084+0000 7fadbd450700 1 -- 172.21.15.99:0/1472357670 >> 172.21.15.99:0/1472357670 conn(0x7fadb8068ce0 msgr2=0x7fadb80d0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:52.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.084+0000 7fadbd450700 1 -- 172.21.15.99:0/1472357670 shutdown_connections 2023-10-08T00:12:52.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.084+0000 7fadbd450700 1 -- 172.21.15.99:0/1472357670 wait complete. 2023-10-08T00:12:52.086 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 26 2023-10-08T00:12:52.097 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T00:12:52.097 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-08T00:12:52.097 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T00:12:52.098 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:12:52.098 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T00:12:52.098 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-08T00:12:52.098 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T00:12:52.098 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:12:52.098 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T00:12:52.099 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-08T00:12:52.099 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T00:12:52.099 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:12:52.099 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T00:12:52.099 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-08T00:12:52.100 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T00:12:52.100 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:12:52.105 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T00:12:52.106 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T00:12:52.106 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T00:12:52.107 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T00:12:52.110 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T00:12:52.110 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T00:12:52.111 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T00:12:52.111 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T00:12:52.148 INFO:tasks.ceph.mds.a:Started 2023-10-08T00:12:52.148 DEBUG:teuthology.parallel:result is None 2023-10-08T00:12:52.149 INFO:tasks.ceph.mds.c:Started 2023-10-08T00:12:52.149 DEBUG:teuthology.parallel:result is None 2023-10-08T00:12:52.152 INFO:tasks.ceph.mds.b:Started 2023-10-08T00:12:52.152 DEBUG:teuthology.parallel:result is None 2023-10-08T00:12:52.153 INFO:tasks.ceph.mds.d:Started 2023-10-08T00:12:52.153 DEBUG:teuthology.parallel:result is None 2023-10-08T00:12:52.153 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-08T00:12:52.222 INFO:tasks.ceph.mds.a.smithi099.stdout:starting mds.a at 2023-10-08T00:12:52.222 INFO:tasks.ceph.mds.c.smithi099.stdout:starting mds.c at 2023-10-08T00:12:52.233 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2023-10-08T00:12:52.233 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2023-10-08T00:12:52.234 INFO:tasks.ceph.mds.b.smithi191.stderr:2023-10-08T00:12:52.233+0000 7f380090c700 -1 PosixStack listen unable to listen on v1:0.0.0.0:6836/0: (98) Address already in use 2023-10-08T00:12:52.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.275+0000 7fd285e80700 1 Processor -- start 2023-10-08T00:12:52.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.275+0000 7fd285e80700 1 -- start start 2023-10-08T00:12:52.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.276+0000 7fd285e80700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd2800cde30 0x7fd2800b0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:52.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.276+0000 7fd285e80700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd2800cbbe0 0x7fd2800cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:52.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.276+0000 7fd285e80700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2800cc350 con 0x7fd280159c10 2023-10-08T00:12:52.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.276+0000 7fd285e80700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2800b0870 con 0x7fd2800cbbe0 2023-10-08T00:12:52.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.276+0000 7fd285e80700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2800cee50 con 0x7fd2800cde30 2023-10-08T00:12:52.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.276+0000 7fd28567f700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd2800cbbe0 0x7fd2800cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:52.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.276+0000 7fd28567f700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd2800cbbe0 0x7fd2800cf2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55702/0 (socket says 172.21.15.99:55702) 2023-10-08T00:12:52.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.276+0000 7fd28567f700 1 -- 172.21.15.99:0/3268985652 learned_addr learned my addr 172.21.15.99:0/3268985652 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:52.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.276+0000 7fd277fff700 1 --2- 172.21.15.99:0/3268985652 >> v2:172.21.15.191:3300/0 conn(0x7fd2800cde30 0x7fd2800b0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:52.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.277+0000 7fd28567f700 1 -- 172.21.15.99:0/3268985652 >> v2:172.21.15.191:3300/0 conn(0x7fd2800cde30 msgr2=0x7fd2800b0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:52.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.277+0000 7fd28567f700 1 --2- 172.21.15.99:0/3268985652 >> v2:172.21.15.191:3300/0 conn(0x7fd2800cde30 0x7fd2800b0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:52.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.277+0000 7fd28567f700 1 -- 172.21.15.99:0/3268985652 >> v1:172.21.15.191:6789/0 conn(0x7fd280159c10 legacy=0x7fd280067660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:52.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.277+0000 7fd28567f700 1 -- 172.21.15.99:0/3268985652 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2800cc070 con 0x7fd2800cbbe0 2023-10-08T00:12:52.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.277+0000 7fd28567f700 1 --2- 172.21.15.99:0/3268985652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2800cbbe0 0x7fd2800cf2e0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fd2800b6b90 tx=0x7fd278009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=862de57545b187ff server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:52.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.277+0000 7fd2777fe700 1 -- 172.21.15.99:0/3268985652 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27800e070 con 0x7fd2800cbbe0 2023-10-08T00:12:52.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.277+0000 7fd2777fe700 1 -- 172.21.15.99:0/3268985652 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd27800ba50 con 0x7fd2800cbbe0 2023-10-08T00:12:52.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.277+0000 7fd2777fe700 1 -- 172.21.15.99:0/3268985652 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2780092e0 con 0x7fd2800cbbe0 2023-10-08T00:12:52.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.277+0000 7fd285e80700 1 -- 172.21.15.99:0/3268985652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2800cbbe0 msgr2=0x7fd2800cf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:52.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.277+0000 7fd285e80700 1 --2- 172.21.15.99:0/3268985652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2800cbbe0 0x7fd2800cf2e0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fd2800b6b90 tx=0x7fd278009840 comp rx=0 tx=0).stop 2023-10-08T00:12:52.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.277+0000 7fd285e80700 1 -- 172.21.15.99:0/3268985652 shutdown_connections 2023-10-08T00:12:52.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.277+0000 7fd285e80700 1 --2- 172.21.15.99:0/3268985652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2800cbbe0 0x7fd2800cf2e0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:52.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.277+0000 7fd285e80700 1 --2- 172.21.15.99:0/3268985652 >> v2:172.21.15.191:3300/0 conn(0x7fd2800cde30 0x7fd2800b0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:52.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.277+0000 7fd285e80700 1 -- 172.21.15.99:0/3268985652 >> 172.21.15.99:0/3268985652 conn(0x7fd2800d1660 msgr2=0x7fd2800cd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:52.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.277+0000 7fd285e80700 1 -- 172.21.15.99:0/3268985652 shutdown_connections 2023-10-08T00:12:52.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.278+0000 7fd285e80700 1 -- 172.21.15.99:0/3268985652 wait complete. 2023-10-08T00:12:52.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.279+0000 7fd285e80700 1 Processor -- start 2023-10-08T00:12:52.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.279+0000 7fd285e80700 1 -- start start 2023-10-08T00:12:52.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.279+0000 7fd285e80700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2800cbbe0 0x7fd28014d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:52.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.279+0000 7fd285e80700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd2800cde30 0x7fd2800b5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:52.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.279+0000 7fd285e80700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd280159c10 0x7fd2800c6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:52.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.279+0000 7fd285e80700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2800c7c40 con 0x7fd2800cbbe0 2023-10-08T00:12:52.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.279+0000 7fd285e80700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2800c7e90 con 0x7fd280159c10 2023-10-08T00:12:52.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.279+0000 7fd285e80700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2800c8170 con 0x7fd2800cde30 2023-10-08T00:12:52.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.280+0000 7fd284e7e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2800cbbe0 0x7fd28014d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:52.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.280+0000 7fd284e7e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2800cbbe0 0x7fd28014d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55712/0 (socket says 172.21.15.99:55712) 2023-10-08T00:12:52.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.280+0000 7fd284e7e700 1 -- 172.21.15.99:0/1628922480 learned_addr learned my addr 172.21.15.99:0/1628922480 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:52.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.280+0000 7fd277fff700 1 --2- 172.21.15.99:0/1628922480 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd2800cde30 0x7fd2800b5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:52.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.280+0000 7fd28567f700 1 --2- 172.21.15.99:0/1628922480 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd280159c10 0x7fd2800c6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:52.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.280+0000 7fd284e7e700 1 -- 172.21.15.99:0/1628922480 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd2800cde30 msgr2=0x7fd2800b5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:52.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.280+0000 7fd284e7e700 1 --2- 172.21.15.99:0/1628922480 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd2800cde30 0x7fd2800b5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:52.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.280+0000 7fd284e7e700 1 -- 172.21.15.99:0/1628922480 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd280159c10 msgr2=0x7fd2800c6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:52.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.280+0000 7fd284e7e700 1 --2- 172.21.15.99:0/1628922480 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd280159c10 0x7fd2800c6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:52.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.280+0000 7fd284e7e700 1 -- 172.21.15.99:0/1628922480 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd278009ae0 con 0x7fd2800cbbe0 2023-10-08T00:12:52.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.280+0000 7fd284e7e700 1 --2- 172.21.15.99:0/1628922480 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2800cbbe0 0x7fd28014d4c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fd27c00c4e0 tx=0x7fd27c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:52.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.280+0000 7fd274ff9700 1 -- 172.21.15.99:0/1628922480 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27c011590 con 0x7fd2800cbbe0 2023-10-08T00:12:52.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.280+0000 7fd274ff9700 1 -- 172.21.15.99:0/1628922480 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd27c011710 con 0x7fd2800cbbe0 2023-10-08T00:12:52.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.280+0000 7fd274ff9700 1 -- 172.21.15.99:0/1628922480 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27c011ad0 con 0x7fd2800cbbe0 2023-10-08T00:12:52.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.280+0000 7fd285e80700 1 -- 172.21.15.99:0/1628922480 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2800c8450 con 0x7fd2800cbbe0 2023-10-08T00:12:52.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.280+0000 7fd285e80700 1 -- 172.21.15.99:0/1628922480 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2800c8a10 con 0x7fd2800cbbe0 2023-10-08T00:12:52.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.281+0000 7fd285e80700 1 -- 172.21.15.99:0/1628922480 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2800ce800 con 0x7fd2800cbbe0 2023-10-08T00:12:52.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.282+0000 7fd274ff9700 1 -- 172.21.15.99:0/1628922480 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd27c01b3b0 con 0x7fd2800cbbe0 2023-10-08T00:12:52.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.282+0000 7fd274ff9700 1 --2- 172.21.15.99:0/1628922480 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd26c066f30 0x7fd26c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:52.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.282+0000 7fd274ff9700 1 -- 172.21.15.99:0/1628922480 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fd27c0835c0 con 0x7fd2800cbbe0 2023-10-08T00:12:52.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.282+0000 7fd277fff700 1 --2- 172.21.15.99:0/1628922480 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd26c066f30 0x7fd26c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:52.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.282+0000 7fd277fff700 1 --2- 172.21.15.99:0/1628922480 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd26c066f30 0x7fd26c0693e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd2801549b0 tx=0x7fd270008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:52.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.286+0000 7fd274ff9700 1 -- 172.21.15.99:0/1628922480 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd27c050b30 con 0x7fd2800cbbe0 2023-10-08T00:12:52.313 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2023-10-08T00:12:52.313 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2023-10-08T00:12:52.313 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-10-08T00:12:52.313 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2023-10-08T00:12:52.485 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:52.484+0000 7fd285e80700 1 -- 172.21.15.99:0/1628922480 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fd28004bb80 con 0x7fd2800cbbe0 2023-10-08T00:12:52.806 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:12:52.808 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:12:53.641 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.640+0000 7fd274ff9700 1 -- 172.21.15.99:0/1628922480 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v65) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd27c017020 con 0x7fd2800cbbe0 2023-10-08T00:12:53.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.642+0000 7fd285e80700 1 -- 172.21.15.99:0/1628922480 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd26c066f30 msgr2=0x7fd26c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:53.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.642+0000 7fd285e80700 1 --2- 172.21.15.99:0/1628922480 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd26c066f30 0x7fd26c0693e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd2801549b0 tx=0x7fd270008040 comp rx=0 tx=0).stop 2023-10-08T00:12:53.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.642+0000 7fd285e80700 1 -- 172.21.15.99:0/1628922480 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2800cbbe0 msgr2=0x7fd28014d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:53.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.642+0000 7fd285e80700 1 --2- 172.21.15.99:0/1628922480 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2800cbbe0 0x7fd28014d4c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fd27c00c4e0 tx=0x7fd27c00c7f0 comp rx=0 tx=0).stop 2023-10-08T00:12:53.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.642+0000 7fd285e80700 1 -- 172.21.15.99:0/1628922480 shutdown_connections 2023-10-08T00:12:53.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.642+0000 7fd285e80700 1 --2- 172.21.15.99:0/1628922480 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd26c066f30 0x7fd26c0693e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:53.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.642+0000 7fd285e80700 1 --2- 172.21.15.99:0/1628922480 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd280159c10 0x7fd2800c6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:53.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.642+0000 7fd285e80700 1 --2- 172.21.15.99:0/1628922480 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2800cbbe0 0x7fd28014d4c0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:53.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.642+0000 7fd285e80700 1 --2- 172.21.15.99:0/1628922480 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd2800cde30 0x7fd2800b5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:53.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.642+0000 7fd285e80700 1 -- 172.21.15.99:0/1628922480 >> 172.21.15.99:0/1628922480 conn(0x7fd2800d1660 msgr2=0x7fd2800b2830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:53.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.643+0000 7fd285e80700 1 -- 172.21.15.99:0/1628922480 shutdown_connections 2023-10-08T00:12:53.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.643+0000 7fd285e80700 1 -- 172.21.15.99:0/1628922480 wait complete. 2023-10-08T00:12:53.645 INFO:teuthology.orchestra.run.smithi099.stderr: removed all blocklist entries 2023-10-08T00:12:53.655 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-08T00:12:53.769 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.768+0000 7fbd990fb700 1 Processor -- start 2023-10-08T00:12:53.769 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.768+0000 7fbd990fb700 1 -- start start 2023-10-08T00:12:53.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.769+0000 7fbd990fb700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fbd94157870 0x7fbd9415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:53.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.769+0000 7fbd990fb700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbd94157c40 0x7fbd94158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:53.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.769+0000 7fbd990fb700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd9415cbb0 con 0x7fbd9415a620 2023-10-08T00:12:53.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.769+0000 7fbd990fb700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd9415bfa0 con 0x7fbd94157c40 2023-10-08T00:12:53.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.769+0000 7fbd990fb700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd9415c280 con 0x7fbd94157870 2023-10-08T00:12:53.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.769+0000 7fbd9359e700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbd94157c40 0x7fbd94158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:53.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.769+0000 7fbd9359e700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbd94157c40 0x7fbd94158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60938/0 (socket says 172.21.15.99:60938) 2023-10-08T00:12:53.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.769+0000 7fbd9359e700 1 -- 172.21.15.99:0/3852958482 learned_addr learned my addr 172.21.15.99:0/3852958482 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:53.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.769+0000 7fbd92d9d700 1 --2- 172.21.15.99:0/3852958482 >> v2:172.21.15.191:3300/0 conn(0x7fbd94157870 0x7fbd9415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:53.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.769+0000 7fbd9359e700 1 -- 172.21.15.99:0/3852958482 >> v2:172.21.15.191:3300/0 conn(0x7fbd94157870 msgr2=0x7fbd9415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:53.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.769+0000 7fbd9359e700 1 --2- 172.21.15.99:0/3852958482 >> v2:172.21.15.191:3300/0 conn(0x7fbd94157870 0x7fbd9415b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:53.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.769+0000 7fbd9359e700 1 -- 172.21.15.99:0/3852958482 >> v1:172.21.15.191:6789/0 conn(0x7fbd9415a620 legacy=0x7fbd9415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:53.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.769+0000 7fbd9359e700 1 -- 172.21.15.99:0/3852958482 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd9415c560 con 0x7fbd94157c40 2023-10-08T00:12:53.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.769+0000 7fbd92d9d700 1 --2- 172.21.15.99:0/3852958482 >> v2:172.21.15.191:3300/0 conn(0x7fbd94157870 0x7fbd9415b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:12:53.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.770+0000 7fbd9359e700 1 --2- 172.21.15.99:0/3852958482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd94157c40 0x7fbd94158110 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fbd94069d80 tx=0x7fbd8c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a03b8f1ecb7a84a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:53.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.770+0000 7fbd9259c700 1 -- 172.21.15.99:0/3852958482 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c00e070 con 0x7fbd94157c40 2023-10-08T00:12:53.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.770+0000 7fbd9259c700 1 -- 172.21.15.99:0/3852958482 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd8c00ba10 con 0x7fbd94157c40 2023-10-08T00:12:53.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.770+0000 7fbd9259c700 1 -- 172.21.15.99:0/3852958482 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c0092e0 con 0x7fbd94157c40 2023-10-08T00:12:53.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.770+0000 7fbd990fb700 1 -- 172.21.15.99:0/3852958482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd94157c40 msgr2=0x7fbd94158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:53.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.770+0000 7fbd990fb700 1 --2- 172.21.15.99:0/3852958482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd94157c40 0x7fbd94158110 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fbd94069d80 tx=0x7fbd8c009840 comp rx=0 tx=0).stop 2023-10-08T00:12:53.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.770+0000 7fbd990fb700 1 -- 172.21.15.99:0/3852958482 shutdown_connections 2023-10-08T00:12:53.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.770+0000 7fbd990fb700 1 --2- 172.21.15.99:0/3852958482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd94157c40 0x7fbd94158110 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:53.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.770+0000 7fbd990fb700 1 --2- 172.21.15.99:0/3852958482 >> v2:172.21.15.191:3300/0 conn(0x7fbd94157870 0x7fbd9415b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:53.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.770+0000 7fbd990fb700 1 -- 172.21.15.99:0/3852958482 >> 172.21.15.99:0/3852958482 conn(0x7fbd94068ce0 msgr2=0x7fbd940690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:53.774 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.770+0000 7fbd990fb700 1 -- 172.21.15.99:0/3852958482 shutdown_connections 2023-10-08T00:12:53.774 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.770+0000 7fbd990fb700 1 -- 172.21.15.99:0/3852958482 wait complete. 2023-10-08T00:12:53.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.772+0000 7fbd990fb700 1 Processor -- start 2023-10-08T00:12:53.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.772+0000 7fbd990fb700 1 -- start start 2023-10-08T00:12:53.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.772+0000 7fbd990fb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd94157870 0x7fbd9414fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:53.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.772+0000 7fbd990fb700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd94157c40 0x7fbd940b1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:53.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.772+0000 7fbd990fb700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd9415a620 0x7fbd9413f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:53.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.772+0000 7fbd990fb700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd940b2ef0 con 0x7fbd94157870 2023-10-08T00:12:53.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.772+0000 7fbd990fb700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd940b22e0 con 0x7fbd9415a620 2023-10-08T00:12:53.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.772+0000 7fbd990fb700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd940b25c0 con 0x7fbd94157c40 2023-10-08T00:12:53.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.772+0000 7fbd92d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd94157870 0x7fbd9414fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:53.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.772+0000 7fbd92d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd94157870 0x7fbd9414fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60940/0 (socket says 172.21.15.99:60940) 2023-10-08T00:12:53.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.772+0000 7fbd92d9d700 1 -- 172.21.15.99:0/3132073804 learned_addr learned my addr 172.21.15.99:0/3132073804 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:53.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.773+0000 7fbd92d9d700 1 -- 172.21.15.99:0/3132073804 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd94157c40 msgr2=0x7fbd940b1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:53.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.773+0000 7fbd92d9d700 1 --2- 172.21.15.99:0/3132073804 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd94157c40 0x7fbd940b1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:53.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.773+0000 7fbd92d9d700 1 -- 172.21.15.99:0/3132073804 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd9415a620 msgr2=0x7fbd9413f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:53.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.773+0000 7fbd92d9d700 1 --2- 172.21.15.99:0/3132073804 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd9415a620 0x7fbd9413f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:53.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.773+0000 7fbd92d9d700 1 -- 172.21.15.99:0/3132073804 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd8c009ae0 con 0x7fbd94157870 2023-10-08T00:12:53.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.773+0000 7fbd92d9d700 1 --2- 172.21.15.99:0/3132073804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd94157870 0x7fbd9414fcf0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fbd8400c830 tx=0x7fbd84008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:53.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.773+0000 7fbd8b7fe700 1 -- 172.21.15.99:0/3132073804 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd84011600 con 0x7fbd94157870 2023-10-08T00:12:53.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.773+0000 7fbd8b7fe700 1 -- 172.21.15.99:0/3132073804 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd84011760 con 0x7fbd94157870 2023-10-08T00:12:53.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.773+0000 7fbd990fb700 1 -- 172.21.15.99:0/3132073804 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd940b28a0 con 0x7fbd94157870 2023-10-08T00:12:53.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.773+0000 7fbd8b7fe700 1 -- 172.21.15.99:0/3132073804 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd84011b20 con 0x7fbd94157870 2023-10-08T00:12:53.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.773+0000 7fbd990fb700 1 -- 172.21.15.99:0/3132073804 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd940b3380 con 0x7fbd94157870 2023-10-08T00:12:53.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.774+0000 7fbd990fb700 1 -- 172.21.15.99:0/3132073804 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd9404c3c0 con 0x7fbd94157870 2023-10-08T00:12:53.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.774+0000 7fbd8b7fe700 1 -- 172.21.15.99:0/3132073804 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbd8400a3e0 con 0x7fbd94157870 2023-10-08T00:12:53.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.775+0000 7fbd8b7fe700 1 --2- 172.21.15.99:0/3132073804 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbd74066f30 0x7fbd740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:53.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.775+0000 7fbd8b7fe700 1 -- 172.21.15.99:0/3132073804 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fbd84084670 con 0x7fbd94157870 2023-10-08T00:12:53.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.775+0000 7fbd8a59c700 1 --2- 172.21.15.99:0/3132073804 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbd74066f30 0x7fbd740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:53.779 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.775+0000 7fbd8a59c700 1 --2- 172.21.15.99:0/3132073804 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbd74066f30 0x7fbd740693e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbd9414d680 tx=0x7fbd7c006c60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:53.779 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:53.779+0000 7fbd8b7fe700 1 -- 172.21.15.99:0/3132073804 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbd84051c30 con 0x7fbd94157870 2023-10-08T00:12:54.033 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.033+0000 7fbd990fb700 1 -- 172.21.15.99:0/3132073804 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fbd9415c840 con 0x7fbd94157870 2023-10-08T00:12:54.034 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.033+0000 7fbd8b7fe700 1 -- 172.21.15.99:0/3132073804 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fbd84055250 con 0x7fbd94157870 2023-10-08T00:12:54.034 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:12:54.034 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:12:54.034 INFO:teuthology.orchestra.run.smithi099.stdout: "auth_dump": [ 2023-10-08T00:12:54.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:54.035 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.a", 2023-10-08T00:12:54.035 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlFlN9AhAAKdbeH0XznSOuyUnReD6RsA==", 2023-10-08T00:12:54.035 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:12:54.035 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:12:54.035 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2023-10-08T00:12:54.035 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2023-10-08T00:12:54.035 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:12:54.035 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:54.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:54.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:54.035 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.b", 2023-10-08T00:12:54.036 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlh2G0DRAA1mYGrUrELHsS9abUxCCG2g==", 2023-10-08T00:12:54.036 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:12:54.036 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:12:54.036 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2023-10-08T00:12:54.036 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2023-10-08T00:12:54.036 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:12:54.036 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:54.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:54.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:54.036 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.c", 2023-10-08T00:12:54.037 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFl/M1DCRAANrfHJGZO1s0KOyCdUzZWIA==", 2023-10-08T00:12:54.037 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:12:54.037 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:12:54.037 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2023-10-08T00:12:54.037 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2023-10-08T00:12:54.037 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:12:54.037 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:54.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:54.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:54.037 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.d", 2023-10-08T00:12:54.037 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlr5F2FBAAYYeG6EM0/N0FiEAwSbYRyA==", 2023-10-08T00:12:54.038 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:12:54.038 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:12:54.038 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2023-10-08T00:12:54.038 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2023-10-08T00:12:54.038 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:12:54.038 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:54.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:54.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:54.038 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.0", 2023-10-08T00:12:54.038 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB08yFla7rpKxAAWTOutLcrjDeR69JbPZMoLg==", 2023-10-08T00:12:54.038 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:12:54.038 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:12:54.039 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:12:54.039 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:12:54.039 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:54.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:54.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:54.039 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.1", 2023-10-08T00:12:54.039 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB08yFlw+y2MhAAAvLDIB/tb49IlaireTqTFg==", 2023-10-08T00:12:54.039 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:12:54.039 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:12:54.039 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:12:54.040 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:12:54.040 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:54.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:54.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:54.040 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.2", 2023-10-08T00:12:54.040 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB18yFlh4wvABAALXOtJZoEfGmt/pOBxReooQ==", 2023-10-08T00:12:54.040 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:12:54.040 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:12:54.040 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:12:54.041 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:12:54.041 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:54.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:54.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:54.041 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.3", 2023-10-08T00:12:54.041 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB18yFl8cVfCRAAoGXTACMT+f5Avll5mxG9OA==", 2023-10-08T00:12:54.041 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:12:54.041 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:12:54.041 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:12:54.041 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:12:54.041 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:54.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:54.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:54.042 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.4", 2023-10-08T00:12:54.042 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB48yFlft9WGhAA8JZN4MjSX9wbXiMKJnfCYg==", 2023-10-08T00:12:54.042 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:12:54.042 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:12:54.042 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:12:54.042 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:12:54.042 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:54.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:54.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:54.042 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.5", 2023-10-08T00:12:54.043 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB48yFlUf1BIRAA2FMBt1jAcm4ia/vYBDyojw==", 2023-10-08T00:12:54.043 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:12:54.043 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:12:54.043 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:12:54.043 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:12:54.043 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:54.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:54.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:54.043 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.6", 2023-10-08T00:12:54.043 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB48yFl0SI6KhAAI6G+Sqfm7OQmGf9nK59x9Q==", 2023-10-08T00:12:54.043 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:12:54.043 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:12:54.044 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:12:54.044 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:12:54.044 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:54.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:54.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:54.044 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.7", 2023-10-08T00:12:54.044 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB48yFlo5xWMxAA4p9OSLK0JpWhEExsifX+3A==", 2023-10-08T00:12:54.044 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:12:54.044 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:12:54.044 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:12:54.044 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:12:54.044 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:54.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:54.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:54.045 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.0", 2023-10-08T00:12:54.045 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlmzhQGBAAReslU/Pvcsr0dwE6l1N3/Q==", 2023-10-08T00:12:54.045 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:12:54.045 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:12:54.045 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow r", 2023-10-08T00:12:54.045 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-08T00:12:54.045 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:54.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:54.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:54.045 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.1", 2023-10-08T00:12:54.046 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFl2NKsIBAAUg6rXynl1J7qB6BkUizmpA==", 2023-10-08T00:12:54.046 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:12:54.046 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:12:54.046 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2023-10-08T00:12:54.046 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2023-10-08T00:12:54.046 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2023-10-08T00:12:54.046 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:54.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:54.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:54.046 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.2", 2023-10-08T00:12:54.046 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlMp6rHRAArVgJfA3mY4n78hOMeVlw1Q==", 2023-10-08T00:12:54.046 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:12:54.047 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:12:54.047 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2023-10-08T00:12:54.047 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2023-10-08T00:12:54.047 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2023-10-08T00:12:54.047 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:54.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:54.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:54.047 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.3", 2023-10-08T00:12:54.047 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlH0gNJhAAEgGVGiabVIWknpWUohOnHA==", 2023-10-08T00:12:54.047 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:12:54.047 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:12:54.047 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2023-10-08T00:12:54.048 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2023-10-08T00:12:54.048 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2023-10-08T00:12:54.048 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:54.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:54.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:54.048 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.admin", 2023-10-08T00:12:54.048 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBw8yFlidUfIxAAIvpvkHbyCx5JFvHo/g5A9g==", 2023-10-08T00:12:54.048 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:12:54.048 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2023-10-08T00:12:54.048 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2023-10-08T00:12:54.048 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2023-10-08T00:12:54.048 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:12:54.049 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:54.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:54.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:54.049 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mgr.x", 2023-10-08T00:12:54.049 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBw8yFl2kMLOxAAsEyuLBLsjQCUS0ERDOi75A==", 2023-10-08T00:12:54.049 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:12:54.049 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2023-10-08T00:12:54.049 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile mgr", 2023-10-08T00:12:54.049 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:12:54.049 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:54.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:12:54.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:54.050 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mgr.y", 2023-10-08T00:12:54.050 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBw8yFl6ht5NxAAFzXnLDjklCePD6hw/pSfFg==", 2023-10-08T00:12:54.050 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:12:54.050 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2023-10-08T00:12:54.050 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile mgr", 2023-10-08T00:12:54.050 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:12:54.050 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:54.050 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:54.050 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:12:54.050 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:12:54.052 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.035+0000 7fbd990fb700 1 -- 172.21.15.99:0/3132073804 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbd74066f30 msgr2=0x7fbd740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:54.052 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.035+0000 7fbd990fb700 1 --2- 172.21.15.99:0/3132073804 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbd74066f30 0x7fbd740693e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbd9414d680 tx=0x7fbd7c006c60 comp rx=0 tx=0).stop 2023-10-08T00:12:54.052 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.035+0000 7fbd990fb700 1 -- 172.21.15.99:0/3132073804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd94157870 msgr2=0x7fbd9414fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:54.052 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.035+0000 7fbd990fb700 1 --2- 172.21.15.99:0/3132073804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd94157870 0x7fbd9414fcf0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fbd8400c830 tx=0x7fbd84008040 comp rx=0 tx=0).stop 2023-10-08T00:12:54.052 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.035+0000 7fbd990fb700 1 -- 172.21.15.99:0/3132073804 shutdown_connections 2023-10-08T00:12:54.052 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.035+0000 7fbd990fb700 1 --2- 172.21.15.99:0/3132073804 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbd74066f30 0x7fbd740693e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:54.052 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.035+0000 7fbd990fb700 1 --2- 172.21.15.99:0/3132073804 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd9415a620 0x7fbd9413f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:54.052 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.035+0000 7fbd990fb700 1 --2- 172.21.15.99:0/3132073804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd94157870 0x7fbd9414fcf0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:54.053 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.035+0000 7fbd990fb700 1 --2- 172.21.15.99:0/3132073804 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd94157c40 0x7fbd940b1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:54.053 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.035+0000 7fbd990fb700 1 -- 172.21.15.99:0/3132073804 >> 172.21.15.99:0/3132073804 conn(0x7fbd94068ce0 msgr2=0x7fbd940b5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:54.053 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.035+0000 7fbd990fb700 1 -- 172.21.15.99:0/3132073804 shutdown_connections 2023-10-08T00:12:54.053 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.035+0000 7fbd990fb700 1 -- 172.21.15.99:0/3132073804 wait complete. 2023-10-08T00:12:54.053 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:12:54.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.166+0000 7f5eeb59e700 1 Processor -- start 2023-10-08T00:12:54.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.166+0000 7f5eeb59e700 1 -- start start 2023-10-08T00:12:54.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.167+0000 7f5eeb59e700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5eec0cd6e0 0x7f5eec0cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:54.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.167+0000 7f5eeb59e700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eec0cb7c0 con 0x7f5eec15cbd0 2023-10-08T00:12:54.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.167+0000 7f5eeb59e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eec15e290 con 0x7f5eec0cdab0 2023-10-08T00:12:54.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.167+0000 7f5eeb59e700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eec15e850 con 0x7f5eec0cd6e0 2023-10-08T00:12:54.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.167+0000 7f5eea59c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5eec0cd6e0 0x7f5eec0cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:54.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.167+0000 7f5eead9d700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f5eec15cbd0 0x7f5eec15d050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:47538/0 (socket says 172.21.15.99:47538) 2023-10-08T00:12:54.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.167+0000 7f5eead9d700 1 -- 172.21.15.99:0/1961444866 learned_addr learned my addr 172.21.15.99:0/1961444866 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:12:54.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.167+0000 7f5eea59c700 1 -- 172.21.15.99:0/1961444866 >> v1:172.21.15.191:6789/0 conn(0x7f5eec0cdab0 legacy=0x7f5eec15d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:54.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.167+0000 7f5eea59c700 1 -- 172.21.15.99:0/1961444866 >> v1:172.21.15.99:6789/0 conn(0x7f5eec15cbd0 legacy=0x7f5eec15d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:54.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.167+0000 7f5eea59c700 1 -- 172.21.15.99:0/1961444866 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eec15e570 con 0x7f5eec0cd6e0 2023-10-08T00:12:54.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.167+0000 7f5eea59c700 1 --2- 172.21.15.99:0/1961444866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5eec0cd6e0 0x7f5eec0cbc50 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f5ee40098e0 tx=0x7f5ee40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1cfc15f5604d9c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:54.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.168+0000 7f5ee959a700 1 -- 172.21.15.99:0/1961444866 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee4009e80 con 0x7f5eec0cd6e0 2023-10-08T00:12:54.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.168+0000 7f5ee959a700 1 -- 172.21.15.99:0/1961444866 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee400b860 con 0x7f5eec0cd6e0 2023-10-08T00:12:54.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.168+0000 7f5ee959a700 1 -- 172.21.15.99:0/1961444866 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee400bbe0 con 0x7f5eec0cd6e0 2023-10-08T00:12:54.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.168+0000 7f5eeb59e700 1 -- 172.21.15.99:0/1961444866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5eec0cd6e0 msgr2=0x7f5eec0cbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:54.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.168+0000 7f5eeb59e700 1 --2- 172.21.15.99:0/1961444866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5eec0cd6e0 0x7f5eec0cbc50 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f5ee40098e0 tx=0x7f5ee40091a0 comp rx=0 tx=0).stop 2023-10-08T00:12:54.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.168+0000 7f5eeb59e700 1 -- 172.21.15.99:0/1961444866 shutdown_connections 2023-10-08T00:12:54.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.168+0000 7f5eeb59e700 1 --2- 172.21.15.99:0/1961444866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5eec0cd6e0 0x7f5eec0cbc50 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:54.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.168+0000 7f5eeb59e700 1 -- 172.21.15.99:0/1961444866 >> 172.21.15.99:0/1961444866 conn(0x7f5eec067660 msgr2=0x7f5eec067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:54.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.168+0000 7f5eeb59e700 1 -- 172.21.15.99:0/1961444866 shutdown_connections 2023-10-08T00:12:54.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.168+0000 7f5eeb59e700 1 -- 172.21.15.99:0/1961444866 wait complete. 2023-10-08T00:12:54.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.169+0000 7f5eeb59e700 1 Processor -- start 2023-10-08T00:12:54.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.169+0000 7f5eeb59e700 1 -- start start 2023-10-08T00:12:54.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.170+0000 7f5eeb59e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5eec0cd6e0 0x7f5eec153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:54.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.170+0000 7f5eeb59e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5eec0cdab0 0x7f5eec0c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:54.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.170+0000 7f5eeb59e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5eec15cbd0 0x7f5eec0c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:54.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.170+0000 7f5eeb59e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec16a260 con 0x7f5eec0cdab0 2023-10-08T00:12:54.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.170+0000 7f5eeb59e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec0b5880 con 0x7f5eec0cd6e0 2023-10-08T00:12:54.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.170+0000 7f5eeb59e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec154010 con 0x7f5eec15cbd0 2023-10-08T00:12:54.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.170+0000 7f5ee9d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5eec0cdab0 0x7f5eec0c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:54.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.170+0000 7f5ee9d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5eec0cdab0 0x7f5eec0c7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60946/0 (socket says 172.21.15.99:60946) 2023-10-08T00:12:54.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.170+0000 7f5ee9d9b700 1 -- 172.21.15.99:0/2140613277 learned_addr learned my addr 172.21.15.99:0/2140613277 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:54.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.170+0000 7f5eead9d700 1 --2- 172.21.15.99:0/2140613277 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5eec15cbd0 0x7f5eec0c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:54.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.170+0000 7f5eea59c700 1 --2- 172.21.15.99:0/2140613277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5eec0cd6e0 0x7f5eec153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:54.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.170+0000 7f5ee9d9b700 1 -- 172.21.15.99:0/2140613277 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5eec15cbd0 msgr2=0x7f5eec0c8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:54.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.170+0000 7f5ee9d9b700 1 --2- 172.21.15.99:0/2140613277 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5eec15cbd0 0x7f5eec0c8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:54.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.170+0000 7f5ee9d9b700 1 -- 172.21.15.99:0/2140613277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5eec0cd6e0 msgr2=0x7f5eec153ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:54.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.170+0000 7f5ee9d9b700 1 --2- 172.21.15.99:0/2140613277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5eec0cd6e0 0x7f5eec153ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:54.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.170+0000 7f5ee9d9b700 1 -- 172.21.15.99:0/2140613277 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee4009420 con 0x7f5eec0cdab0 2023-10-08T00:12:54.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.171+0000 7f5ee9d9b700 1 --2- 172.21.15.99:0/2140613277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5eec0cdab0 0x7f5eec0c7d20 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f5ed400c590 tx=0x7f5ed400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:54.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.171+0000 7f5ee27fc700 1 -- 172.21.15.99:0/2140613277 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed4011590 con 0x7f5eec0cdab0 2023-10-08T00:12:54.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.171+0000 7f5ee27fc700 1 -- 172.21.15.99:0/2140613277 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ed40116f0 con 0x7f5eec0cdab0 2023-10-08T00:12:54.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.171+0000 7f5eeb59e700 1 -- 172.21.15.99:0/2140613277 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eec0b2840 con 0x7f5eec0cdab0 2023-10-08T00:12:54.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.171+0000 7f5ee27fc700 1 -- 172.21.15.99:0/2140613277 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed4011ab0 con 0x7f5eec0cdab0 2023-10-08T00:12:54.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.171+0000 7f5eeb59e700 1 -- 172.21.15.99:0/2140613277 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eec0b4fe0 con 0x7f5eec0cdab0 2023-10-08T00:12:54.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.172+0000 7f5eeb59e700 1 -- 172.21.15.99:0/2140613277 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eec16a3a0 con 0x7f5eec0cdab0 2023-10-08T00:12:54.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.172+0000 7f5ee27fc700 1 -- 172.21.15.99:0/2140613277 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5ed401b3b0 con 0x7f5eec0cdab0 2023-10-08T00:12:54.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.173+0000 7f5ee27fc700 1 --2- 172.21.15.99:0/2140613277 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5ecc066f30 0x7f5ecc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:54.176 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.173+0000 7f5ee27fc700 1 -- 172.21.15.99:0/2140613277 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f5ed4083790 con 0x7f5eec0cdab0 2023-10-08T00:12:54.176 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.173+0000 7f5eea59c700 1 --2- 172.21.15.99:0/2140613277 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5ecc066f30 0x7f5ecc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:54.176 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.173+0000 7f5eea59c700 1 --2- 172.21.15.99:0/2140613277 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5ecc066f30 0x7f5ecc0693e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5ee40093f0 tx=0x7f5ee4017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:54.177 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.177+0000 7f5ee27fc700 1 -- 172.21.15.99:0/2140613277 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5eec16a3a0 con 0x7f5eec0cdab0 2023-10-08T00:12:54.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.361+0000 7f5eeb59e700 1 -- 172.21.15.99:0/2140613277 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5eec16a3a0 con 0x7f5eec0cdab0 2023-10-08T00:12:54.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.361+0000 7f5ee27fc700 1 -- 172.21.15.99:0/2140613277 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+12134 (secure 0 0 0) 0x7f5eec16a3a0 con 0x7f5eec0cdab0 2023-10-08T00:12:54.362 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:12:54.362 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":65,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:12:53.637408+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:12:54.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.363+0000 7f5eeb59e700 1 -- 172.21.15.99:0/2140613277 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5ecc066f30 msgr2=0x7f5ecc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:54.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.363+0000 7f5eeb59e700 1 --2- 172.21.15.99:0/2140613277 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5ecc066f30 0x7f5ecc0693e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5ee40093f0 tx=0x7f5ee4017000 comp rx=0 tx=0).stop 2023-10-08T00:12:54.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.363+0000 7f5eeb59e700 1 -- 172.21.15.99:0/2140613277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5eec0cdab0 msgr2=0x7f5eec0c7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:54.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.363+0000 7f5eeb59e700 1 --2- 172.21.15.99:0/2140613277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5eec0cdab0 0x7f5eec0c7d20 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f5ed400c590 tx=0x7f5ed400c950 comp rx=0 tx=0).stop 2023-10-08T00:12:54.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.364+0000 7f5eeb59e700 1 -- 172.21.15.99:0/2140613277 shutdown_connections 2023-10-08T00:12:54.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.364+0000 7f5eeb59e700 1 --2- 172.21.15.99:0/2140613277 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5ecc066f30 0x7f5ecc0693e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:54.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.364+0000 7f5eeb59e700 1 --2- 172.21.15.99:0/2140613277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5eec0cd6e0 0x7f5eec153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:54.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.364+0000 7f5eeb59e700 1 --2- 172.21.15.99:0/2140613277 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5eec15cbd0 0x7f5eec0c8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:54.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.364+0000 7f5eeb59e700 1 --2- 172.21.15.99:0/2140613277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5eec0cdab0 0x7f5eec0c7d20 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:54.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.364+0000 7f5eeb59e700 1 -- 172.21.15.99:0/2140613277 >> 172.21.15.99:0/2140613277 conn(0x7f5eec067660 msgr2=0x7f5eec1572c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:54.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.364+0000 7f5eeb59e700 1 -- 172.21.15.99:0/2140613277 shutdown_connections 2023-10-08T00:12:54.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.364+0000 7f5eeb59e700 1 -- 172.21.15.99:0/2140613277 wait complete. 2023-10-08T00:12:54.376 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:10:42.063344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-08T00:12:54.377 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T00:12:54.494 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.493+0000 7fe0c13e2700 1 Processor -- start 2023-10-08T00:12:54.495 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.493+0000 7fe0c13e2700 1 -- start start 2023-10-08T00:12:54.497 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.494+0000 7fe0c13e2700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fe0bc06d370 0x7fe0bc06d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:54.498 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.494+0000 7fe0c13e2700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0bc15c8d0 con 0x7fe0bc15b9a0 2023-10-08T00:12:54.498 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.494+0000 7fe0c13e2700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0bc0c9370 con 0x7fe0bc152300 2023-10-08T00:12:54.498 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.494+0000 7fe0c13e2700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0bc0b1400 con 0x7fe0bc06d370 2023-10-08T00:12:54.498 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.494+0000 7fe0ba7fc700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fe0bc15b9a0 0x7fe0bc0caaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:47540/0 (socket says 172.21.15.99:47540) 2023-10-08T00:12:54.498 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.494+0000 7fe0ba7fc700 1 -- 172.21.15.99:0/3016936216 learned_addr learned my addr 172.21.15.99:0/3016936216 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:12:54.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.494+0000 7fe0bb7fe700 1 --2- 172.21.15.99:0/3016936216 >> v2:172.21.15.191:3300/0 conn(0x7fe0bc06d370 0x7fe0bc06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:54.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.495+0000 7fe0b9ffb700 1 -- 172.21.15.99:0/3016936216 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3843171544 0 0) 0x7fe0bc15c8d0 con 0x7fe0bc15b9a0 2023-10-08T00:12:54.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.495+0000 7fe0b9ffb700 1 -- 172.21.15.99:0/3016936216 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0a8003570 con 0x7fe0bc15b9a0 2023-10-08T00:12:54.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.495+0000 7fe0bb7fe700 1 -- 172.21.15.99:0/3016936216 >> v1:172.21.15.191:6789/0 conn(0x7fe0bc152300 legacy=0x7fe0bc158df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:54.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.495+0000 7fe0b9ffb700 1 -- 172.21.15.99:0/3016936216 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2862293558 0 0) 0x7fe0bc0c9370 con 0x7fe0bc152300 2023-10-08T00:12:54.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.495+0000 7fe0bb7fe700 1 -- 172.21.15.99:0/3016936216 >> v1:172.21.15.99:6789/0 conn(0x7fe0bc15b9a0 legacy=0x7fe0bc0caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:54.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.495+0000 7fe0bb7fe700 1 -- 172.21.15.99:0/3016936216 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0bc15c030 con 0x7fe0bc06d370 2023-10-08T00:12:54.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.495+0000 7fe0bb7fe700 1 --2- 172.21.15.99:0/3016936216 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0bc06d370 0x7fe0bc06d790 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe0b40098e0 tx=0x7fe0b40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73f97fc2b2817693 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:54.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.495+0000 7fe0b9ffb700 1 -- 172.21.15.99:0/3016936216 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b400b640 con 0x7fe0bc06d370 2023-10-08T00:12:54.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.495+0000 7fe0b9ffb700 1 -- 172.21.15.99:0/3016936216 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b400b7a0 con 0x7fe0bc06d370 2023-10-08T00:12:54.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.495+0000 7fe0b9ffb700 1 -- 172.21.15.99:0/3016936216 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b400bb40 con 0x7fe0bc06d370 2023-10-08T00:12:54.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.496+0000 7fe0c13e2700 1 -- 172.21.15.99:0/3016936216 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0bc06d370 msgr2=0x7fe0bc06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:54.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.496+0000 7fe0c13e2700 1 --2- 172.21.15.99:0/3016936216 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0bc06d370 0x7fe0bc06d790 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe0b40098e0 tx=0x7fe0b40091a0 comp rx=0 tx=0).stop 2023-10-08T00:12:54.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.496+0000 7fe0c13e2700 1 -- 172.21.15.99:0/3016936216 shutdown_connections 2023-10-08T00:12:54.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.496+0000 7fe0c13e2700 1 --2- 172.21.15.99:0/3016936216 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0bc06d370 0x7fe0bc06d790 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:54.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.496+0000 7fe0c13e2700 1 -- 172.21.15.99:0/3016936216 >> 172.21.15.99:0/3016936216 conn(0x7fe0bc069870 msgr2=0x7fe0bc069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:54.502 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.496+0000 7fe0c13e2700 1 -- 172.21.15.99:0/3016936216 shutdown_connections 2023-10-08T00:12:54.502 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.496+0000 7fe0c13e2700 1 -- 172.21.15.99:0/3016936216 wait complete. 2023-10-08T00:12:54.502 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.497+0000 7fe0c13e2700 1 Processor -- start 2023-10-08T00:12:54.502 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.497+0000 7fe0c13e2700 1 -- start start 2023-10-08T00:12:54.505 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.498+0000 7fe0c13e2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0bc06d370 0x7fe0bc151660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:54.505 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.498+0000 7fe0c13e2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0bc152300 0x7fe0bc1541c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:54.505 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.498+0000 7fe0c13e2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe0bc15b9a0 0x7fe0bc154700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:54.506 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.498+0000 7fe0c13e2700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc0c6ff0 con 0x7fe0bc06d370 2023-10-08T00:12:54.506 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.498+0000 7fe0c13e2700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc0c84d0 con 0x7fe0bc152300 2023-10-08T00:12:54.506 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.498+0000 7fe0c13e2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc0c66c0 con 0x7fe0bc15b9a0 2023-10-08T00:12:54.506 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.498+0000 7fe0baffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0bc06d370 0x7fe0bc151660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:54.506 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.498+0000 7fe0baffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0bc06d370 0x7fe0bc151660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60960/0 (socket says 172.21.15.99:60960) 2023-10-08T00:12:54.507 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.498+0000 7fe0baffd700 1 -- 172.21.15.99:0/2621832329 learned_addr learned my addr 172.21.15.99:0/2621832329 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:54.507 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.498+0000 7fe0baffd700 1 -- 172.21.15.99:0/2621832329 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe0bc15b9a0 msgr2=0x7fe0bc154700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:54.507 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.498+0000 7fe0baffd700 1 --2- 172.21.15.99:0/2621832329 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe0bc15b9a0 0x7fe0bc154700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:54.507 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.498+0000 7fe0baffd700 1 -- 172.21.15.99:0/2621832329 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0bc152300 msgr2=0x7fe0bc1541c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:54.507 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.498+0000 7fe0ba7fc700 1 --2- 172.21.15.99:0/2621832329 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0bc152300 0x7fe0bc1541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:54.508 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.498+0000 7fe0baffd700 1 --2- 172.21.15.99:0/2621832329 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0bc152300 0x7fe0bc1541c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:54.508 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.498+0000 7fe0baffd700 1 -- 172.21.15.99:0/2621832329 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b4009420 con 0x7fe0bc06d370 2023-10-08T00:12:54.508 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.498+0000 7fe0ba7fc700 1 --2- 172.21.15.99:0/2621832329 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0bc152300 0x7fe0bc1541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:12:54.508 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.498+0000 7fe0baffd700 1 --2- 172.21.15.99:0/2621832329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0bc06d370 0x7fe0bc151660 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac00cb60 tx=0x7fe0ac0017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:54.508 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.498+0000 7fe0b2ffd700 1 -- 172.21.15.99:0/2621832329 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac00a7b0 con 0x7fe0bc06d370 2023-10-08T00:12:54.508 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.498+0000 7fe0b2ffd700 1 -- 172.21.15.99:0/2621832329 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0ac00a930 con 0x7fe0bc06d370 2023-10-08T00:12:54.509 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.498+0000 7fe0c13e2700 1 -- 172.21.15.99:0/2621832329 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0bc0c69a0 con 0x7fe0bc06d370 2023-10-08T00:12:54.509 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.498+0000 7fe0b2ffd700 1 -- 172.21.15.99:0/2621832329 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac00acf0 con 0x7fe0bc06d370 2023-10-08T00:12:54.509 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.498+0000 7fe0c13e2700 1 -- 172.21.15.99:0/2621832329 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0bc151ba0 con 0x7fe0bc06d370 2023-10-08T00:12:54.509 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.499+0000 7fe0c13e2700 1 -- 172.21.15.99:0/2621832329 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0bc04c3c0 con 0x7fe0bc06d370 2023-10-08T00:12:54.509 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.500+0000 7fe0b2ffd700 1 -- 172.21.15.99:0/2621832329 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe0ac003030 con 0x7fe0bc06d370 2023-10-08T00:12:54.509 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.500+0000 7fe0b2ffd700 1 --2- 172.21.15.99:0/2621832329 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe09c066f30 0x7fe09c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:54.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.500+0000 7fe0b2ffd700 1 -- 172.21.15.99:0/2621832329 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fe0ac084380 con 0x7fe0bc06d370 2023-10-08T00:12:54.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.500+0000 7fe0ba7fc700 1 --2- 172.21.15.99:0/2621832329 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe09c066f30 0x7fe09c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:54.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.501+0000 7fe0ba7fc700 1 --2- 172.21.15.99:0/2621832329 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe09c066f30 0x7fe09c0693e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe0a40058d0 tx=0x7fe0a4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:54.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.504+0000 7fe0b2ffd700 1 -- 172.21.15.99:0/2621832329 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe0ac051940 con 0x7fe0bc06d370 2023-10-08T00:12:54.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.685+0000 7fe0c13e2700 1 -- 172.21.15.99:0/2621832329 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe0bc0b0ae0 con 0x7fe0bc06d370 2023-10-08T00:12:54.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.685+0000 7fe0b2ffd700 1 -- 172.21.15.99:0/2621832329 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe0ac054f60 con 0x7fe0bc06d370 2023-10-08T00:12:54.687 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2023-10-08T00:12:54.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.687+0000 7fe0c13e2700 1 -- 172.21.15.99:0/2621832329 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe09c066f30 msgr2=0x7fe09c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:54.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.687+0000 7fe0c13e2700 1 --2- 172.21.15.99:0/2621832329 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe09c066f30 0x7fe09c0693e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe0a40058d0 tx=0x7fe0a4009040 comp rx=0 tx=0).stop 2023-10-08T00:12:54.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.687+0000 7fe0c13e2700 1 -- 172.21.15.99:0/2621832329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0bc06d370 msgr2=0x7fe0bc151660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:54.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.687+0000 7fe0c13e2700 1 --2- 172.21.15.99:0/2621832329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0bc06d370 0x7fe0bc151660 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac00cb60 tx=0x7fe0ac0017a0 comp rx=0 tx=0).stop 2023-10-08T00:12:54.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.687+0000 7fe0c13e2700 1 -- 172.21.15.99:0/2621832329 shutdown_connections 2023-10-08T00:12:54.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.687+0000 7fe0c13e2700 1 --2- 172.21.15.99:0/2621832329 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe09c066f30 0x7fe09c0693e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:54.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.687+0000 7fe0c13e2700 1 --2- 172.21.15.99:0/2621832329 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe0bc15b9a0 0x7fe0bc154700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:54.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.687+0000 7fe0c13e2700 1 --2- 172.21.15.99:0/2621832329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0bc06d370 0x7fe0bc151660 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:54.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.687+0000 7fe0c13e2700 1 --2- 172.21.15.99:0/2621832329 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0bc152300 0x7fe0bc1541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:54.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.688+0000 7fe0c13e2700 1 -- 172.21.15.99:0/2621832329 >> 172.21.15.99:0/2621832329 conn(0x7fe0bc069870 msgr2=0x7fe0bc0cb290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:54.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.688+0000 7fe0c13e2700 1 -- 172.21.15.99:0/2621832329 shutdown_connections 2023-10-08T00:12:54.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.688+0000 7fe0c13e2700 1 -- 172.21.15.99:0/2621832329 wait complete. 2023-10-08T00:12:54.700 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T00:12:54.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.814+0000 7fb594864700 1 Processor -- start 2023-10-08T00:12:54.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.814+0000 7fb594864700 1 -- start start 2023-10-08T00:12:54.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.815+0000 7fb594864700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb5900cd6e0 0x7fb5900cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:54.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.815+0000 7fb594864700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb59015cbd0 0x7fb59015d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:54.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.815+0000 7fb594864700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5900cb7c0 con 0x7fb5900cdab0 2023-10-08T00:12:54.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.815+0000 7fb594864700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb59015e850 con 0x7fb59015cbd0 2023-10-08T00:12:54.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.815+0000 7fb594864700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5900cd030 con 0x7fb5900cd6e0 2023-10-08T00:12:54.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.815+0000 7fb58ffff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb59015cbd0 0x7fb59015d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:54.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.815+0000 7fb58effd700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fb5900cdab0 0x7fb59015d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:47552/0 (socket says 172.21.15.99:47552) 2023-10-08T00:12:54.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.815+0000 7fb58effd700 1 -- 172.21.15.99:0/2992872604 learned_addr learned my addr 172.21.15.99:0/2992872604 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:12:54.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.815+0000 7fb58f7fe700 1 --2- 172.21.15.99:0/2992872604 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb5900cd6e0 0x7fb5900cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:54.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.816+0000 7fb58ffff700 1 -- 172.21.15.99:0/2992872604 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb5900cd6e0 msgr2=0x7fb5900cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:54.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.816+0000 7fb58ffff700 1 --2- 172.21.15.99:0/2992872604 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb5900cd6e0 0x7fb5900cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:54.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.816+0000 7fb58ffff700 1 -- 172.21.15.99:0/2992872604 >> v1:172.21.15.99:6789/0 conn(0x7fb5900cdab0 legacy=0x7fb59015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:54.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.816+0000 7fb58ffff700 1 -- 172.21.15.99:0/2992872604 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb59015e290 con 0x7fb59015cbd0 2023-10-08T00:12:54.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.816+0000 7fb58e7fc700 1 -- 172.21.15.99:0/2992872604 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 703551758 0 0) 0x7fb5900cb7c0 con 0x7fb5900cdab0 2023-10-08T00:12:54.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.816+0000 7fb58ffff700 1 --2- 172.21.15.99:0/2992872604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb59015cbd0 0x7fb59015d020 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb588009e60 tx=0x7fb588009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfad66b043af6609 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:54.819 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.816+0000 7fb58e7fc700 1 -- 172.21.15.99:0/2992872604 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58800e070 con 0x7fb59015cbd0 2023-10-08T00:12:54.819 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.816+0000 7fb58e7fc700 1 -- 172.21.15.99:0/2992872604 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb58800bc30 con 0x7fb59015cbd0 2023-10-08T00:12:54.819 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.816+0000 7fb58e7fc700 1 -- 172.21.15.99:0/2992872604 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb588012430 con 0x7fb59015cbd0 2023-10-08T00:12:54.819 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.816+0000 7fb594864700 1 -- 172.21.15.99:0/2992872604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb59015cbd0 msgr2=0x7fb59015d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:54.819 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.816+0000 7fb594864700 1 --2- 172.21.15.99:0/2992872604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb59015cbd0 0x7fb59015d020 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb588009e60 tx=0x7fb588009780 comp rx=0 tx=0).stop 2023-10-08T00:12:54.819 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.816+0000 7fb594864700 1 -- 172.21.15.99:0/2992872604 shutdown_connections 2023-10-08T00:12:54.819 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.816+0000 7fb594864700 1 --2- 172.21.15.99:0/2992872604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb59015cbd0 0x7fb59015d020 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:54.819 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.816+0000 7fb594864700 1 --2- 172.21.15.99:0/2992872604 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb5900cd6e0 0x7fb5900cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:54.819 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.816+0000 7fb594864700 1 -- 172.21.15.99:0/2992872604 >> 172.21.15.99:0/2992872604 conn(0x7fb590067660 msgr2=0x7fb590067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:54.819 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.816+0000 7fb594864700 1 -- 172.21.15.99:0/2992872604 shutdown_connections 2023-10-08T00:12:54.819 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.816+0000 7fb594864700 1 -- 172.21.15.99:0/2992872604 wait complete. 2023-10-08T00:12:54.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.818+0000 7fb594864700 1 Processor -- start 2023-10-08T00:12:54.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.818+0000 7fb594864700 1 -- start start 2023-10-08T00:12:54.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.818+0000 7fb594864700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5900cd6e0 0x7fb590150840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:54.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.818+0000 7fb594864700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb5900cdab0 0x7fb59014eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:54.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.818+0000 7fb594864700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb59015cbd0 0x7fb5900c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:54.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.819+0000 7fb594864700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb590150d80 con 0x7fb5900cd6e0 2023-10-08T00:12:54.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.819+0000 7fb594864700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5900b4200 con 0x7fb59015cbd0 2023-10-08T00:12:54.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.819+0000 7fb594864700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5900b6630 con 0x7fb5900cdab0 2023-10-08T00:12:54.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.819+0000 7fb58f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5900cd6e0 0x7fb590150840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:54.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.819+0000 7fb58f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5900cd6e0 0x7fb590150840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60972/0 (socket says 172.21.15.99:60972) 2023-10-08T00:12:54.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.819+0000 7fb58f7fe700 1 -- 172.21.15.99:0/361070516 learned_addr learned my addr 172.21.15.99:0/361070516 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:54.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.819+0000 7fb58effd700 1 --2- 172.21.15.99:0/361070516 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb5900cdab0 0x7fb59014eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:54.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.819+0000 7fb58f7fe700 1 -- 172.21.15.99:0/361070516 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb5900cdab0 msgr2=0x7fb59014eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:54.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.819+0000 7fb58f7fe700 1 --2- 172.21.15.99:0/361070516 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb5900cdab0 0x7fb59014eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:54.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.819+0000 7fb58f7fe700 1 -- 172.21.15.99:0/361070516 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb59015cbd0 msgr2=0x7fb5900c7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:54.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.819+0000 7fb58f7fe700 1 --2- 172.21.15.99:0/361070516 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb59015cbd0 0x7fb5900c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:54.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.819+0000 7fb58f7fe700 1 -- 172.21.15.99:0/361070516 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb588009a20 con 0x7fb5900cd6e0 2023-10-08T00:12:54.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.819+0000 7fb58f7fe700 1 --2- 172.21.15.99:0/361070516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5900cd6e0 0x7fb590150840 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb58000d860 tx=0x7fb580008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:54.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.819+0000 7fb5777fe700 1 -- 172.21.15.99:0/361070516 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb580009610 con 0x7fb5900cd6e0 2023-10-08T00:12:54.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.819+0000 7fb5777fe700 1 -- 172.21.15.99:0/361070516 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb580009770 con 0x7fb5900cd6e0 2023-10-08T00:12:54.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.819+0000 7fb5777fe700 1 -- 172.21.15.99:0/361070516 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb580009b30 con 0x7fb5900cd6e0 2023-10-08T00:12:54.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.819+0000 7fb594864700 1 -- 172.21.15.99:0/361070516 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5900b3bb0 con 0x7fb5900cd6e0 2023-10-08T00:12:54.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.819+0000 7fb594864700 1 -- 172.21.15.99:0/361070516 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb59014f050 con 0x7fb5900cd6e0 2023-10-08T00:12:54.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.820+0000 7fb594864700 1 -- 172.21.15.99:0/361070516 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5900b3a10 con 0x7fb5900cd6e0 2023-10-08T00:12:54.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.821+0000 7fb5777fe700 1 -- 172.21.15.99:0/361070516 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb58001e460 con 0x7fb5900cd6e0 2023-10-08T00:12:54.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.821+0000 7fb5777fe700 1 --2- 172.21.15.99:0/361070516 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb57c066f30 0x7fb57c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:54.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.821+0000 7fb5777fe700 1 -- 172.21.15.99:0/361070516 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fb580084850 con 0x7fb5900cd6e0 2023-10-08T00:12:54.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.821+0000 7fb58effd700 1 --2- 172.21.15.99:0/361070516 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb57c066f30 0x7fb57c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:54.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.821+0000 7fb58effd700 1 --2- 172.21.15.99:0/361070516 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb57c066f30 0x7fb57c0693e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb584007fd0 tx=0x7fb584009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:54.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:54.825+0000 7fb5777fe700 1 -- 172.21.15.99:0/361070516 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb5900b3a10 con 0x7fb5900cd6e0 2023-10-08T00:12:55.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.007+0000 7fb594864700 1 -- 172.21.15.99:0/361070516 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb5900b4050 con 0x7fb5900cd6e0 2023-10-08T00:12:55.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.008+0000 7fb5777fe700 1 -- 172.21.15.99:0/361070516 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v65) v1 ==== 84+0+65 (secure 0 0 0) 0x7fb580051e10 con 0x7fb5900cd6e0 2023-10-08T00:12:55.009 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:12:55.009 INFO:teuthology.orchestra.run.smithi099.stdout:[ 2023-10-08T00:12:55.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:12:55.009 INFO:teuthology.orchestra.run.smithi099.stdout: "poolnum": 1, 2023-10-08T00:12:55.009 INFO:teuthology.orchestra.run.smithi099.stdout: "poolname": ".mgr" 2023-10-08T00:12:55.009 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:12:55.009 INFO:teuthology.orchestra.run.smithi099.stdout:] 2023-10-08T00:12:55.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.010+0000 7fb594864700 1 -- 172.21.15.99:0/361070516 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb57c066f30 msgr2=0x7fb57c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:55.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.010+0000 7fb594864700 1 --2- 172.21.15.99:0/361070516 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb57c066f30 0x7fb57c0693e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb584007fd0 tx=0x7fb584009040 comp rx=0 tx=0).stop 2023-10-08T00:12:55.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.010+0000 7fb594864700 1 -- 172.21.15.99:0/361070516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5900cd6e0 msgr2=0x7fb590150840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:55.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.010+0000 7fb594864700 1 --2- 172.21.15.99:0/361070516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5900cd6e0 0x7fb590150840 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb58000d860 tx=0x7fb580008040 comp rx=0 tx=0).stop 2023-10-08T00:12:55.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.010+0000 7fb594864700 1 -- 172.21.15.99:0/361070516 shutdown_connections 2023-10-08T00:12:55.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.010+0000 7fb594864700 1 --2- 172.21.15.99:0/361070516 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb57c066f30 0x7fb57c0693e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:55.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.010+0000 7fb594864700 1 --2- 172.21.15.99:0/361070516 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb59015cbd0 0x7fb5900c7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:55.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.010+0000 7fb594864700 1 --2- 172.21.15.99:0/361070516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5900cd6e0 0x7fb590150840 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:55.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.010+0000 7fb594864700 1 --2- 172.21.15.99:0/361070516 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb5900cdab0 0x7fb59014eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:55.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.010+0000 7fb594864700 1 -- 172.21.15.99:0/361070516 >> 172.21.15.99:0/361070516 conn(0x7fb590067660 msgr2=0x7fb590157780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:55.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.010+0000 7fb594864700 1 -- 172.21.15.99:0/361070516 shutdown_connections 2023-10-08T00:12:55.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.010+0000 7fb594864700 1 -- 172.21.15.99:0/361070516 wait complete. 2023-10-08T00:12:55.022 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T00:12:55.023 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T00:12:55.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.140+0000 7f7b22696700 1 Processor -- start 2023-10-08T00:12:55.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.140+0000 7f7b22696700 1 -- start start 2023-10-08T00:12:55.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.141+0000 7f7b22696700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7b1c15d240 0x7f7b1c0b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:55.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.141+0000 7f7b22696700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b1c0ce810 0x7f7b1c0cec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:55.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.141+0000 7f7b22696700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b1c0cd4f0 con 0x7f7b1c0cc940 2023-10-08T00:12:55.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.141+0000 7f7b22696700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b1c0b3f40 con 0x7f7b1c15d240 2023-10-08T00:12:55.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.141+0000 7f7b22696700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1c0cbb80 con 0x7f7b1c0ce810 2023-10-08T00:12:55.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.141+0000 7f7b21694700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7b1c15d240 0x7f7b1c0b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:55.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.141+0000 7f7b21694700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7b1c15d240 0x7f7b1c0b3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60978/0 (socket says 172.21.15.99:60978) 2023-10-08T00:12:55.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.141+0000 7f7b21694700 1 -- 172.21.15.99:0/4148906622 learned_addr learned my addr 172.21.15.99:0/4148906622 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:55.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.141+0000 7f7b21e95700 1 --2- 172.21.15.99:0/4148906622 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b1c0ce810 0x7f7b1c0cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:55.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.141+0000 7f7b21694700 1 -- 172.21.15.99:0/4148906622 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b1c0ce810 msgr2=0x7f7b1c0cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:55.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.141+0000 7f7b21694700 1 --2- 172.21.15.99:0/4148906622 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b1c0ce810 0x7f7b1c0cec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:55.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.141+0000 7f7b21694700 1 -- 172.21.15.99:0/4148906622 >> v1:172.21.15.191:6789/0 conn(0x7f7b1c0cc940 legacy=0x7f7b1c0ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:55.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.141+0000 7f7b21694700 1 -- 172.21.15.99:0/4148906622 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b1c0cbe60 con 0x7f7b1c15d240 2023-10-08T00:12:55.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.141+0000 7f7b21e95700 1 --2- 172.21.15.99:0/4148906622 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b1c0ce810 0x7f7b1c0cec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:12:55.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.142+0000 7f7b21694700 1 --2- 172.21.15.99:0/4148906622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b1c15d240 0x7f7b1c0b3a00 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f7b18009f70 tx=0x7f7b18009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f302d5aa322b8c8b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:55.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.142+0000 7f7b13fff700 1 -- 172.21.15.99:0/4148906622 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1800e070 con 0x7f7b1c15d240 2023-10-08T00:12:55.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.142+0000 7f7b13fff700 1 -- 172.21.15.99:0/4148906622 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b1800ba90 con 0x7f7b1c15d240 2023-10-08T00:12:55.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.142+0000 7f7b13fff700 1 -- 172.21.15.99:0/4148906622 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b180092e0 con 0x7f7b1c15d240 2023-10-08T00:12:55.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.142+0000 7f7b22696700 1 -- 172.21.15.99:0/4148906622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b1c15d240 msgr2=0x7f7b1c0b3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:55.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.142+0000 7f7b22696700 1 --2- 172.21.15.99:0/4148906622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b1c15d240 0x7f7b1c0b3a00 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f7b18009f70 tx=0x7f7b18009840 comp rx=0 tx=0).stop 2023-10-08T00:12:55.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.142+0000 7f7b22696700 1 -- 172.21.15.99:0/4148906622 shutdown_connections 2023-10-08T00:12:55.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.142+0000 7f7b22696700 1 --2- 172.21.15.99:0/4148906622 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b1c0ce810 0x7f7b1c0cec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:55.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.142+0000 7f7b22696700 1 --2- 172.21.15.99:0/4148906622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b1c15d240 0x7f7b1c0b3a00 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:55.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.142+0000 7f7b22696700 1 -- 172.21.15.99:0/4148906622 >> 172.21.15.99:0/4148906622 conn(0x7f7b1c067e40 msgr2=0x7f7b1c0d0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:55.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.142+0000 7f7b22696700 1 -- 172.21.15.99:0/4148906622 shutdown_connections 2023-10-08T00:12:55.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.142+0000 7f7b22696700 1 -- 172.21.15.99:0/4148906622 wait complete. 2023-10-08T00:12:55.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.143+0000 7f7b22696700 1 Processor -- start 2023-10-08T00:12:55.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.143+0000 7f7b22696700 1 -- start start 2023-10-08T00:12:55.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.144+0000 7f7b22696700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b1c0cc940 0x7f7b1c150600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:55.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.144+0000 7f7b22696700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b1c0ce810 0x7f7b1c157a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:55.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.144+0000 7f7b22696700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b1c15d240 0x7f7b1c155750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:55.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.144+0000 7f7b22696700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1c150b40 con 0x7f7b1c0cc940 2023-10-08T00:12:55.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.144+0000 7f7b22696700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1c06a1c0 con 0x7f7b1c0ce810 2023-10-08T00:12:55.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.144+0000 7f7b22696700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1c06a4a0 con 0x7f7b1c15d240 2023-10-08T00:12:55.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.144+0000 7f7b21694700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b1c0cc940 0x7f7b1c150600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:55.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.144+0000 7f7b21694700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b1c0cc940 0x7f7b1c150600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60992/0 (socket says 172.21.15.99:60992) 2023-10-08T00:12:55.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.144+0000 7f7b21694700 1 -- 172.21.15.99:0/3878684179 learned_addr learned my addr 172.21.15.99:0/3878684179 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:55.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.144+0000 7f7b20e93700 1 --2- 172.21.15.99:0/3878684179 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b1c0ce810 0x7f7b1c157a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:55.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.144+0000 7f7b21e95700 1 --2- 172.21.15.99:0/3878684179 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b1c15d240 0x7f7b1c155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:55.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.145+0000 7f7b21694700 1 -- 172.21.15.99:0/3878684179 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b1c15d240 msgr2=0x7f7b1c155750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:55.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.145+0000 7f7b21694700 1 --2- 172.21.15.99:0/3878684179 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b1c15d240 0x7f7b1c155750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:55.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.145+0000 7f7b21694700 1 -- 172.21.15.99:0/3878684179 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b1c0ce810 msgr2=0x7f7b1c157a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:55.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.145+0000 7f7b21694700 1 --2- 172.21.15.99:0/3878684179 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b1c0ce810 0x7f7b1c157a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:55.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.145+0000 7f7b21694700 1 -- 172.21.15.99:0/3878684179 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b18009ae0 con 0x7f7b1c0cc940 2023-10-08T00:12:55.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.145+0000 7f7b21694700 1 --2- 172.21.15.99:0/3878684179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b1c0cc940 0x7f7b1c150600 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c0688d0 tx=0x7f7b18017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:55.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.145+0000 7f7b117fa700 1 -- 172.21.15.99:0/3878684179 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b18012a20 con 0x7f7b1c0cc940 2023-10-08T00:12:55.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.145+0000 7f7b117fa700 1 -- 172.21.15.99:0/3878684179 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b18012ba0 con 0x7f7b1c0cc940 2023-10-08T00:12:55.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.145+0000 7f7b22696700 1 -- 172.21.15.99:0/3878684179 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b1c0c8880 con 0x7f7b1c0cc940 2023-10-08T00:12:55.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.145+0000 7f7b117fa700 1 -- 172.21.15.99:0/3878684179 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1801d420 con 0x7f7b1c0cc940 2023-10-08T00:12:55.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.145+0000 7f7b22696700 1 -- 172.21.15.99:0/3878684179 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b1c157fd0 con 0x7f7b1c0cc940 2023-10-08T00:12:55.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.146+0000 7f7b22696700 1 -- 172.21.15.99:0/3878684179 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b1c0c8da0 con 0x7f7b1c0cc940 2023-10-08T00:12:55.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.146+0000 7f7b117fa700 1 -- 172.21.15.99:0/3878684179 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7b1801d850 con 0x7f7b1c0cc940 2023-10-08T00:12:55.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.147+0000 7f7b117fa700 1 --2- 172.21.15.99:0/3878684179 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7b00066f30 0x7f7b000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:55.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.147+0000 7f7b117fa700 1 -- 172.21.15.99:0/3878684179 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f7b180857f0 con 0x7f7b1c0cc940 2023-10-08T00:12:55.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.147+0000 7f7b20e93700 1 --2- 172.21.15.99:0/3878684179 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7b00066f30 0x7f7b000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:55.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.147+0000 7f7b20e93700 1 --2- 172.21.15.99:0/3878684179 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7b00066f30 0x7f7b000693e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c0cdfe0 tx=0x7f7b0c009060 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:55.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.151+0000 7f7b117fa700 1 -- 172.21.15.99:0/3878684179 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7b18052db0 con 0x7f7b1c0cc940 2023-10-08T00:12:55.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:55.336+0000 7f7b22696700 1 -- 172.21.15.99:0/3878684179 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f7b1c06ac60 con 0x7f7b1c0cc940 2023-10-08T00:12:56.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.198+0000 7f7b117fa700 1 -- 172.21.15.99:0/3878684179 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v66) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7b1802a0b0 con 0x7f7b1c0cc940 2023-10-08T00:12:56.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.201+0000 7f7b22696700 1 -- 172.21.15.99:0/3878684179 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7b00066f30 msgr2=0x7f7b000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:56.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.201+0000 7f7b22696700 1 --2- 172.21.15.99:0/3878684179 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7b00066f30 0x7f7b000693e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c0cdfe0 tx=0x7f7b0c009060 comp rx=0 tx=0).stop 2023-10-08T00:12:56.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.201+0000 7f7b22696700 1 -- 172.21.15.99:0/3878684179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b1c0cc940 msgr2=0x7f7b1c150600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:56.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.201+0000 7f7b22696700 1 --2- 172.21.15.99:0/3878684179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b1c0cc940 0x7f7b1c150600 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c0688d0 tx=0x7f7b18017000 comp rx=0 tx=0).stop 2023-10-08T00:12:56.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.201+0000 7f7b22696700 1 -- 172.21.15.99:0/3878684179 shutdown_connections 2023-10-08T00:12:56.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.201+0000 7f7b22696700 1 --2- 172.21.15.99:0/3878684179 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7b00066f30 0x7f7b000693e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:56.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.201+0000 7f7b22696700 1 --2- 172.21.15.99:0/3878684179 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b1c15d240 0x7f7b1c155750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:56.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.201+0000 7f7b22696700 1 --2- 172.21.15.99:0/3878684179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b1c0cc940 0x7f7b1c150600 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:56.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.201+0000 7f7b22696700 1 --2- 172.21.15.99:0/3878684179 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b1c0ce810 0x7f7b1c157a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:56.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.201+0000 7f7b22696700 1 -- 172.21.15.99:0/3878684179 >> 172.21.15.99:0/3878684179 conn(0x7f7b1c067e40 msgr2=0x7f7b1c068c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:56.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.201+0000 7f7b22696700 1 -- 172.21.15.99:0/3878684179 shutdown_connections 2023-10-08T00:12:56.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.201+0000 7f7b22696700 1 -- 172.21.15.99:0/3878684179 wait complete. 2023-10-08T00:12:56.204 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' created 2023-10-08T00:12:56.215 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T00:12:56.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.337+0000 7f0e7baa7700 1 Processor -- start 2023-10-08T00:12:56.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.338+0000 7f0e7baa7700 1 -- start start 2023-10-08T00:12:56.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.338+0000 7f0e7baa7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e7415a290 0x7f0e7406a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:56.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.338+0000 7f0e7baa7700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0e7406aae0 0x7f0e740cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:56.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.338+0000 7f0e7baa7700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e740b0790 con 0x7f0e74159ec0 2023-10-08T00:12:56.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.338+0000 7f0e7baa7700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e7406af60 con 0x7f0e7406aae0 2023-10-08T00:12:56.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.338+0000 7f0e7baa7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e740ce1f0 con 0x7f0e7415a290 2023-10-08T00:12:56.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.338+0000 7f0e7a044700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0e7406aae0 0x7f0e740cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:56.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.339+0000 7f0e7a044700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0e7406aae0 0x7f0e740cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:32774/0 (socket says 172.21.15.99:32774) 2023-10-08T00:12:56.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.339+0000 7f0e7a044700 1 -- 172.21.15.99:0/2091926082 learned_addr learned my addr 172.21.15.99:0/2091926082 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:56.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.339+0000 7f0e79042700 1 --2- 172.21.15.99:0/2091926082 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e7415a290 0x7f0e7406a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:56.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.339+0000 7f0e7a044700 1 -- 172.21.15.99:0/2091926082 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e7415a290 msgr2=0x7f0e7406a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:56.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.339+0000 7f0e7a044700 1 --2- 172.21.15.99:0/2091926082 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e7415a290 0x7f0e7406a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:56.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.339+0000 7f0e7a044700 1 -- 172.21.15.99:0/2091926082 >> v1:172.21.15.191:6789/0 conn(0x7f0e74159ec0 legacy=0x7f0e740af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:56.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.339+0000 7f0e7a044700 1 -- 172.21.15.99:0/2091926082 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e740b0b20 con 0x7f0e7406aae0 2023-10-08T00:12:56.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.339+0000 7f0e7a044700 1 --2- 172.21.15.99:0/2091926082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e7406aae0 0x7f0e740cf590 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c009f20 tx=0x7f0e6c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c786d8662dba1ca3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:56.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.339+0000 7f0e78841700 1 -- 172.21.15.99:0/2091926082 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c00e070 con 0x7f0e7406aae0 2023-10-08T00:12:56.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.339+0000 7f0e78841700 1 -- 172.21.15.99:0/2091926082 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e6c00b9d0 con 0x7f0e7406aae0 2023-10-08T00:12:56.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.339+0000 7f0e78841700 1 -- 172.21.15.99:0/2091926082 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c0092e0 con 0x7f0e7406aae0 2023-10-08T00:12:56.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.340+0000 7f0e7baa7700 1 -- 172.21.15.99:0/2091926082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e7406aae0 msgr2=0x7f0e740cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:56.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.340+0000 7f0e7baa7700 1 --2- 172.21.15.99:0/2091926082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e7406aae0 0x7f0e740cf590 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c009f20 tx=0x7f0e6c009840 comp rx=0 tx=0).stop 2023-10-08T00:12:56.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.340+0000 7f0e7baa7700 1 -- 172.21.15.99:0/2091926082 shutdown_connections 2023-10-08T00:12:56.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.340+0000 7f0e7baa7700 1 --2- 172.21.15.99:0/2091926082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e7406aae0 0x7f0e740cf590 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:56.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.340+0000 7f0e7baa7700 1 --2- 172.21.15.99:0/2091926082 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e7415a290 0x7f0e7406a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:56.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.340+0000 7f0e7baa7700 1 -- 172.21.15.99:0/2091926082 >> 172.21.15.99:0/2091926082 conn(0x7f0e740d1880 msgr2=0x7f0e740cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:56.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.340+0000 7f0e7baa7700 1 -- 172.21.15.99:0/2091926082 shutdown_connections 2023-10-08T00:12:56.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.340+0000 7f0e7baa7700 1 -- 172.21.15.99:0/2091926082 wait complete. 2023-10-08T00:12:56.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.341+0000 7f0e7baa7700 1 Processor -- start 2023-10-08T00:12:56.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.341+0000 7f0e7baa7700 1 -- start start 2023-10-08T00:12:56.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.342+0000 7f0e7baa7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e7406aae0 0x7f0e74155890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:56.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.342+0000 7f0e7baa7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e74159ec0 0x7f0e740b53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:56.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.342+0000 7f0e7baa7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e7415a290 0x7f0e7416a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:56.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.342+0000 7f0e7baa7700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7415b630 con 0x7f0e74159ec0 2023-10-08T00:12:56.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.342+0000 7f0e7baa7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7415c1b0 con 0x7f0e7406aae0 2023-10-08T00:12:56.347 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.342+0000 7f0e7baa7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7415b880 con 0x7f0e7415a290 2023-10-08T00:12:56.347 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.342+0000 7f0e79042700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e74159ec0 0x7f0e740b53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:56.347 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.342+0000 7f0e79042700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e74159ec0 0x7f0e740b53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:32782/0 (socket says 172.21.15.99:32782) 2023-10-08T00:12:56.347 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.342+0000 7f0e79042700 1 -- 172.21.15.99:0/4002316098 learned_addr learned my addr 172.21.15.99:0/4002316098 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:56.347 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.342+0000 7f0e79843700 1 --2- 172.21.15.99:0/4002316098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e7406aae0 0x7f0e74155890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:56.347 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.342+0000 7f0e7a044700 1 --2- 172.21.15.99:0/4002316098 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e7415a290 0x7f0e7416a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:56.347 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.342+0000 7f0e79042700 1 -- 172.21.15.99:0/4002316098 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e7415a290 msgr2=0x7f0e7416a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:56.348 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.342+0000 7f0e79042700 1 --2- 172.21.15.99:0/4002316098 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e7415a290 0x7f0e7416a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:56.348 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.342+0000 7f0e79042700 1 -- 172.21.15.99:0/4002316098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e7406aae0 msgr2=0x7f0e74155890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:56.348 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.342+0000 7f0e79042700 1 --2- 172.21.15.99:0/4002316098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e7406aae0 0x7f0e74155890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:56.348 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.342+0000 7f0e79042700 1 -- 172.21.15.99:0/4002316098 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e6c009ae0 con 0x7f0e74159ec0 2023-10-08T00:12:56.348 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.342+0000 7f0e79042700 1 --2- 172.21.15.99:0/4002316098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e74159ec0 0x7f0e740b53d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f0e6400e960 tx=0x7f0e640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:56.348 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.342+0000 7f0e69ffb700 1 -- 172.21.15.99:0/4002316098 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e64013680 con 0x7f0e74159ec0 2023-10-08T00:12:56.348 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.342+0000 7f0e69ffb700 1 -- 172.21.15.99:0/4002316098 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e640137e0 con 0x7f0e74159ec0 2023-10-08T00:12:56.348 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.343+0000 7f0e7baa7700 1 -- 172.21.15.99:0/4002316098 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e7415bb60 con 0x7f0e74159ec0 2023-10-08T00:12:56.348 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.343+0000 7f0e69ffb700 1 -- 172.21.15.99:0/4002316098 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e64013b80 con 0x7f0e74159ec0 2023-10-08T00:12:56.348 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.343+0000 7f0e7baa7700 1 -- 172.21.15.99:0/4002316098 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e74155dd0 con 0x7f0e74159ec0 2023-10-08T00:12:56.348 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.344+0000 7f0e7baa7700 1 -- 172.21.15.99:0/4002316098 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e7404c3c0 con 0x7f0e74159ec0 2023-10-08T00:12:56.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.344+0000 7f0e69ffb700 1 -- 172.21.15.99:0/4002316098 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0e640123d0 con 0x7f0e74159ec0 2023-10-08T00:12:56.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.344+0000 7f0e69ffb700 1 --2- 172.21.15.99:0/4002316098 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0e58066f30 0x7f0e580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:56.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.344+0000 7f0e69ffb700 1 -- 172.21.15.99:0/4002316098 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f0e640844e0 con 0x7f0e74159ec0 2023-10-08T00:12:56.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.345+0000 7f0e79843700 1 --2- 172.21.15.99:0/4002316098 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0e58066f30 0x7f0e580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:56.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.345+0000 7f0e79843700 1 --2- 172.21.15.99:0/4002316098 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0e58066f30 0x7f0e580693e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0e70003a80 tx=0x7f0e70007ec0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:56.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.349+0000 7f0e69ffb700 1 -- 172.21.15.99:0/4002316098 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0e64051890 con 0x7f0e74159ec0 2023-10-08T00:12:56.534 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:56.534+0000 7f0e7baa7700 1 -- 172.21.15.99:0/4002316098 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f0e740b04b0 con 0x7f0e74159ec0 2023-10-08T00:12:57.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.202+0000 7f0e69ffb700 1 -- 172.21.15.99:0/4002316098 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v67) v1 ==== 170+0+0 (secure 0 0 0) 0x7f0e64054eb0 con 0x7f0e74159ec0 2023-10-08T00:12:57.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.212+0000 7f0e7baa7700 1 -- 172.21.15.99:0/4002316098 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0e58066f30 msgr2=0x7f0e580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:57.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.212+0000 7f0e7baa7700 1 --2- 172.21.15.99:0/4002316098 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0e58066f30 0x7f0e580693e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0e70003a80 tx=0x7f0e70007ec0 comp rx=0 tx=0).stop 2023-10-08T00:12:57.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.212+0000 7f0e7baa7700 1 -- 172.21.15.99:0/4002316098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e74159ec0 msgr2=0x7f0e740b53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:57.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.212+0000 7f0e7baa7700 1 --2- 172.21.15.99:0/4002316098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e74159ec0 0x7f0e740b53d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f0e6400e960 tx=0x7f0e640091a0 comp rx=0 tx=0).stop 2023-10-08T00:12:57.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.213+0000 7f0e7baa7700 1 -- 172.21.15.99:0/4002316098 shutdown_connections 2023-10-08T00:12:57.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.213+0000 7f0e7baa7700 1 --2- 172.21.15.99:0/4002316098 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0e58066f30 0x7f0e580693e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:57.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.213+0000 7f0e7baa7700 1 --2- 172.21.15.99:0/4002316098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e7406aae0 0x7f0e74155890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:57.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.213+0000 7f0e7baa7700 1 --2- 172.21.15.99:0/4002316098 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e7415a290 0x7f0e7416a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:57.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.213+0000 7f0e7baa7700 1 --2- 172.21.15.99:0/4002316098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e74159ec0 0x7f0e740b53d0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:57.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.213+0000 7f0e7baa7700 1 -- 172.21.15.99:0/4002316098 >> 172.21.15.99:0/4002316098 conn(0x7f0e740d1880 msgr2=0x7f0e74158ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:57.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.213+0000 7f0e7baa7700 1 -- 172.21.15.99:0/4002316098 shutdown_connections 2023-10-08T00:12:57.216 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.213+0000 7f0e7baa7700 1 -- 172.21.15.99:0/4002316098 wait complete. 2023-10-08T00:12:57.216 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' created 2023-10-08T00:12:57.226 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T00:12:57.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.342+0000 7fdebea2b700 1 Processor -- start 2023-10-08T00:12:57.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.342+0000 7fdebea2b700 1 -- start start 2023-10-08T00:12:57.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.343+0000 7fdebea2b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fdeb80b6580 0x7fdeb806a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:57.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.343+0000 7fdebea2b700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdeb806ac90 con 0x7fdeb80cce30 2023-10-08T00:12:57.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.343+0000 7fdebea2b700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdeb80b4d70 con 0x7fdeb806a830 2023-10-08T00:12:57.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.343+0000 7fdebea2b700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdeb815b3b0 con 0x7fdeb80b6580 2023-10-08T00:12:57.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.343+0000 7fdebda29700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fdeb80cce30 0x7fdeb815aca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:47558/0 (socket says 172.21.15.99:47558) 2023-10-08T00:12:57.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.343+0000 7fdebda29700 1 -- 172.21.15.99:0/2255853261 learned_addr learned my addr 172.21.15.99:0/2255853261 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:12:57.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.343+0000 7fdebca27700 1 -- 172.21.15.99:0/2255853261 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 906454613 0 0) 0x7fdeb4001110 con 0x7fdeb80cce30 2023-10-08T00:12:57.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.343+0000 7fdebca27700 1 -- 172.21.15.99:0/2255853261 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdea4003570 con 0x7fdeb80cce30 2023-10-08T00:12:57.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.343+0000 7fdebd228700 1 --2- 172.21.15.99:0/2255853261 >> v2:172.21.15.191:3300/0 conn(0x7fdeb80b6580 0x7fdeb806a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:57.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.344+0000 7fdebca27700 1 -- 172.21.15.99:0/2255853261 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2836487571 0 0) 0x7fdea4003570 con 0x7fdeb80cce30 2023-10-08T00:12:57.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.344+0000 7fdebca27700 1 -- 172.21.15.99:0/2255853261 >> v2:172.21.15.191:3300/0 conn(0x7fdeb80b6580 msgr2=0x7fdeb806a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:57.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.344+0000 7fdebca27700 1 --2- 172.21.15.99:0/2255853261 >> v2:172.21.15.191:3300/0 conn(0x7fdeb80b6580 0x7fdeb806a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:57.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.344+0000 7fdebca27700 1 -- 172.21.15.99:0/2255853261 >> v1:172.21.15.191:6789/0 conn(0x7fdeb806a830 legacy=0x7fdeb80b5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:57.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.344+0000 7fdebca27700 1 -- 172.21.15.99:0/2255853261 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeb80cd8d0 con 0x7fdeb80cce30 2023-10-08T00:12:57.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.344+0000 7fdebca27700 1 -- 172.21.15.99:0/2255853261 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7fdeb4001600 con 0x7fdeb80cce30 2023-10-08T00:12:57.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.344+0000 7fdebca27700 1 -- 172.21.15.99:0/2255853261 >> v1:172.21.15.99:6789/0 conn(0x7fdeb80cce30 legacy=0x7fdeb815aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:57.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.344+0000 7fdebca27700 1 --2- 172.21.15.99:0/2255853261 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdea4003a60 0x7fdea4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:57.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.344+0000 7fdebca27700 1 --2- 172.21.15.99:0/2255853261 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdea4004810 0x7fdea4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:57.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.344+0000 7fdebca27700 1 --2- 172.21.15.99:0/2255853261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdea40051c0 0x7fdea400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:57.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.344+0000 7fdebca27700 1 -- 172.21.15.99:0/2255853261 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb815b3b0 con 0x7fdea40051c0 2023-10-08T00:12:57.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.344+0000 7fdebca27700 1 -- 172.21.15.99:0/2255853261 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea400c160 con 0x7fdea4003a60 2023-10-08T00:12:57.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.344+0000 7fdebca27700 1 -- 172.21.15.99:0/2255853261 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdea400c320 con 0x7fdea4004810 2023-10-08T00:12:57.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.344+0000 7fdebda29700 1 --2- 172.21.15.99:0/2255853261 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdea4003a60 0x7fdea4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:57.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.344+0000 7fdebe22a700 1 --2- 172.21.15.99:0/2255853261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdea40051c0 0x7fdea400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:57.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.344+0000 7fdebd228700 1 --2- 172.21.15.99:0/2255853261 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdea4004810 0x7fdea4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:57.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.344+0000 7fdebe22a700 1 -- 172.21.15.99:0/2255853261 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdea4004810 msgr2=0x7fdea4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:57.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.344+0000 7fdebe22a700 1 --2- 172.21.15.99:0/2255853261 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdea4004810 0x7fdea4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:57.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.344+0000 7fdebe22a700 1 -- 172.21.15.99:0/2255853261 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdea4003a60 msgr2=0x7fdea4003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:57.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.344+0000 7fdebe22a700 1 --2- 172.21.15.99:0/2255853261 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdea4003a60 0x7fdea4003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:57.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.344+0000 7fdebe22a700 1 -- 172.21.15.99:0/2255853261 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea4003570 con 0x7fdea40051c0 2023-10-08T00:12:57.352 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.344+0000 7fdebe22a700 1 -- 172.21.15.99:0/2255853261 reap_dead start 2023-10-08T00:12:57.352 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.344+0000 7fdebe22a700 1 --2- 172.21.15.99:0/2255853261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdea40051c0 0x7fdea400ba60 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0001fc0 tx=0x7fdeb0001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cdd6fe468450748 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:57.352 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.344+0000 7fdebca27700 1 -- 172.21.15.99:0/2255853261 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb0001950 con 0x7fdea40051c0 2023-10-08T00:12:57.352 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.344+0000 7fdebca27700 1 -- 172.21.15.99:0/2255853261 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeb000c8b0 con 0x7fdea40051c0 2023-10-08T00:12:57.352 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.344+0000 7fdebca27700 1 -- 172.21.15.99:0/2255853261 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb000cc90 con 0x7fdea40051c0 2023-10-08T00:12:57.352 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.345+0000 7fdebea2b700 1 -- 172.21.15.99:0/2255853261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdea40051c0 msgr2=0x7fdea400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:57.352 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.345+0000 7fdebea2b700 1 --2- 172.21.15.99:0/2255853261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdea40051c0 0x7fdea400ba60 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0001fc0 tx=0x7fdeb0001640 comp rx=0 tx=0).stop 2023-10-08T00:12:57.352 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.345+0000 7fdebea2b700 1 -- 172.21.15.99:0/2255853261 shutdown_connections 2023-10-08T00:12:57.352 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.345+0000 7fdebea2b700 1 --2- 172.21.15.99:0/2255853261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdea40051c0 0x7fdea400ba60 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:57.352 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.345+0000 7fdebea2b700 1 -- 172.21.15.99:0/2255853261 >> 172.21.15.99:0/2255853261 conn(0x7fdeb8068cb0 msgr2=0x7fdeb80690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:57.352 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.345+0000 7fdebea2b700 1 -- 172.21.15.99:0/2255853261 shutdown_connections 2023-10-08T00:12:57.352 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.345+0000 7fdebea2b700 1 -- 172.21.15.99:0/2255853261 wait complete. 2023-10-08T00:12:57.352 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.346+0000 7fdebea2b700 1 Processor -- start 2023-10-08T00:12:57.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.346+0000 7fdebea2b700 1 -- start start 2023-10-08T00:12:57.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.347+0000 7fdebea2b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdea40051c0 0x7fdeb80b2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:57.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.347+0000 7fdebea2b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdeb80b1f70 0x7fdeb816a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:57.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.347+0000 7fdebea2b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdeb80b23e0 0x7fdeb816a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:57.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.347+0000 7fdebea2b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb8152e10 con 0x7fdeb80b23e0 2023-10-08T00:12:57.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.347+0000 7fdebea2b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb816ac30 con 0x7fdea40051c0 2023-10-08T00:12:57.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.347+0000 7fdebea2b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb8151fb0 con 0x7fdeb80b1f70 2023-10-08T00:12:57.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.347+0000 7fdebe22a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdeb80b23e0 0x7fdeb816a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:57.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.347+0000 7fdebe22a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdeb80b23e0 0x7fdeb816a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:32790/0 (socket says 172.21.15.99:32790) 2023-10-08T00:12:57.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.347+0000 7fdebe22a700 1 -- 172.21.15.99:0/3031091166 learned_addr learned my addr 172.21.15.99:0/3031091166 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:57.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.347+0000 7fdebda29700 1 --2- 172.21.15.99:0/3031091166 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdea40051c0 0x7fdeb80b2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:57.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.347+0000 7fdebd228700 1 --2- 172.21.15.99:0/3031091166 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdeb80b1f70 0x7fdeb816a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:57.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.347+0000 7fdebe22a700 1 -- 172.21.15.99:0/3031091166 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdeb80b1f70 msgr2=0x7fdeb816a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:57.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.347+0000 7fdebe22a700 1 --2- 172.21.15.99:0/3031091166 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdeb80b1f70 0x7fdeb816a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:57.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.347+0000 7fdebe22a700 1 -- 172.21.15.99:0/3031091166 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdea40051c0 msgr2=0x7fdeb80b2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:57.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.347+0000 7fdebe22a700 1 --2- 172.21.15.99:0/3031091166 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdea40051c0 0x7fdeb80b2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:57.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.347+0000 7fdebe22a700 1 -- 172.21.15.99:0/3031091166 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeb8152200 con 0x7fdeb80b23e0 2023-10-08T00:12:57.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.348+0000 7fdebe22a700 1 --2- 172.21.15.99:0/3031091166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdeb80b23e0 0x7fdeb816a6f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fdeb80b3e00 tx=0x7fdeb0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:57.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.348+0000 7fdeadffb700 1 -- 172.21.15.99:0/3031091166 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb0001950 con 0x7fdeb80b23e0 2023-10-08T00:12:57.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.348+0000 7fdeadffb700 1 -- 172.21.15.99:0/3031091166 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeb000c6d0 con 0x7fdeb80b23e0 2023-10-08T00:12:57.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.348+0000 7fdebea2b700 1 -- 172.21.15.99:0/3031091166 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdeb81524e0 con 0x7fdeb80b23e0 2023-10-08T00:12:57.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.348+0000 7fdeadffb700 1 -- 172.21.15.99:0/3031091166 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb001e870 con 0x7fdeb80b23e0 2023-10-08T00:12:57.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.348+0000 7fdebea2b700 1 -- 172.21.15.99:0/3031091166 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeb8152aa0 con 0x7fdeb80b23e0 2023-10-08T00:12:57.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.349+0000 7fdebea2b700 1 -- 172.21.15.99:0/3031091166 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdeb8152fc0 con 0x7fdeb80b23e0 2023-10-08T00:12:57.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.349+0000 7fdeadffb700 1 -- 172.21.15.99:0/3031091166 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdeb001ec20 con 0x7fdeb80b23e0 2023-10-08T00:12:57.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.350+0000 7fdeadffb700 1 --2- 172.21.15.99:0/3031091166 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fde9c066f30 0x7fde9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:57.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.350+0000 7fdeadffb700 1 -- 172.21.15.99:0/3031091166 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5605+0+0 (secure 0 0 0) 0x7fdeb0085e90 con 0x7fdeb80b23e0 2023-10-08T00:12:57.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.350+0000 7fdebda29700 1 --2- 172.21.15.99:0/3031091166 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fde9c066f30 0x7fde9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:57.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.350+0000 7fdebda29700 1 --2- 172.21.15.99:0/3031091166 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fde9c066f30 0x7fde9c0693e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4001730 tx=0x7fdeb80ce540 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:57.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.354+0000 7fdeadffb700 1 -- 172.21.15.99:0/3031091166 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdeb8152fc0 con 0x7fdeb80b23e0 2023-10-08T00:12:57.580 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:57.579+0000 7fdebea2b700 1 -- 172.21.15.99:0/3031091166 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fdeb8152fc0 con 0x7fdeb80b23e0 2023-10-08T00:12:58.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.222+0000 7fdeadffb700 1 -- 172.21.15.99:0/3031091166 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v28) v1 ==== 172+0+0 (secure 0 0 0) 0x7fdeb8152fc0 con 0x7fdeb80b23e0 2023-10-08T00:12:58.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.224+0000 7fdebea2b700 1 -- 172.21.15.99:0/3031091166 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fde9c066f30 msgr2=0x7fde9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:58.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.225+0000 7fdebea2b700 1 --2- 172.21.15.99:0/3031091166 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fde9c066f30 0x7fde9c0693e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4001730 tx=0x7fdeb80ce540 comp rx=0 tx=0).stop 2023-10-08T00:12:58.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.225+0000 7fdebea2b700 1 -- 172.21.15.99:0/3031091166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdeb80b23e0 msgr2=0x7fdeb816a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:58.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.225+0000 7fdebea2b700 1 --2- 172.21.15.99:0/3031091166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdeb80b23e0 0x7fdeb816a6f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fdeb80b3e00 tx=0x7fdeb0015000 comp rx=0 tx=0).stop 2023-10-08T00:12:58.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.225+0000 7fdebea2b700 1 -- 172.21.15.99:0/3031091166 shutdown_connections 2023-10-08T00:12:58.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.225+0000 7fdebea2b700 1 --2- 172.21.15.99:0/3031091166 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fde9c066f30 0x7fde9c0693e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:58.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.225+0000 7fdebea2b700 1 --2- 172.21.15.99:0/3031091166 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdea40051c0 0x7fdeb80b2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:58.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.225+0000 7fdebea2b700 1 --2- 172.21.15.99:0/3031091166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdeb80b23e0 0x7fdeb816a6f0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:58.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.225+0000 7fdebea2b700 1 --2- 172.21.15.99:0/3031091166 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdeb80b1f70 0x7fdeb816a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:58.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.225+0000 7fdebea2b700 1 -- 172.21.15.99:0/3031091166 >> 172.21.15.99:0/3031091166 conn(0x7fdeb8068cb0 msgr2=0x7fdeb815a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:58.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.225+0000 7fdebea2b700 1 -- 172.21.15.99:0/3031091166 shutdown_connections 2023-10-08T00:12:58.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.226+0000 7fdebea2b700 1 -- 172.21.15.99:0/3031091166 wait complete. 2023-10-08T00:12:58.230 INFO:teuthology.orchestra.run.smithi099.stderr:new fs with metadata pool 8 and data pool 9 2023-10-08T00:12:58.241 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:12:58.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.356+0000 7f5e49bd4700 1 Processor -- start 2023-10-08T00:12:58.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.356+0000 7f5e49bd4700 1 -- start start 2023-10-08T00:12:58.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.357+0000 7f5e49bd4700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5e44157c40 0x7f5e44158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:58.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.357+0000 7f5e49bd4700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e4415cbb0 con 0x7f5e44157870 2023-10-08T00:12:58.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.357+0000 7f5e49bd4700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e4415bfa0 con 0x7f5e4415a620 2023-10-08T00:12:58.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.357+0000 7f5e49bd4700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e4415bdb0 con 0x7f5e44157c40 2023-10-08T00:12:58.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.357+0000 7f5e43fff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5e44157c40 0x7f5e44158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:58.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.357+0000 7f5e437fe700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f5e44157870 0x7f5e4415b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:47562/0 (socket says 172.21.15.99:47562) 2023-10-08T00:12:58.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.357+0000 7f5e437fe700 1 -- 172.21.15.99:0/194843470 learned_addr learned my addr 172.21.15.99:0/194843470 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:12:58.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.357+0000 7f5e43fff700 1 -- 172.21.15.99:0/194843470 >> v1:172.21.15.191:6789/0 conn(0x7f5e4415a620 legacy=0x7f5e4415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:58.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.357+0000 7f5e43fff700 1 -- 172.21.15.99:0/194843470 >> v1:172.21.15.99:6789/0 conn(0x7f5e44157870 legacy=0x7f5e4415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:58.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.357+0000 7f5e43fff700 1 -- 172.21.15.99:0/194843470 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e4415c280 con 0x7f5e44157c40 2023-10-08T00:12:58.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.357+0000 7f5e43fff700 1 --2- 172.21.15.99:0/194843470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e44157c40 0x7f5e44158110 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c009e60 tx=0x7f5e3c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c948231818ebed7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:58.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.357+0000 7f5e427fc700 1 -- 172.21.15.99:0/194843470 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c00e070 con 0x7f5e44157c40 2023-10-08T00:12:58.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.357+0000 7f5e427fc700 1 -- 172.21.15.99:0/194843470 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e3c00bbf0 con 0x7f5e44157c40 2023-10-08T00:12:58.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.357+0000 7f5e427fc700 1 -- 172.21.15.99:0/194843470 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c012430 con 0x7f5e44157c40 2023-10-08T00:12:58.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.358+0000 7f5e49bd4700 1 -- 172.21.15.99:0/194843470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e44157c40 msgr2=0x7f5e44158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:58.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.358+0000 7f5e49bd4700 1 --2- 172.21.15.99:0/194843470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e44157c40 0x7f5e44158110 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c009e60 tx=0x7f5e3c009780 comp rx=0 tx=0).stop 2023-10-08T00:12:58.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.358+0000 7f5e49bd4700 1 -- 172.21.15.99:0/194843470 shutdown_connections 2023-10-08T00:12:58.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.358+0000 7f5e49bd4700 1 --2- 172.21.15.99:0/194843470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e44157c40 0x7f5e44158110 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:58.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.358+0000 7f5e49bd4700 1 -- 172.21.15.99:0/194843470 >> 172.21.15.99:0/194843470 conn(0x7f5e44068ce0 msgr2=0x7f5e440690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:58.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.358+0000 7f5e49bd4700 1 -- 172.21.15.99:0/194843470 shutdown_connections 2023-10-08T00:12:58.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.358+0000 7f5e49bd4700 1 -- 172.21.15.99:0/194843470 wait complete. 2023-10-08T00:12:58.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.359+0000 7f5e49bd4700 1 Processor -- start 2023-10-08T00:12:58.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.359+0000 7f5e49bd4700 1 -- start start 2023-10-08T00:12:58.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.360+0000 7f5e49bd4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e44157870 0x7f5e441473a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:58.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.360+0000 7f5e49bd4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e44157c40 0x7f5e4415df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:58.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.360+0000 7f5e49bd4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e4415a620 0x7f5e440c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:58.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.360+0000 7f5e49bd4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e441478e0 con 0x7f5e44157c40 2023-10-08T00:12:58.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.360+0000 7f5e49bd4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4415f800 con 0x7f5e4415a620 2023-10-08T00:12:58.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.360+0000 7f5e49bd4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4415e760 con 0x7f5e44157870 2023-10-08T00:12:58.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.360+0000 7f5e42ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e44157c40 0x7f5e4415df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:58.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.360+0000 7f5e42ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e44157c40 0x7f5e4415df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:32804/0 (socket says 172.21.15.99:32804) 2023-10-08T00:12:58.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.360+0000 7f5e42ffd700 1 -- 172.21.15.99:0/2521503150 learned_addr learned my addr 172.21.15.99:0/2521503150 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:58.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.360+0000 7f5e43fff700 1 --2- 172.21.15.99:0/2521503150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e4415a620 0x7f5e440c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:58.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.360+0000 7f5e437fe700 1 --2- 172.21.15.99:0/2521503150 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e44157870 0x7f5e441473a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:58.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.360+0000 7f5e42ffd700 1 -- 172.21.15.99:0/2521503150 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e44157870 msgr2=0x7f5e441473a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:58.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.360+0000 7f5e42ffd700 1 --2- 172.21.15.99:0/2521503150 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e44157870 0x7f5e441473a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:58.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.360+0000 7f5e42ffd700 1 -- 172.21.15.99:0/2521503150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e4415a620 msgr2=0x7f5e440c7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:58.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.360+0000 7f5e42ffd700 1 --2- 172.21.15.99:0/2521503150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e4415a620 0x7f5e440c7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:58.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.360+0000 7f5e42ffd700 1 -- 172.21.15.99:0/2521503150 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e3c009a20 con 0x7f5e44157c40 2023-10-08T00:12:58.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.360+0000 7f5e437fe700 1 --2- 172.21.15.99:0/2521503150 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e44157870 0x7f5e441473a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:12:58.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.361+0000 7f5e42ffd700 1 --2- 172.21.15.99:0/2521503150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e44157c40 0x7f5e4415df10 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f5e3400b5e0 tx=0x7f5e340090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:58.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.361+0000 7f5e2b7fe700 1 -- 172.21.15.99:0/2521503150 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3400f510 con 0x7f5e44157c40 2023-10-08T00:12:58.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.361+0000 7f5e2b7fe700 1 -- 172.21.15.99:0/2521503150 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e3400f6b0 con 0x7f5e44157c40 2023-10-08T00:12:58.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.361+0000 7f5e2b7fe700 1 -- 172.21.15.99:0/2521503150 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3400fa70 con 0x7f5e44157c40 2023-10-08T00:12:58.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.361+0000 7f5e49bd4700 1 -- 172.21.15.99:0/2521503150 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e4415ea40 con 0x7f5e44157c40 2023-10-08T00:12:58.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.361+0000 7f5e49bd4700 1 -- 172.21.15.99:0/2521503150 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e4415f000 con 0x7f5e44157c40 2023-10-08T00:12:58.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.362+0000 7f5e49bd4700 1 -- 172.21.15.99:0/2521503150 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e4404c3c0 con 0x7f5e44157c40 2023-10-08T00:12:58.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.362+0000 7f5e2b7fe700 1 -- 172.21.15.99:0/2521503150 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5e3401a400 con 0x7f5e44157c40 2023-10-08T00:12:58.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.362+0000 7f5e2b7fe700 1 --2- 172.21.15.99:0/2521503150 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5e30066f30 0x7f5e300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:58.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.362+0000 7f5e2b7fe700 1 -- 172.21.15.99:0/2521503150 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5e34083f90 con 0x7f5e44157c40 2023-10-08T00:12:58.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.363+0000 7f5e437fe700 1 --2- 172.21.15.99:0/2521503150 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5e30066f30 0x7f5e300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:58.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.363+0000 7f5e437fe700 1 --2- 172.21.15.99:0/2521503150 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5e30066f30 0x7f5e300693e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5e380018c0 tx=0x7f5e38009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:58.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.366+0000 7f5e2b7fe700 1 -- 172.21.15.99:0/2521503150 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5e34051280 con 0x7f5e44157c40 2023-10-08T00:12:58.549 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.548+0000 7f5e49bd4700 1 -- 172.21.15.99:0/2521503150 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5e4415c560 con 0x7f5e44157c40 2023-10-08T00:12:58.550 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.549+0000 7f5e2b7fe700 1 -- 172.21.15.99:0/2521503150 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+15363 (secure 0 0 0) 0x7f5e340548a0 con 0x7f5e44157c40 2023-10-08T00:12:58.550 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:12:58.551 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":68,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:12:57.580926+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:12:55.353771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-08T00:12:56.552157+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:12:58.552 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.551+0000 7f5e49bd4700 1 -- 172.21.15.99:0/2521503150 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5e30066f30 msgr2=0x7f5e300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:58.553 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.551+0000 7f5e49bd4700 1 --2- 172.21.15.99:0/2521503150 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5e30066f30 0x7f5e300693e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5e380018c0 tx=0x7f5e38009040 comp rx=0 tx=0).stop 2023-10-08T00:12:58.553 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.551+0000 7f5e49bd4700 1 -- 172.21.15.99:0/2521503150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e44157c40 msgr2=0x7f5e4415df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:58.553 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.551+0000 7f5e49bd4700 1 --2- 172.21.15.99:0/2521503150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e44157c40 0x7f5e4415df10 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f5e3400b5e0 tx=0x7f5e340090f0 comp rx=0 tx=0).stop 2023-10-08T00:12:58.553 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.551+0000 7f5e49bd4700 1 -- 172.21.15.99:0/2521503150 shutdown_connections 2023-10-08T00:12:58.553 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.551+0000 7f5e49bd4700 1 --2- 172.21.15.99:0/2521503150 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5e30066f30 0x7f5e300693e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:58.553 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.551+0000 7f5e49bd4700 1 --2- 172.21.15.99:0/2521503150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e4415a620 0x7f5e440c7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:58.553 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.551+0000 7f5e49bd4700 1 --2- 172.21.15.99:0/2521503150 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e44157870 0x7f5e441473a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:58.553 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.551+0000 7f5e49bd4700 1 --2- 172.21.15.99:0/2521503150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e44157c40 0x7f5e4415df10 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:58.553 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.551+0000 7f5e49bd4700 1 -- 172.21.15.99:0/2521503150 >> 172.21.15.99:0/2521503150 conn(0x7f5e44068ce0 msgr2=0x7f5e440b5880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:58.554 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.551+0000 7f5e49bd4700 1 -- 172.21.15.99:0/2521503150 shutdown_connections 2023-10-08T00:12:58.554 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.551+0000 7f5e49bd4700 1 -- 172.21.15.99:0/2521503150 wait complete. 2023-10-08T00:12:58.564 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:12:58.679 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.678+0000 7f176a653700 1 Processor -- start 2023-10-08T00:12:58.679 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.678+0000 7f176a653700 1 -- start start 2023-10-08T00:12:58.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.679+0000 7f176a653700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f17640cd6e0 0x7f17640cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:58.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.679+0000 7f176a653700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f176415cbd0 0x7f176415d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:58.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.679+0000 7f176a653700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17640cb7c0 con 0x7f17640cdab0 2023-10-08T00:12:58.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.679+0000 7f176a653700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f176415e850 con 0x7f17640cd6e0 2023-10-08T00:12:58.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.679+0000 7f176a653700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17640cd030 con 0x7f176415cbd0 2023-10-08T00:12:58.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.679+0000 7f1769651700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f17640cd6e0 0x7f17640cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:58.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.679+0000 7f1768e50700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f17640cdab0 0x7f176415d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:47568/0 (socket says 172.21.15.99:47568) 2023-10-08T00:12:58.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.679+0000 7f1768e50700 1 -- 172.21.15.99:0/1024392377 learned_addr learned my addr 172.21.15.99:0/1024392377 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:12:58.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.679+0000 7f1769e52700 1 --2- 172.21.15.99:0/1024392377 >> v2:172.21.15.191:3300/0 conn(0x7f176415cbd0 0x7f176415d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:58.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.679+0000 7f1769651700 1 -- 172.21.15.99:0/1024392377 >> v2:172.21.15.191:3300/0 conn(0x7f176415cbd0 msgr2=0x7f176415d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:58.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.679+0000 7f1769651700 1 --2- 172.21.15.99:0/1024392377 >> v2:172.21.15.191:3300/0 conn(0x7f176415cbd0 0x7f176415d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:58.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.679+0000 7f1769651700 1 -- 172.21.15.99:0/1024392377 >> v1:172.21.15.99:6789/0 conn(0x7f17640cdab0 legacy=0x7f176415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:58.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.679+0000 7f175bfff700 1 -- 172.21.15.99:0/1024392377 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2266429346 0 0) 0x7f17640cb7c0 con 0x7f17640cdab0 2023-10-08T00:12:58.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.679+0000 7f1769651700 1 -- 172.21.15.99:0/1024392377 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f176415e290 con 0x7f17640cd6e0 2023-10-08T00:12:58.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.680+0000 7f1769651700 1 --2- 172.21.15.99:0/1024392377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17640cd6e0 0x7f17640cbc50 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f17640cb730 tx=0x7f1760009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c808eefc64e3a82a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:58.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.680+0000 7f175bfff700 1 -- 172.21.15.99:0/1024392377 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176000e070 con 0x7f17640cd6e0 2023-10-08T00:12:58.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.680+0000 7f175bfff700 1 -- 172.21.15.99:0/1024392377 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f176000bc30 con 0x7f17640cd6e0 2023-10-08T00:12:58.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.680+0000 7f175bfff700 1 -- 172.21.15.99:0/1024392377 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1760012430 con 0x7f17640cd6e0 2023-10-08T00:12:58.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.680+0000 7f176a653700 1 -- 172.21.15.99:0/1024392377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17640cd6e0 msgr2=0x7f17640cbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:58.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.680+0000 7f176a653700 1 --2- 172.21.15.99:0/1024392377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17640cd6e0 0x7f17640cbc50 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f17640cb730 tx=0x7f1760009780 comp rx=0 tx=0).stop 2023-10-08T00:12:58.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.680+0000 7f176a653700 1 -- 172.21.15.99:0/1024392377 shutdown_connections 2023-10-08T00:12:58.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.680+0000 7f176a653700 1 --2- 172.21.15.99:0/1024392377 >> v2:172.21.15.191:3300/0 conn(0x7f176415cbd0 0x7f176415d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:58.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.680+0000 7f176a653700 1 --2- 172.21.15.99:0/1024392377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17640cd6e0 0x7f17640cbc50 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:58.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.680+0000 7f176a653700 1 -- 172.21.15.99:0/1024392377 >> 172.21.15.99:0/1024392377 conn(0x7f1764067660 msgr2=0x7f1764067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:58.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.680+0000 7f176a653700 1 -- 172.21.15.99:0/1024392377 shutdown_connections 2023-10-08T00:12:58.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.680+0000 7f176a653700 1 -- 172.21.15.99:0/1024392377 wait complete. 2023-10-08T00:12:58.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.682+0000 7f176a653700 1 Processor -- start 2023-10-08T00:12:58.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.682+0000 7f176a653700 1 -- start start 2023-10-08T00:12:58.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.682+0000 7f176a653700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f17640cd6e0 0x7f17640b6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:58.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.682+0000 7f176a653700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f17640cdab0 0x7f176414d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:58.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.682+0000 7f176a653700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f176415cbd0 0x7f17640b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:58.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.682+0000 7f176a653700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17640b5600 con 0x7f176415cbd0 2023-10-08T00:12:58.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.682+0000 7f176a653700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17640b4a10 con 0x7f17640cd6e0 2023-10-08T00:12:58.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.682+0000 7f176a653700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f176416a400 con 0x7f17640cdab0 2023-10-08T00:12:58.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.683+0000 7f1769e52700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f176415cbd0 0x7f17640b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:58.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.683+0000 7f1769e52700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f176415cbd0 0x7f17640b44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:32812/0 (socket says 172.21.15.99:32812) 2023-10-08T00:12:58.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.683+0000 7f1769e52700 1 -- 172.21.15.99:0/2741849251 learned_addr learned my addr 172.21.15.99:0/2741849251 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:58.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.683+0000 7f1769651700 1 --2- 172.21.15.99:0/2741849251 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f17640cd6e0 0x7f17640b6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:58.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.683+0000 7f1768e50700 1 --2- 172.21.15.99:0/2741849251 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f17640cdab0 0x7f176414d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:58.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.683+0000 7f1769e52700 1 -- 172.21.15.99:0/2741849251 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f17640cdab0 msgr2=0x7f176414d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:58.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.683+0000 7f1769e52700 1 --2- 172.21.15.99:0/2741849251 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f17640cdab0 0x7f176414d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:58.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.683+0000 7f1769e52700 1 -- 172.21.15.99:0/2741849251 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f17640cd6e0 msgr2=0x7f17640b6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:58.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.683+0000 7f1769e52700 1 --2- 172.21.15.99:0/2741849251 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f17640cd6e0 0x7f17640b6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:58.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.683+0000 7f1769e52700 1 -- 172.21.15.99:0/2741849251 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1760009a20 con 0x7f176415cbd0 2023-10-08T00:12:58.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.683+0000 7f1769e52700 1 --2- 172.21.15.99:0/2741849251 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f176415cbd0 0x7f17640b44d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f175c00d890 tx=0x7f175c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:58.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.683+0000 7f17597fa700 1 -- 172.21.15.99:0/2741849251 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c0125f0 con 0x7f176415cbd0 2023-10-08T00:12:58.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.683+0000 7f17597fa700 1 -- 172.21.15.99:0/2741849251 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f175c012770 con 0x7f176415cbd0 2023-10-08T00:12:58.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.683+0000 7f176a653700 1 -- 172.21.15.99:0/2741849251 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17640b5870 con 0x7f176415cbd0 2023-10-08T00:12:58.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.683+0000 7f17597fa700 1 -- 172.21.15.99:0/2741849251 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c012b30 con 0x7f176415cbd0 2023-10-08T00:12:58.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.683+0000 7f176a653700 1 -- 172.21.15.99:0/2741849251 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17640b6790 con 0x7f176415cbd0 2023-10-08T00:12:58.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.684+0000 7f176a653700 1 -- 172.21.15.99:0/2741849251 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1764089280 con 0x7f176415cbd0 2023-10-08T00:12:58.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.684+0000 7f17597fa700 1 -- 172.21.15.99:0/2741849251 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f175c01e460 con 0x7f176415cbd0 2023-10-08T00:12:58.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.685+0000 7f17597fa700 1 --2- 172.21.15.99:0/2741849251 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1748066f60 0x7f1748069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:58.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.685+0000 7f17597fa700 1 -- 172.21.15.99:0/2741849251 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f175c084a90 con 0x7f176415cbd0 2023-10-08T00:12:58.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.685+0000 7f1769651700 1 --2- 172.21.15.99:0/2741849251 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1748066f60 0x7f1748069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:58.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.685+0000 7f1769651700 1 --2- 172.21.15.99:0/2741849251 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1748066f60 0x7f1748069410 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f17640cb730 tx=0x7f1760019000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:58.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.689+0000 7f17597fa700 1 -- 172.21.15.99:0/2741849251 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f175c051d00 con 0x7f176415cbd0 2023-10-08T00:12:58.873 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.872+0000 7f176a653700 1 -- 172.21.15.99:0/2741849251 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f176415e570 con 0x7f176415cbd0 2023-10-08T00:12:58.873 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.873+0000 7f17597fa700 1 -- 172.21.15.99:0/2741849251 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+15363 (secure 0 0 0) 0x7f175c055320 con 0x7f176415cbd0 2023-10-08T00:12:58.873 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:12:58.874 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":68,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:12:57.580926+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:12:55.353771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-08T00:12:56.552157+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:12:58.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.875+0000 7f176a653700 1 -- 172.21.15.99:0/2741849251 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1748066f60 msgr2=0x7f1748069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:58.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.875+0000 7f176a653700 1 --2- 172.21.15.99:0/2741849251 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1748066f60 0x7f1748069410 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f17640cb730 tx=0x7f1760019000 comp rx=0 tx=0).stop 2023-10-08T00:12:58.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.875+0000 7f176a653700 1 -- 172.21.15.99:0/2741849251 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f176415cbd0 msgr2=0x7f17640b44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:58.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.875+0000 7f176a653700 1 --2- 172.21.15.99:0/2741849251 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f176415cbd0 0x7f17640b44d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f175c00d890 tx=0x7f175c008040 comp rx=0 tx=0).stop 2023-10-08T00:12:58.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.875+0000 7f176a653700 1 -- 172.21.15.99:0/2741849251 shutdown_connections 2023-10-08T00:12:58.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.875+0000 7f176a653700 1 --2- 172.21.15.99:0/2741849251 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1748066f60 0x7f1748069410 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:58.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.875+0000 7f176a653700 1 --2- 172.21.15.99:0/2741849251 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f17640cd6e0 0x7f17640b6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:58.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.875+0000 7f176a653700 1 --2- 172.21.15.99:0/2741849251 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f176415cbd0 0x7f17640b44d0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:58.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.875+0000 7f176a653700 1 --2- 172.21.15.99:0/2741849251 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f17640cdab0 0x7f176414d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:58.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.875+0000 7f176a653700 1 -- 172.21.15.99:0/2741849251 >> 172.21.15.99:0/2741849251 conn(0x7f1764067660 msgr2=0x7f176415d610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:58.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.875+0000 7f176a653700 1 -- 172.21.15.99:0/2741849251 shutdown_connections 2023-10-08T00:12:58.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:58.875+0000 7f176a653700 1 -- 172.21.15.99:0/2741849251 wait complete. 2023-10-08T00:12:58.887 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T00:12:59.004 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.003+0000 7f047c2a7700 1 Processor -- start 2023-10-08T00:12:59.004 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.003+0000 7f047c2a7700 1 -- start start 2023-10-08T00:12:59.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.004+0000 7f047c2a7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0474156160 0x7f0474156530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:59.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.004+0000 7f047c2a7700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f04740cb280 0x7f04740c2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:59.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.004+0000 7f047c2a7700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f04740c71d0 0x7f04740c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:59.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.004+0000 7f047c2a7700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04740c7b90 con 0x7f04740c71d0 2023-10-08T00:12:59.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.004+0000 7f047c2a7700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04740cc440 con 0x7f04740cb280 2023-10-08T00:12:59.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.004+0000 7f047c2a7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04740cb830 con 0x7f0474156160 2023-10-08T00:12:59.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.004+0000 7f047a844700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f04740c71d0 0x7f04740c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:59.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.004+0000 7f047a844700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f04740c71d0 0x7f04740c7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:32826/0 (socket says 172.21.15.99:32826) 2023-10-08T00:12:59.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.004+0000 7f047a844700 1 -- 172.21.15.99:0/1374101614 learned_addr learned my addr 172.21.15.99:0/1374101614 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:59.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.004+0000 7f0479842700 1 --2- 172.21.15.99:0/1374101614 >> v2:172.21.15.191:3300/0 conn(0x7f04740cb280 0x7f04740c2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:59.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.004+0000 7f047a043700 1 --2- 172.21.15.99:0/1374101614 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0474156160 0x7f0474156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:59.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.005+0000 7f047a844700 1 -- 172.21.15.99:0/1374101614 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0474156160 msgr2=0x7f0474156530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:59.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.005+0000 7f047a844700 1 --2- 172.21.15.99:0/1374101614 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0474156160 0x7f0474156530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:59.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.005+0000 7f047a844700 1 -- 172.21.15.99:0/1374101614 >> v2:172.21.15.191:3300/0 conn(0x7f04740cb280 msgr2=0x7f04740c2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:12:59.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.005+0000 7f047a844700 1 --2- 172.21.15.99:0/1374101614 >> v2:172.21.15.191:3300/0 conn(0x7f04740cb280 0x7f04740c2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:59.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.005+0000 7f047a844700 1 -- 172.21.15.99:0/1374101614 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04740cbb10 con 0x7f04740c71d0 2023-10-08T00:12:59.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.005+0000 7f047a844700 1 --2- 172.21.15.99:0/1374101614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04740c71d0 0x7f04740c7650 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f047406c0c0 tx=0x7f046c00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=571ca3f4562ee221 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:59.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.005+0000 7f0479041700 1 -- 172.21.15.99:0/1374101614 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046c00db40 con 0x7f04740c71d0 2023-10-08T00:12:59.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.005+0000 7f0479041700 1 -- 172.21.15.99:0/1374101614 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f046c00dca0 con 0x7f04740c71d0 2023-10-08T00:12:59.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.005+0000 7f0479041700 1 -- 172.21.15.99:0/1374101614 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046c0196b0 con 0x7f04740c71d0 2023-10-08T00:12:59.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.005+0000 7f047c2a7700 1 -- 172.21.15.99:0/1374101614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04740c71d0 msgr2=0x7f04740c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:59.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.005+0000 7f047c2a7700 1 --2- 172.21.15.99:0/1374101614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04740c71d0 0x7f04740c7650 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f047406c0c0 tx=0x7f046c00b4f0 comp rx=0 tx=0).stop 2023-10-08T00:12:59.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.006+0000 7f047c2a7700 1 -- 172.21.15.99:0/1374101614 shutdown_connections 2023-10-08T00:12:59.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.006+0000 7f047c2a7700 1 --2- 172.21.15.99:0/1374101614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04740c71d0 0x7f04740c7650 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:59.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.006+0000 7f047c2a7700 1 --2- 172.21.15.99:0/1374101614 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0474156160 0x7f0474156530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:59.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.006+0000 7f047c2a7700 1 --2- 172.21.15.99:0/1374101614 >> v2:172.21.15.191:3300/0 conn(0x7f04740cb280 0x7f04740c2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:59.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.006+0000 7f047c2a7700 1 -- 172.21.15.99:0/1374101614 >> 172.21.15.99:0/1374101614 conn(0x7f0474067660 msgr2=0x7f0474067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:12:59.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.006+0000 7f047c2a7700 1 -- 172.21.15.99:0/1374101614 shutdown_connections 2023-10-08T00:12:59.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.006+0000 7f047c2a7700 1 -- 172.21.15.99:0/1374101614 wait complete. 2023-10-08T00:12:59.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.007+0000 7f047c2a7700 1 Processor -- start 2023-10-08T00:12:59.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.007+0000 7f047c2a7700 1 -- start start 2023-10-08T00:12:59.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.007+0000 7f047c2a7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04740c71d0 0x7f04740c53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:59.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.007+0000 7f047c2a7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f04740cb280 0x7f04740aaa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:59.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.007+0000 7f047c2a7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0474156160 0x7f047414a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:59.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.007+0000 7f047c2a7700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f047416af60 con 0x7f0474156160 2023-10-08T00:12:59.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.007+0000 7f047c2a7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04740aaf70 con 0x7f04740c71d0 2023-10-08T00:12:59.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.007+0000 7f047c2a7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04740c59f0 con 0x7f04740cb280 2023-10-08T00:12:59.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.008+0000 7f047a844700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0474156160 0x7f047414a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:59.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.008+0000 7f047a844700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0474156160 0x7f047414a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:32828/0 (socket says 172.21.15.99:32828) 2023-10-08T00:12:59.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.008+0000 7f047a844700 1 -- 172.21.15.99:0/3142899529 learned_addr learned my addr 172.21.15.99:0/3142899529 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:12:59.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.008+0000 7f047a043700 1 --2- 172.21.15.99:0/3142899529 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04740c71d0 0x7f04740c53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:59.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.008+0000 7f0479842700 1 --2- 172.21.15.99:0/3142899529 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f04740cb280 0x7f04740aaa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:59.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.008+0000 7f047a844700 1 -- 172.21.15.99:0/3142899529 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f04740cb280 msgr2=0x7f04740aaa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:59.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.008+0000 7f047a844700 1 --2- 172.21.15.99:0/3142899529 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f04740cb280 0x7f04740aaa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:59.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.008+0000 7f047a844700 1 -- 172.21.15.99:0/3142899529 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04740c71d0 msgr2=0x7f04740c53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:12:59.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.008+0000 7f047a844700 1 --2- 172.21.15.99:0/3142899529 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04740c71d0 0x7f04740c53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:12:59.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.008+0000 7f047a844700 1 -- 172.21.15.99:0/3142899529 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f046c00b790 con 0x7f0474156160 2023-10-08T00:12:59.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.008+0000 7f047a844700 1 --2- 172.21.15.99:0/3142899529 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0474156160 0x7f047414a2e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f046c00bfd0 tx=0x7f046c01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:59.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.008+0000 7f046a7fc700 1 -- 172.21.15.99:0/3142899529 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046c010680 con 0x7f0474156160 2023-10-08T00:12:59.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.008+0000 7f046a7fc700 1 -- 172.21.15.99:0/3142899529 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f046c0107e0 con 0x7f0474156160 2023-10-08T00:12:59.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.008+0000 7f047c2a7700 1 -- 172.21.15.99:0/3142899529 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04740c5cd0 con 0x7f0474156160 2023-10-08T00:12:59.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.008+0000 7f046a7fc700 1 -- 172.21.15.99:0/3142899529 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046c010ba0 con 0x7f0474156160 2023-10-08T00:12:59.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.008+0000 7f047c2a7700 1 -- 172.21.15.99:0/3142899529 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04740c6290 con 0x7f0474156160 2023-10-08T00:12:59.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.009+0000 7f047c2a7700 1 -- 172.21.15.99:0/3142899529 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f047416b0d0 con 0x7f0474156160 2023-10-08T00:12:59.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.010+0000 7f046a7fc700 1 -- 172.21.15.99:0/3142899529 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f046c02c3f0 con 0x7f0474156160 2023-10-08T00:12:59.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.010+0000 7f046a7fc700 1 --2- 172.21.15.99:0/3142899529 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0458066f30 0x7f04580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:12:59.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.010+0000 7f046a7fc700 1 -- 172.21.15.99:0/3142899529 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f046c014070 con 0x7f0474156160 2023-10-08T00:12:59.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.010+0000 7f047a043700 1 --2- 172.21.15.99:0/3142899529 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0458066f30 0x7f04580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:12:59.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.010+0000 7f047a043700 1 --2- 172.21.15.99:0/3142899529 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0458066f30 0x7f04580693e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0470009fd0 tx=0x7f047000b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:12:59.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.014+0000 7f046a7fc700 1 -- 172.21.15.99:0/3142899529 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f046c053470 con 0x7f0474156160 2023-10-08T00:12:59.025 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2023-10-08T00:12:59.025 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2023-10-08T00:12:59.025 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-10-08T00:12:59.025 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2023-10-08T00:12:59.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:12:59.247+0000 7f047c2a7700 1 -- 172.21.15.99:0/3142899529 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f047404c3c0 con 0x7f0474156160 2023-10-08T00:13:00.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.240+0000 7f046a7fc700 1 -- 172.21.15.99:0/3142899529 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v31) v1 ==== 118+0+0 (secure 0 0 0) 0x7f046c056a90 con 0x7f0474156160 2023-10-08T00:13:00.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.243+0000 7f047c2a7700 1 -- 172.21.15.99:0/3142899529 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0458066f30 msgr2=0x7f04580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:00.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.243+0000 7f047c2a7700 1 --2- 172.21.15.99:0/3142899529 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0458066f30 0x7f04580693e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0470009fd0 tx=0x7f047000b040 comp rx=0 tx=0).stop 2023-10-08T00:13:00.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.243+0000 7f047c2a7700 1 -- 172.21.15.99:0/3142899529 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0474156160 msgr2=0x7f047414a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:00.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.243+0000 7f047c2a7700 1 --2- 172.21.15.99:0/3142899529 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0474156160 0x7f047414a2e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f046c00bfd0 tx=0x7f046c01b000 comp rx=0 tx=0).stop 2023-10-08T00:13:00.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.243+0000 7f047c2a7700 1 -- 172.21.15.99:0/3142899529 shutdown_connections 2023-10-08T00:13:00.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.243+0000 7f047c2a7700 1 --2- 172.21.15.99:0/3142899529 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0458066f30 0x7f04580693e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:00.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.243+0000 7f047c2a7700 1 --2- 172.21.15.99:0/3142899529 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04740c71d0 0x7f04740c53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:00.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.243+0000 7f047c2a7700 1 --2- 172.21.15.99:0/3142899529 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0474156160 0x7f047414a2e0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:00.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.243+0000 7f047c2a7700 1 --2- 172.21.15.99:0/3142899529 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f04740cb280 0x7f04740aaa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:00.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.243+0000 7f047c2a7700 1 -- 172.21.15.99:0/3142899529 >> 172.21.15.99:0/3142899529 conn(0x7f0474067660 msgr2=0x7f04740b2130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:13:00.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.243+0000 7f047c2a7700 1 -- 172.21.15.99:0/3142899529 shutdown_connections 2023-10-08T00:13:00.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.243+0000 7f047c2a7700 1 -- 172.21.15.99:0/3142899529 wait complete. 2023-10-08T00:13:00.258 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T00:13:00.379 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.378+0000 7f74cce70700 1 Processor -- start 2023-10-08T00:13:00.379 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.378+0000 7f74cce70700 1 -- start start 2023-10-08T00:13:00.385 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.379+0000 7f74cce70700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f74c80b4860 0x7f74c80b6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:00.385 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.379+0000 7f74cce70700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f74c80b4c30 0x7f74c8159a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:00.385 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.379+0000 7f74cce70700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74c80b14f0 con 0x7f74c8159fd0 2023-10-08T00:13:00.385 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.379+0000 7f74cce70700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74c80b17d0 con 0x7f74c80b4860 2023-10-08T00:13:00.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.379+0000 7f74cce70700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c80b1ab0 con 0x7f74c80b4c30 2023-10-08T00:13:00.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.379+0000 7f74c6d9d700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f74c8159fd0 0x7f74c80cfe90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:47578/0 (socket says 172.21.15.99:47578) 2023-10-08T00:13:00.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.379+0000 7f74c6d9d700 1 -- 172.21.15.99:0/3945634441 learned_addr learned my addr 172.21.15.99:0/3945634441 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:13:00.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.379+0000 7f74bffff700 1 --2- 172.21.15.99:0/3945634441 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f74c80b4c30 0x7f74c8159a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:00.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.379+0000 7f74c659c700 1 --2- 172.21.15.99:0/3945634441 >> v2:172.21.15.191:3300/0 conn(0x7f74c80b4860 0x7f74c80b6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:00.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.379+0000 7f74c5d9b700 1 -- 172.21.15.99:0/3945634441 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2496872212 0 0) 0x7f74c80b14f0 con 0x7f74c8159fd0 2023-10-08T00:13:00.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.379+0000 7f74c5d9b700 1 -- 172.21.15.99:0/3945634441 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74b4003570 con 0x7f74c8159fd0 2023-10-08T00:13:00.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.380+0000 7f74c5d9b700 1 -- 172.21.15.99:0/3945634441 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1857399557 0 0) 0x7f74b4003570 con 0x7f74c8159fd0 2023-10-08T00:13:00.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.380+0000 7f74c5d9b700 1 -- 172.21.15.99:0/3945634441 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f74c80b4c30 msgr2=0x7f74c8159a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:13:00.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.380+0000 7f74c5d9b700 1 --2- 172.21.15.99:0/3945634441 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f74c80b4c30 0x7f74c8159a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:00.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.380+0000 7f74c5d9b700 1 -- 172.21.15.99:0/3945634441 >> v2:172.21.15.191:3300/0 conn(0x7f74c80b4860 msgr2=0x7f74c80b6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:13:00.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.380+0000 7f74c5d9b700 1 --2- 172.21.15.99:0/3945634441 >> v2:172.21.15.191:3300/0 conn(0x7f74c80b4860 0x7f74c80b6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:00.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.380+0000 7f74c5d9b700 1 -- 172.21.15.99:0/3945634441 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c80b1d90 con 0x7f74c8159fd0 2023-10-08T00:13:00.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.380+0000 7f74c5d9b700 1 -- 172.21.15.99:0/3945634441 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f74c00017f0 con 0x7f74c8159fd0 2023-10-08T00:13:00.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.380+0000 7f74bffff700 1 --2- 172.21.15.99:0/3945634441 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f74c80b4c30 0x7f74c8159a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:13:00.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.380+0000 7f74c659c700 1 --2- 172.21.15.99:0/3945634441 >> v2:172.21.15.191:3300/0 conn(0x7f74c80b4860 0x7f74c80b6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:13:00.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.380+0000 7f74c5d9b700 1 -- 172.21.15.99:0/3945634441 >> v1:172.21.15.99:6789/0 conn(0x7f74c8159fd0 legacy=0x7f74c80cfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:00.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.380+0000 7f74c5d9b700 1 --2- 172.21.15.99:0/3945634441 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f74c80b4c30 0x7f74b4003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:00.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.380+0000 7f74c5d9b700 1 --2- 172.21.15.99:0/3945634441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74b4003fa0 0x7f74b4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:00.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.380+0000 7f74c5d9b700 1 --2- 172.21.15.99:0/3945634441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f74b4004d50 0x7f74b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:00.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.380+0000 7f74c5d9b700 1 -- 172.21.15.99:0/3945634441 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c80b17d0 con 0x7f74b4003fa0 2023-10-08T00:13:00.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.380+0000 7f74c5d9b700 1 -- 172.21.15.99:0/3945634441 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c80b1ab0 con 0x7f74b4004d50 2023-10-08T00:13:00.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.380+0000 7f74c5d9b700 1 -- 172.21.15.99:0/3945634441 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74b400ba60 con 0x7f74c80b4c30 2023-10-08T00:13:00.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.380+0000 7f74bffff700 1 --2- 172.21.15.99:0/3945634441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74b4003fa0 0x7f74b4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:00.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.380+0000 7f74c659c700 1 --2- 172.21.15.99:0/3945634441 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f74c80b4c30 0x7f74b4003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:00.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.380+0000 7f74c6d9d700 1 --2- 172.21.15.99:0/3945634441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f74b4004d50 0x7f74b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:00.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.380+0000 7f74bffff700 1 -- 172.21.15.99:0/3945634441 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f74c80b4c30 msgr2=0x7f74b4003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:13:00.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.380+0000 7f74bffff700 1 --2- 172.21.15.99:0/3945634441 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f74c80b4c30 0x7f74b4003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:00.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.380+0000 7f74bffff700 1 -- 172.21.15.99:0/3945634441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f74b4004d50 msgr2=0x7f74b40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:13:00.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.380+0000 7f74bffff700 1 --2- 172.21.15.99:0/3945634441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f74b4004d50 0x7f74b40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:00.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.380+0000 7f74bffff700 1 -- 172.21.15.99:0/3945634441 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b0009420 con 0x7f74b4003fa0 2023-10-08T00:13:00.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.380+0000 7f74c659c700 1 --2- 172.21.15.99:0/3945634441 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f74c80b4c30 0x7f74b4003a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:13:00.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.380+0000 7f74c6d9d700 1 --2- 172.21.15.99:0/3945634441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f74b4004d50 0x7f74b40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:13:00.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.380+0000 7f74bffff700 1 --2- 172.21.15.99:0/3945634441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74b4003fa0 0x7f74b4004810 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f74b0009df0 tx=0x7f74b00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=492a2b753a6ec077 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:13:00.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.380+0000 7f74c5d9b700 1 -- 172.21.15.99:0/3945634441 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b0009740 con 0x7f74b4003fa0 2023-10-08T00:13:00.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.380+0000 7f74c5d9b700 1 -- 172.21.15.99:0/3945634441 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b000e920 con 0x7f74b4003fa0 2023-10-08T00:13:00.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.380+0000 7f74c5d9b700 1 -- 172.21.15.99:0/3945634441 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b000ece0 con 0x7f74b4003fa0 2023-10-08T00:13:00.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.381+0000 7f74cce70700 1 -- 172.21.15.99:0/3945634441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74b4003fa0 msgr2=0x7f74b4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:00.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.381+0000 7f74cce70700 1 --2- 172.21.15.99:0/3945634441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74b4003fa0 0x7f74b4004810 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f74b0009df0 tx=0x7f74b00091a0 comp rx=0 tx=0).stop 2023-10-08T00:13:00.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.381+0000 7f74c6d9d700 1 -- 172.21.15.99:0/3945634441 reap_dead start 2023-10-08T00:13:00.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.381+0000 7f74cce70700 1 -- 172.21.15.99:0/3945634441 shutdown_connections 2023-10-08T00:13:00.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.381+0000 7f74cce70700 1 -- 172.21.15.99:0/3945634441 >> 172.21.15.99:0/3945634441 conn(0x7f74c809a0f0 msgr2=0x7f74c8067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:13:00.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.381+0000 7f74cce70700 1 -- 172.21.15.99:0/3945634441 shutdown_connections 2023-10-08T00:13:00.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.381+0000 7f74cce70700 1 -- 172.21.15.99:0/3945634441 wait complete. 2023-10-08T00:13:00.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.382+0000 7f74cce70700 1 Processor -- start 2023-10-08T00:13:00.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.382+0000 7f74cce70700 1 -- start start 2023-10-08T00:13:00.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.383+0000 7f74cce70700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f74b4003fa0 0x7f74c809aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:00.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.383+0000 7f74cce70700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f74c80ac260 0x7f74c81471d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:00.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.383+0000 7f74cce70700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74c80af7c0 0x7f74c80accc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:00.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.383+0000 7f74cce70700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c809d970 con 0x7f74c80af7c0 2023-10-08T00:13:00.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.383+0000 7f74cce70700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c809d340 con 0x7f74b4003fa0 2023-10-08T00:13:00.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.383+0000 7f74cce70700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c80b0040 con 0x7f74c80ac260 2023-10-08T00:13:00.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.383+0000 7f74c6d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74c80af7c0 0x7f74c80accc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:00.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.383+0000 7f74c6d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74c80af7c0 0x7f74c80accc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:32858/0 (socket says 172.21.15.99:32858) 2023-10-08T00:13:00.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.383+0000 7f74c6d9d700 1 -- 172.21.15.99:0/1549506071 learned_addr learned my addr 172.21.15.99:0/1549506071 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:13:00.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.383+0000 7f74c659c700 1 --2- 172.21.15.99:0/1549506071 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f74b4003fa0 0x7f74c809aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:00.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.383+0000 7f74bffff700 1 --2- 172.21.15.99:0/1549506071 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f74c80ac260 0x7f74c81471d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:00.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.383+0000 7f74c6d9d700 1 -- 172.21.15.99:0/1549506071 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f74c80ac260 msgr2=0x7f74c81471d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:00.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.383+0000 7f74c6d9d700 1 --2- 172.21.15.99:0/1549506071 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f74c80ac260 0x7f74c81471d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:00.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.383+0000 7f74c6d9d700 1 -- 172.21.15.99:0/1549506071 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f74b4003fa0 msgr2=0x7f74c809aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:00.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.383+0000 7f74c6d9d700 1 --2- 172.21.15.99:0/1549506071 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f74b4003fa0 0x7f74c809aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:00.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.383+0000 7f74c6d9d700 1 -- 172.21.15.99:0/1549506071 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c809d600 con 0x7f74c80af7c0 2023-10-08T00:13:00.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.383+0000 7f74c6d9d700 1 --2- 172.21.15.99:0/1549506071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74c80af7c0 0x7f74c80accc0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f74c815a3a0 tx=0x7f74c0002150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:13:00.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.384+0000 7f74be7fc700 1 -- 172.21.15.99:0/1549506071 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c0007b60 con 0x7f74c80af7c0 2023-10-08T00:13:00.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.384+0000 7f74be7fc700 1 -- 172.21.15.99:0/1549506071 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74c000ebc0 con 0x7f74c80af7c0 2023-10-08T00:13:00.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.384+0000 7f74cce70700 1 -- 172.21.15.99:0/1549506071 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74c8147710 con 0x7f74c80af7c0 2023-10-08T00:13:00.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.384+0000 7f74be7fc700 1 -- 172.21.15.99:0/1549506071 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c0013420 con 0x7f74c80af7c0 2023-10-08T00:13:00.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.384+0000 7f74cce70700 1 -- 172.21.15.99:0/1549506071 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74c80ad200 con 0x7f74c80af7c0 2023-10-08T00:13:00.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.385+0000 7f74cce70700 1 -- 172.21.15.99:0/1549506071 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74c80ac630 con 0x7f74c80af7c0 2023-10-08T00:13:00.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.385+0000 7f74be7fc700 1 -- 172.21.15.99:0/1549506071 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f74c00137d0 con 0x7f74c80af7c0 2023-10-08T00:13:00.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.385+0000 7f74be7fc700 1 --2- 172.21.15.99:0/1549506071 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f74a8066f30 0x7f74a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:00.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.385+0000 7f74be7fc700 1 -- 172.21.15.99:0/1549506071 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f74c00849d0 con 0x7f74c80af7c0 2023-10-08T00:13:00.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.385+0000 7f74c659c700 1 --2- 172.21.15.99:0/1549506071 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f74a8066f30 0x7f74a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:00.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.386+0000 7f74c659c700 1 --2- 172.21.15.99:0/1549506071 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f74a8066f30 0x7f74a80693e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f74b8009fd0 tx=0x7f74b8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:13:00.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.389+0000 7f74be7fc700 1 -- 172.21.15.99:0/1549506071 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f74c80ac630 con 0x7f74c80af7c0 2023-10-08T00:13:00.613 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:00.612+0000 7f74cce70700 1 -- 172.21.15.99:0/1549506071 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f74c80ac630 con 0x7f74c80af7c0 2023-10-08T00:13:01.242 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.242+0000 7f74be7fc700 1 -- 172.21.15.99:0/1549506071 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v32) v1 ==== 122+0+0 (secure 0 0 0) 0x7f74c80ac630 con 0x7f74c80af7c0 2023-10-08T00:13:01.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.244+0000 7f74cce70700 1 -- 172.21.15.99:0/1549506071 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f74a8066f30 msgr2=0x7f74a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:01.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.244+0000 7f74cce70700 1 --2- 172.21.15.99:0/1549506071 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f74a8066f30 0x7f74a80693e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f74b8009fd0 tx=0x7f74b8008040 comp rx=0 tx=0).stop 2023-10-08T00:13:01.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.244+0000 7f74cce70700 1 -- 172.21.15.99:0/1549506071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74c80af7c0 msgr2=0x7f74c80accc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:01.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.244+0000 7f74cce70700 1 --2- 172.21.15.99:0/1549506071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74c80af7c0 0x7f74c80accc0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f74c815a3a0 tx=0x7f74c0002150 comp rx=0 tx=0).stop 2023-10-08T00:13:01.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.244+0000 7f74cce70700 1 -- 172.21.15.99:0/1549506071 shutdown_connections 2023-10-08T00:13:01.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.244+0000 7f74cce70700 1 --2- 172.21.15.99:0/1549506071 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f74a8066f30 0x7f74a80693e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:01.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.244+0000 7f74cce70700 1 --2- 172.21.15.99:0/1549506071 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f74b4003fa0 0x7f74c809aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:01.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.244+0000 7f74cce70700 1 --2- 172.21.15.99:0/1549506071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74c80af7c0 0x7f74c80accc0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:01.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.244+0000 7f74cce70700 1 --2- 172.21.15.99:0/1549506071 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f74c80ac260 0x7f74c81471d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:01.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.244+0000 7f74cce70700 1 -- 172.21.15.99:0/1549506071 >> 172.21.15.99:0/1549506071 conn(0x7f74c809a0f0 msgr2=0x7f74c8158740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:13:01.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.244+0000 7f74cce70700 1 -- 172.21.15.99:0/1549506071 shutdown_connections 2023-10-08T00:13:01.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.244+0000 7f74cce70700 1 -- 172.21.15.99:0/1549506071 wait complete. 2023-10-08T00:13:01.258 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:13:01.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.373+0000 7f4f4531b700 1 Processor -- start 2023-10-08T00:13:01.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.373+0000 7f4f4531b700 1 -- start start 2023-10-08T00:13:01.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.374+0000 7f4f4531b700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4f400cdab0 0x7f4f4015d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:01.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.374+0000 7f4f4531b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f4015dda0 0x7f4f4015cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:01.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.374+0000 7f4f4531b700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f400cb7c0 con 0x7f4f400cd6e0 2023-10-08T00:13:01.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.374+0000 7f4f4531b700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f4015e850 con 0x7f4f400cdab0 2023-10-08T00:13:01.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.374+0000 7f4f4531b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f400cc360 con 0x7f4f4015dda0 2023-10-08T00:13:01.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.374+0000 7f4f3ffff700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f4f400cd6e0 0x7f4f400cbc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:47586/0 (socket says 172.21.15.99:47586) 2023-10-08T00:13:01.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.374+0000 7f4f3ffff700 1 -- 172.21.15.99:0/3798522664 learned_addr learned my addr 172.21.15.99:0/3798522664 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:13:01.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.374+0000 7f4f3f7fe700 1 --2- 172.21.15.99:0/3798522664 >> v2:172.21.15.99:3300/0 conn(0x7f4f400cdab0 0x7f4f4015d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:01.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.374+0000 7f4f3effd700 1 -- 172.21.15.99:0/3798522664 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3685078309 0 0) 0x7f4f400cb7c0 con 0x7f4f400cd6e0 2023-10-08T00:13:01.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.374+0000 7f4f3effd700 1 -- 172.21.15.99:0/3798522664 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f28003570 con 0x7f4f400cd6e0 2023-10-08T00:13:01.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.374+0000 7f4f44b1a700 1 --2- 172.21.15.99:0/3798522664 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f4015dda0 0x7f4f4015cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:01.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.374+0000 7f4f3f7fe700 1 -- 172.21.15.99:0/3798522664 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f4015dda0 msgr2=0x7f4f4015cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:13:01.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.374+0000 7f4f3f7fe700 1 --2- 172.21.15.99:0/3798522664 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f4015dda0 0x7f4f4015cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:01.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.374+0000 7f4f3f7fe700 1 -- 172.21.15.99:0/3798522664 >> v1:172.21.15.99:6789/0 conn(0x7f4f400cd6e0 legacy=0x7f4f400cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:01.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.374+0000 7f4f3f7fe700 1 -- 172.21.15.99:0/3798522664 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f4015e290 con 0x7f4f400cdab0 2023-10-08T00:13:01.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.375+0000 7f4f3f7fe700 1 --2- 172.21.15.99:0/3798522664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f400cdab0 0x7f4f4015d860 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f4f340098e0 tx=0x7f4f340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=293e5e2f5639f430 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:13:01.378 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.375+0000 7f4f3effd700 1 -- 172.21.15.99:0/3798522664 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3400b640 con 0x7f4f400cdab0 2023-10-08T00:13:01.378 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.375+0000 7f4f3effd700 1 -- 172.21.15.99:0/3798522664 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f3400b7a0 con 0x7f4f400cdab0 2023-10-08T00:13:01.378 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.375+0000 7f4f3effd700 1 -- 172.21.15.99:0/3798522664 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3400bb40 con 0x7f4f400cdab0 2023-10-08T00:13:01.378 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.375+0000 7f4f4531b700 1 -- 172.21.15.99:0/3798522664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f400cdab0 msgr2=0x7f4f4015d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:01.378 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.375+0000 7f4f4531b700 1 --2- 172.21.15.99:0/3798522664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f400cdab0 0x7f4f4015d860 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f4f340098e0 tx=0x7f4f340091a0 comp rx=0 tx=0).stop 2023-10-08T00:13:01.378 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.375+0000 7f4f4531b700 1 -- 172.21.15.99:0/3798522664 shutdown_connections 2023-10-08T00:13:01.378 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.375+0000 7f4f4531b700 1 --2- 172.21.15.99:0/3798522664 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f4015dda0 0x7f4f4015cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:01.378 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.375+0000 7f4f4531b700 1 --2- 172.21.15.99:0/3798522664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f400cdab0 0x7f4f4015d860 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:01.378 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.375+0000 7f4f4531b700 1 -- 172.21.15.99:0/3798522664 >> 172.21.15.99:0/3798522664 conn(0x7f4f40067660 msgr2=0x7f4f40067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:13:01.378 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.375+0000 7f4f4531b700 1 -- 172.21.15.99:0/3798522664 shutdown_connections 2023-10-08T00:13:01.378 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.375+0000 7f4f4531b700 1 -- 172.21.15.99:0/3798522664 wait complete. 2023-10-08T00:13:01.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.376+0000 7f4f4531b700 1 Processor -- start 2023-10-08T00:13:01.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.377+0000 7f4f4531b700 1 -- start start 2023-10-08T00:13:01.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.377+0000 7f4f4531b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f400cd6e0 0x7f4f4014cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:01.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.377+0000 7f4f4531b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f400cdab0 0x7f4f400b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:01.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.377+0000 7f4f4531b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f4015dda0 0x7f4f401516b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:01.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.377+0000 7f4f4531b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f400b3f50 con 0x7f4f400cd6e0 2023-10-08T00:13:01.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.377+0000 7f4f4531b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4014e490 con 0x7f4f4015dda0 2023-10-08T00:13:01.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.377+0000 7f4f4531b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4014e770 con 0x7f4f400cdab0 2023-10-08T00:13:01.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.377+0000 7f4f3ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f400cd6e0 0x7f4f4014cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:01.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.377+0000 7f4f3ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f400cd6e0 0x7f4f4014cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:32876/0 (socket says 172.21.15.99:32876) 2023-10-08T00:13:01.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.377+0000 7f4f3ffff700 1 -- 172.21.15.99:0/3617770356 learned_addr learned my addr 172.21.15.99:0/3617770356 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:13:01.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.378+0000 7f4f3f7fe700 1 --2- 172.21.15.99:0/3617770356 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f400cdab0 0x7f4f400b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:01.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.378+0000 7f4f3ffff700 1 -- 172.21.15.99:0/3617770356 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f400cdab0 msgr2=0x7f4f400b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:01.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.378+0000 7f4f3ffff700 1 --2- 172.21.15.99:0/3617770356 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f400cdab0 0x7f4f400b3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:01.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.378+0000 7f4f3ffff700 1 -- 172.21.15.99:0/3617770356 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f4015dda0 msgr2=0x7f4f401516b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:01.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.378+0000 7f4f44b1a700 1 --2- 172.21.15.99:0/3617770356 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f4015dda0 0x7f4f401516b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:01.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.378+0000 7f4f3ffff700 1 --2- 172.21.15.99:0/3617770356 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f4015dda0 0x7f4f401516b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:01.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.378+0000 7f4f3ffff700 1 -- 172.21.15.99:0/3617770356 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f34009420 con 0x7f4f400cd6e0 2023-10-08T00:13:01.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.378+0000 7f4f3ffff700 1 --2- 172.21.15.99:0/3617770356 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f400cd6e0 0x7f4f4014cd90 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f4f3000c590 tx=0x7f4f30007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:13:01.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.378+0000 7f4f27fff700 1 -- 172.21.15.99:0/3617770356 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f30005b30 con 0x7f4f400cd6e0 2023-10-08T00:13:01.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.378+0000 7f4f27fff700 1 -- 172.21.15.99:0/3617770356 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f30005cb0 con 0x7f4f400cd6e0 2023-10-08T00:13:01.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.378+0000 7f4f4531b700 1 -- 172.21.15.99:0/3617770356 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f4014ea50 con 0x7f4f400cd6e0 2023-10-08T00:13:01.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.378+0000 7f4f27fff700 1 -- 172.21.15.99:0/3617770356 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f30007320 con 0x7f4f400cd6e0 2023-10-08T00:13:01.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.378+0000 7f4f4531b700 1 -- 172.21.15.99:0/3617770356 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f4014f010 con 0x7f4f400cd6e0 2023-10-08T00:13:01.384 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.379+0000 7f4f4531b700 1 -- 172.21.15.99:0/3617770356 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f40089280 con 0x7f4f400cd6e0 2023-10-08T00:13:01.384 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.379+0000 7f4f27fff700 1 -- 172.21.15.99:0/3617770356 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4f3001b3b0 con 0x7f4f400cd6e0 2023-10-08T00:13:01.384 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.380+0000 7f4f27fff700 1 --2- 172.21.15.99:0/3617770356 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4f2c066f30 0x7f4f2c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:01.384 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.380+0000 7f4f27fff700 1 -- 172.21.15.99:0/3617770356 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4f30083b60 con 0x7f4f400cd6e0 2023-10-08T00:13:01.384 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.380+0000 7f4f3f7fe700 1 --2- 172.21.15.99:0/3617770356 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4f2c066f30 0x7f4f2c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:01.384 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.380+0000 7f4f3f7fe700 1 --2- 172.21.15.99:0/3617770356 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4f2c066f30 0x7f4f2c0693e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4f40158280 tx=0x7f4f34017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:13:01.384 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.384+0000 7f4f27fff700 1 -- 172.21.15.99:0/3617770356 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4f30050dd0 con 0x7f4f400cd6e0 2023-10-08T00:13:01.609 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.608+0000 7f4f4531b700 1 -- 172.21.15.99:0/3617770356 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4f40150f30 con 0x7f4f400cd6e0 2023-10-08T00:13:01.609 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.609+0000 7f4f27fff700 1 -- 172.21.15.99:0/3617770356 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4600 (secure 0 0 0) 0x7f4f30017020 con 0x7f4f400cd6e0 2023-10-08T00:13:01.610 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:13:01.610 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5087,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/862112794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":862112794},{"type":"v1","addr":"172.21.15.191:6837","nonce":862112794}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5673,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/2694493794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2694493794},{"type":"v1","addr":"172.21.15.99:6834","nonce":2694493794}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1906990531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":1906990531},{"type":"v1","addr":"172.21.15.99:6835","nonce":1906990531}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:12:57.581246+0000","modified":"2023-10-08T00:13:01.240449+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5688},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5688":{"gid":5688,"name":"d","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6836/1779510877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":1779510877},{"type":"v1","addr":"172.21.15.191:6836","nonce":1779510877}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-08T00:13:01.611 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.611+0000 7f4f4531b700 1 -- 172.21.15.99:0/3617770356 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4f2c066f30 msgr2=0x7f4f2c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:01.612 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.611+0000 7f4f4531b700 1 --2- 172.21.15.99:0/3617770356 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4f2c066f30 0x7f4f2c0693e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4f40158280 tx=0x7f4f34017000 comp rx=0 tx=0).stop 2023-10-08T00:13:01.612 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.611+0000 7f4f4531b700 1 -- 172.21.15.99:0/3617770356 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f400cd6e0 msgr2=0x7f4f4014cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:01.612 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.611+0000 7f4f4531b700 1 --2- 172.21.15.99:0/3617770356 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f400cd6e0 0x7f4f4014cd90 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f4f3000c590 tx=0x7f4f30007e00 comp rx=0 tx=0).stop 2023-10-08T00:13:01.612 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.611+0000 7f4f4531b700 1 -- 172.21.15.99:0/3617770356 shutdown_connections 2023-10-08T00:13:01.612 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.611+0000 7f4f4531b700 1 --2- 172.21.15.99:0/3617770356 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4f2c066f30 0x7f4f2c0693e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:01.612 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.611+0000 7f4f4531b700 1 --2- 172.21.15.99:0/3617770356 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f4015dda0 0x7f4f401516b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:01.612 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.611+0000 7f4f4531b700 1 --2- 172.21.15.99:0/3617770356 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f400cd6e0 0x7f4f4014cd90 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:01.612 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.611+0000 7f4f4531b700 1 --2- 172.21.15.99:0/3617770356 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f400cdab0 0x7f4f400b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:01.613 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.611+0000 7f4f4531b700 1 -- 172.21.15.99:0/3617770356 >> 172.21.15.99:0/3617770356 conn(0x7f4f40067660 msgr2=0x7f4f400cfbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:13:01.613 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.611+0000 7f4f4531b700 1 -- 172.21.15.99:0/3617770356 shutdown_connections 2023-10-08T00:13:01.613 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.611+0000 7f4f4531b700 1 -- 172.21.15.99:0/3617770356 wait complete. 2023-10-08T00:13:01.613 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 32 2023-10-08T00:13:01.623 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:13:01.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.736+0000 7fc572c3d700 1 Processor -- start 2023-10-08T00:13:01.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.736+0000 7fc572c3d700 1 -- start start 2023-10-08T00:13:01.740 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.737+0000 7fc572c3d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc56c1577d0 0x7fc56c0cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:01.740 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.737+0000 7fc572c3d700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc56c157ba0 0x7fc56c06a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:01.740 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.737+0000 7fc572c3d700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc56c15c360 con 0x7fc56c15b0d0 2023-10-08T00:13:01.740 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.737+0000 7fc572c3d700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc56c15b790 con 0x7fc56c157ba0 2023-10-08T00:13:01.740 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.737+0000 7fc572c3d700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc56c15d800 con 0x7fc56c1577d0 2023-10-08T00:13:01.740 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.737+0000 7fc563fff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc56c157ba0 0x7fc56c06a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:01.740 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.737+0000 7fc5711da700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fc56c15b0d0 0x7fc56c15da90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:47596/0 (socket says 172.21.15.99:47596) 2023-10-08T00:13:01.740 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.737+0000 7fc5711da700 1 -- 172.21.15.99:0/400499994 learned_addr learned my addr 172.21.15.99:0/400499994 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:13:01.740 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.738+0000 7fc5709d9700 1 --2- 172.21.15.99:0/400499994 >> v2:172.21.15.191:3300/0 conn(0x7fc56c1577d0 0x7fc56c0cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:01.740 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.738+0000 7fc563fff700 1 -- 172.21.15.99:0/400499994 >> v2:172.21.15.191:3300/0 conn(0x7fc56c1577d0 msgr2=0x7fc56c0cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:13:01.741 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.738+0000 7fc563fff700 1 --2- 172.21.15.99:0/400499994 >> v2:172.21.15.191:3300/0 conn(0x7fc56c1577d0 0x7fc56c0cd770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:01.741 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.738+0000 7fc563fff700 1 -- 172.21.15.99:0/400499994 >> v1:172.21.15.99:6789/0 conn(0x7fc56c15b0d0 legacy=0x7fc56c15da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:01.741 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.738+0000 7fc5637fe700 1 -- 172.21.15.99:0/400499994 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1874080572 0 0) 0x7fc56c15c360 con 0x7fc56c15b0d0 2023-10-08T00:13:01.741 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.738+0000 7fc563fff700 1 -- 172.21.15.99:0/400499994 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc56c15ba30 con 0x7fc56c157ba0 2023-10-08T00:13:01.741 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.738+0000 7fc563fff700 1 --2- 172.21.15.99:0/400499994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc56c157ba0 0x7fc56c06a520 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc56c0b6dc0 tx=0x7fc55c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e2ed93532b2912c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:13:01.741 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.738+0000 7fc5637fe700 1 -- 172.21.15.99:0/400499994 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55c00b6b0 con 0x7fc56c157ba0 2023-10-08T00:13:01.741 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.738+0000 7fc5637fe700 1 -- 172.21.15.99:0/400499994 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc55c00b810 con 0x7fc56c157ba0 2023-10-08T00:13:01.741 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.738+0000 7fc5637fe700 1 -- 172.21.15.99:0/400499994 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55c00bbb0 con 0x7fc56c157ba0 2023-10-08T00:13:01.741 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.738+0000 7fc572c3d700 1 -- 172.21.15.99:0/400499994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc56c157ba0 msgr2=0x7fc56c06a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:01.741 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.738+0000 7fc572c3d700 1 --2- 172.21.15.99:0/400499994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc56c157ba0 0x7fc56c06a520 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc56c0b6dc0 tx=0x7fc55c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:13:01.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.738+0000 7fc572c3d700 1 -- 172.21.15.99:0/400499994 shutdown_connections 2023-10-08T00:13:01.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.738+0000 7fc572c3d700 1 --2- 172.21.15.99:0/400499994 >> v2:172.21.15.191:3300/0 conn(0x7fc56c1577d0 0x7fc56c0cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:01.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.738+0000 7fc572c3d700 1 --2- 172.21.15.99:0/400499994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc56c157ba0 0x7fc56c06a520 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:01.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.738+0000 7fc572c3d700 1 -- 172.21.15.99:0/400499994 >> 172.21.15.99:0/400499994 conn(0x7fc56c068ce0 msgr2=0x7fc56c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:13:01.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.739+0000 7fc572c3d700 1 -- 172.21.15.99:0/400499994 shutdown_connections 2023-10-08T00:13:01.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.739+0000 7fc572c3d700 1 -- 172.21.15.99:0/400499994 wait complete. 2023-10-08T00:13:01.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.740+0000 7fc572c3d700 1 Processor -- start 2023-10-08T00:13:01.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.740+0000 7fc572c3d700 1 -- start start 2023-10-08T00:13:01.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.741+0000 7fc572c3d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc56c1577d0 0x7fc56c13c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:01.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.741+0000 7fc572c3d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc56c157ba0 0x7fc56c14f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:01.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.741+0000 7fc572c3d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc56c15b0d0 0x7fc56c14f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:01.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.741+0000 7fc572c3d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc56c14c260 con 0x7fc56c15b0d0 2023-10-08T00:13:01.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.741+0000 7fc572c3d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc56c0d0cb0 con 0x7fc56c157ba0 2023-10-08T00:13:01.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.741+0000 7fc572c3d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc56c14e3b0 con 0x7fc56c1577d0 2023-10-08T00:13:01.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.741+0000 7fc5711da700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc56c15b0d0 0x7fc56c14f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:01.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.741+0000 7fc5711da700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc56c15b0d0 0x7fc56c14f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:32896/0 (socket says 172.21.15.99:32896) 2023-10-08T00:13:01.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.741+0000 7fc5711da700 1 -- 172.21.15.99:0/2635933669 learned_addr learned my addr 172.21.15.99:0/2635933669 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:13:01.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.741+0000 7fc563fff700 1 --2- 172.21.15.99:0/2635933669 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc56c157ba0 0x7fc56c14f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:01.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.741+0000 7fc5709d9700 1 --2- 172.21.15.99:0/2635933669 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc56c1577d0 0x7fc56c13c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:01.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.741+0000 7fc5711da700 1 -- 172.21.15.99:0/2635933669 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc56c1577d0 msgr2=0x7fc56c13c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:01.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.741+0000 7fc5711da700 1 --2- 172.21.15.99:0/2635933669 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc56c1577d0 0x7fc56c13c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:01.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.741+0000 7fc5711da700 1 -- 172.21.15.99:0/2635933669 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc56c157ba0 msgr2=0x7fc56c14f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:01.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.741+0000 7fc5711da700 1 --2- 172.21.15.99:0/2635933669 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc56c157ba0 0x7fc56c14f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:01.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.741+0000 7fc5711da700 1 -- 172.21.15.99:0/2635933669 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc55c009420 con 0x7fc56c15b0d0 2023-10-08T00:13:01.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.741+0000 7fc5709d9700 1 --2- 172.21.15.99:0/2635933669 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc56c1577d0 0x7fc56c13c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:13:01.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.741+0000 7fc5711da700 1 --2- 172.21.15.99:0/2635933669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc56c15b0d0 0x7fc56c14f610 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fc56400c560 tx=0x7fc564007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:13:01.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.741+0000 7fc560ff9700 1 -- 172.21.15.99:0/2635933669 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc564005b30 con 0x7fc56c15b0d0 2023-10-08T00:13:01.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.741+0000 7fc560ff9700 1 -- 172.21.15.99:0/2635933669 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc564005c90 con 0x7fc56c15b0d0 2023-10-08T00:13:01.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.741+0000 7fc572c3d700 1 -- 172.21.15.99:0/2635933669 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc56c14e630 con 0x7fc56c15b0d0 2023-10-08T00:13:01.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.741+0000 7fc560ff9700 1 -- 172.21.15.99:0/2635933669 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc564007320 con 0x7fc56c15b0d0 2023-10-08T00:13:01.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.741+0000 7fc572c3d700 1 -- 172.21.15.99:0/2635933669 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc56c14ebb0 con 0x7fc56c15b0d0 2023-10-08T00:13:01.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.742+0000 7fc572c3d700 1 -- 172.21.15.99:0/2635933669 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc56c04c3c0 con 0x7fc56c15b0d0 2023-10-08T00:13:01.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.743+0000 7fc560ff9700 1 -- 172.21.15.99:0/2635933669 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc56401b3b0 con 0x7fc56c15b0d0 2023-10-08T00:13:01.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.743+0000 7fc560ff9700 1 --2- 172.21.15.99:0/2635933669 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc558066f30 0x7fc5580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:01.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.743+0000 7fc560ff9700 1 -- 172.21.15.99:0/2635933669 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc564083a50 con 0x7fc56c15b0d0 2023-10-08T00:13:01.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.743+0000 7fc5709d9700 1 --2- 172.21.15.99:0/2635933669 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc558066f30 0x7fc5580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:01.748 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.744+0000 7fc5709d9700 1 --2- 172.21.15.99:0/2635933669 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc558066f30 0x7fc5580693e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc56800d460 tx=0x7fc5680075e0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:13:01.748 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.747+0000 7fc560ff9700 1 -- 172.21.15.99:0/2635933669 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc564050cc0 con 0x7fc56c15b0d0 2023-10-08T00:13:01.933 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.933+0000 7fc572c3d700 1 -- 172.21.15.99:0/2635933669 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc56c15bd10 con 0x7fc56c15b0d0 2023-10-08T00:13:01.934 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.934+0000 7fc560ff9700 1 -- 172.21.15.99:0/2635933669 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+15350 (secure 0 0 0) 0x7fc564017090 con 0x7fc56c15b0d0 2023-10-08T00:13:01.934 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:13:01.935 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":69,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:12:59.202744+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:12:55.353771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-08T00:12:56.552157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:13:01.937 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.936+0000 7fc572c3d700 1 -- 172.21.15.99:0/2635933669 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc558066f30 msgr2=0x7fc5580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:01.937 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.936+0000 7fc572c3d700 1 --2- 172.21.15.99:0/2635933669 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc558066f30 0x7fc5580693e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc56800d460 tx=0x7fc5680075e0 comp rx=0 tx=0).stop 2023-10-08T00:13:01.937 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.936+0000 7fc572c3d700 1 -- 172.21.15.99:0/2635933669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc56c15b0d0 msgr2=0x7fc56c14f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:01.937 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.936+0000 7fc572c3d700 1 --2- 172.21.15.99:0/2635933669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc56c15b0d0 0x7fc56c14f610 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fc56400c560 tx=0x7fc564007f90 comp rx=0 tx=0).stop 2023-10-08T00:13:01.937 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.936+0000 7fc572c3d700 1 -- 172.21.15.99:0/2635933669 shutdown_connections 2023-10-08T00:13:01.937 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.936+0000 7fc572c3d700 1 --2- 172.21.15.99:0/2635933669 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc558066f30 0x7fc5580693e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:01.937 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.936+0000 7fc572c3d700 1 --2- 172.21.15.99:0/2635933669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc56c15b0d0 0x7fc56c14f610 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:01.937 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.936+0000 7fc572c3d700 1 --2- 172.21.15.99:0/2635933669 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc56c1577d0 0x7fc56c13c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:01.938 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.936+0000 7fc572c3d700 1 --2- 172.21.15.99:0/2635933669 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc56c157ba0 0x7fc56c14f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:01.938 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.936+0000 7fc572c3d700 1 -- 172.21.15.99:0/2635933669 >> 172.21.15.99:0/2635933669 conn(0x7fc56c068ce0 msgr2=0x7fc56c15f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:13:01.938 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.936+0000 7fc572c3d700 1 -- 172.21.15.99:0/2635933669 shutdown_connections 2023-10-08T00:13:01.938 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:01.936+0000 7fc572c3d700 1 -- 172.21.15.99:0/2635933669 wait complete. 2023-10-08T00:13:01.949 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T00:13:01.949 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T00:13:02.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.062+0000 7f0aa95f7700 1 Processor -- start 2023-10-08T00:13:02.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.062+0000 7f0aa95f7700 1 -- start start 2023-10-08T00:13:02.064 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.063+0000 7f0aa95f7700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0aa40ae550 0x7f0aa4161be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:02.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.063+0000 7f0aa95f7700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aa4068480 con 0x7f0aa4068880 2023-10-08T00:13:02.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.063+0000 7f0aa95f7700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aa40ad060 con 0x7f0aa4156b80 2023-10-08T00:13:02.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.063+0000 7f0aa95f7700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0aa406f030 con 0x7f0aa40ae550 2023-10-08T00:13:02.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.063+0000 7f0aa37fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0aa40ae550 0x7f0aa4161be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:02.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.063+0000 7f0a9a7fc700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f0aa4068880 0x7f0aa4067d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:47612/0 (socket says 172.21.15.99:47612) 2023-10-08T00:13:02.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.063+0000 7f0a9a7fc700 1 -- 172.21.15.99:0/3854311418 learned_addr learned my addr 172.21.15.99:0/3854311418 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:13:02.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.064+0000 7f0aa37fe700 1 -- 172.21.15.99:0/3854311418 >> v1:172.21.15.191:6789/0 conn(0x7f0aa4156b80 legacy=0x7f0aa4067660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:02.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.064+0000 7f0aa37fe700 1 -- 172.21.15.99:0/3854311418 >> v1:172.21.15.99:6789/0 conn(0x7f0aa4068880 legacy=0x7f0aa4067d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:02.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.064+0000 7f0aa37fe700 1 -- 172.21.15.99:0/3854311418 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa406f310 con 0x7f0aa40ae550 2023-10-08T00:13:02.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.064+0000 7f0aa37fe700 1 --2- 172.21.15.99:0/3854311418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0aa40ae550 0x7f0aa4161be0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c009e60 tx=0x7f0a9c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f172fdf0a33342d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:13:02.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.064+0000 7f0aa27fc700 1 -- 172.21.15.99:0/3854311418 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c00e070 con 0x7f0aa40ae550 2023-10-08T00:13:02.066 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.064+0000 7f0aa27fc700 1 -- 172.21.15.99:0/3854311418 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a9c00bc30 con 0x7f0aa40ae550 2023-10-08T00:13:02.066 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.064+0000 7f0aa27fc700 1 -- 172.21.15.99:0/3854311418 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c012430 con 0x7f0aa40ae550 2023-10-08T00:13:02.068 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.064+0000 7f0aa95f7700 1 -- 172.21.15.99:0/3854311418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0aa40ae550 msgr2=0x7f0aa4161be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:02.068 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.064+0000 7f0aa95f7700 1 --2- 172.21.15.99:0/3854311418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0aa40ae550 0x7f0aa4161be0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c009e60 tx=0x7f0a9c009780 comp rx=0 tx=0).stop 2023-10-08T00:13:02.068 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.064+0000 7f0aa95f7700 1 -- 172.21.15.99:0/3854311418 shutdown_connections 2023-10-08T00:13:02.068 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.064+0000 7f0aa95f7700 1 --2- 172.21.15.99:0/3854311418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0aa40ae550 0x7f0aa4161be0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:02.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.065+0000 7f0aa95f7700 1 -- 172.21.15.99:0/3854311418 >> 172.21.15.99:0/3854311418 conn(0x7f0aa406ad40 msgr2=0x7f0aa40cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:13:02.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.065+0000 7f0aa95f7700 1 -- 172.21.15.99:0/3854311418 shutdown_connections 2023-10-08T00:13:02.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.065+0000 7f0aa95f7700 1 -- 172.21.15.99:0/3854311418 wait complete. 2023-10-08T00:13:02.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.066+0000 7f0aa95f7700 1 Processor -- start 2023-10-08T00:13:02.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.066+0000 7f0aa95f7700 1 -- start start 2023-10-08T00:13:02.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.066+0000 7f0aa95f7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0aa4068880 0x7f0aa4151fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:02.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.066+0000 7f0aa95f7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0aa40ae550 0x7f0aa4159200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:02.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.066+0000 7f0aa95f7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0aa4156b80 0x7f0aa4159740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:02.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.067+0000 7f0aa95f7700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa4159dc0 con 0x7f0aa4156b80 2023-10-08T00:13:02.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.067+0000 7f0aa95f7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa4157dd0 con 0x7f0aa40ae550 2023-10-08T00:13:02.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.067+0000 7f0aa95f7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa40a8410 con 0x7f0aa4068880 2023-10-08T00:13:02.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.067+0000 7f0aa37fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0aa4156b80 0x7f0aa4159740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:02.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.067+0000 7f0aa37fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0aa4156b80 0x7f0aa4159740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:32912/0 (socket says 172.21.15.99:32912) 2023-10-08T00:13:02.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.067+0000 7f0aa37fe700 1 -- 172.21.15.99:0/4109061858 learned_addr learned my addr 172.21.15.99:0/4109061858 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:13:02.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.067+0000 7f0a9a7fc700 1 --2- 172.21.15.99:0/4109061858 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0aa40ae550 0x7f0aa4159200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:02.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.067+0000 7f0aa37fe700 1 -- 172.21.15.99:0/4109061858 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0aa4068880 msgr2=0x7f0aa4151fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:02.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.067+0000 7f0aa37fe700 1 --2- 172.21.15.99:0/4109061858 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0aa4068880 0x7f0aa4151fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:02.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.067+0000 7f0aa37fe700 1 -- 172.21.15.99:0/4109061858 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0aa40ae550 msgr2=0x7f0aa4159200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:02.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.067+0000 7f0aa37fe700 1 --2- 172.21.15.99:0/4109061858 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0aa40ae550 0x7f0aa4159200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:02.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.067+0000 7f0aa37fe700 1 -- 172.21.15.99:0/4109061858 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a9c009a20 con 0x7f0aa4156b80 2023-10-08T00:13:02.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.067+0000 7f0a9a7fc700 1 --2- 172.21.15.99:0/4109061858 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0aa40ae550 0x7f0aa4159200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:13:02.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.067+0000 7f0aa37fe700 1 --2- 172.21.15.99:0/4109061858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0aa4156b80 0x7f0aa4159740 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c013000 tx=0x7f0a9c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:13:02.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.067+0000 7f0a9b7fe700 1 -- 172.21.15.99:0/4109061858 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c01e630 con 0x7f0aa4156b80 2023-10-08T00:13:02.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.067+0000 7f0a9b7fe700 1 -- 172.21.15.99:0/4109061858 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a9c01e790 con 0x7f0aa4156b80 2023-10-08T00:13:02.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.067+0000 7f0aa95f7700 1 -- 172.21.15.99:0/4109061858 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aa40a7ae0 con 0x7f0aa4156b80 2023-10-08T00:13:02.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.067+0000 7f0a9b7fe700 1 -- 172.21.15.99:0/4109061858 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c01eb50 con 0x7f0aa4156b80 2023-10-08T00:13:02.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.067+0000 7f0aa95f7700 1 -- 172.21.15.99:0/4109061858 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aa40a80a0 con 0x7f0aa4156b80 2023-10-08T00:13:02.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.068+0000 7f0aa95f7700 1 -- 172.21.15.99:0/4109061858 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0aa404c3c0 con 0x7f0aa4156b80 2023-10-08T00:13:02.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.069+0000 7f0a9b7fe700 1 -- 172.21.15.99:0/4109061858 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0a9c024070 con 0x7f0aa4156b80 2023-10-08T00:13:02.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.069+0000 7f0a9b7fe700 1 --2- 172.21.15.99:0/4109061858 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0a84066f30 0x7f0a840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:02.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.069+0000 7f0a9b7fe700 1 -- 172.21.15.99:0/4109061858 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0a9c086dd0 con 0x7f0aa4156b80 2023-10-08T00:13:02.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.069+0000 7f0aa2ffd700 1 --2- 172.21.15.99:0/4109061858 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0a84066f30 0x7f0a840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:02.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.070+0000 7f0aa2ffd700 1 --2- 172.21.15.99:0/4109061858 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0a84066f30 0x7f0a840693e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0a94001660 tx=0x7f0a94007b00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:13:02.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.073+0000 7f0a9b7fe700 1 -- 172.21.15.99:0/4109061858 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0a9c054040 con 0x7f0aa4156b80 2023-10-08T00:13:02.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.250+0000 7f0aa95f7700 1 -- 172.21.15.99:0/4109061858 --> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0aa40a8550 con 0x7f0a84066f30 2023-10-08T00:13:02.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.258+0000 7f0a9b7fe700 1 -- 172.21.15.99:0/4109061858 <== mgr.4099 v2:172.21.15.191:6832/85985 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+281850 (secure 0 0 0) 0x7f0a94014410 con 0x7f0a84066f30 2023-10-08T00:13:02.262 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:13:02.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.261+0000 7f0aa95f7700 1 -- 172.21.15.99:0/4109061858 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0a84066f30 msgr2=0x7f0a840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:02.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.261+0000 7f0aa95f7700 1 --2- 172.21.15.99:0/4109061858 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0a84066f30 0x7f0a840693e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0a94001660 tx=0x7f0a94007b00 comp rx=0 tx=0).stop 2023-10-08T00:13:02.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.261+0000 7f0aa95f7700 1 -- 172.21.15.99:0/4109061858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0aa4156b80 msgr2=0x7f0aa4159740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:02.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.261+0000 7f0aa95f7700 1 --2- 172.21.15.99:0/4109061858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0aa4156b80 0x7f0aa4159740 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c013000 tx=0x7f0a9c019000 comp rx=0 tx=0).stop 2023-10-08T00:13:02.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.261+0000 7f0aa95f7700 1 -- 172.21.15.99:0/4109061858 shutdown_connections 2023-10-08T00:13:02.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.261+0000 7f0aa95f7700 1 --2- 172.21.15.99:0/4109061858 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0a84066f30 0x7f0a840693e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:02.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.261+0000 7f0aa95f7700 1 --2- 172.21.15.99:0/4109061858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0aa4156b80 0x7f0aa4159740 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:02.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.261+0000 7f0aa95f7700 1 --2- 172.21.15.99:0/4109061858 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0aa4068880 0x7f0aa4151fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:02.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.261+0000 7f0aa95f7700 1 --2- 172.21.15.99:0/4109061858 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0aa40ae550 0x7f0aa4159200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:02.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.261+0000 7f0aa95f7700 1 -- 172.21.15.99:0/4109061858 >> 172.21.15.99:0/4109061858 conn(0x7f0aa406ad40 msgr2=0x7f0aa4155900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:13:02.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.261+0000 7f0aa95f7700 1 -- 172.21.15.99:0/4109061858 shutdown_connections 2023-10-08T00:13:02.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.261+0000 7f0aa95f7700 1 -- 172.21.15.99:0/4109061858 wait complete. 2023-10-08T00:13:02.266 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2023-10-08T00:13:02.274 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":129,"stamp":"2023-10-08T00:13:02.053575+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":3830,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1558669,"statfs":{"total":1600000000,"available":1596077542,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.858362"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.207886+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:42:53.170735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206623+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:29:57.147801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206756+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:15:27.299505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.204518+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:17:47.918131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.208328+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:29:07.443694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.207050+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:19:19.113722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.204971+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:18:06.799190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.214505+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:51:20.513160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.204229+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:57:53.151755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.207215+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:03:56.099847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.207550+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:03:56.099847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.203197+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:04:57.344499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206586+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:36:53.312606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206510+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:56:37.046732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.205079+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:39:24.599431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.207493+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:39:24.599431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206258+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:00:49.167372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206438+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:33:06.564584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.205454+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:47:44.758644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.208478+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:22:06.125225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206453+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:29:07.443694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.209033+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:42:53.170735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.209265+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:56:15.157209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.203882+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:50:17.972931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206009+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:36:24.506117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206105+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:42:53.170735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206249+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:43:06.027907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.213400+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:57:11.917467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.208004+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:17:37.423308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.205536+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:28:33.161628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206770+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:27.657370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.209403+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:29:07.443694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.205320+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:33:16.516139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.213303+0000","last_change":"2023-10-08T00:12:57.213303+0000","last_active":"2023-10-08T00:12:57.213303+0000","last_peered":"2023-10-08T00:12:57.213303+0000","last_clean":"2023-10-08T00:12:57.213303+0000","last_became_active":"2023-10-08T00:12:57.213092+0000","last_became_peered":"2023-10-08T00:12:57.213092+0000","last_unstale":"2023-10-08T00:12:57.213303+0000","last_undegraded":"2023-10-08T00:12:57.213303+0000","last_fullsized":"2023-10-08T00:12:57.213303+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:40:33.741781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.205824+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:51:47.982127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.212771+0000","last_change":"2023-10-08T00:12:57.212771+0000","last_active":"2023-10-08T00:12:57.212771+0000","last_peered":"2023-10-08T00:12:57.212771+0000","last_clean":"2023-10-08T00:12:57.212771+0000","last_became_active":"2023-10-08T00:12:57.212479+0000","last_became_peered":"2023-10-08T00:12:57.212479+0000","last_unstale":"2023-10-08T00:12:57.212771+0000","last_undegraded":"2023-10-08T00:12:57.212771+0000","last_fullsized":"2023-10-08T00:12:57.212771+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:51:18.660991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.207674+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:13:42.175367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.209653+0000","last_change":"2023-10-08T00:12:57.209653+0000","last_active":"2023-10-08T00:12:57.209653+0000","last_peered":"2023-10-08T00:12:57.209653+0000","last_clean":"2023-10-08T00:12:57.209653+0000","last_became_active":"2023-10-08T00:12:57.209408+0000","last_became_peered":"2023-10-08T00:12:57.209408+0000","last_unstale":"2023-10-08T00:12:57.209653+0000","last_undegraded":"2023-10-08T00:12:57.209653+0000","last_fullsized":"2023-10-08T00:12:57.209653+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:26.648145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.207619+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:22:06.125225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.207927+0000","last_change":"2023-10-08T00:12:57.207927+0000","last_active":"2023-10-08T00:12:57.207927+0000","last_peered":"2023-10-08T00:12:57.207927+0000","last_clean":"2023-10-08T00:12:57.207927+0000","last_became_active":"2023-10-08T00:12:57.207442+0000","last_became_peered":"2023-10-08T00:12:57.207442+0000","last_unstale":"2023-10-08T00:12:57.207927+0000","last_undegraded":"2023-10-08T00:12:57.207927+0000","last_fullsized":"2023-10-08T00:12:57.207927+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:57:10.908242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.204176+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:24:53.646854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.212292+0000","last_change":"2023-10-08T00:12:57.212292+0000","last_active":"2023-10-08T00:12:57.212292+0000","last_peered":"2023-10-08T00:12:57.212292+0000","last_clean":"2023-10-08T00:12:57.212292+0000","last_became_active":"2023-10-08T00:12:57.211867+0000","last_became_peered":"2023-10-08T00:12:57.211867+0000","last_unstale":"2023-10-08T00:12:57.212292+0000","last_undegraded":"2023-10-08T00:12:57.212292+0000","last_fullsized":"2023-10-08T00:12:57.212292+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:36:52.303382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.208400+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:39:24.599431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.211573+0000","last_change":"2023-10-08T00:12:57.211573+0000","last_active":"2023-10-08T00:12:57.211573+0000","last_peered":"2023-10-08T00:12:57.211573+0000","last_clean":"2023-10-08T00:12:57.211573+0000","last_became_active":"2023-10-08T00:12:57.211202+0000","last_became_peered":"2023-10-08T00:12:57.211202+0000","last_unstale":"2023-10-08T00:12:57.211573+0000","last_undegraded":"2023-10-08T00:12:57.211573+0000","last_fullsized":"2023-10-08T00:12:57.211573+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:29:56.138577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.208004+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:47:28.589841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.205376+0000","last_change":"2023-10-08T00:12:57.205376+0000","last_active":"2023-10-08T00:12:57.205376+0000","last_peered":"2023-10-08T00:12:57.205376+0000","last_clean":"2023-10-08T00:12:57.205376+0000","last_became_active":"2023-10-08T00:12:57.205158+0000","last_became_peered":"2023-10-08T00:12:57.205158+0000","last_unstale":"2023-10-08T00:12:57.205376+0000","last_undegraded":"2023-10-08T00:12:57.205376+0000","last_fullsized":"2023-10-08T00:12:57.205376+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:15:43.196718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.204806+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:22:04.500698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.214270+0000","last_change":"2023-10-08T00:12:57.214270+0000","last_active":"2023-10-08T00:12:57.214270+0000","last_peered":"2023-10-08T00:12:57.214270+0000","last_clean":"2023-10-08T00:12:57.214270+0000","last_became_active":"2023-10-08T00:12:57.214082+0000","last_became_peered":"2023-10-08T00:12:57.214082+0000","last_unstale":"2023-10-08T00:12:57.214270+0000","last_undegraded":"2023-10-08T00:12:57.214270+0000","last_fullsized":"2023-10-08T00:12:57.214270+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:40:33.741781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.204948+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:03:56.099847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.209199+0000","last_change":"2023-10-08T00:12:57.209199+0000","last_active":"2023-10-08T00:12:57.209199+0000","last_peered":"2023-10-08T00:12:57.209199+0000","last_clean":"2023-10-08T00:12:57.209199+0000","last_became_active":"2023-10-08T00:12:57.208965+0000","last_became_peered":"2023-10-08T00:12:57.208965+0000","last_unstale":"2023-10-08T00:12:57.209199+0000","last_undegraded":"2023-10-08T00:12:57.209199+0000","last_fullsized":"2023-10-08T00:12:57.209199+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:02:39.029498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.207557+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:39:49.309778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.214795+0000","last_change":"2023-10-08T00:12:57.214795+0000","last_active":"2023-10-08T00:12:57.214795+0000","last_peered":"2023-10-08T00:12:57.214795+0000","last_clean":"2023-10-08T00:12:57.214795+0000","last_became_active":"2023-10-08T00:12:57.214269+0000","last_became_peered":"2023-10-08T00:12:57.214269+0000","last_unstale":"2023-10-08T00:12:57.214795+0000","last_undegraded":"2023-10-08T00:12:57.214795+0000","last_fullsized":"2023-10-08T00:12:57.214795+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:28:52.453441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206370+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:52:28.504130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.210087+0000","last_change":"2023-10-08T00:12:57.210087+0000","last_active":"2023-10-08T00:12:57.210087+0000","last_peered":"2023-10-08T00:12:57.210087+0000","last_clean":"2023-10-08T00:12:57.210087+0000","last_became_active":"2023-10-08T00:12:57.209574+0000","last_became_peered":"2023-10-08T00:12:57.209574+0000","last_unstale":"2023-10-08T00:12:57.210087+0000","last_undegraded":"2023-10-08T00:12:57.210087+0000","last_fullsized":"2023-10-08T00:12:57.210087+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:51:19.503936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.208930+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:44:29.429854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.209678+0000","last_change":"2023-10-08T00:12:57.209678+0000","last_active":"2023-10-08T00:12:57.209678+0000","last_peered":"2023-10-08T00:12:57.209678+0000","last_clean":"2023-10-08T00:12:57.209678+0000","last_became_active":"2023-10-08T00:12:57.209454+0000","last_became_peered":"2023-10-08T00:12:57.209454+0000","last_unstale":"2023-10-08T00:12:57.209678+0000","last_undegraded":"2023-10-08T00:12:57.209678+0000","last_fullsized":"2023-10-08T00:12:57.209678+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:51:46.972903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.203919+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:17:10.296378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.215576+0000","last_change":"2023-10-08T00:12:57.215576+0000","last_active":"2023-10-08T00:12:57.215576+0000","last_peered":"2023-10-08T00:12:57.215576+0000","last_clean":"2023-10-08T00:12:57.215576+0000","last_became_active":"2023-10-08T00:12:57.215200+0000","last_became_peered":"2023-10-08T00:12:57.215200+0000","last_unstale":"2023-10-08T00:12:57.215576+0000","last_undegraded":"2023-10-08T00:12:57.215576+0000","last_fullsized":"2023-10-08T00:12:57.215576+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:11:40.080191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.207660+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:50:14.863709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.208075+0000","last_change":"2023-10-08T00:12:57.208075+0000","last_active":"2023-10-08T00:12:57.208075+0000","last_peered":"2023-10-08T00:12:57.208075+0000","last_clean":"2023-10-08T00:12:57.208075+0000","last_became_active":"2023-10-08T00:12:57.207720+0000","last_became_peered":"2023-10-08T00:12:57.207720+0000","last_unstale":"2023-10-08T00:12:57.208075+0000","last_undegraded":"2023-10-08T00:12:57.208075+0000","last_fullsized":"2023-10-08T00:12:57.208075+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:23:41.947350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.203096+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:43:33.796004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.204277+0000","last_change":"2023-10-08T00:12:57.204277+0000","last_active":"2023-10-08T00:12:57.204277+0000","last_peered":"2023-10-08T00:12:57.204277+0000","last_clean":"2023-10-08T00:12:57.204277+0000","last_became_active":"2023-10-08T00:12:57.204039+0000","last_became_peered":"2023-10-08T00:12:57.204039+0000","last_unstale":"2023-10-08T00:12:57.204277+0000","last_undegraded":"2023-10-08T00:12:57.204277+0000","last_fullsized":"2023-10-08T00:12:57.204277+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:09:18.935331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.204614+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:04:37.043481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.210213+0000","last_change":"2023-10-08T00:12:57.210213+0000","last_active":"2023-10-08T00:12:57.210213+0000","last_peered":"2023-10-08T00:12:57.210213+0000","last_clean":"2023-10-08T00:12:57.210213+0000","last_became_active":"2023-10-08T00:12:57.209897+0000","last_became_peered":"2023-10-08T00:12:57.209897+0000","last_unstale":"2023-10-08T00:12:57.210213+0000","last_undegraded":"2023-10-08T00:12:57.210213+0000","last_fullsized":"2023-10-08T00:12:57.210213+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:49:05.184189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.207309+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:24:53.646854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.210506+0000","last_change":"2023-10-08T00:12:57.210506+0000","last_active":"2023-10-08T00:12:57.210506+0000","last_peered":"2023-10-08T00:12:57.210506+0000","last_clean":"2023-10-08T00:12:57.210506+0000","last_became_active":"2023-10-08T00:12:57.210180+0000","last_became_peered":"2023-10-08T00:12:57.210180+0000","last_unstale":"2023-10-08T00:12:57.210506+0000","last_undegraded":"2023-10-08T00:12:57.210506+0000","last_fullsized":"2023-10-08T00:12:57.210506+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:57:52.142531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206202+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:37:04.823729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.215419+0000","last_change":"2023-10-08T00:12:57.215419+0000","last_active":"2023-10-08T00:12:57.215419+0000","last_peered":"2023-10-08T00:12:57.215419+0000","last_clean":"2023-10-08T00:12:57.215419+0000","last_became_active":"2023-10-08T00:12:57.214895+0000","last_became_peered":"2023-10-08T00:12:57.214895+0000","last_unstale":"2023-10-08T00:12:57.215419+0000","last_undegraded":"2023-10-08T00:12:57.215419+0000","last_fullsized":"2023-10-08T00:12:57.215419+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:59:43.036033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.205871+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:25:36.388792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.211758+0000","last_change":"2023-10-08T00:12:57.211758+0000","last_active":"2023-10-08T00:12:57.211758+0000","last_peered":"2023-10-08T00:12:57.211758+0000","last_clean":"2023-10-08T00:12:57.211758+0000","last_became_active":"2023-10-08T00:12:57.211537+0000","last_became_peered":"2023-10-08T00:12:57.211537+0000","last_unstale":"2023-10-08T00:12:57.211758+0000","last_undegraded":"2023-10-08T00:12:57.211758+0000","last_fullsized":"2023-10-08T00:12:57.211758+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:42:52.161510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.205056+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:33:58.167385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.211562+0000","last_change":"2023-10-08T00:12:57.211562+0000","last_active":"2023-10-08T00:12:57.211562+0000","last_peered":"2023-10-08T00:12:57.211562+0000","last_clean":"2023-10-08T00:12:57.211562+0000","last_became_active":"2023-10-08T00:12:57.211290+0000","last_became_peered":"2023-10-08T00:12:57.211290+0000","last_unstale":"2023-10-08T00:12:57.211562+0000","last_undegraded":"2023-10-08T00:12:57.211562+0000","last_fullsized":"2023-10-08T00:12:57.211562+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:36:52.303382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.203529+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:32:16.626313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.212893+0000","last_change":"2023-10-08T00:12:57.212893+0000","last_active":"2023-10-08T00:12:57.212893+0000","last_peered":"2023-10-08T00:12:57.212893+0000","last_clean":"2023-10-08T00:12:57.212893+0000","last_became_active":"2023-10-08T00:12:57.212632+0000","last_became_peered":"2023-10-08T00:12:57.212632+0000","last_unstale":"2023-10-08T00:12:57.212893+0000","last_undegraded":"2023-10-08T00:12:57.212893+0000","last_fullsized":"2023-10-08T00:12:57.212893+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:00:48.158147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.203581+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:56:37.046732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.214282+0000","last_change":"2023-10-08T00:12:57.214282+0000","last_active":"2023-10-08T00:12:57.214282+0000","last_peered":"2023-10-08T00:12:57.214282+0000","last_clean":"2023-10-08T00:12:57.214282+0000","last_became_active":"2023-10-08T00:12:57.214017+0000","last_became_peered":"2023-10-08T00:12:57.214017+0000","last_unstale":"2023-10-08T00:12:57.214282+0000","last_undegraded":"2023-10-08T00:12:57.214282+0000","last_fullsized":"2023-10-08T00:12:57.214282+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:52:20.648298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.203749+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:15:27.299505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.210417+0000","last_change":"2023-10-08T00:12:57.210417+0000","last_active":"2023-10-08T00:12:57.210417+0000","last_peered":"2023-10-08T00:12:57.210417+0000","last_clean":"2023-10-08T00:12:57.210417+0000","last_became_active":"2023-10-08T00:12:57.210168+0000","last_became_peered":"2023-10-08T00:12:57.210168+0000","last_unstale":"2023-10-08T00:12:57.210417+0000","last_undegraded":"2023-10-08T00:12:57.210417+0000","last_fullsized":"2023-10-08T00:12:57.210417+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:51:53.256123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206067+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:11:41.089416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.213694+0000","last_change":"2023-10-08T00:12:57.213694+0000","last_active":"2023-10-08T00:12:57.213694+0000","last_peered":"2023-10-08T00:12:57.213694+0000","last_clean":"2023-10-08T00:12:57.213694+0000","last_became_active":"2023-10-08T00:12:57.213466+0000","last_became_peered":"2023-10-08T00:12:57.213466+0000","last_unstale":"2023-10-08T00:12:57.213694+0000","last_undegraded":"2023-10-08T00:12:57.213694+0000","last_fullsized":"2023-10-08T00:12:57.213694+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:50:16.963706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.205826+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:28:53.462666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.211421+0000","last_change":"2023-10-08T00:12:57.211421+0000","last_active":"2023-10-08T00:12:57.211421+0000","last_peered":"2023-10-08T00:12:57.211421+0000","last_clean":"2023-10-08T00:12:57.211421+0000","last_became_active":"2023-10-08T00:12:57.210901+0000","last_became_peered":"2023-10-08T00:12:57.210901+0000","last_unstale":"2023-10-08T00:12:57.211421+0000","last_undegraded":"2023-10-08T00:12:57.211421+0000","last_fullsized":"2023-10-08T00:12:57.211421+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:43:05.018683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.205536+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:35:32.137447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.206333+0000","last_change":"2023-10-08T00:12:57.206333+0000","last_active":"2023-10-08T00:12:57.206333+0000","last_peered":"2023-10-08T00:12:57.206333+0000","last_clean":"2023-10-08T00:12:57.206333+0000","last_became_active":"2023-10-08T00:12:57.206123+0000","last_became_peered":"2023-10-08T00:12:57.206123+0000","last_unstale":"2023-10-08T00:12:57.206333+0000","last_undegraded":"2023-10-08T00:12:57.206333+0000","last_fullsized":"2023-10-08T00:12:57.206333+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:27:43.372597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.204798+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:47:28.589841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.212538+0000","last_change":"2023-10-08T00:12:57.212538+0000","last_active":"2023-10-08T00:12:57.212538+0000","last_peered":"2023-10-08T00:12:57.212538+0000","last_clean":"2023-10-08T00:12:57.212538+0000","last_became_active":"2023-10-08T00:12:57.212360+0000","last_became_peered":"2023-10-08T00:12:57.212360+0000","last_unstale":"2023-10-08T00:12:57.212538+0000","last_undegraded":"2023-10-08T00:12:57.212538+0000","last_fullsized":"2023-10-08T00:12:57.212538+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:39:23.590206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206666+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:40:34.751005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.213785+0000","last_change":"2023-10-08T00:12:57.213785+0000","last_active":"2023-10-08T00:12:57.213785+0000","last_peered":"2023-10-08T00:12:57.213785+0000","last_clean":"2023-10-08T00:12:57.213785+0000","last_became_active":"2023-10-08T00:12:57.213351+0000","last_became_peered":"2023-10-08T00:12:57.213351+0000","last_unstale":"2023-10-08T00:12:57.213785+0000","last_undegraded":"2023-10-08T00:12:57.213785+0000","last_fullsized":"2023-10-08T00:12:57.213785+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:17:09.287153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.205741+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:22:06.125225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.209114+0000","last_change":"2023-10-08T00:12:57.209114+0000","last_active":"2023-10-08T00:12:57.209114+0000","last_peered":"2023-10-08T00:12:57.209114+0000","last_clean":"2023-10-08T00:12:57.209114+0000","last_became_active":"2023-10-08T00:12:57.208878+0000","last_became_peered":"2023-10-08T00:12:57.208878+0000","last_unstale":"2023-10-08T00:12:57.209114+0000","last_undegraded":"2023-10-08T00:12:57.209114+0000","last_fullsized":"2023-10-08T00:12:57.209114+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:29:06.434469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.204353+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:50:14.863709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.213998+0000","last_change":"2023-10-08T00:12:57.213998+0000","last_active":"2023-10-08T00:12:57.213998+0000","last_peered":"2023-10-08T00:12:57.213998+0000","last_clean":"2023-10-08T00:12:57.213998+0000","last_became_active":"2023-10-08T00:12:57.213768+0000","last_became_peered":"2023-10-08T00:12:57.213768+0000","last_unstale":"2023-10-08T00:12:57.213998+0000","last_undegraded":"2023-10-08T00:12:57.213998+0000","last_fullsized":"2023-10-08T00:12:57.213998+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:36:23.496893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.205083+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:56:09.457506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.209066+0000","last_change":"2023-10-08T00:12:57.209066+0000","last_active":"2023-10-08T00:12:57.209066+0000","last_peered":"2023-10-08T00:12:57.209066+0000","last_clean":"2023-10-08T00:12:57.209066+0000","last_became_active":"2023-10-08T00:12:57.208849+0000","last_became_peered":"2023-10-08T00:12:57.208849+0000","last_unstale":"2023-10-08T00:12:57.209066+0000","last_undegraded":"2023-10-08T00:12:57.209066+0000","last_fullsized":"2023-10-08T00:12:57.209066+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:29:06.434469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.204310+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:39:49.309778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.212524+0000","last_change":"2023-10-08T00:12:57.212524+0000","last_active":"2023-10-08T00:12:57.212524+0000","last_peered":"2023-10-08T00:12:57.212524+0000","last_clean":"2023-10-08T00:12:57.212524+0000","last_became_active":"2023-10-08T00:12:57.212291+0000","last_became_peered":"2023-10-08T00:12:57.212291+0000","last_unstale":"2023-10-08T00:12:57.212524+0000","last_undegraded":"2023-10-08T00:12:57.212524+0000","last_fullsized":"2023-10-08T00:12:57.212524+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:00:48.158147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":199,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-08T00:12:56.201662+0000","last_change":"2023-10-08T00:10:43.728490+0000","last_active":"2023-10-08T00:12:56.201662+0000","last_peered":"2023-10-08T00:12:56.201662+0000","last_clean":"2023-10-08T00:12:56.201662+0000","last_became_active":"2023-10-08T00:10:43.728341+0000","last_became_peered":"2023-10-08T00:10:43.728341+0000","last_unstale":"2023-10-08T00:12:56.201662+0000","last_undegraded":"2023-10-08T00:12:56.201662+0000","last_fullsized":"2023-10-08T00:12:56.201662+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_clean_scrub_stamp":"2023-10-08T00:10:42.716096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:06:31.035293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":336,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194976,"statfs":{"total":200000000,"available":199655719,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.496,"15min":0.49399999999999999},"min":{"1min":0.34599999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63100000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.53800000000000003,"15min":0.54400000000000004},"min":{"1min":0.38100000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.73499999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.40300000000000002}]},{"osd":1,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.53100000000000003,"15min":0.52700000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.437,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.56999999999999995}]},{"osd":2,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.53000000000000003,"15min":0.53300000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.66600000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.51600000000000001,"15min":0.51400000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.71899999999999997,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.55900000000000005}]},{"osd":3,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.51700000000000002,"15min":0.51300000000000001},"min":{"1min":0.36599999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.55200000000000005,"15min":0.55400000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.47099999999999997}]},{"osd":4,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.52200000000000002,"15min":0.52600000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.70799999999999996,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.497},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.52800000000000002,"15min":0.53200000000000003},"min":{"1min":0.35699999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.64300000000000002,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.34599999999999997}]},{"osd":5,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.55300000000000005,"15min":0.55700000000000005},"min":{"1min":0.32100000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.71999999999999997,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.57999999999999996,"15min":0.59199999999999997},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.69399999999999995,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.58099999999999996}]},{"osd":6,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.59399999999999997,"15min":0.59899999999999998},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.71299999999999997,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.627,"15min":0.63800000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76000000000000001,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.52800000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":336,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194976,"statfs":{"total":200000000,"available":199655719,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.48099999999999998,"15min":0.47499999999999998},"min":{"1min":0.377,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.752,"5min":1.012,"15min":1.012},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.68400000000000005,"15min":0.70499999999999996},"min":{"1min":0.34000000000000002,"5min":0.317,"15min":0.317},"max":{"1min":1.2789999999999999,"5min":5.1230000000000002,"15min":5.1230000000000002},"last":0.374}]},{"osd":1,"last update":"Sun Oct 8 00:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.52100000000000002,"15min":0.51400000000000001},"min":{"1min":0.29299999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.82499999999999996,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55900000000000005,"15min":0.56000000000000005},"min":{"1min":0.371,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.69899999999999995,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.45400000000000001}]},{"osd":2,"last update":"Sun Oct 8 00:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.52700000000000002,"15min":0.52500000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.755,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56200000000000006,"15min":0.56499999999999995},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.76100000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.48899999999999999}]},{"osd":3,"last update":"Sun Oct 8 00:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53400000000000003,"15min":0.53000000000000003},"min":{"1min":0.38400000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.81399999999999995,"5min":1.046,"15min":1.046},"last":0.501},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.54400000000000004,"15min":0.53600000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.36099999999999999}]},{"osd":4,"last update":"Sun Oct 8 00:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.502,"15min":0.50600000000000001},"min":{"1min":0.32300000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56499999999999995,"15min":0.56699999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.42999999999999999}]},{"osd":5,"last update":"Sun Oct 8 00:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.48499999999999999,"15min":0.48799999999999999},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71099999999999997,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.55500000000000005,"15min":0.56299999999999994},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.439}]},{"osd":7,"last update":"Sun Oct 8 00:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.54100000000000004,"15min":0.54500000000000004},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.41699999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574877,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":912,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194399,"statfs":{"total":200000000,"available":199065351,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.30399999999999999,"15min":0.29899999999999999},"min":{"1min":0.218,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.44700000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.375,"15min":0.373},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.38300000000000001}]},{"osd":2,"last update":"Sun Oct 8 00:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.33600000000000002,"15min":0.33200000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.49099999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.379,"5min":0.33900000000000002,"15min":0.33200000000000002},"min":{"1min":0.254,"5min":0.252,"15min":0.252},"max":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.34999999999999998}]},{"osd":3,"last update":"Sun Oct 8 00:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.36699999999999999,"15min":0.36499999999999999},"min":{"1min":0.28299999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.307},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.501,"5min":0.501,"15min":0.501},"last":0.41999999999999998}]},{"osd":4,"last update":"Sun Oct 8 00:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.53700000000000003,"15min":0.53400000000000003},"min":{"1min":0.40400000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.56899999999999995,"15min":0.56499999999999995},"min":{"1min":0.41899999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.82299999999999995,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.60399999999999998}]},{"osd":5,"last update":"Sun Oct 8 00:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.53800000000000003,"15min":0.53100000000000003},"min":{"1min":0.372,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57499999999999996,"15min":0.57199999999999995},"min":{"1min":0.42999999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78300000000000003,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.73499999999999999}]},{"osd":6,"last update":"Sun Oct 8 00:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.72999999999999998,"5min":0.877,"15min":0.877},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56399999999999995,"15min":0.55800000000000005},"min":{"1min":0.42899999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.76900000000000002,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.61899999999999999}]},{"osd":7,"last update":"Sun Oct 8 00:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56299999999999994,"15min":0.55900000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.77700000000000002,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.752},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.58899999999999997,"15min":0.58499999999999996},"min":{"1min":0.432,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82299999999999995,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.69999999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":336,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194976,"statfs":{"total":200000000,"available":199655719,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 00:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.32400000000000001,"15min":0.32600000000000001},"min":{"1min":0.23499999999999999,"5min":0.193,"15min":0.193},"max":{"1min":0.41499999999999998,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.308},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.245,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.49299999999999999,"5min":0.87,"15min":0.87},"last":0.40699999999999997}]},{"osd":2,"last update":"Sun Oct 8 00:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.38200000000000001,"15min":0.38600000000000001},"min":{"1min":0.27900000000000003,"5min":0.251,"15min":0.251},"max":{"1min":0.58699999999999997,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.753,"15min":0.82399999999999995},"min":{"1min":0.247,"5min":0.185,"15min":0.185},"max":{"1min":0.40899999999999997,"5min":12.019,"15min":12.019},"last":0.38900000000000001}]},{"osd":3,"last update":"Sun Oct 8 00:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.35599999999999998,"15min":0.35999999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.47399999999999998,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.374,"5min":0.38800000000000001,"15min":0.39000000000000001},"min":{"1min":0.28499999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.55900000000000005,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.33600000000000002}]},{"osd":4,"last update":"Sun Oct 8 00:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.433,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.71899999999999997,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.624,"15min":0.625},"min":{"1min":0.48299999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.73499999999999999,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.71799999999999997}]},{"osd":5,"last update":"Sun Oct 8 00:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.999,"15min":1.069},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.79800000000000004,"5min":11.992000000000001,"15min":11.992000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.73199999999999998,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.61299999999999999}]},{"osd":6,"last update":"Sun Oct 8 00:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.74399999999999999,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.97799999999999998,"15min":1.042},"min":{"1min":0.47299999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76000000000000001,"5min":11.927,"15min":11.927},"last":0.72499999999999998}]},{"osd":7,"last update":"Sun Oct 8 00:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.999,"15min":1.0660000000000001},"min":{"1min":0.434,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82199999999999995,"5min":11.965,"15min":11.965},"last":0.627},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.61899999999999999,"15min":0.621},"min":{"1min":0.48199999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.76100000000000001,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.76900000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":336,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194976,"statfs":{"total":200000000,"available":199655719,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.39000000000000001,"15min":0.40200000000000002},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.437,"5min":2.5139999999999998,"15min":2.5139999999999998},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.40100000000000002,"15min":0.41399999999999998},"min":{"1min":0.248,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.44400000000000001,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":0.35899999999999999}]},{"osd":1,"last update":"Sun Oct 8 00:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.39000000000000001,"15min":0.40200000000000002},"min":{"1min":0.25,"5min":0.246,"15min":0.246},"max":{"1min":0.40100000000000002,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.436,"15min":0.45000000000000001},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.48999999999999999,"5min":2.5379999999999998,"15min":2.5379999999999998},"last":0.39200000000000002}]},{"osd":3,"last update":"Sun Oct 8 00:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.42799999999999999,"15min":0.439},"min":{"1min":0.28499999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.46500000000000002,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":0.378},{"interface":"front","average":{"1min":0.379,"5min":0.46300000000000002,"15min":0.47699999999999998},"min":{"1min":0.315,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.51200000000000001,"5min":2.5619999999999998,"15min":2.5619999999999998},"last":0.41499999999999998}]},{"osd":4,"last update":"Sun Oct 8 00:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.61699999999999999,"15min":0.628},"min":{"1min":0.45800000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.84599999999999997,"5min":2.488,"15min":2.488},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.60399999999999998,"15min":0.60999999999999999},"min":{"1min":0.41099999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85999999999999999,"5min":2.073,"15min":2.073},"last":0.81100000000000005}]},{"osd":5,"last update":"Sun Oct 8 00:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.66300000000000003,"15min":0.66900000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.2130000000000001,"5min":2.2069999999999999,"15min":2.2069999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.62,"5min":0.66300000000000003,"15min":0.67000000000000004},"min":{"1min":0.48799999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0149999999999999,"5min":2.456,"15min":2.456},"last":0.81999999999999995}]},{"osd":6,"last update":"Sun Oct 8 00:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.621,"15min":0.627},"min":{"1min":0.45000000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.79500000000000004,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.60899999999999999,"15min":0.61399999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.97499999999999998,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.77000000000000002}]},{"osd":7,"last update":"Sun Oct 8 00:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.63700000000000001,"15min":0.64100000000000001},"min":{"1min":0.499,"5min":0.438,"15min":0.438},"max":{"1min":0.81699999999999995,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.626,"15min":0.63200000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91200000000000003,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":0.64500000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":330,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194982,"statfs":{"total":200000000,"available":199661982,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.28699999999999998,"5min":0.36699999999999999,"15min":0.38},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.45100000000000001,"5min":1.46,"15min":1.46},"last":0.251},{"interface":"front","average":{"1min":0.29699999999999999,"5min":0.33500000000000002,"15min":0.34100000000000003},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.46300000000000002,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.23300000000000001}]},{"osd":1,"last update":"Sun Oct 8 00:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.82999999999999996,"15min":0.91700000000000004},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.44800000000000001,"5min":12.351000000000001,"15min":12.351000000000001},"last":0.307},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.35899999999999999,"15min":0.36299999999999999},"min":{"1min":0.26500000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.443,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.34300000000000003}]},{"osd":2,"last update":"Sun Oct 8 00:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.41399999999999998,"15min":0.42499999999999999},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.47299999999999998,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.377},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.379,"15min":0.38300000000000001},"min":{"1min":0.27800000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.45900000000000002,"5min":0.753,"15min":0.753},"last":0.41099999999999998}]},{"osd":4,"last update":"Sun Oct 8 00:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":1.004,"15min":1.0860000000000001},"min":{"1min":0.38800000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.75600000000000001,"5min":11.333,"15min":11.333},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":1.0369999999999999,"15min":1.1200000000000001},"min":{"1min":0.434,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.77000000000000002,"5min":12.321999999999999,"15min":12.321999999999999},"last":0.63200000000000001}]},{"osd":5,"last update":"Sun Oct 8 00:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":1.0509999999999999,"15min":1.137},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.72599999999999998,"5min":12.291,"15min":12.291},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.58599999999999997,"15min":0.59299999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.74299999999999999,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.59699999999999998}]},{"osd":6,"last update":"Sun Oct 8 00:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.96999999999999997,"15min":1.048},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.69899999999999995,"5min":11.288,"15min":11.288},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.51800000000000002,"5min":1.0309999999999999,"15min":1.117},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.66000000000000003,"5min":12.253,"15min":12.253},"last":0.67500000000000004}]},{"osd":7,"last update":"Sun Oct 8 00:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56599999999999995,"15min":0.56799999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.56399999999999995,"5min":1.012,"15min":1.0860000000000001},"min":{"1min":0.435,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.381,"5min":11.363,"15min":11.363},"last":0.66600000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":336,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194976,"statfs":{"total":200000000,"available":199655719,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.56000000000000005,"15min":0.55400000000000005},"min":{"1min":0.41299999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.375},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.50600000000000001}]},{"osd":1,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51700000000000002,"15min":0.51800000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.64100000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.42099999999999999}]},{"osd":2,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.51600000000000001,"15min":0.52000000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.51300000000000001,"15min":0.50900000000000001},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.54200000000000004}]},{"osd":3,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.53500000000000003,"15min":0.53900000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.70899999999999996,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.53000000000000003,"15min":0.53200000000000003},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.40200000000000002}]},{"osd":5,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.49299999999999999,"15min":0.503},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69899999999999995,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.50800000000000001,"15min":0.51400000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.71099999999999997,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.57199999999999995}]},{"osd":6,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.48899999999999999,"15min":0.49399999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.55200000000000005,"15min":0.56200000000000006},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.754,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.58599999999999997}]},{"osd":7,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.54600000000000004,"15min":0.55500000000000005},"min":{"1min":0.36299999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.64200000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.57799999999999996,"15min":0.58299999999999996},"min":{"1min":0.41199999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.77700000000000002,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.628}]}]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":906,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194405,"statfs":{"total":200000000,"available":199071614,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.48699999999999999,"15min":0.48799999999999999},"min":{"1min":0.32700000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68799999999999994,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.441},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.55700000000000005,"15min":0.56299999999999994},"min":{"1min":0.374,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.68600000000000005,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.58599999999999997}]},{"osd":1,"last update":"Sun Oct 8 00:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.52100000000000002,"15min":0.52300000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.65800000000000003,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.52400000000000002,"15min":0.52800000000000002},"min":{"1min":0.39300000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.70899999999999996,"5min":0.755,"15min":0.755},"last":0.48999999999999999}]},{"osd":2,"last update":"Sun Oct 8 00:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.52900000000000003,"15min":0.53100000000000003},"min":{"1min":0.40500000000000003,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.64500000000000002,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.56599999999999995,"15min":0.57199999999999995},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.70699999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.59799999999999998}]},{"osd":3,"last update":"Sun Oct 8 00:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54300000000000004,"15min":0.54500000000000004},"min":{"1min":0.38700000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.67100000000000004,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.57999999999999996,"15min":0.58399999999999996},"min":{"1min":0.42799999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.72299999999999998,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.55900000000000005}]},{"osd":4,"last update":"Sun Oct 8 00:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.47299999999999998,"15min":0.47499999999999998},"min":{"1min":0.35799999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.626,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.50900000000000001,"15min":0.51300000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.63700000000000001,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.54600000000000004}]},{"osd":6,"last update":"Sun Oct 8 00:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.499,"15min":0.499},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.61899999999999999,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.66800000000000004,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.53200000000000003}]},{"osd":7,"last update":"Sun Oct 8 00:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.51300000000000001,"15min":0.51500000000000001},"min":{"1min":0.33400000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67300000000000004,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.69799999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.51600000000000001}]}]}],"pool_statfs":[]}} 2023-10-08T00:13:02.281 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T00:13:02.429 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.428+0000 7fbfd3fff700 1 Processor -- start 2023-10-08T00:13:02.429 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.428+0000 7fbfd3fff700 1 -- start start 2023-10-08T00:13:02.431 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.429+0000 7fbfd3fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfd406f2f0 0x7fbfd40c99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:02.431 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.429+0000 7fbfd3fff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fbfd406f6c0 0x7fbfd41575c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:02.431 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.429+0000 7fbfd3fff700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfd40af640 con 0x7fbfd406c440 2023-10-08T00:13:02.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.429+0000 7fbfd3fff700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfd40cbe70 con 0x7fbfd406f6c0 2023-10-08T00:13:02.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.429+0000 7fbfd3fff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd406cc80 con 0x7fbfd406f2f0 2023-10-08T00:13:02.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.429+0000 7fbfd2ffd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfd406f2f0 0x7fbfd40c99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:02.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.429+0000 7fbfd2ffd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfd406f2f0 0x7fbfd40c99d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.99:39514/0 (socket says 172.21.15.99:39514) 2023-10-08T00:13:02.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.429+0000 7fbfd2ffd700 1 -- 172.21.15.99:0/199662659 learned_addr learned my addr 172.21.15.99:0/199662659 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:13:02.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.429+0000 7fbfd37fe700 1 --2- 172.21.15.99:0/199662659 >> v2:172.21.15.191:3300/0 conn(0x7fbfd406f6c0 0x7fbfd41575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:02.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.429+0000 7fbfca7fc700 1 --1- 172.21.15.99:0/199662659 >> v1:172.21.15.191:6789/0 conn(0x7fbfd406c440 0x7fbfd40b0350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:42648/0 (socket says 172.21.15.99:42648) 2023-10-08T00:13:02.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.430+0000 7fbfd2ffd700 1 -- 172.21.15.99:0/199662659 >> v2:172.21.15.191:3300/0 conn(0x7fbfd406f6c0 msgr2=0x7fbfd41575c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:13:02.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.430+0000 7fbfd2ffd700 1 --2- 172.21.15.99:0/199662659 >> v2:172.21.15.191:3300/0 conn(0x7fbfd406f6c0 0x7fbfd41575c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:02.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.430+0000 7fbfd2ffd700 1 -- 172.21.15.99:0/199662659 >> v1:172.21.15.191:6789/0 conn(0x7fbfd406c440 legacy=0x7fbfd40b0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:02.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.430+0000 7fbfd2ffd700 1 -- 172.21.15.99:0/199662659 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd40afec0 con 0x7fbfd406f2f0 2023-10-08T00:13:02.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.430+0000 7fbfd37fe700 1 --2- 172.21.15.99:0/199662659 >> v2:172.21.15.191:3300/0 conn(0x7fbfd406f6c0 0x7fbfd41575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:13:02.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.430+0000 7fbfd2ffd700 1 --2- 172.21.15.99:0/199662659 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfd406f2f0 0x7fbfd40c99d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbfc40099b0 tx=0x7fbfc4009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6d03108c1483e41 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:13:02.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.430+0000 7fbfd27fc700 1 -- 172.21.15.99:0/199662659 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc4009ea0 con 0x7fbfd406f2f0 2023-10-08T00:13:02.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.430+0000 7fbfd27fc700 1 -- 172.21.15.99:0/199662659 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfc400b860 con 0x7fbfd406f2f0 2023-10-08T00:13:02.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.430+0000 7fbfd27fc700 1 -- 172.21.15.99:0/199662659 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc400bc00 con 0x7fbfd406f2f0 2023-10-08T00:13:02.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.431+0000 7fbfd3fff700 1 -- 172.21.15.99:0/199662659 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfd406f2f0 msgr2=0x7fbfd40c99d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:02.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.431+0000 7fbfd3fff700 1 --2- 172.21.15.99:0/199662659 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfd406f2f0 0x7fbfd40c99d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbfc40099b0 tx=0x7fbfc4009250 comp rx=0 tx=0).stop 2023-10-08T00:13:02.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.431+0000 7fbfd3fff700 1 -- 172.21.15.99:0/199662659 shutdown_connections 2023-10-08T00:13:02.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.431+0000 7fbfd3fff700 1 --2- 172.21.15.99:0/199662659 >> v2:172.21.15.191:3300/0 conn(0x7fbfd406f6c0 0x7fbfd41575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:02.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.431+0000 7fbfd3fff700 1 --2- 172.21.15.99:0/199662659 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfd406f2f0 0x7fbfd40c99d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:02.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.431+0000 7fbfd3fff700 1 -- 172.21.15.99:0/199662659 >> 172.21.15.99:0/199662659 conn(0x7fbfd4069870 msgr2=0x7fbfd4069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:13:02.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.431+0000 7fbfd3fff700 1 -- 172.21.15.99:0/199662659 shutdown_connections 2023-10-08T00:13:02.434 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.431+0000 7fbfd3fff700 1 -- 172.21.15.99:0/199662659 wait complete. 2023-10-08T00:13:02.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.432+0000 7fbfd3fff700 1 Processor -- start 2023-10-08T00:13:02.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.432+0000 7fbfd3fff700 1 -- start start 2023-10-08T00:13:02.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.433+0000 7fbfd3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfd406c440 0x7fbfd40c7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:02.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.433+0000 7fbfd3fff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbfd406f2f0 0x7fbfd40c7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:02.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.433+0000 7fbfd3fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfd406f6c0 0x7fbfd40c8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:02.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.433+0000 7fbfd3fff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd40c78c0 con 0x7fbfd406c440 2023-10-08T00:13:02.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.433+0000 7fbfd3fff700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd4155720 con 0x7fbfd406f2f0 2023-10-08T00:13:02.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.433+0000 7fbfd3fff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd41565e0 con 0x7fbfd406f6c0 2023-10-08T00:13:02.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.433+0000 7fbfd2ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfd406c440 0x7fbfd40c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:02.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.433+0000 7fbfd2ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfd406c440 0x7fbfd40c7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:32926/0 (socket says 172.21.15.99:32926) 2023-10-08T00:13:02.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.433+0000 7fbfd2ffd700 1 -- 172.21.15.99:0/2059833461 learned_addr learned my addr 172.21.15.99:0/2059833461 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:13:02.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.433+0000 7fbfd37fe700 1 --2- 172.21.15.99:0/2059833461 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfd406f6c0 0x7fbfd40c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:02.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.433+0000 7fbfca7fc700 1 --2- 172.21.15.99:0/2059833461 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbfd406f2f0 0x7fbfd40c7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:02.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.433+0000 7fbfd2ffd700 1 -- 172.21.15.99:0/2059833461 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfd406f6c0 msgr2=0x7fbfd40c8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:02.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.433+0000 7fbfd2ffd700 1 --2- 172.21.15.99:0/2059833461 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfd406f6c0 0x7fbfd40c8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:02.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.433+0000 7fbfd2ffd700 1 -- 172.21.15.99:0/2059833461 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbfd406f2f0 msgr2=0x7fbfd40c7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:02.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.433+0000 7fbfd2ffd700 1 --2- 172.21.15.99:0/2059833461 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbfd406f2f0 0x7fbfd40c7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:02.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.433+0000 7fbfd2ffd700 1 -- 172.21.15.99:0/2059833461 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfcc009a20 con 0x7fbfd406c440 2023-10-08T00:13:02.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.433+0000 7fbfca7fc700 1 --2- 172.21.15.99:0/2059833461 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbfd406f2f0 0x7fbfd40c7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:13:02.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.433+0000 7fbfd37fe700 1 --2- 172.21.15.99:0/2059833461 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfd406f6c0 0x7fbfd40c8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:13:02.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.433+0000 7fbfd2ffd700 1 --2- 172.21.15.99:0/2059833461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfd406c440 0x7fbfd40c7380 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4006fd0 tx=0x7fbfc4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:13:02.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.433+0000 7fbfcb7fe700 1 -- 172.21.15.99:0/2059833461 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc400bbe0 con 0x7fbfd406c440 2023-10-08T00:13:02.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.433+0000 7fbfcb7fe700 1 -- 172.21.15.99:0/2059833461 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfc4011420 con 0x7fbfd406c440 2023-10-08T00:13:02.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.433+0000 7fbfd3fff700 1 -- 172.21.15.99:0/2059833461 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfc40094f0 con 0x7fbfd406c440 2023-10-08T00:13:02.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.434+0000 7fbfcb7fe700 1 -- 172.21.15.99:0/2059833461 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc40117c0 con 0x7fbfd406c440 2023-10-08T00:13:02.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.434+0000 7fbfd3fff700 1 -- 172.21.15.99:0/2059833461 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfd4155f90 con 0x7fbfd406c440 2023-10-08T00:13:02.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.434+0000 7fbfd3fff700 1 -- 172.21.15.99:0/2059833461 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfd406e3e0 con 0x7fbfd406c440 2023-10-08T00:13:02.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.435+0000 7fbfcb7fe700 1 -- 172.21.15.99:0/2059833461 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbfc4011bb0 con 0x7fbfd406c440 2023-10-08T00:13:02.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.435+0000 7fbfcb7fe700 1 --2- 172.21.15.99:0/2059833461 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbfb4066f30 0x7fbfb40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:02.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.435+0000 7fbfcb7fe700 1 -- 172.21.15.99:0/2059833461 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbfc4085a70 con 0x7fbfd406c440 2023-10-08T00:13:02.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.436+0000 7fbfca7fc700 1 --2- 172.21.15.99:0/2059833461 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbfb4066f30 0x7fbfb40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:02.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.436+0000 7fbfca7fc700 1 --2- 172.21.15.99:0/2059833461 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbfb4066f30 0x7fbfb40693e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc007fd0 tx=0x7fbfbc007e10 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:13:02.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.440+0000 7fbfcb7fe700 1 -- 172.21.15.99:0/2059833461 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbfc4052d60 con 0x7fbfd406c440 2023-10-08T00:13:02.619 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.619+0000 7fbfd3fff700 1 -- 172.21.15.99:0/2059833461 --> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbfd4156720 con 0x7fbfb4066f30 2023-10-08T00:13:02.628 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.627+0000 7fbfcb7fe700 1 -- 172.21.15.99:0/2059833461 <== mgr.4099 v2:172.21.15.191:6832/85985 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+281850 (secure 0 0 0) 0x7fbfbc012380 con 0x7fbfb4066f30 2023-10-08T00:13:02.631 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:13:02.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.630+0000 7fbfd3fff700 1 -- 172.21.15.99:0/2059833461 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbfb4066f30 msgr2=0x7fbfb40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:02.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.630+0000 7fbfd3fff700 1 --2- 172.21.15.99:0/2059833461 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbfb4066f30 0x7fbfb40693e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc007fd0 tx=0x7fbfbc007e10 comp rx=0 tx=0).stop 2023-10-08T00:13:02.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.630+0000 7fbfd3fff700 1 -- 172.21.15.99:0/2059833461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfd406c440 msgr2=0x7fbfd40c7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:02.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.630+0000 7fbfd3fff700 1 --2- 172.21.15.99:0/2059833461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfd406c440 0x7fbfd40c7380 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4006fd0 tx=0x7fbfc4017000 comp rx=0 tx=0).stop 2023-10-08T00:13:02.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.630+0000 7fbfd3fff700 1 -- 172.21.15.99:0/2059833461 shutdown_connections 2023-10-08T00:13:02.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.630+0000 7fbfd3fff700 1 --2- 172.21.15.99:0/2059833461 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbfb4066f30 0x7fbfb40693e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:02.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.630+0000 7fbfd3fff700 1 --2- 172.21.15.99:0/2059833461 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfd406f6c0 0x7fbfd40c8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:02.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.630+0000 7fbfd3fff700 1 --2- 172.21.15.99:0/2059833461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfd406c440 0x7fbfd40c7380 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:02.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.630+0000 7fbfd3fff700 1 --2- 172.21.15.99:0/2059833461 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbfd406f2f0 0x7fbfd40c7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:02.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.631+0000 7fbfd3fff700 1 -- 172.21.15.99:0/2059833461 >> 172.21.15.99:0/2059833461 conn(0x7fbfd4069870 msgr2=0x7fbfd40ca980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:13:02.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.631+0000 7fbfd3fff700 1 -- 172.21.15.99:0/2059833461 shutdown_connections 2023-10-08T00:13:02.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.631+0000 7fbfd3fff700 1 -- 172.21.15.99:0/2059833461 wait complete. 2023-10-08T00:13:02.634 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2023-10-08T00:13:02.643 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":129,"stamp":"2023-10-08T00:13:02.053575+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":3830,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1558669,"statfs":{"total":1600000000,"available":1596077542,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.858362"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.207886+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:42:53.170735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206623+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:29:57.147801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206756+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:15:27.299505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.204518+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:17:47.918131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.208328+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:29:07.443694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.207050+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:19:19.113722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.204971+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:18:06.799190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.214505+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:51:20.513160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.204229+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:57:53.151755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.207215+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:03:56.099847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.207550+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:03:56.099847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.203197+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:04:57.344499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206586+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:36:53.312606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206510+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:56:37.046732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.205079+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:39:24.599431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.207493+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:39:24.599431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206258+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:00:49.167372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206438+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:33:06.564584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.205454+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:47:44.758644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.208478+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:22:06.125225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206453+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:29:07.443694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.209033+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:42:53.170735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.209265+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:56:15.157209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.203882+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:50:17.972931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206009+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:36:24.506117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206105+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:42:53.170735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206249+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:43:06.027907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.213400+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:57:11.917467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.208004+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:17:37.423308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.205536+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:28:33.161628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206770+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:27.657370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.209403+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:29:07.443694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.205320+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:33:16.516139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.213303+0000","last_change":"2023-10-08T00:12:57.213303+0000","last_active":"2023-10-08T00:12:57.213303+0000","last_peered":"2023-10-08T00:12:57.213303+0000","last_clean":"2023-10-08T00:12:57.213303+0000","last_became_active":"2023-10-08T00:12:57.213092+0000","last_became_peered":"2023-10-08T00:12:57.213092+0000","last_unstale":"2023-10-08T00:12:57.213303+0000","last_undegraded":"2023-10-08T00:12:57.213303+0000","last_fullsized":"2023-10-08T00:12:57.213303+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:40:33.741781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.205824+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:51:47.982127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.212771+0000","last_change":"2023-10-08T00:12:57.212771+0000","last_active":"2023-10-08T00:12:57.212771+0000","last_peered":"2023-10-08T00:12:57.212771+0000","last_clean":"2023-10-08T00:12:57.212771+0000","last_became_active":"2023-10-08T00:12:57.212479+0000","last_became_peered":"2023-10-08T00:12:57.212479+0000","last_unstale":"2023-10-08T00:12:57.212771+0000","last_undegraded":"2023-10-08T00:12:57.212771+0000","last_fullsized":"2023-10-08T00:12:57.212771+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:51:18.660991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.207674+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:13:42.175367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.209653+0000","last_change":"2023-10-08T00:12:57.209653+0000","last_active":"2023-10-08T00:12:57.209653+0000","last_peered":"2023-10-08T00:12:57.209653+0000","last_clean":"2023-10-08T00:12:57.209653+0000","last_became_active":"2023-10-08T00:12:57.209408+0000","last_became_peered":"2023-10-08T00:12:57.209408+0000","last_unstale":"2023-10-08T00:12:57.209653+0000","last_undegraded":"2023-10-08T00:12:57.209653+0000","last_fullsized":"2023-10-08T00:12:57.209653+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:26.648145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.207619+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:22:06.125225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.207927+0000","last_change":"2023-10-08T00:12:57.207927+0000","last_active":"2023-10-08T00:12:57.207927+0000","last_peered":"2023-10-08T00:12:57.207927+0000","last_clean":"2023-10-08T00:12:57.207927+0000","last_became_active":"2023-10-08T00:12:57.207442+0000","last_became_peered":"2023-10-08T00:12:57.207442+0000","last_unstale":"2023-10-08T00:12:57.207927+0000","last_undegraded":"2023-10-08T00:12:57.207927+0000","last_fullsized":"2023-10-08T00:12:57.207927+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:57:10.908242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.204176+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:24:53.646854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.212292+0000","last_change":"2023-10-08T00:12:57.212292+0000","last_active":"2023-10-08T00:12:57.212292+0000","last_peered":"2023-10-08T00:12:57.212292+0000","last_clean":"2023-10-08T00:12:57.212292+0000","last_became_active":"2023-10-08T00:12:57.211867+0000","last_became_peered":"2023-10-08T00:12:57.211867+0000","last_unstale":"2023-10-08T00:12:57.212292+0000","last_undegraded":"2023-10-08T00:12:57.212292+0000","last_fullsized":"2023-10-08T00:12:57.212292+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:36:52.303382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.208400+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:39:24.599431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.211573+0000","last_change":"2023-10-08T00:12:57.211573+0000","last_active":"2023-10-08T00:12:57.211573+0000","last_peered":"2023-10-08T00:12:57.211573+0000","last_clean":"2023-10-08T00:12:57.211573+0000","last_became_active":"2023-10-08T00:12:57.211202+0000","last_became_peered":"2023-10-08T00:12:57.211202+0000","last_unstale":"2023-10-08T00:12:57.211573+0000","last_undegraded":"2023-10-08T00:12:57.211573+0000","last_fullsized":"2023-10-08T00:12:57.211573+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:29:56.138577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.208004+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:47:28.589841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.205376+0000","last_change":"2023-10-08T00:12:57.205376+0000","last_active":"2023-10-08T00:12:57.205376+0000","last_peered":"2023-10-08T00:12:57.205376+0000","last_clean":"2023-10-08T00:12:57.205376+0000","last_became_active":"2023-10-08T00:12:57.205158+0000","last_became_peered":"2023-10-08T00:12:57.205158+0000","last_unstale":"2023-10-08T00:12:57.205376+0000","last_undegraded":"2023-10-08T00:12:57.205376+0000","last_fullsized":"2023-10-08T00:12:57.205376+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:15:43.196718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.204806+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:22:04.500698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.214270+0000","last_change":"2023-10-08T00:12:57.214270+0000","last_active":"2023-10-08T00:12:57.214270+0000","last_peered":"2023-10-08T00:12:57.214270+0000","last_clean":"2023-10-08T00:12:57.214270+0000","last_became_active":"2023-10-08T00:12:57.214082+0000","last_became_peered":"2023-10-08T00:12:57.214082+0000","last_unstale":"2023-10-08T00:12:57.214270+0000","last_undegraded":"2023-10-08T00:12:57.214270+0000","last_fullsized":"2023-10-08T00:12:57.214270+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:40:33.741781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.204948+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:03:56.099847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.209199+0000","last_change":"2023-10-08T00:12:57.209199+0000","last_active":"2023-10-08T00:12:57.209199+0000","last_peered":"2023-10-08T00:12:57.209199+0000","last_clean":"2023-10-08T00:12:57.209199+0000","last_became_active":"2023-10-08T00:12:57.208965+0000","last_became_peered":"2023-10-08T00:12:57.208965+0000","last_unstale":"2023-10-08T00:12:57.209199+0000","last_undegraded":"2023-10-08T00:12:57.209199+0000","last_fullsized":"2023-10-08T00:12:57.209199+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:02:39.029498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.207557+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:39:49.309778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.214795+0000","last_change":"2023-10-08T00:12:57.214795+0000","last_active":"2023-10-08T00:12:57.214795+0000","last_peered":"2023-10-08T00:12:57.214795+0000","last_clean":"2023-10-08T00:12:57.214795+0000","last_became_active":"2023-10-08T00:12:57.214269+0000","last_became_peered":"2023-10-08T00:12:57.214269+0000","last_unstale":"2023-10-08T00:12:57.214795+0000","last_undegraded":"2023-10-08T00:12:57.214795+0000","last_fullsized":"2023-10-08T00:12:57.214795+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:28:52.453441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206370+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:52:28.504130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.210087+0000","last_change":"2023-10-08T00:12:57.210087+0000","last_active":"2023-10-08T00:12:57.210087+0000","last_peered":"2023-10-08T00:12:57.210087+0000","last_clean":"2023-10-08T00:12:57.210087+0000","last_became_active":"2023-10-08T00:12:57.209574+0000","last_became_peered":"2023-10-08T00:12:57.209574+0000","last_unstale":"2023-10-08T00:12:57.210087+0000","last_undegraded":"2023-10-08T00:12:57.210087+0000","last_fullsized":"2023-10-08T00:12:57.210087+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:51:19.503936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.208930+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:44:29.429854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.209678+0000","last_change":"2023-10-08T00:12:57.209678+0000","last_active":"2023-10-08T00:12:57.209678+0000","last_peered":"2023-10-08T00:12:57.209678+0000","last_clean":"2023-10-08T00:12:57.209678+0000","last_became_active":"2023-10-08T00:12:57.209454+0000","last_became_peered":"2023-10-08T00:12:57.209454+0000","last_unstale":"2023-10-08T00:12:57.209678+0000","last_undegraded":"2023-10-08T00:12:57.209678+0000","last_fullsized":"2023-10-08T00:12:57.209678+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:51:46.972903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.203919+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:17:10.296378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.215576+0000","last_change":"2023-10-08T00:12:57.215576+0000","last_active":"2023-10-08T00:12:57.215576+0000","last_peered":"2023-10-08T00:12:57.215576+0000","last_clean":"2023-10-08T00:12:57.215576+0000","last_became_active":"2023-10-08T00:12:57.215200+0000","last_became_peered":"2023-10-08T00:12:57.215200+0000","last_unstale":"2023-10-08T00:12:57.215576+0000","last_undegraded":"2023-10-08T00:12:57.215576+0000","last_fullsized":"2023-10-08T00:12:57.215576+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:11:40.080191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.207660+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:50:14.863709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.208075+0000","last_change":"2023-10-08T00:12:57.208075+0000","last_active":"2023-10-08T00:12:57.208075+0000","last_peered":"2023-10-08T00:12:57.208075+0000","last_clean":"2023-10-08T00:12:57.208075+0000","last_became_active":"2023-10-08T00:12:57.207720+0000","last_became_peered":"2023-10-08T00:12:57.207720+0000","last_unstale":"2023-10-08T00:12:57.208075+0000","last_undegraded":"2023-10-08T00:12:57.208075+0000","last_fullsized":"2023-10-08T00:12:57.208075+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:23:41.947350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.203096+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:43:33.796004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.204277+0000","last_change":"2023-10-08T00:12:57.204277+0000","last_active":"2023-10-08T00:12:57.204277+0000","last_peered":"2023-10-08T00:12:57.204277+0000","last_clean":"2023-10-08T00:12:57.204277+0000","last_became_active":"2023-10-08T00:12:57.204039+0000","last_became_peered":"2023-10-08T00:12:57.204039+0000","last_unstale":"2023-10-08T00:12:57.204277+0000","last_undegraded":"2023-10-08T00:12:57.204277+0000","last_fullsized":"2023-10-08T00:12:57.204277+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:09:18.935331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.204614+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:04:37.043481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.210213+0000","last_change":"2023-10-08T00:12:57.210213+0000","last_active":"2023-10-08T00:12:57.210213+0000","last_peered":"2023-10-08T00:12:57.210213+0000","last_clean":"2023-10-08T00:12:57.210213+0000","last_became_active":"2023-10-08T00:12:57.209897+0000","last_became_peered":"2023-10-08T00:12:57.209897+0000","last_unstale":"2023-10-08T00:12:57.210213+0000","last_undegraded":"2023-10-08T00:12:57.210213+0000","last_fullsized":"2023-10-08T00:12:57.210213+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:49:05.184189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.207309+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:24:53.646854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.210506+0000","last_change":"2023-10-08T00:12:57.210506+0000","last_active":"2023-10-08T00:12:57.210506+0000","last_peered":"2023-10-08T00:12:57.210506+0000","last_clean":"2023-10-08T00:12:57.210506+0000","last_became_active":"2023-10-08T00:12:57.210180+0000","last_became_peered":"2023-10-08T00:12:57.210180+0000","last_unstale":"2023-10-08T00:12:57.210506+0000","last_undegraded":"2023-10-08T00:12:57.210506+0000","last_fullsized":"2023-10-08T00:12:57.210506+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:57:52.142531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206202+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:37:04.823729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.215419+0000","last_change":"2023-10-08T00:12:57.215419+0000","last_active":"2023-10-08T00:12:57.215419+0000","last_peered":"2023-10-08T00:12:57.215419+0000","last_clean":"2023-10-08T00:12:57.215419+0000","last_became_active":"2023-10-08T00:12:57.214895+0000","last_became_peered":"2023-10-08T00:12:57.214895+0000","last_unstale":"2023-10-08T00:12:57.215419+0000","last_undegraded":"2023-10-08T00:12:57.215419+0000","last_fullsized":"2023-10-08T00:12:57.215419+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:59:43.036033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.205871+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:25:36.388792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.211758+0000","last_change":"2023-10-08T00:12:57.211758+0000","last_active":"2023-10-08T00:12:57.211758+0000","last_peered":"2023-10-08T00:12:57.211758+0000","last_clean":"2023-10-08T00:12:57.211758+0000","last_became_active":"2023-10-08T00:12:57.211537+0000","last_became_peered":"2023-10-08T00:12:57.211537+0000","last_unstale":"2023-10-08T00:12:57.211758+0000","last_undegraded":"2023-10-08T00:12:57.211758+0000","last_fullsized":"2023-10-08T00:12:57.211758+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:42:52.161510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.205056+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:33:58.167385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.211562+0000","last_change":"2023-10-08T00:12:57.211562+0000","last_active":"2023-10-08T00:12:57.211562+0000","last_peered":"2023-10-08T00:12:57.211562+0000","last_clean":"2023-10-08T00:12:57.211562+0000","last_became_active":"2023-10-08T00:12:57.211290+0000","last_became_peered":"2023-10-08T00:12:57.211290+0000","last_unstale":"2023-10-08T00:12:57.211562+0000","last_undegraded":"2023-10-08T00:12:57.211562+0000","last_fullsized":"2023-10-08T00:12:57.211562+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:36:52.303382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.203529+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:32:16.626313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.212893+0000","last_change":"2023-10-08T00:12:57.212893+0000","last_active":"2023-10-08T00:12:57.212893+0000","last_peered":"2023-10-08T00:12:57.212893+0000","last_clean":"2023-10-08T00:12:57.212893+0000","last_became_active":"2023-10-08T00:12:57.212632+0000","last_became_peered":"2023-10-08T00:12:57.212632+0000","last_unstale":"2023-10-08T00:12:57.212893+0000","last_undegraded":"2023-10-08T00:12:57.212893+0000","last_fullsized":"2023-10-08T00:12:57.212893+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:00:48.158147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.203581+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:56:37.046732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.214282+0000","last_change":"2023-10-08T00:12:57.214282+0000","last_active":"2023-10-08T00:12:57.214282+0000","last_peered":"2023-10-08T00:12:57.214282+0000","last_clean":"2023-10-08T00:12:57.214282+0000","last_became_active":"2023-10-08T00:12:57.214017+0000","last_became_peered":"2023-10-08T00:12:57.214017+0000","last_unstale":"2023-10-08T00:12:57.214282+0000","last_undegraded":"2023-10-08T00:12:57.214282+0000","last_fullsized":"2023-10-08T00:12:57.214282+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:52:20.648298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.203749+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:15:27.299505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.210417+0000","last_change":"2023-10-08T00:12:57.210417+0000","last_active":"2023-10-08T00:12:57.210417+0000","last_peered":"2023-10-08T00:12:57.210417+0000","last_clean":"2023-10-08T00:12:57.210417+0000","last_became_active":"2023-10-08T00:12:57.210168+0000","last_became_peered":"2023-10-08T00:12:57.210168+0000","last_unstale":"2023-10-08T00:12:57.210417+0000","last_undegraded":"2023-10-08T00:12:57.210417+0000","last_fullsized":"2023-10-08T00:12:57.210417+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:51:53.256123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206067+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:11:41.089416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.213694+0000","last_change":"2023-10-08T00:12:57.213694+0000","last_active":"2023-10-08T00:12:57.213694+0000","last_peered":"2023-10-08T00:12:57.213694+0000","last_clean":"2023-10-08T00:12:57.213694+0000","last_became_active":"2023-10-08T00:12:57.213466+0000","last_became_peered":"2023-10-08T00:12:57.213466+0000","last_unstale":"2023-10-08T00:12:57.213694+0000","last_undegraded":"2023-10-08T00:12:57.213694+0000","last_fullsized":"2023-10-08T00:12:57.213694+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:50:16.963706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.205826+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:28:53.462666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.211421+0000","last_change":"2023-10-08T00:12:57.211421+0000","last_active":"2023-10-08T00:12:57.211421+0000","last_peered":"2023-10-08T00:12:57.211421+0000","last_clean":"2023-10-08T00:12:57.211421+0000","last_became_active":"2023-10-08T00:12:57.210901+0000","last_became_peered":"2023-10-08T00:12:57.210901+0000","last_unstale":"2023-10-08T00:12:57.211421+0000","last_undegraded":"2023-10-08T00:12:57.211421+0000","last_fullsized":"2023-10-08T00:12:57.211421+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:43:05.018683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.205536+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:35:32.137447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.206333+0000","last_change":"2023-10-08T00:12:57.206333+0000","last_active":"2023-10-08T00:12:57.206333+0000","last_peered":"2023-10-08T00:12:57.206333+0000","last_clean":"2023-10-08T00:12:57.206333+0000","last_became_active":"2023-10-08T00:12:57.206123+0000","last_became_peered":"2023-10-08T00:12:57.206123+0000","last_unstale":"2023-10-08T00:12:57.206333+0000","last_undegraded":"2023-10-08T00:12:57.206333+0000","last_fullsized":"2023-10-08T00:12:57.206333+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:27:43.372597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.204798+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:47:28.589841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.212538+0000","last_change":"2023-10-08T00:12:57.212538+0000","last_active":"2023-10-08T00:12:57.212538+0000","last_peered":"2023-10-08T00:12:57.212538+0000","last_clean":"2023-10-08T00:12:57.212538+0000","last_became_active":"2023-10-08T00:12:57.212360+0000","last_became_peered":"2023-10-08T00:12:57.212360+0000","last_unstale":"2023-10-08T00:12:57.212538+0000","last_undegraded":"2023-10-08T00:12:57.212538+0000","last_fullsized":"2023-10-08T00:12:57.212538+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:39:23.590206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206666+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:40:34.751005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.213785+0000","last_change":"2023-10-08T00:12:57.213785+0000","last_active":"2023-10-08T00:12:57.213785+0000","last_peered":"2023-10-08T00:12:57.213785+0000","last_clean":"2023-10-08T00:12:57.213785+0000","last_became_active":"2023-10-08T00:12:57.213351+0000","last_became_peered":"2023-10-08T00:12:57.213351+0000","last_unstale":"2023-10-08T00:12:57.213785+0000","last_undegraded":"2023-10-08T00:12:57.213785+0000","last_fullsized":"2023-10-08T00:12:57.213785+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:17:09.287153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.205741+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:22:06.125225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.209114+0000","last_change":"2023-10-08T00:12:57.209114+0000","last_active":"2023-10-08T00:12:57.209114+0000","last_peered":"2023-10-08T00:12:57.209114+0000","last_clean":"2023-10-08T00:12:57.209114+0000","last_became_active":"2023-10-08T00:12:57.208878+0000","last_became_peered":"2023-10-08T00:12:57.208878+0000","last_unstale":"2023-10-08T00:12:57.209114+0000","last_undegraded":"2023-10-08T00:12:57.209114+0000","last_fullsized":"2023-10-08T00:12:57.209114+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:29:06.434469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.204353+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:50:14.863709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.213998+0000","last_change":"2023-10-08T00:12:57.213998+0000","last_active":"2023-10-08T00:12:57.213998+0000","last_peered":"2023-10-08T00:12:57.213998+0000","last_clean":"2023-10-08T00:12:57.213998+0000","last_became_active":"2023-10-08T00:12:57.213768+0000","last_became_peered":"2023-10-08T00:12:57.213768+0000","last_unstale":"2023-10-08T00:12:57.213998+0000","last_undegraded":"2023-10-08T00:12:57.213998+0000","last_fullsized":"2023-10-08T00:12:57.213998+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:36:23.496893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.205083+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:56:09.457506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.209066+0000","last_change":"2023-10-08T00:12:57.209066+0000","last_active":"2023-10-08T00:12:57.209066+0000","last_peered":"2023-10-08T00:12:57.209066+0000","last_clean":"2023-10-08T00:12:57.209066+0000","last_became_active":"2023-10-08T00:12:57.208849+0000","last_became_peered":"2023-10-08T00:12:57.208849+0000","last_unstale":"2023-10-08T00:12:57.209066+0000","last_undegraded":"2023-10-08T00:12:57.209066+0000","last_fullsized":"2023-10-08T00:12:57.209066+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:29:06.434469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.204310+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:39:49.309778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.212524+0000","last_change":"2023-10-08T00:12:57.212524+0000","last_active":"2023-10-08T00:12:57.212524+0000","last_peered":"2023-10-08T00:12:57.212524+0000","last_clean":"2023-10-08T00:12:57.212524+0000","last_became_active":"2023-10-08T00:12:57.212291+0000","last_became_peered":"2023-10-08T00:12:57.212291+0000","last_unstale":"2023-10-08T00:12:57.212524+0000","last_undegraded":"2023-10-08T00:12:57.212524+0000","last_fullsized":"2023-10-08T00:12:57.212524+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:00:48.158147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":199,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-08T00:12:56.201662+0000","last_change":"2023-10-08T00:10:43.728490+0000","last_active":"2023-10-08T00:12:56.201662+0000","last_peered":"2023-10-08T00:12:56.201662+0000","last_clean":"2023-10-08T00:12:56.201662+0000","last_became_active":"2023-10-08T00:10:43.728341+0000","last_became_peered":"2023-10-08T00:10:43.728341+0000","last_unstale":"2023-10-08T00:12:56.201662+0000","last_undegraded":"2023-10-08T00:12:56.201662+0000","last_fullsized":"2023-10-08T00:12:56.201662+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_clean_scrub_stamp":"2023-10-08T00:10:42.716096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:06:31.035293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":336,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194976,"statfs":{"total":200000000,"available":199655719,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.496,"15min":0.49399999999999999},"min":{"1min":0.34599999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63100000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.53800000000000003,"15min":0.54400000000000004},"min":{"1min":0.38100000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.73499999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.40300000000000002}]},{"osd":1,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.53100000000000003,"15min":0.52700000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.437,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.56999999999999995}]},{"osd":2,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.53000000000000003,"15min":0.53300000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.66600000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.51600000000000001,"15min":0.51400000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.71899999999999997,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.55900000000000005}]},{"osd":3,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.51700000000000002,"15min":0.51300000000000001},"min":{"1min":0.36599999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.55200000000000005,"15min":0.55400000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.47099999999999997}]},{"osd":4,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.52200000000000002,"15min":0.52600000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.70799999999999996,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.497},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.52800000000000002,"15min":0.53200000000000003},"min":{"1min":0.35699999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.64300000000000002,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.34599999999999997}]},{"osd":5,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.55300000000000005,"15min":0.55700000000000005},"min":{"1min":0.32100000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.71999999999999997,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.57999999999999996,"15min":0.59199999999999997},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.69399999999999995,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.58099999999999996}]},{"osd":6,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.59399999999999997,"15min":0.59899999999999998},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.71299999999999997,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.627,"15min":0.63800000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76000000000000001,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.52800000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":336,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194976,"statfs":{"total":200000000,"available":199655719,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.48099999999999998,"15min":0.47499999999999998},"min":{"1min":0.377,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.752,"5min":1.012,"15min":1.012},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.68400000000000005,"15min":0.70499999999999996},"min":{"1min":0.34000000000000002,"5min":0.317,"15min":0.317},"max":{"1min":1.2789999999999999,"5min":5.1230000000000002,"15min":5.1230000000000002},"last":0.374}]},{"osd":1,"last update":"Sun Oct 8 00:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.52100000000000002,"15min":0.51400000000000001},"min":{"1min":0.29299999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.82499999999999996,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55900000000000005,"15min":0.56000000000000005},"min":{"1min":0.371,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.69899999999999995,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.45400000000000001}]},{"osd":2,"last update":"Sun Oct 8 00:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.52700000000000002,"15min":0.52500000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.755,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56200000000000006,"15min":0.56499999999999995},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.76100000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.48899999999999999}]},{"osd":3,"last update":"Sun Oct 8 00:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53400000000000003,"15min":0.53000000000000003},"min":{"1min":0.38400000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.81399999999999995,"5min":1.046,"15min":1.046},"last":0.501},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.54400000000000004,"15min":0.53600000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.36099999999999999}]},{"osd":4,"last update":"Sun Oct 8 00:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.502,"15min":0.50600000000000001},"min":{"1min":0.32300000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56499999999999995,"15min":0.56699999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.42999999999999999}]},{"osd":5,"last update":"Sun Oct 8 00:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.48499999999999999,"15min":0.48799999999999999},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71099999999999997,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.55500000000000005,"15min":0.56299999999999994},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.439}]},{"osd":7,"last update":"Sun Oct 8 00:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.54100000000000004,"15min":0.54500000000000004},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.41699999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574877,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":912,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194399,"statfs":{"total":200000000,"available":199065351,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.30399999999999999,"15min":0.29899999999999999},"min":{"1min":0.218,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.44700000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.375,"15min":0.373},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.38300000000000001}]},{"osd":2,"last update":"Sun Oct 8 00:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.33600000000000002,"15min":0.33200000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.49099999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.379,"5min":0.33900000000000002,"15min":0.33200000000000002},"min":{"1min":0.254,"5min":0.252,"15min":0.252},"max":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.34999999999999998}]},{"osd":3,"last update":"Sun Oct 8 00:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.36699999999999999,"15min":0.36499999999999999},"min":{"1min":0.28299999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.307},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.501,"5min":0.501,"15min":0.501},"last":0.41999999999999998}]},{"osd":4,"last update":"Sun Oct 8 00:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.53700000000000003,"15min":0.53400000000000003},"min":{"1min":0.40400000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.56899999999999995,"15min":0.56499999999999995},"min":{"1min":0.41899999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.82299999999999995,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.60399999999999998}]},{"osd":5,"last update":"Sun Oct 8 00:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.53800000000000003,"15min":0.53100000000000003},"min":{"1min":0.372,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57499999999999996,"15min":0.57199999999999995},"min":{"1min":0.42999999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78300000000000003,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.73499999999999999}]},{"osd":6,"last update":"Sun Oct 8 00:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.72999999999999998,"5min":0.877,"15min":0.877},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56399999999999995,"15min":0.55800000000000005},"min":{"1min":0.42899999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.76900000000000002,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.61899999999999999}]},{"osd":7,"last update":"Sun Oct 8 00:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56299999999999994,"15min":0.55900000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.77700000000000002,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.752},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.58899999999999997,"15min":0.58499999999999996},"min":{"1min":0.432,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82299999999999995,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.69999999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":336,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194976,"statfs":{"total":200000000,"available":199655719,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 00:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.32400000000000001,"15min":0.32600000000000001},"min":{"1min":0.23499999999999999,"5min":0.193,"15min":0.193},"max":{"1min":0.41499999999999998,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.308},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.245,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.49299999999999999,"5min":0.87,"15min":0.87},"last":0.40699999999999997}]},{"osd":2,"last update":"Sun Oct 8 00:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.38200000000000001,"15min":0.38600000000000001},"min":{"1min":0.27900000000000003,"5min":0.251,"15min":0.251},"max":{"1min":0.58699999999999997,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.753,"15min":0.82399999999999995},"min":{"1min":0.247,"5min":0.185,"15min":0.185},"max":{"1min":0.40899999999999997,"5min":12.019,"15min":12.019},"last":0.38900000000000001}]},{"osd":3,"last update":"Sun Oct 8 00:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.35599999999999998,"15min":0.35999999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.47399999999999998,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.374,"5min":0.38800000000000001,"15min":0.39000000000000001},"min":{"1min":0.28499999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.55900000000000005,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.33600000000000002}]},{"osd":4,"last update":"Sun Oct 8 00:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.433,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.71899999999999997,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.624,"15min":0.625},"min":{"1min":0.48299999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.73499999999999999,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.71799999999999997}]},{"osd":5,"last update":"Sun Oct 8 00:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.999,"15min":1.069},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.79800000000000004,"5min":11.992000000000001,"15min":11.992000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.73199999999999998,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.61299999999999999}]},{"osd":6,"last update":"Sun Oct 8 00:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.74399999999999999,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.97799999999999998,"15min":1.042},"min":{"1min":0.47299999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76000000000000001,"5min":11.927,"15min":11.927},"last":0.72499999999999998}]},{"osd":7,"last update":"Sun Oct 8 00:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.999,"15min":1.0660000000000001},"min":{"1min":0.434,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82199999999999995,"5min":11.965,"15min":11.965},"last":0.627},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.61899999999999999,"15min":0.621},"min":{"1min":0.48199999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.76100000000000001,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.76900000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":336,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194976,"statfs":{"total":200000000,"available":199655719,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.39000000000000001,"15min":0.40200000000000002},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.437,"5min":2.5139999999999998,"15min":2.5139999999999998},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.40100000000000002,"15min":0.41399999999999998},"min":{"1min":0.248,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.44400000000000001,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":0.35899999999999999}]},{"osd":1,"last update":"Sun Oct 8 00:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.39000000000000001,"15min":0.40200000000000002},"min":{"1min":0.25,"5min":0.246,"15min":0.246},"max":{"1min":0.40100000000000002,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.436,"15min":0.45000000000000001},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.48999999999999999,"5min":2.5379999999999998,"15min":2.5379999999999998},"last":0.39200000000000002}]},{"osd":3,"last update":"Sun Oct 8 00:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.42799999999999999,"15min":0.439},"min":{"1min":0.28499999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.46500000000000002,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":0.378},{"interface":"front","average":{"1min":0.379,"5min":0.46300000000000002,"15min":0.47699999999999998},"min":{"1min":0.315,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.51200000000000001,"5min":2.5619999999999998,"15min":2.5619999999999998},"last":0.41499999999999998}]},{"osd":4,"last update":"Sun Oct 8 00:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.61699999999999999,"15min":0.628},"min":{"1min":0.45800000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.84599999999999997,"5min":2.488,"15min":2.488},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.60399999999999998,"15min":0.60999999999999999},"min":{"1min":0.41099999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85999999999999999,"5min":2.073,"15min":2.073},"last":0.81100000000000005}]},{"osd":5,"last update":"Sun Oct 8 00:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.66300000000000003,"15min":0.66900000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.2130000000000001,"5min":2.2069999999999999,"15min":2.2069999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.62,"5min":0.66300000000000003,"15min":0.67000000000000004},"min":{"1min":0.48799999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0149999999999999,"5min":2.456,"15min":2.456},"last":0.81999999999999995}]},{"osd":6,"last update":"Sun Oct 8 00:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.621,"15min":0.627},"min":{"1min":0.45000000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.79500000000000004,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.60899999999999999,"15min":0.61399999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.97499999999999998,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.77000000000000002}]},{"osd":7,"last update":"Sun Oct 8 00:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.63700000000000001,"15min":0.64100000000000001},"min":{"1min":0.499,"5min":0.438,"15min":0.438},"max":{"1min":0.81699999999999995,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.626,"15min":0.63200000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91200000000000003,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":0.64500000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":330,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194982,"statfs":{"total":200000000,"available":199661982,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.28699999999999998,"5min":0.36699999999999999,"15min":0.38},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.45100000000000001,"5min":1.46,"15min":1.46},"last":0.251},{"interface":"front","average":{"1min":0.29699999999999999,"5min":0.33500000000000002,"15min":0.34100000000000003},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.46300000000000002,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.23300000000000001}]},{"osd":1,"last update":"Sun Oct 8 00:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.82999999999999996,"15min":0.91700000000000004},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.44800000000000001,"5min":12.351000000000001,"15min":12.351000000000001},"last":0.307},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.35899999999999999,"15min":0.36299999999999999},"min":{"1min":0.26500000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.443,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.34300000000000003}]},{"osd":2,"last update":"Sun Oct 8 00:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.41399999999999998,"15min":0.42499999999999999},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.47299999999999998,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.377},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.379,"15min":0.38300000000000001},"min":{"1min":0.27800000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.45900000000000002,"5min":0.753,"15min":0.753},"last":0.41099999999999998}]},{"osd":4,"last update":"Sun Oct 8 00:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":1.004,"15min":1.0860000000000001},"min":{"1min":0.38800000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.75600000000000001,"5min":11.333,"15min":11.333},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":1.0369999999999999,"15min":1.1200000000000001},"min":{"1min":0.434,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.77000000000000002,"5min":12.321999999999999,"15min":12.321999999999999},"last":0.63200000000000001}]},{"osd":5,"last update":"Sun Oct 8 00:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":1.0509999999999999,"15min":1.137},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.72599999999999998,"5min":12.291,"15min":12.291},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.58599999999999997,"15min":0.59299999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.74299999999999999,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.59699999999999998}]},{"osd":6,"last update":"Sun Oct 8 00:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.96999999999999997,"15min":1.048},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.69899999999999995,"5min":11.288,"15min":11.288},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.51800000000000002,"5min":1.0309999999999999,"15min":1.117},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.66000000000000003,"5min":12.253,"15min":12.253},"last":0.67500000000000004}]},{"osd":7,"last update":"Sun Oct 8 00:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56599999999999995,"15min":0.56799999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.56399999999999995,"5min":1.012,"15min":1.0860000000000001},"min":{"1min":0.435,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.381,"5min":11.363,"15min":11.363},"last":0.66600000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":336,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194976,"statfs":{"total":200000000,"available":199655719,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.56000000000000005,"15min":0.55400000000000005},"min":{"1min":0.41299999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.375},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.50600000000000001}]},{"osd":1,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51700000000000002,"15min":0.51800000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.64100000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.42099999999999999}]},{"osd":2,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.51600000000000001,"15min":0.52000000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.51300000000000001,"15min":0.50900000000000001},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.54200000000000004}]},{"osd":3,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.53500000000000003,"15min":0.53900000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.70899999999999996,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.53000000000000003,"15min":0.53200000000000003},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.40200000000000002}]},{"osd":5,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.49299999999999999,"15min":0.503},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69899999999999995,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.50800000000000001,"15min":0.51400000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.71099999999999997,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.57199999999999995}]},{"osd":6,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.48899999999999999,"15min":0.49399999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.55200000000000005,"15min":0.56200000000000006},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.754,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.58599999999999997}]},{"osd":7,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.54600000000000004,"15min":0.55500000000000005},"min":{"1min":0.36299999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.64200000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.57799999999999996,"15min":0.58299999999999996},"min":{"1min":0.41199999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.77700000000000002,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.628}]}]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":906,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194405,"statfs":{"total":200000000,"available":199071614,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.48699999999999999,"15min":0.48799999999999999},"min":{"1min":0.32700000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68799999999999994,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.441},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.55700000000000005,"15min":0.56299999999999994},"min":{"1min":0.374,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.68600000000000005,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.58599999999999997}]},{"osd":1,"last update":"Sun Oct 8 00:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.52100000000000002,"15min":0.52300000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.65800000000000003,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.52400000000000002,"15min":0.52800000000000002},"min":{"1min":0.39300000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.70899999999999996,"5min":0.755,"15min":0.755},"last":0.48999999999999999}]},{"osd":2,"last update":"Sun Oct 8 00:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.52900000000000003,"15min":0.53100000000000003},"min":{"1min":0.40500000000000003,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.64500000000000002,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.56599999999999995,"15min":0.57199999999999995},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.70699999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.59799999999999998}]},{"osd":3,"last update":"Sun Oct 8 00:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54300000000000004,"15min":0.54500000000000004},"min":{"1min":0.38700000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.67100000000000004,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.57999999999999996,"15min":0.58399999999999996},"min":{"1min":0.42799999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.72299999999999998,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.55900000000000005}]},{"osd":4,"last update":"Sun Oct 8 00:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.47299999999999998,"15min":0.47499999999999998},"min":{"1min":0.35799999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.626,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.50900000000000001,"15min":0.51300000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.63700000000000001,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.54600000000000004}]},{"osd":6,"last update":"Sun Oct 8 00:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.499,"15min":0.499},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.61899999999999999,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.66800000000000004,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.53200000000000003}]},{"osd":7,"last update":"Sun Oct 8 00:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.51300000000000001,"15min":0.51500000000000001},"min":{"1min":0.33400000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67300000000000004,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.69799999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.51600000000000001}]}]}],"pool_statfs":[]}} 2023-10-08T00:13:02.649 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T00:13:02.797 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.797+0000 7f713b4ca700 1 Processor -- start 2023-10-08T00:13:02.798 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.797+0000 7f713b4ca700 1 -- start start 2023-10-08T00:13:02.800 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.798+0000 7f713b4ca700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7134157870 0x7f713415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:02.800 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.798+0000 7f713b4ca700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f713415a620 0x7f713415d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:02.800 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.798+0000 7f713b4ca700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f713415d580 0x7f7134158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:02.800 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.798+0000 7f713b4ca700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f713415cbb0 con 0x7f713415d580 2023-10-08T00:13:02.800 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.798+0000 7f713b4ca700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f713415bfa0 con 0x7f7134157870 2023-10-08T00:13:02.800 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.798+0000 7f713b4ca700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f713415c280 con 0x7f713415a620 2023-10-08T00:13:02.800 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.798+0000 7f7139a67700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f713415d580 0x7f7134158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:02.801 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.798+0000 7f7139a67700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f713415d580 0x7f7134158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:32934/0 (socket says 172.21.15.99:32934) 2023-10-08T00:13:02.801 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.798+0000 7f7139a67700 1 -- 172.21.15.99:0/3122532514 learned_addr learned my addr 172.21.15.99:0/3122532514 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:13:02.801 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.798+0000 7f7139266700 1 --2- 172.21.15.99:0/3122532514 >> v2:172.21.15.191:3300/0 conn(0x7f7134157870 0x7f713415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:02.801 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.798+0000 7f7138a65700 1 --2- 172.21.15.99:0/3122532514 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f713415a620 0x7f713415d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:02.801 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.798+0000 7f7139a67700 1 -- 172.21.15.99:0/3122532514 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f713415a620 msgr2=0x7f713415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:13:02.801 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.798+0000 7f7139a67700 1 --2- 172.21.15.99:0/3122532514 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f713415a620 0x7f713415d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:02.801 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.798+0000 7f7139a67700 1 -- 172.21.15.99:0/3122532514 >> v2:172.21.15.191:3300/0 conn(0x7f7134157870 msgr2=0x7f713415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:13:02.801 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.798+0000 7f7139a67700 1 --2- 172.21.15.99:0/3122532514 >> v2:172.21.15.191:3300/0 conn(0x7f7134157870 0x7f713415b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:02.801 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.798+0000 7f7139a67700 1 -- 172.21.15.99:0/3122532514 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f713415c560 con 0x7f713415d580 2023-10-08T00:13:02.801 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.798+0000 7f7139a67700 1 --2- 172.21.15.99:0/3122532514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f713415d580 0x7f7134158110 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f7134069d80 tx=0x7f712c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2232499d9e6daf7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:13:02.801 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.799+0000 7f712bfff700 1 -- 172.21.15.99:0/3122532514 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712c00e070 con 0x7f713415d580 2023-10-08T00:13:02.801 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.799+0000 7f712bfff700 1 -- 172.21.15.99:0/3122532514 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f712c00b970 con 0x7f713415d580 2023-10-08T00:13:02.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.799+0000 7f712bfff700 1 -- 172.21.15.99:0/3122532514 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712c0092e0 con 0x7f713415d580 2023-10-08T00:13:02.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.799+0000 7f713b4ca700 1 -- 172.21.15.99:0/3122532514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f713415d580 msgr2=0x7f7134158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:02.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.799+0000 7f713b4ca700 1 --2- 172.21.15.99:0/3122532514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f713415d580 0x7f7134158110 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f7134069d80 tx=0x7f712c009840 comp rx=0 tx=0).stop 2023-10-08T00:13:02.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.799+0000 7f713b4ca700 1 -- 172.21.15.99:0/3122532514 shutdown_connections 2023-10-08T00:13:02.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.799+0000 7f713b4ca700 1 --2- 172.21.15.99:0/3122532514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f713415d580 0x7f7134158110 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:02.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.799+0000 7f713b4ca700 1 --2- 172.21.15.99:0/3122532514 >> v2:172.21.15.191:3300/0 conn(0x7f7134157870 0x7f713415b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:02.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.799+0000 7f713b4ca700 1 --2- 172.21.15.99:0/3122532514 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f713415a620 0x7f713415d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:02.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.799+0000 7f713b4ca700 1 -- 172.21.15.99:0/3122532514 >> 172.21.15.99:0/3122532514 conn(0x7f7134068ce0 msgr2=0x7f71340690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:13:02.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.799+0000 7f713b4ca700 1 -- 172.21.15.99:0/3122532514 shutdown_connections 2023-10-08T00:13:02.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.799+0000 7f713b4ca700 1 -- 172.21.15.99:0/3122532514 wait complete. 2023-10-08T00:13:02.805 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.800+0000 7f713b4ca700 1 Processor -- start 2023-10-08T00:13:02.805 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.800+0000 7f713b4ca700 1 -- start start 2023-10-08T00:13:02.805 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.801+0000 7f713b4ca700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7134157870 0x7f71340d11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:02.805 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.801+0000 7f713b4ca700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f713415a620 0x7f71340cfc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:02.805 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.801+0000 7f713b4ca700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f713415d580 0x7f713414b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:02.805 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.801+0000 7f713b4ca700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71340b06c0 con 0x7f713415a620 2023-10-08T00:13:02.805 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.801+0000 7f713b4ca700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71340d0190 con 0x7f713415d580 2023-10-08T00:13:02.805 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.801+0000 7f713b4ca700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f713414bf20 con 0x7f7134157870 2023-10-08T00:13:02.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.801+0000 7f7138a65700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f713415a620 0x7f71340cfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:02.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.801+0000 7f7138a65700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f713415a620 0x7f71340cfc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:32936/0 (socket says 172.21.15.99:32936) 2023-10-08T00:13:02.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.801+0000 7f7138a65700 1 -- 172.21.15.99:0/2134123202 learned_addr learned my addr 172.21.15.99:0/2134123202 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:13:02.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.801+0000 7f7139a67700 1 --2- 172.21.15.99:0/2134123202 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f713415d580 0x7f713414b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:02.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.801+0000 7f7139266700 1 --2- 172.21.15.99:0/2134123202 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7134157870 0x7f71340d11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:02.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.802+0000 7f7138a65700 1 -- 172.21.15.99:0/2134123202 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7134157870 msgr2=0x7f71340d11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:02.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.802+0000 7f7138a65700 1 --2- 172.21.15.99:0/2134123202 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7134157870 0x7f71340d11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:02.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.802+0000 7f7138a65700 1 -- 172.21.15.99:0/2134123202 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f713415d580 msgr2=0x7f713414b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:02.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.802+0000 7f7138a65700 1 --2- 172.21.15.99:0/2134123202 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f713415d580 0x7f713414b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:02.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.802+0000 7f7138a65700 1 -- 172.21.15.99:0/2134123202 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f712c009ae0 con 0x7f713415a620 2023-10-08T00:13:02.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.802+0000 7f7138a65700 1 --2- 172.21.15.99:0/2134123202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f713415a620 0x7f71340cfc50 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f712400ea70 tx=0x7f71240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:13:02.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.802+0000 7f71297fa700 1 -- 172.21.15.99:0/2134123202 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712400eeb0 con 0x7f713415a620 2023-10-08T00:13:02.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.802+0000 7f71297fa700 1 -- 172.21.15.99:0/2134123202 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f712400c850 con 0x7f713415a620 2023-10-08T00:13:02.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.802+0000 7f713b4ca700 1 -- 172.21.15.99:0/2134123202 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71340d0400 con 0x7f713415a620 2023-10-08T00:13:02.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.802+0000 7f71297fa700 1 -- 172.21.15.99:0/2134123202 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712400cc30 con 0x7f713415a620 2023-10-08T00:13:02.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.802+0000 7f713b4ca700 1 -- 172.21.15.99:0/2134123202 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71340b0910 con 0x7f713415a620 2023-10-08T00:13:02.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.803+0000 7f713b4ca700 1 -- 172.21.15.99:0/2134123202 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f713404c3c0 con 0x7f713415a620 2023-10-08T00:13:02.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.803+0000 7f71297fa700 1 -- 172.21.15.99:0/2134123202 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7124013680 con 0x7f713415a620 2023-10-08T00:13:02.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.804+0000 7f71297fa700 1 --2- 172.21.15.99:0/2134123202 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7118066f30 0x7f71180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:02.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.804+0000 7f71297fa700 1 -- 172.21.15.99:0/2134123202 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7124084760 con 0x7f713415a620 2023-10-08T00:13:02.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.804+0000 7f7139266700 1 --2- 172.21.15.99:0/2134123202 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7118066f30 0x7f71180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:02.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.804+0000 7f7139266700 1 --2- 172.21.15.99:0/2134123202 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7118066f30 0x7f71180693e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7130009db0 tx=0x7f71300091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:13:02.808 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.808+0000 7f71297fa700 1 -- 172.21.15.99:0/2134123202 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f71240519d0 con 0x7f713415a620 2023-10-08T00:13:02.984 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.983+0000 7f713b4ca700 1 -- 172.21.15.99:0/2134123202 --> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f713415c840 con 0x7f7118066f30 2023-10-08T00:13:02.992 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.992+0000 7f71297fa700 1 -- 172.21.15.99:0/2134123202 <== mgr.4099 v2:172.21.15.191:6832/85985 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+281850 (secure 0 0 0) 0x7f713415c840 con 0x7f7118066f30 2023-10-08T00:13:02.996 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:13:02.998 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.995+0000 7f713b4ca700 1 -- 172.21.15.99:0/2134123202 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7118066f30 msgr2=0x7f71180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:02.998 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.995+0000 7f713b4ca700 1 --2- 172.21.15.99:0/2134123202 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7118066f30 0x7f71180693e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7130009db0 tx=0x7f71300091a0 comp rx=0 tx=0).stop 2023-10-08T00:13:02.998 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.995+0000 7f713b4ca700 1 -- 172.21.15.99:0/2134123202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f713415a620 msgr2=0x7f71340cfc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:02.998 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.995+0000 7f713b4ca700 1 --2- 172.21.15.99:0/2134123202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f713415a620 0x7f71340cfc50 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f712400ea70 tx=0x7f71240091a0 comp rx=0 tx=0).stop 2023-10-08T00:13:02.999 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.995+0000 7f713b4ca700 1 -- 172.21.15.99:0/2134123202 shutdown_connections 2023-10-08T00:13:02.999 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.995+0000 7f713b4ca700 1 --2- 172.21.15.99:0/2134123202 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7118066f30 0x7f71180693e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:02.999 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.995+0000 7f713b4ca700 1 --2- 172.21.15.99:0/2134123202 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f713415d580 0x7f713414b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:02.999 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.995+0000 7f713b4ca700 1 --2- 172.21.15.99:0/2134123202 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7134157870 0x7f71340d11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:02.999 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.995+0000 7f713b4ca700 1 --2- 172.21.15.99:0/2134123202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f713415a620 0x7f71340cfc50 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:02.999 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.995+0000 7f713b4ca700 1 -- 172.21.15.99:0/2134123202 >> 172.21.15.99:0/2134123202 conn(0x7f7134068ce0 msgr2=0x7f713414c280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:13:02.999 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.995+0000 7f713b4ca700 1 -- 172.21.15.99:0/2134123202 shutdown_connections 2023-10-08T00:13:02.999 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:02.995+0000 7f713b4ca700 1 -- 172.21.15.99:0/2134123202 wait complete. 2023-10-08T00:13:02.999 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2023-10-08T00:13:03.007 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":129,"stamp":"2023-10-08T00:13:02.053575+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":3830,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1558669,"statfs":{"total":1600000000,"available":1596077542,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.858362"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.207886+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:42:53.170735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206623+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:29:57.147801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206756+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:15:27.299505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.204518+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:17:47.918131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.208328+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:29:07.443694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.207050+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:19:19.113722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.204971+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:18:06.799190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.214505+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:51:20.513160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.204229+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:57:53.151755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.207215+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:03:56.099847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.207550+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:03:56.099847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.203197+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:04:57.344499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206586+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:36:53.312606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206510+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:56:37.046732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.205079+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:39:24.599431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.207493+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:39:24.599431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206258+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:00:49.167372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206438+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:33:06.564584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.205454+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:47:44.758644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.208478+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:22:06.125225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206453+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:29:07.443694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.209033+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:42:53.170735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.209265+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:56:15.157209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.203882+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:50:17.972931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206009+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:36:24.506117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206105+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:42:53.170735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206249+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:43:06.027907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.213400+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:57:11.917467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.208004+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:17:37.423308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.205536+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:28:33.161628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206770+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:27.657370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.209403+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:29:07.443694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.205320+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:33:16.516139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.213303+0000","last_change":"2023-10-08T00:12:57.213303+0000","last_active":"2023-10-08T00:12:57.213303+0000","last_peered":"2023-10-08T00:12:57.213303+0000","last_clean":"2023-10-08T00:12:57.213303+0000","last_became_active":"2023-10-08T00:12:57.213092+0000","last_became_peered":"2023-10-08T00:12:57.213092+0000","last_unstale":"2023-10-08T00:12:57.213303+0000","last_undegraded":"2023-10-08T00:12:57.213303+0000","last_fullsized":"2023-10-08T00:12:57.213303+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:40:33.741781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.205824+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:51:47.982127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.212771+0000","last_change":"2023-10-08T00:12:57.212771+0000","last_active":"2023-10-08T00:12:57.212771+0000","last_peered":"2023-10-08T00:12:57.212771+0000","last_clean":"2023-10-08T00:12:57.212771+0000","last_became_active":"2023-10-08T00:12:57.212479+0000","last_became_peered":"2023-10-08T00:12:57.212479+0000","last_unstale":"2023-10-08T00:12:57.212771+0000","last_undegraded":"2023-10-08T00:12:57.212771+0000","last_fullsized":"2023-10-08T00:12:57.212771+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:51:18.660991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.207674+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:13:42.175367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.209653+0000","last_change":"2023-10-08T00:12:57.209653+0000","last_active":"2023-10-08T00:12:57.209653+0000","last_peered":"2023-10-08T00:12:57.209653+0000","last_clean":"2023-10-08T00:12:57.209653+0000","last_became_active":"2023-10-08T00:12:57.209408+0000","last_became_peered":"2023-10-08T00:12:57.209408+0000","last_unstale":"2023-10-08T00:12:57.209653+0000","last_undegraded":"2023-10-08T00:12:57.209653+0000","last_fullsized":"2023-10-08T00:12:57.209653+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:26.648145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.207619+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:22:06.125225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.207927+0000","last_change":"2023-10-08T00:12:57.207927+0000","last_active":"2023-10-08T00:12:57.207927+0000","last_peered":"2023-10-08T00:12:57.207927+0000","last_clean":"2023-10-08T00:12:57.207927+0000","last_became_active":"2023-10-08T00:12:57.207442+0000","last_became_peered":"2023-10-08T00:12:57.207442+0000","last_unstale":"2023-10-08T00:12:57.207927+0000","last_undegraded":"2023-10-08T00:12:57.207927+0000","last_fullsized":"2023-10-08T00:12:57.207927+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:57:10.908242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.204176+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:24:53.646854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.212292+0000","last_change":"2023-10-08T00:12:57.212292+0000","last_active":"2023-10-08T00:12:57.212292+0000","last_peered":"2023-10-08T00:12:57.212292+0000","last_clean":"2023-10-08T00:12:57.212292+0000","last_became_active":"2023-10-08T00:12:57.211867+0000","last_became_peered":"2023-10-08T00:12:57.211867+0000","last_unstale":"2023-10-08T00:12:57.212292+0000","last_undegraded":"2023-10-08T00:12:57.212292+0000","last_fullsized":"2023-10-08T00:12:57.212292+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:36:52.303382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.208400+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:39:24.599431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.211573+0000","last_change":"2023-10-08T00:12:57.211573+0000","last_active":"2023-10-08T00:12:57.211573+0000","last_peered":"2023-10-08T00:12:57.211573+0000","last_clean":"2023-10-08T00:12:57.211573+0000","last_became_active":"2023-10-08T00:12:57.211202+0000","last_became_peered":"2023-10-08T00:12:57.211202+0000","last_unstale":"2023-10-08T00:12:57.211573+0000","last_undegraded":"2023-10-08T00:12:57.211573+0000","last_fullsized":"2023-10-08T00:12:57.211573+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:29:56.138577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.208004+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:47:28.589841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.205376+0000","last_change":"2023-10-08T00:12:57.205376+0000","last_active":"2023-10-08T00:12:57.205376+0000","last_peered":"2023-10-08T00:12:57.205376+0000","last_clean":"2023-10-08T00:12:57.205376+0000","last_became_active":"2023-10-08T00:12:57.205158+0000","last_became_peered":"2023-10-08T00:12:57.205158+0000","last_unstale":"2023-10-08T00:12:57.205376+0000","last_undegraded":"2023-10-08T00:12:57.205376+0000","last_fullsized":"2023-10-08T00:12:57.205376+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:15:43.196718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.204806+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:22:04.500698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.214270+0000","last_change":"2023-10-08T00:12:57.214270+0000","last_active":"2023-10-08T00:12:57.214270+0000","last_peered":"2023-10-08T00:12:57.214270+0000","last_clean":"2023-10-08T00:12:57.214270+0000","last_became_active":"2023-10-08T00:12:57.214082+0000","last_became_peered":"2023-10-08T00:12:57.214082+0000","last_unstale":"2023-10-08T00:12:57.214270+0000","last_undegraded":"2023-10-08T00:12:57.214270+0000","last_fullsized":"2023-10-08T00:12:57.214270+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:40:33.741781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.204948+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:03:56.099847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.209199+0000","last_change":"2023-10-08T00:12:57.209199+0000","last_active":"2023-10-08T00:12:57.209199+0000","last_peered":"2023-10-08T00:12:57.209199+0000","last_clean":"2023-10-08T00:12:57.209199+0000","last_became_active":"2023-10-08T00:12:57.208965+0000","last_became_peered":"2023-10-08T00:12:57.208965+0000","last_unstale":"2023-10-08T00:12:57.209199+0000","last_undegraded":"2023-10-08T00:12:57.209199+0000","last_fullsized":"2023-10-08T00:12:57.209199+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:02:39.029498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.207557+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:39:49.309778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.214795+0000","last_change":"2023-10-08T00:12:57.214795+0000","last_active":"2023-10-08T00:12:57.214795+0000","last_peered":"2023-10-08T00:12:57.214795+0000","last_clean":"2023-10-08T00:12:57.214795+0000","last_became_active":"2023-10-08T00:12:57.214269+0000","last_became_peered":"2023-10-08T00:12:57.214269+0000","last_unstale":"2023-10-08T00:12:57.214795+0000","last_undegraded":"2023-10-08T00:12:57.214795+0000","last_fullsized":"2023-10-08T00:12:57.214795+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:28:52.453441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206370+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:52:28.504130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.210087+0000","last_change":"2023-10-08T00:12:57.210087+0000","last_active":"2023-10-08T00:12:57.210087+0000","last_peered":"2023-10-08T00:12:57.210087+0000","last_clean":"2023-10-08T00:12:57.210087+0000","last_became_active":"2023-10-08T00:12:57.209574+0000","last_became_peered":"2023-10-08T00:12:57.209574+0000","last_unstale":"2023-10-08T00:12:57.210087+0000","last_undegraded":"2023-10-08T00:12:57.210087+0000","last_fullsized":"2023-10-08T00:12:57.210087+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:51:19.503936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.208930+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:44:29.429854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.209678+0000","last_change":"2023-10-08T00:12:57.209678+0000","last_active":"2023-10-08T00:12:57.209678+0000","last_peered":"2023-10-08T00:12:57.209678+0000","last_clean":"2023-10-08T00:12:57.209678+0000","last_became_active":"2023-10-08T00:12:57.209454+0000","last_became_peered":"2023-10-08T00:12:57.209454+0000","last_unstale":"2023-10-08T00:12:57.209678+0000","last_undegraded":"2023-10-08T00:12:57.209678+0000","last_fullsized":"2023-10-08T00:12:57.209678+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:51:46.972903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.203919+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:17:10.296378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.215576+0000","last_change":"2023-10-08T00:12:57.215576+0000","last_active":"2023-10-08T00:12:57.215576+0000","last_peered":"2023-10-08T00:12:57.215576+0000","last_clean":"2023-10-08T00:12:57.215576+0000","last_became_active":"2023-10-08T00:12:57.215200+0000","last_became_peered":"2023-10-08T00:12:57.215200+0000","last_unstale":"2023-10-08T00:12:57.215576+0000","last_undegraded":"2023-10-08T00:12:57.215576+0000","last_fullsized":"2023-10-08T00:12:57.215576+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:11:40.080191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.207660+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:50:14.863709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.208075+0000","last_change":"2023-10-08T00:12:57.208075+0000","last_active":"2023-10-08T00:12:57.208075+0000","last_peered":"2023-10-08T00:12:57.208075+0000","last_clean":"2023-10-08T00:12:57.208075+0000","last_became_active":"2023-10-08T00:12:57.207720+0000","last_became_peered":"2023-10-08T00:12:57.207720+0000","last_unstale":"2023-10-08T00:12:57.208075+0000","last_undegraded":"2023-10-08T00:12:57.208075+0000","last_fullsized":"2023-10-08T00:12:57.208075+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:23:41.947350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.203096+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:43:33.796004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.204277+0000","last_change":"2023-10-08T00:12:57.204277+0000","last_active":"2023-10-08T00:12:57.204277+0000","last_peered":"2023-10-08T00:12:57.204277+0000","last_clean":"2023-10-08T00:12:57.204277+0000","last_became_active":"2023-10-08T00:12:57.204039+0000","last_became_peered":"2023-10-08T00:12:57.204039+0000","last_unstale":"2023-10-08T00:12:57.204277+0000","last_undegraded":"2023-10-08T00:12:57.204277+0000","last_fullsized":"2023-10-08T00:12:57.204277+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:09:18.935331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.204614+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:04:37.043481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.210213+0000","last_change":"2023-10-08T00:12:57.210213+0000","last_active":"2023-10-08T00:12:57.210213+0000","last_peered":"2023-10-08T00:12:57.210213+0000","last_clean":"2023-10-08T00:12:57.210213+0000","last_became_active":"2023-10-08T00:12:57.209897+0000","last_became_peered":"2023-10-08T00:12:57.209897+0000","last_unstale":"2023-10-08T00:12:57.210213+0000","last_undegraded":"2023-10-08T00:12:57.210213+0000","last_fullsized":"2023-10-08T00:12:57.210213+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:49:05.184189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.207309+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:24:53.646854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.210506+0000","last_change":"2023-10-08T00:12:57.210506+0000","last_active":"2023-10-08T00:12:57.210506+0000","last_peered":"2023-10-08T00:12:57.210506+0000","last_clean":"2023-10-08T00:12:57.210506+0000","last_became_active":"2023-10-08T00:12:57.210180+0000","last_became_peered":"2023-10-08T00:12:57.210180+0000","last_unstale":"2023-10-08T00:12:57.210506+0000","last_undegraded":"2023-10-08T00:12:57.210506+0000","last_fullsized":"2023-10-08T00:12:57.210506+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:57:52.142531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206202+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:37:04.823729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.215419+0000","last_change":"2023-10-08T00:12:57.215419+0000","last_active":"2023-10-08T00:12:57.215419+0000","last_peered":"2023-10-08T00:12:57.215419+0000","last_clean":"2023-10-08T00:12:57.215419+0000","last_became_active":"2023-10-08T00:12:57.214895+0000","last_became_peered":"2023-10-08T00:12:57.214895+0000","last_unstale":"2023-10-08T00:12:57.215419+0000","last_undegraded":"2023-10-08T00:12:57.215419+0000","last_fullsized":"2023-10-08T00:12:57.215419+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:59:43.036033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.205871+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:25:36.388792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.211758+0000","last_change":"2023-10-08T00:12:57.211758+0000","last_active":"2023-10-08T00:12:57.211758+0000","last_peered":"2023-10-08T00:12:57.211758+0000","last_clean":"2023-10-08T00:12:57.211758+0000","last_became_active":"2023-10-08T00:12:57.211537+0000","last_became_peered":"2023-10-08T00:12:57.211537+0000","last_unstale":"2023-10-08T00:12:57.211758+0000","last_undegraded":"2023-10-08T00:12:57.211758+0000","last_fullsized":"2023-10-08T00:12:57.211758+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:42:52.161510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.205056+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:33:58.167385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.211562+0000","last_change":"2023-10-08T00:12:57.211562+0000","last_active":"2023-10-08T00:12:57.211562+0000","last_peered":"2023-10-08T00:12:57.211562+0000","last_clean":"2023-10-08T00:12:57.211562+0000","last_became_active":"2023-10-08T00:12:57.211290+0000","last_became_peered":"2023-10-08T00:12:57.211290+0000","last_unstale":"2023-10-08T00:12:57.211562+0000","last_undegraded":"2023-10-08T00:12:57.211562+0000","last_fullsized":"2023-10-08T00:12:57.211562+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:36:52.303382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.203529+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:32:16.626313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.212893+0000","last_change":"2023-10-08T00:12:57.212893+0000","last_active":"2023-10-08T00:12:57.212893+0000","last_peered":"2023-10-08T00:12:57.212893+0000","last_clean":"2023-10-08T00:12:57.212893+0000","last_became_active":"2023-10-08T00:12:57.212632+0000","last_became_peered":"2023-10-08T00:12:57.212632+0000","last_unstale":"2023-10-08T00:12:57.212893+0000","last_undegraded":"2023-10-08T00:12:57.212893+0000","last_fullsized":"2023-10-08T00:12:57.212893+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:00:48.158147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.203581+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:56:37.046732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.214282+0000","last_change":"2023-10-08T00:12:57.214282+0000","last_active":"2023-10-08T00:12:57.214282+0000","last_peered":"2023-10-08T00:12:57.214282+0000","last_clean":"2023-10-08T00:12:57.214282+0000","last_became_active":"2023-10-08T00:12:57.214017+0000","last_became_peered":"2023-10-08T00:12:57.214017+0000","last_unstale":"2023-10-08T00:12:57.214282+0000","last_undegraded":"2023-10-08T00:12:57.214282+0000","last_fullsized":"2023-10-08T00:12:57.214282+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:52:20.648298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.203749+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:15:27.299505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.210417+0000","last_change":"2023-10-08T00:12:57.210417+0000","last_active":"2023-10-08T00:12:57.210417+0000","last_peered":"2023-10-08T00:12:57.210417+0000","last_clean":"2023-10-08T00:12:57.210417+0000","last_became_active":"2023-10-08T00:12:57.210168+0000","last_became_peered":"2023-10-08T00:12:57.210168+0000","last_unstale":"2023-10-08T00:12:57.210417+0000","last_undegraded":"2023-10-08T00:12:57.210417+0000","last_fullsized":"2023-10-08T00:12:57.210417+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:51:53.256123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206067+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:11:41.089416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.213694+0000","last_change":"2023-10-08T00:12:57.213694+0000","last_active":"2023-10-08T00:12:57.213694+0000","last_peered":"2023-10-08T00:12:57.213694+0000","last_clean":"2023-10-08T00:12:57.213694+0000","last_became_active":"2023-10-08T00:12:57.213466+0000","last_became_peered":"2023-10-08T00:12:57.213466+0000","last_unstale":"2023-10-08T00:12:57.213694+0000","last_undegraded":"2023-10-08T00:12:57.213694+0000","last_fullsized":"2023-10-08T00:12:57.213694+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:50:16.963706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.205826+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:28:53.462666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.211421+0000","last_change":"2023-10-08T00:12:57.211421+0000","last_active":"2023-10-08T00:12:57.211421+0000","last_peered":"2023-10-08T00:12:57.211421+0000","last_clean":"2023-10-08T00:12:57.211421+0000","last_became_active":"2023-10-08T00:12:57.210901+0000","last_became_peered":"2023-10-08T00:12:57.210901+0000","last_unstale":"2023-10-08T00:12:57.211421+0000","last_undegraded":"2023-10-08T00:12:57.211421+0000","last_fullsized":"2023-10-08T00:12:57.211421+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:43:05.018683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.205536+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:35:32.137447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.206333+0000","last_change":"2023-10-08T00:12:57.206333+0000","last_active":"2023-10-08T00:12:57.206333+0000","last_peered":"2023-10-08T00:12:57.206333+0000","last_clean":"2023-10-08T00:12:57.206333+0000","last_became_active":"2023-10-08T00:12:57.206123+0000","last_became_peered":"2023-10-08T00:12:57.206123+0000","last_unstale":"2023-10-08T00:12:57.206333+0000","last_undegraded":"2023-10-08T00:12:57.206333+0000","last_fullsized":"2023-10-08T00:12:57.206333+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:27:43.372597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.204798+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:47:28.589841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.212538+0000","last_change":"2023-10-08T00:12:57.212538+0000","last_active":"2023-10-08T00:12:57.212538+0000","last_peered":"2023-10-08T00:12:57.212538+0000","last_clean":"2023-10-08T00:12:57.212538+0000","last_became_active":"2023-10-08T00:12:57.212360+0000","last_became_peered":"2023-10-08T00:12:57.212360+0000","last_unstale":"2023-10-08T00:12:57.212538+0000","last_undegraded":"2023-10-08T00:12:57.212538+0000","last_fullsized":"2023-10-08T00:12:57.212538+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:39:23.590206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206666+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:40:34.751005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.213785+0000","last_change":"2023-10-08T00:12:57.213785+0000","last_active":"2023-10-08T00:12:57.213785+0000","last_peered":"2023-10-08T00:12:57.213785+0000","last_clean":"2023-10-08T00:12:57.213785+0000","last_became_active":"2023-10-08T00:12:57.213351+0000","last_became_peered":"2023-10-08T00:12:57.213351+0000","last_unstale":"2023-10-08T00:12:57.213785+0000","last_undegraded":"2023-10-08T00:12:57.213785+0000","last_fullsized":"2023-10-08T00:12:57.213785+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:17:09.287153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.205741+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:22:06.125225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.209114+0000","last_change":"2023-10-08T00:12:57.209114+0000","last_active":"2023-10-08T00:12:57.209114+0000","last_peered":"2023-10-08T00:12:57.209114+0000","last_clean":"2023-10-08T00:12:57.209114+0000","last_became_active":"2023-10-08T00:12:57.208878+0000","last_became_peered":"2023-10-08T00:12:57.208878+0000","last_unstale":"2023-10-08T00:12:57.209114+0000","last_undegraded":"2023-10-08T00:12:57.209114+0000","last_fullsized":"2023-10-08T00:12:57.209114+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:29:06.434469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.204353+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:50:14.863709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.213998+0000","last_change":"2023-10-08T00:12:57.213998+0000","last_active":"2023-10-08T00:12:57.213998+0000","last_peered":"2023-10-08T00:12:57.213998+0000","last_clean":"2023-10-08T00:12:57.213998+0000","last_became_active":"2023-10-08T00:12:57.213768+0000","last_became_peered":"2023-10-08T00:12:57.213768+0000","last_unstale":"2023-10-08T00:12:57.213998+0000","last_undegraded":"2023-10-08T00:12:57.213998+0000","last_fullsized":"2023-10-08T00:12:57.213998+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:36:23.496893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.205083+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:56:09.457506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.209066+0000","last_change":"2023-10-08T00:12:57.209066+0000","last_active":"2023-10-08T00:12:57.209066+0000","last_peered":"2023-10-08T00:12:57.209066+0000","last_clean":"2023-10-08T00:12:57.209066+0000","last_became_active":"2023-10-08T00:12:57.208849+0000","last_became_peered":"2023-10-08T00:12:57.208849+0000","last_unstale":"2023-10-08T00:12:57.209066+0000","last_undegraded":"2023-10-08T00:12:57.209066+0000","last_fullsized":"2023-10-08T00:12:57.209066+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:29:06.434469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.204310+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:39:49.309778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.212524+0000","last_change":"2023-10-08T00:12:57.212524+0000","last_active":"2023-10-08T00:12:57.212524+0000","last_peered":"2023-10-08T00:12:57.212524+0000","last_clean":"2023-10-08T00:12:57.212524+0000","last_became_active":"2023-10-08T00:12:57.212291+0000","last_became_peered":"2023-10-08T00:12:57.212291+0000","last_unstale":"2023-10-08T00:12:57.212524+0000","last_undegraded":"2023-10-08T00:12:57.212524+0000","last_fullsized":"2023-10-08T00:12:57.212524+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:00:48.158147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":199,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-08T00:12:56.201662+0000","last_change":"2023-10-08T00:10:43.728490+0000","last_active":"2023-10-08T00:12:56.201662+0000","last_peered":"2023-10-08T00:12:56.201662+0000","last_clean":"2023-10-08T00:12:56.201662+0000","last_became_active":"2023-10-08T00:10:43.728341+0000","last_became_peered":"2023-10-08T00:10:43.728341+0000","last_unstale":"2023-10-08T00:12:56.201662+0000","last_undegraded":"2023-10-08T00:12:56.201662+0000","last_fullsized":"2023-10-08T00:12:56.201662+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_clean_scrub_stamp":"2023-10-08T00:10:42.716096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:06:31.035293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":336,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194976,"statfs":{"total":200000000,"available":199655719,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.496,"15min":0.49399999999999999},"min":{"1min":0.34599999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63100000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.53800000000000003,"15min":0.54400000000000004},"min":{"1min":0.38100000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.73499999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.40300000000000002}]},{"osd":1,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.53100000000000003,"15min":0.52700000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.437,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.56999999999999995}]},{"osd":2,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.53000000000000003,"15min":0.53300000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.66600000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.51600000000000001,"15min":0.51400000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.71899999999999997,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.55900000000000005}]},{"osd":3,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.51700000000000002,"15min":0.51300000000000001},"min":{"1min":0.36599999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.55200000000000005,"15min":0.55400000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.47099999999999997}]},{"osd":4,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.52200000000000002,"15min":0.52600000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.70799999999999996,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.497},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.52800000000000002,"15min":0.53200000000000003},"min":{"1min":0.35699999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.64300000000000002,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.34599999999999997}]},{"osd":5,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.55300000000000005,"15min":0.55700000000000005},"min":{"1min":0.32100000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.71999999999999997,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.57999999999999996,"15min":0.59199999999999997},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.69399999999999995,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.58099999999999996}]},{"osd":6,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.59399999999999997,"15min":0.59899999999999998},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.71299999999999997,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.627,"15min":0.63800000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76000000000000001,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.52800000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":336,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194976,"statfs":{"total":200000000,"available":199655719,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.48099999999999998,"15min":0.47499999999999998},"min":{"1min":0.377,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.752,"5min":1.012,"15min":1.012},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.68400000000000005,"15min":0.70499999999999996},"min":{"1min":0.34000000000000002,"5min":0.317,"15min":0.317},"max":{"1min":1.2789999999999999,"5min":5.1230000000000002,"15min":5.1230000000000002},"last":0.374}]},{"osd":1,"last update":"Sun Oct 8 00:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.52100000000000002,"15min":0.51400000000000001},"min":{"1min":0.29299999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.82499999999999996,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55900000000000005,"15min":0.56000000000000005},"min":{"1min":0.371,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.69899999999999995,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.45400000000000001}]},{"osd":2,"last update":"Sun Oct 8 00:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.52700000000000002,"15min":0.52500000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.755,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56200000000000006,"15min":0.56499999999999995},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.76100000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.48899999999999999}]},{"osd":3,"last update":"Sun Oct 8 00:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53400000000000003,"15min":0.53000000000000003},"min":{"1min":0.38400000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.81399999999999995,"5min":1.046,"15min":1.046},"last":0.501},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.54400000000000004,"15min":0.53600000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.36099999999999999}]},{"osd":4,"last update":"Sun Oct 8 00:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.502,"15min":0.50600000000000001},"min":{"1min":0.32300000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56499999999999995,"15min":0.56699999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.42999999999999999}]},{"osd":5,"last update":"Sun Oct 8 00:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.48499999999999999,"15min":0.48799999999999999},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71099999999999997,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.55500000000000005,"15min":0.56299999999999994},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.439}]},{"osd":7,"last update":"Sun Oct 8 00:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.54100000000000004,"15min":0.54500000000000004},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.41699999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574877,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":912,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194399,"statfs":{"total":200000000,"available":199065351,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.30399999999999999,"15min":0.29899999999999999},"min":{"1min":0.218,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.44700000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.375,"15min":0.373},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.38300000000000001}]},{"osd":2,"last update":"Sun Oct 8 00:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.33600000000000002,"15min":0.33200000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.49099999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.379,"5min":0.33900000000000002,"15min":0.33200000000000002},"min":{"1min":0.254,"5min":0.252,"15min":0.252},"max":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.34999999999999998}]},{"osd":3,"last update":"Sun Oct 8 00:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.36699999999999999,"15min":0.36499999999999999},"min":{"1min":0.28299999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.307},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.501,"5min":0.501,"15min":0.501},"last":0.41999999999999998}]},{"osd":4,"last update":"Sun Oct 8 00:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.53700000000000003,"15min":0.53400000000000003},"min":{"1min":0.40400000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.56899999999999995,"15min":0.56499999999999995},"min":{"1min":0.41899999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.82299999999999995,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.60399999999999998}]},{"osd":5,"last update":"Sun Oct 8 00:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.53800000000000003,"15min":0.53100000000000003},"min":{"1min":0.372,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57499999999999996,"15min":0.57199999999999995},"min":{"1min":0.42999999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78300000000000003,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.73499999999999999}]},{"osd":6,"last update":"Sun Oct 8 00:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.72999999999999998,"5min":0.877,"15min":0.877},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56399999999999995,"15min":0.55800000000000005},"min":{"1min":0.42899999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.76900000000000002,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.61899999999999999}]},{"osd":7,"last update":"Sun Oct 8 00:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56299999999999994,"15min":0.55900000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.77700000000000002,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.752},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.58899999999999997,"15min":0.58499999999999996},"min":{"1min":0.432,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82299999999999995,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.69999999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":336,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194976,"statfs":{"total":200000000,"available":199655719,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 00:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.32400000000000001,"15min":0.32600000000000001},"min":{"1min":0.23499999999999999,"5min":0.193,"15min":0.193},"max":{"1min":0.41499999999999998,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.308},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.245,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.49299999999999999,"5min":0.87,"15min":0.87},"last":0.40699999999999997}]},{"osd":2,"last update":"Sun Oct 8 00:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.38200000000000001,"15min":0.38600000000000001},"min":{"1min":0.27900000000000003,"5min":0.251,"15min":0.251},"max":{"1min":0.58699999999999997,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.753,"15min":0.82399999999999995},"min":{"1min":0.247,"5min":0.185,"15min":0.185},"max":{"1min":0.40899999999999997,"5min":12.019,"15min":12.019},"last":0.38900000000000001}]},{"osd":3,"last update":"Sun Oct 8 00:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.35599999999999998,"15min":0.35999999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.47399999999999998,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.374,"5min":0.38800000000000001,"15min":0.39000000000000001},"min":{"1min":0.28499999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.55900000000000005,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.33600000000000002}]},{"osd":4,"last update":"Sun Oct 8 00:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.433,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.71899999999999997,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.624,"15min":0.625},"min":{"1min":0.48299999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.73499999999999999,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.71799999999999997}]},{"osd":5,"last update":"Sun Oct 8 00:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.999,"15min":1.069},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.79800000000000004,"5min":11.992000000000001,"15min":11.992000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.73199999999999998,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.61299999999999999}]},{"osd":6,"last update":"Sun Oct 8 00:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.74399999999999999,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.97799999999999998,"15min":1.042},"min":{"1min":0.47299999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76000000000000001,"5min":11.927,"15min":11.927},"last":0.72499999999999998}]},{"osd":7,"last update":"Sun Oct 8 00:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.999,"15min":1.0660000000000001},"min":{"1min":0.434,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82199999999999995,"5min":11.965,"15min":11.965},"last":0.627},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.61899999999999999,"15min":0.621},"min":{"1min":0.48199999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.76100000000000001,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.76900000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":336,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194976,"statfs":{"total":200000000,"available":199655719,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.39000000000000001,"15min":0.40200000000000002},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.437,"5min":2.5139999999999998,"15min":2.5139999999999998},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.40100000000000002,"15min":0.41399999999999998},"min":{"1min":0.248,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.44400000000000001,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":0.35899999999999999}]},{"osd":1,"last update":"Sun Oct 8 00:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.39000000000000001,"15min":0.40200000000000002},"min":{"1min":0.25,"5min":0.246,"15min":0.246},"max":{"1min":0.40100000000000002,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.436,"15min":0.45000000000000001},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.48999999999999999,"5min":2.5379999999999998,"15min":2.5379999999999998},"last":0.39200000000000002}]},{"osd":3,"last update":"Sun Oct 8 00:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.42799999999999999,"15min":0.439},"min":{"1min":0.28499999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.46500000000000002,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":0.378},{"interface":"front","average":{"1min":0.379,"5min":0.46300000000000002,"15min":0.47699999999999998},"min":{"1min":0.315,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.51200000000000001,"5min":2.5619999999999998,"15min":2.5619999999999998},"last":0.41499999999999998}]},{"osd":4,"last update":"Sun Oct 8 00:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.61699999999999999,"15min":0.628},"min":{"1min":0.45800000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.84599999999999997,"5min":2.488,"15min":2.488},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.60399999999999998,"15min":0.60999999999999999},"min":{"1min":0.41099999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85999999999999999,"5min":2.073,"15min":2.073},"last":0.81100000000000005}]},{"osd":5,"last update":"Sun Oct 8 00:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.66300000000000003,"15min":0.66900000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.2130000000000001,"5min":2.2069999999999999,"15min":2.2069999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.62,"5min":0.66300000000000003,"15min":0.67000000000000004},"min":{"1min":0.48799999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0149999999999999,"5min":2.456,"15min":2.456},"last":0.81999999999999995}]},{"osd":6,"last update":"Sun Oct 8 00:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.621,"15min":0.627},"min":{"1min":0.45000000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.79500000000000004,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.60899999999999999,"15min":0.61399999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.97499999999999998,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.77000000000000002}]},{"osd":7,"last update":"Sun Oct 8 00:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.63700000000000001,"15min":0.64100000000000001},"min":{"1min":0.499,"5min":0.438,"15min":0.438},"max":{"1min":0.81699999999999995,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.626,"15min":0.63200000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91200000000000003,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":0.64500000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":330,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194982,"statfs":{"total":200000000,"available":199661982,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.28699999999999998,"5min":0.36699999999999999,"15min":0.38},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.45100000000000001,"5min":1.46,"15min":1.46},"last":0.251},{"interface":"front","average":{"1min":0.29699999999999999,"5min":0.33500000000000002,"15min":0.34100000000000003},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.46300000000000002,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.23300000000000001}]},{"osd":1,"last update":"Sun Oct 8 00:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.82999999999999996,"15min":0.91700000000000004},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.44800000000000001,"5min":12.351000000000001,"15min":12.351000000000001},"last":0.307},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.35899999999999999,"15min":0.36299999999999999},"min":{"1min":0.26500000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.443,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.34300000000000003}]},{"osd":2,"last update":"Sun Oct 8 00:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.41399999999999998,"15min":0.42499999999999999},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.47299999999999998,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.377},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.379,"15min":0.38300000000000001},"min":{"1min":0.27800000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.45900000000000002,"5min":0.753,"15min":0.753},"last":0.41099999999999998}]},{"osd":4,"last update":"Sun Oct 8 00:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":1.004,"15min":1.0860000000000001},"min":{"1min":0.38800000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.75600000000000001,"5min":11.333,"15min":11.333},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":1.0369999999999999,"15min":1.1200000000000001},"min":{"1min":0.434,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.77000000000000002,"5min":12.321999999999999,"15min":12.321999999999999},"last":0.63200000000000001}]},{"osd":5,"last update":"Sun Oct 8 00:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":1.0509999999999999,"15min":1.137},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.72599999999999998,"5min":12.291,"15min":12.291},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.58599999999999997,"15min":0.59299999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.74299999999999999,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.59699999999999998}]},{"osd":6,"last update":"Sun Oct 8 00:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.96999999999999997,"15min":1.048},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.69899999999999995,"5min":11.288,"15min":11.288},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.51800000000000002,"5min":1.0309999999999999,"15min":1.117},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.66000000000000003,"5min":12.253,"15min":12.253},"last":0.67500000000000004}]},{"osd":7,"last update":"Sun Oct 8 00:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56599999999999995,"15min":0.56799999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.56399999999999995,"5min":1.012,"15min":1.0860000000000001},"min":{"1min":0.435,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.381,"5min":11.363,"15min":11.363},"last":0.66600000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":336,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194976,"statfs":{"total":200000000,"available":199655719,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.56000000000000005,"15min":0.55400000000000005},"min":{"1min":0.41299999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.375},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.50600000000000001}]},{"osd":1,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51700000000000002,"15min":0.51800000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.64100000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.42099999999999999}]},{"osd":2,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.51600000000000001,"15min":0.52000000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.51300000000000001,"15min":0.50900000000000001},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.54200000000000004}]},{"osd":3,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.53500000000000003,"15min":0.53900000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.70899999999999996,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.53000000000000003,"15min":0.53200000000000003},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.40200000000000002}]},{"osd":5,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.49299999999999999,"15min":0.503},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69899999999999995,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.50800000000000001,"15min":0.51400000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.71099999999999997,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.57199999999999995}]},{"osd":6,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.48899999999999999,"15min":0.49399999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.55200000000000005,"15min":0.56200000000000006},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.754,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.58599999999999997}]},{"osd":7,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.54600000000000004,"15min":0.55500000000000005},"min":{"1min":0.36299999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.64200000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.57799999999999996,"15min":0.58299999999999996},"min":{"1min":0.41199999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.77700000000000002,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.628}]}]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":906,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194405,"statfs":{"total":200000000,"available":199071614,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.48699999999999999,"15min":0.48799999999999999},"min":{"1min":0.32700000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68799999999999994,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.441},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.55700000000000005,"15min":0.56299999999999994},"min":{"1min":0.374,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.68600000000000005,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.58599999999999997}]},{"osd":1,"last update":"Sun Oct 8 00:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.52100000000000002,"15min":0.52300000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.65800000000000003,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.52400000000000002,"15min":0.52800000000000002},"min":{"1min":0.39300000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.70899999999999996,"5min":0.755,"15min":0.755},"last":0.48999999999999999}]},{"osd":2,"last update":"Sun Oct 8 00:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.52900000000000003,"15min":0.53100000000000003},"min":{"1min":0.40500000000000003,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.64500000000000002,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.56599999999999995,"15min":0.57199999999999995},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.70699999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.59799999999999998}]},{"osd":3,"last update":"Sun Oct 8 00:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54300000000000004,"15min":0.54500000000000004},"min":{"1min":0.38700000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.67100000000000004,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.57999999999999996,"15min":0.58399999999999996},"min":{"1min":0.42799999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.72299999999999998,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.55900000000000005}]},{"osd":4,"last update":"Sun Oct 8 00:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.47299999999999998,"15min":0.47499999999999998},"min":{"1min":0.35799999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.626,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.50900000000000001,"15min":0.51300000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.63700000000000001,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.54600000000000004}]},{"osd":6,"last update":"Sun Oct 8 00:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.499,"15min":0.499},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.61899999999999999,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.66800000000000004,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.53200000000000003}]},{"osd":7,"last update":"Sun Oct 8 00:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.51300000000000001,"15min":0.51500000000000001},"min":{"1min":0.33400000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67300000000000004,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.69799999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.51600000000000001}]}]}],"pool_statfs":[]}} 2023-10-08T00:13:03.013 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3f is not active+clean 2023-10-08T00:13:03.013 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.207886+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T02:42:53.170735+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T00:13:03.013 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3e is not active+clean 2023-10-08T00:13:03.014 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.206623+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T11:29:57.147801+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T00:13:03.014 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3d is not active+clean 2023-10-08T00:13:03.014 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.206756+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T03:15:27.299505+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T00:13:03.014 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3c is not active+clean 2023-10-08T00:13:03.014 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.204518+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T05:17:47.918131+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T00:13:03.014 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3b is not active+clean 2023-10-08T00:13:03.014 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.208328+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T00:29:07.443694+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T00:13:03.015 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3a is not active+clean 2023-10-08T00:13:03.015 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.207050+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T06:19:19.113722+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T00:13:03.015 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.39 is not active+clean 2023-10-08T00:13:03.015 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.204971+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T11:18:06.799190+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T00:13:03.015 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.38 is not active+clean 2023-10-08T00:13:03.015 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.214505+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T11:51:20.513160+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T00:13:03.015 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.37 is not active+clean 2023-10-08T00:13:03.015 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.204229+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T00:57:53.151755+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T00:13:03.016 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.36 is not active+clean 2023-10-08T00:13:03.016 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.207215+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T07:03:56.099847+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T00:13:03.016 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.35 is not active+clean 2023-10-08T00:13:03.016 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.207550+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T07:03:56.099847+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T00:13:03.016 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.34 is not active+clean 2023-10-08T00:13:03.016 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.203197+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T05:04:57.344499+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T00:13:03.016 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.33 is not active+clean 2023-10-08T00:13:03.016 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.206586+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T03:36:53.312606+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T00:13:03.017 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.32 is not active+clean 2023-10-08T00:13:03.017 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.206510+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T09:56:37.046732+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T00:13:03.017 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.31 is not active+clean 2023-10-08T00:13:03.017 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.205079+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T05:39:24.599431+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T00:13:03.017 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.30 is not active+clean 2023-10-08T00:13:03.017 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.207493+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T05:39:24.599431+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T00:13:03.017 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2f is not active+clean 2023-10-08T00:13:03.017 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.206258+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T10:00:49.167372+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T00:13:03.018 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2e is not active+clean 2023-10-08T00:13:03.018 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.206438+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T07:33:06.564584+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T00:13:03.018 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2d is not active+clean 2023-10-08T00:13:03.018 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.205454+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T01:47:44.758644+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T00:13:03.018 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2c is not active+clean 2023-10-08T00:13:03.018 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.208478+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T08:22:06.125225+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T00:13:03.018 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2b is not active+clean 2023-10-08T00:13:03.018 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.206453+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T00:29:07.443694+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T00:13:03.019 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2a is not active+clean 2023-10-08T00:13:03.019 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.209033+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T02:42:53.170735+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T00:13:03.019 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.29 is not active+clean 2023-10-08T00:13:03.019 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.209265+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T04:56:15.157209+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T00:13:03.019 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.28 is not active+clean 2023-10-08T00:13:03.019 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.203882+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T07:50:17.972931+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T00:13:03.019 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.27 is not active+clean 2023-10-08T00:13:03.019 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.206009+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T07:36:24.506117+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T00:13:03.020 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.26 is not active+clean 2023-10-08T00:13:03.020 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.206105+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T02:42:53.170735+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T00:13:03.020 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.25 is not active+clean 2023-10-08T00:13:03.020 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.206249+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T08:43:06.027907+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T00:13:03.020 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.24 is not active+clean 2023-10-08T00:13:03.020 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.213400+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T01:57:11.917467+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T00:13:03.020 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.23 is not active+clean 2023-10-08T00:13:03.020 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.208004+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T11:17:37.423308+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T00:13:03.021 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.22 is not active+clean 2023-10-08T00:13:03.021 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.205536+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T03:28:33.161628+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T00:13:03.021 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.21 is not active+clean 2023-10-08T00:13:03.021 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.206770+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T11:59:27.657370+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T00:13:03.021 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.20 is not active+clean 2023-10-08T00:13:03.021 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.209403+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T00:29:07.443694+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T00:13:03.021 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1e is not active+clean 2023-10-08T00:13:03.021 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.205320+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T01:33:16.516139+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T00:13:03.022 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1f is not active+clean 2023-10-08T00:13:03.022 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.205824+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T08:51:47.982127+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T00:13:03.022 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1c is not active+clean 2023-10-08T00:13:03.022 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.207674+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T00:13:42.175367+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T00:13:03.022 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1d is not active+clean 2023-10-08T00:13:03.022 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.207619+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T08:22:06.125225+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T00:13:03.022 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1a is not active+clean 2023-10-08T00:13:03.022 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.204176+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T04:24:53.646854+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T00:13:03.023 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1b is not active+clean 2023-10-08T00:13:03.023 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.208400+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T05:39:24.599431+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T00:13:03.023 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.18 is not active+clean 2023-10-08T00:13:03.023 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.208004+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T02:47:28.589841+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T00:13:03.023 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.19 is not active+clean 2023-10-08T00:13:03.023 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.204806+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T04:22:04.500698+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T00:13:03.023 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.16 is not active+clean 2023-10-08T00:13:03.023 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.204948+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T07:03:56.099847+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T00:13:03.024 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.17 is not active+clean 2023-10-08T00:13:03.024 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.207557+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T05:39:49.309778+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T00:13:03.024 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.14 is not active+clean 2023-10-08T00:13:03.024 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.206370+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T00:52:28.504130+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T00:13:03.024 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.15 is not active+clean 2023-10-08T00:13:03.024 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.208930+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T11:44:29.429854+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T00:13:03.024 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.12 is not active+clean 2023-10-08T00:13:03.024 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.203919+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T11:17:10.296378+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T00:13:03.025 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.13 is not active+clean 2023-10-08T00:13:03.025 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.207660+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T04:50:14.863709+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T00:13:03.025 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.10 is not active+clean 2023-10-08T00:13:03.025 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.203096+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T07:43:33.796004+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T00:13:03.025 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.11 is not active+clean 2023-10-08T00:13:03.025 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.204614+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T12:04:37.043481+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T00:13:03.025 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.e is not active+clean 2023-10-08T00:13:03.025 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.207309+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T04:24:53.646854+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T00:13:03.025 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.f is not active+clean 2023-10-08T00:13:03.026 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.206202+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T03:37:04.823729+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T00:13:03.026 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.c is not active+clean 2023-10-08T00:13:03.026 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.205871+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T07:25:36.388792+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T00:13:03.026 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.d is not active+clean 2023-10-08T00:13:03.026 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.205056+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T05:33:58.167385+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T00:13:03.026 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.a is not active+clean 2023-10-08T00:13:03.026 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.203529+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T00:32:16.626313+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T00:13:03.026 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.b is not active+clean 2023-10-08T00:13:03.027 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.203581+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T09:56:37.046732+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T00:13:03.027 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.9 is not active+clean 2023-10-08T00:13:03.027 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.203749+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T03:15:27.299505+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T00:13:03.027 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.6 is not active+clean 2023-10-08T00:13:03.027 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.206067+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T09:11:41.089416+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T00:13:03.027 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.7 is not active+clean 2023-10-08T00:13:03.027 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.205826+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T08:28:53.462666+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T00:13:03.027 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.4 is not active+clean 2023-10-08T00:13:03.027 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.205536+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T10:35:32.137447+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T00:13:03.028 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.5 is not active+clean 2023-10-08T00:13:03.028 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.204798+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T02:47:28.589841+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T00:13:03.028 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2 is not active+clean 2023-10-08T00:13:03.028 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.206666+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T09:40:34.751005+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T00:13:03.028 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3 is not active+clean 2023-10-08T00:13:03.028 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.205741+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T08:22:06.125225+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T00:13:03.028 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.0 is not active+clean 2023-10-08T00:13:03.028 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.204353+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T04:50:14.863709+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T00:13:03.029 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1 is not active+clean 2023-10-08T00:13:03.029 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.205083+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T07:56:09.457506+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T00:13:03.029 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.8 is not active+clean 2023-10-08T00:13:03.029 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T00:12:57.194440+0000', 'last_change': '2023-10-08T00:12:57.204310+0000', 'last_active': '2023-10-08T00:12:57.194440+0000', 'last_peered': '2023-10-08T00:12:57.194440+0000', 'last_clean': '2023-10-08T00:12:57.194440+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T00:12:57.194440+0000', 'last_undegraded': '2023-10-08T00:12:57.194440+0000', 'last_fullsized': '2023-10-08T00:12:57.194440+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'last_clean_scrub_stamp': '2023-10-08T00:12:57.194440+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T05:39:49.309778+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T00:13:03.029 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-08T00:13:03.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.160+0000 7faa6e909700 1 Processor -- start 2023-10-08T00:13:03.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.160+0000 7faa6e909700 1 -- start start 2023-10-08T00:13:03.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.160+0000 7faa6e909700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7faa6815a620 0x7faa6815d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:03.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.160+0000 7faa6e909700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa6815cbb0 con 0x7faa6815d580 2023-10-08T00:13:03.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.160+0000 7faa6e909700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa6815bfa0 con 0x7faa68157870 2023-10-08T00:13:03.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.160+0000 7faa6e909700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa6815bdb0 con 0x7faa6815a620 2023-10-08T00:13:03.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.161+0000 7faa6cea6700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7faa6815d580 0x7faa68158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:47628/0 (socket says 172.21.15.99:47628) 2023-10-08T00:13:03.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.161+0000 7faa6cea6700 1 -- 172.21.15.99:0/4257094760 learned_addr learned my addr 172.21.15.99:0/4257094760 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:13:03.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.161+0000 7faa677fe700 1 --2- 172.21.15.99:0/4257094760 >> v2:172.21.15.191:3300/0 conn(0x7faa6815a620 0x7faa6815d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:03.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.161+0000 7faa66ffd700 1 -- 172.21.15.99:0/4257094760 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 428819029 0 0) 0x7faa6815cbb0 con 0x7faa6815d580 2023-10-08T00:13:03.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.161+0000 7faa66ffd700 1 -- 172.21.15.99:0/4257094760 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa50003570 con 0x7faa6815d580 2023-10-08T00:13:03.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.161+0000 7faa66ffd700 1 -- 172.21.15.99:0/4257094760 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3813215343 0 0) 0x7faa6815bfa0 con 0x7faa68157870 2023-10-08T00:13:03.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.161+0000 7faa66ffd700 1 -- 172.21.15.99:0/4257094760 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa6815cbb0 con 0x7faa68157870 2023-10-08T00:13:03.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.161+0000 7faa66ffd700 1 -- 172.21.15.99:0/4257094760 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 537945370 0 0) 0x7faa50003570 con 0x7faa6815d580 2023-10-08T00:13:03.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.161+0000 7faa677fe700 1 -- 172.21.15.99:0/4257094760 >> v1:172.21.15.191:6789/0 conn(0x7faa68157870 legacy=0x7faa6815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:03.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.161+0000 7faa677fe700 1 -- 172.21.15.99:0/4257094760 >> v1:172.21.15.99:6789/0 conn(0x7faa6815d580 legacy=0x7faa68158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:03.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.161+0000 7faa677fe700 1 -- 172.21.15.99:0/4257094760 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa6815c280 con 0x7faa6815a620 2023-10-08T00:13:03.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.161+0000 7faa677fe700 1 --2- 172.21.15.99:0/4257094760 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa6815a620 0x7faa6815d040 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7faa5c0098e0 tx=0x7faa5c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9b805d23a2fe693 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:13:03.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.161+0000 7faa66ffd700 1 -- 172.21.15.99:0/4257094760 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa600017f0 con 0x7faa6815a620 2023-10-08T00:13:03.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.162+0000 7faa66ffd700 1 -- 172.21.15.99:0/4257094760 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa5c011420 con 0x7faa6815a620 2023-10-08T00:13:03.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.162+0000 7faa66ffd700 1 -- 172.21.15.99:0/4257094760 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5c0117c0 con 0x7faa6815a620 2023-10-08T00:13:03.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.162+0000 7faa6e909700 1 -- 172.21.15.99:0/4257094760 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa6815a620 msgr2=0x7faa6815d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:03.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.162+0000 7faa6e909700 1 --2- 172.21.15.99:0/4257094760 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa6815a620 0x7faa6815d040 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7faa5c0098e0 tx=0x7faa5c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:13:03.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.162+0000 7faa6e909700 1 -- 172.21.15.99:0/4257094760 shutdown_connections 2023-10-08T00:13:03.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.162+0000 7faa6e909700 1 --2- 172.21.15.99:0/4257094760 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa6815a620 0x7faa6815d040 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:03.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.162+0000 7faa6e909700 1 -- 172.21.15.99:0/4257094760 >> 172.21.15.99:0/4257094760 conn(0x7faa68068ce0 msgr2=0x7faa680690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:13:03.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.162+0000 7faa6e909700 1 -- 172.21.15.99:0/4257094760 shutdown_connections 2023-10-08T00:13:03.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.162+0000 7faa6e909700 1 -- 172.21.15.99:0/4257094760 wait complete. 2023-10-08T00:13:03.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.163+0000 7faa6e909700 1 Processor -- start 2023-10-08T00:13:03.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.163+0000 7faa6e909700 1 -- start start 2023-10-08T00:13:03.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.164+0000 7faa6e909700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa68157870 0x7faa6814ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:03.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.164+0000 7faa6e909700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa6815a620 0x7faa680cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:03.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.164+0000 7faa6e909700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa6815d580 0x7faa680b5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:03.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.164+0000 7faa6e909700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa6814b240 con 0x7faa6815d580 2023-10-08T00:13:03.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.164+0000 7faa6e909700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa680b1be0 con 0x7faa6815a620 2023-10-08T00:13:03.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.164+0000 7faa6e909700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa680cfd70 con 0x7faa68157870 2023-10-08T00:13:03.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.164+0000 7faa6cea6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa6815d580 0x7faa680b5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:03.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.164+0000 7faa6cea6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa6815d580 0x7faa680b5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:32944/0 (socket says 172.21.15.99:32944) 2023-10-08T00:13:03.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.164+0000 7faa6cea6700 1 -- 172.21.15.99:0/881558833 learned_addr learned my addr 172.21.15.99:0/881558833 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:13:03.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.164+0000 7faa677fe700 1 --2- 172.21.15.99:0/881558833 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa6815a620 0x7faa680cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:03.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.164+0000 7faa67fff700 1 --2- 172.21.15.99:0/881558833 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa68157870 0x7faa6814ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:03.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.164+0000 7faa6cea6700 1 -- 172.21.15.99:0/881558833 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa68157870 msgr2=0x7faa6814ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:03.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.164+0000 7faa6cea6700 1 --2- 172.21.15.99:0/881558833 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa68157870 0x7faa6814ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:03.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.164+0000 7faa6cea6700 1 -- 172.21.15.99:0/881558833 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa6815a620 msgr2=0x7faa680cf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:03.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.164+0000 7faa6cea6700 1 --2- 172.21.15.99:0/881558833 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa6815a620 0x7faa680cf2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:03.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.164+0000 7faa6cea6700 1 -- 172.21.15.99:0/881558833 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa5c009420 con 0x7faa6815d580 2023-10-08T00:13:03.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.165+0000 7faa6cea6700 1 --2- 172.21.15.99:0/881558833 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa6815d580 0x7faa680b5260 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7faa6000cbe0 tx=0x7faa60007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:13:03.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.165+0000 7faa4ffff700 1 -- 172.21.15.99:0/881558833 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa60004dc0 con 0x7faa6815d580 2023-10-08T00:13:03.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.165+0000 7faa4ffff700 1 -- 172.21.15.99:0/881558833 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa6000a5c0 con 0x7faa6815d580 2023-10-08T00:13:03.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.165+0000 7faa6e909700 1 -- 172.21.15.99:0/881558833 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa680d0050 con 0x7faa6815d580 2023-10-08T00:13:03.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.165+0000 7faa4ffff700 1 -- 172.21.15.99:0/881558833 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa6000a980 con 0x7faa6815d580 2023-10-08T00:13:03.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.165+0000 7faa6e909700 1 -- 172.21.15.99:0/881558833 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa680b1800 con 0x7faa6815d580 2023-10-08T00:13:03.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.166+0000 7faa6e909700 1 -- 172.21.15.99:0/881558833 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa680b1d50 con 0x7faa6815d580 2023-10-08T00:13:03.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.166+0000 7faa4ffff700 1 -- 172.21.15.99:0/881558833 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faa6000ad30 con 0x7faa6815d580 2023-10-08T00:13:03.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.166+0000 7faa4ffff700 1 --2- 172.21.15.99:0/881558833 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7faa54066f30 0x7faa540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:03.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.166+0000 7faa4ffff700 1 -- 172.21.15.99:0/881558833 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5705+0+0 (secure 0 0 0) 0x7faa60083e10 con 0x7faa6815d580 2023-10-08T00:13:03.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.167+0000 7faa67fff700 1 --2- 172.21.15.99:0/881558833 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7faa54066f30 0x7faa540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:03.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.167+0000 7faa67fff700 1 --2- 172.21.15.99:0/881558833 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7faa54066f30 0x7faa540693e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7faa58007fd0 tx=0x7faa58007d50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:13:03.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.171+0000 7faa4ffff700 1 -- 172.21.15.99:0/881558833 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faa60051100 con 0x7faa6815d580 2023-10-08T00:13:03.187 INFO:tasks.ceph.mds.d.smithi191.stderr:2023-10-08T00:13:03.186+0000 7fb7ed67f700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T00:13:03.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.432+0000 7faa6e909700 1 -- 172.21.15.99:0/881558833 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7faa6804c3c0 con 0x7faa6815d580 2023-10-08T00:13:03.434 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.433+0000 7faa4ffff700 1 -- 172.21.15.99:0/881558833 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1448 (secure 0 0 0) 0x7faa600185c0 con 0x7faa6815d580 2023-10-08T00:13:03.434 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:13:03.434 INFO:teuthology.orchestra.run.smithi099.stdout:{"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":147,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":69,"num_osds":8,"num_up_osds":8,"osd_up_since":1696723841,"num_in_osds":8,"osd_in_since":1696723838,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":3922458,"bytes_avail":1596077542,"bytes_total":1600000000,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":32,"id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"d","status":"up:active","gid":5688}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2023-10-08T00:12:46.049552+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-08T00:13:03.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.435+0000 7faa6e909700 1 -- 172.21.15.99:0/881558833 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7faa54066f30 msgr2=0x7faa540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:03.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.435+0000 7faa6e909700 1 --2- 172.21.15.99:0/881558833 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7faa54066f30 0x7faa540693e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7faa58007fd0 tx=0x7faa58007d50 comp rx=0 tx=0).stop 2023-10-08T00:13:03.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.435+0000 7faa6e909700 1 -- 172.21.15.99:0/881558833 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa6815d580 msgr2=0x7faa680b5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:03.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.435+0000 7faa6e909700 1 --2- 172.21.15.99:0/881558833 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa6815d580 0x7faa680b5260 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7faa6000cbe0 tx=0x7faa60007d20 comp rx=0 tx=0).stop 2023-10-08T00:13:03.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.435+0000 7faa6e909700 1 -- 172.21.15.99:0/881558833 shutdown_connections 2023-10-08T00:13:03.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.435+0000 7faa6e909700 1 --2- 172.21.15.99:0/881558833 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7faa54066f30 0x7faa540693e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:03.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.435+0000 7faa6e909700 1 --2- 172.21.15.99:0/881558833 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa6815d580 0x7faa680b5260 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:03.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.435+0000 7faa6e909700 1 --2- 172.21.15.99:0/881558833 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa68157870 0x7faa6814ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:03.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.435+0000 7faa6e909700 1 --2- 172.21.15.99:0/881558833 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa6815a620 0x7faa680cf2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:03.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.435+0000 7faa6e909700 1 -- 172.21.15.99:0/881558833 >> 172.21.15.99:0/881558833 conn(0x7faa68068ce0 msgr2=0x7faa68159a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:13:03.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.435+0000 7faa6e909700 1 -- 172.21.15.99:0/881558833 shutdown_connections 2023-10-08T00:13:03.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.435+0000 7faa6e909700 1 -- 172.21.15.99:0/881558833 wait complete. 2023-10-08T00:13:03.447 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-08T00:13:03.448 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T00:13:03.561 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.560+0000 7f626b07c700 1 Processor -- start 2023-10-08T00:13:03.561 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.560+0000 7f626b07c700 1 -- start start 2023-10-08T00:13:03.563 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.561+0000 7f626b07c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f6264157870 0x7f626415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:03.563 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.561+0000 7f626b07c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6264157c40 0x7f6264158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:03.564 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.561+0000 7f626b07c700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f626415cbb0 con 0x7f626415a620 2023-10-08T00:13:03.564 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.561+0000 7f626b07c700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f626415bfa0 con 0x7f6264157870 2023-10-08T00:13:03.564 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.561+0000 7f626b07c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f626415c280 con 0x7f6264157c40 2023-10-08T00:13:03.564 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.561+0000 7f6269619700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6264157c40 0x7f6264158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:03.564 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.561+0000 7f6268e18700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f6264157870 0x7f626415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:03.564 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.561+0000 7f6269619700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6264157c40 0x7f6264158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.99:55854/0 (socket says 172.21.15.99:55854) 2023-10-08T00:13:03.564 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.561+0000 7f625bfff700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f626415a620 0x7f626415d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:56632/0 (socket says 172.21.15.99:56632) 2023-10-08T00:13:03.564 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.561+0000 7f6269619700 1 -- 172.21.15.99:0/2798961279 learned_addr learned my addr 172.21.15.99:0/2798961279 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:13:03.564 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.562+0000 7f6269619700 1 -- 172.21.15.99:0/2798961279 >> v2:172.21.15.191:3300/0 conn(0x7f6264157870 msgr2=0x7f626415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:13:03.564 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.562+0000 7f6269619700 1 --2- 172.21.15.99:0/2798961279 >> v2:172.21.15.191:3300/0 conn(0x7f6264157870 0x7f626415b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:03.564 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.562+0000 7f6269619700 1 -- 172.21.15.99:0/2798961279 >> v1:172.21.15.191:6789/0 conn(0x7f626415a620 legacy=0x7f626415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:03.565 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.562+0000 7f6269619700 1 -- 172.21.15.99:0/2798961279 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f626415c560 con 0x7f6264157c40 2023-10-08T00:13:03.565 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.562+0000 7f6268e18700 1 --2- 172.21.15.99:0/2798961279 >> v2:172.21.15.191:3300/0 conn(0x7f6264157870 0x7f626415b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:13:03.565 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.562+0000 7f6269619700 1 --2- 172.21.15.99:0/2798961279 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6264157c40 0x7f6264158110 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f625c009f20 tx=0x7f625c009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3849acb578ad8a00 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:13:03.565 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.562+0000 7f625b7fe700 1 -- 172.21.15.99:0/2798961279 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625c00e070 con 0x7f6264157c40 2023-10-08T00:13:03.565 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.562+0000 7f625b7fe700 1 -- 172.21.15.99:0/2798961279 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f625c00b9d0 con 0x7f6264157c40 2023-10-08T00:13:03.565 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.562+0000 7f625b7fe700 1 -- 172.21.15.99:0/2798961279 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625c0092e0 con 0x7f6264157c40 2023-10-08T00:13:03.565 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.563+0000 7f626b07c700 1 -- 172.21.15.99:0/2798961279 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6264157c40 msgr2=0x7f6264158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:03.565 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.563+0000 7f626b07c700 1 --2- 172.21.15.99:0/2798961279 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6264157c40 0x7f6264158110 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f625c009f20 tx=0x7f625c009840 comp rx=0 tx=0).stop 2023-10-08T00:13:03.565 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.563+0000 7f626b07c700 1 -- 172.21.15.99:0/2798961279 shutdown_connections 2023-10-08T00:13:03.565 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.563+0000 7f626b07c700 1 --2- 172.21.15.99:0/2798961279 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6264157c40 0x7f6264158110 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:03.565 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.563+0000 7f626b07c700 1 --2- 172.21.15.99:0/2798961279 >> v2:172.21.15.191:3300/0 conn(0x7f6264157870 0x7f626415b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:03.565 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.563+0000 7f626b07c700 1 -- 172.21.15.99:0/2798961279 >> 172.21.15.99:0/2798961279 conn(0x7f6264068ce0 msgr2=0x7f62640690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:13:03.565 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.563+0000 7f626b07c700 1 -- 172.21.15.99:0/2798961279 shutdown_connections 2023-10-08T00:13:03.566 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.563+0000 7f626b07c700 1 -- 172.21.15.99:0/2798961279 wait complete. 2023-10-08T00:13:03.568 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.564+0000 7f626b07c700 1 Processor -- start 2023-10-08T00:13:03.568 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.564+0000 7f626b07c700 1 -- start start 2023-10-08T00:13:03.568 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.565+0000 7f626b07c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6264157870 0x7f626414e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:03.568 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.565+0000 7f626b07c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6264157c40 0x7f62640b0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:03.568 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.565+0000 7f626b07c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f626415a620 0x7f62640b1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:03.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.565+0000 7f626b07c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626414e8f0 con 0x7f6264157c40 2023-10-08T00:13:03.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.565+0000 7f626b07c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6264153c70 con 0x7f626415a620 2023-10-08T00:13:03.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.565+0000 7f626b07c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6264153650 con 0x7f6264157870 2023-10-08T00:13:03.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.565+0000 7f625bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6264157c40 0x7f62640b0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:03.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.565+0000 7f6268e18700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6264157870 0x7f626414e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:03.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.565+0000 7f625bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6264157c40 0x7f62640b0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46276/0 (socket says 172.21.15.99:46276) 2023-10-08T00:13:03.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.565+0000 7f625bfff700 1 -- 172.21.15.99:0/3236453603 learned_addr learned my addr 172.21.15.99:0/3236453603 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:13:03.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.565+0000 7f6269619700 1 --2- 172.21.15.99:0/3236453603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f626415a620 0x7f62640b1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:03.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.565+0000 7f6268e18700 1 -- 172.21.15.99:0/3236453603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f626415a620 msgr2=0x7f62640b1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:03.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.565+0000 7f6268e18700 1 --2- 172.21.15.99:0/3236453603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f626415a620 0x7f62640b1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:03.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.565+0000 7f6268e18700 1 -- 172.21.15.99:0/3236453603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6264157c40 msgr2=0x7f62640b0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:03.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.565+0000 7f6268e18700 1 --2- 172.21.15.99:0/3236453603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6264157c40 0x7f62640b0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:03.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.565+0000 7f6268e18700 1 -- 172.21.15.99:0/3236453603 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6260009420 con 0x7f6264157870 2023-10-08T00:13:03.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.565+0000 7f6268e18700 1 --2- 172.21.15.99:0/3236453603 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6264157870 0x7f626414e3b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f626000ec80 tx=0x7f62600091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:13:03.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.566+0000 7f6258ff9700 1 -- 172.21.15.99:0/3236453603 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626000c850 con 0x7f6264157870 2023-10-08T00:13:03.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.566+0000 7f6258ff9700 1 -- 172.21.15.99:0/3236453603 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f626000c9b0 con 0x7f6264157870 2023-10-08T00:13:03.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.566+0000 7f626b07c700 1 -- 172.21.15.99:0/3236453603 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f625c009ae0 con 0x7f6264157870 2023-10-08T00:13:03.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.566+0000 7f6258ff9700 1 -- 172.21.15.99:0/3236453603 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626000cd70 con 0x7f6264157870 2023-10-08T00:13:03.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.566+0000 7f626b07c700 1 -- 172.21.15.99:0/3236453603 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62641514e0 con 0x7f6264157870 2023-10-08T00:13:03.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.566+0000 7f626b07c700 1 -- 172.21.15.99:0/3236453603 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f626404c3c0 con 0x7f6264157870 2023-10-08T00:13:03.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.567+0000 7f6258ff9700 1 -- 172.21.15.99:0/3236453603 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6260013680 con 0x7f6264157870 2023-10-08T00:13:03.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.567+0000 7f6258ff9700 1 --2- 172.21.15.99:0/3236453603 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6250066f30 0x7f62500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:03.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.567+0000 7f6258ff9700 1 -- 172.21.15.99:0/3236453603 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6260084860 con 0x7f6264157870 2023-10-08T00:13:03.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.567+0000 7f625bfff700 1 --2- 172.21.15.99:0/3236453603 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6250066f30 0x7f62500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:03.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.568+0000 7f625bfff700 1 --2- 172.21.15.99:0/3236453603 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6250066f30 0x7f62500693e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6264158800 tx=0x7f6254000bc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:13:03.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.572+0000 7f6258ff9700 1 -- 172.21.15.99:0/3236453603 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6260051ad0 con 0x7f6264157870 2023-10-08T00:13:03.750 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.749+0000 7f626b07c700 1 -- 172.21.15.99:0/3236453603 --> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6264153db0 con 0x7f6250066f30 2023-10-08T00:13:03.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.758+0000 7f6258ff9700 1 -- 172.21.15.99:0/3236453603 <== mgr.4099 v2:172.21.15.191:6832/85985 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+281850 (secure 0 0 0) 0x7f6254004c80 con 0x7f6250066f30 2023-10-08T00:13:03.761 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:13:03.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.760+0000 7f626b07c700 1 -- 172.21.15.99:0/3236453603 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6250066f30 msgr2=0x7f62500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:03.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.760+0000 7f626b07c700 1 --2- 172.21.15.99:0/3236453603 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6250066f30 0x7f62500693e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6264158800 tx=0x7f6254000bc0 comp rx=0 tx=0).stop 2023-10-08T00:13:03.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.760+0000 7f626b07c700 1 -- 172.21.15.99:0/3236453603 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6264157870 msgr2=0x7f626414e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:03.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.760+0000 7f626b07c700 1 --2- 172.21.15.99:0/3236453603 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6264157870 0x7f626414e3b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f626000ec80 tx=0x7f62600091a0 comp rx=0 tx=0).stop 2023-10-08T00:13:03.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.760+0000 7f626b07c700 1 -- 172.21.15.99:0/3236453603 shutdown_connections 2023-10-08T00:13:03.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.760+0000 7f626b07c700 1 --2- 172.21.15.99:0/3236453603 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6250066f30 0x7f62500693e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:03.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.760+0000 7f626b07c700 1 --2- 172.21.15.99:0/3236453603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f626415a620 0x7f62640b1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:03.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.760+0000 7f626b07c700 1 --2- 172.21.15.99:0/3236453603 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6264157870 0x7f626414e3b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:03.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.760+0000 7f626b07c700 1 --2- 172.21.15.99:0/3236453603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6264157c40 0x7f62640b0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:03.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.760+0000 7f626b07c700 1 -- 172.21.15.99:0/3236453603 >> 172.21.15.99:0/3236453603 conn(0x7f6264068ce0 msgr2=0x7f62640b46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:13:03.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.761+0000 7f626b07c700 1 -- 172.21.15.99:0/3236453603 shutdown_connections 2023-10-08T00:13:03.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:03.761+0000 7f626b07c700 1 -- 172.21.15.99:0/3236453603 wait complete. 2023-10-08T00:13:03.765 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2023-10-08T00:13:03.773 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":129,"stamp":"2023-10-08T00:13:02.053575+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":3830,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1558669,"statfs":{"total":1600000000,"available":1596077542,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.858362"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.207886+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:42:53.170735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206623+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:29:57.147801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206756+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:15:27.299505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.204518+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:17:47.918131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.208328+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:29:07.443694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.207050+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:19:19.113722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.204971+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:18:06.799190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.214505+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:51:20.513160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.204229+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:57:53.151755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.207215+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:03:56.099847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.207550+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:03:56.099847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.203197+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:04:57.344499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206586+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:36:53.312606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206510+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:56:37.046732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.205079+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:39:24.599431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.207493+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:39:24.599431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206258+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:00:49.167372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206438+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:33:06.564584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.205454+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:47:44.758644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.208478+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:22:06.125225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206453+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:29:07.443694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.209033+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:42:53.170735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.209265+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:56:15.157209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.203882+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:50:17.972931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206009+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:36:24.506117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206105+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:42:53.170735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206249+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:43:06.027907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.213400+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:57:11.917467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.208004+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:17:37.423308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.205536+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:28:33.161628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206770+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:27.657370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.209403+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:29:07.443694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.205320+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:33:16.516139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.213303+0000","last_change":"2023-10-08T00:12:57.213303+0000","last_active":"2023-10-08T00:12:57.213303+0000","last_peered":"2023-10-08T00:12:57.213303+0000","last_clean":"2023-10-08T00:12:57.213303+0000","last_became_active":"2023-10-08T00:12:57.213092+0000","last_became_peered":"2023-10-08T00:12:57.213092+0000","last_unstale":"2023-10-08T00:12:57.213303+0000","last_undegraded":"2023-10-08T00:12:57.213303+0000","last_fullsized":"2023-10-08T00:12:57.213303+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:40:33.741781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.205824+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:51:47.982127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.212771+0000","last_change":"2023-10-08T00:12:57.212771+0000","last_active":"2023-10-08T00:12:57.212771+0000","last_peered":"2023-10-08T00:12:57.212771+0000","last_clean":"2023-10-08T00:12:57.212771+0000","last_became_active":"2023-10-08T00:12:57.212479+0000","last_became_peered":"2023-10-08T00:12:57.212479+0000","last_unstale":"2023-10-08T00:12:57.212771+0000","last_undegraded":"2023-10-08T00:12:57.212771+0000","last_fullsized":"2023-10-08T00:12:57.212771+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:51:18.660991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.207674+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:13:42.175367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.209653+0000","last_change":"2023-10-08T00:12:57.209653+0000","last_active":"2023-10-08T00:12:57.209653+0000","last_peered":"2023-10-08T00:12:57.209653+0000","last_clean":"2023-10-08T00:12:57.209653+0000","last_became_active":"2023-10-08T00:12:57.209408+0000","last_became_peered":"2023-10-08T00:12:57.209408+0000","last_unstale":"2023-10-08T00:12:57.209653+0000","last_undegraded":"2023-10-08T00:12:57.209653+0000","last_fullsized":"2023-10-08T00:12:57.209653+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:26.648145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.207619+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:22:06.125225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.207927+0000","last_change":"2023-10-08T00:12:57.207927+0000","last_active":"2023-10-08T00:12:57.207927+0000","last_peered":"2023-10-08T00:12:57.207927+0000","last_clean":"2023-10-08T00:12:57.207927+0000","last_became_active":"2023-10-08T00:12:57.207442+0000","last_became_peered":"2023-10-08T00:12:57.207442+0000","last_unstale":"2023-10-08T00:12:57.207927+0000","last_undegraded":"2023-10-08T00:12:57.207927+0000","last_fullsized":"2023-10-08T00:12:57.207927+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:57:10.908242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.204176+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:24:53.646854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.212292+0000","last_change":"2023-10-08T00:12:57.212292+0000","last_active":"2023-10-08T00:12:57.212292+0000","last_peered":"2023-10-08T00:12:57.212292+0000","last_clean":"2023-10-08T00:12:57.212292+0000","last_became_active":"2023-10-08T00:12:57.211867+0000","last_became_peered":"2023-10-08T00:12:57.211867+0000","last_unstale":"2023-10-08T00:12:57.212292+0000","last_undegraded":"2023-10-08T00:12:57.212292+0000","last_fullsized":"2023-10-08T00:12:57.212292+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:36:52.303382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.208400+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:39:24.599431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.211573+0000","last_change":"2023-10-08T00:12:57.211573+0000","last_active":"2023-10-08T00:12:57.211573+0000","last_peered":"2023-10-08T00:12:57.211573+0000","last_clean":"2023-10-08T00:12:57.211573+0000","last_became_active":"2023-10-08T00:12:57.211202+0000","last_became_peered":"2023-10-08T00:12:57.211202+0000","last_unstale":"2023-10-08T00:12:57.211573+0000","last_undegraded":"2023-10-08T00:12:57.211573+0000","last_fullsized":"2023-10-08T00:12:57.211573+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:29:56.138577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.208004+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:47:28.589841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.205376+0000","last_change":"2023-10-08T00:12:57.205376+0000","last_active":"2023-10-08T00:12:57.205376+0000","last_peered":"2023-10-08T00:12:57.205376+0000","last_clean":"2023-10-08T00:12:57.205376+0000","last_became_active":"2023-10-08T00:12:57.205158+0000","last_became_peered":"2023-10-08T00:12:57.205158+0000","last_unstale":"2023-10-08T00:12:57.205376+0000","last_undegraded":"2023-10-08T00:12:57.205376+0000","last_fullsized":"2023-10-08T00:12:57.205376+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:15:43.196718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.204806+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:22:04.500698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.214270+0000","last_change":"2023-10-08T00:12:57.214270+0000","last_active":"2023-10-08T00:12:57.214270+0000","last_peered":"2023-10-08T00:12:57.214270+0000","last_clean":"2023-10-08T00:12:57.214270+0000","last_became_active":"2023-10-08T00:12:57.214082+0000","last_became_peered":"2023-10-08T00:12:57.214082+0000","last_unstale":"2023-10-08T00:12:57.214270+0000","last_undegraded":"2023-10-08T00:12:57.214270+0000","last_fullsized":"2023-10-08T00:12:57.214270+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:40:33.741781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.204948+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:03:56.099847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.209199+0000","last_change":"2023-10-08T00:12:57.209199+0000","last_active":"2023-10-08T00:12:57.209199+0000","last_peered":"2023-10-08T00:12:57.209199+0000","last_clean":"2023-10-08T00:12:57.209199+0000","last_became_active":"2023-10-08T00:12:57.208965+0000","last_became_peered":"2023-10-08T00:12:57.208965+0000","last_unstale":"2023-10-08T00:12:57.209199+0000","last_undegraded":"2023-10-08T00:12:57.209199+0000","last_fullsized":"2023-10-08T00:12:57.209199+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:02:39.029498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.207557+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:39:49.309778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.214795+0000","last_change":"2023-10-08T00:12:57.214795+0000","last_active":"2023-10-08T00:12:57.214795+0000","last_peered":"2023-10-08T00:12:57.214795+0000","last_clean":"2023-10-08T00:12:57.214795+0000","last_became_active":"2023-10-08T00:12:57.214269+0000","last_became_peered":"2023-10-08T00:12:57.214269+0000","last_unstale":"2023-10-08T00:12:57.214795+0000","last_undegraded":"2023-10-08T00:12:57.214795+0000","last_fullsized":"2023-10-08T00:12:57.214795+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:28:52.453441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206370+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:52:28.504130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.210087+0000","last_change":"2023-10-08T00:12:57.210087+0000","last_active":"2023-10-08T00:12:57.210087+0000","last_peered":"2023-10-08T00:12:57.210087+0000","last_clean":"2023-10-08T00:12:57.210087+0000","last_became_active":"2023-10-08T00:12:57.209574+0000","last_became_peered":"2023-10-08T00:12:57.209574+0000","last_unstale":"2023-10-08T00:12:57.210087+0000","last_undegraded":"2023-10-08T00:12:57.210087+0000","last_fullsized":"2023-10-08T00:12:57.210087+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:51:19.503936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.208930+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:44:29.429854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.209678+0000","last_change":"2023-10-08T00:12:57.209678+0000","last_active":"2023-10-08T00:12:57.209678+0000","last_peered":"2023-10-08T00:12:57.209678+0000","last_clean":"2023-10-08T00:12:57.209678+0000","last_became_active":"2023-10-08T00:12:57.209454+0000","last_became_peered":"2023-10-08T00:12:57.209454+0000","last_unstale":"2023-10-08T00:12:57.209678+0000","last_undegraded":"2023-10-08T00:12:57.209678+0000","last_fullsized":"2023-10-08T00:12:57.209678+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:51:46.972903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.203919+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:17:10.296378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.215576+0000","last_change":"2023-10-08T00:12:57.215576+0000","last_active":"2023-10-08T00:12:57.215576+0000","last_peered":"2023-10-08T00:12:57.215576+0000","last_clean":"2023-10-08T00:12:57.215576+0000","last_became_active":"2023-10-08T00:12:57.215200+0000","last_became_peered":"2023-10-08T00:12:57.215200+0000","last_unstale":"2023-10-08T00:12:57.215576+0000","last_undegraded":"2023-10-08T00:12:57.215576+0000","last_fullsized":"2023-10-08T00:12:57.215576+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:11:40.080191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.207660+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:50:14.863709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.208075+0000","last_change":"2023-10-08T00:12:57.208075+0000","last_active":"2023-10-08T00:12:57.208075+0000","last_peered":"2023-10-08T00:12:57.208075+0000","last_clean":"2023-10-08T00:12:57.208075+0000","last_became_active":"2023-10-08T00:12:57.207720+0000","last_became_peered":"2023-10-08T00:12:57.207720+0000","last_unstale":"2023-10-08T00:12:57.208075+0000","last_undegraded":"2023-10-08T00:12:57.208075+0000","last_fullsized":"2023-10-08T00:12:57.208075+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:23:41.947350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.203096+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:43:33.796004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.204277+0000","last_change":"2023-10-08T00:12:57.204277+0000","last_active":"2023-10-08T00:12:57.204277+0000","last_peered":"2023-10-08T00:12:57.204277+0000","last_clean":"2023-10-08T00:12:57.204277+0000","last_became_active":"2023-10-08T00:12:57.204039+0000","last_became_peered":"2023-10-08T00:12:57.204039+0000","last_unstale":"2023-10-08T00:12:57.204277+0000","last_undegraded":"2023-10-08T00:12:57.204277+0000","last_fullsized":"2023-10-08T00:12:57.204277+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:09:18.935331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.204614+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:04:37.043481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.210213+0000","last_change":"2023-10-08T00:12:57.210213+0000","last_active":"2023-10-08T00:12:57.210213+0000","last_peered":"2023-10-08T00:12:57.210213+0000","last_clean":"2023-10-08T00:12:57.210213+0000","last_became_active":"2023-10-08T00:12:57.209897+0000","last_became_peered":"2023-10-08T00:12:57.209897+0000","last_unstale":"2023-10-08T00:12:57.210213+0000","last_undegraded":"2023-10-08T00:12:57.210213+0000","last_fullsized":"2023-10-08T00:12:57.210213+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:49:05.184189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.207309+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:24:53.646854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.210506+0000","last_change":"2023-10-08T00:12:57.210506+0000","last_active":"2023-10-08T00:12:57.210506+0000","last_peered":"2023-10-08T00:12:57.210506+0000","last_clean":"2023-10-08T00:12:57.210506+0000","last_became_active":"2023-10-08T00:12:57.210180+0000","last_became_peered":"2023-10-08T00:12:57.210180+0000","last_unstale":"2023-10-08T00:12:57.210506+0000","last_undegraded":"2023-10-08T00:12:57.210506+0000","last_fullsized":"2023-10-08T00:12:57.210506+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:57:52.142531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206202+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:37:04.823729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.215419+0000","last_change":"2023-10-08T00:12:57.215419+0000","last_active":"2023-10-08T00:12:57.215419+0000","last_peered":"2023-10-08T00:12:57.215419+0000","last_clean":"2023-10-08T00:12:57.215419+0000","last_became_active":"2023-10-08T00:12:57.214895+0000","last_became_peered":"2023-10-08T00:12:57.214895+0000","last_unstale":"2023-10-08T00:12:57.215419+0000","last_undegraded":"2023-10-08T00:12:57.215419+0000","last_fullsized":"2023-10-08T00:12:57.215419+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:59:43.036033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.205871+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:25:36.388792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.211758+0000","last_change":"2023-10-08T00:12:57.211758+0000","last_active":"2023-10-08T00:12:57.211758+0000","last_peered":"2023-10-08T00:12:57.211758+0000","last_clean":"2023-10-08T00:12:57.211758+0000","last_became_active":"2023-10-08T00:12:57.211537+0000","last_became_peered":"2023-10-08T00:12:57.211537+0000","last_unstale":"2023-10-08T00:12:57.211758+0000","last_undegraded":"2023-10-08T00:12:57.211758+0000","last_fullsized":"2023-10-08T00:12:57.211758+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:42:52.161510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.205056+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:33:58.167385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.211562+0000","last_change":"2023-10-08T00:12:57.211562+0000","last_active":"2023-10-08T00:12:57.211562+0000","last_peered":"2023-10-08T00:12:57.211562+0000","last_clean":"2023-10-08T00:12:57.211562+0000","last_became_active":"2023-10-08T00:12:57.211290+0000","last_became_peered":"2023-10-08T00:12:57.211290+0000","last_unstale":"2023-10-08T00:12:57.211562+0000","last_undegraded":"2023-10-08T00:12:57.211562+0000","last_fullsized":"2023-10-08T00:12:57.211562+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:36:52.303382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.203529+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:32:16.626313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.212893+0000","last_change":"2023-10-08T00:12:57.212893+0000","last_active":"2023-10-08T00:12:57.212893+0000","last_peered":"2023-10-08T00:12:57.212893+0000","last_clean":"2023-10-08T00:12:57.212893+0000","last_became_active":"2023-10-08T00:12:57.212632+0000","last_became_peered":"2023-10-08T00:12:57.212632+0000","last_unstale":"2023-10-08T00:12:57.212893+0000","last_undegraded":"2023-10-08T00:12:57.212893+0000","last_fullsized":"2023-10-08T00:12:57.212893+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:00:48.158147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.203581+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:56:37.046732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.214282+0000","last_change":"2023-10-08T00:12:57.214282+0000","last_active":"2023-10-08T00:12:57.214282+0000","last_peered":"2023-10-08T00:12:57.214282+0000","last_clean":"2023-10-08T00:12:57.214282+0000","last_became_active":"2023-10-08T00:12:57.214017+0000","last_became_peered":"2023-10-08T00:12:57.214017+0000","last_unstale":"2023-10-08T00:12:57.214282+0000","last_undegraded":"2023-10-08T00:12:57.214282+0000","last_fullsized":"2023-10-08T00:12:57.214282+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:52:20.648298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.203749+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:15:27.299505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.210417+0000","last_change":"2023-10-08T00:12:57.210417+0000","last_active":"2023-10-08T00:12:57.210417+0000","last_peered":"2023-10-08T00:12:57.210417+0000","last_clean":"2023-10-08T00:12:57.210417+0000","last_became_active":"2023-10-08T00:12:57.210168+0000","last_became_peered":"2023-10-08T00:12:57.210168+0000","last_unstale":"2023-10-08T00:12:57.210417+0000","last_undegraded":"2023-10-08T00:12:57.210417+0000","last_fullsized":"2023-10-08T00:12:57.210417+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:51:53.256123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206067+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:11:41.089416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.213694+0000","last_change":"2023-10-08T00:12:57.213694+0000","last_active":"2023-10-08T00:12:57.213694+0000","last_peered":"2023-10-08T00:12:57.213694+0000","last_clean":"2023-10-08T00:12:57.213694+0000","last_became_active":"2023-10-08T00:12:57.213466+0000","last_became_peered":"2023-10-08T00:12:57.213466+0000","last_unstale":"2023-10-08T00:12:57.213694+0000","last_undegraded":"2023-10-08T00:12:57.213694+0000","last_fullsized":"2023-10-08T00:12:57.213694+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:50:16.963706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.205826+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:28:53.462666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.211421+0000","last_change":"2023-10-08T00:12:57.211421+0000","last_active":"2023-10-08T00:12:57.211421+0000","last_peered":"2023-10-08T00:12:57.211421+0000","last_clean":"2023-10-08T00:12:57.211421+0000","last_became_active":"2023-10-08T00:12:57.210901+0000","last_became_peered":"2023-10-08T00:12:57.210901+0000","last_unstale":"2023-10-08T00:12:57.211421+0000","last_undegraded":"2023-10-08T00:12:57.211421+0000","last_fullsized":"2023-10-08T00:12:57.211421+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:43:05.018683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.205536+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:35:32.137447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.206333+0000","last_change":"2023-10-08T00:12:57.206333+0000","last_active":"2023-10-08T00:12:57.206333+0000","last_peered":"2023-10-08T00:12:57.206333+0000","last_clean":"2023-10-08T00:12:57.206333+0000","last_became_active":"2023-10-08T00:12:57.206123+0000","last_became_peered":"2023-10-08T00:12:57.206123+0000","last_unstale":"2023-10-08T00:12:57.206333+0000","last_undegraded":"2023-10-08T00:12:57.206333+0000","last_fullsized":"2023-10-08T00:12:57.206333+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:27:43.372597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.204798+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:47:28.589841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.212538+0000","last_change":"2023-10-08T00:12:57.212538+0000","last_active":"2023-10-08T00:12:57.212538+0000","last_peered":"2023-10-08T00:12:57.212538+0000","last_clean":"2023-10-08T00:12:57.212538+0000","last_became_active":"2023-10-08T00:12:57.212360+0000","last_became_peered":"2023-10-08T00:12:57.212360+0000","last_unstale":"2023-10-08T00:12:57.212538+0000","last_undegraded":"2023-10-08T00:12:57.212538+0000","last_fullsized":"2023-10-08T00:12:57.212538+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:39:23.590206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.206666+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:40:34.751005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.213785+0000","last_change":"2023-10-08T00:12:57.213785+0000","last_active":"2023-10-08T00:12:57.213785+0000","last_peered":"2023-10-08T00:12:57.213785+0000","last_clean":"2023-10-08T00:12:57.213785+0000","last_became_active":"2023-10-08T00:12:57.213351+0000","last_became_peered":"2023-10-08T00:12:57.213351+0000","last_unstale":"2023-10-08T00:12:57.213785+0000","last_undegraded":"2023-10-08T00:12:57.213785+0000","last_fullsized":"2023-10-08T00:12:57.213785+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:17:09.287153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.205741+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:22:06.125225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.209114+0000","last_change":"2023-10-08T00:12:57.209114+0000","last_active":"2023-10-08T00:12:57.209114+0000","last_peered":"2023-10-08T00:12:57.209114+0000","last_clean":"2023-10-08T00:12:57.209114+0000","last_became_active":"2023-10-08T00:12:57.208878+0000","last_became_peered":"2023-10-08T00:12:57.208878+0000","last_unstale":"2023-10-08T00:12:57.209114+0000","last_undegraded":"2023-10-08T00:12:57.209114+0000","last_fullsized":"2023-10-08T00:12:57.209114+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:29:06.434469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.204353+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:50:14.863709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.213998+0000","last_change":"2023-10-08T00:12:57.213998+0000","last_active":"2023-10-08T00:12:57.213998+0000","last_peered":"2023-10-08T00:12:57.213998+0000","last_clean":"2023-10-08T00:12:57.213998+0000","last_became_active":"2023-10-08T00:12:57.213768+0000","last_became_peered":"2023-10-08T00:12:57.213768+0000","last_unstale":"2023-10-08T00:12:57.213998+0000","last_undegraded":"2023-10-08T00:12:57.213998+0000","last_fullsized":"2023-10-08T00:12:57.213998+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:36:23.496893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.205083+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:56:09.457506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.209066+0000","last_change":"2023-10-08T00:12:57.209066+0000","last_active":"2023-10-08T00:12:57.209066+0000","last_peered":"2023-10-08T00:12:57.209066+0000","last_clean":"2023-10-08T00:12:57.209066+0000","last_became_active":"2023-10-08T00:12:57.208849+0000","last_became_peered":"2023-10-08T00:12:57.208849+0000","last_unstale":"2023-10-08T00:12:57.209066+0000","last_undegraded":"2023-10-08T00:12:57.209066+0000","last_fullsized":"2023-10-08T00:12:57.209066+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:29:06.434469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:12:57.194440+0000","last_change":"2023-10-08T00:12:57.204310+0000","last_active":"2023-10-08T00:12:57.194440+0000","last_peered":"2023-10-08T00:12:57.194440+0000","last_clean":"2023-10-08T00:12:57.194440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:12:57.194440+0000","last_undegraded":"2023-10-08T00:12:57.194440+0000","last_fullsized":"2023-10-08T00:12:57.194440+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:39:49.309778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":8,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-08T00:12:57.212524+0000","last_change":"2023-10-08T00:12:57.212524+0000","last_active":"2023-10-08T00:12:57.212524+0000","last_peered":"2023-10-08T00:12:57.212524+0000","last_clean":"2023-10-08T00:12:57.212524+0000","last_became_active":"2023-10-08T00:12:57.212291+0000","last_became_peered":"2023-10-08T00:12:57.212291+0000","last_unstale":"2023-10-08T00:12:57.212524+0000","last_undegraded":"2023-10-08T00:12:57.212524+0000","last_fullsized":"2023-10-08T00:12:57.212524+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:00:48.158147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":199,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-08T00:12:56.201662+0000","last_change":"2023-10-08T00:10:43.728490+0000","last_active":"2023-10-08T00:12:56.201662+0000","last_peered":"2023-10-08T00:12:56.201662+0000","last_clean":"2023-10-08T00:12:56.201662+0000","last_became_active":"2023-10-08T00:10:43.728341+0000","last_became_peered":"2023-10-08T00:10:43.728341+0000","last_unstale":"2023-10-08T00:12:56.201662+0000","last_undegraded":"2023-10-08T00:12:56.201662+0000","last_fullsized":"2023-10-08T00:12:56.201662+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_clean_scrub_stamp":"2023-10-08T00:10:42.716096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:06:31.035293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":336,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194976,"statfs":{"total":200000000,"available":199655719,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.496,"15min":0.49399999999999999},"min":{"1min":0.34599999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63100000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.53800000000000003,"15min":0.54400000000000004},"min":{"1min":0.38100000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.73499999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.40300000000000002}]},{"osd":1,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.53100000000000003,"15min":0.52700000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.437,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.56999999999999995}]},{"osd":2,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.53000000000000003,"15min":0.53300000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.66600000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.51600000000000001,"15min":0.51400000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.71899999999999997,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.55900000000000005}]},{"osd":3,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.51700000000000002,"15min":0.51300000000000001},"min":{"1min":0.36599999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.55200000000000005,"15min":0.55400000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.47099999999999997}]},{"osd":4,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.52200000000000002,"15min":0.52600000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.70799999999999996,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.497},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.52800000000000002,"15min":0.53200000000000003},"min":{"1min":0.35699999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.64300000000000002,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.34599999999999997}]},{"osd":5,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.55300000000000005,"15min":0.55700000000000005},"min":{"1min":0.32100000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.71999999999999997,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.57999999999999996,"15min":0.59199999999999997},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.69399999999999995,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.58099999999999996}]},{"osd":6,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.59399999999999997,"15min":0.59899999999999998},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.71299999999999997,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.627,"15min":0.63800000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76000000000000001,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.52800000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":336,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194976,"statfs":{"total":200000000,"available":199655719,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.48099999999999998,"15min":0.47499999999999998},"min":{"1min":0.377,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.752,"5min":1.012,"15min":1.012},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.68400000000000005,"15min":0.70499999999999996},"min":{"1min":0.34000000000000002,"5min":0.317,"15min":0.317},"max":{"1min":1.2789999999999999,"5min":5.1230000000000002,"15min":5.1230000000000002},"last":0.374}]},{"osd":1,"last update":"Sun Oct 8 00:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.52100000000000002,"15min":0.51400000000000001},"min":{"1min":0.29299999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.82499999999999996,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55900000000000005,"15min":0.56000000000000005},"min":{"1min":0.371,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.69899999999999995,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.45400000000000001}]},{"osd":2,"last update":"Sun Oct 8 00:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.52700000000000002,"15min":0.52500000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.755,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56200000000000006,"15min":0.56499999999999995},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.76100000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.48899999999999999}]},{"osd":3,"last update":"Sun Oct 8 00:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53400000000000003,"15min":0.53000000000000003},"min":{"1min":0.38400000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.81399999999999995,"5min":1.046,"15min":1.046},"last":0.501},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.54400000000000004,"15min":0.53600000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.36099999999999999}]},{"osd":4,"last update":"Sun Oct 8 00:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.502,"15min":0.50600000000000001},"min":{"1min":0.32300000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56499999999999995,"15min":0.56699999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.42999999999999999}]},{"osd":5,"last update":"Sun Oct 8 00:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.48499999999999999,"15min":0.48799999999999999},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71099999999999997,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.55500000000000005,"15min":0.56299999999999994},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.439}]},{"osd":7,"last update":"Sun Oct 8 00:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.54100000000000004,"15min":0.54500000000000004},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.41699999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574877,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":912,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194399,"statfs":{"total":200000000,"available":199065351,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.30399999999999999,"15min":0.29899999999999999},"min":{"1min":0.218,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.44700000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.375,"15min":0.373},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.38300000000000001}]},{"osd":2,"last update":"Sun Oct 8 00:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.33600000000000002,"15min":0.33200000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.49099999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.379,"5min":0.33900000000000002,"15min":0.33200000000000002},"min":{"1min":0.254,"5min":0.252,"15min":0.252},"max":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.34999999999999998}]},{"osd":3,"last update":"Sun Oct 8 00:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.36699999999999999,"15min":0.36499999999999999},"min":{"1min":0.28299999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.307},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.501,"5min":0.501,"15min":0.501},"last":0.41999999999999998}]},{"osd":4,"last update":"Sun Oct 8 00:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.53700000000000003,"15min":0.53400000000000003},"min":{"1min":0.40400000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.56899999999999995,"15min":0.56499999999999995},"min":{"1min":0.41899999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.82299999999999995,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.60399999999999998}]},{"osd":5,"last update":"Sun Oct 8 00:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.53800000000000003,"15min":0.53100000000000003},"min":{"1min":0.372,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57499999999999996,"15min":0.57199999999999995},"min":{"1min":0.42999999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78300000000000003,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.73499999999999999}]},{"osd":6,"last update":"Sun Oct 8 00:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.72999999999999998,"5min":0.877,"15min":0.877},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56399999999999995,"15min":0.55800000000000005},"min":{"1min":0.42899999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.76900000000000002,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.61899999999999999}]},{"osd":7,"last update":"Sun Oct 8 00:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56299999999999994,"15min":0.55900000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.77700000000000002,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.752},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.58899999999999997,"15min":0.58499999999999996},"min":{"1min":0.432,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82299999999999995,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.69999999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":336,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194976,"statfs":{"total":200000000,"available":199655719,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 00:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.32400000000000001,"15min":0.32600000000000001},"min":{"1min":0.23499999999999999,"5min":0.193,"15min":0.193},"max":{"1min":0.41499999999999998,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.308},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.245,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.49299999999999999,"5min":0.87,"15min":0.87},"last":0.40699999999999997}]},{"osd":2,"last update":"Sun Oct 8 00:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.38200000000000001,"15min":0.38600000000000001},"min":{"1min":0.27900000000000003,"5min":0.251,"15min":0.251},"max":{"1min":0.58699999999999997,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.753,"15min":0.82399999999999995},"min":{"1min":0.247,"5min":0.185,"15min":0.185},"max":{"1min":0.40899999999999997,"5min":12.019,"15min":12.019},"last":0.38900000000000001}]},{"osd":3,"last update":"Sun Oct 8 00:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.35599999999999998,"15min":0.35999999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.47399999999999998,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.374,"5min":0.38800000000000001,"15min":0.39000000000000001},"min":{"1min":0.28499999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.55900000000000005,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.33600000000000002}]},{"osd":4,"last update":"Sun Oct 8 00:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.433,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.71899999999999997,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.624,"15min":0.625},"min":{"1min":0.48299999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.73499999999999999,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.71799999999999997}]},{"osd":5,"last update":"Sun Oct 8 00:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.999,"15min":1.069},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.79800000000000004,"5min":11.992000000000001,"15min":11.992000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.73199999999999998,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.61299999999999999}]},{"osd":6,"last update":"Sun Oct 8 00:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.74399999999999999,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.97799999999999998,"15min":1.042},"min":{"1min":0.47299999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76000000000000001,"5min":11.927,"15min":11.927},"last":0.72499999999999998}]},{"osd":7,"last update":"Sun Oct 8 00:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.999,"15min":1.0660000000000001},"min":{"1min":0.434,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82199999999999995,"5min":11.965,"15min":11.965},"last":0.627},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.61899999999999999,"15min":0.621},"min":{"1min":0.48199999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.76100000000000001,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.76900000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":336,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194976,"statfs":{"total":200000000,"available":199655719,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.39000000000000001,"15min":0.40200000000000002},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.437,"5min":2.5139999999999998,"15min":2.5139999999999998},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.40100000000000002,"15min":0.41399999999999998},"min":{"1min":0.248,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.44400000000000001,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":0.35899999999999999}]},{"osd":1,"last update":"Sun Oct 8 00:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.39000000000000001,"15min":0.40200000000000002},"min":{"1min":0.25,"5min":0.246,"15min":0.246},"max":{"1min":0.40100000000000002,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.436,"15min":0.45000000000000001},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.48999999999999999,"5min":2.5379999999999998,"15min":2.5379999999999998},"last":0.39200000000000002}]},{"osd":3,"last update":"Sun Oct 8 00:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.42799999999999999,"15min":0.439},"min":{"1min":0.28499999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.46500000000000002,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":0.378},{"interface":"front","average":{"1min":0.379,"5min":0.46300000000000002,"15min":0.47699999999999998},"min":{"1min":0.315,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.51200000000000001,"5min":2.5619999999999998,"15min":2.5619999999999998},"last":0.41499999999999998}]},{"osd":4,"last update":"Sun Oct 8 00:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.61699999999999999,"15min":0.628},"min":{"1min":0.45800000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.84599999999999997,"5min":2.488,"15min":2.488},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.60399999999999998,"15min":0.60999999999999999},"min":{"1min":0.41099999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85999999999999999,"5min":2.073,"15min":2.073},"last":0.81100000000000005}]},{"osd":5,"last update":"Sun Oct 8 00:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.66300000000000003,"15min":0.66900000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.2130000000000001,"5min":2.2069999999999999,"15min":2.2069999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.62,"5min":0.66300000000000003,"15min":0.67000000000000004},"min":{"1min":0.48799999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0149999999999999,"5min":2.456,"15min":2.456},"last":0.81999999999999995}]},{"osd":6,"last update":"Sun Oct 8 00:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.621,"15min":0.627},"min":{"1min":0.45000000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.79500000000000004,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.60899999999999999,"15min":0.61399999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.97499999999999998,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.77000000000000002}]},{"osd":7,"last update":"Sun Oct 8 00:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.63700000000000001,"15min":0.64100000000000001},"min":{"1min":0.499,"5min":0.438,"15min":0.438},"max":{"1min":0.81699999999999995,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.626,"15min":0.63200000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91200000000000003,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":0.64500000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":330,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194982,"statfs":{"total":200000000,"available":199661982,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.28699999999999998,"5min":0.36699999999999999,"15min":0.38},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.45100000000000001,"5min":1.46,"15min":1.46},"last":0.251},{"interface":"front","average":{"1min":0.29699999999999999,"5min":0.33500000000000002,"15min":0.34100000000000003},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.46300000000000002,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.23300000000000001}]},{"osd":1,"last update":"Sun Oct 8 00:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.82999999999999996,"15min":0.91700000000000004},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.44800000000000001,"5min":12.351000000000001,"15min":12.351000000000001},"last":0.307},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.35899999999999999,"15min":0.36299999999999999},"min":{"1min":0.26500000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.443,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.34300000000000003}]},{"osd":2,"last update":"Sun Oct 8 00:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.41399999999999998,"15min":0.42499999999999999},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.47299999999999998,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.377},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.379,"15min":0.38300000000000001},"min":{"1min":0.27800000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.45900000000000002,"5min":0.753,"15min":0.753},"last":0.41099999999999998}]},{"osd":4,"last update":"Sun Oct 8 00:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":1.004,"15min":1.0860000000000001},"min":{"1min":0.38800000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.75600000000000001,"5min":11.333,"15min":11.333},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":1.0369999999999999,"15min":1.1200000000000001},"min":{"1min":0.434,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.77000000000000002,"5min":12.321999999999999,"15min":12.321999999999999},"last":0.63200000000000001}]},{"osd":5,"last update":"Sun Oct 8 00:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":1.0509999999999999,"15min":1.137},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.72599999999999998,"5min":12.291,"15min":12.291},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.58599999999999997,"15min":0.59299999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.74299999999999999,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.59699999999999998}]},{"osd":6,"last update":"Sun Oct 8 00:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.96999999999999997,"15min":1.048},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.69899999999999995,"5min":11.288,"15min":11.288},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.51800000000000002,"5min":1.0309999999999999,"15min":1.117},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.66000000000000003,"5min":12.253,"15min":12.253},"last":0.67500000000000004}]},{"osd":7,"last update":"Sun Oct 8 00:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56599999999999995,"15min":0.56799999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.56399999999999995,"5min":1.012,"15min":1.0860000000000001},"min":{"1min":0.435,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.381,"5min":11.363,"15min":11.363},"last":0.66600000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":336,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194976,"statfs":{"total":200000000,"available":199655719,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.56000000000000005,"15min":0.55400000000000005},"min":{"1min":0.41299999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.375},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.50600000000000001}]},{"osd":1,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51700000000000002,"15min":0.51800000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.64100000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.42099999999999999}]},{"osd":2,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.51600000000000001,"15min":0.52000000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.51300000000000001,"15min":0.50900000000000001},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.54200000000000004}]},{"osd":3,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.53500000000000003,"15min":0.53900000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.70899999999999996,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.53000000000000003,"15min":0.53200000000000003},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.40200000000000002}]},{"osd":5,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.49299999999999999,"15min":0.503},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69899999999999995,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.50800000000000001,"15min":0.51400000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.71099999999999997,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.57199999999999995}]},{"osd":6,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.48899999999999999,"15min":0.49399999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.55200000000000005,"15min":0.56200000000000006},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.754,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.58599999999999997}]},{"osd":7,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.54600000000000004,"15min":0.55500000000000005},"min":{"1min":0.36299999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.64200000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.57799999999999996,"15min":0.58299999999999996},"min":{"1min":0.41199999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.77700000000000002,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.628}]}]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":906,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194405,"statfs":{"total":200000000,"available":199071614,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.48699999999999999,"15min":0.48799999999999999},"min":{"1min":0.32700000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68799999999999994,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.441},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.55700000000000005,"15min":0.56299999999999994},"min":{"1min":0.374,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.68600000000000005,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.58599999999999997}]},{"osd":1,"last update":"Sun Oct 8 00:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.52100000000000002,"15min":0.52300000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.65800000000000003,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.52400000000000002,"15min":0.52800000000000002},"min":{"1min":0.39300000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.70899999999999996,"5min":0.755,"15min":0.755},"last":0.48999999999999999}]},{"osd":2,"last update":"Sun Oct 8 00:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.52900000000000003,"15min":0.53100000000000003},"min":{"1min":0.40500000000000003,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.64500000000000002,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.56599999999999995,"15min":0.57199999999999995},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.70699999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.59799999999999998}]},{"osd":3,"last update":"Sun Oct 8 00:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54300000000000004,"15min":0.54500000000000004},"min":{"1min":0.38700000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.67100000000000004,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.57999999999999996,"15min":0.58399999999999996},"min":{"1min":0.42799999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.72299999999999998,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.55900000000000005}]},{"osd":4,"last update":"Sun Oct 8 00:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.47299999999999998,"15min":0.47499999999999998},"min":{"1min":0.35799999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.626,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.50900000000000001,"15min":0.51300000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.63700000000000001,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.54600000000000004}]},{"osd":6,"last update":"Sun Oct 8 00:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.499,"15min":0.499},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.61899999999999999,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.66800000000000004,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.53200000000000003}]},{"osd":7,"last update":"Sun Oct 8 00:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.51300000000000001,"15min":0.51500000000000001},"min":{"1min":0.33400000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67300000000000004,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.69799999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.51600000000000001}]}]}],"pool_statfs":[]}} 2023-10-08T00:13:06.780 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T00:13:06.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.907+0000 7ff6c492e700 1 Processor -- start 2023-10-08T00:13:06.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.907+0000 7ff6c492e700 1 -- start start 2023-10-08T00:13:06.910 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.908+0000 7ff6c492e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6c00cd6e0 0x7ff6c00cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:06.910 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.908+0000 7ff6c492e700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff6c015cbd0 0x7ff6c015d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:06.910 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.908+0000 7ff6c492e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6c00cb7c0 con 0x7ff6c00cdab0 2023-10-08T00:13:06.910 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.908+0000 7ff6c492e700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6c015e850 con 0x7ff6c015cbd0 2023-10-08T00:13:06.911 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.908+0000 7ff6c492e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c00cd030 con 0x7ff6c00cd6e0 2023-10-08T00:13:06.911 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.908+0000 7ff6bf59e700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff6c015cbd0 0x7ff6c015d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:06.911 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.908+0000 7ff6bf59e700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff6c015cbd0 0x7ff6c015d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46278/0 (socket says 172.21.15.99:46278) 2023-10-08T00:13:06.911 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.908+0000 7ff6bf59e700 1 -- 172.21.15.99:0/17865360 learned_addr learned my addr 172.21.15.99:0/17865360 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:13:06.911 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.908+0000 7ff6bed9d700 1 --2- 172.21.15.99:0/17865360 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6c00cd6e0 0x7ff6c00cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:06.911 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.909+0000 7ff6bf59e700 1 -- 172.21.15.99:0/17865360 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6c00cd6e0 msgr2=0x7ff6c00cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:13:06.911 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.909+0000 7ff6bf59e700 1 --2- 172.21.15.99:0/17865360 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6c00cd6e0 0x7ff6c00cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:06.911 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.909+0000 7ff6bf59e700 1 -- 172.21.15.99:0/17865360 >> v1:172.21.15.191:6789/0 conn(0x7ff6c00cdab0 legacy=0x7ff6c015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:13:06.911 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.909+0000 7ff6bf59e700 1 -- 172.21.15.99:0/17865360 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6c015e290 con 0x7ff6c015cbd0 2023-10-08T00:13:06.911 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.909+0000 7ff6bed9d700 1 --2- 172.21.15.99:0/17865360 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6c00cd6e0 0x7ff6c00cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:13:06.912 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.909+0000 7ff6bf59e700 1 --2- 172.21.15.99:0/17865360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6c015cbd0 0x7ff6c015d020 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8009f20 tx=0x7ff6b8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3be6d2eb6ea83b51 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:13:06.912 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.909+0000 7ff6be59c700 1 -- 172.21.15.99:0/17865360 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b800e070 con 0x7ff6c015cbd0 2023-10-08T00:13:06.912 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.909+0000 7ff6be59c700 1 -- 172.21.15.99:0/17865360 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6b800ba40 con 0x7ff6c015cbd0 2023-10-08T00:13:06.912 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.909+0000 7ff6be59c700 1 -- 172.21.15.99:0/17865360 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b80092e0 con 0x7ff6c015cbd0 2023-10-08T00:13:06.912 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.909+0000 7ff6c492e700 1 -- 172.21.15.99:0/17865360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6c015cbd0 msgr2=0x7ff6c015d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:06.912 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.909+0000 7ff6c492e700 1 --2- 172.21.15.99:0/17865360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6c015cbd0 0x7ff6c015d020 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8009f20 tx=0x7ff6b8009840 comp rx=0 tx=0).stop 2023-10-08T00:13:06.912 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.909+0000 7ff6c492e700 1 -- 172.21.15.99:0/17865360 shutdown_connections 2023-10-08T00:13:06.912 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.909+0000 7ff6c492e700 1 --2- 172.21.15.99:0/17865360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6c015cbd0 0x7ff6c015d020 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:06.912 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.909+0000 7ff6c492e700 1 --2- 172.21.15.99:0/17865360 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6c00cd6e0 0x7ff6c00cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:06.912 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.909+0000 7ff6c492e700 1 -- 172.21.15.99:0/17865360 >> 172.21.15.99:0/17865360 conn(0x7ff6c0067660 msgr2=0x7ff6c0067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:13:06.913 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.909+0000 7ff6c492e700 1 -- 172.21.15.99:0/17865360 shutdown_connections 2023-10-08T00:13:06.913 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.909+0000 7ff6c492e700 1 -- 172.21.15.99:0/17865360 wait complete. 2023-10-08T00:13:06.915 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.911+0000 7ff6c492e700 1 Processor -- start 2023-10-08T00:13:06.915 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.911+0000 7ff6c492e700 1 -- start start 2023-10-08T00:13:06.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.911+0000 7ff6c492e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6c00cd6e0 0x7ff6c0155530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:06.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.911+0000 7ff6c492e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6c00cdab0 0x7ff6c00b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:06.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.911+0000 7ff6c492e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6c015cbd0 0x7ff6c00c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:06.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.911+0000 7ff6c492e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c00b5c00 con 0x7ff6c015cbd0 2023-10-08T00:13:06.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.911+0000 7ff6c492e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c00c9420 con 0x7ff6c00cdab0 2023-10-08T00:13:06.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.911+0000 7ff6c492e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c00b52d0 con 0x7ff6c00cd6e0 2023-10-08T00:13:06.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.911+0000 7ff6bf59e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6c015cbd0 0x7ff6c00c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:06.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.911+0000 7ff6bf59e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6c015cbd0 0x7ff6c00c8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46282/0 (socket says 172.21.15.99:46282) 2023-10-08T00:13:06.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.911+0000 7ff6bf59e700 1 -- 172.21.15.99:0/1262118271 learned_addr learned my addr 172.21.15.99:0/1262118271 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:13:06.917 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.912+0000 7ff6bed9d700 1 --2- 172.21.15.99:0/1262118271 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6c00cd6e0 0x7ff6c0155530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:06.917 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.912+0000 7ff6b7fff700 1 --2- 172.21.15.99:0/1262118271 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6c00cdab0 0x7ff6c00b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:06.917 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.912+0000 7ff6bf59e700 1 -- 172.21.15.99:0/1262118271 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6c00cd6e0 msgr2=0x7ff6c0155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:06.917 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.912+0000 7ff6bf59e700 1 --2- 172.21.15.99:0/1262118271 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6c00cd6e0 0x7ff6c0155530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:06.917 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.912+0000 7ff6bf59e700 1 -- 172.21.15.99:0/1262118271 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6c00cdab0 msgr2=0x7ff6c00b6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:06.917 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.912+0000 7ff6bf59e700 1 --2- 172.21.15.99:0/1262118271 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6c00cdab0 0x7ff6c00b6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:06.917 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.912+0000 7ff6bf59e700 1 -- 172.21.15.99:0/1262118271 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b8009ae0 con 0x7ff6c015cbd0 2023-10-08T00:13:06.917 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.912+0000 7ff6b7fff700 1 --2- 172.21.15.99:0/1262118271 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6c00cdab0 0x7ff6c00b6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:13:06.917 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.912+0000 7ff6bed9d700 1 --2- 172.21.15.99:0/1262118271 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6c00cd6e0 0x7ff6c0155530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:13:06.917 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.912+0000 7ff6bf59e700 1 --2- 172.21.15.99:0/1262118271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6c015cbd0 0x7ff6c00c8a40 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ff6c00c7b10 tx=0x7ff6b8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:13:06.918 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.912+0000 7ff6b6ffd700 1 -- 172.21.15.99:0/1262118271 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b8012e60 con 0x7ff6c015cbd0 2023-10-08T00:13:06.918 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.912+0000 7ff6b6ffd700 1 -- 172.21.15.99:0/1262118271 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6b801d420 con 0x7ff6c015cbd0 2023-10-08T00:13:06.918 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.912+0000 7ff6c492e700 1 -- 172.21.15.99:0/1262118271 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6c00b55b0 con 0x7ff6c015cbd0 2023-10-08T00:13:06.918 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.912+0000 7ff6b6ffd700 1 -- 172.21.15.99:0/1262118271 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b801d7e0 con 0x7ff6c015cbd0 2023-10-08T00:13:06.918 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.912+0000 7ff6c492e700 1 -- 172.21.15.99:0/1262118271 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6c0155a70 con 0x7ff6c015cbd0 2023-10-08T00:13:06.918 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.913+0000 7ff6c492e700 1 -- 172.21.15.99:0/1262118271 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6c00b5db0 con 0x7ff6c015cbd0 2023-10-08T00:13:06.918 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.913+0000 7ff6b6ffd700 1 -- 172.21.15.99:0/1262118271 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff6b801db90 con 0x7ff6c015cbd0 2023-10-08T00:13:06.918 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.914+0000 7ff6b6ffd700 1 --2- 172.21.15.99:0/1262118271 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff6a0066f30 0x7ff6a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:06.918 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.914+0000 7ff6b6ffd700 1 -- 172.21.15.99:0/1262118271 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff6b8085f80 con 0x7ff6c015cbd0 2023-10-08T00:13:06.918 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.914+0000 7ff6bed9d700 1 --2- 172.21.15.99:0/1262118271 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff6a0066f30 0x7ff6a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:06.918 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.914+0000 7ff6bed9d700 1 --2- 172.21.15.99:0/1262118271 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff6a0066f30 0x7ff6a00693e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0006fd0 tx=0x7ff6b0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:13:06.919 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:06.918+0000 7ff6b6ffd700 1 -- 172.21.15.99:0/1262118271 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff6b80b75a0 con 0x7ff6c015cbd0 2023-10-08T00:13:07.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.095+0000 7ff6c492e700 1 -- 172.21.15.99:0/1262118271 --> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff6c015e570 con 0x7ff6a0066f30 2023-10-08T00:13:07.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.104+0000 7ff6b6ffd700 1 -- 172.21.15.99:0/1262118271 <== mgr.4099 v2:172.21.15.191:6832/85985 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284691 (secure 0 0 0) 0x7ff6c015e570 con 0x7ff6a0066f30 2023-10-08T00:13:07.107 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:13:07.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.106+0000 7ff6c492e700 1 -- 172.21.15.99:0/1262118271 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff6a0066f30 msgr2=0x7ff6a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:07.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.106+0000 7ff6c492e700 1 --2- 172.21.15.99:0/1262118271 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff6a0066f30 0x7ff6a00693e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0006fd0 tx=0x7ff6b0008040 comp rx=0 tx=0).stop 2023-10-08T00:13:07.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.106+0000 7ff6c492e700 1 -- 172.21.15.99:0/1262118271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6c015cbd0 msgr2=0x7ff6c00c8a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:07.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.106+0000 7ff6c492e700 1 --2- 172.21.15.99:0/1262118271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6c015cbd0 0x7ff6c00c8a40 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ff6c00c7b10 tx=0x7ff6b8017000 comp rx=0 tx=0).stop 2023-10-08T00:13:07.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.106+0000 7ff6c492e700 1 -- 172.21.15.99:0/1262118271 shutdown_connections 2023-10-08T00:13:07.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.106+0000 7ff6c492e700 1 --2- 172.21.15.99:0/1262118271 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff6a0066f30 0x7ff6a00693e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:07.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.106+0000 7ff6c492e700 1 --2- 172.21.15.99:0/1262118271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6c015cbd0 0x7ff6c00c8a40 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:07.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.106+0000 7ff6c492e700 1 --2- 172.21.15.99:0/1262118271 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6c00cd6e0 0x7ff6c0155530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:07.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.106+0000 7ff6c492e700 1 --2- 172.21.15.99:0/1262118271 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6c00cdab0 0x7ff6c00b6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:07.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.106+0000 7ff6c492e700 1 -- 172.21.15.99:0/1262118271 >> 172.21.15.99:0/1262118271 conn(0x7ff6c0067660 msgr2=0x7ff6c00c5e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:13:07.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.106+0000 7ff6c492e700 1 -- 172.21.15.99:0/1262118271 shutdown_connections 2023-10-08T00:13:07.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.106+0000 7ff6c492e700 1 -- 172.21.15.99:0/1262118271 wait complete. 2023-10-08T00:13:07.110 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2023-10-08T00:13:07.118 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":131,"stamp":"2023-10-08T00:13:06.054740+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":3949,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1558550,"statfs":{"total":1600000000,"available":1595955996,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002225"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.217533+0000","last_change":"2023-10-08T00:12:58.217367+0000","last_active":"2023-10-08T00:12:58.217533+0000","last_peered":"2023-10-08T00:12:58.217533+0000","last_clean":"2023-10-08T00:12:58.217533+0000","last_became_active":"2023-10-08T00:12:58.216807+0000","last_became_peered":"2023-10-08T00:12:58.216807+0000","last_unstale":"2023-10-08T00:12:58.217533+0000","last_undegraded":"2023-10-08T00:12:58.217533+0000","last_fullsized":"2023-10-08T00:12:58.217533+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:37:04.823729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.222932+0000","last_change":"2023-10-08T00:12:58.222859+0000","last_active":"2023-10-08T00:12:58.222932+0000","last_peered":"2023-10-08T00:12:58.222932+0000","last_clean":"2023-10-08T00:12:58.222932+0000","last_became_active":"2023-10-08T00:12:58.222618+0000","last_became_peered":"2023-10-08T00:12:58.222618+0000","last_unstale":"2023-10-08T00:12:58.222932+0000","last_undegraded":"2023-10-08T00:12:58.222932+0000","last_fullsized":"2023-10-08T00:12:58.222932+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:51:47.982127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.218117+0000","last_change":"2023-10-08T00:12:58.218059+0000","last_active":"2023-10-08T00:12:58.218117+0000","last_peered":"2023-10-08T00:12:58.218117+0000","last_clean":"2023-10-08T00:12:58.218117+0000","last_became_active":"2023-10-08T00:12:58.217832+0000","last_became_peered":"2023-10-08T00:12:58.217832+0000","last_unstale":"2023-10-08T00:12:58.218117+0000","last_undegraded":"2023-10-08T00:12:58.218117+0000","last_fullsized":"2023-10-08T00:12:58.218117+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:52:28.504130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.216693+0000","last_change":"2023-10-08T00:12:58.216331+0000","last_active":"2023-10-08T00:12:58.216693+0000","last_peered":"2023-10-08T00:12:58.216693+0000","last_clean":"2023-10-08T00:12:58.216693+0000","last_became_active":"2023-10-08T00:12:58.216072+0000","last_became_peered":"2023-10-08T00:12:58.216072+0000","last_unstale":"2023-10-08T00:12:58.216693+0000","last_undegraded":"2023-10-08T00:12:58.216693+0000","last_fullsized":"2023-10-08T00:12:58.216693+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:57:31.262253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.220468+0000","last_change":"2023-10-08T00:12:58.218686+0000","last_active":"2023-10-08T00:12:58.220468+0000","last_peered":"2023-10-08T00:12:58.220468+0000","last_clean":"2023-10-08T00:12:58.220468+0000","last_became_active":"2023-10-08T00:12:58.217517+0000","last_became_peered":"2023-10-08T00:12:58.217517+0000","last_unstale":"2023-10-08T00:12:58.220468+0000","last_undegraded":"2023-10-08T00:12:58.220468+0000","last_fullsized":"2023-10-08T00:12:58.220468+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:39:49.309778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.217514+0000","last_change":"2023-10-08T00:12:58.217222+0000","last_active":"2023-10-08T00:12:58.217514+0000","last_peered":"2023-10-08T00:12:58.217514+0000","last_clean":"2023-10-08T00:12:58.217514+0000","last_became_active":"2023-10-08T00:12:58.216507+0000","last_became_peered":"2023-10-08T00:12:58.216507+0000","last_unstale":"2023-10-08T00:12:58.217514+0000","last_undegraded":"2023-10-08T00:12:58.217514+0000","last_fullsized":"2023-10-08T00:12:58.217514+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:33:06.564584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.207661+0000","last_change":"2023-10-08T00:12:58.207605+0000","last_active":"2023-10-08T00:12:58.207661+0000","last_peered":"2023-10-08T00:12:58.207661+0000","last_clean":"2023-10-08T00:12:58.207661+0000","last_became_active":"2023-10-08T00:12:58.207259+0000","last_became_peered":"2023-10-08T00:12:58.207259+0000","last_unstale":"2023-10-08T00:12:58.207661+0000","last_undegraded":"2023-10-08T00:12:58.207661+0000","last_fullsized":"2023-10-08T00:12:58.207661+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:04:38.799978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.220775+0000","last_change":"2023-10-08T00:12:58.219278+0000","last_active":"2023-10-08T00:12:58.220775+0000","last_peered":"2023-10-08T00:12:58.220775+0000","last_clean":"2023-10-08T00:12:58.220775+0000","last_became_active":"2023-10-08T00:12:58.219031+0000","last_became_peered":"2023-10-08T00:12:58.219031+0000","last_unstale":"2023-10-08T00:12:58.220775+0000","last_undegraded":"2023-10-08T00:12:58.220775+0000","last_fullsized":"2023-10-08T00:12:58.220775+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:04:37.043481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.224283+0000","last_change":"2023-10-08T00:12:58.224212+0000","last_active":"2023-10-08T00:12:58.224283+0000","last_peered":"2023-10-08T00:12:58.224283+0000","last_clean":"2023-10-08T00:12:58.224283+0000","last_became_active":"2023-10-08T00:12:58.223771+0000","last_became_peered":"2023-10-08T00:12:58.223771+0000","last_unstale":"2023-10-08T00:12:58.224283+0000","last_undegraded":"2023-10-08T00:12:58.224283+0000","last_fullsized":"2023-10-08T00:12:58.224283+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:35:32.137447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.220519+0000","last_change":"2023-10-08T00:12:58.218836+0000","last_active":"2023-10-08T00:12:58.220519+0000","last_peered":"2023-10-08T00:12:58.220519+0000","last_clean":"2023-10-08T00:12:58.220519+0000","last_became_active":"2023-10-08T00:12:58.218315+0000","last_became_peered":"2023-10-08T00:12:58.218315+0000","last_unstale":"2023-10-08T00:12:58.220519+0000","last_undegraded":"2023-10-08T00:12:58.220519+0000","last_fullsized":"2023-10-08T00:12:58.220519+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:24:53.646854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.218852+0000","last_change":"2023-10-08T00:12:58.218488+0000","last_active":"2023-10-08T00:12:58.218852+0000","last_peered":"2023-10-08T00:12:58.218852+0000","last_clean":"2023-10-08T00:12:58.218852+0000","last_became_active":"2023-10-08T00:12:58.218123+0000","last_became_peered":"2023-10-08T00:12:58.218123+0000","last_unstale":"2023-10-08T00:12:58.218852+0000","last_undegraded":"2023-10-08T00:12:58.218852+0000","last_fullsized":"2023-10-08T00:12:58.218852+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:52:21.657523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.219103+0000","last_change":"2023-10-08T00:12:58.219045+0000","last_active":"2023-10-08T00:12:58.219103+0000","last_peered":"2023-10-08T00:12:58.219103+0000","last_clean":"2023-10-08T00:12:58.219103+0000","last_became_active":"2023-10-08T00:12:58.218812+0000","last_became_peered":"2023-10-08T00:12:58.218812+0000","last_unstale":"2023-10-08T00:12:58.219103+0000","last_undegraded":"2023-10-08T00:12:58.219103+0000","last_fullsized":"2023-10-08T00:12:58.219103+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:56:09.457506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.216441+0000","last_change":"2023-10-08T00:12:58.216384+0000","last_active":"2023-10-08T00:12:58.216441+0000","last_peered":"2023-10-08T00:12:58.216441+0000","last_clean":"2023-10-08T00:12:58.216441+0000","last_became_active":"2023-10-08T00:12:58.216178+0000","last_became_peered":"2023-10-08T00:12:58.216178+0000","last_unstale":"2023-10-08T00:12:58.216441+0000","last_undegraded":"2023-10-08T00:12:58.216441+0000","last_fullsized":"2023-10-08T00:12:58.216441+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:33:06.564584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.209518+0000","last_change":"2023-10-08T00:12:58.209465+0000","last_active":"2023-10-08T00:12:58.209518+0000","last_peered":"2023-10-08T00:12:58.209518+0000","last_clean":"2023-10-08T00:12:58.209518+0000","last_became_active":"2023-10-08T00:12:58.208998+0000","last_became_peered":"2023-10-08T00:12:58.208998+0000","last_unstale":"2023-10-08T00:12:58.209518+0000","last_undegraded":"2023-10-08T00:12:58.209518+0000","last_fullsized":"2023-10-08T00:12:58.209518+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:23:42.956575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.217559+0000","last_change":"2023-10-08T00:12:58.217503+0000","last_active":"2023-10-08T00:12:58.217559+0000","last_peered":"2023-10-08T00:12:58.217559+0000","last_clean":"2023-10-08T00:12:58.217559+0000","last_became_active":"2023-10-08T00:12:58.217285+0000","last_became_peered":"2023-10-08T00:12:58.217285+0000","last_unstale":"2023-10-08T00:12:58.217559+0000","last_undegraded":"2023-10-08T00:12:58.217559+0000","last_fullsized":"2023-10-08T00:12:58.217559+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:28:53.462666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.217493+0000","last_change":"2023-10-08T00:12:58.217119+0000","last_active":"2023-10-08T00:12:58.217493+0000","last_peered":"2023-10-08T00:12:58.217493+0000","last_clean":"2023-10-08T00:12:58.217493+0000","last_became_active":"2023-10-08T00:12:58.216194+0000","last_became_peered":"2023-10-08T00:12:58.216194+0000","last_unstale":"2023-10-08T00:12:58.217493+0000","last_undegraded":"2023-10-08T00:12:58.217493+0000","last_fullsized":"2023-10-08T00:12:58.217493+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:28:53.462666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.215551+0000","last_change":"2023-10-08T00:12:58.215405+0000","last_active":"2023-10-08T00:12:58.215551+0000","last_peered":"2023-10-08T00:12:58.215551+0000","last_clean":"2023-10-08T00:12:58.215551+0000","last_became_active":"2023-10-08T00:12:58.214828+0000","last_became_peered":"2023-10-08T00:12:58.214828+0000","last_unstale":"2023-10-08T00:12:58.215551+0000","last_undegraded":"2023-10-08T00:12:58.215551+0000","last_fullsized":"2023-10-08T00:12:58.215551+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:52:21.657523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.214181+0000","last_change":"2023-10-08T00:12:58.214149+0000","last_active":"2023-10-08T00:12:58.214181+0000","last_peered":"2023-10-08T00:12:58.214181+0000","last_clean":"2023-10-08T00:12:58.214181+0000","last_became_active":"2023-10-08T00:12:58.213855+0000","last_became_peered":"2023-10-08T00:12:58.213855+0000","last_unstale":"2023-10-08T00:12:58.214181+0000","last_undegraded":"2023-10-08T00:12:58.214181+0000","last_fullsized":"2023-10-08T00:12:58.214181+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:17:37.423308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.216792+0000","last_change":"2023-10-08T00:12:58.216645+0000","last_active":"2023-10-08T00:12:58.216792+0000","last_peered":"2023-10-08T00:12:58.216792+0000","last_clean":"2023-10-08T00:12:58.216792+0000","last_became_active":"2023-10-08T00:12:58.216417+0000","last_became_peered":"2023-10-08T00:12:58.216417+0000","last_unstale":"2023-10-08T00:12:58.216792+0000","last_undegraded":"2023-10-08T00:12:58.216792+0000","last_fullsized":"2023-10-08T00:12:58.216792+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:00:49.167372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.215453+0000","last_change":"2023-10-08T00:12:58.215354+0000","last_active":"2023-10-08T00:12:58.215453+0000","last_peered":"2023-10-08T00:12:58.215453+0000","last_clean":"2023-10-08T00:12:58.215453+0000","last_became_active":"2023-10-08T00:12:58.215125+0000","last_became_peered":"2023-10-08T00:12:58.215125+0000","last_unstale":"2023-10-08T00:12:58.215453+0000","last_undegraded":"2023-10-08T00:12:58.215453+0000","last_fullsized":"2023-10-08T00:12:58.215453+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:11:41.089416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.215110+0000","last_change":"2023-10-08T00:12:58.215051+0000","last_active":"2023-10-08T00:12:58.215110+0000","last_peered":"2023-10-08T00:12:58.215110+0000","last_clean":"2023-10-08T00:12:58.215110+0000","last_became_active":"2023-10-08T00:12:58.214838+0000","last_became_peered":"2023-10-08T00:12:58.214838+0000","last_unstale":"2023-10-08T00:12:58.215110+0000","last_undegraded":"2023-10-08T00:12:58.215110+0000","last_fullsized":"2023-10-08T00:12:58.215110+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:59:44.045258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.220971+0000","last_change":"2023-10-08T00:12:58.220057+0000","last_active":"2023-10-08T00:12:58.220971+0000","last_peered":"2023-10-08T00:12:58.220971+0000","last_clean":"2023-10-08T00:12:58.220971+0000","last_became_active":"2023-10-08T00:12:58.219542+0000","last_became_peered":"2023-10-08T00:12:58.219542+0000","last_unstale":"2023-10-08T00:12:58.220971+0000","last_undegraded":"2023-10-08T00:12:58.220971+0000","last_fullsized":"2023-10-08T00:12:58.220971+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:59:44.045258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.215180+0000","last_change":"2023-10-08T00:12:58.215049+0000","last_active":"2023-10-08T00:12:58.215180+0000","last_peered":"2023-10-08T00:12:58.215180+0000","last_clean":"2023-10-08T00:12:58.215180+0000","last_became_active":"2023-10-08T00:12:58.214779+0000","last_became_peered":"2023-10-08T00:12:58.214779+0000","last_unstale":"2023-10-08T00:12:58.215180+0000","last_undegraded":"2023-10-08T00:12:58.215180+0000","last_fullsized":"2023-10-08T00:12:58.215180+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:47:22.752537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.224121+0000","last_change":"2023-10-08T00:12:58.224076+0000","last_active":"2023-10-08T00:12:58.224121+0000","last_peered":"2023-10-08T00:12:58.224121+0000","last_clean":"2023-10-08T00:12:58.224121+0000","last_became_active":"2023-10-08T00:12:58.223498+0000","last_became_peered":"2023-10-08T00:12:58.223498+0000","last_unstale":"2023-10-08T00:12:58.224121+0000","last_undegraded":"2023-10-08T00:12:58.224121+0000","last_fullsized":"2023-10-08T00:12:58.224121+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:51:19.670216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.223413+0000","last_change":"2023-10-08T00:12:58.223156+0000","last_active":"2023-10-08T00:12:58.223413+0000","last_peered":"2023-10-08T00:12:58.223413+0000","last_clean":"2023-10-08T00:12:58.223413+0000","last_became_active":"2023-10-08T00:12:58.222753+0000","last_became_peered":"2023-10-08T00:12:58.222753+0000","last_unstale":"2023-10-08T00:12:58.223413+0000","last_undegraded":"2023-10-08T00:12:58.223413+0000","last_fullsized":"2023-10-08T00:12:58.223413+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:22:04.500698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.215747+0000","last_change":"2023-10-08T00:12:58.215657+0000","last_active":"2023-10-08T00:12:58.215747+0000","last_peered":"2023-10-08T00:12:58.215747+0000","last_clean":"2023-10-08T00:12:58.215747+0000","last_became_active":"2023-10-08T00:12:58.215425+0000","last_became_peered":"2023-10-08T00:12:58.215425+0000","last_unstale":"2023-10-08T00:12:58.215747+0000","last_undegraded":"2023-10-08T00:12:58.215747+0000","last_fullsized":"2023-10-08T00:12:58.215747+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:52:21.657523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.220937+0000","last_change":"2023-10-08T00:12:58.220644+0000","last_active":"2023-10-08T00:12:58.220937+0000","last_peered":"2023-10-08T00:12:58.220937+0000","last_clean":"2023-10-08T00:12:58.220937+0000","last_became_active":"2023-10-08T00:12:58.220210+0000","last_became_peered":"2023-10-08T00:12:58.220210+0000","last_unstale":"2023-10-08T00:12:58.220937+0000","last_undegraded":"2023-10-08T00:12:58.220937+0000","last_fullsized":"2023-10-08T00:12:58.220937+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:13:44.142695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.218880+0000","last_change":"2023-10-08T00:12:58.218794+0000","last_active":"2023-10-08T00:12:58.218880+0000","last_peered":"2023-10-08T00:12:58.218880+0000","last_clean":"2023-10-08T00:12:58.218880+0000","last_became_active":"2023-10-08T00:12:58.218422+0000","last_became_peered":"2023-10-08T00:12:58.218422+0000","last_unstale":"2023-10-08T00:12:58.218880+0000","last_undegraded":"2023-10-08T00:12:58.218880+0000","last_fullsized":"2023-10-08T00:12:58.218880+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:17:47.918131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.221231+0000","last_change":"2023-10-08T00:12:58.221011+0000","last_active":"2023-10-08T00:12:58.221231+0000","last_peered":"2023-10-08T00:12:58.221231+0000","last_clean":"2023-10-08T00:12:58.221231+0000","last_became_active":"2023-10-08T00:12:58.220537+0000","last_became_peered":"2023-10-08T00:12:58.220537+0000","last_unstale":"2023-10-08T00:12:58.221231+0000","last_undegraded":"2023-10-08T00:12:58.221231+0000","last_fullsized":"2023-10-08T00:12:58.221231+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:00:12.638297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.215735+0000","last_change":"2023-10-08T00:12:58.215670+0000","last_active":"2023-10-08T00:12:58.215735+0000","last_peered":"2023-10-08T00:12:58.215735+0000","last_clean":"2023-10-08T00:12:58.215735+0000","last_became_active":"2023-10-08T00:12:58.215445+0000","last_became_peered":"2023-10-08T00:12:58.215445+0000","last_unstale":"2023-10-08T00:12:58.215735+0000","last_undegraded":"2023-10-08T00:12:58.215735+0000","last_fullsized":"2023-10-08T00:12:58.215735+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:20:45.018183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.218721+0000","last_change":"2023-10-08T00:12:58.218652+0000","last_active":"2023-10-08T00:12:58.218721+0000","last_peered":"2023-10-08T00:12:58.218721+0000","last_clean":"2023-10-08T00:12:58.218721+0000","last_became_active":"2023-10-08T00:12:58.218371+0000","last_became_peered":"2023-10-08T00:12:58.218371+0000","last_unstale":"2023-10-08T00:12:58.218721+0000","last_undegraded":"2023-10-08T00:12:58.218721+0000","last_fullsized":"2023-10-08T00:12:58.218721+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:33:16.516139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.219987+0000","last_change":"2023-10-08T00:12:58.219768+0000","last_active":"2023-10-08T00:12:58.219987+0000","last_peered":"2023-10-08T00:12:58.219987+0000","last_clean":"2023-10-08T00:12:58.219987+0000","last_became_active":"2023-10-08T00:12:58.218935+0000","last_became_peered":"2023-10-08T00:12:58.218935+0000","last_unstale":"2023-10-08T00:12:58.219987+0000","last_undegraded":"2023-10-08T00:12:58.219987+0000","last_fullsized":"2023-10-08T00:12:58.219987+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:28:53.462666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.222967+0000","last_change":"2023-10-08T00:12:58.222810+0000","last_active":"2023-10-08T00:12:58.222967+0000","last_peered":"2023-10-08T00:12:58.222967+0000","last_clean":"2023-10-08T00:12:58.222967+0000","last_became_active":"2023-10-08T00:12:58.222397+0000","last_became_peered":"2023-10-08T00:12:58.222397+0000","last_unstale":"2023-10-08T00:12:58.222967+0000","last_undegraded":"2023-10-08T00:12:58.222967+0000","last_fullsized":"2023-10-08T00:12:58.222967+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:59:44.045258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"68'4","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.231282+0000","last_change":"2023-10-08T00:12:57.213303+0000","last_active":"2023-10-08T00:12:58.231282+0000","last_peered":"2023-10-08T00:12:58.231282+0000","last_clean":"2023-10-08T00:12:58.231282+0000","last_became_active":"2023-10-08T00:12:57.213092+0000","last_became_peered":"2023-10-08T00:12:57.213092+0000","last_unstale":"2023-10-08T00:12:58.231282+0000","last_undegraded":"2023-10-08T00:12:58.231282+0000","last_fullsized":"2023-10-08T00:12:58.231282+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:40:33.741781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.216620+0000","last_change":"2023-10-08T00:12:58.216453+0000","last_active":"2023-10-08T00:12:58.216620+0000","last_peered":"2023-10-08T00:12:58.216620+0000","last_clean":"2023-10-08T00:12:58.216620+0000","last_became_active":"2023-10-08T00:12:58.216245+0000","last_became_peered":"2023-10-08T00:12:58.216245+0000","last_unstale":"2023-10-08T00:12:58.216620+0000","last_undegraded":"2023-10-08T00:12:58.216620+0000","last_fullsized":"2023-10-08T00:12:58.216620+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:15:27.299505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"68'1","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.229405+0000","last_change":"2023-10-08T00:12:57.212771+0000","last_active":"2023-10-08T00:12:58.229405+0000","last_peered":"2023-10-08T00:12:58.229405+0000","last_clean":"2023-10-08T00:12:58.229405+0000","last_became_active":"2023-10-08T00:12:57.212479+0000","last_became_peered":"2023-10-08T00:12:57.212479+0000","last_unstale":"2023-10-08T00:12:58.229405+0000","last_undegraded":"2023-10-08T00:12:58.229405+0000","last_fullsized":"2023-10-08T00:12:58.229405+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:51:18.660991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.220795+0000","last_change":"2023-10-08T00:12:58.220222+0000","last_active":"2023-10-08T00:12:58.220795+0000","last_peered":"2023-10-08T00:12:58.220795+0000","last_clean":"2023-10-08T00:12:58.220795+0000","last_became_active":"2023-10-08T00:12:58.219839+0000","last_became_peered":"2023-10-08T00:12:58.219839+0000","last_unstale":"2023-10-08T00:12:58.220795+0000","last_undegraded":"2023-10-08T00:12:58.220795+0000","last_fullsized":"2023-10-08T00:12:58.220795+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:09:28.457491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"68'2","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.227279+0000","last_change":"2023-10-08T00:12:57.209653+0000","last_active":"2023-10-08T00:12:58.227279+0000","last_peered":"2023-10-08T00:12:58.227279+0000","last_clean":"2023-10-08T00:12:58.227279+0000","last_became_active":"2023-10-08T00:12:57.209408+0000","last_became_peered":"2023-10-08T00:12:57.209408+0000","last_unstale":"2023-10-08T00:12:58.227279+0000","last_undegraded":"2023-10-08T00:12:58.227279+0000","last_fullsized":"2023-10-08T00:12:58.227279+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:26.648145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.222274+0000","last_change":"2023-10-08T00:12:58.220036+0000","last_active":"2023-10-08T00:12:58.222274+0000","last_peered":"2023-10-08T00:12:58.222274+0000","last_clean":"2023-10-08T00:12:58.222274+0000","last_became_active":"2023-10-08T00:12:58.219765+0000","last_became_peered":"2023-10-08T00:12:58.219765+0000","last_unstale":"2023-10-08T00:12:58.222274+0000","last_undegraded":"2023-10-08T00:12:58.222274+0000","last_fullsized":"2023-10-08T00:12:58.222274+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:56:37.046732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"68'2","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.230038+0000","last_change":"2023-10-08T00:12:57.207927+0000","last_active":"2023-10-08T00:12:58.230038+0000","last_peered":"2023-10-08T00:12:58.230038+0000","last_clean":"2023-10-08T00:12:58.230038+0000","last_became_active":"2023-10-08T00:12:57.207442+0000","last_became_peered":"2023-10-08T00:12:57.207442+0000","last_unstale":"2023-10-08T00:12:58.230038+0000","last_undegraded":"2023-10-08T00:12:58.230038+0000","last_fullsized":"2023-10-08T00:12:58.230038+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:57:10.908242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.218731+0000","last_change":"2023-10-08T00:12:58.218647+0000","last_active":"2023-10-08T00:12:58.218731+0000","last_peered":"2023-10-08T00:12:58.218731+0000","last_clean":"2023-10-08T00:12:58.218731+0000","last_became_active":"2023-10-08T00:12:58.218129+0000","last_became_peered":"2023-10-08T00:12:58.218129+0000","last_unstale":"2023-10-08T00:12:58.218731+0000","last_undegraded":"2023-10-08T00:12:58.218731+0000","last_fullsized":"2023-10-08T00:12:58.218731+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:48:10.822874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"68'1","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.228575+0000","last_change":"2023-10-08T00:12:57.212292+0000","last_active":"2023-10-08T00:12:58.228575+0000","last_peered":"2023-10-08T00:12:58.228575+0000","last_clean":"2023-10-08T00:12:58.228575+0000","last_became_active":"2023-10-08T00:12:57.211867+0000","last_became_peered":"2023-10-08T00:12:57.211867+0000","last_unstale":"2023-10-08T00:12:58.228575+0000","last_undegraded":"2023-10-08T00:12:58.228575+0000","last_fullsized":"2023-10-08T00:12:58.228575+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:36:52.303382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.220011+0000","last_change":"2023-10-08T00:12:58.219921+0000","last_active":"2023-10-08T00:12:58.220011+0000","last_peered":"2023-10-08T00:12:58.220011+0000","last_clean":"2023-10-08T00:12:58.220011+0000","last_became_active":"2023-10-08T00:12:58.219230+0000","last_became_peered":"2023-10-08T00:12:58.219230+0000","last_unstale":"2023-10-08T00:12:58.220011+0000","last_undegraded":"2023-10-08T00:12:58.220011+0000","last_fullsized":"2023-10-08T00:12:58.220011+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:36:53.312606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"68'1","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.228593+0000","last_change":"2023-10-08T00:12:57.211573+0000","last_active":"2023-10-08T00:12:58.228593+0000","last_peered":"2023-10-08T00:12:58.228593+0000","last_clean":"2023-10-08T00:12:58.228593+0000","last_became_active":"2023-10-08T00:12:57.211202+0000","last_became_peered":"2023-10-08T00:12:57.211202+0000","last_unstale":"2023-10-08T00:12:58.228593+0000","last_undegraded":"2023-10-08T00:12:58.228593+0000","last_fullsized":"2023-10-08T00:12:58.228593+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:29:56.138577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.207418+0000","last_change":"2023-10-08T00:12:58.207371+0000","last_active":"2023-10-08T00:12:58.207418+0000","last_peered":"2023-10-08T00:12:58.207418+0000","last_clean":"2023-10-08T00:12:58.207418+0000","last_became_active":"2023-10-08T00:12:58.207156+0000","last_became_peered":"2023-10-08T00:12:58.207156+0000","last_unstale":"2023-10-08T00:12:58.207418+0000","last_undegraded":"2023-10-08T00:12:58.207418+0000","last_fullsized":"2023-10-08T00:12:58.207418+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:10:15.405726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.217703+0000","last_change":"2023-10-08T00:12:57.205376+0000","last_active":"2023-10-08T00:12:58.217703+0000","last_peered":"2023-10-08T00:12:58.217703+0000","last_clean":"2023-10-08T00:12:58.217703+0000","last_became_active":"2023-10-08T00:12:57.205158+0000","last_became_peered":"2023-10-08T00:12:57.205158+0000","last_unstale":"2023-10-08T00:12:58.217703+0000","last_undegraded":"2023-10-08T00:12:58.217703+0000","last_fullsized":"2023-10-08T00:12:58.217703+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:15:43.196718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.215005+0000","last_change":"2023-10-08T00:12:58.214281+0000","last_active":"2023-10-08T00:12:58.215005+0000","last_peered":"2023-10-08T00:12:58.215005+0000","last_clean":"2023-10-08T00:12:58.215005+0000","last_became_active":"2023-10-08T00:12:58.214013+0000","last_became_peered":"2023-10-08T00:12:58.214013+0000","last_unstale":"2023-10-08T00:12:58.215005+0000","last_undegraded":"2023-10-08T00:12:58.215005+0000","last_fullsized":"2023-10-08T00:12:58.215005+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:36:53.312606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.214503+0000","last_change":"2023-10-08T00:12:57.214270+0000","last_active":"2023-10-08T00:12:58.214503+0000","last_peered":"2023-10-08T00:12:58.214503+0000","last_clean":"2023-10-08T00:12:58.214503+0000","last_became_active":"2023-10-08T00:12:57.214082+0000","last_became_peered":"2023-10-08T00:12:57.214082+0000","last_unstale":"2023-10-08T00:12:58.214503+0000","last_undegraded":"2023-10-08T00:12:58.214503+0000","last_fullsized":"2023-10-08T00:12:58.214503+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:40:33.741781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.215730+0000","last_change":"2023-10-08T00:12:58.215648+0000","last_active":"2023-10-08T00:12:58.215730+0000","last_peered":"2023-10-08T00:12:58.215730+0000","last_clean":"2023-10-08T00:12:58.215730+0000","last_became_active":"2023-10-08T00:12:58.215388+0000","last_became_peered":"2023-10-08T00:12:58.215388+0000","last_unstale":"2023-10-08T00:12:58.215730+0000","last_undegraded":"2023-10-08T00:12:58.215730+0000","last_fullsized":"2023-10-08T00:12:58.215730+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:11:41.089416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.217320+0000","last_change":"2023-10-08T00:12:57.209199+0000","last_active":"2023-10-08T00:12:58.217320+0000","last_peered":"2023-10-08T00:12:58.217320+0000","last_clean":"2023-10-08T00:12:58.217320+0000","last_became_active":"2023-10-08T00:12:57.208965+0000","last_became_peered":"2023-10-08T00:12:57.208965+0000","last_unstale":"2023-10-08T00:12:58.217320+0000","last_undegraded":"2023-10-08T00:12:58.217320+0000","last_fullsized":"2023-10-08T00:12:58.217320+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:02:39.029498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.209044+0000","last_change":"2023-10-08T00:12:58.208991+0000","last_active":"2023-10-08T00:12:58.209044+0000","last_peered":"2023-10-08T00:12:58.209044+0000","last_clean":"2023-10-08T00:12:58.209044+0000","last_became_active":"2023-10-08T00:12:58.208766+0000","last_became_peered":"2023-10-08T00:12:58.208766+0000","last_unstale":"2023-10-08T00:12:58.209044+0000","last_undegraded":"2023-10-08T00:12:58.209044+0000","last_fullsized":"2023-10-08T00:12:58.209044+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:00:12.638297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.207486+0000","last_change":"2023-10-08T00:12:57.214795+0000","last_active":"2023-10-08T00:12:58.207486+0000","last_peered":"2023-10-08T00:12:58.207486+0000","last_clean":"2023-10-08T00:12:58.207486+0000","last_became_active":"2023-10-08T00:12:57.214269+0000","last_became_peered":"2023-10-08T00:12:57.214269+0000","last_unstale":"2023-10-08T00:12:58.207486+0000","last_undegraded":"2023-10-08T00:12:58.207486+0000","last_fullsized":"2023-10-08T00:12:58.207486+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:28:52.453441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.217382+0000","last_change":"2023-10-08T00:12:58.216902+0000","last_active":"2023-10-08T00:12:58.217382+0000","last_peered":"2023-10-08T00:12:58.217382+0000","last_clean":"2023-10-08T00:12:58.217382+0000","last_became_active":"2023-10-08T00:12:58.216532+0000","last_became_peered":"2023-10-08T00:12:58.216532+0000","last_unstale":"2023-10-08T00:12:58.217382+0000","last_undegraded":"2023-10-08T00:12:58.217382+0000","last_fullsized":"2023-10-08T00:12:58.217382+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:49:06.193413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"68'3","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.230748+0000","last_change":"2023-10-08T00:12:57.210087+0000","last_active":"2023-10-08T00:12:58.230748+0000","last_peered":"2023-10-08T00:12:58.230748+0000","last_clean":"2023-10-08T00:12:58.230748+0000","last_became_active":"2023-10-08T00:12:57.209574+0000","last_became_peered":"2023-10-08T00:12:57.209574+0000","last_unstale":"2023-10-08T00:12:58.230748+0000","last_undegraded":"2023-10-08T00:12:58.230748+0000","last_fullsized":"2023-10-08T00:12:58.230748+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:51:19.503936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.212489+0000","last_change":"2023-10-08T00:12:58.212450+0000","last_active":"2023-10-08T00:12:58.212489+0000","last_peered":"2023-10-08T00:12:58.212489+0000","last_clean":"2023-10-08T00:12:58.212489+0000","last_became_active":"2023-10-08T00:12:58.212288+0000","last_became_peered":"2023-10-08T00:12:58.212288+0000","last_unstale":"2023-10-08T00:12:58.212489+0000","last_undegraded":"2023-10-08T00:12:58.212489+0000","last_fullsized":"2023-10-08T00:12:58.212489+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:16:35.252392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"68'6","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.230038+0000","last_change":"2023-10-08T00:12:57.209678+0000","last_active":"2023-10-08T00:12:58.230038+0000","last_peered":"2023-10-08T00:12:58.230038+0000","last_clean":"2023-10-08T00:12:58.230038+0000","last_became_active":"2023-10-08T00:12:57.209454+0000","last_became_peered":"2023-10-08T00:12:57.209454+0000","last_unstale":"2023-10-08T00:12:58.230038+0000","last_undegraded":"2023-10-08T00:12:58.230038+0000","last_fullsized":"2023-10-08T00:12:58.230038+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:51:46.972903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.221326+0000","last_change":"2023-10-08T00:12:58.221272+0000","last_active":"2023-10-08T00:12:58.221326+0000","last_peered":"2023-10-08T00:12:58.221326+0000","last_clean":"2023-10-08T00:12:58.221326+0000","last_became_active":"2023-10-08T00:12:58.221038+0000","last_became_peered":"2023-10-08T00:12:58.221038+0000","last_unstale":"2023-10-08T00:12:58.221326+0000","last_undegraded":"2023-10-08T00:12:58.221326+0000","last_fullsized":"2023-10-08T00:12:58.221326+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:25:36.388792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"68'2","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.230064+0000","last_change":"2023-10-08T00:12:57.215576+0000","last_active":"2023-10-08T00:12:58.230064+0000","last_peered":"2023-10-08T00:12:58.230064+0000","last_clean":"2023-10-08T00:12:58.230064+0000","last_became_active":"2023-10-08T00:12:57.215200+0000","last_became_peered":"2023-10-08T00:12:57.215200+0000","last_unstale":"2023-10-08T00:12:58.230064+0000","last_undegraded":"2023-10-08T00:12:58.230064+0000","last_fullsized":"2023-10-08T00:12:58.230064+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:11:40.080191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.218318+0000","last_change":"2023-10-08T00:12:58.218258+0000","last_active":"2023-10-08T00:12:58.218318+0000","last_peered":"2023-10-08T00:12:58.218318+0000","last_clean":"2023-10-08T00:12:58.218318+0000","last_became_active":"2023-10-08T00:12:58.218046+0000","last_became_peered":"2023-10-08T00:12:58.218046+0000","last_unstale":"2023-10-08T00:12:58.218318+0000","last_undegraded":"2023-10-08T00:12:58.218318+0000","last_fullsized":"2023-10-08T00:12:58.218318+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:09:28.457491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.206530+0000","last_change":"2023-10-08T00:12:57.208075+0000","last_active":"2023-10-08T00:12:58.206530+0000","last_peered":"2023-10-08T00:12:58.206530+0000","last_clean":"2023-10-08T00:12:58.206530+0000","last_became_active":"2023-10-08T00:12:57.207720+0000","last_became_peered":"2023-10-08T00:12:57.207720+0000","last_unstale":"2023-10-08T00:12:58.206530+0000","last_undegraded":"2023-10-08T00:12:58.206530+0000","last_fullsized":"2023-10-08T00:12:58.206530+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:23:41.947350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.219120+0000","last_change":"2023-10-08T00:12:58.219063+0000","last_active":"2023-10-08T00:12:58.219120+0000","last_peered":"2023-10-08T00:12:58.219120+0000","last_clean":"2023-10-08T00:12:58.219120+0000","last_became_active":"2023-10-08T00:12:58.218843+0000","last_became_peered":"2023-10-08T00:12:58.218843+0000","last_unstale":"2023-10-08T00:12:58.219120+0000","last_undegraded":"2023-10-08T00:12:58.219120+0000","last_fullsized":"2023-10-08T00:12:58.219120+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:04:44.276710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.206823+0000","last_change":"2023-10-08T00:12:57.204277+0000","last_active":"2023-10-08T00:12:58.206823+0000","last_peered":"2023-10-08T00:12:58.206823+0000","last_clean":"2023-10-08T00:12:58.206823+0000","last_became_active":"2023-10-08T00:12:57.204039+0000","last_became_peered":"2023-10-08T00:12:57.204039+0000","last_unstale":"2023-10-08T00:12:58.206823+0000","last_undegraded":"2023-10-08T00:12:58.206823+0000","last_fullsized":"2023-10-08T00:12:58.206823+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:09:18.935331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.219157+0000","last_change":"2023-10-08T00:12:58.219095+0000","last_active":"2023-10-08T00:12:58.219157+0000","last_peered":"2023-10-08T00:12:58.219157+0000","last_clean":"2023-10-08T00:12:58.219157+0000","last_became_active":"2023-10-08T00:12:58.218867+0000","last_became_peered":"2023-10-08T00:12:58.218867+0000","last_unstale":"2023-10-08T00:12:58.219157+0000","last_undegraded":"2023-10-08T00:12:58.219157+0000","last_fullsized":"2023-10-08T00:12:58.219157+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:39:55.009480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"68'8","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.230615+0000","last_change":"2023-10-08T00:12:57.210213+0000","last_active":"2023-10-08T00:12:58.230615+0000","last_peered":"2023-10-08T00:12:58.230615+0000","last_clean":"2023-10-08T00:12:58.230615+0000","last_became_active":"2023-10-08T00:12:57.209897+0000","last_became_peered":"2023-10-08T00:12:57.209897+0000","last_unstale":"2023-10-08T00:12:58.230615+0000","last_undegraded":"2023-10-08T00:12:58.230615+0000","last_fullsized":"2023-10-08T00:12:58.230615+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:49:05.184189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.207925+0000","last_change":"2023-10-08T00:12:58.207885+0000","last_active":"2023-10-08T00:12:58.207925+0000","last_peered":"2023-10-08T00:12:58.207925+0000","last_clean":"2023-10-08T00:12:58.207925+0000","last_became_active":"2023-10-08T00:12:58.207715+0000","last_became_peered":"2023-10-08T00:12:58.207715+0000","last_unstale":"2023-10-08T00:12:58.207925+0000","last_undegraded":"2023-10-08T00:12:58.207925+0000","last_fullsized":"2023-10-08T00:12:58.207925+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:49:06.193413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"68'1","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.229027+0000","last_change":"2023-10-08T00:12:57.210506+0000","last_active":"2023-10-08T00:12:58.229027+0000","last_peered":"2023-10-08T00:12:58.229027+0000","last_clean":"2023-10-08T00:12:58.229027+0000","last_became_active":"2023-10-08T00:12:57.210180+0000","last_became_peered":"2023-10-08T00:12:57.210180+0000","last_unstale":"2023-10-08T00:12:58.229027+0000","last_undegraded":"2023-10-08T00:12:58.229027+0000","last_fullsized":"2023-10-08T00:12:58.229027+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:57:52.142531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.209605+0000","last_change":"2023-10-08T00:12:58.209560+0000","last_active":"2023-10-08T00:12:58.209605+0000","last_peered":"2023-10-08T00:12:58.209605+0000","last_clean":"2023-10-08T00:12:58.209605+0000","last_became_active":"2023-10-08T00:12:58.209252+0000","last_became_peered":"2023-10-08T00:12:58.209252+0000","last_unstale":"2023-10-08T00:12:58.209605+0000","last_undegraded":"2023-10-08T00:12:58.209605+0000","last_fullsized":"2023-10-08T00:12:58.209605+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:09:19.944555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"68'2","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.229511+0000","last_change":"2023-10-08T00:12:57.215419+0000","last_active":"2023-10-08T00:12:58.229511+0000","last_peered":"2023-10-08T00:12:58.229511+0000","last_clean":"2023-10-08T00:12:58.229511+0000","last_became_active":"2023-10-08T00:12:57.214895+0000","last_became_peered":"2023-10-08T00:12:57.214895+0000","last_unstale":"2023-10-08T00:12:58.229511+0000","last_undegraded":"2023-10-08T00:12:58.229511+0000","last_fullsized":"2023-10-08T00:12:58.229511+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:59:43.036033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.217819+0000","last_change":"2023-10-08T00:12:58.217685+0000","last_active":"2023-10-08T00:12:58.217819+0000","last_peered":"2023-10-08T00:12:58.217819+0000","last_clean":"2023-10-08T00:12:58.217819+0000","last_became_active":"2023-10-08T00:12:58.217414+0000","last_became_peered":"2023-10-08T00:12:58.217414+0000","last_unstale":"2023-10-08T00:12:58.217819+0000","last_undegraded":"2023-10-08T00:12:58.217819+0000","last_fullsized":"2023-10-08T00:12:58.217819+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:40:34.751005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"68'1","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.227173+0000","last_change":"2023-10-08T00:12:57.211758+0000","last_active":"2023-10-08T00:12:58.227173+0000","last_peered":"2023-10-08T00:12:58.227173+0000","last_clean":"2023-10-08T00:12:58.227173+0000","last_became_active":"2023-10-08T00:12:57.211537+0000","last_became_peered":"2023-10-08T00:12:57.211537+0000","last_unstale":"2023-10-08T00:12:58.227173+0000","last_undegraded":"2023-10-08T00:12:58.227173+0000","last_fullsized":"2023-10-08T00:12:58.227173+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:42:52.161510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.222875+0000","last_change":"2023-10-08T00:12:58.222815+0000","last_active":"2023-10-08T00:12:58.222875+0000","last_peered":"2023-10-08T00:12:58.222875+0000","last_clean":"2023-10-08T00:12:58.222875+0000","last_became_active":"2023-10-08T00:12:58.222528+0000","last_became_peered":"2023-10-08T00:12:58.222528+0000","last_unstale":"2023-10-08T00:12:58.222875+0000","last_undegraded":"2023-10-08T00:12:58.222875+0000","last_fullsized":"2023-10-08T00:12:58.222875+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:44:52.647472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.207629+0000","last_change":"2023-10-08T00:12:57.211562+0000","last_active":"2023-10-08T00:12:58.207629+0000","last_peered":"2023-10-08T00:12:58.207629+0000","last_clean":"2023-10-08T00:12:58.207629+0000","last_became_active":"2023-10-08T00:12:57.211290+0000","last_became_peered":"2023-10-08T00:12:57.211290+0000","last_unstale":"2023-10-08T00:12:58.207629+0000","last_undegraded":"2023-10-08T00:12:58.207629+0000","last_fullsized":"2023-10-08T00:12:58.207629+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:36:52.303382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.224044+0000","last_change":"2023-10-08T00:12:58.223986+0000","last_active":"2023-10-08T00:12:58.224044+0000","last_peered":"2023-10-08T00:12:58.224044+0000","last_clean":"2023-10-08T00:12:58.224044+0000","last_became_active":"2023-10-08T00:12:58.223265+0000","last_became_peered":"2023-10-08T00:12:58.223265+0000","last_unstale":"2023-10-08T00:12:58.224044+0000","last_undegraded":"2023-10-08T00:12:58.224044+0000","last_fullsized":"2023-10-08T00:12:58.224044+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:47:44.758644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"68'2","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.231188+0000","last_change":"2023-10-08T00:12:57.212893+0000","last_active":"2023-10-08T00:12:58.231188+0000","last_peered":"2023-10-08T00:12:58.231188+0000","last_clean":"2023-10-08T00:12:58.231188+0000","last_became_active":"2023-10-08T00:12:57.212632+0000","last_became_peered":"2023-10-08T00:12:57.212632+0000","last_unstale":"2023-10-08T00:12:58.231188+0000","last_undegraded":"2023-10-08T00:12:58.231188+0000","last_fullsized":"2023-10-08T00:12:58.231188+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:00:48.158147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.213967+0000","last_change":"2023-10-08T00:12:58.213646+0000","last_active":"2023-10-08T00:12:58.213967+0000","last_peered":"2023-10-08T00:12:58.213967+0000","last_clean":"2023-10-08T00:12:58.213967+0000","last_became_active":"2023-10-08T00:12:58.213321+0000","last_became_peered":"2023-10-08T00:12:58.213321+0000","last_unstale":"2023-10-08T00:12:58.213967+0000","last_undegraded":"2023-10-08T00:12:58.213967+0000","last_fullsized":"2023-10-08T00:12:58.213967+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:55:28.112676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.207807+0000","last_change":"2023-10-08T00:12:57.214282+0000","last_active":"2023-10-08T00:12:58.207807+0000","last_peered":"2023-10-08T00:12:58.207807+0000","last_clean":"2023-10-08T00:12:58.207807+0000","last_became_active":"2023-10-08T00:12:57.214017+0000","last_became_peered":"2023-10-08T00:12:57.214017+0000","last_unstale":"2023-10-08T00:12:58.207807+0000","last_undegraded":"2023-10-08T00:12:58.207807+0000","last_fullsized":"2023-10-08T00:12:58.207807+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:52:20.648298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.215934+0000","last_change":"2023-10-08T00:12:58.215519+0000","last_active":"2023-10-08T00:12:58.215934+0000","last_peered":"2023-10-08T00:12:58.215934+0000","last_clean":"2023-10-08T00:12:58.215934+0000","last_became_active":"2023-10-08T00:12:58.215263+0000","last_became_peered":"2023-10-08T00:12:58.215263+0000","last_unstale":"2023-10-08T00:12:58.215934+0000","last_undegraded":"2023-10-08T00:12:58.215934+0000","last_fullsized":"2023-10-08T00:12:58.215934+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:23:42.956575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.217417+0000","last_change":"2023-10-08T00:12:57.210417+0000","last_active":"2023-10-08T00:12:58.217417+0000","last_peered":"2023-10-08T00:12:58.217417+0000","last_clean":"2023-10-08T00:12:58.217417+0000","last_became_active":"2023-10-08T00:12:57.210168+0000","last_became_peered":"2023-10-08T00:12:57.210168+0000","last_unstale":"2023-10-08T00:12:58.217417+0000","last_undegraded":"2023-10-08T00:12:58.217417+0000","last_fullsized":"2023-10-08T00:12:58.217417+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:51:53.256123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.214100+0000","last_change":"2023-10-08T00:12:58.214050+0000","last_active":"2023-10-08T00:12:58.214100+0000","last_peered":"2023-10-08T00:12:58.214100+0000","last_clean":"2023-10-08T00:12:58.214100+0000","last_became_active":"2023-10-08T00:12:58.213599+0000","last_became_peered":"2023-10-08T00:12:58.213599+0000","last_unstale":"2023-10-08T00:12:58.214100+0000","last_undegraded":"2023-10-08T00:12:58.214100+0000","last_fullsized":"2023-10-08T00:12:58.214100+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:13:42.175367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"68'2","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.228064+0000","last_change":"2023-10-08T00:12:57.213694+0000","last_active":"2023-10-08T00:12:58.228064+0000","last_peered":"2023-10-08T00:12:58.228064+0000","last_clean":"2023-10-08T00:12:58.228064+0000","last_became_active":"2023-10-08T00:12:57.213466+0000","last_became_peered":"2023-10-08T00:12:57.213466+0000","last_unstale":"2023-10-08T00:12:58.228064+0000","last_undegraded":"2023-10-08T00:12:58.228064+0000","last_fullsized":"2023-10-08T00:12:58.228064+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:50:16.963706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.207384+0000","last_change":"2023-10-08T00:12:58.207342+0000","last_active":"2023-10-08T00:12:58.207384+0000","last_peered":"2023-10-08T00:12:58.207384+0000","last_clean":"2023-10-08T00:12:58.207384+0000","last_became_active":"2023-10-08T00:12:58.207150+0000","last_became_peered":"2023-10-08T00:12:58.207150+0000","last_unstale":"2023-10-08T00:12:58.207384+0000","last_undegraded":"2023-10-08T00:12:58.207384+0000","last_fullsized":"2023-10-08T00:12:58.207384+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:03:23.634875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"69'1","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-08T00:13:00.374632+0000","last_change":"2023-10-08T00:12:57.211421+0000","last_active":"2023-10-08T00:13:00.374632+0000","last_peered":"2023-10-08T00:13:00.374632+0000","last_clean":"2023-10-08T00:13:00.374632+0000","last_became_active":"2023-10-08T00:12:57.210901+0000","last_became_peered":"2023-10-08T00:12:57.210901+0000","last_unstale":"2023-10-08T00:13:00.374632+0000","last_undegraded":"2023-10-08T00:13:00.374632+0000","last_fullsized":"2023-10-08T00:13:00.374632+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:43:05.018683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.218206+0000","last_change":"2023-10-08T00:12:58.217959+0000","last_active":"2023-10-08T00:12:58.218206+0000","last_peered":"2023-10-08T00:12:58.218206+0000","last_clean":"2023-10-08T00:12:58.218206+0000","last_became_active":"2023-10-08T00:12:58.217566+0000","last_became_peered":"2023-10-08T00:12:58.217566+0000","last_unstale":"2023-10-08T00:12:58.218206+0000","last_undegraded":"2023-10-08T00:12:58.218206+0000","last_fullsized":"2023-10-08T00:12:58.218206+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:58:43.406027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.217750+0000","last_change":"2023-10-08T00:12:57.206333+0000","last_active":"2023-10-08T00:12:58.217750+0000","last_peered":"2023-10-08T00:12:58.217750+0000","last_clean":"2023-10-08T00:12:58.217750+0000","last_became_active":"2023-10-08T00:12:57.206123+0000","last_became_peered":"2023-10-08T00:12:57.206123+0000","last_unstale":"2023-10-08T00:12:58.217750+0000","last_undegraded":"2023-10-08T00:12:58.217750+0000","last_fullsized":"2023-10-08T00:12:58.217750+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:27:43.372597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.216867+0000","last_change":"2023-10-08T00:12:58.216759+0000","last_active":"2023-10-08T00:12:58.216867+0000","last_peered":"2023-10-08T00:12:58.216867+0000","last_clean":"2023-10-08T00:12:58.216867+0000","last_became_active":"2023-10-08T00:12:58.216207+0000","last_became_peered":"2023-10-08T00:12:58.216207+0000","last_unstale":"2023-10-08T00:12:58.216867+0000","last_undegraded":"2023-10-08T00:12:58.216867+0000","last_fullsized":"2023-10-08T00:12:58.216867+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:09:19.944555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.205120+0000","last_change":"2023-10-08T00:12:57.212538+0000","last_active":"2023-10-08T00:12:58.205120+0000","last_peered":"2023-10-08T00:12:58.205120+0000","last_clean":"2023-10-08T00:12:58.205120+0000","last_became_active":"2023-10-08T00:12:57.212360+0000","last_became_peered":"2023-10-08T00:12:57.212360+0000","last_unstale":"2023-10-08T00:12:58.205120+0000","last_undegraded":"2023-10-08T00:12:58.205120+0000","last_fullsized":"2023-10-08T00:12:58.205120+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:39:23.590206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.216601+0000","last_change":"2023-10-08T00:12:58.215971+0000","last_active":"2023-10-08T00:12:58.216601+0000","last_peered":"2023-10-08T00:12:58.216601+0000","last_clean":"2023-10-08T00:12:58.216601+0000","last_became_active":"2023-10-08T00:12:58.215173+0000","last_became_peered":"2023-10-08T00:12:58.215173+0000","last_unstale":"2023-10-08T00:12:58.216601+0000","last_undegraded":"2023-10-08T00:12:58.216601+0000","last_fullsized":"2023-10-08T00:12:58.216601+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:11:41.089416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.212848+0000","last_change":"2023-10-08T00:12:57.213785+0000","last_active":"2023-10-08T00:12:58.212848+0000","last_peered":"2023-10-08T00:12:58.212848+0000","last_clean":"2023-10-08T00:12:58.212848+0000","last_became_active":"2023-10-08T00:12:57.213351+0000","last_became_peered":"2023-10-08T00:12:57.213351+0000","last_unstale":"2023-10-08T00:12:58.212848+0000","last_undegraded":"2023-10-08T00:12:58.212848+0000","last_fullsized":"2023-10-08T00:12:58.212848+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:17:09.287153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.220048+0000","last_change":"2023-10-08T00:12:58.220013+0000","last_active":"2023-10-08T00:12:58.220048+0000","last_peered":"2023-10-08T00:12:58.220048+0000","last_clean":"2023-10-08T00:12:58.220048+0000","last_became_active":"2023-10-08T00:12:58.219879+0000","last_became_peered":"2023-10-08T00:12:58.219879+0000","last_unstale":"2023-10-08T00:12:58.220048+0000","last_undegraded":"2023-10-08T00:12:58.220048+0000","last_fullsized":"2023-10-08T00:12:58.220048+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:37:04.823729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.204474+0000","last_change":"2023-10-08T00:12:57.209114+0000","last_active":"2023-10-08T00:12:58.204474+0000","last_peered":"2023-10-08T00:12:58.204474+0000","last_clean":"2023-10-08T00:12:58.204474+0000","last_became_active":"2023-10-08T00:12:57.208878+0000","last_became_peered":"2023-10-08T00:12:57.208878+0000","last_unstale":"2023-10-08T00:12:58.204474+0000","last_undegraded":"2023-10-08T00:12:58.204474+0000","last_fullsized":"2023-10-08T00:12:58.204474+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:29:06.434469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.212207+0000","last_change":"2023-10-08T00:12:58.212135+0000","last_active":"2023-10-08T00:12:58.212207+0000","last_peered":"2023-10-08T00:12:58.212207+0000","last_clean":"2023-10-08T00:12:58.212207+0000","last_became_active":"2023-10-08T00:12:58.211767+0000","last_became_peered":"2023-10-08T00:12:58.211767+0000","last_unstale":"2023-10-08T00:12:58.212207+0000","last_undegraded":"2023-10-08T00:12:58.212207+0000","last_fullsized":"2023-10-08T00:12:58.212207+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:03:23.634875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.211537+0000","last_change":"2023-10-08T00:12:57.213998+0000","last_active":"2023-10-08T00:12:58.211537+0000","last_peered":"2023-10-08T00:12:58.211537+0000","last_clean":"2023-10-08T00:12:58.211537+0000","last_became_active":"2023-10-08T00:12:57.213768+0000","last_became_peered":"2023-10-08T00:12:57.213768+0000","last_unstale":"2023-10-08T00:12:58.211537+0000","last_undegraded":"2023-10-08T00:12:58.211537+0000","last_fullsized":"2023-10-08T00:12:58.211537+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:36:23.496893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.216562+0000","last_change":"2023-10-08T00:12:58.216093+0000","last_active":"2023-10-08T00:12:58.216562+0000","last_peered":"2023-10-08T00:12:58.216562+0000","last_clean":"2023-10-08T00:12:58.216562+0000","last_became_active":"2023-10-08T00:12:58.215633+0000","last_became_peered":"2023-10-08T00:12:58.215633+0000","last_unstale":"2023-10-08T00:12:58.216562+0000","last_undegraded":"2023-10-08T00:12:58.216562+0000","last_fullsized":"2023-10-08T00:12:58.216562+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:19:19.113722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.208226+0000","last_change":"2023-10-08T00:12:57.209066+0000","last_active":"2023-10-08T00:12:58.208226+0000","last_peered":"2023-10-08T00:12:58.208226+0000","last_clean":"2023-10-08T00:12:58.208226+0000","last_became_active":"2023-10-08T00:12:57.208849+0000","last_became_peered":"2023-10-08T00:12:57.208849+0000","last_unstale":"2023-10-08T00:12:58.208226+0000","last_undegraded":"2023-10-08T00:12:58.208226+0000","last_fullsized":"2023-10-08T00:12:58.208226+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:29:06.434469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.216756+0000","last_change":"2023-10-08T00:12:58.216473+0000","last_active":"2023-10-08T00:12:58.216756+0000","last_peered":"2023-10-08T00:12:58.216756+0000","last_clean":"2023-10-08T00:12:58.216756+0000","last_became_active":"2023-10-08T00:12:58.215782+0000","last_became_peered":"2023-10-08T00:12:58.215782+0000","last_unstale":"2023-10-08T00:12:58.216756+0000","last_undegraded":"2023-10-08T00:12:58.216756+0000","last_fullsized":"2023-10-08T00:12:58.216756+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:57.194440+0000","last_clean_scrub_stamp":"2023-10-08T00:12:57.194440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:10:15.405726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.214592+0000","last_change":"2023-10-08T00:12:57.212524+0000","last_active":"2023-10-08T00:12:58.214592+0000","last_peered":"2023-10-08T00:12:58.214592+0000","last_clean":"2023-10-08T00:12:58.214592+0000","last_became_active":"2023-10-08T00:12:57.212291+0000","last_became_peered":"2023-10-08T00:12:57.212291+0000","last_unstale":"2023-10-08T00:12:58.214592+0000","last_undegraded":"2023-10-08T00:12:58.214592+0000","last_fullsized":"2023-10-08T00:12:58.214592+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:12:56.185215+0000","last_clean_scrub_stamp":"2023-10-08T00:12:56.185215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:00:48.158147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":201,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-08T00:12:58.217824+0000","last_change":"2023-10-08T00:10:43.728490+0000","last_active":"2023-10-08T00:12:58.217824+0000","last_peered":"2023-10-08T00:12:58.217824+0000","last_clean":"2023-10-08T00:12:58.217824+0000","last_became_active":"2023-10-08T00:10:43.728341+0000","last_became_peered":"2023-10-08T00:10:43.728341+0000","last_unstale":"2023-10-08T00:12:58.217824+0000","last_undegraded":"2023-10-08T00:12:58.217824+0000","last_fullsized":"2023-10-08T00:12:58.217824+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_clean_scrub_stamp":"2023-10-08T00:10:42.716096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:06:31.035293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574878,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":349,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194963,"statfs":{"total":200000000,"available":199642576,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.496,"15min":0.49399999999999999},"min":{"1min":0.34599999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63100000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.53800000000000003,"15min":0.54400000000000004},"min":{"1min":0.38100000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.73499999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.53500000000000003}]},{"osd":1,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.53100000000000003,"15min":0.52700000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.437,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.65100000000000002}]},{"osd":2,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.53000000000000003,"15min":0.53300000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.66600000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.51600000000000001,"15min":0.51400000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.71899999999999997,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.501}]},{"osd":3,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.51700000000000002,"15min":0.51300000000000001},"min":{"1min":0.36599999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.55200000000000005,"15min":0.55400000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.68300000000000005}]},{"osd":4,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.52200000000000002,"15min":0.52600000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.70799999999999996,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.52800000000000002,"15min":0.53200000000000003},"min":{"1min":0.35699999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.64300000000000002,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.47699999999999998}]},{"osd":5,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.55300000000000005,"15min":0.55700000000000005},"min":{"1min":0.32100000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.71999999999999997,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.57999999999999996,"15min":0.59199999999999997},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.69399999999999995,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.66900000000000004}]},{"osd":6,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.59399999999999997,"15min":0.59899999999999998},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.71299999999999997,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.627,"15min":0.63800000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76000000000000001,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.60999999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574878,"num_pgs":31,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":348,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194963,"statfs":{"total":200000000,"available":199642646,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.48099999999999998,"15min":0.47499999999999998},"min":{"1min":0.377,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.752,"5min":1.012,"15min":1.012},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.68400000000000005,"15min":0.70499999999999996},"min":{"1min":0.34000000000000002,"5min":0.317,"15min":0.317},"max":{"1min":1.2789999999999999,"5min":5.1230000000000002,"15min":5.1230000000000002},"last":0.41999999999999998}]},{"osd":1,"last update":"Sun Oct 8 00:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.52100000000000002,"15min":0.51400000000000001},"min":{"1min":0.29299999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.82499999999999996,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55900000000000005,"15min":0.56000000000000005},"min":{"1min":0.371,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.69899999999999995,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.46600000000000003}]},{"osd":2,"last update":"Sun Oct 8 00:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.52700000000000002,"15min":0.52500000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.755,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56200000000000006,"15min":0.56499999999999995},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.76100000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.49399999999999999}]},{"osd":3,"last update":"Sun Oct 8 00:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53400000000000003,"15min":0.53000000000000003},"min":{"1min":0.38400000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.81399999999999995,"5min":1.046,"15min":1.046},"last":0.504},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.54400000000000004,"15min":0.53600000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.48199999999999998}]},{"osd":4,"last update":"Sun Oct 8 00:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.502,"15min":0.50600000000000001},"min":{"1min":0.32300000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56499999999999995,"15min":0.56699999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.54100000000000004}]},{"osd":5,"last update":"Sun Oct 8 00:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.48499999999999999,"15min":0.48799999999999999},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71099999999999997,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.378},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.55500000000000005,"15min":0.56299999999999994},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.53000000000000003}]},{"osd":7,"last update":"Sun Oct 8 00:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.54100000000000004,"15min":0.54500000000000004},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.432}]}]},{"osd":1,"up_from":13,"seq":55834574878,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":926,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194386,"statfs":{"total":200000000,"available":199051746,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.30399999999999999,"15min":0.29899999999999999},"min":{"1min":0.218,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.44700000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.375,"15min":0.373},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.37}]},{"osd":2,"last update":"Sun Oct 8 00:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.33600000000000002,"15min":0.33200000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.49099999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.379,"5min":0.33900000000000002,"15min":0.33200000000000002},"min":{"1min":0.254,"5min":0.252,"15min":0.252},"max":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.32700000000000001}]},{"osd":3,"last update":"Sun Oct 8 00:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.36699999999999999,"15min":0.36499999999999999},"min":{"1min":0.28299999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.501,"5min":0.501,"15min":0.501},"last":0.315}]},{"osd":4,"last update":"Sun Oct 8 00:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.53700000000000003,"15min":0.53400000000000003},"min":{"1min":0.40400000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.56899999999999995,"15min":0.56499999999999995},"min":{"1min":0.41899999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.82299999999999995,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.60099999999999998}]},{"osd":5,"last update":"Sun Oct 8 00:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.53800000000000003,"15min":0.53100000000000003},"min":{"1min":0.372,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57499999999999996,"15min":0.57199999999999995},"min":{"1min":0.42999999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78300000000000003,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.57999999999999996}]},{"osd":6,"last update":"Sun Oct 8 00:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.72999999999999998,"5min":0.877,"15min":0.877},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56399999999999995,"15min":0.55800000000000005},"min":{"1min":0.42899999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.76900000000000002,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.59199999999999997}]},{"osd":7,"last update":"Sun Oct 8 00:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56299999999999994,"15min":0.55900000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.77700000000000002,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.58899999999999997,"15min":0.58499999999999996},"min":{"1min":0.432,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82299999999999995,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.61299999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574878,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":350,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194962,"statfs":{"total":200000000,"available":199641132,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 00:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.32400000000000001,"15min":0.32600000000000001},"min":{"1min":0.23499999999999999,"5min":0.193,"15min":0.193},"max":{"1min":0.41499999999999998,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.215},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.245,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.49299999999999999,"5min":0.87,"15min":0.87},"last":0.28899999999999998}]},{"osd":2,"last update":"Sun Oct 8 00:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.38200000000000001,"15min":0.38600000000000001},"min":{"1min":0.27900000000000003,"5min":0.251,"15min":0.251},"max":{"1min":0.58699999999999997,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.314},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.753,"15min":0.82399999999999995},"min":{"1min":0.247,"5min":0.185,"15min":0.185},"max":{"1min":0.40899999999999997,"5min":12.019,"15min":12.019},"last":0.24099999999999999}]},{"osd":3,"last update":"Sun Oct 8 00:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.35599999999999998,"15min":0.35999999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.47399999999999998,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.374,"5min":0.38800000000000001,"15min":0.39000000000000001},"min":{"1min":0.28499999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.55900000000000005,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.33300000000000002}]},{"osd":4,"last update":"Sun Oct 8 00:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.433,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.71899999999999997,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.44},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.624,"15min":0.625},"min":{"1min":0.48299999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.73499999999999999,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.53000000000000003}]},{"osd":5,"last update":"Sun Oct 8 00:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.999,"15min":1.069},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.79800000000000004,"5min":11.992000000000001,"15min":11.992000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.73199999999999998,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.54100000000000004}]},{"osd":6,"last update":"Sun Oct 8 00:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.74399999999999999,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.97799999999999998,"15min":1.042},"min":{"1min":0.47299999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76000000000000001,"5min":11.927,"15min":11.927},"last":0.50900000000000001}]},{"osd":7,"last update":"Sun Oct 8 00:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.999,"15min":1.0660000000000001},"min":{"1min":0.434,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82199999999999995,"5min":11.965,"15min":11.965},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.61899999999999999,"15min":0.621},"min":{"1min":0.48199999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.76100000000000001,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.56200000000000006}]}]},{"osd":2,"up_from":13,"seq":55834574878,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":350,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194961,"statfs":{"total":200000000,"available":199640972,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.39000000000000001,"15min":0.40200000000000002},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.437,"5min":2.5139999999999998,"15min":2.5139999999999998},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.40100000000000002,"15min":0.41399999999999998},"min":{"1min":0.248,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.44400000000000001,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":0.28199999999999997}]},{"osd":1,"last update":"Sun Oct 8 00:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.39000000000000001,"15min":0.40200000000000002},"min":{"1min":0.25,"5min":0.246,"15min":0.246},"max":{"1min":0.40100000000000002,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.436,"15min":0.45000000000000001},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.48999999999999999,"5min":2.5379999999999998,"15min":2.5379999999999998},"last":0.373}]},{"osd":3,"last update":"Sun Oct 8 00:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.42799999999999999,"15min":0.439},"min":{"1min":0.28499999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.46500000000000002,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.379,"5min":0.46300000000000002,"15min":0.47699999999999998},"min":{"1min":0.315,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.51200000000000001,"5min":2.5619999999999998,"15min":2.5619999999999998},"last":0.313}]},{"osd":4,"last update":"Sun Oct 8 00:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.61699999999999999,"15min":0.628},"min":{"1min":0.45800000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.84599999999999997,"5min":2.488,"15min":2.488},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.60399999999999998,"15min":0.60999999999999999},"min":{"1min":0.41099999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85999999999999999,"5min":2.073,"15min":2.073},"last":0.59399999999999997}]},{"osd":5,"last update":"Sun Oct 8 00:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.66300000000000003,"15min":0.66900000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.2130000000000001,"5min":2.2069999999999999,"15min":2.2069999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.62,"5min":0.66300000000000003,"15min":0.67000000000000004},"min":{"1min":0.48799999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0149999999999999,"5min":2.456,"15min":2.456},"last":0.69499999999999995}]},{"osd":6,"last update":"Sun Oct 8 00:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.621,"15min":0.627},"min":{"1min":0.45000000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.79500000000000004,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.60899999999999999,"15min":0.61399999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.97499999999999998,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.72099999999999997}]},{"osd":7,"last update":"Sun Oct 8 00:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.63700000000000001,"15min":0.64100000000000001},"min":{"1min":0.499,"5min":0.438,"15min":0.438},"max":{"1min":0.81699999999999995,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.626,"15min":0.63200000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91200000000000003,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":0.58199999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":349,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194963,"statfs":{"total":200000000,"available":199642578,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.28699999999999998,"5min":0.36699999999999999,"15min":0.38},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.45100000000000001,"5min":1.46,"15min":1.46},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.29699999999999999,"5min":0.33500000000000002,"15min":0.34100000000000003},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.46300000000000002,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.34499999999999997}]},{"osd":1,"last update":"Sun Oct 8 00:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.82999999999999996,"15min":0.91700000000000004},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.44800000000000001,"5min":12.351000000000001,"15min":12.351000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.35899999999999999,"15min":0.36299999999999999},"min":{"1min":0.26500000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.443,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.29799999999999999}]},{"osd":2,"last update":"Sun Oct 8 00:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.41399999999999998,"15min":0.42499999999999999},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.47299999999999998,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.379,"15min":0.38300000000000001},"min":{"1min":0.27800000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.45900000000000002,"5min":0.753,"15min":0.753},"last":0.33400000000000002}]},{"osd":4,"last update":"Sun Oct 8 00:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":1.004,"15min":1.0860000000000001},"min":{"1min":0.38800000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.75600000000000001,"5min":11.333,"15min":11.333},"last":0.436},{"interface":"front","average":{"1min":0.54100000000000004,"5min":1.0369999999999999,"15min":1.1200000000000001},"min":{"1min":0.434,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.77000000000000002,"5min":12.321999999999999,"15min":12.321999999999999},"last":0.54100000000000004}]},{"osd":5,"last update":"Sun Oct 8 00:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":1.0509999999999999,"15min":1.137},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.72599999999999998,"5min":12.291,"15min":12.291},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.58599999999999997,"15min":0.59299999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.74299999999999999,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.41999999999999998}]},{"osd":6,"last update":"Sun Oct 8 00:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.96999999999999997,"15min":1.048},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.69899999999999995,"5min":11.288,"15min":11.288},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.51800000000000002,"5min":1.0309999999999999,"15min":1.117},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.66000000000000003,"5min":12.253,"15min":12.253},"last":0.44700000000000001}]},{"osd":7,"last update":"Sun Oct 8 00:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56599999999999995,"15min":0.56799999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":1.012,"15min":1.0860000000000001},"min":{"1min":0.435,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.381,"5min":11.363,"15min":11.363},"last":0.55100000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574878,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":349,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194962,"statfs":{"total":200000000,"available":199642080,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.56000000000000005,"15min":0.55400000000000005},"min":{"1min":0.41299999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.52800000000000002}]},{"osd":1,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51700000000000002,"15min":0.51800000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.64100000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.30199999999999999}]},{"osd":2,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.51600000000000001,"15min":0.52000000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.505},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.51300000000000001,"15min":0.50900000000000001},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.372}]},{"osd":3,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.53500000000000003,"15min":0.53900000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.70899999999999996,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.53000000000000003,"15min":0.53200000000000003},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.68600000000000005}]},{"osd":5,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.49299999999999999,"15min":0.503},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69899999999999995,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.50800000000000001,"15min":0.51400000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.71099999999999997,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.48999999999999999}]},{"osd":6,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.48899999999999999,"15min":0.49399999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.55200000000000005,"15min":0.56200000000000006},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.754,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.47199999999999998}]},{"osd":7,"last update":"Sun Oct 8 00:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.54600000000000004,"15min":0.55500000000000005},"min":{"1min":0.36299999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.64200000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.57799999999999996,"15min":0.58299999999999996},"min":{"1min":0.41199999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.77700000000000002,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.72399999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574878,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":925,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194386,"statfs":{"total":200000000,"available":199052266,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.48699999999999999,"15min":0.48799999999999999},"min":{"1min":0.32700000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68799999999999994,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.55700000000000005,"15min":0.56299999999999994},"min":{"1min":0.374,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.68600000000000005,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.56299999999999994}]},{"osd":1,"last update":"Sun Oct 8 00:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.52100000000000002,"15min":0.52300000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.65800000000000003,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.52400000000000002,"15min":0.52800000000000002},"min":{"1min":0.39300000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.70899999999999996,"5min":0.755,"15min":0.755},"last":0.46100000000000002}]},{"osd":2,"last update":"Sun Oct 8 00:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.52900000000000003,"15min":0.53100000000000003},"min":{"1min":0.40500000000000003,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.64500000000000002,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.56599999999999995,"15min":0.57199999999999995},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.70699999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.60499999999999998}]},{"osd":3,"last update":"Sun Oct 8 00:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54300000000000004,"15min":0.54500000000000004},"min":{"1min":0.38700000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.67100000000000004,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.57999999999999996,"15min":0.58399999999999996},"min":{"1min":0.42799999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.72299999999999998,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.66800000000000004}]},{"osd":4,"last update":"Sun Oct 8 00:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.47299999999999998,"15min":0.47499999999999998},"min":{"1min":0.35799999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.626,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.50900000000000001,"15min":0.51300000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.63700000000000001,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.746}]},{"osd":6,"last update":"Sun Oct 8 00:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.499,"15min":0.499},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.61899999999999999,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.66800000000000004,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.58799999999999997}]},{"osd":7,"last update":"Sun Oct 8 00:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.51300000000000001,"15min":0.51500000000000001},"min":{"1min":0.33400000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67300000000000004,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.69799999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.72999999999999998}]}]}],"pool_statfs":[]}} 2023-10-08T00:13:07.125 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T00:13:07.125 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-08T00:13:07.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.280+0000 7f630750f700 1 Processor -- start 2023-10-08T00:13:07.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.280+0000 7f630750f700 1 -- start start 2023-10-08T00:13:07.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.281+0000 7f630750f700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6300141780 0x7f6300070700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:07.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.281+0000 7f630750f700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f630006e2e0 0x7f6300070c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:07.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.281+0000 7f630750f700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f630006e8a0 con 0x7f63001413b0 2023-10-08T00:13:07.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.281+0000 7f630750f700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f630006eb80 con 0x7f6300141780 2023-10-08T00:13:07.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.281+0000 7f630750f700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f630006ee60 con 0x7f630006e2e0 2023-10-08T00:13:07.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.281+0000 7f6304aaa700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6300141780 0x7f6300070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:07.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.281+0000 7f6304aaa700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6300141780 0x7f6300070700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46296/0 (socket says 172.21.15.99:46296) 2023-10-08T00:13:07.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.281+0000 7f6304aaa700 1 -- 172.21.15.99:0/4171152820 learned_addr learned my addr 172.21.15.99:0/4171152820 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:13:07.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.282+0000 7f6305aac700 1 --2- 172.21.15.99:0/4171152820 >> v2:172.21.15.191:3300/0 conn(0x7f630006e2e0 0x7f6300070c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:07.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.282+0000 7f6304aaa700 1 -- 172.21.15.99:0/4171152820 >> v2:172.21.15.191:3300/0 conn(0x7f630006e2e0 msgr2=0x7f6300070c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:13:07.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.282+0000 7f6304aaa700 1 --2- 172.21.15.99:0/4171152820 >> v2:172.21.15.191:3300/0 conn(0x7f630006e2e0 0x7f6300070c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:07.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.282+0000 7f6304aaa700 1 -- 172.21.15.99:0/4171152820 >> v1:172.21.15.191:6789/0 conn(0x7f63001413b0 legacy=0x7f6300068a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:07.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.282+0000 7f6304aaa700 1 -- 172.21.15.99:0/4171152820 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f630006f140 con 0x7f6300141780 2023-10-08T00:13:07.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.282+0000 7f6305aac700 1 --2- 172.21.15.99:0/4171152820 >> v2:172.21.15.191:3300/0 conn(0x7f630006e2e0 0x7f6300070c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:13:07.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.282+0000 7f6304aaa700 1 --2- 172.21.15.99:0/4171152820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6300141780 0x7f6300070700 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f62f00099b0 tx=0x7f62f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4028a0fcd62106e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:13:07.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.282+0000 7f62f7fff700 1 -- 172.21.15.99:0/4171152820 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f000b820 con 0x7f6300141780 2023-10-08T00:13:07.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.282+0000 7f62f7fff700 1 -- 172.21.15.99:0/4171152820 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62f000b980 con 0x7f6300141780 2023-10-08T00:13:07.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.282+0000 7f62f7fff700 1 -- 172.21.15.99:0/4171152820 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f000bd20 con 0x7f6300141780 2023-10-08T00:13:07.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.283+0000 7f630750f700 1 -- 172.21.15.99:0/4171152820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6300141780 msgr2=0x7f6300070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:07.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.283+0000 7f630750f700 1 --2- 172.21.15.99:0/4171152820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6300141780 0x7f6300070700 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f62f00099b0 tx=0x7f62f0009250 comp rx=0 tx=0).stop 2023-10-08T00:13:07.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.283+0000 7f630750f700 1 -- 172.21.15.99:0/4171152820 shutdown_connections 2023-10-08T00:13:07.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.283+0000 7f630750f700 1 --2- 172.21.15.99:0/4171152820 >> v2:172.21.15.191:3300/0 conn(0x7f630006e2e0 0x7f6300070c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:07.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.283+0000 7f630750f700 1 --2- 172.21.15.99:0/4171152820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6300141780 0x7f6300070700 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:07.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.283+0000 7f630750f700 1 -- 172.21.15.99:0/4171152820 >> 172.21.15.99:0/4171152820 conn(0x7f63000ce570 msgr2=0x7f63000ca510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:13:07.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.283+0000 7f630750f700 1 -- 172.21.15.99:0/4171152820 shutdown_connections 2023-10-08T00:13:07.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.283+0000 7f630750f700 1 -- 172.21.15.99:0/4171152820 wait complete. 2023-10-08T00:13:07.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.284+0000 7f630750f700 1 Processor -- start 2023-10-08T00:13:07.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.284+0000 7f630750f700 1 -- start start 2023-10-08T00:13:07.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.285+0000 7f630750f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f630006e2e0 0x7f630016e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:07.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.285+0000 7f630750f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63001413b0 0x7f63001403d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:07.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.285+0000 7f630750f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6300141780 0x7f630013e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:07.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.285+0000 7f630750f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63000c5c60 con 0x7f6300141780 2023-10-08T00:13:07.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.285+0000 7f630750f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630016ee90 con 0x7f630006e2e0 2023-10-08T00:13:07.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.285+0000 7f630750f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63000c4ea0 con 0x7f63001413b0 2023-10-08T00:13:07.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.285+0000 7f6305aac700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6300141780 0x7f630013e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:07.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.285+0000 7f6305aac700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6300141780 0x7f630013e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46306/0 (socket says 172.21.15.99:46306) 2023-10-08T00:13:07.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.285+0000 7f6305aac700 1 -- 172.21.15.99:0/888108119 learned_addr learned my addr 172.21.15.99:0/888108119 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:13:07.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.285+0000 7f63052ab700 1 --2- 172.21.15.99:0/888108119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f630006e2e0 0x7f630016e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:07.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.285+0000 7f6305aac700 1 -- 172.21.15.99:0/888108119 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63001413b0 msgr2=0x7f63001403d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:07.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.285+0000 7f6305aac700 1 --2- 172.21.15.99:0/888108119 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63001413b0 0x7f63001403d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:07.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.285+0000 7f6305aac700 1 -- 172.21.15.99:0/888108119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f630006e2e0 msgr2=0x7f630016e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:07.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.285+0000 7f6305aac700 1 --2- 172.21.15.99:0/888108119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f630006e2e0 0x7f630016e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:07.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.285+0000 7f6305aac700 1 -- 172.21.15.99:0/888108119 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62f00094f0 con 0x7f6300141780 2023-10-08T00:13:07.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.285+0000 7f6305aac700 1 --2- 172.21.15.99:0/888108119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6300141780 0x7f630013e210 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f62f800d0d0 tx=0x7f62f8009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:13:07.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.285+0000 7f62f57fa700 1 -- 172.21.15.99:0/888108119 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f800dd70 con 0x7f6300141780 2023-10-08T00:13:07.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.285+0000 7f62f57fa700 1 -- 172.21.15.99:0/888108119 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62f800f040 con 0x7f6300141780 2023-10-08T00:13:07.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.285+0000 7f630750f700 1 -- 172.21.15.99:0/888108119 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63000c56a0 con 0x7f6300141780 2023-10-08T00:13:07.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.285+0000 7f62f57fa700 1 -- 172.21.15.99:0/888108119 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f8015670 con 0x7f6300141780 2023-10-08T00:13:07.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.285+0000 7f630750f700 1 -- 172.21.15.99:0/888108119 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63000c7050 con 0x7f6300141780 2023-10-08T00:13:07.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.286+0000 7f630750f700 1 -- 172.21.15.99:0/888108119 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63000b2b70 con 0x7f6300141780 2023-10-08T00:13:07.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.287+0000 7f62f57fa700 1 -- 172.21.15.99:0/888108119 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f62f801e3b0 con 0x7f6300141780 2023-10-08T00:13:07.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.287+0000 7f62f57fa700 1 --2- 172.21.15.99:0/888108119 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f62e4066f30 0x7f62e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:07.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.287+0000 7f62f57fa700 1 -- 172.21.15.99:0/888108119 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f62f8084a70 con 0x7f6300141780 2023-10-08T00:13:07.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.287+0000 7f63052ab700 1 --2- 172.21.15.99:0/888108119 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f62e4066f30 0x7f62e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:07.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.288+0000 7f63052ab700 1 --2- 172.21.15.99:0/888108119 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f62e4066f30 0x7f62e40693e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f62fc0016d0 tx=0x7f62fc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:13:07.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.291+0000 7f62f57fa700 1 -- 172.21.15.99:0/888108119 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f62f8051ce0 con 0x7f6300141780 2023-10-08T00:13:07.548 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.547+0000 7f630750f700 1 -- 172.21.15.99:0/888108119 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f63000c7270 con 0x7f6300141780 2023-10-08T00:13:07.551 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.551+0000 7f62f57fa700 1 -- 172.21.15.99:0/888108119 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+0 (secure 0 0 0) 0x7f62f801a0e0 con 0x7f6300141780 2023-10-08T00:13:07.554 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.553+0000 7f630750f700 1 -- 172.21.15.99:0/888108119 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f62e4066f30 msgr2=0x7f62e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:07.554 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.553+0000 7f630750f700 1 --2- 172.21.15.99:0/888108119 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f62e4066f30 0x7f62e40693e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f62fc0016d0 tx=0x7f62fc009040 comp rx=0 tx=0).stop 2023-10-08T00:13:07.554 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.553+0000 7f630750f700 1 -- 172.21.15.99:0/888108119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6300141780 msgr2=0x7f630013e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:07.554 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.553+0000 7f630750f700 1 --2- 172.21.15.99:0/888108119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6300141780 0x7f630013e210 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f62f800d0d0 tx=0x7f62f8009a50 comp rx=0 tx=0).stop 2023-10-08T00:13:07.554 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.553+0000 7f630750f700 1 -- 172.21.15.99:0/888108119 shutdown_connections 2023-10-08T00:13:07.554 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.553+0000 7f630750f700 1 --2- 172.21.15.99:0/888108119 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f62e4066f30 0x7f62e40693e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:07.554 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.553+0000 7f630750f700 1 --2- 172.21.15.99:0/888108119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f630006e2e0 0x7f630016e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:07.554 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.553+0000 7f630750f700 1 --2- 172.21.15.99:0/888108119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6300141780 0x7f630013e210 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:07.555 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.553+0000 7f630750f700 1 --2- 172.21.15.99:0/888108119 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63001413b0 0x7f63001403d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:07.555 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.553+0000 7f630750f700 1 -- 172.21.15.99:0/888108119 >> 172.21.15.99:0/888108119 conn(0x7f63000ce570 msgr2=0x7f630013ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:13:07.555 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.553+0000 7f630750f700 1 -- 172.21.15.99:0/888108119 shutdown_connections 2023-10-08T00:13:07.555 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.553+0000 7f630750f700 1 -- 172.21.15.99:0/888108119 wait complete. 2023-10-08T00:13:07.555 INFO:teuthology.orchestra.run.smithi099.stderr:updated caps for client.0 2023-10-08T00:13:07.565 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:13:07.679 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.678+0000 7fccf7699700 1 Processor -- start 2023-10-08T00:13:07.679 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.678+0000 7fccf7699700 1 -- start start 2023-10-08T00:13:07.680 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.679+0000 7fccf7699700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fccf00cc3c0 0x7fccf006a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:07.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.679+0000 7fccf7699700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fccf00aff60 0x7fccf0163c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:07.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.679+0000 7fccf7699700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccf00b0580 con 0x7fccf006a9f0 2023-10-08T00:13:07.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.679+0000 7fccf7699700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccf00b03a0 con 0x7fccf00cc3c0 2023-10-08T00:13:07.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.679+0000 7fccf7699700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccf006ae90 con 0x7fccf00aff60 2023-10-08T00:13:07.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.679+0000 7fccf5435700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fccf00cc3c0 0x7fccf006a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:07.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.679+0000 7fccf5435700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fccf00cc3c0 0x7fccf006a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.99:36896/0 (socket says 172.21.15.99:36896) 2023-10-08T00:13:07.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.679+0000 7fccf4c34700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7fccf006a9f0 0x7fccf00af850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:56670/0 (socket says 172.21.15.99:56670) 2023-10-08T00:13:07.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.679+0000 7fccf5435700 1 -- 172.21.15.99:0/1889064982 learned_addr learned my addr 172.21.15.99:0/1889064982 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:13:07.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.679+0000 7fccf5c36700 1 --2- 172.21.15.99:0/1889064982 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fccf00aff60 0x7fccf0163c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:07.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.680+0000 7fccf5435700 1 -- 172.21.15.99:0/1889064982 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fccf00aff60 msgr2=0x7fccf0163c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:13:07.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.680+0000 7fccf5435700 1 --2- 172.21.15.99:0/1889064982 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fccf00aff60 0x7fccf0163c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:07.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.680+0000 7fccf5435700 1 -- 172.21.15.99:0/1889064982 >> v1:172.21.15.191:6789/0 conn(0x7fccf006a9f0 legacy=0x7fccf00af850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:07.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.680+0000 7fccf5435700 1 -- 172.21.15.99:0/1889064982 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf015b0c0 con 0x7fccf00cc3c0 2023-10-08T00:13:07.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.680+0000 7fcce7fff700 1 -- 172.21.15.99:0/1889064982 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2103402855 0 0) 0x7fccf00b0580 con 0x7fccf006a9f0 2023-10-08T00:13:07.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.680+0000 7fccf5435700 1 --2- 172.21.15.99:0/1889064982 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fccf00cc3c0 0x7fccf006a4b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fccec009f20 tx=0x7fccec009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6105ae1c73be53db server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:13:07.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.680+0000 7fcce7fff700 1 -- 172.21.15.99:0/1889064982 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccec00e070 con 0x7fccf00cc3c0 2023-10-08T00:13:07.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.680+0000 7fcce7fff700 1 -- 172.21.15.99:0/1889064982 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccec00b9d0 con 0x7fccf00cc3c0 2023-10-08T00:13:07.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.680+0000 7fcce7fff700 1 -- 172.21.15.99:0/1889064982 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccec0092e0 con 0x7fccf00cc3c0 2023-10-08T00:13:07.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.681+0000 7fccf7699700 1 -- 172.21.15.99:0/1889064982 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fccf00cc3c0 msgr2=0x7fccf006a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:07.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.681+0000 7fccf7699700 1 --2- 172.21.15.99:0/1889064982 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fccf00cc3c0 0x7fccf006a4b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fccec009f20 tx=0x7fccec009840 comp rx=0 tx=0).stop 2023-10-08T00:13:07.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.681+0000 7fccf7699700 1 -- 172.21.15.99:0/1889064982 shutdown_connections 2023-10-08T00:13:07.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.681+0000 7fccf7699700 1 --2- 172.21.15.99:0/1889064982 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fccf00aff60 0x7fccf0163c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:07.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.681+0000 7fccf7699700 1 --2- 172.21.15.99:0/1889064982 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fccf00cc3c0 0x7fccf006a4b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:07.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.681+0000 7fccf7699700 1 -- 172.21.15.99:0/1889064982 >> 172.21.15.99:0/1889064982 conn(0x7fccf0069800 msgr2=0x7fccf00d0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:13:07.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.681+0000 7fccf7699700 1 -- 172.21.15.99:0/1889064982 shutdown_connections 2023-10-08T00:13:07.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.681+0000 7fccf7699700 1 -- 172.21.15.99:0/1889064982 wait complete. 2023-10-08T00:13:07.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.682+0000 7fccf7699700 1 Processor -- start 2023-10-08T00:13:07.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.682+0000 7fccf7699700 1 -- start start 2023-10-08T00:13:07.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.683+0000 7fccf7699700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccf006a9f0 0x7fccf0155de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:07.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.683+0000 7fccf7699700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fccf00aff60 0x7fccf00b1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:07.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.683+0000 7fccf7699700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fccf00cc3c0 0x7fccf00b2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:07.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.683+0000 7fccf7699700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf00b48d0 con 0x7fccf006a9f0 2023-10-08T00:13:07.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.683+0000 7fccf7699700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf00b3c50 con 0x7fccf00aff60 2023-10-08T00:13:07.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.683+0000 7fccf7699700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccf00b3f30 con 0x7fccf00cc3c0 2023-10-08T00:13:07.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.683+0000 7fccf5435700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccf006a9f0 0x7fccf0155de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:07.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.683+0000 7fccf5435700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccf006a9f0 0x7fccf0155de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46310/0 (socket says 172.21.15.99:46310) 2023-10-08T00:13:07.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.683+0000 7fccf5435700 1 -- 172.21.15.99:0/3325583423 learned_addr learned my addr 172.21.15.99:0/3325583423 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:13:07.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.683+0000 7fccf5c36700 1 --2- 172.21.15.99:0/3325583423 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fccf00cc3c0 0x7fccf00b2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:07.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.683+0000 7fccf4c34700 1 --2- 172.21.15.99:0/3325583423 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fccf00aff60 0x7fccf00b1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:07.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.683+0000 7fccf5435700 1 -- 172.21.15.99:0/3325583423 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fccf00cc3c0 msgr2=0x7fccf00b2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:07.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.683+0000 7fccf5435700 1 --2- 172.21.15.99:0/3325583423 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fccf00cc3c0 0x7fccf00b2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:07.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.683+0000 7fccf5435700 1 -- 172.21.15.99:0/3325583423 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fccf00aff60 msgr2=0x7fccf00b1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:07.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.683+0000 7fccf5435700 1 --2- 172.21.15.99:0/3325583423 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fccf00aff60 0x7fccf00b1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:07.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.683+0000 7fccf5435700 1 -- 172.21.15.99:0/3325583423 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccec009ae0 con 0x7fccf006a9f0 2023-10-08T00:13:07.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.683+0000 7fccf4c34700 1 --2- 172.21.15.99:0/3325583423 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fccf00aff60 0x7fccf00b1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:13:07.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.683+0000 7fccf5435700 1 --2- 172.21.15.99:0/3325583423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccf006a9f0 0x7fccf0155de0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fccec00b610 tx=0x7fccec018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:13:07.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.683+0000 7fcce57fa700 1 -- 172.21.15.99:0/3325583423 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccec008a50 con 0x7fccf006a9f0 2023-10-08T00:13:07.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.683+0000 7fcce57fa700 1 -- 172.21.15.99:0/3325583423 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccec008bb0 con 0x7fccf006a9f0 2023-10-08T00:13:07.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.683+0000 7fcce57fa700 1 -- 172.21.15.99:0/3325583423 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccec01c410 con 0x7fccf006a9f0 2023-10-08T00:13:07.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.683+0000 7fccf7699700 1 -- 172.21.15.99:0/3325583423 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccf00b4210 con 0x7fccf006a9f0 2023-10-08T00:13:07.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.684+0000 7fccf7699700 1 -- 172.21.15.99:0/3325583423 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccf0156320 con 0x7fccf006a9f0 2023-10-08T00:13:07.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.685+0000 7fccf7699700 1 -- 172.21.15.99:0/3325583423 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccf0089110 con 0x7fccf006a9f0 2023-10-08T00:13:07.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.685+0000 7fcce57fa700 1 -- 172.21.15.99:0/3325583423 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fccec003040 con 0x7fccf006a9f0 2023-10-08T00:13:07.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.685+0000 7fcce57fa700 1 --2- 172.21.15.99:0/3325583423 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fccd4066f30 0x7fccd40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:07.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.685+0000 7fcce57fa700 1 -- 172.21.15.99:0/3325583423 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fccec085bd0 con 0x7fccf006a9f0 2023-10-08T00:13:07.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.685+0000 7fccf4c34700 1 --2- 172.21.15.99:0/3325583423 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fccd4066f30 0x7fccd40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:07.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.686+0000 7fccf4c34700 1 --2- 172.21.15.99:0/3325583423 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fccd4066f30 0x7fccd40693e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fccf016d9f0 tx=0x7fcce0007c90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:13:07.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.689+0000 7fcce57fa700 1 -- 172.21.15.99:0/3325583423 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fccec052e40 con 0x7fccf006a9f0 2023-10-08T00:13:07.912 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.912+0000 7fccf7699700 1 -- 172.21.15.99:0/3325583423 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fccf004c3c0 con 0x7fccf006a9f0 2023-10-08T00:13:07.913 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.912+0000 7fcce57fa700 1 -- 172.21.15.99:0/3325583423 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4600 (secure 0 0 0) 0x7fccec056460 con 0x7fccf006a9f0 2023-10-08T00:13:07.913 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:13:07.913 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5087,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/862112794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":862112794},{"type":"v1","addr":"172.21.15.191:6837","nonce":862112794}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5673,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/2694493794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2694493794},{"type":"v1","addr":"172.21.15.99:6834","nonce":2694493794}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1906990531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":1906990531},{"type":"v1","addr":"172.21.15.99:6835","nonce":1906990531}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:12:57.581246+0000","modified":"2023-10-08T00:13:01.240449+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5688},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5688":{"gid":5688,"name":"d","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6836/1779510877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":1779510877},{"type":"v1","addr":"172.21.15.191:6836","nonce":1779510877}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-08T00:13:07.915 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.914+0000 7fccf7699700 1 -- 172.21.15.99:0/3325583423 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fccd4066f30 msgr2=0x7fccd40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:07.915 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.914+0000 7fccf7699700 1 --2- 172.21.15.99:0/3325583423 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fccd4066f30 0x7fccd40693e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fccf016d9f0 tx=0x7fcce0007c90 comp rx=0 tx=0).stop 2023-10-08T00:13:07.915 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.914+0000 7fccf7699700 1 -- 172.21.15.99:0/3325583423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccf006a9f0 msgr2=0x7fccf0155de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:07.915 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.914+0000 7fccf7699700 1 --2- 172.21.15.99:0/3325583423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccf006a9f0 0x7fccf0155de0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fccec00b610 tx=0x7fccec018000 comp rx=0 tx=0).stop 2023-10-08T00:13:07.915 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.914+0000 7fccf7699700 1 -- 172.21.15.99:0/3325583423 shutdown_connections 2023-10-08T00:13:07.915 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.914+0000 7fccf7699700 1 --2- 172.21.15.99:0/3325583423 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fccd4066f30 0x7fccd40693e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:07.915 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.914+0000 7fccf7699700 1 --2- 172.21.15.99:0/3325583423 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fccf00cc3c0 0x7fccf00b2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:07.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.914+0000 7fccf7699700 1 --2- 172.21.15.99:0/3325583423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccf006a9f0 0x7fccf0155de0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:07.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.914+0000 7fccf7699700 1 --2- 172.21.15.99:0/3325583423 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fccf00aff60 0x7fccf00b1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:07.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.914+0000 7fccf7699700 1 -- 172.21.15.99:0/3325583423 >> 172.21.15.99:0/3325583423 conn(0x7fccf0069800 msgr2=0x7fccf00d0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:13:07.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.914+0000 7fccf7699700 1 -- 172.21.15.99:0/3325583423 shutdown_connections 2023-10-08T00:13:07.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:07.914+0000 7fccf7699700 1 -- 172.21.15.99:0/3325583423 wait complete. 2023-10-08T00:13:07.916 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 32 2023-10-08T00:13:08.329 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 32, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T00:12:57.581246+0000', 'modified': '2023-10-08T00:13:01.240449+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5688}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5688': {'gid': 5688, 'name': 'd', 'rank': 0, 'incarnation': 28, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.191:6836/1779510877', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6835', 'nonce': 1779510877}, {'type': 'v1', 'addr': '172.21.15.191:6836', 'nonce': 1779510877}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-08T00:13:08.329 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T00:13:08.329 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-08T00:13:08.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.445+0000 7fcd5893c700 1 Processor -- start 2023-10-08T00:13:08.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.445+0000 7fcd5893c700 1 -- start start 2023-10-08T00:13:08.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.446+0000 7fcd5893c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fcd54159e10 0x7fcd540b2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:08.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.446+0000 7fcd5893c700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd5415db80 con 0x7fcd5415a1e0 2023-10-08T00:13:08.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.446+0000 7fcd5893c700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd5415de60 con 0x7fcd540cde50 2023-10-08T00:13:08.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.446+0000 7fcd5893c700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd5415e800 con 0x7fcd54159e10 2023-10-08T00:13:08.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.446+0000 7fcd49d9b700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fcd5415a1e0 0x7fcd540cd740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:47524/0 (socket says 172.21.15.99:47524) 2023-10-08T00:13:08.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.446+0000 7fcd49d9b700 1 -- 172.21.15.99:0/930923459 learned_addr learned my addr 172.21.15.99:0/930923459 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:13:08.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.446+0000 7fcd5259c700 1 --2- 172.21.15.99:0/930923459 >> v2:172.21.15.191:3300/0 conn(0x7fcd54159e10 0x7fcd540b2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:08.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.446+0000 7fcd51d9b700 1 -- 172.21.15.99:0/930923459 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4268851568 0 0) 0x7fcd5415db80 con 0x7fcd5415a1e0 2023-10-08T00:13:08.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.446+0000 7fcd51d9b700 1 -- 172.21.15.99:0/930923459 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd40003570 con 0x7fcd5415a1e0 2023-10-08T00:13:08.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.447+0000 7fcd51d9b700 1 -- 172.21.15.99:0/930923459 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1980078110 0 0) 0x7fcd40003570 con 0x7fcd5415a1e0 2023-10-08T00:13:08.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.447+0000 7fcd51d9b700 1 -- 172.21.15.99:0/930923459 >> v2:172.21.15.191:3300/0 conn(0x7fcd54159e10 msgr2=0x7fcd540b2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:13:08.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.447+0000 7fcd51d9b700 1 --2- 172.21.15.99:0/930923459 >> v2:172.21.15.191:3300/0 conn(0x7fcd54159e10 0x7fcd540b2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:08.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.447+0000 7fcd51d9b700 1 -- 172.21.15.99:0/930923459 >> v1:172.21.15.191:6789/0 conn(0x7fcd540cde50 legacy=0x7fcd540cbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:08.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.447+0000 7fcd51d9b700 1 -- 172.21.15.99:0/930923459 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd5415e140 con 0x7fcd5415a1e0 2023-10-08T00:13:08.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.447+0000 7fcd51d9b700 1 -- 172.21.15.99:0/930923459 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7fcd3c001470 con 0x7fcd5415a1e0 2023-10-08T00:13:08.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.447+0000 7fcd51d9b700 1 -- 172.21.15.99:0/930923459 >> v1:172.21.15.99:6789/0 conn(0x7fcd5415a1e0 legacy=0x7fcd540cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:08.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.447+0000 7fcd51d9b700 1 --2- 172.21.15.99:0/930923459 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd40003a60 0x7fcd40003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:08.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.447+0000 7fcd51d9b700 1 --2- 172.21.15.99:0/930923459 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd40004810 0x7fcd40004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:08.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.447+0000 7fcd51d9b700 1 --2- 172.21.15.99:0/930923459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd400051c0 0x7fcd4000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:08.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.447+0000 7fcd51d9b700 1 -- 172.21.15.99:0/930923459 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5415e800 con 0x7fcd400051c0 2023-10-08T00:13:08.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.447+0000 7fcd51d9b700 1 -- 172.21.15.99:0/930923459 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4000c160 con 0x7fcd40003a60 2023-10-08T00:13:08.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.447+0000 7fcd51d9b700 1 -- 172.21.15.99:0/930923459 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4000c320 con 0x7fcd40004810 2023-10-08T00:13:08.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.447+0000 7fcd52d9d700 1 --2- 172.21.15.99:0/930923459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd400051c0 0x7fcd4000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:08.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.447+0000 7fcd5259c700 1 --2- 172.21.15.99:0/930923459 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd40003a60 0x7fcd40003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:08.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.447+0000 7fcd49d9b700 1 --2- 172.21.15.99:0/930923459 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd40004810 0x7fcd40004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:08.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.447+0000 7fcd52d9d700 1 -- 172.21.15.99:0/930923459 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd40004810 msgr2=0x7fcd40004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:13:08.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.447+0000 7fcd52d9d700 1 --2- 172.21.15.99:0/930923459 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd40004810 0x7fcd40004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:08.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.447+0000 7fcd52d9d700 1 -- 172.21.15.99:0/930923459 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd40003a60 msgr2=0x7fcd40003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:13:08.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.447+0000 7fcd52d9d700 1 --2- 172.21.15.99:0/930923459 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd40003a60 0x7fcd40003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:08.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.447+0000 7fcd52d9d700 1 -- 172.21.15.99:0/930923459 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd40003570 con 0x7fcd400051c0 2023-10-08T00:13:08.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.447+0000 7fcd49d9b700 1 --2- 172.21.15.99:0/930923459 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd40004810 0x7fcd40004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:13:08.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.447+0000 7fcd52d9d700 1 -- 172.21.15.99:0/930923459 reap_dead start 2023-10-08T00:13:08.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.447+0000 7fcd52d9d700 1 --2- 172.21.15.99:0/930923459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd400051c0 0x7fcd4000ba60 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c001f90 tx=0x7fcd4c001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f258092d1a2878ba server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:13:08.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.447+0000 7fcd51d9b700 1 -- 172.21.15.99:0/930923459 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c001950 con 0x7fcd400051c0 2023-10-08T00:13:08.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.447+0000 7fcd51d9b700 1 -- 172.21.15.99:0/930923459 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd4c00c8b0 con 0x7fcd400051c0 2023-10-08T00:13:08.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.447+0000 7fcd51d9b700 1 -- 172.21.15.99:0/930923459 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c00cc50 con 0x7fcd400051c0 2023-10-08T00:13:08.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.448+0000 7fcd5893c700 1 -- 172.21.15.99:0/930923459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd400051c0 msgr2=0x7fcd4000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:08.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.448+0000 7fcd5893c700 1 --2- 172.21.15.99:0/930923459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd400051c0 0x7fcd4000ba60 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c001f90 tx=0x7fcd4c001640 comp rx=0 tx=0).stop 2023-10-08T00:13:08.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.448+0000 7fcd5893c700 1 -- 172.21.15.99:0/930923459 shutdown_connections 2023-10-08T00:13:08.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.448+0000 7fcd5893c700 1 --2- 172.21.15.99:0/930923459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd400051c0 0x7fcd4000ba60 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:08.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.448+0000 7fcd5893c700 1 -- 172.21.15.99:0/930923459 >> 172.21.15.99:0/930923459 conn(0x7fcd54067660 msgr2=0x7fcd54067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:13:08.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.448+0000 7fcd5893c700 1 -- 172.21.15.99:0/930923459 shutdown_connections 2023-10-08T00:13:08.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.448+0000 7fcd5893c700 1 -- 172.21.15.99:0/930923459 wait complete. 2023-10-08T00:13:08.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.449+0000 7fcd5893c700 1 Processor -- start 2023-10-08T00:13:08.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.449+0000 7fcd5893c700 1 -- start start 2023-10-08T00:13:08.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.450+0000 7fcd5893c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd400051c0 0x7fcd54157350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:08.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.450+0000 7fcd5893c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd540c9290 0x7fcd540cece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:08.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.450+0000 7fcd5893c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd540c66f0 0x7fcd540c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:08.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.450+0000 7fcd5893c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd540c5730 con 0x7fcd540c9290 2023-10-08T00:13:08.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.450+0000 7fcd5893c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd540ca0d0 con 0x7fcd540c66f0 2023-10-08T00:13:08.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.450+0000 7fcd5893c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd540c7040 con 0x7fcd400051c0 2023-10-08T00:13:08.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.450+0000 7fcd49d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd540c9290 0x7fcd540cece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:08.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.450+0000 7fcd49d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd540c9290 0x7fcd540cece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46330/0 (socket says 172.21.15.99:46330) 2023-10-08T00:13:08.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.450+0000 7fcd49d9b700 1 -- 172.21.15.99:0/1857468697 learned_addr learned my addr 172.21.15.99:0/1857468697 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:13:08.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.450+0000 7fcd5259c700 1 --2- 172.21.15.99:0/1857468697 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd400051c0 0x7fcd54157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:08.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.450+0000 7fcd52d9d700 1 --2- 172.21.15.99:0/1857468697 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd540c66f0 0x7fcd540c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:08.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.450+0000 7fcd49d9b700 1 -- 172.21.15.99:0/1857468697 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd400051c0 msgr2=0x7fcd54157350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:08.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.450+0000 7fcd49d9b700 1 --2- 172.21.15.99:0/1857468697 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd400051c0 0x7fcd54157350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:08.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.450+0000 7fcd49d9b700 1 -- 172.21.15.99:0/1857468697 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd540c66f0 msgr2=0x7fcd540c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:08.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.450+0000 7fcd49d9b700 1 --2- 172.21.15.99:0/1857468697 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd540c66f0 0x7fcd540c86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:08.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.450+0000 7fcd49d9b700 1 -- 172.21.15.99:0/1857468697 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd540c7320 con 0x7fcd540c9290 2023-10-08T00:13:08.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.450+0000 7fcd52d9d700 1 --2- 172.21.15.99:0/1857468697 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd540c66f0 0x7fcd540c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:13:08.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.450+0000 7fcd49d9b700 1 --2- 172.21.15.99:0/1857468697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd540c9290 0x7fcd540cece0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c00e860 tx=0x7fcd3c000d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:13:08.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.451+0000 7fcd4affd700 1 -- 172.21.15.99:0/1857468697 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c012500 con 0x7fcd540c9290 2023-10-08T00:13:08.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.451+0000 7fcd4affd700 1 -- 172.21.15.99:0/1857468697 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd3c010040 con 0x7fcd540c9290 2023-10-08T00:13:08.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.451+0000 7fcd5893c700 1 -- 172.21.15.99:0/1857468697 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd540c9a80 con 0x7fcd540c9290 2023-10-08T00:13:08.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.451+0000 7fcd4affd700 1 -- 172.21.15.99:0/1857468697 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c0128b0 con 0x7fcd540c9290 2023-10-08T00:13:08.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.451+0000 7fcd5893c700 1 -- 172.21.15.99:0/1857468697 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd54157890 con 0x7fcd540c9290 2023-10-08T00:13:08.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.452+0000 7fcd4affd700 1 -- 172.21.15.99:0/1857468697 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcd3c012c20 con 0x7fcd540c9290 2023-10-08T00:13:08.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.452+0000 7fcd4affd700 1 --2- 172.21.15.99:0/1857468697 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcd34066f30 0x7fcd340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:08.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.452+0000 7fcd4affd700 1 -- 172.21.15.99:0/1857468697 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcd3c083820 con 0x7fcd540c9290 2023-10-08T00:13:08.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.453+0000 7fcd5259c700 1 --2- 172.21.15.99:0/1857468697 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcd34066f30 0x7fcd340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:08.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.453+0000 7fcd5259c700 1 --2- 172.21.15.99:0/1857468697 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcd34066f30 0x7fcd340693e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcd44006f00 tx=0x7fcd440091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:13:08.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.454+0000 7fcd61c34b80 1 Processor -- start 2023-10-08T00:13:08.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.454+0000 7fcd61c34b80 1 -- start start 2023-10-08T00:13:08.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.455+0000 7fcd61c34b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ca4328a50 0x562ca4328e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:08.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.455+0000 7fcd61c34b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ca4372d00 0x562ca4351a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:08.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.455+0000 7fcd61c34b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x562ca4351f60 0x562ca4324cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:08.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.455+0000 7fcd61c34b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ca427fa70 con 0x562ca4372d00 2023-10-08T00:13:08.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.455+0000 7fcd61c34b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ca427d270 con 0x562ca4328a50 2023-10-08T00:13:08.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.455+0000 7fcd61c34b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x562ca4279060 con 0x562ca4351f60 2023-10-08T00:13:08.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.455+0000 7fcd49d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ca4372d00 0x562ca4351a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:08.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.455+0000 7fcd49d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ca4372d00 0x562ca4351a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46338/0 (socket says 172.21.15.99:46338) 2023-10-08T00:13:08.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.455+0000 7fcd49d9b700 1 -- 172.21.15.99:0/875402640 learned_addr learned my addr 172.21.15.99:0/875402640 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:13:08.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.455+0000 7fcd5259c700 1 --2- 172.21.15.99:0/875402640 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ca4328a50 0x562ca4328e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:08.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.455+0000 7fcd52d9d700 1 --2- 172.21.15.99:0/875402640 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x562ca4351f60 0x562ca4324cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:08.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.455+0000 7fcd49d9b700 1 -- 172.21.15.99:0/875402640 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x562ca4351f60 msgr2=0x562ca4324cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:13:08.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.455+0000 7fcd49d9b700 1 --2- 172.21.15.99:0/875402640 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x562ca4351f60 0x562ca4324cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:08.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.455+0000 7fcd49d9b700 1 -- 172.21.15.99:0/875402640 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ca4328a50 msgr2=0x562ca4328e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:13:08.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.455+0000 7fcd49d9b700 1 --2- 172.21.15.99:0/875402640 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ca4328a50 0x562ca4328e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:08.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.455+0000 7fcd49d9b700 1 -- 172.21.15.99:0/875402640 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ca4113270 con 0x562ca4372d00 2023-10-08T00:13:08.474 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.456+0000 7fcd52d9d700 1 --2- 172.21.15.99:0/875402640 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x562ca4351f60 0x562ca4324cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:13:08.474 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.456+0000 7fcd49d9b700 1 --2- 172.21.15.99:0/875402640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ca4372d00 0x562ca4351a20 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c00f010 tx=0x7fcd3c007040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5007eee931c89e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:13:08.474 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.456+0000 7fcd337fe700 1 -- 172.21.15.99:0/875402640 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c05b6f0 con 0x562ca4372d00 2023-10-08T00:13:08.474 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.456+0000 7fcd337fe700 1 -- 172.21.15.99:0/875402640 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd3c052940 con 0x562ca4372d00 2023-10-08T00:13:08.474 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.456+0000 7fcd337fe700 1 -- 172.21.15.99:0/875402640 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c053cf0 con 0x562ca4372d00 2023-10-08T00:13:08.474 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.456+0000 7fcd61c34b80 1 -- 172.21.15.99:0/875402640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ca4372d00 msgr2=0x562ca4351a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:08.474 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.456+0000 7fcd61c34b80 1 --2- 172.21.15.99:0/875402640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ca4372d00 0x562ca4351a20 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c00f010 tx=0x7fcd3c007040 comp rx=0 tx=0).stop 2023-10-08T00:13:08.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.456+0000 7fcd61c34b80 1 -- 172.21.15.99:0/875402640 shutdown_connections 2023-10-08T00:13:08.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.456+0000 7fcd61c34b80 1 --2- 172.21.15.99:0/875402640 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ca4328a50 0x562ca4328e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:08.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.456+0000 7fcd61c34b80 1 --2- 172.21.15.99:0/875402640 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x562ca4351f60 0x562ca4324cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:08.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.456+0000 7fcd61c34b80 1 --2- 172.21.15.99:0/875402640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ca4372d00 0x562ca4351a20 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:08.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.456+0000 7fcd61c34b80 1 -- 172.21.15.99:0/875402640 >> 172.21.15.99:0/875402640 conn(0x562ca432a370 msgr2=0x562ca432a740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:13:08.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.457+0000 7fcd61c34b80 1 -- 172.21.15.99:0/875402640 shutdown_connections 2023-10-08T00:13:08.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.457+0000 7fcd61c34b80 1 -- 172.21.15.99:0/875402640 wait complete. 2023-10-08T00:13:08.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.457+0000 7fcd61c34b80 1 -- start start 2023-10-08T00:13:08.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.457+0000 7fcd61c34b80 1 Processor -- start 2023-10-08T00:13:08.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.458+0000 7fcd61c34b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ca4328a50 0x562ca436f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:08.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.458+0000 7fcd61c34b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ca4351f60 0x562ca437fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:08.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.458+0000 7fcd61c34b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x562ca4372d00 0x562ca4381d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:08.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.458+0000 7fcd61c34b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ca4224890 con 0x562ca4328a50 2023-10-08T00:13:08.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.458+0000 7fcd61c34b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ca423fea0 con 0x562ca4351f60 2023-10-08T00:13:08.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.458+0000 7fcd61c34b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x562ca40fbe30 con 0x562ca4372d00 2023-10-08T00:13:08.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.458+0000 7fcd61c34b80 10 client.0 _ll_register_callbacks cb 0x562ca42ec110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T00:13:08.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.458+0000 7fcd5259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ca4328a50 0x562ca436f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:08.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.458+0000 7fcd5259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ca4328a50 0x562ca436f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46352/0 (socket says 172.21.15.99:46352) 2023-10-08T00:13:08.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.458+0000 7fcd5259c700 1 -- 172.21.15.99:0/1247187030 learned_addr learned my addr 172.21.15.99:0/1247187030 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:13:08.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.458+0000 7fcd52d9d700 1 --2- 172.21.15.99:0/1247187030 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x562ca4372d00 0x562ca4381d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:08.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.458+0000 7fcd49d9b700 1 --2- 172.21.15.99:0/1247187030 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ca4351f60 0x562ca437fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:08.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.458+0000 7fcd5259c700 1 -- 172.21.15.99:0/1247187030 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x562ca4372d00 msgr2=0x562ca4381d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:13:08.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.458+0000 7fcd5259c700 1 --2- 172.21.15.99:0/1247187030 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x562ca4372d00 0x562ca4381d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:08.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.458+0000 7fcd5259c700 1 -- 172.21.15.99:0/1247187030 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ca4351f60 msgr2=0x562ca437fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:13:08.479 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.458+0000 7fcd5259c700 1 --2- 172.21.15.99:0/1247187030 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ca4351f60 0x562ca437fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:08.479 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.458+0000 7fcd5259c700 1 -- 172.21.15.99:0/1247187030 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd54157890 con 0x562ca4328a50 2023-10-08T00:13:08.479 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.459+0000 7fcd5259c700 1 --2- 172.21.15.99:0/1247187030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ca4328a50 0x562ca436f0f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fcd44015ad0 tx=0x7fcd4401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=115e3693bb29d640 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:13:08.479 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.459+0000 7fcd30ff9700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T00:13:08.479 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.459+0000 7fcd30ff9700 1 -- 172.21.15.99:0/1247187030 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd44023650 con 0x562ca4328a50 2023-10-08T00:13:08.479 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.459+0000 7fcd30ff9700 1 -- 172.21.15.99:0/1247187030 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(69..69 src has 1..69) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcd440237b0 con 0x562ca4328a50 2023-10-08T00:13:08.479 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.459+0000 7fcd61c34b80 1 -- 172.21.15.99:0/1247187030 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562ca431db70 con 0x562ca4328a50 2023-10-08T00:13:08.480 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.459+0000 7fcd30ff9700 1 -- 172.21.15.99:0/1247187030 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd44023b80 con 0x562ca4328a50 2023-10-08T00:13:08.480 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.459+0000 7fcd61c34b80 10 client.5847 fetch_fsmap learned FSMap version 32 2023-10-08T00:13:08.480 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.459+0000 7fcd61c34b80 1 -- 172.21.15.99:0/1247187030 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x562ca40b5fe0 con 0x562ca4328a50 2023-10-08T00:13:08.480 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.459+0000 7fcd30ff9700 1 -- 172.21.15.99:0/1247187030 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562ca431db70 con 0x562ca4328a50 2023-10-08T00:13:08.480 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.459+0000 7fcd30ff9700 1 -- 172.21.15.99:0/1247187030 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 32) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fcd44023ce0 con 0x562ca4328a50 2023-10-08T00:13:08.480 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.459+0000 7fcd61c34b80 10 client.5847 fetch_fsmap finished waiting for FSMap version 32 2023-10-08T00:13:08.481 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.459+0000 7fcd61c34b80 10 client.5847 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-08T00:13:08.481 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.459+0000 7fcd61c34b80 20 client.5847 populate_metadata read hostname 'smithi099' 2023-10-08T00:13:08.481 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.459+0000 7fcd61c34b80 1 --2- 172.21.15.99:0/1247187030 >> [v2:172.21.15.191:6835/1779510877,v1:172.21.15.191:6836/1779510877] conn(0x562ca4371f90 0x562ca4372360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:08.481 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.459+0000 7fcd61c34b80 4 client.5847 mds_command: new command op to 5688 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-08T00:13:08.481 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.459+0000 7fcd61c34b80 1 -- 172.21.15.99:0/1247187030 --> [v2:172.21.15.191:6835/1779510877,v1:172.21.15.191:6836/1779510877] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562ca42faaa0 con 0x562ca4371f90 2023-10-08T00:13:08.482 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.459+0000 7fcd49d9b700 1 --2- 172.21.15.99:0/1247187030 >> [v2:172.21.15.191:6835/1779510877,v1:172.21.15.191:6836/1779510877] conn(0x562ca4371f90 0x562ca4372360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:08.482 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.460+0000 7fcd30ff9700 10 client.5847 ms_handle_connect on v2:172.21.15.191:6835/1779510877 2023-10-08T00:13:08.482 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.460+0000 7fcd49d9b700 1 --2- 172.21.15.99:0/1247187030 >> [v2:172.21.15.191:6835/1779510877,v1:172.21.15.191:6836/1779510877] conn(0x562ca4371f90 0x562ca4372360 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c437d9874c818b2 server_cookie=7bcd9878fe75c536 in_seq=0 out_seq=0 2023-10-08T00:13:08.482 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.460+0000 7fcd30ff9700 1 -- 172.21.15.99:0/1247187030 <== mds.0 v2:172.21.15.191:6835/1779510877 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x562ca42faaa0 con 0x562ca4371f90 2023-10-08T00:13:08.482 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.460+0000 7fcd30ff9700 10 client.5847 handle_command_reply: tid=0 2023-10-08T00:13:08.482 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.460+0000 7fcd61c34b80 1 client.5847 shutdown 2023-10-08T00:13:08.483 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.460+0000 7fcd61c34b80 1 -- 172.21.15.99:0/1247187030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ca4328a50 msgr2=0x562ca436f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:08.483 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.460+0000 7fcd61c34b80 1 --2- 172.21.15.99:0/1247187030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ca4328a50 0x562ca436f0f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fcd44015ad0 tx=0x7fcd4401c000 comp rx=0 tx=0).stop 2023-10-08T00:13:08.483 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.461+0000 7fcd61c34b80 1 -- 172.21.15.99:0/1247187030 shutdown_connections 2023-10-08T00:13:08.483 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.461+0000 7fcd61c34b80 1 --2- 172.21.15.99:0/1247187030 >> [v2:172.21.15.191:6835/1779510877,v1:172.21.15.191:6836/1779510877] conn(0x562ca4371f90 0x562ca4372360 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:08.483 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.461+0000 7fcd61c34b80 1 --2- 172.21.15.99:0/1247187030 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x562ca4372d00 0x562ca4381d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:08.483 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.461+0000 7fcd30ff9700 0 client.5847 ms_handle_reset on v2:172.21.15.191:6835/1779510877 2023-10-08T00:13:08.483 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.461+0000 7fcd61c34b80 1 --2- 172.21.15.99:0/1247187030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ca4328a50 0x562ca436f0f0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:08.483 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.461+0000 7fcd61c34b80 1 --2- 172.21.15.99:0/1247187030 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ca4351f60 0x562ca437fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:08.483 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.461+0000 7fcd61c34b80 1 -- 172.21.15.99:0/1247187030 >> 172.21.15.99:0/1247187030 conn(0x562ca432a370 msgr2=0x562ca4324c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:13:08.484 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.461+0000 7fcd61c34b80 1 -- 172.21.15.99:0/1247187030 shutdown_connections 2023-10-08T00:13:08.484 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.461+0000 7fcd61c34b80 1 -- 172.21.15.99:0/1247187030 wait complete. 2023-10-08T00:13:08.484 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.461+0000 7fcd61c34b80 20 client.5847 trim_cache size 0 max 16384 2023-10-08T00:13:08.490 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.477+0000 7fcd61c34b80 1 Processor -- start 2023-10-08T00:13:08.490 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.478+0000 7fcd61c34b80 1 -- start start 2023-10-08T00:13:08.490 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.478+0000 7fcd61c34b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ca4328a50 0x562ca43693e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:08.490 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.478+0000 7fcd61c34b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x562ca4351f60 0x562ca436ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:08.490 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.478+0000 7fcd61c34b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ca4372d00 0x562ca4324b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:08.490 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.478+0000 7fcd61c34b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ca431d8e0 con 0x562ca4328a50 2023-10-08T00:13:08.490 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.478+0000 7fcd61c34b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ca4389420 con 0x562ca4372d00 2023-10-08T00:13:08.490 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.478+0000 7fcd61c34b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x562ca4351300 con 0x562ca4351f60 2023-10-08T00:13:08.491 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.478+0000 7fcd5259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ca4328a50 0x562ca43693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:08.491 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.478+0000 7fcd5259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ca4328a50 0x562ca43693e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46362/0 (socket says 172.21.15.99:46362) 2023-10-08T00:13:08.491 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.478+0000 7fcd5259c700 1 -- 172.21.15.99:0/229747763 learned_addr learned my addr 172.21.15.99:0/229747763 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:13:08.491 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.479+0000 7fcd5259c700 1 -- 172.21.15.99:0/229747763 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x562ca4351f60 msgr2=0x562ca436ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:13:08.491 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.479+0000 7fcd52d9d700 1 --2- 172.21.15.99:0/229747763 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ca4372d00 0x562ca4324b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:08.491 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.479+0000 7fcd49d9b700 1 --2- 172.21.15.99:0/229747763 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x562ca4351f60 0x562ca436ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:08.491 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.479+0000 7fcd5259c700 1 --2- 172.21.15.99:0/229747763 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x562ca4351f60 0x562ca436ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:08.491 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.479+0000 7fcd5259c700 1 -- 172.21.15.99:0/229747763 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ca4372d00 msgr2=0x562ca4324b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:13:08.491 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.479+0000 7fcd5259c700 1 --2- 172.21.15.99:0/229747763 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ca4372d00 0x562ca4324b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:08.491 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.479+0000 7fcd5259c700 1 -- 172.21.15.99:0/229747763 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ca4247780 con 0x562ca4328a50 2023-10-08T00:13:08.492 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.479+0000 7fcd5259c700 1 --2- 172.21.15.99:0/229747763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ca4328a50 0x562ca43693e0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fcd4402c7a0 tx=0x7fcd4401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=555da30c1781a3a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:13:08.492 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.479+0000 7fcd33fff700 1 -- 172.21.15.99:0/229747763 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd44006d90 con 0x562ca4328a50 2023-10-08T00:13:08.492 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.479+0000 7fcd33fff700 1 -- 172.21.15.99:0/229747763 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd44019070 con 0x562ca4328a50 2023-10-08T00:13:08.492 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.479+0000 7fcd33fff700 1 -- 172.21.15.99:0/229747763 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4402c560 con 0x562ca4328a50 2023-10-08T00:13:08.492 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.479+0000 7fcd61c34b80 1 -- 172.21.15.99:0/229747763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ca4328a50 msgr2=0x562ca43693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:08.492 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.479+0000 7fcd61c34b80 1 --2- 172.21.15.99:0/229747763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ca4328a50 0x562ca43693e0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fcd4402c7a0 tx=0x7fcd4401e000 comp rx=0 tx=0).stop 2023-10-08T00:13:08.492 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.479+0000 7fcd61c34b80 1 -- 172.21.15.99:0/229747763 shutdown_connections 2023-10-08T00:13:08.492 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.479+0000 7fcd61c34b80 1 --2- 172.21.15.99:0/229747763 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ca4372d00 0x562ca4324b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:08.492 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.479+0000 7fcd61c34b80 1 --2- 172.21.15.99:0/229747763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ca4328a50 0x562ca43693e0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:08.492 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.479+0000 7fcd61c34b80 1 --2- 172.21.15.99:0/229747763 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x562ca4351f60 0x562ca436ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:08.493 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.479+0000 7fcd61c34b80 1 -- 172.21.15.99:0/229747763 >> 172.21.15.99:0/229747763 conn(0x562ca432a370 msgr2=0x562ca4368c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:13:08.493 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.479+0000 7fcd61c34b80 1 -- 172.21.15.99:0/229747763 shutdown_connections 2023-10-08T00:13:08.493 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.479+0000 7fcd61c34b80 1 -- 172.21.15.99:0/229747763 wait complete. 2023-10-08T00:13:08.493 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.480+0000 7fcd61c34b80 1 -- start start 2023-10-08T00:13:08.493 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.480+0000 7fcd61c34b80 1 Processor -- start 2023-10-08T00:13:08.493 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.481+0000 7fcd61c34b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ca4328a50 0x562ca4368c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:08.493 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.481+0000 7fcd61c34b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x562ca4351f60 0x562ca4369170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:08.493 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.481+0000 7fcd61c34b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ca4372d00 0x562ca43d5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:08.493 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.481+0000 7fcd61c34b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ca43cc320 con 0x562ca4328a50 2023-10-08T00:13:08.493 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.481+0000 7fcd61c34b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ca43d69b0 con 0x562ca4372d00 2023-10-08T00:13:08.493 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.481+0000 7fcd61c34b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x562ca436cea0 con 0x562ca4351f60 2023-10-08T00:13:08.493 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.481+0000 7fcd61c34b80 10 client.0 _ll_register_callbacks cb 0x562ca42ec110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T00:13:08.494 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.481+0000 7fcd5259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ca4328a50 0x562ca4368c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:08.494 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.481+0000 7fcd5259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ca4328a50 0x562ca4368c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46372/0 (socket says 172.21.15.99:46372) 2023-10-08T00:13:08.494 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.481+0000 7fcd5259c700 1 -- 172.21.15.99:0/2091632950 learned_addr learned my addr 172.21.15.99:0/2091632950 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:13:08.494 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.481+0000 7fcd49d9b700 1 --2- 172.21.15.99:0/2091632950 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x562ca4351f60 0x562ca4369170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:08.494 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.481+0000 7fcd52d9d700 1 --2- 172.21.15.99:0/2091632950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ca4372d00 0x562ca43d5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:08.494 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.481+0000 7fcd5259c700 1 -- 172.21.15.99:0/2091632950 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x562ca4351f60 msgr2=0x562ca4369170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:13:08.494 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.481+0000 7fcd5259c700 1 --2- 172.21.15.99:0/2091632950 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x562ca4351f60 0x562ca4369170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:08.494 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.481+0000 7fcd5259c700 1 -- 172.21.15.99:0/2091632950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ca4372d00 msgr2=0x562ca43d5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:13:08.494 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.481+0000 7fcd5259c700 1 --2- 172.21.15.99:0/2091632950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ca4372d00 0x562ca43d5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:08.494 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.481+0000 7fcd5259c700 1 -- 172.21.15.99:0/2091632950 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ca40b5fe0 con 0x562ca4328a50 2023-10-08T00:13:08.494 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.481+0000 7fcd49d9b700 1 --2- 172.21.15.99:0/2091632950 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x562ca4351f60 0x562ca4369170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:13:08.495 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.481+0000 7fcd5259c700 1 --2- 172.21.15.99:0/2091632950 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ca4328a50 0x562ca4368c30 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fcd44014860 tx=0x7fcd4403a070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b065ce311616d2d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:13:08.495 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.481+0000 7fcd30ff9700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T00:13:08.495 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.481+0000 7fcd30ff9700 1 -- 172.21.15.99:0/2091632950 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd44023440 con 0x562ca4328a50 2023-10-08T00:13:08.495 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.482+0000 7fcd30ff9700 1 -- 172.21.15.99:0/2091632950 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(69..69 src has 1..69) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcd440235a0 con 0x562ca4328a50 2023-10-08T00:13:08.495 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.482+0000 7fcd61c34b80 1 -- 172.21.15.99:0/2091632950 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcd3c052940 con 0x562ca4328a50 2023-10-08T00:13:08.495 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.482+0000 7fcd30ff9700 1 -- 172.21.15.99:0/2091632950 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd4402c7a0 con 0x562ca4328a50 2023-10-08T00:13:08.495 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.482+0000 7fcd61c34b80 10 client.5853 fetch_fsmap learned FSMap version 32 2023-10-08T00:13:08.495 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.482+0000 7fcd61c34b80 1 -- 172.21.15.99:0/2091632950 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x562ca422e470 con 0x562ca4328a50 2023-10-08T00:13:08.495 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.482+0000 7fcd30ff9700 1 -- 172.21.15.99:0/2091632950 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd4404d070 con 0x562ca4328a50 2023-10-08T00:13:08.495 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.482+0000 7fcd30ff9700 1 -- 172.21.15.99:0/2091632950 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 32) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fcd4402c900 con 0x562ca4328a50 2023-10-08T00:13:08.495 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.482+0000 7fcd61c34b80 10 client.5853 fetch_fsmap finished waiting for FSMap version 32 2023-10-08T00:13:08.495 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.482+0000 7fcd61c34b80 10 client.5853 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-08T00:13:08.496 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.482+0000 7fcd61c34b80 20 client.5853 populate_metadata read hostname 'smithi099' 2023-10-08T00:13:08.496 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.482+0000 7fcd61c34b80 1 --2- 172.21.15.99:0/2091632950 >> [v2:172.21.15.191:6835/1779510877,v1:172.21.15.191:6836/1779510877] conn(0x562ca43d6570 0x562ca43d5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:08.496 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.482+0000 7fcd61c34b80 4 client.5853 mds_command: new command op to 5688 tid=0[{"prefix": "status"}] 2023-10-08T00:13:08.496 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.482+0000 7fcd61c34b80 1 -- 172.21.15.99:0/2091632950 --> [v2:172.21.15.191:6835/1779510877,v1:172.21.15.191:6836/1779510877] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562ca421be00 con 0x562ca43d6570 2023-10-08T00:13:08.496 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.482+0000 7fcd49d9b700 1 --2- 172.21.15.99:0/2091632950 >> [v2:172.21.15.191:6835/1779510877,v1:172.21.15.191:6836/1779510877] conn(0x562ca43d6570 0x562ca43d5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:08.496 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.482+0000 7fcd30ff9700 10 client.5853 ms_handle_connect on v2:172.21.15.191:6835/1779510877 2023-10-08T00:13:08.496 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.482+0000 7fcd49d9b700 1 --2- 172.21.15.99:0/2091632950 >> [v2:172.21.15.191:6835/1779510877,v1:172.21.15.191:6836/1779510877] conn(0x562ca43d6570 0x562ca43d5b90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3dc312a2ad179d2 server_cookie=818d9426e5bed70f in_seq=0 out_seq=0 2023-10-08T00:13:08.496 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.483+0000 7fcd30ff9700 1 -- 172.21.15.99:0/2091632950 <== mds.0 v2:172.21.15.191:6835/1779510877 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+319 (crc 0 0 0) 0x562ca421be00 con 0x562ca43d6570 2023-10-08T00:13:08.496 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.483+0000 7fcd30ff9700 10 client.5853 handle_command_reply: tid=0 2023-10-08T00:13:08.496 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.483+0000 7fcd61c34b80 1 client.5853 shutdown 2023-10-08T00:13:08.496 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.483+0000 7fcd61c34b80 1 -- 172.21.15.99:0/2091632950 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ca4328a50 msgr2=0x562ca4368c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:08.496 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.483+0000 7fcd61c34b80 1 --2- 172.21.15.99:0/2091632950 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ca4328a50 0x562ca4368c30 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fcd44014860 tx=0x7fcd4403a070 comp rx=0 tx=0).stop 2023-10-08T00:13:08.497 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.483+0000 7fcd61c34b80 1 -- 172.21.15.99:0/2091632950 shutdown_connections 2023-10-08T00:13:08.497 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.483+0000 7fcd61c34b80 1 --2- 172.21.15.99:0/2091632950 >> [v2:172.21.15.191:6835/1779510877,v1:172.21.15.191:6836/1779510877] conn(0x562ca43d6570 0x562ca43d5b90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:08.497 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.483+0000 7fcd61c34b80 1 --2- 172.21.15.99:0/2091632950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ca4372d00 0x562ca43d5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:08.497 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.483+0000 7fcd61c34b80 1 --2- 172.21.15.99:0/2091632950 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ca4328a50 0x562ca4368c30 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:08.497 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.483+0000 7fcd61c34b80 1 --2- 172.21.15.99:0/2091632950 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x562ca4351f60 0x562ca4369170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:08.497 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.483+0000 7fcd30ff9700 0 client.5853 ms_handle_reset on v2:172.21.15.191:6835/1779510877 2023-10-08T00:13:08.497 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.483+0000 7fcd61c34b80 1 -- 172.21.15.99:0/2091632950 >> 172.21.15.99:0/2091632950 conn(0x562ca432a370 msgr2=0x562ca436e790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:13:08.497 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.483+0000 7fcd61c34b80 1 -- 172.21.15.99:0/2091632950 shutdown_connections 2023-10-08T00:13:08.497 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.483+0000 7fcd61c34b80 1 -- 172.21.15.99:0/2091632950 wait complete. 2023-10-08T00:13:08.497 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.483+0000 7fcd61c34b80 20 client.5853 trim_cache size 0 max 16384 2023-10-08T00:13:08.498 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:13:08.498 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", 2023-10-08T00:13:08.498 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 0, 2023-10-08T00:13:08.498 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 5688, 2023-10-08T00:13:08.498 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2023-10-08T00:13:08.498 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2023-10-08T00:13:08.498 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2023-10-08T00:13:08.498 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 10.262501793, 2023-10-08T00:13:08.499 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 32, 2023-10-08T00:13:08.499 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 69, 2023-10-08T00:13:08.499 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 69, 2023-10-08T00:13:08.499 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 16.248900098 2023-10-08T00:13:08.499 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:13:08.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.483+0000 7fcd5893c700 1 -- 172.21.15.99:0/1857468697 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcd34066f30 msgr2=0x7fcd340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:08.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.483+0000 7fcd5893c700 1 --2- 172.21.15.99:0/1857468697 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcd34066f30 0x7fcd340693e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcd44006f00 tx=0x7fcd440091a0 comp rx=0 tx=0).stop 2023-10-08T00:13:08.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.483+0000 7fcd5893c700 1 -- 172.21.15.99:0/1857468697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd540c9290 msgr2=0x7fcd540cece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:08.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.483+0000 7fcd5893c700 1 --2- 172.21.15.99:0/1857468697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd540c9290 0x7fcd540cece0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c00e860 tx=0x7fcd3c000d70 comp rx=0 tx=0).stop 2023-10-08T00:13:08.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.483+0000 7fcd5893c700 1 -- 172.21.15.99:0/1857468697 shutdown_connections 2023-10-08T00:13:08.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.483+0000 7fcd5893c700 1 --2- 172.21.15.99:0/1857468697 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcd34066f30 0x7fcd340693e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:08.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.483+0000 7fcd5893c700 1 --2- 172.21.15.99:0/1857468697 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd540c66f0 0x7fcd540c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:08.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.483+0000 7fcd5893c700 1 --2- 172.21.15.99:0/1857468697 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd400051c0 0x7fcd54157350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:08.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.483+0000 7fcd5893c700 1 --2- 172.21.15.99:0/1857468697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd540c9290 0x7fcd540cece0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:08.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.483+0000 7fcd5893c700 1 -- 172.21.15.99:0/1857468697 >> 172.21.15.99:0/1857468697 conn(0x7fcd54067660 msgr2=0x7fcd5415a660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:13:08.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.484+0000 7fcd5893c700 1 -- 172.21.15.99:0/1857468697 shutdown_connections 2023-10-08T00:13:08.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.484+0000 7fcd5893c700 1 -- 172.21.15.99:0/1857468697 wait complete. 2023-10-08T00:13:08.501 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T00:13:08.616 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.615+0000 7fe7f18e4700 1 Processor -- start 2023-10-08T00:13:08.616 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.615+0000 7fe7f18e4700 1 -- start start 2023-10-08T00:13:08.618 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.616+0000 7fe7f18e4700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe7ec0b1220 0x7fe7ec0c9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:08.618 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.616+0000 7fe7f18e4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe7ec0cb4f0 0x7fe7ec0cb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:08.618 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.616+0000 7fe7f18e4700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7ec15de60 con 0x7fe7ec0b0e50 2023-10-08T00:13:08.618 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.616+0000 7fe7f18e4700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7ec0cbed0 con 0x7fe7ec0b1220 2023-10-08T00:13:08.619 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.616+0000 7fe7f18e4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ec15e140 con 0x7fe7ec0cb4f0 2023-10-08T00:13:08.619 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.616+0000 7fe7e27fc700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe7ec0b1220 0x7fe7ec0c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:08.619 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.616+0000 7fe7e27fc700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe7ec0b1220 0x7fe7ec0c9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46374/0 (socket says 172.21.15.99:46374) 2023-10-08T00:13:08.619 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.616+0000 7fe7e27fc700 1 -- 172.21.15.99:0/3256340783 learned_addr learned my addr 172.21.15.99:0/3256340783 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:13:08.619 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.616+0000 7fe7eb7fe700 1 --2- 172.21.15.99:0/3256340783 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe7ec0cb4f0 0x7fe7ec0cb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:08.619 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.617+0000 7fe7e27fc700 1 -- 172.21.15.99:0/3256340783 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe7ec0cb4f0 msgr2=0x7fe7ec0cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:13:08.619 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.617+0000 7fe7e27fc700 1 --2- 172.21.15.99:0/3256340783 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe7ec0cb4f0 0x7fe7ec0cb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:08.619 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.617+0000 7fe7e27fc700 1 -- 172.21.15.99:0/3256340783 >> v1:172.21.15.191:6789/0 conn(0x7fe7ec0b0e50 legacy=0x7fe7ec06d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:08.619 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.617+0000 7fe7e27fc700 1 -- 172.21.15.99:0/3256340783 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7ec15e420 con 0x7fe7ec0b1220 2023-10-08T00:13:08.619 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.617+0000 7fe7e27fc700 1 --2- 172.21.15.99:0/3256340783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7ec0b1220 0x7fe7ec0c9180 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fe7d40099b0 tx=0x7fe7d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58e2241169d4c354 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:13:08.619 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.617+0000 7fe7ea7fc700 1 -- 172.21.15.99:0/3256340783 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d4009ea0 con 0x7fe7ec0b1220 2023-10-08T00:13:08.620 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.617+0000 7fe7ea7fc700 1 -- 172.21.15.99:0/3256340783 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7d400b860 con 0x7fe7ec0b1220 2023-10-08T00:13:08.620 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.617+0000 7fe7ea7fc700 1 -- 172.21.15.99:0/3256340783 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d400bc20 con 0x7fe7ec0b1220 2023-10-08T00:13:08.620 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.617+0000 7fe7f18e4700 1 -- 172.21.15.99:0/3256340783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7ec0b1220 msgr2=0x7fe7ec0c9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:08.620 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.617+0000 7fe7f18e4700 1 --2- 172.21.15.99:0/3256340783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7ec0b1220 0x7fe7ec0c9180 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fe7d40099b0 tx=0x7fe7d4009250 comp rx=0 tx=0).stop 2023-10-08T00:13:08.620 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.617+0000 7fe7f18e4700 1 -- 172.21.15.99:0/3256340783 shutdown_connections 2023-10-08T00:13:08.620 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.617+0000 7fe7f18e4700 1 --2- 172.21.15.99:0/3256340783 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe7ec0cb4f0 0x7fe7ec0cb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:08.620 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.617+0000 7fe7f18e4700 1 --2- 172.21.15.99:0/3256340783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7ec0b1220 0x7fe7ec0c9180 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:08.620 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.617+0000 7fe7f18e4700 1 -- 172.21.15.99:0/3256340783 >> 172.21.15.99:0/3256340783 conn(0x7fe7ec069870 msgr2=0x7fe7ec069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:13:08.620 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.617+0000 7fe7f18e4700 1 -- 172.21.15.99:0/3256340783 shutdown_connections 2023-10-08T00:13:08.620 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.617+0000 7fe7f18e4700 1 -- 172.21.15.99:0/3256340783 wait complete. 2023-10-08T00:13:08.623 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.619+0000 7fe7f18e4700 1 Processor -- start 2023-10-08T00:13:08.623 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.619+0000 7fe7f18e4700 1 -- start start 2023-10-08T00:13:08.623 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.619+0000 7fe7f18e4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7ec0b0e50 0x7fe7ec151790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:08.623 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.619+0000 7fe7f18e4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7ec0b1220 0x7fe7ec1549f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:08.623 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.619+0000 7fe7f18e4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe7ec0cb4f0 0x7fe7ec154f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:08.623 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.619+0000 7fe7f18e4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ec0b2c30 con 0x7fe7ec0b1220 2023-10-08T00:13:08.623 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.619+0000 7fe7f18e4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ec16b2a0 con 0x7fe7ec0b0e50 2023-10-08T00:13:08.623 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.619+0000 7fe7f18e4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ec16a410 con 0x7fe7ec0cb4f0 2023-10-08T00:13:08.623 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.619+0000 7fe7e27fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7ec0b1220 0x7fe7ec1549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:08.623 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.619+0000 7fe7e27fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7ec0b1220 0x7fe7ec1549f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46380/0 (socket says 172.21.15.99:46380) 2023-10-08T00:13:08.623 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.619+0000 7fe7e27fc700 1 -- 172.21.15.99:0/2681618262 learned_addr learned my addr 172.21.15.99:0/2681618262 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:13:08.624 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.619+0000 7fe7eb7fe700 1 --2- 172.21.15.99:0/2681618262 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe7ec0cb4f0 0x7fe7ec154f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:08.624 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.619+0000 7fe7eaffd700 1 --2- 172.21.15.99:0/2681618262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7ec0b0e50 0x7fe7ec151790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:08.624 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.620+0000 7fe7e27fc700 1 -- 172.21.15.99:0/2681618262 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe7ec0cb4f0 msgr2=0x7fe7ec154f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:08.624 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.620+0000 7fe7e27fc700 1 --2- 172.21.15.99:0/2681618262 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe7ec0cb4f0 0x7fe7ec154f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:08.624 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.620+0000 7fe7e27fc700 1 -- 172.21.15.99:0/2681618262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7ec0b0e50 msgr2=0x7fe7ec151790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:08.624 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.620+0000 7fe7e27fc700 1 --2- 172.21.15.99:0/2681618262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7ec0b0e50 0x7fe7ec151790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:08.624 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.620+0000 7fe7e27fc700 1 -- 172.21.15.99:0/2681618262 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7d40094f0 con 0x7fe7ec0b1220 2023-10-08T00:13:08.624 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.620+0000 7fe7eaffd700 1 --2- 172.21.15.99:0/2681618262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7ec0b0e50 0x7fe7ec151790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:13:08.624 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.620+0000 7fe7e27fc700 1 --2- 172.21.15.99:0/2681618262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7ec0b1220 0x7fe7ec1549f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4006fd0 tx=0x7fe7d4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:13:08.624 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.620+0000 7fe7e37fe700 1 -- 172.21.15.99:0/2681618262 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d401f070 con 0x7fe7ec0b1220 2023-10-08T00:13:08.625 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.620+0000 7fe7e37fe700 1 -- 172.21.15.99:0/2681618262 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7d4000bc0 con 0x7fe7ec0b1220 2023-10-08T00:13:08.625 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.620+0000 7fe7f18e4700 1 -- 172.21.15.99:0/2681618262 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7ec16a690 con 0x7fe7ec0b1220 2023-10-08T00:13:08.625 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.620+0000 7fe7e37fe700 1 -- 172.21.15.99:0/2681618262 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d4025420 con 0x7fe7ec0b1220 2023-10-08T00:13:08.625 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.620+0000 7fe7f18e4700 1 -- 172.21.15.99:0/2681618262 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7ec16acd0 con 0x7fe7ec0b1220 2023-10-08T00:13:08.625 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.621+0000 7fe7f18e4700 1 -- 172.21.15.99:0/2681618262 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7ec16b3e0 con 0x7fe7ec0b1220 2023-10-08T00:13:08.625 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.621+0000 7fe7e37fe700 1 -- 172.21.15.99:0/2681618262 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe7d4023070 con 0x7fe7ec0b1220 2023-10-08T00:13:08.625 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.622+0000 7fe7e37fe700 1 --2- 172.21.15.99:0/2681618262 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe7cc066f30 0x7fe7cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:08.625 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.622+0000 7fe7e37fe700 1 -- 172.21.15.99:0/2681618262 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe7d4085850 con 0x7fe7ec0b1220 2023-10-08T00:13:08.625 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.622+0000 7fe7eaffd700 1 --2- 172.21.15.99:0/2681618262 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe7cc066f30 0x7fe7cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:08.626 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.622+0000 7fe7eaffd700 1 --2- 172.21.15.99:0/2681618262 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe7cc066f30 0x7fe7cc0693e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc007fd0 tx=0x7fe7dc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:13:08.626 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.626+0000 7fe7e37fe700 1 -- 172.21.15.99:0/2681618262 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe7ec16b3e0 con 0x7fe7ec0b1220 2023-10-08T00:13:08.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.852+0000 7fe7f18e4700 1 -- 172.21.15.99:0/2681618262 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe7ec16b3e0 con 0x7fe7ec0b1220 2023-10-08T00:13:08.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.852+0000 7fe7e37fe700 1 -- 172.21.15.99:0/2681618262 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v32) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe7ec16b3e0 con 0x7fe7ec0b1220 2023-10-08T00:13:08.853 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T00:13:08.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.854+0000 7fe7f18e4700 1 -- 172.21.15.99:0/2681618262 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe7cc066f30 msgr2=0x7fe7cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:08.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.854+0000 7fe7f18e4700 1 --2- 172.21.15.99:0/2681618262 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe7cc066f30 0x7fe7cc0693e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc007fd0 tx=0x7fe7dc009040 comp rx=0 tx=0).stop 2023-10-08T00:13:08.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.854+0000 7fe7f18e4700 1 -- 172.21.15.99:0/2681618262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7ec0b1220 msgr2=0x7fe7ec1549f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:08.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.854+0000 7fe7f18e4700 1 --2- 172.21.15.99:0/2681618262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7ec0b1220 0x7fe7ec1549f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4006fd0 tx=0x7fe7d4017000 comp rx=0 tx=0).stop 2023-10-08T00:13:08.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.854+0000 7fe7f18e4700 1 -- 172.21.15.99:0/2681618262 shutdown_connections 2023-10-08T00:13:08.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.854+0000 7fe7f18e4700 1 --2- 172.21.15.99:0/2681618262 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe7cc066f30 0x7fe7cc0693e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:08.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.854+0000 7fe7f18e4700 1 --2- 172.21.15.99:0/2681618262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7ec0b0e50 0x7fe7ec151790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:08.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.854+0000 7fe7f18e4700 1 --2- 172.21.15.99:0/2681618262 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe7ec0cb4f0 0x7fe7ec154f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:08.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.854+0000 7fe7f18e4700 1 --2- 172.21.15.99:0/2681618262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7ec0b1220 0x7fe7ec1549f0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:08.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.854+0000 7fe7f18e4700 1 -- 172.21.15.99:0/2681618262 >> 172.21.15.99:0/2681618262 conn(0x7fe7ec069870 msgr2=0x7fe7ec06d060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:13:08.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.854+0000 7fe7f18e4700 1 -- 172.21.15.99:0/2681618262 shutdown_connections 2023-10-08T00:13:08.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:08.854+0000 7fe7f18e4700 1 -- 172.21.15.99:0/2681618262 wait complete. 2023-10-08T00:13:08.867 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T00:13:08.867 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T00:13:08.867 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi099.front.sepia.ceph.com 2023-10-08T00:13:08.867 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T00:13:08.867 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T00:13:08.867 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T00:13:08.867 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T00:13:08.867 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T00:13:08.867 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T00:13:08.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:13:08.868 DEBUG:teuthology.orchestra.run.smithi099:> ip addr 2023-10-08T00:13:08.882 INFO:teuthology.orchestra.run.smithi099.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T00:13:08.883 INFO:teuthology.orchestra.run.smithi099.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T00:13:08.883 INFO:teuthology.orchestra.run.smithi099.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T00:13:08.883 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:13:08.883 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 ::1/128 scope host 2023-10-08T00:13:08.883 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:13:08.883 INFO:teuthology.orchestra.run.smithi099.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:13:08.883 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:10 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:13:08.883 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f0 2023-10-08T00:13:08.884 INFO:teuthology.orchestra.run.smithi099.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:13:08.884 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:11 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:13:08.884 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f1 2023-10-08T00:13:08.884 INFO:teuthology.orchestra.run.smithi099.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:13:08.884 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:02 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:13:08.884 INFO:teuthology.orchestra.run.smithi099.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T00:13:08.884 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:03 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:13:08.884 INFO:teuthology.orchestra.run.smithi099.stdout: inet 172.21.15.99/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T00:13:08.884 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft 42401sec preferred_lft 42401sec 2023-10-08T00:13:08.884 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::ec4:7aff:fe8f:d803/64 scope link 2023-10-08T00:13:08.884 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:13:08.885 INFO:teuthology.orchestra.run.smithi099.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T00:13:08.885 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 22:2c:1c:f6:42:e1 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:13:08.885 INFO:teuthology.orchestra.run.smithi099.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T00:13:08.885 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:13:08.885 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::600d:26ff:fede:5eec/64 scope link 2023-10-08T00:13:08.885 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:13:08.885 INFO:teuthology.orchestra.run.smithi099.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T00:13:08.885 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether ae:2f:b5:94:69:ba brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T00:13:08.885 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::ac2f:b5ff:fe94:69ba/64 scope link 2023-10-08T00:13:08.886 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:13:08.886 INFO:teuthology.orchestra.run.smithi099.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T00:13:08.886 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 22:2c:1c:f6:42:e1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T00:13:08.886 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::202c:1cff:fef6:42e1/64 scope link 2023-10-08T00:13:08.886 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:13:08.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:13:08.886 DEBUG:teuthology.orchestra.run.smithi099:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T00:13:08.946 INFO:teuthology.orchestra.run.smithi099.stdout:1 2023-10-08T00:13:08.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:13:08.947 DEBUG:teuthology.orchestra.run.smithi099:> route 2023-10-08T00:13:09.003 INFO:teuthology.orchestra.run.smithi099.stdout:Kernel IP routing table 2023-10-08T00:13:09.003 INFO:teuthology.orchestra.run.smithi099.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T00:13:09.003 INFO:teuthology.orchestra.run.smithi099.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T00:13:09.003 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T00:13:09.003 INFO:teuthology.orchestra.run.smithi099.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T00:13:09.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:13:09.004 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2023-10-08T00:13:09.004 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2023-10-08T00:13:09.004 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T00:13:09.004 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T00:13:09.004 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T00:13:09.004 DEBUG:teuthology.orchestra.run.smithi099:> ') 2023-10-08T00:13:09.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:13:09.103 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2023-10-08T00:13:09.117 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T00:13:09.117 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T00:13:09.118 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T00:13:09.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:13:09.118 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set brx.0 up 2023-10-08T00:13:09.182 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:13:09.182 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T00:13:09.237 INFO:teuthology.orchestra.run.smithi099.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T00:13:09.237 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:13:09.237 DEBUG:teuthology.orchestra.run.smithi099:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T00:13:09.292 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-08T00:13:09.292 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T00:13:09.292 DEBUG:teuthology.orchestra.run.smithi099:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:13:09.372 INFO:teuthology.orchestra.run.smithi099.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:13:09.372 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:13:09.390 INFO:teuthology.orchestra.run.smithi099.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:13:09.390 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:13:09.391 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: resolved to: "172.21.15.99:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T00:13:09.391 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: trying mount with new device syntax: 0@06962cdb-9a6f-417f-82fd-9e4e3dedbccf.cephfs=/ 2023-10-08T00:13:09.391 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.99:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T00:13:09.391 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T00:13:09.391 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T00:13:09.391 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T00:13:09.412 INFO:teuthology.orchestra.run.smithi099.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:13:09.412 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:13:09.412 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: resolved to: "172.21.15.99:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T00:13:09.412 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: trying mount with new device syntax: 0@06962cdb-9a6f-417f-82fd-9e4e3dedbccf.cephfs=/ 2023-10-08T00:13:09.412 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.99:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T00:13:09.413 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T00:13:09.413 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T00:13:09.413 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T00:13:09.413 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: resolved to: "172.21.15.99:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T00:13:09.413 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.99:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T00:13:09.413 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=06962cdb-9a6f-417f-82fd-9e4e3dedbccf" will pass to kernel 2023-10-08T00:13:09.414 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T00:13:09.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:13:09.415 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T00:13:09.480 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T00:13:09.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.633+0000 7f97c759e700 1 Processor -- start 2023-10-08T00:13:09.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.633+0000 7f97c759e700 1 -- start start 2023-10-08T00:13:09.641 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.634+0000 7f97c759e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f97c80cd6e0 0x7f97c80cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:09.642 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.634+0000 7f97c759e700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97c80cb7c0 con 0x7f97c80cdab0 2023-10-08T00:13:09.642 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.634+0000 7f97c759e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97c815e290 con 0x7f97c815cbd0 2023-10-08T00:13:09.642 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.634+0000 7f97c759e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97c815e850 con 0x7f97c80cd6e0 2023-10-08T00:13:09.642 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.634+0000 7f97bffff700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f97c80cdab0 0x7f97c815d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:47532/0 (socket says 172.21.15.99:47532) 2023-10-08T00:13:09.642 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.634+0000 7f97bffff700 1 -- 172.21.15.99:0/4105059513 learned_addr learned my addr 172.21.15.99:0/4105059513 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:13:09.642 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.634+0000 7f97c659c700 1 --2- 172.21.15.99:0/4105059513 >> v2:172.21.15.191:3300/0 conn(0x7f97c80cd6e0 0x7f97c80cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:09.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.634+0000 7f97c5d9b700 1 -- 172.21.15.99:0/4105059513 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2241893846 0 0) 0x7f97c80cb7c0 con 0x7f97c80cdab0 2023-10-08T00:13:09.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.634+0000 7f97c5d9b700 1 -- 172.21.15.99:0/4105059513 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97b4003570 con 0x7f97c80cdab0 2023-10-08T00:13:09.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.635+0000 7f97c659c700 1 -- 172.21.15.99:0/4105059513 >> v1:172.21.15.191:6789/0 conn(0x7f97c815cbd0 legacy=0x7f97c815d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:09.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.635+0000 7f97c659c700 1 -- 172.21.15.99:0/4105059513 >> v1:172.21.15.99:6789/0 conn(0x7f97c80cdab0 legacy=0x7f97c815d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:09.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.635+0000 7f97c659c700 1 -- 172.21.15.99:0/4105059513 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c815e570 con 0x7f97c80cd6e0 2023-10-08T00:13:09.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.635+0000 7f97c659c700 1 --2- 172.21.15.99:0/4105059513 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f97c80cd6e0 0x7f97c80cbc50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f97b80098e0 tx=0x7f97b80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86716eb968754392 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:13:09.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.635+0000 7f97c5d9b700 1 -- 172.21.15.99:0/4105059513 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b800b640 con 0x7f97c80cd6e0 2023-10-08T00:13:09.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.635+0000 7f97c5d9b700 1 -- 172.21.15.99:0/4105059513 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97b800b7a0 con 0x7f97c80cd6e0 2023-10-08T00:13:09.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.635+0000 7f97c5d9b700 1 -- 172.21.15.99:0/4105059513 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b800bb40 con 0x7f97c80cd6e0 2023-10-08T00:13:09.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.635+0000 7f97c759e700 1 -- 172.21.15.99:0/4105059513 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f97c80cd6e0 msgr2=0x7f97c80cbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:09.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.635+0000 7f97c759e700 1 --2- 172.21.15.99:0/4105059513 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f97c80cd6e0 0x7f97c80cbc50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f97b80098e0 tx=0x7f97b80091a0 comp rx=0 tx=0).stop 2023-10-08T00:13:09.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.635+0000 7f97c759e700 1 -- 172.21.15.99:0/4105059513 shutdown_connections 2023-10-08T00:13:09.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.635+0000 7f97c759e700 1 --2- 172.21.15.99:0/4105059513 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f97c80cd6e0 0x7f97c80cbc50 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:09.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.635+0000 7f97c759e700 1 -- 172.21.15.99:0/4105059513 >> 172.21.15.99:0/4105059513 conn(0x7f97c8067660 msgr2=0x7f97c8067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:13:09.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.636+0000 7f97c759e700 1 -- 172.21.15.99:0/4105059513 shutdown_connections 2023-10-08T00:13:09.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.636+0000 7f97c759e700 1 -- 172.21.15.99:0/4105059513 wait complete. 2023-10-08T00:13:09.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.637+0000 7f97c759e700 1 Processor -- start 2023-10-08T00:13:09.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.637+0000 7f97c759e700 1 -- start start 2023-10-08T00:13:09.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.637+0000 7f97c759e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97c80cd6e0 0x7f97c80c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:09.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.637+0000 7f97c759e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f97c80cdab0 0x7f97c814cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:09.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.637+0000 7f97c759e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f97c815cbd0 0x7f97c80c6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:09.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.637+0000 7f97c759e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c80c8c20 con 0x7f97c80cd6e0 2023-10-08T00:13:09.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.637+0000 7f97c759e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c80c9430 con 0x7f97c80cdab0 2023-10-08T00:13:09.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.637+0000 7f97c759e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97c80c9710 con 0x7f97c815cbd0 2023-10-08T00:13:09.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.637+0000 7f97c659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97c80cd6e0 0x7f97c80c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:09.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.637+0000 7f97c659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97c80cd6e0 0x7f97c80c86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46384/0 (socket says 172.21.15.99:46384) 2023-10-08T00:13:09.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.637+0000 7f97c659c700 1 -- 172.21.15.99:0/347991573 learned_addr learned my addr 172.21.15.99:0/347991573 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:13:09.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.638+0000 7f97c659c700 1 -- 172.21.15.99:0/347991573 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f97c815cbd0 msgr2=0x7f97c80c6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:09.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.638+0000 7f97c659c700 1 --2- 172.21.15.99:0/347991573 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f97c815cbd0 0x7f97c80c6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:09.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.638+0000 7f97c659c700 1 -- 172.21.15.99:0/347991573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f97c80cdab0 msgr2=0x7f97c814cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:09.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.638+0000 7f97c659c700 1 --2- 172.21.15.99:0/347991573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f97c80cdab0 0x7f97c814cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:09.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.638+0000 7f97c659c700 1 -- 172.21.15.99:0/347991573 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b8009420 con 0x7f97c80cd6e0 2023-10-08T00:13:09.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.638+0000 7f97c659c700 1 --2- 172.21.15.99:0/347991573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97c80cd6e0 0x7f97c80c86e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f97c80c7b10 tx=0x7f97b8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:13:09.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.638+0000 7f97be7fc700 1 -- 172.21.15.99:0/347991573 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b800bb20 con 0x7f97c80cd6e0 2023-10-08T00:13:09.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.638+0000 7f97be7fc700 1 -- 172.21.15.99:0/347991573 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97b8011420 con 0x7f97c80cd6e0 2023-10-08T00:13:09.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.638+0000 7f97be7fc700 1 -- 172.21.15.99:0/347991573 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b80117a0 con 0x7f97c80cd6e0 2023-10-08T00:13:09.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.638+0000 7f97c759e700 1 -- 172.21.15.99:0/347991573 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97c80c99f0 con 0x7f97c80cd6e0 2023-10-08T00:13:09.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.638+0000 7f97c759e700 1 -- 172.21.15.99:0/347991573 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97c80c9100 con 0x7f97c80cd6e0 2023-10-08T00:13:09.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.639+0000 7f97c759e700 1 -- 172.21.15.99:0/347991573 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97c80ca000 con 0x7f97c80cd6e0 2023-10-08T00:13:09.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.639+0000 7f97be7fc700 1 -- 172.21.15.99:0/347991573 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f97b8011b50 con 0x7f97c80cd6e0 2023-10-08T00:13:09.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.640+0000 7f97be7fc700 1 --2- 172.21.15.99:0/347991573 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f97a8066f30 0x7f97a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:09.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.640+0000 7f97be7fc700 1 -- 172.21.15.99:0/347991573 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f97b8085c30 con 0x7f97c80cd6e0 2023-10-08T00:13:09.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.640+0000 7f97bffff700 1 --2- 172.21.15.99:0/347991573 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f97a8066f30 0x7f97a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:09.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.640+0000 7f97bffff700 1 --2- 172.21.15.99:0/347991573 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f97a8066f30 0x7f97a80693e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f97b00058d0 tx=0x7f97b00079d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:13:09.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.644+0000 7f97be7fc700 1 -- 172.21.15.99:0/347991573 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f97b8052ea0 con 0x7f97c80cd6e0 2023-10-08T00:13:09.869 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.868+0000 7f97c759e700 1 -- 172.21.15.99:0/347991573 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f97c804bb80 con 0x7f97c80cd6e0 2023-10-08T00:13:09.869 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.868+0000 7f97be7fc700 1 -- 172.21.15.99:0/347991573 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v32) v1 ==== 53+0+73 (secure 0 0 0) 0x7f97b80564c0 con 0x7f97c80cd6e0 2023-10-08T00:13:09.869 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T00:13:09.872 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.871+0000 7f97c759e700 1 -- 172.21.15.99:0/347991573 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f97a8066f30 msgr2=0x7f97a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:09.872 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.871+0000 7f97c759e700 1 --2- 172.21.15.99:0/347991573 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f97a8066f30 0x7f97a80693e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f97b00058d0 tx=0x7f97b00079d0 comp rx=0 tx=0).stop 2023-10-08T00:13:09.872 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.871+0000 7f97c759e700 1 -- 172.21.15.99:0/347991573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97c80cd6e0 msgr2=0x7f97c80c86e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:09.872 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.871+0000 7f97c759e700 1 --2- 172.21.15.99:0/347991573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97c80cd6e0 0x7f97c80c86e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f97c80c7b10 tx=0x7f97b8017000 comp rx=0 tx=0).stop 2023-10-08T00:13:09.872 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.871+0000 7f97c759e700 1 -- 172.21.15.99:0/347991573 shutdown_connections 2023-10-08T00:13:09.872 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.871+0000 7f97c759e700 1 --2- 172.21.15.99:0/347991573 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f97a8066f30 0x7f97a80693e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:09.872 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.871+0000 7f97c759e700 1 --2- 172.21.15.99:0/347991573 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f97c815cbd0 0x7f97c80c6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:09.873 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.871+0000 7f97c759e700 1 --2- 172.21.15.99:0/347991573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97c80cd6e0 0x7f97c80c86e0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:09.873 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.871+0000 7f97c759e700 1 --2- 172.21.15.99:0/347991573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f97c80cdab0 0x7f97c814cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:09.873 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.871+0000 7f97c759e700 1 -- 172.21.15.99:0/347991573 >> 172.21.15.99:0/347991573 conn(0x7f97c8067660 msgr2=0x7f97c80b07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:13:09.873 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.871+0000 7f97c759e700 1 -- 172.21.15.99:0/347991573 shutdown_connections 2023-10-08T00:13:09.873 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:09.871+0000 7f97c759e700 1 -- 172.21.15.99:0/347991573 wait complete. 2023-10-08T00:13:09.883 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T00:13:09.883 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T00:13:09.883 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi191.front.sepia.ceph.com 2023-10-08T00:13:09.883 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-08T00:13:09.884 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-08T00:13:09.884 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T00:13:09.884 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T00:13:09.884 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T00:13:09.884 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T00:13:09.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:13:09.884 DEBUG:teuthology.orchestra.run.smithi191:> ip addr 2023-10-08T00:13:09.903 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T00:13:09.903 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T00:13:09.903 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T00:13:09.903 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:13:09.903 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2023-10-08T00:13:09.903 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:13:09.904 INFO:teuthology.orchestra.run.smithi191.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:13:09.904 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f6 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:13:09.904 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f0 2023-10-08T00:13:09.904 INFO:teuthology.orchestra.run.smithi191.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:13:09.904 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f7 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:13:09.904 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f1 2023-10-08T00:13:09.904 INFO:teuthology.orchestra.run.smithi191.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:13:09.904 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:10 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:13:09.904 INFO:teuthology.orchestra.run.smithi191.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T00:13:09.904 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:11 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:13:09.904 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.21.15.191/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T00:13:09.904 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft 42402sec preferred_lft 42402sec 2023-10-08T00:13:09.905 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec4:7aff:fe88:7211/64 scope link 2023-10-08T00:13:09.905 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:13:09.905 INFO:teuthology.orchestra.run.smithi191.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T00:13:09.905 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 9a:bc:39:72:87:3d brd ff:ff:ff:ff:ff:ff 2023-10-08T00:13:09.905 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T00:13:09.905 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:13:09.905 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec77:a7ff:fea4:fbb/64 scope link 2023-10-08T00:13:09.905 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:13:09.905 INFO:teuthology.orchestra.run.smithi191.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T00:13:09.905 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether ea:1c:d9:46:05:b3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-08T00:13:09.905 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::e81c:d9ff:fe46:5b3/64 scope link 2023-10-08T00:13:09.906 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:13:09.906 INFO:teuthology.orchestra.run.smithi191.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T00:13:09.906 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 9a:bc:39:72:87:3d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-08T00:13:09.906 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::98bc:39ff:fe72:873d/64 scope link 2023-10-08T00:13:09.906 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:13:09.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:13:09.906 DEBUG:teuthology.orchestra.run.smithi191:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T00:13:09.970 INFO:teuthology.orchestra.run.smithi191.stdout:1 2023-10-08T00:13:09.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:13:09.972 DEBUG:teuthology.orchestra.run.smithi191:> route 2023-10-08T00:13:10.030 INFO:teuthology.orchestra.run.smithi191.stdout:Kernel IP routing table 2023-10-08T00:13:10.030 INFO:teuthology.orchestra.run.smithi191.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T00:13:10.031 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T00:13:10.031 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T00:13:10.031 INFO:teuthology.orchestra.run.smithi191.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T00:13:10.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:13:10.031 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2023-10-08T00:13:10.031 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2023-10-08T00:13:10.031 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T00:13:10.031 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T00:13:10.032 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T00:13:10.032 DEBUG:teuthology.orchestra.run.smithi191:> ') 2023-10-08T00:13:10.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:13:10.136 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2023-10-08T00:13:10.152 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-08T00:13:10.152 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-08T00:13:10.152 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T00:13:10.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:13:10.152 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set brx.0 up 2023-10-08T00:13:10.217 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:13:10.217 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-08T00:13:10.272 INFO:teuthology.orchestra.run.smithi191.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-08T00:13:10.272 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:13:10.272 DEBUG:teuthology.orchestra.run.smithi191:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-08T00:13:10.326 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-08T00:13:10.327 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T00:13:10.327 DEBUG:teuthology.orchestra.run.smithi191:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:13:10.407 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:13:10.407 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:13:10.426 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:13:10.426 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:13:10.426 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.99:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T00:13:10.426 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@06962cdb-9a6f-417f-82fd-9e4e3dedbccf.cephfs=/ 2023-10-08T00:13:10.426 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.99:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T00:13:10.426 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T00:13:10.426 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T00:13:10.426 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T00:13:10.447 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:13:10.447 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:13:10.447 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.99:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T00:13:10.447 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@06962cdb-9a6f-417f-82fd-9e4e3dedbccf.cephfs=/ 2023-10-08T00:13:10.447 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.99:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T00:13:10.447 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T00:13:10.448 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T00:13:10.448 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T00:13:10.448 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.99:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T00:13:10.448 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.99:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T00:13:10.448 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=06962cdb-9a6f-417f-82fd-9e4e3dedbccf" will pass to kernel 2023-10-08T00:13:10.449 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T00:13:10.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:13:10.449 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-08T00:13:10.513 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:13:10.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.629+0000 7fa16893d700 1 Processor -- start 2023-10-08T00:13:10.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.629+0000 7fa16893d700 1 -- start start 2023-10-08T00:13:10.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.630+0000 7fa16893d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1640cdab0 0x7fa16415d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:10.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.630+0000 7fa16893d700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1640cb7c0 con 0x7fa16415dda0 2023-10-08T00:13:10.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.630+0000 7fa16893d700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa16415e290 con 0x7fa1640cd6e0 2023-10-08T00:13:10.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.630+0000 7fa16893d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa16415e850 con 0x7fa1640cdab0 2023-10-08T00:13:10.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.630+0000 7fa163fff700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fa16415dda0 0x7fa16415cbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:47548/0 (socket says 172.21.15.99:47548) 2023-10-08T00:13:10.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.630+0000 7fa163fff700 1 -- 172.21.15.99:0/822275029 learned_addr learned my addr 172.21.15.99:0/822275029 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:13:10.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.630+0000 7fa162ffd700 1 --2- 172.21.15.99:0/822275029 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1640cdab0 0x7fa16415d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:10.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.630+0000 7fa1627fc700 1 -- 172.21.15.99:0/822275029 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 406240900 0 0) 0x7fa1640cb7c0 con 0x7fa16415dda0 2023-10-08T00:13:10.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.630+0000 7fa1627fc700 1 -- 172.21.15.99:0/822275029 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa14c003570 con 0x7fa16415dda0 2023-10-08T00:13:10.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.630+0000 7fa1627fc700 1 -- 172.21.15.99:0/822275029 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1913145755 0 0) 0x7fa16415e290 con 0x7fa1640cd6e0 2023-10-08T00:13:10.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.630+0000 7fa1627fc700 1 -- 172.21.15.99:0/822275029 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1640cb7c0 con 0x7fa1640cd6e0 2023-10-08T00:13:10.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.631+0000 7fa162ffd700 1 -- 172.21.15.99:0/822275029 >> v1:172.21.15.191:6789/0 conn(0x7fa1640cd6e0 legacy=0x7fa1640cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:10.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.631+0000 7fa162ffd700 1 -- 172.21.15.99:0/822275029 >> v1:172.21.15.99:6789/0 conn(0x7fa16415dda0 legacy=0x7fa16415cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:10.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.631+0000 7fa162ffd700 1 -- 172.21.15.99:0/822275029 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa16415e570 con 0x7fa1640cdab0 2023-10-08T00:13:10.635 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.631+0000 7fa162ffd700 1 --2- 172.21.15.99:0/822275029 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1640cdab0 0x7fa16415d860 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa158009910 tx=0x7fa1580091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bde9e51f95f1e89 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:13:10.635 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.631+0000 7fa1627fc700 1 -- 172.21.15.99:0/822275029 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa158011630 con 0x7fa1640cdab0 2023-10-08T00:13:10.635 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.631+0000 7fa1627fc700 1 -- 172.21.15.99:0/822275029 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa158011790 con 0x7fa1640cdab0 2023-10-08T00:13:10.635 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.631+0000 7fa1627fc700 1 -- 172.21.15.99:0/822275029 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa158011b30 con 0x7fa1640cdab0 2023-10-08T00:13:10.635 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.631+0000 7fa16893d700 1 -- 172.21.15.99:0/822275029 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1640cdab0 msgr2=0x7fa16415d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:10.635 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.631+0000 7fa16893d700 1 --2- 172.21.15.99:0/822275029 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1640cdab0 0x7fa16415d860 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa158009910 tx=0x7fa1580091a0 comp rx=0 tx=0).stop 2023-10-08T00:13:10.635 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.631+0000 7fa16893d700 1 -- 172.21.15.99:0/822275029 shutdown_connections 2023-10-08T00:13:10.635 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.631+0000 7fa16893d700 1 --2- 172.21.15.99:0/822275029 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1640cdab0 0x7fa16415d860 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:10.635 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.631+0000 7fa16893d700 1 -- 172.21.15.99:0/822275029 >> 172.21.15.99:0/822275029 conn(0x7fa164067660 msgr2=0x7fa164067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:13:10.635 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.631+0000 7fa16893d700 1 -- 172.21.15.99:0/822275029 shutdown_connections 2023-10-08T00:13:10.635 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.631+0000 7fa16893d700 1 -- 172.21.15.99:0/822275029 wait complete. 2023-10-08T00:13:10.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.633+0000 7fa16893d700 1 Processor -- start 2023-10-08T00:13:10.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.633+0000 7fa16893d700 1 -- start start 2023-10-08T00:13:10.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.633+0000 7fa16893d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1640cd6e0 0x7fa16415b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:10.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.633+0000 7fa16893d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1640cdab0 0x7fa1640c9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:10.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.633+0000 7fa16893d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa16415dda0 0x7fa164155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:10.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.633+0000 7fa16893d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1640cc360 con 0x7fa1640cd6e0 2023-10-08T00:13:10.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.633+0000 7fa16893d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16415bba0 con 0x7fa16415dda0 2023-10-08T00:13:10.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.633+0000 7fa16893d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa16415be80 con 0x7fa1640cdab0 2023-10-08T00:13:10.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.634+0000 7fa1637fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1640cd6e0 0x7fa16415b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:10.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.634+0000 7fa1637fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1640cd6e0 0x7fa16415b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46386/0 (socket says 172.21.15.99:46386) 2023-10-08T00:13:10.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.634+0000 7fa1637fe700 1 -- 172.21.15.99:0/1378686182 learned_addr learned my addr 172.21.15.99:0/1378686182 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:13:10.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.634+0000 7fa163fff700 1 --2- 172.21.15.99:0/1378686182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa16415dda0 0x7fa164155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:10.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.634+0000 7fa162ffd700 1 --2- 172.21.15.99:0/1378686182 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1640cdab0 0x7fa1640c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:10.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.634+0000 7fa1637fe700 1 -- 172.21.15.99:0/1378686182 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1640cdab0 msgr2=0x7fa1640c9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:10.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.634+0000 7fa1637fe700 1 --2- 172.21.15.99:0/1378686182 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1640cdab0 0x7fa1640c9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:10.640 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.634+0000 7fa1637fe700 1 -- 172.21.15.99:0/1378686182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa16415dda0 msgr2=0x7fa164155dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:10.640 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.634+0000 7fa1637fe700 1 --2- 172.21.15.99:0/1378686182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa16415dda0 0x7fa164155dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:10.640 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.634+0000 7fa1637fe700 1 -- 172.21.15.99:0/1378686182 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa158009420 con 0x7fa1640cd6e0 2023-10-08T00:13:10.640 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.634+0000 7fa1637fe700 1 --2- 172.21.15.99:0/1378686182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1640cd6e0 0x7fa16415b5a0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fa15400cb10 tx=0x7fa154002820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:13:10.640 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.634+0000 7fa14b7fe700 1 -- 172.21.15.99:0/1378686182 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15400a760 con 0x7fa1640cd6e0 2023-10-08T00:13:10.640 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.634+0000 7fa14b7fe700 1 -- 172.21.15.99:0/1378686182 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa15400a8c0 con 0x7fa1640cd6e0 2023-10-08T00:13:10.640 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.634+0000 7fa16893d700 1 -- 172.21.15.99:0/1378686182 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa16415c160 con 0x7fa1640cd6e0 2023-10-08T00:13:10.640 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.634+0000 7fa14b7fe700 1 -- 172.21.15.99:0/1378686182 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15400ac60 con 0x7fa1640cd6e0 2023-10-08T00:13:10.640 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.634+0000 7fa16893d700 1 -- 172.21.15.99:0/1378686182 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa164156310 con 0x7fa1640cd6e0 2023-10-08T00:13:10.640 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.635+0000 7fa16893d700 1 -- 172.21.15.99:0/1378686182 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa164089280 con 0x7fa1640cd6e0 2023-10-08T00:13:10.640 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.635+0000 7fa14b7fe700 1 -- 172.21.15.99:0/1378686182 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa154012630 con 0x7fa1640cd6e0 2023-10-08T00:13:10.640 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.636+0000 7fa14b7fe700 1 --2- 172.21.15.99:0/1378686182 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa150066f30 0x7fa1500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:13:10.641 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.636+0000 7fa14b7fe700 1 -- 172.21.15.99:0/1378686182 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa154084900 con 0x7fa1640cd6e0 2023-10-08T00:13:10.641 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.636+0000 7fa162ffd700 1 --2- 172.21.15.99:0/1378686182 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa150066f30 0x7fa1500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:13:10.641 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.636+0000 7fa162ffd700 1 --2- 172.21.15.99:0/1378686182 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa150066f30 0x7fa1500693e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa158009860 tx=0x7fa158017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:13:10.641 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.640+0000 7fa14b7fe700 1 -- 172.21.15.99:0/1378686182 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa154051b70 con 0x7fa1640cd6e0 2023-10-08T00:13:10.863 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.862+0000 7fa16893d700 1 -- 172.21.15.99:0/1378686182 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa1640c7d00 con 0x7fa1640cd6e0 2023-10-08T00:13:10.863 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.863+0000 7fa14b7fe700 1 -- 172.21.15.99:0/1378686182 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4600 (secure 0 0 0) 0x7fa154055190 con 0x7fa1640cd6e0 2023-10-08T00:13:10.864 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:13:10.864 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5087,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/862112794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":862112794},{"type":"v1","addr":"172.21.15.191:6837","nonce":862112794}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5673,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/2694493794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2694493794},{"type":"v1","addr":"172.21.15.99:6834","nonce":2694493794}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1906990531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":1906990531},{"type":"v1","addr":"172.21.15.99:6835","nonce":1906990531}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:12:57.581246+0000","modified":"2023-10-08T00:13:01.240449+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5688},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5688":{"gid":5688,"name":"d","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6836/1779510877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":1779510877},{"type":"v1","addr":"172.21.15.191:6836","nonce":1779510877}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-08T00:13:10.866 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.865+0000 7fa16893d700 1 -- 172.21.15.99:0/1378686182 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa150066f30 msgr2=0x7fa1500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:10.866 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.865+0000 7fa16893d700 1 --2- 172.21.15.99:0/1378686182 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa150066f30 0x7fa1500693e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa158009860 tx=0x7fa158017000 comp rx=0 tx=0).stop 2023-10-08T00:13:10.866 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.865+0000 7fa16893d700 1 -- 172.21.15.99:0/1378686182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1640cd6e0 msgr2=0x7fa16415b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:13:10.866 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.865+0000 7fa16893d700 1 --2- 172.21.15.99:0/1378686182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1640cd6e0 0x7fa16415b5a0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fa15400cb10 tx=0x7fa154002820 comp rx=0 tx=0).stop 2023-10-08T00:13:10.866 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.865+0000 7fa16893d700 1 -- 172.21.15.99:0/1378686182 shutdown_connections 2023-10-08T00:13:10.866 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.865+0000 7fa16893d700 1 --2- 172.21.15.99:0/1378686182 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa150066f30 0x7fa1500693e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:10.866 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.865+0000 7fa16893d700 1 --2- 172.21.15.99:0/1378686182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa16415dda0 0x7fa164155dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:10.866 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.865+0000 7fa16893d700 1 --2- 172.21.15.99:0/1378686182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1640cd6e0 0x7fa16415b5a0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:10.866 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.865+0000 7fa16893d700 1 --2- 172.21.15.99:0/1378686182 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1640cdab0 0x7fa1640c9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:13:10.867 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.865+0000 7fa16893d700 1 -- 172.21.15.99:0/1378686182 >> 172.21.15.99:0/1378686182 conn(0x7fa164067660 msgr2=0x7fa1640c62b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:13:10.867 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.865+0000 7fa16893d700 1 -- 172.21.15.99:0/1378686182 shutdown_connections 2023-10-08T00:13:10.867 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:13:10.865+0000 7fa16893d700 1 -- 172.21.15.99:0/1378686182 wait complete. 2023-10-08T00:13:10.867 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 32 2023-10-08T00:13:10.878 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json status 2023-10-08T00:13:11.010 INFO:teuthology.orchestra.run.smithi191.stdout:{"cluster_fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","whoami":0,"id":5688,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":12.779737861999999,"mdsmap_epoch":32,"osdmap_epoch":69,"osdmap_epoch_barrier":69,"uptime":18.766134757} 2023-10-08T00:13:11.011 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", "fs_name": "cephfs", "id": 5688, "mdsmap_epoch": 32, "osdmap_epoch": 69, "osdmap_epoch_barrier": 69, "rank_uptime": 12.779737862, "state": "up:active", "uptime": 18.766134757, "want_state": "up:active", "whoami": 0 } 2023-10-08T00:13:11.011 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-10-08T00:13:11.140 INFO:teuthology.orchestra.run.smithi099.stdout:{"osd_mon_report_interval":"5"} 2023-10-08T00:13:11.141 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-10-08T00:13:11.141 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2023-10-08T00:13:11.262 INFO:teuthology.orchestra.run.smithi099.stdout:{"mon_tick_interval":"5"} 2023-10-08T00:13:11.263 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2023-10-08T00:13:11.263 INFO:tasks.cephfs.test_full:pool capacity 557098560, 531MB should be enough to fill it 2023-10-08T00:13:11.263 DEBUG:teuthology.orchestra.run.smithi099:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-08T00:13:11.263 DEBUG:teuthology.orchestra.run.smithi099:> import time 2023-10-08T00:13:11.263 DEBUG:teuthology.orchestra.run.smithi099:> import datetime 2023-10-08T00:13:11.263 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2023-10-08T00:13:11.263 DEBUG:teuthology.orchestra.run.smithi099:> import os 2023-10-08T00:13:11.264 DEBUG:teuthology.orchestra.run.smithi099:> 2023-10-08T00:13:11.264 DEBUG:teuthology.orchestra.run.smithi099:> # Write some buffered data through before going full, all should be well 2023-10-08T00:13:11.264 DEBUG:teuthology.orchestra.run.smithi099:> print("writing some data through which we expect to succeed") 2023-10-08T00:13:11.264 DEBUG:teuthology.orchestra.run.smithi099:> bytes = 0 2023-10-08T00:13:11.264 DEBUG:teuthology.orchestra.run.smithi099:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2023-10-08T00:13:11.264 DEBUG:teuthology.orchestra.run.smithi099:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2023-10-08T00:13:11.264 DEBUG:teuthology.orchestra.run.smithi099:> bytes += os.write(f, b'"'"'a'"'"' * 512 * 1024) 2023-10-08T00:13:11.264 DEBUG:teuthology.orchestra.run.smithi099:> os.fsync(f) 2023-10-08T00:13:11.264 DEBUG:teuthology.orchestra.run.smithi099:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2023-10-08T00:13:11.264 DEBUG:teuthology.orchestra.run.smithi099:> 2023-10-08T00:13:11.264 DEBUG:teuthology.orchestra.run.smithi099:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2023-10-08T00:13:11.264 DEBUG:teuthology.orchestra.run.smithi099:> # writing until we see an error from fsync. As long as we'"'"'re doing 2023-10-08T00:13:11.264 DEBUG:teuthology.orchestra.run.smithi099:> # buffered IO, the error should always only appear from fsync and not 2023-10-08T00:13:11.265 DEBUG:teuthology.orchestra.run.smithi099:> # from write 2023-10-08T00:13:11.265 DEBUG:teuthology.orchestra.run.smithi099:> full = False 2023-10-08T00:13:11.265 DEBUG:teuthology.orchestra.run.smithi099:> 2023-10-08T00:13:11.265 DEBUG:teuthology.orchestra.run.smithi099:> for n in range(0, int(531 * 0.9)): 2023-10-08T00:13:11.265 DEBUG:teuthology.orchestra.run.smithi099:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2023-10-08T00:13:11.265 DEBUG:teuthology.orchestra.run.smithi099:> print("wrote {0} bytes via buffered write, may repeat".format(bytes)) 2023-10-08T00:13:11.265 DEBUG:teuthology.orchestra.run.smithi099:> print("done writing {0} bytes".format(bytes)) 2023-10-08T00:13:11.265 DEBUG:teuthology.orchestra.run.smithi099:> 2023-10-08T00:13:11.265 DEBUG:teuthology.orchestra.run.smithi099:> # OK, now we should sneak in under the full condition 2023-10-08T00:13:11.265 DEBUG:teuthology.orchestra.run.smithi099:> # due to the time it takes the OSDs to report to the 2023-10-08T00:13:11.265 DEBUG:teuthology.orchestra.run.smithi099:> # mons, and get a successful fsync on our full-making data 2023-10-08T00:13:11.265 DEBUG:teuthology.orchestra.run.smithi099:> os.fsync(f) 2023-10-08T00:13:11.265 DEBUG:teuthology.orchestra.run.smithi099:> print("successfully fsync'"'"'ed prior to getting full state reported") 2023-10-08T00:13:11.266 DEBUG:teuthology.orchestra.run.smithi099:> 2023-10-08T00:13:11.266 DEBUG:teuthology.orchestra.run.smithi099:> # buffered write, add more dirty data to the buffer 2023-10-08T00:13:11.266 DEBUG:teuthology.orchestra.run.smithi099:> print("starting buffered write") 2023-10-08T00:13:11.266 DEBUG:teuthology.orchestra.run.smithi099:> try: 2023-10-08T00:13:11.266 DEBUG:teuthology.orchestra.run.smithi099:> for n in range(0, int(531 * 0.2)): 2023-10-08T00:13:11.266 DEBUG:teuthology.orchestra.run.smithi099:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2023-10-08T00:13:11.266 DEBUG:teuthology.orchestra.run.smithi099:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2023-10-08T00:13:11.266 DEBUG:teuthology.orchestra.run.smithi099:> time.sleep(15.0) 2023-10-08T00:13:11.266 DEBUG:teuthology.orchestra.run.smithi099:> except OSError: 2023-10-08T00:13:11.266 DEBUG:teuthology.orchestra.run.smithi099:> pass; 2023-10-08T00:13:11.266 DEBUG:teuthology.orchestra.run.smithi099:> 2023-10-08T00:13:11.266 DEBUG:teuthology.orchestra.run.smithi099:> print("wrote, now waiting 30s and then doing a close we expect to fail") 2023-10-08T00:13:11.266 DEBUG:teuthology.orchestra.run.smithi099:> 2023-10-08T00:13:11.267 DEBUG:teuthology.orchestra.run.smithi099:> # Wait long enough for a background flush that should fail 2023-10-08T00:13:11.267 DEBUG:teuthology.orchestra.run.smithi099:> time.sleep(30) 2023-10-08T00:13:11.267 DEBUG:teuthology.orchestra.run.smithi099:> 2023-10-08T00:13:11.267 DEBUG:teuthology.orchestra.run.smithi099:> if False: 2023-10-08T00:13:11.267 DEBUG:teuthology.orchestra.run.smithi099:> # ...and check that the failed background flush is reflected in fclose 2023-10-08T00:13:11.267 DEBUG:teuthology.orchestra.run.smithi099:> try: 2023-10-08T00:13:11.267 DEBUG:teuthology.orchestra.run.smithi099:> os.close(f) 2023-10-08T00:13:11.267 DEBUG:teuthology.orchestra.run.smithi099:> except OSError: 2023-10-08T00:13:11.267 DEBUG:teuthology.orchestra.run.smithi099:> print("close() returned an error as expected") 2023-10-08T00:13:11.267 DEBUG:teuthology.orchestra.run.smithi099:> else: 2023-10-08T00:13:11.267 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("close() failed to raise error") 2023-10-08T00:13:11.267 DEBUG:teuthology.orchestra.run.smithi099:> else: 2023-10-08T00:13:11.267 DEBUG:teuthology.orchestra.run.smithi099:> # The kernel cephfs client does not raise errors on fclose 2023-10-08T00:13:11.268 DEBUG:teuthology.orchestra.run.smithi099:> os.close(f) 2023-10-08T00:13:11.268 DEBUG:teuthology.orchestra.run.smithi099:> 2023-10-08T00:13:11.268 DEBUG:teuthology.orchestra.run.smithi099:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2023-10-08T00:13:11.268 DEBUG:teuthology.orchestra.run.smithi099:> ' 2023-10-08T00:13:12.304 INFO:teuthology.orchestra.run.smithi099.stdout:writing some data through which we expect to succeed 2023-10-08T00:13:12.304 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed data successfully, will now attempt to fill fs 2023-10-08T00:13:12.304 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 1572864 bytes via buffered write, may repeat 2023-10-08T00:13:12.304 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 2621440 bytes via buffered write, may repeat 2023-10-08T00:13:12.304 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 3670016 bytes via buffered write, may repeat 2023-10-08T00:13:12.304 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 4718592 bytes via buffered write, may repeat 2023-10-08T00:13:12.304 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 5767168 bytes via buffered write, may repeat 2023-10-08T00:13:12.304 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 6815744 bytes via buffered write, may repeat 2023-10-08T00:13:12.304 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 7864320 bytes via buffered write, may repeat 2023-10-08T00:13:12.304 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 8912896 bytes via buffered write, may repeat 2023-10-08T00:13:12.305 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 9961472 bytes via buffered write, may repeat 2023-10-08T00:13:12.305 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 11010048 bytes via buffered write, may repeat 2023-10-08T00:13:12.305 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 12058624 bytes via buffered write, may repeat 2023-10-08T00:13:12.305 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 13107200 bytes via buffered write, may repeat 2023-10-08T00:13:12.305 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 14155776 bytes via buffered write, may repeat 2023-10-08T00:13:12.305 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 15204352 bytes via buffered write, may repeat 2023-10-08T00:13:12.305 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 16252928 bytes via buffered write, may repeat 2023-10-08T00:13:12.305 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 17301504 bytes via buffered write, may repeat 2023-10-08T00:13:12.305 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 18350080 bytes via buffered write, may repeat 2023-10-08T00:13:12.305 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 19398656 bytes via buffered write, may repeat 2023-10-08T00:13:12.305 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 20447232 bytes via buffered write, may repeat 2023-10-08T00:13:12.305 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 21495808 bytes via buffered write, may repeat 2023-10-08T00:13:12.306 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 22544384 bytes via buffered write, may repeat 2023-10-08T00:13:12.306 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 23592960 bytes via buffered write, may repeat 2023-10-08T00:13:12.306 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 24641536 bytes via buffered write, may repeat 2023-10-08T00:13:12.306 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 25690112 bytes via buffered write, may repeat 2023-10-08T00:13:12.306 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 26738688 bytes via buffered write, may repeat 2023-10-08T00:13:12.306 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 27787264 bytes via buffered write, may repeat 2023-10-08T00:13:12.306 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 28835840 bytes via buffered write, may repeat 2023-10-08T00:13:12.306 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 29884416 bytes via buffered write, may repeat 2023-10-08T00:13:12.306 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 30932992 bytes via buffered write, may repeat 2023-10-08T00:13:12.306 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 31981568 bytes via buffered write, may repeat 2023-10-08T00:13:12.306 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 33030144 bytes via buffered write, may repeat 2023-10-08T00:13:12.306 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 34078720 bytes via buffered write, may repeat 2023-10-08T00:13:12.306 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 35127296 bytes via buffered write, may repeat 2023-10-08T00:13:12.307 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 36175872 bytes via buffered write, may repeat 2023-10-08T00:13:12.307 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 37224448 bytes via buffered write, may repeat 2023-10-08T00:13:12.307 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 38273024 bytes via buffered write, may repeat 2023-10-08T00:13:12.307 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 39321600 bytes via buffered write, may repeat 2023-10-08T00:13:12.307 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 40370176 bytes via buffered write, may repeat 2023-10-08T00:13:12.307 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 41418752 bytes via buffered write, may repeat 2023-10-08T00:13:12.307 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 42467328 bytes via buffered write, may repeat 2023-10-08T00:13:12.307 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 43515904 bytes via buffered write, may repeat 2023-10-08T00:13:12.307 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 44564480 bytes via buffered write, may repeat 2023-10-08T00:13:12.307 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 45613056 bytes via buffered write, may repeat 2023-10-08T00:13:12.307 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 46661632 bytes via buffered write, may repeat 2023-10-08T00:13:12.307 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 47710208 bytes via buffered write, may repeat 2023-10-08T00:13:12.308 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 48758784 bytes via buffered write, may repeat 2023-10-08T00:13:12.308 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 49807360 bytes via buffered write, may repeat 2023-10-08T00:13:12.308 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 50855936 bytes via buffered write, may repeat 2023-10-08T00:13:12.308 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 51904512 bytes via buffered write, may repeat 2023-10-08T00:13:12.308 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 52953088 bytes via buffered write, may repeat 2023-10-08T00:13:12.308 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 54001664 bytes via buffered write, may repeat 2023-10-08T00:13:12.308 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 55050240 bytes via buffered write, may repeat 2023-10-08T00:13:12.308 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 56098816 bytes via buffered write, may repeat 2023-10-08T00:13:12.308 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 57147392 bytes via buffered write, may repeat 2023-10-08T00:13:12.308 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 58195968 bytes via buffered write, may repeat 2023-10-08T00:13:12.308 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 59244544 bytes via buffered write, may repeat 2023-10-08T00:13:12.308 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 60293120 bytes via buffered write, may repeat 2023-10-08T00:13:12.309 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 61341696 bytes via buffered write, may repeat 2023-10-08T00:13:12.309 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 62390272 bytes via buffered write, may repeat 2023-10-08T00:13:12.309 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 63438848 bytes via buffered write, may repeat 2023-10-08T00:13:12.309 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 64487424 bytes via buffered write, may repeat 2023-10-08T00:13:12.309 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 65536000 bytes via buffered write, may repeat 2023-10-08T00:13:12.309 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 66584576 bytes via buffered write, may repeat 2023-10-08T00:13:12.309 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 67633152 bytes via buffered write, may repeat 2023-10-08T00:13:12.309 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 68681728 bytes via buffered write, may repeat 2023-10-08T00:13:12.309 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 69730304 bytes via buffered write, may repeat 2023-10-08T00:13:12.309 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 70778880 bytes via buffered write, may repeat 2023-10-08T00:13:12.309 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 71827456 bytes via buffered write, may repeat 2023-10-08T00:13:12.309 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 72876032 bytes via buffered write, may repeat 2023-10-08T00:13:12.309 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 73924608 bytes via buffered write, may repeat 2023-10-08T00:13:12.310 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 74973184 bytes via buffered write, may repeat 2023-10-08T00:13:12.310 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 76021760 bytes via buffered write, may repeat 2023-10-08T00:13:12.310 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 77070336 bytes via buffered write, may repeat 2023-10-08T00:13:12.310 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 78118912 bytes via buffered write, may repeat 2023-10-08T00:13:12.310 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 79167488 bytes via buffered write, may repeat 2023-10-08T00:13:12.310 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 80216064 bytes via buffered write, may repeat 2023-10-08T00:13:12.310 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 81264640 bytes via buffered write, may repeat 2023-10-08T00:13:12.310 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 82313216 bytes via buffered write, may repeat 2023-10-08T00:13:12.310 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 83361792 bytes via buffered write, may repeat 2023-10-08T00:13:12.310 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 84410368 bytes via buffered write, may repeat 2023-10-08T00:13:12.310 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 85458944 bytes via buffered write, may repeat 2023-10-08T00:13:12.310 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 86507520 bytes via buffered write, may repeat 2023-10-08T00:13:12.311 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 87556096 bytes via buffered write, may repeat 2023-10-08T00:13:12.311 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 88604672 bytes via buffered write, may repeat 2023-10-08T00:13:12.311 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 89653248 bytes via buffered write, may repeat 2023-10-08T00:13:12.311 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 90701824 bytes via buffered write, may repeat 2023-10-08T00:13:12.311 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 91750400 bytes via buffered write, may repeat 2023-10-08T00:13:12.311 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 92798976 bytes via buffered write, may repeat 2023-10-08T00:13:12.311 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 93847552 bytes via buffered write, may repeat 2023-10-08T00:13:12.311 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 94896128 bytes via buffered write, may repeat 2023-10-08T00:13:12.311 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 95944704 bytes via buffered write, may repeat 2023-10-08T00:13:12.311 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 96993280 bytes via buffered write, may repeat 2023-10-08T00:13:12.311 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 98041856 bytes via buffered write, may repeat 2023-10-08T00:13:12.311 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 99090432 bytes via buffered write, may repeat 2023-10-08T00:13:12.312 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 100139008 bytes via buffered write, may repeat 2023-10-08T00:13:12.312 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 101187584 bytes via buffered write, may repeat 2023-10-08T00:13:12.312 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 102236160 bytes via buffered write, may repeat 2023-10-08T00:13:12.312 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 103284736 bytes via buffered write, may repeat 2023-10-08T00:13:12.312 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 104333312 bytes via buffered write, may repeat 2023-10-08T00:13:12.312 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 105381888 bytes via buffered write, may repeat 2023-10-08T00:13:12.312 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 106430464 bytes via buffered write, may repeat 2023-10-08T00:13:12.312 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 107479040 bytes via buffered write, may repeat 2023-10-08T00:13:12.312 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 108527616 bytes via buffered write, may repeat 2023-10-08T00:13:12.312 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 109576192 bytes via buffered write, may repeat 2023-10-08T00:13:12.312 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 110624768 bytes via buffered write, may repeat 2023-10-08T00:13:12.312 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 111673344 bytes via buffered write, may repeat 2023-10-08T00:13:12.313 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 112721920 bytes via buffered write, may repeat 2023-10-08T00:13:12.313 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 113770496 bytes via buffered write, may repeat 2023-10-08T00:13:12.313 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 114819072 bytes via buffered write, may repeat 2023-10-08T00:13:12.313 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 115867648 bytes via buffered write, may repeat 2023-10-08T00:13:12.313 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 116916224 bytes via buffered write, may repeat 2023-10-08T00:13:12.313 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 117964800 bytes via buffered write, may repeat 2023-10-08T00:13:12.313 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 119013376 bytes via buffered write, may repeat 2023-10-08T00:13:12.313 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 120061952 bytes via buffered write, may repeat 2023-10-08T00:13:12.313 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 121110528 bytes via buffered write, may repeat 2023-10-08T00:13:12.313 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 122159104 bytes via buffered write, may repeat 2023-10-08T00:13:12.313 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 123207680 bytes via buffered write, may repeat 2023-10-08T00:13:12.313 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 124256256 bytes via buffered write, may repeat 2023-10-08T00:13:12.314 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 125304832 bytes via buffered write, may repeat 2023-10-08T00:13:12.314 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 126353408 bytes via buffered write, may repeat 2023-10-08T00:13:12.314 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 127401984 bytes via buffered write, may repeat 2023-10-08T00:13:12.314 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 128450560 bytes via buffered write, may repeat 2023-10-08T00:13:12.314 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 129499136 bytes via buffered write, may repeat 2023-10-08T00:13:12.314 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 130547712 bytes via buffered write, may repeat 2023-10-08T00:13:12.314 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 131596288 bytes via buffered write, may repeat 2023-10-08T00:13:12.314 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 132644864 bytes via buffered write, may repeat 2023-10-08T00:13:12.314 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 133693440 bytes via buffered write, may repeat 2023-10-08T00:13:12.314 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 134742016 bytes via buffered write, may repeat 2023-10-08T00:13:12.314 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 135790592 bytes via buffered write, may repeat 2023-10-08T00:13:12.314 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 136839168 bytes via buffered write, may repeat 2023-10-08T00:13:12.315 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 137887744 bytes via buffered write, may repeat 2023-10-08T00:13:12.315 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 138936320 bytes via buffered write, may repeat 2023-10-08T00:13:12.315 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 139984896 bytes via buffered write, may repeat 2023-10-08T00:13:12.315 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 141033472 bytes via buffered write, may repeat 2023-10-08T00:13:12.315 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 142082048 bytes via buffered write, may repeat 2023-10-08T00:13:12.315 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 143130624 bytes via buffered write, may repeat 2023-10-08T00:13:12.315 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 144179200 bytes via buffered write, may repeat 2023-10-08T00:13:12.315 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 145227776 bytes via buffered write, may repeat 2023-10-08T00:13:12.315 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 146276352 bytes via buffered write, may repeat 2023-10-08T00:13:12.315 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 147324928 bytes via buffered write, may repeat 2023-10-08T00:13:12.315 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 148373504 bytes via buffered write, may repeat 2023-10-08T00:13:12.315 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 149422080 bytes via buffered write, may repeat 2023-10-08T00:13:12.315 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 150470656 bytes via buffered write, may repeat 2023-10-08T00:13:12.316 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 151519232 bytes via buffered write, may repeat 2023-10-08T00:13:12.316 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 152567808 bytes via buffered write, may repeat 2023-10-08T00:13:12.316 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 153616384 bytes via buffered write, may repeat 2023-10-08T00:13:12.316 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 154664960 bytes via buffered write, may repeat 2023-10-08T00:13:12.316 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 155713536 bytes via buffered write, may repeat 2023-10-08T00:13:12.316 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 156762112 bytes via buffered write, may repeat 2023-10-08T00:13:12.316 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 157810688 bytes via buffered write, may repeat 2023-10-08T00:13:12.316 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 158859264 bytes via buffered write, may repeat 2023-10-08T00:13:12.316 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 159907840 bytes via buffered write, may repeat 2023-10-08T00:13:12.316 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 160956416 bytes via buffered write, may repeat 2023-10-08T00:13:12.316 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 162004992 bytes via buffered write, may repeat 2023-10-08T00:13:12.352 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 163053568 bytes via buffered write, may repeat 2023-10-08T00:13:12.353 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 164102144 bytes via buffered write, may repeat 2023-10-08T00:13:12.353 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 165150720 bytes via buffered write, may repeat 2023-10-08T00:13:12.353 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 166199296 bytes via buffered write, may repeat 2023-10-08T00:13:12.353 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 167247872 bytes via buffered write, may repeat 2023-10-08T00:13:12.353 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 168296448 bytes via buffered write, may repeat 2023-10-08T00:13:12.353 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 169345024 bytes via buffered write, may repeat 2023-10-08T00:13:12.353 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 170393600 bytes via buffered write, may repeat 2023-10-08T00:13:12.353 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 171442176 bytes via buffered write, may repeat 2023-10-08T00:13:12.353 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 172490752 bytes via buffered write, may repeat 2023-10-08T00:13:12.353 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 173539328 bytes via buffered write, may repeat 2023-10-08T00:13:12.354 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 174587904 bytes via buffered write, may repeat 2023-10-08T00:13:12.354 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 175636480 bytes via buffered write, may repeat 2023-10-08T00:13:12.354 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 176685056 bytes via buffered write, may repeat 2023-10-08T00:13:12.354 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 177733632 bytes via buffered write, may repeat 2023-10-08T00:13:12.354 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 178782208 bytes via buffered write, may repeat 2023-10-08T00:13:12.354 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 179830784 bytes via buffered write, may repeat 2023-10-08T00:13:12.354 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 180879360 bytes via buffered write, may repeat 2023-10-08T00:13:12.354 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 181927936 bytes via buffered write, may repeat 2023-10-08T00:13:12.354 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 182976512 bytes via buffered write, may repeat 2023-10-08T00:13:12.354 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 184025088 bytes via buffered write, may repeat 2023-10-08T00:13:12.354 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 185073664 bytes via buffered write, may repeat 2023-10-08T00:13:12.355 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 186122240 bytes via buffered write, may repeat 2023-10-08T00:13:12.355 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 187170816 bytes via buffered write, may repeat 2023-10-08T00:13:12.355 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 188219392 bytes via buffered write, may repeat 2023-10-08T00:13:12.355 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 189267968 bytes via buffered write, may repeat 2023-10-08T00:13:12.355 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 190316544 bytes via buffered write, may repeat 2023-10-08T00:13:12.355 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 191365120 bytes via buffered write, may repeat 2023-10-08T00:13:12.355 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 192413696 bytes via buffered write, may repeat 2023-10-08T00:13:12.355 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 193462272 bytes via buffered write, may repeat 2023-10-08T00:13:12.355 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 194510848 bytes via buffered write, may repeat 2023-10-08T00:13:12.355 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 195559424 bytes via buffered write, may repeat 2023-10-08T00:13:12.355 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 196608000 bytes via buffered write, may repeat 2023-10-08T00:13:12.355 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 197656576 bytes via buffered write, may repeat 2023-10-08T00:13:12.356 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 198705152 bytes via buffered write, may repeat 2023-10-08T00:13:12.356 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 199753728 bytes via buffered write, may repeat 2023-10-08T00:13:12.356 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 200802304 bytes via buffered write, may repeat 2023-10-08T00:13:12.356 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 201850880 bytes via buffered write, may repeat 2023-10-08T00:13:12.356 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 202899456 bytes via buffered write, may repeat 2023-10-08T00:13:12.356 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 203948032 bytes via buffered write, may repeat 2023-10-08T00:13:12.356 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 204996608 bytes via buffered write, may repeat 2023-10-08T00:13:12.356 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 206045184 bytes via buffered write, may repeat 2023-10-08T00:13:12.356 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 207093760 bytes via buffered write, may repeat 2023-10-08T00:13:12.356 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 208142336 bytes via buffered write, may repeat 2023-10-08T00:13:12.356 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 209190912 bytes via buffered write, may repeat 2023-10-08T00:13:12.356 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 210239488 bytes via buffered write, may repeat 2023-10-08T00:13:12.357 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 211288064 bytes via buffered write, may repeat 2023-10-08T00:13:12.357 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 212336640 bytes via buffered write, may repeat 2023-10-08T00:13:12.357 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 213385216 bytes via buffered write, may repeat 2023-10-08T00:13:12.357 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 214433792 bytes via buffered write, may repeat 2023-10-08T00:13:12.357 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 215482368 bytes via buffered write, may repeat 2023-10-08T00:13:12.357 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 216530944 bytes via buffered write, may repeat 2023-10-08T00:13:12.357 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 217579520 bytes via buffered write, may repeat 2023-10-08T00:13:12.357 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 218628096 bytes via buffered write, may repeat 2023-10-08T00:13:12.357 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 219676672 bytes via buffered write, may repeat 2023-10-08T00:13:12.357 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 220725248 bytes via buffered write, may repeat 2023-10-08T00:13:12.357 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 221773824 bytes via buffered write, may repeat 2023-10-08T00:13:12.357 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 222822400 bytes via buffered write, may repeat 2023-10-08T00:13:12.358 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 223870976 bytes via buffered write, may repeat 2023-10-08T00:13:12.358 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 224919552 bytes via buffered write, may repeat 2023-10-08T00:13:12.358 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 225968128 bytes via buffered write, may repeat 2023-10-08T00:13:12.358 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 227016704 bytes via buffered write, may repeat 2023-10-08T00:13:12.358 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 228065280 bytes via buffered write, may repeat 2023-10-08T00:13:12.358 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 229113856 bytes via buffered write, may repeat 2023-10-08T00:13:12.358 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 230162432 bytes via buffered write, may repeat 2023-10-08T00:13:12.358 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 231211008 bytes via buffered write, may repeat 2023-10-08T00:13:12.358 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 232259584 bytes via buffered write, may repeat 2023-10-08T00:13:12.358 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 233308160 bytes via buffered write, may repeat 2023-10-08T00:13:12.358 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 234356736 bytes via buffered write, may repeat 2023-10-08T00:13:12.359 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 235405312 bytes via buffered write, may repeat 2023-10-08T00:13:12.359 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 236453888 bytes via buffered write, may repeat 2023-10-08T00:13:12.359 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 237502464 bytes via buffered write, may repeat 2023-10-08T00:13:12.359 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 238551040 bytes via buffered write, may repeat 2023-10-08T00:13:12.359 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 239599616 bytes via buffered write, may repeat 2023-10-08T00:13:12.359 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 240648192 bytes via buffered write, may repeat 2023-10-08T00:13:12.359 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 241696768 bytes via buffered write, may repeat 2023-10-08T00:13:12.359 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 242745344 bytes via buffered write, may repeat 2023-10-08T00:13:12.359 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 243793920 bytes via buffered write, may repeat 2023-10-08T00:13:12.359 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 244842496 bytes via buffered write, may repeat 2023-10-08T00:13:12.359 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 245891072 bytes via buffered write, may repeat 2023-10-08T00:13:12.359 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 246939648 bytes via buffered write, may repeat 2023-10-08T00:13:12.360 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 247988224 bytes via buffered write, may repeat 2023-10-08T00:13:12.360 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 249036800 bytes via buffered write, may repeat 2023-10-08T00:13:12.360 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 250085376 bytes via buffered write, may repeat 2023-10-08T00:13:12.360 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 251133952 bytes via buffered write, may repeat 2023-10-08T00:13:12.360 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 252182528 bytes via buffered write, may repeat 2023-10-08T00:13:12.360 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 253231104 bytes via buffered write, may repeat 2023-10-08T00:13:12.360 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 254279680 bytes via buffered write, may repeat 2023-10-08T00:13:12.360 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 255328256 bytes via buffered write, may repeat 2023-10-08T00:13:12.360 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 256376832 bytes via buffered write, may repeat 2023-10-08T00:13:12.360 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 257425408 bytes via buffered write, may repeat 2023-10-08T00:13:12.360 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 258473984 bytes via buffered write, may repeat 2023-10-08T00:13:12.361 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 259522560 bytes via buffered write, may repeat 2023-10-08T00:13:12.361 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 260571136 bytes via buffered write, may repeat 2023-10-08T00:13:12.361 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 261619712 bytes via buffered write, may repeat 2023-10-08T00:13:12.361 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 262668288 bytes via buffered write, may repeat 2023-10-08T00:13:12.361 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 263716864 bytes via buffered write, may repeat 2023-10-08T00:13:12.361 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 264765440 bytes via buffered write, may repeat 2023-10-08T00:13:12.361 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 265814016 bytes via buffered write, may repeat 2023-10-08T00:13:12.361 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 266862592 bytes via buffered write, may repeat 2023-10-08T00:13:12.361 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 267911168 bytes via buffered write, may repeat 2023-10-08T00:13:12.361 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 268959744 bytes via buffered write, may repeat 2023-10-08T00:13:12.361 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 270008320 bytes via buffered write, may repeat 2023-10-08T00:13:12.362 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 271056896 bytes via buffered write, may repeat 2023-10-08T00:13:12.362 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 272105472 bytes via buffered write, may repeat 2023-10-08T00:13:12.362 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 273154048 bytes via buffered write, may repeat 2023-10-08T00:13:12.362 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 274202624 bytes via buffered write, may repeat 2023-10-08T00:13:12.362 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 275251200 bytes via buffered write, may repeat 2023-10-08T00:13:12.362 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 276299776 bytes via buffered write, may repeat 2023-10-08T00:13:12.362 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 277348352 bytes via buffered write, may repeat 2023-10-08T00:13:12.362 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 278396928 bytes via buffered write, may repeat 2023-10-08T00:13:12.362 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 279445504 bytes via buffered write, may repeat 2023-10-08T00:13:12.362 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 280494080 bytes via buffered write, may repeat 2023-10-08T00:13:12.362 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 281542656 bytes via buffered write, may repeat 2023-10-08T00:13:12.362 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 282591232 bytes via buffered write, may repeat 2023-10-08T00:13:12.363 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 283639808 bytes via buffered write, may repeat 2023-10-08T00:13:12.363 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 284688384 bytes via buffered write, may repeat 2023-10-08T00:13:12.363 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 285736960 bytes via buffered write, may repeat 2023-10-08T00:13:12.363 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 286785536 bytes via buffered write, may repeat 2023-10-08T00:13:12.363 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 287834112 bytes via buffered write, may repeat 2023-10-08T00:13:12.363 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 288882688 bytes via buffered write, may repeat 2023-10-08T00:13:12.363 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 289931264 bytes via buffered write, may repeat 2023-10-08T00:13:12.363 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 290979840 bytes via buffered write, may repeat 2023-10-08T00:13:12.363 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 292028416 bytes via buffered write, may repeat 2023-10-08T00:13:12.363 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 293076992 bytes via buffered write, may repeat 2023-10-08T00:13:12.363 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 294125568 bytes via buffered write, may repeat 2023-10-08T00:13:12.364 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 295174144 bytes via buffered write, may repeat 2023-10-08T00:13:12.364 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 296222720 bytes via buffered write, may repeat 2023-10-08T00:13:12.364 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 297271296 bytes via buffered write, may repeat 2023-10-08T00:13:12.364 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 298319872 bytes via buffered write, may repeat 2023-10-08T00:13:12.364 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 299368448 bytes via buffered write, may repeat 2023-10-08T00:13:12.364 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 300417024 bytes via buffered write, may repeat 2023-10-08T00:13:12.364 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 301465600 bytes via buffered write, may repeat 2023-10-08T00:13:12.364 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 302514176 bytes via buffered write, may repeat 2023-10-08T00:13:12.364 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 303562752 bytes via buffered write, may repeat 2023-10-08T00:13:12.364 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 304611328 bytes via buffered write, may repeat 2023-10-08T00:13:12.364 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 305659904 bytes via buffered write, may repeat 2023-10-08T00:13:12.365 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 306708480 bytes via buffered write, may repeat 2023-10-08T00:13:12.365 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 307757056 bytes via buffered write, may repeat 2023-10-08T00:13:12.365 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 308805632 bytes via buffered write, may repeat 2023-10-08T00:13:12.365 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 309854208 bytes via buffered write, may repeat 2023-10-08T00:13:12.365 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 310902784 bytes via buffered write, may repeat 2023-10-08T00:13:12.365 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 311951360 bytes via buffered write, may repeat 2023-10-08T00:13:12.365 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 312999936 bytes via buffered write, may repeat 2023-10-08T00:13:12.365 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 314048512 bytes via buffered write, may repeat 2023-10-08T00:13:12.365 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 315097088 bytes via buffered write, may repeat 2023-10-08T00:13:12.365 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 316145664 bytes via buffered write, may repeat 2023-10-08T00:13:12.365 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 317194240 bytes via buffered write, may repeat 2023-10-08T00:13:12.366 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 318242816 bytes via buffered write, may repeat 2023-10-08T00:13:12.366 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 319291392 bytes via buffered write, may repeat 2023-10-08T00:13:12.366 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 320339968 bytes via buffered write, may repeat 2023-10-08T00:13:12.366 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 321388544 bytes via buffered write, may repeat 2023-10-08T00:13:12.366 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 322437120 bytes via buffered write, may repeat 2023-10-08T00:13:12.366 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 323485696 bytes via buffered write, may repeat 2023-10-08T00:13:12.366 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 324534272 bytes via buffered write, may repeat 2023-10-08T00:13:12.400 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 325582848 bytes via buffered write, may repeat 2023-10-08T00:13:12.400 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 326631424 bytes via buffered write, may repeat 2023-10-08T00:13:12.400 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 327680000 bytes via buffered write, may repeat 2023-10-08T00:13:12.400 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 328728576 bytes via buffered write, may repeat 2023-10-08T00:13:12.401 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 329777152 bytes via buffered write, may repeat 2023-10-08T00:13:12.401 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 330825728 bytes via buffered write, may repeat 2023-10-08T00:13:12.401 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 331874304 bytes via buffered write, may repeat 2023-10-08T00:13:12.401 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 332922880 bytes via buffered write, may repeat 2023-10-08T00:13:12.401 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 333971456 bytes via buffered write, may repeat 2023-10-08T00:13:12.401 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 335020032 bytes via buffered write, may repeat 2023-10-08T00:13:12.401 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 336068608 bytes via buffered write, may repeat 2023-10-08T00:13:12.401 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 337117184 bytes via buffered write, may repeat 2023-10-08T00:13:12.401 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 338165760 bytes via buffered write, may repeat 2023-10-08T00:13:12.401 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 339214336 bytes via buffered write, may repeat 2023-10-08T00:13:12.402 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 340262912 bytes via buffered write, may repeat 2023-10-08T00:13:12.402 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 341311488 bytes via buffered write, may repeat 2023-10-08T00:13:12.402 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 342360064 bytes via buffered write, may repeat 2023-10-08T00:13:12.402 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 343408640 bytes via buffered write, may repeat 2023-10-08T00:13:12.402 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 344457216 bytes via buffered write, may repeat 2023-10-08T00:13:12.402 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 345505792 bytes via buffered write, may repeat 2023-10-08T00:13:12.402 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 346554368 bytes via buffered write, may repeat 2023-10-08T00:13:12.402 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 347602944 bytes via buffered write, may repeat 2023-10-08T00:13:12.402 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 348651520 bytes via buffered write, may repeat 2023-10-08T00:13:12.402 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 349700096 bytes via buffered write, may repeat 2023-10-08T00:13:12.403 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 350748672 bytes via buffered write, may repeat 2023-10-08T00:13:12.403 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 351797248 bytes via buffered write, may repeat 2023-10-08T00:13:12.403 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 352845824 bytes via buffered write, may repeat 2023-10-08T00:13:12.403 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 353894400 bytes via buffered write, may repeat 2023-10-08T00:13:12.403 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 354942976 bytes via buffered write, may repeat 2023-10-08T00:13:12.403 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 355991552 bytes via buffered write, may repeat 2023-10-08T00:13:12.403 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 357040128 bytes via buffered write, may repeat 2023-10-08T00:13:12.403 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 358088704 bytes via buffered write, may repeat 2023-10-08T00:13:12.403 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 359137280 bytes via buffered write, may repeat 2023-10-08T00:13:12.404 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 360185856 bytes via buffered write, may repeat 2023-10-08T00:13:12.404 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 361234432 bytes via buffered write, may repeat 2023-10-08T00:13:12.404 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 362283008 bytes via buffered write, may repeat 2023-10-08T00:13:12.404 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 363331584 bytes via buffered write, may repeat 2023-10-08T00:13:12.404 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 364380160 bytes via buffered write, may repeat 2023-10-08T00:13:12.404 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 365428736 bytes via buffered write, may repeat 2023-10-08T00:13:12.404 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 366477312 bytes via buffered write, may repeat 2023-10-08T00:13:12.404 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 367525888 bytes via buffered write, may repeat 2023-10-08T00:13:12.404 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 368574464 bytes via buffered write, may repeat 2023-10-08T00:13:12.404 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 369623040 bytes via buffered write, may repeat 2023-10-08T00:13:12.405 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 370671616 bytes via buffered write, may repeat 2023-10-08T00:13:12.405 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 371720192 bytes via buffered write, may repeat 2023-10-08T00:13:12.405 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 372768768 bytes via buffered write, may repeat 2023-10-08T00:13:12.405 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 373817344 bytes via buffered write, may repeat 2023-10-08T00:13:12.405 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 374865920 bytes via buffered write, may repeat 2023-10-08T00:13:12.405 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 375914496 bytes via buffered write, may repeat 2023-10-08T00:13:12.405 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 376963072 bytes via buffered write, may repeat 2023-10-08T00:13:12.405 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 378011648 bytes via buffered write, may repeat 2023-10-08T00:13:12.405 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 379060224 bytes via buffered write, may repeat 2023-10-08T00:13:12.405 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 380108800 bytes via buffered write, may repeat 2023-10-08T00:13:12.405 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 381157376 bytes via buffered write, may repeat 2023-10-08T00:13:12.406 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 382205952 bytes via buffered write, may repeat 2023-10-08T00:13:12.406 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 383254528 bytes via buffered write, may repeat 2023-10-08T00:13:12.406 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 384303104 bytes via buffered write, may repeat 2023-10-08T00:13:12.406 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 385351680 bytes via buffered write, may repeat 2023-10-08T00:13:12.406 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 386400256 bytes via buffered write, may repeat 2023-10-08T00:13:12.406 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 387448832 bytes via buffered write, may repeat 2023-10-08T00:13:12.406 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 388497408 bytes via buffered write, may repeat 2023-10-08T00:13:12.406 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 389545984 bytes via buffered write, may repeat 2023-10-08T00:13:12.406 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 390594560 bytes via buffered write, may repeat 2023-10-08T00:13:12.406 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 391643136 bytes via buffered write, may repeat 2023-10-08T00:13:12.406 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 392691712 bytes via buffered write, may repeat 2023-10-08T00:13:12.407 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 393740288 bytes via buffered write, may repeat 2023-10-08T00:13:12.407 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 394788864 bytes via buffered write, may repeat 2023-10-08T00:13:12.407 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 395837440 bytes via buffered write, may repeat 2023-10-08T00:13:12.407 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 396886016 bytes via buffered write, may repeat 2023-10-08T00:13:12.407 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 397934592 bytes via buffered write, may repeat 2023-10-08T00:13:12.407 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 398983168 bytes via buffered write, may repeat 2023-10-08T00:13:12.407 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 400031744 bytes via buffered write, may repeat 2023-10-08T00:13:12.407 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 401080320 bytes via buffered write, may repeat 2023-10-08T00:13:12.407 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 402128896 bytes via buffered write, may repeat 2023-10-08T00:13:12.407 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 403177472 bytes via buffered write, may repeat 2023-10-08T00:13:12.407 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 404226048 bytes via buffered write, may repeat 2023-10-08T00:13:12.408 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 405274624 bytes via buffered write, may repeat 2023-10-08T00:13:12.408 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 406323200 bytes via buffered write, may repeat 2023-10-08T00:13:12.408 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 407371776 bytes via buffered write, may repeat 2023-10-08T00:13:12.408 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 408420352 bytes via buffered write, may repeat 2023-10-08T00:13:12.408 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 409468928 bytes via buffered write, may repeat 2023-10-08T00:13:12.408 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 410517504 bytes via buffered write, may repeat 2023-10-08T00:13:12.408 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 411566080 bytes via buffered write, may repeat 2023-10-08T00:13:12.408 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 412614656 bytes via buffered write, may repeat 2023-10-08T00:13:12.408 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 413663232 bytes via buffered write, may repeat 2023-10-08T00:13:12.408 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 414711808 bytes via buffered write, may repeat 2023-10-08T00:13:12.409 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 415760384 bytes via buffered write, may repeat 2023-10-08T00:13:12.409 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 416808960 bytes via buffered write, may repeat 2023-10-08T00:13:12.409 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 417857536 bytes via buffered write, may repeat 2023-10-08T00:13:12.409 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 418906112 bytes via buffered write, may repeat 2023-10-08T00:13:12.409 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 419954688 bytes via buffered write, may repeat 2023-10-08T00:13:12.409 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 421003264 bytes via buffered write, may repeat 2023-10-08T00:13:12.409 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 422051840 bytes via buffered write, may repeat 2023-10-08T00:13:12.409 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 423100416 bytes via buffered write, may repeat 2023-10-08T00:13:12.409 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 424148992 bytes via buffered write, may repeat 2023-10-08T00:13:12.409 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 425197568 bytes via buffered write, may repeat 2023-10-08T00:13:12.409 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 426246144 bytes via buffered write, may repeat 2023-10-08T00:13:12.411 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 427294720 bytes via buffered write, may repeat 2023-10-08T00:13:12.411 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 428343296 bytes via buffered write, may repeat 2023-10-08T00:13:12.411 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 429391872 bytes via buffered write, may repeat 2023-10-08T00:13:12.411 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 430440448 bytes via buffered write, may repeat 2023-10-08T00:13:12.411 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 431489024 bytes via buffered write, may repeat 2023-10-08T00:13:12.411 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 432537600 bytes via buffered write, may repeat 2023-10-08T00:13:12.412 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 433586176 bytes via buffered write, may repeat 2023-10-08T00:13:12.412 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 434634752 bytes via buffered write, may repeat 2023-10-08T00:13:12.412 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 435683328 bytes via buffered write, may repeat 2023-10-08T00:13:12.412 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 436731904 bytes via buffered write, may repeat 2023-10-08T00:13:12.412 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 437780480 bytes via buffered write, may repeat 2023-10-08T00:13:12.412 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 438829056 bytes via buffered write, may repeat 2023-10-08T00:13:12.412 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 439877632 bytes via buffered write, may repeat 2023-10-08T00:13:12.412 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 440926208 bytes via buffered write, may repeat 2023-10-08T00:13:12.412 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 441974784 bytes via buffered write, may repeat 2023-10-08T00:13:12.412 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 443023360 bytes via buffered write, may repeat 2023-10-08T00:13:12.412 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 444071936 bytes via buffered write, may repeat 2023-10-08T00:13:12.412 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 445120512 bytes via buffered write, may repeat 2023-10-08T00:13:12.413 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 446169088 bytes via buffered write, may repeat 2023-10-08T00:13:12.413 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 447217664 bytes via buffered write, may repeat 2023-10-08T00:13:12.413 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 448266240 bytes via buffered write, may repeat 2023-10-08T00:13:12.413 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 449314816 bytes via buffered write, may repeat 2023-10-08T00:13:12.413 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 450363392 bytes via buffered write, may repeat 2023-10-08T00:13:12.413 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 451411968 bytes via buffered write, may repeat 2023-10-08T00:13:12.413 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 452460544 bytes via buffered write, may repeat 2023-10-08T00:13:12.413 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 453509120 bytes via buffered write, may repeat 2023-10-08T00:13:12.413 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 454557696 bytes via buffered write, may repeat 2023-10-08T00:13:12.413 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 455606272 bytes via buffered write, may repeat 2023-10-08T00:13:12.413 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 456654848 bytes via buffered write, may repeat 2023-10-08T00:13:12.414 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 457703424 bytes via buffered write, may repeat 2023-10-08T00:13:12.414 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 458752000 bytes via buffered write, may repeat 2023-10-08T00:13:12.414 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 459800576 bytes via buffered write, may repeat 2023-10-08T00:13:12.414 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 460849152 bytes via buffered write, may repeat 2023-10-08T00:13:12.414 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 461897728 bytes via buffered write, may repeat 2023-10-08T00:13:12.414 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 462946304 bytes via buffered write, may repeat 2023-10-08T00:13:12.414 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 463994880 bytes via buffered write, may repeat 2023-10-08T00:13:12.414 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 465043456 bytes via buffered write, may repeat 2023-10-08T00:13:12.414 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 466092032 bytes via buffered write, may repeat 2023-10-08T00:13:12.414 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 467140608 bytes via buffered write, may repeat 2023-10-08T00:13:12.414 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 468189184 bytes via buffered write, may repeat 2023-10-08T00:13:12.414 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 469237760 bytes via buffered write, may repeat 2023-10-08T00:13:12.415 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 470286336 bytes via buffered write, may repeat 2023-10-08T00:13:12.415 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 471334912 bytes via buffered write, may repeat 2023-10-08T00:13:12.415 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 472383488 bytes via buffered write, may repeat 2023-10-08T00:13:12.415 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 473432064 bytes via buffered write, may repeat 2023-10-08T00:13:12.415 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 474480640 bytes via buffered write, may repeat 2023-10-08T00:13:12.416 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 475529216 bytes via buffered write, may repeat 2023-10-08T00:13:12.416 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 476577792 bytes via buffered write, may repeat 2023-10-08T00:13:12.416 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 477626368 bytes via buffered write, may repeat 2023-10-08T00:13:12.416 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 478674944 bytes via buffered write, may repeat 2023-10-08T00:13:12.416 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 479723520 bytes via buffered write, may repeat 2023-10-08T00:13:12.416 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 480772096 bytes via buffered write, may repeat 2023-10-08T00:13:12.416 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 481820672 bytes via buffered write, may repeat 2023-10-08T00:13:12.416 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 482869248 bytes via buffered write, may repeat 2023-10-08T00:13:12.416 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 483917824 bytes via buffered write, may repeat 2023-10-08T00:13:12.416 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 484966400 bytes via buffered write, may repeat 2023-10-08T00:13:12.416 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 486014976 bytes via buffered write, may repeat 2023-10-08T00:13:12.416 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 487063552 bytes via buffered write, may repeat 2023-10-08T00:13:15.317 INFO:tasks.ceph.mon.a.smithi099.stderr:2023-10-08T00:13:15.316+0000 7f6c637a7700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 full osd(s) (OSD_FULL) 2023-10-08T00:13:21.072 INFO:tasks.ceph.mon.a.smithi099.stderr:2023-10-08T00:13:21.071+0000 7f6c637a7700 -1 log_channel(cluster) log [ERR] : Health check update: 3 full osd(s) (OSD_FULL) 2023-10-08T00:13:22.861 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:13:22.864 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:13:48.144 INFO:tasks.ceph.mon.a.smithi099.stderr:2023-10-08T00:13:48.143+0000 7f6c637a7700 -1 log_channel(cluster) log [ERR] : Health check failed: Full OSDs blocking recovery: 1 pg recovery_toofull (PG_RECOVERY_FULL) 2023-10-08T00:13:52.923 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:13:52.925 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:14:22.997 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:14:23.000 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:14:27.899 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 488112128 bytes via buffered write, may repeat 2023-10-08T00:14:27.899 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 489160704 bytes via buffered write, may repeat 2023-10-08T00:14:27.900 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 490209280 bytes via buffered write, may repeat 2023-10-08T00:14:27.900 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 491257856 bytes via buffered write, may repeat 2023-10-08T00:14:27.900 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 492306432 bytes via buffered write, may repeat 2023-10-08T00:14:27.900 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 493355008 bytes via buffered write, may repeat 2023-10-08T00:14:27.900 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 494403584 bytes via buffered write, may repeat 2023-10-08T00:14:27.900 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 495452160 bytes via buffered write, may repeat 2023-10-08T00:14:27.900 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 496500736 bytes via buffered write, may repeat 2023-10-08T00:14:27.900 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 497549312 bytes via buffered write, may repeat 2023-10-08T00:14:27.900 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 498597888 bytes via buffered write, may repeat 2023-10-08T00:14:27.900 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 499646464 bytes via buffered write, may repeat 2023-10-08T00:14:27.901 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 500695040 bytes via buffered write, may repeat 2023-10-08T00:14:27.901 INFO:teuthology.orchestra.run.smithi099.stdout:done writing 500695040 bytes 2023-10-08T00:14:27.901 INFO:teuthology.orchestra.run.smithi099.stdout:successfully fsync'ed prior to getting full state reported 2023-10-08T00:14:27.901 INFO:teuthology.orchestra.run.smithi099.stdout:starting buffered write 2023-10-08T00:14:27.901 INFO:teuthology.orchestra.run.smithi099.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-08T00:14:27.901 INFO:teuthology.orchestra.run.smithi099.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-08T00:14:27.901 INFO:teuthology.orchestra.run.smithi099.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-08T00:14:27.901 INFO:teuthology.orchestra.run.smithi099.stdout:wrote, now waiting 30s and then doing a close we expect to fail 2023-10-08T00:14:28.026 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T00:14:28.027 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T00:14:28.079 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T00:14:28.129 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T00:14:28.129 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T00:14:28.129 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:14:28.130 DEBUG:teuthology.orchestra.run.smithi099:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-08T00:14:28.173 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2023-10-08T00:14:28.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:14:28.174 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T00:14:28.229 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-08T00:14:28.230 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:14:28.230 DEBUG:teuthology.orchestra.run.smithi191:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-08T00:14:28.276 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2023-10-08T00:14:28.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:14:28.277 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-08T00:14:28.300 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:14:28.443 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.443+0000 7fe149c9c700 1 Processor -- start 2023-10-08T00:14:28.443 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.443+0000 7fe149c9c700 1 -- start start 2023-10-08T00:14:28.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.443+0000 7fe149c9c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe1440b3430 0x7fe144164e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:28.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.443+0000 7fe149c9c700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe14415db80 con 0x7fe14415a1e0 2023-10-08T00:14:28.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.443+0000 7fe149c9c700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe14415de60 con 0x7fe144159e10 2023-10-08T00:14:28.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.443+0000 7fe149c9c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe14415e800 con 0x7fe1440b3430 2023-10-08T00:14:28.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.444+0000 7fe13affd700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fe14415a1e0 0x7fe1440cd740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:44400/0 (socket says 172.21.15.99:44400) 2023-10-08T00:14:28.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.444+0000 7fe13affd700 1 -- 172.21.15.99:0/608857732 learned_addr learned my addr 172.21.15.99:0/608857732 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:14:28.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.444+0000 7fe143fff700 1 --2- 172.21.15.99:0/608857732 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe1440b3430 0x7fe144164e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:28.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.444+0000 7fe142ffd700 1 -- 172.21.15.99:0/608857732 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1737474426 0 0) 0x7fe14415db80 con 0x7fe14415a1e0 2023-10-08T00:14:28.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.444+0000 7fe142ffd700 1 -- 172.21.15.99:0/608857732 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe130003570 con 0x7fe14415a1e0 2023-10-08T00:14:28.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.444+0000 7fe142ffd700 1 -- 172.21.15.99:0/608857732 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 925435437 0 0) 0x7fe14415de60 con 0x7fe144159e10 2023-10-08T00:14:28.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.444+0000 7fe142ffd700 1 -- 172.21.15.99:0/608857732 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe14415db80 con 0x7fe144159e10 2023-10-08T00:14:28.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.444+0000 7fe142ffd700 1 -- 172.21.15.99:0/608857732 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3017751401 0 0) 0x7fe130003570 con 0x7fe14415a1e0 2023-10-08T00:14:28.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.444+0000 7fe142ffd700 1 -- 172.21.15.99:0/608857732 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe1440b3430 msgr2=0x7fe144164e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:28.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.444+0000 7fe142ffd700 1 --2- 172.21.15.99:0/608857732 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe1440b3430 0x7fe144164e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:28.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.444+0000 7fe142ffd700 1 -- 172.21.15.99:0/608857732 >> v1:172.21.15.191:6789/0 conn(0x7fe144159e10 legacy=0x7fe1440b2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:28.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.444+0000 7fe142ffd700 1 -- 172.21.15.99:0/608857732 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe14415e140 con 0x7fe14415a1e0 2023-10-08T00:14:28.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.444+0000 7fe142ffd700 1 -- 172.21.15.99:0/608857732 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7fe12c001470 con 0x7fe14415a1e0 2023-10-08T00:14:28.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.444+0000 7fe142ffd700 1 -- 172.21.15.99:0/608857732 >> v1:172.21.15.99:6789/0 conn(0x7fe14415a1e0 legacy=0x7fe1440cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:28.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.444+0000 7fe142ffd700 1 --2- 172.21.15.99:0/608857732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe130003c20 0x7fe130004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:28.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.444+0000 7fe142ffd700 1 --2- 172.21.15.99:0/608857732 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe1440b3430 0x7fe130004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:28.449 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.444+0000 7fe142ffd700 1 --2- 172.21.15.99:0/608857732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe130004d50 0x7fe1300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:28.449 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.444+0000 7fe142ffd700 1 -- 172.21.15.99:0/608857732 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe14415e800 con 0x7fe130004d50 2023-10-08T00:14:28.449 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.444+0000 7fe142ffd700 1 -- 172.21.15.99:0/608857732 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13000bb40 con 0x7fe130003c20 2023-10-08T00:14:28.449 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.444+0000 7fe142ffd700 1 -- 172.21.15.99:0/608857732 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe13000bd00 con 0x7fe1440b3430 2023-10-08T00:14:28.449 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.444+0000 7fe13affd700 1 --2- 172.21.15.99:0/608857732 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe1440b3430 0x7fe130004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:28.449 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.444+0000 7fe143fff700 1 --2- 172.21.15.99:0/608857732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe130004d50 0x7fe1300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:28.449 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.444+0000 7fe1437fe700 1 --2- 172.21.15.99:0/608857732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe130003c20 0x7fe130004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:28.449 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.445+0000 7fe13affd700 1 -- 172.21.15.99:0/608857732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe130003c20 msgr2=0x7fe130004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:28.449 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.445+0000 7fe13affd700 1 --2- 172.21.15.99:0/608857732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe130003c20 0x7fe130004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:28.449 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.445+0000 7fe13affd700 1 -- 172.21.15.99:0/608857732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe130004d50 msgr2=0x7fe1300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:28.449 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.445+0000 7fe13affd700 1 --2- 172.21.15.99:0/608857732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe130004d50 0x7fe1300051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:28.449 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.445+0000 7fe13affd700 1 -- 172.21.15.99:0/608857732 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe130003570 con 0x7fe1440b3430 2023-10-08T00:14:28.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.445+0000 7fe143fff700 1 --2- 172.21.15.99:0/608857732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe130004d50 0x7fe1300051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:14:28.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.445+0000 7fe13affd700 1 --2- 172.21.15.99:0/608857732 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe1440b3430 0x7fe130004810 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe14408a0b0 tx=0x7fe12c001e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a943ce7ea04726e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:28.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.445+0000 7fe142ffd700 1 -- 172.21.15.99:0/608857732 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe12c00cb60 con 0x7fe1440b3430 2023-10-08T00:14:28.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.445+0000 7fe142ffd700 1 -- 172.21.15.99:0/608857732 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe12c00ccc0 con 0x7fe1440b3430 2023-10-08T00:14:28.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.445+0000 7fe142ffd700 1 -- 172.21.15.99:0/608857732 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe12c005b70 con 0x7fe1440b3430 2023-10-08T00:14:28.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.445+0000 7fe149c9c700 1 -- 172.21.15.99:0/608857732 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe1440b3430 msgr2=0x7fe130004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:28.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.445+0000 7fe149c9c700 1 --2- 172.21.15.99:0/608857732 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe1440b3430 0x7fe130004810 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe14408a0b0 tx=0x7fe12c001e10 comp rx=0 tx=0).stop 2023-10-08T00:14:28.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.445+0000 7fe143fff700 1 -- 172.21.15.99:0/608857732 reap_dead start 2023-10-08T00:14:28.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.445+0000 7fe149c9c700 1 -- 172.21.15.99:0/608857732 shutdown_connections 2023-10-08T00:14:28.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.445+0000 7fe149c9c700 1 -- 172.21.15.99:0/608857732 >> 172.21.15.99:0/608857732 conn(0x7fe144067660 msgr2=0x7fe144067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:28.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.445+0000 7fe149c9c700 1 -- 172.21.15.99:0/608857732 shutdown_connections 2023-10-08T00:14:28.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.445+0000 7fe149c9c700 1 -- 172.21.15.99:0/608857732 wait complete. 2023-10-08T00:14:28.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.446+0000 7fe149c9c700 1 Processor -- start 2023-10-08T00:14:28.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.447+0000 7fe149c9c700 1 -- start start 2023-10-08T00:14:28.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.447+0000 7fe149c9c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1440b3430 0x7fe1440d17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:28.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.447+0000 7fe149c9c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1440ce870 0x7fe144149fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:28.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.447+0000 7fe149c9c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe14415cd30 0x7fe14415d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:28.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.447+0000 7fe149c9c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1440c5770 con 0x7fe1440ce870 2023-10-08T00:14:28.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.447+0000 7fe149c9c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe14415d6e0 con 0x7fe1440b3430 2023-10-08T00:14:28.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.447+0000 7fe149c9c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1440c4760 con 0x7fe14415cd30 2023-10-08T00:14:28.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.448+0000 7fe13affd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1440ce870 0x7fe144149fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:28.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.448+0000 7fe13affd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1440ce870 0x7fe144149fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43310/0 (socket says 172.21.15.99:43310) 2023-10-08T00:14:28.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.448+0000 7fe13affd700 1 -- 172.21.15.99:0/2582873020 learned_addr learned my addr 172.21.15.99:0/2582873020 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:28.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.448+0000 7fe1437fe700 1 --2- 172.21.15.99:0/2582873020 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1440b3430 0x7fe1440d17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:28.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.448+0000 7fe143fff700 1 --2- 172.21.15.99:0/2582873020 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe14415cd30 0x7fe14415d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:28.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.448+0000 7fe13affd700 1 -- 172.21.15.99:0/2582873020 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe14415cd30 msgr2=0x7fe14415d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:28.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.448+0000 7fe13affd700 1 --2- 172.21.15.99:0/2582873020 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe14415cd30 0x7fe14415d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:28.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.448+0000 7fe13affd700 1 -- 172.21.15.99:0/2582873020 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1440b3430 msgr2=0x7fe1440d17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:28.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.448+0000 7fe13affd700 1 --2- 172.21.15.99:0/2582873020 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1440b3430 0x7fe1440d17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:28.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.448+0000 7fe13affd700 1 -- 172.21.15.99:0/2582873020 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1440c49b0 con 0x7fe1440ce870 2023-10-08T00:14:28.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.448+0000 7fe13affd700 1 --2- 172.21.15.99:0/2582873020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1440ce870 0x7fe144149fb0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe12c00c860 tx=0x7fe12c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:28.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.448+0000 7fe13bfff700 1 -- 172.21.15.99:0/2582873020 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe12c001640 con 0x7fe1440ce870 2023-10-08T00:14:28.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.448+0000 7fe13bfff700 1 -- 172.21.15.99:0/2582873020 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe12c0070d0 con 0x7fe1440ce870 2023-10-08T00:14:28.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.448+0000 7fe149c9c700 1 -- 172.21.15.99:0/2582873020 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1440c4c90 con 0x7fe1440ce870 2023-10-08T00:14:28.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.448+0000 7fe13bfff700 1 -- 172.21.15.99:0/2582873020 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe12c007490 con 0x7fe1440ce870 2023-10-08T00:14:28.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.448+0000 7fe149c9c700 1 -- 172.21.15.99:0/2582873020 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1440c5250 con 0x7fe1440ce870 2023-10-08T00:14:28.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.449+0000 7fe149c9c700 1 -- 172.21.15.99:0/2582873020 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1440c5410 con 0x7fe1440ce870 2023-10-08T00:14:28.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.449+0000 7fe13bfff700 1 -- 172.21.15.99:0/2582873020 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe12c019030 con 0x7fe1440ce870 2023-10-08T00:14:28.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.450+0000 7fe13bfff700 1 --2- 172.21.15.99:0/2582873020 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe124066f30 0x7fe1240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:28.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.450+0000 7fe13bfff700 1 -- 172.21.15.99:0/2582873020 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5763+0+0 (secure 0 0 0) 0x7fe12c085a90 con 0x7fe1440ce870 2023-10-08T00:14:28.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.450+0000 7fe1437fe700 1 --2- 172.21.15.99:0/2582873020 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe124066f30 0x7fe1240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:28.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.450+0000 7fe13bfff700 1 -- 172.21.15.99:0/2582873020 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=75}) v3 -- 0x7fe12406eb10 con 0x7fe1440ce870 2023-10-08T00:14:28.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.450+0000 7fe1437fe700 1 --2- 172.21.15.99:0/2582873020 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe124066f30 0x7fe1240693e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe1340010e0 tx=0x7fe134007d30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:28.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.454+0000 7fe13bfff700 1 -- 172.21.15.99:0/2582873020 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe12c052cc0 con 0x7fe1440ce870 2023-10-08T00:14:28.678 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.678+0000 7fe149c9c700 1 -- 172.21.15.99:0/2582873020 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe14404c3c0 con 0x7fe1440ce870 2023-10-08T00:14:28.679 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.679+0000 7fe13bfff700 1 -- 172.21.15.99:0/2582873020 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4600 (secure 0 0 0) 0x7fe12c017090 con 0x7fe1440ce870 2023-10-08T00:14:28.679 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:14:28.679 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5087,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/862112794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":862112794},{"type":"v1","addr":"172.21.15.191:6837","nonce":862112794}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5673,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/2694493794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2694493794},{"type":"v1","addr":"172.21.15.99:6834","nonce":2694493794}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1906990531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":1906990531},{"type":"v1","addr":"172.21.15.99:6835","nonce":1906990531}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:12:57.581246+0000","modified":"2023-10-08T00:13:01.240449+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5688},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5688":{"gid":5688,"name":"d","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6836/1779510877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":1779510877},{"type":"v1","addr":"172.21.15.191:6836","nonce":1779510877}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-08T00:14:28.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.681+0000 7fe149c9c700 1 -- 172.21.15.99:0/2582873020 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe124066f30 msgr2=0x7fe1240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:28.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.681+0000 7fe149c9c700 1 --2- 172.21.15.99:0/2582873020 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe124066f30 0x7fe1240693e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe1340010e0 tx=0x7fe134007d30 comp rx=0 tx=0).stop 2023-10-08T00:14:28.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.681+0000 7fe149c9c700 1 -- 172.21.15.99:0/2582873020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1440ce870 msgr2=0x7fe144149fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:28.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.681+0000 7fe149c9c700 1 --2- 172.21.15.99:0/2582873020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1440ce870 0x7fe144149fb0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe12c00c860 tx=0x7fe12c015000 comp rx=0 tx=0).stop 2023-10-08T00:14:28.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.681+0000 7fe149c9c700 1 -- 172.21.15.99:0/2582873020 shutdown_connections 2023-10-08T00:14:28.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.681+0000 7fe149c9c700 1 --2- 172.21.15.99:0/2582873020 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe124066f30 0x7fe1240693e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:28.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.681+0000 7fe149c9c700 1 --2- 172.21.15.99:0/2582873020 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1440b3430 0x7fe1440d17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:28.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.681+0000 7fe149c9c700 1 --2- 172.21.15.99:0/2582873020 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe14415cd30 0x7fe14415d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:28.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.681+0000 7fe149c9c700 1 --2- 172.21.15.99:0/2582873020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1440ce870 0x7fe144149fb0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:28.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.681+0000 7fe149c9c700 1 -- 172.21.15.99:0/2582873020 >> 172.21.15.99:0/2582873020 conn(0x7fe144067660 msgr2=0x7fe1440b0780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:28.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.681+0000 7fe149c9c700 1 -- 172.21.15.99:0/2582873020 shutdown_connections 2023-10-08T00:14:28.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.681+0000 7fe149c9c700 1 -- 172.21.15.99:0/2582873020 wait complete. 2023-10-08T00:14:28.683 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 32 2023-10-08T00:14:28.693 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:14:28.808 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.807+0000 7fa36ee2a700 1 Processor -- start 2023-10-08T00:14:28.808 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.807+0000 7fa36ee2a700 1 -- start start 2023-10-08T00:14:28.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.808+0000 7fa36ee2a700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa36815cbd0 0x7fa36815d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:28.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.808+0000 7fa36ee2a700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3680cb7c0 con 0x7fa3680cd6e0 2023-10-08T00:14:28.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.808+0000 7fa36ee2a700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa36815e290 con 0x7fa3680cdab0 2023-10-08T00:14:28.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.808+0000 7fa36ee2a700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa36815e850 con 0x7fa36815cbd0 2023-10-08T00:14:28.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.808+0000 7fa36e629700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa36815cbd0 0x7fa36815d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:28.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.808+0000 7fa36de28700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fa3680cd6e0 0x7fa3680cbc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:44412/0 (socket says 172.21.15.99:44412) 2023-10-08T00:14:28.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.808+0000 7fa36de28700 1 -- 172.21.15.99:0/3448284296 learned_addr learned my addr 172.21.15.99:0/3448284296 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:14:28.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.808+0000 7fa36d627700 1 --1- 172.21.15.99:0/3448284296 >> v1:172.21.15.191:6789/0 conn(0x7fa3680cdab0 0x7fa36815d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:57422/0 (socket says 172.21.15.99:57422) 2023-10-08T00:14:28.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.808+0000 7fa36ce26700 1 -- 172.21.15.99:0/3448284296 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1376169221 0 0) 0x7fa36815e290 con 0x7fa3680cdab0 2023-10-08T00:14:28.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.808+0000 7fa36e629700 1 -- 172.21.15.99:0/3448284296 >> v1:172.21.15.191:6789/0 conn(0x7fa3680cdab0 legacy=0x7fa36815d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:28.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.808+0000 7fa36e629700 1 -- 172.21.15.99:0/3448284296 >> v1:172.21.15.99:6789/0 conn(0x7fa3680cd6e0 legacy=0x7fa3680cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:28.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.808+0000 7fa36e629700 1 -- 172.21.15.99:0/3448284296 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa36815e570 con 0x7fa36815cbd0 2023-10-08T00:14:28.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.808+0000 7fa36e629700 1 --2- 172.21.15.99:0/3448284296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa36815cbd0 0x7fa36815d050 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa3580098e0 tx=0x7fa3580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c14e7fe10c30bff server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:28.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.809+0000 7fa36ce26700 1 -- 172.21.15.99:0/3448284296 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa358009e80 con 0x7fa36815cbd0 2023-10-08T00:14:28.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.809+0000 7fa36ce26700 1 -- 172.21.15.99:0/3448284296 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa35800b760 con 0x7fa36815cbd0 2023-10-08T00:14:28.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.809+0000 7fa36ce26700 1 -- 172.21.15.99:0/3448284296 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35800bae0 con 0x7fa36815cbd0 2023-10-08T00:14:28.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.809+0000 7fa36ee2a700 1 -- 172.21.15.99:0/3448284296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa36815cbd0 msgr2=0x7fa36815d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:28.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.809+0000 7fa36ee2a700 1 --2- 172.21.15.99:0/3448284296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa36815cbd0 0x7fa36815d050 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa3580098e0 tx=0x7fa3580091a0 comp rx=0 tx=0).stop 2023-10-08T00:14:28.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.809+0000 7fa36ee2a700 1 -- 172.21.15.99:0/3448284296 shutdown_connections 2023-10-08T00:14:28.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.809+0000 7fa36ee2a700 1 --2- 172.21.15.99:0/3448284296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa36815cbd0 0x7fa36815d050 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:28.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.809+0000 7fa36ee2a700 1 -- 172.21.15.99:0/3448284296 >> 172.21.15.99:0/3448284296 conn(0x7fa368067660 msgr2=0x7fa368067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:28.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.809+0000 7fa36ee2a700 1 -- 172.21.15.99:0/3448284296 shutdown_connections 2023-10-08T00:14:28.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.809+0000 7fa36ee2a700 1 -- 172.21.15.99:0/3448284296 wait complete. 2023-10-08T00:14:28.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.810+0000 7fa36ee2a700 1 Processor -- start 2023-10-08T00:14:28.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.810+0000 7fa36ee2a700 1 -- start start 2023-10-08T00:14:28.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.811+0000 7fa36ee2a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3680cd6e0 0x7fa3680b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:28.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.811+0000 7fa36ee2a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3680cdab0 0x7fa3680b40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:28.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.811+0000 7fa36ee2a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa36815cbd0 0x7fa36816a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:28.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.811+0000 7fa36ee2a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3680cc360 con 0x7fa3680cd6e0 2023-10-08T00:14:28.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.811+0000 7fa36ee2a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36814d0d0 con 0x7fa3680cdab0 2023-10-08T00:14:28.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.811+0000 7fa36ee2a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3680b4940 con 0x7fa36815cbd0 2023-10-08T00:14:28.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.811+0000 7fa36de28700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3680cd6e0 0x7fa3680b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:28.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.811+0000 7fa36de28700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3680cd6e0 0x7fa3680b3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43316/0 (socket says 172.21.15.99:43316) 2023-10-08T00:14:28.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.811+0000 7fa36de28700 1 -- 172.21.15.99:0/2025292054 learned_addr learned my addr 172.21.15.99:0/2025292054 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:28.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.811+0000 7fa36de28700 1 -- 172.21.15.99:0/2025292054 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa36815cbd0 msgr2=0x7fa36816a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:28.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.811+0000 7fa36de28700 1 --2- 172.21.15.99:0/2025292054 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa36815cbd0 0x7fa36816a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:28.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.811+0000 7fa36de28700 1 -- 172.21.15.99:0/2025292054 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3680cdab0 msgr2=0x7fa3680b40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:28.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.811+0000 7fa36d627700 1 --2- 172.21.15.99:0/2025292054 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3680cdab0 0x7fa3680b40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:28.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.811+0000 7fa36de28700 1 --2- 172.21.15.99:0/2025292054 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3680cdab0 0x7fa3680b40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:28.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.811+0000 7fa36de28700 1 -- 172.21.15.99:0/2025292054 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa358009420 con 0x7fa3680cd6e0 2023-10-08T00:14:28.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.812+0000 7fa36de28700 1 --2- 172.21.15.99:0/2025292054 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3680cd6e0 0x7fa3680b3ba0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa36000cd80 tx=0x7fa360001b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:28.829 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.812+0000 7fa365ffb700 1 -- 172.21.15.99:0/2025292054 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36000ab30 con 0x7fa3680cd6e0 2023-10-08T00:14:28.829 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.812+0000 7fa365ffb700 1 -- 172.21.15.99:0/2025292054 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa36000ac90 con 0x7fa3680cd6e0 2023-10-08T00:14:28.829 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.812+0000 7fa36ee2a700 1 -- 172.21.15.99:0/2025292054 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3680b4c20 con 0x7fa3680cd6e0 2023-10-08T00:14:28.829 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.812+0000 7fa365ffb700 1 -- 172.21.15.99:0/2025292054 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa360007340 con 0x7fa3680cd6e0 2023-10-08T00:14:28.829 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.812+0000 7fa36ee2a700 1 -- 172.21.15.99:0/2025292054 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3680b51e0 con 0x7fa3680cd6e0 2023-10-08T00:14:28.830 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.813+0000 7fa36ee2a700 1 -- 172.21.15.99:0/2025292054 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa368089280 con 0x7fa3680cd6e0 2023-10-08T00:14:28.830 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.813+0000 7fa365ffb700 1 -- 172.21.15.99:0/2025292054 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa360007730 con 0x7fa3680cd6e0 2023-10-08T00:14:28.830 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.813+0000 7fa365ffb700 1 --2- 172.21.15.99:0/2025292054 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa354066f30 0x7fa3540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:28.830 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.814+0000 7fa365ffb700 1 -- 172.21.15.99:0/2025292054 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5763+0+0 (secure 0 0 0) 0x7fa360084a70 con 0x7fa3680cd6e0 2023-10-08T00:14:28.830 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.814+0000 7fa365ffb700 1 -- 172.21.15.99:0/2025292054 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=75}) v3 -- 0x7fa35406eb10 con 0x7fa3680cd6e0 2023-10-08T00:14:28.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.814+0000 7fa36d627700 1 --2- 172.21.15.99:0/2025292054 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa354066f30 0x7fa3540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:28.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.814+0000 7fa36d627700 1 --2- 172.21.15.99:0/2025292054 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa354066f30 0x7fa3540693e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa35c007fd0 tx=0x7fa35c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:28.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:28.818+0000 7fa365ffb700 1 -- 172.21.15.99:0/2025292054 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa360051d20 con 0x7fa3680cd6e0 2023-10-08T00:14:29.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.001+0000 7fa36ee2a700 1 -- 172.21.15.99:0/2025292054 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa36816b5b0 con 0x7fa3680cd6e0 2023-10-08T00:14:29.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.002+0000 7fa365ffb700 1 -- 172.21.15.99:0/2025292054 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+15497 (secure 0 0 0) 0x7fa360055340 con 0x7fa3680cd6e0 2023-10-08T00:14:29.003 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:14:29.003 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":74,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:13:44.132886+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:12:55.353771+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-08T00:12:56.552157+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["backfillfull","exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","full","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","full","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.28","mappings":[{"from":6,"to":4}]},{"pgid":"9.3e","mappings":[{"from":6,"to":1}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:14:29.005 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.004+0000 7fa36ee2a700 1 -- 172.21.15.99:0/2025292054 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa354066f30 msgr2=0x7fa3540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:29.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.004+0000 7fa36ee2a700 1 --2- 172.21.15.99:0/2025292054 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa354066f30 0x7fa3540693e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa35c007fd0 tx=0x7fa35c009040 comp rx=0 tx=0).stop 2023-10-08T00:14:29.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.004+0000 7fa36ee2a700 1 -- 172.21.15.99:0/2025292054 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3680cd6e0 msgr2=0x7fa3680b3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:29.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.004+0000 7fa36ee2a700 1 --2- 172.21.15.99:0/2025292054 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3680cd6e0 0x7fa3680b3ba0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa36000cd80 tx=0x7fa360001b00 comp rx=0 tx=0).stop 2023-10-08T00:14:29.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.004+0000 7fa36ee2a700 1 -- 172.21.15.99:0/2025292054 shutdown_connections 2023-10-08T00:14:29.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.004+0000 7fa36ee2a700 1 --2- 172.21.15.99:0/2025292054 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa354066f30 0x7fa3540693e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:29.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.004+0000 7fa36ee2a700 1 --2- 172.21.15.99:0/2025292054 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa36815cbd0 0x7fa36816a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:29.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.004+0000 7fa36ee2a700 1 --2- 172.21.15.99:0/2025292054 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3680cd6e0 0x7fa3680b3ba0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:29.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.004+0000 7fa36ee2a700 1 --2- 172.21.15.99:0/2025292054 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3680cdab0 0x7fa3680b40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:29.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.004+0000 7fa36ee2a700 1 -- 172.21.15.99:0/2025292054 >> 172.21.15.99:0/2025292054 conn(0x7fa368067660 msgr2=0x7fa3680d10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:29.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.004+0000 7fa36ee2a700 1 -- 172.21.15.99:0/2025292054 shutdown_connections 2023-10-08T00:14:29.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.004+0000 7fa36ee2a700 1 -- 172.21.15.99:0/2025292054 wait complete. 2023-10-08T00:14:29.019 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:10:42.063344+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T00:12:55.353771+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T00:12:56.552157+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T00:14:29.019 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T00:14:29.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.136+0000 7f27ea338700 1 Processor -- start 2023-10-08T00:14:29.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.136+0000 7f27ea338700 1 -- start start 2023-10-08T00:14:29.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.136+0000 7f27ea338700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f27e4159ec0 0x7f27e40af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:29.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.136+0000 7f27ea338700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f27e415a290 0x7f27e40afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:29.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.136+0000 7f27ea338700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27e40b0790 con 0x7f27e406a5a0 2023-10-08T00:14:29.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.136+0000 7f27ea338700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27e40ce1f0 con 0x7f27e4159ec0 2023-10-08T00:14:29.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.136+0000 7f27ea338700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27e40b0b20 con 0x7f27e415a290 2023-10-08T00:14:29.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.136+0000 7f27e3fff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f27e4159ec0 0x7f27e40af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:29.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.136+0000 7f27e88d5700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f27e406a5a0 0x7f27e415db90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:44426/0 (socket says 172.21.15.99:44426) 2023-10-08T00:14:29.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.137+0000 7f27e88d5700 1 -- 172.21.15.99:0/3758290978 learned_addr learned my addr 172.21.15.99:0/3758290978 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:14:29.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.137+0000 7f27e37fe700 1 --2- 172.21.15.99:0/3758290978 >> v2:172.21.15.191:3300/0 conn(0x7f27e415a290 0x7f27e40afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:29.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.137+0000 7f27e3fff700 1 -- 172.21.15.99:0/3758290978 >> v2:172.21.15.191:3300/0 conn(0x7f27e415a290 msgr2=0x7f27e40afe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:29.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.137+0000 7f27e3fff700 1 --2- 172.21.15.99:0/3758290978 >> v2:172.21.15.191:3300/0 conn(0x7f27e415a290 0x7f27e40afe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:29.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.137+0000 7f27e3fff700 1 -- 172.21.15.99:0/3758290978 >> v1:172.21.15.99:6789/0 conn(0x7f27e406a5a0 legacy=0x7f27e415db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:29.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.137+0000 7f27e2ffd700 1 -- 172.21.15.99:0/3758290978 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 578756213 0 0) 0x7f27e40b0790 con 0x7f27e406a5a0 2023-10-08T00:14:29.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.137+0000 7f27e3fff700 1 -- 172.21.15.99:0/3758290978 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e40b04b0 con 0x7f27e4159ec0 2023-10-08T00:14:29.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.137+0000 7f27e3fff700 1 --2- 172.21.15.99:0/3758290978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27e4159ec0 0x7f27e40af940 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f27e4089910 tx=0x7f27d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=789926b93b19ef5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:29.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.137+0000 7f27e2ffd700 1 -- 172.21.15.99:0/3758290978 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d400b6b0 con 0x7f27e4159ec0 2023-10-08T00:14:29.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.137+0000 7f27e2ffd700 1 -- 172.21.15.99:0/3758290978 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27d400b810 con 0x7f27e4159ec0 2023-10-08T00:14:29.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.137+0000 7f27e2ffd700 1 -- 172.21.15.99:0/3758290978 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d400bbb0 con 0x7f27e4159ec0 2023-10-08T00:14:29.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.138+0000 7f27ea338700 1 -- 172.21.15.99:0/3758290978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27e4159ec0 msgr2=0x7f27e40af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:29.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.138+0000 7f27ea338700 1 --2- 172.21.15.99:0/3758290978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27e4159ec0 0x7f27e40af940 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f27e4089910 tx=0x7f27d40091a0 comp rx=0 tx=0).stop 2023-10-08T00:14:29.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.138+0000 7f27ea338700 1 -- 172.21.15.99:0/3758290978 shutdown_connections 2023-10-08T00:14:29.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.138+0000 7f27ea338700 1 --2- 172.21.15.99:0/3758290978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27e4159ec0 0x7f27e40af940 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:29.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.138+0000 7f27ea338700 1 --2- 172.21.15.99:0/3758290978 >> v2:172.21.15.191:3300/0 conn(0x7f27e415a290 0x7f27e40afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:29.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.138+0000 7f27ea338700 1 -- 172.21.15.99:0/3758290978 >> 172.21.15.99:0/3758290978 conn(0x7f27e40d1880 msgr2=0x7f27e40cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:29.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.138+0000 7f27ea338700 1 -- 172.21.15.99:0/3758290978 shutdown_connections 2023-10-08T00:14:29.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.138+0000 7f27ea338700 1 -- 172.21.15.99:0/3758290978 wait complete. 2023-10-08T00:14:29.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.139+0000 7f27ea338700 1 Processor -- start 2023-10-08T00:14:29.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.139+0000 7f27ea338700 1 -- start start 2023-10-08T00:14:29.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.140+0000 7f27ea338700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27e406a5a0 0x7f27e414c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:29.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.140+0000 7f27ea338700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27e4159ec0 0x7f27e4156710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:29.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.140+0000 7f27ea338700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27e415a290 0x7f27e40b4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:29.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.140+0000 7f27ea338700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e40b3b50 con 0x7f27e415a290 2023-10-08T00:14:29.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.140+0000 7f27ea338700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e40b46d0 con 0x7f27e406a5a0 2023-10-08T00:14:29.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.140+0000 7f27ea338700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27e40b3da0 con 0x7f27e4159ec0 2023-10-08T00:14:29.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.140+0000 7f27e88d5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27e415a290 0x7f27e40b4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:29.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.140+0000 7f27e88d5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27e415a290 0x7f27e40b4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43328/0 (socket says 172.21.15.99:43328) 2023-10-08T00:14:29.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.140+0000 7f27e88d5700 1 -- 172.21.15.99:0/525470741 learned_addr learned my addr 172.21.15.99:0/525470741 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:29.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.140+0000 7f27e37fe700 1 --2- 172.21.15.99:0/525470741 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27e4159ec0 0x7f27e4156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:29.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.140+0000 7f27e3fff700 1 --2- 172.21.15.99:0/525470741 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27e406a5a0 0x7f27e414c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:29.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.140+0000 7f27e88d5700 1 -- 172.21.15.99:0/525470741 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27e4159ec0 msgr2=0x7f27e4156710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:29.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.140+0000 7f27e88d5700 1 --2- 172.21.15.99:0/525470741 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27e4159ec0 0x7f27e4156710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:29.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.140+0000 7f27e88d5700 1 -- 172.21.15.99:0/525470741 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27e406a5a0 msgr2=0x7f27e414c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:29.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.140+0000 7f27e88d5700 1 --2- 172.21.15.99:0/525470741 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27e406a5a0 0x7f27e414c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:29.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.140+0000 7f27e88d5700 1 -- 172.21.15.99:0/525470741 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27d4009420 con 0x7f27e415a290 2023-10-08T00:14:29.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.140+0000 7f27e37fe700 1 --2- 172.21.15.99:0/525470741 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27e4159ec0 0x7f27e4156710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:14:29.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.140+0000 7f27e3fff700 1 --2- 172.21.15.99:0/525470741 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27e406a5a0 0x7f27e414c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:14:29.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.140+0000 7f27e88d5700 1 --2- 172.21.15.99:0/525470741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27e415a290 0x7f27e40b4b60 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f27dc00ccc0 tx=0x7f27dc001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:29.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.140+0000 7f27cbfff700 1 -- 172.21.15.99:0/525470741 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27dc00aaf0 con 0x7f27e415a290 2023-10-08T00:14:29.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.140+0000 7f27cbfff700 1 -- 172.21.15.99:0/525470741 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27dc00ac50 con 0x7f27e415a290 2023-10-08T00:14:29.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.140+0000 7f27ea338700 1 -- 172.21.15.99:0/525470741 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27e40b4080 con 0x7f27e415a290 2023-10-08T00:14:29.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.140+0000 7f27cbfff700 1 -- 172.21.15.99:0/525470741 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27dc0070f0 con 0x7f27e415a290 2023-10-08T00:14:29.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.140+0000 7f27ea338700 1 -- 172.21.15.99:0/525470741 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27e414cdc0 con 0x7f27e415a290 2023-10-08T00:14:29.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.141+0000 7f27ea338700 1 -- 172.21.15.99:0/525470741 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27e40b4860 con 0x7f27e415a290 2023-10-08T00:14:29.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.142+0000 7f27cbfff700 1 -- 172.21.15.99:0/525470741 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f27dc0074a0 con 0x7f27e415a290 2023-10-08T00:14:29.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.142+0000 7f27cbfff700 1 --2- 172.21.15.99:0/525470741 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f27d0066f30 0x7f27d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:29.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.142+0000 7f27cbfff700 1 -- 172.21.15.99:0/525470741 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f27dc0849a0 con 0x7f27e415a290 2023-10-08T00:14:29.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.142+0000 7f27e3fff700 1 --2- 172.21.15.99:0/525470741 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f27d0066f30 0x7f27d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:29.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.142+0000 7f27cbfff700 1 -- 172.21.15.99:0/525470741 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=75}) v3 -- 0x7f27d006eb10 con 0x7f27e415a290 2023-10-08T00:14:29.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.143+0000 7f27e3fff700 1 --2- 172.21.15.99:0/525470741 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f27d0066f30 0x7f27d00693e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f27d40093f0 tx=0x7f27d4017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:29.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.146+0000 7f27cbfff700 1 -- 172.21.15.99:0/525470741 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f27dc051c50 con 0x7f27e415a290 2023-10-08T00:14:29.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.328+0000 7f27ea338700 1 -- 172.21.15.99:0/525470741 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f27e404c3c0 con 0x7f27e415a290 2023-10-08T00:14:29.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.328+0000 7f27cbfff700 1 -- 172.21.15.99:0/525470741 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) v1 ==== 93+0+10 (secure 0 0 0) 0x7f27dc055270 con 0x7f27e415a290 2023-10-08T00:14:29.329 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2023-10-08T00:14:29.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.330+0000 7f27ea338700 1 -- 172.21.15.99:0/525470741 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f27d0066f30 msgr2=0x7f27d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:29.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.330+0000 7f27ea338700 1 --2- 172.21.15.99:0/525470741 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f27d0066f30 0x7f27d00693e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f27d40093f0 tx=0x7f27d4017000 comp rx=0 tx=0).stop 2023-10-08T00:14:29.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.330+0000 7f27ea338700 1 -- 172.21.15.99:0/525470741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27e415a290 msgr2=0x7f27e40b4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:29.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.330+0000 7f27ea338700 1 --2- 172.21.15.99:0/525470741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27e415a290 0x7f27e40b4b60 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f27dc00ccc0 tx=0x7f27dc001a20 comp rx=0 tx=0).stop 2023-10-08T00:14:29.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.330+0000 7f27ea338700 1 -- 172.21.15.99:0/525470741 shutdown_connections 2023-10-08T00:14:29.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.330+0000 7f27ea338700 1 --2- 172.21.15.99:0/525470741 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f27d0066f30 0x7f27d00693e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:29.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.330+0000 7f27ea338700 1 --2- 172.21.15.99:0/525470741 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27e406a5a0 0x7f27e414c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:29.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.330+0000 7f27ea338700 1 --2- 172.21.15.99:0/525470741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27e415a290 0x7f27e40b4b60 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:29.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.330+0000 7f27ea338700 1 --2- 172.21.15.99:0/525470741 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27e4159ec0 0x7f27e4156710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:29.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.330+0000 7f27ea338700 1 -- 172.21.15.99:0/525470741 >> 172.21.15.99:0/525470741 conn(0x7f27e40d1880 msgr2=0x7f27e414d750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:29.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.330+0000 7f27ea338700 1 -- 172.21.15.99:0/525470741 shutdown_connections 2023-10-08T00:14:29.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.330+0000 7f27ea338700 1 -- 172.21.15.99:0/525470741 wait complete. 2023-10-08T00:14:29.343 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T00:14:29.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.458+0000 7ff7be8f2700 1 Processor -- start 2023-10-08T00:14:29.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.459+0000 7ff7be8f2700 1 -- start start 2023-10-08T00:14:29.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.459+0000 7ff7be8f2700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff7b815b9a0 0x7ff7b80caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:29.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.459+0000 7ff7be8f2700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7b815c8d0 con 0x7ff7b8152300 2023-10-08T00:14:29.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.459+0000 7ff7be8f2700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7b80c9370 con 0x7ff7b806d370 2023-10-08T00:14:29.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.459+0000 7ff7be8f2700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7b80b1400 con 0x7ff7b815b9a0 2023-10-08T00:14:29.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.459+0000 7ff7b77fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff7b815b9a0 0x7ff7b80caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:29.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.459+0000 7ff7b7fff700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7ff7b8152300 0x7ff7b8158df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:44438/0 (socket says 172.21.15.99:44438) 2023-10-08T00:14:29.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.459+0000 7ff7b7fff700 1 -- 172.21.15.99:0/1876543854 learned_addr learned my addr 172.21.15.99:0/1876543854 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:14:29.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.460+0000 7ff7b6ffd700 1 -- 172.21.15.99:0/1876543854 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1266407595 0 0) 0x7ff7b815c8d0 con 0x7ff7b8152300 2023-10-08T00:14:29.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.460+0000 7ff7b6ffd700 1 -- 172.21.15.99:0/1876543854 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7a0003570 con 0x7ff7b8152300 2023-10-08T00:14:29.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.460+0000 7ff7b77fe700 1 -- 172.21.15.99:0/1876543854 >> v1:172.21.15.191:6789/0 conn(0x7ff7b806d370 legacy=0x7ff7b806d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:29.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.460+0000 7ff7b77fe700 1 -- 172.21.15.99:0/1876543854 >> v1:172.21.15.99:6789/0 conn(0x7ff7b8152300 legacy=0x7ff7b8158df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:29.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.460+0000 7ff7b77fe700 1 -- 172.21.15.99:0/1876543854 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b815c030 con 0x7ff7b815b9a0 2023-10-08T00:14:29.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.460+0000 7ff7b77fe700 1 --2- 172.21.15.99:0/1876543854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7b815b9a0 0x7ff7b80caaf0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac0098e0 tx=0x7ff7ac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e78fe50b91e5deb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:29.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.460+0000 7ff7b6ffd700 1 -- 172.21.15.99:0/1876543854 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac00b640 con 0x7ff7b815b9a0 2023-10-08T00:14:29.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.460+0000 7ff7b6ffd700 1 -- 172.21.15.99:0/1876543854 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7ac00b7a0 con 0x7ff7b815b9a0 2023-10-08T00:14:29.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.460+0000 7ff7b6ffd700 1 -- 172.21.15.99:0/1876543854 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac00bb40 con 0x7ff7b815b9a0 2023-10-08T00:14:29.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.461+0000 7ff7be8f2700 1 -- 172.21.15.99:0/1876543854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7b815b9a0 msgr2=0x7ff7b80caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:29.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.461+0000 7ff7be8f2700 1 --2- 172.21.15.99:0/1876543854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7b815b9a0 0x7ff7b80caaf0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac0098e0 tx=0x7ff7ac0091a0 comp rx=0 tx=0).stop 2023-10-08T00:14:29.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.461+0000 7ff7be8f2700 1 -- 172.21.15.99:0/1876543854 shutdown_connections 2023-10-08T00:14:29.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.461+0000 7ff7be8f2700 1 --2- 172.21.15.99:0/1876543854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7b815b9a0 0x7ff7b80caaf0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:29.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.461+0000 7ff7be8f2700 1 -- 172.21.15.99:0/1876543854 >> 172.21.15.99:0/1876543854 conn(0x7ff7b8069870 msgr2=0x7ff7b8069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:29.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.461+0000 7ff7be8f2700 1 -- 172.21.15.99:0/1876543854 shutdown_connections 2023-10-08T00:14:29.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.461+0000 7ff7be8f2700 1 -- 172.21.15.99:0/1876543854 wait complete. 2023-10-08T00:14:29.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.462+0000 7ff7be8f2700 1 Processor -- start 2023-10-08T00:14:29.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.462+0000 7ff7be8f2700 1 -- start start 2023-10-08T00:14:29.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.463+0000 7ff7be8f2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7b806d370 0x7ff7b813f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:29.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.463+0000 7ff7be8f2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7b8152300 0x7ff7b813fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:29.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.463+0000 7ff7be8f2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7b815b9a0 0x7ff7b80c6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:29.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.463+0000 7ff7be8f2700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b8140f60 con 0x7ff7b806d370 2023-10-08T00:14:29.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.463+0000 7ff7be8f2700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b8150b90 con 0x7ff7b8152300 2023-10-08T00:14:29.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.463+0000 7ff7be8f2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b81402e0 con 0x7ff7b815b9a0 2023-10-08T00:14:29.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.463+0000 7ff7b7fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7b806d370 0x7ff7b813f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:29.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.463+0000 7ff7b7fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7b806d370 0x7ff7b813f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43332/0 (socket says 172.21.15.99:43332) 2023-10-08T00:14:29.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.463+0000 7ff7b7fff700 1 -- 172.21.15.99:0/1637366227 learned_addr learned my addr 172.21.15.99:0/1637366227 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:29.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.463+0000 7ff7b77fe700 1 --2- 172.21.15.99:0/1637366227 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7b8152300 0x7ff7b813fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:29.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.463+0000 7ff7b7fff700 1 -- 172.21.15.99:0/1637366227 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7b815b9a0 msgr2=0x7ff7b80c6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:29.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.463+0000 7ff7b7fff700 1 --2- 172.21.15.99:0/1637366227 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7b815b9a0 0x7ff7b80c6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:29.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.463+0000 7ff7b7fff700 1 -- 172.21.15.99:0/1637366227 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7b8152300 msgr2=0x7ff7b813fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:29.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.463+0000 7ff7b7fff700 1 --2- 172.21.15.99:0/1637366227 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7b8152300 0x7ff7b813fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:29.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.463+0000 7ff7b7fff700 1 -- 172.21.15.99:0/1637366227 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7ac009420 con 0x7ff7b806d370 2023-10-08T00:14:29.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.463+0000 7ff7b7fff700 1 --2- 172.21.15.99:0/1637366227 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7b806d370 0x7ff7b813f740 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7ff7a800c7c0 tx=0x7ff7a8007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:29.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.463+0000 7ff79ffff700 1 -- 172.21.15.99:0/1637366227 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a8005b30 con 0x7ff7b806d370 2023-10-08T00:14:29.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.463+0000 7ff79ffff700 1 -- 172.21.15.99:0/1637366227 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7a8005c90 con 0x7ff7b806d370 2023-10-08T00:14:29.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.463+0000 7ff7be8f2700 1 -- 172.21.15.99:0/1637366227 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7b81405c0 con 0x7ff7b806d370 2023-10-08T00:14:29.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.463+0000 7ff79ffff700 1 -- 172.21.15.99:0/1637366227 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a8007340 con 0x7ff7b806d370 2023-10-08T00:14:29.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.463+0000 7ff7be8f2700 1 -- 172.21.15.99:0/1637366227 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7b8140b80 con 0x7ff7b806d370 2023-10-08T00:14:29.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.464+0000 7ff7be8f2700 1 -- 172.21.15.99:0/1637366227 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7b804c3c0 con 0x7ff7b806d370 2023-10-08T00:14:29.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.464+0000 7ff79ffff700 1 -- 172.21.15.99:0/1637366227 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff7a801b400 con 0x7ff7b806d370 2023-10-08T00:14:29.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.465+0000 7ff79ffff700 1 --2- 172.21.15.99:0/1637366227 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff7a4066f30 0x7ff7a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:29.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.465+0000 7ff79ffff700 1 -- 172.21.15.99:0/1637366227 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5763+0+0 (secure 0 0 0) 0x7ff7a80840c0 con 0x7ff7b806d370 2023-10-08T00:14:29.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.465+0000 7ff79ffff700 1 -- 172.21.15.99:0/1637366227 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=75}) v3 -- 0x7ff7a406eb10 con 0x7ff7b806d370 2023-10-08T00:14:29.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.465+0000 7ff7b77fe700 1 --2- 172.21.15.99:0/1637366227 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff7a4066f30 0x7ff7a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:29.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.465+0000 7ff7b77fe700 1 --2- 172.21.15.99:0/1637366227 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff7a4066f30 0x7ff7a40693e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac003080 tx=0x7ff7ac017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:29.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.469+0000 7ff79ffff700 1 -- 172.21.15.99:0/1637366227 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff7a8051370 con 0x7ff7b806d370 2023-10-08T00:14:29.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.650+0000 7ff7be8f2700 1 -- 172.21.15.99:0/1637366227 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff7b80cfa20 con 0x7ff7b806d370 2023-10-08T00:14:29.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.651+0000 7ff79ffff700 1 -- 172.21.15.99:0/1637366227 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v74) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff7a8054990 con 0x7ff7b806d370 2023-10-08T00:14:29.651 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2023-10-08T00:14:29.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.652+0000 7ff7be8f2700 1 -- 172.21.15.99:0/1637366227 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff7a4066f30 msgr2=0x7ff7a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:29.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.652+0000 7ff7be8f2700 1 --2- 172.21.15.99:0/1637366227 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff7a4066f30 0x7ff7a40693e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac003080 tx=0x7ff7ac017000 comp rx=0 tx=0).stop 2023-10-08T00:14:29.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.652+0000 7ff7be8f2700 1 -- 172.21.15.99:0/1637366227 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7b806d370 msgr2=0x7ff7b813f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:29.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.652+0000 7ff7be8f2700 1 --2- 172.21.15.99:0/1637366227 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7b806d370 0x7ff7b813f740 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7ff7a800c7c0 tx=0x7ff7a8007e00 comp rx=0 tx=0).stop 2023-10-08T00:14:29.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.652+0000 7ff7be8f2700 1 -- 172.21.15.99:0/1637366227 shutdown_connections 2023-10-08T00:14:29.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.652+0000 7ff7be8f2700 1 --2- 172.21.15.99:0/1637366227 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff7a4066f30 0x7ff7a40693e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:29.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.652+0000 7ff7be8f2700 1 --2- 172.21.15.99:0/1637366227 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7b815b9a0 0x7ff7b80c6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:29.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.652+0000 7ff7be8f2700 1 --2- 172.21.15.99:0/1637366227 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7b806d370 0x7ff7b813f740 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:29.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.652+0000 7ff7be8f2700 1 --2- 172.21.15.99:0/1637366227 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7b8152300 0x7ff7b813fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:29.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.652+0000 7ff7be8f2700 1 -- 172.21.15.99:0/1637366227 >> 172.21.15.99:0/1637366227 conn(0x7ff7b8069870 msgr2=0x7ff7b8151690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:29.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.653+0000 7ff7be8f2700 1 -- 172.21.15.99:0/1637366227 shutdown_connections 2023-10-08T00:14:29.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.653+0000 7ff7be8f2700 1 -- 172.21.15.99:0/1637366227 wait complete. 2023-10-08T00:14:29.665 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T00:14:29.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.781+0000 7fda02434700 1 Processor -- start 2023-10-08T00:14:29.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.781+0000 7fda02434700 1 -- start start 2023-10-08T00:14:29.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.782+0000 7fda02434700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd9fc152300 0x7fd9fc15b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:29.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.782+0000 7fda02434700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd9fc06d370 0x7fd9fc06d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:29.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.782+0000 7fda02434700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9fc15c8d0 con 0x7fd9fc158df0 2023-10-08T00:14:29.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.782+0000 7fda02434700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9fc0b1400 con 0x7fd9fc152300 2023-10-08T00:14:29.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.782+0000 7fda02434700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9fc0c9370 con 0x7fd9fc06d370 2023-10-08T00:14:29.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.782+0000 7fd9fbfff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd9fc152300 0x7fd9fc15b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:29.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.782+0000 7fd9fbfff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd9fc152300 0x7fd9fc15b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43334/0 (socket says 172.21.15.99:43334) 2023-10-08T00:14:29.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.782+0000 7fd9fbfff700 1 -- 172.21.15.99:0/1653452281 learned_addr learned my addr 172.21.15.99:0/1653452281 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:29.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.782+0000 7fd9fbfff700 1 -- 172.21.15.99:0/1653452281 >> v2:172.21.15.191:3300/0 conn(0x7fd9fc06d370 msgr2=0x7fd9fc06d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:29.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.782+0000 7fda009d1700 1 --2- 172.21.15.99:0/1653452281 >> v2:172.21.15.191:3300/0 conn(0x7fd9fc06d370 0x7fd9fc06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:29.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.782+0000 7fd9fbfff700 1 --2- 172.21.15.99:0/1653452281 >> v2:172.21.15.191:3300/0 conn(0x7fd9fc06d370 0x7fd9fc06d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:29.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.782+0000 7fd9fbfff700 1 -- 172.21.15.99:0/1653452281 >> v1:172.21.15.191:6789/0 conn(0x7fd9fc158df0 legacy=0x7fd9fc0caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:29.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.783+0000 7fd9fbfff700 1 -- 172.21.15.99:0/1653452281 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9fc15c030 con 0x7fd9fc152300 2023-10-08T00:14:29.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.783+0000 7fd9fbfff700 1 --2- 172.21.15.99:0/1653452281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9fc152300 0x7fd9fc15b9a0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc0cc4d0 tx=0x7fd9ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aff8af091f76f54b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:29.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.783+0000 7fd9faffd700 1 -- 172.21.15.99:0/1653452281 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec00b820 con 0x7fd9fc152300 2023-10-08T00:14:29.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.783+0000 7fd9faffd700 1 -- 172.21.15.99:0/1653452281 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9ec00b980 con 0x7fd9fc152300 2023-10-08T00:14:29.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.783+0000 7fd9faffd700 1 -- 172.21.15.99:0/1653452281 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec00bd20 con 0x7fd9fc152300 2023-10-08T00:14:29.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.783+0000 7fda02434700 1 -- 172.21.15.99:0/1653452281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9fc152300 msgr2=0x7fd9fc15b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:29.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.783+0000 7fda02434700 1 --2- 172.21.15.99:0/1653452281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9fc152300 0x7fd9fc15b9a0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc0cc4d0 tx=0x7fd9ec009250 comp rx=0 tx=0).stop 2023-10-08T00:14:29.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.783+0000 7fda02434700 1 -- 172.21.15.99:0/1653452281 shutdown_connections 2023-10-08T00:14:29.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.783+0000 7fda02434700 1 --2- 172.21.15.99:0/1653452281 >> v2:172.21.15.191:3300/0 conn(0x7fd9fc06d370 0x7fd9fc06d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:29.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.783+0000 7fda02434700 1 --2- 172.21.15.99:0/1653452281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9fc152300 0x7fd9fc15b9a0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:29.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.783+0000 7fda02434700 1 -- 172.21.15.99:0/1653452281 >> 172.21.15.99:0/1653452281 conn(0x7fd9fc069870 msgr2=0x7fd9fc069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:29.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.783+0000 7fda02434700 1 -- 172.21.15.99:0/1653452281 shutdown_connections 2023-10-08T00:14:29.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.783+0000 7fda02434700 1 -- 172.21.15.99:0/1653452281 wait complete. 2023-10-08T00:14:29.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.784+0000 7fda02434700 1 Processor -- start 2023-10-08T00:14:29.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.785+0000 7fda02434700 1 -- start start 2023-10-08T00:14:29.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.785+0000 7fda02434700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9fc06d370 0x7fd9fc151850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:29.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.785+0000 7fda02434700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9fc152300 0x7fd9fc0c6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:29.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.785+0000 7fda02434700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9fc158df0 0x7fd9fc0c6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:29.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.785+0000 7fda02434700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9fc0c7d50 con 0x7fd9fc06d370 2023-10-08T00:14:29.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.785+0000 7fda02434700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9fc151d90 con 0x7fd9fc158df0 2023-10-08T00:14:29.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.785+0000 7fda02434700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9fc0cb570 con 0x7fd9fc152300 2023-10-08T00:14:29.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.785+0000 7fd9fbfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9fc06d370 0x7fd9fc151850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:29.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.786+0000 7fd9fbfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9fc06d370 0x7fd9fc151850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43342/0 (socket says 172.21.15.99:43342) 2023-10-08T00:14:29.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.786+0000 7fd9fbfff700 1 -- 172.21.15.99:0/16980095 learned_addr learned my addr 172.21.15.99:0/16980095 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:29.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.786+0000 7fda009d1700 1 --2- 172.21.15.99:0/16980095 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9fc158df0 0x7fd9fc0c6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:29.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.786+0000 7fd9fb7fe700 1 --2- 172.21.15.99:0/16980095 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9fc152300 0x7fd9fc0c6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:29.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.786+0000 7fd9fbfff700 1 -- 172.21.15.99:0/16980095 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9fc152300 msgr2=0x7fd9fc0c6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:29.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.786+0000 7fd9fbfff700 1 --2- 172.21.15.99:0/16980095 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9fc152300 0x7fd9fc0c6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:29.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.786+0000 7fd9fbfff700 1 -- 172.21.15.99:0/16980095 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9fc158df0 msgr2=0x7fd9fc0c6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:29.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.786+0000 7fd9fbfff700 1 --2- 172.21.15.99:0/16980095 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9fc158df0 0x7fd9fc0c6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:29.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.786+0000 7fd9fbfff700 1 -- 172.21.15.99:0/16980095 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9ec0094f0 con 0x7fd9fc06d370 2023-10-08T00:14:29.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.786+0000 7fd9fbfff700 1 --2- 172.21.15.99:0/16980095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9fc06d370 0x7fd9fc151850 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec009290 tx=0x7fd9ec016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:29.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.786+0000 7fd9e3fff700 1 -- 172.21.15.99:0/16980095 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec011cc0 con 0x7fd9fc06d370 2023-10-08T00:14:29.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.786+0000 7fd9e3fff700 1 -- 172.21.15.99:0/16980095 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9ec020070 con 0x7fd9fc06d370 2023-10-08T00:14:29.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.786+0000 7fd9e3fff700 1 -- 172.21.15.99:0/16980095 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec01b6a0 con 0x7fd9fc06d370 2023-10-08T00:14:29.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.786+0000 7fda02434700 1 -- 172.21.15.99:0/16980095 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9fc0cb850 con 0x7fd9fc06d370 2023-10-08T00:14:29.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.786+0000 7fda02434700 1 -- 172.21.15.99:0/16980095 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9fc0cbe10 con 0x7fd9fc06d370 2023-10-08T00:14:29.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.787+0000 7fda02434700 1 -- 172.21.15.99:0/16980095 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9fc04c3c0 con 0x7fd9fc06d370 2023-10-08T00:14:29.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.787+0000 7fd9e3fff700 1 -- 172.21.15.99:0/16980095 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd9ec022030 con 0x7fd9fc06d370 2023-10-08T00:14:29.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.788+0000 7fd9e3fff700 1 --2- 172.21.15.99:0/16980095 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd9e8066f30 0x7fd9e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:29.792 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.788+0000 7fd9e3fff700 1 -- 172.21.15.99:0/16980095 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5763+0+0 (secure 0 0 0) 0x7fd9ec086420 con 0x7fd9fc06d370 2023-10-08T00:14:29.792 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.788+0000 7fd9fb7fe700 1 --2- 172.21.15.99:0/16980095 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd9e8066f30 0x7fd9e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:29.792 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.788+0000 7fd9e3fff700 1 -- 172.21.15.99:0/16980095 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=75}) v3 -- 0x7fd9e806eb10 con 0x7fd9fc06d370 2023-10-08T00:14:29.792 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.788+0000 7fd9fb7fe700 1 --2- 172.21.15.99:0/16980095 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd9e8066f30 0x7fd9e80693e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc0b28e0 tx=0x7fd9f00090f0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:29.792 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.792+0000 7fd9e3fff700 1 -- 172.21.15.99:0/16980095 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd9ec053650 con 0x7fd9fc06d370 2023-10-08T00:14:29.972 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.971+0000 7fda02434700 1 -- 172.21.15.99:0/16980095 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd9fc04c3c0 con 0x7fd9fc06d370 2023-10-08T00:14:29.972 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.972+0000 7fd9e3fff700 1 -- 172.21.15.99:0/16980095 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v74) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd9ec056c70 con 0x7fd9fc06d370 2023-10-08T00:14:29.973 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2023-10-08T00:14:29.975 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.974+0000 7fda02434700 1 -- 172.21.15.99:0/16980095 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd9e8066f30 msgr2=0x7fd9e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:29.975 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.974+0000 7fda02434700 1 --2- 172.21.15.99:0/16980095 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd9e8066f30 0x7fd9e80693e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc0b28e0 tx=0x7fd9f00090f0 comp rx=0 tx=0).stop 2023-10-08T00:14:29.975 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.974+0000 7fda02434700 1 -- 172.21.15.99:0/16980095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9fc06d370 msgr2=0x7fd9fc151850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:29.975 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.974+0000 7fda02434700 1 --2- 172.21.15.99:0/16980095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9fc06d370 0x7fd9fc151850 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec009290 tx=0x7fd9ec016000 comp rx=0 tx=0).stop 2023-10-08T00:14:29.975 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.974+0000 7fda02434700 1 -- 172.21.15.99:0/16980095 shutdown_connections 2023-10-08T00:14:29.975 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.974+0000 7fda02434700 1 --2- 172.21.15.99:0/16980095 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd9e8066f30 0x7fd9e80693e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:29.975 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.974+0000 7fda02434700 1 --2- 172.21.15.99:0/16980095 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9fc158df0 0x7fd9fc0c6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:29.975 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.974+0000 7fda02434700 1 --2- 172.21.15.99:0/16980095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9fc06d370 0x7fd9fc151850 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:29.976 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.974+0000 7fda02434700 1 --2- 172.21.15.99:0/16980095 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9fc152300 0x7fd9fc0c6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:29.976 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.974+0000 7fda02434700 1 -- 172.21.15.99:0/16980095 >> 172.21.15.99:0/16980095 conn(0x7fd9fc069870 msgr2=0x7fd9fc15a320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:29.976 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.974+0000 7fda02434700 1 -- 172.21.15.99:0/16980095 shutdown_connections 2023-10-08T00:14:29.976 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:29.974+0000 7fda02434700 1 -- 172.21.15.99:0/16980095 wait complete. 2023-10-08T00:14:29.986 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:14:30.101 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.100+0000 7fb5a55a0700 1 Processor -- start 2023-10-08T00:14:30.101 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.101+0000 7fb5a55a0700 1 -- start start 2023-10-08T00:14:30.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.101+0000 7fb5a55a0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb5a01577d0 0x7fb5a00cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:30.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.101+0000 7fb5a55a0700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb5a006ac30 0x7fb5a015c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:30.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.101+0000 7fb5a55a0700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5a015c360 con 0x7fb5a0157ba0 2023-10-08T00:14:30.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.101+0000 7fb5a55a0700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5a015b790 con 0x7fb5a006ac30 2023-10-08T00:14:30.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.101+0000 7fb5a55a0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a015ba30 con 0x7fb5a01577d0 2023-10-08T00:14:30.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.102+0000 7fb5967fc700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fb5a0157ba0 0x7fb5a006a520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:44448/0 (socket says 172.21.15.99:44448) 2023-10-08T00:14:30.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.102+0000 7fb5967fc700 1 -- 172.21.15.99:0/3107267794 learned_addr learned my addr 172.21.15.99:0/3107267794 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:14:30.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.101+0000 7fb59f7fe700 1 --2- 172.21.15.99:0/3107267794 >> v2:172.21.15.99:3300/0 conn(0x7fb5a006ac30 0x7fb5a015c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:30.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.102+0000 7fb59effd700 1 --2- 172.21.15.99:0/3107267794 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb5a01577d0 0x7fb5a00cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:30.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.102+0000 7fb59f7fe700 1 -- 172.21.15.99:0/3107267794 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb5a01577d0 msgr2=0x7fb5a00cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:30.105 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.102+0000 7fb59f7fe700 1 --2- 172.21.15.99:0/3107267794 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb5a01577d0 0x7fb5a00cd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:30.105 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.102+0000 7fb59f7fe700 1 -- 172.21.15.99:0/3107267794 >> v1:172.21.15.99:6789/0 conn(0x7fb5a0157ba0 legacy=0x7fb5a006a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:30.105 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.102+0000 7fb59f7fe700 1 -- 172.21.15.99:0/3107267794 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5a015bd10 con 0x7fb5a006ac30 2023-10-08T00:14:30.105 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.102+0000 7fb59e7fc700 1 -- 172.21.15.99:0/3107267794 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2078753398 0 0) 0x7fb5a015c360 con 0x7fb5a0157ba0 2023-10-08T00:14:30.105 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.102+0000 7fb59f7fe700 1 --2- 172.21.15.99:0/3107267794 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5a006ac30 0x7fb5a015c7f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb598009e60 tx=0x7fb598009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bc8005f60025ed9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:30.105 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.102+0000 7fb59e7fc700 1 -- 172.21.15.99:0/3107267794 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59800e070 con 0x7fb5a006ac30 2023-10-08T00:14:30.105 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.102+0000 7fb59e7fc700 1 -- 172.21.15.99:0/3107267794 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb59800bc30 con 0x7fb5a006ac30 2023-10-08T00:14:30.105 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.102+0000 7fb59e7fc700 1 -- 172.21.15.99:0/3107267794 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb598012430 con 0x7fb5a006ac30 2023-10-08T00:14:30.105 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.103+0000 7fb5a55a0700 1 -- 172.21.15.99:0/3107267794 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5a006ac30 msgr2=0x7fb5a015c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:30.105 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.103+0000 7fb5a55a0700 1 --2- 172.21.15.99:0/3107267794 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5a006ac30 0x7fb5a015c7f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb598009e60 tx=0x7fb598009780 comp rx=0 tx=0).stop 2023-10-08T00:14:30.105 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.103+0000 7fb5a55a0700 1 -- 172.21.15.99:0/3107267794 shutdown_connections 2023-10-08T00:14:30.105 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.103+0000 7fb5a55a0700 1 --2- 172.21.15.99:0/3107267794 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5a006ac30 0x7fb5a015c7f0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:30.106 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.103+0000 7fb5a55a0700 1 --2- 172.21.15.99:0/3107267794 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb5a01577d0 0x7fb5a00cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:30.106 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.103+0000 7fb5a55a0700 1 -- 172.21.15.99:0/3107267794 >> 172.21.15.99:0/3107267794 conn(0x7fb5a0068ce0 msgr2=0x7fb5a00690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:30.106 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.103+0000 7fb5a55a0700 1 -- 172.21.15.99:0/3107267794 shutdown_connections 2023-10-08T00:14:30.106 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.103+0000 7fb5a55a0700 1 -- 172.21.15.99:0/3107267794 wait complete. 2023-10-08T00:14:30.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.104+0000 7fb5a55a0700 1 Processor -- start 2023-10-08T00:14:30.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.104+0000 7fb5a55a0700 1 -- start start 2023-10-08T00:14:30.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.105+0000 7fb5a55a0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb5a006ac30 0x7fb5a016ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:30.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.105+0000 7fb5a55a0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5a01577d0 0x7fb5a015e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:30.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.105+0000 7fb5a55a0700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5a0157ba0 0x7fb5a00b59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:30.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.105+0000 7fb5a55a0700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a016db60 con 0x7fb5a01577d0 2023-10-08T00:14:30.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.105+0000 7fb5a55a0700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a016de40 con 0x7fb5a0157ba0 2023-10-08T00:14:30.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.105+0000 7fb5a55a0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a016e120 con 0x7fb5a006ac30 2023-10-08T00:14:30.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.105+0000 7fb5967fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5a01577d0 0x7fb5a015e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:30.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.105+0000 7fb5967fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5a01577d0 0x7fb5a015e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43362/0 (socket says 172.21.15.99:43362) 2023-10-08T00:14:30.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.105+0000 7fb5967fc700 1 -- 172.21.15.99:0/2930894633 learned_addr learned my addr 172.21.15.99:0/2930894633 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:30.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.105+0000 7fb59f7fe700 1 --2- 172.21.15.99:0/2930894633 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5a0157ba0 0x7fb5a00b59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:30.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.105+0000 7fb59effd700 1 --2- 172.21.15.99:0/2930894633 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb5a006ac30 0x7fb5a016ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:30.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.105+0000 7fb5967fc700 1 -- 172.21.15.99:0/2930894633 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb5a006ac30 msgr2=0x7fb5a016ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:30.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.105+0000 7fb5967fc700 1 --2- 172.21.15.99:0/2930894633 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb5a006ac30 0x7fb5a016ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:30.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.105+0000 7fb5967fc700 1 -- 172.21.15.99:0/2930894633 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5a0157ba0 msgr2=0x7fb5a00b59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:30.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.105+0000 7fb5967fc700 1 --2- 172.21.15.99:0/2930894633 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5a0157ba0 0x7fb5a00b59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:30.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.105+0000 7fb5967fc700 1 -- 172.21.15.99:0/2930894633 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb598009a20 con 0x7fb5a01577d0 2023-10-08T00:14:30.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.105+0000 7fb59effd700 1 --2- 172.21.15.99:0/2930894633 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb5a006ac30 0x7fb5a016ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:14:30.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.105+0000 7fb59f7fe700 1 --2- 172.21.15.99:0/2930894633 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5a0157ba0 0x7fb5a00b59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:14:30.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.105+0000 7fb5967fc700 1 --2- 172.21.15.99:0/2930894633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5a01577d0 0x7fb5a015e690 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb58800c5b0 tx=0x7fb588001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:30.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.105+0000 7fb5977fe700 1 -- 172.21.15.99:0/2930894633 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb588005b30 con 0x7fb5a01577d0 2023-10-08T00:14:30.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.105+0000 7fb5977fe700 1 -- 172.21.15.99:0/2930894633 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb588005c90 con 0x7fb5a01577d0 2023-10-08T00:14:30.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.105+0000 7fb5a55a0700 1 -- 172.21.15.99:0/2930894633 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5a016e400 con 0x7fb5a01577d0 2023-10-08T00:14:30.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.105+0000 7fb5977fe700 1 -- 172.21.15.99:0/2930894633 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb588007300 con 0x7fb5a01577d0 2023-10-08T00:14:30.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.105+0000 7fb5a55a0700 1 -- 172.21.15.99:0/2930894633 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5a016b250 con 0x7fb5a01577d0 2023-10-08T00:14:30.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.106+0000 7fb5a55a0700 1 -- 172.21.15.99:0/2930894633 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5a004c3c0 con 0x7fb5a01577d0 2023-10-08T00:14:30.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.107+0000 7fb5977fe700 1 -- 172.21.15.99:0/2930894633 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb58801b3b0 con 0x7fb5a01577d0 2023-10-08T00:14:30.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.107+0000 7fb5977fe700 1 --2- 172.21.15.99:0/2930894633 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb580066f30 0x7fb5800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:30.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.107+0000 7fb5977fe700 1 -- 172.21.15.99:0/2930894633 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5763+0+0 (secure 0 0 0) 0x7fb588083aa0 con 0x7fb5a01577d0 2023-10-08T00:14:30.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.107+0000 7fb59effd700 1 --2- 172.21.15.99:0/2930894633 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb580066f30 0x7fb5800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:30.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.107+0000 7fb5977fe700 1 -- 172.21.15.99:0/2930894633 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=75}) v3 -- 0x7fb58006eb10 con 0x7fb5a01577d0 2023-10-08T00:14:30.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.108+0000 7fb59effd700 1 --2- 172.21.15.99:0/2930894633 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb580066f30 0x7fb5800693e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb590006fd0 tx=0x7fb590008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:30.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.111+0000 7fb5977fe700 1 -- 172.21.15.99:0/2930894633 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb588017060 con 0x7fb5a01577d0 2023-10-08T00:14:30.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.337+0000 7fb5a55a0700 1 -- 172.21.15.99:0/2930894633 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb5a015bff0 con 0x7fb5a01577d0 2023-10-08T00:14:30.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.337+0000 7fb5977fe700 1 -- 172.21.15.99:0/2930894633 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4600 (secure 0 0 0) 0x7fb588050cd0 con 0x7fb5a01577d0 2023-10-08T00:14:30.338 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:14:30.338 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5087,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/862112794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":862112794},{"type":"v1","addr":"172.21.15.191:6837","nonce":862112794}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5673,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/2694493794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2694493794},{"type":"v1","addr":"172.21.15.99:6834","nonce":2694493794}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1906990531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":1906990531},{"type":"v1","addr":"172.21.15.99:6835","nonce":1906990531}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:12:57.581246+0000","modified":"2023-10-08T00:13:01.240449+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5688},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5688":{"gid":5688,"name":"d","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6836/1779510877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":1779510877},{"type":"v1","addr":"172.21.15.191:6836","nonce":1779510877}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-08T00:14:30.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.339+0000 7fb5a55a0700 1 -- 172.21.15.99:0/2930894633 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb580066f30 msgr2=0x7fb5800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:30.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.339+0000 7fb5a55a0700 1 --2- 172.21.15.99:0/2930894633 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb580066f30 0x7fb5800693e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb590006fd0 tx=0x7fb590008040 comp rx=0 tx=0).stop 2023-10-08T00:14:30.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.339+0000 7fb5a55a0700 1 -- 172.21.15.99:0/2930894633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5a01577d0 msgr2=0x7fb5a015e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:30.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.339+0000 7fb5a55a0700 1 --2- 172.21.15.99:0/2930894633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5a01577d0 0x7fb5a015e690 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb58800c5b0 tx=0x7fb588001070 comp rx=0 tx=0).stop 2023-10-08T00:14:30.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.339+0000 7fb5a55a0700 1 -- 172.21.15.99:0/2930894633 shutdown_connections 2023-10-08T00:14:30.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.339+0000 7fb5a55a0700 1 --2- 172.21.15.99:0/2930894633 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb580066f30 0x7fb5800693e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:30.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.340+0000 7fb5a55a0700 1 --2- 172.21.15.99:0/2930894633 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5a0157ba0 0x7fb5a00b59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:30.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.340+0000 7fb5a55a0700 1 --2- 172.21.15.99:0/2930894633 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb5a006ac30 0x7fb5a016ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:30.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.340+0000 7fb5a55a0700 1 --2- 172.21.15.99:0/2930894633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5a01577d0 0x7fb5a015e690 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:30.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.340+0000 7fb5a55a0700 1 -- 172.21.15.99:0/2930894633 >> 172.21.15.99:0/2930894633 conn(0x7fb5a0068ce0 msgr2=0x7fb5a01563f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:30.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.340+0000 7fb5a55a0700 1 -- 172.21.15.99:0/2930894633 shutdown_connections 2023-10-08T00:14:30.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.340+0000 7fb5a55a0700 1 -- 172.21.15.99:0/2930894633 wait complete. 2023-10-08T00:14:30.342 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 32 2023-10-08T00:14:30.352 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:14:30.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.465+0000 7ff1184e9700 1 Processor -- start 2023-10-08T00:14:30.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.466+0000 7ff1184e9700 1 -- start start 2023-10-08T00:14:30.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.467+0000 7ff1184e9700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff110157870 0x7ff11015b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:30.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.467+0000 7ff1184e9700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff11015cbb0 con 0x7ff110157c40 2023-10-08T00:14:30.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.467+0000 7ff1184e9700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff11015bfa0 con 0x7ff11015a620 2023-10-08T00:14:30.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.467+0000 7ff1184e9700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff11015c280 con 0x7ff110157870 2023-10-08T00:14:30.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.467+0000 7ff116285700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff110157870 0x7ff11015b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:30.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.467+0000 7ff116a86700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7ff110157c40 0x7ff110158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:44462/0 (socket says 172.21.15.99:44462) 2023-10-08T00:14:30.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.467+0000 7ff116a86700 1 -- 172.21.15.99:0/1220675822 learned_addr learned my addr 172.21.15.99:0/1220675822 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:14:30.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.467+0000 7ff116285700 1 -- 172.21.15.99:0/1220675822 >> v1:172.21.15.191:6789/0 conn(0x7ff11015a620 legacy=0x7ff11015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:30.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.467+0000 7ff116285700 1 -- 172.21.15.99:0/1220675822 >> v1:172.21.15.99:6789/0 conn(0x7ff110157c40 legacy=0x7ff110158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:30.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.467+0000 7ff116285700 1 -- 172.21.15.99:0/1220675822 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff11015c560 con 0x7ff110157870 2023-10-08T00:14:30.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.467+0000 7ff115283700 1 -- 172.21.15.99:0/1220675822 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1716612359 0 0) 0x7ff11015cbb0 con 0x7ff110157c40 2023-10-08T00:14:30.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.467+0000 7ff116285700 1 --2- 172.21.15.99:0/1220675822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff110157870 0x7ff11015b6a0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7ff10c009e60 tx=0x7ff10c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53455d1310aefc81 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:30.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.467+0000 7ff115283700 1 -- 172.21.15.99:0/1220675822 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10c00e070 con 0x7ff110157870 2023-10-08T00:14:30.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.467+0000 7ff115283700 1 -- 172.21.15.99:0/1220675822 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff10c00bc30 con 0x7ff110157870 2023-10-08T00:14:30.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.467+0000 7ff115283700 1 -- 172.21.15.99:0/1220675822 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10c012430 con 0x7ff110157870 2023-10-08T00:14:30.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.468+0000 7ff1184e9700 1 -- 172.21.15.99:0/1220675822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff110157870 msgr2=0x7ff11015b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:30.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.468+0000 7ff1184e9700 1 --2- 172.21.15.99:0/1220675822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff110157870 0x7ff11015b6a0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7ff10c009e60 tx=0x7ff10c009780 comp rx=0 tx=0).stop 2023-10-08T00:14:30.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.468+0000 7ff1184e9700 1 -- 172.21.15.99:0/1220675822 shutdown_connections 2023-10-08T00:14:30.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.468+0000 7ff1184e9700 1 --2- 172.21.15.99:0/1220675822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff110157870 0x7ff11015b6a0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:30.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.468+0000 7ff1184e9700 1 -- 172.21.15.99:0/1220675822 >> 172.21.15.99:0/1220675822 conn(0x7ff110068ce0 msgr2=0x7ff1100690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:30.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.468+0000 7ff1184e9700 1 -- 172.21.15.99:0/1220675822 shutdown_connections 2023-10-08T00:14:30.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.468+0000 7ff1184e9700 1 -- 172.21.15.99:0/1220675822 wait complete. 2023-10-08T00:14:30.474 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.469+0000 7ff1184e9700 1 Processor -- start 2023-10-08T00:14:30.474 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.469+0000 7ff1184e9700 1 -- start start 2023-10-08T00:14:30.474 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.470+0000 7ff1184e9700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff110157870 0x7ff11013d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:30.474 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.470+0000 7ff1184e9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff110157c40 0x7ff110153fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:30.474 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.470+0000 7ff1184e9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff11015a620 0x7ff1100b5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:30.474 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.470+0000 7ff1184e9700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff110152b60 con 0x7ff11015a620 2023-10-08T00:14:30.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.470+0000 7ff1184e9700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11013d610 con 0x7ff110157870 2023-10-08T00:14:30.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.470+0000 7ff1184e9700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff110154500 con 0x7ff110157c40 2023-10-08T00:14:30.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.470+0000 7ff116a86700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff11015a620 0x7ff1100b5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:30.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.470+0000 7ff116285700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff110157870 0x7ff11013d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:30.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.470+0000 7ff116a86700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff11015a620 0x7ff1100b5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43384/0 (socket says 172.21.15.99:43384) 2023-10-08T00:14:30.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.470+0000 7ff116a86700 1 -- 172.21.15.99:0/3100788573 learned_addr learned my addr 172.21.15.99:0/3100788573 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:30.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.470+0000 7ff115a84700 1 --2- 172.21.15.99:0/3100788573 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff110157c40 0x7ff110153fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:30.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.470+0000 7ff116a86700 1 -- 172.21.15.99:0/3100788573 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff110157c40 msgr2=0x7ff110153fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:30.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.470+0000 7ff116a86700 1 --2- 172.21.15.99:0/3100788573 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff110157c40 0x7ff110153fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:30.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.470+0000 7ff116a86700 1 -- 172.21.15.99:0/3100788573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff110157870 msgr2=0x7ff11013d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:30.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.470+0000 7ff116a86700 1 --2- 172.21.15.99:0/3100788573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff110157870 0x7ff11013d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:30.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.470+0000 7ff116a86700 1 -- 172.21.15.99:0/3100788573 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff10c009a20 con 0x7ff11015a620 2023-10-08T00:14:30.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.470+0000 7ff116a86700 1 --2- 172.21.15.99:0/3100788573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff11015a620 0x7ff1100b5f10 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff10800c5e0 tx=0x7ff108007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:30.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.470+0000 7ff1067fc700 1 -- 172.21.15.99:0/3100788573 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff108005b30 con 0x7ff11015a620 2023-10-08T00:14:30.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.470+0000 7ff1067fc700 1 -- 172.21.15.99:0/3100788573 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff108005c90 con 0x7ff11015a620 2023-10-08T00:14:30.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.470+0000 7ff1067fc700 1 -- 172.21.15.99:0/3100788573 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1080072e0 con 0x7ff11015a620 2023-10-08T00:14:30.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.470+0000 7ff1184e9700 1 -- 172.21.15.99:0/3100788573 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff11016b7d0 con 0x7ff11015a620 2023-10-08T00:14:30.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.470+0000 7ff1184e9700 1 -- 172.21.15.99:0/3100788573 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1101534e0 con 0x7ff11015a620 2023-10-08T00:14:30.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.471+0000 7ff1184e9700 1 -- 172.21.15.99:0/3100788573 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff11004c3c0 con 0x7ff11015a620 2023-10-08T00:14:30.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.472+0000 7ff1067fc700 1 -- 172.21.15.99:0/3100788573 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff108007650 con 0x7ff11015a620 2023-10-08T00:14:30.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.472+0000 7ff1067fc700 1 --2- 172.21.15.99:0/3100788573 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff0f4066f30 0x7ff0f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:30.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.472+0000 7ff1067fc700 1 -- 172.21.15.99:0/3100788573 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5763+0+0 (secure 0 0 0) 0x7ff1080837e0 con 0x7ff11015a620 2023-10-08T00:14:30.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.472+0000 7ff116285700 1 --2- 172.21.15.99:0/3100788573 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff0f4066f30 0x7ff0f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:30.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.472+0000 7ff1067fc700 1 -- 172.21.15.99:0/3100788573 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=75}) v3 -- 0x7ff0f406eb10 con 0x7ff11015a620 2023-10-08T00:14:30.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.473+0000 7ff116285700 1 --2- 172.21.15.99:0/3100788573 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff0f4066f30 0x7ff0f40693e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff10c009e60 tx=0x7ff10c019000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:30.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.476+0000 7ff1067fc700 1 -- 172.21.15.99:0/3100788573 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff10804b5a0 con 0x7ff11015a620 2023-10-08T00:14:30.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.659+0000 7ff1184e9700 1 -- 172.21.15.99:0/3100788573 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff1100b3c30 con 0x7ff11015a620 2023-10-08T00:14:30.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.660+0000 7ff1067fc700 1 -- 172.21.15.99:0/3100788573 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+15497 (secure 0 0 0) 0x7ff108017020 con 0x7ff11015a620 2023-10-08T00:14:30.660 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:14:30.661 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":74,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:13:44.132886+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:12:55.353771+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-08T00:12:56.552157+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["backfillfull","exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","full","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","full","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.28","mappings":[{"from":6,"to":4}]},{"pgid":"9.3e","mappings":[{"from":6,"to":1}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:14:30.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.662+0000 7ff1184e9700 1 -- 172.21.15.99:0/3100788573 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff0f4066f30 msgr2=0x7ff0f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:30.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.662+0000 7ff1184e9700 1 --2- 172.21.15.99:0/3100788573 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff0f4066f30 0x7ff0f40693e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff10c009e60 tx=0x7ff10c019000 comp rx=0 tx=0).stop 2023-10-08T00:14:30.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.662+0000 7ff1184e9700 1 -- 172.21.15.99:0/3100788573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff11015a620 msgr2=0x7ff1100b5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:30.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.662+0000 7ff1184e9700 1 --2- 172.21.15.99:0/3100788573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff11015a620 0x7ff1100b5f10 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff10800c5e0 tx=0x7ff108007d30 comp rx=0 tx=0).stop 2023-10-08T00:14:30.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.662+0000 7ff1184e9700 1 -- 172.21.15.99:0/3100788573 shutdown_connections 2023-10-08T00:14:30.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.662+0000 7ff1184e9700 1 --2- 172.21.15.99:0/3100788573 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff0f4066f30 0x7ff0f40693e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:30.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.662+0000 7ff1184e9700 1 --2- 172.21.15.99:0/3100788573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff110157870 0x7ff11013d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:30.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.662+0000 7ff1184e9700 1 --2- 172.21.15.99:0/3100788573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff11015a620 0x7ff1100b5f10 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:30.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.662+0000 7ff1184e9700 1 --2- 172.21.15.99:0/3100788573 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff110157c40 0x7ff110153fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:30.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.662+0000 7ff1184e9700 1 -- 172.21.15.99:0/3100788573 >> 172.21.15.99:0/3100788573 conn(0x7ff110068ce0 msgr2=0x7ff11014d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:30.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.662+0000 7ff1184e9700 1 -- 172.21.15.99:0/3100788573 shutdown_connections 2023-10-08T00:14:30.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.662+0000 7ff1184e9700 1 -- 172.21.15.99:0/3100788573 wait complete. 2023-10-08T00:14:30.674 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T00:14:30.674 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:14:30.793 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.793+0000 7fbf14db2700 1 Processor -- start 2023-10-08T00:14:30.793 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.793+0000 7fbf14db2700 1 -- start start 2023-10-08T00:14:30.795 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.793+0000 7fbf14db2700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbf1015a620 0x7fbf1015d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:30.796 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.793+0000 7fbf14db2700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf1015cbb0 con 0x7fbf1015d580 2023-10-08T00:14:30.796 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.793+0000 7fbf14db2700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf1015bfa0 con 0x7fbf10157870 2023-10-08T00:14:30.796 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.793+0000 7fbf14db2700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf1015bdb0 con 0x7fbf1015a620 2023-10-08T00:14:30.796 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.794+0000 7fbf05d9b700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbf1015a620 0x7fbf1015d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:30.796 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.794+0000 7fbf0ed9d700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fbf1015d580 0x7fbf10158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:44478/0 (socket says 172.21.15.99:44478) 2023-10-08T00:14:30.796 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.794+0000 7fbf0ed9d700 1 -- 172.21.15.99:0/1436252596 learned_addr learned my addr 172.21.15.99:0/1436252596 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:14:30.796 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.794+0000 7fbf05d9b700 1 -- 172.21.15.99:0/1436252596 >> v1:172.21.15.191:6789/0 conn(0x7fbf10157870 legacy=0x7fbf1015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:30.796 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.794+0000 7fbf05d9b700 1 -- 172.21.15.99:0/1436252596 >> v1:172.21.15.99:6789/0 conn(0x7fbf1015d580 legacy=0x7fbf10158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:30.796 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.794+0000 7fbf05d9b700 1 -- 172.21.15.99:0/1436252596 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf1015c280 con 0x7fbf1015a620 2023-10-08T00:14:30.797 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.794+0000 7fbf0dd9b700 1 -- 172.21.15.99:0/1436252596 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3073562095 0 0) 0x7fbf1015cbb0 con 0x7fbf1015d580 2023-10-08T00:14:30.797 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.794+0000 7fbf05d9b700 1 --2- 172.21.15.99:0/1436252596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf1015a620 0x7fbf1015d040 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fbef80098e0 tx=0x7fbef80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eab814600bb45f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:30.797 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.794+0000 7fbf0dd9b700 1 -- 172.21.15.99:0/1436252596 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef8009e80 con 0x7fbf1015a620 2023-10-08T00:14:30.797 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.794+0000 7fbf0dd9b700 1 -- 172.21.15.99:0/1436252596 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef800b860 con 0x7fbf1015a620 2023-10-08T00:14:30.797 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.794+0000 7fbf0dd9b700 1 -- 172.21.15.99:0/1436252596 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef800bbe0 con 0x7fbf1015a620 2023-10-08T00:14:30.797 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.795+0000 7fbf14db2700 1 -- 172.21.15.99:0/1436252596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf1015a620 msgr2=0x7fbf1015d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:30.797 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.795+0000 7fbf14db2700 1 --2- 172.21.15.99:0/1436252596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf1015a620 0x7fbf1015d040 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fbef80098e0 tx=0x7fbef80091a0 comp rx=0 tx=0).stop 2023-10-08T00:14:30.797 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.795+0000 7fbf14db2700 1 -- 172.21.15.99:0/1436252596 shutdown_connections 2023-10-08T00:14:30.797 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.795+0000 7fbf14db2700 1 --2- 172.21.15.99:0/1436252596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf1015a620 0x7fbf1015d040 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:30.797 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.795+0000 7fbf14db2700 1 -- 172.21.15.99:0/1436252596 >> 172.21.15.99:0/1436252596 conn(0x7fbf10068ce0 msgr2=0x7fbf100690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:30.797 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.795+0000 7fbf14db2700 1 -- 172.21.15.99:0/1436252596 shutdown_connections 2023-10-08T00:14:30.798 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.795+0000 7fbf14db2700 1 -- 172.21.15.99:0/1436252596 wait complete. 2023-10-08T00:14:30.801 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.796+0000 7fbf14db2700 1 Processor -- start 2023-10-08T00:14:30.801 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.796+0000 7fbf14db2700 1 -- start start 2023-10-08T00:14:30.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.797+0000 7fbf14db2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbf10157870 0x7fbf1014ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:30.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.797+0000 7fbf14db2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbf1015a620 0x7fbf100d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:30.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.797+0000 7fbf14db2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf1015d580 0x7fbf1014b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:30.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.797+0000 7fbf14db2700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf100d09c0 con 0x7fbf1015d580 2023-10-08T00:14:30.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.797+0000 7fbf14db2700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf100cfdf0 con 0x7fbf10157870 2023-10-08T00:14:30.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.797+0000 7fbf14db2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf100d0090 con 0x7fbf1015a620 2023-10-08T00:14:30.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.797+0000 7fbf0ed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf1015d580 0x7fbf1014b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:30.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.797+0000 7fbf0ed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf1015d580 0x7fbf1014b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43400/0 (socket says 172.21.15.99:43400) 2023-10-08T00:14:30.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.797+0000 7fbf0ed9d700 1 -- 172.21.15.99:0/2191294384 learned_addr learned my addr 172.21.15.99:0/2191294384 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:30.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.797+0000 7fbf0e59c700 1 --2- 172.21.15.99:0/2191294384 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbf10157870 0x7fbf1014ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:30.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.797+0000 7fbf05d9b700 1 --2- 172.21.15.99:0/2191294384 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbf1015a620 0x7fbf100d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:30.803 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.797+0000 7fbf0ed9d700 1 -- 172.21.15.99:0/2191294384 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbf1015a620 msgr2=0x7fbf100d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:30.803 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.797+0000 7fbf0ed9d700 1 --2- 172.21.15.99:0/2191294384 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbf1015a620 0x7fbf100d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:30.803 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.797+0000 7fbf0ed9d700 1 -- 172.21.15.99:0/2191294384 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbf10157870 msgr2=0x7fbf1014ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:30.803 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.797+0000 7fbf0ed9d700 1 --2- 172.21.15.99:0/2191294384 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbf10157870 0x7fbf1014ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:30.803 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.797+0000 7fbf0ed9d700 1 -- 172.21.15.99:0/2191294384 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef8009420 con 0x7fbf1015d580 2023-10-08T00:14:30.803 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.798+0000 7fbf0ed9d700 1 --2- 172.21.15.99:0/2191294384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf1015d580 0x7fbf1014b670 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fbf0800cbc0 tx=0x7fbf08001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:30.804 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.798+0000 7fbf06ffd700 1 -- 172.21.15.99:0/2191294384 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0800a870 con 0x7fbf1015d580 2023-10-08T00:14:30.804 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.798+0000 7fbf06ffd700 1 -- 172.21.15.99:0/2191294384 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf0800a9d0 con 0x7fbf1015d580 2023-10-08T00:14:30.804 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.798+0000 7fbf14db2700 1 -- 172.21.15.99:0/2191294384 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf100d0370 con 0x7fbf1015d580 2023-10-08T00:14:30.804 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.798+0000 7fbf06ffd700 1 -- 172.21.15.99:0/2191294384 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0800ad90 con 0x7fbf1015d580 2023-10-08T00:14:30.804 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.798+0000 7fbf14db2700 1 -- 172.21.15.99:0/2191294384 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf100b2f10 con 0x7fbf1015d580 2023-10-08T00:14:30.804 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.799+0000 7fbf14db2700 1 -- 172.21.15.99:0/2191294384 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf100d0b30 con 0x7fbf1015d580 2023-10-08T00:14:30.804 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.799+0000 7fbf06ffd700 1 -- 172.21.15.99:0/2191294384 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbf08007300 con 0x7fbf1015d580 2023-10-08T00:14:30.804 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.799+0000 7fbf06ffd700 1 --2- 172.21.15.99:0/2191294384 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbef0066f30 0x7fbef00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:30.804 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.800+0000 7fbf06ffd700 1 -- 172.21.15.99:0/2191294384 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5763+0+0 (secure 0 0 0) 0x7fbf080848f0 con 0x7fbf1015d580 2023-10-08T00:14:30.804 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.800+0000 7fbf06ffd700 1 -- 172.21.15.99:0/2191294384 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=75}) v3 -- 0x7fbef006eb10 con 0x7fbf1015d580 2023-10-08T00:14:30.804 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.800+0000 7fbf0e59c700 1 --2- 172.21.15.99:0/2191294384 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbef0066f30 0x7fbef00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:30.804 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.800+0000 7fbf0e59c700 1 --2- 172.21.15.99:0/2191294384 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbef0066f30 0x7fbef00693e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbf00003a80 tx=0x7fbf00007ec0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:30.805 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:30.804+0000 7fbf06ffd700 1 -- 172.21.15.99:0/2191294384 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbf08051b20 con 0x7fbf1015d580 2023-10-08T00:14:31.026 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.026+0000 7fbf14db2700 1 -- 172.21.15.99:0/2191294384 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbf1004c3c0 con 0x7fbf1015d580 2023-10-08T00:14:31.027 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.026+0000 7fbf06ffd700 1 -- 172.21.15.99:0/2191294384 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4600 (secure 0 0 0) 0x7fbf08055140 con 0x7fbf1015d580 2023-10-08T00:14:31.028 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:14:31.028 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5087,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/862112794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":862112794},{"type":"v1","addr":"172.21.15.191:6837","nonce":862112794}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5673,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/2694493794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2694493794},{"type":"v1","addr":"172.21.15.99:6834","nonce":2694493794}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1906990531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":1906990531},{"type":"v1","addr":"172.21.15.99:6835","nonce":1906990531}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:12:57.581246+0000","modified":"2023-10-08T00:13:01.240449+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5688},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5688":{"gid":5688,"name":"d","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6836/1779510877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":1779510877},{"type":"v1","addr":"172.21.15.191:6836","nonce":1779510877}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-08T00:14:31.030 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.029+0000 7fbf14db2700 1 -- 172.21.15.99:0/2191294384 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbef0066f30 msgr2=0x7fbef00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:31.030 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.029+0000 7fbf14db2700 1 --2- 172.21.15.99:0/2191294384 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbef0066f30 0x7fbef00693e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbf00003a80 tx=0x7fbf00007ec0 comp rx=0 tx=0).stop 2023-10-08T00:14:31.030 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.029+0000 7fbf14db2700 1 -- 172.21.15.99:0/2191294384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf1015d580 msgr2=0x7fbf1014b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:31.030 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.029+0000 7fbf14db2700 1 --2- 172.21.15.99:0/2191294384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf1015d580 0x7fbf1014b670 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fbf0800cbc0 tx=0x7fbf08001980 comp rx=0 tx=0).stop 2023-10-08T00:14:31.031 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.029+0000 7fbf14db2700 1 -- 172.21.15.99:0/2191294384 shutdown_connections 2023-10-08T00:14:31.031 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.029+0000 7fbf14db2700 1 --2- 172.21.15.99:0/2191294384 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbef0066f30 0x7fbef00693e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:31.031 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.029+0000 7fbf14db2700 1 --2- 172.21.15.99:0/2191294384 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbf10157870 0x7fbf1014ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:31.031 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.029+0000 7fbf14db2700 1 --2- 172.21.15.99:0/2191294384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf1015d580 0x7fbf1014b670 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:31.031 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.029+0000 7fbf14db2700 1 --2- 172.21.15.99:0/2191294384 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbf1015a620 0x7fbf100d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:31.032 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.029+0000 7fbf14db2700 1 -- 172.21.15.99:0/2191294384 >> 172.21.15.99:0/2191294384 conn(0x7fbf10068ce0 msgr2=0x7fbf1014c800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:31.032 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.029+0000 7fbf14db2700 1 -- 172.21.15.99:0/2191294384 shutdown_connections 2023-10-08T00:14:31.032 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.029+0000 7fbf14db2700 1 -- 172.21.15.99:0/2191294384 wait complete. 2023-10-08T00:14:31.032 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 32 2023-10-08T00:14:31.042 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:14:31.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.157+0000 7f2a5b210700 1 Processor -- start 2023-10-08T00:14:31.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.157+0000 7f2a5b210700 1 -- start start 2023-10-08T00:14:31.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.158+0000 7f2a5b210700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2a54157870 0x7f2a5415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:31.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.158+0000 7f2a5b210700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2a5415a620 0x7f2a5415d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:31.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.158+0000 7f2a5b210700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a5415cbb0 con 0x7f2a5415d580 2023-10-08T00:14:31.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.158+0000 7f2a5b210700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a5415bfa0 con 0x7f2a5415a620 2023-10-08T00:14:31.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.158+0000 7f2a5b210700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a5415c280 con 0x7f2a54157870 2023-10-08T00:14:31.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.158+0000 7f2a4bfff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2a5415a620 0x7f2a5415d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:31.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.158+0000 7f2a4bfff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2a5415a620 0x7f2a5415d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43408/0 (socket says 172.21.15.99:43408) 2023-10-08T00:14:31.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.158+0000 7f2a4bfff700 1 -- 172.21.15.99:0/2226532157 learned_addr learned my addr 172.21.15.99:0/2226532157 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:31.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.159+0000 7f2a58fac700 1 --2- 172.21.15.99:0/2226532157 >> v2:172.21.15.191:3300/0 conn(0x7f2a54157870 0x7f2a5415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:31.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.159+0000 7f2a4bfff700 1 -- 172.21.15.99:0/2226532157 >> v2:172.21.15.191:3300/0 conn(0x7f2a54157870 msgr2=0x7f2a5415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:31.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.159+0000 7f2a4bfff700 1 --2- 172.21.15.99:0/2226532157 >> v2:172.21.15.191:3300/0 conn(0x7f2a54157870 0x7f2a5415b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:31.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.159+0000 7f2a4bfff700 1 -- 172.21.15.99:0/2226532157 >> v1:172.21.15.191:6789/0 conn(0x7f2a5415d580 legacy=0x7f2a54158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:31.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.159+0000 7f2a4bfff700 1 -- 172.21.15.99:0/2226532157 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a5415c560 con 0x7f2a5415a620 2023-10-08T00:14:31.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.159+0000 7f2a58fac700 1 --2- 172.21.15.99:0/2226532157 >> v2:172.21.15.191:3300/0 conn(0x7f2a54157870 0x7f2a5415b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:14:31.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.159+0000 7f2a4bfff700 1 --2- 172.21.15.99:0/2226532157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a5415a620 0x7f2a5415d040 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f2a54069d80 tx=0x7f2a44009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dacc173faefae37 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:31.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.159+0000 7f2a4b7fe700 1 -- 172.21.15.99:0/2226532157 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4400b820 con 0x7f2a5415a620 2023-10-08T00:14:31.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.159+0000 7f2a4b7fe700 1 -- 172.21.15.99:0/2226532157 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a4400b980 con 0x7f2a5415a620 2023-10-08T00:14:31.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.159+0000 7f2a4b7fe700 1 -- 172.21.15.99:0/2226532157 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4400bd20 con 0x7f2a5415a620 2023-10-08T00:14:31.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.159+0000 7f2a5b210700 1 -- 172.21.15.99:0/2226532157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a5415a620 msgr2=0x7f2a5415d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:31.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.159+0000 7f2a5b210700 1 --2- 172.21.15.99:0/2226532157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a5415a620 0x7f2a5415d040 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f2a54069d80 tx=0x7f2a44009250 comp rx=0 tx=0).stop 2023-10-08T00:14:31.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.159+0000 7f2a5b210700 1 -- 172.21.15.99:0/2226532157 shutdown_connections 2023-10-08T00:14:31.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.159+0000 7f2a5b210700 1 --2- 172.21.15.99:0/2226532157 >> v2:172.21.15.191:3300/0 conn(0x7f2a54157870 0x7f2a5415b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:31.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.159+0000 7f2a5b210700 1 --2- 172.21.15.99:0/2226532157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a5415a620 0x7f2a5415d040 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:31.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.159+0000 7f2a5b210700 1 -- 172.21.15.99:0/2226532157 >> 172.21.15.99:0/2226532157 conn(0x7f2a54068ce0 msgr2=0x7f2a540690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:31.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.159+0000 7f2a5b210700 1 -- 172.21.15.99:0/2226532157 shutdown_connections 2023-10-08T00:14:31.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.160+0000 7f2a5b210700 1 -- 172.21.15.99:0/2226532157 wait complete. 2023-10-08T00:14:31.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.161+0000 7f2a5b210700 1 Processor -- start 2023-10-08T00:14:31.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.161+0000 7f2a5b210700 1 -- start start 2023-10-08T00:14:31.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.161+0000 7f2a5b210700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a54157870 0x7f2a540cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:31.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.161+0000 7f2a5b210700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a5415a620 0x7f2a540b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:31.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.161+0000 7f2a5b210700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a5415d580 0x7f2a5414bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:31.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.161+0000 7f2a5b210700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a540d02b0 con 0x7f2a54157870 2023-10-08T00:14:31.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.161+0000 7f2a5b210700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a540b1e20 con 0x7f2a5415a620 2023-10-08T00:14:31.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.161+0000 7f2a5b210700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5414d9b0 con 0x7f2a5415d580 2023-10-08T00:14:31.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.162+0000 7f2a58fac700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a54157870 0x7f2a540cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:31.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.162+0000 7f2a58fac700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a54157870 0x7f2a540cfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43410/0 (socket says 172.21.15.99:43410) 2023-10-08T00:14:31.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.162+0000 7f2a58fac700 1 -- 172.21.15.99:0/620865009 learned_addr learned my addr 172.21.15.99:0/620865009 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:31.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.162+0000 7f2a58fac700 1 -- 172.21.15.99:0/620865009 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a5415d580 msgr2=0x7f2a5414bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:31.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.162+0000 7f2a58fac700 1 --2- 172.21.15.99:0/620865009 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a5415d580 0x7f2a5414bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:31.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.162+0000 7f2a58fac700 1 -- 172.21.15.99:0/620865009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a5415a620 msgr2=0x7f2a540b18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:31.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.162+0000 7f2a58fac700 1 --2- 172.21.15.99:0/620865009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a5415a620 0x7f2a540b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:31.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.162+0000 7f2a58fac700 1 -- 172.21.15.99:0/620865009 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a440094f0 con 0x7f2a54157870 2023-10-08T00:14:31.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.162+0000 7f2a58fac700 1 --2- 172.21.15.99:0/620865009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a54157870 0x7f2a540cfd70 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f2a5000c830 tx=0x7f2a50008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:31.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.162+0000 7f2a48ff9700 1 -- 172.21.15.99:0/620865009 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a50011600 con 0x7f2a54157870 2023-10-08T00:14:31.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.162+0000 7f2a48ff9700 1 -- 172.21.15.99:0/620865009 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a50011760 con 0x7f2a54157870 2023-10-08T00:14:31.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.162+0000 7f2a5b210700 1 -- 172.21.15.99:0/620865009 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a540b2050 con 0x7f2a54157870 2023-10-08T00:14:31.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.162+0000 7f2a48ff9700 1 -- 172.21.15.99:0/620865009 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a50011ae0 con 0x7f2a54157870 2023-10-08T00:14:31.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.162+0000 7f2a5b210700 1 -- 172.21.15.99:0/620865009 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a540b2610 con 0x7f2a54157870 2023-10-08T00:14:31.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.163+0000 7f2a5b210700 1 -- 172.21.15.99:0/620865009 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a5404c3c0 con 0x7f2a54157870 2023-10-08T00:14:31.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.164+0000 7f2a48ff9700 1 -- 172.21.15.99:0/620865009 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2a5000a3e0 con 0x7f2a54157870 2023-10-08T00:14:31.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.164+0000 7f2a48ff9700 1 --2- 172.21.15.99:0/620865009 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2a40066f30 0x7f2a400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:31.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.164+0000 7f2a48ff9700 1 -- 172.21.15.99:0/620865009 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f2a50084a80 con 0x7f2a54157870 2023-10-08T00:14:31.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.164+0000 7f2a48ff9700 1 -- 172.21.15.99:0/620865009 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=75}) v3 -- 0x7f2a4006eb10 con 0x7f2a54157870 2023-10-08T00:14:31.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.164+0000 7f2a4bfff700 1 --2- 172.21.15.99:0/620865009 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2a40066f30 0x7f2a400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:31.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.165+0000 7f2a4bfff700 1 --2- 172.21.15.99:0/620865009 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2a40066f30 0x7f2a400693e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2a54069d80 tx=0x7f2a44016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:31.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.168+0000 7f2a48ff9700 1 -- 172.21.15.99:0/620865009 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2a50015060 con 0x7f2a54157870 2023-10-08T00:14:31.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.395+0000 7f2a5b210700 1 -- 172.21.15.99:0/620865009 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a540b29b0 con 0x7f2a54157870 2023-10-08T00:14:31.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.395+0000 7f2a48ff9700 1 -- 172.21.15.99:0/620865009 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4600 (secure 0 0 0) 0x7f2a5001f070 con 0x7f2a54157870 2023-10-08T00:14:31.396 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:14:31.396 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5087,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/862112794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":862112794},{"type":"v1","addr":"172.21.15.191:6837","nonce":862112794}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5673,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/2694493794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2694493794},{"type":"v1","addr":"172.21.15.99:6834","nonce":2694493794}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1906990531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":1906990531},{"type":"v1","addr":"172.21.15.99:6835","nonce":1906990531}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:12:57.581246+0000","modified":"2023-10-08T00:13:01.240449+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5688},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5688":{"gid":5688,"name":"d","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6836/1779510877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":1779510877},{"type":"v1","addr":"172.21.15.191:6836","nonce":1779510877}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-08T00:14:31.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.397+0000 7f2a5b210700 1 -- 172.21.15.99:0/620865009 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2a40066f30 msgr2=0x7f2a400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:31.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.397+0000 7f2a5b210700 1 --2- 172.21.15.99:0/620865009 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2a40066f30 0x7f2a400693e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2a54069d80 tx=0x7f2a44016000 comp rx=0 tx=0).stop 2023-10-08T00:14:31.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.397+0000 7f2a5b210700 1 -- 172.21.15.99:0/620865009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a54157870 msgr2=0x7f2a540cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:31.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.397+0000 7f2a5b210700 1 --2- 172.21.15.99:0/620865009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a54157870 0x7f2a540cfd70 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f2a5000c830 tx=0x7f2a50008040 comp rx=0 tx=0).stop 2023-10-08T00:14:31.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.397+0000 7f2a5b210700 1 -- 172.21.15.99:0/620865009 shutdown_connections 2023-10-08T00:14:31.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.397+0000 7f2a5b210700 1 --2- 172.21.15.99:0/620865009 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2a40066f30 0x7f2a400693e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:31.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.397+0000 7f2a5b210700 1 --2- 172.21.15.99:0/620865009 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a5415d580 0x7f2a5414bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:31.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.397+0000 7f2a5b210700 1 --2- 172.21.15.99:0/620865009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a54157870 0x7f2a540cfd70 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:31.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.397+0000 7f2a5b210700 1 --2- 172.21.15.99:0/620865009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a5415a620 0x7f2a540b18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:31.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.397+0000 7f2a5b210700 1 -- 172.21.15.99:0/620865009 >> 172.21.15.99:0/620865009 conn(0x7f2a54068ce0 msgr2=0x7f2a541582b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:31.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.398+0000 7f2a5b210700 1 -- 172.21.15.99:0/620865009 shutdown_connections 2023-10-08T00:14:31.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.398+0000 7f2a5b210700 1 -- 172.21.15.99:0/620865009 wait complete. 2023-10-08T00:14:31.400 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 32 2023-10-08T00:14:31.411 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:14:31.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.525+0000 7f07bd675700 1 Processor -- start 2023-10-08T00:14:31.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.525+0000 7f07bd675700 1 -- start start 2023-10-08T00:14:31.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.525+0000 7f07bd675700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f07b80a79f0 0x7f07b806eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:31.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.525+0000 7f07bd675700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f07b80b15e0 0x7f07b806eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:31.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.525+0000 7f07bd675700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07b80b26d0 con 0x7f07b80a7dc0 2023-10-08T00:14:31.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.525+0000 7f07bd675700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07b80aeab0 con 0x7f07b80a79f0 2023-10-08T00:14:31.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.525+0000 7f07bd675700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07b80b2b60 con 0x7f07b80b15e0 2023-10-08T00:14:31.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.525+0000 7f07b6ffd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f07b80a79f0 0x7f07b806eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:31.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.525+0000 7f07b77fe700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f07b80a7dc0 0x7f07b815dae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:44484/0 (socket says 172.21.15.99:44484) 2023-10-08T00:14:31.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.526+0000 7f07b77fe700 1 -- 172.21.15.99:0/2141778097 learned_addr learned my addr 172.21.15.99:0/2141778097 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:14:31.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.526+0000 7f07affff700 1 --2- 172.21.15.99:0/2141778097 >> v2:172.21.15.191:3300/0 conn(0x7f07b80b15e0 0x7f07b806eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:31.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.526+0000 7f07b6ffd700 1 -- 172.21.15.99:0/2141778097 >> v2:172.21.15.191:3300/0 conn(0x7f07b80b15e0 msgr2=0x7f07b806eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:31.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.526+0000 7f07b6ffd700 1 --2- 172.21.15.99:0/2141778097 >> v2:172.21.15.191:3300/0 conn(0x7f07b80b15e0 0x7f07b806eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:31.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.526+0000 7f07b6ffd700 1 -- 172.21.15.99:0/2141778097 >> v1:172.21.15.99:6789/0 conn(0x7f07b80a7dc0 legacy=0x7f07b815dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:31.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.526+0000 7f07b6ffd700 1 -- 172.21.15.99:0/2141778097 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07b80b1ac0 con 0x7f07b80a79f0 2023-10-08T00:14:31.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.526+0000 7f07b67fc700 1 -- 172.21.15.99:0/2141778097 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2329973592 0 0) 0x7f07b80b26d0 con 0x7f07b80a7dc0 2023-10-08T00:14:31.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.526+0000 7f07b6ffd700 1 --2- 172.21.15.99:0/2141778097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07b80a79f0 0x7f07b806eab0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f07b8064320 tx=0x7f07a80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21adb0079d495e83 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:31.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.526+0000 7f07b67fc700 1 -- 172.21.15.99:0/2141778097 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a800b6b0 con 0x7f07b80a79f0 2023-10-08T00:14:31.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.526+0000 7f07b67fc700 1 -- 172.21.15.99:0/2141778097 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07a800b810 con 0x7f07b80a79f0 2023-10-08T00:14:31.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.526+0000 7f07b67fc700 1 -- 172.21.15.99:0/2141778097 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a800bbb0 con 0x7f07b80a79f0 2023-10-08T00:14:31.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.527+0000 7f07bd675700 1 -- 172.21.15.99:0/2141778097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07b80a79f0 msgr2=0x7f07b806eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:31.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.527+0000 7f07bd675700 1 --2- 172.21.15.99:0/2141778097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07b80a79f0 0x7f07b806eab0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f07b8064320 tx=0x7f07a80091a0 comp rx=0 tx=0).stop 2023-10-08T00:14:31.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.527+0000 7f07bd675700 1 -- 172.21.15.99:0/2141778097 shutdown_connections 2023-10-08T00:14:31.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.527+0000 7f07bd675700 1 --2- 172.21.15.99:0/2141778097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07b80a79f0 0x7f07b806eab0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:31.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.527+0000 7f07bd675700 1 --2- 172.21.15.99:0/2141778097 >> v2:172.21.15.191:3300/0 conn(0x7f07b80b15e0 0x7f07b806eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:31.530 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.527+0000 7f07bd675700 1 -- 172.21.15.99:0/2141778097 >> 172.21.15.99:0/2141778097 conn(0x7f07b80cd1e0 msgr2=0x7f07b80cd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:31.530 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.527+0000 7f07bd675700 1 -- 172.21.15.99:0/2141778097 shutdown_connections 2023-10-08T00:14:31.530 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.527+0000 7f07bd675700 1 -- 172.21.15.99:0/2141778097 wait complete. 2023-10-08T00:14:31.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.528+0000 7f07bd675700 1 Processor -- start 2023-10-08T00:14:31.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.528+0000 7f07bd675700 1 -- start start 2023-10-08T00:14:31.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.529+0000 7f07bd675700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07b80a79f0 0x7f07b81571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:31.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.529+0000 7f07bd675700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f07b80a7dc0 0x7f07b80cb570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:31.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.529+0000 7f07bd675700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f07b80b15e0 0x7f07b80cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:31.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.529+0000 7f07bd675700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b80cc130 con 0x7f07b80a79f0 2023-10-08T00:14:31.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.529+0000 7f07bd675700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b8099ed0 con 0x7f07b80b15e0 2023-10-08T00:14:31.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.529+0000 7f07bd675700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07b80995a0 con 0x7f07b80a7dc0 2023-10-08T00:14:31.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.529+0000 7f07b6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07b80a79f0 0x7f07b81571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:31.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.529+0000 7f07b6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07b80a79f0 0x7f07b81571a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43428/0 (socket says 172.21.15.99:43428) 2023-10-08T00:14:31.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.529+0000 7f07b6ffd700 1 -- 172.21.15.99:0/3789692040 learned_addr learned my addr 172.21.15.99:0/3789692040 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:31.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.529+0000 7f07b6ffd700 1 -- 172.21.15.99:0/3789692040 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f07b80a7dc0 msgr2=0x7f07b80cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:31.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.529+0000 7f07b6ffd700 1 --2- 172.21.15.99:0/3789692040 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f07b80a7dc0 0x7f07b80cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:31.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.529+0000 7f07b6ffd700 1 -- 172.21.15.99:0/3789692040 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f07b80b15e0 msgr2=0x7f07b80cbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:31.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.529+0000 7f07b6ffd700 1 --2- 172.21.15.99:0/3789692040 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f07b80b15e0 0x7f07b80cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:31.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.529+0000 7f07b6ffd700 1 -- 172.21.15.99:0/3789692040 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a8009420 con 0x7f07b80a79f0 2023-10-08T00:14:31.534 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.529+0000 7f07b6ffd700 1 --2- 172.21.15.99:0/3789692040 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07b80a79f0 0x7f07b81571a0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f07a800b0e0 tx=0x7f07a8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:31.534 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.529+0000 7f07aeffd700 1 -- 172.21.15.99:0/3789692040 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a8011420 con 0x7f07b80a79f0 2023-10-08T00:14:31.534 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.529+0000 7f07aeffd700 1 -- 172.21.15.99:0/3789692040 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07a8011580 con 0x7f07b80a79f0 2023-10-08T00:14:31.534 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.529+0000 7f07bd675700 1 -- 172.21.15.99:0/3789692040 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07b8099880 con 0x7f07b80a79f0 2023-10-08T00:14:31.534 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.529+0000 7f07aeffd700 1 -- 172.21.15.99:0/3789692040 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a8011920 con 0x7f07b80a79f0 2023-10-08T00:14:31.534 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.529+0000 7f07bd675700 1 -- 172.21.15.99:0/3789692040 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07b81576e0 con 0x7f07b80a79f0 2023-10-08T00:14:31.534 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.530+0000 7f07bd675700 1 -- 172.21.15.99:0/3789692040 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07b8157930 con 0x7f07b80a79f0 2023-10-08T00:14:31.534 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.530+0000 7f07aeffd700 1 -- 172.21.15.99:0/3789692040 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f07a8011d10 con 0x7f07b80a79f0 2023-10-08T00:14:31.534 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.531+0000 7f07aeffd700 1 --2- 172.21.15.99:0/3789692040 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0798066f60 0x7f0798069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:31.534 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.531+0000 7f07aeffd700 1 -- 172.21.15.99:0/3789692040 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f07a8086020 con 0x7f07b80a79f0 2023-10-08T00:14:31.534 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.531+0000 7f07affff700 1 --2- 172.21.15.99:0/3789692040 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0798066f60 0x7f0798069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:31.535 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.531+0000 7f07aeffd700 1 -- 172.21.15.99:0/3789692040 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=75}) v3 -- 0x7f079806eb40 con 0x7f07b80a79f0 2023-10-08T00:14:31.535 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.532+0000 7f07affff700 1 --2- 172.21.15.99:0/3789692040 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0798066f60 0x7f0798069410 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f07a0006fd0 tx=0x7f07a0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:31.536 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.535+0000 7f07aeffd700 1 -- 172.21.15.99:0/3789692040 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f07a8015060 con 0x7f07b80a79f0 2023-10-08T00:14:31.719 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.719+0000 7f07bd675700 1 -- 172.21.15.99:0/3789692040 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f07b804c3c0 con 0x7f07b80a79f0 2023-10-08T00:14:31.720 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.720+0000 7f07aeffd700 1 -- 172.21.15.99:0/3789692040 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+15497 (secure 0 0 0) 0x7f07a80532d0 con 0x7f07b80a79f0 2023-10-08T00:14:31.720 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:14:31.720 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":74,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:13:44.132886+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:12:55.353771+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-08T00:12:56.552157+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["backfillfull","exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","full","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","full","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.28","mappings":[{"from":6,"to":4}]},{"pgid":"9.3e","mappings":[{"from":6,"to":1}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:14:31.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.722+0000 7f07bd675700 1 -- 172.21.15.99:0/3789692040 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0798066f60 msgr2=0x7f0798069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:31.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.722+0000 7f07bd675700 1 --2- 172.21.15.99:0/3789692040 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0798066f60 0x7f0798069410 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f07a0006fd0 tx=0x7f07a0008040 comp rx=0 tx=0).stop 2023-10-08T00:14:31.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.722+0000 7f07bd675700 1 -- 172.21.15.99:0/3789692040 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07b80a79f0 msgr2=0x7f07b81571a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:31.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.722+0000 7f07bd675700 1 --2- 172.21.15.99:0/3789692040 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07b80a79f0 0x7f07b81571a0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f07a800b0e0 tx=0x7f07a8017000 comp rx=0 tx=0).stop 2023-10-08T00:14:31.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.722+0000 7f07bd675700 1 -- 172.21.15.99:0/3789692040 shutdown_connections 2023-10-08T00:14:31.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.722+0000 7f07bd675700 1 --2- 172.21.15.99:0/3789692040 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0798066f60 0x7f0798069410 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:31.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.722+0000 7f07bd675700 1 --2- 172.21.15.99:0/3789692040 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f07b80b15e0 0x7f07b80cbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:31.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.722+0000 7f07bd675700 1 --2- 172.21.15.99:0/3789692040 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07b80a79f0 0x7f07b81571a0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:31.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.722+0000 7f07bd675700 1 --2- 172.21.15.99:0/3789692040 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f07b80a7dc0 0x7f07b80cb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:31.724 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.722+0000 7f07bd675700 1 -- 172.21.15.99:0/3789692040 >> 172.21.15.99:0/3789692040 conn(0x7f07b80cd1e0 msgr2=0x7f07b806daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:31.724 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.722+0000 7f07bd675700 1 -- 172.21.15.99:0/3789692040 shutdown_connections 2023-10-08T00:14:31.724 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.722+0000 7f07bd675700 1 -- 172.21.15.99:0/3789692040 wait complete. 2023-10-08T00:14:31.735 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T00:14:31.850 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.849+0000 7f967586f700 1 Processor -- start 2023-10-08T00:14:31.850 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.849+0000 7f967586f700 1 -- start start 2023-10-08T00:14:31.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.850+0000 7f967586f700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f9670157870 0x7f967015b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:31.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.850+0000 7f967586f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f967015a620 0x7f967015d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:31.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.850+0000 7f967586f700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f967015cbb0 con 0x7f967015d580 2023-10-08T00:14:31.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.850+0000 7f967586f700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f967015bfa0 con 0x7f9670157870 2023-10-08T00:14:31.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.850+0000 7f967586f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f967015c280 con 0x7f967015a620 2023-10-08T00:14:31.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.850+0000 7f96667fc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f967015a620 0x7f967015d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:31.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.850+0000 7f966effd700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f9670157870 0x7f967015b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:31.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.850+0000 7f96667fc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f967015a620 0x7f967015d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.99:59166/0 (socket says 172.21.15.99:59166) 2023-10-08T00:14:31.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.850+0000 7f966effd700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f9670157870 0x7f967015b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.99:42838/0 (socket says 172.21.15.99:42838) 2023-10-08T00:14:31.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.850+0000 7f96667fc700 1 -- 172.21.15.99:0/3695103279 learned_addr learned my addr 172.21.15.99:0/3695103279 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:31.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.850+0000 7f966f7fe700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f967015d580 0x7f9670158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:57476/0 (socket says 172.21.15.99:57476) 2023-10-08T00:14:31.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.851+0000 7f96667fc700 1 -- 172.21.15.99:0/3695103279 >> v2:172.21.15.191:3300/0 conn(0x7f9670157870 msgr2=0x7f967015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:31.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.851+0000 7f96667fc700 1 --2- 172.21.15.99:0/3695103279 >> v2:172.21.15.191:3300/0 conn(0x7f9670157870 0x7f967015b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:31.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.851+0000 7f96667fc700 1 -- 172.21.15.99:0/3695103279 >> v1:172.21.15.191:6789/0 conn(0x7f967015d580 legacy=0x7f9670158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:31.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.851+0000 7f96667fc700 1 -- 172.21.15.99:0/3695103279 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f967015c560 con 0x7f967015a620 2023-10-08T00:14:31.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.851+0000 7f966effd700 1 --2- 172.21.15.99:0/3695103279 >> v2:172.21.15.191:3300/0 conn(0x7f9670157870 0x7f967015b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:14:31.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.851+0000 7f96667fc700 1 --2- 172.21.15.99:0/3695103279 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f967015a620 0x7f967015d040 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f96580099b0 tx=0x7f9658009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df655b44f1042695 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:31.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.851+0000 7f966e7fc700 1 -- 172.21.15.99:0/3695103279 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9658009ea0 con 0x7f967015a620 2023-10-08T00:14:31.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.851+0000 7f966e7fc700 1 -- 172.21.15.99:0/3695103279 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f965800b860 con 0x7f967015a620 2023-10-08T00:14:31.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.851+0000 7f966e7fc700 1 -- 172.21.15.99:0/3695103279 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965800bc00 con 0x7f967015a620 2023-10-08T00:14:31.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.851+0000 7f967586f700 1 -- 172.21.15.99:0/3695103279 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f967015a620 msgr2=0x7f967015d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:31.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.851+0000 7f967586f700 1 --2- 172.21.15.99:0/3695103279 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f967015a620 0x7f967015d040 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f96580099b0 tx=0x7f9658009250 comp rx=0 tx=0).stop 2023-10-08T00:14:31.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.851+0000 7f967586f700 1 -- 172.21.15.99:0/3695103279 shutdown_connections 2023-10-08T00:14:31.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.851+0000 7f967586f700 1 --2- 172.21.15.99:0/3695103279 >> v2:172.21.15.191:3300/0 conn(0x7f9670157870 0x7f967015b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:31.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.851+0000 7f967586f700 1 --2- 172.21.15.99:0/3695103279 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f967015a620 0x7f967015d040 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:31.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.852+0000 7f967586f700 1 -- 172.21.15.99:0/3695103279 >> 172.21.15.99:0/3695103279 conn(0x7f9670068ce0 msgr2=0x7f96700690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:31.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.852+0000 7f967586f700 1 -- 172.21.15.99:0/3695103279 shutdown_connections 2023-10-08T00:14:31.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.852+0000 7f967586f700 1 -- 172.21.15.99:0/3695103279 wait complete. 2023-10-08T00:14:31.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.853+0000 7f967586f700 1 Processor -- start 2023-10-08T00:14:31.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.853+0000 7f967586f700 1 -- start start 2023-10-08T00:14:31.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.853+0000 7f967586f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9670157870 0x7f96700b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:31.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.853+0000 7f967586f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f967015a620 0x7f9670151780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:31.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.853+0000 7f967586f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f967015d580 0x7f9670152730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:31.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.853+0000 7f967586f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96700b20d0 con 0x7f967015d580 2023-10-08T00:14:31.859 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.853+0000 7f967586f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f967016b620 con 0x7f967015a620 2023-10-08T00:14:31.859 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.853+0000 7f967586f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f967016aa10 con 0x7f9670157870 2023-10-08T00:14:31.859 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.854+0000 7f966f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f967015d580 0x7f9670152730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:31.859 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.854+0000 7f966f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f967015d580 0x7f9670152730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43434/0 (socket says 172.21.15.99:43434) 2023-10-08T00:14:31.859 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.854+0000 7f966f7fe700 1 -- 172.21.15.99:0/1300204479 learned_addr learned my addr 172.21.15.99:0/1300204479 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:31.859 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.854+0000 7f966effd700 1 --2- 172.21.15.99:0/1300204479 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9670157870 0x7f96700b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:31.859 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.854+0000 7f96667fc700 1 --2- 172.21.15.99:0/1300204479 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f967015a620 0x7f9670151780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:31.859 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.854+0000 7f966f7fe700 1 -- 172.21.15.99:0/1300204479 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9670157870 msgr2=0x7f96700b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:31.859 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.854+0000 7f966f7fe700 1 --2- 172.21.15.99:0/1300204479 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9670157870 0x7f96700b1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:31.859 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.854+0000 7f966f7fe700 1 -- 172.21.15.99:0/1300204479 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f967015a620 msgr2=0x7f9670151780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:31.859 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.854+0000 7f966f7fe700 1 --2- 172.21.15.99:0/1300204479 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f967015a620 0x7f9670151780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:31.859 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.854+0000 7f966f7fe700 1 -- 172.21.15.99:0/1300204479 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9660009420 con 0x7f967015d580 2023-10-08T00:14:31.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.854+0000 7f966f7fe700 1 --2- 172.21.15.99:0/1300204479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f967015d580 0x7f9670152730 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f966800cc70 tx=0x7f9668001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:31.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.854+0000 7f96677fe700 1 -- 172.21.15.99:0/1300204479 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9668005df0 con 0x7f967015d580 2023-10-08T00:14:31.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.854+0000 7f96677fe700 1 -- 172.21.15.99:0/1300204479 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f966800a640 con 0x7f967015d580 2023-10-08T00:14:31.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.854+0000 7f967586f700 1 -- 172.21.15.99:0/1300204479 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96580094f0 con 0x7f967015d580 2023-10-08T00:14:31.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.854+0000 7f96677fe700 1 -- 172.21.15.99:0/1300204479 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f966800a9c0 con 0x7f967015d580 2023-10-08T00:14:31.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.854+0000 7f967586f700 1 -- 172.21.15.99:0/1300204479 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f967016afd0 con 0x7f967015d580 2023-10-08T00:14:31.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.855+0000 7f967586f700 1 -- 172.21.15.99:0/1300204479 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f967016dca0 con 0x7f967015d580 2023-10-08T00:14:31.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.856+0000 7f96677fe700 1 -- 172.21.15.99:0/1300204479 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f96680070f0 con 0x7f967015d580 2023-10-08T00:14:31.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.856+0000 7f96677fe700 1 --2- 172.21.15.99:0/1300204479 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9650066f30 0x7f96500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:31.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.856+0000 7f96677fe700 1 -- 172.21.15.99:0/1300204479 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f96680844b0 con 0x7f967015d580 2023-10-08T00:14:31.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.856+0000 7f966effd700 1 --2- 172.21.15.99:0/1300204479 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9650066f30 0x7f96500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:31.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.856+0000 7f96677fe700 1 -- 172.21.15.99:0/1300204479 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=75}) v3 -- 0x7f965006eb10 con 0x7f967015d580 2023-10-08T00:14:31.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.857+0000 7f966effd700 1 --2- 172.21.15.99:0/1300204479 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9650066f30 0x7f96500693e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9660009f40 tx=0x7f96600091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:31.861 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:31.860+0000 7f96677fe700 1 -- 172.21.15.99:0/1300204479 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9668051810 con 0x7f967015d580 2023-10-08T00:14:32.083 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.083+0000 7f967586f700 1 -- 172.21.15.99:0/1300204479 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f967004c3c0 con 0x7f967015d580 2023-10-08T00:14:32.084 INFO:tasks.ceph.mon.a.smithi099.stderr:2023-10-08T00:14:32.083+0000 7f6c60fa2700 -1 log_channel(cluster) log [ERR] : Health check update: 1 full osd(s) (OSD_FULL) 2023-10-08T00:14:32.084 INFO:tasks.ceph.mon.a.smithi099.stderr:2023-10-08T00:14:32.083+0000 7f6c60fa2700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T00:14:32.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.085+0000 7f96677fe700 1 -- 172.21.15.99:0/1300204479 <== mon.0 v2:172.21.15.99:3300/0 7 ==== osd_map(75..75 src has 1..75) v4 ==== 390+0+0 (secure 0 0 0) 0x7f96680554d0 con 0x7f967015d580 2023-10-08T00:14:32.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.085+0000 7f96677fe700 1 -- 172.21.15.99:0/1300204479 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=76}) v3 -- 0x7f965006f460 con 0x7f967015d580 2023-10-08T00:14:32.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.088+0000 7f96677fe700 1 -- 172.21.15.99:0/1300204479 <== mon.0 v2:172.21.15.99:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v33) v1 ==== 161+0+0 (secure 0 0 0) 0x7f9668054e30 con 0x7f967015d580 2023-10-08T00:14:32.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.090+0000 7f967586f700 1 -- 172.21.15.99:0/1300204479 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9650066f30 msgr2=0x7f96500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:32.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.090+0000 7f967586f700 1 --2- 172.21.15.99:0/1300204479 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9650066f30 0x7f96500693e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9660009f40 tx=0x7f96600091a0 comp rx=0 tx=0).stop 2023-10-08T00:14:32.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.090+0000 7f967586f700 1 -- 172.21.15.99:0/1300204479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f967015d580 msgr2=0x7f9670152730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:32.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.090+0000 7f967586f700 1 --2- 172.21.15.99:0/1300204479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f967015d580 0x7f9670152730 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f966800cc70 tx=0x7f9668001a40 comp rx=0 tx=0).stop 2023-10-08T00:14:32.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.090+0000 7f967586f700 1 -- 172.21.15.99:0/1300204479 shutdown_connections 2023-10-08T00:14:32.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.090+0000 7f967586f700 1 --2- 172.21.15.99:0/1300204479 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9650066f30 0x7f96500693e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:32.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.090+0000 7f967586f700 1 --2- 172.21.15.99:0/1300204479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f967015d580 0x7f9670152730 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:32.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.090+0000 7f967586f700 1 --2- 172.21.15.99:0/1300204479 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9670157870 0x7f96700b1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:32.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.090+0000 7f967586f700 1 --2- 172.21.15.99:0/1300204479 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f967015a620 0x7f9670151780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:32.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.090+0000 7f967586f700 1 -- 172.21.15.99:0/1300204479 >> 172.21.15.99:0/1300204479 conn(0x7f9670068ce0 msgr2=0x7f9670153320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:32.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.090+0000 7f967586f700 1 -- 172.21.15.99:0/1300204479 shutdown_connections 2023-10-08T00:14:32.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.090+0000 7f967586f700 1 -- 172.21.15.99:0/1300204479 wait complete. 2023-10-08T00:14:32.092 INFO:teuthology.orchestra.run.smithi099.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T00:14:32.093 INFO:tasks.ceph.mds.d.smithi191.stderr: -9322> 2023-10-08T00:13:03.186+0000 7fb7ed67f700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T00:14:32.103 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T00:14:32.124 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2023-10-08T00:14:32.221 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.220+0000 7fe98e02b700 1 Processor -- start 2023-10-08T00:14:32.221 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.220+0000 7fe98e02b700 1 -- start start 2023-10-08T00:14:32.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.221+0000 7fe98e02b700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe98815afb0 0x7fe9880d0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:32.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.221+0000 7fe98e02b700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9881513f0 con 0x7fe9880d12a0 2023-10-08T00:14:32.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.221+0000 7fe98e02b700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9880d1700 con 0x7fe9880cd300 2023-10-08T00:14:32.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.221+0000 7fe98e02b700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe98815e7a0 con 0x7fe98815afb0 2023-10-08T00:14:32.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.221+0000 7fe986ffd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe98815afb0 0x7fe9880d0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:32.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.221+0000 7fe987fff700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fe9880d12a0 0x7fe9880cf310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:44490/0 (socket says 172.21.15.99:44490) 2023-10-08T00:14:32.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.221+0000 7fe987fff700 1 -- 172.21.15.99:0/960385584 learned_addr learned my addr 172.21.15.99:0/960385584 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:14:32.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.221+0000 7fe986ffd700 1 -- 172.21.15.99:0/960385584 >> v1:172.21.15.191:6789/0 conn(0x7fe9880cd300 legacy=0x7fe9880b6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:32.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.221+0000 7fe986ffd700 1 -- 172.21.15.99:0/960385584 >> v1:172.21.15.99:6789/0 conn(0x7fe9880d12a0 legacy=0x7fe9880cf310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:32.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.221+0000 7fe986ffd700 1 -- 172.21.15.99:0/960385584 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe98815db20 con 0x7fe98815afb0 2023-10-08T00:14:32.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.221+0000 7fe9867fc700 1 -- 172.21.15.99:0/960385584 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3804350781 0 0) 0x7fe9881513f0 con 0x7fe9880d12a0 2023-10-08T00:14:32.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.221+0000 7fe986ffd700 1 --2- 172.21.15.99:0/960385584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe98815afb0 0x7fe9880d0d60 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fe97c0098e0 tx=0x7fe97c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9760f1505d845997 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:32.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.221+0000 7fe9867fc700 1 -- 172.21.15.99:0/960385584 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe97c009e80 con 0x7fe98815afb0 2023-10-08T00:14:32.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.222+0000 7fe9867fc700 1 -- 172.21.15.99:0/960385584 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe97c00b820 con 0x7fe98815afb0 2023-10-08T00:14:32.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.222+0000 7fe9867fc700 1 -- 172.21.15.99:0/960385584 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe97c00bba0 con 0x7fe98815afb0 2023-10-08T00:14:32.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.222+0000 7fe98e02b700 1 -- 172.21.15.99:0/960385584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe98815afb0 msgr2=0x7fe9880d0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:32.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.222+0000 7fe98e02b700 1 --2- 172.21.15.99:0/960385584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe98815afb0 0x7fe9880d0d60 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fe97c0098e0 tx=0x7fe97c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:14:32.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.222+0000 7fe98e02b700 1 -- 172.21.15.99:0/960385584 shutdown_connections 2023-10-08T00:14:32.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.222+0000 7fe98e02b700 1 --2- 172.21.15.99:0/960385584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe98815afb0 0x7fe9880d0d60 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:32.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.222+0000 7fe98e02b700 1 -- 172.21.15.99:0/960385584 >> 172.21.15.99:0/960385584 conn(0x7fe988068d90 msgr2=0x7fe988069190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:32.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.222+0000 7fe98e02b700 1 -- 172.21.15.99:0/960385584 shutdown_connections 2023-10-08T00:14:32.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.222+0000 7fe98e02b700 1 -- 172.21.15.99:0/960385584 wait complete. 2023-10-08T00:14:32.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.223+0000 7fe98e02b700 1 Processor -- start 2023-10-08T00:14:32.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.223+0000 7fe98e02b700 1 -- start start 2023-10-08T00:14:32.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.224+0000 7fe98e02b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9880cd300 0x7fe98813fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:32.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.224+0000 7fe98e02b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe9880d12a0 0x7fe98813d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:32.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.224+0000 7fe98e02b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe98815afb0 0x7fe9881425f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:32.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.224+0000 7fe98e02b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98813e6d0 con 0x7fe9880cd300 2023-10-08T00:14:32.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.224+0000 7fe98e02b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98813dac0 con 0x7fe9880d12a0 2023-10-08T00:14:32.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.224+0000 7fe98e02b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe98813dda0 con 0x7fe98815afb0 2023-10-08T00:14:32.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.224+0000 7fe9877fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9880cd300 0x7fe98813fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:32.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.224+0000 7fe9877fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9880cd300 0x7fe98813fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43446/0 (socket says 172.21.15.99:43446) 2023-10-08T00:14:32.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.224+0000 7fe9877fe700 1 -- 172.21.15.99:0/2402120457 learned_addr learned my addr 172.21.15.99:0/2402120457 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:32.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.224+0000 7fe986ffd700 1 --2- 172.21.15.99:0/2402120457 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe9880d12a0 0x7fe98813d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:32.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.225+0000 7fe987fff700 1 --2- 172.21.15.99:0/2402120457 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe98815afb0 0x7fe9881425f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:32.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.225+0000 7fe9877fe700 1 -- 172.21.15.99:0/2402120457 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe98815afb0 msgr2=0x7fe9881425f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:32.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.225+0000 7fe9877fe700 1 --2- 172.21.15.99:0/2402120457 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe98815afb0 0x7fe9881425f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:32.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.225+0000 7fe9877fe700 1 -- 172.21.15.99:0/2402120457 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe9880d12a0 msgr2=0x7fe98813d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:32.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.225+0000 7fe9877fe700 1 --2- 172.21.15.99:0/2402120457 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe9880d12a0 0x7fe98813d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:32.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.225+0000 7fe9877fe700 1 -- 172.21.15.99:0/2402120457 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe97c009420 con 0x7fe9880cd300 2023-10-08T00:14:32.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.225+0000 7fe987fff700 1 --2- 172.21.15.99:0/2402120457 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe98815afb0 0x7fe9881425f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:14:32.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.225+0000 7fe9877fe700 1 --2- 172.21.15.99:0/2402120457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9880cd300 0x7fe98813fd80 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fe97800c950 tx=0x7fe978002760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:32.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.225+0000 7fe96f7fe700 1 -- 172.21.15.99:0/2402120457 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe97800a7e0 con 0x7fe9880cd300 2023-10-08T00:14:32.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.225+0000 7fe96f7fe700 1 -- 172.21.15.99:0/2402120457 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe97800a940 con 0x7fe9880cd300 2023-10-08T00:14:32.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.225+0000 7fe96f7fe700 1 -- 172.21.15.99:0/2402120457 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe97800ad00 con 0x7fe9880cd300 2023-10-08T00:14:32.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.225+0000 7fe98e02b700 1 -- 172.21.15.99:0/2402120457 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe98813e080 con 0x7fe9880cd300 2023-10-08T00:14:32.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.225+0000 7fe98e02b700 1 -- 172.21.15.99:0/2402120457 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe98813d7b0 con 0x7fe9880cd300 2023-10-08T00:14:32.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.226+0000 7fe98e02b700 1 -- 172.21.15.99:0/2402120457 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe98804c3c0 con 0x7fe9880cd300 2023-10-08T00:14:32.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.226+0000 7fe96f7fe700 1 -- 172.21.15.99:0/2402120457 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe978012630 con 0x7fe9880cd300 2023-10-08T00:14:32.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.227+0000 7fe96f7fe700 1 --2- 172.21.15.99:0/2402120457 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe974066f30 0x7fe9740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:32.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.227+0000 7fe96f7fe700 1 -- 172.21.15.99:0/2402120457 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5849+0+0 (secure 0 0 0) 0x7fe978084b20 con 0x7fe9880cd300 2023-10-08T00:14:32.231 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.227+0000 7fe96f7fe700 1 -- 172.21.15.99:0/2402120457 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=76}) v3 -- 0x7fe97406ebb0 con 0x7fe9880cd300 2023-10-08T00:14:32.231 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.227+0000 7fe986ffd700 1 --2- 172.21.15.99:0/2402120457 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe974066f30 0x7fe9740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:32.231 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.228+0000 7fe986ffd700 1 --2- 172.21.15.99:0/2402120457 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe974066f30 0x7fe9740693e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe97c0030c0 tx=0x7fe97c016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:32.231 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.231+0000 7fe96f7fe700 1 -- 172.21.15.99:0/2402120457 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe978016020 con 0x7fe9880cd300 2023-10-08T00:14:32.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:32.459+0000 7fe98e02b700 1 -- 172.21.15.99:0/2402120457 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe9880b4010 con 0x7fe9880cd300 2023-10-08T00:14:33.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.088+0000 7fe96f7fe700 1 -- 172.21.15.99:0/2402120457 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v34) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe978051db0 con 0x7fe9880cd300 2023-10-08T00:14:33.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.090+0000 7fe98e02b700 1 -- 172.21.15.99:0/2402120457 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe974066f30 msgr2=0x7fe9740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:33.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.090+0000 7fe98e02b700 1 --2- 172.21.15.99:0/2402120457 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe974066f30 0x7fe9740693e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe97c0030c0 tx=0x7fe97c016000 comp rx=0 tx=0).stop 2023-10-08T00:14:33.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.090+0000 7fe98e02b700 1 -- 172.21.15.99:0/2402120457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9880cd300 msgr2=0x7fe98813fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:33.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.090+0000 7fe98e02b700 1 --2- 172.21.15.99:0/2402120457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9880cd300 0x7fe98813fd80 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fe97800c950 tx=0x7fe978002760 comp rx=0 tx=0).stop 2023-10-08T00:14:33.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.090+0000 7fe98e02b700 1 -- 172.21.15.99:0/2402120457 shutdown_connections 2023-10-08T00:14:33.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.090+0000 7fe98e02b700 1 --2- 172.21.15.99:0/2402120457 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe974066f30 0x7fe9740693e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:33.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.090+0000 7fe98e02b700 1 --2- 172.21.15.99:0/2402120457 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe98815afb0 0x7fe9881425f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:33.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.090+0000 7fe98e02b700 1 --2- 172.21.15.99:0/2402120457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9880cd300 0x7fe98813fd80 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:33.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.090+0000 7fe98e02b700 1 --2- 172.21.15.99:0/2402120457 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe9880d12a0 0x7fe98813d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:33.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.090+0000 7fe98e02b700 1 -- 172.21.15.99:0/2402120457 >> 172.21.15.99:0/2402120457 conn(0x7fe988068d90 msgr2=0x7fe9881508a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:33.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.090+0000 7fe98e02b700 1 -- 172.21.15.99:0/2402120457 shutdown_connections 2023-10-08T00:14:33.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.090+0000 7fe98e02b700 1 -- 172.21.15.99:0/2402120457 wait complete. 2023-10-08T00:14:33.103 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T00:14:33.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.222+0000 7fa0fa761700 1 Processor -- start 2023-10-08T00:14:33.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.222+0000 7fa0fa761700 1 -- start start 2023-10-08T00:14:33.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.223+0000 7fa0fa761700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0f415d240 0x7fa0f40b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:33.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.223+0000 7fa0fa761700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa0f40cc940 0x7fa0f40ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:33.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.223+0000 7fa0fa761700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0f40b3f40 con 0x7fa0f40c59e0 2023-10-08T00:14:33.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.223+0000 7fa0fa761700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0f40cbb80 con 0x7fa0f40cc940 2023-10-08T00:14:33.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.223+0000 7fa0fa761700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f40cbe60 con 0x7fa0f415d240 2023-10-08T00:14:33.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.223+0000 7fa0f9f60700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fa0f40c59e0 0x7fa0f40c5e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:44496/0 (socket says 172.21.15.99:44496) 2023-10-08T00:14:33.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.223+0000 7fa0f9f60700 1 -- 172.21.15.99:0/3328394545 learned_addr learned my addr 172.21.15.99:0/3328394545 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:14:33.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.223+0000 7fa0f8f5e700 1 --2- 172.21.15.99:0/3328394545 >> v2:172.21.15.191:3300/0 conn(0x7fa0f40cc940 0x7fa0f40ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:33.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.223+0000 7fa0f975f700 1 --2- 172.21.15.99:0/3328394545 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0f415d240 0x7fa0f40b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:33.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.224+0000 7fa0ebfff700 1 -- 172.21.15.99:0/3328394545 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4054160590 0 0) 0x7fa0f40b3f40 con 0x7fa0f40c59e0 2023-10-08T00:14:33.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.224+0000 7fa0ebfff700 1 -- 172.21.15.99:0/3328394545 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0e0003570 con 0x7fa0f40c59e0 2023-10-08T00:14:33.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.224+0000 7fa0f975f700 1 -- 172.21.15.99:0/3328394545 >> v2:172.21.15.191:3300/0 conn(0x7fa0f40cc940 msgr2=0x7fa0f40ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:33.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.224+0000 7fa0f975f700 1 --2- 172.21.15.99:0/3328394545 >> v2:172.21.15.191:3300/0 conn(0x7fa0f40cc940 0x7fa0f40ccdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:33.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.224+0000 7fa0f975f700 1 -- 172.21.15.99:0/3328394545 >> v1:172.21.15.99:6789/0 conn(0x7fa0f40c59e0 legacy=0x7fa0f40c5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:33.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.224+0000 7fa0f975f700 1 -- 172.21.15.99:0/3328394545 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f40cc140 con 0x7fa0f415d240 2023-10-08T00:14:33.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.224+0000 7fa0f8f5e700 1 --2- 172.21.15.99:0/3328394545 >> v2:172.21.15.191:3300/0 conn(0x7fa0f40cc940 0x7fa0f40ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:14:33.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.224+0000 7fa0f975f700 1 --2- 172.21.15.99:0/3328394545 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0f415d240 0x7fa0f40b3a00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa0f00098d0 tx=0x7fa0f0009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4027fb1c010f3280 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:33.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.224+0000 7fa0ebfff700 1 -- 172.21.15.99:0/3328394545 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0005c10 con 0x7fa0f415d240 2023-10-08T00:14:33.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.224+0000 7fa0ebfff700 1 -- 172.21.15.99:0/3328394545 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0f0005d70 con 0x7fa0f415d240 2023-10-08T00:14:33.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.224+0000 7fa0ebfff700 1 -- 172.21.15.99:0/3328394545 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0006240 con 0x7fa0f415d240 2023-10-08T00:14:33.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.225+0000 7fa0fa761700 1 -- 172.21.15.99:0/3328394545 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0f415d240 msgr2=0x7fa0f40b3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:33.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.225+0000 7fa0fa761700 1 --2- 172.21.15.99:0/3328394545 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0f415d240 0x7fa0f40b3a00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa0f00098d0 tx=0x7fa0f0009120 comp rx=0 tx=0).stop 2023-10-08T00:14:33.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.225+0000 7fa0fa761700 1 -- 172.21.15.99:0/3328394545 shutdown_connections 2023-10-08T00:14:33.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.225+0000 7fa0fa761700 1 --2- 172.21.15.99:0/3328394545 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0f415d240 0x7fa0f40b3a00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:33.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.225+0000 7fa0fa761700 1 --2- 172.21.15.99:0/3328394545 >> v2:172.21.15.191:3300/0 conn(0x7fa0f40cc940 0x7fa0f40ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:33.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.225+0000 7fa0fa761700 1 -- 172.21.15.99:0/3328394545 >> 172.21.15.99:0/3328394545 conn(0x7fa0f4067e40 msgr2=0x7fa0f40d0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:33.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.225+0000 7fa0fa761700 1 -- 172.21.15.99:0/3328394545 shutdown_connections 2023-10-08T00:14:33.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.225+0000 7fa0fa761700 1 -- 172.21.15.99:0/3328394545 wait complete. 2023-10-08T00:14:33.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.226+0000 7fa0fa761700 1 Processor -- start 2023-10-08T00:14:33.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.226+0000 7fa0fa761700 1 -- start start 2023-10-08T00:14:33.232 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.227+0000 7fa0fa761700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0f40c59e0 0x7fa0f409d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:33.232 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.227+0000 7fa0fa761700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0f40cc940 0x7fa0f409d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:33.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.227+0000 7fa0fa761700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0f415d240 0x7fa0f40c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:33.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.227+0000 7fa0fa761700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f416a790 con 0x7fa0f415d240 2023-10-08T00:14:33.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.227+0000 7fa0fa761700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f40ae390 con 0x7fa0f40cc940 2023-10-08T00:14:33.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.227+0000 7fa0fa761700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f409dcd0 con 0x7fa0f40c59e0 2023-10-08T00:14:33.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.227+0000 7fa0f9f60700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0f415d240 0x7fa0f40c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:33.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.227+0000 7fa0f9f60700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0f415d240 0x7fa0f40c9e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43456/0 (socket says 172.21.15.99:43456) 2023-10-08T00:14:33.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.227+0000 7fa0f9f60700 1 -- 172.21.15.99:0/3220478490 learned_addr learned my addr 172.21.15.99:0/3220478490 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:33.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.227+0000 7fa0f975f700 1 --2- 172.21.15.99:0/3220478490 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0f40c59e0 0x7fa0f409d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:33.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.227+0000 7fa0f8f5e700 1 --2- 172.21.15.99:0/3220478490 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0f40cc940 0x7fa0f409d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:33.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.227+0000 7fa0f9f60700 1 -- 172.21.15.99:0/3220478490 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0f40c59e0 msgr2=0x7fa0f409d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:33.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.227+0000 7fa0f9f60700 1 --2- 172.21.15.99:0/3220478490 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0f40c59e0 0x7fa0f409d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:33.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.227+0000 7fa0f9f60700 1 -- 172.21.15.99:0/3220478490 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0f40cc940 msgr2=0x7fa0f409d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:33.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.227+0000 7fa0f9f60700 1 --2- 172.21.15.99:0/3220478490 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0f40cc940 0x7fa0f409d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:33.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.227+0000 7fa0f9f60700 1 -- 172.21.15.99:0/3220478490 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e4009420 con 0x7fa0f415d240 2023-10-08T00:14:33.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.227+0000 7fa0f975f700 1 --2- 172.21.15.99:0/3220478490 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0f40c59e0 0x7fa0f409d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:14:33.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.227+0000 7fa0f8f5e700 1 --2- 172.21.15.99:0/3220478490 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0f40cc940 0x7fa0f409d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:14:33.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.227+0000 7fa0f9f60700 1 --2- 172.21.15.99:0/3220478490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0f415d240 0x7fa0f40c9e40 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec00cb90 tx=0x7fa0ec0019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:33.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.227+0000 7fa0e97fa700 1 -- 172.21.15.99:0/3220478490 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec00a910 con 0x7fa0f415d240 2023-10-08T00:14:33.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.227+0000 7fa0e97fa700 1 -- 172.21.15.99:0/3220478490 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0ec00aa70 con 0x7fa0f415d240 2023-10-08T00:14:33.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.227+0000 7fa0fa761700 1 -- 172.21.15.99:0/3220478490 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0f00093c0 con 0x7fa0f415d240 2023-10-08T00:14:33.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.227+0000 7fa0e97fa700 1 -- 172.21.15.99:0/3220478490 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec00ae30 con 0x7fa0f415d240 2023-10-08T00:14:33.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.227+0000 7fa0fa761700 1 -- 172.21.15.99:0/3220478490 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0f409e290 con 0x7fa0f415d240 2023-10-08T00:14:33.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.228+0000 7fa0fa761700 1 -- 172.21.15.99:0/3220478490 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0f408aeb0 con 0x7fa0f415d240 2023-10-08T00:14:33.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.229+0000 7fa0e97fa700 1 -- 172.21.15.99:0/3220478490 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa0ec007300 con 0x7fa0f415d240 2023-10-08T00:14:33.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.229+0000 7fa0e97fa700 1 --2- 172.21.15.99:0/3220478490 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa0d8066f30 0x7fa0d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:33.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.229+0000 7fa0e97fa700 1 -- 172.21.15.99:0/3220478490 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5849+0+0 (secure 0 0 0) 0x7fa0ec0849f0 con 0x7fa0f415d240 2023-10-08T00:14:33.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.229+0000 7fa0f975f700 1 --2- 172.21.15.99:0/3220478490 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa0d8066f30 0x7fa0d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:33.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.229+0000 7fa0e97fa700 1 -- 172.21.15.99:0/3220478490 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=76}) v3 -- 0x7fa0d806ebb0 con 0x7fa0f415d240 2023-10-08T00:14:33.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.230+0000 7fa0f975f700 1 --2- 172.21.15.99:0/3220478490 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa0d8066f30 0x7fa0d80693e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa0f00098d0 tx=0x7fa0f0016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:33.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.233+0000 7fa0e97fa700 1 -- 172.21.15.99:0/3220478490 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa0ec051bd0 con 0x7fa0f415d240 2023-10-08T00:14:33.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:33.416+0000 7fa0fa761700 1 -- 172.21.15.99:0/3220478490 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa0f406ac60 con 0x7fa0f415d240 2023-10-08T00:14:34.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.266+0000 7fa0e97fa700 1 -- 172.21.15.99:0/3220478490 <== mon.0 v2:172.21.15.99:3300/0 7 ==== osd_map(76..76 src has 1..76) v4 ==== 1057+0+0 (secure 0 0 0) 0x7fa0ec055f90 con 0x7fa0f415d240 2023-10-08T00:14:34.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.266+0000 7fa0e97fa700 1 -- 172.21.15.99:0/3220478490 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=77}) v3 -- 0x7fa0d806cfe0 con 0x7fa0f415d240 2023-10-08T00:14:34.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.269+0000 7fa0e97fa700 1 -- 172.21.15.99:0/3220478490 <== mon.0 v2:172.21.15.99:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v76) v1 ==== 181+0+0 (secure 0 0 0) 0x7fa0ec0551f0 con 0x7fa0f415d240 2023-10-08T00:14:34.275 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.274+0000 7fa0fa761700 1 -- 172.21.15.99:0/3220478490 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa0d8066f30 msgr2=0x7fa0d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:34.275 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.275+0000 7fa0fa761700 1 --2- 172.21.15.99:0/3220478490 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa0d8066f30 0x7fa0d80693e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa0f00098d0 tx=0x7fa0f0016000 comp rx=0 tx=0).stop 2023-10-08T00:14:34.275 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.275+0000 7fa0fa761700 1 -- 172.21.15.99:0/3220478490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0f415d240 msgr2=0x7fa0f40c9e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:34.275 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.275+0000 7fa0fa761700 1 --2- 172.21.15.99:0/3220478490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0f415d240 0x7fa0f40c9e40 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec00cb90 tx=0x7fa0ec0019c0 comp rx=0 tx=0).stop 2023-10-08T00:14:34.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.276+0000 7fa0fa761700 1 -- 172.21.15.99:0/3220478490 shutdown_connections 2023-10-08T00:14:34.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.276+0000 7fa0fa761700 1 --2- 172.21.15.99:0/3220478490 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa0d8066f30 0x7fa0d80693e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:34.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.276+0000 7fa0fa761700 1 --2- 172.21.15.99:0/3220478490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0f415d240 0x7fa0f40c9e40 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:34.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.276+0000 7fa0fa761700 1 --2- 172.21.15.99:0/3220478490 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0f40c59e0 0x7fa0f409d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:34.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.276+0000 7fa0fa761700 1 --2- 172.21.15.99:0/3220478490 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0f40cc940 0x7fa0f409d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:34.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.276+0000 7fa0fa761700 1 -- 172.21.15.99:0/3220478490 >> 172.21.15.99:0/3220478490 conn(0x7fa0f4067e40 msgr2=0x7fa0f415be40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:34.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.276+0000 7fa0fa761700 1 -- 172.21.15.99:0/3220478490 shutdown_connections 2023-10-08T00:14:34.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.276+0000 7fa0fa761700 1 -- 172.21.15.99:0/3220478490 wait complete. 2023-10-08T00:14:34.277 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' removed 2023-10-08T00:14:34.291 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T00:14:34.414 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.413+0000 7f891838f700 1 Processor -- start 2023-10-08T00:14:34.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.413+0000 7f891838f700 1 -- start start 2023-10-08T00:14:34.424 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.414+0000 7f891838f700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f891015a210 0x7f891006ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:34.424 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.414+0000 7f891838f700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8910068880 con 0x7f8910070e10 2023-10-08T00:14:34.424 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.414+0000 7f891838f700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89100cd650 con 0x7f8910159e40 2023-10-08T00:14:34.425 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.414+0000 7f891838f700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89100cd060 con 0x7f891015a210 2023-10-08T00:14:34.425 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.415+0000 7f891692c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f891015a210 0x7f891006ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:34.425 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.415+0000 7f891592a700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f8910070e10 0x7f891015dba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:41688/0 (socket says 172.21.15.99:41688) 2023-10-08T00:14:34.425 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.415+0000 7f891592a700 1 -- 172.21.15.99:0/190880246 learned_addr learned my addr 172.21.15.99:0/190880246 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:14:34.425 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.415+0000 7f891692c700 1 -- 172.21.15.99:0/190880246 >> v1:172.21.15.191:6789/0 conn(0x7f8910159e40 legacy=0x7f8910070700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:34.425 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.415+0000 7f891692c700 1 -- 172.21.15.99:0/190880246 >> v1:172.21.15.99:6789/0 conn(0x7f8910070e10 legacy=0x7f891015dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:34.426 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.415+0000 7f891692c700 1 -- 172.21.15.99:0/190880246 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8910068ad0 con 0x7f891015a210 2023-10-08T00:14:34.426 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.415+0000 7f8915129700 1 -- 172.21.15.99:0/190880246 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3635445862 0 0) 0x7f8910068880 con 0x7f8910070e10 2023-10-08T00:14:34.426 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.415+0000 7f891692c700 1 --2- 172.21.15.99:0/190880246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f891015a210 0x7f891006ee30 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f89080098e0 tx=0x7f89080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eee18d507886d1f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:34.426 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.415+0000 7f8915129700 1 -- 172.21.15.99:0/190880246 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8908009e80 con 0x7f891015a210 2023-10-08T00:14:34.426 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.415+0000 7f8915129700 1 -- 172.21.15.99:0/190880246 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f890800b860 con 0x7f891015a210 2023-10-08T00:14:34.427 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.415+0000 7f8915129700 1 -- 172.21.15.99:0/190880246 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890800bbe0 con 0x7f891015a210 2023-10-08T00:14:34.427 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.416+0000 7f891838f700 1 -- 172.21.15.99:0/190880246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f891015a210 msgr2=0x7f891006ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:34.427 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.416+0000 7f891838f700 1 --2- 172.21.15.99:0/190880246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f891015a210 0x7f891006ee30 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f89080098e0 tx=0x7f89080091a0 comp rx=0 tx=0).stop 2023-10-08T00:14:34.427 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.416+0000 7f891838f700 1 -- 172.21.15.99:0/190880246 shutdown_connections 2023-10-08T00:14:34.428 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.416+0000 7f891838f700 1 --2- 172.21.15.99:0/190880246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f891015a210 0x7f891006ee30 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:34.428 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.416+0000 7f891838f700 1 -- 172.21.15.99:0/190880246 >> 172.21.15.99:0/190880246 conn(0x7f89100ce5c0 msgr2=0x7f89100ca5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:34.428 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.416+0000 7f891838f700 1 -- 172.21.15.99:0/190880246 shutdown_connections 2023-10-08T00:14:34.428 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.416+0000 7f891838f700 1 -- 172.21.15.99:0/190880246 wait complete. 2023-10-08T00:14:34.428 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.417+0000 7f891838f700 1 Processor -- start 2023-10-08T00:14:34.428 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.417+0000 7f891838f700 1 -- start start 2023-10-08T00:14:34.429 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.418+0000 7f891838f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8910070e10 0x7f89100b2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:34.429 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.418+0000 7f891838f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8910159e40 0x7f89100b0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:34.429 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.418+0000 7f891838f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f891015a210 0x7f891014d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:34.429 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.418+0000 7f891838f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89100b2020 con 0x7f891015a210 2023-10-08T00:14:34.429 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.418+0000 7f891838f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89100b1410 con 0x7f8910159e40 2023-10-08T00:14:34.429 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.418+0000 7f891838f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89100b16f0 con 0x7f8910070e10 2023-10-08T00:14:34.430 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.418+0000 7f891692c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f891015a210 0x7f891014d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:34.430 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.418+0000 7f891692c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f891015a210 0x7f891014d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33580/0 (socket says 172.21.15.99:33580) 2023-10-08T00:14:34.430 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.418+0000 7f891692c700 1 -- 172.21.15.99:0/2150641724 learned_addr learned my addr 172.21.15.99:0/2150641724 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:34.430 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.418+0000 7f891592a700 1 --2- 172.21.15.99:0/2150641724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8910159e40 0x7f89100b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:34.430 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.418+0000 7f891612b700 1 --2- 172.21.15.99:0/2150641724 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8910070e10 0x7f89100b2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:34.431 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.418+0000 7f891692c700 1 -- 172.21.15.99:0/2150641724 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8910070e10 msgr2=0x7f89100b2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:34.431 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.418+0000 7f891692c700 1 --2- 172.21.15.99:0/2150641724 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8910070e10 0x7f89100b2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:34.431 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.418+0000 7f891692c700 1 -- 172.21.15.99:0/2150641724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8910159e40 msgr2=0x7f89100b0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:34.431 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.418+0000 7f891692c700 1 --2- 172.21.15.99:0/2150641724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8910159e40 0x7f89100b0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:34.431 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.418+0000 7f891692c700 1 -- 172.21.15.99:0/2150641724 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8908009420 con 0x7f891015a210 2023-10-08T00:14:34.431 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.418+0000 7f891612b700 1 --2- 172.21.15.99:0/2150641724 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8910070e10 0x7f89100b2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:14:34.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.418+0000 7f891692c700 1 --2- 172.21.15.99:0/2150641724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f891015a210 0x7f891014d0d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f891006dc10 tx=0x7f8908017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:34.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.418+0000 7f89067fc700 1 -- 172.21.15.99:0/2150641724 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890800bb50 con 0x7f891015a210 2023-10-08T00:14:34.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.418+0000 7f89067fc700 1 -- 172.21.15.99:0/2150641724 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8908011630 con 0x7f891015a210 2023-10-08T00:14:34.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.418+0000 7f89067fc700 1 -- 172.21.15.99:0/2150641724 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89080119b0 con 0x7f891015a210 2023-10-08T00:14:34.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.418+0000 7f891838f700 1 -- 172.21.15.99:0/2150641724 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89100b19d0 con 0x7f891015a210 2023-10-08T00:14:34.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.418+0000 7f891838f700 1 -- 172.21.15.99:0/2150641724 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8910158c00 con 0x7f891015a210 2023-10-08T00:14:34.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.419+0000 7f891838f700 1 -- 172.21.15.99:0/2150641724 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f891004c3c0 con 0x7f891015a210 2023-10-08T00:14:34.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.420+0000 7f89067fc700 1 -- 172.21.15.99:0/2150641724 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8908011d20 con 0x7f891015a210 2023-10-08T00:14:34.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.420+0000 7f89067fc700 1 --2- 172.21.15.99:0/2150641724 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f88f4066f30 0x7f88f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:34.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.420+0000 7f89067fc700 1 -- 172.21.15.99:0/2150641724 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5411+0+0 (secure 0 0 0) 0x7f8908085f70 con 0x7f891015a210 2023-10-08T00:14:34.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.420+0000 7f891612b700 1 --2- 172.21.15.99:0/2150641724 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f88f4066f30 0x7f88f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:34.434 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.421+0000 7f891612b700 1 --2- 172.21.15.99:0/2150641724 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f88f4066f30 0x7f88f40693e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f890c007fd0 tx=0x7f890c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:34.434 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.424+0000 7f89067fc700 1 -- 172.21.15.99:0/2150641724 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f89080b75a0 con 0x7f891015a210 2023-10-08T00:14:34.606 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:34.605+0000 7f891838f700 1 -- 172.21.15.99:0/2150641724 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f89100b2160 con 0x7f891015a210 2023-10-08T00:14:35.275 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.274+0000 7f89067fc700 1 -- 172.21.15.99:0/2150641724 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v77) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8908056d90 con 0x7f891015a210 2023-10-08T00:14:35.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.279+0000 7f891838f700 1 -- 172.21.15.99:0/2150641724 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f88f4066f30 msgr2=0x7f88f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:35.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.279+0000 7f891838f700 1 --2- 172.21.15.99:0/2150641724 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f88f4066f30 0x7f88f40693e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f890c007fd0 tx=0x7f890c009040 comp rx=0 tx=0).stop 2023-10-08T00:14:35.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.280+0000 7f891838f700 1 -- 172.21.15.99:0/2150641724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f891015a210 msgr2=0x7f891014d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:35.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.280+0000 7f891838f700 1 --2- 172.21.15.99:0/2150641724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f891015a210 0x7f891014d0d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f891006dc10 tx=0x7f8908017000 comp rx=0 tx=0).stop 2023-10-08T00:14:35.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.280+0000 7f891838f700 1 -- 172.21.15.99:0/2150641724 shutdown_connections 2023-10-08T00:14:35.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.280+0000 7f891838f700 1 --2- 172.21.15.99:0/2150641724 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f88f4066f30 0x7f88f40693e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:35.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.280+0000 7f891838f700 1 --2- 172.21.15.99:0/2150641724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f891015a210 0x7f891014d0d0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:35.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.280+0000 7f891838f700 1 --2- 172.21.15.99:0/2150641724 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8910070e10 0x7f89100b2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:35.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.280+0000 7f891838f700 1 --2- 172.21.15.99:0/2150641724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8910159e40 0x7f89100b0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:35.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.280+0000 7f891838f700 1 -- 172.21.15.99:0/2150641724 >> 172.21.15.99:0/2150641724 conn(0x7f89100ce5c0 msgr2=0x7f89100c81e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:35.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.280+0000 7f891838f700 1 -- 172.21.15.99:0/2150641724 shutdown_connections 2023-10-08T00:14:35.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.280+0000 7f891838f700 1 -- 172.21.15.99:0/2150641724 wait complete. 2023-10-08T00:14:35.284 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' removed 2023-10-08T00:14:35.294 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-08T00:14:35.414 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.413+0000 7f0e9d5d4700 1 Processor -- start 2023-10-08T00:14:35.414 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.413+0000 7f0e9d5d4700 1 -- start start 2023-10-08T00:14:35.416 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.414+0000 7f0e9d5d4700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0e980cd300 0x7f0e9815afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:35.416 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.414+0000 7f0e9d5d4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e980d1470 0x7f0e980cf310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:35.416 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.414+0000 7f0e9d5d4700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e981513f0 con 0x7f0e980b6760 2023-10-08T00:14:35.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.414+0000 7f0e9d5d4700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e9815e7a0 con 0x7f0e980cd300 2023-10-08T00:14:35.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.414+0000 7f0e9d5d4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9815db20 con 0x7f0e980d1470 2023-10-08T00:14:35.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.414+0000 7f0e96ffd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0e980cd300 0x7f0e9815afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:35.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.414+0000 7f0e96ffd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0e980cd300 0x7f0e9815afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33592/0 (socket says 172.21.15.99:33592) 2023-10-08T00:14:35.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.414+0000 7f0e96ffd700 1 -- 172.21.15.99:0/1168092062 learned_addr learned my addr 172.21.15.99:0/1168092062 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:35.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.415+0000 7f0e977fe700 1 --2- 172.21.15.99:0/1168092062 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e980d1470 0x7f0e980cf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:35.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.415+0000 7f0e96ffd700 1 -- 172.21.15.99:0/1168092062 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e980d1470 msgr2=0x7f0e980cf310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:35.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.415+0000 7f0e96ffd700 1 --2- 172.21.15.99:0/1168092062 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e980d1470 0x7f0e980cf310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:35.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.415+0000 7f0e96ffd700 1 -- 172.21.15.99:0/1168092062 >> v1:172.21.15.99:6789/0 conn(0x7f0e980b6760 legacy=0x7f0e980d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:35.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.415+0000 7f0e967fc700 1 -- 172.21.15.99:0/1168092062 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 577494262 0 0) 0x7f0e981513f0 con 0x7f0e980b6760 2023-10-08T00:14:35.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.415+0000 7f0e96ffd700 1 -- 172.21.15.99:0/1168092062 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e9815de00 con 0x7f0e980cd300 2023-10-08T00:14:35.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.415+0000 7f0e96ffd700 1 --2- 172.21.15.99:0/1168092062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e980cd300 0x7f0e9815afb0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f0e880099b0 tx=0x7f0e88009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62403a63447336a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:35.418 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.415+0000 7f0e967fc700 1 -- 172.21.15.99:0/1168092062 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8800b860 con 0x7f0e980cd300 2023-10-08T00:14:35.418 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.415+0000 7f0e967fc700 1 -- 172.21.15.99:0/1168092062 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e8800b9c0 con 0x7f0e980cd300 2023-10-08T00:14:35.418 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.415+0000 7f0e967fc700 1 -- 172.21.15.99:0/1168092062 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8800bd60 con 0x7f0e980cd300 2023-10-08T00:14:35.418 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.415+0000 7f0e9d5d4700 1 -- 172.21.15.99:0/1168092062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e980cd300 msgr2=0x7f0e9815afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:35.418 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.415+0000 7f0e9d5d4700 1 --2- 172.21.15.99:0/1168092062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e980cd300 0x7f0e9815afb0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f0e880099b0 tx=0x7f0e88009250 comp rx=0 tx=0).stop 2023-10-08T00:14:35.418 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.415+0000 7f0e9d5d4700 1 -- 172.21.15.99:0/1168092062 shutdown_connections 2023-10-08T00:14:35.418 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.415+0000 7f0e9d5d4700 1 --2- 172.21.15.99:0/1168092062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e980cd300 0x7f0e9815afb0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:35.418 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.415+0000 7f0e9d5d4700 1 --2- 172.21.15.99:0/1168092062 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e980d1470 0x7f0e980cf310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:35.418 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.415+0000 7f0e9d5d4700 1 -- 172.21.15.99:0/1168092062 >> 172.21.15.99:0/1168092062 conn(0x7f0e98068d90 msgr2=0x7f0e98069190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:35.418 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.415+0000 7f0e9d5d4700 1 -- 172.21.15.99:0/1168092062 shutdown_connections 2023-10-08T00:14:35.418 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.415+0000 7f0e9d5d4700 1 -- 172.21.15.99:0/1168092062 wait complete. 2023-10-08T00:14:35.421 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.417+0000 7f0e9d5d4700 1 Processor -- start 2023-10-08T00:14:35.421 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.417+0000 7f0e9d5d4700 1 -- start start 2023-10-08T00:14:35.421 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.417+0000 7f0e9d5d4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e980b6760 0x7f0e980c5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:35.421 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.417+0000 7f0e9d5d4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e980cd300 0x7f0e980b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:35.421 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.417+0000 7f0e9d5d4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e980d1470 0x7f0e98150390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:35.421 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.417+0000 7f0e9d5d4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9814ff00 con 0x7f0e980d1470 2023-10-08T00:14:35.421 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.417+0000 7f0e9d5d4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9814f5d0 con 0x7f0e980b6760 2023-10-08T00:14:35.422 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.417+0000 7f0e9d5d4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9814f8b0 con 0x7f0e980cd300 2023-10-08T00:14:35.422 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.417+0000 7f0e977fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e980d1470 0x7f0e98150390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:35.422 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.418+0000 7f0e977fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e980d1470 0x7f0e98150390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33602/0 (socket says 172.21.15.99:33602) 2023-10-08T00:14:35.422 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.418+0000 7f0e977fe700 1 -- 172.21.15.99:0/2277455434 learned_addr learned my addr 172.21.15.99:0/2277455434 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:35.422 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.418+0000 7f0e96ffd700 1 --2- 172.21.15.99:0/2277455434 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e980b6760 0x7f0e980c5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:35.422 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.418+0000 7f0e8e7fc700 1 --2- 172.21.15.99:0/2277455434 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e980cd300 0x7f0e980b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:35.422 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.418+0000 7f0e977fe700 1 -- 172.21.15.99:0/2277455434 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e980cd300 msgr2=0x7f0e980b32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:35.422 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.418+0000 7f0e977fe700 1 --2- 172.21.15.99:0/2277455434 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e980cd300 0x7f0e980b32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:35.422 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.418+0000 7f0e977fe700 1 -- 172.21.15.99:0/2277455434 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e980b6760 msgr2=0x7f0e980c5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:35.423 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.418+0000 7f0e977fe700 1 --2- 172.21.15.99:0/2277455434 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e980b6760 0x7f0e980c5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:35.423 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.418+0000 7f0e977fe700 1 -- 172.21.15.99:0/2277455434 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e880094f0 con 0x7f0e980d1470 2023-10-08T00:14:35.423 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.418+0000 7f0e96ffd700 1 --2- 172.21.15.99:0/2277455434 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e980b6760 0x7f0e980c5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:14:35.423 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.418+0000 7f0e977fe700 1 --2- 172.21.15.99:0/2277455434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e980d1470 0x7f0e98150390 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f0e90007ef0 tx=0x7f0e90009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:35.423 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.418+0000 7f0e8f7fe700 1 -- 172.21.15.99:0/2277455434 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e9000ec10 con 0x7f0e980d1470 2023-10-08T00:14:35.423 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.418+0000 7f0e8f7fe700 1 -- 172.21.15.99:0/2277455434 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e9000ed70 con 0x7f0e980d1470 2023-10-08T00:14:35.423 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.418+0000 7f0e9d5d4700 1 -- 172.21.15.99:0/2277455434 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e9814fb90 con 0x7f0e980d1470 2023-10-08T00:14:35.423 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.418+0000 7f0e8f7fe700 1 -- 172.21.15.99:0/2277455434 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e90014670 con 0x7f0e980d1470 2023-10-08T00:14:35.423 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.418+0000 7f0e9d5d4700 1 -- 172.21.15.99:0/2277455434 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e9814d4e0 con 0x7f0e980d1470 2023-10-08T00:14:35.423 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.419+0000 7f0e9d5d4700 1 -- 172.21.15.99:0/2277455434 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e9804c3c0 con 0x7f0e980d1470 2023-10-08T00:14:35.424 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.419+0000 7f0e8f7fe700 1 -- 172.21.15.99:0/2277455434 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0e90014a60 con 0x7f0e980d1470 2023-10-08T00:14:35.424 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.420+0000 7f0e8f7fe700 1 --2- 172.21.15.99:0/2277455434 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0e78066f30 0x7f0e780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:35.424 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.420+0000 7f0e8f7fe700 1 -- 172.21.15.99:0/2277455434 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f0e90084940 con 0x7f0e980d1470 2023-10-08T00:14:35.424 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.420+0000 7f0e96ffd700 1 --2- 172.21.15.99:0/2277455434 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0e78066f30 0x7f0e780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:35.424 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.420+0000 7f0e96ffd700 1 --2- 172.21.15.99:0/2277455434 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0e78066f30 0x7f0e780693e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0e880099b0 tx=0x7f0e88017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:35.424 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.424+0000 7f0e8f7fe700 1 -- 172.21.15.99:0/2277455434 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0e90051eb0 con 0x7f0e980d1470 2023-10-08T00:14:35.606 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.606+0000 7f0e9d5d4700 1 -- 172.21.15.99:0/2277455434 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f0e980b4010 con 0x7f0e980d1470 2023-10-08T00:14:35.610 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.610+0000 7f0e8f7fe700 1 -- 172.21.15.99:0/2277455434 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v7) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0e90009980 con 0x7f0e980d1470 2023-10-08T00:14:35.613 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.612+0000 7f0e9d5d4700 1 -- 172.21.15.99:0/2277455434 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0e78066f30 msgr2=0x7f0e780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:35.617 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.612+0000 7f0e9d5d4700 1 --2- 172.21.15.99:0/2277455434 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0e78066f30 0x7f0e780693e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0e880099b0 tx=0x7f0e88017000 comp rx=0 tx=0).stop 2023-10-08T00:14:35.617 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.612+0000 7f0e9d5d4700 1 -- 172.21.15.99:0/2277455434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e980d1470 msgr2=0x7f0e98150390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:35.617 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.612+0000 7f0e9d5d4700 1 --2- 172.21.15.99:0/2277455434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e980d1470 0x7f0e98150390 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f0e90007ef0 tx=0x7f0e90009840 comp rx=0 tx=0).stop 2023-10-08T00:14:35.617 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.612+0000 7f0e9d5d4700 1 -- 172.21.15.99:0/2277455434 shutdown_connections 2023-10-08T00:14:35.617 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.612+0000 7f0e9d5d4700 1 --2- 172.21.15.99:0/2277455434 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0e78066f30 0x7f0e780693e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:35.617 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.612+0000 7f0e9d5d4700 1 --2- 172.21.15.99:0/2277455434 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e980b6760 0x7f0e980c5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:35.617 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.612+0000 7f0e9d5d4700 1 --2- 172.21.15.99:0/2277455434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e980d1470 0x7f0e98150390 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:35.618 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.612+0000 7f0e9d5d4700 1 --2- 172.21.15.99:0/2277455434 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e980cd300 0x7f0e980b32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:35.618 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.612+0000 7f0e9d5d4700 1 -- 172.21.15.99:0/2277455434 >> 172.21.15.99:0/2277455434 conn(0x7f0e98068d90 msgr2=0x7f0e980b5350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:35.618 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.612+0000 7f0e9d5d4700 1 -- 172.21.15.99:0/2277455434 shutdown_connections 2023-10-08T00:14:35.618 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.612+0000 7f0e9d5d4700 1 -- 172.21.15.99:0/2277455434 wait complete. 2023-10-08T00:14:35.624 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fclose' 2023-10-08T00:14:35.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.737+0000 7f98594f4700 1 Processor -- start 2023-10-08T00:14:35.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.738+0000 7f98594f4700 1 -- start start 2023-10-08T00:14:35.741 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.738+0000 7f98594f4700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f98540b0890 0x7f98540cb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:35.741 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.738+0000 7f98594f4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98540cbd40 0x7f9854067660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:35.741 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.738+0000 7f98594f4700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98540cb370 con 0x7f985406c630 2023-10-08T00:14:35.741 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.738+0000 7f98594f4700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98540b0c60 con 0x7f98540b0890 2023-10-08T00:14:35.741 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.738+0000 7f98594f4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f985406a1e0 con 0x7f98540cbd40 2023-10-08T00:14:35.741 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.739+0000 7f9852ffd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f98540b0890 0x7f98540cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:35.741 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.739+0000 7f9852ffd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f98540b0890 0x7f98540cb800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33606/0 (socket says 172.21.15.99:33606) 2023-10-08T00:14:35.741 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.739+0000 7f9852ffd700 1 -- 172.21.15.99:0/79716969 learned_addr learned my addr 172.21.15.99:0/79716969 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:35.741 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.739+0000 7f98527fc700 1 --2- 172.21.15.99:0/79716969 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98540cbd40 0x7f9854067660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:35.741 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.739+0000 7f9852ffd700 1 -- 172.21.15.99:0/79716969 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98540cbd40 msgr2=0x7f9854067660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:35.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.739+0000 7f9852ffd700 1 --2- 172.21.15.99:0/79716969 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98540cbd40 0x7f9854067660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:35.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.739+0000 7f9852ffd700 1 -- 172.21.15.99:0/79716969 >> v1:172.21.15.191:6789/0 conn(0x7f985406c630 legacy=0x7f985415dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:35.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.739+0000 7f9852ffd700 1 -- 172.21.15.99:0/79716969 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f985406cb80 con 0x7f98540b0890 2023-10-08T00:14:35.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.739+0000 7f9852ffd700 1 --2- 172.21.15.99:0/79716969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98540b0890 0x7f98540cb800 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f98440099b0 tx=0x7f9844009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=348a7e860ec7dded server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:35.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.739+0000 7f9851ffb700 1 -- 172.21.15.99:0/79716969 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984400b820 con 0x7f98540b0890 2023-10-08T00:14:35.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.739+0000 7f9851ffb700 1 -- 172.21.15.99:0/79716969 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f984400b980 con 0x7f98540b0890 2023-10-08T00:14:35.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.739+0000 7f9851ffb700 1 -- 172.21.15.99:0/79716969 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984400bd20 con 0x7f98540b0890 2023-10-08T00:14:35.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.740+0000 7f98594f4700 1 -- 172.21.15.99:0/79716969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98540b0890 msgr2=0x7f98540cb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:35.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.740+0000 7f98594f4700 1 --2- 172.21.15.99:0/79716969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98540b0890 0x7f98540cb800 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f98440099b0 tx=0x7f9844009250 comp rx=0 tx=0).stop 2023-10-08T00:14:35.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.740+0000 7f98594f4700 1 -- 172.21.15.99:0/79716969 shutdown_connections 2023-10-08T00:14:35.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.740+0000 7f98594f4700 1 --2- 172.21.15.99:0/79716969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98540b0890 0x7f98540cb800 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:35.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.740+0000 7f98594f4700 1 --2- 172.21.15.99:0/79716969 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98540cbd40 0x7f9854067660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:35.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.740+0000 7f98594f4700 1 -- 172.21.15.99:0/79716969 >> 172.21.15.99:0/79716969 conn(0x7f985406fe50 msgr2=0x7f9854069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:35.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.740+0000 7f98594f4700 1 -- 172.21.15.99:0/79716969 shutdown_connections 2023-10-08T00:14:35.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.740+0000 7f98594f4700 1 -- 172.21.15.99:0/79716969 wait complete. 2023-10-08T00:14:35.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.741+0000 7f98594f4700 1 Processor -- start 2023-10-08T00:14:35.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.741+0000 7f98594f4700 1 -- start start 2023-10-08T00:14:35.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.742+0000 7f98594f4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f985406c630 0x7f98541590d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:35.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.742+0000 7f98594f4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98540b0890 0x7f985415efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:35.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.742+0000 7f98594f4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98540cbd40 0x7f985415f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:35.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.742+0000 7f98594f4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98540b3a20 con 0x7f98540b0890 2023-10-08T00:14:35.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.742+0000 7f98594f4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985415a650 con 0x7f98540cbd40 2023-10-08T00:14:35.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.742+0000 7f98594f4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9854159d20 con 0x7f985406c630 2023-10-08T00:14:35.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.742+0000 7f98527fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98540b0890 0x7f985415efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:35.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.742+0000 7f98527fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98540b0890 0x7f985415efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33620/0 (socket says 172.21.15.99:33620) 2023-10-08T00:14:35.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.742+0000 7f98527fc700 1 -- 172.21.15.99:0/622880119 learned_addr learned my addr 172.21.15.99:0/622880119 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:35.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.742+0000 7f98537fe700 1 --2- 172.21.15.99:0/622880119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98540cbd40 0x7f985415f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:35.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.742+0000 7f9852ffd700 1 --2- 172.21.15.99:0/622880119 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f985406c630 0x7f98541590d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:35.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.742+0000 7f98527fc700 1 -- 172.21.15.99:0/622880119 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f985406c630 msgr2=0x7f98541590d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:35.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.742+0000 7f98527fc700 1 --2- 172.21.15.99:0/622880119 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f985406c630 0x7f98541590d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:35.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.742+0000 7f98527fc700 1 -- 172.21.15.99:0/622880119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98540cbd40 msgr2=0x7f985415f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:35.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.742+0000 7f98527fc700 1 --2- 172.21.15.99:0/622880119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98540cbd40 0x7f985415f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:35.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.742+0000 7f98527fc700 1 -- 172.21.15.99:0/622880119 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98440094f0 con 0x7f98540b0890 2023-10-08T00:14:35.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.742+0000 7f9852ffd700 1 --2- 172.21.15.99:0/622880119 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f985406c630 0x7f98541590d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:14:35.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.742+0000 7f98527fc700 1 --2- 172.21.15.99:0/622880119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98540b0890 0x7f985415efd0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f984800c880 tx=0x7f9848008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:35.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.742+0000 7f983affd700 1 -- 172.21.15.99:0/622880119 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9848011600 con 0x7f98540b0890 2023-10-08T00:14:35.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.742+0000 7f983affd700 1 -- 172.21.15.99:0/622880119 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9848011760 con 0x7f98540b0890 2023-10-08T00:14:35.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.742+0000 7f98594f4700 1 -- 172.21.15.99:0/622880119 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f985415a000 con 0x7f98540b0890 2023-10-08T00:14:35.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.742+0000 7f983affd700 1 -- 172.21.15.99:0/622880119 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9848011b00 con 0x7f98540b0890 2023-10-08T00:14:35.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.742+0000 7f98594f4700 1 -- 172.21.15.99:0/622880119 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9854159860 con 0x7f98540b0890 2023-10-08T00:14:35.748 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.743+0000 7f98594f4700 1 -- 172.21.15.99:0/622880119 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f985404c3c0 con 0x7f98540b0890 2023-10-08T00:14:35.748 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.744+0000 7f983affd700 1 -- 172.21.15.99:0/622880119 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f98480223a0 con 0x7f98540b0890 2023-10-08T00:14:35.748 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.744+0000 7f983affd700 1 --2- 172.21.15.99:0/622880119 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9840066f30 0x7f98400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:35.748 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.744+0000 7f983affd700 1 -- 172.21.15.99:0/622880119 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f9848084220 con 0x7f98540b0890 2023-10-08T00:14:35.748 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.744+0000 7f9852ffd700 1 --2- 172.21.15.99:0/622880119 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9840066f30 0x7f98400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:35.748 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.745+0000 7f9852ffd700 1 --2- 172.21.15.99:0/622880119 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9840066f30 0x7f98400693e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f98440099b0 tx=0x7f9844016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:35.749 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:35.748+0000 7f983affd700 1 -- 172.21.15.99:0/622880119 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9848051790 con 0x7f98540b0890 2023-10-08T00:14:36.004 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.004+0000 7f98594f4700 1 -- 172.21.15.99:0/622880119 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]} v 0) v1 -- 0x7f985406c410 con 0x7f98540b0890 2023-10-08T00:14:36.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.269+0000 7f983affd700 1 -- 172.21.15.99:0/622880119 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]}]=0 v197) v1 ==== 134+0+0 (secure 0 0 0) 0x7f9848082e60 con 0x7f98540b0890 2023-10-08T00:14:36.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.272+0000 7f98594f4700 1 -- 172.21.15.99:0/622880119 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9840066f30 msgr2=0x7f98400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:36.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.272+0000 7f98594f4700 1 --2- 172.21.15.99:0/622880119 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9840066f30 0x7f98400693e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f98440099b0 tx=0x7f9844016000 comp rx=0 tx=0).stop 2023-10-08T00:14:36.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.272+0000 7f98594f4700 1 -- 172.21.15.99:0/622880119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98540b0890 msgr2=0x7f985415efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:36.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.272+0000 7f98594f4700 1 --2- 172.21.15.99:0/622880119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98540b0890 0x7f985415efd0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f984800c880 tx=0x7f9848008040 comp rx=0 tx=0).stop 2023-10-08T00:14:36.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.272+0000 7f98594f4700 1 -- 172.21.15.99:0/622880119 shutdown_connections 2023-10-08T00:14:36.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.272+0000 7f98594f4700 1 --2- 172.21.15.99:0/622880119 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9840066f30 0x7f98400693e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:36.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.272+0000 7f98594f4700 1 --2- 172.21.15.99:0/622880119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98540cbd40 0x7f985415f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:36.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.272+0000 7f98594f4700 1 --2- 172.21.15.99:0/622880119 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f985406c630 0x7f98541590d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:36.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.272+0000 7f98594f4700 1 --2- 172.21.15.99:0/622880119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98540b0890 0x7f985415efd0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:36.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.272+0000 7f98594f4700 1 -- 172.21.15.99:0/622880119 >> 172.21.15.99:0/622880119 conn(0x7f985406fe50 msgr2=0x7f985415c310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:36.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.272+0000 7f98594f4700 1 -- 172.21.15.99:0/622880119 shutdown_connections 2023-10-08T00:14:36.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.272+0000 7f98594f4700 1 -- 172.21.15.99:0/622880119 wait complete. 2023-10-08T00:14:36.285 INFO:tasks.cephfs_test_runner:test_full_fclose (tasks.cephfs.test_full.TestClusterFull) ... ok 2023-10-08T00:14:36.285 INFO:tasks.cephfs_test_runner:Starting test: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) That when the full flag is encountered during asynchronous 2023-10-08T00:14:36.285 INFO:tasks.cephfs_test_runner:test_full_fsync (tasks.cephfs.test_full.TestClusterFull) 2023-10-08T00:14:36.285 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fsync' 2023-10-08T00:14:36.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.400+0000 7f1119a2f700 1 Processor -- start 2023-10-08T00:14:36.412 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.400+0000 7f1119a2f700 1 -- start start 2023-10-08T00:14:36.412 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.401+0000 7f1119a2f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f11140cd6e0 0x7f11140cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:36.412 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.401+0000 7f1119a2f700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11140cb7c0 con 0x7f111415cbd0 2023-10-08T00:14:36.412 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.401+0000 7f1119a2f700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f111415e290 con 0x7f11140cdab0 2023-10-08T00:14:36.412 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.401+0000 7f1119a2f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f111415e850 con 0x7f11140cd6e0 2023-10-08T00:14:36.413 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.401+0000 7f111922e700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f111415cbd0 0x7f111415d020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:41696/0 (socket says 172.21.15.99:41696) 2023-10-08T00:14:36.413 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.401+0000 7f111922e700 1 -- 172.21.15.99:0/355116111 learned_addr learned my addr 172.21.15.99:0/355116111 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:14:36.413 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.401+0000 7f1118a2d700 1 --2- 172.21.15.99:0/355116111 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f11140cd6e0 0x7f11140cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:36.413 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.401+0000 7f110b7fe700 1 -- 172.21.15.99:0/355116111 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 744755425 0 0) 0x7f11140cb7c0 con 0x7f111415cbd0 2023-10-08T00:14:36.413 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.402+0000 7f110b7fe700 1 -- 172.21.15.99:0/355116111 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10fc003570 con 0x7f111415cbd0 2023-10-08T00:14:36.413 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.402+0000 7f110b7fe700 1 -- 172.21.15.99:0/355116111 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3346296043 0 0) 0x7f111415e290 con 0x7f11140cdab0 2023-10-08T00:14:36.414 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.402+0000 7f110b7fe700 1 -- 172.21.15.99:0/355116111 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11140cb7c0 con 0x7f11140cdab0 2023-10-08T00:14:36.414 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.402+0000 7f110b7fe700 1 -- 172.21.15.99:0/355116111 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3373762843 0 0) 0x7f10fc003570 con 0x7f111415cbd0 2023-10-08T00:14:36.414 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.402+0000 7f110b7fe700 1 -- 172.21.15.99:0/355116111 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f11140cd6e0 msgr2=0x7f11140cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:36.414 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.402+0000 7f110b7fe700 1 --2- 172.21.15.99:0/355116111 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f11140cd6e0 0x7f11140cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:36.414 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.402+0000 7f110b7fe700 1 -- 172.21.15.99:0/355116111 >> v1:172.21.15.191:6789/0 conn(0x7f11140cdab0 legacy=0x7f111415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:36.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.402+0000 7f110b7fe700 1 -- 172.21.15.99:0/355116111 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f111415e570 con 0x7f111415cbd0 2023-10-08T00:14:36.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.402+0000 7f110b7fe700 1 -- 172.21.15.99:0/355116111 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f110c001470 con 0x7f111415cbd0 2023-10-08T00:14:36.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.402+0000 7f110b7fe700 1 -- 172.21.15.99:0/355116111 >> v1:172.21.15.99:6789/0 conn(0x7f111415cbd0 legacy=0x7f111415d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:36.416 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.402+0000 7f110b7fe700 1 --2- 172.21.15.99:0/355116111 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f11140cd6e0 0x7f10fc003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:36.416 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.402+0000 7f110b7fe700 1 --2- 172.21.15.99:0/355116111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10fc004180 0x7f10fc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:36.416 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.402+0000 7f110b7fe700 1 --2- 172.21.15.99:0/355116111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f10fc004d50 0x7f10fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:36.416 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.402+0000 7f110b7fe700 1 -- 172.21.15.99:0/355116111 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111415e850 con 0x7f10fc004180 2023-10-08T00:14:36.416 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.402+0000 7f110b7fe700 1 -- 172.21.15.99:0/355116111 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10fc00bba0 con 0x7f10fc004d50 2023-10-08T00:14:36.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.402+0000 7f110b7fe700 1 -- 172.21.15.99:0/355116111 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10fc00bd60 con 0x7f11140cd6e0 2023-10-08T00:14:36.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.402+0000 7f110bfff700 1 --2- 172.21.15.99:0/355116111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10fc004180 0x7f10fc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:36.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.402+0000 7f1118a2d700 1 --2- 172.21.15.99:0/355116111 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f11140cd6e0 0x7f10fc003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:36.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.402+0000 7f111922e700 1 --2- 172.21.15.99:0/355116111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f10fc004d50 0x7f10fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:36.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.402+0000 7f110bfff700 1 -- 172.21.15.99:0/355116111 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f11140cd6e0 msgr2=0x7f10fc003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:36.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.402+0000 7f110bfff700 1 --2- 172.21.15.99:0/355116111 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f11140cd6e0 0x7f10fc003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:36.418 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.402+0000 7f110bfff700 1 -- 172.21.15.99:0/355116111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f10fc004d50 msgr2=0x7f10fc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:36.418 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.402+0000 7f110bfff700 1 --2- 172.21.15.99:0/355116111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f10fc004d50 0x7f10fc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:36.418 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.402+0000 7f110bfff700 1 -- 172.21.15.99:0/355116111 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10fc003570 con 0x7f10fc004180 2023-10-08T00:14:36.418 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.402+0000 7f111922e700 1 --2- 172.21.15.99:0/355116111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f10fc004d50 0x7f10fc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:14:36.418 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.402+0000 7f110bfff700 1 --2- 172.21.15.99:0/355116111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10fc004180 0x7f10fc004810 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f1104001a10 tx=0x7f1104007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8273a192e15567e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:36.419 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.402+0000 7f110b7fe700 1 -- 172.21.15.99:0/355116111 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110400c5d0 con 0x7f10fc004180 2023-10-08T00:14:36.419 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.403+0000 7f110b7fe700 1 -- 172.21.15.99:0/355116111 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f110400c730 con 0x7f10fc004180 2023-10-08T00:14:36.419 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.403+0000 7f110b7fe700 1 -- 172.21.15.99:0/355116111 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110400cad0 con 0x7f10fc004180 2023-10-08T00:14:36.419 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.403+0000 7f1119a2f700 1 -- 172.21.15.99:0/355116111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10fc004180 msgr2=0x7f10fc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:36.419 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.403+0000 7f1119a2f700 1 --2- 172.21.15.99:0/355116111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10fc004180 0x7f10fc004810 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f1104001a10 tx=0x7f1104007d30 comp rx=0 tx=0).stop 2023-10-08T00:14:36.420 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.403+0000 7f111922e700 1 -- 172.21.15.99:0/355116111 reap_dead start 2023-10-08T00:14:36.420 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.403+0000 7f1119a2f700 1 -- 172.21.15.99:0/355116111 shutdown_connections 2023-10-08T00:14:36.420 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.403+0000 7f1119a2f700 1 -- 172.21.15.99:0/355116111 >> 172.21.15.99:0/355116111 conn(0x7f1114067660 msgr2=0x7f1114067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:36.420 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.403+0000 7f1119a2f700 1 -- 172.21.15.99:0/355116111 shutdown_connections 2023-10-08T00:14:36.420 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.403+0000 7f1119a2f700 1 -- 172.21.15.99:0/355116111 wait complete. 2023-10-08T00:14:36.421 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.404+0000 7f1119a2f700 1 Processor -- start 2023-10-08T00:14:36.421 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.404+0000 7f1119a2f700 1 -- start start 2023-10-08T00:14:36.421 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.405+0000 7f1119a2f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10fc004180 0x7f1114154c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:36.421 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.405+0000 7f1119a2f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f111416d700 0x7f111416db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:36.421 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.405+0000 7f1119a2f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f11140b3d60 0x7f11140b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:36.421 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.405+0000 7f1119a2f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111416e0b0 con 0x7f10fc004180 2023-10-08T00:14:36.422 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.405+0000 7f1119a2f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11140b4ae0 con 0x7f111416d700 2023-10-08T00:14:36.422 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.405+0000 7f1119a2f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11140b4dc0 con 0x7f11140b3d60 2023-10-08T00:14:36.422 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.405+0000 7f1118a2d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10fc004180 0x7f1114154c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:36.422 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.405+0000 7f1118a2d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10fc004180 0x7f1114154c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33634/0 (socket says 172.21.15.99:33634) 2023-10-08T00:14:36.422 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.405+0000 7f1118a2d700 1 -- 172.21.15.99:0/1531578599 learned_addr learned my addr 172.21.15.99:0/1531578599 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:36.422 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.405+0000 7f110bfff700 1 --2- 172.21.15.99:0/1531578599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f111416d700 0x7f111416db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:36.422 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.405+0000 7f111922e700 1 --2- 172.21.15.99:0/1531578599 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f11140b3d60 0x7f11140b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:36.423 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.405+0000 7f1118a2d700 1 -- 172.21.15.99:0/1531578599 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f11140b3d60 msgr2=0x7f11140b41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:36.423 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.405+0000 7f1118a2d700 1 --2- 172.21.15.99:0/1531578599 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f11140b3d60 0x7f11140b41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:36.423 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.405+0000 7f1118a2d700 1 -- 172.21.15.99:0/1531578599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f111416d700 msgr2=0x7f111416db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:36.423 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.405+0000 7f1118a2d700 1 --2- 172.21.15.99:0/1531578599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f111416d700 0x7f111416db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:36.423 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.405+0000 7f1118a2d700 1 -- 172.21.15.99:0/1531578599 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11140b50a0 con 0x7f10fc004180 2023-10-08T00:14:36.423 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.405+0000 7f1118a2d700 1 --2- 172.21.15.99:0/1531578599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10fc004180 0x7f1114154c30 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f1110011f80 tx=0x7f111000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:36.423 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.406+0000 7f1108ff9700 1 -- 172.21.15.99:0/1531578599 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111000e040 con 0x7f10fc004180 2023-10-08T00:14:36.424 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.406+0000 7f1108ff9700 1 -- 172.21.15.99:0/1531578599 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f111000f040 con 0x7f10fc004180 2023-10-08T00:14:36.424 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.406+0000 7f1119a2f700 1 -- 172.21.15.99:0/1531578599 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f111414e420 con 0x7f10fc004180 2023-10-08T00:14:36.424 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.406+0000 7f1108ff9700 1 -- 172.21.15.99:0/1531578599 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1110015430 con 0x7f10fc004180 2023-10-08T00:14:36.424 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.406+0000 7f1119a2f700 1 -- 172.21.15.99:0/1531578599 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11140b5380 con 0x7f10fc004180 2023-10-08T00:14:36.424 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.406+0000 7f1119a2f700 1 -- 172.21.15.99:0/1531578599 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f111414e640 con 0x7f10fc004180 2023-10-08T00:14:36.425 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.407+0000 7f1108ff9700 1 -- 172.21.15.99:0/1531578599 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1110015820 con 0x7f10fc004180 2023-10-08T00:14:36.425 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.407+0000 7f1108ff9700 1 --2- 172.21.15.99:0/1531578599 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1100066f30 0x7f11000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:36.425 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.407+0000 7f1108ff9700 1 -- 172.21.15.99:0/1531578599 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f1110084eb0 con 0x7f10fc004180 2023-10-08T00:14:36.425 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.407+0000 7f110bfff700 1 --2- 172.21.15.99:0/1531578599 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1100066f30 0x7f11000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:36.425 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.408+0000 7f110bfff700 1 --2- 172.21.15.99:0/1531578599 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1100066f30 0x7f11000693e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f111416d660 tx=0x7f1104015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:36.426 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.411+0000 7f1108ff9700 1 -- 172.21.15.99:0/1531578599 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1110055eb0 con 0x7f10fc004180 2023-10-08T00:14:36.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:36.667+0000 7f1119a2f700 1 -- 172.21.15.99:0/1531578599 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]} v 0) v1 -- 0x7f111404bb80 con 0x7f10fc004180 2023-10-08T00:14:37.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.271+0000 7f1108ff9700 1 -- 172.21.15.99:0/1531578599 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]}]=0 v198) v1 ==== 136+0+0 (secure 0 0 0) 0x7f1110055a40 con 0x7f10fc004180 2023-10-08T00:14:37.275 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.274+0000 7f1119a2f700 1 -- 172.21.15.99:0/1531578599 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1100066f30 msgr2=0x7f11000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:37.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.274+0000 7f1119a2f700 1 --2- 172.21.15.99:0/1531578599 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1100066f30 0x7f11000693e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f111416d660 tx=0x7f1104015000 comp rx=0 tx=0).stop 2023-10-08T00:14:37.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.274+0000 7f1119a2f700 1 -- 172.21.15.99:0/1531578599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10fc004180 msgr2=0x7f1114154c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:37.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.274+0000 7f1119a2f700 1 --2- 172.21.15.99:0/1531578599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10fc004180 0x7f1114154c30 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f1110011f80 tx=0x7f111000b040 comp rx=0 tx=0).stop 2023-10-08T00:14:37.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.274+0000 7f1119a2f700 1 -- 172.21.15.99:0/1531578599 shutdown_connections 2023-10-08T00:14:37.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.274+0000 7f1119a2f700 1 --2- 172.21.15.99:0/1531578599 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1100066f30 0x7f11000693e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:37.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.274+0000 7f1119a2f700 1 --2- 172.21.15.99:0/1531578599 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f11140b3d60 0x7f11140b41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:37.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.274+0000 7f1119a2f700 1 --2- 172.21.15.99:0/1531578599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10fc004180 0x7f1114154c30 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:37.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.274+0000 7f1119a2f700 1 --2- 172.21.15.99:0/1531578599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f111416d700 0x7f111416db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:37.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.274+0000 7f1119a2f700 1 -- 172.21.15.99:0/1531578599 >> 172.21.15.99:0/1531578599 conn(0x7f1114067660 msgr2=0x7f11140cf6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:37.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.274+0000 7f1119a2f700 1 -- 172.21.15.99:0/1531578599 shutdown_connections 2023-10-08T00:14:37.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.274+0000 7f1119a2f700 1 -- 172.21.15.99:0/1531578599 wait complete. 2023-10-08T00:14:37.288 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2023-10-08T00:14:37.421 INFO:teuthology.orchestra.run.smithi099.stdout:{"osd_objectstore":"memstore"} 2023-10-08T00:14:37.421 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2023-10-08T00:14:37.421 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T00:14:37.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.537+0000 7f04332d5700 1 Processor -- start 2023-10-08T00:14:37.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.538+0000 7f04332d5700 1 -- start start 2023-10-08T00:14:37.541 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.538+0000 7f04332d5700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f042c0b4860 0x7f042c0b6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:37.541 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.538+0000 7f04332d5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f042c0b3570 0x7f042c15a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:37.541 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.538+0000 7f04332d5700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f042c0b14f0 con 0x7f042c0b4c30 2023-10-08T00:14:37.541 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.538+0000 7f04332d5700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f042c0b17d0 con 0x7f042c0b4860 2023-10-08T00:14:37.541 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.538+0000 7f04332d5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f042c0b1ab0 con 0x7f042c0b3570 2023-10-08T00:14:37.541 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.539+0000 7f0431071700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f042c0b4860 0x7f042c0b6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:37.541 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.539+0000 7f0431071700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f042c0b4860 0x7f042c0b6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33640/0 (socket says 172.21.15.99:33640) 2023-10-08T00:14:37.541 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.539+0000 7f0431071700 1 -- 172.21.15.99:0/1985998189 learned_addr learned my addr 172.21.15.99:0/1985998189 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:37.541 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.539+0000 7f0431872700 1 --2- 172.21.15.99:0/1985998189 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f042c0b3570 0x7f042c15a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:37.541 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.539+0000 7f0431071700 1 -- 172.21.15.99:0/1985998189 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f042c0b3570 msgr2=0x7f042c15a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:37.542 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.539+0000 7f0431071700 1 --2- 172.21.15.99:0/1985998189 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f042c0b3570 0x7f042c15a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:37.542 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.539+0000 7f0431071700 1 -- 172.21.15.99:0/1985998189 >> v1:172.21.15.191:6789/0 conn(0x7f042c0b4c30 legacy=0x7f042c159a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:37.542 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.539+0000 7f0431071700 1 -- 172.21.15.99:0/1985998189 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f042c0b1d90 con 0x7f042c0b4860 2023-10-08T00:14:37.542 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.539+0000 7f0431071700 1 --2- 172.21.15.99:0/1985998189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f042c0b4860 0x7f042c0b6700 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f0428009f70 tx=0x7f0428009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad0aa37076b88eb5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:37.542 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.539+0000 7f0423fff700 1 -- 172.21.15.99:0/1985998189 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042800e070 con 0x7f042c0b4860 2023-10-08T00:14:37.542 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.539+0000 7f0423fff700 1 -- 172.21.15.99:0/1985998189 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f042800bb20 con 0x7f042c0b4860 2023-10-08T00:14:37.542 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.539+0000 7f0423fff700 1 -- 172.21.15.99:0/1985998189 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04280092e0 con 0x7f042c0b4860 2023-10-08T00:14:37.542 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.540+0000 7f04332d5700 1 -- 172.21.15.99:0/1985998189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f042c0b4860 msgr2=0x7f042c0b6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:37.542 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.540+0000 7f04332d5700 1 --2- 172.21.15.99:0/1985998189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f042c0b4860 0x7f042c0b6700 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f0428009f70 tx=0x7f0428009840 comp rx=0 tx=0).stop 2023-10-08T00:14:37.543 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.540+0000 7f04332d5700 1 -- 172.21.15.99:0/1985998189 shutdown_connections 2023-10-08T00:14:37.543 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.540+0000 7f04332d5700 1 --2- 172.21.15.99:0/1985998189 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f042c0b3570 0x7f042c15a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:37.543 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.540+0000 7f04332d5700 1 --2- 172.21.15.99:0/1985998189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f042c0b4860 0x7f042c0b6700 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:37.543 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.540+0000 7f04332d5700 1 -- 172.21.15.99:0/1985998189 >> 172.21.15.99:0/1985998189 conn(0x7f042c09a0f0 msgr2=0x7f042c067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:37.543 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.540+0000 7f04332d5700 1 -- 172.21.15.99:0/1985998189 shutdown_connections 2023-10-08T00:14:37.543 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.540+0000 7f04332d5700 1 -- 172.21.15.99:0/1985998189 wait complete. 2023-10-08T00:14:37.546 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.541+0000 7f04332d5700 1 Processor -- start 2023-10-08T00:14:37.546 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.541+0000 7f04332d5700 1 -- start start 2023-10-08T00:14:37.546 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.542+0000 7f04332d5700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f042c0b3570 0x7f042c09e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:37.547 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.542+0000 7f04332d5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f042c0b4860 0x7f042c16dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:37.547 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.542+0000 7f04332d5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f042c0b4c30 0x7f042c16e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:37.547 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.542+0000 7f04332d5700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042c09ebe0 con 0x7f042c0b4c30 2023-10-08T00:14:37.547 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.542+0000 7f04332d5700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042c16bab0 con 0x7f042c0b3570 2023-10-08T00:14:37.547 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.542+0000 7f04332d5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f042c16bd20 con 0x7f042c0b4860 2023-10-08T00:14:37.547 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.542+0000 7f0431872700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f042c0b4c30 0x7f042c16e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:37.547 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.542+0000 7f0431872700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f042c0b4c30 0x7f042c16e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33654/0 (socket says 172.21.15.99:33654) 2023-10-08T00:14:37.547 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.542+0000 7f0431872700 1 -- 172.21.15.99:0/1980075007 learned_addr learned my addr 172.21.15.99:0/1980075007 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:37.547 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.542+0000 7f0431071700 1 --2- 172.21.15.99:0/1980075007 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f042c0b3570 0x7f042c09e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:37.548 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.542+0000 7f0430870700 1 --2- 172.21.15.99:0/1980075007 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f042c0b4860 0x7f042c16dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:37.548 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.542+0000 7f0431872700 1 -- 172.21.15.99:0/1980075007 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f042c0b4860 msgr2=0x7f042c16dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:37.548 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.542+0000 7f0431872700 1 --2- 172.21.15.99:0/1980075007 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f042c0b4860 0x7f042c16dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:37.548 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.542+0000 7f0431872700 1 -- 172.21.15.99:0/1980075007 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f042c0b3570 msgr2=0x7f042c09e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:37.548 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.542+0000 7f0431872700 1 --2- 172.21.15.99:0/1980075007 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f042c0b3570 0x7f042c09e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:37.548 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.542+0000 7f0431872700 1 -- 172.21.15.99:0/1980075007 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0428009ae0 con 0x7f042c0b4c30 2023-10-08T00:14:37.548 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.542+0000 7f0431872700 1 --2- 172.21.15.99:0/1980075007 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f042c0b4c30 0x7f042c16e4e0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f042400c880 tx=0x7f0424008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:37.548 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.542+0000 7f04217fa700 1 -- 172.21.15.99:0/1980075007 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0424011600 con 0x7f042c0b4c30 2023-10-08T00:14:37.548 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.542+0000 7f04217fa700 1 -- 172.21.15.99:0/1980075007 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0424011760 con 0x7f042c0b4c30 2023-10-08T00:14:37.548 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.542+0000 7f04332d5700 1 -- 172.21.15.99:0/1980075007 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f042c16c000 con 0x7f042c0b4c30 2023-10-08T00:14:37.549 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.542+0000 7f04217fa700 1 -- 172.21.15.99:0/1980075007 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0424011b00 con 0x7f042c0b4c30 2023-10-08T00:14:37.549 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.542+0000 7f04332d5700 1 -- 172.21.15.99:0/1980075007 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f042c16c5c0 con 0x7f042c0b4c30 2023-10-08T00:14:37.549 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.543+0000 7f04332d5700 1 -- 172.21.15.99:0/1980075007 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f042c16c7b0 con 0x7f042c0b4c30 2023-10-08T00:14:37.549 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.544+0000 7f04217fa700 1 -- 172.21.15.99:0/1980075007 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f04240193f0 con 0x7f042c0b4c30 2023-10-08T00:14:37.549 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.544+0000 7f04217fa700 1 --2- 172.21.15.99:0/1980075007 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0410066f30 0x7f04100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:37.549 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.544+0000 7f04217fa700 1 -- 172.21.15.99:0/1980075007 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f04240843d0 con 0x7f042c0b4c30 2023-10-08T00:14:37.549 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.544+0000 7f0431071700 1 --2- 172.21.15.99:0/1980075007 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0410066f30 0x7f04100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:37.549 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.545+0000 7f0431071700 1 --2- 172.21.15.99:0/1980075007 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0410066f30 0x7f04100693e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0428009880 tx=0x7f0428017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:37.549 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.549+0000 7f04217fa700 1 -- 172.21.15.99:0/1980075007 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f04240519f0 con 0x7f042c0b4c30 2023-10-08T00:14:37.731 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.730+0000 7f04332d5700 1 -- 172.21.15.99:0/1980075007 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f042c04c3c0 con 0x7f042c0b4c30 2023-10-08T00:14:37.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.734+0000 7f04217fa700 1 -- 172.21.15.99:0/1980075007 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v8)=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0424055010 con 0x7f042c0b4c30 2023-10-08T00:14:37.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.736+0000 7f04332d5700 1 -- 172.21.15.99:0/1980075007 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0410066f30 msgr2=0x7f04100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:37.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.736+0000 7f04332d5700 1 --2- 172.21.15.99:0/1980075007 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0410066f30 0x7f04100693e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0428009880 tx=0x7f0428017000 comp rx=0 tx=0).stop 2023-10-08T00:14:37.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.736+0000 7f04332d5700 1 -- 172.21.15.99:0/1980075007 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f042c0b4c30 msgr2=0x7f042c16e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:37.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.736+0000 7f04332d5700 1 --2- 172.21.15.99:0/1980075007 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f042c0b4c30 0x7f042c16e4e0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f042400c880 tx=0x7f0424008040 comp rx=0 tx=0).stop 2023-10-08T00:14:37.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.736+0000 7f04332d5700 1 -- 172.21.15.99:0/1980075007 shutdown_connections 2023-10-08T00:14:37.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.736+0000 7f04332d5700 1 --2- 172.21.15.99:0/1980075007 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0410066f30 0x7f04100693e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:37.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.736+0000 7f04332d5700 1 --2- 172.21.15.99:0/1980075007 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f042c0b3570 0x7f042c09e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:37.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.736+0000 7f04332d5700 1 --2- 172.21.15.99:0/1980075007 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f042c0b4c30 0x7f042c16e4e0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:37.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.736+0000 7f04332d5700 1 --2- 172.21.15.99:0/1980075007 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f042c0b4860 0x7f042c16dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:37.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.736+0000 7f04332d5700 1 -- 172.21.15.99:0/1980075007 >> 172.21.15.99:0/1980075007 conn(0x7f042c09a0f0 msgr2=0x7f042c158860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:37.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.736+0000 7f04332d5700 1 -- 172.21.15.99:0/1980075007 shutdown_connections 2023-10-08T00:14:37.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.736+0000 7f04332d5700 1 -- 172.21.15.99:0/1980075007 wait complete. 2023-10-08T00:14:37.748 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T00:14:37.749 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T00:14:37.795 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T00:14:37.845 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T00:14:37.845 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-08T00:14:37.845 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:14:37.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.958+0000 7f023ab45700 1 Processor -- start 2023-10-08T00:14:37.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.958+0000 7f023ab45700 1 -- start start 2023-10-08T00:14:37.961 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.959+0000 7f023ab45700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0234152010 0x7f023415b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:37.962 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.959+0000 7f023ab45700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f0234158e50 0x7f02340cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:37.962 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.959+0000 7f023ab45700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f023406c010 con 0x7f02341523e0 2023-10-08T00:14:37.962 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.959+0000 7f023ab45700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f023406bd10 con 0x7f0234152010 2023-10-08T00:14:37.962 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.959+0000 7f023ab45700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f023415c060 con 0x7f0234158e50 2023-10-08T00:14:37.962 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.959+0000 7f02388e1700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0234152010 0x7f023415b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:37.962 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.959+0000 7f022bfff700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f02341523e0 0x7f023406c4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:41710/0 (socket says 172.21.15.99:41710) 2023-10-08T00:14:37.962 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.959+0000 7f022bfff700 1 -- 172.21.15.99:0/1135507315 learned_addr learned my addr 172.21.15.99:0/1135507315 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:14:37.962 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.959+0000 7f02390e2700 1 --2- 172.21.15.99:0/1135507315 >> v2:172.21.15.191:3300/0 conn(0x7f0234158e50 0x7f02340cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:37.962 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.960+0000 7f02388e1700 1 -- 172.21.15.99:0/1135507315 >> v2:172.21.15.191:3300/0 conn(0x7f0234158e50 msgr2=0x7f02340cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:37.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.960+0000 7f02388e1700 1 --2- 172.21.15.99:0/1135507315 >> v2:172.21.15.191:3300/0 conn(0x7f0234158e50 0x7f02340cdf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:37.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.960+0000 7f02388e1700 1 -- 172.21.15.99:0/1135507315 >> v1:172.21.15.99:6789/0 conn(0x7f02341523e0 legacy=0x7f023406c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:37.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.960+0000 7f02388e1700 1 -- 172.21.15.99:0/1135507315 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f023415c340 con 0x7f0234152010 2023-10-08T00:14:37.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.960+0000 7f022b7fe700 1 -- 172.21.15.99:0/1135507315 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1578208229 0 0) 0x7f023406c010 con 0x7f02341523e0 2023-10-08T00:14:37.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.960+0000 7f02388e1700 1 --2- 172.21.15.99:0/1135507315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0234152010 0x7f023415b810 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f023406f170 tx=0x7f0230009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a238bcb96cfbec10 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:37.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.960+0000 7f022b7fe700 1 -- 172.21.15.99:0/1135507315 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023000e070 con 0x7f0234152010 2023-10-08T00:14:37.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.960+0000 7f022b7fe700 1 -- 172.21.15.99:0/1135507315 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f023000bc30 con 0x7f0234152010 2023-10-08T00:14:37.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.960+0000 7f022b7fe700 1 -- 172.21.15.99:0/1135507315 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0230012430 con 0x7f0234152010 2023-10-08T00:14:37.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.960+0000 7f023ab45700 1 -- 172.21.15.99:0/1135507315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0234152010 msgr2=0x7f023415b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:37.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.960+0000 7f023ab45700 1 --2- 172.21.15.99:0/1135507315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0234152010 0x7f023415b810 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f023406f170 tx=0x7f0230009780 comp rx=0 tx=0).stop 2023-10-08T00:14:37.964 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.960+0000 7f023ab45700 1 -- 172.21.15.99:0/1135507315 shutdown_connections 2023-10-08T00:14:37.964 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.960+0000 7f023ab45700 1 --2- 172.21.15.99:0/1135507315 >> v2:172.21.15.191:3300/0 conn(0x7f0234158e50 0x7f02340cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:37.964 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.960+0000 7f023ab45700 1 --2- 172.21.15.99:0/1135507315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0234152010 0x7f023415b810 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:37.964 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.960+0000 7f023ab45700 1 -- 172.21.15.99:0/1135507315 >> 172.21.15.99:0/1135507315 conn(0x7f0234069870 msgr2=0x7f0234069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:37.964 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.961+0000 7f023ab45700 1 -- 172.21.15.99:0/1135507315 shutdown_connections 2023-10-08T00:14:37.964 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.961+0000 7f023ab45700 1 -- 172.21.15.99:0/1135507315 wait complete. 2023-10-08T00:14:37.967 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.962+0000 7f023ab45700 1 Processor -- start 2023-10-08T00:14:37.967 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.962+0000 7f023ab45700 1 -- start start 2023-10-08T00:14:37.967 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.962+0000 7f023ab45700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0234152010 0x7f02341567a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:37.967 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.962+0000 7f023ab45700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02341523e0 0x7f023416ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:37.967 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.962+0000 7f023ab45700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0234158e50 0x7f023415abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:37.967 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.962+0000 7f023ab45700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02340c8bf0 con 0x7f02341523e0 2023-10-08T00:14:37.967 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.962+0000 7f023ab45700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023415b100 con 0x7f0234152010 2023-10-08T00:14:37.968 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.962+0000 7f023ab45700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f023416a0d0 con 0x7f0234158e50 2023-10-08T00:14:37.968 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.963+0000 7f022bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02341523e0 0x7f023416ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:37.968 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.963+0000 7f022bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02341523e0 0x7f023416ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33682/0 (socket says 172.21.15.99:33682) 2023-10-08T00:14:37.968 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.963+0000 7f022bfff700 1 -- 172.21.15.99:0/1750453647 learned_addr learned my addr 172.21.15.99:0/1750453647 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:37.968 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.963+0000 7f02388e1700 1 --2- 172.21.15.99:0/1750453647 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0234152010 0x7f02341567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:37.968 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.963+0000 7f02390e2700 1 --2- 172.21.15.99:0/1750453647 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0234158e50 0x7f023415abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:37.968 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.963+0000 7f022bfff700 1 -- 172.21.15.99:0/1750453647 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0234158e50 msgr2=0x7f023415abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:37.968 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.963+0000 7f022bfff700 1 --2- 172.21.15.99:0/1750453647 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0234158e50 0x7f023415abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:37.968 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.963+0000 7f022bfff700 1 -- 172.21.15.99:0/1750453647 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0234152010 msgr2=0x7f02341567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:37.968 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.963+0000 7f022bfff700 1 --2- 172.21.15.99:0/1750453647 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0234152010 0x7f02341567a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:37.968 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.963+0000 7f022bfff700 1 -- 172.21.15.99:0/1750453647 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0230009a20 con 0x7f02341523e0 2023-10-08T00:14:37.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.963+0000 7f02388e1700 1 --2- 172.21.15.99:0/1750453647 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0234152010 0x7f02341567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:14:37.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.963+0000 7f022bfff700 1 --2- 172.21.15.99:0/1750453647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02341523e0 0x7f023416ab90 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f022400c7e0 tx=0x7f0224007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:37.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.963+0000 7f0228ff9700 1 -- 172.21.15.99:0/1750453647 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0224005b30 con 0x7f02341523e0 2023-10-08T00:14:37.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.963+0000 7f0228ff9700 1 -- 172.21.15.99:0/1750453647 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0224005c90 con 0x7f02341523e0 2023-10-08T00:14:37.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.963+0000 7f0228ff9700 1 -- 172.21.15.99:0/1750453647 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0224007300 con 0x7f02341523e0 2023-10-08T00:14:37.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.963+0000 7f023ab45700 1 -- 172.21.15.99:0/1750453647 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0234156d80 con 0x7f02341523e0 2023-10-08T00:14:37.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.963+0000 7f023ab45700 1 -- 172.21.15.99:0/1750453647 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f023416a390 con 0x7f02341523e0 2023-10-08T00:14:37.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.964+0000 7f023ab45700 1 -- 172.21.15.99:0/1750453647 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f023404c3c0 con 0x7f02341523e0 2023-10-08T00:14:37.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.964+0000 7f0228ff9700 1 -- 172.21.15.99:0/1750453647 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f022401b400 con 0x7f02341523e0 2023-10-08T00:14:37.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.965+0000 7f0228ff9700 1 --2- 172.21.15.99:0/1750453647 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0220066f30 0x7f02200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:37.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.965+0000 7f0228ff9700 1 -- 172.21.15.99:0/1750453647 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f0224083ce0 con 0x7f02341523e0 2023-10-08T00:14:37.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.965+0000 7f02388e1700 1 --2- 172.21.15.99:0/1750453647 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0220066f30 0x7f02200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:37.970 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.965+0000 7f02388e1700 1 --2- 172.21.15.99:0/1750453647 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0220066f30 0x7f02200693e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f023406f170 tx=0x7f0230019000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:37.970 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:37.969+0000 7f0228ff9700 1 -- 172.21.15.99:0/1750453647 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f02240512d0 con 0x7f02341523e0 2023-10-08T00:14:38.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.192+0000 7f023ab45700 1 -- 172.21.15.99:0/1750453647 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f023406d1b0 con 0x7f02341523e0 2023-10-08T00:14:38.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.193+0000 7f0228ff9700 1 -- 172.21.15.99:0/1750453647 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f02240548f0 con 0x7f02341523e0 2023-10-08T00:14:38.194 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:14:38.194 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":34,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5087,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/862112794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":862112794},{"type":"v1","addr":"172.21.15.191:6837","nonce":862112794}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5216,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/2482826186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":2482826186},{"type":"v1","addr":"172.21.15.191:6836","nonce":2482826186}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5673,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/2694493794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2694493794},{"type":"v1","addr":"172.21.15.99:6834","nonce":2694493794}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1906990531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":1906990531},{"type":"v1","addr":"172.21.15.99:6835","nonce":1906990531}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[]} 2023-10-08T00:14:38.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.195+0000 7f023ab45700 1 -- 172.21.15.99:0/1750453647 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0220066f30 msgr2=0x7f02200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:38.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.195+0000 7f023ab45700 1 --2- 172.21.15.99:0/1750453647 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0220066f30 0x7f02200693e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f023406f170 tx=0x7f0230019000 comp rx=0 tx=0).stop 2023-10-08T00:14:38.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.195+0000 7f023ab45700 1 -- 172.21.15.99:0/1750453647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02341523e0 msgr2=0x7f023416ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:38.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.195+0000 7f023ab45700 1 --2- 172.21.15.99:0/1750453647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02341523e0 0x7f023416ab90 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f022400c7e0 tx=0x7f0224007df0 comp rx=0 tx=0).stop 2023-10-08T00:14:38.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.195+0000 7f023ab45700 1 -- 172.21.15.99:0/1750453647 shutdown_connections 2023-10-08T00:14:38.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.195+0000 7f023ab45700 1 --2- 172.21.15.99:0/1750453647 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0220066f30 0x7f02200693e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:38.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.195+0000 7f023ab45700 1 --2- 172.21.15.99:0/1750453647 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0234152010 0x7f02341567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:38.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.195+0000 7f023ab45700 1 --2- 172.21.15.99:0/1750453647 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0234158e50 0x7f023415abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:38.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.195+0000 7f023ab45700 1 --2- 172.21.15.99:0/1750453647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02341523e0 0x7f023416ab90 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:38.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.195+0000 7f023ab45700 1 -- 172.21.15.99:0/1750453647 >> 172.21.15.99:0/1750453647 conn(0x7f0234069870 msgr2=0x7f02341514c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:38.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.195+0000 7f023ab45700 1 -- 172.21.15.99:0/1750453647 shutdown_connections 2023-10-08T00:14:38.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.195+0000 7f023ab45700 1 -- 172.21.15.99:0/1750453647 wait complete. 2023-10-08T00:14:38.197 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 34 2023-10-08T00:14:38.208 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T00:14:38.208 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-08T00:14:38.209 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T00:14:38.209 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:14:38.209 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T00:14:38.209 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-08T00:14:38.209 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T00:14:38.209 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:14:38.210 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T00:14:38.210 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-08T00:14:38.210 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T00:14:38.210 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:14:38.210 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T00:14:38.210 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-08T00:14:38.211 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T00:14:38.211 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:14:38.217 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T00:14:38.217 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T00:14:38.218 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T00:14:38.218 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T00:14:38.224 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T00:14:38.224 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T00:14:38.225 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T00:14:38.225 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T00:14:38.260 INFO:tasks.ceph.mds.a:Started 2023-10-08T00:14:38.261 INFO:tasks.ceph.mds.c:Started 2023-10-08T00:14:38.261 DEBUG:teuthology.parallel:result is None 2023-10-08T00:14:38.261 DEBUG:teuthology.parallel:result is None 2023-10-08T00:14:38.267 INFO:tasks.ceph.mds.d:Started 2023-10-08T00:14:38.268 DEBUG:teuthology.parallel:result is None 2023-10-08T00:14:38.269 INFO:tasks.ceph.mds.b:Started 2023-10-08T00:14:38.269 DEBUG:teuthology.parallel:result is None 2023-10-08T00:14:38.269 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-08T00:14:38.333 INFO:tasks.ceph.mds.a.smithi099.stdout:starting mds.a at 2023-10-08T00:14:38.333 INFO:tasks.ceph.mds.c.smithi099.stdout:starting mds.c at 2023-10-08T00:14:38.354 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2023-10-08T00:14:38.354 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2023-10-08T00:14:38.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.389+0000 7fa634efd700 1 Processor -- start 2023-10-08T00:14:38.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.389+0000 7fa634efd700 1 -- start start 2023-10-08T00:14:38.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.390+0000 7fa634efd700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa630159ef0 0x7fa6300ce2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:38.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.390+0000 7fa634efd700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6300c91d0 con 0x7fa6300cbca0 2023-10-08T00:14:38.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.390+0000 7fa634efd700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa63015de70 con 0x7fa63015a2c0 2023-10-08T00:14:38.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.390+0000 7fa634efd700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa63015dc60 con 0x7fa630159ef0 2023-10-08T00:14:38.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.390+0000 7fa62ed9d700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fa6300cbca0 0x7fa63006be20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:41730/0 (socket says 172.21.15.99:41730) 2023-10-08T00:14:38.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.390+0000 7fa62ed9d700 1 -- 172.21.15.99:0/443570213 learned_addr learned my addr 172.21.15.99:0/443570213 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:14:38.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.390+0000 7fa62e59c700 1 --2- 172.21.15.99:0/443570213 >> v2:172.21.15.191:3300/0 conn(0x7fa630159ef0 0x7fa6300ce2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:38.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.391+0000 7fa62d59a700 1 -- 172.21.15.99:0/443570213 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2137406444 0 0) 0x7fa6300c91d0 con 0x7fa6300cbca0 2023-10-08T00:14:38.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.391+0000 7fa62d59a700 1 -- 172.21.15.99:0/443570213 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa618003570 con 0x7fa6300cbca0 2023-10-08T00:14:38.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.391+0000 7fa62d59a700 1 -- 172.21.15.99:0/443570213 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3421521021 0 0) 0x7fa63015de70 con 0x7fa63015a2c0 2023-10-08T00:14:38.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.391+0000 7fa62d59a700 1 -- 172.21.15.99:0/443570213 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6300c91d0 con 0x7fa63015a2c0 2023-10-08T00:14:38.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.391+0000 7fa62e59c700 1 -- 172.21.15.99:0/443570213 >> v1:172.21.15.191:6789/0 conn(0x7fa63015a2c0 legacy=0x7fa6300cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:38.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.391+0000 7fa62e59c700 1 -- 172.21.15.99:0/443570213 >> v1:172.21.15.99:6789/0 conn(0x7fa6300cbca0 legacy=0x7fa63006be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:38.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.391+0000 7fa62e59c700 1 -- 172.21.15.99:0/443570213 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa63015e150 con 0x7fa630159ef0 2023-10-08T00:14:38.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.391+0000 7fa62e59c700 1 --2- 172.21.15.99:0/443570213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa630159ef0 0x7fa6300ce2e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa6200098e0 tx=0x7fa6200091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a69352ab8a0253d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:38.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.391+0000 7fa62d59a700 1 -- 172.21.15.99:0/443570213 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa620011630 con 0x7fa630159ef0 2023-10-08T00:14:38.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.391+0000 7fa62d59a700 1 -- 172.21.15.99:0/443570213 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa620011790 con 0x7fa630159ef0 2023-10-08T00:14:38.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.391+0000 7fa62d59a700 1 -- 172.21.15.99:0/443570213 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa620011b30 con 0x7fa630159ef0 2023-10-08T00:14:38.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.392+0000 7fa634efd700 1 -- 172.21.15.99:0/443570213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa630159ef0 msgr2=0x7fa6300ce2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:38.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.392+0000 7fa634efd700 1 --2- 172.21.15.99:0/443570213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa630159ef0 0x7fa6300ce2e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa6200098e0 tx=0x7fa6200091a0 comp rx=0 tx=0).stop 2023-10-08T00:14:38.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.392+0000 7fa634efd700 1 -- 172.21.15.99:0/443570213 shutdown_connections 2023-10-08T00:14:38.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.392+0000 7fa634efd700 1 --2- 172.21.15.99:0/443570213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa630159ef0 0x7fa6300ce2e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:38.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.392+0000 7fa634efd700 1 -- 172.21.15.99:0/443570213 >> 172.21.15.99:0/443570213 conn(0x7fa630069870 msgr2=0x7fa630069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:38.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.392+0000 7fa634efd700 1 -- 172.21.15.99:0/443570213 shutdown_connections 2023-10-08T00:14:38.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.392+0000 7fa634efd700 1 -- 172.21.15.99:0/443570213 wait complete. 2023-10-08T00:14:38.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.393+0000 7fa634efd700 1 Processor -- start 2023-10-08T00:14:38.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.393+0000 7fa634efd700 1 -- start start 2023-10-08T00:14:38.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.393+0000 7fa634efd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6300cbca0 0x7fa630154780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:38.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.393+0000 7fa634efd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa630159ef0 0x7fa6300c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:38.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.393+0000 7fa634efd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa63015a2c0 0x7fa6300c8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:38.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.393+0000 7fa634efd700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6300c7e80 con 0x7fa63015a2c0 2023-10-08T00:14:38.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.393+0000 7fa634efd700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63016e680 con 0x7fa6300cbca0 2023-10-08T00:14:38.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.393+0000 7fa634efd700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa63016d790 con 0x7fa630159ef0 2023-10-08T00:14:38.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.394+0000 7fa62ed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa63015a2c0 0x7fa6300c8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:38.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.394+0000 7fa62ed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa63015a2c0 0x7fa6300c8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33714/0 (socket says 172.21.15.99:33714) 2023-10-08T00:14:38.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.394+0000 7fa62ed9d700 1 -- 172.21.15.99:0/1953215020 learned_addr learned my addr 172.21.15.99:0/1953215020 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:38.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.394+0000 7fa62e59c700 1 --2- 172.21.15.99:0/1953215020 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6300cbca0 0x7fa630154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:38.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.394+0000 7fa62dd9b700 1 --2- 172.21.15.99:0/1953215020 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa630159ef0 0x7fa6300c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:38.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.394+0000 7fa62ed9d700 1 -- 172.21.15.99:0/1953215020 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa630159ef0 msgr2=0x7fa6300c80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:38.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.394+0000 7fa62ed9d700 1 --2- 172.21.15.99:0/1953215020 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa630159ef0 0x7fa6300c80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:38.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.394+0000 7fa62ed9d700 1 -- 172.21.15.99:0/1953215020 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6300cbca0 msgr2=0x7fa630154780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:38.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.394+0000 7fa62ed9d700 1 --2- 172.21.15.99:0/1953215020 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6300cbca0 0x7fa630154780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:38.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.394+0000 7fa62ed9d700 1 -- 172.21.15.99:0/1953215020 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa620009420 con 0x7fa63015a2c0 2023-10-08T00:14:38.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.394+0000 7fa62dd9b700 1 --2- 172.21.15.99:0/1953215020 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa630159ef0 0x7fa6300c80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:14:38.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.394+0000 7fa62ed9d700 1 --2- 172.21.15.99:0/1953215020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa63015a2c0 0x7fa6300c8630 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fa628001460 tx=0x7fa628001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:38.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.394+0000 7fa61e7fc700 1 -- 172.21.15.99:0/1953215020 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62800a960 con 0x7fa63015a2c0 2023-10-08T00:14:38.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.394+0000 7fa61e7fc700 1 -- 172.21.15.99:0/1953215020 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa62800aac0 con 0x7fa63015a2c0 2023-10-08T00:14:38.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.394+0000 7fa634efd700 1 -- 172.21.15.99:0/1953215020 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa63016da70 con 0x7fa63015a2c0 2023-10-08T00:14:38.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.394+0000 7fa61e7fc700 1 -- 172.21.15.99:0/1953215020 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62800ae60 con 0x7fa63015a2c0 2023-10-08T00:14:38.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.394+0000 7fa634efd700 1 -- 172.21.15.99:0/1953215020 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa63016e030 con 0x7fa63015a2c0 2023-10-08T00:14:38.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.395+0000 7fa634efd700 1 -- 172.21.15.99:0/1953215020 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa63015a750 con 0x7fa63015a2c0 2023-10-08T00:14:38.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.396+0000 7fa61e7fc700 1 -- 172.21.15.99:0/1953215020 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa628007300 con 0x7fa63015a2c0 2023-10-08T00:14:38.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.396+0000 7fa61e7fc700 1 --2- 172.21.15.99:0/1953215020 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa610066f30 0x7fa6100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:38.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.396+0000 7fa61e7fc700 1 -- 172.21.15.99:0/1953215020 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fa6280845d0 con 0x7fa63015a2c0 2023-10-08T00:14:38.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.396+0000 7fa62e59c700 1 --2- 172.21.15.99:0/1953215020 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa610066f30 0x7fa6100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:38.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.397+0000 7fa62e59c700 1 --2- 172.21.15.99:0/1953215020 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa610066f30 0x7fa6100693e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa6200093f0 tx=0x7fa620017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:38.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.400+0000 7fa61e7fc700 1 -- 172.21.15.99:0/1953215020 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa63015a750 con 0x7fa63015a2c0 2023-10-08T00:14:38.597 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:38.597+0000 7fa634efd700 1 -- 172.21.15.99:0/1953215020 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fa63015a750 con 0x7fa63015a2c0 2023-10-08T00:14:39.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.758+0000 7fa61e7fc700 1 -- 172.21.15.99:0/1953215020 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v82) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa63015a750 con 0x7fa63015a2c0 2023-10-08T00:14:39.762 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.760+0000 7fa634efd700 1 -- 172.21.15.99:0/1953215020 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa610066f30 msgr2=0x7fa6100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:39.762 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.760+0000 7fa634efd700 1 --2- 172.21.15.99:0/1953215020 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa610066f30 0x7fa6100693e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa6200093f0 tx=0x7fa620017000 comp rx=0 tx=0).stop 2023-10-08T00:14:39.762 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.761+0000 7fa634efd700 1 -- 172.21.15.99:0/1953215020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa63015a2c0 msgr2=0x7fa6300c8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:39.762 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.761+0000 7fa634efd700 1 --2- 172.21.15.99:0/1953215020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa63015a2c0 0x7fa6300c8630 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fa628001460 tx=0x7fa628001ac0 comp rx=0 tx=0).stop 2023-10-08T00:14:39.762 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.761+0000 7fa634efd700 1 -- 172.21.15.99:0/1953215020 shutdown_connections 2023-10-08T00:14:39.762 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.761+0000 7fa634efd700 1 --2- 172.21.15.99:0/1953215020 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa610066f30 0x7fa6100693e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:39.762 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.761+0000 7fa634efd700 1 --2- 172.21.15.99:0/1953215020 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6300cbca0 0x7fa630154780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:39.763 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.761+0000 7fa634efd700 1 --2- 172.21.15.99:0/1953215020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa63015a2c0 0x7fa6300c8630 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:39.763 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.761+0000 7fa634efd700 1 --2- 172.21.15.99:0/1953215020 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa630159ef0 0x7fa6300c80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:39.763 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.761+0000 7fa634efd700 1 -- 172.21.15.99:0/1953215020 >> 172.21.15.99:0/1953215020 conn(0x7fa630069870 msgr2=0x7fa6301578b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:39.763 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.761+0000 7fa634efd700 1 -- 172.21.15.99:0/1953215020 shutdown_connections 2023-10-08T00:14:39.763 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.761+0000 7fa634efd700 1 -- 172.21.15.99:0/1953215020 wait complete. 2023-10-08T00:14:39.763 INFO:teuthology.orchestra.run.smithi099.stderr: removed all blocklist entries 2023-10-08T00:14:39.776 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-08T00:14:39.895 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.894+0000 7fd8edeb0700 1 Processor -- start 2023-10-08T00:14:39.895 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.895+0000 7fd8edeb0700 1 -- start start 2023-10-08T00:14:39.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.895+0000 7fd8edeb0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8e8068280 0x7fd8e80adab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:39.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.895+0000 7fd8edeb0700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8e80c87d0 con 0x7fd8e8156e20 2023-10-08T00:14:39.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.895+0000 7fd8edeb0700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8e8068750 con 0x7fd8e8156a50 2023-10-08T00:14:39.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.895+0000 7fd8edeb0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e8068d10 con 0x7fd8e8068280 2023-10-08T00:14:39.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.895+0000 7fd8e6ffd700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fd8e8156e20 0x7fd8e8067b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:41740/0 (socket says 172.21.15.99:41740) 2023-10-08T00:14:39.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.895+0000 7fd8e6ffd700 1 -- 172.21.15.99:0/2927627076 learned_addr learned my addr 172.21.15.99:0/2927627076 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:14:39.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.896+0000 7fd8e7fff700 1 --2- 172.21.15.99:0/2927627076 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8e8068280 0x7fd8e80adab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:39.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.896+0000 7fd8e67fc700 1 -- 172.21.15.99:0/2927627076 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3550952231 0 0) 0x7fd8e80c87d0 con 0x7fd8e8156e20 2023-10-08T00:14:39.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.896+0000 7fd8e67fc700 1 -- 172.21.15.99:0/2927627076 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8d0003570 con 0x7fd8e8156e20 2023-10-08T00:14:39.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.896+0000 7fd8e67fc700 1 -- 172.21.15.99:0/2927627076 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 871467879 0 0) 0x7fd8e8068750 con 0x7fd8e8156a50 2023-10-08T00:14:39.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.896+0000 7fd8e67fc700 1 -- 172.21.15.99:0/2927627076 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8e80c87d0 con 0x7fd8e8156a50 2023-10-08T00:14:39.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.896+0000 7fd8e67fc700 1 -- 172.21.15.99:0/2927627076 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 374704070 0 0) 0x7fd8d0003570 con 0x7fd8e8156e20 2023-10-08T00:14:39.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.896+0000 7fd8e67fc700 1 -- 172.21.15.99:0/2927627076 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8e8068280 msgr2=0x7fd8e80adab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:39.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.896+0000 7fd8e67fc700 1 --2- 172.21.15.99:0/2927627076 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8e8068280 0x7fd8e80adab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:39.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.896+0000 7fd8e67fc700 1 -- 172.21.15.99:0/2927627076 >> v1:172.21.15.191:6789/0 conn(0x7fd8e8156a50 legacy=0x7fd8e80c8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:39.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.896+0000 7fd8e67fc700 1 -- 172.21.15.99:0/2927627076 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e8068a30 con 0x7fd8e8156e20 2023-10-08T00:14:39.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.896+0000 7fd8e67fc700 1 -- 172.21.15.99:0/2927627076 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7fd8dc001470 con 0x7fd8e8156e20 2023-10-08T00:14:39.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.896+0000 7fd8e67fc700 1 -- 172.21.15.99:0/2927627076 >> v1:172.21.15.99:6789/0 conn(0x7fd8e8156e20 legacy=0x7fd8e8067b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:39.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.896+0000 7fd8e67fc700 1 --2- 172.21.15.99:0/2927627076 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8e8068280 0x7fd8d0003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:39.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.896+0000 7fd8e67fc700 1 --2- 172.21.15.99:0/2927627076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8d00041d0 0x7fd8d0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:39.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.896+0000 7fd8e67fc700 1 --2- 172.21.15.99:0/2927627076 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8d0004d50 0x7fd8d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:39.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.896+0000 7fd8e67fc700 1 -- 172.21.15.99:0/2927627076 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e8068d10 con 0x7fd8d00041d0 2023-10-08T00:14:39.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.896+0000 7fd8e67fc700 1 -- 172.21.15.99:0/2927627076 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d000bbf0 con 0x7fd8d0004d50 2023-10-08T00:14:39.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.896+0000 7fd8e67fc700 1 -- 172.21.15.99:0/2927627076 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d000bdb0 con 0x7fd8e8068280 2023-10-08T00:14:39.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.896+0000 7fd8e6ffd700 1 --2- 172.21.15.99:0/2927627076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8d00041d0 0x7fd8d0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:39.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.896+0000 7fd8e77fe700 1 --2- 172.21.15.99:0/2927627076 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8e8068280 0x7fd8d0003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:39.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.896+0000 7fd8e7fff700 1 --2- 172.21.15.99:0/2927627076 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8d0004d50 0x7fd8d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:39.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.896+0000 7fd8e6ffd700 1 -- 172.21.15.99:0/2927627076 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8e8068280 msgr2=0x7fd8d0003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:39.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.896+0000 7fd8e6ffd700 1 --2- 172.21.15.99:0/2927627076 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8e8068280 0x7fd8d0003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:39.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.896+0000 7fd8e6ffd700 1 -- 172.21.15.99:0/2927627076 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8d0004d50 msgr2=0x7fd8d00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:39.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.896+0000 7fd8e6ffd700 1 --2- 172.21.15.99:0/2927627076 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8d0004d50 0x7fd8d00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:39.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.896+0000 7fd8e6ffd700 1 -- 172.21.15.99:0/2927627076 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d0003570 con 0x7fd8d00041d0 2023-10-08T00:14:39.903 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.897+0000 7fd8e6ffd700 1 --2- 172.21.15.99:0/2927627076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8d00041d0 0x7fd8d0004810 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc007fd0 tx=0x7fd8e80c8740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ace108a3a07ae848 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:39.903 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.897+0000 7fd8e67fc700 1 -- 172.21.15.99:0/2927627076 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc00ca80 con 0x7fd8d00041d0 2023-10-08T00:14:39.903 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.897+0000 7fd8e67fc700 1 -- 172.21.15.99:0/2927627076 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8dc00cbe0 con 0x7fd8d00041d0 2023-10-08T00:14:39.903 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.897+0000 7fd8e67fc700 1 -- 172.21.15.99:0/2927627076 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc005920 con 0x7fd8d00041d0 2023-10-08T00:14:39.903 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.897+0000 7fd8edeb0700 1 -- 172.21.15.99:0/2927627076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8d00041d0 msgr2=0x7fd8d0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:39.903 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.897+0000 7fd8edeb0700 1 --2- 172.21.15.99:0/2927627076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8d00041d0 0x7fd8d0004810 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc007fd0 tx=0x7fd8e80c8740 comp rx=0 tx=0).stop 2023-10-08T00:14:39.903 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.897+0000 7fd8e7fff700 1 -- 172.21.15.99:0/2927627076 reap_dead start 2023-10-08T00:14:39.903 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.897+0000 7fd8edeb0700 1 -- 172.21.15.99:0/2927627076 shutdown_connections 2023-10-08T00:14:39.903 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.897+0000 7fd8edeb0700 1 -- 172.21.15.99:0/2927627076 >> 172.21.15.99:0/2927627076 conn(0x7fd8e806a250 msgr2=0x7fd8e806a650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:39.904 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.897+0000 7fd8edeb0700 1 -- 172.21.15.99:0/2927627076 shutdown_connections 2023-10-08T00:14:39.904 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.897+0000 7fd8edeb0700 1 -- 172.21.15.99:0/2927627076 wait complete. 2023-10-08T00:14:39.904 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.899+0000 7fd8edeb0700 1 Processor -- start 2023-10-08T00:14:39.904 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.899+0000 7fd8edeb0700 1 -- start start 2023-10-08T00:14:39.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.899+0000 7fd8edeb0700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8d00041d0 0x7fd8e8157260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:39.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.899+0000 7fd8edeb0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8e8158f00 0x7fd8e814fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:39.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.899+0000 7fd8edeb0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8e8151ed0 0x7fd8e8152340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:39.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.899+0000 7fd8edeb0700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e815a140 con 0x7fd8e8158f00 2023-10-08T00:14:39.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.899+0000 7fd8edeb0700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e8152880 con 0x7fd8d00041d0 2023-10-08T00:14:39.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.899+0000 7fd8edeb0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e8158650 con 0x7fd8e8151ed0 2023-10-08T00:14:39.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.899+0000 7fd8e6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8e8158f00 0x7fd8e814fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:39.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.899+0000 7fd8e6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8e8158f00 0x7fd8e814fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33734/0 (socket says 172.21.15.99:33734) 2023-10-08T00:14:39.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.899+0000 7fd8e6ffd700 1 -- 172.21.15.99:0/1586229721 learned_addr learned my addr 172.21.15.99:0/1586229721 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:39.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.900+0000 7fd8e77fe700 1 --2- 172.21.15.99:0/1586229721 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8d00041d0 0x7fd8e8157260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:39.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.900+0000 7fd8e7fff700 1 --2- 172.21.15.99:0/1586229721 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8e8151ed0 0x7fd8e8152340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:39.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.900+0000 7fd8e6ffd700 1 -- 172.21.15.99:0/1586229721 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8e8151ed0 msgr2=0x7fd8e8152340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:39.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.900+0000 7fd8e6ffd700 1 --2- 172.21.15.99:0/1586229721 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8e8151ed0 0x7fd8e8152340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:39.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.900+0000 7fd8e6ffd700 1 -- 172.21.15.99:0/1586229721 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8d00041d0 msgr2=0x7fd8e8157260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:39.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.900+0000 7fd8e6ffd700 1 --2- 172.21.15.99:0/1586229721 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8d00041d0 0x7fd8e8157260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:39.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.900+0000 7fd8e6ffd700 1 -- 172.21.15.99:0/1586229721 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e81577a0 con 0x7fd8e8158f00 2023-10-08T00:14:39.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.900+0000 7fd8e7fff700 1 --2- 172.21.15.99:0/1586229721 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8e8151ed0 0x7fd8e8152340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:14:39.909 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.900+0000 7fd8e6ffd700 1 --2- 172.21.15.99:0/1586229721 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8e8158f00 0x7fd8e814fea0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc001ed0 tx=0x7fd8dc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:39.909 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.900+0000 7fd8cf7fe700 1 -- 172.21.15.99:0/1586229721 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc0070d0 con 0x7fd8e8158f00 2023-10-08T00:14:39.909 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.900+0000 7fd8cf7fe700 1 -- 172.21.15.99:0/1586229721 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8dc007230 con 0x7fd8e8158f00 2023-10-08T00:14:39.909 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.900+0000 7fd8edeb0700 1 -- 172.21.15.99:0/1586229721 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8e8157a40 con 0x7fd8e8158f00 2023-10-08T00:14:39.909 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.900+0000 7fd8cf7fe700 1 -- 172.21.15.99:0/1586229721 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc0075b0 con 0x7fd8e8158f00 2023-10-08T00:14:39.909 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.900+0000 7fd8edeb0700 1 -- 172.21.15.99:0/1586229721 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8e8158000 con 0x7fd8e8158f00 2023-10-08T00:14:39.909 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.901+0000 7fd8edeb0700 1 -- 172.21.15.99:0/1586229721 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8e8158800 con 0x7fd8e8158f00 2023-10-08T00:14:39.909 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.901+0000 7fd8cf7fe700 1 -- 172.21.15.99:0/1586229721 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd8dc01f620 con 0x7fd8e8158f00 2023-10-08T00:14:39.910 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.902+0000 7fd8cf7fe700 1 --2- 172.21.15.99:0/1586229721 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd8d4066f30 0x7fd8d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:39.910 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.902+0000 7fd8cf7fe700 1 -- 172.21.15.99:0/1586229721 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fd8dc085b40 con 0x7fd8e8158f00 2023-10-08T00:14:39.910 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.902+0000 7fd8e77fe700 1 --2- 172.21.15.99:0/1586229721 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd8d4066f30 0x7fd8d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:39.910 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.902+0000 7fd8e77fe700 1 --2- 172.21.15.99:0/1586229721 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd8d4066f30 0x7fd8d40693e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd8d8001460 tx=0x7fd8d80017a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:39.910 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:39.906+0000 7fd8cf7fe700 1 -- 172.21.15.99:0/1586229721 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd8e8158800 con 0x7fd8e8158f00 2023-10-08T00:14:40.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.159+0000 7fd8edeb0700 1 -- 172.21.15.99:0/1586229721 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fd8e81587c0 con 0x7fd8e8158f00 2023-10-08T00:14:40.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.159+0000 7fd8cf7fe700 1 -- 172.21.15.99:0/1586229721 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fd8dc053100 con 0x7fd8e8158f00 2023-10-08T00:14:40.160 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:14:40.160 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:14:40.161 INFO:teuthology.orchestra.run.smithi099.stdout: "auth_dump": [ 2023-10-08T00:14:40.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:14:40.161 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.a", 2023-10-08T00:14:40.161 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlFlN9AhAAKdbeH0XznSOuyUnReD6RsA==", 2023-10-08T00:14:40.161 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:14:40.162 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:14:40.162 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2023-10-08T00:14:40.162 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2023-10-08T00:14:40.162 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:14:40.162 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:14:40.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:14:40.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:14:40.163 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.b", 2023-10-08T00:14:40.163 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlh2G0DRAA1mYGrUrELHsS9abUxCCG2g==", 2023-10-08T00:14:40.163 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:14:40.163 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:14:40.164 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2023-10-08T00:14:40.164 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2023-10-08T00:14:40.164 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:14:40.164 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:14:40.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:14:40.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:14:40.165 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.c", 2023-10-08T00:14:40.165 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFl/M1DCRAANrfHJGZO1s0KOyCdUzZWIA==", 2023-10-08T00:14:40.165 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:14:40.165 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:14:40.166 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2023-10-08T00:14:40.166 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2023-10-08T00:14:40.166 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:14:40.166 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:14:40.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:14:40.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:14:40.167 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.d", 2023-10-08T00:14:40.167 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlr5F2FBAAYYeG6EM0/N0FiEAwSbYRyA==", 2023-10-08T00:14:40.167 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:14:40.167 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:14:40.168 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2023-10-08T00:14:40.168 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2023-10-08T00:14:40.168 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:14:40.168 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:14:40.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:14:40.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:14:40.169 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.0", 2023-10-08T00:14:40.169 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB08yFla7rpKxAAWTOutLcrjDeR69JbPZMoLg==", 2023-10-08T00:14:40.169 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:14:40.169 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:14:40.170 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:14:40.170 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:14:40.170 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:14:40.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:14:40.171 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:14:40.172 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.1", 2023-10-08T00:14:40.172 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB08yFlw+y2MhAAAvLDIB/tb49IlaireTqTFg==", 2023-10-08T00:14:40.172 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:14:40.172 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:14:40.172 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:14:40.173 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:14:40.173 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:14:40.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:14:40.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:14:40.173 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.2", 2023-10-08T00:14:40.174 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB18yFlh4wvABAALXOtJZoEfGmt/pOBxReooQ==", 2023-10-08T00:14:40.174 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:14:40.174 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:14:40.174 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:14:40.174 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:14:40.174 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:14:40.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:14:40.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:14:40.175 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.3", 2023-10-08T00:14:40.175 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB18yFl8cVfCRAAoGXTACMT+f5Avll5mxG9OA==", 2023-10-08T00:14:40.175 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:14:40.176 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:14:40.176 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:14:40.176 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:14:40.176 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:14:40.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:14:40.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:14:40.177 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.4", 2023-10-08T00:14:40.177 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB48yFlft9WGhAA8JZN4MjSX9wbXiMKJnfCYg==", 2023-10-08T00:14:40.177 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:14:40.177 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:14:40.177 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:14:40.177 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:14:40.178 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:14:40.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:14:40.178 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:14:40.178 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.5", 2023-10-08T00:14:40.179 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB48yFlUf1BIRAA2FMBt1jAcm4ia/vYBDyojw==", 2023-10-08T00:14:40.179 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:14:40.179 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:14:40.179 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:14:40.179 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:14:40.180 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:14:40.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:14:40.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:14:40.180 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.6", 2023-10-08T00:14:40.180 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB48yFl0SI6KhAAI6G+Sqfm7OQmGf9nK59x9Q==", 2023-10-08T00:14:40.181 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:14:40.181 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:14:40.181 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:14:40.181 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:14:40.181 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:14:40.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:14:40.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:14:40.182 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.7", 2023-10-08T00:14:40.182 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB48yFlo5xWMxAA4p9OSLK0JpWhEExsifX+3A==", 2023-10-08T00:14:40.182 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:14:40.182 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:14:40.182 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:14:40.182 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:14:40.182 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:14:40.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:14:40.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:14:40.183 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.0", 2023-10-08T00:14:40.183 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlmzhQGBAAReslU/Pvcsr0dwE6l1N3/Q==", 2023-10-08T00:14:40.183 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:14:40.183 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:14:40.183 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow r", 2023-10-08T00:14:40.183 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-08T00:14:40.183 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:14:40.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:14:40.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:14:40.184 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.1", 2023-10-08T00:14:40.184 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFl2NKsIBAAUg6rXynl1J7qB6BkUizmpA==", 2023-10-08T00:14:40.184 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:14:40.184 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:14:40.184 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2023-10-08T00:14:40.184 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2023-10-08T00:14:40.184 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2023-10-08T00:14:40.184 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:14:40.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:14:40.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:14:40.184 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.2", 2023-10-08T00:14:40.184 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlMp6rHRAArVgJfA3mY4n78hOMeVlw1Q==", 2023-10-08T00:14:40.185 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:14:40.185 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:14:40.185 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2023-10-08T00:14:40.185 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2023-10-08T00:14:40.185 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2023-10-08T00:14:40.185 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:14:40.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:14:40.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:14:40.185 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.3", 2023-10-08T00:14:40.185 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlH0gNJhAAEgGVGiabVIWknpWUohOnHA==", 2023-10-08T00:14:40.185 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:14:40.185 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:14:40.186 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2023-10-08T00:14:40.186 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2023-10-08T00:14:40.186 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2023-10-08T00:14:40.186 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:14:40.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:14:40.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:14:40.186 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.admin", 2023-10-08T00:14:40.186 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBw8yFlidUfIxAAIvpvkHbyCx5JFvHo/g5A9g==", 2023-10-08T00:14:40.186 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:14:40.186 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2023-10-08T00:14:40.186 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2023-10-08T00:14:40.186 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2023-10-08T00:14:40.187 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:14:40.187 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:14:40.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:14:40.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:14:40.187 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mgr.x", 2023-10-08T00:14:40.187 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBw8yFl2kMLOxAAsEyuLBLsjQCUS0ERDOi75A==", 2023-10-08T00:14:40.187 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:14:40.187 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2023-10-08T00:14:40.187 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile mgr", 2023-10-08T00:14:40.188 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:14:40.188 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:14:40.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:14:40.188 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:14:40.188 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mgr.y", 2023-10-08T00:14:40.188 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBw8yFl6ht5NxAAFzXnLDjklCePD6hw/pSfFg==", 2023-10-08T00:14:40.188 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:14:40.188 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2023-10-08T00:14:40.188 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile mgr", 2023-10-08T00:14:40.188 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:14:40.188 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:14:40.189 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:14:40.189 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:14:40.189 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:14:40.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.161+0000 7fd8edeb0700 1 -- 172.21.15.99:0/1586229721 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd8d4066f30 msgr2=0x7fd8d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:40.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.161+0000 7fd8edeb0700 1 --2- 172.21.15.99:0/1586229721 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd8d4066f30 0x7fd8d40693e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd8d8001460 tx=0x7fd8d80017a0 comp rx=0 tx=0).stop 2023-10-08T00:14:40.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.161+0000 7fd8edeb0700 1 -- 172.21.15.99:0/1586229721 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8e8158f00 msgr2=0x7fd8e814fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:40.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.161+0000 7fd8edeb0700 1 --2- 172.21.15.99:0/1586229721 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8e8158f00 0x7fd8e814fea0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc001ed0 tx=0x7fd8dc015000 comp rx=0 tx=0).stop 2023-10-08T00:14:40.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.161+0000 7fd8edeb0700 1 -- 172.21.15.99:0/1586229721 shutdown_connections 2023-10-08T00:14:40.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.161+0000 7fd8edeb0700 1 --2- 172.21.15.99:0/1586229721 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd8d4066f30 0x7fd8d40693e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:40.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.161+0000 7fd8edeb0700 1 --2- 172.21.15.99:0/1586229721 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8d00041d0 0x7fd8e8157260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:40.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.161+0000 7fd8edeb0700 1 --2- 172.21.15.99:0/1586229721 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8e8151ed0 0x7fd8e8152340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:40.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.161+0000 7fd8edeb0700 1 --2- 172.21.15.99:0/1586229721 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8e8158f00 0x7fd8e814fea0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:40.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.161+0000 7fd8edeb0700 1 -- 172.21.15.99:0/1586229721 >> 172.21.15.99:0/1586229721 conn(0x7fd8e806a250 msgr2=0x7fd8e8154b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:40.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.161+0000 7fd8edeb0700 1 -- 172.21.15.99:0/1586229721 shutdown_connections 2023-10-08T00:14:40.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.161+0000 7fd8edeb0700 1 -- 172.21.15.99:0/1586229721 wait complete. 2023-10-08T00:14:40.192 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:14:40.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.307+0000 7fa3b6326700 1 Processor -- start 2023-10-08T00:14:40.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.307+0000 7fa3b6326700 1 -- start start 2023-10-08T00:14:40.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.308+0000 7fa3b6326700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3b015cbd0 0x7fa3b015d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:40.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.308+0000 7fa3b6326700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3b00cb7c0 con 0x7fa3b00cd6e0 2023-10-08T00:14:40.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.308+0000 7fa3b6326700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3b015e290 con 0x7fa3b00cdab0 2023-10-08T00:14:40.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.308+0000 7fa3b6326700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b015e850 con 0x7fa3b015cbd0 2023-10-08T00:14:40.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.308+0000 7fa3b5324700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fa3b00cd6e0 0x7fa3b00cbc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:41750/0 (socket says 172.21.15.99:41750) 2023-10-08T00:14:40.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.308+0000 7fa3b5324700 1 -- 172.21.15.99:0/931452477 learned_addr learned my addr 172.21.15.99:0/931452477 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:14:40.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.308+0000 7fa3b5b25700 1 --2- 172.21.15.99:0/931452477 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3b015cbd0 0x7fa3b015d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:40.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.308+0000 7fa3a7fff700 1 -- 172.21.15.99:0/931452477 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3072734693 0 0) 0x7fa3b00cb7c0 con 0x7fa3b00cd6e0 2023-10-08T00:14:40.313 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.308+0000 7fa3a7fff700 1 -- 172.21.15.99:0/931452477 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa39c003570 con 0x7fa3b00cd6e0 2023-10-08T00:14:40.313 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.309+0000 7fa3a7fff700 1 -- 172.21.15.99:0/931452477 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 81105446 0 0) 0x7fa3b015e290 con 0x7fa3b00cdab0 2023-10-08T00:14:40.313 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.309+0000 7fa3a7fff700 1 -- 172.21.15.99:0/931452477 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3b00cb7c0 con 0x7fa3b00cdab0 2023-10-08T00:14:40.313 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.309+0000 7fa3a7fff700 1 -- 172.21.15.99:0/931452477 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1141343914 0 0) 0x7fa39c003570 con 0x7fa3b00cd6e0 2023-10-08T00:14:40.313 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.309+0000 7fa3a7fff700 1 -- 172.21.15.99:0/931452477 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3b015cbd0 msgr2=0x7fa3b015d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:40.313 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.309+0000 7fa3a7fff700 1 --2- 172.21.15.99:0/931452477 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3b015cbd0 0x7fa3b015d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:40.313 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.309+0000 7fa3a7fff700 1 -- 172.21.15.99:0/931452477 >> v1:172.21.15.191:6789/0 conn(0x7fa3b00cdab0 legacy=0x7fa3b015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:40.313 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.309+0000 7fa3a7fff700 1 -- 172.21.15.99:0/931452477 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b015e570 con 0x7fa3b00cd6e0 2023-10-08T00:14:40.313 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.309+0000 7fa3a7fff700 1 -- 172.21.15.99:0/931452477 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7fa3ac001470 con 0x7fa3b00cd6e0 2023-10-08T00:14:40.313 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.309+0000 7fa3a7fff700 1 -- 172.21.15.99:0/931452477 >> v1:172.21.15.99:6789/0 conn(0x7fa3b00cd6e0 legacy=0x7fa3b00cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:40.313 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.309+0000 7fa3a7fff700 1 --2- 172.21.15.99:0/931452477 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa39c003c20 0x7fa39c004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:40.314 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.309+0000 7fa3a7fff700 1 --2- 172.21.15.99:0/931452477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa39c004810 0x7fa39c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:40.314 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.309+0000 7fa3a7fff700 1 --2- 172.21.15.99:0/931452477 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3b015cbd0 0x7fa39c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:40.314 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.309+0000 7fa3a7fff700 1 -- 172.21.15.99:0/931452477 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b015e850 con 0x7fa39c004810 2023-10-08T00:14:40.314 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.309+0000 7fa3a7fff700 1 -- 172.21.15.99:0/931452477 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39c00bb40 con 0x7fa39c003c20 2023-10-08T00:14:40.314 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.309+0000 7fa3a7fff700 1 -- 172.21.15.99:0/931452477 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa39c00bd00 con 0x7fa3b015cbd0 2023-10-08T00:14:40.314 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.309+0000 7fa3b4b23700 1 --2- 172.21.15.99:0/931452477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa39c004810 0x7fa39c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:40.314 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.309+0000 7fa3b5324700 1 --2- 172.21.15.99:0/931452477 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa39c003c20 0x7fa39c004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:40.314 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.309+0000 7fa3b5b25700 1 --2- 172.21.15.99:0/931452477 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3b015cbd0 0x7fa39c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:40.314 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.309+0000 7fa3b4b23700 1 -- 172.21.15.99:0/931452477 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3b015cbd0 msgr2=0x7fa39c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:40.314 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.309+0000 7fa3b4b23700 1 --2- 172.21.15.99:0/931452477 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3b015cbd0 0x7fa39c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:40.314 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.309+0000 7fa3b4b23700 1 -- 172.21.15.99:0/931452477 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa39c003c20 msgr2=0x7fa39c004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:40.314 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.309+0000 7fa3b4b23700 1 --2- 172.21.15.99:0/931452477 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa39c003c20 0x7fa39c004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:40.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.309+0000 7fa3b4b23700 1 -- 172.21.15.99:0/931452477 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa39c003570 con 0x7fa39c004810 2023-10-08T00:14:40.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.309+0000 7fa3b4b23700 1 --2- 172.21.15.99:0/931452477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa39c004810 0x7fa39c004c80 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0001a10 tx=0x7fa3a0007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b3ae60802bcf3e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:40.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.309+0000 7fa3a7fff700 1 -- 172.21.15.99:0/931452477 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a000c5d0 con 0x7fa39c004810 2023-10-08T00:14:40.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.309+0000 7fa3a7fff700 1 -- 172.21.15.99:0/931452477 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3a000c730 con 0x7fa39c004810 2023-10-08T00:14:40.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.309+0000 7fa3a7fff700 1 -- 172.21.15.99:0/931452477 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a000cad0 con 0x7fa39c004810 2023-10-08T00:14:40.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.310+0000 7fa3b6326700 1 -- 172.21.15.99:0/931452477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa39c004810 msgr2=0x7fa39c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:40.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.310+0000 7fa3b6326700 1 --2- 172.21.15.99:0/931452477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa39c004810 0x7fa39c004c80 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0001a10 tx=0x7fa3a0007d30 comp rx=0 tx=0).stop 2023-10-08T00:14:40.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.310+0000 7fa3b5b25700 1 -- 172.21.15.99:0/931452477 reap_dead start 2023-10-08T00:14:40.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.310+0000 7fa3b6326700 1 -- 172.21.15.99:0/931452477 shutdown_connections 2023-10-08T00:14:40.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.310+0000 7fa3b6326700 1 -- 172.21.15.99:0/931452477 >> 172.21.15.99:0/931452477 conn(0x7fa3b0067660 msgr2=0x7fa3b0067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:40.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.310+0000 7fa3b6326700 1 -- 172.21.15.99:0/931452477 shutdown_connections 2023-10-08T00:14:40.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.310+0000 7fa3b6326700 1 -- 172.21.15.99:0/931452477 wait complete. 2023-10-08T00:14:40.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.311+0000 7fa3b6326700 1 Processor -- start 2023-10-08T00:14:40.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.311+0000 7fa3b6326700 1 -- start start 2023-10-08T00:14:40.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.312+0000 7fa3b6326700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa39c004810 0x7fa3b0147560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:40.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.312+0000 7fa3b6326700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3b0159030 0x7fa3b006a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:40.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.312+0000 7fa3b6326700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3b01594a0 0x7fa3b006a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:40.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.312+0000 7fa3b6326700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b00cc360 con 0x7fa39c004810 2023-10-08T00:14:40.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.312+0000 7fa3b6326700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b015c220 con 0x7fa3b0159030 2023-10-08T00:14:40.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.312+0000 7fa3b6326700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b0158950 con 0x7fa3b01594a0 2023-10-08T00:14:40.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.312+0000 7fa3b5324700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa39c004810 0x7fa3b0147560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:40.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.312+0000 7fa3b5324700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa39c004810 0x7fa3b0147560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33742/0 (socket says 172.21.15.99:33742) 2023-10-08T00:14:40.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.312+0000 7fa3b5324700 1 -- 172.21.15.99:0/3037556062 learned_addr learned my addr 172.21.15.99:0/3037556062 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:40.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.312+0000 7fa3b4b23700 1 --2- 172.21.15.99:0/3037556062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3b0159030 0x7fa3b006a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:40.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.312+0000 7fa3b5b25700 1 --2- 172.21.15.99:0/3037556062 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3b01594a0 0x7fa3b006a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:40.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.312+0000 7fa3b5324700 1 -- 172.21.15.99:0/3037556062 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3b01594a0 msgr2=0x7fa3b006a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:40.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.312+0000 7fa3b5324700 1 --2- 172.21.15.99:0/3037556062 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3b01594a0 0x7fa3b006a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:40.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.312+0000 7fa3b5324700 1 -- 172.21.15.99:0/3037556062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3b0159030 msgr2=0x7fa3b006a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:40.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.312+0000 7fa3b5324700 1 --2- 172.21.15.99:0/3037556062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3b0159030 0x7fa3b006a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:40.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.312+0000 7fa3b5324700 1 -- 172.21.15.99:0/3037556062 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b00c7e50 con 0x7fa39c004810 2023-10-08T00:14:40.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.312+0000 7fa3b5b25700 1 --2- 172.21.15.99:0/3037556062 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3b01594a0 0x7fa3b006a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:14:40.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.313+0000 7fa3b5324700 1 --2- 172.21.15.99:0/3037556062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa39c004810 0x7fa3b0147560 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac0073c0 tx=0x7fa3ac001df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:40.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.313+0000 7fa3a57fa700 1 -- 172.21.15.99:0/3037556062 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac018030 con 0x7fa39c004810 2023-10-08T00:14:40.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.313+0000 7fa3a57fa700 1 -- 172.21.15.99:0/3037556062 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3ac007780 con 0x7fa39c004810 2023-10-08T00:14:40.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.313+0000 7fa3b6326700 1 -- 172.21.15.99:0/3037556062 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3b015b980 con 0x7fa39c004810 2023-10-08T00:14:40.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.313+0000 7fa3a57fa700 1 -- 172.21.15.99:0/3037556062 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac00fb00 con 0x7fa39c004810 2023-10-08T00:14:40.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.313+0000 7fa3b6326700 1 -- 172.21.15.99:0/3037556062 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3b015bf40 con 0x7fa39c004810 2023-10-08T00:14:40.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.314+0000 7fa3b6326700 1 -- 172.21.15.99:0/3037556062 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3b0089280 con 0x7fa39c004810 2023-10-08T00:14:40.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.314+0000 7fa3a57fa700 1 -- 172.21.15.99:0/3037556062 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa3ac01b070 con 0x7fa39c004810 2023-10-08T00:14:40.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.314+0000 7fa3a57fa700 1 --2- 172.21.15.99:0/3037556062 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa394066f30 0x7fa3940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:40.321 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.314+0000 7fa3a57fa700 1 -- 172.21.15.99:0/3037556062 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fa3ac019070 con 0x7fa39c004810 2023-10-08T00:14:40.321 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.315+0000 7fa3b4b23700 1 --2- 172.21.15.99:0/3037556062 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa394066f30 0x7fa3940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:40.321 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.315+0000 7fa3b4b23700 1 --2- 172.21.15.99:0/3037556062 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa394066f30 0x7fa3940693e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0008fd0 tx=0x7fa3a0015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:40.321 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.318+0000 7fa3a57fa700 1 -- 172.21.15.99:0/3037556062 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa3ac0515a0 con 0x7fa39c004810 2023-10-08T00:14:40.503 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.503+0000 7fa3b6326700 1 -- 172.21.15.99:0/3037556062 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa3b00c7fe0 con 0x7fa39c004810 2023-10-08T00:14:40.504 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.503+0000 7fa3a57fa700 1 -- 172.21.15.99:0/3037556062 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+12134 (secure 0 0 0) 0x7fa3ac054bc0 con 0x7fa39c004810 2023-10-08T00:14:40.504 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:14:40.504 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":82,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:14:39.754902+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:14:40.506 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.505+0000 7fa3b6326700 1 -- 172.21.15.99:0/3037556062 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa394066f30 msgr2=0x7fa3940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:40.506 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.505+0000 7fa3b6326700 1 --2- 172.21.15.99:0/3037556062 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa394066f30 0x7fa3940693e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0008fd0 tx=0x7fa3a0015000 comp rx=0 tx=0).stop 2023-10-08T00:14:40.507 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.505+0000 7fa3b6326700 1 -- 172.21.15.99:0/3037556062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa39c004810 msgr2=0x7fa3b0147560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:40.507 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.505+0000 7fa3b6326700 1 --2- 172.21.15.99:0/3037556062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa39c004810 0x7fa3b0147560 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac0073c0 tx=0x7fa3ac001df0 comp rx=0 tx=0).stop 2023-10-08T00:14:40.507 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.506+0000 7fa3b6326700 1 -- 172.21.15.99:0/3037556062 shutdown_connections 2023-10-08T00:14:40.507 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.506+0000 7fa3b6326700 1 --2- 172.21.15.99:0/3037556062 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa394066f30 0x7fa3940693e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:40.507 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.506+0000 7fa3b6326700 1 --2- 172.21.15.99:0/3037556062 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3b01594a0 0x7fa3b006a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:40.507 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.506+0000 7fa3b6326700 1 --2- 172.21.15.99:0/3037556062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa39c004810 0x7fa3b0147560 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:40.507 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.506+0000 7fa3b6326700 1 --2- 172.21.15.99:0/3037556062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3b0159030 0x7fa3b006a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:40.507 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.506+0000 7fa3b6326700 1 -- 172.21.15.99:0/3037556062 >> 172.21.15.99:0/3037556062 conn(0x7fa3b0067660 msgr2=0x7fa3b00b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:40.507 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.506+0000 7fa3b6326700 1 -- 172.21.15.99:0/3037556062 shutdown_connections 2023-10-08T00:14:40.507 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.506+0000 7fa3b6326700 1 -- 172.21.15.99:0/3037556062 wait complete. 2023-10-08T00:14:40.518 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:10:42.063344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-08T00:14:40.519 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T00:14:40.635 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.634+0000 7f2a3f32e700 1 Processor -- start 2023-10-08T00:14:40.635 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.635+0000 7f2a3f32e700 1 -- start start 2023-10-08T00:14:40.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.635+0000 7f2a3f32e700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2a38159ec0 0x7f2a380af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:40.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.635+0000 7f2a3f32e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2a3815a290 0x7f2a380afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:40.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.635+0000 7f2a3f32e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a380b0790 con 0x7f2a3806a5a0 2023-10-08T00:14:40.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.635+0000 7f2a3f32e700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a380ce1f0 con 0x7f2a38159ec0 2023-10-08T00:14:40.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.635+0000 7f2a3f32e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a380b0b20 con 0x7f2a3815a290 2023-10-08T00:14:40.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.635+0000 7f2a3d0ca700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2a38159ec0 0x7f2a380af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:40.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.636+0000 7f2a3d0ca700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2a38159ec0 0x7f2a380af940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33748/0 (socket says 172.21.15.99:33748) 2023-10-08T00:14:40.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.636+0000 7f2a3d0ca700 1 -- 172.21.15.99:0/3996118778 learned_addr learned my addr 172.21.15.99:0/3996118778 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:40.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.636+0000 7f2a3d0ca700 1 -- 172.21.15.99:0/3996118778 >> v2:172.21.15.191:3300/0 conn(0x7f2a3815a290 msgr2=0x7f2a380afe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:40.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.636+0000 7f2a3c8c9700 1 --2- 172.21.15.99:0/3996118778 >> v2:172.21.15.191:3300/0 conn(0x7f2a3815a290 0x7f2a380afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:40.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.636+0000 7f2a3d0ca700 1 --2- 172.21.15.99:0/3996118778 >> v2:172.21.15.191:3300/0 conn(0x7f2a3815a290 0x7f2a380afe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:40.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.636+0000 7f2a3d0ca700 1 -- 172.21.15.99:0/3996118778 >> v1:172.21.15.191:6789/0 conn(0x7f2a3806a5a0 legacy=0x7f2a3815db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:40.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.636+0000 7f2a3d0ca700 1 -- 172.21.15.99:0/3996118778 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a380b04b0 con 0x7f2a38159ec0 2023-10-08T00:14:40.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.636+0000 7f2a3c8c9700 1 --2- 172.21.15.99:0/3996118778 >> v2:172.21.15.191:3300/0 conn(0x7f2a3815a290 0x7f2a380afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:14:40.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.636+0000 7f2a3d0ca700 1 --2- 172.21.15.99:0/3996118778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a38159ec0 0x7f2a380af940 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f2a38089910 tx=0x7f2a34009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e4ee77c437c39b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:40.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.636+0000 7f2a2ffff700 1 -- 172.21.15.99:0/3996118778 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3400b820 con 0x7f2a38159ec0 2023-10-08T00:14:40.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.636+0000 7f2a2ffff700 1 -- 172.21.15.99:0/3996118778 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a3400b980 con 0x7f2a38159ec0 2023-10-08T00:14:40.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.636+0000 7f2a2ffff700 1 -- 172.21.15.99:0/3996118778 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3400bd20 con 0x7f2a38159ec0 2023-10-08T00:14:40.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.637+0000 7f2a3f32e700 1 -- 172.21.15.99:0/3996118778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a38159ec0 msgr2=0x7f2a380af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:40.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.637+0000 7f2a3f32e700 1 --2- 172.21.15.99:0/3996118778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a38159ec0 0x7f2a380af940 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f2a38089910 tx=0x7f2a34009250 comp rx=0 tx=0).stop 2023-10-08T00:14:40.640 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.637+0000 7f2a3f32e700 1 -- 172.21.15.99:0/3996118778 shutdown_connections 2023-10-08T00:14:40.640 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.637+0000 7f2a3f32e700 1 --2- 172.21.15.99:0/3996118778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a38159ec0 0x7f2a380af940 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:40.640 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.637+0000 7f2a3f32e700 1 --2- 172.21.15.99:0/3996118778 >> v2:172.21.15.191:3300/0 conn(0x7f2a3815a290 0x7f2a380afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:40.640 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.637+0000 7f2a3f32e700 1 -- 172.21.15.99:0/3996118778 >> 172.21.15.99:0/3996118778 conn(0x7f2a380d1880 msgr2=0x7f2a380cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:40.640 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.637+0000 7f2a3f32e700 1 -- 172.21.15.99:0/3996118778 shutdown_connections 2023-10-08T00:14:40.640 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.637+0000 7f2a3f32e700 1 -- 172.21.15.99:0/3996118778 wait complete. 2023-10-08T00:14:40.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.638+0000 7f2a3f32e700 1 Processor -- start 2023-10-08T00:14:40.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.638+0000 7f2a3f32e700 1 -- start start 2023-10-08T00:14:40.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.639+0000 7f2a3f32e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a3806a5a0 0x7f2a381544b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:40.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.639+0000 7f2a3f32e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a38159ec0 0x7f2a3816ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:40.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.639+0000 7f2a3f32e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a3815a290 0x7f2a381571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:40.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.639+0000 7f2a3f32e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a380c9c10 con 0x7f2a3806a5a0 2023-10-08T00:14:40.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.639+0000 7f2a3f32e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a380ca4b0 con 0x7f2a3815a290 2023-10-08T00:14:40.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.639+0000 7f2a3f32e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a380c9720 con 0x7f2a38159ec0 2023-10-08T00:14:40.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.639+0000 7f2a3d0ca700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a3806a5a0 0x7f2a381544b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:40.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.639+0000 7f2a3d0ca700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a3806a5a0 0x7f2a381544b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33758/0 (socket says 172.21.15.99:33758) 2023-10-08T00:14:40.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.639+0000 7f2a3d0ca700 1 -- 172.21.15.99:0/1858677031 learned_addr learned my addr 172.21.15.99:0/1858677031 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:40.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.639+0000 7f2a3d8cb700 1 --2- 172.21.15.99:0/1858677031 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a3815a290 0x7f2a381571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:40.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.639+0000 7f2a3d0ca700 1 -- 172.21.15.99:0/1858677031 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a38159ec0 msgr2=0x7f2a3816ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:40.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.639+0000 7f2a3c8c9700 1 --2- 172.21.15.99:0/1858677031 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a38159ec0 0x7f2a3816ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:40.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.639+0000 7f2a3d0ca700 1 --2- 172.21.15.99:0/1858677031 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a38159ec0 0x7f2a3816ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:40.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.639+0000 7f2a3d0ca700 1 -- 172.21.15.99:0/1858677031 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a3815a290 msgr2=0x7f2a381571e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:40.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.639+0000 7f2a3d0ca700 1 --2- 172.21.15.99:0/1858677031 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a3815a290 0x7f2a381571e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:40.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.639+0000 7f2a3d0ca700 1 -- 172.21.15.99:0/1858677031 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a340094f0 con 0x7f2a3806a5a0 2023-10-08T00:14:40.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.639+0000 7f2a3d8cb700 1 --2- 172.21.15.99:0/1858677031 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a3815a290 0x7f2a381571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:14:40.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.639+0000 7f2a3c8c9700 1 --2- 172.21.15.99:0/1858677031 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a38159ec0 0x7f2a3816ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:14:40.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.639+0000 7f2a3d0ca700 1 --2- 172.21.15.99:0/1858677031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a3806a5a0 0x7f2a381544b0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f2a38071eb0 tx=0x7f2a34016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:40.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.639+0000 7f2a2d7fa700 1 -- 172.21.15.99:0/1858677031 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a34011e20 con 0x7f2a3806a5a0 2023-10-08T00:14:40.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.639+0000 7f2a2d7fa700 1 -- 172.21.15.99:0/1858677031 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a3401b680 con 0x7f2a3806a5a0 2023-10-08T00:14:40.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.639+0000 7f2a3f32e700 1 -- 172.21.15.99:0/1858677031 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a3816b1b0 con 0x7f2a3806a5a0 2023-10-08T00:14:40.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.639+0000 7f2a2d7fa700 1 -- 172.21.15.99:0/1858677031 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3401ba40 con 0x7f2a3806a5a0 2023-10-08T00:14:40.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.639+0000 7f2a3f32e700 1 -- 172.21.15.99:0/1858677031 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a380c9e60 con 0x7f2a3806a5a0 2023-10-08T00:14:40.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.640+0000 7f2a3f32e700 1 -- 172.21.15.99:0/1858677031 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a3804c3c0 con 0x7f2a3806a5a0 2023-10-08T00:14:40.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.641+0000 7f2a2d7fa700 1 -- 172.21.15.99:0/1858677031 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2a3401c450 con 0x7f2a3806a5a0 2023-10-08T00:14:40.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.641+0000 7f2a2d7fa700 1 --2- 172.21.15.99:0/1858677031 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2a1c066f30 0x7f2a1c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:40.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.641+0000 7f2a2d7fa700 1 -- 172.21.15.99:0/1858677031 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f2a34085f00 con 0x7f2a3806a5a0 2023-10-08T00:14:40.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.641+0000 7f2a3c8c9700 1 --2- 172.21.15.99:0/1858677031 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2a1c066f30 0x7f2a1c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:40.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.642+0000 7f2a3c8c9700 1 --2- 172.21.15.99:0/1858677031 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2a1c066f30 0x7f2a1c0693e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2a28006fd0 tx=0x7f2a28008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:40.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.645+0000 7f2a2d7fa700 1 -- 172.21.15.99:0/1858677031 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2a340534c0 con 0x7f2a3806a5a0 2023-10-08T00:14:40.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.827+0000 7f2a3f32e700 1 -- 172.21.15.99:0/1858677031 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2a380ceab0 con 0x7f2a3806a5a0 2023-10-08T00:14:40.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.827+0000 7f2a2d7fa700 1 -- 172.21.15.99:0/1858677031 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2a3402a020 con 0x7f2a3806a5a0 2023-10-08T00:14:40.828 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2023-10-08T00:14:40.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.830+0000 7f2a3f32e700 1 -- 172.21.15.99:0/1858677031 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2a1c066f30 msgr2=0x7f2a1c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:40.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.830+0000 7f2a3f32e700 1 --2- 172.21.15.99:0/1858677031 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2a1c066f30 0x7f2a1c0693e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2a28006fd0 tx=0x7f2a28008040 comp rx=0 tx=0).stop 2023-10-08T00:14:40.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.830+0000 7f2a3f32e700 1 -- 172.21.15.99:0/1858677031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a3806a5a0 msgr2=0x7f2a381544b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:40.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.830+0000 7f2a3f32e700 1 --2- 172.21.15.99:0/1858677031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a3806a5a0 0x7f2a381544b0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f2a38071eb0 tx=0x7f2a34016000 comp rx=0 tx=0).stop 2023-10-08T00:14:40.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.830+0000 7f2a3f32e700 1 -- 172.21.15.99:0/1858677031 shutdown_connections 2023-10-08T00:14:40.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.830+0000 7f2a3f32e700 1 --2- 172.21.15.99:0/1858677031 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2a1c066f30 0x7f2a1c0693e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:40.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.830+0000 7f2a3f32e700 1 --2- 172.21.15.99:0/1858677031 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a3815a290 0x7f2a381571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:40.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.830+0000 7f2a3f32e700 1 --2- 172.21.15.99:0/1858677031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a3806a5a0 0x7f2a381544b0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:40.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.830+0000 7f2a3f32e700 1 --2- 172.21.15.99:0/1858677031 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a38159ec0 0x7f2a3816ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:40.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.830+0000 7f2a3f32e700 1 -- 172.21.15.99:0/1858677031 >> 172.21.15.99:0/1858677031 conn(0x7f2a380d1880 msgr2=0x7f2a381556d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:40.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.830+0000 7f2a3f32e700 1 -- 172.21.15.99:0/1858677031 shutdown_connections 2023-10-08T00:14:40.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.830+0000 7f2a3f32e700 1 -- 172.21.15.99:0/1858677031 wait complete. 2023-10-08T00:14:40.842 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T00:14:40.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.955+0000 7fdd10bcb700 1 Processor -- start 2023-10-08T00:14:40.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.955+0000 7fdd10bcb700 1 -- start start 2023-10-08T00:14:40.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.956+0000 7fdd10bcb700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdd0c152010 0x7fdd0c15b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:40.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.956+0000 7fdd10bcb700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fdd0c1523e0 0x7fdd0c06c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:40.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.956+0000 7fdd10bcb700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd0c06c010 con 0x7fdd0c158e50 2023-10-08T00:14:40.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.956+0000 7fdd10bcb700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd0c06bd10 con 0x7fdd0c152010 2023-10-08T00:14:40.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.956+0000 7fdd10bcb700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd0c15c060 con 0x7fdd0c1523e0 2023-10-08T00:14:40.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.956+0000 7fdd0a59c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdd0c152010 0x7fdd0c15b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:40.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.956+0000 7fdd0ad9d700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fdd0c158e50 0x7fdd0c0cdf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:41764/0 (socket says 172.21.15.99:41764) 2023-10-08T00:14:40.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.956+0000 7fdd0ad9d700 1 -- 172.21.15.99:0/813840380 learned_addr learned my addr 172.21.15.99:0/813840380 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:14:40.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.956+0000 7fdd0a59c700 1 -- 172.21.15.99:0/813840380 >> v2:172.21.15.191:3300/0 conn(0x7fdd0c1523e0 msgr2=0x7fdd0c06c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:40.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.956+0000 7fdd0a59c700 1 --2- 172.21.15.99:0/813840380 >> v2:172.21.15.191:3300/0 conn(0x7fdd0c1523e0 0x7fdd0c06c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:40.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.956+0000 7fdd0a59c700 1 -- 172.21.15.99:0/813840380 >> v1:172.21.15.99:6789/0 conn(0x7fdd0c158e50 legacy=0x7fdd0c0cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:40.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.956+0000 7fdd0a59c700 1 -- 172.21.15.99:0/813840380 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd0c15c340 con 0x7fdd0c152010 2023-10-08T00:14:40.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.957+0000 7fdd0a59c700 1 --2- 172.21.15.99:0/813840380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd0c152010 0x7fdd0c15b810 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c0ceae0 tx=0x7fdcfc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16873e8c381a52e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:40.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.957+0000 7fdd09d9b700 1 -- 172.21.15.99:0/813840380 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc011630 con 0x7fdd0c152010 2023-10-08T00:14:40.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.957+0000 7fdd09d9b700 1 -- 172.21.15.99:0/813840380 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcfc011790 con 0x7fdd0c152010 2023-10-08T00:14:40.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.957+0000 7fdd09d9b700 1 -- 172.21.15.99:0/813840380 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc011b30 con 0x7fdd0c152010 2023-10-08T00:14:40.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.957+0000 7fdd10bcb700 1 -- 172.21.15.99:0/813840380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd0c152010 msgr2=0x7fdd0c15b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:40.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.957+0000 7fdd10bcb700 1 --2- 172.21.15.99:0/813840380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd0c152010 0x7fdd0c15b810 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c0ceae0 tx=0x7fdcfc0091a0 comp rx=0 tx=0).stop 2023-10-08T00:14:40.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.957+0000 7fdd10bcb700 1 -- 172.21.15.99:0/813840380 shutdown_connections 2023-10-08T00:14:40.960 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.957+0000 7fdd10bcb700 1 --2- 172.21.15.99:0/813840380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd0c152010 0x7fdd0c15b810 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:40.960 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.957+0000 7fdd10bcb700 1 --2- 172.21.15.99:0/813840380 >> v2:172.21.15.191:3300/0 conn(0x7fdd0c1523e0 0x7fdd0c06c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:40.960 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.957+0000 7fdd10bcb700 1 -- 172.21.15.99:0/813840380 >> 172.21.15.99:0/813840380 conn(0x7fdd0c069870 msgr2=0x7fdd0c069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:40.960 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.957+0000 7fdd10bcb700 1 -- 172.21.15.99:0/813840380 shutdown_connections 2023-10-08T00:14:40.960 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.957+0000 7fdd10bcb700 1 -- 172.21.15.99:0/813840380 wait complete. 2023-10-08T00:14:40.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.959+0000 7fdd10bcb700 1 Processor -- start 2023-10-08T00:14:40.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.959+0000 7fdd10bcb700 1 -- start start 2023-10-08T00:14:40.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.959+0000 7fdd10bcb700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd0c152010 0x7fdd0c0c5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:40.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.959+0000 7fdd10bcb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd0c1523e0 0x7fdd0c0c6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:40.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.959+0000 7fdd10bcb700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd0c158e50 0x7fdd0c0c7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:40.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.959+0000 7fdd10bcb700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0c0b21e0 con 0x7fdd0c1523e0 2023-10-08T00:14:40.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.959+0000 7fdd10bcb700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0c0b24c0 con 0x7fdd0c158e50 2023-10-08T00:14:40.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.959+0000 7fdd10bcb700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0c0b27a0 con 0x7fdd0c152010 2023-10-08T00:14:40.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.959+0000 7fdd01d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd0c1523e0 0x7fdd0c0c6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:40.964 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.959+0000 7fdd01d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd0c1523e0 0x7fdd0c0c6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33776/0 (socket says 172.21.15.99:33776) 2023-10-08T00:14:40.964 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.959+0000 7fdd01d9b700 1 -- 172.21.15.99:0/4271720910 learned_addr learned my addr 172.21.15.99:0/4271720910 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:40.964 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.960+0000 7fdd0ad9d700 1 --2- 172.21.15.99:0/4271720910 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd0c158e50 0x7fdd0c0c7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:40.964 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.960+0000 7fdd01d9b700 1 -- 172.21.15.99:0/4271720910 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd0c152010 msgr2=0x7fdd0c0c5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:40.964 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.960+0000 7fdd0a59c700 1 --2- 172.21.15.99:0/4271720910 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd0c152010 0x7fdd0c0c5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:40.964 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.960+0000 7fdd01d9b700 1 --2- 172.21.15.99:0/4271720910 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd0c152010 0x7fdd0c0c5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:40.964 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.960+0000 7fdd01d9b700 1 -- 172.21.15.99:0/4271720910 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd0c158e50 msgr2=0x7fdd0c0c7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:40.964 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.960+0000 7fdd01d9b700 1 --2- 172.21.15.99:0/4271720910 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd0c158e50 0x7fdd0c0c7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:40.964 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.960+0000 7fdd01d9b700 1 -- 172.21.15.99:0/4271720910 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcfc009420 con 0x7fdd0c1523e0 2023-10-08T00:14:40.964 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.960+0000 7fdd0ad9d700 1 --2- 172.21.15.99:0/4271720910 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd0c158e50 0x7fdd0c0c7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:14:40.964 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.960+0000 7fdd01d9b700 1 --2- 172.21.15.99:0/4271720910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd0c1523e0 0x7fdd0c0c6130 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fdcf400b810 tx=0x7fdcf4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:40.965 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.960+0000 7fdd02ffd700 1 -- 172.21.15.99:0/4271720910 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf400f5f0 con 0x7fdd0c1523e0 2023-10-08T00:14:40.965 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.960+0000 7fdd02ffd700 1 -- 172.21.15.99:0/4271720910 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcf400f770 con 0x7fdd0c1523e0 2023-10-08T00:14:40.965 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.960+0000 7fdd10bcb700 1 -- 172.21.15.99:0/4271720910 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd0c0b2a80 con 0x7fdd0c1523e0 2023-10-08T00:14:40.965 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.960+0000 7fdd02ffd700 1 -- 172.21.15.99:0/4271720910 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf400fb30 con 0x7fdd0c1523e0 2023-10-08T00:14:40.965 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.960+0000 7fdd10bcb700 1 -- 172.21.15.99:0/4271720910 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd0c0c82f0 con 0x7fdd0c1523e0 2023-10-08T00:14:40.965 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.961+0000 7fdd10bcb700 1 -- 172.21.15.99:0/4271720910 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd0c0b2c70 con 0x7fdd0c1523e0 2023-10-08T00:14:40.965 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.961+0000 7fdd02ffd700 1 -- 172.21.15.99:0/4271720910 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdcf4010480 con 0x7fdd0c1523e0 2023-10-08T00:14:40.965 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.962+0000 7fdd02ffd700 1 --2- 172.21.15.99:0/4271720910 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdcec066f30 0x7fdcec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:40.965 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.962+0000 7fdd02ffd700 1 -- 172.21.15.99:0/4271720910 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fdcf4084140 con 0x7fdd0c1523e0 2023-10-08T00:14:40.965 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.962+0000 7fdd0a59c700 1 --2- 172.21.15.99:0/4271720910 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdcec066f30 0x7fdcec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:40.965 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.962+0000 7fdd0a59c700 1 --2- 172.21.15.99:0/4271720910 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdcec066f30 0x7fdcec0693e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c0cc3f0 tx=0x7fdcfc016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:40.966 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:40.966+0000 7fdd02ffd700 1 -- 172.21.15.99:0/4271720910 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdcf4051780 con 0x7fdd0c1523e0 2023-10-08T00:14:41.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.151+0000 7fdd10bcb700 1 -- 172.21.15.99:0/4271720910 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fdd0c0b2320 con 0x7fdd0c1523e0 2023-10-08T00:14:41.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.151+0000 7fdd02ffd700 1 -- 172.21.15.99:0/4271720910 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) v1 ==== 84+0+65 (secure 0 0 0) 0x7fdd0c0b2320 con 0x7fdd0c1523e0 2023-10-08T00:14:41.152 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:14:41.152 INFO:teuthology.orchestra.run.smithi099.stdout:[ 2023-10-08T00:14:41.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:14:41.152 INFO:teuthology.orchestra.run.smithi099.stdout: "poolnum": 1, 2023-10-08T00:14:41.152 INFO:teuthology.orchestra.run.smithi099.stdout: "poolname": ".mgr" 2023-10-08T00:14:41.152 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:14:41.153 INFO:teuthology.orchestra.run.smithi099.stdout:] 2023-10-08T00:14:41.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.153+0000 7fdd10bcb700 1 -- 172.21.15.99:0/4271720910 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdcec066f30 msgr2=0x7fdcec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:41.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.153+0000 7fdd10bcb700 1 --2- 172.21.15.99:0/4271720910 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdcec066f30 0x7fdcec0693e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c0cc3f0 tx=0x7fdcfc016000 comp rx=0 tx=0).stop 2023-10-08T00:14:41.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.153+0000 7fdd10bcb700 1 -- 172.21.15.99:0/4271720910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd0c1523e0 msgr2=0x7fdd0c0c6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:41.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.153+0000 7fdd10bcb700 1 --2- 172.21.15.99:0/4271720910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd0c1523e0 0x7fdd0c0c6130 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fdcf400b810 tx=0x7fdcf4009250 comp rx=0 tx=0).stop 2023-10-08T00:14:41.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.153+0000 7fdd10bcb700 1 -- 172.21.15.99:0/4271720910 shutdown_connections 2023-10-08T00:14:41.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.153+0000 7fdd10bcb700 1 --2- 172.21.15.99:0/4271720910 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdcec066f30 0x7fdcec0693e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:41.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.153+0000 7fdd10bcb700 1 --2- 172.21.15.99:0/4271720910 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd0c158e50 0x7fdd0c0c7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:41.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.153+0000 7fdd10bcb700 1 --2- 172.21.15.99:0/4271720910 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd0c152010 0x7fdd0c0c5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:41.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.153+0000 7fdd10bcb700 1 --2- 172.21.15.99:0/4271720910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd0c1523e0 0x7fdd0c0c6130 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:41.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.153+0000 7fdd10bcb700 1 -- 172.21.15.99:0/4271720910 >> 172.21.15.99:0/4271720910 conn(0x7fdd0c069870 msgr2=0x7fdd0c151440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:41.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.153+0000 7fdd10bcb700 1 -- 172.21.15.99:0/4271720910 shutdown_connections 2023-10-08T00:14:41.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.153+0000 7fdd10bcb700 1 -- 172.21.15.99:0/4271720910 wait complete. 2023-10-08T00:14:41.166 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T00:14:41.166 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T00:14:41.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.283+0000 7f755d630700 1 Processor -- start 2023-10-08T00:14:41.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.283+0000 7f755d630700 1 -- start start 2023-10-08T00:14:41.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.284+0000 7f755d630700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7558159ef0 0x7f755815a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:41.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.284+0000 7f755d630700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f75580b6a10 0x7f75580b10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:41.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.284+0000 7f755d630700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75580b1a60 con 0x7f75580b15e0 2023-10-08T00:14:41.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.284+0000 7f755d630700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75580b1d40 con 0x7f75580b6a10 2023-10-08T00:14:41.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.284+0000 7f755d630700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75580b2020 con 0x7f7558159ef0 2023-10-08T00:14:41.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.284+0000 7f754ffff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f75580b6a10 0x7f75580b10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:41.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.284+0000 7f754ffff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f75580b6a10 0x7f75580b10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33786/0 (socket says 172.21.15.99:33786) 2023-10-08T00:14:41.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.284+0000 7f754ffff700 1 -- 172.21.15.99:0/4247561338 learned_addr learned my addr 172.21.15.99:0/4247561338 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:41.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.284+0000 7f7556ffd700 1 --2- 172.21.15.99:0/4247561338 >> v2:172.21.15.191:3300/0 conn(0x7f7558159ef0 0x7f755815a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:41.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.284+0000 7f754ffff700 1 -- 172.21.15.99:0/4247561338 >> v2:172.21.15.191:3300/0 conn(0x7f7558159ef0 msgr2=0x7f755815a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:41.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.284+0000 7f754ffff700 1 --2- 172.21.15.99:0/4247561338 >> v2:172.21.15.191:3300/0 conn(0x7f7558159ef0 0x7f755815a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:41.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.284+0000 7f754ffff700 1 -- 172.21.15.99:0/4247561338 >> v1:172.21.15.191:6789/0 conn(0x7f75580b15e0 legacy=0x7f75580b46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:41.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.284+0000 7f754ffff700 1 -- 172.21.15.99:0/4247561338 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75580b2300 con 0x7f75580b6a10 2023-10-08T00:14:41.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.284+0000 7f7556ffd700 1 --2- 172.21.15.99:0/4247561338 >> v2:172.21.15.191:3300/0 conn(0x7f7558159ef0 0x7f755815a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:14:41.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.285+0000 7f754ffff700 1 --2- 172.21.15.99:0/4247561338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75580b6a10 0x7f75580b10a0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f755808a710 tx=0x7f7540009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32f5f086152cd04e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:41.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.285+0000 7f75567fc700 1 -- 172.21.15.99:0/4247561338 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754000b860 con 0x7f75580b6a10 2023-10-08T00:14:41.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.285+0000 7f75567fc700 1 -- 172.21.15.99:0/4247561338 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f754000b9c0 con 0x7f75580b6a10 2023-10-08T00:14:41.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.285+0000 7f75567fc700 1 -- 172.21.15.99:0/4247561338 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754000bd60 con 0x7f75580b6a10 2023-10-08T00:14:41.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.285+0000 7f755d630700 1 -- 172.21.15.99:0/4247561338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75580b6a10 msgr2=0x7f75580b10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:41.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.285+0000 7f755d630700 1 --2- 172.21.15.99:0/4247561338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75580b6a10 0x7f75580b10a0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f755808a710 tx=0x7f7540009250 comp rx=0 tx=0).stop 2023-10-08T00:14:41.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.285+0000 7f755d630700 1 -- 172.21.15.99:0/4247561338 shutdown_connections 2023-10-08T00:14:41.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.285+0000 7f755d630700 1 --2- 172.21.15.99:0/4247561338 >> v2:172.21.15.191:3300/0 conn(0x7f7558159ef0 0x7f755815a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:41.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.285+0000 7f755d630700 1 --2- 172.21.15.99:0/4247561338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75580b6a10 0x7f75580b10a0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:41.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.285+0000 7f755d630700 1 -- 172.21.15.99:0/4247561338 >> 172.21.15.99:0/4247561338 conn(0x7f755809a0a0 msgr2=0x7f7558067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:41.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.285+0000 7f755d630700 1 -- 172.21.15.99:0/4247561338 shutdown_connections 2023-10-08T00:14:41.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.285+0000 7f755d630700 1 -- 172.21.15.99:0/4247561338 wait complete. 2023-10-08T00:14:41.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.286+0000 7f755d630700 1 Processor -- start 2023-10-08T00:14:41.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.286+0000 7f755d630700 1 -- start start 2023-10-08T00:14:41.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.287+0000 7f755d630700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f75580b6a10 0x7f7558153990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:41.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.287+0000 7f755d630700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7558159ef0 0x7f755815adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:41.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.287+0000 7f755d630700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f755815b300 0x7f75581511f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:41.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.287+0000 7f755d630700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755815b7a0 con 0x7f7558159ef0 2023-10-08T00:14:41.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.287+0000 7f755d630700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7558151f80 con 0x7f75580b6a10 2023-10-08T00:14:41.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.287+0000 7f755d630700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7558152260 con 0x7f755815b300 2023-10-08T00:14:41.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.287+0000 7f754ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7558159ef0 0x7f755815adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:41.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.287+0000 7f754ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7558159ef0 0x7f755815adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33794/0 (socket says 172.21.15.99:33794) 2023-10-08T00:14:41.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.287+0000 7f754ffff700 1 -- 172.21.15.99:0/591401040 learned_addr learned my addr 172.21.15.99:0/591401040 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:41.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.287+0000 7f7556ffd700 1 --2- 172.21.15.99:0/591401040 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f75580b6a10 0x7f7558153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:41.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.287+0000 7f75577fe700 1 --2- 172.21.15.99:0/591401040 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f755815b300 0x7f75581511f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:41.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.288+0000 7f754ffff700 1 -- 172.21.15.99:0/591401040 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f755815b300 msgr2=0x7f75581511f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:41.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.288+0000 7f754ffff700 1 --2- 172.21.15.99:0/591401040 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f755815b300 0x7f75581511f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:41.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.288+0000 7f754ffff700 1 -- 172.21.15.99:0/591401040 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f75580b6a10 msgr2=0x7f7558153990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:41.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.288+0000 7f754ffff700 1 --2- 172.21.15.99:0/591401040 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f75580b6a10 0x7f7558153990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:41.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.288+0000 7f754ffff700 1 -- 172.21.15.99:0/591401040 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75400094f0 con 0x7f7558159ef0 2023-10-08T00:14:41.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.288+0000 7f7556ffd700 1 --2- 172.21.15.99:0/591401040 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f75580b6a10 0x7f7558153990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:14:41.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.288+0000 7f754ffff700 1 --2- 172.21.15.99:0/591401040 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7558159ef0 0x7f755815adc0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f7540009290 tx=0x7f7540017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:41.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.288+0000 7f754effd700 1 -- 172.21.15.99:0/591401040 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7540011e60 con 0x7f7558159ef0 2023-10-08T00:14:41.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.288+0000 7f754effd700 1 -- 172.21.15.99:0/591401040 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f754001f070 con 0x7f7558159ef0 2023-10-08T00:14:41.294 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.288+0000 7f755d630700 1 -- 172.21.15.99:0/591401040 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7558152540 con 0x7f7558159ef0 2023-10-08T00:14:41.294 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.288+0000 7f754effd700 1 -- 172.21.15.99:0/591401040 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754001a870 con 0x7f7558159ef0 2023-10-08T00:14:41.294 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.288+0000 7f755d630700 1 -- 172.21.15.99:0/591401040 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7558152b00 con 0x7f7558159ef0 2023-10-08T00:14:41.294 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.289+0000 7f755d630700 1 -- 172.21.15.99:0/591401040 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75580abd50 con 0x7f7558159ef0 2023-10-08T00:14:41.294 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.289+0000 7f754effd700 1 -- 172.21.15.99:0/591401040 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f754001abe0 con 0x7f7558159ef0 2023-10-08T00:14:41.294 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.289+0000 7f754effd700 1 --2- 172.21.15.99:0/591401040 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7538066f30 0x7f75380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:41.294 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.290+0000 7f754effd700 1 -- 172.21.15.99:0/591401040 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f7540021070 con 0x7f7558159ef0 2023-10-08T00:14:41.294 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.290+0000 7f7556ffd700 1 --2- 172.21.15.99:0/591401040 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7538066f30 0x7f75380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:41.294 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.290+0000 7f7556ffd700 1 --2- 172.21.15.99:0/591401040 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7538066f30 0x7f75380693e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7548006ea0 tx=0x7f7548008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:41.295 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.294+0000 7f754effd700 1 -- 172.21.15.99:0/591401040 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f75400537f0 con 0x7f7558159ef0 2023-10-08T00:14:41.479 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:41.478+0000 7f755d630700 1 -- 172.21.15.99:0/591401040 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f7558089a40 con 0x7f7558159ef0 2023-10-08T00:14:42.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.285+0000 7f754effd700 1 -- 172.21.15.99:0/591401040 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7540056e10 con 0x7f7558159ef0 2023-10-08T00:14:42.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.289+0000 7f755d630700 1 -- 172.21.15.99:0/591401040 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7538066f30 msgr2=0x7f75380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:42.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.289+0000 7f755d630700 1 --2- 172.21.15.99:0/591401040 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7538066f30 0x7f75380693e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7548006ea0 tx=0x7f7548008040 comp rx=0 tx=0).stop 2023-10-08T00:14:42.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.289+0000 7f755d630700 1 -- 172.21.15.99:0/591401040 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7558159ef0 msgr2=0x7f755815adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:42.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.289+0000 7f755d630700 1 --2- 172.21.15.99:0/591401040 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7558159ef0 0x7f755815adc0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f7540009290 tx=0x7f7540017000 comp rx=0 tx=0).stop 2023-10-08T00:14:42.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.289+0000 7f755d630700 1 -- 172.21.15.99:0/591401040 shutdown_connections 2023-10-08T00:14:42.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.289+0000 7f755d630700 1 --2- 172.21.15.99:0/591401040 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7538066f30 0x7f75380693e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:42.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.289+0000 7f755d630700 1 --2- 172.21.15.99:0/591401040 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f75580b6a10 0x7f7558153990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:42.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.289+0000 7f755d630700 1 --2- 172.21.15.99:0/591401040 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f755815b300 0x7f75581511f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:42.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.289+0000 7f755d630700 1 --2- 172.21.15.99:0/591401040 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7558159ef0 0x7f755815adc0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:42.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.289+0000 7f755d630700 1 -- 172.21.15.99:0/591401040 >> 172.21.15.99:0/591401040 conn(0x7f755809a0a0 msgr2=0x7f75581592c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:42.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.290+0000 7f755d630700 1 -- 172.21.15.99:0/591401040 shutdown_connections 2023-10-08T00:14:42.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.290+0000 7f755d630700 1 -- 172.21.15.99:0/591401040 wait complete. 2023-10-08T00:14:42.291 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' created 2023-10-08T00:14:42.304 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T00:14:42.429 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.429+0000 7faadffff700 1 Processor -- start 2023-10-08T00:14:42.430 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.429+0000 7faadffff700 1 -- start start 2023-10-08T00:14:42.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.430+0000 7faadffff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faae006aa00 0x7faae00b65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:42.434 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.430+0000 7faadffff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7faae00cd630 0x7faae015c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:42.434 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.430+0000 7faadffff700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faae006b0a0 con 0x7faae01542a0 2023-10-08T00:14:42.434 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.430+0000 7faadffff700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faae015b2e0 con 0x7faae00cd630 2023-10-08T00:14:42.434 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.430+0000 7faadffff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faae015ad20 con 0x7faae006aa00 2023-10-08T00:14:42.434 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.430+0000 7faadeffd700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7faae01542a0 0x7faae006a2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:41772/0 (socket says 172.21.15.99:41772) 2023-10-08T00:14:42.434 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.430+0000 7faadeffd700 1 -- 172.21.15.99:0/204453434 learned_addr learned my addr 172.21.15.99:0/204453434 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:14:42.434 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.430+0000 7faadf7fe700 1 --2- 172.21.15.99:0/204453434 >> v2:172.21.15.191:3300/0 conn(0x7faae00cd630 0x7faae015c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:42.434 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.430+0000 7faade7fc700 1 --2- 172.21.15.99:0/204453434 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faae006aa00 0x7faae00b65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:42.434 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.430+0000 7faaddffb700 1 -- 172.21.15.99:0/204453434 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1501991481 0 0) 0x7faae006b0a0 con 0x7faae01542a0 2023-10-08T00:14:42.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.430+0000 7faaddffb700 1 -- 172.21.15.99:0/204453434 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faac8003570 con 0x7faae01542a0 2023-10-08T00:14:42.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.430+0000 7faaddffb700 1 -- 172.21.15.99:0/204453434 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2496362418 0 0) 0x7faac8003570 con 0x7faae01542a0 2023-10-08T00:14:42.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.430+0000 7faaddffb700 1 -- 172.21.15.99:0/204453434 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faae006aa00 msgr2=0x7faae00b65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:42.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.430+0000 7faaddffb700 1 --2- 172.21.15.99:0/204453434 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faae006aa00 0x7faae00b65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:42.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.430+0000 7faaddffb700 1 -- 172.21.15.99:0/204453434 >> v2:172.21.15.191:3300/0 conn(0x7faae00cd630 msgr2=0x7faae015c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:42.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.430+0000 7faaddffb700 1 --2- 172.21.15.99:0/204453434 >> v2:172.21.15.191:3300/0 conn(0x7faae00cd630 0x7faae015c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:42.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.430+0000 7faaddffb700 1 -- 172.21.15.99:0/204453434 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faae015b000 con 0x7faae01542a0 2023-10-08T00:14:42.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.430+0000 7faaddffb700 1 -- 172.21.15.99:0/204453434 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7faad00015f0 con 0x7faae01542a0 2023-10-08T00:14:42.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.430+0000 7faadf7fe700 1 --2- 172.21.15.99:0/204453434 >> v2:172.21.15.191:3300/0 conn(0x7faae00cd630 0x7faae015c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:14:42.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.430+0000 7faaddffb700 1 -- 172.21.15.99:0/204453434 >> v1:172.21.15.99:6789/0 conn(0x7faae01542a0 legacy=0x7faae006a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:42.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.430+0000 7faaddffb700 1 --2- 172.21.15.99:0/204453434 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faae006aa00 0x7faac8003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:42.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.430+0000 7faaddffb700 1 --2- 172.21.15.99:0/204453434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faac8003fe0 0x7faac8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:42.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.430+0000 7faaddffb700 1 --2- 172.21.15.99:0/204453434 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faac8004d50 0x7faac80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:42.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.430+0000 7faaddffb700 1 -- 172.21.15.99:0/204453434 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae015b2e0 con 0x7faac8003fe0 2023-10-08T00:14:42.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.430+0000 7faaddffb700 1 -- 172.21.15.99:0/204453434 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae015ad20 con 0x7faac8004d50 2023-10-08T00:14:42.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.430+0000 7faaddffb700 1 -- 172.21.15.99:0/204453434 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faac800ba90 con 0x7faae006aa00 2023-10-08T00:14:42.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.431+0000 7faade7fc700 1 --2- 172.21.15.99:0/204453434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faac8003fe0 0x7faac8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:42.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.431+0000 7faadeffd700 1 --2- 172.21.15.99:0/204453434 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faae006aa00 0x7faac8003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:42.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.431+0000 7faadf7fe700 1 --2- 172.21.15.99:0/204453434 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faac8004d50 0x7faac80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:42.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.431+0000 7faade7fc700 1 -- 172.21.15.99:0/204453434 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faae006aa00 msgr2=0x7faac8003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:42.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.431+0000 7faade7fc700 1 --2- 172.21.15.99:0/204453434 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faae006aa00 0x7faac8003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:42.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.431+0000 7faade7fc700 1 -- 172.21.15.99:0/204453434 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faac8004d50 msgr2=0x7faac80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:42.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.431+0000 7faade7fc700 1 --2- 172.21.15.99:0/204453434 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faac8004d50 0x7faac80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:42.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.431+0000 7faade7fc700 1 -- 172.21.15.99:0/204453434 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac8003570 con 0x7faac8003fe0 2023-10-08T00:14:42.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.431+0000 7faade7fc700 1 --2- 172.21.15.99:0/204453434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faac8003fe0 0x7faac8004810 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7faae006ae00 tx=0x7faad40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3e6f55a50535373 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:42.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.431+0000 7faaddffb700 1 -- 172.21.15.99:0/204453434 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad400e6b0 con 0x7faac8003fe0 2023-10-08T00:14:42.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.431+0000 7faaddffb700 1 -- 172.21.15.99:0/204453434 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faad400e810 con 0x7faac8003fe0 2023-10-08T00:14:42.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.431+0000 7faaddffb700 1 -- 172.21.15.99:0/204453434 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad400ebd0 con 0x7faac8003fe0 2023-10-08T00:14:42.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.432+0000 7faadffff700 1 -- 172.21.15.99:0/204453434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faac8003fe0 msgr2=0x7faac8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:42.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.432+0000 7faadffff700 1 --2- 172.21.15.99:0/204453434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faac8003fe0 0x7faac8004810 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7faae006ae00 tx=0x7faad40091a0 comp rx=0 tx=0).stop 2023-10-08T00:14:42.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.432+0000 7faadf7fe700 1 -- 172.21.15.99:0/204453434 reap_dead start 2023-10-08T00:14:42.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.432+0000 7faadffff700 1 -- 172.21.15.99:0/204453434 shutdown_connections 2023-10-08T00:14:42.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.432+0000 7faadffff700 1 -- 172.21.15.99:0/204453434 >> 172.21.15.99:0/204453434 conn(0x7faae0068cb0 msgr2=0x7faae00690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:42.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.432+0000 7faadffff700 1 -- 172.21.15.99:0/204453434 shutdown_connections 2023-10-08T00:14:42.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.432+0000 7faadffff700 1 -- 172.21.15.99:0/204453434 wait complete. 2023-10-08T00:14:42.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.433+0000 7faadffff700 1 Processor -- start 2023-10-08T00:14:42.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.433+0000 7faadffff700 1 -- start start 2023-10-08T00:14:42.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.434+0000 7faadffff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faac8003fe0 0x7faae01520c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:42.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.434+0000 7faadffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faae0152d90 0x7faae00c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:42.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.434+0000 7faadffff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faae01531e0 0x7faae00c72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:42.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.434+0000 7faadffff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae0152be0 con 0x7faae0152d90 2023-10-08T00:14:42.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.434+0000 7faadffff700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae0152600 con 0x7faae01531e0 2023-10-08T00:14:42.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.434+0000 7faadffff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faae01567c0 con 0x7faac8003fe0 2023-10-08T00:14:42.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.434+0000 7faade7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faae0152d90 0x7faae00c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:42.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.434+0000 7faade7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faae0152d90 0x7faae00c6d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33816/0 (socket says 172.21.15.99:33816) 2023-10-08T00:14:42.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.434+0000 7faade7fc700 1 -- 172.21.15.99:0/725460848 learned_addr learned my addr 172.21.15.99:0/725460848 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:42.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.434+0000 7faadeffd700 1 --2- 172.21.15.99:0/725460848 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faac8003fe0 0x7faae01520c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:42.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.434+0000 7faadf7fe700 1 --2- 172.21.15.99:0/725460848 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faae01531e0 0x7faae00c72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:42.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.434+0000 7faade7fc700 1 -- 172.21.15.99:0/725460848 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faac8003fe0 msgr2=0x7faae01520c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:42.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.434+0000 7faade7fc700 1 --2- 172.21.15.99:0/725460848 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faac8003fe0 0x7faae01520c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:42.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.434+0000 7faade7fc700 1 -- 172.21.15.99:0/725460848 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faae01531e0 msgr2=0x7faae00c72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:42.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.434+0000 7faade7fc700 1 --2- 172.21.15.99:0/725460848 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faae01531e0 0x7faae00c72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:42.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.434+0000 7faade7fc700 1 -- 172.21.15.99:0/725460848 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faae00c6750 con 0x7faae0152d90 2023-10-08T00:14:42.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.434+0000 7faade7fc700 1 --2- 172.21.15.99:0/725460848 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faae0152d90 0x7faae00c6d60 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7faad40091e0 tx=0x7faad401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:42.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.434+0000 7faac6ffd700 1 -- 172.21.15.99:0/725460848 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad4022070 con 0x7faae0152d90 2023-10-08T00:14:42.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.434+0000 7faac6ffd700 1 -- 172.21.15.99:0/725460848 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faad400e980 con 0x7faae0152d90 2023-10-08T00:14:42.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.434+0000 7faadffff700 1 -- 172.21.15.99:0/725460848 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faae00c6a30 con 0x7faae0152d90 2023-10-08T00:14:42.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.434+0000 7faac6ffd700 1 -- 172.21.15.99:0/725460848 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad400e6b0 con 0x7faae0152d90 2023-10-08T00:14:42.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.434+0000 7faadffff700 1 -- 172.21.15.99:0/725460848 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faae0156d60 con 0x7faae0152d90 2023-10-08T00:14:42.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.435+0000 7faadffff700 1 -- 172.21.15.99:0/725460848 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faae0156f50 con 0x7faae0152d90 2023-10-08T00:14:42.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.436+0000 7faac6ffd700 1 -- 172.21.15.99:0/725460848 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faad401c450 con 0x7faae0152d90 2023-10-08T00:14:42.449 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.436+0000 7faac6ffd700 1 --2- 172.21.15.99:0/725460848 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7faacc066f60 0x7faacc069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:42.449 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.436+0000 7faac6ffd700 1 -- 172.21.15.99:0/725460848 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5223+0+0 (secure 0 0 0) 0x7faad4085df0 con 0x7faae0152d90 2023-10-08T00:14:42.449 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.436+0000 7faadeffd700 1 --2- 172.21.15.99:0/725460848 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7faacc066f60 0x7faacc069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:42.449 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.437+0000 7faadeffd700 1 --2- 172.21.15.99:0/725460848 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7faacc066f60 0x7faacc069410 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7faad0001260 tx=0x7faad0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:42.449 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.441+0000 7faac6ffd700 1 -- 172.21.15.99:0/725460848 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faad40531a0 con 0x7faae0152d90 2023-10-08T00:14:42.622 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:42.622+0000 7faadffff700 1 -- 172.21.15.99:0/725460848 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7faae004bb80 con 0x7faae0152d90 2023-10-08T00:14:43.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.288+0000 7faac6ffd700 1 -- 172.21.15.99:0/725460848 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v84) v1 ==== 170+0+0 (secure 0 0 0) 0x7faad40567c0 con 0x7faae0152d90 2023-10-08T00:14:43.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.292+0000 7faadffff700 1 -- 172.21.15.99:0/725460848 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7faacc066f60 msgr2=0x7faacc069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:43.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.292+0000 7faadffff700 1 --2- 172.21.15.99:0/725460848 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7faacc066f60 0x7faacc069410 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7faad0001260 tx=0x7faad0009040 comp rx=0 tx=0).stop 2023-10-08T00:14:43.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.292+0000 7faadffff700 1 -- 172.21.15.99:0/725460848 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faae0152d90 msgr2=0x7faae00c6d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:43.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.292+0000 7faadffff700 1 --2- 172.21.15.99:0/725460848 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faae0152d90 0x7faae00c6d60 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7faad40091e0 tx=0x7faad401a000 comp rx=0 tx=0).stop 2023-10-08T00:14:43.294 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.293+0000 7faadffff700 1 -- 172.21.15.99:0/725460848 shutdown_connections 2023-10-08T00:14:43.294 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.293+0000 7faadffff700 1 --2- 172.21.15.99:0/725460848 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7faacc066f60 0x7faacc069410 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:43.294 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.293+0000 7faadffff700 1 --2- 172.21.15.99:0/725460848 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faae01531e0 0x7faae00c72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:43.294 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.293+0000 7faadffff700 1 --2- 172.21.15.99:0/725460848 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faac8003fe0 0x7faae01520c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:43.294 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.293+0000 7faadffff700 1 --2- 172.21.15.99:0/725460848 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faae0152d90 0x7faae00c6d60 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:43.294 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.293+0000 7faadffff700 1 -- 172.21.15.99:0/725460848 >> 172.21.15.99:0/725460848 conn(0x7faae0068cb0 msgr2=0x7faae00cf9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:43.295 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.295+0000 7faadffff700 1 -- 172.21.15.99:0/725460848 shutdown_connections 2023-10-08T00:14:43.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.295+0000 7faadffff700 1 -- 172.21.15.99:0/725460848 wait complete. 2023-10-08T00:14:43.301 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' created 2023-10-08T00:14:43.314 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T00:14:43.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.431+0000 7f97efb46700 1 Processor -- start 2023-10-08T00:14:43.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.431+0000 7f97efb46700 1 -- start start 2023-10-08T00:14:43.434 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.432+0000 7f97efb46700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f97e8159ec0 0x7f97e80af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:43.434 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.432+0000 7f97efb46700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f97e806acb0 0x7f97e80afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:43.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.432+0000 7f97efb46700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97e80b0790 con 0x7f97e815a290 2023-10-08T00:14:43.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.432+0000 7f97efb46700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97e80ce1f0 con 0x7f97e806acb0 2023-10-08T00:14:43.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.432+0000 7f97efb46700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97e80b0b20 con 0x7f97e8159ec0 2023-10-08T00:14:43.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.432+0000 7f97ee0e3700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f97e806acb0 0x7f97e80afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:43.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.432+0000 7f97ed0e1700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f97e815a290 0x7f97e806a5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:34692/0 (socket says 172.21.15.99:34692) 2023-10-08T00:14:43.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.432+0000 7f97ed0e1700 1 -- 172.21.15.99:0/3735202629 learned_addr learned my addr 172.21.15.99:0/3735202629 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:14:43.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.433+0000 7f97ed8e2700 1 --2- 172.21.15.99:0/3735202629 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f97e8159ec0 0x7f97e80af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:43.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.433+0000 7f97ee0e3700 1 -- 172.21.15.99:0/3735202629 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f97e8159ec0 msgr2=0x7f97e80af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:43.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.433+0000 7f97ee0e3700 1 --2- 172.21.15.99:0/3735202629 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f97e8159ec0 0x7f97e80af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:43.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.433+0000 7f97ee0e3700 1 -- 172.21.15.99:0/3735202629 >> v1:172.21.15.99:6789/0 conn(0x7f97e815a290 legacy=0x7f97e806a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:43.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.433+0000 7f97ec8e0700 1 -- 172.21.15.99:0/3735202629 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1003299525 0 0) 0x7f97e80b0790 con 0x7f97e815a290 2023-10-08T00:14:43.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.433+0000 7f97ee0e3700 1 -- 172.21.15.99:0/3735202629 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e80b04b0 con 0x7f97e806acb0 2023-10-08T00:14:43.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.433+0000 7f97ed8e2700 1 --2- 172.21.15.99:0/3735202629 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f97e8159ec0 0x7f97e80af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:14:43.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.433+0000 7f97ee0e3700 1 --2- 172.21.15.99:0/3735202629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97e806acb0 0x7f97e80afe80 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f97e0009e60 tx=0x7f97e0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83a298f3c1472374 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:43.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.433+0000 7f97ec8e0700 1 -- 172.21.15.99:0/3735202629 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e000e070 con 0x7f97e806acb0 2023-10-08T00:14:43.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.433+0000 7f97ec8e0700 1 -- 172.21.15.99:0/3735202629 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e000bc30 con 0x7f97e806acb0 2023-10-08T00:14:43.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.433+0000 7f97ec8e0700 1 -- 172.21.15.99:0/3735202629 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e0012430 con 0x7f97e806acb0 2023-10-08T00:14:43.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.433+0000 7f97efb46700 1 -- 172.21.15.99:0/3735202629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97e806acb0 msgr2=0x7f97e80afe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:43.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.433+0000 7f97efb46700 1 --2- 172.21.15.99:0/3735202629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97e806acb0 0x7f97e80afe80 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f97e0009e60 tx=0x7f97e0009780 comp rx=0 tx=0).stop 2023-10-08T00:14:43.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.434+0000 7f97efb46700 1 -- 172.21.15.99:0/3735202629 shutdown_connections 2023-10-08T00:14:43.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.434+0000 7f97efb46700 1 --2- 172.21.15.99:0/3735202629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97e806acb0 0x7f97e80afe80 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:43.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.434+0000 7f97efb46700 1 --2- 172.21.15.99:0/3735202629 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f97e8159ec0 0x7f97e80af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:43.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.434+0000 7f97efb46700 1 -- 172.21.15.99:0/3735202629 >> 172.21.15.99:0/3735202629 conn(0x7f97e80d1880 msgr2=0x7f97e80cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:43.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.434+0000 7f97efb46700 1 -- 172.21.15.99:0/3735202629 shutdown_connections 2023-10-08T00:14:43.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.434+0000 7f97efb46700 1 -- 172.21.15.99:0/3735202629 wait complete. 2023-10-08T00:14:43.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.435+0000 7f97efb46700 1 Processor -- start 2023-10-08T00:14:43.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.435+0000 7f97efb46700 1 -- start start 2023-10-08T00:14:43.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.435+0000 7f97efb46700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f97e806acb0 0x7f97e8158960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:43.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.435+0000 7f97efb46700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f97e8159ec0 0x7f97e8153270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:43.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.435+0000 7f97efb46700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97e815a290 0x7f97e80b1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:43.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.435+0000 7f97efb46700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e8169df0 con 0x7f97e815a290 2023-10-08T00:14:43.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.435+0000 7f97efb46700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e815a890 con 0x7f97e806acb0 2023-10-08T00:14:43.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.435+0000 7f97efb46700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97e8159190 con 0x7f97e8159ec0 2023-10-08T00:14:43.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.436+0000 7f97ed8e2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f97e806acb0 0x7f97e8158960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:43.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.436+0000 7f97ed0e1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f97e8159ec0 0x7f97e8153270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:43.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.436+0000 7f97ed8e2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f97e806acb0 0x7f97e8158960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.99:53708/0 (socket says 172.21.15.99:53708) 2023-10-08T00:14:43.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.436+0000 7f97ed8e2700 1 -- 172.21.15.99:0/940747134 learned_addr learned my addr 172.21.15.99:0/940747134 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:43.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.436+0000 7f97ee0e3700 1 --2- 172.21.15.99:0/940747134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97e815a290 0x7f97e80b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:43.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.436+0000 7f97ed0e1700 1 -- 172.21.15.99:0/940747134 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f97e806acb0 msgr2=0x7f97e8158960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:43.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.436+0000 7f97ed0e1700 1 --2- 172.21.15.99:0/940747134 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f97e806acb0 0x7f97e8158960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:43.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.436+0000 7f97ed0e1700 1 -- 172.21.15.99:0/940747134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97e815a290 msgr2=0x7f97e80b1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:43.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.436+0000 7f97ed0e1700 1 --2- 172.21.15.99:0/940747134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97e815a290 0x7f97e80b1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:43.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.436+0000 7f97ed0e1700 1 -- 172.21.15.99:0/940747134 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e0009a20 con 0x7f97e8159ec0 2023-10-08T00:14:43.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.436+0000 7f97ed0e1700 1 --2- 172.21.15.99:0/940747134 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f97e8159ec0 0x7f97e8153270 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f97d800c4f0 tx=0x7f97d8007f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:43.443 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.436+0000 7f97ddffb700 1 -- 172.21.15.99:0/940747134 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d8005b30 con 0x7f97e8159ec0 2023-10-08T00:14:43.443 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.436+0000 7f97ddffb700 1 -- 172.21.15.99:0/940747134 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97d8005c90 con 0x7f97e8159ec0 2023-10-08T00:14:43.443 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.436+0000 7f97efb46700 1 -- 172.21.15.99:0/940747134 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97e816a900 con 0x7f97e8159ec0 2023-10-08T00:14:43.443 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.436+0000 7f97ddffb700 1 -- 172.21.15.99:0/940747134 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d8007300 con 0x7f97e8159ec0 2023-10-08T00:14:43.443 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.437+0000 7f97efb46700 1 -- 172.21.15.99:0/940747134 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97e816a080 con 0x7f97e8159ec0 2023-10-08T00:14:43.443 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.437+0000 7f97efb46700 1 -- 172.21.15.99:0/940747134 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97e804c3c0 con 0x7f97e8159ec0 2023-10-08T00:14:43.443 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.438+0000 7f97ddffb700 1 -- 172.21.15.99:0/940747134 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f97d801b3b0 con 0x7f97e8159ec0 2023-10-08T00:14:43.443 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.438+0000 7f97ddffb700 1 --2- 172.21.15.99:0/940747134 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f97cc066f30 0x7f97cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:43.443 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.438+0000 7f97ddffb700 1 -- 172.21.15.99:0/940747134 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f97d8083960 con 0x7f97e8159ec0 2023-10-08T00:14:43.443 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.438+0000 7f97ed8e2700 1 --2- 172.21.15.99:0/940747134 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f97cc066f30 0x7f97cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:43.444 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.439+0000 7f97ed8e2700 1 --2- 172.21.15.99:0/940747134 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f97cc066f30 0x7f97cc0693e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f97e4006dc0 tx=0x7f97e4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:43.444 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.443+0000 7f97ddffb700 1 -- 172.21.15.99:0/940747134 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f97d8050c30 con 0x7f97e8159ec0 2023-10-08T00:14:43.669 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:43.668+0000 7f97efb46700 1 -- 172.21.15.99:0/940747134 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f97e806b5e0 con 0x7f97e8159ec0 2023-10-08T00:14:44.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.319+0000 7f97ddffb700 1 -- 172.21.15.99:0/940747134 <== mon.2 v2:172.21.15.191:3301/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v36) v1 ==== 174+0+0 (secure 0 0 0) 0x7f97d80170c0 con 0x7f97e8159ec0 2023-10-08T00:14:44.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.322+0000 7f97efb46700 1 -- 172.21.15.99:0/940747134 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f97cc066f30 msgr2=0x7f97cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:44.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.322+0000 7f97efb46700 1 --2- 172.21.15.99:0/940747134 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f97cc066f30 0x7f97cc0693e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f97e4006dc0 tx=0x7f97e4008040 comp rx=0 tx=0).stop 2023-10-08T00:14:44.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.322+0000 7f97efb46700 1 -- 172.21.15.99:0/940747134 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f97e8159ec0 msgr2=0x7f97e8153270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:44.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.323+0000 7f97efb46700 1 --2- 172.21.15.99:0/940747134 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f97e8159ec0 0x7f97e8153270 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f97d800c4f0 tx=0x7f97d8007f50 comp rx=0 tx=0).stop 2023-10-08T00:14:44.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.323+0000 7f97efb46700 1 -- 172.21.15.99:0/940747134 shutdown_connections 2023-10-08T00:14:44.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.323+0000 7f97efb46700 1 --2- 172.21.15.99:0/940747134 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f97cc066f30 0x7f97cc0693e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:44.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.323+0000 7f97efb46700 1 --2- 172.21.15.99:0/940747134 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f97e806acb0 0x7f97e8158960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:44.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.323+0000 7f97efb46700 1 --2- 172.21.15.99:0/940747134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97e815a290 0x7f97e80b1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:44.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.323+0000 7f97efb46700 1 --2- 172.21.15.99:0/940747134 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f97e8159ec0 0x7f97e8153270 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:44.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.323+0000 7f97efb46700 1 -- 172.21.15.99:0/940747134 >> 172.21.15.99:0/940747134 conn(0x7f97e80d1880 msgr2=0x7f97e8155970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:44.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.323+0000 7f97efb46700 1 -- 172.21.15.99:0/940747134 shutdown_connections 2023-10-08T00:14:44.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.323+0000 7f97efb46700 1 -- 172.21.15.99:0/940747134 wait complete. 2023-10-08T00:14:44.330 INFO:teuthology.orchestra.run.smithi099.stderr:new fs with metadata pool 10 and data pool 11 2023-10-08T00:14:44.338 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:14:44.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.453+0000 7f5b1eec9700 1 Processor -- start 2023-10-08T00:14:44.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.454+0000 7f5b1eec9700 1 -- start start 2023-10-08T00:14:44.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.454+0000 7f5b1eec9700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5b18152010 0x7f5b1815b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:44.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.454+0000 7f5b1eec9700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b1806c010 con 0x7f5b181523e0 2023-10-08T00:14:44.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.454+0000 7f5b1eec9700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b1806bd10 con 0x7f5b18158e50 2023-10-08T00:14:44.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.454+0000 7f5b1eec9700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b1815c060 con 0x7f5b18152010 2023-10-08T00:14:44.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.454+0000 7f5b1cc65700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5b18152010 0x7f5b1815b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:44.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.454+0000 7f5b0ffff700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f5b181523e0 0x7f5b1806c4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:34696/0 (socket says 172.21.15.99:34696) 2023-10-08T00:14:44.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.454+0000 7f5b0ffff700 1 -- 172.21.15.99:0/3319868394 learned_addr learned my addr 172.21.15.99:0/3319868394 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:14:44.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.455+0000 7f5b1cc65700 1 -- 172.21.15.99:0/3319868394 >> v1:172.21.15.191:6789/0 conn(0x7f5b18158e50 legacy=0x7f5b180cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:44.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.455+0000 7f5b1cc65700 1 -- 172.21.15.99:0/3319868394 >> v1:172.21.15.99:6789/0 conn(0x7f5b181523e0 legacy=0x7f5b1806c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:44.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.455+0000 7f5b0f7fe700 1 -- 172.21.15.99:0/3319868394 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4196196948 0 0) 0x7f5b1806c010 con 0x7f5b181523e0 2023-10-08T00:14:44.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.455+0000 7f5b1cc65700 1 -- 172.21.15.99:0/3319868394 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b1815c340 con 0x7f5b18152010 2023-10-08T00:14:44.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.455+0000 7f5b1cc65700 1 --2- 172.21.15.99:0/3319868394 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b18152010 0x7f5b1815b810 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f5b140098e0 tx=0x7f5b140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3884b880c0c81590 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:44.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.455+0000 7f5b0f7fe700 1 -- 172.21.15.99:0/3319868394 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b14009e80 con 0x7f5b18152010 2023-10-08T00:14:44.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.455+0000 7f5b0f7fe700 1 -- 172.21.15.99:0/3319868394 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b1400b820 con 0x7f5b18152010 2023-10-08T00:14:44.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.455+0000 7f5b0f7fe700 1 -- 172.21.15.99:0/3319868394 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1400bba0 con 0x7f5b18152010 2023-10-08T00:14:44.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.455+0000 7f5b1eec9700 1 -- 172.21.15.99:0/3319868394 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b18152010 msgr2=0x7f5b1815b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:44.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.456+0000 7f5b1eec9700 1 --2- 172.21.15.99:0/3319868394 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b18152010 0x7f5b1815b810 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f5b140098e0 tx=0x7f5b140091a0 comp rx=0 tx=0).stop 2023-10-08T00:14:44.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.456+0000 7f5b1eec9700 1 -- 172.21.15.99:0/3319868394 shutdown_connections 2023-10-08T00:14:44.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.456+0000 7f5b1eec9700 1 --2- 172.21.15.99:0/3319868394 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b18152010 0x7f5b1815b810 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:44.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.456+0000 7f5b1eec9700 1 -- 172.21.15.99:0/3319868394 >> 172.21.15.99:0/3319868394 conn(0x7f5b18069870 msgr2=0x7f5b18069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:44.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.456+0000 7f5b1eec9700 1 -- 172.21.15.99:0/3319868394 shutdown_connections 2023-10-08T00:14:44.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.456+0000 7f5b1eec9700 1 -- 172.21.15.99:0/3319868394 wait complete. 2023-10-08T00:14:44.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.457+0000 7f5b1eec9700 1 Processor -- start 2023-10-08T00:14:44.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.457+0000 7f5b1eec9700 1 -- start start 2023-10-08T00:14:44.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.457+0000 7f5b1eec9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5b18152010 0x7f5b1814d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:44.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.458+0000 7f5b1eec9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b181523e0 0x7f5b1814f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:44.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.458+0000 7f5b1eec9700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b18158e50 0x7f5b180c5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:44.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.458+0000 7f5b1eec9700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b180b4910 con 0x7f5b181523e0 2023-10-08T00:14:44.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.458+0000 7f5b1eec9700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1816b740 con 0x7f5b18158e50 2023-10-08T00:14:44.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.458+0000 7f5b1eec9700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1816b990 con 0x7f5b18152010 2023-10-08T00:14:44.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.458+0000 7f5b0ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b181523e0 0x7f5b1814f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:44.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.458+0000 7f5b0ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b181523e0 0x7f5b1814f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45640/0 (socket says 172.21.15.99:45640) 2023-10-08T00:14:44.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.458+0000 7f5b0ffff700 1 -- 172.21.15.99:0/2489787570 learned_addr learned my addr 172.21.15.99:0/2489787570 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:44.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.458+0000 7f5b1d466700 1 --2- 172.21.15.99:0/2489787570 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b18158e50 0x7f5b180c5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:44.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.458+0000 7f5b1cc65700 1 --2- 172.21.15.99:0/2489787570 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5b18152010 0x7f5b1814d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:44.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.458+0000 7f5b0ffff700 1 -- 172.21.15.99:0/2489787570 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5b18152010 msgr2=0x7f5b1814d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:44.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.458+0000 7f5b0ffff700 1 --2- 172.21.15.99:0/2489787570 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5b18152010 0x7f5b1814d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:44.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.458+0000 7f5b0ffff700 1 -- 172.21.15.99:0/2489787570 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b18158e50 msgr2=0x7f5b180c5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:44.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.458+0000 7f5b0ffff700 1 --2- 172.21.15.99:0/2489787570 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b18158e50 0x7f5b180c5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:44.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.458+0000 7f5b0ffff700 1 -- 172.21.15.99:0/2489787570 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b14009420 con 0x7f5b181523e0 2023-10-08T00:14:44.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.458+0000 7f5b1d466700 1 --2- 172.21.15.99:0/2489787570 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b18158e50 0x7f5b180c5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:14:44.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.458+0000 7f5b0ffff700 1 --2- 172.21.15.99:0/2489787570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b181523e0 0x7f5b1814f330 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f5b0800c810 tx=0x7f5b08007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:44.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.458+0000 7f5b0cff9700 1 -- 172.21.15.99:0/2489787570 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b08005b30 con 0x7f5b181523e0 2023-10-08T00:14:44.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.458+0000 7f5b0cff9700 1 -- 172.21.15.99:0/2489787570 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b08005c90 con 0x7f5b181523e0 2023-10-08T00:14:44.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.458+0000 7f5b0cff9700 1 -- 172.21.15.99:0/2489787570 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b08007300 con 0x7f5b181523e0 2023-10-08T00:14:44.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.458+0000 7f5b1eec9700 1 -- 172.21.15.99:0/2489787570 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b1816bc70 con 0x7f5b181523e0 2023-10-08T00:14:44.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.458+0000 7f5b1eec9700 1 -- 172.21.15.99:0/2489787570 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b1816c230 con 0x7f5b181523e0 2023-10-08T00:14:44.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.459+0000 7f5b1eec9700 1 -- 172.21.15.99:0/2489787570 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b180c38e0 con 0x7f5b181523e0 2023-10-08T00:14:44.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.460+0000 7f5b0cff9700 1 -- 172.21.15.99:0/2489787570 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5b0801b400 con 0x7f5b181523e0 2023-10-08T00:14:44.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.460+0000 7f5b0cff9700 1 --2- 172.21.15.99:0/2489787570 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5b04066f30 0x7f5b040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:44.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.460+0000 7f5b0cff9700 1 -- 172.21.15.99:0/2489787570 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5b08084030 con 0x7f5b181523e0 2023-10-08T00:14:44.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.460+0000 7f5b1cc65700 1 --2- 172.21.15.99:0/2489787570 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5b04066f30 0x7f5b040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:44.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.461+0000 7f5b1cc65700 1 --2- 172.21.15.99:0/2489787570 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5b04066f30 0x7f5b040693e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5b140091e0 tx=0x7f5b14016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:44.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.465+0000 7f5b0cff9700 1 -- 172.21.15.99:0/2489787570 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5b08051320 con 0x7f5b181523e0 2023-10-08T00:14:44.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.648+0000 7f5b1eec9700 1 -- 172.21.15.99:0/2489787570 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5b1804c3c0 con 0x7f5b181523e0 2023-10-08T00:14:44.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.649+0000 7f5b0cff9700 1 -- 172.21.15.99:0/2489787570 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+15366 (secure 0 0 0) 0x7f5b08054940 con 0x7f5b181523e0 2023-10-08T00:14:44.649 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:14:44.650 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":85,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:14:43.670752+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:14:41.496823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-08T00:14:42.640029+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:14:44.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.651+0000 7f5b1eec9700 1 -- 172.21.15.99:0/2489787570 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5b04066f30 msgr2=0x7f5b040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:44.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.651+0000 7f5b1eec9700 1 --2- 172.21.15.99:0/2489787570 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5b04066f30 0x7f5b040693e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5b140091e0 tx=0x7f5b14016000 comp rx=0 tx=0).stop 2023-10-08T00:14:44.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.651+0000 7f5b1eec9700 1 -- 172.21.15.99:0/2489787570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b181523e0 msgr2=0x7f5b1814f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:44.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.651+0000 7f5b1eec9700 1 --2- 172.21.15.99:0/2489787570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b181523e0 0x7f5b1814f330 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f5b0800c810 tx=0x7f5b08007df0 comp rx=0 tx=0).stop 2023-10-08T00:14:44.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.651+0000 7f5b1eec9700 1 -- 172.21.15.99:0/2489787570 shutdown_connections 2023-10-08T00:14:44.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.651+0000 7f5b1eec9700 1 --2- 172.21.15.99:0/2489787570 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5b04066f30 0x7f5b040693e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:44.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.651+0000 7f5b1eec9700 1 --2- 172.21.15.99:0/2489787570 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b18158e50 0x7f5b180c5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:44.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.651+0000 7f5b1eec9700 1 --2- 172.21.15.99:0/2489787570 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5b18152010 0x7f5b1814d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:44.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.651+0000 7f5b1eec9700 1 --2- 172.21.15.99:0/2489787570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b181523e0 0x7f5b1814f330 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:44.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.651+0000 7f5b1eec9700 1 -- 172.21.15.99:0/2489787570 >> 172.21.15.99:0/2489787570 conn(0x7f5b18069870 msgr2=0x7f5b181514c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:44.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.651+0000 7f5b1eec9700 1 -- 172.21.15.99:0/2489787570 shutdown_connections 2023-10-08T00:14:44.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.651+0000 7f5b1eec9700 1 -- 172.21.15.99:0/2489787570 wait complete. 2023-10-08T00:14:44.664 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:14:44.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.777+0000 7f6c4313c700 1 Processor -- start 2023-10-08T00:14:44.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.777+0000 7f6c4313c700 1 -- start start 2023-10-08T00:14:44.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.778+0000 7f6c4313c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c3c157ba0 0x7f6c3c06a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:44.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.778+0000 7f6c4313c700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c3c15c360 con 0x7f6c3c1577d0 2023-10-08T00:14:44.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.778+0000 7f6c4313c700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c3c15b790 con 0x7f6c3c15b0d0 2023-10-08T00:14:44.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.778+0000 7f6c4313c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3c15ba30 con 0x7f6c3c157ba0 2023-10-08T00:14:44.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.778+0000 7f6c40ed8700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f6c3c1577d0 0x7f6c3c0cd770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:34702/0 (socket says 172.21.15.99:34702) 2023-10-08T00:14:44.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.778+0000 7f6c40ed8700 1 -- 172.21.15.99:0/2148782340 learned_addr learned my addr 172.21.15.99:0/2148782340 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:14:44.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.779+0000 7f6c33fff700 1 --2- 172.21.15.99:0/2148782340 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c3c157ba0 0x7f6c3c06a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:44.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.779+0000 7f6c337fe700 1 -- 172.21.15.99:0/2148782340 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1244415897 0 0) 0x7f6c3c15c360 con 0x7f6c3c1577d0 2023-10-08T00:14:44.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.779+0000 7f6c337fe700 1 -- 172.21.15.99:0/2148782340 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c24003570 con 0x7f6c3c1577d0 2023-10-08T00:14:44.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.779+0000 7f6c337fe700 1 -- 172.21.15.99:0/2148782340 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3172759023 0 0) 0x7f6c3c15b790 con 0x7f6c3c15b0d0 2023-10-08T00:14:44.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.779+0000 7f6c337fe700 1 -- 172.21.15.99:0/2148782340 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c3c15c360 con 0x7f6c3c15b0d0 2023-10-08T00:14:44.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.779+0000 7f6c337fe700 1 -- 172.21.15.99:0/2148782340 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2316022880 0 0) 0x7f6c24003570 con 0x7f6c3c1577d0 2023-10-08T00:14:44.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.779+0000 7f6c337fe700 1 -- 172.21.15.99:0/2148782340 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c3c157ba0 msgr2=0x7f6c3c06a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:44.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.779+0000 7f6c337fe700 1 --2- 172.21.15.99:0/2148782340 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c3c157ba0 0x7f6c3c06a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:44.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.779+0000 7f6c337fe700 1 -- 172.21.15.99:0/2148782340 >> v1:172.21.15.191:6789/0 conn(0x7f6c3c15b0d0 legacy=0x7f6c3c15c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:44.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.779+0000 7f6c337fe700 1 -- 172.21.15.99:0/2148782340 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c3c15bd10 con 0x7f6c3c1577d0 2023-10-08T00:14:44.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.779+0000 7f6c337fe700 1 -- 172.21.15.99:0/2148782340 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f6c380017f0 con 0x7f6c3c1577d0 2023-10-08T00:14:44.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.779+0000 7f6c337fe700 1 -- 172.21.15.99:0/2148782340 >> v1:172.21.15.99:6789/0 conn(0x7f6c3c1577d0 legacy=0x7f6c3c0cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:44.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.779+0000 7f6c337fe700 1 --2- 172.21.15.99:0/2148782340 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c3c157ba0 0x7f6c24003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:44.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.779+0000 7f6c337fe700 1 --2- 172.21.15.99:0/2148782340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c24004180 0x7f6c24004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:44.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.779+0000 7f6c337fe700 1 --2- 172.21.15.99:0/2148782340 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c24004d50 0x7f6c240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:44.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.779+0000 7f6c337fe700 1 -- 172.21.15.99:0/2148782340 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3c15ba30 con 0x7f6c24004180 2023-10-08T00:14:44.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.779+0000 7f6c337fe700 1 -- 172.21.15.99:0/2148782340 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2400bba0 con 0x7f6c24004d50 2023-10-08T00:14:44.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.779+0000 7f6c337fe700 1 -- 172.21.15.99:0/2148782340 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2400bd60 con 0x7f6c3c157ba0 2023-10-08T00:14:44.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.779+0000 7f6c33fff700 1 --2- 172.21.15.99:0/2148782340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c24004180 0x7f6c24004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:44.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.779+0000 7f6c40ed8700 1 --2- 172.21.15.99:0/2148782340 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c3c157ba0 0x7f6c24003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:44.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.779+0000 7f6c416d9700 1 --2- 172.21.15.99:0/2148782340 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c24004d50 0x7f6c240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:44.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.779+0000 7f6c33fff700 1 -- 172.21.15.99:0/2148782340 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c3c157ba0 msgr2=0x7f6c24003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:44.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.779+0000 7f6c33fff700 1 --2- 172.21.15.99:0/2148782340 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c3c157ba0 0x7f6c24003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:44.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.779+0000 7f6c33fff700 1 -- 172.21.15.99:0/2148782340 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c24004d50 msgr2=0x7f6c240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:44.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.779+0000 7f6c33fff700 1 --2- 172.21.15.99:0/2148782340 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c24004d50 0x7f6c240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:44.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.779+0000 7f6c33fff700 1 -- 172.21.15.99:0/2148782340 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c24003570 con 0x7f6c24004180 2023-10-08T00:14:44.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.779+0000 7f6c416d9700 1 --2- 172.21.15.99:0/2148782340 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c24004d50 0x7f6c240051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:14:44.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.779+0000 7f6c40ed8700 1 --2- 172.21.15.99:0/2148782340 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c3c157ba0 0x7f6c24003c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:14:44.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.780+0000 7f6c33fff700 1 --2- 172.21.15.99:0/2148782340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c24004180 0x7f6c24004810 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c009c00 tx=0x7f6c2c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e6737d5fb75871 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:44.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.780+0000 7f6c337fe700 1 -- 172.21.15.99:0/2148782340 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c00e710 con 0x7f6c24004180 2023-10-08T00:14:44.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.780+0000 7f6c337fe700 1 -- 172.21.15.99:0/2148782340 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c2c00e870 con 0x7f6c24004180 2023-10-08T00:14:44.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.780+0000 7f6c337fe700 1 -- 172.21.15.99:0/2148782340 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c00ec30 con 0x7f6c24004180 2023-10-08T00:14:44.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.780+0000 7f6c4313c700 1 -- 172.21.15.99:0/2148782340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c24004180 msgr2=0x7f6c24004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:44.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.780+0000 7f6c4313c700 1 --2- 172.21.15.99:0/2148782340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c24004180 0x7f6c24004810 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c009c00 tx=0x7f6c2c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:14:44.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.780+0000 7f6c416d9700 1 -- 172.21.15.99:0/2148782340 reap_dead start 2023-10-08T00:14:44.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.780+0000 7f6c4313c700 1 -- 172.21.15.99:0/2148782340 shutdown_connections 2023-10-08T00:14:44.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.780+0000 7f6c4313c700 1 -- 172.21.15.99:0/2148782340 >> 172.21.15.99:0/2148782340 conn(0x7f6c3c068ce0 msgr2=0x7f6c3c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:44.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.780+0000 7f6c4313c700 1 -- 172.21.15.99:0/2148782340 shutdown_connections 2023-10-08T00:14:44.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.780+0000 7f6c4313c700 1 -- 172.21.15.99:0/2148782340 wait complete. 2023-10-08T00:14:44.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.781+0000 7f6c4313c700 1 Processor -- start 2023-10-08T00:14:44.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.782+0000 7f6c4313c700 1 -- start start 2023-10-08T00:14:44.788 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.782+0000 7f6c4313c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c24004180 0x7f6c3c0b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:44.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.782+0000 7f6c4313c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c3c0b2e90 0x7f6c3c16d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:44.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.782+0000 7f6c4313c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c3c153e40 0x7f6c3c1542a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:44.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.782+0000 7f6c4313c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3c0b21a0 con 0x7f6c3c0b2e90 2023-10-08T00:14:44.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.782+0000 7f6c4313c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3c16a750 con 0x7f6c24004180 2023-10-08T00:14:44.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.782+0000 7f6c4313c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3c153c40 con 0x7f6c3c153e40 2023-10-08T00:14:44.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.782+0000 7f6c33fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c3c0b2e90 0x7f6c3c16d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:44.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.782+0000 7f6c33fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c3c0b2e90 0x7f6c3c16d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45654/0 (socket says 172.21.15.99:45654) 2023-10-08T00:14:44.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.782+0000 7f6c33fff700 1 -- 172.21.15.99:0/2165989815 learned_addr learned my addr 172.21.15.99:0/2165989815 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:44.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.783+0000 7f6c40ed8700 1 --2- 172.21.15.99:0/2165989815 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c24004180 0x7f6c3c0b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:44.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.783+0000 7f6c416d9700 1 --2- 172.21.15.99:0/2165989815 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c3c153e40 0x7f6c3c1542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:44.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.783+0000 7f6c33fff700 1 -- 172.21.15.99:0/2165989815 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c3c153e40 msgr2=0x7f6c3c1542a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:44.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.783+0000 7f6c33fff700 1 --2- 172.21.15.99:0/2165989815 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c3c153e40 0x7f6c3c1542a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:44.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.783+0000 7f6c33fff700 1 -- 172.21.15.99:0/2165989815 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c24004180 msgr2=0x7f6c3c0b0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:44.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.783+0000 7f6c33fff700 1 --2- 172.21.15.99:0/2165989815 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c24004180 0x7f6c3c0b0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:44.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.783+0000 7f6c33fff700 1 -- 172.21.15.99:0/2165989815 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c3c1547e0 con 0x7f6c3c0b2e90 2023-10-08T00:14:44.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.783+0000 7f6c33fff700 1 --2- 172.21.15.99:0/2165989815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c3c0b2e90 0x7f6c3c16d500 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f6c2400bb70 tx=0x7f6c2c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:44.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.783+0000 7f6c30ff9700 1 -- 172.21.15.99:0/2165989815 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c022070 con 0x7f6c3c0b2e90 2023-10-08T00:14:44.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.783+0000 7f6c30ff9700 1 -- 172.21.15.99:0/2165989815 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c2c00e9e0 con 0x7f6c3c0b2e90 2023-10-08T00:14:44.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.783+0000 7f6c30ff9700 1 -- 172.21.15.99:0/2165989815 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c027470 con 0x7f6c3c0b2e90 2023-10-08T00:14:44.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.783+0000 7f6c4313c700 1 -- 172.21.15.99:0/2165989815 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c3c0b1620 con 0x7f6c3c0b2e90 2023-10-08T00:14:44.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.783+0000 7f6c4313c700 1 -- 172.21.15.99:0/2165989815 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c3c0b19b0 con 0x7f6c3c0b2e90 2023-10-08T00:14:44.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.784+0000 7f6c4313c700 1 -- 172.21.15.99:0/2165989815 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c3c04c3c0 con 0x7f6c3c0b2e90 2023-10-08T00:14:44.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.784+0000 7f6c30ff9700 1 -- 172.21.15.99:0/2165989815 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6c2c01c450 con 0x7f6c3c0b2e90 2023-10-08T00:14:44.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.785+0000 7f6c30ff9700 1 --2- 172.21.15.99:0/2165989815 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6c28066f30 0x7f6c280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:44.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.785+0000 7f6c30ff9700 1 -- 172.21.15.99:0/2165989815 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6c2c086020 con 0x7f6c3c0b2e90 2023-10-08T00:14:44.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.785+0000 7f6c40ed8700 1 --2- 172.21.15.99:0/2165989815 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6c28066f30 0x7f6c280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:44.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.785+0000 7f6c40ed8700 1 --2- 172.21.15.99:0/2165989815 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6c28066f30 0x7f6c280693e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c159c40 tx=0x7f6c38009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:44.792 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.789+0000 7f6c30ff9700 1 -- 172.21.15.99:0/2165989815 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6c2c053290 con 0x7f6c3c0b2e90 2023-10-08T00:14:44.972 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.972+0000 7f6c4313c700 1 -- 172.21.15.99:0/2165989815 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c3c1547e0 con 0x7f6c3c0b2e90 2023-10-08T00:14:44.973 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.972+0000 7f6c30ff9700 1 -- 172.21.15.99:0/2165989815 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+15366 (secure 0 0 0) 0x7f6c3c1547e0 con 0x7f6c3c0b2e90 2023-10-08T00:14:44.973 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:14:44.973 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":85,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:14:43.670752+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:14:41.496823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-08T00:14:42.640029+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:14:44.975 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.974+0000 7f6c4313c700 1 -- 172.21.15.99:0/2165989815 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6c28066f30 msgr2=0x7f6c280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:44.975 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.974+0000 7f6c4313c700 1 --2- 172.21.15.99:0/2165989815 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6c28066f30 0x7f6c280693e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c159c40 tx=0x7f6c38009040 comp rx=0 tx=0).stop 2023-10-08T00:14:44.975 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.974+0000 7f6c4313c700 1 -- 172.21.15.99:0/2165989815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c3c0b2e90 msgr2=0x7f6c3c16d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:44.976 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.974+0000 7f6c4313c700 1 --2- 172.21.15.99:0/2165989815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c3c0b2e90 0x7f6c3c16d500 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f6c2400bb70 tx=0x7f6c2c01a000 comp rx=0 tx=0).stop 2023-10-08T00:14:44.976 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.974+0000 7f6c4313c700 1 -- 172.21.15.99:0/2165989815 shutdown_connections 2023-10-08T00:14:44.976 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.974+0000 7f6c4313c700 1 --2- 172.21.15.99:0/2165989815 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6c28066f30 0x7f6c280693e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:44.976 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.974+0000 7f6c4313c700 1 --2- 172.21.15.99:0/2165989815 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c24004180 0x7f6c3c0b0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:44.976 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.974+0000 7f6c4313c700 1 --2- 172.21.15.99:0/2165989815 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c3c153e40 0x7f6c3c1542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:44.976 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.974+0000 7f6c4313c700 1 --2- 172.21.15.99:0/2165989815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c3c0b2e90 0x7f6c3c16d500 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:44.976 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.974+0000 7f6c4313c700 1 -- 172.21.15.99:0/2165989815 >> 172.21.15.99:0/2165989815 conn(0x7f6c3c068ce0 msgr2=0x7f6c3c0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:44.977 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.974+0000 7f6c4313c700 1 -- 172.21.15.99:0/2165989815 shutdown_connections 2023-10-08T00:14:44.977 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:44.974+0000 7f6c4313c700 1 -- 172.21.15.99:0/2165989815 wait complete. 2023-10-08T00:14:44.987 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T00:14:45.105 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.104+0000 7fd0c143b700 1 Processor -- start 2023-10-08T00:14:45.105 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.104+0000 7fd0c143b700 1 -- start start 2023-10-08T00:14:45.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.105+0000 7fd0c143b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd0bc0cce30 0x7fd0bc15aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:45.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.105+0000 7fd0c143b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd0bc06aa00 0x7fd0bc0b5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:45.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.105+0000 7fd0c143b700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0bc0b4d70 con 0x7fd0bc0b6580 2023-10-08T00:14:45.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.105+0000 7fd0c143b700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0bc0cd8d0 con 0x7fd0bc06aa00 2023-10-08T00:14:45.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.105+0000 7fd0c143b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0bc0b4440 con 0x7fd0bc0cce30 2023-10-08T00:14:45.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.105+0000 7fd0bb7fe700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fd0bc0b6580 0x7fd0bc06a2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:34704/0 (socket says 172.21.15.99:34704) 2023-10-08T00:14:45.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.105+0000 7fd0bb7fe700 1 -- 172.21.15.99:0/1401337494 learned_addr learned my addr 172.21.15.99:0/1401337494 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:14:45.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.105+0000 7fd0baffd700 1 -- 172.21.15.99:0/1401337494 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3147500845 0 0) 0x7fd0bc0b4d70 con 0x7fd0bc0b6580 2023-10-08T00:14:45.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.105+0000 7fd0bbfff700 1 --2- 172.21.15.99:0/1401337494 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd0bc0cce30 0x7fd0bc15aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:45.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.105+0000 7fd0c0c3a700 1 --2- 172.21.15.99:0/1401337494 >> v2:172.21.15.191:3300/0 conn(0x7fd0bc06aa00 0x7fd0bc0b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:45.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.105+0000 7fd0baffd700 1 -- 172.21.15.99:0/1401337494 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0a4003570 con 0x7fd0bc0b6580 2023-10-08T00:14:45.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.106+0000 7fd0baffd700 1 -- 172.21.15.99:0/1401337494 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4114632466 0 0) 0x7fd0a4003570 con 0x7fd0bc0b6580 2023-10-08T00:14:45.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.106+0000 7fd0baffd700 1 -- 172.21.15.99:0/1401337494 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd0bc0cce30 msgr2=0x7fd0bc15aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:45.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.106+0000 7fd0baffd700 1 --2- 172.21.15.99:0/1401337494 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd0bc0cce30 0x7fd0bc15aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:45.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.106+0000 7fd0baffd700 1 -- 172.21.15.99:0/1401337494 >> v2:172.21.15.191:3300/0 conn(0x7fd0bc06aa00 msgr2=0x7fd0bc0b5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:45.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.106+0000 7fd0baffd700 1 --2- 172.21.15.99:0/1401337494 >> v2:172.21.15.191:3300/0 conn(0x7fd0bc06aa00 0x7fd0bc0b5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:45.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.106+0000 7fd0baffd700 1 -- 172.21.15.99:0/1401337494 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0bc0b4720 con 0x7fd0bc0b6580 2023-10-08T00:14:45.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.106+0000 7fd0baffd700 1 -- 172.21.15.99:0/1401337494 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7fd0b0001470 con 0x7fd0bc0b6580 2023-10-08T00:14:45.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.106+0000 7fd0baffd700 1 -- 172.21.15.99:0/1401337494 >> v1:172.21.15.99:6789/0 conn(0x7fd0bc0b6580 legacy=0x7fd0bc06a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:45.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.106+0000 7fd0baffd700 1 --2- 172.21.15.99:0/1401337494 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0a4003a60 0x7fd0a4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:45.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.106+0000 7fd0baffd700 1 --2- 172.21.15.99:0/1401337494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0a4004810 0x7fd0a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:45.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.106+0000 7fd0baffd700 1 --2- 172.21.15.99:0/1401337494 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd0bc0cce30 0x7fd0a40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:45.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.106+0000 7fd0baffd700 1 -- 172.21.15.99:0/1401337494 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0bc0cd8d0 con 0x7fd0a4004810 2023-10-08T00:14:45.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.106+0000 7fd0baffd700 1 -- 172.21.15.99:0/1401337494 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0bc0b4440 con 0x7fd0a4003a60 2023-10-08T00:14:45.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.106+0000 7fd0baffd700 1 -- 172.21.15.99:0/1401337494 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a400ba60 con 0x7fd0bc0cce30 2023-10-08T00:14:45.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.106+0000 7fd0bb7fe700 1 --2- 172.21.15.99:0/1401337494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0a4004810 0x7fd0a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:45.113 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.106+0000 7fd0bbfff700 1 --2- 172.21.15.99:0/1401337494 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0a4003a60 0x7fd0a4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:45.113 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.106+0000 7fd0c0c3a700 1 --2- 172.21.15.99:0/1401337494 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd0bc0cce30 0x7fd0a40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:45.113 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.106+0000 7fd0bb7fe700 1 -- 172.21.15.99:0/1401337494 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd0bc0cce30 msgr2=0x7fd0a40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:45.113 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.106+0000 7fd0bb7fe700 1 --2- 172.21.15.99:0/1401337494 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd0bc0cce30 0x7fd0a40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:45.113 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.106+0000 7fd0bb7fe700 1 -- 172.21.15.99:0/1401337494 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0a4003a60 msgr2=0x7fd0a4003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:45.113 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.106+0000 7fd0bb7fe700 1 --2- 172.21.15.99:0/1401337494 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0a4003a60 0x7fd0a4003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:45.113 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.106+0000 7fd0bb7fe700 1 -- 172.21.15.99:0/1401337494 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a4003570 con 0x7fd0a4004810 2023-10-08T00:14:45.114 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.106+0000 7fd0bb7fe700 1 --2- 172.21.15.99:0/1401337494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0a4004810 0x7fd0a4004c80 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fd0b000c3c0 tx=0x7fd0bc0cdca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acd32b77844b7582 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:45.114 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.106+0000 7fd0baffd700 1 -- 172.21.15.99:0/1401337494 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b000cc50 con 0x7fd0a4004810 2023-10-08T00:14:45.114 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.106+0000 7fd0baffd700 1 -- 172.21.15.99:0/1401337494 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0b000cdb0 con 0x7fd0a4004810 2023-10-08T00:14:45.114 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.106+0000 7fd0baffd700 1 -- 172.21.15.99:0/1401337494 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b0005b70 con 0x7fd0a4004810 2023-10-08T00:14:45.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.107+0000 7fd0c143b700 1 -- 172.21.15.99:0/1401337494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0a4004810 msgr2=0x7fd0a4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:45.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.107+0000 7fd0c143b700 1 --2- 172.21.15.99:0/1401337494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0a4004810 0x7fd0a4004c80 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fd0b000c3c0 tx=0x7fd0bc0cdca0 comp rx=0 tx=0).stop 2023-10-08T00:14:45.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.107+0000 7fd0c0c3a700 1 -- 172.21.15.99:0/1401337494 reap_dead start 2023-10-08T00:14:45.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.107+0000 7fd0c143b700 1 -- 172.21.15.99:0/1401337494 shutdown_connections 2023-10-08T00:14:45.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.107+0000 7fd0c143b700 1 -- 172.21.15.99:0/1401337494 >> 172.21.15.99:0/1401337494 conn(0x7fd0bc068cb0 msgr2=0x7fd0bc0690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:45.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.107+0000 7fd0c143b700 1 -- 172.21.15.99:0/1401337494 shutdown_connections 2023-10-08T00:14:45.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.107+0000 7fd0c143b700 1 -- 172.21.15.99:0/1401337494 wait complete. 2023-10-08T00:14:45.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.108+0000 7fd0c143b700 1 Processor -- start 2023-10-08T00:14:45.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.108+0000 7fd0c143b700 1 -- start start 2023-10-08T00:14:45.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.109+0000 7fd0c143b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd0a4004810 0x7fd0bc169460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:45.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.109+0000 7fd0c143b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0bc15ebc0 0x7fd0bc15f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:45.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.109+0000 7fd0c143b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0bc154890 0x7fd0bc154d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:45.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.109+0000 7fd0c143b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0bc156a40 con 0x7fd0bc154890 2023-10-08T00:14:45.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.109+0000 7fd0c143b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0bc155240 con 0x7fd0bc15ebc0 2023-10-08T00:14:45.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.109+0000 7fd0c143b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0bc15f640 con 0x7fd0a4004810 2023-10-08T00:14:45.119 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.109+0000 7fd0c0c3a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0bc154890 0x7fd0bc154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:45.119 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.109+0000 7fd0c0c3a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0bc154890 0x7fd0bc154d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45660/0 (socket says 172.21.15.99:45660) 2023-10-08T00:14:45.119 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.109+0000 7fd0c0c3a700 1 -- 172.21.15.99:0/2842775741 learned_addr learned my addr 172.21.15.99:0/2842775741 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:45.119 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.109+0000 7fd0bbfff700 1 --2- 172.21.15.99:0/2842775741 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd0a4004810 0x7fd0bc169460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:45.119 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.109+0000 7fd0bb7fe700 1 --2- 172.21.15.99:0/2842775741 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0bc15ebc0 0x7fd0bc15f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:45.119 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.109+0000 7fd0c0c3a700 1 -- 172.21.15.99:0/2842775741 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd0a4004810 msgr2=0x7fd0bc169460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:45.119 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.109+0000 7fd0c0c3a700 1 --2- 172.21.15.99:0/2842775741 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd0a4004810 0x7fd0bc169460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:45.119 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.109+0000 7fd0c0c3a700 1 -- 172.21.15.99:0/2842775741 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0bc15ebc0 msgr2=0x7fd0bc15f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:45.119 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.109+0000 7fd0c0c3a700 1 --2- 172.21.15.99:0/2842775741 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0bc15ebc0 0x7fd0bc15f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:45.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.109+0000 7fd0c0c3a700 1 -- 172.21.15.99:0/2842775741 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0bc1575a0 con 0x7fd0bc154890 2023-10-08T00:14:45.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.110+0000 7fd0c0c3a700 1 --2- 172.21.15.99:0/2842775741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0bc154890 0x7fd0bc154d00 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc06ae00 tx=0x7fd0b400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:45.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.110+0000 7fd0a3fff700 1 -- 172.21.15.99:0/2842775741 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b400e040 con 0x7fd0bc154890 2023-10-08T00:14:45.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.110+0000 7fd0a3fff700 1 -- 172.21.15.99:0/2842775741 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0b400f040 con 0x7fd0bc154890 2023-10-08T00:14:45.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.110+0000 7fd0a3fff700 1 -- 172.21.15.99:0/2842775741 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b400fda0 con 0x7fd0bc154890 2023-10-08T00:14:45.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.110+0000 7fd0c143b700 1 -- 172.21.15.99:0/2842775741 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0bc1699a0 con 0x7fd0bc154890 2023-10-08T00:14:45.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.110+0000 7fd0c143b700 1 -- 172.21.15.99:0/2842775741 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0bc156f50 con 0x7fd0bc154890 2023-10-08T00:14:45.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.111+0000 7fd0c143b700 1 -- 172.21.15.99:0/2842775741 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0bc157730 con 0x7fd0bc154890 2023-10-08T00:14:45.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.111+0000 7fd0a3fff700 1 -- 172.21.15.99:0/2842775741 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd0b4015640 con 0x7fd0bc154890 2023-10-08T00:14:45.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.112+0000 7fd0a3fff700 1 --2- 172.21.15.99:0/2842775741 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd0a8066f30 0x7fd0a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:45.121 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.112+0000 7fd0a3fff700 1 -- 172.21.15.99:0/2842775741 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd0b4084dd0 con 0x7fd0bc154890 2023-10-08T00:14:45.121 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.112+0000 7fd0bbfff700 1 --2- 172.21.15.99:0/2842775741 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd0a8066f30 0x7fd0a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:45.121 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.112+0000 7fd0bbfff700 1 --2- 172.21.15.99:0/2842775741 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd0a8066f30 0x7fd0a80693e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac006f00 tx=0x7fd0ac0091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:45.121 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.116+0000 7fd0a3fff700 1 -- 172.21.15.99:0/2842775741 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd0b4052040 con 0x7fd0bc154890 2023-10-08T00:14:45.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:45.342+0000 7fd0c143b700 1 -- 172.21.15.99:0/2842775741 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd0bc0b4a00 con 0x7fd0bc154890 2023-10-08T00:14:46.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.326+0000 7fd0a3fff700 1 -- 172.21.15.99:0/2842775741 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v39) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd0b4055660 con 0x7fd0bc154890 2023-10-08T00:14:46.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.329+0000 7fd0c143b700 1 -- 172.21.15.99:0/2842775741 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd0a8066f30 msgr2=0x7fd0a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:46.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.329+0000 7fd0c143b700 1 --2- 172.21.15.99:0/2842775741 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd0a8066f30 0x7fd0a80693e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac006f00 tx=0x7fd0ac0091a0 comp rx=0 tx=0).stop 2023-10-08T00:14:46.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.329+0000 7fd0c143b700 1 -- 172.21.15.99:0/2842775741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0bc154890 msgr2=0x7fd0bc154d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:46.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.329+0000 7fd0c143b700 1 --2- 172.21.15.99:0/2842775741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0bc154890 0x7fd0bc154d00 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc06ae00 tx=0x7fd0b400b040 comp rx=0 tx=0).stop 2023-10-08T00:14:46.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.329+0000 7fd0c143b700 1 -- 172.21.15.99:0/2842775741 shutdown_connections 2023-10-08T00:14:46.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.329+0000 7fd0c143b700 1 --2- 172.21.15.99:0/2842775741 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd0a8066f30 0x7fd0a80693e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:46.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.329+0000 7fd0c143b700 1 --2- 172.21.15.99:0/2842775741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0bc154890 0x7fd0bc154d00 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:46.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.329+0000 7fd0c143b700 1 --2- 172.21.15.99:0/2842775741 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd0a4004810 0x7fd0bc169460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:46.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.329+0000 7fd0c143b700 1 --2- 172.21.15.99:0/2842775741 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0bc15ebc0 0x7fd0bc15f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:46.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.329+0000 7fd0c143b700 1 -- 172.21.15.99:0/2842775741 >> 172.21.15.99:0/2842775741 conn(0x7fd0bc068cb0 msgr2=0x7fd0bc1507c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:46.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.329+0000 7fd0c143b700 1 -- 172.21.15.99:0/2842775741 shutdown_connections 2023-10-08T00:14:46.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.329+0000 7fd0c143b700 1 -- 172.21.15.99:0/2842775741 wait complete. 2023-10-08T00:14:46.345 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T00:14:46.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.467+0000 7fd8e492e700 1 Processor -- start 2023-10-08T00:14:46.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.467+0000 7fd8e492e700 1 -- start start 2023-10-08T00:14:46.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.468+0000 7fd8e492e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8e015b9a0 0x7fd8e00caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:46.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.468+0000 7fd8e492e700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd8e006d370 0x7fd8e006d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:46.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.468+0000 7fd8e492e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8e015bdf0 con 0x7fd8e0152300 2023-10-08T00:14:46.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.468+0000 7fd8e492e700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8e00b1400 con 0x7fd8e006d370 2023-10-08T00:14:46.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.468+0000 7fd8e492e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e015c8d0 con 0x7fd8e015b9a0 2023-10-08T00:14:46.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.468+0000 7fd8ded9d700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd8e006d370 0x7fd8e006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:46.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.468+0000 7fd8ded9d700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd8e006d370 0x7fd8e006d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45668/0 (socket says 172.21.15.99:45668) 2023-10-08T00:14:46.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.468+0000 7fd8ded9d700 1 -- 172.21.15.99:0/781799526 learned_addr learned my addr 172.21.15.99:0/781799526 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:46.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.468+0000 7fd8d5d9b700 1 --2- 172.21.15.99:0/781799526 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8e015b9a0 0x7fd8e00caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:46.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.469+0000 7fd8ded9d700 1 -- 172.21.15.99:0/781799526 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8e015b9a0 msgr2=0x7fd8e00caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:46.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.469+0000 7fd8ded9d700 1 --2- 172.21.15.99:0/781799526 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8e015b9a0 0x7fd8e00caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:46.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.469+0000 7fd8ded9d700 1 -- 172.21.15.99:0/781799526 >> v1:172.21.15.191:6789/0 conn(0x7fd8e0152300 legacy=0x7fd8e0158df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:46.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.469+0000 7fd8ded9d700 1 -- 172.21.15.99:0/781799526 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e00c9370 con 0x7fd8e006d370 2023-10-08T00:14:46.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.469+0000 7fd8ded9d700 1 --2- 172.21.15.99:0/781799526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8e006d370 0x7fd8e006d790 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fd8d8009f20 tx=0x7fd8d8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44516e816b1740f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:46.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.469+0000 7fd8ddd9b700 1 -- 172.21.15.99:0/781799526 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d800e070 con 0x7fd8e006d370 2023-10-08T00:14:46.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.469+0000 7fd8ddd9b700 1 -- 172.21.15.99:0/781799526 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8d800b9d0 con 0x7fd8e006d370 2023-10-08T00:14:46.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.469+0000 7fd8ddd9b700 1 -- 172.21.15.99:0/781799526 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d80092e0 con 0x7fd8e006d370 2023-10-08T00:14:46.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.469+0000 7fd8e492e700 1 -- 172.21.15.99:0/781799526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8e006d370 msgr2=0x7fd8e006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:46.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.469+0000 7fd8e492e700 1 --2- 172.21.15.99:0/781799526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8e006d370 0x7fd8e006d790 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fd8d8009f20 tx=0x7fd8d8009840 comp rx=0 tx=0).stop 2023-10-08T00:14:46.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.469+0000 7fd8e492e700 1 -- 172.21.15.99:0/781799526 shutdown_connections 2023-10-08T00:14:46.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.469+0000 7fd8e492e700 1 --2- 172.21.15.99:0/781799526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8e006d370 0x7fd8e006d790 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:46.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.469+0000 7fd8e492e700 1 --2- 172.21.15.99:0/781799526 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8e015b9a0 0x7fd8e00caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:46.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.469+0000 7fd8e492e700 1 -- 172.21.15.99:0/781799526 >> 172.21.15.99:0/781799526 conn(0x7fd8e0069870 msgr2=0x7fd8e0069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:46.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.469+0000 7fd8e492e700 1 -- 172.21.15.99:0/781799526 shutdown_connections 2023-10-08T00:14:46.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.469+0000 7fd8e492e700 1 -- 172.21.15.99:0/781799526 wait complete. 2023-10-08T00:14:46.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.471+0000 7fd8e492e700 1 Processor -- start 2023-10-08T00:14:46.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.471+0000 7fd8e492e700 1 -- start start 2023-10-08T00:14:46.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.471+0000 7fd8e492e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8e006d370 0x7fd8e0150300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:46.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.471+0000 7fd8e492e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8e0152300 0x7fd8e00680f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:46.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.471+0000 7fd8e492e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8e015b9a0 0x7fd8e00c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:46.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.471+0000 7fd8e492e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e00689d0 con 0x7fd8e0152300 2023-10-08T00:14:46.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.471+0000 7fd8e492e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e0068cb0 con 0x7fd8e015b9a0 2023-10-08T00:14:46.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.471+0000 7fd8e492e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e0068f90 con 0x7fd8e006d370 2023-10-08T00:14:46.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.471+0000 7fd8d5d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8e0152300 0x7fd8e00680f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:46.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.471+0000 7fd8d5d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8e0152300 0x7fd8e00680f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45672/0 (socket says 172.21.15.99:45672) 2023-10-08T00:14:46.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.471+0000 7fd8d5d9b700 1 -- 172.21.15.99:0/2483950850 learned_addr learned my addr 172.21.15.99:0/2483950850 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:46.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.472+0000 7fd8ded9d700 1 --2- 172.21.15.99:0/2483950850 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8e015b9a0 0x7fd8e00c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:46.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.472+0000 7fd8de59c700 1 --2- 172.21.15.99:0/2483950850 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8e006d370 0x7fd8e0150300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:46.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.472+0000 7fd8d5d9b700 1 -- 172.21.15.99:0/2483950850 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8e006d370 msgr2=0x7fd8e0150300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:46.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.472+0000 7fd8d5d9b700 1 --2- 172.21.15.99:0/2483950850 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8e006d370 0x7fd8e0150300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:46.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.472+0000 7fd8d5d9b700 1 -- 172.21.15.99:0/2483950850 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8e015b9a0 msgr2=0x7fd8e00c6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:46.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.472+0000 7fd8d5d9b700 1 --2- 172.21.15.99:0/2483950850 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8e015b9a0 0x7fd8e00c6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:46.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.472+0000 7fd8d5d9b700 1 -- 172.21.15.99:0/2483950850 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d8009ae0 con 0x7fd8e0152300 2023-10-08T00:14:46.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.472+0000 7fd8d5d9b700 1 --2- 172.21.15.99:0/2483950850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8e0152300 0x7fd8e00680f0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fd8c800e930 tx=0x7fd8c80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:46.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.472+0000 7fd8d6ffd700 1 -- 172.21.15.99:0/2483950850 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c800eed0 con 0x7fd8e0152300 2023-10-08T00:14:46.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.472+0000 7fd8d6ffd700 1 -- 172.21.15.99:0/2483950850 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8c800c850 con 0x7fd8e0152300 2023-10-08T00:14:46.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.472+0000 7fd8e492e700 1 -- 172.21.15.99:0/2483950850 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8e016a490 con 0x7fd8e0152300 2023-10-08T00:14:46.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.472+0000 7fd8d6ffd700 1 -- 172.21.15.99:0/2483950850 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c800cbf0 con 0x7fd8e0152300 2023-10-08T00:14:46.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.472+0000 7fd8e492e700 1 -- 172.21.15.99:0/2483950850 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8e0150840 con 0x7fd8e0152300 2023-10-08T00:14:46.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.473+0000 7fd8e492e700 1 -- 172.21.15.99:0/2483950850 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8e0069150 con 0x7fd8e0152300 2023-10-08T00:14:46.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.473+0000 7fd8d6ffd700 1 -- 172.21.15.99:0/2483950850 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd8c8013470 con 0x7fd8e0152300 2023-10-08T00:14:46.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.474+0000 7fd8d6ffd700 1 --2- 172.21.15.99:0/2483950850 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd8c0066f30 0x7fd8c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:46.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.474+0000 7fd8d6ffd700 1 -- 172.21.15.99:0/2483950850 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd8c8084760 con 0x7fd8e0152300 2023-10-08T00:14:46.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.474+0000 7fd8de59c700 1 --2- 172.21.15.99:0/2483950850 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd8c0066f30 0x7fd8c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:46.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.474+0000 7fd8de59c700 1 --2- 172.21.15.99:0/2483950850 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd8c0066f30 0x7fd8c00693e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0003860 tx=0x7fd8d0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:46.479 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.479+0000 7fd8d6ffd700 1 -- 172.21.15.99:0/2483950850 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd8c80519d0 con 0x7fd8e0152300 2023-10-08T00:14:46.702 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:46.701+0000 7fd8e492e700 1 -- 172.21.15.99:0/2483950850 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd8e0068b10 con 0x7fd8e0152300 2023-10-08T00:14:47.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.327+0000 7fd8d6ffd700 1 -- 172.21.15.99:0/2483950850 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v40) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd8e0068b10 con 0x7fd8e0152300 2023-10-08T00:14:47.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.329+0000 7fd8e492e700 1 -- 172.21.15.99:0/2483950850 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd8c0066f30 msgr2=0x7fd8c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:47.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.329+0000 7fd8e492e700 1 --2- 172.21.15.99:0/2483950850 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd8c0066f30 0x7fd8c00693e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0003860 tx=0x7fd8d0009040 comp rx=0 tx=0).stop 2023-10-08T00:14:47.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.329+0000 7fd8e492e700 1 -- 172.21.15.99:0/2483950850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8e0152300 msgr2=0x7fd8e00680f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:47.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.329+0000 7fd8e492e700 1 --2- 172.21.15.99:0/2483950850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8e0152300 0x7fd8e00680f0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fd8c800e930 tx=0x7fd8c80091a0 comp rx=0 tx=0).stop 2023-10-08T00:14:47.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.329+0000 7fd8e492e700 1 -- 172.21.15.99:0/2483950850 shutdown_connections 2023-10-08T00:14:47.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.329+0000 7fd8e492e700 1 --2- 172.21.15.99:0/2483950850 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd8c0066f30 0x7fd8c00693e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:47.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.329+0000 7fd8e492e700 1 --2- 172.21.15.99:0/2483950850 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8e015b9a0 0x7fd8e00c6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:47.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.329+0000 7fd8e492e700 1 --2- 172.21.15.99:0/2483950850 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8e006d370 0x7fd8e0150300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:47.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.329+0000 7fd8e492e700 1 --2- 172.21.15.99:0/2483950850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8e0152300 0x7fd8e00680f0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:47.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.329+0000 7fd8e492e700 1 -- 172.21.15.99:0/2483950850 >> 172.21.15.99:0/2483950850 conn(0x7fd8e0069870 msgr2=0x7fd8e00cc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:47.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.329+0000 7fd8e492e700 1 -- 172.21.15.99:0/2483950850 shutdown_connections 2023-10-08T00:14:47.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.329+0000 7fd8e492e700 1 -- 172.21.15.99:0/2483950850 wait complete. 2023-10-08T00:14:47.344 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:14:47.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.459+0000 7fe68272a700 1 Processor -- start 2023-10-08T00:14:47.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.459+0000 7fe68272a700 1 -- start start 2023-10-08T00:14:47.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.459+0000 7fe68272a700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fe67c157870 0x7fe67c15b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:47.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.459+0000 7fe68272a700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe67c157c40 0x7fe67c158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:47.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.459+0000 7fe68272a700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe67c15cbb0 con 0x7fe67c15a620 2023-10-08T00:14:47.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.459+0000 7fe68272a700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe67c15bfa0 con 0x7fe67c157c40 2023-10-08T00:14:47.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.459+0000 7fe68272a700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe67c15c280 con 0x7fe67c157870 2023-10-08T00:14:47.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.459+0000 7fe680cc7700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe67c157c40 0x7fe67c158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:47.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.459+0000 7fe67b7fe700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fe67c15a620 0x7fe67c15d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:34718/0 (socket says 172.21.15.99:34718) 2023-10-08T00:14:47.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.459+0000 7fe67b7fe700 1 -- 172.21.15.99:0/2714643113 learned_addr learned my addr 172.21.15.99:0/2714643113 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:14:47.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.459+0000 7fe67bfff700 1 --2- 172.21.15.99:0/2714643113 >> v2:172.21.15.191:3300/0 conn(0x7fe67c157870 0x7fe67c15b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:47.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.459+0000 7fe67affd700 1 -- 172.21.15.99:0/2714643113 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 853061737 0 0) 0x7fe67c15cbb0 con 0x7fe67c15a620 2023-10-08T00:14:47.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.459+0000 7fe67affd700 1 -- 172.21.15.99:0/2714643113 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe664003570 con 0x7fe67c15a620 2023-10-08T00:14:47.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.459+0000 7fe680cc7700 1 -- 172.21.15.99:0/2714643113 >> v2:172.21.15.191:3300/0 conn(0x7fe67c157870 msgr2=0x7fe67c15b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:47.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.459+0000 7fe680cc7700 1 --2- 172.21.15.99:0/2714643113 >> v2:172.21.15.191:3300/0 conn(0x7fe67c157870 0x7fe67c15b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:47.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.459+0000 7fe680cc7700 1 -- 172.21.15.99:0/2714643113 >> v1:172.21.15.99:6789/0 conn(0x7fe67c15a620 legacy=0x7fe67c15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:47.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.459+0000 7fe680cc7700 1 -- 172.21.15.99:0/2714643113 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe67c15c560 con 0x7fe67c157c40 2023-10-08T00:14:47.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.459+0000 7fe67bfff700 1 --2- 172.21.15.99:0/2714643113 >> v2:172.21.15.191:3300/0 conn(0x7fe67c157870 0x7fe67c15b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:14:47.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.459+0000 7fe680cc7700 1 --2- 172.21.15.99:0/2714643113 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe67c157c40 0x7fe67c158110 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fe67c069d80 tx=0x7fe674009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39cc75c4555e56e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:47.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.459+0000 7fe67affd700 1 -- 172.21.15.99:0/2714643113 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67400e070 con 0x7fe67c157c40 2023-10-08T00:14:47.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.459+0000 7fe67affd700 1 -- 172.21.15.99:0/2714643113 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe67400bb60 con 0x7fe67c157c40 2023-10-08T00:14:47.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.459+0000 7fe67affd700 1 -- 172.21.15.99:0/2714643113 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe674012430 con 0x7fe67c157c40 2023-10-08T00:14:47.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.460+0000 7fe68272a700 1 -- 172.21.15.99:0/2714643113 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe67c157c40 msgr2=0x7fe67c158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:47.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.460+0000 7fe68272a700 1 --2- 172.21.15.99:0/2714643113 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe67c157c40 0x7fe67c158110 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fe67c069d80 tx=0x7fe674009780 comp rx=0 tx=0).stop 2023-10-08T00:14:47.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.460+0000 7fe68272a700 1 -- 172.21.15.99:0/2714643113 shutdown_connections 2023-10-08T00:14:47.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.460+0000 7fe68272a700 1 --2- 172.21.15.99:0/2714643113 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe67c157c40 0x7fe67c158110 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:47.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.460+0000 7fe68272a700 1 --2- 172.21.15.99:0/2714643113 >> v2:172.21.15.191:3300/0 conn(0x7fe67c157870 0x7fe67c15b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:47.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.460+0000 7fe68272a700 1 -- 172.21.15.99:0/2714643113 >> 172.21.15.99:0/2714643113 conn(0x7fe67c068ce0 msgr2=0x7fe67c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:47.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.460+0000 7fe68272a700 1 -- 172.21.15.99:0/2714643113 shutdown_connections 2023-10-08T00:14:47.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.460+0000 7fe68272a700 1 -- 172.21.15.99:0/2714643113 wait complete. 2023-10-08T00:14:47.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.462+0000 7fe68272a700 1 Processor -- start 2023-10-08T00:14:47.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.462+0000 7fe68272a700 1 -- start start 2023-10-08T00:14:47.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.463+0000 7fe68272a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe67c157870 0x7fe67c14d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:47.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.463+0000 7fe68272a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe67c157c40 0x7fe67c151830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:47.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.463+0000 7fe68272a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe67c15a620 0x7fe67c151d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:47.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.463+0000 7fe68272a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67c1522b0 con 0x7fe67c157c40 2023-10-08T00:14:47.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.463+0000 7fe68272a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67c160a80 con 0x7fe67c15a620 2023-10-08T00:14:47.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.463+0000 7fe68272a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe67c0b0c10 con 0x7fe67c157870 2023-10-08T00:14:47.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.463+0000 7fe67b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe67c157c40 0x7fe67c151830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:47.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.463+0000 7fe67b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe67c157c40 0x7fe67c151830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45688/0 (socket says 172.21.15.99:45688) 2023-10-08T00:14:47.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.463+0000 7fe67b7fe700 1 -- 172.21.15.99:0/3669648331 learned_addr learned my addr 172.21.15.99:0/3669648331 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:47.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.463+0000 7fe67b7fe700 1 -- 172.21.15.99:0/3669648331 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe67c157870 msgr2=0x7fe67c14d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:47.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.463+0000 7fe67b7fe700 1 --2- 172.21.15.99:0/3669648331 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe67c157870 0x7fe67c14d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:47.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.463+0000 7fe67b7fe700 1 -- 172.21.15.99:0/3669648331 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe67c15a620 msgr2=0x7fe67c151d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:47.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.463+0000 7fe67b7fe700 1 --2- 172.21.15.99:0/3669648331 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe67c15a620 0x7fe67c151d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:47.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.463+0000 7fe67b7fe700 1 -- 172.21.15.99:0/3669648331 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe674009a20 con 0x7fe67c157c40 2023-10-08T00:14:47.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.463+0000 7fe67b7fe700 1 --2- 172.21.15.99:0/3669648331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe67c157c40 0x7fe67c151830 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fe67000c560 tx=0x7fe670007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:47.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.463+0000 7fe663fff700 1 -- 172.21.15.99:0/3669648331 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe670005c00 con 0x7fe67c157c40 2023-10-08T00:14:47.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.463+0000 7fe663fff700 1 -- 172.21.15.99:0/3669648331 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe670005d60 con 0x7fe67c157c40 2023-10-08T00:14:47.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.463+0000 7fe68272a700 1 -- 172.21.15.99:0/3669648331 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe67c0b11a0 con 0x7fe67c157c40 2023-10-08T00:14:47.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.463+0000 7fe663fff700 1 -- 172.21.15.99:0/3669648331 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe670007300 con 0x7fe67c157c40 2023-10-08T00:14:47.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.463+0000 7fe68272a700 1 -- 172.21.15.99:0/3669648331 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe67c14d5e0 con 0x7fe67c157c40 2023-10-08T00:14:47.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.463+0000 7fe68272a700 1 -- 172.21.15.99:0/3669648331 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe67c04c3c0 con 0x7fe67c157c40 2023-10-08T00:14:47.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.465+0000 7fe663fff700 1 -- 172.21.15.99:0/3669648331 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe67001b3b0 con 0x7fe67c157c40 2023-10-08T00:14:47.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.465+0000 7fe663fff700 1 --2- 172.21.15.99:0/3669648331 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe668066f30 0x7fe6680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:47.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.465+0000 7fe663fff700 1 -- 172.21.15.99:0/3669648331 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe670083a80 con 0x7fe67c157c40 2023-10-08T00:14:47.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.465+0000 7fe67bfff700 1 --2- 172.21.15.99:0/3669648331 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe668066f30 0x7fe6680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:47.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.465+0000 7fe67bfff700 1 --2- 172.21.15.99:0/3669648331 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe668066f30 0x7fe6680693e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe66c006fd0 tx=0x7fe66c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:47.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.469+0000 7fe663fff700 1 -- 172.21.15.99:0/3669648331 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe670050cf0 con 0x7fe67c157c40 2023-10-08T00:14:47.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.691+0000 7fe68272a700 1 -- 172.21.15.99:0/3669648331 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe67c14d7a0 con 0x7fe67c157c40 2023-10-08T00:14:47.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.692+0000 7fe663fff700 1 -- 172.21.15.99:0/3669648331 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4599 (secure 0 0 0) 0x7fe670017020 con 0x7fe67c157c40 2023-10-08T00:14:47.692 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:14:47.692 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5650,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/2166055423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2166055423},{"type":"v1","addr":"172.21.15.99:6834","nonce":2166055423}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5665,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/4087832300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":4087832300},{"type":"v1","addr":"172.21.15.191:6836","nonce":4087832300}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5668,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/730261712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":730261712},{"type":"v1","addr":"172.21.15.191:6837","nonce":730261712}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:14:43.671064+0000","modified":"2023-10-08T00:14:47.326922+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6039},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6039":{"gid":6039,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/104443412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":104443412},{"type":"v1","addr":"172.21.15.99:6835","nonce":104443412}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-08T00:14:47.694 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.694+0000 7fe68272a700 1 -- 172.21.15.99:0/3669648331 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe668066f30 msgr2=0x7fe6680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:47.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.694+0000 7fe68272a700 1 --2- 172.21.15.99:0/3669648331 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe668066f30 0x7fe6680693e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe66c006fd0 tx=0x7fe66c008040 comp rx=0 tx=0).stop 2023-10-08T00:14:47.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.694+0000 7fe68272a700 1 -- 172.21.15.99:0/3669648331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe67c157c40 msgr2=0x7fe67c151830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:47.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.694+0000 7fe68272a700 1 --2- 172.21.15.99:0/3669648331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe67c157c40 0x7fe67c151830 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fe67000c560 tx=0x7fe670007dd0 comp rx=0 tx=0).stop 2023-10-08T00:14:47.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.694+0000 7fe68272a700 1 -- 172.21.15.99:0/3669648331 shutdown_connections 2023-10-08T00:14:47.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.694+0000 7fe68272a700 1 --2- 172.21.15.99:0/3669648331 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe668066f30 0x7fe6680693e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:47.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.694+0000 7fe68272a700 1 --2- 172.21.15.99:0/3669648331 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe67c15a620 0x7fe67c151d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:47.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.694+0000 7fe68272a700 1 --2- 172.21.15.99:0/3669648331 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe67c157870 0x7fe67c14d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:47.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.694+0000 7fe68272a700 1 --2- 172.21.15.99:0/3669648331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe67c157c40 0x7fe67c151830 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:47.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.694+0000 7fe68272a700 1 -- 172.21.15.99:0/3669648331 >> 172.21.15.99:0/3669648331 conn(0x7fe67c068ce0 msgr2=0x7fe67c1595d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:47.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.694+0000 7fe68272a700 1 -- 172.21.15.99:0/3669648331 shutdown_connections 2023-10-08T00:14:47.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.694+0000 7fe68272a700 1 -- 172.21.15.99:0/3669648331 wait complete. 2023-10-08T00:14:47.696 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 40 2023-10-08T00:14:47.706 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:14:47.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.820+0000 7ff6f9767700 1 Processor -- start 2023-10-08T00:14:47.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.820+0000 7ff6f9767700 1 -- start start 2023-10-08T00:14:47.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.820+0000 7ff6f9767700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff6f4157c40 0x7ff6f4158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:47.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.821+0000 7ff6f9767700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6f415cbb0 con 0x7ff6f4157870 2023-10-08T00:14:47.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.821+0000 7ff6f9767700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6f415bfa0 con 0x7ff6f415a620 2023-10-08T00:14:47.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.821+0000 7ff6f9767700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6f415bdb0 con 0x7ff6f4157c40 2023-10-08T00:14:47.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.821+0000 7ff6f2ffd700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7ff6f4157870 0x7ff6f415b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:34726/0 (socket says 172.21.15.99:34726) 2023-10-08T00:14:47.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.821+0000 7ff6f2ffd700 1 -- 172.21.15.99:0/3304320886 learned_addr learned my addr 172.21.15.99:0/3304320886 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:14:47.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.821+0000 7ff6f1ffb700 1 -- 172.21.15.99:0/3304320886 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 512817586 0 0) 0x7ff6f415cbb0 con 0x7ff6f4157870 2023-10-08T00:14:47.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.821+0000 7ff6f1ffb700 1 -- 172.21.15.99:0/3304320886 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6e0003570 con 0x7ff6f4157870 2023-10-08T00:14:47.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.821+0000 7ff6f37fe700 1 --2- 172.21.15.99:0/3304320886 >> v2:172.21.15.191:3300/0 conn(0x7ff6f4157c40 0x7ff6f4158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:47.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.821+0000 7ff6f1ffb700 1 -- 172.21.15.99:0/3304320886 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2680659336 0 0) 0x7ff6e0003570 con 0x7ff6f4157870 2023-10-08T00:14:47.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.821+0000 7ff6f1ffb700 1 -- 172.21.15.99:0/3304320886 >> v2:172.21.15.191:3300/0 conn(0x7ff6f4157c40 msgr2=0x7ff6f4158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:47.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.821+0000 7ff6f1ffb700 1 --2- 172.21.15.99:0/3304320886 >> v2:172.21.15.191:3300/0 conn(0x7ff6f4157c40 0x7ff6f4158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:47.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.821+0000 7ff6f1ffb700 1 -- 172.21.15.99:0/3304320886 >> v1:172.21.15.191:6789/0 conn(0x7ff6f415a620 legacy=0x7ff6f415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:47.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.821+0000 7ff6f1ffb700 1 -- 172.21.15.99:0/3304320886 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f415c280 con 0x7ff6f4157870 2023-10-08T00:14:47.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.821+0000 7ff6f1ffb700 1 -- 172.21.15.99:0/3304320886 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7ff6ec0017f0 con 0x7ff6f4157870 2023-10-08T00:14:47.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.821+0000 7ff6f1ffb700 1 -- 172.21.15.99:0/3304320886 >> v1:172.21.15.99:6789/0 conn(0x7ff6f4157870 legacy=0x7ff6f415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:47.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.821+0000 7ff6f1ffb700 1 --2- 172.21.15.99:0/3304320886 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6e0003a60 0x7ff6e0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:47.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.821+0000 7ff6f1ffb700 1 --2- 172.21.15.99:0/3304320886 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6e0004810 0x7ff6e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:47.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.821+0000 7ff6f1ffb700 1 --2- 172.21.15.99:0/3304320886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6e00051c0 0x7ff6e000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:47.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.821+0000 7ff6f1ffb700 1 -- 172.21.15.99:0/3304320886 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e000c130 con 0x7ff6e00051c0 2023-10-08T00:14:47.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.821+0000 7ff6f1ffb700 1 -- 172.21.15.99:0/3304320886 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e000c2a0 con 0x7ff6e0004810 2023-10-08T00:14:47.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.821+0000 7ff6f1ffb700 1 -- 172.21.15.99:0/3304320886 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e000c460 con 0x7ff6e0003a60 2023-10-08T00:14:47.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.821+0000 7ff6f37fe700 1 --2- 172.21.15.99:0/3304320886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6e00051c0 0x7ff6e000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:47.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.821+0000 7ff6f2ffd700 1 --2- 172.21.15.99:0/3304320886 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6e0003a60 0x7ff6e0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:47.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.821+0000 7ff6f27fc700 1 --2- 172.21.15.99:0/3304320886 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6e0004810 0x7ff6e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:47.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.822+0000 7ff6f37fe700 1 -- 172.21.15.99:0/3304320886 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6e0003a60 msgr2=0x7ff6e0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:47.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.822+0000 7ff6f37fe700 1 --2- 172.21.15.99:0/3304320886 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6e0003a60 0x7ff6e0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:47.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.822+0000 7ff6f37fe700 1 -- 172.21.15.99:0/3304320886 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6e0004810 msgr2=0x7ff6e0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:47.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.822+0000 7ff6f37fe700 1 --2- 172.21.15.99:0/3304320886 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6e0004810 0x7ff6e0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:47.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.822+0000 7ff6f37fe700 1 -- 172.21.15.99:0/3304320886 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e0003570 con 0x7ff6e00051c0 2023-10-08T00:14:47.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.822+0000 7ff6f27fc700 1 --2- 172.21.15.99:0/3304320886 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6e0004810 0x7ff6e0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:14:47.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.822+0000 7ff6f37fe700 1 -- 172.21.15.99:0/3304320886 reap_dead start 2023-10-08T00:14:47.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.822+0000 7ff6f37fe700 1 --2- 172.21.15.99:0/3304320886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6e00051c0 0x7ff6e000ba60 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ff6f41580c0 tx=0x7ff6e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c322b219296711 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:47.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.822+0000 7ff6f1ffb700 1 -- 172.21.15.99:0/3304320886 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e400e710 con 0x7ff6e00051c0 2023-10-08T00:14:47.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.822+0000 7ff6f1ffb700 1 -- 172.21.15.99:0/3304320886 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e400e870 con 0x7ff6e00051c0 2023-10-08T00:14:47.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.822+0000 7ff6f1ffb700 1 -- 172.21.15.99:0/3304320886 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e400ec30 con 0x7ff6e00051c0 2023-10-08T00:14:47.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.822+0000 7ff6f9767700 1 -- 172.21.15.99:0/3304320886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6e00051c0 msgr2=0x7ff6e000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:47.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.822+0000 7ff6f9767700 1 --2- 172.21.15.99:0/3304320886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6e00051c0 0x7ff6e000ba60 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ff6f41580c0 tx=0x7ff6e40091a0 comp rx=0 tx=0).stop 2023-10-08T00:14:47.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.822+0000 7ff6f9767700 1 -- 172.21.15.99:0/3304320886 shutdown_connections 2023-10-08T00:14:47.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.822+0000 7ff6f9767700 1 --2- 172.21.15.99:0/3304320886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6e00051c0 0x7ff6e000ba60 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:47.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.822+0000 7ff6f9767700 1 -- 172.21.15.99:0/3304320886 >> 172.21.15.99:0/3304320886 conn(0x7ff6f4068ce0 msgr2=0x7ff6f40690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:47.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.822+0000 7ff6f9767700 1 -- 172.21.15.99:0/3304320886 shutdown_connections 2023-10-08T00:14:47.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.822+0000 7ff6f9767700 1 -- 172.21.15.99:0/3304320886 wait complete. 2023-10-08T00:14:47.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.824+0000 7ff6f9767700 1 Processor -- start 2023-10-08T00:14:47.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.824+0000 7ff6f9767700 1 -- start start 2023-10-08T00:14:47.830 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.824+0000 7ff6f9767700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6e00051c0 0x7ff6f413f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:47.830 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.824+0000 7ff6f9767700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6f4150b20 0x7ff6f416e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:47.830 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.824+0000 7ff6f9767700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6f414fdb0 0x7ff6f4150230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:47.830 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.824+0000 7ff6f9767700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f4153860 con 0x7ff6f4150b20 2023-10-08T00:14:47.830 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.824+0000 7ff6f9767700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f414e9e0 con 0x7ff6f414fdb0 2023-10-08T00:14:47.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.824+0000 7ff6f9767700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f414e140 con 0x7ff6e00051c0 2023-10-08T00:14:47.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.824+0000 7ff6f27fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6f4150b20 0x7ff6f416e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:47.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.825+0000 7ff6f27fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6f4150b20 0x7ff6f416e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45710/0 (socket says 172.21.15.99:45710) 2023-10-08T00:14:47.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.825+0000 7ff6f27fc700 1 -- 172.21.15.99:0/837838851 learned_addr learned my addr 172.21.15.99:0/837838851 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:47.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.825+0000 7ff6f37fe700 1 --2- 172.21.15.99:0/837838851 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6f414fdb0 0x7ff6f4150230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:47.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.825+0000 7ff6f2ffd700 1 --2- 172.21.15.99:0/837838851 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6e00051c0 0x7ff6f413f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:47.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.825+0000 7ff6f27fc700 1 -- 172.21.15.99:0/837838851 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6e00051c0 msgr2=0x7ff6f413f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:47.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.825+0000 7ff6f27fc700 1 --2- 172.21.15.99:0/837838851 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6e00051c0 0x7ff6f413f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:47.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.825+0000 7ff6f27fc700 1 -- 172.21.15.99:0/837838851 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6f414fdb0 msgr2=0x7ff6f4150230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:47.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.825+0000 7ff6f27fc700 1 --2- 172.21.15.99:0/837838851 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6f414fdb0 0x7ff6f4150230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:47.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.825+0000 7ff6f27fc700 1 -- 172.21.15.99:0/837838851 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f416ec10 con 0x7ff6f4150b20 2023-10-08T00:14:47.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.825+0000 7ff6f37fe700 1 --2- 172.21.15.99:0/837838851 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6f414fdb0 0x7ff6f4150230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:14:47.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.825+0000 7ff6f2ffd700 1 --2- 172.21.15.99:0/837838851 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6e00051c0 0x7ff6f413f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:14:47.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.825+0000 7ff6f27fc700 1 --2- 172.21.15.99:0/837838851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6f4150b20 0x7ff6f416e6d0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc00e550 tx=0x7ff6dc009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:47.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.825+0000 7ff6eaffd700 1 -- 172.21.15.99:0/837838851 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc011460 con 0x7ff6f4150b20 2023-10-08T00:14:47.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.825+0000 7ff6eaffd700 1 -- 172.21.15.99:0/837838851 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6dc010040 con 0x7ff6f4150b20 2023-10-08T00:14:47.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.825+0000 7ff6f9767700 1 -- 172.21.15.99:0/837838851 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6f4153f30 con 0x7ff6f4150b20 2023-10-08T00:14:47.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.825+0000 7ff6eaffd700 1 -- 172.21.15.99:0/837838851 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc011810 con 0x7ff6f4150b20 2023-10-08T00:14:47.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.825+0000 7ff6f9767700 1 -- 172.21.15.99:0/837838851 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6f414e670 con 0x7ff6f4150b20 2023-10-08T00:14:47.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.826+0000 7ff6f9767700 1 -- 172.21.15.99:0/837838851 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6f404c3c0 con 0x7ff6f4150b20 2023-10-08T00:14:47.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.827+0000 7ff6eaffd700 1 -- 172.21.15.99:0/837838851 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff6dc011bc0 con 0x7ff6f4150b20 2023-10-08T00:14:47.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.827+0000 7ff6eaffd700 1 --2- 172.21.15.99:0/837838851 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff6d4066f30 0x7ff6d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:47.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.827+0000 7ff6eaffd700 1 -- 172.21.15.99:0/837838851 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff6dc0836c0 con 0x7ff6f4150b20 2023-10-08T00:14:47.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.827+0000 7ff6f2ffd700 1 --2- 172.21.15.99:0/837838851 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff6d4066f30 0x7ff6d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:47.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.828+0000 7ff6f2ffd700 1 --2- 172.21.15.99:0/837838851 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff6d4066f30 0x7ff6d40693e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec00a0c0 tx=0x7ff6ec006040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:47.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:47.831+0000 7ff6eaffd700 1 -- 172.21.15.99:0/837838851 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff6dc04d5a0 con 0x7ff6f4150b20 2023-10-08T00:14:48.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.014+0000 7ff6f9767700 1 -- 172.21.15.99:0/837838851 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6f415c560 con 0x7ff6f4150b20 2023-10-08T00:14:48.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.015+0000 7ff6eaffd700 1 -- 172.21.15.99:0/837838851 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+15353 (secure 0 0 0) 0x7ff6dc0509e0 con 0x7ff6f4150b20 2023-10-08T00:14:48.016 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:14:48.016 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":86,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:14:45.299289+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:14:41.496823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-08T00:14:42.640029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:14:48.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.017+0000 7ff6f9767700 1 -- 172.21.15.99:0/837838851 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff6d4066f30 msgr2=0x7ff6d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:48.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.017+0000 7ff6f9767700 1 --2- 172.21.15.99:0/837838851 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff6d4066f30 0x7ff6d40693e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec00a0c0 tx=0x7ff6ec006040 comp rx=0 tx=0).stop 2023-10-08T00:14:48.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.017+0000 7ff6f9767700 1 -- 172.21.15.99:0/837838851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6f4150b20 msgr2=0x7ff6f416e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:48.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.017+0000 7ff6f9767700 1 --2- 172.21.15.99:0/837838851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6f4150b20 0x7ff6f416e6d0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc00e550 tx=0x7ff6dc009040 comp rx=0 tx=0).stop 2023-10-08T00:14:48.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.017+0000 7ff6f9767700 1 -- 172.21.15.99:0/837838851 shutdown_connections 2023-10-08T00:14:48.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.017+0000 7ff6f9767700 1 --2- 172.21.15.99:0/837838851 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff6d4066f30 0x7ff6d40693e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:48.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.017+0000 7ff6f9767700 1 --2- 172.21.15.99:0/837838851 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6f414fdb0 0x7ff6f4150230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:48.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.017+0000 7ff6f9767700 1 --2- 172.21.15.99:0/837838851 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6e00051c0 0x7ff6f413f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:48.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.017+0000 7ff6f9767700 1 --2- 172.21.15.99:0/837838851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6f4150b20 0x7ff6f416e6d0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:48.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.017+0000 7ff6f9767700 1 -- 172.21.15.99:0/837838851 >> 172.21.15.99:0/837838851 conn(0x7ff6f4068ce0 msgr2=0x7ff6f40b47c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:48.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.017+0000 7ff6f9767700 1 -- 172.21.15.99:0/837838851 shutdown_connections 2023-10-08T00:14:48.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.017+0000 7ff6f9767700 1 -- 172.21.15.99:0/837838851 wait complete. 2023-10-08T00:14:48.030 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T00:14:48.030 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T00:14:48.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.144+0000 7f20cf916700 1 Processor -- start 2023-10-08T00:14:48.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.144+0000 7f20cf916700 1 -- start start 2023-10-08T00:14:48.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.145+0000 7f20cf916700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f20c80a7a20 0x7f20c80af520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:48.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.145+0000 7f20cf916700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20c80ae760 con 0x7f20c80a7df0 2023-10-08T00:14:48.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.145+0000 7f20cf916700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20c80aea40 con 0x7f20c81572a0 2023-10-08T00:14:48.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.145+0000 7f20cf916700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20c80aed20 con 0x7f20c80a7a20 2023-10-08T00:14:48.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.145+0000 7f20cd6b2700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f20c80a7a20 0x7f20c80af520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:48.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.145+0000 7f20cceb1700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f20c80a7df0 0x7f20c80b29c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:34738/0 (socket says 172.21.15.99:34738) 2023-10-08T00:14:48.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.145+0000 7f20cceb1700 1 -- 172.21.15.99:0/891501956 learned_addr learned my addr 172.21.15.99:0/891501956 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:14:48.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.146+0000 7f20cd6b2700 1 -- 172.21.15.99:0/891501956 >> v1:172.21.15.191:6789/0 conn(0x7f20c81572a0 legacy=0x7f20c81576b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:48.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.146+0000 7f20cd6b2700 1 -- 172.21.15.99:0/891501956 >> v1:172.21.15.99:6789/0 conn(0x7f20c80a7df0 legacy=0x7f20c80b29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:48.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.146+0000 7f20cd6b2700 1 -- 172.21.15.99:0/891501956 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c80af000 con 0x7f20c80a7a20 2023-10-08T00:14:48.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.146+0000 7f20cd6b2700 1 --2- 172.21.15.99:0/891501956 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20c80a7a20 0x7f20c80af520 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f20c40098e0 tx=0x7f20c40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d005e854279b8a26 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:48.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.146+0000 7f20bffff700 1 -- 172.21.15.99:0/891501956 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c4009e80 con 0x7f20c80a7a20 2023-10-08T00:14:48.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.146+0000 7f20bffff700 1 -- 172.21.15.99:0/891501956 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20c400b8f0 con 0x7f20c80a7a20 2023-10-08T00:14:48.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.146+0000 7f20bffff700 1 -- 172.21.15.99:0/891501956 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c400bc70 con 0x7f20c80a7a20 2023-10-08T00:14:48.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.146+0000 7f20cf916700 1 -- 172.21.15.99:0/891501956 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20c80a7a20 msgr2=0x7f20c80af520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:48.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.146+0000 7f20cf916700 1 --2- 172.21.15.99:0/891501956 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20c80a7a20 0x7f20c80af520 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f20c40098e0 tx=0x7f20c40091a0 comp rx=0 tx=0).stop 2023-10-08T00:14:48.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.147+0000 7f20cf916700 1 -- 172.21.15.99:0/891501956 shutdown_connections 2023-10-08T00:14:48.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.147+0000 7f20cf916700 1 --2- 172.21.15.99:0/891501956 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20c80a7a20 0x7f20c80af520 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:48.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.147+0000 7f20cf916700 1 -- 172.21.15.99:0/891501956 >> 172.21.15.99:0/891501956 conn(0x7f20c80cd1e0 msgr2=0x7f20c80cd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:48.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.147+0000 7f20cf916700 1 -- 172.21.15.99:0/891501956 shutdown_connections 2023-10-08T00:14:48.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.147+0000 7f20cf916700 1 -- 172.21.15.99:0/891501956 wait complete. 2023-10-08T00:14:48.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.148+0000 7f20cf916700 1 Processor -- start 2023-10-08T00:14:48.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.148+0000 7f20cf916700 1 -- start start 2023-10-08T00:14:48.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.148+0000 7f20cf916700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20c80a7a20 0x7f20c8146470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:48.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.149+0000 7f20cf916700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20c80a7df0 0x7f20c80ab730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:48.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.149+0000 7f20cf916700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f20c81572a0 0x7f20c80abc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:48.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.149+0000 7f20cf916700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c809a950 con 0x7f20c80a7a20 2023-10-08T00:14:48.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.149+0000 7f20cf916700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c8099dd0 con 0x7f20c80a7df0 2023-10-08T00:14:48.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.149+0000 7f20cf916700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20c809a0b0 con 0x7f20c81572a0 2023-10-08T00:14:48.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.149+0000 7f20cd6b2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20c80a7a20 0x7f20c8146470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:48.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.149+0000 7f20cd6b2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20c80a7a20 0x7f20c8146470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45740/0 (socket says 172.21.15.99:45740) 2023-10-08T00:14:48.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.149+0000 7f20cd6b2700 1 -- 172.21.15.99:0/1933668236 learned_addr learned my addr 172.21.15.99:0/1933668236 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:48.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.149+0000 7f20cceb1700 1 --2- 172.21.15.99:0/1933668236 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20c80a7df0 0x7f20c80ab730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:48.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.149+0000 7f20cdeb3700 1 --2- 172.21.15.99:0/1933668236 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f20c81572a0 0x7f20c80abc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:48.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.149+0000 7f20cd6b2700 1 -- 172.21.15.99:0/1933668236 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f20c81572a0 msgr2=0x7f20c80abc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:48.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.149+0000 7f20cd6b2700 1 --2- 172.21.15.99:0/1933668236 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f20c81572a0 0x7f20c80abc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:48.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.149+0000 7f20cd6b2700 1 -- 172.21.15.99:0/1933668236 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20c80a7df0 msgr2=0x7f20c80ab730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:48.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.149+0000 7f20cd6b2700 1 --2- 172.21.15.99:0/1933668236 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20c80a7df0 0x7f20c80ab730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:48.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.149+0000 7f20cd6b2700 1 -- 172.21.15.99:0/1933668236 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c4009420 con 0x7f20c80a7a20 2023-10-08T00:14:48.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.149+0000 7f20cceb1700 1 --2- 172.21.15.99:0/1933668236 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20c80a7df0 0x7f20c80ab730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:14:48.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.149+0000 7f20cd6b2700 1 --2- 172.21.15.99:0/1933668236 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20c80a7a20 0x7f20c8146470 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f20c8072020 tx=0x7f20c4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:48.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.149+0000 7f20bd7fa700 1 -- 172.21.15.99:0/1933668236 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c401a420 con 0x7f20c80a7a20 2023-10-08T00:14:48.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.149+0000 7f20bd7fa700 1 -- 172.21.15.99:0/1933668236 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20c401a580 con 0x7f20c80a7a20 2023-10-08T00:14:48.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.149+0000 7f20cf916700 1 -- 172.21.15.99:0/1933668236 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20c809a390 con 0x7f20c80a7a20 2023-10-08T00:14:48.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.149+0000 7f20bd7fa700 1 -- 172.21.15.99:0/1933668236 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c401a940 con 0x7f20c80a7a20 2023-10-08T00:14:48.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.149+0000 7f20cf916700 1 -- 172.21.15.99:0/1933668236 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20c81469b0 con 0x7f20c80a7a20 2023-10-08T00:14:48.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.150+0000 7f20cf916700 1 -- 172.21.15.99:0/1933668236 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20c80cc3d0 con 0x7f20c80a7a20 2023-10-08T00:14:48.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.151+0000 7f20bd7fa700 1 -- 172.21.15.99:0/1933668236 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f20c401acf0 con 0x7f20c80a7a20 2023-10-08T00:14:48.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.151+0000 7f20bd7fa700 1 --2- 172.21.15.99:0/1933668236 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f20ac066f30 0x7f20ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:48.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.151+0000 7f20bd7fa700 1 -- 172.21.15.99:0/1933668236 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f20c4087f60 con 0x7f20c80a7a20 2023-10-08T00:14:48.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.151+0000 7f20cceb1700 1 --2- 172.21.15.99:0/1933668236 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f20ac066f30 0x7f20ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:48.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.152+0000 7f20cceb1700 1 --2- 172.21.15.99:0/1933668236 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f20ac066f30 0x7f20ac0693e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f20c809ab50 tx=0x7f20b8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:48.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.155+0000 7f20bd7fa700 1 -- 172.21.15.99:0/1933668236 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f20c4055250 con 0x7f20c80a7a20 2023-10-08T00:14:48.333 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.332+0000 7f20cf916700 1 -- 172.21.15.99:0/1933668236 --> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f20c804c3c0 con 0x7f20ac066f30 2023-10-08T00:14:48.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.341+0000 7f20bd7fa700 1 -- 172.21.15.99:0/1933668236 <== mgr.4099 v2:172.21.15.191:6832/85985 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284849 (secure 0 0 0) 0x7f20c804c3c0 con 0x7f20ac066f30 2023-10-08T00:14:48.345 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:14:48.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.344+0000 7f20cf916700 1 -- 172.21.15.99:0/1933668236 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f20ac066f30 msgr2=0x7f20ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:48.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.344+0000 7f20cf916700 1 --2- 172.21.15.99:0/1933668236 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f20ac066f30 0x7f20ac0693e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f20c809ab50 tx=0x7f20b8009040 comp rx=0 tx=0).stop 2023-10-08T00:14:48.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.344+0000 7f20cf916700 1 -- 172.21.15.99:0/1933668236 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20c80a7a20 msgr2=0x7f20c8146470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:48.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.344+0000 7f20cf916700 1 --2- 172.21.15.99:0/1933668236 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20c80a7a20 0x7f20c8146470 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f20c8072020 tx=0x7f20c4017000 comp rx=0 tx=0).stop 2023-10-08T00:14:48.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.344+0000 7f20cf916700 1 -- 172.21.15.99:0/1933668236 shutdown_connections 2023-10-08T00:14:48.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.344+0000 7f20cf916700 1 --2- 172.21.15.99:0/1933668236 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f20ac066f30 0x7f20ac0693e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:48.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.344+0000 7f20cf916700 1 --2- 172.21.15.99:0/1933668236 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f20c81572a0 0x7f20c80abc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:48.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.344+0000 7f20cf916700 1 --2- 172.21.15.99:0/1933668236 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20c80a7a20 0x7f20c8146470 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:48.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.344+0000 7f20cf916700 1 --2- 172.21.15.99:0/1933668236 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20c80a7df0 0x7f20c80ab730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:48.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.344+0000 7f20cf916700 1 -- 172.21.15.99:0/1933668236 >> 172.21.15.99:0/1933668236 conn(0x7f20c80cd1e0 msgr2=0x7f20c81564d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:48.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.344+0000 7f20cf916700 1 -- 172.21.15.99:0/1933668236 shutdown_connections 2023-10-08T00:14:48.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.344+0000 7f20cf916700 1 -- 172.21.15.99:0/1933668236 wait complete. 2023-10-08T00:14:48.350 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2023-10-08T00:14:48.356 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":199,"stamp":"2023-10-08T00:14:48.081007+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":4741,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1557758,"statfs":{"total":1600000000,"available":1595145140,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.794598"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.314368+0000","last_change":"2023-10-08T00:14:44.314214+0000","last_active":"2023-10-08T00:14:44.314368+0000","last_peered":"2023-10-08T00:14:44.314368+0000","last_clean":"2023-10-08T00:14:44.314368+0000","last_became_active":"2023-10-08T00:14:44.313766+0000","last_became_peered":"2023-10-08T00:14:44.313766+0000","last_unstale":"2023-10-08T00:14:44.314368+0000","last_undegraded":"2023-10-08T00:14:44.314368+0000","last_fullsized":"2023-10-08T00:14:44.314368+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:33:13.734581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.312659+0000","last_change":"2023-10-08T00:14:44.312494+0000","last_active":"2023-10-08T00:14:44.312659+0000","last_peered":"2023-10-08T00:14:44.312659+0000","last_clean":"2023-10-08T00:14:44.312659+0000","last_became_active":"2023-10-08T00:14:44.312205+0000","last_became_peered":"2023-10-08T00:14:44.312205+0000","last_unstale":"2023-10-08T00:14:44.312659+0000","last_undegraded":"2023-10-08T00:14:44.312659+0000","last_fullsized":"2023-10-08T00:14:44.312659+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:02:01.051881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.313626+0000","last_change":"2023-10-08T00:14:44.313561+0000","last_active":"2023-10-08T00:14:44.313626+0000","last_peered":"2023-10-08T00:14:44.313626+0000","last_clean":"2023-10-08T00:14:44.313626+0000","last_became_active":"2023-10-08T00:14:44.313034+0000","last_became_peered":"2023-10-08T00:14:44.313034+0000","last_unstale":"2023-10-08T00:14:44.313626+0000","last_undegraded":"2023-10-08T00:14:44.313626+0000","last_fullsized":"2023-10-08T00:14:44.313626+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:22.234376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.312164+0000","last_change":"2023-10-08T00:14:44.311989+0000","last_active":"2023-10-08T00:14:44.312164+0000","last_peered":"2023-10-08T00:14:44.312164+0000","last_clean":"2023-10-08T00:14:44.312164+0000","last_became_active":"2023-10-08T00:14:44.311648+0000","last_became_peered":"2023-10-08T00:14:44.311648+0000","last_unstale":"2023-10-08T00:14:44.312164+0000","last_undegraded":"2023-10-08T00:14:44.312164+0000","last_fullsized":"2023-10-08T00:14:44.312164+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:35:38.085349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.313897+0000","last_change":"2023-10-08T00:14:44.313757+0000","last_active":"2023-10-08T00:14:44.313897+0000","last_peered":"2023-10-08T00:14:44.313897+0000","last_clean":"2023-10-08T00:14:44.313897+0000","last_became_active":"2023-10-08T00:14:44.313528+0000","last_became_peered":"2023-10-08T00:14:44.313528+0000","last_unstale":"2023-10-08T00:14:44.313897+0000","last_undegraded":"2023-10-08T00:14:44.313897+0000","last_fullsized":"2023-10-08T00:14:44.313897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:07:31.992120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.308712+0000","last_change":"2023-10-08T00:14:44.307713+0000","last_active":"2023-10-08T00:14:44.308712+0000","last_peered":"2023-10-08T00:14:44.308712+0000","last_clean":"2023-10-08T00:14:44.308712+0000","last_became_active":"2023-10-08T00:14:44.307467+0000","last_became_peered":"2023-10-08T00:14:44.307467+0000","last_unstale":"2023-10-08T00:14:44.308712+0000","last_undegraded":"2023-10-08T00:14:44.308712+0000","last_fullsized":"2023-10-08T00:14:44.308712+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:21:40.091830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.313020+0000","last_change":"2023-10-08T00:14:44.312916+0000","last_active":"2023-10-08T00:14:44.313020+0000","last_peered":"2023-10-08T00:14:44.313020+0000","last_clean":"2023-10-08T00:14:44.313020+0000","last_became_active":"2023-10-08T00:14:44.312247+0000","last_became_peered":"2023-10-08T00:14:44.312247+0000","last_unstale":"2023-10-08T00:14:44.313020+0000","last_undegraded":"2023-10-08T00:14:44.313020+0000","last_fullsized":"2023-10-08T00:14:44.313020+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:26:21.446680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.313189+0000","last_change":"2023-10-08T00:14:44.312725+0000","last_active":"2023-10-08T00:14:44.313189+0000","last_peered":"2023-10-08T00:14:44.313189+0000","last_clean":"2023-10-08T00:14:44.313189+0000","last_became_active":"2023-10-08T00:14:44.312475+0000","last_became_peered":"2023-10-08T00:14:44.312475+0000","last_unstale":"2023-10-08T00:14:44.313189+0000","last_undegraded":"2023-10-08T00:14:44.313189+0000","last_fullsized":"2023-10-08T00:14:44.313189+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:05:00.741767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.312125+0000","last_change":"2023-10-08T00:14:44.311433+0000","last_active":"2023-10-08T00:14:44.312125+0000","last_peered":"2023-10-08T00:14:44.312125+0000","last_clean":"2023-10-08T00:14:44.312125+0000","last_became_active":"2023-10-08T00:14:44.310859+0000","last_became_peered":"2023-10-08T00:14:44.310859+0000","last_unstale":"2023-10-08T00:14:44.312125+0000","last_undegraded":"2023-10-08T00:14:44.312125+0000","last_fullsized":"2023-10-08T00:14:44.312125+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:27:52.229759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.312689+0000","last_change":"2023-10-08T00:14:44.312635+0000","last_active":"2023-10-08T00:14:44.312689+0000","last_peered":"2023-10-08T00:14:44.312689+0000","last_clean":"2023-10-08T00:14:44.312689+0000","last_became_active":"2023-10-08T00:14:44.312108+0000","last_became_peered":"2023-10-08T00:14:44.312108+0000","last_unstale":"2023-10-08T00:14:44.312689+0000","last_undegraded":"2023-10-08T00:14:44.312689+0000","last_fullsized":"2023-10-08T00:14:44.312689+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:52:17.712934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.307330+0000","last_change":"2023-10-08T00:14:44.307239+0000","last_active":"2023-10-08T00:14:44.307330+0000","last_peered":"2023-10-08T00:14:44.307330+0000","last_clean":"2023-10-08T00:14:44.307330+0000","last_became_active":"2023-10-08T00:14:44.306964+0000","last_became_peered":"2023-10-08T00:14:44.306964+0000","last_unstale":"2023-10-08T00:14:44.307330+0000","last_undegraded":"2023-10-08T00:14:44.307330+0000","last_fullsized":"2023-10-08T00:14:44.307330+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:38:03.052039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.308182+0000","last_change":"2023-10-08T00:14:44.307951+0000","last_active":"2023-10-08T00:14:44.308182+0000","last_peered":"2023-10-08T00:14:44.308182+0000","last_clean":"2023-10-08T00:14:44.308182+0000","last_became_active":"2023-10-08T00:14:44.307713+0000","last_became_peered":"2023-10-08T00:14:44.307713+0000","last_unstale":"2023-10-08T00:14:44.308182+0000","last_undegraded":"2023-10-08T00:14:44.308182+0000","last_fullsized":"2023-10-08T00:14:44.308182+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:52:24.163704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.314791+0000","last_change":"2023-10-08T00:14:44.314121+0000","last_active":"2023-10-08T00:14:44.314791+0000","last_peered":"2023-10-08T00:14:44.314791+0000","last_clean":"2023-10-08T00:14:44.314791+0000","last_became_active":"2023-10-08T00:14:44.313333+0000","last_became_peered":"2023-10-08T00:14:44.313333+0000","last_unstale":"2023-10-08T00:14:44.314791+0000","last_undegraded":"2023-10-08T00:14:44.314791+0000","last_fullsized":"2023-10-08T00:14:44.314791+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:52:17.712934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.311047+0000","last_change":"2023-10-08T00:14:44.310772+0000","last_active":"2023-10-08T00:14:44.311047+0000","last_peered":"2023-10-08T00:14:44.311047+0000","last_clean":"2023-10-08T00:14:44.311047+0000","last_became_active":"2023-10-08T00:14:44.309865+0000","last_became_peered":"2023-10-08T00:14:44.309865+0000","last_unstale":"2023-10-08T00:14:44.311047+0000","last_undegraded":"2023-10-08T00:14:44.311047+0000","last_fullsized":"2023-10-08T00:14:44.311047+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:50:50.579902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.311987+0000","last_change":"2023-10-08T00:14:44.311937+0000","last_active":"2023-10-08T00:14:44.311987+0000","last_peered":"2023-10-08T00:14:44.311987+0000","last_clean":"2023-10-08T00:14:44.311987+0000","last_became_active":"2023-10-08T00:14:44.310528+0000","last_became_peered":"2023-10-08T00:14:44.310528+0000","last_unstale":"2023-10-08T00:14:44.311987+0000","last_undegraded":"2023-10-08T00:14:44.311987+0000","last_fullsized":"2023-10-08T00:14:44.311987+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:47:52.704667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.311282+0000","last_change":"2023-10-08T00:14:44.311178+0000","last_active":"2023-10-08T00:14:44.311282+0000","last_peered":"2023-10-08T00:14:44.311282+0000","last_clean":"2023-10-08T00:14:44.311282+0000","last_became_active":"2023-10-08T00:14:44.310766+0000","last_became_peered":"2023-10-08T00:14:44.310766+0000","last_unstale":"2023-10-08T00:14:44.311282+0000","last_undegraded":"2023-10-08T00:14:44.311282+0000","last_fullsized":"2023-10-08T00:14:44.311282+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:18:05.629462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.312246+0000","last_change":"2023-10-08T00:14:44.312184+0000","last_active":"2023-10-08T00:14:44.312246+0000","last_peered":"2023-10-08T00:14:44.312246+0000","last_clean":"2023-10-08T00:14:44.312246+0000","last_became_active":"2023-10-08T00:14:44.310422+0000","last_became_peered":"2023-10-08T00:14:44.310422+0000","last_unstale":"2023-10-08T00:14:44.312246+0000","last_undegraded":"2023-10-08T00:14:44.312246+0000","last_fullsized":"2023-10-08T00:14:44.312246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:33:18.254151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.312023+0000","last_change":"2023-10-08T00:14:44.311931+0000","last_active":"2023-10-08T00:14:44.312023+0000","last_peered":"2023-10-08T00:14:44.312023+0000","last_clean":"2023-10-08T00:14:44.312023+0000","last_became_active":"2023-10-08T00:14:44.311396+0000","last_became_peered":"2023-10-08T00:14:44.311396+0000","last_unstale":"2023-10-08T00:14:44.312023+0000","last_undegraded":"2023-10-08T00:14:44.312023+0000","last_fullsized":"2023-10-08T00:14:44.312023+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:44:45.812545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.314339+0000","last_change":"2023-10-08T00:14:44.313535+0000","last_active":"2023-10-08T00:14:44.314339+0000","last_peered":"2023-10-08T00:14:44.314339+0000","last_clean":"2023-10-08T00:14:44.314339+0000","last_became_active":"2023-10-08T00:14:44.313306+0000","last_became_peered":"2023-10-08T00:14:44.313306+0000","last_unstale":"2023-10-08T00:14:44.314339+0000","last_undegraded":"2023-10-08T00:14:44.314339+0000","last_fullsized":"2023-10-08T00:14:44.314339+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:31:06.244548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.310921+0000","last_change":"2023-10-08T00:14:44.310816+0000","last_active":"2023-10-08T00:14:44.310921+0000","last_peered":"2023-10-08T00:14:44.310921+0000","last_clean":"2023-10-08T00:14:44.310921+0000","last_became_active":"2023-10-08T00:14:44.310156+0000","last_became_peered":"2023-10-08T00:14:44.310156+0000","last_unstale":"2023-10-08T00:14:44.310921+0000","last_undegraded":"2023-10-08T00:14:44.310921+0000","last_fullsized":"2023-10-08T00:14:44.310921+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:59:35.179200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.316019+0000","last_change":"2023-10-08T00:14:44.315911+0000","last_active":"2023-10-08T00:14:44.316019+0000","last_peered":"2023-10-08T00:14:44.316019+0000","last_clean":"2023-10-08T00:14:44.316019+0000","last_became_active":"2023-10-08T00:14:44.315664+0000","last_became_peered":"2023-10-08T00:14:44.315664+0000","last_unstale":"2023-10-08T00:14:44.316019+0000","last_undegraded":"2023-10-08T00:14:44.316019+0000","last_fullsized":"2023-10-08T00:14:44.316019+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:31:58.708770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.314516+0000","last_change":"2023-10-08T00:14:44.314458+0000","last_active":"2023-10-08T00:14:44.314516+0000","last_peered":"2023-10-08T00:14:44.314516+0000","last_clean":"2023-10-08T00:14:44.314516+0000","last_became_active":"2023-10-08T00:14:44.314241+0000","last_became_peered":"2023-10-08T00:14:44.314241+0000","last_unstale":"2023-10-08T00:14:44.314516+0000","last_undegraded":"2023-10-08T00:14:44.314516+0000","last_fullsized":"2023-10-08T00:14:44.314516+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:50:20.430773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.315866+0000","last_change":"2023-10-08T00:14:44.315817+0000","last_active":"2023-10-08T00:14:44.315866+0000","last_peered":"2023-10-08T00:14:44.315866+0000","last_clean":"2023-10-08T00:14:44.315866+0000","last_became_active":"2023-10-08T00:14:44.315436+0000","last_became_peered":"2023-10-08T00:14:44.315436+0000","last_unstale":"2023-10-08T00:14:44.315866+0000","last_undegraded":"2023-10-08T00:14:44.315866+0000","last_fullsized":"2023-10-08T00:14:44.315866+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:27:52.229759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.306748+0000","last_change":"2023-10-08T00:14:44.306703+0000","last_active":"2023-10-08T00:14:44.306748+0000","last_peered":"2023-10-08T00:14:44.306748+0000","last_clean":"2023-10-08T00:14:44.306748+0000","last_became_active":"2023-10-08T00:14:44.306441+0000","last_became_peered":"2023-10-08T00:14:44.306441+0000","last_unstale":"2023-10-08T00:14:44.306748+0000","last_undegraded":"2023-10-08T00:14:44.306748+0000","last_fullsized":"2023-10-08T00:14:44.306748+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:41:11.473838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.312862+0000","last_change":"2023-10-08T00:14:44.312776+0000","last_active":"2023-10-08T00:14:44.312862+0000","last_peered":"2023-10-08T00:14:44.312862+0000","last_clean":"2023-10-08T00:14:44.312862+0000","last_became_active":"2023-10-08T00:14:44.312414+0000","last_became_peered":"2023-10-08T00:14:44.312414+0000","last_unstale":"2023-10-08T00:14:44.312862+0000","last_undegraded":"2023-10-08T00:14:44.312862+0000","last_fullsized":"2023-10-08T00:14:44.312862+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:43:42.699284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.307299+0000","last_change":"2023-10-08T00:14:44.307119+0000","last_active":"2023-10-08T00:14:44.307299+0000","last_peered":"2023-10-08T00:14:44.307299+0000","last_clean":"2023-10-08T00:14:44.307299+0000","last_became_active":"2023-10-08T00:14:44.306662+0000","last_became_peered":"2023-10-08T00:14:44.306662+0000","last_unstale":"2023-10-08T00:14:44.307299+0000","last_undegraded":"2023-10-08T00:14:44.307299+0000","last_fullsized":"2023-10-08T00:14:44.307299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:01:30.785141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.310904+0000","last_change":"2023-10-08T00:14:44.310747+0000","last_active":"2023-10-08T00:14:44.310904+0000","last_peered":"2023-10-08T00:14:44.310904+0000","last_clean":"2023-10-08T00:14:44.310904+0000","last_became_active":"2023-10-08T00:14:44.309809+0000","last_became_peered":"2023-10-08T00:14:44.309809+0000","last_unstale":"2023-10-08T00:14:44.310904+0000","last_undegraded":"2023-10-08T00:14:44.310904+0000","last_fullsized":"2023-10-08T00:14:44.310904+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:02:49.544839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.316004+0000","last_change":"2023-10-08T00:14:44.315598+0000","last_active":"2023-10-08T00:14:44.316004+0000","last_peered":"2023-10-08T00:14:44.316004+0000","last_clean":"2023-10-08T00:14:44.316004+0000","last_became_active":"2023-10-08T00:14:44.315374+0000","last_became_peered":"2023-10-08T00:14:44.315374+0000","last_unstale":"2023-10-08T00:14:44.316004+0000","last_undegraded":"2023-10-08T00:14:44.316004+0000","last_fullsized":"2023-10-08T00:14:44.316004+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:46:13.400999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.312147+0000","last_change":"2023-10-08T00:14:44.311860+0000","last_active":"2023-10-08T00:14:44.312147+0000","last_peered":"2023-10-08T00:14:44.312147+0000","last_clean":"2023-10-08T00:14:44.312147+0000","last_became_active":"2023-10-08T00:14:44.311178+0000","last_became_peered":"2023-10-08T00:14:44.311178+0000","last_unstale":"2023-10-08T00:14:44.312147+0000","last_undegraded":"2023-10-08T00:14:44.312147+0000","last_fullsized":"2023-10-08T00:14:44.312147+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:17:57.649959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.314353+0000","last_change":"2023-10-08T00:14:44.313917+0000","last_active":"2023-10-08T00:14:44.314353+0000","last_peered":"2023-10-08T00:14:44.314353+0000","last_clean":"2023-10-08T00:14:44.314353+0000","last_became_active":"2023-10-08T00:14:44.313683+0000","last_became_peered":"2023-10-08T00:14:44.313683+0000","last_unstale":"2023-10-08T00:14:44.314353+0000","last_undegraded":"2023-10-08T00:14:44.314353+0000","last_fullsized":"2023-10-08T00:14:44.314353+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:31:35.202882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.315971+0000","last_change":"2023-10-08T00:14:44.315281+0000","last_active":"2023-10-08T00:14:44.315971+0000","last_peered":"2023-10-08T00:14:44.315971+0000","last_clean":"2023-10-08T00:14:44.315971+0000","last_became_active":"2023-10-08T00:14:44.312848+0000","last_became_peered":"2023-10-08T00:14:44.312848+0000","last_unstale":"2023-10-08T00:14:44.315971+0000","last_undegraded":"2023-10-08T00:14:44.315971+0000","last_fullsized":"2023-10-08T00:14:44.315971+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:26:51.881390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.311210+0000","last_change":"2023-10-08T00:14:44.311150+0000","last_active":"2023-10-08T00:14:44.311210+0000","last_peered":"2023-10-08T00:14:44.311210+0000","last_clean":"2023-10-08T00:14:44.311210+0000","last_became_active":"2023-10-08T00:14:44.311012+0000","last_became_peered":"2023-10-08T00:14:44.311012+0000","last_unstale":"2023-10-08T00:14:44.311210+0000","last_undegraded":"2023-10-08T00:14:44.311210+0000","last_fullsized":"2023-10-08T00:14:44.311210+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:47:52.704667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.311250+0000","last_change":"2023-10-08T00:14:44.310661+0000","last_active":"2023-10-08T00:14:44.311250+0000","last_peered":"2023-10-08T00:14:44.311250+0000","last_clean":"2023-10-08T00:14:44.311250+0000","last_became_active":"2023-10-08T00:14:44.310437+0000","last_became_peered":"2023-10-08T00:14:44.310437+0000","last_unstale":"2023-10-08T00:14:44.311250+0000","last_undegraded":"2023-10-08T00:14:44.311250+0000","last_fullsized":"2023-10-08T00:14:44.311250+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:25:53.638687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"85'4","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.323071+0000","last_change":"2023-10-08T00:14:43.296853+0000","last_active":"2023-10-08T00:14:44.323071+0000","last_peered":"2023-10-08T00:14:44.323071+0000","last_clean":"2023-10-08T00:14:44.323071+0000","last_became_active":"2023-10-08T00:14:43.296688+0000","last_became_peered":"2023-10-08T00:14:43.296688+0000","last_unstale":"2023-10-08T00:14:44.323071+0000","last_undegraded":"2023-10-08T00:14:44.323071+0000","last_fullsized":"2023-10-08T00:14:44.323071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:22:58.707158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.314810+0000","last_change":"2023-10-08T00:14:44.314316+0000","last_active":"2023-10-08T00:14:44.314810+0000","last_peered":"2023-10-08T00:14:44.314810+0000","last_clean":"2023-10-08T00:14:44.314810+0000","last_became_active":"2023-10-08T00:14:44.313787+0000","last_became_peered":"2023-10-08T00:14:44.313787+0000","last_unstale":"2023-10-08T00:14:44.314810+0000","last_undegraded":"2023-10-08T00:14:44.314810+0000","last_fullsized":"2023-10-08T00:14:44.314810+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:43:42.699284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"85'1","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.326526+0000","last_change":"2023-10-08T00:14:43.295617+0000","last_active":"2023-10-08T00:14:44.326526+0000","last_peered":"2023-10-08T00:14:44.326526+0000","last_clean":"2023-10-08T00:14:44.326526+0000","last_became_active":"2023-10-08T00:14:43.295397+0000","last_became_peered":"2023-10-08T00:14:43.295397+0000","last_unstale":"2023-10-08T00:14:44.326526+0000","last_undegraded":"2023-10-08T00:14:44.326526+0000","last_fullsized":"2023-10-08T00:14:44.326526+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:01:41.952885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.305180+0000","last_change":"2023-10-08T00:14:44.305058+0000","last_active":"2023-10-08T00:14:44.305180+0000","last_peered":"2023-10-08T00:14:44.305180+0000","last_clean":"2023-10-08T00:14:44.305180+0000","last_became_active":"2023-10-08T00:14:44.304828+0000","last_became_peered":"2023-10-08T00:14:44.304828+0000","last_unstale":"2023-10-08T00:14:44.305180+0000","last_undegraded":"2023-10-08T00:14:44.305180+0000","last_fullsized":"2023-10-08T00:14:44.305180+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:31:58.708770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"85'2","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.322798+0000","last_change":"2023-10-08T00:14:43.294237+0000","last_active":"2023-10-08T00:14:44.322798+0000","last_peered":"2023-10-08T00:14:44.322798+0000","last_clean":"2023-10-08T00:14:44.322798+0000","last_became_active":"2023-10-08T00:14:43.293992+0000","last_became_peered":"2023-10-08T00:14:43.293992+0000","last_unstale":"2023-10-08T00:14:44.322798+0000","last_undegraded":"2023-10-08T00:14:44.322798+0000","last_fullsized":"2023-10-08T00:14:44.322798+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:23:10.497366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.308389+0000","last_change":"2023-10-08T00:14:44.308326+0000","last_active":"2023-10-08T00:14:44.308389+0000","last_peered":"2023-10-08T00:14:44.308389+0000","last_clean":"2023-10-08T00:14:44.308389+0000","last_became_active":"2023-10-08T00:14:44.308153+0000","last_became_peered":"2023-10-08T00:14:44.308153+0000","last_unstale":"2023-10-08T00:14:44.308389+0000","last_undegraded":"2023-10-08T00:14:44.308389+0000","last_fullsized":"2023-10-08T00:14:44.308389+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:10:49.087366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"85'2","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.324309+0000","last_change":"2023-10-08T00:14:43.293651+0000","last_active":"2023-10-08T00:14:44.324309+0000","last_peered":"2023-10-08T00:14:44.324309+0000","last_clean":"2023-10-08T00:14:44.324309+0000","last_became_active":"2023-10-08T00:14:43.293063+0000","last_became_peered":"2023-10-08T00:14:43.293063+0000","last_unstale":"2023-10-08T00:14:44.324309+0000","last_undegraded":"2023-10-08T00:14:44.324309+0000","last_fullsized":"2023-10-08T00:14:44.324309+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:01:41.952885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.305928+0000","last_change":"2023-10-08T00:14:44.305769+0000","last_active":"2023-10-08T00:14:44.305928+0000","last_peered":"2023-10-08T00:14:44.305928+0000","last_clean":"2023-10-08T00:14:44.305928+0000","last_became_active":"2023-10-08T00:14:44.305504+0000","last_became_peered":"2023-10-08T00:14:44.305504+0000","last_unstale":"2023-10-08T00:14:44.305928+0000","last_undegraded":"2023-10-08T00:14:44.305928+0000","last_fullsized":"2023-10-08T00:14:44.305928+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:46:21.620833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"85'1","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.322291+0000","last_change":"2023-10-08T00:14:43.300434+0000","last_active":"2023-10-08T00:14:44.322291+0000","last_peered":"2023-10-08T00:14:44.322291+0000","last_clean":"2023-10-08T00:14:44.322291+0000","last_became_active":"2023-10-08T00:14:43.300297+0000","last_became_peered":"2023-10-08T00:14:43.300297+0000","last_unstale":"2023-10-08T00:14:44.322291+0000","last_undegraded":"2023-10-08T00:14:44.322291+0000","last_fullsized":"2023-10-08T00:14:44.322291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:52:16.707995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.303553+0000","last_change":"2023-10-08T00:14:44.303466+0000","last_active":"2023-10-08T00:14:44.303553+0000","last_peered":"2023-10-08T00:14:44.303553+0000","last_clean":"2023-10-08T00:14:44.303553+0000","last_became_active":"2023-10-08T00:14:44.303243+0000","last_became_peered":"2023-10-08T00:14:44.303243+0000","last_unstale":"2023-10-08T00:14:44.303553+0000","last_undegraded":"2023-10-08T00:14:44.303553+0000","last_fullsized":"2023-10-08T00:14:44.303553+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:47:00.324016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"85'1","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.326611+0000","last_change":"2023-10-08T00:14:43.303622+0000","last_active":"2023-10-08T00:14:44.326611+0000","last_peered":"2023-10-08T00:14:44.326611+0000","last_clean":"2023-10-08T00:14:44.326611+0000","last_became_active":"2023-10-08T00:14:43.303406+0000","last_became_peered":"2023-10-08T00:14:43.303406+0000","last_unstale":"2023-10-08T00:14:44.326611+0000","last_undegraded":"2023-10-08T00:14:44.326611+0000","last_fullsized":"2023-10-08T00:14:44.326611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:32:10.788785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.314882+0000","last_change":"2023-10-08T00:14:44.314120+0000","last_active":"2023-10-08T00:14:44.314882+0000","last_peered":"2023-10-08T00:14:44.314882+0000","last_clean":"2023-10-08T00:14:44.314882+0000","last_became_active":"2023-10-08T00:14:44.313891+0000","last_became_peered":"2023-10-08T00:14:44.313891+0000","last_unstale":"2023-10-08T00:14:44.314882+0000","last_undegraded":"2023-10-08T00:14:44.314882+0000","last_fullsized":"2023-10-08T00:14:44.314882+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:33:22.070538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.306422+0000","last_change":"2023-10-08T00:14:43.298762+0000","last_active":"2023-10-08T00:14:44.306422+0000","last_peered":"2023-10-08T00:14:44.306422+0000","last_clean":"2023-10-08T00:14:44.306422+0000","last_became_active":"2023-10-08T00:14:43.298520+0000","last_became_peered":"2023-10-08T00:14:43.298520+0000","last_unstale":"2023-10-08T00:14:44.306422+0000","last_undegraded":"2023-10-08T00:14:44.306422+0000","last_fullsized":"2023-10-08T00:14:44.306422+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:27:51.224820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.310782+0000","last_change":"2023-10-08T00:14:44.310218+0000","last_active":"2023-10-08T00:14:44.310782+0000","last_peered":"2023-10-08T00:14:44.310782+0000","last_clean":"2023-10-08T00:14:44.310782+0000","last_became_active":"2023-10-08T00:14:44.308468+0000","last_became_peered":"2023-10-08T00:14:44.308468+0000","last_unstale":"2023-10-08T00:14:44.310782+0000","last_undegraded":"2023-10-08T00:14:44.310782+0000","last_fullsized":"2023-10-08T00:14:44.310782+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:33:22.070538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.304327+0000","last_change":"2023-10-08T00:14:43.298044+0000","last_active":"2023-10-08T00:14:44.304327+0000","last_peered":"2023-10-08T00:14:44.304327+0000","last_clean":"2023-10-08T00:14:44.304327+0000","last_became_active":"2023-10-08T00:14:43.295683+0000","last_became_peered":"2023-10-08T00:14:43.295683+0000","last_unstale":"2023-10-08T00:14:44.304327+0000","last_undegraded":"2023-10-08T00:14:44.304327+0000","last_fullsized":"2023-10-08T00:14:44.304327+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:26:20.441741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.311924+0000","last_change":"2023-10-08T00:14:44.311825+0000","last_active":"2023-10-08T00:14:44.311924+0000","last_peered":"2023-10-08T00:14:44.311924+0000","last_clean":"2023-10-08T00:14:44.311924+0000","last_became_active":"2023-10-08T00:14:44.311570+0000","last_became_peered":"2023-10-08T00:14:44.311570+0000","last_unstale":"2023-10-08T00:14:44.311924+0000","last_undegraded":"2023-10-08T00:14:44.311924+0000","last_fullsized":"2023-10-08T00:14:44.311924+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:38:11.833743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.304468+0000","last_change":"2023-10-08T00:14:43.301418+0000","last_active":"2023-10-08T00:14:44.304468+0000","last_peered":"2023-10-08T00:14:44.304468+0000","last_clean":"2023-10-08T00:14:44.304468+0000","last_became_active":"2023-10-08T00:14:43.300950+0000","last_became_peered":"2023-10-08T00:14:43.300950+0000","last_unstale":"2023-10-08T00:14:44.304468+0000","last_undegraded":"2023-10-08T00:14:44.304468+0000","last_fullsized":"2023-10-08T00:14:44.304468+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:02:48.539900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.315988+0000","last_change":"2023-10-08T00:14:44.315424+0000","last_active":"2023-10-08T00:14:44.315988+0000","last_peered":"2023-10-08T00:14:44.315988+0000","last_clean":"2023-10-08T00:14:44.315988+0000","last_became_active":"2023-10-08T00:14:44.313161+0000","last_became_peered":"2023-10-08T00:14:44.313161+0000","last_unstale":"2023-10-08T00:14:44.315988+0000","last_undegraded":"2023-10-08T00:14:44.315988+0000","last_fullsized":"2023-10-08T00:14:44.315988+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:47:00.324016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.309080+0000","last_change":"2023-10-08T00:14:43.294524+0000","last_active":"2023-10-08T00:14:44.309080+0000","last_peered":"2023-10-08T00:14:44.309080+0000","last_clean":"2023-10-08T00:14:44.309080+0000","last_became_active":"2023-10-08T00:14:43.294281+0000","last_became_peered":"2023-10-08T00:14:43.294281+0000","last_unstale":"2023-10-08T00:14:44.309080+0000","last_undegraded":"2023-10-08T00:14:44.309080+0000","last_fullsized":"2023-10-08T00:14:44.309080+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:35:37.080411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.309358+0000","last_change":"2023-10-08T00:14:44.309284+0000","last_active":"2023-10-08T00:14:44.309358+0000","last_peered":"2023-10-08T00:14:44.309358+0000","last_clean":"2023-10-08T00:14:44.309358+0000","last_became_active":"2023-10-08T00:14:44.308537+0000","last_became_peered":"2023-10-08T00:14:44.308537+0000","last_unstale":"2023-10-08T00:14:44.309358+0000","last_undegraded":"2023-10-08T00:14:44.309358+0000","last_fullsized":"2023-10-08T00:14:44.309358+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:29:06.196423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"85'3","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.322204+0000","last_change":"2023-10-08T00:14:43.297383+0000","last_active":"2023-10-08T00:14:44.322204+0000","last_peered":"2023-10-08T00:14:44.322204+0000","last_clean":"2023-10-08T00:14:44.322204+0000","last_became_active":"2023-10-08T00:14:43.296565+0000","last_became_peered":"2023-10-08T00:14:43.296565+0000","last_unstale":"2023-10-08T00:14:44.322204+0000","last_undegraded":"2023-10-08T00:14:44.322204+0000","last_fullsized":"2023-10-08T00:14:44.322204+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:10:48.082428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.312449+0000","last_change":"2023-10-08T00:14:44.312318+0000","last_active":"2023-10-08T00:14:44.312449+0000","last_peered":"2023-10-08T00:14:44.312449+0000","last_clean":"2023-10-08T00:14:44.312449+0000","last_became_active":"2023-10-08T00:14:44.311706+0000","last_became_peered":"2023-10-08T00:14:44.311706+0000","last_unstale":"2023-10-08T00:14:44.312449+0000","last_undegraded":"2023-10-08T00:14:44.312449+0000","last_fullsized":"2023-10-08T00:14:44.312449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:04:04.824912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"85'6","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.326516+0000","last_change":"2023-10-08T00:14:43.295216+0000","last_active":"2023-10-08T00:14:44.326516+0000","last_peered":"2023-10-08T00:14:44.326516+0000","last_clean":"2023-10-08T00:14:44.326516+0000","last_became_active":"2023-10-08T00:14:43.294989+0000","last_became_peered":"2023-10-08T00:14:43.294989+0000","last_unstale":"2023-10-08T00:14:44.326516+0000","last_undegraded":"2023-10-08T00:14:44.326516+0000","last_fullsized":"2023-10-08T00:14:44.326516+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:26:20.441741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.305892+0000","last_change":"2023-10-08T00:14:44.305779+0000","last_active":"2023-10-08T00:14:44.305892+0000","last_peered":"2023-10-08T00:14:44.305892+0000","last_clean":"2023-10-08T00:14:44.305892+0000","last_became_active":"2023-10-08T00:14:44.305560+0000","last_became_peered":"2023-10-08T00:14:44.305560+0000","last_unstale":"2023-10-08T00:14:44.305892+0000","last_undegraded":"2023-10-08T00:14:44.305892+0000","last_fullsized":"2023-10-08T00:14:44.305892+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:09:33.137344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"85'2","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.323938+0000","last_change":"2023-10-08T00:14:43.294068+0000","last_active":"2023-10-08T00:14:44.323938+0000","last_peered":"2023-10-08T00:14:44.323938+0000","last_clean":"2023-10-08T00:14:44.323938+0000","last_became_active":"2023-10-08T00:14:43.293845+0000","last_became_peered":"2023-10-08T00:14:43.293845+0000","last_unstale":"2023-10-08T00:14:44.323938+0000","last_undegraded":"2023-10-08T00:14:44.323938+0000","last_fullsized":"2023-10-08T00:14:44.323938+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:43:41.694345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.307940+0000","last_change":"2023-10-08T00:14:44.307384+0000","last_active":"2023-10-08T00:14:44.307940+0000","last_peered":"2023-10-08T00:14:44.307940+0000","last_clean":"2023-10-08T00:14:44.307940+0000","last_became_active":"2023-10-08T00:14:44.307151+0000","last_became_peered":"2023-10-08T00:14:44.307151+0000","last_unstale":"2023-10-08T00:14:44.307940+0000","last_undegraded":"2023-10-08T00:14:44.307940+0000","last_fullsized":"2023-10-08T00:14:44.307940+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:49:23.052153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.305799+0000","last_change":"2023-10-08T00:14:43.296275+0000","last_active":"2023-10-08T00:14:44.305799+0000","last_peered":"2023-10-08T00:14:44.305799+0000","last_clean":"2023-10-08T00:14:44.305799+0000","last_became_active":"2023-10-08T00:14:43.296041+0000","last_became_peered":"2023-10-08T00:14:43.296041+0000","last_unstale":"2023-10-08T00:14:44.305799+0000","last_undegraded":"2023-10-08T00:14:44.305799+0000","last_fullsized":"2023-10-08T00:14:44.305799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:51:24.797873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.314571+0000","last_change":"2023-10-08T00:14:44.314491+0000","last_active":"2023-10-08T00:14:44.314571+0000","last_peered":"2023-10-08T00:14:44.314571+0000","last_clean":"2023-10-08T00:14:44.314571+0000","last_became_active":"2023-10-08T00:14:44.314252+0000","last_became_peered":"2023-10-08T00:14:44.314252+0000","last_unstale":"2023-10-08T00:14:44.314571+0000","last_undegraded":"2023-10-08T00:14:44.314571+0000","last_fullsized":"2023-10-08T00:14:44.314571+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:54:13.055140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.306932+0000","last_change":"2023-10-08T00:14:43.298585+0000","last_active":"2023-10-08T00:14:44.306932+0000","last_peered":"2023-10-08T00:14:44.306932+0000","last_clean":"2023-10-08T00:14:44.306932+0000","last_became_active":"2023-10-08T00:14:43.298455+0000","last_became_peered":"2023-10-08T00:14:43.298455+0000","last_unstale":"2023-10-08T00:14:44.306932+0000","last_undegraded":"2023-10-08T00:14:44.306932+0000","last_fullsized":"2023-10-08T00:14:44.306932+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:27:24.902226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.314054+0000","last_change":"2023-10-08T00:14:44.313732+0000","last_active":"2023-10-08T00:14:44.314054+0000","last_peered":"2023-10-08T00:14:44.314054+0000","last_clean":"2023-10-08T00:14:44.314054+0000","last_became_active":"2023-10-08T00:14:44.313476+0000","last_became_peered":"2023-10-08T00:14:44.313476+0000","last_unstale":"2023-10-08T00:14:44.314054+0000","last_undegraded":"2023-10-08T00:14:44.314054+0000","last_fullsized":"2023-10-08T00:14:44.314054+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:10:49.087366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"85'8","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.326523+0000","last_change":"2023-10-08T00:14:43.301832+0000","last_active":"2023-10-08T00:14:44.326523+0000","last_peered":"2023-10-08T00:14:44.326523+0000","last_clean":"2023-10-08T00:14:44.326523+0000","last_became_active":"2023-10-08T00:14:43.301610+0000","last_became_peered":"2023-10-08T00:14:43.301610+0000","last_unstale":"2023-10-08T00:14:44.326523+0000","last_undegraded":"2023-10-08T00:14:44.326523+0000","last_fullsized":"2023-10-08T00:14:44.326523+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:10:10.820384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.315206+0000","last_change":"2023-10-08T00:14:44.315143+0000","last_active":"2023-10-08T00:14:44.315206+0000","last_peered":"2023-10-08T00:14:44.315206+0000","last_clean":"2023-10-08T00:14:44.315206+0000","last_became_active":"2023-10-08T00:14:44.314895+0000","last_became_peered":"2023-10-08T00:14:44.314895+0000","last_unstale":"2023-10-08T00:14:44.315206+0000","last_undegraded":"2023-10-08T00:14:44.315206+0000","last_fullsized":"2023-10-08T00:14:44.315206+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:50:20.430773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"85'1","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.322942+0000","last_change":"2023-10-08T00:14:43.301356+0000","last_active":"2023-10-08T00:14:44.322942+0000","last_peered":"2023-10-08T00:14:44.322942+0000","last_clean":"2023-10-08T00:14:44.322942+0000","last_became_active":"2023-10-08T00:14:43.301143+0000","last_became_peered":"2023-10-08T00:14:43.301143+0000","last_unstale":"2023-10-08T00:14:44.322942+0000","last_undegraded":"2023-10-08T00:14:44.322942+0000","last_fullsized":"2023-10-08T00:14:44.322942+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:04:03.819974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.314841+0000","last_change":"2023-10-08T00:14:44.314800+0000","last_active":"2023-10-08T00:14:44.314841+0000","last_peered":"2023-10-08T00:14:44.314841+0000","last_clean":"2023-10-08T00:14:44.314841+0000","last_became_active":"2023-10-08T00:14:44.314638+0000","last_became_peered":"2023-10-08T00:14:44.314638+0000","last_unstale":"2023-10-08T00:14:44.314841+0000","last_undegraded":"2023-10-08T00:14:44.314841+0000","last_fullsized":"2023-10-08T00:14:44.314841+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:13:05.430418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"85'2","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.327091+0000","last_change":"2023-10-08T00:14:43.298391+0000","last_active":"2023-10-08T00:14:44.327091+0000","last_peered":"2023-10-08T00:14:44.327091+0000","last_clean":"2023-10-08T00:14:44.327091+0000","last_became_active":"2023-10-08T00:14:43.298175+0000","last_became_peered":"2023-10-08T00:14:43.298175+0000","last_unstale":"2023-10-08T00:14:44.327091+0000","last_undegraded":"2023-10-08T00:14:44.327091+0000","last_fullsized":"2023-10-08T00:14:44.327091+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:45:12.140067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.314826+0000","last_change":"2023-10-08T00:14:44.314581+0000","last_active":"2023-10-08T00:14:44.314826+0000","last_peered":"2023-10-08T00:14:44.314826+0000","last_clean":"2023-10-08T00:14:44.314826+0000","last_became_active":"2023-10-08T00:14:44.314306+0000","last_became_peered":"2023-10-08T00:14:44.314306+0000","last_unstale":"2023-10-08T00:14:44.314826+0000","last_undegraded":"2023-10-08T00:14:44.314826+0000","last_fullsized":"2023-10-08T00:14:44.314826+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:26:21.446680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"85'1","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.324975+0000","last_change":"2023-10-08T00:14:43.304417+0000","last_active":"2023-10-08T00:14:44.324975+0000","last_peered":"2023-10-08T00:14:44.324975+0000","last_clean":"2023-10-08T00:14:44.324975+0000","last_became_active":"2023-10-08T00:14:43.304168+0000","last_became_peered":"2023-10-08T00:14:43.304168+0000","last_unstale":"2023-10-08T00:14:44.324975+0000","last_undegraded":"2023-10-08T00:14:44.324975+0000","last_fullsized":"2023-10-08T00:14:44.324975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:17:56.645020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.313180+0000","last_change":"2023-10-08T00:14:44.313095+0000","last_active":"2023-10-08T00:14:44.313180+0000","last_peered":"2023-10-08T00:14:44.313180+0000","last_clean":"2023-10-08T00:14:44.313180+0000","last_became_active":"2023-10-08T00:14:44.312843+0000","last_became_peered":"2023-10-08T00:14:44.312843+0000","last_unstale":"2023-10-08T00:14:44.313180+0000","last_undegraded":"2023-10-08T00:14:44.313180+0000","last_fullsized":"2023-10-08T00:14:44.313180+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:45:13.145006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.300817+0000","last_change":"2023-10-08T00:14:43.291042+0000","last_active":"2023-10-08T00:14:44.300817+0000","last_peered":"2023-10-08T00:14:44.300817+0000","last_clean":"2023-10-08T00:14:44.300817+0000","last_became_active":"2023-10-08T00:14:43.290806+0000","last_became_peered":"2023-10-08T00:14:43.290806+0000","last_unstale":"2023-10-08T00:14:44.300817+0000","last_undegraded":"2023-10-08T00:14:44.300817+0000","last_fullsized":"2023-10-08T00:14:44.300817+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:50:49.574964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.310716+0000","last_change":"2023-10-08T00:14:44.310650+0000","last_active":"2023-10-08T00:14:44.310716+0000","last_peered":"2023-10-08T00:14:44.310716+0000","last_clean":"2023-10-08T00:14:44.310716+0000","last_became_active":"2023-10-08T00:14:44.310419+0000","last_became_peered":"2023-10-08T00:14:44.310419+0000","last_unstale":"2023-10-08T00:14:44.310716+0000","last_undegraded":"2023-10-08T00:14:44.310716+0000","last_fullsized":"2023-10-08T00:14:44.310716+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:56:39.552066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.300190+0000","last_change":"2023-10-08T00:14:43.301524+0000","last_active":"2023-10-08T00:14:44.300190+0000","last_peered":"2023-10-08T00:14:44.300190+0000","last_clean":"2023-10-08T00:14:44.300190+0000","last_became_active":"2023-10-08T00:14:43.301054+0000","last_became_peered":"2023-10-08T00:14:43.301054+0000","last_unstale":"2023-10-08T00:14:44.300190+0000","last_undegraded":"2023-10-08T00:14:44.300190+0000","last_fullsized":"2023-10-08T00:14:44.300190+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:39:26.312424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.313117+0000","last_change":"2023-10-08T00:14:44.313067+0000","last_active":"2023-10-08T00:14:44.313117+0000","last_peered":"2023-10-08T00:14:44.313117+0000","last_clean":"2023-10-08T00:14:44.313117+0000","last_became_active":"2023-10-08T00:14:44.312844+0000","last_became_peered":"2023-10-08T00:14:44.312844+0000","last_unstale":"2023-10-08T00:14:44.313117+0000","last_undegraded":"2023-10-08T00:14:44.313117+0000","last_fullsized":"2023-10-08T00:14:44.313117+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:52:24.163704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.310629+0000","last_change":"2023-10-08T00:14:43.299330+0000","last_active":"2023-10-08T00:14:44.310629+0000","last_peered":"2023-10-08T00:14:44.310629+0000","last_clean":"2023-10-08T00:14:44.310629+0000","last_became_active":"2023-10-08T00:14:43.298909+0000","last_became_peered":"2023-10-08T00:14:43.298909+0000","last_unstale":"2023-10-08T00:14:44.310629+0000","last_undegraded":"2023-10-08T00:14:44.310629+0000","last_fullsized":"2023-10-08T00:14:44.310629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:21.229437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.311954+0000","last_change":"2023-10-08T00:14:44.311787+0000","last_active":"2023-10-08T00:14:44.311954+0000","last_peered":"2023-10-08T00:14:44.311954+0000","last_clean":"2023-10-08T00:14:44.311954+0000","last_became_active":"2023-10-08T00:14:44.310198+0000","last_became_peered":"2023-10-08T00:14:44.310198+0000","last_unstale":"2023-10-08T00:14:44.311954+0000","last_undegraded":"2023-10-08T00:14:44.311954+0000","last_fullsized":"2023-10-08T00:14:44.311954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:26:44.694537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.303547+0000","last_change":"2023-10-08T00:14:43.298576+0000","last_active":"2023-10-08T00:14:44.303547+0000","last_peered":"2023-10-08T00:14:44.303547+0000","last_clean":"2023-10-08T00:14:44.303547+0000","last_became_active":"2023-10-08T00:14:43.298336+0000","last_became_peered":"2023-10-08T00:14:43.298336+0000","last_unstale":"2023-10-08T00:14:44.303547+0000","last_undegraded":"2023-10-08T00:14:44.303547+0000","last_fullsized":"2023-10-08T00:14:44.303547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:51:57.785852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.309626+0000","last_change":"2023-10-08T00:14:44.308193+0000","last_active":"2023-10-08T00:14:44.309626+0000","last_peered":"2023-10-08T00:14:44.309626+0000","last_clean":"2023-10-08T00:14:44.309626+0000","last_became_active":"2023-10-08T00:14:44.307903+0000","last_became_peered":"2023-10-08T00:14:44.307903+0000","last_unstale":"2023-10-08T00:14:44.309626+0000","last_undegraded":"2023-10-08T00:14:44.309626+0000","last_fullsized":"2023-10-08T00:14:44.309626+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:24:42.915827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"85'2","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.323070+0000","last_change":"2023-10-08T00:14:43.302228+0000","last_active":"2023-10-08T00:14:44.323070+0000","last_peered":"2023-10-08T00:14:44.323070+0000","last_clean":"2023-10-08T00:14:44.323070+0000","last_became_active":"2023-10-08T00:14:43.302099+0000","last_became_peered":"2023-10-08T00:14:43.302099+0000","last_unstale":"2023-10-08T00:14:44.323070+0000","last_undegraded":"2023-10-08T00:14:44.323070+0000","last_fullsized":"2023-10-08T00:14:44.323070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:24:41.910888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.313225+0000","last_change":"2023-10-08T00:14:44.313123+0000","last_active":"2023-10-08T00:14:44.313225+0000","last_peered":"2023-10-08T00:14:44.313225+0000","last_clean":"2023-10-08T00:14:44.313225+0000","last_became_active":"2023-10-08T00:14:44.312546+0000","last_became_peered":"2023-10-08T00:14:44.312546+0000","last_unstale":"2023-10-08T00:14:44.313225+0000","last_undegraded":"2023-10-08T00:14:44.313225+0000","last_fullsized":"2023-10-08T00:14:44.313225+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:48:38.177938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.303082+0000","last_change":"2023-10-08T00:14:43.299951+0000","last_active":"2023-10-08T00:14:44.303082+0000","last_peered":"2023-10-08T00:14:44.303082+0000","last_clean":"2023-10-08T00:14:44.303082+0000","last_became_active":"2023-10-08T00:14:43.299758+0000","last_became_peered":"2023-10-08T00:14:43.299758+0000","last_unstale":"2023-10-08T00:14:44.303082+0000","last_undegraded":"2023-10-08T00:14:44.303082+0000","last_fullsized":"2023-10-08T00:14:44.303082+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:44:44.807607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.312636+0000","last_change":"2023-10-08T00:14:44.312408+0000","last_active":"2023-10-08T00:14:44.312636+0000","last_peered":"2023-10-08T00:14:44.312636+0000","last_clean":"2023-10-08T00:14:44.312636+0000","last_became_active":"2023-10-08T00:14:44.312187+0000","last_became_peered":"2023-10-08T00:14:44.312187+0000","last_unstale":"2023-10-08T00:14:44.312636+0000","last_undegraded":"2023-10-08T00:14:44.312636+0000","last_fullsized":"2023-10-08T00:14:44.312636+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:09:33.137344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.303284+0000","last_change":"2023-10-08T00:14:43.295854+0000","last_active":"2023-10-08T00:14:44.303284+0000","last_peered":"2023-10-08T00:14:44.303284+0000","last_clean":"2023-10-08T00:14:44.303284+0000","last_became_active":"2023-10-08T00:14:43.295593+0000","last_became_peered":"2023-10-08T00:14:43.295593+0000","last_unstale":"2023-10-08T00:14:44.303284+0000","last_undegraded":"2023-10-08T00:14:44.303284+0000","last_fullsized":"2023-10-08T00:14:44.303284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:43:41.694345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.307962+0000","last_change":"2023-10-08T00:14:44.307664+0000","last_active":"2023-10-08T00:14:44.307962+0000","last_peered":"2023-10-08T00:14:44.307962+0000","last_clean":"2023-10-08T00:14:44.307962+0000","last_became_active":"2023-10-08T00:14:44.307457+0000","last_became_peered":"2023-10-08T00:14:44.307457+0000","last_unstale":"2023-10-08T00:14:44.307962+0000","last_undegraded":"2023-10-08T00:14:44.307962+0000","last_fullsized":"2023-10-08T00:14:44.307962+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:35:38.085349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.304418+0000","last_change":"2023-10-08T00:14:43.298666+0000","last_active":"2023-10-08T00:14:44.304418+0000","last_peered":"2023-10-08T00:14:44.304418+0000","last_clean":"2023-10-08T00:14:44.304418+0000","last_became_active":"2023-10-08T00:14:43.297793+0000","last_became_peered":"2023-10-08T00:14:43.297793+0000","last_unstale":"2023-10-08T00:14:44.304418+0000","last_undegraded":"2023-10-08T00:14:44.304418+0000","last_fullsized":"2023-10-08T00:14:44.304418+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:49:22.047215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.312162+0000","last_change":"2023-10-08T00:14:44.312014+0000","last_active":"2023-10-08T00:14:44.312162+0000","last_peered":"2023-10-08T00:14:44.312162+0000","last_clean":"2023-10-08T00:14:44.312162+0000","last_became_active":"2023-10-08T00:14:44.310128+0000","last_became_peered":"2023-10-08T00:14:44.310128+0000","last_unstale":"2023-10-08T00:14:44.312162+0000","last_undegraded":"2023-10-08T00:14:44.312162+0000","last_fullsized":"2023-10-08T00:14:44.312162+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:46:26.233800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.302814+0000","last_change":"2023-10-08T00:14:43.301528+0000","last_active":"2023-10-08T00:14:44.302814+0000","last_peered":"2023-10-08T00:14:44.302814+0000","last_clean":"2023-10-08T00:14:44.302814+0000","last_became_active":"2023-10-08T00:14:43.301291+0000","last_became_peered":"2023-10-08T00:14:43.301291+0000","last_unstale":"2023-10-08T00:14:44.302814+0000","last_undegraded":"2023-10-08T00:14:44.302814+0000","last_fullsized":"2023-10-08T00:14:44.302814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:27:51.224820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.311267+0000","last_change":"2023-10-08T00:14:44.311005+0000","last_active":"2023-10-08T00:14:44.311267+0000","last_peered":"2023-10-08T00:14:44.311267+0000","last_clean":"2023-10-08T00:14:44.311267+0000","last_became_active":"2023-10-08T00:14:44.310466+0000","last_became_peered":"2023-10-08T00:14:44.310466+0000","last_unstale":"2023-10-08T00:14:44.311267+0000","last_undegraded":"2023-10-08T00:14:44.311267+0000","last_fullsized":"2023-10-08T00:14:44.311267+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:56:38.021830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.306219+0000","last_change":"2023-10-08T00:14:43.292988+0000","last_active":"2023-10-08T00:14:44.306219+0000","last_peered":"2023-10-08T00:14:44.306219+0000","last_clean":"2023-10-08T00:14:44.306219+0000","last_became_active":"2023-10-08T00:14:43.292285+0000","last_became_peered":"2023-10-08T00:14:43.292285+0000","last_unstale":"2023-10-08T00:14:44.306219+0000","last_undegraded":"2023-10-08T00:14:44.306219+0000","last_fullsized":"2023-10-08T00:14:44.306219+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:02:00.046943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.312638+0000","last_change":"2023-10-08T00:14:44.311482+0000","last_active":"2023-10-08T00:14:44.312638+0000","last_peered":"2023-10-08T00:14:44.312638+0000","last_clean":"2023-10-08T00:14:44.312638+0000","last_became_active":"2023-10-08T00:14:44.311250+0000","last_became_peered":"2023-10-08T00:14:44.311250+0000","last_unstale":"2023-10-08T00:14:44.312638+0000","last_undegraded":"2023-10-08T00:14:44.312638+0000","last_fullsized":"2023-10-08T00:14:44.312638+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:26:44.694537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.313616+0000","last_change":"2023-10-08T00:14:43.300700+0000","last_active":"2023-10-08T00:14:44.313616+0000","last_peered":"2023-10-08T00:14:44.313616+0000","last_clean":"2023-10-08T00:14:44.313616+0000","last_became_active":"2023-10-08T00:14:43.300471+0000","last_became_peered":"2023-10-08T00:14:43.300471+0000","last_unstale":"2023-10-08T00:14:44.313616+0000","last_undegraded":"2023-10-08T00:14:44.313616+0000","last_fullsized":"2023-10-08T00:14:44.313616+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:27:51.224820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.314189+0000","last_change":"2023-10-08T00:14:44.314125+0000","last_active":"2023-10-08T00:14:44.314189+0000","last_peered":"2023-10-08T00:14:44.314189+0000","last_clean":"2023-10-08T00:14:44.314189+0000","last_became_active":"2023-10-08T00:14:44.313903+0000","last_became_peered":"2023-10-08T00:14:44.313903+0000","last_unstale":"2023-10-08T00:14:44.314189+0000","last_undegraded":"2023-10-08T00:14:44.314189+0000","last_fullsized":"2023-10-08T00:14:44.314189+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:21:40.091830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.300284+0000","last_change":"2023-10-08T00:14:43.297408+0000","last_active":"2023-10-08T00:14:44.300284+0000","last_peered":"2023-10-08T00:14:44.300284+0000","last_clean":"2023-10-08T00:14:44.300284+0000","last_became_active":"2023-10-08T00:14:43.297186+0000","last_became_peered":"2023-10-08T00:14:43.297186+0000","last_unstale":"2023-10-08T00:14:44.300284+0000","last_undegraded":"2023-10-08T00:14:44.300284+0000","last_fullsized":"2023-10-08T00:14:44.300284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:47:51.699728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.309607+0000","last_change":"2023-10-08T00:14:44.307156+0000","last_active":"2023-10-08T00:14:44.309607+0000","last_peered":"2023-10-08T00:14:44.309607+0000","last_clean":"2023-10-08T00:14:44.309607+0000","last_became_active":"2023-10-08T00:14:44.306915+0000","last_became_peered":"2023-10-08T00:14:44.306915+0000","last_unstale":"2023-10-08T00:14:44.309607+0000","last_undegraded":"2023-10-08T00:14:44.309607+0000","last_fullsized":"2023-10-08T00:14:44.309607+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:23:11.502305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"85'2","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.326701+0000","last_change":"2023-10-08T00:14:43.299421+0000","last_active":"2023-10-08T00:14:44.326701+0000","last_peered":"2023-10-08T00:14:44.326701+0000","last_clean":"2023-10-08T00:14:44.326701+0000","last_became_active":"2023-10-08T00:14:43.298880+0000","last_became_peered":"2023-10-08T00:14:43.298880+0000","last_unstale":"2023-10-08T00:14:44.326701+0000","last_undegraded":"2023-10-08T00:14:44.326701+0000","last_fullsized":"2023-10-08T00:14:44.326701+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:43:54.613305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":217,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.307113+0000","last_change":"2023-10-08T00:10:43.728490+0000","last_active":"2023-10-08T00:14:44.307113+0000","last_peered":"2023-10-08T00:14:44.307113+0000","last_clean":"2023-10-08T00:14:44.307113+0000","last_became_active":"2023-10-08T00:10:43.728341+0000","last_became_peered":"2023-10-08T00:10:43.728341+0000","last_unstale":"2023-10-08T00:14:44.307113+0000","last_undegraded":"2023-10-08T00:14:44.307113+0000","last_fullsized":"2023-10-08T00:14:44.307113+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_clean_scrub_stamp":"2023-10-08T00:10:42.716096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:06:31.035293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574899,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":449,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194863,"statfs":{"total":200000000,"available":199539775,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:13:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.51000000000000001,"15min":0.499},"min":{"1min":0.41699999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.53800000000000003,"15min":0.54400000000000004},"min":{"1min":0.374,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.77900000000000003,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.56200000000000006}]},{"osd":1,"last update":"Sun Oct 8 00:13:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.54100000000000004,"15min":0.53100000000000003},"min":{"1min":0.37,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.56899999999999995,"15min":0.56100000000000005},"min":{"1min":0.44400000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.76800000000000002,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.53000000000000003}]},{"osd":2,"last update":"Sun Oct 8 00:13:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54200000000000004,"15min":0.53700000000000003},"min":{"1min":0.40300000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.435},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.52800000000000002,"15min":0.51800000000000002},"min":{"1min":0.37,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.504}]},{"osd":3,"last update":"Sun Oct 8 00:13:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52600000000000002,"15min":0.51600000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.56200000000000006,"15min":0.55700000000000005},"min":{"1min":0.45700000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.54400000000000004}]},{"osd":4,"last update":"Sun Oct 8 00:13:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.52300000000000002,"15min":0.52600000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52300000000000002,"15min":0.53000000000000003},"min":{"1min":0.32800000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.747,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.61899999999999999}]},{"osd":5,"last update":"Sun Oct 8 00:13:45 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.54200000000000004,"15min":0.55400000000000005},"min":{"1min":0.315,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57299999999999995,"15min":0.58999999999999997},"min":{"1min":0.38500000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.68899999999999995}]},{"osd":6,"last update":"Sun Oct 8 00:13:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59399999999999997,"15min":0.59899999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76900000000000002,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.62,"15min":0.63600000000000001},"min":{"1min":0.36099999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.67600000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574899,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":449,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194863,"statfs":{"total":200000000,"available":199539741,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.48999999999999999,"15min":0.47799999999999998},"min":{"1min":0.34599999999999997,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.71399999999999997,"5min":1.012,"15min":1.012},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.64800000000000002,"15min":0.69299999999999995},"min":{"1min":0.374,"5min":0.317,"15min":0.317},"max":{"1min":0.70999999999999996,"5min":5.1230000000000002,"15min":5.1230000000000002},"last":0.57099999999999995}]},{"osd":1,"last update":"Sun Oct 8 00:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.53400000000000003,"15min":0.51800000000000002},"min":{"1min":0.39500000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.752,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.56599999999999995,"15min":0.56200000000000006},"min":{"1min":0.45400000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.73099999999999998,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.61299999999999999}]},{"osd":2,"last update":"Sun Oct 8 00:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.53600000000000003,"15min":0.52800000000000002},"min":{"1min":0.42099999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72199999999999998,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.56799999999999995,"15min":0.56699999999999995},"min":{"1min":0.48899999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.71599999999999997,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.59899999999999998}]},{"osd":3,"last update":"Sun Oct 8 00:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.54600000000000004,"15min":0.53400000000000003},"min":{"1min":0.501,"5min":0.371,"15min":0.371},"max":{"1min":0.745,"5min":1.046,"15min":1.046},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.55400000000000005,"15min":0.54000000000000004},"min":{"1min":0.36099999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.69199999999999995,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.55000000000000004}]},{"osd":4,"last update":"Sun Oct 8 00:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.49099999999999999,"15min":0.502},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.628,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55700000000000005,"15min":0.56499999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73599999999999999,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.46000000000000002}]},{"osd":5,"last update":"Sun Oct 8 00:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.48599999999999999,"15min":0.48799999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.70999999999999996,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.54400000000000004,"15min":0.55900000000000005},"min":{"1min":0.39800000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.71799999999999997,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.379}]},{"osd":7,"last update":"Sun Oct 8 00:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.53500000000000003,"15min":0.54300000000000004},"min":{"1min":0.36399999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76500000000000001,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54500000000000004,"15min":0.54800000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.73999999999999999,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.502}]}]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1024,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194288,"statfs":{"total":200000000,"available":198950943,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:13:49 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.315,"15min":0.30299999999999999},"min":{"1min":0.20799999999999999,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"last":0.22},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.38,"15min":0.374},"min":{"1min":0.25700000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.32300000000000001}]},{"osd":2,"last update":"Sun Oct 8 00:13:49 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.33800000000000002,"15min":0.33300000000000002},"min":{"1min":0.22900000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.254},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.34999999999999998,"15min":0.33600000000000002},"min":{"1min":0.27300000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.28599999999999998}]},{"osd":3,"last update":"Sun Oct 8 00:13:49 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.375,"15min":0.36699999999999999},"min":{"1min":0.307,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.54900000000000004,"5min":0.623,"15min":0.623},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.34499999999999997,"15min":0.34100000000000003},"min":{"1min":0.23400000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.30399999999999999}]},{"osd":4,"last update":"Sun Oct 8 00:13:49 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.54500000000000004,"15min":0.53700000000000003},"min":{"1min":0.40000000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57199999999999995,"15min":0.56599999999999995},"min":{"1min":0.39500000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.76800000000000002,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.55800000000000005}]},{"osd":5,"last update":"Sun Oct 8 00:13:49 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.54700000000000004,"15min":0.53400000000000003},"min":{"1min":0.42099999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.504},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57999999999999996,"15min":0.57399999999999995},"min":{"1min":0.38600000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.58799999999999997}]},{"osd":6,"last update":"Sun Oct 8 00:13:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.57099999999999995,"15min":0.56399999999999995},"min":{"1min":0.43099999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.82299999999999995,"5min":0.877,"15min":0.877},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56699999999999995,"15min":0.55900000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.83899999999999997,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.60299999999999998}]},{"osd":7,"last update":"Sun Oct 8 00:13:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.56999999999999995,"15min":0.56200000000000006},"min":{"1min":0.42399999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.622,"5min":0.59699999999999998,"15min":0.58699999999999997},"min":{"1min":0.45200000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.85599999999999998,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.61599999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574899,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":448,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194864,"statfs":{"total":200000000,"available":199541241,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 00:13:49 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.33100000000000002,"15min":0.32900000000000001},"min":{"1min":0.215,"5min":0.193,"15min":0.193},"max":{"1min":0.60499999999999998,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.314},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.34999999999999998,"15min":0.34599999999999997},"min":{"1min":0.23100000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.57699999999999996,"5min":0.87,"15min":0.87},"last":0.39400000000000002}]},{"osd":2,"last update":"Sun Oct 8 00:13:49 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.38500000000000001,"15min":0.38700000000000001},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.68000000000000005,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.66200000000000003,"15min":0.79400000000000004},"min":{"1min":0.24099999999999999,"5min":0.185,"15min":0.185},"max":{"1min":0.622,"5min":12.019,"15min":12.019},"last":0.35899999999999999}]},{"osd":3,"last update":"Sun Oct 8 00:13:49 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.36399999999999999,"15min":0.36199999999999999},"min":{"1min":0.25700000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.69999999999999996,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.439,"5min":0.39700000000000002,"15min":0.39300000000000002},"min":{"1min":0.28499999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.72899999999999998,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.42999999999999999}]},{"osd":4,"last update":"Sun Oct 8 00:13:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.62,"15min":0.61899999999999999},"min":{"1min":0.40000000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82599999999999996,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.628,"15min":0.626},"min":{"1min":0.42399999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.94599999999999995,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.66600000000000004}]},{"osd":5,"last update":"Sun Oct 8 00:13:49 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.90300000000000002,"15min":1.0369999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.85599999999999998,"5min":11.992000000000001,"15min":11.992000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59299999999999997,"15min":0.58599999999999997},"min":{"1min":0.502,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.90300000000000002,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.70199999999999996}]},{"osd":6,"last update":"Sun Oct 8 00:13:49 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.59399999999999997,"15min":0.58699999999999997},"min":{"1min":0.43099999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.877,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.89100000000000001,"15min":1.0129999999999999},"min":{"1min":0.434,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.96599999999999997,"5min":11.927,"15min":11.927},"last":0.60999999999999999}]},{"osd":7,"last update":"Sun Oct 8 00:13:49 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.90800000000000003,"15min":1.036},"min":{"1min":0.44500000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0069999999999999,"5min":11.965,"15min":11.965},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.629,"15min":0.625},"min":{"1min":0.54200000000000004,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.98299999999999998,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.748}]}]},{"osd":2,"up_from":13,"seq":55834574899,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":448,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194864,"statfs":{"total":200000000,"available":199540757,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.37,"15min":0.39500000000000002},"min":{"1min":0.24199999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.498,"5min":2.5139999999999998,"15min":2.5139999999999998},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.376,"15min":0.40600000000000003},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.56399999999999995,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":0.255}]},{"osd":1,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.36899999999999999,"15min":0.39500000000000002},"min":{"1min":0.249,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.48499999999999999,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.372,"5min":0.40799999999999997,"15min":0.44},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.53400000000000003,"5min":2.5379999999999998,"15min":2.5379999999999998},"last":0.33700000000000002}]},{"osd":3,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.41099999999999998,"15min":0.433},"min":{"1min":0.255,"5min":0.254,"15min":0.254},"max":{"1min":0.54800000000000004,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.436,"15min":0.46800000000000003},"min":{"1min":0.30399999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.501,"5min":2.5619999999999998,"15min":2.5619999999999998},"last":0.35799999999999998}]},{"osd":4,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.59699999999999998,"15min":0.621},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.073,"5min":2.488,"15min":2.488},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.59599999999999997,"15min":0.60799999999999998},"min":{"1min":0.41299999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.75800000000000001,"5min":2.073,"15min":2.073},"last":0.41799999999999998}]},{"osd":5,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63900000000000001,"15min":0.66100000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.754,"5min":2.2069999999999999,"15min":2.2069999999999999},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.65200000000000002,"15min":0.66600000000000004},"min":{"1min":0.45900000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0369999999999999,"5min":2.456,"15min":2.456},"last":0.47499999999999998}]},{"osd":6,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.625},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.05,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":0.504},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60599999999999998,"15min":0.61299999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0940000000000001,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.439}]},{"osd":7,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63,"15min":0.63900000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0229999999999999,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.61899999999999999,"15min":0.63},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0569999999999999,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":0.48299999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574899,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":448,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194864,"statfs":{"total":200000000,"available":199541185,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.34000000000000002,"15min":0.371},"min":{"1min":0.218,"5min":0.193,"15min":0.193},"max":{"1min":0.52700000000000002,"5min":1.46,"15min":1.46},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.33400000000000002,"15min":0.34100000000000003},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":0.496,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.34599999999999997}]},{"osd":1,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.58999999999999997,"15min":0.83699999999999997},"min":{"1min":0.25,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.57599999999999996,"5min":12.351000000000001,"15min":12.351000000000001},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.28399999999999997,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.58899999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.438}]},{"osd":2,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.40100000000000002,"15min":0.42099999999999999},"min":{"1min":0.26300000000000001,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.70499999999999996,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.26800000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.63500000000000001,"5min":0.753,"15min":0.753},"last":0.38100000000000001}]},{"osd":4,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.80000000000000004,"15min":1.018},"min":{"1min":0.38400000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.876,"5min":11.333,"15min":11.333},"last":1.024},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.82899999999999996,"15min":1.0509999999999999},"min":{"1min":0.41299999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.94899999999999995,"5min":12.321999999999999,"15min":12.321999999999999},"last":0.73399999999999999}]},{"osd":5,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.82499999999999996,"15min":1.0620000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.92000000000000004,"5min":12.291,"15min":12.291},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.60099999999999998,"15min":0.59799999999999998},"min":{"1min":0.46300000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.99399999999999999}]},{"osd":6,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.77700000000000002,"15min":0.98299999999999998},"min":{"1min":0.42299999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.96199999999999997,"5min":11.288,"15min":11.288},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.622,"5min":0.81299999999999994,"15min":1.044},"min":{"1min":0.433,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.89400000000000002,"5min":12.253,"15min":12.253},"last":0.91200000000000003}]},{"osd":7,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.58899999999999997,"15min":0.57499999999999996},"min":{"1min":0.40400000000000003,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.97299999999999998,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.81200000000000006,"15min":1.02},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.92400000000000004,"5min":11.363,"15min":11.363},"last":0.67100000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574899,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":448,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194863,"statfs":{"total":200000000,"available":199540645,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56000000000000005,"15min":0.55400000000000005},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.74199999999999999,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.441},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.58199999999999996,"15min":0.58599999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.63600000000000001}]},{"osd":1,"last update":"Sun Oct 8 00:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.38800000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.52600000000000002,"15min":0.53200000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.73699999999999999,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.53300000000000003}]},{"osd":2,"last update":"Sun Oct 8 00:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51100000000000001,"15min":0.51800000000000002},"min":{"1min":0.33900000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.63400000000000001,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.51600000000000001,"15min":0.51000000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.48099999999999998}]},{"osd":3,"last update":"Sun Oct 8 00:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53400000000000003,"15min":0.53900000000000003},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.71499999999999997,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53100000000000003,"15min":0.53300000000000003},"min":{"1min":0.40200000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.64500000000000002}]},{"osd":5,"last update":"Sun Oct 8 00:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.48499999999999999,"15min":0.5},"min":{"1min":0.35899999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69599999999999995,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.498,"15min":0.51100000000000001},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.71399999999999997,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.375}]},{"osd":6,"last update":"Sun Oct 8 00:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.48499999999999999,"15min":0.49299999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.67600000000000005,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54400000000000004,"15min":0.55900000000000005},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.72199999999999998,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.39800000000000002}]},{"osd":7,"last update":"Sun Oct 8 00:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.53900000000000003,"15min":0.55300000000000005},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.73399999999999999,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57799999999999996,"15min":0.58299999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.78000000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.57899999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574899,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1024,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194287,"statfs":{"total":200000000,"available":198950853,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.48399999999999999,"15min":0.48699999999999999},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.65600000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55400000000000005,"15min":0.56200000000000006},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.70999999999999996,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.48999999999999999}]},{"osd":1,"last update":"Sun Oct 8 00:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.51600000000000001,"15min":0.52200000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.68200000000000005,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51600000000000001,"15min":0.52600000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.747,"5min":0.755,"15min":0.755},"last":0.441}]},{"osd":2,"last update":"Sun Oct 8 00:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52700000000000002,"15min":0.53000000000000003},"min":{"1min":0.34399999999999997,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.71799999999999997,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.56100000000000005,"15min":0.57099999999999995},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.79500000000000004,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.58199999999999996}]},{"osd":3,"last update":"Sun Oct 8 00:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.54900000000000004,"15min":0.54700000000000004},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.57199999999999995,"15min":0.58099999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.71999999999999997,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.55700000000000005}]},{"osd":4,"last update":"Sun Oct 8 00:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.46500000000000002,"15min":0.47199999999999998},"min":{"1min":0.30399999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.57499999999999996,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.51300000000000001},"min":{"1min":0.35699999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.746,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.52500000000000002}]},{"osd":6,"last update":"Sun Oct 8 00:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.496,"15min":0.498},"min":{"1min":0.34399999999999997,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.70599999999999996,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48999999999999999,"15min":0.49099999999999999},"min":{"1min":0.38400000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.61399999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.503}]},{"osd":7,"last update":"Sun Oct 8 00:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.33300000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.68400000000000005,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.55000000000000004,"15min":0.55500000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.72999999999999998,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.60399999999999998}]}]}],"pool_statfs":[]}} 2023-10-08T00:14:48.364 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T00:14:48.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.510+0000 7fe8a8a4b700 1 Processor -- start 2023-10-08T00:14:48.511 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.510+0000 7fe8a8a4b700 1 -- start start 2023-10-08T00:14:48.513 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.511+0000 7fe8a8a4b700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe8a40cffa0 0x7fe8a406a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:48.513 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.511+0000 7fe8a8a4b700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8a40b3d40 con 0x7fe8a4159e60 2023-10-08T00:14:48.513 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.511+0000 7fe8a8a4b700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8a40b4300 con 0x7fe8a415a230 2023-10-08T00:14:48.513 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.511+0000 7fe8a8a4b700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8a4069d70 con 0x7fe8a40cffa0 2023-10-08T00:14:48.513 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.511+0000 7fe8a1d9b700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe8a40cffa0 0x7fe8a406a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:48.513 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.511+0000 7fe8a259c700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fe8a4159e60 0x7fe8a40b6890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:34748/0 (socket says 172.21.15.99:34748) 2023-10-08T00:14:48.513 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.511+0000 7fe8a259c700 1 -- 172.21.15.99:0/2057613837 learned_addr learned my addr 172.21.15.99:0/2057613837 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:14:48.513 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.511+0000 7fe8a1d9b700 1 -- 172.21.15.99:0/2057613837 >> v1:172.21.15.191:6789/0 conn(0x7fe8a415a230 legacy=0x7fe8a4163c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:48.513 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.511+0000 7fe8a1d9b700 1 -- 172.21.15.99:0/2057613837 >> v1:172.21.15.99:6789/0 conn(0x7fe8a4159e60 legacy=0x7fe8a40b6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:48.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.511+0000 7fe8a1d9b700 1 -- 172.21.15.99:0/2057613837 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a40d0480 con 0x7fe8a40cffa0 2023-10-08T00:14:48.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.511+0000 7fe8a159a700 1 -- 172.21.15.99:0/2057613837 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3650882519 0 0) 0x7fe8a40b3d40 con 0x7fe8a4159e60 2023-10-08T00:14:48.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.511+0000 7fe8a1d9b700 1 --2- 172.21.15.99:0/2057613837 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8a40cffa0 0x7fe8a406a6a0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fe88c0098e0 tx=0x7fe88c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41da8c655f058e57 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:48.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.511+0000 7fe8a159a700 1 -- 172.21.15.99:0/2057613837 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88c009e80 con 0x7fe8a40cffa0 2023-10-08T00:14:48.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.511+0000 7fe8a159a700 1 -- 172.21.15.99:0/2057613837 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe88c00b820 con 0x7fe8a40cffa0 2023-10-08T00:14:48.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.511+0000 7fe8a159a700 1 -- 172.21.15.99:0/2057613837 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88c00bba0 con 0x7fe8a40cffa0 2023-10-08T00:14:48.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.512+0000 7fe8a8a4b700 1 -- 172.21.15.99:0/2057613837 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8a40cffa0 msgr2=0x7fe8a406a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:48.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.512+0000 7fe8a8a4b700 1 --2- 172.21.15.99:0/2057613837 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8a40cffa0 0x7fe8a406a6a0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fe88c0098e0 tx=0x7fe88c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:14:48.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.512+0000 7fe8a8a4b700 1 -- 172.21.15.99:0/2057613837 shutdown_connections 2023-10-08T00:14:48.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.512+0000 7fe8a8a4b700 1 --2- 172.21.15.99:0/2057613837 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8a40cffa0 0x7fe8a406a6a0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:48.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.512+0000 7fe8a8a4b700 1 -- 172.21.15.99:0/2057613837 >> 172.21.15.99:0/2057613837 conn(0x7fe8a40d1a50 msgr2=0x7fe8a4068f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:48.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.512+0000 7fe8a8a4b700 1 -- 172.21.15.99:0/2057613837 shutdown_connections 2023-10-08T00:14:48.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.512+0000 7fe8a8a4b700 1 -- 172.21.15.99:0/2057613837 wait complete. 2023-10-08T00:14:48.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.513+0000 7fe8a8a4b700 1 Processor -- start 2023-10-08T00:14:48.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.513+0000 7fe8a8a4b700 1 -- start start 2023-10-08T00:14:48.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.514+0000 7fe8a8a4b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8a40cffa0 0x7fe8a40c7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:48.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.514+0000 7fe8a8a4b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8a4159e60 0x7fe8a40cd670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:48.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.514+0000 7fe8a8a4b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8a415a230 0x7fe8a4154be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:48.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.514+0000 7fe8a8a4b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a40c9540 con 0x7fe8a415a230 2023-10-08T00:14:48.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.514+0000 7fe8a8a4b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a40cdbb0 con 0x7fe8a40cffa0 2023-10-08T00:14:48.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.514+0000 7fe8a8a4b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a4155120 con 0x7fe8a4159e60 2023-10-08T00:14:48.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.514+0000 7fe8a2d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8a415a230 0x7fe8a4154be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:48.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.514+0000 7fe8a2d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8a415a230 0x7fe8a4154be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45746/0 (socket says 172.21.15.99:45746) 2023-10-08T00:14:48.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.514+0000 7fe8a2d9d700 1 -- 172.21.15.99:0/1462155196 learned_addr learned my addr 172.21.15.99:0/1462155196 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:48.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.514+0000 7fe8a259c700 1 --2- 172.21.15.99:0/1462155196 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8a40cffa0 0x7fe8a40c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:48.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.514+0000 7fe8a1d9b700 1 --2- 172.21.15.99:0/1462155196 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8a4159e60 0x7fe8a40cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:48.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.514+0000 7fe8a2d9d700 1 -- 172.21.15.99:0/1462155196 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8a4159e60 msgr2=0x7fe8a40cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:48.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.514+0000 7fe8a2d9d700 1 --2- 172.21.15.99:0/1462155196 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8a4159e60 0x7fe8a40cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:48.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.514+0000 7fe8a2d9d700 1 -- 172.21.15.99:0/1462155196 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8a40cffa0 msgr2=0x7fe8a40c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:48.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.514+0000 7fe8a2d9d700 1 --2- 172.21.15.99:0/1462155196 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8a40cffa0 0x7fe8a40c7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:48.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.514+0000 7fe8a2d9d700 1 -- 172.21.15.99:0/1462155196 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe88c009420 con 0x7fe8a415a230 2023-10-08T00:14:48.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.514+0000 7fe8a2d9d700 1 --2- 172.21.15.99:0/1462155196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8a415a230 0x7fe8a4154be0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fe89400b4e0 tx=0x7fe894009060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:48.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.515+0000 7fe89a7fc700 1 -- 172.21.15.99:0/1462155196 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89400f510 con 0x7fe8a415a230 2023-10-08T00:14:48.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.515+0000 7fe89a7fc700 1 -- 172.21.15.99:0/1462155196 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe89400f690 con 0x7fe8a415a230 2023-10-08T00:14:48.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.515+0000 7fe89a7fc700 1 -- 172.21.15.99:0/1462155196 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89400fa50 con 0x7fe8a415a230 2023-10-08T00:14:48.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.515+0000 7fe8a8a4b700 1 -- 172.21.15.99:0/1462155196 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8a416ade0 con 0x7fe8a415a230 2023-10-08T00:14:48.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.515+0000 7fe8a8a4b700 1 -- 172.21.15.99:0/1462155196 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8a40c8070 con 0x7fe8a415a230 2023-10-08T00:14:48.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.516+0000 7fe8a8a4b700 1 -- 172.21.15.99:0/1462155196 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8a416af70 con 0x7fe8a415a230 2023-10-08T00:14:48.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.516+0000 7fe89a7fc700 1 -- 172.21.15.99:0/1462155196 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe89401a400 con 0x7fe8a415a230 2023-10-08T00:14:48.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.516+0000 7fe89a7fc700 1 --2- 172.21.15.99:0/1462155196 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe884066f30 0x7fe8840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:48.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.516+0000 7fe89a7fc700 1 -- 172.21.15.99:0/1462155196 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe894083e80 con 0x7fe8a415a230 2023-10-08T00:14:48.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.517+0000 7fe8a259c700 1 --2- 172.21.15.99:0/1462155196 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe884066f30 0x7fe8840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:48.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.517+0000 7fe8a259c700 1 --2- 172.21.15.99:0/1462155196 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe884066f30 0x7fe8840693e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe89c002250 tx=0x7fe89c001a20 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:48.521 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.520+0000 7fe89a7fc700 1 -- 172.21.15.99:0/1462155196 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe894051170 con 0x7fe8a415a230 2023-10-08T00:14:48.702 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.700+0000 7fe8a8a4b700 1 -- 172.21.15.99:0/1462155196 --> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe8a404c450 con 0x7fe884066f30 2023-10-08T00:14:48.709 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.709+0000 7fe89a7fc700 1 -- 172.21.15.99:0/1462155196 <== mgr.4099 v2:172.21.15.191:6832/85985 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284849 (secure 0 0 0) 0x7fe89c014450 con 0x7fe884066f30 2023-10-08T00:14:48.715 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:14:48.720 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.712+0000 7fe8a8a4b700 1 -- 172.21.15.99:0/1462155196 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe884066f30 msgr2=0x7fe8840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:48.721 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.712+0000 7fe8a8a4b700 1 --2- 172.21.15.99:0/1462155196 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe884066f30 0x7fe8840693e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe89c002250 tx=0x7fe89c001a20 comp rx=0 tx=0).stop 2023-10-08T00:14:48.721 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.712+0000 7fe8a8a4b700 1 -- 172.21.15.99:0/1462155196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8a415a230 msgr2=0x7fe8a4154be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:48.721 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.712+0000 7fe8a8a4b700 1 --2- 172.21.15.99:0/1462155196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8a415a230 0x7fe8a4154be0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fe89400b4e0 tx=0x7fe894009060 comp rx=0 tx=0).stop 2023-10-08T00:14:48.721 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.712+0000 7fe8a8a4b700 1 -- 172.21.15.99:0/1462155196 shutdown_connections 2023-10-08T00:14:48.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.712+0000 7fe8a8a4b700 1 --2- 172.21.15.99:0/1462155196 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe884066f30 0x7fe8840693e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:48.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.712+0000 7fe8a8a4b700 1 --2- 172.21.15.99:0/1462155196 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8a40cffa0 0x7fe8a40c7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:48.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.712+0000 7fe8a8a4b700 1 --2- 172.21.15.99:0/1462155196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8a415a230 0x7fe8a4154be0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:48.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.712+0000 7fe8a8a4b700 1 --2- 172.21.15.99:0/1462155196 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8a4159e60 0x7fe8a40cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:48.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.712+0000 7fe8a8a4b700 1 -- 172.21.15.99:0/1462155196 >> 172.21.15.99:0/1462155196 conn(0x7fe8a40d1a50 msgr2=0x7fe8a4068f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:48.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.712+0000 7fe8a8a4b700 1 -- 172.21.15.99:0/1462155196 shutdown_connections 2023-10-08T00:14:48.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.712+0000 7fe8a8a4b700 1 -- 172.21.15.99:0/1462155196 wait complete. 2023-10-08T00:14:48.723 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2023-10-08T00:14:48.725 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":199,"stamp":"2023-10-08T00:14:48.081007+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":4741,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1557758,"statfs":{"total":1600000000,"available":1595145140,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.794598"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.314368+0000","last_change":"2023-10-08T00:14:44.314214+0000","last_active":"2023-10-08T00:14:44.314368+0000","last_peered":"2023-10-08T00:14:44.314368+0000","last_clean":"2023-10-08T00:14:44.314368+0000","last_became_active":"2023-10-08T00:14:44.313766+0000","last_became_peered":"2023-10-08T00:14:44.313766+0000","last_unstale":"2023-10-08T00:14:44.314368+0000","last_undegraded":"2023-10-08T00:14:44.314368+0000","last_fullsized":"2023-10-08T00:14:44.314368+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:33:13.734581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.312659+0000","last_change":"2023-10-08T00:14:44.312494+0000","last_active":"2023-10-08T00:14:44.312659+0000","last_peered":"2023-10-08T00:14:44.312659+0000","last_clean":"2023-10-08T00:14:44.312659+0000","last_became_active":"2023-10-08T00:14:44.312205+0000","last_became_peered":"2023-10-08T00:14:44.312205+0000","last_unstale":"2023-10-08T00:14:44.312659+0000","last_undegraded":"2023-10-08T00:14:44.312659+0000","last_fullsized":"2023-10-08T00:14:44.312659+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:02:01.051881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.313626+0000","last_change":"2023-10-08T00:14:44.313561+0000","last_active":"2023-10-08T00:14:44.313626+0000","last_peered":"2023-10-08T00:14:44.313626+0000","last_clean":"2023-10-08T00:14:44.313626+0000","last_became_active":"2023-10-08T00:14:44.313034+0000","last_became_peered":"2023-10-08T00:14:44.313034+0000","last_unstale":"2023-10-08T00:14:44.313626+0000","last_undegraded":"2023-10-08T00:14:44.313626+0000","last_fullsized":"2023-10-08T00:14:44.313626+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:22.234376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.312164+0000","last_change":"2023-10-08T00:14:44.311989+0000","last_active":"2023-10-08T00:14:44.312164+0000","last_peered":"2023-10-08T00:14:44.312164+0000","last_clean":"2023-10-08T00:14:44.312164+0000","last_became_active":"2023-10-08T00:14:44.311648+0000","last_became_peered":"2023-10-08T00:14:44.311648+0000","last_unstale":"2023-10-08T00:14:44.312164+0000","last_undegraded":"2023-10-08T00:14:44.312164+0000","last_fullsized":"2023-10-08T00:14:44.312164+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:35:38.085349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.313897+0000","last_change":"2023-10-08T00:14:44.313757+0000","last_active":"2023-10-08T00:14:44.313897+0000","last_peered":"2023-10-08T00:14:44.313897+0000","last_clean":"2023-10-08T00:14:44.313897+0000","last_became_active":"2023-10-08T00:14:44.313528+0000","last_became_peered":"2023-10-08T00:14:44.313528+0000","last_unstale":"2023-10-08T00:14:44.313897+0000","last_undegraded":"2023-10-08T00:14:44.313897+0000","last_fullsized":"2023-10-08T00:14:44.313897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:07:31.992120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.308712+0000","last_change":"2023-10-08T00:14:44.307713+0000","last_active":"2023-10-08T00:14:44.308712+0000","last_peered":"2023-10-08T00:14:44.308712+0000","last_clean":"2023-10-08T00:14:44.308712+0000","last_became_active":"2023-10-08T00:14:44.307467+0000","last_became_peered":"2023-10-08T00:14:44.307467+0000","last_unstale":"2023-10-08T00:14:44.308712+0000","last_undegraded":"2023-10-08T00:14:44.308712+0000","last_fullsized":"2023-10-08T00:14:44.308712+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:21:40.091830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.313020+0000","last_change":"2023-10-08T00:14:44.312916+0000","last_active":"2023-10-08T00:14:44.313020+0000","last_peered":"2023-10-08T00:14:44.313020+0000","last_clean":"2023-10-08T00:14:44.313020+0000","last_became_active":"2023-10-08T00:14:44.312247+0000","last_became_peered":"2023-10-08T00:14:44.312247+0000","last_unstale":"2023-10-08T00:14:44.313020+0000","last_undegraded":"2023-10-08T00:14:44.313020+0000","last_fullsized":"2023-10-08T00:14:44.313020+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:26:21.446680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.313189+0000","last_change":"2023-10-08T00:14:44.312725+0000","last_active":"2023-10-08T00:14:44.313189+0000","last_peered":"2023-10-08T00:14:44.313189+0000","last_clean":"2023-10-08T00:14:44.313189+0000","last_became_active":"2023-10-08T00:14:44.312475+0000","last_became_peered":"2023-10-08T00:14:44.312475+0000","last_unstale":"2023-10-08T00:14:44.313189+0000","last_undegraded":"2023-10-08T00:14:44.313189+0000","last_fullsized":"2023-10-08T00:14:44.313189+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:05:00.741767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.312125+0000","last_change":"2023-10-08T00:14:44.311433+0000","last_active":"2023-10-08T00:14:44.312125+0000","last_peered":"2023-10-08T00:14:44.312125+0000","last_clean":"2023-10-08T00:14:44.312125+0000","last_became_active":"2023-10-08T00:14:44.310859+0000","last_became_peered":"2023-10-08T00:14:44.310859+0000","last_unstale":"2023-10-08T00:14:44.312125+0000","last_undegraded":"2023-10-08T00:14:44.312125+0000","last_fullsized":"2023-10-08T00:14:44.312125+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:27:52.229759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.312689+0000","last_change":"2023-10-08T00:14:44.312635+0000","last_active":"2023-10-08T00:14:44.312689+0000","last_peered":"2023-10-08T00:14:44.312689+0000","last_clean":"2023-10-08T00:14:44.312689+0000","last_became_active":"2023-10-08T00:14:44.312108+0000","last_became_peered":"2023-10-08T00:14:44.312108+0000","last_unstale":"2023-10-08T00:14:44.312689+0000","last_undegraded":"2023-10-08T00:14:44.312689+0000","last_fullsized":"2023-10-08T00:14:44.312689+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:52:17.712934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.307330+0000","last_change":"2023-10-08T00:14:44.307239+0000","last_active":"2023-10-08T00:14:44.307330+0000","last_peered":"2023-10-08T00:14:44.307330+0000","last_clean":"2023-10-08T00:14:44.307330+0000","last_became_active":"2023-10-08T00:14:44.306964+0000","last_became_peered":"2023-10-08T00:14:44.306964+0000","last_unstale":"2023-10-08T00:14:44.307330+0000","last_undegraded":"2023-10-08T00:14:44.307330+0000","last_fullsized":"2023-10-08T00:14:44.307330+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:38:03.052039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.308182+0000","last_change":"2023-10-08T00:14:44.307951+0000","last_active":"2023-10-08T00:14:44.308182+0000","last_peered":"2023-10-08T00:14:44.308182+0000","last_clean":"2023-10-08T00:14:44.308182+0000","last_became_active":"2023-10-08T00:14:44.307713+0000","last_became_peered":"2023-10-08T00:14:44.307713+0000","last_unstale":"2023-10-08T00:14:44.308182+0000","last_undegraded":"2023-10-08T00:14:44.308182+0000","last_fullsized":"2023-10-08T00:14:44.308182+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:52:24.163704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.314791+0000","last_change":"2023-10-08T00:14:44.314121+0000","last_active":"2023-10-08T00:14:44.314791+0000","last_peered":"2023-10-08T00:14:44.314791+0000","last_clean":"2023-10-08T00:14:44.314791+0000","last_became_active":"2023-10-08T00:14:44.313333+0000","last_became_peered":"2023-10-08T00:14:44.313333+0000","last_unstale":"2023-10-08T00:14:44.314791+0000","last_undegraded":"2023-10-08T00:14:44.314791+0000","last_fullsized":"2023-10-08T00:14:44.314791+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:52:17.712934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.311047+0000","last_change":"2023-10-08T00:14:44.310772+0000","last_active":"2023-10-08T00:14:44.311047+0000","last_peered":"2023-10-08T00:14:44.311047+0000","last_clean":"2023-10-08T00:14:44.311047+0000","last_became_active":"2023-10-08T00:14:44.309865+0000","last_became_peered":"2023-10-08T00:14:44.309865+0000","last_unstale":"2023-10-08T00:14:44.311047+0000","last_undegraded":"2023-10-08T00:14:44.311047+0000","last_fullsized":"2023-10-08T00:14:44.311047+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:50:50.579902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.311987+0000","last_change":"2023-10-08T00:14:44.311937+0000","last_active":"2023-10-08T00:14:44.311987+0000","last_peered":"2023-10-08T00:14:44.311987+0000","last_clean":"2023-10-08T00:14:44.311987+0000","last_became_active":"2023-10-08T00:14:44.310528+0000","last_became_peered":"2023-10-08T00:14:44.310528+0000","last_unstale":"2023-10-08T00:14:44.311987+0000","last_undegraded":"2023-10-08T00:14:44.311987+0000","last_fullsized":"2023-10-08T00:14:44.311987+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:47:52.704667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.311282+0000","last_change":"2023-10-08T00:14:44.311178+0000","last_active":"2023-10-08T00:14:44.311282+0000","last_peered":"2023-10-08T00:14:44.311282+0000","last_clean":"2023-10-08T00:14:44.311282+0000","last_became_active":"2023-10-08T00:14:44.310766+0000","last_became_peered":"2023-10-08T00:14:44.310766+0000","last_unstale":"2023-10-08T00:14:44.311282+0000","last_undegraded":"2023-10-08T00:14:44.311282+0000","last_fullsized":"2023-10-08T00:14:44.311282+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:18:05.629462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.312246+0000","last_change":"2023-10-08T00:14:44.312184+0000","last_active":"2023-10-08T00:14:44.312246+0000","last_peered":"2023-10-08T00:14:44.312246+0000","last_clean":"2023-10-08T00:14:44.312246+0000","last_became_active":"2023-10-08T00:14:44.310422+0000","last_became_peered":"2023-10-08T00:14:44.310422+0000","last_unstale":"2023-10-08T00:14:44.312246+0000","last_undegraded":"2023-10-08T00:14:44.312246+0000","last_fullsized":"2023-10-08T00:14:44.312246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:33:18.254151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.312023+0000","last_change":"2023-10-08T00:14:44.311931+0000","last_active":"2023-10-08T00:14:44.312023+0000","last_peered":"2023-10-08T00:14:44.312023+0000","last_clean":"2023-10-08T00:14:44.312023+0000","last_became_active":"2023-10-08T00:14:44.311396+0000","last_became_peered":"2023-10-08T00:14:44.311396+0000","last_unstale":"2023-10-08T00:14:44.312023+0000","last_undegraded":"2023-10-08T00:14:44.312023+0000","last_fullsized":"2023-10-08T00:14:44.312023+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:44:45.812545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.314339+0000","last_change":"2023-10-08T00:14:44.313535+0000","last_active":"2023-10-08T00:14:44.314339+0000","last_peered":"2023-10-08T00:14:44.314339+0000","last_clean":"2023-10-08T00:14:44.314339+0000","last_became_active":"2023-10-08T00:14:44.313306+0000","last_became_peered":"2023-10-08T00:14:44.313306+0000","last_unstale":"2023-10-08T00:14:44.314339+0000","last_undegraded":"2023-10-08T00:14:44.314339+0000","last_fullsized":"2023-10-08T00:14:44.314339+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:31:06.244548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.310921+0000","last_change":"2023-10-08T00:14:44.310816+0000","last_active":"2023-10-08T00:14:44.310921+0000","last_peered":"2023-10-08T00:14:44.310921+0000","last_clean":"2023-10-08T00:14:44.310921+0000","last_became_active":"2023-10-08T00:14:44.310156+0000","last_became_peered":"2023-10-08T00:14:44.310156+0000","last_unstale":"2023-10-08T00:14:44.310921+0000","last_undegraded":"2023-10-08T00:14:44.310921+0000","last_fullsized":"2023-10-08T00:14:44.310921+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:59:35.179200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.316019+0000","last_change":"2023-10-08T00:14:44.315911+0000","last_active":"2023-10-08T00:14:44.316019+0000","last_peered":"2023-10-08T00:14:44.316019+0000","last_clean":"2023-10-08T00:14:44.316019+0000","last_became_active":"2023-10-08T00:14:44.315664+0000","last_became_peered":"2023-10-08T00:14:44.315664+0000","last_unstale":"2023-10-08T00:14:44.316019+0000","last_undegraded":"2023-10-08T00:14:44.316019+0000","last_fullsized":"2023-10-08T00:14:44.316019+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:31:58.708770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.314516+0000","last_change":"2023-10-08T00:14:44.314458+0000","last_active":"2023-10-08T00:14:44.314516+0000","last_peered":"2023-10-08T00:14:44.314516+0000","last_clean":"2023-10-08T00:14:44.314516+0000","last_became_active":"2023-10-08T00:14:44.314241+0000","last_became_peered":"2023-10-08T00:14:44.314241+0000","last_unstale":"2023-10-08T00:14:44.314516+0000","last_undegraded":"2023-10-08T00:14:44.314516+0000","last_fullsized":"2023-10-08T00:14:44.314516+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:50:20.430773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.315866+0000","last_change":"2023-10-08T00:14:44.315817+0000","last_active":"2023-10-08T00:14:44.315866+0000","last_peered":"2023-10-08T00:14:44.315866+0000","last_clean":"2023-10-08T00:14:44.315866+0000","last_became_active":"2023-10-08T00:14:44.315436+0000","last_became_peered":"2023-10-08T00:14:44.315436+0000","last_unstale":"2023-10-08T00:14:44.315866+0000","last_undegraded":"2023-10-08T00:14:44.315866+0000","last_fullsized":"2023-10-08T00:14:44.315866+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:27:52.229759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.306748+0000","last_change":"2023-10-08T00:14:44.306703+0000","last_active":"2023-10-08T00:14:44.306748+0000","last_peered":"2023-10-08T00:14:44.306748+0000","last_clean":"2023-10-08T00:14:44.306748+0000","last_became_active":"2023-10-08T00:14:44.306441+0000","last_became_peered":"2023-10-08T00:14:44.306441+0000","last_unstale":"2023-10-08T00:14:44.306748+0000","last_undegraded":"2023-10-08T00:14:44.306748+0000","last_fullsized":"2023-10-08T00:14:44.306748+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:41:11.473838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.312862+0000","last_change":"2023-10-08T00:14:44.312776+0000","last_active":"2023-10-08T00:14:44.312862+0000","last_peered":"2023-10-08T00:14:44.312862+0000","last_clean":"2023-10-08T00:14:44.312862+0000","last_became_active":"2023-10-08T00:14:44.312414+0000","last_became_peered":"2023-10-08T00:14:44.312414+0000","last_unstale":"2023-10-08T00:14:44.312862+0000","last_undegraded":"2023-10-08T00:14:44.312862+0000","last_fullsized":"2023-10-08T00:14:44.312862+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:43:42.699284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.307299+0000","last_change":"2023-10-08T00:14:44.307119+0000","last_active":"2023-10-08T00:14:44.307299+0000","last_peered":"2023-10-08T00:14:44.307299+0000","last_clean":"2023-10-08T00:14:44.307299+0000","last_became_active":"2023-10-08T00:14:44.306662+0000","last_became_peered":"2023-10-08T00:14:44.306662+0000","last_unstale":"2023-10-08T00:14:44.307299+0000","last_undegraded":"2023-10-08T00:14:44.307299+0000","last_fullsized":"2023-10-08T00:14:44.307299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:01:30.785141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.310904+0000","last_change":"2023-10-08T00:14:44.310747+0000","last_active":"2023-10-08T00:14:44.310904+0000","last_peered":"2023-10-08T00:14:44.310904+0000","last_clean":"2023-10-08T00:14:44.310904+0000","last_became_active":"2023-10-08T00:14:44.309809+0000","last_became_peered":"2023-10-08T00:14:44.309809+0000","last_unstale":"2023-10-08T00:14:44.310904+0000","last_undegraded":"2023-10-08T00:14:44.310904+0000","last_fullsized":"2023-10-08T00:14:44.310904+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:02:49.544839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.316004+0000","last_change":"2023-10-08T00:14:44.315598+0000","last_active":"2023-10-08T00:14:44.316004+0000","last_peered":"2023-10-08T00:14:44.316004+0000","last_clean":"2023-10-08T00:14:44.316004+0000","last_became_active":"2023-10-08T00:14:44.315374+0000","last_became_peered":"2023-10-08T00:14:44.315374+0000","last_unstale":"2023-10-08T00:14:44.316004+0000","last_undegraded":"2023-10-08T00:14:44.316004+0000","last_fullsized":"2023-10-08T00:14:44.316004+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:46:13.400999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.312147+0000","last_change":"2023-10-08T00:14:44.311860+0000","last_active":"2023-10-08T00:14:44.312147+0000","last_peered":"2023-10-08T00:14:44.312147+0000","last_clean":"2023-10-08T00:14:44.312147+0000","last_became_active":"2023-10-08T00:14:44.311178+0000","last_became_peered":"2023-10-08T00:14:44.311178+0000","last_unstale":"2023-10-08T00:14:44.312147+0000","last_undegraded":"2023-10-08T00:14:44.312147+0000","last_fullsized":"2023-10-08T00:14:44.312147+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:17:57.649959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.314353+0000","last_change":"2023-10-08T00:14:44.313917+0000","last_active":"2023-10-08T00:14:44.314353+0000","last_peered":"2023-10-08T00:14:44.314353+0000","last_clean":"2023-10-08T00:14:44.314353+0000","last_became_active":"2023-10-08T00:14:44.313683+0000","last_became_peered":"2023-10-08T00:14:44.313683+0000","last_unstale":"2023-10-08T00:14:44.314353+0000","last_undegraded":"2023-10-08T00:14:44.314353+0000","last_fullsized":"2023-10-08T00:14:44.314353+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:31:35.202882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.315971+0000","last_change":"2023-10-08T00:14:44.315281+0000","last_active":"2023-10-08T00:14:44.315971+0000","last_peered":"2023-10-08T00:14:44.315971+0000","last_clean":"2023-10-08T00:14:44.315971+0000","last_became_active":"2023-10-08T00:14:44.312848+0000","last_became_peered":"2023-10-08T00:14:44.312848+0000","last_unstale":"2023-10-08T00:14:44.315971+0000","last_undegraded":"2023-10-08T00:14:44.315971+0000","last_fullsized":"2023-10-08T00:14:44.315971+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:26:51.881390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.311210+0000","last_change":"2023-10-08T00:14:44.311150+0000","last_active":"2023-10-08T00:14:44.311210+0000","last_peered":"2023-10-08T00:14:44.311210+0000","last_clean":"2023-10-08T00:14:44.311210+0000","last_became_active":"2023-10-08T00:14:44.311012+0000","last_became_peered":"2023-10-08T00:14:44.311012+0000","last_unstale":"2023-10-08T00:14:44.311210+0000","last_undegraded":"2023-10-08T00:14:44.311210+0000","last_fullsized":"2023-10-08T00:14:44.311210+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:47:52.704667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.311250+0000","last_change":"2023-10-08T00:14:44.310661+0000","last_active":"2023-10-08T00:14:44.311250+0000","last_peered":"2023-10-08T00:14:44.311250+0000","last_clean":"2023-10-08T00:14:44.311250+0000","last_became_active":"2023-10-08T00:14:44.310437+0000","last_became_peered":"2023-10-08T00:14:44.310437+0000","last_unstale":"2023-10-08T00:14:44.311250+0000","last_undegraded":"2023-10-08T00:14:44.311250+0000","last_fullsized":"2023-10-08T00:14:44.311250+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:25:53.638687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"85'4","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.323071+0000","last_change":"2023-10-08T00:14:43.296853+0000","last_active":"2023-10-08T00:14:44.323071+0000","last_peered":"2023-10-08T00:14:44.323071+0000","last_clean":"2023-10-08T00:14:44.323071+0000","last_became_active":"2023-10-08T00:14:43.296688+0000","last_became_peered":"2023-10-08T00:14:43.296688+0000","last_unstale":"2023-10-08T00:14:44.323071+0000","last_undegraded":"2023-10-08T00:14:44.323071+0000","last_fullsized":"2023-10-08T00:14:44.323071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:22:58.707158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.314810+0000","last_change":"2023-10-08T00:14:44.314316+0000","last_active":"2023-10-08T00:14:44.314810+0000","last_peered":"2023-10-08T00:14:44.314810+0000","last_clean":"2023-10-08T00:14:44.314810+0000","last_became_active":"2023-10-08T00:14:44.313787+0000","last_became_peered":"2023-10-08T00:14:44.313787+0000","last_unstale":"2023-10-08T00:14:44.314810+0000","last_undegraded":"2023-10-08T00:14:44.314810+0000","last_fullsized":"2023-10-08T00:14:44.314810+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:43:42.699284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"85'1","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.326526+0000","last_change":"2023-10-08T00:14:43.295617+0000","last_active":"2023-10-08T00:14:44.326526+0000","last_peered":"2023-10-08T00:14:44.326526+0000","last_clean":"2023-10-08T00:14:44.326526+0000","last_became_active":"2023-10-08T00:14:43.295397+0000","last_became_peered":"2023-10-08T00:14:43.295397+0000","last_unstale":"2023-10-08T00:14:44.326526+0000","last_undegraded":"2023-10-08T00:14:44.326526+0000","last_fullsized":"2023-10-08T00:14:44.326526+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:01:41.952885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.305180+0000","last_change":"2023-10-08T00:14:44.305058+0000","last_active":"2023-10-08T00:14:44.305180+0000","last_peered":"2023-10-08T00:14:44.305180+0000","last_clean":"2023-10-08T00:14:44.305180+0000","last_became_active":"2023-10-08T00:14:44.304828+0000","last_became_peered":"2023-10-08T00:14:44.304828+0000","last_unstale":"2023-10-08T00:14:44.305180+0000","last_undegraded":"2023-10-08T00:14:44.305180+0000","last_fullsized":"2023-10-08T00:14:44.305180+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:31:58.708770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"85'2","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.322798+0000","last_change":"2023-10-08T00:14:43.294237+0000","last_active":"2023-10-08T00:14:44.322798+0000","last_peered":"2023-10-08T00:14:44.322798+0000","last_clean":"2023-10-08T00:14:44.322798+0000","last_became_active":"2023-10-08T00:14:43.293992+0000","last_became_peered":"2023-10-08T00:14:43.293992+0000","last_unstale":"2023-10-08T00:14:44.322798+0000","last_undegraded":"2023-10-08T00:14:44.322798+0000","last_fullsized":"2023-10-08T00:14:44.322798+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:23:10.497366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.308389+0000","last_change":"2023-10-08T00:14:44.308326+0000","last_active":"2023-10-08T00:14:44.308389+0000","last_peered":"2023-10-08T00:14:44.308389+0000","last_clean":"2023-10-08T00:14:44.308389+0000","last_became_active":"2023-10-08T00:14:44.308153+0000","last_became_peered":"2023-10-08T00:14:44.308153+0000","last_unstale":"2023-10-08T00:14:44.308389+0000","last_undegraded":"2023-10-08T00:14:44.308389+0000","last_fullsized":"2023-10-08T00:14:44.308389+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:10:49.087366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"85'2","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.324309+0000","last_change":"2023-10-08T00:14:43.293651+0000","last_active":"2023-10-08T00:14:44.324309+0000","last_peered":"2023-10-08T00:14:44.324309+0000","last_clean":"2023-10-08T00:14:44.324309+0000","last_became_active":"2023-10-08T00:14:43.293063+0000","last_became_peered":"2023-10-08T00:14:43.293063+0000","last_unstale":"2023-10-08T00:14:44.324309+0000","last_undegraded":"2023-10-08T00:14:44.324309+0000","last_fullsized":"2023-10-08T00:14:44.324309+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:01:41.952885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.305928+0000","last_change":"2023-10-08T00:14:44.305769+0000","last_active":"2023-10-08T00:14:44.305928+0000","last_peered":"2023-10-08T00:14:44.305928+0000","last_clean":"2023-10-08T00:14:44.305928+0000","last_became_active":"2023-10-08T00:14:44.305504+0000","last_became_peered":"2023-10-08T00:14:44.305504+0000","last_unstale":"2023-10-08T00:14:44.305928+0000","last_undegraded":"2023-10-08T00:14:44.305928+0000","last_fullsized":"2023-10-08T00:14:44.305928+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:46:21.620833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"85'1","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.322291+0000","last_change":"2023-10-08T00:14:43.300434+0000","last_active":"2023-10-08T00:14:44.322291+0000","last_peered":"2023-10-08T00:14:44.322291+0000","last_clean":"2023-10-08T00:14:44.322291+0000","last_became_active":"2023-10-08T00:14:43.300297+0000","last_became_peered":"2023-10-08T00:14:43.300297+0000","last_unstale":"2023-10-08T00:14:44.322291+0000","last_undegraded":"2023-10-08T00:14:44.322291+0000","last_fullsized":"2023-10-08T00:14:44.322291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:52:16.707995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.303553+0000","last_change":"2023-10-08T00:14:44.303466+0000","last_active":"2023-10-08T00:14:44.303553+0000","last_peered":"2023-10-08T00:14:44.303553+0000","last_clean":"2023-10-08T00:14:44.303553+0000","last_became_active":"2023-10-08T00:14:44.303243+0000","last_became_peered":"2023-10-08T00:14:44.303243+0000","last_unstale":"2023-10-08T00:14:44.303553+0000","last_undegraded":"2023-10-08T00:14:44.303553+0000","last_fullsized":"2023-10-08T00:14:44.303553+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:47:00.324016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"85'1","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.326611+0000","last_change":"2023-10-08T00:14:43.303622+0000","last_active":"2023-10-08T00:14:44.326611+0000","last_peered":"2023-10-08T00:14:44.326611+0000","last_clean":"2023-10-08T00:14:44.326611+0000","last_became_active":"2023-10-08T00:14:43.303406+0000","last_became_peered":"2023-10-08T00:14:43.303406+0000","last_unstale":"2023-10-08T00:14:44.326611+0000","last_undegraded":"2023-10-08T00:14:44.326611+0000","last_fullsized":"2023-10-08T00:14:44.326611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:32:10.788785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.314882+0000","last_change":"2023-10-08T00:14:44.314120+0000","last_active":"2023-10-08T00:14:44.314882+0000","last_peered":"2023-10-08T00:14:44.314882+0000","last_clean":"2023-10-08T00:14:44.314882+0000","last_became_active":"2023-10-08T00:14:44.313891+0000","last_became_peered":"2023-10-08T00:14:44.313891+0000","last_unstale":"2023-10-08T00:14:44.314882+0000","last_undegraded":"2023-10-08T00:14:44.314882+0000","last_fullsized":"2023-10-08T00:14:44.314882+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:33:22.070538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.306422+0000","last_change":"2023-10-08T00:14:43.298762+0000","last_active":"2023-10-08T00:14:44.306422+0000","last_peered":"2023-10-08T00:14:44.306422+0000","last_clean":"2023-10-08T00:14:44.306422+0000","last_became_active":"2023-10-08T00:14:43.298520+0000","last_became_peered":"2023-10-08T00:14:43.298520+0000","last_unstale":"2023-10-08T00:14:44.306422+0000","last_undegraded":"2023-10-08T00:14:44.306422+0000","last_fullsized":"2023-10-08T00:14:44.306422+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:27:51.224820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.310782+0000","last_change":"2023-10-08T00:14:44.310218+0000","last_active":"2023-10-08T00:14:44.310782+0000","last_peered":"2023-10-08T00:14:44.310782+0000","last_clean":"2023-10-08T00:14:44.310782+0000","last_became_active":"2023-10-08T00:14:44.308468+0000","last_became_peered":"2023-10-08T00:14:44.308468+0000","last_unstale":"2023-10-08T00:14:44.310782+0000","last_undegraded":"2023-10-08T00:14:44.310782+0000","last_fullsized":"2023-10-08T00:14:44.310782+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:33:22.070538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.304327+0000","last_change":"2023-10-08T00:14:43.298044+0000","last_active":"2023-10-08T00:14:44.304327+0000","last_peered":"2023-10-08T00:14:44.304327+0000","last_clean":"2023-10-08T00:14:44.304327+0000","last_became_active":"2023-10-08T00:14:43.295683+0000","last_became_peered":"2023-10-08T00:14:43.295683+0000","last_unstale":"2023-10-08T00:14:44.304327+0000","last_undegraded":"2023-10-08T00:14:44.304327+0000","last_fullsized":"2023-10-08T00:14:44.304327+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:26:20.441741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.311924+0000","last_change":"2023-10-08T00:14:44.311825+0000","last_active":"2023-10-08T00:14:44.311924+0000","last_peered":"2023-10-08T00:14:44.311924+0000","last_clean":"2023-10-08T00:14:44.311924+0000","last_became_active":"2023-10-08T00:14:44.311570+0000","last_became_peered":"2023-10-08T00:14:44.311570+0000","last_unstale":"2023-10-08T00:14:44.311924+0000","last_undegraded":"2023-10-08T00:14:44.311924+0000","last_fullsized":"2023-10-08T00:14:44.311924+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:38:11.833743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.304468+0000","last_change":"2023-10-08T00:14:43.301418+0000","last_active":"2023-10-08T00:14:44.304468+0000","last_peered":"2023-10-08T00:14:44.304468+0000","last_clean":"2023-10-08T00:14:44.304468+0000","last_became_active":"2023-10-08T00:14:43.300950+0000","last_became_peered":"2023-10-08T00:14:43.300950+0000","last_unstale":"2023-10-08T00:14:44.304468+0000","last_undegraded":"2023-10-08T00:14:44.304468+0000","last_fullsized":"2023-10-08T00:14:44.304468+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:02:48.539900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.315988+0000","last_change":"2023-10-08T00:14:44.315424+0000","last_active":"2023-10-08T00:14:44.315988+0000","last_peered":"2023-10-08T00:14:44.315988+0000","last_clean":"2023-10-08T00:14:44.315988+0000","last_became_active":"2023-10-08T00:14:44.313161+0000","last_became_peered":"2023-10-08T00:14:44.313161+0000","last_unstale":"2023-10-08T00:14:44.315988+0000","last_undegraded":"2023-10-08T00:14:44.315988+0000","last_fullsized":"2023-10-08T00:14:44.315988+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:47:00.324016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.309080+0000","last_change":"2023-10-08T00:14:43.294524+0000","last_active":"2023-10-08T00:14:44.309080+0000","last_peered":"2023-10-08T00:14:44.309080+0000","last_clean":"2023-10-08T00:14:44.309080+0000","last_became_active":"2023-10-08T00:14:43.294281+0000","last_became_peered":"2023-10-08T00:14:43.294281+0000","last_unstale":"2023-10-08T00:14:44.309080+0000","last_undegraded":"2023-10-08T00:14:44.309080+0000","last_fullsized":"2023-10-08T00:14:44.309080+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:35:37.080411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.309358+0000","last_change":"2023-10-08T00:14:44.309284+0000","last_active":"2023-10-08T00:14:44.309358+0000","last_peered":"2023-10-08T00:14:44.309358+0000","last_clean":"2023-10-08T00:14:44.309358+0000","last_became_active":"2023-10-08T00:14:44.308537+0000","last_became_peered":"2023-10-08T00:14:44.308537+0000","last_unstale":"2023-10-08T00:14:44.309358+0000","last_undegraded":"2023-10-08T00:14:44.309358+0000","last_fullsized":"2023-10-08T00:14:44.309358+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:29:06.196423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"85'3","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.322204+0000","last_change":"2023-10-08T00:14:43.297383+0000","last_active":"2023-10-08T00:14:44.322204+0000","last_peered":"2023-10-08T00:14:44.322204+0000","last_clean":"2023-10-08T00:14:44.322204+0000","last_became_active":"2023-10-08T00:14:43.296565+0000","last_became_peered":"2023-10-08T00:14:43.296565+0000","last_unstale":"2023-10-08T00:14:44.322204+0000","last_undegraded":"2023-10-08T00:14:44.322204+0000","last_fullsized":"2023-10-08T00:14:44.322204+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:10:48.082428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.312449+0000","last_change":"2023-10-08T00:14:44.312318+0000","last_active":"2023-10-08T00:14:44.312449+0000","last_peered":"2023-10-08T00:14:44.312449+0000","last_clean":"2023-10-08T00:14:44.312449+0000","last_became_active":"2023-10-08T00:14:44.311706+0000","last_became_peered":"2023-10-08T00:14:44.311706+0000","last_unstale":"2023-10-08T00:14:44.312449+0000","last_undegraded":"2023-10-08T00:14:44.312449+0000","last_fullsized":"2023-10-08T00:14:44.312449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:04:04.824912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"85'6","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.326516+0000","last_change":"2023-10-08T00:14:43.295216+0000","last_active":"2023-10-08T00:14:44.326516+0000","last_peered":"2023-10-08T00:14:44.326516+0000","last_clean":"2023-10-08T00:14:44.326516+0000","last_became_active":"2023-10-08T00:14:43.294989+0000","last_became_peered":"2023-10-08T00:14:43.294989+0000","last_unstale":"2023-10-08T00:14:44.326516+0000","last_undegraded":"2023-10-08T00:14:44.326516+0000","last_fullsized":"2023-10-08T00:14:44.326516+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:26:20.441741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.305892+0000","last_change":"2023-10-08T00:14:44.305779+0000","last_active":"2023-10-08T00:14:44.305892+0000","last_peered":"2023-10-08T00:14:44.305892+0000","last_clean":"2023-10-08T00:14:44.305892+0000","last_became_active":"2023-10-08T00:14:44.305560+0000","last_became_peered":"2023-10-08T00:14:44.305560+0000","last_unstale":"2023-10-08T00:14:44.305892+0000","last_undegraded":"2023-10-08T00:14:44.305892+0000","last_fullsized":"2023-10-08T00:14:44.305892+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:09:33.137344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"85'2","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.323938+0000","last_change":"2023-10-08T00:14:43.294068+0000","last_active":"2023-10-08T00:14:44.323938+0000","last_peered":"2023-10-08T00:14:44.323938+0000","last_clean":"2023-10-08T00:14:44.323938+0000","last_became_active":"2023-10-08T00:14:43.293845+0000","last_became_peered":"2023-10-08T00:14:43.293845+0000","last_unstale":"2023-10-08T00:14:44.323938+0000","last_undegraded":"2023-10-08T00:14:44.323938+0000","last_fullsized":"2023-10-08T00:14:44.323938+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:43:41.694345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.307940+0000","last_change":"2023-10-08T00:14:44.307384+0000","last_active":"2023-10-08T00:14:44.307940+0000","last_peered":"2023-10-08T00:14:44.307940+0000","last_clean":"2023-10-08T00:14:44.307940+0000","last_became_active":"2023-10-08T00:14:44.307151+0000","last_became_peered":"2023-10-08T00:14:44.307151+0000","last_unstale":"2023-10-08T00:14:44.307940+0000","last_undegraded":"2023-10-08T00:14:44.307940+0000","last_fullsized":"2023-10-08T00:14:44.307940+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:49:23.052153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.305799+0000","last_change":"2023-10-08T00:14:43.296275+0000","last_active":"2023-10-08T00:14:44.305799+0000","last_peered":"2023-10-08T00:14:44.305799+0000","last_clean":"2023-10-08T00:14:44.305799+0000","last_became_active":"2023-10-08T00:14:43.296041+0000","last_became_peered":"2023-10-08T00:14:43.296041+0000","last_unstale":"2023-10-08T00:14:44.305799+0000","last_undegraded":"2023-10-08T00:14:44.305799+0000","last_fullsized":"2023-10-08T00:14:44.305799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:51:24.797873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.314571+0000","last_change":"2023-10-08T00:14:44.314491+0000","last_active":"2023-10-08T00:14:44.314571+0000","last_peered":"2023-10-08T00:14:44.314571+0000","last_clean":"2023-10-08T00:14:44.314571+0000","last_became_active":"2023-10-08T00:14:44.314252+0000","last_became_peered":"2023-10-08T00:14:44.314252+0000","last_unstale":"2023-10-08T00:14:44.314571+0000","last_undegraded":"2023-10-08T00:14:44.314571+0000","last_fullsized":"2023-10-08T00:14:44.314571+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:54:13.055140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.306932+0000","last_change":"2023-10-08T00:14:43.298585+0000","last_active":"2023-10-08T00:14:44.306932+0000","last_peered":"2023-10-08T00:14:44.306932+0000","last_clean":"2023-10-08T00:14:44.306932+0000","last_became_active":"2023-10-08T00:14:43.298455+0000","last_became_peered":"2023-10-08T00:14:43.298455+0000","last_unstale":"2023-10-08T00:14:44.306932+0000","last_undegraded":"2023-10-08T00:14:44.306932+0000","last_fullsized":"2023-10-08T00:14:44.306932+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:27:24.902226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.314054+0000","last_change":"2023-10-08T00:14:44.313732+0000","last_active":"2023-10-08T00:14:44.314054+0000","last_peered":"2023-10-08T00:14:44.314054+0000","last_clean":"2023-10-08T00:14:44.314054+0000","last_became_active":"2023-10-08T00:14:44.313476+0000","last_became_peered":"2023-10-08T00:14:44.313476+0000","last_unstale":"2023-10-08T00:14:44.314054+0000","last_undegraded":"2023-10-08T00:14:44.314054+0000","last_fullsized":"2023-10-08T00:14:44.314054+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:10:49.087366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"85'8","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.326523+0000","last_change":"2023-10-08T00:14:43.301832+0000","last_active":"2023-10-08T00:14:44.326523+0000","last_peered":"2023-10-08T00:14:44.326523+0000","last_clean":"2023-10-08T00:14:44.326523+0000","last_became_active":"2023-10-08T00:14:43.301610+0000","last_became_peered":"2023-10-08T00:14:43.301610+0000","last_unstale":"2023-10-08T00:14:44.326523+0000","last_undegraded":"2023-10-08T00:14:44.326523+0000","last_fullsized":"2023-10-08T00:14:44.326523+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:10:10.820384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.315206+0000","last_change":"2023-10-08T00:14:44.315143+0000","last_active":"2023-10-08T00:14:44.315206+0000","last_peered":"2023-10-08T00:14:44.315206+0000","last_clean":"2023-10-08T00:14:44.315206+0000","last_became_active":"2023-10-08T00:14:44.314895+0000","last_became_peered":"2023-10-08T00:14:44.314895+0000","last_unstale":"2023-10-08T00:14:44.315206+0000","last_undegraded":"2023-10-08T00:14:44.315206+0000","last_fullsized":"2023-10-08T00:14:44.315206+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:50:20.430773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"85'1","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.322942+0000","last_change":"2023-10-08T00:14:43.301356+0000","last_active":"2023-10-08T00:14:44.322942+0000","last_peered":"2023-10-08T00:14:44.322942+0000","last_clean":"2023-10-08T00:14:44.322942+0000","last_became_active":"2023-10-08T00:14:43.301143+0000","last_became_peered":"2023-10-08T00:14:43.301143+0000","last_unstale":"2023-10-08T00:14:44.322942+0000","last_undegraded":"2023-10-08T00:14:44.322942+0000","last_fullsized":"2023-10-08T00:14:44.322942+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:04:03.819974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.314841+0000","last_change":"2023-10-08T00:14:44.314800+0000","last_active":"2023-10-08T00:14:44.314841+0000","last_peered":"2023-10-08T00:14:44.314841+0000","last_clean":"2023-10-08T00:14:44.314841+0000","last_became_active":"2023-10-08T00:14:44.314638+0000","last_became_peered":"2023-10-08T00:14:44.314638+0000","last_unstale":"2023-10-08T00:14:44.314841+0000","last_undegraded":"2023-10-08T00:14:44.314841+0000","last_fullsized":"2023-10-08T00:14:44.314841+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:13:05.430418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"85'2","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.327091+0000","last_change":"2023-10-08T00:14:43.298391+0000","last_active":"2023-10-08T00:14:44.327091+0000","last_peered":"2023-10-08T00:14:44.327091+0000","last_clean":"2023-10-08T00:14:44.327091+0000","last_became_active":"2023-10-08T00:14:43.298175+0000","last_became_peered":"2023-10-08T00:14:43.298175+0000","last_unstale":"2023-10-08T00:14:44.327091+0000","last_undegraded":"2023-10-08T00:14:44.327091+0000","last_fullsized":"2023-10-08T00:14:44.327091+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:45:12.140067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.314826+0000","last_change":"2023-10-08T00:14:44.314581+0000","last_active":"2023-10-08T00:14:44.314826+0000","last_peered":"2023-10-08T00:14:44.314826+0000","last_clean":"2023-10-08T00:14:44.314826+0000","last_became_active":"2023-10-08T00:14:44.314306+0000","last_became_peered":"2023-10-08T00:14:44.314306+0000","last_unstale":"2023-10-08T00:14:44.314826+0000","last_undegraded":"2023-10-08T00:14:44.314826+0000","last_fullsized":"2023-10-08T00:14:44.314826+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:26:21.446680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"85'1","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.324975+0000","last_change":"2023-10-08T00:14:43.304417+0000","last_active":"2023-10-08T00:14:44.324975+0000","last_peered":"2023-10-08T00:14:44.324975+0000","last_clean":"2023-10-08T00:14:44.324975+0000","last_became_active":"2023-10-08T00:14:43.304168+0000","last_became_peered":"2023-10-08T00:14:43.304168+0000","last_unstale":"2023-10-08T00:14:44.324975+0000","last_undegraded":"2023-10-08T00:14:44.324975+0000","last_fullsized":"2023-10-08T00:14:44.324975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:17:56.645020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.313180+0000","last_change":"2023-10-08T00:14:44.313095+0000","last_active":"2023-10-08T00:14:44.313180+0000","last_peered":"2023-10-08T00:14:44.313180+0000","last_clean":"2023-10-08T00:14:44.313180+0000","last_became_active":"2023-10-08T00:14:44.312843+0000","last_became_peered":"2023-10-08T00:14:44.312843+0000","last_unstale":"2023-10-08T00:14:44.313180+0000","last_undegraded":"2023-10-08T00:14:44.313180+0000","last_fullsized":"2023-10-08T00:14:44.313180+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:45:13.145006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.300817+0000","last_change":"2023-10-08T00:14:43.291042+0000","last_active":"2023-10-08T00:14:44.300817+0000","last_peered":"2023-10-08T00:14:44.300817+0000","last_clean":"2023-10-08T00:14:44.300817+0000","last_became_active":"2023-10-08T00:14:43.290806+0000","last_became_peered":"2023-10-08T00:14:43.290806+0000","last_unstale":"2023-10-08T00:14:44.300817+0000","last_undegraded":"2023-10-08T00:14:44.300817+0000","last_fullsized":"2023-10-08T00:14:44.300817+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:50:49.574964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.310716+0000","last_change":"2023-10-08T00:14:44.310650+0000","last_active":"2023-10-08T00:14:44.310716+0000","last_peered":"2023-10-08T00:14:44.310716+0000","last_clean":"2023-10-08T00:14:44.310716+0000","last_became_active":"2023-10-08T00:14:44.310419+0000","last_became_peered":"2023-10-08T00:14:44.310419+0000","last_unstale":"2023-10-08T00:14:44.310716+0000","last_undegraded":"2023-10-08T00:14:44.310716+0000","last_fullsized":"2023-10-08T00:14:44.310716+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:56:39.552066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.300190+0000","last_change":"2023-10-08T00:14:43.301524+0000","last_active":"2023-10-08T00:14:44.300190+0000","last_peered":"2023-10-08T00:14:44.300190+0000","last_clean":"2023-10-08T00:14:44.300190+0000","last_became_active":"2023-10-08T00:14:43.301054+0000","last_became_peered":"2023-10-08T00:14:43.301054+0000","last_unstale":"2023-10-08T00:14:44.300190+0000","last_undegraded":"2023-10-08T00:14:44.300190+0000","last_fullsized":"2023-10-08T00:14:44.300190+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:39:26.312424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.313117+0000","last_change":"2023-10-08T00:14:44.313067+0000","last_active":"2023-10-08T00:14:44.313117+0000","last_peered":"2023-10-08T00:14:44.313117+0000","last_clean":"2023-10-08T00:14:44.313117+0000","last_became_active":"2023-10-08T00:14:44.312844+0000","last_became_peered":"2023-10-08T00:14:44.312844+0000","last_unstale":"2023-10-08T00:14:44.313117+0000","last_undegraded":"2023-10-08T00:14:44.313117+0000","last_fullsized":"2023-10-08T00:14:44.313117+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:52:24.163704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.310629+0000","last_change":"2023-10-08T00:14:43.299330+0000","last_active":"2023-10-08T00:14:44.310629+0000","last_peered":"2023-10-08T00:14:44.310629+0000","last_clean":"2023-10-08T00:14:44.310629+0000","last_became_active":"2023-10-08T00:14:43.298909+0000","last_became_peered":"2023-10-08T00:14:43.298909+0000","last_unstale":"2023-10-08T00:14:44.310629+0000","last_undegraded":"2023-10-08T00:14:44.310629+0000","last_fullsized":"2023-10-08T00:14:44.310629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:21.229437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.311954+0000","last_change":"2023-10-08T00:14:44.311787+0000","last_active":"2023-10-08T00:14:44.311954+0000","last_peered":"2023-10-08T00:14:44.311954+0000","last_clean":"2023-10-08T00:14:44.311954+0000","last_became_active":"2023-10-08T00:14:44.310198+0000","last_became_peered":"2023-10-08T00:14:44.310198+0000","last_unstale":"2023-10-08T00:14:44.311954+0000","last_undegraded":"2023-10-08T00:14:44.311954+0000","last_fullsized":"2023-10-08T00:14:44.311954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:26:44.694537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.303547+0000","last_change":"2023-10-08T00:14:43.298576+0000","last_active":"2023-10-08T00:14:44.303547+0000","last_peered":"2023-10-08T00:14:44.303547+0000","last_clean":"2023-10-08T00:14:44.303547+0000","last_became_active":"2023-10-08T00:14:43.298336+0000","last_became_peered":"2023-10-08T00:14:43.298336+0000","last_unstale":"2023-10-08T00:14:44.303547+0000","last_undegraded":"2023-10-08T00:14:44.303547+0000","last_fullsized":"2023-10-08T00:14:44.303547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:51:57.785852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.309626+0000","last_change":"2023-10-08T00:14:44.308193+0000","last_active":"2023-10-08T00:14:44.309626+0000","last_peered":"2023-10-08T00:14:44.309626+0000","last_clean":"2023-10-08T00:14:44.309626+0000","last_became_active":"2023-10-08T00:14:44.307903+0000","last_became_peered":"2023-10-08T00:14:44.307903+0000","last_unstale":"2023-10-08T00:14:44.309626+0000","last_undegraded":"2023-10-08T00:14:44.309626+0000","last_fullsized":"2023-10-08T00:14:44.309626+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:24:42.915827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"85'2","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.323070+0000","last_change":"2023-10-08T00:14:43.302228+0000","last_active":"2023-10-08T00:14:44.323070+0000","last_peered":"2023-10-08T00:14:44.323070+0000","last_clean":"2023-10-08T00:14:44.323070+0000","last_became_active":"2023-10-08T00:14:43.302099+0000","last_became_peered":"2023-10-08T00:14:43.302099+0000","last_unstale":"2023-10-08T00:14:44.323070+0000","last_undegraded":"2023-10-08T00:14:44.323070+0000","last_fullsized":"2023-10-08T00:14:44.323070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:24:41.910888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.313225+0000","last_change":"2023-10-08T00:14:44.313123+0000","last_active":"2023-10-08T00:14:44.313225+0000","last_peered":"2023-10-08T00:14:44.313225+0000","last_clean":"2023-10-08T00:14:44.313225+0000","last_became_active":"2023-10-08T00:14:44.312546+0000","last_became_peered":"2023-10-08T00:14:44.312546+0000","last_unstale":"2023-10-08T00:14:44.313225+0000","last_undegraded":"2023-10-08T00:14:44.313225+0000","last_fullsized":"2023-10-08T00:14:44.313225+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:48:38.177938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.303082+0000","last_change":"2023-10-08T00:14:43.299951+0000","last_active":"2023-10-08T00:14:44.303082+0000","last_peered":"2023-10-08T00:14:44.303082+0000","last_clean":"2023-10-08T00:14:44.303082+0000","last_became_active":"2023-10-08T00:14:43.299758+0000","last_became_peered":"2023-10-08T00:14:43.299758+0000","last_unstale":"2023-10-08T00:14:44.303082+0000","last_undegraded":"2023-10-08T00:14:44.303082+0000","last_fullsized":"2023-10-08T00:14:44.303082+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:44:44.807607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.312636+0000","last_change":"2023-10-08T00:14:44.312408+0000","last_active":"2023-10-08T00:14:44.312636+0000","last_peered":"2023-10-08T00:14:44.312636+0000","last_clean":"2023-10-08T00:14:44.312636+0000","last_became_active":"2023-10-08T00:14:44.312187+0000","last_became_peered":"2023-10-08T00:14:44.312187+0000","last_unstale":"2023-10-08T00:14:44.312636+0000","last_undegraded":"2023-10-08T00:14:44.312636+0000","last_fullsized":"2023-10-08T00:14:44.312636+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:09:33.137344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.303284+0000","last_change":"2023-10-08T00:14:43.295854+0000","last_active":"2023-10-08T00:14:44.303284+0000","last_peered":"2023-10-08T00:14:44.303284+0000","last_clean":"2023-10-08T00:14:44.303284+0000","last_became_active":"2023-10-08T00:14:43.295593+0000","last_became_peered":"2023-10-08T00:14:43.295593+0000","last_unstale":"2023-10-08T00:14:44.303284+0000","last_undegraded":"2023-10-08T00:14:44.303284+0000","last_fullsized":"2023-10-08T00:14:44.303284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:43:41.694345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.307962+0000","last_change":"2023-10-08T00:14:44.307664+0000","last_active":"2023-10-08T00:14:44.307962+0000","last_peered":"2023-10-08T00:14:44.307962+0000","last_clean":"2023-10-08T00:14:44.307962+0000","last_became_active":"2023-10-08T00:14:44.307457+0000","last_became_peered":"2023-10-08T00:14:44.307457+0000","last_unstale":"2023-10-08T00:14:44.307962+0000","last_undegraded":"2023-10-08T00:14:44.307962+0000","last_fullsized":"2023-10-08T00:14:44.307962+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:35:38.085349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.304418+0000","last_change":"2023-10-08T00:14:43.298666+0000","last_active":"2023-10-08T00:14:44.304418+0000","last_peered":"2023-10-08T00:14:44.304418+0000","last_clean":"2023-10-08T00:14:44.304418+0000","last_became_active":"2023-10-08T00:14:43.297793+0000","last_became_peered":"2023-10-08T00:14:43.297793+0000","last_unstale":"2023-10-08T00:14:44.304418+0000","last_undegraded":"2023-10-08T00:14:44.304418+0000","last_fullsized":"2023-10-08T00:14:44.304418+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:49:22.047215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.312162+0000","last_change":"2023-10-08T00:14:44.312014+0000","last_active":"2023-10-08T00:14:44.312162+0000","last_peered":"2023-10-08T00:14:44.312162+0000","last_clean":"2023-10-08T00:14:44.312162+0000","last_became_active":"2023-10-08T00:14:44.310128+0000","last_became_peered":"2023-10-08T00:14:44.310128+0000","last_unstale":"2023-10-08T00:14:44.312162+0000","last_undegraded":"2023-10-08T00:14:44.312162+0000","last_fullsized":"2023-10-08T00:14:44.312162+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:46:26.233800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.302814+0000","last_change":"2023-10-08T00:14:43.301528+0000","last_active":"2023-10-08T00:14:44.302814+0000","last_peered":"2023-10-08T00:14:44.302814+0000","last_clean":"2023-10-08T00:14:44.302814+0000","last_became_active":"2023-10-08T00:14:43.301291+0000","last_became_peered":"2023-10-08T00:14:43.301291+0000","last_unstale":"2023-10-08T00:14:44.302814+0000","last_undegraded":"2023-10-08T00:14:44.302814+0000","last_fullsized":"2023-10-08T00:14:44.302814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:27:51.224820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.311267+0000","last_change":"2023-10-08T00:14:44.311005+0000","last_active":"2023-10-08T00:14:44.311267+0000","last_peered":"2023-10-08T00:14:44.311267+0000","last_clean":"2023-10-08T00:14:44.311267+0000","last_became_active":"2023-10-08T00:14:44.310466+0000","last_became_peered":"2023-10-08T00:14:44.310466+0000","last_unstale":"2023-10-08T00:14:44.311267+0000","last_undegraded":"2023-10-08T00:14:44.311267+0000","last_fullsized":"2023-10-08T00:14:44.311267+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:56:38.021830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.306219+0000","last_change":"2023-10-08T00:14:43.292988+0000","last_active":"2023-10-08T00:14:44.306219+0000","last_peered":"2023-10-08T00:14:44.306219+0000","last_clean":"2023-10-08T00:14:44.306219+0000","last_became_active":"2023-10-08T00:14:43.292285+0000","last_became_peered":"2023-10-08T00:14:43.292285+0000","last_unstale":"2023-10-08T00:14:44.306219+0000","last_undegraded":"2023-10-08T00:14:44.306219+0000","last_fullsized":"2023-10-08T00:14:44.306219+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:02:00.046943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.312638+0000","last_change":"2023-10-08T00:14:44.311482+0000","last_active":"2023-10-08T00:14:44.312638+0000","last_peered":"2023-10-08T00:14:44.312638+0000","last_clean":"2023-10-08T00:14:44.312638+0000","last_became_active":"2023-10-08T00:14:44.311250+0000","last_became_peered":"2023-10-08T00:14:44.311250+0000","last_unstale":"2023-10-08T00:14:44.312638+0000","last_undegraded":"2023-10-08T00:14:44.312638+0000","last_fullsized":"2023-10-08T00:14:44.312638+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:26:44.694537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.313616+0000","last_change":"2023-10-08T00:14:43.300700+0000","last_active":"2023-10-08T00:14:44.313616+0000","last_peered":"2023-10-08T00:14:44.313616+0000","last_clean":"2023-10-08T00:14:44.313616+0000","last_became_active":"2023-10-08T00:14:43.300471+0000","last_became_peered":"2023-10-08T00:14:43.300471+0000","last_unstale":"2023-10-08T00:14:44.313616+0000","last_undegraded":"2023-10-08T00:14:44.313616+0000","last_fullsized":"2023-10-08T00:14:44.313616+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:27:51.224820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.314189+0000","last_change":"2023-10-08T00:14:44.314125+0000","last_active":"2023-10-08T00:14:44.314189+0000","last_peered":"2023-10-08T00:14:44.314189+0000","last_clean":"2023-10-08T00:14:44.314189+0000","last_became_active":"2023-10-08T00:14:44.313903+0000","last_became_peered":"2023-10-08T00:14:44.313903+0000","last_unstale":"2023-10-08T00:14:44.314189+0000","last_undegraded":"2023-10-08T00:14:44.314189+0000","last_fullsized":"2023-10-08T00:14:44.314189+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:21:40.091830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.300284+0000","last_change":"2023-10-08T00:14:43.297408+0000","last_active":"2023-10-08T00:14:44.300284+0000","last_peered":"2023-10-08T00:14:44.300284+0000","last_clean":"2023-10-08T00:14:44.300284+0000","last_became_active":"2023-10-08T00:14:43.297186+0000","last_became_peered":"2023-10-08T00:14:43.297186+0000","last_unstale":"2023-10-08T00:14:44.300284+0000","last_undegraded":"2023-10-08T00:14:44.300284+0000","last_fullsized":"2023-10-08T00:14:44.300284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:47:51.699728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.309607+0000","last_change":"2023-10-08T00:14:44.307156+0000","last_active":"2023-10-08T00:14:44.309607+0000","last_peered":"2023-10-08T00:14:44.309607+0000","last_clean":"2023-10-08T00:14:44.309607+0000","last_became_active":"2023-10-08T00:14:44.306915+0000","last_became_peered":"2023-10-08T00:14:44.306915+0000","last_unstale":"2023-10-08T00:14:44.309607+0000","last_undegraded":"2023-10-08T00:14:44.309607+0000","last_fullsized":"2023-10-08T00:14:44.309607+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:43.284339+0000","last_clean_scrub_stamp":"2023-10-08T00:14:43.284339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:23:11.502305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"85'2","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.326701+0000","last_change":"2023-10-08T00:14:43.299421+0000","last_active":"2023-10-08T00:14:44.326701+0000","last_peered":"2023-10-08T00:14:44.326701+0000","last_clean":"2023-10-08T00:14:44.326701+0000","last_became_active":"2023-10-08T00:14:43.298880+0000","last_became_peered":"2023-10-08T00:14:43.298880+0000","last_unstale":"2023-10-08T00:14:44.326701+0000","last_undegraded":"2023-10-08T00:14:44.326701+0000","last_fullsized":"2023-10-08T00:14:44.326701+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:14:42.279401+0000","last_clean_scrub_stamp":"2023-10-08T00:14:42.279401+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:43:54.613305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":217,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T00:14:44.307113+0000","last_change":"2023-10-08T00:10:43.728490+0000","last_active":"2023-10-08T00:14:44.307113+0000","last_peered":"2023-10-08T00:14:44.307113+0000","last_clean":"2023-10-08T00:14:44.307113+0000","last_became_active":"2023-10-08T00:10:43.728341+0000","last_became_peered":"2023-10-08T00:10:43.728341+0000","last_unstale":"2023-10-08T00:14:44.307113+0000","last_undegraded":"2023-10-08T00:14:44.307113+0000","last_fullsized":"2023-10-08T00:14:44.307113+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_clean_scrub_stamp":"2023-10-08T00:10:42.716096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:06:31.035293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574899,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":449,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194863,"statfs":{"total":200000000,"available":199539775,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:13:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.51000000000000001,"15min":0.499},"min":{"1min":0.41699999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.53800000000000003,"15min":0.54400000000000004},"min":{"1min":0.374,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.77900000000000003,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.56200000000000006}]},{"osd":1,"last update":"Sun Oct 8 00:13:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.54100000000000004,"15min":0.53100000000000003},"min":{"1min":0.37,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.56899999999999995,"15min":0.56100000000000005},"min":{"1min":0.44400000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.76800000000000002,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.53000000000000003}]},{"osd":2,"last update":"Sun Oct 8 00:13:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54200000000000004,"15min":0.53700000000000003},"min":{"1min":0.40300000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.435},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.52800000000000002,"15min":0.51800000000000002},"min":{"1min":0.37,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.504}]},{"osd":3,"last update":"Sun Oct 8 00:13:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52600000000000002,"15min":0.51600000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.56200000000000006,"15min":0.55700000000000005},"min":{"1min":0.45700000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.54400000000000004}]},{"osd":4,"last update":"Sun Oct 8 00:13:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.52300000000000002,"15min":0.52600000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52300000000000002,"15min":0.53000000000000003},"min":{"1min":0.32800000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.747,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.61899999999999999}]},{"osd":5,"last update":"Sun Oct 8 00:13:45 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.54200000000000004,"15min":0.55400000000000005},"min":{"1min":0.315,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57299999999999995,"15min":0.58999999999999997},"min":{"1min":0.38500000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.68899999999999995}]},{"osd":6,"last update":"Sun Oct 8 00:13:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59399999999999997,"15min":0.59899999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76900000000000002,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.62,"15min":0.63600000000000001},"min":{"1min":0.36099999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.67600000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574899,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":449,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194863,"statfs":{"total":200000000,"available":199539741,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.48999999999999999,"15min":0.47799999999999998},"min":{"1min":0.34599999999999997,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.71399999999999997,"5min":1.012,"15min":1.012},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.64800000000000002,"15min":0.69299999999999995},"min":{"1min":0.374,"5min":0.317,"15min":0.317},"max":{"1min":0.70999999999999996,"5min":5.1230000000000002,"15min":5.1230000000000002},"last":0.57099999999999995}]},{"osd":1,"last update":"Sun Oct 8 00:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.53400000000000003,"15min":0.51800000000000002},"min":{"1min":0.39500000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.752,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.56599999999999995,"15min":0.56200000000000006},"min":{"1min":0.45400000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.73099999999999998,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.61299999999999999}]},{"osd":2,"last update":"Sun Oct 8 00:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.53600000000000003,"15min":0.52800000000000002},"min":{"1min":0.42099999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72199999999999998,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.56799999999999995,"15min":0.56699999999999995},"min":{"1min":0.48899999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.71599999999999997,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.59899999999999998}]},{"osd":3,"last update":"Sun Oct 8 00:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.54600000000000004,"15min":0.53400000000000003},"min":{"1min":0.501,"5min":0.371,"15min":0.371},"max":{"1min":0.745,"5min":1.046,"15min":1.046},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.55400000000000005,"15min":0.54000000000000004},"min":{"1min":0.36099999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.69199999999999995,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.55000000000000004}]},{"osd":4,"last update":"Sun Oct 8 00:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.49099999999999999,"15min":0.502},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.628,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55700000000000005,"15min":0.56499999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73599999999999999,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.46000000000000002}]},{"osd":5,"last update":"Sun Oct 8 00:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.48599999999999999,"15min":0.48799999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.70999999999999996,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.54400000000000004,"15min":0.55900000000000005},"min":{"1min":0.39800000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.71799999999999997,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.379}]},{"osd":7,"last update":"Sun Oct 8 00:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.53500000000000003,"15min":0.54300000000000004},"min":{"1min":0.36399999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76500000000000001,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54500000000000004,"15min":0.54800000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.73999999999999999,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.502}]}]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1024,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194288,"statfs":{"total":200000000,"available":198950943,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:13:49 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.315,"15min":0.30299999999999999},"min":{"1min":0.20799999999999999,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"last":0.22},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.38,"15min":0.374},"min":{"1min":0.25700000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.32300000000000001}]},{"osd":2,"last update":"Sun Oct 8 00:13:49 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.33800000000000002,"15min":0.33300000000000002},"min":{"1min":0.22900000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.254},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.34999999999999998,"15min":0.33600000000000002},"min":{"1min":0.27300000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.28599999999999998}]},{"osd":3,"last update":"Sun Oct 8 00:13:49 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.375,"15min":0.36699999999999999},"min":{"1min":0.307,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.54900000000000004,"5min":0.623,"15min":0.623},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.34499999999999997,"15min":0.34100000000000003},"min":{"1min":0.23400000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.30399999999999999}]},{"osd":4,"last update":"Sun Oct 8 00:13:49 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.54500000000000004,"15min":0.53700000000000003},"min":{"1min":0.40000000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57199999999999995,"15min":0.56599999999999995},"min":{"1min":0.39500000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.76800000000000002,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.55800000000000005}]},{"osd":5,"last update":"Sun Oct 8 00:13:49 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.54700000000000004,"15min":0.53400000000000003},"min":{"1min":0.42099999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.504},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57999999999999996,"15min":0.57399999999999995},"min":{"1min":0.38600000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.58799999999999997}]},{"osd":6,"last update":"Sun Oct 8 00:13:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.57099999999999995,"15min":0.56399999999999995},"min":{"1min":0.43099999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.82299999999999995,"5min":0.877,"15min":0.877},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56699999999999995,"15min":0.55900000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.83899999999999997,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.60299999999999998}]},{"osd":7,"last update":"Sun Oct 8 00:13:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.56999999999999995,"15min":0.56200000000000006},"min":{"1min":0.42399999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.622,"5min":0.59699999999999998,"15min":0.58699999999999997},"min":{"1min":0.45200000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.85599999999999998,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.61599999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574899,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":448,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194864,"statfs":{"total":200000000,"available":199541241,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 00:13:49 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.33100000000000002,"15min":0.32900000000000001},"min":{"1min":0.215,"5min":0.193,"15min":0.193},"max":{"1min":0.60499999999999998,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.314},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.34999999999999998,"15min":0.34599999999999997},"min":{"1min":0.23100000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.57699999999999996,"5min":0.87,"15min":0.87},"last":0.39400000000000002}]},{"osd":2,"last update":"Sun Oct 8 00:13:49 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.38500000000000001,"15min":0.38700000000000001},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.68000000000000005,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.66200000000000003,"15min":0.79400000000000004},"min":{"1min":0.24099999999999999,"5min":0.185,"15min":0.185},"max":{"1min":0.622,"5min":12.019,"15min":12.019},"last":0.35899999999999999}]},{"osd":3,"last update":"Sun Oct 8 00:13:49 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.36399999999999999,"15min":0.36199999999999999},"min":{"1min":0.25700000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.69999999999999996,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.439,"5min":0.39700000000000002,"15min":0.39300000000000002},"min":{"1min":0.28499999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.72899999999999998,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.42999999999999999}]},{"osd":4,"last update":"Sun Oct 8 00:13:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.62,"15min":0.61899999999999999},"min":{"1min":0.40000000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82599999999999996,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.628,"15min":0.626},"min":{"1min":0.42399999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.94599999999999995,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.66600000000000004}]},{"osd":5,"last update":"Sun Oct 8 00:13:49 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.90300000000000002,"15min":1.0369999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.85599999999999998,"5min":11.992000000000001,"15min":11.992000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59299999999999997,"15min":0.58599999999999997},"min":{"1min":0.502,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.90300000000000002,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.70199999999999996}]},{"osd":6,"last update":"Sun Oct 8 00:13:49 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.59399999999999997,"15min":0.58699999999999997},"min":{"1min":0.43099999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.877,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.89100000000000001,"15min":1.0129999999999999},"min":{"1min":0.434,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.96599999999999997,"5min":11.927,"15min":11.927},"last":0.60999999999999999}]},{"osd":7,"last update":"Sun Oct 8 00:13:49 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.90800000000000003,"15min":1.036},"min":{"1min":0.44500000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0069999999999999,"5min":11.965,"15min":11.965},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.629,"15min":0.625},"min":{"1min":0.54200000000000004,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.98299999999999998,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.748}]}]},{"osd":2,"up_from":13,"seq":55834574899,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":448,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194864,"statfs":{"total":200000000,"available":199540757,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.37,"15min":0.39500000000000002},"min":{"1min":0.24199999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.498,"5min":2.5139999999999998,"15min":2.5139999999999998},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.376,"15min":0.40600000000000003},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.56399999999999995,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":0.255}]},{"osd":1,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.36899999999999999,"15min":0.39500000000000002},"min":{"1min":0.249,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.48499999999999999,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.372,"5min":0.40799999999999997,"15min":0.44},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.53400000000000003,"5min":2.5379999999999998,"15min":2.5379999999999998},"last":0.33700000000000002}]},{"osd":3,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.41099999999999998,"15min":0.433},"min":{"1min":0.255,"5min":0.254,"15min":0.254},"max":{"1min":0.54800000000000004,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.436,"15min":0.46800000000000003},"min":{"1min":0.30399999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.501,"5min":2.5619999999999998,"15min":2.5619999999999998},"last":0.35799999999999998}]},{"osd":4,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.59699999999999998,"15min":0.621},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.073,"5min":2.488,"15min":2.488},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.59599999999999997,"15min":0.60799999999999998},"min":{"1min":0.41299999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.75800000000000001,"5min":2.073,"15min":2.073},"last":0.41799999999999998}]},{"osd":5,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63900000000000001,"15min":0.66100000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.754,"5min":2.2069999999999999,"15min":2.2069999999999999},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.65200000000000002,"15min":0.66600000000000004},"min":{"1min":0.45900000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0369999999999999,"5min":2.456,"15min":2.456},"last":0.47499999999999998}]},{"osd":6,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.625},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.05,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":0.504},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60599999999999998,"15min":0.61299999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0940000000000001,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.439}]},{"osd":7,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63,"15min":0.63900000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0229999999999999,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.61899999999999999,"15min":0.63},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0569999999999999,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":0.48299999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574899,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":448,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194864,"statfs":{"total":200000000,"available":199541185,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.34000000000000002,"15min":0.371},"min":{"1min":0.218,"5min":0.193,"15min":0.193},"max":{"1min":0.52700000000000002,"5min":1.46,"15min":1.46},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.33400000000000002,"15min":0.34100000000000003},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":0.496,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.34599999999999997}]},{"osd":1,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.58999999999999997,"15min":0.83699999999999997},"min":{"1min":0.25,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.57599999999999996,"5min":12.351000000000001,"15min":12.351000000000001},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.28399999999999997,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.58899999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.438}]},{"osd":2,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.40100000000000002,"15min":0.42099999999999999},"min":{"1min":0.26300000000000001,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.70499999999999996,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.26800000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.63500000000000001,"5min":0.753,"15min":0.753},"last":0.38100000000000001}]},{"osd":4,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.80000000000000004,"15min":1.018},"min":{"1min":0.38400000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.876,"5min":11.333,"15min":11.333},"last":1.024},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.82899999999999996,"15min":1.0509999999999999},"min":{"1min":0.41299999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.94899999999999995,"5min":12.321999999999999,"15min":12.321999999999999},"last":0.73399999999999999}]},{"osd":5,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.82499999999999996,"15min":1.0620000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.92000000000000004,"5min":12.291,"15min":12.291},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.60099999999999998,"15min":0.59799999999999998},"min":{"1min":0.46300000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.99399999999999999}]},{"osd":6,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.77700000000000002,"15min":0.98299999999999998},"min":{"1min":0.42299999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.96199999999999997,"5min":11.288,"15min":11.288},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.622,"5min":0.81299999999999994,"15min":1.044},"min":{"1min":0.433,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.89400000000000002,"5min":12.253,"15min":12.253},"last":0.91200000000000003}]},{"osd":7,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.58899999999999997,"15min":0.57499999999999996},"min":{"1min":0.40400000000000003,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.97299999999999998,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.81200000000000006,"15min":1.02},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.92400000000000004,"5min":11.363,"15min":11.363},"last":0.67100000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574899,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":448,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194863,"statfs":{"total":200000000,"available":199540645,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56000000000000005,"15min":0.55400000000000005},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.74199999999999999,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.441},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.58199999999999996,"15min":0.58599999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.63600000000000001}]},{"osd":1,"last update":"Sun Oct 8 00:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.38800000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.52600000000000002,"15min":0.53200000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.73699999999999999,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.53300000000000003}]},{"osd":2,"last update":"Sun Oct 8 00:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51100000000000001,"15min":0.51800000000000002},"min":{"1min":0.33900000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.63400000000000001,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.51600000000000001,"15min":0.51000000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.48099999999999998}]},{"osd":3,"last update":"Sun Oct 8 00:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53400000000000003,"15min":0.53900000000000003},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.71499999999999997,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53100000000000003,"15min":0.53300000000000003},"min":{"1min":0.40200000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.64500000000000002}]},{"osd":5,"last update":"Sun Oct 8 00:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.48499999999999999,"15min":0.5},"min":{"1min":0.35899999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69599999999999995,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.498,"15min":0.51100000000000001},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.71399999999999997,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.375}]},{"osd":6,"last update":"Sun Oct 8 00:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.48499999999999999,"15min":0.49299999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.67600000000000005,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54400000000000004,"15min":0.55900000000000005},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.72199999999999998,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.39800000000000002}]},{"osd":7,"last update":"Sun Oct 8 00:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.53900000000000003,"15min":0.55300000000000005},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.73399999999999999,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57799999999999996,"15min":0.58299999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.78000000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.57899999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574899,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1024,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194287,"statfs":{"total":200000000,"available":198950853,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.48399999999999999,"15min":0.48699999999999999},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.65600000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55400000000000005,"15min":0.56200000000000006},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.70999999999999996,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.48999999999999999}]},{"osd":1,"last update":"Sun Oct 8 00:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.51600000000000001,"15min":0.52200000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.68200000000000005,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51600000000000001,"15min":0.52600000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.747,"5min":0.755,"15min":0.755},"last":0.441}]},{"osd":2,"last update":"Sun Oct 8 00:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52700000000000002,"15min":0.53000000000000003},"min":{"1min":0.34399999999999997,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.71799999999999997,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.56100000000000005,"15min":0.57099999999999995},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.79500000000000004,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.58199999999999996}]},{"osd":3,"last update":"Sun Oct 8 00:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.54900000000000004,"15min":0.54700000000000004},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.57199999999999995,"15min":0.58099999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.71999999999999997,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.55700000000000005}]},{"osd":4,"last update":"Sun Oct 8 00:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.46500000000000002,"15min":0.47199999999999998},"min":{"1min":0.30399999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.57499999999999996,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.51300000000000001},"min":{"1min":0.35699999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.746,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.52500000000000002}]},{"osd":6,"last update":"Sun Oct 8 00:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.496,"15min":0.498},"min":{"1min":0.34399999999999997,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.70599999999999996,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48999999999999999,"15min":0.49099999999999999},"min":{"1min":0.38400000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.61399999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.503}]},{"osd":7,"last update":"Sun Oct 8 00:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.33300000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.68400000000000005,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.55000000000000004,"15min":0.55500000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.72999999999999998,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.60399999999999998}]}]}],"pool_statfs":[]}} 2023-10-08T00:14:48.733 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T00:14:48.734 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-08T00:14:48.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.886+0000 7f61a9ff2700 1 Processor -- start 2023-10-08T00:14:48.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.886+0000 7f61a9ff2700 1 -- start start 2023-10-08T00:14:48.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.887+0000 7f61a9ff2700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f61a40b3720 0x7f61a40b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:48.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.887+0000 7f61a9ff2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f61a415c3e0 0x7f61a40ce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:48.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.887+0000 7f61a9ff2700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61a415bf50 con 0x7f61a40cee00 2023-10-08T00:14:48.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.887+0000 7f61a9ff2700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61a4158f00 con 0x7f61a40b3720 2023-10-08T00:14:48.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.887+0000 7f61a9ff2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61a415b900 con 0x7f61a415c3e0 2023-10-08T00:14:48.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.887+0000 7f61a8ff0700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f61a40b3720 0x7f61a40b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:48.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.887+0000 7f61a97f1700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f61a40cee00 0x7f61a40b17f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:34752/0 (socket says 172.21.15.99:34752) 2023-10-08T00:14:48.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.887+0000 7f61a97f1700 1 -- 172.21.15.99:0/1395571839 learned_addr learned my addr 172.21.15.99:0/1395571839 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:14:48.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.887+0000 7f61a3fff700 1 --2- 172.21.15.99:0/1395571839 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f61a415c3e0 0x7f61a40ce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:48.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.887+0000 7f61a8ff0700 1 -- 172.21.15.99:0/1395571839 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f61a415c3e0 msgr2=0x7f61a40ce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:48.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.887+0000 7f61a8ff0700 1 --2- 172.21.15.99:0/1395571839 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f61a415c3e0 0x7f61a40ce8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:48.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.887+0000 7f61a8ff0700 1 -- 172.21.15.99:0/1395571839 >> v1:172.21.15.99:6789/0 conn(0x7f61a40cee00 legacy=0x7f61a40b17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:48.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.887+0000 7f61a37fe700 1 -- 172.21.15.99:0/1395571839 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1326400565 0 0) 0x7f61a415bf50 con 0x7f61a40cee00 2023-10-08T00:14:48.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.887+0000 7f61a8ff0700 1 -- 172.21.15.99:0/1395571839 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a415bbe0 con 0x7f61a40b3720 2023-10-08T00:14:48.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.887+0000 7f61a8ff0700 1 --2- 172.21.15.99:0/1395571839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61a40b3720 0x7f61a40b3af0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f619c009e60 tx=0x7f619c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ddb23b6249fcb11 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:48.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.887+0000 7f61a37fe700 1 -- 172.21.15.99:0/1395571839 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619c00e070 con 0x7f61a40b3720 2023-10-08T00:14:48.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.888+0000 7f61a37fe700 1 -- 172.21.15.99:0/1395571839 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f619c00bc30 con 0x7f61a40b3720 2023-10-08T00:14:48.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.888+0000 7f61a37fe700 1 -- 172.21.15.99:0/1395571839 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619c012430 con 0x7f61a40b3720 2023-10-08T00:14:48.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.888+0000 7f61a9ff2700 1 -- 172.21.15.99:0/1395571839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61a40b3720 msgr2=0x7f61a40b3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:48.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.888+0000 7f61a9ff2700 1 --2- 172.21.15.99:0/1395571839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61a40b3720 0x7f61a40b3af0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f619c009e60 tx=0x7f619c009780 comp rx=0 tx=0).stop 2023-10-08T00:14:48.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.888+0000 7f61a9ff2700 1 -- 172.21.15.99:0/1395571839 shutdown_connections 2023-10-08T00:14:48.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.888+0000 7f61a9ff2700 1 --2- 172.21.15.99:0/1395571839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61a40b3720 0x7f61a40b3af0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:48.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.888+0000 7f61a9ff2700 1 --2- 172.21.15.99:0/1395571839 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f61a415c3e0 0x7f61a40ce8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:48.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.888+0000 7f61a9ff2700 1 -- 172.21.15.99:0/1395571839 >> 172.21.15.99:0/1395571839 conn(0x7f61a406c6f0 msgr2=0x7f61a4069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:48.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.888+0000 7f61a9ff2700 1 -- 172.21.15.99:0/1395571839 shutdown_connections 2023-10-08T00:14:48.893 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.888+0000 7f61a9ff2700 1 -- 172.21.15.99:0/1395571839 wait complete. 2023-10-08T00:14:48.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.889+0000 7f61a9ff2700 1 Processor -- start 2023-10-08T00:14:48.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.889+0000 7f61a9ff2700 1 -- start start 2023-10-08T00:14:48.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.890+0000 7f61a9ff2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f61a40b3720 0x7f61a4139390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:48.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.890+0000 7f61a9ff2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f61a40cee00 0x7f61a414e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:48.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.890+0000 7f61a9ff2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61a415c3e0 0x7f61a414ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:48.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.890+0000 7f61a9ff2700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a414e0e0 con 0x7f61a415c3e0 2023-10-08T00:14:48.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.890+0000 7f61a9ff2700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a413b070 con 0x7f61a40cee00 2023-10-08T00:14:48.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.890+0000 7f61a9ff2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61a413b350 con 0x7f61a40b3720 2023-10-08T00:14:48.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.890+0000 7f61a97f1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61a415c3e0 0x7f61a414ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:48.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.890+0000 7f61a97f1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61a415c3e0 0x7f61a414ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45766/0 (socket says 172.21.15.99:45766) 2023-10-08T00:14:48.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.890+0000 7f61a97f1700 1 -- 172.21.15.99:0/1690271657 learned_addr learned my addr 172.21.15.99:0/1690271657 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:48.898 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.890+0000 7f61a8ff0700 1 --2- 172.21.15.99:0/1690271657 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f61a40b3720 0x7f61a4139390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:48.898 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.890+0000 7f61a3fff700 1 --2- 172.21.15.99:0/1690271657 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f61a40cee00 0x7f61a414e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:48.898 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.890+0000 7f61a97f1700 1 -- 172.21.15.99:0/1690271657 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f61a40b3720 msgr2=0x7f61a4139390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:48.898 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.890+0000 7f61a97f1700 1 --2- 172.21.15.99:0/1690271657 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f61a40b3720 0x7f61a4139390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:48.898 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.890+0000 7f61a97f1700 1 -- 172.21.15.99:0/1690271657 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f61a40cee00 msgr2=0x7f61a414e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:48.898 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.890+0000 7f61a97f1700 1 --2- 172.21.15.99:0/1690271657 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f61a40cee00 0x7f61a414e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:48.898 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.890+0000 7f61a97f1700 1 -- 172.21.15.99:0/1690271657 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f619c009a20 con 0x7f61a415c3e0 2023-10-08T00:14:48.898 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.891+0000 7f61a97f1700 1 --2- 172.21.15.99:0/1690271657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61a415c3e0 0x7f61a414ea90 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f619400c7e0 tx=0x7f6194007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:48.898 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.891+0000 7f61a0ff9700 1 -- 172.21.15.99:0/1690271657 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6194005c00 con 0x7f61a415c3e0 2023-10-08T00:14:48.898 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.891+0000 7f61a0ff9700 1 -- 172.21.15.99:0/1690271657 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6194005d80 con 0x7f61a415c3e0 2023-10-08T00:14:48.898 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.891+0000 7f61a9ff2700 1 -- 172.21.15.99:0/1690271657 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61a413b630 con 0x7f61a415c3e0 2023-10-08T00:14:48.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.891+0000 7f61a0ff9700 1 -- 172.21.15.99:0/1690271657 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6194007320 con 0x7f61a415c3e0 2023-10-08T00:14:48.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.891+0000 7f61a9ff2700 1 -- 172.21.15.99:0/1690271657 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61a41398d0 con 0x7f61a415c3e0 2023-10-08T00:14:48.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.892+0000 7f61a9ff2700 1 -- 172.21.15.99:0/1690271657 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61a414e220 con 0x7f61a415c3e0 2023-10-08T00:14:48.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.892+0000 7f61a0ff9700 1 -- 172.21.15.99:0/1690271657 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6194007690 con 0x7f61a415c3e0 2023-10-08T00:14:48.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.893+0000 7f61a0ff9700 1 --2- 172.21.15.99:0/1690271657 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6190066f30 0x7f61900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:48.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.893+0000 7f61a0ff9700 1 -- 172.21.15.99:0/1690271657 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6194083950 con 0x7f61a415c3e0 2023-10-08T00:14:48.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.893+0000 7f61a8ff0700 1 --2- 172.21.15.99:0/1690271657 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6190066f30 0x7f61900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:48.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.893+0000 7f61a8ff0700 1 --2- 172.21.15.99:0/1690271657 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6190066f30 0x7f61900693e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f619c009e60 tx=0x7f619c019000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:48.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:48.897+0000 7f61a0ff9700 1 -- 172.21.15.99:0/1690271657 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6194050bc0 con 0x7f61a415c3e0 2023-10-08T00:14:49.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.151+0000 7f61a9ff2700 1 -- 172.21.15.99:0/1690271657 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f61a40cf280 con 0x7f61a415c3e0 2023-10-08T00:14:49.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.154+0000 7f61a0ff9700 1 -- 172.21.15.99:0/1690271657 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+0 (secure 0 0 0) 0x7f61940170e0 con 0x7f61a415c3e0 2023-10-08T00:14:49.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.156+0000 7f61a9ff2700 1 -- 172.21.15.99:0/1690271657 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6190066f30 msgr2=0x7f61900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:49.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.156+0000 7f61a9ff2700 1 --2- 172.21.15.99:0/1690271657 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6190066f30 0x7f61900693e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f619c009e60 tx=0x7f619c019000 comp rx=0 tx=0).stop 2023-10-08T00:14:49.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.156+0000 7f61a9ff2700 1 -- 172.21.15.99:0/1690271657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61a415c3e0 msgr2=0x7f61a414ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:49.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.156+0000 7f61a9ff2700 1 --2- 172.21.15.99:0/1690271657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61a415c3e0 0x7f61a414ea90 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f619400c7e0 tx=0x7f6194007d90 comp rx=0 tx=0).stop 2023-10-08T00:14:49.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.156+0000 7f61a9ff2700 1 -- 172.21.15.99:0/1690271657 shutdown_connections 2023-10-08T00:14:49.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.156+0000 7f61a9ff2700 1 --2- 172.21.15.99:0/1690271657 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6190066f30 0x7f61900693e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:49.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.156+0000 7f61a9ff2700 1 --2- 172.21.15.99:0/1690271657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61a415c3e0 0x7f61a414ea90 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:49.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.156+0000 7f61a9ff2700 1 --2- 172.21.15.99:0/1690271657 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f61a40b3720 0x7f61a4139390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:49.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.156+0000 7f61a9ff2700 1 --2- 172.21.15.99:0/1690271657 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f61a40cee00 0x7f61a414e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:49.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.156+0000 7f61a9ff2700 1 -- 172.21.15.99:0/1690271657 >> 172.21.15.99:0/1690271657 conn(0x7f61a406c6f0 msgr2=0x7f61a4153ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:49.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.157+0000 7f61a9ff2700 1 -- 172.21.15.99:0/1690271657 shutdown_connections 2023-10-08T00:14:49.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.157+0000 7f61a9ff2700 1 -- 172.21.15.99:0/1690271657 wait complete. 2023-10-08T00:14:49.159 INFO:teuthology.orchestra.run.smithi099.stderr:updated caps for client.0 2023-10-08T00:14:49.169 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:14:49.281 INFO:tasks.ceph.mds.a.smithi099.stderr:2023-10-08T00:14:49.280+0000 7f380635b700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T00:14:49.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.282+0000 7f974ed97700 1 Processor -- start 2023-10-08T00:14:49.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.282+0000 7f974ed97700 1 -- start start 2023-10-08T00:14:49.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.283+0000 7f974ed97700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f974815a620 0x7f974815d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:49.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.283+0000 7f974ed97700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f974815d580 0x7f9748158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:49.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.283+0000 7f974ed97700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f974815cbb0 con 0x7f9748157870 2023-10-08T00:14:49.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.283+0000 7f974ed97700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f974815bdb0 con 0x7f974815a620 2023-10-08T00:14:49.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.283+0000 7f974ed97700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f974815bfa0 con 0x7f974815d580 2023-10-08T00:14:49.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.283+0000 7f973ffff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f974815a620 0x7f974815d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:49.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.283+0000 7f973ffff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f974815a620 0x7f974815d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45776/0 (socket says 172.21.15.99:45776) 2023-10-08T00:14:49.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.283+0000 7f973ffff700 1 -- 172.21.15.99:0/1211235846 learned_addr learned my addr 172.21.15.99:0/1211235846 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:49.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.283+0000 7f974d334700 1 --2- 172.21.15.99:0/1211235846 >> v2:172.21.15.191:3300/0 conn(0x7f974815d580 0x7f9748158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:49.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.284+0000 7f973ffff700 1 -- 172.21.15.99:0/1211235846 >> v2:172.21.15.191:3300/0 conn(0x7f974815d580 msgr2=0x7f9748158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:49.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.284+0000 7f973ffff700 1 --2- 172.21.15.99:0/1211235846 >> v2:172.21.15.191:3300/0 conn(0x7f974815d580 0x7f9748158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:49.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.284+0000 7f973ffff700 1 -- 172.21.15.99:0/1211235846 >> v1:172.21.15.191:6789/0 conn(0x7f9748157870 legacy=0x7f974815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:49.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.284+0000 7f973ffff700 1 -- 172.21.15.99:0/1211235846 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f974815c280 con 0x7f974815a620 2023-10-08T00:14:49.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.284+0000 7f973ffff700 1 --2- 172.21.15.99:0/1211235846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f974815a620 0x7f974815d040 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f974806b4e0 tx=0x7f9738009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ea33165bd8b203 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:49.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.284+0000 7f973f7fe700 1 -- 172.21.15.99:0/1211235846 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9738009e70 con 0x7f974815a620 2023-10-08T00:14:49.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.284+0000 7f973f7fe700 1 -- 172.21.15.99:0/1211235846 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f973800b860 con 0x7f974815a620 2023-10-08T00:14:49.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.284+0000 7f973f7fe700 1 -- 172.21.15.99:0/1211235846 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f973800bc00 con 0x7f974815a620 2023-10-08T00:14:49.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.284+0000 7f974ed97700 1 -- 172.21.15.99:0/1211235846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f974815a620 msgr2=0x7f974815d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:49.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.284+0000 7f974ed97700 1 --2- 172.21.15.99:0/1211235846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f974815a620 0x7f974815d040 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f974806b4e0 tx=0x7f9738009250 comp rx=0 tx=0).stop 2023-10-08T00:14:49.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.284+0000 7f974ed97700 1 -- 172.21.15.99:0/1211235846 shutdown_connections 2023-10-08T00:14:49.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.284+0000 7f974ed97700 1 --2- 172.21.15.99:0/1211235846 >> v2:172.21.15.191:3300/0 conn(0x7f974815d580 0x7f9748158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:49.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.284+0000 7f974ed97700 1 --2- 172.21.15.99:0/1211235846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f974815a620 0x7f974815d040 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:49.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.284+0000 7f974ed97700 1 -- 172.21.15.99:0/1211235846 >> 172.21.15.99:0/1211235846 conn(0x7f9748068ce0 msgr2=0x7f97480690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:49.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.284+0000 7f974ed97700 1 -- 172.21.15.99:0/1211235846 shutdown_connections 2023-10-08T00:14:49.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.284+0000 7f974ed97700 1 -- 172.21.15.99:0/1211235846 wait complete. 2023-10-08T00:14:49.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.285+0000 7f974ed97700 1 Processor -- start 2023-10-08T00:14:49.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.286+0000 7f974ed97700 1 -- start start 2023-10-08T00:14:49.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.286+0000 7f974ed97700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9748157870 0x7f974814c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:49.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.286+0000 7f974ed97700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f974815a620 0x7f974814eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:49.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.286+0000 7f974ed97700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f974815d580 0x7f97481530a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:49.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.286+0000 7f974ed97700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974814f5c0 con 0x7f974815a620 2023-10-08T00:14:49.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.286+0000 7f974ed97700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974814f8a0 con 0x7f974815d580 2023-10-08T00:14:49.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.286+0000 7f974ed97700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f974814fb80 con 0x7f9748157870 2023-10-08T00:14:49.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.286+0000 7f973ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f974815a620 0x7f974814eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:49.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.287+0000 7f973ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f974815a620 0x7f974814eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45792/0 (socket says 172.21.15.99:45792) 2023-10-08T00:14:49.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.287+0000 7f973ffff700 1 -- 172.21.15.99:0/3015470097 learned_addr learned my addr 172.21.15.99:0/3015470097 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:49.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.287+0000 7f974d334700 1 --2- 172.21.15.99:0/3015470097 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f974815d580 0x7f97481530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:49.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.287+0000 7f974cb33700 1 --2- 172.21.15.99:0/3015470097 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9748157870 0x7f974814c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:49.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.287+0000 7f973ffff700 1 -- 172.21.15.99:0/3015470097 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9748157870 msgr2=0x7f974814c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:49.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.287+0000 7f973ffff700 1 --2- 172.21.15.99:0/3015470097 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9748157870 0x7f974814c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:49.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.287+0000 7f973ffff700 1 -- 172.21.15.99:0/3015470097 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f974815d580 msgr2=0x7f97481530a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:49.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.287+0000 7f973ffff700 1 --2- 172.21.15.99:0/3015470097 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f974815d580 0x7f97481530a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:49.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.287+0000 7f973ffff700 1 -- 172.21.15.99:0/3015470097 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97380094f0 con 0x7f974815a620 2023-10-08T00:14:49.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.287+0000 7f973ffff700 1 --2- 172.21.15.99:0/3015470097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f974815a620 0x7f974814eb00 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f974806b530 tx=0x7f9738017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:49.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.287+0000 7f973cff9700 1 -- 172.21.15.99:0/3015470097 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f973800bbe0 con 0x7f974815a620 2023-10-08T00:14:49.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.287+0000 7f973cff9700 1 -- 172.21.15.99:0/3015470097 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9738011420 con 0x7f974815a620 2023-10-08T00:14:49.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.287+0000 7f973cff9700 1 -- 172.21.15.99:0/3015470097 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97380117c0 con 0x7f974815a620 2023-10-08T00:14:49.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.287+0000 7f974ed97700 1 -- 172.21.15.99:0/3015470097 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f974814fe60 con 0x7f974815a620 2023-10-08T00:14:49.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.287+0000 7f974ed97700 1 -- 172.21.15.99:0/3015470097 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f974814ce00 con 0x7f974815a620 2023-10-08T00:14:49.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.288+0000 7f974ed97700 1 -- 172.21.15.99:0/3015470097 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9748063230 con 0x7f974815a620 2023-10-08T00:14:49.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.288+0000 7f973cff9700 1 -- 172.21.15.99:0/3015470097 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9738011bb0 con 0x7f974815a620 2023-10-08T00:14:49.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.289+0000 7f973cff9700 1 --2- 172.21.15.99:0/3015470097 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9734066f30 0x7f97340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:49.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.289+0000 7f973cff9700 1 -- 172.21.15.99:0/3015470097 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9738085ba0 con 0x7f974815a620 2023-10-08T00:14:49.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.289+0000 7f974cb33700 1 --2- 172.21.15.99:0/3015470097 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9734066f30 0x7f97340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:49.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.289+0000 7f974cb33700 1 --2- 172.21.15.99:0/3015470097 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9734066f30 0x7f97340693e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9744007fd0 tx=0x7f9744009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:49.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.293+0000 7f973cff9700 1 -- 172.21.15.99:0/3015470097 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9738052e10 con 0x7f974815a620 2023-10-08T00:14:49.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.519+0000 7f974ed97700 1 -- 172.21.15.99:0/3015470097 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f974804c3c0 con 0x7f974815a620 2023-10-08T00:14:49.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.520+0000 7f973cff9700 1 -- 172.21.15.99:0/3015470097 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4599 (secure 0 0 0) 0x7f9738015090 con 0x7f974815a620 2023-10-08T00:14:49.520 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:14:49.520 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5650,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/2166055423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2166055423},{"type":"v1","addr":"172.21.15.99:6834","nonce":2166055423}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5665,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/4087832300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":4087832300},{"type":"v1","addr":"172.21.15.191:6836","nonce":4087832300}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5668,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/730261712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":730261712},{"type":"v1","addr":"172.21.15.191:6837","nonce":730261712}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:14:43.671064+0000","modified":"2023-10-08T00:14:47.326922+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6039},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6039":{"gid":6039,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/104443412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":104443412},{"type":"v1","addr":"172.21.15.99:6835","nonce":104443412}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-08T00:14:49.522 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.521+0000 7f974ed97700 1 -- 172.21.15.99:0/3015470097 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9734066f30 msgr2=0x7f97340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:49.522 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.521+0000 7f974ed97700 1 --2- 172.21.15.99:0/3015470097 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9734066f30 0x7f97340693e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9744007fd0 tx=0x7f9744009040 comp rx=0 tx=0).stop 2023-10-08T00:14:49.522 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.521+0000 7f974ed97700 1 -- 172.21.15.99:0/3015470097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f974815a620 msgr2=0x7f974814eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:49.523 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.521+0000 7f974ed97700 1 --2- 172.21.15.99:0/3015470097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f974815a620 0x7f974814eb00 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f974806b530 tx=0x7f9738017000 comp rx=0 tx=0).stop 2023-10-08T00:14:49.523 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.521+0000 7f974ed97700 1 -- 172.21.15.99:0/3015470097 shutdown_connections 2023-10-08T00:14:49.523 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.521+0000 7f974ed97700 1 --2- 172.21.15.99:0/3015470097 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9734066f30 0x7f97340693e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:49.523 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.521+0000 7f974ed97700 1 --2- 172.21.15.99:0/3015470097 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f974815d580 0x7f97481530a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:49.523 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.521+0000 7f974ed97700 1 --2- 172.21.15.99:0/3015470097 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9748157870 0x7f974814c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:49.523 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.521+0000 7f974ed97700 1 --2- 172.21.15.99:0/3015470097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f974815a620 0x7f974814eb00 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:49.523 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.521+0000 7f974ed97700 1 -- 172.21.15.99:0/3015470097 >> 172.21.15.99:0/3015470097 conn(0x7f9748068ce0 msgr2=0x7f9748159960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:49.523 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.522+0000 7f974ed97700 1 -- 172.21.15.99:0/3015470097 shutdown_connections 2023-10-08T00:14:49.523 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:49.522+0000 7f974ed97700 1 -- 172.21.15.99:0/3015470097 wait complete. 2023-10-08T00:14:49.523 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 40 2023-10-08T00:14:49.937 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 40, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T00:14:43.671064+0000', 'modified': '2023-10-08T00:14:47.326922+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6039}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6039': {'gid': 6039, 'name': 'a', 'rank': 0, 'incarnation': 36, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.99:6835/104443412', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6833', 'nonce': 104443412}, {'type': 'v1', 'addr': '172.21.15.99:6835', 'nonce': 104443412}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-08T00:14:49.937 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T00:14:49.937 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-08T00:14:50.053 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.052+0000 7fc6cc0d2700 1 Processor -- start 2023-10-08T00:14:50.053 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.052+0000 7fc6cc0d2700 1 -- start start 2023-10-08T00:14:50.055 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.053+0000 7fc6cc0d2700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc6c40cf1c0 0x7fc6c415e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:50.056 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.053+0000 7fc6cc0d2700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6c40cca20 con 0x7fc6c4159cf0 2023-10-08T00:14:50.056 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.053+0000 7fc6cc0d2700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6c415ea40 con 0x7fc6c415a0c0 2023-10-08T00:14:50.056 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.053+0000 7fc6cc0d2700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6c4067660 con 0x7fc6c40cf1c0 2023-10-08T00:14:50.056 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.053+0000 7fc6c966d700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc6c40cf1c0 0x7fc6c415e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:50.056 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.053+0000 7fc6c9e6e700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fc6c4159cf0 0x7fc6c40ceab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:34762/0 (socket says 172.21.15.99:34762) 2023-10-08T00:14:50.056 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.053+0000 7fc6c9e6e700 1 -- 172.21.15.99:0/3151913509 learned_addr learned my addr 172.21.15.99:0/3151913509 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:14:50.056 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.054+0000 7fc6c966d700 1 -- 172.21.15.99:0/3151913509 >> v1:172.21.15.191:6789/0 conn(0x7fc6c415a0c0 legacy=0x7fc6c4068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:50.056 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.054+0000 7fc6c966d700 1 -- 172.21.15.99:0/3151913509 >> v1:172.21.15.99:6789/0 conn(0x7fc6c4159cf0 legacy=0x7fc6c40ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:50.056 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.054+0000 7fc6c966d700 1 -- 172.21.15.99:0/3151913509 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c4067f00 con 0x7fc6c40cf1c0 2023-10-08T00:14:50.056 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.054+0000 7fc6c8e6c700 1 -- 172.21.15.99:0/3151913509 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 861876898 0 0) 0x7fc6c40cca20 con 0x7fc6c4159cf0 2023-10-08T00:14:50.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.054+0000 7fc6c966d700 1 --2- 172.21.15.99:0/3151913509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6c40cf1c0 0x7fc6c415e500 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fc6b40098e0 tx=0x7fc6b40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e548f7369d9e23d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:50.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.054+0000 7fc6c8e6c700 1 -- 172.21.15.99:0/3151913509 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b4009e80 con 0x7fc6c40cf1c0 2023-10-08T00:14:50.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.054+0000 7fc6c8e6c700 1 -- 172.21.15.99:0/3151913509 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b400b860 con 0x7fc6c40cf1c0 2023-10-08T00:14:50.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.054+0000 7fc6c8e6c700 1 -- 172.21.15.99:0/3151913509 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b400bbe0 con 0x7fc6c40cf1c0 2023-10-08T00:14:50.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.055+0000 7fc6cc0d2700 1 -- 172.21.15.99:0/3151913509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6c40cf1c0 msgr2=0x7fc6c415e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:50.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.055+0000 7fc6cc0d2700 1 --2- 172.21.15.99:0/3151913509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6c40cf1c0 0x7fc6c415e500 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fc6b40098e0 tx=0x7fc6b40091a0 comp rx=0 tx=0).stop 2023-10-08T00:14:50.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.055+0000 7fc6cc0d2700 1 -- 172.21.15.99:0/3151913509 shutdown_connections 2023-10-08T00:14:50.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.055+0000 7fc6cc0d2700 1 --2- 172.21.15.99:0/3151913509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6c40cf1c0 0x7fc6c415e500 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:50.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.055+0000 7fc6cc0d2700 1 -- 172.21.15.99:0/3151913509 >> 172.21.15.99:0/3151913509 conn(0x7fc6c406c6e0 msgr2=0x7fc6c4069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:50.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.055+0000 7fc6cc0d2700 1 -- 172.21.15.99:0/3151913509 shutdown_connections 2023-10-08T00:14:50.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.055+0000 7fc6cc0d2700 1 -- 172.21.15.99:0/3151913509 wait complete. 2023-10-08T00:14:50.060 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.056+0000 7fc6cc0d2700 1 Processor -- start 2023-10-08T00:14:50.060 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.056+0000 7fc6cc0d2700 1 -- start start 2023-10-08T00:14:50.060 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.056+0000 7fc6cc0d2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc6c40cf1c0 0x7fc6c40abfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:50.060 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.056+0000 7fc6cc0d2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6c4159cf0 0x7fc6c4158a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:50.060 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.056+0000 7fc6cc0d2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc6c415a0c0 0x7fc6c4155420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:50.060 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.056+0000 7fc6cc0d2700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c40b2580 con 0x7fc6c4159cf0 2023-10-08T00:14:50.060 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.056+0000 7fc6cc0d2700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c4158fd0 con 0x7fc6c415a0c0 2023-10-08T00:14:50.060 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.056+0000 7fc6cc0d2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c40ac600 con 0x7fc6c40cf1c0 2023-10-08T00:14:50.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.057+0000 7fc6c966d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6c4159cf0 0x7fc6c4158a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:50.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.057+0000 7fc6c966d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6c4159cf0 0x7fc6c4158a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45808/0 (socket says 172.21.15.99:45808) 2023-10-08T00:14:50.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.057+0000 7fc6c966d700 1 -- 172.21.15.99:0/3982870249 learned_addr learned my addr 172.21.15.99:0/3982870249 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:50.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.057+0000 7fc6c9e6e700 1 --2- 172.21.15.99:0/3982870249 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc6c40cf1c0 0x7fc6c40abfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:50.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.057+0000 7fc6ca66f700 1 --2- 172.21.15.99:0/3982870249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc6c415a0c0 0x7fc6c4155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:50.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.057+0000 7fc6c966d700 1 -- 172.21.15.99:0/3982870249 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc6c40cf1c0 msgr2=0x7fc6c40abfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:50.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.057+0000 7fc6c966d700 1 --2- 172.21.15.99:0/3982870249 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc6c40cf1c0 0x7fc6c40abfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:50.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.057+0000 7fc6c966d700 1 -- 172.21.15.99:0/3982870249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc6c415a0c0 msgr2=0x7fc6c4155420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:50.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.057+0000 7fc6c966d700 1 --2- 172.21.15.99:0/3982870249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc6c415a0c0 0x7fc6c4155420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:50.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.057+0000 7fc6c966d700 1 -- 172.21.15.99:0/3982870249 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b4009420 con 0x7fc6c4159cf0 2023-10-08T00:14:50.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.057+0000 7fc6ca66f700 1 --2- 172.21.15.99:0/3982870249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc6c415a0c0 0x7fc6c4155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:14:50.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.057+0000 7fc6c9e6e700 1 --2- 172.21.15.99:0/3982870249 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc6c40cf1c0 0x7fc6c40abfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:14:50.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.057+0000 7fc6c966d700 1 --2- 172.21.15.99:0/3982870249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6c4159cf0 0x7fc6c4158a90 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fc6c406de20 tx=0x7fc6b4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:50.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.057+0000 7fc6b9ffb700 1 -- 172.21.15.99:0/3982870249 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b401f070 con 0x7fc6c4159cf0 2023-10-08T00:14:50.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.057+0000 7fc6b9ffb700 1 -- 172.21.15.99:0/3982870249 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b400b9d0 con 0x7fc6c4159cf0 2023-10-08T00:14:50.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.057+0000 7fc6cc0d2700 1 -- 172.21.15.99:0/3982870249 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6c40ac8e0 con 0x7fc6c4159cf0 2023-10-08T00:14:50.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.057+0000 7fc6b9ffb700 1 -- 172.21.15.99:0/3982870249 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b4025670 con 0x7fc6c4159cf0 2023-10-08T00:14:50.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.057+0000 7fc6cc0d2700 1 -- 172.21.15.99:0/3982870249 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6c40acea0 con 0x7fc6c4159cf0 2023-10-08T00:14:50.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.059+0000 7fc6b9ffb700 1 -- 172.21.15.99:0/3982870249 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc6b4023070 con 0x7fc6c4159cf0 2023-10-08T00:14:50.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.059+0000 7fc6b9ffb700 1 --2- 172.21.15.99:0/3982870249 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc6a8066f30 0x7fc6a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:50.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.059+0000 7fc6b9ffb700 1 -- 172.21.15.99:0/3982870249 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc6b4085bc0 con 0x7fc6c4159cf0 2023-10-08T00:14:50.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.059+0000 7fc6c9e6e700 1 --2- 172.21.15.99:0/3982870249 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc6a8066f30 0x7fc6a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:50.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.060+0000 7fc6c9e6e700 1 --2- 172.21.15.99:0/3982870249 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc6a8066f30 0x7fc6a80693e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0000f80 tx=0x7fc6c0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:50.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.061+0000 7fc6d53cab80 1 Processor -- start 2023-10-08T00:14:50.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.061+0000 7fc6d53cab80 1 -- start start 2023-10-08T00:14:50.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.062+0000 7fc6d53cab80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a8744a3a50 0x55a8744a3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:50.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.062+0000 7fc6d53cab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8744edd00 0x55a8744cca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:50.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.062+0000 7fc6d53cab80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8744ccf60 0x55a87449fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:50.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.062+0000 7fc6d53cab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8743faa70 con 0x55a8744edd00 2023-10-08T00:14:50.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.062+0000 7fc6d53cab80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8743f8270 con 0x55a8744ccf60 2023-10-08T00:14:50.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.062+0000 7fc6d53cab80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a8743f4060 con 0x55a8744a3a50 2023-10-08T00:14:50.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.062+0000 7fc6c966d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8744edd00 0x55a8744cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:50.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.062+0000 7fc6c966d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8744edd00 0x55a8744cca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45816/0 (socket says 172.21.15.99:45816) 2023-10-08T00:14:50.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.062+0000 7fc6c966d700 1 -- 172.21.15.99:0/960314057 learned_addr learned my addr 172.21.15.99:0/960314057 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:50.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.062+0000 7fc6c966d700 1 -- 172.21.15.99:0/960314057 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a8744a3a50 msgr2=0x55a8744a3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:50.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.062+0000 7fc6c9e6e700 1 --2- 172.21.15.99:0/960314057 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a8744a3a50 0x55a8744a3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:50.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.062+0000 7fc6ca66f700 1 --2- 172.21.15.99:0/960314057 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8744ccf60 0x55a87449fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:50.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.062+0000 7fc6c966d700 1 --2- 172.21.15.99:0/960314057 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a8744a3a50 0x55a8744a3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:50.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.062+0000 7fc6c966d700 1 -- 172.21.15.99:0/960314057 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8744ccf60 msgr2=0x55a87449fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:50.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.062+0000 7fc6c966d700 1 --2- 172.21.15.99:0/960314057 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8744ccf60 0x55a87449fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:50.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.062+0000 7fc6c966d700 1 -- 172.21.15.99:0/960314057 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a87428e270 con 0x55a8744edd00 2023-10-08T00:14:50.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.062+0000 7fc6c966d700 1 --2- 172.21.15.99:0/960314057 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8744edd00 0x55a8744cca20 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fc6b40840e0 tx=0x7fc6b4084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=334fe18f358459ea server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:50.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.062+0000 7fc6aeffd700 1 -- 172.21.15.99:0/960314057 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b4056af0 con 0x55a8744edd00 2023-10-08T00:14:50.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.062+0000 7fc6aeffd700 1 -- 172.21.15.99:0/960314057 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b4059d40 con 0x55a8744edd00 2023-10-08T00:14:50.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.062+0000 7fc6aeffd700 1 -- 172.21.15.99:0/960314057 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b405da90 con 0x55a8744edd00 2023-10-08T00:14:50.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.063+0000 7fc6d53cab80 1 -- 172.21.15.99:0/960314057 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8744edd00 msgr2=0x55a8744cca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:50.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.063+0000 7fc6d53cab80 1 --2- 172.21.15.99:0/960314057 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8744edd00 0x55a8744cca20 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fc6b40840e0 tx=0x7fc6b4084040 comp rx=0 tx=0).stop 2023-10-08T00:14:50.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.063+0000 7fc6d53cab80 1 -- 172.21.15.99:0/960314057 shutdown_connections 2023-10-08T00:14:50.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.063+0000 7fc6d53cab80 1 --2- 172.21.15.99:0/960314057 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8744ccf60 0x55a87449fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:50.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.063+0000 7fc6d53cab80 1 --2- 172.21.15.99:0/960314057 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a8744a3a50 0x55a8744a3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:50.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.063+0000 7fc6d53cab80 1 --2- 172.21.15.99:0/960314057 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8744edd00 0x55a8744cca20 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:50.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.063+0000 7fc6d53cab80 1 -- 172.21.15.99:0/960314057 >> 172.21.15.99:0/960314057 conn(0x55a8744a5370 msgr2=0x55a8744a5740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:50.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.063+0000 7fc6d53cab80 1 -- 172.21.15.99:0/960314057 shutdown_connections 2023-10-08T00:14:50.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.063+0000 7fc6d53cab80 1 -- 172.21.15.99:0/960314057 wait complete. 2023-10-08T00:14:50.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.063+0000 7fc6d53cab80 1 -- start start 2023-10-08T00:14:50.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.064+0000 7fc6d53cab80 1 Processor -- start 2023-10-08T00:14:50.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.064+0000 7fc6d53cab80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8744a3a50 0x55a8744eabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:50.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.064+0000 7fc6d53cab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8744ccf60 0x55a8744e58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:50.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.064+0000 7fc6d53cab80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a8744edd00 0x55a87449ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:50.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.064+0000 7fc6d53cab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a87439f890 con 0x55a8744ccf60 2023-10-08T00:14:50.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.064+0000 7fc6d53cab80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8743baea0 con 0x55a8744a3a50 2023-10-08T00:14:50.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.064+0000 7fc6d53cab80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a874276e30 con 0x55a8744edd00 2023-10-08T00:14:50.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.064+0000 7fc6d53cab80 10 client.0 _ll_register_callbacks cb 0x55a874467110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T00:14:50.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.065+0000 7fc6c966d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8744ccf60 0x55a8744e58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:50.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.065+0000 7fc6c966d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8744ccf60 0x55a8744e58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45824/0 (socket says 172.21.15.99:45824) 2023-10-08T00:14:50.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.065+0000 7fc6c966d700 1 -- 172.21.15.99:0/2011534751 learned_addr learned my addr 172.21.15.99:0/2011534751 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:50.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.065+0000 7fc6ca66f700 1 --2- 172.21.15.99:0/2011534751 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a8744edd00 0x55a87449ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:50.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.065+0000 7fc6c9e6e700 1 --2- 172.21.15.99:0/2011534751 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8744a3a50 0x55a8744eabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:50.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.065+0000 7fc6c966d700 1 -- 172.21.15.99:0/2011534751 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a8744edd00 msgr2=0x55a87449ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:50.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.065+0000 7fc6c966d700 1 --2- 172.21.15.99:0/2011534751 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a8744edd00 0x55a87449ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:50.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.065+0000 7fc6c966d700 1 -- 172.21.15.99:0/2011534751 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8744a3a50 msgr2=0x55a8744eabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:50.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.065+0000 7fc6c966d700 1 --2- 172.21.15.99:0/2011534751 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8744a3a50 0x55a8744eabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:50.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.065+0000 7fc6c966d700 1 -- 172.21.15.99:0/2011534751 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6c40acea0 con 0x55a8744ccf60 2023-10-08T00:14:50.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.065+0000 7fc6c9e6e700 1 --2- 172.21.15.99:0/2011534751 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8744a3a50 0x55a8744eabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:14:50.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.065+0000 7fc6ca66f700 1 --2- 172.21.15.99:0/2011534751 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a8744edd00 0x55a87449ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:14:50.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.065+0000 7fc6c966d700 1 --2- 172.21.15.99:0/2011534751 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8744ccf60 0x55a8744e58e0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fc6b406d080 tx=0x7fc6b406c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f1c7eaad30fe90f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:50.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.065+0000 7fc6a3fff700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T00:14:50.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.065+0000 7fc6a3fff700 1 -- 172.21.15.99:0/2011534751 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b405da90 con 0x55a8744ccf60 2023-10-08T00:14:50.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.065+0000 7fc6a3fff700 1 -- 172.21.15.99:0/2011534751 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc6b4082030 con 0x55a8744ccf60 2023-10-08T00:14:50.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.065+0000 7fc6d53cab80 1 -- 172.21.15.99:0/2011534751 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a874498b70 con 0x55a8744ccf60 2023-10-08T00:14:50.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.065+0000 7fc6a3fff700 1 -- 172.21.15.99:0/2011534751 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6b400bb40 con 0x55a8744ccf60 2023-10-08T00:14:50.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.065+0000 7fc6d53cab80 10 client.6195 fetch_fsmap learned FSMap version 40 2023-10-08T00:14:50.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.065+0000 7fc6d53cab80 1 -- 172.21.15.99:0/2011534751 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x55a874230fe0 con 0x55a8744ccf60 2023-10-08T00:14:50.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.065+0000 7fc6a3fff700 1 -- 172.21.15.99:0/2011534751 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a874498b70 con 0x55a8744ccf60 2023-10-08T00:14:50.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.065+0000 7fc6a3fff700 1 -- 172.21.15.99:0/2011534751 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 40) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fc6b4055fe0 con 0x55a8744ccf60 2023-10-08T00:14:50.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.065+0000 7fc6d53cab80 10 client.6195 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T00:14:50.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.065+0000 7fc6d53cab80 10 client.6195 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-08T00:14:50.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.065+0000 7fc6d53cab80 20 client.6195 populate_metadata read hostname 'smithi099' 2023-10-08T00:14:50.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.065+0000 7fc6d53cab80 1 --2- 172.21.15.99:0/2011534751 >> [v2:172.21.15.99:6833/104443412,v1:172.21.15.99:6835/104443412] conn(0x55a8744e69c0 0x55a8744e6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:50.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.065+0000 7fc6d53cab80 4 client.6195 mds_command: new command op to 6039 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-08T00:14:50.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.065+0000 7fc6d53cab80 1 -- 172.21.15.99:0/2011534751 --> [v2:172.21.15.99:6833/104443412,v1:172.21.15.99:6835/104443412] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a874475aa0 con 0x55a8744e69c0 2023-10-08T00:14:50.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.066+0000 7fc6c9e6e700 1 --2- 172.21.15.99:0/2011534751 >> [v2:172.21.15.99:6833/104443412,v1:172.21.15.99:6835/104443412] conn(0x55a8744e69c0 0x55a8744e6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:50.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.066+0000 7fc6a3fff700 10 client.6195 ms_handle_connect on v2:172.21.15.99:6833/104443412 2023-10-08T00:14:50.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.066+0000 7fc6c9e6e700 1 --2- 172.21.15.99:0/2011534751 >> [v2:172.21.15.99:6833/104443412,v1:172.21.15.99:6835/104443412] conn(0x55a8744e69c0 0x55a8744e6d90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=baf0bb50b973655c server_cookie=dbe567bc7468e409 in_seq=0 out_seq=0 2023-10-08T00:14:50.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.066+0000 7fc6a3fff700 1 -- 172.21.15.99:0/2011534751 <== mds.0 v2:172.21.15.99:6833/104443412 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55a874475aa0 con 0x55a8744e69c0 2023-10-08T00:14:50.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.066+0000 7fc6a3fff700 10 client.6195 handle_command_reply: tid=0 2023-10-08T00:14:50.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.066+0000 7fc6d53cab80 1 client.6195 shutdown 2023-10-08T00:14:50.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.066+0000 7fc6d53cab80 1 -- 172.21.15.99:0/2011534751 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8744ccf60 msgr2=0x55a8744e58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:50.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.066+0000 7fc6d53cab80 1 --2- 172.21.15.99:0/2011534751 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8744ccf60 0x55a8744e58e0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fc6b406d080 tx=0x7fc6b406c000 comp rx=0 tx=0).stop 2023-10-08T00:14:50.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.066+0000 7fc6d53cab80 1 -- 172.21.15.99:0/2011534751 shutdown_connections 2023-10-08T00:14:50.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.066+0000 7fc6d53cab80 1 --2- 172.21.15.99:0/2011534751 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8744a3a50 0x55a8744eabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:50.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.066+0000 7fc6d53cab80 1 --2- 172.21.15.99:0/2011534751 >> [v2:172.21.15.99:6833/104443412,v1:172.21.15.99:6835/104443412] conn(0x55a8744e69c0 0x55a8744e6d90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:50.077 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.066+0000 7fc6d53cab80 1 --2- 172.21.15.99:0/2011534751 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a8744edd00 0x55a87449ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:50.077 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.066+0000 7fc6d53cab80 1 --2- 172.21.15.99:0/2011534751 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8744ccf60 0x55a8744e58e0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:50.077 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.066+0000 7fc6a3fff700 0 client.6195 ms_handle_reset on v2:172.21.15.99:6833/104443412 2023-10-08T00:14:50.077 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.066+0000 7fc6d53cab80 1 -- 172.21.15.99:0/2011534751 >> 172.21.15.99:0/2011534751 conn(0x55a8744a5370 msgr2=0x55a8744a0780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:50.077 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.067+0000 7fc6d53cab80 1 -- 172.21.15.99:0/2011534751 shutdown_connections 2023-10-08T00:14:50.077 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.067+0000 7fc6d53cab80 1 -- 172.21.15.99:0/2011534751 wait complete. 2023-10-08T00:14:50.077 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.067+0000 7fc6d53cab80 20 client.6195 trim_cache size 0 max 16384 2023-10-08T00:14:50.084 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.083+0000 7fc6d53cab80 1 Processor -- start 2023-10-08T00:14:50.084 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.083+0000 7fc6d53cab80 1 -- start start 2023-10-08T00:14:50.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.084+0000 7fc6d53cab80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8744a3a50 0x55a8744e7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:50.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.084+0000 7fc6d53cab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8744ccf60 0x55a8744f9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:50.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.084+0000 7fc6d53cab80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a8744edd00 0x55a8744d3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:50.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.084+0000 7fc6d53cab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a874483510 con 0x55a8744ccf60 2023-10-08T00:14:50.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.084+0000 7fc6d53cab80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a87439f810 con 0x55a8744a3a50 2023-10-08T00:14:50.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.084+0000 7fc6d53cab80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a8744cc7e0 con 0x55a8744edd00 2023-10-08T00:14:50.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.084+0000 7fc6c966d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8744ccf60 0x55a8744f9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:50.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.084+0000 7fc6c966d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8744ccf60 0x55a8744f9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45826/0 (socket says 172.21.15.99:45826) 2023-10-08T00:14:50.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.084+0000 7fc6c966d700 1 -- 172.21.15.99:0/2367018761 learned_addr learned my addr 172.21.15.99:0/2367018761 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:50.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.084+0000 7fc6c9e6e700 1 --2- 172.21.15.99:0/2367018761 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8744a3a50 0x55a8744e7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:50.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.084+0000 7fc6ca66f700 1 --2- 172.21.15.99:0/2367018761 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a8744edd00 0x55a8744d3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:50.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.084+0000 7fc6c966d700 1 -- 172.21.15.99:0/2367018761 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a8744edd00 msgr2=0x55a8744d3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:50.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.084+0000 7fc6c966d700 1 --2- 172.21.15.99:0/2367018761 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a8744edd00 0x55a8744d3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:50.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.084+0000 7fc6c966d700 1 -- 172.21.15.99:0/2367018761 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8744a3a50 msgr2=0x55a8744e7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:50.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.084+0000 7fc6c966d700 1 --2- 172.21.15.99:0/2367018761 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8744a3a50 0x55a8744e7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:50.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.084+0000 7fc6c966d700 1 -- 172.21.15.99:0/2367018761 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8744a65f0 con 0x55a8744ccf60 2023-10-08T00:14:50.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.084+0000 7fc6ca66f700 1 --2- 172.21.15.99:0/2367018761 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a8744edd00 0x55a8744d3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:14:50.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.085+0000 7fc6c966d700 1 --2- 172.21.15.99:0/2367018761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8744ccf60 0x55a8744f9590 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fc6b405d860 tx=0x7fc6b406f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2008c0da3d37f12c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:50.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.085+0000 7fc6af7fe700 1 -- 172.21.15.99:0/2367018761 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b400bb40 con 0x55a8744ccf60 2023-10-08T00:14:50.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.085+0000 7fc6af7fe700 1 -- 172.21.15.99:0/2367018761 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b405d600 con 0x55a8744ccf60 2023-10-08T00:14:50.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.085+0000 7fc6af7fe700 1 -- 172.21.15.99:0/2367018761 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b405da90 con 0x55a8744ccf60 2023-10-08T00:14:50.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.085+0000 7fc6d53cab80 1 -- 172.21.15.99:0/2367018761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8744ccf60 msgr2=0x55a8744f9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:50.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.085+0000 7fc6d53cab80 1 --2- 172.21.15.99:0/2367018761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8744ccf60 0x55a8744f9590 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fc6b405d860 tx=0x7fc6b406f040 comp rx=0 tx=0).stop 2023-10-08T00:14:50.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.085+0000 7fc6d53cab80 1 -- 172.21.15.99:0/2367018761 shutdown_connections 2023-10-08T00:14:50.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.085+0000 7fc6d53cab80 1 --2- 172.21.15.99:0/2367018761 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8744a3a50 0x55a8744e7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:50.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.085+0000 7fc6d53cab80 1 --2- 172.21.15.99:0/2367018761 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a8744edd00 0x55a8744d3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:50.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.085+0000 7fc6d53cab80 1 --2- 172.21.15.99:0/2367018761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8744ccf60 0x55a8744f9590 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:50.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.085+0000 7fc6d53cab80 1 -- 172.21.15.99:0/2367018761 >> 172.21.15.99:0/2367018761 conn(0x55a8744a5370 msgr2=0x55a8744e6d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:50.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.085+0000 7fc6d53cab80 1 -- 172.21.15.99:0/2367018761 shutdown_connections 2023-10-08T00:14:50.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.085+0000 7fc6d53cab80 1 -- 172.21.15.99:0/2367018761 wait complete. 2023-10-08T00:14:50.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.086+0000 7fc6d53cab80 1 -- start start 2023-10-08T00:14:50.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.086+0000 7fc6d53cab80 1 Processor -- start 2023-10-08T00:14:50.093 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.087+0000 7fc6d53cab80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8744a3a50 0x55a8744d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:50.093 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.087+0000 7fc6d53cab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8744ccf60 0x55a87454eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:50.093 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.087+0000 7fc6d53cab80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a8744edd00 0x55a8744e3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:50.093 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.087+0000 7fc6d53cab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8744f9250 con 0x55a8744ccf60 2023-10-08T00:14:50.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.087+0000 7fc6d53cab80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a874550a90 con 0x55a8744a3a50 2023-10-08T00:14:50.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.087+0000 7fc6d53cab80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a8744f8320 con 0x55a8744edd00 2023-10-08T00:14:50.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.087+0000 7fc6d53cab80 10 client.0 _ll_register_callbacks cb 0x55a874467110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T00:14:50.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.087+0000 7fc6c966d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8744ccf60 0x55a87454eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:50.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.087+0000 7fc6c966d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8744ccf60 0x55a87454eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45828/0 (socket says 172.21.15.99:45828) 2023-10-08T00:14:50.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.087+0000 7fc6c966d700 1 -- 172.21.15.99:0/2224541180 learned_addr learned my addr 172.21.15.99:0/2224541180 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:50.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.087+0000 7fc6ca66f700 1 --2- 172.21.15.99:0/2224541180 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a8744edd00 0x55a8744e3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:50.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.087+0000 7fc6c9e6e700 1 --2- 172.21.15.99:0/2224541180 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8744a3a50 0x55a8744d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:50.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.087+0000 7fc6c966d700 1 -- 172.21.15.99:0/2224541180 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a8744edd00 msgr2=0x55a8744e3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:50.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.087+0000 7fc6c966d700 1 --2- 172.21.15.99:0/2224541180 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a8744edd00 0x55a8744e3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:50.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.087+0000 7fc6c966d700 1 -- 172.21.15.99:0/2224541180 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8744a3a50 msgr2=0x55a8744d3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:50.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.087+0000 7fc6c966d700 1 --2- 172.21.15.99:0/2224541180 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8744a3a50 0x55a8744d3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:50.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.087+0000 7fc6c966d700 1 -- 172.21.15.99:0/2224541180 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a874230fe0 con 0x55a8744ccf60 2023-10-08T00:14:50.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.087+0000 7fc6ca66f700 1 --2- 172.21.15.99:0/2224541180 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a8744edd00 0x55a8744e3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:14:50.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.087+0000 7fc6c9e6e700 1 --2- 172.21.15.99:0/2224541180 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8744a3a50 0x55a8744d3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:14:50.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.087+0000 7fc6c966d700 1 --2- 172.21.15.99:0/2224541180 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8744ccf60 0x55a87454eeb0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4052ff0 tx=0x7fc6b4082000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb36ca6a276ed61b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:50.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.087+0000 7fc6a3fff700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T00:14:50.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.087+0000 7fc6a3fff700 1 -- 172.21.15.99:0/2224541180 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b405d860 con 0x55a8744ccf60 2023-10-08T00:14:50.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.087+0000 7fc6a3fff700 1 -- 172.21.15.99:0/2224541180 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc6b4082070 con 0x55a8744ccf60 2023-10-08T00:14:50.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.087+0000 7fc6d53cab80 1 -- 172.21.15.99:0/2224541180 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc6b4059d40 con 0x55a8744ccf60 2023-10-08T00:14:50.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.087+0000 7fc6a3fff700 1 -- 172.21.15.99:0/2224541180 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6b4056af0 con 0x55a8744ccf60 2023-10-08T00:14:50.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.087+0000 7fc6d53cab80 10 client.6201 fetch_fsmap learned FSMap version 40 2023-10-08T00:14:50.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.087+0000 7fc6d53cab80 1 -- 172.21.15.99:0/2224541180 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x55a8743c2780 con 0x55a8744ccf60 2023-10-08T00:14:50.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.088+0000 7fc6a3fff700 1 -- 172.21.15.99:0/2224541180 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b4059d40 con 0x55a8744ccf60 2023-10-08T00:14:50.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.088+0000 7fc6a3fff700 1 -- 172.21.15.99:0/2224541180 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 40) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fc6b4055fe0 con 0x55a8744ccf60 2023-10-08T00:14:50.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.088+0000 7fc6d53cab80 10 client.6201 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T00:14:50.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.088+0000 7fc6d53cab80 10 client.6201 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-08T00:14:50.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.088+0000 7fc6d53cab80 20 client.6201 populate_metadata read hostname 'smithi099' 2023-10-08T00:14:50.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.088+0000 7fc6d53cab80 1 --2- 172.21.15.99:0/2224541180 >> [v2:172.21.15.99:6833/104443412,v1:172.21.15.99:6835/104443412] conn(0x55a8744a3e20 0x55a8744a41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:50.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.088+0000 7fc6d53cab80 4 client.6201 mds_command: new command op to 6039 tid=0[{"prefix": "status"}] 2023-10-08T00:14:50.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.088+0000 7fc6d53cab80 1 -- 172.21.15.99:0/2224541180 --> [v2:172.21.15.99:6833/104443412,v1:172.21.15.99:6835/104443412] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a874396e00 con 0x55a8744a3e20 2023-10-08T00:14:50.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.088+0000 7fc6c9e6e700 1 --2- 172.21.15.99:0/2224541180 >> [v2:172.21.15.99:6833/104443412,v1:172.21.15.99:6835/104443412] conn(0x55a8744a3e20 0x55a8744a41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:50.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.088+0000 7fc6a3fff700 10 client.6201 ms_handle_connect on v2:172.21.15.99:6833/104443412 2023-10-08T00:14:50.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.088+0000 7fc6c9e6e700 1 --2- 172.21.15.99:0/2224541180 >> [v2:172.21.15.99:6833/104443412,v1:172.21.15.99:6835/104443412] conn(0x55a8744a3e20 0x55a8744a41f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6485682ed2e1984b server_cookie=e8a50dbc35c0ed1d in_seq=0 out_seq=0 2023-10-08T00:14:50.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.088+0000 7fc6a3fff700 1 -- 172.21.15.99:0/2224541180 <== mds.0 v2:172.21.15.99:6833/104443412 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55a874396e00 con 0x55a8744a3e20 2023-10-08T00:14:50.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.088+0000 7fc6a3fff700 10 client.6201 handle_command_reply: tid=0 2023-10-08T00:14:50.097 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.088+0000 7fc6d53cab80 1 client.6201 shutdown 2023-10-08T00:14:50.097 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.088+0000 7fc6d53cab80 1 -- 172.21.15.99:0/2224541180 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8744ccf60 msgr2=0x55a87454eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:50.097 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.088+0000 7fc6d53cab80 1 --2- 172.21.15.99:0/2224541180 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8744ccf60 0x55a87454eeb0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4052ff0 tx=0x7fc6b4082000 comp rx=0 tx=0).stop 2023-10-08T00:14:50.097 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.088+0000 7fc6d53cab80 1 -- 172.21.15.99:0/2224541180 shutdown_connections 2023-10-08T00:14:50.097 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.088+0000 7fc6d53cab80 1 --2- 172.21.15.99:0/2224541180 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8744a3a50 0x55a8744d3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:50.097 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.088+0000 7fc6d53cab80 1 --2- 172.21.15.99:0/2224541180 >> [v2:172.21.15.99:6833/104443412,v1:172.21.15.99:6835/104443412] conn(0x55a8744a3e20 0x55a8744a41f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:50.097 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.089+0000 7fc6d53cab80 1 --2- 172.21.15.99:0/2224541180 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a8744edd00 0x55a8744e3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:50.097 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.089+0000 7fc6d53cab80 1 --2- 172.21.15.99:0/2224541180 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8744ccf60 0x55a87454eeb0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:50.097 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.089+0000 7fc6a3fff700 0 client.6201 ms_handle_reset on v2:172.21.15.99:6833/104443412 2023-10-08T00:14:50.097 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.089+0000 7fc6d53cab80 1 -- 172.21.15.99:0/2224541180 >> 172.21.15.99:0/2224541180 conn(0x55a8744a5370 msgr2=0x55a8744a0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:50.097 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.089+0000 7fc6d53cab80 1 -- 172.21.15.99:0/2224541180 shutdown_connections 2023-10-08T00:14:50.097 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.089+0000 7fc6d53cab80 1 -- 172.21.15.99:0/2224541180 wait complete. 2023-10-08T00:14:50.098 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.089+0000 7fc6d53cab80 20 client.6201 trim_cache size 0 max 16384 2023-10-08T00:14:50.098 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:14:50.098 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", 2023-10-08T00:14:50.098 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 0, 2023-10-08T00:14:50.098 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 6039, 2023-10-08T00:14:50.098 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2023-10-08T00:14:50.098 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2023-10-08T00:14:50.098 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2023-10-08T00:14:50.099 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 5.7730350819999998, 2023-10-08T00:14:50.099 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 40, 2023-10-08T00:14:50.099 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 86, 2023-10-08T00:14:50.099 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 86, 2023-10-08T00:14:50.099 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 11.754842249999999 2023-10-08T00:14:50.099 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:14:50.100 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.089+0000 7fc6cc0d2700 1 -- 172.21.15.99:0/3982870249 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc6a8066f30 msgr2=0x7fc6a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:50.100 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.089+0000 7fc6cc0d2700 1 --2- 172.21.15.99:0/3982870249 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc6a8066f30 0x7fc6a80693e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0000f80 tx=0x7fc6c0009040 comp rx=0 tx=0).stop 2023-10-08T00:14:50.100 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.089+0000 7fc6cc0d2700 1 -- 172.21.15.99:0/3982870249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6c4159cf0 msgr2=0x7fc6c4158a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:50.100 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.089+0000 7fc6cc0d2700 1 --2- 172.21.15.99:0/3982870249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6c4159cf0 0x7fc6c4158a90 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fc6c406de20 tx=0x7fc6b4017000 comp rx=0 tx=0).stop 2023-10-08T00:14:50.100 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.089+0000 7fc6cc0d2700 1 -- 172.21.15.99:0/3982870249 shutdown_connections 2023-10-08T00:14:50.100 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.089+0000 7fc6cc0d2700 1 --2- 172.21.15.99:0/3982870249 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc6a8066f30 0x7fc6a80693e0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:50.100 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.089+0000 7fc6cc0d2700 1 --2- 172.21.15.99:0/3982870249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc6c415a0c0 0x7fc6c4155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:50.100 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.089+0000 7fc6cc0d2700 1 --2- 172.21.15.99:0/3982870249 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc6c40cf1c0 0x7fc6c40abfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:50.100 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.089+0000 7fc6cc0d2700 1 --2- 172.21.15.99:0/3982870249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6c4159cf0 0x7fc6c4158a90 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:50.100 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.089+0000 7fc6cc0d2700 1 -- 172.21.15.99:0/3982870249 >> 172.21.15.99:0/3982870249 conn(0x7fc6c406c6e0 msgr2=0x7fc6c415dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:50.101 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.089+0000 7fc6cc0d2700 1 -- 172.21.15.99:0/3982870249 shutdown_connections 2023-10-08T00:14:50.101 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.089+0000 7fc6cc0d2700 1 -- 172.21.15.99:0/3982870249 wait complete. 2023-10-08T00:14:50.102 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T00:14:50.216 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.215+0000 7f267d1d6700 1 Processor -- start 2023-10-08T00:14:50.216 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.215+0000 7f267d1d6700 1 -- start start 2023-10-08T00:14:50.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.216+0000 7f267d1d6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26780cd6e0 0x7f26780cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:50.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.216+0000 7f267d1d6700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26780cb7c0 con 0x7f267815cbd0 2023-10-08T00:14:50.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.216+0000 7f267d1d6700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f267815e290 con 0x7f26780cdab0 2023-10-08T00:14:50.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.216+0000 7f267d1d6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f267815e850 con 0x7f26780cd6e0 2023-10-08T00:14:50.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.216+0000 7f267c9d5700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f267815cbd0 0x7f267815d020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:34778/0 (socket says 172.21.15.99:34778) 2023-10-08T00:14:50.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.216+0000 7f267c9d5700 1 -- 172.21.15.99:0/3291790976 learned_addr learned my addr 172.21.15.99:0/3291790976 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:14:50.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.216+0000 7f2677fff700 1 --2- 172.21.15.99:0/3291790976 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26780cd6e0 0x7f26780cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:50.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.216+0000 7f2676ffd700 1 -- 172.21.15.99:0/3291790976 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 47362384 0 0) 0x7f26780cb7c0 con 0x7f267815cbd0 2023-10-08T00:14:50.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.216+0000 7f2676ffd700 1 -- 172.21.15.99:0/3291790976 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2660003570 con 0x7f267815cbd0 2023-10-08T00:14:50.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.216+0000 7f2676ffd700 1 -- 172.21.15.99:0/3291790976 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 476585179 0 0) 0x7f267815e290 con 0x7f26780cdab0 2023-10-08T00:14:50.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.216+0000 7f2676ffd700 1 -- 172.21.15.99:0/3291790976 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26780cb7c0 con 0x7f26780cdab0 2023-10-08T00:14:50.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.216+0000 7f2676ffd700 1 -- 172.21.15.99:0/3291790976 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 672950537 0 0) 0x7f2660003570 con 0x7f267815cbd0 2023-10-08T00:14:50.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.216+0000 7f2676ffd700 1 -- 172.21.15.99:0/3291790976 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26780cd6e0 msgr2=0x7f26780cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:50.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.216+0000 7f2676ffd700 1 --2- 172.21.15.99:0/3291790976 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26780cd6e0 0x7f26780cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:50.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.216+0000 7f2676ffd700 1 -- 172.21.15.99:0/3291790976 >> v1:172.21.15.191:6789/0 conn(0x7f26780cdab0 legacy=0x7f267815d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:50.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.216+0000 7f2676ffd700 1 -- 172.21.15.99:0/3291790976 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f267815e570 con 0x7f267815cbd0 2023-10-08T00:14:50.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.216+0000 7f2676ffd700 1 -- 172.21.15.99:0/3291790976 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f26700017f0 con 0x7f267815cbd0 2023-10-08T00:14:50.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.216+0000 7f2676ffd700 1 -- 172.21.15.99:0/3291790976 >> v1:172.21.15.99:6789/0 conn(0x7f267815cbd0 legacy=0x7f267815d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:50.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.216+0000 7f2676ffd700 1 --2- 172.21.15.99:0/3291790976 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26780cd6e0 0x7f2660003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:50.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.217+0000 7f2676ffd700 1 --2- 172.21.15.99:0/3291790976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2660004180 0x7f2660004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:50.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.217+0000 7f2676ffd700 1 --2- 172.21.15.99:0/3291790976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2660004d50 0x7f26600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:50.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.217+0000 7f2676ffd700 1 -- 172.21.15.99:0/3291790976 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267815e850 con 0x7f2660004180 2023-10-08T00:14:50.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.217+0000 7f2676ffd700 1 -- 172.21.15.99:0/3291790976 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f266000bba0 con 0x7f2660004d50 2023-10-08T00:14:50.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.217+0000 7f2676ffd700 1 -- 172.21.15.99:0/3291790976 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f266000bd60 con 0x7f26780cd6e0 2023-10-08T00:14:50.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.217+0000 7f26777fe700 1 --2- 172.21.15.99:0/3291790976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2660004180 0x7f2660004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:50.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.217+0000 7f2677fff700 1 --2- 172.21.15.99:0/3291790976 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26780cd6e0 0x7f2660003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:50.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.217+0000 7f267c9d5700 1 --2- 172.21.15.99:0/3291790976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2660004d50 0x7f26600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:50.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.217+0000 7f26777fe700 1 -- 172.21.15.99:0/3291790976 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26780cd6e0 msgr2=0x7f2660003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:50.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.217+0000 7f26777fe700 1 --2- 172.21.15.99:0/3291790976 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26780cd6e0 0x7f2660003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:50.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.217+0000 7f26777fe700 1 -- 172.21.15.99:0/3291790976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2660004d50 msgr2=0x7f26600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:50.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.217+0000 7f26777fe700 1 --2- 172.21.15.99:0/3291790976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2660004d50 0x7f26600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:50.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.217+0000 7f26777fe700 1 -- 172.21.15.99:0/3291790976 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2660003570 con 0x7f2660004180 2023-10-08T00:14:50.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.217+0000 7f26777fe700 1 --2- 172.21.15.99:0/3291790976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2660004180 0x7f2660004810 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f266c001a10 tx=0x7f266c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4745b8af47aa3f4a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:50.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.217+0000 7f2676ffd700 1 -- 172.21.15.99:0/3291790976 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266c015960 con 0x7f2660004180 2023-10-08T00:14:50.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.217+0000 7f2676ffd700 1 -- 172.21.15.99:0/3291790976 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f266c015ac0 con 0x7f2660004180 2023-10-08T00:14:50.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.217+0000 7f2676ffd700 1 -- 172.21.15.99:0/3291790976 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266c015e60 con 0x7f2660004180 2023-10-08T00:14:50.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.218+0000 7f267d1d6700 1 -- 172.21.15.99:0/3291790976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2660004180 msgr2=0x7f2660004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:50.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.218+0000 7f267d1d6700 1 --2- 172.21.15.99:0/3291790976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2660004180 0x7f2660004810 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f266c001a10 tx=0x7f266c007d30 comp rx=0 tx=0).stop 2023-10-08T00:14:50.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.218+0000 7f267c9d5700 1 -- 172.21.15.99:0/3291790976 reap_dead start 2023-10-08T00:14:50.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.218+0000 7f267d1d6700 1 -- 172.21.15.99:0/3291790976 shutdown_connections 2023-10-08T00:14:50.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.218+0000 7f267d1d6700 1 -- 172.21.15.99:0/3291790976 >> 172.21.15.99:0/3291790976 conn(0x7f2678067660 msgr2=0x7f2678067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:50.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.218+0000 7f267d1d6700 1 -- 172.21.15.99:0/3291790976 shutdown_connections 2023-10-08T00:14:50.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.218+0000 7f267d1d6700 1 -- 172.21.15.99:0/3291790976 wait complete. 2023-10-08T00:14:50.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.219+0000 7f267d1d6700 1 Processor -- start 2023-10-08T00:14:50.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.219+0000 7f267d1d6700 1 -- start start 2023-10-08T00:14:50.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.220+0000 7f267d1d6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2660004180 0x7f26780b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:50.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.220+0000 7f267d1d6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26780b47f0 0x7f267814a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:50.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.220+0000 7f267d1d6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2678154af0 0x7f2678154f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:50.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.220+0000 7f267d1d6700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2678154660 con 0x7f26780b47f0 2023-10-08T00:14:50.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.220+0000 7f267d1d6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26780b2730 con 0x7f2678154af0 2023-10-08T00:14:50.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.220+0000 7f267d1d6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26780b2a10 con 0x7f2660004180 2023-10-08T00:14:50.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.220+0000 7f26777fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26780b47f0 0x7f267814a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:50.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.220+0000 7f26777fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26780b47f0 0x7f267814a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45834/0 (socket says 172.21.15.99:45834) 2023-10-08T00:14:50.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.220+0000 7f26777fe700 1 -- 172.21.15.99:0/1615695678 learned_addr learned my addr 172.21.15.99:0/1615695678 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:50.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.220+0000 7f2677fff700 1 --2- 172.21.15.99:0/1615695678 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2660004180 0x7f26780b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:50.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.220+0000 7f267c9d5700 1 --2- 172.21.15.99:0/1615695678 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2678154af0 0x7f2678154f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:50.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.220+0000 7f26777fe700 1 -- 172.21.15.99:0/1615695678 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2660004180 msgr2=0x7f26780b6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:50.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.220+0000 7f26777fe700 1 --2- 172.21.15.99:0/1615695678 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2660004180 0x7f26780b6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:50.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.220+0000 7f26777fe700 1 -- 172.21.15.99:0/1615695678 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2678154af0 msgr2=0x7f2678154f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:50.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.220+0000 7f26777fe700 1 --2- 172.21.15.99:0/1615695678 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2678154af0 0x7f2678154f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:50.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.220+0000 7f26777fe700 1 -- 172.21.15.99:0/1615695678 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26780b65d0 con 0x7f26780b47f0 2023-10-08T00:14:50.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.220+0000 7f26777fe700 1 --2- 172.21.15.99:0/1615695678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26780b47f0 0x7f267814a830 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f266000bb70 tx=0x7f266c003000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:50.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.220+0000 7f265ffff700 1 -- 172.21.15.99:0/1615695678 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266c025070 con 0x7f26780b47f0 2023-10-08T00:14:50.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.221+0000 7f265ffff700 1 -- 172.21.15.99:0/1615695678 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f266c001360 con 0x7f26780b47f0 2023-10-08T00:14:50.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.221+0000 7f265ffff700 1 -- 172.21.15.99:0/1615695678 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266c015960 con 0x7f26780b47f0 2023-10-08T00:14:50.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.221+0000 7f267d1d6700 1 -- 172.21.15.99:0/1615695678 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f267814ad70 con 0x7f26780b47f0 2023-10-08T00:14:50.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.221+0000 7f267d1d6700 1 -- 172.21.15.99:0/1615695678 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26780c7ee0 con 0x7f26780b47f0 2023-10-08T00:14:50.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.222+0000 7f267d1d6700 1 -- 172.21.15.99:0/1615695678 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2678154450 con 0x7f26780b47f0 2023-10-08T00:14:50.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.222+0000 7f265ffff700 1 -- 172.21.15.99:0/1615695678 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f266c02a460 con 0x7f26780b47f0 2023-10-08T00:14:50.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.222+0000 7f265ffff700 1 --2- 172.21.15.99:0/1615695678 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2664066f30 0x7f26640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:50.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.222+0000 7f265ffff700 1 -- 172.21.15.99:0/1615695678 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f266c08d6d0 con 0x7f26780b47f0 2023-10-08T00:14:50.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.223+0000 7f2677fff700 1 --2- 172.21.15.99:0/1615695678 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2664066f30 0x7f26640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:50.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.223+0000 7f2677fff700 1 --2- 172.21.15.99:0/1615695678 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2664066f30 0x7f26640693e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2668006f00 tx=0x7f26680091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:50.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.226+0000 7f265ffff700 1 -- 172.21.15.99:0/1615695678 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f266c05a940 con 0x7f26780b47f0 2023-10-08T00:14:50.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.452+0000 7f267d1d6700 1 -- 172.21.15.99:0/1615695678 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f267804bb80 con 0x7f26780b47f0 2023-10-08T00:14:50.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.453+0000 7f265ffff700 1 -- 172.21.15.99:0/1615695678 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v40) v1 ==== 53+0+73 (secure 0 0 0) 0x7f266c02f070 con 0x7f26780b47f0 2023-10-08T00:14:50.454 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T00:14:50.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.455+0000 7f267d1d6700 1 -- 172.21.15.99:0/1615695678 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2664066f30 msgr2=0x7f26640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:50.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.455+0000 7f267d1d6700 1 --2- 172.21.15.99:0/1615695678 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2664066f30 0x7f26640693e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2668006f00 tx=0x7f26680091a0 comp rx=0 tx=0).stop 2023-10-08T00:14:50.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.455+0000 7f267d1d6700 1 -- 172.21.15.99:0/1615695678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26780b47f0 msgr2=0x7f267814a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:50.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.455+0000 7f267d1d6700 1 --2- 172.21.15.99:0/1615695678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26780b47f0 0x7f267814a830 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f266000bb70 tx=0x7f266c003000 comp rx=0 tx=0).stop 2023-10-08T00:14:50.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.455+0000 7f267d1d6700 1 -- 172.21.15.99:0/1615695678 shutdown_connections 2023-10-08T00:14:50.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.455+0000 7f267d1d6700 1 --2- 172.21.15.99:0/1615695678 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2664066f30 0x7f26640693e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:50.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.455+0000 7f267d1d6700 1 --2- 172.21.15.99:0/1615695678 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2678154af0 0x7f2678154f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:50.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.455+0000 7f267d1d6700 1 --2- 172.21.15.99:0/1615695678 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2660004180 0x7f26780b6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:50.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.455+0000 7f267d1d6700 1 --2- 172.21.15.99:0/1615695678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26780b47f0 0x7f267814a830 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:50.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.455+0000 7f267d1d6700 1 -- 172.21.15.99:0/1615695678 >> 172.21.15.99:0/1615695678 conn(0x7f2678067660 msgr2=0x7f26780c5de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:50.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.455+0000 7f267d1d6700 1 -- 172.21.15.99:0/1615695678 shutdown_connections 2023-10-08T00:14:50.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:50.455+0000 7f267d1d6700 1 -- 172.21.15.99:0/1615695678 wait complete. 2023-10-08T00:14:50.467 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T00:14:50.468 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T00:14:50.468 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi099.front.sepia.ceph.com 2023-10-08T00:14:50.468 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T00:14:50.468 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T00:14:50.468 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T00:14:50.468 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T00:14:50.468 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T00:14:50.468 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T00:14:50.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:14:50.468 DEBUG:teuthology.orchestra.run.smithi099:> ip addr 2023-10-08T00:14:50.484 INFO:teuthology.orchestra.run.smithi099.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T00:14:50.484 INFO:teuthology.orchestra.run.smithi099.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T00:14:50.484 INFO:teuthology.orchestra.run.smithi099.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T00:14:50.484 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:14:50.484 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 ::1/128 scope host 2023-10-08T00:14:50.485 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:14:50.485 INFO:teuthology.orchestra.run.smithi099.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:14:50.485 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:10 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:14:50.485 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f0 2023-10-08T00:14:50.485 INFO:teuthology.orchestra.run.smithi099.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:14:50.485 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:11 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:14:50.485 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f1 2023-10-08T00:14:50.485 INFO:teuthology.orchestra.run.smithi099.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:14:50.485 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:02 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:14:50.485 INFO:teuthology.orchestra.run.smithi099.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T00:14:50.485 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:03 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:14:50.485 INFO:teuthology.orchestra.run.smithi099.stdout: inet 172.21.15.99/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T00:14:50.486 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft 42299sec preferred_lft 42299sec 2023-10-08T00:14:50.486 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::ec4:7aff:fe8f:d803/64 scope link 2023-10-08T00:14:50.486 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:14:50.486 INFO:teuthology.orchestra.run.smithi099.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T00:14:50.486 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 22:2c:1c:f6:42:e1 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:14:50.486 INFO:teuthology.orchestra.run.smithi099.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T00:14:50.486 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:14:50.486 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::600d:26ff:fede:5eec/64 scope link 2023-10-08T00:14:50.486 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:14:50.486 INFO:teuthology.orchestra.run.smithi099.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T00:14:50.486 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether ae:2f:b5:94:69:ba brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T00:14:50.486 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::ac2f:b5ff:fe94:69ba/64 scope link 2023-10-08T00:14:50.487 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:14:50.487 INFO:teuthology.orchestra.run.smithi099.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T00:14:50.487 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 22:2c:1c:f6:42:e1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T00:14:50.487 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::202c:1cff:fef6:42e1/64 scope link 2023-10-08T00:14:50.487 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:14:50.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:14:50.487 DEBUG:teuthology.orchestra.run.smithi099:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T00:14:50.547 INFO:teuthology.orchestra.run.smithi099.stdout:1 2023-10-08T00:14:50.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:14:50.548 DEBUG:teuthology.orchestra.run.smithi099:> route 2023-10-08T00:14:50.604 INFO:teuthology.orchestra.run.smithi099.stdout:Kernel IP routing table 2023-10-08T00:14:50.605 INFO:teuthology.orchestra.run.smithi099.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T00:14:50.605 INFO:teuthology.orchestra.run.smithi099.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T00:14:50.605 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T00:14:50.605 INFO:teuthology.orchestra.run.smithi099.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T00:14:50.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:14:50.605 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2023-10-08T00:14:50.605 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2023-10-08T00:14:50.605 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T00:14:50.606 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T00:14:50.606 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T00:14:50.606 DEBUG:teuthology.orchestra.run.smithi099:> ') 2023-10-08T00:14:50.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:14:50.704 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2023-10-08T00:14:50.719 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T00:14:50.719 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T00:14:50.719 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T00:14:50.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:14:50.719 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set brx.0 up 2023-10-08T00:14:50.783 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:14:50.784 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T00:14:50.838 INFO:teuthology.orchestra.run.smithi099.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T00:14:50.838 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:14:50.839 DEBUG:teuthology.orchestra.run.smithi099:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T00:14:50.893 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-08T00:14:50.893 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T00:14:50.893 DEBUG:teuthology.orchestra.run.smithi099:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:14:50.974 INFO:teuthology.orchestra.run.smithi099.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:14:50.974 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:14:50.993 INFO:teuthology.orchestra.run.smithi099.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:14:50.993 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:14:50.993 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: resolved to: "172.21.15.99:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T00:14:50.993 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: trying mount with new device syntax: 0@06962cdb-9a6f-417f-82fd-9e4e3dedbccf.cephfs=/ 2023-10-08T00:14:50.993 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.99:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T00:14:50.993 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T00:14:50.994 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T00:14:50.994 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T00:14:51.014 INFO:teuthology.orchestra.run.smithi099.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:14:51.014 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:14:51.014 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: resolved to: "172.21.15.99:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T00:14:51.014 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: trying mount with new device syntax: 0@06962cdb-9a6f-417f-82fd-9e4e3dedbccf.cephfs=/ 2023-10-08T00:14:51.015 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.99:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T00:14:51.015 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T00:14:51.015 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T00:14:51.015 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T00:14:51.015 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: resolved to: "172.21.15.99:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T00:14:51.015 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.99:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T00:14:51.015 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=06962cdb-9a6f-417f-82fd-9e4e3dedbccf" will pass to kernel 2023-10-08T00:14:51.016 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T00:14:51.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:14:51.016 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T00:14:51.081 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T00:14:51.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.235+0000 7f42950a9700 1 Processor -- start 2023-10-08T00:14:51.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.235+0000 7f42950a9700 1 -- start start 2023-10-08T00:14:51.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.236+0000 7f42950a9700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f42900abae0 0x7f42900c6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:51.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.236+0000 7f42950a9700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f42900abeb0 0x7f42900c73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:51.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.236+0000 7f42950a9700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42900c9cb0 con 0x7f42900c8400 2023-10-08T00:14:51.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.236+0000 7f42950a9700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42900c90a0 con 0x7f42900abeb0 2023-10-08T00:14:51.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.236+0000 7f42950a9700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42900c9380 con 0x7f42900abae0 2023-10-08T00:14:51.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.236+0000 7f428659c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f42900abeb0 0x7f42900c73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:51.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.236+0000 7f428f59e700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f42900c8400 0x7f42900c8840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:34788/0 (socket says 172.21.15.99:34788) 2023-10-08T00:14:51.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.236+0000 7f428f59e700 1 -- 172.21.15.99:0/3334491881 learned_addr learned my addr 172.21.15.99:0/3334491881 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:14:51.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.236+0000 7f428ed9d700 1 --2- 172.21.15.99:0/3334491881 >> v2:172.21.15.191:3300/0 conn(0x7f42900abae0 0x7f42900c6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:51.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.236+0000 7f428659c700 1 -- 172.21.15.99:0/3334491881 >> v2:172.21.15.191:3300/0 conn(0x7f42900abae0 msgr2=0x7f42900c6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:51.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.236+0000 7f428659c700 1 --2- 172.21.15.99:0/3334491881 >> v2:172.21.15.191:3300/0 conn(0x7f42900abae0 0x7f42900c6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:51.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.236+0000 7f428659c700 1 -- 172.21.15.99:0/3334491881 >> v1:172.21.15.99:6789/0 conn(0x7f42900c8400 legacy=0x7f42900c8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:51.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.236+0000 7f428e59c700 1 -- 172.21.15.99:0/3334491881 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3414496631 0 0) 0x7f42900c9cb0 con 0x7f42900c8400 2023-10-08T00:14:51.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.236+0000 7f428659c700 1 -- 172.21.15.99:0/3334491881 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42900c9660 con 0x7f42900abeb0 2023-10-08T00:14:51.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.237+0000 7f428659c700 1 --2- 172.21.15.99:0/3334491881 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42900abeb0 0x7f42900c73f0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f42900cc7f0 tx=0x7f42780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fb232d499877bcd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:51.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.237+0000 7f428e59c700 1 -- 172.21.15.99:0/3334491881 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427800b6b0 con 0x7f42900abeb0 2023-10-08T00:14:51.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.237+0000 7f428e59c700 1 -- 172.21.15.99:0/3334491881 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f427800b810 con 0x7f42900abeb0 2023-10-08T00:14:51.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.237+0000 7f428e59c700 1 -- 172.21.15.99:0/3334491881 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427800bbb0 con 0x7f42900abeb0 2023-10-08T00:14:51.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.237+0000 7f42950a9700 1 -- 172.21.15.99:0/3334491881 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42900abeb0 msgr2=0x7f42900c73f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:51.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.237+0000 7f42950a9700 1 --2- 172.21.15.99:0/3334491881 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42900abeb0 0x7f42900c73f0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f42900cc7f0 tx=0x7f42780091a0 comp rx=0 tx=0).stop 2023-10-08T00:14:51.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.237+0000 7f42950a9700 1 -- 172.21.15.99:0/3334491881 shutdown_connections 2023-10-08T00:14:51.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.237+0000 7f42950a9700 1 --2- 172.21.15.99:0/3334491881 >> v2:172.21.15.191:3300/0 conn(0x7f42900abae0 0x7f42900c6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:51.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.237+0000 7f42950a9700 1 --2- 172.21.15.99:0/3334491881 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42900abeb0 0x7f42900c73f0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:51.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.237+0000 7f42950a9700 1 -- 172.21.15.99:0/3334491881 >> 172.21.15.99:0/3334491881 conn(0x7f4290067660 msgr2=0x7f4290067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:51.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.237+0000 7f42950a9700 1 -- 172.21.15.99:0/3334491881 shutdown_connections 2023-10-08T00:14:51.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.237+0000 7f42950a9700 1 -- 172.21.15.99:0/3334491881 wait complete. 2023-10-08T00:14:51.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.239+0000 7f42950a9700 1 Processor -- start 2023-10-08T00:14:51.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.239+0000 7f42950a9700 1 -- start start 2023-10-08T00:14:51.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.239+0000 7f42950a9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42900abae0 0x7f4290145de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:51.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.239+0000 7f42950a9700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42900abeb0 0x7f42900c42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:51.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.239+0000 7f42950a9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f42900c8400 0x7f42900ad5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:51.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.239+0000 7f42950a9700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4290146320 con 0x7f42900abae0 2023-10-08T00:14:51.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.239+0000 7f42950a9700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429015be70 con 0x7f42900abeb0 2023-10-08T00:14:51.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.239+0000 7f42950a9700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f429015c150 con 0x7f42900c8400 2023-10-08T00:14:51.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.239+0000 7f428ed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42900abae0 0x7f4290145de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:51.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.239+0000 7f428ed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42900abae0 0x7f4290145de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45856/0 (socket says 172.21.15.99:45856) 2023-10-08T00:14:51.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.239+0000 7f428ed9d700 1 -- 172.21.15.99:0/3043143884 learned_addr learned my addr 172.21.15.99:0/3043143884 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:51.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.240+0000 7f428f59e700 1 --2- 172.21.15.99:0/3043143884 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f42900c8400 0x7f42900ad5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:51.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.240+0000 7f428659c700 1 --2- 172.21.15.99:0/3043143884 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42900abeb0 0x7f42900c42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:51.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.240+0000 7f428ed9d700 1 -- 172.21.15.99:0/3043143884 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f42900c8400 msgr2=0x7f42900ad5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:51.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.240+0000 7f428ed9d700 1 --2- 172.21.15.99:0/3043143884 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f42900c8400 0x7f42900ad5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:51.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.240+0000 7f428ed9d700 1 -- 172.21.15.99:0/3043143884 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42900abeb0 msgr2=0x7f42900c42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:51.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.240+0000 7f428ed9d700 1 --2- 172.21.15.99:0/3043143884 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42900abeb0 0x7f42900c42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:51.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.240+0000 7f428ed9d700 1 -- 172.21.15.99:0/3043143884 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4278009420 con 0x7f42900abae0 2023-10-08T00:14:51.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.240+0000 7f428f59e700 1 --2- 172.21.15.99:0/3043143884 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f42900c8400 0x7f42900ad5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:14:51.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.240+0000 7f428659c700 1 --2- 172.21.15.99:0/3043143884 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42900abeb0 0x7f42900c42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:14:51.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.240+0000 7f428ed9d700 1 --2- 172.21.15.99:0/3043143884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42900abae0 0x7f4290145de0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f428000d860 tx=0x7f4280008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:51.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.240+0000 7f42877fe700 1 -- 172.21.15.99:0/3043143884 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4280009610 con 0x7f42900abae0 2023-10-08T00:14:51.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.240+0000 7f42877fe700 1 -- 172.21.15.99:0/3043143884 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4280009770 con 0x7f42900abae0 2023-10-08T00:14:51.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.240+0000 7f42950a9700 1 -- 172.21.15.99:0/3043143884 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f429015c430 con 0x7f42900abae0 2023-10-08T00:14:51.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.240+0000 7f42877fe700 1 -- 172.21.15.99:0/3043143884 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4280009b30 con 0x7f42900abae0 2023-10-08T00:14:51.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.240+0000 7f42950a9700 1 -- 172.21.15.99:0/3043143884 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42900c5380 con 0x7f42900abae0 2023-10-08T00:14:51.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.241+0000 7f42950a9700 1 -- 172.21.15.99:0/3043143884 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42900c56f0 con 0x7f42900abae0 2023-10-08T00:14:51.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.241+0000 7f42877fe700 1 -- 172.21.15.99:0/3043143884 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f428001e460 con 0x7f42900abae0 2023-10-08T00:14:51.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.241+0000 7f42877fe700 1 --2- 172.21.15.99:0/3043143884 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4270066f30 0x7f42700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:51.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.242+0000 7f42877fe700 1 -- 172.21.15.99:0/3043143884 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4280084ba0 con 0x7f42900abae0 2023-10-08T00:14:51.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.242+0000 7f428659c700 1 --2- 172.21.15.99:0/3043143884 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4270066f30 0x7f42700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:51.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.242+0000 7f428659c700 1 --2- 172.21.15.99:0/3043143884 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4270066f30 0x7f42700693e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f42900cc7f0 tx=0x7f4278017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:51.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.246+0000 7f42877fe700 1 -- 172.21.15.99:0/3043143884 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4280051e10 con 0x7f42900abae0 2023-10-08T00:14:51.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.472+0000 7f42950a9700 1 -- 172.21.15.99:0/3043143884 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f429004c3c0 con 0x7f42900abae0 2023-10-08T00:14:51.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.473+0000 7f42877fe700 1 -- 172.21.15.99:0/3043143884 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v40) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4280055430 con 0x7f42900abae0 2023-10-08T00:14:51.473 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T00:14:51.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.475+0000 7f42950a9700 1 -- 172.21.15.99:0/3043143884 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4270066f30 msgr2=0x7f42700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:51.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.475+0000 7f42950a9700 1 --2- 172.21.15.99:0/3043143884 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4270066f30 0x7f42700693e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f42900cc7f0 tx=0x7f4278017000 comp rx=0 tx=0).stop 2023-10-08T00:14:51.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.475+0000 7f42950a9700 1 -- 172.21.15.99:0/3043143884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42900abae0 msgr2=0x7f4290145de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:51.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.475+0000 7f42950a9700 1 --2- 172.21.15.99:0/3043143884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42900abae0 0x7f4290145de0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f428000d860 tx=0x7f4280008040 comp rx=0 tx=0).stop 2023-10-08T00:14:51.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.475+0000 7f42950a9700 1 -- 172.21.15.99:0/3043143884 shutdown_connections 2023-10-08T00:14:51.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.475+0000 7f42950a9700 1 --2- 172.21.15.99:0/3043143884 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4270066f30 0x7f42700693e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:51.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.475+0000 7f42950a9700 1 --2- 172.21.15.99:0/3043143884 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f42900c8400 0x7f42900ad5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:51.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.475+0000 7f42950a9700 1 --2- 172.21.15.99:0/3043143884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42900abae0 0x7f4290145de0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:51.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.475+0000 7f42950a9700 1 --2- 172.21.15.99:0/3043143884 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42900abeb0 0x7f42900c42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:51.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.475+0000 7f42950a9700 1 -- 172.21.15.99:0/3043143884 >> 172.21.15.99:0/3043143884 conn(0x7f4290067660 msgr2=0x7f4290159040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:51.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.475+0000 7f42950a9700 1 -- 172.21.15.99:0/3043143884 shutdown_connections 2023-10-08T00:14:51.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:51.475+0000 7f42950a9700 1 -- 172.21.15.99:0/3043143884 wait complete. 2023-10-08T00:14:51.487 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T00:14:51.487 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T00:14:51.487 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi191.front.sepia.ceph.com 2023-10-08T00:14:51.487 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-08T00:14:51.487 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-08T00:14:51.487 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T00:14:51.487 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T00:14:51.487 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T00:14:51.488 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T00:14:51.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:14:51.488 DEBUG:teuthology.orchestra.run.smithi191:> ip addr 2023-10-08T00:14:51.507 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T00:14:51.507 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T00:14:51.507 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T00:14:51.507 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:14:51.507 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2023-10-08T00:14:51.507 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:14:51.507 INFO:teuthology.orchestra.run.smithi191.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:14:51.508 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f6 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:14:51.508 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f0 2023-10-08T00:14:51.508 INFO:teuthology.orchestra.run.smithi191.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:14:51.508 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f7 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:14:51.508 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f1 2023-10-08T00:14:51.508 INFO:teuthology.orchestra.run.smithi191.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:14:51.508 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:10 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:14:51.508 INFO:teuthology.orchestra.run.smithi191.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T00:14:51.508 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:11 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:14:51.508 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.21.15.191/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T00:14:51.508 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft 42300sec preferred_lft 42300sec 2023-10-08T00:14:51.508 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec4:7aff:fe88:7211/64 scope link 2023-10-08T00:14:51.509 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:14:51.509 INFO:teuthology.orchestra.run.smithi191.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T00:14:51.509 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 9a:bc:39:72:87:3d brd ff:ff:ff:ff:ff:ff 2023-10-08T00:14:51.509 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T00:14:51.509 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:14:51.509 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec77:a7ff:fea4:fbb/64 scope link 2023-10-08T00:14:51.509 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:14:51.509 INFO:teuthology.orchestra.run.smithi191.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T00:14:51.509 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether ea:1c:d9:46:05:b3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-08T00:14:51.509 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::e81c:d9ff:fe46:5b3/64 scope link 2023-10-08T00:14:51.509 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:14:51.509 INFO:teuthology.orchestra.run.smithi191.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T00:14:51.510 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 9a:bc:39:72:87:3d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-08T00:14:51.510 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::98bc:39ff:fe72:873d/64 scope link 2023-10-08T00:14:51.510 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:14:51.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:14:51.510 DEBUG:teuthology.orchestra.run.smithi191:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T00:14:51.575 INFO:teuthology.orchestra.run.smithi191.stdout:1 2023-10-08T00:14:51.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:14:51.577 DEBUG:teuthology.orchestra.run.smithi191:> route 2023-10-08T00:14:51.635 INFO:teuthology.orchestra.run.smithi191.stdout:Kernel IP routing table 2023-10-08T00:14:51.635 INFO:teuthology.orchestra.run.smithi191.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T00:14:51.635 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T00:14:51.636 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T00:14:51.636 INFO:teuthology.orchestra.run.smithi191.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T00:14:51.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:14:51.637 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2023-10-08T00:14:51.637 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2023-10-08T00:14:51.637 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T00:14:51.637 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T00:14:51.637 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T00:14:51.637 DEBUG:teuthology.orchestra.run.smithi191:> ') 2023-10-08T00:14:51.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:14:51.743 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2023-10-08T00:14:51.758 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-08T00:14:51.758 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-08T00:14:51.759 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T00:14:51.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:14:51.759 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set brx.0 up 2023-10-08T00:14:51.824 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:14:51.824 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-08T00:14:51.879 INFO:teuthology.orchestra.run.smithi191.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-08T00:14:51.879 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:14:51.879 DEBUG:teuthology.orchestra.run.smithi191:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-08T00:14:51.933 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-08T00:14:51.933 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T00:14:51.933 DEBUG:teuthology.orchestra.run.smithi191:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:14:52.014 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:14:52.014 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:14:52.032 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:14:52.032 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:14:52.032 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.99:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T00:14:52.032 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@06962cdb-9a6f-417f-82fd-9e4e3dedbccf.cephfs=/ 2023-10-08T00:14:52.033 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.99:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T00:14:52.033 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T00:14:52.033 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T00:14:52.033 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T00:14:52.054 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:14:52.054 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:14:52.054 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.99:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T00:14:52.054 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@06962cdb-9a6f-417f-82fd-9e4e3dedbccf.cephfs=/ 2023-10-08T00:14:52.054 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.99:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T00:14:52.054 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T00:14:52.055 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T00:14:52.055 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T00:14:52.055 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.99:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T00:14:52.055 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.99:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T00:14:52.055 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=06962cdb-9a6f-417f-82fd-9e4e3dedbccf" will pass to kernel 2023-10-08T00:14:52.057 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T00:14:52.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:14:52.057 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-08T00:14:52.121 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:14:52.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.235+0000 7fbb83fff700 1 Processor -- start 2023-10-08T00:14:52.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.235+0000 7fbb83fff700 1 -- start start 2023-10-08T00:14:52.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.236+0000 7fbb83fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb840cd6e0 0x7fbb840cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:52.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.236+0000 7fbb83fff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fbb8415cbd0 0x7fbb8415d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:52.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.236+0000 7fbb83fff700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb840cb7c0 con 0x7fbb840cdab0 2023-10-08T00:14:52.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.236+0000 7fbb83fff700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb8415e850 con 0x7fbb8415cbd0 2023-10-08T00:14:52.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.236+0000 7fbb83fff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb840cd030 con 0x7fbb840cd6e0 2023-10-08T00:14:52.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.236+0000 7fbb837fe700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fbb8415cbd0 0x7fbb8415d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:52.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.236+0000 7fbb82ffd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb840cd6e0 0x7fbb840cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:52.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.236+0000 7fbb82ffd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb840cd6e0 0x7fbb840cbc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.99:49720/0 (socket says 172.21.15.99:49720) 2023-10-08T00:14:52.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.236+0000 7fbb837fe700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fbb8415cbd0 0x7fbb8415d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.99:53978/0 (socket says 172.21.15.99:53978) 2023-10-08T00:14:52.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.236+0000 7fbb7a7fc700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7fbb840cdab0 0x7fbb8415d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:42830/0 (socket says 172.21.15.99:42830) 2023-10-08T00:14:52.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.236+0000 7fbb82ffd700 1 -- 172.21.15.99:0/2691901378 learned_addr learned my addr 172.21.15.99:0/2691901378 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:52.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.237+0000 7fbb837fe700 1 -- 172.21.15.99:0/2691901378 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb840cd6e0 msgr2=0x7fbb840cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:14:52.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.237+0000 7fbb837fe700 1 --2- 172.21.15.99:0/2691901378 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb840cd6e0 0x7fbb840cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:52.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.237+0000 7fbb837fe700 1 -- 172.21.15.99:0/2691901378 >> v1:172.21.15.191:6789/0 conn(0x7fbb840cdab0 legacy=0x7fbb8415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:52.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.237+0000 7fbb837fe700 1 -- 172.21.15.99:0/2691901378 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb8415e290 con 0x7fbb8415cbd0 2023-10-08T00:14:52.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.237+0000 7fbb827fc700 1 -- 172.21.15.99:0/2691901378 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 328649314 0 0) 0x7fbb840cb7c0 con 0x7fbb840cdab0 2023-10-08T00:14:52.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.237+0000 7fbb82ffd700 1 --2- 172.21.15.99:0/2691901378 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb840cd6e0 0x7fbb840cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:14:52.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.237+0000 7fbb837fe700 1 --2- 172.21.15.99:0/2691901378 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb8415cbd0 0x7fbb8415d020 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c009e60 tx=0x7fbb7c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=560f3ffd2e2896a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:52.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.237+0000 7fbb827fc700 1 -- 172.21.15.99:0/2691901378 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c00e070 con 0x7fbb8415cbd0 2023-10-08T00:14:52.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.237+0000 7fbb827fc700 1 -- 172.21.15.99:0/2691901378 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb7c00bb60 con 0x7fbb8415cbd0 2023-10-08T00:14:52.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.237+0000 7fbb827fc700 1 -- 172.21.15.99:0/2691901378 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c012430 con 0x7fbb8415cbd0 2023-10-08T00:14:52.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.237+0000 7fbb83fff700 1 -- 172.21.15.99:0/2691901378 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb8415cbd0 msgr2=0x7fbb8415d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:52.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.237+0000 7fbb83fff700 1 --2- 172.21.15.99:0/2691901378 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb8415cbd0 0x7fbb8415d020 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c009e60 tx=0x7fbb7c009780 comp rx=0 tx=0).stop 2023-10-08T00:14:52.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.238+0000 7fbb83fff700 1 -- 172.21.15.99:0/2691901378 shutdown_connections 2023-10-08T00:14:52.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.238+0000 7fbb83fff700 1 --2- 172.21.15.99:0/2691901378 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb8415cbd0 0x7fbb8415d020 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:52.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.238+0000 7fbb83fff700 1 --2- 172.21.15.99:0/2691901378 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb840cd6e0 0x7fbb840cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:52.242 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.238+0000 7fbb83fff700 1 -- 172.21.15.99:0/2691901378 >> 172.21.15.99:0/2691901378 conn(0x7fbb84067660 msgr2=0x7fbb84067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:52.242 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.238+0000 7fbb83fff700 1 -- 172.21.15.99:0/2691901378 shutdown_connections 2023-10-08T00:14:52.242 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.238+0000 7fbb83fff700 1 -- 172.21.15.99:0/2691901378 wait complete. 2023-10-08T00:14:52.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.239+0000 7fbb83fff700 1 Processor -- start 2023-10-08T00:14:52.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.239+0000 7fbb83fff700 1 -- start start 2023-10-08T00:14:52.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.240+0000 7fbb83fff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb840cd6e0 0x7fbb84150840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:52.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.240+0000 7fbb83fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb840cdab0 0x7fbb8414eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:52.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.240+0000 7fbb83fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb8415cbd0 0x7fbb840c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:52.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.240+0000 7fbb83fff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb84150d80 con 0x7fbb840cdab0 2023-10-08T00:14:52.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.240+0000 7fbb83fff700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb840b4200 con 0x7fbb840cd6e0 2023-10-08T00:14:52.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.240+0000 7fbb83fff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb840b6630 con 0x7fbb8415cbd0 2023-10-08T00:14:52.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.240+0000 7fbb7a7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb840cdab0 0x7fbb8414eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:52.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.240+0000 7fbb7a7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb840cdab0 0x7fbb8414eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45864/0 (socket says 172.21.15.99:45864) 2023-10-08T00:14:52.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.240+0000 7fbb7a7fc700 1 -- 172.21.15.99:0/3236927873 learned_addr learned my addr 172.21.15.99:0/3236927873 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:14:52.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.240+0000 7fbb82ffd700 1 --2- 172.21.15.99:0/3236927873 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb840cd6e0 0x7fbb84150840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:52.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.240+0000 7fbb837fe700 1 --2- 172.21.15.99:0/3236927873 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb8415cbd0 0x7fbb840c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:52.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.240+0000 7fbb7a7fc700 1 -- 172.21.15.99:0/3236927873 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb8415cbd0 msgr2=0x7fbb840c7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:52.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.240+0000 7fbb7a7fc700 1 --2- 172.21.15.99:0/3236927873 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb8415cbd0 0x7fbb840c7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:52.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.240+0000 7fbb7a7fc700 1 -- 172.21.15.99:0/3236927873 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb840cd6e0 msgr2=0x7fbb84150840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:52.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.240+0000 7fbb7a7fc700 1 --2- 172.21.15.99:0/3236927873 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb840cd6e0 0x7fbb84150840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:52.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.240+0000 7fbb7a7fc700 1 -- 172.21.15.99:0/3236927873 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb740094f0 con 0x7fbb840cdab0 2023-10-08T00:14:52.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.240+0000 7fbb7a7fc700 1 --2- 172.21.15.99:0/3236927873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb840cdab0 0x7fbb8414eb10 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c0011e0 tx=0x7fbb6c007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:52.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.240+0000 7fbb7b7fe700 1 -- 172.21.15.99:0/3236927873 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c005a70 con 0x7fbb840cdab0 2023-10-08T00:14:52.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.240+0000 7fbb7b7fe700 1 -- 172.21.15.99:0/3236927873 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb6c005bd0 con 0x7fbb840cdab0 2023-10-08T00:14:52.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.240+0000 7fbb83fff700 1 -- 172.21.15.99:0/3236927873 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb7c009a20 con 0x7fbb840cdab0 2023-10-08T00:14:52.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.240+0000 7fbb7b7fe700 1 -- 172.21.15.99:0/3236927873 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c0070d0 con 0x7fbb840cdab0 2023-10-08T00:14:52.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.241+0000 7fbb83fff700 1 -- 172.21.15.99:0/3236927873 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb840b3e90 con 0x7fbb840cdab0 2023-10-08T00:14:52.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.241+0000 7fbb83fff700 1 -- 172.21.15.99:0/3236927873 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb840b3a10 con 0x7fbb840cdab0 2023-10-08T00:14:52.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.242+0000 7fbb7b7fe700 1 -- 172.21.15.99:0/3236927873 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbb6c007480 con 0x7fbb840cdab0 2023-10-08T00:14:52.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.242+0000 7fbb7b7fe700 1 --2- 172.21.15.99:0/3236927873 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbb64066f30 0x7fbb640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:14:52.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.242+0000 7fbb7b7fe700 1 -- 172.21.15.99:0/3236927873 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbb6c0839e0 con 0x7fbb840cdab0 2023-10-08T00:14:52.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.242+0000 7fbb82ffd700 1 --2- 172.21.15.99:0/3236927873 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbb64066f30 0x7fbb640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:14:52.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.243+0000 7fbb82ffd700 1 --2- 172.21.15.99:0/3236927873 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbb64066f30 0x7fbb640693e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fbb74006d20 tx=0x7fbb74009250 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:14:52.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.246+0000 7fbb7b7fe700 1 -- 172.21.15.99:0/3236927873 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbb840b3a10 con 0x7fbb840cdab0 2023-10-08T00:14:52.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.474+0000 7fbb83fff700 1 -- 172.21.15.99:0/3236927873 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbb840b3a10 con 0x7fbb840cdab0 2023-10-08T00:14:52.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.475+0000 7fbb7b7fe700 1 -- 172.21.15.99:0/3236927873 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4599 (secure 0 0 0) 0x7fbb840b3a10 con 0x7fbb840cdab0 2023-10-08T00:14:52.476 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:14:52.476 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5650,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/2166055423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2166055423},{"type":"v1","addr":"172.21.15.99:6834","nonce":2166055423}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5665,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/4087832300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":4087832300},{"type":"v1","addr":"172.21.15.191:6836","nonce":4087832300}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5668,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/730261712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":730261712},{"type":"v1","addr":"172.21.15.191:6837","nonce":730261712}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:14:43.671064+0000","modified":"2023-10-08T00:14:47.326922+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6039},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6039":{"gid":6039,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/104443412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":104443412},{"type":"v1","addr":"172.21.15.99:6835","nonce":104443412}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-08T00:14:52.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.477+0000 7fbb83fff700 1 -- 172.21.15.99:0/3236927873 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbb64066f30 msgr2=0x7fbb640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:52.479 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.477+0000 7fbb83fff700 1 --2- 172.21.15.99:0/3236927873 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbb64066f30 0x7fbb640693e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fbb74006d20 tx=0x7fbb74009250 comp rx=0 tx=0).stop 2023-10-08T00:14:52.479 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.477+0000 7fbb83fff700 1 -- 172.21.15.99:0/3236927873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb840cdab0 msgr2=0x7fbb8414eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:14:52.479 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.477+0000 7fbb83fff700 1 --2- 172.21.15.99:0/3236927873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb840cdab0 0x7fbb8414eb10 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c0011e0 tx=0x7fbb6c007e10 comp rx=0 tx=0).stop 2023-10-08T00:14:52.479 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.477+0000 7fbb83fff700 1 -- 172.21.15.99:0/3236927873 shutdown_connections 2023-10-08T00:14:52.479 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.477+0000 7fbb83fff700 1 --2- 172.21.15.99:0/3236927873 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbb64066f30 0x7fbb640693e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:52.479 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.477+0000 7fbb83fff700 1 --2- 172.21.15.99:0/3236927873 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb840cd6e0 0x7fbb84150840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:52.479 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.477+0000 7fbb83fff700 1 --2- 172.21.15.99:0/3236927873 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb8415cbd0 0x7fbb840c7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:52.479 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.477+0000 7fbb83fff700 1 --2- 172.21.15.99:0/3236927873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb840cdab0 0x7fbb8414eb10 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:14:52.479 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.477+0000 7fbb83fff700 1 -- 172.21.15.99:0/3236927873 >> 172.21.15.99:0/3236927873 conn(0x7fbb84067660 msgr2=0x7fbb84157780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:14:52.479 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.477+0000 7fbb83fff700 1 -- 172.21.15.99:0/3236927873 shutdown_connections 2023-10-08T00:14:52.479 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:14:52.477+0000 7fbb83fff700 1 -- 172.21.15.99:0/3236927873 wait complete. 2023-10-08T00:14:52.480 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 40 2023-10-08T00:14:52.489 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2023-10-08T00:14:52.622 INFO:teuthology.orchestra.run.smithi099.stdout:{"cluster_fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","whoami":0,"id":6039,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":8.2960326630000001,"mdsmap_epoch":40,"osdmap_epoch":86,"osdmap_epoch_barrier":86,"uptime":14.277842437} 2023-10-08T00:14:52.622 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", "fs_name": "cephfs", "id": 6039, "mdsmap_epoch": 40, "osdmap_epoch": 86, "osdmap_epoch_barrier": 86, "rank_uptime": 8.296032663, "state": "up:active", "uptime": 14.277842437, "want_state": "up:active", "whoami": 0 } 2023-10-08T00:14:52.623 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-10-08T00:14:52.752 INFO:teuthology.orchestra.run.smithi099.stdout:{"osd_mon_report_interval":"5"} 2023-10-08T00:14:52.753 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-10-08T00:14:52.753 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2023-10-08T00:14:52.876 INFO:teuthology.orchestra.run.smithi099.stdout:{"mon_tick_interval":"5"} 2023-10-08T00:14:52.876 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2023-10-08T00:14:52.876 INFO:tasks.cephfs.test_full:pool capacity 557098560, 531MB should be enough to fill it 2023-10-08T00:14:52.876 DEBUG:teuthology.orchestra.run.smithi099:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-08T00:14:52.876 DEBUG:teuthology.orchestra.run.smithi099:> import time 2023-10-08T00:14:52.876 DEBUG:teuthology.orchestra.run.smithi099:> import datetime 2023-10-08T00:14:52.876 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2023-10-08T00:14:52.877 DEBUG:teuthology.orchestra.run.smithi099:> import os 2023-10-08T00:14:52.877 DEBUG:teuthology.orchestra.run.smithi099:> 2023-10-08T00:14:52.877 DEBUG:teuthology.orchestra.run.smithi099:> # Write some buffered data through before going full, all should be well 2023-10-08T00:14:52.877 DEBUG:teuthology.orchestra.run.smithi099:> print("writing some data through which we expect to succeed") 2023-10-08T00:14:52.877 DEBUG:teuthology.orchestra.run.smithi099:> bytes = 0 2023-10-08T00:14:52.877 DEBUG:teuthology.orchestra.run.smithi099:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2023-10-08T00:14:52.877 DEBUG:teuthology.orchestra.run.smithi099:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2023-10-08T00:14:52.877 DEBUG:teuthology.orchestra.run.smithi099:> bytes += os.write(f, b'"'"'a'"'"' * 4096) 2023-10-08T00:14:52.877 DEBUG:teuthology.orchestra.run.smithi099:> os.fsync(f) 2023-10-08T00:14:52.877 DEBUG:teuthology.orchestra.run.smithi099:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2023-10-08T00:14:52.877 DEBUG:teuthology.orchestra.run.smithi099:> 2023-10-08T00:14:52.877 DEBUG:teuthology.orchestra.run.smithi099:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2023-10-08T00:14:52.877 DEBUG:teuthology.orchestra.run.smithi099:> # writing until we see an error from fsync. As long as we'"'"'re doing 2023-10-08T00:14:52.878 DEBUG:teuthology.orchestra.run.smithi099:> # buffered IO, the error should always only appear from fsync and not 2023-10-08T00:14:52.878 DEBUG:teuthology.orchestra.run.smithi099:> # from write 2023-10-08T00:14:52.878 DEBUG:teuthology.orchestra.run.smithi099:> full = False 2023-10-08T00:14:52.878 DEBUG:teuthology.orchestra.run.smithi099:> 2023-10-08T00:14:52.878 DEBUG:teuthology.orchestra.run.smithi099:> for n in range(0, int(531 * 1.1)): 2023-10-08T00:14:52.878 DEBUG:teuthology.orchestra.run.smithi099:> try: 2023-10-08T00:14:52.878 DEBUG:teuthology.orchestra.run.smithi099:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2023-10-08T00:14:52.878 DEBUG:teuthology.orchestra.run.smithi099:> print("wrote bytes via buffered write, moving on to fsync") 2023-10-08T00:14:52.878 DEBUG:teuthology.orchestra.run.smithi099:> except OSError as e: 2023-10-08T00:14:52.878 DEBUG:teuthology.orchestra.run.smithi099:> if False: 2023-10-08T00:14:52.878 DEBUG:teuthology.orchestra.run.smithi099:> print("Unexpected error %s from write() instead of fsync()" % e) 2023-10-08T00:14:52.878 DEBUG:teuthology.orchestra.run.smithi099:> raise 2023-10-08T00:14:52.878 DEBUG:teuthology.orchestra.run.smithi099:> else: 2023-10-08T00:14:52.879 DEBUG:teuthology.orchestra.run.smithi099:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2023-10-08T00:14:52.879 DEBUG:teuthology.orchestra.run.smithi099:> full = True 2023-10-08T00:14:52.879 DEBUG:teuthology.orchestra.run.smithi099:> break 2023-10-08T00:14:52.879 DEBUG:teuthology.orchestra.run.smithi099:> 2023-10-08T00:14:52.879 DEBUG:teuthology.orchestra.run.smithi099:> try: 2023-10-08T00:14:52.879 DEBUG:teuthology.orchestra.run.smithi099:> os.fsync(f) 2023-10-08T00:14:52.879 DEBUG:teuthology.orchestra.run.smithi099:> print("fsync'"'"'ed successfully") 2023-10-08T00:14:52.879 DEBUG:teuthology.orchestra.run.smithi099:> except OSError as e: 2023-10-08T00:14:52.879 DEBUG:teuthology.orchestra.run.smithi099:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2023-10-08T00:14:52.879 DEBUG:teuthology.orchestra.run.smithi099:> full = True 2023-10-08T00:14:52.879 DEBUG:teuthology.orchestra.run.smithi099:> break 2023-10-08T00:14:52.879 DEBUG:teuthology.orchestra.run.smithi099:> else: 2023-10-08T00:14:52.879 DEBUG:teuthology.orchestra.run.smithi099:> print("Not full yet after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2023-10-08T00:14:52.880 DEBUG:teuthology.orchestra.run.smithi099:> 2023-10-08T00:14:52.880 DEBUG:teuthology.orchestra.run.smithi099:> if n > 531 * 0.9: 2023-10-08T00:14:52.880 DEBUG:teuthology.orchestra.run.smithi099:> # Be cautious in the last region where we expect to hit 2023-10-08T00:14:52.880 DEBUG:teuthology.orchestra.run.smithi099:> # the full condition, so that we don'"'"'t overshoot too dramatically 2023-10-08T00:14:52.880 DEBUG:teuthology.orchestra.run.smithi099:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2023-10-08T00:14:52.880 DEBUG:teuthology.orchestra.run.smithi099:> time.sleep(15.0) 2023-10-08T00:14:52.880 DEBUG:teuthology.orchestra.run.smithi099:> 2023-10-08T00:14:52.880 DEBUG:teuthology.orchestra.run.smithi099:> if not full: 2023-10-08T00:14:52.880 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Failed to reach fullness after writing %d bytes" % bytes) 2023-10-08T00:14:52.881 DEBUG:teuthology.orchestra.run.smithi099:> 2023-10-08T00:14:52.881 DEBUG:teuthology.orchestra.run.smithi099:> # close() should not raise an error because we already caught it in 2023-10-08T00:14:52.881 DEBUG:teuthology.orchestra.run.smithi099:> # fsync. There shouldn'"'"'t have been any more writeback errors 2023-10-08T00:14:52.881 DEBUG:teuthology.orchestra.run.smithi099:> # since then because all IOs got cancelled on the full flag. 2023-10-08T00:14:52.881 DEBUG:teuthology.orchestra.run.smithi099:> print("calling close") 2023-10-08T00:14:52.881 DEBUG:teuthology.orchestra.run.smithi099:> os.close(f) 2023-10-08T00:14:52.881 DEBUG:teuthology.orchestra.run.smithi099:> print("close() did not raise error") 2023-10-08T00:14:52.881 DEBUG:teuthology.orchestra.run.smithi099:> 2023-10-08T00:14:52.881 DEBUG:teuthology.orchestra.run.smithi099:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2023-10-08T00:14:52.881 DEBUG:teuthology.orchestra.run.smithi099:> ' 2023-10-08T00:14:53.058 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:14:53.061 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:14:53.551 INFO:teuthology.orchestra.run.smithi099.stdout:writing some data through which we expect to succeed 2023-10-08T00:14:53.551 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed data successfully, will now attempt to fill fs 2023-10-08T00:14:53.551 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.551 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.551 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 1.00 MB 2023-10-08T00:14:53.551 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.551 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.552 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 2.00 MB 2023-10-08T00:14:53.552 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.552 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.552 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 3.00 MB 2023-10-08T00:14:53.552 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.552 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.552 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 4.00 MB 2023-10-08T00:14:53.552 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.552 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.552 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 5.00 MB 2023-10-08T00:14:53.552 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.552 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.553 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 6.00 MB 2023-10-08T00:14:53.553 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.553 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.553 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 7.00 MB 2023-10-08T00:14:53.553 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.553 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.553 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 8.00 MB 2023-10-08T00:14:53.553 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.553 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.553 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 9.00 MB 2023-10-08T00:14:53.553 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.554 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.554 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 10.00 MB 2023-10-08T00:14:53.554 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.554 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.554 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 11.00 MB 2023-10-08T00:14:53.554 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.554 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.554 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 12.00 MB 2023-10-08T00:14:53.554 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.554 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.554 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 13.00 MB 2023-10-08T00:14:53.555 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.555 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.555 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 14.00 MB 2023-10-08T00:14:53.555 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.555 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.555 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 15.00 MB 2023-10-08T00:14:53.555 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.555 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.555 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 16.00 MB 2023-10-08T00:14:53.555 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.555 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.556 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 17.00 MB 2023-10-08T00:14:53.556 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.556 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.556 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 18.00 MB 2023-10-08T00:14:53.556 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.556 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.556 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 19.00 MB 2023-10-08T00:14:53.556 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.556 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.556 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 20.00 MB 2023-10-08T00:14:53.556 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.556 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.557 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 21.00 MB 2023-10-08T00:14:53.557 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.557 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.557 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 22.00 MB 2023-10-08T00:14:53.557 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.557 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.557 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 23.00 MB 2023-10-08T00:14:53.557 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.557 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.557 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 24.00 MB 2023-10-08T00:14:53.557 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.557 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.558 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 25.00 MB 2023-10-08T00:14:53.558 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.558 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.558 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 26.00 MB 2023-10-08T00:14:53.558 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.558 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.558 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 27.00 MB 2023-10-08T00:14:53.558 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.558 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.558 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 28.00 MB 2023-10-08T00:14:53.558 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.559 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.559 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 29.00 MB 2023-10-08T00:14:53.559 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.559 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.559 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 30.00 MB 2023-10-08T00:14:53.559 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.559 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.559 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 31.00 MB 2023-10-08T00:14:53.559 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.559 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.559 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 32.00 MB 2023-10-08T00:14:53.559 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.560 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.560 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 33.00 MB 2023-10-08T00:14:53.560 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.560 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.560 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 34.00 MB 2023-10-08T00:14:53.560 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.560 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.560 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 35.00 MB 2023-10-08T00:14:53.560 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.560 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.560 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 36.00 MB 2023-10-08T00:14:53.560 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.561 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.561 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 37.00 MB 2023-10-08T00:14:53.561 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.561 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.561 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 38.00 MB 2023-10-08T00:14:53.561 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.561 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.561 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 39.00 MB 2023-10-08T00:14:53.561 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.561 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.561 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 40.00 MB 2023-10-08T00:14:53.561 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.562 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.562 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 41.00 MB 2023-10-08T00:14:53.562 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.562 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.562 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 42.00 MB 2023-10-08T00:14:53.562 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.562 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.562 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 43.00 MB 2023-10-08T00:14:53.562 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.562 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.562 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 44.00 MB 2023-10-08T00:14:53.563 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.563 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.563 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 45.00 MB 2023-10-08T00:14:53.563 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.563 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.563 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 46.00 MB 2023-10-08T00:14:53.563 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.563 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.563 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 47.00 MB 2023-10-08T00:14:53.563 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.563 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.563 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 48.00 MB 2023-10-08T00:14:53.564 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.564 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.564 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 49.00 MB 2023-10-08T00:14:53.564 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.564 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.564 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 50.00 MB 2023-10-08T00:14:53.564 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.564 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.564 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 51.00 MB 2023-10-08T00:14:53.564 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.564 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.564 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 52.00 MB 2023-10-08T00:14:53.565 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.565 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.565 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 53.00 MB 2023-10-08T00:14:53.565 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.565 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.565 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 54.00 MB 2023-10-08T00:14:53.565 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.565 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.565 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 55.00 MB 2023-10-08T00:14:53.565 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.565 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.565 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 56.00 MB 2023-10-08T00:14:53.565 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.566 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.566 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 57.00 MB 2023-10-08T00:14:53.566 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.566 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.566 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 58.00 MB 2023-10-08T00:14:53.566 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.566 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.566 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 59.00 MB 2023-10-08T00:14:53.567 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.567 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.567 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 60.00 MB 2023-10-08T00:14:53.567 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.567 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.567 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 61.00 MB 2023-10-08T00:14:53.567 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.567 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.567 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 62.00 MB 2023-10-08T00:14:53.567 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.568 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.568 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 63.00 MB 2023-10-08T00:14:53.568 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.568 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.568 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 64.00 MB 2023-10-08T00:14:53.568 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.568 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.568 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 65.00 MB 2023-10-08T00:14:53.568 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.568 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.568 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 66.00 MB 2023-10-08T00:14:53.569 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.569 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.569 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 67.00 MB 2023-10-08T00:14:53.569 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.569 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.569 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 68.00 MB 2023-10-08T00:14:53.569 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.569 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.569 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 69.00 MB 2023-10-08T00:14:53.570 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.570 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.570 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 70.00 MB 2023-10-08T00:14:53.570 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.570 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.570 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 71.00 MB 2023-10-08T00:14:53.570 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.570 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.570 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 72.00 MB 2023-10-08T00:14:53.570 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.570 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.570 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 73.00 MB 2023-10-08T00:14:53.571 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.571 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.571 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 74.00 MB 2023-10-08T00:14:53.571 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.571 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.571 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 75.00 MB 2023-10-08T00:14:53.571 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.571 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.571 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 76.00 MB 2023-10-08T00:14:53.571 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.571 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.571 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 77.00 MB 2023-10-08T00:14:53.572 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.572 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.572 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 78.00 MB 2023-10-08T00:14:53.572 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.572 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.572 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 79.00 MB 2023-10-08T00:14:53.572 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.572 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.572 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 80.00 MB 2023-10-08T00:14:53.760 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.760 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.760 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 81.00 MB 2023-10-08T00:14:53.760 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.760 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.760 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 82.00 MB 2023-10-08T00:14:53.760 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.761 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.761 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 83.00 MB 2023-10-08T00:14:53.761 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.761 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.761 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 84.00 MB 2023-10-08T00:14:53.761 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.761 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.761 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 85.00 MB 2023-10-08T00:14:53.761 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.762 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.762 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 86.00 MB 2023-10-08T00:14:53.762 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.762 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.762 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 87.00 MB 2023-10-08T00:14:53.762 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.762 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.762 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 88.00 MB 2023-10-08T00:14:53.762 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.762 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.762 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 89.00 MB 2023-10-08T00:14:53.762 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.763 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.763 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 90.00 MB 2023-10-08T00:14:53.763 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.763 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.763 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 91.00 MB 2023-10-08T00:14:53.763 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.763 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.763 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 92.00 MB 2023-10-08T00:14:53.763 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.763 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.764 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 93.00 MB 2023-10-08T00:14:53.764 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.764 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.764 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 94.00 MB 2023-10-08T00:14:53.764 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.764 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.764 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 95.00 MB 2023-10-08T00:14:53.764 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.764 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.764 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 96.00 MB 2023-10-08T00:14:53.764 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.764 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.765 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 97.00 MB 2023-10-08T00:14:53.765 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.765 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.765 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 98.00 MB 2023-10-08T00:14:53.765 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.765 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.765 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 99.00 MB 2023-10-08T00:14:53.765 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.765 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.765 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 100.00 MB 2023-10-08T00:14:53.765 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.765 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.765 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 101.00 MB 2023-10-08T00:14:53.766 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.766 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.766 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 102.00 MB 2023-10-08T00:14:53.766 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.766 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.766 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 103.00 MB 2023-10-08T00:14:53.766 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.766 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.766 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 104.00 MB 2023-10-08T00:14:53.766 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.766 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.767 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 105.00 MB 2023-10-08T00:14:53.767 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.767 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.767 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 106.00 MB 2023-10-08T00:14:53.767 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.767 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.767 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 107.00 MB 2023-10-08T00:14:53.767 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.767 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.767 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 108.00 MB 2023-10-08T00:14:53.767 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.767 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.768 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 109.00 MB 2023-10-08T00:14:53.768 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.768 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.768 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 110.00 MB 2023-10-08T00:14:53.768 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.768 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.768 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 111.00 MB 2023-10-08T00:14:53.768 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.768 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.768 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 112.00 MB 2023-10-08T00:14:53.768 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.768 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.769 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 113.00 MB 2023-10-08T00:14:53.769 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.769 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.769 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 114.00 MB 2023-10-08T00:14:53.769 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.769 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.769 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 115.00 MB 2023-10-08T00:14:53.769 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.769 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.769 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 116.00 MB 2023-10-08T00:14:53.769 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.769 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.770 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 117.00 MB 2023-10-08T00:14:53.770 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.770 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.770 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 118.00 MB 2023-10-08T00:14:53.770 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.770 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.770 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 119.00 MB 2023-10-08T00:14:53.770 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.770 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.770 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 120.00 MB 2023-10-08T00:14:53.771 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.771 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.771 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 121.00 MB 2023-10-08T00:14:53.771 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.771 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.771 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 122.00 MB 2023-10-08T00:14:53.771 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.771 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.771 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 123.00 MB 2023-10-08T00:14:53.772 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.772 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.772 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 124.00 MB 2023-10-08T00:14:53.772 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.772 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.772 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 125.00 MB 2023-10-08T00:14:53.772 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.772 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.772 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 126.00 MB 2023-10-08T00:14:53.772 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.772 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.772 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 127.00 MB 2023-10-08T00:14:53.773 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.773 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.773 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 128.00 MB 2023-10-08T00:14:53.773 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.773 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.773 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 129.00 MB 2023-10-08T00:14:53.773 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.773 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.773 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 130.00 MB 2023-10-08T00:14:53.773 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.773 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.773 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 131.00 MB 2023-10-08T00:14:53.774 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.774 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.774 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 132.00 MB 2023-10-08T00:14:53.774 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.774 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.774 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 133.00 MB 2023-10-08T00:14:53.774 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.774 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.774 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 134.00 MB 2023-10-08T00:14:53.774 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.774 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.774 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 135.00 MB 2023-10-08T00:14:53.775 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.775 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.775 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 136.00 MB 2023-10-08T00:14:53.775 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.775 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.775 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 137.00 MB 2023-10-08T00:14:53.775 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.775 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.775 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 138.00 MB 2023-10-08T00:14:53.775 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.776 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.776 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 139.00 MB 2023-10-08T00:14:53.776 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.776 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.776 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 140.00 MB 2023-10-08T00:14:53.776 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.776 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.776 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 141.00 MB 2023-10-08T00:14:53.776 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.776 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.777 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 142.00 MB 2023-10-08T00:14:53.777 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.777 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.777 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 143.00 MB 2023-10-08T00:14:53.777 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.777 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.777 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 144.00 MB 2023-10-08T00:14:53.777 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.777 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.777 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 145.00 MB 2023-10-08T00:14:53.777 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.777 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.778 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 146.00 MB 2023-10-08T00:14:53.778 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.778 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.778 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 147.00 MB 2023-10-08T00:14:53.778 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.778 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.778 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 148.00 MB 2023-10-08T00:14:53.778 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.778 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.778 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 149.00 MB 2023-10-08T00:14:53.778 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.779 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.779 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 150.00 MB 2023-10-08T00:14:53.779 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.779 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.779 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 151.00 MB 2023-10-08T00:14:53.779 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.779 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.779 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 152.00 MB 2023-10-08T00:14:53.779 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.779 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.779 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 153.00 MB 2023-10-08T00:14:53.779 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.780 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.780 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 154.00 MB 2023-10-08T00:14:53.780 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.780 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.780 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 155.00 MB 2023-10-08T00:14:53.780 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.780 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.780 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 156.00 MB 2023-10-08T00:14:53.780 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.780 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.780 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 157.00 MB 2023-10-08T00:14:53.780 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.781 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.781 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 158.00 MB 2023-10-08T00:14:53.781 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.781 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.781 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 159.00 MB 2023-10-08T00:14:53.781 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.781 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.781 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 160.00 MB 2023-10-08T00:14:53.781 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.781 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.781 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 161.00 MB 2023-10-08T00:14:53.957 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.957 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.957 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 162.00 MB 2023-10-08T00:14:53.957 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.957 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.958 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 163.00 MB 2023-10-08T00:14:53.958 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.958 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.958 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 164.00 MB 2023-10-08T00:14:53.958 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.958 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.958 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 165.00 MB 2023-10-08T00:14:53.958 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.958 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.958 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 166.00 MB 2023-10-08T00:14:53.958 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.959 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.959 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 167.00 MB 2023-10-08T00:14:53.959 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.959 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.959 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 168.00 MB 2023-10-08T00:14:53.959 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.959 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.959 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 169.00 MB 2023-10-08T00:14:53.959 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.959 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.959 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 170.00 MB 2023-10-08T00:14:53.960 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.960 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.960 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 171.00 MB 2023-10-08T00:14:53.960 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.960 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.960 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 172.00 MB 2023-10-08T00:14:53.960 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.960 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.960 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 173.00 MB 2023-10-08T00:14:53.960 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.960 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.960 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 174.00 MB 2023-10-08T00:14:53.961 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.961 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.961 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 175.00 MB 2023-10-08T00:14:53.961 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.961 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.961 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 176.00 MB 2023-10-08T00:14:53.961 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.961 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.961 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 177.00 MB 2023-10-08T00:14:53.961 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.961 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.961 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 178.00 MB 2023-10-08T00:14:53.961 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.962 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.962 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 179.00 MB 2023-10-08T00:14:53.962 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.962 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.962 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 180.00 MB 2023-10-08T00:14:53.962 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.962 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.962 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 181.00 MB 2023-10-08T00:14:53.962 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.962 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.962 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 182.00 MB 2023-10-08T00:14:53.963 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.963 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.963 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 183.00 MB 2023-10-08T00:14:53.963 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.963 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.963 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 184.00 MB 2023-10-08T00:14:53.963 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.963 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.963 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 185.00 MB 2023-10-08T00:14:53.963 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.963 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.963 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 186.00 MB 2023-10-08T00:14:53.964 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.964 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.964 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 187.00 MB 2023-10-08T00:14:53.964 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.964 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.964 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 188.00 MB 2023-10-08T00:14:53.964 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.964 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.964 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 189.00 MB 2023-10-08T00:14:53.964 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.964 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.964 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 190.00 MB 2023-10-08T00:14:53.965 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.965 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.965 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 191.00 MB 2023-10-08T00:14:53.965 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.965 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.965 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 192.00 MB 2023-10-08T00:14:53.965 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.965 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.965 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 193.00 MB 2023-10-08T00:14:53.965 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.965 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.965 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 194.00 MB 2023-10-08T00:14:53.965 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.966 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.966 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 195.00 MB 2023-10-08T00:14:53.966 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.966 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.966 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 196.00 MB 2023-10-08T00:14:53.966 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.966 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.966 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 197.00 MB 2023-10-08T00:14:53.967 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.967 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.967 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 198.00 MB 2023-10-08T00:14:53.967 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.967 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.967 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 199.00 MB 2023-10-08T00:14:53.967 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.967 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.967 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 200.00 MB 2023-10-08T00:14:53.967 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.967 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.968 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 201.00 MB 2023-10-08T00:14:53.968 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.968 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.968 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 202.00 MB 2023-10-08T00:14:53.969 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.969 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.969 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 203.00 MB 2023-10-08T00:14:53.969 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.969 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.969 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 204.00 MB 2023-10-08T00:14:53.969 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.969 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.969 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 205.00 MB 2023-10-08T00:14:53.970 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.970 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.970 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 206.00 MB 2023-10-08T00:14:53.970 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.970 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.970 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 207.00 MB 2023-10-08T00:14:53.970 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.970 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.970 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 208.00 MB 2023-10-08T00:14:53.970 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.970 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.971 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 209.00 MB 2023-10-08T00:14:53.971 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.971 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.971 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 210.00 MB 2023-10-08T00:14:53.971 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.971 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.971 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 211.00 MB 2023-10-08T00:14:53.971 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.971 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.971 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 212.00 MB 2023-10-08T00:14:53.972 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.972 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.972 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 213.00 MB 2023-10-08T00:14:53.972 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.972 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.972 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 214.00 MB 2023-10-08T00:14:53.972 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.972 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.972 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 215.00 MB 2023-10-08T00:14:53.972 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.972 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.972 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 216.00 MB 2023-10-08T00:14:53.973 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.973 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.973 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 217.00 MB 2023-10-08T00:14:53.973 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.973 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.973 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 218.00 MB 2023-10-08T00:14:53.973 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.973 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.973 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 219.00 MB 2023-10-08T00:14:53.973 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.973 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.974 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 220.00 MB 2023-10-08T00:14:53.974 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.974 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.974 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 221.00 MB 2023-10-08T00:14:53.974 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.974 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.974 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 222.00 MB 2023-10-08T00:14:53.974 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.975 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.975 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 223.00 MB 2023-10-08T00:14:53.975 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.975 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.975 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 224.00 MB 2023-10-08T00:14:53.975 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.975 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.975 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 225.00 MB 2023-10-08T00:14:53.975 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.975 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.976 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 226.00 MB 2023-10-08T00:14:53.976 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.976 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.976 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 227.00 MB 2023-10-08T00:14:53.976 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.976 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.976 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 228.00 MB 2023-10-08T00:14:53.976 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.977 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.977 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 229.00 MB 2023-10-08T00:14:53.977 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.977 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.977 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 230.00 MB 2023-10-08T00:14:53.977 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.977 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.977 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 231.00 MB 2023-10-08T00:14:53.978 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.978 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.978 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 232.00 MB 2023-10-08T00:14:53.978 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.978 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.978 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 233.00 MB 2023-10-08T00:14:53.978 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.978 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.978 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 234.00 MB 2023-10-08T00:14:53.978 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.978 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.979 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 235.00 MB 2023-10-08T00:14:53.979 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.979 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.979 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 236.00 MB 2023-10-08T00:14:53.979 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.979 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.979 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 237.00 MB 2023-10-08T00:14:53.979 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.979 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.979 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 238.00 MB 2023-10-08T00:14:53.979 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.979 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.980 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 239.00 MB 2023-10-08T00:14:53.980 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.980 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.980 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 240.00 MB 2023-10-08T00:14:53.980 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:53.980 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:53.980 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 241.00 MB 2023-10-08T00:14:54.152 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.153 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.153 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 242.00 MB 2023-10-08T00:14:54.153 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.153 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.153 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 243.00 MB 2023-10-08T00:14:54.153 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.153 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.153 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 244.00 MB 2023-10-08T00:14:54.153 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.153 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.153 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 245.00 MB 2023-10-08T00:14:54.153 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.154 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.154 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 246.00 MB 2023-10-08T00:14:54.154 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.154 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.154 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 247.00 MB 2023-10-08T00:14:54.154 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.154 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.154 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 248.00 MB 2023-10-08T00:14:54.154 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.154 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.154 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 249.00 MB 2023-10-08T00:14:54.154 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.155 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.155 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 250.00 MB 2023-10-08T00:14:54.155 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.155 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.155 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 251.00 MB 2023-10-08T00:14:54.155 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.155 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.155 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 252.00 MB 2023-10-08T00:14:54.155 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.155 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.155 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 253.00 MB 2023-10-08T00:14:54.155 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.156 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.156 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 254.00 MB 2023-10-08T00:14:54.156 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.156 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.156 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 255.00 MB 2023-10-08T00:14:54.156 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.156 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.156 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 256.00 MB 2023-10-08T00:14:54.156 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.156 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.156 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 257.00 MB 2023-10-08T00:14:54.157 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.157 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.157 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 258.00 MB 2023-10-08T00:14:54.157 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.157 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.157 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 259.00 MB 2023-10-08T00:14:54.157 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.157 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.157 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 260.00 MB 2023-10-08T00:14:54.157 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.157 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.157 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 261.00 MB 2023-10-08T00:14:54.158 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.158 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.158 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 262.00 MB 2023-10-08T00:14:54.158 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.158 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.158 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 263.00 MB 2023-10-08T00:14:54.158 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.158 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.158 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 264.00 MB 2023-10-08T00:14:54.158 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.158 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.158 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 265.00 MB 2023-10-08T00:14:54.159 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.159 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.159 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 266.00 MB 2023-10-08T00:14:54.159 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.159 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.159 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 267.00 MB 2023-10-08T00:14:54.159 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.159 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.159 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 268.00 MB 2023-10-08T00:14:54.159 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.159 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.159 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 269.00 MB 2023-10-08T00:14:54.160 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.160 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.160 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 270.00 MB 2023-10-08T00:14:54.160 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.160 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.160 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 271.00 MB 2023-10-08T00:14:54.160 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.160 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.160 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 272.00 MB 2023-10-08T00:14:54.160 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.160 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.160 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 273.00 MB 2023-10-08T00:14:54.161 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.161 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.161 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 274.00 MB 2023-10-08T00:14:54.161 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.161 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.161 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 275.00 MB 2023-10-08T00:14:54.161 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.161 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.161 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 276.00 MB 2023-10-08T00:14:54.161 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.161 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.161 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 277.00 MB 2023-10-08T00:14:54.161 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.162 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.162 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 278.00 MB 2023-10-08T00:14:54.162 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.162 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.162 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 279.00 MB 2023-10-08T00:14:54.162 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.162 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.162 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 280.00 MB 2023-10-08T00:14:54.162 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.162 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.162 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 281.00 MB 2023-10-08T00:14:54.162 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.163 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.163 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 282.00 MB 2023-10-08T00:14:54.163 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.163 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.163 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 283.00 MB 2023-10-08T00:14:54.163 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.163 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.163 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 284.00 MB 2023-10-08T00:14:54.163 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.163 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.163 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 285.00 MB 2023-10-08T00:14:54.163 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.163 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.164 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 286.00 MB 2023-10-08T00:14:54.164 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.164 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.164 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 287.00 MB 2023-10-08T00:14:54.164 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.164 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.164 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 288.00 MB 2023-10-08T00:14:54.164 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.164 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.164 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 289.00 MB 2023-10-08T00:14:54.164 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.164 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.164 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 290.00 MB 2023-10-08T00:14:54.165 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.165 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.165 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 291.00 MB 2023-10-08T00:14:54.165 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.165 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.165 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 292.00 MB 2023-10-08T00:14:54.165 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.165 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.165 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 293.00 MB 2023-10-08T00:14:54.165 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.165 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.165 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 294.00 MB 2023-10-08T00:14:54.166 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.166 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.166 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 295.00 MB 2023-10-08T00:14:54.166 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.166 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.166 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 296.00 MB 2023-10-08T00:14:54.166 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.166 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.166 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 297.00 MB 2023-10-08T00:14:54.166 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.166 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.166 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 298.00 MB 2023-10-08T00:14:54.167 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.167 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.167 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 299.00 MB 2023-10-08T00:14:54.167 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.167 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.167 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 300.00 MB 2023-10-08T00:14:54.167 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.167 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.167 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 301.00 MB 2023-10-08T00:14:54.167 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.167 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.167 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 302.00 MB 2023-10-08T00:14:54.167 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.168 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.168 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 303.00 MB 2023-10-08T00:14:54.168 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.168 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.168 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 304.00 MB 2023-10-08T00:14:54.168 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.168 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.168 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 305.00 MB 2023-10-08T00:14:54.168 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.168 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.168 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 306.00 MB 2023-10-08T00:14:54.168 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.169 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.169 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 307.00 MB 2023-10-08T00:14:54.169 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.169 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.169 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 308.00 MB 2023-10-08T00:14:54.169 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.169 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.169 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 309.00 MB 2023-10-08T00:14:54.169 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.169 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.169 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 310.00 MB 2023-10-08T00:14:54.169 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.169 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.170 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 311.00 MB 2023-10-08T00:14:54.170 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.170 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.170 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 312.00 MB 2023-10-08T00:14:54.170 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.170 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.170 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 313.00 MB 2023-10-08T00:14:54.170 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.170 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.170 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 314.00 MB 2023-10-08T00:14:54.170 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.170 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.171 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 315.00 MB 2023-10-08T00:14:54.171 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.171 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.171 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 316.00 MB 2023-10-08T00:14:54.171 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.171 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.171 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 317.00 MB 2023-10-08T00:14:54.171 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.171 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.171 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 318.00 MB 2023-10-08T00:14:54.171 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.171 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.172 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 319.00 MB 2023-10-08T00:14:54.172 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.172 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.172 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 320.00 MB 2023-10-08T00:14:54.172 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.172 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.172 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 321.00 MB 2023-10-08T00:14:54.172 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.172 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.342 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 322.00 MB 2023-10-08T00:14:54.343 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.343 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.343 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 323.00 MB 2023-10-08T00:14:54.343 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.343 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.343 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 324.00 MB 2023-10-08T00:14:54.343 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.343 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.343 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 325.00 MB 2023-10-08T00:14:54.343 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.344 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.344 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 326.00 MB 2023-10-08T00:14:54.344 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.344 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.344 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 327.00 MB 2023-10-08T00:14:54.344 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.344 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.344 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 328.00 MB 2023-10-08T00:14:54.344 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.344 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.344 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 329.00 MB 2023-10-08T00:14:54.344 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.345 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.345 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 330.00 MB 2023-10-08T00:14:54.345 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.345 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.345 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 331.00 MB 2023-10-08T00:14:54.345 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.345 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.345 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 332.00 MB 2023-10-08T00:14:54.345 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.345 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.345 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 333.00 MB 2023-10-08T00:14:54.345 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.346 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.346 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 334.00 MB 2023-10-08T00:14:54.346 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.346 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.346 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 335.00 MB 2023-10-08T00:14:54.346 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.346 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.346 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 336.00 MB 2023-10-08T00:14:54.346 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.346 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.346 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 337.00 MB 2023-10-08T00:14:54.347 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.347 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.347 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 338.00 MB 2023-10-08T00:14:54.347 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.347 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.347 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 339.00 MB 2023-10-08T00:14:54.347 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.347 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.347 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 340.00 MB 2023-10-08T00:14:54.347 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.347 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.347 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 341.00 MB 2023-10-08T00:14:54.348 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.348 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.348 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 342.00 MB 2023-10-08T00:14:54.348 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.348 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.348 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 343.00 MB 2023-10-08T00:14:54.348 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.348 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.348 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 344.00 MB 2023-10-08T00:14:54.348 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.348 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.348 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 345.00 MB 2023-10-08T00:14:54.349 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.349 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.349 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 346.00 MB 2023-10-08T00:14:54.349 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.349 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.349 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 347.00 MB 2023-10-08T00:14:54.349 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.349 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.349 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 348.00 MB 2023-10-08T00:14:54.349 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.349 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.349 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 349.00 MB 2023-10-08T00:14:54.350 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.350 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.350 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 350.00 MB 2023-10-08T00:14:54.350 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.350 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.350 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 351.00 MB 2023-10-08T00:14:54.350 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.350 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.350 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 352.00 MB 2023-10-08T00:14:54.350 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.350 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.350 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 353.00 MB 2023-10-08T00:14:54.351 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.351 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.351 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 354.00 MB 2023-10-08T00:14:54.351 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.351 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.351 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 355.00 MB 2023-10-08T00:14:54.351 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.351 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.351 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 356.00 MB 2023-10-08T00:14:54.351 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.351 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.352 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 357.00 MB 2023-10-08T00:14:54.352 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.352 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.352 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 358.00 MB 2023-10-08T00:14:54.352 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.352 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.352 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 359.00 MB 2023-10-08T00:14:54.352 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.352 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.352 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 360.00 MB 2023-10-08T00:14:54.352 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.352 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.353 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 361.00 MB 2023-10-08T00:14:54.353 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.353 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.353 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 362.00 MB 2023-10-08T00:14:54.353 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.353 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.353 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 363.00 MB 2023-10-08T00:14:54.353 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.353 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.353 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 364.00 MB 2023-10-08T00:14:54.353 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.353 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.354 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 365.00 MB 2023-10-08T00:14:54.354 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.354 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.354 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 366.00 MB 2023-10-08T00:14:54.354 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.354 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.354 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 367.00 MB 2023-10-08T00:14:54.354 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.354 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.354 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 368.00 MB 2023-10-08T00:14:54.354 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.355 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.355 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 369.00 MB 2023-10-08T00:14:54.355 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.355 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.355 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 370.00 MB 2023-10-08T00:14:54.355 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.355 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.355 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 371.00 MB 2023-10-08T00:14:54.355 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.355 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.355 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 372.00 MB 2023-10-08T00:14:54.355 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.356 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.356 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 373.00 MB 2023-10-08T00:14:54.356 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.356 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.356 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 374.00 MB 2023-10-08T00:14:54.356 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.356 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.356 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 375.00 MB 2023-10-08T00:14:54.356 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.356 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.356 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 376.00 MB 2023-10-08T00:14:54.356 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.357 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.357 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 377.00 MB 2023-10-08T00:14:54.357 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.357 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.357 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 378.00 MB 2023-10-08T00:14:54.357 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.357 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.357 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 379.00 MB 2023-10-08T00:14:54.357 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.357 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.357 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 380.00 MB 2023-10-08T00:14:54.357 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.358 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.358 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 381.00 MB 2023-10-08T00:14:54.358 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.358 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.358 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 382.00 MB 2023-10-08T00:14:54.358 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.358 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.358 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 383.00 MB 2023-10-08T00:14:54.358 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.358 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.358 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 384.00 MB 2023-10-08T00:14:54.358 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.359 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.359 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 385.00 MB 2023-10-08T00:14:54.359 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.359 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.359 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 386.00 MB 2023-10-08T00:14:54.359 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.359 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.359 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 387.00 MB 2023-10-08T00:14:54.359 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.359 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.359 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 388.00 MB 2023-10-08T00:14:54.359 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.360 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.360 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 389.00 MB 2023-10-08T00:14:54.360 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.360 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.360 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 390.00 MB 2023-10-08T00:14:54.360 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.360 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.360 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 391.00 MB 2023-10-08T00:14:54.360 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.360 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.360 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 392.00 MB 2023-10-08T00:14:54.360 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.361 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.361 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 393.00 MB 2023-10-08T00:14:54.361 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.361 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.361 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 394.00 MB 2023-10-08T00:14:54.361 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.361 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.361 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 395.00 MB 2023-10-08T00:14:54.361 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.361 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.361 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 396.00 MB 2023-10-08T00:14:54.361 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.362 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.362 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 397.00 MB 2023-10-08T00:14:54.362 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.362 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.362 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 398.00 MB 2023-10-08T00:14:54.362 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.362 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.362 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 399.00 MB 2023-10-08T00:14:54.362 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.362 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.362 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 400.00 MB 2023-10-08T00:14:54.362 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.363 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.363 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 401.00 MB 2023-10-08T00:14:54.363 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:14:54.363 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:14:54.363 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 402.00 MB 2023-10-08T00:14:56.346 INFO:tasks.ceph.mon.a.smithi099.stderr:2023-10-08T00:14:56.345+0000 7f6c637a7700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 full osd(s) (OSD_FULL) 2023-10-08T00:15:06.091 INFO:tasks.ceph.mon.a.smithi099.stderr:2023-10-08T00:15:06.091+0000 7f6c637a7700 -1 log_channel(cluster) log [ERR] : Health check update: 2 full osd(s) (OSD_FULL) 2023-10-08T00:15:23.156 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:15:23.159 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:15:24.593 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.594 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.594 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 403.00 MB 2023-10-08T00:15:24.594 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.594 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.594 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 404.00 MB 2023-10-08T00:15:24.595 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.595 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.595 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 405.00 MB 2023-10-08T00:15:24.595 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.595 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.595 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 406.00 MB 2023-10-08T00:15:24.595 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.596 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.596 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 407.00 MB 2023-10-08T00:15:24.596 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.596 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.596 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 408.00 MB 2023-10-08T00:15:24.596 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.597 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.597 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 409.00 MB 2023-10-08T00:15:24.597 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.597 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.597 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 410.00 MB 2023-10-08T00:15:24.598 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.598 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.598 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 411.00 MB 2023-10-08T00:15:24.598 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.598 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.598 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 412.00 MB 2023-10-08T00:15:24.599 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.599 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.599 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 413.00 MB 2023-10-08T00:15:24.599 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.599 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.599 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 414.00 MB 2023-10-08T00:15:24.599 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.599 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.600 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 415.00 MB 2023-10-08T00:15:24.600 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.600 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.600 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 416.00 MB 2023-10-08T00:15:24.600 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.600 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.600 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 417.00 MB 2023-10-08T00:15:24.600 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.600 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.600 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 418.00 MB 2023-10-08T00:15:24.600 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.601 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.601 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 419.00 MB 2023-10-08T00:15:24.601 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.601 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.601 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 420.00 MB 2023-10-08T00:15:24.601 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.601 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.601 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 421.00 MB 2023-10-08T00:15:24.601 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.601 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.601 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 422.00 MB 2023-10-08T00:15:24.601 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.602 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.602 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 423.00 MB 2023-10-08T00:15:24.602 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.602 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.602 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 424.00 MB 2023-10-08T00:15:24.602 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.602 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.602 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 425.00 MB 2023-10-08T00:15:24.602 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.602 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.602 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 426.00 MB 2023-10-08T00:15:24.603 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.603 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.603 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 427.00 MB 2023-10-08T00:15:24.603 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.603 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.603 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 428.00 MB 2023-10-08T00:15:24.603 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.603 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.603 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 429.00 MB 2023-10-08T00:15:24.604 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.604 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.604 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 430.00 MB 2023-10-08T00:15:24.604 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.604 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.604 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 431.00 MB 2023-10-08T00:15:24.604 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.604 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.604 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 432.00 MB 2023-10-08T00:15:24.604 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.605 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.605 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 433.00 MB 2023-10-08T00:15:24.605 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.605 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.605 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 434.00 MB 2023-10-08T00:15:24.605 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.605 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.605 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 435.00 MB 2023-10-08T00:15:24.605 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.605 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.605 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 436.00 MB 2023-10-08T00:15:24.606 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.606 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.606 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 437.00 MB 2023-10-08T00:15:24.606 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.606 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.606 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 438.00 MB 2023-10-08T00:15:24.606 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.606 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.606 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 439.00 MB 2023-10-08T00:15:24.607 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.607 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.607 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 440.00 MB 2023-10-08T00:15:24.607 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.607 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.607 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 441.00 MB 2023-10-08T00:15:24.607 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.607 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.607 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 442.00 MB 2023-10-08T00:15:24.607 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.607 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.607 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 443.00 MB 2023-10-08T00:15:24.608 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.608 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.608 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 444.00 MB 2023-10-08T00:15:24.608 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.608 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.608 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 445.00 MB 2023-10-08T00:15:24.608 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.608 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.608 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 446.00 MB 2023-10-08T00:15:24.608 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.608 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.608 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 447.00 MB 2023-10-08T00:15:24.608 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.609 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.609 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 448.00 MB 2023-10-08T00:15:24.609 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.609 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.609 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 449.00 MB 2023-10-08T00:15:24.609 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.609 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.609 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 450.00 MB 2023-10-08T00:15:24.610 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.610 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.610 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 451.00 MB 2023-10-08T00:15:24.610 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.610 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.610 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 452.00 MB 2023-10-08T00:15:24.610 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.610 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.610 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 453.00 MB 2023-10-08T00:15:24.610 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.610 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.611 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 454.00 MB 2023-10-08T00:15:24.611 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.611 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.611 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 455.00 MB 2023-10-08T00:15:24.611 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.611 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.611 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 456.00 MB 2023-10-08T00:15:24.611 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.611 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.611 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 457.00 MB 2023-10-08T00:15:24.611 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.611 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.612 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 458.00 MB 2023-10-08T00:15:24.612 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.612 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.612 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 459.00 MB 2023-10-08T00:15:24.612 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.612 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.612 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 460.00 MB 2023-10-08T00:15:24.612 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.612 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.612 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 461.00 MB 2023-10-08T00:15:24.612 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.612 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.612 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 462.00 MB 2023-10-08T00:15:24.613 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.613 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.613 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 463.00 MB 2023-10-08T00:15:24.613 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.613 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.613 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 464.00 MB 2023-10-08T00:15:24.613 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.613 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.613 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 465.00 MB 2023-10-08T00:15:24.613 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.613 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.613 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 466.00 MB 2023-10-08T00:15:24.614 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.614 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.614 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 467.00 MB 2023-10-08T00:15:24.614 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.614 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.614 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 468.00 MB 2023-10-08T00:15:24.614 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.614 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.614 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 469.00 MB 2023-10-08T00:15:24.614 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.614 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.614 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 470.00 MB 2023-10-08T00:15:24.615 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.615 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.615 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 471.00 MB 2023-10-08T00:15:24.615 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.615 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.615 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 472.00 MB 2023-10-08T00:15:24.615 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.615 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.615 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 473.00 MB 2023-10-08T00:15:24.615 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.616 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.616 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 474.00 MB 2023-10-08T00:15:24.616 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.616 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.616 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 475.00 MB 2023-10-08T00:15:24.616 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.617 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.617 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 476.00 MB 2023-10-08T00:15:24.617 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.617 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.617 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 477.00 MB 2023-10-08T00:15:24.617 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.618 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.618 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 478.00 MB 2023-10-08T00:15:24.618 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.618 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.618 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 479.00 MB 2023-10-08T00:15:24.618 INFO:teuthology.orchestra.run.smithi099.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-08T00:15:24.619 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:15:24.619 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:15:24.619 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 480.00 MB 2023-10-08T00:15:24.619 INFO:teuthology.orchestra.run.smithi099.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-08T00:15:24.619 INFO:teuthology.orchestra.run.smithi099.stdout:Reached fullness after 480.00 MB 2023-10-08T00:15:24.619 INFO:teuthology.orchestra.run.smithi099.stdout:calling close 2023-10-08T00:15:24.620 INFO:teuthology.orchestra.run.smithi099.stdout:close() did not raise error 2023-10-08T00:15:24.777 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T00:15:24.777 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T00:15:24.827 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T00:15:24.865 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T00:15:24.866 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T00:15:24.866 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:15:24.866 DEBUG:teuthology.orchestra.run.smithi099:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-08T00:15:24.905 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2023-10-08T00:15:24.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:15:24.905 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T00:15:24.961 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-08T00:15:24.962 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:15:24.962 DEBUG:teuthology.orchestra.run.smithi191:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-08T00:15:25.000 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2023-10-08T00:15:25.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:15:25.001 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-08T00:15:25.057 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:15:25.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.174+0000 7fe89250f700 1 Processor -- start 2023-10-08T00:15:25.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.175+0000 7fe89250f700 1 -- start start 2023-10-08T00:15:25.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.175+0000 7fe89250f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe88c15a620 0x7fe88c15d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:25.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.175+0000 7fe89250f700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe88c15d580 0x7fe88c158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:25.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.175+0000 7fe89250f700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe88c15cbb0 con 0x7fe88c157870 2023-10-08T00:15:25.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.175+0000 7fe89250f700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe88c15bdb0 con 0x7fe88c15d580 2023-10-08T00:15:25.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.175+0000 7fe89250f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe88c15bfa0 con 0x7fe88c15a620 2023-10-08T00:15:25.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.175+0000 7fe88bfff700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fe88c157870 0x7fe88c15b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:52138/0 (socket says 172.21.15.99:52138) 2023-10-08T00:15:25.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.175+0000 7fe88bfff700 1 -- 172.21.15.99:0/3709167593 learned_addr learned my addr 172.21.15.99:0/3709167593 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:15:25.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.176+0000 7fe890aac700 1 --2- 172.21.15.99:0/3709167593 >> v2:172.21.15.99:3300/0 conn(0x7fe88c15d580 0x7fe88c158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:25.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.176+0000 7fe88affd700 1 -- 172.21.15.99:0/3709167593 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1597052187 0 0) 0x7fe88c15cbb0 con 0x7fe88c157870 2023-10-08T00:15:25.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.176+0000 7fe88b7fe700 1 --2- 172.21.15.99:0/3709167593 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe88c15a620 0x7fe88c15d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:25.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.176+0000 7fe88affd700 1 -- 172.21.15.99:0/3709167593 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe874003570 con 0x7fe88c157870 2023-10-08T00:15:25.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.176+0000 7fe890aac700 1 -- 172.21.15.99:0/3709167593 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe88c15a620 msgr2=0x7fe88c15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:25.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.176+0000 7fe890aac700 1 --2- 172.21.15.99:0/3709167593 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe88c15a620 0x7fe88c15d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:25.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.176+0000 7fe890aac700 1 -- 172.21.15.99:0/3709167593 >> v1:172.21.15.99:6789/0 conn(0x7fe88c157870 legacy=0x7fe88c15b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:25.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.176+0000 7fe890aac700 1 -- 172.21.15.99:0/3709167593 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe88c15c280 con 0x7fe88c15d580 2023-10-08T00:15:25.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.176+0000 7fe88b7fe700 1 --2- 172.21.15.99:0/3709167593 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe88c15a620 0x7fe88c15d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:15:25.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.176+0000 7fe890aac700 1 --2- 172.21.15.99:0/3709167593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe88c15d580 0x7fe88c158110 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fe884009e60 tx=0x7fe884009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c73b0b2ccdcb985 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:25.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.176+0000 7fe88affd700 1 -- 172.21.15.99:0/3709167593 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88400e070 con 0x7fe88c15d580 2023-10-08T00:15:25.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.176+0000 7fe88affd700 1 -- 172.21.15.99:0/3709167593 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe88400bb60 con 0x7fe88c15d580 2023-10-08T00:15:25.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.176+0000 7fe88affd700 1 -- 172.21.15.99:0/3709167593 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe884012430 con 0x7fe88c15d580 2023-10-08T00:15:25.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.177+0000 7fe89250f700 1 -- 172.21.15.99:0/3709167593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe88c15d580 msgr2=0x7fe88c158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:25.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.177+0000 7fe89250f700 1 --2- 172.21.15.99:0/3709167593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe88c15d580 0x7fe88c158110 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fe884009e60 tx=0x7fe884009780 comp rx=0 tx=0).stop 2023-10-08T00:15:25.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.177+0000 7fe89250f700 1 -- 172.21.15.99:0/3709167593 shutdown_connections 2023-10-08T00:15:25.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.177+0000 7fe89250f700 1 --2- 172.21.15.99:0/3709167593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe88c15d580 0x7fe88c158110 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:25.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.177+0000 7fe89250f700 1 --2- 172.21.15.99:0/3709167593 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe88c15a620 0x7fe88c15d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:25.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.177+0000 7fe89250f700 1 -- 172.21.15.99:0/3709167593 >> 172.21.15.99:0/3709167593 conn(0x7fe88c068ce0 msgr2=0x7fe88c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:25.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.177+0000 7fe89250f700 1 -- 172.21.15.99:0/3709167593 shutdown_connections 2023-10-08T00:15:25.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.177+0000 7fe89250f700 1 -- 172.21.15.99:0/3709167593 wait complete. 2023-10-08T00:15:25.183 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.178+0000 7fe89250f700 1 Processor -- start 2023-10-08T00:15:25.183 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.178+0000 7fe89250f700 1 -- start start 2023-10-08T00:15:25.183 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.179+0000 7fe89250f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe88c157870 0x7fe88c14b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:25.183 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.179+0000 7fe89250f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe88c15a620 0x7fe88c0b1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:25.183 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.179+0000 7fe89250f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe88c15d580 0x7fe88c0b16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:25.183 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.179+0000 7fe89250f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe88c0b27a0 con 0x7fe88c15a620 2023-10-08T00:15:25.183 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.179+0000 7fe89250f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe88c14d530 con 0x7fe88c157870 2023-10-08T00:15:25.183 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.179+0000 7fe89250f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe88c0b1f00 con 0x7fe88c15d580 2023-10-08T00:15:25.184 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.179+0000 7fe88b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe88c15a620 0x7fe88c0b1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:25.184 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.179+0000 7fe88b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe88c15a620 0x7fe88c0b1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55810/0 (socket says 172.21.15.99:55810) 2023-10-08T00:15:25.184 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.179+0000 7fe88b7fe700 1 -- 172.21.15.99:0/2605152624 learned_addr learned my addr 172.21.15.99:0/2605152624 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:25.184 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.179+0000 7fe88bfff700 1 --2- 172.21.15.99:0/2605152624 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe88c157870 0x7fe88c14b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:25.184 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.179+0000 7fe890aac700 1 --2- 172.21.15.99:0/2605152624 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe88c15d580 0x7fe88c0b16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:25.184 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.179+0000 7fe88b7fe700 1 -- 172.21.15.99:0/2605152624 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe88c15d580 msgr2=0x7fe88c0b16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:25.184 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.179+0000 7fe88b7fe700 1 --2- 172.21.15.99:0/2605152624 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe88c15d580 0x7fe88c0b16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:25.184 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.179+0000 7fe88b7fe700 1 -- 172.21.15.99:0/2605152624 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe88c157870 msgr2=0x7fe88c14b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:25.184 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.179+0000 7fe88b7fe700 1 --2- 172.21.15.99:0/2605152624 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe88c157870 0x7fe88c14b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:25.184 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.179+0000 7fe88b7fe700 1 -- 172.21.15.99:0/2605152624 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe884009a20 con 0x7fe88c15a620 2023-10-08T00:15:25.184 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.179+0000 7fe88b7fe700 1 --2- 172.21.15.99:0/2605152624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe88c15a620 0x7fe88c0b1190 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fe88000c880 tx=0x7fe880008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:25.185 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.179+0000 7fe873fff700 1 -- 172.21.15.99:0/2605152624 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe880011600 con 0x7fe88c15a620 2023-10-08T00:15:25.185 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.179+0000 7fe873fff700 1 -- 172.21.15.99:0/2605152624 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe880011760 con 0x7fe88c15a620 2023-10-08T00:15:25.185 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.179+0000 7fe89250f700 1 -- 172.21.15.99:0/2605152624 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe88c0b21e0 con 0x7fe88c15a620 2023-10-08T00:15:25.185 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.179+0000 7fe873fff700 1 -- 172.21.15.99:0/2605152624 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe880011ae0 con 0x7fe88c15a620 2023-10-08T00:15:25.185 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.179+0000 7fe89250f700 1 -- 172.21.15.99:0/2605152624 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe88c14d190 con 0x7fe88c15a620 2023-10-08T00:15:25.185 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.180+0000 7fe89250f700 1 -- 172.21.15.99:0/2605152624 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe88c0b23d0 con 0x7fe88c15a620 2023-10-08T00:15:25.185 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.181+0000 7fe873fff700 1 -- 172.21.15.99:0/2605152624 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe8800073d0 con 0x7fe88c15a620 2023-10-08T00:15:25.185 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.181+0000 7fe873fff700 1 --2- 172.21.15.99:0/2605152624 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe878066f30 0x7fe8780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:25.185 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.181+0000 7fe873fff700 1 -- 172.21.15.99:0/2605152624 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe8800845c0 con 0x7fe88c15a620 2023-10-08T00:15:25.186 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.181+0000 7fe88bfff700 1 --2- 172.21.15.99:0/2605152624 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe878066f30 0x7fe8780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:25.186 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.181+0000 7fe873fff700 1 -- 172.21.15.99:0/2605152624 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=90}) v3 -- 0x7fe87806ea50 con 0x7fe88c15a620 2023-10-08T00:15:25.186 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.182+0000 7fe88bfff700 1 --2- 172.21.15.99:0/2605152624 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe878066f30 0x7fe8780693e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe87c0058d0 tx=0x7fe87c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:25.186 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.185+0000 7fe873fff700 1 -- 172.21.15.99:0/2605152624 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe880051830 con 0x7fe88c15a620 2023-10-08T00:15:25.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.466+0000 7fe89250f700 1 -- 172.21.15.99:0/2605152624 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe88c14d6a0 con 0x7fe88c15a620 2023-10-08T00:15:25.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.467+0000 7fe873fff700 1 -- 172.21.15.99:0/2605152624 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4599 (secure 0 0 0) 0x7fe880015090 con 0x7fe88c15a620 2023-10-08T00:15:25.468 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:15:25.468 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5650,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/2166055423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2166055423},{"type":"v1","addr":"172.21.15.99:6834","nonce":2166055423}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5665,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/4087832300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":4087832300},{"type":"v1","addr":"172.21.15.191:6836","nonce":4087832300}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5668,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/730261712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":730261712},{"type":"v1","addr":"172.21.15.191:6837","nonce":730261712}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:14:43.671064+0000","modified":"2023-10-08T00:14:47.326922+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6039},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6039":{"gid":6039,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/104443412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":104443412},{"type":"v1","addr":"172.21.15.99:6835","nonce":104443412}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-08T00:15:25.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.469+0000 7fe89250f700 1 -- 172.21.15.99:0/2605152624 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe878066f30 msgr2=0x7fe8780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:25.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.469+0000 7fe89250f700 1 --2- 172.21.15.99:0/2605152624 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe878066f30 0x7fe8780693e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe87c0058d0 tx=0x7fe87c009040 comp rx=0 tx=0).stop 2023-10-08T00:15:25.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.469+0000 7fe89250f700 1 -- 172.21.15.99:0/2605152624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe88c15a620 msgr2=0x7fe88c0b1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:25.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.469+0000 7fe89250f700 1 --2- 172.21.15.99:0/2605152624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe88c15a620 0x7fe88c0b1190 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fe88000c880 tx=0x7fe880008040 comp rx=0 tx=0).stop 2023-10-08T00:15:25.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.469+0000 7fe89250f700 1 -- 172.21.15.99:0/2605152624 shutdown_connections 2023-10-08T00:15:25.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.469+0000 7fe89250f700 1 --2- 172.21.15.99:0/2605152624 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe878066f30 0x7fe8780693e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:25.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.469+0000 7fe89250f700 1 --2- 172.21.15.99:0/2605152624 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe88c157870 0x7fe88c14b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:25.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.469+0000 7fe89250f700 1 --2- 172.21.15.99:0/2605152624 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe88c15d580 0x7fe88c0b16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:25.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.469+0000 7fe89250f700 1 --2- 172.21.15.99:0/2605152624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe88c15a620 0x7fe88c0b1190 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:25.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.469+0000 7fe89250f700 1 -- 172.21.15.99:0/2605152624 >> 172.21.15.99:0/2605152624 conn(0x7fe88c068ce0 msgr2=0x7fe88c14c340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:25.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.469+0000 7fe89250f700 1 -- 172.21.15.99:0/2605152624 shutdown_connections 2023-10-08T00:15:25.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.469+0000 7fe89250f700 1 -- 172.21.15.99:0/2605152624 wait complete. 2023-10-08T00:15:25.471 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 40 2023-10-08T00:15:25.482 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:15:25.596 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.595+0000 7f02cf207700 1 Processor -- start 2023-10-08T00:15:25.596 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.595+0000 7f02cf207700 1 -- start start 2023-10-08T00:15:25.602 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.596+0000 7f02cf207700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02c80b0e50 0x7f02c80c9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:25.602 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.596+0000 7f02cf207700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02c815de60 con 0x7f02c80b1220 2023-10-08T00:15:25.602 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.596+0000 7f02cf207700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02c815e140 con 0x7f02c80cb4f0 2023-10-08T00:15:25.602 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.596+0000 7f02cf207700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02c815e420 con 0x7f02c80b0e50 2023-10-08T00:15:25.603 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.596+0000 7f02bffff700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f02c80b1220 0x7f02c806d590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:52144/0 (socket says 172.21.15.99:52144) 2023-10-08T00:15:25.603 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.596+0000 7f02bffff700 1 -- 172.21.15.99:0/2922467558 learned_addr learned my addr 172.21.15.99:0/2922467558 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:15:25.603 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.597+0000 7f02ccfa3700 1 --2- 172.21.15.99:0/2922467558 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02c80b0e50 0x7f02c80c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:25.603 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.597+0000 7f02bf7fe700 1 -- 172.21.15.99:0/2922467558 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3285098801 0 0) 0x7f02c815de60 con 0x7f02c80b1220 2023-10-08T00:15:25.603 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.597+0000 7f02bf7fe700 1 -- 172.21.15.99:0/2922467558 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02b0003570 con 0x7f02c80b1220 2023-10-08T00:15:25.603 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.597+0000 7f02bf7fe700 1 -- 172.21.15.99:0/2922467558 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3666305084 0 0) 0x7f02b0003570 con 0x7f02c80b1220 2023-10-08T00:15:25.603 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.597+0000 7f02bf7fe700 1 -- 172.21.15.99:0/2922467558 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02c80b0e50 msgr2=0x7f02c80c9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:25.603 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.597+0000 7f02bf7fe700 1 --2- 172.21.15.99:0/2922467558 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02c80b0e50 0x7f02c80c9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:25.603 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.597+0000 7f02bf7fe700 1 -- 172.21.15.99:0/2922467558 >> v1:172.21.15.191:6789/0 conn(0x7f02c80cb4f0 legacy=0x7f02c80679d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:25.604 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.597+0000 7f02bf7fe700 1 -- 172.21.15.99:0/2922467558 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c815e700 con 0x7f02c80b1220 2023-10-08T00:15:25.604 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.597+0000 7f02bf7fe700 1 -- 172.21.15.99:0/2922467558 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f02b8001470 con 0x7f02c80b1220 2023-10-08T00:15:25.604 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.597+0000 7f02ccfa3700 1 --2- 172.21.15.99:0/2922467558 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02c80b0e50 0x7f02c80c9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:15:25.604 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.597+0000 7f02bf7fe700 1 -- 172.21.15.99:0/2922467558 >> v1:172.21.15.99:6789/0 conn(0x7f02c80b1220 legacy=0x7f02c806d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:25.604 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.597+0000 7f02bf7fe700 1 --2- 172.21.15.99:0/2922467558 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02b0003a90 0x7f02b0003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:25.604 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.597+0000 7f02bf7fe700 1 --2- 172.21.15.99:0/2922467558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02b0004810 0x7f02b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:25.604 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.597+0000 7f02bf7fe700 1 --2- 172.21.15.99:0/2922467558 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02c80b0e50 0x7f02b000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:25.604 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.597+0000 7f02bf7fe700 1 -- 172.21.15.99:0/2922467558 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c815e420 con 0x7f02b0004810 2023-10-08T00:15:25.604 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.597+0000 7f02bf7fe700 1 -- 172.21.15.99:0/2922467558 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b0005500 con 0x7f02b0003a90 2023-10-08T00:15:25.605 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.597+0000 7f02bf7fe700 1 -- 172.21.15.99:0/2922467558 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02b00056c0 con 0x7f02c80b0e50 2023-10-08T00:15:25.605 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.597+0000 7f02bffff700 1 --2- 172.21.15.99:0/2922467558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02b0004810 0x7f02b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:25.605 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.597+0000 7f02bffff700 1 -- 172.21.15.99:0/2922467558 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02c80b0e50 msgr2=0x7f02b000ba60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T00:15:25.605 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.597+0000 7f02bffff700 1 --2- 172.21.15.99:0/2922467558 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02c80b0e50 0x7f02b000ba60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:25.605 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.597+0000 7f02bffff700 1 -- 172.21.15.99:0/2922467558 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02b0003a90 msgr2=0x7f02b0003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:25.605 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.597+0000 7f02bffff700 1 --2- 172.21.15.99:0/2922467558 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02b0003a90 0x7f02b0003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:25.605 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.597+0000 7f02bffff700 1 -- 172.21.15.99:0/2922467558 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b0003570 con 0x7f02b0004810 2023-10-08T00:15:25.605 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.597+0000 7f02bffff700 1 --2- 172.21.15.99:0/2922467558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02b0004810 0x7f02b0004c80 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f02b800c3c0 tx=0x7f02b8001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80fa961903407e12 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:25.605 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.598+0000 7f02bf7fe700 1 -- 172.21.15.99:0/2922467558 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b800cc90 con 0x7f02b0004810 2023-10-08T00:15:25.605 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.598+0000 7f02bf7fe700 1 -- 172.21.15.99:0/2922467558 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02b800cdf0 con 0x7f02b0004810 2023-10-08T00:15:25.606 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.598+0000 7f02bf7fe700 1 -- 172.21.15.99:0/2922467558 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b8005b70 con 0x7f02b0004810 2023-10-08T00:15:25.606 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.598+0000 7f02cf207700 1 -- 172.21.15.99:0/2922467558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02b0004810 msgr2=0x7f02b0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:25.606 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.598+0000 7f02cf207700 1 --2- 172.21.15.99:0/2922467558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02b0004810 0x7f02b0004c80 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f02b800c3c0 tx=0x7f02b8001e50 comp rx=0 tx=0).stop 2023-10-08T00:15:25.606 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.598+0000 7f02cd7a4700 1 -- 172.21.15.99:0/2922467558 reap_dead start 2023-10-08T00:15:25.606 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.598+0000 7f02cf207700 1 -- 172.21.15.99:0/2922467558 shutdown_connections 2023-10-08T00:15:25.606 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.598+0000 7f02cf207700 1 -- 172.21.15.99:0/2922467558 >> 172.21.15.99:0/2922467558 conn(0x7f02c8069870 msgr2=0x7f02c8069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:25.606 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.598+0000 7f02cf207700 1 -- 172.21.15.99:0/2922467558 shutdown_connections 2023-10-08T00:15:25.606 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.598+0000 7f02cf207700 1 -- 172.21.15.99:0/2922467558 wait complete. 2023-10-08T00:15:25.606 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.599+0000 7f02cf207700 1 Processor -- start 2023-10-08T00:15:25.606 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.599+0000 7f02cf207700 1 -- start start 2023-10-08T00:15:25.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.600+0000 7f02cf207700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02c8155f60 0x7f02c80cc240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:25.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.600+0000 7f02cf207700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02c8156330 0x7f02c80cc780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:25.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.600+0000 7f02cf207700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02c815c470 0x7f02c815c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:25.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.600+0000 7f02cf207700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c80c66a0 con 0x7f02c815c470 2023-10-08T00:15:25.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.600+0000 7f02cf207700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c80c5d70 con 0x7f02c8155f60 2023-10-08T00:15:25.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.600+0000 7f02cf207700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02c80c6050 con 0x7f02c8156330 2023-10-08T00:15:25.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.600+0000 7f02cd7a4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02c815c470 0x7f02c815c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:25.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.600+0000 7f02cd7a4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02c815c470 0x7f02c815c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55832/0 (socket says 172.21.15.99:55832) 2023-10-08T00:15:25.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.600+0000 7f02cd7a4700 1 -- 172.21.15.99:0/978100233 learned_addr learned my addr 172.21.15.99:0/978100233 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:25.609 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.600+0000 7f02ccfa3700 1 --2- 172.21.15.99:0/978100233 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02c8155f60 0x7f02c80cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:25.609 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.600+0000 7f02cd7a4700 1 -- 172.21.15.99:0/978100233 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02c8156330 msgr2=0x7f02c80cc780 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T00:15:25.609 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.600+0000 7f02cd7a4700 1 --2- 172.21.15.99:0/978100233 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02c8156330 0x7f02c80cc780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:25.609 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.600+0000 7f02cd7a4700 1 -- 172.21.15.99:0/978100233 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02c8155f60 msgr2=0x7f02c80cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:25.609 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.600+0000 7f02cd7a4700 1 --2- 172.21.15.99:0/978100233 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02c8155f60 0x7f02c80cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:25.609 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.600+0000 7f02cd7a4700 1 -- 172.21.15.99:0/978100233 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c80c6330 con 0x7f02c815c470 2023-10-08T00:15:25.609 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.601+0000 7f02cd7a4700 1 --2- 172.21.15.99:0/978100233 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02c815c470 0x7f02c815c910 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f02c815e9e0 tx=0x7f02c0007de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:25.610 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.601+0000 7f02bcff9700 1 -- 172.21.15.99:0/978100233 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c00070d0 con 0x7f02c815c470 2023-10-08T00:15:25.610 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.601+0000 7f02bcff9700 1 -- 172.21.15.99:0/978100233 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02c0001e90 con 0x7f02c815c470 2023-10-08T00:15:25.610 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.601+0000 7f02cf207700 1 -- 172.21.15.99:0/978100233 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02c815ce50 con 0x7f02c815c470 2023-10-08T00:15:25.610 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.601+0000 7f02bcff9700 1 -- 172.21.15.99:0/978100233 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c0007480 con 0x7f02c815c470 2023-10-08T00:15:25.610 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.601+0000 7f02cf207700 1 -- 172.21.15.99:0/978100233 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02c815a4e0 con 0x7f02c815c470 2023-10-08T00:15:25.610 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.602+0000 7f02cf207700 1 -- 172.21.15.99:0/978100233 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02c804c3c0 con 0x7f02c815c470 2023-10-08T00:15:25.611 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.602+0000 7f02bcff9700 1 -- 172.21.15.99:0/978100233 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f02c0005a20 con 0x7f02c815c470 2023-10-08T00:15:25.611 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.602+0000 7f02bcff9700 1 --2- 172.21.15.99:0/978100233 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f02b4066f30 0x7f02b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:25.611 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.603+0000 7f02bcff9700 1 -- 172.21.15.99:0/978100233 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f02c00833f0 con 0x7f02c815c470 2023-10-08T00:15:25.611 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.603+0000 7f02bcff9700 1 -- 172.21.15.99:0/978100233 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=90}) v3 -- 0x7f02b406ea50 con 0x7f02c815c470 2023-10-08T00:15:25.611 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.603+0000 7f02ccfa3700 1 --2- 172.21.15.99:0/978100233 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f02b4066f30 0x7f02b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:25.611 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.603+0000 7f02ccfa3700 1 --2- 172.21.15.99:0/978100233 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f02b4066f30 0x7f02b40693e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f02c40077a0 tx=0x7f02c4009900 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:25.612 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.607+0000 7f02bcff9700 1 -- 172.21.15.99:0/978100233 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f02c004a5a0 con 0x7f02c815c470 2023-10-08T00:15:25.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.790+0000 7f02cf207700 1 -- 172.21.15.99:0/978100233 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f02c8068ba0 con 0x7f02c815c470 2023-10-08T00:15:25.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.791+0000 7f02bcff9700 1 -- 172.21.15.99:0/978100233 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+15442 (secure 0 0 0) 0x7f02c0050660 con 0x7f02c815c470 2023-10-08T00:15:25.792 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:15:25.792 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":89,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:14:59.098654+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:14:41.496823+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-08T00:14:42.640029+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["backfillfull","exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","full","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["backfillfull","exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["backfillfull","exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["backfillfull","exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:15:25.793 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.793+0000 7f02cf207700 1 -- 172.21.15.99:0/978100233 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f02b4066f30 msgr2=0x7f02b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:25.794 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.793+0000 7f02cf207700 1 --2- 172.21.15.99:0/978100233 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f02b4066f30 0x7f02b40693e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f02c40077a0 tx=0x7f02c4009900 comp rx=0 tx=0).stop 2023-10-08T00:15:25.794 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.793+0000 7f02cf207700 1 -- 172.21.15.99:0/978100233 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02c815c470 msgr2=0x7f02c815c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:25.794 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.793+0000 7f02cf207700 1 --2- 172.21.15.99:0/978100233 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02c815c470 0x7f02c815c910 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f02c815e9e0 tx=0x7f02c0007de0 comp rx=0 tx=0).stop 2023-10-08T00:15:25.795 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.793+0000 7f02cf207700 1 -- 172.21.15.99:0/978100233 shutdown_connections 2023-10-08T00:15:25.795 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.793+0000 7f02cf207700 1 --2- 172.21.15.99:0/978100233 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f02b4066f30 0x7f02b40693e0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:25.795 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.793+0000 7f02cf207700 1 --2- 172.21.15.99:0/978100233 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02c8155f60 0x7f02c80cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:25.795 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.793+0000 7f02cf207700 1 --2- 172.21.15.99:0/978100233 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02c815c470 0x7f02c815c910 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:25.795 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.793+0000 7f02cf207700 1 --2- 172.21.15.99:0/978100233 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02c8156330 0x7f02c80cc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:25.795 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.793+0000 7f02cf207700 1 -- 172.21.15.99:0/978100233 >> 172.21.15.99:0/978100233 conn(0x7f02c8069870 msgr2=0x7f02c80c4170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:25.795 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.793+0000 7f02cf207700 1 -- 172.21.15.99:0/978100233 shutdown_connections 2023-10-08T00:15:25.796 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.793+0000 7f02cf207700 1 -- 172.21.15.99:0/978100233 wait complete. 2023-10-08T00:15:25.806 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:10:42.063344+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T00:14:41.496823+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T00:14:42.640029+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T00:15:25.807 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T00:15:25.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.923+0000 7f7ec7b86700 1 Processor -- start 2023-10-08T00:15:25.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.923+0000 7f7ec7b86700 1 -- start start 2023-10-08T00:15:25.927 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.924+0000 7f7ec7b86700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7ec015a290 0x7f7ec006a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:25.927 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.924+0000 7f7ec7b86700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ec00b0790 con 0x7f7ec006aae0 2023-10-08T00:15:25.927 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.924+0000 7f7ec7b86700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ec006af20 con 0x7f7ec0159ec0 2023-10-08T00:15:25.927 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.924+0000 7f7ec7b86700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ec00ce1f0 con 0x7f7ec015a290 2023-10-08T00:15:25.927 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.924+0000 7f7ec6123700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f7ec006aae0 0x7f7ec00cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:52146/0 (socket says 172.21.15.99:52146) 2023-10-08T00:15:25.927 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.924+0000 7f7ec6123700 1 -- 172.21.15.99:0/4121186397 learned_addr learned my addr 172.21.15.99:0/4121186397 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:15:25.927 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.924+0000 7f7ec5121700 1 --2- 172.21.15.99:0/4121186397 >> v2:172.21.15.191:3300/0 conn(0x7f7ec015a290 0x7f7ec006a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:25.927 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.924+0000 7f7ec4920700 1 -- 172.21.15.99:0/4121186397 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4206697597 0 0) 0x7f7ec006af20 con 0x7f7ec0159ec0 2023-10-08T00:15:25.928 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.924+0000 7f7ec4920700 1 -- 172.21.15.99:0/4121186397 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7eac003570 con 0x7f7ec0159ec0 2023-10-08T00:15:25.928 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.924+0000 7f7ec4920700 1 -- 172.21.15.99:0/4121186397 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2244409443 0 0) 0x7f7ec00b0790 con 0x7f7ec006aae0 2023-10-08T00:15:25.928 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.924+0000 7f7ec4920700 1 -- 172.21.15.99:0/4121186397 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ec006af20 con 0x7f7ec006aae0 2023-10-08T00:15:25.928 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.925+0000 7f7ec5121700 1 -- 172.21.15.99:0/4121186397 >> v1:172.21.15.191:6789/0 conn(0x7f7ec0159ec0 legacy=0x7f7ec00af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:25.928 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.925+0000 7f7ec5121700 1 -- 172.21.15.99:0/4121186397 >> v1:172.21.15.99:6789/0 conn(0x7f7ec006aae0 legacy=0x7f7ec00cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:25.928 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.925+0000 7f7ec5121700 1 -- 172.21.15.99:0/4121186397 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec00b0b20 con 0x7f7ec015a290 2023-10-08T00:15:25.928 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.925+0000 7f7ec5121700 1 --2- 172.21.15.99:0/4121186397 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ec015a290 0x7f7ec006a5a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7eb00098e0 tx=0x7f7eb00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=101608f7ecf5d1d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:25.928 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.925+0000 7f7ec4920700 1 -- 172.21.15.99:0/4121186397 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb0011630 con 0x7f7ec015a290 2023-10-08T00:15:25.928 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.925+0000 7f7ec4920700 1 -- 172.21.15.99:0/4121186397 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eb0011790 con 0x7f7ec015a290 2023-10-08T00:15:25.929 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.925+0000 7f7ec4920700 1 -- 172.21.15.99:0/4121186397 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb0011b30 con 0x7f7ec015a290 2023-10-08T00:15:25.929 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.925+0000 7f7ec7b86700 1 -- 172.21.15.99:0/4121186397 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ec015a290 msgr2=0x7f7ec006a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:25.929 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.925+0000 7f7ec7b86700 1 --2- 172.21.15.99:0/4121186397 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ec015a290 0x7f7ec006a5a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7eb00098e0 tx=0x7f7eb00091a0 comp rx=0 tx=0).stop 2023-10-08T00:15:25.929 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.925+0000 7f7ec7b86700 1 -- 172.21.15.99:0/4121186397 shutdown_connections 2023-10-08T00:15:25.929 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.925+0000 7f7ec7b86700 1 --2- 172.21.15.99:0/4121186397 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ec015a290 0x7f7ec006a5a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:25.929 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.925+0000 7f7ec7b86700 1 -- 172.21.15.99:0/4121186397 >> 172.21.15.99:0/4121186397 conn(0x7f7ec00d1880 msgr2=0x7f7ec00cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:25.929 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.926+0000 7f7ec7b86700 1 -- 172.21.15.99:0/4121186397 shutdown_connections 2023-10-08T00:15:25.929 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.926+0000 7f7ec7b86700 1 -- 172.21.15.99:0/4121186397 wait complete. 2023-10-08T00:15:25.932 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.927+0000 7f7ec7b86700 1 Processor -- start 2023-10-08T00:15:25.932 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.927+0000 7f7ec7b86700 1 -- start start 2023-10-08T00:15:25.932 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.927+0000 7f7ec7b86700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7ec006aae0 0x7f7ec00c9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:25.932 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.927+0000 7f7ec7b86700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ec0159ec0 0x7f7ec015b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:25.932 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.927+0000 7f7ec7b86700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ec015a290 0x7f7ec0155890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:25.932 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.927+0000 7f7ec7b86700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec00ca550 con 0x7f7ec0159ec0 2023-10-08T00:15:25.932 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.927+0000 7f7ec7b86700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec00ca830 con 0x7f7ec015a290 2023-10-08T00:15:25.932 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.927+0000 7f7ec7b86700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec00cab10 con 0x7f7ec006aae0 2023-10-08T00:15:25.932 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.927+0000 7f7ec5121700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ec0159ec0 0x7f7ec015b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:25.933 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.927+0000 7f7ec5121700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ec0159ec0 0x7f7ec015b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55844/0 (socket says 172.21.15.99:55844) 2023-10-08T00:15:25.933 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.928+0000 7f7ec5121700 1 -- 172.21.15.99:0/2761944308 learned_addr learned my addr 172.21.15.99:0/2761944308 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:25.933 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.928+0000 7f7ec6123700 1 --2- 172.21.15.99:0/2761944308 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ec015a290 0x7f7ec0155890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:25.933 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.928+0000 7f7ec5922700 1 --2- 172.21.15.99:0/2761944308 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7ec006aae0 0x7f7ec00c9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:25.933 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.928+0000 7f7ec5121700 1 -- 172.21.15.99:0/2761944308 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7ec006aae0 msgr2=0x7f7ec00c9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:25.933 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.928+0000 7f7ec5121700 1 --2- 172.21.15.99:0/2761944308 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7ec006aae0 0x7f7ec00c9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:25.933 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.928+0000 7f7ec5121700 1 -- 172.21.15.99:0/2761944308 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ec015a290 msgr2=0x7f7ec0155890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:25.933 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.928+0000 7f7ec5121700 1 --2- 172.21.15.99:0/2761944308 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ec015a290 0x7f7ec0155890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:25.933 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.928+0000 7f7ec5121700 1 -- 172.21.15.99:0/2761944308 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb0009420 con 0x7f7ec0159ec0 2023-10-08T00:15:25.934 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.928+0000 7f7ec5121700 1 --2- 172.21.15.99:0/2761944308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ec0159ec0 0x7f7ec015b1e0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f7ec006b0b0 tx=0x7f7eb0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:25.934 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.928+0000 7f7eb5ffb700 1 -- 172.21.15.99:0/2761944308 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb001f070 con 0x7f7ec0159ec0 2023-10-08T00:15:25.934 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.928+0000 7f7ec7b86700 1 -- 172.21.15.99:0/2761944308 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ec00cadf0 con 0x7f7ec0159ec0 2023-10-08T00:15:25.934 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.928+0000 7f7eb5ffb700 1 -- 172.21.15.99:0/2761944308 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eb0002f70 con 0x7f7ec0159ec0 2023-10-08T00:15:25.934 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.928+0000 7f7eb5ffb700 1 -- 172.21.15.99:0/2761944308 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb0023410 con 0x7f7ec0159ec0 2023-10-08T00:15:25.934 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.928+0000 7f7ec7b86700 1 -- 172.21.15.99:0/2761944308 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ec015b720 con 0x7f7ec0159ec0 2023-10-08T00:15:25.934 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.929+0000 7f7ec7b86700 1 -- 172.21.15.99:0/2761944308 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ec00d0b50 con 0x7f7ec0159ec0 2023-10-08T00:15:25.934 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.930+0000 7f7eb5ffb700 1 -- 172.21.15.99:0/2761944308 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7eb002c3b0 con 0x7f7ec0159ec0 2023-10-08T00:15:25.934 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.930+0000 7f7eb5ffb700 1 --2- 172.21.15.99:0/2761944308 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7ea4066f30 0x7f7ea40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:25.934 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.930+0000 7f7eb5ffb700 1 -- 172.21.15.99:0/2761944308 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7eb00856b0 con 0x7f7ec0159ec0 2023-10-08T00:15:25.935 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.930+0000 7f7eb5ffb700 1 -- 172.21.15.99:0/2761944308 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=90}) v3 -- 0x7f7ea406ea50 con 0x7f7ec0159ec0 2023-10-08T00:15:25.935 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.930+0000 7f7ec5922700 1 --2- 172.21.15.99:0/2761944308 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7ea4066f30 0x7f7ea40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:25.935 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.930+0000 7f7ec5922700 1 --2- 172.21.15.99:0/2761944308 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7ea4066f30 0x7f7ea40693e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc004140 tx=0x7f7ebc007f90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:25.935 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:25.934+0000 7f7eb5ffb700 1 -- 172.21.15.99:0/2761944308 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7eb0028060 con 0x7f7ec0159ec0 2023-10-08T00:15:26.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.116+0000 7f7ec7b86700 1 -- 172.21.15.99:0/2761944308 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7ec004c3c0 con 0x7f7ec0159ec0 2023-10-08T00:15:26.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.117+0000 7f7eb5ffb700 1 -- 172.21.15.99:0/2761944308 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7eb0015020 con 0x7f7ec0159ec0 2023-10-08T00:15:26.118 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2023-10-08T00:15:26.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.119+0000 7f7ec7b86700 1 -- 172.21.15.99:0/2761944308 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7ea4066f30 msgr2=0x7f7ea40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:26.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.119+0000 7f7ec7b86700 1 --2- 172.21.15.99:0/2761944308 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7ea4066f30 0x7f7ea40693e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc004140 tx=0x7f7ebc007f90 comp rx=0 tx=0).stop 2023-10-08T00:15:26.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.119+0000 7f7ec7b86700 1 -- 172.21.15.99:0/2761944308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ec0159ec0 msgr2=0x7f7ec015b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:26.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.119+0000 7f7ec7b86700 1 --2- 172.21.15.99:0/2761944308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ec0159ec0 0x7f7ec015b1e0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f7ec006b0b0 tx=0x7f7eb0017000 comp rx=0 tx=0).stop 2023-10-08T00:15:26.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.119+0000 7f7ec7b86700 1 -- 172.21.15.99:0/2761944308 shutdown_connections 2023-10-08T00:15:26.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.119+0000 7f7ec7b86700 1 --2- 172.21.15.99:0/2761944308 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7ea4066f30 0x7f7ea40693e0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:26.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.119+0000 7f7ec7b86700 1 --2- 172.21.15.99:0/2761944308 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ec015a290 0x7f7ec0155890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:26.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.119+0000 7f7ec7b86700 1 --2- 172.21.15.99:0/2761944308 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7ec006aae0 0x7f7ec00c9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:26.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.119+0000 7f7ec7b86700 1 --2- 172.21.15.99:0/2761944308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ec0159ec0 0x7f7ec015b1e0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:26.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.119+0000 7f7ec7b86700 1 -- 172.21.15.99:0/2761944308 >> 172.21.15.99:0/2761944308 conn(0x7f7ec00d1880 msgr2=0x7f7ec00b2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:26.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.119+0000 7f7ec7b86700 1 -- 172.21.15.99:0/2761944308 shutdown_connections 2023-10-08T00:15:26.121 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.119+0000 7f7ec7b86700 1 -- 172.21.15.99:0/2761944308 wait complete. 2023-10-08T00:15:26.131 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T00:15:26.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.248+0000 7fc739d1d700 1 Processor -- start 2023-10-08T00:15:26.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.248+0000 7fc739d1d700 1 -- start start 2023-10-08T00:15:26.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.249+0000 7fc739d1d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc734159ec0 0x7fc7340af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:26.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.249+0000 7fc739d1d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc73415a290 0x7fc7340afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:26.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.249+0000 7fc739d1d700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7340b0790 con 0x7fc73406a5a0 2023-10-08T00:15:26.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.249+0000 7fc739d1d700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7340ce1f0 con 0x7fc73415a290 2023-10-08T00:15:26.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.249+0000 7fc739d1d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7340b0b20 con 0x7fc734159ec0 2023-10-08T00:15:26.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.249+0000 7fc732ffd700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc73415a290 0x7fc7340afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:26.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.249+0000 7fc7337fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc734159ec0 0x7fc7340af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:26.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.249+0000 7fc733fff700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7fc73406a5a0 0x7fc73415db90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:46230/0 (socket says 172.21.15.99:46230) 2023-10-08T00:15:26.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.249+0000 7fc733fff700 1 -- 172.21.15.99:0/3051971654 learned_addr learned my addr 172.21.15.99:0/3051971654 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:15:26.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.250+0000 7fc7327fc700 1 -- 172.21.15.99:0/3051971654 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 695323239 0 0) 0x7fc7340b0790 con 0x7fc73406a5a0 2023-10-08T00:15:26.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.250+0000 7fc7327fc700 1 -- 172.21.15.99:0/3051971654 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc71c003570 con 0x7fc73406a5a0 2023-10-08T00:15:26.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.250+0000 7fc732ffd700 1 -- 172.21.15.99:0/3051971654 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc734159ec0 msgr2=0x7fc7340af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:26.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.250+0000 7fc732ffd700 1 --2- 172.21.15.99:0/3051971654 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc734159ec0 0x7fc7340af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:26.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.250+0000 7fc732ffd700 1 -- 172.21.15.99:0/3051971654 >> v1:172.21.15.191:6789/0 conn(0x7fc73406a5a0 legacy=0x7fc73415db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:26.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.250+0000 7fc732ffd700 1 -- 172.21.15.99:0/3051971654 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7340b04b0 con 0x7fc73415a290 2023-10-08T00:15:26.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.250+0000 7fc7337fe700 1 --2- 172.21.15.99:0/3051971654 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc734159ec0 0x7fc7340af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:15:26.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.250+0000 7fc732ffd700 1 --2- 172.21.15.99:0/3051971654 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc73415a290 0x7fc7340afe80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc7280098e0 tx=0x7fc7280091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=263fde96bd5e0589 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:26.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.250+0000 7fc7327fc700 1 -- 172.21.15.99:0/3051971654 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc728011630 con 0x7fc73415a290 2023-10-08T00:15:26.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.250+0000 7fc7327fc700 1 -- 172.21.15.99:0/3051971654 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc728011790 con 0x7fc73415a290 2023-10-08T00:15:26.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.250+0000 7fc7327fc700 1 -- 172.21.15.99:0/3051971654 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc728011b30 con 0x7fc73415a290 2023-10-08T00:15:26.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.251+0000 7fc739d1d700 1 -- 172.21.15.99:0/3051971654 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc73415a290 msgr2=0x7fc7340afe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:26.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.251+0000 7fc739d1d700 1 --2- 172.21.15.99:0/3051971654 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc73415a290 0x7fc7340afe80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc7280098e0 tx=0x7fc7280091a0 comp rx=0 tx=0).stop 2023-10-08T00:15:26.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.251+0000 7fc739d1d700 1 -- 172.21.15.99:0/3051971654 shutdown_connections 2023-10-08T00:15:26.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.251+0000 7fc739d1d700 1 --2- 172.21.15.99:0/3051971654 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc734159ec0 0x7fc7340af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:26.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.251+0000 7fc739d1d700 1 --2- 172.21.15.99:0/3051971654 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc73415a290 0x7fc7340afe80 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:26.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.251+0000 7fc739d1d700 1 -- 172.21.15.99:0/3051971654 >> 172.21.15.99:0/3051971654 conn(0x7fc7340d1880 msgr2=0x7fc7340cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:26.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.251+0000 7fc739d1d700 1 -- 172.21.15.99:0/3051971654 shutdown_connections 2023-10-08T00:15:26.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.251+0000 7fc739d1d700 1 -- 172.21.15.99:0/3051971654 wait complete. 2023-10-08T00:15:26.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.252+0000 7fc739d1d700 1 Processor -- start 2023-10-08T00:15:26.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.252+0000 7fc739d1d700 1 -- start start 2023-10-08T00:15:26.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.253+0000 7fc739d1d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc73406a5a0 0x7fc734148b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:26.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.253+0000 7fc739d1d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc734159ec0 0x7fc7340c89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:26.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.253+0000 7fc739d1d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc73415a290 0x7fc7340cbc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:26.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.253+0000 7fc739d1d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc734154eb0 con 0x7fc734159ec0 2023-10-08T00:15:26.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.253+0000 7fc739d1d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7340c8510 con 0x7fc73415a290 2023-10-08T00:15:26.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.253+0000 7fc739d1d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7340c8ee0 con 0x7fc73406a5a0 2023-10-08T00:15:26.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.253+0000 7fc732ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc734159ec0 0x7fc7340c89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:26.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.253+0000 7fc732ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc734159ec0 0x7fc7340c89a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55854/0 (socket says 172.21.15.99:55854) 2023-10-08T00:15:26.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.253+0000 7fc732ffd700 1 -- 172.21.15.99:0/3616281939 learned_addr learned my addr 172.21.15.99:0/3616281939 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:26.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.253+0000 7fc733fff700 1 --2- 172.21.15.99:0/3616281939 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc73415a290 0x7fc7340cbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:26.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.253+0000 7fc7337fe700 1 --2- 172.21.15.99:0/3616281939 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc73406a5a0 0x7fc734148b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:26.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.253+0000 7fc732ffd700 1 -- 172.21.15.99:0/3616281939 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc73406a5a0 msgr2=0x7fc734148b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:26.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.253+0000 7fc732ffd700 1 --2- 172.21.15.99:0/3616281939 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc73406a5a0 0x7fc734148b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:26.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.253+0000 7fc732ffd700 1 -- 172.21.15.99:0/3616281939 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc73415a290 msgr2=0x7fc7340cbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:26.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.253+0000 7fc732ffd700 1 --2- 172.21.15.99:0/3616281939 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc73415a290 0x7fc7340cbc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:26.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.253+0000 7fc732ffd700 1 -- 172.21.15.99:0/3616281939 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc724009a20 con 0x7fc734159ec0 2023-10-08T00:15:26.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.253+0000 7fc733fff700 1 --2- 172.21.15.99:0/3616281939 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc73415a290 0x7fc7340cbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:15:26.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.253+0000 7fc732ffd700 1 --2- 172.21.15.99:0/3616281939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc734159ec0 0x7fc7340c89a0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fc734071eb0 tx=0x7fc728017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:26.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.254+0000 7fc71b7fe700 1 -- 172.21.15.99:0/3616281939 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc728000d10 con 0x7fc734159ec0 2023-10-08T00:15:26.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.254+0000 7fc71b7fe700 1 -- 172.21.15.99:0/3616281939 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc728011b10 con 0x7fc734159ec0 2023-10-08T00:15:26.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.254+0000 7fc739d1d700 1 -- 172.21.15.99:0/3616281939 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc728009420 con 0x7fc734159ec0 2023-10-08T00:15:26.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.254+0000 7fc71b7fe700 1 -- 172.21.15.99:0/3616281939 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc728025660 con 0x7fc734159ec0 2023-10-08T00:15:26.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.254+0000 7fc739d1d700 1 -- 172.21.15.99:0/3616281939 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc73416aea0 con 0x7fc734159ec0 2023-10-08T00:15:26.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.254+0000 7fc739d1d700 1 -- 172.21.15.99:0/3616281939 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc73404c3c0 con 0x7fc734159ec0 2023-10-08T00:15:26.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.255+0000 7fc71b7fe700 1 -- 172.21.15.99:0/3616281939 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc728025a90 con 0x7fc734159ec0 2023-10-08T00:15:26.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.255+0000 7fc71b7fe700 1 --2- 172.21.15.99:0/3616281939 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc720066f30 0x7fc7200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:26.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.255+0000 7fc71b7fe700 1 -- 172.21.15.99:0/3616281939 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc728085cf0 con 0x7fc734159ec0 2023-10-08T00:15:26.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.255+0000 7fc71b7fe700 1 -- 172.21.15.99:0/3616281939 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=90}) v3 -- 0x7fc72006ea50 con 0x7fc734159ec0 2023-10-08T00:15:26.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.255+0000 7fc7337fe700 1 --2- 172.21.15.99:0/3616281939 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc720066f30 0x7fc7200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:26.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.256+0000 7fc7337fe700 1 --2- 172.21.15.99:0/3616281939 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc720066f30 0x7fc7200693e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc724009fd0 tx=0x7fc72400b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:26.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.260+0000 7fc71b7fe700 1 -- 172.21.15.99:0/3616281939 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc72808a070 con 0x7fc734159ec0 2023-10-08T00:15:26.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.444+0000 7fc739d1d700 1 -- 172.21.15.99:0/3616281939 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc7340ceb00 con 0x7fc734159ec0 2023-10-08T00:15:26.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.444+0000 7fc71b7fe700 1 -- 172.21.15.99:0/3616281939 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc728015030 con 0x7fc734159ec0 2023-10-08T00:15:26.445 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2023-10-08T00:15:26.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.446+0000 7fc739d1d700 1 -- 172.21.15.99:0/3616281939 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc720066f30 msgr2=0x7fc7200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:26.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.446+0000 7fc739d1d700 1 --2- 172.21.15.99:0/3616281939 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc720066f30 0x7fc7200693e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc724009fd0 tx=0x7fc72400b040 comp rx=0 tx=0).stop 2023-10-08T00:15:26.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.446+0000 7fc739d1d700 1 -- 172.21.15.99:0/3616281939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc734159ec0 msgr2=0x7fc7340c89a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:26.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.446+0000 7fc739d1d700 1 --2- 172.21.15.99:0/3616281939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc734159ec0 0x7fc7340c89a0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fc734071eb0 tx=0x7fc728017000 comp rx=0 tx=0).stop 2023-10-08T00:15:26.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.446+0000 7fc739d1d700 1 -- 172.21.15.99:0/3616281939 shutdown_connections 2023-10-08T00:15:26.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.446+0000 7fc739d1d700 1 --2- 172.21.15.99:0/3616281939 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc720066f30 0x7fc7200693e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:26.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.446+0000 7fc739d1d700 1 --2- 172.21.15.99:0/3616281939 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc73415a290 0x7fc7340cbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:26.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.446+0000 7fc739d1d700 1 --2- 172.21.15.99:0/3616281939 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc73406a5a0 0x7fc734148b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:26.449 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.446+0000 7fc739d1d700 1 --2- 172.21.15.99:0/3616281939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc734159ec0 0x7fc7340c89a0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:26.449 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.446+0000 7fc739d1d700 1 -- 172.21.15.99:0/3616281939 >> 172.21.15.99:0/3616281939 conn(0x7fc7340d1880 msgr2=0x7fc7340afca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:26.449 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.446+0000 7fc739d1d700 1 -- 172.21.15.99:0/3616281939 shutdown_connections 2023-10-08T00:15:26.449 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.446+0000 7fc739d1d700 1 -- 172.21.15.99:0/3616281939 wait complete. 2023-10-08T00:15:26.459 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T00:15:26.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.575+0000 7f38fb52d700 1 Processor -- start 2023-10-08T00:15:26.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.576+0000 7f38fb52d700 1 -- start start 2023-10-08T00:15:26.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.576+0000 7f38fb52d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f38f40c5180 0x7f38f40b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:26.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.576+0000 7f38fb52d700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38f40af8e0 con 0x7f38f40cb5d0 2023-10-08T00:15:26.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.576+0000 7f38fb52d700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38f40c9960 con 0x7f38f40cb9a0 2023-10-08T00:15:26.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.576+0000 7f38fb52d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38f40c3610 con 0x7f38f40c5180 2023-10-08T00:15:26.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.577+0000 7f38f92c9700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f38f40cb5d0 0x7f38f40c4a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:52156/0 (socket says 172.21.15.99:52156) 2023-10-08T00:15:26.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.577+0000 7f38f92c9700 1 -- 172.21.15.99:0/1341925617 learned_addr learned my addr 172.21.15.99:0/1341925617 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:15:26.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.577+0000 7f38f8ac8700 1 --2- 172.21.15.99:0/1341925617 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f38f40c5180 0x7f38f40b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:26.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.577+0000 7f38ebfff700 1 -- 172.21.15.99:0/1341925617 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3956515069 0 0) 0x7f38f40af8e0 con 0x7f38f40cb5d0 2023-10-08T00:15:26.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.577+0000 7f38ebfff700 1 -- 172.21.15.99:0/1341925617 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38e0003570 con 0x7f38f40cb5d0 2023-10-08T00:15:26.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.577+0000 7f38ebfff700 1 -- 172.21.15.99:0/1341925617 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1571890622 0 0) 0x7f38f40c9960 con 0x7f38f40cb9a0 2023-10-08T00:15:26.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.577+0000 7f38ebfff700 1 -- 172.21.15.99:0/1341925617 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38f40af8e0 con 0x7f38f40cb9a0 2023-10-08T00:15:26.580 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.577+0000 7f38f8ac8700 1 -- 172.21.15.99:0/1341925617 >> v1:172.21.15.191:6789/0 conn(0x7f38f40cb9a0 legacy=0x7f38f41597f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:26.580 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.577+0000 7f38f8ac8700 1 -- 172.21.15.99:0/1341925617 >> v1:172.21.15.99:6789/0 conn(0x7f38f40cb5d0 legacy=0x7f38f40c4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:26.580 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.577+0000 7f38f8ac8700 1 -- 172.21.15.99:0/1341925617 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f40c9bb0 con 0x7f38f40c5180 2023-10-08T00:15:26.580 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.577+0000 7f38f8ac8700 1 --2- 172.21.15.99:0/1341925617 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f38f40c5180 0x7f38f40b0cf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f38e4009910 tx=0x7f38e40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c50e3ea3c02f2061 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:26.580 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.577+0000 7f38ebfff700 1 -- 172.21.15.99:0/1341925617 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e400b690 con 0x7f38f40c5180 2023-10-08T00:15:26.580 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.577+0000 7f38ebfff700 1 -- 172.21.15.99:0/1341925617 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e400b7f0 con 0x7f38f40c5180 2023-10-08T00:15:26.580 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.577+0000 7f38ebfff700 1 -- 172.21.15.99:0/1341925617 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e400bb70 con 0x7f38f40c5180 2023-10-08T00:15:26.580 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.578+0000 7f38fb52d700 1 -- 172.21.15.99:0/1341925617 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f38f40c5180 msgr2=0x7f38f40b0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:26.580 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.578+0000 7f38fb52d700 1 --2- 172.21.15.99:0/1341925617 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f38f40c5180 0x7f38f40b0cf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f38e4009910 tx=0x7f38e40091a0 comp rx=0 tx=0).stop 2023-10-08T00:15:26.580 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.578+0000 7f38fb52d700 1 -- 172.21.15.99:0/1341925617 shutdown_connections 2023-10-08T00:15:26.580 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.578+0000 7f38fb52d700 1 --2- 172.21.15.99:0/1341925617 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f38f40c5180 0x7f38f40b0cf0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:26.581 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.578+0000 7f38fb52d700 1 -- 172.21.15.99:0/1341925617 >> 172.21.15.99:0/1341925617 conn(0x7f38f406f3e0 msgr2=0x7f38f406f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:26.581 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.578+0000 7f38fb52d700 1 -- 172.21.15.99:0/1341925617 shutdown_connections 2023-10-08T00:15:26.581 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.578+0000 7f38fb52d700 1 -- 172.21.15.99:0/1341925617 wait complete. 2023-10-08T00:15:26.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.579+0000 7f38fb52d700 1 Processor -- start 2023-10-08T00:15:26.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.579+0000 7f38fb52d700 1 -- start start 2023-10-08T00:15:26.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.580+0000 7f38fb52d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f38f40c5180 0x7f38f415cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:26.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.580+0000 7f38fb52d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38f40cb5d0 0x7f38f4068870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:26.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.580+0000 7f38fb52d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38f40cb9a0 0x7f38f40cef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:26.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.580+0000 7f38fb52d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f406a360 con 0x7f38f40cb9a0 2023-10-08T00:15:26.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.580+0000 7f38fb52d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f4069ed0 con 0x7f38f40cb5d0 2023-10-08T00:15:26.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.580+0000 7f38fb52d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38f40692c0 con 0x7f38f40c5180 2023-10-08T00:15:26.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.580+0000 7f38f9aca700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38f40cb9a0 0x7f38f40cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:26.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.580+0000 7f38f9aca700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38f40cb9a0 0x7f38f40cef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55856/0 (socket says 172.21.15.99:55856) 2023-10-08T00:15:26.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.580+0000 7f38f9aca700 1 -- 172.21.15.99:0/2994893282 learned_addr learned my addr 172.21.15.99:0/2994893282 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:26.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.580+0000 7f38f8ac8700 1 --2- 172.21.15.99:0/2994893282 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38f40cb5d0 0x7f38f4068870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:26.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.580+0000 7f38f92c9700 1 --2- 172.21.15.99:0/2994893282 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f38f40c5180 0x7f38f415cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:26.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.580+0000 7f38f9aca700 1 -- 172.21.15.99:0/2994893282 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f38f40c5180 msgr2=0x7f38f415cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:26.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.580+0000 7f38f9aca700 1 --2- 172.21.15.99:0/2994893282 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f38f40c5180 0x7f38f415cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:26.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.580+0000 7f38f9aca700 1 -- 172.21.15.99:0/2994893282 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38f40cb5d0 msgr2=0x7f38f4068870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:26.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.580+0000 7f38f9aca700 1 --2- 172.21.15.99:0/2994893282 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38f40cb5d0 0x7f38f4068870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:26.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.580+0000 7f38f9aca700 1 -- 172.21.15.99:0/2994893282 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e4009420 con 0x7f38f40cb9a0 2023-10-08T00:15:26.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.580+0000 7f38f8ac8700 1 --2- 172.21.15.99:0/2994893282 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38f40cb5d0 0x7f38f4068870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:15:26.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.580+0000 7f38f92c9700 1 --2- 172.21.15.99:0/2994893282 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f38f40c5180 0x7f38f415cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:15:26.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.580+0000 7f38f9aca700 1 --2- 172.21.15.99:0/2994893282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38f40cb9a0 0x7f38f40cef10 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f38ec00c4e0 tx=0x7f38ec007d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:26.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.580+0000 7f38e97fa700 1 -- 172.21.15.99:0/2994893282 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ec011660 con 0x7f38f40cb9a0 2023-10-08T00:15:26.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.580+0000 7f38e97fa700 1 -- 172.21.15.99:0/2994893282 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38ec0117e0 con 0x7f38f40cb9a0 2023-10-08T00:15:26.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.580+0000 7f38fb52d700 1 -- 172.21.15.99:0/2994893282 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38f40695a0 con 0x7f38f40cb9a0 2023-10-08T00:15:26.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.580+0000 7f38e97fa700 1 -- 172.21.15.99:0/2994893282 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ec011ba0 con 0x7f38f40cb9a0 2023-10-08T00:15:26.586 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.581+0000 7f38fb52d700 1 -- 172.21.15.99:0/2994893282 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38f4069b60 con 0x7f38f40cb9a0 2023-10-08T00:15:26.586 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.581+0000 7f38fb52d700 1 -- 172.21.15.99:0/2994893282 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38f406a060 con 0x7f38f40cb9a0 2023-10-08T00:15:26.586 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.582+0000 7f38e97fa700 1 -- 172.21.15.99:0/2994893282 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f38ec012420 con 0x7f38f40cb9a0 2023-10-08T00:15:26.586 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.582+0000 7f38e97fa700 1 --2- 172.21.15.99:0/2994893282 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f38d8066f30 0x7f38d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:26.586 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.582+0000 7f38e97fa700 1 -- 172.21.15.99:0/2994893282 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f38ec002030 con 0x7f38f40cb9a0 2023-10-08T00:15:26.586 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.583+0000 7f38f92c9700 1 --2- 172.21.15.99:0/2994893282 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f38d8066f30 0x7f38d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:26.586 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.583+0000 7f38e97fa700 1 -- 172.21.15.99:0/2994893282 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=90}) v3 -- 0x7f38d806ea50 con 0x7f38f40cb9a0 2023-10-08T00:15:26.586 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.583+0000 7f38f92c9700 1 --2- 172.21.15.99:0/2994893282 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f38d8066f30 0x7f38d80693e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f38f0005c50 tx=0x7f38f0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:26.587 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.587+0000 7f38e97fa700 1 -- 172.21.15.99:0/2994893282 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f38ec051530 con 0x7f38f40cb9a0 2023-10-08T00:15:26.770 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.770+0000 7f38fb52d700 1 -- 172.21.15.99:0/2994893282 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f38f404c3c0 con 0x7f38f40cb9a0 2023-10-08T00:15:26.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.770+0000 7f38e97fa700 1 -- 172.21.15.99:0/2994893282 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7f38ec054b50 con 0x7f38f40cb9a0 2023-10-08T00:15:26.771 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2023-10-08T00:15:26.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.772+0000 7f38fb52d700 1 -- 172.21.15.99:0/2994893282 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f38d8066f30 msgr2=0x7f38d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:26.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.772+0000 7f38fb52d700 1 --2- 172.21.15.99:0/2994893282 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f38d8066f30 0x7f38d80693e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f38f0005c50 tx=0x7f38f0009040 comp rx=0 tx=0).stop 2023-10-08T00:15:26.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.772+0000 7f38fb52d700 1 -- 172.21.15.99:0/2994893282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38f40cb9a0 msgr2=0x7f38f40cef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:26.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.772+0000 7f38fb52d700 1 --2- 172.21.15.99:0/2994893282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38f40cb9a0 0x7f38f40cef10 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f38ec00c4e0 tx=0x7f38ec007d80 comp rx=0 tx=0).stop 2023-10-08T00:15:26.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.772+0000 7f38fb52d700 1 -- 172.21.15.99:0/2994893282 shutdown_connections 2023-10-08T00:15:26.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.772+0000 7f38fb52d700 1 --2- 172.21.15.99:0/2994893282 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f38d8066f30 0x7f38d80693e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:26.774 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.772+0000 7f38fb52d700 1 --2- 172.21.15.99:0/2994893282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38f40cb9a0 0x7f38f40cef10 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:26.774 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.772+0000 7f38fb52d700 1 --2- 172.21.15.99:0/2994893282 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f38f40c5180 0x7f38f415cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:26.774 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.772+0000 7f38fb52d700 1 --2- 172.21.15.99:0/2994893282 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38f40cb5d0 0x7f38f4068870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:26.774 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.772+0000 7f38fb52d700 1 -- 172.21.15.99:0/2994893282 >> 172.21.15.99:0/2994893282 conn(0x7f38f406f3e0 msgr2=0x7f38f40b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:26.774 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.772+0000 7f38fb52d700 1 -- 172.21.15.99:0/2994893282 shutdown_connections 2023-10-08T00:15:26.774 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.772+0000 7f38fb52d700 1 -- 172.21.15.99:0/2994893282 wait complete. 2023-10-08T00:15:26.785 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:15:26.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.898+0000 7fac1dc7d700 1 Processor -- start 2023-10-08T00:15:26.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.898+0000 7fac1dc7d700 1 -- start start 2023-10-08T00:15:26.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.899+0000 7fac1dc7d700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fac18152010 0x7fac1815b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:26.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.899+0000 7fac1dc7d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fac181523e0 0x7fac1806c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:26.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.899+0000 7fac1dc7d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fac18158e50 0x7fac180cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:26.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.899+0000 7fac1dc7d700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac1806c010 con 0x7fac18152010 2023-10-08T00:15:26.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.899+0000 7fac1dc7d700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac1806bd10 con 0x7fac18158e50 2023-10-08T00:15:26.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.899+0000 7fac1dc7d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac1815c060 con 0x7fac181523e0 2023-10-08T00:15:26.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.899+0000 7fac177fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fac18152010 0x7fac1815b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:26.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.899+0000 7fac177fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fac18152010 0x7fac1815b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55870/0 (socket says 172.21.15.99:55870) 2023-10-08T00:15:26.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.899+0000 7fac177fe700 1 -- 172.21.15.99:0/4226420870 learned_addr learned my addr 172.21.15.99:0/4226420870 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:26.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.899+0000 7fac16ffd700 1 --2- 172.21.15.99:0/4226420870 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fac181523e0 0x7fac1806c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:26.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.899+0000 7fac17fff700 1 --2- 172.21.15.99:0/4226420870 >> v2:172.21.15.191:3300/0 conn(0x7fac18158e50 0x7fac180cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:26.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.900+0000 7fac177fe700 1 -- 172.21.15.99:0/4226420870 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fac181523e0 msgr2=0x7fac1806c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:26.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.900+0000 7fac177fe700 1 --2- 172.21.15.99:0/4226420870 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fac181523e0 0x7fac1806c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:26.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.900+0000 7fac177fe700 1 -- 172.21.15.99:0/4226420870 >> v2:172.21.15.191:3300/0 conn(0x7fac18158e50 msgr2=0x7fac180cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:26.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.900+0000 7fac177fe700 1 --2- 172.21.15.99:0/4226420870 >> v2:172.21.15.191:3300/0 conn(0x7fac18158e50 0x7fac180cdf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:26.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.900+0000 7fac177fe700 1 -- 172.21.15.99:0/4226420870 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac1815c340 con 0x7fac18152010 2023-10-08T00:15:26.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.900+0000 7fac177fe700 1 --2- 172.21.15.99:0/4226420870 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac18152010 0x7fac1815b810 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fac18155d40 tx=0x7fac08009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba3a1a43903eee39 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:26.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.900+0000 7fac167fc700 1 -- 172.21.15.99:0/4226420870 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0800e070 con 0x7fac18152010 2023-10-08T00:15:26.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.900+0000 7fac167fc700 1 -- 172.21.15.99:0/4226420870 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac0800b970 con 0x7fac18152010 2023-10-08T00:15:26.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.900+0000 7fac167fc700 1 -- 172.21.15.99:0/4226420870 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac080092e0 con 0x7fac18152010 2023-10-08T00:15:26.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.900+0000 7fac1dc7d700 1 -- 172.21.15.99:0/4226420870 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac18152010 msgr2=0x7fac1815b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:26.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.900+0000 7fac1dc7d700 1 --2- 172.21.15.99:0/4226420870 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac18152010 0x7fac1815b810 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fac18155d40 tx=0x7fac08009840 comp rx=0 tx=0).stop 2023-10-08T00:15:26.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.900+0000 7fac1dc7d700 1 -- 172.21.15.99:0/4226420870 shutdown_connections 2023-10-08T00:15:26.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.900+0000 7fac1dc7d700 1 --2- 172.21.15.99:0/4226420870 >> v2:172.21.15.191:3300/0 conn(0x7fac18158e50 0x7fac180cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:26.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.900+0000 7fac1dc7d700 1 --2- 172.21.15.99:0/4226420870 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac18152010 0x7fac1815b810 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:26.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.900+0000 7fac1dc7d700 1 --2- 172.21.15.99:0/4226420870 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fac181523e0 0x7fac1806c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:26.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.900+0000 7fac1dc7d700 1 -- 172.21.15.99:0/4226420870 >> 172.21.15.99:0/4226420870 conn(0x7fac18069870 msgr2=0x7fac18069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:26.909 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.900+0000 7fac1dc7d700 1 -- 172.21.15.99:0/4226420870 shutdown_connections 2023-10-08T00:15:26.909 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.900+0000 7fac1dc7d700 1 -- 172.21.15.99:0/4226420870 wait complete. 2023-10-08T00:15:26.909 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.902+0000 7fac1dc7d700 1 Processor -- start 2023-10-08T00:15:26.909 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.902+0000 7fac1dc7d700 1 -- start start 2023-10-08T00:15:26.909 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.902+0000 7fac1dc7d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac18152010 0x7fac18168d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:26.909 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.902+0000 7fac1dc7d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fac181523e0 0x7fac1815a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:26.909 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.902+0000 7fac1dc7d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac18158e50 0x7fac1815aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:26.909 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.902+0000 7fac1dc7d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac18169920 con 0x7fac18158e50 2023-10-08T00:15:26.909 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.902+0000 7fac1dc7d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1814fd70 con 0x7fac18152010 2023-10-08T00:15:26.909 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.902+0000 7fac1dc7d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac18169b90 con 0x7fac181523e0 2023-10-08T00:15:26.909 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.902+0000 7fac17fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac18158e50 0x7fac1815aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:26.909 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.903+0000 7fac17fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac18158e50 0x7fac1815aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55884/0 (socket says 172.21.15.99:55884) 2023-10-08T00:15:26.910 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.903+0000 7fac17fff700 1 -- 172.21.15.99:0/254428466 learned_addr learned my addr 172.21.15.99:0/254428466 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:26.910 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.903+0000 7fac177fe700 1 --2- 172.21.15.99:0/254428466 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac18152010 0x7fac18168d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:26.910 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.903+0000 7fac16ffd700 1 --2- 172.21.15.99:0/254428466 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fac181523e0 0x7fac1815a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:26.910 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.903+0000 7fac17fff700 1 -- 172.21.15.99:0/254428466 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fac181523e0 msgr2=0x7fac1815a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:26.910 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.903+0000 7fac17fff700 1 --2- 172.21.15.99:0/254428466 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fac181523e0 0x7fac1815a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:26.910 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.903+0000 7fac17fff700 1 -- 172.21.15.99:0/254428466 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac18152010 msgr2=0x7fac18168d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:26.910 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.903+0000 7fac17fff700 1 --2- 172.21.15.99:0/254428466 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac18152010 0x7fac18168d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:26.910 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.903+0000 7fac17fff700 1 -- 172.21.15.99:0/254428466 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac08009ae0 con 0x7fac18158e50 2023-10-08T00:15:26.910 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.903+0000 7fac17fff700 1 --2- 172.21.15.99:0/254428466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac18158e50 0x7fac1815aa70 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fac1000e8d0 tx=0x7fac100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:26.910 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.903+0000 7fabff7fe700 1 -- 172.21.15.99:0/254428466 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac10013680 con 0x7fac18158e50 2023-10-08T00:15:26.910 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.903+0000 7fabff7fe700 1 -- 172.21.15.99:0/254428466 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac100137e0 con 0x7fac18158e50 2023-10-08T00:15:26.911 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.903+0000 7fac1dc7d700 1 -- 172.21.15.99:0/254428466 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac18169e70 con 0x7fac18158e50 2023-10-08T00:15:26.911 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.903+0000 7fabff7fe700 1 -- 172.21.15.99:0/254428466 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac10013ba0 con 0x7fac18158e50 2023-10-08T00:15:26.911 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.903+0000 7fac1dc7d700 1 -- 172.21.15.99:0/254428466 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac18150050 con 0x7fac18158e50 2023-10-08T00:15:26.911 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.904+0000 7fac1dc7d700 1 -- 172.21.15.99:0/254428466 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac1804c3c0 con 0x7fac18158e50 2023-10-08T00:15:26.911 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.904+0000 7fabff7fe700 1 -- 172.21.15.99:0/254428466 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fac100123d0 con 0x7fac18158e50 2023-10-08T00:15:26.911 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.905+0000 7fabff7fe700 1 --2- 172.21.15.99:0/254428466 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fac04066f30 0x7fac040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:26.911 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.905+0000 7fabff7fe700 1 -- 172.21.15.99:0/254428466 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fac10084660 con 0x7fac18158e50 2023-10-08T00:15:26.911 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.905+0000 7fac177fe700 1 --2- 172.21.15.99:0/254428466 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fac04066f30 0x7fac040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:26.911 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.905+0000 7fabff7fe700 1 -- 172.21.15.99:0/254428466 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=90}) v3 -- 0x7fac0406ea50 con 0x7fac18158e50 2023-10-08T00:15:26.911 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.905+0000 7fac177fe700 1 --2- 172.21.15.99:0/254428466 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fac04066f30 0x7fac040693e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fac08009880 tx=0x7fac08018000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:26.912 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:26.909+0000 7fabff7fe700 1 -- 172.21.15.99:0/254428466 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fac100518d0 con 0x7fac18158e50 2023-10-08T00:15:27.131 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.131+0000 7fac1dc7d700 1 -- 172.21.15.99:0/254428466 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fac1806d600 con 0x7fac18158e50 2023-10-08T00:15:27.132 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.131+0000 7fabff7fe700 1 -- 172.21.15.99:0/254428466 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4599 (secure 0 0 0) 0x7fac10017090 con 0x7fac18158e50 2023-10-08T00:15:27.132 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:15:27.132 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5650,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/2166055423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2166055423},{"type":"v1","addr":"172.21.15.99:6834","nonce":2166055423}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5665,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/4087832300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":4087832300},{"type":"v1","addr":"172.21.15.191:6836","nonce":4087832300}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5668,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/730261712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":730261712},{"type":"v1","addr":"172.21.15.191:6837","nonce":730261712}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:14:43.671064+0000","modified":"2023-10-08T00:14:47.326922+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6039},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6039":{"gid":6039,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/104443412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":104443412},{"type":"v1","addr":"172.21.15.99:6835","nonce":104443412}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-08T00:15:27.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.133+0000 7fac1dc7d700 1 -- 172.21.15.99:0/254428466 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fac04066f30 msgr2=0x7fac040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:27.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.133+0000 7fac1dc7d700 1 --2- 172.21.15.99:0/254428466 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fac04066f30 0x7fac040693e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fac08009880 tx=0x7fac08018000 comp rx=0 tx=0).stop 2023-10-08T00:15:27.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.133+0000 7fac1dc7d700 1 -- 172.21.15.99:0/254428466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac18158e50 msgr2=0x7fac1815aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:27.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.133+0000 7fac1dc7d700 1 --2- 172.21.15.99:0/254428466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac18158e50 0x7fac1815aa70 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fac1000e8d0 tx=0x7fac100091a0 comp rx=0 tx=0).stop 2023-10-08T00:15:27.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.133+0000 7fac1dc7d700 1 -- 172.21.15.99:0/254428466 shutdown_connections 2023-10-08T00:15:27.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.133+0000 7fac1dc7d700 1 --2- 172.21.15.99:0/254428466 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fac04066f30 0x7fac040693e0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:27.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.133+0000 7fac1dc7d700 1 --2- 172.21.15.99:0/254428466 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac18152010 0x7fac18168d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:27.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.133+0000 7fac1dc7d700 1 --2- 172.21.15.99:0/254428466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac18158e50 0x7fac1815aa70 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:27.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.133+0000 7fac1dc7d700 1 --2- 172.21.15.99:0/254428466 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fac181523e0 0x7fac1815a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:27.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.133+0000 7fac1dc7d700 1 -- 172.21.15.99:0/254428466 >> 172.21.15.99:0/254428466 conn(0x7fac18069870 msgr2=0x7fac18069cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:27.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.134+0000 7fac1dc7d700 1 -- 172.21.15.99:0/254428466 shutdown_connections 2023-10-08T00:15:27.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.134+0000 7fac1dc7d700 1 -- 172.21.15.99:0/254428466 wait complete. 2023-10-08T00:15:27.136 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 40 2023-10-08T00:15:27.146 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:15:27.152 INFO:tasks.ceph.mon.a.smithi099.stderr:2023-10-08T00:15:27.152+0000 7f6c637a7700 -1 log_channel(cluster) log [ERR] : Health check update: 1 full osd(s) (OSD_FULL) 2023-10-08T00:15:27.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.261+0000 7f9acf716700 1 Processor -- start 2023-10-08T00:15:27.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.261+0000 7f9acf716700 1 -- start start 2023-10-08T00:15:27.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.262+0000 7f9acf716700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9ac8157c40 0x7f9ac8158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:27.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.262+0000 7f9acf716700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ac815cbb0 con 0x7f9ac8157870 2023-10-08T00:15:27.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.262+0000 7f9acf716700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ac815bfa0 con 0x7f9ac815a620 2023-10-08T00:15:27.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.262+0000 7f9acf716700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac815bdb0 con 0x7f9ac8157c40 2023-10-08T00:15:27.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.262+0000 7f9acd4b2700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f9ac8157870 0x7f9ac815b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:52160/0 (socket says 172.21.15.99:52160) 2023-10-08T00:15:27.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.262+0000 7f9acd4b2700 1 -- 172.21.15.99:0/73014553 learned_addr learned my addr 172.21.15.99:0/73014553 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:15:27.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.262+0000 7f9abffff700 1 -- 172.21.15.99:0/73014553 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2972148532 0 0) 0x7f9ac815cbb0 con 0x7f9ac8157870 2023-10-08T00:15:27.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.262+0000 7f9abffff700 1 -- 172.21.15.99:0/73014553 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ab4003570 con 0x7f9ac8157870 2023-10-08T00:15:27.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.262+0000 7f9acdcb3700 1 --2- 172.21.15.99:0/73014553 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9ac8157c40 0x7f9ac8158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:27.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.262+0000 7f9abffff700 1 -- 172.21.15.99:0/73014553 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2014950021 0 0) 0x7f9ab4003570 con 0x7f9ac8157870 2023-10-08T00:15:27.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.262+0000 7f9abffff700 1 -- 172.21.15.99:0/73014553 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9ac8157c40 msgr2=0x7f9ac8158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:27.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.262+0000 7f9abffff700 1 --2- 172.21.15.99:0/73014553 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9ac8157c40 0x7f9ac8158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:27.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.262+0000 7f9abffff700 1 -- 172.21.15.99:0/73014553 >> v1:172.21.15.191:6789/0 conn(0x7f9ac815a620 legacy=0x7f9ac815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:27.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.262+0000 7f9abffff700 1 -- 172.21.15.99:0/73014553 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac815c280 con 0x7f9ac8157870 2023-10-08T00:15:27.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.262+0000 7f9abffff700 1 -- 172.21.15.99:0/73014553 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f9ac4001470 con 0x7f9ac8157870 2023-10-08T00:15:27.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.262+0000 7f9abffff700 1 -- 172.21.15.99:0/73014553 >> v1:172.21.15.99:6789/0 conn(0x7f9ac8157870 legacy=0x7f9ac815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:27.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.262+0000 7f9abffff700 1 --2- 172.21.15.99:0/73014553 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9ac8157c40 0x7f9ab4003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:27.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.263+0000 7f9abffff700 1 --2- 172.21.15.99:0/73014553 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ab4004010 0x7f9ab4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:27.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.263+0000 7f9abffff700 1 --2- 172.21.15.99:0/73014553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ab4004d50 0x7f9ab40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:27.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.263+0000 7f9abffff700 1 -- 172.21.15.99:0/73014553 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab400bbf0 con 0x7f9ab4004d50 2023-10-08T00:15:27.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.263+0000 7f9abffff700 1 -- 172.21.15.99:0/73014553 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab400bd30 con 0x7f9ab4004010 2023-10-08T00:15:27.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.263+0000 7f9abffff700 1 -- 172.21.15.99:0/73014553 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab400bef0 con 0x7f9ac8157c40 2023-10-08T00:15:27.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.263+0000 7f9acdcb3700 1 --2- 172.21.15.99:0/73014553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ab4004d50 0x7f9ab40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:27.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.263+0000 7f9acd4b2700 1 --2- 172.21.15.99:0/73014553 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9ac8157c40 0x7f9ab4003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:27.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.263+0000 7f9acccb1700 1 --2- 172.21.15.99:0/73014553 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ab4004010 0x7f9ab4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:27.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.263+0000 7f9acdcb3700 1 -- 172.21.15.99:0/73014553 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9ac8157c40 msgr2=0x7f9ab4003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:27.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.263+0000 7f9acdcb3700 1 --2- 172.21.15.99:0/73014553 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9ac8157c40 0x7f9ab4003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:27.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.263+0000 7f9acdcb3700 1 -- 172.21.15.99:0/73014553 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ab4004010 msgr2=0x7f9ab4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:27.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.263+0000 7f9acdcb3700 1 --2- 172.21.15.99:0/73014553 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ab4004010 0x7f9ab4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:27.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.263+0000 7f9acdcb3700 1 -- 172.21.15.99:0/73014553 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab4003570 con 0x7f9ab4004d50 2023-10-08T00:15:27.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.263+0000 7f9acdcb3700 1 --2- 172.21.15.99:0/73014553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ab4004d50 0x7f9ab40051c0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f9ac00094d0 tx=0x7f9ac000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a25c0e4389eb5dbe server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:27.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.263+0000 7f9abffff700 1 -- 172.21.15.99:0/73014553 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac000ea60 con 0x7f9ab4004d50 2023-10-08T00:15:27.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.263+0000 7f9abffff700 1 -- 172.21.15.99:0/73014553 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ac000f040 con 0x7f9ab4004d50 2023-10-08T00:15:27.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.263+0000 7f9abffff700 1 -- 172.21.15.99:0/73014553 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac000edf0 con 0x7f9ab4004d50 2023-10-08T00:15:27.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.264+0000 7f9acf716700 1 -- 172.21.15.99:0/73014553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ab4004d50 msgr2=0x7f9ab40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:27.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.264+0000 7f9acf716700 1 --2- 172.21.15.99:0/73014553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ab4004d50 0x7f9ab40051c0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f9ac00094d0 tx=0x7f9ac000b040 comp rx=0 tx=0).stop 2023-10-08T00:15:27.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.264+0000 7f9acdcb3700 1 -- 172.21.15.99:0/73014553 reap_dead start 2023-10-08T00:15:27.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.264+0000 7f9acf716700 1 -- 172.21.15.99:0/73014553 shutdown_connections 2023-10-08T00:15:27.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.264+0000 7f9acf716700 1 -- 172.21.15.99:0/73014553 >> 172.21.15.99:0/73014553 conn(0x7f9ac8068ce0 msgr2=0x7f9ac80690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:27.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.264+0000 7f9acf716700 1 -- 172.21.15.99:0/73014553 shutdown_connections 2023-10-08T00:15:27.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.264+0000 7f9acf716700 1 -- 172.21.15.99:0/73014553 wait complete. 2023-10-08T00:15:27.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.265+0000 7f9acf716700 1 Processor -- start 2023-10-08T00:15:27.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.265+0000 7f9acf716700 1 -- start start 2023-10-08T00:15:27.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.266+0000 7f9acf716700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9ab4004d50 0x7f9ac80aef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:27.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.266+0000 7f9acf716700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ac816b240 0x7f9ac816d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:27.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.266+0000 7f9acf716700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ac816b6b0 0x7f9ac816dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:27.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.266+0000 7f9acf716700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac80af470 con 0x7f9ac816b240 2023-10-08T00:15:27.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.266+0000 7f9acf716700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac80a77b0 con 0x7f9ac816b6b0 2023-10-08T00:15:27.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.266+0000 7f9acf716700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac80a7a90 con 0x7f9ab4004d50 2023-10-08T00:15:27.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.266+0000 7f9acccb1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ac816b240 0x7f9ac816d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:27.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.266+0000 7f9acccb1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ac816b240 0x7f9ac816d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55904/0 (socket says 172.21.15.99:55904) 2023-10-08T00:15:27.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.266+0000 7f9acccb1700 1 -- 172.21.15.99:0/2338364909 learned_addr learned my addr 172.21.15.99:0/2338364909 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:27.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.266+0000 7f9acd4b2700 1 --2- 172.21.15.99:0/2338364909 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9ab4004d50 0x7f9ac80aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:27.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.266+0000 7f9acdcb3700 1 --2- 172.21.15.99:0/2338364909 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ac816b6b0 0x7f9ac816dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:27.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.266+0000 7f9acccb1700 1 -- 172.21.15.99:0/2338364909 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9ab4004d50 msgr2=0x7f9ac80aef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:27.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.266+0000 7f9acccb1700 1 --2- 172.21.15.99:0/2338364909 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9ab4004d50 0x7f9ac80aef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:27.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.266+0000 7f9acccb1700 1 -- 172.21.15.99:0/2338364909 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ac816b6b0 msgr2=0x7f9ac816dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:27.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.266+0000 7f9acccb1700 1 --2- 172.21.15.99:0/2338364909 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ac816b6b0 0x7f9ac816dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:27.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.266+0000 7f9acccb1700 1 -- 172.21.15.99:0/2338364909 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac80a7d70 con 0x7f9ac816b240 2023-10-08T00:15:27.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.266+0000 7f9acccb1700 1 --2- 172.21.15.99:0/2338364909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ac816b240 0x7f9ac816d730 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8002920 tx=0x7f9ab8007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:27.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.267+0000 7f9abd7fa700 1 -- 172.21.15.99:0/2338364909 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab8013600 con 0x7f9ac816b240 2023-10-08T00:15:27.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.267+0000 7f9abd7fa700 1 -- 172.21.15.99:0/2338364909 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ab8013760 con 0x7f9ac816b240 2023-10-08T00:15:27.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.267+0000 7f9acf716700 1 -- 172.21.15.99:0/2338364909 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ac80a8050 con 0x7f9ac816b240 2023-10-08T00:15:27.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.267+0000 7f9abd7fa700 1 -- 172.21.15.99:0/2338364909 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab8013b00 con 0x7f9ac816b240 2023-10-08T00:15:27.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.267+0000 7f9acf716700 1 -- 172.21.15.99:0/2338364909 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ac80a8850 con 0x7f9ac816b240 2023-10-08T00:15:27.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.268+0000 7f9acf716700 1 -- 172.21.15.99:0/2338364909 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ac80a86a0 con 0x7f9ac816b240 2023-10-08T00:15:27.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.268+0000 7f9abd7fa700 1 -- 172.21.15.99:0/2338364909 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9ab800f3e0 con 0x7f9ac816b240 2023-10-08T00:15:27.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.268+0000 7f9abd7fa700 1 --2- 172.21.15.99:0/2338364909 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9aac066f30 0x7f9aac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:27.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.268+0000 7f9abd7fa700 1 -- 172.21.15.99:0/2338364909 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9ab80841a0 con 0x7f9ac816b240 2023-10-08T00:15:27.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.269+0000 7f9abd7fa700 1 -- 172.21.15.99:0/2338364909 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=91}) v3 -- 0x7f9aac06ea50 con 0x7f9ac816b240 2023-10-08T00:15:27.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.269+0000 7f9acd4b2700 1 --2- 172.21.15.99:0/2338364909 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9aac066f30 0x7f9aac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:27.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.269+0000 7f9acd4b2700 1 --2- 172.21.15.99:0/2338364909 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9aac066f30 0x7f9aac0693e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4000e50 tx=0x7f9ac4001e50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:27.275 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.273+0000 7f9abd7fa700 1 -- 172.21.15.99:0/2338364909 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9ab80514c0 con 0x7f9ac816b240 2023-10-08T00:15:27.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.457+0000 7f9acf716700 1 -- 172.21.15.99:0/2338364909 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ac804c3c0 con 0x7f9ac816b240 2023-10-08T00:15:27.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.458+0000 7f9abd7fa700 1 -- 172.21.15.99:0/2338364909 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+15405 (secure 0 0 0) 0x7f9ab8054ae0 con 0x7f9ac816b240 2023-10-08T00:15:27.459 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:15:27.459 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":90,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:15:27.153339+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:14:41.496823+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-08T00:14:42.640029+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","full","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["backfillfull","exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["backfillfull","exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:15:27.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.460+0000 7f9acf716700 1 -- 172.21.15.99:0/2338364909 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9aac066f30 msgr2=0x7f9aac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:27.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.460+0000 7f9acf716700 1 --2- 172.21.15.99:0/2338364909 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9aac066f30 0x7f9aac0693e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4000e50 tx=0x7f9ac4001e50 comp rx=0 tx=0).stop 2023-10-08T00:15:27.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.460+0000 7f9acf716700 1 -- 172.21.15.99:0/2338364909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ac816b240 msgr2=0x7f9ac816d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:27.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.460+0000 7f9acf716700 1 --2- 172.21.15.99:0/2338364909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ac816b240 0x7f9ac816d730 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8002920 tx=0x7f9ab8007d30 comp rx=0 tx=0).stop 2023-10-08T00:15:27.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.460+0000 7f9acf716700 1 -- 172.21.15.99:0/2338364909 shutdown_connections 2023-10-08T00:15:27.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.460+0000 7f9acf716700 1 --2- 172.21.15.99:0/2338364909 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9aac066f30 0x7f9aac0693e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:27.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.460+0000 7f9acf716700 1 --2- 172.21.15.99:0/2338364909 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ac816b6b0 0x7f9ac816dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:27.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.460+0000 7f9acf716700 1 --2- 172.21.15.99:0/2338364909 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9ab4004d50 0x7f9ac80aef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:27.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.460+0000 7f9acf716700 1 --2- 172.21.15.99:0/2338364909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ac816b240 0x7f9ac816d730 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:27.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.460+0000 7f9acf716700 1 -- 172.21.15.99:0/2338364909 >> 172.21.15.99:0/2338364909 conn(0x7f9ac8068ce0 msgr2=0x7f9ac80690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:27.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.460+0000 7f9acf716700 1 -- 172.21.15.99:0/2338364909 shutdown_connections 2023-10-08T00:15:27.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.460+0000 7f9acf716700 1 -- 172.21.15.99:0/2338364909 wait complete. 2023-10-08T00:15:27.473 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T00:15:27.473 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:15:27.627 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.627+0000 7efd89ba2700 1 Processor -- start 2023-10-08T00:15:27.628 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.627+0000 7efd89ba2700 1 -- start start 2023-10-08T00:15:27.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.627+0000 7efd89ba2700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7efd840cd6e0 0x7efd840cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:27.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.628+0000 7efd89ba2700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd840cb7c0 con 0x7efd840cdab0 2023-10-08T00:15:27.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.628+0000 7efd89ba2700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd8415e290 con 0x7efd8415cbd0 2023-10-08T00:15:27.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.628+0000 7efd89ba2700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd8415e850 con 0x7efd840cd6e0 2023-10-08T00:15:27.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.628+0000 7efd7bfff700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7efd840cdab0 0x7efd8415d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:52174/0 (socket says 172.21.15.99:52174) 2023-10-08T00:15:27.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.628+0000 7efd7bfff700 1 -- 172.21.15.99:0/811975644 learned_addr learned my addr 172.21.15.99:0/811975644 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:15:27.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.628+0000 7efd88ba0700 1 --2- 172.21.15.99:0/811975644 >> v2:172.21.15.191:3300/0 conn(0x7efd840cd6e0 0x7efd840cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:27.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.628+0000 7efd7b7fe700 1 -- 172.21.15.99:0/811975644 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 911322726 0 0) 0x7efd840cb7c0 con 0x7efd840cdab0 2023-10-08T00:15:27.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.628+0000 7efd7b7fe700 1 -- 172.21.15.99:0/811975644 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd6c003570 con 0x7efd840cdab0 2023-10-08T00:15:27.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.628+0000 7efd7b7fe700 1 -- 172.21.15.99:0/811975644 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 880114506 0 0) 0x7efd6c003570 con 0x7efd840cdab0 2023-10-08T00:15:27.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.628+0000 7efd7b7fe700 1 -- 172.21.15.99:0/811975644 >> v2:172.21.15.191:3300/0 conn(0x7efd840cd6e0 msgr2=0x7efd840cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:27.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.628+0000 7efd7b7fe700 1 --2- 172.21.15.99:0/811975644 >> v2:172.21.15.191:3300/0 conn(0x7efd840cd6e0 0x7efd840cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:27.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.628+0000 7efd7b7fe700 1 -- 172.21.15.99:0/811975644 >> v1:172.21.15.191:6789/0 conn(0x7efd8415cbd0 legacy=0x7efd8415d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:27.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.628+0000 7efd7b7fe700 1 -- 172.21.15.99:0/811975644 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd8415e570 con 0x7efd840cdab0 2023-10-08T00:15:27.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.628+0000 7efd7b7fe700 1 -- 172.21.15.99:0/811975644 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7efd74001470 con 0x7efd840cdab0 2023-10-08T00:15:27.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.628+0000 7efd7b7fe700 1 -- 172.21.15.99:0/811975644 >> v1:172.21.15.99:6789/0 conn(0x7efd840cdab0 legacy=0x7efd8415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:27.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.628+0000 7efd7b7fe700 1 --2- 172.21.15.99:0/811975644 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efd6c003a60 0x7efd6c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:27.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.628+0000 7efd7b7fe700 1 --2- 172.21.15.99:0/811975644 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efd6c004810 0x7efd6c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:27.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.628+0000 7efd7b7fe700 1 --2- 172.21.15.99:0/811975644 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd6c0051c0 0x7efd6c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:27.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.628+0000 7efd7b7fe700 1 -- 172.21.15.99:0/811975644 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd8415e850 con 0x7efd6c003a60 2023-10-08T00:15:27.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.628+0000 7efd7b7fe700 1 -- 172.21.15.99:0/811975644 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c00c0d0 con 0x7efd6c0051c0 2023-10-08T00:15:27.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.628+0000 7efd7b7fe700 1 -- 172.21.15.99:0/811975644 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c00c290 con 0x7efd6c004810 2023-10-08T00:15:27.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.628+0000 7efd88ba0700 1 --2- 172.21.15.99:0/811975644 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efd6c003a60 0x7efd6c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:27.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.628+0000 7efd893a1700 1 --2- 172.21.15.99:0/811975644 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd6c0051c0 0x7efd6c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:27.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.628+0000 7efd7bfff700 1 --2- 172.21.15.99:0/811975644 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efd6c004810 0x7efd6c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:27.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.629+0000 7efd88ba0700 1 -- 172.21.15.99:0/811975644 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efd6c004810 msgr2=0x7efd6c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:27.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.629+0000 7efd88ba0700 1 --2- 172.21.15.99:0/811975644 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efd6c004810 0x7efd6c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:27.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.629+0000 7efd88ba0700 1 -- 172.21.15.99:0/811975644 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd6c0051c0 msgr2=0x7efd6c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:27.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.629+0000 7efd88ba0700 1 --2- 172.21.15.99:0/811975644 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd6c0051c0 0x7efd6c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:27.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.629+0000 7efd88ba0700 1 -- 172.21.15.99:0/811975644 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd6c003570 con 0x7efd6c003a60 2023-10-08T00:15:27.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.629+0000 7efd893a1700 1 --2- 172.21.15.99:0/811975644 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd6c0051c0 0x7efd6c00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:15:27.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.629+0000 7efd893a1700 1 -- 172.21.15.99:0/811975644 reap_dead start 2023-10-08T00:15:27.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.629+0000 7efd88ba0700 1 --2- 172.21.15.99:0/811975644 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efd6c003a60 0x7efd6c003e70 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7efd80009c00 tx=0x7efd800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42569871fd0bb199 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:27.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.629+0000 7efd7b7fe700 1 -- 172.21.15.99:0/811975644 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8000e710 con 0x7efd6c003a60 2023-10-08T00:15:27.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.629+0000 7efd7b7fe700 1 -- 172.21.15.99:0/811975644 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd8000e870 con 0x7efd6c003a60 2023-10-08T00:15:27.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.629+0000 7efd7b7fe700 1 -- 172.21.15.99:0/811975644 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8000ec30 con 0x7efd6c003a60 2023-10-08T00:15:27.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.629+0000 7efd89ba2700 1 -- 172.21.15.99:0/811975644 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efd6c003a60 msgr2=0x7efd6c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:27.635 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.629+0000 7efd89ba2700 1 --2- 172.21.15.99:0/811975644 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efd6c003a60 0x7efd6c003e70 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7efd80009c00 tx=0x7efd800091a0 comp rx=0 tx=0).stop 2023-10-08T00:15:27.635 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.629+0000 7efd89ba2700 1 -- 172.21.15.99:0/811975644 shutdown_connections 2023-10-08T00:15:27.635 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.629+0000 7efd89ba2700 1 --2- 172.21.15.99:0/811975644 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efd6c003a60 0x7efd6c003e70 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:27.635 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.629+0000 7efd89ba2700 1 -- 172.21.15.99:0/811975644 >> 172.21.15.99:0/811975644 conn(0x7efd84067660 msgr2=0x7efd84067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:27.635 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.629+0000 7efd89ba2700 1 -- 172.21.15.99:0/811975644 shutdown_connections 2023-10-08T00:15:27.635 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.629+0000 7efd89ba2700 1 -- 172.21.15.99:0/811975644 wait complete. 2023-10-08T00:15:27.635 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.630+0000 7efd89ba2700 1 Processor -- start 2023-10-08T00:15:27.635 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.631+0000 7efd89ba2700 1 -- start start 2023-10-08T00:15:27.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.631+0000 7efd89ba2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd6c003a60 0x7efd840b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:27.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.631+0000 7efd89ba2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efd840ca200 0x7efd8416de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:27.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.631+0000 7efd89ba2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efd8416e390 0x7efd8416e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:27.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.631+0000 7efd89ba2700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd8416b7d0 con 0x7efd840ca200 2023-10-08T00:15:27.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.631+0000 7efd89ba2700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd840b4070 con 0x7efd6c003a60 2023-10-08T00:15:27.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.631+0000 7efd89ba2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd8416ba20 con 0x7efd8416e390 2023-10-08T00:15:27.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.631+0000 7efd7bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efd840ca200 0x7efd8416de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:27.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.631+0000 7efd7bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efd840ca200 0x7efd8416de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55910/0 (socket says 172.21.15.99:55910) 2023-10-08T00:15:27.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.631+0000 7efd7bfff700 1 -- 172.21.15.99:0/641435540 learned_addr learned my addr 172.21.15.99:0/641435540 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:27.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.632+0000 7efd893a1700 1 --2- 172.21.15.99:0/641435540 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efd8416e390 0x7efd8416e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:27.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.632+0000 7efd88ba0700 1 --2- 172.21.15.99:0/641435540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd6c003a60 0x7efd840b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:27.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.632+0000 7efd7bfff700 1 -- 172.21.15.99:0/641435540 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efd8416e390 msgr2=0x7efd8416e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:27.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.632+0000 7efd7bfff700 1 --2- 172.21.15.99:0/641435540 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efd8416e390 0x7efd8416e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:27.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.632+0000 7efd7bfff700 1 -- 172.21.15.99:0/641435540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd6c003a60 msgr2=0x7efd840b3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:27.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.632+0000 7efd7bfff700 1 --2- 172.21.15.99:0/641435540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd6c003a60 0x7efd840b3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:27.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.632+0000 7efd7bfff700 1 -- 172.21.15.99:0/641435540 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd8413cf00 con 0x7efd840ca200 2023-10-08T00:15:27.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.632+0000 7efd893a1700 1 --2- 172.21.15.99:0/641435540 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efd8416e390 0x7efd8416e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:15:27.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.632+0000 7efd7bfff700 1 --2- 172.21.15.99:0/641435540 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efd840ca200 0x7efd8416de50 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7efd7400eb90 tx=0x7efd74001c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:27.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.632+0000 7efd78ff9700 1 -- 172.21.15.99:0/641435540 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7400c7e0 con 0x7efd840ca200 2023-10-08T00:15:27.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.632+0000 7efd78ff9700 1 -- 172.21.15.99:0/641435540 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd7400c940 con 0x7efd840ca200 2023-10-08T00:15:27.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.632+0000 7efd89ba2700 1 -- 172.21.15.99:0/641435540 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd840ca700 con 0x7efd840ca200 2023-10-08T00:15:27.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.632+0000 7efd78ff9700 1 -- 172.21.15.99:0/641435540 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7400cd00 con 0x7efd840ca200 2023-10-08T00:15:27.640 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.632+0000 7efd89ba2700 1 -- 172.21.15.99:0/641435540 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd840b13e0 con 0x7efd840ca200 2023-10-08T00:15:27.640 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.633+0000 7efd89ba2700 1 -- 172.21.15.99:0/641435540 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd84089280 con 0x7efd840ca200 2023-10-08T00:15:27.640 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.633+0000 7efd78ff9700 1 -- 172.21.15.99:0/641435540 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efd74013630 con 0x7efd840ca200 2023-10-08T00:15:27.640 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.634+0000 7efd78ff9700 1 --2- 172.21.15.99:0/641435540 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efd70066f30 0x7efd700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:27.640 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.634+0000 7efd78ff9700 1 -- 172.21.15.99:0/641435540 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5705+0+0 (secure 0 0 0) 0x7efd740845a0 con 0x7efd840ca200 2023-10-08T00:15:27.640 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.634+0000 7efd88ba0700 1 --2- 172.21.15.99:0/641435540 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efd70066f30 0x7efd700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:27.640 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.634+0000 7efd78ff9700 1 -- 172.21.15.99:0/641435540 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=91}) v3 -- 0x7efd7006ea50 con 0x7efd840ca200 2023-10-08T00:15:27.640 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.634+0000 7efd88ba0700 1 --2- 172.21.15.99:0/641435540 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efd70066f30 0x7efd700693e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7efd800091e0 tx=0x7efd8001a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:27.641 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.638+0000 7efd78ff9700 1 -- 172.21.15.99:0/641435540 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efd74051810 con 0x7efd840ca200 2023-10-08T00:15:27.865 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.864+0000 7efd89ba2700 1 -- 172.21.15.99:0/641435540 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efd840c7100 con 0x7efd840ca200 2023-10-08T00:15:27.866 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.865+0000 7efd78ff9700 1 -- 172.21.15.99:0/641435540 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4599 (secure 0 0 0) 0x7efd7400ce60 con 0x7efd840ca200 2023-10-08T00:15:27.866 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:15:27.866 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5650,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/2166055423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2166055423},{"type":"v1","addr":"172.21.15.99:6834","nonce":2166055423}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5665,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/4087832300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":4087832300},{"type":"v1","addr":"172.21.15.191:6836","nonce":4087832300}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5668,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/730261712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":730261712},{"type":"v1","addr":"172.21.15.191:6837","nonce":730261712}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:14:43.671064+0000","modified":"2023-10-08T00:14:47.326922+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6039},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6039":{"gid":6039,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/104443412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":104443412},{"type":"v1","addr":"172.21.15.99:6835","nonce":104443412}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-08T00:15:27.868 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.867+0000 7efd89ba2700 1 -- 172.21.15.99:0/641435540 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efd70066f30 msgr2=0x7efd700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:27.868 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.867+0000 7efd89ba2700 1 --2- 172.21.15.99:0/641435540 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efd70066f30 0x7efd700693e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7efd800091e0 tx=0x7efd8001a000 comp rx=0 tx=0).stop 2023-10-08T00:15:27.868 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.867+0000 7efd89ba2700 1 -- 172.21.15.99:0/641435540 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efd840ca200 msgr2=0x7efd8416de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:27.868 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.867+0000 7efd89ba2700 1 --2- 172.21.15.99:0/641435540 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efd840ca200 0x7efd8416de50 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7efd7400eb90 tx=0x7efd74001c70 comp rx=0 tx=0).stop 2023-10-08T00:15:27.868 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.867+0000 7efd89ba2700 1 -- 172.21.15.99:0/641435540 shutdown_connections 2023-10-08T00:15:27.868 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.867+0000 7efd89ba2700 1 --2- 172.21.15.99:0/641435540 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efd70066f30 0x7efd700693e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:27.868 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.867+0000 7efd89ba2700 1 --2- 172.21.15.99:0/641435540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd6c003a60 0x7efd840b3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:27.869 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.867+0000 7efd89ba2700 1 --2- 172.21.15.99:0/641435540 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efd8416e390 0x7efd8416e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:27.869 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.867+0000 7efd89ba2700 1 --2- 172.21.15.99:0/641435540 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efd840ca200 0x7efd8416de50 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:27.869 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.867+0000 7efd89ba2700 1 -- 172.21.15.99:0/641435540 >> 172.21.15.99:0/641435540 conn(0x7efd84067660 msgr2=0x7efd840b0810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:27.869 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.867+0000 7efd89ba2700 1 -- 172.21.15.99:0/641435540 shutdown_connections 2023-10-08T00:15:27.869 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.867+0000 7efd89ba2700 1 -- 172.21.15.99:0/641435540 wait complete. 2023-10-08T00:15:27.869 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 40 2023-10-08T00:15:27.880 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:15:27.992 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.992+0000 7ff30b59e700 1 Processor -- start 2023-10-08T00:15:27.993 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.992+0000 7ff30b59e700 1 -- start start 2023-10-08T00:15:27.995 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.993+0000 7ff30b59e700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff30c0d0970 0x7ff30c15b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:27.995 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.993+0000 7ff30b59e700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff30c0cf0c0 con 0x7ff30c153e50 2023-10-08T00:15:27.995 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.993+0000 7ff30b59e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff30c15e8f0 con 0x7ff30c15aea0 2023-10-08T00:15:27.995 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.993+0000 7ff30b59e700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff30c15b320 con 0x7ff30c0d0970 2023-10-08T00:15:27.995 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.993+0000 7ff30ad9d700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7ff30c153e50 0x7ff30c1542b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:52188/0 (socket says 172.21.15.99:52188) 2023-10-08T00:15:27.995 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.993+0000 7ff30ad9d700 1 -- 172.21.15.99:0/1938166684 learned_addr learned my addr 172.21.15.99:0/1938166684 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:15:27.995 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.993+0000 7ff30a59c700 1 --2- 172.21.15.99:0/1938166684 >> v2:172.21.15.99:3300/0 conn(0x7ff30c0d0970 0x7ff30c15b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:27.996 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.993+0000 7ff30959a700 1 -- 172.21.15.99:0/1938166684 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2346774611 0 0) 0x7ff30c0cf0c0 con 0x7ff30c153e50 2023-10-08T00:15:27.996 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.993+0000 7ff30959a700 1 -- 172.21.15.99:0/1938166684 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2f8003570 con 0x7ff30c153e50 2023-10-08T00:15:27.996 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.993+0000 7ff30a59c700 1 -- 172.21.15.99:0/1938166684 >> v1:172.21.15.191:6789/0 conn(0x7ff30c15aea0 legacy=0x7ff30c0d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:27.996 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.993+0000 7ff30a59c700 1 -- 172.21.15.99:0/1938166684 >> v1:172.21.15.99:6789/0 conn(0x7ff30c153e50 legacy=0x7ff30c1542b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:27.996 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.993+0000 7ff30a59c700 1 -- 172.21.15.99:0/1938166684 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff30c15e050 con 0x7ff30c0d0970 2023-10-08T00:15:27.996 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.994+0000 7ff30a59c700 1 --2- 172.21.15.99:0/1938166684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff30c0d0970 0x7ff30c15b540 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7ff304009e90 tx=0x7ff304009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ab4e77f8ac1e8df server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:27.996 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.994+0000 7ff30959a700 1 -- 172.21.15.99:0/1938166684 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30400e070 con 0x7ff30c0d0970 2023-10-08T00:15:27.996 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.994+0000 7ff30959a700 1 -- 172.21.15.99:0/1938166684 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff30400bc40 con 0x7ff30c0d0970 2023-10-08T00:15:27.996 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.994+0000 7ff30959a700 1 -- 172.21.15.99:0/1938166684 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff304012430 con 0x7ff30c0d0970 2023-10-08T00:15:27.996 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.994+0000 7ff30b59e700 1 -- 172.21.15.99:0/1938166684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff30c0d0970 msgr2=0x7ff30c15b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:27.997 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.994+0000 7ff30b59e700 1 --2- 172.21.15.99:0/1938166684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff30c0d0970 0x7ff30c15b540 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7ff304009e90 tx=0x7ff304009780 comp rx=0 tx=0).stop 2023-10-08T00:15:27.997 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.994+0000 7ff30b59e700 1 -- 172.21.15.99:0/1938166684 shutdown_connections 2023-10-08T00:15:27.997 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.994+0000 7ff30b59e700 1 --2- 172.21.15.99:0/1938166684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff30c0d0970 0x7ff30c15b540 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:27.997 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.994+0000 7ff30b59e700 1 -- 172.21.15.99:0/1938166684 >> 172.21.15.99:0/1938166684 conn(0x7ff30c068c00 msgr2=0x7ff30c069000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:27.997 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.994+0000 7ff30b59e700 1 -- 172.21.15.99:0/1938166684 shutdown_connections 2023-10-08T00:15:27.997 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.994+0000 7ff30b59e700 1 -- 172.21.15.99:0/1938166684 wait complete. 2023-10-08T00:15:28.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.995+0000 7ff30b59e700 1 Processor -- start 2023-10-08T00:15:28.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.995+0000 7ff30b59e700 1 -- start start 2023-10-08T00:15:28.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.996+0000 7ff30b59e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff30c0d0970 0x7ff30c157c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:28.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.996+0000 7ff30b59e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff30c153e50 0x7ff30c1475d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:28.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.996+0000 7ff30b59e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff30c15aea0 0x7ff30c15caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:28.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.996+0000 7ff30b59e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30c159040 con 0x7ff30c153e50 2023-10-08T00:15:28.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.996+0000 7ff30b59e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30c158430 con 0x7ff30c0d0970 2023-10-08T00:15:28.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.996+0000 7ff30b59e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff30c158710 con 0x7ff30c15aea0 2023-10-08T00:15:28.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.996+0000 7ff309d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff30c153e50 0x7ff30c1475d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:28.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.996+0000 7ff309d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff30c153e50 0x7ff30c1475d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55934/0 (socket says 172.21.15.99:55934) 2023-10-08T00:15:28.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.996+0000 7ff309d9b700 1 -- 172.21.15.99:0/997899550 learned_addr learned my addr 172.21.15.99:0/997899550 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:28.001 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.996+0000 7ff30a59c700 1 --2- 172.21.15.99:0/997899550 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff30c0d0970 0x7ff30c157c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:28.001 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.996+0000 7ff30ad9d700 1 --2- 172.21.15.99:0/997899550 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff30c15aea0 0x7ff30c15caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:28.001 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.996+0000 7ff309d9b700 1 -- 172.21.15.99:0/997899550 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff30c15aea0 msgr2=0x7ff30c15caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:28.001 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.996+0000 7ff309d9b700 1 --2- 172.21.15.99:0/997899550 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff30c15aea0 0x7ff30c15caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:28.001 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.996+0000 7ff309d9b700 1 -- 172.21.15.99:0/997899550 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff30c0d0970 msgr2=0x7ff30c157c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:28.001 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.996+0000 7ff309d9b700 1 --2- 172.21.15.99:0/997899550 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff30c0d0970 0x7ff30c157c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:28.001 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.996+0000 7ff309d9b700 1 -- 172.21.15.99:0/997899550 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff304009a20 con 0x7ff30c153e50 2023-10-08T00:15:28.001 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.997+0000 7ff309d9b700 1 --2- 172.21.15.99:0/997899550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff30c153e50 0x7ff30c1475d0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ff2f400c4e0 tx=0x7ff2f400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:28.001 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.997+0000 7ff3027fc700 1 -- 172.21.15.99:0/997899550 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4011590 con 0x7ff30c153e50 2023-10-08T00:15:28.001 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.997+0000 7ff3027fc700 1 -- 172.21.15.99:0/997899550 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2f4011710 con 0x7ff30c153e50 2023-10-08T00:15:28.001 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.997+0000 7ff3027fc700 1 -- 172.21.15.99:0/997899550 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4011ad0 con 0x7ff30c153e50 2023-10-08T00:15:28.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.997+0000 7ff30b59e700 1 -- 172.21.15.99:0/997899550 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff30c1589f0 con 0x7ff30c153e50 2023-10-08T00:15:28.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.997+0000 7ff30b59e700 1 -- 172.21.15.99:0/997899550 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff30c158140 con 0x7ff30c153e50 2023-10-08T00:15:28.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.998+0000 7ff30b59e700 1 -- 172.21.15.99:0/997899550 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff30c1591f0 con 0x7ff30c153e50 2023-10-08T00:15:28.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.998+0000 7ff3027fc700 1 -- 172.21.15.99:0/997899550 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff2f401b3b0 con 0x7ff30c153e50 2023-10-08T00:15:28.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.998+0000 7ff3027fc700 1 --2- 172.21.15.99:0/997899550 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff2ec066f30 0x7ff2ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:28.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.999+0000 7ff3027fc700 1 -- 172.21.15.99:0/997899550 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff2f4083960 con 0x7ff30c153e50 2023-10-08T00:15:28.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.999+0000 7ff30a59c700 1 --2- 172.21.15.99:0/997899550 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff2ec066f30 0x7ff2ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:28.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.999+0000 7ff3027fc700 1 -- 172.21.15.99:0/997899550 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=91}) v3 -- 0x7ff2ec06ea50 con 0x7ff30c153e50 2023-10-08T00:15:28.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:27.999+0000 7ff30a59c700 1 --2- 172.21.15.99:0/997899550 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff2ec066f30 0x7ff2ec0693e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff3040099f0 tx=0x7ff304019000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:28.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.003+0000 7ff3027fc700 1 -- 172.21.15.99:0/997899550 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff2f4017070 con 0x7ff30c153e50 2023-10-08T00:15:28.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.166+0000 7ff3027fc700 1 -- 172.21.15.99:0/997899550 <== mon.0 v2:172.21.15.99:3300/0 7 ==== osd_map(91..91 src has 1..91) v4 ==== 288+0+0 (secure 0 0 0) 0x7ff2f4054f90 con 0x7ff30c153e50 2023-10-08T00:15:28.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.166+0000 7ff3027fc700 1 -- 172.21.15.99:0/997899550 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=92}) v3 -- 0x7ff2ec06f3a0 con 0x7ff30c153e50 2023-10-08T00:15:28.232 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.232+0000 7ff30b59e700 1 -- 172.21.15.99:0/997899550 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff30c1593d0 con 0x7ff30c153e50 2023-10-08T00:15:28.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.232+0000 7ff3027fc700 1 -- 172.21.15.99:0/997899550 <== mon.0 v2:172.21.15.99:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4599 (secure 0 0 0) 0x7ff30c1591f0 con 0x7ff30c153e50 2023-10-08T00:15:28.233 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:15:28.233 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5650,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/2166055423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2166055423},{"type":"v1","addr":"172.21.15.99:6834","nonce":2166055423}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5665,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/4087832300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":4087832300},{"type":"v1","addr":"172.21.15.191:6836","nonce":4087832300}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5668,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/730261712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":730261712},{"type":"v1","addr":"172.21.15.191:6837","nonce":730261712}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:14:43.671064+0000","modified":"2023-10-08T00:14:47.326922+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6039},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6039":{"gid":6039,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/104443412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":104443412},{"type":"v1","addr":"172.21.15.99:6835","nonce":104443412}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-08T00:15:28.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.234+0000 7ff30b59e700 1 -- 172.21.15.99:0/997899550 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff2ec066f30 msgr2=0x7ff2ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:28.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.234+0000 7ff30b59e700 1 --2- 172.21.15.99:0/997899550 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff2ec066f30 0x7ff2ec0693e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff3040099f0 tx=0x7ff304019000 comp rx=0 tx=0).stop 2023-10-08T00:15:28.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.234+0000 7ff30b59e700 1 -- 172.21.15.99:0/997899550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff30c153e50 msgr2=0x7ff30c1475d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:28.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.234+0000 7ff30b59e700 1 --2- 172.21.15.99:0/997899550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff30c153e50 0x7ff30c1475d0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ff2f400c4e0 tx=0x7ff2f400c7f0 comp rx=0 tx=0).stop 2023-10-08T00:15:28.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.234+0000 7ff30b59e700 1 -- 172.21.15.99:0/997899550 shutdown_connections 2023-10-08T00:15:28.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.234+0000 7ff30b59e700 1 --2- 172.21.15.99:0/997899550 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff2ec066f30 0x7ff2ec0693e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:28.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.234+0000 7ff30b59e700 1 --2- 172.21.15.99:0/997899550 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff30c0d0970 0x7ff30c157c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:28.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.234+0000 7ff30b59e700 1 --2- 172.21.15.99:0/997899550 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff30c15aea0 0x7ff30c15caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:28.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.234+0000 7ff30b59e700 1 --2- 172.21.15.99:0/997899550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff30c153e50 0x7ff30c1475d0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:28.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.234+0000 7ff30b59e700 1 -- 172.21.15.99:0/997899550 >> 172.21.15.99:0/997899550 conn(0x7ff30c068c00 msgr2=0x7ff30c0b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:28.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.234+0000 7ff30b59e700 1 -- 172.21.15.99:0/997899550 shutdown_connections 2023-10-08T00:15:28.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.234+0000 7ff30b59e700 1 -- 172.21.15.99:0/997899550 wait complete. 2023-10-08T00:15:28.236 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 40 2023-10-08T00:15:28.247 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:15:28.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.362+0000 7f86da2e7700 1 Processor -- start 2023-10-08T00:15:28.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.362+0000 7f86da2e7700 1 -- start start 2023-10-08T00:15:28.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.363+0000 7f86da2e7700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f86d40afff0 0x7f86d415e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:28.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.363+0000 7f86da2e7700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86d40b03c0 con 0x7f86d4070e10 2023-10-08T00:15:28.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.363+0000 7f86da2e7700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86d406a870 con 0x7f86d4069e20 2023-10-08T00:15:28.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.363+0000 7f86da2e7700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86d406ab50 con 0x7f86d40afff0 2023-10-08T00:15:28.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.363+0000 7f86d8884700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f86d4070e10 0x7f86d406ea90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:52196/0 (socket says 172.21.15.99:52196) 2023-10-08T00:15:28.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.363+0000 7f86d8884700 1 -- 172.21.15.99:0/1611569615 learned_addr learned my addr 172.21.15.99:0/1611569615 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:15:28.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.363+0000 7f86d3fff700 1 --2- 172.21.15.99:0/1611569615 >> v2:172.21.15.191:3300/0 conn(0x7f86d40afff0 0x7f86d415e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:28.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.364+0000 7f86d2ffd700 1 -- 172.21.15.99:0/1611569615 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 793269395 0 0) 0x7f86d40b03c0 con 0x7f86d4070e10 2023-10-08T00:15:28.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.364+0000 7f86d2ffd700 1 -- 172.21.15.99:0/1611569615 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86bc003570 con 0x7f86d4070e10 2023-10-08T00:15:28.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.364+0000 7f86d3fff700 1 -- 172.21.15.99:0/1611569615 >> v1:172.21.15.191:6789/0 conn(0x7f86d4069e20 legacy=0x7f86d4070700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:28.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.364+0000 7f86d3fff700 1 -- 172.21.15.99:0/1611569615 >> v1:172.21.15.99:6789/0 conn(0x7f86d4070e10 legacy=0x7f86d406ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:28.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.364+0000 7f86d2ffd700 1 -- 172.21.15.99:0/1611569615 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2492979934 0 0) 0x7f86bc003570 con 0x7f86d4070e10 2023-10-08T00:15:28.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.364+0000 7f86d3fff700 1 -- 172.21.15.99:0/1611569615 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d415da60 con 0x7f86d40afff0 2023-10-08T00:15:28.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.364+0000 7f86d3fff700 1 --2- 172.21.15.99:0/1611569615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86d40afff0 0x7f86d415e540 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f86c40098e0 tx=0x7f86c40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=777719c0fa9cf793 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:28.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.364+0000 7f86d2ffd700 1 -- 172.21.15.99:0/1611569615 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc0017f0 con 0x7f86d40afff0 2023-10-08T00:15:28.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.364+0000 7f86d2ffd700 1 -- 172.21.15.99:0/1611569615 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86c4011420 con 0x7f86d40afff0 2023-10-08T00:15:28.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.364+0000 7f86d2ffd700 1 -- 172.21.15.99:0/1611569615 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c40117c0 con 0x7f86d40afff0 2023-10-08T00:15:28.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.365+0000 7f86da2e7700 1 -- 172.21.15.99:0/1611569615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86d40afff0 msgr2=0x7f86d415e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:28.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.365+0000 7f86da2e7700 1 --2- 172.21.15.99:0/1611569615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86d40afff0 0x7f86d415e540 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f86c40098e0 tx=0x7f86c40091a0 comp rx=0 tx=0).stop 2023-10-08T00:15:28.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.365+0000 7f86da2e7700 1 -- 172.21.15.99:0/1611569615 shutdown_connections 2023-10-08T00:15:28.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.365+0000 7f86da2e7700 1 --2- 172.21.15.99:0/1611569615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86d40afff0 0x7f86d415e540 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:28.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.365+0000 7f86da2e7700 1 -- 172.21.15.99:0/1611569615 >> 172.21.15.99:0/1611569615 conn(0x7f86d40cda50 msgr2=0x7f86d40cde50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:28.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.365+0000 7f86da2e7700 1 -- 172.21.15.99:0/1611569615 shutdown_connections 2023-10-08T00:15:28.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.365+0000 7f86da2e7700 1 -- 172.21.15.99:0/1611569615 wait complete. 2023-10-08T00:15:28.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.366+0000 7f86da2e7700 1 Processor -- start 2023-10-08T00:15:28.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.366+0000 7f86da2e7700 1 -- start start 2023-10-08T00:15:28.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.367+0000 7f86da2e7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86d4069e20 0x7f86d416ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:28.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.367+0000 7f86da2e7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86d4070e10 0x7f86d40b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:28.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.367+0000 7f86da2e7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86d40afff0 0x7f86d40aef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:28.371 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.367+0000 7f86da2e7700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d40b39a0 con 0x7f86d4069e20 2023-10-08T00:15:28.371 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.367+0000 7f86da2e7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d416b270 con 0x7f86d4070e10 2023-10-08T00:15:28.371 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.367+0000 7f86da2e7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86d40b3720 con 0x7f86d40afff0 2023-10-08T00:15:28.371 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.367+0000 7f86d3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86d4069e20 0x7f86d416ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:28.371 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.367+0000 7f86d3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86d4069e20 0x7f86d416ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55944/0 (socket says 172.21.15.99:55944) 2023-10-08T00:15:28.371 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.367+0000 7f86d3fff700 1 -- 172.21.15.99:0/915247508 learned_addr learned my addr 172.21.15.99:0/915247508 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:28.371 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.367+0000 7f86d37fe700 1 --2- 172.21.15.99:0/915247508 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86d4070e10 0x7f86d40b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:28.371 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.367+0000 7f86d8884700 1 --2- 172.21.15.99:0/915247508 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86d40afff0 0x7f86d40aef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:28.371 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.367+0000 7f86d3fff700 1 -- 172.21.15.99:0/915247508 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86d40afff0 msgr2=0x7f86d40aef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:28.371 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.367+0000 7f86d3fff700 1 --2- 172.21.15.99:0/915247508 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86d40afff0 0x7f86d40aef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:28.371 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.367+0000 7f86d3fff700 1 -- 172.21.15.99:0/915247508 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86d4070e10 msgr2=0x7f86d40b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:28.371 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.367+0000 7f86d3fff700 1 --2- 172.21.15.99:0/915247508 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86d4070e10 0x7f86d40b31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:28.372 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.367+0000 7f86d3fff700 1 -- 172.21.15.99:0/915247508 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c4009420 con 0x7f86d4069e20 2023-10-08T00:15:28.372 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.367+0000 7f86d3fff700 1 --2- 172.21.15.99:0/915247508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86d4069e20 0x7f86d416ad30 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f86c40091e0 tx=0x7f86c4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:28.372 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.367+0000 7f86bbfff700 1 -- 172.21.15.99:0/915247508 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c401cc90 con 0x7f86d4069e20 2023-10-08T00:15:28.372 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.367+0000 7f86bbfff700 1 -- 172.21.15.99:0/915247508 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86c401ce10 con 0x7f86d4069e20 2023-10-08T00:15:28.372 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.367+0000 7f86da2e7700 1 -- 172.21.15.99:0/915247508 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86d40b0d30 con 0x7f86d4069e20 2023-10-08T00:15:28.372 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.367+0000 7f86bbfff700 1 -- 172.21.15.99:0/915247508 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c401a6a0 con 0x7f86d4069e20 2023-10-08T00:15:28.372 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.367+0000 7f86da2e7700 1 -- 172.21.15.99:0/915247508 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86d40b12f0 con 0x7f86d4069e20 2023-10-08T00:15:28.372 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.368+0000 7f86da2e7700 1 -- 172.21.15.99:0/915247508 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86d404c3c0 con 0x7f86d4069e20 2023-10-08T00:15:28.372 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.369+0000 7f86bbfff700 1 -- 172.21.15.99:0/915247508 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f86c401aa50 con 0x7f86d4069e20 2023-10-08T00:15:28.372 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.369+0000 7f86bbfff700 1 --2- 172.21.15.99:0/915247508 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f86c0066f30 0x7f86c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:28.372 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.369+0000 7f86bbfff700 1 -- 172.21.15.99:0/915247508 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f86c4086240 con 0x7f86d4069e20 2023-10-08T00:15:28.372 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.369+0000 7f86d37fe700 1 --2- 172.21.15.99:0/915247508 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f86c0066f30 0x7f86c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:28.373 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.369+0000 7f86bbfff700 1 -- 172.21.15.99:0/915247508 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=92}) v3 -- 0x7f86c006ea50 con 0x7f86d4069e20 2023-10-08T00:15:28.373 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.370+0000 7f86d37fe700 1 --2- 172.21.15.99:0/915247508 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f86c0066f30 0x7f86c00693e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f86d40cc260 tx=0x7f86c8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:28.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.373+0000 7f86bbfff700 1 -- 172.21.15.99:0/915247508 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f86c40534b0 con 0x7f86d4069e20 2023-10-08T00:15:28.557 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.557+0000 7f86da2e7700 1 -- 172.21.15.99:0/915247508 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86d406e870 con 0x7f86d4069e20 2023-10-08T00:15:28.558 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.558+0000 7f86bbfff700 1 -- 172.21.15.99:0/915247508 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+15390 (secure 0 0 0) 0x7f86c4056ad0 con 0x7f86d4069e20 2023-10-08T00:15:28.559 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:15:28.559 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":91,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:15:28.157681+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:14:41.496823+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-08T00:14:42.640029+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","full","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["backfillfull","exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:15:28.560 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.560+0000 7f86da2e7700 1 -- 172.21.15.99:0/915247508 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f86c0066f30 msgr2=0x7f86c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:28.561 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.560+0000 7f86da2e7700 1 --2- 172.21.15.99:0/915247508 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f86c0066f30 0x7f86c00693e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f86d40cc260 tx=0x7f86c8009040 comp rx=0 tx=0).stop 2023-10-08T00:15:28.561 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.560+0000 7f86da2e7700 1 -- 172.21.15.99:0/915247508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86d4069e20 msgr2=0x7f86d416ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:28.561 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.560+0000 7f86da2e7700 1 --2- 172.21.15.99:0/915247508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86d4069e20 0x7f86d416ad30 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f86c40091e0 tx=0x7f86c4016000 comp rx=0 tx=0).stop 2023-10-08T00:15:28.561 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.560+0000 7f86da2e7700 1 -- 172.21.15.99:0/915247508 shutdown_connections 2023-10-08T00:15:28.561 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.560+0000 7f86da2e7700 1 --2- 172.21.15.99:0/915247508 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f86c0066f30 0x7f86c00693e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:28.561 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.560+0000 7f86da2e7700 1 --2- 172.21.15.99:0/915247508 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86d40afff0 0x7f86d40aef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:28.561 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.560+0000 7f86da2e7700 1 --2- 172.21.15.99:0/915247508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86d4069e20 0x7f86d416ad30 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:28.561 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.560+0000 7f86da2e7700 1 --2- 172.21.15.99:0/915247508 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86d4070e10 0x7f86d40b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:28.562 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.560+0000 7f86da2e7700 1 -- 172.21.15.99:0/915247508 >> 172.21.15.99:0/915247508 conn(0x7f86d40cda50 msgr2=0x7f86d40cb8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:28.562 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.560+0000 7f86da2e7700 1 -- 172.21.15.99:0/915247508 shutdown_connections 2023-10-08T00:15:28.562 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.560+0000 7f86da2e7700 1 -- 172.21.15.99:0/915247508 wait complete. 2023-10-08T00:15:28.573 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T00:15:28.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.687+0000 7f007107a700 1 Processor -- start 2023-10-08T00:15:28.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.687+0000 7f007107a700 1 -- start start 2023-10-08T00:15:28.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.688+0000 7f007107a700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f006c152010 0x7f006c15b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:28.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.688+0000 7f007107a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f006c158e50 0x7f006c0cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:28.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.688+0000 7f007107a700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f006c06c010 con 0x7f006c1523e0 2023-10-08T00:15:28.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.688+0000 7f007107a700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f006c06bd10 con 0x7f006c152010 2023-10-08T00:15:28.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.688+0000 7f007107a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f006c15c060 con 0x7f006c158e50 2023-10-08T00:15:28.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.688+0000 7f006259c700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f006c1523e0 0x7f006c06c4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:52202/0 (socket says 172.21.15.99:52202) 2023-10-08T00:15:28.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.688+0000 7f006259c700 1 -- 172.21.15.99:0/1164946152 learned_addr learned my addr 172.21.15.99:0/1164946152 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:15:28.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.688+0000 7f006ad9d700 1 --2- 172.21.15.99:0/1164946152 >> v2:172.21.15.191:3300/0 conn(0x7f006c152010 0x7f006c15b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:28.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.688+0000 7f006b59e700 1 --2- 172.21.15.99:0/1164946152 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f006c158e50 0x7f006c0cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:28.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.689+0000 7f006a59c700 1 -- 172.21.15.99:0/1164946152 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3842873749 0 0) 0x7f006c06c010 con 0x7f006c1523e0 2023-10-08T00:15:28.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.689+0000 7f006a59c700 1 -- 172.21.15.99:0/1164946152 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0058003570 con 0x7f006c1523e0 2023-10-08T00:15:28.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.689+0000 7f006ad9d700 1 -- 172.21.15.99:0/1164946152 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f006c158e50 msgr2=0x7f006c0cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:28.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.689+0000 7f006ad9d700 1 --2- 172.21.15.99:0/1164946152 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f006c158e50 0x7f006c0cdf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:28.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.689+0000 7f006ad9d700 1 -- 172.21.15.99:0/1164946152 >> v1:172.21.15.99:6789/0 conn(0x7f006c1523e0 legacy=0x7f006c06c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:28.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.689+0000 7f006ad9d700 1 -- 172.21.15.99:0/1164946152 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f006c15c340 con 0x7f006c152010 2023-10-08T00:15:28.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.689+0000 7f006ad9d700 1 --2- 172.21.15.99:0/1164946152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f006c152010 0x7f006c15b810 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f005c0098e0 tx=0x7f005c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b97a7e174c1466a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:28.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.689+0000 7f006a59c700 1 -- 172.21.15.99:0/1164946152 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005c011630 con 0x7f006c152010 2023-10-08T00:15:28.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.689+0000 7f006a59c700 1 -- 172.21.15.99:0/1164946152 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f005c011790 con 0x7f006c152010 2023-10-08T00:15:28.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.689+0000 7f006a59c700 1 -- 172.21.15.99:0/1164946152 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005c011b30 con 0x7f006c152010 2023-10-08T00:15:28.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.690+0000 7f007107a700 1 -- 172.21.15.99:0/1164946152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f006c152010 msgr2=0x7f006c15b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:28.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.690+0000 7f007107a700 1 --2- 172.21.15.99:0/1164946152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f006c152010 0x7f006c15b810 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f005c0098e0 tx=0x7f005c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:15:28.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.690+0000 7f007107a700 1 -- 172.21.15.99:0/1164946152 shutdown_connections 2023-10-08T00:15:28.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.690+0000 7f007107a700 1 --2- 172.21.15.99:0/1164946152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f006c152010 0x7f006c15b810 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:28.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.690+0000 7f007107a700 1 --2- 172.21.15.99:0/1164946152 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f006c158e50 0x7f006c0cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:28.694 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.690+0000 7f007107a700 1 -- 172.21.15.99:0/1164946152 >> 172.21.15.99:0/1164946152 conn(0x7f006c069870 msgr2=0x7f006c069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:28.694 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.690+0000 7f007107a700 1 -- 172.21.15.99:0/1164946152 shutdown_connections 2023-10-08T00:15:28.694 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.690+0000 7f007107a700 1 -- 172.21.15.99:0/1164946152 wait complete. 2023-10-08T00:15:28.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.691+0000 7f007107a700 1 Processor -- start 2023-10-08T00:15:28.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.691+0000 7f007107a700 1 -- start start 2023-10-08T00:15:28.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.692+0000 7f007107a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f006c152010 0x7f006c13eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:28.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.692+0000 7f007107a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f006c1523e0 0x7f006c0b2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:28.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.692+0000 7f007107a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f006c158e50 0x7f006c0c48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:28.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.692+0000 7f007107a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006c16bfc0 con 0x7f006c152010 2023-10-08T00:15:28.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.692+0000 7f007107a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006c16b3b0 con 0x7f006c158e50 2023-10-08T00:15:28.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.692+0000 7f007107a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f006c16b690 con 0x7f006c1523e0 2023-10-08T00:15:28.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.692+0000 7f006ad9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f006c152010 0x7f006c13eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:28.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.692+0000 7f006ad9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f006c152010 0x7f006c13eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55960/0 (socket says 172.21.15.99:55960) 2023-10-08T00:15:28.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.692+0000 7f006ad9d700 1 -- 172.21.15.99:0/2170275982 learned_addr learned my addr 172.21.15.99:0/2170275982 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:28.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.692+0000 7f006259c700 1 --2- 172.21.15.99:0/2170275982 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f006c1523e0 0x7f006c0b2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:28.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.692+0000 7f006ad9d700 1 -- 172.21.15.99:0/2170275982 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f006c1523e0 msgr2=0x7f006c0b2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:28.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.692+0000 7f006b59e700 1 --2- 172.21.15.99:0/2170275982 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f006c158e50 0x7f006c0c48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:28.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.692+0000 7f006ad9d700 1 --2- 172.21.15.99:0/2170275982 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f006c1523e0 0x7f006c0b2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:28.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.692+0000 7f006ad9d700 1 -- 172.21.15.99:0/2170275982 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f006c158e50 msgr2=0x7f006c0c48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:28.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.692+0000 7f006ad9d700 1 --2- 172.21.15.99:0/2170275982 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f006c158e50 0x7f006c0c48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:28.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.692+0000 7f006ad9d700 1 -- 172.21.15.99:0/2170275982 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f005c009420 con 0x7f006c152010 2023-10-08T00:15:28.699 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.692+0000 7f006259c700 1 --2- 172.21.15.99:0/2170275982 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f006c1523e0 0x7f006c0b2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:15:28.699 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.692+0000 7f006b59e700 1 --2- 172.21.15.99:0/2170275982 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f006c158e50 0x7f006c0c48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:15:28.699 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.692+0000 7f006ad9d700 1 --2- 172.21.15.99:0/2170275982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f006c152010 0x7f006c13eef0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f006c0c59e0 tx=0x7f005c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:28.699 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.692+0000 7f00637fe700 1 -- 172.21.15.99:0/2170275982 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005c011b10 con 0x7f006c152010 2023-10-08T00:15:28.699 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.692+0000 7f00637fe700 1 -- 172.21.15.99:0/2170275982 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f005c0233f0 con 0x7f006c152010 2023-10-08T00:15:28.699 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.692+0000 7f007107a700 1 -- 172.21.15.99:0/2170275982 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f006c16b970 con 0x7f006c152010 2023-10-08T00:15:28.699 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.692+0000 7f00637fe700 1 -- 172.21.15.99:0/2170275982 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005c023770 con 0x7f006c152010 2023-10-08T00:15:28.699 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.692+0000 7f007107a700 1 -- 172.21.15.99:0/2170275982 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f006c150890 con 0x7f006c152010 2023-10-08T00:15:28.699 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.693+0000 7f007107a700 1 -- 172.21.15.99:0/2170275982 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f006c04c3c0 con 0x7f006c152010 2023-10-08T00:15:28.699 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.694+0000 7f00637fe700 1 -- 172.21.15.99:0/2170275982 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f005c02c3b0 con 0x7f006c152010 2023-10-08T00:15:28.699 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.694+0000 7f00637fe700 1 --2- 172.21.15.99:0/2170275982 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f004c066fb0 0x7f004c069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:28.700 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.694+0000 7f00637fe700 1 -- 172.21.15.99:0/2170275982 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f005c085a90 con 0x7f006c152010 2023-10-08T00:15:28.700 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.694+0000 7f00637fe700 1 -- 172.21.15.99:0/2170275982 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=92}) v3 -- 0x7f004c06ead0 con 0x7f006c152010 2023-10-08T00:15:28.700 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.694+0000 7f006259c700 1 --2- 172.21.15.99:0/2170275982 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f004c066fb0 0x7f004c069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:28.700 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.695+0000 7f006259c700 1 --2- 172.21.15.99:0/2170275982 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f004c066fb0 0x7f004c069460 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0054007fd0 tx=0x7f0054009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:28.700 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.698+0000 7f00637fe700 1 -- 172.21.15.99:0/2170275982 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f005c028060 con 0x7f006c152010 2023-10-08T00:15:28.921 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.921+0000 7f007107a700 1 -- 172.21.15.99:0/2170275982 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f006c150c30 con 0x7f006c152010 2023-10-08T00:15:28.922 INFO:tasks.ceph.mon.a.smithi099.stderr:2023-10-08T00:15:28.921+0000 7f6c60fa2700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T00:15:28.926 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.926+0000 7f00637fe700 1 -- 172.21.15.99:0/2170275982 <== mon.0 v2:172.21.15.99:3300/0 7 ==== osd_map(92..92 src has 1..92) v4 ==== 1546+0+0 (secure 0 0 0) 0x7f005c0570a0 con 0x7f006c152010 2023-10-08T00:15:28.926 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.926+0000 7f00637fe700 1 -- 172.21.15.99:0/2170275982 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=93}) v3 -- 0x7f004c06fde0 con 0x7f006c152010 2023-10-08T00:15:28.928 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.928+0000 7f00637fe700 1 -- 172.21.15.99:0/2170275982 <== mon.0 v2:172.21.15.99:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v41) v1 ==== 161+0+0 (secure 0 0 0) 0x7f005c015020 con 0x7f006c152010 2023-10-08T00:15:28.932 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.931+0000 7f007107a700 1 -- 172.21.15.99:0/2170275982 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f004c066fb0 msgr2=0x7f004c069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:28.932 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.931+0000 7f007107a700 1 --2- 172.21.15.99:0/2170275982 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f004c066fb0 0x7f004c069460 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0054007fd0 tx=0x7f0054009040 comp rx=0 tx=0).stop 2023-10-08T00:15:28.932 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.931+0000 7f007107a700 1 -- 172.21.15.99:0/2170275982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f006c152010 msgr2=0x7f006c13eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:28.932 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.931+0000 7f007107a700 1 --2- 172.21.15.99:0/2170275982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f006c152010 0x7f006c13eef0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f006c0c59e0 tx=0x7f005c017000 comp rx=0 tx=0).stop 2023-10-08T00:15:28.932 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.931+0000 7f007107a700 1 -- 172.21.15.99:0/2170275982 shutdown_connections 2023-10-08T00:15:28.932 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.931+0000 7f007107a700 1 --2- 172.21.15.99:0/2170275982 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f004c066fb0 0x7f004c069460 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:28.932 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.931+0000 7f007107a700 1 --2- 172.21.15.99:0/2170275982 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f006c158e50 0x7f006c0c48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:28.932 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.931+0000 7f007107a700 1 --2- 172.21.15.99:0/2170275982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f006c152010 0x7f006c13eef0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:28.933 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.931+0000 7f007107a700 1 --2- 172.21.15.99:0/2170275982 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f006c1523e0 0x7f006c0b2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:28.933 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.931+0000 7f007107a700 1 -- 172.21.15.99:0/2170275982 >> 172.21.15.99:0/2170275982 conn(0x7f006c069870 msgr2=0x7f006c0cb2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:28.933 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.931+0000 7f007107a700 1 -- 172.21.15.99:0/2170275982 shutdown_connections 2023-10-08T00:15:28.933 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:28.931+0000 7f007107a700 1 -- 172.21.15.99:0/2170275982 wait complete. 2023-10-08T00:15:28.933 INFO:teuthology.orchestra.run.smithi099.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T00:15:28.948 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T00:15:28.966 INFO:tasks.ceph.mds.a.smithi099.stdout:starting mds.a at 2023-10-08T00:15:29.064 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.063+0000 7fa54c582700 1 Processor -- start 2023-10-08T00:15:29.064 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.063+0000 7fa54c582700 1 -- start start 2023-10-08T00:15:29.066 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.064+0000 7fa54c582700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa5440cc250 0x7fa544068ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:29.066 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.064+0000 7fa54c582700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa5440cc620 0x7fa544156530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:29.066 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.064+0000 7fa54c582700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa544068620 con 0x7fa544156a70 2023-10-08T00:15:29.066 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.064+0000 7fa54c582700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa544068ff0 con 0x7fa5440cc620 2023-10-08T00:15:29.067 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.064+0000 7fa54c582700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa544067a50 con 0x7fa5440cc250 2023-10-08T00:15:29.067 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.064+0000 7fa549b1d700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa5440cc620 0x7fa544156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:29.067 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.064+0000 7fa549b1d700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa5440cc620 0x7fa544156530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55964/0 (socket says 172.21.15.99:55964) 2023-10-08T00:15:29.067 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.064+0000 7fa549b1d700 1 -- 172.21.15.99:0/99685004 learned_addr learned my addr 172.21.15.99:0/99685004 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:29.067 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.065+0000 7fa54a31e700 1 --2- 172.21.15.99:0/99685004 >> v2:172.21.15.191:3300/0 conn(0x7fa5440cc250 0x7fa544068ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:29.067 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.065+0000 7fa549b1d700 1 -- 172.21.15.99:0/99685004 >> v2:172.21.15.191:3300/0 conn(0x7fa5440cc250 msgr2=0x7fa544068ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:29.067 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.065+0000 7fa549b1d700 1 --2- 172.21.15.99:0/99685004 >> v2:172.21.15.191:3300/0 conn(0x7fa5440cc250 0x7fa544068ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:29.067 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.065+0000 7fa549b1d700 1 -- 172.21.15.99:0/99685004 >> v1:172.21.15.191:6789/0 conn(0x7fa544156a70 legacy=0x7fa5440ca4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:29.067 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.065+0000 7fa549b1d700 1 -- 172.21.15.99:0/99685004 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa544067cf0 con 0x7fa5440cc620 2023-10-08T00:15:29.067 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.065+0000 7fa549b1d700 1 --2- 172.21.15.99:0/99685004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5440cc620 0x7fa544156530 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fa54406b7a0 tx=0x7fa538009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c24f8ecde115c7b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:29.067 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.065+0000 7fa54931c700 1 -- 172.21.15.99:0/99685004 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa538009e70 con 0x7fa5440cc620 2023-10-08T00:15:29.068 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.065+0000 7fa54931c700 1 -- 172.21.15.99:0/99685004 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa53800b860 con 0x7fa5440cc620 2023-10-08T00:15:29.068 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.065+0000 7fa54931c700 1 -- 172.21.15.99:0/99685004 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa53800bc00 con 0x7fa5440cc620 2023-10-08T00:15:29.068 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.065+0000 7fa54c582700 1 -- 172.21.15.99:0/99685004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5440cc620 msgr2=0x7fa544156530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:29.068 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.065+0000 7fa54c582700 1 --2- 172.21.15.99:0/99685004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5440cc620 0x7fa544156530 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fa54406b7a0 tx=0x7fa538009250 comp rx=0 tx=0).stop 2023-10-08T00:15:29.068 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.066+0000 7fa54c582700 1 -- 172.21.15.99:0/99685004 shutdown_connections 2023-10-08T00:15:29.068 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.066+0000 7fa54c582700 1 --2- 172.21.15.99:0/99685004 >> v2:172.21.15.191:3300/0 conn(0x7fa5440cc250 0x7fa544068ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:29.068 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.066+0000 7fa54c582700 1 --2- 172.21.15.99:0/99685004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5440cc620 0x7fa544156530 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:29.068 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.066+0000 7fa54c582700 1 -- 172.21.15.99:0/99685004 >> 172.21.15.99:0/99685004 conn(0x7fa54406be80 msgr2=0x7fa54406c280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:29.068 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.066+0000 7fa54c582700 1 -- 172.21.15.99:0/99685004 shutdown_connections 2023-10-08T00:15:29.068 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.066+0000 7fa54c582700 1 -- 172.21.15.99:0/99685004 wait complete. 2023-10-08T00:15:29.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.067+0000 7fa54c582700 1 Processor -- start 2023-10-08T00:15:29.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.067+0000 7fa54c582700 1 -- start start 2023-10-08T00:15:29.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.067+0000 7fa54c582700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa5440cc250 0x7fa5441597a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:29.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.067+0000 7fa54c582700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5440cc620 0x7fa54415ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:29.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.067+0000 7fa54c582700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa544156a70 0x7fa5440c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:29.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.067+0000 7fa54c582700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5440b38c0 con 0x7fa5440cc620 2023-10-08T00:15:29.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.067+0000 7fa54c582700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5440b3020 con 0x7fa544156a70 2023-10-08T00:15:29.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.067+0000 7fa54c582700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5440b3300 con 0x7fa5440cc250 2023-10-08T00:15:29.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.068+0000 7fa549b1d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5440cc620 0x7fa54415ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:29.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.068+0000 7fa549b1d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5440cc620 0x7fa54415ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55974/0 (socket says 172.21.15.99:55974) 2023-10-08T00:15:29.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.068+0000 7fa549b1d700 1 -- 172.21.15.99:0/20143312 learned_addr learned my addr 172.21.15.99:0/20143312 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:29.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.068+0000 7fa54ab1f700 1 --2- 172.21.15.99:0/20143312 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa544156a70 0x7fa5440c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:29.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.068+0000 7fa54a31e700 1 --2- 172.21.15.99:0/20143312 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa5440cc250 0x7fa5441597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:29.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.068+0000 7fa549b1d700 1 -- 172.21.15.99:0/20143312 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa5440cc250 msgr2=0x7fa5441597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:29.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.068+0000 7fa549b1d700 1 --2- 172.21.15.99:0/20143312 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa5440cc250 0x7fa5441597a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:29.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.068+0000 7fa549b1d700 1 -- 172.21.15.99:0/20143312 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa544156a70 msgr2=0x7fa5440c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:29.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.068+0000 7fa549b1d700 1 --2- 172.21.15.99:0/20143312 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa544156a70 0x7fa5440c5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:29.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.068+0000 7fa549b1d700 1 -- 172.21.15.99:0/20143312 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5380094f0 con 0x7fa5440cc620 2023-10-08T00:15:29.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.068+0000 7fa549b1d700 1 --2- 172.21.15.99:0/20143312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5440cc620 0x7fa54415ab20 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fa5440c4a70 tx=0x7fa538017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:29.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.068+0000 7fa5427fc700 1 -- 172.21.15.99:0/20143312 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa53800bbe0 con 0x7fa5440cc620 2023-10-08T00:15:29.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.068+0000 7fa5427fc700 1 -- 172.21.15.99:0/20143312 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa538011420 con 0x7fa5440cc620 2023-10-08T00:15:29.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.068+0000 7fa54c582700 1 -- 172.21.15.99:0/20143312 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5440b35e0 con 0x7fa5440cc620 2023-10-08T00:15:29.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.068+0000 7fa5427fc700 1 -- 172.21.15.99:0/20143312 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5380117c0 con 0x7fa5440cc620 2023-10-08T00:15:29.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.068+0000 7fa54c582700 1 -- 172.21.15.99:0/20143312 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa544159ce0 con 0x7fa5440cc620 2023-10-08T00:15:29.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.069+0000 7fa54c582700 1 -- 172.21.15.99:0/20143312 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa544156eb0 con 0x7fa5440cc620 2023-10-08T00:15:29.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.069+0000 7fa5427fc700 1 -- 172.21.15.99:0/20143312 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa538011bf0 con 0x7fa5440cc620 2023-10-08T00:15:29.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.070+0000 7fa5427fc700 1 --2- 172.21.15.99:0/20143312 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa530066fb0 0x7fa530069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:29.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.070+0000 7fa5427fc700 1 -- 172.21.15.99:0/20143312 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5791+0+0 (secure 0 0 0) 0x7fa538085af0 con 0x7fa5440cc620 2023-10-08T00:15:29.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.070+0000 7fa54a31e700 1 --2- 172.21.15.99:0/20143312 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa530066fb0 0x7fa530069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:29.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.070+0000 7fa54a31e700 1 --2- 172.21.15.99:0/20143312 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa530066fb0 0x7fa530069460 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa53400f5b0 tx=0x7fa53400b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:29.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.074+0000 7fa5427fc700 1 -- 172.21.15.99:0/20143312 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa544156eb0 con 0x7fa5440cc620 2023-10-08T00:15:29.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.298+0000 7fa54c582700 1 -- 172.21.15.99:0/20143312 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa544157290 con 0x7fa5440cc620 2023-10-08T00:15:29.926 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.925+0000 7fa5427fc700 1 -- 172.21.15.99:0/20143312 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v42) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa544157290 con 0x7fa5440cc620 2023-10-08T00:15:29.929 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.927+0000 7fa54c582700 1 -- 172.21.15.99:0/20143312 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa530066fb0 msgr2=0x7fa530069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:29.929 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.927+0000 7fa54c582700 1 --2- 172.21.15.99:0/20143312 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa530066fb0 0x7fa530069460 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa53400f5b0 tx=0x7fa53400b040 comp rx=0 tx=0).stop 2023-10-08T00:15:29.929 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.927+0000 7fa54c582700 1 -- 172.21.15.99:0/20143312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5440cc620 msgr2=0x7fa54415ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:29.930 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.927+0000 7fa54c582700 1 --2- 172.21.15.99:0/20143312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5440cc620 0x7fa54415ab20 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fa5440c4a70 tx=0x7fa538017000 comp rx=0 tx=0).stop 2023-10-08T00:15:29.930 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.927+0000 7fa54c582700 1 -- 172.21.15.99:0/20143312 shutdown_connections 2023-10-08T00:15:29.930 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.927+0000 7fa54c582700 1 --2- 172.21.15.99:0/20143312 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa530066fb0 0x7fa530069460 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:29.930 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.927+0000 7fa54c582700 1 --2- 172.21.15.99:0/20143312 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa544156a70 0x7fa5440c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:29.930 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.927+0000 7fa54c582700 1 --2- 172.21.15.99:0/20143312 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa5440cc250 0x7fa5441597a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:29.930 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.927+0000 7fa54c582700 1 --2- 172.21.15.99:0/20143312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5440cc620 0x7fa54415ab20 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:29.930 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.927+0000 7fa54c582700 1 -- 172.21.15.99:0/20143312 >> 172.21.15.99:0/20143312 conn(0x7fa54406be80 msgr2=0x7fa5440c2880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:29.930 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.927+0000 7fa54c582700 1 -- 172.21.15.99:0/20143312 shutdown_connections 2023-10-08T00:15:29.930 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:29.927+0000 7fa54c582700 1 -- 172.21.15.99:0/20143312 wait complete. 2023-10-08T00:15:29.941 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T00:15:30.100 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.098+0000 7f15e2c12700 1 Processor -- start 2023-10-08T00:15:30.100 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.098+0000 7f15e2c12700 1 -- start start 2023-10-08T00:15:30.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.099+0000 7f15e2c12700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f15dc0b6a10 0x7f15dc15a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:30.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.099+0000 7f15e2c12700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15dc0b17b0 con 0x7f15dc159ef0 2023-10-08T00:15:30.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.099+0000 7f15e2c12700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15dc0b1a60 con 0x7f15dc0b46f0 2023-10-08T00:15:30.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.099+0000 7f15e2c12700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15dc0b1d40 con 0x7f15dc0b6a10 2023-10-08T00:15:30.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.099+0000 7f15e09ae700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f15dc159ef0 0x7f15dc0b10a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:52216/0 (socket says 172.21.15.99:52216) 2023-10-08T00:15:30.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.099+0000 7f15e09ae700 1 -- 172.21.15.99:0/3950323393 learned_addr learned my addr 172.21.15.99:0/3950323393 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:15:30.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.099+0000 7f15d3fff700 1 --2- 172.21.15.99:0/3950323393 >> v2:172.21.15.191:3300/0 conn(0x7f15dc0b6a10 0x7f15dc15a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:30.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.099+0000 7f15d37fe700 1 -- 172.21.15.99:0/3950323393 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4083173304 0 0) 0x7f15dc0b17b0 con 0x7f15dc159ef0 2023-10-08T00:15:30.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.099+0000 7f15d37fe700 1 -- 172.21.15.99:0/3950323393 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15c4003570 con 0x7f15dc159ef0 2023-10-08T00:15:30.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.100+0000 7f15d37fe700 1 -- 172.21.15.99:0/3950323393 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1939910561 0 0) 0x7f15c4003570 con 0x7f15dc159ef0 2023-10-08T00:15:30.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.100+0000 7f15d37fe700 1 -- 172.21.15.99:0/3950323393 >> v2:172.21.15.191:3300/0 conn(0x7f15dc0b6a10 msgr2=0x7f15dc15a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:30.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.100+0000 7f15d37fe700 1 --2- 172.21.15.99:0/3950323393 >> v2:172.21.15.191:3300/0 conn(0x7f15dc0b6a10 0x7f15dc15a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:30.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.100+0000 7f15d37fe700 1 -- 172.21.15.99:0/3950323393 >> v1:172.21.15.191:6789/0 conn(0x7f15dc0b46f0 legacy=0x7f15dc0b4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:30.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.100+0000 7f15d37fe700 1 -- 172.21.15.99:0/3950323393 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15dc0b2020 con 0x7f15dc159ef0 2023-10-08T00:15:30.105 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.100+0000 7f15d37fe700 1 -- 172.21.15.99:0/3950323393 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f15d80015f0 con 0x7f15dc159ef0 2023-10-08T00:15:30.105 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.100+0000 7f15d37fe700 1 -- 172.21.15.99:0/3950323393 >> v1:172.21.15.99:6789/0 conn(0x7f15dc159ef0 legacy=0x7f15dc0b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:30.105 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.100+0000 7f15d37fe700 1 --2- 172.21.15.99:0/3950323393 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15c4003a60 0x7f15c4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:30.105 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.100+0000 7f15d37fe700 1 --2- 172.21.15.99:0/3950323393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15c4004810 0x7f15c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:30.105 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.100+0000 7f15d37fe700 1 --2- 172.21.15.99:0/3950323393 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15c40051c0 0x7f15c400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:30.105 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.100+0000 7f15d37fe700 1 -- 172.21.15.99:0/3950323393 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15dc0b1d40 con 0x7f15c4004810 2023-10-08T00:15:30.105 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.100+0000 7f15d37fe700 1 -- 172.21.15.99:0/3950323393 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c400c050 con 0x7f15c40051c0 2023-10-08T00:15:30.105 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.100+0000 7f15d37fe700 1 -- 172.21.15.99:0/3950323393 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15c400c210 con 0x7f15c4003a60 2023-10-08T00:15:30.105 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.100+0000 7f15d3fff700 1 --2- 172.21.15.99:0/3950323393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15c4004810 0x7f15c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:30.106 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.100+0000 7f15e09ae700 1 --2- 172.21.15.99:0/3950323393 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15c4003a60 0x7f15c4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:30.106 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.100+0000 7f15e11af700 1 --2- 172.21.15.99:0/3950323393 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15c40051c0 0x7f15c400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:30.106 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.100+0000 7f15d3fff700 1 -- 172.21.15.99:0/3950323393 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15c4003a60 msgr2=0x7f15c4003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:30.106 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.100+0000 7f15d3fff700 1 --2- 172.21.15.99:0/3950323393 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15c4003a60 0x7f15c4003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:30.106 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.100+0000 7f15d3fff700 1 -- 172.21.15.99:0/3950323393 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15c40051c0 msgr2=0x7f15c400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:30.106 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.100+0000 7f15d3fff700 1 --2- 172.21.15.99:0/3950323393 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15c40051c0 0x7f15c400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:30.106 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.100+0000 7f15e11af700 1 -- 172.21.15.99:0/3950323393 reap_dead start 2023-10-08T00:15:30.106 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.100+0000 7f15d3fff700 1 -- 172.21.15.99:0/3950323393 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15c4003570 con 0x7f15c4004810 2023-10-08T00:15:30.106 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.100+0000 7f15e09ae700 1 --2- 172.21.15.99:0/3950323393 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15c4003a60 0x7f15c4003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:15:30.106 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.100+0000 7f15d3fff700 1 --2- 172.21.15.99:0/3950323393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15c4004810 0x7f15c4004c80 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f15dc0b6e10 tx=0x7f15cc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=973d3f848f37907a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:30.106 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.100+0000 7f15d37fe700 1 -- 172.21.15.99:0/3950323393 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15cc00e710 con 0x7f15c4004810 2023-10-08T00:15:30.106 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.100+0000 7f15d37fe700 1 -- 172.21.15.99:0/3950323393 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15cc00e870 con 0x7f15c4004810 2023-10-08T00:15:30.107 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.100+0000 7f15d37fe700 1 -- 172.21.15.99:0/3950323393 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15cc00ec30 con 0x7f15c4004810 2023-10-08T00:15:30.107 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.101+0000 7f15e2c12700 1 -- 172.21.15.99:0/3950323393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15c4004810 msgr2=0x7f15c4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:30.107 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.101+0000 7f15e2c12700 1 --2- 172.21.15.99:0/3950323393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15c4004810 0x7f15c4004c80 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f15dc0b6e10 tx=0x7f15cc0091a0 comp rx=0 tx=0).stop 2023-10-08T00:15:30.107 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.101+0000 7f15e2c12700 1 -- 172.21.15.99:0/3950323393 shutdown_connections 2023-10-08T00:15:30.107 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.101+0000 7f15e2c12700 1 --2- 172.21.15.99:0/3950323393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15c4004810 0x7f15c4004c80 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:30.107 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.101+0000 7f15e2c12700 1 -- 172.21.15.99:0/3950323393 >> 172.21.15.99:0/3950323393 conn(0x7f15dc09a0a0 msgr2=0x7f15dc067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:30.107 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.101+0000 7f15e2c12700 1 -- 172.21.15.99:0/3950323393 shutdown_connections 2023-10-08T00:15:30.107 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.101+0000 7f15e2c12700 1 -- 172.21.15.99:0/3950323393 wait complete. 2023-10-08T00:15:30.107 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.102+0000 7f15e2c12700 1 Processor -- start 2023-10-08T00:15:30.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.102+0000 7f15e2c12700 1 -- start start 2023-10-08T00:15:30.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.103+0000 7f15e2c12700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15c4004810 0x7f15dc1574c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:30.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.103+0000 7f15e2c12700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15dc15e7f0 0x7f15dc15ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:30.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.103+0000 7f15e2c12700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15dc0ae140 0x7f15dc0ae5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:30.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.103+0000 7f15e2c12700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15dc15f1a0 con 0x7f15dc15e7f0 2023-10-08T00:15:30.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.103+0000 7f15e2c12700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15dc157a00 con 0x7f15c4004810 2023-10-08T00:15:30.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.103+0000 7f15e2c12700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15dc0aeaf0 con 0x7f15dc0ae140 2023-10-08T00:15:30.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.103+0000 7f15d3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15dc15e7f0 0x7f15dc15ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:30.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.103+0000 7f15d3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15dc15e7f0 0x7f15dc15ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55988/0 (socket says 172.21.15.99:55988) 2023-10-08T00:15:30.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.103+0000 7f15d3fff700 1 -- 172.21.15.99:0/1355343982 learned_addr learned my addr 172.21.15.99:0/1355343982 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:30.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.103+0000 7f15e09ae700 1 --2- 172.21.15.99:0/1355343982 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15c4004810 0x7f15dc1574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:30.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.103+0000 7f15e11af700 1 --2- 172.21.15.99:0/1355343982 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15dc0ae140 0x7f15dc0ae5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:30.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.103+0000 7f15d3fff700 1 -- 172.21.15.99:0/1355343982 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15dc0ae140 msgr2=0x7f15dc0ae5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:30.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.103+0000 7f15d3fff700 1 --2- 172.21.15.99:0/1355343982 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15dc0ae140 0x7f15dc0ae5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:30.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.103+0000 7f15d3fff700 1 -- 172.21.15.99:0/1355343982 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15c4004810 msgr2=0x7f15dc1574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:30.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.103+0000 7f15d3fff700 1 --2- 172.21.15.99:0/1355343982 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15c4004810 0x7f15dc1574c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:30.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.103+0000 7f15d3fff700 1 -- 172.21.15.99:0/1355343982 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15dc0ac000 con 0x7f15dc15e7f0 2023-10-08T00:15:30.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.104+0000 7f15d3fff700 1 --2- 172.21.15.99:0/1355343982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15dc15e7f0 0x7f15dc15ec60 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f15cc013ef0 tx=0x7f15cc01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:30.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.104+0000 7f15d0ff9700 1 -- 172.21.15.99:0/1355343982 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15cc022070 con 0x7f15dc15e7f0 2023-10-08T00:15:30.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.104+0000 7f15d0ff9700 1 -- 172.21.15.99:0/1355343982 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15cc00e9e0 con 0x7f15dc15e7f0 2023-10-08T00:15:30.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.104+0000 7f15d0ff9700 1 -- 172.21.15.99:0/1355343982 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15cc01c450 con 0x7f15dc15e7f0 2023-10-08T00:15:30.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.104+0000 7f15e2c12700 1 -- 172.21.15.99:0/1355343982 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15dc0ac2e0 con 0x7f15dc15e7f0 2023-10-08T00:15:30.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.104+0000 7f15e2c12700 1 -- 172.21.15.99:0/1355343982 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15dc0ac8a0 con 0x7f15dc15e7f0 2023-10-08T00:15:30.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.105+0000 7f15e2c12700 1 -- 172.21.15.99:0/1355343982 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15dc0abd50 con 0x7f15dc15e7f0 2023-10-08T00:15:30.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.105+0000 7f15d0ff9700 1 -- 172.21.15.99:0/1355343982 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f15cc01c860 con 0x7f15dc15e7f0 2023-10-08T00:15:30.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.105+0000 7f15d0ff9700 1 --2- 172.21.15.99:0/1355343982 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f15c8066f60 0x7f15c8069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:30.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.105+0000 7f15d0ff9700 1 -- 172.21.15.99:0/1355343982 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f15cc085c10 con 0x7f15dc15e7f0 2023-10-08T00:15:30.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.106+0000 7f15e09ae700 1 --2- 172.21.15.99:0/1355343982 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f15c8066f60 0x7f15c8069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:30.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.106+0000 7f15e09ae700 1 --2- 172.21.15.99:0/1355343982 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f15c8066f60 0x7f15c8069410 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f15d8000e10 tx=0x7f15d8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:30.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.110+0000 7f15d0ff9700 1 -- 172.21.15.99:0/1355343982 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f15cc052ef0 con 0x7f15dc15e7f0 2023-10-08T00:15:30.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:30.297+0000 7f15e2c12700 1 -- 172.21.15.99:0/1355343982 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f15dc04c3c0 con 0x7f15dc15e7f0 2023-10-08T00:15:31.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.176+0000 7f15d0ff9700 1 -- 172.21.15.99:0/1355343982 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v94) v1 ==== 181+0+0 (secure 0 0 0) 0x7f15cc0170d0 con 0x7f15dc15e7f0 2023-10-08T00:15:31.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.179+0000 7f15e2c12700 1 -- 172.21.15.99:0/1355343982 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f15c8066f60 msgr2=0x7f15c8069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:31.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.179+0000 7f15e2c12700 1 --2- 172.21.15.99:0/1355343982 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f15c8066f60 0x7f15c8069410 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f15d8000e10 tx=0x7f15d8009040 comp rx=0 tx=0).stop 2023-10-08T00:15:31.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.179+0000 7f15e2c12700 1 -- 172.21.15.99:0/1355343982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15dc15e7f0 msgr2=0x7f15dc15ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:31.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.179+0000 7f15e2c12700 1 --2- 172.21.15.99:0/1355343982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15dc15e7f0 0x7f15dc15ec60 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f15cc013ef0 tx=0x7f15cc01a000 comp rx=0 tx=0).stop 2023-10-08T00:15:31.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.179+0000 7f15e2c12700 1 -- 172.21.15.99:0/1355343982 shutdown_connections 2023-10-08T00:15:31.182 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.179+0000 7f15e2c12700 1 --2- 172.21.15.99:0/1355343982 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f15c8066f60 0x7f15c8069410 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:31.182 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.179+0000 7f15e2c12700 1 --2- 172.21.15.99:0/1355343982 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15c4004810 0x7f15dc1574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:31.182 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.179+0000 7f15e2c12700 1 --2- 172.21.15.99:0/1355343982 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15dc0ae140 0x7f15dc0ae5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:31.182 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.179+0000 7f15e2c12700 1 --2- 172.21.15.99:0/1355343982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15dc15e7f0 0x7f15dc15ec60 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:31.182 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.179+0000 7f15e2c12700 1 -- 172.21.15.99:0/1355343982 >> 172.21.15.99:0/1355343982 conn(0x7f15dc09a0a0 msgr2=0x7f15dc06a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:31.182 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.179+0000 7f15e2c12700 1 -- 172.21.15.99:0/1355343982 shutdown_connections 2023-10-08T00:15:31.182 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.179+0000 7f15e2c12700 1 -- 172.21.15.99:0/1355343982 wait complete. 2023-10-08T00:15:31.182 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' removed 2023-10-08T00:15:31.195 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T00:15:31.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.315+0000 7f1ccda64700 1 Processor -- start 2023-10-08T00:15:31.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.315+0000 7f1ccda64700 1 -- start start 2023-10-08T00:15:31.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.315+0000 7f1ccda64700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1cc80af940 0x7f1cc80cb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:31.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.315+0000 7f1ccda64700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cc80b1ea0 con 0x7f1cc80b1950 2023-10-08T00:15:31.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.315+0000 7f1ccda64700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cc80b2180 con 0x7f1cc80b36c0 2023-10-08T00:15:31.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.315+0000 7f1ccda64700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cc80cb5f0 con 0x7f1cc80af940 2023-10-08T00:15:31.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.315+0000 7f1cc6ffd700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f1cc80b1950 0x7f1cc80ca9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:52230/0 (socket says 172.21.15.99:52230) 2023-10-08T00:15:31.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.315+0000 7f1cc6ffd700 1 -- 172.21.15.99:0/2149506650 learned_addr learned my addr 172.21.15.99:0/2149506650 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:15:31.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.316+0000 7f1cc67fc700 1 -- 172.21.15.99:0/2149506650 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4038663114 0 0) 0x7f1cc80b1ea0 con 0x7f1cc80b1950 2023-10-08T00:15:31.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.316+0000 7f1cc67fc700 1 -- 172.21.15.99:0/2149506650 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cb0003570 con 0x7f1cc80b1950 2023-10-08T00:15:31.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.316+0000 7f1cc7fff700 1 --2- 172.21.15.99:0/2149506650 >> v2:172.21.15.191:3300/0 conn(0x7f1cc80af940 0x7f1cc80cb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:31.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.316+0000 7f1cc67fc700 1 -- 172.21.15.99:0/2149506650 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 968251600 0 0) 0x7f1cb0003570 con 0x7f1cc80b1950 2023-10-08T00:15:31.321 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.316+0000 7f1cc67fc700 1 -- 172.21.15.99:0/2149506650 >> v2:172.21.15.191:3300/0 conn(0x7f1cc80af940 msgr2=0x7f1cc80cb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:31.321 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.316+0000 7f1cc67fc700 1 --2- 172.21.15.99:0/2149506650 >> v2:172.21.15.191:3300/0 conn(0x7f1cc80af940 0x7f1cc80cb0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:31.321 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.316+0000 7f1cc67fc700 1 -- 172.21.15.99:0/2149506650 >> v1:172.21.15.191:6789/0 conn(0x7f1cc80b36c0 legacy=0x7f1cc8151e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:31.321 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.316+0000 7f1cc67fc700 1 -- 172.21.15.99:0/2149506650 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc80b2460 con 0x7f1cc80b1950 2023-10-08T00:15:31.321 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.316+0000 7f1cc67fc700 1 -- 172.21.15.99:0/2149506650 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f1cbc001470 con 0x7f1cc80b1950 2023-10-08T00:15:31.321 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.316+0000 7f1cc7fff700 1 --2- 172.21.15.99:0/2149506650 >> v2:172.21.15.191:3300/0 conn(0x7f1cc80af940 0x7f1cc80cb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:15:31.321 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.316+0000 7f1cc67fc700 1 -- 172.21.15.99:0/2149506650 >> v1:172.21.15.99:6789/0 conn(0x7f1cc80b1950 legacy=0x7f1cc80ca9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:31.321 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.316+0000 7f1cc67fc700 1 --2- 172.21.15.99:0/2149506650 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1cb0003ab0 0x7f1cb0003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:31.321 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.316+0000 7f1cc67fc700 1 --2- 172.21.15.99:0/2149506650 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1cb0004810 0x7f1cb0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:31.321 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.316+0000 7f1cc67fc700 1 --2- 172.21.15.99:0/2149506650 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cb00052e0 0x7f1cb000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:31.321 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.316+0000 7f1cc67fc700 1 -- 172.21.15.99:0/2149506650 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc80cb5f0 con 0x7f1cb00052e0 2023-10-08T00:15:31.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.316+0000 7f1cc67fc700 1 -- 172.21.15.99:0/2149506650 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb000c1e0 con 0x7f1cb0003ab0 2023-10-08T00:15:31.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.316+0000 7f1cc67fc700 1 -- 172.21.15.99:0/2149506650 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb000c3a0 con 0x7f1cb0004810 2023-10-08T00:15:31.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.316+0000 7f1cc7fff700 1 --2- 172.21.15.99:0/2149506650 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cb00052e0 0x7f1cb000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:31.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.316+0000 7f1cc77fe700 1 --2- 172.21.15.99:0/2149506650 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1cb0003ab0 0x7f1cb0003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:31.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.316+0000 7f1cc6ffd700 1 --2- 172.21.15.99:0/2149506650 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1cb0004810 0x7f1cb0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:31.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.316+0000 7f1cc7fff700 1 -- 172.21.15.99:0/2149506650 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1cb0004810 msgr2=0x7f1cb0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:31.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.316+0000 7f1cc7fff700 1 --2- 172.21.15.99:0/2149506650 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1cb0004810 0x7f1cb0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:31.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.316+0000 7f1cc7fff700 1 -- 172.21.15.99:0/2149506650 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1cb0003ab0 msgr2=0x7f1cb0003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:31.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.316+0000 7f1cc7fff700 1 --2- 172.21.15.99:0/2149506650 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1cb0003ab0 0x7f1cb0003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:31.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.316+0000 7f1cc7fff700 1 -- 172.21.15.99:0/2149506650 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb0003570 con 0x7f1cb00052e0 2023-10-08T00:15:31.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.317+0000 7f1cc7fff700 1 -- 172.21.15.99:0/2149506650 reap_dead start 2023-10-08T00:15:31.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.317+0000 7f1cc7fff700 1 --2- 172.21.15.99:0/2149506650 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cb00052e0 0x7f1cb000ba60 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0009ba0 tx=0x7f1cc0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cad5b187fba149dc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:31.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.317+0000 7f1cc67fc700 1 -- 172.21.15.99:0/2149506650 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc0009510 con 0x7f1cb00052e0 2023-10-08T00:15:31.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.317+0000 7f1cc67fc700 1 -- 172.21.15.99:0/2149506650 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cc000e5e0 con 0x7f1cb00052e0 2023-10-08T00:15:31.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.317+0000 7f1cc67fc700 1 -- 172.21.15.99:0/2149506650 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc000e770 con 0x7f1cb00052e0 2023-10-08T00:15:31.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.317+0000 7f1ccda64700 1 -- 172.21.15.99:0/2149506650 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cb00052e0 msgr2=0x7f1cb000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:31.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.317+0000 7f1ccda64700 1 --2- 172.21.15.99:0/2149506650 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cb00052e0 0x7f1cb000ba60 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0009ba0 tx=0x7f1cc0008040 comp rx=0 tx=0).stop 2023-10-08T00:15:31.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.317+0000 7f1ccda64700 1 -- 172.21.15.99:0/2149506650 shutdown_connections 2023-10-08T00:15:31.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.317+0000 7f1ccda64700 1 --2- 172.21.15.99:0/2149506650 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cb00052e0 0x7f1cb000ba60 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:31.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.317+0000 7f1ccda64700 1 -- 172.21.15.99:0/2149506650 >> 172.21.15.99:0/2149506650 conn(0x7f1cc8069870 msgr2=0x7f1cc8069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:31.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.317+0000 7f1ccda64700 1 -- 172.21.15.99:0/2149506650 shutdown_connections 2023-10-08T00:15:31.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.317+0000 7f1ccda64700 1 -- 172.21.15.99:0/2149506650 wait complete. 2023-10-08T00:15:31.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.319+0000 7f1ccda64700 1 Processor -- start 2023-10-08T00:15:31.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.319+0000 7f1ccda64700 1 -- start start 2023-10-08T00:15:31.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.319+0000 7f1ccda64700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cb00052e0 0x7f1cc8154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:31.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.319+0000 7f1ccda64700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1cc80a6b00 0x7f1cc80a6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:31.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.319+0000 7f1ccda64700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1cc80aaef0 0x7f1cc80ab320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:31.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.319+0000 7f1ccda64700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc80a74b0 con 0x7f1cb00052e0 2023-10-08T00:15:31.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.319+0000 7f1ccda64700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc815ac70 con 0x7f1cc80a6b00 2023-10-08T00:15:31.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.319+0000 7f1ccda64700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc8154ff0 con 0x7f1cc80aaef0 2023-10-08T00:15:31.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.319+0000 7f1cc77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cb00052e0 0x7f1cc8154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:31.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.319+0000 7f1cc77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cb00052e0 0x7f1cc8154ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56004/0 (socket says 172.21.15.99:56004) 2023-10-08T00:15:31.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.319+0000 7f1cc77fe700 1 -- 172.21.15.99:0/3316529224 learned_addr learned my addr 172.21.15.99:0/3316529224 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:31.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.320+0000 7f1cc77fe700 1 -- 172.21.15.99:0/3316529224 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1cc80aaef0 msgr2=0x7f1cc80ab320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:31.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.320+0000 7f1cc77fe700 1 --2- 172.21.15.99:0/3316529224 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1cc80aaef0 0x7f1cc80ab320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:31.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.320+0000 7f1cc77fe700 1 -- 172.21.15.99:0/3316529224 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1cc80a6b00 msgr2=0x7f1cc80a6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:31.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.320+0000 7f1cc6ffd700 1 --2- 172.21.15.99:0/3316529224 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1cc80a6b00 0x7f1cc80a6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:31.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.320+0000 7f1cc77fe700 1 --2- 172.21.15.99:0/3316529224 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1cc80a6b00 0x7f1cc80a6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:31.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.320+0000 7f1cc77fe700 1 -- 172.21.15.99:0/3316529224 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc815a060 con 0x7f1cb00052e0 2023-10-08T00:15:31.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.320+0000 7f1cc77fe700 1 --2- 172.21.15.99:0/3316529224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cb00052e0 0x7f1cc8154ab0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f1cb800ebf0 tx=0x7f1cb8001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:31.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.320+0000 7f1caf7fe700 1 -- 172.21.15.99:0/3316529224 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb8019070 con 0x7f1cb00052e0 2023-10-08T00:15:31.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.320+0000 7f1caf7fe700 1 -- 172.21.15.99:0/3316529224 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cb8006ba0 con 0x7f1cb00052e0 2023-10-08T00:15:31.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.320+0000 7f1ccda64700 1 -- 172.21.15.99:0/3316529224 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cc815a340 con 0x7f1cb00052e0 2023-10-08T00:15:31.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.320+0000 7f1caf7fe700 1 -- 172.21.15.99:0/3316529224 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb800c8b0 con 0x7f1cb00052e0 2023-10-08T00:15:31.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.320+0000 7f1ccda64700 1 -- 172.21.15.99:0/3316529224 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cc815a900 con 0x7f1cb00052e0 2023-10-08T00:15:31.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.321+0000 7f1ccda64700 1 -- 172.21.15.99:0/3316529224 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cc806b900 con 0x7f1cb00052e0 2023-10-08T00:15:31.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.321+0000 7f1caf7fe700 1 -- 172.21.15.99:0/3316529224 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1cb800cc60 con 0x7f1cb00052e0 2023-10-08T00:15:31.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.322+0000 7f1caf7fe700 1 --2- 172.21.15.99:0/3316529224 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1cb4066f30 0x7f1cb40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:31.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.322+0000 7f1caf7fe700 1 -- 172.21.15.99:0/3316529224 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5353+0+0 (secure 0 0 0) 0x7f1cb8084760 con 0x7f1cb00052e0 2023-10-08T00:15:31.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.322+0000 7f1cc6ffd700 1 --2- 172.21.15.99:0/3316529224 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1cb4066f30 0x7f1cb40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:31.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.323+0000 7f1cc6ffd700 1 --2- 172.21.15.99:0/3316529224 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1cb4066f30 0x7f1cb40693e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc001050 tx=0x7f1cbc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:31.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.326+0000 7f1caf7fe700 1 -- 172.21.15.99:0/3316529224 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1cb8051b30 con 0x7f1cb00052e0 2023-10-08T00:15:31.513 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:31.512+0000 7f1ccda64700 1 -- 172.21.15.99:0/3316529224 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1cc804c3c0 con 0x7f1cb00052e0 2023-10-08T00:15:32.183 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.183+0000 7f1caf7fe700 1 -- 172.21.15.99:0/3316529224 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v95) v1 ==== 169+0+0 (secure 0 0 0) 0x7f1cb8055150 con 0x7f1cb00052e0 2023-10-08T00:15:32.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.186+0000 7f1ccda64700 1 -- 172.21.15.99:0/3316529224 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1cb4066f30 msgr2=0x7f1cb40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:32.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.186+0000 7f1ccda64700 1 --2- 172.21.15.99:0/3316529224 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1cb4066f30 0x7f1cb40693e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc001050 tx=0x7f1cbc009040 comp rx=0 tx=0).stop 2023-10-08T00:15:32.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.186+0000 7f1ccda64700 1 -- 172.21.15.99:0/3316529224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cb00052e0 msgr2=0x7f1cc8154ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:32.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.186+0000 7f1ccda64700 1 --2- 172.21.15.99:0/3316529224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cb00052e0 0x7f1cc8154ab0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f1cb800ebf0 tx=0x7f1cb8001640 comp rx=0 tx=0).stop 2023-10-08T00:15:32.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.186+0000 7f1ccda64700 1 -- 172.21.15.99:0/3316529224 shutdown_connections 2023-10-08T00:15:32.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.186+0000 7f1ccda64700 1 --2- 172.21.15.99:0/3316529224 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1cb4066f30 0x7f1cb40693e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:32.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.186+0000 7f1ccda64700 1 --2- 172.21.15.99:0/3316529224 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1cc80aaef0 0x7f1cc80ab320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:32.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.186+0000 7f1ccda64700 1 --2- 172.21.15.99:0/3316529224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cb00052e0 0x7f1cc8154ab0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:32.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.186+0000 7f1ccda64700 1 --2- 172.21.15.99:0/3316529224 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1cc80a6b00 0x7f1cc80a6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:32.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.186+0000 7f1ccda64700 1 -- 172.21.15.99:0/3316529224 >> 172.21.15.99:0/3316529224 conn(0x7f1cc8069870 msgr2=0x7f1cc81568f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:32.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.186+0000 7f1ccda64700 1 -- 172.21.15.99:0/3316529224 shutdown_connections 2023-10-08T00:15:32.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.186+0000 7f1ccda64700 1 -- 172.21.15.99:0/3316529224 wait complete. 2023-10-08T00:15:32.188 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' removed 2023-10-08T00:15:32.201 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-08T00:15:32.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.319+0000 7fb90e066700 1 Processor -- start 2023-10-08T00:15:32.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.319+0000 7fb90e066700 1 -- start start 2023-10-08T00:15:32.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.320+0000 7fb90e066700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb90806aa00 0x7fb908164c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:32.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.320+0000 7fb90e066700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9080b4d70 con 0x7fb9080cce30 2023-10-08T00:15:32.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.320+0000 7fb90e066700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb90815b3b0 con 0x7fb9080b6580 2023-10-08T00:15:32.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.320+0000 7fb90e066700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9080cd8d0 con 0x7fb90806aa00 2023-10-08T00:15:32.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.320+0000 7fb90d064700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fb9080cce30 0x7fb90815aca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:52240/0 (socket says 172.21.15.99:52240) 2023-10-08T00:15:32.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.320+0000 7fb90d064700 1 -- 172.21.15.99:0/2566977516 learned_addr learned my addr 172.21.15.99:0/2566977516 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:15:32.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.320+0000 7fb90d865700 1 --2- 172.21.15.99:0/2566977516 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb90806aa00 0x7fb908164c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:32.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.320+0000 7fb8fffff700 1 -- 172.21.15.99:0/2566977516 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2767545301 0 0) 0x7fb9080b4d70 con 0x7fb9080cce30 2023-10-08T00:15:32.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.320+0000 7fb8fffff700 1 -- 172.21.15.99:0/2566977516 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8f4003570 con 0x7fb9080cce30 2023-10-08T00:15:32.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.321+0000 7fb8fffff700 1 -- 172.21.15.99:0/2566977516 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3804225652 0 0) 0x7fb90815b3b0 con 0x7fb9080b6580 2023-10-08T00:15:32.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.321+0000 7fb8fffff700 1 -- 172.21.15.99:0/2566977516 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9080b4d70 con 0x7fb9080b6580 2023-10-08T00:15:32.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.321+0000 7fb8fffff700 1 -- 172.21.15.99:0/2566977516 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2310075936 0 0) 0x7fb8f4003570 con 0x7fb9080cce30 2023-10-08T00:15:32.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.321+0000 7fb8fffff700 1 -- 172.21.15.99:0/2566977516 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb90806aa00 msgr2=0x7fb908164c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:32.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.321+0000 7fb8fffff700 1 --2- 172.21.15.99:0/2566977516 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb90806aa00 0x7fb908164c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:32.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.321+0000 7fb8fffff700 1 -- 172.21.15.99:0/2566977516 >> v1:172.21.15.191:6789/0 conn(0x7fb9080b6580 legacy=0x7fb90806a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:32.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.321+0000 7fb8fffff700 1 -- 172.21.15.99:0/2566977516 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9080b4440 con 0x7fb9080cce30 2023-10-08T00:15:32.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.321+0000 7fb8fffff700 1 -- 172.21.15.99:0/2566977516 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7fb904001470 con 0x7fb9080cce30 2023-10-08T00:15:32.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.321+0000 7fb8fffff700 1 -- 172.21.15.99:0/2566977516 >> v1:172.21.15.99:6789/0 conn(0x7fb9080cce30 legacy=0x7fb90815aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:32.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.321+0000 7fb8fffff700 1 --2- 172.21.15.99:0/2566977516 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb90806aa00 0x7fb8f4003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:32.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.321+0000 7fb8fffff700 1 --2- 172.21.15.99:0/2566977516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8f4004180 0x7fb8f4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:32.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.321+0000 7fb8fffff700 1 --2- 172.21.15.99:0/2566977516 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8f4004d50 0x7fb8f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:32.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.321+0000 7fb8fffff700 1 -- 172.21.15.99:0/2566977516 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9080cd8d0 con 0x7fb8f4004180 2023-10-08T00:15:32.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.321+0000 7fb8fffff700 1 -- 172.21.15.99:0/2566977516 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f400bba0 con 0x7fb8f4004d50 2023-10-08T00:15:32.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.321+0000 7fb8fffff700 1 -- 172.21.15.99:0/2566977516 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f400bd60 con 0x7fb90806aa00 2023-10-08T00:15:32.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.321+0000 7fb90c863700 1 --2- 172.21.15.99:0/2566977516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8f4004180 0x7fb8f4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:32.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.321+0000 7fb90d064700 1 --2- 172.21.15.99:0/2566977516 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb90806aa00 0x7fb8f4003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:32.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.321+0000 7fb90d865700 1 --2- 172.21.15.99:0/2566977516 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8f4004d50 0x7fb8f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:32.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.321+0000 7fb90c863700 1 -- 172.21.15.99:0/2566977516 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb90806aa00 msgr2=0x7fb8f4003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:32.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.321+0000 7fb90c863700 1 --2- 172.21.15.99:0/2566977516 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb90806aa00 0x7fb8f4003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:32.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.321+0000 7fb90c863700 1 -- 172.21.15.99:0/2566977516 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8f4004d50 msgr2=0x7fb8f40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:32.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.321+0000 7fb90c863700 1 --2- 172.21.15.99:0/2566977516 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8f4004d50 0x7fb8f40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:32.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.321+0000 7fb90c863700 1 -- 172.21.15.99:0/2566977516 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f4003570 con 0x7fb8f4004180 2023-10-08T00:15:32.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.321+0000 7fb90c863700 1 --2- 172.21.15.99:0/2566977516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8f4004180 0x7fb8f4004810 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb8f8001a50 tx=0x7fb8f8007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67fcef377d883f74 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:32.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.321+0000 7fb8fffff700 1 -- 172.21.15.99:0/2566977516 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f800c5d0 con 0x7fb8f4004180 2023-10-08T00:15:32.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.321+0000 7fb8fffff700 1 -- 172.21.15.99:0/2566977516 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8f800c730 con 0x7fb8f4004180 2023-10-08T00:15:32.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.321+0000 7fb8fffff700 1 -- 172.21.15.99:0/2566977516 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f800cad0 con 0x7fb8f4004180 2023-10-08T00:15:32.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.322+0000 7fb90e066700 1 -- 172.21.15.99:0/2566977516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8f4004180 msgr2=0x7fb8f4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:32.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.322+0000 7fb90e066700 1 --2- 172.21.15.99:0/2566977516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8f4004180 0x7fb8f4004810 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb8f8001a50 tx=0x7fb8f8007d30 comp rx=0 tx=0).stop 2023-10-08T00:15:32.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.322+0000 7fb90d865700 1 -- 172.21.15.99:0/2566977516 reap_dead start 2023-10-08T00:15:32.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.322+0000 7fb90e066700 1 -- 172.21.15.99:0/2566977516 shutdown_connections 2023-10-08T00:15:32.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.322+0000 7fb90e066700 1 -- 172.21.15.99:0/2566977516 >> 172.21.15.99:0/2566977516 conn(0x7fb908068cb0 msgr2=0x7fb9080690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:32.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.322+0000 7fb90e066700 1 -- 172.21.15.99:0/2566977516 shutdown_connections 2023-10-08T00:15:32.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.322+0000 7fb90e066700 1 -- 172.21.15.99:0/2566977516 wait complete. 2023-10-08T00:15:32.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.323+0000 7fb90e066700 1 Processor -- start 2023-10-08T00:15:32.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.323+0000 7fb90e066700 1 -- start start 2023-10-08T00:15:32.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.324+0000 7fb90e066700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8f4004180 0x7fb9081559d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:32.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.324+0000 7fb90e066700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb908153c10 0x7fb9081595d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:32.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.324+0000 7fb90e066700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb908150320 0x7fb908156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:32.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.324+0000 7fb90e066700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb908155540 con 0x7fb908153c10 2023-10-08T00:15:32.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.324+0000 7fb90e066700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb908155f10 con 0x7fb8f4004180 2023-10-08T00:15:32.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.324+0000 7fb90e066700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb908154930 con 0x7fb908150320 2023-10-08T00:15:32.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.324+0000 7fb90c863700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb908153c10 0x7fb9081595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:32.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.324+0000 7fb90c863700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb908153c10 0x7fb9081595d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56018/0 (socket says 172.21.15.99:56018) 2023-10-08T00:15:32.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.324+0000 7fb90c863700 1 -- 172.21.15.99:0/3673284168 learned_addr learned my addr 172.21.15.99:0/3673284168 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:32.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.324+0000 7fb90d064700 1 --2- 172.21.15.99:0/3673284168 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8f4004180 0x7fb9081559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:32.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.324+0000 7fb90d865700 1 --2- 172.21.15.99:0/3673284168 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb908150320 0x7fb908156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:32.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.324+0000 7fb90c863700 1 -- 172.21.15.99:0/3673284168 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb908150320 msgr2=0x7fb908156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:32.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.324+0000 7fb90c863700 1 --2- 172.21.15.99:0/3673284168 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb908150320 0x7fb908156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:32.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.324+0000 7fb90c863700 1 -- 172.21.15.99:0/3673284168 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8f4004180 msgr2=0x7fb9081559d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:32.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.324+0000 7fb90c863700 1 --2- 172.21.15.99:0/3673284168 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8f4004180 0x7fb9081559d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:32.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.324+0000 7fb90c863700 1 -- 172.21.15.99:0/3673284168 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb908154c10 con 0x7fb908153c10 2023-10-08T00:15:32.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.325+0000 7fb90c863700 1 --2- 172.21.15.99:0/3673284168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb908153c10 0x7fb9081595d0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fb8f8001ed0 tx=0x7fb8f8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:32.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.325+0000 7fb8fd7fa700 1 -- 172.21.15.99:0/3673284168 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f800c3f0 con 0x7fb908153c10 2023-10-08T00:15:32.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.325+0000 7fb8fd7fa700 1 -- 172.21.15.99:0/3673284168 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8f800cab0 con 0x7fb908153c10 2023-10-08T00:15:32.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.325+0000 7fb90e066700 1 -- 172.21.15.99:0/3673284168 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb908154ef0 con 0x7fb908153c10 2023-10-08T00:15:32.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.325+0000 7fb8fd7fa700 1 -- 172.21.15.99:0/3673284168 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f8006a70 con 0x7fb908153c10 2023-10-08T00:15:32.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.325+0000 7fb90e066700 1 -- 172.21.15.99:0/3673284168 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb908159b10 con 0x7fb908153c10 2023-10-08T00:15:32.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.326+0000 7fb90e066700 1 -- 172.21.15.99:0/3673284168 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9081556f0 con 0x7fb908153c10 2023-10-08T00:15:32.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.326+0000 7fb8fd7fa700 1 -- 172.21.15.99:0/3673284168 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb8f8019070 con 0x7fb908153c10 2023-10-08T00:15:32.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.326+0000 7fb8fd7fa700 1 --2- 172.21.15.99:0/3673284168 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb8ec066f60 0x7fb8ec069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:32.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.327+0000 7fb8fd7fa700 1 -- 172.21.15.99:0/3673284168 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fb8f8085680 con 0x7fb908153c10 2023-10-08T00:15:32.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.327+0000 7fb90d064700 1 --2- 172.21.15.99:0/3673284168 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb8ec066f60 0x7fb8ec069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:32.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.327+0000 7fb90d064700 1 --2- 172.21.15.99:0/3673284168 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb8ec066f60 0x7fb8ec069410 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb904001200 tx=0x7fb9080ce820 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:32.333 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.331+0000 7fb8fd7fa700 1 -- 172.21.15.99:0/3673284168 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb9081556f0 con 0x7fb908153c10 2023-10-08T00:15:32.513 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.512+0000 7fb90e066700 1 -- 172.21.15.99:0/3673284168 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb9080b4720 con 0x7fb908153c10 2023-10-08T00:15:32.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.516+0000 7fb8fd7fa700 1 -- 172.21.15.99:0/3673284168 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb8f8052bf0 con 0x7fb908153c10 2023-10-08T00:15:32.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.518+0000 7fb90e066700 1 -- 172.21.15.99:0/3673284168 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb8ec066f60 msgr2=0x7fb8ec069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:32.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.518+0000 7fb90e066700 1 --2- 172.21.15.99:0/3673284168 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb8ec066f60 0x7fb8ec069410 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb904001200 tx=0x7fb9080ce820 comp rx=0 tx=0).stop 2023-10-08T00:15:32.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.518+0000 7fb90e066700 1 -- 172.21.15.99:0/3673284168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb908153c10 msgr2=0x7fb9081595d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:32.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.518+0000 7fb90e066700 1 --2- 172.21.15.99:0/3673284168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb908153c10 0x7fb9081595d0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fb8f8001ed0 tx=0x7fb8f8015000 comp rx=0 tx=0).stop 2023-10-08T00:15:32.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.518+0000 7fb90e066700 1 -- 172.21.15.99:0/3673284168 shutdown_connections 2023-10-08T00:15:32.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.518+0000 7fb90e066700 1 --2- 172.21.15.99:0/3673284168 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb8ec066f60 0x7fb8ec069410 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:32.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.518+0000 7fb90e066700 1 --2- 172.21.15.99:0/3673284168 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8f4004180 0x7fb9081559d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:32.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.518+0000 7fb90e066700 1 --2- 172.21.15.99:0/3673284168 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb908150320 0x7fb908156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:32.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.518+0000 7fb90e066700 1 --2- 172.21.15.99:0/3673284168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb908153c10 0x7fb9081595d0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:32.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.518+0000 7fb90e066700 1 -- 172.21.15.99:0/3673284168 >> 172.21.15.99:0/3673284168 conn(0x7fb908068cb0 msgr2=0x7fb9080b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:32.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.518+0000 7fb90e066700 1 -- 172.21.15.99:0/3673284168 shutdown_connections 2023-10-08T00:15:32.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.518+0000 7fb90e066700 1 -- 172.21.15.99:0/3673284168 wait complete. 2023-10-08T00:15:32.531 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fsync' 2023-10-08T00:15:32.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.644+0000 7f8e2e83a700 1 Processor -- start 2023-10-08T00:15:32.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.644+0000 7f8e2e83a700 1 -- start start 2023-10-08T00:15:32.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.645+0000 7f8e2e83a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8e2815a1e0 0x7f8e280cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:32.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.645+0000 7f8e2e83a700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8e280b3430 0x7f8e280cdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:32.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.645+0000 7f8e2e83a700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e2815db80 con 0x7f8e28159e10 2023-10-08T00:15:32.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.645+0000 7f8e2e83a700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e2815e800 con 0x7f8e280b3430 2023-10-08T00:15:32.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.645+0000 7f8e2e83a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2815de60 con 0x7f8e2815a1e0 2023-10-08T00:15:32.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.645+0000 7f8e27fff700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f8e28159e10 0x7f8e280b2d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:52244/0 (socket says 172.21.15.99:52244) 2023-10-08T00:15:32.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.645+0000 7f8e27fff700 1 -- 172.21.15.99:0/38753778 learned_addr learned my addr 172.21.15.99:0/38753778 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:15:32.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.645+0000 7f8e26ffd700 1 -- 172.21.15.99:0/38753778 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2528002170 0 0) 0x7f8e2815db80 con 0x7f8e28159e10 2023-10-08T00:15:32.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.645+0000 7f8e26ffd700 1 -- 172.21.15.99:0/38753778 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e10003570 con 0x7f8e28159e10 2023-10-08T00:15:32.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.645+0000 7f8e277fe700 1 --2- 172.21.15.99:0/38753778 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8e2815a1e0 0x7f8e280cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:32.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.645+0000 7f8e2cdd7700 1 --2- 172.21.15.99:0/38753778 >> v2:172.21.15.191:3300/0 conn(0x7f8e280b3430 0x7f8e280cdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:32.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.645+0000 7f8e26ffd700 1 -- 172.21.15.99:0/38753778 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1831875905 0 0) 0x7f8e10003570 con 0x7f8e28159e10 2023-10-08T00:15:32.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.645+0000 7f8e26ffd700 1 -- 172.21.15.99:0/38753778 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8e2815a1e0 msgr2=0x7f8e280cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:32.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.645+0000 7f8e26ffd700 1 --2- 172.21.15.99:0/38753778 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8e2815a1e0 0x7f8e280cd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:32.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.645+0000 7f8e26ffd700 1 -- 172.21.15.99:0/38753778 >> v2:172.21.15.191:3300/0 conn(0x7f8e280b3430 msgr2=0x7f8e280cdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:32.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.645+0000 7f8e26ffd700 1 --2- 172.21.15.99:0/38753778 >> v2:172.21.15.191:3300/0 conn(0x7f8e280b3430 0x7f8e280cdc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:32.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.645+0000 7f8e26ffd700 1 -- 172.21.15.99:0/38753778 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e2815e140 con 0x7f8e28159e10 2023-10-08T00:15:32.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.645+0000 7f8e26ffd700 1 -- 172.21.15.99:0/38753778 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f8e1c001470 con 0x7f8e28159e10 2023-10-08T00:15:32.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.645+0000 7f8e26ffd700 1 -- 172.21.15.99:0/38753778 >> v1:172.21.15.99:6789/0 conn(0x7f8e28159e10 legacy=0x7f8e280b2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:32.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.645+0000 7f8e26ffd700 1 --2- 172.21.15.99:0/38753778 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8e2815a1e0 0x7f8e10003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:32.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.645+0000 7f8e26ffd700 1 --2- 172.21.15.99:0/38753778 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e10003fe0 0x7f8e10004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:32.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.645+0000 7f8e26ffd700 1 --2- 172.21.15.99:0/38753778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e10004d50 0x7f8e100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:32.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.645+0000 7f8e26ffd700 1 -- 172.21.15.99:0/38753778 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2815e800 con 0x7f8e10004d50 2023-10-08T00:15:32.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.645+0000 7f8e26ffd700 1 -- 172.21.15.99:0/38753778 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2815de60 con 0x7f8e10003fe0 2023-10-08T00:15:32.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.645+0000 7f8e26ffd700 1 -- 172.21.15.99:0/38753778 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1000ba90 con 0x7f8e2815a1e0 2023-10-08T00:15:32.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.646+0000 7f8e2cdd7700 1 --2- 172.21.15.99:0/38753778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e10004d50 0x7f8e100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:32.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.646+0000 7f8e27fff700 1 --2- 172.21.15.99:0/38753778 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8e2815a1e0 0x7f8e10003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:32.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.646+0000 7f8e277fe700 1 --2- 172.21.15.99:0/38753778 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e10003fe0 0x7f8e10004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:32.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.646+0000 7f8e2cdd7700 1 -- 172.21.15.99:0/38753778 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8e2815a1e0 msgr2=0x7f8e10003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:32.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.646+0000 7f8e2cdd7700 1 --2- 172.21.15.99:0/38753778 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8e2815a1e0 0x7f8e10003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:32.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.646+0000 7f8e2cdd7700 1 -- 172.21.15.99:0/38753778 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e10003fe0 msgr2=0x7f8e10004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:32.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.646+0000 7f8e2cdd7700 1 --2- 172.21.15.99:0/38753778 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e10003fe0 0x7f8e10004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:32.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.646+0000 7f8e2cdd7700 1 -- 172.21.15.99:0/38753778 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e10003570 con 0x7f8e10004d50 2023-10-08T00:15:32.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.646+0000 7f8e277fe700 1 --2- 172.21.15.99:0/38753778 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e10003fe0 0x7f8e10004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:15:32.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.646+0000 7f8e2cdd7700 1 --2- 172.21.15.99:0/38753778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e10004d50 0x7f8e100051c0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f8e28069d20 tx=0x7f8e2000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28c9c5fc8aa41da0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:32.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.646+0000 7f8e26ffd700 1 -- 172.21.15.99:0/38753778 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2000ea30 con 0x7f8e10004d50 2023-10-08T00:15:32.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.646+0000 7f8e26ffd700 1 -- 172.21.15.99:0/38753778 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e2000f040 con 0x7f8e10004d50 2023-10-08T00:15:32.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.646+0000 7f8e26ffd700 1 -- 172.21.15.99:0/38753778 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2000ede0 con 0x7f8e10004d50 2023-10-08T00:15:32.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.646+0000 7f8e2e83a700 1 -- 172.21.15.99:0/38753778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e10004d50 msgr2=0x7f8e100051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:32.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.646+0000 7f8e2e83a700 1 --2- 172.21.15.99:0/38753778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e10004d50 0x7f8e100051c0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f8e28069d20 tx=0x7f8e2000b040 comp rx=0 tx=0).stop 2023-10-08T00:15:32.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.647+0000 7f8e2cdd7700 1 -- 172.21.15.99:0/38753778 reap_dead start 2023-10-08T00:15:32.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.647+0000 7f8e2e83a700 1 -- 172.21.15.99:0/38753778 shutdown_connections 2023-10-08T00:15:32.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.647+0000 7f8e2e83a700 1 -- 172.21.15.99:0/38753778 >> 172.21.15.99:0/38753778 conn(0x7f8e28067660 msgr2=0x7f8e28067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:32.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.647+0000 7f8e2e83a700 1 -- 172.21.15.99:0/38753778 shutdown_connections 2023-10-08T00:15:32.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.647+0000 7f8e2e83a700 1 -- 172.21.15.99:0/38753778 wait complete. 2023-10-08T00:15:32.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.648+0000 7f8e2e83a700 1 Processor -- start 2023-10-08T00:15:32.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.648+0000 7f8e2e83a700 1 -- start start 2023-10-08T00:15:32.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.649+0000 7f8e2e83a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e10004d50 0x7f8e280c4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:32.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.649+0000 7f8e2e83a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8e280c38f0 0x7f8e280c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:32.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.649+0000 7f8e2e83a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e280c3d60 0x7f8e280c8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:32.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.649+0000 7f8e2e83a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e280c87d0 con 0x7f8e280c3d60 2023-10-08T00:15:32.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.649+0000 7f8e2e83a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e280cf8d0 con 0x7f8e10004d50 2023-10-08T00:15:32.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.649+0000 7f8e2e83a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e280ca680 con 0x7f8e280c38f0 2023-10-08T00:15:32.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.649+0000 7f8e2cdd7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e280c3d60 0x7f8e280c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:32.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.649+0000 7f8e2cdd7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e280c3d60 0x7f8e280c8290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56042/0 (socket says 172.21.15.99:56042) 2023-10-08T00:15:32.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.649+0000 7f8e2cdd7700 1 -- 172.21.15.99:0/287630055 learned_addr learned my addr 172.21.15.99:0/287630055 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:32.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.649+0000 7f8e277fe700 1 --2- 172.21.15.99:0/287630055 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8e280c38f0 0x7f8e280c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:32.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.649+0000 7f8e27fff700 1 --2- 172.21.15.99:0/287630055 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e10004d50 0x7f8e280c4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:32.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.649+0000 7f8e2cdd7700 1 -- 172.21.15.99:0/287630055 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8e280c38f0 msgr2=0x7f8e280c7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:32.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.649+0000 7f8e2cdd7700 1 --2- 172.21.15.99:0/287630055 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8e280c38f0 0x7f8e280c7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:32.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.649+0000 7f8e2cdd7700 1 -- 172.21.15.99:0/287630055 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e10004d50 msgr2=0x7f8e280c4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:32.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.649+0000 7f8e2cdd7700 1 --2- 172.21.15.99:0/287630055 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e10004d50 0x7f8e280c4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:32.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.649+0000 7f8e2cdd7700 1 -- 172.21.15.99:0/287630055 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e280c51b0 con 0x7f8e280c3d60 2023-10-08T00:15:32.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.649+0000 7f8e2cdd7700 1 --2- 172.21.15.99:0/287630055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e280c3d60 0x7f8e280c8290 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f8e2000afd0 tx=0x7f8e2001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:32.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.649+0000 7f8e0ffff700 1 -- 172.21.15.99:0/287630055 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2000eeb0 con 0x7f8e280c3d60 2023-10-08T00:15:32.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.649+0000 7f8e0ffff700 1 -- 172.21.15.99:0/287630055 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e20022070 con 0x7f8e280c3d60 2023-10-08T00:15:32.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.649+0000 7f8e2e83a700 1 -- 172.21.15.99:0/287630055 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e280ca920 con 0x7f8e280c3d60 2023-10-08T00:15:32.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.649+0000 7f8e0ffff700 1 -- 172.21.15.99:0/287630055 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2001e890 con 0x7f8e280c3d60 2023-10-08T00:15:32.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.649+0000 7f8e2e83a700 1 -- 172.21.15.99:0/287630055 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e280caee0 con 0x7f8e280c3d60 2023-10-08T00:15:32.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.650+0000 7f8e2e83a700 1 -- 172.21.15.99:0/287630055 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e2804c3c0 con 0x7f8e280c3d60 2023-10-08T00:15:32.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.651+0000 7f8e0ffff700 1 -- 172.21.15.99:0/287630055 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8e20003040 con 0x7f8e280c3d60 2023-10-08T00:15:32.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.651+0000 7f8e0ffff700 1 --2- 172.21.15.99:0/287630055 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8e14066f30 0x7f8e140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:32.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.651+0000 7f8e0ffff700 1 -- 172.21.15.99:0/287630055 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f8e20024070 con 0x7f8e280c3d60 2023-10-08T00:15:32.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.651+0000 7f8e27fff700 1 --2- 172.21.15.99:0/287630055 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8e14066f30 0x7f8e140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:32.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.652+0000 7f8e27fff700 1 --2- 172.21.15.99:0/287630055 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8e14066f30 0x7f8e140693e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c0010e0 tx=0x7f8e1c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:32.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.655+0000 7f8e0ffff700 1 -- 172.21.15.99:0/287630055 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8e20052d40 con 0x7f8e280c3d60 2023-10-08T00:15:32.912 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:32.912+0000 7f8e2e83a700 1 -- 172.21.15.99:0/287630055 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]} v 0) v1 -- 0x7f8e280cfa10 con 0x7f8e280c3d60 2023-10-08T00:15:33.184 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.184+0000 7f8e0ffff700 1 -- 172.21.15.99:0/287630055 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]}]=0 v240) v1 ==== 133+0+0 (secure 0 0 0) 0x7f8e20056360 con 0x7f8e280c3d60 2023-10-08T00:15:33.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.186+0000 7f8e2e83a700 1 -- 172.21.15.99:0/287630055 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8e14066f30 msgr2=0x7f8e140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:33.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.186+0000 7f8e2e83a700 1 --2- 172.21.15.99:0/287630055 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8e14066f30 0x7f8e140693e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c0010e0 tx=0x7f8e1c009040 comp rx=0 tx=0).stop 2023-10-08T00:15:33.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.186+0000 7f8e2e83a700 1 -- 172.21.15.99:0/287630055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e280c3d60 msgr2=0x7f8e280c8290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:33.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.186+0000 7f8e2e83a700 1 --2- 172.21.15.99:0/287630055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e280c3d60 0x7f8e280c8290 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f8e2000afd0 tx=0x7f8e2001a000 comp rx=0 tx=0).stop 2023-10-08T00:15:33.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.186+0000 7f8e2e83a700 1 -- 172.21.15.99:0/287630055 shutdown_connections 2023-10-08T00:15:33.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.186+0000 7f8e2e83a700 1 --2- 172.21.15.99:0/287630055 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8e14066f30 0x7f8e140693e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:33.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.186+0000 7f8e2e83a700 1 --2- 172.21.15.99:0/287630055 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e10004d50 0x7f8e280c4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:33.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.186+0000 7f8e2e83a700 1 --2- 172.21.15.99:0/287630055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e280c3d60 0x7f8e280c8290 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:33.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.186+0000 7f8e2e83a700 1 --2- 172.21.15.99:0/287630055 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8e280c38f0 0x7f8e280c7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:33.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.186+0000 7f8e2e83a700 1 -- 172.21.15.99:0/287630055 >> 172.21.15.99:0/287630055 conn(0x7f8e28067660 msgr2=0x7f8e2815a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:33.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.186+0000 7f8e2e83a700 1 -- 172.21.15.99:0/287630055 shutdown_connections 2023-10-08T00:15:33.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.186+0000 7f8e2e83a700 1 -- 172.21.15.99:0/287630055 wait complete. 2023-10-08T00:15:33.199 INFO:tasks.cephfs_test_runner:That when the full flag is encountered during asynchronous ... ok 2023-10-08T00:15:33.200 INFO:tasks.cephfs_test_runner:Starting test: test_full_same_file (tasks.cephfs.test_full.TestClusterFull) 2023-10-08T00:15:33.200 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_same_file' 2023-10-08T00:15:33.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.314+0000 7ff26d925700 1 Processor -- start 2023-10-08T00:15:33.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.314+0000 7ff26d925700 1 -- start start 2023-10-08T00:15:33.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.315+0000 7ff26d925700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff268152010 0x7ff26815b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:33.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.315+0000 7ff26d925700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff26806c010 con 0x7ff268158e50 2023-10-08T00:15:33.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.315+0000 7ff26d925700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff26806bd10 con 0x7ff2681523e0 2023-10-08T00:15:33.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.315+0000 7ff26d925700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff26815c060 con 0x7ff268152010 2023-10-08T00:15:33.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.315+0000 7ff266ffd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff268152010 0x7ff26815b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:33.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.315+0000 7ff2677fe700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7ff268158e50 0x7ff2680cdf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:52254/0 (socket says 172.21.15.99:52254) 2023-10-08T00:15:33.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.315+0000 7ff266ffd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff268152010 0x7ff26815b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56054/0 (socket says 172.21.15.99:56054) 2023-10-08T00:15:33.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.315+0000 7ff2677fe700 1 -- 172.21.15.99:0/3175496187 learned_addr learned my addr 172.21.15.99:0/3175496187 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:15:33.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.316+0000 7ff266ffd700 1 -- 172.21.15.99:0/3175496187 >> v1:172.21.15.191:6789/0 conn(0x7ff2681523e0 legacy=0x7ff26806c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:33.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.316+0000 7ff266ffd700 1 -- 172.21.15.99:0/3175496187 >> v1:172.21.15.99:6789/0 conn(0x7ff268158e50 legacy=0x7ff2680cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:33.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.316+0000 7ff266ffd700 1 -- 172.21.15.99:0/3175496187 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff26815c340 con 0x7ff268152010 2023-10-08T00:15:33.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.316+0000 7ff2667fc700 1 -- 172.21.15.99:0/3175496187 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2515440515 0 0) 0x7ff26806c010 con 0x7ff268158e50 2023-10-08T00:15:33.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.316+0000 7ff266ffd700 1 --2- 172.21.15.99:0/3175496187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff268152010 0x7ff26815b810 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7ff2580098e0 tx=0x7ff2580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a091640ed04709e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:33.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.316+0000 7ff2667fc700 1 -- 172.21.15.99:0/3175496187 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff258009e80 con 0x7ff268152010 2023-10-08T00:15:33.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.316+0000 7ff2667fc700 1 -- 172.21.15.99:0/3175496187 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff25800b860 con 0x7ff268152010 2023-10-08T00:15:33.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.316+0000 7ff2667fc700 1 -- 172.21.15.99:0/3175496187 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25800bbe0 con 0x7ff268152010 2023-10-08T00:15:33.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.316+0000 7ff26d925700 1 -- 172.21.15.99:0/3175496187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff268152010 msgr2=0x7ff26815b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:33.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.316+0000 7ff26d925700 1 --2- 172.21.15.99:0/3175496187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff268152010 0x7ff26815b810 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7ff2580098e0 tx=0x7ff2580091a0 comp rx=0 tx=0).stop 2023-10-08T00:15:33.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.316+0000 7ff26d925700 1 -- 172.21.15.99:0/3175496187 shutdown_connections 2023-10-08T00:15:33.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.316+0000 7ff26d925700 1 --2- 172.21.15.99:0/3175496187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff268152010 0x7ff26815b810 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:33.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.316+0000 7ff26d925700 1 -- 172.21.15.99:0/3175496187 >> 172.21.15.99:0/3175496187 conn(0x7ff268069870 msgr2=0x7ff268069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:33.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.317+0000 7ff26d925700 1 -- 172.21.15.99:0/3175496187 shutdown_connections 2023-10-08T00:15:33.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.317+0000 7ff26d925700 1 -- 172.21.15.99:0/3175496187 wait complete. 2023-10-08T00:15:33.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.318+0000 7ff26d925700 1 Processor -- start 2023-10-08T00:15:33.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.318+0000 7ff26d925700 1 -- start start 2023-10-08T00:15:33.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.318+0000 7ff26d925700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff268152010 0x7ff2680c6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:33.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.318+0000 7ff26d925700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff2681523e0 0x7ff2680b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:33.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.318+0000 7ff26d925700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff268158e50 0x7ff26814f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:33.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.318+0000 7ff26d925700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2680b3d90 con 0x7ff268158e50 2023-10-08T00:15:33.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.318+0000 7ff26d925700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26814fcd0 con 0x7ff268152010 2023-10-08T00:15:33.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.318+0000 7ff26d925700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2680b3180 con 0x7ff2681523e0 2023-10-08T00:15:33.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.319+0000 7ff2677fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff268158e50 0x7ff26814f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:33.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.319+0000 7ff2677fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff268158e50 0x7ff26814f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56066/0 (socket says 172.21.15.99:56066) 2023-10-08T00:15:33.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.319+0000 7ff2677fe700 1 -- 172.21.15.99:0/497134841 learned_addr learned my addr 172.21.15.99:0/497134841 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:33.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.319+0000 7ff2677fe700 1 -- 172.21.15.99:0/497134841 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff2681523e0 msgr2=0x7ff2680b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:33.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.319+0000 7ff266ffd700 1 --2- 172.21.15.99:0/497134841 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff268152010 0x7ff2680c6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:33.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.319+0000 7ff25e7fc700 1 --2- 172.21.15.99:0/497134841 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff2681523e0 0x7ff2680b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:33.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.319+0000 7ff2677fe700 1 --2- 172.21.15.99:0/497134841 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff2681523e0 0x7ff2680b2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:33.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.319+0000 7ff2677fe700 1 -- 172.21.15.99:0/497134841 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff268152010 msgr2=0x7ff2680c6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:33.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.319+0000 7ff2677fe700 1 --2- 172.21.15.99:0/497134841 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff268152010 0x7ff2680c6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:33.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.319+0000 7ff2677fe700 1 -- 172.21.15.99:0/497134841 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff258009420 con 0x7ff268158e50 2023-10-08T00:15:33.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.319+0000 7ff2677fe700 1 --2- 172.21.15.99:0/497134841 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff268158e50 0x7ff26814f790 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7ff26000ccc0 tx=0x7ff260001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:33.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.319+0000 7ff25f7fe700 1 -- 172.21.15.99:0/497134841 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26000ab30 con 0x7ff268158e50 2023-10-08T00:15:33.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.319+0000 7ff25f7fe700 1 -- 172.21.15.99:0/497134841 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff26000ac90 con 0x7ff268158e50 2023-10-08T00:15:33.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.319+0000 7ff26d925700 1 -- 172.21.15.99:0/497134841 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2680b3460 con 0x7ff268158e50 2023-10-08T00:15:33.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.319+0000 7ff25f7fe700 1 -- 172.21.15.99:0/497134841 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff260007340 con 0x7ff268158e50 2023-10-08T00:15:33.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.319+0000 7ff26d925700 1 -- 172.21.15.99:0/497134841 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2680b3a20 con 0x7ff268158e50 2023-10-08T00:15:33.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.320+0000 7ff26d925700 1 -- 172.21.15.99:0/497134841 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff26804c3c0 con 0x7ff268158e50 2023-10-08T00:15:33.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.320+0000 7ff25f7fe700 1 -- 172.21.15.99:0/497134841 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff2600076f0 con 0x7ff268158e50 2023-10-08T00:15:33.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.321+0000 7ff25f7fe700 1 --2- 172.21.15.99:0/497134841 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff248066f30 0x7ff2480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:33.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.321+0000 7ff25f7fe700 1 -- 172.21.15.99:0/497134841 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 4939+0+0 (secure 0 0 0) 0x7ff2600846e0 con 0x7ff268158e50 2023-10-08T00:15:33.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.321+0000 7ff266ffd700 1 --2- 172.21.15.99:0/497134841 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff248066f30 0x7ff2480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:33.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.322+0000 7ff266ffd700 1 --2- 172.21.15.99:0/497134841 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff248066f30 0x7ff2480693e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff2580098e0 tx=0x7ff258017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:33.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.325+0000 7ff25f7fe700 1 -- 172.21.15.99:0/497134841 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff260051cd0 con 0x7ff268158e50 2023-10-08T00:15:33.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:33.588+0000 7ff26d925700 1 -- 172.21.15.99:0/497134841 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]} v 0) v1 -- 0x7ff26806d1b0 con 0x7ff268158e50 2023-10-08T00:15:34.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.187+0000 7ff25f7fe700 1 -- 172.21.15.99:0/497134841 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]}]=0 v241) v1 ==== 140+0+0 (secure 0 0 0) 0x7ff2600552f0 con 0x7ff268158e50 2023-10-08T00:15:34.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.190+0000 7ff26d925700 1 -- 172.21.15.99:0/497134841 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff248066f30 msgr2=0x7ff2480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:34.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.190+0000 7ff26d925700 1 --2- 172.21.15.99:0/497134841 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff248066f30 0x7ff2480693e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff2580098e0 tx=0x7ff258017000 comp rx=0 tx=0).stop 2023-10-08T00:15:34.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.190+0000 7ff26d925700 1 -- 172.21.15.99:0/497134841 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff268158e50 msgr2=0x7ff26814f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:34.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.190+0000 7ff26d925700 1 --2- 172.21.15.99:0/497134841 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff268158e50 0x7ff26814f790 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7ff26000ccc0 tx=0x7ff260001a80 comp rx=0 tx=0).stop 2023-10-08T00:15:34.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.190+0000 7ff26d925700 1 -- 172.21.15.99:0/497134841 shutdown_connections 2023-10-08T00:15:34.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.190+0000 7ff26d925700 1 --2- 172.21.15.99:0/497134841 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff248066f30 0x7ff2480693e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:34.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.190+0000 7ff26d925700 1 --2- 172.21.15.99:0/497134841 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff268152010 0x7ff2680c6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:34.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.190+0000 7ff26d925700 1 --2- 172.21.15.99:0/497134841 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff268158e50 0x7ff26814f790 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:34.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.190+0000 7ff26d925700 1 --2- 172.21.15.99:0/497134841 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff2681523e0 0x7ff2680b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:34.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.190+0000 7ff26d925700 1 -- 172.21.15.99:0/497134841 >> 172.21.15.99:0/497134841 conn(0x7ff268069870 msgr2=0x7ff26815d090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:34.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.190+0000 7ff26d925700 1 -- 172.21.15.99:0/497134841 shutdown_connections 2023-10-08T00:15:34.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.190+0000 7ff26d925700 1 -- 172.21.15.99:0/497134841 wait complete. 2023-10-08T00:15:34.204 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2023-10-08T00:15:34.336 INFO:teuthology.orchestra.run.smithi099.stdout:{"osd_objectstore":"memstore"} 2023-10-08T00:15:34.337 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2023-10-08T00:15:34.337 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T00:15:34.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.454+0000 7fc4808d7700 1 Processor -- start 2023-10-08T00:15:34.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.454+0000 7fc4808d7700 1 -- start start 2023-10-08T00:15:34.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.455+0000 7fc4808d7700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc47c159ec0 0x7fc47c0b0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:34.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.455+0000 7fc4808d7700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc47c0ce7e0 con 0x7fc47c0b06b0 2023-10-08T00:15:34.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.455+0000 7fc4808d7700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc47c0ceac0 con 0x7fc47c0cf5a0 2023-10-08T00:15:34.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.455+0000 7fc4808d7700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc47c0ceda0 con 0x7fc47c159ec0 2023-10-08T00:15:34.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.455+0000 7fc47bfff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc47c159ec0 0x7fc47c0b0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:34.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.455+0000 7fc47b7fe700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fc47c0b06b0 0x7fc47c1597b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45270/0 (socket says 172.21.15.99:45270) 2023-10-08T00:15:34.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.455+0000 7fc47b7fe700 1 -- 172.21.15.99:0/245783463 learned_addr learned my addr 172.21.15.99:0/245783463 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:15:34.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.456+0000 7fc47bfff700 1 -- 172.21.15.99:0/245783463 >> v1:172.21.15.191:6789/0 conn(0x7fc47c0cf5a0 legacy=0x7fc47c067660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:34.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.456+0000 7fc47bfff700 1 -- 172.21.15.99:0/245783463 >> v1:172.21.15.99:6789/0 conn(0x7fc47c0b06b0 legacy=0x7fc47c1597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:34.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.456+0000 7fc47bfff700 1 -- 172.21.15.99:0/245783463 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc47c0cf080 con 0x7fc47c159ec0 2023-10-08T00:15:34.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.456+0000 7fc47a7fc700 1 -- 172.21.15.99:0/245783463 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2106162893 0 0) 0x7fc47c0ce7e0 con 0x7fc47c0b06b0 2023-10-08T00:15:34.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.456+0000 7fc47bfff700 1 --2- 172.21.15.99:0/245783463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc47c159ec0 0x7fc47c0b0a80 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fc4740098e0 tx=0x7fc4740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38dd7261c18e848 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:34.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.456+0000 7fc47a7fc700 1 -- 172.21.15.99:0/245783463 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc474009e80 con 0x7fc47c159ec0 2023-10-08T00:15:34.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.456+0000 7fc47a7fc700 1 -- 172.21.15.99:0/245783463 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc47400b820 con 0x7fc47c159ec0 2023-10-08T00:15:34.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.456+0000 7fc47a7fc700 1 -- 172.21.15.99:0/245783463 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47400bba0 con 0x7fc47c159ec0 2023-10-08T00:15:34.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.457+0000 7fc4808d7700 1 -- 172.21.15.99:0/245783463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc47c159ec0 msgr2=0x7fc47c0b0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:34.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.457+0000 7fc4808d7700 1 --2- 172.21.15.99:0/245783463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc47c159ec0 0x7fc47c0b0a80 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fc4740098e0 tx=0x7fc4740091a0 comp rx=0 tx=0).stop 2023-10-08T00:15:34.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.457+0000 7fc4808d7700 1 -- 172.21.15.99:0/245783463 shutdown_connections 2023-10-08T00:15:34.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.457+0000 7fc4808d7700 1 --2- 172.21.15.99:0/245783463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc47c159ec0 0x7fc47c0b0a80 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:34.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.457+0000 7fc4808d7700 1 -- 172.21.15.99:0/245783463 >> 172.21.15.99:0/245783463 conn(0x7fc47c06c700 msgr2=0x7fc47c069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:34.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.457+0000 7fc4808d7700 1 -- 172.21.15.99:0/245783463 shutdown_connections 2023-10-08T00:15:34.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.457+0000 7fc4808d7700 1 -- 172.21.15.99:0/245783463 wait complete. 2023-10-08T00:15:34.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.458+0000 7fc4808d7700 1 Processor -- start 2023-10-08T00:15:34.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.458+0000 7fc4808d7700 1 -- start start 2023-10-08T00:15:34.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.459+0000 7fc4808d7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc47c0b06b0 0x7fc47c0c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:34.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.459+0000 7fc4808d7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc47c0cf5a0 0x7fc47c0ca380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:34.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.459+0000 7fc4808d7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc47c159ec0 0x7fc47c0cba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:34.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.459+0000 7fc4808d7700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47c0cb390 con 0x7fc47c159ec0 2023-10-08T00:15:34.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.459+0000 7fc4808d7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47c0cb600 con 0x7fc47c0cf5a0 2023-10-08T00:15:34.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.459+0000 7fc4808d7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc47c0ca960 con 0x7fc47c0b06b0 2023-10-08T00:15:34.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.459+0000 7fc47bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc47c159ec0 0x7fc47c0cba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:34.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.459+0000 7fc47bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc47c159ec0 0x7fc47c0cba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51140/0 (socket says 172.21.15.99:51140) 2023-10-08T00:15:34.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.459+0000 7fc47bfff700 1 -- 172.21.15.99:0/2087997109 learned_addr learned my addr 172.21.15.99:0/2087997109 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:34.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.459+0000 7fc47bfff700 1 -- 172.21.15.99:0/2087997109 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc47c0b06b0 msgr2=0x7fc47c0c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:34.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.459+0000 7fc47bfff700 1 --2- 172.21.15.99:0/2087997109 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc47c0b06b0 0x7fc47c0c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:34.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.459+0000 7fc47bfff700 1 -- 172.21.15.99:0/2087997109 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc47c0cf5a0 msgr2=0x7fc47c0ca380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:34.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.459+0000 7fc47bfff700 1 --2- 172.21.15.99:0/2087997109 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc47c0cf5a0 0x7fc47c0ca380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:34.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.459+0000 7fc47bfff700 1 -- 172.21.15.99:0/2087997109 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc474009420 con 0x7fc47c159ec0 2023-10-08T00:15:34.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.459+0000 7fc47bfff700 1 --2- 172.21.15.99:0/2087997109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc47c159ec0 0x7fc47c0cba90 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fc4740093f0 tx=0x7fc474016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:34.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.459+0000 7fc4637fe700 1 -- 172.21.15.99:0/2087997109 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47401b450 con 0x7fc47c159ec0 2023-10-08T00:15:34.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.459+0000 7fc4637fe700 1 -- 172.21.15.99:0/2087997109 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc474020070 con 0x7fc47c159ec0 2023-10-08T00:15:34.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.459+0000 7fc4808d7700 1 -- 172.21.15.99:0/2087997109 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc47c0cac40 con 0x7fc47c159ec0 2023-10-08T00:15:34.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.459+0000 7fc4637fe700 1 -- 172.21.15.99:0/2087997109 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47401b820 con 0x7fc47c159ec0 2023-10-08T00:15:34.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.459+0000 7fc4808d7700 1 -- 172.21.15.99:0/2087997109 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc47c0cdec0 con 0x7fc47c159ec0 2023-10-08T00:15:34.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.460+0000 7fc4808d7700 1 -- 172.21.15.99:0/2087997109 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc47c04c3c0 con 0x7fc47c159ec0 2023-10-08T00:15:34.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.461+0000 7fc4637fe700 1 -- 172.21.15.99:0/2087997109 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc474022030 con 0x7fc47c159ec0 2023-10-08T00:15:34.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.461+0000 7fc4637fe700 1 --2- 172.21.15.99:0/2087997109 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc468066f30 0x7fc4680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:34.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.461+0000 7fc4637fe700 1 -- 172.21.15.99:0/2087997109 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fc474086420 con 0x7fc47c159ec0 2023-10-08T00:15:34.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.461+0000 7fc47b7fe700 1 --2- 172.21.15.99:0/2087997109 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc468066f30 0x7fc4680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:34.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.462+0000 7fc47b7fe700 1 --2- 172.21.15.99:0/2087997109 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc468066f30 0x7fc4680693e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc46c001560 tx=0x7fc46c0016c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:34.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.465+0000 7fc4637fe700 1 -- 172.21.15.99:0/2087997109 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc474053990 con 0x7fc47c159ec0 2023-10-08T00:15:34.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.649+0000 7fc4808d7700 1 -- 172.21.15.99:0/2087997109 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fc47c0cb740 con 0x7fc47c159ec0 2023-10-08T00:15:34.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.652+0000 7fc4637fe700 1 -- 172.21.15.99:0/2087997109 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc47c0cb740 con 0x7fc47c159ec0 2023-10-08T00:15:34.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.654+0000 7fc4808d7700 1 -- 172.21.15.99:0/2087997109 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc468066f30 msgr2=0x7fc4680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:34.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.654+0000 7fc4808d7700 1 --2- 172.21.15.99:0/2087997109 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc468066f30 0x7fc4680693e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc46c001560 tx=0x7fc46c0016c0 comp rx=0 tx=0).stop 2023-10-08T00:15:34.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.654+0000 7fc4808d7700 1 -- 172.21.15.99:0/2087997109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc47c159ec0 msgr2=0x7fc47c0cba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:34.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.654+0000 7fc4808d7700 1 --2- 172.21.15.99:0/2087997109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc47c159ec0 0x7fc47c0cba90 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fc4740093f0 tx=0x7fc474016000 comp rx=0 tx=0).stop 2023-10-08T00:15:34.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.654+0000 7fc4808d7700 1 -- 172.21.15.99:0/2087997109 shutdown_connections 2023-10-08T00:15:34.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.654+0000 7fc4808d7700 1 --2- 172.21.15.99:0/2087997109 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc468066f30 0x7fc4680693e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:34.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.654+0000 7fc4808d7700 1 --2- 172.21.15.99:0/2087997109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc47c159ec0 0x7fc47c0cba90 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:34.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.654+0000 7fc4808d7700 1 --2- 172.21.15.99:0/2087997109 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc47c0b06b0 0x7fc47c0c9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:34.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.654+0000 7fc4808d7700 1 --2- 172.21.15.99:0/2087997109 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc47c0cf5a0 0x7fc47c0ca380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:34.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.654+0000 7fc4808d7700 1 -- 172.21.15.99:0/2087997109 >> 172.21.15.99:0/2087997109 conn(0x7fc47c06c700 msgr2=0x7fc47c155e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:34.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.654+0000 7fc4808d7700 1 -- 172.21.15.99:0/2087997109 shutdown_connections 2023-10-08T00:15:34.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.654+0000 7fc4808d7700 1 -- 172.21.15.99:0/2087997109 wait complete. 2023-10-08T00:15:34.668 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T00:15:34.669 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T00:15:34.708 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T00:15:34.758 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T00:15:34.759 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-08T00:15:34.759 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:15:34.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.872+0000 7fc494a36700 1 Processor -- start 2023-10-08T00:15:34.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.872+0000 7fc494a36700 1 -- start start 2023-10-08T00:15:34.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.872+0000 7fc494a36700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4900cdab0 0x7fc49015d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:34.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.872+0000 7fc494a36700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4900cb7c0 con 0x7fc49015dda0 2023-10-08T00:15:34.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.872+0000 7fc494a36700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc49015e290 con 0x7fc4900cd6e0 2023-10-08T00:15:34.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.872+0000 7fc494a36700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc49015e850 con 0x7fc4900cdab0 2023-10-08T00:15:34.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.873+0000 7fc48f59e700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fc49015dda0 0x7fc49015cbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45278/0 (socket says 172.21.15.99:45278) 2023-10-08T00:15:34.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.873+0000 7fc48f59e700 1 -- 172.21.15.99:0/2640256771 learned_addr learned my addr 172.21.15.99:0/2640256771 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:15:34.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.873+0000 7fc48e59c700 1 --2- 172.21.15.99:0/2640256771 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4900cdab0 0x7fc49015d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:34.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.873+0000 7fc48dd9b700 1 -- 172.21.15.99:0/2640256771 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1073935346 0 0) 0x7fc4900cb7c0 con 0x7fc49015dda0 2023-10-08T00:15:34.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.873+0000 7fc48dd9b700 1 -- 172.21.15.99:0/2640256771 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc47c003570 con 0x7fc49015dda0 2023-10-08T00:15:34.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.873+0000 7fc48dd9b700 1 -- 172.21.15.99:0/2640256771 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2981129771 0 0) 0x7fc47c003570 con 0x7fc49015dda0 2023-10-08T00:15:34.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.873+0000 7fc48dd9b700 1 -- 172.21.15.99:0/2640256771 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4900cdab0 msgr2=0x7fc49015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:34.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.873+0000 7fc48dd9b700 1 --2- 172.21.15.99:0/2640256771 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4900cdab0 0x7fc49015d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:34.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.873+0000 7fc48dd9b700 1 -- 172.21.15.99:0/2640256771 >> v1:172.21.15.191:6789/0 conn(0x7fc4900cd6e0 legacy=0x7fc4900cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:34.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.873+0000 7fc48dd9b700 1 -- 172.21.15.99:0/2640256771 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc49015e570 con 0x7fc49015dda0 2023-10-08T00:15:34.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.873+0000 7fc48dd9b700 1 -- 172.21.15.99:0/2640256771 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7fc480001470 con 0x7fc49015dda0 2023-10-08T00:15:34.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.873+0000 7fc48dd9b700 1 -- 172.21.15.99:0/2640256771 >> v1:172.21.15.99:6789/0 conn(0x7fc49015dda0 legacy=0x7fc49015cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:34.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.873+0000 7fc48dd9b700 1 --2- 172.21.15.99:0/2640256771 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc47c003a90 0x7fc47c003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:34.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.873+0000 7fc48dd9b700 1 --2- 172.21.15.99:0/2640256771 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4900cdab0 0x7fc47c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:34.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.873+0000 7fc48dd9b700 1 --2- 172.21.15.99:0/2640256771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc47c004d50 0x7fc47c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:34.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.873+0000 7fc48dd9b700 1 -- 172.21.15.99:0/2640256771 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc49015e850 con 0x7fc47c004d50 2023-10-08T00:15:34.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.873+0000 7fc48dd9b700 1 -- 172.21.15.99:0/2640256771 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47c00bb40 con 0x7fc47c003a90 2023-10-08T00:15:34.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.873+0000 7fc48dd9b700 1 -- 172.21.15.99:0/2640256771 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc47c00bd00 con 0x7fc4900cdab0 2023-10-08T00:15:34.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.873+0000 7fc48f59e700 1 --2- 172.21.15.99:0/2640256771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc47c004d50 0x7fc47c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:34.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.873+0000 7fc48ed9d700 1 --2- 172.21.15.99:0/2640256771 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc47c003a90 0x7fc47c003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:34.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.873+0000 7fc48e59c700 1 --2- 172.21.15.99:0/2640256771 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4900cdab0 0x7fc47c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:34.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.874+0000 7fc48f59e700 1 -- 172.21.15.99:0/2640256771 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4900cdab0 msgr2=0x7fc47c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:34.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.874+0000 7fc48f59e700 1 --2- 172.21.15.99:0/2640256771 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4900cdab0 0x7fc47c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:34.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.874+0000 7fc48f59e700 1 -- 172.21.15.99:0/2640256771 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc47c003a90 msgr2=0x7fc47c003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:34.880 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.874+0000 7fc48f59e700 1 --2- 172.21.15.99:0/2640256771 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc47c003a90 0x7fc47c003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:34.880 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.874+0000 7fc48f59e700 1 -- 172.21.15.99:0/2640256771 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc47c003570 con 0x7fc47c004d50 2023-10-08T00:15:34.880 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.874+0000 7fc48f59e700 1 --2- 172.21.15.99:0/2640256771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc47c004d50 0x7fc47c0051c0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fc48000c3c0 tx=0x7fc480001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92ef9263912b9d86 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:34.880 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.874+0000 7fc48dd9b700 1 -- 172.21.15.99:0/2640256771 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48000cc90 con 0x7fc47c004d50 2023-10-08T00:15:34.880 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.874+0000 7fc48dd9b700 1 -- 172.21.15.99:0/2640256771 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc48000cdf0 con 0x7fc47c004d50 2023-10-08T00:15:34.880 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.874+0000 7fc48dd9b700 1 -- 172.21.15.99:0/2640256771 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc480005b70 con 0x7fc47c004d50 2023-10-08T00:15:34.880 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.874+0000 7fc494a36700 1 -- 172.21.15.99:0/2640256771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc47c004d50 msgr2=0x7fc47c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:34.880 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.874+0000 7fc494a36700 1 --2- 172.21.15.99:0/2640256771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc47c004d50 0x7fc47c0051c0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fc48000c3c0 tx=0x7fc480001e70 comp rx=0 tx=0).stop 2023-10-08T00:15:34.880 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.874+0000 7fc48f59e700 1 -- 172.21.15.99:0/2640256771 reap_dead start 2023-10-08T00:15:34.880 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.874+0000 7fc494a36700 1 -- 172.21.15.99:0/2640256771 shutdown_connections 2023-10-08T00:15:34.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.874+0000 7fc494a36700 1 -- 172.21.15.99:0/2640256771 >> 172.21.15.99:0/2640256771 conn(0x7fc490067660 msgr2=0x7fc490067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:34.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.874+0000 7fc494a36700 1 -- 172.21.15.99:0/2640256771 shutdown_connections 2023-10-08T00:15:34.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.874+0000 7fc494a36700 1 -- 172.21.15.99:0/2640256771 wait complete. 2023-10-08T00:15:34.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.876+0000 7fc494a36700 1 Processor -- start 2023-10-08T00:15:34.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.876+0000 7fc494a36700 1 -- start start 2023-10-08T00:15:34.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.876+0000 7fc494a36700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc47c004d50 0x7fc49014c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:34.884 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.876+0000 7fc494a36700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc490150b30 0x7fc4900b24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:34.884 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.876+0000 7fc494a36700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc490150fa0 0x7fc4900b2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:34.884 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.876+0000 7fc494a36700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4900cc360 con 0x7fc490150b30 2023-10-08T00:15:34.884 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.876+0000 7fc494a36700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4900b4780 con 0x7fc47c004d50 2023-10-08T00:15:34.884 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.876+0000 7fc494a36700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc49014e340 con 0x7fc490150fa0 2023-10-08T00:15:34.884 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.876+0000 7fc48e59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc490150b30 0x7fc4900b24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:34.884 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.877+0000 7fc48e59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc490150b30 0x7fc4900b24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51158/0 (socket says 172.21.15.99:51158) 2023-10-08T00:15:34.884 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.877+0000 7fc48e59c700 1 -- 172.21.15.99:0/3329885036 learned_addr learned my addr 172.21.15.99:0/3329885036 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:34.884 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.877+0000 7fc48f59e700 1 --2- 172.21.15.99:0/3329885036 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc490150fa0 0x7fc4900b2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:34.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.877+0000 7fc48ed9d700 1 --2- 172.21.15.99:0/3329885036 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc47c004d50 0x7fc49014c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:34.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.877+0000 7fc48e59c700 1 -- 172.21.15.99:0/3329885036 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc490150fa0 msgr2=0x7fc4900b2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:34.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.877+0000 7fc48e59c700 1 --2- 172.21.15.99:0/3329885036 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc490150fa0 0x7fc4900b2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:34.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.877+0000 7fc48e59c700 1 -- 172.21.15.99:0/3329885036 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc47c004d50 msgr2=0x7fc49014c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:34.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.877+0000 7fc48e59c700 1 --2- 172.21.15.99:0/3329885036 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc47c004d50 0x7fc49014c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:34.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.877+0000 7fc48e59c700 1 -- 172.21.15.99:0/3329885036 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4900b3bb0 con 0x7fc490150b30 2023-10-08T00:15:34.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.877+0000 7fc48e59c700 1 --2- 172.21.15.99:0/3329885036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc490150b30 0x7fc4900b24f0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fc478011b10 tx=0x7fc4780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:34.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.877+0000 7fc486ffd700 1 -- 172.21.15.99:0/3329885036 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47800f9f0 con 0x7fc490150b30 2023-10-08T00:15:34.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.877+0000 7fc486ffd700 1 -- 172.21.15.99:0/3329885036 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc47800fb50 con 0x7fc490150b30 2023-10-08T00:15:34.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.877+0000 7fc486ffd700 1 -- 172.21.15.99:0/3329885036 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47800fed0 con 0x7fc490150b30 2023-10-08T00:15:34.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.877+0000 7fc494a36700 1 -- 172.21.15.99:0/3329885036 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4900b3e50 con 0x7fc490150b30 2023-10-08T00:15:34.886 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.877+0000 7fc494a36700 1 -- 172.21.15.99:0/3329885036 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4900b4410 con 0x7fc490150b30 2023-10-08T00:15:34.886 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.878+0000 7fc494a36700 1 -- 172.21.15.99:0/3329885036 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4900b4930 con 0x7fc490150b30 2023-10-08T00:15:34.886 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.878+0000 7fc486ffd700 1 -- 172.21.15.99:0/3329885036 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc478014650 con 0x7fc490150b30 2023-10-08T00:15:34.886 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.879+0000 7fc486ffd700 1 --2- 172.21.15.99:0/3329885036 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc470066f30 0x7fc4700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:34.886 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.879+0000 7fc486ffd700 1 -- 172.21.15.99:0/3329885036 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fc478018030 con 0x7fc490150b30 2023-10-08T00:15:34.886 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.879+0000 7fc48ed9d700 1 --2- 172.21.15.99:0/3329885036 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc470066f30 0x7fc4700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:34.886 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.879+0000 7fc48ed9d700 1 --2- 172.21.15.99:0/3329885036 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc470066f30 0x7fc4700693e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc488004290 tx=0x7fc4880016d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:34.886 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:34.883+0000 7fc486ffd700 1 -- 172.21.15.99:0/3329885036 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc4900b4930 con 0x7fc490150b30 2023-10-08T00:15:35.105 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.104+0000 7fc494a36700 1 -- 172.21.15.99:0/3329885036 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc4900b48f0 con 0x7fc490150b30 2023-10-08T00:15:35.105 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.105+0000 7fc486ffd700 1 -- 172.21.15.99:0/3329885036 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+3529 (secure 0 0 0) 0x7fc478022de0 con 0x7fc490150b30 2023-10-08T00:15:35.105 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:15:35.106 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":42,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5650,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/2166055423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2166055423},{"type":"v1","addr":"172.21.15.99:6834","nonce":2166055423}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5665,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/4087832300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":4087832300},{"type":"v1","addr":"172.21.15.191:6836","nonce":4087832300}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5668,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/730261712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":730261712},{"type":"v1","addr":"172.21.15.191:6837","nonce":730261712}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":6297,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/290014273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":290014273},{"type":"v1","addr":"172.21.15.99:6835","nonce":290014273}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[]} 2023-10-08T00:15:35.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.107+0000 7fc494a36700 1 -- 172.21.15.99:0/3329885036 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc470066f30 msgr2=0x7fc4700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:35.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.107+0000 7fc494a36700 1 --2- 172.21.15.99:0/3329885036 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc470066f30 0x7fc4700693e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc488004290 tx=0x7fc4880016d0 comp rx=0 tx=0).stop 2023-10-08T00:15:35.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.107+0000 7fc494a36700 1 -- 172.21.15.99:0/3329885036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc490150b30 msgr2=0x7fc4900b24f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:35.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.107+0000 7fc494a36700 1 --2- 172.21.15.99:0/3329885036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc490150b30 0x7fc4900b24f0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fc478011b10 tx=0x7fc4780091a0 comp rx=0 tx=0).stop 2023-10-08T00:15:35.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.107+0000 7fc494a36700 1 -- 172.21.15.99:0/3329885036 shutdown_connections 2023-10-08T00:15:35.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.107+0000 7fc494a36700 1 --2- 172.21.15.99:0/3329885036 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc470066f30 0x7fc4700693e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:35.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.107+0000 7fc494a36700 1 --2- 172.21.15.99:0/3329885036 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc47c004d50 0x7fc49014c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:35.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.107+0000 7fc494a36700 1 --2- 172.21.15.99:0/3329885036 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc490150fa0 0x7fc4900b2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:35.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.107+0000 7fc494a36700 1 --2- 172.21.15.99:0/3329885036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc490150b30 0x7fc4900b24f0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:35.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.107+0000 7fc494a36700 1 -- 172.21.15.99:0/3329885036 >> 172.21.15.99:0/3329885036 conn(0x7fc490067660 msgr2=0x7fc49006aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:35.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.107+0000 7fc494a36700 1 -- 172.21.15.99:0/3329885036 shutdown_connections 2023-10-08T00:15:35.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.107+0000 7fc494a36700 1 -- 172.21.15.99:0/3329885036 wait complete. 2023-10-08T00:15:35.109 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 42 2023-10-08T00:15:35.120 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T00:15:35.120 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-08T00:15:35.120 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T00:15:35.120 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:15:35.121 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T00:15:35.121 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-08T00:15:35.121 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T00:15:35.121 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:15:35.121 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T00:15:35.121 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-08T00:15:35.122 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T00:15:35.122 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:15:35.122 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T00:15:35.122 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-08T00:15:35.122 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T00:15:35.122 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:15:35.130 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T00:15:35.130 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T00:15:35.131 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T00:15:35.131 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T00:15:35.135 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T00:15:35.135 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T00:15:35.136 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T00:15:35.136 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T00:15:35.173 INFO:tasks.ceph.mds.a:Started 2023-10-08T00:15:35.175 DEBUG:teuthology.parallel:result is None 2023-10-08T00:15:35.176 INFO:tasks.ceph.mds.c:Started 2023-10-08T00:15:35.176 DEBUG:teuthology.parallel:result is None 2023-10-08T00:15:35.178 INFO:tasks.ceph.mds.d:Started 2023-10-08T00:15:35.178 DEBUG:teuthology.parallel:result is None 2023-10-08T00:15:35.179 INFO:tasks.ceph.mds.b:Started 2023-10-08T00:15:35.179 DEBUG:teuthology.parallel:result is None 2023-10-08T00:15:35.179 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-08T00:15:35.246 INFO:tasks.ceph.mds.a.smithi099.stdout:starting mds.a at 2023-10-08T00:15:35.246 INFO:tasks.ceph.mds.c.smithi099.stdout:starting mds.c at 2023-10-08T00:15:35.247 INFO:tasks.ceph.mds.a.smithi099.stderr:2023-10-08T00:15:35.246+0000 7f77c23eb700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2023-10-08T00:15:35.260 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2023-10-08T00:15:35.260 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2023-10-08T00:15:35.261 INFO:tasks.ceph.mds.b.smithi191.stderr:2023-10-08T00:15:35.260+0000 7f8cd4cb2700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-10-08T00:15:35.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.299+0000 7f6522895700 1 Processor -- start 2023-10-08T00:15:35.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.299+0000 7f6522895700 1 -- start start 2023-10-08T00:15:35.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.300+0000 7f6522895700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f651c0afa70 0x7f651c0cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:35.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.300+0000 7f6522895700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f651c0cb810 con 0x7f651c0d18d0 2023-10-08T00:15:35.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.300+0000 7f6522895700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f651c0b0370 con 0x7f651c0afe40 2023-10-08T00:15:35.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.300+0000 7f6522895700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f651c0b0f80 con 0x7f651c0afa70 2023-10-08T00:15:35.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.300+0000 7f651bfff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f651c0afa70 0x7f651c0cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:35.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.300+0000 7f6520e32700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f651c0d18d0 0x7f651c068cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45302/0 (socket says 172.21.15.99:45302) 2023-10-08T00:15:35.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.300+0000 7f6520e32700 1 -- 172.21.15.99:0/3907251121 learned_addr learned my addr 172.21.15.99:0/3907251121 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:15:35.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.301+0000 7f651bfff700 1 -- 172.21.15.99:0/3907251121 >> v1:172.21.15.191:6789/0 conn(0x7f651c0afe40 legacy=0x7f651c0b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:35.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.301+0000 7f651bfff700 1 -- 172.21.15.99:0/3907251121 >> v1:172.21.15.99:6789/0 conn(0x7f651c0d18d0 legacy=0x7f651c068cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:35.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.301+0000 7f651bfff700 1 -- 172.21.15.99:0/3907251121 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f651c0b0650 con 0x7f651c0afa70 2023-10-08T00:15:35.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.301+0000 7f651bfff700 1 --2- 172.21.15.99:0/3907251121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f651c0afa70 0x7f651c0cb2d0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f650c0098e0 tx=0x7f650c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81d129c38ed7f11 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:35.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.301+0000 7f651affd700 1 -- 172.21.15.99:0/3907251121 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650c009e80 con 0x7f651c0afa70 2023-10-08T00:15:35.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.301+0000 7f651affd700 1 -- 172.21.15.99:0/3907251121 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f650c00b860 con 0x7f651c0afa70 2023-10-08T00:15:35.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.301+0000 7f651affd700 1 -- 172.21.15.99:0/3907251121 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650c00bbe0 con 0x7f651c0afa70 2023-10-08T00:15:35.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.301+0000 7f6522895700 1 -- 172.21.15.99:0/3907251121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f651c0afa70 msgr2=0x7f651c0cb2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:35.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.301+0000 7f6522895700 1 --2- 172.21.15.99:0/3907251121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f651c0afa70 0x7f651c0cb2d0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f650c0098e0 tx=0x7f650c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:15:35.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.301+0000 7f6522895700 1 -- 172.21.15.99:0/3907251121 shutdown_connections 2023-10-08T00:15:35.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.301+0000 7f6522895700 1 --2- 172.21.15.99:0/3907251121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f651c0afa70 0x7f651c0cb2d0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:35.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.301+0000 7f6522895700 1 -- 172.21.15.99:0/3907251121 >> 172.21.15.99:0/3907251121 conn(0x7f651c068130 msgr2=0x7f651c0d0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:35.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.301+0000 7f6522895700 1 -- 172.21.15.99:0/3907251121 shutdown_connections 2023-10-08T00:15:35.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.302+0000 7f6522895700 1 -- 172.21.15.99:0/3907251121 wait complete. 2023-10-08T00:15:35.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.303+0000 7f6522895700 1 Processor -- start 2023-10-08T00:15:35.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.303+0000 7f6522895700 1 -- start start 2023-10-08T00:15:35.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.303+0000 7f6522895700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f651c0afa70 0x7f651c0ca790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:35.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.303+0000 7f6522895700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f651c0d18d0 0x7f651c15a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:35.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.303+0000 7f6522895700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f651c15c0e0 0x7f651c152170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:35.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.303+0000 7f6522895700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651c15ada0 con 0x7f651c0d18d0 2023-10-08T00:15:35.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.303+0000 7f6522895700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651c15b040 con 0x7f651c15c0e0 2023-10-08T00:15:35.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.303+0000 7f6522895700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f651c15b320 con 0x7f651c0afa70 2023-10-08T00:15:35.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.304+0000 7f651b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f651c0d18d0 0x7f651c15a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:35.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.304+0000 7f651b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f651c0d18d0 0x7f651c15a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51196/0 (socket says 172.21.15.99:51196) 2023-10-08T00:15:35.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.304+0000 7f651b7fe700 1 -- 172.21.15.99:0/1961283487 learned_addr learned my addr 172.21.15.99:0/1961283487 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:35.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.304+0000 7f6520e32700 1 --2- 172.21.15.99:0/1961283487 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f651c15c0e0 0x7f651c152170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:35.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.304+0000 7f651bfff700 1 --2- 172.21.15.99:0/1961283487 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f651c0afa70 0x7f651c0ca790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:35.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.304+0000 7f651b7fe700 1 -- 172.21.15.99:0/1961283487 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f651c0afa70 msgr2=0x7f651c0ca790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:35.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.304+0000 7f651b7fe700 1 --2- 172.21.15.99:0/1961283487 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f651c0afa70 0x7f651c0ca790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:35.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.304+0000 7f651b7fe700 1 -- 172.21.15.99:0/1961283487 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f651c15c0e0 msgr2=0x7f651c152170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:35.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.304+0000 7f651b7fe700 1 --2- 172.21.15.99:0/1961283487 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f651c15c0e0 0x7f651c152170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:35.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.304+0000 7f651b7fe700 1 -- 172.21.15.99:0/1961283487 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f650c009420 con 0x7f651c0d18d0 2023-10-08T00:15:35.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.304+0000 7f651bfff700 1 --2- 172.21.15.99:0/1961283487 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f651c0afa70 0x7f651c0ca790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:15:35.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.304+0000 7f6520e32700 1 --2- 172.21.15.99:0/1961283487 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f651c15c0e0 0x7f651c152170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:15:35.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.304+0000 7f651b7fe700 1 --2- 172.21.15.99:0/1961283487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f651c0d18d0 0x7f651c15a860 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f651000c790 tx=0x7f6510007f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:35.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.304+0000 7f6503fff700 1 -- 172.21.15.99:0/1961283487 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6510011590 con 0x7f651c0d18d0 2023-10-08T00:15:35.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.304+0000 7f6503fff700 1 -- 172.21.15.99:0/1961283487 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6510011710 con 0x7f651c0d18d0 2023-10-08T00:15:35.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.304+0000 7f6503fff700 1 -- 172.21.15.99:0/1961283487 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6510011ad0 con 0x7f651c0d18d0 2023-10-08T00:15:35.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.304+0000 7f6522895700 1 -- 172.21.15.99:0/1961283487 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f651c15b600 con 0x7f651c0d18d0 2023-10-08T00:15:35.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.304+0000 7f6522895700 1 -- 172.21.15.99:0/1961283487 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f651c15bbc0 con 0x7f651c0d18d0 2023-10-08T00:15:35.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.305+0000 7f6522895700 1 -- 172.21.15.99:0/1961283487 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f651c15bde0 con 0x7f651c0d18d0 2023-10-08T00:15:35.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.305+0000 7f6503fff700 1 -- 172.21.15.99:0/1961283487 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f651001a360 con 0x7f651c0d18d0 2023-10-08T00:15:35.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.306+0000 7f6503fff700 1 --2- 172.21.15.99:0/1961283487 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6508066f30 0x7f65080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:35.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.306+0000 7f6503fff700 1 -- 172.21.15.99:0/1961283487 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f65100831b0 con 0x7f651c0d18d0 2023-10-08T00:15:35.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.306+0000 7f651bfff700 1 --2- 172.21.15.99:0/1961283487 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6508066f30 0x7f65080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:35.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.306+0000 7f651bfff700 1 --2- 172.21.15.99:0/1961283487 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6508066f30 0x7f65080693e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f650c003080 tx=0x7f650c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:35.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.310+0000 7f6503fff700 1 -- 172.21.15.99:0/1961283487 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6510050720 con 0x7f651c0d18d0 2023-10-08T00:15:35.508 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:35.508+0000 7f6522895700 1 -- 172.21.15.99:0/1961283487 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f651c04c3c0 con 0x7f651c0d18d0 2023-10-08T00:15:37.121 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.121+0000 7f6503fff700 1 -- 172.21.15.99:0/1961283487 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v100) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6510053d40 con 0x7f651c0d18d0 2023-10-08T00:15:37.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.123+0000 7f6522895700 1 -- 172.21.15.99:0/1961283487 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6508066f30 msgr2=0x7f65080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:37.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.123+0000 7f6522895700 1 --2- 172.21.15.99:0/1961283487 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6508066f30 0x7f65080693e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f650c003080 tx=0x7f650c017000 comp rx=0 tx=0).stop 2023-10-08T00:15:37.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.123+0000 7f6522895700 1 -- 172.21.15.99:0/1961283487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f651c0d18d0 msgr2=0x7f651c15a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:37.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.123+0000 7f6522895700 1 --2- 172.21.15.99:0/1961283487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f651c0d18d0 0x7f651c15a860 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f651000c790 tx=0x7f6510007f20 comp rx=0 tx=0).stop 2023-10-08T00:15:37.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.123+0000 7f6522895700 1 -- 172.21.15.99:0/1961283487 shutdown_connections 2023-10-08T00:15:37.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.123+0000 7f6522895700 1 --2- 172.21.15.99:0/1961283487 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6508066f30 0x7f65080693e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:37.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.123+0000 7f6522895700 1 --2- 172.21.15.99:0/1961283487 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f651c15c0e0 0x7f651c152170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:37.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.123+0000 7f6522895700 1 --2- 172.21.15.99:0/1961283487 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f651c0afa70 0x7f651c0ca790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:37.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.123+0000 7f6522895700 1 --2- 172.21.15.99:0/1961283487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f651c0d18d0 0x7f651c15a860 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:37.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.123+0000 7f6522895700 1 -- 172.21.15.99:0/1961283487 >> 172.21.15.99:0/1961283487 conn(0x7f651c068130 msgr2=0x7f651c158150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:37.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.123+0000 7f6522895700 1 -- 172.21.15.99:0/1961283487 shutdown_connections 2023-10-08T00:15:37.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.123+0000 7f6522895700 1 -- 172.21.15.99:0/1961283487 wait complete. 2023-10-08T00:15:37.126 INFO:teuthology.orchestra.run.smithi099.stderr: removed all blocklist entries 2023-10-08T00:15:37.138 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-08T00:15:37.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.256+0000 7fcaffb9b700 1 Processor -- start 2023-10-08T00:15:37.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.257+0000 7fcaffb9b700 1 -- start start 2023-10-08T00:15:37.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.257+0000 7fcaffb9b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fcaf8152010 0x7fcaf815b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:37.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.257+0000 7fcaffb9b700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fcaf81523e0 0x7fcaf806c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:37.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.257+0000 7fcaffb9b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcaf8158e50 0x7fcaf80cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:37.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.257+0000 7fcaffb9b700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaf806c010 con 0x7fcaf81523e0 2023-10-08T00:15:37.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.257+0000 7fcaffb9b700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaf806bd10 con 0x7fcaf8152010 2023-10-08T00:15:37.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.257+0000 7fcaffb9b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf815c060 con 0x7fcaf8158e50 2023-10-08T00:15:37.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.257+0000 7fcafd136700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fcaf81523e0 0x7fcaf806c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:37.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.258+0000 7fcafd136700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fcaf81523e0 0x7fcaf806c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51204/0 (socket says 172.21.15.99:51204) 2023-10-08T00:15:37.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.258+0000 7fcafd136700 1 -- 172.21.15.99:0/2129908479 learned_addr learned my addr 172.21.15.99:0/2129908479 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:37.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.258+0000 7fcafd937700 1 --2- 172.21.15.99:0/2129908479 >> v2:172.21.15.191:3300/0 conn(0x7fcaf8152010 0x7fcaf815b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:37.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.258+0000 7fcafd136700 1 -- 172.21.15.99:0/2129908479 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcaf8158e50 msgr2=0x7fcaf80cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:37.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.258+0000 7fcafd136700 1 --2- 172.21.15.99:0/2129908479 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcaf8158e50 0x7fcaf80cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:37.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.258+0000 7fcafd136700 1 -- 172.21.15.99:0/2129908479 >> v2:172.21.15.191:3300/0 conn(0x7fcaf8152010 msgr2=0x7fcaf815b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:37.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.258+0000 7fcafd136700 1 --2- 172.21.15.99:0/2129908479 >> v2:172.21.15.191:3300/0 conn(0x7fcaf8152010 0x7fcaf815b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:37.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.258+0000 7fcafd136700 1 -- 172.21.15.99:0/2129908479 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf815c340 con 0x7fcaf81523e0 2023-10-08T00:15:37.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.258+0000 7fcafd136700 1 --2- 172.21.15.99:0/2129908479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcaf81523e0 0x7fcaf806c4a0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fcaf80cc680 tx=0x7fcae8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92ea12b63f3f2e7f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:37.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.258+0000 7fcafc935700 1 -- 172.21.15.99:0/2129908479 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae800b860 con 0x7fcaf81523e0 2023-10-08T00:15:37.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.258+0000 7fcafc935700 1 -- 172.21.15.99:0/2129908479 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcae800b9c0 con 0x7fcaf81523e0 2023-10-08T00:15:37.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.258+0000 7fcafc935700 1 -- 172.21.15.99:0/2129908479 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae800bd60 con 0x7fcaf81523e0 2023-10-08T00:15:37.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.258+0000 7fcaffb9b700 1 -- 172.21.15.99:0/2129908479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcaf81523e0 msgr2=0x7fcaf806c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:37.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.258+0000 7fcaffb9b700 1 --2- 172.21.15.99:0/2129908479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcaf81523e0 0x7fcaf806c4a0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fcaf80cc680 tx=0x7fcae8009250 comp rx=0 tx=0).stop 2023-10-08T00:15:37.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.259+0000 7fcaffb9b700 1 -- 172.21.15.99:0/2129908479 shutdown_connections 2023-10-08T00:15:37.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.259+0000 7fcaffb9b700 1 --2- 172.21.15.99:0/2129908479 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcaf8158e50 0x7fcaf80cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:37.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.259+0000 7fcaffb9b700 1 --2- 172.21.15.99:0/2129908479 >> v2:172.21.15.191:3300/0 conn(0x7fcaf8152010 0x7fcaf815b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:37.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.259+0000 7fcaffb9b700 1 --2- 172.21.15.99:0/2129908479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcaf81523e0 0x7fcaf806c4a0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:37.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.259+0000 7fcaffb9b700 1 -- 172.21.15.99:0/2129908479 >> 172.21.15.99:0/2129908479 conn(0x7fcaf8069870 msgr2=0x7fcaf8069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:37.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.259+0000 7fcaffb9b700 1 -- 172.21.15.99:0/2129908479 shutdown_connections 2023-10-08T00:15:37.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.259+0000 7fcaffb9b700 1 -- 172.21.15.99:0/2129908479 wait complete. 2023-10-08T00:15:37.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.260+0000 7fcaffb9b700 1 Processor -- start 2023-10-08T00:15:37.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.260+0000 7fcaffb9b700 1 -- start start 2023-10-08T00:15:37.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.260+0000 7fcaffb9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcaf8152010 0x7fcaf80b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:37.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.260+0000 7fcaffb9b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcaf81523e0 0x7fcaf8169fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:37.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.260+0000 7fcaffb9b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcaf8158e50 0x7fcaf8156c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:37.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.260+0000 7fcaffb9b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf80679d0 con 0x7fcaf8152010 2023-10-08T00:15:37.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.260+0000 7fcaffb9b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf81514b0 con 0x7fcaf8158e50 2023-10-08T00:15:37.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.260+0000 7fcaffb9b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf8151790 con 0x7fcaf81523e0 2023-10-08T00:15:37.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.261+0000 7fcafd937700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcaf8152010 0x7fcaf80b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:37.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.261+0000 7fcafd937700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcaf8152010 0x7fcaf80b18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51214/0 (socket says 172.21.15.99:51214) 2023-10-08T00:15:37.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.261+0000 7fcafd937700 1 -- 172.21.15.99:0/497122951 learned_addr learned my addr 172.21.15.99:0/497122951 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:37.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.261+0000 7fcafd937700 1 -- 172.21.15.99:0/497122951 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcaf81523e0 msgr2=0x7fcaf8169fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:37.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.261+0000 7fcafd937700 1 --2- 172.21.15.99:0/497122951 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcaf81523e0 0x7fcaf8169fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:37.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.261+0000 7fcafd937700 1 -- 172.21.15.99:0/497122951 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcaf8158e50 msgr2=0x7fcaf8156c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:37.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.261+0000 7fcafd937700 1 --2- 172.21.15.99:0/497122951 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcaf8158e50 0x7fcaf8156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:37.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.261+0000 7fcafd937700 1 -- 172.21.15.99:0/497122951 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae80094f0 con 0x7fcaf8152010 2023-10-08T00:15:37.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.261+0000 7fcafd937700 1 --2- 172.21.15.99:0/497122951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcaf8152010 0x7fcaf80b18e0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fcaf400c7d0 tx=0x7fcaf4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:37.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.261+0000 7fcaedffb700 1 -- 172.21.15.99:0/497122951 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf4011600 con 0x7fcaf8152010 2023-10-08T00:15:37.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.261+0000 7fcaedffb700 1 -- 172.21.15.99:0/497122951 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaf4011760 con 0x7fcaf8152010 2023-10-08T00:15:37.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.261+0000 7fcaffb9b700 1 -- 172.21.15.99:0/497122951 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaf8151a70 con 0x7fcaf8152010 2023-10-08T00:15:37.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.261+0000 7fcaedffb700 1 -- 172.21.15.99:0/497122951 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf4011b20 con 0x7fcaf8152010 2023-10-08T00:15:37.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.261+0000 7fcaffb9b700 1 -- 172.21.15.99:0/497122951 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaf816a4f0 con 0x7fcaf8152010 2023-10-08T00:15:37.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.262+0000 7fcaffb9b700 1 -- 172.21.15.99:0/497122951 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaf804c3c0 con 0x7fcaf8152010 2023-10-08T00:15:37.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.263+0000 7fcaedffb700 1 -- 172.21.15.99:0/497122951 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcaf40193f0 con 0x7fcaf8152010 2023-10-08T00:15:37.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.263+0000 7fcaedffb700 1 --2- 172.21.15.99:0/497122951 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcadc066f30 0x7fcadc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:37.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.263+0000 7fcaedffb700 1 -- 172.21.15.99:0/497122951 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fcaf40845f0 con 0x7fcaf8152010 2023-10-08T00:15:37.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.263+0000 7fcafd136700 1 --2- 172.21.15.99:0/497122951 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcadc066f30 0x7fcadc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:37.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.264+0000 7fcafd136700 1 --2- 172.21.15.99:0/497122951 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcadc066f30 0x7fcadc0693e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fcaf80cc680 tx=0x7fcae8017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:37.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.267+0000 7fcaedffb700 1 -- 172.21.15.99:0/497122951 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcaf4051bb0 con 0x7fcaf8152010 2023-10-08T00:15:37.526 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.524+0000 7fcaffb9b700 1 -- 172.21.15.99:0/497122951 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fcaf806d1b0 con 0x7fcaf8152010 2023-10-08T00:15:37.526 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.525+0000 7fcaedffb700 1 -- 172.21.15.99:0/497122951 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fcaf40551d0 con 0x7fcaf8152010 2023-10-08T00:15:37.527 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:15:37.527 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:15:37.527 INFO:teuthology.orchestra.run.smithi099.stdout: "auth_dump": [ 2023-10-08T00:15:37.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:37.527 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.a", 2023-10-08T00:15:37.527 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlFlN9AhAAKdbeH0XznSOuyUnReD6RsA==", 2023-10-08T00:15:37.527 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:15:37.527 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:15:37.528 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2023-10-08T00:15:37.528 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2023-10-08T00:15:37.528 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:15:37.528 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:37.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:37.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:37.528 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.b", 2023-10-08T00:15:37.528 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlh2G0DRAA1mYGrUrELHsS9abUxCCG2g==", 2023-10-08T00:15:37.528 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:15:37.528 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:15:37.528 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2023-10-08T00:15:37.528 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2023-10-08T00:15:37.529 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:15:37.529 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:37.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:37.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:37.529 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.c", 2023-10-08T00:15:37.529 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFl/M1DCRAANrfHJGZO1s0KOyCdUzZWIA==", 2023-10-08T00:15:37.529 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:15:37.529 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:15:37.529 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2023-10-08T00:15:37.529 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2023-10-08T00:15:37.529 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:15:37.529 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:37.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:37.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:37.530 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.d", 2023-10-08T00:15:37.530 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlr5F2FBAAYYeG6EM0/N0FiEAwSbYRyA==", 2023-10-08T00:15:37.530 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:15:37.530 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:15:37.530 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2023-10-08T00:15:37.530 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2023-10-08T00:15:37.530 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:15:37.530 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:37.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:37.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:37.531 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.0", 2023-10-08T00:15:37.531 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB08yFla7rpKxAAWTOutLcrjDeR69JbPZMoLg==", 2023-10-08T00:15:37.531 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:15:37.531 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:15:37.531 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:15:37.531 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:15:37.531 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:37.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:37.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:37.531 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.1", 2023-10-08T00:15:37.531 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB08yFlw+y2MhAAAvLDIB/tb49IlaireTqTFg==", 2023-10-08T00:15:37.531 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:15:37.532 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:15:37.532 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:15:37.532 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:15:37.532 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:37.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:37.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:37.532 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.2", 2023-10-08T00:15:37.532 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB18yFlh4wvABAALXOtJZoEfGmt/pOBxReooQ==", 2023-10-08T00:15:37.532 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:15:37.533 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:15:37.533 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:15:37.533 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:15:37.533 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:37.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:37.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:37.533 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.3", 2023-10-08T00:15:37.533 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB18yFl8cVfCRAAoGXTACMT+f5Avll5mxG9OA==", 2023-10-08T00:15:37.533 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:15:37.533 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:15:37.534 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:15:37.534 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:15:37.534 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:37.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:37.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:37.534 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.4", 2023-10-08T00:15:37.534 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB48yFlft9WGhAA8JZN4MjSX9wbXiMKJnfCYg==", 2023-10-08T00:15:37.534 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:15:37.534 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:15:37.534 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:15:37.534 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:15:37.534 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:37.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:37.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:37.535 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.5", 2023-10-08T00:15:37.535 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB48yFlUf1BIRAA2FMBt1jAcm4ia/vYBDyojw==", 2023-10-08T00:15:37.535 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:15:37.535 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:15:37.535 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:15:37.535 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:15:37.535 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:37.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:37.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:37.536 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.6", 2023-10-08T00:15:37.536 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB48yFl0SI6KhAAI6G+Sqfm7OQmGf9nK59x9Q==", 2023-10-08T00:15:37.536 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:15:37.536 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:15:37.536 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:15:37.536 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:15:37.536 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:37.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:37.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:37.536 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.7", 2023-10-08T00:15:37.536 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB48yFlo5xWMxAA4p9OSLK0JpWhEExsifX+3A==", 2023-10-08T00:15:37.537 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:15:37.537 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:15:37.537 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:15:37.537 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:15:37.537 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:37.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:37.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:37.537 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.0", 2023-10-08T00:15:37.537 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlmzhQGBAAReslU/Pvcsr0dwE6l1N3/Q==", 2023-10-08T00:15:37.537 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:15:37.537 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:15:37.538 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow r", 2023-10-08T00:15:37.538 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-08T00:15:37.538 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:37.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:37.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:37.538 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.1", 2023-10-08T00:15:37.538 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFl2NKsIBAAUg6rXynl1J7qB6BkUizmpA==", 2023-10-08T00:15:37.538 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:15:37.538 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:15:37.538 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2023-10-08T00:15:37.539 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2023-10-08T00:15:37.539 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2023-10-08T00:15:37.539 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:37.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:37.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:37.539 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.2", 2023-10-08T00:15:37.539 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlMp6rHRAArVgJfA3mY4n78hOMeVlw1Q==", 2023-10-08T00:15:37.539 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:15:37.539 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:15:37.539 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2023-10-08T00:15:37.539 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2023-10-08T00:15:37.539 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2023-10-08T00:15:37.540 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:37.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:37.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:37.540 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.3", 2023-10-08T00:15:37.540 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlH0gNJhAAEgGVGiabVIWknpWUohOnHA==", 2023-10-08T00:15:37.540 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:15:37.540 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:15:37.540 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2023-10-08T00:15:37.540 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2023-10-08T00:15:37.540 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2023-10-08T00:15:37.540 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:37.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:37.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:37.541 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.admin", 2023-10-08T00:15:37.541 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBw8yFlidUfIxAAIvpvkHbyCx5JFvHo/g5A9g==", 2023-10-08T00:15:37.541 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:15:37.541 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2023-10-08T00:15:37.541 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2023-10-08T00:15:37.541 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2023-10-08T00:15:37.541 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:15:37.541 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:37.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:37.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:37.542 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mgr.x", 2023-10-08T00:15:37.542 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBw8yFl2kMLOxAAsEyuLBLsjQCUS0ERDOi75A==", 2023-10-08T00:15:37.542 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:15:37.542 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2023-10-08T00:15:37.542 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile mgr", 2023-10-08T00:15:37.542 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:15:37.542 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:37.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:37.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:37.542 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mgr.y", 2023-10-08T00:15:37.542 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBw8yFl6ht5NxAAFzXnLDjklCePD6hw/pSfFg==", 2023-10-08T00:15:37.543 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:15:37.543 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2023-10-08T00:15:37.543 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile mgr", 2023-10-08T00:15:37.543 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:15:37.543 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:37.543 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:37.543 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:37.543 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:15:37.545 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.527+0000 7fcaffb9b700 1 -- 172.21.15.99:0/497122951 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcadc066f30 msgr2=0x7fcadc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:37.545 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.527+0000 7fcaffb9b700 1 --2- 172.21.15.99:0/497122951 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcadc066f30 0x7fcadc0693e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fcaf80cc680 tx=0x7fcae8017000 comp rx=0 tx=0).stop 2023-10-08T00:15:37.545 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.527+0000 7fcaffb9b700 1 -- 172.21.15.99:0/497122951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcaf8152010 msgr2=0x7fcaf80b18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:37.546 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.527+0000 7fcaffb9b700 1 --2- 172.21.15.99:0/497122951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcaf8152010 0x7fcaf80b18e0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fcaf400c7d0 tx=0x7fcaf4008040 comp rx=0 tx=0).stop 2023-10-08T00:15:37.546 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.527+0000 7fcaffb9b700 1 -- 172.21.15.99:0/497122951 shutdown_connections 2023-10-08T00:15:37.546 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.527+0000 7fcaffb9b700 1 --2- 172.21.15.99:0/497122951 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcadc066f30 0x7fcadc0693e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:37.546 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.527+0000 7fcaffb9b700 1 --2- 172.21.15.99:0/497122951 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcaf8158e50 0x7fcaf8156c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:37.546 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.527+0000 7fcaffb9b700 1 --2- 172.21.15.99:0/497122951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcaf8152010 0x7fcaf80b18e0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:37.546 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.527+0000 7fcaffb9b700 1 --2- 172.21.15.99:0/497122951 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcaf81523e0 0x7fcaf8169fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:37.546 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.527+0000 7fcaffb9b700 1 -- 172.21.15.99:0/497122951 >> 172.21.15.99:0/497122951 conn(0x7fcaf8069870 msgr2=0x7fcaf80cadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:37.546 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.527+0000 7fcaffb9b700 1 -- 172.21.15.99:0/497122951 shutdown_connections 2023-10-08T00:15:37.546 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.527+0000 7fcaffb9b700 1 -- 172.21.15.99:0/497122951 wait complete. 2023-10-08T00:15:37.547 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:15:37.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.660+0000 7f2bdb59e700 1 Processor -- start 2023-10-08T00:15:37.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.660+0000 7f2bdb59e700 1 -- start start 2023-10-08T00:15:37.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.661+0000 7f2bdb59e700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2bdc0cd6e0 0x7f2bdc0cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:37.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.661+0000 7f2bdb59e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2bdc0cdab0 0x7f2bdc15d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:37.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.661+0000 7f2bdb59e700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bdc0cb7c0 con 0x7f2bdc15dda0 2023-10-08T00:15:37.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.661+0000 7f2bdb59e700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bdc15e850 con 0x7f2bdc0cd6e0 2023-10-08T00:15:37.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.661+0000 7f2bdb59e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bdc0cd030 con 0x7f2bdc0cdab0 2023-10-08T00:15:37.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.661+0000 7f2bda59c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2bdc0cd6e0 0x7f2bdc0cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:37.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.661+0000 7f2bdad9d700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f2bdc15dda0 0x7f2bdc15cbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45314/0 (socket says 172.21.15.99:45314) 2023-10-08T00:15:37.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.661+0000 7f2bdad9d700 1 -- 172.21.15.99:0/2676274960 learned_addr learned my addr 172.21.15.99:0/2676274960 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:15:37.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.661+0000 7f2bd1d9b700 1 --2- 172.21.15.99:0/2676274960 >> v2:172.21.15.191:3300/0 conn(0x7f2bdc0cdab0 0x7f2bdc15d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:37.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.661+0000 7f2bda59c700 1 -- 172.21.15.99:0/2676274960 >> v2:172.21.15.191:3300/0 conn(0x7f2bdc0cdab0 msgr2=0x7f2bdc15d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:37.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.661+0000 7f2bda59c700 1 --2- 172.21.15.99:0/2676274960 >> v2:172.21.15.191:3300/0 conn(0x7f2bdc0cdab0 0x7f2bdc15d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:37.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.661+0000 7f2bda59c700 1 -- 172.21.15.99:0/2676274960 >> v1:172.21.15.99:6789/0 conn(0x7f2bdc15dda0 legacy=0x7f2bdc15cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:37.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.661+0000 7f2bda59c700 1 -- 172.21.15.99:0/2676274960 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bdc15e290 con 0x7f2bdc0cd6e0 2023-10-08T00:15:37.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.662+0000 7f2bd1d9b700 1 --2- 172.21.15.99:0/2676274960 >> v2:172.21.15.191:3300/0 conn(0x7f2bdc0cdab0 0x7f2bdc15d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:15:37.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.662+0000 7f2bda59c700 1 --2- 172.21.15.99:0/2676274960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bdc0cd6e0 0x7f2bdc0cbc50 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc0b6c70 tx=0x7f2bcc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=253e7879bbb87b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:37.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.662+0000 7f2bd9d9b700 1 -- 172.21.15.99:0/2676274960 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc00b6b0 con 0x7f2bdc0cd6e0 2023-10-08T00:15:37.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.662+0000 7f2bd9d9b700 1 -- 172.21.15.99:0/2676274960 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bcc00b810 con 0x7f2bdc0cd6e0 2023-10-08T00:15:37.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.662+0000 7f2bd9d9b700 1 -- 172.21.15.99:0/2676274960 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc00bbb0 con 0x7f2bdc0cd6e0 2023-10-08T00:15:37.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.662+0000 7f2bdb59e700 1 -- 172.21.15.99:0/2676274960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bdc0cd6e0 msgr2=0x7f2bdc0cbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:37.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.662+0000 7f2bdb59e700 1 --2- 172.21.15.99:0/2676274960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bdc0cd6e0 0x7f2bdc0cbc50 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc0b6c70 tx=0x7f2bcc0091a0 comp rx=0 tx=0).stop 2023-10-08T00:15:37.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.662+0000 7f2bdb59e700 1 -- 172.21.15.99:0/2676274960 shutdown_connections 2023-10-08T00:15:37.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.662+0000 7f2bdb59e700 1 --2- 172.21.15.99:0/2676274960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bdc0cd6e0 0x7f2bdc0cbc50 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:37.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.662+0000 7f2bdb59e700 1 --2- 172.21.15.99:0/2676274960 >> v2:172.21.15.191:3300/0 conn(0x7f2bdc0cdab0 0x7f2bdc15d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:37.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.662+0000 7f2bdb59e700 1 -- 172.21.15.99:0/2676274960 >> 172.21.15.99:0/2676274960 conn(0x7f2bdc067660 msgr2=0x7f2bdc067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:37.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.662+0000 7f2bdb59e700 1 -- 172.21.15.99:0/2676274960 shutdown_connections 2023-10-08T00:15:37.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.662+0000 7f2bdb59e700 1 -- 172.21.15.99:0/2676274960 wait complete. 2023-10-08T00:15:37.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.664+0000 7f2bdb59e700 1 Processor -- start 2023-10-08T00:15:37.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.664+0000 7f2bdb59e700 1 -- start start 2023-10-08T00:15:37.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.664+0000 7f2bdb59e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2bdc0cd6e0 0x7f2bdc1589c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:37.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.664+0000 7f2bdb59e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bdc0cdab0 0x7f2bdc0c6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:37.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.664+0000 7f2bdb59e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2bdc15dda0 0x7f2bdc0c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:37.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.664+0000 7f2bdb59e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bdc0c8fd0 con 0x7f2bdc0cdab0 2023-10-08T00:15:37.669 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.664+0000 7f2bdb59e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bdc15ba60 con 0x7f2bdc15dda0 2023-10-08T00:15:37.669 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.664+0000 7f2bdb59e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bdc15bd40 con 0x7f2bdc0cd6e0 2023-10-08T00:15:37.669 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.664+0000 7f2bd1d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bdc0cdab0 0x7f2bdc0c6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:37.669 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.664+0000 7f2bd1d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bdc0cdab0 0x7f2bdc0c6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51236/0 (socket says 172.21.15.99:51236) 2023-10-08T00:15:37.669 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.664+0000 7f2bd1d9b700 1 -- 172.21.15.99:0/3720356055 learned_addr learned my addr 172.21.15.99:0/3720356055 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:37.669 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.665+0000 7f2bdad9d700 1 --2- 172.21.15.99:0/3720356055 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2bdc15dda0 0x7f2bdc0c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:37.669 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.665+0000 7f2bda59c700 1 --2- 172.21.15.99:0/3720356055 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2bdc0cd6e0 0x7f2bdc1589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:37.669 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.665+0000 7f2bd1d9b700 1 -- 172.21.15.99:0/3720356055 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2bdc0cd6e0 msgr2=0x7f2bdc1589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:37.670 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.665+0000 7f2bd1d9b700 1 --2- 172.21.15.99:0/3720356055 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2bdc0cd6e0 0x7f2bdc1589c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:37.670 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.665+0000 7f2bd1d9b700 1 -- 172.21.15.99:0/3720356055 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2bdc15dda0 msgr2=0x7f2bdc0c7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:37.670 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.665+0000 7f2bd1d9b700 1 --2- 172.21.15.99:0/3720356055 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2bdc15dda0 0x7f2bdc0c7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:37.670 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.665+0000 7f2bd1d9b700 1 -- 172.21.15.99:0/3720356055 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bcc009420 con 0x7f2bdc0cdab0 2023-10-08T00:15:37.670 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.665+0000 7f2bd1d9b700 1 --2- 172.21.15.99:0/3720356055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bdc0cdab0 0x7f2bdc0c6500 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f2bc400c830 tx=0x7f2bc4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:37.670 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.665+0000 7f2bd2ffd700 1 -- 172.21.15.99:0/3720356055 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4011600 con 0x7f2bdc0cdab0 2023-10-08T00:15:37.670 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.665+0000 7f2bd2ffd700 1 -- 172.21.15.99:0/3720356055 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc4011760 con 0x7f2bdc0cdab0 2023-10-08T00:15:37.670 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.665+0000 7f2bdb59e700 1 -- 172.21.15.99:0/3720356055 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bdc15c020 con 0x7f2bdc0cdab0 2023-10-08T00:15:37.670 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.665+0000 7f2bd2ffd700 1 -- 172.21.15.99:0/3720356055 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4011b20 con 0x7f2bdc0cdab0 2023-10-08T00:15:37.670 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.665+0000 7f2bdb59e700 1 -- 172.21.15.99:0/3720356055 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bdc1590c0 con 0x7f2bdc0cdab0 2023-10-08T00:15:37.670 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.666+0000 7f2bdb59e700 1 -- 172.21.15.99:0/3720356055 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bdc15c210 con 0x7f2bdc0cdab0 2023-10-08T00:15:37.671 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.666+0000 7f2bd2ffd700 1 -- 172.21.15.99:0/3720356055 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2bc400a3e0 con 0x7f2bdc0cdab0 2023-10-08T00:15:37.671 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.667+0000 7f2bd2ffd700 1 --2- 172.21.15.99:0/3720356055 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2bbc066f30 0x7f2bbc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:37.671 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.667+0000 7f2bd2ffd700 1 -- 172.21.15.99:0/3720356055 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f2bc4084670 con 0x7f2bdc0cdab0 2023-10-08T00:15:37.671 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.667+0000 7f2bda59c700 1 --2- 172.21.15.99:0/3720356055 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2bbc066f30 0x7f2bbc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:37.671 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.667+0000 7f2bda59c700 1 --2- 172.21.15.99:0/3720356055 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2bbc066f30 0x7f2bbc0693e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc0093f0 tx=0x7f2bcc017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:37.671 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.671+0000 7f2bd2ffd700 1 -- 172.21.15.99:0/3720356055 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2bc4051c30 con 0x7f2bdc0cdab0 2023-10-08T00:15:37.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.855+0000 7f2bdb59e700 1 -- 172.21.15.99:0/3720356055 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2bdc15e570 con 0x7f2bdc0cdab0 2023-10-08T00:15:37.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.855+0000 7f2bd2ffd700 1 -- 172.21.15.99:0/3720356055 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+12136 (secure 0 0 0) 0x7f2bc4055250 con 0x7f2bdc0cdab0 2023-10-08T00:15:37.856 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:15:37.856 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":100,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:15:37.118806+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:15:37.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.857+0000 7f2bdb59e700 1 -- 172.21.15.99:0/3720356055 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2bbc066f30 msgr2=0x7f2bbc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:37.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.857+0000 7f2bdb59e700 1 --2- 172.21.15.99:0/3720356055 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2bbc066f30 0x7f2bbc0693e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc0093f0 tx=0x7f2bcc017000 comp rx=0 tx=0).stop 2023-10-08T00:15:37.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.857+0000 7f2bdb59e700 1 -- 172.21.15.99:0/3720356055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bdc0cdab0 msgr2=0x7f2bdc0c6500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:37.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.857+0000 7f2bdb59e700 1 --2- 172.21.15.99:0/3720356055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bdc0cdab0 0x7f2bdc0c6500 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f2bc400c830 tx=0x7f2bc4008040 comp rx=0 tx=0).stop 2023-10-08T00:15:37.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.857+0000 7f2bdb59e700 1 -- 172.21.15.99:0/3720356055 shutdown_connections 2023-10-08T00:15:37.859 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.857+0000 7f2bdb59e700 1 --2- 172.21.15.99:0/3720356055 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2bbc066f30 0x7f2bbc0693e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:37.859 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.857+0000 7f2bdb59e700 1 --2- 172.21.15.99:0/3720356055 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2bdc15dda0 0x7f2bdc0c7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:37.859 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.857+0000 7f2bdb59e700 1 --2- 172.21.15.99:0/3720356055 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2bdc0cd6e0 0x7f2bdc1589c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:37.859 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.857+0000 7f2bdb59e700 1 --2- 172.21.15.99:0/3720356055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bdc0cdab0 0x7f2bdc0c6500 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:37.859 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.857+0000 7f2bdb59e700 1 -- 172.21.15.99:0/3720356055 >> 172.21.15.99:0/3720356055 conn(0x7f2bdc067660 msgr2=0x7f2bdc1571d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:37.859 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.857+0000 7f2bdb59e700 1 -- 172.21.15.99:0/3720356055 shutdown_connections 2023-10-08T00:15:37.859 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.857+0000 7f2bdb59e700 1 -- 172.21.15.99:0/3720356055 wait complete. 2023-10-08T00:15:37.871 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:10:42.063344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-08T00:15:37.871 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T00:15:37.986 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.986+0000 7fb578ba9700 1 Processor -- start 2023-10-08T00:15:37.987 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.986+0000 7fb578ba9700 1 -- start start 2023-10-08T00:15:37.989 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.987+0000 7fb578ba9700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb574159ec0 0x7fb5740af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:37.989 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.987+0000 7fb578ba9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb57406acb0 0x7fb5740afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:37.989 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.987+0000 7fb578ba9700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5740b0790 con 0x7fb57415a290 2023-10-08T00:15:37.989 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.987+0000 7fb578ba9700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5740ce1f0 con 0x7fb574159ec0 2023-10-08T00:15:37.989 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.987+0000 7fb578ba9700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5740b0b20 con 0x7fb57406acb0 2023-10-08T00:15:37.989 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.987+0000 7fb56bfff700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fb57415a290 0x7fb57406a5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45326/0 (socket says 172.21.15.99:45326) 2023-10-08T00:15:37.990 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.987+0000 7fb56bfff700 1 -- 172.21.15.99:0/2927627711 learned_addr learned my addr 172.21.15.99:0/2927627711 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:15:37.990 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.987+0000 7fb57259c700 1 --2- 172.21.15.99:0/2927627711 >> v2:172.21.15.191:3300/0 conn(0x7fb574159ec0 0x7fb5740af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:37.990 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.987+0000 7fb572d9d700 1 --2- 172.21.15.99:0/2927627711 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb57406acb0 0x7fb5740afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:37.990 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.987+0000 7fb571d9b700 1 -- 172.21.15.99:0/2927627711 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2929856570 0 0) 0x7fb5740b0790 con 0x7fb57415a290 2023-10-08T00:15:37.990 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.987+0000 7fb571d9b700 1 -- 172.21.15.99:0/2927627711 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb560003570 con 0x7fb57415a290 2023-10-08T00:15:37.990 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.987+0000 7fb57259c700 1 -- 172.21.15.99:0/2927627711 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb57406acb0 msgr2=0x7fb5740afe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:37.990 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.987+0000 7fb57259c700 1 --2- 172.21.15.99:0/2927627711 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb57406acb0 0x7fb5740afe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:37.990 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.987+0000 7fb57259c700 1 -- 172.21.15.99:0/2927627711 >> v1:172.21.15.99:6789/0 conn(0x7fb57415a290 legacy=0x7fb57406a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:37.990 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.987+0000 7fb57259c700 1 -- 172.21.15.99:0/2927627711 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5740b04b0 con 0x7fb574159ec0 2023-10-08T00:15:37.990 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.987+0000 7fb572d9d700 1 --2- 172.21.15.99:0/2927627711 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb57406acb0 0x7fb5740afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:15:37.991 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.987+0000 7fb57259c700 1 --2- 172.21.15.99:0/2927627711 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb574159ec0 0x7fb5740af940 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb5640098e0 tx=0x7fb5640091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=576daee26633d540 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:37.991 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.987+0000 7fb571d9b700 1 -- 172.21.15.99:0/2927627711 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb564011630 con 0x7fb574159ec0 2023-10-08T00:15:37.991 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.988+0000 7fb571d9b700 1 -- 172.21.15.99:0/2927627711 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb564011790 con 0x7fb574159ec0 2023-10-08T00:15:37.991 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.988+0000 7fb571d9b700 1 -- 172.21.15.99:0/2927627711 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb564011b30 con 0x7fb574159ec0 2023-10-08T00:15:37.991 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.988+0000 7fb578ba9700 1 -- 172.21.15.99:0/2927627711 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb574159ec0 msgr2=0x7fb5740af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:37.991 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.988+0000 7fb578ba9700 1 --2- 172.21.15.99:0/2927627711 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb574159ec0 0x7fb5740af940 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb5640098e0 tx=0x7fb5640091a0 comp rx=0 tx=0).stop 2023-10-08T00:15:37.991 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.988+0000 7fb578ba9700 1 -- 172.21.15.99:0/2927627711 shutdown_connections 2023-10-08T00:15:37.991 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.988+0000 7fb578ba9700 1 --2- 172.21.15.99:0/2927627711 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb574159ec0 0x7fb5740af940 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:37.991 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.988+0000 7fb578ba9700 1 --2- 172.21.15.99:0/2927627711 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb57406acb0 0x7fb5740afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:37.991 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.988+0000 7fb578ba9700 1 -- 172.21.15.99:0/2927627711 >> 172.21.15.99:0/2927627711 conn(0x7fb5740d1880 msgr2=0x7fb5740cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:37.991 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.988+0000 7fb578ba9700 1 -- 172.21.15.99:0/2927627711 shutdown_connections 2023-10-08T00:15:37.992 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.988+0000 7fb578ba9700 1 -- 172.21.15.99:0/2927627711 wait complete. 2023-10-08T00:15:37.994 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.989+0000 7fb578ba9700 1 Processor -- start 2023-10-08T00:15:37.995 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.989+0000 7fb578ba9700 1 -- start start 2023-10-08T00:15:37.995 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.990+0000 7fb578ba9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb57406acb0 0x7fb57416d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:37.995 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.990+0000 7fb578ba9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb574159ec0 0x7fb57416ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:37.995 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.990+0000 7fb578ba9700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb57415a290 0x7fb5741583a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:37.995 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.990+0000 7fb578ba9700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57415cb20 con 0x7fb574159ec0 2023-10-08T00:15:37.995 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.990+0000 7fb578ba9700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57416dfc0 con 0x7fb57415a290 2023-10-08T00:15:37.995 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.990+0000 7fb578ba9700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb57416e2a0 con 0x7fb57406acb0 2023-10-08T00:15:37.995 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.990+0000 7fb56bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb574159ec0 0x7fb57416ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:37.995 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.990+0000 7fb56bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb574159ec0 0x7fb57416ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51244/0 (socket says 172.21.15.99:51244) 2023-10-08T00:15:37.995 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.990+0000 7fb56bfff700 1 -- 172.21.15.99:0/1933471724 learned_addr learned my addr 172.21.15.99:0/1933471724 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:37.996 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.990+0000 7fb572d9d700 1 --2- 172.21.15.99:0/1933471724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb57415a290 0x7fb5741583a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:37.996 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.990+0000 7fb57259c700 1 --2- 172.21.15.99:0/1933471724 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb57406acb0 0x7fb57416d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:37.996 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.990+0000 7fb56bfff700 1 -- 172.21.15.99:0/1933471724 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb57406acb0 msgr2=0x7fb57416d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:37.996 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.990+0000 7fb56bfff700 1 --2- 172.21.15.99:0/1933471724 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb57406acb0 0x7fb57416d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:37.996 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.990+0000 7fb56bfff700 1 -- 172.21.15.99:0/1933471724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb57415a290 msgr2=0x7fb5741583a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:37.996 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.990+0000 7fb56bfff700 1 --2- 172.21.15.99:0/1933471724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb57415a290 0x7fb5741583a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:37.996 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.990+0000 7fb56bfff700 1 -- 172.21.15.99:0/1933471724 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb56c009a20 con 0x7fb574159ec0 2023-10-08T00:15:37.996 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.991+0000 7fb56bfff700 1 --2- 172.21.15.99:0/1933471724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb574159ec0 0x7fb57416ced0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fb55c00c590 tx=0x7fb55c007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:37.996 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.991+0000 7fb56a7fc700 1 -- 172.21.15.99:0/1933471724 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55c005b30 con 0x7fb574159ec0 2023-10-08T00:15:37.996 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.991+0000 7fb56a7fc700 1 -- 172.21.15.99:0/1933471724 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb55c005cb0 con 0x7fb574159ec0 2023-10-08T00:15:37.997 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.991+0000 7fb56a7fc700 1 -- 172.21.15.99:0/1933471724 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55c007320 con 0x7fb574159ec0 2023-10-08T00:15:37.997 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.991+0000 7fb578ba9700 1 -- 172.21.15.99:0/1933471724 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb564009420 con 0x7fb574159ec0 2023-10-08T00:15:37.997 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.991+0000 7fb578ba9700 1 -- 172.21.15.99:0/1933471724 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb57416dcd0 con 0x7fb574159ec0 2023-10-08T00:15:37.997 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.992+0000 7fb578ba9700 1 -- 172.21.15.99:0/1933471724 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb57415ccd0 con 0x7fb574159ec0 2023-10-08T00:15:37.997 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.992+0000 7fb56a7fc700 1 -- 172.21.15.99:0/1933471724 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb55c01b3b0 con 0x7fb574159ec0 2023-10-08T00:15:37.997 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.992+0000 7fb56a7fc700 1 --2- 172.21.15.99:0/1933471724 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb554066f30 0x7fb5540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:37.997 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.992+0000 7fb56a7fc700 1 -- 172.21.15.99:0/1933471724 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fb55c083610 con 0x7fb574159ec0 2023-10-08T00:15:37.997 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.993+0000 7fb57259c700 1 --2- 172.21.15.99:0/1933471724 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb554066f30 0x7fb5540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:37.997 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.993+0000 7fb57259c700 1 --2- 172.21.15.99:0/1933471724 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb554066f30 0x7fb5540693e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb5640091e0 tx=0x7fb564017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:37.998 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:37.996+0000 7fb56a7fc700 1 -- 172.21.15.99:0/1933471724 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb57415ccd0 con 0x7fb574159ec0 2023-10-08T00:15:38.177 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.176+0000 7fb578ba9700 1 -- 172.21.15.99:0/1933471724 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb57404c3c0 con 0x7fb574159ec0 2023-10-08T00:15:38.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.177+0000 7fb56a7fc700 1 -- 172.21.15.99:0/1933471724 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb55c0170a0 con 0x7fb574159ec0 2023-10-08T00:15:38.178 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2023-10-08T00:15:38.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.179+0000 7fb578ba9700 1 -- 172.21.15.99:0/1933471724 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb554066f30 msgr2=0x7fb5540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:38.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.179+0000 7fb578ba9700 1 --2- 172.21.15.99:0/1933471724 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb554066f30 0x7fb5540693e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb5640091e0 tx=0x7fb564017000 comp rx=0 tx=0).stop 2023-10-08T00:15:38.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.179+0000 7fb578ba9700 1 -- 172.21.15.99:0/1933471724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb574159ec0 msgr2=0x7fb57416ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:38.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.179+0000 7fb578ba9700 1 --2- 172.21.15.99:0/1933471724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb574159ec0 0x7fb57416ced0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fb55c00c590 tx=0x7fb55c007e00 comp rx=0 tx=0).stop 2023-10-08T00:15:38.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.179+0000 7fb578ba9700 1 -- 172.21.15.99:0/1933471724 shutdown_connections 2023-10-08T00:15:38.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.179+0000 7fb578ba9700 1 --2- 172.21.15.99:0/1933471724 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb554066f30 0x7fb5540693e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:38.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.179+0000 7fb578ba9700 1 --2- 172.21.15.99:0/1933471724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb57415a290 0x7fb5741583a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:38.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.179+0000 7fb578ba9700 1 --2- 172.21.15.99:0/1933471724 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb57406acb0 0x7fb57416d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:38.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.179+0000 7fb578ba9700 1 --2- 172.21.15.99:0/1933471724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb574159ec0 0x7fb57416ced0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:38.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.179+0000 7fb578ba9700 1 -- 172.21.15.99:0/1933471724 >> 172.21.15.99:0/1933471724 conn(0x7fb5740d1880 msgr2=0x7fb5740c5b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:38.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.179+0000 7fb578ba9700 1 -- 172.21.15.99:0/1933471724 shutdown_connections 2023-10-08T00:15:38.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.179+0000 7fb578ba9700 1 -- 172.21.15.99:0/1933471724 wait complete. 2023-10-08T00:15:38.192 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T00:15:38.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.306+0000 7feaf5da5700 1 Processor -- start 2023-10-08T00:15:38.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.306+0000 7feaf5da5700 1 -- start start 2023-10-08T00:15:38.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.307+0000 7feaf5da5700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7feaf0157870 0x7feaf015b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:38.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.307+0000 7feaf5da5700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7feaf0157c40 0x7feaf0158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:38.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.307+0000 7feaf5da5700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaf015cbb0 con 0x7feaf015a620 2023-10-08T00:15:38.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.307+0000 7feaf5da5700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaf015bfa0 con 0x7feaf0157870 2023-10-08T00:15:38.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.307+0000 7feaf5da5700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaf015c280 con 0x7feaf0157c40 2023-10-08T00:15:38.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.307+0000 7feaef7fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7feaf0157870 0x7feaf015b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:38.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.307+0000 7feaeeffd700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7feaf015a620 0x7feaf015d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45342/0 (socket says 172.21.15.99:45342) 2023-10-08T00:15:38.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.307+0000 7feaeeffd700 1 -- 172.21.15.99:0/175939297 learned_addr learned my addr 172.21.15.99:0/175939297 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:15:38.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.307+0000 7feaeffff700 1 --2- 172.21.15.99:0/175939297 >> v2:172.21.15.191:3300/0 conn(0x7feaf0157c40 0x7feaf0158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:38.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.307+0000 7feaef7fe700 1 -- 172.21.15.99:0/175939297 >> v2:172.21.15.191:3300/0 conn(0x7feaf0157c40 msgr2=0x7feaf0158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:38.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.307+0000 7feaef7fe700 1 --2- 172.21.15.99:0/175939297 >> v2:172.21.15.191:3300/0 conn(0x7feaf0157c40 0x7feaf0158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:38.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.307+0000 7feaef7fe700 1 -- 172.21.15.99:0/175939297 >> v1:172.21.15.99:6789/0 conn(0x7feaf015a620 legacy=0x7feaf015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:38.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.307+0000 7feaef7fe700 1 -- 172.21.15.99:0/175939297 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf015c560 con 0x7feaf0157870 2023-10-08T00:15:38.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.307+0000 7feaee7fc700 1 -- 172.21.15.99:0/175939297 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2276192147 0 0) 0x7feaf015cbb0 con 0x7feaf015a620 2023-10-08T00:15:38.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.308+0000 7feaef7fe700 1 --2- 172.21.15.99:0/175939297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feaf0157870 0x7feaf015b6a0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7feaf006b4e0 tx=0x7feae0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e900515883bb8e38 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:38.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.308+0000 7feaee7fc700 1 -- 172.21.15.99:0/175939297 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae000e070 con 0x7feaf0157870 2023-10-08T00:15:38.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.308+0000 7feaee7fc700 1 -- 172.21.15.99:0/175939297 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae000bc30 con 0x7feaf0157870 2023-10-08T00:15:38.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.308+0000 7feaee7fc700 1 -- 172.21.15.99:0/175939297 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae0012430 con 0x7feaf0157870 2023-10-08T00:15:38.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.308+0000 7feaf5da5700 1 -- 172.21.15.99:0/175939297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feaf0157870 msgr2=0x7feaf015b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:38.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.308+0000 7feaf5da5700 1 --2- 172.21.15.99:0/175939297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feaf0157870 0x7feaf015b6a0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7feaf006b4e0 tx=0x7feae0009780 comp rx=0 tx=0).stop 2023-10-08T00:15:38.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.308+0000 7feaf5da5700 1 -- 172.21.15.99:0/175939297 shutdown_connections 2023-10-08T00:15:38.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.308+0000 7feaf5da5700 1 --2- 172.21.15.99:0/175939297 >> v2:172.21.15.191:3300/0 conn(0x7feaf0157c40 0x7feaf0158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:38.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.308+0000 7feaf5da5700 1 --2- 172.21.15.99:0/175939297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feaf0157870 0x7feaf015b6a0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:38.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.308+0000 7feaf5da5700 1 -- 172.21.15.99:0/175939297 >> 172.21.15.99:0/175939297 conn(0x7feaf0068ce0 msgr2=0x7feaf00690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:38.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.308+0000 7feaf5da5700 1 -- 172.21.15.99:0/175939297 shutdown_connections 2023-10-08T00:15:38.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.308+0000 7feaf5da5700 1 -- 172.21.15.99:0/175939297 wait complete. 2023-10-08T00:15:38.314 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.309+0000 7feaf5da5700 1 Processor -- start 2023-10-08T00:15:38.314 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.310+0000 7feaf5da5700 1 -- start start 2023-10-08T00:15:38.314 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.310+0000 7feaf5da5700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feaf0157870 0x7feaf0153750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:38.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.310+0000 7feaf5da5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feaf0157c40 0x7feaf00b2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:38.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.310+0000 7feaf5da5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feaf015a620 0x7feaf00b1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:38.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.310+0000 7feaf5da5700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf014ee40 con 0x7feaf0157c40 2023-10-08T00:15:38.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.310+0000 7feaf5da5700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf014f120 con 0x7feaf0157870 2023-10-08T00:15:38.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.310+0000 7feaf5da5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaf014f400 con 0x7feaf015a620 2023-10-08T00:15:38.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.310+0000 7feaeeffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feaf0157c40 0x7feaf00b2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:38.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.310+0000 7feaeeffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feaf0157c40 0x7feaf00b2b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51270/0 (socket says 172.21.15.99:51270) 2023-10-08T00:15:38.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.310+0000 7feaeeffd700 1 -- 172.21.15.99:0/946712418 learned_addr learned my addr 172.21.15.99:0/946712418 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:38.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.311+0000 7feaef7fe700 1 --2- 172.21.15.99:0/946712418 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feaf0157870 0x7feaf0153750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:38.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.311+0000 7feaeeffd700 1 -- 172.21.15.99:0/946712418 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feaf015a620 msgr2=0x7feaf00b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:38.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.311+0000 7feaeeffd700 1 --2- 172.21.15.99:0/946712418 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feaf015a620 0x7feaf00b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:38.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.311+0000 7feaeeffd700 1 -- 172.21.15.99:0/946712418 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feaf0157870 msgr2=0x7feaf0153750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:38.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.311+0000 7feaeeffd700 1 --2- 172.21.15.99:0/946712418 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feaf0157870 0x7feaf0153750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:38.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.311+0000 7feaeeffd700 1 -- 172.21.15.99:0/946712418 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae0009a20 con 0x7feaf0157c40 2023-10-08T00:15:38.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.311+0000 7feaeeffd700 1 --2- 172.21.15.99:0/946712418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feaf0157c40 0x7feaf00b2b40 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7feae400c5e0 tx=0x7feae4000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:38.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.311+0000 7fead77fe700 1 -- 172.21.15.99:0/946712418 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae4005b30 con 0x7feaf0157c40 2023-10-08T00:15:38.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.311+0000 7fead77fe700 1 -- 172.21.15.99:0/946712418 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae4005c90 con 0x7feaf0157c40 2023-10-08T00:15:38.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.311+0000 7feaf5da5700 1 -- 172.21.15.99:0/946712418 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaf014f6e0 con 0x7feaf0157c40 2023-10-08T00:15:38.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.311+0000 7fead77fe700 1 -- 172.21.15.99:0/946712418 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae4007300 con 0x7feaf0157c40 2023-10-08T00:15:38.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.311+0000 7feaf5da5700 1 -- 172.21.15.99:0/946712418 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaf014e9f0 con 0x7feaf0157c40 2023-10-08T00:15:38.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.312+0000 7feaf5da5700 1 -- 172.21.15.99:0/946712418 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaf014ec10 con 0x7feaf0157c40 2023-10-08T00:15:38.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.312+0000 7fead77fe700 1 -- 172.21.15.99:0/946712418 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7feae40076b0 con 0x7feaf0157c40 2023-10-08T00:15:38.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.313+0000 7fead77fe700 1 --2- 172.21.15.99:0/946712418 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7feadc066f30 0x7feadc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:38.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.313+0000 7fead77fe700 1 -- 172.21.15.99:0/946712418 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 4853+0+0 (secure 0 0 0) 0x7feae4083720 con 0x7feaf0157c40 2023-10-08T00:15:38.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.313+0000 7feaef7fe700 1 --2- 172.21.15.99:0/946712418 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7feadc066f30 0x7feadc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:38.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.313+0000 7feaef7fe700 1 --2- 172.21.15.99:0/946712418 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7feadc066f30 0x7feadc0693e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7feaf006b4e0 tx=0x7feae0019000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:38.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.317+0000 7fead77fe700 1 -- 172.21.15.99:0/946712418 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feae4050ce0 con 0x7feaf0157c40 2023-10-08T00:15:38.502 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.502+0000 7feaf5da5700 1 -- 172.21.15.99:0/946712418 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7feaf014fc00 con 0x7feaf0157c40 2023-10-08T00:15:38.503 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.502+0000 7fead77fe700 1 -- 172.21.15.99:0/946712418 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v100) v1 ==== 84+0+65 (secure 0 0 0) 0x7feae4017060 con 0x7feaf0157c40 2023-10-08T00:15:38.503 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:15:38.503 INFO:teuthology.orchestra.run.smithi099.stdout:[ 2023-10-08T00:15:38.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:38.503 INFO:teuthology.orchestra.run.smithi099.stdout: "poolnum": 1, 2023-10-08T00:15:38.503 INFO:teuthology.orchestra.run.smithi099.stdout: "poolname": ".mgr" 2023-10-08T00:15:38.503 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:38.503 INFO:teuthology.orchestra.run.smithi099.stdout:] 2023-10-08T00:15:38.505 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.504+0000 7feaf5da5700 1 -- 172.21.15.99:0/946712418 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7feadc066f30 msgr2=0x7feadc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:38.505 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.504+0000 7feaf5da5700 1 --2- 172.21.15.99:0/946712418 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7feadc066f30 0x7feadc0693e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7feaf006b4e0 tx=0x7feae0019000 comp rx=0 tx=0).stop 2023-10-08T00:15:38.505 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.504+0000 7feaf5da5700 1 -- 172.21.15.99:0/946712418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feaf0157c40 msgr2=0x7feaf00b2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:38.505 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.504+0000 7feaf5da5700 1 --2- 172.21.15.99:0/946712418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feaf0157c40 0x7feaf00b2b40 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7feae400c5e0 tx=0x7feae4000fe0 comp rx=0 tx=0).stop 2023-10-08T00:15:38.505 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.504+0000 7feaf5da5700 1 -- 172.21.15.99:0/946712418 shutdown_connections 2023-10-08T00:15:38.505 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.504+0000 7feaf5da5700 1 --2- 172.21.15.99:0/946712418 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7feadc066f30 0x7feadc0693e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:38.505 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.504+0000 7feaf5da5700 1 --2- 172.21.15.99:0/946712418 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feaf0157870 0x7feaf0153750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:38.506 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.504+0000 7feaf5da5700 1 --2- 172.21.15.99:0/946712418 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feaf015a620 0x7feaf00b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:38.506 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.504+0000 7feaf5da5700 1 --2- 172.21.15.99:0/946712418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feaf0157c40 0x7feaf00b2b40 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:38.506 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.504+0000 7feaf5da5700 1 -- 172.21.15.99:0/946712418 >> 172.21.15.99:0/946712418 conn(0x7feaf0068ce0 msgr2=0x7feaf00b4810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:38.506 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.504+0000 7feaf5da5700 1 -- 172.21.15.99:0/946712418 shutdown_connections 2023-10-08T00:15:38.506 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.504+0000 7feaf5da5700 1 -- 172.21.15.99:0/946712418 wait complete. 2023-10-08T00:15:38.517 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T00:15:38.517 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T00:15:38.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.634+0000 7fe9e1a93700 1 Processor -- start 2023-10-08T00:15:38.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.634+0000 7fe9e1a93700 1 -- start start 2023-10-08T00:15:38.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.634+0000 7fe9e1a93700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe9dc159ef0 0x7fe9dc15a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:38.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.634+0000 7fe9e1a93700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe9dc0b6a10 0x7fe9dc0b10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:38.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.634+0000 7fe9e1a93700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fe9dc0b15e0 0x7fe9dc0b46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:38.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.634+0000 7fe9e1a93700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9dc0b1a60 con 0x7fe9dc159ef0 2023-10-08T00:15:38.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.634+0000 7fe9e1a93700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9dc0b1d40 con 0x7fe9dc0b15e0 2023-10-08T00:15:38.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.634+0000 7fe9e1a93700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc0b2020 con 0x7fe9dc0b6a10 2023-10-08T00:15:38.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.635+0000 7fe9db7fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe9dc159ef0 0x7fe9dc15a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:38.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.635+0000 7fe9db7fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe9dc159ef0 0x7fe9dc15a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51272/0 (socket says 172.21.15.99:51272) 2023-10-08T00:15:38.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.635+0000 7fe9db7fe700 1 -- 172.21.15.99:0/3220021929 learned_addr learned my addr 172.21.15.99:0/3220021929 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:38.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.635+0000 7fe9daffd700 1 --2- 172.21.15.99:0/3220021929 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe9dc0b6a10 0x7fe9dc0b10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:38.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.635+0000 7fe9dbfff700 1 --2- 172.21.15.99:0/3220021929 >> v2:172.21.15.191:3300/0 conn(0x7fe9dc0b15e0 0x7fe9dc0b46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:38.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.635+0000 7fe9db7fe700 1 -- 172.21.15.99:0/3220021929 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe9dc0b6a10 msgr2=0x7fe9dc0b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:38.640 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.635+0000 7fe9db7fe700 1 --2- 172.21.15.99:0/3220021929 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe9dc0b6a10 0x7fe9dc0b10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:38.640 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.635+0000 7fe9db7fe700 1 -- 172.21.15.99:0/3220021929 >> v2:172.21.15.191:3300/0 conn(0x7fe9dc0b15e0 msgr2=0x7fe9dc0b46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:38.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.635+0000 7fe9db7fe700 1 --2- 172.21.15.99:0/3220021929 >> v2:172.21.15.191:3300/0 conn(0x7fe9dc0b15e0 0x7fe9dc0b46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:38.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.635+0000 7fe9db7fe700 1 -- 172.21.15.99:0/3220021929 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9dc0b2300 con 0x7fe9dc159ef0 2023-10-08T00:15:38.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.635+0000 7fe9db7fe700 1 --2- 172.21.15.99:0/3220021929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9dc159ef0 0x7fe9dc15a2c0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc0890b0 tx=0x7fe9cc009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed621de753ade10d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:38.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.635+0000 7fe9da7fc700 1 -- 172.21.15.99:0/3220021929 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc005c10 con 0x7fe9dc159ef0 2023-10-08T00:15:38.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.635+0000 7fe9da7fc700 1 -- 172.21.15.99:0/3220021929 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9cc005d70 con 0x7fe9dc159ef0 2023-10-08T00:15:38.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.635+0000 7fe9da7fc700 1 -- 172.21.15.99:0/3220021929 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc00b880 con 0x7fe9dc159ef0 2023-10-08T00:15:38.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.636+0000 7fe9e1a93700 1 -- 172.21.15.99:0/3220021929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9dc159ef0 msgr2=0x7fe9dc15a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:38.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.636+0000 7fe9e1a93700 1 --2- 172.21.15.99:0/3220021929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9dc159ef0 0x7fe9dc15a2c0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc0890b0 tx=0x7fe9cc009220 comp rx=0 tx=0).stop 2023-10-08T00:15:38.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.636+0000 7fe9e1a93700 1 -- 172.21.15.99:0/3220021929 shutdown_connections 2023-10-08T00:15:38.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.636+0000 7fe9e1a93700 1 --2- 172.21.15.99:0/3220021929 >> v2:172.21.15.191:3300/0 conn(0x7fe9dc0b15e0 0x7fe9dc0b46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:38.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.636+0000 7fe9e1a93700 1 --2- 172.21.15.99:0/3220021929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9dc159ef0 0x7fe9dc15a2c0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:38.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.636+0000 7fe9e1a93700 1 --2- 172.21.15.99:0/3220021929 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe9dc0b6a10 0x7fe9dc0b10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:38.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.636+0000 7fe9e1a93700 1 -- 172.21.15.99:0/3220021929 >> 172.21.15.99:0/3220021929 conn(0x7fe9dc09a0a0 msgr2=0x7fe9dc067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:38.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.636+0000 7fe9e1a93700 1 -- 172.21.15.99:0/3220021929 shutdown_connections 2023-10-08T00:15:38.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.636+0000 7fe9e1a93700 1 -- 172.21.15.99:0/3220021929 wait complete. 2023-10-08T00:15:38.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.637+0000 7fe9e1a93700 1 Processor -- start 2023-10-08T00:15:38.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.637+0000 7fe9e1a93700 1 -- start start 2023-10-08T00:15:38.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.638+0000 7fe9e1a93700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe9dc0b6a10 0x7fe9dc14f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:38.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.638+0000 7fe9e1a93700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe9dc159ef0 0x7fe9dc16d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:38.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.638+0000 7fe9e1a93700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9dc15e230 0x7fe9dc16db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:38.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.638+0000 7fe9e1a93700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc16e460 con 0x7fe9dc15e230 2023-10-08T00:15:38.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.638+0000 7fe9e1a93700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc15e6c0 con 0x7fe9dc0b6a10 2023-10-08T00:15:38.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.638+0000 7fe9e1a93700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc15e9a0 con 0x7fe9dc159ef0 2023-10-08T00:15:38.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.638+0000 7fe9dbfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9dc15e230 0x7fe9dc16db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:38.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.638+0000 7fe9dbfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9dc15e230 0x7fe9dc16db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51278/0 (socket says 172.21.15.99:51278) 2023-10-08T00:15:38.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.638+0000 7fe9dbfff700 1 -- 172.21.15.99:0/1820808769 learned_addr learned my addr 172.21.15.99:0/1820808769 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:38.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.638+0000 7fe9db7fe700 1 --2- 172.21.15.99:0/1820808769 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe9dc0b6a10 0x7fe9dc14f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:38.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.638+0000 7fe9daffd700 1 --2- 172.21.15.99:0/1820808769 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe9dc159ef0 0x7fe9dc16d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:38.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.638+0000 7fe9dbfff700 1 -- 172.21.15.99:0/1820808769 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe9dc159ef0 msgr2=0x7fe9dc16d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:38.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.638+0000 7fe9dbfff700 1 --2- 172.21.15.99:0/1820808769 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe9dc159ef0 0x7fe9dc16d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:38.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.638+0000 7fe9dbfff700 1 -- 172.21.15.99:0/1820808769 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe9dc0b6a10 msgr2=0x7fe9dc14f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:38.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.638+0000 7fe9dbfff700 1 --2- 172.21.15.99:0/1820808769 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe9dc0b6a10 0x7fe9dc14f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:38.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.638+0000 7fe9dbfff700 1 -- 172.21.15.99:0/1820808769 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9cc009480 con 0x7fe9dc15e230 2023-10-08T00:15:38.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.638+0000 7fe9dbfff700 1 --2- 172.21.15.99:0/1820808769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9dc15e230 0x7fe9dc16db10 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4007f30 tx=0x7fe9d40098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:38.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.639+0000 7fe9c37fe700 1 -- 172.21.15.99:0/1820808769 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d400eb90 con 0x7fe9dc15e230 2023-10-08T00:15:38.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.639+0000 7fe9c37fe700 1 -- 172.21.15.99:0/1820808769 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9d400ecf0 con 0x7fe9dc15e230 2023-10-08T00:15:38.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.639+0000 7fe9e1a93700 1 -- 172.21.15.99:0/1820808769 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9dc15ec80 con 0x7fe9dc15e230 2023-10-08T00:15:38.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.639+0000 7fe9c37fe700 1 -- 172.21.15.99:0/1820808769 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d4014670 con 0x7fe9dc15e230 2023-10-08T00:15:38.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.639+0000 7fe9e1a93700 1 -- 172.21.15.99:0/1820808769 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9dc14fb20 con 0x7fe9dc15e230 2023-10-08T00:15:38.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.640+0000 7fe9e1a93700 1 -- 172.21.15.99:0/1820808769 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9dc04c3c0 con 0x7fe9dc15e230 2023-10-08T00:15:38.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.640+0000 7fe9c37fe700 1 -- 172.21.15.99:0/1820808769 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe9d4014a20 con 0x7fe9dc15e230 2023-10-08T00:15:38.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.640+0000 7fe9c37fe700 1 --2- 172.21.15.99:0/1820808769 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe9c8066f30 0x7fe9c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:38.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.640+0000 7fe9c37fe700 1 -- 172.21.15.99:0/1820808769 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fe9d4084820 con 0x7fe9dc15e230 2023-10-08T00:15:38.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.640+0000 7fe9db7fe700 1 --2- 172.21.15.99:0/1820808769 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe9c8066f30 0x7fe9c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:38.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.641+0000 7fe9db7fe700 1 --2- 172.21.15.99:0/1820808769 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe9c8066f30 0x7fe9c80693e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc009260 tx=0x7fe9cc016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:38.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.644+0000 7fe9c37fe700 1 -- 172.21.15.99:0/1820808769 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe9d4055870 con 0x7fe9dc15e230 2023-10-08T00:15:38.830 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:38.830+0000 7fe9e1a93700 1 -- 172.21.15.99:0/1820808769 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe9dc0ab0f0 con 0x7fe9dc15e230 2023-10-08T00:15:39.212 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.212+0000 7fe9c37fe700 1 -- 172.21.15.99:0/1820808769 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v101) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe9d4009a20 con 0x7fe9dc15e230 2023-10-08T00:15:39.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.215+0000 7fe9e1a93700 1 -- 172.21.15.99:0/1820808769 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe9c8066f30 msgr2=0x7fe9c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:39.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.215+0000 7fe9e1a93700 1 --2- 172.21.15.99:0/1820808769 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe9c8066f30 0x7fe9c80693e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc009260 tx=0x7fe9cc016000 comp rx=0 tx=0).stop 2023-10-08T00:15:39.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.215+0000 7fe9e1a93700 1 -- 172.21.15.99:0/1820808769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9dc15e230 msgr2=0x7fe9dc16db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:39.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.215+0000 7fe9e1a93700 1 --2- 172.21.15.99:0/1820808769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9dc15e230 0x7fe9dc16db10 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4007f30 tx=0x7fe9d40098e0 comp rx=0 tx=0).stop 2023-10-08T00:15:39.216 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.215+0000 7fe9e1a93700 1 -- 172.21.15.99:0/1820808769 shutdown_connections 2023-10-08T00:15:39.216 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.215+0000 7fe9e1a93700 1 --2- 172.21.15.99:0/1820808769 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe9c8066f30 0x7fe9c80693e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:39.216 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.215+0000 7fe9e1a93700 1 --2- 172.21.15.99:0/1820808769 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe9dc0b6a10 0x7fe9dc14f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:39.216 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.215+0000 7fe9e1a93700 1 --2- 172.21.15.99:0/1820808769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9dc15e230 0x7fe9dc16db10 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:39.216 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.215+0000 7fe9e1a93700 1 --2- 172.21.15.99:0/1820808769 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe9dc159ef0 0x7fe9dc16d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:39.216 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.215+0000 7fe9e1a93700 1 -- 172.21.15.99:0/1820808769 >> 172.21.15.99:0/1820808769 conn(0x7fe9dc09a0a0 msgr2=0x7fe9dc0b2660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:39.216 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.215+0000 7fe9e1a93700 1 -- 172.21.15.99:0/1820808769 shutdown_connections 2023-10-08T00:15:39.216 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.215+0000 7fe9e1a93700 1 -- 172.21.15.99:0/1820808769 wait complete. 2023-10-08T00:15:39.218 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' created 2023-10-08T00:15:39.230 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T00:15:39.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.351+0000 7fc7a693c700 1 Processor -- start 2023-10-08T00:15:39.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.351+0000 7fc7a693c700 1 -- start start 2023-10-08T00:15:39.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.352+0000 7fc7a693c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc7a0156460 0x7fc7a00cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:39.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.352+0000 7fc7a693c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7a006e040 0x7fc7a00aa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:39.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.352+0000 7fc7a693c700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7a00cc110 con 0x7fc7a00cbc20 2023-10-08T00:15:39.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.352+0000 7fc7a693c700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7a0156830 con 0x7fc7a0156460 2023-10-08T00:15:39.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.352+0000 7fc7a693c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a00cc6d0 con 0x7fc7a006e040 2023-10-08T00:15:39.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.352+0000 7fc79ffff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc7a0156460 0x7fc7a00cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:39.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.352+0000 7fc79f7fe700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fc7a00cbc20 0x7fc7a006d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45352/0 (socket says 172.21.15.99:45352) 2023-10-08T00:15:39.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.352+0000 7fc79f7fe700 1 -- 172.21.15.99:0/3443626424 learned_addr learned my addr 172.21.15.99:0/3443626424 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:15:39.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.353+0000 7fc79ffff700 1 -- 172.21.15.99:0/3443626424 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7a006e040 msgr2=0x7fc7a00aa6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:39.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.353+0000 7fc7a4ed9700 1 --2- 172.21.15.99:0/3443626424 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7a006e040 0x7fc7a00aa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:39.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.353+0000 7fc79ffff700 1 --2- 172.21.15.99:0/3443626424 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7a006e040 0x7fc7a00aa6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:39.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.353+0000 7fc79ffff700 1 -- 172.21.15.99:0/3443626424 >> v1:172.21.15.99:6789/0 conn(0x7fc7a00cbc20 legacy=0x7fc7a006d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:39.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.353+0000 7fc79effd700 1 -- 172.21.15.99:0/3443626424 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 611403998 0 0) 0x7fc7a00cc110 con 0x7fc7a00cbc20 2023-10-08T00:15:39.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.353+0000 7fc79ffff700 1 -- 172.21.15.99:0/3443626424 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a00cc3f0 con 0x7fc7a0156460 2023-10-08T00:15:39.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.353+0000 7fc7a4ed9700 1 --2- 172.21.15.99:0/3443626424 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7a006e040 0x7fc7a00aa6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:15:39.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.353+0000 7fc79ffff700 1 --2- 172.21.15.99:0/3443626424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7a0156460 0x7fc7a00cb6e0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fc7900098e0 tx=0x7fc7900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a16a0cffed63eb8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:39.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.353+0000 7fc79effd700 1 -- 172.21.15.99:0/3443626424 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc790009e80 con 0x7fc7a0156460 2023-10-08T00:15:39.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.353+0000 7fc79effd700 1 -- 172.21.15.99:0/3443626424 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc79000b860 con 0x7fc7a0156460 2023-10-08T00:15:39.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.353+0000 7fc79effd700 1 -- 172.21.15.99:0/3443626424 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79000bbe0 con 0x7fc7a0156460 2023-10-08T00:15:39.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.353+0000 7fc7a693c700 1 -- 172.21.15.99:0/3443626424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7a0156460 msgr2=0x7fc7a00cb6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:39.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.353+0000 7fc7a693c700 1 --2- 172.21.15.99:0/3443626424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7a0156460 0x7fc7a00cb6e0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fc7900098e0 tx=0x7fc7900091a0 comp rx=0 tx=0).stop 2023-10-08T00:15:39.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.354+0000 7fc7a693c700 1 -- 172.21.15.99:0/3443626424 shutdown_connections 2023-10-08T00:15:39.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.354+0000 7fc7a693c700 1 --2- 172.21.15.99:0/3443626424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7a0156460 0x7fc7a00cb6e0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:39.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.354+0000 7fc7a693c700 1 --2- 172.21.15.99:0/3443626424 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7a006e040 0x7fc7a00aa6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:39.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.354+0000 7fc7a693c700 1 -- 172.21.15.99:0/3443626424 >> 172.21.15.99:0/3443626424 conn(0x7fc7a0068170 msgr2=0x7fc7a00cd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:39.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.354+0000 7fc7a693c700 1 -- 172.21.15.99:0/3443626424 shutdown_connections 2023-10-08T00:15:39.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.354+0000 7fc7a693c700 1 -- 172.21.15.99:0/3443626424 wait complete. 2023-10-08T00:15:39.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.355+0000 7fc7a693c700 1 Processor -- start 2023-10-08T00:15:39.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.355+0000 7fc7a693c700 1 -- start start 2023-10-08T00:15:39.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.356+0000 7fc7a693c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7a006e040 0x7fc7a01526c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:39.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.356+0000 7fc7a693c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7a00cbc20 0x7fc7a00c8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:39.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.356+0000 7fc7a693c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7a0156460 0x7fc7a00c9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:39.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.356+0000 7fc7a693c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a01575b0 con 0x7fc7a00cbc20 2023-10-08T00:15:39.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.356+0000 7fc7a693c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a0156a10 con 0x7fc7a0156460 2023-10-08T00:15:39.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.356+0000 7fc7a693c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a0156c80 con 0x7fc7a006e040 2023-10-08T00:15:39.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.356+0000 7fc79f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7a00cbc20 0x7fc7a00c8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:39.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.356+0000 7fc79f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7a00cbc20 0x7fc7a00c8bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51284/0 (socket says 172.21.15.99:51284) 2023-10-08T00:15:39.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.356+0000 7fc79f7fe700 1 -- 172.21.15.99:0/2393670023 learned_addr learned my addr 172.21.15.99:0/2393670023 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:39.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.356+0000 7fc79ffff700 1 --2- 172.21.15.99:0/2393670023 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7a006e040 0x7fc7a01526c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:39.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.356+0000 7fc7a4ed9700 1 --2- 172.21.15.99:0/2393670023 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7a0156460 0x7fc7a00c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:39.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.356+0000 7fc79f7fe700 1 -- 172.21.15.99:0/2393670023 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7a006e040 msgr2=0x7fc7a01526c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:39.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.356+0000 7fc79f7fe700 1 --2- 172.21.15.99:0/2393670023 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7a006e040 0x7fc7a01526c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:39.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.356+0000 7fc79f7fe700 1 -- 172.21.15.99:0/2393670023 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7a0156460 msgr2=0x7fc7a00c9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:39.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.356+0000 7fc79f7fe700 1 --2- 172.21.15.99:0/2393670023 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7a0156460 0x7fc7a00c9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:39.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.356+0000 7fc79f7fe700 1 -- 172.21.15.99:0/2393670023 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc790009420 con 0x7fc7a00cbc20 2023-10-08T00:15:39.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.356+0000 7fc79f7fe700 1 --2- 172.21.15.99:0/2393670023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7a00cbc20 0x7fc7a00c8bd0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fc79400c810 tx=0x7fc794007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:39.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.356+0000 7fc787fff700 1 -- 172.21.15.99:0/2393670023 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc794005c00 con 0x7fc7a00cbc20 2023-10-08T00:15:39.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.356+0000 7fc787fff700 1 -- 172.21.15.99:0/2393670023 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc794005d60 con 0x7fc7a00cbc20 2023-10-08T00:15:39.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.356+0000 7fc7a693c700 1 -- 172.21.15.99:0/2393670023 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7a0156f60 con 0x7fc7a00cbc20 2023-10-08T00:15:39.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.356+0000 7fc787fff700 1 -- 172.21.15.99:0/2393670023 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc794007320 con 0x7fc7a00cbc20 2023-10-08T00:15:39.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.356+0000 7fc7a693c700 1 -- 172.21.15.99:0/2393670023 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7a0152c00 con 0x7fc7a00cbc20 2023-10-08T00:15:39.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.357+0000 7fc7a693c700 1 -- 172.21.15.99:0/2393670023 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7a00ac750 con 0x7fc7a00cbc20 2023-10-08T00:15:39.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.358+0000 7fc787fff700 1 -- 172.21.15.99:0/2393670023 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc79401b400 con 0x7fc7a00cbc20 2023-10-08T00:15:39.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.358+0000 7fc787fff700 1 --2- 172.21.15.99:0/2393670023 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc78c066fb0 0x7fc78c069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:39.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.358+0000 7fc787fff700 1 -- 172.21.15.99:0/2393670023 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5223+0+0 (secure 0 0 0) 0x7fc794083ed0 con 0x7fc7a00cbc20 2023-10-08T00:15:39.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.358+0000 7fc79ffff700 1 --2- 172.21.15.99:0/2393670023 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc78c066fb0 0x7fc78c069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:39.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.359+0000 7fc79ffff700 1 --2- 172.21.15.99:0/2393670023 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc78c066fb0 0x7fc78c069460 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc7900091e0 tx=0x7fc790017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:39.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.363+0000 7fc787fff700 1 -- 172.21.15.99:0/2393670023 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc794051300 con 0x7fc7a00cbc20 2023-10-08T00:15:39.544 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:39.543+0000 7fc7a693c700 1 -- 172.21.15.99:0/2393670023 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fc7a004c3c0 con 0x7fc7a00cbc20 2023-10-08T00:15:40.218 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.217+0000 7fc787fff700 1 -- 172.21.15.99:0/2393670023 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v102) v1 ==== 170+0+0 (secure 0 0 0) 0x7fc794054920 con 0x7fc7a00cbc20 2023-10-08T00:15:40.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.221+0000 7fc7a693c700 1 -- 172.21.15.99:0/2393670023 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc78c066fb0 msgr2=0x7fc78c069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:40.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.221+0000 7fc7a693c700 1 --2- 172.21.15.99:0/2393670023 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc78c066fb0 0x7fc78c069460 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc7900091e0 tx=0x7fc790017000 comp rx=0 tx=0).stop 2023-10-08T00:15:40.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.221+0000 7fc7a693c700 1 -- 172.21.15.99:0/2393670023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7a00cbc20 msgr2=0x7fc7a00c8bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:40.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.221+0000 7fc7a693c700 1 --2- 172.21.15.99:0/2393670023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7a00cbc20 0x7fc7a00c8bd0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fc79400c810 tx=0x7fc794007db0 comp rx=0 tx=0).stop 2023-10-08T00:15:40.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.222+0000 7fc7a693c700 1 -- 172.21.15.99:0/2393670023 shutdown_connections 2023-10-08T00:15:40.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.222+0000 7fc7a693c700 1 --2- 172.21.15.99:0/2393670023 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc78c066fb0 0x7fc78c069460 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:40.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.222+0000 7fc7a693c700 1 --2- 172.21.15.99:0/2393670023 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7a0156460 0x7fc7a00c9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:40.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.222+0000 7fc7a693c700 1 --2- 172.21.15.99:0/2393670023 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7a006e040 0x7fc7a01526c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:40.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.222+0000 7fc7a693c700 1 --2- 172.21.15.99:0/2393670023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7a00cbc20 0x7fc7a00c8bd0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:40.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.222+0000 7fc7a693c700 1 -- 172.21.15.99:0/2393670023 >> 172.21.15.99:0/2393670023 conn(0x7fc7a0068170 msgr2=0x7fc7a00aead0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:40.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.223+0000 7fc7a693c700 1 -- 172.21.15.99:0/2393670023 shutdown_connections 2023-10-08T00:15:40.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.224+0000 7fc7a693c700 1 -- 172.21.15.99:0/2393670023 wait complete. 2023-10-08T00:15:40.228 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' created 2023-10-08T00:15:40.242 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T00:15:40.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.361+0000 7f0abc790700 1 Processor -- start 2023-10-08T00:15:40.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.361+0000 7f0abc790700 1 -- start start 2023-10-08T00:15:40.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.362+0000 7f0abc790700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0ab41569c0 0x7f0ab40a9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:40.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.362+0000 7f0abc790700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ab406ea00 0x7f0ab40cbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:40.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.362+0000 7f0abc790700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ab40cdb90 con 0x7f0ab4156d90 2023-10-08T00:15:40.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.362+0000 7f0abc790700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ab40c78c0 con 0x7f0ab41569c0 2023-10-08T00:15:40.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.362+0000 7f0abc790700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab40c7ba0 con 0x7f0ab406ea00 2023-10-08T00:15:40.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.362+0000 7f0aba52c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0ab41569c0 0x7f0ab40a9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:40.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.362+0000 7f0aba52c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0ab41569c0 0x7f0ab40a9ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51290/0 (socket says 172.21.15.99:51290) 2023-10-08T00:15:40.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.362+0000 7f0aba52c700 1 -- 172.21.15.99:0/870711826 learned_addr learned my addr 172.21.15.99:0/870711826 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:40.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.362+0000 7f0aba52c700 1 -- 172.21.15.99:0/870711826 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ab406ea00 msgr2=0x7f0ab40cbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:40.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.362+0000 7f0aba52c700 1 --2- 172.21.15.99:0/870711826 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ab406ea00 0x7f0ab40cbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:40.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.362+0000 7f0aba52c700 1 -- 172.21.15.99:0/870711826 >> v1:172.21.15.191:6789/0 conn(0x7f0ab4156d90 legacy=0x7f0ab406e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:40.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.362+0000 7f0aba52c700 1 -- 172.21.15.99:0/870711826 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab40c7e80 con 0x7f0ab41569c0 2023-10-08T00:15:40.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.362+0000 7f0aba52c700 1 --2- 172.21.15.99:0/870711826 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ab41569c0 0x7f0ab40a9ff0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f0ab00099e0 tx=0x7f0ab0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26f9abfd093d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:40.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.362+0000 7f0ab952a700 1 -- 172.21.15.99:0/870711826 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab000b7e0 con 0x7f0ab41569c0 2023-10-08T00:15:40.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.362+0000 7f0ab952a700 1 -- 172.21.15.99:0/870711826 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab000b940 con 0x7f0ab41569c0 2023-10-08T00:15:40.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.362+0000 7f0ab952a700 1 -- 172.21.15.99:0/870711826 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab000bce0 con 0x7f0ab41569c0 2023-10-08T00:15:40.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.363+0000 7f0abc790700 1 -- 172.21.15.99:0/870711826 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ab41569c0 msgr2=0x7f0ab40a9ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:40.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.363+0000 7f0abc790700 1 --2- 172.21.15.99:0/870711826 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ab41569c0 0x7f0ab40a9ff0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f0ab00099e0 tx=0x7f0ab0009250 comp rx=0 tx=0).stop 2023-10-08T00:15:40.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.363+0000 7f0abc790700 1 -- 172.21.15.99:0/870711826 shutdown_connections 2023-10-08T00:15:40.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.363+0000 7f0abc790700 1 --2- 172.21.15.99:0/870711826 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ab406ea00 0x7f0ab40cbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:40.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.363+0000 7f0abc790700 1 --2- 172.21.15.99:0/870711826 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ab41569c0 0x7f0ab40a9ff0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:40.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.363+0000 7f0abc790700 1 -- 172.21.15.99:0/870711826 >> 172.21.15.99:0/870711826 conn(0x7f0ab4068170 msgr2=0x7f0ab40cd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:40.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.363+0000 7f0abc790700 1 -- 172.21.15.99:0/870711826 shutdown_connections 2023-10-08T00:15:40.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.363+0000 7f0abc790700 1 -- 172.21.15.99:0/870711826 wait complete. 2023-10-08T00:15:40.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.364+0000 7f0abc790700 1 Processor -- start 2023-10-08T00:15:40.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.364+0000 7f0abc790700 1 -- start start 2023-10-08T00:15:40.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.365+0000 7f0abc790700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ab406ea00 0x7f0ab4154a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:40.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.365+0000 7f0abc790700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ab41569c0 0x7f0ab40c3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:40.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.365+0000 7f0abc790700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ab4156d90 0x7f0ab40c3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:40.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.365+0000 7f0abc790700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab4154f50 con 0x7f0ab4156d90 2023-10-08T00:15:40.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.365+0000 7f0abc790700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab40c5850 con 0x7f0ab406ea00 2023-10-08T00:15:40.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.365+0000 7f0abc790700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab40c4c40 con 0x7f0ab41569c0 2023-10-08T00:15:40.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.365+0000 7f0abad2d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ab4156d90 0x7f0ab40c3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:40.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.365+0000 7f0abad2d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ab4156d90 0x7f0ab40c3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51298/0 (socket says 172.21.15.99:51298) 2023-10-08T00:15:40.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.365+0000 7f0abad2d700 1 -- 172.21.15.99:0/623036079 learned_addr learned my addr 172.21.15.99:0/623036079 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:40.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.365+0000 7f0ab9d2b700 1 --2- 172.21.15.99:0/623036079 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ab41569c0 0x7f0ab40c3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:40.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.365+0000 7f0aba52c700 1 --2- 172.21.15.99:0/623036079 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ab406ea00 0x7f0ab4154a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:40.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.365+0000 7f0abad2d700 1 -- 172.21.15.99:0/623036079 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ab41569c0 msgr2=0x7f0ab40c3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:40.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.365+0000 7f0abad2d700 1 --2- 172.21.15.99:0/623036079 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ab41569c0 0x7f0ab40c3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:40.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.365+0000 7f0abad2d700 1 -- 172.21.15.99:0/623036079 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ab406ea00 msgr2=0x7f0ab4154a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:40.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.365+0000 7f0abad2d700 1 --2- 172.21.15.99:0/623036079 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ab406ea00 0x7f0ab4154a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:40.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.365+0000 7f0abad2d700 1 -- 172.21.15.99:0/623036079 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab00094f0 con 0x7f0ab4156d90 2023-10-08T00:15:40.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.365+0000 7f0abad2d700 1 --2- 172.21.15.99:0/623036079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ab4156d90 0x7f0ab40c3650 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f0aac00bd70 tx=0x7f0aac00ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:40.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.365+0000 7f0aaa7fc700 1 -- 172.21.15.99:0/623036079 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac00b2e0 con 0x7f0ab4156d90 2023-10-08T00:15:40.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.365+0000 7f0aaa7fc700 1 -- 172.21.15.99:0/623036079 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aac00b460 con 0x7f0ab4156d90 2023-10-08T00:15:40.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.365+0000 7f0abc790700 1 -- 172.21.15.99:0/623036079 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ab40c4f20 con 0x7f0ab4156d90 2023-10-08T00:15:40.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.365+0000 7f0aaa7fc700 1 -- 172.21.15.99:0/623036079 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac00ebe0 con 0x7f0ab4156d90 2023-10-08T00:15:40.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.365+0000 7f0abc790700 1 -- 172.21.15.99:0/623036079 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ab40c54e0 con 0x7f0ab4156d90 2023-10-08T00:15:40.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.366+0000 7f0abc790700 1 -- 172.21.15.99:0/623036079 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ab404c3c0 con 0x7f0ab4156d90 2023-10-08T00:15:40.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.367+0000 7f0aaa7fc700 1 -- 172.21.15.99:0/623036079 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0aac014070 con 0x7f0ab4156d90 2023-10-08T00:15:40.371 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.367+0000 7f0aaa7fc700 1 --2- 172.21.15.99:0/623036079 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0a98066f30 0x7f0a980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:40.371 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.367+0000 7f0aaa7fc700 1 -- 172.21.15.99:0/623036079 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f0aac086350 con 0x7f0ab4156d90 2023-10-08T00:15:40.371 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.367+0000 7f0aba52c700 1 --2- 172.21.15.99:0/623036079 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0a98066f30 0x7f0a980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:40.371 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.368+0000 7f0aba52c700 1 --2- 172.21.15.99:0/623036079 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0a98066f30 0x7f0a980693e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0ab00099e0 tx=0x7f0ab0016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:40.372 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.371+0000 7f0aaa7fc700 1 -- 172.21.15.99:0/623036079 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0aac053620 con 0x7f0ab4156d90 2023-10-08T00:15:40.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:40.593+0000 7f0abc790700 1 -- 172.21.15.99:0/623036079 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f0ab40c8160 con 0x7f0ab4156d90 2023-10-08T00:15:41.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.224+0000 7f0aaa7fc700 1 -- 172.21.15.99:0/623036079 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v44) v1 ==== 174+0+0 (secure 0 0 0) 0x7f0aac056c40 con 0x7f0ab4156d90 2023-10-08T00:15:41.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.227+0000 7f0abc790700 1 -- 172.21.15.99:0/623036079 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0a98066f30 msgr2=0x7f0a980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:41.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.227+0000 7f0abc790700 1 --2- 172.21.15.99:0/623036079 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0a98066f30 0x7f0a980693e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0ab00099e0 tx=0x7f0ab0016000 comp rx=0 tx=0).stop 2023-10-08T00:15:41.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.227+0000 7f0abc790700 1 -- 172.21.15.99:0/623036079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ab4156d90 msgr2=0x7f0ab40c3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:41.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.227+0000 7f0abc790700 1 --2- 172.21.15.99:0/623036079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ab4156d90 0x7f0ab40c3650 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f0aac00bd70 tx=0x7f0aac00ba20 comp rx=0 tx=0).stop 2023-10-08T00:15:41.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.228+0000 7f0abc790700 1 -- 172.21.15.99:0/623036079 shutdown_connections 2023-10-08T00:15:41.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.228+0000 7f0abc790700 1 --2- 172.21.15.99:0/623036079 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0a98066f30 0x7f0a980693e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:41.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.228+0000 7f0abc790700 1 --2- 172.21.15.99:0/623036079 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ab406ea00 0x7f0ab4154a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:41.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.228+0000 7f0abc790700 1 --2- 172.21.15.99:0/623036079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ab4156d90 0x7f0ab40c3650 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:41.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.228+0000 7f0abc790700 1 --2- 172.21.15.99:0/623036079 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ab41569c0 0x7f0ab40c3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:41.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.228+0000 7f0abc790700 1 -- 172.21.15.99:0/623036079 >> 172.21.15.99:0/623036079 conn(0x7f0ab4068170 msgr2=0x7f0ab406a920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:41.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.228+0000 7f0abc790700 1 -- 172.21.15.99:0/623036079 shutdown_connections 2023-10-08T00:15:41.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.228+0000 7f0abc790700 1 -- 172.21.15.99:0/623036079 wait complete. 2023-10-08T00:15:41.233 INFO:teuthology.orchestra.run.smithi099.stderr:new fs with metadata pool 12 and data pool 13 2023-10-08T00:15:41.242 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:15:41.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.356+0000 7ffbed76c700 1 Processor -- start 2023-10-08T00:15:41.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.356+0000 7ffbed76c700 1 -- start start 2023-10-08T00:15:41.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.357+0000 7ffbed76c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffbe81523e0 0x7ffbe806c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:41.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.357+0000 7ffbed76c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ffbe8158e50 0x7ffbe80cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:41.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.357+0000 7ffbed76c700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbe806c010 con 0x7ffbe8152010 2023-10-08T00:15:41.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.357+0000 7ffbed76c700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbe806bd10 con 0x7ffbe8158e50 2023-10-08T00:15:41.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.357+0000 7ffbed76c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe815c060 con 0x7ffbe81523e0 2023-10-08T00:15:41.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.357+0000 7ffbe6ffd700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7ffbe8152010 0x7ffbe815b810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45366/0 (socket says 172.21.15.99:45366) 2023-10-08T00:15:41.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.357+0000 7ffbe6ffd700 1 -- 172.21.15.99:0/1301428704 learned_addr learned my addr 172.21.15.99:0/1301428704 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:15:41.361 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.358+0000 7ffbe67fc700 1 -- 172.21.15.99:0/1301428704 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2450033702 0 0) 0x7ffbe806c010 con 0x7ffbe8152010 2023-10-08T00:15:41.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.358+0000 7ffbe67fc700 1 -- 172.21.15.99:0/1301428704 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffbd4003570 con 0x7ffbe8152010 2023-10-08T00:15:41.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.358+0000 7ffbde7fc700 1 --2- 172.21.15.99:0/1301428704 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffbe81523e0 0x7ffbe806c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:41.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.358+0000 7ffbe77fe700 1 --2- 172.21.15.99:0/1301428704 >> v2:172.21.15.191:3300/0 conn(0x7ffbe8158e50 0x7ffbe80cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:41.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.358+0000 7ffbe67fc700 1 -- 172.21.15.99:0/1301428704 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3342504340 0 0) 0x7ffbd4003570 con 0x7ffbe8152010 2023-10-08T00:15:41.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.358+0000 7ffbe67fc700 1 -- 172.21.15.99:0/1301428704 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffbe81523e0 msgr2=0x7ffbe806c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:41.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.358+0000 7ffbe67fc700 1 --2- 172.21.15.99:0/1301428704 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffbe81523e0 0x7ffbe806c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:41.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.358+0000 7ffbe67fc700 1 -- 172.21.15.99:0/1301428704 >> v2:172.21.15.191:3300/0 conn(0x7ffbe8158e50 msgr2=0x7ffbe80cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:41.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.358+0000 7ffbe67fc700 1 --2- 172.21.15.99:0/1301428704 >> v2:172.21.15.191:3300/0 conn(0x7ffbe8158e50 0x7ffbe80cdf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:41.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.358+0000 7ffbe67fc700 1 -- 172.21.15.99:0/1301428704 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbe815c340 con 0x7ffbe8152010 2023-10-08T00:15:41.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.358+0000 7ffbe67fc700 1 -- 172.21.15.99:0/1301428704 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7ffbd8001470 con 0x7ffbe8152010 2023-10-08T00:15:41.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.358+0000 7ffbe67fc700 1 -- 172.21.15.99:0/1301428704 >> v1:172.21.15.99:6789/0 conn(0x7ffbe8152010 legacy=0x7ffbe815b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:41.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.358+0000 7ffbe67fc700 1 --2- 172.21.15.99:0/1301428704 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffbd4003a60 0x7ffbd4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:41.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.358+0000 7ffbe67fc700 1 --2- 172.21.15.99:0/1301428704 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffbe81523e0 0x7ffbd4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:41.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.358+0000 7ffbe67fc700 1 --2- 172.21.15.99:0/1301428704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbd4004d50 0x7ffbd40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:41.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.358+0000 7ffbe67fc700 1 -- 172.21.15.99:0/1301428704 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe806bd10 con 0x7ffbd4004d50 2023-10-08T00:15:41.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.358+0000 7ffbe67fc700 1 -- 172.21.15.99:0/1301428704 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe815c060 con 0x7ffbd4003a60 2023-10-08T00:15:41.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.358+0000 7ffbe67fc700 1 -- 172.21.15.99:0/1301428704 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd400ba60 con 0x7ffbe81523e0 2023-10-08T00:15:41.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.358+0000 7ffbe77fe700 1 --2- 172.21.15.99:0/1301428704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbd4004d50 0x7ffbd40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:41.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.358+0000 7ffbe6ffd700 1 --2- 172.21.15.99:0/1301428704 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffbd4003a60 0x7ffbd4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:41.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.358+0000 7ffbde7fc700 1 --2- 172.21.15.99:0/1301428704 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffbe81523e0 0x7ffbd4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:41.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.358+0000 7ffbe77fe700 1 -- 172.21.15.99:0/1301428704 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffbe81523e0 msgr2=0x7ffbd4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:41.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.358+0000 7ffbe77fe700 1 --2- 172.21.15.99:0/1301428704 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffbe81523e0 0x7ffbd4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:41.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.358+0000 7ffbe77fe700 1 -- 172.21.15.99:0/1301428704 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffbd4003a60 msgr2=0x7ffbd4003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:41.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.358+0000 7ffbe77fe700 1 --2- 172.21.15.99:0/1301428704 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffbd4003a60 0x7ffbd4003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:41.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.358+0000 7ffbe77fe700 1 -- 172.21.15.99:0/1301428704 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd4003570 con 0x7ffbd4004d50 2023-10-08T00:15:41.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.359+0000 7ffbe77fe700 1 --2- 172.21.15.99:0/1301428704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbd4004d50 0x7ffbd40051c0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7ffbe000e210 tx=0x7ffbe000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7effd2a79eeb7a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:41.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.359+0000 7ffbe67fc700 1 -- 172.21.15.99:0/1301428704 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe000ea60 con 0x7ffbd4004d50 2023-10-08T00:15:41.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.359+0000 7ffbe67fc700 1 -- 172.21.15.99:0/1301428704 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbe000f040 con 0x7ffbd4004d50 2023-10-08T00:15:41.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.359+0000 7ffbe67fc700 1 -- 172.21.15.99:0/1301428704 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe000ee10 con 0x7ffbd4004d50 2023-10-08T00:15:41.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.359+0000 7ffbed76c700 1 -- 172.21.15.99:0/1301428704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbd4004d50 msgr2=0x7ffbd40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:41.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.359+0000 7ffbed76c700 1 --2- 172.21.15.99:0/1301428704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbd4004d50 0x7ffbd40051c0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7ffbe000e210 tx=0x7ffbe000b040 comp rx=0 tx=0).stop 2023-10-08T00:15:41.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.359+0000 7ffbe77fe700 1 -- 172.21.15.99:0/1301428704 reap_dead start 2023-10-08T00:15:41.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.359+0000 7ffbed76c700 1 -- 172.21.15.99:0/1301428704 shutdown_connections 2023-10-08T00:15:41.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.359+0000 7ffbed76c700 1 -- 172.21.15.99:0/1301428704 >> 172.21.15.99:0/1301428704 conn(0x7ffbe8069870 msgr2=0x7ffbe8069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:41.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.359+0000 7ffbed76c700 1 -- 172.21.15.99:0/1301428704 shutdown_connections 2023-10-08T00:15:41.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.359+0000 7ffbed76c700 1 -- 172.21.15.99:0/1301428704 wait complete. 2023-10-08T00:15:41.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.360+0000 7ffbed76c700 1 Processor -- start 2023-10-08T00:15:41.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.361+0000 7ffbed76c700 1 -- start start 2023-10-08T00:15:41.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.361+0000 7ffbed76c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffbe8167160 0x7ffbe80c6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:41.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.361+0000 7ffbed76c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffbe8167530 0x7ffbe8168420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:41.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.361+0000 7ffbed76c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbe80c80f0 0x7ffbe80c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:41.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.361+0000 7ffbed76c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe8168960 con 0x7ffbe80c80f0 2023-10-08T00:15:41.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.361+0000 7ffbed76c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe80b26c0 con 0x7ffbe8167530 2023-10-08T00:15:41.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.361+0000 7ffbed76c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe8157100 con 0x7ffbe8167160 2023-10-08T00:15:41.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.361+0000 7ffbe77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbe80c80f0 0x7ffbe80c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:41.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.361+0000 7ffbe77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbe80c80f0 0x7ffbe80c8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51320/0 (socket says 172.21.15.99:51320) 2023-10-08T00:15:41.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.361+0000 7ffbe77fe700 1 -- 172.21.15.99:0/3053201883 learned_addr learned my addr 172.21.15.99:0/3053201883 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:41.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.362+0000 7ffbde7fc700 1 --2- 172.21.15.99:0/3053201883 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffbe8167530 0x7ffbe8168420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:41.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.362+0000 7ffbe6ffd700 1 --2- 172.21.15.99:0/3053201883 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffbe8167160 0x7ffbe80c6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:41.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.362+0000 7ffbe77fe700 1 -- 172.21.15.99:0/3053201883 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffbe8167160 msgr2=0x7ffbe80c6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:41.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.362+0000 7ffbe77fe700 1 --2- 172.21.15.99:0/3053201883 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffbe8167160 0x7ffbe80c6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:41.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.362+0000 7ffbe77fe700 1 -- 172.21.15.99:0/3053201883 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffbe8167530 msgr2=0x7ffbe8168420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:41.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.362+0000 7ffbe77fe700 1 --2- 172.21.15.99:0/3053201883 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffbe8167530 0x7ffbe8168420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:41.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.362+0000 7ffbe77fe700 1 -- 172.21.15.99:0/3053201883 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbe80b2960 con 0x7ffbe80c80f0 2023-10-08T00:15:41.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.362+0000 7ffbde7fc700 1 --2- 172.21.15.99:0/3053201883 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffbe8167530 0x7ffbe8168420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:15:41.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.362+0000 7ffbe77fe700 1 --2- 172.21.15.99:0/3053201883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbe80c80f0 0x7ffbe80c8560 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7ffbe0016000 tx=0x7ffbe001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:41.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.362+0000 7ffbdf7fe700 1 -- 172.21.15.99:0/3053201883 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe001d630 con 0x7ffbe80c80f0 2023-10-08T00:15:41.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.362+0000 7ffbdf7fe700 1 -- 172.21.15.99:0/3053201883 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbe001d790 con 0x7ffbe80c80f0 2023-10-08T00:15:41.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.362+0000 7ffbed76c700 1 -- 172.21.15.99:0/3053201883 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbe80b2c40 con 0x7ffbe80c80f0 2023-10-08T00:15:41.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.362+0000 7ffbdf7fe700 1 -- 172.21.15.99:0/3053201883 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe001db30 con 0x7ffbe80c80f0 2023-10-08T00:15:41.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.362+0000 7ffbed76c700 1 -- 172.21.15.99:0/3053201883 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbe80c6b70 con 0x7ffbe80c80f0 2023-10-08T00:15:41.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.363+0000 7ffbed76c700 1 -- 172.21.15.99:0/3053201883 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbe804c3c0 con 0x7ffbe80c80f0 2023-10-08T00:15:41.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.363+0000 7ffbdf7fe700 1 -- 172.21.15.99:0/3053201883 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffbe0003040 con 0x7ffbe80c80f0 2023-10-08T00:15:41.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.364+0000 7ffbdf7fe700 1 --2- 172.21.15.99:0/3053201883 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ffbc8066f30 0x7ffbc80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:41.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.364+0000 7ffbdf7fe700 1 -- 172.21.15.99:0/3053201883 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ffbe0085ae0 con 0x7ffbe80c80f0 2023-10-08T00:15:41.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.364+0000 7ffbe6ffd700 1 --2- 172.21.15.99:0/3053201883 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ffbc8066f30 0x7ffbc80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:41.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.365+0000 7ffbe6ffd700 1 --2- 172.21.15.99:0/3053201883 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ffbc8066f30 0x7ffbc80693e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8000e10 tx=0x7ffbd8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:41.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.368+0000 7ffbdf7fe700 1 -- 172.21.15.99:0/3053201883 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffbe0052d50 con 0x7ffbe80c80f0 2023-10-08T00:15:41.551 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.550+0000 7ffbed76c700 1 -- 172.21.15.99:0/3053201883 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffbe806d1b0 con 0x7ffbe80c80f0 2023-10-08T00:15:41.551 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.551+0000 7ffbdf7fe700 1 -- 172.21.15.99:0/3053201883 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+15377 (secure 0 0 0) 0x7ffbe0018090 con 0x7ffbe80c80f0 2023-10-08T00:15:41.551 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:15:41.552 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":103,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:15:40.594854+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:15:38.848324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-08T00:15:39.561379+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:15:41.554 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.553+0000 7ffbed76c700 1 -- 172.21.15.99:0/3053201883 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ffbc8066f30 msgr2=0x7ffbc80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:41.554 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.553+0000 7ffbed76c700 1 --2- 172.21.15.99:0/3053201883 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ffbc8066f30 0x7ffbc80693e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8000e10 tx=0x7ffbd8009040 comp rx=0 tx=0).stop 2023-10-08T00:15:41.554 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.553+0000 7ffbed76c700 1 -- 172.21.15.99:0/3053201883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbe80c80f0 msgr2=0x7ffbe80c8560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:41.554 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.553+0000 7ffbed76c700 1 --2- 172.21.15.99:0/3053201883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbe80c80f0 0x7ffbe80c8560 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7ffbe0016000 tx=0x7ffbe001a000 comp rx=0 tx=0).stop 2023-10-08T00:15:41.555 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.553+0000 7ffbed76c700 1 -- 172.21.15.99:0/3053201883 shutdown_connections 2023-10-08T00:15:41.555 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.553+0000 7ffbed76c700 1 --2- 172.21.15.99:0/3053201883 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ffbc8066f30 0x7ffbc80693e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:41.555 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.553+0000 7ffbed76c700 1 --2- 172.21.15.99:0/3053201883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbe80c80f0 0x7ffbe80c8560 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:41.556 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.553+0000 7ffbed76c700 1 --2- 172.21.15.99:0/3053201883 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffbe8167160 0x7ffbe80c6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:41.556 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.553+0000 7ffbed76c700 1 --2- 172.21.15.99:0/3053201883 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffbe8167530 0x7ffbe8168420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:41.556 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.553+0000 7ffbed76c700 1 -- 172.21.15.99:0/3053201883 >> 172.21.15.99:0/3053201883 conn(0x7ffbe8069870 msgr2=0x7ffbe806c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:41.556 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.553+0000 7ffbed76c700 1 -- 172.21.15.99:0/3053201883 shutdown_connections 2023-10-08T00:15:41.556 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.553+0000 7ffbed76c700 1 -- 172.21.15.99:0/3053201883 wait complete. 2023-10-08T00:15:41.566 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:15:41.680 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.679+0000 7fa9ac8a1700 1 Processor -- start 2023-10-08T00:15:41.680 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.679+0000 7fa9ac8a1700 1 -- start start 2023-10-08T00:15:41.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.680+0000 7fa9ac8a1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9a8157a90 0x7fa9a80b1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:41.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.680+0000 7fa9ac8a1700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa9a8157e60 0x7fa9a80b0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:41.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.680+0000 7fa9ac8a1700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa9a80b0660 0x7fa9a80ce950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:41.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.680+0000 7fa9ac8a1700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9a80b0ad0 con 0x7fa9a8157e60 2023-10-08T00:15:41.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.680+0000 7fa9ac8a1700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9a80b4970 con 0x7fa9a80b0660 2023-10-08T00:15:41.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.680+0000 7fa9ac8a1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a80cb0b0 con 0x7fa9a8157a90 2023-10-08T00:15:41.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.680+0000 7fa9a6ffd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa9a8157e60 0x7fa9a80b0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:41.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.680+0000 7fa9a6ffd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa9a8157e60 0x7fa9a80b0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51324/0 (socket says 172.21.15.99:51324) 2023-10-08T00:15:41.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.680+0000 7fa9a6ffd700 1 -- 172.21.15.99:0/3068241315 learned_addr learned my addr 172.21.15.99:0/3068241315 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:41.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.680+0000 7fa9a77fe700 1 --2- 172.21.15.99:0/3068241315 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9a8157a90 0x7fa9a80b1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:41.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.680+0000 7fa9a7fff700 1 --2- 172.21.15.99:0/3068241315 >> v2:172.21.15.191:3300/0 conn(0x7fa9a80b0660 0x7fa9a80ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:41.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.680+0000 7fa9a6ffd700 1 -- 172.21.15.99:0/3068241315 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9a8157a90 msgr2=0x7fa9a80b1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:41.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.680+0000 7fa9a6ffd700 1 --2- 172.21.15.99:0/3068241315 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9a8157a90 0x7fa9a80b1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:41.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.680+0000 7fa9a6ffd700 1 -- 172.21.15.99:0/3068241315 >> v2:172.21.15.191:3300/0 conn(0x7fa9a80b0660 msgr2=0x7fa9a80ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:41.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.680+0000 7fa9a6ffd700 1 --2- 172.21.15.99:0/3068241315 >> v2:172.21.15.191:3300/0 conn(0x7fa9a80b0660 0x7fa9a80ce950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:41.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.680+0000 7fa9a6ffd700 1 -- 172.21.15.99:0/3068241315 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a80ca780 con 0x7fa9a8157e60 2023-10-08T00:15:41.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.680+0000 7fa9a7fff700 1 --2- 172.21.15.99:0/3068241315 >> v2:172.21.15.191:3300/0 conn(0x7fa9a80b0660 0x7fa9a80ce950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:15:41.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.680+0000 7fa9a6ffd700 1 --2- 172.21.15.99:0/3068241315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9a8157e60 0x7fa9a80b0120 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fa9a80cf890 tx=0x7fa99c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9850e7bf6dfc63a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:41.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.681+0000 7fa9a67fc700 1 -- 172.21.15.99:0/3068241315 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99c00b820 con 0x7fa9a8157e60 2023-10-08T00:15:41.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.681+0000 7fa9a67fc700 1 -- 172.21.15.99:0/3068241315 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa99c00b980 con 0x7fa9a8157e60 2023-10-08T00:15:41.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.681+0000 7fa9a67fc700 1 -- 172.21.15.99:0/3068241315 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99c00bd20 con 0x7fa9a8157e60 2023-10-08T00:15:41.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.681+0000 7fa9ac8a1700 1 -- 172.21.15.99:0/3068241315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9a8157e60 msgr2=0x7fa9a80b0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:41.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.681+0000 7fa9ac8a1700 1 --2- 172.21.15.99:0/3068241315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9a8157e60 0x7fa9a80b0120 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fa9a80cf890 tx=0x7fa99c009250 comp rx=0 tx=0).stop 2023-10-08T00:15:41.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.681+0000 7fa9ac8a1700 1 -- 172.21.15.99:0/3068241315 shutdown_connections 2023-10-08T00:15:41.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.681+0000 7fa9ac8a1700 1 --2- 172.21.15.99:0/3068241315 >> v2:172.21.15.191:3300/0 conn(0x7fa9a80b0660 0x7fa9a80ce950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:41.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.681+0000 7fa9ac8a1700 1 --2- 172.21.15.99:0/3068241315 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9a8157a90 0x7fa9a80b1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:41.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.681+0000 7fa9ac8a1700 1 --2- 172.21.15.99:0/3068241315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9a8157e60 0x7fa9a80b0120 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:41.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.681+0000 7fa9ac8a1700 1 -- 172.21.15.99:0/3068241315 >> 172.21.15.99:0/3068241315 conn(0x7fa9a806bc70 msgr2=0x7fa9a806c070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:41.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.681+0000 7fa9ac8a1700 1 -- 172.21.15.99:0/3068241315 shutdown_connections 2023-10-08T00:15:41.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.681+0000 7fa9ac8a1700 1 -- 172.21.15.99:0/3068241315 wait complete. 2023-10-08T00:15:41.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.682+0000 7fa9ac8a1700 1 Processor -- start 2023-10-08T00:15:41.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.682+0000 7fa9ac8a1700 1 -- start start 2023-10-08T00:15:41.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.683+0000 7fa9ac8a1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9a80b0660 0x7fa9a80c7020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:41.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.683+0000 7fa9ac8a1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9a8157a90 0x7fa9a80c9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:41.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.683+0000 7fa9ac8a1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9a8157e60 0x7fa9a8167b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:41.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.683+0000 7fa9ac8a1700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a80c7bc0 con 0x7fa9a80b0660 2023-10-08T00:15:41.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.683+0000 7fa9ac8a1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a80c97d0 con 0x7fa9a8157e60 2023-10-08T00:15:41.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.683+0000 7fa9ac8a1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a80c7600 con 0x7fa9a8157a90 2023-10-08T00:15:41.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.683+0000 7fa9a77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9a80b0660 0x7fa9a80c7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:41.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.683+0000 7fa9a77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9a80b0660 0x7fa9a80c7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51330/0 (socket says 172.21.15.99:51330) 2023-10-08T00:15:41.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.683+0000 7fa9a77fe700 1 -- 172.21.15.99:0/746991157 learned_addr learned my addr 172.21.15.99:0/746991157 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:41.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.683+0000 7fa9a7fff700 1 --2- 172.21.15.99:0/746991157 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9a8157e60 0x7fa9a8167b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:41.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.683+0000 7fa9a6ffd700 1 --2- 172.21.15.99:0/746991157 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9a8157a90 0x7fa9a80c9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:41.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.684+0000 7fa9a77fe700 1 -- 172.21.15.99:0/746991157 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9a8157a90 msgr2=0x7fa9a80c9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:41.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.684+0000 7fa9a77fe700 1 --2- 172.21.15.99:0/746991157 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9a8157a90 0x7fa9a80c9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:41.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.684+0000 7fa9a77fe700 1 -- 172.21.15.99:0/746991157 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9a8157e60 msgr2=0x7fa9a8167b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:41.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.684+0000 7fa9a77fe700 1 --2- 172.21.15.99:0/746991157 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9a8157e60 0x7fa9a8167b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:41.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.684+0000 7fa9a77fe700 1 -- 172.21.15.99:0/746991157 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa99c0094f0 con 0x7fa9a80b0660 2023-10-08T00:15:41.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.684+0000 7fa9a77fe700 1 --2- 172.21.15.99:0/746991157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9a80b0660 0x7fa9a80c7020 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fa99800e190 tx=0x7fa998009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:41.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.684+0000 7fa98f7fe700 1 -- 172.21.15.99:0/746991157 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99800ecd0 con 0x7fa9a80b0660 2023-10-08T00:15:41.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.684+0000 7fa98f7fe700 1 -- 172.21.15.99:0/746991157 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa99800ee30 con 0x7fa9a80b0660 2023-10-08T00:15:41.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.684+0000 7fa98f7fe700 1 -- 172.21.15.99:0/746991157 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa998014670 con 0x7fa9a80b0660 2023-10-08T00:15:41.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.684+0000 7fa9ac8a1700 1 -- 172.21.15.99:0/746991157 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9a80c78e0 con 0x7fa9a80b0660 2023-10-08T00:15:41.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.684+0000 7fa9ac8a1700 1 -- 172.21.15.99:0/746991157 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9a80c5e50 con 0x7fa9a80b0660 2023-10-08T00:15:41.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.685+0000 7fa9ac8a1700 1 -- 172.21.15.99:0/746991157 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9a80c61c0 con 0x7fa9a80b0660 2023-10-08T00:15:41.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.685+0000 7fa98f7fe700 1 -- 172.21.15.99:0/746991157 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa998014a20 con 0x7fa9a80b0660 2023-10-08T00:15:41.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.686+0000 7fa98f7fe700 1 --2- 172.21.15.99:0/746991157 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa994066f30 0x7fa9940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:41.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.686+0000 7fa98f7fe700 1 -- 172.21.15.99:0/746991157 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa998084bc0 con 0x7fa9a80b0660 2023-10-08T00:15:41.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.686+0000 7fa9a6ffd700 1 --2- 172.21.15.99:0/746991157 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa994066f30 0x7fa9940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:41.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.686+0000 7fa9a6ffd700 1 --2- 172.21.15.99:0/746991157 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa994066f30 0x7fa9940693e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa9a80cf890 tx=0x7fa99c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:41.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.690+0000 7fa98f7fe700 1 -- 172.21.15.99:0/746991157 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa998051e30 con 0x7fa9a80b0660 2023-10-08T00:15:41.872 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.871+0000 7fa9ac8a1700 1 -- 172.21.15.99:0/746991157 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9a80caa60 con 0x7fa9a80b0660 2023-10-08T00:15:41.873 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.872+0000 7fa98f7fe700 1 -- 172.21.15.99:0/746991157 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+15377 (secure 0 0 0) 0x7fa998009a60 con 0x7fa9a80b0660 2023-10-08T00:15:41.873 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:15:41.873 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":103,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:15:40.594854+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:15:38.848324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-08T00:15:39.561379+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:15:41.875 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.874+0000 7fa9ac8a1700 1 -- 172.21.15.99:0/746991157 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa994066f30 msgr2=0x7fa9940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:41.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.874+0000 7fa9ac8a1700 1 --2- 172.21.15.99:0/746991157 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa994066f30 0x7fa9940693e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa9a80cf890 tx=0x7fa99c017000 comp rx=0 tx=0).stop 2023-10-08T00:15:41.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.874+0000 7fa9ac8a1700 1 -- 172.21.15.99:0/746991157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9a80b0660 msgr2=0x7fa9a80c7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:41.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.874+0000 7fa9ac8a1700 1 --2- 172.21.15.99:0/746991157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9a80b0660 0x7fa9a80c7020 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fa99800e190 tx=0x7fa998009920 comp rx=0 tx=0).stop 2023-10-08T00:15:41.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.874+0000 7fa9ac8a1700 1 -- 172.21.15.99:0/746991157 shutdown_connections 2023-10-08T00:15:41.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.874+0000 7fa9ac8a1700 1 --2- 172.21.15.99:0/746991157 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa994066f30 0x7fa9940693e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:41.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.874+0000 7fa9ac8a1700 1 --2- 172.21.15.99:0/746991157 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9a8157e60 0x7fa9a8167b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:41.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.874+0000 7fa9ac8a1700 1 --2- 172.21.15.99:0/746991157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9a80b0660 0x7fa9a80c7020 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:41.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.874+0000 7fa9ac8a1700 1 --2- 172.21.15.99:0/746991157 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9a8157a90 0x7fa9a80c9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:41.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.874+0000 7fa9ac8a1700 1 -- 172.21.15.99:0/746991157 >> 172.21.15.99:0/746991157 conn(0x7fa9a806bc70 msgr2=0x7fa9a8069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:41.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.874+0000 7fa9ac8a1700 1 -- 172.21.15.99:0/746991157 shutdown_connections 2023-10-08T00:15:41.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:41.875+0000 7fa9ac8a1700 1 -- 172.21.15.99:0/746991157 wait complete. 2023-10-08T00:15:41.887 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T00:15:42.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.005+0000 7f1a2cb2b700 1 Processor -- start 2023-10-08T00:15:42.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.005+0000 7f1a2cb2b700 1 -- start start 2023-10-08T00:15:42.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.006+0000 7f1a2cb2b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1a280678b0 0x7f1a2815db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:42.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.006+0000 7f1a2cb2b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a280cbf10 0x7f1a2815e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:42.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.006+0000 7f1a2cb2b700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a280cae70 con 0x7f1a280c9390 2023-10-08T00:15:42.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.006+0000 7f1a2cb2b700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a280cb150 con 0x7f1a280678b0 2023-10-08T00:15:42.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.006+0000 7f1a2cb2b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a280cb430 con 0x7f1a280cbf10 2023-10-08T00:15:42.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.006+0000 7f1a2659c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1a280678b0 0x7f1a2815db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:42.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.006+0000 7f1a1dd9b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a280cbf10 0x7f1a2815e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:42.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.006+0000 7f1a2659c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1a280678b0 0x7f1a2815db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.99:32866/0 (socket says 172.21.15.99:32866) 2023-10-08T00:15:42.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.006+0000 7f1a2659c700 1 -- 172.21.15.99:0/3425410194 learned_addr learned my addr 172.21.15.99:0/3425410194 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:42.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.007+0000 7f1a1dd9b700 1 -- 172.21.15.99:0/3425410194 >> v2:172.21.15.191:3300/0 conn(0x7f1a280678b0 msgr2=0x7f1a2815db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:42.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.007+0000 7f1a1dd9b700 1 --2- 172.21.15.99:0/3425410194 >> v2:172.21.15.191:3300/0 conn(0x7f1a280678b0 0x7f1a2815db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:42.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.007+0000 7f1a1dd9b700 1 -- 172.21.15.99:0/3425410194 >> v1:172.21.15.191:6789/0 conn(0x7f1a280c9390 legacy=0x7f1a2806bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:42.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.007+0000 7f1a1dd9b700 1 -- 172.21.15.99:0/3425410194 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a280cb710 con 0x7f1a280cbf10 2023-10-08T00:15:42.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.007+0000 7f1a1dd9b700 1 --2- 172.21.15.99:0/3425410194 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a280cbf10 0x7f1a2815e0c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1a100098e0 tx=0x7f1a100091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=391c18d062d29ba6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:42.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.007+0000 7f1a25d9b700 1 -- 172.21.15.99:0/3425410194 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1000b640 con 0x7f1a280cbf10 2023-10-08T00:15:42.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.007+0000 7f1a25d9b700 1 -- 172.21.15.99:0/3425410194 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a1000b7a0 con 0x7f1a280cbf10 2023-10-08T00:15:42.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.007+0000 7f1a25d9b700 1 -- 172.21.15.99:0/3425410194 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1000bb40 con 0x7f1a280cbf10 2023-10-08T00:15:42.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.008+0000 7f1a2cb2b700 1 -- 172.21.15.99:0/3425410194 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a280cbf10 msgr2=0x7f1a2815e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:42.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.008+0000 7f1a2cb2b700 1 --2- 172.21.15.99:0/3425410194 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a280cbf10 0x7f1a2815e0c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1a100098e0 tx=0x7f1a100091a0 comp rx=0 tx=0).stop 2023-10-08T00:15:42.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.008+0000 7f1a2cb2b700 1 -- 172.21.15.99:0/3425410194 shutdown_connections 2023-10-08T00:15:42.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.008+0000 7f1a2cb2b700 1 --2- 172.21.15.99:0/3425410194 >> v2:172.21.15.191:3300/0 conn(0x7f1a280678b0 0x7f1a2815db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:42.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.008+0000 7f1a2cb2b700 1 --2- 172.21.15.99:0/3425410194 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a280cbf10 0x7f1a2815e0c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:42.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.008+0000 7f1a2cb2b700 1 -- 172.21.15.99:0/3425410194 >> 172.21.15.99:0/3425410194 conn(0x7f1a28069870 msgr2=0x7f1a28069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:42.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.008+0000 7f1a2cb2b700 1 -- 172.21.15.99:0/3425410194 shutdown_connections 2023-10-08T00:15:42.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.008+0000 7f1a2cb2b700 1 -- 172.21.15.99:0/3425410194 wait complete. 2023-10-08T00:15:42.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.009+0000 7f1a2cb2b700 1 Processor -- start 2023-10-08T00:15:42.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.009+0000 7f1a2cb2b700 1 -- start start 2023-10-08T00:15:42.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.010+0000 7f1a2cb2b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a280678b0 0x7f1a28150610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:42.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.010+0000 7f1a2cb2b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a280c9390 0x7f1a280c6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:42.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.010+0000 7f1a2cb2b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a280cbf10 0x7f1a280c4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:42.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.010+0000 7f1a2cb2b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a280cc7d0 con 0x7f1a280cbf10 2023-10-08T00:15:42.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.010+0000 7f1a2cb2b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a28150b50 con 0x7f1a280c9390 2023-10-08T00:15:42.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.010+0000 7f1a2cb2b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a280cca70 con 0x7f1a280678b0 2023-10-08T00:15:42.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.010+0000 7f1a26d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a280cbf10 0x7f1a280c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:42.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.010+0000 7f1a26d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a280cbf10 0x7f1a280c4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51332/0 (socket says 172.21.15.99:51332) 2023-10-08T00:15:42.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.010+0000 7f1a26d9d700 1 -- 172.21.15.99:0/1522666331 learned_addr learned my addr 172.21.15.99:0/1522666331 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:42.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.010+0000 7f1a1dd9b700 1 --2- 172.21.15.99:0/1522666331 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a280c9390 0x7f1a280c6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:42.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.010+0000 7f1a2659c700 1 --2- 172.21.15.99:0/1522666331 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a280678b0 0x7f1a28150610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:42.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.010+0000 7f1a26d9d700 1 -- 172.21.15.99:0/1522666331 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a280678b0 msgr2=0x7f1a28150610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:42.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.010+0000 7f1a26d9d700 1 --2- 172.21.15.99:0/1522666331 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a280678b0 0x7f1a28150610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:42.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.010+0000 7f1a26d9d700 1 -- 172.21.15.99:0/1522666331 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a280c9390 msgr2=0x7f1a280c6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:42.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.010+0000 7f1a26d9d700 1 --2- 172.21.15.99:0/1522666331 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a280c9390 0x7f1a280c6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:42.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.010+0000 7f1a26d9d700 1 -- 172.21.15.99:0/1522666331 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a10009420 con 0x7f1a280cbf10 2023-10-08T00:15:42.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.010+0000 7f1a1dd9b700 1 --2- 172.21.15.99:0/1522666331 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a280c9390 0x7f1a280c6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:15:42.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.010+0000 7f1a26d9d700 1 --2- 172.21.15.99:0/1522666331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a280cbf10 0x7f1a280c4830 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f1a2000c8e0 tx=0x7f1a20001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:42.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.011+0000 7f1a1effd700 1 -- 172.21.15.99:0/1522666331 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a20012630 con 0x7f1a280cbf10 2023-10-08T00:15:42.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.011+0000 7f1a1effd700 1 -- 172.21.15.99:0/1522666331 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a20012790 con 0x7f1a280cbf10 2023-10-08T00:15:42.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.011+0000 7f1a2cb2b700 1 -- 172.21.15.99:0/1522666331 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a280ccd50 con 0x7f1a280cbf10 2023-10-08T00:15:42.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.011+0000 7f1a1effd700 1 -- 172.21.15.99:0/1522666331 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a20012b50 con 0x7f1a280cbf10 2023-10-08T00:15:42.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.011+0000 7f1a2cb2b700 1 -- 172.21.15.99:0/1522666331 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a280cd310 con 0x7f1a280cbf10 2023-10-08T00:15:42.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.012+0000 7f1a2cb2b700 1 -- 172.21.15.99:0/1522666331 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a280cd890 con 0x7f1a280cbf10 2023-10-08T00:15:42.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.012+0000 7f1a1effd700 1 -- 172.21.15.99:0/1522666331 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1a2001a3a0 con 0x7f1a280cbf10 2023-10-08T00:15:42.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.012+0000 7f1a1effd700 1 --2- 172.21.15.99:0/1522666331 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1a08066f30 0x7f1a080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:42.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.012+0000 7f1a1effd700 1 -- 172.21.15.99:0/1522666331 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1a200846a0 con 0x7f1a280cbf10 2023-10-08T00:15:42.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.012+0000 7f1a2659c700 1 --2- 172.21.15.99:0/1522666331 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1a08066f30 0x7f1a080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:42.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.013+0000 7f1a2659c700 1 --2- 172.21.15.99:0/1522666331 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1a08066f30 0x7f1a080693e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1a2815f3a0 tx=0x7f1a18009250 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:42.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.017+0000 7f1a1effd700 1 -- 172.21.15.99:0/1522666331 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1a200553a0 con 0x7f1a280cbf10 2023-10-08T00:15:42.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:42.244+0000 7f1a2cb2b700 1 -- 172.21.15.99:0/1522666331 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f1a2804c3c0 con 0x7f1a280cbf10 2023-10-08T00:15:43.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.232+0000 7f1a1effd700 1 -- 172.21.15.99:0/1522666331 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v47) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1a20054f30 con 0x7f1a280cbf10 2023-10-08T00:15:43.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.235+0000 7f1a2cb2b700 1 -- 172.21.15.99:0/1522666331 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1a08066f30 msgr2=0x7f1a080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:43.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.235+0000 7f1a2cb2b700 1 --2- 172.21.15.99:0/1522666331 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1a08066f30 0x7f1a080693e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1a2815f3a0 tx=0x7f1a18009250 comp rx=0 tx=0).stop 2023-10-08T00:15:43.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.235+0000 7f1a2cb2b700 1 -- 172.21.15.99:0/1522666331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a280cbf10 msgr2=0x7f1a280c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:43.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.235+0000 7f1a2cb2b700 1 --2- 172.21.15.99:0/1522666331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a280cbf10 0x7f1a280c4830 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f1a2000c8e0 tx=0x7f1a20001710 comp rx=0 tx=0).stop 2023-10-08T00:15:43.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.235+0000 7f1a2cb2b700 1 -- 172.21.15.99:0/1522666331 shutdown_connections 2023-10-08T00:15:43.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.235+0000 7f1a2cb2b700 1 --2- 172.21.15.99:0/1522666331 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1a08066f30 0x7f1a080693e0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:43.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.235+0000 7f1a2cb2b700 1 --2- 172.21.15.99:0/1522666331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a280cbf10 0x7f1a280c4830 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:43.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.235+0000 7f1a2cb2b700 1 --2- 172.21.15.99:0/1522666331 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a280678b0 0x7f1a28150610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:43.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.235+0000 7f1a2cb2b700 1 --2- 172.21.15.99:0/1522666331 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a280c9390 0x7f1a280c6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:43.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.235+0000 7f1a2cb2b700 1 -- 172.21.15.99:0/1522666331 >> 172.21.15.99:0/1522666331 conn(0x7f1a28069870 msgr2=0x7f1a2806bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:43.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.235+0000 7f1a2cb2b700 1 -- 172.21.15.99:0/1522666331 shutdown_connections 2023-10-08T00:15:43.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.235+0000 7f1a2cb2b700 1 -- 172.21.15.99:0/1522666331 wait complete. 2023-10-08T00:15:43.250 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T00:15:43.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.373+0000 7f6e5b57d700 1 Processor -- start 2023-10-08T00:15:43.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.373+0000 7f6e5b57d700 1 -- start start 2023-10-08T00:15:43.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.374+0000 7f6e5b57d700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6e540678b0 0x7f6e5415db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:43.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.374+0000 7f6e5b57d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f6e540cbf10 0x7f6e5415e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:43.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.374+0000 7f6e5b57d700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e540cae70 con 0x7f6e540c9390 2023-10-08T00:15:43.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.374+0000 7f6e5b57d700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e540cb150 con 0x7f6e540678b0 2023-10-08T00:15:43.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.374+0000 7f6e5b57d700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e540cb430 con 0x7f6e540cbf10 2023-10-08T00:15:43.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.374+0000 7f6e59319700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6e540678b0 0x7f6e5415db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:43.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.374+0000 7f6e59319700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6e540678b0 0x7f6e5415db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40864/0 (socket says 172.21.15.99:40864) 2023-10-08T00:15:43.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.374+0000 7f6e59319700 1 -- 172.21.15.99:0/609405415 learned_addr learned my addr 172.21.15.99:0/609405415 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:43.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.375+0000 7f6e58b18700 1 --2- 172.21.15.99:0/609405415 >> v2:172.21.15.191:3300/0 conn(0x7f6e540cbf10 0x7f6e5415e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:43.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.375+0000 7f6e59319700 1 -- 172.21.15.99:0/609405415 >> v2:172.21.15.191:3300/0 conn(0x7f6e540cbf10 msgr2=0x7f6e5415e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:43.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.375+0000 7f6e59319700 1 --2- 172.21.15.99:0/609405415 >> v2:172.21.15.191:3300/0 conn(0x7f6e540cbf10 0x7f6e5415e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:43.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.375+0000 7f6e59319700 1 -- 172.21.15.99:0/609405415 >> v1:172.21.15.191:6789/0 conn(0x7f6e540c9390 legacy=0x7f6e5406bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:43.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.375+0000 7f6e59319700 1 -- 172.21.15.99:0/609405415 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e540cb710 con 0x7f6e540678b0 2023-10-08T00:15:43.378 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.375+0000 7f6e59319700 1 --2- 172.21.15.99:0/609405415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e540678b0 0x7f6e5415db80 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f6e540ca0a0 tx=0x7f6e44009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f1f5b4a0ae1063 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:43.378 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.375+0000 7f6e53fff700 1 -- 172.21.15.99:0/609405415 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4400e070 con 0x7f6e540678b0 2023-10-08T00:15:43.378 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.375+0000 7f6e53fff700 1 -- 172.21.15.99:0/609405415 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e4400ba10 con 0x7f6e540678b0 2023-10-08T00:15:43.378 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.375+0000 7f6e53fff700 1 -- 172.21.15.99:0/609405415 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e440092e0 con 0x7f6e540678b0 2023-10-08T00:15:43.378 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.375+0000 7f6e5b57d700 1 -- 172.21.15.99:0/609405415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e540678b0 msgr2=0x7f6e5415db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:43.378 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.375+0000 7f6e5b57d700 1 --2- 172.21.15.99:0/609405415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e540678b0 0x7f6e5415db80 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f6e540ca0a0 tx=0x7f6e44009840 comp rx=0 tx=0).stop 2023-10-08T00:15:43.378 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.376+0000 7f6e5b57d700 1 -- 172.21.15.99:0/609405415 shutdown_connections 2023-10-08T00:15:43.378 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.376+0000 7f6e5b57d700 1 --2- 172.21.15.99:0/609405415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e540678b0 0x7f6e5415db80 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:43.378 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.376+0000 7f6e5b57d700 1 --2- 172.21.15.99:0/609405415 >> v2:172.21.15.191:3300/0 conn(0x7f6e540cbf10 0x7f6e5415e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:43.378 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.376+0000 7f6e5b57d700 1 -- 172.21.15.99:0/609405415 >> 172.21.15.99:0/609405415 conn(0x7f6e54069870 msgr2=0x7f6e54069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:43.378 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.376+0000 7f6e5b57d700 1 -- 172.21.15.99:0/609405415 shutdown_connections 2023-10-08T00:15:43.379 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.376+0000 7f6e5b57d700 1 -- 172.21.15.99:0/609405415 wait complete. 2023-10-08T00:15:43.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.377+0000 7f6e5b57d700 1 Processor -- start 2023-10-08T00:15:43.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.377+0000 7f6e5b57d700 1 -- start start 2023-10-08T00:15:43.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.378+0000 7f6e5b57d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e540678b0 0x7f6e541508c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:43.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.378+0000 7f6e5b57d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e540c9390 0x7f6e540c4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:43.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.378+0000 7f6e5b57d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e540cbf10 0x7f6e540c4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:43.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.378+0000 7f6e5b57d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e54150e00 con 0x7f6e540cbf10 2023-10-08T00:15:43.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.378+0000 7f6e5b57d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e540c7070 con 0x7f6e540c9390 2023-10-08T00:15:43.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.378+0000 7f6e5b57d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e540c7350 con 0x7f6e540678b0 2023-10-08T00:15:43.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.378+0000 7f6e59b1a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e540cbf10 0x7f6e540c4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:43.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.378+0000 7f6e59b1a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e540cbf10 0x7f6e540c4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40878/0 (socket says 172.21.15.99:40878) 2023-10-08T00:15:43.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.378+0000 7f6e59b1a700 1 -- 172.21.15.99:0/794565575 learned_addr learned my addr 172.21.15.99:0/794565575 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:43.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.378+0000 7f6e59319700 1 --2- 172.21.15.99:0/794565575 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e540678b0 0x7f6e541508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:43.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.378+0000 7f6e58b18700 1 --2- 172.21.15.99:0/794565575 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e540c9390 0x7f6e540c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:43.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.378+0000 7f6e59b1a700 1 -- 172.21.15.99:0/794565575 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e540678b0 msgr2=0x7f6e541508c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:43.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.378+0000 7f6e59b1a700 1 --2- 172.21.15.99:0/794565575 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e540678b0 0x7f6e541508c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:43.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.378+0000 7f6e59b1a700 1 -- 172.21.15.99:0/794565575 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e540c9390 msgr2=0x7f6e540c4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:43.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.378+0000 7f6e59b1a700 1 --2- 172.21.15.99:0/794565575 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e540c9390 0x7f6e540c4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:43.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.378+0000 7f6e59b1a700 1 -- 172.21.15.99:0/794565575 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e44009ae0 con 0x7f6e540cbf10 2023-10-08T00:15:43.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.378+0000 7f6e59b1a700 1 --2- 172.21.15.99:0/794565575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e540cbf10 0x7f6e540c4d70 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c00c5b0 tx=0x7f6e4c007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:43.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.378+0000 7f6e517fa700 1 -- 172.21.15.99:0/794565575 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c011590 con 0x7f6e540cbf10 2023-10-08T00:15:43.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.378+0000 7f6e517fa700 1 -- 172.21.15.99:0/794565575 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e4c011710 con 0x7f6e540cbf10 2023-10-08T00:15:43.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.378+0000 7f6e5b57d700 1 -- 172.21.15.99:0/794565575 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e540c7630 con 0x7f6e540cbf10 2023-10-08T00:15:43.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.378+0000 7f6e517fa700 1 -- 172.21.15.99:0/794565575 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c011ad0 con 0x7f6e540cbf10 2023-10-08T00:15:43.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.378+0000 7f6e5b57d700 1 -- 172.21.15.99:0/794565575 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e540cc730 con 0x7f6e540cbf10 2023-10-08T00:15:43.384 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.379+0000 7f6e5b57d700 1 -- 172.21.15.99:0/794565575 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e5404c3c0 con 0x7f6e540cbf10 2023-10-08T00:15:43.384 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.380+0000 7f6e517fa700 1 -- 172.21.15.99:0/794565575 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6e4c01c460 con 0x7f6e540cbf10 2023-10-08T00:15:43.384 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.380+0000 7f6e517fa700 1 --2- 172.21.15.99:0/794565575 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6e40066f30 0x7f6e400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:43.384 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.380+0000 7f6e517fa700 1 -- 172.21.15.99:0/794565575 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6e4c0840d0 con 0x7f6e540cbf10 2023-10-08T00:15:43.384 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.380+0000 7f6e59319700 1 --2- 172.21.15.99:0/794565575 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6e40066f30 0x7f6e400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:43.384 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.381+0000 7f6e59319700 1 --2- 172.21.15.99:0/794565575 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6e40066f30 0x7f6e400693e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f6e44009880 tx=0x7f6e44017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:43.385 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.384+0000 7f6e517fa700 1 -- 172.21.15.99:0/794565575 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6e4c051340 con 0x7f6e540cbf10 2023-10-08T00:15:43.611 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:43.609+0000 7f6e5b57d700 1 -- 172.21.15.99:0/794565575 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f6e540c71b0 con 0x7f6e540cbf10 2023-10-08T00:15:44.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.239+0000 7f6e517fa700 1 -- 172.21.15.99:0/794565575 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v48) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6e540c71b0 con 0x7f6e540cbf10 2023-10-08T00:15:44.243 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.242+0000 7f6e5b57d700 1 -- 172.21.15.99:0/794565575 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6e40066f30 msgr2=0x7f6e400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:44.243 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.242+0000 7f6e5b57d700 1 --2- 172.21.15.99:0/794565575 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6e40066f30 0x7f6e400693e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f6e44009880 tx=0x7f6e44017000 comp rx=0 tx=0).stop 2023-10-08T00:15:44.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.242+0000 7f6e5b57d700 1 -- 172.21.15.99:0/794565575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e540cbf10 msgr2=0x7f6e540c4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:44.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.242+0000 7f6e5b57d700 1 --2- 172.21.15.99:0/794565575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e540cbf10 0x7f6e540c4d70 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c00c5b0 tx=0x7f6e4c007f60 comp rx=0 tx=0).stop 2023-10-08T00:15:44.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.242+0000 7f6e5b57d700 1 -- 172.21.15.99:0/794565575 shutdown_connections 2023-10-08T00:15:44.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.242+0000 7f6e5b57d700 1 --2- 172.21.15.99:0/794565575 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6e40066f30 0x7f6e400693e0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:44.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.242+0000 7f6e5b57d700 1 --2- 172.21.15.99:0/794565575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e540cbf10 0x7f6e540c4d70 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:44.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.242+0000 7f6e5b57d700 1 --2- 172.21.15.99:0/794565575 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e540678b0 0x7f6e541508c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:44.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.242+0000 7f6e5b57d700 1 --2- 172.21.15.99:0/794565575 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e540c9390 0x7f6e540c4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:44.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.242+0000 7f6e5b57d700 1 -- 172.21.15.99:0/794565575 >> 172.21.15.99:0/794565575 conn(0x7f6e54069870 msgr2=0x7f6e5415bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:44.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.242+0000 7f6e5b57d700 1 -- 172.21.15.99:0/794565575 shutdown_connections 2023-10-08T00:15:44.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.242+0000 7f6e5b57d700 1 -- 172.21.15.99:0/794565575 wait complete. 2023-10-08T00:15:44.256 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:15:44.373 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.372+0000 7fd9bbada700 1 Processor -- start 2023-10-08T00:15:44.373 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.372+0000 7fd9bbada700 1 -- start start 2023-10-08T00:15:44.373 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.373+0000 7fd9bbada700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9b40d1c30 0x7fd9b40ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:44.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.373+0000 7fd9bbada700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd9b415a680 0x7fd9b40cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:44.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.373+0000 7fd9bbada700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9b40c5d80 con 0x7fd9b415a2b0 2023-10-08T00:15:44.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.373+0000 7fd9bbada700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9b4068470 con 0x7fd9b415a680 2023-10-08T00:15:44.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.373+0000 7fd9bbada700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b40c6060 con 0x7fd9b40d1c30 2023-10-08T00:15:44.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.373+0000 7fd9b9075700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9b40d1c30 0x7fd9b40ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:44.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.373+0000 7fd9ba077700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd9b415a680 0x7fd9b40cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:44.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.373+0000 7fd9b9075700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9b40d1c30 0x7fd9b40ccdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.99:56964/0 (socket says 172.21.15.99:56964) 2023-10-08T00:15:44.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.373+0000 7fd9ba077700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd9b415a680 0x7fd9b40cf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.99:39102/0 (socket says 172.21.15.99:39102) 2023-10-08T00:15:44.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.373+0000 7fd9b9075700 1 -- 172.21.15.99:0/2236594788 learned_addr learned my addr 172.21.15.99:0/2236594788 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:44.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.374+0000 7fd9ba077700 1 -- 172.21.15.99:0/2236594788 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9b40d1c30 msgr2=0x7fd9b40ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:44.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.374+0000 7fd9ba077700 1 --2- 172.21.15.99:0/2236594788 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9b40d1c30 0x7fd9b40ccdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:44.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.374+0000 7fd9ba077700 1 -- 172.21.15.99:0/2236594788 >> v1:172.21.15.191:6789/0 conn(0x7fd9b415a2b0 legacy=0x7fd9b40b3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:44.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.374+0000 7fd9ba077700 1 -- 172.21.15.99:0/2236594788 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b40c6340 con 0x7fd9b415a680 2023-10-08T00:15:44.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.374+0000 7fd9b8874700 1 -- 172.21.15.99:0/2236594788 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1620657921 0 0) 0x7fd9b40c5d80 con 0x7fd9b415a2b0 2023-10-08T00:15:44.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.374+0000 7fd9b9075700 1 --2- 172.21.15.99:0/2236594788 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9b40d1c30 0x7fd9b40ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:15:44.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.374+0000 7fd9ba077700 1 --2- 172.21.15.99:0/2236594788 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9b415a680 0x7fd9b40cf710 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac0098e0 tx=0x7fd9ac0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee180f7fa2e67fb7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:44.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.374+0000 7fd9b8874700 1 -- 172.21.15.99:0/2236594788 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac00b640 con 0x7fd9b415a680 2023-10-08T00:15:44.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.374+0000 7fd9b8874700 1 -- 172.21.15.99:0/2236594788 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9ac00b7a0 con 0x7fd9b415a680 2023-10-08T00:15:44.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.374+0000 7fd9b8874700 1 -- 172.21.15.99:0/2236594788 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac00bb40 con 0x7fd9b415a680 2023-10-08T00:15:44.378 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.375+0000 7fd9bbada700 1 -- 172.21.15.99:0/2236594788 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9b415a680 msgr2=0x7fd9b40cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:44.378 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.375+0000 7fd9bbada700 1 --2- 172.21.15.99:0/2236594788 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9b415a680 0x7fd9b40cf710 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac0098e0 tx=0x7fd9ac0091a0 comp rx=0 tx=0).stop 2023-10-08T00:15:44.378 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.375+0000 7fd9bbada700 1 -- 172.21.15.99:0/2236594788 shutdown_connections 2023-10-08T00:15:44.378 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.375+0000 7fd9bbada700 1 --2- 172.21.15.99:0/2236594788 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9b415a680 0x7fd9b40cf710 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:44.378 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.375+0000 7fd9bbada700 1 --2- 172.21.15.99:0/2236594788 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9b40d1c30 0x7fd9b40ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:44.378 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.375+0000 7fd9bbada700 1 -- 172.21.15.99:0/2236594788 >> 172.21.15.99:0/2236594788 conn(0x7fd9b4067e40 msgr2=0x7fd9b40d0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:44.378 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.375+0000 7fd9bbada700 1 -- 172.21.15.99:0/2236594788 shutdown_connections 2023-10-08T00:15:44.378 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.375+0000 7fd9bbada700 1 -- 172.21.15.99:0/2236594788 wait complete. 2023-10-08T00:15:44.378 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.376+0000 7fd9bbada700 1 Processor -- start 2023-10-08T00:15:44.379 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.376+0000 7fd9bbada700 1 -- start start 2023-10-08T00:15:44.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.376+0000 7fd9bbada700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9b40d1c30 0x7fd9b415d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:44.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.376+0000 7fd9bbada700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9b415a2b0 0x7fd9b415ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:44.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.376+0000 7fd9bbada700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9b415a680 0x7fd9b416d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:44.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.376+0000 7fd9bbada700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b415c0c0 con 0x7fd9b415a2b0 2023-10-08T00:15:44.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.376+0000 7fd9bbada700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b415b540 con 0x7fd9b415a680 2023-10-08T00:15:44.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.376+0000 7fd9bbada700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b415b820 con 0x7fd9b40d1c30 2023-10-08T00:15:44.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.377+0000 7fd9b9075700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9b415a2b0 0x7fd9b415ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:44.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.377+0000 7fd9b9075700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9b415a2b0 0x7fd9b415ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40892/0 (socket says 172.21.15.99:40892) 2023-10-08T00:15:44.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.377+0000 7fd9b9075700 1 -- 172.21.15.99:0/3053094093 learned_addr learned my addr 172.21.15.99:0/3053094093 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:44.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.377+0000 7fd9ba077700 1 --2- 172.21.15.99:0/3053094093 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9b415a680 0x7fd9b416d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:44.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.377+0000 7fd9b9876700 1 --2- 172.21.15.99:0/3053094093 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9b40d1c30 0x7fd9b415d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:44.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.377+0000 7fd9b9075700 1 -- 172.21.15.99:0/3053094093 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9b40d1c30 msgr2=0x7fd9b415d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:44.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.377+0000 7fd9b9075700 1 --2- 172.21.15.99:0/3053094093 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9b40d1c30 0x7fd9b415d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:44.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.377+0000 7fd9b9075700 1 -- 172.21.15.99:0/3053094093 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9b415a680 msgr2=0x7fd9b416d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:44.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.377+0000 7fd9b9075700 1 --2- 172.21.15.99:0/3053094093 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9b415a680 0x7fd9b416d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:44.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.377+0000 7fd9b9075700 1 -- 172.21.15.99:0/3053094093 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a80094f0 con 0x7fd9b415a2b0 2023-10-08T00:15:44.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.377+0000 7fd9b9075700 1 --2- 172.21.15.99:0/3053094093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9b415a2b0 0x7fd9b415ab60 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fd9a800ec80 tx=0x7fd9a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:44.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.377+0000 7fd9b1ffb700 1 -- 172.21.15.99:0/3053094093 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a800c850 con 0x7fd9b415a2b0 2023-10-08T00:15:44.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.377+0000 7fd9b1ffb700 1 -- 172.21.15.99:0/3053094093 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a800c9b0 con 0x7fd9b415a2b0 2023-10-08T00:15:44.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.377+0000 7fd9bbada700 1 -- 172.21.15.99:0/3053094093 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9ac009420 con 0x7fd9b415a2b0 2023-10-08T00:15:44.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.377+0000 7fd9b1ffb700 1 -- 172.21.15.99:0/3053094093 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a800cd30 con 0x7fd9b415a2b0 2023-10-08T00:15:44.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.378+0000 7fd9bbada700 1 -- 172.21.15.99:0/3053094093 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9b415cc40 con 0x7fd9b415a2b0 2023-10-08T00:15:44.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.378+0000 7fd9bbada700 1 -- 172.21.15.99:0/3053094093 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9b40c8090 con 0x7fd9b415a2b0 2023-10-08T00:15:44.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.379+0000 7fd9b1ffb700 1 -- 172.21.15.99:0/3053094093 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd9a8013680 con 0x7fd9b415a2b0 2023-10-08T00:15:44.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.379+0000 7fd9b1ffb700 1 --2- 172.21.15.99:0/3053094093 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd9a0066f30 0x7fd9a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:44.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.379+0000 7fd9b1ffb700 1 -- 172.21.15.99:0/3053094093 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd9a8084780 con 0x7fd9b415a2b0 2023-10-08T00:15:44.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.380+0000 7fd9b9876700 1 --2- 172.21.15.99:0/3053094093 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd9a0066f30 0x7fd9a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:44.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.380+0000 7fd9b9876700 1 --2- 172.21.15.99:0/3053094093 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd9a0066f30 0x7fd9a00693e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd9a400c5f0 tx=0x7fd9a4001a70 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:44.384 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.383+0000 7fd9b1ffb700 1 -- 172.21.15.99:0/3053094093 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd9b40c8090 con 0x7fd9b415a2b0 2023-10-08T00:15:44.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.606+0000 7fd9bbada700 1 -- 172.21.15.99:0/3053094093 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9b40c8090 con 0x7fd9b415a2b0 2023-10-08T00:15:44.607 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.607+0000 7fd9b1ffb700 1 -- 172.21.15.99:0/3053094093 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4599 (secure 0 0 0) 0x7fd9b40c8090 con 0x7fd9b415a2b0 2023-10-08T00:15:44.608 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:15:44.608 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":48,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5426,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/280880696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":280880696},{"type":"v1","addr":"172.21.15.191:6836","nonce":280880696}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":5902,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3148129562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":3148129562},{"type":"v1","addr":"172.21.15.99:6835","nonce":3148129562}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":5917,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/1825600944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":1825600944},{"type":"v1","addr":"172.21.15.191:6837","nonce":1825600944}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:15:40.595166+0000","modified":"2023-10-08T00:15:44.232916+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6360},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6360":{"gid":6360,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6834/146609901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":146609901},{"type":"v1","addr":"172.21.15.99:6834","nonce":146609901}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-08T00:15:44.611 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.609+0000 7fd9bbada700 1 -- 172.21.15.99:0/3053094093 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd9a0066f30 msgr2=0x7fd9a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:44.611 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.609+0000 7fd9bbada700 1 --2- 172.21.15.99:0/3053094093 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd9a0066f30 0x7fd9a00693e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd9a400c5f0 tx=0x7fd9a4001a70 comp rx=0 tx=0).stop 2023-10-08T00:15:44.611 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.609+0000 7fd9bbada700 1 -- 172.21.15.99:0/3053094093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9b415a2b0 msgr2=0x7fd9b415ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:44.612 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.609+0000 7fd9bbada700 1 --2- 172.21.15.99:0/3053094093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9b415a2b0 0x7fd9b415ab60 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fd9a800ec80 tx=0x7fd9a8009250 comp rx=0 tx=0).stop 2023-10-08T00:15:44.612 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.609+0000 7fd9bbada700 1 -- 172.21.15.99:0/3053094093 shutdown_connections 2023-10-08T00:15:44.612 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.609+0000 7fd9bbada700 1 --2- 172.21.15.99:0/3053094093 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd9a0066f30 0x7fd9a00693e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:44.612 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.609+0000 7fd9bbada700 1 --2- 172.21.15.99:0/3053094093 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9b415a680 0x7fd9b416d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:44.612 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.609+0000 7fd9bbada700 1 --2- 172.21.15.99:0/3053094093 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9b40d1c30 0x7fd9b415d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:44.612 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.609+0000 7fd9bbada700 1 --2- 172.21.15.99:0/3053094093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9b415a2b0 0x7fd9b415ab60 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:44.612 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.609+0000 7fd9bbada700 1 -- 172.21.15.99:0/3053094093 >> 172.21.15.99:0/3053094093 conn(0x7fd9b4067e40 msgr2=0x7fd9b4069530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:44.612 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.609+0000 7fd9bbada700 1 -- 172.21.15.99:0/3053094093 shutdown_connections 2023-10-08T00:15:44.612 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.609+0000 7fd9bbada700 1 -- 172.21.15.99:0/3053094093 wait complete. 2023-10-08T00:15:44.612 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 48 2023-10-08T00:15:44.622 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:15:44.735 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.735+0000 7f6f52fbf700 1 Processor -- start 2023-10-08T00:15:44.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.735+0000 7f6f52fbf700 1 -- start start 2023-10-08T00:15:44.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.736+0000 7f6f52fbf700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6f4c0cd6e0 0x7f6f4c0cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:44.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.736+0000 7f6f52fbf700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f4c0cb7c0 con 0x7f6f4c15cbd0 2023-10-08T00:15:44.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.736+0000 7f6f52fbf700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f4c15e290 con 0x7f6f4c0cdab0 2023-10-08T00:15:44.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.736+0000 7f6f52fbf700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f4c15e850 con 0x7f6f4c0cd6e0 2023-10-08T00:15:44.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.736+0000 7f6f51fbd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6f4c0cd6e0 0x7f6f4c0cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:44.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.736+0000 7f6f51fbd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6f4c0cd6e0 0x7f6f4c0cbc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40898/0 (socket says 172.21.15.99:40898) 2023-10-08T00:15:44.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.736+0000 7f6f51fbd700 1 -- 172.21.15.99:0/135201272 learned_addr learned my addr 172.21.15.99:0/135201272 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:44.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.736+0000 7f6f51fbd700 1 -- 172.21.15.99:0/135201272 >> v1:172.21.15.191:6789/0 conn(0x7f6f4c0cdab0 legacy=0x7f6f4c15d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:44.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.736+0000 7f6f51fbd700 1 -- 172.21.15.99:0/135201272 >> v1:172.21.15.99:6789/0 conn(0x7f6f4c15cbd0 legacy=0x7f6f4c15d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:44.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.736+0000 7f6f51fbd700 1 -- 172.21.15.99:0/135201272 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f4c15e570 con 0x7f6f4c0cd6e0 2023-10-08T00:15:44.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.736+0000 7f6f50fbb700 1 -- 172.21.15.99:0/135201272 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 481954622 0 0) 0x7f6f4c0cb7c0 con 0x7f6f4c15cbd0 2023-10-08T00:15:44.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.736+0000 7f6f51fbd700 1 --2- 172.21.15.99:0/135201272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f4c0cd6e0 0x7f6f4c0cbc50 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f6f44009f20 tx=0x7f6f44009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a8c445a73babee server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:44.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.736+0000 7f6f50fbb700 1 -- 172.21.15.99:0/135201272 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4400e070 con 0x7f6f4c0cd6e0 2023-10-08T00:15:44.739 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.736+0000 7f6f50fbb700 1 -- 172.21.15.99:0/135201272 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f4400ba40 con 0x7f6f4c0cd6e0 2023-10-08T00:15:44.739 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.736+0000 7f6f50fbb700 1 -- 172.21.15.99:0/135201272 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f440092e0 con 0x7f6f4c0cd6e0 2023-10-08T00:15:44.739 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.737+0000 7f6f52fbf700 1 -- 172.21.15.99:0/135201272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f4c0cd6e0 msgr2=0x7f6f4c0cbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:44.739 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.737+0000 7f6f52fbf700 1 --2- 172.21.15.99:0/135201272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f4c0cd6e0 0x7f6f4c0cbc50 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f6f44009f20 tx=0x7f6f44009840 comp rx=0 tx=0).stop 2023-10-08T00:15:44.739 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.737+0000 7f6f52fbf700 1 -- 172.21.15.99:0/135201272 shutdown_connections 2023-10-08T00:15:44.739 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.737+0000 7f6f52fbf700 1 --2- 172.21.15.99:0/135201272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f4c0cd6e0 0x7f6f4c0cbc50 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:44.739 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.737+0000 7f6f52fbf700 1 -- 172.21.15.99:0/135201272 >> 172.21.15.99:0/135201272 conn(0x7f6f4c067660 msgr2=0x7f6f4c067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:44.739 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.737+0000 7f6f52fbf700 1 -- 172.21.15.99:0/135201272 shutdown_connections 2023-10-08T00:15:44.739 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.737+0000 7f6f52fbf700 1 -- 172.21.15.99:0/135201272 wait complete. 2023-10-08T00:15:44.741 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.738+0000 7f6f52fbf700 1 Processor -- start 2023-10-08T00:15:44.741 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.738+0000 7f6f52fbf700 1 -- start start 2023-10-08T00:15:44.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.739+0000 7f6f52fbf700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f4c0cd6e0 0x7f6f4c150220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:44.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.739+0000 7f6f52fbf700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f4c0cdab0 0x7f6f4c0b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:44.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.739+0000 7f6f52fbf700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f4c15cbd0 0x7f6f4c151cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:44.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.739+0000 7f6f52fbf700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4c150760 con 0x7f6f4c0cd6e0 2023-10-08T00:15:44.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.739+0000 7f6f52fbf700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4c0b3f50 con 0x7f6f4c15cbd0 2023-10-08T00:15:44.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.739+0000 7f6f52fbf700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4c153550 con 0x7f6f4c0cdab0 2023-10-08T00:15:44.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.739+0000 7f6f51fbd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f4c0cd6e0 0x7f6f4c150220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:44.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.739+0000 7f6f51fbd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f4c0cd6e0 0x7f6f4c150220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40902/0 (socket says 172.21.15.99:40902) 2023-10-08T00:15:44.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.739+0000 7f6f51fbd700 1 -- 172.21.15.99:0/2436717712 learned_addr learned my addr 172.21.15.99:0/2436717712 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:44.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.739+0000 7f6f527be700 1 --2- 172.21.15.99:0/2436717712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f4c15cbd0 0x7f6f4c151cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:44.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.739+0000 7f6f517bc700 1 --2- 172.21.15.99:0/2436717712 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f4c0cdab0 0x7f6f4c0b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:44.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.739+0000 7f6f51fbd700 1 -- 172.21.15.99:0/2436717712 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f4c0cdab0 msgr2=0x7f6f4c0b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:44.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.739+0000 7f6f51fbd700 1 --2- 172.21.15.99:0/2436717712 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f4c0cdab0 0x7f6f4c0b3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:44.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.739+0000 7f6f51fbd700 1 -- 172.21.15.99:0/2436717712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f4c15cbd0 msgr2=0x7f6f4c151cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:44.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.739+0000 7f6f51fbd700 1 --2- 172.21.15.99:0/2436717712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f4c15cbd0 0x7f6f4c151cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:44.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.739+0000 7f6f51fbd700 1 -- 172.21.15.99:0/2436717712 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f44009ae0 con 0x7f6f4c0cd6e0 2023-10-08T00:15:44.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.739+0000 7f6f517bc700 1 --2- 172.21.15.99:0/2436717712 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f4c0cdab0 0x7f6f4c0b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:15:44.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.739+0000 7f6f51fbd700 1 --2- 172.21.15.99:0/2436717712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f4c0cd6e0 0x7f6f4c150220 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f6f4400b6c0 tx=0x7f6f44017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:44.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.739+0000 7f6f41ffb700 1 -- 172.21.15.99:0/2436717712 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f44012aa0 con 0x7f6f4c0cd6e0 2023-10-08T00:15:44.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.739+0000 7f6f41ffb700 1 -- 172.21.15.99:0/2436717712 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f44012c00 con 0x7f6f4c0cd6e0 2023-10-08T00:15:44.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.739+0000 7f6f52fbf700 1 -- 172.21.15.99:0/2436717712 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f4c0b4180 con 0x7f6f4c0cd6e0 2023-10-08T00:15:44.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.739+0000 7f6f41ffb700 1 -- 172.21.15.99:0/2436717712 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4401e430 con 0x7f6f4c0cd6e0 2023-10-08T00:15:44.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.740+0000 7f6f52fbf700 1 -- 172.21.15.99:0/2436717712 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f4c0b4740 con 0x7f6f4c0cd6e0 2023-10-08T00:15:44.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.741+0000 7f6f52fbf700 1 -- 172.21.15.99:0/2436717712 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f4c151610 con 0x7f6f4c0cd6e0 2023-10-08T00:15:44.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.741+0000 7f6f41ffb700 1 -- 172.21.15.99:0/2436717712 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6f4401e820 con 0x7f6f4c0cd6e0 2023-10-08T00:15:44.744 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.741+0000 7f6f41ffb700 1 --2- 172.21.15.99:0/2436717712 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6f30066f30 0x7f6f300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:44.744 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.742+0000 7f6f41ffb700 1 -- 172.21.15.99:0/2436717712 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6f44086000 con 0x7f6f4c0cd6e0 2023-10-08T00:15:44.744 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.742+0000 7f6f517bc700 1 --2- 172.21.15.99:0/2436717712 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6f30066f30 0x7f6f300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:44.744 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.742+0000 7f6f517bc700 1 --2- 172.21.15.99:0/2436717712 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6f30066f30 0x7f6f300693e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c0b5fe0 tx=0x7f6f3c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:44.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.746+0000 7f6f41ffb700 1 -- 172.21.15.99:0/2436717712 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6f4c151610 con 0x7f6f4c0cd6e0 2023-10-08T00:15:44.929 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.927+0000 7f6f52fbf700 1 -- 172.21.15.99:0/2436717712 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6f4c151610 con 0x7f6f4c0cd6e0 2023-10-08T00:15:44.929 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.928+0000 7f6f41ffb700 1 -- 172.21.15.99:0/2436717712 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+15364 (secure 0 0 0) 0x7f6f4c151610 con 0x7f6f4c0cd6e0 2023-10-08T00:15:44.930 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:15:44.930 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":104,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:15:42.099867+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:15:38.848324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-08T00:15:39.561379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:15:44.932 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.930+0000 7f6f52fbf700 1 -- 172.21.15.99:0/2436717712 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6f30066f30 msgr2=0x7f6f300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:44.932 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.930+0000 7f6f52fbf700 1 --2- 172.21.15.99:0/2436717712 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6f30066f30 0x7f6f300693e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c0b5fe0 tx=0x7f6f3c009040 comp rx=0 tx=0).stop 2023-10-08T00:15:44.932 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.930+0000 7f6f52fbf700 1 -- 172.21.15.99:0/2436717712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f4c0cd6e0 msgr2=0x7f6f4c150220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:44.932 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.930+0000 7f6f52fbf700 1 --2- 172.21.15.99:0/2436717712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f4c0cd6e0 0x7f6f4c150220 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f6f4400b6c0 tx=0x7f6f44017000 comp rx=0 tx=0).stop 2023-10-08T00:15:44.933 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.930+0000 7f6f52fbf700 1 -- 172.21.15.99:0/2436717712 shutdown_connections 2023-10-08T00:15:44.933 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.930+0000 7f6f52fbf700 1 --2- 172.21.15.99:0/2436717712 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6f30066f30 0x7f6f300693e0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:44.933 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.930+0000 7f6f52fbf700 1 --2- 172.21.15.99:0/2436717712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f4c15cbd0 0x7f6f4c151cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:44.933 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.930+0000 7f6f52fbf700 1 --2- 172.21.15.99:0/2436717712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f4c0cd6e0 0x7f6f4c150220 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:44.933 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.930+0000 7f6f52fbf700 1 --2- 172.21.15.99:0/2436717712 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f4c0cdab0 0x7f6f4c0b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:44.934 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.930+0000 7f6f52fbf700 1 -- 172.21.15.99:0/2436717712 >> 172.21.15.99:0/2436717712 conn(0x7f6f4c067660 msgr2=0x7f6f4c0d15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:44.934 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.930+0000 7f6f52fbf700 1 -- 172.21.15.99:0/2436717712 shutdown_connections 2023-10-08T00:15:44.934 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:44.930+0000 7f6f52fbf700 1 -- 172.21.15.99:0/2436717712 wait complete. 2023-10-08T00:15:44.944 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T00:15:44.944 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T00:15:45.059 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.058+0000 7f8a54f16700 1 Processor -- start 2023-10-08T00:15:45.059 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.058+0000 7f8a54f16700 1 -- start start 2023-10-08T00:15:45.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.059+0000 7f8a54f16700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8a50157870 0x7f8a5015b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:45.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.059+0000 7f8a54f16700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a50157c40 0x7f8a50158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:45.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.059+0000 7f8a54f16700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a5015cbb0 con 0x7f8a5015a620 2023-10-08T00:15:45.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.059+0000 7f8a54f16700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a5015bfa0 con 0x7f8a50157870 2023-10-08T00:15:45.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.059+0000 7f8a54f16700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5015c280 con 0x7f8a50157c40 2023-10-08T00:15:45.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.059+0000 7f8a4e59c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8a50157870 0x7f8a5015b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:45.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.059+0000 7f8a4e59c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8a50157870 0x7f8a5015b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40908/0 (socket says 172.21.15.99:40908) 2023-10-08T00:15:45.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.059+0000 7f8a4e59c700 1 -- 172.21.15.99:0/1866284810 learned_addr learned my addr 172.21.15.99:0/1866284810 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:45.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.059+0000 7f8a4ed9d700 1 --2- 172.21.15.99:0/1866284810 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a50157c40 0x7f8a50158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:45.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.059+0000 7f8a4e59c700 1 -- 172.21.15.99:0/1866284810 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a50157c40 msgr2=0x7f8a50158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:45.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.059+0000 7f8a4e59c700 1 --2- 172.21.15.99:0/1866284810 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a50157c40 0x7f8a50158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:45.064 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.059+0000 7f8a4e59c700 1 -- 172.21.15.99:0/1866284810 >> v1:172.21.15.191:6789/0 conn(0x7f8a5015a620 legacy=0x7f8a5015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:45.064 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.059+0000 7f8a4e59c700 1 -- 172.21.15.99:0/1866284810 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a5015c560 con 0x7f8a50157870 2023-10-08T00:15:45.064 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.059+0000 7f8a4ed9d700 1 --2- 172.21.15.99:0/1866284810 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a50157c40 0x7f8a50158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:15:45.064 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.060+0000 7f8a4e59c700 1 --2- 172.21.15.99:0/1866284810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a50157870 0x7f8a5015b6a0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f8a400099b0 tx=0x7f8a40009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4f90e0ca232321e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:45.064 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.060+0000 7f8a4dd9b700 1 -- 172.21.15.99:0/1866284810 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4000b860 con 0x7f8a50157870 2023-10-08T00:15:45.064 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.060+0000 7f8a4dd9b700 1 -- 172.21.15.99:0/1866284810 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a4000b9c0 con 0x7f8a50157870 2023-10-08T00:15:45.064 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.060+0000 7f8a4dd9b700 1 -- 172.21.15.99:0/1866284810 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4000bd60 con 0x7f8a50157870 2023-10-08T00:15:45.064 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.060+0000 7f8a54f16700 1 -- 172.21.15.99:0/1866284810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a50157870 msgr2=0x7f8a5015b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:45.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.060+0000 7f8a54f16700 1 --2- 172.21.15.99:0/1866284810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a50157870 0x7f8a5015b6a0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f8a400099b0 tx=0x7f8a40009250 comp rx=0 tx=0).stop 2023-10-08T00:15:45.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.060+0000 7f8a54f16700 1 -- 172.21.15.99:0/1866284810 shutdown_connections 2023-10-08T00:15:45.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.060+0000 7f8a54f16700 1 --2- 172.21.15.99:0/1866284810 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a50157c40 0x7f8a50158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:45.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.060+0000 7f8a54f16700 1 --2- 172.21.15.99:0/1866284810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a50157870 0x7f8a5015b6a0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:45.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.060+0000 7f8a54f16700 1 -- 172.21.15.99:0/1866284810 >> 172.21.15.99:0/1866284810 conn(0x7f8a50068ce0 msgr2=0x7f8a500690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:45.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.060+0000 7f8a54f16700 1 -- 172.21.15.99:0/1866284810 shutdown_connections 2023-10-08T00:15:45.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.060+0000 7f8a54f16700 1 -- 172.21.15.99:0/1866284810 wait complete. 2023-10-08T00:15:45.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.062+0000 7f8a54f16700 1 Processor -- start 2023-10-08T00:15:45.066 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.062+0000 7f8a54f16700 1 -- start start 2023-10-08T00:15:45.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.062+0000 7f8a54f16700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a50157870 0x7f8a50147ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:45.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.062+0000 7f8a54f16700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a50157c40 0x7f8a500c5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:45.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.062+0000 7f8a54f16700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a5015a620 0x7f8a500c6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:45.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.062+0000 7f8a54f16700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a501486d0 con 0x7f8a5015a620 2023-10-08T00:15:45.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.062+0000 7f8a54f16700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5015d300 con 0x7f8a50157870 2023-10-08T00:15:45.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.062+0000 7f8a54f16700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a500c6680 con 0x7f8a50157c40 2023-10-08T00:15:45.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.062+0000 7f8a4ed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a5015a620 0x7f8a500c6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:45.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.062+0000 7f8a4ed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a5015a620 0x7f8a500c6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40922/0 (socket says 172.21.15.99:40922) 2023-10-08T00:15:45.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.062+0000 7f8a4ed9d700 1 -- 172.21.15.99:0/1313601036 learned_addr learned my addr 172.21.15.99:0/1313601036 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:45.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.063+0000 7f8a4e59c700 1 --2- 172.21.15.99:0/1313601036 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a50157870 0x7f8a50147ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:45.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.063+0000 7f8a45d9b700 1 --2- 172.21.15.99:0/1313601036 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a50157c40 0x7f8a500c5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:45.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.063+0000 7f8a4ed9d700 1 -- 172.21.15.99:0/1313601036 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a50157c40 msgr2=0x7f8a500c5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:45.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.063+0000 7f8a4ed9d700 1 --2- 172.21.15.99:0/1313601036 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a50157c40 0x7f8a500c5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:45.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.063+0000 7f8a4ed9d700 1 -- 172.21.15.99:0/1313601036 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a50157870 msgr2=0x7f8a50147ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:45.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.063+0000 7f8a4ed9d700 1 --2- 172.21.15.99:0/1313601036 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a50157870 0x7f8a50147ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:45.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.063+0000 7f8a4ed9d700 1 -- 172.21.15.99:0/1313601036 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a400094f0 con 0x7f8a5015a620 2023-10-08T00:15:45.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.063+0000 7f8a4ed9d700 1 --2- 172.21.15.99:0/1313601036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a5015a620 0x7f8a500c6050 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f8a4800d150 tx=0x7f8a48009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:45.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.063+0000 7f8a46ffd700 1 -- 172.21.15.99:0/1313601036 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4800ddf0 con 0x7f8a5015a620 2023-10-08T00:15:45.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.063+0000 7f8a46ffd700 1 -- 172.21.15.99:0/1313601036 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a4800f040 con 0x7f8a5015a620 2023-10-08T00:15:45.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.063+0000 7f8a54f16700 1 -- 172.21.15.99:0/1313601036 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a500c6960 con 0x7f8a5015a620 2023-10-08T00:15:45.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.063+0000 7f8a46ffd700 1 -- 172.21.15.99:0/1313601036 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a48015690 con 0x7f8a5015a620 2023-10-08T00:15:45.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.063+0000 7f8a54f16700 1 -- 172.21.15.99:0/1313601036 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a500c6f20 con 0x7f8a5015a620 2023-10-08T00:15:45.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.064+0000 7f8a54f16700 1 -- 172.21.15.99:0/1313601036 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a5015d520 con 0x7f8a5015a620 2023-10-08T00:15:45.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.064+0000 7f8a46ffd700 1 -- 172.21.15.99:0/1313601036 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8a4801e3b0 con 0x7f8a5015a620 2023-10-08T00:15:45.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.065+0000 7f8a46ffd700 1 --2- 172.21.15.99:0/1313601036 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8a30066f30 0x7f8a300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:45.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.065+0000 7f8a46ffd700 1 -- 172.21.15.99:0/1313601036 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8a48084b00 con 0x7f8a5015a620 2023-10-08T00:15:45.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.065+0000 7f8a4e59c700 1 --2- 172.21.15.99:0/1313601036 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8a30066f30 0x7f8a300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:45.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.065+0000 7f8a4e59c700 1 --2- 172.21.15.99:0/1313601036 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8a30066f30 0x7f8a300693e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8a400099b0 tx=0x7f8a40017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:45.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.069+0000 7f8a46ffd700 1 -- 172.21.15.99:0/1313601036 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8a5015d520 con 0x7f8a5015a620 2023-10-08T00:15:45.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.248+0000 7f8a54f16700 1 -- 172.21.15.99:0/1313601036 --> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8a5015d520 con 0x7f8a30066f30 2023-10-08T00:15:45.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.257+0000 7f8a46ffd700 1 -- 172.21.15.99:0/1313601036 <== mgr.4099 v2:172.21.15.191:6832/85985 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285544 (secure 0 0 0) 0x7f8a5015d520 con 0x7f8a30066f30 2023-10-08T00:15:45.261 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:15:45.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.260+0000 7f8a54f16700 1 -- 172.21.15.99:0/1313601036 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8a30066f30 msgr2=0x7f8a300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:45.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.260+0000 7f8a54f16700 1 --2- 172.21.15.99:0/1313601036 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8a30066f30 0x7f8a300693e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8a400099b0 tx=0x7f8a40017000 comp rx=0 tx=0).stop 2023-10-08T00:15:45.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.260+0000 7f8a54f16700 1 -- 172.21.15.99:0/1313601036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a5015a620 msgr2=0x7f8a500c6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:45.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.260+0000 7f8a54f16700 1 --2- 172.21.15.99:0/1313601036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a5015a620 0x7f8a500c6050 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f8a4800d150 tx=0x7f8a48009a50 comp rx=0 tx=0).stop 2023-10-08T00:15:45.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.260+0000 7f8a54f16700 1 -- 172.21.15.99:0/1313601036 shutdown_connections 2023-10-08T00:15:45.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.260+0000 7f8a54f16700 1 --2- 172.21.15.99:0/1313601036 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8a30066f30 0x7f8a300693e0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:45.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.260+0000 7f8a54f16700 1 --2- 172.21.15.99:0/1313601036 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a50157870 0x7f8a50147ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:45.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.260+0000 7f8a54f16700 1 --2- 172.21.15.99:0/1313601036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a5015a620 0x7f8a500c6050 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:45.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.260+0000 7f8a54f16700 1 --2- 172.21.15.99:0/1313601036 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a50157c40 0x7f8a500c5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:45.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.260+0000 7f8a54f16700 1 -- 172.21.15.99:0/1313601036 >> 172.21.15.99:0/1313601036 conn(0x7f8a50068ce0 msgr2=0x7f8a500b46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:45.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.260+0000 7f8a54f16700 1 -- 172.21.15.99:0/1313601036 shutdown_connections 2023-10-08T00:15:45.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.260+0000 7f8a54f16700 1 -- 172.21.15.99:0/1313601036 wait complete. 2023-10-08T00:15:45.265 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2023-10-08T00:15:45.273 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":245,"stamp":"2023-10-08T00:15:44.093941+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":135,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":5562,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1556937,"statfs":{"total":1600000000,"available":1594304396,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.877858"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.112547+0000","last_change":"2023-10-08T00:15:41.112491+0000","last_active":"2023-10-08T00:15:41.112547+0000","last_peered":"2023-10-08T00:15:41.112547+0000","last_clean":"2023-10-08T00:15:41.112547+0000","last_became_active":"2023-10-08T00:15:41.112292+0000","last_became_peered":"2023-10-08T00:15:41.112292+0000","last_unstale":"2023-10-08T00:15:41.112547+0000","last_undegraded":"2023-10-08T00:15:41.112547+0000","last_fullsized":"2023-10-08T00:15:41.112547+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:23:45.765124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.111895+0000","last_change":"2023-10-08T00:15:41.111823+0000","last_active":"2023-10-08T00:15:41.111895+0000","last_peered":"2023-10-08T00:15:41.111895+0000","last_clean":"2023-10-08T00:15:41.111895+0000","last_became_active":"2023-10-08T00:15:41.111520+0000","last_became_peered":"2023-10-08T00:15:41.111520+0000","last_unstale":"2023-10-08T00:15:41.111895+0000","last_undegraded":"2023-10-08T00:15:41.111895+0000","last_fullsized":"2023-10-08T00:15:41.111895+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:23:45.765124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.106044+0000","last_change":"2023-10-08T00:15:41.105499+0000","last_active":"2023-10-08T00:15:41.106044+0000","last_peered":"2023-10-08T00:15:41.106044+0000","last_clean":"2023-10-08T00:15:41.106044+0000","last_became_active":"2023-10-08T00:15:41.105224+0000","last_became_peered":"2023-10-08T00:15:41.105224+0000","last_unstale":"2023-10-08T00:15:41.106044+0000","last_undegraded":"2023-10-08T00:15:41.106044+0000","last_fullsized":"2023-10-08T00:15:41.106044+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:48:22.786963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.107949+0000","last_change":"2023-10-08T00:15:41.107872+0000","last_active":"2023-10-08T00:15:41.107949+0000","last_peered":"2023-10-08T00:15:41.107949+0000","last_clean":"2023-10-08T00:15:41.107949+0000","last_became_active":"2023-10-08T00:15:41.107608+0000","last_became_peered":"2023-10-08T00:15:41.107608+0000","last_unstale":"2023-10-08T00:15:41.107949+0000","last_undegraded":"2023-10-08T00:15:41.107949+0000","last_fullsized":"2023-10-08T00:15:41.107949+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:07:58.834149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.106621+0000","last_change":"2023-10-08T00:15:41.106566+0000","last_active":"2023-10-08T00:15:41.106621+0000","last_peered":"2023-10-08T00:15:41.106621+0000","last_clean":"2023-10-08T00:15:41.106621+0000","last_became_active":"2023-10-08T00:15:41.106418+0000","last_became_peered":"2023-10-08T00:15:41.106418+0000","last_unstale":"2023-10-08T00:15:41.106621+0000","last_undegraded":"2023-10-08T00:15:41.106621+0000","last_fullsized":"2023-10-08T00:15:41.106621+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:36:32.485025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.106061+0000","last_change":"2023-10-08T00:15:41.105616+0000","last_active":"2023-10-08T00:15:41.106061+0000","last_peered":"2023-10-08T00:15:41.106061+0000","last_clean":"2023-10-08T00:15:41.106061+0000","last_became_active":"2023-10-08T00:15:41.105113+0000","last_became_peered":"2023-10-08T00:15:41.105113+0000","last_unstale":"2023-10-08T00:15:41.106061+0000","last_undegraded":"2023-10-08T00:15:41.106061+0000","last_fullsized":"2023-10-08T00:15:41.106061+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:39:53.717841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.110919+0000","last_change":"2023-10-08T00:15:41.110051+0000","last_active":"2023-10-08T00:15:41.110919+0000","last_peered":"2023-10-08T00:15:41.110919+0000","last_clean":"2023-10-08T00:15:41.110919+0000","last_became_active":"2023-10-08T00:15:41.109788+0000","last_became_peered":"2023-10-08T00:15:41.109788+0000","last_unstale":"2023-10-08T00:15:41.110919+0000","last_undegraded":"2023-10-08T00:15:41.110919+0000","last_fullsized":"2023-10-08T00:15:41.110919+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:35:02.136255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.116566+0000","last_change":"2023-10-08T00:15:41.116506+0000","last_active":"2023-10-08T00:15:41.116566+0000","last_peered":"2023-10-08T00:15:41.116566+0000","last_clean":"2023-10-08T00:15:41.116566+0000","last_became_active":"2023-10-08T00:15:41.116266+0000","last_became_peered":"2023-10-08T00:15:41.116266+0000","last_unstale":"2023-10-08T00:15:41.116566+0000","last_undegraded":"2023-10-08T00:15:41.116566+0000","last_fullsized":"2023-10-08T00:15:41.116566+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:19:15.421134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.113328+0000","last_change":"2023-10-08T00:15:41.113218+0000","last_active":"2023-10-08T00:15:41.113328+0000","last_peered":"2023-10-08T00:15:41.113328+0000","last_clean":"2023-10-08T00:15:41.113328+0000","last_became_active":"2023-10-08T00:15:41.112860+0000","last_became_peered":"2023-10-08T00:15:41.112860+0000","last_unstale":"2023-10-08T00:15:41.113328+0000","last_undegraded":"2023-10-08T00:15:41.113328+0000","last_fullsized":"2023-10-08T00:15:41.113328+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:46:54.920309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.111913+0000","last_change":"2023-10-08T00:15:41.111859+0000","last_active":"2023-10-08T00:15:41.111913+0000","last_peered":"2023-10-08T00:15:41.111913+0000","last_clean":"2023-10-08T00:15:41.111913+0000","last_became_active":"2023-10-08T00:15:41.111610+0000","last_became_peered":"2023-10-08T00:15:41.111610+0000","last_unstale":"2023-10-08T00:15:41.111913+0000","last_undegraded":"2023-10-08T00:15:41.111913+0000","last_fullsized":"2023-10-08T00:15:41.111913+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:50:30.128925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.115601+0000","last_change":"2023-10-08T00:15:41.115543+0000","last_active":"2023-10-08T00:15:41.115601+0000","last_peered":"2023-10-08T00:15:41.115601+0000","last_clean":"2023-10-08T00:15:41.115601+0000","last_became_active":"2023-10-08T00:15:41.115204+0000","last_became_peered":"2023-10-08T00:15:41.115204+0000","last_unstale":"2023-10-08T00:15:41.115601+0000","last_undegraded":"2023-10-08T00:15:41.115601+0000","last_fullsized":"2023-10-08T00:15:41.115601+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:31:07.729785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.108863+0000","last_change":"2023-10-08T00:15:41.108679+0000","last_active":"2023-10-08T00:15:41.108863+0000","last_peered":"2023-10-08T00:15:41.108863+0000","last_clean":"2023-10-08T00:15:41.108863+0000","last_became_active":"2023-10-08T00:15:41.108464+0000","last_became_peered":"2023-10-08T00:15:41.108464+0000","last_unstale":"2023-10-08T00:15:41.108863+0000","last_undegraded":"2023-10-08T00:15:41.108863+0000","last_fullsized":"2023-10-08T00:15:41.108863+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:39:53.717841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.107046+0000","last_change":"2023-10-08T00:15:41.106987+0000","last_active":"2023-10-08T00:15:41.107046+0000","last_peered":"2023-10-08T00:15:41.107046+0000","last_clean":"2023-10-08T00:15:41.107046+0000","last_became_active":"2023-10-08T00:15:41.106703+0000","last_became_peered":"2023-10-08T00:15:41.106703+0000","last_unstale":"2023-10-08T00:15:41.107046+0000","last_undegraded":"2023-10-08T00:15:41.107046+0000","last_fullsized":"2023-10-08T00:15:41.107046+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:52:35.721354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.115133+0000","last_change":"2023-10-08T00:15:41.113678+0000","last_active":"2023-10-08T00:15:41.115133+0000","last_peered":"2023-10-08T00:15:41.115133+0000","last_clean":"2023-10-08T00:15:41.115133+0000","last_became_active":"2023-10-08T00:15:41.108892+0000","last_became_peered":"2023-10-08T00:15:41.108892+0000","last_unstale":"2023-10-08T00:15:41.115133+0000","last_undegraded":"2023-10-08T00:15:41.115133+0000","last_fullsized":"2023-10-08T00:15:41.115133+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:29:22.992476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.115233+0000","last_change":"2023-10-08T00:15:41.115164+0000","last_active":"2023-10-08T00:15:41.115233+0000","last_peered":"2023-10-08T00:15:41.115233+0000","last_clean":"2023-10-08T00:15:41.115233+0000","last_became_active":"2023-10-08T00:15:41.114890+0000","last_became_peered":"2023-10-08T00:15:41.114890+0000","last_unstale":"2023-10-08T00:15:41.115233+0000","last_undegraded":"2023-10-08T00:15:41.115233+0000","last_fullsized":"2023-10-08T00:15:41.115233+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:03:01.394697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.104980+0000","last_change":"2023-10-08T00:15:41.104211+0000","last_active":"2023-10-08T00:15:41.104980+0000","last_peered":"2023-10-08T00:15:41.104980+0000","last_clean":"2023-10-08T00:15:41.104980+0000","last_became_active":"2023-10-08T00:15:41.103996+0000","last_became_peered":"2023-10-08T00:15:41.103996+0000","last_unstale":"2023-10-08T00:15:41.104980+0000","last_undegraded":"2023-10-08T00:15:41.104980+0000","last_fullsized":"2023-10-08T00:15:41.104980+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:54:58.905309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.115634+0000","last_change":"2023-10-08T00:15:41.115577+0000","last_active":"2023-10-08T00:15:41.115634+0000","last_peered":"2023-10-08T00:15:41.115634+0000","last_clean":"2023-10-08T00:15:41.115634+0000","last_became_active":"2023-10-08T00:15:41.115336+0000","last_became_peered":"2023-10-08T00:15:41.115336+0000","last_unstale":"2023-10-08T00:15:41.115634+0000","last_undegraded":"2023-10-08T00:15:41.115634+0000","last_fullsized":"2023-10-08T00:15:41.115634+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:49:44.152669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.117365+0000","last_change":"2023-10-08T00:15:41.117314+0000","last_active":"2023-10-08T00:15:41.117365+0000","last_peered":"2023-10-08T00:15:41.117365+0000","last_clean":"2023-10-08T00:15:41.117365+0000","last_became_active":"2023-10-08T00:15:41.116388+0000","last_became_peered":"2023-10-08T00:15:41.116388+0000","last_unstale":"2023-10-08T00:15:41.117365+0000","last_undegraded":"2023-10-08T00:15:41.117365+0000","last_fullsized":"2023-10-08T00:15:41.117365+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:35:04.242399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.115153+0000","last_change":"2023-10-08T00:15:41.113825+0000","last_active":"2023-10-08T00:15:41.115153+0000","last_peered":"2023-10-08T00:15:41.115153+0000","last_clean":"2023-10-08T00:15:41.115153+0000","last_became_active":"2023-10-08T00:15:41.109229+0000","last_became_peered":"2023-10-08T00:15:41.109229+0000","last_unstale":"2023-10-08T00:15:41.115153+0000","last_undegraded":"2023-10-08T00:15:41.115153+0000","last_fullsized":"2023-10-08T00:15:41.115153+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:15:38.694742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.110825+0000","last_change":"2023-10-08T00:15:41.109240+0000","last_active":"2023-10-08T00:15:41.110825+0000","last_peered":"2023-10-08T00:15:41.110825+0000","last_clean":"2023-10-08T00:15:41.110825+0000","last_became_active":"2023-10-08T00:15:41.108693+0000","last_became_peered":"2023-10-08T00:15:41.108693+0000","last_unstale":"2023-10-08T00:15:41.110825+0000","last_undegraded":"2023-10-08T00:15:41.110825+0000","last_fullsized":"2023-10-08T00:15:41.110825+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:31:07.729785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.113966+0000","last_change":"2023-10-08T00:15:41.113848+0000","last_active":"2023-10-08T00:15:41.113966+0000","last_peered":"2023-10-08T00:15:41.113966+0000","last_clean":"2023-10-08T00:15:41.113966+0000","last_became_active":"2023-10-08T00:15:41.113560+0000","last_became_peered":"2023-10-08T00:15:41.113560+0000","last_unstale":"2023-10-08T00:15:41.113966+0000","last_undegraded":"2023-10-08T00:15:41.113966+0000","last_fullsized":"2023-10-08T00:15:41.113966+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:16:04.386776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.108544+0000","last_change":"2023-10-08T00:15:41.108457+0000","last_active":"2023-10-08T00:15:41.108544+0000","last_peered":"2023-10-08T00:15:41.108544+0000","last_clean":"2023-10-08T00:15:41.108544+0000","last_became_active":"2023-10-08T00:15:41.108218+0000","last_became_peered":"2023-10-08T00:15:41.108218+0000","last_unstale":"2023-10-08T00:15:41.108544+0000","last_undegraded":"2023-10-08T00:15:41.108544+0000","last_fullsized":"2023-10-08T00:15:41.108544+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:13:13.842849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.115485+0000","last_change":"2023-10-08T00:15:41.115417+0000","last_active":"2023-10-08T00:15:41.115485+0000","last_peered":"2023-10-08T00:15:41.115485+0000","last_clean":"2023-10-08T00:15:41.115485+0000","last_became_active":"2023-10-08T00:15:41.115200+0000","last_became_peered":"2023-10-08T00:15:41.115200+0000","last_unstale":"2023-10-08T00:15:41.115485+0000","last_undegraded":"2023-10-08T00:15:41.115485+0000","last_fullsized":"2023-10-08T00:15:41.115485+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:50:30.128925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.115853+0000","last_change":"2023-10-08T00:15:41.115769+0000","last_active":"2023-10-08T00:15:41.115853+0000","last_peered":"2023-10-08T00:15:41.115853+0000","last_clean":"2023-10-08T00:15:41.115853+0000","last_became_active":"2023-10-08T00:15:41.113214+0000","last_became_peered":"2023-10-08T00:15:41.113214+0000","last_unstale":"2023-10-08T00:15:41.115853+0000","last_undegraded":"2023-10-08T00:15:41.115853+0000","last_fullsized":"2023-10-08T00:15:41.115853+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:38:56.511100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.109001+0000","last_change":"2023-10-08T00:15:41.108934+0000","last_active":"2023-10-08T00:15:41.109001+0000","last_peered":"2023-10-08T00:15:41.109001+0000","last_clean":"2023-10-08T00:15:41.109001+0000","last_became_active":"2023-10-08T00:15:41.108701+0000","last_became_peered":"2023-10-08T00:15:41.108701+0000","last_unstale":"2023-10-08T00:15:41.109001+0000","last_undegraded":"2023-10-08T00:15:41.109001+0000","last_fullsized":"2023-10-08T00:15:41.109001+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:48:51.428292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.111745+0000","last_change":"2023-10-08T00:15:41.111689+0000","last_active":"2023-10-08T00:15:41.111745+0000","last_peered":"2023-10-08T00:15:41.111745+0000","last_clean":"2023-10-08T00:15:41.111745+0000","last_became_active":"2023-10-08T00:15:41.111429+0000","last_became_peered":"2023-10-08T00:15:41.111429+0000","last_unstale":"2023-10-08T00:15:41.111745+0000","last_undegraded":"2023-10-08T00:15:41.111745+0000","last_fullsized":"2023-10-08T00:15:41.111745+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:15:30.961386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.109528+0000","last_change":"2023-10-08T00:15:41.109166+0000","last_active":"2023-10-08T00:15:41.109528+0000","last_peered":"2023-10-08T00:15:41.109528+0000","last_clean":"2023-10-08T00:15:41.109528+0000","last_became_active":"2023-10-08T00:15:41.108621+0000","last_became_peered":"2023-10-08T00:15:41.108621+0000","last_unstale":"2023-10-08T00:15:41.109528+0000","last_undegraded":"2023-10-08T00:15:41.109528+0000","last_fullsized":"2023-10-08T00:15:41.109528+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:03:25.736301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.112092+0000","last_change":"2023-10-08T00:15:41.112057+0000","last_active":"2023-10-08T00:15:41.112092+0000","last_peered":"2023-10-08T00:15:41.112092+0000","last_clean":"2023-10-08T00:15:41.112092+0000","last_became_active":"2023-10-08T00:15:41.111797+0000","last_became_peered":"2023-10-08T00:15:41.111797+0000","last_unstale":"2023-10-08T00:15:41.112092+0000","last_undegraded":"2023-10-08T00:15:41.112092+0000","last_fullsized":"2023-10-08T00:15:41.112092+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:17:01.578223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.110515+0000","last_change":"2023-10-08T00:15:41.110429+0000","last_active":"2023-10-08T00:15:41.110515+0000","last_peered":"2023-10-08T00:15:41.110515+0000","last_clean":"2023-10-08T00:15:41.110515+0000","last_became_active":"2023-10-08T00:15:41.110195+0000","last_became_peered":"2023-10-08T00:15:41.110195+0000","last_unstale":"2023-10-08T00:15:41.110515+0000","last_undegraded":"2023-10-08T00:15:41.110515+0000","last_fullsized":"2023-10-08T00:15:41.110515+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:29:08.856896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.111544+0000","last_change":"2023-10-08T00:15:41.111281+0000","last_active":"2023-10-08T00:15:41.111544+0000","last_peered":"2023-10-08T00:15:41.111544+0000","last_clean":"2023-10-08T00:15:41.111544+0000","last_became_active":"2023-10-08T00:15:41.109002+0000","last_became_peered":"2023-10-08T00:15:41.109002+0000","last_unstale":"2023-10-08T00:15:41.111544+0000","last_undegraded":"2023-10-08T00:15:41.111544+0000","last_fullsized":"2023-10-08T00:15:41.111544+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:35:04.242399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.106076+0000","last_change":"2023-10-08T00:15:41.105740+0000","last_active":"2023-10-08T00:15:41.106076+0000","last_peered":"2023-10-08T00:15:41.106076+0000","last_clean":"2023-10-08T00:15:41.106076+0000","last_became_active":"2023-10-08T00:15:41.105406+0000","last_became_peered":"2023-10-08T00:15:41.105406+0000","last_unstale":"2023-10-08T00:15:41.106076+0000","last_undegraded":"2023-10-08T00:15:41.106076+0000","last_fullsized":"2023-10-08T00:15:41.106076+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:44:23.612344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.110286+0000","last_change":"2023-10-08T00:15:41.110224+0000","last_active":"2023-10-08T00:15:41.110286+0000","last_peered":"2023-10-08T00:15:41.110286+0000","last_clean":"2023-10-08T00:15:41.110286+0000","last_became_active":"2023-10-08T00:15:41.110008+0000","last_became_peered":"2023-10-08T00:15:41.110008+0000","last_unstale":"2023-10-08T00:15:41.110286+0000","last_undegraded":"2023-10-08T00:15:41.110286+0000","last_fullsized":"2023-10-08T00:15:41.110286+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:42:20.087509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.110392+0000","last_change":"2023-10-08T00:15:41.110335+0000","last_active":"2023-10-08T00:15:41.110392+0000","last_peered":"2023-10-08T00:15:41.110392+0000","last_clean":"2023-10-08T00:15:41.110392+0000","last_became_active":"2023-10-08T00:15:41.110119+0000","last_became_peered":"2023-10-08T00:15:41.110119+0000","last_unstale":"2023-10-08T00:15:41.110392+0000","last_undegraded":"2023-10-08T00:15:41.110392+0000","last_fullsized":"2023-10-08T00:15:41.110392+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:42:20.087509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"103'4","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.231623+0000","last_change":"2023-10-08T00:15:40.227818+0000","last_active":"2023-10-08T00:15:41.231623+0000","last_peered":"2023-10-08T00:15:41.231623+0000","last_clean":"2023-10-08T00:15:41.231623+0000","last_became_active":"2023-10-08T00:15:40.227592+0000","last_became_peered":"2023-10-08T00:15:40.227592+0000","last_unstale":"2023-10-08T00:15:41.231623+0000","last_undegraded":"2023-10-08T00:15:41.231623+0000","last_fullsized":"2023-10-08T00:15:41.231623+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:28:10.454328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.112008+0000","last_change":"2023-10-08T00:15:41.111951+0000","last_active":"2023-10-08T00:15:41.112008+0000","last_peered":"2023-10-08T00:15:41.112008+0000","last_clean":"2023-10-08T00:15:41.112008+0000","last_became_active":"2023-10-08T00:15:41.111570+0000","last_became_peered":"2023-10-08T00:15:41.111570+0000","last_unstale":"2023-10-08T00:15:41.112008+0000","last_undegraded":"2023-10-08T00:15:41.112008+0000","last_fullsized":"2023-10-08T00:15:41.112008+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:47:58.612685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"103'1","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.229599+0000","last_change":"2023-10-08T00:15:40.227055+0000","last_active":"2023-10-08T00:15:41.229599+0000","last_peered":"2023-10-08T00:15:41.229599+0000","last_clean":"2023-10-08T00:15:41.229599+0000","last_became_active":"2023-10-08T00:15:40.226844+0000","last_became_peered":"2023-10-08T00:15:40.226844+0000","last_unstale":"2023-10-08T00:15:41.229599+0000","last_undegraded":"2023-10-08T00:15:41.229599+0000","last_fullsized":"2023-10-08T00:15:41.229599+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:31:06.719620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.114719+0000","last_change":"2023-10-08T00:15:41.114667+0000","last_active":"2023-10-08T00:15:41.114719+0000","last_peered":"2023-10-08T00:15:41.114719+0000","last_clean":"2023-10-08T00:15:41.114719+0000","last_became_active":"2023-10-08T00:15:41.111339+0000","last_became_peered":"2023-10-08T00:15:41.111339+0000","last_unstale":"2023-10-08T00:15:41.114719+0000","last_undegraded":"2023-10-08T00:15:41.114719+0000","last_fullsized":"2023-10-08T00:15:41.114719+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:07:58.834149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"103'2","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.231179+0000","last_change":"2023-10-08T00:15:40.229368+0000","last_active":"2023-10-08T00:15:41.231179+0000","last_peered":"2023-10-08T00:15:41.231179+0000","last_clean":"2023-10-08T00:15:41.231179+0000","last_became_active":"2023-10-08T00:15:40.228729+0000","last_became_peered":"2023-10-08T00:15:40.228729+0000","last_unstale":"2023-10-08T00:15:41.231179+0000","last_undegraded":"2023-10-08T00:15:41.231179+0000","last_fullsized":"2023-10-08T00:15:41.231179+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:46:53.910144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.114055+0000","last_change":"2023-10-08T00:15:41.114002+0000","last_active":"2023-10-08T00:15:41.114055+0000","last_peered":"2023-10-08T00:15:41.114055+0000","last_clean":"2023-10-08T00:15:41.114055+0000","last_became_active":"2023-10-08T00:15:41.113373+0000","last_became_peered":"2023-10-08T00:15:41.113373+0000","last_unstale":"2023-10-08T00:15:41.114055+0000","last_undegraded":"2023-10-08T00:15:41.114055+0000","last_fullsized":"2023-10-08T00:15:41.114055+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:13:13.842849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"103'2","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.230093+0000","last_change":"2023-10-08T00:15:40.228429+0000","last_active":"2023-10-08T00:15:41.230093+0000","last_peered":"2023-10-08T00:15:41.230093+0000","last_clean":"2023-10-08T00:15:41.230093+0000","last_became_active":"2023-10-08T00:15:40.227815+0000","last_became_peered":"2023-10-08T00:15:40.227815+0000","last_unstale":"2023-10-08T00:15:41.230093+0000","last_undegraded":"2023-10-08T00:15:41.230093+0000","last_fullsized":"2023-10-08T00:15:41.230093+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:42:37.339496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.115967+0000","last_change":"2023-10-08T00:15:41.115898+0000","last_active":"2023-10-08T00:15:41.115967+0000","last_peered":"2023-10-08T00:15:41.115967+0000","last_clean":"2023-10-08T00:15:41.115967+0000","last_became_active":"2023-10-08T00:15:41.111380+0000","last_became_peered":"2023-10-08T00:15:41.111380+0000","last_unstale":"2023-10-08T00:15:41.115967+0000","last_undegraded":"2023-10-08T00:15:41.115967+0000","last_fullsized":"2023-10-08T00:15:41.115967+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:50:30.128925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"103'1","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.228885+0000","last_change":"2023-10-08T00:15:40.231216+0000","last_active":"2023-10-08T00:15:41.228885+0000","last_peered":"2023-10-08T00:15:41.228885+0000","last_clean":"2023-10-08T00:15:41.228885+0000","last_became_active":"2023-10-08T00:15:40.230960+0000","last_became_peered":"2023-10-08T00:15:40.230960+0000","last_unstale":"2023-10-08T00:15:41.228885+0000","last_undegraded":"2023-10-08T00:15:41.228885+0000","last_fullsized":"2023-10-08T00:15:41.228885+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:48:50.418127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.104996+0000","last_change":"2023-10-08T00:15:41.104468+0000","last_active":"2023-10-08T00:15:41.104996+0000","last_peered":"2023-10-08T00:15:41.104996+0000","last_clean":"2023-10-08T00:15:41.104996+0000","last_became_active":"2023-10-08T00:15:41.104261+0000","last_became_peered":"2023-10-08T00:15:41.104261+0000","last_unstale":"2023-10-08T00:15:41.104996+0000","last_undegraded":"2023-10-08T00:15:41.104996+0000","last_fullsized":"2023-10-08T00:15:41.104996+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:16:04.386776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"103'1","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.228404+0000","last_change":"2023-10-08T00:15:40.226651+0000","last_active":"2023-10-08T00:15:41.228404+0000","last_peered":"2023-10-08T00:15:41.228404+0000","last_clean":"2023-10-08T00:15:41.228404+0000","last_became_active":"2023-10-08T00:15:40.226430+0000","last_became_peered":"2023-10-08T00:15:40.226430+0000","last_unstale":"2023-10-08T00:15:41.228404+0000","last_undegraded":"2023-10-08T00:15:41.228404+0000","last_fullsized":"2023-10-08T00:15:41.228404+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:38:55.500935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.113926+0000","last_change":"2023-10-08T00:15:41.113869+0000","last_active":"2023-10-08T00:15:41.113926+0000","last_peered":"2023-10-08T00:15:41.113926+0000","last_clean":"2023-10-08T00:15:41.113926+0000","last_became_active":"2023-10-08T00:15:41.113076+0000","last_became_peered":"2023-10-08T00:15:41.113076+0000","last_unstale":"2023-10-08T00:15:41.113926+0000","last_undegraded":"2023-10-08T00:15:41.113926+0000","last_fullsized":"2023-10-08T00:15:41.113926+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:50:11.772158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.104280+0000","last_change":"2023-10-08T00:15:40.231974+0000","last_active":"2023-10-08T00:15:41.104280+0000","last_peered":"2023-10-08T00:15:41.104280+0000","last_clean":"2023-10-08T00:15:41.104280+0000","last_became_active":"2023-10-08T00:15:40.231828+0000","last_became_peered":"2023-10-08T00:15:40.231828+0000","last_unstale":"2023-10-08T00:15:41.104280+0000","last_undegraded":"2023-10-08T00:15:41.104280+0000","last_fullsized":"2023-10-08T00:15:41.104280+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:57:50.777303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.107064+0000","last_change":"2023-10-08T00:15:41.106829+0000","last_active":"2023-10-08T00:15:41.107064+0000","last_peered":"2023-10-08T00:15:41.107064+0000","last_clean":"2023-10-08T00:15:41.107064+0000","last_became_active":"2023-10-08T00:15:41.106603+0000","last_became_peered":"2023-10-08T00:15:41.106603+0000","last_unstale":"2023-10-08T00:15:41.107064+0000","last_undegraded":"2023-10-08T00:15:41.107064+0000","last_fullsized":"2023-10-08T00:15:41.107064+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:38:56.511100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.102306+0000","last_change":"2023-10-08T00:15:40.229099+0000","last_active":"2023-10-08T00:15:41.102306+0000","last_peered":"2023-10-08T00:15:41.102306+0000","last_clean":"2023-10-08T00:15:41.102306+0000","last_became_active":"2023-10-08T00:15:40.227964+0000","last_became_peered":"2023-10-08T00:15:40.227964+0000","last_unstale":"2023-10-08T00:15:41.102306+0000","last_undegraded":"2023-10-08T00:15:41.102306+0000","last_fullsized":"2023-10-08T00:15:41.102306+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:49:01.369417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.112518+0000","last_change":"2023-10-08T00:15:41.112449+0000","last_active":"2023-10-08T00:15:41.112518+0000","last_peered":"2023-10-08T00:15:41.112518+0000","last_clean":"2023-10-08T00:15:41.112518+0000","last_became_active":"2023-10-08T00:15:41.111938+0000","last_became_peered":"2023-10-08T00:15:41.111938+0000","last_unstale":"2023-10-08T00:15:41.112518+0000","last_undegraded":"2023-10-08T00:15:41.112518+0000","last_fullsized":"2023-10-08T00:15:41.112518+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:50:11.772158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.107680+0000","last_change":"2023-10-08T00:15:40.222404+0000","last_active":"2023-10-08T00:15:41.107680+0000","last_peered":"2023-10-08T00:15:41.107680+0000","last_clean":"2023-10-08T00:15:41.107680+0000","last_became_active":"2023-10-08T00:15:40.222177+0000","last_became_peered":"2023-10-08T00:15:40.222177+0000","last_unstale":"2023-10-08T00:15:41.107680+0000","last_undegraded":"2023-10-08T00:15:41.107680+0000","last_fullsized":"2023-10-08T00:15:41.107680+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:25:12.720956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.111578+0000","last_change":"2023-10-08T00:15:41.111219+0000","last_active":"2023-10-08T00:15:41.111578+0000","last_peered":"2023-10-08T00:15:41.111578+0000","last_clean":"2023-10-08T00:15:41.111578+0000","last_became_active":"2023-10-08T00:15:41.110919+0000","last_became_peered":"2023-10-08T00:15:41.110919+0000","last_unstale":"2023-10-08T00:15:41.111578+0000","last_undegraded":"2023-10-08T00:15:41.111578+0000","last_fullsized":"2023-10-08T00:15:41.111578+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:48:14.900929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.101492+0000","last_change":"2023-10-08T00:15:40.233351+0000","last_active":"2023-10-08T00:15:41.101492+0000","last_peered":"2023-10-08T00:15:41.101492+0000","last_clean":"2023-10-08T00:15:41.101492+0000","last_became_active":"2023-10-08T00:15:40.233144+0000","last_became_peered":"2023-10-08T00:15:40.233144+0000","last_unstale":"2023-10-08T00:15:41.101492+0000","last_undegraded":"2023-10-08T00:15:41.101492+0000","last_fullsized":"2023-10-08T00:15:41.101492+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:04:45.369149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.112879+0000","last_change":"2023-10-08T00:15:41.112732+0000","last_active":"2023-10-08T00:15:41.112879+0000","last_peered":"2023-10-08T00:15:41.112879+0000","last_clean":"2023-10-08T00:15:41.112879+0000","last_became_active":"2023-10-08T00:15:41.112384+0000","last_became_peered":"2023-10-08T00:15:41.112384+0000","last_unstale":"2023-10-08T00:15:41.112879+0000","last_undegraded":"2023-10-08T00:15:41.112879+0000","last_fullsized":"2023-10-08T00:15:41.112879+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:45:38.522163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"103'3","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.231005+0000","last_change":"2023-10-08T00:15:40.227901+0000","last_active":"2023-10-08T00:15:41.231005+0000","last_peered":"2023-10-08T00:15:41.231005+0000","last_clean":"2023-10-08T00:15:41.231005+0000","last_became_active":"2023-10-08T00:15:40.227460+0000","last_became_peered":"2023-10-08T00:15:40.227460+0000","last_unstale":"2023-10-08T00:15:41.231005+0000","last_undegraded":"2023-10-08T00:15:41.231005+0000","last_fullsized":"2023-10-08T00:15:41.231005+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:30:09.994691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.115202+0000","last_change":"2023-10-08T00:15:41.115058+0000","last_active":"2023-10-08T00:15:41.115202+0000","last_peered":"2023-10-08T00:15:41.115202+0000","last_clean":"2023-10-08T00:15:41.115202+0000","last_became_active":"2023-10-08T00:15:41.114617+0000","last_became_peered":"2023-10-08T00:15:41.114617+0000","last_unstale":"2023-10-08T00:15:41.115202+0000","last_undegraded":"2023-10-08T00:15:41.115202+0000","last_fullsized":"2023-10-08T00:15:41.115202+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:46:47.561539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"103'6","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.231490+0000","last_change":"2023-10-08T00:15:40.232058+0000","last_active":"2023-10-08T00:15:41.231490+0000","last_peered":"2023-10-08T00:15:41.231490+0000","last_clean":"2023-10-08T00:15:41.231490+0000","last_became_active":"2023-10-08T00:15:40.231094+0000","last_became_peered":"2023-10-08T00:15:40.231094+0000","last_unstale":"2023-10-08T00:15:41.231490+0000","last_undegraded":"2023-10-08T00:15:41.231490+0000","last_fullsized":"2023-10-08T00:15:41.231490+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:32:34.940111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.112006+0000","last_change":"2023-10-08T00:15:41.111447+0000","last_active":"2023-10-08T00:15:41.112006+0000","last_peered":"2023-10-08T00:15:41.112006+0000","last_clean":"2023-10-08T00:15:41.112006+0000","last_became_active":"2023-10-08T00:15:41.111226+0000","last_became_peered":"2023-10-08T00:15:41.111226+0000","last_unstale":"2023-10-08T00:15:41.112006+0000","last_undegraded":"2023-10-08T00:15:41.112006+0000","last_fullsized":"2023-10-08T00:15:41.112006+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:15:38.694742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"103'2","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.228961+0000","last_change":"2023-10-08T00:15:40.224566+0000","last_active":"2023-10-08T00:15:41.228961+0000","last_peered":"2023-10-08T00:15:41.228961+0000","last_clean":"2023-10-08T00:15:41.228961+0000","last_became_active":"2023-10-08T00:15:40.224289+0000","last_became_peered":"2023-10-08T00:15:40.224289+0000","last_unstale":"2023-10-08T00:15:41.228961+0000","last_undegraded":"2023-10-08T00:15:41.228961+0000","last_fullsized":"2023-10-08T00:15:41.228961+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:11:18.658704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.114182+0000","last_change":"2023-10-08T00:15:41.114128+0000","last_active":"2023-10-08T00:15:41.114182+0000","last_peered":"2023-10-08T00:15:41.114182+0000","last_clean":"2023-10-08T00:15:41.114182+0000","last_became_active":"2023-10-08T00:15:41.113663+0000","last_became_peered":"2023-10-08T00:15:41.113663+0000","last_unstale":"2023-10-08T00:15:41.114182+0000","last_undegraded":"2023-10-08T00:15:41.114182+0000","last_fullsized":"2023-10-08T00:15:41.114182+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:03:01.394697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.110882+0000","last_change":"2023-10-08T00:15:40.230264+0000","last_active":"2023-10-08T00:15:41.110882+0000","last_peered":"2023-10-08T00:15:41.110882+0000","last_clean":"2023-10-08T00:15:41.110882+0000","last_became_active":"2023-10-08T00:15:40.230046+0000","last_became_peered":"2023-10-08T00:15:40.230046+0000","last_unstale":"2023-10-08T00:15:41.110882+0000","last_undegraded":"2023-10-08T00:15:41.110882+0000","last_fullsized":"2023-10-08T00:15:41.110882+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:46:38.300061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.113339+0000","last_change":"2023-10-08T00:15:41.112956+0000","last_active":"2023-10-08T00:15:41.113339+0000","last_peered":"2023-10-08T00:15:41.113339+0000","last_clean":"2023-10-08T00:15:41.113339+0000","last_became_active":"2023-10-08T00:15:41.112699+0000","last_became_peered":"2023-10-08T00:15:41.112699+0000","last_unstale":"2023-10-08T00:15:41.113339+0000","last_undegraded":"2023-10-08T00:15:41.113339+0000","last_fullsized":"2023-10-08T00:15:41.113339+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:48:14.900929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.101564+0000","last_change":"2023-10-08T00:15:40.223493+0000","last_active":"2023-10-08T00:15:41.101564+0000","last_peered":"2023-10-08T00:15:41.101564+0000","last_clean":"2023-10-08T00:15:41.101564+0000","last_became_active":"2023-10-08T00:15:40.223199+0000","last_became_peered":"2023-10-08T00:15:40.223199+0000","last_unstale":"2023-10-08T00:15:41.101564+0000","last_undegraded":"2023-10-08T00:15:41.101564+0000","last_fullsized":"2023-10-08T00:15:41.101564+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:32:34.940111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.113988+0000","last_change":"2023-10-08T00:15:41.113279+0000","last_active":"2023-10-08T00:15:41.113988+0000","last_peered":"2023-10-08T00:15:41.113988+0000","last_clean":"2023-10-08T00:15:41.113988+0000","last_became_active":"2023-10-08T00:15:41.112946+0000","last_became_peered":"2023-10-08T00:15:41.112946+0000","last_unstale":"2023-10-08T00:15:41.113988+0000","last_undegraded":"2023-10-08T00:15:41.113988+0000","last_fullsized":"2023-10-08T00:15:41.113988+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:32:35.950276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"103'8","reported_seq":18,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.233377+0000","last_change":"2023-10-08T00:15:40.229794+0000","last_active":"2023-10-08T00:15:41.233377+0000","last_peered":"2023-10-08T00:15:41.233377+0000","last_clean":"2023-10-08T00:15:41.233377+0000","last_became_active":"2023-10-08T00:15:40.228910+0000","last_became_peered":"2023-10-08T00:15:40.228910+0000","last_unstale":"2023-10-08T00:15:41.233377+0000","last_undegraded":"2023-10-08T00:15:41.233377+0000","last_fullsized":"2023-10-08T00:15:41.233377+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:52:20.615200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.109402+0000","last_change":"2023-10-08T00:15:41.109324+0000","last_active":"2023-10-08T00:15:41.109402+0000","last_peered":"2023-10-08T00:15:41.109402+0000","last_clean":"2023-10-08T00:15:41.109402+0000","last_became_active":"2023-10-08T00:15:41.109104+0000","last_became_peered":"2023-10-08T00:15:41.109104+0000","last_unstale":"2023-10-08T00:15:41.109402+0000","last_undegraded":"2023-10-08T00:15:41.109402+0000","last_fullsized":"2023-10-08T00:15:41.109402+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:44:23.612344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"103'1","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.228676+0000","last_change":"2023-10-08T00:15:40.232336+0000","last_active":"2023-10-08T00:15:41.228676+0000","last_peered":"2023-10-08T00:15:41.228676+0000","last_clean":"2023-10-08T00:15:41.228676+0000","last_became_active":"2023-10-08T00:15:40.232022+0000","last_became_peered":"2023-10-08T00:15:40.232022+0000","last_unstale":"2023-10-08T00:15:41.228676+0000","last_undegraded":"2023-10-08T00:15:41.228676+0000","last_fullsized":"2023-10-08T00:15:41.228676+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:35:07.451703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.104944+0000","last_change":"2023-10-08T00:15:41.104185+0000","last_active":"2023-10-08T00:15:41.104944+0000","last_peered":"2023-10-08T00:15:41.104944+0000","last_clean":"2023-10-08T00:15:41.104944+0000","last_became_active":"2023-10-08T00:15:41.103973+0000","last_became_peered":"2023-10-08T00:15:41.103973+0000","last_unstale":"2023-10-08T00:15:41.104944+0000","last_undegraded":"2023-10-08T00:15:41.104944+0000","last_fullsized":"2023-10-08T00:15:41.104944+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:07:58.834149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"103'2","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.230836+0000","last_change":"2023-10-08T00:15:40.229211+0000","last_active":"2023-10-08T00:15:41.230836+0000","last_peered":"2023-10-08T00:15:41.230836+0000","last_clean":"2023-10-08T00:15:41.230836+0000","last_became_active":"2023-10-08T00:15:40.228280+0000","last_became_peered":"2023-10-08T00:15:40.228280+0000","last_unstale":"2023-10-08T00:15:41.230836+0000","last_undegraded":"2023-10-08T00:15:41.230836+0000","last_fullsized":"2023-10-08T00:15:41.230836+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:50:00.645163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.109427+0000","last_change":"2023-10-08T00:15:41.109376+0000","last_active":"2023-10-08T00:15:41.109427+0000","last_peered":"2023-10-08T00:15:41.109427+0000","last_clean":"2023-10-08T00:15:41.109427+0000","last_became_active":"2023-10-08T00:15:41.109163+0000","last_became_peered":"2023-10-08T00:15:41.109163+0000","last_unstale":"2023-10-08T00:15:41.109427+0000","last_undegraded":"2023-10-08T00:15:41.109427+0000","last_fullsized":"2023-10-08T00:15:41.109427+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:31:17.421281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.102834+0000","last_change":"2023-10-08T00:15:40.223887+0000","last_active":"2023-10-08T00:15:41.102834+0000","last_peered":"2023-10-08T00:15:41.102834+0000","last_clean":"2023-10-08T00:15:41.102834+0000","last_became_active":"2023-10-08T00:15:40.223656+0000","last_became_peered":"2023-10-08T00:15:40.223656+0000","last_unstale":"2023-10-08T00:15:41.102834+0000","last_undegraded":"2023-10-08T00:15:41.102834+0000","last_fullsized":"2023-10-08T00:15:41.102834+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:48:08.234731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.113915+0000","last_change":"2023-10-08T00:15:41.113833+0000","last_active":"2023-10-08T00:15:41.113915+0000","last_peered":"2023-10-08T00:15:41.113915+0000","last_clean":"2023-10-08T00:15:41.113915+0000","last_became_active":"2023-10-08T00:15:41.113566+0000","last_became_peered":"2023-10-08T00:15:41.113566+0000","last_unstale":"2023-10-08T00:15:41.113915+0000","last_undegraded":"2023-10-08T00:15:41.113915+0000","last_fullsized":"2023-10-08T00:15:41.113915+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:41:30.260910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"103'2","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.231765+0000","last_change":"2023-10-08T00:15:40.223627+0000","last_active":"2023-10-08T00:15:41.231765+0000","last_peered":"2023-10-08T00:15:41.231765+0000","last_clean":"2023-10-08T00:15:41.231765+0000","last_became_active":"2023-10-08T00:15:40.223380+0000","last_became_peered":"2023-10-08T00:15:40.223380+0000","last_unstale":"2023-10-08T00:15:41.231765+0000","last_undegraded":"2023-10-08T00:15:41.231765+0000","last_fullsized":"2023-10-08T00:15:41.231765+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:46:38.300061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.115192+0000","last_change":"2023-10-08T00:15:41.114928+0000","last_active":"2023-10-08T00:15:41.115192+0000","last_peered":"2023-10-08T00:15:41.115192+0000","last_clean":"2023-10-08T00:15:41.115192+0000","last_became_active":"2023-10-08T00:15:41.114278+0000","last_became_peered":"2023-10-08T00:15:41.114278+0000","last_unstale":"2023-10-08T00:15:41.115192+0000","last_undegraded":"2023-10-08T00:15:41.115192+0000","last_fullsized":"2023-10-08T00:15:41.115192+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:03:49.910579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.105484+0000","last_change":"2023-10-08T00:15:40.226286+0000","last_active":"2023-10-08T00:15:41.105484+0000","last_peered":"2023-10-08T00:15:41.105484+0000","last_clean":"2023-10-08T00:15:41.105484+0000","last_became_active":"2023-10-08T00:15:40.226046+0000","last_became_peered":"2023-10-08T00:15:40.226046+0000","last_unstale":"2023-10-08T00:15:41.105484+0000","last_undegraded":"2023-10-08T00:15:41.105484+0000","last_fullsized":"2023-10-08T00:15:41.105484+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:49:01.369417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.113447+0000","last_change":"2023-10-08T00:15:41.110696+0000","last_active":"2023-10-08T00:15:41.113447+0000","last_peered":"2023-10-08T00:15:41.113447+0000","last_clean":"2023-10-08T00:15:41.113447+0000","last_became_active":"2023-10-08T00:15:41.110435+0000","last_became_peered":"2023-10-08T00:15:41.110435+0000","last_unstale":"2023-10-08T00:15:41.113447+0000","last_undegraded":"2023-10-08T00:15:41.113447+0000","last_fullsized":"2023-10-08T00:15:41.113447+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:54:58.905309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.106691+0000","last_change":"2023-10-08T00:15:40.229512+0000","last_active":"2023-10-08T00:15:41.106691+0000","last_peered":"2023-10-08T00:15:41.106691+0000","last_clean":"2023-10-08T00:15:41.106691+0000","last_became_active":"2023-10-08T00:15:40.228978+0000","last_became_peered":"2023-10-08T00:15:40.228978+0000","last_unstale":"2023-10-08T00:15:41.106691+0000","last_undegraded":"2023-10-08T00:15:41.106691+0000","last_fullsized":"2023-10-08T00:15:41.106691+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:50:00.645163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.110209+0000","last_change":"2023-10-08T00:15:41.110160+0000","last_active":"2023-10-08T00:15:41.110209+0000","last_peered":"2023-10-08T00:15:41.110209+0000","last_clean":"2023-10-08T00:15:41.110209+0000","last_became_active":"2023-10-08T00:15:41.109933+0000","last_became_peered":"2023-10-08T00:15:41.109933+0000","last_unstale":"2023-10-08T00:15:41.110209+0000","last_undegraded":"2023-10-08T00:15:41.110209+0000","last_fullsized":"2023-10-08T00:15:41.110209+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:48:22.786963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.102100+0000","last_change":"2023-10-08T00:15:40.228701+0000","last_active":"2023-10-08T00:15:41.102100+0000","last_peered":"2023-10-08T00:15:41.102100+0000","last_clean":"2023-10-08T00:15:41.102100+0000","last_became_active":"2023-10-08T00:15:40.228460+0000","last_became_peered":"2023-10-08T00:15:40.228460+0000","last_unstale":"2023-10-08T00:15:41.102100+0000","last_undegraded":"2023-10-08T00:15:41.102100+0000","last_fullsized":"2023-10-08T00:15:41.102100+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:30:56.106888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.114866+0000","last_change":"2023-10-08T00:15:41.114813+0000","last_active":"2023-10-08T00:15:41.114866+0000","last_peered":"2023-10-08T00:15:41.114866+0000","last_clean":"2023-10-08T00:15:41.114866+0000","last_became_active":"2023-10-08T00:15:41.113886+0000","last_became_peered":"2023-10-08T00:15:41.113886+0000","last_unstale":"2023-10-08T00:15:41.114866+0000","last_undegraded":"2023-10-08T00:15:41.114866+0000","last_fullsized":"2023-10-08T00:15:41.114866+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:10:50.560796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"103'2","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.231521+0000","last_change":"2023-10-08T00:15:40.226242+0000","last_active":"2023-10-08T00:15:41.231521+0000","last_peered":"2023-10-08T00:15:41.231521+0000","last_clean":"2023-10-08T00:15:41.231521+0000","last_became_active":"2023-10-08T00:15:40.225988+0000","last_became_peered":"2023-10-08T00:15:40.225988+0000","last_unstale":"2023-10-08T00:15:41.231521+0000","last_undegraded":"2023-10-08T00:15:41.231521+0000","last_fullsized":"2023-10-08T00:15:41.231521+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:28:07.849819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.109548+0000","last_change":"2023-10-08T00:15:41.109468+0000","last_active":"2023-10-08T00:15:41.109548+0000","last_peered":"2023-10-08T00:15:41.109548+0000","last_clean":"2023-10-08T00:15:41.109548+0000","last_became_active":"2023-10-08T00:15:41.108928+0000","last_became_peered":"2023-10-08T00:15:41.108928+0000","last_unstale":"2023-10-08T00:15:41.109548+0000","last_undegraded":"2023-10-08T00:15:41.109548+0000","last_fullsized":"2023-10-08T00:15:41.109548+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:01:43.941131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.108467+0000","last_change":"2023-10-08T00:15:40.229650+0000","last_active":"2023-10-08T00:15:41.108467+0000","last_peered":"2023-10-08T00:15:41.108467+0000","last_clean":"2023-10-08T00:15:41.108467+0000","last_became_active":"2023-10-08T00:15:40.228540+0000","last_became_peered":"2023-10-08T00:15:40.228540+0000","last_unstale":"2023-10-08T00:15:41.108467+0000","last_undegraded":"2023-10-08T00:15:41.108467+0000","last_fullsized":"2023-10-08T00:15:41.108467+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:48:08.234731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.111724+0000","last_change":"2023-10-08T00:15:41.111661+0000","last_active":"2023-10-08T00:15:41.111724+0000","last_peered":"2023-10-08T00:15:41.111724+0000","last_clean":"2023-10-08T00:15:41.111724+0000","last_became_active":"2023-10-08T00:15:41.111414+0000","last_became_peered":"2023-10-08T00:15:41.111414+0000","last_unstale":"2023-10-08T00:15:41.111724+0000","last_undegraded":"2023-10-08T00:15:41.111724+0000","last_fullsized":"2023-10-08T00:15:41.111724+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:37:30.435787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.105947+0000","last_change":"2023-10-08T00:15:40.231914+0000","last_active":"2023-10-08T00:15:41.105947+0000","last_peered":"2023-10-08T00:15:41.105947+0000","last_clean":"2023-10-08T00:15:41.105947+0000","last_became_active":"2023-10-08T00:15:40.230640+0000","last_became_peered":"2023-10-08T00:15:40.230640+0000","last_unstale":"2023-10-08T00:15:41.105947+0000","last_undegraded":"2023-10-08T00:15:41.105947+0000","last_fullsized":"2023-10-08T00:15:41.105947+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:11:18.658704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.106468+0000","last_change":"2023-10-08T00:15:41.106411+0000","last_active":"2023-10-08T00:15:41.106468+0000","last_peered":"2023-10-08T00:15:41.106468+0000","last_clean":"2023-10-08T00:15:41.106468+0000","last_became_active":"2023-10-08T00:15:41.106197+0000","last_became_peered":"2023-10-08T00:15:41.106197+0000","last_unstale":"2023-10-08T00:15:41.106468+0000","last_undegraded":"2023-10-08T00:15:41.106468+0000","last_fullsized":"2023-10-08T00:15:41.106468+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:09:52.052703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.101239+0000","last_change":"2023-10-08T00:15:40.229408+0000","last_active":"2023-10-08T00:15:41.101239+0000","last_peered":"2023-10-08T00:15:41.101239+0000","last_clean":"2023-10-08T00:15:41.101239+0000","last_became_active":"2023-10-08T00:15:40.229004+0000","last_became_peered":"2023-10-08T00:15:40.229004+0000","last_unstale":"2023-10-08T00:15:41.101239+0000","last_undegraded":"2023-10-08T00:15:41.101239+0000","last_fullsized":"2023-10-08T00:15:41.101239+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:45:37.511998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.110827+0000","last_change":"2023-10-08T00:15:41.110785+0000","last_active":"2023-10-08T00:15:41.110827+0000","last_peered":"2023-10-08T00:15:41.110827+0000","last_clean":"2023-10-08T00:15:41.110827+0000","last_became_active":"2023-10-08T00:15:41.110584+0000","last_became_peered":"2023-10-08T00:15:41.110584+0000","last_unstale":"2023-10-08T00:15:41.110827+0000","last_undegraded":"2023-10-08T00:15:41.110827+0000","last_fullsized":"2023-10-08T00:15:41.110827+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:35:02.136255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.110999+0000","last_change":"2023-10-08T00:15:40.228342+0000","last_active":"2023-10-08T00:15:41.110999+0000","last_peered":"2023-10-08T00:15:41.110999+0000","last_clean":"2023-10-08T00:15:41.110999+0000","last_became_active":"2023-10-08T00:15:40.225794+0000","last_became_peered":"2023-10-08T00:15:40.225794+0000","last_unstale":"2023-10-08T00:15:41.110999+0000","last_undegraded":"2023-10-08T00:15:41.110999+0000","last_fullsized":"2023-10-08T00:15:41.110999+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:57:50.777303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.115584+0000","last_change":"2023-10-08T00:15:41.115390+0000","last_active":"2023-10-08T00:15:41.115584+0000","last_peered":"2023-10-08T00:15:41.115584+0000","last_clean":"2023-10-08T00:15:41.115584+0000","last_became_active":"2023-10-08T00:15:41.113010+0000","last_became_peered":"2023-10-08T00:15:41.113010+0000","last_unstale":"2023-10-08T00:15:41.115584+0000","last_undegraded":"2023-10-08T00:15:41.115584+0000","last_fullsized":"2023-10-08T00:15:41.115584+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:29:22.992476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.101639+0000","last_change":"2023-10-08T00:15:40.224730+0000","last_active":"2023-10-08T00:15:41.101639+0000","last_peered":"2023-10-08T00:15:41.101639+0000","last_clean":"2023-10-08T00:15:41.101639+0000","last_became_active":"2023-10-08T00:15:40.224515+0000","last_became_peered":"2023-10-08T00:15:40.224515+0000","last_unstale":"2023-10-08T00:15:41.101639+0000","last_undegraded":"2023-10-08T00:15:41.101639+0000","last_fullsized":"2023-10-08T00:15:41.101639+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:35:03.232234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.113312+0000","last_change":"2023-10-08T00:15:41.113084+0000","last_active":"2023-10-08T00:15:41.113312+0000","last_peered":"2023-10-08T00:15:41.113312+0000","last_clean":"2023-10-08T00:15:41.113312+0000","last_became_active":"2023-10-08T00:15:41.112570+0000","last_became_peered":"2023-10-08T00:15:41.112570+0000","last_unstale":"2023-10-08T00:15:41.113312+0000","last_undegraded":"2023-10-08T00:15:41.113312+0000","last_fullsized":"2023-10-08T00:15:41.113312+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:15:38.694742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.110283+0000","last_change":"2023-10-08T00:15:40.228031+0000","last_active":"2023-10-08T00:15:41.110283+0000","last_peered":"2023-10-08T00:15:41.110283+0000","last_clean":"2023-10-08T00:15:41.110283+0000","last_became_active":"2023-10-08T00:15:40.227683+0000","last_became_peered":"2023-10-08T00:15:40.227683+0000","last_unstale":"2023-10-08T00:15:41.110283+0000","last_undegraded":"2023-10-08T00:15:41.110283+0000","last_fullsized":"2023-10-08T00:15:41.110283+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:45:21.339247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.113759+0000","last_change":"2023-10-08T00:15:41.113698+0000","last_active":"2023-10-08T00:15:41.113759+0000","last_peered":"2023-10-08T00:15:41.113759+0000","last_clean":"2023-10-08T00:15:41.113759+0000","last_became_active":"2023-10-08T00:15:41.113473+0000","last_became_peered":"2023-10-08T00:15:41.113473+0000","last_unstale":"2023-10-08T00:15:41.113759+0000","last_undegraded":"2023-10-08T00:15:41.113759+0000","last_fullsized":"2023-10-08T00:15:41.113759+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:05:33.248306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.105838+0000","last_change":"2023-10-08T00:15:40.232170+0000","last_active":"2023-10-08T00:15:41.105838+0000","last_peered":"2023-10-08T00:15:41.105838+0000","last_clean":"2023-10-08T00:15:41.105838+0000","last_became_active":"2023-10-08T00:15:40.231705+0000","last_became_peered":"2023-10-08T00:15:40.231705+0000","last_unstale":"2023-10-08T00:15:41.105838+0000","last_undegraded":"2023-10-08T00:15:41.105838+0000","last_fullsized":"2023-10-08T00:15:41.105838+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:45:37.511998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.108513+0000","last_change":"2023-10-08T00:15:41.108421+0000","last_active":"2023-10-08T00:15:41.108513+0000","last_peered":"2023-10-08T00:15:41.108513+0000","last_clean":"2023-10-08T00:15:41.108513+0000","last_became_active":"2023-10-08T00:15:41.108160+0000","last_became_peered":"2023-10-08T00:15:41.108160+0000","last_unstale":"2023-10-08T00:15:41.108513+0000","last_undegraded":"2023-10-08T00:15:41.108513+0000","last_fullsized":"2023-10-08T00:15:41.108513+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:04:46.379314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"103'1","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.229393+0000","last_change":"2023-10-08T00:15:40.228623+0000","last_active":"2023-10-08T00:15:41.229393+0000","last_peered":"2023-10-08T00:15:41.229393+0000","last_clean":"2023-10-08T00:15:41.229393+0000","last_became_active":"2023-10-08T00:15:40.228332+0000","last_became_peered":"2023-10-08T00:15:40.228332+0000","last_unstale":"2023-10-08T00:15:41.229393+0000","last_undegraded":"2023-10-08T00:15:41.229393+0000","last_fullsized":"2023-10-08T00:15:41.229393+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:54:08.808620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":232,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.107323+0000","last_change":"2023-10-08T00:10:43.728490+0000","last_active":"2023-10-08T00:15:41.107323+0000","last_peered":"2023-10-08T00:15:41.107323+0000","last_clean":"2023-10-08T00:15:41.107323+0000","last_became_active":"2023-10-08T00:10:43.728341+0000","last_became_peered":"2023-10-08T00:10:43.728341+0000","last_unstale":"2023-10-08T00:15:41.107323+0000","last_undegraded":"2023-10-08T00:15:41.107323+0000","last_fullsized":"2023-10-08T00:15:41.107323+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_clean_scrub_stamp":"2023-10-08T00:10:42.716096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:06:31.035293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574910,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":552,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194759,"statfs":{"total":200000000,"available":199433790,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.51700000000000002,"15min":0.501},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53700000000000003,"15min":0.54300000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.72699999999999998,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.47799999999999998}]},{"osd":1,"last update":"Sun Oct 8 00:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.55200000000000005,"15min":0.53400000000000003},"min":{"1min":0.42399999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78200000000000003,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.57899999999999996,"15min":0.56399999999999995},"min":{"1min":0.438,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.79800000000000004,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.56699999999999995}]},{"osd":2,"last update":"Sun Oct 8 00:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54800000000000004,"15min":0.53900000000000003},"min":{"1min":0.41499999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.72499999999999998,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.53500000000000003,"15min":0.52000000000000002},"min":{"1min":0.33400000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.72399999999999998,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.55300000000000005}]},{"osd":3,"last update":"Sun Oct 8 00:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53500000000000003,"15min":0.51900000000000002},"min":{"1min":0.41199999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.74299999999999999,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.371},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56499999999999995,"15min":0.55800000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.76400000000000001,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.40000000000000002}]},{"osd":4,"last update":"Sun Oct 8 00:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.51000000000000001,"15min":0.52200000000000002},"min":{"1min":0.30399999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.72399999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.502},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.51100000000000001,"15min":0.52600000000000002},"min":{"1min":0.32600000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76500000000000001,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.59399999999999997}]},{"osd":5,"last update":"Sun Oct 8 00:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.52600000000000002,"15min":0.54800000000000004},"min":{"1min":0.33100000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.69399999999999995,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.55500000000000005,"15min":0.58399999999999996},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.75,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.622}]},{"osd":6,"last update":"Sun Oct 8 00:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.58799999999999997,"15min":0.59699999999999998},"min":{"1min":0.438,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.81699999999999995,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.60199999999999998,"15min":0.63},"min":{"1min":0.38300000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.82599999999999996,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.63900000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574910,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":553,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194759,"statfs":{"total":200000000,"available":199433216,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:14:53 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.502,"15min":0.48199999999999998},"min":{"1min":0.373,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.68799999999999994,"5min":1.012,"15min":1.012},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.60699999999999998,"15min":0.67900000000000005},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.68799999999999994,"5min":5.1230000000000002,"15min":5.1230000000000002},"last":0.42799999999999999}]},{"osd":1,"last update":"Sun Oct 8 00:14:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.54300000000000004,"15min":0.52100000000000002},"min":{"1min":0.379,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.71199999999999997,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56899999999999995,"15min":0.56299999999999994},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.79400000000000004,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.59399999999999997}]},{"osd":2,"last update":"Sun Oct 8 00:14:53 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54000000000000004,"15min":0.53000000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.75700000000000001,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57299999999999995,"15min":0.56899999999999995},"min":{"1min":0.38900000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.747,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.497}]},{"osd":3,"last update":"Sun Oct 8 00:14:53 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.55900000000000005,"15min":0.53800000000000003},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.76600000000000001,"5min":1.046,"15min":1.046},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56100000000000005,"15min":0.54200000000000004},"min":{"1min":0.35099999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78400000000000003,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.46100000000000002}]},{"osd":4,"last update":"Sun Oct 8 00:14:53 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.47099999999999997,"15min":0.495},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.55100000000000005,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.54700000000000004,"15min":0.56100000000000005},"min":{"1min":0.36199999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73599999999999999,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.626}]},{"osd":5,"last update":"Sun Oct 8 00:14:53 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.47899999999999998,"15min":0.48599999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.74199999999999999,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.52400000000000002,"15min":0.55200000000000005},"min":{"1min":0.32900000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.65200000000000002,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.54400000000000004}]},{"osd":7,"last update":"Sun Oct 8 00:14:53 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.52300000000000002,"15min":0.53900000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.72899999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.53600000000000003,"15min":0.54500000000000004},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.71799999999999997,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.53300000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1123,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194189,"statfs":{"total":200000000,"available":198849690,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.32500000000000001,"15min":0.30599999999999999},"min":{"1min":0.22,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.49199999999999999,"5min":0.56899999999999995,"15min":0.56899999999999995},"last":0.20399999999999999},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.38500000000000001,"15min":0.376},"min":{"1min":0.314,"5min":0.255,"15min":0.255},"max":{"1min":0.53000000000000003,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.32300000000000001}]},{"osd":2,"last update":"Sun Oct 8 00:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.34000000000000002,"15min":0.33400000000000002},"min":{"1min":0.254,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.50800000000000001,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.36099999999999999,"15min":0.33900000000000002},"min":{"1min":0.27900000000000003,"5min":0.252,"15min":0.252},"max":{"1min":0.54500000000000004,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.252}]},{"osd":3,"last update":"Sun Oct 8 00:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.38200000000000001,"15min":0.37},"min":{"1min":0.28399999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.57199999999999995,"5min":0.623,"15min":0.623},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.34799999999999998,"15min":0.34200000000000003},"min":{"1min":0.253,"5min":0.222,"15min":0.222},"max":{"1min":0.46700000000000003,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.30499999999999999}]},{"osd":4,"last update":"Sun Oct 8 00:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55200000000000005,"15min":0.53900000000000003},"min":{"1min":0.36299999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.504},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.56699999999999995},"min":{"1min":0.41499999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.72099999999999997,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.55400000000000005}]},{"osd":5,"last update":"Sun Oct 8 00:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55200000000000005,"15min":0.53600000000000003},"min":{"1min":0.42099999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.752,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58299999999999996,"15min":0.57499999999999996},"min":{"1min":0.39600000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.72499999999999998,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.47599999999999998}]},{"osd":6,"last update":"Sun Oct 8 00:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57899999999999996,"15min":0.56699999999999995},"min":{"1min":0.44500000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.73499999999999999,"5min":0.877,"15min":0.877},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57199999999999995,"15min":0.56100000000000005},"min":{"1min":0.45000000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77200000000000002,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.48699999999999999}]},{"osd":7,"last update":"Sun Oct 8 00:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57399999999999995,"15min":0.56299999999999994},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.76600000000000001,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.60799999999999998,"15min":0.59099999999999997},"min":{"1min":0.45200000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.53400000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574910,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":554,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194758,"statfs":{"total":200000000,"available":199432302,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 00:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.33500000000000002,"15min":0.33000000000000002},"min":{"1min":0.25,"5min":0.193,"15min":0.193},"max":{"1min":0.53200000000000003,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.20699999999999999},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.35099999999999998,"15min":0.34599999999999997},"min":{"1min":0.23799999999999999,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.48399999999999999,"5min":0.87,"15min":0.87},"last":0.38300000000000001}]},{"osd":2,"last update":"Sun Oct 8 00:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.38300000000000001,"15min":0.38600000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.56200000000000006,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.56000000000000005,"15min":0.76000000000000001},"min":{"1min":0.27000000000000002,"5min":0.185,"15min":0.185},"max":{"1min":0.55100000000000005,"5min":12.019,"15min":12.019},"last":0.25700000000000001}]},{"osd":3,"last update":"Sun Oct 8 00:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.36399999999999999,"15min":0.36199999999999999},"min":{"1min":0.27400000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.46999999999999997,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.40300000000000002,"15min":0.39500000000000002},"min":{"1min":0.28299999999999997,"5min":0.246,"15min":0.246},"max":{"1min":0.61399999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.438}]},{"osd":4,"last update":"Sun Oct 8 00:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.61399999999999999,"15min":0.61699999999999999},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.96399999999999997,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.625,"15min":0.625},"min":{"1min":0.435,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.94999999999999996,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.505}]},{"osd":5,"last update":"Sun Oct 8 00:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.79800000000000004,"15min":1.002},"min":{"1min":0.39500000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.84799999999999998,"5min":11.992000000000001,"15min":11.992000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.58699999999999997},"min":{"1min":0.40500000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.90300000000000002,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.51600000000000001}]},{"osd":6,"last update":"Sun Oct 8 00:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59499999999999997,"15min":0.58699999999999997},"min":{"1min":0.44,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.88700000000000001,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.79000000000000004,"15min":0.97899999999999998},"min":{"1min":0.41199999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.73999999999999999,"5min":11.927,"15min":11.927},"last":0.41699999999999998}]},{"osd":7,"last update":"Sun Oct 8 00:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.80200000000000005,"15min":1},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.81100000000000005,"5min":11.965,"15min":11.965},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.626,"15min":0.624},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.94099999999999995,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.61299999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574910,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":552,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194759,"statfs":{"total":200000000,"available":199433860,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.37,"15min":0.39500000000000002},"min":{"1min":0.24199999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.498,"5min":2.5139999999999998,"15min":2.5139999999999998},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.376,"15min":0.40600000000000003},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.56399999999999995,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":0.31900000000000001}]},{"osd":1,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.36899999999999999,"15min":0.39500000000000002},"min":{"1min":0.249,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.48499999999999999,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.372,"5min":0.40799999999999997,"15min":0.44},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.53400000000000003,"5min":2.5379999999999998,"15min":2.5379999999999998},"last":0.34699999999999998}]},{"osd":3,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.41099999999999998,"15min":0.433},"min":{"1min":0.255,"5min":0.254,"15min":0.254},"max":{"1min":0.54800000000000004,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.436,"15min":0.46800000000000003},"min":{"1min":0.30399999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.501,"5min":2.5619999999999998,"15min":2.5619999999999998},"last":0.38100000000000001}]},{"osd":4,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.59699999999999998,"15min":0.621},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.073,"5min":2.488,"15min":2.488},"last":0.497},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.59599999999999997,"15min":0.60799999999999998},"min":{"1min":0.41299999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.75800000000000001,"5min":2.073,"15min":2.073},"last":0.58699999999999997}]},{"osd":5,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63900000000000001,"15min":0.66100000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.754,"5min":2.2069999999999999,"15min":2.2069999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.65200000000000002,"15min":0.66600000000000004},"min":{"1min":0.45900000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0369999999999999,"5min":2.456,"15min":2.456},"last":0.56799999999999995}]},{"osd":6,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.625},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.05,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60599999999999998,"15min":0.61299999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0940000000000001,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.59599999999999997}]},{"osd":7,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63,"15min":0.63900000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0229999999999999,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.61899999999999999,"15min":0.63},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0569999999999999,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":0.53300000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574910,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":547,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194765,"statfs":{"total":200000000,"available":199439502,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.34000000000000002,"15min":0.371},"min":{"1min":0.218,"5min":0.193,"15min":0.193},"max":{"1min":0.52700000000000002,"5min":1.46,"15min":1.46},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.33400000000000002,"15min":0.34100000000000003},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":0.496,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.24199999999999999}]},{"osd":1,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.58999999999999997,"15min":0.83699999999999997},"min":{"1min":0.25,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.57599999999999996,"5min":12.351000000000001,"15min":12.351000000000001},"last":0.441},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.28399999999999997,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.58899999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.26700000000000002}]},{"osd":2,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.40100000000000002,"15min":0.42099999999999999},"min":{"1min":0.26300000000000001,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.70499999999999996,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.26800000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.63500000000000001,"5min":0.753,"15min":0.753},"last":0.45100000000000001}]},{"osd":4,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.80000000000000004,"15min":1.018},"min":{"1min":0.38400000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.876,"5min":11.333,"15min":11.333},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.82899999999999996,"15min":1.0509999999999999},"min":{"1min":0.41299999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.94899999999999995,"5min":12.321999999999999,"15min":12.321999999999999},"last":0.46300000000000002}]},{"osd":5,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.82499999999999996,"15min":1.0620000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.92000000000000004,"5min":12.291,"15min":12.291},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.60099999999999998,"15min":0.59799999999999998},"min":{"1min":0.46300000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.42999999999999999}]},{"osd":6,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.77700000000000002,"15min":0.98299999999999998},"min":{"1min":0.42299999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.96199999999999997,"5min":11.288,"15min":11.288},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.81299999999999994,"15min":1.044},"min":{"1min":0.433,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.89400000000000002,"5min":12.253,"15min":12.253},"last":0.47899999999999998}]},{"osd":7,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.58899999999999997,"15min":0.57499999999999996},"min":{"1min":0.40400000000000003,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.97299999999999998,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.81200000000000006,"15min":1.02},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.92400000000000004,"5min":11.363,"15min":11.363},"last":0.38600000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574910,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":546,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194765,"statfs":{"total":200000000,"available":199440114,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:14:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.56000000000000005,"15min":0.55400000000000005},"min":{"1min":0.316,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.72899999999999998,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57599999999999996,"15min":0.58399999999999996},"min":{"1min":0.41299999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.68899999999999995,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.67700000000000005}]},{"osd":1,"last update":"Sun Oct 8 00:14:52 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51700000000000002,"15min":0.51800000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.73599999999999999,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.52600000000000002,"15min":0.53200000000000003},"min":{"1min":0.36799999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71199999999999997,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.48799999999999999}]},{"osd":2,"last update":"Sun Oct 8 00:14:52 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.50900000000000001,"15min":0.51700000000000002},"min":{"1min":0.42699999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.72699999999999998,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51700000000000002,"15min":0.51100000000000001},"min":{"1min":0.34300000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.70499999999999996,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.57099999999999995}]},{"osd":3,"last update":"Sun Oct 8 00:14:52 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.52900000000000003,"15min":0.53700000000000003},"min":{"1min":0.40999999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.70599999999999996,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53200000000000003,"15min":0.53300000000000003},"min":{"1min":0.39500000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.72399999999999998}]},{"osd":5,"last update":"Sun Oct 8 00:14:52 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.47199999999999998,"15min":0.496},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.65100000000000002,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.49199999999999999,"15min":0.50900000000000001},"min":{"1min":0.35099999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.74399999999999999,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.79000000000000004}]},{"osd":6,"last update":"Sun Oct 8 00:14:52 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.47499999999999998,"15min":0.48899999999999999},"min":{"1min":0.30599999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.67300000000000004,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.52700000000000002,"15min":0.55300000000000005},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.71899999999999997,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.80100000000000005}]},{"osd":7,"last update":"Sun Oct 8 00:14:52 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.53200000000000003,"15min":0.55000000000000004},"min":{"1min":0.31900000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.77000000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.751},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.56799999999999995,"15min":0.57999999999999996},"min":{"1min":0.39000000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.70899999999999996,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.77500000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574910,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1130,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194181,"statfs":{"total":200000000,"available":198841922,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48299999999999998,"15min":0.48699999999999999},"min":{"1min":0.33800000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.65600000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.54500000000000004,"15min":0.55900000000000005},"min":{"1min":0.36799999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.752,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.61799999999999999}]},{"osd":1,"last update":"Sun Oct 8 00:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.52300000000000002,"15min":0.52400000000000002},"min":{"1min":0.32900000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.73799999999999999,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.51200000000000001,"15min":0.52400000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.63400000000000001,"5min":0.755,"15min":0.755},"last":0.56499999999999995}]},{"osd":2,"last update":"Sun Oct 8 00:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.52800000000000002,"15min":0.53000000000000003},"min":{"1min":0.36899999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.72199999999999998,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56000000000000005,"15min":0.56999999999999995},"min":{"1min":0.41399999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.76500000000000001,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.63300000000000001}]},{"osd":3,"last update":"Sun Oct 8 00:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.55400000000000005,"15min":0.54800000000000004},"min":{"1min":0.41499999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.72299999999999998,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56799999999999995,"15min":0.57999999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.72199999999999998,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.59399999999999997}]},{"osd":4,"last update":"Sun Oct 8 00:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.46700000000000003,"15min":0.47299999999999998},"min":{"1min":0.33300000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.76800000000000002,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.51100000000000001,"15min":0.51400000000000001},"min":{"1min":0.32200000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.80300000000000005,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.44800000000000001}]},{"osd":6,"last update":"Sun Oct 8 00:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.50900000000000001,"15min":0.502},"min":{"1min":0.442,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.78900000000000003,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.501,"15min":0.495},"min":{"1min":0.36899999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.39500000000000002}]},{"osd":7,"last update":"Sun Oct 8 00:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.52400000000000002,"15min":0.51800000000000002},"min":{"1min":0.33800000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.34899999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.78700000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.436}]}]}],"pool_statfs":[]}} 2023-10-08T00:15:45.280 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T00:15:45.428 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.427+0000 7f40cbfff700 1 Processor -- start 2023-10-08T00:15:45.428 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.427+0000 7f40cbfff700 1 -- start start 2023-10-08T00:15:45.430 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.428+0000 7f40cbfff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f40cc0cd6e0 0x7f40cc0cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:45.430 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.428+0000 7f40cbfff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f40cc0cdab0 0x7f40cc15d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:45.430 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.428+0000 7f40cbfff700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40cc0cb7c0 con 0x7f40cc15dda0 2023-10-08T00:15:45.430 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.428+0000 7f40cbfff700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40cc15e850 con 0x7f40cc0cd6e0 2023-10-08T00:15:45.430 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.428+0000 7f40cbfff700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40cc0cd030 con 0x7f40cc0cdab0 2023-10-08T00:15:45.431 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.428+0000 7f40caffd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f40cc0cd6e0 0x7f40cc0cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:45.431 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.428+0000 7f40cb7fe700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f40cc15dda0 0x7f40cc15cbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:60810/0 (socket says 172.21.15.99:60810) 2023-10-08T00:15:45.431 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.428+0000 7f40cb7fe700 1 -- 172.21.15.99:0/1153302927 learned_addr learned my addr 172.21.15.99:0/1153302927 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:15:45.431 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.428+0000 7f40ca7fc700 1 --2- 172.21.15.99:0/1153302927 >> v2:172.21.15.191:3300/0 conn(0x7f40cc0cdab0 0x7f40cc15d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:45.431 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.428+0000 7f40caffd700 1 -- 172.21.15.99:0/1153302927 >> v2:172.21.15.191:3300/0 conn(0x7f40cc0cdab0 msgr2=0x7f40cc15d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:45.431 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.428+0000 7f40caffd700 1 --2- 172.21.15.99:0/1153302927 >> v2:172.21.15.191:3300/0 conn(0x7f40cc0cdab0 0x7f40cc15d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:45.431 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.428+0000 7f40caffd700 1 -- 172.21.15.99:0/1153302927 >> v1:172.21.15.99:6789/0 conn(0x7f40cc15dda0 legacy=0x7f40cc15cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:45.431 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.428+0000 7f40c9ffb700 1 -- 172.21.15.99:0/1153302927 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1490957649 0 0) 0x7f40cc0cb7c0 con 0x7f40cc15dda0 2023-10-08T00:15:45.431 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.428+0000 7f40caffd700 1 -- 172.21.15.99:0/1153302927 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40cc15e290 con 0x7f40cc0cd6e0 2023-10-08T00:15:45.431 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.429+0000 7f40caffd700 1 --2- 172.21.15.99:0/1153302927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40cc0cd6e0 0x7f40cc0cbc50 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f40cc0b6c70 tx=0x7f40bc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb16b03bd0c0c481 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:45.431 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.429+0000 7f40c9ffb700 1 -- 172.21.15.99:0/1153302927 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40bc00e070 con 0x7f40cc0cd6e0 2023-10-08T00:15:45.431 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.429+0000 7f40c9ffb700 1 -- 172.21.15.99:0/1153302927 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40bc00bb60 con 0x7f40cc0cd6e0 2023-10-08T00:15:45.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.429+0000 7f40c9ffb700 1 -- 172.21.15.99:0/1153302927 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40bc012430 con 0x7f40cc0cd6e0 2023-10-08T00:15:45.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.429+0000 7f40cbfff700 1 -- 172.21.15.99:0/1153302927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40cc0cd6e0 msgr2=0x7f40cc0cbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:45.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.429+0000 7f40cbfff700 1 --2- 172.21.15.99:0/1153302927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40cc0cd6e0 0x7f40cc0cbc50 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f40cc0b6c70 tx=0x7f40bc009780 comp rx=0 tx=0).stop 2023-10-08T00:15:45.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.429+0000 7f40cbfff700 1 -- 172.21.15.99:0/1153302927 shutdown_connections 2023-10-08T00:15:45.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.429+0000 7f40cbfff700 1 --2- 172.21.15.99:0/1153302927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40cc0cd6e0 0x7f40cc0cbc50 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:45.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.429+0000 7f40cbfff700 1 --2- 172.21.15.99:0/1153302927 >> v2:172.21.15.191:3300/0 conn(0x7f40cc0cdab0 0x7f40cc15d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:45.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.429+0000 7f40cbfff700 1 -- 172.21.15.99:0/1153302927 >> 172.21.15.99:0/1153302927 conn(0x7f40cc067660 msgr2=0x7f40cc067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:45.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.429+0000 7f40cbfff700 1 -- 172.21.15.99:0/1153302927 shutdown_connections 2023-10-08T00:15:45.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.429+0000 7f40cbfff700 1 -- 172.21.15.99:0/1153302927 wait complete. 2023-10-08T00:15:45.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.431+0000 7f40cbfff700 1 Processor -- start 2023-10-08T00:15:45.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.431+0000 7f40cbfff700 1 -- start start 2023-10-08T00:15:45.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.431+0000 7f40cbfff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40cc0cd6e0 0x7f40cc15bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:45.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.431+0000 7f40cbfff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40cc0cdab0 0x7f40cc15ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:45.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.431+0000 7f40cbfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40cc15dda0 0x7f40cc15b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:45.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.431+0000 7f40cbfff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40cc147cd0 con 0x7f40cc15dda0 2023-10-08T00:15:45.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.431+0000 7f40cbfff700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40cc0b5c90 con 0x7f40cc0cd6e0 2023-10-08T00:15:45.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.431+0000 7f40cbfff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40cc15c320 con 0x7f40cc0cdab0 2023-10-08T00:15:45.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.431+0000 7f40cb7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40cc15dda0 0x7f40cc15b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:45.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.431+0000 7f40cb7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40cc15dda0 0x7f40cc15b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40942/0 (socket says 172.21.15.99:40942) 2023-10-08T00:15:45.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.431+0000 7f40cb7fe700 1 -- 172.21.15.99:0/540358844 learned_addr learned my addr 172.21.15.99:0/540358844 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:45.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.432+0000 7f40caffd700 1 --2- 172.21.15.99:0/540358844 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40cc0cd6e0 0x7f40cc15bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:45.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.432+0000 7f40ca7fc700 1 --2- 172.21.15.99:0/540358844 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40cc0cdab0 0x7f40cc15ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:45.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.432+0000 7f40cb7fe700 1 -- 172.21.15.99:0/540358844 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40cc0cdab0 msgr2=0x7f40cc15ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:45.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.432+0000 7f40cb7fe700 1 --2- 172.21.15.99:0/540358844 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40cc0cdab0 0x7f40cc15ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:45.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.432+0000 7f40cb7fe700 1 -- 172.21.15.99:0/540358844 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40cc0cd6e0 msgr2=0x7f40cc15bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:45.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.432+0000 7f40cb7fe700 1 --2- 172.21.15.99:0/540358844 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40cc0cd6e0 0x7f40cc15bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:45.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.432+0000 7f40cb7fe700 1 -- 172.21.15.99:0/540358844 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40bc009a20 con 0x7f40cc15dda0 2023-10-08T00:15:45.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.432+0000 7f40cb7fe700 1 --2- 172.21.15.99:0/540358844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40cc15dda0 0x7f40cc15b1a0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f40c400c530 tx=0x7f40c4007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:45.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.432+0000 7f40c2ffd700 1 -- 172.21.15.99:0/540358844 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c4005b30 con 0x7f40cc15dda0 2023-10-08T00:15:45.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.432+0000 7f40c2ffd700 1 -- 172.21.15.99:0/540358844 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c4005c90 con 0x7f40cc15dda0 2023-10-08T00:15:45.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.432+0000 7f40cbfff700 1 -- 172.21.15.99:0/540358844 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40cc0b53a0 con 0x7f40cc15dda0 2023-10-08T00:15:45.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.432+0000 7f40c2ffd700 1 -- 172.21.15.99:0/540358844 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c4007320 con 0x7f40cc15dda0 2023-10-08T00:15:45.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.432+0000 7f40cbfff700 1 -- 172.21.15.99:0/540358844 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40cc0b5640 con 0x7f40cc15dda0 2023-10-08T00:15:45.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.433+0000 7f40cbfff700 1 -- 172.21.15.99:0/540358844 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40cc0b5e20 con 0x7f40cc15dda0 2023-10-08T00:15:45.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.433+0000 7f40c2ffd700 1 -- 172.21.15.99:0/540358844 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f40c401b3b0 con 0x7f40cc15dda0 2023-10-08T00:15:45.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.434+0000 7f40c2ffd700 1 --2- 172.21.15.99:0/540358844 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f40ac066f30 0x7f40ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:45.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.434+0000 7f40c2ffd700 1 -- 172.21.15.99:0/540358844 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f40c4083a50 con 0x7f40cc15dda0 2023-10-08T00:15:45.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.434+0000 7f40caffd700 1 --2- 172.21.15.99:0/540358844 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f40ac066f30 0x7f40ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:45.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.434+0000 7f40caffd700 1 --2- 172.21.15.99:0/540358844 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f40ac066f30 0x7f40ac0693e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f40bc0097c0 tx=0x7f40bc019000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:45.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.438+0000 7f40c2ffd700 1 -- 172.21.15.99:0/540358844 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f40c4050cc0 con 0x7f40cc15dda0 2023-10-08T00:15:45.614 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.613+0000 7f40cbfff700 1 -- 172.21.15.99:0/540358844 --> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f40cc15e570 con 0x7f40ac066f30 2023-10-08T00:15:45.622 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.621+0000 7f40c2ffd700 1 -- 172.21.15.99:0/540358844 <== mgr.4099 v2:172.21.15.191:6832/85985 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285544 (secure 0 0 0) 0x7f40cc15e570 con 0x7f40ac066f30 2023-10-08T00:15:45.625 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:15:45.627 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.624+0000 7f40cbfff700 1 -- 172.21.15.99:0/540358844 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f40ac066f30 msgr2=0x7f40ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:45.628 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.624+0000 7f40cbfff700 1 --2- 172.21.15.99:0/540358844 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f40ac066f30 0x7f40ac0693e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f40bc0097c0 tx=0x7f40bc019000 comp rx=0 tx=0).stop 2023-10-08T00:15:45.628 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.624+0000 7f40cbfff700 1 -- 172.21.15.99:0/540358844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40cc15dda0 msgr2=0x7f40cc15b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:45.628 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.624+0000 7f40cbfff700 1 --2- 172.21.15.99:0/540358844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40cc15dda0 0x7f40cc15b1a0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f40c400c530 tx=0x7f40c4007f60 comp rx=0 tx=0).stop 2023-10-08T00:15:45.628 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.624+0000 7f40cbfff700 1 -- 172.21.15.99:0/540358844 shutdown_connections 2023-10-08T00:15:45.628 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.624+0000 7f40cbfff700 1 --2- 172.21.15.99:0/540358844 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f40ac066f30 0x7f40ac0693e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:45.628 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.624+0000 7f40cbfff700 1 --2- 172.21.15.99:0/540358844 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40cc0cd6e0 0x7f40cc15bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:45.628 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.624+0000 7f40cbfff700 1 --2- 172.21.15.99:0/540358844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40cc15dda0 0x7f40cc15b1a0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:45.628 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.624+0000 7f40cbfff700 1 --2- 172.21.15.99:0/540358844 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40cc0cdab0 0x7f40cc15ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:45.628 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.624+0000 7f40cbfff700 1 -- 172.21.15.99:0/540358844 >> 172.21.15.99:0/540358844 conn(0x7f40cc067660 msgr2=0x7f40cc1576f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:45.628 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.625+0000 7f40cbfff700 1 -- 172.21.15.99:0/540358844 shutdown_connections 2023-10-08T00:15:45.628 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.625+0000 7f40cbfff700 1 -- 172.21.15.99:0/540358844 wait complete. 2023-10-08T00:15:45.629 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2023-10-08T00:15:45.637 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":245,"stamp":"2023-10-08T00:15:44.093941+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":135,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":5562,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1556937,"statfs":{"total":1600000000,"available":1594304396,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.877858"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.112547+0000","last_change":"2023-10-08T00:15:41.112491+0000","last_active":"2023-10-08T00:15:41.112547+0000","last_peered":"2023-10-08T00:15:41.112547+0000","last_clean":"2023-10-08T00:15:41.112547+0000","last_became_active":"2023-10-08T00:15:41.112292+0000","last_became_peered":"2023-10-08T00:15:41.112292+0000","last_unstale":"2023-10-08T00:15:41.112547+0000","last_undegraded":"2023-10-08T00:15:41.112547+0000","last_fullsized":"2023-10-08T00:15:41.112547+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:23:45.765124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.111895+0000","last_change":"2023-10-08T00:15:41.111823+0000","last_active":"2023-10-08T00:15:41.111895+0000","last_peered":"2023-10-08T00:15:41.111895+0000","last_clean":"2023-10-08T00:15:41.111895+0000","last_became_active":"2023-10-08T00:15:41.111520+0000","last_became_peered":"2023-10-08T00:15:41.111520+0000","last_unstale":"2023-10-08T00:15:41.111895+0000","last_undegraded":"2023-10-08T00:15:41.111895+0000","last_fullsized":"2023-10-08T00:15:41.111895+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:23:45.765124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.106044+0000","last_change":"2023-10-08T00:15:41.105499+0000","last_active":"2023-10-08T00:15:41.106044+0000","last_peered":"2023-10-08T00:15:41.106044+0000","last_clean":"2023-10-08T00:15:41.106044+0000","last_became_active":"2023-10-08T00:15:41.105224+0000","last_became_peered":"2023-10-08T00:15:41.105224+0000","last_unstale":"2023-10-08T00:15:41.106044+0000","last_undegraded":"2023-10-08T00:15:41.106044+0000","last_fullsized":"2023-10-08T00:15:41.106044+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:48:22.786963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.107949+0000","last_change":"2023-10-08T00:15:41.107872+0000","last_active":"2023-10-08T00:15:41.107949+0000","last_peered":"2023-10-08T00:15:41.107949+0000","last_clean":"2023-10-08T00:15:41.107949+0000","last_became_active":"2023-10-08T00:15:41.107608+0000","last_became_peered":"2023-10-08T00:15:41.107608+0000","last_unstale":"2023-10-08T00:15:41.107949+0000","last_undegraded":"2023-10-08T00:15:41.107949+0000","last_fullsized":"2023-10-08T00:15:41.107949+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:07:58.834149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.106621+0000","last_change":"2023-10-08T00:15:41.106566+0000","last_active":"2023-10-08T00:15:41.106621+0000","last_peered":"2023-10-08T00:15:41.106621+0000","last_clean":"2023-10-08T00:15:41.106621+0000","last_became_active":"2023-10-08T00:15:41.106418+0000","last_became_peered":"2023-10-08T00:15:41.106418+0000","last_unstale":"2023-10-08T00:15:41.106621+0000","last_undegraded":"2023-10-08T00:15:41.106621+0000","last_fullsized":"2023-10-08T00:15:41.106621+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:36:32.485025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.106061+0000","last_change":"2023-10-08T00:15:41.105616+0000","last_active":"2023-10-08T00:15:41.106061+0000","last_peered":"2023-10-08T00:15:41.106061+0000","last_clean":"2023-10-08T00:15:41.106061+0000","last_became_active":"2023-10-08T00:15:41.105113+0000","last_became_peered":"2023-10-08T00:15:41.105113+0000","last_unstale":"2023-10-08T00:15:41.106061+0000","last_undegraded":"2023-10-08T00:15:41.106061+0000","last_fullsized":"2023-10-08T00:15:41.106061+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:39:53.717841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.110919+0000","last_change":"2023-10-08T00:15:41.110051+0000","last_active":"2023-10-08T00:15:41.110919+0000","last_peered":"2023-10-08T00:15:41.110919+0000","last_clean":"2023-10-08T00:15:41.110919+0000","last_became_active":"2023-10-08T00:15:41.109788+0000","last_became_peered":"2023-10-08T00:15:41.109788+0000","last_unstale":"2023-10-08T00:15:41.110919+0000","last_undegraded":"2023-10-08T00:15:41.110919+0000","last_fullsized":"2023-10-08T00:15:41.110919+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:35:02.136255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.116566+0000","last_change":"2023-10-08T00:15:41.116506+0000","last_active":"2023-10-08T00:15:41.116566+0000","last_peered":"2023-10-08T00:15:41.116566+0000","last_clean":"2023-10-08T00:15:41.116566+0000","last_became_active":"2023-10-08T00:15:41.116266+0000","last_became_peered":"2023-10-08T00:15:41.116266+0000","last_unstale":"2023-10-08T00:15:41.116566+0000","last_undegraded":"2023-10-08T00:15:41.116566+0000","last_fullsized":"2023-10-08T00:15:41.116566+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:19:15.421134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.113328+0000","last_change":"2023-10-08T00:15:41.113218+0000","last_active":"2023-10-08T00:15:41.113328+0000","last_peered":"2023-10-08T00:15:41.113328+0000","last_clean":"2023-10-08T00:15:41.113328+0000","last_became_active":"2023-10-08T00:15:41.112860+0000","last_became_peered":"2023-10-08T00:15:41.112860+0000","last_unstale":"2023-10-08T00:15:41.113328+0000","last_undegraded":"2023-10-08T00:15:41.113328+0000","last_fullsized":"2023-10-08T00:15:41.113328+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:46:54.920309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.111913+0000","last_change":"2023-10-08T00:15:41.111859+0000","last_active":"2023-10-08T00:15:41.111913+0000","last_peered":"2023-10-08T00:15:41.111913+0000","last_clean":"2023-10-08T00:15:41.111913+0000","last_became_active":"2023-10-08T00:15:41.111610+0000","last_became_peered":"2023-10-08T00:15:41.111610+0000","last_unstale":"2023-10-08T00:15:41.111913+0000","last_undegraded":"2023-10-08T00:15:41.111913+0000","last_fullsized":"2023-10-08T00:15:41.111913+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:50:30.128925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.115601+0000","last_change":"2023-10-08T00:15:41.115543+0000","last_active":"2023-10-08T00:15:41.115601+0000","last_peered":"2023-10-08T00:15:41.115601+0000","last_clean":"2023-10-08T00:15:41.115601+0000","last_became_active":"2023-10-08T00:15:41.115204+0000","last_became_peered":"2023-10-08T00:15:41.115204+0000","last_unstale":"2023-10-08T00:15:41.115601+0000","last_undegraded":"2023-10-08T00:15:41.115601+0000","last_fullsized":"2023-10-08T00:15:41.115601+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:31:07.729785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.108863+0000","last_change":"2023-10-08T00:15:41.108679+0000","last_active":"2023-10-08T00:15:41.108863+0000","last_peered":"2023-10-08T00:15:41.108863+0000","last_clean":"2023-10-08T00:15:41.108863+0000","last_became_active":"2023-10-08T00:15:41.108464+0000","last_became_peered":"2023-10-08T00:15:41.108464+0000","last_unstale":"2023-10-08T00:15:41.108863+0000","last_undegraded":"2023-10-08T00:15:41.108863+0000","last_fullsized":"2023-10-08T00:15:41.108863+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:39:53.717841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.107046+0000","last_change":"2023-10-08T00:15:41.106987+0000","last_active":"2023-10-08T00:15:41.107046+0000","last_peered":"2023-10-08T00:15:41.107046+0000","last_clean":"2023-10-08T00:15:41.107046+0000","last_became_active":"2023-10-08T00:15:41.106703+0000","last_became_peered":"2023-10-08T00:15:41.106703+0000","last_unstale":"2023-10-08T00:15:41.107046+0000","last_undegraded":"2023-10-08T00:15:41.107046+0000","last_fullsized":"2023-10-08T00:15:41.107046+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:52:35.721354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.115133+0000","last_change":"2023-10-08T00:15:41.113678+0000","last_active":"2023-10-08T00:15:41.115133+0000","last_peered":"2023-10-08T00:15:41.115133+0000","last_clean":"2023-10-08T00:15:41.115133+0000","last_became_active":"2023-10-08T00:15:41.108892+0000","last_became_peered":"2023-10-08T00:15:41.108892+0000","last_unstale":"2023-10-08T00:15:41.115133+0000","last_undegraded":"2023-10-08T00:15:41.115133+0000","last_fullsized":"2023-10-08T00:15:41.115133+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:29:22.992476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.115233+0000","last_change":"2023-10-08T00:15:41.115164+0000","last_active":"2023-10-08T00:15:41.115233+0000","last_peered":"2023-10-08T00:15:41.115233+0000","last_clean":"2023-10-08T00:15:41.115233+0000","last_became_active":"2023-10-08T00:15:41.114890+0000","last_became_peered":"2023-10-08T00:15:41.114890+0000","last_unstale":"2023-10-08T00:15:41.115233+0000","last_undegraded":"2023-10-08T00:15:41.115233+0000","last_fullsized":"2023-10-08T00:15:41.115233+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:03:01.394697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.104980+0000","last_change":"2023-10-08T00:15:41.104211+0000","last_active":"2023-10-08T00:15:41.104980+0000","last_peered":"2023-10-08T00:15:41.104980+0000","last_clean":"2023-10-08T00:15:41.104980+0000","last_became_active":"2023-10-08T00:15:41.103996+0000","last_became_peered":"2023-10-08T00:15:41.103996+0000","last_unstale":"2023-10-08T00:15:41.104980+0000","last_undegraded":"2023-10-08T00:15:41.104980+0000","last_fullsized":"2023-10-08T00:15:41.104980+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:54:58.905309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.115634+0000","last_change":"2023-10-08T00:15:41.115577+0000","last_active":"2023-10-08T00:15:41.115634+0000","last_peered":"2023-10-08T00:15:41.115634+0000","last_clean":"2023-10-08T00:15:41.115634+0000","last_became_active":"2023-10-08T00:15:41.115336+0000","last_became_peered":"2023-10-08T00:15:41.115336+0000","last_unstale":"2023-10-08T00:15:41.115634+0000","last_undegraded":"2023-10-08T00:15:41.115634+0000","last_fullsized":"2023-10-08T00:15:41.115634+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:49:44.152669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.117365+0000","last_change":"2023-10-08T00:15:41.117314+0000","last_active":"2023-10-08T00:15:41.117365+0000","last_peered":"2023-10-08T00:15:41.117365+0000","last_clean":"2023-10-08T00:15:41.117365+0000","last_became_active":"2023-10-08T00:15:41.116388+0000","last_became_peered":"2023-10-08T00:15:41.116388+0000","last_unstale":"2023-10-08T00:15:41.117365+0000","last_undegraded":"2023-10-08T00:15:41.117365+0000","last_fullsized":"2023-10-08T00:15:41.117365+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:35:04.242399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.115153+0000","last_change":"2023-10-08T00:15:41.113825+0000","last_active":"2023-10-08T00:15:41.115153+0000","last_peered":"2023-10-08T00:15:41.115153+0000","last_clean":"2023-10-08T00:15:41.115153+0000","last_became_active":"2023-10-08T00:15:41.109229+0000","last_became_peered":"2023-10-08T00:15:41.109229+0000","last_unstale":"2023-10-08T00:15:41.115153+0000","last_undegraded":"2023-10-08T00:15:41.115153+0000","last_fullsized":"2023-10-08T00:15:41.115153+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:15:38.694742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.110825+0000","last_change":"2023-10-08T00:15:41.109240+0000","last_active":"2023-10-08T00:15:41.110825+0000","last_peered":"2023-10-08T00:15:41.110825+0000","last_clean":"2023-10-08T00:15:41.110825+0000","last_became_active":"2023-10-08T00:15:41.108693+0000","last_became_peered":"2023-10-08T00:15:41.108693+0000","last_unstale":"2023-10-08T00:15:41.110825+0000","last_undegraded":"2023-10-08T00:15:41.110825+0000","last_fullsized":"2023-10-08T00:15:41.110825+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:31:07.729785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.113966+0000","last_change":"2023-10-08T00:15:41.113848+0000","last_active":"2023-10-08T00:15:41.113966+0000","last_peered":"2023-10-08T00:15:41.113966+0000","last_clean":"2023-10-08T00:15:41.113966+0000","last_became_active":"2023-10-08T00:15:41.113560+0000","last_became_peered":"2023-10-08T00:15:41.113560+0000","last_unstale":"2023-10-08T00:15:41.113966+0000","last_undegraded":"2023-10-08T00:15:41.113966+0000","last_fullsized":"2023-10-08T00:15:41.113966+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:16:04.386776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.108544+0000","last_change":"2023-10-08T00:15:41.108457+0000","last_active":"2023-10-08T00:15:41.108544+0000","last_peered":"2023-10-08T00:15:41.108544+0000","last_clean":"2023-10-08T00:15:41.108544+0000","last_became_active":"2023-10-08T00:15:41.108218+0000","last_became_peered":"2023-10-08T00:15:41.108218+0000","last_unstale":"2023-10-08T00:15:41.108544+0000","last_undegraded":"2023-10-08T00:15:41.108544+0000","last_fullsized":"2023-10-08T00:15:41.108544+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:13:13.842849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.115485+0000","last_change":"2023-10-08T00:15:41.115417+0000","last_active":"2023-10-08T00:15:41.115485+0000","last_peered":"2023-10-08T00:15:41.115485+0000","last_clean":"2023-10-08T00:15:41.115485+0000","last_became_active":"2023-10-08T00:15:41.115200+0000","last_became_peered":"2023-10-08T00:15:41.115200+0000","last_unstale":"2023-10-08T00:15:41.115485+0000","last_undegraded":"2023-10-08T00:15:41.115485+0000","last_fullsized":"2023-10-08T00:15:41.115485+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:50:30.128925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.115853+0000","last_change":"2023-10-08T00:15:41.115769+0000","last_active":"2023-10-08T00:15:41.115853+0000","last_peered":"2023-10-08T00:15:41.115853+0000","last_clean":"2023-10-08T00:15:41.115853+0000","last_became_active":"2023-10-08T00:15:41.113214+0000","last_became_peered":"2023-10-08T00:15:41.113214+0000","last_unstale":"2023-10-08T00:15:41.115853+0000","last_undegraded":"2023-10-08T00:15:41.115853+0000","last_fullsized":"2023-10-08T00:15:41.115853+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:38:56.511100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.109001+0000","last_change":"2023-10-08T00:15:41.108934+0000","last_active":"2023-10-08T00:15:41.109001+0000","last_peered":"2023-10-08T00:15:41.109001+0000","last_clean":"2023-10-08T00:15:41.109001+0000","last_became_active":"2023-10-08T00:15:41.108701+0000","last_became_peered":"2023-10-08T00:15:41.108701+0000","last_unstale":"2023-10-08T00:15:41.109001+0000","last_undegraded":"2023-10-08T00:15:41.109001+0000","last_fullsized":"2023-10-08T00:15:41.109001+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:48:51.428292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.111745+0000","last_change":"2023-10-08T00:15:41.111689+0000","last_active":"2023-10-08T00:15:41.111745+0000","last_peered":"2023-10-08T00:15:41.111745+0000","last_clean":"2023-10-08T00:15:41.111745+0000","last_became_active":"2023-10-08T00:15:41.111429+0000","last_became_peered":"2023-10-08T00:15:41.111429+0000","last_unstale":"2023-10-08T00:15:41.111745+0000","last_undegraded":"2023-10-08T00:15:41.111745+0000","last_fullsized":"2023-10-08T00:15:41.111745+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:15:30.961386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.109528+0000","last_change":"2023-10-08T00:15:41.109166+0000","last_active":"2023-10-08T00:15:41.109528+0000","last_peered":"2023-10-08T00:15:41.109528+0000","last_clean":"2023-10-08T00:15:41.109528+0000","last_became_active":"2023-10-08T00:15:41.108621+0000","last_became_peered":"2023-10-08T00:15:41.108621+0000","last_unstale":"2023-10-08T00:15:41.109528+0000","last_undegraded":"2023-10-08T00:15:41.109528+0000","last_fullsized":"2023-10-08T00:15:41.109528+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:03:25.736301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.112092+0000","last_change":"2023-10-08T00:15:41.112057+0000","last_active":"2023-10-08T00:15:41.112092+0000","last_peered":"2023-10-08T00:15:41.112092+0000","last_clean":"2023-10-08T00:15:41.112092+0000","last_became_active":"2023-10-08T00:15:41.111797+0000","last_became_peered":"2023-10-08T00:15:41.111797+0000","last_unstale":"2023-10-08T00:15:41.112092+0000","last_undegraded":"2023-10-08T00:15:41.112092+0000","last_fullsized":"2023-10-08T00:15:41.112092+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:17:01.578223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.110515+0000","last_change":"2023-10-08T00:15:41.110429+0000","last_active":"2023-10-08T00:15:41.110515+0000","last_peered":"2023-10-08T00:15:41.110515+0000","last_clean":"2023-10-08T00:15:41.110515+0000","last_became_active":"2023-10-08T00:15:41.110195+0000","last_became_peered":"2023-10-08T00:15:41.110195+0000","last_unstale":"2023-10-08T00:15:41.110515+0000","last_undegraded":"2023-10-08T00:15:41.110515+0000","last_fullsized":"2023-10-08T00:15:41.110515+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:29:08.856896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.111544+0000","last_change":"2023-10-08T00:15:41.111281+0000","last_active":"2023-10-08T00:15:41.111544+0000","last_peered":"2023-10-08T00:15:41.111544+0000","last_clean":"2023-10-08T00:15:41.111544+0000","last_became_active":"2023-10-08T00:15:41.109002+0000","last_became_peered":"2023-10-08T00:15:41.109002+0000","last_unstale":"2023-10-08T00:15:41.111544+0000","last_undegraded":"2023-10-08T00:15:41.111544+0000","last_fullsized":"2023-10-08T00:15:41.111544+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:35:04.242399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.106076+0000","last_change":"2023-10-08T00:15:41.105740+0000","last_active":"2023-10-08T00:15:41.106076+0000","last_peered":"2023-10-08T00:15:41.106076+0000","last_clean":"2023-10-08T00:15:41.106076+0000","last_became_active":"2023-10-08T00:15:41.105406+0000","last_became_peered":"2023-10-08T00:15:41.105406+0000","last_unstale":"2023-10-08T00:15:41.106076+0000","last_undegraded":"2023-10-08T00:15:41.106076+0000","last_fullsized":"2023-10-08T00:15:41.106076+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:44:23.612344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.110286+0000","last_change":"2023-10-08T00:15:41.110224+0000","last_active":"2023-10-08T00:15:41.110286+0000","last_peered":"2023-10-08T00:15:41.110286+0000","last_clean":"2023-10-08T00:15:41.110286+0000","last_became_active":"2023-10-08T00:15:41.110008+0000","last_became_peered":"2023-10-08T00:15:41.110008+0000","last_unstale":"2023-10-08T00:15:41.110286+0000","last_undegraded":"2023-10-08T00:15:41.110286+0000","last_fullsized":"2023-10-08T00:15:41.110286+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:42:20.087509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.110392+0000","last_change":"2023-10-08T00:15:41.110335+0000","last_active":"2023-10-08T00:15:41.110392+0000","last_peered":"2023-10-08T00:15:41.110392+0000","last_clean":"2023-10-08T00:15:41.110392+0000","last_became_active":"2023-10-08T00:15:41.110119+0000","last_became_peered":"2023-10-08T00:15:41.110119+0000","last_unstale":"2023-10-08T00:15:41.110392+0000","last_undegraded":"2023-10-08T00:15:41.110392+0000","last_fullsized":"2023-10-08T00:15:41.110392+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:42:20.087509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"103'4","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.231623+0000","last_change":"2023-10-08T00:15:40.227818+0000","last_active":"2023-10-08T00:15:41.231623+0000","last_peered":"2023-10-08T00:15:41.231623+0000","last_clean":"2023-10-08T00:15:41.231623+0000","last_became_active":"2023-10-08T00:15:40.227592+0000","last_became_peered":"2023-10-08T00:15:40.227592+0000","last_unstale":"2023-10-08T00:15:41.231623+0000","last_undegraded":"2023-10-08T00:15:41.231623+0000","last_fullsized":"2023-10-08T00:15:41.231623+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:28:10.454328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.112008+0000","last_change":"2023-10-08T00:15:41.111951+0000","last_active":"2023-10-08T00:15:41.112008+0000","last_peered":"2023-10-08T00:15:41.112008+0000","last_clean":"2023-10-08T00:15:41.112008+0000","last_became_active":"2023-10-08T00:15:41.111570+0000","last_became_peered":"2023-10-08T00:15:41.111570+0000","last_unstale":"2023-10-08T00:15:41.112008+0000","last_undegraded":"2023-10-08T00:15:41.112008+0000","last_fullsized":"2023-10-08T00:15:41.112008+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:47:58.612685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"103'1","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.229599+0000","last_change":"2023-10-08T00:15:40.227055+0000","last_active":"2023-10-08T00:15:41.229599+0000","last_peered":"2023-10-08T00:15:41.229599+0000","last_clean":"2023-10-08T00:15:41.229599+0000","last_became_active":"2023-10-08T00:15:40.226844+0000","last_became_peered":"2023-10-08T00:15:40.226844+0000","last_unstale":"2023-10-08T00:15:41.229599+0000","last_undegraded":"2023-10-08T00:15:41.229599+0000","last_fullsized":"2023-10-08T00:15:41.229599+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:31:06.719620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.114719+0000","last_change":"2023-10-08T00:15:41.114667+0000","last_active":"2023-10-08T00:15:41.114719+0000","last_peered":"2023-10-08T00:15:41.114719+0000","last_clean":"2023-10-08T00:15:41.114719+0000","last_became_active":"2023-10-08T00:15:41.111339+0000","last_became_peered":"2023-10-08T00:15:41.111339+0000","last_unstale":"2023-10-08T00:15:41.114719+0000","last_undegraded":"2023-10-08T00:15:41.114719+0000","last_fullsized":"2023-10-08T00:15:41.114719+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:07:58.834149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"103'2","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.231179+0000","last_change":"2023-10-08T00:15:40.229368+0000","last_active":"2023-10-08T00:15:41.231179+0000","last_peered":"2023-10-08T00:15:41.231179+0000","last_clean":"2023-10-08T00:15:41.231179+0000","last_became_active":"2023-10-08T00:15:40.228729+0000","last_became_peered":"2023-10-08T00:15:40.228729+0000","last_unstale":"2023-10-08T00:15:41.231179+0000","last_undegraded":"2023-10-08T00:15:41.231179+0000","last_fullsized":"2023-10-08T00:15:41.231179+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:46:53.910144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.114055+0000","last_change":"2023-10-08T00:15:41.114002+0000","last_active":"2023-10-08T00:15:41.114055+0000","last_peered":"2023-10-08T00:15:41.114055+0000","last_clean":"2023-10-08T00:15:41.114055+0000","last_became_active":"2023-10-08T00:15:41.113373+0000","last_became_peered":"2023-10-08T00:15:41.113373+0000","last_unstale":"2023-10-08T00:15:41.114055+0000","last_undegraded":"2023-10-08T00:15:41.114055+0000","last_fullsized":"2023-10-08T00:15:41.114055+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:13:13.842849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"103'2","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.230093+0000","last_change":"2023-10-08T00:15:40.228429+0000","last_active":"2023-10-08T00:15:41.230093+0000","last_peered":"2023-10-08T00:15:41.230093+0000","last_clean":"2023-10-08T00:15:41.230093+0000","last_became_active":"2023-10-08T00:15:40.227815+0000","last_became_peered":"2023-10-08T00:15:40.227815+0000","last_unstale":"2023-10-08T00:15:41.230093+0000","last_undegraded":"2023-10-08T00:15:41.230093+0000","last_fullsized":"2023-10-08T00:15:41.230093+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:42:37.339496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.115967+0000","last_change":"2023-10-08T00:15:41.115898+0000","last_active":"2023-10-08T00:15:41.115967+0000","last_peered":"2023-10-08T00:15:41.115967+0000","last_clean":"2023-10-08T00:15:41.115967+0000","last_became_active":"2023-10-08T00:15:41.111380+0000","last_became_peered":"2023-10-08T00:15:41.111380+0000","last_unstale":"2023-10-08T00:15:41.115967+0000","last_undegraded":"2023-10-08T00:15:41.115967+0000","last_fullsized":"2023-10-08T00:15:41.115967+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:50:30.128925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"103'1","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.228885+0000","last_change":"2023-10-08T00:15:40.231216+0000","last_active":"2023-10-08T00:15:41.228885+0000","last_peered":"2023-10-08T00:15:41.228885+0000","last_clean":"2023-10-08T00:15:41.228885+0000","last_became_active":"2023-10-08T00:15:40.230960+0000","last_became_peered":"2023-10-08T00:15:40.230960+0000","last_unstale":"2023-10-08T00:15:41.228885+0000","last_undegraded":"2023-10-08T00:15:41.228885+0000","last_fullsized":"2023-10-08T00:15:41.228885+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:48:50.418127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.104996+0000","last_change":"2023-10-08T00:15:41.104468+0000","last_active":"2023-10-08T00:15:41.104996+0000","last_peered":"2023-10-08T00:15:41.104996+0000","last_clean":"2023-10-08T00:15:41.104996+0000","last_became_active":"2023-10-08T00:15:41.104261+0000","last_became_peered":"2023-10-08T00:15:41.104261+0000","last_unstale":"2023-10-08T00:15:41.104996+0000","last_undegraded":"2023-10-08T00:15:41.104996+0000","last_fullsized":"2023-10-08T00:15:41.104996+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:16:04.386776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"103'1","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.228404+0000","last_change":"2023-10-08T00:15:40.226651+0000","last_active":"2023-10-08T00:15:41.228404+0000","last_peered":"2023-10-08T00:15:41.228404+0000","last_clean":"2023-10-08T00:15:41.228404+0000","last_became_active":"2023-10-08T00:15:40.226430+0000","last_became_peered":"2023-10-08T00:15:40.226430+0000","last_unstale":"2023-10-08T00:15:41.228404+0000","last_undegraded":"2023-10-08T00:15:41.228404+0000","last_fullsized":"2023-10-08T00:15:41.228404+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:38:55.500935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.113926+0000","last_change":"2023-10-08T00:15:41.113869+0000","last_active":"2023-10-08T00:15:41.113926+0000","last_peered":"2023-10-08T00:15:41.113926+0000","last_clean":"2023-10-08T00:15:41.113926+0000","last_became_active":"2023-10-08T00:15:41.113076+0000","last_became_peered":"2023-10-08T00:15:41.113076+0000","last_unstale":"2023-10-08T00:15:41.113926+0000","last_undegraded":"2023-10-08T00:15:41.113926+0000","last_fullsized":"2023-10-08T00:15:41.113926+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:50:11.772158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.104280+0000","last_change":"2023-10-08T00:15:40.231974+0000","last_active":"2023-10-08T00:15:41.104280+0000","last_peered":"2023-10-08T00:15:41.104280+0000","last_clean":"2023-10-08T00:15:41.104280+0000","last_became_active":"2023-10-08T00:15:40.231828+0000","last_became_peered":"2023-10-08T00:15:40.231828+0000","last_unstale":"2023-10-08T00:15:41.104280+0000","last_undegraded":"2023-10-08T00:15:41.104280+0000","last_fullsized":"2023-10-08T00:15:41.104280+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:57:50.777303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.107064+0000","last_change":"2023-10-08T00:15:41.106829+0000","last_active":"2023-10-08T00:15:41.107064+0000","last_peered":"2023-10-08T00:15:41.107064+0000","last_clean":"2023-10-08T00:15:41.107064+0000","last_became_active":"2023-10-08T00:15:41.106603+0000","last_became_peered":"2023-10-08T00:15:41.106603+0000","last_unstale":"2023-10-08T00:15:41.107064+0000","last_undegraded":"2023-10-08T00:15:41.107064+0000","last_fullsized":"2023-10-08T00:15:41.107064+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:38:56.511100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.102306+0000","last_change":"2023-10-08T00:15:40.229099+0000","last_active":"2023-10-08T00:15:41.102306+0000","last_peered":"2023-10-08T00:15:41.102306+0000","last_clean":"2023-10-08T00:15:41.102306+0000","last_became_active":"2023-10-08T00:15:40.227964+0000","last_became_peered":"2023-10-08T00:15:40.227964+0000","last_unstale":"2023-10-08T00:15:41.102306+0000","last_undegraded":"2023-10-08T00:15:41.102306+0000","last_fullsized":"2023-10-08T00:15:41.102306+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:49:01.369417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.112518+0000","last_change":"2023-10-08T00:15:41.112449+0000","last_active":"2023-10-08T00:15:41.112518+0000","last_peered":"2023-10-08T00:15:41.112518+0000","last_clean":"2023-10-08T00:15:41.112518+0000","last_became_active":"2023-10-08T00:15:41.111938+0000","last_became_peered":"2023-10-08T00:15:41.111938+0000","last_unstale":"2023-10-08T00:15:41.112518+0000","last_undegraded":"2023-10-08T00:15:41.112518+0000","last_fullsized":"2023-10-08T00:15:41.112518+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:50:11.772158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.107680+0000","last_change":"2023-10-08T00:15:40.222404+0000","last_active":"2023-10-08T00:15:41.107680+0000","last_peered":"2023-10-08T00:15:41.107680+0000","last_clean":"2023-10-08T00:15:41.107680+0000","last_became_active":"2023-10-08T00:15:40.222177+0000","last_became_peered":"2023-10-08T00:15:40.222177+0000","last_unstale":"2023-10-08T00:15:41.107680+0000","last_undegraded":"2023-10-08T00:15:41.107680+0000","last_fullsized":"2023-10-08T00:15:41.107680+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:25:12.720956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.111578+0000","last_change":"2023-10-08T00:15:41.111219+0000","last_active":"2023-10-08T00:15:41.111578+0000","last_peered":"2023-10-08T00:15:41.111578+0000","last_clean":"2023-10-08T00:15:41.111578+0000","last_became_active":"2023-10-08T00:15:41.110919+0000","last_became_peered":"2023-10-08T00:15:41.110919+0000","last_unstale":"2023-10-08T00:15:41.111578+0000","last_undegraded":"2023-10-08T00:15:41.111578+0000","last_fullsized":"2023-10-08T00:15:41.111578+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:48:14.900929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.101492+0000","last_change":"2023-10-08T00:15:40.233351+0000","last_active":"2023-10-08T00:15:41.101492+0000","last_peered":"2023-10-08T00:15:41.101492+0000","last_clean":"2023-10-08T00:15:41.101492+0000","last_became_active":"2023-10-08T00:15:40.233144+0000","last_became_peered":"2023-10-08T00:15:40.233144+0000","last_unstale":"2023-10-08T00:15:41.101492+0000","last_undegraded":"2023-10-08T00:15:41.101492+0000","last_fullsized":"2023-10-08T00:15:41.101492+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:04:45.369149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.112879+0000","last_change":"2023-10-08T00:15:41.112732+0000","last_active":"2023-10-08T00:15:41.112879+0000","last_peered":"2023-10-08T00:15:41.112879+0000","last_clean":"2023-10-08T00:15:41.112879+0000","last_became_active":"2023-10-08T00:15:41.112384+0000","last_became_peered":"2023-10-08T00:15:41.112384+0000","last_unstale":"2023-10-08T00:15:41.112879+0000","last_undegraded":"2023-10-08T00:15:41.112879+0000","last_fullsized":"2023-10-08T00:15:41.112879+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:45:38.522163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"103'3","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.231005+0000","last_change":"2023-10-08T00:15:40.227901+0000","last_active":"2023-10-08T00:15:41.231005+0000","last_peered":"2023-10-08T00:15:41.231005+0000","last_clean":"2023-10-08T00:15:41.231005+0000","last_became_active":"2023-10-08T00:15:40.227460+0000","last_became_peered":"2023-10-08T00:15:40.227460+0000","last_unstale":"2023-10-08T00:15:41.231005+0000","last_undegraded":"2023-10-08T00:15:41.231005+0000","last_fullsized":"2023-10-08T00:15:41.231005+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:30:09.994691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.115202+0000","last_change":"2023-10-08T00:15:41.115058+0000","last_active":"2023-10-08T00:15:41.115202+0000","last_peered":"2023-10-08T00:15:41.115202+0000","last_clean":"2023-10-08T00:15:41.115202+0000","last_became_active":"2023-10-08T00:15:41.114617+0000","last_became_peered":"2023-10-08T00:15:41.114617+0000","last_unstale":"2023-10-08T00:15:41.115202+0000","last_undegraded":"2023-10-08T00:15:41.115202+0000","last_fullsized":"2023-10-08T00:15:41.115202+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:46:47.561539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"103'6","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.231490+0000","last_change":"2023-10-08T00:15:40.232058+0000","last_active":"2023-10-08T00:15:41.231490+0000","last_peered":"2023-10-08T00:15:41.231490+0000","last_clean":"2023-10-08T00:15:41.231490+0000","last_became_active":"2023-10-08T00:15:40.231094+0000","last_became_peered":"2023-10-08T00:15:40.231094+0000","last_unstale":"2023-10-08T00:15:41.231490+0000","last_undegraded":"2023-10-08T00:15:41.231490+0000","last_fullsized":"2023-10-08T00:15:41.231490+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:32:34.940111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.112006+0000","last_change":"2023-10-08T00:15:41.111447+0000","last_active":"2023-10-08T00:15:41.112006+0000","last_peered":"2023-10-08T00:15:41.112006+0000","last_clean":"2023-10-08T00:15:41.112006+0000","last_became_active":"2023-10-08T00:15:41.111226+0000","last_became_peered":"2023-10-08T00:15:41.111226+0000","last_unstale":"2023-10-08T00:15:41.112006+0000","last_undegraded":"2023-10-08T00:15:41.112006+0000","last_fullsized":"2023-10-08T00:15:41.112006+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:15:38.694742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"103'2","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.228961+0000","last_change":"2023-10-08T00:15:40.224566+0000","last_active":"2023-10-08T00:15:41.228961+0000","last_peered":"2023-10-08T00:15:41.228961+0000","last_clean":"2023-10-08T00:15:41.228961+0000","last_became_active":"2023-10-08T00:15:40.224289+0000","last_became_peered":"2023-10-08T00:15:40.224289+0000","last_unstale":"2023-10-08T00:15:41.228961+0000","last_undegraded":"2023-10-08T00:15:41.228961+0000","last_fullsized":"2023-10-08T00:15:41.228961+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:11:18.658704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.114182+0000","last_change":"2023-10-08T00:15:41.114128+0000","last_active":"2023-10-08T00:15:41.114182+0000","last_peered":"2023-10-08T00:15:41.114182+0000","last_clean":"2023-10-08T00:15:41.114182+0000","last_became_active":"2023-10-08T00:15:41.113663+0000","last_became_peered":"2023-10-08T00:15:41.113663+0000","last_unstale":"2023-10-08T00:15:41.114182+0000","last_undegraded":"2023-10-08T00:15:41.114182+0000","last_fullsized":"2023-10-08T00:15:41.114182+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:03:01.394697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.110882+0000","last_change":"2023-10-08T00:15:40.230264+0000","last_active":"2023-10-08T00:15:41.110882+0000","last_peered":"2023-10-08T00:15:41.110882+0000","last_clean":"2023-10-08T00:15:41.110882+0000","last_became_active":"2023-10-08T00:15:40.230046+0000","last_became_peered":"2023-10-08T00:15:40.230046+0000","last_unstale":"2023-10-08T00:15:41.110882+0000","last_undegraded":"2023-10-08T00:15:41.110882+0000","last_fullsized":"2023-10-08T00:15:41.110882+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:46:38.300061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.113339+0000","last_change":"2023-10-08T00:15:41.112956+0000","last_active":"2023-10-08T00:15:41.113339+0000","last_peered":"2023-10-08T00:15:41.113339+0000","last_clean":"2023-10-08T00:15:41.113339+0000","last_became_active":"2023-10-08T00:15:41.112699+0000","last_became_peered":"2023-10-08T00:15:41.112699+0000","last_unstale":"2023-10-08T00:15:41.113339+0000","last_undegraded":"2023-10-08T00:15:41.113339+0000","last_fullsized":"2023-10-08T00:15:41.113339+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:48:14.900929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.101564+0000","last_change":"2023-10-08T00:15:40.223493+0000","last_active":"2023-10-08T00:15:41.101564+0000","last_peered":"2023-10-08T00:15:41.101564+0000","last_clean":"2023-10-08T00:15:41.101564+0000","last_became_active":"2023-10-08T00:15:40.223199+0000","last_became_peered":"2023-10-08T00:15:40.223199+0000","last_unstale":"2023-10-08T00:15:41.101564+0000","last_undegraded":"2023-10-08T00:15:41.101564+0000","last_fullsized":"2023-10-08T00:15:41.101564+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:32:34.940111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.113988+0000","last_change":"2023-10-08T00:15:41.113279+0000","last_active":"2023-10-08T00:15:41.113988+0000","last_peered":"2023-10-08T00:15:41.113988+0000","last_clean":"2023-10-08T00:15:41.113988+0000","last_became_active":"2023-10-08T00:15:41.112946+0000","last_became_peered":"2023-10-08T00:15:41.112946+0000","last_unstale":"2023-10-08T00:15:41.113988+0000","last_undegraded":"2023-10-08T00:15:41.113988+0000","last_fullsized":"2023-10-08T00:15:41.113988+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:32:35.950276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"103'8","reported_seq":18,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.233377+0000","last_change":"2023-10-08T00:15:40.229794+0000","last_active":"2023-10-08T00:15:41.233377+0000","last_peered":"2023-10-08T00:15:41.233377+0000","last_clean":"2023-10-08T00:15:41.233377+0000","last_became_active":"2023-10-08T00:15:40.228910+0000","last_became_peered":"2023-10-08T00:15:40.228910+0000","last_unstale":"2023-10-08T00:15:41.233377+0000","last_undegraded":"2023-10-08T00:15:41.233377+0000","last_fullsized":"2023-10-08T00:15:41.233377+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:52:20.615200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.109402+0000","last_change":"2023-10-08T00:15:41.109324+0000","last_active":"2023-10-08T00:15:41.109402+0000","last_peered":"2023-10-08T00:15:41.109402+0000","last_clean":"2023-10-08T00:15:41.109402+0000","last_became_active":"2023-10-08T00:15:41.109104+0000","last_became_peered":"2023-10-08T00:15:41.109104+0000","last_unstale":"2023-10-08T00:15:41.109402+0000","last_undegraded":"2023-10-08T00:15:41.109402+0000","last_fullsized":"2023-10-08T00:15:41.109402+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:44:23.612344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"103'1","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.228676+0000","last_change":"2023-10-08T00:15:40.232336+0000","last_active":"2023-10-08T00:15:41.228676+0000","last_peered":"2023-10-08T00:15:41.228676+0000","last_clean":"2023-10-08T00:15:41.228676+0000","last_became_active":"2023-10-08T00:15:40.232022+0000","last_became_peered":"2023-10-08T00:15:40.232022+0000","last_unstale":"2023-10-08T00:15:41.228676+0000","last_undegraded":"2023-10-08T00:15:41.228676+0000","last_fullsized":"2023-10-08T00:15:41.228676+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:35:07.451703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.104944+0000","last_change":"2023-10-08T00:15:41.104185+0000","last_active":"2023-10-08T00:15:41.104944+0000","last_peered":"2023-10-08T00:15:41.104944+0000","last_clean":"2023-10-08T00:15:41.104944+0000","last_became_active":"2023-10-08T00:15:41.103973+0000","last_became_peered":"2023-10-08T00:15:41.103973+0000","last_unstale":"2023-10-08T00:15:41.104944+0000","last_undegraded":"2023-10-08T00:15:41.104944+0000","last_fullsized":"2023-10-08T00:15:41.104944+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:07:58.834149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"103'2","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.230836+0000","last_change":"2023-10-08T00:15:40.229211+0000","last_active":"2023-10-08T00:15:41.230836+0000","last_peered":"2023-10-08T00:15:41.230836+0000","last_clean":"2023-10-08T00:15:41.230836+0000","last_became_active":"2023-10-08T00:15:40.228280+0000","last_became_peered":"2023-10-08T00:15:40.228280+0000","last_unstale":"2023-10-08T00:15:41.230836+0000","last_undegraded":"2023-10-08T00:15:41.230836+0000","last_fullsized":"2023-10-08T00:15:41.230836+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:50:00.645163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.109427+0000","last_change":"2023-10-08T00:15:41.109376+0000","last_active":"2023-10-08T00:15:41.109427+0000","last_peered":"2023-10-08T00:15:41.109427+0000","last_clean":"2023-10-08T00:15:41.109427+0000","last_became_active":"2023-10-08T00:15:41.109163+0000","last_became_peered":"2023-10-08T00:15:41.109163+0000","last_unstale":"2023-10-08T00:15:41.109427+0000","last_undegraded":"2023-10-08T00:15:41.109427+0000","last_fullsized":"2023-10-08T00:15:41.109427+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:31:17.421281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.102834+0000","last_change":"2023-10-08T00:15:40.223887+0000","last_active":"2023-10-08T00:15:41.102834+0000","last_peered":"2023-10-08T00:15:41.102834+0000","last_clean":"2023-10-08T00:15:41.102834+0000","last_became_active":"2023-10-08T00:15:40.223656+0000","last_became_peered":"2023-10-08T00:15:40.223656+0000","last_unstale":"2023-10-08T00:15:41.102834+0000","last_undegraded":"2023-10-08T00:15:41.102834+0000","last_fullsized":"2023-10-08T00:15:41.102834+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:48:08.234731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.113915+0000","last_change":"2023-10-08T00:15:41.113833+0000","last_active":"2023-10-08T00:15:41.113915+0000","last_peered":"2023-10-08T00:15:41.113915+0000","last_clean":"2023-10-08T00:15:41.113915+0000","last_became_active":"2023-10-08T00:15:41.113566+0000","last_became_peered":"2023-10-08T00:15:41.113566+0000","last_unstale":"2023-10-08T00:15:41.113915+0000","last_undegraded":"2023-10-08T00:15:41.113915+0000","last_fullsized":"2023-10-08T00:15:41.113915+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:41:30.260910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"103'2","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.231765+0000","last_change":"2023-10-08T00:15:40.223627+0000","last_active":"2023-10-08T00:15:41.231765+0000","last_peered":"2023-10-08T00:15:41.231765+0000","last_clean":"2023-10-08T00:15:41.231765+0000","last_became_active":"2023-10-08T00:15:40.223380+0000","last_became_peered":"2023-10-08T00:15:40.223380+0000","last_unstale":"2023-10-08T00:15:41.231765+0000","last_undegraded":"2023-10-08T00:15:41.231765+0000","last_fullsized":"2023-10-08T00:15:41.231765+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:46:38.300061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.115192+0000","last_change":"2023-10-08T00:15:41.114928+0000","last_active":"2023-10-08T00:15:41.115192+0000","last_peered":"2023-10-08T00:15:41.115192+0000","last_clean":"2023-10-08T00:15:41.115192+0000","last_became_active":"2023-10-08T00:15:41.114278+0000","last_became_peered":"2023-10-08T00:15:41.114278+0000","last_unstale":"2023-10-08T00:15:41.115192+0000","last_undegraded":"2023-10-08T00:15:41.115192+0000","last_fullsized":"2023-10-08T00:15:41.115192+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:03:49.910579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.105484+0000","last_change":"2023-10-08T00:15:40.226286+0000","last_active":"2023-10-08T00:15:41.105484+0000","last_peered":"2023-10-08T00:15:41.105484+0000","last_clean":"2023-10-08T00:15:41.105484+0000","last_became_active":"2023-10-08T00:15:40.226046+0000","last_became_peered":"2023-10-08T00:15:40.226046+0000","last_unstale":"2023-10-08T00:15:41.105484+0000","last_undegraded":"2023-10-08T00:15:41.105484+0000","last_fullsized":"2023-10-08T00:15:41.105484+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:49:01.369417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.113447+0000","last_change":"2023-10-08T00:15:41.110696+0000","last_active":"2023-10-08T00:15:41.113447+0000","last_peered":"2023-10-08T00:15:41.113447+0000","last_clean":"2023-10-08T00:15:41.113447+0000","last_became_active":"2023-10-08T00:15:41.110435+0000","last_became_peered":"2023-10-08T00:15:41.110435+0000","last_unstale":"2023-10-08T00:15:41.113447+0000","last_undegraded":"2023-10-08T00:15:41.113447+0000","last_fullsized":"2023-10-08T00:15:41.113447+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:54:58.905309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.106691+0000","last_change":"2023-10-08T00:15:40.229512+0000","last_active":"2023-10-08T00:15:41.106691+0000","last_peered":"2023-10-08T00:15:41.106691+0000","last_clean":"2023-10-08T00:15:41.106691+0000","last_became_active":"2023-10-08T00:15:40.228978+0000","last_became_peered":"2023-10-08T00:15:40.228978+0000","last_unstale":"2023-10-08T00:15:41.106691+0000","last_undegraded":"2023-10-08T00:15:41.106691+0000","last_fullsized":"2023-10-08T00:15:41.106691+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:50:00.645163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.110209+0000","last_change":"2023-10-08T00:15:41.110160+0000","last_active":"2023-10-08T00:15:41.110209+0000","last_peered":"2023-10-08T00:15:41.110209+0000","last_clean":"2023-10-08T00:15:41.110209+0000","last_became_active":"2023-10-08T00:15:41.109933+0000","last_became_peered":"2023-10-08T00:15:41.109933+0000","last_unstale":"2023-10-08T00:15:41.110209+0000","last_undegraded":"2023-10-08T00:15:41.110209+0000","last_fullsized":"2023-10-08T00:15:41.110209+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:48:22.786963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.102100+0000","last_change":"2023-10-08T00:15:40.228701+0000","last_active":"2023-10-08T00:15:41.102100+0000","last_peered":"2023-10-08T00:15:41.102100+0000","last_clean":"2023-10-08T00:15:41.102100+0000","last_became_active":"2023-10-08T00:15:40.228460+0000","last_became_peered":"2023-10-08T00:15:40.228460+0000","last_unstale":"2023-10-08T00:15:41.102100+0000","last_undegraded":"2023-10-08T00:15:41.102100+0000","last_fullsized":"2023-10-08T00:15:41.102100+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:30:56.106888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.114866+0000","last_change":"2023-10-08T00:15:41.114813+0000","last_active":"2023-10-08T00:15:41.114866+0000","last_peered":"2023-10-08T00:15:41.114866+0000","last_clean":"2023-10-08T00:15:41.114866+0000","last_became_active":"2023-10-08T00:15:41.113886+0000","last_became_peered":"2023-10-08T00:15:41.113886+0000","last_unstale":"2023-10-08T00:15:41.114866+0000","last_undegraded":"2023-10-08T00:15:41.114866+0000","last_fullsized":"2023-10-08T00:15:41.114866+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:10:50.560796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"103'2","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.231521+0000","last_change":"2023-10-08T00:15:40.226242+0000","last_active":"2023-10-08T00:15:41.231521+0000","last_peered":"2023-10-08T00:15:41.231521+0000","last_clean":"2023-10-08T00:15:41.231521+0000","last_became_active":"2023-10-08T00:15:40.225988+0000","last_became_peered":"2023-10-08T00:15:40.225988+0000","last_unstale":"2023-10-08T00:15:41.231521+0000","last_undegraded":"2023-10-08T00:15:41.231521+0000","last_fullsized":"2023-10-08T00:15:41.231521+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:28:07.849819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.109548+0000","last_change":"2023-10-08T00:15:41.109468+0000","last_active":"2023-10-08T00:15:41.109548+0000","last_peered":"2023-10-08T00:15:41.109548+0000","last_clean":"2023-10-08T00:15:41.109548+0000","last_became_active":"2023-10-08T00:15:41.108928+0000","last_became_peered":"2023-10-08T00:15:41.108928+0000","last_unstale":"2023-10-08T00:15:41.109548+0000","last_undegraded":"2023-10-08T00:15:41.109548+0000","last_fullsized":"2023-10-08T00:15:41.109548+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:01:43.941131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.108467+0000","last_change":"2023-10-08T00:15:40.229650+0000","last_active":"2023-10-08T00:15:41.108467+0000","last_peered":"2023-10-08T00:15:41.108467+0000","last_clean":"2023-10-08T00:15:41.108467+0000","last_became_active":"2023-10-08T00:15:40.228540+0000","last_became_peered":"2023-10-08T00:15:40.228540+0000","last_unstale":"2023-10-08T00:15:41.108467+0000","last_undegraded":"2023-10-08T00:15:41.108467+0000","last_fullsized":"2023-10-08T00:15:41.108467+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:48:08.234731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.111724+0000","last_change":"2023-10-08T00:15:41.111661+0000","last_active":"2023-10-08T00:15:41.111724+0000","last_peered":"2023-10-08T00:15:41.111724+0000","last_clean":"2023-10-08T00:15:41.111724+0000","last_became_active":"2023-10-08T00:15:41.111414+0000","last_became_peered":"2023-10-08T00:15:41.111414+0000","last_unstale":"2023-10-08T00:15:41.111724+0000","last_undegraded":"2023-10-08T00:15:41.111724+0000","last_fullsized":"2023-10-08T00:15:41.111724+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:37:30.435787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.105947+0000","last_change":"2023-10-08T00:15:40.231914+0000","last_active":"2023-10-08T00:15:41.105947+0000","last_peered":"2023-10-08T00:15:41.105947+0000","last_clean":"2023-10-08T00:15:41.105947+0000","last_became_active":"2023-10-08T00:15:40.230640+0000","last_became_peered":"2023-10-08T00:15:40.230640+0000","last_unstale":"2023-10-08T00:15:41.105947+0000","last_undegraded":"2023-10-08T00:15:41.105947+0000","last_fullsized":"2023-10-08T00:15:41.105947+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:11:18.658704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.106468+0000","last_change":"2023-10-08T00:15:41.106411+0000","last_active":"2023-10-08T00:15:41.106468+0000","last_peered":"2023-10-08T00:15:41.106468+0000","last_clean":"2023-10-08T00:15:41.106468+0000","last_became_active":"2023-10-08T00:15:41.106197+0000","last_became_peered":"2023-10-08T00:15:41.106197+0000","last_unstale":"2023-10-08T00:15:41.106468+0000","last_undegraded":"2023-10-08T00:15:41.106468+0000","last_fullsized":"2023-10-08T00:15:41.106468+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:09:52.052703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.101239+0000","last_change":"2023-10-08T00:15:40.229408+0000","last_active":"2023-10-08T00:15:41.101239+0000","last_peered":"2023-10-08T00:15:41.101239+0000","last_clean":"2023-10-08T00:15:41.101239+0000","last_became_active":"2023-10-08T00:15:40.229004+0000","last_became_peered":"2023-10-08T00:15:40.229004+0000","last_unstale":"2023-10-08T00:15:41.101239+0000","last_undegraded":"2023-10-08T00:15:41.101239+0000","last_fullsized":"2023-10-08T00:15:41.101239+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:45:37.511998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.110827+0000","last_change":"2023-10-08T00:15:41.110785+0000","last_active":"2023-10-08T00:15:41.110827+0000","last_peered":"2023-10-08T00:15:41.110827+0000","last_clean":"2023-10-08T00:15:41.110827+0000","last_became_active":"2023-10-08T00:15:41.110584+0000","last_became_peered":"2023-10-08T00:15:41.110584+0000","last_unstale":"2023-10-08T00:15:41.110827+0000","last_undegraded":"2023-10-08T00:15:41.110827+0000","last_fullsized":"2023-10-08T00:15:41.110827+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:35:02.136255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.110999+0000","last_change":"2023-10-08T00:15:40.228342+0000","last_active":"2023-10-08T00:15:41.110999+0000","last_peered":"2023-10-08T00:15:41.110999+0000","last_clean":"2023-10-08T00:15:41.110999+0000","last_became_active":"2023-10-08T00:15:40.225794+0000","last_became_peered":"2023-10-08T00:15:40.225794+0000","last_unstale":"2023-10-08T00:15:41.110999+0000","last_undegraded":"2023-10-08T00:15:41.110999+0000","last_fullsized":"2023-10-08T00:15:41.110999+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:57:50.777303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.115584+0000","last_change":"2023-10-08T00:15:41.115390+0000","last_active":"2023-10-08T00:15:41.115584+0000","last_peered":"2023-10-08T00:15:41.115584+0000","last_clean":"2023-10-08T00:15:41.115584+0000","last_became_active":"2023-10-08T00:15:41.113010+0000","last_became_peered":"2023-10-08T00:15:41.113010+0000","last_unstale":"2023-10-08T00:15:41.115584+0000","last_undegraded":"2023-10-08T00:15:41.115584+0000","last_fullsized":"2023-10-08T00:15:41.115584+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:29:22.992476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.101639+0000","last_change":"2023-10-08T00:15:40.224730+0000","last_active":"2023-10-08T00:15:41.101639+0000","last_peered":"2023-10-08T00:15:41.101639+0000","last_clean":"2023-10-08T00:15:41.101639+0000","last_became_active":"2023-10-08T00:15:40.224515+0000","last_became_peered":"2023-10-08T00:15:40.224515+0000","last_unstale":"2023-10-08T00:15:41.101639+0000","last_undegraded":"2023-10-08T00:15:41.101639+0000","last_fullsized":"2023-10-08T00:15:41.101639+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:35:03.232234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.113312+0000","last_change":"2023-10-08T00:15:41.113084+0000","last_active":"2023-10-08T00:15:41.113312+0000","last_peered":"2023-10-08T00:15:41.113312+0000","last_clean":"2023-10-08T00:15:41.113312+0000","last_became_active":"2023-10-08T00:15:41.112570+0000","last_became_peered":"2023-10-08T00:15:41.112570+0000","last_unstale":"2023-10-08T00:15:41.113312+0000","last_undegraded":"2023-10-08T00:15:41.113312+0000","last_fullsized":"2023-10-08T00:15:41.113312+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:15:38.694742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.110283+0000","last_change":"2023-10-08T00:15:40.228031+0000","last_active":"2023-10-08T00:15:41.110283+0000","last_peered":"2023-10-08T00:15:41.110283+0000","last_clean":"2023-10-08T00:15:41.110283+0000","last_became_active":"2023-10-08T00:15:40.227683+0000","last_became_peered":"2023-10-08T00:15:40.227683+0000","last_unstale":"2023-10-08T00:15:41.110283+0000","last_undegraded":"2023-10-08T00:15:41.110283+0000","last_fullsized":"2023-10-08T00:15:41.110283+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:45:21.339247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.113759+0000","last_change":"2023-10-08T00:15:41.113698+0000","last_active":"2023-10-08T00:15:41.113759+0000","last_peered":"2023-10-08T00:15:41.113759+0000","last_clean":"2023-10-08T00:15:41.113759+0000","last_became_active":"2023-10-08T00:15:41.113473+0000","last_became_peered":"2023-10-08T00:15:41.113473+0000","last_unstale":"2023-10-08T00:15:41.113759+0000","last_undegraded":"2023-10-08T00:15:41.113759+0000","last_fullsized":"2023-10-08T00:15:41.113759+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:05:33.248306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.105838+0000","last_change":"2023-10-08T00:15:40.232170+0000","last_active":"2023-10-08T00:15:41.105838+0000","last_peered":"2023-10-08T00:15:41.105838+0000","last_clean":"2023-10-08T00:15:41.105838+0000","last_became_active":"2023-10-08T00:15:40.231705+0000","last_became_peered":"2023-10-08T00:15:40.231705+0000","last_unstale":"2023-10-08T00:15:41.105838+0000","last_undegraded":"2023-10-08T00:15:41.105838+0000","last_fullsized":"2023-10-08T00:15:41.105838+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:45:37.511998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.108513+0000","last_change":"2023-10-08T00:15:41.108421+0000","last_active":"2023-10-08T00:15:41.108513+0000","last_peered":"2023-10-08T00:15:41.108513+0000","last_clean":"2023-10-08T00:15:41.108513+0000","last_became_active":"2023-10-08T00:15:41.108160+0000","last_became_peered":"2023-10-08T00:15:41.108160+0000","last_unstale":"2023-10-08T00:15:41.108513+0000","last_undegraded":"2023-10-08T00:15:41.108513+0000","last_fullsized":"2023-10-08T00:15:41.108513+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:40.212497+0000","last_clean_scrub_stamp":"2023-10-08T00:15:40.212497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:04:46.379314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"103'1","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.229393+0000","last_change":"2023-10-08T00:15:40.228623+0000","last_active":"2023-10-08T00:15:41.229393+0000","last_peered":"2023-10-08T00:15:41.229393+0000","last_clean":"2023-10-08T00:15:41.229393+0000","last_became_active":"2023-10-08T00:15:40.228332+0000","last_became_peered":"2023-10-08T00:15:40.228332+0000","last_unstale":"2023-10-08T00:15:41.229393+0000","last_undegraded":"2023-10-08T00:15:41.229393+0000","last_fullsized":"2023-10-08T00:15:41.229393+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:15:39.202332+0000","last_clean_scrub_stamp":"2023-10-08T00:15:39.202332+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:54:08.808620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":232,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T00:15:41.107323+0000","last_change":"2023-10-08T00:10:43.728490+0000","last_active":"2023-10-08T00:15:41.107323+0000","last_peered":"2023-10-08T00:15:41.107323+0000","last_clean":"2023-10-08T00:15:41.107323+0000","last_became_active":"2023-10-08T00:10:43.728341+0000","last_became_peered":"2023-10-08T00:10:43.728341+0000","last_unstale":"2023-10-08T00:15:41.107323+0000","last_undegraded":"2023-10-08T00:15:41.107323+0000","last_fullsized":"2023-10-08T00:15:41.107323+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_clean_scrub_stamp":"2023-10-08T00:10:42.716096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:06:31.035293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574910,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":552,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194759,"statfs":{"total":200000000,"available":199433790,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.51700000000000002,"15min":0.501},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53700000000000003,"15min":0.54300000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.72699999999999998,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.47799999999999998}]},{"osd":1,"last update":"Sun Oct 8 00:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.55200000000000005,"15min":0.53400000000000003},"min":{"1min":0.42399999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78200000000000003,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.57899999999999996,"15min":0.56399999999999995},"min":{"1min":0.438,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.79800000000000004,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.56699999999999995}]},{"osd":2,"last update":"Sun Oct 8 00:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54800000000000004,"15min":0.53900000000000003},"min":{"1min":0.41499999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.72499999999999998,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.53500000000000003,"15min":0.52000000000000002},"min":{"1min":0.33400000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.72399999999999998,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.55300000000000005}]},{"osd":3,"last update":"Sun Oct 8 00:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53500000000000003,"15min":0.51900000000000002},"min":{"1min":0.41199999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.74299999999999999,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.371},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56499999999999995,"15min":0.55800000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.76400000000000001,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.40000000000000002}]},{"osd":4,"last update":"Sun Oct 8 00:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.51000000000000001,"15min":0.52200000000000002},"min":{"1min":0.30399999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.72399999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.502},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.51100000000000001,"15min":0.52600000000000002},"min":{"1min":0.32600000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76500000000000001,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.59399999999999997}]},{"osd":5,"last update":"Sun Oct 8 00:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.52600000000000002,"15min":0.54800000000000004},"min":{"1min":0.33100000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.69399999999999995,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.55500000000000005,"15min":0.58399999999999996},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.75,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.622}]},{"osd":6,"last update":"Sun Oct 8 00:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.58799999999999997,"15min":0.59699999999999998},"min":{"1min":0.438,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.81699999999999995,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.60199999999999998,"15min":0.63},"min":{"1min":0.38300000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.82599999999999996,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.63900000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574910,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":553,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194759,"statfs":{"total":200000000,"available":199433216,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:14:53 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.502,"15min":0.48199999999999998},"min":{"1min":0.373,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.68799999999999994,"5min":1.012,"15min":1.012},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.60699999999999998,"15min":0.67900000000000005},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.68799999999999994,"5min":5.1230000000000002,"15min":5.1230000000000002},"last":0.42799999999999999}]},{"osd":1,"last update":"Sun Oct 8 00:14:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.54300000000000004,"15min":0.52100000000000002},"min":{"1min":0.379,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.71199999999999997,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56899999999999995,"15min":0.56299999999999994},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.79400000000000004,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.59399999999999997}]},{"osd":2,"last update":"Sun Oct 8 00:14:53 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54000000000000004,"15min":0.53000000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.75700000000000001,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57299999999999995,"15min":0.56899999999999995},"min":{"1min":0.38900000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.747,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.497}]},{"osd":3,"last update":"Sun Oct 8 00:14:53 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.55900000000000005,"15min":0.53800000000000003},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.76600000000000001,"5min":1.046,"15min":1.046},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56100000000000005,"15min":0.54200000000000004},"min":{"1min":0.35099999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78400000000000003,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.46100000000000002}]},{"osd":4,"last update":"Sun Oct 8 00:14:53 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.47099999999999997,"15min":0.495},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.55100000000000005,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.54700000000000004,"15min":0.56100000000000005},"min":{"1min":0.36199999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73599999999999999,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.626}]},{"osd":5,"last update":"Sun Oct 8 00:14:53 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.47899999999999998,"15min":0.48599999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.74199999999999999,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.52400000000000002,"15min":0.55200000000000005},"min":{"1min":0.32900000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.65200000000000002,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.54400000000000004}]},{"osd":7,"last update":"Sun Oct 8 00:14:53 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.52300000000000002,"15min":0.53900000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.72899999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.53600000000000003,"15min":0.54500000000000004},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.71799999999999997,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.53300000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1123,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194189,"statfs":{"total":200000000,"available":198849690,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.32500000000000001,"15min":0.30599999999999999},"min":{"1min":0.22,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.49199999999999999,"5min":0.56899999999999995,"15min":0.56899999999999995},"last":0.20399999999999999},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.38500000000000001,"15min":0.376},"min":{"1min":0.314,"5min":0.255,"15min":0.255},"max":{"1min":0.53000000000000003,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.32300000000000001}]},{"osd":2,"last update":"Sun Oct 8 00:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.34000000000000002,"15min":0.33400000000000002},"min":{"1min":0.254,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.50800000000000001,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.36099999999999999,"15min":0.33900000000000002},"min":{"1min":0.27900000000000003,"5min":0.252,"15min":0.252},"max":{"1min":0.54500000000000004,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.252}]},{"osd":3,"last update":"Sun Oct 8 00:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.38200000000000001,"15min":0.37},"min":{"1min":0.28399999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.57199999999999995,"5min":0.623,"15min":0.623},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.34799999999999998,"15min":0.34200000000000003},"min":{"1min":0.253,"5min":0.222,"15min":0.222},"max":{"1min":0.46700000000000003,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.30499999999999999}]},{"osd":4,"last update":"Sun Oct 8 00:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55200000000000005,"15min":0.53900000000000003},"min":{"1min":0.36299999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.504},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.56699999999999995},"min":{"1min":0.41499999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.72099999999999997,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.55400000000000005}]},{"osd":5,"last update":"Sun Oct 8 00:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55200000000000005,"15min":0.53600000000000003},"min":{"1min":0.42099999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.752,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58299999999999996,"15min":0.57499999999999996},"min":{"1min":0.39600000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.72499999999999998,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.47599999999999998}]},{"osd":6,"last update":"Sun Oct 8 00:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57899999999999996,"15min":0.56699999999999995},"min":{"1min":0.44500000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.73499999999999999,"5min":0.877,"15min":0.877},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57199999999999995,"15min":0.56100000000000005},"min":{"1min":0.45000000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77200000000000002,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.48699999999999999}]},{"osd":7,"last update":"Sun Oct 8 00:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57399999999999995,"15min":0.56299999999999994},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.76600000000000001,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.60799999999999998,"15min":0.59099999999999997},"min":{"1min":0.45200000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.53400000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574910,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":554,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194758,"statfs":{"total":200000000,"available":199432302,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 00:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.33500000000000002,"15min":0.33000000000000002},"min":{"1min":0.25,"5min":0.193,"15min":0.193},"max":{"1min":0.53200000000000003,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.20699999999999999},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.35099999999999998,"15min":0.34599999999999997},"min":{"1min":0.23799999999999999,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.48399999999999999,"5min":0.87,"15min":0.87},"last":0.38300000000000001}]},{"osd":2,"last update":"Sun Oct 8 00:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.38300000000000001,"15min":0.38600000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.56200000000000006,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.56000000000000005,"15min":0.76000000000000001},"min":{"1min":0.27000000000000002,"5min":0.185,"15min":0.185},"max":{"1min":0.55100000000000005,"5min":12.019,"15min":12.019},"last":0.25700000000000001}]},{"osd":3,"last update":"Sun Oct 8 00:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.36399999999999999,"15min":0.36199999999999999},"min":{"1min":0.27400000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.46999999999999997,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.40300000000000002,"15min":0.39500000000000002},"min":{"1min":0.28299999999999997,"5min":0.246,"15min":0.246},"max":{"1min":0.61399999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.438}]},{"osd":4,"last update":"Sun Oct 8 00:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.61399999999999999,"15min":0.61699999999999999},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.96399999999999997,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.625,"15min":0.625},"min":{"1min":0.435,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.94999999999999996,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.505}]},{"osd":5,"last update":"Sun Oct 8 00:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.79800000000000004,"15min":1.002},"min":{"1min":0.39500000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.84799999999999998,"5min":11.992000000000001,"15min":11.992000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.58699999999999997},"min":{"1min":0.40500000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.90300000000000002,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.51600000000000001}]},{"osd":6,"last update":"Sun Oct 8 00:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59499999999999997,"15min":0.58699999999999997},"min":{"1min":0.44,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.88700000000000001,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.79000000000000004,"15min":0.97899999999999998},"min":{"1min":0.41199999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.73999999999999999,"5min":11.927,"15min":11.927},"last":0.41699999999999998}]},{"osd":7,"last update":"Sun Oct 8 00:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.80200000000000005,"15min":1},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.81100000000000005,"5min":11.965,"15min":11.965},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.626,"15min":0.624},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.94099999999999995,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.61299999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574910,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":552,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194759,"statfs":{"total":200000000,"available":199433860,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.37,"15min":0.39500000000000002},"min":{"1min":0.24199999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.498,"5min":2.5139999999999998,"15min":2.5139999999999998},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.376,"15min":0.40600000000000003},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.56399999999999995,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":0.31900000000000001}]},{"osd":1,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.36899999999999999,"15min":0.39500000000000002},"min":{"1min":0.249,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.48499999999999999,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.372,"5min":0.40799999999999997,"15min":0.44},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.53400000000000003,"5min":2.5379999999999998,"15min":2.5379999999999998},"last":0.34699999999999998}]},{"osd":3,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.41099999999999998,"15min":0.433},"min":{"1min":0.255,"5min":0.254,"15min":0.254},"max":{"1min":0.54800000000000004,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.436,"15min":0.46800000000000003},"min":{"1min":0.30399999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.501,"5min":2.5619999999999998,"15min":2.5619999999999998},"last":0.38100000000000001}]},{"osd":4,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.59699999999999998,"15min":0.621},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.073,"5min":2.488,"15min":2.488},"last":0.497},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.59599999999999997,"15min":0.60799999999999998},"min":{"1min":0.41299999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.75800000000000001,"5min":2.073,"15min":2.073},"last":0.58699999999999997}]},{"osd":5,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63900000000000001,"15min":0.66100000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.754,"5min":2.2069999999999999,"15min":2.2069999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.65200000000000002,"15min":0.66600000000000004},"min":{"1min":0.45900000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0369999999999999,"5min":2.456,"15min":2.456},"last":0.56799999999999995}]},{"osd":6,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.625},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.05,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60599999999999998,"15min":0.61299999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0940000000000001,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.59599999999999997}]},{"osd":7,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63,"15min":0.63900000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0229999999999999,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.61899999999999999,"15min":0.63},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0569999999999999,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":0.53300000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574910,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":547,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194765,"statfs":{"total":200000000,"available":199439502,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.34000000000000002,"15min":0.371},"min":{"1min":0.218,"5min":0.193,"15min":0.193},"max":{"1min":0.52700000000000002,"5min":1.46,"15min":1.46},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.33400000000000002,"15min":0.34100000000000003},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":0.496,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.24199999999999999}]},{"osd":1,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.58999999999999997,"15min":0.83699999999999997},"min":{"1min":0.25,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.57599999999999996,"5min":12.351000000000001,"15min":12.351000000000001},"last":0.441},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.28399999999999997,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.58899999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.26700000000000002}]},{"osd":2,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.40100000000000002,"15min":0.42099999999999999},"min":{"1min":0.26300000000000001,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.70499999999999996,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.26800000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.63500000000000001,"5min":0.753,"15min":0.753},"last":0.45100000000000001}]},{"osd":4,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.80000000000000004,"15min":1.018},"min":{"1min":0.38400000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.876,"5min":11.333,"15min":11.333},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.82899999999999996,"15min":1.0509999999999999},"min":{"1min":0.41299999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.94899999999999995,"5min":12.321999999999999,"15min":12.321999999999999},"last":0.46300000000000002}]},{"osd":5,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.82499999999999996,"15min":1.0620000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.92000000000000004,"5min":12.291,"15min":12.291},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.60099999999999998,"15min":0.59799999999999998},"min":{"1min":0.46300000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.42999999999999999}]},{"osd":6,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.77700000000000002,"15min":0.98299999999999998},"min":{"1min":0.42299999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.96199999999999997,"5min":11.288,"15min":11.288},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.81299999999999994,"15min":1.044},"min":{"1min":0.433,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.89400000000000002,"5min":12.253,"15min":12.253},"last":0.47899999999999998}]},{"osd":7,"last update":"Sun Oct 8 00:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.58899999999999997,"15min":0.57499999999999996},"min":{"1min":0.40400000000000003,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.97299999999999998,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.81200000000000006,"15min":1.02},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.92400000000000004,"5min":11.363,"15min":11.363},"last":0.38600000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574910,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":546,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194765,"statfs":{"total":200000000,"available":199440114,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:14:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.56000000000000005,"15min":0.55400000000000005},"min":{"1min":0.316,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.72899999999999998,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57599999999999996,"15min":0.58399999999999996},"min":{"1min":0.41299999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.68899999999999995,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.67700000000000005}]},{"osd":1,"last update":"Sun Oct 8 00:14:52 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51700000000000002,"15min":0.51800000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.73599999999999999,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.52600000000000002,"15min":0.53200000000000003},"min":{"1min":0.36799999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71199999999999997,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.48799999999999999}]},{"osd":2,"last update":"Sun Oct 8 00:14:52 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.50900000000000001,"15min":0.51700000000000002},"min":{"1min":0.42699999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.72699999999999998,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51700000000000002,"15min":0.51100000000000001},"min":{"1min":0.34300000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.70499999999999996,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.57099999999999995}]},{"osd":3,"last update":"Sun Oct 8 00:14:52 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.52900000000000003,"15min":0.53700000000000003},"min":{"1min":0.40999999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.70599999999999996,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53200000000000003,"15min":0.53300000000000003},"min":{"1min":0.39500000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.72399999999999998}]},{"osd":5,"last update":"Sun Oct 8 00:14:52 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.47199999999999998,"15min":0.496},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.65100000000000002,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.49199999999999999,"15min":0.50900000000000001},"min":{"1min":0.35099999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.74399999999999999,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.79000000000000004}]},{"osd":6,"last update":"Sun Oct 8 00:14:52 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.47499999999999998,"15min":0.48899999999999999},"min":{"1min":0.30599999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.67300000000000004,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.52700000000000002,"15min":0.55300000000000005},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.71899999999999997,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.80100000000000005}]},{"osd":7,"last update":"Sun Oct 8 00:14:52 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.53200000000000003,"15min":0.55000000000000004},"min":{"1min":0.31900000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.77000000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.751},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.56799999999999995,"15min":0.57999999999999996},"min":{"1min":0.39000000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.70899999999999996,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.77500000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574910,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1130,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194181,"statfs":{"total":200000000,"available":198841922,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48299999999999998,"15min":0.48699999999999999},"min":{"1min":0.33800000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.65600000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.54500000000000004,"15min":0.55900000000000005},"min":{"1min":0.36799999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.752,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.61799999999999999}]},{"osd":1,"last update":"Sun Oct 8 00:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.52300000000000002,"15min":0.52400000000000002},"min":{"1min":0.32900000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.73799999999999999,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.51200000000000001,"15min":0.52400000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.63400000000000001,"5min":0.755,"15min":0.755},"last":0.56499999999999995}]},{"osd":2,"last update":"Sun Oct 8 00:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.52800000000000002,"15min":0.53000000000000003},"min":{"1min":0.36899999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.72199999999999998,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56000000000000005,"15min":0.56999999999999995},"min":{"1min":0.41399999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.76500000000000001,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.63300000000000001}]},{"osd":3,"last update":"Sun Oct 8 00:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.55400000000000005,"15min":0.54800000000000004},"min":{"1min":0.41499999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.72299999999999998,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56799999999999995,"15min":0.57999999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.72199999999999998,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.59399999999999997}]},{"osd":4,"last update":"Sun Oct 8 00:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.46700000000000003,"15min":0.47299999999999998},"min":{"1min":0.33300000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.76800000000000002,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.51100000000000001,"15min":0.51400000000000001},"min":{"1min":0.32200000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.80300000000000005,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.44800000000000001}]},{"osd":6,"last update":"Sun Oct 8 00:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.50900000000000001,"15min":0.502},"min":{"1min":0.442,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.78900000000000003,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.501,"15min":0.495},"min":{"1min":0.36899999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.39500000000000002}]},{"osd":7,"last update":"Sun Oct 8 00:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.52400000000000002,"15min":0.51800000000000002},"min":{"1min":0.33800000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.34899999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.78700000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.436}]}]}],"pool_statfs":[]}} 2023-10-08T00:15:45.645 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T00:15:45.645 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-08T00:15:45.797 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.797+0000 7f7e26ca3700 1 Processor -- start 2023-10-08T00:15:45.798 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.797+0000 7f7e26ca3700 1 -- start start 2023-10-08T00:15:45.800 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.798+0000 7f7e26ca3700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7e201542a0 0x7f7e200b65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:45.800 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.798+0000 7f7e26ca3700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e2006b0a0 con 0x7f7e200cd630 2023-10-08T00:15:45.800 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.798+0000 7f7e26ca3700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e2015ad20 con 0x7f7e2006aa00 2023-10-08T00:15:45.800 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.798+0000 7f7e26ca3700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e2015b2e0 con 0x7f7e201542a0 2023-10-08T00:15:45.801 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.798+0000 7f7e254a0700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f7e200cd630 0x7f7e2006a2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:60814/0 (socket says 172.21.15.99:60814) 2023-10-08T00:15:45.801 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.798+0000 7f7e254a0700 1 -- 172.21.15.99:0/2603178277 learned_addr learned my addr 172.21.15.99:0/2603178277 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:15:45.801 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.798+0000 7f7e25ca1700 1 --2- 172.21.15.99:0/2603178277 >> v2:172.21.15.191:3300/0 conn(0x7f7e201542a0 0x7f7e200b65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:45.801 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.798+0000 7f7e24c9f700 1 -- 172.21.15.99:0/2603178277 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 992888372 0 0) 0x7f7e2006b0a0 con 0x7f7e200cd630 2023-10-08T00:15:45.801 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.798+0000 7f7e24c9f700 1 -- 172.21.15.99:0/2603178277 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e0c003570 con 0x7f7e200cd630 2023-10-08T00:15:45.801 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.798+0000 7f7e24c9f700 1 -- 172.21.15.99:0/2603178277 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 660186141 0 0) 0x7f7e0c003570 con 0x7f7e200cd630 2023-10-08T00:15:45.801 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.798+0000 7f7e25ca1700 1 -- 172.21.15.99:0/2603178277 >> v1:172.21.15.191:6789/0 conn(0x7f7e2006aa00 legacy=0x7f7e2015c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:45.801 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.798+0000 7f7e25ca1700 1 -- 172.21.15.99:0/2603178277 >> v1:172.21.15.99:6789/0 conn(0x7f7e200cd630 legacy=0x7f7e2006a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:45.801 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.798+0000 7f7e25ca1700 1 -- 172.21.15.99:0/2603178277 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e2015b000 con 0x7f7e201542a0 2023-10-08T00:15:45.801 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.799+0000 7f7e25ca1700 1 --2- 172.21.15.99:0/2603178277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e201542a0 0x7f7e200b65a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c0098d0 tx=0x7f7e1c009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1512cdcab6beb4f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:45.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.799+0000 7f7e24c9f700 1 -- 172.21.15.99:0/2603178277 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e10001470 con 0x7f7e201542a0 2023-10-08T00:15:45.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.799+0000 7f7e24c9f700 1 -- 172.21.15.99:0/2603178277 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e1c011420 con 0x7f7e201542a0 2023-10-08T00:15:45.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.799+0000 7f7e24c9f700 1 -- 172.21.15.99:0/2603178277 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c0117a0 con 0x7f7e201542a0 2023-10-08T00:15:45.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.799+0000 7f7e26ca3700 1 -- 172.21.15.99:0/2603178277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e201542a0 msgr2=0x7f7e200b65a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:45.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.799+0000 7f7e26ca3700 1 --2- 172.21.15.99:0/2603178277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e201542a0 0x7f7e200b65a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c0098d0 tx=0x7f7e1c009120 comp rx=0 tx=0).stop 2023-10-08T00:15:45.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.799+0000 7f7e26ca3700 1 -- 172.21.15.99:0/2603178277 shutdown_connections 2023-10-08T00:15:45.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.799+0000 7f7e26ca3700 1 --2- 172.21.15.99:0/2603178277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e201542a0 0x7f7e200b65a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:45.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.799+0000 7f7e26ca3700 1 -- 172.21.15.99:0/2603178277 >> 172.21.15.99:0/2603178277 conn(0x7f7e20068cb0 msgr2=0x7f7e200690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:45.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.799+0000 7f7e26ca3700 1 -- 172.21.15.99:0/2603178277 shutdown_connections 2023-10-08T00:15:45.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.799+0000 7f7e26ca3700 1 -- 172.21.15.99:0/2603178277 wait complete. 2023-10-08T00:15:45.805 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.800+0000 7f7e26ca3700 1 Processor -- start 2023-10-08T00:15:45.805 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.801+0000 7f7e26ca3700 1 -- start start 2023-10-08T00:15:45.805 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.801+0000 7f7e26ca3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e2006aa00 0x7f7e20169ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:45.805 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.801+0000 7f7e26ca3700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e200cd630 0x7f7e200c7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:45.805 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.801+0000 7f7e26ca3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e201542a0 0x7f7e200c7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:45.805 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.801+0000 7f7e26ca3700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2016a740 con 0x7f7e201542a0 2023-10-08T00:15:45.805 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.801+0000 7f7e26ca3700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e201575a0 con 0x7f7e200cd630 2023-10-08T00:15:45.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.801+0000 7f7e26ca3700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e20157880 con 0x7f7e2006aa00 2023-10-08T00:15:45.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.801+0000 7f7e264a2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e201542a0 0x7f7e200c7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:45.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.801+0000 7f7e264a2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e201542a0 0x7f7e200c7690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40950/0 (socket says 172.21.15.99:40950) 2023-10-08T00:15:45.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.801+0000 7f7e264a2700 1 -- 172.21.15.99:0/1720109152 learned_addr learned my addr 172.21.15.99:0/1720109152 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:45.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.801+0000 7f7e254a0700 1 --2- 172.21.15.99:0/1720109152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e200cd630 0x7f7e200c7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:45.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.801+0000 7f7e25ca1700 1 --2- 172.21.15.99:0/1720109152 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e2006aa00 0x7f7e20169ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:45.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.802+0000 7f7e264a2700 1 -- 172.21.15.99:0/1720109152 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e2006aa00 msgr2=0x7f7e20169ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:45.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.802+0000 7f7e264a2700 1 --2- 172.21.15.99:0/1720109152 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e2006aa00 0x7f7e20169ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:45.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.802+0000 7f7e264a2700 1 -- 172.21.15.99:0/1720109152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e200cd630 msgr2=0x7f7e200c7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:45.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.802+0000 7f7e264a2700 1 --2- 172.21.15.99:0/1720109152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e200cd630 0x7f7e200c7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:45.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.802+0000 7f7e264a2700 1 -- 172.21.15.99:0/1720109152 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e1c0093c0 con 0x7f7e201542a0 2023-10-08T00:15:45.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.802+0000 7f7e264a2700 1 --2- 172.21.15.99:0/1720109152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e201542a0 0x7f7e200c7690 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f7e1800c8e0 tx=0x7f7e180016c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:45.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.802+0000 7f7e15ffb700 1 -- 172.21.15.99:0/1720109152 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e18012630 con 0x7f7e201542a0 2023-10-08T00:15:45.808 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.802+0000 7f7e15ffb700 1 -- 172.21.15.99:0/1720109152 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e18012790 con 0x7f7e201542a0 2023-10-08T00:15:45.808 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.802+0000 7f7e26ca3700 1 -- 172.21.15.99:0/1720109152 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e20157b60 con 0x7f7e201542a0 2023-10-08T00:15:45.808 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.802+0000 7f7e15ffb700 1 -- 172.21.15.99:0/1720109152 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e18012b50 con 0x7f7e201542a0 2023-10-08T00:15:45.808 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.802+0000 7f7e26ca3700 1 -- 172.21.15.99:0/1720109152 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e2016a420 con 0x7f7e201542a0 2023-10-08T00:15:45.808 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.803+0000 7f7e26ca3700 1 -- 172.21.15.99:0/1720109152 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e2016a8b0 con 0x7f7e201542a0 2023-10-08T00:15:45.809 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.803+0000 7f7e15ffb700 1 -- 172.21.15.99:0/1720109152 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7e18011380 con 0x7f7e201542a0 2023-10-08T00:15:45.809 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.804+0000 7f7e15ffb700 1 --2- 172.21.15.99:0/1720109152 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7e04066f30 0x7f7e040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:45.809 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.804+0000 7f7e15ffb700 1 -- 172.21.15.99:0/1720109152 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7e18084710 con 0x7f7e201542a0 2023-10-08T00:15:45.809 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.804+0000 7f7e25ca1700 1 --2- 172.21.15.99:0/1720109152 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7e04066f30 0x7f7e040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:45.809 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.804+0000 7f7e25ca1700 1 --2- 172.21.15.99:0/1720109152 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7e04066f30 0x7f7e040693e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c003080 tx=0x7f7e1c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:45.810 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:45.808+0000 7f7e15ffb700 1 -- 172.21.15.99:0/1720109152 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7e2016a8b0 con 0x7f7e201542a0 2023-10-08T00:15:46.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.063+0000 7f7e26ca3700 1 -- 172.21.15.99:0/1720109152 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f7e2016a8b0 con 0x7f7e201542a0 2023-10-08T00:15:46.067 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.067+0000 7f7e15ffb700 1 -- 172.21.15.99:0/1720109152 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+0 (secure 0 0 0) 0x7f7e2016a8b0 con 0x7f7e201542a0 2023-10-08T00:15:46.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.069+0000 7f7e26ca3700 1 -- 172.21.15.99:0/1720109152 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7e04066f30 msgr2=0x7f7e040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:46.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.069+0000 7f7e26ca3700 1 --2- 172.21.15.99:0/1720109152 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7e04066f30 0x7f7e040693e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c003080 tx=0x7f7e1c017000 comp rx=0 tx=0).stop 2023-10-08T00:15:46.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.069+0000 7f7e26ca3700 1 -- 172.21.15.99:0/1720109152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e201542a0 msgr2=0x7f7e200c7690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:46.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.069+0000 7f7e26ca3700 1 --2- 172.21.15.99:0/1720109152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e201542a0 0x7f7e200c7690 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f7e1800c8e0 tx=0x7f7e180016c0 comp rx=0 tx=0).stop 2023-10-08T00:15:46.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.069+0000 7f7e26ca3700 1 -- 172.21.15.99:0/1720109152 shutdown_connections 2023-10-08T00:15:46.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.069+0000 7f7e26ca3700 1 --2- 172.21.15.99:0/1720109152 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7e04066f30 0x7f7e040693e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:46.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.069+0000 7f7e26ca3700 1 --2- 172.21.15.99:0/1720109152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e201542a0 0x7f7e200c7690 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:46.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.069+0000 7f7e26ca3700 1 --2- 172.21.15.99:0/1720109152 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e2006aa00 0x7f7e20169ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:46.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.069+0000 7f7e26ca3700 1 --2- 172.21.15.99:0/1720109152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e200cd630 0x7f7e200c7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:46.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.069+0000 7f7e26ca3700 1 -- 172.21.15.99:0/1720109152 >> 172.21.15.99:0/1720109152 conn(0x7f7e20068cb0 msgr2=0x7f7e200cf7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:46.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.069+0000 7f7e26ca3700 1 -- 172.21.15.99:0/1720109152 shutdown_connections 2023-10-08T00:15:46.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.069+0000 7f7e26ca3700 1 -- 172.21.15.99:0/1720109152 wait complete. 2023-10-08T00:15:46.071 INFO:teuthology.orchestra.run.smithi099.stderr:updated caps for client.0 2023-10-08T00:15:46.082 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:15:46.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.196+0000 7f01ca64a700 1 Processor -- start 2023-10-08T00:15:46.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.196+0000 7f01ca64a700 1 -- start start 2023-10-08T00:15:46.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.197+0000 7f01ca64a700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f01c4157870 0x7f01c415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:46.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.197+0000 7f01ca64a700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f01c415a620 0x7f01c415d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:46.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.197+0000 7f01ca64a700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01c415cbb0 con 0x7f01c415d580 2023-10-08T00:15:46.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.197+0000 7f01ca64a700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01c415bfa0 con 0x7f01c4157870 2023-10-08T00:15:46.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.197+0000 7f01ca64a700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01c415c280 con 0x7f01c415a620 2023-10-08T00:15:46.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.197+0000 7f01c3fff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f01c4157870 0x7f01c415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:46.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.197+0000 7f01c8be7700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f01c415d580 0x7f01c4158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:60820/0 (socket says 172.21.15.99:60820) 2023-10-08T00:15:46.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.197+0000 7f01c3fff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f01c4157870 0x7f01c415b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40952/0 (socket says 172.21.15.99:40952) 2023-10-08T00:15:46.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.197+0000 7f01c8be7700 1 -- 172.21.15.99:0/2793562466 learned_addr learned my addr 172.21.15.99:0/2793562466 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:15:46.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.197+0000 7f01c37fe700 1 --2- 172.21.15.99:0/2793562466 >> v2:172.21.15.191:3300/0 conn(0x7f01c415a620 0x7f01c415d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:46.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.197+0000 7f01c3fff700 1 -- 172.21.15.99:0/2793562466 >> v2:172.21.15.191:3300/0 conn(0x7f01c415a620 msgr2=0x7f01c415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:46.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.197+0000 7f01c3fff700 1 --2- 172.21.15.99:0/2793562466 >> v2:172.21.15.191:3300/0 conn(0x7f01c415a620 0x7f01c415d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:46.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.197+0000 7f01c3fff700 1 -- 172.21.15.99:0/2793562466 >> v1:172.21.15.99:6789/0 conn(0x7f01c415d580 legacy=0x7f01c4158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:46.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.197+0000 7f01c2ffd700 1 -- 172.21.15.99:0/2793562466 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3428445367 0 0) 0x7f01c415cbb0 con 0x7f01c415d580 2023-10-08T00:15:46.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.197+0000 7f01c3fff700 1 -- 172.21.15.99:0/2793562466 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c415c560 con 0x7f01c4157870 2023-10-08T00:15:46.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.197+0000 7f01c3fff700 1 --2- 172.21.15.99:0/2793562466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01c4157870 0x7f01c415b6a0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f01c415cb20 tx=0x7f01b80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd1a0c62fde10cc7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:46.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.197+0000 7f01c2ffd700 1 -- 172.21.15.99:0/2793562466 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b800b6b0 con 0x7f01c4157870 2023-10-08T00:15:46.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.197+0000 7f01c2ffd700 1 -- 172.21.15.99:0/2793562466 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01b800b810 con 0x7f01c4157870 2023-10-08T00:15:46.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.197+0000 7f01c2ffd700 1 -- 172.21.15.99:0/2793562466 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b800bbb0 con 0x7f01c4157870 2023-10-08T00:15:46.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.198+0000 7f01ca64a700 1 -- 172.21.15.99:0/2793562466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01c4157870 msgr2=0x7f01c415b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:46.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.198+0000 7f01ca64a700 1 --2- 172.21.15.99:0/2793562466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01c4157870 0x7f01c415b6a0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f01c415cb20 tx=0x7f01b80091a0 comp rx=0 tx=0).stop 2023-10-08T00:15:46.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.198+0000 7f01ca64a700 1 -- 172.21.15.99:0/2793562466 shutdown_connections 2023-10-08T00:15:46.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.198+0000 7f01ca64a700 1 --2- 172.21.15.99:0/2793562466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01c4157870 0x7f01c415b6a0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:46.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.198+0000 7f01ca64a700 1 --2- 172.21.15.99:0/2793562466 >> v2:172.21.15.191:3300/0 conn(0x7f01c415a620 0x7f01c415d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:46.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.198+0000 7f01ca64a700 1 -- 172.21.15.99:0/2793562466 >> 172.21.15.99:0/2793562466 conn(0x7f01c4068ce0 msgr2=0x7f01c40690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:46.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.198+0000 7f01ca64a700 1 -- 172.21.15.99:0/2793562466 shutdown_connections 2023-10-08T00:15:46.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.198+0000 7f01ca64a700 1 -- 172.21.15.99:0/2793562466 wait complete. 2023-10-08T00:15:46.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.199+0000 7f01ca64a700 1 Processor -- start 2023-10-08T00:15:46.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.199+0000 7f01ca64a700 1 -- start start 2023-10-08T00:15:46.202 INFO:tasks.ceph.mds.c.smithi099.stderr:2023-10-08T00:15:46.199+0000 7f8c00ff9700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T00:15:46.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.200+0000 7f01ca64a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01c4157870 0x7f01c414bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:46.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.200+0000 7f01ca64a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01c415a620 0x7f01c40b0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:46.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.200+0000 7f01ca64a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01c415d580 0x7f01c4154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:46.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.200+0000 7f01ca64a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c4150760 con 0x7f01c415a620 2023-10-08T00:15:46.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.200+0000 7f01ca64a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c40b1060 con 0x7f01c4157870 2023-10-08T00:15:46.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.200+0000 7f01ca64a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01c40b1340 con 0x7f01c415d580 2023-10-08T00:15:46.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.200+0000 7f01c37fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01c415a620 0x7f01c40b0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:46.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.200+0000 7f01c37fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01c415a620 0x7f01c40b0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40966/0 (socket says 172.21.15.99:40966) 2023-10-08T00:15:46.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.200+0000 7f01c37fe700 1 -- 172.21.15.99:0/4042043152 learned_addr learned my addr 172.21.15.99:0/4042043152 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:46.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.200+0000 7f01c3fff700 1 --2- 172.21.15.99:0/4042043152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01c4157870 0x7f01c414bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:46.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.200+0000 7f01c8be7700 1 --2- 172.21.15.99:0/4042043152 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01c415d580 0x7f01c4154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:46.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.201+0000 7f01c37fe700 1 -- 172.21.15.99:0/4042043152 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01c415d580 msgr2=0x7f01c4154ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:46.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.201+0000 7f01c37fe700 1 --2- 172.21.15.99:0/4042043152 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01c415d580 0x7f01c4154ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:46.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.201+0000 7f01c37fe700 1 -- 172.21.15.99:0/4042043152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01c4157870 msgr2=0x7f01c414bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:46.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.201+0000 7f01c37fe700 1 --2- 172.21.15.99:0/4042043152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01c4157870 0x7f01c414bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:46.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.201+0000 7f01c37fe700 1 -- 172.21.15.99:0/4042043152 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b8009420 con 0x7f01c415a620 2023-10-08T00:15:46.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.201+0000 7f01c37fe700 1 --2- 172.21.15.99:0/4042043152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01c415a620 0x7f01c40b0610 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f01b400c880 tx=0x7f01b4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:46.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.201+0000 7f01abfff700 1 -- 172.21.15.99:0/4042043152 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b4011600 con 0x7f01c415a620 2023-10-08T00:15:46.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.201+0000 7f01abfff700 1 -- 172.21.15.99:0/4042043152 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01b4011760 con 0x7f01c415a620 2023-10-08T00:15:46.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.201+0000 7f01abfff700 1 -- 172.21.15.99:0/4042043152 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b4011b00 con 0x7f01c415a620 2023-10-08T00:15:46.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.201+0000 7f01ca64a700 1 -- 172.21.15.99:0/4042043152 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01c40b1620 con 0x7f01c415a620 2023-10-08T00:15:46.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.201+0000 7f01ca64a700 1 -- 172.21.15.99:0/4042043152 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01c40cf590 con 0x7f01c415a620 2023-10-08T00:15:46.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.202+0000 7f01ca64a700 1 -- 172.21.15.99:0/4042043152 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01c40cf7b0 con 0x7f01c415a620 2023-10-08T00:15:46.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.202+0000 7f01abfff700 1 -- 172.21.15.99:0/4042043152 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f01b4018360 con 0x7f01c415a620 2023-10-08T00:15:46.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.202+0000 7f01abfff700 1 --2- 172.21.15.99:0/4042043152 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f01b0066f30 0x7f01b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:46.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.203+0000 7f01abfff700 1 -- 172.21.15.99:0/4042043152 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f01b40843d0 con 0x7f01c415a620 2023-10-08T00:15:46.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.203+0000 7f01c3fff700 1 --2- 172.21.15.99:0/4042043152 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f01b0066f30 0x7f01b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:46.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.203+0000 7f01c3fff700 1 --2- 172.21.15.99:0/4042043152 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f01b0066f30 0x7f01b00693e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f01b80093f0 tx=0x7f01b8017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:46.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.207+0000 7f01abfff700 1 -- 172.21.15.99:0/4042043152 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f01b4051640 con 0x7f01c415a620 2023-10-08T00:15:46.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.432+0000 7f01ca64a700 1 -- 172.21.15.99:0/4042043152 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f01c404c3c0 con 0x7f01c415a620 2023-10-08T00:15:46.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.432+0000 7f01abfff700 1 -- 172.21.15.99:0/4042043152 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4599 (secure 0 0 0) 0x7f01b4054c60 con 0x7f01c415a620 2023-10-08T00:15:46.433 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:15:46.433 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":48,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5426,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/280880696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":280880696},{"type":"v1","addr":"172.21.15.191:6836","nonce":280880696}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":5902,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3148129562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":3148129562},{"type":"v1","addr":"172.21.15.99:6835","nonce":3148129562}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":5917,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/1825600944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":1825600944},{"type":"v1","addr":"172.21.15.191:6837","nonce":1825600944}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:15:40.595166+0000","modified":"2023-10-08T00:15:44.232916+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6360},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6360":{"gid":6360,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6834/146609901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":146609901},{"type":"v1","addr":"172.21.15.99:6834","nonce":146609901}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-08T00:15:46.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.434+0000 7f01ca64a700 1 -- 172.21.15.99:0/4042043152 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f01b0066f30 msgr2=0x7f01b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:46.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.434+0000 7f01ca64a700 1 --2- 172.21.15.99:0/4042043152 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f01b0066f30 0x7f01b00693e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f01b80093f0 tx=0x7f01b8017000 comp rx=0 tx=0).stop 2023-10-08T00:15:46.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.434+0000 7f01ca64a700 1 -- 172.21.15.99:0/4042043152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01c415a620 msgr2=0x7f01c40b0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:46.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.434+0000 7f01ca64a700 1 --2- 172.21.15.99:0/4042043152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01c415a620 0x7f01c40b0610 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f01b400c880 tx=0x7f01b4008040 comp rx=0 tx=0).stop 2023-10-08T00:15:46.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.434+0000 7f01ca64a700 1 -- 172.21.15.99:0/4042043152 shutdown_connections 2023-10-08T00:15:46.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.434+0000 7f01ca64a700 1 --2- 172.21.15.99:0/4042043152 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f01b0066f30 0x7f01b00693e0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:46.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.434+0000 7f01ca64a700 1 --2- 172.21.15.99:0/4042043152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01c4157870 0x7f01c414bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:46.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.434+0000 7f01ca64a700 1 --2- 172.21.15.99:0/4042043152 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01c415d580 0x7f01c4154ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:46.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.434+0000 7f01ca64a700 1 --2- 172.21.15.99:0/4042043152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01c415a620 0x7f01c40b0610 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:46.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.434+0000 7f01ca64a700 1 -- 172.21.15.99:0/4042043152 >> 172.21.15.99:0/4042043152 conn(0x7f01c4068ce0 msgr2=0x7f01c415b650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:46.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.434+0000 7f01ca64a700 1 -- 172.21.15.99:0/4042043152 shutdown_connections 2023-10-08T00:15:46.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.434+0000 7f01ca64a700 1 -- 172.21.15.99:0/4042043152 wait complete. 2023-10-08T00:15:46.436 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 48 2023-10-08T00:15:46.850 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 48, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T00:15:40.595166+0000', 'modified': '2023-10-08T00:15:44.232916+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6360}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6360': {'gid': 6360, 'name': 'c', 'rank': 0, 'incarnation': 44, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.99:6834/146609901', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6832', 'nonce': 146609901}, {'type': 'v1', 'addr': '172.21.15.99:6834', 'nonce': 146609901}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-08T00:15:46.850 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T00:15:46.851 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-08T00:15:46.966 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.965+0000 7f59d7ef3700 1 Processor -- start 2023-10-08T00:15:46.966 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.965+0000 7f59d7ef3700 1 -- start start 2023-10-08T00:15:46.968 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.966+0000 7f59d7ef3700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f59d0157870 0x7f59d015b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:46.968 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.966+0000 7f59d7ef3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f59d015a620 0x7f59d015d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:46.968 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.966+0000 7f59d7ef3700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f59d015d580 0x7f59d0158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:46.968 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.966+0000 7f59d7ef3700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59d015cbb0 con 0x7f59d0157870 2023-10-08T00:15:46.968 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.966+0000 7f59d7ef3700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59d015bfa0 con 0x7f59d015d580 2023-10-08T00:15:46.968 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.966+0000 7f59d7ef3700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59d015c280 con 0x7f59d015a620 2023-10-08T00:15:46.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.966+0000 7f59d5c8f700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f59d0157870 0x7f59d015b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:46.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.966+0000 7f59d5c8f700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f59d0157870 0x7f59d015b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40974/0 (socket says 172.21.15.99:40974) 2023-10-08T00:15:46.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.966+0000 7f59d5c8f700 1 -- 172.21.15.99:0/19831580 learned_addr learned my addr 172.21.15.99:0/19831580 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:46.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.966+0000 7f59d6490700 1 --2- 172.21.15.99:0/19831580 >> v2:172.21.15.191:3300/0 conn(0x7f59d015d580 0x7f59d0158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:46.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.966+0000 7f59d548e700 1 --2- 172.21.15.99:0/19831580 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f59d015a620 0x7f59d015d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:46.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.966+0000 7f59d5c8f700 1 -- 172.21.15.99:0/19831580 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f59d015a620 msgr2=0x7f59d015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:46.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.966+0000 7f59d5c8f700 1 --2- 172.21.15.99:0/19831580 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f59d015a620 0x7f59d015d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:46.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.966+0000 7f59d5c8f700 1 -- 172.21.15.99:0/19831580 >> v2:172.21.15.191:3300/0 conn(0x7f59d015d580 msgr2=0x7f59d0158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:46.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.966+0000 7f59d5c8f700 1 --2- 172.21.15.99:0/19831580 >> v2:172.21.15.191:3300/0 conn(0x7f59d015d580 0x7f59d0158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:46.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.966+0000 7f59d5c8f700 1 -- 172.21.15.99:0/19831580 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d015c560 con 0x7f59d0157870 2023-10-08T00:15:46.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.967+0000 7f59d548e700 1 --2- 172.21.15.99:0/19831580 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f59d015a620 0x7f59d015d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:15:46.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.967+0000 7f59d5c8f700 1 --2- 172.21.15.99:0/19831580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59d0157870 0x7f59d015b6a0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f59d00b03b0 tx=0x7f59cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=642b89143bbc8ac6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:46.970 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.967+0000 7f59d4c8d700 1 -- 172.21.15.99:0/19831580 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc009e70 con 0x7f59d0157870 2023-10-08T00:15:46.970 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.967+0000 7f59d4c8d700 1 -- 172.21.15.99:0/19831580 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59cc00b860 con 0x7f59d0157870 2023-10-08T00:15:46.970 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.967+0000 7f59d4c8d700 1 -- 172.21.15.99:0/19831580 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc00bc00 con 0x7f59d0157870 2023-10-08T00:15:46.970 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.967+0000 7f59d7ef3700 1 -- 172.21.15.99:0/19831580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59d0157870 msgr2=0x7f59d015b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:46.970 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.967+0000 7f59d7ef3700 1 --2- 172.21.15.99:0/19831580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59d0157870 0x7f59d015b6a0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f59d00b03b0 tx=0x7f59cc009250 comp rx=0 tx=0).stop 2023-10-08T00:15:46.970 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.967+0000 7f59d7ef3700 1 -- 172.21.15.99:0/19831580 shutdown_connections 2023-10-08T00:15:46.970 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.967+0000 7f59d7ef3700 1 --2- 172.21.15.99:0/19831580 >> v2:172.21.15.191:3300/0 conn(0x7f59d015d580 0x7f59d0158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:46.970 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.967+0000 7f59d7ef3700 1 --2- 172.21.15.99:0/19831580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59d0157870 0x7f59d015b6a0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:46.970 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.967+0000 7f59d7ef3700 1 --2- 172.21.15.99:0/19831580 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f59d015a620 0x7f59d015d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:46.970 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.967+0000 7f59d7ef3700 1 -- 172.21.15.99:0/19831580 >> 172.21.15.99:0/19831580 conn(0x7f59d0068ce0 msgr2=0x7f59d00690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:46.971 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.967+0000 7f59d7ef3700 1 -- 172.21.15.99:0/19831580 shutdown_connections 2023-10-08T00:15:46.971 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.968+0000 7f59d7ef3700 1 -- 172.21.15.99:0/19831580 wait complete. 2023-10-08T00:15:46.973 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.968+0000 7f59d7ef3700 1 Processor -- start 2023-10-08T00:15:46.973 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.969+0000 7f59d7ef3700 1 -- start start 2023-10-08T00:15:46.973 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.969+0000 7f59d7ef3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59d0157870 0x7f59d016e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:46.973 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.969+0000 7f59d7ef3700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59d015a620 0x7f59d00cfc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:46.973 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.969+0000 7f59d7ef3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f59d015d580 0x7f59d00d01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:46.973 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.969+0000 7f59d7ef3700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d016ed60 con 0x7f59d0157870 2023-10-08T00:15:46.974 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.969+0000 7f59d7ef3700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d014ef70 con 0x7f59d015a620 2023-10-08T00:15:46.974 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.969+0000 7f59d7ef3700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59d016bab0 con 0x7f59d015d580 2023-10-08T00:15:46.974 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.969+0000 7f59d5c8f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59d0157870 0x7f59d016e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:46.974 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.969+0000 7f59d5c8f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59d0157870 0x7f59d016e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40980/0 (socket says 172.21.15.99:40980) 2023-10-08T00:15:46.974 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.969+0000 7f59d5c8f700 1 -- 172.21.15.99:0/3425426865 learned_addr learned my addr 172.21.15.99:0/3425426865 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:46.974 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.969+0000 7f59d548e700 1 --2- 172.21.15.99:0/3425426865 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59d015a620 0x7f59d00cfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:46.974 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.970+0000 7f59d6490700 1 --2- 172.21.15.99:0/3425426865 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f59d015d580 0x7f59d00d01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:46.974 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.970+0000 7f59d5c8f700 1 -- 172.21.15.99:0/3425426865 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f59d015d580 msgr2=0x7f59d00d01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:46.974 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.970+0000 7f59d5c8f700 1 --2- 172.21.15.99:0/3425426865 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f59d015d580 0x7f59d00d01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:46.974 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.970+0000 7f59d5c8f700 1 -- 172.21.15.99:0/3425426865 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59d015a620 msgr2=0x7f59d00cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:46.974 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.970+0000 7f59d5c8f700 1 --2- 172.21.15.99:0/3425426865 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59d015a620 0x7f59d00cfc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:46.975 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.970+0000 7f59d5c8f700 1 -- 172.21.15.99:0/3425426865 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59cc0094f0 con 0x7f59d0157870 2023-10-08T00:15:46.975 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.970+0000 7f59d5c8f700 1 --2- 172.21.15.99:0/3425426865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59d0157870 0x7f59d016e130 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f59cc009290 tx=0x7f59cc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:46.975 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.970+0000 7f59c5ffb700 1 -- 172.21.15.99:0/3425426865 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc00bbe0 con 0x7f59d0157870 2023-10-08T00:15:46.975 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.970+0000 7f59c5ffb700 1 -- 172.21.15.99:0/3425426865 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59cc01f070 con 0x7f59d0157870 2023-10-08T00:15:46.975 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.970+0000 7f59d7ef3700 1 -- 172.21.15.99:0/3425426865 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59d00d0890 con 0x7f59d0157870 2023-10-08T00:15:46.975 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.970+0000 7f59c5ffb700 1 -- 172.21.15.99:0/3425426865 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc026690 con 0x7f59d0157870 2023-10-08T00:15:46.975 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.970+0000 7f59d7ef3700 1 -- 172.21.15.99:0/3425426865 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59d014ec00 con 0x7f59d0157870 2023-10-08T00:15:46.975 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.971+0000 7f59c5ffb700 1 -- 172.21.15.99:0/3425426865 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f59cc01a430 con 0x7f59d0157870 2023-10-08T00:15:46.975 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.972+0000 7f59c5ffb700 1 --2- 172.21.15.99:0/3425426865 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f59b4066f30 0x7f59b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:46.975 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.972+0000 7f59c5ffb700 1 -- 172.21.15.99:0/3425426865 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f59cc021070 con 0x7f59d0157870 2023-10-08T00:15:46.975 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.972+0000 7f59d548e700 1 --2- 172.21.15.99:0/3425426865 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f59b4066f30 0x7f59b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:46.975 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.973+0000 7f59d548e700 1 --2- 172.21.15.99:0/3425426865 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f59b4066f30 0x7f59b40693e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f59c0006f00 tx=0x7f59c0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:46.982 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.974+0000 7f59e11ebb80 1 Processor -- start 2023-10-08T00:15:46.982 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.974+0000 7f59e11ebb80 1 -- start start 2023-10-08T00:15:46.982 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.975+0000 7f59e11ebb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ba7522a50 0x560ba7522e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:46.982 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.975+0000 7f59e11ebb80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560ba756cd00 0x560ba754ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:46.982 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.975+0000 7f59e11ebb80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560ba754bf60 0x560ba751ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:46.982 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.975+0000 7f59e11ebb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ba7479a70 con 0x560ba7522a50 2023-10-08T00:15:46.982 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.975+0000 7f59e11ebb80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ba7477270 con 0x560ba756cd00 2023-10-08T00:15:46.983 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.975+0000 7f59e11ebb80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x560ba7473060 con 0x560ba754bf60 2023-10-08T00:15:46.983 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.975+0000 7f59d5c8f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ba7522a50 0x560ba7522e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:46.983 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.975+0000 7f59d5c8f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ba7522a50 0x560ba7522e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40990/0 (socket says 172.21.15.99:40990) 2023-10-08T00:15:46.983 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.975+0000 7f59d5c8f700 1 -- 172.21.15.99:0/3845603918 learned_addr learned my addr 172.21.15.99:0/3845603918 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:46.983 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.975+0000 7f59d548e700 1 --2- 172.21.15.99:0/3845603918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560ba756cd00 0x560ba754ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:46.983 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.975+0000 7f59d6490700 1 --2- 172.21.15.99:0/3845603918 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560ba754bf60 0x560ba751ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:46.983 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.975+0000 7f59d5c8f700 1 -- 172.21.15.99:0/3845603918 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560ba754bf60 msgr2=0x560ba751ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:46.983 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.975+0000 7f59d5c8f700 1 --2- 172.21.15.99:0/3845603918 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560ba754bf60 0x560ba751ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:46.983 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.975+0000 7f59d5c8f700 1 -- 172.21.15.99:0/3845603918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560ba756cd00 msgr2=0x560ba754ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:46.983 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.975+0000 7f59d5c8f700 1 --2- 172.21.15.99:0/3845603918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560ba756cd00 0x560ba754ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:46.983 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.975+0000 7f59d5c8f700 1 -- 172.21.15.99:0/3845603918 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ba730d270 con 0x560ba7522a50 2023-10-08T00:15:46.983 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.975+0000 7f59d548e700 1 --2- 172.21.15.99:0/3845603918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560ba756cd00 0x560ba754ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:15:46.984 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.975+0000 7f59d5c8f700 1 --2- 172.21.15.99:0/3845603918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ba7522a50 0x560ba7522e20 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f59cc056e60 tx=0x7f59cc0847f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88dbf0f61be3e109 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:46.984 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.975+0000 7f59baffd700 1 -- 172.21.15.99:0/3845603918 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc0572a0 con 0x560ba7522a50 2023-10-08T00:15:46.984 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.975+0000 7f59baffd700 1 -- 172.21.15.99:0/3845603918 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59cc05a4f0 con 0x560ba7522a50 2023-10-08T00:15:46.984 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.975+0000 7f59baffd700 1 -- 172.21.15.99:0/3845603918 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc05e240 con 0x560ba7522a50 2023-10-08T00:15:46.984 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.976+0000 7f59e11ebb80 1 -- 172.21.15.99:0/3845603918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ba7522a50 msgr2=0x560ba7522e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:46.984 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.976+0000 7f59e11ebb80 1 --2- 172.21.15.99:0/3845603918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ba7522a50 0x560ba7522e20 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f59cc056e60 tx=0x7f59cc0847f0 comp rx=0 tx=0).stop 2023-10-08T00:15:46.984 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.976+0000 7f59e11ebb80 1 -- 172.21.15.99:0/3845603918 shutdown_connections 2023-10-08T00:15:46.984 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.976+0000 7f59e11ebb80 1 --2- 172.21.15.99:0/3845603918 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560ba754bf60 0x560ba751ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:46.984 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.976+0000 7f59e11ebb80 1 --2- 172.21.15.99:0/3845603918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ba7522a50 0x560ba7522e20 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:46.984 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.976+0000 7f59e11ebb80 1 --2- 172.21.15.99:0/3845603918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560ba756cd00 0x560ba754ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:46.984 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.976+0000 7f59e11ebb80 1 -- 172.21.15.99:0/3845603918 >> 172.21.15.99:0/3845603918 conn(0x560ba7524370 msgr2=0x560ba7524740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:46.984 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.976+0000 7f59e11ebb80 1 -- 172.21.15.99:0/3845603918 shutdown_connections 2023-10-08T00:15:46.985 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.976+0000 7f59e11ebb80 1 -- 172.21.15.99:0/3845603918 wait complete. 2023-10-08T00:15:46.985 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.976+0000 7f59e11ebb80 1 -- start start 2023-10-08T00:15:46.985 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.977+0000 7f59e11ebb80 1 Processor -- start 2023-10-08T00:15:46.985 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.977+0000 7f59e11ebb80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560ba7522a50 0x560ba7567030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:46.985 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.977+0000 7f59e11ebb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ba754bf60 0x560ba7547c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:46.985 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.977+0000 7f59e11ebb80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560ba756cd00 0x560ba7563170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:46.985 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.977+0000 7f59e11ebb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ba741e890 con 0x560ba754bf60 2023-10-08T00:15:46.985 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.977+0000 7f59e11ebb80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ba7439ea0 con 0x560ba756cd00 2023-10-08T00:15:46.985 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.977+0000 7f59e11ebb80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x560ba72f5e30 con 0x560ba7522a50 2023-10-08T00:15:46.985 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.977+0000 7f59e11ebb80 10 client.0 _ll_register_callbacks cb 0x560ba74e6110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T00:15:46.985 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.977+0000 7f59d548e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ba754bf60 0x560ba7547c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:46.986 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.978+0000 7f59d548e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ba754bf60 0x560ba7547c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41002/0 (socket says 172.21.15.99:41002) 2023-10-08T00:15:46.986 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.978+0000 7f59d548e700 1 -- 172.21.15.99:0/345298862 learned_addr learned my addr 172.21.15.99:0/345298862 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:46.986 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.978+0000 7f59d5c8f700 1 --2- 172.21.15.99:0/345298862 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560ba7522a50 0x560ba7567030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:46.986 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.978+0000 7f59d6490700 1 --2- 172.21.15.99:0/345298862 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560ba756cd00 0x560ba7563170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:46.986 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.978+0000 7f59d548e700 1 -- 172.21.15.99:0/345298862 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560ba7522a50 msgr2=0x560ba7567030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:46.986 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.978+0000 7f59d548e700 1 --2- 172.21.15.99:0/345298862 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560ba7522a50 0x560ba7567030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:46.986 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.978+0000 7f59d548e700 1 -- 172.21.15.99:0/345298862 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560ba756cd00 msgr2=0x560ba7563170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:46.986 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.978+0000 7f59d548e700 1 --2- 172.21.15.99:0/345298862 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560ba756cd00 0x560ba7563170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:46.986 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.978+0000 7f59d548e700 1 -- 172.21.15.99:0/345298862 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59d014ec00 con 0x560ba754bf60 2023-10-08T00:15:46.986 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.978+0000 7f59d548e700 1 --2- 172.21.15.99:0/345298862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ba754bf60 0x560ba7547c00 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f59d015aa20 tx=0x7f59c0019040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d722c874fcb8bb5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:46.986 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.978+0000 7f59affff700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T00:15:46.986 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.978+0000 7f59affff700 1 -- 172.21.15.99:0/345298862 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c0022630 con 0x560ba754bf60 2023-10-08T00:15:46.987 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.978+0000 7f59affff700 1 -- 172.21.15.99:0/345298862 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f59c0022790 con 0x560ba754bf60 2023-10-08T00:15:46.987 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.978+0000 7f59e11ebb80 1 -- 172.21.15.99:0/345298862 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560ba7517b70 con 0x560ba754bf60 2023-10-08T00:15:46.987 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.978+0000 7f59affff700 1 -- 172.21.15.99:0/345298862 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59c0022b60 con 0x560ba754bf60 2023-10-08T00:15:46.987 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.978+0000 7f59e11ebb80 10 client.6510 fetch_fsmap learned FSMap version 48 2023-10-08T00:15:46.987 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.978+0000 7f59e11ebb80 1 -- 172.21.15.99:0/345298862 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x560ba72affe0 con 0x560ba754bf60 2023-10-08T00:15:46.987 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.978+0000 7f59affff700 1 -- 172.21.15.99:0/345298862 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560ba7517b70 con 0x560ba754bf60 2023-10-08T00:15:46.987 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.979+0000 7f59affff700 1 -- 172.21.15.99:0/345298862 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 48) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f59c0022cc0 con 0x560ba754bf60 2023-10-08T00:15:46.987 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.979+0000 7f59e11ebb80 10 client.6510 fetch_fsmap finished waiting for FSMap version 48 2023-10-08T00:15:46.987 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.979+0000 7f59e11ebb80 10 client.6510 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-08T00:15:46.987 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.979+0000 7f59e11ebb80 20 client.6510 populate_metadata read hostname 'smithi099' 2023-10-08T00:15:46.987 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.979+0000 7f59e11ebb80 1 --2- 172.21.15.99:0/345298862 >> [v2:172.21.15.99:6832/146609901,v1:172.21.15.99:6834/146609901] conn(0x560ba75663b0 0x560ba7566780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:46.987 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.979+0000 7f59e11ebb80 4 client.6510 mds_command: new command op to 6360 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-08T00:15:46.988 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.979+0000 7f59e11ebb80 1 -- 172.21.15.99:0/345298862 --> [v2:172.21.15.99:6832/146609901,v1:172.21.15.99:6834/146609901] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560ba74f4aa0 con 0x560ba75663b0 2023-10-08T00:15:46.988 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.979+0000 7f59d5c8f700 1 --2- 172.21.15.99:0/345298862 >> [v2:172.21.15.99:6832/146609901,v1:172.21.15.99:6834/146609901] conn(0x560ba75663b0 0x560ba7566780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:46.988 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.979+0000 7f59affff700 10 client.6510 ms_handle_connect on v2:172.21.15.99:6832/146609901 2023-10-08T00:15:46.988 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.979+0000 7f59d5c8f700 1 --2- 172.21.15.99:0/345298862 >> [v2:172.21.15.99:6832/146609901,v1:172.21.15.99:6834/146609901] conn(0x560ba75663b0 0x560ba7566780 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4d92ad807b644d8 server_cookie=8f0a51ad20c98146 in_seq=0 out_seq=0 2023-10-08T00:15:46.988 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.980+0000 7f59affff700 1 -- 172.21.15.99:0/345298862 <== mds.0 v2:172.21.15.99:6832/146609901 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x560ba74f4aa0 con 0x560ba75663b0 2023-10-08T00:15:46.988 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.980+0000 7f59affff700 10 client.6510 handle_command_reply: tid=0 2023-10-08T00:15:46.988 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.980+0000 7f59e11ebb80 1 client.6510 shutdown 2023-10-08T00:15:46.988 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.980+0000 7f59e11ebb80 1 -- 172.21.15.99:0/345298862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ba754bf60 msgr2=0x560ba7547c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:46.988 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.980+0000 7f59e11ebb80 1 --2- 172.21.15.99:0/345298862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ba754bf60 0x560ba7547c00 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f59d015aa20 tx=0x7f59c0019040 comp rx=0 tx=0).stop 2023-10-08T00:15:46.988 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.980+0000 7f59e11ebb80 1 -- 172.21.15.99:0/345298862 shutdown_connections 2023-10-08T00:15:46.988 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.980+0000 7f59e11ebb80 1 --2- 172.21.15.99:0/345298862 >> [v2:172.21.15.99:6832/146609901,v1:172.21.15.99:6834/146609901] conn(0x560ba75663b0 0x560ba7566780 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:46.988 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.980+0000 7f59e11ebb80 1 --2- 172.21.15.99:0/345298862 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560ba756cd00 0x560ba7563170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:46.989 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.980+0000 7f59e11ebb80 1 --2- 172.21.15.99:0/345298862 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560ba7522a50 0x560ba7567030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:46.989 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.980+0000 7f59e11ebb80 1 --2- 172.21.15.99:0/345298862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ba754bf60 0x560ba7547c00 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:46.989 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.980+0000 7f59affff700 0 client.6510 ms_handle_reset on v2:172.21.15.99:6832/146609901 2023-10-08T00:15:46.989 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.980+0000 7f59e11ebb80 1 -- 172.21.15.99:0/345298862 >> 172.21.15.99:0/345298862 conn(0x560ba7524370 msgr2=0x560ba7568550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:46.989 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.980+0000 7f59e11ebb80 1 -- 172.21.15.99:0/345298862 shutdown_connections 2023-10-08T00:15:46.989 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.980+0000 7f59e11ebb80 1 -- 172.21.15.99:0/345298862 wait complete. 2023-10-08T00:15:46.989 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.980+0000 7f59e11ebb80 20 client.6510 trim_cache size 0 max 16384 2023-10-08T00:15:46.997 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.997+0000 7f59e11ebb80 1 Processor -- start 2023-10-08T00:15:46.997 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.997+0000 7f59e11ebb80 1 -- start start 2023-10-08T00:15:47.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.997+0000 7f59e11ebb80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560ba7522a50 0x560ba7566950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:47.004 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.997+0000 7f59e11ebb80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560ba754bf60 0x560ba7560c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:47.004 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.997+0000 7f59e11ebb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ba756cd00 0x560ba7561160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:47.004 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.997+0000 7f59e11ebb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ba751d5a0 con 0x560ba756cd00 2023-10-08T00:15:47.004 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.997+0000 7f59e11ebb80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ba751d830 con 0x560ba754bf60 2023-10-08T00:15:47.004 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.997+0000 7f59e11ebb80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x560ba758c6b0 con 0x560ba7522a50 2023-10-08T00:15:47.004 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.997+0000 7f59d6490700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ba756cd00 0x560ba7561160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:47.004 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.997+0000 7f59d6490700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ba756cd00 0x560ba7561160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41004/0 (socket says 172.21.15.99:41004) 2023-10-08T00:15:47.004 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.997+0000 7f59d6490700 1 -- 172.21.15.99:0/1161806882 learned_addr learned my addr 172.21.15.99:0/1161806882 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:47.005 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.998+0000 7f59d548e700 1 --2- 172.21.15.99:0/1161806882 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560ba754bf60 0x560ba7560c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:47.005 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.998+0000 7f59d6490700 1 -- 172.21.15.99:0/1161806882 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560ba7522a50 msgr2=0x560ba7566950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:47.005 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.998+0000 7f59d5c8f700 1 --2- 172.21.15.99:0/1161806882 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560ba7522a50 0x560ba7566950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:47.005 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.998+0000 7f59d6490700 1 --2- 172.21.15.99:0/1161806882 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560ba7522a50 0x560ba7566950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:47.005 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.998+0000 7f59d6490700 1 -- 172.21.15.99:0/1161806882 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560ba754bf60 msgr2=0x560ba7560c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:47.005 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.998+0000 7f59d6490700 1 --2- 172.21.15.99:0/1161806882 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560ba754bf60 0x560ba7560c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:47.005 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.998+0000 7f59d6490700 1 -- 172.21.15.99:0/1161806882 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ba7441780 con 0x560ba756cd00 2023-10-08T00:15:47.005 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.998+0000 7f59d548e700 1 --2- 172.21.15.99:0/1161806882 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560ba754bf60 0x560ba7560c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:15:47.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.998+0000 7f59d5c8f700 1 --2- 172.21.15.99:0/1161806882 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560ba7522a50 0x560ba7566950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:15:47.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.998+0000 7f59d6490700 1 --2- 172.21.15.99:0/1161806882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ba756cd00 0x560ba7561160 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f59d015d9e0 tx=0x7f59c800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dffc538b5db3ff11 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:47.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.998+0000 7f59bb7fe700 1 -- 172.21.15.99:0/1161806882 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c8017650 con 0x560ba756cd00 2023-10-08T00:15:47.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.998+0000 7f59bb7fe700 1 -- 172.21.15.99:0/1161806882 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c800f070 con 0x560ba756cd00 2023-10-08T00:15:47.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.998+0000 7f59bb7fe700 1 -- 172.21.15.99:0/1161806882 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c80179c0 con 0x560ba756cd00 2023-10-08T00:15:47.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.998+0000 7f59e11ebb80 1 -- 172.21.15.99:0/1161806882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ba756cd00 msgr2=0x560ba7561160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:47.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.998+0000 7f59e11ebb80 1 --2- 172.21.15.99:0/1161806882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ba756cd00 0x560ba7561160 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f59d015d9e0 tx=0x7f59c800b040 comp rx=0 tx=0).stop 2023-10-08T00:15:47.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.998+0000 7f59e11ebb80 1 -- 172.21.15.99:0/1161806882 shutdown_connections 2023-10-08T00:15:47.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.998+0000 7f59e11ebb80 1 --2- 172.21.15.99:0/1161806882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ba756cd00 0x560ba7561160 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:47.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.998+0000 7f59e11ebb80 1 --2- 172.21.15.99:0/1161806882 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560ba7522a50 0x560ba7566950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:47.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.998+0000 7f59e11ebb80 1 --2- 172.21.15.99:0/1161806882 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560ba754bf60 0x560ba7560c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:47.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.998+0000 7f59e11ebb80 1 -- 172.21.15.99:0/1161806882 >> 172.21.15.99:0/1161806882 conn(0x560ba7524370 msgr2=0x560ba7552790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:47.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.999+0000 7f59e11ebb80 1 -- 172.21.15.99:0/1161806882 shutdown_connections 2023-10-08T00:15:47.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.999+0000 7f59e11ebb80 1 -- 172.21.15.99:0/1161806882 wait complete. 2023-10-08T00:15:47.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.999+0000 7f59e11ebb80 1 -- start start 2023-10-08T00:15:47.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:46.999+0000 7f59e11ebb80 1 Processor -- start 2023-10-08T00:15:47.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.000+0000 7f59e11ebb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ba7522a50 0x560ba7550590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:47.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.000+0000 7f59e11ebb80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560ba754bf60 0x560ba7560c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:47.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.000+0000 7f59e11ebb80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560ba756cd00 0x560ba75529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:47.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.000+0000 7f59e11ebb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ba756d940 con 0x560ba7522a50 2023-10-08T00:15:47.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.000+0000 7f59e11ebb80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ba756e040 con 0x560ba754bf60 2023-10-08T00:15:47.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.000+0000 7f59e11ebb80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x560ba756dbb0 con 0x560ba756cd00 2023-10-08T00:15:47.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.000+0000 7f59e11ebb80 10 client.0 _ll_register_callbacks cb 0x560ba74e6110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T00:15:47.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.000+0000 7f59d5c8f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ba7522a50 0x560ba7550590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:47.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.000+0000 7f59d5c8f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ba7522a50 0x560ba7550590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41020/0 (socket says 172.21.15.99:41020) 2023-10-08T00:15:47.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.000+0000 7f59d5c8f700 1 -- 172.21.15.99:0/1243710347 learned_addr learned my addr 172.21.15.99:0/1243710347 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:47.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.000+0000 7f59d6490700 1 --2- 172.21.15.99:0/1243710347 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560ba756cd00 0x560ba75529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:47.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.000+0000 7f59d548e700 1 --2- 172.21.15.99:0/1243710347 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560ba754bf60 0x560ba7560c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:47.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.000+0000 7f59d5c8f700 1 -- 172.21.15.99:0/1243710347 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560ba756cd00 msgr2=0x560ba75529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:47.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.000+0000 7f59d5c8f700 1 --2- 172.21.15.99:0/1243710347 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560ba756cd00 0x560ba75529d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:47.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.000+0000 7f59d5c8f700 1 -- 172.21.15.99:0/1243710347 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560ba754bf60 msgr2=0x560ba7560c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:47.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.000+0000 7f59d5c8f700 1 --2- 172.21.15.99:0/1243710347 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560ba754bf60 0x560ba7560c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:47.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.000+0000 7f59d5c8f700 1 -- 172.21.15.99:0/1243710347 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59c80073a0 con 0x560ba7522a50 2023-10-08T00:15:47.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.000+0000 7f59d5c8f700 1 --2- 172.21.15.99:0/1243710347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ba7522a50 0x560ba7550590 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f59cc084970 tx=0x7f59cc06f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1071baab73660a98 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:47.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.000+0000 7f59affff700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T00:15:47.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.001+0000 7f59affff700 1 -- 172.21.15.99:0/1243710347 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc0553e0 con 0x560ba7522a50 2023-10-08T00:15:47.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.001+0000 7f59affff700 1 -- 172.21.15.99:0/1243710347 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f59cc086040 con 0x560ba7522a50 2023-10-08T00:15:47.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.001+0000 7f59e11ebb80 1 -- 172.21.15.99:0/1243710347 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f59cc05a4f0 con 0x560ba7522a50 2023-10-08T00:15:47.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.001+0000 7f59affff700 1 -- 172.21.15.99:0/1243710347 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59cc086200 con 0x560ba7522a50 2023-10-08T00:15:47.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.001+0000 7f59e11ebb80 10 client.6516 fetch_fsmap learned FSMap version 48 2023-10-08T00:15:47.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.001+0000 7f59e11ebb80 1 -- 172.21.15.99:0/1243710347 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x560ba7428470 con 0x560ba7522a50 2023-10-08T00:15:47.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.001+0000 7f59affff700 1 -- 172.21.15.99:0/1243710347 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59cc05a4f0 con 0x560ba7522a50 2023-10-08T00:15:47.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.001+0000 7f59affff700 1 -- 172.21.15.99:0/1243710347 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 48) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f59cc059070 con 0x560ba7522a50 2023-10-08T00:15:47.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.001+0000 7f59e11ebb80 10 client.6516 fetch_fsmap finished waiting for FSMap version 48 2023-10-08T00:15:47.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.001+0000 7f59e11ebb80 10 client.6516 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-08T00:15:47.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.001+0000 7f59e11ebb80 20 client.6516 populate_metadata read hostname 'smithi099' 2023-10-08T00:15:47.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.001+0000 7f59e11ebb80 1 --2- 172.21.15.99:0/1243710347 >> [v2:172.21.15.99:6832/146609901,v1:172.21.15.99:6834/146609901] conn(0x560ba7547ee0 0x560ba75482b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:47.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.001+0000 7f59e11ebb80 4 client.6516 mds_command: new command op to 6360 tid=0[{"prefix": "status"}] 2023-10-08T00:15:47.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.001+0000 7f59e11ebb80 1 -- 172.21.15.99:0/1243710347 --> [v2:172.21.15.99:6832/146609901,v1:172.21.15.99:6834/146609901] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560ba7415e00 con 0x560ba7547ee0 2023-10-08T00:15:47.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.001+0000 7f59d548e700 1 --2- 172.21.15.99:0/1243710347 >> [v2:172.21.15.99:6832/146609901,v1:172.21.15.99:6834/146609901] conn(0x560ba7547ee0 0x560ba75482b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:47.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.001+0000 7f59affff700 10 client.6516 ms_handle_connect on v2:172.21.15.99:6832/146609901 2023-10-08T00:15:47.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.001+0000 7f59d548e700 1 --2- 172.21.15.99:0/1243710347 >> [v2:172.21.15.99:6832/146609901,v1:172.21.15.99:6834/146609901] conn(0x560ba7547ee0 0x560ba75482b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc8365bfbb0f780f server_cookie=afe253c5a84d0931 in_seq=0 out_seq=0 2023-10-08T00:15:47.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.001+0000 7f59affff700 1 -- 172.21.15.99:0/1243710347 <== mds.0 v2:172.21.15.99:6832/146609901 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x560ba7415e00 con 0x560ba7547ee0 2023-10-08T00:15:47.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.001+0000 7f59affff700 10 client.6516 handle_command_reply: tid=0 2023-10-08T00:15:47.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.001+0000 7f59e11ebb80 1 client.6516 shutdown 2023-10-08T00:15:47.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.002+0000 7f59e11ebb80 1 -- 172.21.15.99:0/1243710347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ba7522a50 msgr2=0x560ba7550590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:47.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.002+0000 7f59e11ebb80 1 --2- 172.21.15.99:0/1243710347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ba7522a50 0x560ba7550590 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f59cc084970 tx=0x7f59cc06f040 comp rx=0 tx=0).stop 2023-10-08T00:15:47.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.002+0000 7f59e11ebb80 1 -- 172.21.15.99:0/1243710347 shutdown_connections 2023-10-08T00:15:47.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.002+0000 7f59e11ebb80 1 --2- 172.21.15.99:0/1243710347 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560ba756cd00 0x560ba75529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:47.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.002+0000 7f59e11ebb80 1 --2- 172.21.15.99:0/1243710347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ba7522a50 0x560ba7550590 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:47.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.002+0000 7f59e11ebb80 1 --2- 172.21.15.99:0/1243710347 >> [v2:172.21.15.99:6832/146609901,v1:172.21.15.99:6834/146609901] conn(0x560ba7547ee0 0x560ba75482b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:47.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.002+0000 7f59e11ebb80 1 --2- 172.21.15.99:0/1243710347 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560ba754bf60 0x560ba7560c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:47.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.002+0000 7f59affff700 0 client.6516 ms_handle_reset on v2:172.21.15.99:6832/146609901 2023-10-08T00:15:47.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.002+0000 7f59e11ebb80 1 -- 172.21.15.99:0/1243710347 >> 172.21.15.99:0/1243710347 conn(0x560ba7524370 msgr2=0x560ba751f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:47.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.002+0000 7f59e11ebb80 1 -- 172.21.15.99:0/1243710347 shutdown_connections 2023-10-08T00:15:47.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.002+0000 7f59e11ebb80 1 -- 172.21.15.99:0/1243710347 wait complete. 2023-10-08T00:15:47.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.002+0000 7f59e11ebb80 20 client.6516 trim_cache size 0 max 16384 2023-10-08T00:15:47.011 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:15:47.012 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", 2023-10-08T00:15:47.012 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 0, 2023-10-08T00:15:47.012 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 6360, 2023-10-08T00:15:47.012 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2023-10-08T00:15:47.012 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2023-10-08T00:15:47.012 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2023-10-08T00:15:47.012 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 5.7788561290000002, 2023-10-08T00:15:47.012 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 48, 2023-10-08T00:15:47.012 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 104, 2023-10-08T00:15:47.012 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 104, 2023-10-08T00:15:47.012 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 11.754627995 2023-10-08T00:15:47.012 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:15:47.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.002+0000 7f59d7ef3700 1 -- 172.21.15.99:0/3425426865 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f59b4066f30 msgr2=0x7f59b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:47.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.002+0000 7f59d7ef3700 1 --2- 172.21.15.99:0/3425426865 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f59b4066f30 0x7f59b40693e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f59c0006f00 tx=0x7f59c0008040 comp rx=0 tx=0).stop 2023-10-08T00:15:47.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.002+0000 7f59d7ef3700 1 -- 172.21.15.99:0/3425426865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59d0157870 msgr2=0x7f59d016e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:47.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.002+0000 7f59d7ef3700 1 --2- 172.21.15.99:0/3425426865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59d0157870 0x7f59d016e130 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f59cc009290 tx=0x7f59cc017000 comp rx=0 tx=0).stop 2023-10-08T00:15:47.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.002+0000 7f59d7ef3700 1 -- 172.21.15.99:0/3425426865 shutdown_connections 2023-10-08T00:15:47.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.002+0000 7f59d7ef3700 1 --2- 172.21.15.99:0/3425426865 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f59b4066f30 0x7f59b40693e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:47.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.002+0000 7f59d7ef3700 1 --2- 172.21.15.99:0/3425426865 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f59d015d580 0x7f59d00d01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:47.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.002+0000 7f59d7ef3700 1 --2- 172.21.15.99:0/3425426865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59d0157870 0x7f59d016e130 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:47.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.002+0000 7f59d7ef3700 1 --2- 172.21.15.99:0/3425426865 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59d015a620 0x7f59d00cfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:47.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.002+0000 7f59d7ef3700 1 -- 172.21.15.99:0/3425426865 >> 172.21.15.99:0/3425426865 conn(0x7f59d0068ce0 msgr2=0x7f59d015da10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:47.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.002+0000 7f59d7ef3700 1 -- 172.21.15.99:0/3425426865 shutdown_connections 2023-10-08T00:15:47.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.002+0000 7f59d7ef3700 1 -- 172.21.15.99:0/3425426865 wait complete. 2023-10-08T00:15:47.015 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T00:15:47.129 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.128+0000 7fa63e3bf700 1 Processor -- start 2023-10-08T00:15:47.129 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.128+0000 7fa63e3bf700 1 -- start start 2023-10-08T00:15:47.132 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.129+0000 7fa63e3bf700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa638152010 0x7fa63815b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:47.132 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.129+0000 7fa63e3bf700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6381523e0 0x7fa63806c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:47.132 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.129+0000 7fa63e3bf700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa638158e50 0x7fa6380cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:47.133 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.129+0000 7fa63e3bf700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa63806c010 con 0x7fa638158e50 2023-10-08T00:15:47.133 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.129+0000 7fa63e3bf700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa63806bd10 con 0x7fa638152010 2023-10-08T00:15:47.133 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.129+0000 7fa63e3bf700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa63815c060 con 0x7fa6381523e0 2023-10-08T00:15:47.133 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.129+0000 7fa63c95c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa638158e50 0x7fa6380cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:47.133 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.129+0000 7fa63c95c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa638158e50 0x7fa6380cdf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41032/0 (socket says 172.21.15.99:41032) 2023-10-08T00:15:47.133 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.129+0000 7fa63c95c700 1 -- 172.21.15.99:0/1084139112 learned_addr learned my addr 172.21.15.99:0/1084139112 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:47.133 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.130+0000 7fa6377fe700 1 --2- 172.21.15.99:0/1084139112 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6381523e0 0x7fa63806c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:47.133 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.130+0000 7fa637fff700 1 --2- 172.21.15.99:0/1084139112 >> v2:172.21.15.191:3300/0 conn(0x7fa638152010 0x7fa63815b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:47.133 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.130+0000 7fa63c95c700 1 -- 172.21.15.99:0/1084139112 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6381523e0 msgr2=0x7fa63806c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:47.133 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.130+0000 7fa63c95c700 1 --2- 172.21.15.99:0/1084139112 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6381523e0 0x7fa63806c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:47.133 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.130+0000 7fa63c95c700 1 -- 172.21.15.99:0/1084139112 >> v2:172.21.15.191:3300/0 conn(0x7fa638152010 msgr2=0x7fa63815b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:47.133 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.130+0000 7fa63c95c700 1 --2- 172.21.15.99:0/1084139112 >> v2:172.21.15.191:3300/0 conn(0x7fa638152010 0x7fa63815b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:47.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.130+0000 7fa63c95c700 1 -- 172.21.15.99:0/1084139112 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa63815c340 con 0x7fa638158e50 2023-10-08T00:15:47.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.130+0000 7fa6377fe700 1 --2- 172.21.15.99:0/1084139112 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6381523e0 0x7fa63806c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:15:47.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.130+0000 7fa637fff700 1 --2- 172.21.15.99:0/1084139112 >> v2:172.21.15.191:3300/0 conn(0x7fa638152010 0x7fa63815b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:15:47.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.130+0000 7fa63c95c700 1 --2- 172.21.15.99:0/1084139112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa638158e50 0x7fa6380cdf30 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fa6380cc680 tx=0x7fa630009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ab30fe738e720fe server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:47.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.130+0000 7fa636ffd700 1 -- 172.21.15.99:0/1084139112 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63000e070 con 0x7fa638158e50 2023-10-08T00:15:47.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.130+0000 7fa636ffd700 1 -- 172.21.15.99:0/1084139112 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa63000ba50 con 0x7fa638158e50 2023-10-08T00:15:47.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.130+0000 7fa636ffd700 1 -- 172.21.15.99:0/1084139112 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6300092e0 con 0x7fa638158e50 2023-10-08T00:15:47.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.130+0000 7fa63e3bf700 1 -- 172.21.15.99:0/1084139112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa638158e50 msgr2=0x7fa6380cdf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:47.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.130+0000 7fa63e3bf700 1 --2- 172.21.15.99:0/1084139112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa638158e50 0x7fa6380cdf30 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fa6380cc680 tx=0x7fa630009840 comp rx=0 tx=0).stop 2023-10-08T00:15:47.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.131+0000 7fa63e3bf700 1 -- 172.21.15.99:0/1084139112 shutdown_connections 2023-10-08T00:15:47.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.131+0000 7fa63e3bf700 1 --2- 172.21.15.99:0/1084139112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa638158e50 0x7fa6380cdf30 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:47.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.131+0000 7fa63e3bf700 1 --2- 172.21.15.99:0/1084139112 >> v2:172.21.15.191:3300/0 conn(0x7fa638152010 0x7fa63815b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:47.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.131+0000 7fa63e3bf700 1 --2- 172.21.15.99:0/1084139112 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6381523e0 0x7fa63806c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:47.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.131+0000 7fa63e3bf700 1 -- 172.21.15.99:0/1084139112 >> 172.21.15.99:0/1084139112 conn(0x7fa638069870 msgr2=0x7fa638069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:47.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.131+0000 7fa63e3bf700 1 -- 172.21.15.99:0/1084139112 shutdown_connections 2023-10-08T00:15:47.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.131+0000 7fa63e3bf700 1 -- 172.21.15.99:0/1084139112 wait complete. 2023-10-08T00:15:47.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.132+0000 7fa63e3bf700 1 Processor -- start 2023-10-08T00:15:47.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.132+0000 7fa63e3bf700 1 -- start start 2023-10-08T00:15:47.137 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.132+0000 7fa63e3bf700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa638152010 0x7fa6380c42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:47.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.132+0000 7fa63e3bf700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6381523e0 0x7fa638167c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:47.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.133+0000 7fa63e3bf700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa638158e50 0x7fa63814e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:47.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.133+0000 7fa63e3bf700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6380c47e0 con 0x7fa638158e50 2023-10-08T00:15:47.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.133+0000 7fa63e3bf700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa638168160 con 0x7fa6381523e0 2023-10-08T00:15:47.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.133+0000 7fa63e3bf700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa638168c20 con 0x7fa638152010 2023-10-08T00:15:47.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.133+0000 7fa63c95c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa638158e50 0x7fa63814e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:47.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.133+0000 7fa63c95c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa638158e50 0x7fa63814e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41044/0 (socket says 172.21.15.99:41044) 2023-10-08T00:15:47.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.133+0000 7fa63c95c700 1 -- 172.21.15.99:0/3779960450 learned_addr learned my addr 172.21.15.99:0/3779960450 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:47.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.133+0000 7fa6377fe700 1 --2- 172.21.15.99:0/3779960450 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6381523e0 0x7fa638167c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:47.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.133+0000 7fa637fff700 1 --2- 172.21.15.99:0/3779960450 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa638152010 0x7fa6380c42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:47.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.133+0000 7fa63c95c700 1 -- 172.21.15.99:0/3779960450 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa638152010 msgr2=0x7fa6380c42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:47.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.133+0000 7fa63c95c700 1 --2- 172.21.15.99:0/3779960450 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa638152010 0x7fa6380c42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:47.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.133+0000 7fa63c95c700 1 -- 172.21.15.99:0/3779960450 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6381523e0 msgr2=0x7fa638167c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:47.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.133+0000 7fa63c95c700 1 --2- 172.21.15.99:0/3779960450 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6381523e0 0x7fa638167c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:47.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.133+0000 7fa63c95c700 1 -- 172.21.15.99:0/3779960450 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa630009ae0 con 0x7fa638158e50 2023-10-08T00:15:47.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.133+0000 7fa63c95c700 1 --2- 172.21.15.99:0/3779960450 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa638158e50 0x7fa63814e770 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fa630009fd0 tx=0x7fa630018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:47.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.133+0000 7fa61ffff700 1 -- 172.21.15.99:0/3779960450 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6300094e0 con 0x7fa638158e50 2023-10-08T00:15:47.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.133+0000 7fa61ffff700 1 -- 172.21.15.99:0/3779960450 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6300032f0 con 0x7fa638158e50 2023-10-08T00:15:47.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.133+0000 7fa63e3bf700 1 -- 172.21.15.99:0/3779960450 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6380b2cd0 con 0x7fa638158e50 2023-10-08T00:15:47.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.133+0000 7fa61ffff700 1 -- 172.21.15.99:0/3779960450 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6300256b0 con 0x7fa638158e50 2023-10-08T00:15:47.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.133+0000 7fa63e3bf700 1 -- 172.21.15.99:0/3779960450 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa638168940 con 0x7fa638158e50 2023-10-08T00:15:47.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.134+0000 7fa63e3bf700 1 -- 172.21.15.99:0/3779960450 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa63804c3c0 con 0x7fa638158e50 2023-10-08T00:15:47.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.134+0000 7fa61ffff700 1 -- 172.21.15.99:0/3779960450 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa630003040 con 0x7fa638158e50 2023-10-08T00:15:47.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.135+0000 7fa61ffff700 1 --2- 172.21.15.99:0/3779960450 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa624066f30 0x7fa6240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:47.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.135+0000 7fa61ffff700 1 -- 172.21.15.99:0/3779960450 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa630086300 con 0x7fa638158e50 2023-10-08T00:15:47.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.135+0000 7fa637fff700 1 --2- 172.21.15.99:0/3779960450 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa624066f30 0x7fa6240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:47.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.136+0000 7fa637fff700 1 --2- 172.21.15.99:0/3779960450 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa624066f30 0x7fa6240693e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa62800ab30 tx=0x7fa628008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:47.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.139+0000 7fa61ffff700 1 -- 172.21.15.99:0/3779960450 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa630053620 con 0x7fa638158e50 2023-10-08T00:15:47.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.366+0000 7fa63e3bf700 1 -- 172.21.15.99:0/3779960450 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa6380b2e60 con 0x7fa638158e50 2023-10-08T00:15:47.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.366+0000 7fa61ffff700 1 -- 172.21.15.99:0/3779960450 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v48) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa630056c40 con 0x7fa638158e50 2023-10-08T00:15:47.367 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T00:15:47.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.368+0000 7fa63e3bf700 1 -- 172.21.15.99:0/3779960450 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa624066f30 msgr2=0x7fa6240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:47.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.368+0000 7fa63e3bf700 1 --2- 172.21.15.99:0/3779960450 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa624066f30 0x7fa6240693e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa62800ab30 tx=0x7fa628008040 comp rx=0 tx=0).stop 2023-10-08T00:15:47.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.368+0000 7fa63e3bf700 1 -- 172.21.15.99:0/3779960450 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa638158e50 msgr2=0x7fa63814e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:47.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.368+0000 7fa63e3bf700 1 --2- 172.21.15.99:0/3779960450 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa638158e50 0x7fa63814e770 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fa630009fd0 tx=0x7fa630018000 comp rx=0 tx=0).stop 2023-10-08T00:15:47.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.368+0000 7fa63e3bf700 1 -- 172.21.15.99:0/3779960450 shutdown_connections 2023-10-08T00:15:47.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.368+0000 7fa63e3bf700 1 --2- 172.21.15.99:0/3779960450 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa624066f30 0x7fa6240693e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:47.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.368+0000 7fa63e3bf700 1 --2- 172.21.15.99:0/3779960450 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa638158e50 0x7fa63814e770 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:47.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.368+0000 7fa63e3bf700 1 --2- 172.21.15.99:0/3779960450 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa638152010 0x7fa6380c42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:47.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.368+0000 7fa63e3bf700 1 --2- 172.21.15.99:0/3779960450 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6381523e0 0x7fa638167c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:47.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.368+0000 7fa63e3bf700 1 -- 172.21.15.99:0/3779960450 >> 172.21.15.99:0/3779960450 conn(0x7fa638069870 msgr2=0x7fa63814f160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:47.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.369+0000 7fa63e3bf700 1 -- 172.21.15.99:0/3779960450 shutdown_connections 2023-10-08T00:15:47.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:47.369+0000 7fa63e3bf700 1 -- 172.21.15.99:0/3779960450 wait complete. 2023-10-08T00:15:47.381 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T00:15:47.381 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T00:15:47.381 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi099.front.sepia.ceph.com 2023-10-08T00:15:47.381 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T00:15:47.381 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T00:15:47.382 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T00:15:47.382 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T00:15:47.382 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T00:15:47.382 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T00:15:47.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:15:47.382 DEBUG:teuthology.orchestra.run.smithi099:> ip addr 2023-10-08T00:15:47.397 INFO:teuthology.orchestra.run.smithi099.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T00:15:47.397 INFO:teuthology.orchestra.run.smithi099.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T00:15:47.397 INFO:teuthology.orchestra.run.smithi099.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T00:15:47.397 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:15:47.397 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 ::1/128 scope host 2023-10-08T00:15:47.398 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:15:47.398 INFO:teuthology.orchestra.run.smithi099.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:15:47.398 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:10 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:15:47.398 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f0 2023-10-08T00:15:47.398 INFO:teuthology.orchestra.run.smithi099.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:15:47.398 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:11 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:15:47.398 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f1 2023-10-08T00:15:47.398 INFO:teuthology.orchestra.run.smithi099.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:15:47.398 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:02 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:15:47.398 INFO:teuthology.orchestra.run.smithi099.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T00:15:47.398 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:03 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:15:47.399 INFO:teuthology.orchestra.run.smithi099.stdout: inet 172.21.15.99/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T00:15:47.399 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft 42242sec preferred_lft 42242sec 2023-10-08T00:15:47.399 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::ec4:7aff:fe8f:d803/64 scope link 2023-10-08T00:15:47.399 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:15:47.399 INFO:teuthology.orchestra.run.smithi099.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T00:15:47.399 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 22:2c:1c:f6:42:e1 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:15:47.399 INFO:teuthology.orchestra.run.smithi099.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T00:15:47.399 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:15:47.399 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::600d:26ff:fede:5eec/64 scope link 2023-10-08T00:15:47.399 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:15:47.399 INFO:teuthology.orchestra.run.smithi099.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T00:15:47.399 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether ae:2f:b5:94:69:ba brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T00:15:47.400 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::ac2f:b5ff:fe94:69ba/64 scope link 2023-10-08T00:15:47.400 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:15:47.400 INFO:teuthology.orchestra.run.smithi099.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T00:15:47.400 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 22:2c:1c:f6:42:e1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T00:15:47.400 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::202c:1cff:fef6:42e1/64 scope link 2023-10-08T00:15:47.400 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:15:47.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:15:47.400 DEBUG:teuthology.orchestra.run.smithi099:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T00:15:47.460 INFO:teuthology.orchestra.run.smithi099.stdout:1 2023-10-08T00:15:47.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:15:47.462 DEBUG:teuthology.orchestra.run.smithi099:> route 2023-10-08T00:15:47.517 INFO:teuthology.orchestra.run.smithi099.stdout:Kernel IP routing table 2023-10-08T00:15:47.517 INFO:teuthology.orchestra.run.smithi099.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T00:15:47.517 INFO:teuthology.orchestra.run.smithi099.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T00:15:47.517 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T00:15:47.518 INFO:teuthology.orchestra.run.smithi099.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T00:15:47.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:15:47.518 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2023-10-08T00:15:47.518 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2023-10-08T00:15:47.518 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T00:15:47.518 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T00:15:47.518 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T00:15:47.518 DEBUG:teuthology.orchestra.run.smithi099:> ') 2023-10-08T00:15:47.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:15:47.617 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2023-10-08T00:15:47.631 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T00:15:47.632 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T00:15:47.632 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T00:15:47.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:15:47.632 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set brx.0 up 2023-10-08T00:15:47.696 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:15:47.696 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T00:15:47.750 INFO:teuthology.orchestra.run.smithi099.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T00:15:47.751 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:15:47.751 DEBUG:teuthology.orchestra.run.smithi099:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T00:15:47.804 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-08T00:15:47.805 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T00:15:47.805 DEBUG:teuthology.orchestra.run.smithi099:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:15:47.885 INFO:teuthology.orchestra.run.smithi099.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:15:47.885 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:15:47.904 INFO:teuthology.orchestra.run.smithi099.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:15:47.904 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:15:47.904 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: resolved to: "172.21.15.99:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T00:15:47.904 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: trying mount with new device syntax: 0@06962cdb-9a6f-417f-82fd-9e4e3dedbccf.cephfs=/ 2023-10-08T00:15:47.904 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.99:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T00:15:47.904 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T00:15:47.904 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T00:15:47.904 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T00:15:47.926 INFO:teuthology.orchestra.run.smithi099.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:15:47.926 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:15:47.926 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: resolved to: "172.21.15.99:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T00:15:47.926 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: trying mount with new device syntax: 0@06962cdb-9a6f-417f-82fd-9e4e3dedbccf.cephfs=/ 2023-10-08T00:15:47.926 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.99:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T00:15:47.926 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T00:15:47.926 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T00:15:47.927 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T00:15:47.927 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: resolved to: "172.21.15.99:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T00:15:47.927 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.99:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T00:15:47.927 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=06962cdb-9a6f-417f-82fd-9e4e3dedbccf" will pass to kernel 2023-10-08T00:15:47.928 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T00:15:47.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:15:47.929 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T00:15:47.993 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T00:15:48.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.147+0000 7efc16de7700 1 Processor -- start 2023-10-08T00:15:48.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.147+0000 7efc16de7700 1 -- start start 2023-10-08T00:15:48.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.148+0000 7efc16de7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc100cdab0 0x7efc1015d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:48.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.148+0000 7efc16de7700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7efc1015dda0 0x7efc1015cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:48.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.148+0000 7efc16de7700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc100cb7c0 con 0x7efc100cd6e0 2023-10-08T00:15:48.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.148+0000 7efc16de7700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc1015e850 con 0x7efc1015dda0 2023-10-08T00:15:48.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.148+0000 7efc16de7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc100cc360 con 0x7efc100cdab0 2023-10-08T00:15:48.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.148+0000 7efc165e6700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7efc1015dda0 0x7efc1015cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:48.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.148+0000 7efc165e6700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7efc1015dda0 0x7efc1015cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41052/0 (socket says 172.21.15.99:41052) 2023-10-08T00:15:48.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.148+0000 7efc165e6700 1 -- 172.21.15.99:0/3716415446 learned_addr learned my addr 172.21.15.99:0/3716415446 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:48.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.148+0000 7efc155e4700 1 --2- 172.21.15.99:0/3716415446 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc100cdab0 0x7efc1015d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:48.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.148+0000 7efc165e6700 1 -- 172.21.15.99:0/3716415446 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc100cdab0 msgr2=0x7efc1015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:48.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.148+0000 7efc165e6700 1 --2- 172.21.15.99:0/3716415446 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc100cdab0 0x7efc1015d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:48.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.148+0000 7efc165e6700 1 -- 172.21.15.99:0/3716415446 >> v1:172.21.15.191:6789/0 conn(0x7efc100cd6e0 legacy=0x7efc100cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:48.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.148+0000 7efc165e6700 1 -- 172.21.15.99:0/3716415446 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc1015e290 con 0x7efc1015dda0 2023-10-08T00:15:48.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.149+0000 7efc165e6700 1 --2- 172.21.15.99:0/3716415446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc1015dda0 0x7efc1015cbd0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7efc08009f20 tx=0x7efc08009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62b740e34f2e981e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:48.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.149+0000 7efc14de3700 1 -- 172.21.15.99:0/3716415446 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0800e070 con 0x7efc1015dda0 2023-10-08T00:15:48.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.149+0000 7efc14de3700 1 -- 172.21.15.99:0/3716415446 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc0800ba80 con 0x7efc1015dda0 2023-10-08T00:15:48.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.149+0000 7efc14de3700 1 -- 172.21.15.99:0/3716415446 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc080092e0 con 0x7efc1015dda0 2023-10-08T00:15:48.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.149+0000 7efc16de7700 1 -- 172.21.15.99:0/3716415446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc1015dda0 msgr2=0x7efc1015cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:48.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.149+0000 7efc16de7700 1 --2- 172.21.15.99:0/3716415446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc1015dda0 0x7efc1015cbd0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7efc08009f20 tx=0x7efc08009840 comp rx=0 tx=0).stop 2023-10-08T00:15:48.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.149+0000 7efc16de7700 1 -- 172.21.15.99:0/3716415446 shutdown_connections 2023-10-08T00:15:48.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.149+0000 7efc16de7700 1 --2- 172.21.15.99:0/3716415446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc1015dda0 0x7efc1015cbd0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:48.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.149+0000 7efc16de7700 1 --2- 172.21.15.99:0/3716415446 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc100cdab0 0x7efc1015d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:48.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.149+0000 7efc16de7700 1 -- 172.21.15.99:0/3716415446 >> 172.21.15.99:0/3716415446 conn(0x7efc10067660 msgr2=0x7efc10067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:48.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.149+0000 7efc16de7700 1 -- 172.21.15.99:0/3716415446 shutdown_connections 2023-10-08T00:15:48.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.149+0000 7efc16de7700 1 -- 172.21.15.99:0/3716415446 wait complete. 2023-10-08T00:15:48.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.150+0000 7efc16de7700 1 Processor -- start 2023-10-08T00:15:48.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.150+0000 7efc16de7700 1 -- start start 2023-10-08T00:15:48.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.151+0000 7efc16de7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc100cd6e0 0x7efc100c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:48.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.151+0000 7efc16de7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc100cdab0 0x7efc100b23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:48.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.151+0000 7efc16de7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efc1015dda0 0x7efc100b5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:48.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.151+0000 7efc16de7700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc1016d7b0 con 0x7efc100cd6e0 2023-10-08T00:15:48.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.151+0000 7efc16de7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc1016e610 con 0x7efc1015dda0 2023-10-08T00:15:48.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.151+0000 7efc16de7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc1016da00 con 0x7efc100cdab0 2023-10-08T00:15:48.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.151+0000 7efc15de5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc100cd6e0 0x7efc100c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:48.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.151+0000 7efc15de5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc100cd6e0 0x7efc100c7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41066/0 (socket says 172.21.15.99:41066) 2023-10-08T00:15:48.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.151+0000 7efc15de5700 1 -- 172.21.15.99:0/3431559532 learned_addr learned my addr 172.21.15.99:0/3431559532 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:48.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.151+0000 7efc165e6700 1 --2- 172.21.15.99:0/3431559532 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efc1015dda0 0x7efc100b5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:48.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.151+0000 7efc155e4700 1 --2- 172.21.15.99:0/3431559532 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc100cdab0 0x7efc100b23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:48.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.151+0000 7efc15de5700 1 -- 172.21.15.99:0/3431559532 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc100cdab0 msgr2=0x7efc100b23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:48.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.151+0000 7efc15de5700 1 --2- 172.21.15.99:0/3431559532 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc100cdab0 0x7efc100b23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:48.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.151+0000 7efc15de5700 1 -- 172.21.15.99:0/3431559532 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efc1015dda0 msgr2=0x7efc100b5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:48.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.151+0000 7efc15de5700 1 --2- 172.21.15.99:0/3431559532 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efc1015dda0 0x7efc100b5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:48.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.151+0000 7efc15de5700 1 -- 172.21.15.99:0/3431559532 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc08009ae0 con 0x7efc100cd6e0 2023-10-08T00:15:48.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.151+0000 7efc165e6700 1 --2- 172.21.15.99:0/3431559532 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efc1015dda0 0x7efc100b5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:15:48.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.151+0000 7efc155e4700 1 --2- 172.21.15.99:0/3431559532 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc100cdab0 0x7efc100b23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:15:48.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.152+0000 7efc15de5700 1 --2- 172.21.15.99:0/3431559532 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc100cd6e0 0x7efc100c7d20 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7efc0c00c790 tx=0x7efc0c007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:48.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.152+0000 7efc01ffb700 1 -- 172.21.15.99:0/3431559532 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0c011590 con 0x7efc100cd6e0 2023-10-08T00:15:48.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.152+0000 7efc01ffb700 1 -- 172.21.15.99:0/3431559532 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc0c011710 con 0x7efc100cd6e0 2023-10-08T00:15:48.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.152+0000 7efc16de7700 1 -- 172.21.15.99:0/3431559532 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc1016dce0 con 0x7efc100cd6e0 2023-10-08T00:15:48.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.152+0000 7efc01ffb700 1 -- 172.21.15.99:0/3431559532 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0c011ad0 con 0x7efc100cd6e0 2023-10-08T00:15:48.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.152+0000 7efc16de7700 1 -- 172.21.15.99:0/3431559532 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc1016e2a0 con 0x7efc100cd6e0 2023-10-08T00:15:48.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.153+0000 7efc16de7700 1 -- 172.21.15.99:0/3431559532 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc10089280 con 0x7efc100cd6e0 2023-10-08T00:15:48.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.153+0000 7efc01ffb700 1 -- 172.21.15.99:0/3431559532 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efc0c01b3b0 con 0x7efc100cd6e0 2023-10-08T00:15:48.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.153+0000 7efc01ffb700 1 --2- 172.21.15.99:0/3431559532 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efbf4066f30 0x7efbf40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:48.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.154+0000 7efc01ffb700 1 -- 172.21.15.99:0/3431559532 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5705+0+0 (secure 0 0 0) 0x7efc0c083a50 con 0x7efc100cd6e0 2023-10-08T00:15:48.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.154+0000 7efc155e4700 1 --2- 172.21.15.99:0/3431559532 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efbf4066f30 0x7efbf40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:48.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.154+0000 7efc155e4700 1 --2- 172.21.15.99:0/3431559532 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efbf4066f30 0x7efbf40693e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7efc04006fd0 tx=0x7efc04008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:48.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.158+0000 7efc01ffb700 1 -- 172.21.15.99:0/3431559532 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efc0c050cc0 con 0x7efc100cd6e0 2023-10-08T00:15:48.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.385+0000 7efc16de7700 1 -- 172.21.15.99:0/3431559532 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7efc100cabc0 con 0x7efc100cd6e0 2023-10-08T00:15:48.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.385+0000 7efc01ffb700 1 -- 172.21.15.99:0/3431559532 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v48) v1 ==== 53+0+73 (secure 0 0 0) 0x7efc0c017070 con 0x7efc100cd6e0 2023-10-08T00:15:48.386 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T00:15:48.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.388+0000 7efc16de7700 1 -- 172.21.15.99:0/3431559532 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efbf4066f30 msgr2=0x7efbf40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:48.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.388+0000 7efc16de7700 1 --2- 172.21.15.99:0/3431559532 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efbf4066f30 0x7efbf40693e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7efc04006fd0 tx=0x7efc04008040 comp rx=0 tx=0).stop 2023-10-08T00:15:48.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.388+0000 7efc16de7700 1 -- 172.21.15.99:0/3431559532 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc100cd6e0 msgr2=0x7efc100c7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:48.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.388+0000 7efc16de7700 1 --2- 172.21.15.99:0/3431559532 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc100cd6e0 0x7efc100c7d20 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7efc0c00c790 tx=0x7efc0c007f40 comp rx=0 tx=0).stop 2023-10-08T00:15:48.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.388+0000 7efc16de7700 1 -- 172.21.15.99:0/3431559532 shutdown_connections 2023-10-08T00:15:48.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.388+0000 7efc16de7700 1 --2- 172.21.15.99:0/3431559532 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efbf4066f30 0x7efbf40693e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:48.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.388+0000 7efc16de7700 1 --2- 172.21.15.99:0/3431559532 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efc1015dda0 0x7efc100b5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:48.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.388+0000 7efc16de7700 1 --2- 172.21.15.99:0/3431559532 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc100cd6e0 0x7efc100c7d20 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:48.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.388+0000 7efc16de7700 1 --2- 172.21.15.99:0/3431559532 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc100cdab0 0x7efc100b23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:48.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.388+0000 7efc16de7700 1 -- 172.21.15.99:0/3431559532 >> 172.21.15.99:0/3431559532 conn(0x7efc10067660 msgr2=0x7efc100d1060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:48.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.388+0000 7efc16de7700 1 -- 172.21.15.99:0/3431559532 shutdown_connections 2023-10-08T00:15:48.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:48.388+0000 7efc16de7700 1 -- 172.21.15.99:0/3431559532 wait complete. 2023-10-08T00:15:48.400 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T00:15:48.400 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T00:15:48.400 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi191.front.sepia.ceph.com 2023-10-08T00:15:48.400 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-08T00:15:48.400 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-08T00:15:48.401 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T00:15:48.401 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T00:15:48.401 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T00:15:48.401 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T00:15:48.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:15:48.401 DEBUG:teuthology.orchestra.run.smithi191:> ip addr 2023-10-08T00:15:48.421 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T00:15:48.421 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T00:15:48.421 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T00:15:48.421 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:15:48.421 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2023-10-08T00:15:48.422 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:15:48.422 INFO:teuthology.orchestra.run.smithi191.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:15:48.422 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f6 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:15:48.422 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f0 2023-10-08T00:15:48.423 INFO:teuthology.orchestra.run.smithi191.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:15:48.423 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f7 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:15:48.423 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f1 2023-10-08T00:15:48.423 INFO:teuthology.orchestra.run.smithi191.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:15:48.423 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:10 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:15:48.424 INFO:teuthology.orchestra.run.smithi191.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T00:15:48.424 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:11 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:15:48.424 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.21.15.191/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T00:15:48.424 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft 42243sec preferred_lft 42243sec 2023-10-08T00:15:48.424 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec4:7aff:fe88:7211/64 scope link 2023-10-08T00:15:48.424 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:15:48.424 INFO:teuthology.orchestra.run.smithi191.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T00:15:48.424 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 9a:bc:39:72:87:3d brd ff:ff:ff:ff:ff:ff 2023-10-08T00:15:48.424 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T00:15:48.424 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:15:48.425 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec77:a7ff:fea4:fbb/64 scope link 2023-10-08T00:15:48.425 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:15:48.425 INFO:teuthology.orchestra.run.smithi191.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T00:15:48.425 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether ea:1c:d9:46:05:b3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-08T00:15:48.425 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::e81c:d9ff:fe46:5b3/64 scope link 2023-10-08T00:15:48.425 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:15:48.425 INFO:teuthology.orchestra.run.smithi191.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T00:15:48.425 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 9a:bc:39:72:87:3d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-08T00:15:48.425 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::98bc:39ff:fe72:873d/64 scope link 2023-10-08T00:15:48.425 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:15:48.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:15:48.426 DEBUG:teuthology.orchestra.run.smithi191:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T00:15:48.489 INFO:teuthology.orchestra.run.smithi191.stdout:1 2023-10-08T00:15:48.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:15:48.491 DEBUG:teuthology.orchestra.run.smithi191:> route 2023-10-08T00:15:48.550 INFO:teuthology.orchestra.run.smithi191.stdout:Kernel IP routing table 2023-10-08T00:15:48.550 INFO:teuthology.orchestra.run.smithi191.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T00:15:48.550 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T00:15:48.550 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T00:15:48.550 INFO:teuthology.orchestra.run.smithi191.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T00:15:48.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:15:48.551 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2023-10-08T00:15:48.551 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2023-10-08T00:15:48.551 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T00:15:48.551 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T00:15:48.551 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T00:15:48.551 DEBUG:teuthology.orchestra.run.smithi191:> ') 2023-10-08T00:15:48.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:15:48.656 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2023-10-08T00:15:48.671 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-08T00:15:48.671 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-08T00:15:48.671 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T00:15:48.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:15:48.672 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set brx.0 up 2023-10-08T00:15:48.737 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:15:48.737 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-08T00:15:48.792 INFO:teuthology.orchestra.run.smithi191.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-08T00:15:48.792 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:15:48.792 DEBUG:teuthology.orchestra.run.smithi191:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-08T00:15:48.846 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-08T00:15:48.846 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T00:15:48.846 DEBUG:teuthology.orchestra.run.smithi191:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:15:48.926 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:15:48.926 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:15:48.945 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:15:48.945 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:15:48.945 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.99:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T00:15:48.945 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@06962cdb-9a6f-417f-82fd-9e4e3dedbccf.cephfs=/ 2023-10-08T00:15:48.945 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.99:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T00:15:48.945 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T00:15:48.946 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T00:15:48.946 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T00:15:48.967 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:15:48.967 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:15:48.967 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.99:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T00:15:48.968 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@06962cdb-9a6f-417f-82fd-9e4e3dedbccf.cephfs=/ 2023-10-08T00:15:48.968 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.99:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T00:15:48.968 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T00:15:48.968 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T00:15:48.968 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T00:15:48.968 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.99:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T00:15:48.969 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.99:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T00:15:48.969 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=06962cdb-9a6f-417f-82fd-9e4e3dedbccf" will pass to kernel 2023-10-08T00:15:48.971 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T00:15:48.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:15:48.971 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-08T00:15:49.035 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:15:49.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.154+0000 7f835ef8e700 1 Processor -- start 2023-10-08T00:15:49.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.154+0000 7f835ef8e700 1 -- start start 2023-10-08T00:15:49.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.155+0000 7f835ef8e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83580b1220 0x7f83580c9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:49.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.155+0000 7f835ef8e700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f83580cb4f0 0x7f83580cb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:49.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.155+0000 7f835ef8e700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f835815de60 con 0x7f83580b0e50 2023-10-08T00:15:49.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.155+0000 7f835ef8e700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83580cbed0 con 0x7f83580cb4f0 2023-10-08T00:15:49.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.155+0000 7f835ef8e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f835815e140 con 0x7f83580b1220 2023-10-08T00:15:49.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.155+0000 7f835d52b700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f83580cb4f0 0x7f83580cb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:49.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.155+0000 7f835cd2a700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f83580b0e50 0x7f835806d590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:60828/0 (socket says 172.21.15.99:60828) 2023-10-08T00:15:49.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.155+0000 7f835cd2a700 1 -- 172.21.15.99:0/1444823957 learned_addr learned my addr 172.21.15.99:0/1444823957 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:15:49.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.155+0000 7f8357fff700 1 --2- 172.21.15.99:0/1444823957 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83580b1220 0x7f83580c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:49.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.155+0000 7f835d52b700 1 -- 172.21.15.99:0/1444823957 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83580b1220 msgr2=0x7f83580c9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:49.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.155+0000 7f835d52b700 1 --2- 172.21.15.99:0/1444823957 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83580b1220 0x7f83580c9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:49.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.155+0000 7f835d52b700 1 -- 172.21.15.99:0/1444823957 >> v1:172.21.15.99:6789/0 conn(0x7f83580b0e50 legacy=0x7f835806d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:49.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.155+0000 7f835d52b700 1 -- 172.21.15.99:0/1444823957 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f835815e420 con 0x7f83580cb4f0 2023-10-08T00:15:49.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.155+0000 7f83577fe700 1 -- 172.21.15.99:0/1444823957 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 148105712 0 0) 0x7f835815de60 con 0x7f83580b0e50 2023-10-08T00:15:49.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.155+0000 7f8357fff700 1 --2- 172.21.15.99:0/1444823957 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83580b1220 0x7f83580c9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:15:49.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.155+0000 7f835d52b700 1 --2- 172.21.15.99:0/1444823957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83580cb4f0 0x7f83580cb990 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f8350009e60 tx=0x7f8350009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48bf734cff590c7a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:49.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.155+0000 7f83577fe700 1 -- 172.21.15.99:0/1444823957 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835000e070 con 0x7f83580cb4f0 2023-10-08T00:15:49.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.155+0000 7f83577fe700 1 -- 172.21.15.99:0/1444823957 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f835000bc30 con 0x7f83580cb4f0 2023-10-08T00:15:49.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.155+0000 7f83577fe700 1 -- 172.21.15.99:0/1444823957 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8350012430 con 0x7f83580cb4f0 2023-10-08T00:15:49.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.156+0000 7f835ef8e700 1 -- 172.21.15.99:0/1444823957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83580cb4f0 msgr2=0x7f83580cb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:49.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.156+0000 7f835ef8e700 1 --2- 172.21.15.99:0/1444823957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83580cb4f0 0x7f83580cb990 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f8350009e60 tx=0x7f8350009780 comp rx=0 tx=0).stop 2023-10-08T00:15:49.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.156+0000 7f835ef8e700 1 -- 172.21.15.99:0/1444823957 shutdown_connections 2023-10-08T00:15:49.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.156+0000 7f835ef8e700 1 --2- 172.21.15.99:0/1444823957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83580cb4f0 0x7f83580cb990 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:49.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.156+0000 7f835ef8e700 1 --2- 172.21.15.99:0/1444823957 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83580b1220 0x7f83580c9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:49.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.156+0000 7f835ef8e700 1 -- 172.21.15.99:0/1444823957 >> 172.21.15.99:0/1444823957 conn(0x7f8358069870 msgr2=0x7f8358069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:49.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.156+0000 7f835ef8e700 1 -- 172.21.15.99:0/1444823957 shutdown_connections 2023-10-08T00:15:49.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.156+0000 7f835ef8e700 1 -- 172.21.15.99:0/1444823957 wait complete. 2023-10-08T00:15:49.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.157+0000 7f835ef8e700 1 Processor -- start 2023-10-08T00:15:49.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.157+0000 7f835ef8e700 1 -- start start 2023-10-08T00:15:49.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.158+0000 7f835ef8e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83580b0e50 0x7f835815d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:49.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.158+0000 7f835ef8e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83580b1220 0x7f83580b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:49.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.158+0000 7f835ef8e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83580cb4f0 0x7f835816cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:49.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.158+0000 7f835ef8e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f835815da60 con 0x7f83580b1220 2023-10-08T00:15:49.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.158+0000 7f835ef8e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83580b3360 con 0x7f83580cb4f0 2023-10-08T00:15:49.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.158+0000 7f835ef8e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83580b2750 con 0x7f83580b0e50 2023-10-08T00:15:49.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.158+0000 7f8357fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83580b1220 0x7f83580b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:49.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.158+0000 7f8357fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83580b1220 0x7f83580b18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41080/0 (socket says 172.21.15.99:41080) 2023-10-08T00:15:49.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.158+0000 7f8357fff700 1 -- 172.21.15.99:0/2073638834 learned_addr learned my addr 172.21.15.99:0/2073638834 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:49.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.158+0000 7f835cd2a700 1 --2- 172.21.15.99:0/2073638834 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83580b0e50 0x7f835815d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:49.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.158+0000 7f835d52b700 1 --2- 172.21.15.99:0/2073638834 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83580cb4f0 0x7f835816cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:49.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.158+0000 7f8357fff700 1 -- 172.21.15.99:0/2073638834 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83580b0e50 msgr2=0x7f835815d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:49.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.158+0000 7f8357fff700 1 --2- 172.21.15.99:0/2073638834 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83580b0e50 0x7f835815d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:49.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.158+0000 7f8357fff700 1 -- 172.21.15.99:0/2073638834 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83580cb4f0 msgr2=0x7f835816cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:49.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.158+0000 7f8357fff700 1 --2- 172.21.15.99:0/2073638834 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83580cb4f0 0x7f835816cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:49.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.158+0000 7f8357fff700 1 -- 172.21.15.99:0/2073638834 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8350009a20 con 0x7f83580b1220 2023-10-08T00:15:49.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.158+0000 7f835cd2a700 1 --2- 172.21.15.99:0/2073638834 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83580b0e50 0x7f835815d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:15:49.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.158+0000 7f835d52b700 1 --2- 172.21.15.99:0/2073638834 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83580cb4f0 0x7f835816cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:15:49.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.159+0000 7f8357fff700 1 --2- 172.21.15.99:0/2073638834 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83580b1220 0x7f83580b18e0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f834400e8d0 tx=0x7f8344008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:49.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.159+0000 7f8354ff9700 1 -- 172.21.15.99:0/2073638834 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834400ee70 con 0x7f83580b1220 2023-10-08T00:15:49.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.159+0000 7f8354ff9700 1 -- 172.21.15.99:0/2073638834 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f834400c5e0 con 0x7f83580b1220 2023-10-08T00:15:49.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.159+0000 7f8354ff9700 1 -- 172.21.15.99:0/2073638834 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834400c980 con 0x7f83580b1220 2023-10-08T00:15:49.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.159+0000 7f835ef8e700 1 -- 172.21.15.99:0/2073638834 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83580b2a30 con 0x7f83580b1220 2023-10-08T00:15:49.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.159+0000 7f835ef8e700 1 -- 172.21.15.99:0/2073638834 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83580b2ff0 con 0x7f83580b1220 2023-10-08T00:15:49.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.160+0000 7f835ef8e700 1 -- 172.21.15.99:0/2073638834 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f835804c3c0 con 0x7f83580b1220 2023-10-08T00:15:49.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.160+0000 7f8354ff9700 1 -- 172.21.15.99:0/2073638834 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8344013410 con 0x7f83580b1220 2023-10-08T00:15:49.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.160+0000 7f8354ff9700 1 --2- 172.21.15.99:0/2073638834 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f833c066f30 0x7f833c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:49.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.161+0000 7f8354ff9700 1 -- 172.21.15.99:0/2073638834 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8344084360 con 0x7f83580b1220 2023-10-08T00:15:49.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.161+0000 7f835cd2a700 1 --2- 172.21.15.99:0/2073638834 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f833c066f30 0x7f833c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:49.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.161+0000 7f835cd2a700 1 --2- 172.21.15.99:0/2073638834 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f833c066f30 0x7f833c0693e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f834c0011e0 tx=0x7f834c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:49.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.165+0000 7f8354ff9700 1 -- 172.21.15.99:0/2073638834 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8344055060 con 0x7f83580b1220 2023-10-08T00:15:49.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.389+0000 7f835ef8e700 1 -- 172.21.15.99:0/2073638834 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f835806db10 con 0x7f83580b1220 2023-10-08T00:15:49.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.389+0000 7f8354ff9700 1 -- 172.21.15.99:0/2073638834 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4599 (secure 0 0 0) 0x7f8344054bf0 con 0x7f83580b1220 2023-10-08T00:15:49.391 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:15:49.391 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":48,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5426,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/280880696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":280880696},{"type":"v1","addr":"172.21.15.191:6836","nonce":280880696}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":5902,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3148129562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":3148129562},{"type":"v1","addr":"172.21.15.99:6835","nonce":3148129562}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":5917,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/1825600944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":1825600944},{"type":"v1","addr":"172.21.15.191:6837","nonce":1825600944}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:15:40.595166+0000","modified":"2023-10-08T00:15:44.232916+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6360},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6360":{"gid":6360,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6834/146609901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":146609901},{"type":"v1","addr":"172.21.15.99:6834","nonce":146609901}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-08T00:15:49.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.391+0000 7f835ef8e700 1 -- 172.21.15.99:0/2073638834 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f833c066f30 msgr2=0x7f833c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:49.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.391+0000 7f835ef8e700 1 --2- 172.21.15.99:0/2073638834 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f833c066f30 0x7f833c0693e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f834c0011e0 tx=0x7f834c009040 comp rx=0 tx=0).stop 2023-10-08T00:15:49.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.391+0000 7f835ef8e700 1 -- 172.21.15.99:0/2073638834 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83580b1220 msgr2=0x7f83580b18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:49.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.391+0000 7f835ef8e700 1 --2- 172.21.15.99:0/2073638834 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83580b1220 0x7f83580b18e0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f834400e8d0 tx=0x7f8344008040 comp rx=0 tx=0).stop 2023-10-08T00:15:49.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.391+0000 7f835ef8e700 1 -- 172.21.15.99:0/2073638834 shutdown_connections 2023-10-08T00:15:49.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.391+0000 7f835ef8e700 1 --2- 172.21.15.99:0/2073638834 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f833c066f30 0x7f833c0693e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:49.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.391+0000 7f835ef8e700 1 --2- 172.21.15.99:0/2073638834 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83580cb4f0 0x7f835816cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:49.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.391+0000 7f835ef8e700 1 --2- 172.21.15.99:0/2073638834 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83580b0e50 0x7f835815d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:49.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.391+0000 7f835ef8e700 1 --2- 172.21.15.99:0/2073638834 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83580b1220 0x7f83580b18e0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:49.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.391+0000 7f835ef8e700 1 -- 172.21.15.99:0/2073638834 >> 172.21.15.99:0/2073638834 conn(0x7f8358069870 msgr2=0x7f835806cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:49.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.391+0000 7f835ef8e700 1 -- 172.21.15.99:0/2073638834 shutdown_connections 2023-10-08T00:15:49.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:49.391+0000 7f835ef8e700 1 -- 172.21.15.99:0/2073638834 wait complete. 2023-10-08T00:15:49.395 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 48 2023-10-08T00:15:49.405 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-10-08T00:15:49.539 INFO:teuthology.orchestra.run.smithi099.stdout:{"cluster_fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","whoami":0,"id":6360,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":8.3038369630000002,"mdsmap_epoch":48,"osdmap_epoch":104,"osdmap_epoch_barrier":104,"uptime":14.279613980000001} 2023-10-08T00:15:49.539 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", "fs_name": "cephfs", "id": 6360, "mdsmap_epoch": 48, "osdmap_epoch": 104, "osdmap_epoch_barrier": 104, "rank_uptime": 8.303836963, "state": "up:active", "uptime": 14.27961398, "want_state": "up:active", "whoami": 0 } 2023-10-08T00:15:49.540 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-10-08T00:15:49.667 INFO:teuthology.orchestra.run.smithi099.stdout:{"osd_mon_report_interval":"5"} 2023-10-08T00:15:49.667 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-10-08T00:15:49.667 INFO:tasks.cephfs.test_full:Writing 531MB should fill this cluster 2023-10-08T00:15:49.667 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:15:49.667 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=265 seek=0) 2023-10-08T00:15:51.039 INFO:teuthology.orchestra.run.smithi099.stderr:265+0 records in 2023-10-08T00:15:51.040 INFO:teuthology.orchestra.run.smithi099.stderr:265+0 records out 2023-10-08T00:15:51.040 INFO:teuthology.orchestra.run.smithi099.stderr:277872640 bytes (278 MB, 265 MiB) copied, 1.35502 s, 205 MB/s 2023-10-08T00:15:51.041 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:15:51.041 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=292 seek=0) 2023-10-08T00:15:52.547 INFO:teuthology.orchestra.run.smithi099.stderr:292+0 records in 2023-10-08T00:15:52.547 INFO:teuthology.orchestra.run.smithi099.stderr:292+0 records out 2023-10-08T00:15:52.547 INFO:teuthology.orchestra.run.smithi099.stderr:306184192 bytes (306 MB, 292 MiB) copied, 1.49031 s, 205 MB/s 2023-10-08T00:15:52.548 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2023-10-08T00:15:52.548 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-08T00:15:52.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.661+0000 7ffa9719a700 1 Processor -- start 2023-10-08T00:15:52.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.661+0000 7ffa9719a700 1 -- start start 2023-10-08T00:15:52.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.662+0000 7ffa9719a700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ffa90157870 0x7ffa9015b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:52.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.662+0000 7ffa9719a700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ffa90157c40 0x7ffa90158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:52.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.662+0000 7ffa9719a700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa9015cbb0 con 0x7ffa9015a620 2023-10-08T00:15:52.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.662+0000 7ffa9719a700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa9015bfa0 con 0x7ffa90157870 2023-10-08T00:15:52.673 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.662+0000 7ffa9719a700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa9015c280 con 0x7ffa90157c40 2023-10-08T00:15:52.673 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.662+0000 7ffa94f36700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ffa90157870 0x7ffa9015b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:52.673 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.662+0000 7ffa87fff700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7ffa9015a620 0x7ffa9015d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:60834/0 (socket says 172.21.15.99:60834) 2023-10-08T00:15:52.673 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.662+0000 7ffa87fff700 1 -- 172.21.15.99:0/2291725936 learned_addr learned my addr 172.21.15.99:0/2291725936 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:15:52.673 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.662+0000 7ffa95737700 1 --2- 172.21.15.99:0/2291725936 >> v2:172.21.15.191:3300/0 conn(0x7ffa90157c40 0x7ffa90158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:52.673 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.662+0000 7ffa94f36700 1 -- 172.21.15.99:0/2291725936 >> v2:172.21.15.191:3300/0 conn(0x7ffa90157c40 msgr2=0x7ffa90158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:52.673 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.663+0000 7ffa94f36700 1 --2- 172.21.15.99:0/2291725936 >> v2:172.21.15.191:3300/0 conn(0x7ffa90157c40 0x7ffa90158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:52.673 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.663+0000 7ffa94f36700 1 -- 172.21.15.99:0/2291725936 >> v1:172.21.15.99:6789/0 conn(0x7ffa9015a620 legacy=0x7ffa9015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:52.673 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.663+0000 7ffa94f36700 1 -- 172.21.15.99:0/2291725936 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa9015c560 con 0x7ffa90157870 2023-10-08T00:15:52.673 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.663+0000 7ffa877fe700 1 -- 172.21.15.99:0/2291725936 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 226573167 0 0) 0x7ffa9015cbb0 con 0x7ffa9015a620 2023-10-08T00:15:52.674 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.663+0000 7ffa94f36700 1 --2- 172.21.15.99:0/2291725936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa90157870 0x7ffa9015b6a0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7ffa9006b4e0 tx=0x7ffa8c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6ee1f515adedcf4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:52.674 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.663+0000 7ffa877fe700 1 -- 172.21.15.99:0/2291725936 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c00e070 con 0x7ffa90157870 2023-10-08T00:15:52.674 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.663+0000 7ffa877fe700 1 -- 172.21.15.99:0/2291725936 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa8c00bc30 con 0x7ffa90157870 2023-10-08T00:15:52.674 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.663+0000 7ffa877fe700 1 -- 172.21.15.99:0/2291725936 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c012430 con 0x7ffa90157870 2023-10-08T00:15:52.674 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.663+0000 7ffa9719a700 1 -- 172.21.15.99:0/2291725936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa90157870 msgr2=0x7ffa9015b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:52.674 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.663+0000 7ffa9719a700 1 --2- 172.21.15.99:0/2291725936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa90157870 0x7ffa9015b6a0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7ffa9006b4e0 tx=0x7ffa8c009780 comp rx=0 tx=0).stop 2023-10-08T00:15:52.674 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.663+0000 7ffa9719a700 1 -- 172.21.15.99:0/2291725936 shutdown_connections 2023-10-08T00:15:52.674 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.663+0000 7ffa9719a700 1 --2- 172.21.15.99:0/2291725936 >> v2:172.21.15.191:3300/0 conn(0x7ffa90157c40 0x7ffa90158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:52.674 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.663+0000 7ffa9719a700 1 --2- 172.21.15.99:0/2291725936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa90157870 0x7ffa9015b6a0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:52.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.663+0000 7ffa9719a700 1 -- 172.21.15.99:0/2291725936 >> 172.21.15.99:0/2291725936 conn(0x7ffa90068ce0 msgr2=0x7ffa900690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:52.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.663+0000 7ffa9719a700 1 -- 172.21.15.99:0/2291725936 shutdown_connections 2023-10-08T00:15:52.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.663+0000 7ffa9719a700 1 -- 172.21.15.99:0/2291725936 wait complete. 2023-10-08T00:15:52.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.664+0000 7ffa9719a700 1 Processor -- start 2023-10-08T00:15:52.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.665+0000 7ffa9719a700 1 -- start start 2023-10-08T00:15:52.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.665+0000 7ffa9719a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa90157870 0x7ffa90153750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:52.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.665+0000 7ffa9719a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa90157c40 0x7ffa900b2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:52.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.665+0000 7ffa9719a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffa9015a620 0x7ffa900b1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:52.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.665+0000 7ffa9719a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9014ee40 con 0x7ffa90157870 2023-10-08T00:15:52.676 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.665+0000 7ffa9719a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9014f120 con 0x7ffa90157c40 2023-10-08T00:15:52.676 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.665+0000 7ffa9719a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9014f400 con 0x7ffa9015a620 2023-10-08T00:15:52.676 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.665+0000 7ffa94f36700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa90157870 0x7ffa90153750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:52.676 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.665+0000 7ffa94f36700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa90157870 0x7ffa90153750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41094/0 (socket says 172.21.15.99:41094) 2023-10-08T00:15:52.676 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.665+0000 7ffa94f36700 1 -- 172.21.15.99:0/3742598976 learned_addr learned my addr 172.21.15.99:0/3742598976 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:52.676 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.666+0000 7ffa87fff700 1 --2- 172.21.15.99:0/3742598976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa90157c40 0x7ffa900b2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:52.676 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.666+0000 7ffa95737700 1 --2- 172.21.15.99:0/3742598976 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffa9015a620 0x7ffa900b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:52.676 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.666+0000 7ffa94f36700 1 -- 172.21.15.99:0/3742598976 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffa9015a620 msgr2=0x7ffa900b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:52.676 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.666+0000 7ffa94f36700 1 --2- 172.21.15.99:0/3742598976 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffa9015a620 0x7ffa900b1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:52.676 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.666+0000 7ffa94f36700 1 -- 172.21.15.99:0/3742598976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa90157c40 msgr2=0x7ffa900b2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:52.677 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.666+0000 7ffa94f36700 1 --2- 172.21.15.99:0/3742598976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa90157c40 0x7ffa900b2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:52.677 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.666+0000 7ffa94f36700 1 -- 172.21.15.99:0/3742598976 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa8c009a20 con 0x7ffa90157870 2023-10-08T00:15:52.677 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.666+0000 7ffa87fff700 1 --2- 172.21.15.99:0/3742598976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa90157c40 0x7ffa900b2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:15:52.677 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.666+0000 7ffa94f36700 1 --2- 172.21.15.99:0/3742598976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa90157870 0x7ffa90153750 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c0097c0 tx=0x7ffa8c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:52.677 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.666+0000 7ffa84ff9700 1 -- 172.21.15.99:0/3742598976 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c027470 con 0x7ffa90157870 2023-10-08T00:15:52.677 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.666+0000 7ffa84ff9700 1 -- 172.21.15.99:0/3742598976 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa8c0275f0 con 0x7ffa90157870 2023-10-08T00:15:52.677 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.666+0000 7ffa9719a700 1 -- 172.21.15.99:0/3742598976 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa9014f6e0 con 0x7ffa90157870 2023-10-08T00:15:52.677 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.666+0000 7ffa84ff9700 1 -- 172.21.15.99:0/3742598976 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c0279b0 con 0x7ffa90157870 2023-10-08T00:15:52.677 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.666+0000 7ffa9719a700 1 -- 172.21.15.99:0/3742598976 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa9014e9f0 con 0x7ffa90157870 2023-10-08T00:15:52.677 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.667+0000 7ffa9719a700 1 -- 172.21.15.99:0/3742598976 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa9004c3c0 con 0x7ffa90157870 2023-10-08T00:15:52.677 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.667+0000 7ffa84ff9700 1 -- 172.21.15.99:0/3742598976 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffa8c024070 con 0x7ffa90157870 2023-10-08T00:15:52.677 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.668+0000 7ffa84ff9700 1 --2- 172.21.15.99:0/3742598976 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ffa7c066f30 0x7ffa7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:52.678 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.668+0000 7ffa84ff9700 1 -- 172.21.15.99:0/3742598976 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ffa8c086bc0 con 0x7ffa90157870 2023-10-08T00:15:52.678 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.668+0000 7ffa87fff700 1 --2- 172.21.15.99:0/3742598976 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ffa7c066f30 0x7ffa7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:52.678 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.668+0000 7ffa87fff700 1 --2- 172.21.15.99:0/3742598976 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ffa7c066f30 0x7ffa7c0693e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ffa90153000 tx=0x7ffa80009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:52.678 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.672+0000 7ffa84ff9700 1 -- 172.21.15.99:0/3742598976 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffa8c053e30 con 0x7ffa90157870 2023-10-08T00:15:52.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.854+0000 7ffa9719a700 1 -- 172.21.15.99:0/3742598976 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7ffa9015c840 con 0x7ffa90157870 2023-10-08T00:15:52.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.855+0000 7ffa84ff9700 1 -- 172.21.15.99:0/3742598976 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v104) v1 ==== 81+0+32274 (secure 0 0 0) 0x7ffa8c01c3a0 con 0x7ffa90157870 2023-10-08T00:15:52.856 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:15:52.856 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:15:52.856 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 104, 2023-10-08T00:15:52.856 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", 2023-10-08T00:15:52.856 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2023-10-08T00:10:36.303601+0000", 2023-10-08T00:15:52.856 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2023-10-08T00:15:42.099867+0000", 2023-10-08T00:15:52.856 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2023-10-08T00:10:41.709171+0000", 2023-10-08T00:15:52.856 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2023-10-08T00:10:38.690404+0000", 2023-10-08T00:15:52.856 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-08T00:15:52.856 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2023-10-08T00:15:52.857 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2023-10-08T00:15:52.857 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2023-10-08T00:15:52.857 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2023-10-08T00:15:52.857 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2023-10-08T00:15:52.857 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2023-10-08T00:15:52.857 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:15:52.857 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2023-10-08T00:15:52.857 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2023-10-08T00:15:52.857 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-08T00:15:52.858 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-08T00:15:52.858 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2023-10-08T00:15:52.858 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 13, 2023-10-08T00:15:52.858 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2023-10-08T00:15:52.858 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2023-10-08T00:15:52.858 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "jewel", 2023-10-08T00:15:52.859 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "quincy", 2023-10-08T00:15:52.859 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2023-10-08T00:15:52.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.859 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2023-10-08T00:15:52.859 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2023-10-08T00:15:52.860 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:10:42.063344+0000", 2023-10-08T00:15:52.860 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:15:52.860 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:15:52.860 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:15:52.860 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:15:52.860 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:15:52.860 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:15:52.860 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:15:52.860 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:15:52.860 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:15:52.860 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:15:52.860 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:15:52.861 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:15:52.861 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2023-10-08T00:15:52.861 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2023-10-08T00:15:52.861 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2023-10-08T00:15:52.861 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2023-10-08T00:15:52.861 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2023-10-08T00:15:52.861 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:15:52.861 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:15:52.861 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:15:52.861 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:15:52.861 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:15:52.861 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:15:52.862 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:15:52.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.862 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "93", 2023-10-08T00:15:52.862 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:15:52.862 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:15:52.862 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:15:52.862 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:15:52.862 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:15:52.862 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:15:52.862 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:15:52.862 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:15:52.863 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:15:52.863 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:15:52.863 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:15:52.863 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:15:52.863 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:15:52.863 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:15:52.863 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:15:52.863 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:15:52.863 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:15:52.863 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:15:52.863 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:15:52.863 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:15:52.864 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:15:52.864 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:15:52.864 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:15:52.864 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:15:52.864 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:15:52.864 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:15:52.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.864 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:15:52.864 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:15:52.864 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:15:52.864 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:15:52.864 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:15:52.865 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:15:52.865 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:15:52.865 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:15:52.865 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:15:52.865 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:15:52.865 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:15:52.866 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:15:52.866 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2023-10-08T00:15:52.866 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2023-10-08T00:15:52.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.866 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:15:52.866 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2023-10-08T00:15:52.866 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:52.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.867 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 12, 2023-10-08T00:15:52.867 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2023-10-08T00:15:52.867 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:15:38.848324+0000", 2023-10-08T00:15:52.867 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:15:52.867 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:15:52.867 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:15:52.867 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:15:52.867 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:15:52.867 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:15:52.868 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:15:52.868 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:15:52.868 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:15:52.868 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:15:52.868 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:15:52.868 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:15:52.868 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2023-10-08T00:15:52.868 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2023-10-08T00:15:52.868 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2023-10-08T00:15:52.869 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2023-10-08T00:15:52.869 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2023-10-08T00:15:52.869 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:15:52.869 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:15:52.869 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:15:52.870 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:15:52.870 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:15:52.870 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:15:52.870 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:15:52.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.870 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "103", 2023-10-08T00:15:52.870 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:15:52.870 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:15:52.870 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:15:52.871 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:15:52.871 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:15:52.871 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:15:52.871 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:15:52.871 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:15:52.871 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:15:52.871 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:15:52.871 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:15:52.871 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:15:52.871 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:15:52.871 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:15:52.871 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:15:52.872 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:15:52.872 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:15:52.872 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:15:52.872 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:15:52.872 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:15:52.872 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:15:52.872 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:15:52.872 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:15:52.872 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:15:52.872 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:15:52.872 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:15:52.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.873 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:15:52.873 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:15:52.873 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:15:52.873 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:15:52.873 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:15:52.873 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:15:52.873 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:15:52.873 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:15:52.873 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:15:52.873 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:15:52.874 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:15:52.874 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:15:52.874 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2023-10-08T00:15:52.874 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2023-10-08T00:15:52.874 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2023-10-08T00:15:52.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.874 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:15:52.874 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:15:52.874 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2023-10-08T00:15:52.874 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:52.874 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:52.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.875 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 13, 2023-10-08T00:15:52.875 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2023-10-08T00:15:52.875 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:15:39.561379+0000", 2023-10-08T00:15:52.875 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:15:52.875 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:15:52.875 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:15:52.875 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:15:52.875 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:15:52.875 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:15:52.875 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:15:52.875 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:15:52.876 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:15:52.876 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:15:52.876 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:15:52.876 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:15:52.876 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2023-10-08T00:15:52.876 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2023-10-08T00:15:52.876 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2023-10-08T00:15:52.876 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2023-10-08T00:15:52.876 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2023-10-08T00:15:52.877 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:15:52.877 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:15:52.877 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:15:52.877 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:15:52.877 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:15:52.877 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:15:52.878 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:15:52.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.878 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "104", 2023-10-08T00:15:52.878 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:15:52.878 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:15:52.879 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:15:52.879 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:15:52.879 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:15:52.879 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:15:52.879 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:15:52.879 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:15:52.880 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:15:52.880 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:15:52.880 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:15:52.881 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:15:52.881 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:15:52.881 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:15:52.881 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:15:52.881 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:15:52.881 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:15:52.881 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:15:52.881 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:15:52.882 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:15:52.882 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:15:52.882 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:15:52.882 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:15:52.882 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:15:52.882 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:15:52.882 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:15:52.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.883 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:15:52.883 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:15:52.883 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:15:52.883 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:15:52.883 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:15:52.883 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:15:52.884 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:15:52.884 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:15:52.884 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:15:52.884 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:15:52.884 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:15:52.884 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:15:52.884 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2023-10-08T00:15:52.884 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2023-10-08T00:15:52.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.884 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:15:52.884 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:15:52.884 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2023-10-08T00:15:52.885 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:52.885 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:52.885 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:52.885 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:15:52.885 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2023-10-08T00:15:52.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.885 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:15:52.885 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:15:52.885 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:15:52.885 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:15:52.886 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:15:52.886 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:15:52.886 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:15:52.886 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:15:52.886 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:15:52.886 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 102, 2023-10-08T00:15:52.886 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:15:52.886 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:15:52.886 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:15:52.886 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:52.886 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.887 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:52.887 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2023-10-08T00:15:52.887 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:15:52.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.887 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:52.887 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2023-10-08T00:15:52.887 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:15:52.887 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:52.888 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:52.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.888 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:15:52.888 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:52.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.888 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:52.888 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2023-10-08T00:15:52.888 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:15:52.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.888 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:52.888 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2023-10-08T00:15:52.889 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:15:52.889 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:52.889 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:52.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.889 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:15:52.889 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:52.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.889 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:52.889 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2023-10-08T00:15:52.889 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:15:52.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.890 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:52.890 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2023-10-08T00:15:52.890 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:15:52.890 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:52.890 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:52.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.893 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:15:52.893 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:52.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.893 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:52.893 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2023-10-08T00:15:52.893 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:15:52.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.893 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:52.894 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2023-10-08T00:15:52.894 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:15:52.894 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:52.894 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:52.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.895 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/86365", 2023-10-08T00:15:52.895 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/86365", 2023-10-08T00:15:52.895 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/86365", 2023-10-08T00:15:52.895 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/86365", 2023-10-08T00:15:52.895 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:15:52.895 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:15:52.895 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:15:52.895 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:52.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.895 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:15:52.896 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:15:52.896 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:15:52.896 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:15:52.896 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:15:52.896 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:15:52.896 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:15:52.896 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:15:52.896 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:15:52.896 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 102, 2023-10-08T00:15:52.896 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:15:52.896 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:15:52.896 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:15:52.897 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:52.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.897 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:52.897 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2023-10-08T00:15:52.897 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:15:52.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.898 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:52.898 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2023-10-08T00:15:52.898 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:15:52.898 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:52.898 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:52.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.899 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:15:52.899 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:52.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.899 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:52.899 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2023-10-08T00:15:52.899 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:15:52.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.899 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:52.899 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2023-10-08T00:15:52.899 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:15:52.900 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:52.900 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:52.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.900 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:15:52.900 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:52.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.900 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:52.900 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2023-10-08T00:15:52.900 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:15:52.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.900 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:52.900 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2023-10-08T00:15:52.901 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:15:52.901 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:52.901 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:52.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.901 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:15:52.901 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:52.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.901 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:52.901 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2023-10-08T00:15:52.901 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:15:52.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.902 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:52.902 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2023-10-08T00:15:52.902 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:15:52.902 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:52.902 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:52.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.902 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/86366", 2023-10-08T00:15:52.902 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/86366", 2023-10-08T00:15:52.902 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/86366", 2023-10-08T00:15:52.903 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/86366", 2023-10-08T00:15:52.903 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:15:52.903 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:15:52.903 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:15:52.903 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:52.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.903 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:15:52.903 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:15:52.903 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:15:52.903 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:15:52.903 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:15:52.904 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:15:52.904 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:15:52.904 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:15:52.904 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:15:52.904 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 102, 2023-10-08T00:15:52.904 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:15:52.904 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:15:52.904 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:15:52.904 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:52.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.904 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:52.904 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2023-10-08T00:15:52.905 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:15:52.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.905 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:52.905 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2023-10-08T00:15:52.905 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:15:52.905 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:52.905 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:52.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.905 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:15:52.905 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:52.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.906 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:52.906 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2023-10-08T00:15:52.906 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:15:52.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.906 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:52.906 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2023-10-08T00:15:52.906 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:15:52.906 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:52.906 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:52.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.906 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:15:52.907 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:52.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.907 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:52.907 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2023-10-08T00:15:52.907 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:15:52.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.907 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:52.907 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2023-10-08T00:15:52.908 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:15:52.908 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:52.908 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:52.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.908 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:15:52.908 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:52.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.908 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:52.908 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2023-10-08T00:15:52.908 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:15:52.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.909 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:52.909 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2023-10-08T00:15:52.909 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:15:52.909 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:52.909 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:52.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.909 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6817/86382", 2023-10-08T00:15:52.909 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/86382", 2023-10-08T00:15:52.909 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/86382", 2023-10-08T00:15:52.909 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/86382", 2023-10-08T00:15:52.910 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:15:52.910 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:15:52.910 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:15:52.910 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:52.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.910 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:15:52.910 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:15:52.910 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:15:52.910 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:15:52.910 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:15:52.911 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:15:52.911 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:15:52.911 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:15:52.911 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:15:52.911 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 102, 2023-10-08T00:15:52.911 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:15:52.911 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:15:52.911 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:15:52.911 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:52.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.911 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:52.911 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2023-10-08T00:15:52.911 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:15:52.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.912 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:52.912 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2023-10-08T00:15:52.912 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:15:52.912 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:52.912 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:52.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.912 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:15:52.912 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:52.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.912 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:52.913 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2023-10-08T00:15:52.913 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:15:52.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.913 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:52.913 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2023-10-08T00:15:52.913 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:15:52.913 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:52.913 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:52.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.913 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:15:52.913 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:52.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.914 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:52.914 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2023-10-08T00:15:52.914 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:15:52.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.914 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:52.914 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2023-10-08T00:15:52.914 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:15:52.914 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:52.914 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:52.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.914 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:15:52.915 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:52.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.915 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:52.915 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2023-10-08T00:15:52.915 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:15:52.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.915 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:52.915 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2023-10-08T00:15:52.915 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:15:52.916 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:52.916 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:52.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.916 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6820/86393", 2023-10-08T00:15:52.916 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/86393", 2023-10-08T00:15:52.916 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/86393", 2023-10-08T00:15:52.916 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/86393", 2023-10-08T00:15:52.916 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:15:52.916 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:15:52.916 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:15:52.916 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:52.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.917 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:15:52.917 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:15:52.917 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:15:52.917 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:15:52.917 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:15:52.917 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:15:52.917 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:15:52.917 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:15:52.917 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:15:52.917 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 102, 2023-10-08T00:15:52.917 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:15:52.917 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:15:52.918 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:15:52.918 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:52.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.918 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:52.918 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6800", 2023-10-08T00:15:52.918 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:15:52.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.918 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:52.918 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6801", 2023-10-08T00:15:52.918 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:15:52.919 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:52.919 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:52.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.919 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:15:52.919 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:52.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.919 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:52.919 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6802", 2023-10-08T00:15:52.919 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:15:52.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.919 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:52.919 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6803", 2023-10-08T00:15:52.920 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:15:52.920 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:52.920 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:52.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.920 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:15:52.920 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:52.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.920 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:52.920 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6806", 2023-10-08T00:15:52.920 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:15:52.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.921 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:52.921 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6807", 2023-10-08T00:15:52.921 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:15:52.921 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:52.921 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:52.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.921 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:15:52.921 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:52.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.921 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:52.921 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6804", 2023-10-08T00:15:52.921 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:15:52.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.922 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:52.922 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6805", 2023-10-08T00:15:52.922 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:15:52.922 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:52.922 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:52.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.922 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6801/86570", 2023-10-08T00:15:52.923 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6803/86570", 2023-10-08T00:15:52.923 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6807/86570", 2023-10-08T00:15:52.923 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6805/86570", 2023-10-08T00:15:52.923 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:15:52.923 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:15:52.923 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:15:52.923 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:52.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.923 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:15:52.923 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:15:52.923 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:15:52.923 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:15:52.924 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:15:52.924 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:15:52.924 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:15:52.924 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:15:52.924 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:15:52.924 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 102, 2023-10-08T00:15:52.924 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:15:52.924 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:15:52.924 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:15:52.924 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:52.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.924 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:52.925 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6816", 2023-10-08T00:15:52.925 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:15:52.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.925 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:52.925 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6818", 2023-10-08T00:15:52.925 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:15:52.925 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:52.925 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:52.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.925 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:15:52.925 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:52.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.926 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:52.926 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6820", 2023-10-08T00:15:52.926 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:15:52.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.926 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:52.926 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6821", 2023-10-08T00:15:52.926 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:15:52.926 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:52.926 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:52.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.926 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:15:52.927 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:52.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.927 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:52.927 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6826", 2023-10-08T00:15:52.927 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:15:52.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.927 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:52.927 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6828", 2023-10-08T00:15:52.927 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:15:52.927 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:52.928 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:52.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.928 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:15:52.928 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:52.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.928 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:52.928 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6823", 2023-10-08T00:15:52.928 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:15:52.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.928 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:52.928 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6825", 2023-10-08T00:15:52.929 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:15:52.929 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:52.929 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:52.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.929 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6818/86608", 2023-10-08T00:15:52.929 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6821/86608", 2023-10-08T00:15:52.929 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6828/86608", 2023-10-08T00:15:52.929 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6825/86608", 2023-10-08T00:15:52.929 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:15:52.929 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:15:52.929 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:15:52.929 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:52.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.930 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:15:52.930 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:15:52.930 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:15:52.930 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:15:52.930 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:15:52.930 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:15:52.930 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:15:52.930 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:15:52.930 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:15:52.931 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 102, 2023-10-08T00:15:52.931 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:15:52.931 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:15:52.931 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:15:52.931 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:52.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.931 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:52.931 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6808", 2023-10-08T00:15:52.931 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:15:52.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.931 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:52.932 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6809", 2023-10-08T00:15:52.932 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:15:52.932 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:52.932 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:52.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.932 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:15:52.932 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:52.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.932 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:52.932 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6810", 2023-10-08T00:15:52.932 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:15:52.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.933 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:52.933 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6811", 2023-10-08T00:15:52.933 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:15:52.933 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:52.933 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:52.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.933 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:15:52.933 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:52.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.933 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:52.933 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6814", 2023-10-08T00:15:52.933 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:15:52.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.934 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:52.934 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6815", 2023-10-08T00:15:52.934 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:15:52.934 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:52.934 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:52.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.934 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:15:52.934 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:52.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.934 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:52.935 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6812", 2023-10-08T00:15:52.935 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:15:52.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.935 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:52.935 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6813", 2023-10-08T00:15:52.935 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:15:52.935 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:52.935 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:52.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.935 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6809/86571", 2023-10-08T00:15:52.935 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6811/86571", 2023-10-08T00:15:52.936 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6815/86571", 2023-10-08T00:15:52.936 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6813/86571", 2023-10-08T00:15:52.936 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:15:52.936 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:15:52.936 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:15:52.936 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:52.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.936 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:15:52.936 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:15:52.936 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:15:52.936 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:15:52.937 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:15:52.937 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:15:52.937 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:15:52.937 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:15:52.937 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:15:52.937 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 102, 2023-10-08T00:15:52.937 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:15:52.937 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:15:52.937 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:15:52.937 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:52.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.937 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:52.937 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6817", 2023-10-08T00:15:52.938 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:15:52.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.938 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:52.938 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6819", 2023-10-08T00:15:52.938 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:15:52.938 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:52.938 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:52.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.938 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:15:52.938 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:52.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.939 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:52.939 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6822", 2023-10-08T00:15:52.939 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:15:52.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.939 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:52.939 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6824", 2023-10-08T00:15:52.939 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:15:52.939 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:52.939 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:52.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.940 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:15:52.940 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:52.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.940 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:52.940 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6830", 2023-10-08T00:15:52.940 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:15:52.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.940 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:52.940 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6831", 2023-10-08T00:15:52.940 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:15:52.940 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:52.941 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:52.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.941 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:15:52.941 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:52.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.941 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:52.941 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6827", 2023-10-08T00:15:52.941 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:15:52.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.941 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:52.941 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6829", 2023-10-08T00:15:52.942 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:15:52.942 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:52.942 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:52.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.942 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6819/86612", 2023-10-08T00:15:52.942 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6824/86612", 2023-10-08T00:15:52.942 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6831/86612", 2023-10-08T00:15:52.942 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6829/86612", 2023-10-08T00:15:52.942 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:15:52.942 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:15:52.942 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:15:52.942 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:52.943 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:52.943 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:15:52.943 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2023-10-08T00:15:52.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.943 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:15:52.943 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:15:52.943 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:15:52.943 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:15:52.943 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:15:52.943 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:15:52.943 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.905615+0000", 2023-10-08T00:15:52.943 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:15:52.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.944 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:15:52.944 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:15:52.944 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:15:52.944 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:15:52.944 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:15:52.944 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:15:52.944 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.880709+0000", 2023-10-08T00:15:52.944 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:15:52.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.944 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:15:52.945 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:15:52.945 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:15:52.945 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:15:52.945 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:15:52.945 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:15:52.945 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.874704+0000", 2023-10-08T00:15:52.945 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:15:52.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.945 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.945 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:15:52.945 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:15:52.945 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:15:52.946 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:15:52.946 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:15:52.946 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:15:52.946 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.899604+0000", 2023-10-08T00:15:52.946 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:15:52.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.946 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.946 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:15:52.946 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:15:52.946 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:15:52.946 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:15:52.946 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:15:52.947 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:15:52.947 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.844371+0000", 2023-10-08T00:15:52.947 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:15:52.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.947 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.947 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:15:52.947 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:15:52.947 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:15:52.947 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:15:52.947 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:15:52.947 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:15:52.947 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.794810+0000", 2023-10-08T00:15:52.948 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:15:52.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.948 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.948 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:15:52.948 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:15:52.948 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:15:52.948 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:15:52.948 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:15:52.948 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:15:52.948 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.859474+0000", 2023-10-08T00:15:52.948 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:15:52.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.948 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:52.949 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:15:52.949 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:15:52.949 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:15:52.949 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:15:52.949 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:15:52.949 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:15:52.949 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.893130+0000", 2023-10-08T00:15:52.949 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:15:52.949 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:52.949 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:15:52.949 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2023-10-08T00:15:52.949 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [], 2023-10-08T00:15:52.950 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2023-10-08T00:15:52.950 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2023-10-08T00:15:52.950 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2023-10-08T00:15:52.950 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2023-10-08T00:15:52.950 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2023-10-08T00:15:52.950 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2023-10-08T00:15:52.950 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2023-10-08T00:15:52.950 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2023-10-08T00:15:52.950 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2023-10-08T00:15:52.950 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2023-10-08T00:15:52.950 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2023-10-08T00:15:52.950 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:52.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:52.951 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2023-10-08T00:15:52.951 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2023-10-08T00:15:52.951 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2023-10-08T00:15:52.951 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2023-10-08T00:15:52.951 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2023-10-08T00:15:52.951 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2023-10-08T00:15:52.951 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2023-10-08T00:15:52.951 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2023-10-08T00:15:52.951 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2023-10-08T00:15:52.951 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2023-10-08T00:15:52.951 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2023-10-08T00:15:52.952 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:52.952 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:15:52.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.857+0000 7ffa9719a700 1 -- 172.21.15.99:0/3742598976 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ffa7c066f30 msgr2=0x7ffa7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:52.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.857+0000 7ffa9719a700 1 --2- 172.21.15.99:0/3742598976 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ffa7c066f30 0x7ffa7c0693e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ffa90153000 tx=0x7ffa80009040 comp rx=0 tx=0).stop 2023-10-08T00:15:52.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.857+0000 7ffa9719a700 1 -- 172.21.15.99:0/3742598976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa90157870 msgr2=0x7ffa90153750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:52.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.857+0000 7ffa9719a700 1 --2- 172.21.15.99:0/3742598976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa90157870 0x7ffa90153750 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c0097c0 tx=0x7ffa8c019000 comp rx=0 tx=0).stop 2023-10-08T00:15:52.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.857+0000 7ffa9719a700 1 -- 172.21.15.99:0/3742598976 shutdown_connections 2023-10-08T00:15:52.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.857+0000 7ffa9719a700 1 --2- 172.21.15.99:0/3742598976 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ffa7c066f30 0x7ffa7c0693e0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:52.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.857+0000 7ffa9719a700 1 --2- 172.21.15.99:0/3742598976 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffa9015a620 0x7ffa900b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:52.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.857+0000 7ffa9719a700 1 --2- 172.21.15.99:0/3742598976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa90157870 0x7ffa90153750 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:52.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.857+0000 7ffa9719a700 1 --2- 172.21.15.99:0/3742598976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa90157c40 0x7ffa900b2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:52.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.857+0000 7ffa9719a700 1 -- 172.21.15.99:0/3742598976 >> 172.21.15.99:0/3742598976 conn(0x7ffa90068ce0 msgr2=0x7ffa900b4810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:52.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.857+0000 7ffa9719a700 1 -- 172.21.15.99:0/3742598976 shutdown_connections 2023-10-08T00:15:52.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:52.857+0000 7ffa9719a700 1 -- 172.21.15.99:0/3742598976 wait complete. 2023-10-08T00:15:52.955 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-08T00:15:53.236 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:15:53.239 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:15:54.262 INFO:tasks.ceph.mon.a.smithi099.stderr:2023-10-08T00:15:54.262+0000 7f6c637a7700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 full osd(s) (OSD_FULL) 2023-10-08T00:15:57.956 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-08T00:15:58.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.081+0000 7f7585c2e700 1 Processor -- start 2023-10-08T00:15:58.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.082+0000 7f7585c2e700 1 -- start start 2023-10-08T00:15:58.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.082+0000 7f7585c2e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f758015a620 0x7f758015d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:58.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.082+0000 7f7585c2e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f758015d580 0x7f7580158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:58.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.082+0000 7f7585c2e700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f758015cbb0 con 0x7f7580157870 2023-10-08T00:15:58.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.082+0000 7f7585c2e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f758015bdb0 con 0x7f758015a620 2023-10-08T00:15:58.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.082+0000 7f7585c2e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f758015bfa0 con 0x7f758015d580 2023-10-08T00:15:58.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.082+0000 7f757f7fe700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f7580157870 0x7f758015b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:47622/0 (socket says 172.21.15.99:47622) 2023-10-08T00:15:58.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.083+0000 7f757f7fe700 1 -- 172.21.15.99:0/3111418057 learned_addr learned my addr 172.21.15.99:0/3111418057 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:15:58.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.083+0000 7f757ffff700 1 --2- 172.21.15.99:0/3111418057 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f758015d580 0x7f7580158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:58.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.083+0000 7f757effd700 1 --2- 172.21.15.99:0/3111418057 >> v2:172.21.15.191:3300/0 conn(0x7f758015a620 0x7f758015d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:58.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.083+0000 7f757e7fc700 1 -- 172.21.15.99:0/3111418057 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2087771257 0 0) 0x7f758015cbb0 con 0x7f7580157870 2023-10-08T00:15:58.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.083+0000 7f757e7fc700 1 -- 172.21.15.99:0/3111418057 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7568003570 con 0x7f7580157870 2023-10-08T00:15:58.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.083+0000 7f757e7fc700 1 -- 172.21.15.99:0/3111418057 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1209129530 0 0) 0x7f7568003570 con 0x7f7580157870 2023-10-08T00:15:58.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.083+0000 7f757e7fc700 1 -- 172.21.15.99:0/3111418057 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f758015d580 msgr2=0x7f7580158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:58.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.083+0000 7f757e7fc700 1 --2- 172.21.15.99:0/3111418057 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f758015d580 0x7f7580158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:58.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.083+0000 7f757e7fc700 1 -- 172.21.15.99:0/3111418057 >> v2:172.21.15.191:3300/0 conn(0x7f758015a620 msgr2=0x7f758015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:58.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.083+0000 7f757e7fc700 1 --2- 172.21.15.99:0/3111418057 >> v2:172.21.15.191:3300/0 conn(0x7f758015a620 0x7f758015d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:58.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.083+0000 7f757e7fc700 1 -- 172.21.15.99:0/3111418057 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f758015c280 con 0x7f7580157870 2023-10-08T00:15:58.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.083+0000 7f757e7fc700 1 -- 172.21.15.99:0/3111418057 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f75700017f0 con 0x7f7580157870 2023-10-08T00:15:58.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.083+0000 7f757e7fc700 1 -- 172.21.15.99:0/3111418057 >> v1:172.21.15.99:6789/0 conn(0x7f7580157870 legacy=0x7f758015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:58.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.083+0000 7f757e7fc700 1 --2- 172.21.15.99:0/3111418057 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f758015d580 0x7f7568003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:58.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.083+0000 7f757e7fc700 1 --2- 172.21.15.99:0/3111418057 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7568003fe0 0x7f7568004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:58.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.083+0000 7f757e7fc700 1 --2- 172.21.15.99:0/3111418057 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7568004d50 0x7f75680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:58.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.083+0000 7f757e7fc700 1 -- 172.21.15.99:0/3111418057 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758015bfa0 con 0x7f7568003fe0 2023-10-08T00:15:58.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.083+0000 7f757e7fc700 1 -- 172.21.15.99:0/3111418057 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756800ba60 con 0x7f7568004d50 2023-10-08T00:15:58.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.083+0000 7f757e7fc700 1 -- 172.21.15.99:0/3111418057 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f756800bbd0 con 0x7f758015d580 2023-10-08T00:15:58.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.083+0000 7f757effd700 1 --2- 172.21.15.99:0/3111418057 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7568003fe0 0x7f7568004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:58.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.083+0000 7f757f7fe700 1 --2- 172.21.15.99:0/3111418057 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f758015d580 0x7f7568003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:58.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.083+0000 7f757ffff700 1 --2- 172.21.15.99:0/3111418057 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7568004d50 0x7f75680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:58.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.083+0000 7f757effd700 1 -- 172.21.15.99:0/3111418057 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f758015d580 msgr2=0x7f7568003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:58.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.083+0000 7f757effd700 1 --2- 172.21.15.99:0/3111418057 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f758015d580 0x7f7568003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:58.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.083+0000 7f757effd700 1 -- 172.21.15.99:0/3111418057 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7568004d50 msgr2=0x7f75680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:58.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.083+0000 7f757effd700 1 --2- 172.21.15.99:0/3111418057 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7568004d50 0x7f75680051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:58.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.084+0000 7f757effd700 1 -- 172.21.15.99:0/3111418057 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7568003570 con 0x7f7568003fe0 2023-10-08T00:15:58.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.084+0000 7f757ffff700 1 --2- 172.21.15.99:0/3111418057 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7568004d50 0x7f75680051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:15:58.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.084+0000 7f757effd700 1 --2- 172.21.15.99:0/3111418057 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7568003fe0 0x7f7568004810 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f758015aa20 tx=0x7f75740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94764dcff40651f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:58.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.084+0000 7f757e7fc700 1 -- 172.21.15.99:0/3111418057 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757400e6b0 con 0x7f7568003fe0 2023-10-08T00:15:58.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.084+0000 7f757e7fc700 1 -- 172.21.15.99:0/3111418057 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f757400e810 con 0x7f7568003fe0 2023-10-08T00:15:58.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.084+0000 7f757e7fc700 1 -- 172.21.15.99:0/3111418057 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757400ebd0 con 0x7f7568003fe0 2023-10-08T00:15:58.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.084+0000 7f7585c2e700 1 -- 172.21.15.99:0/3111418057 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7568003fe0 msgr2=0x7f7568004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:58.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.084+0000 7f7585c2e700 1 --2- 172.21.15.99:0/3111418057 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7568003fe0 0x7f7568004810 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f758015aa20 tx=0x7f75740091a0 comp rx=0 tx=0).stop 2023-10-08T00:15:58.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.084+0000 7f757ffff700 1 -- 172.21.15.99:0/3111418057 reap_dead start 2023-10-08T00:15:58.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.084+0000 7f7585c2e700 1 -- 172.21.15.99:0/3111418057 shutdown_connections 2023-10-08T00:15:58.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.084+0000 7f7585c2e700 1 -- 172.21.15.99:0/3111418057 >> 172.21.15.99:0/3111418057 conn(0x7f7580068ce0 msgr2=0x7f75800690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:58.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.084+0000 7f7585c2e700 1 -- 172.21.15.99:0/3111418057 shutdown_connections 2023-10-08T00:15:58.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.084+0000 7f7585c2e700 1 -- 172.21.15.99:0/3111418057 wait complete. 2023-10-08T00:15:58.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.086+0000 7f7585c2e700 1 Processor -- start 2023-10-08T00:15:58.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.086+0000 7f7585c2e700 1 -- start start 2023-10-08T00:15:58.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.086+0000 7f7585c2e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7568003fe0 0x7f75800b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:58.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.086+0000 7f7585c2e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f75800b1470 0x7f75800b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:58.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.086+0000 7f7585c2e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7580149bc0 0x7f758014a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:58.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.086+0000 7f7585c2e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75800b1e20 con 0x7f7580149bc0 2023-10-08T00:15:58.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.086+0000 7f7585c2e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75800b3610 con 0x7f75800b1470 2023-10-08T00:15:58.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.086+0000 7f7585c2e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75800b2c40 con 0x7f7568003fe0 2023-10-08T00:15:58.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.087+0000 7f757ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7580149bc0 0x7f758014a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:58.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.087+0000 7f757ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7580149bc0 0x7f758014a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58756/0 (socket says 172.21.15.99:58756) 2023-10-08T00:15:58.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.087+0000 7f757f7fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7568003fe0 0x7f75800b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:58.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.087+0000 7f757ffff700 1 -- 172.21.15.99:0/3288141748 learned_addr learned my addr 172.21.15.99:0/3288141748 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:58.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.087+0000 7f757effd700 1 --2- 172.21.15.99:0/3288141748 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f75800b1470 0x7f75800b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:58.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.087+0000 7f757ffff700 1 -- 172.21.15.99:0/3288141748 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7568003fe0 msgr2=0x7f75800b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:58.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.087+0000 7f757ffff700 1 --2- 172.21.15.99:0/3288141748 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7568003fe0 0x7f75800b30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:58.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.087+0000 7f757ffff700 1 -- 172.21.15.99:0/3288141748 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f75800b1470 msgr2=0x7f75800b18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:58.097 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.087+0000 7f757ffff700 1 --2- 172.21.15.99:0/3288141748 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f75800b1470 0x7f75800b18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:58.097 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.087+0000 7f757ffff700 1 -- 172.21.15.99:0/3288141748 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75801499c0 con 0x7f7580149bc0 2023-10-08T00:15:58.097 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.087+0000 7f757ffff700 1 --2- 172.21.15.99:0/3288141748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7580149bc0 0x7f758014a030 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f758015d9e0 tx=0x7f7578008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:58.097 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.087+0000 7f75677fe700 1 -- 172.21.15.99:0/3288141748 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7578011ea0 con 0x7f7580149bc0 2023-10-08T00:15:58.097 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.087+0000 7f75677fe700 1 -- 172.21.15.99:0/3288141748 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7578015670 con 0x7f7580149bc0 2023-10-08T00:15:58.097 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.087+0000 7f7585c2e700 1 -- 172.21.15.99:0/3288141748 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75800b2030 con 0x7f7580149bc0 2023-10-08T00:15:58.098 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.087+0000 7f75677fe700 1 -- 172.21.15.99:0/3288141748 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7578015a30 con 0x7f7580149bc0 2023-10-08T00:15:58.098 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.087+0000 7f7585c2e700 1 -- 172.21.15.99:0/3288141748 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75800b25f0 con 0x7f7580149bc0 2023-10-08T00:15:58.098 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.088+0000 7f7585c2e700 1 -- 172.21.15.99:0/3288141748 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f758004c3c0 con 0x7f7580149bc0 2023-10-08T00:15:58.098 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.088+0000 7f75677fe700 1 -- 172.21.15.99:0/3288141748 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7578020460 con 0x7f7580149bc0 2023-10-08T00:15:58.098 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.089+0000 7f75677fe700 1 --2- 172.21.15.99:0/3288141748 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f756c066f30 0x7f756c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:58.098 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.089+0000 7f75677fe700 1 -- 172.21.15.99:0/3288141748 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f75780846f0 con 0x7f7580149bc0 2023-10-08T00:15:58.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.089+0000 7f75677fe700 1 -- 172.21.15.99:0/3288141748 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=109}) v3 -- 0x7f756c06ea50 con 0x7f7580149bc0 2023-10-08T00:15:58.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.089+0000 7f757f7fe700 1 --2- 172.21.15.99:0/3288141748 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f756c066f30 0x7f756c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:58.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.089+0000 7f757f7fe700 1 --2- 172.21.15.99:0/3288141748 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f756c066f30 0x7f756c0693e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f7570004f40 tx=0x7f7570006040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:58.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.093+0000 7f75677fe700 1 -- 172.21.15.99:0/3288141748 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7578051960 con 0x7f7580149bc0 2023-10-08T00:15:58.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.273+0000 7f7585c2e700 1 -- 172.21.15.99:0/3288141748 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f75800b2d80 con 0x7f7580149bc0 2023-10-08T00:15:58.275 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.274+0000 7f75677fe700 1 -- 172.21.15.99:0/3288141748 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v108) v1 ==== 81+0+32458 (secure 0 0 0) 0x7f7578054f80 con 0x7f7580149bc0 2023-10-08T00:15:58.275 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:15:58.275 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:15:58.276 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 108, 2023-10-08T00:15:58.276 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", 2023-10-08T00:15:58.276 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2023-10-08T00:10:36.303601+0000", 2023-10-08T00:15:58.276 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2023-10-08T00:15:57.327150+0000", 2023-10-08T00:15:58.276 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2023-10-08T00:10:41.709171+0000", 2023-10-08T00:15:58.276 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2023-10-08T00:10:38.690404+0000", 2023-10-08T00:15:58.276 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-08T00:15:58.276 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2023-10-08T00:15:58.276 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2023-10-08T00:15:58.276 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2023-10-08T00:15:58.276 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2023-10-08T00:15:58.277 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2023-10-08T00:15:58.277 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2023-10-08T00:15:58.277 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:15:58.277 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2023-10-08T00:15:58.277 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2023-10-08T00:15:58.277 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-08T00:15:58.277 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-08T00:15:58.277 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2023-10-08T00:15:58.277 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 13, 2023-10-08T00:15:58.277 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2023-10-08T00:15:58.277 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2023-10-08T00:15:58.278 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "jewel", 2023-10-08T00:15:58.278 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "quincy", 2023-10-08T00:15:58.278 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2023-10-08T00:15:58.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.278 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2023-10-08T00:15:58.278 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2023-10-08T00:15:58.278 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:10:42.063344+0000", 2023-10-08T00:15:58.278 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 3, 2023-10-08T00:15:58.278 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full", 2023-10-08T00:15:58.278 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:15:58.278 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:15:58.279 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:15:58.279 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:15:58.279 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:15:58.279 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:15:58.279 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:15:58.279 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:15:58.280 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:15:58.280 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:15:58.280 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2023-10-08T00:15:58.280 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2023-10-08T00:15:58.280 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2023-10-08T00:15:58.280 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2023-10-08T00:15:58.280 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2023-10-08T00:15:58.280 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:15:58.280 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:15:58.280 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:15:58.281 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:15:58.281 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:15:58.281 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:15:58.281 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:15:58.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.281 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "105", 2023-10-08T00:15:58.281 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:15:58.281 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:15:58.281 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:15:58.281 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:15:58.282 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:15:58.282 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:15:58.282 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:15:58.282 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:15:58.282 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:15:58.283 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:15:58.283 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:15:58.283 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:15:58.284 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:15:58.284 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:15:58.284 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:15:58.284 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:15:58.284 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:15:58.284 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:15:58.284 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:15:58.285 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:15:58.285 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:15:58.285 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:15:58.285 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:15:58.285 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:15:58.285 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:15:58.285 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:15:58.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.285 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:15:58.285 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:15:58.285 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:15:58.286 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:15:58.286 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:15:58.286 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:15:58.286 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:15:58.286 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:15:58.286 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:15:58.286 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:15:58.286 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:15:58.286 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:15:58.286 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2023-10-08T00:15:58.286 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2023-10-08T00:15:58.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.287 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:15:58.287 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2023-10-08T00:15:58.287 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.287 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 12, 2023-10-08T00:15:58.287 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2023-10-08T00:15:58.287 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:15:38.848324+0000", 2023-10-08T00:15:58.287 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 3, 2023-10-08T00:15:58.287 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full", 2023-10-08T00:15:58.288 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:15:58.288 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:15:58.288 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:15:58.288 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:15:58.288 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:15:58.288 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:15:58.288 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:15:58.288 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:15:58.289 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:15:58.289 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:15:58.289 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2023-10-08T00:15:58.289 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2023-10-08T00:15:58.289 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2023-10-08T00:15:58.289 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2023-10-08T00:15:58.289 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2023-10-08T00:15:58.289 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:15:58.289 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:15:58.289 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:15:58.289 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:15:58.289 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:15:58.290 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:15:58.290 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:15:58.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.290 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "105", 2023-10-08T00:15:58.290 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:15:58.290 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:15:58.290 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:15:58.290 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:15:58.291 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:15:58.291 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:15:58.291 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:15:58.291 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:15:58.291 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:15:58.291 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:15:58.291 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:15:58.291 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:15:58.291 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:15:58.292 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:15:58.292 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:15:58.292 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:15:58.292 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:15:58.292 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:15:58.292 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:15:58.292 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:15:58.293 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:15:58.293 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:15:58.293 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:15:58.293 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:15:58.293 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:15:58.293 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:15:58.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.293 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:15:58.293 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:15:58.293 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:15:58.293 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:15:58.293 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:15:58.294 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:15:58.294 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:15:58.294 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:15:58.294 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:15:58.294 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:15:58.294 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:15:58.294 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:15:58.294 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2023-10-08T00:15:58.294 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2023-10-08T00:15:58.295 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2023-10-08T00:15:58.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.295 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:15:58.295 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:15:58.295 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2023-10-08T00:15:58.295 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.295 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.295 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 13, 2023-10-08T00:15:58.295 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2023-10-08T00:15:58.295 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:15:39.561379+0000", 2023-10-08T00:15:58.296 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 3, 2023-10-08T00:15:58.296 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full", 2023-10-08T00:15:58.296 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:15:58.296 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:15:58.296 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:15:58.296 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:15:58.296 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:15:58.296 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:15:58.296 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:15:58.296 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:15:58.296 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:15:58.296 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:15:58.297 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2023-10-08T00:15:58.297 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2023-10-08T00:15:58.297 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2023-10-08T00:15:58.297 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2023-10-08T00:15:58.297 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2023-10-08T00:15:58.297 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:15:58.297 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:15:58.297 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:15:58.297 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:15:58.297 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:15:58.297 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:15:58.297 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:15:58.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.298 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "105", 2023-10-08T00:15:58.298 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:15:58.298 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:15:58.298 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:15:58.298 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:15:58.299 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:15:58.299 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:15:58.299 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:15:58.299 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:15:58.299 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:15:58.299 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:15:58.299 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:15:58.299 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:15:58.299 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:15:58.299 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:15:58.299 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:15:58.299 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:15:58.300 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:15:58.300 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:15:58.300 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:15:58.300 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:15:58.300 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:15:58.300 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:15:58.300 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:15:58.300 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:15:58.300 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:15:58.301 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:15:58.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.301 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:15:58.301 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:15:58.301 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:15:58.301 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:15:58.301 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:15:58.301 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:15:58.301 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:15:58.301 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:15:58.302 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:15:58.302 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:15:58.302 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:15:58.302 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:15:58.302 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2023-10-08T00:15:58.302 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2023-10-08T00:15:58.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.302 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:15:58.302 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:15:58.302 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2023-10-08T00:15:58.303 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.303 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.303 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.303 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:15:58.303 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2023-10-08T00:15:58.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.303 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:15:58.303 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:15:58.303 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:15:58.303 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:15:58.303 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:15:58.304 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:15:58.304 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:15:58.304 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:15:58.304 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:15:58.304 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 102, 2023-10-08T00:15:58.304 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:15:58.304 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:15:58.304 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:15:58.305 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.305 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.305 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2023-10-08T00:15:58.305 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:15:58.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.305 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.305 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2023-10-08T00:15:58.305 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:15:58.305 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.305 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.306 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:15:58.306 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.306 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.306 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2023-10-08T00:15:58.306 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:15:58.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.306 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.307 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2023-10-08T00:15:58.307 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:15:58.307 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.307 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.307 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:15:58.307 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.307 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.308 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2023-10-08T00:15:58.308 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:15:58.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.308 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.308 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2023-10-08T00:15:58.308 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:15:58.308 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.308 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.308 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:15:58.308 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.309 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.309 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2023-10-08T00:15:58.309 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:15:58.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.309 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.309 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2023-10-08T00:15:58.309 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:15:58.309 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.310 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.310 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/86365", 2023-10-08T00:15:58.310 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/86365", 2023-10-08T00:15:58.310 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/86365", 2023-10-08T00:15:58.310 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/86365", 2023-10-08T00:15:58.310 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:15:58.310 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull", 2023-10-08T00:15:58.310 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:15:58.310 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:15:58.310 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.311 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:15:58.311 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:15:58.311 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:15:58.311 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:15:58.311 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:15:58.311 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:15:58.311 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:15:58.311 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:15:58.311 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:15:58.312 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 102, 2023-10-08T00:15:58.312 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:15:58.312 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:15:58.312 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:15:58.312 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.312 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.312 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2023-10-08T00:15:58.312 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:15:58.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.313 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.313 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2023-10-08T00:15:58.313 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:15:58.313 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.313 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.314 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:15:58.314 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.314 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.315 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2023-10-08T00:15:58.315 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:15:58.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.315 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.315 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2023-10-08T00:15:58.315 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:15:58.315 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.315 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.315 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:15:58.316 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.316 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.316 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2023-10-08T00:15:58.316 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:15:58.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.316 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.316 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2023-10-08T00:15:58.316 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:15:58.316 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.316 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.317 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:15:58.317 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.317 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.317 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2023-10-08T00:15:58.317 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:15:58.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.317 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.317 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2023-10-08T00:15:58.318 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:15:58.318 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.318 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.318 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/86366", 2023-10-08T00:15:58.318 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/86366", 2023-10-08T00:15:58.318 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/86366", 2023-10-08T00:15:58.318 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/86366", 2023-10-08T00:15:58.318 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:15:58.318 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull", 2023-10-08T00:15:58.318 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:15:58.319 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:15:58.319 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.319 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:15:58.319 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:15:58.319 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:15:58.319 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:15:58.319 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:15:58.319 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:15:58.320 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:15:58.320 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:15:58.320 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:15:58.320 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 102, 2023-10-08T00:15:58.320 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:15:58.320 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:15:58.320 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:15:58.320 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.320 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.320 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2023-10-08T00:15:58.321 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:15:58.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.321 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.321 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2023-10-08T00:15:58.321 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:15:58.321 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.321 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.321 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:15:58.321 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.322 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.322 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2023-10-08T00:15:58.322 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:15:58.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.322 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.322 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2023-10-08T00:15:58.322 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:15:58.322 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.322 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.323 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:15:58.323 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.323 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.323 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2023-10-08T00:15:58.323 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:15:58.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.323 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.323 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2023-10-08T00:15:58.323 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:15:58.323 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.324 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.324 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:15:58.324 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.324 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.324 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2023-10-08T00:15:58.324 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:15:58.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.324 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.324 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2023-10-08T00:15:58.325 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:15:58.325 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.325 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.325 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6817/86382", 2023-10-08T00:15:58.325 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/86382", 2023-10-08T00:15:58.325 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/86382", 2023-10-08T00:15:58.325 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/86382", 2023-10-08T00:15:58.325 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:15:58.325 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull", 2023-10-08T00:15:58.325 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:15:58.326 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:15:58.326 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.326 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:15:58.326 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:15:58.326 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:15:58.326 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:15:58.326 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:15:58.326 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:15:58.326 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:15:58.327 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:15:58.327 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:15:58.327 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 102, 2023-10-08T00:15:58.327 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:15:58.327 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:15:58.327 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:15:58.327 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.327 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.327 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2023-10-08T00:15:58.327 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:15:58.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.328 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.328 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2023-10-08T00:15:58.328 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:15:58.328 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.328 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.328 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:15:58.328 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.328 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.328 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2023-10-08T00:15:58.329 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:15:58.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.329 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.329 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2023-10-08T00:15:58.329 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:15:58.329 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.329 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.329 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:15:58.329 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.330 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.330 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2023-10-08T00:15:58.330 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:15:58.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.330 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.330 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2023-10-08T00:15:58.330 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:15:58.330 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.330 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.331 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:15:58.331 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.331 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.331 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2023-10-08T00:15:58.331 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:15:58.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.331 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.331 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2023-10-08T00:15:58.331 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:15:58.331 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.332 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.332 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6820/86393", 2023-10-08T00:15:58.332 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/86393", 2023-10-08T00:15:58.332 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/86393", 2023-10-08T00:15:58.332 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/86393", 2023-10-08T00:15:58.332 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:15:58.332 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:15:58.332 INFO:teuthology.orchestra.run.smithi099.stdout: "full", 2023-10-08T00:15:58.332 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:15:58.332 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.333 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:15:58.333 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:15:58.333 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:15:58.333 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:15:58.333 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:15:58.333 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:15:58.333 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:15:58.333 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:15:58.333 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:15:58.333 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 102, 2023-10-08T00:15:58.333 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:15:58.334 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:15:58.334 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:15:58.334 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.334 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.334 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6800", 2023-10-08T00:15:58.334 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:15:58.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.334 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.334 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6801", 2023-10-08T00:15:58.335 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:15:58.335 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.335 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.335 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:15:58.335 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.335 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.335 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6802", 2023-10-08T00:15:58.336 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:15:58.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.336 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.336 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6803", 2023-10-08T00:15:58.336 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:15:58.336 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.336 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.336 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:15:58.337 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.337 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.337 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6806", 2023-10-08T00:15:58.337 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:15:58.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.337 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.337 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6807", 2023-10-08T00:15:58.337 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:15:58.337 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.337 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.338 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:15:58.338 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.338 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.338 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6804", 2023-10-08T00:15:58.338 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:15:58.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.338 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.338 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6805", 2023-10-08T00:15:58.338 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:15:58.339 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.339 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.339 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6801/86570", 2023-10-08T00:15:58.339 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6803/86570", 2023-10-08T00:15:58.339 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6807/86570", 2023-10-08T00:15:58.339 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6805/86570", 2023-10-08T00:15:58.339 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:15:58.339 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:15:58.339 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:15:58.339 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.340 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:15:58.340 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:15:58.340 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:15:58.340 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:15:58.340 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:15:58.340 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:15:58.340 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:15:58.340 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:15:58.340 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:15:58.340 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 102, 2023-10-08T00:15:58.340 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:15:58.341 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:15:58.341 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:15:58.341 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.341 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.341 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6816", 2023-10-08T00:15:58.341 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:15:58.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.341 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.341 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6818", 2023-10-08T00:15:58.341 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:15:58.342 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.342 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.342 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:15:58.342 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.342 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.342 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6820", 2023-10-08T00:15:58.342 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:15:58.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.342 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.343 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6821", 2023-10-08T00:15:58.343 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:15:58.343 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.343 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.343 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:15:58.343 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.343 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.343 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6826", 2023-10-08T00:15:58.343 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:15:58.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.344 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.344 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6828", 2023-10-08T00:15:58.344 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:15:58.344 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.344 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.344 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:15:58.344 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.344 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.345 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6823", 2023-10-08T00:15:58.345 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:15:58.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.345 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.345 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6825", 2023-10-08T00:15:58.345 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:15:58.345 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.345 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.345 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6818/86608", 2023-10-08T00:15:58.345 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6821/86608", 2023-10-08T00:15:58.345 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6828/86608", 2023-10-08T00:15:58.346 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6825/86608", 2023-10-08T00:15:58.346 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:15:58.346 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:15:58.346 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:15:58.346 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.346 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:15:58.346 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:15:58.346 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:15:58.346 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:15:58.347 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:15:58.347 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:15:58.347 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:15:58.347 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:15:58.347 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:15:58.347 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 102, 2023-10-08T00:15:58.347 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:15:58.347 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:15:58.347 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:15:58.347 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.347 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.347 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6808", 2023-10-08T00:15:58.347 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:15:58.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.348 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.348 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6809", 2023-10-08T00:15:58.348 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:15:58.348 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.348 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.348 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:15:58.348 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.348 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.349 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6810", 2023-10-08T00:15:58.349 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:15:58.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.349 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.349 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6811", 2023-10-08T00:15:58.349 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:15:58.349 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.349 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.349 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:15:58.350 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.350 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.350 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6814", 2023-10-08T00:15:58.350 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:15:58.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.350 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.350 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6815", 2023-10-08T00:15:58.350 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:15:58.350 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.351 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.351 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:15:58.351 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.351 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.351 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6812", 2023-10-08T00:15:58.351 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:15:58.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.351 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.351 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6813", 2023-10-08T00:15:58.352 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:15:58.352 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.352 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.352 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6809/86571", 2023-10-08T00:15:58.352 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6811/86571", 2023-10-08T00:15:58.352 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6815/86571", 2023-10-08T00:15:58.352 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6813/86571", 2023-10-08T00:15:58.352 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:15:58.352 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:15:58.352 INFO:teuthology.orchestra.run.smithi099.stdout: "full", 2023-10-08T00:15:58.352 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:15:58.353 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.353 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:15:58.353 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:15:58.353 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:15:58.353 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:15:58.353 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:15:58.353 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:15:58.353 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:15:58.353 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:15:58.353 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:15:58.354 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 102, 2023-10-08T00:15:58.354 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:15:58.354 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:15:58.354 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:15:58.354 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.354 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.354 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6817", 2023-10-08T00:15:58.354 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:15:58.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.354 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.355 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6819", 2023-10-08T00:15:58.355 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:15:58.355 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.355 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.355 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:15:58.355 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.355 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.355 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6822", 2023-10-08T00:15:58.355 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:15:58.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.356 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.356 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6824", 2023-10-08T00:15:58.356 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:15:58.356 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.356 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.356 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:15:58.356 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.356 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.356 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6830", 2023-10-08T00:15:58.356 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:15:58.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.357 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.357 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6831", 2023-10-08T00:15:58.357 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:15:58.357 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.357 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.357 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:15:58.357 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.357 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.357 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6827", 2023-10-08T00:15:58.358 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:15:58.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.358 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.358 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6829", 2023-10-08T00:15:58.358 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:15:58.358 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.358 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.358 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6819/86612", 2023-10-08T00:15:58.358 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6824/86612", 2023-10-08T00:15:58.358 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6831/86612", 2023-10-08T00:15:58.359 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6829/86612", 2023-10-08T00:15:58.359 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:15:58.359 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:15:58.359 INFO:teuthology.orchestra.run.smithi099.stdout: "full", 2023-10-08T00:15:58.359 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:15:58.359 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.359 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.359 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:15:58.359 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2023-10-08T00:15:58.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.359 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:15:58.359 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:15:58.360 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:15:58.360 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:15:58.360 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:15:58.360 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:15:58.360 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.905615+0000", 2023-10-08T00:15:58.360 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:15:58.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.360 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:15:58.360 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:15:58.360 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:15:58.360 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:15:58.360 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:15:58.361 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:15:58.361 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.880709+0000", 2023-10-08T00:15:58.361 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:15:58.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.361 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:15:58.361 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:15:58.361 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:15:58.361 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:15:58.361 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:15:58.361 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:15:58.361 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.874704+0000", 2023-10-08T00:15:58.362 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:15:58.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.362 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:15:58.362 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:15:58.362 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:15:58.362 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:15:58.362 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:15:58.362 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:15:58.362 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.899604+0000", 2023-10-08T00:15:58.362 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:15:58.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.363 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:15:58.363 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:15:58.363 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:15:58.363 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:15:58.363 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:15:58.363 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:15:58.363 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.844371+0000", 2023-10-08T00:15:58.363 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:15:58.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.363 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:15:58.364 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:15:58.364 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:15:58.364 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:15:58.364 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:15:58.364 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:15:58.364 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.794810+0000", 2023-10-08T00:15:58.364 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:15:58.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.364 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:15:58.364 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:15:58.364 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:15:58.365 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:15:58.365 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:15:58.365 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:15:58.365 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.859474+0000", 2023-10-08T00:15:58.365 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:15:58.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.365 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:15:58.365 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:15:58.365 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:15:58.365 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:15:58.365 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:15:58.366 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:15:58.366 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.893130+0000", 2023-10-08T00:15:58.366 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:15:58.366 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.366 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:15:58.366 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2023-10-08T00:15:58.366 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [], 2023-10-08T00:15:58.366 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2023-10-08T00:15:58.366 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2023-10-08T00:15:58.366 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2023-10-08T00:15:58.366 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2023-10-08T00:15:58.366 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2023-10-08T00:15:58.367 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2023-10-08T00:15:58.367 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2023-10-08T00:15:58.367 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2023-10-08T00:15:58.367 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2023-10-08T00:15:58.367 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2023-10-08T00:15:58.367 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2023-10-08T00:15:58.367 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.367 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2023-10-08T00:15:58.367 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2023-10-08T00:15:58.367 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2023-10-08T00:15:58.367 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2023-10-08T00:15:58.367 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2023-10-08T00:15:58.368 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2023-10-08T00:15:58.368 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2023-10-08T00:15:58.368 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2023-10-08T00:15:58.368 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2023-10-08T00:15:58.368 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2023-10-08T00:15:58.368 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2023-10-08T00:15:58.368 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.368 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:15:58.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.276+0000 7f7585c2e700 1 -- 172.21.15.99:0/3288141748 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f756c066f30 msgr2=0x7f756c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:58.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.276+0000 7f7585c2e700 1 --2- 172.21.15.99:0/3288141748 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f756c066f30 0x7f756c0693e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f7570004f40 tx=0x7f7570006040 comp rx=0 tx=0).stop 2023-10-08T00:15:58.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.276+0000 7f7585c2e700 1 -- 172.21.15.99:0/3288141748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7580149bc0 msgr2=0x7f758014a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:58.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.276+0000 7f7585c2e700 1 --2- 172.21.15.99:0/3288141748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7580149bc0 0x7f758014a030 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f758015d9e0 tx=0x7f7578008040 comp rx=0 tx=0).stop 2023-10-08T00:15:58.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.276+0000 7f7585c2e700 1 -- 172.21.15.99:0/3288141748 shutdown_connections 2023-10-08T00:15:58.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.276+0000 7f7585c2e700 1 --2- 172.21.15.99:0/3288141748 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f756c066f30 0x7f756c0693e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:58.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.276+0000 7f7585c2e700 1 --2- 172.21.15.99:0/3288141748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7580149bc0 0x7f758014a030 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:58.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.276+0000 7f7585c2e700 1 --2- 172.21.15.99:0/3288141748 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7568003fe0 0x7f75800b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:58.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.276+0000 7f7585c2e700 1 --2- 172.21.15.99:0/3288141748 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f75800b1470 0x7f75800b18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:58.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.276+0000 7f7585c2e700 1 -- 172.21.15.99:0/3288141748 >> 172.21.15.99:0/3288141748 conn(0x7f7580068ce0 msgr2=0x7f758014c580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:58.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.276+0000 7f7585c2e700 1 -- 172.21.15.99:0/3288141748 shutdown_connections 2023-10-08T00:15:58.371 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.276+0000 7f7585c2e700 1 -- 172.21.15.99:0/3288141748 wait complete. 2023-10-08T00:15:58.371 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-10-08T00:15:58.371 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:15:58.371 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=265) 2023-10-08T00:15:58.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T00:15:58.623 INFO:teuthology.orchestra.run.smithi099.stderr:dd: fdatasync failed for 'large_file_b': No space left on device 2023-10-08T00:15:58.623 INFO:teuthology.orchestra.run.smithi099.stderr:50+0 records in 2023-10-08T00:15:58.623 INFO:teuthology.orchestra.run.smithi099.stderr:50+0 records out 2023-10-08T00:15:58.623 INFO:teuthology.orchestra.run.smithi099.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.230509 s, 227 MB/s 2023-10-08T00:15:58.623 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-08T00:15:58.744 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.744+0000 7f5c5f813700 1 Processor -- start 2023-10-08T00:15:58.744 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.744+0000 7f5c5f813700 1 -- start start 2023-10-08T00:15:58.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.745+0000 7f5c5f813700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5c58157c40 0x7f5c58158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:58.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.745+0000 7f5c5f813700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c5815cbb0 con 0x7f5c58157870 2023-10-08T00:15:58.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.745+0000 7f5c5f813700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c5815bfa0 con 0x7f5c5815a620 2023-10-08T00:15:58.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.745+0000 7f5c5f813700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c5815bdb0 con 0x7f5c58157c40 2023-10-08T00:15:58.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.745+0000 7f5c5ddb0700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5c58157c40 0x7f5c58158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:58.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.745+0000 7f5c5d5af700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f5c58157870 0x7f5c5815b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:47628/0 (socket says 172.21.15.99:47628) 2023-10-08T00:15:58.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.745+0000 7f5c5d5af700 1 -- 172.21.15.99:0/4180426173 learned_addr learned my addr 172.21.15.99:0/4180426173 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:15:58.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.745+0000 7f5c5ddb0700 1 -- 172.21.15.99:0/4180426173 >> v1:172.21.15.191:6789/0 conn(0x7f5c5815a620 legacy=0x7f5c5815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:58.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.745+0000 7f5c5ddb0700 1 -- 172.21.15.99:0/4180426173 >> v1:172.21.15.99:6789/0 conn(0x7f5c58157870 legacy=0x7f5c5815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:58.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.745+0000 7f5c5ddb0700 1 -- 172.21.15.99:0/4180426173 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c5815c280 con 0x7f5c58157c40 2023-10-08T00:15:58.748 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.745+0000 7f5c4ffff700 1 -- 172.21.15.99:0/4180426173 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2345603905 0 0) 0x7f5c5815cbb0 con 0x7f5c58157870 2023-10-08T00:15:58.748 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.745+0000 7f5c5ddb0700 1 --2- 172.21.15.99:0/4180426173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c58157c40 0x7f5c58158110 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f5c500098e0 tx=0x7f5c500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e8ca513cd225b5d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:58.748 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.745+0000 7f5c4ffff700 1 -- 172.21.15.99:0/4180426173 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c50009e80 con 0x7f5c58157c40 2023-10-08T00:15:58.748 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.745+0000 7f5c4ffff700 1 -- 172.21.15.99:0/4180426173 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c5000b860 con 0x7f5c58157c40 2023-10-08T00:15:58.748 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.745+0000 7f5c4ffff700 1 -- 172.21.15.99:0/4180426173 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5000bbe0 con 0x7f5c58157c40 2023-10-08T00:15:58.748 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.746+0000 7f5c5f813700 1 -- 172.21.15.99:0/4180426173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c58157c40 msgr2=0x7f5c58158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:58.748 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.746+0000 7f5c5f813700 1 --2- 172.21.15.99:0/4180426173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c58157c40 0x7f5c58158110 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f5c500098e0 tx=0x7f5c500091a0 comp rx=0 tx=0).stop 2023-10-08T00:15:58.748 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.746+0000 7f5c5f813700 1 -- 172.21.15.99:0/4180426173 shutdown_connections 2023-10-08T00:15:58.748 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.746+0000 7f5c5f813700 1 --2- 172.21.15.99:0/4180426173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c58157c40 0x7f5c58158110 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:58.749 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.746+0000 7f5c5f813700 1 -- 172.21.15.99:0/4180426173 >> 172.21.15.99:0/4180426173 conn(0x7f5c58068ce0 msgr2=0x7f5c580690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:58.749 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.746+0000 7f5c5f813700 1 -- 172.21.15.99:0/4180426173 shutdown_connections 2023-10-08T00:15:58.749 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.746+0000 7f5c5f813700 1 -- 172.21.15.99:0/4180426173 wait complete. 2023-10-08T00:15:58.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.747+0000 7f5c5f813700 1 Processor -- start 2023-10-08T00:15:58.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.747+0000 7f5c5f813700 1 -- start start 2023-10-08T00:15:58.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.748+0000 7f5c5f813700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c58157870 0x7f5c58138e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:58.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.748+0000 7f5c5f813700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c58157c40 0x7f5c58150f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:58.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.748+0000 7f5c5f813700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c5815a620 0x7f5c58151480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:58.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.748+0000 7f5c5f813700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c580b3530 con 0x7f5c5815a620 2023-10-08T00:15:58.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.748+0000 7f5c5f813700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c580b3250 con 0x7f5c58157c40 2023-10-08T00:15:58.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.748+0000 7f5c5f813700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5814e840 con 0x7f5c58157870 2023-10-08T00:15:58.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.748+0000 7f5c5ddb0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c5815a620 0x7f5c58151480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:58.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.748+0000 7f5c5ddb0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c5815a620 0x7f5c58151480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58774/0 (socket says 172.21.15.99:58774) 2023-10-08T00:15:58.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.748+0000 7f5c5ddb0700 1 -- 172.21.15.99:0/204867262 learned_addr learned my addr 172.21.15.99:0/204867262 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:58.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.748+0000 7f5c5cdae700 1 --2- 172.21.15.99:0/204867262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c58157c40 0x7f5c58150f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:58.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.748+0000 7f5c5ddb0700 1 -- 172.21.15.99:0/204867262 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c58157870 msgr2=0x7f5c58138e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:58.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.748+0000 7f5c5d5af700 1 --2- 172.21.15.99:0/204867262 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c58157870 0x7f5c58138e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:58.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.748+0000 7f5c5ddb0700 1 --2- 172.21.15.99:0/204867262 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c58157870 0x7f5c58138e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:58.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.748+0000 7f5c5ddb0700 1 -- 172.21.15.99:0/204867262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c58157c40 msgr2=0x7f5c58150f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:58.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.748+0000 7f5c5ddb0700 1 --2- 172.21.15.99:0/204867262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c58157c40 0x7f5c58150f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:58.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.748+0000 7f5c5ddb0700 1 -- 172.21.15.99:0/204867262 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c50009420 con 0x7f5c5815a620 2023-10-08T00:15:58.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.748+0000 7f5c5cdae700 1 --2- 172.21.15.99:0/204867262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c58157c40 0x7f5c58150f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:15:58.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.748+0000 7f5c5d5af700 1 --2- 172.21.15.99:0/204867262 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c58157870 0x7f5c58138e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:15:58.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.748+0000 7f5c5ddb0700 1 --2- 172.21.15.99:0/204867262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c5815a620 0x7f5c58151480 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f5c50013000 tx=0x7f5c50017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:58.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.749+0000 7f5c4d7fa700 1 -- 172.21.15.99:0/204867262 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5001f070 con 0x7f5c5815a620 2023-10-08T00:15:58.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.749+0000 7f5c4d7fa700 1 -- 172.21.15.99:0/204867262 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c50011420 con 0x7f5c5815a620 2023-10-08T00:15:58.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.749+0000 7f5c4d7fa700 1 -- 172.21.15.99:0/204867262 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c500117e0 con 0x7f5c5815a620 2023-10-08T00:15:58.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.749+0000 7f5c5f813700 1 -- 172.21.15.99:0/204867262 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c5814eb20 con 0x7f5c5815a620 2023-10-08T00:15:58.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.749+0000 7f5c5f813700 1 -- 172.21.15.99:0/204867262 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c58139380 con 0x7f5c5815a620 2023-10-08T00:15:58.759 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.750+0000 7f5c5f813700 1 -- 172.21.15.99:0/204867262 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c58139570 con 0x7f5c5815a620 2023-10-08T00:15:58.760 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.750+0000 7f5c4d7fa700 1 -- 172.21.15.99:0/204867262 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5c50011b50 con 0x7f5c5815a620 2023-10-08T00:15:58.760 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.750+0000 7f5c4d7fa700 1 --2- 172.21.15.99:0/204867262 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5c3c066f30 0x7f5c3c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:58.760 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.750+0000 7f5c4d7fa700 1 -- 172.21.15.99:0/204867262 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5c50085cb0 con 0x7f5c5815a620 2023-10-08T00:15:58.760 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.751+0000 7f5c4d7fa700 1 -- 172.21.15.99:0/204867262 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=109}) v3 -- 0x7f5c3c06ea50 con 0x7f5c5815a620 2023-10-08T00:15:58.760 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.751+0000 7f5c5d5af700 1 --2- 172.21.15.99:0/204867262 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5c3c066f30 0x7f5c3c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:58.760 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.751+0000 7f5c5d5af700 1 --2- 172.21.15.99:0/204867262 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5c3c066f30 0x7f5c3c0693e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5c54000f80 tx=0x7f5c54009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:58.761 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.754+0000 7f5c4d7fa700 1 -- 172.21.15.99:0/204867262 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5c50052f20 con 0x7f5c5815a620 2023-10-08T00:15:58.939 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.938+0000 7f5c5f813700 1 -- 172.21.15.99:0/204867262 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f5c5804c3c0 con 0x7f5c5815a620 2023-10-08T00:15:58.940 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.939+0000 7f5c4d7fa700 1 -- 172.21.15.99:0/204867262 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v108) v1 ==== 81+0+32458 (secure 0 0 0) 0x7f5c50028070 con 0x7f5c5815a620 2023-10-08T00:15:58.940 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:15:58.940 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:15:58.940 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 108, 2023-10-08T00:15:58.941 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", 2023-10-08T00:15:58.941 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2023-10-08T00:10:36.303601+0000", 2023-10-08T00:15:58.941 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2023-10-08T00:15:57.327150+0000", 2023-10-08T00:15:58.941 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2023-10-08T00:10:41.709171+0000", 2023-10-08T00:15:58.941 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2023-10-08T00:10:38.690404+0000", 2023-10-08T00:15:58.941 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-08T00:15:58.941 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2023-10-08T00:15:58.941 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2023-10-08T00:15:58.941 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2023-10-08T00:15:58.941 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2023-10-08T00:15:58.942 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2023-10-08T00:15:58.942 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2023-10-08T00:15:58.942 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:15:58.942 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2023-10-08T00:15:58.942 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2023-10-08T00:15:58.942 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-08T00:15:58.942 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-08T00:15:58.942 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2023-10-08T00:15:58.942 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 13, 2023-10-08T00:15:58.942 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2023-10-08T00:15:58.942 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2023-10-08T00:15:58.943 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "jewel", 2023-10-08T00:15:58.943 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "quincy", 2023-10-08T00:15:58.943 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2023-10-08T00:15:58.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.943 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2023-10-08T00:15:58.943 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2023-10-08T00:15:58.943 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:10:42.063344+0000", 2023-10-08T00:15:58.943 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 3, 2023-10-08T00:15:58.944 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full", 2023-10-08T00:15:58.944 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:15:58.944 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:15:58.944 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:15:58.944 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:15:58.944 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:15:58.944 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:15:58.944 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:15:58.944 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:15:58.944 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:15:58.944 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:15:58.944 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2023-10-08T00:15:58.945 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2023-10-08T00:15:58.945 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2023-10-08T00:15:58.945 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2023-10-08T00:15:58.945 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2023-10-08T00:15:58.945 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:15:58.945 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:15:58.945 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:15:58.945 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:15:58.945 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:15:58.945 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:15:58.945 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:15:58.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.946 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "105", 2023-10-08T00:15:58.946 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:15:58.946 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:15:58.946 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:15:58.946 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:15:58.946 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:15:58.946 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:15:58.946 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:15:58.946 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:15:58.946 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:15:58.946 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:15:58.947 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:15:58.947 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:15:58.947 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:15:58.947 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:15:58.947 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:15:58.947 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:15:58.947 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:15:58.947 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:15:58.947 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:15:58.947 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:15:58.947 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:15:58.948 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:15:58.948 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:15:58.948 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:15:58.948 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:15:58.948 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:15:58.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.948 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:15:58.948 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:15:58.948 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:15:58.948 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:15:58.948 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:15:58.948 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:15:58.949 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:15:58.949 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:15:58.949 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:15:58.949 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:15:58.949 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:15:58.949 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:15:58.949 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2023-10-08T00:15:58.949 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2023-10-08T00:15:58.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.949 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:15:58.949 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2023-10-08T00:15:58.950 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.950 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 12, 2023-10-08T00:15:58.950 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2023-10-08T00:15:58.950 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:15:38.848324+0000", 2023-10-08T00:15:58.950 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 3, 2023-10-08T00:15:58.950 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full", 2023-10-08T00:15:58.950 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:15:58.951 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:15:58.951 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:15:58.951 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:15:58.951 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:15:58.951 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:15:58.951 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:15:58.951 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:15:58.951 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:15:58.951 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:15:58.951 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2023-10-08T00:15:58.952 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2023-10-08T00:15:58.952 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2023-10-08T00:15:58.952 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2023-10-08T00:15:58.952 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2023-10-08T00:15:58.952 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:15:58.952 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:15:58.952 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:15:58.952 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:15:58.952 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:15:58.952 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:15:58.952 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:15:58.953 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.953 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "105", 2023-10-08T00:15:58.953 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:15:58.953 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:15:58.953 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:15:58.953 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:15:58.953 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:15:58.953 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:15:58.953 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:15:58.953 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:15:58.954 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:15:58.954 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:15:58.954 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:15:58.954 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:15:58.954 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:15:58.954 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:15:58.954 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:15:58.954 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:15:58.954 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:15:58.954 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:15:58.954 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:15:58.954 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:15:58.955 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:15:58.955 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:15:58.955 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:15:58.955 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:15:58.955 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:15:58.955 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:15:58.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.955 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:15:58.955 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:15:58.955 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:15:58.955 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:15:58.956 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:15:58.956 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:15:58.956 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:15:58.956 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:15:58.956 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:15:58.956 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:15:58.956 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:15:58.956 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:15:58.956 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2023-10-08T00:15:58.956 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2023-10-08T00:15:58.956 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2023-10-08T00:15:58.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.957 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:15:58.957 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:15:58.957 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2023-10-08T00:15:58.957 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.957 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.957 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 13, 2023-10-08T00:15:58.957 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2023-10-08T00:15:58.957 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:15:39.561379+0000", 2023-10-08T00:15:58.957 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 3, 2023-10-08T00:15:58.958 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full", 2023-10-08T00:15:58.958 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:15:58.958 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:15:58.958 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:15:58.958 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:15:58.958 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:15:58.958 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:15:58.958 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:15:58.958 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:15:58.958 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:15:58.958 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:15:58.959 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2023-10-08T00:15:58.959 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2023-10-08T00:15:58.959 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2023-10-08T00:15:58.959 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2023-10-08T00:15:58.959 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2023-10-08T00:15:58.959 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:15:58.959 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:15:58.959 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:15:58.959 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:15:58.959 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:15:58.960 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:15:58.960 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:15:58.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.960 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "105", 2023-10-08T00:15:58.960 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:15:58.960 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:15:58.960 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:15:58.960 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:15:58.960 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:15:58.960 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:15:58.960 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:15:58.960 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:15:58.961 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:15:58.961 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:15:58.961 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:15:58.961 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:15:58.961 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:15:58.961 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:15:58.961 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:15:58.961 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:15:58.961 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:15:58.961 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:15:58.961 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:15:58.961 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:15:58.962 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:15:58.962 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:15:58.962 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:15:58.962 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:15:58.962 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:15:58.962 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:15:58.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.962 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:15:58.963 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:15:58.963 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:15:58.963 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:15:58.963 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:15:58.963 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:15:58.963 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:15:58.963 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:15:58.963 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:15:58.963 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:15:58.963 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:15:58.963 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:15:58.963 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2023-10-08T00:15:58.964 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2023-10-08T00:15:58.964 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.964 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:15:58.964 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:15:58.964 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2023-10-08T00:15:58.964 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.964 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.964 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.964 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:15:58.965 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2023-10-08T00:15:58.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.965 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:15:58.965 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:15:58.965 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:15:58.965 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:15:58.965 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:15:58.965 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:15:58.965 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:15:58.965 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:15:58.965 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:15:58.966 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 102, 2023-10-08T00:15:58.966 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:15:58.966 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:15:58.966 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:15:58.966 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.966 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.966 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2023-10-08T00:15:58.966 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:15:58.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.967 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.967 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2023-10-08T00:15:58.967 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:15:58.967 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.967 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.967 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:15:58.967 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.967 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.967 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.967 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2023-10-08T00:15:58.967 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:15:58.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.968 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.968 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2023-10-08T00:15:58.968 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:15:58.968 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.968 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.968 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:15:58.968 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.969 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.969 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2023-10-08T00:15:58.969 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:15:58.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.969 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.969 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.969 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2023-10-08T00:15:58.969 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:15:58.969 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.969 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.969 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:15:58.970 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.970 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.970 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.970 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2023-10-08T00:15:58.970 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:15:58.970 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.970 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.970 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.970 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2023-10-08T00:15:58.970 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:15:58.970 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.971 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.971 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/86365", 2023-10-08T00:15:58.971 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/86365", 2023-10-08T00:15:58.971 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/86365", 2023-10-08T00:15:58.971 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/86365", 2023-10-08T00:15:58.971 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:15:58.971 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull", 2023-10-08T00:15:58.971 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:15:58.971 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:15:58.971 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.972 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:15:58.972 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:15:58.972 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:15:58.972 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:15:58.972 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:15:58.972 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:15:58.972 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:15:58.972 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:15:58.972 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:15:58.973 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 102, 2023-10-08T00:15:58.973 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:15:58.973 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:15:58.973 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:15:58.973 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.973 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.973 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2023-10-08T00:15:58.973 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:15:58.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.974 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.974 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2023-10-08T00:15:58.974 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:15:58.974 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.974 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.974 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:15:58.974 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.974 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.974 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.975 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2023-10-08T00:15:58.975 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:15:58.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.975 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.975 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.975 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2023-10-08T00:15:58.975 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:15:58.975 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.975 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.975 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:15:58.975 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.976 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.976 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2023-10-08T00:15:58.976 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:15:58.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.976 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.976 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2023-10-08T00:15:58.976 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:15:58.976 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.976 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.977 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:15:58.977 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.977 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.977 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2023-10-08T00:15:58.977 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:15:58.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.977 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.978 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2023-10-08T00:15:58.978 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:15:58.978 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.978 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.978 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/86366", 2023-10-08T00:15:58.978 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/86366", 2023-10-08T00:15:58.978 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/86366", 2023-10-08T00:15:58.978 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/86366", 2023-10-08T00:15:58.978 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:15:58.978 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull", 2023-10-08T00:15:58.979 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:15:58.979 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:15:58.979 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.979 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:15:58.979 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:15:58.979 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:15:58.979 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:15:58.979 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:15:58.979 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:15:58.979 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:15:58.980 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:15:58.980 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:15:58.980 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 102, 2023-10-08T00:15:58.980 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:15:58.980 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:15:58.980 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:15:58.980 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.980 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.980 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2023-10-08T00:15:58.980 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:15:58.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.981 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.981 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2023-10-08T00:15:58.981 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:15:58.981 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.981 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.981 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:15:58.981 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.982 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.982 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2023-10-08T00:15:58.982 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:15:58.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.982 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.982 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2023-10-08T00:15:58.982 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:15:58.982 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.982 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.983 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:15:58.983 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.983 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.983 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2023-10-08T00:15:58.983 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:15:58.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.983 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.983 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2023-10-08T00:15:58.984 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:15:58.984 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.984 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.984 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:15:58.984 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.984 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.984 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2023-10-08T00:15:58.984 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:15:58.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.985 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.985 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2023-10-08T00:15:58.985 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:15:58.985 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.985 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.985 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6817/86382", 2023-10-08T00:15:58.985 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/86382", 2023-10-08T00:15:58.985 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/86382", 2023-10-08T00:15:58.985 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/86382", 2023-10-08T00:15:58.985 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:15:58.986 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull", 2023-10-08T00:15:58.986 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:15:58.986 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:15:58.986 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.986 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:15:58.986 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:15:58.986 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:15:58.986 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:15:58.986 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:15:58.987 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:15:58.987 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:15:58.987 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:15:58.987 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:15:58.987 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 102, 2023-10-08T00:15:58.987 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:15:58.987 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:15:58.987 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:15:58.987 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.987 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.987 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2023-10-08T00:15:58.988 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:15:58.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.988 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.988 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2023-10-08T00:15:58.988 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:15:58.988 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.988 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.988 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:15:58.988 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.989 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.989 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2023-10-08T00:15:58.989 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:15:58.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.989 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.989 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2023-10-08T00:15:58.989 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:15:58.989 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.989 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.990 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:15:58.990 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.990 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.990 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.990 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2023-10-08T00:15:58.990 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:15:58.990 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.990 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.990 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.990 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2023-10-08T00:15:58.990 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:15:58.991 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.991 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.991 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:15:58.991 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.991 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.991 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.991 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2023-10-08T00:15:58.991 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:15:58.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.991 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.992 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.992 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2023-10-08T00:15:58.992 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:15:58.992 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.992 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.992 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6820/86393", 2023-10-08T00:15:58.992 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/86393", 2023-10-08T00:15:58.992 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/86393", 2023-10-08T00:15:58.992 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/86393", 2023-10-08T00:15:58.992 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:15:58.992 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:15:58.993 INFO:teuthology.orchestra.run.smithi099.stdout: "full", 2023-10-08T00:15:58.993 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:15:58.993 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.993 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:15:58.993 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:15:58.993 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:15:58.993 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:15:58.993 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:15:58.993 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:15:58.994 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:15:58.994 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:15:58.994 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:15:58.994 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 102, 2023-10-08T00:15:58.994 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:15:58.994 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:15:58.994 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:15:58.994 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.994 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.994 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6800", 2023-10-08T00:15:58.995 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:15:58.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.995 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.995 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.995 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6801", 2023-10-08T00:15:58.995 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:15:58.995 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.995 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.995 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:15:58.996 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.996 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.996 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.996 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6802", 2023-10-08T00:15:58.996 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:15:58.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.996 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.996 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.996 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6803", 2023-10-08T00:15:58.996 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:15:58.996 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.996 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.997 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:15:58.997 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.997 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.997 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6806", 2023-10-08T00:15:58.997 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:15:58.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.997 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.997 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6807", 2023-10-08T00:15:58.997 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:15:58.998 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.998 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.998 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:15:58.998 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:58.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.998 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:58.998 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6804", 2023-10-08T00:15:58.998 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:15:58.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:58.999 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:58.999 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6805", 2023-10-08T00:15:58.999 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:15:58.999 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:58.999 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:58.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:58.999 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6801/86570", 2023-10-08T00:15:58.999 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6803/86570", 2023-10-08T00:15:58.999 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6807/86570", 2023-10-08T00:15:58.999 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6805/86570", 2023-10-08T00:15:58.999 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:15:58.999 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:15:59.000 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:15:59.000 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:59.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:59.000 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:59.000 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:15:59.000 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:15:59.000 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:15:59.000 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:15:59.000 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:15:59.000 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:15:59.000 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:15:59.001 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:15:59.001 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:15:59.001 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 102, 2023-10-08T00:15:59.001 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:15:59.001 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:15:59.001 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:15:59.001 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:59.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:59.001 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:59.001 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6816", 2023-10-08T00:15:59.001 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:15:59.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:59.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:59.002 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:59.002 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6818", 2023-10-08T00:15:59.002 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:15:59.002 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:59.002 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:59.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:59.002 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:15:59.002 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:59.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:59.003 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:59.003 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6820", 2023-10-08T00:15:59.003 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:15:59.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:59.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:59.003 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:59.003 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6821", 2023-10-08T00:15:59.003 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:15:59.003 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:59.003 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:59.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:59.003 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:15:59.004 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:59.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:59.004 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:59.004 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6826", 2023-10-08T00:15:59.004 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:15:59.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:59.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:59.004 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:59.004 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6828", 2023-10-08T00:15:59.004 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:15:59.004 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:59.004 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:59.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:59.005 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:15:59.005 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:59.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:59.005 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:59.005 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6823", 2023-10-08T00:15:59.005 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:15:59.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:59.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:59.005 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:59.005 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6825", 2023-10-08T00:15:59.006 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:15:59.006 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:59.006 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:59.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:59.006 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6818/86608", 2023-10-08T00:15:59.006 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6821/86608", 2023-10-08T00:15:59.006 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6828/86608", 2023-10-08T00:15:59.006 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6825/86608", 2023-10-08T00:15:59.006 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:15:59.006 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:15:59.006 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:15:59.006 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:59.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:59.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:59.007 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:15:59.007 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:15:59.007 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:15:59.007 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:15:59.007 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:15:59.007 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:15:59.007 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:15:59.007 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:15:59.007 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:15:59.008 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 102, 2023-10-08T00:15:59.008 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:15:59.008 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:15:59.008 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:15:59.008 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:59.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:59.008 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:59.008 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6808", 2023-10-08T00:15:59.008 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:15:59.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:59.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:59.009 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:59.009 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6809", 2023-10-08T00:15:59.009 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:15:59.009 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:59.009 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:59.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:59.009 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:15:59.009 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:59.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:59.009 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:59.009 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6810", 2023-10-08T00:15:59.010 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:15:59.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:59.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:59.010 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:59.010 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6811", 2023-10-08T00:15:59.010 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:15:59.010 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:59.010 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:59.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:59.010 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:15:59.010 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:59.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:59.011 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:59.011 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6814", 2023-10-08T00:15:59.011 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:15:59.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:59.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:59.011 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:59.011 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6815", 2023-10-08T00:15:59.011 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:15:59.011 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:59.011 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:59.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:59.012 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:15:59.012 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:59.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:59.012 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:59.012 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6812", 2023-10-08T00:15:59.012 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:15:59.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:59.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:59.012 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:59.012 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6813", 2023-10-08T00:15:59.012 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:15:59.012 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:59.013 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:59.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:59.013 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6809/86571", 2023-10-08T00:15:59.013 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6811/86571", 2023-10-08T00:15:59.013 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6815/86571", 2023-10-08T00:15:59.013 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6813/86571", 2023-10-08T00:15:59.013 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:15:59.013 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:15:59.013 INFO:teuthology.orchestra.run.smithi099.stdout: "full", 2023-10-08T00:15:59.013 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:15:59.013 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:59.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:59.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:59.014 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:15:59.014 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:15:59.014 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:15:59.014 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:15:59.014 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:15:59.014 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:15:59.014 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:15:59.014 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:15:59.014 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:15:59.014 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 102, 2023-10-08T00:15:59.015 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:15:59.015 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:15:59.015 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:15:59.015 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:59.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:59.015 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:59.015 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6817", 2023-10-08T00:15:59.015 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:15:59.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:59.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:59.015 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:59.016 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6819", 2023-10-08T00:15:59.016 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:15:59.016 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:59.016 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:59.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:59.016 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:15:59.016 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:59.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:59.016 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:59.016 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6822", 2023-10-08T00:15:59.016 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:15:59.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:59.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:59.017 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:59.017 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6824", 2023-10-08T00:15:59.017 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:15:59.017 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:59.017 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:59.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:59.017 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:15:59.017 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:59.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:59.018 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:59.018 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6830", 2023-10-08T00:15:59.018 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:15:59.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:59.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:59.018 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:59.018 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6831", 2023-10-08T00:15:59.018 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:15:59.018 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:59.018 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:59.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:59.019 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:15:59.019 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:15:59.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:59.019 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:15:59.019 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6827", 2023-10-08T00:15:59.019 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:15:59.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:59.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:59.019 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:15:59.020 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6829", 2023-10-08T00:15:59.020 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:15:59.020 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:59.020 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:59.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:59.020 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6819/86612", 2023-10-08T00:15:59.020 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6824/86612", 2023-10-08T00:15:59.020 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6831/86612", 2023-10-08T00:15:59.020 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6829/86612", 2023-10-08T00:15:59.020 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:15:59.020 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:15:59.021 INFO:teuthology.orchestra.run.smithi099.stdout: "full", 2023-10-08T00:15:59.021 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:15:59.021 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:15:59.021 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:59.021 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:15:59.021 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2023-10-08T00:15:59.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:59.021 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:15:59.022 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:15:59.022 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:15:59.022 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:15:59.022 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:15:59.022 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:15:59.022 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.905615+0000", 2023-10-08T00:15:59.022 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:15:59.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:59.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:59.023 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:15:59.023 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:15:59.023 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:15:59.023 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:15:59.023 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:15:59.023 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:15:59.023 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.880709+0000", 2023-10-08T00:15:59.023 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:15:59.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:59.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:59.024 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:15:59.024 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:15:59.024 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:15:59.024 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:15:59.024 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:15:59.024 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:15:59.024 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.874704+0000", 2023-10-08T00:15:59.024 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:15:59.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:59.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:59.025 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:15:59.025 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:15:59.025 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:15:59.025 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:15:59.025 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:15:59.025 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:15:59.025 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.899604+0000", 2023-10-08T00:15:59.025 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:15:59.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:59.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:59.026 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:15:59.026 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:15:59.026 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:15:59.026 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:15:59.026 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:15:59.026 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:15:59.026 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.844371+0000", 2023-10-08T00:15:59.026 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:15:59.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:59.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:59.026 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:15:59.027 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:15:59.027 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:15:59.027 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:15:59.027 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:15:59.027 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:15:59.027 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.794810+0000", 2023-10-08T00:15:59.027 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:15:59.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:59.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:59.027 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:15:59.027 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:15:59.027 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:15:59.028 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:15:59.028 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:15:59.028 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:15:59.028 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.859474+0000", 2023-10-08T00:15:59.028 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:15:59.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:59.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:15:59.028 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:15:59.028 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:15:59.028 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:15:59.028 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:15:59.028 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:15:59.029 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:15:59.029 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.893130+0000", 2023-10-08T00:15:59.029 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:15:59.029 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:59.029 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:15:59.029 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2023-10-08T00:15:59.029 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [], 2023-10-08T00:15:59.029 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2023-10-08T00:15:59.029 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2023-10-08T00:15:59.029 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2023-10-08T00:15:59.029 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2023-10-08T00:15:59.029 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2023-10-08T00:15:59.030 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2023-10-08T00:15:59.030 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2023-10-08T00:15:59.030 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2023-10-08T00:15:59.030 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2023-10-08T00:15:59.030 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2023-10-08T00:15:59.030 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2023-10-08T00:15:59.030 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:59.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:15:59.030 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2023-10-08T00:15:59.030 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2023-10-08T00:15:59.030 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2023-10-08T00:15:59.031 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2023-10-08T00:15:59.031 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2023-10-08T00:15:59.031 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2023-10-08T00:15:59.031 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2023-10-08T00:15:59.031 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2023-10-08T00:15:59.031 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2023-10-08T00:15:59.031 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2023-10-08T00:15:59.031 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2023-10-08T00:15:59.031 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:15:59.031 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:15:59.033 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.941+0000 7f5c5f813700 1 -- 172.21.15.99:0/204867262 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5c3c066f30 msgr2=0x7f5c3c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:59.033 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.941+0000 7f5c5f813700 1 --2- 172.21.15.99:0/204867262 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5c3c066f30 0x7f5c3c0693e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5c54000f80 tx=0x7f5c54009040 comp rx=0 tx=0).stop 2023-10-08T00:15:59.033 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.941+0000 7f5c5f813700 1 -- 172.21.15.99:0/204867262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c5815a620 msgr2=0x7f5c58151480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:59.033 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.941+0000 7f5c5f813700 1 --2- 172.21.15.99:0/204867262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c5815a620 0x7f5c58151480 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f5c50013000 tx=0x7f5c50017000 comp rx=0 tx=0).stop 2023-10-08T00:15:59.033 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.941+0000 7f5c5f813700 1 -- 172.21.15.99:0/204867262 shutdown_connections 2023-10-08T00:15:59.033 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.941+0000 7f5c5f813700 1 --2- 172.21.15.99:0/204867262 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5c3c066f30 0x7f5c3c0693e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:59.033 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.941+0000 7f5c5f813700 1 --2- 172.21.15.99:0/204867262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c5815a620 0x7f5c58151480 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:59.033 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.941+0000 7f5c5f813700 1 --2- 172.21.15.99:0/204867262 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c58157870 0x7f5c58138e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:59.033 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.941+0000 7f5c5f813700 1 --2- 172.21.15.99:0/204867262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c58157c40 0x7f5c58150f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:59.033 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.941+0000 7f5c5f813700 1 -- 172.21.15.99:0/204867262 >> 172.21.15.99:0/204867262 conn(0x7f5c58068ce0 msgr2=0x7f5c580690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:59.034 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.941+0000 7f5c5f813700 1 -- 172.21.15.99:0/204867262 shutdown_connections 2023-10-08T00:15:59.034 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:58.941+0000 7f5c5f813700 1 -- 172.21.15.99:0/204867262 wait complete. 2023-10-08T00:15:59.034 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:15:59.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.147+0000 7f804a055700 1 Processor -- start 2023-10-08T00:15:59.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.147+0000 7f804a055700 1 -- start start 2023-10-08T00:15:59.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.148+0000 7f804a055700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f804415a620 0x7f804415d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:59.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.148+0000 7f804a055700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f804415d580 0x7f8044158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:59.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.148+0000 7f804a055700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f804415cbb0 con 0x7f8044157870 2023-10-08T00:15:59.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.148+0000 7f804a055700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f804415bdb0 con 0x7f804415a620 2023-10-08T00:15:59.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.148+0000 7f804a055700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f804415bfa0 con 0x7f804415d580 2023-10-08T00:15:59.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.148+0000 7f8043fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f804415d580 0x7f8044158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:59.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.148+0000 7f8042ffd700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f804415a620 0x7f804415d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:59.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.148+0000 7f80437fe700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f8044157870 0x7f804415b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:36252/0 (socket says 172.21.15.99:36252) 2023-10-08T00:15:59.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.148+0000 7f8043fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f804415d580 0x7f8044158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.99:47876/0 (socket says 172.21.15.99:47876) 2023-10-08T00:15:59.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.148+0000 7f80437fe700 1 -- 172.21.15.99:0/596871716 learned_addr learned my addr 172.21.15.99:0/596871716 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:15:59.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.148+0000 7f8042ffd700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f804415a620 0x7f804415d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.99:36008/0 (socket says 172.21.15.99:36008) 2023-10-08T00:15:59.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.149+0000 7f8043fff700 1 -- 172.21.15.99:0/596871716 >> v2:172.21.15.191:3300/0 conn(0x7f804415a620 msgr2=0x7f804415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:59.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.149+0000 7f8043fff700 1 --2- 172.21.15.99:0/596871716 >> v2:172.21.15.191:3300/0 conn(0x7f804415a620 0x7f804415d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:59.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.149+0000 7f8043fff700 1 -- 172.21.15.99:0/596871716 >> v1:172.21.15.191:6789/0 conn(0x7f8044157870 legacy=0x7f804415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:59.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.149+0000 7f8043fff700 1 -- 172.21.15.99:0/596871716 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f804415c280 con 0x7f804415d580 2023-10-08T00:15:59.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.149+0000 7f80427fc700 1 -- 172.21.15.99:0/596871716 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1339465205 0 0) 0x7f804415cbb0 con 0x7f8044157870 2023-10-08T00:15:59.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.149+0000 7f8042ffd700 1 --2- 172.21.15.99:0/596871716 >> v2:172.21.15.191:3300/0 conn(0x7f804415a620 0x7f804415d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:15:59.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.149+0000 7f8043fff700 1 --2- 172.21.15.99:0/596871716 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f804415d580 0x7f8044158110 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f803c0098e0 tx=0x7f803c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b208c39ff2a6300f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:59.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.149+0000 7f80427fc700 1 -- 172.21.15.99:0/596871716 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803c00b640 con 0x7f804415d580 2023-10-08T00:15:59.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.149+0000 7f80427fc700 1 -- 172.21.15.99:0/596871716 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803c00b7a0 con 0x7f804415d580 2023-10-08T00:15:59.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.149+0000 7f80427fc700 1 -- 172.21.15.99:0/596871716 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803c00bb40 con 0x7f804415d580 2023-10-08T00:15:59.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.150+0000 7f804a055700 1 -- 172.21.15.99:0/596871716 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f804415d580 msgr2=0x7f8044158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:59.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.150+0000 7f804a055700 1 --2- 172.21.15.99:0/596871716 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f804415d580 0x7f8044158110 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f803c0098e0 tx=0x7f803c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:15:59.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.150+0000 7f804a055700 1 -- 172.21.15.99:0/596871716 shutdown_connections 2023-10-08T00:15:59.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.150+0000 7f804a055700 1 --2- 172.21.15.99:0/596871716 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f804415d580 0x7f8044158110 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:59.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.150+0000 7f804a055700 1 --2- 172.21.15.99:0/596871716 >> v2:172.21.15.191:3300/0 conn(0x7f804415a620 0x7f804415d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:59.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.150+0000 7f804a055700 1 -- 172.21.15.99:0/596871716 >> 172.21.15.99:0/596871716 conn(0x7f8044068ce0 msgr2=0x7f80440690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:59.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.150+0000 7f804a055700 1 -- 172.21.15.99:0/596871716 shutdown_connections 2023-10-08T00:15:59.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.150+0000 7f804a055700 1 -- 172.21.15.99:0/596871716 wait complete. 2023-10-08T00:15:59.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.151+0000 7f804a055700 1 Processor -- start 2023-10-08T00:15:59.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.151+0000 7f804a055700 1 -- start start 2023-10-08T00:15:59.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.152+0000 7f804a055700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8044157870 0x7f80441530d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:59.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.152+0000 7f804a055700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f804415a620 0x7f80441494d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:59.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.152+0000 7f804a055700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f804415d580 0x7f80440b10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:59.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.152+0000 7f804a055700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80440b19d0 con 0x7f8044157870 2023-10-08T00:15:59.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.152+0000 7f804a055700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8044158fa0 con 0x7f804415a620 2023-10-08T00:15:59.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.152+0000 7f804a055700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80441586b0 con 0x7f804415d580 2023-10-08T00:15:59.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.152+0000 7f80437fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8044157870 0x7f80441530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:59.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.152+0000 7f80437fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8044157870 0x7f80441530d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58780/0 (socket says 172.21.15.99:58780) 2023-10-08T00:15:59.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.152+0000 7f80437fe700 1 -- 172.21.15.99:0/975844876 learned_addr learned my addr 172.21.15.99:0/975844876 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:59.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.152+0000 7f8042ffd700 1 --2- 172.21.15.99:0/975844876 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f804415a620 0x7f80441494d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:59.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.152+0000 7f8043fff700 1 --2- 172.21.15.99:0/975844876 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f804415d580 0x7f80440b10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:59.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.152+0000 7f80437fe700 1 -- 172.21.15.99:0/975844876 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f804415d580 msgr2=0x7f80440b10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:59.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.152+0000 7f80437fe700 1 --2- 172.21.15.99:0/975844876 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f804415d580 0x7f80440b10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:59.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.152+0000 7f80437fe700 1 -- 172.21.15.99:0/975844876 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f804415a620 msgr2=0x7f80441494d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:59.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.152+0000 7f80437fe700 1 --2- 172.21.15.99:0/975844876 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f804415a620 0x7f80441494d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:59.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.152+0000 7f80437fe700 1 -- 172.21.15.99:0/975844876 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8038009420 con 0x7f8044157870 2023-10-08T00:15:59.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.152+0000 7f80437fe700 1 --2- 172.21.15.99:0/975844876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8044157870 0x7f80441530d0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f803400cd80 tx=0x7f8034001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:59.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.152+0000 7f802b7fe700 1 -- 172.21.15.99:0/975844876 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803400ab10 con 0x7f8044157870 2023-10-08T00:15:59.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.152+0000 7f802b7fe700 1 -- 172.21.15.99:0/975844876 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803400ac70 con 0x7f8044157870 2023-10-08T00:15:59.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.152+0000 7f802b7fe700 1 -- 172.21.15.99:0/975844876 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80340070f0 con 0x7f8044157870 2023-10-08T00:15:59.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.152+0000 7f804a055700 1 -- 172.21.15.99:0/975844876 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f803c009420 con 0x7f8044157870 2023-10-08T00:15:59.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.152+0000 7f804a055700 1 -- 172.21.15.99:0/975844876 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8044158cb0 con 0x7f8044157870 2023-10-08T00:15:59.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.153+0000 7f804a055700 1 -- 172.21.15.99:0/975844876 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80441536d0 con 0x7f8044157870 2023-10-08T00:15:59.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.154+0000 7f802b7fe700 1 -- 172.21.15.99:0/975844876 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f80340074a0 con 0x7f8044157870 2023-10-08T00:15:59.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.154+0000 7f802b7fe700 1 --2- 172.21.15.99:0/975844876 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8030066f30 0x7f80300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:59.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.154+0000 7f802b7fe700 1 -- 172.21.15.99:0/975844876 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8034084a40 con 0x7f8044157870 2023-10-08T00:15:59.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.154+0000 7f802b7fe700 1 -- 172.21.15.99:0/975844876 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=109}) v3 -- 0x7f803006ea50 con 0x7f8044157870 2023-10-08T00:15:59.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.154+0000 7f8042ffd700 1 --2- 172.21.15.99:0/975844876 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8030066f30 0x7f80300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:59.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.154+0000 7f8042ffd700 1 --2- 172.21.15.99:0/975844876 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8030066f30 0x7f80300693e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f80380093f0 tx=0x7f8038008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:59.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.158+0000 7f802b7fe700 1 -- 172.21.15.99:0/975844876 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8034086e60 con 0x7f8044157870 2023-10-08T00:15:59.336 INFO:tasks.ceph.mon.a.smithi099.stderr:2023-10-08T00:15:59.335+0000 7f6c637a7700 -1 log_channel(cluster) log [ERR] : Health check update: 4 full osd(s) (OSD_FULL) 2023-10-08T00:15:59.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.340+0000 7f802b7fe700 1 -- 172.21.15.99:0/975844876 <== mon.0 v2:172.21.15.99:3300/0 7 ==== osd_map(109..109 src has 1..109) v4 ==== 288+0+0 (secure 0 0 0) 0x7f8034086e60 con 0x7f8044157870 2023-10-08T00:15:59.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.340+0000 7f802b7fe700 1 -- 172.21.15.99:0/975844876 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=110}) v3 -- 0x7f803006f340 con 0x7f8044157870 2023-10-08T00:15:59.385 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.384+0000 7f804a055700 1 -- 172.21.15.99:0/975844876 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f804404c3c0 con 0x7f8044157870 2023-10-08T00:15:59.385 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.385+0000 7f802b7fe700 1 -- 172.21.15.99:0/975844876 <== mon.0 v2:172.21.15.99:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4599 (secure 0 0 0) 0x7f8034051d30 con 0x7f8044157870 2023-10-08T00:15:59.385 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:15:59.386 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":48,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5426,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/280880696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":280880696},{"type":"v1","addr":"172.21.15.191:6836","nonce":280880696}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":5902,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3148129562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":3148129562},{"type":"v1","addr":"172.21.15.99:6835","nonce":3148129562}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":5917,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/1825600944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":1825600944},{"type":"v1","addr":"172.21.15.191:6837","nonce":1825600944}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:15:40.595166+0000","modified":"2023-10-08T00:15:44.232916+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6360},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6360":{"gid":6360,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6834/146609901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":146609901},{"type":"v1","addr":"172.21.15.99:6834","nonce":146609901}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-08T00:15:59.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.386+0000 7f804a055700 1 -- 172.21.15.99:0/975844876 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8030066f30 msgr2=0x7f80300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:59.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.386+0000 7f804a055700 1 --2- 172.21.15.99:0/975844876 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8030066f30 0x7f80300693e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f80380093f0 tx=0x7f8038008040 comp rx=0 tx=0).stop 2023-10-08T00:15:59.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.386+0000 7f804a055700 1 -- 172.21.15.99:0/975844876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8044157870 msgr2=0x7f80441530d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:59.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.386+0000 7f804a055700 1 --2- 172.21.15.99:0/975844876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8044157870 0x7f80441530d0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f803400cd80 tx=0x7f8034001ae0 comp rx=0 tx=0).stop 2023-10-08T00:15:59.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.386+0000 7f804a055700 1 -- 172.21.15.99:0/975844876 shutdown_connections 2023-10-08T00:15:59.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.386+0000 7f804a055700 1 --2- 172.21.15.99:0/975844876 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8030066f30 0x7f80300693e0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:59.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.387+0000 7f804a055700 1 --2- 172.21.15.99:0/975844876 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f804415d580 0x7f80440b10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:59.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.387+0000 7f804a055700 1 --2- 172.21.15.99:0/975844876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8044157870 0x7f80441530d0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:59.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.387+0000 7f804a055700 1 --2- 172.21.15.99:0/975844876 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f804415a620 0x7f80441494d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:59.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.387+0000 7f804a055700 1 -- 172.21.15.99:0/975844876 >> 172.21.15.99:0/975844876 conn(0x7f8044068ce0 msgr2=0x7f804414c440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:59.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.387+0000 7f804a055700 1 -- 172.21.15.99:0/975844876 shutdown_connections 2023-10-08T00:15:59.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.387+0000 7f804a055700 1 -- 172.21.15.99:0/975844876 wait complete. 2023-10-08T00:15:59.389 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 48 2023-10-08T00:15:59.399 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-10-08T00:15:59.531 INFO:teuthology.orchestra.run.smithi099.stdout:{"cluster_fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","whoami":0,"id":6360,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":18.297952201000001,"mdsmap_epoch":48,"osdmap_epoch":109,"osdmap_epoch_barrier":108,"uptime":24.273726778} 2023-10-08T00:15:59.531 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", "fs_name": "cephfs", "id": 6360, "mdsmap_epoch": 48, "osdmap_epoch": 109, "osdmap_epoch_barrier": 108, "rank_uptime": 18.297952201, "state": "up:active", "uptime": 24.273726778, "want_state": "up:active", "whoami": 0 } 2023-10-08T00:15:59.532 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-08T00:15:59.532 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:15:59.532 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=small_file_1 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T00:15:59.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T00:15:59.547 INFO:teuthology.orchestra.run.smithi099.stderr:dd: failed to open 'small_file_1': Permission denied 2023-10-08T00:15:59.548 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:15:59.548 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2023-10-08T00:15:59.603 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:15:59.603 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2023-10-08T00:15:59.679 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-08T00:15:59.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.834+0000 7f9c3aabb700 1 Processor -- start 2023-10-08T00:15:59.835 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.834+0000 7f9c3aabb700 1 -- start start 2023-10-08T00:15:59.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.835+0000 7f9c3aabb700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f9c340cdab0 0x7f9c3415d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:59.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.835+0000 7f9c3aabb700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c3415dda0 0x7f9c3415cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:59.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.835+0000 7f9c3aabb700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c340cb7c0 con 0x7f9c340cd6e0 2023-10-08T00:15:59.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.835+0000 7f9c3aabb700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c3415e850 con 0x7f9c340cdab0 2023-10-08T00:15:59.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.835+0000 7f9c3aabb700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c340cc360 con 0x7f9c3415dda0 2023-10-08T00:15:59.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.835+0000 7f9c39ab9700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f9c340cd6e0 0x7f9c340cbc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:47638/0 (socket says 172.21.15.99:47638) 2023-10-08T00:15:59.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.835+0000 7f9c39ab9700 1 -- 172.21.15.99:0/1326566883 learned_addr learned my addr 172.21.15.99:0/1326566883 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:15:59.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.835+0000 7f9c38ab7700 1 -- 172.21.15.99:0/1326566883 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2365691828 0 0) 0x7f9c340cb7c0 con 0x7f9c340cd6e0 2023-10-08T00:15:59.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.835+0000 7f9c38ab7700 1 -- 172.21.15.99:0/1326566883 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c20003570 con 0x7f9c340cd6e0 2023-10-08T00:15:59.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.835+0000 7f9c392b8700 1 --2- 172.21.15.99:0/1326566883 >> v2:172.21.15.191:3300/0 conn(0x7f9c340cdab0 0x7f9c3415d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:59.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.835+0000 7f9c3a2ba700 1 --2- 172.21.15.99:0/1326566883 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c3415dda0 0x7f9c3415cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:59.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.835+0000 7f9c38ab7700 1 -- 172.21.15.99:0/1326566883 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2636228750 0 0) 0x7f9c20003570 con 0x7f9c340cd6e0 2023-10-08T00:15:59.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.835+0000 7f9c38ab7700 1 -- 172.21.15.99:0/1326566883 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c3415dda0 msgr2=0x7f9c3415cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:59.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.835+0000 7f9c38ab7700 1 --2- 172.21.15.99:0/1326566883 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c3415dda0 0x7f9c3415cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:59.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.835+0000 7f9c38ab7700 1 -- 172.21.15.99:0/1326566883 >> v2:172.21.15.191:3300/0 conn(0x7f9c340cdab0 msgr2=0x7f9c3415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:59.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.835+0000 7f9c38ab7700 1 --2- 172.21.15.99:0/1326566883 >> v2:172.21.15.191:3300/0 conn(0x7f9c340cdab0 0x7f9c3415d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:59.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.835+0000 7f9c38ab7700 1 -- 172.21.15.99:0/1326566883 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c3415e290 con 0x7f9c340cd6e0 2023-10-08T00:15:59.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.835+0000 7f9c38ab7700 1 -- 172.21.15.99:0/1326566883 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f9c30001470 con 0x7f9c340cd6e0 2023-10-08T00:15:59.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.835+0000 7f9c38ab7700 1 -- 172.21.15.99:0/1326566883 >> v1:172.21.15.99:6789/0 conn(0x7f9c340cd6e0 legacy=0x7f9c340cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:59.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.835+0000 7f9c38ab7700 1 --2- 172.21.15.99:0/1326566883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c20003a60 0x7f9c20003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:59.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.836+0000 7f9c38ab7700 1 --2- 172.21.15.99:0/1326566883 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c20004810 0x7f9c20004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:59.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.836+0000 7f9c38ab7700 1 --2- 172.21.15.99:0/1326566883 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c3415dda0 0x7f9c200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:59.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.836+0000 7f9c38ab7700 1 -- 172.21.15.99:0/1326566883 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3415e850 con 0x7f9c20003a60 2023-10-08T00:15:59.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.836+0000 7f9c38ab7700 1 -- 172.21.15.99:0/1326566883 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c340cc360 con 0x7f9c20004810 2023-10-08T00:15:59.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.836+0000 7f9c38ab7700 1 -- 172.21.15.99:0/1326566883 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2000ba60 con 0x7f9c3415dda0 2023-10-08T00:15:59.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.836+0000 7f9c39ab9700 1 --2- 172.21.15.99:0/1326566883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c20003a60 0x7f9c20003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:59.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.836+0000 7f9c392b8700 1 --2- 172.21.15.99:0/1326566883 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c20004810 0x7f9c20004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:59.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.836+0000 7f9c3a2ba700 1 --2- 172.21.15.99:0/1326566883 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c3415dda0 0x7f9c200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:59.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.836+0000 7f9c39ab9700 1 -- 172.21.15.99:0/1326566883 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c3415dda0 msgr2=0x7f9c200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:59.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.836+0000 7f9c39ab9700 1 --2- 172.21.15.99:0/1326566883 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c3415dda0 0x7f9c200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:59.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.836+0000 7f9c39ab9700 1 -- 172.21.15.99:0/1326566883 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c20004810 msgr2=0x7f9c20004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:15:59.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.836+0000 7f9c39ab9700 1 --2- 172.21.15.99:0/1326566883 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c20004810 0x7f9c20004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:59.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.836+0000 7f9c39ab9700 1 -- 172.21.15.99:0/1326566883 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c20003570 con 0x7f9c20003a60 2023-10-08T00:15:59.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.836+0000 7f9c3a2ba700 1 --2- 172.21.15.99:0/1326566883 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c3415dda0 0x7f9c200051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:15:59.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.836+0000 7f9c39ab9700 1 --2- 172.21.15.99:0/1326566883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c20003a60 0x7f9c20003e70 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f9c3000c3f0 tx=0x7f9c30001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b95bfc17a6ac690 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:59.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.836+0000 7f9c38ab7700 1 -- 172.21.15.99:0/1326566883 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3000cc80 con 0x7f9c20003a60 2023-10-08T00:15:59.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.836+0000 7f9c38ab7700 1 -- 172.21.15.99:0/1326566883 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c3000ce00 con 0x7f9c20003a60 2023-10-08T00:15:59.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.836+0000 7f9c38ab7700 1 -- 172.21.15.99:0/1326566883 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c30005b70 con 0x7f9c20003a60 2023-10-08T00:15:59.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.837+0000 7f9c3aabb700 1 -- 172.21.15.99:0/1326566883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c20003a60 msgr2=0x7f9c20003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:59.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.837+0000 7f9c3aabb700 1 --2- 172.21.15.99:0/1326566883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c20003a60 0x7f9c20003e70 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f9c3000c3f0 tx=0x7f9c30001e90 comp rx=0 tx=0).stop 2023-10-08T00:15:59.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.837+0000 7f9c3a2ba700 1 -- 172.21.15.99:0/1326566883 reap_dead start 2023-10-08T00:15:59.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.837+0000 7f9c3aabb700 1 -- 172.21.15.99:0/1326566883 shutdown_connections 2023-10-08T00:15:59.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.837+0000 7f9c3aabb700 1 -- 172.21.15.99:0/1326566883 >> 172.21.15.99:0/1326566883 conn(0x7f9c34067660 msgr2=0x7f9c34067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:15:59.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.837+0000 7f9c3aabb700 1 -- 172.21.15.99:0/1326566883 shutdown_connections 2023-10-08T00:15:59.844 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.837+0000 7f9c3aabb700 1 -- 172.21.15.99:0/1326566883 wait complete. 2023-10-08T00:15:59.844 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.838+0000 7f9c3aabb700 1 Processor -- start 2023-10-08T00:15:59.844 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.838+0000 7f9c3aabb700 1 -- start start 2023-10-08T00:15:59.847 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.839+0000 7f9c3aabb700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c20003a60 0x7f9c3416abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:59.847 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.839+0000 7f9c3aabb700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c3415b060 0x7f9c34159440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:59.847 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.839+0000 7f9c3aabb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c34159980 0x7f9c34147200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:59.847 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.839+0000 7f9c3aabb700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3416b130 con 0x7f9c34159980 2023-10-08T00:15:59.847 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.839+0000 7f9c3aabb700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3415c3e0 con 0x7f9c20003a60 2023-10-08T00:15:59.847 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.839+0000 7f9c3aabb700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3415b7d0 con 0x7f9c3415b060 2023-10-08T00:15:59.847 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.839+0000 7f9c3a2ba700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c34159980 0x7f9c34147200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:59.847 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.839+0000 7f9c3a2ba700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c34159980 0x7f9c34147200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58792/0 (socket says 172.21.15.99:58792) 2023-10-08T00:15:59.848 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.839+0000 7f9c3a2ba700 1 -- 172.21.15.99:0/3288367206 learned_addr learned my addr 172.21.15.99:0/3288367206 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:15:59.848 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.839+0000 7f9c392b8700 1 --2- 172.21.15.99:0/3288367206 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c3415b060 0x7f9c34159440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:59.848 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.839+0000 7f9c39ab9700 1 --2- 172.21.15.99:0/3288367206 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c20003a60 0x7f9c3416abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:59.848 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.839+0000 7f9c3a2ba700 1 -- 172.21.15.99:0/3288367206 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c3415b060 msgr2=0x7f9c34159440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:59.848 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.839+0000 7f9c3a2ba700 1 --2- 172.21.15.99:0/3288367206 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c3415b060 0x7f9c34159440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:59.848 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.839+0000 7f9c3a2ba700 1 -- 172.21.15.99:0/3288367206 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c20003a60 msgr2=0x7f9c3416abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:15:59.848 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.839+0000 7f9c3a2ba700 1 --2- 172.21.15.99:0/3288367206 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c20003a60 0x7f9c3416abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:15:59.848 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.839+0000 7f9c3a2ba700 1 -- 172.21.15.99:0/3288367206 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c3415bab0 con 0x7f9c34159980 2023-10-08T00:15:59.849 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.839+0000 7f9c3a2ba700 1 --2- 172.21.15.99:0/3288367206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c34159980 0x7f9c34147200 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f9c34089ac0 tx=0x7f9c2c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:59.849 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.839+0000 7f9c29ffb700 1 -- 172.21.15.99:0/3288367206 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c01b070 con 0x7f9c34159980 2023-10-08T00:15:59.849 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.839+0000 7f9c29ffb700 1 -- 172.21.15.99:0/3288367206 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c2c00f040 con 0x7f9c34159980 2023-10-08T00:15:59.849 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.839+0000 7f9c3aabb700 1 -- 172.21.15.99:0/3288367206 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c3415bd90 con 0x7f9c34159980 2023-10-08T00:15:59.849 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.839+0000 7f9c29ffb700 1 -- 172.21.15.99:0/3288367206 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c00e050 con 0x7f9c34159980 2023-10-08T00:15:59.849 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.840+0000 7f9c3aabb700 1 -- 172.21.15.99:0/3288367206 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c3415b4d0 con 0x7f9c34159980 2023-10-08T00:15:59.849 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.840+0000 7f9c3aabb700 1 -- 172.21.15.99:0/3288367206 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c34089280 con 0x7f9c34159980 2023-10-08T00:15:59.849 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.841+0000 7f9c29ffb700 1 -- 172.21.15.99:0/3288367206 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9c2c003020 con 0x7f9c34159980 2023-10-08T00:15:59.849 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.842+0000 7f9c29ffb700 1 --2- 172.21.15.99:0/3288367206 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9c18066f30 0x7f9c180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:15:59.849 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.842+0000 7f9c29ffb700 1 -- 172.21.15.99:0/3288367206 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9c2c0858a0 con 0x7f9c34159980 2023-10-08T00:15:59.849 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.842+0000 7f9c29ffb700 1 -- 172.21.15.99:0/3288367206 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=110}) v3 -- 0x7f9c1806ea50 con 0x7f9c34159980 2023-10-08T00:15:59.850 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.842+0000 7f9c39ab9700 1 --2- 172.21.15.99:0/3288367206 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9c18066f30 0x7f9c180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:15:59.850 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.843+0000 7f9c39ab9700 1 --2- 172.21.15.99:0/3288367206 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9c18066f30 0x7f9c180693e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9c30001ed0 tx=0x7f9c30015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:15:59.850 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:15:59.845+0000 7f9c29ffb700 1 -- 172.21.15.99:0/3288367206 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9c2c052af0 con 0x7f9c34159980 2023-10-08T00:16:00.026 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:00.026+0000 7f9c3aabb700 1 -- 172.21.15.99:0/3288367206 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f9c3404bb80 con 0x7f9c34159980 2023-10-08T00:16:00.027 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:00.027+0000 7f9c29ffb700 1 -- 172.21.15.99:0/3288367206 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v109) v1 ==== 81+0+32450 (secure 0 0 0) 0x7f9c2c056110 con 0x7f9c34159980 2023-10-08T00:16:00.028 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:16:00.028 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:16:00.028 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 109, 2023-10-08T00:16:00.028 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", 2023-10-08T00:16:00.028 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2023-10-08T00:10:36.303601+0000", 2023-10-08T00:16:00.029 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2023-10-08T00:15:59.336657+0000", 2023-10-08T00:16:00.029 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2023-10-08T00:10:41.709171+0000", 2023-10-08T00:16:00.029 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2023-10-08T00:10:38.690404+0000", 2023-10-08T00:16:00.029 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-08T00:16:00.029 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2023-10-08T00:16:00.029 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2023-10-08T00:16:00.029 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2023-10-08T00:16:00.029 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2023-10-08T00:16:00.029 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2023-10-08T00:16:00.030 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2023-10-08T00:16:00.030 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:16:00.030 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2023-10-08T00:16:00.030 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2023-10-08T00:16:00.030 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-08T00:16:00.030 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-08T00:16:00.030 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2023-10-08T00:16:00.030 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 13, 2023-10-08T00:16:00.030 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2023-10-08T00:16:00.031 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2023-10-08T00:16:00.031 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "jewel", 2023-10-08T00:16:00.031 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "quincy", 2023-10-08T00:16:00.031 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2023-10-08T00:16:00.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.031 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2023-10-08T00:16:00.031 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2023-10-08T00:16:00.031 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:10:42.063344+0000", 2023-10-08T00:16:00.031 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 3, 2023-10-08T00:16:00.031 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full", 2023-10-08T00:16:00.031 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:16:00.032 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:16:00.032 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:16:00.032 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:16:00.032 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:16:00.032 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:16:00.032 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:16:00.032 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:16:00.032 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:16:00.032 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:16:00.033 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2023-10-08T00:16:00.033 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2023-10-08T00:16:00.033 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2023-10-08T00:16:00.033 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2023-10-08T00:16:00.033 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2023-10-08T00:16:00.033 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:16:00.033 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:16:00.033 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:16:00.033 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:16:00.034 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:16:00.034 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:16:00.034 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:16:00.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.034 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "105", 2023-10-08T00:16:00.034 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:16:00.034 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:16:00.034 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:16:00.034 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:16:00.035 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:16:00.035 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:16:00.035 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:16:00.035 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:16:00.035 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:16:00.035 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:16:00.035 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:16:00.035 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:16:00.035 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:16:00.036 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:16:00.036 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:16:00.036 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:16:00.036 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:16:00.036 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:16:00.036 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:16:00.036 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:16:00.036 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:16:00.036 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:16:00.037 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:16:00.037 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:16:00.037 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:16:00.037 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:16:00.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.037 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:16:00.037 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:16:00.037 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:16:00.037 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:16:00.038 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:16:00.038 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:16:00.038 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:16:00.038 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:16:00.038 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:16:00.038 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:16:00.039 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:16:00.039 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:16:00.039 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2023-10-08T00:16:00.039 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2023-10-08T00:16:00.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.039 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:16:00.039 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2023-10-08T00:16:00.039 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:00.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.040 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 12, 2023-10-08T00:16:00.040 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2023-10-08T00:16:00.040 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:15:38.848324+0000", 2023-10-08T00:16:00.040 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 3, 2023-10-08T00:16:00.040 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full", 2023-10-08T00:16:00.040 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:16:00.040 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:16:00.040 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:16:00.040 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:16:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:16:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:16:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:16:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:16:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:16:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:16:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2023-10-08T00:16:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2023-10-08T00:16:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2023-10-08T00:16:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2023-10-08T00:16:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2023-10-08T00:16:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:16:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:16:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:16:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:16:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:16:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:16:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:16:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "105", 2023-10-08T00:16:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:16:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:16:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:16:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:16:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:16:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:16:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:16:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:16:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:16:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:16:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:16:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:16:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:16:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:16:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:16:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:16:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:16:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:16:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:16:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:16:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:16:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:16:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:16:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:16:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:16:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:16:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:16:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:16:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:16:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:16:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:16:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:16:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:16:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:16:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:16:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:16:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:16:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:16:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2023-10-08T00:16:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2023-10-08T00:16:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2023-10-08T00:16:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:16:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:16:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2023-10-08T00:16:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 13, 2023-10-08T00:16:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2023-10-08T00:16:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:15:39.561379+0000", 2023-10-08T00:16:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 3, 2023-10-08T00:16:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full", 2023-10-08T00:16:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:16:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:16:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:16:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:16:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:16:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:16:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:16:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:16:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:16:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:16:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2023-10-08T00:16:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2023-10-08T00:16:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2023-10-08T00:16:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2023-10-08T00:16:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2023-10-08T00:16:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:16:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:16:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:16:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:16:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:16:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:16:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:16:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "105", 2023-10-08T00:16:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:16:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:16:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:16:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:16:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:16:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:16:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:16:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:16:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:16:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:16:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:16:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:16:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:16:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:16:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:16:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:16:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:16:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:16:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:16:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:16:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:16:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:16:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:16:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:16:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:16:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:16:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:16:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:16:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:16:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:16:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:16:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:16:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:16:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:16:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:16:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:16:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:16:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:16:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2023-10-08T00:16:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2023-10-08T00:16:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:16:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:16:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2023-10-08T00:16:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:16:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2023-10-08T00:16:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:16:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:16:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:16:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:16:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:16:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:16:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:16:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:16:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:16:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 102, 2023-10-08T00:16:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:16:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:16:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:16:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2023-10-08T00:16:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:16:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2023-10-08T00:16:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:16:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:16:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2023-10-08T00:16:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:16:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2023-10-08T00:16:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:16:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:16:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2023-10-08T00:16:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:16:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2023-10-08T00:16:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:16:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:16:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2023-10-08T00:16:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:16:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2023-10-08T00:16:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:16:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/86365", 2023-10-08T00:16:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/86365", 2023-10-08T00:16:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/86365", 2023-10-08T00:16:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/86365", 2023-10-08T00:16:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:16:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:16:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: "full", 2023-10-08T00:16:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:16:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:16:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:16:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:16:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:16:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:16:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:16:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:16:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:16:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:16:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 102, 2023-10-08T00:16:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:16:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:16:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:16:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2023-10-08T00:16:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:16:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2023-10-08T00:16:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:16:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:16:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2023-10-08T00:16:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:16:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2023-10-08T00:16:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:16:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:16:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2023-10-08T00:16:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:16:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2023-10-08T00:16:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:16:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:16:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2023-10-08T00:16:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:16:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2023-10-08T00:16:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:16:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/86366", 2023-10-08T00:16:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/86366", 2023-10-08T00:16:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/86366", 2023-10-08T00:16:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/86366", 2023-10-08T00:16:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:16:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull", 2023-10-08T00:16:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:16:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:16:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:16:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:16:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:16:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:16:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:16:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:16:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:16:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:16:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:16:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 102, 2023-10-08T00:16:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:16:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:16:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:16:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2023-10-08T00:16:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:16:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2023-10-08T00:16:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:16:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:16:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2023-10-08T00:16:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:16:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2023-10-08T00:16:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:16:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:16:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2023-10-08T00:16:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:16:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2023-10-08T00:16:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:16:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:16:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2023-10-08T00:16:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:16:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2023-10-08T00:16:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:16:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6817/86382", 2023-10-08T00:16:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/86382", 2023-10-08T00:16:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/86382", 2023-10-08T00:16:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/86382", 2023-10-08T00:16:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:16:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull", 2023-10-08T00:16:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:16:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:16:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:16:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:16:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:16:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:16:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:16:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:16:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:16:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:16:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:16:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 102, 2023-10-08T00:16:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:16:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:16:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:16:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2023-10-08T00:16:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:16:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2023-10-08T00:16:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:16:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:16:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2023-10-08T00:16:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:16:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2023-10-08T00:16:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:16:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:16:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2023-10-08T00:16:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:16:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2023-10-08T00:16:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:16:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:16:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2023-10-08T00:16:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:16:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2023-10-08T00:16:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:16:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6820/86393", 2023-10-08T00:16:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/86393", 2023-10-08T00:16:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/86393", 2023-10-08T00:16:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/86393", 2023-10-08T00:16:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:16:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:16:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: "full", 2023-10-08T00:16:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:16:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:16:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:16:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:16:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:16:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:16:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:16:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:16:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:16:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:16:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 102, 2023-10-08T00:16:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:16:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:16:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:16:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6800", 2023-10-08T00:16:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:16:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6801", 2023-10-08T00:16:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:16:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:16:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6802", 2023-10-08T00:16:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:16:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6803", 2023-10-08T00:16:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:16:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:16:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6806", 2023-10-08T00:16:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:16:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6807", 2023-10-08T00:16:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:16:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:16:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6804", 2023-10-08T00:16:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:16:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6805", 2023-10-08T00:16:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:16:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6801/86570", 2023-10-08T00:16:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6803/86570", 2023-10-08T00:16:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6807/86570", 2023-10-08T00:16:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6805/86570", 2023-10-08T00:16:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:16:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:16:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:16:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:16:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:16:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:16:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:16:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:16:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:16:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:16:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:16:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:16:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 102, 2023-10-08T00:16:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:16:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:16:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:16:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6816", 2023-10-08T00:16:00.105 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:16:00.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.105 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.105 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:00.105 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6818", 2023-10-08T00:16:00.105 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:16:00.105 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:00.105 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:00.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.105 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:16:00.105 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:00.105 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.106 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:00.106 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6820", 2023-10-08T00:16:00.106 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:16:00.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.106 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:00.106 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6821", 2023-10-08T00:16:00.106 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:16:00.106 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:00.106 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:00.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.106 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:16:00.107 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:00.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.107 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:00.107 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6826", 2023-10-08T00:16:00.107 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:16:00.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.107 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:00.107 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6828", 2023-10-08T00:16:00.107 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:16:00.107 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:00.107 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:00.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.108 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:16:00.108 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:00.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.108 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:00.108 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6823", 2023-10-08T00:16:00.108 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:16:00.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.108 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:00.108 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6825", 2023-10-08T00:16:00.108 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:16:00.109 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:00.109 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:00.109 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.109 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6818/86608", 2023-10-08T00:16:00.109 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6821/86608", 2023-10-08T00:16:00.109 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6828/86608", 2023-10-08T00:16:00.109 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6825/86608", 2023-10-08T00:16:00.109 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:16:00.109 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:16:00.109 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:16:00.109 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:00.109 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.110 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:16:00.110 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:16:00.110 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:16:00.110 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:16:00.110 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:16:00.110 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:16:00.110 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:16:00.110 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:16:00.111 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:16:00.111 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 102, 2023-10-08T00:16:00.111 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:16:00.111 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:16:00.111 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:16:00.111 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:00.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.111 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:00.111 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6808", 2023-10-08T00:16:00.111 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:16:00.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.112 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:00.112 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6809", 2023-10-08T00:16:00.112 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:16:00.112 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:00.112 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:00.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.112 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:16:00.112 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:00.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.112 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:00.112 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6810", 2023-10-08T00:16:00.113 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:16:00.113 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.113 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.113 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:00.113 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6811", 2023-10-08T00:16:00.113 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:16:00.113 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:00.113 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:00.113 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.113 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:16:00.113 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:00.113 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.113 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:00.114 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6814", 2023-10-08T00:16:00.114 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:16:00.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.114 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.114 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:00.114 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6815", 2023-10-08T00:16:00.114 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:16:00.114 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:00.114 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:00.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.114 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:16:00.115 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:00.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.115 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:00.115 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6812", 2023-10-08T00:16:00.115 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:16:00.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.115 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:00.115 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6813", 2023-10-08T00:16:00.115 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:16:00.115 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:00.115 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:00.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.116 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6809/86571", 2023-10-08T00:16:00.116 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6811/86571", 2023-10-08T00:16:00.116 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6815/86571", 2023-10-08T00:16:00.116 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6813/86571", 2023-10-08T00:16:00.116 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:16:00.116 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:16:00.116 INFO:teuthology.orchestra.run.smithi099.stdout: "full", 2023-10-08T00:16:00.116 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:16:00.116 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:00.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.117 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:16:00.117 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:16:00.117 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:16:00.117 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:16:00.117 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:16:00.117 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:16:00.117 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:16:00.117 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:16:00.117 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:16:00.117 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 102, 2023-10-08T00:16:00.117 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:16:00.117 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:16:00.117 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:16:00.118 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:00.118 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.118 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:00.118 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6817", 2023-10-08T00:16:00.118 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:16:00.118 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.118 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.118 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:00.118 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6819", 2023-10-08T00:16:00.118 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:16:00.118 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:00.118 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:00.119 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.119 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:16:00.119 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:00.119 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.119 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:00.119 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6822", 2023-10-08T00:16:00.119 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:16:00.119 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.119 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.119 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:00.119 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6824", 2023-10-08T00:16:00.119 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:16:00.120 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:00.120 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:00.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.120 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:16:00.120 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:00.120 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.120 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:00.120 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6830", 2023-10-08T00:16:00.120 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:16:00.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.120 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.120 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:00.121 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6831", 2023-10-08T00:16:00.121 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:16:00.121 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:00.121 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:00.121 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.121 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:16:00.121 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:00.121 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.121 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:00.121 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6827", 2023-10-08T00:16:00.121 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:16:00.121 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.122 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.122 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:00.122 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6829", 2023-10-08T00:16:00.122 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:16:00.122 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:00.122 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:00.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.122 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6819/86612", 2023-10-08T00:16:00.122 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6824/86612", 2023-10-08T00:16:00.122 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6831/86612", 2023-10-08T00:16:00.122 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6829/86612", 2023-10-08T00:16:00.122 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:16:00.123 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:16:00.123 INFO:teuthology.orchestra.run.smithi099.stdout: "full", 2023-10-08T00:16:00.123 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:16:00.123 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:00.123 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:00.123 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:16:00.123 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2023-10-08T00:16:00.123 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.123 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:16:00.123 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:16:00.123 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:16:00.123 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:16:00.124 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:16:00.124 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:16:00.124 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.905615+0000", 2023-10-08T00:16:00.124 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:16:00.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.124 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.124 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:16:00.124 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:16:00.124 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:16:00.124 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:16:00.124 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:16:00.125 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:16:00.125 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.880709+0000", 2023-10-08T00:16:00.125 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:16:00.125 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.125 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.125 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:16:00.125 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:16:00.125 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:16:00.125 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:16:00.125 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:16:00.125 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:16:00.125 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.874704+0000", 2023-10-08T00:16:00.125 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:16:00.126 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.126 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.126 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:16:00.126 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:16:00.126 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:16:00.126 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:16:00.126 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:16:00.126 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:16:00.126 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.899604+0000", 2023-10-08T00:16:00.126 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:16:00.126 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.126 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.127 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:16:00.127 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:16:00.127 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:16:00.127 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:16:00.127 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:16:00.127 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:16:00.127 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.844371+0000", 2023-10-08T00:16:00.127 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:16:00.127 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.127 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.127 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:16:00.127 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:16:00.128 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:16:00.128 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:16:00.128 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:16:00.128 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:16:00.128 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.794810+0000", 2023-10-08T00:16:00.128 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:16:00.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.128 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.128 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:16:00.128 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:16:00.128 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:16:00.128 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:16:00.129 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:16:00.129 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:16:00.129 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.859474+0000", 2023-10-08T00:16:00.129 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:16:00.129 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.129 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:00.129 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:16:00.129 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:16:00.129 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:16:00.129 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:16:00.129 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:16:00.129 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:16:00.130 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.893130+0000", 2023-10-08T00:16:00.130 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:16:00.130 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:00.130 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:16:00.130 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2023-10-08T00:16:00.130 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [], 2023-10-08T00:16:00.130 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2023-10-08T00:16:00.130 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2023-10-08T00:16:00.130 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2023-10-08T00:16:00.130 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2023-10-08T00:16:00.130 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2023-10-08T00:16:00.131 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2023-10-08T00:16:00.131 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2023-10-08T00:16:00.131 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2023-10-08T00:16:00.131 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2023-10-08T00:16:00.131 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2023-10-08T00:16:00.131 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2023-10-08T00:16:00.131 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:00.131 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:00.131 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2023-10-08T00:16:00.131 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2023-10-08T00:16:00.131 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2023-10-08T00:16:00.131 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2023-10-08T00:16:00.132 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2023-10-08T00:16:00.132 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2023-10-08T00:16:00.132 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2023-10-08T00:16:00.132 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2023-10-08T00:16:00.132 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2023-10-08T00:16:00.132 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2023-10-08T00:16:00.132 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2023-10-08T00:16:00.132 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:00.132 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:16:00.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:00.029+0000 7f9c3aabb700 1 -- 172.21.15.99:0/3288367206 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9c18066f30 msgr2=0x7f9c180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:00.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:00.029+0000 7f9c3aabb700 1 --2- 172.21.15.99:0/3288367206 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9c18066f30 0x7f9c180693e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9c30001ed0 tx=0x7f9c30015000 comp rx=0 tx=0).stop 2023-10-08T00:16:00.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:00.029+0000 7f9c3aabb700 1 -- 172.21.15.99:0/3288367206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c34159980 msgr2=0x7f9c34147200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:00.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:00.029+0000 7f9c3aabb700 1 --2- 172.21.15.99:0/3288367206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c34159980 0x7f9c34147200 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f9c34089ac0 tx=0x7f9c2c00b040 comp rx=0 tx=0).stop 2023-10-08T00:16:00.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:00.029+0000 7f9c3aabb700 1 -- 172.21.15.99:0/3288367206 shutdown_connections 2023-10-08T00:16:00.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:00.029+0000 7f9c3aabb700 1 --2- 172.21.15.99:0/3288367206 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9c18066f30 0x7f9c180693e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:00.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:00.029+0000 7f9c3aabb700 1 --2- 172.21.15.99:0/3288367206 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c20003a60 0x7f9c3416abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:00.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:00.029+0000 7f9c3aabb700 1 --2- 172.21.15.99:0/3288367206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c34159980 0x7f9c34147200 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:00.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:00.029+0000 7f9c3aabb700 1 --2- 172.21.15.99:0/3288367206 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c3415b060 0x7f9c34159440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:00.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:00.029+0000 7f9c3aabb700 1 -- 172.21.15.99:0/3288367206 >> 172.21.15.99:0/3288367206 conn(0x7f9c34067660 msgr2=0x7f9c341569d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:00.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:00.029+0000 7f9c3aabb700 1 -- 172.21.15.99:0/3288367206 shutdown_connections 2023-10-08T00:16:00.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:00.029+0000 7f9c3aabb700 1 -- 172.21.15.99:0/3288367206 wait complete. 2023-10-08T00:16:00.135 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-08T00:16:05.136 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-08T00:16:05.555 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.554+0000 7fdb634fc700 1 Processor -- start 2023-10-08T00:16:05.555 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.555+0000 7fdb634fc700 1 -- start start 2023-10-08T00:16:05.559 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.555+0000 7fdb634fc700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdb5c157870 0x7fdb5c15b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:05.559 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.555+0000 7fdb634fc700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fdb5c15a620 0x7fdb5c15d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:05.559 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.555+0000 7fdb634fc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdb5c15d580 0x7fdb5c158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:05.559 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.555+0000 7fdb634fc700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb5c15cbb0 con 0x7fdb5c157870 2023-10-08T00:16:05.559 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.555+0000 7fdb634fc700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb5c15bfa0 con 0x7fdb5c15a620 2023-10-08T00:16:05.559 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.555+0000 7fdb634fc700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c15c280 con 0x7fdb5c15d580 2023-10-08T00:16:05.559 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.556+0000 7fdb61298700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdb5c157870 0x7fdb5c15b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:05.559 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.556+0000 7fdb61298700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdb5c157870 0x7fdb5c15b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46044/0 (socket says 172.21.15.99:46044) 2023-10-08T00:16:05.559 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.556+0000 7fdb61298700 1 -- 172.21.15.99:0/374329266 learned_addr learned my addr 172.21.15.99:0/374329266 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:05.560 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.556+0000 7fdb60a97700 1 --2- 172.21.15.99:0/374329266 >> v2:172.21.15.191:3300/0 conn(0x7fdb5c15a620 0x7fdb5c15d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:05.560 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.556+0000 7fdb61a99700 1 --2- 172.21.15.99:0/374329266 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdb5c15d580 0x7fdb5c158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:05.560 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.556+0000 7fdb61298700 1 -- 172.21.15.99:0/374329266 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdb5c15d580 msgr2=0x7fdb5c158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:05.560 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.556+0000 7fdb61298700 1 --2- 172.21.15.99:0/374329266 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdb5c15d580 0x7fdb5c158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:05.560 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.556+0000 7fdb61298700 1 -- 172.21.15.99:0/374329266 >> v2:172.21.15.191:3300/0 conn(0x7fdb5c15a620 msgr2=0x7fdb5c15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:05.560 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.556+0000 7fdb61298700 1 --2- 172.21.15.99:0/374329266 >> v2:172.21.15.191:3300/0 conn(0x7fdb5c15a620 0x7fdb5c15d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:05.560 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.556+0000 7fdb61298700 1 -- 172.21.15.99:0/374329266 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb5c15c560 con 0x7fdb5c157870 2023-10-08T00:16:05.560 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.556+0000 7fdb61298700 1 --2- 172.21.15.99:0/374329266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb5c157870 0x7fdb5c15b6a0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c15cb20 tx=0x7fdb4c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d63bc5720353d531 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:05.560 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.556+0000 7fdb5bfff700 1 -- 172.21.15.99:0/374329266 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c00e070 con 0x7fdb5c157870 2023-10-08T00:16:05.560 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.556+0000 7fdb5bfff700 1 -- 172.21.15.99:0/374329266 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb4c00b970 con 0x7fdb5c157870 2023-10-08T00:16:05.560 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.556+0000 7fdb5bfff700 1 -- 172.21.15.99:0/374329266 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c0092e0 con 0x7fdb5c157870 2023-10-08T00:16:05.561 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.557+0000 7fdb634fc700 1 -- 172.21.15.99:0/374329266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb5c157870 msgr2=0x7fdb5c15b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:05.561 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.557+0000 7fdb634fc700 1 --2- 172.21.15.99:0/374329266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb5c157870 0x7fdb5c15b6a0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c15cb20 tx=0x7fdb4c009840 comp rx=0 tx=0).stop 2023-10-08T00:16:05.561 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.557+0000 7fdb634fc700 1 -- 172.21.15.99:0/374329266 shutdown_connections 2023-10-08T00:16:05.561 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.557+0000 7fdb634fc700 1 --2- 172.21.15.99:0/374329266 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdb5c15d580 0x7fdb5c158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:05.561 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.557+0000 7fdb634fc700 1 --2- 172.21.15.99:0/374329266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb5c157870 0x7fdb5c15b6a0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:05.561 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.557+0000 7fdb634fc700 1 --2- 172.21.15.99:0/374329266 >> v2:172.21.15.191:3300/0 conn(0x7fdb5c15a620 0x7fdb5c15d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:05.561 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.557+0000 7fdb634fc700 1 -- 172.21.15.99:0/374329266 >> 172.21.15.99:0/374329266 conn(0x7fdb5c068ce0 msgr2=0x7fdb5c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:05.561 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.557+0000 7fdb634fc700 1 -- 172.21.15.99:0/374329266 shutdown_connections 2023-10-08T00:16:05.561 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.557+0000 7fdb634fc700 1 -- 172.21.15.99:0/374329266 wait complete. 2023-10-08T00:16:05.561 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.558+0000 7fdb634fc700 1 Processor -- start 2023-10-08T00:16:05.561 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.558+0000 7fdb634fc700 1 -- start start 2023-10-08T00:16:05.565 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.559+0000 7fdb634fc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdb5c157870 0x7fdb5c0b0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:05.566 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.559+0000 7fdb634fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb5c15a620 0x7fdb5c0b2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:05.566 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.559+0000 7fdb634fc700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb5c15d580 0x7fdb5c154850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:05.567 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.559+0000 7fdb634fc700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c0b21d0 con 0x7fdb5c15a620 2023-10-08T00:16:05.567 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.559+0000 7fdb634fc700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c0b18b0 con 0x7fdb5c15d580 2023-10-08T00:16:05.567 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.559+0000 7fdb634fc700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c0b0ca0 con 0x7fdb5c157870 2023-10-08T00:16:05.567 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.559+0000 7fdb60a97700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb5c15a620 0x7fdb5c0b2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:05.567 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.559+0000 7fdb60a97700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb5c15a620 0x7fdb5c0b2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46054/0 (socket says 172.21.15.99:46054) 2023-10-08T00:16:05.567 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.559+0000 7fdb60a97700 1 -- 172.21.15.99:0/4053063955 learned_addr learned my addr 172.21.15.99:0/4053063955 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:05.567 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.559+0000 7fdb61a99700 1 --2- 172.21.15.99:0/4053063955 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb5c15d580 0x7fdb5c154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:05.567 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.559+0000 7fdb61298700 1 --2- 172.21.15.99:0/4053063955 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdb5c157870 0x7fdb5c0b0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:05.567 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.559+0000 7fdb60a97700 1 -- 172.21.15.99:0/4053063955 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdb5c157870 msgr2=0x7fdb5c0b0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:05.567 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.559+0000 7fdb60a97700 1 --2- 172.21.15.99:0/4053063955 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdb5c157870 0x7fdb5c0b0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:05.568 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.559+0000 7fdb60a97700 1 -- 172.21.15.99:0/4053063955 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb5c15d580 msgr2=0x7fdb5c154850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:05.568 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.559+0000 7fdb60a97700 1 --2- 172.21.15.99:0/4053063955 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb5c15d580 0x7fdb5c154850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:05.568 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.559+0000 7fdb60a97700 1 -- 172.21.15.99:0/4053063955 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb4c009ae0 con 0x7fdb5c15a620 2023-10-08T00:16:05.568 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.559+0000 7fdb60a97700 1 --2- 172.21.15.99:0/4053063955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb5c15a620 0x7fdb5c0b2570 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fdb5000ec50 tx=0x7fdb500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:05.568 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.560+0000 7fdb597fa700 1 -- 172.21.15.99:0/4053063955 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5000c850 con 0x7fdb5c15a620 2023-10-08T00:16:05.568 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.560+0000 7fdb597fa700 1 -- 172.21.15.99:0/4053063955 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb5000c9b0 con 0x7fdb5c15a620 2023-10-08T00:16:05.568 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.560+0000 7fdb634fc700 1 -- 172.21.15.99:0/4053063955 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb5c0b0f80 con 0x7fdb5c15a620 2023-10-08T00:16:05.568 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.560+0000 7fdb597fa700 1 -- 172.21.15.99:0/4053063955 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5000cd70 con 0x7fdb5c15a620 2023-10-08T00:16:05.568 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.560+0000 7fdb634fc700 1 -- 172.21.15.99:0/4053063955 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb5c0b1540 con 0x7fdb5c15a620 2023-10-08T00:16:05.568 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.560+0000 7fdb634fc700 1 -- 172.21.15.99:0/4053063955 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb5c0b1a40 con 0x7fdb5c15a620 2023-10-08T00:16:05.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.561+0000 7fdb597fa700 1 -- 172.21.15.99:0/4053063955 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdb50013680 con 0x7fdb5c15a620 2023-10-08T00:16:05.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.561+0000 7fdb597fa700 1 --2- 172.21.15.99:0/4053063955 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdb48066f30 0x7fdb480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:05.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.561+0000 7fdb597fa700 1 -- 172.21.15.99:0/4053063955 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdb50084740 con 0x7fdb5c15a620 2023-10-08T00:16:05.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.562+0000 7fdb61298700 1 --2- 172.21.15.99:0/4053063955 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdb48066f30 0x7fdb480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:05.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.562+0000 7fdb61298700 1 --2- 172.21.15.99:0/4053063955 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdb48066f30 0x7fdb480693e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c009880 tx=0x7fdb4c018000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:05.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.566+0000 7fdb597fa700 1 -- 172.21.15.99:0/4053063955 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdb500519b0 con 0x7fdb5c15a620 2023-10-08T00:16:05.748 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.748+0000 7fdb634fc700 1 -- 172.21.15.99:0/4053063955 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fdb5c04c3c0 con 0x7fdb5c15a620 2023-10-08T00:16:05.749 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.749+0000 7fdb597fa700 1 -- 172.21.15.99:0/4053063955 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v112) v1 ==== 81+0+32275 (secure 0 0 0) 0x7fdb50054fd0 con 0x7fdb5c15a620 2023-10-08T00:16:05.750 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:16:05.750 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:16:05.750 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 112, 2023-10-08T00:16:05.750 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", 2023-10-08T00:16:05.751 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2023-10-08T00:10:36.303601+0000", 2023-10-08T00:16:05.751 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2023-10-08T00:16:05.407718+0000", 2023-10-08T00:16:05.751 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2023-10-08T00:10:41.709171+0000", 2023-10-08T00:16:05.751 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2023-10-08T00:10:38.690404+0000", 2023-10-08T00:16:05.751 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-08T00:16:05.751 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2023-10-08T00:16:05.751 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2023-10-08T00:16:05.751 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2023-10-08T00:16:05.752 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2023-10-08T00:16:05.752 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2023-10-08T00:16:05.752 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2023-10-08T00:16:05.752 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:16:05.752 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2023-10-08T00:16:05.752 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2023-10-08T00:16:05.752 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-08T00:16:05.752 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-08T00:16:05.752 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2023-10-08T00:16:05.752 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 13, 2023-10-08T00:16:05.753 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2023-10-08T00:16:05.753 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2023-10-08T00:16:05.753 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "jewel", 2023-10-08T00:16:05.753 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "quincy", 2023-10-08T00:16:05.753 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2023-10-08T00:16:05.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.753 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2023-10-08T00:16:05.753 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2023-10-08T00:16:05.754 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:10:42.063344+0000", 2023-10-08T00:16:05.754 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:16:05.754 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:16:05.754 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:16:05.754 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:16:05.754 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:16:05.754 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:16:05.754 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:16:05.755 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:16:05.755 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:16:05.755 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:16:05.755 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:16:05.755 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:16:05.755 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2023-10-08T00:16:05.755 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2023-10-08T00:16:05.755 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2023-10-08T00:16:05.756 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2023-10-08T00:16:05.756 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2023-10-08T00:16:05.756 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:16:05.756 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:16:05.756 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:16:05.756 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:16:05.756 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:16:05.756 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:16:05.756 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:16:05.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.757 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "112", 2023-10-08T00:16:05.757 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:16:05.757 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:16:05.757 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:16:05.757 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:16:05.757 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:16:05.757 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:16:05.758 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:16:05.758 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:16:05.758 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:16:05.758 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:16:05.758 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:16:05.758 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:16:05.758 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:16:05.758 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:16:05.758 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:16:05.759 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:16:05.759 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:16:05.759 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:16:05.759 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:16:05.759 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:16:05.759 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:16:05.759 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:16:05.760 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:16:05.760 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:16:05.760 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:16:05.760 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:16:05.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.760 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:16:05.760 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:16:05.760 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:16:05.760 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:16:05.760 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:16:05.761 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:16:05.761 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:16:05.761 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:16:05.761 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:16:05.761 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:16:05.761 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:16:05.761 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:16:05.761 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2023-10-08T00:16:05.761 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2023-10-08T00:16:05.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.761 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:16:05.762 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2023-10-08T00:16:05.762 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:05.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.762 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 12, 2023-10-08T00:16:05.762 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2023-10-08T00:16:05.762 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:15:38.848324+0000", 2023-10-08T00:16:05.762 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:16:05.762 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:16:05.763 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:16:05.763 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:16:05.763 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:16:05.763 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:16:05.763 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:16:05.763 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:16:05.763 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:16:05.764 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:16:05.764 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:16:05.764 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:16:05.764 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2023-10-08T00:16:05.764 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2023-10-08T00:16:05.764 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2023-10-08T00:16:05.764 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2023-10-08T00:16:05.764 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2023-10-08T00:16:05.765 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:16:05.765 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:16:05.765 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:16:05.765 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:16:05.765 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:16:05.765 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:16:05.765 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:16:05.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.766 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "112", 2023-10-08T00:16:05.766 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:16:05.766 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:16:05.766 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:16:05.766 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:16:05.766 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:16:05.766 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:16:05.767 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:16:05.767 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:16:05.767 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:16:05.767 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:16:05.767 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:16:05.767 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:16:05.767 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:16:05.767 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:16:05.768 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:16:05.768 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:16:05.768 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:16:05.768 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:16:05.768 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:16:05.768 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:16:05.768 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:16:05.768 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:16:05.768 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:16:05.769 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:16:05.769 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:16:05.769 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:16:05.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.769 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:16:05.769 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:16:05.769 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:16:05.769 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:16:05.769 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:16:05.770 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:16:05.770 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:16:05.770 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:16:05.770 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:16:05.770 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:16:05.770 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:16:05.770 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:16:05.770 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2023-10-08T00:16:05.770 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2023-10-08T00:16:05.770 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2023-10-08T00:16:05.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.771 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:16:05.771 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:16:05.771 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2023-10-08T00:16:05.771 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:05.771 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:05.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.771 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 13, 2023-10-08T00:16:05.771 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2023-10-08T00:16:05.771 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:15:39.561379+0000", 2023-10-08T00:16:05.772 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:16:05.772 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:16:05.772 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:16:05.772 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:16:05.772 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:16:05.772 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:16:05.772 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:16:05.772 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:16:05.772 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:16:05.772 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:16:05.773 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:16:05.773 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:16:05.773 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2023-10-08T00:16:05.773 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2023-10-08T00:16:05.773 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2023-10-08T00:16:05.773 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2023-10-08T00:16:05.773 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2023-10-08T00:16:05.773 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:16:05.774 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:16:05.774 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:16:05.774 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:16:05.774 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:16:05.774 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:16:05.774 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:16:05.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.774 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "112", 2023-10-08T00:16:05.775 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:16:05.775 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:16:05.775 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:16:05.775 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:16:05.775 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:16:05.775 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:16:05.775 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:16:05.775 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:16:05.775 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:16:05.776 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:16:05.776 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:16:05.776 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:16:05.776 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:16:05.776 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:16:05.776 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:16:05.776 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:16:05.776 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:16:05.777 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:16:05.777 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:16:05.777 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:16:05.777 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:16:05.777 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:16:05.777 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:16:05.777 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:16:05.777 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:16:05.778 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:16:05.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.778 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:16:05.778 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:16:05.778 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:16:05.778 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:16:05.778 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:16:05.778 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:16:05.778 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:16:05.778 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:16:05.778 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:16:05.779 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:16:05.779 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:16:05.779 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:16:05.779 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2023-10-08T00:16:05.779 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2023-10-08T00:16:05.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.779 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:16:05.779 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:16:05.779 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2023-10-08T00:16:05.779 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:05.780 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:05.780 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:05.780 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:16:05.780 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2023-10-08T00:16:05.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.780 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:16:05.780 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:16:05.780 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:16:05.780 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:16:05.780 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:16:05.780 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:16:05.781 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:16:05.781 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:16:05.781 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:16:05.781 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 102, 2023-10-08T00:16:05.781 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:16:05.781 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:16:05.781 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:16:05.781 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:05.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.781 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:05.781 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2023-10-08T00:16:05.781 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:16:05.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.782 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:05.782 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2023-10-08T00:16:05.782 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:16:05.782 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:05.782 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:05.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.782 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:16:05.782 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:05.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.783 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:05.783 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2023-10-08T00:16:05.783 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:16:05.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.783 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:05.783 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2023-10-08T00:16:05.783 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:16:05.783 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:05.784 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:05.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.784 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:16:05.784 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:05.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.784 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:05.784 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2023-10-08T00:16:05.784 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:16:05.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.785 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:05.785 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2023-10-08T00:16:05.785 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:16:05.785 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:05.785 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:05.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.785 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:16:05.785 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:05.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.785 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:05.786 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2023-10-08T00:16:05.786 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:16:05.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.786 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:05.786 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2023-10-08T00:16:05.786 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:16:05.786 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:05.786 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:05.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.787 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/86365", 2023-10-08T00:16:05.787 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/86365", 2023-10-08T00:16:05.787 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/86365", 2023-10-08T00:16:05.787 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/86365", 2023-10-08T00:16:05.787 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:16:05.787 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:16:05.787 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:16:05.787 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:05.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.787 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:16:05.788 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:16:05.788 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:16:05.788 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:16:05.788 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:16:05.788 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:16:05.788 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:16:05.788 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:16:05.788 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:16:05.788 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 102, 2023-10-08T00:16:05.788 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:16:05.789 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:16:05.789 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:16:05.789 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:05.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.789 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:05.789 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2023-10-08T00:16:05.789 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:16:05.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.789 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:05.790 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2023-10-08T00:16:05.790 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:16:05.790 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:05.790 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:05.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.790 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:16:05.790 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:05.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.790 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:05.790 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2023-10-08T00:16:05.791 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:16:05.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.791 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:05.791 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2023-10-08T00:16:05.791 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:16:05.791 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:05.791 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:05.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.792 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:16:05.792 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:05.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.792 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:05.792 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2023-10-08T00:16:05.792 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:16:05.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.792 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:05.793 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2023-10-08T00:16:05.793 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:16:05.793 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:05.793 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:05.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.793 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:16:05.793 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:05.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.793 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:05.793 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2023-10-08T00:16:05.794 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:16:05.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.794 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:05.794 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2023-10-08T00:16:05.794 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:16:05.794 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:05.794 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:05.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.794 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/86366", 2023-10-08T00:16:05.795 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/86366", 2023-10-08T00:16:05.795 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/86366", 2023-10-08T00:16:05.795 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/86366", 2023-10-08T00:16:05.795 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:16:05.795 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:16:05.795 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:16:05.795 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:05.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.795 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:16:05.796 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:16:05.796 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:16:05.796 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:16:05.796 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:16:05.796 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:16:05.796 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:16:05.796 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:16:05.796 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:16:05.796 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 102, 2023-10-08T00:16:05.796 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:16:05.796 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:16:05.797 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:16:05.797 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:05.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.797 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:05.797 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2023-10-08T00:16:05.797 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:16:05.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.797 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:05.797 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2023-10-08T00:16:05.797 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:16:05.798 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:05.798 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:05.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.798 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:16:05.798 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:05.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.798 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:05.798 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2023-10-08T00:16:05.798 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:16:05.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.799 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:05.799 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2023-10-08T00:16:05.799 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:16:05.799 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:05.799 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:05.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.799 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:16:05.799 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:05.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.799 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:05.800 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2023-10-08T00:16:05.800 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:16:05.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.800 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:05.800 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2023-10-08T00:16:05.800 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:16:05.800 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:05.800 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:05.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.800 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:16:05.801 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:05.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.801 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:05.801 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2023-10-08T00:16:05.801 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:16:05.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.801 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:05.801 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2023-10-08T00:16:05.801 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:16:05.802 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:05.802 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:05.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.802 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6817/86382", 2023-10-08T00:16:05.802 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/86382", 2023-10-08T00:16:05.802 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/86382", 2023-10-08T00:16:05.802 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/86382", 2023-10-08T00:16:05.802 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:16:05.802 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:16:05.802 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:16:05.803 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:05.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.803 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:16:05.803 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:16:05.803 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:16:05.803 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:16:05.804 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:16:05.804 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:16:05.804 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:16:05.804 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:16:05.804 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:16:05.804 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 102, 2023-10-08T00:16:05.805 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:16:05.805 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:16:05.805 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:16:05.805 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:05.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.805 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:05.805 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2023-10-08T00:16:05.805 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:16:05.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.806 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:05.806 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2023-10-08T00:16:05.806 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:16:05.806 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:05.806 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:05.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.806 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:16:05.806 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:05.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.807 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:05.807 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2023-10-08T00:16:05.807 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:16:05.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.807 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:05.807 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2023-10-08T00:16:05.807 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:16:05.808 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:05.808 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:05.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.808 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:16:05.808 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:05.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.808 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:05.808 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2023-10-08T00:16:05.808 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:16:05.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.809 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:05.809 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2023-10-08T00:16:05.809 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:16:05.809 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:05.809 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:05.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.810 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:16:05.810 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:05.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.810 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:05.810 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2023-10-08T00:16:05.810 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:16:05.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.810 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:05.810 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2023-10-08T00:16:05.810 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:16:05.810 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:05.811 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:05.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.811 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6820/86393", 2023-10-08T00:16:05.811 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/86393", 2023-10-08T00:16:05.811 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/86393", 2023-10-08T00:16:05.811 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/86393", 2023-10-08T00:16:05.811 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:16:05.811 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:16:05.811 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:16:05.811 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:05.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.812 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:16:05.812 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:16:05.812 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:16:05.813 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:16:05.813 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:16:05.813 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:16:05.813 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:16:05.813 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:16:05.813 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:16:05.813 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 102, 2023-10-08T00:16:05.813 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:16:05.814 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:16:05.814 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:16:05.814 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:05.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.814 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:05.814 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6800", 2023-10-08T00:16:05.814 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:16:05.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.814 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:05.815 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6801", 2023-10-08T00:16:05.815 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:16:05.815 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:05.815 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:05.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.815 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:16:05.815 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:05.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.815 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:05.816 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6802", 2023-10-08T00:16:05.816 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:16:05.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.816 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:05.816 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6803", 2023-10-08T00:16:05.816 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:16:05.816 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:05.816 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:05.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.817 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:16:05.817 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:05.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.817 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:05.817 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6806", 2023-10-08T00:16:05.817 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:16:05.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.817 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:05.818 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6807", 2023-10-08T00:16:05.818 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:16:05.818 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:05.818 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:05.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.818 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:16:05.818 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:05.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.818 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:05.819 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6804", 2023-10-08T00:16:05.819 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:16:05.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.819 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:05.819 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6805", 2023-10-08T00:16:05.819 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:16:05.819 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:05.819 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:05.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.819 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6801/86570", 2023-10-08T00:16:05.820 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6803/86570", 2023-10-08T00:16:05.820 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6807/86570", 2023-10-08T00:16:05.820 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6805/86570", 2023-10-08T00:16:05.820 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:16:05.820 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:16:05.820 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:16:05.820 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:05.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.820 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:16:05.820 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:16:05.821 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:16:05.821 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:16:05.821 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:16:05.821 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:16:05.821 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:16:05.821 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:16:05.821 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:16:05.821 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 102, 2023-10-08T00:16:05.821 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:16:05.822 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:16:05.822 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:16:05.822 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:05.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.822 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:05.822 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6816", 2023-10-08T00:16:05.822 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:16:05.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.823 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:05.823 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6818", 2023-10-08T00:16:05.823 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:16:05.823 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:05.823 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:05.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.823 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:16:05.823 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:05.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.824 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:05.824 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6820", 2023-10-08T00:16:05.824 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:16:05.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.824 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:05.824 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6821", 2023-10-08T00:16:05.824 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:16:05.825 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:05.825 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:05.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.825 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:16:05.825 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:05.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.825 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:05.825 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6826", 2023-10-08T00:16:05.825 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:16:05.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.826 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:05.826 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6828", 2023-10-08T00:16:05.826 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:16:05.826 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:05.826 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:05.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.826 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:16:05.826 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:05.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.827 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:05.827 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6823", 2023-10-08T00:16:05.827 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:16:05.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.827 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:05.827 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6825", 2023-10-08T00:16:05.827 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:16:05.827 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:05.828 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:05.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.828 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6818/86608", 2023-10-08T00:16:05.828 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6821/86608", 2023-10-08T00:16:05.828 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6828/86608", 2023-10-08T00:16:05.828 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6825/86608", 2023-10-08T00:16:05.828 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:16:05.828 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:16:05.828 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:16:05.828 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:05.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.829 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:16:05.829 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:16:05.829 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:16:05.829 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:16:05.829 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:16:05.829 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:16:05.829 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:16:05.829 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:16:05.829 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:16:05.830 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 102, 2023-10-08T00:16:05.830 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:16:05.830 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:16:05.830 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:16:05.830 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:05.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.830 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:05.830 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6808", 2023-10-08T00:16:05.830 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:16:05.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.831 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:05.831 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6809", 2023-10-08T00:16:05.831 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:16:05.831 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:05.831 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:05.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.831 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:16:05.831 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:05.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.831 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:05.832 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6810", 2023-10-08T00:16:05.832 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:16:05.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.832 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:05.832 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6811", 2023-10-08T00:16:05.832 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:16:05.832 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:05.832 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:05.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.833 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:16:05.833 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:05.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.833 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:05.833 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6814", 2023-10-08T00:16:05.833 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:16:05.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.833 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:05.833 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6815", 2023-10-08T00:16:05.833 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:16:05.834 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:05.834 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:05.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.834 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:16:05.834 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:05.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.834 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:05.834 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6812", 2023-10-08T00:16:05.834 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:16:05.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.835 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:05.835 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6813", 2023-10-08T00:16:05.835 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:16:05.835 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:05.835 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:05.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.835 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6809/86571", 2023-10-08T00:16:05.835 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6811/86571", 2023-10-08T00:16:05.835 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6815/86571", 2023-10-08T00:16:05.835 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6813/86571", 2023-10-08T00:16:05.836 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:16:05.836 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:16:05.836 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:16:05.836 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:05.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.836 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:16:05.836 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:16:05.836 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:16:05.836 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:16:05.836 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:16:05.837 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:16:05.837 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:16:05.837 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:16:05.837 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:16:05.837 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 102, 2023-10-08T00:16:05.837 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:16:05.837 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:16:05.837 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:16:05.837 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:05.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.838 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:05.838 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6817", 2023-10-08T00:16:05.838 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:16:05.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.838 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:05.838 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6819", 2023-10-08T00:16:05.838 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:16:05.838 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:05.838 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:05.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.839 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:16:05.839 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:05.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.839 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:05.839 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6822", 2023-10-08T00:16:05.839 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:16:05.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.839 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:05.839 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6824", 2023-10-08T00:16:05.839 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:16:05.840 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:05.840 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:05.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.840 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:16:05.840 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:05.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.840 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:05.840 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6830", 2023-10-08T00:16:05.840 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:16:05.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.841 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:05.841 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6831", 2023-10-08T00:16:05.841 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:16:05.841 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:05.841 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:05.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.841 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:16:05.842 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:05.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.842 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:05.842 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6827", 2023-10-08T00:16:05.842 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:16:05.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.842 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:05.843 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6829", 2023-10-08T00:16:05.843 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:16:05.843 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:05.843 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:05.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.843 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6819/86612", 2023-10-08T00:16:05.843 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6824/86612", 2023-10-08T00:16:05.843 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6831/86612", 2023-10-08T00:16:05.844 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6829/86612", 2023-10-08T00:16:05.844 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:16:05.844 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:16:05.844 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:16:05.844 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:05.844 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:05.844 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:16:05.844 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2023-10-08T00:16:05.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.845 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:16:05.845 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:16:05.845 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:16:05.845 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:16:05.845 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:16:05.845 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:16:05.845 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.905615+0000", 2023-10-08T00:16:05.846 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:16:05.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.846 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:16:05.846 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:16:05.846 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:16:05.846 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:16:05.846 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:16:05.846 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:16:05.847 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.880709+0000", 2023-10-08T00:16:05.847 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:16:05.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.847 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:16:05.847 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:16:05.847 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:16:05.847 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:16:05.847 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:16:05.847 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:16:05.848 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.874704+0000", 2023-10-08T00:16:05.848 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:16:05.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.848 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:16:05.848 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:16:05.848 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:16:05.848 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:16:05.848 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:16:05.849 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:16:05.849 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.899604+0000", 2023-10-08T00:16:05.849 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:16:05.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.849 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:16:05.849 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:16:05.849 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:16:05.849 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:16:05.849 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:16:05.850 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:16:05.850 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.844371+0000", 2023-10-08T00:16:05.850 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:16:05.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.850 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:16:05.850 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:16:05.850 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:16:05.850 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:16:05.850 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:16:05.851 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:16:05.851 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.794810+0000", 2023-10-08T00:16:05.851 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:16:05.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.851 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:16:05.851 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:16:05.851 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:16:05.851 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:16:05.851 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:16:05.851 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:16:05.852 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.859474+0000", 2023-10-08T00:16:05.852 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:16:05.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:05.852 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:16:05.852 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:16:05.852 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:16:05.852 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:16:05.852 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:16:05.852 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:16:05.853 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.893130+0000", 2023-10-08T00:16:05.853 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:16:05.853 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:05.853 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:16:05.853 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2023-10-08T00:16:05.853 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [], 2023-10-08T00:16:05.853 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2023-10-08T00:16:05.853 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2023-10-08T00:16:05.853 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2023-10-08T00:16:05.854 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2023-10-08T00:16:05.854 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2023-10-08T00:16:05.854 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2023-10-08T00:16:05.854 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2023-10-08T00:16:05.854 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2023-10-08T00:16:05.854 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2023-10-08T00:16:05.854 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2023-10-08T00:16:05.854 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2023-10-08T00:16:05.854 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:05.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:05.855 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2023-10-08T00:16:05.855 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2023-10-08T00:16:05.855 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2023-10-08T00:16:05.855 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2023-10-08T00:16:05.855 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2023-10-08T00:16:05.855 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2023-10-08T00:16:05.855 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2023-10-08T00:16:05.855 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2023-10-08T00:16:05.855 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2023-10-08T00:16:05.855 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2023-10-08T00:16:05.856 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2023-10-08T00:16:05.856 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:05.856 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:16:05.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.751+0000 7fdb634fc700 1 -- 172.21.15.99:0/4053063955 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdb48066f30 msgr2=0x7fdb480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:05.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.751+0000 7fdb634fc700 1 --2- 172.21.15.99:0/4053063955 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdb48066f30 0x7fdb480693e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c009880 tx=0x7fdb4c018000 comp rx=0 tx=0).stop 2023-10-08T00:16:05.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.751+0000 7fdb634fc700 1 -- 172.21.15.99:0/4053063955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb5c15a620 msgr2=0x7fdb5c0b2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:05.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.751+0000 7fdb634fc700 1 --2- 172.21.15.99:0/4053063955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb5c15a620 0x7fdb5c0b2570 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fdb5000ec50 tx=0x7fdb500091a0 comp rx=0 tx=0).stop 2023-10-08T00:16:05.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.751+0000 7fdb634fc700 1 -- 172.21.15.99:0/4053063955 shutdown_connections 2023-10-08T00:16:05.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.751+0000 7fdb634fc700 1 --2- 172.21.15.99:0/4053063955 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdb48066f30 0x7fdb480693e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:05.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.751+0000 7fdb634fc700 1 --2- 172.21.15.99:0/4053063955 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb5c15d580 0x7fdb5c154850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:05.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.751+0000 7fdb634fc700 1 --2- 172.21.15.99:0/4053063955 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdb5c157870 0x7fdb5c0b0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:05.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.751+0000 7fdb634fc700 1 --2- 172.21.15.99:0/4053063955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb5c15a620 0x7fdb5c0b2570 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:05.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.751+0000 7fdb634fc700 1 -- 172.21.15.99:0/4053063955 >> 172.21.15.99:0/4053063955 conn(0x7fdb5c068ce0 msgr2=0x7fdb5c1582e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:05.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.751+0000 7fdb634fc700 1 -- 172.21.15.99:0/4053063955 shutdown_connections 2023-10-08T00:16:05.859 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.751+0000 7fdb634fc700 1 -- 172.21.15.99:0/4053063955 wait complete. 2023-10-08T00:16:05.859 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-10-08T00:16:05.859 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-08T00:16:05.975 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.975+0000 7fe3d4d94700 1 Processor -- start 2023-10-08T00:16:05.976 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.975+0000 7fe3d4d94700 1 -- start start 2023-10-08T00:16:05.979 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.976+0000 7fe3d4d94700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fe3d0157870 0x7fe3d015b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:05.979 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.976+0000 7fe3d4d94700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe3d015a620 0x7fe3d015d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:05.979 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.976+0000 7fe3d4d94700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3d015cbb0 con 0x7fe3d015d580 2023-10-08T00:16:05.979 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.976+0000 7fe3d4d94700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3d015bfa0 con 0x7fe3d015a620 2023-10-08T00:16:05.979 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.976+0000 7fe3d4d94700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3d015c280 con 0x7fe3d0157870 2023-10-08T00:16:05.979 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.976+0000 7fe3cdd9b700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe3d015a620 0x7fe3d015d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:05.979 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.976+0000 7fe3ced9d700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fe3d015d580 0x7fe3d0158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:43672/0 (socket says 172.21.15.99:43672) 2023-10-08T00:16:05.979 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.976+0000 7fe3ced9d700 1 -- 172.21.15.99:0/2418243151 learned_addr learned my addr 172.21.15.99:0/2418243151 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:05.979 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.976+0000 7fe3ce59c700 1 --2- 172.21.15.99:0/2418243151 >> v2:172.21.15.191:3300/0 conn(0x7fe3d0157870 0x7fe3d015b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:05.979 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.976+0000 7fe3cdd9b700 1 -- 172.21.15.99:0/2418243151 >> v2:172.21.15.191:3300/0 conn(0x7fe3d0157870 msgr2=0x7fe3d015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:05.980 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.976+0000 7fe3cdd9b700 1 --2- 172.21.15.99:0/2418243151 >> v2:172.21.15.191:3300/0 conn(0x7fe3d0157870 0x7fe3d015b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:05.980 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.976+0000 7fe3cdd9b700 1 -- 172.21.15.99:0/2418243151 >> v1:172.21.15.99:6789/0 conn(0x7fe3d015d580 legacy=0x7fe3d0158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:05.980 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.976+0000 7fe3cdd9b700 1 -- 172.21.15.99:0/2418243151 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3d015c560 con 0x7fe3d015a620 2023-10-08T00:16:05.980 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.976+0000 7fe3ce59c700 1 --2- 172.21.15.99:0/2418243151 >> v2:172.21.15.191:3300/0 conn(0x7fe3d0157870 0x7fe3d015b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:16:05.980 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.976+0000 7fe3cdd9b700 1 --2- 172.21.15.99:0/2418243151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3d015a620 0x7fe3d015d040 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0069d80 tx=0x7fe3c40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd2425af6bbf268e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:05.980 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.976+0000 7fe3cd59a700 1 -- 172.21.15.99:0/2418243151 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c400b6b0 con 0x7fe3d015a620 2023-10-08T00:16:05.980 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.977+0000 7fe3cd59a700 1 -- 172.21.15.99:0/2418243151 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3c400b810 con 0x7fe3d015a620 2023-10-08T00:16:05.980 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.977+0000 7fe3cd59a700 1 -- 172.21.15.99:0/2418243151 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c400bbb0 con 0x7fe3d015a620 2023-10-08T00:16:05.981 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.977+0000 7fe3d4d94700 1 -- 172.21.15.99:0/2418243151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3d015a620 msgr2=0x7fe3d015d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:05.981 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.977+0000 7fe3d4d94700 1 --2- 172.21.15.99:0/2418243151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3d015a620 0x7fe3d015d040 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0069d80 tx=0x7fe3c40091a0 comp rx=0 tx=0).stop 2023-10-08T00:16:05.981 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.977+0000 7fe3d4d94700 1 -- 172.21.15.99:0/2418243151 shutdown_connections 2023-10-08T00:16:05.981 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.977+0000 7fe3d4d94700 1 --2- 172.21.15.99:0/2418243151 >> v2:172.21.15.191:3300/0 conn(0x7fe3d0157870 0x7fe3d015b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:05.981 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.977+0000 7fe3d4d94700 1 --2- 172.21.15.99:0/2418243151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3d015a620 0x7fe3d015d040 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:05.981 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.977+0000 7fe3d4d94700 1 -- 172.21.15.99:0/2418243151 >> 172.21.15.99:0/2418243151 conn(0x7fe3d0068ce0 msgr2=0x7fe3d00690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:05.981 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.977+0000 7fe3d4d94700 1 -- 172.21.15.99:0/2418243151 shutdown_connections 2023-10-08T00:16:05.981 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.977+0000 7fe3d4d94700 1 -- 172.21.15.99:0/2418243151 wait complete. 2023-10-08T00:16:05.981 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.978+0000 7fe3d4d94700 1 Processor -- start 2023-10-08T00:16:05.981 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.978+0000 7fe3d4d94700 1 -- start start 2023-10-08T00:16:05.986 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.979+0000 7fe3d4d94700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3d0157870 0x7fe3d014e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:05.986 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.979+0000 7fe3d4d94700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe3d015a620 0x7fe3d014c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:05.986 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.979+0000 7fe3d4d94700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe3d015d580 0x7fe3d0152ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:05.986 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.979+0000 7fe3d4d94700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d016ab50 con 0x7fe3d0157870 2023-10-08T00:16:05.986 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.979+0000 7fe3d4d94700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d0150ea0 con 0x7fe3d015a620 2023-10-08T00:16:05.986 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.979+0000 7fe3d4d94700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d014ce20 con 0x7fe3d015d580 2023-10-08T00:16:05.986 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.979+0000 7fe3ce59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3d0157870 0x7fe3d014e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:05.986 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.979+0000 7fe3ce59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3d0157870 0x7fe3d014e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46070/0 (socket says 172.21.15.99:46070) 2023-10-08T00:16:05.987 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.979+0000 7fe3ce59c700 1 -- 172.21.15.99:0/3605223208 learned_addr learned my addr 172.21.15.99:0/3605223208 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:05.987 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.979+0000 7fe3cdd9b700 1 --2- 172.21.15.99:0/3605223208 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe3d015a620 0x7fe3d014c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:05.987 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.979+0000 7fe3ced9d700 1 --2- 172.21.15.99:0/3605223208 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe3d015d580 0x7fe3d0152ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:05.987 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.979+0000 7fe3ce59c700 1 -- 172.21.15.99:0/3605223208 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe3d015d580 msgr2=0x7fe3d0152ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:05.987 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.979+0000 7fe3ce59c700 1 --2- 172.21.15.99:0/3605223208 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe3d015d580 0x7fe3d0152ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:05.987 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.979+0000 7fe3ce59c700 1 -- 172.21.15.99:0/3605223208 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe3d015a620 msgr2=0x7fe3d014c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:05.987 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.979+0000 7fe3ce59c700 1 --2- 172.21.15.99:0/3605223208 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe3d015a620 0x7fe3d014c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:05.987 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.979+0000 7fe3ce59c700 1 -- 172.21.15.99:0/3605223208 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c4009420 con 0x7fe3d0157870 2023-10-08T00:16:05.987 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.979+0000 7fe3ce59c700 1 --2- 172.21.15.99:0/3605223208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3d0157870 0x7fe3d014e8b0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fe3c800c830 tx=0x7fe3c8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:05.987 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.980+0000 7fe3be7fc700 1 -- 172.21.15.99:0/3605223208 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c8011600 con 0x7fe3d0157870 2023-10-08T00:16:05.987 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.980+0000 7fe3be7fc700 1 -- 172.21.15.99:0/3605223208 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3c8011760 con 0x7fe3d0157870 2023-10-08T00:16:05.988 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.980+0000 7fe3be7fc700 1 -- 172.21.15.99:0/3605223208 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c8011b20 con 0x7fe3d0157870 2023-10-08T00:16:05.988 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.980+0000 7fe3d4d94700 1 -- 172.21.15.99:0/3605223208 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3d0153780 con 0x7fe3d0157870 2023-10-08T00:16:05.988 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.980+0000 7fe3d4d94700 1 -- 172.21.15.99:0/3605223208 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3d0150b30 con 0x7fe3d0157870 2023-10-08T00:16:05.988 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.980+0000 7fe3d4d94700 1 -- 172.21.15.99:0/3605223208 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3d0153c80 con 0x7fe3d0157870 2023-10-08T00:16:05.988 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.981+0000 7fe3be7fc700 1 -- 172.21.15.99:0/3605223208 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe3c800a3e0 con 0x7fe3d0157870 2023-10-08T00:16:05.988 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.981+0000 7fe3be7fc700 1 --2- 172.21.15.99:0/3605223208 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe3b0066f30 0x7fe3b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:05.988 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.982+0000 7fe3be7fc700 1 -- 172.21.15.99:0/3605223208 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe3c8084620 con 0x7fe3d0157870 2023-10-08T00:16:05.988 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.982+0000 7fe3cdd9b700 1 --2- 172.21.15.99:0/3605223208 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe3b0066f30 0x7fe3b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:05.988 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.982+0000 7fe3cdd9b700 1 --2- 172.21.15.99:0/3605223208 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe3b0066f30 0x7fe3b00693e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe3d015aa20 tx=0x7fe3c4017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:05.989 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:05.986+0000 7fe3be7fc700 1 -- 172.21.15.99:0/3605223208 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe3c8051890 con 0x7fe3d0157870 2023-10-08T00:16:06.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.171+0000 7fe3d4d94700 1 -- 172.21.15.99:0/3605223208 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fe3d004c3c0 con 0x7fe3d0157870 2023-10-08T00:16:06.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.172+0000 7fe3be7fc700 1 -- 172.21.15.99:0/3605223208 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v112) v1 ==== 81+0+32275 (secure 0 0 0) 0x7fe3c8054eb0 con 0x7fe3d0157870 2023-10-08T00:16:06.173 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:16:06.173 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:16:06.173 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 112, 2023-10-08T00:16:06.173 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", 2023-10-08T00:16:06.173 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2023-10-08T00:10:36.303601+0000", 2023-10-08T00:16:06.173 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2023-10-08T00:16:05.407718+0000", 2023-10-08T00:16:06.173 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2023-10-08T00:10:41.709171+0000", 2023-10-08T00:16:06.173 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2023-10-08T00:10:38.690404+0000", 2023-10-08T00:16:06.174 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-08T00:16:06.174 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2023-10-08T00:16:06.174 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2023-10-08T00:16:06.174 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2023-10-08T00:16:06.174 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2023-10-08T00:16:06.174 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2023-10-08T00:16:06.174 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2023-10-08T00:16:06.174 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:16:06.174 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2023-10-08T00:16:06.174 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2023-10-08T00:16:06.175 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-08T00:16:06.175 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-08T00:16:06.175 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2023-10-08T00:16:06.175 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 13, 2023-10-08T00:16:06.175 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2023-10-08T00:16:06.175 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2023-10-08T00:16:06.175 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "jewel", 2023-10-08T00:16:06.175 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "quincy", 2023-10-08T00:16:06.175 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2023-10-08T00:16:06.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.175 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2023-10-08T00:16:06.176 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2023-10-08T00:16:06.176 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:10:42.063344+0000", 2023-10-08T00:16:06.176 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:16:06.176 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:16:06.176 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:16:06.176 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:16:06.176 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:16:06.176 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:16:06.176 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:16:06.176 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:16:06.176 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:16:06.176 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:16:06.177 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:16:06.177 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:16:06.177 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2023-10-08T00:16:06.177 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2023-10-08T00:16:06.177 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2023-10-08T00:16:06.177 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2023-10-08T00:16:06.177 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2023-10-08T00:16:06.177 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:16:06.177 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:16:06.177 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:16:06.177 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:16:06.178 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:16:06.178 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:16:06.178 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:16:06.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.178 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "112", 2023-10-08T00:16:06.178 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:16:06.178 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:16:06.178 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:16:06.178 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:16:06.178 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:16:06.178 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:16:06.179 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:16:06.179 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:16:06.179 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:16:06.179 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:16:06.179 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:16:06.179 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:16:06.179 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:16:06.179 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:16:06.179 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:16:06.179 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:16:06.180 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:16:06.180 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:16:06.180 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:16:06.180 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:16:06.180 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:16:06.180 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:16:06.180 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:16:06.180 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:16:06.180 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:16:06.180 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:16:06.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.181 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:16:06.181 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:16:06.181 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:16:06.181 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:16:06.181 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:16:06.181 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:16:06.182 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:16:06.182 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:16:06.182 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:16:06.182 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:16:06.182 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:16:06.182 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:16:06.182 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2023-10-08T00:16:06.182 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2023-10-08T00:16:06.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.183 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:16:06.183 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2023-10-08T00:16:06.183 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:06.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.183 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 12, 2023-10-08T00:16:06.183 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2023-10-08T00:16:06.183 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:15:38.848324+0000", 2023-10-08T00:16:06.183 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:16:06.183 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:16:06.183 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:16:06.184 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:16:06.184 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:16:06.184 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:16:06.184 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:16:06.184 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:16:06.184 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:16:06.184 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:16:06.184 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:16:06.185 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:16:06.185 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2023-10-08T00:16:06.185 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2023-10-08T00:16:06.185 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2023-10-08T00:16:06.185 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2023-10-08T00:16:06.186 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2023-10-08T00:16:06.186 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:16:06.186 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:16:06.186 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:16:06.186 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:16:06.186 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:16:06.186 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:16:06.186 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:16:06.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.187 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "112", 2023-10-08T00:16:06.187 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:16:06.187 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:16:06.187 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:16:06.187 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:16:06.187 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:16:06.187 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:16:06.187 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:16:06.187 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:16:06.188 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:16:06.188 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:16:06.188 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:16:06.188 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:16:06.188 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:16:06.188 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:16:06.188 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:16:06.188 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:16:06.188 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:16:06.188 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:16:06.189 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:16:06.189 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:16:06.189 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:16:06.189 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:16:06.189 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:16:06.189 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:16:06.189 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:16:06.189 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:16:06.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.189 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:16:06.189 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:16:06.189 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:16:06.190 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:16:06.190 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:16:06.190 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:16:06.190 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:16:06.190 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:16:06.190 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:16:06.190 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:16:06.190 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:16:06.190 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:16:06.190 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2023-10-08T00:16:06.190 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2023-10-08T00:16:06.190 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2023-10-08T00:16:06.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.191 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:16:06.191 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:16:06.191 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2023-10-08T00:16:06.191 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:06.191 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:06.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.191 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 13, 2023-10-08T00:16:06.191 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2023-10-08T00:16:06.192 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:15:39.561379+0000", 2023-10-08T00:16:06.192 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:16:06.192 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:16:06.192 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:16:06.192 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:16:06.192 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:16:06.192 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:16:06.192 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:16:06.192 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:16:06.193 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:16:06.193 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:16:06.193 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:16:06.193 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:16:06.193 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2023-10-08T00:16:06.193 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2023-10-08T00:16:06.193 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2023-10-08T00:16:06.193 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2023-10-08T00:16:06.193 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2023-10-08T00:16:06.193 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:16:06.193 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:16:06.193 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:16:06.194 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:16:06.194 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:16:06.194 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:16:06.194 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:16:06.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.194 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "112", 2023-10-08T00:16:06.194 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:16:06.194 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:16:06.194 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:16:06.194 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:16:06.194 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:16:06.195 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:16:06.195 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:16:06.195 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:16:06.195 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:16:06.195 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:16:06.195 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:16:06.195 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:16:06.195 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:16:06.195 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:16:06.195 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:16:06.195 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:16:06.195 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:16:06.195 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:16:06.196 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:16:06.196 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:16:06.196 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:16:06.196 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:16:06.196 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:16:06.196 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:16:06.196 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:16:06.196 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:16:06.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.196 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:16:06.196 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:16:06.197 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:16:06.197 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:16:06.197 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:16:06.197 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:16:06.197 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:16:06.197 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:16:06.197 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:16:06.197 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:16:06.197 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:16:06.197 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:16:06.197 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2023-10-08T00:16:06.197 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2023-10-08T00:16:06.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.198 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:16:06.198 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:16:06.198 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2023-10-08T00:16:06.198 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:06.198 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:06.198 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:06.198 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:16:06.198 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2023-10-08T00:16:06.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.198 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:16:06.199 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:16:06.199 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:16:06.199 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:16:06.199 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:16:06.199 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:16:06.199 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:16:06.199 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:16:06.199 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:16:06.199 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 102, 2023-10-08T00:16:06.199 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:16:06.199 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:16:06.200 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:16:06.200 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:06.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.200 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:06.200 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2023-10-08T00:16:06.200 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:16:06.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.200 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:06.200 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2023-10-08T00:16:06.200 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:16:06.200 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:06.201 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:06.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.201 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:16:06.201 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:06.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.201 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:06.201 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2023-10-08T00:16:06.201 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:16:06.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.201 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:06.202 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2023-10-08T00:16:06.202 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:16:06.202 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:06.202 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:06.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.202 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:16:06.202 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:06.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.202 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:06.202 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2023-10-08T00:16:06.202 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:16:06.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.203 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:06.203 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2023-10-08T00:16:06.203 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:16:06.203 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:06.203 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:06.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.203 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:16:06.203 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:06.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.204 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:06.204 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2023-10-08T00:16:06.204 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:16:06.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.204 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:06.204 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2023-10-08T00:16:06.204 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:16:06.204 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:06.204 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:06.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.205 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/86365", 2023-10-08T00:16:06.205 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/86365", 2023-10-08T00:16:06.205 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/86365", 2023-10-08T00:16:06.205 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/86365", 2023-10-08T00:16:06.205 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:16:06.205 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:16:06.205 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:16:06.205 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:06.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.205 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:16:06.205 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:16:06.206 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:16:06.206 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:16:06.206 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:16:06.206 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:16:06.206 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:16:06.206 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:16:06.206 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:16:06.206 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 102, 2023-10-08T00:16:06.206 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:16:06.206 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:16:06.206 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:16:06.207 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:06.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.207 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:06.207 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2023-10-08T00:16:06.207 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:16:06.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.207 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:06.207 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2023-10-08T00:16:06.207 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:16:06.207 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:06.208 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:06.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.208 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:16:06.208 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:06.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.208 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:06.208 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2023-10-08T00:16:06.208 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:16:06.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.208 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:06.208 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2023-10-08T00:16:06.209 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:16:06.209 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:06.209 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:06.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.209 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:16:06.209 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:06.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.209 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:06.209 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2023-10-08T00:16:06.209 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:16:06.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.210 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:06.210 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2023-10-08T00:16:06.210 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:16:06.210 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:06.210 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:06.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.210 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:16:06.210 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:06.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.210 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:06.210 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2023-10-08T00:16:06.211 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:16:06.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.211 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:06.211 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2023-10-08T00:16:06.211 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:16:06.211 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:06.211 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:06.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.211 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/86366", 2023-10-08T00:16:06.211 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/86366", 2023-10-08T00:16:06.212 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/86366", 2023-10-08T00:16:06.212 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/86366", 2023-10-08T00:16:06.212 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:16:06.212 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:16:06.212 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:16:06.212 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:06.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.212 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:16:06.212 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:16:06.212 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:16:06.212 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:16:06.213 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:16:06.213 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:16:06.213 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:16:06.213 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:16:06.213 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:16:06.213 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 102, 2023-10-08T00:16:06.213 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:16:06.213 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:16:06.213 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:16:06.213 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:06.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.213 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:06.214 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2023-10-08T00:16:06.214 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:16:06.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.214 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.214 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:06.214 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2023-10-08T00:16:06.214 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:16:06.214 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:06.214 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:06.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.214 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:16:06.214 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:06.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.215 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:06.215 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2023-10-08T00:16:06.215 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:16:06.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.215 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:06.215 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2023-10-08T00:16:06.215 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:16:06.215 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:06.215 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:06.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.216 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:16:06.216 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:06.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.216 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:06.216 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2023-10-08T00:16:06.216 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:16:06.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.216 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:06.216 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2023-10-08T00:16:06.216 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:16:06.217 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:06.217 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:06.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.217 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:16:06.217 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:06.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.217 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:06.217 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2023-10-08T00:16:06.217 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:16:06.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.217 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:06.218 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2023-10-08T00:16:06.218 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:16:06.218 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:06.218 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:06.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.218 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6817/86382", 2023-10-08T00:16:06.218 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/86382", 2023-10-08T00:16:06.218 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/86382", 2023-10-08T00:16:06.218 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/86382", 2023-10-08T00:16:06.218 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:16:06.218 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:16:06.218 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:16:06.218 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:06.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.219 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:16:06.219 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:16:06.219 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:16:06.219 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:16:06.219 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:16:06.219 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:16:06.219 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:16:06.219 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:16:06.219 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:16:06.219 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 102, 2023-10-08T00:16:06.219 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:16:06.220 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:16:06.221 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:16:06.221 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:06.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.221 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:06.221 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2023-10-08T00:16:06.221 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:16:06.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.222 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:06.222 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2023-10-08T00:16:06.222 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:16:06.222 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:06.222 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:06.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.222 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:16:06.222 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:06.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.222 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:06.222 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2023-10-08T00:16:06.222 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:16:06.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.223 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:06.223 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2023-10-08T00:16:06.223 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:16:06.223 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:06.223 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:06.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.223 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:16:06.223 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:06.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.223 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:06.224 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2023-10-08T00:16:06.224 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:16:06.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.224 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.224 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:06.224 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2023-10-08T00:16:06.224 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:16:06.224 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:06.224 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:06.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.224 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:16:06.225 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:06.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.225 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:06.225 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2023-10-08T00:16:06.225 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:16:06.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.225 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:06.225 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2023-10-08T00:16:06.225 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:16:06.225 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:06.226 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:06.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.226 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6820/86393", 2023-10-08T00:16:06.226 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/86393", 2023-10-08T00:16:06.226 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/86393", 2023-10-08T00:16:06.226 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/86393", 2023-10-08T00:16:06.226 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:16:06.226 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:16:06.226 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:16:06.226 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:06.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.227 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:16:06.227 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:16:06.227 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:16:06.227 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:16:06.227 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:16:06.227 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:16:06.227 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:16:06.227 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:16:06.227 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:16:06.227 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 102, 2023-10-08T00:16:06.228 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:16:06.228 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:16:06.228 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:16:06.228 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:06.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.228 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:06.228 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6800", 2023-10-08T00:16:06.228 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:16:06.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.228 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:06.229 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6801", 2023-10-08T00:16:06.229 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:16:06.229 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:06.229 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:06.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.229 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:16:06.229 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:06.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.229 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:06.229 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6802", 2023-10-08T00:16:06.230 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:16:06.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.230 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:06.230 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6803", 2023-10-08T00:16:06.230 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:16:06.230 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:06.230 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:06.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.230 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:16:06.230 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:06.231 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.231 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:06.231 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6806", 2023-10-08T00:16:06.231 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:16:06.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.231 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.231 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:06.231 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6807", 2023-10-08T00:16:06.231 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:16:06.231 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:06.231 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:06.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.232 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:16:06.232 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:06.232 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.232 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:06.232 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6804", 2023-10-08T00:16:06.232 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:16:06.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.232 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.232 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:06.232 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6805", 2023-10-08T00:16:06.233 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:16:06.233 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:06.233 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:06.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.233 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6801/86570", 2023-10-08T00:16:06.233 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6803/86570", 2023-10-08T00:16:06.233 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6807/86570", 2023-10-08T00:16:06.233 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6805/86570", 2023-10-08T00:16:06.233 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:16:06.233 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:16:06.234 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:16:06.234 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:06.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.234 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:16:06.234 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:16:06.234 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:16:06.234 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:16:06.234 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:16:06.234 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:16:06.234 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:16:06.235 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:16:06.235 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:16:06.235 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 102, 2023-10-08T00:16:06.235 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:16:06.235 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:16:06.235 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:16:06.235 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:06.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.235 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:06.235 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6816", 2023-10-08T00:16:06.235 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:16:06.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.236 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:06.236 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6818", 2023-10-08T00:16:06.236 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:16:06.236 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:06.236 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:06.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.236 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:16:06.236 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:06.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.237 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:06.237 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6820", 2023-10-08T00:16:06.237 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:16:06.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.237 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:06.237 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6821", 2023-10-08T00:16:06.237 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:16:06.237 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:06.237 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:06.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.238 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:16:06.238 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:06.238 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.238 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:06.238 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6826", 2023-10-08T00:16:06.238 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:16:06.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.238 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.238 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:06.238 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6828", 2023-10-08T00:16:06.238 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:16:06.239 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:06.239 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:06.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.239 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:16:06.239 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:06.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.239 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:06.239 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6823", 2023-10-08T00:16:06.239 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:16:06.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.240 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.240 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:06.240 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6825", 2023-10-08T00:16:06.240 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:16:06.240 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:06.240 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:06.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.240 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6818/86608", 2023-10-08T00:16:06.240 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6821/86608", 2023-10-08T00:16:06.240 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6828/86608", 2023-10-08T00:16:06.240 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6825/86608", 2023-10-08T00:16:06.241 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:16:06.241 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:16:06.241 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:16:06.241 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:06.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.241 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:16:06.241 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:16:06.241 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:16:06.241 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:16:06.241 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:16:06.242 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:16:06.242 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:16:06.242 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:16:06.242 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:16:06.242 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 102, 2023-10-08T00:16:06.242 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:16:06.242 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:16:06.242 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:16:06.242 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:06.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.242 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:06.243 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6808", 2023-10-08T00:16:06.243 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:16:06.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.243 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:06.243 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6809", 2023-10-08T00:16:06.243 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:16:06.243 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:06.243 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:06.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.243 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:16:06.244 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:06.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.244 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:06.244 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6810", 2023-10-08T00:16:06.244 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:16:06.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.244 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:06.244 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6811", 2023-10-08T00:16:06.244 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:16:06.245 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:06.245 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:06.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.245 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:16:06.245 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:06.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.245 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:06.245 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6814", 2023-10-08T00:16:06.245 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:16:06.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.246 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:06.246 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6815", 2023-10-08T00:16:06.246 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:16:06.246 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:06.246 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:06.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.246 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:16:06.246 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:06.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.246 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:06.246 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6812", 2023-10-08T00:16:06.247 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:16:06.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.247 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:06.247 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6813", 2023-10-08T00:16:06.247 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:16:06.247 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:06.247 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:06.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.247 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6809/86571", 2023-10-08T00:16:06.247 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6811/86571", 2023-10-08T00:16:06.248 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6815/86571", 2023-10-08T00:16:06.248 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6813/86571", 2023-10-08T00:16:06.248 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:16:06.248 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:16:06.248 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:16:06.248 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:06.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.248 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:16:06.248 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:16:06.248 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:16:06.249 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:16:06.249 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:16:06.249 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:16:06.249 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:16:06.249 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:16:06.249 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:16:06.249 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 102, 2023-10-08T00:16:06.249 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:16:06.249 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:16:06.249 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:16:06.249 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:06.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.250 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:06.250 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6817", 2023-10-08T00:16:06.250 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:16:06.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.250 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:06.250 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6819", 2023-10-08T00:16:06.250 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:16:06.250 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:06.251 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:06.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.251 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:16:06.251 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:06.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.251 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:06.251 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6822", 2023-10-08T00:16:06.251 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:16:06.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.251 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:06.252 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6824", 2023-10-08T00:16:06.252 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:16:06.252 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:06.252 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:06.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.252 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:16:06.252 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:06.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.252 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:06.252 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6830", 2023-10-08T00:16:06.252 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:16:06.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.253 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:06.253 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6831", 2023-10-08T00:16:06.253 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:16:06.253 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:06.253 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:06.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.253 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:16:06.253 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:16:06.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.253 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:16:06.254 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6827", 2023-10-08T00:16:06.254 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:16:06.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.254 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:16:06.254 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6829", 2023-10-08T00:16:06.254 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:16:06.254 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:06.254 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:06.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.255 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6819/86612", 2023-10-08T00:16:06.255 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6824/86612", 2023-10-08T00:16:06.255 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6831/86612", 2023-10-08T00:16:06.255 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6829/86612", 2023-10-08T00:16:06.255 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:16:06.255 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:16:06.255 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:16:06.255 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:06.255 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:06.255 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:16:06.256 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2023-10-08T00:16:06.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.256 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:16:06.256 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:16:06.256 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:16:06.256 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:16:06.256 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:16:06.256 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:16:06.256 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.905615+0000", 2023-10-08T00:16:06.256 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:16:06.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.257 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:16:06.257 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:16:06.257 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:16:06.257 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:16:06.257 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:16:06.257 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:16:06.257 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.880709+0000", 2023-10-08T00:16:06.257 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:16:06.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.258 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:16:06.258 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:16:06.258 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:16:06.258 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:16:06.258 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:16:06.258 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:16:06.258 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.874704+0000", 2023-10-08T00:16:06.258 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:16:06.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.258 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:16:06.259 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:16:06.259 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:16:06.259 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:16:06.259 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:16:06.259 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:16:06.259 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.899604+0000", 2023-10-08T00:16:06.259 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:16:06.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.259 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:16:06.259 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:16:06.260 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:16:06.260 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:16:06.260 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:16:06.260 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:16:06.260 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.844371+0000", 2023-10-08T00:16:06.260 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:16:06.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.260 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:16:06.260 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:16:06.260 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:16:06.261 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:16:06.261 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:16:06.261 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:16:06.261 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.794810+0000", 2023-10-08T00:16:06.261 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:16:06.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.261 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:16:06.261 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:16:06.261 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:16:06.261 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:16:06.262 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:16:06.262 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:16:06.262 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.859474+0000", 2023-10-08T00:16:06.262 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:16:06.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:06.262 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:16:06.262 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:16:06.262 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:16:06.262 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:16:06.262 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:16:06.263 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:16:06.263 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.893130+0000", 2023-10-08T00:16:06.263 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:16:06.263 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:06.263 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:16:06.263 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2023-10-08T00:16:06.263 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [], 2023-10-08T00:16:06.263 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2023-10-08T00:16:06.263 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2023-10-08T00:16:06.263 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2023-10-08T00:16:06.263 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2023-10-08T00:16:06.264 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2023-10-08T00:16:06.264 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2023-10-08T00:16:06.264 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2023-10-08T00:16:06.264 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2023-10-08T00:16:06.264 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2023-10-08T00:16:06.264 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2023-10-08T00:16:06.264 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2023-10-08T00:16:06.264 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:06.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:06.264 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2023-10-08T00:16:06.264 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2023-10-08T00:16:06.265 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2023-10-08T00:16:06.265 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2023-10-08T00:16:06.265 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2023-10-08T00:16:06.265 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2023-10-08T00:16:06.265 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2023-10-08T00:16:06.265 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2023-10-08T00:16:06.265 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2023-10-08T00:16:06.265 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2023-10-08T00:16:06.265 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2023-10-08T00:16:06.265 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:06.265 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:16:06.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.174+0000 7fe3d4d94700 1 -- 172.21.15.99:0/3605223208 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe3b0066f30 msgr2=0x7fe3b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:06.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.174+0000 7fe3d4d94700 1 --2- 172.21.15.99:0/3605223208 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe3b0066f30 0x7fe3b00693e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe3d015aa20 tx=0x7fe3c4017000 comp rx=0 tx=0).stop 2023-10-08T00:16:06.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.174+0000 7fe3d4d94700 1 -- 172.21.15.99:0/3605223208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3d0157870 msgr2=0x7fe3d014e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:06.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.174+0000 7fe3d4d94700 1 --2- 172.21.15.99:0/3605223208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3d0157870 0x7fe3d014e8b0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fe3c800c830 tx=0x7fe3c8008040 comp rx=0 tx=0).stop 2023-10-08T00:16:06.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.174+0000 7fe3d4d94700 1 -- 172.21.15.99:0/3605223208 shutdown_connections 2023-10-08T00:16:06.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.174+0000 7fe3d4d94700 1 --2- 172.21.15.99:0/3605223208 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe3b0066f30 0x7fe3b00693e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:06.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.174+0000 7fe3d4d94700 1 --2- 172.21.15.99:0/3605223208 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe3d015d580 0x7fe3d0152ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:06.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.174+0000 7fe3d4d94700 1 --2- 172.21.15.99:0/3605223208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3d0157870 0x7fe3d014e8b0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:06.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.174+0000 7fe3d4d94700 1 --2- 172.21.15.99:0/3605223208 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe3d015a620 0x7fe3d014c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:06.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.174+0000 7fe3d4d94700 1 -- 172.21.15.99:0/3605223208 >> 172.21.15.99:0/3605223208 conn(0x7fe3d0068ce0 msgr2=0x7fe3d01584c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:06.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.174+0000 7fe3d4d94700 1 -- 172.21.15.99:0/3605223208 shutdown_connections 2023-10-08T00:16:06.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.174+0000 7fe3d4d94700 1 -- 172.21.15.99:0/3605223208 wait complete. 2023-10-08T00:16:06.269 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:16:06.384 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.383+0000 7fba41814700 1 Processor -- start 2023-10-08T00:16:06.384 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.383+0000 7fba41814700 1 -- start start 2023-10-08T00:16:06.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.384+0000 7fba41814700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fba3c0cd6e0 0x7fba3c0cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:06.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.384+0000 7fba41814700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fba3c15cbd0 0x7fba3c15d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:06.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.384+0000 7fba41814700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba3c0cb7c0 con 0x7fba3c0cdab0 2023-10-08T00:16:06.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.384+0000 7fba41814700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba3c15e850 con 0x7fba3c15cbd0 2023-10-08T00:16:06.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.384+0000 7fba41814700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba3c0cd030 con 0x7fba3c0cd6e0 2023-10-08T00:16:06.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.384+0000 7fba41013700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fba3c15cbd0 0x7fba3c15d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:06.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.384+0000 7fba41013700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fba3c15cbd0 0x7fba3c15d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46072/0 (socket says 172.21.15.99:46072) 2023-10-08T00:16:06.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.384+0000 7fba41013700 1 -- 172.21.15.99:0/40313618 learned_addr learned my addr 172.21.15.99:0/40313618 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:06.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.384+0000 7fba40812700 1 --2- 172.21.15.99:0/40313618 >> v2:172.21.15.191:3300/0 conn(0x7fba3c0cd6e0 0x7fba3c0cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:06.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.384+0000 7fba41013700 1 -- 172.21.15.99:0/40313618 >> v2:172.21.15.191:3300/0 conn(0x7fba3c0cd6e0 msgr2=0x7fba3c0cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:06.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.384+0000 7fba41013700 1 --2- 172.21.15.99:0/40313618 >> v2:172.21.15.191:3300/0 conn(0x7fba3c0cd6e0 0x7fba3c0cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:06.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.384+0000 7fba41013700 1 -- 172.21.15.99:0/40313618 >> v1:172.21.15.99:6789/0 conn(0x7fba3c0cdab0 legacy=0x7fba3c15d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:06.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.384+0000 7fba41013700 1 -- 172.21.15.99:0/40313618 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba3c15e290 con 0x7fba3c15cbd0 2023-10-08T00:16:06.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.384+0000 7fba337fe700 1 -- 172.21.15.99:0/40313618 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1592693433 0 0) 0x7fba3c0cb7c0 con 0x7fba3c0cdab0 2023-10-08T00:16:06.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.385+0000 7fba41013700 1 --2- 172.21.15.99:0/40313618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba3c15cbd0 0x7fba3c15d050 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fba3c06b730 tx=0x7fba34009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0089c24ca67da0c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:06.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.385+0000 7fba337fe700 1 -- 172.21.15.99:0/40313618 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3400e070 con 0x7fba3c15cbd0 2023-10-08T00:16:06.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.385+0000 7fba337fe700 1 -- 172.21.15.99:0/40313618 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba3400ba50 con 0x7fba3c15cbd0 2023-10-08T00:16:06.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.385+0000 7fba337fe700 1 -- 172.21.15.99:0/40313618 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba340092e0 con 0x7fba3c15cbd0 2023-10-08T00:16:06.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.385+0000 7fba41814700 1 -- 172.21.15.99:0/40313618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba3c15cbd0 msgr2=0x7fba3c15d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:06.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.385+0000 7fba41814700 1 --2- 172.21.15.99:0/40313618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba3c15cbd0 0x7fba3c15d050 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fba3c06b730 tx=0x7fba34009840 comp rx=0 tx=0).stop 2023-10-08T00:16:06.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.385+0000 7fba41814700 1 -- 172.21.15.99:0/40313618 shutdown_connections 2023-10-08T00:16:06.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.385+0000 7fba41814700 1 --2- 172.21.15.99:0/40313618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba3c15cbd0 0x7fba3c15d050 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:06.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.385+0000 7fba41814700 1 --2- 172.21.15.99:0/40313618 >> v2:172.21.15.191:3300/0 conn(0x7fba3c0cd6e0 0x7fba3c0cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:06.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.385+0000 7fba41814700 1 -- 172.21.15.99:0/40313618 >> 172.21.15.99:0/40313618 conn(0x7fba3c067660 msgr2=0x7fba3c067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:06.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.385+0000 7fba41814700 1 -- 172.21.15.99:0/40313618 shutdown_connections 2023-10-08T00:16:06.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.385+0000 7fba41814700 1 -- 172.21.15.99:0/40313618 wait complete. 2023-10-08T00:16:06.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.386+0000 7fba41814700 1 Processor -- start 2023-10-08T00:16:06.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.387+0000 7fba41814700 1 -- start start 2023-10-08T00:16:06.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.387+0000 7fba41814700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba3c0cd6e0 0x7fba3c14c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:06.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.387+0000 7fba41814700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba3c0cdab0 0x7fba3c0b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:06.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.387+0000 7fba41814700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba3c15cbd0 0x7fba3c155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:06.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.387+0000 7fba41814700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3c0b31b0 con 0x7fba3c0cd6e0 2023-10-08T00:16:06.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.387+0000 7fba41814700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3c0b25a0 con 0x7fba3c15cbd0 2023-10-08T00:16:06.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.387+0000 7fba41814700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba3c0b2880 con 0x7fba3c0cdab0 2023-10-08T00:16:06.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.387+0000 7fba40812700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba3c0cd6e0 0x7fba3c14c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:06.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.387+0000 7fba40812700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba3c0cd6e0 0x7fba3c14c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46088/0 (socket says 172.21.15.99:46088) 2023-10-08T00:16:06.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.387+0000 7fba40812700 1 -- 172.21.15.99:0/1635704523 learned_addr learned my addr 172.21.15.99:0/1635704523 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:06.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.388+0000 7fba41013700 1 --2- 172.21.15.99:0/1635704523 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba3c15cbd0 0x7fba3c155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:06.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.388+0000 7fba33fff700 1 --2- 172.21.15.99:0/1635704523 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba3c0cdab0 0x7fba3c0b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:06.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.388+0000 7fba40812700 1 -- 172.21.15.99:0/1635704523 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba3c0cdab0 msgr2=0x7fba3c0b54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:06.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.388+0000 7fba40812700 1 --2- 172.21.15.99:0/1635704523 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba3c0cdab0 0x7fba3c0b54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:06.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.388+0000 7fba40812700 1 -- 172.21.15.99:0/1635704523 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba3c15cbd0 msgr2=0x7fba3c155870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:06.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.388+0000 7fba40812700 1 --2- 172.21.15.99:0/1635704523 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba3c15cbd0 0x7fba3c155870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:06.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.388+0000 7fba40812700 1 -- 172.21.15.99:0/1635704523 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba34009ae0 con 0x7fba3c0cd6e0 2023-10-08T00:16:06.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.388+0000 7fba33fff700 1 --2- 172.21.15.99:0/1635704523 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba3c0cdab0 0x7fba3c0b54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:16:06.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.388+0000 7fba40812700 1 --2- 172.21.15.99:0/1635704523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba3c0cd6e0 0x7fba3c14c7e0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fba3800d890 tx=0x7fba38008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:06.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.388+0000 7fba30ff9700 1 -- 172.21.15.99:0/1635704523 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3800b600 con 0x7fba3c0cd6e0 2023-10-08T00:16:06.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.388+0000 7fba30ff9700 1 -- 172.21.15.99:0/1635704523 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba3800b760 con 0x7fba3c0cd6e0 2023-10-08T00:16:06.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.388+0000 7fba41814700 1 -- 172.21.15.99:0/1635704523 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba3c0b2b60 con 0x7fba3c0cd6e0 2023-10-08T00:16:06.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.388+0000 7fba30ff9700 1 -- 172.21.15.99:0/1635704523 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3800bae0 con 0x7fba3c0cd6e0 2023-10-08T00:16:06.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.388+0000 7fba41814700 1 -- 172.21.15.99:0/1635704523 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba3c14cd20 con 0x7fba3c0cd6e0 2023-10-08T00:16:06.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.389+0000 7fba41814700 1 -- 172.21.15.99:0/1635704523 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba3c0b3490 con 0x7fba3c0cd6e0 2023-10-08T00:16:06.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.389+0000 7fba30ff9700 1 -- 172.21.15.99:0/1635704523 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fba3801e460 con 0x7fba3c0cd6e0 2023-10-08T00:16:06.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.390+0000 7fba30ff9700 1 --2- 172.21.15.99:0/1635704523 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fba28066f30 0x7fba280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:06.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.390+0000 7fba30ff9700 1 -- 172.21.15.99:0/1635704523 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fba38084be0 con 0x7fba3c0cd6e0 2023-10-08T00:16:06.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.390+0000 7fba33fff700 1 --2- 172.21.15.99:0/1635704523 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fba28066f30 0x7fba280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:06.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.390+0000 7fba33fff700 1 --2- 172.21.15.99:0/1635704523 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fba28066f30 0x7fba280693e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fba2c0011e0 tx=0x7fba2c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:06.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.394+0000 7fba30ff9700 1 -- 172.21.15.99:0/1635704523 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fba38051e50 con 0x7fba3c0cd6e0 2023-10-08T00:16:06.621 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.620+0000 7fba41814700 1 -- 172.21.15.99:0/1635704523 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fba3c15e570 con 0x7fba3c0cd6e0 2023-10-08T00:16:06.621 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.621+0000 7fba30ff9700 1 -- 172.21.15.99:0/1635704523 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4599 (secure 0 0 0) 0x7fba38055470 con 0x7fba3c0cd6e0 2023-10-08T00:16:06.622 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:16:06.622 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":48,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5426,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/280880696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":280880696},{"type":"v1","addr":"172.21.15.191:6836","nonce":280880696}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":5902,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3148129562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":3148129562},{"type":"v1","addr":"172.21.15.99:6835","nonce":3148129562}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":5917,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/1825600944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":1825600944},{"type":"v1","addr":"172.21.15.191:6837","nonce":1825600944}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:15:40.595166+0000","modified":"2023-10-08T00:15:44.232916+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6360},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6360":{"gid":6360,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6834/146609901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":146609901},{"type":"v1","addr":"172.21.15.99:6834","nonce":146609901}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-08T00:16:06.624 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.623+0000 7fba41814700 1 -- 172.21.15.99:0/1635704523 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fba28066f30 msgr2=0x7fba280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:06.624 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.623+0000 7fba41814700 1 --2- 172.21.15.99:0/1635704523 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fba28066f30 0x7fba280693e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fba2c0011e0 tx=0x7fba2c009040 comp rx=0 tx=0).stop 2023-10-08T00:16:06.625 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.623+0000 7fba41814700 1 -- 172.21.15.99:0/1635704523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba3c0cd6e0 msgr2=0x7fba3c14c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:06.625 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.623+0000 7fba41814700 1 --2- 172.21.15.99:0/1635704523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba3c0cd6e0 0x7fba3c14c7e0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fba3800d890 tx=0x7fba38008040 comp rx=0 tx=0).stop 2023-10-08T00:16:06.625 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.623+0000 7fba41814700 1 -- 172.21.15.99:0/1635704523 shutdown_connections 2023-10-08T00:16:06.625 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.623+0000 7fba41814700 1 --2- 172.21.15.99:0/1635704523 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fba28066f30 0x7fba280693e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:06.625 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.623+0000 7fba41814700 1 --2- 172.21.15.99:0/1635704523 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba3c15cbd0 0x7fba3c155870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:06.625 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.623+0000 7fba41814700 1 --2- 172.21.15.99:0/1635704523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba3c0cd6e0 0x7fba3c14c7e0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:06.625 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.623+0000 7fba41814700 1 --2- 172.21.15.99:0/1635704523 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba3c0cdab0 0x7fba3c0b54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:06.625 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.623+0000 7fba41814700 1 -- 172.21.15.99:0/1635704523 >> 172.21.15.99:0/1635704523 conn(0x7fba3c067660 msgr2=0x7fba3c0c6420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:06.625 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.623+0000 7fba41814700 1 -- 172.21.15.99:0/1635704523 shutdown_connections 2023-10-08T00:16:06.625 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:06.623+0000 7fba41814700 1 -- 172.21.15.99:0/1635704523 wait complete. 2023-10-08T00:16:06.626 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 48 2023-10-08T00:16:06.636 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-10-08T00:16:06.768 INFO:teuthology.orchestra.run.smithi099.stdout:{"cluster_fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","whoami":0,"id":6360,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":25.535246553,"mdsmap_epoch":48,"osdmap_epoch":112,"osdmap_epoch_barrier":108,"uptime":31.511020371000001} 2023-10-08T00:16:06.769 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", "fs_name": "cephfs", "id": 6360, "mdsmap_epoch": 48, "osdmap_epoch": 112, "osdmap_epoch_barrier": 108, "rank_uptime": 25.535246553, "state": "up:active", "uptime": 31.511020371, "want_state": "up:active", "whoami": 0 } 2023-10-08T00:16:06.769 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-08T00:16:06.769 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:16:06.769 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2023-10-08T00:16:07.048 INFO:teuthology.orchestra.run.smithi099.stderr:50+0 records in 2023-10-08T00:16:07.049 INFO:teuthology.orchestra.run.smithi099.stderr:50+0 records out 2023-10-08T00:16:07.049 INFO:teuthology.orchestra.run.smithi099.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.263715 s, 199 MB/s 2023-10-08T00:16:07.049 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T00:16:07.049 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T00:16:07.085 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T00:16:07.128 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T00:16:07.128 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T00:16:07.128 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:16:07.128 DEBUG:teuthology.orchestra.run.smithi099:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-08T00:16:07.209 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2023-10-08T00:16:07.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:16:07.210 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T00:16:07.225 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-08T00:16:07.225 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:16:07.226 DEBUG:teuthology.orchestra.run.smithi191:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-08T00:16:07.261 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2023-10-08T00:16:07.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:16:07.261 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-08T00:16:07.316 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:16:07.430 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.429+0000 7fd3ba603700 1 Processor -- start 2023-10-08T00:16:07.430 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.429+0000 7fd3ba603700 1 -- start start 2023-10-08T00:16:07.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.430+0000 7fd3ba603700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd3b40cdab0 0x7fd3b415d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:07.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.430+0000 7fd3ba603700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3b40cb7c0 con 0x7fd3b415dda0 2023-10-08T00:16:07.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.430+0000 7fd3ba603700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3b415e290 con 0x7fd3b40cd6e0 2023-10-08T00:16:07.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.430+0000 7fd3ba603700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3b415e850 con 0x7fd3b40cdab0 2023-10-08T00:16:07.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.430+0000 7fd3b8e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd3b40cdab0 0x7fd3b415d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:07.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.430+0000 7fd3b8e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd3b40cdab0 0x7fd3b415d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46100/0 (socket says 172.21.15.99:46100) 2023-10-08T00:16:07.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.430+0000 7fd3b8e00700 1 -- 172.21.15.99:0/1583821025 learned_addr learned my addr 172.21.15.99:0/1583821025 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:07.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.431+0000 7fd3b8e00700 1 -- 172.21.15.99:0/1583821025 >> v1:172.21.15.191:6789/0 conn(0x7fd3b40cd6e0 legacy=0x7fd3b40cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:07.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.431+0000 7fd3b8e00700 1 -- 172.21.15.99:0/1583821025 >> v1:172.21.15.99:6789/0 conn(0x7fd3b415dda0 legacy=0x7fd3b415cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:07.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.431+0000 7fd3b8e00700 1 -- 172.21.15.99:0/1583821025 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b415e570 con 0x7fd3b40cdab0 2023-10-08T00:16:07.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.431+0000 7fd3b8e00700 1 --2- 172.21.15.99:0/1583821025 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3b40cdab0 0x7fd3b415d860 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fd3a80099b0 tx=0x7fd3a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbe4765bf89ebca3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:07.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.431+0000 7fd3b3fff700 1 -- 172.21.15.99:0/1583821025 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a800b860 con 0x7fd3b40cdab0 2023-10-08T00:16:07.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.431+0000 7fd3b3fff700 1 -- 172.21.15.99:0/1583821025 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3a800b9c0 con 0x7fd3b40cdab0 2023-10-08T00:16:07.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.431+0000 7fd3b3fff700 1 -- 172.21.15.99:0/1583821025 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a800bd60 con 0x7fd3b40cdab0 2023-10-08T00:16:07.434 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.432+0000 7fd3ba603700 1 -- 172.21.15.99:0/1583821025 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3b40cdab0 msgr2=0x7fd3b415d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:07.434 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.432+0000 7fd3ba603700 1 --2- 172.21.15.99:0/1583821025 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3b40cdab0 0x7fd3b415d860 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fd3a80099b0 tx=0x7fd3a8009250 comp rx=0 tx=0).stop 2023-10-08T00:16:07.434 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.432+0000 7fd3ba603700 1 -- 172.21.15.99:0/1583821025 shutdown_connections 2023-10-08T00:16:07.434 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.432+0000 7fd3ba603700 1 --2- 172.21.15.99:0/1583821025 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3b40cdab0 0x7fd3b415d860 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:07.434 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.432+0000 7fd3ba603700 1 -- 172.21.15.99:0/1583821025 >> 172.21.15.99:0/1583821025 conn(0x7fd3b4067660 msgr2=0x7fd3b4067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:07.434 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.432+0000 7fd3ba603700 1 -- 172.21.15.99:0/1583821025 shutdown_connections 2023-10-08T00:16:07.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.432+0000 7fd3ba603700 1 -- 172.21.15.99:0/1583821025 wait complete. 2023-10-08T00:16:07.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.433+0000 7fd3ba603700 1 Processor -- start 2023-10-08T00:16:07.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.433+0000 7fd3ba603700 1 -- start start 2023-10-08T00:16:07.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.434+0000 7fd3ba603700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3b40cd6e0 0x7fd3b414d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:07.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.434+0000 7fd3ba603700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3b40cdab0 0x7fd3b40b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:07.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.434+0000 7fd3ba603700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd3b415dda0 0x7fd3b40b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:07.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.434+0000 7fd3ba603700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b40cc360 con 0x7fd3b40cdab0 2023-10-08T00:16:07.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.434+0000 7fd3ba603700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b40c8ba0 con 0x7fd3b40cd6e0 2023-10-08T00:16:07.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.434+0000 7fd3ba603700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b415b290 con 0x7fd3b415dda0 2023-10-08T00:16:07.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.434+0000 7fd3b8e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3b40cdab0 0x7fd3b40b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:07.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.434+0000 7fd3b8e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3b40cdab0 0x7fd3b40b3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46108/0 (socket says 172.21.15.99:46108) 2023-10-08T00:16:07.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.434+0000 7fd3b8e00700 1 -- 172.21.15.99:0/298043789 learned_addr learned my addr 172.21.15.99:0/298043789 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:07.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.434+0000 7fd3b9601700 1 --2- 172.21.15.99:0/298043789 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3b40cd6e0 0x7fd3b414d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:07.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.434+0000 7fd3b9e02700 1 --2- 172.21.15.99:0/298043789 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd3b415dda0 0x7fd3b40b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:07.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.434+0000 7fd3b8e00700 1 -- 172.21.15.99:0/298043789 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd3b415dda0 msgr2=0x7fd3b40b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:07.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.434+0000 7fd3b8e00700 1 --2- 172.21.15.99:0/298043789 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd3b415dda0 0x7fd3b40b3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:07.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.434+0000 7fd3b8e00700 1 -- 172.21.15.99:0/298043789 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3b40cd6e0 msgr2=0x7fd3b414d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:07.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.434+0000 7fd3b8e00700 1 --2- 172.21.15.99:0/298043789 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3b40cd6e0 0x7fd3b414d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:07.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.434+0000 7fd3b8e00700 1 -- 172.21.15.99:0/298043789 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a80094f0 con 0x7fd3b40cdab0 2023-10-08T00:16:07.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.435+0000 7fd3b8e00700 1 --2- 172.21.15.99:0/298043789 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3b40cdab0 0x7fd3b40b3a10 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fd3a80094c0 tx=0x7fd3a8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:07.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.435+0000 7fd3b17fa700 1 -- 172.21.15.99:0/298043789 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a8024440 con 0x7fd3b40cdab0 2023-10-08T00:16:07.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.435+0000 7fd3b17fa700 1 -- 172.21.15.99:0/298043789 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3a80245a0 con 0x7fd3b40cdab0 2023-10-08T00:16:07.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.435+0000 7fd3ba603700 1 -- 172.21.15.99:0/298043789 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3b40c8270 con 0x7fd3b40cdab0 2023-10-08T00:16:07.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.435+0000 7fd3b17fa700 1 -- 172.21.15.99:0/298043789 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a8024960 con 0x7fd3b40cdab0 2023-10-08T00:16:07.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.435+0000 7fd3ba603700 1 -- 172.21.15.99:0/298043789 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3b40c8830 con 0x7fd3b40cdab0 2023-10-08T00:16:07.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.436+0000 7fd3ba603700 1 -- 172.21.15.99:0/298043789 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3b40c8d30 con 0x7fd3b40cdab0 2023-10-08T00:16:07.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.436+0000 7fd3b17fa700 1 -- 172.21.15.99:0/298043789 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd3a801a420 con 0x7fd3b40cdab0 2023-10-08T00:16:07.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.436+0000 7fd3b17fa700 1 --2- 172.21.15.99:0/298043789 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd3a0066f30 0x7fd3a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:07.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.436+0000 7fd3b17fa700 1 -- 172.21.15.99:0/298043789 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd3a8086460 con 0x7fd3b40cdab0 2023-10-08T00:16:07.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.437+0000 7fd3b9601700 1 --2- 172.21.15.99:0/298043789 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd3a0066f30 0x7fd3a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:07.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.437+0000 7fd3b9601700 1 --2- 172.21.15.99:0/298043789 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd3a0066f30 0x7fd3a00693e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4007fd0 tx=0x7fd3a40016f0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:07.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.440+0000 7fd3b17fa700 1 -- 172.21.15.99:0/298043789 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd3a8053630 con 0x7fd3b40cdab0 2023-10-08T00:16:07.667 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.666+0000 7fd3ba603700 1 -- 172.21.15.99:0/298043789 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd3b414dc70 con 0x7fd3b40cdab0 2023-10-08T00:16:07.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.667+0000 7fd3b17fa700 1 -- 172.21.15.99:0/298043789 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4599 (secure 0 0 0) 0x7fd3a8056c50 con 0x7fd3b40cdab0 2023-10-08T00:16:07.668 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:16:07.668 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":48,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5426,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/280880696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":280880696},{"type":"v1","addr":"172.21.15.191:6836","nonce":280880696}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":5902,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3148129562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":3148129562},{"type":"v1","addr":"172.21.15.99:6835","nonce":3148129562}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":5917,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/1825600944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":1825600944},{"type":"v1","addr":"172.21.15.191:6837","nonce":1825600944}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:15:40.595166+0000","modified":"2023-10-08T00:15:44.232916+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6360},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6360":{"gid":6360,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6834/146609901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":146609901},{"type":"v1","addr":"172.21.15.99:6834","nonce":146609901}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-08T00:16:07.670 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.669+0000 7fd3ba603700 1 -- 172.21.15.99:0/298043789 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd3a0066f30 msgr2=0x7fd3a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:07.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.669+0000 7fd3ba603700 1 --2- 172.21.15.99:0/298043789 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd3a0066f30 0x7fd3a00693e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4007fd0 tx=0x7fd3a40016f0 comp rx=0 tx=0).stop 2023-10-08T00:16:07.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.669+0000 7fd3ba603700 1 -- 172.21.15.99:0/298043789 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3b40cdab0 msgr2=0x7fd3b40b3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:07.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.669+0000 7fd3ba603700 1 --2- 172.21.15.99:0/298043789 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3b40cdab0 0x7fd3b40b3a10 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fd3a80094c0 tx=0x7fd3a8017000 comp rx=0 tx=0).stop 2023-10-08T00:16:07.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.669+0000 7fd3ba603700 1 -- 172.21.15.99:0/298043789 shutdown_connections 2023-10-08T00:16:07.673 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.669+0000 7fd3ba603700 1 --2- 172.21.15.99:0/298043789 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd3a0066f30 0x7fd3a00693e0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:07.673 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.669+0000 7fd3ba603700 1 --2- 172.21.15.99:0/298043789 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3b40cd6e0 0x7fd3b414d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:07.673 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.669+0000 7fd3ba603700 1 --2- 172.21.15.99:0/298043789 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd3b415dda0 0x7fd3b40b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:07.673 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.669+0000 7fd3ba603700 1 --2- 172.21.15.99:0/298043789 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3b40cdab0 0x7fd3b40b3a10 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:07.673 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.669+0000 7fd3ba603700 1 -- 172.21.15.99:0/298043789 >> 172.21.15.99:0/298043789 conn(0x7fd3b4067660 msgr2=0x7fd3b41573e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:07.673 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.669+0000 7fd3ba603700 1 -- 172.21.15.99:0/298043789 shutdown_connections 2023-10-08T00:16:07.673 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.669+0000 7fd3ba603700 1 -- 172.21.15.99:0/298043789 wait complete. 2023-10-08T00:16:07.673 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 48 2023-10-08T00:16:07.681 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:16:07.795 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.794+0000 7fba79802700 1 Processor -- start 2023-10-08T00:16:07.796 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.794+0000 7fba79802700 1 -- start start 2023-10-08T00:16:07.805 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.795+0000 7fba79802700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba741577d0 0x7fba74157ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:07.805 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.795+0000 7fba79802700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fba74069320 0x7fba7406ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:07.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.795+0000 7fba79802700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba740cac40 con 0x7fba7415b6b0 2023-10-08T00:16:07.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.795+0000 7fba79802700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba7415bc70 con 0x7fba74069320 2023-10-08T00:16:07.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.795+0000 7fba79802700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba7415bf50 con 0x7fba741577d0 2023-10-08T00:16:07.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.795+0000 7fba73fff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fba74069320 0x7fba7406ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:07.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.795+0000 7fba73fff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fba74069320 0x7fba7406ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46120/0 (socket says 172.21.15.99:46120) 2023-10-08T00:16:07.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.795+0000 7fba73fff700 1 -- 172.21.15.99:0/1698864131 learned_addr learned my addr 172.21.15.99:0/1698864131 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:07.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.796+0000 7fba78800700 1 --2- 172.21.15.99:0/1698864131 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba741577d0 0x7fba74157ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:07.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.796+0000 7fba73fff700 1 -- 172.21.15.99:0/1698864131 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba741577d0 msgr2=0x7fba74157ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:07.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.796+0000 7fba73fff700 1 --2- 172.21.15.99:0/1698864131 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba741577d0 0x7fba74157ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:07.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.796+0000 7fba73fff700 1 -- 172.21.15.99:0/1698864131 >> v1:172.21.15.191:6789/0 conn(0x7fba7415b6b0 legacy=0x7fba7406a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:07.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.796+0000 7fba73fff700 1 -- 172.21.15.99:0/1698864131 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba7415c230 con 0x7fba74069320 2023-10-08T00:16:07.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.796+0000 7fba73fff700 1 --2- 172.21.15.99:0/1698864131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba74069320 0x7fba7406ed00 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fba600099b0 tx=0x7fba60009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f987431ad87408e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:07.808 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.796+0000 7fba737fe700 1 -- 172.21.15.99:0/1698864131 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6000b820 con 0x7fba74069320 2023-10-08T00:16:07.808 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.796+0000 7fba737fe700 1 -- 172.21.15.99:0/1698864131 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba6000b980 con 0x7fba74069320 2023-10-08T00:16:07.808 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.796+0000 7fba737fe700 1 -- 172.21.15.99:0/1698864131 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6000bd20 con 0x7fba74069320 2023-10-08T00:16:07.808 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.797+0000 7fba79802700 1 -- 172.21.15.99:0/1698864131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba74069320 msgr2=0x7fba7406ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:07.808 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.797+0000 7fba79802700 1 --2- 172.21.15.99:0/1698864131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba74069320 0x7fba7406ed00 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fba600099b0 tx=0x7fba60009250 comp rx=0 tx=0).stop 2023-10-08T00:16:07.808 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.797+0000 7fba79802700 1 -- 172.21.15.99:0/1698864131 shutdown_connections 2023-10-08T00:16:07.809 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.797+0000 7fba79802700 1 --2- 172.21.15.99:0/1698864131 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba741577d0 0x7fba74157ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:07.809 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.797+0000 7fba79802700 1 --2- 172.21.15.99:0/1698864131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba74069320 0x7fba7406ed00 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:07.809 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.797+0000 7fba79802700 1 -- 172.21.15.99:0/1698864131 >> 172.21.15.99:0/1698864131 conn(0x7fba740cd070 msgr2=0x7fba740cd470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:07.809 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.797+0000 7fba79802700 1 -- 172.21.15.99:0/1698864131 shutdown_connections 2023-10-08T00:16:07.809 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.797+0000 7fba79802700 1 -- 172.21.15.99:0/1698864131 wait complete. 2023-10-08T00:16:07.809 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.798+0000 7fba79802700 1 Processor -- start 2023-10-08T00:16:07.810 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.798+0000 7fba79802700 1 -- start start 2023-10-08T00:16:07.810 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.799+0000 7fba79802700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba74069320 0x7fba740c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:07.810 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.799+0000 7fba79802700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba741577d0 0x7fba740c6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:07.810 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.799+0000 7fba79802700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba7415b6b0 0x7fba740c67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:07.811 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.799+0000 7fba79802700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba740992d0 con 0x7fba74069320 2023-10-08T00:16:07.811 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.799+0000 7fba79802700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7416a370 con 0x7fba741577d0 2023-10-08T00:16:07.811 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.799+0000 7fba79802700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba740c54d0 con 0x7fba7415b6b0 2023-10-08T00:16:07.811 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.799+0000 7fba78800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba74069320 0x7fba740c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:07.812 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.799+0000 7fba78800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba74069320 0x7fba740c8f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46134/0 (socket says 172.21.15.99:46134) 2023-10-08T00:16:07.812 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.799+0000 7fba78800700 1 -- 172.21.15.99:0/1055199655 learned_addr learned my addr 172.21.15.99:0/1055199655 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:07.812 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.799+0000 7fba78800700 1 -- 172.21.15.99:0/1055199655 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba7415b6b0 msgr2=0x7fba740c67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:07.812 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.799+0000 7fba79001700 1 --2- 172.21.15.99:0/1055199655 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba7415b6b0 0x7fba740c67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:07.812 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.799+0000 7fba73fff700 1 --2- 172.21.15.99:0/1055199655 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba741577d0 0x7fba740c6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:07.812 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.799+0000 7fba78800700 1 --2- 172.21.15.99:0/1055199655 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba7415b6b0 0x7fba740c67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:07.813 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.799+0000 7fba78800700 1 -- 172.21.15.99:0/1055199655 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba741577d0 msgr2=0x7fba740c6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:07.813 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.799+0000 7fba78800700 1 --2- 172.21.15.99:0/1055199655 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba741577d0 0x7fba740c6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:07.813 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.799+0000 7fba78800700 1 -- 172.21.15.99:0/1055199655 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba600094f0 con 0x7fba74069320 2023-10-08T00:16:07.813 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.799+0000 7fba79001700 1 --2- 172.21.15.99:0/1055199655 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba7415b6b0 0x7fba740c67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:16:07.813 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.799+0000 7fba73fff700 1 --2- 172.21.15.99:0/1055199655 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba741577d0 0x7fba740c6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:16:07.814 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.799+0000 7fba78800700 1 --2- 172.21.15.99:0/1055199655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba74069320 0x7fba740c8f50 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fba6800d890 tx=0x7fba68008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:07.814 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.799+0000 7fba70ff9700 1 -- 172.21.15.99:0/1055199655 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6800b600 con 0x7fba74069320 2023-10-08T00:16:07.814 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.799+0000 7fba70ff9700 1 -- 172.21.15.99:0/1055199655 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba6800b760 con 0x7fba74069320 2023-10-08T00:16:07.814 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.800+0000 7fba70ff9700 1 -- 172.21.15.99:0/1055199655 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6800bae0 con 0x7fba74069320 2023-10-08T00:16:07.814 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.800+0000 7fba79802700 1 -- 172.21.15.99:0/1055199655 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba740c57b0 con 0x7fba74069320 2023-10-08T00:16:07.814 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.800+0000 7fba79802700 1 -- 172.21.15.99:0/1055199655 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba740c5d70 con 0x7fba74069320 2023-10-08T00:16:07.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.800+0000 7fba79802700 1 -- 172.21.15.99:0/1055199655 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba7406c790 con 0x7fba74069320 2023-10-08T00:16:07.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.801+0000 7fba70ff9700 1 -- 172.21.15.99:0/1055199655 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fba6801e460 con 0x7fba74069320 2023-10-08T00:16:07.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.801+0000 7fba70ff9700 1 --2- 172.21.15.99:0/1055199655 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fba58066f30 0x7fba580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:07.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.801+0000 7fba70ff9700 1 -- 172.21.15.99:0/1055199655 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fba68084bf0 con 0x7fba74069320 2023-10-08T00:16:07.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.801+0000 7fba73fff700 1 --2- 172.21.15.99:0/1055199655 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fba58066f30 0x7fba580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:07.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.802+0000 7fba73fff700 1 --2- 172.21.15.99:0/1055199655 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fba58066f30 0x7fba580693e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fba600099b0 tx=0x7fba60016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:07.819 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.805+0000 7fba70ff9700 1 -- 172.21.15.99:0/1055199655 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fba68051e60 con 0x7fba74069320 2023-10-08T00:16:07.989 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.989+0000 7fba79802700 1 -- 172.21.15.99:0/1055199655 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fba7415c510 con 0x7fba74069320 2023-10-08T00:16:07.990 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.989+0000 7fba70ff9700 1 -- 172.21.15.99:0/1055199655 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+15365 (secure 0 0 0) 0x7fba68055480 con 0x7fba74069320 2023-10-08T00:16:07.990 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:16:07.990 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":112,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:16:05.407718+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:15:38.848324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-08T00:15:39.561379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:16:07.992 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.991+0000 7fba79802700 1 -- 172.21.15.99:0/1055199655 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fba58066f30 msgr2=0x7fba580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:07.992 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.991+0000 7fba79802700 1 --2- 172.21.15.99:0/1055199655 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fba58066f30 0x7fba580693e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fba600099b0 tx=0x7fba60016000 comp rx=0 tx=0).stop 2023-10-08T00:16:07.992 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.991+0000 7fba79802700 1 -- 172.21.15.99:0/1055199655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba74069320 msgr2=0x7fba740c8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:07.992 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.991+0000 7fba79802700 1 --2- 172.21.15.99:0/1055199655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba74069320 0x7fba740c8f50 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fba6800d890 tx=0x7fba68008040 comp rx=0 tx=0).stop 2023-10-08T00:16:07.993 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.991+0000 7fba79802700 1 -- 172.21.15.99:0/1055199655 shutdown_connections 2023-10-08T00:16:07.993 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.991+0000 7fba79802700 1 --2- 172.21.15.99:0/1055199655 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fba58066f30 0x7fba580693e0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:07.993 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.991+0000 7fba79802700 1 --2- 172.21.15.99:0/1055199655 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba7415b6b0 0x7fba740c67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:07.993 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.991+0000 7fba79802700 1 --2- 172.21.15.99:0/1055199655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba74069320 0x7fba740c8f50 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:07.993 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.991+0000 7fba79802700 1 --2- 172.21.15.99:0/1055199655 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba741577d0 0x7fba740c6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:07.993 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.991+0000 7fba79802700 1 -- 172.21.15.99:0/1055199655 >> 172.21.15.99:0/1055199655 conn(0x7fba740cd070 msgr2=0x7fba740c3060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:07.993 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.991+0000 7fba79802700 1 -- 172.21.15.99:0/1055199655 shutdown_connections 2023-10-08T00:16:07.993 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:07.991+0000 7fba79802700 1 -- 172.21.15.99:0/1055199655 wait complete. 2023-10-08T00:16:08.005 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:10:42.063344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T00:15:38.848324+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T00:15:39.561379+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T00:16:08.006 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T00:16:08.123 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.123+0000 7f49ba31d700 1 Processor -- start 2023-10-08T00:16:08.123 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.123+0000 7f49ba31d700 1 -- start start 2023-10-08T00:16:08.127 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.124+0000 7f49ba31d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49b40cce30 0x7f49b415aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:08.127 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.124+0000 7f49ba31d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f49b40b6580 0x7f49b406a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:08.127 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.124+0000 7f49ba31d700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f49b406a830 0x7f49b415d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:08.127 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.124+0000 7f49ba31d700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49b40b4d70 con 0x7f49b406a830 2023-10-08T00:16:08.128 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.124+0000 7f49ba31d700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49b406ac70 con 0x7f49b40b6580 2023-10-08T00:16:08.128 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.124+0000 7f49ba31d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49b40cd8d0 con 0x7f49b40cce30 2023-10-08T00:16:08.128 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.124+0000 7f49b9b1c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f49b406a830 0x7f49b415d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:08.128 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.124+0000 7f49b9b1c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f49b406a830 0x7f49b415d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46136/0 (socket says 172.21.15.99:46136) 2023-10-08T00:16:08.128 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.124+0000 7f49b9b1c700 1 -- 172.21.15.99:0/616338890 learned_addr learned my addr 172.21.15.99:0/616338890 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:08.129 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.124+0000 7f49b931b700 1 --2- 172.21.15.99:0/616338890 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49b40cce30 0x7f49b415aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:08.129 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.124+0000 7f49b8b1a700 1 --2- 172.21.15.99:0/616338890 >> v2:172.21.15.191:3300/0 conn(0x7f49b40b6580 0x7f49b406a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:08.129 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.124+0000 7f49b9b1c700 1 -- 172.21.15.99:0/616338890 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49b40cce30 msgr2=0x7f49b415aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:08.129 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.124+0000 7f49b9b1c700 1 --2- 172.21.15.99:0/616338890 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49b40cce30 0x7f49b415aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:08.129 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.124+0000 7f49b9b1c700 1 -- 172.21.15.99:0/616338890 >> v2:172.21.15.191:3300/0 conn(0x7f49b40b6580 msgr2=0x7f49b406a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:08.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.124+0000 7f49b9b1c700 1 --2- 172.21.15.99:0/616338890 >> v2:172.21.15.191:3300/0 conn(0x7f49b40b6580 0x7f49b406a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:08.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.124+0000 7f49b9b1c700 1 -- 172.21.15.99:0/616338890 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b40b4440 con 0x7f49b406a830 2023-10-08T00:16:08.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.124+0000 7f49b9b1c700 1 --2- 172.21.15.99:0/616338890 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49b406a830 0x7f49b415d880 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f49b40b36d0 tx=0x7f49ac009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22833f5523597445 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:08.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.124+0000 7f49abfff700 1 -- 172.21.15.99:0/616338890 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ac00e070 con 0x7f49b406a830 2023-10-08T00:16:08.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.124+0000 7f49abfff700 1 -- 172.21.15.99:0/616338890 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49ac00b970 con 0x7f49b406a830 2023-10-08T00:16:08.131 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.124+0000 7f49abfff700 1 -- 172.21.15.99:0/616338890 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ac0092e0 con 0x7f49b406a830 2023-10-08T00:16:08.131 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.125+0000 7f49ba31d700 1 -- 172.21.15.99:0/616338890 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49b406a830 msgr2=0x7f49b415d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:08.131 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.125+0000 7f49ba31d700 1 --2- 172.21.15.99:0/616338890 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49b406a830 0x7f49b415d880 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f49b40b36d0 tx=0x7f49ac009840 comp rx=0 tx=0).stop 2023-10-08T00:16:08.131 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.125+0000 7f49ba31d700 1 -- 172.21.15.99:0/616338890 shutdown_connections 2023-10-08T00:16:08.131 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.125+0000 7f49ba31d700 1 --2- 172.21.15.99:0/616338890 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49b406a830 0x7f49b415d880 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:08.131 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.125+0000 7f49ba31d700 1 --2- 172.21.15.99:0/616338890 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49b40cce30 0x7f49b415aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:08.132 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.125+0000 7f49ba31d700 1 --2- 172.21.15.99:0/616338890 >> v2:172.21.15.191:3300/0 conn(0x7f49b40b6580 0x7f49b406a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:08.132 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.125+0000 7f49ba31d700 1 -- 172.21.15.99:0/616338890 >> 172.21.15.99:0/616338890 conn(0x7f49b4068cb0 msgr2=0x7f49b40690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:08.132 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.125+0000 7f49ba31d700 1 -- 172.21.15.99:0/616338890 shutdown_connections 2023-10-08T00:16:08.132 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.125+0000 7f49ba31d700 1 -- 172.21.15.99:0/616338890 wait complete. 2023-10-08T00:16:08.132 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.126+0000 7f49ba31d700 1 Processor -- start 2023-10-08T00:16:08.133 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.126+0000 7f49ba31d700 1 -- start start 2023-10-08T00:16:08.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.127+0000 7f49ba31d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49b406a830 0x7f49b40b2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:08.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.127+0000 7f49ba31d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49b40b6580 0x7f49b4152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:08.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.127+0000 7f49ba31d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49b40cce30 0x7f49b414e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:08.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.127+0000 7f49ba31d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b40b2a50 con 0x7f49b40b6580 2023-10-08T00:16:08.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.127+0000 7f49ba31d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b40b2d30 con 0x7f49b40cce30 2023-10-08T00:16:08.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.127+0000 7f49ba31d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49b40b3010 con 0x7f49b406a830 2023-10-08T00:16:08.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.127+0000 7f49b8b1a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49b40b6580 0x7f49b4152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:08.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.127+0000 7f49b8b1a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49b40b6580 0x7f49b4152610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46138/0 (socket says 172.21.15.99:46138) 2023-10-08T00:16:08.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.127+0000 7f49b8b1a700 1 -- 172.21.15.99:0/2155485008 learned_addr learned my addr 172.21.15.99:0/2155485008 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:08.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.127+0000 7f49b931b700 1 --2- 172.21.15.99:0/2155485008 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49b406a830 0x7f49b40b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:08.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.127+0000 7f49b9b1c700 1 --2- 172.21.15.99:0/2155485008 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49b40cce30 0x7f49b414e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:08.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.127+0000 7f49b8b1a700 1 -- 172.21.15.99:0/2155485008 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49b406a830 msgr2=0x7f49b40b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:08.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.127+0000 7f49b8b1a700 1 --2- 172.21.15.99:0/2155485008 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49b406a830 0x7f49b40b2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:08.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.127+0000 7f49b8b1a700 1 -- 172.21.15.99:0/2155485008 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49b40cce30 msgr2=0x7f49b414e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:08.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.127+0000 7f49b8b1a700 1 --2- 172.21.15.99:0/2155485008 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49b40cce30 0x7f49b414e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:08.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.127+0000 7f49b8b1a700 1 -- 172.21.15.99:0/2155485008 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49ac009ae0 con 0x7f49b40b6580 2023-10-08T00:16:08.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.127+0000 7f49b8b1a700 1 --2- 172.21.15.99:0/2155485008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49b40b6580 0x7f49b4152610 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f49b000ec50 tx=0x7f49b00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:08.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.128+0000 7f49a97fa700 1 -- 172.21.15.99:0/2155485008 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b000c850 con 0x7f49b40b6580 2023-10-08T00:16:08.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.128+0000 7f49a97fa700 1 -- 172.21.15.99:0/2155485008 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b000c9b0 con 0x7f49b40b6580 2023-10-08T00:16:08.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.128+0000 7f49ba31d700 1 -- 172.21.15.99:0/2155485008 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49b40b32f0 con 0x7f49b40b6580 2023-10-08T00:16:08.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.128+0000 7f49a97fa700 1 -- 172.21.15.99:0/2155485008 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b000cd70 con 0x7f49b40b6580 2023-10-08T00:16:08.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.128+0000 7f49ba31d700 1 -- 172.21.15.99:0/2155485008 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49b4152b50 con 0x7f49b40b6580 2023-10-08T00:16:08.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.129+0000 7f49ba31d700 1 -- 172.21.15.99:0/2155485008 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49b416a030 con 0x7f49b40b6580 2023-10-08T00:16:08.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.129+0000 7f49a97fa700 1 -- 172.21.15.99:0/2155485008 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f49b0013680 con 0x7f49b40b6580 2023-10-08T00:16:08.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.129+0000 7f49a97fa700 1 --2- 172.21.15.99:0/2155485008 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f49a0066f30 0x7f49a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:08.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.129+0000 7f49a97fa700 1 -- 172.21.15.99:0/2155485008 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f49b0084720 con 0x7f49b40b6580 2023-10-08T00:16:08.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.129+0000 7f49b931b700 1 --2- 172.21.15.99:0/2155485008 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f49a0066f30 0x7f49a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:08.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.130+0000 7f49b931b700 1 --2- 172.21.15.99:0/2155485008 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f49a0066f30 0x7f49a00693e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f49a4009d10 tx=0x7f49a40091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:08.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.134+0000 7f49a97fa700 1 -- 172.21.15.99:0/2155485008 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f49b416a030 con 0x7f49b40b6580 2023-10-08T00:16:08.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.314+0000 7f49ba31d700 1 -- 172.21.15.99:0/2155485008 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f49b416a030 con 0x7f49b40b6580 2023-10-08T00:16:08.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.315+0000 7f49a97fa700 1 -- 172.21.15.99:0/2155485008 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) v1 ==== 93+0+10 (secure 0 0 0) 0x7f49b416a030 con 0x7f49b40b6580 2023-10-08T00:16:08.315 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2023-10-08T00:16:08.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.316+0000 7f49ba31d700 1 -- 172.21.15.99:0/2155485008 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f49a0066f30 msgr2=0x7f49a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:08.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.316+0000 7f49ba31d700 1 --2- 172.21.15.99:0/2155485008 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f49a0066f30 0x7f49a00693e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f49a4009d10 tx=0x7f49a40091a0 comp rx=0 tx=0).stop 2023-10-08T00:16:08.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.316+0000 7f49ba31d700 1 -- 172.21.15.99:0/2155485008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49b40b6580 msgr2=0x7f49b4152610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:08.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.316+0000 7f49ba31d700 1 --2- 172.21.15.99:0/2155485008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49b40b6580 0x7f49b4152610 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f49b000ec50 tx=0x7f49b00091a0 comp rx=0 tx=0).stop 2023-10-08T00:16:08.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.317+0000 7f49ba31d700 1 -- 172.21.15.99:0/2155485008 shutdown_connections 2023-10-08T00:16:08.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.317+0000 7f49ba31d700 1 --2- 172.21.15.99:0/2155485008 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f49a0066f30 0x7f49a00693e0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:08.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.317+0000 7f49ba31d700 1 --2- 172.21.15.99:0/2155485008 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49b40cce30 0x7f49b414e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:08.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.317+0000 7f49ba31d700 1 --2- 172.21.15.99:0/2155485008 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49b406a830 0x7f49b40b2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:08.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.317+0000 7f49ba31d700 1 --2- 172.21.15.99:0/2155485008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49b40b6580 0x7f49b4152610 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:08.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.317+0000 7f49ba31d700 1 -- 172.21.15.99:0/2155485008 >> 172.21.15.99:0/2155485008 conn(0x7f49b4068cb0 msgr2=0x7f49b40cfe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:08.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.317+0000 7f49ba31d700 1 -- 172.21.15.99:0/2155485008 shutdown_connections 2023-10-08T00:16:08.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.317+0000 7f49ba31d700 1 -- 172.21.15.99:0/2155485008 wait complete. 2023-10-08T00:16:08.329 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T00:16:08.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.445+0000 7f061e526700 1 Processor -- start 2023-10-08T00:16:08.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.445+0000 7f061e526700 1 -- start start 2023-10-08T00:16:08.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.446+0000 7f061e526700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0618159ec0 0x7f06180af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:08.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.446+0000 7f061e526700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f061815a290 0x7f06180afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:08.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.446+0000 7f061e526700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06180b0790 con 0x7f061806a5a0 2023-10-08T00:16:08.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.446+0000 7f061e526700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06180ce1f0 con 0x7f0618159ec0 2023-10-08T00:16:08.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.446+0000 7f061e526700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06180b0b20 con 0x7f061815a290 2023-10-08T00:16:08.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.446+0000 7f0617fff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0618159ec0 0x7f06180af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:08.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.446+0000 7f0617fff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0618159ec0 0x7f06180af940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46146/0 (socket says 172.21.15.99:46146) 2023-10-08T00:16:08.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.446+0000 7f0617fff700 1 -- 172.21.15.99:0/2899063859 learned_addr learned my addr 172.21.15.99:0/2899063859 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:08.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.446+0000 7f061cac3700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f061806a5a0 0x7f061815db90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:43702/0 (socket says 172.21.15.99:43702) 2023-10-08T00:16:08.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.446+0000 7f06177fe700 1 --2- 172.21.15.99:0/2899063859 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f061815a290 0x7f06180afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:08.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.447+0000 7f0617fff700 1 -- 172.21.15.99:0/2899063859 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f061815a290 msgr2=0x7f06180afe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:08.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.447+0000 7f0617fff700 1 --2- 172.21.15.99:0/2899063859 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f061815a290 0x7f06180afe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:08.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.447+0000 7f0617fff700 1 -- 172.21.15.99:0/2899063859 >> v1:172.21.15.99:6789/0 conn(0x7f061806a5a0 legacy=0x7f061815db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:08.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.447+0000 7f0617fff700 1 -- 172.21.15.99:0/2899063859 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06180b04b0 con 0x7f0618159ec0 2023-10-08T00:16:08.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.447+0000 7f0616ffd700 1 -- 172.21.15.99:0/2899063859 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4251453975 0 0) 0x7f06180b0790 con 0x7f061806a5a0 2023-10-08T00:16:08.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.447+0000 7f0617fff700 1 --2- 172.21.15.99:0/2899063859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0618159ec0 0x7f06180af940 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f06080099b0 tx=0x7f0608009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=141ae74d14239988 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:08.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.447+0000 7f0616ffd700 1 -- 172.21.15.99:0/2899063859 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0608009ea0 con 0x7f0618159ec0 2023-10-08T00:16:08.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.447+0000 7f0616ffd700 1 -- 172.21.15.99:0/2899063859 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f060800b860 con 0x7f0618159ec0 2023-10-08T00:16:08.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.447+0000 7f0616ffd700 1 -- 172.21.15.99:0/2899063859 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060800bc00 con 0x7f0618159ec0 2023-10-08T00:16:08.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.447+0000 7f061e526700 1 -- 172.21.15.99:0/2899063859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0618159ec0 msgr2=0x7f06180af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:08.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.447+0000 7f061e526700 1 --2- 172.21.15.99:0/2899063859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0618159ec0 0x7f06180af940 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f06080099b0 tx=0x7f0608009250 comp rx=0 tx=0).stop 2023-10-08T00:16:08.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.447+0000 7f061e526700 1 -- 172.21.15.99:0/2899063859 shutdown_connections 2023-10-08T00:16:08.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.447+0000 7f061e526700 1 --2- 172.21.15.99:0/2899063859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0618159ec0 0x7f06180af940 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:08.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.447+0000 7f061e526700 1 --2- 172.21.15.99:0/2899063859 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f061815a290 0x7f06180afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:08.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.447+0000 7f061e526700 1 -- 172.21.15.99:0/2899063859 >> 172.21.15.99:0/2899063859 conn(0x7f06180d1880 msgr2=0x7f06180cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:08.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.447+0000 7f061e526700 1 -- 172.21.15.99:0/2899063859 shutdown_connections 2023-10-08T00:16:08.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.447+0000 7f061e526700 1 -- 172.21.15.99:0/2899063859 wait complete. 2023-10-08T00:16:08.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.448+0000 7f061e526700 1 Processor -- start 2023-10-08T00:16:08.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.449+0000 7f061e526700 1 -- start start 2023-10-08T00:16:08.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.449+0000 7f061e526700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f061806a5a0 0x7f06181605d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:08.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.449+0000 7f061e526700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0618159ec0 0x7f0618155a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:08.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.449+0000 7f061e526700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f061815a290 0x7f061815e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:08.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.449+0000 7f061e526700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0618155f80 con 0x7f061815a290 2023-10-08T00:16:08.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.449+0000 7f061e526700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061815b720 con 0x7f061806a5a0 2023-10-08T00:16:08.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.449+0000 7f061e526700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f061815c2a0 con 0x7f0618159ec0 2023-10-08T00:16:08.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.449+0000 7f061cac3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f061815a290 0x7f061815e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:08.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.449+0000 7f061cac3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f061815a290 0x7f061815e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46150/0 (socket says 172.21.15.99:46150) 2023-10-08T00:16:08.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.450+0000 7f061cac3700 1 -- 172.21.15.99:0/4245264685 learned_addr learned my addr 172.21.15.99:0/4245264685 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:08.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.450+0000 7f06177fe700 1 --2- 172.21.15.99:0/4245264685 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0618159ec0 0x7f0618155a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:08.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.450+0000 7f0617fff700 1 --2- 172.21.15.99:0/4245264685 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f061806a5a0 0x7f06181605d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:08.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.450+0000 7f061cac3700 1 -- 172.21.15.99:0/4245264685 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0618159ec0 msgr2=0x7f0618155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:08.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.450+0000 7f061cac3700 1 --2- 172.21.15.99:0/4245264685 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0618159ec0 0x7f0618155a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:08.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.450+0000 7f061cac3700 1 -- 172.21.15.99:0/4245264685 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f061806a5a0 msgr2=0x7f06181605d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:08.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.450+0000 7f061cac3700 1 --2- 172.21.15.99:0/4245264685 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f061806a5a0 0x7f06181605d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:08.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.450+0000 7f061cac3700 1 -- 172.21.15.99:0/4245264685 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06080094f0 con 0x7f061815a290 2023-10-08T00:16:08.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.450+0000 7f06177fe700 1 --2- 172.21.15.99:0/4245264685 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0618159ec0 0x7f0618155a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:08.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.450+0000 7f0617fff700 1 --2- 172.21.15.99:0/4245264685 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f061806a5a0 0x7f06181605d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:08.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.450+0000 7f061cac3700 1 --2- 172.21.15.99:0/4245264685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f061815a290 0x7f061815e930 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f061000ccf0 tx=0x7f0610001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:08.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.450+0000 7f05fffff700 1 -- 172.21.15.99:0/4245264685 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061000ce30 con 0x7f061815a290 2023-10-08T00:16:08.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.450+0000 7f05fffff700 1 -- 172.21.15.99:0/4245264685 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0610005df0 con 0x7f061815a290 2023-10-08T00:16:08.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.450+0000 7f061e526700 1 -- 172.21.15.99:0/4245264685 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f061815b970 con 0x7f061815a290 2023-10-08T00:16:08.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.450+0000 7f05fffff700 1 -- 172.21.15.99:0/4245264685 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061000ab70 con 0x7f061815a290 2023-10-08T00:16:08.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.450+0000 7f061e526700 1 -- 172.21.15.99:0/4245264685 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f061815bf30 con 0x7f061815a290 2023-10-08T00:16:08.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.451+0000 7f061e526700 1 -- 172.21.15.99:0/4245264685 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f061804c3c0 con 0x7f061815a290 2023-10-08T00:16:08.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.451+0000 7f05fffff700 1 -- 172.21.15.99:0/4245264685 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f06100070f0 con 0x7f061815a290 2023-10-08T00:16:08.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.452+0000 7f05fffff700 1 --2- 172.21.15.99:0/4245264685 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0604066f30 0x7f06040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:08.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.452+0000 7f05fffff700 1 -- 172.21.15.99:0/4245264685 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f06100845b0 con 0x7f061815a290 2023-10-08T00:16:08.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.452+0000 7f0617fff700 1 --2- 172.21.15.99:0/4245264685 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0604066f30 0x7f06040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:08.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.453+0000 7f0617fff700 1 --2- 172.21.15.99:0/4245264685 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0604066f30 0x7f06040693e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f06080099b0 tx=0x7f0608017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:08.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.456+0000 7f05fffff700 1 -- 172.21.15.99:0/4245264685 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f06100518a0 con 0x7f061815a290 2023-10-08T00:16:08.641 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.640+0000 7f061e526700 1 -- 172.21.15.99:0/4245264685 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f06180ceab0 con 0x7f061815a290 2023-10-08T00:16:08.641 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.641+0000 7f05fffff700 1 -- 172.21.15.99:0/4245264685 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v112) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0610054ec0 con 0x7f061815a290 2023-10-08T00:16:08.642 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2023-10-08T00:16:08.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.643+0000 7f061e526700 1 -- 172.21.15.99:0/4245264685 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0604066f30 msgr2=0x7f06040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:08.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.643+0000 7f061e526700 1 --2- 172.21.15.99:0/4245264685 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0604066f30 0x7f06040693e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f06080099b0 tx=0x7f0608017000 comp rx=0 tx=0).stop 2023-10-08T00:16:08.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.643+0000 7f061e526700 1 -- 172.21.15.99:0/4245264685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f061815a290 msgr2=0x7f061815e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:08.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.643+0000 7f061e526700 1 --2- 172.21.15.99:0/4245264685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f061815a290 0x7f061815e930 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f061000ccf0 tx=0x7f0610001a70 comp rx=0 tx=0).stop 2023-10-08T00:16:08.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.643+0000 7f061e526700 1 -- 172.21.15.99:0/4245264685 shutdown_connections 2023-10-08T00:16:08.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.643+0000 7f061e526700 1 --2- 172.21.15.99:0/4245264685 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0604066f30 0x7f06040693e0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:08.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.643+0000 7f061e526700 1 --2- 172.21.15.99:0/4245264685 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f061806a5a0 0x7f06181605d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:08.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.643+0000 7f061e526700 1 --2- 172.21.15.99:0/4245264685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f061815a290 0x7f061815e930 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:08.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.643+0000 7f061e526700 1 --2- 172.21.15.99:0/4245264685 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0618159ec0 0x7f0618155a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:08.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.643+0000 7f061e526700 1 -- 172.21.15.99:0/4245264685 >> 172.21.15.99:0/4245264685 conn(0x7f06180d1880 msgr2=0x7f06180b5b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:08.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.643+0000 7f061e526700 1 -- 172.21.15.99:0/4245264685 shutdown_connections 2023-10-08T00:16:08.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.643+0000 7f061e526700 1 -- 172.21.15.99:0/4245264685 wait complete. 2023-10-08T00:16:08.655 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T00:16:08.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.771+0000 7f1b2d8a2700 1 Processor -- start 2023-10-08T00:16:08.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.771+0000 7f1b2d8a2700 1 -- start start 2023-10-08T00:16:08.774 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.772+0000 7f1b2d8a2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1b28159ec0 0x7f1b280af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:08.774 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.772+0000 7f1b2d8a2700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1b2806acb0 0x7f1b280afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:08.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.772+0000 7f1b2d8a2700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b280b0790 con 0x7f1b2815a290 2023-10-08T00:16:08.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.772+0000 7f1b2d8a2700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b280ce1f0 con 0x7f1b2806acb0 2023-10-08T00:16:08.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.772+0000 7f1b2d8a2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b280b0b20 con 0x7f1b28159ec0 2023-10-08T00:16:08.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.772+0000 7f1b1e7fc700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f1b2815a290 0x7f1b2806a5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:43714/0 (socket says 172.21.15.99:43714) 2023-10-08T00:16:08.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.772+0000 7f1b1e7fc700 1 -- 172.21.15.99:0/3092584860 learned_addr learned my addr 172.21.15.99:0/3092584860 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:08.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.772+0000 7f1b277fe700 1 --2- 172.21.15.99:0/3092584860 >> v2:172.21.15.191:3300/0 conn(0x7f1b2806acb0 0x7f1b280afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:08.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.772+0000 7f1b26ffd700 1 --2- 172.21.15.99:0/3092584860 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1b28159ec0 0x7f1b280af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:08.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.772+0000 7f1b267fc700 1 -- 172.21.15.99:0/3092584860 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3598771281 0 0) 0x7f1b280b0790 con 0x7f1b2815a290 2023-10-08T00:16:08.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.772+0000 7f1b267fc700 1 -- 172.21.15.99:0/3092584860 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b14003570 con 0x7f1b2815a290 2023-10-08T00:16:08.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.772+0000 7f1b267fc700 1 -- 172.21.15.99:0/3092584860 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 672061622 0 0) 0x7f1b14003570 con 0x7f1b2815a290 2023-10-08T00:16:08.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.772+0000 7f1b26ffd700 1 -- 172.21.15.99:0/3092584860 >> v2:172.21.15.191:3300/0 conn(0x7f1b2806acb0 msgr2=0x7f1b280afe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:08.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.772+0000 7f1b26ffd700 1 --2- 172.21.15.99:0/3092584860 >> v2:172.21.15.191:3300/0 conn(0x7f1b2806acb0 0x7f1b280afe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:08.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.772+0000 7f1b26ffd700 1 -- 172.21.15.99:0/3092584860 >> v1:172.21.15.99:6789/0 conn(0x7f1b2815a290 legacy=0x7f1b2806a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:08.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.772+0000 7f1b26ffd700 1 -- 172.21.15.99:0/3092584860 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b280b04b0 con 0x7f1b28159ec0 2023-10-08T00:16:08.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.772+0000 7f1b277fe700 1 --2- 172.21.15.99:0/3092584860 >> v2:172.21.15.191:3300/0 conn(0x7f1b2806acb0 0x7f1b280afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:16:08.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.773+0000 7f1b26ffd700 1 --2- 172.21.15.99:0/3092584860 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1b28159ec0 0x7f1b280af940 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1b180098e0 tx=0x7f1b180091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6e6e1bde443f1aa server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:08.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.773+0000 7f1b267fc700 1 -- 172.21.15.99:0/3092584860 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b10001470 con 0x7f1b28159ec0 2023-10-08T00:16:08.779 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.773+0000 7f1b267fc700 1 -- 172.21.15.99:0/3092584860 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b18011420 con 0x7f1b28159ec0 2023-10-08T00:16:08.779 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.773+0000 7f1b267fc700 1 -- 172.21.15.99:0/3092584860 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b180117c0 con 0x7f1b28159ec0 2023-10-08T00:16:08.779 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.773+0000 7f1b2d8a2700 1 -- 172.21.15.99:0/3092584860 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1b28159ec0 msgr2=0x7f1b280af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:08.779 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.773+0000 7f1b2d8a2700 1 --2- 172.21.15.99:0/3092584860 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1b28159ec0 0x7f1b280af940 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1b180098e0 tx=0x7f1b180091a0 comp rx=0 tx=0).stop 2023-10-08T00:16:08.779 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.773+0000 7f1b2d8a2700 1 -- 172.21.15.99:0/3092584860 shutdown_connections 2023-10-08T00:16:08.780 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.773+0000 7f1b2d8a2700 1 --2- 172.21.15.99:0/3092584860 >> v2:172.21.15.191:3300/0 conn(0x7f1b2806acb0 0x7f1b280afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:08.780 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.773+0000 7f1b2d8a2700 1 --2- 172.21.15.99:0/3092584860 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1b28159ec0 0x7f1b280af940 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:08.780 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.773+0000 7f1b2d8a2700 1 -- 172.21.15.99:0/3092584860 >> 172.21.15.99:0/3092584860 conn(0x7f1b280d1880 msgr2=0x7f1b280cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:08.780 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.773+0000 7f1b2d8a2700 1 -- 172.21.15.99:0/3092584860 shutdown_connections 2023-10-08T00:16:08.780 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.773+0000 7f1b2d8a2700 1 -- 172.21.15.99:0/3092584860 wait complete. 2023-10-08T00:16:08.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.774+0000 7f1b2d8a2700 1 Processor -- start 2023-10-08T00:16:08.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.775+0000 7f1b2d8a2700 1 -- start start 2023-10-08T00:16:08.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.775+0000 7f1b2d8a2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1b2806acb0 0x7f1b28159120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:08.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.775+0000 7f1b2d8a2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b28159ec0 0x7f1b281558a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:08.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.775+0000 7f1b2d8a2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1b2815a290 0x7f1b280b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:08.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.775+0000 7f1b2d8a2700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b28159660 con 0x7f1b28159ec0 2023-10-08T00:16:08.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.775+0000 7f1b2d8a2700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b28158c90 con 0x7f1b2815a290 2023-10-08T00:16:08.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.775+0000 7f1b2d8a2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b28158080 con 0x7f1b2806acb0 2023-10-08T00:16:08.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.775+0000 7f1b1e7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b28159ec0 0x7f1b281558a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:08.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.775+0000 7f1b1e7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b28159ec0 0x7f1b281558a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46156/0 (socket says 172.21.15.99:46156) 2023-10-08T00:16:08.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.775+0000 7f1b1e7fc700 1 -- 172.21.15.99:0/2860318369 learned_addr learned my addr 172.21.15.99:0/2860318369 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:08.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.775+0000 7f1b26ffd700 1 --2- 172.21.15.99:0/2860318369 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1b2806acb0 0x7f1b28159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:08.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.775+0000 7f1b277fe700 1 --2- 172.21.15.99:0/2860318369 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1b2815a290 0x7f1b280b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:08.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.776+0000 7f1b1e7fc700 1 -- 172.21.15.99:0/2860318369 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1b2806acb0 msgr2=0x7f1b28159120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:08.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.776+0000 7f1b1e7fc700 1 --2- 172.21.15.99:0/2860318369 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1b2806acb0 0x7f1b28159120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:08.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.776+0000 7f1b1e7fc700 1 -- 172.21.15.99:0/2860318369 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1b2815a290 msgr2=0x7f1b280b4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:08.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.776+0000 7f1b1e7fc700 1 --2- 172.21.15.99:0/2860318369 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1b2815a290 0x7f1b280b4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:08.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.776+0000 7f1b1e7fc700 1 -- 172.21.15.99:0/2860318369 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b20009a20 con 0x7f1b28159ec0 2023-10-08T00:16:08.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.776+0000 7f1b1e7fc700 1 --2- 172.21.15.99:0/2860318369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b28159ec0 0x7f1b281558a0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f1b1000cb90 tx=0x7f1b10001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:08.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.776+0000 7f1b1f7fe700 1 -- 172.21.15.99:0/2860318369 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b10007150 con 0x7f1b28159ec0 2023-10-08T00:16:08.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.776+0000 7f1b1f7fe700 1 -- 172.21.15.99:0/2860318369 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b100072d0 con 0x7f1b28159ec0 2023-10-08T00:16:08.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.776+0000 7f1b2d8a2700 1 -- 172.21.15.99:0/2860318369 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b18009420 con 0x7f1b28159ec0 2023-10-08T00:16:08.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.776+0000 7f1b1f7fe700 1 -- 172.21.15.99:0/2860318369 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b10007690 con 0x7f1b28159ec0 2023-10-08T00:16:08.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.776+0000 7f1b2d8a2700 1 -- 172.21.15.99:0/2860318369 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b28158640 con 0x7f1b28159ec0 2023-10-08T00:16:08.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.777+0000 7f1b2d8a2700 1 -- 172.21.15.99:0/2860318369 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b2804c3c0 con 0x7f1b28159ec0 2023-10-08T00:16:08.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.777+0000 7f1b1f7fe700 1 -- 172.21.15.99:0/2860318369 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1b10004a40 con 0x7f1b28159ec0 2023-10-08T00:16:08.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.778+0000 7f1b1f7fe700 1 --2- 172.21.15.99:0/2860318369 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1b08066f30 0x7f1b080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:08.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.778+0000 7f1b1f7fe700 1 -- 172.21.15.99:0/2860318369 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1b10083c40 con 0x7f1b28159ec0 2023-10-08T00:16:08.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.778+0000 7f1b26ffd700 1 --2- 172.21.15.99:0/2860318369 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1b08066f30 0x7f1b080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:08.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.778+0000 7f1b26ffd700 1 --2- 172.21.15.99:0/2860318369 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1b08066f30 0x7f1b080693e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1b180030c0 tx=0x7f1b18016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:08.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.782+0000 7f1b1f7fe700 1 -- 172.21.15.99:0/2860318369 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1b10050f30 con 0x7f1b28159ec0 2023-10-08T00:16:08.964 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.963+0000 7f1b2d8a2700 1 -- 172.21.15.99:0/2860318369 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1b280ceab0 con 0x7f1b28159ec0 2023-10-08T00:16:08.964 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.963+0000 7f1b1f7fe700 1 -- 172.21.15.99:0/2860318369 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v112) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1b10018020 con 0x7f1b28159ec0 2023-10-08T00:16:08.964 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2023-10-08T00:16:08.966 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.965+0000 7f1b2d8a2700 1 -- 172.21.15.99:0/2860318369 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1b08066f30 msgr2=0x7f1b080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:08.966 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.965+0000 7f1b2d8a2700 1 --2- 172.21.15.99:0/2860318369 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1b08066f30 0x7f1b080693e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1b180030c0 tx=0x7f1b18016000 comp rx=0 tx=0).stop 2023-10-08T00:16:08.966 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.965+0000 7f1b2d8a2700 1 -- 172.21.15.99:0/2860318369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b28159ec0 msgr2=0x7f1b281558a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:08.967 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.965+0000 7f1b2d8a2700 1 --2- 172.21.15.99:0/2860318369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b28159ec0 0x7f1b281558a0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f1b1000cb90 tx=0x7f1b10001ab0 comp rx=0 tx=0).stop 2023-10-08T00:16:08.967 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.965+0000 7f1b2d8a2700 1 -- 172.21.15.99:0/2860318369 shutdown_connections 2023-10-08T00:16:08.967 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.965+0000 7f1b2d8a2700 1 --2- 172.21.15.99:0/2860318369 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1b08066f30 0x7f1b080693e0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:08.967 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.965+0000 7f1b2d8a2700 1 --2- 172.21.15.99:0/2860318369 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1b2815a290 0x7f1b280b4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:08.967 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.965+0000 7f1b2d8a2700 1 --2- 172.21.15.99:0/2860318369 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1b2806acb0 0x7f1b28159120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:08.967 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.965+0000 7f1b2d8a2700 1 --2- 172.21.15.99:0/2860318369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b28159ec0 0x7f1b281558a0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:08.967 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.965+0000 7f1b2d8a2700 1 -- 172.21.15.99:0/2860318369 >> 172.21.15.99:0/2860318369 conn(0x7f1b280d1880 msgr2=0x7f1b280b1510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:08.967 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.966+0000 7f1b2d8a2700 1 -- 172.21.15.99:0/2860318369 shutdown_connections 2023-10-08T00:16:08.967 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:08.966+0000 7f1b2d8a2700 1 -- 172.21.15.99:0/2860318369 wait complete. 2023-10-08T00:16:08.978 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:16:09.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.091+0000 7fe7bd3e6700 1 Processor -- start 2023-10-08T00:16:09.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.091+0000 7fe7bd3e6700 1 -- start start 2023-10-08T00:16:09.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.092+0000 7fe7bd3e6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe7b815a620 0x7fe7b815d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:09.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.092+0000 7fe7bd3e6700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe7b815d580 0x7fe7b8158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:09.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.092+0000 7fe7bd3e6700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7b815cbb0 con 0x7fe7b8157870 2023-10-08T00:16:09.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.092+0000 7fe7bd3e6700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7b815bdb0 con 0x7fe7b815d580 2023-10-08T00:16:09.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.092+0000 7fe7bd3e6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b815bfa0 con 0x7fe7b815a620 2023-10-08T00:16:09.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.092+0000 7fe7b77fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe7b815d580 0x7fe7b8158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:09.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.092+0000 7fe7b77fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe7b815d580 0x7fe7b8158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46168/0 (socket says 172.21.15.99:46168) 2023-10-08T00:16:09.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.092+0000 7fe7b77fe700 1 -- 172.21.15.99:0/2709116751 learned_addr learned my addr 172.21.15.99:0/2709116751 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:09.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.092+0000 7fe7b67fc700 1 --2- 172.21.15.99:0/2709116751 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe7b815a620 0x7fe7b815d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:09.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.092+0000 7fe7b77fe700 1 -- 172.21.15.99:0/2709116751 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe7b815a620 msgr2=0x7fe7b815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:09.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.092+0000 7fe7b77fe700 1 --2- 172.21.15.99:0/2709116751 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe7b815a620 0x7fe7b815d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:09.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.092+0000 7fe7b77fe700 1 -- 172.21.15.99:0/2709116751 >> v1:172.21.15.191:6789/0 conn(0x7fe7b8157870 legacy=0x7fe7b815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:09.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.092+0000 7fe7b77fe700 1 -- 172.21.15.99:0/2709116751 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b815c280 con 0x7fe7b815d580 2023-10-08T00:16:09.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.092+0000 7fe7b77fe700 1 --2- 172.21.15.99:0/2709116751 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7b815d580 0x7fe7b8158110 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fe7a80099b0 tx=0x7fe7a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe36029e9ef7f5db server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:09.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.092+0000 7fe7b5ffb700 1 -- 172.21.15.99:0/2709116751 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a800b860 con 0x7fe7b815d580 2023-10-08T00:16:09.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.092+0000 7fe7b5ffb700 1 -- 172.21.15.99:0/2709116751 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7a800b9c0 con 0x7fe7b815d580 2023-10-08T00:16:09.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.092+0000 7fe7b5ffb700 1 -- 172.21.15.99:0/2709116751 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a800bd60 con 0x7fe7b815d580 2023-10-08T00:16:09.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.093+0000 7fe7bd3e6700 1 -- 172.21.15.99:0/2709116751 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7b815d580 msgr2=0x7fe7b8158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:09.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.093+0000 7fe7bd3e6700 1 --2- 172.21.15.99:0/2709116751 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7b815d580 0x7fe7b8158110 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fe7a80099b0 tx=0x7fe7a8009250 comp rx=0 tx=0).stop 2023-10-08T00:16:09.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.093+0000 7fe7bd3e6700 1 -- 172.21.15.99:0/2709116751 shutdown_connections 2023-10-08T00:16:09.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.093+0000 7fe7bd3e6700 1 --2- 172.21.15.99:0/2709116751 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7b815d580 0x7fe7b8158110 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:09.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.093+0000 7fe7bd3e6700 1 --2- 172.21.15.99:0/2709116751 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe7b815a620 0x7fe7b815d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:09.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.093+0000 7fe7bd3e6700 1 -- 172.21.15.99:0/2709116751 >> 172.21.15.99:0/2709116751 conn(0x7fe7b8068ce0 msgr2=0x7fe7b80690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:09.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.093+0000 7fe7bd3e6700 1 -- 172.21.15.99:0/2709116751 shutdown_connections 2023-10-08T00:16:09.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.093+0000 7fe7bd3e6700 1 -- 172.21.15.99:0/2709116751 wait complete. 2023-10-08T00:16:09.100 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.094+0000 7fe7bd3e6700 1 Processor -- start 2023-10-08T00:16:09.100 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.094+0000 7fe7bd3e6700 1 -- start start 2023-10-08T00:16:09.100 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.095+0000 7fe7bd3e6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe7b8157870 0x7fe7b8154f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:09.100 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.095+0000 7fe7bd3e6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7b815a620 0x7fe7b8159120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:09.100 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.095+0000 7fe7bd3e6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7b815d580 0x7fe7b814b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:09.100 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.095+0000 7fe7bd3e6700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b8150d10 con 0x7fe7b815d580 2023-10-08T00:16:09.101 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.095+0000 7fe7bd3e6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b80b06c0 con 0x7fe7b815a620 2023-10-08T00:16:09.101 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.095+0000 7fe7bd3e6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b8150f20 con 0x7fe7b8157870 2023-10-08T00:16:09.101 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.095+0000 7fe7b77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7b815d580 0x7fe7b814b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:09.101 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.095+0000 7fe7b77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7b815d580 0x7fe7b814b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46170/0 (socket says 172.21.15.99:46170) 2023-10-08T00:16:09.101 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.095+0000 7fe7b77fe700 1 -- 172.21.15.99:0/4133566924 learned_addr learned my addr 172.21.15.99:0/4133566924 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:09.102 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.095+0000 7fe7b67fc700 1 --2- 172.21.15.99:0/4133566924 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7b815a620 0x7fe7b8159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:09.102 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.095+0000 7fe7b6ffd700 1 --2- 172.21.15.99:0/4133566924 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe7b8157870 0x7fe7b8154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:09.102 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.095+0000 7fe7b77fe700 1 -- 172.21.15.99:0/4133566924 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe7b8157870 msgr2=0x7fe7b8154f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:09.102 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.095+0000 7fe7b77fe700 1 --2- 172.21.15.99:0/4133566924 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe7b8157870 0x7fe7b8154f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:09.102 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.095+0000 7fe7b77fe700 1 -- 172.21.15.99:0/4133566924 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7b815a620 msgr2=0x7fe7b8159120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:09.102 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.095+0000 7fe7b77fe700 1 --2- 172.21.15.99:0/4133566924 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7b815a620 0x7fe7b8159120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:09.102 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.095+0000 7fe7b77fe700 1 -- 172.21.15.99:0/4133566924 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a80094f0 con 0x7fe7b815d580 2023-10-08T00:16:09.102 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.095+0000 7fe7b77fe700 1 --2- 172.21.15.99:0/4133566924 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7b815d580 0x7fe7b814b680 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fe7a80099b0 tx=0x7fe7a8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:09.102 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.096+0000 7fe79effd700 1 -- 172.21.15.99:0/4133566924 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a800b860 con 0x7fe7b815d580 2023-10-08T00:16:09.102 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.096+0000 7fe79effd700 1 -- 172.21.15.99:0/4133566924 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7a801a630 con 0x7fe7b815d580 2023-10-08T00:16:09.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.096+0000 7fe7bd3e6700 1 -- 172.21.15.99:0/4133566924 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7b8151200 con 0x7fe7b815d580 2023-10-08T00:16:09.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.096+0000 7fe79effd700 1 -- 172.21.15.99:0/4133566924 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a801aa10 con 0x7fe7b815d580 2023-10-08T00:16:09.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.096+0000 7fe7bd3e6700 1 -- 172.21.15.99:0/4133566924 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7b81517c0 con 0x7fe7b815d580 2023-10-08T00:16:09.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.097+0000 7fe7bd3e6700 1 -- 172.21.15.99:0/4133566924 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7b804c3c0 con 0x7fe7b815d580 2023-10-08T00:16:09.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.097+0000 7fe79effd700 1 -- 172.21.15.99:0/4133566924 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe7a802d440 con 0x7fe7b815d580 2023-10-08T00:16:09.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.097+0000 7fe79effd700 1 --2- 172.21.15.99:0/4133566924 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe7a4066f30 0x7fe7a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:09.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.097+0000 7fe79effd700 1 -- 172.21.15.99:0/4133566924 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe7a8086070 con 0x7fe7b815d580 2023-10-08T00:16:09.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.097+0000 7fe7b6ffd700 1 --2- 172.21.15.99:0/4133566924 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe7a4066f30 0x7fe7a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:09.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.098+0000 7fe7b6ffd700 1 --2- 172.21.15.99:0/4133566924 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe7a4066f30 0x7fe7a40693e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0007fd0 tx=0x7fe7b00016f0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:09.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.101+0000 7fe79effd700 1 -- 172.21.15.99:0/4133566924 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe7a80532e0 con 0x7fe7b815d580 2023-10-08T00:16:09.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.325+0000 7fe7bd3e6700 1 -- 172.21.15.99:0/4133566924 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe7b815c560 con 0x7fe7b815d580 2023-10-08T00:16:09.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.326+0000 7fe79effd700 1 -- 172.21.15.99:0/4133566924 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4599 (secure 0 0 0) 0x7fe7a8056900 con 0x7fe7b815d580 2023-10-08T00:16:09.326 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:16:09.326 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":48,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5426,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/280880696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":280880696},{"type":"v1","addr":"172.21.15.191:6836","nonce":280880696}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":5902,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3148129562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":3148129562},{"type":"v1","addr":"172.21.15.99:6835","nonce":3148129562}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":5917,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/1825600944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":1825600944},{"type":"v1","addr":"172.21.15.191:6837","nonce":1825600944}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:15:40.595166+0000","modified":"2023-10-08T00:15:44.232916+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6360},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6360":{"gid":6360,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6834/146609901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":146609901},{"type":"v1","addr":"172.21.15.99:6834","nonce":146609901}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-08T00:16:09.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.328+0000 7fe7bd3e6700 1 -- 172.21.15.99:0/4133566924 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe7a4066f30 msgr2=0x7fe7a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:09.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.328+0000 7fe7bd3e6700 1 --2- 172.21.15.99:0/4133566924 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe7a4066f30 0x7fe7a40693e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0007fd0 tx=0x7fe7b00016f0 comp rx=0 tx=0).stop 2023-10-08T00:16:09.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.328+0000 7fe7bd3e6700 1 -- 172.21.15.99:0/4133566924 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7b815d580 msgr2=0x7fe7b814b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:09.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.328+0000 7fe7bd3e6700 1 --2- 172.21.15.99:0/4133566924 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7b815d580 0x7fe7b814b680 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fe7a80099b0 tx=0x7fe7a8017000 comp rx=0 tx=0).stop 2023-10-08T00:16:09.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.328+0000 7fe7bd3e6700 1 -- 172.21.15.99:0/4133566924 shutdown_connections 2023-10-08T00:16:09.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.328+0000 7fe7bd3e6700 1 --2- 172.21.15.99:0/4133566924 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe7a4066f30 0x7fe7a40693e0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:09.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.328+0000 7fe7bd3e6700 1 --2- 172.21.15.99:0/4133566924 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7b815d580 0x7fe7b814b680 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:09.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.328+0000 7fe7bd3e6700 1 --2- 172.21.15.99:0/4133566924 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe7b8157870 0x7fe7b8154f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:09.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.328+0000 7fe7bd3e6700 1 --2- 172.21.15.99:0/4133566924 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7b815a620 0x7fe7b8159120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:09.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.328+0000 7fe7bd3e6700 1 -- 172.21.15.99:0/4133566924 >> 172.21.15.99:0/4133566924 conn(0x7fe7b8068ce0 msgr2=0x7fe7b80b5a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:09.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.328+0000 7fe7bd3e6700 1 -- 172.21.15.99:0/4133566924 shutdown_connections 2023-10-08T00:16:09.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.328+0000 7fe7bd3e6700 1 -- 172.21.15.99:0/4133566924 wait complete. 2023-10-08T00:16:09.331 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 48 2023-10-08T00:16:09.340 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:16:09.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.453+0000 7f9325904700 1 Processor -- start 2023-10-08T00:16:09.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.453+0000 7f9325904700 1 -- start start 2023-10-08T00:16:09.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.454+0000 7f9325904700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9320157c40 0x7f9320158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:09.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.454+0000 7f9325904700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f932015cbb0 con 0x7f9320157870 2023-10-08T00:16:09.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.454+0000 7f9325904700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f932015bfa0 con 0x7f932015a620 2023-10-08T00:16:09.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.454+0000 7f9325904700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f932015bdb0 con 0x7f9320157c40 2023-10-08T00:16:09.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.454+0000 7f931effd700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f9320157870 0x7f932015b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:43728/0 (socket says 172.21.15.99:43728) 2023-10-08T00:16:09.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.454+0000 7f931effd700 1 -- 172.21.15.99:0/1488574062 learned_addr learned my addr 172.21.15.99:0/1488574062 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:09.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.454+0000 7f931f7fe700 1 --2- 172.21.15.99:0/1488574062 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9320157c40 0x7f9320158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:09.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.454+0000 7f931dffb700 1 -- 172.21.15.99:0/1488574062 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1926551735 0 0) 0x7f932015cbb0 con 0x7f9320157870 2023-10-08T00:16:09.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.454+0000 7f931dffb700 1 -- 172.21.15.99:0/1488574062 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9308003570 con 0x7f9320157870 2023-10-08T00:16:09.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.455+0000 7f931dffb700 1 -- 172.21.15.99:0/1488574062 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3667580803 0 0) 0x7f9308003570 con 0x7f9320157870 2023-10-08T00:16:09.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.455+0000 7f931dffb700 1 -- 172.21.15.99:0/1488574062 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9320157c40 msgr2=0x7f9320158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:09.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.455+0000 7f931dffb700 1 --2- 172.21.15.99:0/1488574062 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9320157c40 0x7f9320158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:09.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.455+0000 7f931dffb700 1 -- 172.21.15.99:0/1488574062 >> v1:172.21.15.191:6789/0 conn(0x7f932015a620 legacy=0x7f932015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:09.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.455+0000 7f931dffb700 1 -- 172.21.15.99:0/1488574062 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f932015c280 con 0x7f9320157870 2023-10-08T00:16:09.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.455+0000 7f931dffb700 1 -- 172.21.15.99:0/1488574062 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f93100017f0 con 0x7f9320157870 2023-10-08T00:16:09.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.455+0000 7f931dffb700 1 -- 172.21.15.99:0/1488574062 >> v1:172.21.15.99:6789/0 conn(0x7f9320157870 legacy=0x7f932015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:09.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.455+0000 7f931dffb700 1 --2- 172.21.15.99:0/1488574062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9308003a90 0x7f9308003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:09.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.455+0000 7f931dffb700 1 --2- 172.21.15.99:0/1488574062 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9320157c40 0x7f9308004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:09.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.455+0000 7f931dffb700 1 --2- 172.21.15.99:0/1488574062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9308004d50 0x7f93080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:09.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.455+0000 7f931dffb700 1 -- 172.21.15.99:0/1488574062 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930800bb10 con 0x7f9308003a90 2023-10-08T00:16:09.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.455+0000 7f931dffb700 1 -- 172.21.15.99:0/1488574062 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930800bc50 con 0x7f9308004d50 2023-10-08T00:16:09.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.455+0000 7f931dffb700 1 -- 172.21.15.99:0/1488574062 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f930800be10 con 0x7f9320157c40 2023-10-08T00:16:09.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.455+0000 7f931effd700 1 --2- 172.21.15.99:0/1488574062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9308003a90 0x7f9308003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:09.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.455+0000 7f931f7fe700 1 --2- 172.21.15.99:0/1488574062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9308004d50 0x7f93080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:09.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.455+0000 7f931e7fc700 1 --2- 172.21.15.99:0/1488574062 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9320157c40 0x7f9308004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:09.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.455+0000 7f931effd700 1 -- 172.21.15.99:0/1488574062 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9320157c40 msgr2=0x7f9308004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:09.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.455+0000 7f931effd700 1 --2- 172.21.15.99:0/1488574062 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9320157c40 0x7f9308004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:09.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.455+0000 7f931effd700 1 -- 172.21.15.99:0/1488574062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9308004d50 msgr2=0x7f93080051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:09.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.455+0000 7f931effd700 1 --2- 172.21.15.99:0/1488574062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9308004d50 0x7f93080051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:09.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.455+0000 7f931effd700 1 -- 172.21.15.99:0/1488574062 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9308003570 con 0x7f9308003a90 2023-10-08T00:16:09.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.455+0000 7f931f7fe700 1 --2- 172.21.15.99:0/1488574062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9308004d50 0x7f93080051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:09.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.455+0000 7f931effd700 1 --2- 172.21.15.99:0/1488574062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9308003a90 0x7f9308003e60 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f931000a950 tx=0x7f93100021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=620d5196289b042e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:09.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.455+0000 7f931dffb700 1 -- 172.21.15.99:0/1488574062 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93100070d0 con 0x7f9308003a90 2023-10-08T00:16:09.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.455+0000 7f931dffb700 1 -- 172.21.15.99:0/1488574062 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9310007250 con 0x7f9308003a90 2023-10-08T00:16:09.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.455+0000 7f931dffb700 1 -- 172.21.15.99:0/1488574062 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9310007610 con 0x7f9308003a90 2023-10-08T00:16:09.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.456+0000 7f9325904700 1 -- 172.21.15.99:0/1488574062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9308003a90 msgr2=0x7f9308003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:09.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.456+0000 7f9325904700 1 --2- 172.21.15.99:0/1488574062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9308003a90 0x7f9308003e60 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f931000a950 tx=0x7f93100021f0 comp rx=0 tx=0).stop 2023-10-08T00:16:09.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.456+0000 7f931f7fe700 1 -- 172.21.15.99:0/1488574062 reap_dead start 2023-10-08T00:16:09.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.456+0000 7f9325904700 1 -- 172.21.15.99:0/1488574062 shutdown_connections 2023-10-08T00:16:09.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.456+0000 7f9325904700 1 -- 172.21.15.99:0/1488574062 >> 172.21.15.99:0/1488574062 conn(0x7f9320068ce0 msgr2=0x7f93200690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:09.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.456+0000 7f9325904700 1 -- 172.21.15.99:0/1488574062 shutdown_connections 2023-10-08T00:16:09.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.456+0000 7f9325904700 1 -- 172.21.15.99:0/1488574062 wait complete. 2023-10-08T00:16:09.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.457+0000 7f9325904700 1 Processor -- start 2023-10-08T00:16:09.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.457+0000 7f9325904700 1 -- start start 2023-10-08T00:16:09.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.458+0000 7f9325904700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9308003a90 0x7f9320155420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:09.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.458+0000 7f9325904700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93200b5a40 0x7f93200b5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:09.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.458+0000 7f9325904700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93200c73d0 0x7f93200c7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:09.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.458+0000 7f9325904700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93200b63f0 con 0x7f93200b5a40 2023-10-08T00:16:09.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.458+0000 7f9325904700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93200c7d80 con 0x7f93200c73d0 2023-10-08T00:16:09.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.458+0000 7f9325904700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93200c6e90 con 0x7f9308003a90 2023-10-08T00:16:09.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.458+0000 7f931e7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93200b5a40 0x7f93200b5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:09.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.458+0000 7f931e7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93200b5a40 0x7f93200b5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46184/0 (socket says 172.21.15.99:46184) 2023-10-08T00:16:09.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.458+0000 7f931e7fc700 1 -- 172.21.15.99:0/322562483 learned_addr learned my addr 172.21.15.99:0/322562483 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:09.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.458+0000 7f931effd700 1 --2- 172.21.15.99:0/322562483 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9308003a90 0x7f9320155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:09.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.458+0000 7f931f7fe700 1 --2- 172.21.15.99:0/322562483 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93200c73d0 0x7f93200c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:09.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.458+0000 7f931e7fc700 1 -- 172.21.15.99:0/322562483 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9308003a90 msgr2=0x7f9320155420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:09.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.458+0000 7f931e7fc700 1 --2- 172.21.15.99:0/322562483 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9308003a90 0x7f9320155420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:09.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.458+0000 7f931e7fc700 1 -- 172.21.15.99:0/322562483 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93200c73d0 msgr2=0x7f93200c7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:09.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.458+0000 7f931e7fc700 1 --2- 172.21.15.99:0/322562483 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93200c73d0 0x7f93200c7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:09.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.458+0000 7f931e7fc700 1 -- 172.21.15.99:0/322562483 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93200c5b10 con 0x7f93200b5a40 2023-10-08T00:16:09.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.458+0000 7f931e7fc700 1 --2- 172.21.15.99:0/322562483 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93200b5a40 0x7f93200b5eb0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f931400e890 tx=0x7f9314009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:09.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.458+0000 7f9306ffd700 1 -- 172.21.15.99:0/322562483 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9314012590 con 0x7f93200b5a40 2023-10-08T00:16:09.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.459+0000 7f9306ffd700 1 -- 172.21.15.99:0/322562483 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93140126f0 con 0x7f93200b5a40 2023-10-08T00:16:09.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.459+0000 7f9306ffd700 1 -- 172.21.15.99:0/322562483 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9314012ab0 con 0x7f93200b5a40 2023-10-08T00:16:09.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.459+0000 7f9325904700 1 -- 172.21.15.99:0/322562483 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93200c5df0 con 0x7f93200b5a40 2023-10-08T00:16:09.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.459+0000 7f9325904700 1 -- 172.21.15.99:0/322562483 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93200c63b0 con 0x7f93200b5a40 2023-10-08T00:16:09.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.459+0000 7f9325904700 1 -- 172.21.15.99:0/322562483 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f932004c3c0 con 0x7f93200b5a40 2023-10-08T00:16:09.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.460+0000 7f9306ffd700 1 -- 172.21.15.99:0/322562483 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f931401d460 con 0x7f93200b5a40 2023-10-08T00:16:09.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.460+0000 7f9306ffd700 1 --2- 172.21.15.99:0/322562483 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f930c066f30 0x7f930c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:09.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.460+0000 7f9306ffd700 1 -- 172.21.15.99:0/322562483 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f93140846d0 con 0x7f93200b5a40 2023-10-08T00:16:09.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.460+0000 7f931effd700 1 --2- 172.21.15.99:0/322562483 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f930c066f30 0x7f930c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:09.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.461+0000 7f931effd700 1 --2- 172.21.15.99:0/322562483 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f930c066f30 0x7f930c0693e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9310002250 tx=0x7f9310015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:09.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.464+0000 7f9306ffd700 1 -- 172.21.15.99:0/322562483 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9314051940 con 0x7f93200b5a40 2023-10-08T00:16:09.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.648+0000 7f9325904700 1 -- 172.21.15.99:0/322562483 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f932015c560 con 0x7f93200b5a40 2023-10-08T00:16:09.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.649+0000 7f9306ffd700 1 -- 172.21.15.99:0/322562483 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+15365 (secure 0 0 0) 0x7f9314054f60 con 0x7f93200b5a40 2023-10-08T00:16:09.650 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:16:09.650 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":112,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:16:05.407718+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:15:38.848324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-08T00:15:39.561379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:16:09.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.651+0000 7f9325904700 1 -- 172.21.15.99:0/322562483 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f930c066f30 msgr2=0x7f930c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:09.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.651+0000 7f9325904700 1 --2- 172.21.15.99:0/322562483 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f930c066f30 0x7f930c0693e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9310002250 tx=0x7f9310015000 comp rx=0 tx=0).stop 2023-10-08T00:16:09.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.651+0000 7f9325904700 1 -- 172.21.15.99:0/322562483 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93200b5a40 msgr2=0x7f93200b5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:09.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.651+0000 7f9325904700 1 --2- 172.21.15.99:0/322562483 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93200b5a40 0x7f93200b5eb0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f931400e890 tx=0x7f9314009040 comp rx=0 tx=0).stop 2023-10-08T00:16:09.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.651+0000 7f9325904700 1 -- 172.21.15.99:0/322562483 shutdown_connections 2023-10-08T00:16:09.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.651+0000 7f9325904700 1 --2- 172.21.15.99:0/322562483 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f930c066f30 0x7f930c0693e0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:09.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.651+0000 7f9325904700 1 --2- 172.21.15.99:0/322562483 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93200c73d0 0x7f93200c7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:09.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.651+0000 7f9325904700 1 --2- 172.21.15.99:0/322562483 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9308003a90 0x7f9320155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:09.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.651+0000 7f9325904700 1 --2- 172.21.15.99:0/322562483 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93200b5a40 0x7f93200b5eb0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:09.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.651+0000 7f9325904700 1 -- 172.21.15.99:0/322562483 >> 172.21.15.99:0/322562483 conn(0x7f9320068ce0 msgr2=0x7f9320159820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:09.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.651+0000 7f9325904700 1 -- 172.21.15.99:0/322562483 shutdown_connections 2023-10-08T00:16:09.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.651+0000 7f9325904700 1 -- 172.21.15.99:0/322562483 wait complete. 2023-10-08T00:16:09.664 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T00:16:09.665 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:16:09.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.777+0000 7f0e91d87700 1 Processor -- start 2023-10-08T00:16:09.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.777+0000 7f0e91d87700 1 -- start start 2023-10-08T00:16:09.781 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.778+0000 7f0e91d87700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f0e8c15a620 0x7f0e8c15d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:09.781 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.778+0000 7f0e91d87700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0e8c15d580 0x7f0e8c158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:09.781 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.778+0000 7f0e91d87700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e8c15cbb0 con 0x7f0e8c157870 2023-10-08T00:16:09.781 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.778+0000 7f0e91d87700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e8c15bdb0 con 0x7f0e8c15d580 2023-10-08T00:16:09.781 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.778+0000 7f0e91d87700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e8c15bfa0 con 0x7f0e8c15a620 2023-10-08T00:16:09.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.778+0000 7f0e8bfff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0e8c15d580 0x7f0e8c158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:09.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.778+0000 7f0e8bfff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0e8c15d580 0x7f0e8c158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46192/0 (socket says 172.21.15.99:46192) 2023-10-08T00:16:09.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.778+0000 7f0e8bfff700 1 -- 172.21.15.99:0/1138152108 learned_addr learned my addr 172.21.15.99:0/1138152108 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:09.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.778+0000 7f0e8affd700 1 --2- 172.21.15.99:0/1138152108 >> v2:172.21.15.191:3300/0 conn(0x7f0e8c15a620 0x7f0e8c15d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:09.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.779+0000 7f0e8bfff700 1 -- 172.21.15.99:0/1138152108 >> v2:172.21.15.191:3300/0 conn(0x7f0e8c15a620 msgr2=0x7f0e8c15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:09.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.779+0000 7f0e8bfff700 1 --2- 172.21.15.99:0/1138152108 >> v2:172.21.15.191:3300/0 conn(0x7f0e8c15a620 0x7f0e8c15d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:09.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.779+0000 7f0e8bfff700 1 -- 172.21.15.99:0/1138152108 >> v1:172.21.15.191:6789/0 conn(0x7f0e8c157870 legacy=0x7f0e8c15b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:09.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.779+0000 7f0e8bfff700 1 -- 172.21.15.99:0/1138152108 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e8c15c280 con 0x7f0e8c15d580 2023-10-08T00:16:09.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.779+0000 7f0e8bfff700 1 --2- 172.21.15.99:0/1138152108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e8c15d580 0x7f0e8c158110 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c15cb20 tx=0x7f0e84009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=295f2d3629a66158 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:09.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.779+0000 7f0e8a7fc700 1 -- 172.21.15.99:0/1138152108 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8400e070 con 0x7f0e8c15d580 2023-10-08T00:16:09.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.779+0000 7f0e8a7fc700 1 -- 172.21.15.99:0/1138152108 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e8400ba10 con 0x7f0e8c15d580 2023-10-08T00:16:09.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.779+0000 7f0e8a7fc700 1 -- 172.21.15.99:0/1138152108 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e840092e0 con 0x7f0e8c15d580 2023-10-08T00:16:09.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.779+0000 7f0e91d87700 1 -- 172.21.15.99:0/1138152108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e8c15d580 msgr2=0x7f0e8c158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:09.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.779+0000 7f0e91d87700 1 --2- 172.21.15.99:0/1138152108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e8c15d580 0x7f0e8c158110 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c15cb20 tx=0x7f0e84009840 comp rx=0 tx=0).stop 2023-10-08T00:16:09.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.779+0000 7f0e91d87700 1 -- 172.21.15.99:0/1138152108 shutdown_connections 2023-10-08T00:16:09.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.779+0000 7f0e91d87700 1 --2- 172.21.15.99:0/1138152108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e8c15d580 0x7f0e8c158110 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:09.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.779+0000 7f0e91d87700 1 --2- 172.21.15.99:0/1138152108 >> v2:172.21.15.191:3300/0 conn(0x7f0e8c15a620 0x7f0e8c15d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:09.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.779+0000 7f0e91d87700 1 -- 172.21.15.99:0/1138152108 >> 172.21.15.99:0/1138152108 conn(0x7f0e8c068ce0 msgr2=0x7f0e8c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:09.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.779+0000 7f0e91d87700 1 -- 172.21.15.99:0/1138152108 shutdown_connections 2023-10-08T00:16:09.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.779+0000 7f0e91d87700 1 -- 172.21.15.99:0/1138152108 wait complete. 2023-10-08T00:16:09.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.781+0000 7f0e91d87700 1 Processor -- start 2023-10-08T00:16:09.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.781+0000 7f0e91d87700 1 -- start start 2023-10-08T00:16:09.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.781+0000 7f0e91d87700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e8c157870 0x7f0e8c16a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:09.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.781+0000 7f0e91d87700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e8c15a620 0x7f0e8c0b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:09.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.781+0000 7f0e91d87700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e8c15d580 0x7f0e8c0b2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:09.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.781+0000 7f0e91d87700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8c14c110 con 0x7f0e8c15a620 2023-10-08T00:16:09.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.781+0000 7f0e91d87700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8c0b1b20 con 0x7f0e8c15d580 2023-10-08T00:16:09.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.781+0000 7f0e91d87700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8c0b0f10 con 0x7f0e8c157870 2023-10-08T00:16:09.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.781+0000 7f0e8affd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e8c15a620 0x7f0e8c0b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:09.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.781+0000 7f0e8affd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e8c15a620 0x7f0e8c0b06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46202/0 (socket says 172.21.15.99:46202) 2023-10-08T00:16:09.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.781+0000 7f0e8affd700 1 -- 172.21.15.99:0/1109585338 learned_addr learned my addr 172.21.15.99:0/1109585338 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:09.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.782+0000 7f0e8bfff700 1 --2- 172.21.15.99:0/1109585338 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e8c15d580 0x7f0e8c0b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:09.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.782+0000 7f0e8b7fe700 1 --2- 172.21.15.99:0/1109585338 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e8c157870 0x7f0e8c16a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:09.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.782+0000 7f0e8affd700 1 -- 172.21.15.99:0/1109585338 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e8c157870 msgr2=0x7f0e8c16a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:09.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.782+0000 7f0e8affd700 1 --2- 172.21.15.99:0/1109585338 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e8c157870 0x7f0e8c16a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:09.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.782+0000 7f0e8affd700 1 -- 172.21.15.99:0/1109585338 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e8c15d580 msgr2=0x7f0e8c0b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:09.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.782+0000 7f0e8affd700 1 --2- 172.21.15.99:0/1109585338 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e8c15d580 0x7f0e8c0b2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:09.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.782+0000 7f0e8affd700 1 -- 172.21.15.99:0/1109585338 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e84009ae0 con 0x7f0e8c15a620 2023-10-08T00:16:09.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.782+0000 7f0e8b7fe700 1 --2- 172.21.15.99:0/1109585338 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e8c157870 0x7f0e8c16a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:16:09.788 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.782+0000 7f0e8affd700 1 --2- 172.21.15.99:0/1109585338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e8c15a620 0x7f0e8c0b06c0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f0e8000d860 tx=0x7f0e80008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:09.788 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.782+0000 7f0e737fe700 1 -- 172.21.15.99:0/1109585338 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e800125f0 con 0x7f0e8c15a620 2023-10-08T00:16:09.788 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.782+0000 7f0e737fe700 1 -- 172.21.15.99:0/1109585338 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e80012750 con 0x7f0e8c15a620 2023-10-08T00:16:09.788 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.782+0000 7f0e91d87700 1 -- 172.21.15.99:0/1109585338 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e8c0b11f0 con 0x7f0e8c15a620 2023-10-08T00:16:09.788 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.782+0000 7f0e737fe700 1 -- 172.21.15.99:0/1109585338 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e80012ad0 con 0x7f0e8c15a620 2023-10-08T00:16:09.788 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.782+0000 7f0e91d87700 1 -- 172.21.15.99:0/1109585338 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e8c0b17b0 con 0x7f0e8c15a620 2023-10-08T00:16:09.788 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.783+0000 7f0e91d87700 1 -- 172.21.15.99:0/1109585338 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e8c04c3c0 con 0x7f0e8c15a620 2023-10-08T00:16:09.788 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.783+0000 7f0e737fe700 1 -- 172.21.15.99:0/1109585338 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0e8001e460 con 0x7f0e8c15a620 2023-10-08T00:16:09.788 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.784+0000 7f0e737fe700 1 --2- 172.21.15.99:0/1109585338 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0e78066f30 0x7f0e780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:09.788 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.784+0000 7f0e737fe700 1 -- 172.21.15.99:0/1109585338 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0e80084ba0 con 0x7f0e8c15a620 2023-10-08T00:16:09.788 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.784+0000 7f0e8b7fe700 1 --2- 172.21.15.99:0/1109585338 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0e78066f30 0x7f0e780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:09.788 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.784+0000 7f0e8b7fe700 1 --2- 172.21.15.99:0/1109585338 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0e78066f30 0x7f0e780693e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c16b670 tx=0x7f0e7c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:09.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:09.788+0000 7f0e737fe700 1 -- 172.21.15.99:0/1109585338 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0e800558a0 con 0x7f0e8c15a620 2023-10-08T00:16:10.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.012+0000 7f0e91d87700 1 -- 172.21.15.99:0/1109585338 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e8c15c560 con 0x7f0e8c15a620 2023-10-08T00:16:10.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.013+0000 7f0e737fe700 1 -- 172.21.15.99:0/1109585338 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4599 (secure 0 0 0) 0x7f0e80055430 con 0x7f0e8c15a620 2023-10-08T00:16:10.013 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:16:10.013 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":48,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5426,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/280880696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":280880696},{"type":"v1","addr":"172.21.15.191:6836","nonce":280880696}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":5902,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3148129562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":3148129562},{"type":"v1","addr":"172.21.15.99:6835","nonce":3148129562}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":5917,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/1825600944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":1825600944},{"type":"v1","addr":"172.21.15.191:6837","nonce":1825600944}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:15:40.595166+0000","modified":"2023-10-08T00:15:44.232916+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6360},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6360":{"gid":6360,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6834/146609901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":146609901},{"type":"v1","addr":"172.21.15.99:6834","nonce":146609901}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-08T00:16:10.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.014+0000 7f0e91d87700 1 -- 172.21.15.99:0/1109585338 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0e78066f30 msgr2=0x7f0e780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:10.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.014+0000 7f0e91d87700 1 --2- 172.21.15.99:0/1109585338 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0e78066f30 0x7f0e780693e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c16b670 tx=0x7f0e7c009040 comp rx=0 tx=0).stop 2023-10-08T00:16:10.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.014+0000 7f0e91d87700 1 -- 172.21.15.99:0/1109585338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e8c15a620 msgr2=0x7f0e8c0b06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:10.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.014+0000 7f0e91d87700 1 --2- 172.21.15.99:0/1109585338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e8c15a620 0x7f0e8c0b06c0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f0e8000d860 tx=0x7f0e80008040 comp rx=0 tx=0).stop 2023-10-08T00:16:10.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.014+0000 7f0e91d87700 1 -- 172.21.15.99:0/1109585338 shutdown_connections 2023-10-08T00:16:10.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.014+0000 7f0e91d87700 1 --2- 172.21.15.99:0/1109585338 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0e78066f30 0x7f0e780693e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:10.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.014+0000 7f0e91d87700 1 --2- 172.21.15.99:0/1109585338 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e8c15d580 0x7f0e8c0b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:10.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.014+0000 7f0e91d87700 1 --2- 172.21.15.99:0/1109585338 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e8c157870 0x7f0e8c16a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:10.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.015+0000 7f0e91d87700 1 --2- 172.21.15.99:0/1109585338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e8c15a620 0x7f0e8c0b06c0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:10.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.015+0000 7f0e91d87700 1 -- 172.21.15.99:0/1109585338 >> 172.21.15.99:0/1109585338 conn(0x7f0e8c068ce0 msgr2=0x7f0e8c15da10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:10.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.015+0000 7f0e91d87700 1 -- 172.21.15.99:0/1109585338 shutdown_connections 2023-10-08T00:16:10.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.015+0000 7f0e91d87700 1 -- 172.21.15.99:0/1109585338 wait complete. 2023-10-08T00:16:10.017 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 48 2023-10-08T00:16:10.027 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:16:10.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.142+0000 7f588f838700 1 Processor -- start 2023-10-08T00:16:10.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.143+0000 7f588f838700 1 -- start start 2023-10-08T00:16:10.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.143+0000 7f588f838700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5888159cf0 0x7f588815e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:10.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.143+0000 7f588f838700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f58880cf1c0 0x7f5888068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:10.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.143+0000 7f588f838700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58880cca20 con 0x7f588815a0c0 2023-10-08T00:16:10.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.143+0000 7f588f838700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f588815ea40 con 0x7f58880cf1c0 2023-10-08T00:16:10.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.143+0000 7f588f838700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5888067660 con 0x7f5888159cf0 2023-10-08T00:16:10.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.144+0000 7f588ddd5700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f58880cf1c0 0x7f5888068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:10.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.144+0000 7f588ddd5700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f58880cf1c0 0x7f5888068390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46218/0 (socket says 172.21.15.99:46218) 2023-10-08T00:16:10.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.144+0000 7f588ddd5700 1 -- 172.21.15.99:0/3600097175 learned_addr learned my addr 172.21.15.99:0/3600097175 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:10.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.144+0000 7f588d5d4700 1 --2- 172.21.15.99:0/3600097175 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5888159cf0 0x7f588815e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:10.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.144+0000 7f588ddd5700 1 -- 172.21.15.99:0/3600097175 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5888159cf0 msgr2=0x7f588815e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:10.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.144+0000 7f588ddd5700 1 --2- 172.21.15.99:0/3600097175 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5888159cf0 0x7f588815e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:10.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.144+0000 7f588ddd5700 1 -- 172.21.15.99:0/3600097175 >> v1:172.21.15.191:6789/0 conn(0x7f588815a0c0 legacy=0x7f58880ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:10.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.144+0000 7f588ddd5700 1 -- 172.21.15.99:0/3600097175 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5888067f00 con 0x7f58880cf1c0 2023-10-08T00:16:10.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.144+0000 7f588ddd5700 1 --2- 172.21.15.99:0/3600097175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58880cf1c0 0x7f5888068390 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f5880009f20 tx=0x7f5880009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de2d39c95da90a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:10.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.144+0000 7f587ffff700 1 -- 172.21.15.99:0/3600097175 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588000e070 con 0x7f58880cf1c0 2023-10-08T00:16:10.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.144+0000 7f587ffff700 1 -- 172.21.15.99:0/3600097175 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f588000ba40 con 0x7f58880cf1c0 2023-10-08T00:16:10.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.144+0000 7f587ffff700 1 -- 172.21.15.99:0/3600097175 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58800092e0 con 0x7f58880cf1c0 2023-10-08T00:16:10.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.145+0000 7f588f838700 1 -- 172.21.15.99:0/3600097175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58880cf1c0 msgr2=0x7f5888068390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:10.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.145+0000 7f588f838700 1 --2- 172.21.15.99:0/3600097175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58880cf1c0 0x7f5888068390 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f5880009f20 tx=0x7f5880009840 comp rx=0 tx=0).stop 2023-10-08T00:16:10.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.145+0000 7f588f838700 1 -- 172.21.15.99:0/3600097175 shutdown_connections 2023-10-08T00:16:10.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.145+0000 7f588f838700 1 --2- 172.21.15.99:0/3600097175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58880cf1c0 0x7f5888068390 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:10.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.145+0000 7f588f838700 1 --2- 172.21.15.99:0/3600097175 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5888159cf0 0x7f588815e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:10.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.145+0000 7f588f838700 1 -- 172.21.15.99:0/3600097175 >> 172.21.15.99:0/3600097175 conn(0x7f588806c6e0 msgr2=0x7f5888069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:10.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.145+0000 7f588f838700 1 -- 172.21.15.99:0/3600097175 shutdown_connections 2023-10-08T00:16:10.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.145+0000 7f588f838700 1 -- 172.21.15.99:0/3600097175 wait complete. 2023-10-08T00:16:10.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.146+0000 7f588f838700 1 Processor -- start 2023-10-08T00:16:10.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.146+0000 7f588f838700 1 -- start start 2023-10-08T00:16:10.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.147+0000 7f588f838700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f58880cf1c0 0x7f58880b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:10.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.147+0000 7f588f838700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5888159cf0 0x7f588815a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:10.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.147+0000 7f588f838700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f588815a0c0 0x7f588815aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:10.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.147+0000 7f588f838700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58880b38b0 con 0x7f588815a0c0 2023-10-08T00:16:10.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.147+0000 7f588f838700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58880b4280 con 0x7f5888159cf0 2023-10-08T00:16:10.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.147+0000 7f588f838700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58880b2ca0 con 0x7f58880cf1c0 2023-10-08T00:16:10.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.147+0000 7f588ddd5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f588815a0c0 0x7f588815aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:10.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.147+0000 7f588ddd5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f588815a0c0 0x7f588815aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46234/0 (socket says 172.21.15.99:46234) 2023-10-08T00:16:10.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.147+0000 7f588ddd5700 1 -- 172.21.15.99:0/3875051241 learned_addr learned my addr 172.21.15.99:0/3875051241 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:10.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.147+0000 7f588cdd3700 1 --2- 172.21.15.99:0/3875051241 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5888159cf0 0x7f588815a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:10.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.147+0000 7f588d5d4700 1 --2- 172.21.15.99:0/3875051241 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f58880cf1c0 0x7f58880b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:10.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.147+0000 7f588ddd5700 1 -- 172.21.15.99:0/3875051241 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f58880cf1c0 msgr2=0x7f58880b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:10.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.147+0000 7f588ddd5700 1 --2- 172.21.15.99:0/3875051241 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f58880cf1c0 0x7f58880b3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:10.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.147+0000 7f588ddd5700 1 -- 172.21.15.99:0/3875051241 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5888159cf0 msgr2=0x7f588815a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:10.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.147+0000 7f588ddd5700 1 --2- 172.21.15.99:0/3875051241 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5888159cf0 0x7f588815a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:10.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.147+0000 7f588ddd5700 1 -- 172.21.15.99:0/3875051241 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5880009ae0 con 0x7f588815a0c0 2023-10-08T00:16:10.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.147+0000 7f588ddd5700 1 --2- 172.21.15.99:0/3875051241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f588815a0c0 0x7f588815aaa0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f588806de20 tx=0x7f5880017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:10.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.148+0000 7f587d7fa700 1 -- 172.21.15.99:0/3875051241 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58800092e0 con 0x7f588815a0c0 2023-10-08T00:16:10.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.148+0000 7f587d7fa700 1 -- 172.21.15.99:0/3875051241 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5880012d00 con 0x7f588815a0c0 2023-10-08T00:16:10.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.148+0000 7f588f838700 1 -- 172.21.15.99:0/3875051241 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58880b2f80 con 0x7f588815a0c0 2023-10-08T00:16:10.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.148+0000 7f587d7fa700 1 -- 172.21.15.99:0/3875051241 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588001e660 con 0x7f588815a0c0 2023-10-08T00:16:10.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.148+0000 7f588f838700 1 -- 172.21.15.99:0/3875051241 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58880b3540 con 0x7f588815a0c0 2023-10-08T00:16:10.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.148+0000 7f588f838700 1 -- 172.21.15.99:0/3875051241 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f588804c3c0 con 0x7f588815a0c0 2023-10-08T00:16:10.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.149+0000 7f587d7fa700 1 -- 172.21.15.99:0/3875051241 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f588001ea50 con 0x7f588815a0c0 2023-10-08T00:16:10.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.149+0000 7f587d7fa700 1 --2- 172.21.15.99:0/3875051241 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f586c066f30 0x7f586c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:10.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.149+0000 7f587d7fa700 1 -- 172.21.15.99:0/3875051241 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f58800860d0 con 0x7f588815a0c0 2023-10-08T00:16:10.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.149+0000 7f588d5d4700 1 --2- 172.21.15.99:0/3875051241 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f586c066f30 0x7f586c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:10.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.150+0000 7f588d5d4700 1 --2- 172.21.15.99:0/3875051241 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f586c066f30 0x7f586c0693e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f58880ce5e0 tx=0x7f5884008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:10.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.153+0000 7f587d7fa700 1 -- 172.21.15.99:0/3875051241 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5880053340 con 0x7f588815a0c0 2023-10-08T00:16:10.379 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.379+0000 7f588f838700 1 -- 172.21.15.99:0/3875051241 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58880b39f0 con 0x7f588815a0c0 2023-10-08T00:16:10.380 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.379+0000 7f587d7fa700 1 -- 172.21.15.99:0/3875051241 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4599 (secure 0 0 0) 0x7f5880056960 con 0x7f588815a0c0 2023-10-08T00:16:10.380 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:16:10.380 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":48,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5426,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/280880696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":280880696},{"type":"v1","addr":"172.21.15.191:6836","nonce":280880696}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":5902,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3148129562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":3148129562},{"type":"v1","addr":"172.21.15.99:6835","nonce":3148129562}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":5917,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/1825600944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":1825600944},{"type":"v1","addr":"172.21.15.191:6837","nonce":1825600944}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:15:40.595166+0000","modified":"2023-10-08T00:15:44.232916+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6360},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6360":{"gid":6360,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6834/146609901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":146609901},{"type":"v1","addr":"172.21.15.99:6834","nonce":146609901}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-08T00:16:10.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.381+0000 7f588f838700 1 -- 172.21.15.99:0/3875051241 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f586c066f30 msgr2=0x7f586c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:10.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.381+0000 7f588f838700 1 --2- 172.21.15.99:0/3875051241 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f586c066f30 0x7f586c0693e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f58880ce5e0 tx=0x7f5884008040 comp rx=0 tx=0).stop 2023-10-08T00:16:10.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.381+0000 7f588f838700 1 -- 172.21.15.99:0/3875051241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f588815a0c0 msgr2=0x7f588815aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:10.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.381+0000 7f588f838700 1 --2- 172.21.15.99:0/3875051241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f588815a0c0 0x7f588815aaa0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f588806de20 tx=0x7f5880017000 comp rx=0 tx=0).stop 2023-10-08T00:16:10.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.381+0000 7f588f838700 1 -- 172.21.15.99:0/3875051241 shutdown_connections 2023-10-08T00:16:10.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.381+0000 7f588f838700 1 --2- 172.21.15.99:0/3875051241 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f586c066f30 0x7f586c0693e0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:10.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.381+0000 7f588f838700 1 --2- 172.21.15.99:0/3875051241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f588815a0c0 0x7f588815aaa0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:10.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.381+0000 7f588f838700 1 --2- 172.21.15.99:0/3875051241 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f58880cf1c0 0x7f58880b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:10.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.381+0000 7f588f838700 1 --2- 172.21.15.99:0/3875051241 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5888159cf0 0x7f588815a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:10.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.381+0000 7f588f838700 1 -- 172.21.15.99:0/3875051241 >> 172.21.15.99:0/3875051241 conn(0x7f588806c6e0 msgr2=0x7f58880c9150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:10.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.381+0000 7f588f838700 1 -- 172.21.15.99:0/3875051241 shutdown_connections 2023-10-08T00:16:10.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.381+0000 7f588f838700 1 -- 172.21.15.99:0/3875051241 wait complete. 2023-10-08T00:16:10.383 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 48 2023-10-08T00:16:10.394 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:16:10.509 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.509+0000 7f1e00b02700 1 Processor -- start 2023-10-08T00:16:10.509 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.509+0000 7f1e00b02700 1 -- start start 2023-10-08T00:16:10.512 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.510+0000 7f1e00b02700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1dfc152010 0x7f1dfc15b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:10.512 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.510+0000 7f1e00b02700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1dfc1523e0 0x7f1dfc06c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:10.512 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.510+0000 7f1e00b02700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dfc06c010 con 0x7f1dfc158e50 2023-10-08T00:16:10.512 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.510+0000 7f1e00b02700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dfc06bd10 con 0x7f1dfc1523e0 2023-10-08T00:16:10.512 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.510+0000 7f1e00b02700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dfc15c060 con 0x7f1dfc152010 2023-10-08T00:16:10.512 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.510+0000 7f1df9d9b700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1dfc1523e0 0x7f1dfc06c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:10.512 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.510+0000 7f1df9d9b700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1dfc1523e0 0x7f1dfc06c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46242/0 (socket says 172.21.15.99:46242) 2023-10-08T00:16:10.513 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.510+0000 7f1df9d9b700 1 -- 172.21.15.99:0/2462218479 learned_addr learned my addr 172.21.15.99:0/2462218479 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:10.513 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.510+0000 7f1df9d9b700 1 -- 172.21.15.99:0/2462218479 >> v2:172.21.15.191:3300/0 conn(0x7f1dfc152010 msgr2=0x7f1dfc15b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:10.513 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.510+0000 7f1dfa59c700 1 --2- 172.21.15.99:0/2462218479 >> v2:172.21.15.191:3300/0 conn(0x7f1dfc152010 0x7f1dfc15b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:10.513 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.510+0000 7f1df9d9b700 1 --2- 172.21.15.99:0/2462218479 >> v2:172.21.15.191:3300/0 conn(0x7f1dfc152010 0x7f1dfc15b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:10.513 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.510+0000 7f1df9d9b700 1 -- 172.21.15.99:0/2462218479 >> v1:172.21.15.191:6789/0 conn(0x7f1dfc158e50 legacy=0x7f1dfc0cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:10.513 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.510+0000 7f1df9d9b700 1 -- 172.21.15.99:0/2462218479 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dfc15c340 con 0x7f1dfc1523e0 2023-10-08T00:16:10.513 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.510+0000 7f1df9d9b700 1 --2- 172.21.15.99:0/2462218479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dfc1523e0 0x7f1dfc06c4a0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc0cc680 tx=0x7f1df0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3bda027fdc88fb7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:10.513 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.510+0000 7f1df959a700 1 -- 172.21.15.99:0/2462218479 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df000b820 con 0x7f1dfc1523e0 2023-10-08T00:16:10.513 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.510+0000 7f1df959a700 1 -- 172.21.15.99:0/2462218479 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1df000b980 con 0x7f1dfc1523e0 2023-10-08T00:16:10.513 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.510+0000 7f1df959a700 1 -- 172.21.15.99:0/2462218479 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df000bd20 con 0x7f1dfc1523e0 2023-10-08T00:16:10.513 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.511+0000 7f1e00b02700 1 -- 172.21.15.99:0/2462218479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dfc1523e0 msgr2=0x7f1dfc06c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:10.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.511+0000 7f1e00b02700 1 --2- 172.21.15.99:0/2462218479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dfc1523e0 0x7f1dfc06c4a0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc0cc680 tx=0x7f1df0009250 comp rx=0 tx=0).stop 2023-10-08T00:16:10.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.511+0000 7f1e00b02700 1 -- 172.21.15.99:0/2462218479 shutdown_connections 2023-10-08T00:16:10.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.511+0000 7f1e00b02700 1 --2- 172.21.15.99:0/2462218479 >> v2:172.21.15.191:3300/0 conn(0x7f1dfc152010 0x7f1dfc15b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:10.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.511+0000 7f1e00b02700 1 --2- 172.21.15.99:0/2462218479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dfc1523e0 0x7f1dfc06c4a0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:10.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.511+0000 7f1e00b02700 1 -- 172.21.15.99:0/2462218479 >> 172.21.15.99:0/2462218479 conn(0x7f1dfc069870 msgr2=0x7f1dfc069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:10.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.511+0000 7f1e00b02700 1 -- 172.21.15.99:0/2462218479 shutdown_connections 2023-10-08T00:16:10.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.511+0000 7f1e00b02700 1 -- 172.21.15.99:0/2462218479 wait complete. 2023-10-08T00:16:10.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.512+0000 7f1e00b02700 1 Processor -- start 2023-10-08T00:16:10.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.512+0000 7f1e00b02700 1 -- start start 2023-10-08T00:16:10.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.513+0000 7f1e00b02700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1dfc152010 0x7f1dfc168c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:10.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.513+0000 7f1e00b02700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dfc1523e0 0x7f1dfc152820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:10.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.513+0000 7f1e00b02700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1dfc158e50 0x7f1dfc14faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:10.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.513+0000 7f1e00b02700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc152d60 con 0x7f1dfc1523e0 2023-10-08T00:16:10.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.513+0000 7f1e00b02700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc14ffe0 con 0x7f1dfc152010 2023-10-08T00:16:10.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.513+0000 7f1e00b02700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc168790 con 0x7f1dfc158e50 2023-10-08T00:16:10.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.513+0000 7f1df9d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dfc1523e0 0x7f1dfc152820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:10.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.513+0000 7f1df9d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dfc1523e0 0x7f1dfc152820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46250/0 (socket says 172.21.15.99:46250) 2023-10-08T00:16:10.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.513+0000 7f1df9d9b700 1 -- 172.21.15.99:0/1252641039 learned_addr learned my addr 172.21.15.99:0/1252641039 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:10.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.513+0000 7f1dfa59c700 1 --2- 172.21.15.99:0/1252641039 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1dfc152010 0x7f1dfc168c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:10.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.513+0000 7f1dfad9d700 1 --2- 172.21.15.99:0/1252641039 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1dfc158e50 0x7f1dfc14faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:10.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.513+0000 7f1df9d9b700 1 -- 172.21.15.99:0/1252641039 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1dfc158e50 msgr2=0x7f1dfc14faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:10.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.513+0000 7f1df9d9b700 1 --2- 172.21.15.99:0/1252641039 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1dfc158e50 0x7f1dfc14faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:10.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.513+0000 7f1df9d9b700 1 -- 172.21.15.99:0/1252641039 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1dfc152010 msgr2=0x7f1dfc168c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:10.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.513+0000 7f1df9d9b700 1 --2- 172.21.15.99:0/1252641039 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1dfc152010 0x7f1dfc168c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:10.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.513+0000 7f1df9d9b700 1 -- 172.21.15.99:0/1252641039 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df00094f0 con 0x7f1dfc1523e0 2023-10-08T00:16:10.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.513+0000 7f1dfad9d700 1 --2- 172.21.15.99:0/1252641039 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1dfc158e50 0x7f1dfc14faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:16:10.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.513+0000 7f1df9d9b700 1 --2- 172.21.15.99:0/1252641039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dfc1523e0 0x7f1dfc152820 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f1df0009290 tx=0x7f1df0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:10.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.513+0000 7f1dea7fc700 1 -- 172.21.15.99:0/1252641039 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df0011e20 con 0x7f1dfc1523e0 2023-10-08T00:16:10.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.513+0000 7f1dea7fc700 1 -- 172.21.15.99:0/1252641039 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1df001b680 con 0x7f1dfc1523e0 2023-10-08T00:16:10.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.513+0000 7f1dea7fc700 1 -- 172.21.15.99:0/1252641039 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df001ba40 con 0x7f1dfc1523e0 2023-10-08T00:16:10.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.513+0000 7f1e00b02700 1 -- 172.21.15.99:0/1252641039 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dfc168160 con 0x7f1dfc1523e0 2023-10-08T00:16:10.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.513+0000 7f1e00b02700 1 -- 172.21.15.99:0/1252641039 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dfc153640 con 0x7f1dfc1523e0 2023-10-08T00:16:10.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.514+0000 7f1e00b02700 1 -- 172.21.15.99:0/1252641039 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dfc169160 con 0x7f1dfc1523e0 2023-10-08T00:16:10.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.515+0000 7f1dea7fc700 1 -- 172.21.15.99:0/1252641039 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1df001c450 con 0x7f1dfc1523e0 2023-10-08T00:16:10.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.515+0000 7f1dea7fc700 1 --2- 172.21.15.99:0/1252641039 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1ddc066f30 0x7f1ddc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:10.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.515+0000 7f1dea7fc700 1 -- 172.21.15.99:0/1252641039 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1df0086290 con 0x7f1dfc1523e0 2023-10-08T00:16:10.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.515+0000 7f1dfa59c700 1 --2- 172.21.15.99:0/1252641039 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1ddc066f30 0x7f1ddc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:10.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.516+0000 7f1dfa59c700 1 --2- 172.21.15.99:0/1252641039 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1ddc066f30 0x7f1ddc0693e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1dec006fd0 tx=0x7f1dec008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:10.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.519+0000 7f1dea7fc700 1 -- 172.21.15.99:0/1252641039 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1df0053500 con 0x7f1dfc1523e0 2023-10-08T00:16:10.702 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.702+0000 7f1e00b02700 1 -- 172.21.15.99:0/1252641039 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1dfc153830 con 0x7f1dfc1523e0 2023-10-08T00:16:10.703 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.703+0000 7f1dea7fc700 1 -- 172.21.15.99:0/1252641039 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+15365 (secure 0 0 0) 0x7f1df0056b20 con 0x7f1dfc1523e0 2023-10-08T00:16:10.703 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:16:10.704 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":112,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:16:05.407718+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:15:38.848324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-08T00:15:39.561379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:16:10.706 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.705+0000 7f1e00b02700 1 -- 172.21.15.99:0/1252641039 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1ddc066f30 msgr2=0x7f1ddc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:10.706 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.705+0000 7f1e00b02700 1 --2- 172.21.15.99:0/1252641039 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1ddc066f30 0x7f1ddc0693e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1dec006fd0 tx=0x7f1dec008040 comp rx=0 tx=0).stop 2023-10-08T00:16:10.706 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.705+0000 7f1e00b02700 1 -- 172.21.15.99:0/1252641039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dfc1523e0 msgr2=0x7f1dfc152820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:10.706 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.705+0000 7f1e00b02700 1 --2- 172.21.15.99:0/1252641039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dfc1523e0 0x7f1dfc152820 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f1df0009290 tx=0x7f1df0016000 comp rx=0 tx=0).stop 2023-10-08T00:16:10.706 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.705+0000 7f1e00b02700 1 -- 172.21.15.99:0/1252641039 shutdown_connections 2023-10-08T00:16:10.706 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.705+0000 7f1e00b02700 1 --2- 172.21.15.99:0/1252641039 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1ddc066f30 0x7f1ddc0693e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:10.706 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.705+0000 7f1e00b02700 1 --2- 172.21.15.99:0/1252641039 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1dfc152010 0x7f1dfc168c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:10.706 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.705+0000 7f1e00b02700 1 --2- 172.21.15.99:0/1252641039 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1dfc158e50 0x7f1dfc14faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:10.706 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.705+0000 7f1e00b02700 1 --2- 172.21.15.99:0/1252641039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dfc1523e0 0x7f1dfc152820 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:10.707 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.705+0000 7f1e00b02700 1 -- 172.21.15.99:0/1252641039 >> 172.21.15.99:0/1252641039 conn(0x7f1dfc069870 msgr2=0x7f1dfc06c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:10.707 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.705+0000 7f1e00b02700 1 -- 172.21.15.99:0/1252641039 shutdown_connections 2023-10-08T00:16:10.707 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.705+0000 7f1e00b02700 1 -- 172.21.15.99:0/1252641039 wait complete. 2023-10-08T00:16:10.717 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T00:16:10.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.830+0000 7f7e2ec52700 1 Processor -- start 2023-10-08T00:16:10.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.831+0000 7f7e2ec52700 1 -- start start 2023-10-08T00:16:10.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.831+0000 7f7e2ec52700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e2815a620 0x7f7e2815d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:10.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.831+0000 7f7e2ec52700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e2815cbb0 con 0x7f7e2815d580 2023-10-08T00:16:10.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.831+0000 7f7e2ec52700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e2815bfa0 con 0x7f7e28157870 2023-10-08T00:16:10.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.831+0000 7f7e2ec52700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2815bdb0 con 0x7f7e2815a620 2023-10-08T00:16:10.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.832+0000 7f7e2d1ef700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f7e2815d580 0x7f7e28158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:43730/0 (socket says 172.21.15.99:43730) 2023-10-08T00:16:10.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.832+0000 7f7e2d1ef700 1 -- 172.21.15.99:0/1828723415 learned_addr learned my addr 172.21.15.99:0/1828723415 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:10.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.832+0000 7f7e1ffff700 1 --2- 172.21.15.99:0/1828723415 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e2815a620 0x7f7e2815d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:10.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.832+0000 7f7e1f7fe700 1 -- 172.21.15.99:0/1828723415 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3158052737 0 0) 0x7f7e2815cbb0 con 0x7f7e2815d580 2023-10-08T00:16:10.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.832+0000 7f7e1f7fe700 1 -- 172.21.15.99:0/1828723415 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e10003570 con 0x7f7e2815d580 2023-10-08T00:16:10.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.832+0000 7f7e1f7fe700 1 -- 172.21.15.99:0/1828723415 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2204387094 0 0) 0x7f7e2815bfa0 con 0x7f7e28157870 2023-10-08T00:16:10.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.832+0000 7f7e1f7fe700 1 -- 172.21.15.99:0/1828723415 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e2815cbb0 con 0x7f7e28157870 2023-10-08T00:16:10.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.832+0000 7f7e1f7fe700 1 -- 172.21.15.99:0/1828723415 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3344415080 0 0) 0x7f7e10003570 con 0x7f7e2815d580 2023-10-08T00:16:10.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.832+0000 7f7e1f7fe700 1 -- 172.21.15.99:0/1828723415 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e2815a620 msgr2=0x7f7e2815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:10.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.832+0000 7f7e1f7fe700 1 --2- 172.21.15.99:0/1828723415 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e2815a620 0x7f7e2815d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:10.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.832+0000 7f7e1f7fe700 1 -- 172.21.15.99:0/1828723415 >> v1:172.21.15.191:6789/0 conn(0x7f7e28157870 legacy=0x7f7e2815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:10.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.832+0000 7f7e1f7fe700 1 -- 172.21.15.99:0/1828723415 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e2815c280 con 0x7f7e2815d580 2023-10-08T00:16:10.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.832+0000 7f7e1f7fe700 1 -- 172.21.15.99:0/1828723415 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f7e20001470 con 0x7f7e2815d580 2023-10-08T00:16:10.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.832+0000 7f7e1f7fe700 1 -- 172.21.15.99:0/1828723415 >> v1:172.21.15.99:6789/0 conn(0x7f7e2815d580 legacy=0x7f7e28158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:10.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.832+0000 7f7e1f7fe700 1 --2- 172.21.15.99:0/1828723415 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e2815a620 0x7f7e10003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:10.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.832+0000 7f7e1f7fe700 1 --2- 172.21.15.99:0/1828723415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e10004180 0x7f7e10004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:10.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.832+0000 7f7e1f7fe700 1 --2- 172.21.15.99:0/1828723415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e10004d50 0x7f7e100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:10.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.832+0000 7f7e1f7fe700 1 -- 172.21.15.99:0/1828723415 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1000bb70 con 0x7f7e10004d50 2023-10-08T00:16:10.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.832+0000 7f7e1f7fe700 1 -- 172.21.15.99:0/1828723415 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1000bce0 con 0x7f7e10004180 2023-10-08T00:16:10.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.832+0000 7f7e1f7fe700 1 -- 172.21.15.99:0/1828723415 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1000bea0 con 0x7f7e2815a620 2023-10-08T00:16:10.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.832+0000 7f7e2d1ef700 1 --2- 172.21.15.99:0/1828723415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e10004d50 0x7f7e100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:10.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.832+0000 7f7e2c9ee700 1 --2- 172.21.15.99:0/1828723415 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e2815a620 0x7f7e10003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:10.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.832+0000 7f7e1ffff700 1 --2- 172.21.15.99:0/1828723415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e10004180 0x7f7e10004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:10.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.833+0000 7f7e2d1ef700 1 -- 172.21.15.99:0/1828723415 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e2815a620 msgr2=0x7f7e10003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:10.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.833+0000 7f7e2d1ef700 1 --2- 172.21.15.99:0/1828723415 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e2815a620 0x7f7e10003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:10.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.833+0000 7f7e2d1ef700 1 -- 172.21.15.99:0/1828723415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e10004180 msgr2=0x7f7e10004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:10.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.833+0000 7f7e2d1ef700 1 --2- 172.21.15.99:0/1828723415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e10004180 0x7f7e10004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:10.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.833+0000 7f7e2d1ef700 1 -- 172.21.15.99:0/1828723415 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e10003570 con 0x7f7e10004d50 2023-10-08T00:16:10.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.833+0000 7f7e2d1ef700 1 --2- 172.21.15.99:0/1828723415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e10004d50 0x7f7e100051c0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f7e2000c2a0 tx=0x7f7e20001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d59c8d62db9df338 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:10.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.833+0000 7f7e1f7fe700 1 -- 172.21.15.99:0/1828723415 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2000cb90 con 0x7f7e10004d50 2023-10-08T00:16:10.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.833+0000 7f7e1f7fe700 1 -- 172.21.15.99:0/1828723415 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e2000ccf0 con 0x7f7e10004d50 2023-10-08T00:16:10.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.833+0000 7f7e1f7fe700 1 -- 172.21.15.99:0/1828723415 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e20005b70 con 0x7f7e10004d50 2023-10-08T00:16:10.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.833+0000 7f7e2ec52700 1 -- 172.21.15.99:0/1828723415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e10004d50 msgr2=0x7f7e100051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:10.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.833+0000 7f7e2ec52700 1 --2- 172.21.15.99:0/1828723415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e10004d50 0x7f7e100051c0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f7e2000c2a0 tx=0x7f7e20001e10 comp rx=0 tx=0).stop 2023-10-08T00:16:10.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.833+0000 7f7e2d1ef700 1 -- 172.21.15.99:0/1828723415 reap_dead start 2023-10-08T00:16:10.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.833+0000 7f7e2ec52700 1 -- 172.21.15.99:0/1828723415 shutdown_connections 2023-10-08T00:16:10.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.833+0000 7f7e2ec52700 1 -- 172.21.15.99:0/1828723415 >> 172.21.15.99:0/1828723415 conn(0x7f7e28068ce0 msgr2=0x7f7e280690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:10.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.833+0000 7f7e2ec52700 1 -- 172.21.15.99:0/1828723415 shutdown_connections 2023-10-08T00:16:10.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.833+0000 7f7e2ec52700 1 -- 172.21.15.99:0/1828723415 wait complete. 2023-10-08T00:16:10.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.834+0000 7f7e2ec52700 1 Processor -- start 2023-10-08T00:16:10.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.835+0000 7f7e2ec52700 1 -- start start 2023-10-08T00:16:10.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.835+0000 7f7e2ec52700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e10004d50 0x7f7e28154340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:10.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.835+0000 7f7e2ec52700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e2814b9e0 0x7f7e28154d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:10.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.835+0000 7f7e2ec52700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e28155260 0x7f7e28151f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:10.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.835+0000 7f7e2ec52700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e281556e0 con 0x7f7e2814b9e0 2023-10-08T00:16:10.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.835+0000 7f7e2ec52700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e28151ae0 con 0x7f7e28155260 2023-10-08T00:16:10.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.835+0000 7f7e2ec52700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e28154880 con 0x7f7e10004d50 2023-10-08T00:16:10.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.835+0000 7f7e1ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e2814b9e0 0x7f7e28154d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:10.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.835+0000 7f7e1ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e2814b9e0 0x7f7e28154d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46272/0 (socket says 172.21.15.99:46272) 2023-10-08T00:16:10.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.835+0000 7f7e1ffff700 1 -- 172.21.15.99:0/4096194513 learned_addr learned my addr 172.21.15.99:0/4096194513 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:10.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.836+0000 7f7e2d1ef700 1 --2- 172.21.15.99:0/4096194513 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e28155260 0x7f7e28151f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:10.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.836+0000 7f7e2c9ee700 1 --2- 172.21.15.99:0/4096194513 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e10004d50 0x7f7e28154340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:10.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.836+0000 7f7e1ffff700 1 -- 172.21.15.99:0/4096194513 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e10004d50 msgr2=0x7f7e28154340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:10.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.836+0000 7f7e1ffff700 1 --2- 172.21.15.99:0/4096194513 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e10004d50 0x7f7e28154340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:10.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.836+0000 7f7e1ffff700 1 -- 172.21.15.99:0/4096194513 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e28155260 msgr2=0x7f7e28151f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:10.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.836+0000 7f7e1ffff700 1 --2- 172.21.15.99:0/4096194513 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e28155260 0x7f7e28151f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:10.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.836+0000 7f7e1ffff700 1 -- 172.21.15.99:0/4096194513 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e28150ed0 con 0x7f7e2814b9e0 2023-10-08T00:16:10.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.836+0000 7f7e2c9ee700 1 --2- 172.21.15.99:0/4096194513 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e10004d50 0x7f7e28154340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:10.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.836+0000 7f7e1ffff700 1 --2- 172.21.15.99:0/4096194513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e2814b9e0 0x7f7e28154d20 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f7e18011b70 tx=0x7f7e180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:10.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.836+0000 7f7e1cff9700 1 -- 172.21.15.99:0/4096194513 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1800f940 con 0x7f7e2814b9e0 2023-10-08T00:16:10.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.836+0000 7f7e1cff9700 1 -- 172.21.15.99:0/4096194513 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e1800faa0 con 0x7f7e2814b9e0 2023-10-08T00:16:10.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.836+0000 7f7e2ec52700 1 -- 172.21.15.99:0/4096194513 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e281511b0 con 0x7f7e2814b9e0 2023-10-08T00:16:10.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.836+0000 7f7e1cff9700 1 -- 172.21.15.99:0/4096194513 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1800fe20 con 0x7f7e2814b9e0 2023-10-08T00:16:10.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.836+0000 7f7e2ec52700 1 -- 172.21.15.99:0/4096194513 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e28151770 con 0x7f7e2814b9e0 2023-10-08T00:16:10.844 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.837+0000 7f7e2ec52700 1 -- 172.21.15.99:0/4096194513 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e2804c3c0 con 0x7f7e2814b9e0 2023-10-08T00:16:10.844 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.838+0000 7f7e1cff9700 1 -- 172.21.15.99:0/4096194513 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7e18014650 con 0x7f7e2814b9e0 2023-10-08T00:16:10.844 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.838+0000 7f7e1cff9700 1 --2- 172.21.15.99:0/4096194513 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7e14066f30 0x7f7e140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:10.844 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.838+0000 7f7e1cff9700 1 -- 172.21.15.99:0/4096194513 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7e18018030 con 0x7f7e2814b9e0 2023-10-08T00:16:10.844 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.838+0000 7f7e2c9ee700 1 --2- 172.21.15.99:0/4096194513 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7e14066f30 0x7f7e140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:10.844 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.839+0000 7f7e2c9ee700 1 --2- 172.21.15.99:0/4096194513 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7e14066f30 0x7f7e140693e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7e240018b0 tx=0x7f7e240017a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:10.845 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:10.842+0000 7f7e1cff9700 1 -- 172.21.15.99:0/4096194513 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7e18052230 con 0x7f7e2814b9e0 2023-10-08T00:16:11.067 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.066+0000 7f7e2ec52700 1 -- 172.21.15.99:0/4096194513 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7e28151c20 con 0x7f7e2814b9e0 2023-10-08T00:16:11.067 INFO:tasks.ceph.mon.a.smithi099.stderr:2023-10-08T00:16:11.067+0000 7f6c60fa2700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T00:16:11.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.074+0000 7f7e1cff9700 1 -- 172.21.15.99:0/4096194513 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v49) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7e18022090 con 0x7f7e2814b9e0 2023-10-08T00:16:11.075 INFO:tasks.ceph.mds.c.smithi099.stderr: -8769> 2023-10-08T00:15:46.199+0000 7f8c00ff9700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T00:16:11.078 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.076+0000 7f7e2ec52700 1 -- 172.21.15.99:0/4096194513 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7e14066f30 msgr2=0x7f7e140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:11.078 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.076+0000 7f7e2ec52700 1 --2- 172.21.15.99:0/4096194513 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7e14066f30 0x7f7e140693e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7e240018b0 tx=0x7f7e240017a0 comp rx=0 tx=0).stop 2023-10-08T00:16:11.078 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.076+0000 7f7e2ec52700 1 -- 172.21.15.99:0/4096194513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e2814b9e0 msgr2=0x7f7e28154d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:11.078 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.076+0000 7f7e2ec52700 1 --2- 172.21.15.99:0/4096194513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e2814b9e0 0x7f7e28154d20 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f7e18011b70 tx=0x7f7e180091a0 comp rx=0 tx=0).stop 2023-10-08T00:16:11.079 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.076+0000 7f7e2ec52700 1 -- 172.21.15.99:0/4096194513 shutdown_connections 2023-10-08T00:16:11.079 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.076+0000 7f7e2ec52700 1 --2- 172.21.15.99:0/4096194513 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7e14066f30 0x7f7e140693e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:11.079 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.076+0000 7f7e2ec52700 1 --2- 172.21.15.99:0/4096194513 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e28155260 0x7f7e28151f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:11.079 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.076+0000 7f7e2ec52700 1 --2- 172.21.15.99:0/4096194513 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e10004d50 0x7f7e28154340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:11.080 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.076+0000 7f7e2ec52700 1 --2- 172.21.15.99:0/4096194513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e2814b9e0 0x7f7e28154d20 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:11.080 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.076+0000 7f7e2ec52700 1 -- 172.21.15.99:0/4096194513 >> 172.21.15.99:0/4096194513 conn(0x7f7e28068ce0 msgr2=0x7f7e280b6470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:11.080 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.077+0000 7f7e2ec52700 1 -- 172.21.15.99:0/4096194513 shutdown_connections 2023-10-08T00:16:11.080 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.077+0000 7f7e2ec52700 1 -- 172.21.15.99:0/4096194513 wait complete. 2023-10-08T00:16:11.080 INFO:teuthology.orchestra.run.smithi099.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T00:16:11.090 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T00:16:11.108 INFO:tasks.ceph.mds.c.smithi099.stdout:starting mds.c at 2023-10-08T00:16:11.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.208+0000 7fe4a5fae700 1 Processor -- start 2023-10-08T00:16:11.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.209+0000 7fe4a5fae700 1 -- start start 2023-10-08T00:16:11.212 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.209+0000 7fe4a5fae700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fe4a0143490 0x7fe4a00ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:11.212 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.209+0000 7fe4a5fae700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4a0143860 0x7fe4a006a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:11.212 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.209+0000 7fe4a5fae700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe4a0070700 0x7fe4a0070b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:11.212 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.209+0000 7fe4a5fae700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4a00710c0 con 0x7fe4a0070700 2023-10-08T00:16:11.212 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.209+0000 7fe4a5fae700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4a00cb9a0 con 0x7fe4a0143490 2023-10-08T00:16:11.212 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.209+0000 7fe4a5fae700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a006ec70 con 0x7fe4a0143860 2023-10-08T00:16:11.212 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.209+0000 7fe49ffff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe4a0070700 0x7fe4a0070b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:11.212 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.210+0000 7fe49ffff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe4a0070700 0x7fe4a0070b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46300/0 (socket says 172.21.15.99:46300) 2023-10-08T00:16:11.212 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.210+0000 7fe49ffff700 1 -- 172.21.15.99:0/598958826 learned_addr learned my addr 172.21.15.99:0/598958826 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:11.212 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.210+0000 7fe49f7fe700 1 --2- 172.21.15.99:0/598958826 >> v2:172.21.15.191:3300/0 conn(0x7fe4a0143490 0x7fe4a00ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:11.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.210+0000 7fe49effd700 1 --2- 172.21.15.99:0/598958826 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4a0143860 0x7fe4a006a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:11.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.210+0000 7fe49ffff700 1 -- 172.21.15.99:0/598958826 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4a0143860 msgr2=0x7fe4a006a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:11.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.210+0000 7fe49ffff700 1 --2- 172.21.15.99:0/598958826 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4a0143860 0x7fe4a006a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:11.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.210+0000 7fe49ffff700 1 -- 172.21.15.99:0/598958826 >> v2:172.21.15.191:3300/0 conn(0x7fe4a0143490 msgr2=0x7fe4a00ccfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:11.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.210+0000 7fe49ffff700 1 --2- 172.21.15.99:0/598958826 >> v2:172.21.15.191:3300/0 conn(0x7fe4a0143490 0x7fe4a00ccfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:11.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.210+0000 7fe49ffff700 1 -- 172.21.15.99:0/598958826 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a006f510 con 0x7fe4a0070700 2023-10-08T00:16:11.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.210+0000 7fe49f7fe700 1 --2- 172.21.15.99:0/598958826 >> v2:172.21.15.191:3300/0 conn(0x7fe4a0143490 0x7fe4a00ccfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:11.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.210+0000 7fe49ffff700 1 --2- 172.21.15.99:0/598958826 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4a0070700 0x7fe4a0070b80 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fe4a00ce740 tx=0x7fe498009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d286292281d7f3e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:11.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.210+0000 7fe49e7fc700 1 -- 172.21.15.99:0/598958826 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe498005c60 con 0x7fe4a0070700 2023-10-08T00:16:11.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.210+0000 7fe49e7fc700 1 -- 172.21.15.99:0/598958826 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe498005dc0 con 0x7fe4a0070700 2023-10-08T00:16:11.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.210+0000 7fe49e7fc700 1 -- 172.21.15.99:0/598958826 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49800b860 con 0x7fe4a0070700 2023-10-08T00:16:11.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.211+0000 7fe4a5fae700 1 -- 172.21.15.99:0/598958826 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4a0070700 msgr2=0x7fe4a0070b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:11.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.211+0000 7fe4a5fae700 1 --2- 172.21.15.99:0/598958826 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4a0070700 0x7fe4a0070b80 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fe4a00ce740 tx=0x7fe498009250 comp rx=0 tx=0).stop 2023-10-08T00:16:11.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.211+0000 7fe4a5fae700 1 -- 172.21.15.99:0/598958826 shutdown_connections 2023-10-08T00:16:11.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.211+0000 7fe4a5fae700 1 --2- 172.21.15.99:0/598958826 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4a0070700 0x7fe4a0070b80 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:11.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.211+0000 7fe4a5fae700 1 --2- 172.21.15.99:0/598958826 >> v2:172.21.15.191:3300/0 conn(0x7fe4a0143490 0x7fe4a00ccfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:11.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.211+0000 7fe4a5fae700 1 --2- 172.21.15.99:0/598958826 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4a0143860 0x7fe4a006a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:11.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.211+0000 7fe4a5fae700 1 -- 172.21.15.99:0/598958826 >> 172.21.15.99:0/598958826 conn(0x7fe4a00cdaa0 msgr2=0x7fe4a00cdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:11.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.211+0000 7fe4a5fae700 1 -- 172.21.15.99:0/598958826 shutdown_connections 2023-10-08T00:16:11.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.211+0000 7fe4a5fae700 1 -- 172.21.15.99:0/598958826 wait complete. 2023-10-08T00:16:11.217 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.212+0000 7fe4a5fae700 1 Processor -- start 2023-10-08T00:16:11.217 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.212+0000 7fe4a5fae700 1 -- start start 2023-10-08T00:16:11.217 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.213+0000 7fe4a5fae700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4a0070700 0x7fe4a0132370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:11.217 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.213+0000 7fe4a5fae700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4a0143490 0x7fe4a015e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:11.217 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.213+0000 7fe4a5fae700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4a0143860 0x7fe4a00b2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:11.217 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.213+0000 7fe4a5fae700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a016d2e0 con 0x7fe4a0070700 2023-10-08T00:16:11.218 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.213+0000 7fe4a5fae700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a015f3b0 con 0x7fe4a0143490 2023-10-08T00:16:11.218 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.213+0000 7fe4a5fae700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a016d530 con 0x7fe4a0143860 2023-10-08T00:16:11.218 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.213+0000 7fe49f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4a0070700 0x7fe4a0132370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:11.218 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.213+0000 7fe49f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4a0070700 0x7fe4a0132370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46316/0 (socket says 172.21.15.99:46316) 2023-10-08T00:16:11.218 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.213+0000 7fe49f7fe700 1 -- 172.21.15.99:0/311891135 learned_addr learned my addr 172.21.15.99:0/311891135 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:11.218 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.213+0000 7fe49f7fe700 1 -- 172.21.15.99:0/311891135 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4a0143860 msgr2=0x7fe4a00b2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:11.218 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.213+0000 7fe49f7fe700 1 --2- 172.21.15.99:0/311891135 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4a0143860 0x7fe4a00b2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:11.218 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.213+0000 7fe49effd700 1 --2- 172.21.15.99:0/311891135 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4a0143490 0x7fe4a015e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:11.218 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.213+0000 7fe49f7fe700 1 -- 172.21.15.99:0/311891135 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4a0143490 msgr2=0x7fe4a015e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:11.218 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.213+0000 7fe49f7fe700 1 --2- 172.21.15.99:0/311891135 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4a0143490 0x7fe4a015e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:11.218 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.213+0000 7fe49f7fe700 1 -- 172.21.15.99:0/311891135 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4980094d0 con 0x7fe4a0070700 2023-10-08T00:16:11.219 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.213+0000 7fe49effd700 1 --2- 172.21.15.99:0/311891135 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4a0143490 0x7fe4a015e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:16:11.219 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.213+0000 7fe49f7fe700 1 --2- 172.21.15.99:0/311891135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4a0070700 0x7fe4a0132370 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fe49000d1d0 tx=0x7fe490009a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:11.219 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.213+0000 7fe4877fe700 1 -- 172.21.15.99:0/311891135 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49000de70 con 0x7fe4a0070700 2023-10-08T00:16:11.219 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.213+0000 7fe4877fe700 1 -- 172.21.15.99:0/311891135 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe49000f040 con 0x7fe4a0070700 2023-10-08T00:16:11.219 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.213+0000 7fe4a5fae700 1 -- 172.21.15.99:0/311891135 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4a016d810 con 0x7fe4a0070700 2023-10-08T00:16:11.219 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.213+0000 7fe4877fe700 1 -- 172.21.15.99:0/311891135 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe490015670 con 0x7fe4a0070700 2023-10-08T00:16:11.219 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.213+0000 7fe4a5fae700 1 -- 172.21.15.99:0/311891135 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4a01328b0 con 0x7fe4a0070700 2023-10-08T00:16:11.219 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.214+0000 7fe4a5fae700 1 -- 172.21.15.99:0/311891135 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4a004c3c0 con 0x7fe4a0070700 2023-10-08T00:16:11.219 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.215+0000 7fe4877fe700 1 -- 172.21.15.99:0/311891135 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe490015a60 con 0x7fe4a0070700 2023-10-08T00:16:11.219 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.215+0000 7fe4877fe700 1 --2- 172.21.15.99:0/311891135 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe48c066f30 0x7fe48c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:11.220 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.215+0000 7fe4877fe700 1 -- 172.21.15.99:0/311891135 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5791+0+0 (secure 0 0 0) 0x7fe490084a60 con 0x7fe4a0070700 2023-10-08T00:16:11.220 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.215+0000 7fe49effd700 1 --2- 172.21.15.99:0/311891135 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe48c066f30 0x7fe48c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:11.220 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.216+0000 7fe49effd700 1 --2- 172.21.15.99:0/311891135 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe48c066f30 0x7fe48c0693e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe4a016d240 tx=0x7fe494008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:11.220 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.219+0000 7fe4877fe700 1 -- 172.21.15.99:0/311891135 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe49004e5a0 con 0x7fe4a0070700 2023-10-08T00:16:11.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:11.446+0000 7fe4a5fae700 1 -- 172.21.15.99:0/311891135 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe4a00638a0 con 0x7fe4a0070700 2023-10-08T00:16:12.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.107+0000 7fe4877fe700 1 -- 172.21.15.99:0/311891135 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v50) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe490051d30 con 0x7fe4a0070700 2023-10-08T00:16:12.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.109+0000 7fe4a5fae700 1 -- 172.21.15.99:0/311891135 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe48c066f30 msgr2=0x7fe48c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:12.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.109+0000 7fe4a5fae700 1 --2- 172.21.15.99:0/311891135 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe48c066f30 0x7fe48c0693e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe4a016d240 tx=0x7fe494008040 comp rx=0 tx=0).stop 2023-10-08T00:16:12.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.110+0000 7fe4a5fae700 1 -- 172.21.15.99:0/311891135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4a0070700 msgr2=0x7fe4a0132370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:12.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.110+0000 7fe4a5fae700 1 --2- 172.21.15.99:0/311891135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4a0070700 0x7fe4a0132370 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fe49000d1d0 tx=0x7fe490009a90 comp rx=0 tx=0).stop 2023-10-08T00:16:12.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.110+0000 7fe4a5fae700 1 -- 172.21.15.99:0/311891135 shutdown_connections 2023-10-08T00:16:12.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.110+0000 7fe4a5fae700 1 --2- 172.21.15.99:0/311891135 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe48c066f30 0x7fe48c0693e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:12.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.110+0000 7fe4a5fae700 1 --2- 172.21.15.99:0/311891135 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4a0143860 0x7fe4a00b2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:12.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.110+0000 7fe4a5fae700 1 --2- 172.21.15.99:0/311891135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4a0070700 0x7fe4a0132370 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:12.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.110+0000 7fe4a5fae700 1 --2- 172.21.15.99:0/311891135 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4a0143490 0x7fe4a015e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:12.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.110+0000 7fe4a5fae700 1 -- 172.21.15.99:0/311891135 >> 172.21.15.99:0/311891135 conn(0x7fe4a00cdaa0 msgr2=0x7fe4a00ac570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:12.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.110+0000 7fe4a5fae700 1 -- 172.21.15.99:0/311891135 shutdown_connections 2023-10-08T00:16:12.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.110+0000 7fe4a5fae700 1 -- 172.21.15.99:0/311891135 wait complete. 2023-10-08T00:16:12.123 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T00:16:12.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.284+0000 7f6093ae8700 1 Processor -- start 2023-10-08T00:16:12.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.284+0000 7f6093ae8700 1 -- start start 2023-10-08T00:16:12.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.285+0000 7f6093ae8700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f608c159e50 0x7f608c0b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:12.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.285+0000 7f6093ae8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f608c068cb0 0x7f608c0690e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:12.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.285+0000 7f6093ae8700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f608c0b0d60 con 0x7f608c0b3370 2023-10-08T00:16:12.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.285+0000 7f6093ae8700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f608c069620 con 0x7f608c159e50 2023-10-08T00:16:12.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.285+0000 7f6093ae8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f608c0d15b0 con 0x7f608c068cb0 2023-10-08T00:16:12.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.285+0000 7f6091884700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f608c159e50 0x7f608c0b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:12.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.285+0000 7f6091884700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f608c159e50 0x7f608c0b2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46332/0 (socket says 172.21.15.99:46332) 2023-10-08T00:16:12.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.285+0000 7f6091083700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f608c0b3370 0x7f608c0b11f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:43740/0 (socket says 172.21.15.99:43740) 2023-10-08T00:16:12.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.285+0000 7f6091884700 1 -- 172.21.15.99:0/2896837867 learned_addr learned my addr 172.21.15.99:0/2896837867 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:12.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.285+0000 7f6091884700 1 -- 172.21.15.99:0/2896837867 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f608c068cb0 msgr2=0x7f608c0690e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:12.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.285+0000 7f6092085700 1 --2- 172.21.15.99:0/2896837867 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f608c068cb0 0x7f608c0690e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:12.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.285+0000 7f6091884700 1 --2- 172.21.15.99:0/2896837867 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f608c068cb0 0x7f608c0690e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:12.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.285+0000 7f6090882700 1 -- 172.21.15.99:0/2896837867 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2275235052 0 0) 0x7f608c0b0d60 con 0x7f608c0b3370 2023-10-08T00:16:12.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.285+0000 7f6091884700 1 -- 172.21.15.99:0/2896837867 >> v1:172.21.15.99:6789/0 conn(0x7f608c0b3370 legacy=0x7f608c0b11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:12.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.285+0000 7f6091884700 1 -- 172.21.15.99:0/2896837867 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f608c0d1890 con 0x7f608c159e50 2023-10-08T00:16:12.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.285+0000 7f6091884700 1 --2- 172.21.15.99:0/2896837867 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f608c159e50 0x7f608c0b2e30 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f60880099b0 tx=0x7f6088009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=538a7d318419e119 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:12.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.285+0000 7f6090882700 1 -- 172.21.15.99:0/2896837867 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608800b860 con 0x7f608c159e50 2023-10-08T00:16:12.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.286+0000 7f6090882700 1 -- 172.21.15.99:0/2896837867 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f608800b9c0 con 0x7f608c159e50 2023-10-08T00:16:12.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.286+0000 7f6090882700 1 -- 172.21.15.99:0/2896837867 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608800bd60 con 0x7f608c159e50 2023-10-08T00:16:12.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.286+0000 7f6093ae8700 1 -- 172.21.15.99:0/2896837867 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f608c159e50 msgr2=0x7f608c0b2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:12.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.286+0000 7f6093ae8700 1 --2- 172.21.15.99:0/2896837867 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f608c159e50 0x7f608c0b2e30 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f60880099b0 tx=0x7f6088009250 comp rx=0 tx=0).stop 2023-10-08T00:16:12.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.286+0000 7f6093ae8700 1 -- 172.21.15.99:0/2896837867 shutdown_connections 2023-10-08T00:16:12.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.286+0000 7f6093ae8700 1 --2- 172.21.15.99:0/2896837867 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f608c159e50 0x7f608c0b2e30 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:12.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.286+0000 7f6093ae8700 1 --2- 172.21.15.99:0/2896837867 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f608c068cb0 0x7f608c0690e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:12.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.286+0000 7f6093ae8700 1 -- 172.21.15.99:0/2896837867 >> 172.21.15.99:0/2896837867 conn(0x7f608c068130 msgr2=0x7f608c0d0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:12.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.286+0000 7f6093ae8700 1 -- 172.21.15.99:0/2896837867 shutdown_connections 2023-10-08T00:16:12.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.286+0000 7f6093ae8700 1 -- 172.21.15.99:0/2896837867 wait complete. 2023-10-08T00:16:12.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.287+0000 7f6093ae8700 1 Processor -- start 2023-10-08T00:16:12.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.287+0000 7f6093ae8700 1 -- start start 2023-10-08T00:16:12.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.288+0000 7f6093ae8700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f608c068cb0 0x7f608c0a8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:12.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.288+0000 7f6093ae8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f608c0b3370 0x7f608c16a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:12.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.288+0000 7f6093ae8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f608c159e50 0x7f608c15d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:12.294 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.288+0000 7f6093ae8700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608c158ce0 con 0x7f608c159e50 2023-10-08T00:16:12.294 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.288+0000 7f6093ae8700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608c15e9f0 con 0x7f608c068cb0 2023-10-08T00:16:12.294 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.288+0000 7f6093ae8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f608c15dde0 con 0x7f608c0b3370 2023-10-08T00:16:12.294 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.288+0000 7f6092085700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f608c159e50 0x7f608c15d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:12.294 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.288+0000 7f6092085700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f608c159e50 0x7f608c15d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46338/0 (socket says 172.21.15.99:46338) 2023-10-08T00:16:12.295 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.288+0000 7f6092085700 1 -- 172.21.15.99:0/3524111561 learned_addr learned my addr 172.21.15.99:0/3524111561 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:12.295 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.288+0000 7f6091884700 1 --2- 172.21.15.99:0/3524111561 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f608c068cb0 0x7f608c0a8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:12.295 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.288+0000 7f6091083700 1 --2- 172.21.15.99:0/3524111561 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f608c0b3370 0x7f608c16a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:12.295 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.288+0000 7f6092085700 1 -- 172.21.15.99:0/3524111561 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f608c0b3370 msgr2=0x7f608c16a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:12.295 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.288+0000 7f6092085700 1 --2- 172.21.15.99:0/3524111561 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f608c0b3370 0x7f608c16a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:12.295 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.288+0000 7f6092085700 1 -- 172.21.15.99:0/3524111561 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f608c068cb0 msgr2=0x7f608c0a8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:12.295 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.288+0000 7f6092085700 1 --2- 172.21.15.99:0/3524111561 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f608c068cb0 0x7f608c0a8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:12.295 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.288+0000 7f6092085700 1 -- 172.21.15.99:0/3524111561 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60880094f0 con 0x7f608c159e50 2023-10-08T00:16:12.295 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.288+0000 7f6091884700 1 --2- 172.21.15.99:0/3524111561 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f608c068cb0 0x7f608c0a8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:16:12.295 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.288+0000 7f6092085700 1 --2- 172.21.15.99:0/3524111561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f608c159e50 0x7f608c15d270 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f608400ceb0 tx=0x7f6084007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:12.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.289+0000 7f6081ffb700 1 -- 172.21.15.99:0/3524111561 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608400ae40 con 0x7f608c159e50 2023-10-08T00:16:12.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.289+0000 7f6081ffb700 1 -- 172.21.15.99:0/3524111561 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6084016430 con 0x7f608c159e50 2023-10-08T00:16:12.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.289+0000 7f6093ae8700 1 -- 172.21.15.99:0/3524111561 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f608c15e0c0 con 0x7f608c159e50 2023-10-08T00:16:12.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.289+0000 7f6081ffb700 1 -- 172.21.15.99:0/3524111561 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60840167d0 con 0x7f608c159e50 2023-10-08T00:16:12.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.289+0000 7f6093ae8700 1 -- 172.21.15.99:0/3524111561 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f608c15e680 con 0x7f608c159e50 2023-10-08T00:16:12.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.290+0000 7f6093ae8700 1 -- 172.21.15.99:0/3524111561 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f608c15eba0 con 0x7f608c159e50 2023-10-08T00:16:12.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.290+0000 7f6081ffb700 1 -- 172.21.15.99:0/3524111561 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6084003020 con 0x7f608c159e50 2023-10-08T00:16:12.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.290+0000 7f6081ffb700 1 --2- 172.21.15.99:0/3524111561 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6070066f30 0x7f60700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:12.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.290+0000 7f6081ffb700 1 -- 172.21.15.99:0/3524111561 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f6084012070 con 0x7f608c159e50 2023-10-08T00:16:12.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.291+0000 7f6091884700 1 --2- 172.21.15.99:0/3524111561 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6070066f30 0x7f60700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:12.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.291+0000 7f6091884700 1 --2- 172.21.15.99:0/3524111561 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6070066f30 0x7f60700693e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f60880099b0 tx=0x7f6088017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:12.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.294+0000 7f6081ffb700 1 -- 172.21.15.99:0/3524111561 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f608c15eba0 con 0x7f608c159e50 2023-10-08T00:16:12.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:12.476+0000 7f6093ae8700 1 -- 172.21.15.99:0/3524111561 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f608c15e840 con 0x7f608c159e50 2023-10-08T00:16:13.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.108+0000 7f6081ffb700 1 -- 172.21.15.99:0/3524111561 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v114) v1 ==== 181+0+0 (secure 0 0 0) 0x7f6084055ec0 con 0x7f608c159e50 2023-10-08T00:16:13.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.111+0000 7f6093ae8700 1 -- 172.21.15.99:0/3524111561 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6070066f30 msgr2=0x7f60700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:13.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.111+0000 7f6093ae8700 1 --2- 172.21.15.99:0/3524111561 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6070066f30 0x7f60700693e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f60880099b0 tx=0x7f6088017000 comp rx=0 tx=0).stop 2023-10-08T00:16:13.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.111+0000 7f6093ae8700 1 -- 172.21.15.99:0/3524111561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f608c159e50 msgr2=0x7f608c15d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:13.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.111+0000 7f6093ae8700 1 --2- 172.21.15.99:0/3524111561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f608c159e50 0x7f608c15d270 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f608400ceb0 tx=0x7f6084007870 comp rx=0 tx=0).stop 2023-10-08T00:16:13.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.111+0000 7f6093ae8700 1 -- 172.21.15.99:0/3524111561 shutdown_connections 2023-10-08T00:16:13.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.111+0000 7f6093ae8700 1 --2- 172.21.15.99:0/3524111561 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6070066f30 0x7f60700693e0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:13.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.111+0000 7f6093ae8700 1 --2- 172.21.15.99:0/3524111561 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f608c068cb0 0x7f608c0a8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:13.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.111+0000 7f6093ae8700 1 --2- 172.21.15.99:0/3524111561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f608c159e50 0x7f608c15d270 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:13.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.111+0000 7f6093ae8700 1 --2- 172.21.15.99:0/3524111561 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f608c0b3370 0x7f608c16a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:13.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.111+0000 7f6093ae8700 1 -- 172.21.15.99:0/3524111561 >> 172.21.15.99:0/3524111561 conn(0x7f608c068130 msgr2=0x7f608c159040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:13.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.111+0000 7f6093ae8700 1 -- 172.21.15.99:0/3524111561 shutdown_connections 2023-10-08T00:16:13.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.111+0000 7f6093ae8700 1 -- 172.21.15.99:0/3524111561 wait complete. 2023-10-08T00:16:13.113 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' removed 2023-10-08T00:16:13.125 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T00:16:13.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.244+0000 7f680c0cc700 1 Processor -- start 2023-10-08T00:16:13.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.244+0000 7f680c0cc700 1 -- start start 2023-10-08T00:16:13.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.245+0000 7f680c0cc700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6804070e10 0x7f6804164ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:13.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.245+0000 7f680c0cc700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6804067dc0 con 0x7f680406f2c0 2023-10-08T00:16:13.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.245+0000 7f680c0cc700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f680406fa70 con 0x7f6804159e50 2023-10-08T00:16:13.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.245+0000 7f680c0cc700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68040cb9e0 con 0x7f6804070e10 2023-10-08T00:16:13.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.245+0000 7f680a669700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6804070e10 0x7f6804164ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:13.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.246+0000 7f6809667700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f680406f2c0 0x7f6804070700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:43756/0 (socket says 172.21.15.99:43756) 2023-10-08T00:16:13.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.246+0000 7f680a669700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6804070e10 0x7f6804164ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46346/0 (socket says 172.21.15.99:46346) 2023-10-08T00:16:13.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.246+0000 7f6809667700 1 -- 172.21.15.99:0/2977367915 learned_addr learned my addr 172.21.15.99:0/2977367915 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:13.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.246+0000 7f680a669700 1 -- 172.21.15.99:0/2977367915 >> v1:172.21.15.191:6789/0 conn(0x7f6804159e50 legacy=0x7f680406ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:13.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.246+0000 7f680a669700 1 -- 172.21.15.99:0/2977367915 >> v1:172.21.15.99:6789/0 conn(0x7f680406f2c0 legacy=0x7f6804070700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:13.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.246+0000 7f680a669700 1 -- 172.21.15.99:0/2977367915 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f680406f790 con 0x7f6804070e10 2023-10-08T00:16:13.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.246+0000 7f6808e66700 1 -- 172.21.15.99:0/2977367915 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2507786536 0 0) 0x7f6804067dc0 con 0x7f680406f2c0 2023-10-08T00:16:13.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.246+0000 7f680a669700 1 --2- 172.21.15.99:0/2977367915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6804070e10 0x7f6804164ea0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f67fc009e60 tx=0x7f67fc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37e59f47a8e29f73 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:13.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.246+0000 7f6808e66700 1 -- 172.21.15.99:0/2977367915 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc00e070 con 0x7f6804070e10 2023-10-08T00:16:13.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.246+0000 7f6808e66700 1 -- 172.21.15.99:0/2977367915 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67fc00bc30 con 0x7f6804070e10 2023-10-08T00:16:13.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.246+0000 7f6808e66700 1 -- 172.21.15.99:0/2977367915 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc012430 con 0x7f6804070e10 2023-10-08T00:16:13.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.247+0000 7f680c0cc700 1 -- 172.21.15.99:0/2977367915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6804070e10 msgr2=0x7f6804164ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:13.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.247+0000 7f680c0cc700 1 --2- 172.21.15.99:0/2977367915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6804070e10 0x7f6804164ea0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f67fc009e60 tx=0x7f67fc009780 comp rx=0 tx=0).stop 2023-10-08T00:16:13.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.247+0000 7f680c0cc700 1 -- 172.21.15.99:0/2977367915 shutdown_connections 2023-10-08T00:16:13.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.247+0000 7f680c0cc700 1 --2- 172.21.15.99:0/2977367915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6804070e10 0x7f6804164ea0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:13.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.247+0000 7f680c0cc700 1 -- 172.21.15.99:0/2977367915 >> 172.21.15.99:0/2977367915 conn(0x7f68040cdae0 msgr2=0x7f68040cdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:13.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.247+0000 7f680c0cc700 1 -- 172.21.15.99:0/2977367915 shutdown_connections 2023-10-08T00:16:13.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.247+0000 7f680c0cc700 1 -- 172.21.15.99:0/2977367915 wait complete. 2023-10-08T00:16:13.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.248+0000 7f680c0cc700 1 Processor -- start 2023-10-08T00:16:13.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.248+0000 7f680c0cc700 1 -- start start 2023-10-08T00:16:13.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.249+0000 7f680c0cc700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f680406f2c0 0x7f6804153de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:13.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.249+0000 7f680c0cc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6804070e10 0x7f68040b28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:13.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.249+0000 7f680c0cc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6804159e50 0x7f68040abe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:13.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.249+0000 7f680c0cc700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68040adad0 con 0x7f6804070e10 2023-10-08T00:16:13.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.249+0000 7f680c0cc700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6804154320 con 0x7f680406f2c0 2023-10-08T00:16:13.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.249+0000 7f680c0cc700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68040b36b0 con 0x7f6804159e50 2023-10-08T00:16:13.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.249+0000 7f6809667700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6804070e10 0x7f68040b28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:13.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.249+0000 7f6809667700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6804070e10 0x7f68040b28d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46356/0 (socket says 172.21.15.99:46356) 2023-10-08T00:16:13.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.249+0000 7f6809667700 1 -- 172.21.15.99:0/371723776 learned_addr learned my addr 172.21.15.99:0/371723776 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:13.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.249+0000 7f6809e68700 1 --2- 172.21.15.99:0/371723776 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f680406f2c0 0x7f6804153de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:13.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.249+0000 7f680a669700 1 --2- 172.21.15.99:0/371723776 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6804159e50 0x7f68040abe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:13.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.249+0000 7f6809667700 1 -- 172.21.15.99:0/371723776 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6804159e50 msgr2=0x7f68040abe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:13.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.249+0000 7f6809667700 1 --2- 172.21.15.99:0/371723776 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6804159e50 0x7f68040abe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:13.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.249+0000 7f6809667700 1 -- 172.21.15.99:0/371723776 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f680406f2c0 msgr2=0x7f6804153de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:13.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.249+0000 7f6809667700 1 --2- 172.21.15.99:0/371723776 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f680406f2c0 0x7f6804153de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:13.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.249+0000 7f6809667700 1 -- 172.21.15.99:0/371723776 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67fc009a20 con 0x7f6804070e10 2023-10-08T00:16:13.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.249+0000 7f6809667700 1 --2- 172.21.15.99:0/371723776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6804070e10 0x7f68040b28d0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f67f400c7e0 tx=0x7f67f4007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:13.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.249+0000 7f67f9ffb700 1 -- 172.21.15.99:0/371723776 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f4005b30 con 0x7f6804070e10 2023-10-08T00:16:13.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.249+0000 7f67f9ffb700 1 -- 172.21.15.99:0/371723776 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67f4005c90 con 0x7f6804070e10 2023-10-08T00:16:13.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.249+0000 7f680c0cc700 1 -- 172.21.15.99:0/371723776 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68040b33d0 con 0x7f6804070e10 2023-10-08T00:16:13.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.249+0000 7f67f9ffb700 1 -- 172.21.15.99:0/371723776 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f4007300 con 0x7f6804070e10 2023-10-08T00:16:13.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.250+0000 7f680c0cc700 1 -- 172.21.15.99:0/371723776 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68040ad760 con 0x7f6804070e10 2023-10-08T00:16:13.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.250+0000 7f680c0cc700 1 -- 172.21.15.99:0/371723776 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68040adc80 con 0x7f6804070e10 2023-10-08T00:16:13.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.251+0000 7f67f9ffb700 1 -- 172.21.15.99:0/371723776 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f67f401b3b0 con 0x7f6804070e10 2023-10-08T00:16:13.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.251+0000 7f67f9ffb700 1 --2- 172.21.15.99:0/371723776 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f67e8066f30 0x7f67e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:13.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.252+0000 7f67f9ffb700 1 -- 172.21.15.99:0/371723776 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5353+0+0 (secure 0 0 0) 0x7f67f40838f0 con 0x7f6804070e10 2023-10-08T00:16:13.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.254+0000 7f6809e68700 1 --2- 172.21.15.99:0/371723776 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f67e8066f30 0x7f67e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:13.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.254+0000 7f6809e68700 1 --2- 172.21.15.99:0/371723776 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f67e8066f30 0x7f67e80693e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f68000039e0 tx=0x7f6800007ec0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:13.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.257+0000 7f67f9ffb700 1 -- 172.21.15.99:0/371723776 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f68040adc80 con 0x7f6804070e10 2023-10-08T00:16:13.443 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:13.443+0000 7f680c0cc700 1 -- 172.21.15.99:0/371723776 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f680404c3c0 con 0x7f6804070e10 2023-10-08T00:16:14.114 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.112+0000 7f67f9ffb700 1 -- 172.21.15.99:0/371723776 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v115) v1 ==== 169+0+0 (secure 0 0 0) 0x7f67f4017020 con 0x7f6804070e10 2023-10-08T00:16:14.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.115+0000 7f680c0cc700 1 -- 172.21.15.99:0/371723776 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f67e8066f30 msgr2=0x7f67e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:14.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.115+0000 7f680c0cc700 1 --2- 172.21.15.99:0/371723776 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f67e8066f30 0x7f67e80693e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f68000039e0 tx=0x7f6800007ec0 comp rx=0 tx=0).stop 2023-10-08T00:16:14.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.115+0000 7f680c0cc700 1 -- 172.21.15.99:0/371723776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6804070e10 msgr2=0x7f68040b28d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:14.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.115+0000 7f680c0cc700 1 --2- 172.21.15.99:0/371723776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6804070e10 0x7f68040b28d0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f67f400c7e0 tx=0x7f67f4007df0 comp rx=0 tx=0).stop 2023-10-08T00:16:14.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.115+0000 7f680c0cc700 1 -- 172.21.15.99:0/371723776 shutdown_connections 2023-10-08T00:16:14.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.115+0000 7f680c0cc700 1 --2- 172.21.15.99:0/371723776 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f67e8066f30 0x7f67e80693e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:14.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.115+0000 7f680c0cc700 1 --2- 172.21.15.99:0/371723776 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f680406f2c0 0x7f6804153de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:14.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.115+0000 7f680c0cc700 1 --2- 172.21.15.99:0/371723776 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6804159e50 0x7f68040abe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:14.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.115+0000 7f680c0cc700 1 --2- 172.21.15.99:0/371723776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6804070e10 0x7f68040b28d0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:14.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.115+0000 7f680c0cc700 1 -- 172.21.15.99:0/371723776 >> 172.21.15.99:0/371723776 conn(0x7f68040cdae0 msgr2=0x7f6804155b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:14.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.115+0000 7f680c0cc700 1 -- 172.21.15.99:0/371723776 shutdown_connections 2023-10-08T00:16:14.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.115+0000 7f680c0cc700 1 -- 172.21.15.99:0/371723776 wait complete. 2023-10-08T00:16:14.118 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' removed 2023-10-08T00:16:14.129 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-08T00:16:14.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.243+0000 7f2d18131700 1 Processor -- start 2023-10-08T00:16:14.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.243+0000 7f2d18131700 1 -- start start 2023-10-08T00:16:14.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.244+0000 7f2d18131700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2d10069f80 0x7f2d100b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:14.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.244+0000 7f2d18131700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d100afb50 con 0x7f2d10157cc0 2023-10-08T00:16:14.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.244+0000 7f2d18131700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d100afe30 con 0x7f2d100cb0f0 2023-10-08T00:16:14.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.244+0000 7f2d18131700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d100b0110 con 0x7f2d10069f80 2023-10-08T00:16:14.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.244+0000 7f2d15ecd700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f2d10157cc0 0x7f2d100ca9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56250/0 (socket says 172.21.15.99:56250) 2023-10-08T00:16:14.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.244+0000 7f2d15ecd700 1 -- 172.21.15.99:0/3875336015 learned_addr learned my addr 172.21.15.99:0/3875336015 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:14.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.245+0000 7f2d166ce700 1 --2- 172.21.15.99:0/3875336015 >> v2:172.21.15.191:3300/0 conn(0x7f2d10069f80 0x7f2d100b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:14.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.245+0000 7f2d14ecb700 1 -- 172.21.15.99:0/3875336015 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2055681658 0 0) 0x7f2d100afb50 con 0x7f2d10157cc0 2023-10-08T00:16:14.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.245+0000 7f2d14ecb700 1 -- 172.21.15.99:0/3875336015 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cfc003570 con 0x7f2d10157cc0 2023-10-08T00:16:14.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.245+0000 7f2d14ecb700 1 -- 172.21.15.99:0/3875336015 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2239967445 0 0) 0x7f2cfc003570 con 0x7f2d10157cc0 2023-10-08T00:16:14.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.245+0000 7f2d14ecb700 1 -- 172.21.15.99:0/3875336015 >> v2:172.21.15.191:3300/0 conn(0x7f2d10069f80 msgr2=0x7f2d100b0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:14.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.245+0000 7f2d14ecb700 1 --2- 172.21.15.99:0/3875336015 >> v2:172.21.15.191:3300/0 conn(0x7f2d10069f80 0x7f2d100b0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:14.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.245+0000 7f2d14ecb700 1 -- 172.21.15.99:0/3875336015 >> v1:172.21.15.191:6789/0 conn(0x7f2d100cb0f0 legacy=0x7f2d10069870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:14.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.245+0000 7f2d14ecb700 1 -- 172.21.15.99:0/3875336015 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d100b03f0 con 0x7f2d10157cc0 2023-10-08T00:16:14.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.245+0000 7f2d14ecb700 1 -- 172.21.15.99:0/3875336015 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f2d0c001470 con 0x7f2d10157cc0 2023-10-08T00:16:14.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.245+0000 7f2d14ecb700 1 -- 172.21.15.99:0/3875336015 >> v1:172.21.15.99:6789/0 conn(0x7f2d10157cc0 legacy=0x7f2d100ca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:14.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.245+0000 7f2d14ecb700 1 --2- 172.21.15.99:0/3875336015 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2cfc003a60 0x7f2cfc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:14.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.245+0000 7f2d14ecb700 1 --2- 172.21.15.99:0/3875336015 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cfc004810 0x7f2cfc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:14.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.245+0000 7f2d14ecb700 1 --2- 172.21.15.99:0/3875336015 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2cfc0051c0 0x7f2cfc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:14.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.245+0000 7f2d14ecb700 1 -- 172.21.15.99:0/3875336015 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d100b0110 con 0x7f2cfc004810 2023-10-08T00:16:14.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.245+0000 7f2d14ecb700 1 -- 172.21.15.99:0/3875336015 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cfc00c0d0 con 0x7f2cfc0051c0 2023-10-08T00:16:14.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.245+0000 7f2d14ecb700 1 -- 172.21.15.99:0/3875336015 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cfc00c290 con 0x7f2cfc003a60 2023-10-08T00:16:14.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.245+0000 7f2d156cc700 1 --2- 172.21.15.99:0/3875336015 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cfc004810 0x7f2cfc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:14.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.245+0000 7f2d166ce700 1 --2- 172.21.15.99:0/3875336015 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2cfc0051c0 0x7f2cfc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:14.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.245+0000 7f2d15ecd700 1 --2- 172.21.15.99:0/3875336015 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2cfc003a60 0x7f2cfc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:14.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.245+0000 7f2d156cc700 1 -- 172.21.15.99:0/3875336015 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2cfc003a60 msgr2=0x7f2cfc003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:14.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.245+0000 7f2d156cc700 1 --2- 172.21.15.99:0/3875336015 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2cfc003a60 0x7f2cfc003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:14.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.245+0000 7f2d156cc700 1 -- 172.21.15.99:0/3875336015 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2cfc0051c0 msgr2=0x7f2cfc00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:14.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.245+0000 7f2d156cc700 1 --2- 172.21.15.99:0/3875336015 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2cfc0051c0 0x7f2cfc00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:14.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.245+0000 7f2d156cc700 1 -- 172.21.15.99:0/3875336015 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cfc003570 con 0x7f2cfc004810 2023-10-08T00:16:14.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.245+0000 7f2d166ce700 1 -- 172.21.15.99:0/3875336015 reap_dead start 2023-10-08T00:16:14.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.245+0000 7f2d156cc700 1 --2- 172.21.15.99:0/3875336015 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cfc004810 0x7f2cfc004c80 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f2d00001880 tx=0x7f2d00007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13dc0b2c57535663 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:14.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.246+0000 7f2d14ecb700 1 -- 172.21.15.99:0/3875336015 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d00012420 con 0x7f2cfc004810 2023-10-08T00:16:14.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.246+0000 7f2d14ecb700 1 -- 172.21.15.99:0/3875336015 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d000125a0 con 0x7f2cfc004810 2023-10-08T00:16:14.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.246+0000 7f2d14ecb700 1 -- 172.21.15.99:0/3875336015 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d00012960 con 0x7f2cfc004810 2023-10-08T00:16:14.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.246+0000 7f2d18131700 1 -- 172.21.15.99:0/3875336015 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cfc004810 msgr2=0x7f2cfc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:14.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.246+0000 7f2d18131700 1 --2- 172.21.15.99:0/3875336015 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cfc004810 0x7f2cfc004c80 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f2d00001880 tx=0x7f2d00007d30 comp rx=0 tx=0).stop 2023-10-08T00:16:14.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.246+0000 7f2d18131700 1 -- 172.21.15.99:0/3875336015 shutdown_connections 2023-10-08T00:16:14.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.246+0000 7f2d18131700 1 --2- 172.21.15.99:0/3875336015 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cfc004810 0x7f2cfc004c80 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:14.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.246+0000 7f2d18131700 1 -- 172.21.15.99:0/3875336015 >> 172.21.15.99:0/3875336015 conn(0x7f2d100cb5c0 msgr2=0x7f2d100cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:14.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.246+0000 7f2d18131700 1 -- 172.21.15.99:0/3875336015 shutdown_connections 2023-10-08T00:16:14.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.246+0000 7f2d18131700 1 -- 172.21.15.99:0/3875336015 wait complete. 2023-10-08T00:16:14.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.247+0000 7f2d18131700 1 Processor -- start 2023-10-08T00:16:14.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.247+0000 7f2d18131700 1 -- start start 2023-10-08T00:16:14.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.248+0000 7f2d18131700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2cfc004810 0x7f2d10068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:14.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.248+0000 7f2d18131700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2d1015a700 0x7f2d1016ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:14.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.248+0000 7f2d18131700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d1015bc80 0x7f2d1015ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:14.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.248+0000 7f2d18131700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d10069140 con 0x7f2d1015bc80 2023-10-08T00:16:14.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.248+0000 7f2d18131700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1015b7f0 con 0x7f2cfc004810 2023-10-08T00:16:14.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.248+0000 7f2d18131700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1016b3a0 con 0x7f2d1015a700 2023-10-08T00:16:14.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.248+0000 7f2d166ce700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d1015bc80 0x7f2d1015ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:14.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.248+0000 7f2d166ce700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d1015bc80 0x7f2d1015ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35770/0 (socket says 172.21.15.99:35770) 2023-10-08T00:16:14.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.248+0000 7f2d166ce700 1 -- 172.21.15.99:0/3066096433 learned_addr learned my addr 172.21.15.99:0/3066096433 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:14.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.248+0000 7f2d15ecd700 1 --2- 172.21.15.99:0/3066096433 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2cfc004810 0x7f2d10068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:14.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.248+0000 7f2d156cc700 1 --2- 172.21.15.99:0/3066096433 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2d1015a700 0x7f2d1016ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:14.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.248+0000 7f2d166ce700 1 -- 172.21.15.99:0/3066096433 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2d1015a700 msgr2=0x7f2d1016ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:14.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.248+0000 7f2d166ce700 1 --2- 172.21.15.99:0/3066096433 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2d1015a700 0x7f2d1016ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:14.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.248+0000 7f2d166ce700 1 -- 172.21.15.99:0/3066096433 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2cfc004810 msgr2=0x7f2d10068c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:14.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.248+0000 7f2d166ce700 1 --2- 172.21.15.99:0/3066096433 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2cfc004810 0x7f2d10068c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:14.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.248+0000 7f2d166ce700 1 -- 172.21.15.99:0/3066096433 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d100c2850 con 0x7f2d1015bc80 2023-10-08T00:16:14.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.249+0000 7f2d166ce700 1 --2- 172.21.15.99:0/3066096433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d1015bc80 0x7f2d1015ca80 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f2d1006a870 tx=0x7f2d0800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:14.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.249+0000 7f2d05ffb700 1 -- 172.21.15.99:0/3066096433 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0800e040 con 0x7f2d1015bc80 2023-10-08T00:16:14.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.249+0000 7f2d05ffb700 1 -- 172.21.15.99:0/3066096433 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d0800f040 con 0x7f2d1015bc80 2023-10-08T00:16:14.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.249+0000 7f2d05ffb700 1 -- 172.21.15.99:0/3066096433 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0800fe20 con 0x7f2d1015bc80 2023-10-08T00:16:14.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.249+0000 7f2d18131700 1 -- 172.21.15.99:0/3066096433 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d1015abe0 con 0x7f2d1015bc80 2023-10-08T00:16:14.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.249+0000 7f2d18131700 1 -- 172.21.15.99:0/3066096433 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d1015b1a0 con 0x7f2d1015bc80 2023-10-08T00:16:14.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.250+0000 7f2d18131700 1 -- 172.21.15.99:0/3066096433 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d1015b960 con 0x7f2d1015bc80 2023-10-08T00:16:14.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.250+0000 7f2d05ffb700 1 -- 172.21.15.99:0/3066096433 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2d08015640 con 0x7f2d1015bc80 2023-10-08T00:16:14.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.251+0000 7f2d05ffb700 1 --2- 172.21.15.99:0/3066096433 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2cf4066f30 0x7f2cf40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:14.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.251+0000 7f2d05ffb700 1 -- 172.21.15.99:0/3066096433 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f2d08084ab0 con 0x7f2d1015bc80 2023-10-08T00:16:14.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.251+0000 7f2d15ecd700 1 --2- 172.21.15.99:0/3066096433 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2cf4066f30 0x7f2cf40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:14.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.251+0000 7f2d15ecd700 1 --2- 172.21.15.99:0/3066096433 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2cf4066f30 0x7f2cf40693e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c001ce0 tx=0x7f2d0c001e50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:14.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.255+0000 7f2d05ffb700 1 -- 172.21.15.99:0/3066096433 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2d080520e0 con 0x7f2d1015bc80 2023-10-08T00:16:14.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.439+0000 7f2d18131700 1 -- 172.21.15.99:0/3066096433 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f2d1004c3c0 con 0x7f2d1015bc80 2023-10-08T00:16:14.444 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.443+0000 7f2d05ffb700 1 -- 172.21.15.99:0/3066096433 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2d08055700 con 0x7f2d1015bc80 2023-10-08T00:16:14.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.445+0000 7f2d18131700 1 -- 172.21.15.99:0/3066096433 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2cf4066f30 msgr2=0x7f2cf40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:14.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.445+0000 7f2d18131700 1 --2- 172.21.15.99:0/3066096433 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2cf4066f30 0x7f2cf40693e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c001ce0 tx=0x7f2d0c001e50 comp rx=0 tx=0).stop 2023-10-08T00:16:14.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.445+0000 7f2d18131700 1 -- 172.21.15.99:0/3066096433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d1015bc80 msgr2=0x7f2d1015ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:14.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.445+0000 7f2d18131700 1 --2- 172.21.15.99:0/3066096433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d1015bc80 0x7f2d1015ca80 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f2d1006a870 tx=0x7f2d0800b040 comp rx=0 tx=0).stop 2023-10-08T00:16:14.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.445+0000 7f2d18131700 1 -- 172.21.15.99:0/3066096433 shutdown_connections 2023-10-08T00:16:14.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.445+0000 7f2d18131700 1 --2- 172.21.15.99:0/3066096433 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2cf4066f30 0x7f2cf40693e0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:14.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.445+0000 7f2d18131700 1 --2- 172.21.15.99:0/3066096433 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2cfc004810 0x7f2d10068c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:14.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.445+0000 7f2d18131700 1 --2- 172.21.15.99:0/3066096433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d1015bc80 0x7f2d1015ca80 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:14.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.445+0000 7f2d18131700 1 --2- 172.21.15.99:0/3066096433 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2d1015a700 0x7f2d1016ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:14.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.445+0000 7f2d18131700 1 -- 172.21.15.99:0/3066096433 >> 172.21.15.99:0/3066096433 conn(0x7f2d100cb5c0 msgr2=0x7f2d10156e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:14.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.445+0000 7f2d18131700 1 -- 172.21.15.99:0/3066096433 shutdown_connections 2023-10-08T00:16:14.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.445+0000 7f2d18131700 1 -- 172.21.15.99:0/3066096433 wait complete. 2023-10-08T00:16:14.459 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_same_file' 2023-10-08T00:16:14.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.571+0000 7f2ae02a2700 1 Processor -- start 2023-10-08T00:16:14.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.571+0000 7f2ae02a2700 1 -- start start 2023-10-08T00:16:14.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.572+0000 7f2ae02a2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ad8157870 0x7f2ad815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:14.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.572+0000 7f2ae02a2700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2ad815a620 0x7f2ad815d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:14.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.572+0000 7f2ae02a2700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2ad815d580 0x7f2ad8158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:14.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.572+0000 7f2ae02a2700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ad815cbb0 con 0x7f2ad815a620 2023-10-08T00:16:14.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.572+0000 7f2ae02a2700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ad815bfa0 con 0x7f2ad815d580 2023-10-08T00:16:14.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.572+0000 7f2ae02a2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad815c280 con 0x7f2ad8157870 2023-10-08T00:16:14.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.572+0000 7f2add83d700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2ad815a620 0x7f2ad815d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:14.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.573+0000 7f2add83d700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2ad815a620 0x7f2ad815d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35782/0 (socket says 172.21.15.99:35782) 2023-10-08T00:16:14.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.573+0000 7f2add83d700 1 -- 172.21.15.99:0/1068645441 learned_addr learned my addr 172.21.15.99:0/1068645441 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:14.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.573+0000 7f2ade03e700 1 --2- 172.21.15.99:0/1068645441 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ad8157870 0x7f2ad815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:14.577 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.573+0000 7f2ade83f700 1 --2- 172.21.15.99:0/1068645441 >> v2:172.21.15.191:3300/0 conn(0x7f2ad815d580 0x7f2ad8158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:14.577 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.573+0000 7f2add83d700 1 -- 172.21.15.99:0/1068645441 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ad8157870 msgr2=0x7f2ad815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:14.577 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.573+0000 7f2add83d700 1 --2- 172.21.15.99:0/1068645441 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ad8157870 0x7f2ad815b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:14.577 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.573+0000 7f2add83d700 1 -- 172.21.15.99:0/1068645441 >> v2:172.21.15.191:3300/0 conn(0x7f2ad815d580 msgr2=0x7f2ad8158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:14.577 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.573+0000 7f2add83d700 1 --2- 172.21.15.99:0/1068645441 >> v2:172.21.15.191:3300/0 conn(0x7f2ad815d580 0x7f2ad8158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:14.578 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.573+0000 7f2add83d700 1 -- 172.21.15.99:0/1068645441 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad815c560 con 0x7f2ad815a620 2023-10-08T00:16:14.578 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.573+0000 7f2add83d700 1 --2- 172.21.15.99:0/1068645441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ad815a620 0x7f2ad815d040 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f2ad80b03b0 tx=0x7f2ac8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=438d5090d7148728 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:14.578 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.573+0000 7f2add03c700 1 -- 172.21.15.99:0/1068645441 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac800b860 con 0x7f2ad815a620 2023-10-08T00:16:14.578 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.573+0000 7f2add03c700 1 -- 172.21.15.99:0/1068645441 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ac800b9c0 con 0x7f2ad815a620 2023-10-08T00:16:14.578 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.573+0000 7f2add03c700 1 -- 172.21.15.99:0/1068645441 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac800bd60 con 0x7f2ad815a620 2023-10-08T00:16:14.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.574+0000 7f2ae02a2700 1 -- 172.21.15.99:0/1068645441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ad815a620 msgr2=0x7f2ad815d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:14.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.574+0000 7f2ae02a2700 1 --2- 172.21.15.99:0/1068645441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ad815a620 0x7f2ad815d040 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f2ad80b03b0 tx=0x7f2ac8009250 comp rx=0 tx=0).stop 2023-10-08T00:16:14.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.574+0000 7f2ae02a2700 1 -- 172.21.15.99:0/1068645441 shutdown_connections 2023-10-08T00:16:14.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.574+0000 7f2ae02a2700 1 --2- 172.21.15.99:0/1068645441 >> v2:172.21.15.191:3300/0 conn(0x7f2ad815d580 0x7f2ad8158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:14.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.574+0000 7f2ae02a2700 1 --2- 172.21.15.99:0/1068645441 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ad8157870 0x7f2ad815b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:14.580 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.574+0000 7f2ae02a2700 1 --2- 172.21.15.99:0/1068645441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ad815a620 0x7f2ad815d040 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:14.580 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.574+0000 7f2ae02a2700 1 -- 172.21.15.99:0/1068645441 >> 172.21.15.99:0/1068645441 conn(0x7f2ad8068ce0 msgr2=0x7f2ad80690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:14.580 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.574+0000 7f2ae02a2700 1 -- 172.21.15.99:0/1068645441 shutdown_connections 2023-10-08T00:16:14.580 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.574+0000 7f2ae02a2700 1 -- 172.21.15.99:0/1068645441 wait complete. 2023-10-08T00:16:14.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.575+0000 7f2ae02a2700 1 Processor -- start 2023-10-08T00:16:14.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.575+0000 7f2ae02a2700 1 -- start start 2023-10-08T00:16:14.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.576+0000 7f2ae02a2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ad8157870 0x7f2ad80cf950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:14.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.576+0000 7f2ae02a2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ad815a620 0x7f2ad80b0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:14.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.576+0000 7f2ae02a2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ad815d580 0x7f2ad80b1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:14.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.576+0000 7f2ae02a2700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad80d0ba0 con 0x7f2ad8157870 2023-10-08T00:16:14.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.576+0000 7f2ae02a2700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad80d0020 con 0x7f2ad815a620 2023-10-08T00:16:14.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.576+0000 7f2ae02a2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad80d0300 con 0x7f2ad815d580 2023-10-08T00:16:14.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.576+0000 7f2ade03e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ad8157870 0x7f2ad80cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:14.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.576+0000 7f2ade03e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ad8157870 0x7f2ad80cf950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35786/0 (socket says 172.21.15.99:35786) 2023-10-08T00:16:14.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.576+0000 7f2ade03e700 1 -- 172.21.15.99:0/1526523923 learned_addr learned my addr 172.21.15.99:0/1526523923 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:14.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.576+0000 7f2add83d700 1 --2- 172.21.15.99:0/1526523923 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ad815a620 0x7f2ad80b0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:14.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.576+0000 7f2ade83f700 1 --2- 172.21.15.99:0/1526523923 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ad815d580 0x7f2ad80b1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:14.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.576+0000 7f2ade03e700 1 -- 172.21.15.99:0/1526523923 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ad815d580 msgr2=0x7f2ad80b1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:14.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.576+0000 7f2ade03e700 1 --2- 172.21.15.99:0/1526523923 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ad815d580 0x7f2ad80b1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:14.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.576+0000 7f2ade03e700 1 -- 172.21.15.99:0/1526523923 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ad815a620 msgr2=0x7f2ad80b0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:14.586 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.576+0000 7f2ade03e700 1 --2- 172.21.15.99:0/1526523923 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ad815a620 0x7f2ad80b0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:14.586 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.576+0000 7f2ade03e700 1 -- 172.21.15.99:0/1526523923 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ac80094f0 con 0x7f2ad8157870 2023-10-08T00:16:14.586 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.576+0000 7f2add83d700 1 --2- 172.21.15.99:0/1526523923 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ad815a620 0x7f2ad80b0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:16:14.586 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.576+0000 7f2ade83f700 1 --2- 172.21.15.99:0/1526523923 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ad815d580 0x7f2ad80b1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:16:14.586 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.576+0000 7f2ade03e700 1 --2- 172.21.15.99:0/1526523923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ad8157870 0x7f2ad80cf950 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f2ad400c7d0 tx=0x7f2ad4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:14.587 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.576+0000 7f2ace7fc700 1 -- 172.21.15.99:0/1526523923 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad4011600 con 0x7f2ad8157870 2023-10-08T00:16:14.587 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.576+0000 7f2ace7fc700 1 -- 172.21.15.99:0/1526523923 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ad4011760 con 0x7f2ad8157870 2023-10-08T00:16:14.587 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.576+0000 7f2ae02a2700 1 -- 172.21.15.99:0/1526523923 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ad80d05e0 con 0x7f2ad8157870 2023-10-08T00:16:14.587 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.576+0000 7f2ace7fc700 1 -- 172.21.15.99:0/1526523923 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad4011b20 con 0x7f2ad8157870 2023-10-08T00:16:14.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.576+0000 7f2ae02a2700 1 -- 172.21.15.99:0/1526523923 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ad80b1570 con 0x7f2ad8157870 2023-10-08T00:16:14.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.577+0000 7f2ae02a2700 1 -- 172.21.15.99:0/1526523923 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ad804c3c0 con 0x7f2ad8157870 2023-10-08T00:16:14.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.578+0000 7f2ace7fc700 1 -- 172.21.15.99:0/1526523923 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2ad40193f0 con 0x7f2ad8157870 2023-10-08T00:16:14.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.578+0000 7f2ace7fc700 1 --2- 172.21.15.99:0/1526523923 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2abc066f30 0x7f2abc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:14.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.578+0000 7f2ace7fc700 1 -- 172.21.15.99:0/1526523923 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f2ad4084640 con 0x7f2ad8157870 2023-10-08T00:16:14.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.578+0000 7f2add83d700 1 --2- 172.21.15.99:0/1526523923 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2abc066f30 0x7f2abc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:14.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.579+0000 7f2add83d700 1 --2- 172.21.15.99:0/1526523923 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2abc066f30 0x7f2abc0693e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2ad80b03b0 tx=0x7f2ac8017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:14.590 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.582+0000 7f2ace7fc700 1 -- 172.21.15.99:0/1526523923 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2ad4051bb0 con 0x7f2ad8157870 2023-10-08T00:16:14.850 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:14.848+0000 7f2ae02a2700 1 -- 172.21.15.99:0/1526523923 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]} v 0) v1 -- 0x7f2ad815c840 con 0x7f2ad8157870 2023-10-08T00:16:15.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.114+0000 7f2ace7fc700 1 -- 172.21.15.99:0/1526523923 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]}]=0 v279) v1 ==== 137+0+0 (secure 0 0 0) 0x7f2ad40551d0 con 0x7f2ad8157870 2023-10-08T00:16:15.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.116+0000 7f2ae02a2700 1 -- 172.21.15.99:0/1526523923 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2abc066f30 msgr2=0x7f2abc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:15.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.116+0000 7f2ae02a2700 1 --2- 172.21.15.99:0/1526523923 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2abc066f30 0x7f2abc0693e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2ad80b03b0 tx=0x7f2ac8017000 comp rx=0 tx=0).stop 2023-10-08T00:16:15.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.116+0000 7f2ae02a2700 1 -- 172.21.15.99:0/1526523923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ad8157870 msgr2=0x7f2ad80cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:15.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.116+0000 7f2ae02a2700 1 --2- 172.21.15.99:0/1526523923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ad8157870 0x7f2ad80cf950 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f2ad400c7d0 tx=0x7f2ad4008040 comp rx=0 tx=0).stop 2023-10-08T00:16:15.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.116+0000 7f2ae02a2700 1 -- 172.21.15.99:0/1526523923 shutdown_connections 2023-10-08T00:16:15.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.116+0000 7f2ae02a2700 1 --2- 172.21.15.99:0/1526523923 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2abc066f30 0x7f2abc0693e0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:15.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.116+0000 7f2ae02a2700 1 --2- 172.21.15.99:0/1526523923 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ad815d580 0x7f2ad80b1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:15.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.116+0000 7f2ae02a2700 1 --2- 172.21.15.99:0/1526523923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ad8157870 0x7f2ad80cf950 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:15.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.116+0000 7f2ae02a2700 1 --2- 172.21.15.99:0/1526523923 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ad815a620 0x7f2ad80b0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:15.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.116+0000 7f2ae02a2700 1 -- 172.21.15.99:0/1526523923 >> 172.21.15.99:0/1526523923 conn(0x7f2ad8068ce0 msgr2=0x7f2ad815da10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:15.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.116+0000 7f2ae02a2700 1 -- 172.21.15.99:0/1526523923 shutdown_connections 2023-10-08T00:16:15.119 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.116+0000 7f2ae02a2700 1 -- 172.21.15.99:0/1526523923 wait complete. 2023-10-08T00:16:15.130 INFO:tasks.cephfs_test_runner:test_full_same_file (tasks.cephfs.test_full.TestClusterFull) ... ok 2023-10-08T00:16:15.130 INFO:tasks.cephfs_test_runner:Starting test: test_barrier (tasks.cephfs.test_full.TestQuotaFull) That when an OSD epoch barrier is set on an MDS, subsequently 2023-10-08T00:16:15.130 INFO:tasks.cephfs_test_runner:test_barrier (tasks.cephfs.test_full.TestQuotaFull) 2023-10-08T00:16:15.130 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_barrier' 2023-10-08T00:16:15.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.243+0000 7f341578f700 1 Processor -- start 2023-10-08T00:16:15.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.243+0000 7f341578f700 1 -- start start 2023-10-08T00:16:15.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.244+0000 7f341578f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34100cdab0 0x7f341015d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:15.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.244+0000 7f341578f700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f341015dda0 0x7f341015cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:15.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.244+0000 7f341578f700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34100cb7c0 con 0x7f34100cd6e0 2023-10-08T00:16:15.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.244+0000 7f341578f700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f341015e850 con 0x7f341015dda0 2023-10-08T00:16:15.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.244+0000 7f341578f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34100cc360 con 0x7f34100cdab0 2023-10-08T00:16:15.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.244+0000 7f340f7fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34100cdab0 0x7f341015d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:15.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.244+0000 7f3414f8e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f341015dda0 0x7f341015cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:15.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.244+0000 7f3414f8e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f341015dda0 0x7f341015cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.99:47470/0 (socket says 172.21.15.99:47470) 2023-10-08T00:16:15.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.244+0000 7f3414f8e700 1 -- 172.21.15.99:0/1731178846 learned_addr learned my addr 172.21.15.99:0/1731178846 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:15.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.244+0000 7f340f7fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34100cdab0 0x7f341015d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.99:40570/0 (socket says 172.21.15.99:40570) 2023-10-08T00:16:15.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.245+0000 7f3414f8e700 1 -- 172.21.15.99:0/1731178846 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34100cdab0 msgr2=0x7f341015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:15.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.245+0000 7f3414f8e700 1 --2- 172.21.15.99:0/1731178846 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34100cdab0 0x7f341015d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:15.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.245+0000 7f3414f8e700 1 -- 172.21.15.99:0/1731178846 >> v1:172.21.15.191:6789/0 conn(0x7f34100cd6e0 legacy=0x7f34100cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:15.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.245+0000 7f3414f8e700 1 -- 172.21.15.99:0/1731178846 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f341015e290 con 0x7f341015dda0 2023-10-08T00:16:15.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.245+0000 7f340f7fe700 1 --2- 172.21.15.99:0/1731178846 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34100cdab0 0x7f341015d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:16:15.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.245+0000 7f3414f8e700 1 --2- 172.21.15.99:0/1731178846 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f341015dda0 0x7f341015cbd0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3408009f20 tx=0x7f3408009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b811cae74ffbe76b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:15.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.245+0000 7f340effd700 1 -- 172.21.15.99:0/1731178846 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340800e070 con 0x7f341015dda0 2023-10-08T00:16:15.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.245+0000 7f340effd700 1 -- 172.21.15.99:0/1731178846 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f340800b9d0 con 0x7f341015dda0 2023-10-08T00:16:15.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.245+0000 7f340effd700 1 -- 172.21.15.99:0/1731178846 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34080092e0 con 0x7f341015dda0 2023-10-08T00:16:15.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.246+0000 7f341578f700 1 -- 172.21.15.99:0/1731178846 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f341015dda0 msgr2=0x7f341015cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:15.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.246+0000 7f341578f700 1 --2- 172.21.15.99:0/1731178846 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f341015dda0 0x7f341015cbd0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3408009f20 tx=0x7f3408009840 comp rx=0 tx=0).stop 2023-10-08T00:16:15.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.246+0000 7f341578f700 1 -- 172.21.15.99:0/1731178846 shutdown_connections 2023-10-08T00:16:15.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.246+0000 7f341578f700 1 --2- 172.21.15.99:0/1731178846 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f341015dda0 0x7f341015cbd0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:15.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.246+0000 7f341578f700 1 --2- 172.21.15.99:0/1731178846 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34100cdab0 0x7f341015d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:15.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.246+0000 7f341578f700 1 -- 172.21.15.99:0/1731178846 >> 172.21.15.99:0/1731178846 conn(0x7f3410067660 msgr2=0x7f3410067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:15.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.246+0000 7f341578f700 1 -- 172.21.15.99:0/1731178846 shutdown_connections 2023-10-08T00:16:15.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.246+0000 7f341578f700 1 -- 172.21.15.99:0/1731178846 wait complete. 2023-10-08T00:16:15.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.247+0000 7f341578f700 1 Processor -- start 2023-10-08T00:16:15.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.247+0000 7f341578f700 1 -- start start 2023-10-08T00:16:15.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.248+0000 7f341578f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34100cd6e0 0x7f341016ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:15.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.248+0000 7f341578f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34100cdab0 0x7f3410144270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:15.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.248+0000 7f341578f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f341015dda0 0x7f3410158550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:15.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.248+0000 7f341578f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34101447b0 con 0x7f34100cd6e0 2023-10-08T00:16:15.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.248+0000 7f341578f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341016d8b0 con 0x7f34100cdab0 2023-10-08T00:16:15.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.248+0000 7f341578f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f341016db20 con 0x7f341015dda0 2023-10-08T00:16:15.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.248+0000 7f340ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34100cd6e0 0x7f341016ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:15.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.248+0000 7f340ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34100cd6e0 0x7f341016ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35792/0 (socket says 172.21.15.99:35792) 2023-10-08T00:16:15.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.248+0000 7f340ffff700 1 -- 172.21.15.99:0/2952738431 learned_addr learned my addr 172.21.15.99:0/2952738431 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:15.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.248+0000 7f340f7fe700 1 --2- 172.21.15.99:0/2952738431 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34100cdab0 0x7f3410144270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:15.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.248+0000 7f3414f8e700 1 --2- 172.21.15.99:0/2952738431 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f341015dda0 0x7f3410158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:15.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.248+0000 7f340ffff700 1 -- 172.21.15.99:0/2952738431 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f341015dda0 msgr2=0x7f3410158550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:15.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.248+0000 7f340ffff700 1 --2- 172.21.15.99:0/2952738431 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f341015dda0 0x7f3410158550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:15.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.248+0000 7f340ffff700 1 -- 172.21.15.99:0/2952738431 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34100cdab0 msgr2=0x7f3410144270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:15.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.248+0000 7f340ffff700 1 --2- 172.21.15.99:0/2952738431 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34100cdab0 0x7f3410144270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:15.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.248+0000 7f340ffff700 1 -- 172.21.15.99:0/2952738431 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3404009420 con 0x7f34100cd6e0 2023-10-08T00:16:15.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.248+0000 7f340ffff700 1 --2- 172.21.15.99:0/2952738431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34100cd6e0 0x7f341016ad80 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f340000c590 tx=0x7f340000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:15.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.249+0000 7f33f7fff700 1 -- 172.21.15.99:0/2952738431 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3400011590 con 0x7f34100cd6e0 2023-10-08T00:16:15.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.249+0000 7f33f7fff700 1 -- 172.21.15.99:0/2952738431 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3400011710 con 0x7f34100cd6e0 2023-10-08T00:16:15.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.249+0000 7f341578f700 1 -- 172.21.15.99:0/2952738431 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3408009ae0 con 0x7f34100cd6e0 2023-10-08T00:16:15.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.249+0000 7f33f7fff700 1 -- 172.21.15.99:0/2952738431 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3400011ad0 con 0x7f34100cd6e0 2023-10-08T00:16:15.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.249+0000 7f341578f700 1 -- 172.21.15.99:0/2952738431 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f341016e0e0 con 0x7f34100cd6e0 2023-10-08T00:16:15.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.250+0000 7f341578f700 1 -- 172.21.15.99:0/2952738431 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f341016e430 con 0x7f34100cd6e0 2023-10-08T00:16:15.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.250+0000 7f33f7fff700 1 -- 172.21.15.99:0/2952738431 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f340001b3b0 con 0x7f34100cd6e0 2023-10-08T00:16:15.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.250+0000 7f33f7fff700 1 --2- 172.21.15.99:0/2952738431 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f33fc066f30 0x7f33fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:15.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.251+0000 7f33f7fff700 1 -- 172.21.15.99:0/2952738431 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f3400083560 con 0x7f34100cd6e0 2023-10-08T00:16:15.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.251+0000 7f340f7fe700 1 --2- 172.21.15.99:0/2952738431 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f33fc066f30 0x7f33fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:15.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.251+0000 7f340f7fe700 1 --2- 172.21.15.99:0/2952738431 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f33fc066f30 0x7f33fc0693e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f34100b5df0 tx=0x7f34040091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:15.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.255+0000 7f33f7fff700 1 -- 172.21.15.99:0/2952738431 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3400050b80 con 0x7f34100cd6e0 2023-10-08T00:16:15.511 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:15.509+0000 7f341578f700 1 -- 172.21.15.99:0/2952738431 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]} v 0) v1 -- 0x7f341004bb80 con 0x7f34100cd6e0 2023-10-08T00:16:16.121 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.121+0000 7f33f7fff700 1 -- 172.21.15.99:0/2952738431 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]}]=0 v280) v1 ==== 131+0+0 (secure 0 0 0) 0x7f3400017020 con 0x7f34100cd6e0 2023-10-08T00:16:16.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.123+0000 7f341578f700 1 -- 172.21.15.99:0/2952738431 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f33fc066f30 msgr2=0x7f33fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:16.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.123+0000 7f341578f700 1 --2- 172.21.15.99:0/2952738431 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f33fc066f30 0x7f33fc0693e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f34100b5df0 tx=0x7f34040091a0 comp rx=0 tx=0).stop 2023-10-08T00:16:16.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.123+0000 7f341578f700 1 -- 172.21.15.99:0/2952738431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34100cd6e0 msgr2=0x7f341016ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:16.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.123+0000 7f341578f700 1 --2- 172.21.15.99:0/2952738431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34100cd6e0 0x7f341016ad80 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f340000c590 tx=0x7f340000c8a0 comp rx=0 tx=0).stop 2023-10-08T00:16:16.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.123+0000 7f341578f700 1 -- 172.21.15.99:0/2952738431 shutdown_connections 2023-10-08T00:16:16.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.123+0000 7f341578f700 1 --2- 172.21.15.99:0/2952738431 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f33fc066f30 0x7f33fc0693e0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:16.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.123+0000 7f341578f700 1 --2- 172.21.15.99:0/2952738431 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f341015dda0 0x7f3410158550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:16.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.123+0000 7f341578f700 1 --2- 172.21.15.99:0/2952738431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34100cd6e0 0x7f341016ad80 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:16.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.123+0000 7f341578f700 1 --2- 172.21.15.99:0/2952738431 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34100cdab0 0x7f3410144270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:16.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.123+0000 7f341578f700 1 -- 172.21.15.99:0/2952738431 >> 172.21.15.99:0/2952738431 conn(0x7f3410067660 msgr2=0x7f3410156350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:16.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.123+0000 7f341578f700 1 -- 172.21.15.99:0/2952738431 shutdown_connections 2023-10-08T00:16:16.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.123+0000 7f341578f700 1 -- 172.21.15.99:0/2952738431 wait complete. 2023-10-08T00:16:16.137 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T00:16:16.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.256+0000 7f75d9399700 1 Processor -- start 2023-10-08T00:16:16.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.256+0000 7f75d9399700 1 -- start start 2023-10-08T00:16:16.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.257+0000 7f75d9399700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f75d4159e80 0x7f75d40b4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:16.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.257+0000 7f75d9399700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f75d40aada0 0x7f75d40ab210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:16.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.257+0000 7f75d9399700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f75d40ab750 0x7f75d40b2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:16.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.257+0000 7f75d9399700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75d40b50a0 con 0x7f75d40ab750 2023-10-08T00:16:16.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.257+0000 7f75d9399700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75d40b6b10 con 0x7f75d4159e80 2023-10-08T00:16:16.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.257+0000 7f75d9399700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75d40b0e70 con 0x7f75d40aada0 2023-10-08T00:16:16.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.257+0000 7f75d8b98700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f75d40ab750 0x7f75d40b2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:16.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.257+0000 7f75d8b98700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f75d40ab750 0x7f75d40b2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35796/0 (socket says 172.21.15.99:35796) 2023-10-08T00:16:16.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.257+0000 7f75d8b98700 1 -- 172.21.15.99:0/1339097278 learned_addr learned my addr 172.21.15.99:0/1339097278 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:16.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.257+0000 7f75d3fff700 1 --2- 172.21.15.99:0/1339097278 >> v2:172.21.15.191:3300/0 conn(0x7f75d4159e80 0x7f75d40b4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:16.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.257+0000 7f75d37fe700 1 --2- 172.21.15.99:0/1339097278 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f75d40aada0 0x7f75d40ab210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:16.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.257+0000 7f75d8b98700 1 -- 172.21.15.99:0/1339097278 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f75d40aada0 msgr2=0x7f75d40ab210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:16.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.257+0000 7f75d8b98700 1 --2- 172.21.15.99:0/1339097278 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f75d40aada0 0x7f75d40ab210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:16.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.257+0000 7f75d8b98700 1 -- 172.21.15.99:0/1339097278 >> v2:172.21.15.191:3300/0 conn(0x7f75d4159e80 msgr2=0x7f75d40b4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:16.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.257+0000 7f75d8b98700 1 --2- 172.21.15.99:0/1339097278 >> v2:172.21.15.191:3300/0 conn(0x7f75d4159e80 0x7f75d40b4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:16.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.257+0000 7f75d8b98700 1 -- 172.21.15.99:0/1339097278 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d40b52c0 con 0x7f75d40ab750 2023-10-08T00:16:16.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.258+0000 7f75d8b98700 1 --2- 172.21.15.99:0/1339097278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75d40ab750 0x7f75d40b2570 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f75d408a5c0 tx=0x7f75cc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5492f293a49cf67c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:16.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.258+0000 7f75d2ffd700 1 -- 172.21.15.99:0/1339097278 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75cc00e070 con 0x7f75d40ab750 2023-10-08T00:16:16.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.258+0000 7f75d2ffd700 1 -- 172.21.15.99:0/1339097278 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75cc00ba40 con 0x7f75d40ab750 2023-10-08T00:16:16.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.258+0000 7f75d2ffd700 1 -- 172.21.15.99:0/1339097278 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75cc0092e0 con 0x7f75d40ab750 2023-10-08T00:16:16.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.258+0000 7f75d9399700 1 -- 172.21.15.99:0/1339097278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75d40ab750 msgr2=0x7f75d40b2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:16.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.258+0000 7f75d9399700 1 --2- 172.21.15.99:0/1339097278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75d40ab750 0x7f75d40b2570 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f75d408a5c0 tx=0x7f75cc009840 comp rx=0 tx=0).stop 2023-10-08T00:16:16.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.258+0000 7f75d9399700 1 -- 172.21.15.99:0/1339097278 shutdown_connections 2023-10-08T00:16:16.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.258+0000 7f75d9399700 1 --2- 172.21.15.99:0/1339097278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75d40ab750 0x7f75d40b2570 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:16.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.258+0000 7f75d9399700 1 --2- 172.21.15.99:0/1339097278 >> v2:172.21.15.191:3300/0 conn(0x7f75d4159e80 0x7f75d40b4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:16.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.258+0000 7f75d9399700 1 --2- 172.21.15.99:0/1339097278 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f75d40aada0 0x7f75d40ab210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:16.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.258+0000 7f75d9399700 1 -- 172.21.15.99:0/1339097278 >> 172.21.15.99:0/1339097278 conn(0x7f75d4099f50 msgr2=0x7f75d4067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:16.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.258+0000 7f75d9399700 1 -- 172.21.15.99:0/1339097278 shutdown_connections 2023-10-08T00:16:16.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.258+0000 7f75d9399700 1 -- 172.21.15.99:0/1339097278 wait complete. 2023-10-08T00:16:16.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.259+0000 7f75d9399700 1 Processor -- start 2023-10-08T00:16:16.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.260+0000 7f75d9399700 1 -- start start 2023-10-08T00:16:16.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.260+0000 7f75d9399700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75d40aada0 0x7f75d40ce740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:16.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.260+0000 7f75d9399700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f75d40ab750 0x7f75d4157060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:16.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.260+0000 7f75d9399700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f75d4159e80 0x7f75d406a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:16.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.260+0000 7f75d9399700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d415cec0 con 0x7f75d40aada0 2023-10-08T00:16:16.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.260+0000 7f75d9399700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d409d300 con 0x7f75d4159e80 2023-10-08T00:16:16.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.260+0000 7f75d9399700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75d40cec80 con 0x7f75d40ab750 2023-10-08T00:16:16.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.260+0000 7f75d3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75d40aada0 0x7f75d40ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:16.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.260+0000 7f75d3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75d40aada0 0x7f75d40ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35802/0 (socket says 172.21.15.99:35802) 2023-10-08T00:16:16.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.260+0000 7f75d3fff700 1 -- 172.21.15.99:0/1417446691 learned_addr learned my addr 172.21.15.99:0/1417446691 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:16.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.261+0000 7f75d37fe700 1 --2- 172.21.15.99:0/1417446691 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f75d40ab750 0x7f75d4157060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:16.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.261+0000 7f75d8b98700 1 --2- 172.21.15.99:0/1417446691 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f75d4159e80 0x7f75d406a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:16.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.261+0000 7f75d3fff700 1 -- 172.21.15.99:0/1417446691 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f75d40ab750 msgr2=0x7f75d4157060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:16.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.261+0000 7f75d3fff700 1 --2- 172.21.15.99:0/1417446691 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f75d40ab750 0x7f75d4157060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:16.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.261+0000 7f75d3fff700 1 -- 172.21.15.99:0/1417446691 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f75d4159e80 msgr2=0x7f75d406a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:16.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.261+0000 7f75d3fff700 1 --2- 172.21.15.99:0/1417446691 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f75d4159e80 0x7f75d406a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:16.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.261+0000 7f75d3fff700 1 -- 172.21.15.99:0/1417446691 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75cc009ae0 con 0x7f75d40aada0 2023-10-08T00:16:16.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.261+0000 7f75d8b98700 1 --2- 172.21.15.99:0/1417446691 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f75d4159e80 0x7f75d406a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:16.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.261+0000 7f75d3fff700 1 --2- 172.21.15.99:0/1417446691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75d40aada0 0x7f75d40ce740 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f75c400d890 tx=0x7f75c4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:16.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.261+0000 7f75bbfff700 1 -- 172.21.15.99:0/1417446691 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c400b600 con 0x7f75d40aada0 2023-10-08T00:16:16.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.261+0000 7f75bbfff700 1 -- 172.21.15.99:0/1417446691 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75c400b760 con 0x7f75d40aada0 2023-10-08T00:16:16.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.261+0000 7f75bbfff700 1 -- 172.21.15.99:0/1417446691 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c400bae0 con 0x7f75d40aada0 2023-10-08T00:16:16.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.261+0000 7f75d9399700 1 -- 172.21.15.99:0/1417446691 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75d40ceeb0 con 0x7f75d40aada0 2023-10-08T00:16:16.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.261+0000 7f75d9399700 1 -- 172.21.15.99:0/1417446691 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75d40cf470 con 0x7f75d40aada0 2023-10-08T00:16:16.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.262+0000 7f75d9399700 1 -- 172.21.15.99:0/1417446691 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75d4068050 con 0x7f75d40aada0 2023-10-08T00:16:16.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.262+0000 7f75bbfff700 1 -- 172.21.15.99:0/1417446691 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f75c401e460 con 0x7f75d40aada0 2023-10-08T00:16:16.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.263+0000 7f75bbfff700 1 --2- 172.21.15.99:0/1417446691 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f75c0066f30 0x7f75c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:16.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.263+0000 7f75bbfff700 1 -- 172.21.15.99:0/1417446691 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f75c40848e0 con 0x7f75d40aada0 2023-10-08T00:16:16.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.263+0000 7f75d37fe700 1 --2- 172.21.15.99:0/1417446691 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f75c0066f30 0x7f75c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:16.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.263+0000 7f75d37fe700 1 --2- 172.21.15.99:0/1417446691 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f75c0066f30 0x7f75c00693e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f75d415d170 tx=0x7f75c8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:16.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.267+0000 7f75bbfff700 1 -- 172.21.15.99:0/1417446691 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f75c4051e50 con 0x7f75d40aada0 2023-10-08T00:16:16.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.450+0000 7f75d9399700 1 -- 172.21.15.99:0/1417446691 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f75d4089110 con 0x7f75d40aada0 2023-10-08T00:16:16.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.453+0000 7f75bbfff700 1 -- 172.21.15.99:0/1417446691 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f75c4055470 con 0x7f75d40aada0 2023-10-08T00:16:16.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.456+0000 7f75d9399700 1 -- 172.21.15.99:0/1417446691 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f75c0066f30 msgr2=0x7f75c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:16.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.456+0000 7f75d9399700 1 --2- 172.21.15.99:0/1417446691 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f75c0066f30 0x7f75c00693e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f75d415d170 tx=0x7f75c8008040 comp rx=0 tx=0).stop 2023-10-08T00:16:16.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.456+0000 7f75d9399700 1 -- 172.21.15.99:0/1417446691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75d40aada0 msgr2=0x7f75d40ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:16.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.456+0000 7f75d9399700 1 --2- 172.21.15.99:0/1417446691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75d40aada0 0x7f75d40ce740 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f75c400d890 tx=0x7f75c4008040 comp rx=0 tx=0).stop 2023-10-08T00:16:16.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.456+0000 7f75d9399700 1 -- 172.21.15.99:0/1417446691 shutdown_connections 2023-10-08T00:16:16.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.456+0000 7f75d9399700 1 --2- 172.21.15.99:0/1417446691 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f75c0066f30 0x7f75c00693e0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:16.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.456+0000 7f75d9399700 1 --2- 172.21.15.99:0/1417446691 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f75d4159e80 0x7f75d406a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:16.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.456+0000 7f75d9399700 1 --2- 172.21.15.99:0/1417446691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75d40aada0 0x7f75d40ce740 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:16.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.456+0000 7f75d9399700 1 --2- 172.21.15.99:0/1417446691 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f75d40ab750 0x7f75d4157060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:16.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.456+0000 7f75d9399700 1 -- 172.21.15.99:0/1417446691 >> 172.21.15.99:0/1417446691 conn(0x7f75d4099f50 msgr2=0x7f75d415f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:16.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.456+0000 7f75d9399700 1 -- 172.21.15.99:0/1417446691 shutdown_connections 2023-10-08T00:16:16.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.456+0000 7f75d9399700 1 -- 172.21.15.99:0/1417446691 wait complete. 2023-10-08T00:16:16.470 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T00:16:16.470 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T00:16:16.515 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T00:16:16.568 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T00:16:16.569 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-08T00:16:16.569 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:16:16.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.682+0000 7f96a12a9700 1 Processor -- start 2023-10-08T00:16:16.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.682+0000 7f96a12a9700 1 -- start start 2023-10-08T00:16:16.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.683+0000 7f96a12a9700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f969c0b0890 0x7f969c0cb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:16.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.683+0000 7f96a12a9700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f969c0cbd40 0x7f969c067660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:16.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.683+0000 7f96a12a9700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f969c0cb370 con 0x7f969c06c630 2023-10-08T00:16:16.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.683+0000 7f96a12a9700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f969c0b0c60 con 0x7f969c0cbd40 2023-10-08T00:16:16.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.683+0000 7f96a12a9700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f969c06a1e0 con 0x7f969c0b0890 2023-10-08T00:16:16.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.683+0000 7f96927fc700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f969c0cbd40 0x7f969c067660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:16.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.683+0000 7f96927fc700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f969c0cbd40 0x7f969c067660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35814/0 (socket says 172.21.15.99:35814) 2023-10-08T00:16:16.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.683+0000 7f969b7fe700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f969c06c630 0x7f969c15dc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56262/0 (socket says 172.21.15.99:56262) 2023-10-08T00:16:16.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.683+0000 7f96927fc700 1 -- 172.21.15.99:0/2900821420 learned_addr learned my addr 172.21.15.99:0/2900821420 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:16.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.683+0000 7f969affd700 1 --2- 172.21.15.99:0/2900821420 >> v2:172.21.15.191:3300/0 conn(0x7f969c0b0890 0x7f969c0cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:16.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.684+0000 7f96927fc700 1 -- 172.21.15.99:0/2900821420 >> v2:172.21.15.191:3300/0 conn(0x7f969c0b0890 msgr2=0x7f969c0cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:16.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.684+0000 7f96927fc700 1 --2- 172.21.15.99:0/2900821420 >> v2:172.21.15.191:3300/0 conn(0x7f969c0b0890 0x7f969c0cb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:16.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.684+0000 7f96927fc700 1 -- 172.21.15.99:0/2900821420 >> v1:172.21.15.99:6789/0 conn(0x7f969c06c630 legacy=0x7f969c15dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:16.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.684+0000 7f96927fc700 1 -- 172.21.15.99:0/2900821420 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f969c06cb80 con 0x7f969c0cbd40 2023-10-08T00:16:16.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.684+0000 7f969a7fc700 1 -- 172.21.15.99:0/2900821420 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1352443647 0 0) 0x7f969c0cb370 con 0x7f969c06c630 2023-10-08T00:16:16.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.684+0000 7f96927fc700 1 --2- 172.21.15.99:0/2900821420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f969c0cbd40 0x7f969c067660 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f969c06aae0 tx=0x7f9684009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=951efe4ef71a5457 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:16.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.684+0000 7f969a7fc700 1 -- 172.21.15.99:0/2900821420 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968400b860 con 0x7f969c0cbd40 2023-10-08T00:16:16.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.684+0000 7f969a7fc700 1 -- 172.21.15.99:0/2900821420 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f968400b9c0 con 0x7f969c0cbd40 2023-10-08T00:16:16.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.684+0000 7f969a7fc700 1 -- 172.21.15.99:0/2900821420 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968400bd60 con 0x7f969c0cbd40 2023-10-08T00:16:16.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.684+0000 7f96a12a9700 1 -- 172.21.15.99:0/2900821420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f969c0cbd40 msgr2=0x7f969c067660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:16.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.684+0000 7f96a12a9700 1 --2- 172.21.15.99:0/2900821420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f969c0cbd40 0x7f969c067660 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f969c06aae0 tx=0x7f9684009250 comp rx=0 tx=0).stop 2023-10-08T00:16:16.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.684+0000 7f96a12a9700 1 -- 172.21.15.99:0/2900821420 shutdown_connections 2023-10-08T00:16:16.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.684+0000 7f96a12a9700 1 --2- 172.21.15.99:0/2900821420 >> v2:172.21.15.191:3300/0 conn(0x7f969c0b0890 0x7f969c0cb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:16.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.684+0000 7f96a12a9700 1 --2- 172.21.15.99:0/2900821420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f969c0cbd40 0x7f969c067660 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:16.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.684+0000 7f96a12a9700 1 -- 172.21.15.99:0/2900821420 >> 172.21.15.99:0/2900821420 conn(0x7f969c06fe50 msgr2=0x7f969c069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:16.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.684+0000 7f96a12a9700 1 -- 172.21.15.99:0/2900821420 shutdown_connections 2023-10-08T00:16:16.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.684+0000 7f96a12a9700 1 -- 172.21.15.99:0/2900821420 wait complete. 2023-10-08T00:16:16.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.686+0000 7f96a12a9700 1 Processor -- start 2023-10-08T00:16:16.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.686+0000 7f96a12a9700 1 -- start start 2023-10-08T00:16:16.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.686+0000 7f96a12a9700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f969c06c630 0x7f969c0c3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:16.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.686+0000 7f96a12a9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f969c0b0890 0x7f969c15a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:16.694 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.686+0000 7f96a12a9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f969c0cbd40 0x7f969c0cde10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:16.694 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.686+0000 7f96a12a9700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969c0c2a80 con 0x7f969c0b0890 2023-10-08T00:16:16.694 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.686+0000 7f96a12a9700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969c0c2d60 con 0x7f969c06c630 2023-10-08T00:16:16.694 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.686+0000 7f96a12a9700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f969c0c3040 con 0x7f969c0cbd40 2023-10-08T00:16:16.694 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.687+0000 7f96927fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f969c0b0890 0x7f969c15a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:16.694 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.687+0000 7f96927fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f969c0b0890 0x7f969c15a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35820/0 (socket says 172.21.15.99:35820) 2023-10-08T00:16:16.694 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.687+0000 7f96927fc700 1 -- 172.21.15.99:0/1335927969 learned_addr learned my addr 172.21.15.99:0/1335927969 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:16.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.687+0000 7f969affd700 1 --2- 172.21.15.99:0/1335927969 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f969c06c630 0x7f969c0c3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:16.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.687+0000 7f969b7fe700 1 --2- 172.21.15.99:0/1335927969 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f969c0cbd40 0x7f969c0cde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:16.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.687+0000 7f96927fc700 1 -- 172.21.15.99:0/1335927969 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f969c0cbd40 msgr2=0x7f969c0cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:16.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.687+0000 7f96927fc700 1 --2- 172.21.15.99:0/1335927969 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f969c0cbd40 0x7f969c0cde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:16.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.687+0000 7f96927fc700 1 -- 172.21.15.99:0/1335927969 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f969c06c630 msgr2=0x7f969c0c3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:16.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.687+0000 7f96927fc700 1 --2- 172.21.15.99:0/1335927969 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f969c06c630 0x7f969c0c3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:16.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.687+0000 7f96927fc700 1 -- 172.21.15.99:0/1335927969 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96840094f0 con 0x7f969c0b0890 2023-10-08T00:16:16.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.687+0000 7f96927fc700 1 --2- 172.21.15.99:0/1335927969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f969c0b0890 0x7f969c15a190 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f9684006f50 tx=0x7f9684017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:16.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.687+0000 7f96937fe700 1 -- 172.21.15.99:0/1335927969 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9684024440 con 0x7f969c0b0890 2023-10-08T00:16:16.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.687+0000 7f96937fe700 1 -- 172.21.15.99:0/1335927969 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96840245e0 con 0x7f969c0b0890 2023-10-08T00:16:16.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.687+0000 7f96a12a9700 1 -- 172.21.15.99:0/1335927969 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f969c0c3320 con 0x7f969c0b0890 2023-10-08T00:16:16.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.687+0000 7f96937fe700 1 -- 172.21.15.99:0/1335927969 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96840249a0 con 0x7f969c0b0890 2023-10-08T00:16:16.696 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.687+0000 7f96a12a9700 1 -- 172.21.15.99:0/1335927969 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f969c0c4060 con 0x7f969c0b0890 2023-10-08T00:16:16.696 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.688+0000 7f96a12a9700 1 -- 172.21.15.99:0/1335927969 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f969c04c3c0 con 0x7f969c0b0890 2023-10-08T00:16:16.696 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.689+0000 7f96937fe700 1 -- 172.21.15.99:0/1335927969 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9684024d50 con 0x7f969c0b0890 2023-10-08T00:16:16.696 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.689+0000 7f96937fe700 1 --2- 172.21.15.99:0/1335927969 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f967c066f30 0x7f967c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:16.696 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.689+0000 7f96937fe700 1 -- 172.21.15.99:0/1335927969 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f9684085bd0 con 0x7f969c0b0890 2023-10-08T00:16:16.696 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.689+0000 7f969affd700 1 --2- 172.21.15.99:0/1335927969 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f967c066f30 0x7f967c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:16.696 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.690+0000 7f969affd700 1 --2- 172.21.15.99:0/1335927969 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f967c066f30 0x7f967c0693e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f968c006fd0 tx=0x7f968c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:16.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.693+0000 7f96937fe700 1 -- 172.21.15.99:0/1335927969 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9684053140 con 0x7f969c0b0890 2023-10-08T00:16:16.917 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.917+0000 7f96a12a9700 1 -- 172.21.15.99:0/1335927969 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f969c06cdc0 con 0x7f969c0b0890 2023-10-08T00:16:16.918 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.917+0000 7f96937fe700 1 -- 172.21.15.99:0/1335927969 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f9684015de0 con 0x7f969c0b0890 2023-10-08T00:16:16.918 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:16:16.918 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":50,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5426,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/280880696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":280880696},{"type":"v1","addr":"172.21.15.191:6836","nonce":280880696}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":5902,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3148129562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":3148129562},{"type":"v1","addr":"172.21.15.99:6835","nonce":3148129562}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":5917,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/1825600944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":1825600944},{"type":"v1","addr":"172.21.15.191:6837","nonce":1825600944}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6675,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/4125613017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4125613017},{"type":"v1","addr":"172.21.15.99:6834","nonce":4125613017}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50}],"filesystems":[]} 2023-10-08T00:16:16.921 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.919+0000 7f96a12a9700 1 -- 172.21.15.99:0/1335927969 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f967c066f30 msgr2=0x7f967c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:16.921 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.919+0000 7f96a12a9700 1 --2- 172.21.15.99:0/1335927969 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f967c066f30 0x7f967c0693e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f968c006fd0 tx=0x7f968c008040 comp rx=0 tx=0).stop 2023-10-08T00:16:16.921 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.919+0000 7f96a12a9700 1 -- 172.21.15.99:0/1335927969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f969c0b0890 msgr2=0x7f969c15a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:16.922 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.919+0000 7f96a12a9700 1 --2- 172.21.15.99:0/1335927969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f969c0b0890 0x7f969c15a190 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f9684006f50 tx=0x7f9684017000 comp rx=0 tx=0).stop 2023-10-08T00:16:16.922 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.920+0000 7f96a12a9700 1 -- 172.21.15.99:0/1335927969 shutdown_connections 2023-10-08T00:16:16.922 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.920+0000 7f96a12a9700 1 --2- 172.21.15.99:0/1335927969 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f967c066f30 0x7f967c0693e0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:16.922 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.920+0000 7f96a12a9700 1 --2- 172.21.15.99:0/1335927969 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f969c06c630 0x7f969c0c3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:16.922 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.920+0000 7f96a12a9700 1 --2- 172.21.15.99:0/1335927969 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f969c0cbd40 0x7f969c0cde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:16.922 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.920+0000 7f96a12a9700 1 --2- 172.21.15.99:0/1335927969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f969c0b0890 0x7f969c15a190 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:16.922 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.920+0000 7f96a12a9700 1 -- 172.21.15.99:0/1335927969 >> 172.21.15.99:0/1335927969 conn(0x7f969c06fe50 msgr2=0x7f969c15c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:16.922 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.920+0000 7f96a12a9700 1 -- 172.21.15.99:0/1335927969 shutdown_connections 2023-10-08T00:16:16.922 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:16.920+0000 7f96a12a9700 1 -- 172.21.15.99:0/1335927969 wait complete. 2023-10-08T00:16:16.922 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 50 2023-10-08T00:16:16.933 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T00:16:16.934 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-08T00:16:16.934 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T00:16:16.934 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:16:16.935 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T00:16:16.935 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-08T00:16:16.935 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T00:16:16.935 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:16:16.936 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T00:16:16.936 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-08T00:16:16.936 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T00:16:16.936 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:16:16.936 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T00:16:16.936 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-08T00:16:16.937 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T00:16:16.937 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:16:16.942 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T00:16:16.943 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T00:16:16.944 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T00:16:16.945 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T00:16:16.947 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T00:16:16.948 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T00:16:16.949 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T00:16:16.949 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T00:16:16.986 INFO:tasks.ceph.mds.a:Started 2023-10-08T00:16:16.987 DEBUG:teuthology.parallel:result is None 2023-10-08T00:16:16.988 INFO:tasks.ceph.mds.c:Started 2023-10-08T00:16:16.988 DEBUG:teuthology.parallel:result is None 2023-10-08T00:16:16.990 INFO:tasks.ceph.mds.b:Started 2023-10-08T00:16:16.990 DEBUG:teuthology.parallel:result is None 2023-10-08T00:16:16.991 INFO:tasks.ceph.mds.d:Started 2023-10-08T00:16:16.991 DEBUG:teuthology.parallel:result is None 2023-10-08T00:16:16.992 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-08T00:16:17.059 INFO:tasks.ceph.mds.a.smithi099.stdout:starting mds.a at 2023-10-08T00:16:17.060 INFO:tasks.ceph.mds.c.smithi099.stdout:starting mds.c at 2023-10-08T00:16:17.074 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2023-10-08T00:16:17.075 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2023-10-08T00:16:17.113 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.112+0000 7fd7d383e700 1 Processor -- start 2023-10-08T00:16:17.113 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.113+0000 7fd7d383e700 1 -- start start 2023-10-08T00:16:17.116 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.113+0000 7fd7d383e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd7cc0c37a0 0x7fd7cc06dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:17.116 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.113+0000 7fd7d383e700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7cc06f320 con 0x7fd7cc0cc9a0 2023-10-08T00:16:17.116 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.113+0000 7fd7d383e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7cc06e710 con 0x7fd7cc0ccd70 2023-10-08T00:16:17.116 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.113+0000 7fd7d383e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7cc06e530 con 0x7fd7cc0c37a0 2023-10-08T00:16:17.116 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.114+0000 7fd7d15da700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fd7cc0cc9a0 0x7fd7cc0c3090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56282/0 (socket says 172.21.15.99:56282) 2023-10-08T00:16:17.116 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.114+0000 7fd7d15da700 1 -- 172.21.15.99:0/2827301918 learned_addr learned my addr 172.21.15.99:0/2827301918 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:17.116 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.114+0000 7fd7d0dd9700 1 --2- 172.21.15.99:0/2827301918 >> v2:172.21.15.191:3300/0 conn(0x7fd7cc0c37a0 0x7fd7cc06dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:17.116 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.114+0000 7fd7c3fff700 1 -- 172.21.15.99:0/2827301918 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 411869716 0 0) 0x7fd7cc06f320 con 0x7fd7cc0cc9a0 2023-10-08T00:16:17.116 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.114+0000 7fd7c3fff700 1 -- 172.21.15.99:0/2827301918 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7b8003570 con 0x7fd7cc0cc9a0 2023-10-08T00:16:17.116 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.114+0000 7fd7c3fff700 1 -- 172.21.15.99:0/2827301918 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 125138368 0 0) 0x7fd7cc06e710 con 0x7fd7cc0ccd70 2023-10-08T00:16:17.116 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.114+0000 7fd7c3fff700 1 -- 172.21.15.99:0/2827301918 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7cc06f320 con 0x7fd7cc0ccd70 2023-10-08T00:16:17.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.114+0000 7fd7d0dd9700 1 -- 172.21.15.99:0/2827301918 >> v1:172.21.15.191:6789/0 conn(0x7fd7cc0ccd70 legacy=0x7fd7cc0cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:17.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.114+0000 7fd7d0dd9700 1 -- 172.21.15.99:0/2827301918 >> v1:172.21.15.99:6789/0 conn(0x7fd7cc0cc9a0 legacy=0x7fd7cc0c3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:17.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.114+0000 7fd7c3fff700 1 -- 172.21.15.99:0/2827301918 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2250116934 0 0) 0x7fd7b8003570 con 0x7fd7cc0cc9a0 2023-10-08T00:16:17.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.114+0000 7fd7d0dd9700 1 -- 172.21.15.99:0/2827301918 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7cc06e9f0 con 0x7fd7cc0c37a0 2023-10-08T00:16:17.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.114+0000 7fd7d0dd9700 1 --2- 172.21.15.99:0/2827301918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7cc0c37a0 0x7fd7cc06dff0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc0098e0 tx=0x7fd7bc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42bb12b484a710b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:17.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.114+0000 7fd7c3fff700 1 -- 172.21.15.99:0/2827301918 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c8001470 con 0x7fd7cc0c37a0 2023-10-08T00:16:17.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.114+0000 7fd7c3fff700 1 -- 172.21.15.99:0/2827301918 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7bc011420 con 0x7fd7cc0c37a0 2023-10-08T00:16:17.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.114+0000 7fd7c3fff700 1 -- 172.21.15.99:0/2827301918 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc0117c0 con 0x7fd7cc0c37a0 2023-10-08T00:16:17.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.115+0000 7fd7d383e700 1 -- 172.21.15.99:0/2827301918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7cc0c37a0 msgr2=0x7fd7cc06dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:17.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.115+0000 7fd7d383e700 1 --2- 172.21.15.99:0/2827301918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7cc0c37a0 0x7fd7cc06dff0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc0098e0 tx=0x7fd7bc0091a0 comp rx=0 tx=0).stop 2023-10-08T00:16:17.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.115+0000 7fd7d383e700 1 -- 172.21.15.99:0/2827301918 shutdown_connections 2023-10-08T00:16:17.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.115+0000 7fd7d383e700 1 --2- 172.21.15.99:0/2827301918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7cc0c37a0 0x7fd7cc06dff0 secure :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc0098e0 tx=0x7fd7bc0091a0 comp rx=0 tx=0).stop 2023-10-08T00:16:17.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.115+0000 7fd7d383e700 1 -- 172.21.15.99:0/2827301918 >> 172.21.15.99:0/2827301918 conn(0x7fd7cc0cd1e0 msgr2=0x7fd7cc0cd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:17.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.115+0000 7fd7d383e700 1 -- 172.21.15.99:0/2827301918 shutdown_connections 2023-10-08T00:16:17.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.115+0000 7fd7d383e700 1 -- 172.21.15.99:0/2827301918 wait complete. 2023-10-08T00:16:17.121 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.116+0000 7fd7d383e700 1 Processor -- start 2023-10-08T00:16:17.121 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.116+0000 7fd7d383e700 1 -- start start 2023-10-08T00:16:17.121 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.117+0000 7fd7d383e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7cc0cc9a0 0x7fd7cc099c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:17.121 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.117+0000 7fd7d383e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7cc0ccd70 0x7fd7cc0ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:17.121 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.117+0000 7fd7d383e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7cc14d5f0 0x7fd7cc0ae1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:17.121 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.117+0000 7fd7d383e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc0b0020 con 0x7fd7cc14d5f0 2023-10-08T00:16:17.122 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.117+0000 7fd7d383e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc09a4b0 con 0x7fd7cc0cc9a0 2023-10-08T00:16:17.122 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.117+0000 7fd7d383e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc09a790 con 0x7fd7cc0ccd70 2023-10-08T00:16:17.122 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.117+0000 7fd7d1ddb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7cc14d5f0 0x7fd7cc0ae1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:17.122 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.117+0000 7fd7d15da700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7cc0cc9a0 0x7fd7cc099c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:17.122 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.117+0000 7fd7d0dd9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7cc0ccd70 0x7fd7cc0ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:17.122 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.117+0000 7fd7d1ddb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7cc14d5f0 0x7fd7cc0ae1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35860/0 (socket says 172.21.15.99:35860) 2023-10-08T00:16:17.122 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.117+0000 7fd7d1ddb700 1 -- 172.21.15.99:0/770829774 learned_addr learned my addr 172.21.15.99:0/770829774 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:17.122 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.117+0000 7fd7d1ddb700 1 -- 172.21.15.99:0/770829774 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7cc0ccd70 msgr2=0x7fd7cc0ca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:17.122 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.117+0000 7fd7d1ddb700 1 --2- 172.21.15.99:0/770829774 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7cc0ccd70 0x7fd7cc0ca1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:17.122 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.118+0000 7fd7d1ddb700 1 -- 172.21.15.99:0/770829774 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7cc0cc9a0 msgr2=0x7fd7cc099c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:17.122 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.118+0000 7fd7d1ddb700 1 --2- 172.21.15.99:0/770829774 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7cc0cc9a0 0x7fd7cc099c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:17.123 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.118+0000 7fd7d1ddb700 1 -- 172.21.15.99:0/770829774 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7bc009420 con 0x7fd7cc14d5f0 2023-10-08T00:16:17.123 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.118+0000 7fd7d1ddb700 1 --2- 172.21.15.99:0/770829774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7cc14d5f0 0x7fd7cc0ae1c0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fd7c400c910 tx=0x7fd7c40027a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:17.123 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.118+0000 7fd7c17fa700 1 -- 172.21.15.99:0/770829774 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c4012630 con 0x7fd7cc14d5f0 2023-10-08T00:16:17.123 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.118+0000 7fd7c17fa700 1 -- 172.21.15.99:0/770829774 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c40127b0 con 0x7fd7cc14d5f0 2023-10-08T00:16:17.123 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.118+0000 7fd7d383e700 1 -- 172.21.15.99:0/770829774 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7cc0b0290 con 0x7fd7cc14d5f0 2023-10-08T00:16:17.123 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.118+0000 7fd7c17fa700 1 -- 172.21.15.99:0/770829774 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c4012b70 con 0x7fd7cc14d5f0 2023-10-08T00:16:17.123 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.118+0000 7fd7d383e700 1 -- 172.21.15.99:0/770829774 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7cc09aa70 con 0x7fd7cc14d5f0 2023-10-08T00:16:17.123 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.119+0000 7fd7d383e700 1 -- 172.21.15.99:0/770829774 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7cc04c3c0 con 0x7fd7cc14d5f0 2023-10-08T00:16:17.123 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.119+0000 7fd7c17fa700 1 -- 172.21.15.99:0/770829774 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd7c401b400 con 0x7fd7cc14d5f0 2023-10-08T00:16:17.123 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.120+0000 7fd7c17fa700 1 --2- 172.21.15.99:0/770829774 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd7b0066f30 0x7fd7b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:17.123 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.120+0000 7fd7c17fa700 1 -- 172.21.15.99:0/770829774 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fd7c4084560 con 0x7fd7cc14d5f0 2023-10-08T00:16:17.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.120+0000 7fd7d15da700 1 --2- 172.21.15.99:0/770829774 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd7b0066f30 0x7fd7b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:17.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.120+0000 7fd7d15da700 1 --2- 172.21.15.99:0/770829774 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd7b0066f30 0x7fd7b00693e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd7c80015a0 tx=0x7fd7c8000bc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:17.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.124+0000 7fd7c17fa700 1 -- 172.21.15.99:0/770829774 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd7c4051ad0 con 0x7fd7cc14d5f0 2023-10-08T00:16:17.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:17.323+0000 7fd7d383e700 1 -- 172.21.15.99:0/770829774 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fd7cc06ecd0 con 0x7fd7cc14d5f0 2023-10-08T00:16:18.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.139+0000 7fd7c17fa700 1 -- 172.21.15.99:0/770829774 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v120) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd7c40550f0 con 0x7fd7cc14d5f0 2023-10-08T00:16:18.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.141+0000 7fd7d383e700 1 -- 172.21.15.99:0/770829774 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd7b0066f30 msgr2=0x7fd7b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:18.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.141+0000 7fd7d383e700 1 --2- 172.21.15.99:0/770829774 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd7b0066f30 0x7fd7b00693e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd7c80015a0 tx=0x7fd7c8000bc0 comp rx=0 tx=0).stop 2023-10-08T00:16:18.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.141+0000 7fd7d383e700 1 -- 172.21.15.99:0/770829774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7cc14d5f0 msgr2=0x7fd7cc0ae1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:18.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.141+0000 7fd7d383e700 1 --2- 172.21.15.99:0/770829774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7cc14d5f0 0x7fd7cc0ae1c0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fd7c400c910 tx=0x7fd7c40027a0 comp rx=0 tx=0).stop 2023-10-08T00:16:18.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.141+0000 7fd7d383e700 1 -- 172.21.15.99:0/770829774 shutdown_connections 2023-10-08T00:16:18.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.141+0000 7fd7d383e700 1 --2- 172.21.15.99:0/770829774 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd7b0066f30 0x7fd7b00693e0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:18.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.141+0000 7fd7d383e700 1 --2- 172.21.15.99:0/770829774 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7cc0cc9a0 0x7fd7cc099c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:18.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.141+0000 7fd7d383e700 1 --2- 172.21.15.99:0/770829774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7cc14d5f0 0x7fd7cc0ae1c0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:18.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.141+0000 7fd7d383e700 1 --2- 172.21.15.99:0/770829774 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7cc0ccd70 0x7fd7cc0ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:18.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.141+0000 7fd7d383e700 1 -- 172.21.15.99:0/770829774 >> 172.21.15.99:0/770829774 conn(0x7fd7cc0cd1e0 msgr2=0x7fd7cc0c2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:18.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.142+0000 7fd7d383e700 1 -- 172.21.15.99:0/770829774 shutdown_connections 2023-10-08T00:16:18.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.142+0000 7fd7d383e700 1 -- 172.21.15.99:0/770829774 wait complete. 2023-10-08T00:16:18.144 INFO:teuthology.orchestra.run.smithi099.stderr: removed all blocklist entries 2023-10-08T00:16:18.155 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-08T00:16:18.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.271+0000 7f120e2e9700 1 Processor -- start 2023-10-08T00:16:18.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.271+0000 7f120e2e9700 1 -- start start 2023-10-08T00:16:18.275 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.272+0000 7f120e2e9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1208157870 0x7f120815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:18.275 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.272+0000 7f120e2e9700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1208157c40 0x7f1208158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:18.275 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.272+0000 7f120e2e9700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f120815cbb0 con 0x7f120815a620 2023-10-08T00:16:18.275 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.272+0000 7f120e2e9700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f120815bfa0 con 0x7f1208157c40 2023-10-08T00:16:18.275 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.272+0000 7f120e2e9700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f120815c280 con 0x7f1208157870 2023-10-08T00:16:18.275 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.272+0000 7f120c886700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1208157c40 0x7f1208158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:18.275 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.272+0000 7f120c886700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1208157c40 0x7f1208158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35862/0 (socket says 172.21.15.99:35862) 2023-10-08T00:16:18.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.272+0000 7f120c886700 1 -- 172.21.15.99:0/2079865197 learned_addr learned my addr 172.21.15.99:0/2079865197 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:18.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.272+0000 7f1207fff700 1 --2- 172.21.15.99:0/2079865197 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1208157870 0x7f120815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:18.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.272+0000 7f120c886700 1 -- 172.21.15.99:0/2079865197 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1208157870 msgr2=0x7f120815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:18.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.272+0000 7f120c886700 1 --2- 172.21.15.99:0/2079865197 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1208157870 0x7f120815b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:18.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.272+0000 7f120c886700 1 -- 172.21.15.99:0/2079865197 >> v1:172.21.15.191:6789/0 conn(0x7f120815a620 legacy=0x7f120815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:18.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.272+0000 7f120c886700 1 -- 172.21.15.99:0/2079865197 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f120815c560 con 0x7f1208157c40 2023-10-08T00:16:18.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.272+0000 7f120c886700 1 --2- 172.21.15.99:0/2079865197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1208157c40 0x7f1208158110 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f1200009f20 tx=0x7f1200009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9402e8a845e9ead server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:18.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.272+0000 7f1206ffd700 1 -- 172.21.15.99:0/2079865197 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120000e070 con 0x7f1208157c40 2023-10-08T00:16:18.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.272+0000 7f1206ffd700 1 -- 172.21.15.99:0/2079865197 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f120000ba40 con 0x7f1208157c40 2023-10-08T00:16:18.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.272+0000 7f1206ffd700 1 -- 172.21.15.99:0/2079865197 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12000092e0 con 0x7f1208157c40 2023-10-08T00:16:18.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.273+0000 7f120e2e9700 1 -- 172.21.15.99:0/2079865197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1208157c40 msgr2=0x7f1208158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:18.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.273+0000 7f120e2e9700 1 --2- 172.21.15.99:0/2079865197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1208157c40 0x7f1208158110 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f1200009f20 tx=0x7f1200009840 comp rx=0 tx=0).stop 2023-10-08T00:16:18.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.273+0000 7f120e2e9700 1 -- 172.21.15.99:0/2079865197 shutdown_connections 2023-10-08T00:16:18.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.273+0000 7f120e2e9700 1 --2- 172.21.15.99:0/2079865197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1208157c40 0x7f1208158110 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:18.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.273+0000 7f120e2e9700 1 --2- 172.21.15.99:0/2079865197 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1208157870 0x7f120815b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:18.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.273+0000 7f120e2e9700 1 -- 172.21.15.99:0/2079865197 >> 172.21.15.99:0/2079865197 conn(0x7f1208068ce0 msgr2=0x7f12080690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:18.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.273+0000 7f120e2e9700 1 -- 172.21.15.99:0/2079865197 shutdown_connections 2023-10-08T00:16:18.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.273+0000 7f120e2e9700 1 -- 172.21.15.99:0/2079865197 wait complete. 2023-10-08T00:16:18.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.274+0000 7f120e2e9700 1 Processor -- start 2023-10-08T00:16:18.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.274+0000 7f120e2e9700 1 -- start start 2023-10-08T00:16:18.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.275+0000 7f120e2e9700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1208157870 0x7f12081507c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:18.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.275+0000 7f120e2e9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1208157c40 0x7f12080b2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:18.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.275+0000 7f120e2e9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f120815a620 0x7f12081538d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:18.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.275+0000 7f120e2e9700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1208150d00 con 0x7f1208157c40 2023-10-08T00:16:18.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.275+0000 7f120e2e9700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12080b1690 con 0x7f1208157870 2023-10-08T00:16:18.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.275+0000 7f120e2e9700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12080b24f0 con 0x7f120815a620 2023-10-08T00:16:18.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.275+0000 7f12077fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1208157c40 0x7f12080b2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:18.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.275+0000 7f12077fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1208157c40 0x7f12080b2980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35876/0 (socket says 172.21.15.99:35876) 2023-10-08T00:16:18.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.275+0000 7f12077fe700 1 -- 172.21.15.99:0/3221838409 learned_addr learned my addr 172.21.15.99:0/3221838409 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:18.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.275+0000 7f1207fff700 1 --2- 172.21.15.99:0/3221838409 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1208157870 0x7f12081507c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:18.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.275+0000 7f120c886700 1 --2- 172.21.15.99:0/3221838409 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f120815a620 0x7f12081538d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:18.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.275+0000 7f12077fe700 1 -- 172.21.15.99:0/3221838409 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f120815a620 msgr2=0x7f12081538d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:18.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.275+0000 7f12077fe700 1 --2- 172.21.15.99:0/3221838409 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f120815a620 0x7f12081538d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:18.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.275+0000 7f12077fe700 1 -- 172.21.15.99:0/3221838409 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1208157870 msgr2=0x7f12081507c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:18.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.275+0000 7f12077fe700 1 --2- 172.21.15.99:0/3221838409 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1208157870 0x7f12081507c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:18.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.275+0000 7f12077fe700 1 -- 172.21.15.99:0/3221838409 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1200009ae0 con 0x7f1208157c40 2023-10-08T00:16:18.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.275+0000 7f120c886700 1 --2- 172.21.15.99:0/3221838409 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f120815a620 0x7f12081538d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:18.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.275+0000 7f12077fe700 1 --2- 172.21.15.99:0/3221838409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1208157c40 0x7f12080b2980 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f11fc00b610 tx=0x7f11fc0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:18.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.275+0000 7f11effff700 1 -- 172.21.15.99:0/3221838409 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11fc00f510 con 0x7f1208157c40 2023-10-08T00:16:18.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.276+0000 7f11effff700 1 -- 172.21.15.99:0/3221838409 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11fc00f690 con 0x7f1208157c40 2023-10-08T00:16:18.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.276+0000 7f11effff700 1 -- 172.21.15.99:0/3221838409 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11fc00fa50 con 0x7f1208157c40 2023-10-08T00:16:18.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.276+0000 7f120e2e9700 1 -- 172.21.15.99:0/3221838409 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12080b18e0 con 0x7f1208157c40 2023-10-08T00:16:18.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.276+0000 7f120e2e9700 1 -- 172.21.15.99:0/3221838409 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12080b1ea0 con 0x7f1208157c40 2023-10-08T00:16:18.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.276+0000 7f120e2e9700 1 -- 172.21.15.99:0/3221838409 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12080b2680 con 0x7f1208157c40 2023-10-08T00:16:18.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.277+0000 7f11effff700 1 -- 172.21.15.99:0/3221838409 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f11fc01a400 con 0x7f1208157c40 2023-10-08T00:16:18.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.277+0000 7f11effff700 1 --2- 172.21.15.99:0/3221838409 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f11f4066f30 0x7f11f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:18.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.277+0000 7f11effff700 1 -- 172.21.15.99:0/3221838409 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f11fc083cd0 con 0x7f1208157c40 2023-10-08T00:16:18.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.277+0000 7f1207fff700 1 --2- 172.21.15.99:0/3221838409 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f11f4066f30 0x7f11f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:18.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.278+0000 7f1207fff700 1 --2- 172.21.15.99:0/3221838409 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f11f4066f30 0x7f11f40693e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f120814d8c0 tx=0x7f11f8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:18.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.281+0000 7f11effff700 1 -- 172.21.15.99:0/3221838409 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f11fc051310 con 0x7f1208157c40 2023-10-08T00:16:18.539 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.539+0000 7f120e2e9700 1 -- 172.21.15.99:0/3221838409 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f120804c3c0 con 0x7f1208157c40 2023-10-08T00:16:18.540 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.539+0000 7f11effff700 1 -- 172.21.15.99:0/3221838409 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f11fc054930 con 0x7f1208157c40 2023-10-08T00:16:18.540 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:16:18.540 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:16:18.540 INFO:teuthology.orchestra.run.smithi099.stdout: "auth_dump": [ 2023-10-08T00:16:18.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:18.540 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.a", 2023-10-08T00:16:18.541 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlFlN9AhAAKdbeH0XznSOuyUnReD6RsA==", 2023-10-08T00:16:18.541 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:16:18.541 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:16:18.541 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2023-10-08T00:16:18.541 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2023-10-08T00:16:18.541 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:16:18.541 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:18.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:18.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:18.541 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.b", 2023-10-08T00:16:18.541 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlh2G0DRAA1mYGrUrELHsS9abUxCCG2g==", 2023-10-08T00:16:18.541 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:16:18.542 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:16:18.542 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2023-10-08T00:16:18.542 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2023-10-08T00:16:18.542 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:16:18.542 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:18.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:18.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:18.542 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.c", 2023-10-08T00:16:18.542 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFl/M1DCRAANrfHJGZO1s0KOyCdUzZWIA==", 2023-10-08T00:16:18.542 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:16:18.542 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:16:18.542 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2023-10-08T00:16:18.543 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2023-10-08T00:16:18.543 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:16:18.543 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:18.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:18.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:18.543 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.d", 2023-10-08T00:16:18.543 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlr5F2FBAAYYeG6EM0/N0FiEAwSbYRyA==", 2023-10-08T00:16:18.543 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:16:18.543 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:16:18.543 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2023-10-08T00:16:18.543 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2023-10-08T00:16:18.543 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:16:18.544 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:18.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:18.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:18.544 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.0", 2023-10-08T00:16:18.544 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB08yFla7rpKxAAWTOutLcrjDeR69JbPZMoLg==", 2023-10-08T00:16:18.544 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:16:18.544 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:16:18.544 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:16:18.544 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:16:18.544 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:18.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:18.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:18.545 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.1", 2023-10-08T00:16:18.545 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB08yFlw+y2MhAAAvLDIB/tb49IlaireTqTFg==", 2023-10-08T00:16:18.545 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:16:18.545 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:16:18.545 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:16:18.545 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:16:18.545 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:18.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:18.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:18.545 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.2", 2023-10-08T00:16:18.546 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB18yFlh4wvABAALXOtJZoEfGmt/pOBxReooQ==", 2023-10-08T00:16:18.546 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:16:18.546 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:16:18.546 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:16:18.546 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:16:18.546 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:18.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:18.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:18.546 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.3", 2023-10-08T00:16:18.546 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB18yFl8cVfCRAAoGXTACMT+f5Avll5mxG9OA==", 2023-10-08T00:16:18.546 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:16:18.546 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:16:18.547 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:16:18.547 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:16:18.547 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:18.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:18.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:18.547 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.4", 2023-10-08T00:16:18.547 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB48yFlft9WGhAA8JZN4MjSX9wbXiMKJnfCYg==", 2023-10-08T00:16:18.547 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:16:18.547 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:16:18.547 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:16:18.547 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:16:18.547 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:18.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:18.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:18.548 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.5", 2023-10-08T00:16:18.548 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB48yFlUf1BIRAA2FMBt1jAcm4ia/vYBDyojw==", 2023-10-08T00:16:18.548 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:16:18.548 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:16:18.548 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:16:18.548 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:16:18.548 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:18.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:18.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:18.548 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.6", 2023-10-08T00:16:18.548 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB48yFl0SI6KhAAI6G+Sqfm7OQmGf9nK59x9Q==", 2023-10-08T00:16:18.549 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:16:18.549 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:16:18.549 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:16:18.549 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:16:18.549 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:18.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:18.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:18.549 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.7", 2023-10-08T00:16:18.549 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB48yFlo5xWMxAA4p9OSLK0JpWhEExsifX+3A==", 2023-10-08T00:16:18.549 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:16:18.549 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:16:18.549 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:16:18.550 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:16:18.550 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:18.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:18.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:18.550 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.0", 2023-10-08T00:16:18.550 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlmzhQGBAAReslU/Pvcsr0dwE6l1N3/Q==", 2023-10-08T00:16:18.550 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:16:18.550 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:16:18.550 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow r", 2023-10-08T00:16:18.550 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-08T00:16:18.550 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:18.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:18.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:18.551 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.1", 2023-10-08T00:16:18.551 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFl2NKsIBAAUg6rXynl1J7qB6BkUizmpA==", 2023-10-08T00:16:18.551 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:16:18.551 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:16:18.551 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2023-10-08T00:16:18.551 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2023-10-08T00:16:18.551 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2023-10-08T00:16:18.551 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:18.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:18.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:18.551 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.2", 2023-10-08T00:16:18.551 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlMp6rHRAArVgJfA3mY4n78hOMeVlw1Q==", 2023-10-08T00:16:18.552 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:16:18.552 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:16:18.552 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2023-10-08T00:16:18.552 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2023-10-08T00:16:18.552 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2023-10-08T00:16:18.552 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:18.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:18.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:18.552 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.3", 2023-10-08T00:16:18.553 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlH0gNJhAAEgGVGiabVIWknpWUohOnHA==", 2023-10-08T00:16:18.553 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:16:18.553 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:16:18.553 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2023-10-08T00:16:18.553 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2023-10-08T00:16:18.553 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2023-10-08T00:16:18.553 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:18.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:18.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:18.553 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.admin", 2023-10-08T00:16:18.553 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBw8yFlidUfIxAAIvpvkHbyCx5JFvHo/g5A9g==", 2023-10-08T00:16:18.553 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:16:18.553 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2023-10-08T00:16:18.554 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2023-10-08T00:16:18.554 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2023-10-08T00:16:18.554 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:16:18.554 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:18.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:18.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:18.554 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mgr.x", 2023-10-08T00:16:18.554 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBw8yFl2kMLOxAAsEyuLBLsjQCUS0ERDOi75A==", 2023-10-08T00:16:18.554 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:16:18.554 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2023-10-08T00:16:18.554 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile mgr", 2023-10-08T00:16:18.555 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:16:18.555 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:18.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:18.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:18.555 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mgr.y", 2023-10-08T00:16:18.555 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBw8yFl6ht5NxAAFzXnLDjklCePD6hw/pSfFg==", 2023-10-08T00:16:18.555 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:16:18.555 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2023-10-08T00:16:18.555 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile mgr", 2023-10-08T00:16:18.555 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:16:18.555 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:18.555 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:18.555 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:18.556 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:16:18.557 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.541+0000 7f120e2e9700 1 -- 172.21.15.99:0/3221838409 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f11f4066f30 msgr2=0x7f11f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:18.557 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.541+0000 7f120e2e9700 1 --2- 172.21.15.99:0/3221838409 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f11f4066f30 0x7f11f40693e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f120814d8c0 tx=0x7f11f8008040 comp rx=0 tx=0).stop 2023-10-08T00:16:18.557 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.541+0000 7f120e2e9700 1 -- 172.21.15.99:0/3221838409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1208157c40 msgr2=0x7f12080b2980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:18.557 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.541+0000 7f120e2e9700 1 --2- 172.21.15.99:0/3221838409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1208157c40 0x7f12080b2980 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f11fc00b610 tx=0x7f11fc0090f0 comp rx=0 tx=0).stop 2023-10-08T00:16:18.557 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.542+0000 7f120e2e9700 1 -- 172.21.15.99:0/3221838409 shutdown_connections 2023-10-08T00:16:18.557 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.542+0000 7f120e2e9700 1 --2- 172.21.15.99:0/3221838409 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f11f4066f30 0x7f11f40693e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:18.558 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.542+0000 7f120e2e9700 1 --2- 172.21.15.99:0/3221838409 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1208157870 0x7f12081507c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:18.558 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.542+0000 7f120e2e9700 1 --2- 172.21.15.99:0/3221838409 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f120815a620 0x7f12081538d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:18.558 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.542+0000 7f120e2e9700 1 --2- 172.21.15.99:0/3221838409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1208157c40 0x7f12080b2980 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:18.558 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.542+0000 7f120e2e9700 1 -- 172.21.15.99:0/3221838409 >> 172.21.15.99:0/3221838409 conn(0x7f1208068ce0 msgr2=0x7f12080b46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:18.558 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.542+0000 7f120e2e9700 1 -- 172.21.15.99:0/3221838409 shutdown_connections 2023-10-08T00:16:18.558 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.542+0000 7f120e2e9700 1 -- 172.21.15.99:0/3221838409 wait complete. 2023-10-08T00:16:18.558 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:16:18.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.671+0000 7f812af96700 1 Processor -- start 2023-10-08T00:16:18.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.671+0000 7f812af96700 1 -- start start 2023-10-08T00:16:18.674 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.672+0000 7f812af96700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f812415a620 0x7f812415d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:18.674 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.672+0000 7f812af96700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f812415d580 0x7f8124158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:18.674 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.672+0000 7f812af96700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f812415cbb0 con 0x7f8124157870 2023-10-08T00:16:18.674 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.672+0000 7f812af96700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f812415bdb0 con 0x7f812415a620 2023-10-08T00:16:18.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.672+0000 7f812af96700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f812415bfa0 con 0x7f812415d580 2023-10-08T00:16:18.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.672+0000 7f811bfff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f812415a620 0x7f812415d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:18.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.672+0000 7f8128d32700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f8124157870 0x7f812415b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56290/0 (socket says 172.21.15.99:56290) 2023-10-08T00:16:18.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.672+0000 7f8128d32700 1 -- 172.21.15.99:0/4116534894 learned_addr learned my addr 172.21.15.99:0/4116534894 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:18.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.672+0000 7f8129533700 1 --2- 172.21.15.99:0/4116534894 >> v2:172.21.15.191:3300/0 conn(0x7f812415d580 0x7f8124158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:18.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.672+0000 7f811bfff700 1 -- 172.21.15.99:0/4116534894 >> v2:172.21.15.191:3300/0 conn(0x7f812415d580 msgr2=0x7f8124158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:18.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.672+0000 7f811bfff700 1 --2- 172.21.15.99:0/4116534894 >> v2:172.21.15.191:3300/0 conn(0x7f812415d580 0x7f8124158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:18.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.672+0000 7f811bfff700 1 -- 172.21.15.99:0/4116534894 >> v1:172.21.15.99:6789/0 conn(0x7f8124157870 legacy=0x7f812415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:18.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.672+0000 7f811bfff700 1 -- 172.21.15.99:0/4116534894 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f812415c280 con 0x7f812415a620 2023-10-08T00:16:18.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.672+0000 7f811b7fe700 1 -- 172.21.15.99:0/4116534894 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 509450852 0 0) 0x7f812415cbb0 con 0x7f8124157870 2023-10-08T00:16:18.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.673+0000 7f811bfff700 1 --2- 172.21.15.99:0/4116534894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f812415a620 0x7f812415d040 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f812406b4e0 tx=0x7f81140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b4e6792e3ab7d56 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:18.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.673+0000 7f811b7fe700 1 -- 172.21.15.99:0/4116534894 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811400b6b0 con 0x7f812415a620 2023-10-08T00:16:18.676 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.673+0000 7f811b7fe700 1 -- 172.21.15.99:0/4116534894 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f811400b810 con 0x7f812415a620 2023-10-08T00:16:18.676 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.673+0000 7f811b7fe700 1 -- 172.21.15.99:0/4116534894 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811400bbb0 con 0x7f812415a620 2023-10-08T00:16:18.676 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.673+0000 7f812af96700 1 -- 172.21.15.99:0/4116534894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f812415a620 msgr2=0x7f812415d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:18.676 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.673+0000 7f812af96700 1 --2- 172.21.15.99:0/4116534894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f812415a620 0x7f812415d040 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f812406b4e0 tx=0x7f81140091a0 comp rx=0 tx=0).stop 2023-10-08T00:16:18.676 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.673+0000 7f812af96700 1 -- 172.21.15.99:0/4116534894 shutdown_connections 2023-10-08T00:16:18.676 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.673+0000 7f812af96700 1 --2- 172.21.15.99:0/4116534894 >> v2:172.21.15.191:3300/0 conn(0x7f812415d580 0x7f8124158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:18.676 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.673+0000 7f812af96700 1 --2- 172.21.15.99:0/4116534894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f812415a620 0x7f812415d040 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:18.676 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.673+0000 7f812af96700 1 -- 172.21.15.99:0/4116534894 >> 172.21.15.99:0/4116534894 conn(0x7f8124068ce0 msgr2=0x7f81240690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:18.676 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.673+0000 7f812af96700 1 -- 172.21.15.99:0/4116534894 shutdown_connections 2023-10-08T00:16:18.676 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.673+0000 7f812af96700 1 -- 172.21.15.99:0/4116534894 wait complete. 2023-10-08T00:16:18.679 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.674+0000 7f812af96700 1 Processor -- start 2023-10-08T00:16:18.679 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.675+0000 7f812af96700 1 -- start start 2023-10-08T00:16:18.679 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.675+0000 7f812af96700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8124157870 0x7f81240cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:18.679 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.675+0000 7f812af96700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f812415a620 0x7f812414bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:18.680 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.675+0000 7f812af96700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f812415d580 0x7f81240b5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:18.680 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.675+0000 7f812af96700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8124151030 con 0x7f812415d580 2023-10-08T00:16:18.680 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.675+0000 7f812af96700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8124151310 con 0x7f812415a620 2023-10-08T00:16:18.680 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.675+0000 7f812af96700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81241515f0 con 0x7f8124157870 2023-10-08T00:16:18.680 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.675+0000 7f8129533700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f812415d580 0x7f81240b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:18.680 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.675+0000 7f8129533700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f812415d580 0x7f81240b5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35892/0 (socket says 172.21.15.99:35892) 2023-10-08T00:16:18.680 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.675+0000 7f8129533700 1 -- 172.21.15.99:0/3351852696 learned_addr learned my addr 172.21.15.99:0/3351852696 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:18.680 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.675+0000 7f8128d32700 1 --2- 172.21.15.99:0/3351852696 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8124157870 0x7f81240cfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:18.680 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.676+0000 7f811bfff700 1 --2- 172.21.15.99:0/3351852696 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f812415a620 0x7f812414bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:18.680 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.676+0000 7f8129533700 1 -- 172.21.15.99:0/3351852696 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8124157870 msgr2=0x7f81240cfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:18.680 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.676+0000 7f8129533700 1 --2- 172.21.15.99:0/3351852696 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8124157870 0x7f81240cfc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:18.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.676+0000 7f8129533700 1 -- 172.21.15.99:0/3351852696 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f812415a620 msgr2=0x7f812414bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:18.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.676+0000 7f8129533700 1 --2- 172.21.15.99:0/3351852696 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f812415a620 0x7f812414bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:18.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.676+0000 7f8129533700 1 -- 172.21.15.99:0/3351852696 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8114009420 con 0x7f812415d580 2023-10-08T00:16:18.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.676+0000 7f8128d32700 1 --2- 172.21.15.99:0/3351852696 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8124157870 0x7f81240cfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:18.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.676+0000 7f8129533700 1 --2- 172.21.15.99:0/3351852696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f812415d580 0x7f81240b5400 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f811c00d860 tx=0x7f811c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:18.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.676+0000 7f8118ff9700 1 -- 172.21.15.99:0/3351852696 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811c0125f0 con 0x7f812415d580 2023-10-08T00:16:18.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.676+0000 7f8118ff9700 1 -- 172.21.15.99:0/3351852696 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f811c012750 con 0x7f812415d580 2023-10-08T00:16:18.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.676+0000 7f8118ff9700 1 -- 172.21.15.99:0/3351852696 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811c012b10 con 0x7f812415d580 2023-10-08T00:16:18.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.676+0000 7f812af96700 1 -- 172.21.15.99:0/3351852696 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81241518d0 con 0x7f812415d580 2023-10-08T00:16:18.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.676+0000 7f812af96700 1 -- 172.21.15.99:0/3351852696 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81240d01d0 con 0x7f812415d580 2023-10-08T00:16:18.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.677+0000 7f812af96700 1 -- 172.21.15.99:0/3351852696 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8124151df0 con 0x7f812415d580 2023-10-08T00:16:18.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.677+0000 7f8118ff9700 1 -- 172.21.15.99:0/3351852696 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f811c01e460 con 0x7f812415d580 2023-10-08T00:16:18.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.678+0000 7f8118ff9700 1 --2- 172.21.15.99:0/3351852696 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8110066f30 0x7f81100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:18.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.678+0000 7f8118ff9700 1 -- 172.21.15.99:0/3351852696 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f811c0847d0 con 0x7f812415d580 2023-10-08T00:16:18.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.678+0000 7f8128d32700 1 --2- 172.21.15.99:0/3351852696 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8110066f30 0x7f81100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:18.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.678+0000 7f8128d32700 1 --2- 172.21.15.99:0/3351852696 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8110066f30 0x7f81100693e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8120001560 tx=0x7f8120009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:18.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.682+0000 7f8118ff9700 1 -- 172.21.15.99:0/3351852696 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8124151df0 con 0x7f812415d580 2023-10-08T00:16:18.865 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.864+0000 7f812af96700 1 -- 172.21.15.99:0/3351852696 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8124151df0 con 0x7f812415d580 2023-10-08T00:16:18.865 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.865+0000 7f8118ff9700 1 -- 172.21.15.99:0/3351852696 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+12145 (secure 0 0 0) 0x7f8124151df0 con 0x7f812415d580 2023-10-08T00:16:18.866 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:16:18.866 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":120,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:16:18.137582+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:16:18.868 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.867+0000 7f812af96700 1 -- 172.21.15.99:0/3351852696 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8110066f30 msgr2=0x7f81100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:18.868 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.867+0000 7f812af96700 1 --2- 172.21.15.99:0/3351852696 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8110066f30 0x7f81100693e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8120001560 tx=0x7f8120009040 comp rx=0 tx=0).stop 2023-10-08T00:16:18.868 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.867+0000 7f812af96700 1 -- 172.21.15.99:0/3351852696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f812415d580 msgr2=0x7f81240b5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:18.869 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.867+0000 7f812af96700 1 --2- 172.21.15.99:0/3351852696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f812415d580 0x7f81240b5400 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f811c00d860 tx=0x7f811c008040 comp rx=0 tx=0).stop 2023-10-08T00:16:18.869 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.867+0000 7f812af96700 1 -- 172.21.15.99:0/3351852696 shutdown_connections 2023-10-08T00:16:18.869 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.867+0000 7f812af96700 1 --2- 172.21.15.99:0/3351852696 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8110066f30 0x7f81100693e0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:18.869 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.867+0000 7f812af96700 1 --2- 172.21.15.99:0/3351852696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f812415d580 0x7f81240b5400 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:18.869 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.867+0000 7f812af96700 1 --2- 172.21.15.99:0/3351852696 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8124157870 0x7f81240cfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:18.869 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.867+0000 7f812af96700 1 --2- 172.21.15.99:0/3351852696 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f812415a620 0x7f812414bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:18.870 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.867+0000 7f812af96700 1 -- 172.21.15.99:0/3351852696 >> 172.21.15.99:0/3351852696 conn(0x7f8124068ce0 msgr2=0x7f812415fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:18.870 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.867+0000 7f812af96700 1 -- 172.21.15.99:0/3351852696 shutdown_connections 2023-10-08T00:16:18.870 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.867+0000 7f812af96700 1 -- 172.21.15.99:0/3351852696 wait complete. 2023-10-08T00:16:18.880 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:10:42.063344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-08T00:16:18.881 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T00:16:18.999 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.998+0000 7fdeba5c9700 1 Processor -- start 2023-10-08T00:16:18.999 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.998+0000 7fdeba5c9700 1 -- start start 2023-10-08T00:16:19.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.999+0000 7fdeba5c9700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fdeb415a290 0x7fdeb406a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:19.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.999+0000 7fdeba5c9700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdeb406aae0 0x7fdeb40cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:19.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.999+0000 7fdeba5c9700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdeb40b0790 con 0x7fdeb4159ec0 2023-10-08T00:16:19.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.999+0000 7fdeba5c9700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdeb406af20 con 0x7fdeb406aae0 2023-10-08T00:16:19.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.999+0000 7fdeba5c9700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdeb40ce1f0 con 0x7fdeb415a290 2023-10-08T00:16:19.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.999+0000 7fdeb8b66700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdeb406aae0 0x7fdeb40cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:19.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.999+0000 7fdeb8b66700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdeb406aae0 0x7fdeb40cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35894/0 (socket says 172.21.15.99:35894) 2023-10-08T00:16:19.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:18.999+0000 7fdeb8b66700 1 -- 172.21.15.99:0/3687031131 learned_addr learned my addr 172.21.15.99:0/3687031131 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:19.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.000+0000 7fdeb37fe700 1 --2- 172.21.15.99:0/3687031131 >> v2:172.21.15.191:3300/0 conn(0x7fdeb415a290 0x7fdeb406a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:19.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.000+0000 7fdeb8b66700 1 -- 172.21.15.99:0/3687031131 >> v2:172.21.15.191:3300/0 conn(0x7fdeb415a290 msgr2=0x7fdeb406a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:19.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.000+0000 7fdeb8b66700 1 --2- 172.21.15.99:0/3687031131 >> v2:172.21.15.191:3300/0 conn(0x7fdeb415a290 0x7fdeb406a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:19.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.000+0000 7fdeb8b66700 1 -- 172.21.15.99:0/3687031131 >> v1:172.21.15.191:6789/0 conn(0x7fdeb4159ec0 legacy=0x7fdeb40af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:19.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.000+0000 7fdeb8b66700 1 -- 172.21.15.99:0/3687031131 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeb40b0b20 con 0x7fdeb406aae0 2023-10-08T00:16:19.004 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.000+0000 7fdeb8b66700 1 --2- 172.21.15.99:0/3687031131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdeb406aae0 0x7fdeb40cf590 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4089910 tx=0x7fdeac009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b70fe0930a84ec75 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:19.004 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.000+0000 7fdeb2ffd700 1 -- 172.21.15.99:0/3687031131 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeac00e070 con 0x7fdeb406aae0 2023-10-08T00:16:19.004 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.000+0000 7fdeb2ffd700 1 -- 172.21.15.99:0/3687031131 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeac00ba10 con 0x7fdeb406aae0 2023-10-08T00:16:19.004 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.000+0000 7fdeb2ffd700 1 -- 172.21.15.99:0/3687031131 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeac0092e0 con 0x7fdeb406aae0 2023-10-08T00:16:19.004 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.001+0000 7fdeba5c9700 1 -- 172.21.15.99:0/3687031131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdeb406aae0 msgr2=0x7fdeb40cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:19.004 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.001+0000 7fdeba5c9700 1 --2- 172.21.15.99:0/3687031131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdeb406aae0 0x7fdeb40cf590 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4089910 tx=0x7fdeac009840 comp rx=0 tx=0).stop 2023-10-08T00:16:19.004 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.001+0000 7fdeba5c9700 1 -- 172.21.15.99:0/3687031131 shutdown_connections 2023-10-08T00:16:19.004 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.001+0000 7fdeba5c9700 1 --2- 172.21.15.99:0/3687031131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdeb406aae0 0x7fdeb40cf590 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:19.005 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.001+0000 7fdeba5c9700 1 --2- 172.21.15.99:0/3687031131 >> v2:172.21.15.191:3300/0 conn(0x7fdeb415a290 0x7fdeb406a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:19.005 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.001+0000 7fdeba5c9700 1 -- 172.21.15.99:0/3687031131 >> 172.21.15.99:0/3687031131 conn(0x7fdeb40d1880 msgr2=0x7fdeb40cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:19.005 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.001+0000 7fdeba5c9700 1 -- 172.21.15.99:0/3687031131 shutdown_connections 2023-10-08T00:16:19.005 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.001+0000 7fdeba5c9700 1 -- 172.21.15.99:0/3687031131 wait complete. 2023-10-08T00:16:19.005 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.002+0000 7fdeba5c9700 1 Processor -- start 2023-10-08T00:16:19.005 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.002+0000 7fdeba5c9700 1 -- start start 2023-10-08T00:16:19.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.003+0000 7fdeba5c9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdeb406aae0 0x7fdeb41549b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:19.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.003+0000 7fdeba5c9700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdeb4159ec0 0x7fdeb40c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:19.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.003+0000 7fdeba5c9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdeb415a290 0x7fdeb40c6f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:19.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.003+0000 7fdeba5c9700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb40c8830 con 0x7fdeb406aae0 2023-10-08T00:16:19.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.003+0000 7fdeba5c9700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb40cad10 con 0x7fdeb4159ec0 2023-10-08T00:16:19.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.003+0000 7fdeba5c9700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb40c7f00 con 0x7fdeb415a290 2023-10-08T00:16:19.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.003+0000 7fdeb3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdeb406aae0 0x7fdeb41549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:19.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.003+0000 7fdeb3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdeb406aae0 0x7fdeb41549b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35900/0 (socket says 172.21.15.99:35900) 2023-10-08T00:16:19.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.003+0000 7fdeb3fff700 1 -- 172.21.15.99:0/892401437 learned_addr learned my addr 172.21.15.99:0/892401437 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:19.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.003+0000 7fdeb3fff700 1 -- 172.21.15.99:0/892401437 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdeb415a290 msgr2=0x7fdeb40c6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:19.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.003+0000 7fdeb3fff700 1 --2- 172.21.15.99:0/892401437 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdeb415a290 0x7fdeb40c6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:19.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.003+0000 7fdeb3fff700 1 -- 172.21.15.99:0/892401437 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdeb4159ec0 msgr2=0x7fdeb40c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:19.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.003+0000 7fdeb3fff700 1 --2- 172.21.15.99:0/892401437 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdeb4159ec0 0x7fdeb40c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:19.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.003+0000 7fdeb3fff700 1 -- 172.21.15.99:0/892401437 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeac009ae0 con 0x7fdeb406aae0 2023-10-08T00:16:19.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.003+0000 7fdeb3fff700 1 --2- 172.21.15.99:0/892401437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdeb406aae0 0x7fdeb41549b0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fdea400c5d0 tx=0x7fdea4007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:19.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.003+0000 7fde9bfff700 1 -- 172.21.15.99:0/892401437 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea4011590 con 0x7fdeb406aae0 2023-10-08T00:16:19.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.003+0000 7fde9bfff700 1 -- 172.21.15.99:0/892401437 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdea4011710 con 0x7fdeb406aae0 2023-10-08T00:16:19.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.003+0000 7fdeba5c9700 1 -- 172.21.15.99:0/892401437 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdeb40c81e0 con 0x7fdeb406aae0 2023-10-08T00:16:19.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.003+0000 7fde9bfff700 1 -- 172.21.15.99:0/892401437 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea4011ad0 con 0x7fdeb406aae0 2023-10-08T00:16:19.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.003+0000 7fdeba5c9700 1 -- 172.21.15.99:0/892401437 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeb4154ef0 con 0x7fdeb406aae0 2023-10-08T00:16:19.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.004+0000 7fdeba5c9700 1 -- 172.21.15.99:0/892401437 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdeb404c3c0 con 0x7fdeb406aae0 2023-10-08T00:16:19.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.005+0000 7fde9bfff700 1 -- 172.21.15.99:0/892401437 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdea401c460 con 0x7fdeb406aae0 2023-10-08T00:16:19.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.005+0000 7fde9bfff700 1 --2- 172.21.15.99:0/892401437 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdea0066f30 0x7fdea00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:19.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.005+0000 7fde9bfff700 1 -- 172.21.15.99:0/892401437 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fdea4084060 con 0x7fdeb406aae0 2023-10-08T00:16:19.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.005+0000 7fdeb37fe700 1 --2- 172.21.15.99:0/892401437 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdea0066f30 0x7fdea00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:19.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.006+0000 7fdeb37fe700 1 --2- 172.21.15.99:0/892401437 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdea0066f30 0x7fdea00693e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fdeb40ceda0 tx=0x7fdea8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:19.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.009+0000 7fde9bfff700 1 -- 172.21.15.99:0/892401437 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdea4051620 con 0x7fdeb406aae0 2023-10-08T00:16:19.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.189+0000 7fdeba5c9700 1 -- 172.21.15.99:0/892401437 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdeb40b04b0 con 0x7fdeb406aae0 2023-10-08T00:16:19.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.189+0000 7fde9bfff700 1 -- 172.21.15.99:0/892401437 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v120) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdea4054c40 con 0x7fdeb406aae0 2023-10-08T00:16:19.191 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2023-10-08T00:16:19.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.191+0000 7fdeba5c9700 1 -- 172.21.15.99:0/892401437 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdea0066f30 msgr2=0x7fdea00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:19.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.191+0000 7fdeba5c9700 1 --2- 172.21.15.99:0/892401437 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdea0066f30 0x7fdea00693e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fdeb40ceda0 tx=0x7fdea8008040 comp rx=0 tx=0).stop 2023-10-08T00:16:19.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.191+0000 7fdeba5c9700 1 -- 172.21.15.99:0/892401437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdeb406aae0 msgr2=0x7fdeb41549b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:19.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.191+0000 7fdeba5c9700 1 --2- 172.21.15.99:0/892401437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdeb406aae0 0x7fdeb41549b0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fdea400c5d0 tx=0x7fdea4007f60 comp rx=0 tx=0).stop 2023-10-08T00:16:19.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.191+0000 7fdeba5c9700 1 -- 172.21.15.99:0/892401437 shutdown_connections 2023-10-08T00:16:19.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.191+0000 7fdeba5c9700 1 --2- 172.21.15.99:0/892401437 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdea0066f30 0x7fdea00693e0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:19.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.191+0000 7fdeba5c9700 1 --2- 172.21.15.99:0/892401437 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdeb415a290 0x7fdeb40c6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:19.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.191+0000 7fdeba5c9700 1 --2- 172.21.15.99:0/892401437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdeb406aae0 0x7fdeb41549b0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:19.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.191+0000 7fdeba5c9700 1 --2- 172.21.15.99:0/892401437 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdeb4159ec0 0x7fdeb40c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:19.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.191+0000 7fdeba5c9700 1 -- 172.21.15.99:0/892401437 >> 172.21.15.99:0/892401437 conn(0x7fdeb40d1880 msgr2=0x7fdeb41557d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:19.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.191+0000 7fdeba5c9700 1 -- 172.21.15.99:0/892401437 shutdown_connections 2023-10-08T00:16:19.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.191+0000 7fdeba5c9700 1 -- 172.21.15.99:0/892401437 wait complete. 2023-10-08T00:16:19.204 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T00:16:19.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.319+0000 7fea648f4700 1 Processor -- start 2023-10-08T00:16:19.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.319+0000 7fea648f4700 1 -- start start 2023-10-08T00:16:19.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.320+0000 7fea648f4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea600cd6e0 0x7fea600cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:19.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.320+0000 7fea648f4700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fea600cdab0 0x7fea6015d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:19.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.320+0000 7fea648f4700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fea6015dda0 0x7fea6015cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:19.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.320+0000 7fea648f4700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea6015e850 con 0x7fea600cdab0 2023-10-08T00:16:19.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.320+0000 7fea648f4700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea600cd030 con 0x7fea6015dda0 2023-10-08T00:16:19.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.320+0000 7fea648f4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea600cb7c0 con 0x7fea600cd6e0 2023-10-08T00:16:19.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.320+0000 7fea5effd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fea600cdab0 0x7fea6015d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:19.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.320+0000 7fea5effd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fea600cdab0 0x7fea6015d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35906/0 (socket says 172.21.15.99:35906) 2023-10-08T00:16:19.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.320+0000 7fea5effd700 1 -- 172.21.15.99:0/745292516 learned_addr learned my addr 172.21.15.99:0/745292516 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:19.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.320+0000 7fea5f7fe700 1 --2- 172.21.15.99:0/745292516 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea600cd6e0 0x7fea600cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:19.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.320+0000 7fea5ffff700 1 --2- 172.21.15.99:0/745292516 >> v2:172.21.15.191:3300/0 conn(0x7fea6015dda0 0x7fea6015cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:19.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.320+0000 7fea5effd700 1 -- 172.21.15.99:0/745292516 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea600cd6e0 msgr2=0x7fea600cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:19.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.320+0000 7fea5effd700 1 --2- 172.21.15.99:0/745292516 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea600cd6e0 0x7fea600cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:19.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.320+0000 7fea5effd700 1 -- 172.21.15.99:0/745292516 >> v2:172.21.15.191:3300/0 conn(0x7fea6015dda0 msgr2=0x7fea6015cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:19.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.320+0000 7fea5effd700 1 --2- 172.21.15.99:0/745292516 >> v2:172.21.15.191:3300/0 conn(0x7fea6015dda0 0x7fea6015cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:19.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.320+0000 7fea5effd700 1 -- 172.21.15.99:0/745292516 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea6015e290 con 0x7fea600cdab0 2023-10-08T00:16:19.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.320+0000 7fea5ffff700 1 --2- 172.21.15.99:0/745292516 >> v2:172.21.15.191:3300/0 conn(0x7fea6015dda0 0x7fea6015cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:19.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.321+0000 7fea5effd700 1 --2- 172.21.15.99:0/745292516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea600cdab0 0x7fea6015d860 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fea60089ac0 tx=0x7fea54009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ba4df1a5bd67f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:19.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.321+0000 7fea5e7fc700 1 -- 172.21.15.99:0/745292516 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea54009e70 con 0x7fea600cdab0 2023-10-08T00:16:19.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.321+0000 7fea5e7fc700 1 -- 172.21.15.99:0/745292516 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea5400b860 con 0x7fea600cdab0 2023-10-08T00:16:19.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.321+0000 7fea5e7fc700 1 -- 172.21.15.99:0/745292516 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5400bc00 con 0x7fea600cdab0 2023-10-08T00:16:19.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.321+0000 7fea648f4700 1 -- 172.21.15.99:0/745292516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea600cdab0 msgr2=0x7fea6015d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:19.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.321+0000 7fea648f4700 1 --2- 172.21.15.99:0/745292516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea600cdab0 0x7fea6015d860 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fea60089ac0 tx=0x7fea54009250 comp rx=0 tx=0).stop 2023-10-08T00:16:19.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.321+0000 7fea648f4700 1 -- 172.21.15.99:0/745292516 shutdown_connections 2023-10-08T00:16:19.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.321+0000 7fea648f4700 1 --2- 172.21.15.99:0/745292516 >> v2:172.21.15.191:3300/0 conn(0x7fea6015dda0 0x7fea6015cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:19.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.321+0000 7fea648f4700 1 --2- 172.21.15.99:0/745292516 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea600cd6e0 0x7fea600cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:19.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.321+0000 7fea648f4700 1 --2- 172.21.15.99:0/745292516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea600cdab0 0x7fea6015d860 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:19.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.321+0000 7fea648f4700 1 -- 172.21.15.99:0/745292516 >> 172.21.15.99:0/745292516 conn(0x7fea60067660 msgr2=0x7fea60067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:19.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.321+0000 7fea648f4700 1 -- 172.21.15.99:0/745292516 shutdown_connections 2023-10-08T00:16:19.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.321+0000 7fea648f4700 1 -- 172.21.15.99:0/745292516 wait complete. 2023-10-08T00:16:19.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.323+0000 7fea648f4700 1 Processor -- start 2023-10-08T00:16:19.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.323+0000 7fea648f4700 1 -- start start 2023-10-08T00:16:19.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.323+0000 7fea648f4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea600cd6e0 0x7fea60153660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:19.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.323+0000 7fea648f4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea600cdab0 0x7fea6016e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:19.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.323+0000 7fea648f4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea6015dda0 0x7fea6014c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:19.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.323+0000 7fea648f4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6016ea60 con 0x7fea6015dda0 2023-10-08T00:16:19.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.323+0000 7fea648f4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea60150d90 con 0x7fea600cd6e0 2023-10-08T00:16:19.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.323+0000 7fea648f4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea600c7dc0 con 0x7fea600cdab0 2023-10-08T00:16:19.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.323+0000 7fea5ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea6015dda0 0x7fea6014c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:19.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.323+0000 7fea5ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea6015dda0 0x7fea6014c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35918/0 (socket says 172.21.15.99:35918) 2023-10-08T00:16:19.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.324+0000 7fea5ffff700 1 -- 172.21.15.99:0/2942199515 learned_addr learned my addr 172.21.15.99:0/2942199515 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:19.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.324+0000 7fea5effd700 1 --2- 172.21.15.99:0/2942199515 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea600cdab0 0x7fea6016e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:19.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.324+0000 7fea5f7fe700 1 --2- 172.21.15.99:0/2942199515 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea600cd6e0 0x7fea60153660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:19.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.324+0000 7fea5ffff700 1 -- 172.21.15.99:0/2942199515 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea600cdab0 msgr2=0x7fea6016e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:19.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.324+0000 7fea5ffff700 1 --2- 172.21.15.99:0/2942199515 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea600cdab0 0x7fea6016e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:19.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.324+0000 7fea5ffff700 1 -- 172.21.15.99:0/2942199515 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea600cd6e0 msgr2=0x7fea60153660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:19.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.324+0000 7fea5ffff700 1 --2- 172.21.15.99:0/2942199515 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea600cd6e0 0x7fea60153660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:19.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.324+0000 7fea5ffff700 1 -- 172.21.15.99:0/2942199515 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea540094f0 con 0x7fea6015dda0 2023-10-08T00:16:19.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.324+0000 7fea5effd700 1 --2- 172.21.15.99:0/2942199515 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea600cdab0 0x7fea6016e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:19.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.324+0000 7fea5ffff700 1 --2- 172.21.15.99:0/2942199515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea6015dda0 0x7fea6014c880 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fea5800e870 tx=0x7fea58008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:19.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.324+0000 7fea477fe700 1 -- 172.21.15.99:0/2942199515 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5800c430 con 0x7fea6015dda0 2023-10-08T00:16:19.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.324+0000 7fea477fe700 1 -- 172.21.15.99:0/2942199515 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea5800c590 con 0x7fea6015dda0 2023-10-08T00:16:19.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.324+0000 7fea648f4700 1 -- 172.21.15.99:0/2942199515 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea600c80a0 con 0x7fea6015dda0 2023-10-08T00:16:19.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.324+0000 7fea477fe700 1 -- 172.21.15.99:0/2942199515 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5800c910 con 0x7fea6015dda0 2023-10-08T00:16:19.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.324+0000 7fea648f4700 1 -- 172.21.15.99:0/2942199515 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea600c8660 con 0x7fea6015dda0 2023-10-08T00:16:19.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.325+0000 7fea648f4700 1 -- 172.21.15.99:0/2942199515 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea60089280 con 0x7fea6015dda0 2023-10-08T00:16:19.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.326+0000 7fea477fe700 1 -- 172.21.15.99:0/2942199515 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fea5800cd40 con 0x7fea6015dda0 2023-10-08T00:16:19.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.326+0000 7fea477fe700 1 --2- 172.21.15.99:0/2942199515 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fea4c066f30 0x7fea4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:19.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.326+0000 7fea477fe700 1 -- 172.21.15.99:0/2942199515 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fea58083f50 con 0x7fea6015dda0 2023-10-08T00:16:19.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.326+0000 7fea5f7fe700 1 --2- 172.21.15.99:0/2942199515 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fea4c066f30 0x7fea4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:19.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.326+0000 7fea5f7fe700 1 --2- 172.21.15.99:0/2942199515 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fea4c066f30 0x7fea4c0693e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fea50007600 tx=0x7fea5000b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:19.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.330+0000 7fea477fe700 1 -- 172.21.15.99:0/2942199515 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fea58051510 con 0x7fea6015dda0 2023-10-08T00:16:19.513 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.512+0000 7fea648f4700 1 -- 172.21.15.99:0/2942199515 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fea6015e570 con 0x7fea6015dda0 2023-10-08T00:16:19.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.513+0000 7fea477fe700 1 -- 172.21.15.99:0/2942199515 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v120) v1 ==== 84+0+65 (secure 0 0 0) 0x7fea58054b30 con 0x7fea6015dda0 2023-10-08T00:16:19.517 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:16:19.518 INFO:teuthology.orchestra.run.smithi099.stdout:[ 2023-10-08T00:16:19.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:19.518 INFO:teuthology.orchestra.run.smithi099.stdout: "poolnum": 1, 2023-10-08T00:16:19.518 INFO:teuthology.orchestra.run.smithi099.stdout: "poolname": ".mgr" 2023-10-08T00:16:19.518 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:19.518 INFO:teuthology.orchestra.run.smithi099.stdout:] 2023-10-08T00:16:19.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.514+0000 7fea648f4700 1 -- 172.21.15.99:0/2942199515 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fea4c066f30 msgr2=0x7fea4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:19.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.514+0000 7fea648f4700 1 --2- 172.21.15.99:0/2942199515 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fea4c066f30 0x7fea4c0693e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fea50007600 tx=0x7fea5000b040 comp rx=0 tx=0).stop 2023-10-08T00:16:19.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.514+0000 7fea648f4700 1 -- 172.21.15.99:0/2942199515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea6015dda0 msgr2=0x7fea6014c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:19.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.514+0000 7fea648f4700 1 --2- 172.21.15.99:0/2942199515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea6015dda0 0x7fea6014c880 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fea5800e870 tx=0x7fea58008040 comp rx=0 tx=0).stop 2023-10-08T00:16:19.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.514+0000 7fea648f4700 1 -- 172.21.15.99:0/2942199515 shutdown_connections 2023-10-08T00:16:19.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.514+0000 7fea648f4700 1 --2- 172.21.15.99:0/2942199515 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fea4c066f30 0x7fea4c0693e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:19.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.514+0000 7fea648f4700 1 --2- 172.21.15.99:0/2942199515 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea600cd6e0 0x7fea60153660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:19.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.514+0000 7fea648f4700 1 --2- 172.21.15.99:0/2942199515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea6015dda0 0x7fea6014c880 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:19.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.514+0000 7fea648f4700 1 --2- 172.21.15.99:0/2942199515 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea600cdab0 0x7fea6016e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:19.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.514+0000 7fea648f4700 1 -- 172.21.15.99:0/2942199515 >> 172.21.15.99:0/2942199515 conn(0x7fea60067660 msgr2=0x7fea60156740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:19.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.515+0000 7fea648f4700 1 -- 172.21.15.99:0/2942199515 shutdown_connections 2023-10-08T00:16:19.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.515+0000 7fea648f4700 1 -- 172.21.15.99:0/2942199515 wait complete. 2023-10-08T00:16:19.527 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T00:16:19.527 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T00:16:19.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.645+0000 7f2dda27a700 1 Processor -- start 2023-10-08T00:16:19.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.645+0000 7f2dda27a700 1 -- start start 2023-10-08T00:16:19.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.646+0000 7f2dda27a700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2dd415d240 0x7f2dd40b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:19.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.646+0000 7f2dda27a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2dd40ce810 0x7f2dd40cec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:19.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.646+0000 7f2dda27a700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dd40cd4f0 con 0x7f2dd40cc940 2023-10-08T00:16:19.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.646+0000 7f2dda27a700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dd40b3f40 con 0x7f2dd415d240 2023-10-08T00:16:19.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.646+0000 7f2dda27a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd40cbb80 con 0x7f2dd40ce810 2023-10-08T00:16:19.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.646+0000 7f2dd9278700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2dd415d240 0x7f2dd40b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:19.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.646+0000 7f2dd8a77700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f2dd40cc940 0x7f2dd40ccde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56304/0 (socket says 172.21.15.99:56304) 2023-10-08T00:16:19.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.646+0000 7f2dd8a77700 1 -- 172.21.15.99:0/1821522397 learned_addr learned my addr 172.21.15.99:0/1821522397 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:19.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.646+0000 7f2dd9a79700 1 --2- 172.21.15.99:0/1821522397 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2dd40ce810 0x7f2dd40cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:19.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.646+0000 7f2dd9278700 1 -- 172.21.15.99:0/1821522397 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2dd40ce810 msgr2=0x7f2dd40cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:19.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.646+0000 7f2dd9278700 1 --2- 172.21.15.99:0/1821522397 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2dd40ce810 0x7f2dd40cec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:19.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.646+0000 7f2dd9278700 1 -- 172.21.15.99:0/1821522397 >> v1:172.21.15.99:6789/0 conn(0x7f2dd40cc940 legacy=0x7f2dd40ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:19.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.646+0000 7f2dd9278700 1 -- 172.21.15.99:0/1821522397 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dd40cbe60 con 0x7f2dd415d240 2023-10-08T00:16:19.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.646+0000 7f2dd9a79700 1 --2- 172.21.15.99:0/1821522397 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2dd40ce810 0x7f2dd40cec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:16:19.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.646+0000 7f2dcbfff700 1 -- 172.21.15.99:0/1821522397 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2550875008 0 0) 0x7f2dd40cd4f0 con 0x7f2dd40cc940 2023-10-08T00:16:19.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.646+0000 7f2dd9278700 1 --2- 172.21.15.99:0/1821522397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2dd415d240 0x7f2dd40b3a00 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f2dd00098d0 tx=0x7f2dd0009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a7a0a6d5a9e907 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:19.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.646+0000 7f2dcbfff700 1 -- 172.21.15.99:0/1821522397 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd0009e70 con 0x7f2dd415d240 2023-10-08T00:16:19.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.646+0000 7f2dcbfff700 1 -- 172.21.15.99:0/1821522397 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dd0005c10 con 0x7f2dd415d240 2023-10-08T00:16:19.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.646+0000 7f2dcbfff700 1 -- 172.21.15.99:0/1821522397 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd000b650 con 0x7f2dd415d240 2023-10-08T00:16:19.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.647+0000 7f2dda27a700 1 -- 172.21.15.99:0/1821522397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2dd415d240 msgr2=0x7f2dd40b3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:19.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.647+0000 7f2dda27a700 1 --2- 172.21.15.99:0/1821522397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2dd415d240 0x7f2dd40b3a00 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f2dd00098d0 tx=0x7f2dd0009120 comp rx=0 tx=0).stop 2023-10-08T00:16:19.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.647+0000 7f2dda27a700 1 -- 172.21.15.99:0/1821522397 shutdown_connections 2023-10-08T00:16:19.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.647+0000 7f2dda27a700 1 --2- 172.21.15.99:0/1821522397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2dd415d240 0x7f2dd40b3a00 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:19.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.647+0000 7f2dda27a700 1 --2- 172.21.15.99:0/1821522397 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2dd40ce810 0x7f2dd40cec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:19.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.647+0000 7f2dda27a700 1 -- 172.21.15.99:0/1821522397 >> 172.21.15.99:0/1821522397 conn(0x7f2dd4067e40 msgr2=0x7f2dd40d0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:19.651 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.647+0000 7f2dda27a700 1 -- 172.21.15.99:0/1821522397 shutdown_connections 2023-10-08T00:16:19.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.647+0000 7f2dda27a700 1 -- 172.21.15.99:0/1821522397 wait complete. 2023-10-08T00:16:19.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.648+0000 7f2dda27a700 1 Processor -- start 2023-10-08T00:16:19.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.648+0000 7f2dda27a700 1 -- start start 2023-10-08T00:16:19.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.649+0000 7f2dda27a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2dd40cc940 0x7f2dd414a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:19.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.649+0000 7f2dda27a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2dd40ce810 0x7f2dd40ca930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:19.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.649+0000 7f2dda27a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2dd415d240 0x7f2dd415c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:19.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.649+0000 7f2dda27a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd40c9640 con 0x7f2dd415d240 2023-10-08T00:16:19.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.649+0000 7f2dda27a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd40ca4a0 con 0x7f2dd40ce810 2023-10-08T00:16:19.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.649+0000 7f2dda27a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd414ac30 con 0x7f2dd40cc940 2023-10-08T00:16:19.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.649+0000 7f2dd9a79700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2dd415d240 0x7f2dd415c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:19.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.649+0000 7f2dd9a79700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2dd415d240 0x7f2dd415c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35930/0 (socket says 172.21.15.99:35930) 2023-10-08T00:16:19.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.649+0000 7f2dd9a79700 1 -- 172.21.15.99:0/2441736782 learned_addr learned my addr 172.21.15.99:0/2441736782 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:19.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.649+0000 7f2dd9278700 1 --2- 172.21.15.99:0/2441736782 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2dd40cc940 0x7f2dd414a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:19.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.649+0000 7f2dd8a77700 1 --2- 172.21.15.99:0/2441736782 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2dd40ce810 0x7f2dd40ca930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:19.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.650+0000 7f2dd9a79700 1 -- 172.21.15.99:0/2441736782 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2dd40cc940 msgr2=0x7f2dd414a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:19.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.650+0000 7f2dd9a79700 1 --2- 172.21.15.99:0/2441736782 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2dd40cc940 0x7f2dd414a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:19.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.650+0000 7f2dd9a79700 1 -- 172.21.15.99:0/2441736782 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2dd40ce810 msgr2=0x7f2dd40ca930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:19.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.650+0000 7f2dd9a79700 1 --2- 172.21.15.99:0/2441736782 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2dd40ce810 0x7f2dd40ca930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:19.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.650+0000 7f2dd9a79700 1 -- 172.21.15.99:0/2441736782 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dd00093c0 con 0x7f2dd415d240 2023-10-08T00:16:19.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.650+0000 7f2dd9a79700 1 --2- 172.21.15.99:0/2441736782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2dd415d240 0x7f2dd415c340 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc00cef0 tx=0x7f2dcc007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:19.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.650+0000 7f2dc97fa700 1 -- 172.21.15.99:0/2441736782 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc00ad20 con 0x7f2dd415d240 2023-10-08T00:16:19.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.650+0000 7f2dc97fa700 1 -- 172.21.15.99:0/2441736782 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dcc016420 con 0x7f2dd415d240 2023-10-08T00:16:19.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.650+0000 7f2dda27a700 1 -- 172.21.15.99:0/2441736782 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dd40c9890 con 0x7f2dd415d240 2023-10-08T00:16:19.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.650+0000 7f2dc97fa700 1 -- 172.21.15.99:0/2441736782 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc00aea0 con 0x7f2dd415d240 2023-10-08T00:16:19.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.650+0000 7f2dda27a700 1 -- 172.21.15.99:0/2441736782 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dd40c9e50 con 0x7f2dd415d240 2023-10-08T00:16:19.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.651+0000 7f2dda27a700 1 -- 172.21.15.99:0/2441736782 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dd40ca610 con 0x7f2dd415d240 2023-10-08T00:16:19.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.651+0000 7f2dc97fa700 1 -- 172.21.15.99:0/2441736782 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2dcc016a70 con 0x7f2dd415d240 2023-10-08T00:16:19.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.652+0000 7f2dc97fa700 1 --2- 172.21.15.99:0/2441736782 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2db8066f30 0x7f2db80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:19.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.652+0000 7f2dc97fa700 1 -- 172.21.15.99:0/2441736782 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f2dcc012070 con 0x7f2dd415d240 2023-10-08T00:16:19.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.652+0000 7f2dd9278700 1 --2- 172.21.15.99:0/2441736782 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2db8066f30 0x7f2db80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:19.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.652+0000 7f2dd9278700 1 --2- 172.21.15.99:0/2441736782 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2db8066f30 0x7f2db80693e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0009160 tx=0x7f2dd0016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:19.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.656+0000 7f2dc97fa700 1 -- 172.21.15.99:0/2441736782 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2dcc04e5a0 con 0x7f2dd415d240 2023-10-08T00:16:19.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:19.838+0000 7f2dda27a700 1 -- 172.21.15.99:0/2441736782 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f2dd406ac60 con 0x7f2dd415d240 2023-10-08T00:16:20.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.145+0000 7f2dc97fa700 1 -- 172.21.15.99:0/2441736782 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v121) v1 ==== 138+0+0 (secure 0 0 0) 0x7f2dcc051fc0 con 0x7f2dd415d240 2023-10-08T00:16:20.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.149+0000 7f2dda27a700 1 -- 172.21.15.99:0/2441736782 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2db8066f30 msgr2=0x7f2db80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:20.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.149+0000 7f2dda27a700 1 --2- 172.21.15.99:0/2441736782 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2db8066f30 0x7f2db80693e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0009160 tx=0x7f2dd0016000 comp rx=0 tx=0).stop 2023-10-08T00:16:20.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.149+0000 7f2dda27a700 1 -- 172.21.15.99:0/2441736782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2dd415d240 msgr2=0x7f2dd415c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:20.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.149+0000 7f2dda27a700 1 --2- 172.21.15.99:0/2441736782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2dd415d240 0x7f2dd415c340 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc00cef0 tx=0x7f2dcc007860 comp rx=0 tx=0).stop 2023-10-08T00:16:20.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.149+0000 7f2dda27a700 1 -- 172.21.15.99:0/2441736782 shutdown_connections 2023-10-08T00:16:20.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.149+0000 7f2dda27a700 1 --2- 172.21.15.99:0/2441736782 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2db8066f30 0x7f2db80693e0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:20.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.149+0000 7f2dda27a700 1 --2- 172.21.15.99:0/2441736782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2dd415d240 0x7f2dd415c340 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:20.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.149+0000 7f2dda27a700 1 --2- 172.21.15.99:0/2441736782 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2dd40cc940 0x7f2dd414a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:20.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.149+0000 7f2dda27a700 1 --2- 172.21.15.99:0/2441736782 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2dd40ce810 0x7f2dd40ca930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:20.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.149+0000 7f2dda27a700 1 -- 172.21.15.99:0/2441736782 >> 172.21.15.99:0/2441736782 conn(0x7f2dd4067e40 msgr2=0x7f2dd415d610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:20.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.149+0000 7f2dda27a700 1 -- 172.21.15.99:0/2441736782 shutdown_connections 2023-10-08T00:16:20.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.149+0000 7f2dda27a700 1 -- 172.21.15.99:0/2441736782 wait complete. 2023-10-08T00:16:20.151 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' created 2023-10-08T00:16:20.162 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T00:16:20.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.283+0000 7fb2c23b4700 1 Processor -- start 2023-10-08T00:16:20.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.283+0000 7fb2c23b4700 1 -- start start 2023-10-08T00:16:20.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.284+0000 7fb2c23b4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2bc152300 0x7fb2bc15b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:20.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.284+0000 7fb2c23b4700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb2bc06d370 0x7fb2bc06d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:20.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.284+0000 7fb2c23b4700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2bc15c8d0 con 0x7fb2bc158df0 2023-10-08T00:16:20.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.284+0000 7fb2c23b4700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2bc0b1400 con 0x7fb2bc06d370 2023-10-08T00:16:20.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.284+0000 7fb2c23b4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc0c9370 con 0x7fb2bc152300 2023-10-08T00:16:20.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.284+0000 7fb2c0951700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb2bc06d370 0x7fb2bc06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:20.287 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.284+0000 7fb2bb7fe700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fb2bc158df0 0x7fb2bc0caaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56316/0 (socket says 172.21.15.99:56316) 2023-10-08T00:16:20.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.284+0000 7fb2bb7fe700 1 -- 172.21.15.99:0/2684583989 learned_addr learned my addr 172.21.15.99:0/2684583989 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:20.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.284+0000 7fb2bbfff700 1 --2- 172.21.15.99:0/2684583989 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2bc152300 0x7fb2bc15b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:20.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.284+0000 7fb2c0951700 1 -- 172.21.15.99:0/2684583989 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2bc152300 msgr2=0x7fb2bc15b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:20.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.284+0000 7fb2c0951700 1 --2- 172.21.15.99:0/2684583989 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2bc152300 0x7fb2bc15b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:20.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.284+0000 7fb2c0951700 1 -- 172.21.15.99:0/2684583989 >> v1:172.21.15.99:6789/0 conn(0x7fb2bc158df0 legacy=0x7fb2bc0caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:20.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.284+0000 7fb2c0951700 1 -- 172.21.15.99:0/2684583989 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2bc15c030 con 0x7fb2bc06d370 2023-10-08T00:16:20.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.284+0000 7fb2c0951700 1 --2- 172.21.15.99:0/2684583989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2bc06d370 0x7fb2bc06d790 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4009e60 tx=0x7fb2b4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9c494f9eb12e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:20.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.284+0000 7fb2baffd700 1 -- 172.21.15.99:0/2684583989 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b400e070 con 0x7fb2bc06d370 2023-10-08T00:16:20.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.284+0000 7fb2baffd700 1 -- 172.21.15.99:0/2684583989 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2b400bc30 con 0x7fb2bc06d370 2023-10-08T00:16:20.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.284+0000 7fb2baffd700 1 -- 172.21.15.99:0/2684583989 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b4012430 con 0x7fb2bc06d370 2023-10-08T00:16:20.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.285+0000 7fb2c23b4700 1 -- 172.21.15.99:0/2684583989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2bc06d370 msgr2=0x7fb2bc06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:20.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.285+0000 7fb2c23b4700 1 --2- 172.21.15.99:0/2684583989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2bc06d370 0x7fb2bc06d790 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4009e60 tx=0x7fb2b4009780 comp rx=0 tx=0).stop 2023-10-08T00:16:20.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.285+0000 7fb2c23b4700 1 -- 172.21.15.99:0/2684583989 shutdown_connections 2023-10-08T00:16:20.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.285+0000 7fb2c23b4700 1 --2- 172.21.15.99:0/2684583989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2bc06d370 0x7fb2bc06d790 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:20.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.285+0000 7fb2c23b4700 1 --2- 172.21.15.99:0/2684583989 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2bc152300 0x7fb2bc15b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:20.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.285+0000 7fb2c23b4700 1 -- 172.21.15.99:0/2684583989 >> 172.21.15.99:0/2684583989 conn(0x7fb2bc069870 msgr2=0x7fb2bc069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:20.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.285+0000 7fb2c23b4700 1 -- 172.21.15.99:0/2684583989 shutdown_connections 2023-10-08T00:16:20.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.285+0000 7fb2c23b4700 1 -- 172.21.15.99:0/2684583989 wait complete. 2023-10-08T00:16:20.295 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.286+0000 7fb2c23b4700 1 Processor -- start 2023-10-08T00:16:20.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.286+0000 7fb2c23b4700 1 -- start start 2023-10-08T00:16:20.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.287+0000 7fb2c23b4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2bc06d370 0x7fb2bc0c8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:20.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.287+0000 7fb2c23b4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2bc152300 0x7fb2bc1516f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:20.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.287+0000 7fb2c23b4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2bc158df0 0x7fb2bc1530e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:20.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.287+0000 7fb2c23b4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc151c30 con 0x7fb2bc158df0 2023-10-08T00:16:20.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.287+0000 7fb2c23b4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc153620 con 0x7fb2bc06d370 2023-10-08T00:16:20.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.287+0000 7fb2c23b4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc153850 con 0x7fb2bc152300 2023-10-08T00:16:20.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.287+0000 7fb2c0951700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2bc158df0 0x7fb2bc1530e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:20.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.287+0000 7fb2c0951700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2bc158df0 0x7fb2bc1530e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35948/0 (socket says 172.21.15.99:35948) 2023-10-08T00:16:20.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.287+0000 7fb2c0951700 1 -- 172.21.15.99:0/1457572932 learned_addr learned my addr 172.21.15.99:0/1457572932 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:20.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.287+0000 7fb2bb7fe700 1 --2- 172.21.15.99:0/1457572932 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2bc152300 0x7fb2bc1516f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:20.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.287+0000 7fb2bbfff700 1 --2- 172.21.15.99:0/1457572932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2bc06d370 0x7fb2bc0c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:20.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.287+0000 7fb2c0951700 1 -- 172.21.15.99:0/1457572932 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2bc152300 msgr2=0x7fb2bc1516f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:20.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.287+0000 7fb2c0951700 1 --2- 172.21.15.99:0/1457572932 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2bc152300 0x7fb2bc1516f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:20.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.287+0000 7fb2c0951700 1 -- 172.21.15.99:0/1457572932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2bc06d370 msgr2=0x7fb2bc0c8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:20.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.287+0000 7fb2c0951700 1 --2- 172.21.15.99:0/1457572932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2bc06d370 0x7fb2bc0c8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:20.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.287+0000 7fb2c0951700 1 -- 172.21.15.99:0/1457572932 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b4009a20 con 0x7fb2bc158df0 2023-10-08T00:16:20.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.287+0000 7fb2bbfff700 1 --2- 172.21.15.99:0/1457572932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2bc06d370 0x7fb2bc0c8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:20.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.287+0000 7fb2c0951700 1 --2- 172.21.15.99:0/1457572932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2bc158df0 0x7fb2bc1530e0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4013000 tx=0x7fb2b4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:20.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.287+0000 7fb2a3fff700 1 -- 172.21.15.99:0/1457572932 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b4027470 con 0x7fb2bc158df0 2023-10-08T00:16:20.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.288+0000 7fb2a3fff700 1 -- 172.21.15.99:0/1457572932 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2b40275f0 con 0x7fb2bc158df0 2023-10-08T00:16:20.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.288+0000 7fb2a3fff700 1 -- 172.21.15.99:0/1457572932 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b40279b0 con 0x7fb2bc158df0 2023-10-08T00:16:20.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.288+0000 7fb2c23b4700 1 -- 172.21.15.99:0/1457572932 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2bc153b30 con 0x7fb2bc158df0 2023-10-08T00:16:20.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.288+0000 7fb2c23b4700 1 -- 172.21.15.99:0/1457572932 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2bc1540f0 con 0x7fb2bc158df0 2023-10-08T00:16:20.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.289+0000 7fb2c23b4700 1 -- 172.21.15.99:0/1457572932 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2bc04c3c0 con 0x7fb2bc158df0 2023-10-08T00:16:20.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.289+0000 7fb2a3fff700 1 -- 172.21.15.99:0/1457572932 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb2b4024070 con 0x7fb2bc158df0 2023-10-08T00:16:20.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.289+0000 7fb2a3fff700 1 --2- 172.21.15.99:0/1457572932 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb2a8066f30 0x7fb2a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:20.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.289+0000 7fb2a3fff700 1 -- 172.21.15.99:0/1457572932 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5223+0+0 (secure 0 0 0) 0x7fb2b4086e50 con 0x7fb2bc158df0 2023-10-08T00:16:20.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.289+0000 7fb2bbfff700 1 --2- 172.21.15.99:0/1457572932 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb2a8066f30 0x7fb2a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:20.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.290+0000 7fb2bbfff700 1 --2- 172.21.15.99:0/1457572932 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb2a8066f30 0x7fb2a80693e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac006fd0 tx=0x7fb2ac008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:20.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.294+0000 7fb2a3fff700 1 -- 172.21.15.99:0/1457572932 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb2b40542a0 con 0x7fb2bc158df0 2023-10-08T00:16:20.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:20.476+0000 7fb2c23b4700 1 -- 172.21.15.99:0/1457572932 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb2bc0cdf10 con 0x7fb2bc158df0 2023-10-08T00:16:21.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.150+0000 7fb2a3fff700 1 -- 172.21.15.99:0/1457572932 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v122) v1 ==== 170+0+0 (secure 0 0 0) 0x7fb2b402c0d0 con 0x7fb2bc158df0 2023-10-08T00:16:21.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.154+0000 7fb2c23b4700 1 -- 172.21.15.99:0/1457572932 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb2a8066f30 msgr2=0x7fb2a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:21.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.154+0000 7fb2c23b4700 1 --2- 172.21.15.99:0/1457572932 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb2a8066f30 0x7fb2a80693e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac006fd0 tx=0x7fb2ac008040 comp rx=0 tx=0).stop 2023-10-08T00:16:21.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.154+0000 7fb2c23b4700 1 -- 172.21.15.99:0/1457572932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2bc158df0 msgr2=0x7fb2bc1530e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:21.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.154+0000 7fb2c23b4700 1 --2- 172.21.15.99:0/1457572932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2bc158df0 0x7fb2bc1530e0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4013000 tx=0x7fb2b4019000 comp rx=0 tx=0).stop 2023-10-08T00:16:21.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.154+0000 7fb2c23b4700 1 -- 172.21.15.99:0/1457572932 shutdown_connections 2023-10-08T00:16:21.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.154+0000 7fb2c23b4700 1 --2- 172.21.15.99:0/1457572932 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb2a8066f30 0x7fb2a80693e0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:21.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.154+0000 7fb2c23b4700 1 --2- 172.21.15.99:0/1457572932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2bc06d370 0x7fb2bc0c8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:21.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.154+0000 7fb2c23b4700 1 --2- 172.21.15.99:0/1457572932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2bc158df0 0x7fb2bc1530e0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:21.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.154+0000 7fb2c23b4700 1 --2- 172.21.15.99:0/1457572932 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2bc152300 0x7fb2bc1516f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:21.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.154+0000 7fb2c23b4700 1 -- 172.21.15.99:0/1457572932 >> 172.21.15.99:0/1457572932 conn(0x7fb2bc069870 msgr2=0x7fb2bc0c39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:21.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.154+0000 7fb2c23b4700 1 -- 172.21.15.99:0/1457572932 shutdown_connections 2023-10-08T00:16:21.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.155+0000 7fb2c23b4700 1 -- 172.21.15.99:0/1457572932 wait complete. 2023-10-08T00:16:21.159 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' created 2023-10-08T00:16:21.173 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T00:16:21.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.288+0000 7fdec84fd700 1 Processor -- start 2023-10-08T00:16:21.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.288+0000 7fdec84fd700 1 -- start start 2023-10-08T00:16:21.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.289+0000 7fdec84fd700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fdec015a290 0x7fdec006a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:21.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.289+0000 7fdec84fd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdec006aae0 0x7fdec00cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:21.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.289+0000 7fdec84fd700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdec00b0790 con 0x7fdec0159ec0 2023-10-08T00:16:21.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.289+0000 7fdec84fd700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdec006af60 con 0x7fdec015a290 2023-10-08T00:16:21.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.289+0000 7fdec84fd700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdec00ce1f0 con 0x7fdec006aae0 2023-10-08T00:16:21.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.289+0000 7fdec5a98700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fdec015a290 0x7fdec006a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:21.294 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.289+0000 7fdec6a9a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdec006aae0 0x7fdec00cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:21.294 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.290+0000 7fdec6a9a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdec006aae0 0x7fdec00cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.99:40770/0 (socket says 172.21.15.99:40770) 2023-10-08T00:16:21.294 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.290+0000 7fdec5a98700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fdec015a290 0x7fdec006a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.99:47638/0 (socket says 172.21.15.99:47638) 2023-10-08T00:16:21.294 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.290+0000 7fdec6a9a700 1 -- 172.21.15.99:0/2028072840 learned_addr learned my addr 172.21.15.99:0/2028072840 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:21.294 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.290+0000 7fdec6299700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7fdec0159ec0 0x7fdec00af940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:37578/0 (socket says 172.21.15.99:37578) 2023-10-08T00:16:21.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.290+0000 7fdec5a98700 1 -- 172.21.15.99:0/2028072840 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdec006aae0 msgr2=0x7fdec00cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:21.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.290+0000 7fdec5a98700 1 --2- 172.21.15.99:0/2028072840 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdec006aae0 0x7fdec00cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:21.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.290+0000 7fdec5a98700 1 -- 172.21.15.99:0/2028072840 >> v1:172.21.15.191:6789/0 conn(0x7fdec0159ec0 legacy=0x7fdec00af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:21.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.290+0000 7fdec5a98700 1 -- 172.21.15.99:0/2028072840 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec00b0b20 con 0x7fdec015a290 2023-10-08T00:16:21.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.290+0000 7fdec6a9a700 1 --2- 172.21.15.99:0/2028072840 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdec006aae0 0x7fdec00cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:16:21.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.290+0000 7fdec5a98700 1 --2- 172.21.15.99:0/2028072840 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdec015a290 0x7fdec006a5a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fdeb00098e0 tx=0x7fdeb00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2c1d0d419e641f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:21.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.290+0000 7fdec5297700 1 -- 172.21.15.99:0/2028072840 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb000b640 con 0x7fdec015a290 2023-10-08T00:16:21.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.290+0000 7fdec5297700 1 -- 172.21.15.99:0/2028072840 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeb000b7a0 con 0x7fdec015a290 2023-10-08T00:16:21.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.290+0000 7fdec5297700 1 -- 172.21.15.99:0/2028072840 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb000bb40 con 0x7fdec015a290 2023-10-08T00:16:21.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.291+0000 7fdec84fd700 1 -- 172.21.15.99:0/2028072840 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdec015a290 msgr2=0x7fdec006a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:21.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.291+0000 7fdec84fd700 1 --2- 172.21.15.99:0/2028072840 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdec015a290 0x7fdec006a5a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fdeb00098e0 tx=0x7fdeb00091a0 comp rx=0 tx=0).stop 2023-10-08T00:16:21.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.291+0000 7fdec84fd700 1 -- 172.21.15.99:0/2028072840 shutdown_connections 2023-10-08T00:16:21.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.291+0000 7fdec84fd700 1 --2- 172.21.15.99:0/2028072840 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdec006aae0 0x7fdec00cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:21.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.291+0000 7fdec84fd700 1 --2- 172.21.15.99:0/2028072840 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdec015a290 0x7fdec006a5a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:21.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.291+0000 7fdec84fd700 1 -- 172.21.15.99:0/2028072840 >> 172.21.15.99:0/2028072840 conn(0x7fdec00d1880 msgr2=0x7fdec00cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:21.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.291+0000 7fdec84fd700 1 -- 172.21.15.99:0/2028072840 shutdown_connections 2023-10-08T00:16:21.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.291+0000 7fdec84fd700 1 -- 172.21.15.99:0/2028072840 wait complete. 2023-10-08T00:16:21.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.292+0000 7fdec84fd700 1 Processor -- start 2023-10-08T00:16:21.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.292+0000 7fdec84fd700 1 -- start start 2023-10-08T00:16:21.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.293+0000 7fdec84fd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdec006aae0 0x7fdec01581e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:21.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.293+0000 7fdec84fd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdec0159ec0 0x7fdec016a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:21.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.293+0000 7fdec84fd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdec015a290 0x7fdec016cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:21.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.293+0000 7fdec84fd700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec00c8f00 con 0x7fdec015a290 2023-10-08T00:16:21.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.293+0000 7fdec84fd700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec00c82f0 con 0x7fdec0159ec0 2023-10-08T00:16:21.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.293+0000 7fdec84fd700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdec00c85d0 con 0x7fdec006aae0 2023-10-08T00:16:21.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.293+0000 7fdec6a9a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdec015a290 0x7fdec016cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:21.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.293+0000 7fdec6a9a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdec015a290 0x7fdec016cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35960/0 (socket says 172.21.15.99:35960) 2023-10-08T00:16:21.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.293+0000 7fdec6a9a700 1 -- 172.21.15.99:0/2940154743 learned_addr learned my addr 172.21.15.99:0/2940154743 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:21.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.293+0000 7fdec6299700 1 --2- 172.21.15.99:0/2940154743 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdec006aae0 0x7fdec01581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:21.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.293+0000 7fdec5a98700 1 --2- 172.21.15.99:0/2940154743 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdec0159ec0 0x7fdec016a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:21.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.293+0000 7fdec6a9a700 1 -- 172.21.15.99:0/2940154743 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdec006aae0 msgr2=0x7fdec01581e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:21.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.293+0000 7fdec6a9a700 1 --2- 172.21.15.99:0/2940154743 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdec006aae0 0x7fdec01581e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:21.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.293+0000 7fdec6a9a700 1 -- 172.21.15.99:0/2940154743 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdec0159ec0 msgr2=0x7fdec016a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:21.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.293+0000 7fdec6a9a700 1 --2- 172.21.15.99:0/2940154743 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdec0159ec0 0x7fdec016a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:21.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.293+0000 7fdec6a9a700 1 -- 172.21.15.99:0/2940154743 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeb80094f0 con 0x7fdec015a290 2023-10-08T00:16:21.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.293+0000 7fdec6a9a700 1 --2- 172.21.15.99:0/2940154743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdec015a290 0x7fdec016cf90 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fdeb800ec50 tx=0x7fdeb8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:21.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.293+0000 7fdeb67fc700 1 -- 172.21.15.99:0/2940154743 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb800c850 con 0x7fdec015a290 2023-10-08T00:16:21.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.294+0000 7fdeb67fc700 1 -- 172.21.15.99:0/2940154743 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeb800c9b0 con 0x7fdec015a290 2023-10-08T00:16:21.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.294+0000 7fdeb67fc700 1 -- 172.21.15.99:0/2940154743 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb800cd50 con 0x7fdec015a290 2023-10-08T00:16:21.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.294+0000 7fdec84fd700 1 -- 172.21.15.99:0/2940154743 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdeb0009420 con 0x7fdec015a290 2023-10-08T00:16:21.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.294+0000 7fdec84fd700 1 -- 172.21.15.99:0/2940154743 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdec00c8b90 con 0x7fdec015a290 2023-10-08T00:16:21.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.295+0000 7fdec84fd700 1 -- 172.21.15.99:0/2940154743 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdec004c3c0 con 0x7fdec015a290 2023-10-08T00:16:21.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.295+0000 7fdeb67fc700 1 -- 172.21.15.99:0/2940154743 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdeb8013680 con 0x7fdec015a290 2023-10-08T00:16:21.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.295+0000 7fdeb67fc700 1 --2- 172.21.15.99:0/2940154743 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdea4066f30 0x7fdea40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:21.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.295+0000 7fdeb67fc700 1 -- 172.21.15.99:0/2940154743 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5605+0+0 (secure 0 0 0) 0x7fdeb80846f0 con 0x7fdec015a290 2023-10-08T00:16:21.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.296+0000 7fdec6299700 1 --2- 172.21.15.99:0/2940154743 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdea4066f30 0x7fdea40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:21.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.296+0000 7fdec6299700 1 --2- 172.21.15.99:0/2940154743 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdea4066f30 0x7fdea40693e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fdebc000f80 tx=0x7fdebc001a90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:21.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.300+0000 7fdeb67fc700 1 -- 172.21.15.99:0/2940154743 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdeb80519c0 con 0x7fdec015a290 2023-10-08T00:16:21.526 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:21.526+0000 7fdec84fd700 1 -- 172.21.15.99:0/2940154743 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fdec00b04b0 con 0x7fdec015a290 2023-10-08T00:16:22.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.168+0000 7fdeb67fc700 1 -- 172.21.15.99:0/2940154743 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v52) v1 ==== 174+0+0 (secure 0 0 0) 0x7fdeb80200c0 con 0x7fdec015a290 2023-10-08T00:16:22.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.173+0000 7fdec84fd700 1 -- 172.21.15.99:0/2940154743 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdea4066f30 msgr2=0x7fdea40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:22.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.173+0000 7fdec84fd700 1 --2- 172.21.15.99:0/2940154743 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdea4066f30 0x7fdea40693e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fdebc000f80 tx=0x7fdebc001a90 comp rx=0 tx=0).stop 2023-10-08T00:16:22.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.173+0000 7fdec84fd700 1 -- 172.21.15.99:0/2940154743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdec015a290 msgr2=0x7fdec016cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:22.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.173+0000 7fdec84fd700 1 --2- 172.21.15.99:0/2940154743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdec015a290 0x7fdec016cf90 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fdeb800ec50 tx=0x7fdeb8009250 comp rx=0 tx=0).stop 2023-10-08T00:16:22.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.173+0000 7fdec84fd700 1 -- 172.21.15.99:0/2940154743 shutdown_connections 2023-10-08T00:16:22.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.173+0000 7fdec84fd700 1 --2- 172.21.15.99:0/2940154743 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdea4066f30 0x7fdea40693e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:22.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.173+0000 7fdec84fd700 1 --2- 172.21.15.99:0/2940154743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdec015a290 0x7fdec016cf90 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:22.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.173+0000 7fdec84fd700 1 --2- 172.21.15.99:0/2940154743 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdec006aae0 0x7fdec01581e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:22.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.174+0000 7fdec84fd700 1 --2- 172.21.15.99:0/2940154743 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdec0159ec0 0x7fdec016a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:22.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.174+0000 7fdec84fd700 1 -- 172.21.15.99:0/2940154743 >> 172.21.15.99:0/2940154743 conn(0x7fdec00d1880 msgr2=0x7fdec00b1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:22.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.174+0000 7fdec84fd700 1 -- 172.21.15.99:0/2940154743 shutdown_connections 2023-10-08T00:16:22.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.174+0000 7fdec84fd700 1 -- 172.21.15.99:0/2940154743 wait complete. 2023-10-08T00:16:22.176 INFO:teuthology.orchestra.run.smithi099.stderr:new fs with metadata pool 14 and data pool 15 2023-10-08T00:16:22.188 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:16:22.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.303+0000 7f8dfb59e700 1 Processor -- start 2023-10-08T00:16:22.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.303+0000 7f8dfb59e700 1 -- start start 2023-10-08T00:16:22.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.304+0000 7f8dfb59e700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8dfc157e60 0x7f8dfc0b1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:22.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.304+0000 7f8dfb59e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8dfc06fef0 0x7f8dfc0ce950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:22.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.304+0000 7f8dfb59e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dfc0cb0b0 con 0x7f8dfc157a90 2023-10-08T00:16:22.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.304+0000 7f8dfb59e700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dfc0b0ad0 con 0x7f8dfc157e60 2023-10-08T00:16:22.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.304+0000 7f8dfb59e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dfc0b4970 con 0x7f8dfc06fef0 2023-10-08T00:16:22.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.304+0000 7f8df9d9b700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8dfc157e60 0x7f8dfc0b1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:22.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.304+0000 7f8df9d9b700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8dfc157e60 0x7f8dfc0b1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35964/0 (socket says 172.21.15.99:35964) 2023-10-08T00:16:22.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.304+0000 7f8df9d9b700 1 -- 172.21.15.99:0/1926895715 learned_addr learned my addr 172.21.15.99:0/1926895715 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:22.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.304+0000 7f8df9d9b700 1 -- 172.21.15.99:0/1926895715 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8dfc06fef0 msgr2=0x7f8dfc0ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:22.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.304+0000 7f8df9d9b700 1 --2- 172.21.15.99:0/1926895715 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8dfc06fef0 0x7f8dfc0ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:22.313 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.304+0000 7f8df9d9b700 1 -- 172.21.15.99:0/1926895715 >> v1:172.21.15.191:6789/0 conn(0x7f8dfc157a90 legacy=0x7f8dfc0b0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:22.313 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.304+0000 7f8df9d9b700 1 -- 172.21.15.99:0/1926895715 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dfc0b0830 con 0x7f8dfc157e60 2023-10-08T00:16:22.313 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.305+0000 7f8df9d9b700 1 --2- 172.21.15.99:0/1926895715 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8dfc157e60 0x7f8dfc0b1790 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f8df00099b0 tx=0x7f8df0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=156f52a0297619e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:22.313 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.305+0000 7f8df959a700 1 -- 172.21.15.99:0/1926895715 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df000b820 con 0x7f8dfc157e60 2023-10-08T00:16:22.313 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.305+0000 7f8df959a700 1 -- 172.21.15.99:0/1926895715 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8df000b980 con 0x7f8dfc157e60 2023-10-08T00:16:22.313 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.305+0000 7f8df959a700 1 -- 172.21.15.99:0/1926895715 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df000bd20 con 0x7f8dfc157e60 2023-10-08T00:16:22.314 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.305+0000 7f8dfb59e700 1 -- 172.21.15.99:0/1926895715 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8dfc157e60 msgr2=0x7f8dfc0b1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:22.314 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.305+0000 7f8dfb59e700 1 --2- 172.21.15.99:0/1926895715 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8dfc157e60 0x7f8dfc0b1790 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f8df00099b0 tx=0x7f8df0009250 comp rx=0 tx=0).stop 2023-10-08T00:16:22.314 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.305+0000 7f8dfb59e700 1 -- 172.21.15.99:0/1926895715 shutdown_connections 2023-10-08T00:16:22.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.305+0000 7f8dfb59e700 1 --2- 172.21.15.99:0/1926895715 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8dfc06fef0 0x7f8dfc0ce950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:22.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.305+0000 7f8dfb59e700 1 --2- 172.21.15.99:0/1926895715 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8dfc157e60 0x7f8dfc0b1790 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:22.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.305+0000 7f8dfb59e700 1 -- 172.21.15.99:0/1926895715 >> 172.21.15.99:0/1926895715 conn(0x7f8dfc06bc70 msgr2=0x7f8dfc06c070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:22.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.305+0000 7f8dfb59e700 1 -- 172.21.15.99:0/1926895715 shutdown_connections 2023-10-08T00:16:22.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.305+0000 7f8dfb59e700 1 -- 172.21.15.99:0/1926895715 wait complete. 2023-10-08T00:16:22.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.306+0000 7f8dfb59e700 1 Processor -- start 2023-10-08T00:16:22.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.306+0000 7f8dfb59e700 1 -- start start 2023-10-08T00:16:22.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.307+0000 7f8dfb59e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8dfc06fef0 0x7f8dfc14f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:22.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.307+0000 7f8dfb59e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8dfc157a90 0x7f8dfc0cd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:22.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.307+0000 7f8dfb59e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8dfc157e60 0x7f8dfc1562b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:22.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.307+0000 7f8dfb59e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dfc158250 con 0x7f8dfc157a90 2023-10-08T00:16:22.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.307+0000 7f8dfb59e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dfc14f900 con 0x7f8dfc06fef0 2023-10-08T00:16:22.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.307+0000 7f8dfb59e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dfc0cdd10 con 0x7f8dfc157e60 2023-10-08T00:16:22.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.307+0000 7f8df9d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8dfc157a90 0x7f8dfc0cd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:22.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.307+0000 7f8df9d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8dfc157a90 0x7f8dfc0cd4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35972/0 (socket says 172.21.15.99:35972) 2023-10-08T00:16:22.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.307+0000 7f8df9d9b700 1 -- 172.21.15.99:0/566065627 learned_addr learned my addr 172.21.15.99:0/566065627 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:22.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.307+0000 7f8dfa59c700 1 --2- 172.21.15.99:0/566065627 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8dfc06fef0 0x7f8dfc14f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:22.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.307+0000 7f8dfad9d700 1 --2- 172.21.15.99:0/566065627 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8dfc157e60 0x7f8dfc1562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:22.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.308+0000 7f8df9d9b700 1 -- 172.21.15.99:0/566065627 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8dfc157e60 msgr2=0x7f8dfc1562b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:22.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.308+0000 7f8df9d9b700 1 --2- 172.21.15.99:0/566065627 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8dfc157e60 0x7f8dfc1562b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:22.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.308+0000 7f8df9d9b700 1 -- 172.21.15.99:0/566065627 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8dfc06fef0 msgr2=0x7f8dfc14f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:22.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.308+0000 7f8df9d9b700 1 --2- 172.21.15.99:0/566065627 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8dfc06fef0 0x7f8dfc14f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:22.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.308+0000 7f8df9d9b700 1 -- 172.21.15.99:0/566065627 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df00094f0 con 0x7f8dfc157a90 2023-10-08T00:16:22.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.308+0000 7f8df9d9b700 1 --2- 172.21.15.99:0/566065627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8dfc157a90 0x7f8dfc0cd4c0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f8df00090b0 tx=0x7f8df0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:22.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.308+0000 7f8dea7fc700 1 -- 172.21.15.99:0/566065627 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df0011e00 con 0x7f8dfc157a90 2023-10-08T00:16:22.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.308+0000 7f8dea7fc700 1 -- 172.21.15.99:0/566065627 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8df001a460 con 0x7f8dfc157a90 2023-10-08T00:16:22.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.308+0000 7f8dfb59e700 1 -- 172.21.15.99:0/566065627 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dfc0cdff0 con 0x7f8dfc157a90 2023-10-08T00:16:22.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.308+0000 7f8dea7fc700 1 -- 172.21.15.99:0/566065627 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df001a820 con 0x7f8dfc157a90 2023-10-08T00:16:22.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.308+0000 7f8dfb59e700 1 -- 172.21.15.99:0/566065627 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dfc1584a0 con 0x7f8dfc157a90 2023-10-08T00:16:22.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.309+0000 7f8dfb59e700 1 -- 172.21.15.99:0/566065627 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dfc06a650 con 0x7f8dfc157a90 2023-10-08T00:16:22.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.309+0000 7f8dea7fc700 1 -- 172.21.15.99:0/566065627 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8df001ac50 con 0x7f8dfc157a90 2023-10-08T00:16:22.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.310+0000 7f8dea7fc700 1 --2- 172.21.15.99:0/566065627 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8ddc066f30 0x7f8ddc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:22.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.310+0000 7f8dea7fc700 1 -- 172.21.15.99:0/566065627 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8df0085d70 con 0x7f8dfc157a90 2023-10-08T00:16:22.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.310+0000 7f8dfa59c700 1 --2- 172.21.15.99:0/566065627 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8ddc066f30 0x7f8ddc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:22.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.310+0000 7f8dfa59c700 1 --2- 172.21.15.99:0/566065627 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8ddc066f30 0x7f8ddc0693e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc0c5220 tx=0x7f8dec0090f0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:22.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.314+0000 7f8dea7fc700 1 -- 172.21.15.99:0/566065627 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8df0052f40 con 0x7f8dfc157a90 2023-10-08T00:16:22.497 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.497+0000 7f8dfb59e700 1 -- 172.21.15.99:0/566065627 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8dfc158690 con 0x7f8dfc157a90 2023-10-08T00:16:22.498 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.498+0000 7f8dea7fc700 1 -- 172.21.15.99:0/566065627 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+15378 (secure 0 0 0) 0x7f8df0020090 con 0x7f8dfc157a90 2023-10-08T00:16:22.498 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:16:22.498 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":123,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:16:21.527302+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:16:19.856259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-08T00:16:20.494851+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:16:22.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.500+0000 7f8dfb59e700 1 -- 172.21.15.99:0/566065627 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8ddc066f30 msgr2=0x7f8ddc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:22.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.500+0000 7f8dfb59e700 1 --2- 172.21.15.99:0/566065627 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8ddc066f30 0x7f8ddc0693e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc0c5220 tx=0x7f8dec0090f0 comp rx=0 tx=0).stop 2023-10-08T00:16:22.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.500+0000 7f8dfb59e700 1 -- 172.21.15.99:0/566065627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8dfc157a90 msgr2=0x7f8dfc0cd4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:22.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.500+0000 7f8dfb59e700 1 --2- 172.21.15.99:0/566065627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8dfc157a90 0x7f8dfc0cd4c0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f8df00090b0 tx=0x7f8df0016000 comp rx=0 tx=0).stop 2023-10-08T00:16:22.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.500+0000 7f8dfb59e700 1 -- 172.21.15.99:0/566065627 shutdown_connections 2023-10-08T00:16:22.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.500+0000 7f8dfb59e700 1 --2- 172.21.15.99:0/566065627 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8ddc066f30 0x7f8ddc0693e0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:22.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.500+0000 7f8dfb59e700 1 --2- 172.21.15.99:0/566065627 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8dfc06fef0 0x7f8dfc14f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:22.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.500+0000 7f8dfb59e700 1 --2- 172.21.15.99:0/566065627 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8dfc157e60 0x7f8dfc1562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:22.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.500+0000 7f8dfb59e700 1 --2- 172.21.15.99:0/566065627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8dfc157a90 0x7f8dfc0cd4c0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:22.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.500+0000 7f8dfb59e700 1 -- 172.21.15.99:0/566065627 >> 172.21.15.99:0/566065627 conn(0x7f8dfc06bc70 msgr2=0x7f8dfc150180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:22.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.500+0000 7f8dfb59e700 1 -- 172.21.15.99:0/566065627 shutdown_connections 2023-10-08T00:16:22.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.500+0000 7f8dfb59e700 1 -- 172.21.15.99:0/566065627 wait complete. 2023-10-08T00:16:22.513 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:16:22.627 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.626+0000 7fcb07d50700 1 Processor -- start 2023-10-08T00:16:22.627 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.626+0000 7fcb07d50700 1 -- start start 2023-10-08T00:16:22.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.627+0000 7fcb07d50700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fcb0015a620 0x7fcb0015d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:22.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.627+0000 7fcb07d50700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb0015cbb0 con 0x7fcb00157870 2023-10-08T00:16:22.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.627+0000 7fcb07d50700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb0015bfa0 con 0x7fcb0015d580 2023-10-08T00:16:22.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.627+0000 7fcb07d50700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb0015bdb0 con 0x7fcb0015a620 2023-10-08T00:16:22.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.627+0000 7fcb05aec700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fcb00157870 0x7fcb0015b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56322/0 (socket says 172.21.15.99:56322) 2023-10-08T00:16:22.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.627+0000 7fcb05aec700 1 -- 172.21.15.99:0/2975381809 learned_addr learned my addr 172.21.15.99:0/2975381809 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:22.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.628+0000 7fcb04aea700 1 -- 172.21.15.99:0/2975381809 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1960870944 0 0) 0x7fcb0015cbb0 con 0x7fcb00157870 2023-10-08T00:16:22.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.628+0000 7fcb052eb700 1 --2- 172.21.15.99:0/2975381809 >> v2:172.21.15.191:3300/0 conn(0x7fcb0015a620 0x7fcb0015d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:22.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.628+0000 7fcb04aea700 1 -- 172.21.15.99:0/2975381809 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcaec003570 con 0x7fcb00157870 2023-10-08T00:16:22.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.628+0000 7fcb04aea700 1 -- 172.21.15.99:0/2975381809 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4188743539 0 0) 0x7fcaec003570 con 0x7fcb00157870 2023-10-08T00:16:22.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.628+0000 7fcb04aea700 1 -- 172.21.15.99:0/2975381809 >> v2:172.21.15.191:3300/0 conn(0x7fcb0015a620 msgr2=0x7fcb0015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:22.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.628+0000 7fcb04aea700 1 --2- 172.21.15.99:0/2975381809 >> v2:172.21.15.191:3300/0 conn(0x7fcb0015a620 0x7fcb0015d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:22.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.628+0000 7fcb04aea700 1 -- 172.21.15.99:0/2975381809 >> v1:172.21.15.191:6789/0 conn(0x7fcb0015d580 legacy=0x7fcb00158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:22.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.628+0000 7fcb04aea700 1 -- 172.21.15.99:0/2975381809 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb0015c280 con 0x7fcb00157870 2023-10-08T00:16:22.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.628+0000 7fcb04aea700 1 -- 172.21.15.99:0/2975381809 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7fcafc001470 con 0x7fcb00157870 2023-10-08T00:16:22.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.628+0000 7fcb04aea700 1 -- 172.21.15.99:0/2975381809 >> v1:172.21.15.99:6789/0 conn(0x7fcb00157870 legacy=0x7fcb0015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:22.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.628+0000 7fcb04aea700 1 --2- 172.21.15.99:0/2975381809 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcaec003a60 0x7fcaec003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:22.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.628+0000 7fcb04aea700 1 --2- 172.21.15.99:0/2975381809 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcaec004810 0x7fcaec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:22.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.628+0000 7fcb04aea700 1 --2- 172.21.15.99:0/2975381809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcaec0051c0 0x7fcaec00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:22.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.628+0000 7fcb04aea700 1 -- 172.21.15.99:0/2975381809 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaec00c130 con 0x7fcaec0051c0 2023-10-08T00:16:22.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.628+0000 7fcb04aea700 1 -- 172.21.15.99:0/2975381809 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaec00c2a0 con 0x7fcaec003a60 2023-10-08T00:16:22.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.628+0000 7fcb04aea700 1 -- 172.21.15.99:0/2975381809 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaec00c460 con 0x7fcaec004810 2023-10-08T00:16:22.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.628+0000 7fcb062ed700 1 --2- 172.21.15.99:0/2975381809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcaec0051c0 0x7fcaec00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:22.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.628+0000 7fcb052eb700 1 --2- 172.21.15.99:0/2975381809 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcaec004810 0x7fcaec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:22.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.628+0000 7fcb05aec700 1 --2- 172.21.15.99:0/2975381809 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcaec003a60 0x7fcaec003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:22.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.628+0000 7fcb062ed700 1 -- 172.21.15.99:0/2975381809 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcaec004810 msgr2=0x7fcaec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:22.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.628+0000 7fcb062ed700 1 --2- 172.21.15.99:0/2975381809 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcaec004810 0x7fcaec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:22.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.628+0000 7fcb062ed700 1 -- 172.21.15.99:0/2975381809 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcaec003a60 msgr2=0x7fcaec003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:22.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.628+0000 7fcb062ed700 1 --2- 172.21.15.99:0/2975381809 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcaec003a60 0x7fcaec003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:22.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.628+0000 7fcb062ed700 1 -- 172.21.15.99:0/2975381809 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaec003570 con 0x7fcaec0051c0 2023-10-08T00:16:22.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.628+0000 7fcb062ed700 1 -- 172.21.15.99:0/2975381809 reap_dead start 2023-10-08T00:16:22.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.628+0000 7fcb062ed700 1 --2- 172.21.15.99:0/2975381809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcaec0051c0 0x7fcaec00ba60 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8001f90 tx=0x7fcaf8001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ba72ee8eb572525 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:22.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.629+0000 7fcb04aea700 1 -- 172.21.15.99:0/2975381809 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf8001950 con 0x7fcaec0051c0 2023-10-08T00:16:22.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.629+0000 7fcb04aea700 1 -- 172.21.15.99:0/2975381809 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaf800c8b0 con 0x7fcaec0051c0 2023-10-08T00:16:22.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.629+0000 7fcb04aea700 1 -- 172.21.15.99:0/2975381809 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf800cc50 con 0x7fcaec0051c0 2023-10-08T00:16:22.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.629+0000 7fcb07d50700 1 -- 172.21.15.99:0/2975381809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcaec0051c0 msgr2=0x7fcaec00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:22.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.629+0000 7fcb07d50700 1 --2- 172.21.15.99:0/2975381809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcaec0051c0 0x7fcaec00ba60 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8001f90 tx=0x7fcaf8001640 comp rx=0 tx=0).stop 2023-10-08T00:16:22.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.629+0000 7fcb07d50700 1 -- 172.21.15.99:0/2975381809 shutdown_connections 2023-10-08T00:16:22.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.629+0000 7fcb07d50700 1 --2- 172.21.15.99:0/2975381809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcaec0051c0 0x7fcaec00ba60 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:22.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.629+0000 7fcb07d50700 1 -- 172.21.15.99:0/2975381809 >> 172.21.15.99:0/2975381809 conn(0x7fcb00068ce0 msgr2=0x7fcb000690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:22.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.629+0000 7fcb07d50700 1 -- 172.21.15.99:0/2975381809 shutdown_connections 2023-10-08T00:16:22.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.629+0000 7fcb07d50700 1 -- 172.21.15.99:0/2975381809 wait complete. 2023-10-08T00:16:22.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.630+0000 7fcb07d50700 1 Processor -- start 2023-10-08T00:16:22.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.630+0000 7fcb07d50700 1 -- start start 2023-10-08T00:16:22.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.631+0000 7fcb07d50700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcaec0051c0 0x7fcb0014ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:22.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.631+0000 7fcb07d50700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb0014e980 0x7fcb000b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:22.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.631+0000 7fcb07d50700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb000d08e0 0x7fcb000d0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:22.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.631+0000 7fcb07d50700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb000d12c0 con 0x7fcaec0051c0 2023-10-08T00:16:22.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.631+0000 7fcb07d50700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb000b19a0 con 0x7fcb0014e980 2023-10-08T00:16:22.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.631+0000 7fcb07d50700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb000b5f90 con 0x7fcb000d08e0 2023-10-08T00:16:22.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.631+0000 7fcb05aec700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcaec0051c0 0x7fcb0014ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:22.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.631+0000 7fcb05aec700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcaec0051c0 0x7fcb0014ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35994/0 (socket says 172.21.15.99:35994) 2023-10-08T00:16:22.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.631+0000 7fcb05aec700 1 -- 172.21.15.99:0/2428862581 learned_addr learned my addr 172.21.15.99:0/2428862581 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:22.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.631+0000 7fcb052eb700 1 --2- 172.21.15.99:0/2428862581 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb0014e980 0x7fcb000b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:22.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.631+0000 7fcb062ed700 1 --2- 172.21.15.99:0/2428862581 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb000d08e0 0x7fcb000d0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:22.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.631+0000 7fcb05aec700 1 -- 172.21.15.99:0/2428862581 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb000d08e0 msgr2=0x7fcb000d0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:22.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.631+0000 7fcb05aec700 1 --2- 172.21.15.99:0/2428862581 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb000d08e0 0x7fcb000d0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:22.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.631+0000 7fcb05aec700 1 -- 172.21.15.99:0/2428862581 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb0014e980 msgr2=0x7fcb000b5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:22.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.631+0000 7fcb05aec700 1 --2- 172.21.15.99:0/2428862581 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb0014e980 0x7fcb000b5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:22.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.631+0000 7fcb05aec700 1 -- 172.21.15.99:0/2428862581 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb0014edf0 con 0x7fcaec0051c0 2023-10-08T00:16:22.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.631+0000 7fcb062ed700 1 --2- 172.21.15.99:0/2428862581 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb000d08e0 0x7fcb000d0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:22.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.632+0000 7fcb05aec700 1 --2- 172.21.15.99:0/2428862581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcaec0051c0 0x7fcb0014ceb0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fcafc0073c0 tx=0x7fcafc001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:22.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.632+0000 7fcaf5ffb700 1 -- 172.21.15.99:0/2428862581 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc018030 con 0x7fcaec0051c0 2023-10-08T00:16:22.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.632+0000 7fcaf5ffb700 1 -- 172.21.15.99:0/2428862581 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcafc00f6c0 con 0x7fcaec0051c0 2023-10-08T00:16:22.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.632+0000 7fcb07d50700 1 -- 172.21.15.99:0/2428862581 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb000b61f0 con 0x7fcaec0051c0 2023-10-08T00:16:22.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.632+0000 7fcaf5ffb700 1 -- 172.21.15.99:0/2428862581 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc00faa0 con 0x7fcaec0051c0 2023-10-08T00:16:22.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.632+0000 7fcb07d50700 1 -- 172.21.15.99:0/2428862581 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb000b67b0 con 0x7fcaec0051c0 2023-10-08T00:16:22.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.633+0000 7fcb07d50700 1 -- 172.21.15.99:0/2428862581 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb0004c3c0 con 0x7fcaec0051c0 2023-10-08T00:16:22.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.633+0000 7fcaf5ffb700 1 -- 172.21.15.99:0/2428862581 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcafc01b070 con 0x7fcaec0051c0 2023-10-08T00:16:22.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.633+0000 7fcaf5ffb700 1 --2- 172.21.15.99:0/2428862581 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcae4066f30 0x7fcae40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:22.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.634+0000 7fcaf5ffb700 1 -- 172.21.15.99:0/2428862581 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcafc019070 con 0x7fcaec0051c0 2023-10-08T00:16:22.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.634+0000 7fcb052eb700 1 --2- 172.21.15.99:0/2428862581 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcae4066f30 0x7fcae40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:22.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.634+0000 7fcb052eb700 1 --2- 172.21.15.99:0/2428862581 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcae4066f30 0x7fcae40693e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fcb0015b250 tx=0x7fcaf00091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:22.641 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.638+0000 7fcaf5ffb700 1 -- 172.21.15.99:0/2428862581 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcafc051470 con 0x7fcaec0051c0 2023-10-08T00:16:22.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.821+0000 7fcb07d50700 1 -- 172.21.15.99:0/2428862581 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb000b1770 con 0x7fcaec0051c0 2023-10-08T00:16:22.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.822+0000 7fcaf5ffb700 1 -- 172.21.15.99:0/2428862581 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+15378 (secure 0 0 0) 0x7fcafc054a90 con 0x7fcaec0051c0 2023-10-08T00:16:22.823 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:16:22.823 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":123,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:16:21.527302+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:16:19.856259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-08T00:16:20.494851+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:16:22.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.824+0000 7fcb07d50700 1 -- 172.21.15.99:0/2428862581 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcae4066f30 msgr2=0x7fcae40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:22.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.824+0000 7fcb07d50700 1 --2- 172.21.15.99:0/2428862581 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcae4066f30 0x7fcae40693e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fcb0015b250 tx=0x7fcaf00091a0 comp rx=0 tx=0).stop 2023-10-08T00:16:22.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.824+0000 7fcb07d50700 1 -- 172.21.15.99:0/2428862581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcaec0051c0 msgr2=0x7fcb0014ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:22.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.824+0000 7fcb07d50700 1 --2- 172.21.15.99:0/2428862581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcaec0051c0 0x7fcb0014ceb0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fcafc0073c0 tx=0x7fcafc001e50 comp rx=0 tx=0).stop 2023-10-08T00:16:22.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.824+0000 7fcb07d50700 1 -- 172.21.15.99:0/2428862581 shutdown_connections 2023-10-08T00:16:22.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.824+0000 7fcb07d50700 1 --2- 172.21.15.99:0/2428862581 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcae4066f30 0x7fcae40693e0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:22.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.824+0000 7fcb07d50700 1 --2- 172.21.15.99:0/2428862581 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb000d08e0 0x7fcb000d0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:22.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.824+0000 7fcb07d50700 1 --2- 172.21.15.99:0/2428862581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcaec0051c0 0x7fcb0014ceb0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:22.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.824+0000 7fcb07d50700 1 --2- 172.21.15.99:0/2428862581 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb0014e980 0x7fcb000b5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:22.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.824+0000 7fcb07d50700 1 -- 172.21.15.99:0/2428862581 >> 172.21.15.99:0/2428862581 conn(0x7fcb00068ce0 msgr2=0x7fcb00159630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:22.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.824+0000 7fcb07d50700 1 -- 172.21.15.99:0/2428862581 shutdown_connections 2023-10-08T00:16:22.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.824+0000 7fcb07d50700 1 -- 172.21.15.99:0/2428862581 wait complete. 2023-10-08T00:16:22.837 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T00:16:22.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.953+0000 7fcedc984700 1 Processor -- start 2023-10-08T00:16:22.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.953+0000 7fcedc984700 1 -- start start 2023-10-08T00:16:22.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.954+0000 7fcedc984700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fced815a290 0x7fced806a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:22.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.954+0000 7fcedc984700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fced806aae0 0x7fced80cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:22.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.954+0000 7fcedc984700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fced80b0790 con 0x7fced8159ec0 2023-10-08T00:16:22.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.954+0000 7fcedc984700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fced806af60 con 0x7fced815a290 2023-10-08T00:16:22.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.954+0000 7fcedc984700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fced80ce1f0 con 0x7fced806aae0 2023-10-08T00:16:22.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.954+0000 7fced6d9d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fced806aae0 0x7fced80cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:22.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.954+0000 7fced5d9b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fced815a290 0x7fced806a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:22.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.954+0000 7fced659c700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7fced8159ec0 0x7fced80af940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:37588/0 (socket says 172.21.15.99:37588) 2023-10-08T00:16:22.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.954+0000 7fced659c700 1 -- 172.21.15.99:0/3590868736 learned_addr learned my addr 172.21.15.99:0/3590868736 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:22.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.954+0000 7fced559a700 1 -- 172.21.15.99:0/3590868736 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1819606033 0 0) 0x7fced80b0790 con 0x7fced8159ec0 2023-10-08T00:16:22.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.954+0000 7fced559a700 1 -- 172.21.15.99:0/3590868736 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcec0003570 con 0x7fced8159ec0 2023-10-08T00:16:22.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.954+0000 7fced6d9d700 1 -- 172.21.15.99:0/3590868736 >> v2:172.21.15.191:3300/0 conn(0x7fced815a290 msgr2=0x7fced806a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:22.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.954+0000 7fced6d9d700 1 --2- 172.21.15.99:0/3590868736 >> v2:172.21.15.191:3300/0 conn(0x7fced815a290 0x7fced806a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:22.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.954+0000 7fced6d9d700 1 -- 172.21.15.99:0/3590868736 >> v1:172.21.15.191:6789/0 conn(0x7fced8159ec0 legacy=0x7fced80af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:22.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.954+0000 7fced6d9d700 1 -- 172.21.15.99:0/3590868736 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced80b0b20 con 0x7fced806aae0 2023-10-08T00:16:22.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.954+0000 7fced5d9b700 1 --2- 172.21.15.99:0/3590868736 >> v2:172.21.15.191:3300/0 conn(0x7fced815a290 0x7fced806a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:16:22.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.955+0000 7fced6d9d700 1 --2- 172.21.15.99:0/3590868736 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fced806aae0 0x7fced80cf590 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fced0009e60 tx=0x7fced0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56075c6ea26ea248 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:22.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.955+0000 7fced559a700 1 -- 172.21.15.99:0/3590868736 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced000e070 con 0x7fced806aae0 2023-10-08T00:16:22.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.955+0000 7fced559a700 1 -- 172.21.15.99:0/3590868736 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fced000bab0 con 0x7fced806aae0 2023-10-08T00:16:22.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.955+0000 7fced559a700 1 -- 172.21.15.99:0/3590868736 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced000be30 con 0x7fced806aae0 2023-10-08T00:16:22.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.955+0000 7fcedc984700 1 -- 172.21.15.99:0/3590868736 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fced806aae0 msgr2=0x7fced80cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:22.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.955+0000 7fcedc984700 1 --2- 172.21.15.99:0/3590868736 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fced806aae0 0x7fced80cf590 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fced0009e60 tx=0x7fced0009780 comp rx=0 tx=0).stop 2023-10-08T00:16:22.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.955+0000 7fcedc984700 1 -- 172.21.15.99:0/3590868736 shutdown_connections 2023-10-08T00:16:22.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.955+0000 7fcedc984700 1 --2- 172.21.15.99:0/3590868736 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fced806aae0 0x7fced80cf590 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:22.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.955+0000 7fcedc984700 1 --2- 172.21.15.99:0/3590868736 >> v2:172.21.15.191:3300/0 conn(0x7fced815a290 0x7fced806a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:22.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.955+0000 7fcedc984700 1 -- 172.21.15.99:0/3590868736 >> 172.21.15.99:0/3590868736 conn(0x7fced80d1880 msgr2=0x7fced80cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:22.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.955+0000 7fcedc984700 1 -- 172.21.15.99:0/3590868736 shutdown_connections 2023-10-08T00:16:22.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.955+0000 7fcedc984700 1 -- 172.21.15.99:0/3590868736 wait complete. 2023-10-08T00:16:22.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.956+0000 7fcedc984700 1 Processor -- start 2023-10-08T00:16:22.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.957+0000 7fcedc984700 1 -- start start 2023-10-08T00:16:22.961 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.957+0000 7fcedc984700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fced806aae0 0x7fced81487c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:22.961 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.957+0000 7fcedc984700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fced8159ec0 0x7fced80cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:22.961 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.957+0000 7fcedc984700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fced815a290 0x7fced80b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:22.961 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.957+0000 7fcedc984700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced8156840 con 0x7fced806aae0 2023-10-08T00:16:22.961 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.957+0000 7fcedc984700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced8156b20 con 0x7fced815a290 2023-10-08T00:16:22.961 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.957+0000 7fcedc984700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fced8156e00 con 0x7fced8159ec0 2023-10-08T00:16:22.961 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.957+0000 7fced659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fced806aae0 0x7fced81487c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:22.962 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.957+0000 7fced659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fced806aae0 0x7fced81487c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36008/0 (socket says 172.21.15.99:36008) 2023-10-08T00:16:22.962 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.957+0000 7fced659c700 1 -- 172.21.15.99:0/2010290493 learned_addr learned my addr 172.21.15.99:0/2010290493 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:22.962 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.957+0000 7fced6d9d700 1 --2- 172.21.15.99:0/2010290493 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fced815a290 0x7fced80b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:22.962 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.957+0000 7fced5d9b700 1 --2- 172.21.15.99:0/2010290493 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fced8159ec0 0x7fced80cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:22.962 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.958+0000 7fced659c700 1 -- 172.21.15.99:0/2010290493 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fced8159ec0 msgr2=0x7fced80cb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:22.962 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.958+0000 7fced659c700 1 --2- 172.21.15.99:0/2010290493 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fced8159ec0 0x7fced80cb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:22.962 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.958+0000 7fced659c700 1 -- 172.21.15.99:0/2010290493 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fced815a290 msgr2=0x7fced80b4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:22.962 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.958+0000 7fced659c700 1 --2- 172.21.15.99:0/2010290493 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fced815a290 0x7fced80b4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:22.962 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.958+0000 7fced659c700 1 -- 172.21.15.99:0/2010290493 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcecc009420 con 0x7fced806aae0 2023-10-08T00:16:22.962 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.958+0000 7fced6d9d700 1 --2- 172.21.15.99:0/2010290493 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fced815a290 0x7fced80b4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:22.962 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.958+0000 7fced5d9b700 1 --2- 172.21.15.99:0/2010290493 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fced8159ec0 0x7fced80cb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:22.962 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.958+0000 7fced659c700 1 --2- 172.21.15.99:0/2010290493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fced806aae0 0x7fced81487c0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fcec80011e0 tx=0x7fcec8007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:22.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.958+0000 7fcec67fc700 1 -- 172.21.15.99:0/2010290493 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec8005b30 con 0x7fced806aae0 2023-10-08T00:16:22.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.958+0000 7fcec67fc700 1 -- 172.21.15.99:0/2010290493 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcec8005c90 con 0x7fced806aae0 2023-10-08T00:16:22.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.958+0000 7fcedc984700 1 -- 172.21.15.99:0/2010290493 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fced0009a20 con 0x7fced806aae0 2023-10-08T00:16:22.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.958+0000 7fcec67fc700 1 -- 172.21.15.99:0/2010290493 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec80072e0 con 0x7fced806aae0 2023-10-08T00:16:22.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.958+0000 7fcedc984700 1 -- 172.21.15.99:0/2010290493 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fced8148d00 con 0x7fced806aae0 2023-10-08T00:16:22.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.959+0000 7fcedc984700 1 -- 172.21.15.99:0/2010290493 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fced804c3c0 con 0x7fced806aae0 2023-10-08T00:16:22.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.959+0000 7fcec67fc700 1 -- 172.21.15.99:0/2010290493 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcec8007650 con 0x7fced806aae0 2023-10-08T00:16:22.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.960+0000 7fcec67fc700 1 --2- 172.21.15.99:0/2010290493 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fceb8066f30 0x7fceb80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:22.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.960+0000 7fcec67fc700 1 -- 172.21.15.99:0/2010290493 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcec8083980 con 0x7fced806aae0 2023-10-08T00:16:22.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.960+0000 7fced5d9b700 1 --2- 172.21.15.99:0/2010290493 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fceb8066f30 0x7fceb80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:22.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.960+0000 7fced5d9b700 1 --2- 172.21.15.99:0/2010290493 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fceb8066f30 0x7fceb80693e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fcecc0093f0 tx=0x7fcecc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:22.964 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:22.964+0000 7fcec67fc700 1 -- 172.21.15.99:0/2010290493 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcec804b5a0 con 0x7fced806aae0 2023-10-08T00:16:23.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:23.190+0000 7fcedc984700 1 -- 172.21.15.99:0/2010290493 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fced80b04b0 con 0x7fced806aae0 2023-10-08T00:16:23.310 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:16:23.313 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:16:24.186 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.185+0000 7fcec67fc700 1 -- 172.21.15.99:0/2010290493 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v55) v1 ==== 118+0+0 (secure 0 0 0) 0x7fcec8017020 con 0x7fced806aae0 2023-10-08T00:16:24.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.187+0000 7fcedc984700 1 -- 172.21.15.99:0/2010290493 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fceb8066f30 msgr2=0x7fceb80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:24.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.187+0000 7fcedc984700 1 --2- 172.21.15.99:0/2010290493 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fceb8066f30 0x7fceb80693e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fcecc0093f0 tx=0x7fcecc008040 comp rx=0 tx=0).stop 2023-10-08T00:16:24.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.187+0000 7fcedc984700 1 -- 172.21.15.99:0/2010290493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fced806aae0 msgr2=0x7fced81487c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:24.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.187+0000 7fcedc984700 1 --2- 172.21.15.99:0/2010290493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fced806aae0 0x7fced81487c0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fcec80011e0 tx=0x7fcec8007df0 comp rx=0 tx=0).stop 2023-10-08T00:16:24.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.188+0000 7fcedc984700 1 -- 172.21.15.99:0/2010290493 shutdown_connections 2023-10-08T00:16:24.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.188+0000 7fcedc984700 1 --2- 172.21.15.99:0/2010290493 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fceb8066f30 0x7fceb80693e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:24.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.188+0000 7fcedc984700 1 --2- 172.21.15.99:0/2010290493 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fced815a290 0x7fced80b4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:24.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.188+0000 7fcedc984700 1 --2- 172.21.15.99:0/2010290493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fced806aae0 0x7fced81487c0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:24.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.188+0000 7fcedc984700 1 --2- 172.21.15.99:0/2010290493 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fced8159ec0 0x7fced80cb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:24.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.188+0000 7fcedc984700 1 -- 172.21.15.99:0/2010290493 >> 172.21.15.99:0/2010290493 conn(0x7fced80d1880 msgr2=0x7fced80b14f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:24.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.188+0000 7fcedc984700 1 -- 172.21.15.99:0/2010290493 shutdown_connections 2023-10-08T00:16:24.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.188+0000 7fcedc984700 1 -- 172.21.15.99:0/2010290493 wait complete. 2023-10-08T00:16:24.201 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T00:16:24.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.318+0000 7f35b4f88700 1 Processor -- start 2023-10-08T00:16:24.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.318+0000 7f35b4f88700 1 -- start start 2023-10-08T00:16:24.321 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.319+0000 7f35b4f88700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f35b0159ec0 0x7f35b00af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:24.321 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.319+0000 7f35b4f88700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f35b006acb0 0x7f35b00afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:24.321 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.319+0000 7f35b4f88700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35b00b0790 con 0x7f35b015a290 2023-10-08T00:16:24.321 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.319+0000 7f35b4f88700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35b00ce1f0 con 0x7f35b0159ec0 2023-10-08T00:16:24.321 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.319+0000 7f35b4f88700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35b00b0b20 con 0x7f35b006acb0 2023-10-08T00:16:24.321 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.319+0000 7f35ae59c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f35b0159ec0 0x7f35b00af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:24.321 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.319+0000 7f35add9b700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f35b015a290 0x7f35b006a5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50650/0 (socket says 172.21.15.99:50650) 2023-10-08T00:16:24.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.319+0000 7f35ae59c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f35b0159ec0 0x7f35b00af940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36996/0 (socket says 172.21.15.99:36996) 2023-10-08T00:16:24.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.319+0000 7f35add9b700 1 -- 172.21.15.99:0/183748815 learned_addr learned my addr 172.21.15.99:0/183748815 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:24.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.319+0000 7f35aed9d700 1 --2- 172.21.15.99:0/183748815 >> v2:172.21.15.191:3300/0 conn(0x7f35b006acb0 0x7f35b00afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:24.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.319+0000 7f35ae59c700 1 -- 172.21.15.99:0/183748815 >> v2:172.21.15.191:3300/0 conn(0x7f35b006acb0 msgr2=0x7f35b00afe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:24.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.319+0000 7f35ae59c700 1 --2- 172.21.15.99:0/183748815 >> v2:172.21.15.191:3300/0 conn(0x7f35b006acb0 0x7f35b00afe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:24.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.319+0000 7f35ae59c700 1 -- 172.21.15.99:0/183748815 >> v1:172.21.15.99:6789/0 conn(0x7f35b015a290 legacy=0x7f35b006a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:24.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.319+0000 7f35ae59c700 1 -- 172.21.15.99:0/183748815 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b00b04b0 con 0x7f35b0159ec0 2023-10-08T00:16:24.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.319+0000 7f35ad59a700 1 -- 172.21.15.99:0/183748815 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2724829810 0 0) 0x7f35b00b0790 con 0x7f35b015a290 2023-10-08T00:16:24.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.319+0000 7f35ae59c700 1 --2- 172.21.15.99:0/183748815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35b0159ec0 0x7f35b00af940 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f35b00711f0 tx=0x7f35a00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67c4e38fbf5de8aa server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:24.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.319+0000 7f35ad59a700 1 -- 172.21.15.99:0/183748815 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a000b6b0 con 0x7f35b0159ec0 2023-10-08T00:16:24.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.319+0000 7f35ad59a700 1 -- 172.21.15.99:0/183748815 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a000b810 con 0x7f35b0159ec0 2023-10-08T00:16:24.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.320+0000 7f35ad59a700 1 -- 172.21.15.99:0/183748815 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a000bbb0 con 0x7f35b0159ec0 2023-10-08T00:16:24.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.320+0000 7f35b4f88700 1 -- 172.21.15.99:0/183748815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35b0159ec0 msgr2=0x7f35b00af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:24.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.320+0000 7f35b4f88700 1 --2- 172.21.15.99:0/183748815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35b0159ec0 0x7f35b00af940 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f35b00711f0 tx=0x7f35a00091a0 comp rx=0 tx=0).stop 2023-10-08T00:16:24.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.320+0000 7f35b4f88700 1 -- 172.21.15.99:0/183748815 shutdown_connections 2023-10-08T00:16:24.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.320+0000 7f35b4f88700 1 --2- 172.21.15.99:0/183748815 >> v2:172.21.15.191:3300/0 conn(0x7f35b006acb0 0x7f35b00afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:24.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.320+0000 7f35b4f88700 1 --2- 172.21.15.99:0/183748815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35b0159ec0 0x7f35b00af940 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:24.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.320+0000 7f35b4f88700 1 -- 172.21.15.99:0/183748815 >> 172.21.15.99:0/183748815 conn(0x7f35b00d1880 msgr2=0x7f35b00cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:24.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.320+0000 7f35b4f88700 1 -- 172.21.15.99:0/183748815 shutdown_connections 2023-10-08T00:16:24.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.320+0000 7f35b4f88700 1 -- 172.21.15.99:0/183748815 wait complete. 2023-10-08T00:16:24.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.321+0000 7f35b4f88700 1 Processor -- start 2023-10-08T00:16:24.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.321+0000 7f35b4f88700 1 -- start start 2023-10-08T00:16:24.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.322+0000 7f35b4f88700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35b006acb0 0x7f35b00c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:24.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.322+0000 7f35b4f88700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f35b0159ec0 0x7f35b00c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:24.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.322+0000 7f35b4f88700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35b015a290 0x7f35b00c9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:24.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.322+0000 7f35b4f88700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b016e290 con 0x7f35b006acb0 2023-10-08T00:16:24.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.322+0000 7f35b4f88700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b0159620 con 0x7f35b015a290 2023-10-08T00:16:24.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.322+0000 7f35b4f88700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35b016d680 con 0x7f35b0159ec0 2023-10-08T00:16:24.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.322+0000 7f35ae59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35b006acb0 0x7f35b00c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:24.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.322+0000 7f35ae59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35b006acb0 0x7f35b00c7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36998/0 (socket says 172.21.15.99:36998) 2023-10-08T00:16:24.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.322+0000 7f35ae59c700 1 -- 172.21.15.99:0/1030309595 learned_addr learned my addr 172.21.15.99:0/1030309595 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:24.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.322+0000 7f35aed9d700 1 --2- 172.21.15.99:0/1030309595 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35b015a290 0x7f35b00c9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:24.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.322+0000 7f35add9b700 1 --2- 172.21.15.99:0/1030309595 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f35b0159ec0 0x7f35b00c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:24.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.322+0000 7f35ae59c700 1 -- 172.21.15.99:0/1030309595 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f35b0159ec0 msgr2=0x7f35b00c8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:24.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.322+0000 7f35ae59c700 1 --2- 172.21.15.99:0/1030309595 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f35b0159ec0 0x7f35b00c8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:24.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.322+0000 7f35ae59c700 1 -- 172.21.15.99:0/1030309595 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35b015a290 msgr2=0x7f35b00c9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:24.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.322+0000 7f35ae59c700 1 --2- 172.21.15.99:0/1030309595 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35b015a290 0x7f35b00c9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:24.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.322+0000 7f35ae59c700 1 -- 172.21.15.99:0/1030309595 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a0009420 con 0x7f35b006acb0 2023-10-08T00:16:24.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.322+0000 7f35aed9d700 1 --2- 172.21.15.99:0/1030309595 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35b015a290 0x7f35b00c9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:24.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.322+0000 7f35ae59c700 1 --2- 172.21.15.99:0/1030309595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35b006acb0 0x7f35b00c7e60 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f35b0071eb0 tx=0x7f35a0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:24.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.323+0000 7f359e7fc700 1 -- 172.21.15.99:0/1030309595 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a0011420 con 0x7f35b006acb0 2023-10-08T00:16:24.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.323+0000 7f359e7fc700 1 -- 172.21.15.99:0/1030309595 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a0011580 con 0x7f35b006acb0 2023-10-08T00:16:24.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.323+0000 7f359e7fc700 1 -- 172.21.15.99:0/1030309595 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a0011940 con 0x7f35b006acb0 2023-10-08T00:16:24.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.323+0000 7f35b4f88700 1 -- 172.21.15.99:0/1030309595 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35b016d960 con 0x7f35b006acb0 2023-10-08T00:16:24.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.323+0000 7f35b4f88700 1 -- 172.21.15.99:0/1030309595 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35b016df20 con 0x7f35b006acb0 2023-10-08T00:16:24.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.323+0000 7f35b4f88700 1 -- 172.21.15.99:0/1030309595 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35b004c3c0 con 0x7f35b006acb0 2023-10-08T00:16:24.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.324+0000 7f359e7fc700 1 -- 172.21.15.99:0/1030309595 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f35a0011d30 con 0x7f35b006acb0 2023-10-08T00:16:24.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.324+0000 7f359e7fc700 1 --2- 172.21.15.99:0/1030309595 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3590066f30 0x7f35900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:24.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.324+0000 7f359e7fc700 1 -- 172.21.15.99:0/1030309595 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f35a0085a60 con 0x7f35b006acb0 2023-10-08T00:16:24.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.324+0000 7f35add9b700 1 --2- 172.21.15.99:0/1030309595 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3590066f30 0x7f35900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:24.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.325+0000 7f35add9b700 1 --2- 172.21.15.99:0/1030309595 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3590066f30 0x7f35900693e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f35b00c9400 tx=0x7f35a4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:24.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.328+0000 7f359e7fc700 1 -- 172.21.15.99:0/1030309595 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f35a0052cd0 con 0x7f35b006acb0 2023-10-08T00:16:24.553 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:24.553+0000 7f35b4f88700 1 -- 172.21.15.99:0/1030309595 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f35b00ceab0 con 0x7f35b006acb0 2023-10-08T00:16:25.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.189+0000 7f359e7fc700 1 -- 172.21.15.99:0/1030309595 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v56) v1 ==== 122+0+0 (secure 0 0 0) 0x7f35a00150c0 con 0x7f35b006acb0 2023-10-08T00:16:25.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.191+0000 7f35b4f88700 1 -- 172.21.15.99:0/1030309595 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3590066f30 msgr2=0x7f35900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:25.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.191+0000 7f35b4f88700 1 --2- 172.21.15.99:0/1030309595 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3590066f30 0x7f35900693e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f35b00c9400 tx=0x7f35a4009040 comp rx=0 tx=0).stop 2023-10-08T00:16:25.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.191+0000 7f35b4f88700 1 -- 172.21.15.99:0/1030309595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35b006acb0 msgr2=0x7f35b00c7e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:25.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.191+0000 7f35b4f88700 1 --2- 172.21.15.99:0/1030309595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35b006acb0 0x7f35b00c7e60 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f35b0071eb0 tx=0x7f35a0017000 comp rx=0 tx=0).stop 2023-10-08T00:16:25.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.191+0000 7f35b4f88700 1 -- 172.21.15.99:0/1030309595 shutdown_connections 2023-10-08T00:16:25.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.191+0000 7f35b4f88700 1 --2- 172.21.15.99:0/1030309595 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3590066f30 0x7f35900693e0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:25.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.191+0000 7f35b4f88700 1 --2- 172.21.15.99:0/1030309595 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35b015a290 0x7f35b00c9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:25.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.191+0000 7f35b4f88700 1 --2- 172.21.15.99:0/1030309595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35b006acb0 0x7f35b00c7e60 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:25.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.191+0000 7f35b4f88700 1 --2- 172.21.15.99:0/1030309595 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f35b0159ec0 0x7f35b00c8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:25.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.191+0000 7f35b4f88700 1 -- 172.21.15.99:0/1030309595 >> 172.21.15.99:0/1030309595 conn(0x7f35b00d1880 msgr2=0x7f35b00b1780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:25.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.191+0000 7f35b4f88700 1 -- 172.21.15.99:0/1030309595 shutdown_connections 2023-10-08T00:16:25.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.191+0000 7f35b4f88700 1 -- 172.21.15.99:0/1030309595 wait complete. 2023-10-08T00:16:25.205 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:16:25.321 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.320+0000 7fef08359700 1 Processor -- start 2023-10-08T00:16:25.321 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.320+0000 7fef08359700 1 -- start start 2023-10-08T00:16:25.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.321+0000 7fef08359700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fef00157c40 0x7fef00158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:25.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.321+0000 7fef08359700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef0015cbb0 con 0x7fef0015a620 2023-10-08T00:16:25.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.321+0000 7fef08359700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef0015bfa0 con 0x7fef00157870 2023-10-08T00:16:25.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.321+0000 7fef08359700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef0015bdb0 con 0x7fef00157c40 2023-10-08T00:16:25.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.321+0000 7fef058f4700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fef0015a620 0x7fef0015d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50654/0 (socket says 172.21.15.99:50654) 2023-10-08T00:16:25.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.321+0000 7fef058f4700 1 -- 172.21.15.99:0/3594058546 learned_addr learned my addr 172.21.15.99:0/3594058546 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:25.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.322+0000 7fef068f6700 1 --2- 172.21.15.99:0/3594058546 >> v2:172.21.15.191:3300/0 conn(0x7fef00157c40 0x7fef00158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:25.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.322+0000 7fef050f3700 1 -- 172.21.15.99:0/3594058546 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2197261737 0 0) 0x7fef0015cbb0 con 0x7fef0015a620 2023-10-08T00:16:25.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.322+0000 7fef050f3700 1 -- 172.21.15.99:0/3594058546 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feeec003570 con 0x7fef0015a620 2023-10-08T00:16:25.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.322+0000 7fef050f3700 1 -- 172.21.15.99:0/3594058546 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3944604689 0 0) 0x7feeec003570 con 0x7fef0015a620 2023-10-08T00:16:25.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.322+0000 7fef050f3700 1 -- 172.21.15.99:0/3594058546 >> v2:172.21.15.191:3300/0 conn(0x7fef00157c40 msgr2=0x7fef00158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:25.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.322+0000 7fef050f3700 1 --2- 172.21.15.99:0/3594058546 >> v2:172.21.15.191:3300/0 conn(0x7fef00157c40 0x7fef00158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:25.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.322+0000 7fef050f3700 1 -- 172.21.15.99:0/3594058546 >> v1:172.21.15.191:6789/0 conn(0x7fef00157870 legacy=0x7fef0015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:25.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.322+0000 7fef050f3700 1 -- 172.21.15.99:0/3594058546 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef0015c280 con 0x7fef0015a620 2023-10-08T00:16:25.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.322+0000 7fef050f3700 1 -- 172.21.15.99:0/3594058546 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7feef0001470 con 0x7fef0015a620 2023-10-08T00:16:25.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.322+0000 7fef050f3700 1 -- 172.21.15.99:0/3594058546 >> v1:172.21.15.99:6789/0 conn(0x7fef0015a620 legacy=0x7fef0015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:25.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.322+0000 7fef050f3700 1 --2- 172.21.15.99:0/3594058546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feeec003a60 0x7feeec003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:25.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.322+0000 7fef050f3700 1 --2- 172.21.15.99:0/3594058546 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feeec004810 0x7feeec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:25.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.322+0000 7fef050f3700 1 --2- 172.21.15.99:0/3594058546 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feeec0051c0 0x7feeec00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:25.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.322+0000 7fef050f3700 1 -- 172.21.15.99:0/3594058546 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeec00c130 con 0x7feeec003a60 2023-10-08T00:16:25.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.322+0000 7fef050f3700 1 -- 172.21.15.99:0/3594058546 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeec00c2a0 con 0x7feeec0051c0 2023-10-08T00:16:25.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.322+0000 7fef050f3700 1 -- 172.21.15.99:0/3594058546 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeec00c460 con 0x7feeec004810 2023-10-08T00:16:25.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.322+0000 7fef060f5700 1 --2- 172.21.15.99:0/3594058546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feeec003a60 0x7feeec003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:25.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.322+0000 7fef068f6700 1 --2- 172.21.15.99:0/3594058546 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feeec0051c0 0x7feeec00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:25.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.322+0000 7fef058f4700 1 --2- 172.21.15.99:0/3594058546 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feeec004810 0x7feeec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:25.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.322+0000 7fef060f5700 1 -- 172.21.15.99:0/3594058546 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feeec004810 msgr2=0x7feeec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:25.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.322+0000 7fef060f5700 1 --2- 172.21.15.99:0/3594058546 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feeec004810 0x7feeec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:25.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.322+0000 7fef060f5700 1 -- 172.21.15.99:0/3594058546 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feeec0051c0 msgr2=0x7feeec00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:25.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.322+0000 7fef060f5700 1 --2- 172.21.15.99:0/3594058546 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feeec0051c0 0x7feeec00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:25.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.322+0000 7fef060f5700 1 -- 172.21.15.99:0/3594058546 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeec003570 con 0x7feeec003a60 2023-10-08T00:16:25.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.322+0000 7fef068f6700 1 --2- 172.21.15.99:0/3594058546 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feeec0051c0 0x7feeec00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:25.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.322+0000 7fef068f6700 1 -- 172.21.15.99:0/3594058546 reap_dead start 2023-10-08T00:16:25.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.322+0000 7fef060f5700 1 --2- 172.21.15.99:0/3594058546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feeec003a60 0x7feeec003e70 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7feefc0018b0 tx=0x7feefc007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e849335b1a7429b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:25.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.323+0000 7fef050f3700 1 -- 172.21.15.99:0/3594058546 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feefc012420 con 0x7feeec003a60 2023-10-08T00:16:25.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.323+0000 7fef050f3700 1 -- 172.21.15.99:0/3594058546 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feefc0125a0 con 0x7feeec003a60 2023-10-08T00:16:25.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.323+0000 7fef050f3700 1 -- 172.21.15.99:0/3594058546 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feefc012960 con 0x7feeec003a60 2023-10-08T00:16:25.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.323+0000 7fef08359700 1 -- 172.21.15.99:0/3594058546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feeec003a60 msgr2=0x7feeec003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:25.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.323+0000 7fef08359700 1 --2- 172.21.15.99:0/3594058546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feeec003a60 0x7feeec003e70 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7feefc0018b0 tx=0x7feefc007d30 comp rx=0 tx=0).stop 2023-10-08T00:16:25.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.323+0000 7fef08359700 1 -- 172.21.15.99:0/3594058546 shutdown_connections 2023-10-08T00:16:25.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.323+0000 7fef08359700 1 --2- 172.21.15.99:0/3594058546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feeec003a60 0x7feeec003e70 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:25.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.323+0000 7fef08359700 1 -- 172.21.15.99:0/3594058546 >> 172.21.15.99:0/3594058546 conn(0x7fef00068ce0 msgr2=0x7fef000690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:25.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.323+0000 7fef08359700 1 -- 172.21.15.99:0/3594058546 shutdown_connections 2023-10-08T00:16:25.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.323+0000 7fef08359700 1 -- 172.21.15.99:0/3594058546 wait complete. 2023-10-08T00:16:25.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.324+0000 7fef08359700 1 Processor -- start 2023-10-08T00:16:25.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.324+0000 7fef08359700 1 -- start start 2023-10-08T00:16:25.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.325+0000 7fef08359700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feeec003a60 0x7fef00153e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:25.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.325+0000 7fef08359700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef00150880 0x7fef0016b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:25.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.325+0000 7fef08359700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef000cd560 0x7fef0014bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:25.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.325+0000 7fef08359700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0016b680 con 0x7fef00150880 2023-10-08T00:16:25.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.325+0000 7fef08359700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0016a680 con 0x7fef000cd560 2023-10-08T00:16:25.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.325+0000 7fef08359700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef001519a0 con 0x7feeec003a60 2023-10-08T00:16:25.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.325+0000 7fef058f4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef00150880 0x7fef0016b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:25.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.325+0000 7fef058f4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef00150880 0x7fef0016b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37016/0 (socket says 172.21.15.99:37016) 2023-10-08T00:16:25.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.325+0000 7fef058f4700 1 -- 172.21.15.99:0/3561621295 learned_addr learned my addr 172.21.15.99:0/3561621295 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:25.333 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.325+0000 7fef068f6700 1 --2- 172.21.15.99:0/3561621295 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef000cd560 0x7fef0014bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:25.333 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.325+0000 7fef060f5700 1 --2- 172.21.15.99:0/3561621295 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feeec003a60 0x7fef00153e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:25.333 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.326+0000 7fef058f4700 1 -- 172.21.15.99:0/3561621295 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feeec003a60 msgr2=0x7fef00153e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:25.333 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.326+0000 7fef058f4700 1 --2- 172.21.15.99:0/3561621295 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feeec003a60 0x7fef00153e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:25.333 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.326+0000 7fef058f4700 1 -- 172.21.15.99:0/3561621295 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef000cd560 msgr2=0x7fef0014bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:25.333 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.326+0000 7fef058f4700 1 --2- 172.21.15.99:0/3561621295 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef000cd560 0x7fef0014bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:25.333 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.326+0000 7fef058f4700 1 -- 172.21.15.99:0/3561621295 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef00151c80 con 0x7fef00150880 2023-10-08T00:16:25.333 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.326+0000 7fef058f4700 1 --2- 172.21.15.99:0/3561621295 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef00150880 0x7fef0016b140 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7feef000eb90 tx=0x7feef0001d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:25.333 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.326+0000 7feef67fc700 1 -- 172.21.15.99:0/3561621295 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef0004a40 con 0x7fef00150880 2023-10-08T00:16:25.333 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.326+0000 7feef67fc700 1 -- 172.21.15.99:0/3561621295 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef0004bc0 con 0x7fef00150880 2023-10-08T00:16:25.333 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.326+0000 7fef08359700 1 -- 172.21.15.99:0/3561621295 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef0016a940 con 0x7fef00150880 2023-10-08T00:16:25.333 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.326+0000 7feef67fc700 1 -- 172.21.15.99:0/3561621295 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef0005050 con 0x7fef00150880 2023-10-08T00:16:25.334 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.326+0000 7fef08359700 1 -- 172.21.15.99:0/3561621295 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef000cd9d0 con 0x7fef00150880 2023-10-08T00:16:25.334 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.327+0000 7fef08359700 1 -- 172.21.15.99:0/3561621295 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef000cdc50 con 0x7fef00150880 2023-10-08T00:16:25.334 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.327+0000 7feef67fc700 1 -- 172.21.15.99:0/3561621295 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7feef00053c0 con 0x7fef00150880 2023-10-08T00:16:25.334 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.327+0000 7feef67fc700 1 --2- 172.21.15.99:0/3561621295 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7feee4066f30 0x7feee40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:25.334 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.328+0000 7feef67fc700 1 -- 172.21.15.99:0/3561621295 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5705+0+0 (secure 0 0 0) 0x7feef0084010 con 0x7fef00150880 2023-10-08T00:16:25.334 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.328+0000 7fef060f5700 1 --2- 172.21.15.99:0/3561621295 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7feee4066f30 0x7feee40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:25.334 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.328+0000 7fef060f5700 1 --2- 172.21.15.99:0/3561621295 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7feee4066f30 0x7feee40693e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7feefc001010 tx=0x7feefc015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:25.334 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.332+0000 7feef67fc700 1 -- 172.21.15.99:0/3561621295 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feef0051280 con 0x7fef00150880 2023-10-08T00:16:25.554 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.554+0000 7fef08359700 1 -- 172.21.15.99:0/3561621295 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef0004c3c0 con 0x7fef00150880 2023-10-08T00:16:25.555 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.554+0000 7feef67fc700 1 -- 172.21.15.99:0/3561621295 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4605 (secure 0 0 0) 0x7feef0019090 con 0x7fef00150880 2023-10-08T00:16:25.555 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:16:25.555 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":56,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5627,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/2782755379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":2782755379},{"type":"v1","addr":"172.21.15.191:6837","nonce":2782755379}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6166,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/2980541392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2980541392},{"type":"v1","addr":"172.21.15.99:6834","nonce":2980541392}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6184,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/1342206989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1342206989},{"type":"v1","addr":"172.21.15.191:6836","nonce":1342206989}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:16:21.527609+0000","modified":"2023-10-08T00:16:25.185437+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6747},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6747":{"gid":6747,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/1495315849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":1495315849},{"type":"v1","addr":"172.21.15.99:6835","nonce":1495315849}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-08T00:16:25.557 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.556+0000 7fef08359700 1 -- 172.21.15.99:0/3561621295 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7feee4066f30 msgr2=0x7feee40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:25.557 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.556+0000 7fef08359700 1 --2- 172.21.15.99:0/3561621295 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7feee4066f30 0x7feee40693e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7feefc001010 tx=0x7feefc015000 comp rx=0 tx=0).stop 2023-10-08T00:16:25.557 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.556+0000 7fef08359700 1 -- 172.21.15.99:0/3561621295 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef00150880 msgr2=0x7fef0016b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:25.557 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.556+0000 7fef08359700 1 --2- 172.21.15.99:0/3561621295 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef00150880 0x7fef0016b140 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7feef000eb90 tx=0x7feef0001d00 comp rx=0 tx=0).stop 2023-10-08T00:16:25.557 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.556+0000 7fef08359700 1 -- 172.21.15.99:0/3561621295 shutdown_connections 2023-10-08T00:16:25.558 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.556+0000 7fef08359700 1 --2- 172.21.15.99:0/3561621295 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7feee4066f30 0x7feee40693e0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:25.558 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.556+0000 7fef08359700 1 --2- 172.21.15.99:0/3561621295 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef000cd560 0x7fef0014bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:25.558 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.556+0000 7fef08359700 1 --2- 172.21.15.99:0/3561621295 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feeec003a60 0x7fef00153e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:25.558 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.556+0000 7fef08359700 1 --2- 172.21.15.99:0/3561621295 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef00150880 0x7fef0016b140 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:25.558 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.556+0000 7fef08359700 1 -- 172.21.15.99:0/3561621295 >> 172.21.15.99:0/3561621295 conn(0x7fef00068ce0 msgr2=0x7fef00159820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:25.558 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.556+0000 7fef08359700 1 -- 172.21.15.99:0/3561621295 shutdown_connections 2023-10-08T00:16:25.558 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.556+0000 7fef08359700 1 -- 172.21.15.99:0/3561621295 wait complete. 2023-10-08T00:16:25.558 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 56 2023-10-08T00:16:25.569 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:16:25.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.681+0000 7f34238fb700 1 Processor -- start 2023-10-08T00:16:25.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.681+0000 7f34238fb700 1 -- start start 2023-10-08T00:16:25.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.682+0000 7f34238fb700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f341c157870 0x7f341c15b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:25.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.682+0000 7f34238fb700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f341c157c40 0x7f341c158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:25.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.682+0000 7f34238fb700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f341c15cbb0 con 0x7f341c15a620 2023-10-08T00:16:25.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.682+0000 7f34238fb700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f341c15bfa0 con 0x7f341c157c40 2023-10-08T00:16:25.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.682+0000 7f34238fb700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f341c15c280 con 0x7f341c157870 2023-10-08T00:16:25.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.682+0000 7f3421e98700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f341c157c40 0x7f341c158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:25.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.682+0000 7f3420e96700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f341c15a620 0x7f341c15d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50660/0 (socket says 172.21.15.99:50660) 2023-10-08T00:16:25.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.682+0000 7f3420e96700 1 -- 172.21.15.99:0/599811380 learned_addr learned my addr 172.21.15.99:0/599811380 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:25.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.682+0000 7f3421697700 1 --2- 172.21.15.99:0/599811380 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f341c157870 0x7f341c15b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:25.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.683+0000 7f3421e98700 1 -- 172.21.15.99:0/599811380 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f341c157870 msgr2=0x7f341c15b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:25.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.683+0000 7f3421e98700 1 --2- 172.21.15.99:0/599811380 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f341c157870 0x7f341c15b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:25.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.683+0000 7f3421e98700 1 -- 172.21.15.99:0/599811380 >> v1:172.21.15.99:6789/0 conn(0x7f341c15a620 legacy=0x7f341c15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:25.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.683+0000 7f3421e98700 1 -- 172.21.15.99:0/599811380 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f341c15c560 con 0x7f341c157c40 2023-10-08T00:16:25.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.683+0000 7f3413fff700 1 -- 172.21.15.99:0/599811380 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4053418798 0 0) 0x7f341c15cbb0 con 0x7f341c15a620 2023-10-08T00:16:25.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.683+0000 7f3421e98700 1 --2- 172.21.15.99:0/599811380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f341c157c40 0x7f341c158110 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f34140098e0 tx=0x7f34140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a12fde7496ae9db4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:25.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.683+0000 7f3413fff700 1 -- 172.21.15.99:0/599811380 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341400b640 con 0x7f341c157c40 2023-10-08T00:16:25.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.683+0000 7f3413fff700 1 -- 172.21.15.99:0/599811380 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f341400b7a0 con 0x7f341c157c40 2023-10-08T00:16:25.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.683+0000 7f3413fff700 1 -- 172.21.15.99:0/599811380 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341400bb40 con 0x7f341c157c40 2023-10-08T00:16:25.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.683+0000 7f34238fb700 1 -- 172.21.15.99:0/599811380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f341c157c40 msgr2=0x7f341c158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:25.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.683+0000 7f34238fb700 1 --2- 172.21.15.99:0/599811380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f341c157c40 0x7f341c158110 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f34140098e0 tx=0x7f34140091a0 comp rx=0 tx=0).stop 2023-10-08T00:16:25.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.684+0000 7f34238fb700 1 -- 172.21.15.99:0/599811380 shutdown_connections 2023-10-08T00:16:25.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.684+0000 7f34238fb700 1 --2- 172.21.15.99:0/599811380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f341c157c40 0x7f341c158110 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:25.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.684+0000 7f34238fb700 1 --2- 172.21.15.99:0/599811380 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f341c157870 0x7f341c15b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:25.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.684+0000 7f34238fb700 1 -- 172.21.15.99:0/599811380 >> 172.21.15.99:0/599811380 conn(0x7f341c068ce0 msgr2=0x7f341c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:25.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.684+0000 7f34238fb700 1 -- 172.21.15.99:0/599811380 shutdown_connections 2023-10-08T00:16:25.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.684+0000 7f34238fb700 1 -- 172.21.15.99:0/599811380 wait complete. 2023-10-08T00:16:25.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.685+0000 7f34238fb700 1 Processor -- start 2023-10-08T00:16:25.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.685+0000 7f34238fb700 1 -- start start 2023-10-08T00:16:25.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.686+0000 7f34238fb700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f341c157870 0x7f341c14e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:25.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.686+0000 7f34238fb700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f341c157c40 0x7f341c141fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:25.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.686+0000 7f34238fb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f341c15a620 0x7f341c0b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:25.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.686+0000 7f34238fb700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341c153110 con 0x7f341c15a620 2023-10-08T00:16:25.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.686+0000 7f34238fb700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341c153c90 con 0x7f341c157870 2023-10-08T00:16:25.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.686+0000 7f34238fb700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f341c14e770 con 0x7f341c157c40 2023-10-08T00:16:25.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.686+0000 7f3421e98700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f341c15a620 0x7f341c0b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:25.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.686+0000 7f3421e98700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f341c15a620 0x7f341c0b2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37040/0 (socket says 172.21.15.99:37040) 2023-10-08T00:16:25.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.686+0000 7f3421e98700 1 -- 172.21.15.99:0/1041732060 learned_addr learned my addr 172.21.15.99:0/1041732060 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:25.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.686+0000 7f3421697700 1 --2- 172.21.15.99:0/1041732060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f341c157870 0x7f341c14e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:25.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.686+0000 7f3420e96700 1 --2- 172.21.15.99:0/1041732060 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f341c157c40 0x7f341c141fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:25.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.686+0000 7f3421e98700 1 -- 172.21.15.99:0/1041732060 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f341c157c40 msgr2=0x7f341c141fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:25.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.686+0000 7f3421e98700 1 --2- 172.21.15.99:0/1041732060 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f341c157c40 0x7f341c141fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:25.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.686+0000 7f3421e98700 1 -- 172.21.15.99:0/1041732060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f341c157870 msgr2=0x7f341c14e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:25.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.686+0000 7f3421e98700 1 --2- 172.21.15.99:0/1041732060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f341c157870 0x7f341c14e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:25.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.686+0000 7f3421e98700 1 -- 172.21.15.99:0/1041732060 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3414009420 con 0x7f341c15a620 2023-10-08T00:16:25.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.686+0000 7f3421697700 1 --2- 172.21.15.99:0/1041732060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f341c157870 0x7f341c14e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:16:25.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.686+0000 7f3420e96700 1 --2- 172.21.15.99:0/1041732060 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f341c157c40 0x7f341c141fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:16:25.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.686+0000 7f3421e98700 1 --2- 172.21.15.99:0/1041732060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f341c15a620 0x7f341c0b2e30 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f3414009f80 tx=0x7f3414017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:25.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.686+0000 7f34117fa700 1 -- 172.21.15.99:0/1041732060 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3414011420 con 0x7f341c15a620 2023-10-08T00:16:25.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.686+0000 7f34117fa700 1 -- 172.21.15.99:0/1041732060 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3414011580 con 0x7f341c15a620 2023-10-08T00:16:25.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.686+0000 7f34238fb700 1 -- 172.21.15.99:0/1041732060 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f341c153360 con 0x7f341c15a620 2023-10-08T00:16:25.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.686+0000 7f34117fa700 1 -- 172.21.15.99:0/1041732060 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3414011920 con 0x7f341c15a620 2023-10-08T00:16:25.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.686+0000 7f34238fb700 1 -- 172.21.15.99:0/1041732060 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f341c153920 con 0x7f341c15a620 2023-10-08T00:16:25.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.687+0000 7f34238fb700 1 -- 172.21.15.99:0/1041732060 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f341c153e20 con 0x7f341c15a620 2023-10-08T00:16:25.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.687+0000 7f34117fa700 1 -- 172.21.15.99:0/1041732060 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3414011c90 con 0x7f341c15a620 2023-10-08T00:16:25.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.688+0000 7f34117fa700 1 --2- 172.21.15.99:0/1041732060 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3400066f30 0x7f34000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:25.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.688+0000 7f34117fa700 1 -- 172.21.15.99:0/1041732060 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3414086120 con 0x7f341c15a620 2023-10-08T00:16:25.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.688+0000 7f3421697700 1 --2- 172.21.15.99:0/1041732060 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3400066f30 0x7f34000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:25.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.689+0000 7f3421697700 1 --2- 172.21.15.99:0/1041732060 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3400066f30 0x7f34000693e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f3418007780 tx=0x7f341800b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:25.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.692+0000 7f34117fa700 1 -- 172.21.15.99:0/1041732060 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3414053390 con 0x7f341c15a620 2023-10-08T00:16:25.875 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.875+0000 7f34238fb700 1 -- 172.21.15.99:0/1041732060 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f341c04c3c0 con 0x7f341c15a620 2023-10-08T00:16:25.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.876+0000 7f34117fa700 1 -- 172.21.15.99:0/1041732060 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+15365 (secure 0 0 0) 0x7f34140569b0 con 0x7f341c15a620 2023-10-08T00:16:25.876 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:16:25.876 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":124,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:16:23.151406+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:16:19.856259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-08T00:16:20.494851+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:16:25.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.877+0000 7f34238fb700 1 -- 172.21.15.99:0/1041732060 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3400066f30 msgr2=0x7f34000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:25.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.877+0000 7f34238fb700 1 --2- 172.21.15.99:0/1041732060 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3400066f30 0x7f34000693e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f3418007780 tx=0x7f341800b040 comp rx=0 tx=0).stop 2023-10-08T00:16:25.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.877+0000 7f34238fb700 1 -- 172.21.15.99:0/1041732060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f341c15a620 msgr2=0x7f341c0b2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:25.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.877+0000 7f34238fb700 1 --2- 172.21.15.99:0/1041732060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f341c15a620 0x7f341c0b2e30 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f3414009f80 tx=0x7f3414017000 comp rx=0 tx=0).stop 2023-10-08T00:16:25.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.878+0000 7f34238fb700 1 -- 172.21.15.99:0/1041732060 shutdown_connections 2023-10-08T00:16:25.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.878+0000 7f34238fb700 1 --2- 172.21.15.99:0/1041732060 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3400066f30 0x7f34000693e0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:25.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.878+0000 7f34238fb700 1 --2- 172.21.15.99:0/1041732060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f341c157870 0x7f341c14e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:25.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.878+0000 7f34238fb700 1 --2- 172.21.15.99:0/1041732060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f341c15a620 0x7f341c0b2e30 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:25.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.878+0000 7f34238fb700 1 --2- 172.21.15.99:0/1041732060 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f341c157c40 0x7f341c141fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:25.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.878+0000 7f34238fb700 1 -- 172.21.15.99:0/1041732060 >> 172.21.15.99:0/1041732060 conn(0x7f341c068ce0 msgr2=0x7f341c0d1010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:25.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.878+0000 7f34238fb700 1 -- 172.21.15.99:0/1041732060 shutdown_connections 2023-10-08T00:16:25.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:25.878+0000 7f34238fb700 1 -- 172.21.15.99:0/1041732060 wait complete. 2023-10-08T00:16:25.890 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T00:16:25.891 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T00:16:26.005 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.004+0000 7f5cd9813700 1 Processor -- start 2023-10-08T00:16:26.005 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.004+0000 7f5cd9813700 1 -- start start 2023-10-08T00:16:26.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.005+0000 7f5cd9813700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f5cd415a620 0x7f5cd415d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:26.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.005+0000 7f5cd9813700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cd415cbb0 con 0x7f5cd415d580 2023-10-08T00:16:26.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.005+0000 7f5cd9813700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cd415bfa0 con 0x7f5cd4157870 2023-10-08T00:16:26.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.005+0000 7f5cd9813700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cd415bdb0 con 0x7f5cd415a620 2023-10-08T00:16:26.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.005+0000 7f5cd37fe700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f5cd415d580 0x7f5cd4158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50664/0 (socket says 172.21.15.99:50664) 2023-10-08T00:16:26.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.005+0000 7f5cd37fe700 1 -- 172.21.15.99:0/3843051390 learned_addr learned my addr 172.21.15.99:0/3843051390 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:26.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.005+0000 7f5cca7fc700 1 --2- 172.21.15.99:0/3843051390 >> v2:172.21.15.191:3300/0 conn(0x7f5cd415a620 0x7f5cd415d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:26.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.005+0000 7f5cd27fc700 1 -- 172.21.15.99:0/3843051390 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2752013583 0 0) 0x7f5cd415cbb0 con 0x7f5cd415d580 2023-10-08T00:16:26.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.005+0000 7f5cd27fc700 1 -- 172.21.15.99:0/3843051390 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cc0003570 con 0x7f5cd415d580 2023-10-08T00:16:26.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.005+0000 7f5cd27fc700 1 -- 172.21.15.99:0/3843051390 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1314696302 0 0) 0x7f5cc0003570 con 0x7f5cd415d580 2023-10-08T00:16:26.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.005+0000 7f5cd27fc700 1 -- 172.21.15.99:0/3843051390 >> v2:172.21.15.191:3300/0 conn(0x7f5cd415a620 msgr2=0x7f5cd415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:26.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.005+0000 7f5cd27fc700 1 --2- 172.21.15.99:0/3843051390 >> v2:172.21.15.191:3300/0 conn(0x7f5cd415a620 0x7f5cd415d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:26.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.005+0000 7f5cd27fc700 1 -- 172.21.15.99:0/3843051390 >> v1:172.21.15.191:6789/0 conn(0x7f5cd4157870 legacy=0x7f5cd415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:26.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.006+0000 7f5cd27fc700 1 -- 172.21.15.99:0/3843051390 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd415c280 con 0x7f5cd415d580 2023-10-08T00:16:26.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.006+0000 7f5cd27fc700 1 -- 172.21.15.99:0/3843051390 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f5ccc0017f0 con 0x7f5cd415d580 2023-10-08T00:16:26.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.006+0000 7f5cd27fc700 1 -- 172.21.15.99:0/3843051390 >> v1:172.21.15.99:6789/0 conn(0x7f5cd415d580 legacy=0x7f5cd4158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:26.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.006+0000 7f5cd27fc700 1 --2- 172.21.15.99:0/3843051390 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5cc0003a60 0x7f5cc0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:26.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.006+0000 7f5cd27fc700 1 --2- 172.21.15.99:0/3843051390 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5cc0004810 0x7f5cc0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:26.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.006+0000 7f5cd27fc700 1 --2- 172.21.15.99:0/3843051390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cc00051c0 0x7f5cc000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:26.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.006+0000 7f5cd27fc700 1 -- 172.21.15.99:0/3843051390 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc000c130 con 0x7f5cc00051c0 2023-10-08T00:16:26.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.006+0000 7f5cd27fc700 1 -- 172.21.15.99:0/3843051390 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc000c2a0 con 0x7f5cc0004810 2023-10-08T00:16:26.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.006+0000 7f5cd27fc700 1 -- 172.21.15.99:0/3843051390 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc000c460 con 0x7f5cc0003a60 2023-10-08T00:16:26.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.006+0000 7f5cd37fe700 1 --2- 172.21.15.99:0/3843051390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cc00051c0 0x7f5cc000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:26.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.006+0000 7f5cca7fc700 1 --2- 172.21.15.99:0/3843051390 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5cc0004810 0x7f5cc0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:26.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.006+0000 7f5cd2ffd700 1 --2- 172.21.15.99:0/3843051390 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5cc0003a60 0x7f5cc0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:26.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.006+0000 7f5cd37fe700 1 -- 172.21.15.99:0/3843051390 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5cc0003a60 msgr2=0x7f5cc0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:26.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.006+0000 7f5cd37fe700 1 --2- 172.21.15.99:0/3843051390 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5cc0003a60 0x7f5cc0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:26.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.006+0000 7f5cd37fe700 1 -- 172.21.15.99:0/3843051390 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5cc0004810 msgr2=0x7f5cc0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:26.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.006+0000 7f5cd37fe700 1 --2- 172.21.15.99:0/3843051390 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5cc0004810 0x7f5cc0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:26.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.006+0000 7f5cd37fe700 1 -- 172.21.15.99:0/3843051390 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc0003570 con 0x7f5cc00051c0 2023-10-08T00:16:26.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.006+0000 7f5cd2ffd700 1 --2- 172.21.15.99:0/3843051390 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5cc0003a60 0x7f5cc0003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:26.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.006+0000 7f5cd37fe700 1 -- 172.21.15.99:0/3843051390 reap_dead start 2023-10-08T00:16:26.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.006+0000 7f5cd37fe700 1 --2- 172.21.15.99:0/3843051390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cc00051c0 0x7f5cc000ba60 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc00a900 tx=0x7f5ccc0021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c05ff327b52cd5bc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:26.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.006+0000 7f5cd27fc700 1 -- 172.21.15.99:0/3843051390 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc0070d0 con 0x7f5cc00051c0 2023-10-08T00:16:26.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.006+0000 7f5cd27fc700 1 -- 172.21.15.99:0/3843051390 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ccc007230 con 0x7f5cc00051c0 2023-10-08T00:16:26.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.006+0000 7f5cd27fc700 1 -- 172.21.15.99:0/3843051390 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc0075d0 con 0x7f5cc00051c0 2023-10-08T00:16:26.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.007+0000 7f5cd9813700 1 -- 172.21.15.99:0/3843051390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cc00051c0 msgr2=0x7f5cc000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:26.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.007+0000 7f5cd9813700 1 --2- 172.21.15.99:0/3843051390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cc00051c0 0x7f5cc000ba60 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc00a900 tx=0x7f5ccc0021d0 comp rx=0 tx=0).stop 2023-10-08T00:16:26.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.007+0000 7f5cd9813700 1 -- 172.21.15.99:0/3843051390 shutdown_connections 2023-10-08T00:16:26.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.007+0000 7f5cd9813700 1 --2- 172.21.15.99:0/3843051390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cc00051c0 0x7f5cc000ba60 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:26.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.007+0000 7f5cd9813700 1 -- 172.21.15.99:0/3843051390 >> 172.21.15.99:0/3843051390 conn(0x7f5cd4068ce0 msgr2=0x7f5cd40690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:26.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.007+0000 7f5cd9813700 1 -- 172.21.15.99:0/3843051390 shutdown_connections 2023-10-08T00:16:26.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.007+0000 7f5cd9813700 1 -- 172.21.15.99:0/3843051390 wait complete. 2023-10-08T00:16:26.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.008+0000 7f5cd9813700 1 Processor -- start 2023-10-08T00:16:26.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.008+0000 7f5cd9813700 1 -- start start 2023-10-08T00:16:26.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.009+0000 7f5cd9813700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cc00051c0 0x7f5cd40cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:26.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.009+0000 7f5cd9813700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5cd40c5b10 0x7f5cd40c5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:26.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.009+0000 7f5cd9813700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5cd4158930 0x7f5cd4158d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:26.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.009+0000 7f5cd9813700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd40c64c0 con 0x7f5cc00051c0 2023-10-08T00:16:26.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.009+0000 7f5cd9813700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd40cd900 con 0x7f5cd4158930 2023-10-08T00:16:26.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.009+0000 7f5cd9813700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd40b2150 con 0x7f5cd40c5b10 2023-10-08T00:16:26.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.009+0000 7f5cd2ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cc00051c0 0x7f5cd40cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:26.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.009+0000 7f5cd2ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cc00051c0 0x7f5cd40cd3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37048/0 (socket says 172.21.15.99:37048) 2023-10-08T00:16:26.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.009+0000 7f5cd2ffd700 1 -- 172.21.15.99:0/3668283722 learned_addr learned my addr 172.21.15.99:0/3668283722 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:26.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.009+0000 7f5cca7fc700 1 --2- 172.21.15.99:0/3668283722 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5cd40c5b10 0x7f5cd40c5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:26.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.009+0000 7f5cd37fe700 1 --2- 172.21.15.99:0/3668283722 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5cd4158930 0x7f5cd4158d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:26.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.009+0000 7f5cd2ffd700 1 -- 172.21.15.99:0/3668283722 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5cd40c5b10 msgr2=0x7f5cd40c5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:26.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.009+0000 7f5cd2ffd700 1 --2- 172.21.15.99:0/3668283722 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5cd40c5b10 0x7f5cd40c5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:26.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.009+0000 7f5cd2ffd700 1 -- 172.21.15.99:0/3668283722 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5cd4158930 msgr2=0x7f5cd4158d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:26.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.009+0000 7f5cd2ffd700 1 --2- 172.21.15.99:0/3668283722 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5cd4158930 0x7f5cd4158d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:26.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.009+0000 7f5cd2ffd700 1 -- 172.21.15.99:0/3668283722 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd41596f0 con 0x7f5cc00051c0 2023-10-08T00:16:26.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.009+0000 7f5cd37fe700 1 --2- 172.21.15.99:0/3668283722 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5cd4158930 0x7f5cd4158d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:16:26.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.009+0000 7f5cd2ffd700 1 --2- 172.21.15.99:0/3668283722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cc00051c0 0x7f5cd40cd3c0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f5cc400e780 tx=0x7f5cc4009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:26.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.009+0000 7f5ccb7fe700 1 -- 172.21.15.99:0/3668283722 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4010490 con 0x7f5cc00051c0 2023-10-08T00:16:26.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.009+0000 7f5ccb7fe700 1 -- 172.21.15.99:0/3668283722 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cc4010040 con 0x7f5cc00051c0 2023-10-08T00:16:26.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.009+0000 7f5cd9813700 1 -- 172.21.15.99:0/3668283722 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cd40b1820 con 0x7f5cc00051c0 2023-10-08T00:16:26.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.009+0000 7f5ccb7fe700 1 -- 172.21.15.99:0/3668283722 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4010820 con 0x7f5cc00051c0 2023-10-08T00:16:26.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.010+0000 7f5cd9813700 1 -- 172.21.15.99:0/3668283722 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cd40b1de0 con 0x7f5cc00051c0 2023-10-08T00:16:26.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.010+0000 7f5cd9813700 1 -- 172.21.15.99:0/3668283722 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cd40cdca0 con 0x7f5cc00051c0 2023-10-08T00:16:26.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.011+0000 7f5ccb7fe700 1 -- 172.21.15.99:0/3668283722 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5cc4010b90 con 0x7f5cc00051c0 2023-10-08T00:16:26.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.011+0000 7f5ccb7fe700 1 --2- 172.21.15.99:0/3668283722 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5cb4066f30 0x7f5cb40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:26.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.011+0000 7f5ccb7fe700 1 -- 172.21.15.99:0/3668283722 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5cc4083810 con 0x7f5cc00051c0 2023-10-08T00:16:26.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.011+0000 7f5cca7fc700 1 --2- 172.21.15.99:0/3668283722 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5cb4066f30 0x7f5cb40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:26.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.012+0000 7f5cca7fc700 1 --2- 172.21.15.99:0/3668283722 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5cb4066f30 0x7f5cb40693e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f5cd40b65d0 tx=0x7f5cbc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:26.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.015+0000 7f5ccb7fe700 1 -- 172.21.15.99:0/3668283722 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5cc404d5a0 con 0x7f5cc00051c0 2023-10-08T00:16:26.195 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.194+0000 7f5cd9813700 1 -- 172.21.15.99:0/3668283722 --> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5cd40b2290 con 0x7f5cb4066f30 2023-10-08T00:16:26.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.203+0000 7f5ccb7fe700 1 -- 172.21.15.99:0/3668283722 <== mgr.4099 v2:172.21.15.191:6832/85985 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285657 (secure 0 0 0) 0x7f5cbc009da0 con 0x7f5cb4066f30 2023-10-08T00:16:26.207 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:16:26.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.206+0000 7f5cd9813700 1 -- 172.21.15.99:0/3668283722 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5cb4066f30 msgr2=0x7f5cb40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:26.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.206+0000 7f5cd9813700 1 --2- 172.21.15.99:0/3668283722 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5cb4066f30 0x7f5cb40693e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f5cd40b65d0 tx=0x7f5cbc008040 comp rx=0 tx=0).stop 2023-10-08T00:16:26.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.206+0000 7f5cd9813700 1 -- 172.21.15.99:0/3668283722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cc00051c0 msgr2=0x7f5cd40cd3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:26.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.206+0000 7f5cd9813700 1 --2- 172.21.15.99:0/3668283722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cc00051c0 0x7f5cd40cd3c0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f5cc400e780 tx=0x7f5cc4009040 comp rx=0 tx=0).stop 2023-10-08T00:16:26.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.206+0000 7f5cd9813700 1 -- 172.21.15.99:0/3668283722 shutdown_connections 2023-10-08T00:16:26.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.206+0000 7f5cd9813700 1 --2- 172.21.15.99:0/3668283722 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5cb4066f30 0x7f5cb40693e0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:26.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.206+0000 7f5cd9813700 1 --2- 172.21.15.99:0/3668283722 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5cd4158930 0x7f5cd4158d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:26.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.206+0000 7f5cd9813700 1 --2- 172.21.15.99:0/3668283722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cc00051c0 0x7f5cd40cd3c0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:26.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.206+0000 7f5cd9813700 1 --2- 172.21.15.99:0/3668283722 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5cd40c5b10 0x7f5cd40c5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:26.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.206+0000 7f5cd9813700 1 -- 172.21.15.99:0/3668283722 >> 172.21.15.99:0/3668283722 conn(0x7f5cd4068ce0 msgr2=0x7f5cd415f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:26.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.206+0000 7f5cd9813700 1 -- 172.21.15.99:0/3668283722 shutdown_connections 2023-10-08T00:16:26.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.206+0000 7f5cd9813700 1 -- 172.21.15.99:0/3668283722 wait complete. 2023-10-08T00:16:26.210 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2023-10-08T00:16:26.218 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":286,"stamp":"2023-10-08T00:16:26.103515+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":92,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6453,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1556046,"statfs":{"total":1600000000,"available":1593391876,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.957491"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.168926+0000","last_change":"2023-10-08T00:16:22.168926+0000","last_active":"2023-10-08T00:16:22.168926+0000","last_peered":"2023-10-08T00:16:22.168926+0000","last_clean":"2023-10-08T00:16:22.168926+0000","last_became_active":"2023-10-08T00:16:22.168678+0000","last_became_peered":"2023-10-08T00:16:22.168678+0000","last_unstale":"2023-10-08T00:16:22.168926+0000","last_undegraded":"2023-10-08T00:16:22.168926+0000","last_fullsized":"2023-10-08T00:16:22.168926+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:50:42.638829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.166603+0000","last_change":"2023-10-08T00:16:22.166603+0000","last_active":"2023-10-08T00:16:22.166603+0000","last_peered":"2023-10-08T00:16:22.166603+0000","last_clean":"2023-10-08T00:16:22.166603+0000","last_became_active":"2023-10-08T00:16:22.166090+0000","last_became_peered":"2023-10-08T00:16:22.166090+0000","last_unstale":"2023-10-08T00:16:22.166603+0000","last_undegraded":"2023-10-08T00:16:22.166603+0000","last_fullsized":"2023-10-08T00:16:22.166603+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:35:16.330010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.169784+0000","last_change":"2023-10-08T00:16:22.169784+0000","last_active":"2023-10-08T00:16:22.169784+0000","last_peered":"2023-10-08T00:16:22.169784+0000","last_clean":"2023-10-08T00:16:22.169784+0000","last_became_active":"2023-10-08T00:16:22.166646+0000","last_became_peered":"2023-10-08T00:16:22.166646+0000","last_unstale":"2023-10-08T00:16:22.169784+0000","last_undegraded":"2023-10-08T00:16:22.169784+0000","last_fullsized":"2023-10-08T00:16:22.169784+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:32:41.484943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.162701+0000","last_change":"2023-10-08T00:16:22.162701+0000","last_active":"2023-10-08T00:16:22.162701+0000","last_peered":"2023-10-08T00:16:22.162701+0000","last_clean":"2023-10-08T00:16:22.162701+0000","last_became_active":"2023-10-08T00:16:22.162131+0000","last_became_peered":"2023-10-08T00:16:22.162131+0000","last_unstale":"2023-10-08T00:16:22.162701+0000","last_undegraded":"2023-10-08T00:16:22.162701+0000","last_fullsized":"2023-10-08T00:16:22.162701+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:35:56.520235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.163038+0000","last_change":"2023-10-08T00:16:22.163038+0000","last_active":"2023-10-08T00:16:22.163038+0000","last_peered":"2023-10-08T00:16:22.163038+0000","last_clean":"2023-10-08T00:16:22.163038+0000","last_became_active":"2023-10-08T00:16:22.162809+0000","last_became_peered":"2023-10-08T00:16:22.162809+0000","last_unstale":"2023-10-08T00:16:22.163038+0000","last_undegraded":"2023-10-08T00:16:22.163038+0000","last_fullsized":"2023-10-08T00:16:22.163038+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:52:50.789133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.164226+0000","last_change":"2023-10-08T00:16:22.164226+0000","last_active":"2023-10-08T00:16:22.164226+0000","last_peered":"2023-10-08T00:16:22.164226+0000","last_clean":"2023-10-08T00:16:22.164226+0000","last_became_active":"2023-10-08T00:16:22.163362+0000","last_became_peered":"2023-10-08T00:16:22.163362+0000","last_unstale":"2023-10-08T00:16:22.164226+0000","last_undegraded":"2023-10-08T00:16:22.164226+0000","last_fullsized":"2023-10-08T00:16:22.164226+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:47:35.001042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.164274+0000","last_change":"2023-10-08T00:16:22.164274+0000","last_active":"2023-10-08T00:16:22.164274+0000","last_peered":"2023-10-08T00:16:22.164274+0000","last_clean":"2023-10-08T00:16:22.164274+0000","last_became_active":"2023-10-08T00:16:22.164026+0000","last_became_peered":"2023-10-08T00:16:22.164026+0000","last_unstale":"2023-10-08T00:16:22.164274+0000","last_undegraded":"2023-10-08T00:16:22.164274+0000","last_fullsized":"2023-10-08T00:16:22.164274+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:51:24.231446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.169877+0000","last_change":"2023-10-08T00:16:22.169877+0000","last_active":"2023-10-08T00:16:22.169877+0000","last_peered":"2023-10-08T00:16:22.169877+0000","last_clean":"2023-10-08T00:16:22.169877+0000","last_became_active":"2023-10-08T00:16:22.166990+0000","last_became_peered":"2023-10-08T00:16:22.166990+0000","last_unstale":"2023-10-08T00:16:22.169877+0000","last_undegraded":"2023-10-08T00:16:22.169877+0000","last_fullsized":"2023-10-08T00:16:22.169877+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:09:25.675403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.162665+0000","last_change":"2023-10-08T00:16:22.162665+0000","last_active":"2023-10-08T00:16:22.162665+0000","last_peered":"2023-10-08T00:16:22.162665+0000","last_clean":"2023-10-08T00:16:22.162665+0000","last_became_active":"2023-10-08T00:16:22.162451+0000","last_became_peered":"2023-10-08T00:16:22.162451+0000","last_unstale":"2023-10-08T00:16:22.162665+0000","last_undegraded":"2023-10-08T00:16:22.162665+0000","last_fullsized":"2023-10-08T00:16:22.162665+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:21:05.119155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.168752+0000","last_change":"2023-10-08T00:16:22.168752+0000","last_active":"2023-10-08T00:16:22.168752+0000","last_peered":"2023-10-08T00:16:22.168752+0000","last_clean":"2023-10-08T00:16:22.168752+0000","last_became_active":"2023-10-08T00:16:22.168518+0000","last_became_peered":"2023-10-08T00:16:22.168518+0000","last_unstale":"2023-10-08T00:16:22.168752+0000","last_undegraded":"2023-10-08T00:16:22.168752+0000","last_fullsized":"2023-10-08T00:16:22.168752+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:36:52.374149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.164446+0000","last_change":"2023-10-08T00:16:22.164446+0000","last_active":"2023-10-08T00:16:22.164446+0000","last_peered":"2023-10-08T00:16:22.164446+0000","last_clean":"2023-10-08T00:16:22.164446+0000","last_became_active":"2023-10-08T00:16:22.163950+0000","last_became_peered":"2023-10-08T00:16:22.163950+0000","last_unstale":"2023-10-08T00:16:22.164446+0000","last_undegraded":"2023-10-08T00:16:22.164446+0000","last_fullsized":"2023-10-08T00:16:22.164446+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:42:45.613813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.167538+0000","last_change":"2023-10-08T00:16:22.167538+0000","last_active":"2023-10-08T00:16:22.167538+0000","last_peered":"2023-10-08T00:16:22.167538+0000","last_clean":"2023-10-08T00:16:22.167538+0000","last_became_active":"2023-10-08T00:16:22.167134+0000","last_became_peered":"2023-10-08T00:16:22.167134+0000","last_unstale":"2023-10-08T00:16:22.167538+0000","last_undegraded":"2023-10-08T00:16:22.167538+0000","last_fullsized":"2023-10-08T00:16:22.167538+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:54:43.701059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.156907+0000","last_change":"2023-10-08T00:16:22.156907+0000","last_active":"2023-10-08T00:16:22.156907+0000","last_peered":"2023-10-08T00:16:22.156907+0000","last_clean":"2023-10-08T00:16:22.156907+0000","last_became_active":"2023-10-08T00:16:22.156691+0000","last_became_peered":"2023-10-08T00:16:22.156691+0000","last_unstale":"2023-10-08T00:16:22.156907+0000","last_undegraded":"2023-10-08T00:16:22.156907+0000","last_fullsized":"2023-10-08T00:16:22.156907+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:32:41.484943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.162851+0000","last_change":"2023-10-08T00:16:22.162851+0000","last_active":"2023-10-08T00:16:22.162851+0000","last_peered":"2023-10-08T00:16:22.162851+0000","last_clean":"2023-10-08T00:16:22.162851+0000","last_became_active":"2023-10-08T00:16:22.162466+0000","last_became_peered":"2023-10-08T00:16:22.162466+0000","last_unstale":"2023-10-08T00:16:22.162851+0000","last_undegraded":"2023-10-08T00:16:22.162851+0000","last_fullsized":"2023-10-08T00:16:22.162851+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:05:55.518464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.166798+0000","last_change":"2023-10-08T00:16:22.166798+0000","last_active":"2023-10-08T00:16:22.166798+0000","last_peered":"2023-10-08T00:16:22.166798+0000","last_clean":"2023-10-08T00:16:22.166798+0000","last_became_active":"2023-10-08T00:16:22.166492+0000","last_became_peered":"2023-10-08T00:16:22.166492+0000","last_unstale":"2023-10-08T00:16:22.166798+0000","last_undegraded":"2023-10-08T00:16:22.166798+0000","last_fullsized":"2023-10-08T00:16:22.166798+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:44:20.663579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.159081+0000","last_change":"2023-10-08T00:16:22.159081+0000","last_active":"2023-10-08T00:16:22.159081+0000","last_peered":"2023-10-08T00:16:22.159081+0000","last_clean":"2023-10-08T00:16:22.159081+0000","last_became_active":"2023-10-08T00:16:22.158851+0000","last_became_peered":"2023-10-08T00:16:22.158851+0000","last_unstale":"2023-10-08T00:16:22.159081+0000","last_undegraded":"2023-10-08T00:16:22.159081+0000","last_fullsized":"2023-10-08T00:16:22.159081+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:48:39.888939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.158983+0000","last_change":"2023-10-08T00:16:22.158983+0000","last_active":"2023-10-08T00:16:22.158983+0000","last_peered":"2023-10-08T00:16:22.158983+0000","last_clean":"2023-10-08T00:16:22.158983+0000","last_became_active":"2023-10-08T00:16:22.158200+0000","last_became_peered":"2023-10-08T00:16:22.158200+0000","last_unstale":"2023-10-08T00:16:22.158983+0000","last_undegraded":"2023-10-08T00:16:22.158983+0000","last_fullsized":"2023-10-08T00:16:22.158983+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:21:44.003712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.165593+0000","last_change":"2023-10-08T00:16:22.165593+0000","last_active":"2023-10-08T00:16:22.165593+0000","last_peered":"2023-10-08T00:16:22.165593+0000","last_clean":"2023-10-08T00:16:22.165593+0000","last_became_active":"2023-10-08T00:16:22.165367+0000","last_became_peered":"2023-10-08T00:16:22.165367+0000","last_unstale":"2023-10-08T00:16:22.165593+0000","last_undegraded":"2023-10-08T00:16:22.165593+0000","last_fullsized":"2023-10-08T00:16:22.165593+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:53:34.263422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.165305+0000","last_change":"2023-10-08T00:16:22.165305+0000","last_active":"2023-10-08T00:16:22.165305+0000","last_peered":"2023-10-08T00:16:22.165305+0000","last_clean":"2023-10-08T00:16:22.165305+0000","last_became_active":"2023-10-08T00:16:22.164891+0000","last_became_peered":"2023-10-08T00:16:22.164891+0000","last_unstale":"2023-10-08T00:16:22.165305+0000","last_undegraded":"2023-10-08T00:16:22.165305+0000","last_fullsized":"2023-10-08T00:16:22.165305+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:40:12.567676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.164925+0000","last_change":"2023-10-08T00:16:22.164925+0000","last_active":"2023-10-08T00:16:22.164925+0000","last_peered":"2023-10-08T00:16:22.164925+0000","last_clean":"2023-10-08T00:16:22.164925+0000","last_became_active":"2023-10-08T00:16:22.164785+0000","last_became_peered":"2023-10-08T00:16:22.164785+0000","last_unstale":"2023-10-08T00:16:22.164925+0000","last_undegraded":"2023-10-08T00:16:22.164925+0000","last_fullsized":"2023-10-08T00:16:22.164925+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:26:37.495930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.168020+0000","last_change":"2023-10-08T00:16:22.168020+0000","last_active":"2023-10-08T00:16:22.168020+0000","last_peered":"2023-10-08T00:16:22.168020+0000","last_clean":"2023-10-08T00:16:22.168020+0000","last_became_active":"2023-10-08T00:16:22.167398+0000","last_became_peered":"2023-10-08T00:16:22.167398+0000","last_unstale":"2023-10-08T00:16:22.168020+0000","last_undegraded":"2023-10-08T00:16:22.168020+0000","last_fullsized":"2023-10-08T00:16:22.168020+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:59:13.288375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.161097+0000","last_change":"2023-10-08T00:16:22.161097+0000","last_active":"2023-10-08T00:16:22.161097+0000","last_peered":"2023-10-08T00:16:22.161097+0000","last_clean":"2023-10-08T00:16:22.161097+0000","last_became_active":"2023-10-08T00:16:22.160843+0000","last_became_peered":"2023-10-08T00:16:22.160843+0000","last_unstale":"2023-10-08T00:16:22.161097+0000","last_undegraded":"2023-10-08T00:16:22.161097+0000","last_fullsized":"2023-10-08T00:16:22.161097+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:34:33.394447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.168529+0000","last_change":"2023-10-08T00:16:22.168529+0000","last_active":"2023-10-08T00:16:22.168529+0000","last_peered":"2023-10-08T00:16:22.168529+0000","last_clean":"2023-10-08T00:16:22.168529+0000","last_became_active":"2023-10-08T00:16:22.167019+0000","last_became_peered":"2023-10-08T00:16:22.167019+0000","last_unstale":"2023-10-08T00:16:22.168529+0000","last_undegraded":"2023-10-08T00:16:22.168529+0000","last_fullsized":"2023-10-08T00:16:22.168529+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:07:56.723107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.163557+0000","last_change":"2023-10-08T00:16:22.163557+0000","last_active":"2023-10-08T00:16:22.163557+0000","last_peered":"2023-10-08T00:16:22.163557+0000","last_clean":"2023-10-08T00:16:22.163557+0000","last_became_active":"2023-10-08T00:16:22.163320+0000","last_became_peered":"2023-10-08T00:16:22.163320+0000","last_unstale":"2023-10-08T00:16:22.163557+0000","last_undegraded":"2023-10-08T00:16:22.163557+0000","last_fullsized":"2023-10-08T00:16:22.163557+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:09:11.213631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.165451+0000","last_change":"2023-10-08T00:16:22.165451+0000","last_active":"2023-10-08T00:16:22.165451+0000","last_peered":"2023-10-08T00:16:22.165451+0000","last_clean":"2023-10-08T00:16:22.165451+0000","last_became_active":"2023-10-08T00:16:22.165236+0000","last_became_peered":"2023-10-08T00:16:22.165236+0000","last_unstale":"2023-10-08T00:16:22.165451+0000","last_undegraded":"2023-10-08T00:16:22.165451+0000","last_fullsized":"2023-10-08T00:16:22.165451+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:50:42.638829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.156398+0000","last_change":"2023-10-08T00:16:22.156398+0000","last_active":"2023-10-08T00:16:22.156398+0000","last_peered":"2023-10-08T00:16:22.156398+0000","last_clean":"2023-10-08T00:16:22.156398+0000","last_became_active":"2023-10-08T00:16:22.156016+0000","last_became_peered":"2023-10-08T00:16:22.156016+0000","last_unstale":"2023-10-08T00:16:22.156398+0000","last_undegraded":"2023-10-08T00:16:22.156398+0000","last_fullsized":"2023-10-08T00:16:22.156398+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:26:37.495930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.157036+0000","last_change":"2023-10-08T00:16:22.157036+0000","last_active":"2023-10-08T00:16:22.157036+0000","last_peered":"2023-10-08T00:16:22.157036+0000","last_clean":"2023-10-08T00:16:22.157036+0000","last_became_active":"2023-10-08T00:16:22.156786+0000","last_became_peered":"2023-10-08T00:16:22.156786+0000","last_unstale":"2023-10-08T00:16:22.157036+0000","last_undegraded":"2023-10-08T00:16:22.157036+0000","last_fullsized":"2023-10-08T00:16:22.157036+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:35:56.520235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.168747+0000","last_change":"2023-10-08T00:16:22.168747+0000","last_active":"2023-10-08T00:16:22.168747+0000","last_peered":"2023-10-08T00:16:22.168747+0000","last_clean":"2023-10-08T00:16:22.168747+0000","last_became_active":"2023-10-08T00:16:22.168522+0000","last_became_peered":"2023-10-08T00:16:22.168522+0000","last_unstale":"2023-10-08T00:16:22.168747+0000","last_undegraded":"2023-10-08T00:16:22.168747+0000","last_fullsized":"2023-10-08T00:16:22.168747+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:23:11.199659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.164916+0000","last_change":"2023-10-08T00:16:22.164916+0000","last_active":"2023-10-08T00:16:22.164916+0000","last_peered":"2023-10-08T00:16:22.164916+0000","last_clean":"2023-10-08T00:16:22.164916+0000","last_became_active":"2023-10-08T00:16:22.164544+0000","last_became_peered":"2023-10-08T00:16:22.164544+0000","last_unstale":"2023-10-08T00:16:22.164916+0000","last_undegraded":"2023-10-08T00:16:22.164916+0000","last_fullsized":"2023-10-08T00:16:22.164916+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:09:25.675403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.155761+0000","last_change":"2023-10-08T00:16:22.155761+0000","last_active":"2023-10-08T00:16:22.155761+0000","last_peered":"2023-10-08T00:16:22.155761+0000","last_clean":"2023-10-08T00:16:22.155761+0000","last_became_active":"2023-10-08T00:16:22.155520+0000","last_became_peered":"2023-10-08T00:16:22.155520+0000","last_unstale":"2023-10-08T00:16:22.155761+0000","last_undegraded":"2023-10-08T00:16:22.155761+0000","last_fullsized":"2023-10-08T00:16:22.155761+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:34:33.394447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.167870+0000","last_change":"2023-10-08T00:16:22.167870+0000","last_active":"2023-10-08T00:16:22.167870+0000","last_peered":"2023-10-08T00:16:22.167870+0000","last_clean":"2023-10-08T00:16:22.167870+0000","last_became_active":"2023-10-08T00:16:22.167061+0000","last_became_peered":"2023-10-08T00:16:22.167061+0000","last_unstale":"2023-10-08T00:16:22.167870+0000","last_undegraded":"2023-10-08T00:16:22.167870+0000","last_fullsized":"2023-10-08T00:16:22.167870+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:54:17.969681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.166465+0000","last_change":"2023-10-08T00:16:22.166465+0000","last_active":"2023-10-08T00:16:22.166465+0000","last_peered":"2023-10-08T00:16:22.166465+0000","last_clean":"2023-10-08T00:16:22.166465+0000","last_became_active":"2023-10-08T00:16:22.165825+0000","last_became_peered":"2023-10-08T00:16:22.165825+0000","last_unstale":"2023-10-08T00:16:22.166465+0000","last_undegraded":"2023-10-08T00:16:22.166465+0000","last_fullsized":"2023-10-08T00:16:22.166465+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:35:07.078898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.166018+0000","last_change":"2023-10-08T00:16:22.166018+0000","last_active":"2023-10-08T00:16:22.166018+0000","last_peered":"2023-10-08T00:16:22.166018+0000","last_clean":"2023-10-08T00:16:22.166018+0000","last_became_active":"2023-10-08T00:16:22.165795+0000","last_became_peered":"2023-10-08T00:16:22.165795+0000","last_unstale":"2023-10-08T00:16:22.166018+0000","last_undegraded":"2023-10-08T00:16:22.166018+0000","last_fullsized":"2023-10-08T00:16:22.166018+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:47:08.037802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"123'4","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.175178+0000","last_change":"2023-10-08T00:16:21.164000+0000","last_active":"2023-10-08T00:16:22.175178+0000","last_peered":"2023-10-08T00:16:22.175178+0000","last_clean":"2023-10-08T00:16:22.175178+0000","last_became_active":"2023-10-08T00:16:21.163786+0000","last_became_peered":"2023-10-08T00:16:21.163786+0000","last_unstale":"2023-10-08T00:16:22.175178+0000","last_undegraded":"2023-10-08T00:16:22.175178+0000","last_fullsized":"2023-10-08T00:16:22.175178+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:45:18.775774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.165690+0000","last_change":"2023-10-08T00:16:22.165690+0000","last_active":"2023-10-08T00:16:22.165690+0000","last_peered":"2023-10-08T00:16:22.165690+0000","last_clean":"2023-10-08T00:16:22.165690+0000","last_became_active":"2023-10-08T00:16:22.165379+0000","last_became_peered":"2023-10-08T00:16:22.165379+0000","last_unstale":"2023-10-08T00:16:22.165690+0000","last_undegraded":"2023-10-08T00:16:22.165690+0000","last_fullsized":"2023-10-08T00:16:22.165690+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:54:43.701059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"123'1","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.157921+0000","last_change":"2023-10-08T00:16:21.166992+0000","last_active":"2023-10-08T00:16:22.157921+0000","last_peered":"2023-10-08T00:16:22.157921+0000","last_clean":"2023-10-08T00:16:22.157921+0000","last_became_active":"2023-10-08T00:16:21.166731+0000","last_became_peered":"2023-10-08T00:16:21.166731+0000","last_unstale":"2023-10-08T00:16:22.157921+0000","last_undegraded":"2023-10-08T00:16:22.157921+0000","last_fullsized":"2023-10-08T00:16:22.157921+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:41:14.388380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.164171+0000","last_change":"2023-10-08T00:16:22.164171+0000","last_active":"2023-10-08T00:16:22.164171+0000","last_peered":"2023-10-08T00:16:22.164171+0000","last_clean":"2023-10-08T00:16:22.164171+0000","last_became_active":"2023-10-08T00:16:22.163771+0000","last_became_peered":"2023-10-08T00:16:22.163771+0000","last_unstale":"2023-10-08T00:16:22.164171+0000","last_undegraded":"2023-10-08T00:16:22.164171+0000","last_fullsized":"2023-10-08T00:16:22.164171+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:54:00.451741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"123'2","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.172979+0000","last_change":"2023-10-08T00:16:21.164021+0000","last_active":"2023-10-08T00:16:22.172979+0000","last_peered":"2023-10-08T00:16:22.172979+0000","last_clean":"2023-10-08T00:16:22.172979+0000","last_became_active":"2023-10-08T00:16:21.163600+0000","last_became_peered":"2023-10-08T00:16:21.163600+0000","last_unstale":"2023-10-08T00:16:22.172979+0000","last_undegraded":"2023-10-08T00:16:22.172979+0000","last_fullsized":"2023-10-08T00:16:22.172979+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:47:27.489699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.163743+0000","last_change":"2023-10-08T00:16:22.163743+0000","last_active":"2023-10-08T00:16:22.163743+0000","last_peered":"2023-10-08T00:16:22.163743+0000","last_clean":"2023-10-08T00:16:22.163743+0000","last_became_active":"2023-10-08T00:16:22.163532+0000","last_became_peered":"2023-10-08T00:16:22.163532+0000","last_unstale":"2023-10-08T00:16:22.163743+0000","last_undegraded":"2023-10-08T00:16:22.163743+0000","last_fullsized":"2023-10-08T00:16:22.163743+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:53:34.263422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"123'2","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.175614+0000","last_change":"2023-10-08T00:16:21.162920+0000","last_active":"2023-10-08T00:16:22.175614+0000","last_peered":"2023-10-08T00:16:22.175614+0000","last_clean":"2023-10-08T00:16:22.175614+0000","last_became_active":"2023-10-08T00:16:21.162488+0000","last_became_peered":"2023-10-08T00:16:21.162488+0000","last_unstale":"2023-10-08T00:16:22.175614+0000","last_undegraded":"2023-10-08T00:16:22.175614+0000","last_fullsized":"2023-10-08T00:16:22.175614+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:15:13.307047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.163846+0000","last_change":"2023-10-08T00:16:22.163846+0000","last_active":"2023-10-08T00:16:22.163846+0000","last_peered":"2023-10-08T00:16:22.163846+0000","last_clean":"2023-10-08T00:16:22.163846+0000","last_became_active":"2023-10-08T00:16:22.162480+0000","last_became_peered":"2023-10-08T00:16:22.162480+0000","last_unstale":"2023-10-08T00:16:22.163846+0000","last_undegraded":"2023-10-08T00:16:22.163846+0000","last_fullsized":"2023-10-08T00:16:22.163846+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:09:25.675403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"123'1","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.161529+0000","last_change":"2023-10-08T00:16:21.158912+0000","last_active":"2023-10-08T00:16:22.161529+0000","last_peered":"2023-10-08T00:16:22.161529+0000","last_clean":"2023-10-08T00:16:22.161529+0000","last_became_active":"2023-10-08T00:16:21.158683+0000","last_became_peered":"2023-10-08T00:16:21.158683+0000","last_unstale":"2023-10-08T00:16:22.161529+0000","last_undegraded":"2023-10-08T00:16:22.161529+0000","last_fullsized":"2023-10-08T00:16:22.161529+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:34:48.715478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.167201+0000","last_change":"2023-10-08T00:16:22.167201+0000","last_active":"2023-10-08T00:16:22.167201+0000","last_peered":"2023-10-08T00:16:22.167201+0000","last_clean":"2023-10-08T00:16:22.167201+0000","last_became_active":"2023-10-08T00:16:22.166944+0000","last_became_peered":"2023-10-08T00:16:22.166944+0000","last_unstale":"2023-10-08T00:16:22.167201+0000","last_undegraded":"2023-10-08T00:16:22.167201+0000","last_fullsized":"2023-10-08T00:16:22.167201+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:16:19.343525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"123'1","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.162159+0000","last_change":"2023-10-08T00:16:21.160966+0000","last_active":"2023-10-08T00:16:22.162159+0000","last_peered":"2023-10-08T00:16:22.162159+0000","last_clean":"2023-10-08T00:16:22.162159+0000","last_became_active":"2023-10-08T00:16:21.159717+0000","last_became_peered":"2023-10-08T00:16:21.159717+0000","last_unstale":"2023-10-08T00:16:22.162159+0000","last_undegraded":"2023-10-08T00:16:22.162159+0000","last_fullsized":"2023-10-08T00:16:22.162159+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:16:10.889545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.164400+0000","last_change":"2023-10-08T00:16:22.164400+0000","last_active":"2023-10-08T00:16:22.164400+0000","last_peered":"2023-10-08T00:16:22.164400+0000","last_clean":"2023-10-08T00:16:22.164400+0000","last_became_active":"2023-10-08T00:16:22.164128+0000","last_became_peered":"2023-10-08T00:16:22.164128+0000","last_unstale":"2023-10-08T00:16:22.164400+0000","last_undegraded":"2023-10-08T00:16:22.164400+0000","last_fullsized":"2023-10-08T00:16:22.164400+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:54:17.969681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-08T00:16:21.157979+0000","last_change":"2023-10-08T00:16:21.157919+0000","last_active":"2023-10-08T00:16:21.157979+0000","last_peered":"2023-10-08T00:16:21.157979+0000","last_clean":"2023-10-08T00:16:21.157979+0000","last_became_active":"2023-10-08T00:16:21.157695+0000","last_became_peered":"2023-10-08T00:16:21.157695+0000","last_unstale":"2023-10-08T00:16:21.157979+0000","last_undegraded":"2023-10-08T00:16:21.157979+0000","last_fullsized":"2023-10-08T00:16:21.157979+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:59:48.625941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.165112+0000","last_change":"2023-10-08T00:16:22.165112+0000","last_active":"2023-10-08T00:16:22.165112+0000","last_peered":"2023-10-08T00:16:22.165112+0000","last_clean":"2023-10-08T00:16:22.165112+0000","last_became_active":"2023-10-08T00:16:22.164779+0000","last_became_peered":"2023-10-08T00:16:22.164779+0000","last_unstale":"2023-10-08T00:16:22.165112+0000","last_undegraded":"2023-10-08T00:16:22.165112+0000","last_fullsized":"2023-10-08T00:16:22.165112+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:15:14.311899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-08T00:16:21.165975+0000","last_change":"2023-10-08T00:16:21.165191+0000","last_active":"2023-10-08T00:16:21.165975+0000","last_peered":"2023-10-08T00:16:21.165975+0000","last_clean":"2023-10-08T00:16:21.165975+0000","last_became_active":"2023-10-08T00:16:21.164824+0000","last_became_peered":"2023-10-08T00:16:21.164824+0000","last_unstale":"2023-10-08T00:16:21.165975+0000","last_undegraded":"2023-10-08T00:16:21.165975+0000","last_fullsized":"2023-10-08T00:16:21.165975+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:04:07.976115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.162534+0000","last_change":"2023-10-08T00:16:22.162534+0000","last_active":"2023-10-08T00:16:22.162534+0000","last_peered":"2023-10-08T00:16:22.162534+0000","last_clean":"2023-10-08T00:16:22.162534+0000","last_became_active":"2023-10-08T00:16:22.162303+0000","last_became_peered":"2023-10-08T00:16:22.162303+0000","last_unstale":"2023-10-08T00:16:22.162534+0000","last_undegraded":"2023-10-08T00:16:22.162534+0000","last_fullsized":"2023-10-08T00:16:22.162534+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:26:47.791012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-08T00:16:21.161043+0000","last_change":"2023-10-08T00:16:21.160185+0000","last_active":"2023-10-08T00:16:21.161043+0000","last_peered":"2023-10-08T00:16:21.161043+0000","last_clean":"2023-10-08T00:16:21.161043+0000","last_became_active":"2023-10-08T00:16:21.159932+0000","last_became_peered":"2023-10-08T00:16:21.159932+0000","last_unstale":"2023-10-08T00:16:21.161043+0000","last_undegraded":"2023-10-08T00:16:21.161043+0000","last_fullsized":"2023-10-08T00:16:21.161043+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:04:29.838739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.162528+0000","last_change":"2023-10-08T00:16:22.162528+0000","last_active":"2023-10-08T00:16:22.162528+0000","last_peered":"2023-10-08T00:16:22.162528+0000","last_clean":"2023-10-08T00:16:22.162528+0000","last_became_active":"2023-10-08T00:16:22.162320+0000","last_became_peered":"2023-10-08T00:16:22.162320+0000","last_unstale":"2023-10-08T00:16:22.162528+0000","last_undegraded":"2023-10-08T00:16:22.162528+0000","last_fullsized":"2023-10-08T00:16:22.162528+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:47:08.037802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-08T00:16:21.159766+0000","last_change":"2023-10-08T00:16:21.159661+0000","last_active":"2023-10-08T00:16:21.159766+0000","last_peered":"2023-10-08T00:16:21.159766+0000","last_clean":"2023-10-08T00:16:21.159766+0000","last_became_active":"2023-10-08T00:16:21.159125+0000","last_became_peered":"2023-10-08T00:16:21.159125+0000","last_unstale":"2023-10-08T00:16:21.159766+0000","last_undegraded":"2023-10-08T00:16:21.159766+0000","last_fullsized":"2023-10-08T00:16:21.159766+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:02:42.055051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.167493+0000","last_change":"2023-10-08T00:16:22.167493+0000","last_active":"2023-10-08T00:16:22.167493+0000","last_peered":"2023-10-08T00:16:22.167493+0000","last_clean":"2023-10-08T00:16:22.167493+0000","last_became_active":"2023-10-08T00:16:22.167268+0000","last_became_peered":"2023-10-08T00:16:22.167268+0000","last_unstale":"2023-10-08T00:16:22.167493+0000","last_undegraded":"2023-10-08T00:16:22.167493+0000","last_fullsized":"2023-10-08T00:16:22.167493+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:08:28.127337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"123'3","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.175313+0000","last_change":"2023-10-08T00:16:21.158687+0000","last_active":"2023-10-08T00:16:22.175313+0000","last_peered":"2023-10-08T00:16:22.175313+0000","last_clean":"2023-10-08T00:16:22.175313+0000","last_became_active":"2023-10-08T00:16:21.158476+0000","last_became_peered":"2023-10-08T00:16:21.158476+0000","last_unstale":"2023-10-08T00:16:22.175313+0000","last_undegraded":"2023-10-08T00:16:22.175313+0000","last_fullsized":"2023-10-08T00:16:22.175313+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:30:47.742876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.160017+0000","last_change":"2023-10-08T00:16:22.160017+0000","last_active":"2023-10-08T00:16:22.160017+0000","last_peered":"2023-10-08T00:16:22.160017+0000","last_clean":"2023-10-08T00:16:22.160017+0000","last_became_active":"2023-10-08T00:16:22.159727+0000","last_became_peered":"2023-10-08T00:16:22.159727+0000","last_unstale":"2023-10-08T00:16:22.160017+0000","last_undegraded":"2023-10-08T00:16:22.160017+0000","last_fullsized":"2023-10-08T00:16:22.160017+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:40:12.567676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"123'6","reported_seq":15,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.174977+0000","last_change":"2023-10-08T00:16:21.166994+0000","last_active":"2023-10-08T00:16:22.174977+0000","last_peered":"2023-10-08T00:16:22.174977+0000","last_clean":"2023-10-08T00:16:22.174977+0000","last_became_active":"2023-10-08T00:16:21.166843+0000","last_became_peered":"2023-10-08T00:16:21.166843+0000","last_unstale":"2023-10-08T00:16:22.174977+0000","last_undegraded":"2023-10-08T00:16:22.174977+0000","last_fullsized":"2023-10-08T00:16:22.174977+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:34:47.325200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.164023+0000","last_change":"2023-10-08T00:16:22.164023+0000","last_active":"2023-10-08T00:16:22.164023+0000","last_peered":"2023-10-08T00:16:22.164023+0000","last_clean":"2023-10-08T00:16:22.164023+0000","last_became_active":"2023-10-08T00:16:22.163472+0000","last_became_peered":"2023-10-08T00:16:22.163472+0000","last_unstale":"2023-10-08T00:16:22.164023+0000","last_undegraded":"2023-10-08T00:16:22.164023+0000","last_fullsized":"2023-10-08T00:16:22.164023+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:36:52.374149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"123'2","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.175101+0000","last_change":"2023-10-08T00:16:21.165052+0000","last_active":"2023-10-08T00:16:22.175101+0000","last_peered":"2023-10-08T00:16:22.175101+0000","last_clean":"2023-10-08T00:16:22.175101+0000","last_became_active":"2023-10-08T00:16:21.164426+0000","last_became_peered":"2023-10-08T00:16:21.164426+0000","last_unstale":"2023-10-08T00:16:22.175101+0000","last_undegraded":"2023-10-08T00:16:22.175101+0000","last_fullsized":"2023-10-08T00:16:22.175101+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:17:44.815549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.163946+0000","last_change":"2023-10-08T00:16:22.163946+0000","last_active":"2023-10-08T00:16:22.163946+0000","last_peered":"2023-10-08T00:16:22.163946+0000","last_clean":"2023-10-08T00:16:22.163946+0000","last_became_active":"2023-10-08T00:16:22.163402+0000","last_became_peered":"2023-10-08T00:16:22.163402+0000","last_unstale":"2023-10-08T00:16:22.163946+0000","last_undegraded":"2023-10-08T00:16:22.163946+0000","last_fullsized":"2023-10-08T00:16:22.163946+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:35:04.496031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-08T00:16:21.162860+0000","last_change":"2023-10-08T00:16:21.162715+0000","last_active":"2023-10-08T00:16:21.162860+0000","last_peered":"2023-10-08T00:16:21.162860+0000","last_clean":"2023-10-08T00:16:21.162860+0000","last_became_active":"2023-10-08T00:16:21.161125+0000","last_became_peered":"2023-10-08T00:16:21.161125+0000","last_unstale":"2023-10-08T00:16:21.162860+0000","last_undegraded":"2023-10-08T00:16:21.162860+0000","last_fullsized":"2023-10-08T00:16:21.162860+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:59:12.283523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.163296+0000","last_change":"2023-10-08T00:16:22.163296+0000","last_active":"2023-10-08T00:16:22.163296+0000","last_peered":"2023-10-08T00:16:22.163296+0000","last_clean":"2023-10-08T00:16:22.163296+0000","last_became_active":"2023-10-08T00:16:22.162982+0000","last_became_peered":"2023-10-08T00:16:22.162982+0000","last_unstale":"2023-10-08T00:16:22.163296+0000","last_undegraded":"2023-10-08T00:16:22.163296+0000","last_fullsized":"2023-10-08T00:16:22.163296+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:43:01.385831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-08T00:16:21.160253+0000","last_change":"2023-10-08T00:16:21.160061+0000","last_active":"2023-10-08T00:16:21.160253+0000","last_peered":"2023-10-08T00:16:21.160253+0000","last_clean":"2023-10-08T00:16:21.160253+0000","last_became_active":"2023-10-08T00:16:21.159709+0000","last_became_peered":"2023-10-08T00:16:21.159709+0000","last_unstale":"2023-10-08T00:16:21.160253+0000","last_undegraded":"2023-10-08T00:16:21.160253+0000","last_fullsized":"2023-10-08T00:16:21.160253+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:16:10.889545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.166743+0000","last_change":"2023-10-08T00:16:22.166743+0000","last_active":"2023-10-08T00:16:22.166743+0000","last_peered":"2023-10-08T00:16:22.166743+0000","last_clean":"2023-10-08T00:16:22.166743+0000","last_became_active":"2023-10-08T00:16:22.166462+0000","last_became_peered":"2023-10-08T00:16:22.166462+0000","last_unstale":"2023-10-08T00:16:22.166743+0000","last_undegraded":"2023-10-08T00:16:22.166743+0000","last_fullsized":"2023-10-08T00:16:22.166743+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:54:43.701059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"123'8","reported_seq":17,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.176492+0000","last_change":"2023-10-08T00:16:21.160781+0000","last_active":"2023-10-08T00:16:22.176492+0000","last_peered":"2023-10-08T00:16:22.176492+0000","last_clean":"2023-10-08T00:16:22.176492+0000","last_became_active":"2023-10-08T00:16:21.160010+0000","last_became_peered":"2023-10-08T00:16:21.160010+0000","last_unstale":"2023-10-08T00:16:22.176492+0000","last_undegraded":"2023-10-08T00:16:22.176492+0000","last_fullsized":"2023-10-08T00:16:22.176492+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:54:16.964828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.166017+0000","last_change":"2023-10-08T00:16:22.166017+0000","last_active":"2023-10-08T00:16:22.166017+0000","last_peered":"2023-10-08T00:16:22.166017+0000","last_clean":"2023-10-08T00:16:22.166017+0000","last_became_active":"2023-10-08T00:16:22.165772+0000","last_became_peered":"2023-10-08T00:16:22.165772+0000","last_unstale":"2023-10-08T00:16:22.166017+0000","last_undegraded":"2023-10-08T00:16:22.166017+0000","last_fullsized":"2023-10-08T00:16:22.166017+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:30:42.577974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"123'2","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.174868+0000","last_change":"2023-10-08T00:16:21.165739+0000","last_active":"2023-10-08T00:16:22.174868+0000","last_peered":"2023-10-08T00:16:22.174868+0000","last_clean":"2023-10-08T00:16:22.174868+0000","last_became_active":"2023-10-08T00:16:21.165596+0000","last_became_peered":"2023-10-08T00:16:21.165596+0000","last_unstale":"2023-10-08T00:16:22.174868+0000","last_undegraded":"2023-10-08T00:16:22.174868+0000","last_fullsized":"2023-10-08T00:16:22.174868+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:11:16.353357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.166357+0000","last_change":"2023-10-08T00:16:22.166357+0000","last_active":"2023-10-08T00:16:22.166357+0000","last_peered":"2023-10-08T00:16:22.166357+0000","last_clean":"2023-10-08T00:16:22.166357+0000","last_became_active":"2023-10-08T00:16:22.166141+0000","last_became_peered":"2023-10-08T00:16:22.166141+0000","last_unstale":"2023-10-08T00:16:22.166357+0000","last_undegraded":"2023-10-08T00:16:22.166357+0000","last_fullsized":"2023-10-08T00:16:22.166357+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:40:19.056813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"123'1","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.157943+0000","last_change":"2023-10-08T00:16:21.159455+0000","last_active":"2023-10-08T00:16:22.157943+0000","last_peered":"2023-10-08T00:16:22.157943+0000","last_clean":"2023-10-08T00:16:22.157943+0000","last_became_active":"2023-10-08T00:16:21.159198+0000","last_became_peered":"2023-10-08T00:16:21.159198+0000","last_unstale":"2023-10-08T00:16:22.157943+0000","last_undegraded":"2023-10-08T00:16:22.157943+0000","last_fullsized":"2023-10-08T00:16:22.157943+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:11:11.190696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.164268+0000","last_change":"2023-10-08T00:16:22.164268+0000","last_active":"2023-10-08T00:16:22.164268+0000","last_peered":"2023-10-08T00:16:22.164268+0000","last_clean":"2023-10-08T00:16:22.164268+0000","last_became_active":"2023-10-08T00:16:22.164025+0000","last_became_peered":"2023-10-08T00:16:22.164025+0000","last_unstale":"2023-10-08T00:16:22.164268+0000","last_undegraded":"2023-10-08T00:16:22.164268+0000","last_fullsized":"2023-10-08T00:16:22.164268+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:59:13.288375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-08T00:16:21.159608+0000","last_change":"2023-10-08T00:16:21.159488+0000","last_active":"2023-10-08T00:16:21.159608+0000","last_peered":"2023-10-08T00:16:21.159608+0000","last_clean":"2023-10-08T00:16:21.159608+0000","last_became_active":"2023-10-08T00:16:21.158888+0000","last_became_peered":"2023-10-08T00:16:21.158888+0000","last_unstale":"2023-10-08T00:16:21.159608+0000","last_undegraded":"2023-10-08T00:16:21.159608+0000","last_fullsized":"2023-10-08T00:16:21.159608+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:27:12.467258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.162930+0000","last_change":"2023-10-08T00:16:22.162930+0000","last_active":"2023-10-08T00:16:22.162930+0000","last_peered":"2023-10-08T00:16:22.162930+0000","last_clean":"2023-10-08T00:16:22.162930+0000","last_became_active":"2023-10-08T00:16:22.162721+0000","last_became_peered":"2023-10-08T00:16:22.162721+0000","last_unstale":"2023-10-08T00:16:22.162930+0000","last_undegraded":"2023-10-08T00:16:22.162930+0000","last_fullsized":"2023-10-08T00:16:22.162930+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:23:11.199659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"123'2","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.176794+0000","last_change":"2023-10-08T00:16:21.160930+0000","last_active":"2023-10-08T00:16:22.176794+0000","last_peered":"2023-10-08T00:16:22.176794+0000","last_clean":"2023-10-08T00:16:22.176794+0000","last_became_active":"2023-10-08T00:16:21.160562+0000","last_became_peered":"2023-10-08T00:16:21.160562+0000","last_unstale":"2023-10-08T00:16:22.176794+0000","last_undegraded":"2023-10-08T00:16:22.176794+0000","last_fullsized":"2023-10-08T00:16:22.176794+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:20:42.677235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.163843+0000","last_change":"2023-10-08T00:16:22.163843+0000","last_active":"2023-10-08T00:16:22.163843+0000","last_peered":"2023-10-08T00:16:22.163843+0000","last_clean":"2023-10-08T00:16:22.163843+0000","last_became_active":"2023-10-08T00:16:22.163619+0000","last_became_peered":"2023-10-08T00:16:22.163619+0000","last_unstale":"2023-10-08T00:16:22.163843+0000","last_undegraded":"2023-10-08T00:16:22.163843+0000","last_fullsized":"2023-10-08T00:16:22.163843+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:44:20.663579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-08T00:16:21.155458+0000","last_change":"2023-10-08T00:16:21.154822+0000","last_active":"2023-10-08T00:16:21.155458+0000","last_peered":"2023-10-08T00:16:21.155458+0000","last_clean":"2023-10-08T00:16:21.155458+0000","last_became_active":"2023-10-08T00:16:21.154614+0000","last_became_peered":"2023-10-08T00:16:21.154614+0000","last_unstale":"2023-10-08T00:16:21.155458+0000","last_undegraded":"2023-10-08T00:16:21.155458+0000","last_fullsized":"2023-10-08T00:16:21.155458+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:04:07.976115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.168462+0000","last_change":"2023-10-08T00:16:22.168462+0000","last_active":"2023-10-08T00:16:22.168462+0000","last_peered":"2023-10-08T00:16:22.168462+0000","last_clean":"2023-10-08T00:16:22.168462+0000","last_became_active":"2023-10-08T00:16:22.168230+0000","last_became_peered":"2023-10-08T00:16:22.168230+0000","last_unstale":"2023-10-08T00:16:22.168462+0000","last_undegraded":"2023-10-08T00:16:22.168462+0000","last_fullsized":"2023-10-08T00:16:22.168462+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:21:44.003712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-08T00:16:21.159223+0000","last_change":"2023-10-08T00:16:21.159058+0000","last_active":"2023-10-08T00:16:21.159223+0000","last_peered":"2023-10-08T00:16:21.159223+0000","last_clean":"2023-10-08T00:16:21.159223+0000","last_became_active":"2023-10-08T00:16:21.158618+0000","last_became_peered":"2023-10-08T00:16:21.158618+0000","last_unstale":"2023-10-08T00:16:21.159223+0000","last_undegraded":"2023-10-08T00:16:21.159223+0000","last_fullsized":"2023-10-08T00:16:21.159223+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:35:15.325157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.168156+0000","last_change":"2023-10-08T00:16:22.168156+0000","last_active":"2023-10-08T00:16:22.168156+0000","last_peered":"2023-10-08T00:16:22.168156+0000","last_clean":"2023-10-08T00:16:22.168156+0000","last_became_active":"2023-10-08T00:16:22.167982+0000","last_became_peered":"2023-10-08T00:16:22.167982+0000","last_unstale":"2023-10-08T00:16:22.168156+0000","last_undegraded":"2023-10-08T00:16:22.168156+0000","last_fullsized":"2023-10-08T00:16:22.168156+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:29:35.744364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-08T00:16:21.165127+0000","last_change":"2023-10-08T00:16:21.165066+0000","last_active":"2023-10-08T00:16:21.165127+0000","last_peered":"2023-10-08T00:16:21.165127+0000","last_clean":"2023-10-08T00:16:21.165127+0000","last_became_active":"2023-10-08T00:16:21.164371+0000","last_became_peered":"2023-10-08T00:16:21.164371+0000","last_unstale":"2023-10-08T00:16:21.165127+0000","last_undegraded":"2023-10-08T00:16:21.165127+0000","last_fullsized":"2023-10-08T00:16:21.165127+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:38:10.363947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.157037+0000","last_change":"2023-10-08T00:16:22.157037+0000","last_active":"2023-10-08T00:16:22.157037+0000","last_peered":"2023-10-08T00:16:22.157037+0000","last_clean":"2023-10-08T00:16:22.157037+0000","last_became_active":"2023-10-08T00:16:22.156823+0000","last_became_peered":"2023-10-08T00:16:22.156823+0000","last_unstale":"2023-10-08T00:16:22.157037+0000","last_undegraded":"2023-10-08T00:16:22.157037+0000","last_fullsized":"2023-10-08T00:16:22.157037+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:53:34.263422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"123'2","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.174197+0000","last_change":"2023-10-08T00:16:21.159923+0000","last_active":"2023-10-08T00:16:22.174197+0000","last_peered":"2023-10-08T00:16:22.174197+0000","last_clean":"2023-10-08T00:16:22.174197+0000","last_became_active":"2023-10-08T00:16:21.159425+0000","last_became_peered":"2023-10-08T00:16:21.159425+0000","last_unstale":"2023-10-08T00:16:22.174197+0000","last_undegraded":"2023-10-08T00:16:22.174197+0000","last_fullsized":"2023-10-08T00:16:22.174197+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:37:12.413185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.164592+0000","last_change":"2023-10-08T00:16:22.164592+0000","last_active":"2023-10-08T00:16:22.164592+0000","last_peered":"2023-10-08T00:16:22.164592+0000","last_clean":"2023-10-08T00:16:22.164592+0000","last_became_active":"2023-10-08T00:16:22.164252+0000","last_became_peered":"2023-10-08T00:16:22.164252+0000","last_unstale":"2023-10-08T00:16:22.164592+0000","last_undegraded":"2023-10-08T00:16:22.164592+0000","last_fullsized":"2023-10-08T00:16:22.164592+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:36:52.374149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-08T00:16:21.166957+0000","last_change":"2023-10-08T00:16:21.166879+0000","last_active":"2023-10-08T00:16:21.166957+0000","last_peered":"2023-10-08T00:16:21.166957+0000","last_clean":"2023-10-08T00:16:21.166957+0000","last_became_active":"2023-10-08T00:16:21.166638+0000","last_became_peered":"2023-10-08T00:16:21.166638+0000","last_unstale":"2023-10-08T00:16:21.166957+0000","last_undegraded":"2023-10-08T00:16:21.166957+0000","last_fullsized":"2023-10-08T00:16:21.166957+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:46:05.494965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.164094+0000","last_change":"2023-10-08T00:16:22.164094+0000","last_active":"2023-10-08T00:16:22.164094+0000","last_peered":"2023-10-08T00:16:22.164094+0000","last_clean":"2023-10-08T00:16:22.164094+0000","last_became_active":"2023-10-08T00:16:22.163717+0000","last_became_peered":"2023-10-08T00:16:22.163717+0000","last_unstale":"2023-10-08T00:16:22.164094+0000","last_undegraded":"2023-10-08T00:16:22.164094+0000","last_fullsized":"2023-10-08T00:16:22.164094+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:26:37.495930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-08T00:16:21.167140+0000","last_change":"2023-10-08T00:16:21.167100+0000","last_active":"2023-10-08T00:16:21.167140+0000","last_peered":"2023-10-08T00:16:21.167140+0000","last_clean":"2023-10-08T00:16:21.167140+0000","last_became_active":"2023-10-08T00:16:21.166283+0000","last_became_peered":"2023-10-08T00:16:21.166283+0000","last_unstale":"2023-10-08T00:16:21.167140+0000","last_undegraded":"2023-10-08T00:16:21.167140+0000","last_fullsized":"2023-10-08T00:16:21.167140+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:50:55.146686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.163628+0000","last_change":"2023-10-08T00:16:22.163628+0000","last_active":"2023-10-08T00:16:22.163628+0000","last_peered":"2023-10-08T00:16:22.163628+0000","last_clean":"2023-10-08T00:16:22.163628+0000","last_became_active":"2023-10-08T00:16:22.163402+0000","last_became_peered":"2023-10-08T00:16:22.163402+0000","last_unstale":"2023-10-08T00:16:22.163628+0000","last_undegraded":"2023-10-08T00:16:22.163628+0000","last_fullsized":"2023-10-08T00:16:22.163628+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:46.070540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-08T00:16:21.161596+0000","last_change":"2023-10-08T00:16:21.161536+0000","last_active":"2023-10-08T00:16:21.161596+0000","last_peered":"2023-10-08T00:16:21.161596+0000","last_clean":"2023-10-08T00:16:21.161596+0000","last_became_active":"2023-10-08T00:16:21.161356+0000","last_became_peered":"2023-10-08T00:16:21.161356+0000","last_unstale":"2023-10-08T00:16:21.161596+0000","last_undegraded":"2023-10-08T00:16:21.161596+0000","last_fullsized":"2023-10-08T00:16:21.161596+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:05:54.513612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.158829+0000","last_change":"2023-10-08T00:16:22.158829+0000","last_active":"2023-10-08T00:16:22.158829+0000","last_peered":"2023-10-08T00:16:22.158829+0000","last_clean":"2023-10-08T00:16:22.158829+0000","last_became_active":"2023-10-08T00:16:22.157910+0000","last_became_peered":"2023-10-08T00:16:22.157910+0000","last_unstale":"2023-10-08T00:16:22.158829+0000","last_undegraded":"2023-10-08T00:16:22.158829+0000","last_fullsized":"2023-10-08T00:16:22.158829+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:50:42.638829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-08T00:16:21.165337+0000","last_change":"2023-10-08T00:16:21.165275+0000","last_active":"2023-10-08T00:16:21.165337+0000","last_peered":"2023-10-08T00:16:21.165337+0000","last_clean":"2023-10-08T00:16:21.165337+0000","last_became_active":"2023-10-08T00:16:21.164768+0000","last_became_peered":"2023-10-08T00:16:21.164768+0000","last_unstale":"2023-10-08T00:16:21.165337+0000","last_undegraded":"2023-10-08T00:16:21.165337+0000","last_fullsized":"2023-10-08T00:16:21.165337+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:11:30.488955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.164327+0000","last_change":"2023-10-08T00:16:22.164327+0000","last_active":"2023-10-08T00:16:22.164327+0000","last_peered":"2023-10-08T00:16:22.164327+0000","last_clean":"2023-10-08T00:16:22.164327+0000","last_became_active":"2023-10-08T00:16:22.163648+0000","last_became_peered":"2023-10-08T00:16:22.163648+0000","last_unstale":"2023-10-08T00:16:22.164327+0000","last_undegraded":"2023-10-08T00:16:22.164327+0000","last_fullsized":"2023-10-08T00:16:22.164327+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:19:31.079138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-08T00:16:21.162693+0000","last_change":"2023-10-08T00:16:21.162634+0000","last_active":"2023-10-08T00:16:21.162693+0000","last_peered":"2023-10-08T00:16:21.162693+0000","last_clean":"2023-10-08T00:16:21.162693+0000","last_became_active":"2023-10-08T00:16:21.162455+0000","last_became_peered":"2023-10-08T00:16:21.162455+0000","last_unstale":"2023-10-08T00:16:21.162693+0000","last_undegraded":"2023-10-08T00:16:21.162693+0000","last_fullsized":"2023-10-08T00:16:21.162693+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:46:05.494965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.158995+0000","last_change":"2023-10-08T00:16:22.158995+0000","last_active":"2023-10-08T00:16:22.158995+0000","last_peered":"2023-10-08T00:16:22.158995+0000","last_clean":"2023-10-08T00:16:22.158995+0000","last_became_active":"2023-10-08T00:16:22.158755+0000","last_became_peered":"2023-10-08T00:16:22.158755+0000","last_unstale":"2023-10-08T00:16:22.158995+0000","last_undegraded":"2023-10-08T00:16:22.158995+0000","last_fullsized":"2023-10-08T00:16:22.158995+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:29:35.744364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-08T00:16:21.161822+0000","last_change":"2023-10-08T00:16:21.161744+0000","last_active":"2023-10-08T00:16:21.161822+0000","last_peered":"2023-10-08T00:16:21.161822+0000","last_clean":"2023-10-08T00:16:21.161822+0000","last_became_active":"2023-10-08T00:16:21.161514+0000","last_became_peered":"2023-10-08T00:16:21.161514+0000","last_unstale":"2023-10-08T00:16:21.161822+0000","last_undegraded":"2023-10-08T00:16:21.161822+0000","last_fullsized":"2023-10-08T00:16:21.161822+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:59:12.063065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.166013+0000","last_change":"2023-10-08T00:16:22.166013+0000","last_active":"2023-10-08T00:16:22.166013+0000","last_peered":"2023-10-08T00:16:22.166013+0000","last_clean":"2023-10-08T00:16:22.166013+0000","last_became_active":"2023-10-08T00:16:22.165795+0000","last_became_peered":"2023-10-08T00:16:22.165795+0000","last_unstale":"2023-10-08T00:16:22.166013+0000","last_undegraded":"2023-10-08T00:16:22.166013+0000","last_fullsized":"2023-10-08T00:16:22.166013+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:54:17.969681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-08T00:16:21.168074+0000","last_change":"2023-10-08T00:16:21.167985+0000","last_active":"2023-10-08T00:16:21.168074+0000","last_peered":"2023-10-08T00:16:21.168074+0000","last_clean":"2023-10-08T00:16:21.168074+0000","last_became_active":"2023-10-08T00:16:21.167755+0000","last_became_peered":"2023-10-08T00:16:21.167755+0000","last_unstale":"2023-10-08T00:16:21.168074+0000","last_undegraded":"2023-10-08T00:16:21.168074+0000","last_fullsized":"2023-10-08T00:16:21.168074+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:37:12.413185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.167814+0000","last_change":"2023-10-08T00:16:22.167814+0000","last_active":"2023-10-08T00:16:22.167814+0000","last_peered":"2023-10-08T00:16:22.167814+0000","last_clean":"2023-10-08T00:16:22.167814+0000","last_became_active":"2023-10-08T00:16:22.167655+0000","last_became_peered":"2023-10-08T00:16:22.167655+0000","last_unstale":"2023-10-08T00:16:22.167814+0000","last_undegraded":"2023-10-08T00:16:22.167814+0000","last_fullsized":"2023-10-08T00:16:22.167814+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:25:07.879702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"123'1","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.164456+0000","last_change":"2023-10-08T00:16:21.166400+0000","last_active":"2023-10-08T00:16:22.164456+0000","last_peered":"2023-10-08T00:16:22.164456+0000","last_clean":"2023-10-08T00:16:22.164456+0000","last_became_active":"2023-10-08T00:16:21.166279+0000","last_became_peered":"2023-10-08T00:16:21.166279+0000","last_unstale":"2023-10-08T00:16:22.164456+0000","last_undegraded":"2023-10-08T00:16:22.164456+0000","last_fullsized":"2023-10-08T00:16:22.164456+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:02:23.869290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":250,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-08T00:16:21.157581+0000","last_change":"2023-10-08T00:10:43.728490+0000","last_active":"2023-10-08T00:16:21.157581+0000","last_peered":"2023-10-08T00:16:21.157581+0000","last_clean":"2023-10-08T00:16:21.157581+0000","last_became_active":"2023-10-08T00:10:43.728341+0000","last_became_peered":"2023-10-08T00:10:43.728341+0000","last_unstale":"2023-10-08T00:16:21.157581+0000","last_undegraded":"2023-10-08T00:16:21.157581+0000","last_fullsized":"2023-10-08T00:16:21.157581+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_clean_scrub_stamp":"2023-10-08T00:10:42.716096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:06:31.035293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574918,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":662,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194649,"statfs":{"total":200000000,"available":199321599,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.53300000000000003,"15min":0.50700000000000001},"min":{"1min":0.39700000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.54300000000000004,"15min":0.54500000000000004},"min":{"1min":0.36099999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.52300000000000002}]},{"osd":1,"last update":"Sun Oct 8 00:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56100000000000005,"15min":0.53700000000000003},"min":{"1min":0.38,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.72799999999999998,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.58699999999999997,"15min":0.56699999999999995},"min":{"1min":0.42399999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.78400000000000003,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.47799999999999998}]},{"osd":2,"last update":"Sun Oct 8 00:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55100000000000005,"15min":0.54000000000000004},"min":{"1min":0.34599999999999997,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.501},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.55200000000000005,"15min":0.52600000000000002},"min":{"1min":0.433,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77200000000000002,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.46800000000000003}]},{"osd":3,"last update":"Sun Oct 8 00:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54500000000000004,"15min":0.52300000000000002},"min":{"1min":0.35599999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.75600000000000001,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56899999999999995,"15min":0.55900000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.371}]},{"osd":4,"last update":"Sun Oct 8 00:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.497,"15min":0.51700000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.65100000000000002,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.5,"5min":0.504,"15min":0.52400000000000002},"min":{"1min":0.315,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.75900000000000001,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.442}]},{"osd":5,"last update":"Sun Oct 8 00:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.51500000000000001,"15min":0.54500000000000004},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.78000000000000003,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54300000000000004,"15min":0.57999999999999996},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.76900000000000002,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.46000000000000002}]},{"osd":6,"last update":"Sun Oct 8 00:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.57799999999999996,"15min":0.59399999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.74399999999999999,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58899999999999997,"15min":0.625},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.80100000000000005,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.39100000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574918,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":661,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194650,"statfs":{"total":200000000,"available":199322153,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.51500000000000001,"15min":0.48699999999999999},"min":{"1min":0.39400000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.71199999999999997,"5min":1.012,"15min":1.012},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.56999999999999995,"15min":0.66700000000000004},"min":{"1min":0.39900000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.0529999999999999,"5min":5.1230000000000002,"15min":5.1230000000000002},"last":0.61399999999999999}]},{"osd":1,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55200000000000005,"15min":0.52400000000000002},"min":{"1min":0.34799999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.78100000000000003,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57599999999999996,"15min":0.56599999999999995},"min":{"1min":0.42499999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.98999999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.56100000000000005}]},{"osd":2,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.54700000000000004,"15min":0.53200000000000003},"min":{"1min":0.439,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.753,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.39700000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.76900000000000002,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.58899999999999997}]},{"osd":3,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57299999999999995,"15min":0.54300000000000004},"min":{"1min":0.36699999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.84199999999999997,"5min":1.046,"15min":1.046},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56899999999999995,"15min":0.54500000000000004},"min":{"1min":0.39800000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.81000000000000005,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.60299999999999998}]},{"osd":4,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.47299999999999998,"15min":0.496},"min":{"1min":0.371,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.435},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.55200000000000005,"15min":0.56299999999999994},"min":{"1min":0.38100000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.36099999999999999}]},{"osd":5,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.48299999999999998,"15min":0.48699999999999999},"min":{"1min":0.34499999999999997,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.69499999999999995,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.52300000000000002,"15min":0.55200000000000005},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.39200000000000002}]},{"osd":7,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.52700000000000002,"15min":0.54000000000000004},"min":{"1min":0.31,"5min":0.307,"15min":0.307},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.54500000000000004,"15min":0.54800000000000004},"min":{"1min":0.40200000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76000000000000001,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.52000000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1240,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194072,"statfs":{"total":200000000,"available":198730137,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:15:49 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.33300000000000002,"15min":0.309},"min":{"1min":0.20300000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.51600000000000001,"5min":0.56899999999999995,"15min":0.56899999999999995},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.39300000000000002,"15min":0.379},"min":{"1min":0.27300000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.35799999999999998}]},{"osd":2,"last update":"Sun Oct 8 00:15:49 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.34699999999999998,"15min":0.33600000000000002},"min":{"1min":0.26300000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.58999999999999997,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.376,"15min":0.34399999999999997},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.58999999999999997,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.317}]},{"osd":3,"last update":"Sun Oct 8 00:15:49 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.39000000000000001,"15min":0.372},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.35299999999999998,"15min":0.34399999999999997},"min":{"1min":0.28299999999999997,"5min":0.222,"15min":0.222},"max":{"1min":0.53600000000000003,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.36599999999999999}]},{"osd":4,"last update":"Sun Oct 8 00:15:49 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55900000000000005,"15min":0.54100000000000004},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58099999999999996,"15min":0.56899999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.42899999999999999}]},{"osd":5,"last update":"Sun Oct 8 00:15:49 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56200000000000006,"15min":0.53900000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.83799999999999997,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.58999999999999997,"15min":0.57699999999999996},"min":{"1min":0.38300000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.46400000000000002}]},{"osd":6,"last update":"Sun Oct 8 00:15:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58499999999999996,"15min":0.56899999999999995},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.57999999999999996,"15min":0.56399999999999995},"min":{"1min":0.39600000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77000000000000002,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.47399999999999998}]},{"osd":7,"last update":"Sun Oct 8 00:15:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.58299999999999996,"15min":0.56599999999999995},"min":{"1min":0.39500000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.61599999999999999,"15min":0.59399999999999997},"min":{"1min":0.40500000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.89500000000000002,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.439}]}]},{"osd":0,"up_from":13,"seq":55834574918,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":661,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194650,"statfs":{"total":200000000,"available":199322153,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 00:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.34100000000000003,"15min":0.33200000000000002},"min":{"1min":0.20699999999999999,"5min":0.193,"15min":0.193},"max":{"1min":0.57599999999999996,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.35599999999999998,"15min":0.34799999999999998},"min":{"1min":0.218,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.57199999999999995,"5min":0.87,"15min":0.87},"last":0.377}]},{"osd":2,"last update":"Sun Oct 8 00:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.38900000000000001,"15min":0.38800000000000001},"min":{"1min":0.27600000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.623,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.379,"5min":0.46400000000000002,"15min":0.72799999999999998},"min":{"1min":0.245,"5min":0.185,"15min":0.185},"max":{"1min":0.65600000000000003,"5min":12.019,"15min":12.019},"last":0.30299999999999999}]},{"osd":3,"last update":"Sun Oct 8 00:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.36799999999999999,"15min":0.36399999999999999},"min":{"1min":0.27500000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.54300000000000004,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.318},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.41399999999999998,"15min":0.39900000000000002},"min":{"1min":0.29999999999999999,"5min":0.246,"15min":0.246},"max":{"1min":0.67400000000000004,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.41199999999999998}]},{"osd":4,"last update":"Sun Oct 8 00:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61399999999999999,"15min":0.61699999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.90300000000000002,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.496},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.626,"15min":0.626},"min":{"1min":0.439,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.873,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.59899999999999998}]},{"osd":5,"last update":"Sun Oct 8 00:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.69099999999999995,"15min":0.96699999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.751,"5min":11.992000000000001,"15min":11.992000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60099999999999998,"15min":0.58899999999999997},"min":{"1min":0.46800000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.85599999999999998,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.626}]},{"osd":6,"last update":"Sun Oct 8 00:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.59899999999999998,"15min":0.58899999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.88100000000000001,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.69599999999999995,"15min":0.94799999999999995},"min":{"1min":0.41299999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84499999999999997,"5min":11.927,"15min":11.927},"last":0.61199999999999999}]},{"osd":7,"last update":"Sun Oct 8 00:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.70799999999999996,"15min":0.96899999999999997},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.84099999999999997,"5min":11.965,"15min":11.965},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.626,"15min":0.624},"min":{"1min":0.42699999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81100000000000005,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.64900000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574918,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":662,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194650,"statfs":{"total":200000000,"available":199322083,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.35299999999999998,"15min":0.38900000000000001},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.45600000000000002,"5min":2.5139999999999998,"15min":2.5139999999999998},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.35799999999999998,"15min":0.40000000000000002},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.53800000000000003,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":0.32600000000000001}]},{"osd":1,"last update":"Sun Oct 8 00:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.35999999999999999,"15min":0.39200000000000002},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.65300000000000002,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.39000000000000001,"15min":0.434},"min":{"1min":0.254,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.58499999999999996,"5min":2.5379999999999998,"15min":2.5379999999999998},"last":0.38800000000000001}]},{"osd":3,"last update":"Sun Oct 8 00:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.39800000000000002,"15min":0.42899999999999999},"min":{"1min":0.27400000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.60599999999999998,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.41799999999999998,"15min":0.46200000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.63,"5min":2.5619999999999998,"15min":2.5619999999999998},"last":0.45600000000000002}]},{"osd":4,"last update":"Sun Oct 8 00:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.57899999999999996,"15min":0.61499999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.81499999999999995,"5min":2.488,"15min":2.488},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59299999999999997,"15min":0.60699999999999998},"min":{"1min":0.41799999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.81000000000000005,"5min":2.073,"15min":2.073},"last":0.68500000000000005}]},{"osd":5,"last update":"Sun Oct 8 00:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.625,"15min":0.65700000000000003},"min":{"1min":0.36899999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83099999999999996,"5min":2.2069999999999999,"15min":2.2069999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.621,"5min":0.64200000000000002,"15min":0.66300000000000003},"min":{"1min":0.41599999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83799999999999997,"5min":2.456,"15min":2.456},"last":0.65300000000000002}]},{"osd":6,"last update":"Sun Oct 8 00:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.60599999999999998,"15min":0.622},"min":{"1min":0.40899999999999997,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.80000000000000004,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.60899999999999999,"15min":0.61399999999999999},"min":{"1min":0.439,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.92200000000000004,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.59099999999999997}]},{"osd":7,"last update":"Sun Oct 8 00:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.621,"15min":0.63600000000000001},"min":{"1min":0.45700000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.75700000000000001,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61499999999999999,"15min":0.629},"min":{"1min":0.45000000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85999999999999999,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":0.73499999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574918,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":662,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194650,"statfs":{"total":200000000,"available":199321959,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.33300000000000002,"15min":0.36899999999999999},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":0.71199999999999997,"5min":1.46,"15min":1.46},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.33700000000000002,"15min":0.34200000000000003},"min":{"1min":0.20399999999999999,"5min":0.192,"15min":0.192},"max":{"1min":0.504,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.40799999999999997}]},{"osd":1,"last update":"Sun Oct 8 00:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.46899999999999997,"15min":0.79700000000000004},"min":{"1min":0.22700000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.55200000000000005,"5min":12.351000000000001,"15min":12.351000000000001},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.375,"5min":0.36699999999999999,"15min":0.36599999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.60199999999999998,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.54200000000000004}]},{"osd":2,"last update":"Sun Oct 8 00:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.39500000000000002,"15min":0.41899999999999998},"min":{"1min":0.27700000000000002,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.60899999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.39300000000000002,"15min":0.38800000000000001},"min":{"1min":0.28699999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.64700000000000002,"5min":0.753,"15min":0.753},"last":0.442}]},{"osd":4,"last update":"Sun Oct 8 00:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.70799999999999996,"15min":0.98699999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.024,"5min":11.333,"15min":11.333},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.72699999999999998,"15min":1.0169999999999999},"min":{"1min":0.43099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84999999999999998,"5min":12.321999999999999,"15min":12.321999999999999},"last":0.64500000000000002}]},{"osd":5,"last update":"Sun Oct 8 00:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.72099999999999997,"15min":1.0269999999999999},"min":{"1min":0.38300000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91000000000000003,"5min":12.291,"15min":12.291},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.61799999999999999,"15min":0.60399999999999998},"min":{"1min":0.40999999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.73199999999999998}]},{"osd":6,"last update":"Sun Oct 8 00:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.68200000000000005,"15min":0.95199999999999996},"min":{"1min":0.35299999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.88,"5min":11.288,"15min":11.288},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.71199999999999997,"15min":1.01},"min":{"1min":0.40600000000000003,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.002,"5min":12.253,"15min":12.253},"last":0.71899999999999997}]},{"osd":7,"last update":"Sun Oct 8 00:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.60099999999999998,"15min":0.57899999999999996},"min":{"1min":0.39700000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80300000000000005,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.72199999999999998,"15min":0.98999999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0549999999999999,"5min":11.363,"15min":11.363},"last":0.74399999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574918,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":662,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194649,"statfs":{"total":200000000,"available":199321599,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.56899999999999995,"15min":0.55700000000000005},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.78100000000000003,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57399999999999995,"15min":0.58399999999999996},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.72899999999999998,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.49299999999999999}]},{"osd":1,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.52500000000000002,"15min":0.52100000000000002},"min":{"1min":0.33900000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.53100000000000003,"15min":0.53400000000000003},"min":{"1min":0.314,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.80400000000000005,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.50600000000000001}]},{"osd":2,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.51500000000000001,"15min":0.51900000000000002},"min":{"1min":0.41699999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.68700000000000006,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.52400000000000002,"15min":0.51300000000000001},"min":{"1min":0.374,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.433}]},{"osd":3,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.53700000000000003,"15min":0.54000000000000004},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.79800000000000004,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.53500000000000003,"15min":0.53400000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.47199999999999998}]},{"osd":5,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.46600000000000003,"15min":0.49399999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.317},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.495,"15min":0.51000000000000001},"min":{"1min":0.26400000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.41299999999999998}]},{"osd":6,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.47199999999999998,"15min":0.48799999999999999},"min":{"1min":0.32200000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.65000000000000002,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52200000000000002,"15min":0.55200000000000005},"min":{"1min":0.35399999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.52600000000000002}]},{"osd":7,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.53400000000000003,"15min":0.55100000000000005},"min":{"1min":0.35399999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.79700000000000004,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.56899999999999995,"15min":0.57999999999999996},"min":{"1min":0.45500000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.77500000000000002,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.54900000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574918,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1240,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194072,"statfs":{"total":200000000,"available":198730193,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48599999999999999},"min":{"1min":0.28799999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.66700000000000004,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53800000000000003,"15min":0.55700000000000005},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.70999999999999996,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.55200000000000005}]},{"osd":1,"last update":"Sun Oct 8 00:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.39000000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.71799999999999997,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50700000000000001,"15min":0.52300000000000002},"min":{"1min":0.33100000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70699999999999996,"5min":0.755,"15min":0.755},"last":0.56599999999999995}]},{"osd":2,"last update":"Sun Oct 8 00:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.52300000000000002,"15min":0.52900000000000003},"min":{"1min":0.39500000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.64700000000000002,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.55000000000000004,"15min":0.56699999999999995},"min":{"1min":0.33700000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.68799999999999994,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.42299999999999999}]},{"osd":3,"last update":"Sun Oct 8 00:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.55100000000000005,"15min":0.54700000000000004},"min":{"1min":0.35499999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.73199999999999998,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.436},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.55800000000000005,"15min":0.57599999999999996},"min":{"1min":0.38,"5min":0.371,"15min":0.371},"max":{"1min":0.71999999999999997,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.58099999999999996}]},{"osd":4,"last update":"Sun Oct 8 00:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.46999999999999997,"15min":0.47399999999999998},"min":{"1min":0.312,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.69399999999999995,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.51300000000000001,"15min":0.51500000000000001},"min":{"1min":0.40000000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.68300000000000005,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.50800000000000001}]},{"osd":6,"last update":"Sun Oct 8 00:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.505,"15min":0.501},"min":{"1min":0.36299999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.63200000000000001,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.505,"15min":0.496},"min":{"1min":0.36199999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68300000000000005,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.52400000000000002}]},{"osd":7,"last update":"Sun Oct 8 00:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52600000000000002,"15min":0.51900000000000002},"min":{"1min":0.379,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.66600000000000004,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55600000000000005,"15min":0.55700000000000005},"min":{"1min":0.34499999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.69599999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.47799999999999998}]}]}],"pool_statfs":[]}} 2023-10-08T00:16:26.226 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T00:16:26.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.373+0000 7fcd4bebe700 1 Processor -- start 2023-10-08T00:16:26.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.374+0000 7fcd4bebe700 1 -- start start 2023-10-08T00:16:26.375 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.374+0000 7fcd4bebe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd44152010 0x7fcd4415b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:26.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.374+0000 7fcd4bebe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fcd441523e0 0x7fcd4406c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:26.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.374+0000 7fcd4bebe700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd4406c010 con 0x7fcd44158e50 2023-10-08T00:16:26.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.374+0000 7fcd4bebe700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd4406bd10 con 0x7fcd441523e0 2023-10-08T00:16:26.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.374+0000 7fcd4bebe700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4415c060 con 0x7fcd44152010 2023-10-08T00:16:26.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.374+0000 7fcd49459700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fcd441523e0 0x7fcd4406c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:26.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.374+0000 7fcd4a45b700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fcd44158e50 0x7fcd440cdf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50666/0 (socket says 172.21.15.99:50666) 2023-10-08T00:16:26.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.374+0000 7fcd4a45b700 1 -- 172.21.15.99:0/2730674901 learned_addr learned my addr 172.21.15.99:0/2730674901 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:26.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.375+0000 7fcd49c5a700 1 --2- 172.21.15.99:0/2730674901 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd44152010 0x7fcd4415b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:26.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.375+0000 7fcd49459700 1 -- 172.21.15.99:0/2730674901 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd44152010 msgr2=0x7fcd4415b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:26.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.375+0000 7fcd49459700 1 --2- 172.21.15.99:0/2730674901 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd44152010 0x7fcd4415b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:26.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.375+0000 7fcd49459700 1 -- 172.21.15.99:0/2730674901 >> v1:172.21.15.99:6789/0 conn(0x7fcd44158e50 legacy=0x7fcd440cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:26.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.375+0000 7fcd49459700 1 -- 172.21.15.99:0/2730674901 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd4415c340 con 0x7fcd441523e0 2023-10-08T00:16:26.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.375+0000 7fcd48c58700 1 -- 172.21.15.99:0/2730674901 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 638703632 0 0) 0x7fcd4406c010 con 0x7fcd44158e50 2023-10-08T00:16:26.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.375+0000 7fcd49459700 1 --2- 172.21.15.99:0/2730674901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd441523e0 0x7fcd4406c4a0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fcd340098e0 tx=0x7fcd340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6392c21fe1c3e9fc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:26.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.375+0000 7fcd48c58700 1 -- 172.21.15.99:0/2730674901 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd34009e80 con 0x7fcd441523e0 2023-10-08T00:16:26.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.375+0000 7fcd48c58700 1 -- 172.21.15.99:0/2730674901 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd3400b860 con 0x7fcd441523e0 2023-10-08T00:16:26.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.375+0000 7fcd48c58700 1 -- 172.21.15.99:0/2730674901 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3400bbe0 con 0x7fcd441523e0 2023-10-08T00:16:26.380 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.375+0000 7fcd4bebe700 1 -- 172.21.15.99:0/2730674901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd441523e0 msgr2=0x7fcd4406c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:26.380 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.375+0000 7fcd4bebe700 1 --2- 172.21.15.99:0/2730674901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd441523e0 0x7fcd4406c4a0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fcd340098e0 tx=0x7fcd340091a0 comp rx=0 tx=0).stop 2023-10-08T00:16:26.380 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.376+0000 7fcd4bebe700 1 -- 172.21.15.99:0/2730674901 shutdown_connections 2023-10-08T00:16:26.380 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.376+0000 7fcd4bebe700 1 --2- 172.21.15.99:0/2730674901 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd44152010 0x7fcd4415b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:26.380 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.376+0000 7fcd4bebe700 1 --2- 172.21.15.99:0/2730674901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd441523e0 0x7fcd4406c4a0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:26.380 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.376+0000 7fcd4bebe700 1 -- 172.21.15.99:0/2730674901 >> 172.21.15.99:0/2730674901 conn(0x7fcd44069870 msgr2=0x7fcd44069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:26.380 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.376+0000 7fcd4bebe700 1 -- 172.21.15.99:0/2730674901 shutdown_connections 2023-10-08T00:16:26.380 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.376+0000 7fcd4bebe700 1 -- 172.21.15.99:0/2730674901 wait complete. 2023-10-08T00:16:26.380 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.377+0000 7fcd4bebe700 1 Processor -- start 2023-10-08T00:16:26.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.377+0000 7fcd4bebe700 1 -- start start 2023-10-08T00:16:26.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.377+0000 7fcd4bebe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd44152010 0x7fcd4416b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:26.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.377+0000 7fcd4bebe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd441523e0 0x7fcd4414d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:26.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.377+0000 7fcd4bebe700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd44158e50 0x7fcd44167ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:26.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.377+0000 7fcd4bebe700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4416c880 con 0x7fcd441523e0 2023-10-08T00:16:26.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.377+0000 7fcd4bebe700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4416bfe0 con 0x7fcd44158e50 2023-10-08T00:16:26.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.377+0000 7fcd4bebe700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4416c2c0 con 0x7fcd44152010 2023-10-08T00:16:26.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.378+0000 7fcd49459700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd441523e0 0x7fcd4414d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:26.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.378+0000 7fcd49459700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd441523e0 0x7fcd4414d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37066/0 (socket says 172.21.15.99:37066) 2023-10-08T00:16:26.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.378+0000 7fcd49459700 1 -- 172.21.15.99:0/146752424 learned_addr learned my addr 172.21.15.99:0/146752424 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:26.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.378+0000 7fcd4a45b700 1 --2- 172.21.15.99:0/146752424 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd44158e50 0x7fcd44167ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:26.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.378+0000 7fcd49c5a700 1 --2- 172.21.15.99:0/146752424 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd44152010 0x7fcd4416b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:26.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.378+0000 7fcd49459700 1 -- 172.21.15.99:0/146752424 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd44152010 msgr2=0x7fcd4416b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:26.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.378+0000 7fcd49459700 1 --2- 172.21.15.99:0/146752424 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd44152010 0x7fcd4416b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:26.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.378+0000 7fcd49459700 1 -- 172.21.15.99:0/146752424 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd44158e50 msgr2=0x7fcd44167ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:26.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.378+0000 7fcd49459700 1 --2- 172.21.15.99:0/146752424 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd44158e50 0x7fcd44167ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:26.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.378+0000 7fcd49459700 1 -- 172.21.15.99:0/146752424 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd34009420 con 0x7fcd441523e0 2023-10-08T00:16:26.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.378+0000 7fcd49459700 1 --2- 172.21.15.99:0/146752424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd441523e0 0x7fcd4414d460 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fcd34013000 tx=0x7fcd34017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:26.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.378+0000 7fcd39ffb700 1 -- 172.21.15.99:0/146752424 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3400b9d0 con 0x7fcd441523e0 2023-10-08T00:16:26.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.378+0000 7fcd39ffb700 1 -- 172.21.15.99:0/146752424 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd3400bb30 con 0x7fcd441523e0 2023-10-08T00:16:26.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.378+0000 7fcd4bebe700 1 -- 172.21.15.99:0/146752424 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd4416c5a0 con 0x7fcd441523e0 2023-10-08T00:16:26.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.378+0000 7fcd39ffb700 1 -- 172.21.15.99:0/146752424 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd34025670 con 0x7fcd441523e0 2023-10-08T00:16:26.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.378+0000 7fcd4bebe700 1 -- 172.21.15.99:0/146752424 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd44167ff0 con 0x7fcd441523e0 2023-10-08T00:16:26.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.379+0000 7fcd4bebe700 1 -- 172.21.15.99:0/146752424 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd440b3670 con 0x7fcd441523e0 2023-10-08T00:16:26.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.380+0000 7fcd39ffb700 1 -- 172.21.15.99:0/146752424 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcd34023070 con 0x7fcd441523e0 2023-10-08T00:16:26.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.380+0000 7fcd39ffb700 1 --2- 172.21.15.99:0/146752424 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcd28066fb0 0x7fcd28069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:26.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.380+0000 7fcd39ffb700 1 -- 172.21.15.99:0/146752424 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcd34085bd0 con 0x7fcd441523e0 2023-10-08T00:16:26.384 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.381+0000 7fcd49c5a700 1 --2- 172.21.15.99:0/146752424 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcd28066fb0 0x7fcd28069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:26.384 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.381+0000 7fcd49c5a700 1 --2- 172.21.15.99:0/146752424 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcd28066fb0 0x7fcd28069460 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fcd40006fd0 tx=0x7fcd40008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:26.385 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.385+0000 7fcd39ffb700 1 -- 172.21.15.99:0/146752424 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcd440b3670 con 0x7fcd441523e0 2023-10-08T00:16:26.562 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.561+0000 7fcd4bebe700 1 -- 172.21.15.99:0/146752424 --> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcd440b3670 con 0x7fcd28066fb0 2023-10-08T00:16:26.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.570+0000 7fcd39ffb700 1 -- 172.21.15.99:0/146752424 <== mgr.4099 v2:172.21.15.191:6832/85985 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285657 (secure 0 0 0) 0x7fcd440b3670 con 0x7fcd28066fb0 2023-10-08T00:16:26.573 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:16:26.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.572+0000 7fcd4bebe700 1 -- 172.21.15.99:0/146752424 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcd28066fb0 msgr2=0x7fcd28069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:26.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.572+0000 7fcd4bebe700 1 --2- 172.21.15.99:0/146752424 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcd28066fb0 0x7fcd28069460 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fcd40006fd0 tx=0x7fcd40008040 comp rx=0 tx=0).stop 2023-10-08T00:16:26.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.572+0000 7fcd4bebe700 1 -- 172.21.15.99:0/146752424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd441523e0 msgr2=0x7fcd4414d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:26.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.572+0000 7fcd4bebe700 1 --2- 172.21.15.99:0/146752424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd441523e0 0x7fcd4414d460 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fcd34013000 tx=0x7fcd34017000 comp rx=0 tx=0).stop 2023-10-08T00:16:26.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.572+0000 7fcd4bebe700 1 -- 172.21.15.99:0/146752424 shutdown_connections 2023-10-08T00:16:26.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.572+0000 7fcd4bebe700 1 --2- 172.21.15.99:0/146752424 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcd28066fb0 0x7fcd28069460 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:26.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.572+0000 7fcd4bebe700 1 --2- 172.21.15.99:0/146752424 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd44158e50 0x7fcd44167ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:26.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.572+0000 7fcd4bebe700 1 --2- 172.21.15.99:0/146752424 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd44152010 0x7fcd4416b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:26.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.572+0000 7fcd4bebe700 1 --2- 172.21.15.99:0/146752424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd441523e0 0x7fcd4414d460 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:26.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.572+0000 7fcd4bebe700 1 -- 172.21.15.99:0/146752424 >> 172.21.15.99:0/146752424 conn(0x7fcd44069870 msgr2=0x7fcd440cacc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:26.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.572+0000 7fcd4bebe700 1 -- 172.21.15.99:0/146752424 shutdown_connections 2023-10-08T00:16:26.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.573+0000 7fcd4bebe700 1 -- 172.21.15.99:0/146752424 wait complete. 2023-10-08T00:16:26.576 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2023-10-08T00:16:26.585 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":286,"stamp":"2023-10-08T00:16:26.103515+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":92,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6453,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1556046,"statfs":{"total":1600000000,"available":1593391876,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.957491"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.168926+0000","last_change":"2023-10-08T00:16:22.168926+0000","last_active":"2023-10-08T00:16:22.168926+0000","last_peered":"2023-10-08T00:16:22.168926+0000","last_clean":"2023-10-08T00:16:22.168926+0000","last_became_active":"2023-10-08T00:16:22.168678+0000","last_became_peered":"2023-10-08T00:16:22.168678+0000","last_unstale":"2023-10-08T00:16:22.168926+0000","last_undegraded":"2023-10-08T00:16:22.168926+0000","last_fullsized":"2023-10-08T00:16:22.168926+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:50:42.638829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.166603+0000","last_change":"2023-10-08T00:16:22.166603+0000","last_active":"2023-10-08T00:16:22.166603+0000","last_peered":"2023-10-08T00:16:22.166603+0000","last_clean":"2023-10-08T00:16:22.166603+0000","last_became_active":"2023-10-08T00:16:22.166090+0000","last_became_peered":"2023-10-08T00:16:22.166090+0000","last_unstale":"2023-10-08T00:16:22.166603+0000","last_undegraded":"2023-10-08T00:16:22.166603+0000","last_fullsized":"2023-10-08T00:16:22.166603+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:35:16.330010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.169784+0000","last_change":"2023-10-08T00:16:22.169784+0000","last_active":"2023-10-08T00:16:22.169784+0000","last_peered":"2023-10-08T00:16:22.169784+0000","last_clean":"2023-10-08T00:16:22.169784+0000","last_became_active":"2023-10-08T00:16:22.166646+0000","last_became_peered":"2023-10-08T00:16:22.166646+0000","last_unstale":"2023-10-08T00:16:22.169784+0000","last_undegraded":"2023-10-08T00:16:22.169784+0000","last_fullsized":"2023-10-08T00:16:22.169784+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:32:41.484943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.162701+0000","last_change":"2023-10-08T00:16:22.162701+0000","last_active":"2023-10-08T00:16:22.162701+0000","last_peered":"2023-10-08T00:16:22.162701+0000","last_clean":"2023-10-08T00:16:22.162701+0000","last_became_active":"2023-10-08T00:16:22.162131+0000","last_became_peered":"2023-10-08T00:16:22.162131+0000","last_unstale":"2023-10-08T00:16:22.162701+0000","last_undegraded":"2023-10-08T00:16:22.162701+0000","last_fullsized":"2023-10-08T00:16:22.162701+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:35:56.520235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.163038+0000","last_change":"2023-10-08T00:16:22.163038+0000","last_active":"2023-10-08T00:16:22.163038+0000","last_peered":"2023-10-08T00:16:22.163038+0000","last_clean":"2023-10-08T00:16:22.163038+0000","last_became_active":"2023-10-08T00:16:22.162809+0000","last_became_peered":"2023-10-08T00:16:22.162809+0000","last_unstale":"2023-10-08T00:16:22.163038+0000","last_undegraded":"2023-10-08T00:16:22.163038+0000","last_fullsized":"2023-10-08T00:16:22.163038+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:52:50.789133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.164226+0000","last_change":"2023-10-08T00:16:22.164226+0000","last_active":"2023-10-08T00:16:22.164226+0000","last_peered":"2023-10-08T00:16:22.164226+0000","last_clean":"2023-10-08T00:16:22.164226+0000","last_became_active":"2023-10-08T00:16:22.163362+0000","last_became_peered":"2023-10-08T00:16:22.163362+0000","last_unstale":"2023-10-08T00:16:22.164226+0000","last_undegraded":"2023-10-08T00:16:22.164226+0000","last_fullsized":"2023-10-08T00:16:22.164226+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:47:35.001042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.164274+0000","last_change":"2023-10-08T00:16:22.164274+0000","last_active":"2023-10-08T00:16:22.164274+0000","last_peered":"2023-10-08T00:16:22.164274+0000","last_clean":"2023-10-08T00:16:22.164274+0000","last_became_active":"2023-10-08T00:16:22.164026+0000","last_became_peered":"2023-10-08T00:16:22.164026+0000","last_unstale":"2023-10-08T00:16:22.164274+0000","last_undegraded":"2023-10-08T00:16:22.164274+0000","last_fullsized":"2023-10-08T00:16:22.164274+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:51:24.231446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.169877+0000","last_change":"2023-10-08T00:16:22.169877+0000","last_active":"2023-10-08T00:16:22.169877+0000","last_peered":"2023-10-08T00:16:22.169877+0000","last_clean":"2023-10-08T00:16:22.169877+0000","last_became_active":"2023-10-08T00:16:22.166990+0000","last_became_peered":"2023-10-08T00:16:22.166990+0000","last_unstale":"2023-10-08T00:16:22.169877+0000","last_undegraded":"2023-10-08T00:16:22.169877+0000","last_fullsized":"2023-10-08T00:16:22.169877+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:09:25.675403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.162665+0000","last_change":"2023-10-08T00:16:22.162665+0000","last_active":"2023-10-08T00:16:22.162665+0000","last_peered":"2023-10-08T00:16:22.162665+0000","last_clean":"2023-10-08T00:16:22.162665+0000","last_became_active":"2023-10-08T00:16:22.162451+0000","last_became_peered":"2023-10-08T00:16:22.162451+0000","last_unstale":"2023-10-08T00:16:22.162665+0000","last_undegraded":"2023-10-08T00:16:22.162665+0000","last_fullsized":"2023-10-08T00:16:22.162665+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:21:05.119155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.168752+0000","last_change":"2023-10-08T00:16:22.168752+0000","last_active":"2023-10-08T00:16:22.168752+0000","last_peered":"2023-10-08T00:16:22.168752+0000","last_clean":"2023-10-08T00:16:22.168752+0000","last_became_active":"2023-10-08T00:16:22.168518+0000","last_became_peered":"2023-10-08T00:16:22.168518+0000","last_unstale":"2023-10-08T00:16:22.168752+0000","last_undegraded":"2023-10-08T00:16:22.168752+0000","last_fullsized":"2023-10-08T00:16:22.168752+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:36:52.374149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.164446+0000","last_change":"2023-10-08T00:16:22.164446+0000","last_active":"2023-10-08T00:16:22.164446+0000","last_peered":"2023-10-08T00:16:22.164446+0000","last_clean":"2023-10-08T00:16:22.164446+0000","last_became_active":"2023-10-08T00:16:22.163950+0000","last_became_peered":"2023-10-08T00:16:22.163950+0000","last_unstale":"2023-10-08T00:16:22.164446+0000","last_undegraded":"2023-10-08T00:16:22.164446+0000","last_fullsized":"2023-10-08T00:16:22.164446+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:42:45.613813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.167538+0000","last_change":"2023-10-08T00:16:22.167538+0000","last_active":"2023-10-08T00:16:22.167538+0000","last_peered":"2023-10-08T00:16:22.167538+0000","last_clean":"2023-10-08T00:16:22.167538+0000","last_became_active":"2023-10-08T00:16:22.167134+0000","last_became_peered":"2023-10-08T00:16:22.167134+0000","last_unstale":"2023-10-08T00:16:22.167538+0000","last_undegraded":"2023-10-08T00:16:22.167538+0000","last_fullsized":"2023-10-08T00:16:22.167538+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:54:43.701059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.156907+0000","last_change":"2023-10-08T00:16:22.156907+0000","last_active":"2023-10-08T00:16:22.156907+0000","last_peered":"2023-10-08T00:16:22.156907+0000","last_clean":"2023-10-08T00:16:22.156907+0000","last_became_active":"2023-10-08T00:16:22.156691+0000","last_became_peered":"2023-10-08T00:16:22.156691+0000","last_unstale":"2023-10-08T00:16:22.156907+0000","last_undegraded":"2023-10-08T00:16:22.156907+0000","last_fullsized":"2023-10-08T00:16:22.156907+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:32:41.484943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.162851+0000","last_change":"2023-10-08T00:16:22.162851+0000","last_active":"2023-10-08T00:16:22.162851+0000","last_peered":"2023-10-08T00:16:22.162851+0000","last_clean":"2023-10-08T00:16:22.162851+0000","last_became_active":"2023-10-08T00:16:22.162466+0000","last_became_peered":"2023-10-08T00:16:22.162466+0000","last_unstale":"2023-10-08T00:16:22.162851+0000","last_undegraded":"2023-10-08T00:16:22.162851+0000","last_fullsized":"2023-10-08T00:16:22.162851+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:05:55.518464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.166798+0000","last_change":"2023-10-08T00:16:22.166798+0000","last_active":"2023-10-08T00:16:22.166798+0000","last_peered":"2023-10-08T00:16:22.166798+0000","last_clean":"2023-10-08T00:16:22.166798+0000","last_became_active":"2023-10-08T00:16:22.166492+0000","last_became_peered":"2023-10-08T00:16:22.166492+0000","last_unstale":"2023-10-08T00:16:22.166798+0000","last_undegraded":"2023-10-08T00:16:22.166798+0000","last_fullsized":"2023-10-08T00:16:22.166798+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:44:20.663579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.159081+0000","last_change":"2023-10-08T00:16:22.159081+0000","last_active":"2023-10-08T00:16:22.159081+0000","last_peered":"2023-10-08T00:16:22.159081+0000","last_clean":"2023-10-08T00:16:22.159081+0000","last_became_active":"2023-10-08T00:16:22.158851+0000","last_became_peered":"2023-10-08T00:16:22.158851+0000","last_unstale":"2023-10-08T00:16:22.159081+0000","last_undegraded":"2023-10-08T00:16:22.159081+0000","last_fullsized":"2023-10-08T00:16:22.159081+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:48:39.888939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.158983+0000","last_change":"2023-10-08T00:16:22.158983+0000","last_active":"2023-10-08T00:16:22.158983+0000","last_peered":"2023-10-08T00:16:22.158983+0000","last_clean":"2023-10-08T00:16:22.158983+0000","last_became_active":"2023-10-08T00:16:22.158200+0000","last_became_peered":"2023-10-08T00:16:22.158200+0000","last_unstale":"2023-10-08T00:16:22.158983+0000","last_undegraded":"2023-10-08T00:16:22.158983+0000","last_fullsized":"2023-10-08T00:16:22.158983+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:21:44.003712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.165593+0000","last_change":"2023-10-08T00:16:22.165593+0000","last_active":"2023-10-08T00:16:22.165593+0000","last_peered":"2023-10-08T00:16:22.165593+0000","last_clean":"2023-10-08T00:16:22.165593+0000","last_became_active":"2023-10-08T00:16:22.165367+0000","last_became_peered":"2023-10-08T00:16:22.165367+0000","last_unstale":"2023-10-08T00:16:22.165593+0000","last_undegraded":"2023-10-08T00:16:22.165593+0000","last_fullsized":"2023-10-08T00:16:22.165593+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:53:34.263422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.165305+0000","last_change":"2023-10-08T00:16:22.165305+0000","last_active":"2023-10-08T00:16:22.165305+0000","last_peered":"2023-10-08T00:16:22.165305+0000","last_clean":"2023-10-08T00:16:22.165305+0000","last_became_active":"2023-10-08T00:16:22.164891+0000","last_became_peered":"2023-10-08T00:16:22.164891+0000","last_unstale":"2023-10-08T00:16:22.165305+0000","last_undegraded":"2023-10-08T00:16:22.165305+0000","last_fullsized":"2023-10-08T00:16:22.165305+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:40:12.567676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.164925+0000","last_change":"2023-10-08T00:16:22.164925+0000","last_active":"2023-10-08T00:16:22.164925+0000","last_peered":"2023-10-08T00:16:22.164925+0000","last_clean":"2023-10-08T00:16:22.164925+0000","last_became_active":"2023-10-08T00:16:22.164785+0000","last_became_peered":"2023-10-08T00:16:22.164785+0000","last_unstale":"2023-10-08T00:16:22.164925+0000","last_undegraded":"2023-10-08T00:16:22.164925+0000","last_fullsized":"2023-10-08T00:16:22.164925+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:26:37.495930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.168020+0000","last_change":"2023-10-08T00:16:22.168020+0000","last_active":"2023-10-08T00:16:22.168020+0000","last_peered":"2023-10-08T00:16:22.168020+0000","last_clean":"2023-10-08T00:16:22.168020+0000","last_became_active":"2023-10-08T00:16:22.167398+0000","last_became_peered":"2023-10-08T00:16:22.167398+0000","last_unstale":"2023-10-08T00:16:22.168020+0000","last_undegraded":"2023-10-08T00:16:22.168020+0000","last_fullsized":"2023-10-08T00:16:22.168020+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:59:13.288375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.161097+0000","last_change":"2023-10-08T00:16:22.161097+0000","last_active":"2023-10-08T00:16:22.161097+0000","last_peered":"2023-10-08T00:16:22.161097+0000","last_clean":"2023-10-08T00:16:22.161097+0000","last_became_active":"2023-10-08T00:16:22.160843+0000","last_became_peered":"2023-10-08T00:16:22.160843+0000","last_unstale":"2023-10-08T00:16:22.161097+0000","last_undegraded":"2023-10-08T00:16:22.161097+0000","last_fullsized":"2023-10-08T00:16:22.161097+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:34:33.394447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.168529+0000","last_change":"2023-10-08T00:16:22.168529+0000","last_active":"2023-10-08T00:16:22.168529+0000","last_peered":"2023-10-08T00:16:22.168529+0000","last_clean":"2023-10-08T00:16:22.168529+0000","last_became_active":"2023-10-08T00:16:22.167019+0000","last_became_peered":"2023-10-08T00:16:22.167019+0000","last_unstale":"2023-10-08T00:16:22.168529+0000","last_undegraded":"2023-10-08T00:16:22.168529+0000","last_fullsized":"2023-10-08T00:16:22.168529+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:07:56.723107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.163557+0000","last_change":"2023-10-08T00:16:22.163557+0000","last_active":"2023-10-08T00:16:22.163557+0000","last_peered":"2023-10-08T00:16:22.163557+0000","last_clean":"2023-10-08T00:16:22.163557+0000","last_became_active":"2023-10-08T00:16:22.163320+0000","last_became_peered":"2023-10-08T00:16:22.163320+0000","last_unstale":"2023-10-08T00:16:22.163557+0000","last_undegraded":"2023-10-08T00:16:22.163557+0000","last_fullsized":"2023-10-08T00:16:22.163557+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:09:11.213631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.165451+0000","last_change":"2023-10-08T00:16:22.165451+0000","last_active":"2023-10-08T00:16:22.165451+0000","last_peered":"2023-10-08T00:16:22.165451+0000","last_clean":"2023-10-08T00:16:22.165451+0000","last_became_active":"2023-10-08T00:16:22.165236+0000","last_became_peered":"2023-10-08T00:16:22.165236+0000","last_unstale":"2023-10-08T00:16:22.165451+0000","last_undegraded":"2023-10-08T00:16:22.165451+0000","last_fullsized":"2023-10-08T00:16:22.165451+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:50:42.638829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.156398+0000","last_change":"2023-10-08T00:16:22.156398+0000","last_active":"2023-10-08T00:16:22.156398+0000","last_peered":"2023-10-08T00:16:22.156398+0000","last_clean":"2023-10-08T00:16:22.156398+0000","last_became_active":"2023-10-08T00:16:22.156016+0000","last_became_peered":"2023-10-08T00:16:22.156016+0000","last_unstale":"2023-10-08T00:16:22.156398+0000","last_undegraded":"2023-10-08T00:16:22.156398+0000","last_fullsized":"2023-10-08T00:16:22.156398+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:26:37.495930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.157036+0000","last_change":"2023-10-08T00:16:22.157036+0000","last_active":"2023-10-08T00:16:22.157036+0000","last_peered":"2023-10-08T00:16:22.157036+0000","last_clean":"2023-10-08T00:16:22.157036+0000","last_became_active":"2023-10-08T00:16:22.156786+0000","last_became_peered":"2023-10-08T00:16:22.156786+0000","last_unstale":"2023-10-08T00:16:22.157036+0000","last_undegraded":"2023-10-08T00:16:22.157036+0000","last_fullsized":"2023-10-08T00:16:22.157036+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:35:56.520235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.168747+0000","last_change":"2023-10-08T00:16:22.168747+0000","last_active":"2023-10-08T00:16:22.168747+0000","last_peered":"2023-10-08T00:16:22.168747+0000","last_clean":"2023-10-08T00:16:22.168747+0000","last_became_active":"2023-10-08T00:16:22.168522+0000","last_became_peered":"2023-10-08T00:16:22.168522+0000","last_unstale":"2023-10-08T00:16:22.168747+0000","last_undegraded":"2023-10-08T00:16:22.168747+0000","last_fullsized":"2023-10-08T00:16:22.168747+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:23:11.199659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.164916+0000","last_change":"2023-10-08T00:16:22.164916+0000","last_active":"2023-10-08T00:16:22.164916+0000","last_peered":"2023-10-08T00:16:22.164916+0000","last_clean":"2023-10-08T00:16:22.164916+0000","last_became_active":"2023-10-08T00:16:22.164544+0000","last_became_peered":"2023-10-08T00:16:22.164544+0000","last_unstale":"2023-10-08T00:16:22.164916+0000","last_undegraded":"2023-10-08T00:16:22.164916+0000","last_fullsized":"2023-10-08T00:16:22.164916+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:09:25.675403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.155761+0000","last_change":"2023-10-08T00:16:22.155761+0000","last_active":"2023-10-08T00:16:22.155761+0000","last_peered":"2023-10-08T00:16:22.155761+0000","last_clean":"2023-10-08T00:16:22.155761+0000","last_became_active":"2023-10-08T00:16:22.155520+0000","last_became_peered":"2023-10-08T00:16:22.155520+0000","last_unstale":"2023-10-08T00:16:22.155761+0000","last_undegraded":"2023-10-08T00:16:22.155761+0000","last_fullsized":"2023-10-08T00:16:22.155761+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:34:33.394447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.167870+0000","last_change":"2023-10-08T00:16:22.167870+0000","last_active":"2023-10-08T00:16:22.167870+0000","last_peered":"2023-10-08T00:16:22.167870+0000","last_clean":"2023-10-08T00:16:22.167870+0000","last_became_active":"2023-10-08T00:16:22.167061+0000","last_became_peered":"2023-10-08T00:16:22.167061+0000","last_unstale":"2023-10-08T00:16:22.167870+0000","last_undegraded":"2023-10-08T00:16:22.167870+0000","last_fullsized":"2023-10-08T00:16:22.167870+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:54:17.969681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.166465+0000","last_change":"2023-10-08T00:16:22.166465+0000","last_active":"2023-10-08T00:16:22.166465+0000","last_peered":"2023-10-08T00:16:22.166465+0000","last_clean":"2023-10-08T00:16:22.166465+0000","last_became_active":"2023-10-08T00:16:22.165825+0000","last_became_peered":"2023-10-08T00:16:22.165825+0000","last_unstale":"2023-10-08T00:16:22.166465+0000","last_undegraded":"2023-10-08T00:16:22.166465+0000","last_fullsized":"2023-10-08T00:16:22.166465+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:35:07.078898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.166018+0000","last_change":"2023-10-08T00:16:22.166018+0000","last_active":"2023-10-08T00:16:22.166018+0000","last_peered":"2023-10-08T00:16:22.166018+0000","last_clean":"2023-10-08T00:16:22.166018+0000","last_became_active":"2023-10-08T00:16:22.165795+0000","last_became_peered":"2023-10-08T00:16:22.165795+0000","last_unstale":"2023-10-08T00:16:22.166018+0000","last_undegraded":"2023-10-08T00:16:22.166018+0000","last_fullsized":"2023-10-08T00:16:22.166018+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:47:08.037802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"123'4","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.175178+0000","last_change":"2023-10-08T00:16:21.164000+0000","last_active":"2023-10-08T00:16:22.175178+0000","last_peered":"2023-10-08T00:16:22.175178+0000","last_clean":"2023-10-08T00:16:22.175178+0000","last_became_active":"2023-10-08T00:16:21.163786+0000","last_became_peered":"2023-10-08T00:16:21.163786+0000","last_unstale":"2023-10-08T00:16:22.175178+0000","last_undegraded":"2023-10-08T00:16:22.175178+0000","last_fullsized":"2023-10-08T00:16:22.175178+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:45:18.775774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.165690+0000","last_change":"2023-10-08T00:16:22.165690+0000","last_active":"2023-10-08T00:16:22.165690+0000","last_peered":"2023-10-08T00:16:22.165690+0000","last_clean":"2023-10-08T00:16:22.165690+0000","last_became_active":"2023-10-08T00:16:22.165379+0000","last_became_peered":"2023-10-08T00:16:22.165379+0000","last_unstale":"2023-10-08T00:16:22.165690+0000","last_undegraded":"2023-10-08T00:16:22.165690+0000","last_fullsized":"2023-10-08T00:16:22.165690+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:54:43.701059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"123'1","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.157921+0000","last_change":"2023-10-08T00:16:21.166992+0000","last_active":"2023-10-08T00:16:22.157921+0000","last_peered":"2023-10-08T00:16:22.157921+0000","last_clean":"2023-10-08T00:16:22.157921+0000","last_became_active":"2023-10-08T00:16:21.166731+0000","last_became_peered":"2023-10-08T00:16:21.166731+0000","last_unstale":"2023-10-08T00:16:22.157921+0000","last_undegraded":"2023-10-08T00:16:22.157921+0000","last_fullsized":"2023-10-08T00:16:22.157921+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:41:14.388380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.164171+0000","last_change":"2023-10-08T00:16:22.164171+0000","last_active":"2023-10-08T00:16:22.164171+0000","last_peered":"2023-10-08T00:16:22.164171+0000","last_clean":"2023-10-08T00:16:22.164171+0000","last_became_active":"2023-10-08T00:16:22.163771+0000","last_became_peered":"2023-10-08T00:16:22.163771+0000","last_unstale":"2023-10-08T00:16:22.164171+0000","last_undegraded":"2023-10-08T00:16:22.164171+0000","last_fullsized":"2023-10-08T00:16:22.164171+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:54:00.451741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"123'2","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.172979+0000","last_change":"2023-10-08T00:16:21.164021+0000","last_active":"2023-10-08T00:16:22.172979+0000","last_peered":"2023-10-08T00:16:22.172979+0000","last_clean":"2023-10-08T00:16:22.172979+0000","last_became_active":"2023-10-08T00:16:21.163600+0000","last_became_peered":"2023-10-08T00:16:21.163600+0000","last_unstale":"2023-10-08T00:16:22.172979+0000","last_undegraded":"2023-10-08T00:16:22.172979+0000","last_fullsized":"2023-10-08T00:16:22.172979+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:47:27.489699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.163743+0000","last_change":"2023-10-08T00:16:22.163743+0000","last_active":"2023-10-08T00:16:22.163743+0000","last_peered":"2023-10-08T00:16:22.163743+0000","last_clean":"2023-10-08T00:16:22.163743+0000","last_became_active":"2023-10-08T00:16:22.163532+0000","last_became_peered":"2023-10-08T00:16:22.163532+0000","last_unstale":"2023-10-08T00:16:22.163743+0000","last_undegraded":"2023-10-08T00:16:22.163743+0000","last_fullsized":"2023-10-08T00:16:22.163743+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:53:34.263422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"123'2","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.175614+0000","last_change":"2023-10-08T00:16:21.162920+0000","last_active":"2023-10-08T00:16:22.175614+0000","last_peered":"2023-10-08T00:16:22.175614+0000","last_clean":"2023-10-08T00:16:22.175614+0000","last_became_active":"2023-10-08T00:16:21.162488+0000","last_became_peered":"2023-10-08T00:16:21.162488+0000","last_unstale":"2023-10-08T00:16:22.175614+0000","last_undegraded":"2023-10-08T00:16:22.175614+0000","last_fullsized":"2023-10-08T00:16:22.175614+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:15:13.307047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.163846+0000","last_change":"2023-10-08T00:16:22.163846+0000","last_active":"2023-10-08T00:16:22.163846+0000","last_peered":"2023-10-08T00:16:22.163846+0000","last_clean":"2023-10-08T00:16:22.163846+0000","last_became_active":"2023-10-08T00:16:22.162480+0000","last_became_peered":"2023-10-08T00:16:22.162480+0000","last_unstale":"2023-10-08T00:16:22.163846+0000","last_undegraded":"2023-10-08T00:16:22.163846+0000","last_fullsized":"2023-10-08T00:16:22.163846+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:09:25.675403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"123'1","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.161529+0000","last_change":"2023-10-08T00:16:21.158912+0000","last_active":"2023-10-08T00:16:22.161529+0000","last_peered":"2023-10-08T00:16:22.161529+0000","last_clean":"2023-10-08T00:16:22.161529+0000","last_became_active":"2023-10-08T00:16:21.158683+0000","last_became_peered":"2023-10-08T00:16:21.158683+0000","last_unstale":"2023-10-08T00:16:22.161529+0000","last_undegraded":"2023-10-08T00:16:22.161529+0000","last_fullsized":"2023-10-08T00:16:22.161529+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:34:48.715478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.167201+0000","last_change":"2023-10-08T00:16:22.167201+0000","last_active":"2023-10-08T00:16:22.167201+0000","last_peered":"2023-10-08T00:16:22.167201+0000","last_clean":"2023-10-08T00:16:22.167201+0000","last_became_active":"2023-10-08T00:16:22.166944+0000","last_became_peered":"2023-10-08T00:16:22.166944+0000","last_unstale":"2023-10-08T00:16:22.167201+0000","last_undegraded":"2023-10-08T00:16:22.167201+0000","last_fullsized":"2023-10-08T00:16:22.167201+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:16:19.343525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"123'1","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.162159+0000","last_change":"2023-10-08T00:16:21.160966+0000","last_active":"2023-10-08T00:16:22.162159+0000","last_peered":"2023-10-08T00:16:22.162159+0000","last_clean":"2023-10-08T00:16:22.162159+0000","last_became_active":"2023-10-08T00:16:21.159717+0000","last_became_peered":"2023-10-08T00:16:21.159717+0000","last_unstale":"2023-10-08T00:16:22.162159+0000","last_undegraded":"2023-10-08T00:16:22.162159+0000","last_fullsized":"2023-10-08T00:16:22.162159+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:16:10.889545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.164400+0000","last_change":"2023-10-08T00:16:22.164400+0000","last_active":"2023-10-08T00:16:22.164400+0000","last_peered":"2023-10-08T00:16:22.164400+0000","last_clean":"2023-10-08T00:16:22.164400+0000","last_became_active":"2023-10-08T00:16:22.164128+0000","last_became_peered":"2023-10-08T00:16:22.164128+0000","last_unstale":"2023-10-08T00:16:22.164400+0000","last_undegraded":"2023-10-08T00:16:22.164400+0000","last_fullsized":"2023-10-08T00:16:22.164400+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:54:17.969681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-08T00:16:21.157979+0000","last_change":"2023-10-08T00:16:21.157919+0000","last_active":"2023-10-08T00:16:21.157979+0000","last_peered":"2023-10-08T00:16:21.157979+0000","last_clean":"2023-10-08T00:16:21.157979+0000","last_became_active":"2023-10-08T00:16:21.157695+0000","last_became_peered":"2023-10-08T00:16:21.157695+0000","last_unstale":"2023-10-08T00:16:21.157979+0000","last_undegraded":"2023-10-08T00:16:21.157979+0000","last_fullsized":"2023-10-08T00:16:21.157979+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:59:48.625941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.165112+0000","last_change":"2023-10-08T00:16:22.165112+0000","last_active":"2023-10-08T00:16:22.165112+0000","last_peered":"2023-10-08T00:16:22.165112+0000","last_clean":"2023-10-08T00:16:22.165112+0000","last_became_active":"2023-10-08T00:16:22.164779+0000","last_became_peered":"2023-10-08T00:16:22.164779+0000","last_unstale":"2023-10-08T00:16:22.165112+0000","last_undegraded":"2023-10-08T00:16:22.165112+0000","last_fullsized":"2023-10-08T00:16:22.165112+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:15:14.311899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-08T00:16:21.165975+0000","last_change":"2023-10-08T00:16:21.165191+0000","last_active":"2023-10-08T00:16:21.165975+0000","last_peered":"2023-10-08T00:16:21.165975+0000","last_clean":"2023-10-08T00:16:21.165975+0000","last_became_active":"2023-10-08T00:16:21.164824+0000","last_became_peered":"2023-10-08T00:16:21.164824+0000","last_unstale":"2023-10-08T00:16:21.165975+0000","last_undegraded":"2023-10-08T00:16:21.165975+0000","last_fullsized":"2023-10-08T00:16:21.165975+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:04:07.976115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.162534+0000","last_change":"2023-10-08T00:16:22.162534+0000","last_active":"2023-10-08T00:16:22.162534+0000","last_peered":"2023-10-08T00:16:22.162534+0000","last_clean":"2023-10-08T00:16:22.162534+0000","last_became_active":"2023-10-08T00:16:22.162303+0000","last_became_peered":"2023-10-08T00:16:22.162303+0000","last_unstale":"2023-10-08T00:16:22.162534+0000","last_undegraded":"2023-10-08T00:16:22.162534+0000","last_fullsized":"2023-10-08T00:16:22.162534+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:26:47.791012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-08T00:16:21.161043+0000","last_change":"2023-10-08T00:16:21.160185+0000","last_active":"2023-10-08T00:16:21.161043+0000","last_peered":"2023-10-08T00:16:21.161043+0000","last_clean":"2023-10-08T00:16:21.161043+0000","last_became_active":"2023-10-08T00:16:21.159932+0000","last_became_peered":"2023-10-08T00:16:21.159932+0000","last_unstale":"2023-10-08T00:16:21.161043+0000","last_undegraded":"2023-10-08T00:16:21.161043+0000","last_fullsized":"2023-10-08T00:16:21.161043+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:04:29.838739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.162528+0000","last_change":"2023-10-08T00:16:22.162528+0000","last_active":"2023-10-08T00:16:22.162528+0000","last_peered":"2023-10-08T00:16:22.162528+0000","last_clean":"2023-10-08T00:16:22.162528+0000","last_became_active":"2023-10-08T00:16:22.162320+0000","last_became_peered":"2023-10-08T00:16:22.162320+0000","last_unstale":"2023-10-08T00:16:22.162528+0000","last_undegraded":"2023-10-08T00:16:22.162528+0000","last_fullsized":"2023-10-08T00:16:22.162528+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:47:08.037802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-08T00:16:21.159766+0000","last_change":"2023-10-08T00:16:21.159661+0000","last_active":"2023-10-08T00:16:21.159766+0000","last_peered":"2023-10-08T00:16:21.159766+0000","last_clean":"2023-10-08T00:16:21.159766+0000","last_became_active":"2023-10-08T00:16:21.159125+0000","last_became_peered":"2023-10-08T00:16:21.159125+0000","last_unstale":"2023-10-08T00:16:21.159766+0000","last_undegraded":"2023-10-08T00:16:21.159766+0000","last_fullsized":"2023-10-08T00:16:21.159766+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:02:42.055051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.167493+0000","last_change":"2023-10-08T00:16:22.167493+0000","last_active":"2023-10-08T00:16:22.167493+0000","last_peered":"2023-10-08T00:16:22.167493+0000","last_clean":"2023-10-08T00:16:22.167493+0000","last_became_active":"2023-10-08T00:16:22.167268+0000","last_became_peered":"2023-10-08T00:16:22.167268+0000","last_unstale":"2023-10-08T00:16:22.167493+0000","last_undegraded":"2023-10-08T00:16:22.167493+0000","last_fullsized":"2023-10-08T00:16:22.167493+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:08:28.127337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"123'3","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.175313+0000","last_change":"2023-10-08T00:16:21.158687+0000","last_active":"2023-10-08T00:16:22.175313+0000","last_peered":"2023-10-08T00:16:22.175313+0000","last_clean":"2023-10-08T00:16:22.175313+0000","last_became_active":"2023-10-08T00:16:21.158476+0000","last_became_peered":"2023-10-08T00:16:21.158476+0000","last_unstale":"2023-10-08T00:16:22.175313+0000","last_undegraded":"2023-10-08T00:16:22.175313+0000","last_fullsized":"2023-10-08T00:16:22.175313+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:30:47.742876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.160017+0000","last_change":"2023-10-08T00:16:22.160017+0000","last_active":"2023-10-08T00:16:22.160017+0000","last_peered":"2023-10-08T00:16:22.160017+0000","last_clean":"2023-10-08T00:16:22.160017+0000","last_became_active":"2023-10-08T00:16:22.159727+0000","last_became_peered":"2023-10-08T00:16:22.159727+0000","last_unstale":"2023-10-08T00:16:22.160017+0000","last_undegraded":"2023-10-08T00:16:22.160017+0000","last_fullsized":"2023-10-08T00:16:22.160017+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:40:12.567676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"123'6","reported_seq":15,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.174977+0000","last_change":"2023-10-08T00:16:21.166994+0000","last_active":"2023-10-08T00:16:22.174977+0000","last_peered":"2023-10-08T00:16:22.174977+0000","last_clean":"2023-10-08T00:16:22.174977+0000","last_became_active":"2023-10-08T00:16:21.166843+0000","last_became_peered":"2023-10-08T00:16:21.166843+0000","last_unstale":"2023-10-08T00:16:22.174977+0000","last_undegraded":"2023-10-08T00:16:22.174977+0000","last_fullsized":"2023-10-08T00:16:22.174977+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:34:47.325200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.164023+0000","last_change":"2023-10-08T00:16:22.164023+0000","last_active":"2023-10-08T00:16:22.164023+0000","last_peered":"2023-10-08T00:16:22.164023+0000","last_clean":"2023-10-08T00:16:22.164023+0000","last_became_active":"2023-10-08T00:16:22.163472+0000","last_became_peered":"2023-10-08T00:16:22.163472+0000","last_unstale":"2023-10-08T00:16:22.164023+0000","last_undegraded":"2023-10-08T00:16:22.164023+0000","last_fullsized":"2023-10-08T00:16:22.164023+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:36:52.374149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"123'2","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.175101+0000","last_change":"2023-10-08T00:16:21.165052+0000","last_active":"2023-10-08T00:16:22.175101+0000","last_peered":"2023-10-08T00:16:22.175101+0000","last_clean":"2023-10-08T00:16:22.175101+0000","last_became_active":"2023-10-08T00:16:21.164426+0000","last_became_peered":"2023-10-08T00:16:21.164426+0000","last_unstale":"2023-10-08T00:16:22.175101+0000","last_undegraded":"2023-10-08T00:16:22.175101+0000","last_fullsized":"2023-10-08T00:16:22.175101+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:17:44.815549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.163946+0000","last_change":"2023-10-08T00:16:22.163946+0000","last_active":"2023-10-08T00:16:22.163946+0000","last_peered":"2023-10-08T00:16:22.163946+0000","last_clean":"2023-10-08T00:16:22.163946+0000","last_became_active":"2023-10-08T00:16:22.163402+0000","last_became_peered":"2023-10-08T00:16:22.163402+0000","last_unstale":"2023-10-08T00:16:22.163946+0000","last_undegraded":"2023-10-08T00:16:22.163946+0000","last_fullsized":"2023-10-08T00:16:22.163946+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:35:04.496031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-08T00:16:21.162860+0000","last_change":"2023-10-08T00:16:21.162715+0000","last_active":"2023-10-08T00:16:21.162860+0000","last_peered":"2023-10-08T00:16:21.162860+0000","last_clean":"2023-10-08T00:16:21.162860+0000","last_became_active":"2023-10-08T00:16:21.161125+0000","last_became_peered":"2023-10-08T00:16:21.161125+0000","last_unstale":"2023-10-08T00:16:21.162860+0000","last_undegraded":"2023-10-08T00:16:21.162860+0000","last_fullsized":"2023-10-08T00:16:21.162860+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:59:12.283523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.163296+0000","last_change":"2023-10-08T00:16:22.163296+0000","last_active":"2023-10-08T00:16:22.163296+0000","last_peered":"2023-10-08T00:16:22.163296+0000","last_clean":"2023-10-08T00:16:22.163296+0000","last_became_active":"2023-10-08T00:16:22.162982+0000","last_became_peered":"2023-10-08T00:16:22.162982+0000","last_unstale":"2023-10-08T00:16:22.163296+0000","last_undegraded":"2023-10-08T00:16:22.163296+0000","last_fullsized":"2023-10-08T00:16:22.163296+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:43:01.385831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-08T00:16:21.160253+0000","last_change":"2023-10-08T00:16:21.160061+0000","last_active":"2023-10-08T00:16:21.160253+0000","last_peered":"2023-10-08T00:16:21.160253+0000","last_clean":"2023-10-08T00:16:21.160253+0000","last_became_active":"2023-10-08T00:16:21.159709+0000","last_became_peered":"2023-10-08T00:16:21.159709+0000","last_unstale":"2023-10-08T00:16:21.160253+0000","last_undegraded":"2023-10-08T00:16:21.160253+0000","last_fullsized":"2023-10-08T00:16:21.160253+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:16:10.889545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.166743+0000","last_change":"2023-10-08T00:16:22.166743+0000","last_active":"2023-10-08T00:16:22.166743+0000","last_peered":"2023-10-08T00:16:22.166743+0000","last_clean":"2023-10-08T00:16:22.166743+0000","last_became_active":"2023-10-08T00:16:22.166462+0000","last_became_peered":"2023-10-08T00:16:22.166462+0000","last_unstale":"2023-10-08T00:16:22.166743+0000","last_undegraded":"2023-10-08T00:16:22.166743+0000","last_fullsized":"2023-10-08T00:16:22.166743+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:54:43.701059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"123'8","reported_seq":17,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.176492+0000","last_change":"2023-10-08T00:16:21.160781+0000","last_active":"2023-10-08T00:16:22.176492+0000","last_peered":"2023-10-08T00:16:22.176492+0000","last_clean":"2023-10-08T00:16:22.176492+0000","last_became_active":"2023-10-08T00:16:21.160010+0000","last_became_peered":"2023-10-08T00:16:21.160010+0000","last_unstale":"2023-10-08T00:16:22.176492+0000","last_undegraded":"2023-10-08T00:16:22.176492+0000","last_fullsized":"2023-10-08T00:16:22.176492+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:54:16.964828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.166017+0000","last_change":"2023-10-08T00:16:22.166017+0000","last_active":"2023-10-08T00:16:22.166017+0000","last_peered":"2023-10-08T00:16:22.166017+0000","last_clean":"2023-10-08T00:16:22.166017+0000","last_became_active":"2023-10-08T00:16:22.165772+0000","last_became_peered":"2023-10-08T00:16:22.165772+0000","last_unstale":"2023-10-08T00:16:22.166017+0000","last_undegraded":"2023-10-08T00:16:22.166017+0000","last_fullsized":"2023-10-08T00:16:22.166017+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:30:42.577974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"123'2","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.174868+0000","last_change":"2023-10-08T00:16:21.165739+0000","last_active":"2023-10-08T00:16:22.174868+0000","last_peered":"2023-10-08T00:16:22.174868+0000","last_clean":"2023-10-08T00:16:22.174868+0000","last_became_active":"2023-10-08T00:16:21.165596+0000","last_became_peered":"2023-10-08T00:16:21.165596+0000","last_unstale":"2023-10-08T00:16:22.174868+0000","last_undegraded":"2023-10-08T00:16:22.174868+0000","last_fullsized":"2023-10-08T00:16:22.174868+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:11:16.353357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.166357+0000","last_change":"2023-10-08T00:16:22.166357+0000","last_active":"2023-10-08T00:16:22.166357+0000","last_peered":"2023-10-08T00:16:22.166357+0000","last_clean":"2023-10-08T00:16:22.166357+0000","last_became_active":"2023-10-08T00:16:22.166141+0000","last_became_peered":"2023-10-08T00:16:22.166141+0000","last_unstale":"2023-10-08T00:16:22.166357+0000","last_undegraded":"2023-10-08T00:16:22.166357+0000","last_fullsized":"2023-10-08T00:16:22.166357+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:40:19.056813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"123'1","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.157943+0000","last_change":"2023-10-08T00:16:21.159455+0000","last_active":"2023-10-08T00:16:22.157943+0000","last_peered":"2023-10-08T00:16:22.157943+0000","last_clean":"2023-10-08T00:16:22.157943+0000","last_became_active":"2023-10-08T00:16:21.159198+0000","last_became_peered":"2023-10-08T00:16:21.159198+0000","last_unstale":"2023-10-08T00:16:22.157943+0000","last_undegraded":"2023-10-08T00:16:22.157943+0000","last_fullsized":"2023-10-08T00:16:22.157943+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:11:11.190696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.164268+0000","last_change":"2023-10-08T00:16:22.164268+0000","last_active":"2023-10-08T00:16:22.164268+0000","last_peered":"2023-10-08T00:16:22.164268+0000","last_clean":"2023-10-08T00:16:22.164268+0000","last_became_active":"2023-10-08T00:16:22.164025+0000","last_became_peered":"2023-10-08T00:16:22.164025+0000","last_unstale":"2023-10-08T00:16:22.164268+0000","last_undegraded":"2023-10-08T00:16:22.164268+0000","last_fullsized":"2023-10-08T00:16:22.164268+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:59:13.288375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-08T00:16:21.159608+0000","last_change":"2023-10-08T00:16:21.159488+0000","last_active":"2023-10-08T00:16:21.159608+0000","last_peered":"2023-10-08T00:16:21.159608+0000","last_clean":"2023-10-08T00:16:21.159608+0000","last_became_active":"2023-10-08T00:16:21.158888+0000","last_became_peered":"2023-10-08T00:16:21.158888+0000","last_unstale":"2023-10-08T00:16:21.159608+0000","last_undegraded":"2023-10-08T00:16:21.159608+0000","last_fullsized":"2023-10-08T00:16:21.159608+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:27:12.467258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.162930+0000","last_change":"2023-10-08T00:16:22.162930+0000","last_active":"2023-10-08T00:16:22.162930+0000","last_peered":"2023-10-08T00:16:22.162930+0000","last_clean":"2023-10-08T00:16:22.162930+0000","last_became_active":"2023-10-08T00:16:22.162721+0000","last_became_peered":"2023-10-08T00:16:22.162721+0000","last_unstale":"2023-10-08T00:16:22.162930+0000","last_undegraded":"2023-10-08T00:16:22.162930+0000","last_fullsized":"2023-10-08T00:16:22.162930+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:23:11.199659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"123'2","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.176794+0000","last_change":"2023-10-08T00:16:21.160930+0000","last_active":"2023-10-08T00:16:22.176794+0000","last_peered":"2023-10-08T00:16:22.176794+0000","last_clean":"2023-10-08T00:16:22.176794+0000","last_became_active":"2023-10-08T00:16:21.160562+0000","last_became_peered":"2023-10-08T00:16:21.160562+0000","last_unstale":"2023-10-08T00:16:22.176794+0000","last_undegraded":"2023-10-08T00:16:22.176794+0000","last_fullsized":"2023-10-08T00:16:22.176794+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:20:42.677235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.163843+0000","last_change":"2023-10-08T00:16:22.163843+0000","last_active":"2023-10-08T00:16:22.163843+0000","last_peered":"2023-10-08T00:16:22.163843+0000","last_clean":"2023-10-08T00:16:22.163843+0000","last_became_active":"2023-10-08T00:16:22.163619+0000","last_became_peered":"2023-10-08T00:16:22.163619+0000","last_unstale":"2023-10-08T00:16:22.163843+0000","last_undegraded":"2023-10-08T00:16:22.163843+0000","last_fullsized":"2023-10-08T00:16:22.163843+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:44:20.663579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-08T00:16:21.155458+0000","last_change":"2023-10-08T00:16:21.154822+0000","last_active":"2023-10-08T00:16:21.155458+0000","last_peered":"2023-10-08T00:16:21.155458+0000","last_clean":"2023-10-08T00:16:21.155458+0000","last_became_active":"2023-10-08T00:16:21.154614+0000","last_became_peered":"2023-10-08T00:16:21.154614+0000","last_unstale":"2023-10-08T00:16:21.155458+0000","last_undegraded":"2023-10-08T00:16:21.155458+0000","last_fullsized":"2023-10-08T00:16:21.155458+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:04:07.976115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.168462+0000","last_change":"2023-10-08T00:16:22.168462+0000","last_active":"2023-10-08T00:16:22.168462+0000","last_peered":"2023-10-08T00:16:22.168462+0000","last_clean":"2023-10-08T00:16:22.168462+0000","last_became_active":"2023-10-08T00:16:22.168230+0000","last_became_peered":"2023-10-08T00:16:22.168230+0000","last_unstale":"2023-10-08T00:16:22.168462+0000","last_undegraded":"2023-10-08T00:16:22.168462+0000","last_fullsized":"2023-10-08T00:16:22.168462+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:21:44.003712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-08T00:16:21.159223+0000","last_change":"2023-10-08T00:16:21.159058+0000","last_active":"2023-10-08T00:16:21.159223+0000","last_peered":"2023-10-08T00:16:21.159223+0000","last_clean":"2023-10-08T00:16:21.159223+0000","last_became_active":"2023-10-08T00:16:21.158618+0000","last_became_peered":"2023-10-08T00:16:21.158618+0000","last_unstale":"2023-10-08T00:16:21.159223+0000","last_undegraded":"2023-10-08T00:16:21.159223+0000","last_fullsized":"2023-10-08T00:16:21.159223+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:35:15.325157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.168156+0000","last_change":"2023-10-08T00:16:22.168156+0000","last_active":"2023-10-08T00:16:22.168156+0000","last_peered":"2023-10-08T00:16:22.168156+0000","last_clean":"2023-10-08T00:16:22.168156+0000","last_became_active":"2023-10-08T00:16:22.167982+0000","last_became_peered":"2023-10-08T00:16:22.167982+0000","last_unstale":"2023-10-08T00:16:22.168156+0000","last_undegraded":"2023-10-08T00:16:22.168156+0000","last_fullsized":"2023-10-08T00:16:22.168156+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:29:35.744364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-08T00:16:21.165127+0000","last_change":"2023-10-08T00:16:21.165066+0000","last_active":"2023-10-08T00:16:21.165127+0000","last_peered":"2023-10-08T00:16:21.165127+0000","last_clean":"2023-10-08T00:16:21.165127+0000","last_became_active":"2023-10-08T00:16:21.164371+0000","last_became_peered":"2023-10-08T00:16:21.164371+0000","last_unstale":"2023-10-08T00:16:21.165127+0000","last_undegraded":"2023-10-08T00:16:21.165127+0000","last_fullsized":"2023-10-08T00:16:21.165127+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:38:10.363947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.157037+0000","last_change":"2023-10-08T00:16:22.157037+0000","last_active":"2023-10-08T00:16:22.157037+0000","last_peered":"2023-10-08T00:16:22.157037+0000","last_clean":"2023-10-08T00:16:22.157037+0000","last_became_active":"2023-10-08T00:16:22.156823+0000","last_became_peered":"2023-10-08T00:16:22.156823+0000","last_unstale":"2023-10-08T00:16:22.157037+0000","last_undegraded":"2023-10-08T00:16:22.157037+0000","last_fullsized":"2023-10-08T00:16:22.157037+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:53:34.263422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"123'2","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.174197+0000","last_change":"2023-10-08T00:16:21.159923+0000","last_active":"2023-10-08T00:16:22.174197+0000","last_peered":"2023-10-08T00:16:22.174197+0000","last_clean":"2023-10-08T00:16:22.174197+0000","last_became_active":"2023-10-08T00:16:21.159425+0000","last_became_peered":"2023-10-08T00:16:21.159425+0000","last_unstale":"2023-10-08T00:16:22.174197+0000","last_undegraded":"2023-10-08T00:16:22.174197+0000","last_fullsized":"2023-10-08T00:16:22.174197+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:37:12.413185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.164592+0000","last_change":"2023-10-08T00:16:22.164592+0000","last_active":"2023-10-08T00:16:22.164592+0000","last_peered":"2023-10-08T00:16:22.164592+0000","last_clean":"2023-10-08T00:16:22.164592+0000","last_became_active":"2023-10-08T00:16:22.164252+0000","last_became_peered":"2023-10-08T00:16:22.164252+0000","last_unstale":"2023-10-08T00:16:22.164592+0000","last_undegraded":"2023-10-08T00:16:22.164592+0000","last_fullsized":"2023-10-08T00:16:22.164592+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:36:52.374149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-08T00:16:21.166957+0000","last_change":"2023-10-08T00:16:21.166879+0000","last_active":"2023-10-08T00:16:21.166957+0000","last_peered":"2023-10-08T00:16:21.166957+0000","last_clean":"2023-10-08T00:16:21.166957+0000","last_became_active":"2023-10-08T00:16:21.166638+0000","last_became_peered":"2023-10-08T00:16:21.166638+0000","last_unstale":"2023-10-08T00:16:21.166957+0000","last_undegraded":"2023-10-08T00:16:21.166957+0000","last_fullsized":"2023-10-08T00:16:21.166957+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:46:05.494965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.164094+0000","last_change":"2023-10-08T00:16:22.164094+0000","last_active":"2023-10-08T00:16:22.164094+0000","last_peered":"2023-10-08T00:16:22.164094+0000","last_clean":"2023-10-08T00:16:22.164094+0000","last_became_active":"2023-10-08T00:16:22.163717+0000","last_became_peered":"2023-10-08T00:16:22.163717+0000","last_unstale":"2023-10-08T00:16:22.164094+0000","last_undegraded":"2023-10-08T00:16:22.164094+0000","last_fullsized":"2023-10-08T00:16:22.164094+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:26:37.495930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-08T00:16:21.167140+0000","last_change":"2023-10-08T00:16:21.167100+0000","last_active":"2023-10-08T00:16:21.167140+0000","last_peered":"2023-10-08T00:16:21.167140+0000","last_clean":"2023-10-08T00:16:21.167140+0000","last_became_active":"2023-10-08T00:16:21.166283+0000","last_became_peered":"2023-10-08T00:16:21.166283+0000","last_unstale":"2023-10-08T00:16:21.167140+0000","last_undegraded":"2023-10-08T00:16:21.167140+0000","last_fullsized":"2023-10-08T00:16:21.167140+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:50:55.146686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.163628+0000","last_change":"2023-10-08T00:16:22.163628+0000","last_active":"2023-10-08T00:16:22.163628+0000","last_peered":"2023-10-08T00:16:22.163628+0000","last_clean":"2023-10-08T00:16:22.163628+0000","last_became_active":"2023-10-08T00:16:22.163402+0000","last_became_peered":"2023-10-08T00:16:22.163402+0000","last_unstale":"2023-10-08T00:16:22.163628+0000","last_undegraded":"2023-10-08T00:16:22.163628+0000","last_fullsized":"2023-10-08T00:16:22.163628+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:46.070540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-08T00:16:21.161596+0000","last_change":"2023-10-08T00:16:21.161536+0000","last_active":"2023-10-08T00:16:21.161596+0000","last_peered":"2023-10-08T00:16:21.161596+0000","last_clean":"2023-10-08T00:16:21.161596+0000","last_became_active":"2023-10-08T00:16:21.161356+0000","last_became_peered":"2023-10-08T00:16:21.161356+0000","last_unstale":"2023-10-08T00:16:21.161596+0000","last_undegraded":"2023-10-08T00:16:21.161596+0000","last_fullsized":"2023-10-08T00:16:21.161596+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:05:54.513612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.158829+0000","last_change":"2023-10-08T00:16:22.158829+0000","last_active":"2023-10-08T00:16:22.158829+0000","last_peered":"2023-10-08T00:16:22.158829+0000","last_clean":"2023-10-08T00:16:22.158829+0000","last_became_active":"2023-10-08T00:16:22.157910+0000","last_became_peered":"2023-10-08T00:16:22.157910+0000","last_unstale":"2023-10-08T00:16:22.158829+0000","last_undegraded":"2023-10-08T00:16:22.158829+0000","last_fullsized":"2023-10-08T00:16:22.158829+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:50:42.638829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-08T00:16:21.165337+0000","last_change":"2023-10-08T00:16:21.165275+0000","last_active":"2023-10-08T00:16:21.165337+0000","last_peered":"2023-10-08T00:16:21.165337+0000","last_clean":"2023-10-08T00:16:21.165337+0000","last_became_active":"2023-10-08T00:16:21.164768+0000","last_became_peered":"2023-10-08T00:16:21.164768+0000","last_unstale":"2023-10-08T00:16:21.165337+0000","last_undegraded":"2023-10-08T00:16:21.165337+0000","last_fullsized":"2023-10-08T00:16:21.165337+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:11:30.488955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.164327+0000","last_change":"2023-10-08T00:16:22.164327+0000","last_active":"2023-10-08T00:16:22.164327+0000","last_peered":"2023-10-08T00:16:22.164327+0000","last_clean":"2023-10-08T00:16:22.164327+0000","last_became_active":"2023-10-08T00:16:22.163648+0000","last_became_peered":"2023-10-08T00:16:22.163648+0000","last_unstale":"2023-10-08T00:16:22.164327+0000","last_undegraded":"2023-10-08T00:16:22.164327+0000","last_fullsized":"2023-10-08T00:16:22.164327+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:19:31.079138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-08T00:16:21.162693+0000","last_change":"2023-10-08T00:16:21.162634+0000","last_active":"2023-10-08T00:16:21.162693+0000","last_peered":"2023-10-08T00:16:21.162693+0000","last_clean":"2023-10-08T00:16:21.162693+0000","last_became_active":"2023-10-08T00:16:21.162455+0000","last_became_peered":"2023-10-08T00:16:21.162455+0000","last_unstale":"2023-10-08T00:16:21.162693+0000","last_undegraded":"2023-10-08T00:16:21.162693+0000","last_fullsized":"2023-10-08T00:16:21.162693+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:46:05.494965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.158995+0000","last_change":"2023-10-08T00:16:22.158995+0000","last_active":"2023-10-08T00:16:22.158995+0000","last_peered":"2023-10-08T00:16:22.158995+0000","last_clean":"2023-10-08T00:16:22.158995+0000","last_became_active":"2023-10-08T00:16:22.158755+0000","last_became_peered":"2023-10-08T00:16:22.158755+0000","last_unstale":"2023-10-08T00:16:22.158995+0000","last_undegraded":"2023-10-08T00:16:22.158995+0000","last_fullsized":"2023-10-08T00:16:22.158995+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:29:35.744364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-08T00:16:21.161822+0000","last_change":"2023-10-08T00:16:21.161744+0000","last_active":"2023-10-08T00:16:21.161822+0000","last_peered":"2023-10-08T00:16:21.161822+0000","last_clean":"2023-10-08T00:16:21.161822+0000","last_became_active":"2023-10-08T00:16:21.161514+0000","last_became_peered":"2023-10-08T00:16:21.161514+0000","last_unstale":"2023-10-08T00:16:21.161822+0000","last_undegraded":"2023-10-08T00:16:21.161822+0000","last_fullsized":"2023-10-08T00:16:21.161822+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:59:12.063065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.166013+0000","last_change":"2023-10-08T00:16:22.166013+0000","last_active":"2023-10-08T00:16:22.166013+0000","last_peered":"2023-10-08T00:16:22.166013+0000","last_clean":"2023-10-08T00:16:22.166013+0000","last_became_active":"2023-10-08T00:16:22.165795+0000","last_became_peered":"2023-10-08T00:16:22.165795+0000","last_unstale":"2023-10-08T00:16:22.166013+0000","last_undegraded":"2023-10-08T00:16:22.166013+0000","last_fullsized":"2023-10-08T00:16:22.166013+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:54:17.969681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-08T00:16:21.168074+0000","last_change":"2023-10-08T00:16:21.167985+0000","last_active":"2023-10-08T00:16:21.168074+0000","last_peered":"2023-10-08T00:16:21.168074+0000","last_clean":"2023-10-08T00:16:21.168074+0000","last_became_active":"2023-10-08T00:16:21.167755+0000","last_became_peered":"2023-10-08T00:16:21.167755+0000","last_unstale":"2023-10-08T00:16:21.168074+0000","last_undegraded":"2023-10-08T00:16:21.168074+0000","last_fullsized":"2023-10-08T00:16:21.168074+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:37:12.413185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":8,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.167814+0000","last_change":"2023-10-08T00:16:22.167814+0000","last_active":"2023-10-08T00:16:22.167814+0000","last_peered":"2023-10-08T00:16:22.167814+0000","last_clean":"2023-10-08T00:16:22.167814+0000","last_became_active":"2023-10-08T00:16:22.167655+0000","last_became_peered":"2023-10-08T00:16:22.167655+0000","last_unstale":"2023-10-08T00:16:22.167814+0000","last_undegraded":"2023-10-08T00:16:22.167814+0000","last_fullsized":"2023-10-08T00:16:22.167814+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:21.145509+0000","last_clean_scrub_stamp":"2023-10-08T00:16:21.145509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:25:07.879702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"123'1","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-08T00:16:22.164456+0000","last_change":"2023-10-08T00:16:21.166400+0000","last_active":"2023-10-08T00:16:22.164456+0000","last_peered":"2023-10-08T00:16:22.164456+0000","last_clean":"2023-10-08T00:16:22.164456+0000","last_became_active":"2023-10-08T00:16:21.166279+0000","last_became_peered":"2023-10-08T00:16:21.166279+0000","last_unstale":"2023-10-08T00:16:22.164456+0000","last_undegraded":"2023-10-08T00:16:22.164456+0000","last_fullsized":"2023-10-08T00:16:22.164456+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:20.140657+0000","last_clean_scrub_stamp":"2023-10-08T00:16:20.140657+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:02:23.869290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":250,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-08T00:16:21.157581+0000","last_change":"2023-10-08T00:10:43.728490+0000","last_active":"2023-10-08T00:16:21.157581+0000","last_peered":"2023-10-08T00:16:21.157581+0000","last_clean":"2023-10-08T00:16:21.157581+0000","last_became_active":"2023-10-08T00:10:43.728341+0000","last_became_peered":"2023-10-08T00:10:43.728341+0000","last_unstale":"2023-10-08T00:16:21.157581+0000","last_undegraded":"2023-10-08T00:16:21.157581+0000","last_fullsized":"2023-10-08T00:16:21.157581+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_clean_scrub_stamp":"2023-10-08T00:10:42.716096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:06:31.035293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574918,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":662,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194649,"statfs":{"total":200000000,"available":199321599,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.53300000000000003,"15min":0.50700000000000001},"min":{"1min":0.39700000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.54300000000000004,"15min":0.54500000000000004},"min":{"1min":0.36099999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.52300000000000002}]},{"osd":1,"last update":"Sun Oct 8 00:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56100000000000005,"15min":0.53700000000000003},"min":{"1min":0.38,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.72799999999999998,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.58699999999999997,"15min":0.56699999999999995},"min":{"1min":0.42399999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.78400000000000003,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.47799999999999998}]},{"osd":2,"last update":"Sun Oct 8 00:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55100000000000005,"15min":0.54000000000000004},"min":{"1min":0.34599999999999997,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.501},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.55200000000000005,"15min":0.52600000000000002},"min":{"1min":0.433,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77200000000000002,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.46800000000000003}]},{"osd":3,"last update":"Sun Oct 8 00:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54500000000000004,"15min":0.52300000000000002},"min":{"1min":0.35599999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.75600000000000001,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56899999999999995,"15min":0.55900000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.371}]},{"osd":4,"last update":"Sun Oct 8 00:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.497,"15min":0.51700000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.65100000000000002,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.5,"5min":0.504,"15min":0.52400000000000002},"min":{"1min":0.315,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.75900000000000001,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.442}]},{"osd":5,"last update":"Sun Oct 8 00:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.51500000000000001,"15min":0.54500000000000004},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.78000000000000003,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54300000000000004,"15min":0.57999999999999996},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.76900000000000002,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.46000000000000002}]},{"osd":6,"last update":"Sun Oct 8 00:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.57799999999999996,"15min":0.59399999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.74399999999999999,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58899999999999997,"15min":0.625},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.80100000000000005,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.39100000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574918,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":661,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194650,"statfs":{"total":200000000,"available":199322153,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.51500000000000001,"15min":0.48699999999999999},"min":{"1min":0.39400000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.71199999999999997,"5min":1.012,"15min":1.012},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.56999999999999995,"15min":0.66700000000000004},"min":{"1min":0.39900000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.0529999999999999,"5min":5.1230000000000002,"15min":5.1230000000000002},"last":0.61399999999999999}]},{"osd":1,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55200000000000005,"15min":0.52400000000000002},"min":{"1min":0.34799999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.78100000000000003,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57599999999999996,"15min":0.56599999999999995},"min":{"1min":0.42499999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.98999999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.56100000000000005}]},{"osd":2,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.54700000000000004,"15min":0.53200000000000003},"min":{"1min":0.439,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.753,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.39700000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.76900000000000002,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.58899999999999997}]},{"osd":3,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57299999999999995,"15min":0.54300000000000004},"min":{"1min":0.36699999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.84199999999999997,"5min":1.046,"15min":1.046},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56899999999999995,"15min":0.54500000000000004},"min":{"1min":0.39800000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.81000000000000005,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.60299999999999998}]},{"osd":4,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.47299999999999998,"15min":0.496},"min":{"1min":0.371,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.435},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.55200000000000005,"15min":0.56299999999999994},"min":{"1min":0.38100000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.36099999999999999}]},{"osd":5,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.48299999999999998,"15min":0.48699999999999999},"min":{"1min":0.34499999999999997,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.69499999999999995,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.52300000000000002,"15min":0.55200000000000005},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.39200000000000002}]},{"osd":7,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.52700000000000002,"15min":0.54000000000000004},"min":{"1min":0.31,"5min":0.307,"15min":0.307},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.54500000000000004,"15min":0.54800000000000004},"min":{"1min":0.40200000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76000000000000001,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.52000000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1240,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194072,"statfs":{"total":200000000,"available":198730137,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:15:49 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.33300000000000002,"15min":0.309},"min":{"1min":0.20300000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.51600000000000001,"5min":0.56899999999999995,"15min":0.56899999999999995},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.39300000000000002,"15min":0.379},"min":{"1min":0.27300000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.35799999999999998}]},{"osd":2,"last update":"Sun Oct 8 00:15:49 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.34699999999999998,"15min":0.33600000000000002},"min":{"1min":0.26300000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.58999999999999997,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.376,"15min":0.34399999999999997},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.58999999999999997,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.317}]},{"osd":3,"last update":"Sun Oct 8 00:15:49 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.39000000000000001,"15min":0.372},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.35299999999999998,"15min":0.34399999999999997},"min":{"1min":0.28299999999999997,"5min":0.222,"15min":0.222},"max":{"1min":0.53600000000000003,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.36599999999999999}]},{"osd":4,"last update":"Sun Oct 8 00:15:49 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55900000000000005,"15min":0.54100000000000004},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58099999999999996,"15min":0.56899999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.42899999999999999}]},{"osd":5,"last update":"Sun Oct 8 00:15:49 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56200000000000006,"15min":0.53900000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.83799999999999997,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.58999999999999997,"15min":0.57699999999999996},"min":{"1min":0.38300000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.46400000000000002}]},{"osd":6,"last update":"Sun Oct 8 00:15:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58499999999999996,"15min":0.56899999999999995},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.57999999999999996,"15min":0.56399999999999995},"min":{"1min":0.39600000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77000000000000002,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.47399999999999998}]},{"osd":7,"last update":"Sun Oct 8 00:15:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.58299999999999996,"15min":0.56599999999999995},"min":{"1min":0.39500000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.61599999999999999,"15min":0.59399999999999997},"min":{"1min":0.40500000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.89500000000000002,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.439}]}]},{"osd":0,"up_from":13,"seq":55834574918,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":661,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194650,"statfs":{"total":200000000,"available":199322153,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 00:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.34100000000000003,"15min":0.33200000000000002},"min":{"1min":0.20699999999999999,"5min":0.193,"15min":0.193},"max":{"1min":0.57599999999999996,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.35599999999999998,"15min":0.34799999999999998},"min":{"1min":0.218,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.57199999999999995,"5min":0.87,"15min":0.87},"last":0.377}]},{"osd":2,"last update":"Sun Oct 8 00:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.38900000000000001,"15min":0.38800000000000001},"min":{"1min":0.27600000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.623,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.379,"5min":0.46400000000000002,"15min":0.72799999999999998},"min":{"1min":0.245,"5min":0.185,"15min":0.185},"max":{"1min":0.65600000000000003,"5min":12.019,"15min":12.019},"last":0.30299999999999999}]},{"osd":3,"last update":"Sun Oct 8 00:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.36799999999999999,"15min":0.36399999999999999},"min":{"1min":0.27500000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.54300000000000004,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.318},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.41399999999999998,"15min":0.39900000000000002},"min":{"1min":0.29999999999999999,"5min":0.246,"15min":0.246},"max":{"1min":0.67400000000000004,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.41199999999999998}]},{"osd":4,"last update":"Sun Oct 8 00:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61399999999999999,"15min":0.61699999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.90300000000000002,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.496},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.626,"15min":0.626},"min":{"1min":0.439,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.873,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.59899999999999998}]},{"osd":5,"last update":"Sun Oct 8 00:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.69099999999999995,"15min":0.96699999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.751,"5min":11.992000000000001,"15min":11.992000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60099999999999998,"15min":0.58899999999999997},"min":{"1min":0.46800000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.85599999999999998,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.626}]},{"osd":6,"last update":"Sun Oct 8 00:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.59899999999999998,"15min":0.58899999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.88100000000000001,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.69599999999999995,"15min":0.94799999999999995},"min":{"1min":0.41299999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84499999999999997,"5min":11.927,"15min":11.927},"last":0.61199999999999999}]},{"osd":7,"last update":"Sun Oct 8 00:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.70799999999999996,"15min":0.96899999999999997},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.84099999999999997,"5min":11.965,"15min":11.965},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.626,"15min":0.624},"min":{"1min":0.42699999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81100000000000005,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.64900000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574918,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":662,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194650,"statfs":{"total":200000000,"available":199322083,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.35299999999999998,"15min":0.38900000000000001},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.45600000000000002,"5min":2.5139999999999998,"15min":2.5139999999999998},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.35799999999999998,"15min":0.40000000000000002},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.53800000000000003,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":0.32600000000000001}]},{"osd":1,"last update":"Sun Oct 8 00:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.35999999999999999,"15min":0.39200000000000002},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.65300000000000002,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.39000000000000001,"15min":0.434},"min":{"1min":0.254,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.58499999999999996,"5min":2.5379999999999998,"15min":2.5379999999999998},"last":0.38800000000000001}]},{"osd":3,"last update":"Sun Oct 8 00:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.39800000000000002,"15min":0.42899999999999999},"min":{"1min":0.27400000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.60599999999999998,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.41799999999999998,"15min":0.46200000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.63,"5min":2.5619999999999998,"15min":2.5619999999999998},"last":0.45600000000000002}]},{"osd":4,"last update":"Sun Oct 8 00:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.57899999999999996,"15min":0.61499999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.81499999999999995,"5min":2.488,"15min":2.488},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59299999999999997,"15min":0.60699999999999998},"min":{"1min":0.41799999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.81000000000000005,"5min":2.073,"15min":2.073},"last":0.68500000000000005}]},{"osd":5,"last update":"Sun Oct 8 00:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.625,"15min":0.65700000000000003},"min":{"1min":0.36899999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83099999999999996,"5min":2.2069999999999999,"15min":2.2069999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.621,"5min":0.64200000000000002,"15min":0.66300000000000003},"min":{"1min":0.41599999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83799999999999997,"5min":2.456,"15min":2.456},"last":0.65300000000000002}]},{"osd":6,"last update":"Sun Oct 8 00:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.60599999999999998,"15min":0.622},"min":{"1min":0.40899999999999997,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.80000000000000004,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.60899999999999999,"15min":0.61399999999999999},"min":{"1min":0.439,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.92200000000000004,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.59099999999999997}]},{"osd":7,"last update":"Sun Oct 8 00:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.621,"15min":0.63600000000000001},"min":{"1min":0.45700000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.75700000000000001,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61499999999999999,"15min":0.629},"min":{"1min":0.45000000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85999999999999999,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":0.73499999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574918,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":662,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194650,"statfs":{"total":200000000,"available":199321959,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.33300000000000002,"15min":0.36899999999999999},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":0.71199999999999997,"5min":1.46,"15min":1.46},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.33700000000000002,"15min":0.34200000000000003},"min":{"1min":0.20399999999999999,"5min":0.192,"15min":0.192},"max":{"1min":0.504,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.40799999999999997}]},{"osd":1,"last update":"Sun Oct 8 00:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.46899999999999997,"15min":0.79700000000000004},"min":{"1min":0.22700000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.55200000000000005,"5min":12.351000000000001,"15min":12.351000000000001},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.375,"5min":0.36699999999999999,"15min":0.36599999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.60199999999999998,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.54200000000000004}]},{"osd":2,"last update":"Sun Oct 8 00:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.39500000000000002,"15min":0.41899999999999998},"min":{"1min":0.27700000000000002,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.60899999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.39300000000000002,"15min":0.38800000000000001},"min":{"1min":0.28699999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.64700000000000002,"5min":0.753,"15min":0.753},"last":0.442}]},{"osd":4,"last update":"Sun Oct 8 00:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.70799999999999996,"15min":0.98699999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.024,"5min":11.333,"15min":11.333},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.72699999999999998,"15min":1.0169999999999999},"min":{"1min":0.43099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84999999999999998,"5min":12.321999999999999,"15min":12.321999999999999},"last":0.64500000000000002}]},{"osd":5,"last update":"Sun Oct 8 00:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.72099999999999997,"15min":1.0269999999999999},"min":{"1min":0.38300000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91000000000000003,"5min":12.291,"15min":12.291},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.61799999999999999,"15min":0.60399999999999998},"min":{"1min":0.40999999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.73199999999999998}]},{"osd":6,"last update":"Sun Oct 8 00:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.68200000000000005,"15min":0.95199999999999996},"min":{"1min":0.35299999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.88,"5min":11.288,"15min":11.288},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.71199999999999997,"15min":1.01},"min":{"1min":0.40600000000000003,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.002,"5min":12.253,"15min":12.253},"last":0.71899999999999997}]},{"osd":7,"last update":"Sun Oct 8 00:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.60099999999999998,"15min":0.57899999999999996},"min":{"1min":0.39700000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80300000000000005,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.72199999999999998,"15min":0.98999999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0549999999999999,"5min":11.363,"15min":11.363},"last":0.74399999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574918,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":662,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194649,"statfs":{"total":200000000,"available":199321599,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.56899999999999995,"15min":0.55700000000000005},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.78100000000000003,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57399999999999995,"15min":0.58399999999999996},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.72899999999999998,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.49299999999999999}]},{"osd":1,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.52500000000000002,"15min":0.52100000000000002},"min":{"1min":0.33900000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.53100000000000003,"15min":0.53400000000000003},"min":{"1min":0.314,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.80400000000000005,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.50600000000000001}]},{"osd":2,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.51500000000000001,"15min":0.51900000000000002},"min":{"1min":0.41699999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.68700000000000006,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.52400000000000002,"15min":0.51300000000000001},"min":{"1min":0.374,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.433}]},{"osd":3,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.53700000000000003,"15min":0.54000000000000004},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.79800000000000004,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.53500000000000003,"15min":0.53400000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.47199999999999998}]},{"osd":5,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.46600000000000003,"15min":0.49399999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.317},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.495,"15min":0.51000000000000001},"min":{"1min":0.26400000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.41299999999999998}]},{"osd":6,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.47199999999999998,"15min":0.48799999999999999},"min":{"1min":0.32200000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.65000000000000002,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52200000000000002,"15min":0.55200000000000005},"min":{"1min":0.35399999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.52600000000000002}]},{"osd":7,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.53400000000000003,"15min":0.55100000000000005},"min":{"1min":0.35399999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.79700000000000004,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.56899999999999995,"15min":0.57999999999999996},"min":{"1min":0.45500000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.77500000000000002,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.54900000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574918,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1240,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194072,"statfs":{"total":200000000,"available":198730193,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48599999999999999},"min":{"1min":0.28799999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.66700000000000004,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53800000000000003,"15min":0.55700000000000005},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.70999999999999996,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.55200000000000005}]},{"osd":1,"last update":"Sun Oct 8 00:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.39000000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.71799999999999997,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50700000000000001,"15min":0.52300000000000002},"min":{"1min":0.33100000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70699999999999996,"5min":0.755,"15min":0.755},"last":0.56599999999999995}]},{"osd":2,"last update":"Sun Oct 8 00:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.52300000000000002,"15min":0.52900000000000003},"min":{"1min":0.39500000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.64700000000000002,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.55000000000000004,"15min":0.56699999999999995},"min":{"1min":0.33700000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.68799999999999994,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.42299999999999999}]},{"osd":3,"last update":"Sun Oct 8 00:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.55100000000000005,"15min":0.54700000000000004},"min":{"1min":0.35499999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.73199999999999998,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.436},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.55800000000000005,"15min":0.57599999999999996},"min":{"1min":0.38,"5min":0.371,"15min":0.371},"max":{"1min":0.71999999999999997,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.58099999999999996}]},{"osd":4,"last update":"Sun Oct 8 00:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.46999999999999997,"15min":0.47399999999999998},"min":{"1min":0.312,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.69399999999999995,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.51300000000000001,"15min":0.51500000000000001},"min":{"1min":0.40000000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.68300000000000005,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.50800000000000001}]},{"osd":6,"last update":"Sun Oct 8 00:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.505,"15min":0.501},"min":{"1min":0.36299999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.63200000000000001,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.505,"15min":0.496},"min":{"1min":0.36199999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68300000000000005,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.52400000000000002}]},{"osd":7,"last update":"Sun Oct 8 00:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52600000000000002,"15min":0.51900000000000002},"min":{"1min":0.379,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.66600000000000004,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55600000000000005,"15min":0.55700000000000005},"min":{"1min":0.34499999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.69599999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.47799999999999998}]}]}],"pool_statfs":[]}} 2023-10-08T00:16:26.594 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T00:16:26.594 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-08T00:16:26.748 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.744+0000 7f35b7075700 1 Processor -- start 2023-10-08T00:16:26.748 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.744+0000 7f35b7075700 1 -- start start 2023-10-08T00:16:26.748 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.745+0000 7f35b7075700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f35b015a290 0x7f35b006a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:26.748 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.745+0000 7f35b7075700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f35b006aae0 0x7f35b00cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:26.748 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.745+0000 7f35b7075700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35b00b0790 con 0x7f35b0159ec0 2023-10-08T00:16:26.748 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.745+0000 7f35b7075700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35b006af60 con 0x7f35b015a290 2023-10-08T00:16:26.748 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.745+0000 7f35b7075700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35b00ce1f0 con 0x7f35b006aae0 2023-10-08T00:16:26.748 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.745+0000 7f35a7fff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f35b015a290 0x7f35b006a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:26.749 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.745+0000 7f35b5612700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f35b006aae0 0x7f35b00cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:26.749 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.746+0000 7f35b5612700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f35b006aae0 0x7f35b00cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.99:50526/0 (socket says 172.21.15.99:50526) 2023-10-08T00:16:26.749 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.746+0000 7f35a7fff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f35b015a290 0x7f35b006a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.99:37426/0 (socket says 172.21.15.99:37426) 2023-10-08T00:16:26.749 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.746+0000 7f35b5612700 1 -- 172.21.15.99:0/4004407901 learned_addr learned my addr 172.21.15.99:0/4004407901 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:26.749 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.746+0000 7f35b4e11700 1 --1- 172.21.15.99:0/4004407901 >> v1:172.21.15.191:6789/0 conn(0x7f35b0159ec0 0x7f35b00af940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:43512/0 (socket says 172.21.15.99:43512) 2023-10-08T00:16:26.749 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.746+0000 7f35a7fff700 1 -- 172.21.15.99:0/4004407901 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f35b006aae0 msgr2=0x7f35b00cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:26.749 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.746+0000 7f35a7fff700 1 --2- 172.21.15.99:0/4004407901 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f35b006aae0 0x7f35b00cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:26.749 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.746+0000 7f35a7fff700 1 -- 172.21.15.99:0/4004407901 >> v1:172.21.15.191:6789/0 conn(0x7f35b0159ec0 legacy=0x7f35b00af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:26.749 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.746+0000 7f35a77fe700 1 -- 172.21.15.99:0/4004407901 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3411124975 0 0) 0x7f35b00b0790 con 0x7f35b0159ec0 2023-10-08T00:16:26.750 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.746+0000 7f35a7fff700 1 -- 172.21.15.99:0/4004407901 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b00b0b20 con 0x7f35b015a290 2023-10-08T00:16:26.750 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.746+0000 7f35b5612700 1 --2- 172.21.15.99:0/4004407901 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f35b006aae0 0x7f35b00cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:16:26.750 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.746+0000 7f35a7fff700 1 --2- 172.21.15.99:0/4004407901 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35b015a290 0x7f35b006a5a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f35a00098e0 tx=0x7f35a00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a9c9a4a543b232c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:26.750 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.746+0000 7f35a77fe700 1 -- 172.21.15.99:0/4004407901 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a000b640 con 0x7f35b015a290 2023-10-08T00:16:26.750 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.746+0000 7f35a77fe700 1 -- 172.21.15.99:0/4004407901 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a000b7a0 con 0x7f35b015a290 2023-10-08T00:16:26.750 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.746+0000 7f35a77fe700 1 -- 172.21.15.99:0/4004407901 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a000bb40 con 0x7f35b015a290 2023-10-08T00:16:26.750 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.747+0000 7f35b7075700 1 -- 172.21.15.99:0/4004407901 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35b015a290 msgr2=0x7f35b006a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:26.750 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.747+0000 7f35b7075700 1 --2- 172.21.15.99:0/4004407901 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35b015a290 0x7f35b006a5a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f35a00098e0 tx=0x7f35a00091a0 comp rx=0 tx=0).stop 2023-10-08T00:16:26.750 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.747+0000 7f35b7075700 1 -- 172.21.15.99:0/4004407901 shutdown_connections 2023-10-08T00:16:26.750 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.747+0000 7f35b7075700 1 --2- 172.21.15.99:0/4004407901 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f35b006aae0 0x7f35b00cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:26.750 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.747+0000 7f35b7075700 1 --2- 172.21.15.99:0/4004407901 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35b015a290 0x7f35b006a5a0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:26.751 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.747+0000 7f35b7075700 1 -- 172.21.15.99:0/4004407901 >> 172.21.15.99:0/4004407901 conn(0x7f35b00d1880 msgr2=0x7f35b00cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:26.751 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.747+0000 7f35b7075700 1 -- 172.21.15.99:0/4004407901 shutdown_connections 2023-10-08T00:16:26.751 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.747+0000 7f35b7075700 1 -- 172.21.15.99:0/4004407901 wait complete. 2023-10-08T00:16:26.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.748+0000 7f35b7075700 1 Processor -- start 2023-10-08T00:16:26.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.748+0000 7f35b7075700 1 -- start start 2023-10-08T00:16:26.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.749+0000 7f35b7075700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f35b006aae0 0x7f35b00c8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:26.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.749+0000 7f35b7075700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35b0159ec0 0x7f35b016d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:26.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.749+0000 7f35b7075700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35b015a290 0x7f35b015b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:26.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.749+0000 7f35b7075700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b00cac40 con 0x7f35b015a290 2023-10-08T00:16:26.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.749+0000 7f35b7075700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b015b590 con 0x7f35b0159ec0 2023-10-08T00:16:26.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.749+0000 7f35b7075700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35b00ca310 con 0x7f35b006aae0 2023-10-08T00:16:26.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.749+0000 7f35b5612700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35b015a290 0x7f35b015b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:26.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.749+0000 7f35b5612700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35b015a290 0x7f35b015b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37080/0 (socket says 172.21.15.99:37080) 2023-10-08T00:16:26.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.749+0000 7f35b5612700 1 -- 172.21.15.99:0/2604941549 learned_addr learned my addr 172.21.15.99:0/2604941549 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:26.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.749+0000 7f35b4e11700 1 --2- 172.21.15.99:0/2604941549 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f35b006aae0 0x7f35b00c8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:26.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.749+0000 7f35a7fff700 1 --2- 172.21.15.99:0/2604941549 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35b0159ec0 0x7f35b016d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:26.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.749+0000 7f35b5612700 1 -- 172.21.15.99:0/2604941549 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f35b006aae0 msgr2=0x7f35b00c8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:26.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.749+0000 7f35b5612700 1 --2- 172.21.15.99:0/2604941549 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f35b006aae0 0x7f35b00c8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:26.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.749+0000 7f35b5612700 1 -- 172.21.15.99:0/2604941549 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35b0159ec0 msgr2=0x7f35b016d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:26.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.749+0000 7f35b5612700 1 --2- 172.21.15.99:0/2604941549 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35b0159ec0 0x7f35b016d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:26.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.749+0000 7f35b5612700 1 -- 172.21.15.99:0/2604941549 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a8009ae0 con 0x7f35b015a290 2023-10-08T00:16:26.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.750+0000 7f35b5612700 1 --2- 172.21.15.99:0/2604941549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35b015a290 0x7f35b015b050 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f35a8009fd0 tx=0x7f35a800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:26.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.750+0000 7f35a4ff9700 1 -- 172.21.15.99:0/2604941549 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a8018070 con 0x7f35b015a290 2023-10-08T00:16:26.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.750+0000 7f35a4ff9700 1 -- 172.21.15.99:0/2604941549 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a800f040 con 0x7f35b015a290 2023-10-08T00:16:26.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.750+0000 7f35b7075700 1 -- 172.21.15.99:0/2604941549 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35a0009420 con 0x7f35b015a290 2023-10-08T00:16:26.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.750+0000 7f35a4ff9700 1 -- 172.21.15.99:0/2604941549 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a800cbd0 con 0x7f35b015a290 2023-10-08T00:16:26.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.750+0000 7f35b7075700 1 -- 172.21.15.99:0/2604941549 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35b00ca8d0 con 0x7f35b015a290 2023-10-08T00:16:26.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.751+0000 7f35b7075700 1 -- 172.21.15.99:0/2604941549 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35b00cadb0 con 0x7f35b015a290 2023-10-08T00:16:26.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.751+0000 7f35a4ff9700 1 -- 172.21.15.99:0/2604941549 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f35a8003020 con 0x7f35b015a290 2023-10-08T00:16:26.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.751+0000 7f35a4ff9700 1 --2- 172.21.15.99:0/2604941549 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f359c066f30 0x7f359c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:26.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.752+0000 7f35a4ff9700 1 -- 172.21.15.99:0/2604941549 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f35a8084420 con 0x7f35b015a290 2023-10-08T00:16:26.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.752+0000 7f35b4e11700 1 --2- 172.21.15.99:0/2604941549 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f359c066f30 0x7f359c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:26.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.752+0000 7f35b4e11700 1 --2- 172.21.15.99:0/2604941549 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f359c066f30 0x7f359c0693e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f35ac007fd0 tx=0x7f35ac007d50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:26.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:26.756+0000 7f35a4ff9700 1 -- 172.21.15.99:0/2604941549 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f35a804d5a0 con 0x7f35b015a290 2023-10-08T00:16:27.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.011+0000 7f35b7075700 1 -- 172.21.15.99:0/2604941549 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f35b004c3c0 con 0x7f35b015a290 2023-10-08T00:16:27.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.014+0000 7f35a4ff9700 1 -- 172.21.15.99:0/2604941549 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+0 (secure 0 0 0) 0x7f35a8055120 con 0x7f35b015a290 2023-10-08T00:16:27.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.017+0000 7f35b7075700 1 -- 172.21.15.99:0/2604941549 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f359c066f30 msgr2=0x7f359c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:27.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.017+0000 7f35b7075700 1 --2- 172.21.15.99:0/2604941549 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f359c066f30 0x7f359c0693e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f35ac007fd0 tx=0x7f35ac007d50 comp rx=0 tx=0).stop 2023-10-08T00:16:27.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.017+0000 7f35b7075700 1 -- 172.21.15.99:0/2604941549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35b015a290 msgr2=0x7f35b015b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:27.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.017+0000 7f35b7075700 1 --2- 172.21.15.99:0/2604941549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35b015a290 0x7f35b015b050 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f35a8009fd0 tx=0x7f35a800b040 comp rx=0 tx=0).stop 2023-10-08T00:16:27.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.017+0000 7f35b7075700 1 -- 172.21.15.99:0/2604941549 shutdown_connections 2023-10-08T00:16:27.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.017+0000 7f35b7075700 1 --2- 172.21.15.99:0/2604941549 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f359c066f30 0x7f359c0693e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:27.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.017+0000 7f35b7075700 1 --2- 172.21.15.99:0/2604941549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35b015a290 0x7f35b015b050 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:27.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.017+0000 7f35b7075700 1 --2- 172.21.15.99:0/2604941549 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f35b006aae0 0x7f35b00c8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:27.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.017+0000 7f35b7075700 1 --2- 172.21.15.99:0/2604941549 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35b0159ec0 0x7f35b016d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:27.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.017+0000 7f35b7075700 1 -- 172.21.15.99:0/2604941549 >> 172.21.15.99:0/2604941549 conn(0x7f35b00d1880 msgr2=0x7f35b00c7360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:27.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.017+0000 7f35b7075700 1 -- 172.21.15.99:0/2604941549 shutdown_connections 2023-10-08T00:16:27.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.017+0000 7f35b7075700 1 -- 172.21.15.99:0/2604941549 wait complete. 2023-10-08T00:16:27.019 INFO:teuthology.orchestra.run.smithi099.stderr:updated caps for client.0 2023-10-08T00:16:27.029 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:16:27.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.142+0000 7f2cc1727700 1 Processor -- start 2023-10-08T00:16:27.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.142+0000 7f2cc1727700 1 -- start start 2023-10-08T00:16:27.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.143+0000 7f2cc1727700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2cbc159cf0 0x7f2cbc15e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:27.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.143+0000 7f2cc1727700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cbc0cca20 con 0x7f2cbc15a0c0 2023-10-08T00:16:27.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.143+0000 7f2cc1727700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cbc15ea40 con 0x7f2cbc0cf1c0 2023-10-08T00:16:27.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.143+0000 7f2cc1727700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cbc067660 con 0x7f2cbc159cf0 2023-10-08T00:16:27.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.143+0000 7f2cba7fc700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f2cbc15a0c0 0x7f2cbc0ceab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50670/0 (socket says 172.21.15.99:50670) 2023-10-08T00:16:27.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.143+0000 7f2cba7fc700 1 -- 172.21.15.99:0/4118262770 learned_addr learned my addr 172.21.15.99:0/4118262770 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:27.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.143+0000 7f2cbaffd700 1 --2- 172.21.15.99:0/4118262770 >> v2:172.21.15.191:3300/0 conn(0x7f2cbc159cf0 0x7f2cbc15e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:27.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.143+0000 7f2cb9ffb700 1 -- 172.21.15.99:0/4118262770 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4046052461 0 0) 0x7f2cbc0cca20 con 0x7f2cbc15a0c0 2023-10-08T00:16:27.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.143+0000 7f2cb9ffb700 1 -- 172.21.15.99:0/4118262770 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ca4003570 con 0x7f2cbc15a0c0 2023-10-08T00:16:27.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.144+0000 7f2cb9ffb700 1 -- 172.21.15.99:0/4118262770 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1365640415 0 0) 0x7f2cb4001110 con 0x7f2cbc0cf1c0 2023-10-08T00:16:27.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.144+0000 7f2cb9ffb700 1 -- 172.21.15.99:0/4118262770 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cbc0cca20 con 0x7f2cbc0cf1c0 2023-10-08T00:16:27.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.144+0000 7f2cbaffd700 1 -- 172.21.15.99:0/4118262770 >> v1:172.21.15.191:6789/0 conn(0x7f2cbc0cf1c0 legacy=0x7f2cbc068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:27.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.144+0000 7f2cbaffd700 1 -- 172.21.15.99:0/4118262770 >> v1:172.21.15.99:6789/0 conn(0x7f2cbc15a0c0 legacy=0x7f2cbc0ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:27.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.144+0000 7f2cbaffd700 1 -- 172.21.15.99:0/4118262770 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cbc067f00 con 0x7f2cbc159cf0 2023-10-08T00:16:27.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.144+0000 7f2cbaffd700 1 --2- 172.21.15.99:0/4118262770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2cbc159cf0 0x7f2cbc15e500 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2cac009e60 tx=0x7f2cac009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a78b7694cc00fd79 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:27.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.144+0000 7f2cb9ffb700 1 -- 172.21.15.99:0/4118262770 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cac00e070 con 0x7f2cbc159cf0 2023-10-08T00:16:27.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.144+0000 7f2cb9ffb700 1 -- 172.21.15.99:0/4118262770 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cac00bab0 con 0x7f2cbc159cf0 2023-10-08T00:16:27.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.144+0000 7f2cb9ffb700 1 -- 172.21.15.99:0/4118262770 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cac00be30 con 0x7f2cbc159cf0 2023-10-08T00:16:27.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.144+0000 7f2cc1727700 1 -- 172.21.15.99:0/4118262770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2cbc159cf0 msgr2=0x7f2cbc15e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:27.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.144+0000 7f2cc1727700 1 --2- 172.21.15.99:0/4118262770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2cbc159cf0 0x7f2cbc15e500 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2cac009e60 tx=0x7f2cac009780 comp rx=0 tx=0).stop 2023-10-08T00:16:27.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.145+0000 7f2cc1727700 1 -- 172.21.15.99:0/4118262770 shutdown_connections 2023-10-08T00:16:27.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.145+0000 7f2cc1727700 1 --2- 172.21.15.99:0/4118262770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2cbc159cf0 0x7f2cbc15e500 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:27.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.145+0000 7f2cc1727700 1 -- 172.21.15.99:0/4118262770 >> 172.21.15.99:0/4118262770 conn(0x7f2cbc06c6e0 msgr2=0x7f2cbc069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:27.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.145+0000 7f2cc1727700 1 -- 172.21.15.99:0/4118262770 shutdown_connections 2023-10-08T00:16:27.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.145+0000 7f2cc1727700 1 -- 172.21.15.99:0/4118262770 wait complete. 2023-10-08T00:16:27.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.146+0000 7f2cc1727700 1 Processor -- start 2023-10-08T00:16:27.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.146+0000 7f2cc1727700 1 -- start start 2023-10-08T00:16:27.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.147+0000 7f2cc1727700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2cbc0cf1c0 0x7f2cbc1571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:27.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.147+0000 7f2cc1727700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cbc159cf0 0x7f2cbc0c5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:27.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.147+0000 7f2cc1727700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2cbc15a0c0 0x7f2cbc0b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:27.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.147+0000 7f2cc1727700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc0c7f30 con 0x7f2cbc159cf0 2023-10-08T00:16:27.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.147+0000 7f2cc1727700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc0c8210 con 0x7f2cbc0cf1c0 2023-10-08T00:16:27.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.147+0000 7f2cc1727700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc0c84f0 con 0x7f2cbc15a0c0 2023-10-08T00:16:27.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.147+0000 7f2cba7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cbc159cf0 0x7f2cbc0c5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:27.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.147+0000 7f2cba7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cbc159cf0 0x7f2cbc0c5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37086/0 (socket says 172.21.15.99:37086) 2023-10-08T00:16:27.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.147+0000 7f2cba7fc700 1 -- 172.21.15.99:0/2453277063 learned_addr learned my addr 172.21.15.99:0/2453277063 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:27.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.147+0000 7f2cbaffd700 1 --2- 172.21.15.99:0/2453277063 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2cbc0cf1c0 0x7f2cbc1571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:27.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.147+0000 7f2cbb7fe700 1 --2- 172.21.15.99:0/2453277063 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2cbc15a0c0 0x7f2cbc0b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:27.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.147+0000 7f2cba7fc700 1 -- 172.21.15.99:0/2453277063 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2cbc15a0c0 msgr2=0x7f2cbc0b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:27.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.147+0000 7f2cba7fc700 1 --2- 172.21.15.99:0/2453277063 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2cbc15a0c0 0x7f2cbc0b28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:27.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.147+0000 7f2cba7fc700 1 -- 172.21.15.99:0/2453277063 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2cbc0cf1c0 msgr2=0x7f2cbc1571a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:27.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.147+0000 7f2cba7fc700 1 --2- 172.21.15.99:0/2453277063 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2cbc0cf1c0 0x7f2cbc1571a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:27.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.147+0000 7f2cba7fc700 1 -- 172.21.15.99:0/2453277063 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cac009a20 con 0x7f2cbc159cf0 2023-10-08T00:16:27.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.147+0000 7f2cbaffd700 1 --2- 172.21.15.99:0/2453277063 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2cbc0cf1c0 0x7f2cbc1571a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:16:27.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.147+0000 7f2cbb7fe700 1 --2- 172.21.15.99:0/2453277063 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2cbc15a0c0 0x7f2cbc0b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:16:27.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.147+0000 7f2cba7fc700 1 --2- 172.21.15.99:0/2453277063 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cbc159cf0 0x7f2cbc0c5dc0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f2cb000c790 tx=0x7f2cb0007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:27.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.147+0000 7f2ca2ffd700 1 -- 172.21.15.99:0/2453277063 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb0005b30 con 0x7f2cbc159cf0 2023-10-08T00:16:27.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.147+0000 7f2ca2ffd700 1 -- 172.21.15.99:0/2453277063 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cb0005c90 con 0x7f2cbc159cf0 2023-10-08T00:16:27.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.147+0000 7f2cc1727700 1 -- 172.21.15.99:0/2453277063 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cbc0c87d0 con 0x7f2cbc159cf0 2023-10-08T00:16:27.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.147+0000 7f2ca2ffd700 1 -- 172.21.15.99:0/2453277063 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb00072e0 con 0x7f2cbc159cf0 2023-10-08T00:16:27.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.147+0000 7f2cc1727700 1 -- 172.21.15.99:0/2453277063 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cbc0c7a70 con 0x7f2cbc159cf0 2023-10-08T00:16:27.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.148+0000 7f2cc1727700 1 -- 172.21.15.99:0/2453277063 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cbc0aa870 con 0x7f2cbc159cf0 2023-10-08T00:16:27.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.149+0000 7f2ca2ffd700 1 -- 172.21.15.99:0/2453277063 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2cb0007690 con 0x7f2cbc159cf0 2023-10-08T00:16:27.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.149+0000 7f2ca2ffd700 1 --2- 172.21.15.99:0/2453277063 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2ca8066fb0 0x7f2ca8069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:27.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.149+0000 7f2ca2ffd700 1 -- 172.21.15.99:0/2453277063 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2cb0083a30 con 0x7f2cbc159cf0 2023-10-08T00:16:27.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.149+0000 7f2cbaffd700 1 --2- 172.21.15.99:0/2453277063 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2ca8066fb0 0x7f2ca8069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:27.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.150+0000 7f2cbaffd700 1 --2- 172.21.15.99:0/2453277063 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2ca8066fb0 0x7f2ca8069460 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc0b3c40 tx=0x7f2cac019000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:27.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.153+0000 7f2ca2ffd700 1 -- 172.21.15.99:0/2453277063 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2cb004b5a0 con 0x7f2cbc159cf0 2023-10-08T00:16:27.379 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.378+0000 7f2cc1727700 1 -- 172.21.15.99:0/2453277063 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2cbc0c7c90 con 0x7f2cbc159cf0 2023-10-08T00:16:27.379 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.379+0000 7f2ca2ffd700 1 -- 172.21.15.99:0/2453277063 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f2cb0017020 con 0x7f2cbc159cf0 2023-10-08T00:16:27.380 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:16:27.380 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":56,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5627,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/2782755379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":2782755379},{"type":"v1","addr":"172.21.15.191:6837","nonce":2782755379}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6166,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/2980541392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2980541392},{"type":"v1","addr":"172.21.15.99:6834","nonce":2980541392}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6184,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/1342206989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1342206989},{"type":"v1","addr":"172.21.15.191:6836","nonce":1342206989}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:16:21.527609+0000","modified":"2023-10-08T00:16:25.185437+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6747},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6747":{"gid":6747,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/1495315849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":1495315849},{"type":"v1","addr":"172.21.15.99:6835","nonce":1495315849}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-08T00:16:27.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.381+0000 7f2cc1727700 1 -- 172.21.15.99:0/2453277063 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2ca8066fb0 msgr2=0x7f2ca8069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:27.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.381+0000 7f2cc1727700 1 --2- 172.21.15.99:0/2453277063 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2ca8066fb0 0x7f2ca8069460 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc0b3c40 tx=0x7f2cac019000 comp rx=0 tx=0).stop 2023-10-08T00:16:27.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.381+0000 7f2cc1727700 1 -- 172.21.15.99:0/2453277063 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cbc159cf0 msgr2=0x7f2cbc0c5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:27.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.381+0000 7f2cc1727700 1 --2- 172.21.15.99:0/2453277063 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cbc159cf0 0x7f2cbc0c5dc0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f2cb000c790 tx=0x7f2cb0007ea0 comp rx=0 tx=0).stop 2023-10-08T00:16:27.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.381+0000 7f2cc1727700 1 -- 172.21.15.99:0/2453277063 shutdown_connections 2023-10-08T00:16:27.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.381+0000 7f2cc1727700 1 --2- 172.21.15.99:0/2453277063 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2ca8066fb0 0x7f2ca8069460 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:27.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.381+0000 7f2cc1727700 1 --2- 172.21.15.99:0/2453277063 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2cbc0cf1c0 0x7f2cbc1571a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:27.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.381+0000 7f2cc1727700 1 --2- 172.21.15.99:0/2453277063 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2cbc15a0c0 0x7f2cbc0b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:27.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.381+0000 7f2cc1727700 1 --2- 172.21.15.99:0/2453277063 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cbc159cf0 0x7f2cbc0c5dc0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:27.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.381+0000 7f2cc1727700 1 -- 172.21.15.99:0/2453277063 >> 172.21.15.99:0/2453277063 conn(0x7f2cbc06c6e0 msgr2=0x7f2cbc0cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:27.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.381+0000 7f2cc1727700 1 -- 172.21.15.99:0/2453277063 shutdown_connections 2023-10-08T00:16:27.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.381+0000 7f2cc1727700 1 -- 172.21.15.99:0/2453277063 wait complete. 2023-10-08T00:16:27.383 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 56 2023-10-08T00:16:27.459 INFO:tasks.ceph.mds.c.smithi099.stderr:2023-10-08T00:16:27.458+0000 7f59355b3700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T00:16:27.796 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 56, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T00:16:21.527609+0000', 'modified': '2023-10-08T00:16:25.185437+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6747}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6747': {'gid': 6747, 'name': 'c', 'rank': 0, 'incarnation': 52, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.99:6835/1495315849', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6833', 'nonce': 1495315849}, {'type': 'v1', 'addr': '172.21.15.99:6835', 'nonce': 1495315849}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-08T00:16:27.796 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T00:16:27.797 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-08T00:16:27.913 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.912+0000 7f6c2c37a700 1 Processor -- start 2023-10-08T00:16:27.913 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.913+0000 7f6c2c37a700 1 -- start start 2023-10-08T00:16:27.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.913+0000 7f6c2c37a700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f6c2415a2b0 0x7f6c240b3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:27.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.913+0000 7f6c2c37a700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6c240d1c30 0x7f6c240cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:27.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.913+0000 7f6c2c37a700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c240c5d80 con 0x7f6c2415a680 2023-10-08T00:16:27.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.913+0000 7f6c2c37a700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c24068470 con 0x7f6c240d1c30 2023-10-08T00:16:27.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.913+0000 7f6c2c37a700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c240c6060 con 0x7f6c2415a2b0 2023-10-08T00:16:27.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.914+0000 7f6c29915700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6c240d1c30 0x7f6c240cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:27.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.914+0000 7f6c29915700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6c240d1c30 0x7f6c240cf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37088/0 (socket says 172.21.15.99:37088) 2023-10-08T00:16:27.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.914+0000 7f6c29915700 1 -- 172.21.15.99:0/3675830523 learned_addr learned my addr 172.21.15.99:0/3675830523 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:27.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.914+0000 7f6c2a116700 1 --2- 172.21.15.99:0/3675830523 >> v2:172.21.15.191:3300/0 conn(0x7f6c2415a2b0 0x7f6c240b3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:27.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.914+0000 7f6c29915700 1 -- 172.21.15.99:0/3675830523 >> v2:172.21.15.191:3300/0 conn(0x7f6c2415a2b0 msgr2=0x7f6c240b3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:27.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.914+0000 7f6c29915700 1 --2- 172.21.15.99:0/3675830523 >> v2:172.21.15.191:3300/0 conn(0x7f6c2415a2b0 0x7f6c240b3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:27.917 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.914+0000 7f6c29915700 1 -- 172.21.15.99:0/3675830523 >> v1:172.21.15.191:6789/0 conn(0x7f6c2415a680 legacy=0x7f6c240c7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:27.917 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.914+0000 7f6c29915700 1 -- 172.21.15.99:0/3675830523 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c240c6340 con 0x7f6c240d1c30 2023-10-08T00:16:27.917 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.914+0000 7f6c2a116700 1 --2- 172.21.15.99:0/3675830523 >> v2:172.21.15.191:3300/0 conn(0x7f6c2415a2b0 0x7f6c240b3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:27.917 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.914+0000 7f6c29915700 1 --2- 172.21.15.99:0/3675830523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c240d1c30 0x7f6c240cf710 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f6c24089220 tx=0x7f6c18009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fb2a02ad19a18a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:27.917 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.914+0000 7f6c29114700 1 -- 172.21.15.99:0/3675830523 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1800b860 con 0x7f6c240d1c30 2023-10-08T00:16:27.917 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.914+0000 7f6c29114700 1 -- 172.21.15.99:0/3675830523 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c1800b9c0 con 0x7f6c240d1c30 2023-10-08T00:16:27.917 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.914+0000 7f6c29114700 1 -- 172.21.15.99:0/3675830523 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1800bd60 con 0x7f6c240d1c30 2023-10-08T00:16:27.917 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.915+0000 7f6c2c37a700 1 -- 172.21.15.99:0/3675830523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c240d1c30 msgr2=0x7f6c240cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:27.917 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.915+0000 7f6c2c37a700 1 --2- 172.21.15.99:0/3675830523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c240d1c30 0x7f6c240cf710 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f6c24089220 tx=0x7f6c18009250 comp rx=0 tx=0).stop 2023-10-08T00:16:27.917 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.915+0000 7f6c2c37a700 1 -- 172.21.15.99:0/3675830523 shutdown_connections 2023-10-08T00:16:27.917 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.915+0000 7f6c2c37a700 1 --2- 172.21.15.99:0/3675830523 >> v2:172.21.15.191:3300/0 conn(0x7f6c2415a2b0 0x7f6c240b3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:27.917 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.915+0000 7f6c2c37a700 1 --2- 172.21.15.99:0/3675830523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c240d1c30 0x7f6c240cf710 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:27.918 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.915+0000 7f6c2c37a700 1 -- 172.21.15.99:0/3675830523 >> 172.21.15.99:0/3675830523 conn(0x7f6c24067e40 msgr2=0x7f6c240d0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:27.918 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.915+0000 7f6c2c37a700 1 -- 172.21.15.99:0/3675830523 shutdown_connections 2023-10-08T00:16:27.918 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.915+0000 7f6c2c37a700 1 -- 172.21.15.99:0/3675830523 wait complete. 2023-10-08T00:16:27.920 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.916+0000 7f6c2c37a700 1 Processor -- start 2023-10-08T00:16:27.920 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.916+0000 7f6c2c37a700 1 -- start start 2023-10-08T00:16:27.920 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.917+0000 7f6c2c37a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c240d1c30 0x7f6c24155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:27.920 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.917+0000 7f6c2c37a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c2415a2b0 0x7f6c2416b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:27.920 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.917+0000 7f6c2c37a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c2415a680 0x7f6c240c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:27.921 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.917+0000 7f6c2c37a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c240c98d0 con 0x7f6c2415a680 2023-10-08T00:16:27.921 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.917+0000 7f6c2c37a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c240b3d80 con 0x7f6c240d1c30 2023-10-08T00:16:27.921 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.917+0000 7f6c2c37a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c240c9100 con 0x7f6c2415a2b0 2023-10-08T00:16:27.921 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.917+0000 7f6c2a917700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c2415a680 0x7f6c240c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:27.921 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.917+0000 7f6c2a917700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c2415a680 0x7f6c240c8bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37096/0 (socket says 172.21.15.99:37096) 2023-10-08T00:16:27.921 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.917+0000 7f6c2a917700 1 -- 172.21.15.99:0/1865099600 learned_addr learned my addr 172.21.15.99:0/1865099600 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:27.921 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.917+0000 7f6c2a116700 1 --2- 172.21.15.99:0/1865099600 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c240d1c30 0x7f6c24155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:27.921 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.917+0000 7f6c29915700 1 --2- 172.21.15.99:0/1865099600 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c2415a2b0 0x7f6c2416b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:27.921 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.917+0000 7f6c2a917700 1 -- 172.21.15.99:0/1865099600 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c2415a2b0 msgr2=0x7f6c2416b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:27.921 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.917+0000 7f6c2a917700 1 --2- 172.21.15.99:0/1865099600 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c2415a2b0 0x7f6c2416b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:27.921 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.917+0000 7f6c2a917700 1 -- 172.21.15.99:0/1865099600 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c240d1c30 msgr2=0x7f6c24155b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:27.921 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.917+0000 7f6c2a917700 1 --2- 172.21.15.99:0/1865099600 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c240d1c30 0x7f6c24155b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:27.922 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.917+0000 7f6c2a917700 1 -- 172.21.15.99:0/1865099600 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c180094f0 con 0x7f6c2415a680 2023-10-08T00:16:27.922 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.917+0000 7f6c2a116700 1 --2- 172.21.15.99:0/1865099600 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c240d1c30 0x7f6c24155b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:27.922 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.917+0000 7f6c2a917700 1 --2- 172.21.15.99:0/1865099600 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c2415a680 0x7f6c240c8bc0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c00c4c0 tx=0x7f6c1c007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:27.922 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.917+0000 7f6c227fc700 1 -- 172.21.15.99:0/1865099600 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c011590 con 0x7f6c2415a680 2023-10-08T00:16:27.922 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.917+0000 7f6c227fc700 1 -- 172.21.15.99:0/1865099600 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c1c011710 con 0x7f6c2415a680 2023-10-08T00:16:27.922 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.917+0000 7f6c227fc700 1 -- 172.21.15.99:0/1865099600 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c011ad0 con 0x7f6c2415a680 2023-10-08T00:16:27.922 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.917+0000 7f6c2c37a700 1 -- 172.21.15.99:0/1865099600 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c240b4300 con 0x7f6c2415a680 2023-10-08T00:16:27.922 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.917+0000 7f6c2c37a700 1 -- 172.21.15.99:0/1865099600 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c240c95f0 con 0x7f6c2415a680 2023-10-08T00:16:27.922 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.919+0000 7f6c227fc700 1 -- 172.21.15.99:0/1865099600 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6c1c01b3b0 con 0x7f6c2415a680 2023-10-08T00:16:27.922 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.919+0000 7f6c227fc700 1 --2- 172.21.15.99:0/1865099600 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6c10066f30 0x7f6c100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:27.922 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.919+0000 7f6c227fc700 1 -- 172.21.15.99:0/1865099600 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6c1c0838e0 con 0x7f6c2415a680 2023-10-08T00:16:27.922 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.919+0000 7f6c2a116700 1 --2- 172.21.15.99:0/1865099600 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6c10066f30 0x7f6c100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:27.923 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.920+0000 7f6c2a116700 1 --2- 172.21.15.99:0/1865099600 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6c10066f30 0x7f6c100693e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6c14009fd0 tx=0x7f6c140098c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:27.932 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.921+0000 7f6c35672b80 1 Processor -- start 2023-10-08T00:16:27.933 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.921+0000 7f6c35672b80 1 -- start start 2023-10-08T00:16:27.933 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.922+0000 7f6c35672b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609663faa50 0x5609663fae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:27.933 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.922+0000 7f6c35672b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560966444d00 0x560966423a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:27.933 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.922+0000 7f6c35672b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560966423f60 0x5609663f6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:27.933 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.922+0000 7f6c35672b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560966351a70 con 0x560966423f60 2023-10-08T00:16:27.933 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.922+0000 7f6c35672b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x56096634f270 con 0x5609663faa50 2023-10-08T00:16:27.933 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.922+0000 7f6c35672b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x56096634b060 con 0x560966444d00 2023-10-08T00:16:27.933 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.922+0000 7f6c2a917700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560966423f60 0x5609663f6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:27.933 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.922+0000 7f6c2a917700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560966423f60 0x5609663f6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37102/0 (socket says 172.21.15.99:37102) 2023-10-08T00:16:27.933 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.922+0000 7f6c2a917700 1 -- 172.21.15.99:0/3336404810 learned_addr learned my addr 172.21.15.99:0/3336404810 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:27.933 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.922+0000 7f6c2a116700 1 --2- 172.21.15.99:0/3336404810 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609663faa50 0x5609663fae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:27.933 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.922+0000 7f6c29915700 1 --2- 172.21.15.99:0/3336404810 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560966444d00 0x560966423a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:27.934 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.922+0000 7f6c2a917700 1 -- 172.21.15.99:0/3336404810 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560966444d00 msgr2=0x560966423a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:27.934 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.922+0000 7f6c2a917700 1 --2- 172.21.15.99:0/3336404810 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560966444d00 0x560966423a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:27.934 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.923+0000 7f6c2a917700 1 -- 172.21.15.99:0/3336404810 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609663faa50 msgr2=0x5609663fae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:27.934 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.923+0000 7f6c2a917700 1 --2- 172.21.15.99:0/3336404810 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609663faa50 0x5609663fae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:27.934 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.923+0000 7f6c2a917700 1 -- 172.21.15.99:0/3336404810 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609661e5270 con 0x560966423f60 2023-10-08T00:16:27.934 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.923+0000 7f6c2a917700 1 --2- 172.21.15.99:0/3336404810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560966423f60 0x5609663f6cf0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c081dd0 tx=0x7f6c1c081d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2df1466fbeca1d27 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:27.935 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.923+0000 7f6c0b7fe700 1 -- 172.21.15.99:0/3336404810 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c057a60 con 0x560966423f60 2023-10-08T00:16:27.935 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.923+0000 7f6c0b7fe700 1 -- 172.21.15.99:0/3336404810 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c1c05b7b0 con 0x560966423f60 2023-10-08T00:16:27.935 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.923+0000 7f6c0b7fe700 1 -- 172.21.15.99:0/3336404810 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c052950 con 0x560966423f60 2023-10-08T00:16:27.935 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.923+0000 7f6c35672b80 1 -- 172.21.15.99:0/3336404810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560966423f60 msgr2=0x5609663f6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:27.935 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.923+0000 7f6c35672b80 1 --2- 172.21.15.99:0/3336404810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560966423f60 0x5609663f6cf0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c081dd0 tx=0x7f6c1c081d60 comp rx=0 tx=0).stop 2023-10-08T00:16:27.935 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.923+0000 7f6c35672b80 1 -- 172.21.15.99:0/3336404810 shutdown_connections 2023-10-08T00:16:27.935 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.923+0000 7f6c35672b80 1 --2- 172.21.15.99:0/3336404810 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609663faa50 0x5609663fae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:27.936 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.923+0000 7f6c35672b80 1 --2- 172.21.15.99:0/3336404810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560966423f60 0x5609663f6cf0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:27.936 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.923+0000 7f6c35672b80 1 --2- 172.21.15.99:0/3336404810 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560966444d00 0x560966423a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:27.936 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.923+0000 7f6c35672b80 1 -- 172.21.15.99:0/3336404810 >> 172.21.15.99:0/3336404810 conn(0x5609663fc370 msgr2=0x5609663fc740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:27.936 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.923+0000 7f6c35672b80 1 -- 172.21.15.99:0/3336404810 shutdown_connections 2023-10-08T00:16:27.936 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.924+0000 7f6c35672b80 1 -- 172.21.15.99:0/3336404810 wait complete. 2023-10-08T00:16:27.936 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.924+0000 7f6c35672b80 1 -- start start 2023-10-08T00:16:27.936 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.924+0000 7f6c35672b80 1 Processor -- start 2023-10-08T00:16:27.937 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.925+0000 7f6c35672b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5609663faa50 0x560966451d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:27.937 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.925+0000 7f6c35672b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560966423f60 0x56096643e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:27.937 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.925+0000 7f6c35672b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560966444d00 0x56096643e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:27.937 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.925+0000 7f6c35672b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609662f6890 con 0x560966423f60 2023-10-08T00:16:27.937 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.925+0000 7f6c35672b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560966311ea0 con 0x560966444d00 2023-10-08T00:16:27.937 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.925+0000 7f6c35672b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x5609661cde30 con 0x5609663faa50 2023-10-08T00:16:27.938 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.925+0000 7f6c35672b80 10 client.0 _ll_register_callbacks cb 0x5609663be110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T00:16:27.938 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.925+0000 7f6c29915700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560966423f60 0x56096643e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:27.938 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.925+0000 7f6c29915700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560966423f60 0x56096643e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37114/0 (socket says 172.21.15.99:37114) 2023-10-08T00:16:27.938 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.925+0000 7f6c29915700 1 -- 172.21.15.99:0/1400017035 learned_addr learned my addr 172.21.15.99:0/1400017035 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:27.938 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.925+0000 7f6c2a116700 1 --2- 172.21.15.99:0/1400017035 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5609663faa50 0x560966451d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:27.938 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.925+0000 7f6c2a917700 1 --2- 172.21.15.99:0/1400017035 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560966444d00 0x56096643e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:27.939 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.925+0000 7f6c29915700 1 -- 172.21.15.99:0/1400017035 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5609663faa50 msgr2=0x560966451d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:27.939 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.925+0000 7f6c29915700 1 --2- 172.21.15.99:0/1400017035 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5609663faa50 0x560966451d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:27.939 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.925+0000 7f6c29915700 1 -- 172.21.15.99:0/1400017035 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560966444d00 msgr2=0x56096643e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:27.939 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.925+0000 7f6c29915700 1 --2- 172.21.15.99:0/1400017035 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560966444d00 0x56096643e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:27.939 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.925+0000 7f6c29915700 1 -- 172.21.15.99:0/1400017035 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c240c95f0 con 0x560966423f60 2023-10-08T00:16:27.939 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.926+0000 7f6c29915700 1 --2- 172.21.15.99:0/1400017035 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560966423f60 0x56096643e420 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f6c24089120 tx=0x7f6c18017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb8c0b3e803f9cd8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:27.940 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.926+0000 7f6c08ff9700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T00:16:27.940 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.926+0000 7f6c08ff9700 1 -- 172.21.15.99:0/1400017035 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c18022070 con 0x560966423f60 2023-10-08T00:16:27.940 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.926+0000 7f6c08ff9700 1 -- 172.21.15.99:0/1400017035 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6c18028b00 con 0x560966423f60 2023-10-08T00:16:27.940 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.926+0000 7f6c35672b80 1 -- 172.21.15.99:0/1400017035 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5609663efb70 con 0x560966423f60 2023-10-08T00:16:27.940 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.926+0000 7f6c08ff9700 1 -- 172.21.15.99:0/1400017035 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c18011630 con 0x560966423f60 2023-10-08T00:16:27.940 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.926+0000 7f6c35672b80 10 client.6888 fetch_fsmap learned FSMap version 56 2023-10-08T00:16:27.940 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.926+0000 7f6c35672b80 1 -- 172.21.15.99:0/1400017035 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x560966187fe0 con 0x560966423f60 2023-10-08T00:16:27.940 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.926+0000 7f6c08ff9700 1 -- 172.21.15.99:0/1400017035 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5609663efb70 con 0x560966423f60 2023-10-08T00:16:27.941 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.926+0000 7f6c08ff9700 1 -- 172.21.15.99:0/1400017035 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 56) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f6c18011790 con 0x560966423f60 2023-10-08T00:16:27.941 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.926+0000 7f6c35672b80 10 client.6888 fetch_fsmap finished waiting for FSMap version 56 2023-10-08T00:16:27.941 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.926+0000 7f6c35672b80 10 client.6888 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-08T00:16:27.941 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.926+0000 7f6c35672b80 20 client.6888 populate_metadata read hostname 'smithi099' 2023-10-08T00:16:27.941 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.926+0000 7f6c35672b80 1 --2- 172.21.15.99:0/1400017035 >> [v2:172.21.15.99:6833/1495315849,v1:172.21.15.99:6835/1495315849] conn(0x56096643aed0 0x56096643b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:27.941 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.926+0000 7f6c35672b80 4 client.6888 mds_command: new command op to 6747 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-08T00:16:27.942 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.926+0000 7f6c35672b80 1 -- 172.21.15.99:0/1400017035 --> [v2:172.21.15.99:6833/1495315849,v1:172.21.15.99:6835/1495315849] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5609663ccaa0 con 0x56096643aed0 2023-10-08T00:16:27.942 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.926+0000 7f6c2a116700 1 --2- 172.21.15.99:0/1400017035 >> [v2:172.21.15.99:6833/1495315849,v1:172.21.15.99:6835/1495315849] conn(0x56096643aed0 0x56096643b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:27.942 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.927+0000 7f6c2a116700 1 --2- 172.21.15.99:0/1400017035 >> [v2:172.21.15.99:6833/1495315849,v1:172.21.15.99:6835/1495315849] conn(0x56096643aed0 0x56096643b2a0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7591c58194361cc6 server_cookie=7dd3062d5b2adaf6 in_seq=0 out_seq=0 2023-10-08T00:16:27.942 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.927+0000 7f6c08ff9700 10 client.6888 ms_handle_connect on v2:172.21.15.99:6833/1495315849 2023-10-08T00:16:27.942 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.927+0000 7f6c08ff9700 1 -- 172.21.15.99:0/1400017035 <== mds.0 v2:172.21.15.99:6833/1495315849 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5609663ccaa0 con 0x56096643aed0 2023-10-08T00:16:27.942 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.927+0000 7f6c08ff9700 10 client.6888 handle_command_reply: tid=0 2023-10-08T00:16:27.943 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.927+0000 7f6c35672b80 1 client.6888 shutdown 2023-10-08T00:16:27.943 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.927+0000 7f6c35672b80 1 -- 172.21.15.99:0/1400017035 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560966423f60 msgr2=0x56096643e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:27.943 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.927+0000 7f6c35672b80 1 --2- 172.21.15.99:0/1400017035 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560966423f60 0x56096643e420 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f6c24089120 tx=0x7f6c18017000 comp rx=0 tx=0).stop 2023-10-08T00:16:27.943 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.927+0000 7f6c35672b80 1 -- 172.21.15.99:0/1400017035 shutdown_connections 2023-10-08T00:16:27.943 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.927+0000 7f6c35672b80 1 --2- 172.21.15.99:0/1400017035 >> [v2:172.21.15.99:6833/1495315849,v1:172.21.15.99:6835/1495315849] conn(0x56096643aed0 0x56096643b2a0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:27.943 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.927+0000 7f6c35672b80 1 --2- 172.21.15.99:0/1400017035 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560966444d00 0x56096643e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:27.944 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.927+0000 7f6c35672b80 1 --2- 172.21.15.99:0/1400017035 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5609663faa50 0x560966451d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:27.944 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.927+0000 7f6c08ff9700 0 client.6888 ms_handle_reset on v2:172.21.15.99:6833/1495315849 2023-10-08T00:16:27.944 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.927+0000 7f6c35672b80 1 --2- 172.21.15.99:0/1400017035 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560966423f60 0x56096643e420 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:27.944 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.927+0000 7f6c35672b80 1 -- 172.21.15.99:0/1400017035 >> 172.21.15.99:0/1400017035 conn(0x5609663fc370 msgr2=0x5609663f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:27.944 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.927+0000 7f6c35672b80 1 -- 172.21.15.99:0/1400017035 shutdown_connections 2023-10-08T00:16:27.944 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.927+0000 7f6c35672b80 1 -- 172.21.15.99:0/1400017035 wait complete. 2023-10-08T00:16:27.944 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.928+0000 7f6c35672b80 20 client.6888 trim_cache size 0 max 16384 2023-10-08T00:16:27.947 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.944+0000 7f6c35672b80 1 Processor -- start 2023-10-08T00:16:27.947 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.944+0000 7f6c35672b80 1 -- start start 2023-10-08T00:16:27.947 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.945+0000 7f6c35672b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609663faa50 0x56096643e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:27.947 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.945+0000 7f6c35672b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560966423f60 0x56096643e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:27.947 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.945+0000 7f6c35672b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560966444d00 0x5609664a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:27.947 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.945+0000 7f6c35672b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609663fd140 con 0x560966444d00 2023-10-08T00:16:27.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.945+0000 7f6c35672b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560966442bb0 con 0x5609663faa50 2023-10-08T00:16:27.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.945+0000 7f6c35672b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x5609663fa370 con 0x560966423f60 2023-10-08T00:16:27.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.945+0000 7f6c2a917700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560966444d00 0x5609664a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:27.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.945+0000 7f6c2a917700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560966444d00 0x5609664a0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37128/0 (socket says 172.21.15.99:37128) 2023-10-08T00:16:27.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.945+0000 7f6c2a917700 1 -- 172.21.15.99:0/2644058322 learned_addr learned my addr 172.21.15.99:0/2644058322 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:27.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.945+0000 7f6c2a917700 1 -- 172.21.15.99:0/2644058322 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560966423f60 msgr2=0x56096643e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:27.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.945+0000 7f6c2a917700 1 --2- 172.21.15.99:0/2644058322 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560966423f60 0x56096643e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:27.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.945+0000 7f6c2a917700 1 -- 172.21.15.99:0/2644058322 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609663faa50 msgr2=0x56096643e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:27.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.945+0000 7f6c2a917700 1 --2- 172.21.15.99:0/2644058322 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609663faa50 0x56096643e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:27.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.945+0000 7f6c2a917700 1 -- 172.21.15.99:0/2644058322 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560966319780 con 0x560966444d00 2023-10-08T00:16:27.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.945+0000 7f6c2a917700 1 --2- 172.21.15.99:0/2644058322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560966444d00 0x5609664a0b00 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c00c490 tx=0x7f6c1c003000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=192d1221d74eff4e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:27.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.945+0000 7f6c0bfff700 1 -- 172.21.15.99:0/2644058322 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c054810 con 0x560966444d00 2023-10-08T00:16:27.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.945+0000 7f6c0bfff700 1 -- 172.21.15.99:0/2644058322 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c1c057a60 con 0x560966444d00 2023-10-08T00:16:27.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.945+0000 7f6c0bfff700 1 -- 172.21.15.99:0/2644058322 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c011e40 con 0x560966444d00 2023-10-08T00:16:27.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.946+0000 7f6c35672b80 1 -- 172.21.15.99:0/2644058322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560966444d00 msgr2=0x5609664a0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:27.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.946+0000 7f6c35672b80 1 --2- 172.21.15.99:0/2644058322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560966444d00 0x5609664a0b00 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c00c490 tx=0x7f6c1c003000 comp rx=0 tx=0).stop 2023-10-08T00:16:27.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.946+0000 7f6c35672b80 1 -- 172.21.15.99:0/2644058322 shutdown_connections 2023-10-08T00:16:27.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.946+0000 7f6c35672b80 1 --2- 172.21.15.99:0/2644058322 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609663faa50 0x56096643e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:27.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.946+0000 7f6c35672b80 1 --2- 172.21.15.99:0/2644058322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560966444d00 0x5609664a0b00 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:27.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.946+0000 7f6c35672b80 1 --2- 172.21.15.99:0/2644058322 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560966423f60 0x56096643e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:27.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.946+0000 7f6c35672b80 1 -- 172.21.15.99:0/2644058322 >> 172.21.15.99:0/2644058322 conn(0x5609663fc370 msgr2=0x5609663fae20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:27.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.946+0000 7f6c35672b80 1 -- 172.21.15.99:0/2644058322 shutdown_connections 2023-10-08T00:16:27.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.946+0000 7f6c35672b80 1 -- 172.21.15.99:0/2644058322 wait complete. 2023-10-08T00:16:27.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.946+0000 7f6c35672b80 1 -- start start 2023-10-08T00:16:27.950 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.947+0000 7f6c35672b80 1 Processor -- start 2023-10-08T00:16:27.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.947+0000 7f6c35672b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5609663faa50 0x5609663fb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:27.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.947+0000 7f6c35672b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560966423f60 0x56096649ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:27.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.947+0000 7f6c35672b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560966444d00 0x56096649f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:27.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.947+0000 7f6c35672b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609663f5fa0 con 0x560966444d00 2023-10-08T00:16:27.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.947+0000 7f6c35672b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560966462070 con 0x560966423f60 2023-10-08T00:16:27.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.947+0000 7f6c35672b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x56096643e910 con 0x5609663faa50 2023-10-08T00:16:27.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.947+0000 7f6c35672b80 10 client.0 _ll_register_callbacks cb 0x5609663be110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T00:16:27.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.947+0000 7f6c2a917700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560966444d00 0x56096649f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:27.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.947+0000 7f6c2a917700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560966444d00 0x56096649f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37138/0 (socket says 172.21.15.99:37138) 2023-10-08T00:16:27.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.948+0000 7f6c2a917700 1 -- 172.21.15.99:0/3704281491 learned_addr learned my addr 172.21.15.99:0/3704281491 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:27.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.948+0000 7f6c2a116700 1 --2- 172.21.15.99:0/3704281491 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5609663faa50 0x5609663fb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:27.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.948+0000 7f6c29915700 1 --2- 172.21.15.99:0/3704281491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560966423f60 0x56096649ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:27.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.948+0000 7f6c2a917700 1 -- 172.21.15.99:0/3704281491 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5609663faa50 msgr2=0x5609663fb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:27.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.948+0000 7f6c2a917700 1 --2- 172.21.15.99:0/3704281491 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5609663faa50 0x5609663fb320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:27.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.948+0000 7f6c2a917700 1 -- 172.21.15.99:0/3704281491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560966423f60 msgr2=0x56096649ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:27.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.948+0000 7f6c2a917700 1 --2- 172.21.15.99:0/3704281491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560966423f60 0x56096649ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:27.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.948+0000 7f6c2a917700 1 -- 172.21.15.99:0/3704281491 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5609661e5270 con 0x560966444d00 2023-10-08T00:16:27.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.948+0000 7f6c29915700 1 --2- 172.21.15.99:0/3704281491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560966423f60 0x56096649ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:27.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.948+0000 7f6c2a116700 1 --2- 172.21.15.99:0/3704281491 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5609663faa50 0x5609663fb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:27.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.948+0000 7f6c2a917700 1 --2- 172.21.15.99:0/3704281491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560966444d00 0x56096649f1a0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c05d040 tx=0x7f6c1c07e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f14f097cb2b73d25 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:27.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.948+0000 7f6c08ff9700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T00:16:27.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.948+0000 7f6c08ff9700 1 -- 172.21.15.99:0/3704281491 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c053d00 con 0x560966444d00 2023-10-08T00:16:27.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.948+0000 7f6c08ff9700 1 -- 172.21.15.99:0/3704281491 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6c1c053e60 con 0x560966444d00 2023-10-08T00:16:27.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.948+0000 7f6c35672b80 1 -- 172.21.15.99:0/3704281491 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6c1c05b7b0 con 0x560966444d00 2023-10-08T00:16:27.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.948+0000 7f6c08ff9700 1 -- 172.21.15.99:0/3704281491 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c1c05b530 con 0x560966444d00 2023-10-08T00:16:27.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.948+0000 7f6c35672b80 10 client.6894 fetch_fsmap learned FSMap version 56 2023-10-08T00:16:27.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.948+0000 7f6c35672b80 1 -- 172.21.15.99:0/3704281491 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x560966300470 con 0x560966444d00 2023-10-08T00:16:27.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.948+0000 7f6c08ff9700 1 -- 172.21.15.99:0/3704281491 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c1c05b7b0 con 0x560966444d00 2023-10-08T00:16:27.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.948+0000 7f6c08ff9700 1 -- 172.21.15.99:0/3704281491 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 56) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f6c1c070170 con 0x560966444d00 2023-10-08T00:16:27.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.948+0000 7f6c35672b80 10 client.6894 fetch_fsmap finished waiting for FSMap version 56 2023-10-08T00:16:27.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.948+0000 7f6c35672b80 10 client.6894 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-08T00:16:27.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.948+0000 7f6c35672b80 20 client.6894 populate_metadata read hostname 'smithi099' 2023-10-08T00:16:27.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.948+0000 7f6c35672b80 1 --2- 172.21.15.99:0/3704281491 >> [v2:172.21.15.99:6833/1495315849,v1:172.21.15.99:6835/1495315849] conn(0x56096649d290 0x56096649d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:27.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.948+0000 7f6c35672b80 4 client.6894 mds_command: new command op to 6747 tid=0[{"prefix": "status"}] 2023-10-08T00:16:27.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.948+0000 7f6c35672b80 1 -- 172.21.15.99:0/3704281491 --> [v2:172.21.15.99:6833/1495315849,v1:172.21.15.99:6835/1495315849] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5609662ede00 con 0x56096649d290 2023-10-08T00:16:27.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.949+0000 7f6c2a116700 1 --2- 172.21.15.99:0/3704281491 >> [v2:172.21.15.99:6833/1495315849,v1:172.21.15.99:6835/1495315849] conn(0x56096649d290 0x56096649d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:27.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.949+0000 7f6c08ff9700 10 client.6894 ms_handle_connect on v2:172.21.15.99:6833/1495315849 2023-10-08T00:16:27.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.949+0000 7f6c2a116700 1 --2- 172.21.15.99:0/3704281491 >> [v2:172.21.15.99:6833/1495315849,v1:172.21.15.99:6835/1495315849] conn(0x56096649d290 0x56096649d660 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a377262e38fd4829 server_cookie=30ce6ed43e637c4a in_seq=0 out_seq=0 2023-10-08T00:16:27.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.949+0000 7f6c08ff9700 1 -- 172.21.15.99:0/3704281491 <== mds.0 v2:172.21.15.99:6833/1495315849 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5609662ede00 con 0x56096649d290 2023-10-08T00:16:27.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.949+0000 7f6c08ff9700 10 client.6894 handle_command_reply: tid=0 2023-10-08T00:16:27.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.949+0000 7f6c35672b80 1 client.6894 shutdown 2023-10-08T00:16:27.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.949+0000 7f6c35672b80 1 -- 172.21.15.99:0/3704281491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560966444d00 msgr2=0x56096649f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:27.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.949+0000 7f6c35672b80 1 --2- 172.21.15.99:0/3704281491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560966444d00 0x56096649f1a0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c05d040 tx=0x7f6c1c07e070 comp rx=0 tx=0).stop 2023-10-08T00:16:27.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.949+0000 7f6c35672b80 1 -- 172.21.15.99:0/3704281491 shutdown_connections 2023-10-08T00:16:27.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.949+0000 7f6c35672b80 1 --2- 172.21.15.99:0/3704281491 >> [v2:172.21.15.99:6833/1495315849,v1:172.21.15.99:6835/1495315849] conn(0x56096649d290 0x56096649d660 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:27.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.949+0000 7f6c35672b80 1 --2- 172.21.15.99:0/3704281491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560966444d00 0x56096649f1a0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:27.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.949+0000 7f6c35672b80 1 --2- 172.21.15.99:0/3704281491 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5609663faa50 0x5609663fb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:27.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.949+0000 7f6c08ff9700 0 client.6894 ms_handle_reset on v2:172.21.15.99:6833/1495315849 2023-10-08T00:16:27.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.949+0000 7f6c35672b80 1 --2- 172.21.15.99:0/3704281491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560966423f60 0x56096649ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:27.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.949+0000 7f6c35672b80 1 -- 172.21.15.99:0/3704281491 >> 172.21.15.99:0/3704281491 conn(0x5609663fc370 msgr2=0x5609664a08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:27.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.949+0000 7f6c35672b80 1 -- 172.21.15.99:0/3704281491 shutdown_connections 2023-10-08T00:16:27.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.949+0000 7f6c35672b80 1 -- 172.21.15.99:0/3704281491 wait complete. 2023-10-08T00:16:27.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.949+0000 7f6c35672b80 20 client.6894 trim_cache size 0 max 16384 2023-10-08T00:16:27.959 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:16:27.959 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", 2023-10-08T00:16:27.959 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 0, 2023-10-08T00:16:27.959 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 6747, 2023-10-08T00:16:27.959 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2023-10-08T00:16:27.959 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2023-10-08T00:16:27.960 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2023-10-08T00:16:27.960 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 5.7842129230000001, 2023-10-08T00:16:27.960 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 56, 2023-10-08T00:16:27.960 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 124, 2023-10-08T00:16:27.960 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 124, 2023-10-08T00:16:27.960 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 10.888891251 2023-10-08T00:16:27.960 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:16:27.961 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.950+0000 7f6c2c37a700 1 -- 172.21.15.99:0/1865099600 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6c10066f30 msgr2=0x7f6c100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:27.961 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.950+0000 7f6c2c37a700 1 --2- 172.21.15.99:0/1865099600 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6c10066f30 0x7f6c100693e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6c14009fd0 tx=0x7f6c140098c0 comp rx=0 tx=0).stop 2023-10-08T00:16:27.961 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.950+0000 7f6c2c37a700 1 -- 172.21.15.99:0/1865099600 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c2415a680 msgr2=0x7f6c240c8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:27.961 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.950+0000 7f6c2c37a700 1 --2- 172.21.15.99:0/1865099600 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c2415a680 0x7f6c240c8bc0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c00c4c0 tx=0x7f6c1c007f50 comp rx=0 tx=0).stop 2023-10-08T00:16:27.961 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.950+0000 7f6c2c37a700 1 -- 172.21.15.99:0/1865099600 shutdown_connections 2023-10-08T00:16:27.962 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.950+0000 7f6c2c37a700 1 --2- 172.21.15.99:0/1865099600 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6c10066f30 0x7f6c100693e0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:27.962 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.950+0000 7f6c2c37a700 1 --2- 172.21.15.99:0/1865099600 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c240d1c30 0x7f6c24155b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:27.962 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.950+0000 7f6c2c37a700 1 --2- 172.21.15.99:0/1865099600 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c2415a680 0x7f6c240c8bc0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:27.962 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.950+0000 7f6c2c37a700 1 --2- 172.21.15.99:0/1865099600 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c2415a2b0 0x7f6c2416b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:27.962 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.950+0000 7f6c2c37a700 1 -- 172.21.15.99:0/1865099600 >> 172.21.15.99:0/1865099600 conn(0x7f6c24067e40 msgr2=0x7f6c240692d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:27.962 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.950+0000 7f6c2c37a700 1 -- 172.21.15.99:0/1865099600 shutdown_connections 2023-10-08T00:16:27.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:27.950+0000 7f6c2c37a700 1 -- 172.21.15.99:0/1865099600 wait complete. 2023-10-08T00:16:27.964 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T00:16:28.078 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.077+0000 7fa260792700 1 Processor -- start 2023-10-08T00:16:28.078 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.077+0000 7fa260792700 1 -- start start 2023-10-08T00:16:28.080 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.078+0000 7fa260792700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa258157870 0x7fa25815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:28.081 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.078+0000 7fa260792700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa25815cbb0 con 0x7fa258157c40 2023-10-08T00:16:28.081 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.078+0000 7fa260792700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa25815bfa0 con 0x7fa25815a620 2023-10-08T00:16:28.081 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.078+0000 7fa260792700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa25815c280 con 0x7fa258157870 2023-10-08T00:16:28.081 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.079+0000 7fa25ed2f700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fa258157c40 0x7fa258158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50682/0 (socket says 172.21.15.99:50682) 2023-10-08T00:16:28.081 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.079+0000 7fa25ed2f700 1 -- 172.21.15.99:0/518995495 learned_addr learned my addr 172.21.15.99:0/518995495 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:28.081 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.079+0000 7fa25e52e700 1 --2- 172.21.15.99:0/518995495 >> v2:172.21.15.191:3300/0 conn(0x7fa258157870 0x7fa25815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:28.081 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.079+0000 7fa25d52c700 1 -- 172.21.15.99:0/518995495 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 376613492 0 0) 0x7fa25815cbb0 con 0x7fa258157c40 2023-10-08T00:16:28.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.079+0000 7fa25d52c700 1 -- 172.21.15.99:0/518995495 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa244003570 con 0x7fa258157c40 2023-10-08T00:16:28.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.079+0000 7fa25e52e700 1 -- 172.21.15.99:0/518995495 >> v1:172.21.15.191:6789/0 conn(0x7fa25815a620 legacy=0x7fa25815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:28.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.079+0000 7fa25d52c700 1 -- 172.21.15.99:0/518995495 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2292445377 0 0) 0x7fa244003570 con 0x7fa258157c40 2023-10-08T00:16:28.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.079+0000 7fa25e52e700 1 -- 172.21.15.99:0/518995495 >> v1:172.21.15.99:6789/0 conn(0x7fa258157c40 legacy=0x7fa258158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:28.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.079+0000 7fa25e52e700 1 -- 172.21.15.99:0/518995495 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa25815c560 con 0x7fa258157870 2023-10-08T00:16:28.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.079+0000 7fa25e52e700 1 --2- 172.21.15.99:0/518995495 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa258157870 0x7fa25815b6a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa2540098e0 tx=0x7fa2540091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43551ba983c30729 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:28.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.079+0000 7fa25d52c700 1 -- 172.21.15.99:0/518995495 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2500017f0 con 0x7fa258157870 2023-10-08T00:16:28.083 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.079+0000 7fa25d52c700 1 -- 172.21.15.99:0/518995495 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa254011420 con 0x7fa258157870 2023-10-08T00:16:28.083 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.079+0000 7fa25d52c700 1 -- 172.21.15.99:0/518995495 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2540117c0 con 0x7fa258157870 2023-10-08T00:16:28.083 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.080+0000 7fa260792700 1 -- 172.21.15.99:0/518995495 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa258157870 msgr2=0x7fa25815b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:28.083 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.080+0000 7fa260792700 1 --2- 172.21.15.99:0/518995495 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa258157870 0x7fa25815b6a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa2540098e0 tx=0x7fa2540091a0 comp rx=0 tx=0).stop 2023-10-08T00:16:28.083 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.080+0000 7fa260792700 1 -- 172.21.15.99:0/518995495 shutdown_connections 2023-10-08T00:16:28.083 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.080+0000 7fa260792700 1 --2- 172.21.15.99:0/518995495 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa258157870 0x7fa25815b6a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:28.083 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.080+0000 7fa260792700 1 -- 172.21.15.99:0/518995495 >> 172.21.15.99:0/518995495 conn(0x7fa258068ce0 msgr2=0x7fa2580690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:28.083 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.080+0000 7fa260792700 1 -- 172.21.15.99:0/518995495 shutdown_connections 2023-10-08T00:16:28.084 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.080+0000 7fa260792700 1 -- 172.21.15.99:0/518995495 wait complete. 2023-10-08T00:16:28.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.081+0000 7fa260792700 1 Processor -- start 2023-10-08T00:16:28.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.081+0000 7fa260792700 1 -- start start 2023-10-08T00:16:28.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.082+0000 7fa260792700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa258157870 0x7fa25815f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:28.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.082+0000 7fa260792700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa258157c40 0x7fa258154240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:28.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.082+0000 7fa260792700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa25815a620 0x7fa25816ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:28.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.082+0000 7fa260792700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2581601e0 con 0x7fa258157870 2023-10-08T00:16:28.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.082+0000 7fa260792700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2581604c0 con 0x7fa258157c40 2023-10-08T00:16:28.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.082+0000 7fa260792700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2581607a0 con 0x7fa25815a620 2023-10-08T00:16:28.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.082+0000 7fa25e52e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa258157870 0x7fa25815f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:28.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.082+0000 7fa25e52e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa258157870 0x7fa25815f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37140/0 (socket says 172.21.15.99:37140) 2023-10-08T00:16:28.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.082+0000 7fa25e52e700 1 -- 172.21.15.99:0/1659658926 learned_addr learned my addr 172.21.15.99:0/1659658926 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:28.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.082+0000 7fa25ed2f700 1 --2- 172.21.15.99:0/1659658926 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa25815a620 0x7fa25816ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:28.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.082+0000 7fa25dd2d700 1 --2- 172.21.15.99:0/1659658926 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa258157c40 0x7fa258154240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:28.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.082+0000 7fa25e52e700 1 -- 172.21.15.99:0/1659658926 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa25815a620 msgr2=0x7fa25816ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:28.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.082+0000 7fa25e52e700 1 --2- 172.21.15.99:0/1659658926 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa25815a620 0x7fa25816ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:28.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.082+0000 7fa25e52e700 1 -- 172.21.15.99:0/1659658926 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa258157c40 msgr2=0x7fa258154240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:28.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.082+0000 7fa25e52e700 1 --2- 172.21.15.99:0/1659658926 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa258157c40 0x7fa258154240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:28.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.082+0000 7fa25e52e700 1 -- 172.21.15.99:0/1659658926 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa254009420 con 0x7fa258157870 2023-10-08T00:16:28.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.082+0000 7fa25ed2f700 1 --2- 172.21.15.99:0/1659658926 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa25815a620 0x7fa25816ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:28.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.082+0000 7fa25e52e700 1 --2- 172.21.15.99:0/1659658926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa258157870 0x7fa25815f760 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fa25806b530 tx=0x7fa254016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:28.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.083+0000 7fa24e7fc700 1 -- 172.21.15.99:0/1659658926 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25401cb60 con 0x7fa258157870 2023-10-08T00:16:28.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.083+0000 7fa24e7fc700 1 -- 172.21.15.99:0/1659658926 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa25401ccc0 con 0x7fa258157870 2023-10-08T00:16:28.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.083+0000 7fa260792700 1 -- 172.21.15.99:0/1659658926 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa258160a80 con 0x7fa258157870 2023-10-08T00:16:28.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.083+0000 7fa24e7fc700 1 -- 172.21.15.99:0/1659658926 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25401a6a0 con 0x7fa258157870 2023-10-08T00:16:28.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.083+0000 7fa260792700 1 -- 172.21.15.99:0/1659658926 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa258154e00 con 0x7fa258157870 2023-10-08T00:16:28.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.084+0000 7fa260792700 1 -- 172.21.15.99:0/1659658926 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa25804c3c0 con 0x7fa258157870 2023-10-08T00:16:28.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.084+0000 7fa24e7fc700 1 -- 172.21.15.99:0/1659658926 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa25401aa90 con 0x7fa258157870 2023-10-08T00:16:28.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.084+0000 7fa24e7fc700 1 --2- 172.21.15.99:0/1659658926 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa23c066f30 0x7fa23c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:28.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.084+0000 7fa24e7fc700 1 -- 172.21.15.99:0/1659658926 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa254085e10 con 0x7fa258157870 2023-10-08T00:16:28.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.085+0000 7fa25dd2d700 1 --2- 172.21.15.99:0/1659658926 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa23c066f30 0x7fa23c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:28.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.085+0000 7fa25dd2d700 1 --2- 172.21.15.99:0/1659658926 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa23c066f30 0x7fa23c0693e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa25815aa20 tx=0x7fa248007ec0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:28.093 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.089+0000 7fa24e7fc700 1 -- 172.21.15.99:0/1659658926 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa254053080 con 0x7fa258157870 2023-10-08T00:16:28.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.316+0000 7fa260792700 1 -- 172.21.15.99:0/1659658926 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa25815c840 con 0x7fa258157870 2023-10-08T00:16:28.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.316+0000 7fa24e7fc700 1 -- 172.21.15.99:0/1659658926 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v56) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa2540566a0 con 0x7fa258157870 2023-10-08T00:16:28.317 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T00:16:28.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.318+0000 7fa260792700 1 -- 172.21.15.99:0/1659658926 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa23c066f30 msgr2=0x7fa23c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:28.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.318+0000 7fa260792700 1 --2- 172.21.15.99:0/1659658926 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa23c066f30 0x7fa23c0693e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa25815aa20 tx=0x7fa248007ec0 comp rx=0 tx=0).stop 2023-10-08T00:16:28.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.318+0000 7fa260792700 1 -- 172.21.15.99:0/1659658926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa258157870 msgr2=0x7fa25815f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:28.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.318+0000 7fa260792700 1 --2- 172.21.15.99:0/1659658926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa258157870 0x7fa25815f760 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fa25806b530 tx=0x7fa254016000 comp rx=0 tx=0).stop 2023-10-08T00:16:28.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.318+0000 7fa260792700 1 -- 172.21.15.99:0/1659658926 shutdown_connections 2023-10-08T00:16:28.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.318+0000 7fa260792700 1 --2- 172.21.15.99:0/1659658926 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa23c066f30 0x7fa23c0693e0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:28.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.318+0000 7fa260792700 1 --2- 172.21.15.99:0/1659658926 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa25815a620 0x7fa25816ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:28.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.318+0000 7fa260792700 1 --2- 172.21.15.99:0/1659658926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa258157870 0x7fa25815f760 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:28.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.318+0000 7fa260792700 1 --2- 172.21.15.99:0/1659658926 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa258157c40 0x7fa258154240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:28.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.318+0000 7fa260792700 1 -- 172.21.15.99:0/1659658926 >> 172.21.15.99:0/1659658926 conn(0x7fa258068ce0 msgr2=0x7fa2580b2620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:28.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.318+0000 7fa260792700 1 -- 172.21.15.99:0/1659658926 shutdown_connections 2023-10-08T00:16:28.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:28.318+0000 7fa260792700 1 -- 172.21.15.99:0/1659658926 wait complete. 2023-10-08T00:16:28.331 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T00:16:28.331 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T00:16:28.331 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi099.front.sepia.ceph.com 2023-10-08T00:16:28.331 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T00:16:28.331 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T00:16:28.331 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T00:16:28.331 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T00:16:28.331 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T00:16:28.331 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T00:16:28.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:16:28.332 DEBUG:teuthology.orchestra.run.smithi099:> ip addr 2023-10-08T00:16:28.347 INFO:teuthology.orchestra.run.smithi099.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T00:16:28.347 INFO:teuthology.orchestra.run.smithi099.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T00:16:28.347 INFO:teuthology.orchestra.run.smithi099.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T00:16:28.347 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:16:28.347 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 ::1/128 scope host 2023-10-08T00:16:28.347 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:16:28.347 INFO:teuthology.orchestra.run.smithi099.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:16:28.347 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:10 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:16:28.348 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f0 2023-10-08T00:16:28.348 INFO:teuthology.orchestra.run.smithi099.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:16:28.348 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:11 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:16:28.348 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f1 2023-10-08T00:16:28.348 INFO:teuthology.orchestra.run.smithi099.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:16:28.348 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:02 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:16:28.348 INFO:teuthology.orchestra.run.smithi099.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T00:16:28.348 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:03 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:16:28.348 INFO:teuthology.orchestra.run.smithi099.stdout: inet 172.21.15.99/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T00:16:28.348 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft 42201sec preferred_lft 42201sec 2023-10-08T00:16:28.348 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::ec4:7aff:fe8f:d803/64 scope link 2023-10-08T00:16:28.348 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:16:28.349 INFO:teuthology.orchestra.run.smithi099.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T00:16:28.349 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 22:2c:1c:f6:42:e1 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:16:28.349 INFO:teuthology.orchestra.run.smithi099.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T00:16:28.349 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:16:28.349 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::600d:26ff:fede:5eec/64 scope link 2023-10-08T00:16:28.349 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:16:28.349 INFO:teuthology.orchestra.run.smithi099.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T00:16:28.349 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether ae:2f:b5:94:69:ba brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T00:16:28.349 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::ac2f:b5ff:fe94:69ba/64 scope link 2023-10-08T00:16:28.349 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:16:28.350 INFO:teuthology.orchestra.run.smithi099.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T00:16:28.350 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 22:2c:1c:f6:42:e1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T00:16:28.350 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::202c:1cff:fef6:42e1/64 scope link 2023-10-08T00:16:28.350 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:16:28.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:16:28.350 DEBUG:teuthology.orchestra.run.smithi099:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T00:16:28.413 INFO:teuthology.orchestra.run.smithi099.stdout:1 2023-10-08T00:16:28.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:16:28.414 DEBUG:teuthology.orchestra.run.smithi099:> route 2023-10-08T00:16:28.470 INFO:teuthology.orchestra.run.smithi099.stdout:Kernel IP routing table 2023-10-08T00:16:28.470 INFO:teuthology.orchestra.run.smithi099.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T00:16:28.470 INFO:teuthology.orchestra.run.smithi099.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T00:16:28.470 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T00:16:28.470 INFO:teuthology.orchestra.run.smithi099.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T00:16:28.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:16:28.471 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2023-10-08T00:16:28.471 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2023-10-08T00:16:28.471 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T00:16:28.471 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T00:16:28.471 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T00:16:28.471 DEBUG:teuthology.orchestra.run.smithi099:> ') 2023-10-08T00:16:28.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:16:28.570 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2023-10-08T00:16:28.584 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T00:16:28.584 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T00:16:28.584 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T00:16:28.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:16:28.585 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set brx.0 up 2023-10-08T00:16:28.648 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:16:28.648 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T00:16:28.703 INFO:teuthology.orchestra.run.smithi099.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T00:16:28.703 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:16:28.703 DEBUG:teuthology.orchestra.run.smithi099:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T00:16:28.757 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-08T00:16:28.757 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T00:16:28.757 DEBUG:teuthology.orchestra.run.smithi099:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:16:28.836 INFO:teuthology.orchestra.run.smithi099.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:16:28.836 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:16:28.855 INFO:teuthology.orchestra.run.smithi099.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:16:28.855 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:16:28.856 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: resolved to: "172.21.15.99:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T00:16:28.856 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: trying mount with new device syntax: 0@06962cdb-9a6f-417f-82fd-9e4e3dedbccf.cephfs=/ 2023-10-08T00:16:28.856 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.99:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T00:16:28.856 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T00:16:28.856 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T00:16:28.856 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T00:16:28.877 INFO:teuthology.orchestra.run.smithi099.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:16:28.877 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:16:28.877 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: resolved to: "172.21.15.99:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T00:16:28.877 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: trying mount with new device syntax: 0@06962cdb-9a6f-417f-82fd-9e4e3dedbccf.cephfs=/ 2023-10-08T00:16:28.877 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.99:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T00:16:28.877 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T00:16:28.877 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T00:16:28.877 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T00:16:28.878 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: resolved to: "172.21.15.99:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T00:16:28.878 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.99:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T00:16:28.878 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=06962cdb-9a6f-417f-82fd-9e4e3dedbccf" will pass to kernel 2023-10-08T00:16:28.880 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T00:16:28.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:16:28.880 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T00:16:28.945 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T00:16:29.100 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.099+0000 7f12ee88a700 1 Processor -- start 2023-10-08T00:16:29.100 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.099+0000 7f12ee88a700 1 -- start start 2023-10-08T00:16:29.102 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.100+0000 7f12ee88a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f12e80c75c0 0x7f12e80c8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:29.102 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.100+0000 7f12ee88a700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f12e80abeb0 0x7f12e80c8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:29.102 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.100+0000 7f12ee88a700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12e80c9cb0 con 0x7f12e80abae0 2023-10-08T00:16:29.102 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.100+0000 7f12ee88a700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12e80c8e80 con 0x7f12e80abeb0 2023-10-08T00:16:29.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.100+0000 7f12ee88a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12e80c90a0 con 0x7f12e80c75c0 2023-10-08T00:16:29.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.100+0000 7f12ece27700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f12e80abeb0 0x7f12e80c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:29.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.100+0000 7f12e7fff700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f12e80abae0 0x7f12e80c6eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50690/0 (socket says 172.21.15.99:50690) 2023-10-08T00:16:29.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.100+0000 7f12ece27700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f12e80abeb0 0x7f12e80c8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37144/0 (socket says 172.21.15.99:37144) 2023-10-08T00:16:29.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.100+0000 7f12e7fff700 1 -- 172.21.15.99:0/1893648708 learned_addr learned my addr 172.21.15.99:0/1893648708 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:29.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.100+0000 7f12e77fe700 1 --2- 172.21.15.99:0/1893648708 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f12e80c75c0 0x7f12e80c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:29.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.100+0000 7f12ece27700 1 -- 172.21.15.99:0/1893648708 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f12e80c75c0 msgr2=0x7f12e80c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:29.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.101+0000 7f12ece27700 1 --2- 172.21.15.99:0/1893648708 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f12e80c75c0 0x7f12e80c8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:29.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.101+0000 7f12ece27700 1 -- 172.21.15.99:0/1893648708 >> v1:172.21.15.99:6789/0 conn(0x7f12e80abae0 legacy=0x7f12e80c6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:29.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.101+0000 7f12ece27700 1 -- 172.21.15.99:0/1893648708 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e80c9380 con 0x7f12e80abeb0 2023-10-08T00:16:29.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.101+0000 7f12e6ffd700 1 -- 172.21.15.99:0/1893648708 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2110072393 0 0) 0x7f12e80c9cb0 con 0x7f12e80abae0 2023-10-08T00:16:29.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.101+0000 7f12e77fe700 1 --2- 172.21.15.99:0/1893648708 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f12e80c75c0 0x7f12e80c8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:16:29.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.101+0000 7f12ece27700 1 --2- 172.21.15.99:0/1893648708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12e80abeb0 0x7f12e80c8940 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f12e0009e60 tx=0x7f12e0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c61011fd7a01f33 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:29.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.101+0000 7f12e6ffd700 1 -- 172.21.15.99:0/1893648708 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e000e070 con 0x7f12e80abeb0 2023-10-08T00:16:29.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.101+0000 7f12e6ffd700 1 -- 172.21.15.99:0/1893648708 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12e000bc30 con 0x7f12e80abeb0 2023-10-08T00:16:29.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.101+0000 7f12e6ffd700 1 -- 172.21.15.99:0/1893648708 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e0012430 con 0x7f12e80abeb0 2023-10-08T00:16:29.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.101+0000 7f12ee88a700 1 -- 172.21.15.99:0/1893648708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12e80abeb0 msgr2=0x7f12e80c8940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:29.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.101+0000 7f12ee88a700 1 --2- 172.21.15.99:0/1893648708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12e80abeb0 0x7f12e80c8940 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f12e0009e60 tx=0x7f12e0009780 comp rx=0 tx=0).stop 2023-10-08T00:16:29.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.101+0000 7f12ee88a700 1 -- 172.21.15.99:0/1893648708 shutdown_connections 2023-10-08T00:16:29.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.101+0000 7f12ee88a700 1 --2- 172.21.15.99:0/1893648708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12e80abeb0 0x7f12e80c8940 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:29.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.101+0000 7f12ee88a700 1 --2- 172.21.15.99:0/1893648708 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f12e80c75c0 0x7f12e80c8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:29.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.101+0000 7f12ee88a700 1 -- 172.21.15.99:0/1893648708 >> 172.21.15.99:0/1893648708 conn(0x7f12e8067660 msgr2=0x7f12e8067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:29.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.101+0000 7f12ee88a700 1 -- 172.21.15.99:0/1893648708 shutdown_connections 2023-10-08T00:16:29.104 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.101+0000 7f12ee88a700 1 -- 172.21.15.99:0/1893648708 wait complete. 2023-10-08T00:16:29.107 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.102+0000 7f12ee88a700 1 Processor -- start 2023-10-08T00:16:29.107 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.103+0000 7f12ee88a700 1 -- start start 2023-10-08T00:16:29.107 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.103+0000 7f12ee88a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f12e80abae0 0x7f12e80c3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:29.107 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.103+0000 7f12ee88a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12e80abeb0 0x7f12e8168620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:29.107 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.103+0000 7f12ee88a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12e80c75c0 0x7f12e8168da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:29.107 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.103+0000 7f12ee88a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e80c3780 con 0x7f12e80c75c0 2023-10-08T00:16:29.107 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.103+0000 7f12ee88a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e8168b60 con 0x7f12e80abeb0 2023-10-08T00:16:29.107 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.103+0000 7f12ee88a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12e80a7070 con 0x7f12e80abae0 2023-10-08T00:16:29.107 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.103+0000 7f12ece27700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12e80c75c0 0x7f12e8168da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:29.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.103+0000 7f12ece27700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12e80c75c0 0x7f12e8168da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37154/0 (socket says 172.21.15.99:37154) 2023-10-08T00:16:29.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.103+0000 7f12ece27700 1 -- 172.21.15.99:0/1810996895 learned_addr learned my addr 172.21.15.99:0/1810996895 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:29.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.103+0000 7f12e7fff700 1 --2- 172.21.15.99:0/1810996895 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f12e80abae0 0x7f12e80c3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:29.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.103+0000 7f12e77fe700 1 --2- 172.21.15.99:0/1810996895 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12e80abeb0 0x7f12e8168620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:29.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.104+0000 7f12ece27700 1 -- 172.21.15.99:0/1810996895 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f12e80abae0 msgr2=0x7f12e80c3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:29.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.104+0000 7f12ece27700 1 --2- 172.21.15.99:0/1810996895 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f12e80abae0 0x7f12e80c3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:29.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.104+0000 7f12ece27700 1 -- 172.21.15.99:0/1810996895 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12e80abeb0 msgr2=0x7f12e8168620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:29.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.104+0000 7f12ece27700 1 --2- 172.21.15.99:0/1810996895 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12e80abeb0 0x7f12e8168620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:29.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.104+0000 7f12ece27700 1 -- 172.21.15.99:0/1810996895 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e0009a20 con 0x7f12e80c75c0 2023-10-08T00:16:29.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.104+0000 7f12ece27700 1 --2- 172.21.15.99:0/1810996895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12e80c75c0 0x7f12e8168da0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f12e806ba00 tx=0x7f12e0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:29.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.104+0000 7f12cffff700 1 -- 172.21.15.99:0/1810996895 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e0012e00 con 0x7f12e80c75c0 2023-10-08T00:16:29.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.104+0000 7f12cffff700 1 -- 172.21.15.99:0/1810996895 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12e001d630 con 0x7f12e80c75c0 2023-10-08T00:16:29.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.104+0000 7f12ee88a700 1 -- 172.21.15.99:0/1810996895 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12e80a7350 con 0x7f12e80c75c0 2023-10-08T00:16:29.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.104+0000 7f12cffff700 1 -- 172.21.15.99:0/1810996895 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e001d9d0 con 0x7f12e80c75c0 2023-10-08T00:16:29.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.104+0000 7f12ee88a700 1 -- 172.21.15.99:0/1810996895 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12e80a7910 con 0x7f12e80c75c0 2023-10-08T00:16:29.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.105+0000 7f12ee88a700 1 -- 172.21.15.99:0/1810996895 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12e804c3c0 con 0x7f12e80c75c0 2023-10-08T00:16:29.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.105+0000 7f12cffff700 1 -- 172.21.15.99:0/1810996895 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f12e0024070 con 0x7f12e80c75c0 2023-10-08T00:16:29.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.106+0000 7f12cffff700 1 --2- 172.21.15.99:0/1810996895 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f12d4066f30 0x7f12d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:29.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.106+0000 7f12cffff700 1 -- 172.21.15.99:0/1810996895 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f12e0086d90 con 0x7f12e80c75c0 2023-10-08T00:16:29.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.106+0000 7f12e7fff700 1 --2- 172.21.15.99:0/1810996895 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f12d4066f30 0x7f12d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:29.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.106+0000 7f12e7fff700 1 --2- 172.21.15.99:0/1810996895 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f12d4066f30 0x7f12d40693e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f12e80ab860 tx=0x7f12d8007d30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:29.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.110+0000 7f12cffff700 1 -- 172.21.15.99:0/1810996895 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f12e0054000 con 0x7f12e80c75c0 2023-10-08T00:16:29.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.336+0000 7f12ee88a700 1 -- 172.21.15.99:0/1810996895 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f12e806a0d0 con 0x7f12e80c75c0 2023-10-08T00:16:29.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.336+0000 7f12cffff700 1 -- 172.21.15.99:0/1810996895 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v56) v1 ==== 53+0+73 (secure 0 0 0) 0x7f12e002c070 con 0x7f12e80c75c0 2023-10-08T00:16:29.337 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T00:16:29.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.338+0000 7f12ee88a700 1 -- 172.21.15.99:0/1810996895 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f12d4066f30 msgr2=0x7f12d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:29.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.338+0000 7f12ee88a700 1 --2- 172.21.15.99:0/1810996895 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f12d4066f30 0x7f12d40693e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f12e80ab860 tx=0x7f12d8007d30 comp rx=0 tx=0).stop 2023-10-08T00:16:29.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.338+0000 7f12ee88a700 1 -- 172.21.15.99:0/1810996895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12e80c75c0 msgr2=0x7f12e8168da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:29.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.338+0000 7f12ee88a700 1 --2- 172.21.15.99:0/1810996895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12e80c75c0 0x7f12e8168da0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f12e806ba00 tx=0x7f12e0019000 comp rx=0 tx=0).stop 2023-10-08T00:16:29.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.338+0000 7f12ee88a700 1 -- 172.21.15.99:0/1810996895 shutdown_connections 2023-10-08T00:16:29.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.338+0000 7f12ee88a700 1 --2- 172.21.15.99:0/1810996895 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f12d4066f30 0x7f12d40693e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:29.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.338+0000 7f12ee88a700 1 --2- 172.21.15.99:0/1810996895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12e80c75c0 0x7f12e8168da0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:29.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.338+0000 7f12ee88a700 1 --2- 172.21.15.99:0/1810996895 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f12e80abae0 0x7f12e80c3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:29.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.338+0000 7f12ee88a700 1 --2- 172.21.15.99:0/1810996895 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12e80abeb0 0x7f12e8168620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:29.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.338+0000 7f12ee88a700 1 -- 172.21.15.99:0/1810996895 >> 172.21.15.99:0/1810996895 conn(0x7f12e8067660 msgr2=0x7f12e80adc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:29.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.338+0000 7f12ee88a700 1 -- 172.21.15.99:0/1810996895 shutdown_connections 2023-10-08T00:16:29.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:29.338+0000 7f12ee88a700 1 -- 172.21.15.99:0/1810996895 wait complete. 2023-10-08T00:16:29.351 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T00:16:29.351 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T00:16:29.351 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi191.front.sepia.ceph.com 2023-10-08T00:16:29.351 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-08T00:16:29.351 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-08T00:16:29.351 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T00:16:29.351 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T00:16:29.351 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T00:16:29.351 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T00:16:29.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:16:29.352 DEBUG:teuthology.orchestra.run.smithi191:> ip addr 2023-10-08T00:16:29.370 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T00:16:29.371 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T00:16:29.371 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T00:16:29.371 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:16:29.371 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2023-10-08T00:16:29.371 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:16:29.371 INFO:teuthology.orchestra.run.smithi191.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:16:29.371 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f6 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:16:29.371 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f0 2023-10-08T00:16:29.371 INFO:teuthology.orchestra.run.smithi191.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:16:29.372 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f7 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:16:29.372 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f1 2023-10-08T00:16:29.372 INFO:teuthology.orchestra.run.smithi191.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:16:29.372 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:10 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:16:29.372 INFO:teuthology.orchestra.run.smithi191.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T00:16:29.372 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:11 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:16:29.372 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.21.15.191/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T00:16:29.372 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft 42202sec preferred_lft 42202sec 2023-10-08T00:16:29.372 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec4:7aff:fe88:7211/64 scope link 2023-10-08T00:16:29.372 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:16:29.372 INFO:teuthology.orchestra.run.smithi191.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T00:16:29.372 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 9a:bc:39:72:87:3d brd ff:ff:ff:ff:ff:ff 2023-10-08T00:16:29.373 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T00:16:29.373 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:16:29.373 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec77:a7ff:fea4:fbb/64 scope link 2023-10-08T00:16:29.373 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:16:29.373 INFO:teuthology.orchestra.run.smithi191.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T00:16:29.373 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether ea:1c:d9:46:05:b3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-08T00:16:29.373 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::e81c:d9ff:fe46:5b3/64 scope link 2023-10-08T00:16:29.373 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:16:29.373 INFO:teuthology.orchestra.run.smithi191.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T00:16:29.373 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 9a:bc:39:72:87:3d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-08T00:16:29.373 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::98bc:39ff:fe72:873d/64 scope link 2023-10-08T00:16:29.373 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:16:29.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:16:29.374 DEBUG:teuthology.orchestra.run.smithi191:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T00:16:29.438 INFO:teuthology.orchestra.run.smithi191.stdout:1 2023-10-08T00:16:29.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:16:29.439 DEBUG:teuthology.orchestra.run.smithi191:> route 2023-10-08T00:16:29.497 INFO:teuthology.orchestra.run.smithi191.stdout:Kernel IP routing table 2023-10-08T00:16:29.497 INFO:teuthology.orchestra.run.smithi191.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T00:16:29.498 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T00:16:29.498 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T00:16:29.498 INFO:teuthology.orchestra.run.smithi191.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T00:16:29.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:16:29.498 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2023-10-08T00:16:29.498 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2023-10-08T00:16:29.498 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T00:16:29.499 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T00:16:29.499 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T00:16:29.499 DEBUG:teuthology.orchestra.run.smithi191:> ') 2023-10-08T00:16:29.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:16:29.603 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2023-10-08T00:16:29.617 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-08T00:16:29.618 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-08T00:16:29.618 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T00:16:29.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:16:29.619 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set brx.0 up 2023-10-08T00:16:29.683 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:16:29.683 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-08T00:16:29.738 INFO:teuthology.orchestra.run.smithi191.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-08T00:16:29.738 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:16:29.738 DEBUG:teuthology.orchestra.run.smithi191:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-08T00:16:29.792 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-08T00:16:29.792 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T00:16:29.792 DEBUG:teuthology.orchestra.run.smithi191:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:16:29.872 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:16:29.966 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:16:29.968 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:16:29.968 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:16:29.968 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.99:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T00:16:29.968 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@06962cdb-9a6f-417f-82fd-9e4e3dedbccf.cephfs=/ 2023-10-08T00:16:29.968 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.99:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T00:16:29.968 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T00:16:29.968 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T00:16:29.969 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T00:16:29.969 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:16:29.969 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:16:29.969 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.99:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T00:16:29.969 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@06962cdb-9a6f-417f-82fd-9e4e3dedbccf.cephfs=/ 2023-10-08T00:16:29.969 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.99:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T00:16:29.969 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T00:16:29.969 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T00:16:29.969 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T00:16:29.969 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.99:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T00:16:29.969 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.99:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T00:16:29.969 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=06962cdb-9a6f-417f-82fd-9e4e3dedbccf" will pass to kernel 2023-10-08T00:16:29.970 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T00:16:29.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:16:29.970 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-08T00:16:30.033 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:16:30.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.148+0000 7fed1359e700 1 Processor -- start 2023-10-08T00:16:30.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.148+0000 7fed1359e700 1 -- start start 2023-10-08T00:16:30.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.149+0000 7fed1359e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fed140cd6e0 0x7fed140cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:30.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.149+0000 7fed1359e700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fed1415cbd0 0x7fed1415d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:30.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.149+0000 7fed1359e700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed140cb7c0 con 0x7fed140cdab0 2023-10-08T00:16:30.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.149+0000 7fed1359e700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed1415e850 con 0x7fed1415cbd0 2023-10-08T00:16:30.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.149+0000 7fed1359e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed140cd030 con 0x7fed140cd6e0 2023-10-08T00:16:30.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.149+0000 7fed12d9d700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fed1415cbd0 0x7fed1415d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:30.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.149+0000 7fed0bfff700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fed140cdab0 0x7fed1415d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50702/0 (socket says 172.21.15.99:50702) 2023-10-08T00:16:30.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.149+0000 7fed0bfff700 1 -- 172.21.15.99:0/156116218 learned_addr learned my addr 172.21.15.99:0/156116218 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:30.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.149+0000 7fed1259c700 1 --2- 172.21.15.99:0/156116218 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fed140cd6e0 0x7fed140cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:30.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.149+0000 7fed12d9d700 1 -- 172.21.15.99:0/156116218 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fed140cd6e0 msgr2=0x7fed140cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:30.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.149+0000 7fed12d9d700 1 --2- 172.21.15.99:0/156116218 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fed140cd6e0 0x7fed140cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:30.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.149+0000 7fed12d9d700 1 -- 172.21.15.99:0/156116218 >> v1:172.21.15.99:6789/0 conn(0x7fed140cdab0 legacy=0x7fed1415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:30.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.149+0000 7fed12d9d700 1 -- 172.21.15.99:0/156116218 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed1415e290 con 0x7fed1415cbd0 2023-10-08T00:16:30.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.149+0000 7fed1259c700 1 --2- 172.21.15.99:0/156116218 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fed140cd6e0 0x7fed140cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:30.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.150+0000 7fed12d9d700 1 --2- 172.21.15.99:0/156116218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed1415cbd0 0x7fed1415d020 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fed0c009e60 tx=0x7fed0c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f07477c61af8ba04 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:30.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.150+0000 7fed11d9b700 1 -- 172.21.15.99:0/156116218 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0c00e070 con 0x7fed1415cbd0 2023-10-08T00:16:30.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.150+0000 7fed11d9b700 1 -- 172.21.15.99:0/156116218 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed0c00bc30 con 0x7fed1415cbd0 2023-10-08T00:16:30.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.150+0000 7fed11d9b700 1 -- 172.21.15.99:0/156116218 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0c012430 con 0x7fed1415cbd0 2023-10-08T00:16:30.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.150+0000 7fed1359e700 1 -- 172.21.15.99:0/156116218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed1415cbd0 msgr2=0x7fed1415d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:30.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.150+0000 7fed1359e700 1 --2- 172.21.15.99:0/156116218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed1415cbd0 0x7fed1415d020 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fed0c009e60 tx=0x7fed0c009780 comp rx=0 tx=0).stop 2023-10-08T00:16:30.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.150+0000 7fed1359e700 1 -- 172.21.15.99:0/156116218 shutdown_connections 2023-10-08T00:16:30.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.150+0000 7fed1359e700 1 --2- 172.21.15.99:0/156116218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed1415cbd0 0x7fed1415d020 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:30.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.150+0000 7fed1359e700 1 --2- 172.21.15.99:0/156116218 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fed140cd6e0 0x7fed140cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:30.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.150+0000 7fed1359e700 1 -- 172.21.15.99:0/156116218 >> 172.21.15.99:0/156116218 conn(0x7fed14067660 msgr2=0x7fed14067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:30.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.150+0000 7fed1359e700 1 -- 172.21.15.99:0/156116218 shutdown_connections 2023-10-08T00:16:30.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.150+0000 7fed1359e700 1 -- 172.21.15.99:0/156116218 wait complete. 2023-10-08T00:16:30.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.151+0000 7fed1359e700 1 Processor -- start 2023-10-08T00:16:30.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.151+0000 7fed1359e700 1 -- start start 2023-10-08T00:16:30.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.152+0000 7fed1359e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fed140cd6e0 0x7fed140c92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:30.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.152+0000 7fed1359e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed140cdab0 0x7fed140b1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:30.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.152+0000 7fed1359e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fed1415cbd0 0x7fed140b24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:30.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.152+0000 7fed1359e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed1416a8e0 con 0x7fed140cdab0 2023-10-08T00:16:30.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.152+0000 7fed1359e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed1416ab60 con 0x7fed140cd6e0 2023-10-08T00:16:30.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.152+0000 7fed1359e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed1416ae40 con 0x7fed1415cbd0 2023-10-08T00:16:30.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.152+0000 7fed0bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed140cdab0 0x7fed140b1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:30.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.152+0000 7fed0bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed140cdab0 0x7fed140b1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37184/0 (socket says 172.21.15.99:37184) 2023-10-08T00:16:30.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.152+0000 7fed0bfff700 1 -- 172.21.15.99:0/909960631 learned_addr learned my addr 172.21.15.99:0/909960631 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:30.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.152+0000 7fed1259c700 1 --2- 172.21.15.99:0/909960631 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fed140cd6e0 0x7fed140c92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:30.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.152+0000 7fed12d9d700 1 --2- 172.21.15.99:0/909960631 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fed1415cbd0 0x7fed140b24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:30.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.152+0000 7fed0bfff700 1 -- 172.21.15.99:0/909960631 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fed1415cbd0 msgr2=0x7fed140b24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:30.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.152+0000 7fed0bfff700 1 --2- 172.21.15.99:0/909960631 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fed1415cbd0 0x7fed140b24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:30.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.152+0000 7fed0bfff700 1 -- 172.21.15.99:0/909960631 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fed140cd6e0 msgr2=0x7fed140c92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:30.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.152+0000 7fed0bfff700 1 --2- 172.21.15.99:0/909960631 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fed140cd6e0 0x7fed140c92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:30.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.152+0000 7fed0bfff700 1 -- 172.21.15.99:0/909960631 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed0c009a20 con 0x7fed140cdab0 2023-10-08T00:16:30.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.152+0000 7fed1259c700 1 --2- 172.21.15.99:0/909960631 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fed140cd6e0 0x7fed140c92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:16:30.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.153+0000 7fed0bfff700 1 --2- 172.21.15.99:0/909960631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed140cdab0 0x7fed140b1f90 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fecfc00c7c0 tx=0x7fecfc007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:30.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.153+0000 7fed0a7fc700 1 -- 172.21.15.99:0/909960631 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecfc005b30 con 0x7fed140cdab0 2023-10-08T00:16:30.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.153+0000 7fed0a7fc700 1 -- 172.21.15.99:0/909960631 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecfc005cb0 con 0x7fed140cdab0 2023-10-08T00:16:30.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.153+0000 7fed1359e700 1 -- 172.21.15.99:0/909960631 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed1416b120 con 0x7fed140cdab0 2023-10-08T00:16:30.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.153+0000 7fed0a7fc700 1 -- 172.21.15.99:0/909960631 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecfc004a80 con 0x7fed140cdab0 2023-10-08T00:16:30.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.153+0000 7fed1359e700 1 -- 172.21.15.99:0/909960631 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed140c97e0 con 0x7fed140cdab0 2023-10-08T00:16:30.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.154+0000 7fed1359e700 1 -- 172.21.15.99:0/909960631 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed1416b640 con 0x7fed140cdab0 2023-10-08T00:16:30.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.154+0000 7fed0a7fc700 1 -- 172.21.15.99:0/909960631 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fecfc0070f0 con 0x7fed140cdab0 2023-10-08T00:16:30.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.154+0000 7fed0a7fc700 1 --2- 172.21.15.99:0/909960631 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fecf4066f30 0x7fecf40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:30.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.154+0000 7fed0a7fc700 1 -- 172.21.15.99:0/909960631 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fecfc0839b0 con 0x7fed140cdab0 2023-10-08T00:16:30.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.154+0000 7fed1259c700 1 --2- 172.21.15.99:0/909960631 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fecf4066f30 0x7fecf40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:30.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.155+0000 7fed1259c700 1 --2- 172.21.15.99:0/909960631 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fecf4066f30 0x7fecf40693e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fed04006f00 tx=0x7fed04008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:30.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.159+0000 7fed0a7fc700 1 -- 172.21.15.99:0/909960631 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fed1416b640 con 0x7fed140cdab0 2023-10-08T00:16:30.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.387+0000 7fed1359e700 1 -- 172.21.15.99:0/909960631 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fed1416b640 con 0x7fed140cdab0 2023-10-08T00:16:30.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.387+0000 7fed0a7fc700 1 -- 172.21.15.99:0/909960631 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4605 (secure 0 0 0) 0x7fed1416b640 con 0x7fed140cdab0 2023-10-08T00:16:30.388 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:16:30.388 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":56,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5627,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/2782755379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":2782755379},{"type":"v1","addr":"172.21.15.191:6837","nonce":2782755379}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6166,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/2980541392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2980541392},{"type":"v1","addr":"172.21.15.99:6834","nonce":2980541392}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6184,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/1342206989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1342206989},{"type":"v1","addr":"172.21.15.191:6836","nonce":1342206989}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:16:21.527609+0000","modified":"2023-10-08T00:16:25.185437+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6747},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6747":{"gid":6747,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/1495315849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":1495315849},{"type":"v1","addr":"172.21.15.99:6835","nonce":1495315849}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-08T00:16:30.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.389+0000 7fed1359e700 1 -- 172.21.15.99:0/909960631 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fecf4066f30 msgr2=0x7fecf40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:30.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.389+0000 7fed1359e700 1 --2- 172.21.15.99:0/909960631 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fecf4066f30 0x7fecf40693e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fed04006f00 tx=0x7fed04008040 comp rx=0 tx=0).stop 2023-10-08T00:16:30.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.389+0000 7fed1359e700 1 -- 172.21.15.99:0/909960631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed140cdab0 msgr2=0x7fed140b1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:30.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.389+0000 7fed1359e700 1 --2- 172.21.15.99:0/909960631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed140cdab0 0x7fed140b1f90 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fecfc00c7c0 tx=0x7fecfc007f70 comp rx=0 tx=0).stop 2023-10-08T00:16:30.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.389+0000 7fed1359e700 1 -- 172.21.15.99:0/909960631 shutdown_connections 2023-10-08T00:16:30.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.389+0000 7fed1359e700 1 --2- 172.21.15.99:0/909960631 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fecf4066f30 0x7fecf40693e0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:30.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.389+0000 7fed1359e700 1 --2- 172.21.15.99:0/909960631 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fed140cd6e0 0x7fed140c92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:30.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.389+0000 7fed1359e700 1 --2- 172.21.15.99:0/909960631 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fed1415cbd0 0x7fed140b24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:30.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.389+0000 7fed1359e700 1 --2- 172.21.15.99:0/909960631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed140cdab0 0x7fed140b1f90 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:30.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.389+0000 7fed1359e700 1 -- 172.21.15.99:0/909960631 >> 172.21.15.99:0/909960631 conn(0x7fed14067660 msgr2=0x7fed14067de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:30.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.389+0000 7fed1359e700 1 -- 172.21.15.99:0/909960631 shutdown_connections 2023-10-08T00:16:30.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.389+0000 7fed1359e700 1 -- 172.21.15.99:0/909960631 wait complete. 2023-10-08T00:16:30.392 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 56 2023-10-08T00:16:30.402 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-10-08T00:16:30.535 INFO:teuthology.orchestra.run.smithi099.stdout:{"cluster_fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","whoami":0,"id":6747,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":8.3599197099999998,"mdsmap_epoch":56,"osdmap_epoch":124,"osdmap_epoch_barrier":124,"uptime":13.464600171000001} 2023-10-08T00:16:30.536 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", "fs_name": "cephfs", "id": 6747, "mdsmap_epoch": 56, "osdmap_epoch": 124, "osdmap_epoch_barrier": 124, "rank_uptime": 8.35991971, "state": "up:active", "uptime": 13.464600171, "want_state": "up:active", "whoami": 0 } 2023-10-08T00:16:30.536 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2023-10-08T00:16:30.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.653+0000 7f83133c6700 1 Processor -- start 2023-10-08T00:16:30.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.653+0000 7f83133c6700 1 -- start start 2023-10-08T00:16:30.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.654+0000 7f83133c6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f830c15d240 0x7f830c0b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:30.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.654+0000 7f83133c6700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f830c0cc940 0x7f830c0ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:30.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.654+0000 7f83133c6700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f830c0b3f40 con 0x7f830c0c59e0 2023-10-08T00:16:30.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.654+0000 7f83133c6700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f830c0cbb80 con 0x7f830c0cc940 2023-10-08T00:16:30.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.654+0000 7f83133c6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f830c0cbe60 con 0x7f830c15d240 2023-10-08T00:16:30.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.654+0000 7f8312bc5700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f830c0c59e0 0x7f830c0c5e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50712/0 (socket says 172.21.15.99:50712) 2023-10-08T00:16:30.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.654+0000 7f8312bc5700 1 -- 172.21.15.99:0/2442015724 learned_addr learned my addr 172.21.15.99:0/2442015724 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:30.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.654+0000 7f83123c4700 1 --2- 172.21.15.99:0/2442015724 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f830c15d240 0x7f830c0b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:30.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.654+0000 7f8311bc3700 1 --2- 172.21.15.99:0/2442015724 >> v2:172.21.15.191:3300/0 conn(0x7f830c0cc940 0x7f830c0ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:30.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.654+0000 7f83113c2700 1 -- 172.21.15.99:0/2442015724 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2411996557 0 0) 0x7f830c0b3f40 con 0x7f830c0c59e0 2023-10-08T00:16:30.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.654+0000 7f83113c2700 1 -- 172.21.15.99:0/2442015724 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82f4003570 con 0x7f830c0c59e0 2023-10-08T00:16:30.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.654+0000 7f83123c4700 1 -- 172.21.15.99:0/2442015724 >> v2:172.21.15.191:3300/0 conn(0x7f830c0cc940 msgr2=0x7f830c0ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:30.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.654+0000 7f83123c4700 1 --2- 172.21.15.99:0/2442015724 >> v2:172.21.15.191:3300/0 conn(0x7f830c0cc940 0x7f830c0ccdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:30.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.654+0000 7f83123c4700 1 -- 172.21.15.99:0/2442015724 >> v1:172.21.15.99:6789/0 conn(0x7f830c0c59e0 legacy=0x7f830c0c5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:30.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.654+0000 7f83123c4700 1 -- 172.21.15.99:0/2442015724 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f830c0cc140 con 0x7f830c15d240 2023-10-08T00:16:30.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.655+0000 7f83123c4700 1 --2- 172.21.15.99:0/2442015724 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f830c15d240 0x7f830c0b3a00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8304009eb0 tx=0x7f8304009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f579d076cae6f555 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:30.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.655+0000 7f83113c2700 1 -- 172.21.15.99:0/2442015724 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830400e070 con 0x7f830c15d240 2023-10-08T00:16:30.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.655+0000 7f83113c2700 1 -- 172.21.15.99:0/2442015724 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f830400bb80 con 0x7f830c15d240 2023-10-08T00:16:30.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.655+0000 7f83113c2700 1 -- 172.21.15.99:0/2442015724 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8304012430 con 0x7f830c15d240 2023-10-08T00:16:30.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.655+0000 7f83133c6700 1 -- 172.21.15.99:0/2442015724 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f830c15d240 msgr2=0x7f830c0b3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:30.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.655+0000 7f83133c6700 1 --2- 172.21.15.99:0/2442015724 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f830c15d240 0x7f830c0b3a00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8304009eb0 tx=0x7f8304009780 comp rx=0 tx=0).stop 2023-10-08T00:16:30.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.655+0000 7f83133c6700 1 -- 172.21.15.99:0/2442015724 shutdown_connections 2023-10-08T00:16:30.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.655+0000 7f83133c6700 1 --2- 172.21.15.99:0/2442015724 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f830c15d240 0x7f830c0b3a00 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:30.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.655+0000 7f83133c6700 1 --2- 172.21.15.99:0/2442015724 >> v2:172.21.15.191:3300/0 conn(0x7f830c0cc940 0x7f830c0ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:30.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.655+0000 7f83133c6700 1 -- 172.21.15.99:0/2442015724 >> 172.21.15.99:0/2442015724 conn(0x7f830c067e40 msgr2=0x7f830c0d0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:30.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.655+0000 7f83133c6700 1 -- 172.21.15.99:0/2442015724 shutdown_connections 2023-10-08T00:16:30.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.655+0000 7f83133c6700 1 -- 172.21.15.99:0/2442015724 wait complete. 2023-10-08T00:16:30.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.656+0000 7f83133c6700 1 Processor -- start 2023-10-08T00:16:30.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.656+0000 7f83133c6700 1 -- start start 2023-10-08T00:16:30.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.657+0000 7f83133c6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f830c0c59e0 0x7f830c1587d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:30.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.657+0000 7f83133c6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f830c0cc940 0x7f830c09c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:30.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.657+0000 7f83133c6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f830c15d240 0x7f830c09c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:30.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.657+0000 7f83133c6700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830c16b1f0 con 0x7f830c0c59e0 2023-10-08T00:16:30.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.657+0000 7f83133c6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830c16a670 con 0x7f830c0cc940 2023-10-08T00:16:30.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.657+0000 7f83133c6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f830c16a950 con 0x7f830c15d240 2023-10-08T00:16:30.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.657+0000 7f83123c4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f830c0c59e0 0x7f830c1587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:30.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.657+0000 7f83123c4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f830c0c59e0 0x7f830c1587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37192/0 (socket says 172.21.15.99:37192) 2023-10-08T00:16:30.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.657+0000 7f83123c4700 1 -- 172.21.15.99:0/1870658987 learned_addr learned my addr 172.21.15.99:0/1870658987 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:30.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.657+0000 7f8312bc5700 1 --2- 172.21.15.99:0/1870658987 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f830c15d240 0x7f830c09c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:30.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.657+0000 7f8311bc3700 1 --2- 172.21.15.99:0/1870658987 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f830c0cc940 0x7f830c09c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:30.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.657+0000 7f83123c4700 1 -- 172.21.15.99:0/1870658987 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f830c15d240 msgr2=0x7f830c09c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:30.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.657+0000 7f83123c4700 1 --2- 172.21.15.99:0/1870658987 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f830c15d240 0x7f830c09c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:30.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.657+0000 7f83123c4700 1 -- 172.21.15.99:0/1870658987 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f830c0cc940 msgr2=0x7f830c09c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:30.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.657+0000 7f83123c4700 1 --2- 172.21.15.99:0/1870658987 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f830c0cc940 0x7f830c09c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:30.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.657+0000 7f83123c4700 1 -- 172.21.15.99:0/1870658987 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8304009a20 con 0x7f830c0c59e0 2023-10-08T00:16:30.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.657+0000 7f8311bc3700 1 --2- 172.21.15.99:0/1870658987 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f830c0cc940 0x7f830c09c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:16:30.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.658+0000 7f83123c4700 1 --2- 172.21.15.99:0/1870658987 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f830c0c59e0 0x7f830c1587d0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f830400b610 tx=0x7f8304019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:30.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.658+0000 7f830a7fc700 1 -- 172.21.15.99:0/1870658987 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8304012e20 con 0x7f830c0c59e0 2023-10-08T00:16:30.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.658+0000 7f830a7fc700 1 -- 172.21.15.99:0/1870658987 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f830400be40 con 0x7f830c0c59e0 2023-10-08T00:16:30.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.658+0000 7f83133c6700 1 -- 172.21.15.99:0/1870658987 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f830c16ac30 con 0x7f830c0c59e0 2023-10-08T00:16:30.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.658+0000 7f830a7fc700 1 -- 172.21.15.99:0/1870658987 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830401d880 con 0x7f830c0c59e0 2023-10-08T00:16:30.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.658+0000 7f83133c6700 1 -- 172.21.15.99:0/1870658987 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f830c158d10 con 0x7f830c0c59e0 2023-10-08T00:16:30.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.659+0000 7f83133c6700 1 -- 172.21.15.99:0/1870658987 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f830c06ac60 con 0x7f830c0c59e0 2023-10-08T00:16:30.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.659+0000 7f830a7fc700 1 -- 172.21.15.99:0/1870658987 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8304024070 con 0x7f830c0c59e0 2023-10-08T00:16:30.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.659+0000 7f830a7fc700 1 --2- 172.21.15.99:0/1870658987 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f82f8066f30 0x7f82f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:30.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.660+0000 7f830a7fc700 1 -- 172.21.15.99:0/1870658987 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8304086be0 con 0x7f830c0c59e0 2023-10-08T00:16:30.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.660+0000 7f8311bc3700 1 --2- 172.21.15.99:0/1870658987 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f82f8066f30 0x7f82f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:30.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.660+0000 7f8311bc3700 1 --2- 172.21.15.99:0/1870658987 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f82f8066f30 0x7f82f80693e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8300006f00 tx=0x7f8300008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:30.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.664+0000 7f830a7fc700 1 -- 172.21.15.99:0/1870658987 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8304053e50 con 0x7f830c0c59e0 2023-10-08T00:16:30.850 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:30.849+0000 7f83133c6700 1 -- 172.21.15.99:0/1870658987 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7f830c0cc420 con 0x7f830c0c59e0 2023-10-08T00:16:31.212 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.212+0000 7f830a7fc700 1 -- 172.21.15.99:0/1870658987 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v125) v1 ==== 181+0+0 (secure 0 0 0) 0x7f830402c070 con 0x7f830c0c59e0 2023-10-08T00:16:31.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.214+0000 7f83133c6700 1 -- 172.21.15.99:0/1870658987 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f82f8066f30 msgr2=0x7f82f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:31.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.214+0000 7f83133c6700 1 --2- 172.21.15.99:0/1870658987 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f82f8066f30 0x7f82f80693e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8300006f00 tx=0x7f8300008040 comp rx=0 tx=0).stop 2023-10-08T00:16:31.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.214+0000 7f83133c6700 1 -- 172.21.15.99:0/1870658987 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f830c0c59e0 msgr2=0x7f830c1587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:31.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.214+0000 7f83133c6700 1 --2- 172.21.15.99:0/1870658987 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f830c0c59e0 0x7f830c1587d0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f830400b610 tx=0x7f8304019000 comp rx=0 tx=0).stop 2023-10-08T00:16:31.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.214+0000 7f83133c6700 1 -- 172.21.15.99:0/1870658987 shutdown_connections 2023-10-08T00:16:31.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.214+0000 7f83133c6700 1 --2- 172.21.15.99:0/1870658987 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f82f8066f30 0x7f82f80693e0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:31.216 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.214+0000 7f83133c6700 1 --2- 172.21.15.99:0/1870658987 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f830c15d240 0x7f830c09c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:31.216 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.214+0000 7f83133c6700 1 --2- 172.21.15.99:0/1870658987 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f830c0c59e0 0x7f830c1587d0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:31.216 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.214+0000 7f83133c6700 1 --2- 172.21.15.99:0/1870658987 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f830c0cc940 0x7f830c09c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:31.216 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.214+0000 7f83133c6700 1 -- 172.21.15.99:0/1870658987 >> 172.21.15.99:0/1870658987 conn(0x7f830c067e40 msgr2=0x7f830c159790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:31.216 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.214+0000 7f83133c6700 1 -- 172.21.15.99:0/1870658987 shutdown_connections 2023-10-08T00:16:31.216 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.214+0000 7f83133c6700 1 -- 172.21.15.99:0/1870658987 wait complete. 2023-10-08T00:16:31.216 INFO:teuthology.orchestra.run.smithi099.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2023-10-08T00:16:31.227 DEBUG:teuthology.orchestra.run.smithi099:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-08T00:16:31.227 DEBUG:teuthology.orchestra.run.smithi099:> import os 2023-10-08T00:16:31.227 DEBUG:teuthology.orchestra.run.smithi099:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2023-10-08T00:16:31.227 DEBUG:teuthology.orchestra.run.smithi099:> os.fchmod(fd, 0O666) 2023-10-08T00:16:31.227 DEBUG:teuthology.orchestra.run.smithi099:> os.fsync(fd) 2023-10-08T00:16:31.227 DEBUG:teuthology.orchestra.run.smithi099:> os.close(fd) 2023-10-08T00:16:31.227 DEBUG:teuthology.orchestra.run.smithi099:> ' 2023-10-08T00:16:31.480 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:16:31.480 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-08T00:16:31.549 INFO:teuthology.orchestra.run.smithi099.stdout:client6316 2023-10-08T00:16:31.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:16:31.550 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/06962cdb-9a6f-417f-82fd-9e4e3dedbccf.client6316/osdmap') 2023-10-08T00:16:31.625 INFO:teuthology.orchestra.run.smithi099.stdout:epoch 124 barrier 124 flags 0x588000 2023-10-08T00:16:31.625 INFO:teuthology.orchestra.run.smithi099.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-08T00:16:31.625 INFO:teuthology.orchestra.run.smithi099.stdout:pool 14 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-08T00:16:31.625 INFO:teuthology.orchestra.run.smithi099.stdout:pool 15 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-08T00:16:31.625 INFO:teuthology.orchestra.run.smithi099.stdout:osd0 (1)172.21.15.99:6801 100% (exists, up) 100% -1 2023-10-08T00:16:31.625 INFO:teuthology.orchestra.run.smithi099.stdout:osd1 (1)172.21.15.99:6809 100% (exists, up) 100% -1 2023-10-08T00:16:31.626 INFO:teuthology.orchestra.run.smithi099.stdout:osd2 (1)172.21.15.99:6817 100% (exists, up) 100% -1 2023-10-08T00:16:31.626 INFO:teuthology.orchestra.run.smithi099.stdout:osd3 (1)172.21.15.99:6820 100% (exists, up) 100% -1 2023-10-08T00:16:31.626 INFO:teuthology.orchestra.run.smithi099.stdout:osd4 (1)172.21.15.191:6801 100% (exists, up) 100% -1 2023-10-08T00:16:31.626 INFO:teuthology.orchestra.run.smithi099.stdout:osd5 (1)172.21.15.191:6818 100% (exists, up) 100% -1 2023-10-08T00:16:31.626 INFO:teuthology.orchestra.run.smithi099.stdout:osd6 (1)172.21.15.191:6809 100% (exists, up) 100% -1 2023-10-08T00:16:31.626 INFO:teuthology.orchestra.run.smithi099.stdout:osd7 (1)172.21.15.191:6819 100% (exists, up) 100% -1 2023-10-08T00:16:31.627 INFO:teuthology.orchestra.run.smithi099.stderr:1+1 records in 2023-10-08T00:16:31.627 INFO:teuthology.orchestra.run.smithi099.stderr:1+1 records out 2023-10-08T00:16:31.627 INFO:teuthology.orchestra.run.smithi099.stderr:799 bytes copied, 3.68e-05 s, 21.7 MB/s 2023-10-08T00:16:31.627 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set pause 2023-10-08T00:16:31.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.781+0000 7fb22a005700 1 Processor -- start 2023-10-08T00:16:31.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.781+0000 7fb22a005700 1 -- start start 2023-10-08T00:16:31.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.782+0000 7fb22a005700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb2240d1160 0x7fb2240cf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:31.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.782+0000 7fb22a005700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2241579c0 0x7fb22415ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:31.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.782+0000 7fb22a005700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2240d0540 con 0x7fb2241575f0 2023-10-08T00:16:31.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.782+0000 7fb22a005700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2240697b0 con 0x7fb2240d1160 2023-10-08T00:16:31.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.782+0000 7fb22a005700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb22415b240 con 0x7fb2241579c0 2023-10-08T00:16:31.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.782+0000 7fb229003700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7fb2241575f0 0x7fb2240d0a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:43548/0 (socket says 172.21.15.99:43548) 2023-10-08T00:16:31.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.782+0000 7fb228802700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb2240d1160 0x7fb2240cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:31.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.782+0000 7fb229804700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2241579c0 0x7fb22415ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:31.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.782+0000 7fb229003700 1 -- 172.21.15.99:0/3355004937 learned_addr learned my addr 172.21.15.99:0/3355004937 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:31.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.783+0000 7fb223fff700 1 -- 172.21.15.99:0/3355004937 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1253091802 0 0) 0x7fb2240d0540 con 0x7fb2241575f0 2023-10-08T00:16:31.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.783+0000 7fb223fff700 1 -- 172.21.15.99:0/3355004937 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb20c003570 con 0x7fb2241575f0 2023-10-08T00:16:31.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.783+0000 7fb229804700 1 -- 172.21.15.99:0/3355004937 >> v2:172.21.15.191:3300/0 conn(0x7fb2240d1160 msgr2=0x7fb2240cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:31.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.783+0000 7fb229804700 1 --2- 172.21.15.99:0/3355004937 >> v2:172.21.15.191:3300/0 conn(0x7fb2240d1160 0x7fb2240cf270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:31.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.783+0000 7fb229804700 1 -- 172.21.15.99:0/3355004937 >> v1:172.21.15.191:6789/0 conn(0x7fb2241575f0 legacy=0x7fb2240d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:31.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.783+0000 7fb229804700 1 -- 172.21.15.99:0/3355004937 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2240cf930 con 0x7fb2241579c0 2023-10-08T00:16:31.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.783+0000 7fb228802700 1 --2- 172.21.15.99:0/3355004937 >> v2:172.21.15.191:3300/0 conn(0x7fb2240d1160 0x7fb2240cf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:16:31.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.783+0000 7fb229804700 1 --2- 172.21.15.99:0/3355004937 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2241579c0 0x7fb22415ad00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb214009e60 tx=0x7fb214009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe2a42a19853ffad server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:31.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.783+0000 7fb223fff700 1 -- 172.21.15.99:0/3355004937 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21400e070 con 0x7fb2241579c0 2023-10-08T00:16:31.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.783+0000 7fb223fff700 1 -- 172.21.15.99:0/3355004937 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb21400bab0 con 0x7fb2241579c0 2023-10-08T00:16:31.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.783+0000 7fb223fff700 1 -- 172.21.15.99:0/3355004937 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21400be70 con 0x7fb2241579c0 2023-10-08T00:16:31.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.784+0000 7fb22a005700 1 -- 172.21.15.99:0/3355004937 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2241579c0 msgr2=0x7fb22415ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:31.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.784+0000 7fb22a005700 1 --2- 172.21.15.99:0/3355004937 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2241579c0 0x7fb22415ad00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb214009e60 tx=0x7fb214009780 comp rx=0 tx=0).stop 2023-10-08T00:16:31.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.784+0000 7fb22a005700 1 -- 172.21.15.99:0/3355004937 shutdown_connections 2023-10-08T00:16:31.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.784+0000 7fb22a005700 1 --2- 172.21.15.99:0/3355004937 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2241579c0 0x7fb22415ad00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:31.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.784+0000 7fb22a005700 1 --2- 172.21.15.99:0/3355004937 >> v2:172.21.15.191:3300/0 conn(0x7fb2240d1160 0x7fb2240cf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:31.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.784+0000 7fb22a005700 1 -- 172.21.15.99:0/3355004937 >> 172.21.15.99:0/3355004937 conn(0x7fb224068130 msgr2=0x7fb224068ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:31.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.784+0000 7fb22a005700 1 -- 172.21.15.99:0/3355004937 shutdown_connections 2023-10-08T00:16:31.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.784+0000 7fb22a005700 1 -- 172.21.15.99:0/3355004937 wait complete. 2023-10-08T00:16:31.788 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.785+0000 7fb22a005700 1 Processor -- start 2023-10-08T00:16:31.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.785+0000 7fb22a005700 1 -- start start 2023-10-08T00:16:31.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.786+0000 7fb22a005700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2240d1160 0x7fb2240ca3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:31.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.786+0000 7fb22a005700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2241575f0 0x7fb224143880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:31.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.786+0000 7fb22a005700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2241579c0 0x7fb2240cc750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:31.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.786+0000 7fb22a005700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2240cdb70 con 0x7fb2241579c0 2023-10-08T00:16:31.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.786+0000 7fb22a005700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb224143dc0 con 0x7fb2240d1160 2023-10-08T00:16:31.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.786+0000 7fb22a005700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2240ccc90 con 0x7fb2241575f0 2023-10-08T00:16:31.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.786+0000 7fb229804700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2241579c0 0x7fb2240cc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:31.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.786+0000 7fb229804700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2241579c0 0x7fb2240cc750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37194/0 (socket says 172.21.15.99:37194) 2023-10-08T00:16:31.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.786+0000 7fb229804700 1 -- 172.21.15.99:0/2884005219 learned_addr learned my addr 172.21.15.99:0/2884005219 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:31.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.786+0000 7fb229003700 1 --2- 172.21.15.99:0/2884005219 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2240d1160 0x7fb2240ca3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:31.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.786+0000 7fb228802700 1 --2- 172.21.15.99:0/2884005219 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2241575f0 0x7fb224143880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:31.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.787+0000 7fb229804700 1 -- 172.21.15.99:0/2884005219 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2241575f0 msgr2=0x7fb224143880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:31.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.787+0000 7fb229804700 1 --2- 172.21.15.99:0/2884005219 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2241575f0 0x7fb224143880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:31.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.787+0000 7fb229804700 1 -- 172.21.15.99:0/2884005219 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2240d1160 msgr2=0x7fb2240ca3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:31.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.787+0000 7fb229804700 1 --2- 172.21.15.99:0/2884005219 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2240d1160 0x7fb2240ca3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:31.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.787+0000 7fb229804700 1 -- 172.21.15.99:0/2884005219 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb218009420 con 0x7fb2241579c0 2023-10-08T00:16:31.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.787+0000 7fb229804700 1 --2- 172.21.15.99:0/2884005219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2241579c0 0x7fb2240cc750 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fb22415b380 tx=0x7fb214019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:31.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.787+0000 7fb2217fa700 1 -- 172.21.15.99:0/2884005219 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb214012e10 con 0x7fb2241579c0 2023-10-08T00:16:31.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.787+0000 7fb22a005700 1 -- 172.21.15.99:0/2884005219 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb214009a20 con 0x7fb2241579c0 2023-10-08T00:16:31.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.787+0000 7fb2217fa700 1 -- 172.21.15.99:0/2884005219 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb214027470 con 0x7fb2241579c0 2023-10-08T00:16:31.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.787+0000 7fb2217fa700 1 -- 172.21.15.99:0/2884005219 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb214027830 con 0x7fb2241579c0 2023-10-08T00:16:31.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.787+0000 7fb22a005700 1 -- 172.21.15.99:0/2884005219 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2240cd240 con 0x7fb2241579c0 2023-10-08T00:16:31.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.788+0000 7fb22a005700 1 -- 172.21.15.99:0/2884005219 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2240891c0 con 0x7fb2241579c0 2023-10-08T00:16:31.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.788+0000 7fb2217fa700 1 -- 172.21.15.99:0/2884005219 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb214024070 con 0x7fb2241579c0 2023-10-08T00:16:31.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.789+0000 7fb2217fa700 1 --2- 172.21.15.99:0/2884005219 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb210066f30 0x7fb2100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:31.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.789+0000 7fb2217fa700 1 -- 172.21.15.99:0/2884005219 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb214086ff0 con 0x7fb2241579c0 2023-10-08T00:16:31.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.789+0000 7fb229003700 1 --2- 172.21.15.99:0/2884005219 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb210066f30 0x7fb2100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:31.792 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.789+0000 7fb229003700 1 --2- 172.21.15.99:0/2884005219 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb210066f30 0x7fb2100693e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb21c007fd0 tx=0x7fb21c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:31.793 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.793+0000 7fb2217fa700 1 -- 172.21.15.99:0/2884005219 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb214054260 con 0x7fb2241579c0 2023-10-08T00:16:31.987 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:31.986+0000 7fb22a005700 1 -- 172.21.15.99:0/2884005219 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd set", "key": "pause"} v 0) v1 -- 0x7fb2240b6a90 con 0x7fb2241579c0 2023-10-08T00:16:32.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.224+0000 7fb2217fa700 1 -- 172.21.15.99:0/2884005219 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "pause"}]=0 pauserd,pausewr is set v126) v1 ==== 93+0+0 (secure 0 0 0) 0x7fb214057880 con 0x7fb2241579c0 2023-10-08T00:16:32.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.227+0000 7fb22a005700 1 -- 172.21.15.99:0/2884005219 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb210066f30 msgr2=0x7fb2100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:32.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.227+0000 7fb22a005700 1 --2- 172.21.15.99:0/2884005219 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb210066f30 0x7fb2100693e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb21c007fd0 tx=0x7fb21c009040 comp rx=0 tx=0).stop 2023-10-08T00:16:32.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.227+0000 7fb22a005700 1 -- 172.21.15.99:0/2884005219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2241579c0 msgr2=0x7fb2240cc750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:32.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.227+0000 7fb22a005700 1 --2- 172.21.15.99:0/2884005219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2241579c0 0x7fb2240cc750 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fb22415b380 tx=0x7fb214019000 comp rx=0 tx=0).stop 2023-10-08T00:16:32.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.227+0000 7fb22a005700 1 -- 172.21.15.99:0/2884005219 shutdown_connections 2023-10-08T00:16:32.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.227+0000 7fb22a005700 1 --2- 172.21.15.99:0/2884005219 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb210066f30 0x7fb2100693e0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:32.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.227+0000 7fb22a005700 1 --2- 172.21.15.99:0/2884005219 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2240d1160 0x7fb2240ca3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:32.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.227+0000 7fb22a005700 1 --2- 172.21.15.99:0/2884005219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2241579c0 0x7fb2240cc750 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:32.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.227+0000 7fb22a005700 1 --2- 172.21.15.99:0/2884005219 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2241575f0 0x7fb224143880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:32.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.227+0000 7fb22a005700 1 -- 172.21.15.99:0/2884005219 >> 172.21.15.99:0/2884005219 conn(0x7fb224068130 msgr2=0x7fb2241594f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:32.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.227+0000 7fb22a005700 1 -- 172.21.15.99:0/2884005219 shutdown_connections 2023-10-08T00:16:32.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.227+0000 7fb22a005700 1 -- 172.21.15.99:0/2884005219 wait complete. 2023-10-08T00:16:32.229 INFO:teuthology.orchestra.run.smithi099.stderr:pauserd,pausewr is set 2023-10-08T00:16:32.240 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset pause 2023-10-08T00:16:32.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.354+0000 7f07cc32c700 1 Processor -- start 2023-10-08T00:16:32.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.354+0000 7f07cc32c700 1 -- start start 2023-10-08T00:16:32.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.355+0000 7f07cc32c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f07c40aedb0 0x7f07c40af180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:32.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.355+0000 7f07cc32c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f07c40c90f0 0x7f07c40c9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:32.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.355+0000 7f07cc32c700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07c406efc0 con 0x7f07c406f450 2023-10-08T00:16:32.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.355+0000 7f07cc32c700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07c4158470 con 0x7f07c40c90f0 2023-10-08T00:16:32.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.355+0000 7f07cc32c700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07c40af6c0 con 0x7f07c40aedb0 2023-10-08T00:16:32.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.355+0000 7f07ca8c9700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f07c40c90f0 0x7f07c40c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:32.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.355+0000 7f07ca8c9700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f07c40c90f0 0x7f07c40c9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37206/0 (socket says 172.21.15.99:37206) 2023-10-08T00:16:32.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.355+0000 7f07ca8c9700 1 -- 172.21.15.99:0/3773950595 learned_addr learned my addr 172.21.15.99:0/3773950595 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:32.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.355+0000 7f07ca0c8700 1 --2- 172.21.15.99:0/3773950595 >> v2:172.21.15.191:3300/0 conn(0x7f07c40aedb0 0x7f07c40af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:32.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.356+0000 7f07ca8c9700 1 -- 172.21.15.99:0/3773950595 >> v2:172.21.15.191:3300/0 conn(0x7f07c40aedb0 msgr2=0x7f07c40af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:32.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.356+0000 7f07ca8c9700 1 --2- 172.21.15.99:0/3773950595 >> v2:172.21.15.191:3300/0 conn(0x7f07c40aedb0 0x7f07c40af180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:32.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.356+0000 7f07ca8c9700 1 -- 172.21.15.99:0/3773950595 >> v1:172.21.15.191:6789/0 conn(0x7f07c406f450 legacy=0x7f07c40b16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:32.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.356+0000 7f07ca8c9700 1 -- 172.21.15.99:0/3773950595 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07c40b2100 con 0x7f07c40c90f0 2023-10-08T00:16:32.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.356+0000 7f07ca8c9700 1 --2- 172.21.15.99:0/3773950595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07c40c90f0 0x7f07c40c9530 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f07c40711c0 tx=0x7f07bc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b914bdd21d246163 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:32.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.356+0000 7f07c90c6700 1 -- 172.21.15.99:0/3773950595 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07bc00e070 con 0x7f07c40c90f0 2023-10-08T00:16:32.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.356+0000 7f07c90c6700 1 -- 172.21.15.99:0/3773950595 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07bc00ba50 con 0x7f07c40c90f0 2023-10-08T00:16:32.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.356+0000 7f07c90c6700 1 -- 172.21.15.99:0/3773950595 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07bc0092e0 con 0x7f07c40c90f0 2023-10-08T00:16:32.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.356+0000 7f07cc32c700 1 -- 172.21.15.99:0/3773950595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07c40c90f0 msgr2=0x7f07c40c9530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:32.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.356+0000 7f07cc32c700 1 --2- 172.21.15.99:0/3773950595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07c40c90f0 0x7f07c40c9530 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f07c40711c0 tx=0x7f07bc009840 comp rx=0 tx=0).stop 2023-10-08T00:16:32.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.357+0000 7f07cc32c700 1 -- 172.21.15.99:0/3773950595 shutdown_connections 2023-10-08T00:16:32.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.357+0000 7f07cc32c700 1 --2- 172.21.15.99:0/3773950595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07c40c90f0 0x7f07c40c9530 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:32.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.357+0000 7f07cc32c700 1 --2- 172.21.15.99:0/3773950595 >> v2:172.21.15.191:3300/0 conn(0x7f07c40aedb0 0x7f07c40af180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:32.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.357+0000 7f07cc32c700 1 -- 172.21.15.99:0/3773950595 >> 172.21.15.99:0/3773950595 conn(0x7f07c40cd1e0 msgr2=0x7f07c40cd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:32.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.357+0000 7f07cc32c700 1 -- 172.21.15.99:0/3773950595 shutdown_connections 2023-10-08T00:16:32.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.357+0000 7f07cc32c700 1 -- 172.21.15.99:0/3773950595 wait complete. 2023-10-08T00:16:32.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.358+0000 7f07cc32c700 1 Processor -- start 2023-10-08T00:16:32.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.358+0000 7f07cc32c700 1 -- start start 2023-10-08T00:16:32.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.358+0000 7f07cc32c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f07c406f450 0x7f07c4158bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:32.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.358+0000 7f07cc32c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f07c40aedb0 0x7f07c415a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:32.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.358+0000 7f07cc32c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07c40c90f0 0x7f07c415ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:32.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.358+0000 7f07cc32c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c415b2d0 con 0x7f07c40c90f0 2023-10-08T00:16:32.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.358+0000 7f07cc32c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c414c8f0 con 0x7f07c40aedb0 2023-10-08T00:16:32.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.358+0000 7f07cc32c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07c40ae670 con 0x7f07c406f450 2023-10-08T00:16:32.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.359+0000 7f07ca8c9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07c40c90f0 0x7f07c415ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:32.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.359+0000 7f07ca8c9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07c40c90f0 0x7f07c415ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37214/0 (socket says 172.21.15.99:37214) 2023-10-08T00:16:32.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.359+0000 7f07ca8c9700 1 -- 172.21.15.99:0/2182479882 learned_addr learned my addr 172.21.15.99:0/2182479882 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:32.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.359+0000 7f07ca8c9700 1 -- 172.21.15.99:0/2182479882 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f07c406f450 msgr2=0x7f07c4158bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:32.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.359+0000 7f07ca8c9700 1 --2- 172.21.15.99:0/2182479882 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f07c406f450 0x7f07c4158bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:32.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.359+0000 7f07ca8c9700 1 -- 172.21.15.99:0/2182479882 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f07c40aedb0 msgr2=0x7f07c415a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:32.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.359+0000 7f07c98c7700 1 --2- 172.21.15.99:0/2182479882 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f07c40aedb0 0x7f07c415a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:32.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.359+0000 7f07ca8c9700 1 --2- 172.21.15.99:0/2182479882 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f07c40aedb0 0x7f07c415a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:32.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.359+0000 7f07ca8c9700 1 -- 172.21.15.99:0/2182479882 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07bc009ae0 con 0x7f07c40c90f0 2023-10-08T00:16:32.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.359+0000 7f07ca8c9700 1 --2- 172.21.15.99:0/2182479882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07c40c90f0 0x7f07c415ad90 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f07c4072020 tx=0x7f07bc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:32.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.359+0000 7f07ba7fc700 1 -- 172.21.15.99:0/2182479882 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07bc0094e0 con 0x7f07c40c90f0 2023-10-08T00:16:32.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.359+0000 7f07ba7fc700 1 -- 172.21.15.99:0/2182479882 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07bc008e80 con 0x7f07c40c90f0 2023-10-08T00:16:32.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.359+0000 7f07ba7fc700 1 -- 172.21.15.99:0/2182479882 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07bc0256b0 con 0x7f07c40c90f0 2023-10-08T00:16:32.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.359+0000 7f07cc32c700 1 -- 172.21.15.99:0/2182479882 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07c40c6be0 con 0x7f07c40c90f0 2023-10-08T00:16:32.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.359+0000 7f07cc32c700 1 -- 172.21.15.99:0/2182479882 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07c40c71a0 con 0x7f07c40c90f0 2023-10-08T00:16:32.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.360+0000 7f07cc32c700 1 -- 172.21.15.99:0/2182479882 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07c404c3c0 con 0x7f07c40c90f0 2023-10-08T00:16:32.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.360+0000 7f07ba7fc700 1 -- 172.21.15.99:0/2182479882 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f07bc003040 con 0x7f07c40c90f0 2023-10-08T00:16:32.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.361+0000 7f07ba7fc700 1 --2- 172.21.15.99:0/2182479882 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f07a8066f30 0x7f07a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:32.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.361+0000 7f07ba7fc700 1 -- 172.21.15.99:0/2182479882 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f07bc086190 con 0x7f07c40c90f0 2023-10-08T00:16:32.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.361+0000 7f07ba7fc700 1 -- 172.21.15.99:0/2182479882 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=127+}) v3 -- 0x7f07a806ea50 con 0x7f07c40c90f0 2023-10-08T00:16:32.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.361+0000 7f07ca0c8700 1 --2- 172.21.15.99:0/2182479882 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f07a8066f30 0x7f07a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:32.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.362+0000 7f07ca0c8700 1 --2- 172.21.15.99:0/2182479882 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f07a8066f30 0x7f07a80693e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f07c0009a60 tx=0x7f07c0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:32.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.365+0000 7f07ba7fc700 1 -- 172.21.15.99:0/2182479882 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f07bc053400 con 0x7f07c40c90f0 2023-10-08T00:16:32.562 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:32.562+0000 7f07cc32c700 1 -- 172.21.15.99:0/2182479882 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd unset", "key": "pause"} v 0) v1 -- 0x7f07c40b23e0 con 0x7f07c40c90f0 2023-10-08T00:16:33.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.230+0000 7f07ba7fc700 1 -- 172.21.15.99:0/2182479882 <== mon.0 v2:172.21.15.99:3300/0 7 ==== osd_map(127..127 src has 1..127) v4 ==== 280+0+0 (secure 0 0 0) 0x7f07bc00be60 con 0x7f07c40c90f0 2023-10-08T00:16:33.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.230+0000 7f07ba7fc700 1 -- 172.21.15.99:0/2182479882 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=128}) v3 -- 0x7f07a806f3a0 con 0x7f07c40c90f0 2023-10-08T00:16:33.232 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.231+0000 7f07ba7fc700 1 -- 172.21.15.99:0/2182479882 <== mon.0 v2:172.21.15.99:3300/0 8 ==== mon_command_ack([{"prefix": "osd unset", "key": "pause"}]=0 pauserd,pausewr is unset v127) v1 ==== 97+0+0 (secure 0 0 0) 0x7f07bc02a090 con 0x7f07c40c90f0 2023-10-08T00:16:33.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.234+0000 7f07cc32c700 1 -- 172.21.15.99:0/2182479882 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f07a8066f30 msgr2=0x7f07a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:33.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.234+0000 7f07cc32c700 1 --2- 172.21.15.99:0/2182479882 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f07a8066f30 0x7f07a80693e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f07c0009a60 tx=0x7f07c0008040 comp rx=0 tx=0).stop 2023-10-08T00:16:33.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.234+0000 7f07cc32c700 1 -- 172.21.15.99:0/2182479882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07c40c90f0 msgr2=0x7f07c415ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:33.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.234+0000 7f07cc32c700 1 --2- 172.21.15.99:0/2182479882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07c40c90f0 0x7f07c415ad90 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f07c4072020 tx=0x7f07bc018000 comp rx=0 tx=0).stop 2023-10-08T00:16:33.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.234+0000 7f07cc32c700 1 -- 172.21.15.99:0/2182479882 shutdown_connections 2023-10-08T00:16:33.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.234+0000 7f07cc32c700 1 --2- 172.21.15.99:0/2182479882 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f07a8066f30 0x7f07a80693e0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:33.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.234+0000 7f07cc32c700 1 --2- 172.21.15.99:0/2182479882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07c40c90f0 0x7f07c415ad90 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:33.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.234+0000 7f07cc32c700 1 --2- 172.21.15.99:0/2182479882 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f07c406f450 0x7f07c4158bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:33.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.234+0000 7f07cc32c700 1 --2- 172.21.15.99:0/2182479882 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f07c40aedb0 0x7f07c415a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:33.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.234+0000 7f07cc32c700 1 -- 172.21.15.99:0/2182479882 >> 172.21.15.99:0/2182479882 conn(0x7f07c40cd1e0 msgr2=0x7f07c414d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:33.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.234+0000 7f07cc32c700 1 -- 172.21.15.99:0/2182479882 shutdown_connections 2023-10-08T00:16:33.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.234+0000 7f07cc32c700 1 -- 172.21.15.99:0/2182479882 wait complete. 2023-10-08T00:16:33.236 INFO:teuthology.orchestra.run.smithi099.stderr:pauserd,pausewr is unset 2023-10-08T00:16:33.248 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:16:33.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.363+0000 7fdde9a22700 1 Processor -- start 2023-10-08T00:16:33.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.363+0000 7fdde9a22700 1 -- start start 2023-10-08T00:16:33.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.364+0000 7fdde9a22700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdde4157870 0x7fdde415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:33.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.364+0000 7fdde9a22700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdde4157c40 0x7fdde4158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:33.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.364+0000 7fdde9a22700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdde415cbb0 con 0x7fdde415a620 2023-10-08T00:16:33.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.364+0000 7fdde9a22700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdde415bfa0 con 0x7fdde4157870 2023-10-08T00:16:33.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.364+0000 7fdde9a22700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdde415c280 con 0x7fdde4157c40 2023-10-08T00:16:33.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.364+0000 7fdde2ffd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdde4157870 0x7fdde415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:33.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.364+0000 7fdde2ffd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdde4157870 0x7fdde415b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39876/0 (socket says 172.21.15.99:39876) 2023-10-08T00:16:33.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.364+0000 7fdde2ffd700 1 -- 172.21.15.99:0/1866238586 learned_addr learned my addr 172.21.15.99:0/1866238586 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:33.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.364+0000 7fdde37fe700 1 --2- 172.21.15.99:0/1866238586 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdde4157c40 0x7fdde4158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:33.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.365+0000 7fdde2ffd700 1 -- 172.21.15.99:0/1866238586 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdde4157c40 msgr2=0x7fdde4158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:33.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.365+0000 7fdde2ffd700 1 --2- 172.21.15.99:0/1866238586 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdde4157c40 0x7fdde4158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:33.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.365+0000 7fdde2ffd700 1 -- 172.21.15.99:0/1866238586 >> v1:172.21.15.191:6789/0 conn(0x7fdde415a620 legacy=0x7fdde415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:33.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.365+0000 7fdde2ffd700 1 -- 172.21.15.99:0/1866238586 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde415c560 con 0x7fdde4157870 2023-10-08T00:16:33.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.365+0000 7fdde2ffd700 1 --2- 172.21.15.99:0/1866238586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdde4157870 0x7fdde415b6a0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fddd40099b0 tx=0x7fddd4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b14345f9aa43a9a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:33.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.365+0000 7fdde27fc700 1 -- 172.21.15.99:0/1866238586 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd400b860 con 0x7fdde4157870 2023-10-08T00:16:33.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.365+0000 7fdde27fc700 1 -- 172.21.15.99:0/1866238586 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd400b9c0 con 0x7fdde4157870 2023-10-08T00:16:33.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.365+0000 7fdde27fc700 1 -- 172.21.15.99:0/1866238586 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd400bd60 con 0x7fdde4157870 2023-10-08T00:16:33.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.365+0000 7fdde9a22700 1 -- 172.21.15.99:0/1866238586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdde4157870 msgr2=0x7fdde415b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:33.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.365+0000 7fdde9a22700 1 --2- 172.21.15.99:0/1866238586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdde4157870 0x7fdde415b6a0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fddd40099b0 tx=0x7fddd4009250 comp rx=0 tx=0).stop 2023-10-08T00:16:33.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.365+0000 7fdde9a22700 1 -- 172.21.15.99:0/1866238586 shutdown_connections 2023-10-08T00:16:33.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.365+0000 7fdde9a22700 1 --2- 172.21.15.99:0/1866238586 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdde4157c40 0x7fdde4158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:33.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.365+0000 7fdde9a22700 1 --2- 172.21.15.99:0/1866238586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdde4157870 0x7fdde415b6a0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:33.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.365+0000 7fdde9a22700 1 -- 172.21.15.99:0/1866238586 >> 172.21.15.99:0/1866238586 conn(0x7fdde4068ce0 msgr2=0x7fdde40690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:33.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.365+0000 7fdde9a22700 1 -- 172.21.15.99:0/1866238586 shutdown_connections 2023-10-08T00:16:33.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.365+0000 7fdde9a22700 1 -- 172.21.15.99:0/1866238586 wait complete. 2023-10-08T00:16:33.371 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.367+0000 7fdde9a22700 1 Processor -- start 2023-10-08T00:16:33.372 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.367+0000 7fdde9a22700 1 -- start start 2023-10-08T00:16:33.372 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.367+0000 7fdde9a22700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdde4157870 0x7fdde41458e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:33.372 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.367+0000 7fdde9a22700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdde4157c40 0x7fdde40c5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:33.372 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.367+0000 7fdde9a22700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdde415a620 0x7fdde40c74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:33.372 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.367+0000 7fdde9a22700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde415d5f0 con 0x7fdde4157c40 2023-10-08T00:16:33.372 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.367+0000 7fdde9a22700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde40c65d0 con 0x7fdde4157870 2023-10-08T00:16:33.372 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.367+0000 7fdde9a22700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdde40c68b0 con 0x7fdde415a620 2023-10-08T00:16:33.372 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.367+0000 7fddda7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdde4157c40 0x7fdde40c5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:33.372 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.368+0000 7fddda7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdde4157c40 0x7fdde40c5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39892/0 (socket says 172.21.15.99:39892) 2023-10-08T00:16:33.373 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.368+0000 7fddda7fc700 1 -- 172.21.15.99:0/1046985491 learned_addr learned my addr 172.21.15.99:0/1046985491 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:33.373 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.368+0000 7fdde2ffd700 1 --2- 172.21.15.99:0/1046985491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdde4157870 0x7fdde41458e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:33.373 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.368+0000 7fddda7fc700 1 -- 172.21.15.99:0/1046985491 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdde415a620 msgr2=0x7fdde40c74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:33.373 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.368+0000 7fddda7fc700 1 --2- 172.21.15.99:0/1046985491 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdde415a620 0x7fdde40c74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:33.373 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.368+0000 7fddda7fc700 1 -- 172.21.15.99:0/1046985491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdde4157870 msgr2=0x7fdde41458e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:33.373 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.368+0000 7fddda7fc700 1 --2- 172.21.15.99:0/1046985491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdde4157870 0x7fdde41458e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:33.373 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.368+0000 7fddda7fc700 1 -- 172.21.15.99:0/1046985491 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd40094f0 con 0x7fdde4157c40 2023-10-08T00:16:33.373 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.368+0000 7fddda7fc700 1 --2- 172.21.15.99:0/1046985491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdde4157c40 0x7fdde40c5b10 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fddcc00c590 tx=0x7fddcc007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:33.373 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.368+0000 7fdddb7fe700 1 -- 172.21.15.99:0/1046985491 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddcc011590 con 0x7fdde4157c40 2023-10-08T00:16:33.373 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.368+0000 7fdddb7fe700 1 -- 172.21.15.99:0/1046985491 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddcc011710 con 0x7fdde4157c40 2023-10-08T00:16:33.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.368+0000 7fdde9a22700 1 -- 172.21.15.99:0/1046985491 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdde40c6b90 con 0x7fdde4157c40 2023-10-08T00:16:33.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.368+0000 7fdddb7fe700 1 -- 172.21.15.99:0/1046985491 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddcc011ad0 con 0x7fdde4157c40 2023-10-08T00:16:33.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.368+0000 7fdde9a22700 1 -- 172.21.15.99:0/1046985491 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdde4156010 con 0x7fdde4157c40 2023-10-08T00:16:33.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.369+0000 7fdde9a22700 1 -- 172.21.15.99:0/1046985491 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdde415d760 con 0x7fdde4157c40 2023-10-08T00:16:33.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.369+0000 7fdddb7fe700 1 -- 172.21.15.99:0/1046985491 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fddcc01b3b0 con 0x7fdde4157c40 2023-10-08T00:16:33.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.370+0000 7fdddb7fe700 1 --2- 172.21.15.99:0/1046985491 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fddc4066f30 0x7fddc40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:33.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.370+0000 7fdddb7fe700 1 -- 172.21.15.99:0/1046985491 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fddcc083980 con 0x7fdde4157c40 2023-10-08T00:16:33.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.370+0000 7fdde2ffd700 1 --2- 172.21.15.99:0/1046985491 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fddc4066f30 0x7fddc40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:33.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.370+0000 7fdde2ffd700 1 --2- 172.21.15.99:0/1046985491 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fddc4066f30 0x7fddc40693e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fddd40099b0 tx=0x7fddd4017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:33.375 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.374+0000 7fdddb7fe700 1 -- 172.21.15.99:0/1046985491 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fddcc050bf0 con 0x7fdde4157c40 2023-10-08T00:16:33.557 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.556+0000 7fdde9a22700 1 -- 172.21.15.99:0/1046985491 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdde404c3c0 con 0x7fdde4157c40 2023-10-08T00:16:33.558 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.557+0000 7fdddb7fe700 1 -- 172.21.15.99:0/1046985491 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+15372 (secure 0 0 0) 0x7fddcc017070 con 0x7fdde4157c40 2023-10-08T00:16:33.558 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:16:33.558 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":127,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:16:33.221921+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:16:19.856259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-08T00:16:20.494851+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:16:33.560 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.559+0000 7fdde9a22700 1 -- 172.21.15.99:0/1046985491 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fddc4066f30 msgr2=0x7fddc40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:33.560 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.559+0000 7fdde9a22700 1 --2- 172.21.15.99:0/1046985491 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fddc4066f30 0x7fddc40693e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fddd40099b0 tx=0x7fddd4017000 comp rx=0 tx=0).stop 2023-10-08T00:16:33.560 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.559+0000 7fdde9a22700 1 -- 172.21.15.99:0/1046985491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdde4157c40 msgr2=0x7fdde40c5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:33.560 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.559+0000 7fdde9a22700 1 --2- 172.21.15.99:0/1046985491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdde4157c40 0x7fdde40c5b10 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fddcc00c590 tx=0x7fddcc007f50 comp rx=0 tx=0).stop 2023-10-08T00:16:33.561 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.559+0000 7fdde9a22700 1 -- 172.21.15.99:0/1046985491 shutdown_connections 2023-10-08T00:16:33.561 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.559+0000 7fdde9a22700 1 --2- 172.21.15.99:0/1046985491 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fddc4066f30 0x7fddc40693e0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:33.561 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.559+0000 7fdde9a22700 1 --2- 172.21.15.99:0/1046985491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdde4157870 0x7fdde41458e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:33.561 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.559+0000 7fdde9a22700 1 --2- 172.21.15.99:0/1046985491 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdde415a620 0x7fdde40c74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:33.561 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.559+0000 7fdde9a22700 1 --2- 172.21.15.99:0/1046985491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdde4157c40 0x7fdde40c5b10 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:33.561 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.559+0000 7fdde9a22700 1 -- 172.21.15.99:0/1046985491 >> 172.21.15.99:0/1046985491 conn(0x7fdde4068ce0 msgr2=0x7fdde40690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:33.561 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.559+0000 7fdde9a22700 1 -- 172.21.15.99:0/1046985491 shutdown_connections 2023-10-08T00:16:33.561 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:33.559+0000 7fdde9a22700 1 -- 172.21.15.99:0/1046985491 wait complete. 2023-10-08T00:16:33.572 DEBUG:teuthology.orchestra.run.smithi099:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-08T00:16:33.572 DEBUG:teuthology.orchestra.run.smithi099:> import os 2023-10-08T00:16:33.573 DEBUG:teuthology.orchestra.run.smithi099:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2023-10-08T00:16:33.573 DEBUG:teuthology.orchestra.run.smithi099:> os.fchmod(fd, 0O666) 2023-10-08T00:16:33.573 DEBUG:teuthology.orchestra.run.smithi099:> os.fsync(fd) 2023-10-08T00:16:33.573 DEBUG:teuthology.orchestra.run.smithi099:> os.close(fd) 2023-10-08T00:16:33.573 DEBUG:teuthology.orchestra.run.smithi099:> ' 2023-10-08T00:16:33.825 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:16:33.825 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-08T00:16:33.894 INFO:teuthology.orchestra.run.smithi099.stdout:client6316 2023-10-08T00:16:33.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:16:33.894 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/06962cdb-9a6f-417f-82fd-9e4e3dedbccf.client6316/osdmap') 2023-10-08T00:16:33.970 INFO:teuthology.orchestra.run.smithi099.stdout:epoch 124 barrier 124 flags 0x588000 2023-10-08T00:16:33.971 INFO:teuthology.orchestra.run.smithi099.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-08T00:16:33.971 INFO:teuthology.orchestra.run.smithi099.stdout:pool 14 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-08T00:16:33.971 INFO:teuthology.orchestra.run.smithi099.stdout:pool 15 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-08T00:16:33.971 INFO:teuthology.orchestra.run.smithi099.stdout:osd0 (1)172.21.15.99:6801 100% (exists, up) 100% -1 2023-10-08T00:16:33.971 INFO:teuthology.orchestra.run.smithi099.stdout:osd1 (1)172.21.15.99:6809 100% (exists, up) 100% -1 2023-10-08T00:16:33.971 INFO:teuthology.orchestra.run.smithi099.stdout:osd2 (1)172.21.15.99:6817 100% (exists, up) 100% -1 2023-10-08T00:16:33.971 INFO:teuthology.orchestra.run.smithi099.stdout:osd3 (1)172.21.15.99:6820 100% (exists, up) 100% -1 2023-10-08T00:16:33.971 INFO:teuthology.orchestra.run.smithi099.stdout:osd4 (1)172.21.15.191:6801 100% (exists, up) 100% -1 2023-10-08T00:16:33.971 INFO:teuthology.orchestra.run.smithi099.stdout:osd5 (1)172.21.15.191:6818 100% (exists, up) 100% -1 2023-10-08T00:16:33.971 INFO:teuthology.orchestra.run.smithi099.stdout:osd6 (1)172.21.15.191:6809 100% (exists, up) 100% -1 2023-10-08T00:16:33.971 INFO:teuthology.orchestra.run.smithi099.stdout:osd7 (1)172.21.15.191:6819 100% (exists, up) 100% -1 2023-10-08T00:16:33.972 INFO:teuthology.orchestra.run.smithi099.stderr:1+1 records in 2023-10-08T00:16:33.972 INFO:teuthology.orchestra.run.smithi099.stderr:1+1 records out 2023-10-08T00:16:33.973 INFO:teuthology.orchestra.run.smithi099.stderr:799 bytes copied, 3.4246e-05 s, 23.3 MB/s 2023-10-08T00:16:33.973 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:16:34.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.127+0000 7ff4eb59e700 1 Processor -- start 2023-10-08T00:16:34.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.127+0000 7ff4eb59e700 1 -- start start 2023-10-08T00:16:34.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.128+0000 7ff4eb59e700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff4ec0b09c0 0x7ff4ec0b0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:34.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.128+0000 7ff4eb59e700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4ec0ca050 con 0x7ff4ec06f0a0 2023-10-08T00:16:34.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.128+0000 7ff4eb59e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4ec0b12d0 con 0x7ff4ec0c8f30 2023-10-08T00:16:34.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.128+0000 7ff4eb59e700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4ec0c94d0 con 0x7ff4ec0b09c0 2023-10-08T00:16:34.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.128+0000 7ff4ea59c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff4ec0b09c0 0x7ff4ec0b0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:34.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.128+0000 7ff4ead9d700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7ff4ec06f0a0 0x7ff4ec0ad4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50796/0 (socket says 172.21.15.99:50796) 2023-10-08T00:16:34.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.128+0000 7ff4ead9d700 1 -- 172.21.15.99:0/3982023956 learned_addr learned my addr 172.21.15.99:0/3982023956 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:34.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.128+0000 7ff4ea59c700 1 -- 172.21.15.99:0/3982023956 >> v1:172.21.15.191:6789/0 conn(0x7ff4ec0c8f30 legacy=0x7ff4ec158b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:34.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.128+0000 7ff4ea59c700 1 -- 172.21.15.99:0/3982023956 >> v1:172.21.15.99:6789/0 conn(0x7ff4ec06f0a0 legacy=0x7ff4ec0ad4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:34.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.128+0000 7ff4e9d9b700 1 -- 172.21.15.99:0/3982023956 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 948249582 0 0) 0x7ff4ec0ca050 con 0x7ff4ec06f0a0 2023-10-08T00:16:34.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.128+0000 7ff4ea59c700 1 -- 172.21.15.99:0/3982023956 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4ec0c97b0 con 0x7ff4ec0b09c0 2023-10-08T00:16:34.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.129+0000 7ff4ea59c700 1 --2- 172.21.15.99:0/3982023956 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4ec0b09c0 0x7ff4ec0b0d90 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc0098e0 tx=0x7ff4dc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba622eee1e6868d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:34.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.129+0000 7ff4e9d9b700 1 -- 172.21.15.99:0/3982023956 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc009e80 con 0x7ff4ec0b09c0 2023-10-08T00:16:34.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.129+0000 7ff4e9d9b700 1 -- 172.21.15.99:0/3982023956 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4dc00b860 con 0x7ff4ec0b09c0 2023-10-08T00:16:34.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.129+0000 7ff4e9d9b700 1 -- 172.21.15.99:0/3982023956 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc00bbe0 con 0x7ff4ec0b09c0 2023-10-08T00:16:34.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.129+0000 7ff4eb59e700 1 -- 172.21.15.99:0/3982023956 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4ec0b09c0 msgr2=0x7ff4ec0b0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:34.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.129+0000 7ff4eb59e700 1 --2- 172.21.15.99:0/3982023956 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4ec0b09c0 0x7ff4ec0b0d90 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc0098e0 tx=0x7ff4dc0091a0 comp rx=0 tx=0).stop 2023-10-08T00:16:34.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.129+0000 7ff4eb59e700 1 -- 172.21.15.99:0/3982023956 shutdown_connections 2023-10-08T00:16:34.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.129+0000 7ff4eb59e700 1 --2- 172.21.15.99:0/3982023956 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4ec0b09c0 0x7ff4ec0b0d90 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:34.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.129+0000 7ff4eb59e700 1 -- 172.21.15.99:0/3982023956 >> 172.21.15.99:0/3982023956 conn(0x7ff4ec069870 msgr2=0x7ff4ec069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:34.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.129+0000 7ff4eb59e700 1 -- 172.21.15.99:0/3982023956 shutdown_connections 2023-10-08T00:16:34.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.129+0000 7ff4eb59e700 1 -- 172.21.15.99:0/3982023956 wait complete. 2023-10-08T00:16:34.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.130+0000 7ff4eb59e700 1 Processor -- start 2023-10-08T00:16:34.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.130+0000 7ff4eb59e700 1 -- start start 2023-10-08T00:16:34.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.131+0000 7ff4eb59e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4ec06f0a0 0x7ff4ec0c4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:34.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.131+0000 7ff4eb59e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff4ec0b09c0 0x7ff4ec16b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:34.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.131+0000 7ff4eb59e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff4ec0c8f30 0x7ff4ec16ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:34.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.131+0000 7ff4eb59e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ec0c3350 con 0x7ff4ec06f0a0 2023-10-08T00:16:34.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.131+0000 7ff4eb59e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ec0c2740 con 0x7ff4ec0c8f30 2023-10-08T00:16:34.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.131+0000 7ff4eb59e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ec0c2a20 con 0x7ff4ec0b09c0 2023-10-08T00:16:34.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.131+0000 7ff4ea59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4ec06f0a0 0x7ff4ec0c4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:34.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.131+0000 7ff4ea59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4ec06f0a0 0x7ff4ec0c4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39908/0 (socket says 172.21.15.99:39908) 2023-10-08T00:16:34.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.131+0000 7ff4ea59c700 1 -- 172.21.15.99:0/1992093854 learned_addr learned my addr 172.21.15.99:0/1992093854 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:34.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.131+0000 7ff4ead9d700 1 --2- 172.21.15.99:0/1992093854 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff4ec0c8f30 0x7ff4ec16ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:34.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.131+0000 7ff4e3fff700 1 --2- 172.21.15.99:0/1992093854 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff4ec0b09c0 0x7ff4ec16b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:34.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.131+0000 7ff4ea59c700 1 -- 172.21.15.99:0/1992093854 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff4ec0b09c0 msgr2=0x7ff4ec16b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:34.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.131+0000 7ff4ea59c700 1 --2- 172.21.15.99:0/1992093854 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff4ec0b09c0 0x7ff4ec16b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:34.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.131+0000 7ff4ea59c700 1 -- 172.21.15.99:0/1992093854 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff4ec0c8f30 msgr2=0x7ff4ec16ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:34.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.131+0000 7ff4ea59c700 1 --2- 172.21.15.99:0/1992093854 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff4ec0c8f30 0x7ff4ec16ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:34.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.131+0000 7ff4ea59c700 1 -- 172.21.15.99:0/1992093854 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4dc009420 con 0x7ff4ec06f0a0 2023-10-08T00:16:34.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.132+0000 7ff4e3fff700 1 --2- 172.21.15.99:0/1992093854 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff4ec0b09c0 0x7ff4ec16b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:34.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.132+0000 7ff4ea59c700 1 --2- 172.21.15.99:0/1992093854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4ec06f0a0 0x7ff4ec0c4cb0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec06dbb0 tx=0x7ff4dc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:34.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.132+0000 7ff4e27fc700 1 -- 172.21.15.99:0/1992093854 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc00bbe0 con 0x7ff4ec06f0a0 2023-10-08T00:16:34.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.132+0000 7ff4e27fc700 1 -- 172.21.15.99:0/1992093854 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4dc011420 con 0x7ff4ec06f0a0 2023-10-08T00:16:34.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.132+0000 7ff4eb59e700 1 -- 172.21.15.99:0/1992093854 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4ec0c2d00 con 0x7ff4ec06f0a0 2023-10-08T00:16:34.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.132+0000 7ff4e27fc700 1 -- 172.21.15.99:0/1992093854 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc0117c0 con 0x7ff4ec06f0a0 2023-10-08T00:16:34.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.132+0000 7ff4eb59e700 1 -- 172.21.15.99:0/1992093854 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4ec0c37e0 con 0x7ff4ec06f0a0 2023-10-08T00:16:34.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.133+0000 7ff4eb59e700 1 -- 172.21.15.99:0/1992093854 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4ec06e3e0 con 0x7ff4ec06f0a0 2023-10-08T00:16:34.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.133+0000 7ff4e27fc700 1 -- 172.21.15.99:0/1992093854 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff4dc011bf0 con 0x7ff4ec06f0a0 2023-10-08T00:16:34.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.133+0000 7ff4e27fc700 1 --2- 172.21.15.99:0/1992093854 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff4cc066f30 0x7ff4cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:34.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.134+0000 7ff4e27fc700 1 -- 172.21.15.99:0/1992093854 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff4dc085eb0 con 0x7ff4ec06f0a0 2023-10-08T00:16:34.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.134+0000 7ff4e3fff700 1 --2- 172.21.15.99:0/1992093854 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff4cc066f30 0x7ff4cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:34.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.134+0000 7ff4e3fff700 1 --2- 172.21.15.99:0/1992093854 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff4cc066f30 0x7ff4cc0693e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4007fd0 tx=0x7ff4d4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:34.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.138+0000 7ff4e27fc700 1 -- 172.21.15.99:0/1992093854 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff4dc053120 con 0x7ff4ec06f0a0 2023-10-08T00:16:34.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.364+0000 7ff4eb59e700 1 -- 172.21.15.99:0/1992093854 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff4ec0c3630 con 0x7ff4ec06f0a0 2023-10-08T00:16:34.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.365+0000 7ff4e27fc700 1 -- 172.21.15.99:0/1992093854 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4605 (secure 0 0 0) 0x7ff4dc009280 con 0x7ff4ec06f0a0 2023-10-08T00:16:34.366 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:16:34.366 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":56,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5627,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/2782755379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":2782755379},{"type":"v1","addr":"172.21.15.191:6837","nonce":2782755379}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6166,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/2980541392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2980541392},{"type":"v1","addr":"172.21.15.99:6834","nonce":2980541392}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6184,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/1342206989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1342206989},{"type":"v1","addr":"172.21.15.191:6836","nonce":1342206989}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:16:21.527609+0000","modified":"2023-10-08T00:16:25.185437+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6747},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6747":{"gid":6747,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/1495315849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":1495315849},{"type":"v1","addr":"172.21.15.99:6835","nonce":1495315849}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-08T00:16:34.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.366+0000 7ff4eb59e700 1 -- 172.21.15.99:0/1992093854 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff4cc066f30 msgr2=0x7ff4cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:34.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.366+0000 7ff4eb59e700 1 --2- 172.21.15.99:0/1992093854 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff4cc066f30 0x7ff4cc0693e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4007fd0 tx=0x7ff4d4009040 comp rx=0 tx=0).stop 2023-10-08T00:16:34.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.366+0000 7ff4eb59e700 1 -- 172.21.15.99:0/1992093854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4ec06f0a0 msgr2=0x7ff4ec0c4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:34.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.366+0000 7ff4eb59e700 1 --2- 172.21.15.99:0/1992093854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4ec06f0a0 0x7ff4ec0c4cb0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec06dbb0 tx=0x7ff4dc017000 comp rx=0 tx=0).stop 2023-10-08T00:16:34.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.366+0000 7ff4eb59e700 1 -- 172.21.15.99:0/1992093854 shutdown_connections 2023-10-08T00:16:34.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.366+0000 7ff4eb59e700 1 --2- 172.21.15.99:0/1992093854 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff4cc066f30 0x7ff4cc0693e0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:34.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.366+0000 7ff4eb59e700 1 --2- 172.21.15.99:0/1992093854 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff4ec0c8f30 0x7ff4ec16ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:34.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.366+0000 7ff4eb59e700 1 --2- 172.21.15.99:0/1992093854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4ec06f0a0 0x7ff4ec0c4cb0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:34.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.366+0000 7ff4eb59e700 1 --2- 172.21.15.99:0/1992093854 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff4ec0b09c0 0x7ff4ec16b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:34.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.366+0000 7ff4eb59e700 1 -- 172.21.15.99:0/1992093854 >> 172.21.15.99:0/1992093854 conn(0x7ff4ec069870 msgr2=0x7ff4ec155c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:34.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.367+0000 7ff4eb59e700 1 -- 172.21.15.99:0/1992093854 shutdown_connections 2023-10-08T00:16:34.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:34.367+0000 7ff4eb59e700 1 -- 172.21.15.99:0/1992093854 wait complete. 2023-10-08T00:16:34.519 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 56 2023-10-08T00:16:34.520 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json osdmap barrier 127 2023-10-08T00:16:34.669 DEBUG:tasks.cephfs.filesystem:_json_asok output empty 2023-10-08T00:16:34.669 DEBUG:teuthology.orchestra.run.smithi099:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-08T00:16:34.669 DEBUG:teuthology.orchestra.run.smithi099:> import os 2023-10-08T00:16:34.669 DEBUG:teuthology.orchestra.run.smithi099:> fd = os.open("/home/ubuntu/cephtest/mnt.0/baz", os.O_CREAT | os.O_RDWR, 0O600) 2023-10-08T00:16:34.669 DEBUG:teuthology.orchestra.run.smithi099:> os.fchmod(fd, 0O666) 2023-10-08T00:16:34.669 DEBUG:teuthology.orchestra.run.smithi099:> os.fsync(fd) 2023-10-08T00:16:34.669 DEBUG:teuthology.orchestra.run.smithi099:> os.close(fd) 2023-10-08T00:16:34.669 DEBUG:teuthology.orchestra.run.smithi099:> ' 2023-10-08T00:16:34.922 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:16:35.315 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-08T00:16:35.345 INFO:teuthology.orchestra.run.smithi099.stdout:client6316 2023-10-08T00:16:35.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:16:35.393 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/06962cdb-9a6f-417f-82fd-9e4e3dedbccf.client6316/osdmap') 2023-10-08T00:16:35.429 INFO:teuthology.orchestra.run.smithi099.stdout:epoch 127 barrier 127 flags 0x588000 2023-10-08T00:16:35.464 INFO:teuthology.orchestra.run.smithi099.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-08T00:16:35.464 INFO:teuthology.orchestra.run.smithi099.stdout:pool 14 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-08T00:16:35.464 INFO:teuthology.orchestra.run.smithi099.stdout:pool 15 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-08T00:16:35.464 INFO:teuthology.orchestra.run.smithi099.stdout:osd0 (1)172.21.15.99:6801 100% (exists, up) 100% -1 2023-10-08T00:16:35.464 INFO:teuthology.orchestra.run.smithi099.stdout:osd1 (1)172.21.15.99:6809 100% (exists, up) 100% -1 2023-10-08T00:16:35.464 INFO:teuthology.orchestra.run.smithi099.stdout:osd2 (1)172.21.15.99:6817 100% (exists, up) 100% -1 2023-10-08T00:16:35.464 INFO:teuthology.orchestra.run.smithi099.stdout:osd3 (1)172.21.15.99:6820 100% (exists, up) 100% -1 2023-10-08T00:16:35.464 INFO:teuthology.orchestra.run.smithi099.stdout:osd4 (1)172.21.15.191:6801 100% (exists, up) 100% -1 2023-10-08T00:16:35.464 INFO:teuthology.orchestra.run.smithi099.stdout:osd5 (1)172.21.15.191:6818 100% (exists, up) 100% -1 2023-10-08T00:16:35.464 INFO:teuthology.orchestra.run.smithi099.stdout:osd6 (1)172.21.15.191:6809 100% (exists, up) 100% -1 2023-10-08T00:16:35.465 INFO:teuthology.orchestra.run.smithi099.stdout:osd7 (1)172.21.15.191:6819 100% (exists, up) 100% -1 2023-10-08T00:16:35.466 INFO:teuthology.orchestra.run.smithi099.stderr:1+1 records in 2023-10-08T00:16:35.466 INFO:teuthology.orchestra.run.smithi099.stderr:1+1 records out 2023-10-08T00:16:35.466 INFO:teuthology.orchestra.run.smithi099.stderr:799 bytes copied, 3.6478e-05 s, 21.9 MB/s 2023-10-08T00:16:35.466 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:16:35.466 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-08T00:16:35.535 INFO:teuthology.orchestra.run.smithi099.stdout:client6316 2023-10-08T00:16:35.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:16:35.535 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/06962cdb-9a6f-417f-82fd-9e4e3dedbccf.client6316/osdmap') 2023-10-08T00:16:35.612 INFO:teuthology.orchestra.run.smithi099.stdout:epoch 127 barrier 127 flags 0x588000 2023-10-08T00:16:35.612 INFO:teuthology.orchestra.run.smithi099.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-08T00:16:35.612 INFO:teuthology.orchestra.run.smithi099.stdout:pool 14 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-08T00:16:35.612 INFO:teuthology.orchestra.run.smithi099.stdout:pool 15 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-08T00:16:35.612 INFO:teuthology.orchestra.run.smithi099.stdout:osd0 (1)172.21.15.99:6801 100% (exists, up) 100% -1 2023-10-08T00:16:35.612 INFO:teuthology.orchestra.run.smithi099.stdout:osd1 (1)172.21.15.99:6809 100% (exists, up) 100% -1 2023-10-08T00:16:35.612 INFO:teuthology.orchestra.run.smithi099.stdout:osd2 (1)172.21.15.99:6817 100% (exists, up) 100% -1 2023-10-08T00:16:35.613 INFO:teuthology.orchestra.run.smithi099.stdout:osd3 (1)172.21.15.99:6820 100% (exists, up) 100% -1 2023-10-08T00:16:35.613 INFO:teuthology.orchestra.run.smithi099.stdout:osd4 (1)172.21.15.191:6801 100% (exists, up) 100% -1 2023-10-08T00:16:35.613 INFO:teuthology.orchestra.run.smithi099.stdout:osd5 (1)172.21.15.191:6818 100% (exists, up) 100% -1 2023-10-08T00:16:35.613 INFO:teuthology.orchestra.run.smithi099.stdout:osd6 (1)172.21.15.191:6809 100% (exists, up) 100% -1 2023-10-08T00:16:35.613 INFO:teuthology.orchestra.run.smithi099.stdout:osd7 (1)172.21.15.191:6819 100% (exists, up) 100% -1 2023-10-08T00:16:35.614 INFO:teuthology.orchestra.run.smithi099.stderr:1+1 records in 2023-10-08T00:16:35.614 INFO:teuthology.orchestra.run.smithi099.stderr:1+1 records out 2023-10-08T00:16:35.614 INFO:teuthology.orchestra.run.smithi099.stderr:799 bytes copied, 3.4146e-05 s, 23.4 MB/s 2023-10-08T00:16:35.614 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-08T00:16:35.614 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T00:16:35.615 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T00:16:35.693 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T00:16:35.733 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T00:16:35.734 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T00:16:35.734 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:16:35.734 DEBUG:teuthology.orchestra.run.smithi099:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-08T00:16:35.799 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2023-10-08T00:16:35.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:16:35.799 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T00:16:35.816 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-08T00:16:35.816 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:16:35.816 DEBUG:teuthology.orchestra.run.smithi191:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-08T00:16:35.858 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2023-10-08T00:16:35.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:16:35.858 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-08T00:16:35.914 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:16:36.028 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.028+0000 7fd5c9a45700 1 Processor -- start 2023-10-08T00:16:36.029 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.028+0000 7fd5c9a45700 1 -- start start 2023-10-08T00:16:36.031 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.028+0000 7fd5c9a45700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd5c4157870 0x7fd5c415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:36.031 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.028+0000 7fd5c9a45700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd5c415a620 0x7fd5c415d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:36.031 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.028+0000 7fd5c9a45700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5c415cbb0 con 0x7fd5c415d580 2023-10-08T00:16:36.031 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.028+0000 7fd5c9a45700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5c415bfa0 con 0x7fd5c415a620 2023-10-08T00:16:36.031 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.028+0000 7fd5c9a45700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5c415c280 con 0x7fd5c4157870 2023-10-08T00:16:36.031 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.029+0000 7fd5c27fc700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd5c415a620 0x7fd5c415d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:36.031 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.029+0000 7fd5c27fc700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd5c415a620 0x7fd5c415d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39924/0 (socket says 172.21.15.99:39924) 2023-10-08T00:16:36.032 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.029+0000 7fd5c27fc700 1 -- 172.21.15.99:0/232959574 learned_addr learned my addr 172.21.15.99:0/232959574 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:36.032 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.029+0000 7fd5c2ffd700 1 --2- 172.21.15.99:0/232959574 >> v2:172.21.15.191:3300/0 conn(0x7fd5c4157870 0x7fd5c415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:36.032 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.029+0000 7fd5c27fc700 1 -- 172.21.15.99:0/232959574 >> v2:172.21.15.191:3300/0 conn(0x7fd5c4157870 msgr2=0x7fd5c415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:36.032 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.029+0000 7fd5c27fc700 1 --2- 172.21.15.99:0/232959574 >> v2:172.21.15.191:3300/0 conn(0x7fd5c4157870 0x7fd5c415b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:36.032 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.029+0000 7fd5c27fc700 1 -- 172.21.15.99:0/232959574 >> v1:172.21.15.191:6789/0 conn(0x7fd5c415d580 legacy=0x7fd5c4158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:36.032 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.029+0000 7fd5c27fc700 1 -- 172.21.15.99:0/232959574 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c415c560 con 0x7fd5c415a620 2023-10-08T00:16:36.032 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.029+0000 7fd5c2ffd700 1 --2- 172.21.15.99:0/232959574 >> v2:172.21.15.191:3300/0 conn(0x7fd5c4157870 0x7fd5c415b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:36.032 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.029+0000 7fd5c27fc700 1 --2- 172.21.15.99:0/232959574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5c415a620 0x7fd5c415d040 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4069d80 tx=0x7fd5ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12f59dbd2ab7a4bb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:36.032 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.029+0000 7fd5c1ffb700 1 -- 172.21.15.99:0/232959574 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac00b860 con 0x7fd5c415a620 2023-10-08T00:16:36.032 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.029+0000 7fd5c1ffb700 1 -- 172.21.15.99:0/232959574 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5ac00b9c0 con 0x7fd5c415a620 2023-10-08T00:16:36.032 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.029+0000 7fd5c1ffb700 1 -- 172.21.15.99:0/232959574 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac00bd60 con 0x7fd5c415a620 2023-10-08T00:16:36.032 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.030+0000 7fd5c9a45700 1 -- 172.21.15.99:0/232959574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5c415a620 msgr2=0x7fd5c415d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:36.033 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.030+0000 7fd5c9a45700 1 --2- 172.21.15.99:0/232959574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5c415a620 0x7fd5c415d040 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4069d80 tx=0x7fd5ac009250 comp rx=0 tx=0).stop 2023-10-08T00:16:36.033 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.030+0000 7fd5c9a45700 1 -- 172.21.15.99:0/232959574 shutdown_connections 2023-10-08T00:16:36.033 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.030+0000 7fd5c9a45700 1 --2- 172.21.15.99:0/232959574 >> v2:172.21.15.191:3300/0 conn(0x7fd5c4157870 0x7fd5c415b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:36.033 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.030+0000 7fd5c9a45700 1 --2- 172.21.15.99:0/232959574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5c415a620 0x7fd5c415d040 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:36.033 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.030+0000 7fd5c9a45700 1 -- 172.21.15.99:0/232959574 >> 172.21.15.99:0/232959574 conn(0x7fd5c4068ce0 msgr2=0x7fd5c40690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:36.033 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.030+0000 7fd5c9a45700 1 -- 172.21.15.99:0/232959574 shutdown_connections 2023-10-08T00:16:36.033 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.030+0000 7fd5c9a45700 1 -- 172.21.15.99:0/232959574 wait complete. 2023-10-08T00:16:36.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.031+0000 7fd5c9a45700 1 Processor -- start 2023-10-08T00:16:36.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.031+0000 7fd5c9a45700 1 -- start start 2023-10-08T00:16:36.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.032+0000 7fd5c9a45700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5c4157870 0x7fd5c4149820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:36.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.032+0000 7fd5c9a45700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd5c415a620 0x7fd5c40b65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:36.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.032+0000 7fd5c9a45700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd5c415d580 0x7fd5c415e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:36.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.032+0000 7fd5c9a45700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c40b1cd0 con 0x7fd5c4157870 2023-10-08T00:16:36.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.032+0000 7fd5c9a45700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c4149d60 con 0x7fd5c415d580 2023-10-08T00:16:36.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.032+0000 7fd5c9a45700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c40cf7b0 con 0x7fd5c415a620 2023-10-08T00:16:36.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.032+0000 7fd5c2ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5c4157870 0x7fd5c4149820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:36.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.032+0000 7fd5c2ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5c4157870 0x7fd5c4149820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39930/0 (socket says 172.21.15.99:39930) 2023-10-08T00:16:36.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.032+0000 7fd5c2ffd700 1 -- 172.21.15.99:0/3037904392 learned_addr learned my addr 172.21.15.99:0/3037904392 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:36.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.032+0000 7fd5c37fe700 1 --2- 172.21.15.99:0/3037904392 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd5c415d580 0x7fd5c415e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:36.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.032+0000 7fd5c27fc700 1 --2- 172.21.15.99:0/3037904392 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd5c415a620 0x7fd5c40b65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:36.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.032+0000 7fd5c2ffd700 1 -- 172.21.15.99:0/3037904392 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd5c415a620 msgr2=0x7fd5c40b65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:36.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.032+0000 7fd5c2ffd700 1 --2- 172.21.15.99:0/3037904392 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd5c415a620 0x7fd5c40b65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:36.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.032+0000 7fd5c2ffd700 1 -- 172.21.15.99:0/3037904392 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd5c415d580 msgr2=0x7fd5c415e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:36.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.032+0000 7fd5c2ffd700 1 --2- 172.21.15.99:0/3037904392 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd5c415d580 0x7fd5c415e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:36.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.032+0000 7fd5c2ffd700 1 -- 172.21.15.99:0/3037904392 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5ac0094f0 con 0x7fd5c4157870 2023-10-08T00:16:36.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.032+0000 7fd5c2ffd700 1 --2- 172.21.15.99:0/3037904392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5c4157870 0x7fd5c4149820 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc00d1d0 tx=0x7fd5bc009a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:36.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.033+0000 7fd5baffd700 1 -- 172.21.15.99:0/3037904392 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc00ddc0 con 0x7fd5c4157870 2023-10-08T00:16:36.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.033+0000 7fd5baffd700 1 -- 172.21.15.99:0/3037904392 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5bc00f040 con 0x7fd5c4157870 2023-10-08T00:16:36.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.033+0000 7fd5c9a45700 1 -- 172.21.15.99:0/3037904392 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5c40cfa90 con 0x7fd5c4157870 2023-10-08T00:16:36.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.033+0000 7fd5baffd700 1 -- 172.21.15.99:0/3037904392 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc015670 con 0x7fd5c4157870 2023-10-08T00:16:36.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.033+0000 7fd5c9a45700 1 -- 172.21.15.99:0/3037904392 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5c40d0050 con 0x7fd5c4157870 2023-10-08T00:16:36.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.033+0000 7fd5c9a45700 1 -- 172.21.15.99:0/3037904392 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5c415ba10 con 0x7fd5c4157870 2023-10-08T00:16:36.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.034+0000 7fd5baffd700 1 -- 172.21.15.99:0/3037904392 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd5bc015a20 con 0x7fd5c4157870 2023-10-08T00:16:36.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.034+0000 7fd5baffd700 1 --2- 172.21.15.99:0/3037904392 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd5a4066f30 0x7fd5a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:36.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.034+0000 7fd5baffd700 1 -- 172.21.15.99:0/3037904392 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd5bc084900 con 0x7fd5c4157870 2023-10-08T00:16:36.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.034+0000 7fd5c27fc700 1 --2- 172.21.15.99:0/3037904392 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd5a4066f30 0x7fd5a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:36.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.035+0000 7fd5c27fc700 1 --2- 172.21.15.99:0/3037904392 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd5a4066f30 0x7fd5a40693e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4069d80 tx=0x7fd5ac017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:36.039 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.038+0000 7fd5baffd700 1 -- 172.21.15.99:0/3037904392 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd5bc04e5a0 con 0x7fd5c4157870 2023-10-08T00:16:36.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.263+0000 7fd5c9a45700 1 -- 172.21.15.99:0/3037904392 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd5c40b1f90 con 0x7fd5c4157870 2023-10-08T00:16:36.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.264+0000 7fd5baffd700 1 -- 172.21.15.99:0/3037904392 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4605 (secure 0 0 0) 0x7fd5bc01a090 con 0x7fd5c4157870 2023-10-08T00:16:36.265 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:16:36.265 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":56,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5627,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/2782755379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":2782755379},{"type":"v1","addr":"172.21.15.191:6837","nonce":2782755379}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6166,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/2980541392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2980541392},{"type":"v1","addr":"172.21.15.99:6834","nonce":2980541392}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6184,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/1342206989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1342206989},{"type":"v1","addr":"172.21.15.191:6836","nonce":1342206989}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:16:21.527609+0000","modified":"2023-10-08T00:16:25.185437+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6747},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6747":{"gid":6747,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/1495315849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":1495315849},{"type":"v1","addr":"172.21.15.99:6835","nonce":1495315849}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-08T00:16:36.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.266+0000 7fd5c9a45700 1 -- 172.21.15.99:0/3037904392 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd5a4066f30 msgr2=0x7fd5a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:36.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.266+0000 7fd5c9a45700 1 --2- 172.21.15.99:0/3037904392 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd5a4066f30 0x7fd5a40693e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4069d80 tx=0x7fd5ac017000 comp rx=0 tx=0).stop 2023-10-08T00:16:36.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.266+0000 7fd5c9a45700 1 -- 172.21.15.99:0/3037904392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5c4157870 msgr2=0x7fd5c4149820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:36.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.266+0000 7fd5c9a45700 1 --2- 172.21.15.99:0/3037904392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5c4157870 0x7fd5c4149820 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc00d1d0 tx=0x7fd5bc009a90 comp rx=0 tx=0).stop 2023-10-08T00:16:36.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.266+0000 7fd5c9a45700 1 -- 172.21.15.99:0/3037904392 shutdown_connections 2023-10-08T00:16:36.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.266+0000 7fd5c9a45700 1 --2- 172.21.15.99:0/3037904392 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd5a4066f30 0x7fd5a40693e0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:36.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.266+0000 7fd5c9a45700 1 --2- 172.21.15.99:0/3037904392 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd5c415d580 0x7fd5c415e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:36.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.266+0000 7fd5c9a45700 1 --2- 172.21.15.99:0/3037904392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5c4157870 0x7fd5c4149820 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:36.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.266+0000 7fd5c9a45700 1 --2- 172.21.15.99:0/3037904392 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd5c415a620 0x7fd5c40b65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:36.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.266+0000 7fd5c9a45700 1 -- 172.21.15.99:0/3037904392 >> 172.21.15.99:0/3037904392 conn(0x7fd5c4068ce0 msgr2=0x7fd5c4159840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:36.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.266+0000 7fd5c9a45700 1 -- 172.21.15.99:0/3037904392 shutdown_connections 2023-10-08T00:16:36.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.266+0000 7fd5c9a45700 1 -- 172.21.15.99:0/3037904392 wait complete. 2023-10-08T00:16:36.268 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 56 2023-10-08T00:16:36.279 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:16:36.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.392+0000 7fcbcedb3700 1 Processor -- start 2023-10-08T00:16:36.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.392+0000 7fcbcedb3700 1 -- start start 2023-10-08T00:16:36.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.393+0000 7fcbcedb3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcbc815b540 0x7fcbc815aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:36.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.393+0000 7fcbcedb3700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fcbc8153e50 0x7fcbc81542b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:36.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.393+0000 7fcbcedb3700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbc80cf0c0 con 0x7fcbc80d0970 2023-10-08T00:16:36.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.393+0000 7fcbcedb3700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbc815b3e0 con 0x7fcbc8153e50 2023-10-08T00:16:36.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.393+0000 7fcbcedb3700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc815e8f0 con 0x7fcbc815b540 2023-10-08T00:16:36.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.393+0000 7fcbcddb1700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fcbc80d0970 0x7fcbc80d0d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50806/0 (socket says 172.21.15.99:50806) 2023-10-08T00:16:36.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.393+0000 7fcbcddb1700 1 -- 172.21.15.99:0/3404451240 learned_addr learned my addr 172.21.15.99:0/3404451240 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:36.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.393+0000 7fcbcd5b0700 1 --2- 172.21.15.99:0/3404451240 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcbc815b540 0x7fcbc815aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:36.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.393+0000 7fcbce5b2700 1 --2- 172.21.15.99:0/3404451240 >> v2:172.21.15.191:3300/0 conn(0x7fcbc8153e50 0x7fcbc81542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:36.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.393+0000 7fcbccdaf700 1 -- 172.21.15.99:0/3404451240 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3852419679 0 0) 0x7fcbc80cf0c0 con 0x7fcbc80d0970 2023-10-08T00:16:36.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.393+0000 7fcbccdaf700 1 -- 172.21.15.99:0/3404451240 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbb4003570 con 0x7fcbc80d0970 2023-10-08T00:16:36.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.394+0000 7fcbccdaf700 1 -- 172.21.15.99:0/3404451240 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2518940685 0 0) 0x7fcbb4003570 con 0x7fcbc80d0970 2023-10-08T00:16:36.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.394+0000 7fcbccdaf700 1 -- 172.21.15.99:0/3404451240 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcbc815b540 msgr2=0x7fcbc815aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:36.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.394+0000 7fcbccdaf700 1 --2- 172.21.15.99:0/3404451240 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcbc815b540 0x7fcbc815aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:36.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.394+0000 7fcbccdaf700 1 -- 172.21.15.99:0/3404451240 >> v2:172.21.15.191:3300/0 conn(0x7fcbc8153e50 msgr2=0x7fcbc81542b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:36.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.394+0000 7fcbccdaf700 1 --2- 172.21.15.99:0/3404451240 >> v2:172.21.15.191:3300/0 conn(0x7fcbc8153e50 0x7fcbc81542b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:36.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.394+0000 7fcbccdaf700 1 -- 172.21.15.99:0/3404451240 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc81547f0 con 0x7fcbc80d0970 2023-10-08T00:16:36.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.394+0000 7fcbccdaf700 1 -- 172.21.15.99:0/3404451240 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7fcbc4001470 con 0x7fcbc80d0970 2023-10-08T00:16:36.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.394+0000 7fcbccdaf700 1 -- 172.21.15.99:0/3404451240 >> v1:172.21.15.99:6789/0 conn(0x7fcbc80d0970 legacy=0x7fcbc80d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:36.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.394+0000 7fcbccdaf700 1 --2- 172.21.15.99:0/3404451240 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcbc815b540 0x7fcbb4003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:36.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.394+0000 7fcbccdaf700 1 --2- 172.21.15.99:0/3404451240 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcbb4003fe0 0x7fcbb4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:36.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.394+0000 7fcbccdaf700 1 --2- 172.21.15.99:0/3404451240 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbb4004d50 0x7fcbb40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:36.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.394+0000 7fcbccdaf700 1 -- 172.21.15.99:0/3404451240 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc815b3e0 con 0x7fcbb4004d50 2023-10-08T00:16:36.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.394+0000 7fcbccdaf700 1 -- 172.21.15.99:0/3404451240 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc815e8f0 con 0x7fcbb4003fe0 2023-10-08T00:16:36.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.394+0000 7fcbccdaf700 1 -- 172.21.15.99:0/3404451240 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb400ba90 con 0x7fcbc815b540 2023-10-08T00:16:36.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.394+0000 7fcbce5b2700 1 --2- 172.21.15.99:0/3404451240 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbb4004d50 0x7fcbb40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:36.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.394+0000 7fcbcd5b0700 1 --2- 172.21.15.99:0/3404451240 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcbb4003fe0 0x7fcbb4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:36.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.394+0000 7fcbcddb1700 1 --2- 172.21.15.99:0/3404451240 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcbc815b540 0x7fcbb4003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:36.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.394+0000 7fcbce5b2700 1 -- 172.21.15.99:0/3404451240 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcbc815b540 msgr2=0x7fcbb4003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:36.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.394+0000 7fcbce5b2700 1 --2- 172.21.15.99:0/3404451240 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcbc815b540 0x7fcbb4003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:36.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.394+0000 7fcbce5b2700 1 -- 172.21.15.99:0/3404451240 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcbb4003fe0 msgr2=0x7fcbb4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:36.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.394+0000 7fcbce5b2700 1 --2- 172.21.15.99:0/3404451240 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcbb4003fe0 0x7fcbb4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:36.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.394+0000 7fcbce5b2700 1 -- 172.21.15.99:0/3404451240 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb4003570 con 0x7fcbb4004d50 2023-10-08T00:16:36.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.394+0000 7fcbce5b2700 1 --2- 172.21.15.99:0/3404451240 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbb4004d50 0x7fcbb40051c0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fcbc000e210 tx=0x7fcbc000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9821a95d3367b1e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:36.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.394+0000 7fcbccdaf700 1 -- 172.21.15.99:0/3404451240 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc000ea60 con 0x7fcbb4004d50 2023-10-08T00:16:36.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.394+0000 7fcbccdaf700 1 -- 172.21.15.99:0/3404451240 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbc000f040 con 0x7fcbb4004d50 2023-10-08T00:16:36.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.395+0000 7fcbccdaf700 1 -- 172.21.15.99:0/3404451240 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc000ee10 con 0x7fcbb4004d50 2023-10-08T00:16:36.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.395+0000 7fcbcedb3700 1 -- 172.21.15.99:0/3404451240 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbb4004d50 msgr2=0x7fcbb40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:36.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.395+0000 7fcbcedb3700 1 --2- 172.21.15.99:0/3404451240 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbb4004d50 0x7fcbb40051c0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fcbc000e210 tx=0x7fcbc000b040 comp rx=0 tx=0).stop 2023-10-08T00:16:36.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.395+0000 7fcbce5b2700 1 -- 172.21.15.99:0/3404451240 reap_dead start 2023-10-08T00:16:36.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.395+0000 7fcbcedb3700 1 -- 172.21.15.99:0/3404451240 shutdown_connections 2023-10-08T00:16:36.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.395+0000 7fcbcedb3700 1 -- 172.21.15.99:0/3404451240 >> 172.21.15.99:0/3404451240 conn(0x7fcbc8068c00 msgr2=0x7fcbc8069000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:36.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.395+0000 7fcbcedb3700 1 -- 172.21.15.99:0/3404451240 shutdown_connections 2023-10-08T00:16:36.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.395+0000 7fcbcedb3700 1 -- 172.21.15.99:0/3404451240 wait complete. 2023-10-08T00:16:36.402 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.396+0000 7fcbcedb3700 1 Processor -- start 2023-10-08T00:16:36.402 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.396+0000 7fcbcedb3700 1 -- start start 2023-10-08T00:16:36.402 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.397+0000 7fcbcedb3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcbc8152800 0x7fcbc816b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:36.402 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.397+0000 7fcbcedb3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbc816a240 0x7fcbc816a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:36.402 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.397+0000 7fcbcedb3700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcbc8069000 0x7fcbc8069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:36.402 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.397+0000 7fcbcedb3700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc80c66f0 con 0x7fcbc816a240 2023-10-08T00:16:36.402 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.397+0000 7fcbcedb3700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc80699b0 con 0x7fcbc8069000 2023-10-08T00:16:36.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.397+0000 7fcbcedb3700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc8158970 con 0x7fcbc8152800 2023-10-08T00:16:36.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.397+0000 7fcbcd5b0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbc816a240 0x7fcbc816a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:36.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.397+0000 7fcbcd5b0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbc816a240 0x7fcbc816a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39948/0 (socket says 172.21.15.99:39948) 2023-10-08T00:16:36.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.397+0000 7fcbcd5b0700 1 -- 172.21.15.99:0/3883172574 learned_addr learned my addr 172.21.15.99:0/3883172574 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:36.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.397+0000 7fcbcddb1700 1 --2- 172.21.15.99:0/3883172574 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcbc8152800 0x7fcbc816b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:36.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.397+0000 7fcbce5b2700 1 --2- 172.21.15.99:0/3883172574 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcbc8069000 0x7fcbc8069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:36.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.397+0000 7fcbcd5b0700 1 -- 172.21.15.99:0/3883172574 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcbc8152800 msgr2=0x7fcbc816b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:36.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.397+0000 7fcbcd5b0700 1 --2- 172.21.15.99:0/3883172574 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcbc8152800 0x7fcbc816b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:36.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.397+0000 7fcbcd5b0700 1 -- 172.21.15.99:0/3883172574 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcbc8069000 msgr2=0x7fcbc8069470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:36.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.397+0000 7fcbcd5b0700 1 --2- 172.21.15.99:0/3883172574 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcbc8069000 0x7fcbc8069470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:36.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.397+0000 7fcbcd5b0700 1 -- 172.21.15.99:0/3883172574 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc816ad00 con 0x7fcbc816a240 2023-10-08T00:16:36.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.397+0000 7fcbcd5b0700 1 --2- 172.21.15.99:0/3883172574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbc816a240 0x7fcbc816a6b0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8011b10 tx=0x7fcbb80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:36.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.397+0000 7fcbbdffb700 1 -- 172.21.15.99:0/3883172574 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb800f940 con 0x7fcbc816a240 2023-10-08T00:16:36.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.398+0000 7fcbbdffb700 1 -- 172.21.15.99:0/3883172574 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbb800faa0 con 0x7fcbc816a240 2023-10-08T00:16:36.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.398+0000 7fcbbdffb700 1 -- 172.21.15.99:0/3883172574 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb800fe20 con 0x7fcbc816a240 2023-10-08T00:16:36.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.398+0000 7fcbcedb3700 1 -- 172.21.15.99:0/3883172574 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbc816afe0 con 0x7fcbc816a240 2023-10-08T00:16:36.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.398+0000 7fcbcedb3700 1 -- 172.21.15.99:0/3883172574 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbc8152bd0 con 0x7fcbc816a240 2023-10-08T00:16:36.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.399+0000 7fcbcedb3700 1 -- 172.21.15.99:0/3883172574 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbc8158460 con 0x7fcbc816a240 2023-10-08T00:16:36.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.399+0000 7fcbbdffb700 1 -- 172.21.15.99:0/3883172574 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcbb8014650 con 0x7fcbc816a240 2023-10-08T00:16:36.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.399+0000 7fcbbdffb700 1 --2- 172.21.15.99:0/3883172574 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcbac066f30 0x7fcbac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:36.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.399+0000 7fcbbdffb700 1 -- 172.21.15.99:0/3883172574 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcbb8018030 con 0x7fcbc816a240 2023-10-08T00:16:36.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.400+0000 7fcbcddb1700 1 --2- 172.21.15.99:0/3883172574 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcbac066f30 0x7fcbac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:36.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.400+0000 7fcbcddb1700 1 --2- 172.21.15.99:0/3883172574 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcbac066f30 0x7fcbac0693e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fcbc40079e0 tx=0x7fcbc4001e90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:36.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.404+0000 7fcbbdffb700 1 -- 172.21.15.99:0/3883172574 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcbc8158460 con 0x7fcbc816a240 2023-10-08T00:16:36.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.589+0000 7fcbcedb3700 1 -- 172.21.15.99:0/3883172574 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcbc8158460 con 0x7fcbc816a240 2023-10-08T00:16:36.590 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.589+0000 7fcbbdffb700 1 -- 172.21.15.99:0/3883172574 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+15372 (secure 0 0 0) 0x7fcbc8158460 con 0x7fcbc816a240 2023-10-08T00:16:36.590 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:16:36.591 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":127,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:16:33.221921+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:16:19.856259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-08T00:16:20.494851+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:16:36.592 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.592+0000 7fcbcedb3700 1 -- 172.21.15.99:0/3883172574 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcbac066f30 msgr2=0x7fcbac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:36.592 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.592+0000 7fcbcedb3700 1 --2- 172.21.15.99:0/3883172574 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcbac066f30 0x7fcbac0693e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fcbc40079e0 tx=0x7fcbc4001e90 comp rx=0 tx=0).stop 2023-10-08T00:16:36.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.592+0000 7fcbcedb3700 1 -- 172.21.15.99:0/3883172574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbc816a240 msgr2=0x7fcbc816a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:36.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.592+0000 7fcbcedb3700 1 --2- 172.21.15.99:0/3883172574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbc816a240 0x7fcbc816a6b0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8011b10 tx=0x7fcbb80091a0 comp rx=0 tx=0).stop 2023-10-08T00:16:36.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.592+0000 7fcbcedb3700 1 -- 172.21.15.99:0/3883172574 shutdown_connections 2023-10-08T00:16:36.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.592+0000 7fcbcedb3700 1 --2- 172.21.15.99:0/3883172574 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcbac066f30 0x7fcbac0693e0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:36.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.592+0000 7fcbcedb3700 1 --2- 172.21.15.99:0/3883172574 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcbc8069000 0x7fcbc8069470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:36.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.592+0000 7fcbcedb3700 1 --2- 172.21.15.99:0/3883172574 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcbc8152800 0x7fcbc816b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:36.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.592+0000 7fcbcedb3700 1 --2- 172.21.15.99:0/3883172574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbc816a240 0x7fcbc816a6b0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:36.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.592+0000 7fcbcedb3700 1 -- 172.21.15.99:0/3883172574 >> 172.21.15.99:0/3883172574 conn(0x7fcbc8068c00 msgr2=0x7fcbc8153660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:36.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.592+0000 7fcbcedb3700 1 -- 172.21.15.99:0/3883172574 shutdown_connections 2023-10-08T00:16:36.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.592+0000 7fcbcedb3700 1 -- 172.21.15.99:0/3883172574 wait complete. 2023-10-08T00:16:36.604 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:10:42.063344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T00:16:19.856259+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T00:16:20.494851+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '125', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T00:16:36.605 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T00:16:36.721 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.720+0000 7f9a79ab2700 1 Processor -- start 2023-10-08T00:16:36.721 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.720+0000 7f9a79ab2700 1 -- start start 2023-10-08T00:16:36.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.721+0000 7f9a79ab2700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9a74159ec0 0x7f9a740af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:36.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.721+0000 7f9a79ab2700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a740b0790 con 0x7f9a7415a290 2023-10-08T00:16:36.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.721+0000 7f9a79ab2700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a740ce1f0 con 0x7f9a7406acb0 2023-10-08T00:16:36.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.721+0000 7f9a79ab2700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a740b0b20 con 0x7f9a74159ec0 2023-10-08T00:16:36.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.721+0000 7f9a737fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9a74159ec0 0x7f9a740af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:36.724 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.721+0000 7f9a72ffd700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f9a7415a290 0x7f9a7406a5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50808/0 (socket says 172.21.15.99:50808) 2023-10-08T00:16:36.724 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.721+0000 7f9a72ffd700 1 -- 172.21.15.99:0/3769148648 learned_addr learned my addr 172.21.15.99:0/3769148648 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:36.724 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.722+0000 7f9a737fe700 1 -- 172.21.15.99:0/3769148648 >> v1:172.21.15.191:6789/0 conn(0x7f9a7406acb0 legacy=0x7f9a740cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:36.724 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.722+0000 7f9a737fe700 1 -- 172.21.15.99:0/3769148648 >> v1:172.21.15.99:6789/0 conn(0x7f9a7415a290 legacy=0x7f9a7406a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:36.724 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.722+0000 7f9a727fc700 1 -- 172.21.15.99:0/3769148648 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2825622392 0 0) 0x7f9a740b0790 con 0x7f9a7415a290 2023-10-08T00:16:36.724 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.722+0000 7f9a737fe700 1 -- 172.21.15.99:0/3769148648 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a740b04b0 con 0x7f9a74159ec0 2023-10-08T00:16:36.724 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.722+0000 7f9a737fe700 1 --2- 172.21.15.99:0/3769148648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a74159ec0 0x7f9a740af940 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f9a64009e60 tx=0x7f9a64009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=619a085b88cee94f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:36.724 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.722+0000 7f9a727fc700 1 -- 172.21.15.99:0/3769148648 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6400e070 con 0x7f9a74159ec0 2023-10-08T00:16:36.724 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.722+0000 7f9a727fc700 1 -- 172.21.15.99:0/3769148648 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a6400bc30 con 0x7f9a74159ec0 2023-10-08T00:16:36.724 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.722+0000 7f9a727fc700 1 -- 172.21.15.99:0/3769148648 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a64012430 con 0x7f9a74159ec0 2023-10-08T00:16:36.724 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.722+0000 7f9a79ab2700 1 -- 172.21.15.99:0/3769148648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a74159ec0 msgr2=0x7f9a740af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:36.724 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.722+0000 7f9a79ab2700 1 --2- 172.21.15.99:0/3769148648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a74159ec0 0x7f9a740af940 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f9a64009e60 tx=0x7f9a64009780 comp rx=0 tx=0).stop 2023-10-08T00:16:36.725 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.722+0000 7f9a79ab2700 1 -- 172.21.15.99:0/3769148648 shutdown_connections 2023-10-08T00:16:36.725 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.722+0000 7f9a79ab2700 1 --2- 172.21.15.99:0/3769148648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a74159ec0 0x7f9a740af940 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:36.725 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.723+0000 7f9a79ab2700 1 -- 172.21.15.99:0/3769148648 >> 172.21.15.99:0/3769148648 conn(0x7f9a740d1880 msgr2=0x7f9a740cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:36.725 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.723+0000 7f9a79ab2700 1 -- 172.21.15.99:0/3769148648 shutdown_connections 2023-10-08T00:16:36.725 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.723+0000 7f9a79ab2700 1 -- 172.21.15.99:0/3769148648 wait complete. 2023-10-08T00:16:36.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.724+0000 7f9a79ab2700 1 Processor -- start 2023-10-08T00:16:36.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.724+0000 7f9a79ab2700 1 -- start start 2023-10-08T00:16:36.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.724+0000 7f9a79ab2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a7406acb0 0x7f9a740cb190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:36.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.724+0000 7f9a79ab2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a74159ec0 0x7f9a74159020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:36.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.724+0000 7f9a79ab2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a7415a290 0x7f9a74155a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:36.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.724+0000 7f9a79ab2700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a74159560 con 0x7f9a74159ec0 2023-10-08T00:16:36.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.725+0000 7f9a79ab2700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a74158b90 con 0x7f9a7415a290 2023-10-08T00:16:36.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.725+0000 7f9a79ab2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a74155f70 con 0x7f9a7406acb0 2023-10-08T00:16:36.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.725+0000 7f9a72ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a74159ec0 0x7f9a74159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:36.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.725+0000 7f9a72ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a74159ec0 0x7f9a74159020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39958/0 (socket says 172.21.15.99:39958) 2023-10-08T00:16:36.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.725+0000 7f9a72ffd700 1 -- 172.21.15.99:0/3233573536 learned_addr learned my addr 172.21.15.99:0/3233573536 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:36.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.725+0000 7f9a737fe700 1 --2- 172.21.15.99:0/3233573536 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a7406acb0 0x7f9a740cb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:36.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.725+0000 7f9a73fff700 1 --2- 172.21.15.99:0/3233573536 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a7415a290 0x7f9a74155a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:36.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.725+0000 7f9a72ffd700 1 -- 172.21.15.99:0/3233573536 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a7406acb0 msgr2=0x7f9a740cb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:36.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.725+0000 7f9a72ffd700 1 --2- 172.21.15.99:0/3233573536 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a7406acb0 0x7f9a740cb190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:36.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.725+0000 7f9a72ffd700 1 -- 172.21.15.99:0/3233573536 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a7415a290 msgr2=0x7f9a74155a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:36.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.725+0000 7f9a72ffd700 1 --2- 172.21.15.99:0/3233573536 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a7415a290 0x7f9a74155a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:36.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.725+0000 7f9a72ffd700 1 -- 172.21.15.99:0/3233573536 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a64009a20 con 0x7f9a74159ec0 2023-10-08T00:16:36.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.725+0000 7f9a73fff700 1 --2- 172.21.15.99:0/3233573536 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a7415a290 0x7f9a74155a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:16:36.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.725+0000 7f9a72ffd700 1 --2- 172.21.15.99:0/3233573536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a74159ec0 0x7f9a74159020 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f9a6800c5b0 tx=0x7f9a68001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:36.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.725+0000 7f9a5b7fe700 1 -- 172.21.15.99:0/3233573536 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a68005b30 con 0x7f9a74159ec0 2023-10-08T00:16:36.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.725+0000 7f9a5b7fe700 1 -- 172.21.15.99:0/3233573536 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a68005c90 con 0x7f9a74159ec0 2023-10-08T00:16:36.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.725+0000 7f9a79ab2700 1 -- 172.21.15.99:0/3233573536 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a74157f80 con 0x7f9a74159ec0 2023-10-08T00:16:36.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.725+0000 7f9a5b7fe700 1 -- 172.21.15.99:0/3233573536 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a68007300 con 0x7f9a74159ec0 2023-10-08T00:16:36.730 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.725+0000 7f9a79ab2700 1 -- 172.21.15.99:0/3233573536 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a74158540 con 0x7f9a74159ec0 2023-10-08T00:16:36.730 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.726+0000 7f9a79ab2700 1 -- 172.21.15.99:0/3233573536 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a7404c3c0 con 0x7f9a74159ec0 2023-10-08T00:16:36.730 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.727+0000 7f9a5b7fe700 1 -- 172.21.15.99:0/3233573536 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9a680076b0 con 0x7f9a74159ec0 2023-10-08T00:16:36.730 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.727+0000 7f9a5b7fe700 1 --2- 172.21.15.99:0/3233573536 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9a60066f30 0x7f9a600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:36.730 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.727+0000 7f9a5b7fe700 1 -- 172.21.15.99:0/3233573536 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9a680839d0 con 0x7f9a74159ec0 2023-10-08T00:16:36.730 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.727+0000 7f9a737fe700 1 --2- 172.21.15.99:0/3233573536 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9a60066f30 0x7f9a600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:36.730 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.728+0000 7f9a737fe700 1 --2- 172.21.15.99:0/3233573536 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9a60066f30 0x7f9a600693e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f9a64009e60 tx=0x7f9a64019000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:36.732 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.731+0000 7f9a5b7fe700 1 -- 172.21.15.99:0/3233573536 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9a68050c40 con 0x7f9a74159ec0 2023-10-08T00:16:36.912 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.912+0000 7f9a79ab2700 1 -- 172.21.15.99:0/3233573536 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9a74158cd0 con 0x7f9a74159ec0 2023-10-08T00:16:36.913 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.912+0000 7f9a5b7fe700 1 -- 172.21.15.99:0/3233573536 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v127) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9a680170a0 con 0x7f9a74159ec0 2023-10-08T00:16:36.913 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2023-10-08T00:16:36.915 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.914+0000 7f9a79ab2700 1 -- 172.21.15.99:0/3233573536 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9a60066f30 msgr2=0x7f9a600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:36.915 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.914+0000 7f9a79ab2700 1 --2- 172.21.15.99:0/3233573536 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9a60066f30 0x7f9a600693e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f9a64009e60 tx=0x7f9a64019000 comp rx=0 tx=0).stop 2023-10-08T00:16:36.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.914+0000 7f9a79ab2700 1 -- 172.21.15.99:0/3233573536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a74159ec0 msgr2=0x7f9a74159020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:36.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.914+0000 7f9a79ab2700 1 --2- 172.21.15.99:0/3233573536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a74159ec0 0x7f9a74159020 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f9a6800c5b0 tx=0x7f9a68001070 comp rx=0 tx=0).stop 2023-10-08T00:16:36.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.914+0000 7f9a79ab2700 1 -- 172.21.15.99:0/3233573536 shutdown_connections 2023-10-08T00:16:36.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.914+0000 7f9a79ab2700 1 --2- 172.21.15.99:0/3233573536 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9a60066f30 0x7f9a600693e0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:36.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.914+0000 7f9a79ab2700 1 --2- 172.21.15.99:0/3233573536 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a7415a290 0x7f9a74155a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:36.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.914+0000 7f9a79ab2700 1 --2- 172.21.15.99:0/3233573536 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a7406acb0 0x7f9a740cb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:36.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.914+0000 7f9a79ab2700 1 --2- 172.21.15.99:0/3233573536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a74159ec0 0x7f9a74159020 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:36.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.914+0000 7f9a79ab2700 1 -- 172.21.15.99:0/3233573536 >> 172.21.15.99:0/3233573536 conn(0x7f9a740d1880 msgr2=0x7f9a740c5b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:36.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.915+0000 7f9a79ab2700 1 -- 172.21.15.99:0/3233573536 shutdown_connections 2023-10-08T00:16:36.917 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:36.915+0000 7f9a79ab2700 1 -- 172.21.15.99:0/3233573536 wait complete. 2023-10-08T00:16:36.927 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T00:16:37.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.042+0000 7f80c31c1700 1 Processor -- start 2023-10-08T00:16:37.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.042+0000 7f80c31c1700 1 -- start start 2023-10-08T00:16:37.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.043+0000 7f80c31c1700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f80bc15a290 0x7f80bc06a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:37.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.043+0000 7f80c31c1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80bc06aae0 0x7f80bc0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:37.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.043+0000 7f80c31c1700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80bc0b0790 con 0x7f80bc159ec0 2023-10-08T00:16:37.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.043+0000 7f80c31c1700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80bc06af60 con 0x7f80bc15a290 2023-10-08T00:16:37.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.043+0000 7f80c31c1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80bc0ce1f0 con 0x7f80bc06aae0 2023-10-08T00:16:37.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.043+0000 7f80c0f5d700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f80bc159ec0 0x7f80bc0af940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50822/0 (socket says 172.21.15.99:50822) 2023-10-08T00:16:37.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.043+0000 7f80c0f5d700 1 -- 172.21.15.99:0/2585483815 learned_addr learned my addr 172.21.15.99:0/2585483815 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:37.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.043+0000 7f80c175e700 1 --2- 172.21.15.99:0/2585483815 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80bc06aae0 0x7f80bc0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:37.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.043+0000 7f80b3fff700 1 --2- 172.21.15.99:0/2585483815 >> v2:172.21.15.191:3300/0 conn(0x7f80bc15a290 0x7f80bc06a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:37.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.043+0000 7f80b37fe700 1 -- 172.21.15.99:0/2585483815 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2951872356 0 0) 0x7f80bc0b0790 con 0x7f80bc159ec0 2023-10-08T00:16:37.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.043+0000 7f80b37fe700 1 -- 172.21.15.99:0/2585483815 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80a4003570 con 0x7f80bc159ec0 2023-10-08T00:16:37.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.044+0000 7f80b3fff700 1 -- 172.21.15.99:0/2585483815 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80bc06aae0 msgr2=0x7f80bc0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:37.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.044+0000 7f80b3fff700 1 --2- 172.21.15.99:0/2585483815 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80bc06aae0 0x7f80bc0cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:37.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.044+0000 7f80b3fff700 1 -- 172.21.15.99:0/2585483815 >> v1:172.21.15.99:6789/0 conn(0x7f80bc159ec0 legacy=0x7f80bc0af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:37.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.044+0000 7f80b3fff700 1 -- 172.21.15.99:0/2585483815 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80bc0b0b20 con 0x7f80bc15a290 2023-10-08T00:16:37.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.044+0000 7f80c175e700 1 --2- 172.21.15.99:0/2585483815 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80bc06aae0 0x7f80bc0cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:16:37.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.044+0000 7f80b3fff700 1 --2- 172.21.15.99:0/2585483815 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80bc15a290 0x7f80bc06a5a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f80ac0098e0 tx=0x7f80ac0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=952ed2d5156e5523 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:37.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.044+0000 7f80b37fe700 1 -- 172.21.15.99:0/2585483815 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ac011630 con 0x7f80bc15a290 2023-10-08T00:16:37.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.044+0000 7f80b37fe700 1 -- 172.21.15.99:0/2585483815 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80ac011790 con 0x7f80bc15a290 2023-10-08T00:16:37.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.044+0000 7f80b37fe700 1 -- 172.21.15.99:0/2585483815 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ac011b30 con 0x7f80bc15a290 2023-10-08T00:16:37.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.044+0000 7f80c31c1700 1 -- 172.21.15.99:0/2585483815 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80bc15a290 msgr2=0x7f80bc06a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:37.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.044+0000 7f80c31c1700 1 --2- 172.21.15.99:0/2585483815 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80bc15a290 0x7f80bc06a5a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f80ac0098e0 tx=0x7f80ac0091a0 comp rx=0 tx=0).stop 2023-10-08T00:16:37.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.044+0000 7f80c31c1700 1 -- 172.21.15.99:0/2585483815 shutdown_connections 2023-10-08T00:16:37.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.044+0000 7f80c31c1700 1 --2- 172.21.15.99:0/2585483815 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80bc06aae0 0x7f80bc0cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:37.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.044+0000 7f80c31c1700 1 --2- 172.21.15.99:0/2585483815 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80bc15a290 0x7f80bc06a5a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:37.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.044+0000 7f80c31c1700 1 -- 172.21.15.99:0/2585483815 >> 172.21.15.99:0/2585483815 conn(0x7f80bc0d1880 msgr2=0x7f80bc0cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:37.048 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.045+0000 7f80c31c1700 1 -- 172.21.15.99:0/2585483815 shutdown_connections 2023-10-08T00:16:37.048 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.045+0000 7f80c31c1700 1 -- 172.21.15.99:0/2585483815 wait complete. 2023-10-08T00:16:37.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.046+0000 7f80c31c1700 1 Processor -- start 2023-10-08T00:16:37.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.046+0000 7f80c31c1700 1 -- start start 2023-10-08T00:16:37.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.046+0000 7f80c31c1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80bc06aae0 0x7f80bc148070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:37.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.046+0000 7f80c31c1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80bc159ec0 0x7f80bc156820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:37.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.046+0000 7f80c31c1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80bc15a290 0x7f80bc158230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:37.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.046+0000 7f80c31c1700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80bc157da0 con 0x7f80bc159ec0 2023-10-08T00:16:37.051 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.046+0000 7f80c31c1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80bc157190 con 0x7f80bc15a290 2023-10-08T00:16:37.051 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.046+0000 7f80c31c1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80bc157470 con 0x7f80bc06aae0 2023-10-08T00:16:37.051 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.047+0000 7f80b3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80bc159ec0 0x7f80bc156820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:37.051 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.047+0000 7f80b3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80bc159ec0 0x7f80bc156820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39966/0 (socket says 172.21.15.99:39966) 2023-10-08T00:16:37.051 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.047+0000 7f80b3fff700 1 -- 172.21.15.99:0/3284547796 learned_addr learned my addr 172.21.15.99:0/3284547796 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:37.051 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.047+0000 7f80c0f5d700 1 --2- 172.21.15.99:0/3284547796 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80bc06aae0 0x7f80bc148070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:37.051 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.047+0000 7f80c175e700 1 --2- 172.21.15.99:0/3284547796 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80bc15a290 0x7f80bc158230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:37.051 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.047+0000 7f80b3fff700 1 -- 172.21.15.99:0/3284547796 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80bc06aae0 msgr2=0x7f80bc148070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:37.051 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.047+0000 7f80b3fff700 1 --2- 172.21.15.99:0/3284547796 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80bc06aae0 0x7f80bc148070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:37.051 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.047+0000 7f80b3fff700 1 -- 172.21.15.99:0/3284547796 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80bc15a290 msgr2=0x7f80bc158230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:37.051 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.047+0000 7f80b3fff700 1 --2- 172.21.15.99:0/3284547796 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80bc15a290 0x7f80bc158230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:37.051 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.047+0000 7f80b3fff700 1 -- 172.21.15.99:0/3284547796 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b4009420 con 0x7f80bc159ec0 2023-10-08T00:16:37.052 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.047+0000 7f80c0f5d700 1 --2- 172.21.15.99:0/3284547796 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80bc06aae0 0x7f80bc148070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:37.052 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.047+0000 7f80c175e700 1 --2- 172.21.15.99:0/3284547796 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80bc15a290 0x7f80bc158230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:37.052 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.047+0000 7f80b3fff700 1 --2- 172.21.15.99:0/3284547796 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80bc159ec0 0x7f80bc156820 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f80bc071eb0 tx=0x7f80ac017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:37.052 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.047+0000 7f80b0ff9700 1 -- 172.21.15.99:0/3284547796 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ac011b10 con 0x7f80bc159ec0 2023-10-08T00:16:37.052 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.047+0000 7f80b0ff9700 1 -- 172.21.15.99:0/3284547796 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80ac0253f0 con 0x7f80bc159ec0 2023-10-08T00:16:37.052 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.047+0000 7f80c31c1700 1 -- 172.21.15.99:0/3284547796 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80ac009420 con 0x7f80bc159ec0 2023-10-08T00:16:37.052 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.047+0000 7f80b0ff9700 1 -- 172.21.15.99:0/3284547796 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ac0257b0 con 0x7f80bc159ec0 2023-10-08T00:16:37.052 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.047+0000 7f80c31c1700 1 -- 172.21.15.99:0/3284547796 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80bc157a30 con 0x7f80bc159ec0 2023-10-08T00:16:37.052 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.048+0000 7f80c31c1700 1 -- 172.21.15.99:0/3284547796 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80bc157f10 con 0x7f80bc159ec0 2023-10-08T00:16:37.052 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.049+0000 7f80b0ff9700 1 -- 172.21.15.99:0/3284547796 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f80ac025b60 con 0x7f80bc159ec0 2023-10-08T00:16:37.052 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.049+0000 7f80b0ff9700 1 --2- 172.21.15.99:0/3284547796 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f80a8066f30 0x7f80a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:37.052 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.049+0000 7f80b0ff9700 1 -- 172.21.15.99:0/3284547796 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f80ac085c80 con 0x7f80bc159ec0 2023-10-08T00:16:37.052 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.049+0000 7f80c0f5d700 1 --2- 172.21.15.99:0/3284547796 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f80a8066f30 0x7f80a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:37.053 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.050+0000 7f80c0f5d700 1 --2- 172.21.15.99:0/3284547796 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f80a8066f30 0x7f80a80693e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f80b8005c50 tx=0x7f80b8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:37.053 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.053+0000 7f80b0ff9700 1 -- 172.21.15.99:0/3284547796 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f80ac052ef0 con 0x7f80bc159ec0 2023-10-08T00:16:37.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.237+0000 7f80c31c1700 1 -- 172.21.15.99:0/3284547796 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f80bc04c3c0 con 0x7f80bc159ec0 2023-10-08T00:16:37.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.238+0000 7f80b0ff9700 1 -- 172.21.15.99:0/3284547796 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v127) v1 ==== 104+0+11 (secure 0 0 0) 0x7f80ac0150a0 con 0x7f80bc159ec0 2023-10-08T00:16:37.238 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2023-10-08T00:16:37.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.239+0000 7f80c31c1700 1 -- 172.21.15.99:0/3284547796 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f80a8066f30 msgr2=0x7f80a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:37.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.239+0000 7f80c31c1700 1 --2- 172.21.15.99:0/3284547796 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f80a8066f30 0x7f80a80693e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f80b8005c50 tx=0x7f80b8009040 comp rx=0 tx=0).stop 2023-10-08T00:16:37.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.239+0000 7f80c31c1700 1 -- 172.21.15.99:0/3284547796 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80bc159ec0 msgr2=0x7f80bc156820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:37.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.239+0000 7f80c31c1700 1 --2- 172.21.15.99:0/3284547796 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80bc159ec0 0x7f80bc156820 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f80bc071eb0 tx=0x7f80ac017000 comp rx=0 tx=0).stop 2023-10-08T00:16:37.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.240+0000 7f80c31c1700 1 -- 172.21.15.99:0/3284547796 shutdown_connections 2023-10-08T00:16:37.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.240+0000 7f80c31c1700 1 --2- 172.21.15.99:0/3284547796 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f80a8066f30 0x7f80a80693e0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:37.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.240+0000 7f80c31c1700 1 --2- 172.21.15.99:0/3284547796 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80bc15a290 0x7f80bc158230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:37.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.240+0000 7f80c31c1700 1 --2- 172.21.15.99:0/3284547796 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80bc06aae0 0x7f80bc148070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:37.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.240+0000 7f80c31c1700 1 --2- 172.21.15.99:0/3284547796 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80bc159ec0 0x7f80bc156820 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:37.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.240+0000 7f80c31c1700 1 -- 172.21.15.99:0/3284547796 >> 172.21.15.99:0/3284547796 conn(0x7f80bc0d1880 msgr2=0x7f80bc0b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:37.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.240+0000 7f80c31c1700 1 -- 172.21.15.99:0/3284547796 shutdown_connections 2023-10-08T00:16:37.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.240+0000 7f80c31c1700 1 -- 172.21.15.99:0/3284547796 wait complete. 2023-10-08T00:16:37.252 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T00:16:37.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.368+0000 7fc3fa736700 1 Processor -- start 2023-10-08T00:16:37.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.368+0000 7fc3fa736700 1 -- start start 2023-10-08T00:16:37.372 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.369+0000 7fc3fa736700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc3f4159de0 0x7fc3f40cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:37.372 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.369+0000 7fc3fa736700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc3f40cc680 0x7fc3f415da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:37.372 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.369+0000 7fc3fa736700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc3f415df70 0x7fc3f40b1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:37.372 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.369+0000 7fc3fa736700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3f40ce160 con 0x7fc3f415df70 2023-10-08T00:16:37.372 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.369+0000 7fc3fa736700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3f40cb440 con 0x7fc3f40cc680 2023-10-08T00:16:37.372 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.369+0000 7fc3fa736700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f415e3f0 con 0x7fc3f4159de0 2023-10-08T00:16:37.372 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.369+0000 7fc3f8cd3700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc3f415df70 0x7fc3f40b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:37.373 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.369+0000 7fc3f8cd3700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc3f415df70 0x7fc3f40b1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39968/0 (socket says 172.21.15.99:39968) 2023-10-08T00:16:37.373 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.369+0000 7fc3f8cd3700 1 -- 172.21.15.99:0/3058739354 learned_addr learned my addr 172.21.15.99:0/3058739354 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:37.373 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.369+0000 7fc3f3fff700 1 --2- 172.21.15.99:0/3058739354 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc3f4159de0 0x7fc3f40cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:37.373 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.369+0000 7fc3f37fe700 1 --2- 172.21.15.99:0/3058739354 >> v2:172.21.15.191:3300/0 conn(0x7fc3f40cc680 0x7fc3f415da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:37.373 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.369+0000 7fc3f8cd3700 1 -- 172.21.15.99:0/3058739354 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc3f4159de0 msgr2=0x7fc3f40cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:37.373 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.369+0000 7fc3f8cd3700 1 --2- 172.21.15.99:0/3058739354 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc3f4159de0 0x7fc3f40cdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:37.373 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.369+0000 7fc3f8cd3700 1 -- 172.21.15.99:0/3058739354 >> v2:172.21.15.191:3300/0 conn(0x7fc3f40cc680 msgr2=0x7fc3f415da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:37.373 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.369+0000 7fc3f8cd3700 1 --2- 172.21.15.99:0/3058739354 >> v2:172.21.15.191:3300/0 conn(0x7fc3f40cc680 0x7fc3f415da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:37.373 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.370+0000 7fc3f8cd3700 1 -- 172.21.15.99:0/3058739354 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f40cd790 con 0x7fc3f415df70 2023-10-08T00:16:37.373 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.370+0000 7fc3f8cd3700 1 --2- 172.21.15.99:0/3058739354 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3f415df70 0x7fc3f40b1900 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4071100 tx=0x7fc3ec009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e521fa729efeb9d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:37.373 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.370+0000 7fc3f2ffd700 1 -- 172.21.15.99:0/3058739354 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec00e070 con 0x7fc3f415df70 2023-10-08T00:16:37.373 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.370+0000 7fc3f2ffd700 1 -- 172.21.15.99:0/3058739354 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3ec00ba40 con 0x7fc3f415df70 2023-10-08T00:16:37.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.370+0000 7fc3f2ffd700 1 -- 172.21.15.99:0/3058739354 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec0092e0 con 0x7fc3f415df70 2023-10-08T00:16:37.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.370+0000 7fc3fa736700 1 -- 172.21.15.99:0/3058739354 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3f415df70 msgr2=0x7fc3f40b1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:37.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.370+0000 7fc3fa736700 1 --2- 172.21.15.99:0/3058739354 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3f415df70 0x7fc3f40b1900 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4071100 tx=0x7fc3ec009840 comp rx=0 tx=0).stop 2023-10-08T00:16:37.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.370+0000 7fc3fa736700 1 -- 172.21.15.99:0/3058739354 shutdown_connections 2023-10-08T00:16:37.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.370+0000 7fc3fa736700 1 --2- 172.21.15.99:0/3058739354 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3f415df70 0x7fc3f40b1900 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:37.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.370+0000 7fc3fa736700 1 --2- 172.21.15.99:0/3058739354 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc3f4159de0 0x7fc3f40cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:37.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.370+0000 7fc3fa736700 1 --2- 172.21.15.99:0/3058739354 >> v2:172.21.15.191:3300/0 conn(0x7fc3f40cc680 0x7fc3f415da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:37.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.370+0000 7fc3fa736700 1 -- 172.21.15.99:0/3058739354 >> 172.21.15.99:0/3058739354 conn(0x7fc3f40cb5c0 msgr2=0x7fc3f40cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:37.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.370+0000 7fc3fa736700 1 -- 172.21.15.99:0/3058739354 shutdown_connections 2023-10-08T00:16:37.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.370+0000 7fc3fa736700 1 -- 172.21.15.99:0/3058739354 wait complete. 2023-10-08T00:16:37.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.371+0000 7fc3fa736700 1 Processor -- start 2023-10-08T00:16:37.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.372+0000 7fc3fa736700 1 -- start start 2023-10-08T00:16:37.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.372+0000 7fc3fa736700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3f40cc680 0x7fc3f40c8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:37.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.372+0000 7fc3fa736700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc3f4159de0 0x7fc3f4157ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:37.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.372+0000 7fc3fa736700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3f415df70 0x7fc3f415c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:37.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.372+0000 7fc3fa736700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f415bbe0 con 0x7fc3f415df70 2023-10-08T00:16:37.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.372+0000 7fc3fa736700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f415bec0 con 0x7fc3f40cc680 2023-10-08T00:16:37.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.372+0000 7fc3fa736700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f415c1a0 con 0x7fc3f4159de0 2023-10-08T00:16:37.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.372+0000 7fc3f8cd3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3f415df70 0x7fc3f415c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:37.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.372+0000 7fc3f8cd3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3f415df70 0x7fc3f415c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39976/0 (socket says 172.21.15.99:39976) 2023-10-08T00:16:37.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.372+0000 7fc3f8cd3700 1 -- 172.21.15.99:0/1719190675 learned_addr learned my addr 172.21.15.99:0/1719190675 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:37.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.373+0000 7fc3f37fe700 1 --2- 172.21.15.99:0/1719190675 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc3f4159de0 0x7fc3f4157ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:37.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.373+0000 7fc3f3fff700 1 --2- 172.21.15.99:0/1719190675 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3f40cc680 0x7fc3f40c8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:37.384 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.373+0000 7fc3f8cd3700 1 -- 172.21.15.99:0/1719190675 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc3f4159de0 msgr2=0x7fc3f4157ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:37.384 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.373+0000 7fc3f8cd3700 1 --2- 172.21.15.99:0/1719190675 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc3f4159de0 0x7fc3f4157ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:37.384 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.373+0000 7fc3f8cd3700 1 -- 172.21.15.99:0/1719190675 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3f40cc680 msgr2=0x7fc3f40c8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:37.384 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.373+0000 7fc3f8cd3700 1 --2- 172.21.15.99:0/1719190675 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3f40cc680 0x7fc3f40c8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:37.384 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.373+0000 7fc3f8cd3700 1 -- 172.21.15.99:0/1719190675 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3ec009ae0 con 0x7fc3f415df70 2023-10-08T00:16:37.384 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.373+0000 7fc3f8cd3700 1 --2- 172.21.15.99:0/1719190675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3f415df70 0x7fc3f415c9a0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fc3f406de60 tx=0x7fc3ec018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:37.384 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.373+0000 7fc3dbfff700 1 -- 172.21.15.99:0/1719190675 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec008e40 con 0x7fc3f415df70 2023-10-08T00:16:37.384 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.373+0000 7fc3dbfff700 1 -- 172.21.15.99:0/1719190675 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3ec025460 con 0x7fc3f415df70 2023-10-08T00:16:37.385 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.373+0000 7fc3fa736700 1 -- 172.21.15.99:0/1719190675 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3f415c480 con 0x7fc3f415df70 2023-10-08T00:16:37.385 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.373+0000 7fc3dbfff700 1 -- 172.21.15.99:0/1719190675 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec025800 con 0x7fc3f415df70 2023-10-08T00:16:37.385 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.373+0000 7fc3fa736700 1 -- 172.21.15.99:0/1719190675 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3f40c8840 con 0x7fc3f415df70 2023-10-08T00:16:37.385 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.374+0000 7fc3fa736700 1 -- 172.21.15.99:0/1719190675 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3f415d740 con 0x7fc3f415df70 2023-10-08T00:16:37.385 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.374+0000 7fc3dbfff700 1 -- 172.21.15.99:0/1719190675 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc3ec003040 con 0x7fc3f415df70 2023-10-08T00:16:37.385 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.375+0000 7fc3dbfff700 1 --2- 172.21.15.99:0/1719190675 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc3e0066f30 0x7fc3e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:37.385 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.375+0000 7fc3dbfff700 1 -- 172.21.15.99:0/1719190675 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc3ec086350 con 0x7fc3f415df70 2023-10-08T00:16:37.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.375+0000 7fc3f3fff700 1 --2- 172.21.15.99:0/1719190675 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc3e0066f30 0x7fc3e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:37.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.375+0000 7fc3f3fff700 1 --2- 172.21.15.99:0/1719190675 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc3e0066f30 0x7fc3e00693e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc3e40089b0 tx=0x7fc3e4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:37.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.379+0000 7fc3dbfff700 1 -- 172.21.15.99:0/1719190675 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc3ec0535c0 con 0x7fc3f415df70 2023-10-08T00:16:37.560 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.559+0000 7fc3fa736700 1 -- 172.21.15.99:0/1719190675 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc3f415bd20 con 0x7fc3f415df70 2023-10-08T00:16:37.560 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.560+0000 7fc3dbfff700 1 -- 172.21.15.99:0/1719190675 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v127) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc3f415bd20 con 0x7fc3f415df70 2023-10-08T00:16:37.560 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2023-10-08T00:16:37.563 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.562+0000 7fc3fa736700 1 -- 172.21.15.99:0/1719190675 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc3e0066f30 msgr2=0x7fc3e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:37.563 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.562+0000 7fc3fa736700 1 --2- 172.21.15.99:0/1719190675 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc3e0066f30 0x7fc3e00693e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc3e40089b0 tx=0x7fc3e4008040 comp rx=0 tx=0).stop 2023-10-08T00:16:37.563 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.562+0000 7fc3fa736700 1 -- 172.21.15.99:0/1719190675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3f415df70 msgr2=0x7fc3f415c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:37.563 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.562+0000 7fc3fa736700 1 --2- 172.21.15.99:0/1719190675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3f415df70 0x7fc3f415c9a0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fc3f406de60 tx=0x7fc3ec018000 comp rx=0 tx=0).stop 2023-10-08T00:16:37.563 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.562+0000 7fc3fa736700 1 -- 172.21.15.99:0/1719190675 shutdown_connections 2023-10-08T00:16:37.563 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.562+0000 7fc3fa736700 1 --2- 172.21.15.99:0/1719190675 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc3e0066f30 0x7fc3e00693e0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:37.563 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.562+0000 7fc3fa736700 1 --2- 172.21.15.99:0/1719190675 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3f40cc680 0x7fc3f40c8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:37.563 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.562+0000 7fc3fa736700 1 --2- 172.21.15.99:0/1719190675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3f415df70 0x7fc3f415c9a0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:37.563 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.562+0000 7fc3fa736700 1 --2- 172.21.15.99:0/1719190675 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc3f4159de0 0x7fc3f4157ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:37.563 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.562+0000 7fc3fa736700 1 -- 172.21.15.99:0/1719190675 >> 172.21.15.99:0/1719190675 conn(0x7fc3f40cb5c0 msgr2=0x7fc3f4158460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:37.564 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.562+0000 7fc3fa736700 1 -- 172.21.15.99:0/1719190675 shutdown_connections 2023-10-08T00:16:37.564 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.562+0000 7fc3fa736700 1 -- 172.21.15.99:0/1719190675 wait complete. 2023-10-08T00:16:37.575 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:16:37.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.687+0000 7f1e3b31c700 1 Processor -- start 2023-10-08T00:16:37.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.687+0000 7f1e3b31c700 1 -- start start 2023-10-08T00:16:37.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.688+0000 7f1e3b31c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1e34159cf0 0x7f1e3415e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:37.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.688+0000 7f1e3b31c700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e340cca20 con 0x7f1e340cf1c0 2023-10-08T00:16:37.690 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.688+0000 7f1e3b31c700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e3415ea40 con 0x7f1e3415a0c0 2023-10-08T00:16:37.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.688+0000 7f1e3b31c700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e34067660 con 0x7f1e34159cf0 2023-10-08T00:16:37.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.688+0000 7f1e390b8700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1e34159cf0 0x7f1e3415e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:37.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.688+0000 7f1e398b9700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f1e340cf1c0 0x7f1e34068390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50824/0 (socket says 172.21.15.99:50824) 2023-10-08T00:16:37.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.688+0000 7f1e398b9700 1 -- 172.21.15.99:0/2555585683 learned_addr learned my addr 172.21.15.99:0/2555585683 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:37.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.689+0000 7f1e390b8700 1 -- 172.21.15.99:0/2555585683 >> v1:172.21.15.191:6789/0 conn(0x7f1e3415a0c0 legacy=0x7f1e340ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:37.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.689+0000 7f1e390b8700 1 -- 172.21.15.99:0/2555585683 >> v1:172.21.15.99:6789/0 conn(0x7f1e340cf1c0 legacy=0x7f1e34068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:37.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.689+0000 7f1e390b8700 1 -- 172.21.15.99:0/2555585683 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e34067f00 con 0x7f1e34159cf0 2023-10-08T00:16:37.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.689+0000 7f1e2bfff700 1 -- 172.21.15.99:0/2555585683 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2237816644 0 0) 0x7f1e340cca20 con 0x7f1e340cf1c0 2023-10-08T00:16:37.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.689+0000 7f1e390b8700 1 --2- 172.21.15.99:0/2555585683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e34159cf0 0x7f1e3415e500 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f1e300098e0 tx=0x7f1e300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac47c31005925088 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:37.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.689+0000 7f1e2bfff700 1 -- 172.21.15.99:0/2555585683 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e30009e80 con 0x7f1e34159cf0 2023-10-08T00:16:37.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.689+0000 7f1e2bfff700 1 -- 172.21.15.99:0/2555585683 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e3000b860 con 0x7f1e34159cf0 2023-10-08T00:16:37.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.689+0000 7f1e2bfff700 1 -- 172.21.15.99:0/2555585683 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3000bbe0 con 0x7f1e34159cf0 2023-10-08T00:16:37.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.689+0000 7f1e3b31c700 1 -- 172.21.15.99:0/2555585683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e34159cf0 msgr2=0x7f1e3415e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:37.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.689+0000 7f1e3b31c700 1 --2- 172.21.15.99:0/2555585683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e34159cf0 0x7f1e3415e500 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f1e300098e0 tx=0x7f1e300091a0 comp rx=0 tx=0).stop 2023-10-08T00:16:37.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.689+0000 7f1e3b31c700 1 -- 172.21.15.99:0/2555585683 shutdown_connections 2023-10-08T00:16:37.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.689+0000 7f1e3b31c700 1 --2- 172.21.15.99:0/2555585683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e34159cf0 0x7f1e3415e500 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:37.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.689+0000 7f1e3b31c700 1 -- 172.21.15.99:0/2555585683 >> 172.21.15.99:0/2555585683 conn(0x7f1e3406c6e0 msgr2=0x7f1e34069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:37.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.690+0000 7f1e3b31c700 1 -- 172.21.15.99:0/2555585683 shutdown_connections 2023-10-08T00:16:37.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.690+0000 7f1e3b31c700 1 -- 172.21.15.99:0/2555585683 wait complete. 2023-10-08T00:16:37.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.691+0000 7f1e3b31c700 1 Processor -- start 2023-10-08T00:16:37.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.691+0000 7f1e3b31c700 1 -- start start 2023-10-08T00:16:37.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.691+0000 7f1e3b31c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e340cf1c0 0x7f1e34155730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:37.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.691+0000 7f1e3b31c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e34159cf0 0x7f1e3416a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:37.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.691+0000 7f1e3b31c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e3415a0c0 0x7f1e340afec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:37.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.691+0000 7f1e3b31c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e341561e0 con 0x7f1e3415a0c0 2023-10-08T00:16:37.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.692+0000 7f1e3b31c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e34156480 con 0x7f1e34159cf0 2023-10-08T00:16:37.696 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.692+0000 7f1e3b31c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e34156760 con 0x7f1e340cf1c0 2023-10-08T00:16:37.696 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.692+0000 7f1e398b9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e3415a0c0 0x7f1e340afec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:37.696 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.692+0000 7f1e398b9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e3415a0c0 0x7f1e340afec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39986/0 (socket says 172.21.15.99:39986) 2023-10-08T00:16:37.696 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.692+0000 7f1e398b9700 1 -- 172.21.15.99:0/3912188083 learned_addr learned my addr 172.21.15.99:0/3912188083 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:37.696 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.692+0000 7f1e390b8700 1 --2- 172.21.15.99:0/3912188083 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e340cf1c0 0x7f1e34155730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:37.696 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.692+0000 7f1e398b9700 1 -- 172.21.15.99:0/3912188083 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e340cf1c0 msgr2=0x7f1e34155730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:37.696 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.692+0000 7f1e388b7700 1 --2- 172.21.15.99:0/3912188083 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e34159cf0 0x7f1e3416a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:37.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.692+0000 7f1e398b9700 1 --2- 172.21.15.99:0/3912188083 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e340cf1c0 0x7f1e34155730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:37.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.692+0000 7f1e398b9700 1 -- 172.21.15.99:0/3912188083 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e34159cf0 msgr2=0x7f1e3416a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:37.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.692+0000 7f1e398b9700 1 --2- 172.21.15.99:0/3912188083 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e34159cf0 0x7f1e3416a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:37.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.692+0000 7f1e398b9700 1 -- 172.21.15.99:0/3912188083 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e30009420 con 0x7f1e3415a0c0 2023-10-08T00:16:37.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.692+0000 7f1e398b9700 1 --2- 172.21.15.99:0/3912188083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e3415a0c0 0x7f1e340afec0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c00cc90 tx=0x7f1e2c001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:37.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.692+0000 7f1e297fa700 1 -- 172.21.15.99:0/3912188083 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c00cea0 con 0x7f1e3415a0c0 2023-10-08T00:16:37.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.692+0000 7f1e297fa700 1 -- 172.21.15.99:0/3912188083 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e2c00ab70 con 0x7f1e3415a0c0 2023-10-08T00:16:37.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.692+0000 7f1e3b31c700 1 -- 172.21.15.99:0/3912188083 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e34156a40 con 0x7f1e3415a0c0 2023-10-08T00:16:37.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.692+0000 7f1e297fa700 1 -- 172.21.15.99:0/3912188083 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c0070f0 con 0x7f1e3415a0c0 2023-10-08T00:16:37.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.692+0000 7f1e3b31c700 1 -- 172.21.15.99:0/3912188083 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e34155c70 con 0x7f1e3415a0c0 2023-10-08T00:16:37.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.693+0000 7f1e3b31c700 1 -- 172.21.15.99:0/3912188083 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e3404c3c0 con 0x7f1e3415a0c0 2023-10-08T00:16:37.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.694+0000 7f1e297fa700 1 -- 172.21.15.99:0/3912188083 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1e2c0074a0 con 0x7f1e3415a0c0 2023-10-08T00:16:37.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.694+0000 7f1e297fa700 1 --2- 172.21.15.99:0/3912188083 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1e18066f30 0x7f1e180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:37.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.694+0000 7f1e297fa700 1 -- 172.21.15.99:0/3912188083 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1e2c0849b0 con 0x7f1e3415a0c0 2023-10-08T00:16:37.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.694+0000 7f1e390b8700 1 --2- 172.21.15.99:0/3912188083 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1e18066f30 0x7f1e180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:37.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.695+0000 7f1e390b8700 1 --2- 172.21.15.99:0/3912188083 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1e18066f30 0x7f1e180693e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f1e300091e0 tx=0x7f1e30016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:37.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.698+0000 7f1e297fa700 1 -- 172.21.15.99:0/3912188083 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1e2c051ca0 con 0x7f1e3415a0c0 2023-10-08T00:16:37.921 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.921+0000 7f1e3b31c700 1 -- 172.21.15.99:0/3912188083 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1e34156c30 con 0x7f1e3415a0c0 2023-10-08T00:16:37.922 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.922+0000 7f1e297fa700 1 -- 172.21.15.99:0/3912188083 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f1e2c0552c0 con 0x7f1e3415a0c0 2023-10-08T00:16:37.922 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:16:37.922 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":56,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5627,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/2782755379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":2782755379},{"type":"v1","addr":"172.21.15.191:6837","nonce":2782755379}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6166,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/2980541392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2980541392},{"type":"v1","addr":"172.21.15.99:6834","nonce":2980541392}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6184,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/1342206989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1342206989},{"type":"v1","addr":"172.21.15.191:6836","nonce":1342206989}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:16:21.527609+0000","modified":"2023-10-08T00:16:25.185437+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6747},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6747":{"gid":6747,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/1495315849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":1495315849},{"type":"v1","addr":"172.21.15.99:6835","nonce":1495315849}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-08T00:16:37.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.924+0000 7f1e3b31c700 1 -- 172.21.15.99:0/3912188083 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1e18066f30 msgr2=0x7f1e180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:37.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.924+0000 7f1e3b31c700 1 --2- 172.21.15.99:0/3912188083 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1e18066f30 0x7f1e180693e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f1e300091e0 tx=0x7f1e30016000 comp rx=0 tx=0).stop 2023-10-08T00:16:37.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.924+0000 7f1e3b31c700 1 -- 172.21.15.99:0/3912188083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e3415a0c0 msgr2=0x7f1e340afec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:37.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.924+0000 7f1e3b31c700 1 --2- 172.21.15.99:0/3912188083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e3415a0c0 0x7f1e340afec0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c00cc90 tx=0x7f1e2c001a80 comp rx=0 tx=0).stop 2023-10-08T00:16:37.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.924+0000 7f1e3b31c700 1 -- 172.21.15.99:0/3912188083 shutdown_connections 2023-10-08T00:16:37.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.924+0000 7f1e3b31c700 1 --2- 172.21.15.99:0/3912188083 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1e18066f30 0x7f1e180693e0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:37.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.924+0000 7f1e3b31c700 1 --2- 172.21.15.99:0/3912188083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e3415a0c0 0x7f1e340afec0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:37.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.924+0000 7f1e3b31c700 1 --2- 172.21.15.99:0/3912188083 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e340cf1c0 0x7f1e34155730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:37.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.924+0000 7f1e3b31c700 1 --2- 172.21.15.99:0/3912188083 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e34159cf0 0x7f1e3416a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:37.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.924+0000 7f1e3b31c700 1 -- 172.21.15.99:0/3912188083 >> 172.21.15.99:0/3912188083 conn(0x7f1e3406c6e0 msgr2=0x7f1e34069560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:37.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.924+0000 7f1e3b31c700 1 -- 172.21.15.99:0/3912188083 shutdown_connections 2023-10-08T00:16:37.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:37.924+0000 7f1e3b31c700 1 -- 172.21.15.99:0/3912188083 wait complete. 2023-10-08T00:16:37.926 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 56 2023-10-08T00:16:37.936 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:16:38.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.049+0000 7fc2c9b4f700 1 Processor -- start 2023-10-08T00:16:38.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.049+0000 7fc2c9b4f700 1 -- start start 2023-10-08T00:16:38.053 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.050+0000 7fc2c9b4f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc2c40c75c0 0x7fc2c40c8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:38.053 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.050+0000 7fc2c9b4f700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc2c40abeb0 0x7fc2c40c8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:38.054 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.050+0000 7fc2c9b4f700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2c40c9cb0 con 0x7fc2c40abae0 2023-10-08T00:16:38.054 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.050+0000 7fc2c9b4f700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2c40c8e80 con 0x7fc2c40abeb0 2023-10-08T00:16:38.054 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.050+0000 7fc2c9b4f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c40c90a0 con 0x7fc2c40c75c0 2023-10-08T00:16:38.054 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.050+0000 7fc2c37fe700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fc2c40abae0 0x7fc2c40c6eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50830/0 (socket says 172.21.15.99:50830) 2023-10-08T00:16:38.054 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.050+0000 7fc2c37fe700 1 -- 172.21.15.99:0/66161426 learned_addr learned my addr 172.21.15.99:0/66161426 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:38.054 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.050+0000 7fc2c3fff700 1 --2- 172.21.15.99:0/66161426 >> v2:172.21.15.191:3300/0 conn(0x7fc2c40abeb0 0x7fc2c40c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:38.054 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.050+0000 7fc2c2ffd700 1 --2- 172.21.15.99:0/66161426 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc2c40c75c0 0x7fc2c40c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:38.054 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.051+0000 7fc2c27fc700 1 -- 172.21.15.99:0/66161426 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4222287640 0 0) 0x7fc2c40c9cb0 con 0x7fc2c40abae0 2023-10-08T00:16:38.054 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.051+0000 7fc2c27fc700 1 -- 172.21.15.99:0/66161426 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2ac003570 con 0x7fc2c40abae0 2023-10-08T00:16:38.054 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.051+0000 7fc2c27fc700 1 -- 172.21.15.99:0/66161426 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2411817786 0 0) 0x7fc2ac003570 con 0x7fc2c40abae0 2023-10-08T00:16:38.054 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.051+0000 7fc2c27fc700 1 -- 172.21.15.99:0/66161426 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc2c40c75c0 msgr2=0x7fc2c40c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:38.055 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.051+0000 7fc2c27fc700 1 --2- 172.21.15.99:0/66161426 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc2c40c75c0 0x7fc2c40c8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:38.055 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.051+0000 7fc2c27fc700 1 -- 172.21.15.99:0/66161426 >> v2:172.21.15.191:3300/0 conn(0x7fc2c40abeb0 msgr2=0x7fc2c40c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:38.055 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.051+0000 7fc2c27fc700 1 --2- 172.21.15.99:0/66161426 >> v2:172.21.15.191:3300/0 conn(0x7fc2c40abeb0 0x7fc2c40c8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:38.055 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.051+0000 7fc2c27fc700 1 -- 172.21.15.99:0/66161426 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c40c9380 con 0x7fc2c40abae0 2023-10-08T00:16:38.055 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.051+0000 7fc2c27fc700 1 -- 172.21.15.99:0/66161426 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7fc2b40017f0 con 0x7fc2c40abae0 2023-10-08T00:16:38.055 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.051+0000 7fc2c3fff700 1 --2- 172.21.15.99:0/66161426 >> v2:172.21.15.191:3300/0 conn(0x7fc2c40abeb0 0x7fc2c40c8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:16:38.055 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.051+0000 7fc2c27fc700 1 -- 172.21.15.99:0/66161426 >> v1:172.21.15.99:6789/0 conn(0x7fc2c40abae0 legacy=0x7fc2c40c6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:38.055 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.051+0000 7fc2c27fc700 1 --2- 172.21.15.99:0/66161426 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc2c40c75c0 0x7fc2ac003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:38.055 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.051+0000 7fc2c27fc700 1 --2- 172.21.15.99:0/66161426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2ac003fe0 0x7fc2ac004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:38.055 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.051+0000 7fc2c27fc700 1 --2- 172.21.15.99:0/66161426 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2ac004d50 0x7fc2ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:38.055 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.051+0000 7fc2c27fc700 1 -- 172.21.15.99:0/66161426 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c40c8e80 con 0x7fc2ac003fe0 2023-10-08T00:16:38.055 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.051+0000 7fc2c27fc700 1 -- 172.21.15.99:0/66161426 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c40c90a0 con 0x7fc2ac004d50 2023-10-08T00:16:38.056 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.051+0000 7fc2c27fc700 1 -- 172.21.15.99:0/66161426 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ac00dca0 con 0x7fc2c40c75c0 2023-10-08T00:16:38.056 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.051+0000 7fc2c2ffd700 1 --2- 172.21.15.99:0/66161426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2ac003fe0 0x7fc2ac004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:38.056 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.051+0000 7fc2c37fe700 1 --2- 172.21.15.99:0/66161426 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc2c40c75c0 0x7fc2ac003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:38.056 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.051+0000 7fc2c3fff700 1 --2- 172.21.15.99:0/66161426 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2ac004d50 0x7fc2ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:38.056 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.051+0000 7fc2c2ffd700 1 -- 172.21.15.99:0/66161426 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc2c40c75c0 msgr2=0x7fc2ac003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:38.056 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.051+0000 7fc2c2ffd700 1 --2- 172.21.15.99:0/66161426 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc2c40c75c0 0x7fc2ac003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:38.056 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.051+0000 7fc2c2ffd700 1 -- 172.21.15.99:0/66161426 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2ac004d50 msgr2=0x7fc2ac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:38.056 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.051+0000 7fc2c2ffd700 1 --2- 172.21.15.99:0/66161426 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2ac004d50 0x7fc2ac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:38.056 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.051+0000 7fc2c2ffd700 1 -- 172.21.15.99:0/66161426 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2ac003570 con 0x7fc2ac003fe0 2023-10-08T00:16:38.056 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.051+0000 7fc2c37fe700 1 --2- 172.21.15.99:0/66161426 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc2c40c75c0 0x7fc2ac003a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:38.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.051+0000 7fc2c2ffd700 1 --2- 172.21.15.99:0/66161426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2ac003fe0 0x7fc2ac004810 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8009c00 tx=0x7fc2b80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8f2880871bb6d0a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:38.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.052+0000 7fc2c27fc700 1 -- 172.21.15.99:0/66161426 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b800e6b0 con 0x7fc2ac003fe0 2023-10-08T00:16:38.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.052+0000 7fc2c27fc700 1 -- 172.21.15.99:0/66161426 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2b800e810 con 0x7fc2ac003fe0 2023-10-08T00:16:38.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.052+0000 7fc2c27fc700 1 -- 172.21.15.99:0/66161426 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b800ebd0 con 0x7fc2ac003fe0 2023-10-08T00:16:38.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.052+0000 7fc2c9b4f700 1 -- 172.21.15.99:0/66161426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2ac003fe0 msgr2=0x7fc2ac004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:38.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.052+0000 7fc2c9b4f700 1 --2- 172.21.15.99:0/66161426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2ac003fe0 0x7fc2ac004810 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8009c00 tx=0x7fc2b80091a0 comp rx=0 tx=0).stop 2023-10-08T00:16:38.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.052+0000 7fc2c3fff700 1 -- 172.21.15.99:0/66161426 reap_dead start 2023-10-08T00:16:38.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.052+0000 7fc2c9b4f700 1 -- 172.21.15.99:0/66161426 shutdown_connections 2023-10-08T00:16:38.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.052+0000 7fc2c9b4f700 1 -- 172.21.15.99:0/66161426 >> 172.21.15.99:0/66161426 conn(0x7fc2c4067660 msgr2=0x7fc2c4067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:38.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.052+0000 7fc2c9b4f700 1 -- 172.21.15.99:0/66161426 shutdown_connections 2023-10-08T00:16:38.058 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.052+0000 7fc2c9b4f700 1 -- 172.21.15.99:0/66161426 wait complete. 2023-10-08T00:16:38.060 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.053+0000 7fc2c9b4f700 1 Processor -- start 2023-10-08T00:16:38.060 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.053+0000 7fc2c9b4f700 1 -- start start 2023-10-08T00:16:38.060 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.054+0000 7fc2c9b4f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2ac003fe0 0x7fc2c40af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:38.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.054+0000 7fc2c9b4f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2c4153d00 0x7fc2c41572b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:38.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.054+0000 7fc2c9b4f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc2c4154170 0x7fc2c40ca440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:38.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.054+0000 7fc2c9b4f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c4156e20 con 0x7fc2c4153d00 2023-10-08T00:16:38.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.054+0000 7fc2c9b4f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c40b2420 con 0x7fc2ac003fe0 2023-10-08T00:16:38.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.054+0000 7fc2c9b4f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c4156220 con 0x7fc2c4154170 2023-10-08T00:16:38.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.054+0000 7fc2c2ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2c4153d00 0x7fc2c41572b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:38.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.054+0000 7fc2c2ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2c4153d00 0x7fc2c41572b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40002/0 (socket says 172.21.15.99:40002) 2023-10-08T00:16:38.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.054+0000 7fc2c2ffd700 1 -- 172.21.15.99:0/201564818 learned_addr learned my addr 172.21.15.99:0/201564818 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:38.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.054+0000 7fc2c3fff700 1 --2- 172.21.15.99:0/201564818 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc2c4154170 0x7fc2c40ca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:38.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.054+0000 7fc2c37fe700 1 --2- 172.21.15.99:0/201564818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2ac003fe0 0x7fc2c40af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:38.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.054+0000 7fc2c2ffd700 1 -- 172.21.15.99:0/201564818 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc2c4154170 msgr2=0x7fc2c40ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:38.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.054+0000 7fc2c2ffd700 1 --2- 172.21.15.99:0/201564818 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc2c4154170 0x7fc2c40ca440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:38.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.054+0000 7fc2c2ffd700 1 -- 172.21.15.99:0/201564818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2ac003fe0 msgr2=0x7fc2c40af6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:38.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.054+0000 7fc2c2ffd700 1 --2- 172.21.15.99:0/201564818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2ac003fe0 0x7fc2c40af6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:38.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.054+0000 7fc2c2ffd700 1 -- 172.21.15.99:0/201564818 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c41567d0 con 0x7fc2c4153d00 2023-10-08T00:16:38.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.054+0000 7fc2c3fff700 1 --2- 172.21.15.99:0/201564818 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc2c4154170 0x7fc2c40ca440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:38.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.054+0000 7fc2c37fe700 1 --2- 172.21.15.99:0/201564818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2ac003fe0 0x7fc2c40af6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:38.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.055+0000 7fc2c2ffd700 1 --2- 172.21.15.99:0/201564818 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2c4153d00 0x7fc2c41572b0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac004730 tx=0x7fc2b801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:38.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.055+0000 7fc2ab7fe700 1 -- 172.21.15.99:0/201564818 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b8022070 con 0x7fc2c4153d00 2023-10-08T00:16:38.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.055+0000 7fc2ab7fe700 1 -- 172.21.15.99:0/201564818 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2b800e980 con 0x7fc2c4153d00 2023-10-08T00:16:38.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.055+0000 7fc2ab7fe700 1 -- 172.21.15.99:0/201564818 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b8027470 con 0x7fc2c4153d00 2023-10-08T00:16:38.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.055+0000 7fc2c9b4f700 1 -- 172.21.15.99:0/201564818 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2c4156e20 con 0x7fc2c4153d00 2023-10-08T00:16:38.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.055+0000 7fc2c9b4f700 1 -- 172.21.15.99:0/201564818 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2c40afc00 con 0x7fc2c4153d00 2023-10-08T00:16:38.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.056+0000 7fc2c9b4f700 1 -- 172.21.15.99:0/201564818 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2c4156630 con 0x7fc2c4153d00 2023-10-08T00:16:38.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.056+0000 7fc2ab7fe700 1 -- 172.21.15.99:0/201564818 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc2b801c450 con 0x7fc2c4153d00 2023-10-08T00:16:38.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.056+0000 7fc2ab7fe700 1 --2- 172.21.15.99:0/201564818 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc2b0066f30 0x7fc2b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:38.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.057+0000 7fc2ab7fe700 1 -- 172.21.15.99:0/201564818 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc2b80860d0 con 0x7fc2c4153d00 2023-10-08T00:16:38.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.057+0000 7fc2c37fe700 1 --2- 172.21.15.99:0/201564818 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc2b0066f30 0x7fc2b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:38.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.057+0000 7fc2c37fe700 1 --2- 172.21.15.99:0/201564818 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc2b0066f30 0x7fc2b00693e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4001190 tx=0x7fc2b4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:38.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.061+0000 7fc2ab7fe700 1 -- 172.21.15.99:0/201564818 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc2c4156630 con 0x7fc2c4153d00 2023-10-08T00:16:38.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.251+0000 7fc2c9b4f700 1 -- 172.21.15.99:0/201564818 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc2c4156c70 con 0x7fc2c4153d00 2023-10-08T00:16:38.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.252+0000 7fc2ab7fe700 1 -- 172.21.15.99:0/201564818 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+15372 (secure 0 0 0) 0x7fc2b8053400 con 0x7fc2c4153d00 2023-10-08T00:16:38.253 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:16:38.253 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":127,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:16:33.221921+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:16:19.856259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-08T00:16:20.494851+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:16:38.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.254+0000 7fc2c9b4f700 1 -- 172.21.15.99:0/201564818 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc2b0066f30 msgr2=0x7fc2b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:38.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.254+0000 7fc2c9b4f700 1 --2- 172.21.15.99:0/201564818 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc2b0066f30 0x7fc2b00693e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4001190 tx=0x7fc2b4009040 comp rx=0 tx=0).stop 2023-10-08T00:16:38.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.254+0000 7fc2c9b4f700 1 -- 172.21.15.99:0/201564818 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2c4153d00 msgr2=0x7fc2c41572b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:38.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.254+0000 7fc2c9b4f700 1 --2- 172.21.15.99:0/201564818 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2c4153d00 0x7fc2c41572b0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac004730 tx=0x7fc2b801a000 comp rx=0 tx=0).stop 2023-10-08T00:16:38.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.255+0000 7fc2c9b4f700 1 -- 172.21.15.99:0/201564818 shutdown_connections 2023-10-08T00:16:38.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.255+0000 7fc2c9b4f700 1 --2- 172.21.15.99:0/201564818 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc2b0066f30 0x7fc2b00693e0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:38.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.255+0000 7fc2c9b4f700 1 --2- 172.21.15.99:0/201564818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2ac003fe0 0x7fc2c40af6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:38.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.255+0000 7fc2c9b4f700 1 --2- 172.21.15.99:0/201564818 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc2c4154170 0x7fc2c40ca440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:38.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.255+0000 7fc2c9b4f700 1 --2- 172.21.15.99:0/201564818 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2c4153d00 0x7fc2c41572b0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:38.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.255+0000 7fc2c9b4f700 1 -- 172.21.15.99:0/201564818 >> 172.21.15.99:0/201564818 conn(0x7fc2c4067660 msgr2=0x7fc2c4158fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:38.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.255+0000 7fc2c9b4f700 1 -- 172.21.15.99:0/201564818 shutdown_connections 2023-10-08T00:16:38.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.255+0000 7fc2c9b4f700 1 -- 172.21.15.99:0/201564818 wait complete. 2023-10-08T00:16:38.267 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T00:16:38.267 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:16:38.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.381+0000 7fd86f799700 1 Processor -- start 2023-10-08T00:16:38.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.381+0000 7fd86f799700 1 -- start start 2023-10-08T00:16:38.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.382+0000 7fd86f799700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd8681577d0 0x7fd8680cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:38.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.382+0000 7fd86f799700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd868157ba0 0x7fd86806a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:38.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.382+0000 7fd86f799700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd86815c360 con 0x7fd86815b0d0 2023-10-08T00:16:38.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.382+0000 7fd86f799700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd86815b790 con 0x7fd8681577d0 2023-10-08T00:16:38.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.382+0000 7fd86f799700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd86815d800 con 0x7fd868157ba0 2023-10-08T00:16:38.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.382+0000 7fd86dd36700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fd86815b0d0 0x7fd86815da90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50846/0 (socket says 172.21.15.99:50846) 2023-10-08T00:16:38.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.382+0000 7fd86dd36700 1 -- 172.21.15.99:0/17325568 learned_addr learned my addr 172.21.15.99:0/17325568 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:38.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.383+0000 7fd86cd34700 1 --2- 172.21.15.99:0/17325568 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd868157ba0 0x7fd86806a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:38.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.383+0000 7fd86d535700 1 --2- 172.21.15.99:0/17325568 >> v2:172.21.15.191:3300/0 conn(0x7fd8681577d0 0x7fd8680cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:38.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.383+0000 7fd85ffff700 1 -- 172.21.15.99:0/17325568 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2503088452 0 0) 0x7fd86815c360 con 0x7fd86815b0d0 2023-10-08T00:16:38.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.383+0000 7fd85ffff700 1 -- 172.21.15.99:0/17325568 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd854003570 con 0x7fd86815b0d0 2023-10-08T00:16:38.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.383+0000 7fd85ffff700 1 -- 172.21.15.99:0/17325568 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 833176817 0 0) 0x7fd854003570 con 0x7fd86815b0d0 2023-10-08T00:16:38.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.383+0000 7fd85ffff700 1 -- 172.21.15.99:0/17325568 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd868157ba0 msgr2=0x7fd86806a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:38.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.383+0000 7fd85ffff700 1 --2- 172.21.15.99:0/17325568 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd868157ba0 0x7fd86806a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:38.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.383+0000 7fd85ffff700 1 -- 172.21.15.99:0/17325568 >> v2:172.21.15.191:3300/0 conn(0x7fd8681577d0 msgr2=0x7fd8680cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:38.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.383+0000 7fd85ffff700 1 --2- 172.21.15.99:0/17325568 >> v2:172.21.15.191:3300/0 conn(0x7fd8681577d0 0x7fd8680cd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:38.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.383+0000 7fd85ffff700 1 -- 172.21.15.99:0/17325568 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd86815ba30 con 0x7fd86815b0d0 2023-10-08T00:16:38.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.383+0000 7fd85ffff700 1 -- 172.21.15.99:0/17325568 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7fd8600017f0 con 0x7fd86815b0d0 2023-10-08T00:16:38.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.383+0000 7fd85ffff700 1 -- 172.21.15.99:0/17325568 >> v1:172.21.15.99:6789/0 conn(0x7fd86815b0d0 legacy=0x7fd86815da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:38.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.383+0000 7fd85ffff700 1 --2- 172.21.15.99:0/17325568 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd868157ba0 0x7fd854003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:38.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.383+0000 7fd85ffff700 1 --2- 172.21.15.99:0/17325568 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd854003fe0 0x7fd854004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:38.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.383+0000 7fd85ffff700 1 --2- 172.21.15.99:0/17325568 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd854004d50 0x7fd8540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:38.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.383+0000 7fd85ffff700 1 -- 172.21.15.99:0/17325568 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd86815b790 con 0x7fd854004d50 2023-10-08T00:16:38.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.383+0000 7fd85ffff700 1 -- 172.21.15.99:0/17325568 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd86815d800 con 0x7fd854003fe0 2023-10-08T00:16:38.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.383+0000 7fd85ffff700 1 -- 172.21.15.99:0/17325568 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd85400ba90 con 0x7fd868157ba0 2023-10-08T00:16:38.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.383+0000 7fd86dd36700 1 --2- 172.21.15.99:0/17325568 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd854004d50 0x7fd8540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:38.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.383+0000 7fd86cd34700 1 --2- 172.21.15.99:0/17325568 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd854003fe0 0x7fd854004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:38.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.383+0000 7fd86d535700 1 --2- 172.21.15.99:0/17325568 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd868157ba0 0x7fd854003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:38.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.383+0000 7fd86dd36700 1 -- 172.21.15.99:0/17325568 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd868157ba0 msgr2=0x7fd854003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:38.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.383+0000 7fd86dd36700 1 --2- 172.21.15.99:0/17325568 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd868157ba0 0x7fd854003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:38.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.383+0000 7fd86dd36700 1 -- 172.21.15.99:0/17325568 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd854003fe0 msgr2=0x7fd854004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:38.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.383+0000 7fd86dd36700 1 --2- 172.21.15.99:0/17325568 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd854003fe0 0x7fd854004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:38.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.383+0000 7fd86dd36700 1 -- 172.21.15.99:0/17325568 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd854003570 con 0x7fd854004d50 2023-10-08T00:16:38.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.383+0000 7fd86d535700 1 --2- 172.21.15.99:0/17325568 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd868157ba0 0x7fd854003a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:38.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.384+0000 7fd86dd36700 1 --2- 172.21.15.99:0/17325568 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd854004d50 0x7fd8540051c0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fd86000a970 tx=0x7fd860002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae250c3f4bfcc8ad server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:38.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.384+0000 7fd85ffff700 1 -- 172.21.15.99:0/17325568 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd860012440 con 0x7fd854004d50 2023-10-08T00:16:38.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.384+0000 7fd85ffff700 1 -- 172.21.15.99:0/17325568 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8600125c0 con 0x7fd854004d50 2023-10-08T00:16:38.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.384+0000 7fd85ffff700 1 -- 172.21.15.99:0/17325568 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd860012980 con 0x7fd854004d50 2023-10-08T00:16:38.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.384+0000 7fd86f799700 1 -- 172.21.15.99:0/17325568 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd854004d50 msgr2=0x7fd8540051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:38.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.384+0000 7fd86f799700 1 --2- 172.21.15.99:0/17325568 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd854004d50 0x7fd8540051c0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fd86000a970 tx=0x7fd860002210 comp rx=0 tx=0).stop 2023-10-08T00:16:38.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.384+0000 7fd86dd36700 1 -- 172.21.15.99:0/17325568 reap_dead start 2023-10-08T00:16:38.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.384+0000 7fd86f799700 1 -- 172.21.15.99:0/17325568 shutdown_connections 2023-10-08T00:16:38.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.384+0000 7fd86f799700 1 -- 172.21.15.99:0/17325568 >> 172.21.15.99:0/17325568 conn(0x7fd868068ce0 msgr2=0x7fd8680690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:38.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.384+0000 7fd86f799700 1 -- 172.21.15.99:0/17325568 shutdown_connections 2023-10-08T00:16:38.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.384+0000 7fd86f799700 1 -- 172.21.15.99:0/17325568 wait complete. 2023-10-08T00:16:38.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.385+0000 7fd86f799700 1 Processor -- start 2023-10-08T00:16:38.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.385+0000 7fd86f799700 1 -- start start 2023-10-08T00:16:38.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.386+0000 7fd86f799700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd854004d50 0x7fd8680a8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:38.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.386+0000 7fd86f799700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd86816d890 0x7fd86816a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:38.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.386+0000 7fd86f799700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8680b1390 0x7fd86816dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:38.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.386+0000 7fd86f799700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8680a86c0 con 0x7fd854004d50 2023-10-08T00:16:38.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.386+0000 7fd86f799700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8680b1950 con 0x7fd8680b1390 2023-10-08T00:16:38.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.386+0000 7fd86f799700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8680b1c30 con 0x7fd86816d890 2023-10-08T00:16:38.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.386+0000 7fd86d535700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd854004d50 0x7fd8680a8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:38.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.386+0000 7fd86d535700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd854004d50 0x7fd8680a8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40018/0 (socket says 172.21.15.99:40018) 2023-10-08T00:16:38.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.386+0000 7fd86d535700 1 -- 172.21.15.99:0/3463516060 learned_addr learned my addr 172.21.15.99:0/3463516060 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:38.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.387+0000 7fd86cd34700 1 --2- 172.21.15.99:0/3463516060 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd86816d890 0x7fd86816a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:38.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.387+0000 7fd86dd36700 1 --2- 172.21.15.99:0/3463516060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8680b1390 0x7fd86816dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:38.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.387+0000 7fd86d535700 1 -- 172.21.15.99:0/3463516060 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd86816d890 msgr2=0x7fd86816a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:38.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.387+0000 7fd86d535700 1 --2- 172.21.15.99:0/3463516060 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd86816d890 0x7fd86816a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:38.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.387+0000 7fd86d535700 1 -- 172.21.15.99:0/3463516060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8680b1390 msgr2=0x7fd86816dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:38.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.387+0000 7fd86d535700 1 --2- 172.21.15.99:0/3463516060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8680b1390 0x7fd86816dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:38.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.387+0000 7fd86d535700 1 -- 172.21.15.99:0/3463516060 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8680b1f10 con 0x7fd854004d50 2023-10-08T00:16:38.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.387+0000 7fd86d535700 1 --2- 172.21.15.99:0/3463516060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd854004d50 0x7fd8680a8180 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fd864011960 tx=0x7fd864008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:38.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.387+0000 7fd85d7fa700 1 -- 172.21.15.99:0/3463516060 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86400f6d0 con 0x7fd854004d50 2023-10-08T00:16:38.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.387+0000 7fd85d7fa700 1 -- 172.21.15.99:0/3463516060 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd86400f830 con 0x7fd854004d50 2023-10-08T00:16:38.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.387+0000 7fd86f799700 1 -- 172.21.15.99:0/3463516060 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8680b21f0 con 0x7fd854004d50 2023-10-08T00:16:38.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.387+0000 7fd85d7fa700 1 -- 172.21.15.99:0/3463516060 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86400fbd0 con 0x7fd854004d50 2023-10-08T00:16:38.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.387+0000 7fd86f799700 1 -- 172.21.15.99:0/3463516060 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd86816ac40 con 0x7fd854004d50 2023-10-08T00:16:38.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.388+0000 7fd86f799700 1 -- 172.21.15.99:0/3463516060 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8680b2840 con 0x7fd854004d50 2023-10-08T00:16:38.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.388+0000 7fd85d7fa700 1 -- 172.21.15.99:0/3463516060 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd864015460 con 0x7fd854004d50 2023-10-08T00:16:38.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.389+0000 7fd85d7fa700 1 --2- 172.21.15.99:0/3463516060 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd84c066f30 0x7fd84c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:38.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.389+0000 7fd85d7fa700 1 -- 172.21.15.99:0/3463516060 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd864084d00 con 0x7fd854004d50 2023-10-08T00:16:38.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.389+0000 7fd86cd34700 1 --2- 172.21.15.99:0/3463516060 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd84c066f30 0x7fd84c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:38.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.389+0000 7fd86cd34700 1 --2- 172.21.15.99:0/3463516060 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd84c066f30 0x7fd84c0693e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd868153ec0 tx=0x7fd858008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:38.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.393+0000 7fd85d7fa700 1 -- 172.21.15.99:0/3463516060 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd864051f70 con 0x7fd854004d50 2023-10-08T00:16:38.618 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.618+0000 7fd86f799700 1 -- 172.21.15.99:0/3463516060 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd86804c3c0 con 0x7fd854004d50 2023-10-08T00:16:38.619 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.618+0000 7fd85d7fa700 1 -- 172.21.15.99:0/3463516060 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4605 (secure 0 0 0) 0x7fd864055590 con 0x7fd854004d50 2023-10-08T00:16:38.619 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:16:38.619 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":56,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5627,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/2782755379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":2782755379},{"type":"v1","addr":"172.21.15.191:6837","nonce":2782755379}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6166,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/2980541392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2980541392},{"type":"v1","addr":"172.21.15.99:6834","nonce":2980541392}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6184,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/1342206989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1342206989},{"type":"v1","addr":"172.21.15.191:6836","nonce":1342206989}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:16:21.527609+0000","modified":"2023-10-08T00:16:25.185437+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6747},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6747":{"gid":6747,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/1495315849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":1495315849},{"type":"v1","addr":"172.21.15.99:6835","nonce":1495315849}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-08T00:16:38.621 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.620+0000 7fd86f799700 1 -- 172.21.15.99:0/3463516060 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd84c066f30 msgr2=0x7fd84c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:38.621 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.620+0000 7fd86f799700 1 --2- 172.21.15.99:0/3463516060 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd84c066f30 0x7fd84c0693e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd868153ec0 tx=0x7fd858008040 comp rx=0 tx=0).stop 2023-10-08T00:16:38.621 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.620+0000 7fd86f799700 1 -- 172.21.15.99:0/3463516060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd854004d50 msgr2=0x7fd8680a8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:38.621 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.620+0000 7fd86f799700 1 --2- 172.21.15.99:0/3463516060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd854004d50 0x7fd8680a8180 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fd864011960 tx=0x7fd864008040 comp rx=0 tx=0).stop 2023-10-08T00:16:38.622 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.620+0000 7fd86f799700 1 -- 172.21.15.99:0/3463516060 shutdown_connections 2023-10-08T00:16:38.622 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.620+0000 7fd86f799700 1 --2- 172.21.15.99:0/3463516060 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd84c066f30 0x7fd84c0693e0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:38.622 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.620+0000 7fd86f799700 1 --2- 172.21.15.99:0/3463516060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8680b1390 0x7fd86816dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:38.622 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.620+0000 7fd86f799700 1 --2- 172.21.15.99:0/3463516060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd854004d50 0x7fd8680a8180 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:38.622 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.620+0000 7fd86f799700 1 --2- 172.21.15.99:0/3463516060 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd86816d890 0x7fd86816a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:38.622 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.620+0000 7fd86f799700 1 -- 172.21.15.99:0/3463516060 >> 172.21.15.99:0/3463516060 conn(0x7fd868068ce0 msgr2=0x7fd86815dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:38.622 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.620+0000 7fd86f799700 1 -- 172.21.15.99:0/3463516060 shutdown_connections 2023-10-08T00:16:38.622 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.620+0000 7fd86f799700 1 -- 172.21.15.99:0/3463516060 wait complete. 2023-10-08T00:16:38.623 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 56 2023-10-08T00:16:38.633 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:16:38.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.746+0000 7f4685f2d700 1 Processor -- start 2023-10-08T00:16:38.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.746+0000 7f4685f2d700 1 -- start start 2023-10-08T00:16:38.749 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.747+0000 7f4685f2d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f468015a2b0 0x7f46800b3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:38.749 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.747+0000 7f4685f2d700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f46800d1c30 0x7f46800cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:38.749 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.747+0000 7f4685f2d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f468015a680 0x7f46800cfc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:38.749 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.747+0000 7f4685f2d700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4680068470 con 0x7f46800d1c30 2023-10-08T00:16:38.749 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.747+0000 7f4685f2d700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46800c5d80 con 0x7f468015a680 2023-10-08T00:16:38.750 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.747+0000 7f4685f2d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46800c6060 con 0x7f468015a2b0 2023-10-08T00:16:38.750 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.747+0000 7f467effd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f46800d1c30 0x7f46800cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:38.750 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.747+0000 7f467effd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f46800d1c30 0x7f46800cf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40020/0 (socket says 172.21.15.99:40020) 2023-10-08T00:16:38.750 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.747+0000 7f467effd700 1 -- 172.21.15.99:0/502263665 learned_addr learned my addr 172.21.15.99:0/502263665 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:38.750 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.747+0000 7f467ffff700 1 --2- 172.21.15.99:0/502263665 >> v2:172.21.15.191:3300/0 conn(0x7f468015a680 0x7f46800cfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:38.750 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.747+0000 7f467f7fe700 1 --2- 172.21.15.99:0/502263665 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f468015a2b0 0x7f46800b3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:38.750 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.748+0000 7f467effd700 1 -- 172.21.15.99:0/502263665 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f468015a2b0 msgr2=0x7f46800b3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:38.750 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.748+0000 7f467effd700 1 --2- 172.21.15.99:0/502263665 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f468015a2b0 0x7f46800b3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:38.750 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.748+0000 7f467effd700 1 -- 172.21.15.99:0/502263665 >> v2:172.21.15.191:3300/0 conn(0x7f468015a680 msgr2=0x7f46800cfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:38.750 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.748+0000 7f467effd700 1 --2- 172.21.15.99:0/502263665 >> v2:172.21.15.191:3300/0 conn(0x7f468015a680 0x7f46800cfc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:38.750 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.748+0000 7f467effd700 1 -- 172.21.15.99:0/502263665 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46800c6340 con 0x7f46800d1c30 2023-10-08T00:16:38.751 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.748+0000 7f467f7fe700 1 --2- 172.21.15.99:0/502263665 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f468015a2b0 0x7f46800b3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:16:38.751 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.748+0000 7f467ffff700 1 --2- 172.21.15.99:0/502263665 >> v2:172.21.15.191:3300/0 conn(0x7f468015a680 0x7f46800cfc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:16:38.751 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.748+0000 7f467effd700 1 --2- 172.21.15.99:0/502263665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46800d1c30 0x7f46800cf710 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f468008a810 tx=0x7f4674009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11970112db335c30 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:38.751 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.748+0000 7f467e7fc700 1 -- 172.21.15.99:0/502263665 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467400b860 con 0x7f46800d1c30 2023-10-08T00:16:38.751 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.748+0000 7f467e7fc700 1 -- 172.21.15.99:0/502263665 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f467400b9c0 con 0x7f46800d1c30 2023-10-08T00:16:38.751 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.748+0000 7f467e7fc700 1 -- 172.21.15.99:0/502263665 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467400bd60 con 0x7f46800d1c30 2023-10-08T00:16:38.751 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.748+0000 7f4685f2d700 1 -- 172.21.15.99:0/502263665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46800d1c30 msgr2=0x7f46800cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:38.751 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.748+0000 7f4685f2d700 1 --2- 172.21.15.99:0/502263665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46800d1c30 0x7f46800cf710 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f468008a810 tx=0x7f4674009250 comp rx=0 tx=0).stop 2023-10-08T00:16:38.751 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.748+0000 7f4685f2d700 1 -- 172.21.15.99:0/502263665 shutdown_connections 2023-10-08T00:16:38.751 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.748+0000 7f4685f2d700 1 --2- 172.21.15.99:0/502263665 >> v2:172.21.15.191:3300/0 conn(0x7f468015a680 0x7f46800cfc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:38.751 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.748+0000 7f4685f2d700 1 --2- 172.21.15.99:0/502263665 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f468015a2b0 0x7f46800b3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:38.751 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.748+0000 7f4685f2d700 1 --2- 172.21.15.99:0/502263665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46800d1c30 0x7f46800cf710 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:38.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.748+0000 7f4685f2d700 1 -- 172.21.15.99:0/502263665 >> 172.21.15.99:0/502263665 conn(0x7f4680067e40 msgr2=0x7f46800d0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:38.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.748+0000 7f4685f2d700 1 -- 172.21.15.99:0/502263665 shutdown_connections 2023-10-08T00:16:38.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.748+0000 7f4685f2d700 1 -- 172.21.15.99:0/502263665 wait complete. 2023-10-08T00:16:38.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.750+0000 7f4685f2d700 1 Processor -- start 2023-10-08T00:16:38.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.750+0000 7f4685f2d700 1 -- start start 2023-10-08T00:16:38.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.750+0000 7f4685f2d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46800d1c30 0x7f468015f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:38.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.750+0000 7f4685f2d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f468015a2b0 0x7f46800b4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:38.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.750+0000 7f4685f2d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f468015a680 0x7f468015bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:38.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.750+0000 7f4685f2d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468015e400 con 0x7f468015a2b0 2023-10-08T00:16:38.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.750+0000 7f4685f2d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468015e6e0 con 0x7f468015a680 2023-10-08T00:16:38.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.750+0000 7f4685f2d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f468015e9c0 con 0x7f46800d1c30 2023-10-08T00:16:38.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.750+0000 7f467effd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f468015a2b0 0x7f46800b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:38.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.751+0000 7f467effd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f468015a2b0 0x7f46800b4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40034/0 (socket says 172.21.15.99:40034) 2023-10-08T00:16:38.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.751+0000 7f467effd700 1 -- 172.21.15.99:0/1949856302 learned_addr learned my addr 172.21.15.99:0/1949856302 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:38.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.751+0000 7f467f7fe700 1 --2- 172.21.15.99:0/1949856302 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46800d1c30 0x7f468015f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:38.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.751+0000 7f467ffff700 1 --2- 172.21.15.99:0/1949856302 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f468015a680 0x7f468015bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:38.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.751+0000 7f467effd700 1 -- 172.21.15.99:0/1949856302 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46800d1c30 msgr2=0x7f468015f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:38.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.751+0000 7f467effd700 1 --2- 172.21.15.99:0/1949856302 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46800d1c30 0x7f468015f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:38.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.751+0000 7f467effd700 1 -- 172.21.15.99:0/1949856302 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f468015a680 msgr2=0x7f468015bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:38.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.751+0000 7f467effd700 1 --2- 172.21.15.99:0/1949856302 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f468015a680 0x7f468015bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:38.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.751+0000 7f467effd700 1 -- 172.21.15.99:0/1949856302 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46740094f0 con 0x7f468015a2b0 2023-10-08T00:16:38.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.751+0000 7f467f7fe700 1 --2- 172.21.15.99:0/1949856302 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46800d1c30 0x7f468015f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:16:38.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.751+0000 7f467ffff700 1 --2- 172.21.15.99:0/1949856302 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f468015a680 0x7f468015bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:16:38.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.751+0000 7f467effd700 1 --2- 172.21.15.99:0/1949856302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f468015a2b0 0x7f46800b4920 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f4680089120 tx=0x7f4674017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:38.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.751+0000 7f46677fe700 1 -- 172.21.15.99:0/1949856302 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4674024440 con 0x7f468015a2b0 2023-10-08T00:16:38.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.751+0000 7f46677fe700 1 -- 172.21.15.99:0/1949856302 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46740245a0 con 0x7f468015a2b0 2023-10-08T00:16:38.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.751+0000 7f46677fe700 1 -- 172.21.15.99:0/1949856302 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4674024960 con 0x7f468015a2b0 2023-10-08T00:16:38.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.751+0000 7f4685f2d700 1 -- 172.21.15.99:0/1949856302 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f468015eca0 con 0x7f468015a2b0 2023-10-08T00:16:38.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.751+0000 7f4685f2d700 1 -- 172.21.15.99:0/1949856302 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f468015d970 con 0x7f468015a2b0 2023-10-08T00:16:38.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.752+0000 7f4685f2d700 1 -- 172.21.15.99:0/1949856302 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f468015f9e0 con 0x7f468015a2b0 2023-10-08T00:16:38.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.752+0000 7f46677fe700 1 -- 172.21.15.99:0/1949856302 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f467401a420 con 0x7f468015a2b0 2023-10-08T00:16:38.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.753+0000 7f46677fe700 1 --2- 172.21.15.99:0/1949856302 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f466c066f30 0x7f466c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:38.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.753+0000 7f46677fe700 1 -- 172.21.15.99:0/1949856302 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4674086290 con 0x7f468015a2b0 2023-10-08T00:16:38.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.753+0000 7f467f7fe700 1 --2- 172.21.15.99:0/1949856302 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f466c066f30 0x7f466c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:38.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.753+0000 7f467f7fe700 1 --2- 172.21.15.99:0/1949856302 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f466c066f30 0x7f466c0693e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f4670008a60 tx=0x7f4670008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:38.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.757+0000 7f46677fe700 1 -- 172.21.15.99:0/1949856302 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4674053500 con 0x7f468015a2b0 2023-10-08T00:16:38.982 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.982+0000 7f4685f2d700 1 -- 172.21.15.99:0/1949856302 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f468015f1c0 con 0x7f468015a2b0 2023-10-08T00:16:38.983 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.982+0000 7f46677fe700 1 -- 172.21.15.99:0/1949856302 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f468015f1c0 con 0x7f468015a2b0 2023-10-08T00:16:38.983 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:16:38.983 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":56,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5627,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/2782755379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":2782755379},{"type":"v1","addr":"172.21.15.191:6837","nonce":2782755379}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6166,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/2980541392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2980541392},{"type":"v1","addr":"172.21.15.99:6834","nonce":2980541392}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6184,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/1342206989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1342206989},{"type":"v1","addr":"172.21.15.191:6836","nonce":1342206989}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:16:21.527609+0000","modified":"2023-10-08T00:16:25.185437+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6747},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6747":{"gid":6747,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/1495315849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":1495315849},{"type":"v1","addr":"172.21.15.99:6835","nonce":1495315849}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-08T00:16:38.985 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.985+0000 7f4685f2d700 1 -- 172.21.15.99:0/1949856302 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f466c066f30 msgr2=0x7f466c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:38.986 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.985+0000 7f4685f2d700 1 --2- 172.21.15.99:0/1949856302 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f466c066f30 0x7f466c0693e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f4670008a60 tx=0x7f4670008040 comp rx=0 tx=0).stop 2023-10-08T00:16:38.986 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.985+0000 7f4685f2d700 1 -- 172.21.15.99:0/1949856302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f468015a2b0 msgr2=0x7f46800b4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:38.986 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.985+0000 7f4685f2d700 1 --2- 172.21.15.99:0/1949856302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f468015a2b0 0x7f46800b4920 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f4680089120 tx=0x7f4674017000 comp rx=0 tx=0).stop 2023-10-08T00:16:38.986 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.985+0000 7f4685f2d700 1 -- 172.21.15.99:0/1949856302 shutdown_connections 2023-10-08T00:16:38.986 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.985+0000 7f4685f2d700 1 --2- 172.21.15.99:0/1949856302 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f466c066f30 0x7f466c0693e0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:38.987 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.985+0000 7f4685f2d700 1 --2- 172.21.15.99:0/1949856302 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f468015a680 0x7f468015bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:38.988 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.985+0000 7f4685f2d700 1 --2- 172.21.15.99:0/1949856302 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46800d1c30 0x7f468015f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:38.988 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.985+0000 7f4685f2d700 1 --2- 172.21.15.99:0/1949856302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f468015a2b0 0x7f46800b4920 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:38.988 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.985+0000 7f4685f2d700 1 -- 172.21.15.99:0/1949856302 >> 172.21.15.99:0/1949856302 conn(0x7f4680067e40 msgr2=0x7f4680158ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:38.988 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.985+0000 7f4685f2d700 1 -- 172.21.15.99:0/1949856302 shutdown_connections 2023-10-08T00:16:38.988 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:38.985+0000 7f4685f2d700 1 -- 172.21.15.99:0/1949856302 wait complete. 2023-10-08T00:16:38.989 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 56 2023-10-08T00:16:38.997 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:16:39.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.110+0000 7f0708ab9700 1 Processor -- start 2023-10-08T00:16:39.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.110+0000 7f0708ab9700 1 -- start start 2023-10-08T00:16:39.113 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.111+0000 7f0708ab9700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f070406ab50 0x7f07040cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:39.113 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.111+0000 7f0708ab9700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f07041438f0 0x7f07040b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:39.113 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.111+0000 7f0708ab9700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07040d0870 con 0x7f0704143520 2023-10-08T00:16:39.113 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.111+0000 7f0708ab9700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07040cdfb0 con 0x7f070406ab50 2023-10-08T00:16:39.113 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.111+0000 7f0708ab9700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07040cfc60 con 0x7f07041438f0 2023-10-08T00:16:39.113 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.111+0000 7f0702ffd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f070406ab50 0x7f07040cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:39.114 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.111+0000 7f07037fe700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f0704143520 0x7f070406a440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50858/0 (socket says 172.21.15.99:50858) 2023-10-08T00:16:39.114 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.111+0000 7f07037fe700 1 -- 172.21.15.99:0/1798709454 learned_addr learned my addr 172.21.15.99:0/1798709454 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:39.114 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.111+0000 7f0703fff700 1 --2- 172.21.15.99:0/1798709454 >> v2:172.21.15.191:3300/0 conn(0x7f07041438f0 0x7f07040b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:39.114 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.111+0000 7f0702ffd700 1 -- 172.21.15.99:0/1798709454 >> v2:172.21.15.191:3300/0 conn(0x7f07041438f0 msgr2=0x7f07040b3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:39.114 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.111+0000 7f0702ffd700 1 --2- 172.21.15.99:0/1798709454 >> v2:172.21.15.191:3300/0 conn(0x7f07041438f0 0x7f07040b3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:39.114 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.111+0000 7f0702ffd700 1 -- 172.21.15.99:0/1798709454 >> v1:172.21.15.99:6789/0 conn(0x7f0704143520 legacy=0x7f070406a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:39.114 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.111+0000 7f0702ffd700 1 -- 172.21.15.99:0/1798709454 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07040cff40 con 0x7f070406ab50 2023-10-08T00:16:39.114 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.111+0000 7f07027fc700 1 -- 172.21.15.99:0/1798709454 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 167797497 0 0) 0x7f07040d0870 con 0x7f0704143520 2023-10-08T00:16:39.114 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.112+0000 7f0702ffd700 1 --2- 172.21.15.99:0/1798709454 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f070406ab50 0x7f07040cf580 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f0704063e50 tx=0x7f06f80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ca5da8086c23fdd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:39.114 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.112+0000 7f07027fc700 1 -- 172.21.15.99:0/1798709454 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f800b6b0 con 0x7f070406ab50 2023-10-08T00:16:39.114 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.112+0000 7f07027fc700 1 -- 172.21.15.99:0/1798709454 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06f800b810 con 0x7f070406ab50 2023-10-08T00:16:39.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.112+0000 7f07027fc700 1 -- 172.21.15.99:0/1798709454 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f800bbb0 con 0x7f070406ab50 2023-10-08T00:16:39.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.112+0000 7f0708ab9700 1 -- 172.21.15.99:0/1798709454 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f070406ab50 msgr2=0x7f07040cf580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:39.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.112+0000 7f0708ab9700 1 --2- 172.21.15.99:0/1798709454 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f070406ab50 0x7f07040cf580 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f0704063e50 tx=0x7f06f80091a0 comp rx=0 tx=0).stop 2023-10-08T00:16:39.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.112+0000 7f0708ab9700 1 -- 172.21.15.99:0/1798709454 shutdown_connections 2023-10-08T00:16:39.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.112+0000 7f0708ab9700 1 --2- 172.21.15.99:0/1798709454 >> v2:172.21.15.191:3300/0 conn(0x7f07041438f0 0x7f07040b3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:39.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.112+0000 7f0708ab9700 1 --2- 172.21.15.99:0/1798709454 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f070406ab50 0x7f07040cf580 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:39.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.112+0000 7f0708ab9700 1 -- 172.21.15.99:0/1798709454 >> 172.21.15.99:0/1798709454 conn(0x7f0704069750 msgr2=0x7f07040b25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:39.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.112+0000 7f0708ab9700 1 -- 172.21.15.99:0/1798709454 shutdown_connections 2023-10-08T00:16:39.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.112+0000 7f0708ab9700 1 -- 172.21.15.99:0/1798709454 wait complete. 2023-10-08T00:16:39.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.113+0000 7f0708ab9700 1 Processor -- start 2023-10-08T00:16:39.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.113+0000 7f0708ab9700 1 -- start start 2023-10-08T00:16:39.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.114+0000 7f0708ab9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f070406ab50 0x7f0704138140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:39.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.114+0000 7f0708ab9700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0704143520 0x7f070413bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:39.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.114+0000 7f0708ab9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07041438f0 0x7f070413ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:39.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.114+0000 7f0708ab9700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07041392a0 con 0x7f07041438f0 2023-10-08T00:16:39.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.114+0000 7f0708ab9700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070413b680 con 0x7f0704143520 2023-10-08T00:16:39.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.114+0000 7f0708ab9700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0704138680 con 0x7f070406ab50 2023-10-08T00:16:39.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.114+0000 7f0703fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07041438f0 0x7f070413ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:39.119 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.114+0000 7f0703fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07041438f0 0x7f070413ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40056/0 (socket says 172.21.15.99:40056) 2023-10-08T00:16:39.119 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.114+0000 7f0703fff700 1 -- 172.21.15.99:0/4179964808 learned_addr learned my addr 172.21.15.99:0/4179964808 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:39.119 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.114+0000 7f0702ffd700 1 --2- 172.21.15.99:0/4179964808 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0704143520 0x7f070413bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:39.119 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.114+0000 7f07037fe700 1 --2- 172.21.15.99:0/4179964808 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f070406ab50 0x7f0704138140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:39.119 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.115+0000 7f0703fff700 1 -- 172.21.15.99:0/4179964808 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f070406ab50 msgr2=0x7f0704138140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:39.119 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.115+0000 7f0703fff700 1 --2- 172.21.15.99:0/4179964808 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f070406ab50 0x7f0704138140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:39.119 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.115+0000 7f0703fff700 1 -- 172.21.15.99:0/4179964808 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0704143520 msgr2=0x7f070413bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:39.119 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.115+0000 7f0703fff700 1 --2- 172.21.15.99:0/4179964808 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0704143520 0x7f070413bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:39.119 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.115+0000 7f0703fff700 1 -- 172.21.15.99:0/4179964808 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06f8009420 con 0x7f07041438f0 2023-10-08T00:16:39.119 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.115+0000 7f07037fe700 1 --2- 172.21.15.99:0/4179964808 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f070406ab50 0x7f0704138140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:39.119 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.115+0000 7f0702ffd700 1 --2- 172.21.15.99:0/4179964808 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0704143520 0x7f070413bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:39.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.115+0000 7f0703fff700 1 --2- 172.21.15.99:0/4179964808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07041438f0 0x7f070413ec10 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f06fc00d890 tx=0x7f06fc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:39.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.115+0000 7f06eb7fe700 1 -- 172.21.15.99:0/4179964808 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06fc0125f0 con 0x7f07041438f0 2023-10-08T00:16:39.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.115+0000 7f06eb7fe700 1 -- 172.21.15.99:0/4179964808 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06fc012770 con 0x7f07041438f0 2023-10-08T00:16:39.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.115+0000 7f0708ab9700 1 -- 172.21.15.99:0/4179964808 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f070413aa70 con 0x7f07041438f0 2023-10-08T00:16:39.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.115+0000 7f06eb7fe700 1 -- 172.21.15.99:0/4179964808 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06fc012b30 con 0x7f07041438f0 2023-10-08T00:16:39.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.115+0000 7f0708ab9700 1 -- 172.21.15.99:0/4179964808 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f070413b030 con 0x7f07041438f0 2023-10-08T00:16:39.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.116+0000 7f0708ab9700 1 -- 172.21.15.99:0/4179964808 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0704089030 con 0x7f07041438f0 2023-10-08T00:16:39.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.116+0000 7f06eb7fe700 1 -- 172.21.15.99:0/4179964808 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f06fc01e460 con 0x7f07041438f0 2023-10-08T00:16:39.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.117+0000 7f06eb7fe700 1 --2- 172.21.15.99:0/4179964808 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f06f0066f30 0x7f06f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:39.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.117+0000 7f06eb7fe700 1 -- 172.21.15.99:0/4179964808 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f06fc084b60 con 0x7f07041438f0 2023-10-08T00:16:39.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.117+0000 7f07037fe700 1 --2- 172.21.15.99:0/4179964808 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f06f0066f30 0x7f06f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:39.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.117+0000 7f07037fe700 1 --2- 172.21.15.99:0/4179964808 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f06f0066f30 0x7f06f00693e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f06f4001560 tx=0x7f06f4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:39.121 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.121+0000 7f06eb7fe700 1 -- 172.21.15.99:0/4179964808 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f06fc051dd0 con 0x7f07041438f0 2023-10-08T00:16:39.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.305+0000 7f0708ab9700 1 -- 172.21.15.99:0/4179964808 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f07040d0220 con 0x7f07041438f0 2023-10-08T00:16:39.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.305+0000 7f06eb7fe700 1 -- 172.21.15.99:0/4179964808 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+15372 (secure 0 0 0) 0x7f06fc0553f0 con 0x7f07041438f0 2023-10-08T00:16:39.306 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:16:39.306 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":127,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:16:33.221921+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:16:19.856259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-08T00:16:20.494851+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:16:39.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.308+0000 7f0708ab9700 1 -- 172.21.15.99:0/4179964808 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f06f0066f30 msgr2=0x7f06f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:39.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.308+0000 7f0708ab9700 1 --2- 172.21.15.99:0/4179964808 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f06f0066f30 0x7f06f00693e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f06f4001560 tx=0x7f06f4009040 comp rx=0 tx=0).stop 2023-10-08T00:16:39.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.308+0000 7f0708ab9700 1 -- 172.21.15.99:0/4179964808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07041438f0 msgr2=0x7f070413ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:39.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.308+0000 7f0708ab9700 1 --2- 172.21.15.99:0/4179964808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07041438f0 0x7f070413ec10 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f06fc00d890 tx=0x7f06fc008040 comp rx=0 tx=0).stop 2023-10-08T00:16:39.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.308+0000 7f0708ab9700 1 -- 172.21.15.99:0/4179964808 shutdown_connections 2023-10-08T00:16:39.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.308+0000 7f0708ab9700 1 --2- 172.21.15.99:0/4179964808 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f06f0066f30 0x7f06f00693e0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:39.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.308+0000 7f0708ab9700 1 --2- 172.21.15.99:0/4179964808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07041438f0 0x7f070413ec10 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:39.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.308+0000 7f0708ab9700 1 --2- 172.21.15.99:0/4179964808 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f070406ab50 0x7f0704138140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:39.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.308+0000 7f0708ab9700 1 --2- 172.21.15.99:0/4179964808 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0704143520 0x7f070413bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:39.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.308+0000 7f0708ab9700 1 -- 172.21.15.99:0/4179964808 >> 172.21.15.99:0/4179964808 conn(0x7f0704069750 msgr2=0x7f070413fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:39.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.308+0000 7f0708ab9700 1 -- 172.21.15.99:0/4179964808 shutdown_connections 2023-10-08T00:16:39.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.308+0000 7f0708ab9700 1 -- 172.21.15.99:0/4179964808 wait complete. 2023-10-08T00:16:39.320 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T00:16:39.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.435+0000 7fde70c5c700 1 Processor -- start 2023-10-08T00:16:39.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.435+0000 7fde70c5c700 1 -- start start 2023-10-08T00:16:39.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.435+0000 7fde70c5c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fde6c157870 0x7fde6c15b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:39.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.435+0000 7fde70c5c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde6c15a620 0x7fde6c15d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:39.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.435+0000 7fde70c5c700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde6c15cbb0 con 0x7fde6c15d580 2023-10-08T00:16:39.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.435+0000 7fde70c5c700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde6c15bfa0 con 0x7fde6c157870 2023-10-08T00:16:39.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.435+0000 7fde70c5c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c15c280 con 0x7fde6c15a620 2023-10-08T00:16:39.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.436+0000 7fde6a59c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fde6c157870 0x7fde6c15b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:39.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.436+0000 7fde6a59c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fde6c157870 0x7fde6c15b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40058/0 (socket says 172.21.15.99:40058) 2023-10-08T00:16:39.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.436+0000 7fde6a59c700 1 -- 172.21.15.99:0/3568571233 learned_addr learned my addr 172.21.15.99:0/3568571233 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:39.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.436+0000 7fde61d9b700 1 --2- 172.21.15.99:0/3568571233 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde6c15a620 0x7fde6c15d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:39.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.436+0000 7fde6a59c700 1 -- 172.21.15.99:0/3568571233 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde6c15a620 msgr2=0x7fde6c15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:39.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.436+0000 7fde6a59c700 1 --2- 172.21.15.99:0/3568571233 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde6c15a620 0x7fde6c15d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:39.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.436+0000 7fde6a59c700 1 -- 172.21.15.99:0/3568571233 >> v1:172.21.15.191:6789/0 conn(0x7fde6c15d580 legacy=0x7fde6c158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:39.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.436+0000 7fde6a59c700 1 -- 172.21.15.99:0/3568571233 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde6c15c560 con 0x7fde6c157870 2023-10-08T00:16:39.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.436+0000 7fde6a59c700 1 --2- 172.21.15.99:0/3568571233 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde6c157870 0x7fde6c15b6a0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fde5c0099b0 tx=0x7fde5c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf5f2c890567e71 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:39.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.436+0000 7fde69d9b700 1 -- 172.21.15.99:0/3568571233 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5c00b860 con 0x7fde6c157870 2023-10-08T00:16:39.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.436+0000 7fde69d9b700 1 -- 172.21.15.99:0/3568571233 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde5c00b9c0 con 0x7fde6c157870 2023-10-08T00:16:39.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.436+0000 7fde69d9b700 1 -- 172.21.15.99:0/3568571233 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5c00bd60 con 0x7fde6c157870 2023-10-08T00:16:39.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.437+0000 7fde70c5c700 1 -- 172.21.15.99:0/3568571233 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde6c157870 msgr2=0x7fde6c15b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:39.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.437+0000 7fde70c5c700 1 --2- 172.21.15.99:0/3568571233 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde6c157870 0x7fde6c15b6a0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fde5c0099b0 tx=0x7fde5c009250 comp rx=0 tx=0).stop 2023-10-08T00:16:39.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.437+0000 7fde70c5c700 1 -- 172.21.15.99:0/3568571233 shutdown_connections 2023-10-08T00:16:39.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.437+0000 7fde70c5c700 1 --2- 172.21.15.99:0/3568571233 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde6c157870 0x7fde6c15b6a0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:39.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.437+0000 7fde70c5c700 1 --2- 172.21.15.99:0/3568571233 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde6c15a620 0x7fde6c15d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:39.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.437+0000 7fde70c5c700 1 -- 172.21.15.99:0/3568571233 >> 172.21.15.99:0/3568571233 conn(0x7fde6c068ce0 msgr2=0x7fde6c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:39.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.437+0000 7fde70c5c700 1 -- 172.21.15.99:0/3568571233 shutdown_connections 2023-10-08T00:16:39.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.437+0000 7fde70c5c700 1 -- 172.21.15.99:0/3568571233 wait complete. 2023-10-08T00:16:39.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.438+0000 7fde70c5c700 1 Processor -- start 2023-10-08T00:16:39.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.438+0000 7fde70c5c700 1 -- start start 2023-10-08T00:16:39.443 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.439+0000 7fde70c5c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde6c157870 0x7fde6c156360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:39.444 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.439+0000 7fde70c5c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde6c15a620 0x7fde6c0b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:39.444 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.439+0000 7fde70c5c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde6c15d580 0x7fde6c0b2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:39.444 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.439+0000 7fde70c5c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c152450 con 0x7fde6c15a620 2023-10-08T00:16:39.444 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.439+0000 7fde70c5c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c152730 con 0x7fde6c15d580 2023-10-08T00:16:39.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.439+0000 7fde70c5c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c152a10 con 0x7fde6c157870 2023-10-08T00:16:39.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.439+0000 7fde61d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde6c15a620 0x7fde6c0b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:39.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.439+0000 7fde61d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde6c15a620 0x7fde6c0b2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40070/0 (socket says 172.21.15.99:40070) 2023-10-08T00:16:39.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.439+0000 7fde61d9b700 1 -- 172.21.15.99:0/43112934 learned_addr learned my addr 172.21.15.99:0/43112934 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:39.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.439+0000 7fde6ad9d700 1 --2- 172.21.15.99:0/43112934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde6c15d580 0x7fde6c0b2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:39.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.439+0000 7fde6a59c700 1 --2- 172.21.15.99:0/43112934 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde6c157870 0x7fde6c156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:39.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.439+0000 7fde61d9b700 1 -- 172.21.15.99:0/43112934 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde6c157870 msgr2=0x7fde6c156360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:39.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.439+0000 7fde61d9b700 1 --2- 172.21.15.99:0/43112934 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde6c157870 0x7fde6c156360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:39.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.439+0000 7fde61d9b700 1 -- 172.21.15.99:0/43112934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde6c15d580 msgr2=0x7fde6c0b2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:39.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.439+0000 7fde61d9b700 1 --2- 172.21.15.99:0/43112934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde6c15d580 0x7fde6c0b2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:39.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.439+0000 7fde61d9b700 1 -- 172.21.15.99:0/43112934 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde5c0094f0 con 0x7fde6c15a620 2023-10-08T00:16:39.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.439+0000 7fde61d9b700 1 --2- 172.21.15.99:0/43112934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde6c15a620 0x7fde6c0b2a90 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fde5400d860 tx=0x7fde54008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:39.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.439+0000 7fde62ffd700 1 -- 172.21.15.99:0/43112934 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde540125f0 con 0x7fde6c15a620 2023-10-08T00:16:39.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.439+0000 7fde62ffd700 1 -- 172.21.15.99:0/43112934 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde54012750 con 0x7fde6c15a620 2023-10-08T00:16:39.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.439+0000 7fde70c5c700 1 -- 172.21.15.99:0/43112934 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde6c152cf0 con 0x7fde6c15a620 2023-10-08T00:16:39.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.439+0000 7fde62ffd700 1 -- 172.21.15.99:0/43112934 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde54012ad0 con 0x7fde6c15a620 2023-10-08T00:16:39.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.439+0000 7fde70c5c700 1 -- 172.21.15.99:0/43112934 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde6c0d1330 con 0x7fde6c15a620 2023-10-08T00:16:39.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.440+0000 7fde70c5c700 1 -- 172.21.15.99:0/43112934 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde6c153210 con 0x7fde6c15a620 2023-10-08T00:16:39.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.441+0000 7fde62ffd700 1 -- 172.21.15.99:0/43112934 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fde5401e460 con 0x7fde6c15a620 2023-10-08T00:16:39.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.441+0000 7fde62ffd700 1 --2- 172.21.15.99:0/43112934 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fde4c066f30 0x7fde4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:39.452 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.441+0000 7fde62ffd700 1 -- 172.21.15.99:0/43112934 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fde54084b90 con 0x7fde6c15a620 2023-10-08T00:16:39.452 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.441+0000 7fde6a59c700 1 --2- 172.21.15.99:0/43112934 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fde4c066f30 0x7fde4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:39.452 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.442+0000 7fde6a59c700 1 --2- 172.21.15.99:0/43112934 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fde4c066f30 0x7fde4c0693e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fde5c0099b0 tx=0x7fde5c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:39.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.445+0000 7fde62ffd700 1 -- 172.21.15.99:0/43112934 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fde6c153210 con 0x7fde6c15a620 2023-10-08T00:16:39.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.671+0000 7fde70c5c700 1 -- 172.21.15.99:0/43112934 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fde6c153210 con 0x7fde6c15a620 2023-10-08T00:16:39.672 INFO:tasks.ceph.mon.a.smithi099.stderr:2023-10-08T00:16:39.672+0000 7f6c60fa2700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T00:16:39.677 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.677+0000 7fde62ffd700 1 -- 172.21.15.99:0/43112934 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v57) v1 ==== 161+0+0 (secure 0 0 0) 0x7fde6c153210 con 0x7fde6c15a620 2023-10-08T00:16:39.678 INFO:tasks.ceph.mds.c.smithi099.stderr: -3331> 2023-10-08T00:16:27.458+0000 7f59355b3700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T00:16:39.680 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.679+0000 7fde70c5c700 1 -- 172.21.15.99:0/43112934 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fde4c066f30 msgr2=0x7fde4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:39.680 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.679+0000 7fde70c5c700 1 --2- 172.21.15.99:0/43112934 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fde4c066f30 0x7fde4c0693e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fde5c0099b0 tx=0x7fde5c017000 comp rx=0 tx=0).stop 2023-10-08T00:16:39.680 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.679+0000 7fde70c5c700 1 -- 172.21.15.99:0/43112934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde6c15a620 msgr2=0x7fde6c0b2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:39.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.679+0000 7fde70c5c700 1 --2- 172.21.15.99:0/43112934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde6c15a620 0x7fde6c0b2a90 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fde5400d860 tx=0x7fde54008040 comp rx=0 tx=0).stop 2023-10-08T00:16:39.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.679+0000 7fde70c5c700 1 -- 172.21.15.99:0/43112934 shutdown_connections 2023-10-08T00:16:39.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.679+0000 7fde70c5c700 1 --2- 172.21.15.99:0/43112934 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fde4c066f30 0x7fde4c0693e0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:39.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.679+0000 7fde70c5c700 1 --2- 172.21.15.99:0/43112934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde6c15d580 0x7fde6c0b2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:39.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.679+0000 7fde70c5c700 1 --2- 172.21.15.99:0/43112934 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde6c157870 0x7fde6c156360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:39.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.679+0000 7fde70c5c700 1 --2- 172.21.15.99:0/43112934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde6c15a620 0x7fde6c0b2a90 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:39.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.679+0000 7fde70c5c700 1 -- 172.21.15.99:0/43112934 >> 172.21.15.99:0/43112934 conn(0x7fde6c068ce0 msgr2=0x7fde6c15f400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:39.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.679+0000 7fde70c5c700 1 -- 172.21.15.99:0/43112934 shutdown_connections 2023-10-08T00:16:39.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.679+0000 7fde70c5c700 1 -- 172.21.15.99:0/43112934 wait complete. 2023-10-08T00:16:39.682 INFO:teuthology.orchestra.run.smithi099.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T00:16:39.693 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T00:16:39.704 INFO:tasks.ceph.mds.c.smithi099.stdout:starting mds.c at 2023-10-08T00:16:39.812 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.811+0000 7f1069918700 1 Processor -- start 2023-10-08T00:16:39.812 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.811+0000 7f1069918700 1 -- start start 2023-10-08T00:16:39.814 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.812+0000 7f1069918700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f10640cce30 0x7f106415aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:39.814 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.812+0000 7f1069918700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f106406aa00 0x7f10640b5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:39.814 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.812+0000 7f1069918700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10640b4d70 con 0x7f10640b6580 2023-10-08T00:16:39.814 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.812+0000 7f1069918700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10640cd8d0 con 0x7f106406aa00 2023-10-08T00:16:39.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.812+0000 7f1069918700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10640b4440 con 0x7f10640cce30 2023-10-08T00:16:39.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.812+0000 7f1069117700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f106406aa00 0x7f10640b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:39.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.812+0000 7f1069117700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f106406aa00 0x7f10640b5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40098/0 (socket says 172.21.15.99:40098) 2023-10-08T00:16:39.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.812+0000 7f1069117700 1 -- 172.21.15.99:0/3265933860 learned_addr learned my addr 172.21.15.99:0/3265933860 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:39.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.812+0000 7f1068916700 1 --2- 172.21.15.99:0/3265933860 >> v2:172.21.15.191:3300/0 conn(0x7f10640cce30 0x7f106415aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:39.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.812+0000 7f1069117700 1 -- 172.21.15.99:0/3265933860 >> v2:172.21.15.191:3300/0 conn(0x7f10640cce30 msgr2=0x7f106415aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:39.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.812+0000 7f1069117700 1 --2- 172.21.15.99:0/3265933860 >> v2:172.21.15.191:3300/0 conn(0x7f10640cce30 0x7f106415aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:39.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.812+0000 7f1069117700 1 -- 172.21.15.99:0/3265933860 >> v1:172.21.15.191:6789/0 conn(0x7f10640b6580 legacy=0x7f106406a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:39.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.812+0000 7f1069117700 1 -- 172.21.15.99:0/3265933860 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10640b4720 con 0x7f106406aa00 2023-10-08T00:16:39.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.812+0000 7f1068916700 1 --2- 172.21.15.99:0/3265933860 >> v2:172.21.15.191:3300/0 conn(0x7f10640cce30 0x7f106415aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:16:39.816 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.813+0000 7f1069117700 1 --2- 172.21.15.99:0/3265933860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f106406aa00 0x7f10640b5200 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f10640b3770 tx=0x7f105c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f2416b4980da154 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:39.816 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.813+0000 7f105b7fe700 1 -- 172.21.15.99:0/3265933860 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105c00b820 con 0x7f106406aa00 2023-10-08T00:16:39.816 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.813+0000 7f105b7fe700 1 -- 172.21.15.99:0/3265933860 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f105c00b980 con 0x7f106406aa00 2023-10-08T00:16:39.816 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.813+0000 7f105b7fe700 1 -- 172.21.15.99:0/3265933860 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105c00bd20 con 0x7f106406aa00 2023-10-08T00:16:39.816 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.813+0000 7f1069918700 1 -- 172.21.15.99:0/3265933860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f106406aa00 msgr2=0x7f10640b5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:39.816 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.813+0000 7f1069918700 1 --2- 172.21.15.99:0/3265933860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f106406aa00 0x7f10640b5200 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f10640b3770 tx=0x7f105c009250 comp rx=0 tx=0).stop 2023-10-08T00:16:39.816 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.813+0000 7f1069918700 1 -- 172.21.15.99:0/3265933860 shutdown_connections 2023-10-08T00:16:39.816 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.813+0000 7f1069918700 1 --2- 172.21.15.99:0/3265933860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f106406aa00 0x7f10640b5200 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:39.816 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.813+0000 7f1069918700 1 --2- 172.21.15.99:0/3265933860 >> v2:172.21.15.191:3300/0 conn(0x7f10640cce30 0x7f106415aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:39.816 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.813+0000 7f1069918700 1 -- 172.21.15.99:0/3265933860 >> 172.21.15.99:0/3265933860 conn(0x7f1064068cb0 msgr2=0x7f10640690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:39.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.813+0000 7f1069918700 1 -- 172.21.15.99:0/3265933860 shutdown_connections 2023-10-08T00:16:39.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.813+0000 7f1069918700 1 -- 172.21.15.99:0/3265933860 wait complete. 2023-10-08T00:16:39.819 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.814+0000 7f1069918700 1 Processor -- start 2023-10-08T00:16:39.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.815+0000 7f1069918700 1 -- start start 2023-10-08T00:16:39.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.815+0000 7f1069918700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f106406aa00 0x7f10641406c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:39.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.815+0000 7f1069918700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10640b6580 0x7f1064159530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:39.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.815+0000 7f1069918700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f10640cce30 0x7f1064154890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:39.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.815+0000 7f1069918700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10641582a0 con 0x7f10640b6580 2023-10-08T00:16:39.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.815+0000 7f1069918700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1064140c00 con 0x7f106406aa00 2023-10-08T00:16:39.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.815+0000 7f1069918700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10641590a0 con 0x7f10640cce30 2023-10-08T00:16:39.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.815+0000 7f105bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10640b6580 0x7f1064159530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:39.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.816+0000 7f105bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10640b6580 0x7f1064159530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40102/0 (socket says 172.21.15.99:40102) 2023-10-08T00:16:39.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.816+0000 7f105bfff700 1 -- 172.21.15.99:0/470059934 learned_addr learned my addr 172.21.15.99:0/470059934 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:39.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.816+0000 7f1069117700 1 --2- 172.21.15.99:0/470059934 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f10640cce30 0x7f1064154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:39.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.816+0000 7f105bfff700 1 -- 172.21.15.99:0/470059934 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f10640cce30 msgr2=0x7f1064154890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:39.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.816+0000 7f105bfff700 1 --2- 172.21.15.99:0/470059934 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f10640cce30 0x7f1064154890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:39.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.816+0000 7f105bfff700 1 -- 172.21.15.99:0/470059934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f106406aa00 msgr2=0x7f10641406c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:39.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.816+0000 7f1068916700 1 --2- 172.21.15.99:0/470059934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f106406aa00 0x7f10641406c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:39.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.816+0000 7f105bfff700 1 --2- 172.21.15.99:0/470059934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f106406aa00 0x7f10641406c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:39.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.816+0000 7f105bfff700 1 -- 172.21.15.99:0/470059934 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f105c0094f0 con 0x7f10640b6580 2023-10-08T00:16:39.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.816+0000 7f105bfff700 1 --2- 172.21.15.99:0/470059934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10640b6580 0x7f1064159530 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f105400c600 tx=0x7f1054007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:39.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.816+0000 7f1058ff9700 1 -- 172.21.15.99:0/470059934 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1054011590 con 0x7f10640b6580 2023-10-08T00:16:39.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.816+0000 7f1058ff9700 1 -- 172.21.15.99:0/470059934 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1054011710 con 0x7f10640b6580 2023-10-08T00:16:39.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.816+0000 7f1058ff9700 1 -- 172.21.15.99:0/470059934 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1054011ad0 con 0x7f10640b6580 2023-10-08T00:16:39.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.816+0000 7f1069918700 1 -- 172.21.15.99:0/470059934 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1064158490 con 0x7f10640b6580 2023-10-08T00:16:39.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.816+0000 7f1069918700 1 -- 172.21.15.99:0/470059934 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1064158a50 con 0x7f10640b6580 2023-10-08T00:16:39.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.817+0000 7f1069918700 1 -- 172.21.15.99:0/470059934 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1064089550 con 0x7f10640b6580 2023-10-08T00:16:39.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.817+0000 7f1058ff9700 1 -- 172.21.15.99:0/470059934 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f105401c460 con 0x7f10640b6580 2023-10-08T00:16:39.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.818+0000 7f1058ff9700 1 --2- 172.21.15.99:0/470059934 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1050066f30 0x7f10500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:39.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.818+0000 7f1058ff9700 1 -- 172.21.15.99:0/470059934 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f1054084360 con 0x7f10640b6580 2023-10-08T00:16:39.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.818+0000 7f1068916700 1 --2- 172.21.15.99:0/470059934 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1050066f30 0x7f10500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:39.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.818+0000 7f1068916700 1 --2- 172.21.15.99:0/470059934 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1050066f30 0x7f10500693e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f10640b2810 tx=0x7f1060007520 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:39.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:39.822+0000 7f1058ff9700 1 -- 172.21.15.99:0/470059934 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1054051580 con 0x7f10640b6580 2023-10-08T00:16:40.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.044+0000 7f1069918700 1 -- 172.21.15.99:0/470059934 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f10641591e0 con 0x7f10640b6580 2023-10-08T00:16:40.679 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.678+0000 7f1058ff9700 1 -- 172.21.15.99:0/470059934 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v58) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1054054ba0 con 0x7f10640b6580 2023-10-08T00:16:40.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.681+0000 7f1069918700 1 -- 172.21.15.99:0/470059934 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1050066f30 msgr2=0x7f10500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:40.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.681+0000 7f1069918700 1 --2- 172.21.15.99:0/470059934 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1050066f30 0x7f10500693e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f10640b2810 tx=0x7f1060007520 comp rx=0 tx=0).stop 2023-10-08T00:16:40.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.681+0000 7f1069918700 1 -- 172.21.15.99:0/470059934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10640b6580 msgr2=0x7f1064159530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:40.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.681+0000 7f1069918700 1 --2- 172.21.15.99:0/470059934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10640b6580 0x7f1064159530 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f105400c600 tx=0x7f1054007f60 comp rx=0 tx=0).stop 2023-10-08T00:16:40.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.681+0000 7f1069918700 1 -- 172.21.15.99:0/470059934 shutdown_connections 2023-10-08T00:16:40.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.681+0000 7f1069918700 1 --2- 172.21.15.99:0/470059934 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1050066f30 0x7f10500693e0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:40.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.681+0000 7f1069918700 1 --2- 172.21.15.99:0/470059934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f106406aa00 0x7f10641406c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:40.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.681+0000 7f1069918700 1 --2- 172.21.15.99:0/470059934 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f10640cce30 0x7f1064154890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:40.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.681+0000 7f1069918700 1 --2- 172.21.15.99:0/470059934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10640b6580 0x7f1064159530 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:40.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.681+0000 7f1069918700 1 -- 172.21.15.99:0/470059934 >> 172.21.15.99:0/470059934 conn(0x7f1064068cb0 msgr2=0x7f106415d650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:40.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.681+0000 7f1069918700 1 -- 172.21.15.99:0/470059934 shutdown_connections 2023-10-08T00:16:40.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.681+0000 7f1069918700 1 -- 172.21.15.99:0/470059934 wait complete. 2023-10-08T00:16:40.694 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T00:16:40.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.853+0000 7fee80b82700 1 Processor -- start 2023-10-08T00:16:40.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.854+0000 7fee80b82700 1 -- start start 2023-10-08T00:16:40.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.854+0000 7fee80b82700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fee7c0b6a10 0x7fee7c15a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:40.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.854+0000 7fee80b82700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee7c0b17b0 con 0x7fee7c0b46f0 2023-10-08T00:16:40.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.854+0000 7fee80b82700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee7c0b1a60 con 0x7fee7c159ef0 2023-10-08T00:16:40.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.854+0000 7fee80b82700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee7c0b1d40 con 0x7fee7c0b6a10 2023-10-08T00:16:40.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.855+0000 7fee71d9b700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fee7c0b6a10 0x7fee7c15a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:40.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.855+0000 7fee7ad9d700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fee7c0b46f0 0x7fee7c0b4ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50876/0 (socket says 172.21.15.99:50876) 2023-10-08T00:16:40.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.855+0000 7fee7ad9d700 1 -- 172.21.15.99:0/1432394795 learned_addr learned my addr 172.21.15.99:0/1432394795 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:40.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.855+0000 7fee79d9b700 1 -- 172.21.15.99:0/1432394795 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2941147507 0 0) 0x7fee7c0b17b0 con 0x7fee7c0b46f0 2023-10-08T00:16:40.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.855+0000 7fee79d9b700 1 -- 172.21.15.99:0/1432394795 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee68003570 con 0x7fee7c0b46f0 2023-10-08T00:16:40.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.855+0000 7fee71d9b700 1 -- 172.21.15.99:0/1432394795 >> v1:172.21.15.191:6789/0 conn(0x7fee7c159ef0 legacy=0x7fee7c0b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:40.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.855+0000 7fee71d9b700 1 -- 172.21.15.99:0/1432394795 >> v1:172.21.15.99:6789/0 conn(0x7fee7c0b46f0 legacy=0x7fee7c0b4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:40.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.855+0000 7fee71d9b700 1 -- 172.21.15.99:0/1432394795 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee7c0b2020 con 0x7fee7c0b6a10 2023-10-08T00:16:40.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.855+0000 7fee71d9b700 1 --2- 172.21.15.99:0/1432394795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee7c0b6a10 0x7fee7c15a2c0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fee640098e0 tx=0x7fee640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9ed796c63cafe7b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:40.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.855+0000 7fee79d9b700 1 -- 172.21.15.99:0/1432394795 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6400b690 con 0x7fee7c0b6a10 2023-10-08T00:16:40.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.855+0000 7fee79d9b700 1 -- 172.21.15.99:0/1432394795 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee6400b7f0 con 0x7fee7c0b6a10 2023-10-08T00:16:40.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.855+0000 7fee79d9b700 1 -- 172.21.15.99:0/1432394795 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6400bb90 con 0x7fee7c0b6a10 2023-10-08T00:16:40.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.856+0000 7fee80b82700 1 -- 172.21.15.99:0/1432394795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee7c0b6a10 msgr2=0x7fee7c15a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:40.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.856+0000 7fee80b82700 1 --2- 172.21.15.99:0/1432394795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee7c0b6a10 0x7fee7c15a2c0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fee640098e0 tx=0x7fee640091a0 comp rx=0 tx=0).stop 2023-10-08T00:16:40.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.856+0000 7fee80b82700 1 -- 172.21.15.99:0/1432394795 shutdown_connections 2023-10-08T00:16:40.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.856+0000 7fee80b82700 1 --2- 172.21.15.99:0/1432394795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee7c0b6a10 0x7fee7c15a2c0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:40.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.856+0000 7fee80b82700 1 -- 172.21.15.99:0/1432394795 >> 172.21.15.99:0/1432394795 conn(0x7fee7c09a0a0 msgr2=0x7fee7c067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:40.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.856+0000 7fee80b82700 1 -- 172.21.15.99:0/1432394795 shutdown_connections 2023-10-08T00:16:40.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.856+0000 7fee80b82700 1 -- 172.21.15.99:0/1432394795 wait complete. 2023-10-08T00:16:40.862 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.857+0000 7fee80b82700 1 Processor -- start 2023-10-08T00:16:40.862 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.857+0000 7fee80b82700 1 -- start start 2023-10-08T00:16:40.862 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.858+0000 7fee80b82700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee7c0b46f0 0x7fee7c153560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:40.862 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.858+0000 7fee80b82700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fee7c0b6a10 0x7fee7c1557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:40.862 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.858+0000 7fee80b82700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fee7c159ef0 0x7fee7c09bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:40.862 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.858+0000 7fee80b82700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7c155330 con 0x7fee7c0b46f0 2023-10-08T00:16:40.862 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.858+0000 7fee80b82700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7c153aa0 con 0x7fee7c159ef0 2023-10-08T00:16:40.863 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.858+0000 7fee80b82700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee7c154720 con 0x7fee7c0b6a10 2023-10-08T00:16:40.863 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.858+0000 7fee7a59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee7c0b46f0 0x7fee7c153560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:40.863 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.858+0000 7fee7a59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee7c0b46f0 0x7fee7c153560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40126/0 (socket says 172.21.15.99:40126) 2023-10-08T00:16:40.863 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.858+0000 7fee7a59c700 1 -- 172.21.15.99:0/2212789385 learned_addr learned my addr 172.21.15.99:0/2212789385 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:40.863 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.858+0000 7fee7ad9d700 1 --2- 172.21.15.99:0/2212789385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fee7c159ef0 0x7fee7c09bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:40.863 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.858+0000 7fee71d9b700 1 --2- 172.21.15.99:0/2212789385 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fee7c0b6a10 0x7fee7c1557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:40.863 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.858+0000 7fee7a59c700 1 -- 172.21.15.99:0/2212789385 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fee7c0b6a10 msgr2=0x7fee7c1557c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:40.863 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.858+0000 7fee7a59c700 1 --2- 172.21.15.99:0/2212789385 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fee7c0b6a10 0x7fee7c1557c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:40.863 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.858+0000 7fee7a59c700 1 -- 172.21.15.99:0/2212789385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fee7c159ef0 msgr2=0x7fee7c09bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:40.863 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.858+0000 7fee7a59c700 1 --2- 172.21.15.99:0/2212789385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fee7c159ef0 0x7fee7c09bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:40.863 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.858+0000 7fee7a59c700 1 -- 172.21.15.99:0/2212789385 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee64009420 con 0x7fee7c0b46f0 2023-10-08T00:16:40.864 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.858+0000 7fee7a59c700 1 --2- 172.21.15.99:0/2212789385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee7c0b46f0 0x7fee7c153560 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fee6c00c5b0 tx=0x7fee6c0020e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:40.864 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.858+0000 7fee72ffd700 1 -- 172.21.15.99:0/2212789385 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6c011590 con 0x7fee7c0b46f0 2023-10-08T00:16:40.864 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.858+0000 7fee72ffd700 1 -- 172.21.15.99:0/2212789385 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee6c0116f0 con 0x7fee7c0b46f0 2023-10-08T00:16:40.864 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.858+0000 7fee80b82700 1 -- 172.21.15.99:0/2212789385 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee7c154a00 con 0x7fee7c0b46f0 2023-10-08T00:16:40.864 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.859+0000 7fee72ffd700 1 -- 172.21.15.99:0/2212789385 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6c011a90 con 0x7fee7c0b46f0 2023-10-08T00:16:40.864 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.859+0000 7fee80b82700 1 -- 172.21.15.99:0/2212789385 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee7c154fc0 con 0x7fee7c0b46f0 2023-10-08T00:16:40.864 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.860+0000 7fee80b82700 1 -- 172.21.15.99:0/2212789385 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee7c09e490 con 0x7fee7c0b46f0 2023-10-08T00:16:40.864 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.860+0000 7fee72ffd700 1 -- 172.21.15.99:0/2212789385 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fee6c012420 con 0x7fee7c0b46f0 2023-10-08T00:16:40.864 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.860+0000 7fee72ffd700 1 --2- 172.21.15.99:0/2212789385 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fee5c066f30 0x7fee5c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:40.864 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.860+0000 7fee72ffd700 1 -- 172.21.15.99:0/2212789385 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5791+0+0 (secure 0 0 0) 0x7fee6c084010 con 0x7fee7c0b46f0 2023-10-08T00:16:40.864 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.860+0000 7fee71d9b700 1 --2- 172.21.15.99:0/2212789385 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fee5c066f30 0x7fee5c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:40.864 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.861+0000 7fee71d9b700 1 --2- 172.21.15.99:0/2212789385 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fee5c066f30 0x7fee5c0693e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fee7c0b6e10 tx=0x7fee64016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:40.865 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:40.864+0000 7fee72ffd700 1 -- 172.21.15.99:0/2212789385 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fee6c051230 con 0x7fee7c0b46f0 2023-10-08T00:16:41.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.045+0000 7fee80b82700 1 -- 172.21.15.99:0/2212789385 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fee7c04c3c0 con 0x7fee7c0b46f0 2023-10-08T00:16:41.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.681+0000 7fee72ffd700 1 -- 172.21.15.99:0/2212789385 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v129) v1 ==== 181+0+0 (secure 0 0 0) 0x7fee6c054850 con 0x7fee7c0b46f0 2023-10-08T00:16:41.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.683+0000 7fee80b82700 1 -- 172.21.15.99:0/2212789385 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fee5c066f30 msgr2=0x7fee5c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:41.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.683+0000 7fee80b82700 1 --2- 172.21.15.99:0/2212789385 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fee5c066f30 0x7fee5c0693e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fee7c0b6e10 tx=0x7fee64016000 comp rx=0 tx=0).stop 2023-10-08T00:16:41.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.683+0000 7fee80b82700 1 -- 172.21.15.99:0/2212789385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee7c0b46f0 msgr2=0x7fee7c153560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:41.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.683+0000 7fee80b82700 1 --2- 172.21.15.99:0/2212789385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee7c0b46f0 0x7fee7c153560 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fee6c00c5b0 tx=0x7fee6c0020e0 comp rx=0 tx=0).stop 2023-10-08T00:16:41.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.684+0000 7fee80b82700 1 -- 172.21.15.99:0/2212789385 shutdown_connections 2023-10-08T00:16:41.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.684+0000 7fee80b82700 1 --2- 172.21.15.99:0/2212789385 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fee5c066f30 0x7fee5c0693e0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:41.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.684+0000 7fee80b82700 1 --2- 172.21.15.99:0/2212789385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fee7c159ef0 0x7fee7c09bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:41.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.684+0000 7fee80b82700 1 --2- 172.21.15.99:0/2212789385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee7c0b46f0 0x7fee7c153560 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:41.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.684+0000 7fee80b82700 1 --2- 172.21.15.99:0/2212789385 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fee7c0b6a10 0x7fee7c1557c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:41.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.684+0000 7fee80b82700 1 -- 172.21.15.99:0/2212789385 >> 172.21.15.99:0/2212789385 conn(0x7fee7c09a0a0 msgr2=0x7fee7c068e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:41.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.684+0000 7fee80b82700 1 -- 172.21.15.99:0/2212789385 shutdown_connections 2023-10-08T00:16:41.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.684+0000 7fee80b82700 1 -- 172.21.15.99:0/2212789385 wait complete. 2023-10-08T00:16:41.686 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' removed 2023-10-08T00:16:41.697 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T00:16:41.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.815+0000 7f2c5d211700 1 Processor -- start 2023-10-08T00:16:41.816 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.815+0000 7f2c5d211700 1 -- start start 2023-10-08T00:16:41.819 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.816+0000 7f2c5d211700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c5815d240 0x7f2c580b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:41.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.816+0000 7f2c5d211700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c580cd4c0 con 0x7f2c580cc940 2023-10-08T00:16:41.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.816+0000 7f2c5d211700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c580b3f40 con 0x7f2c580ce810 2023-10-08T00:16:41.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.816+0000 7f2c5d211700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c580cbb80 con 0x7f2c5815d240 2023-10-08T00:16:41.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.816+0000 7f2c577fe700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f2c580cc940 0x7f2c580ccdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50888/0 (socket says 172.21.15.99:50888) 2023-10-08T00:16:41.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.816+0000 7f2c577fe700 1 -- 172.21.15.99:0/473639835 learned_addr learned my addr 172.21.15.99:0/473639835 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:41.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.816+0000 7f2c56ffd700 1 -- 172.21.15.99:0/473639835 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3956350366 0 0) 0x7f2c580cd4c0 con 0x7f2c580cc940 2023-10-08T00:16:41.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.816+0000 7f2c56ffd700 1 -- 172.21.15.99:0/473639835 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c40003570 con 0x7f2c580cc940 2023-10-08T00:16:41.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.816+0000 7f2c57fff700 1 --2- 172.21.15.99:0/473639835 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c5815d240 0x7f2c580b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:41.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.816+0000 7f2c56ffd700 1 -- 172.21.15.99:0/473639835 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2546866706 0 0) 0x7f2c40003570 con 0x7f2c580cc940 2023-10-08T00:16:41.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.816+0000 7f2c56ffd700 1 -- 172.21.15.99:0/473639835 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c5815d240 msgr2=0x7f2c580b3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:41.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.816+0000 7f2c56ffd700 1 --2- 172.21.15.99:0/473639835 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c5815d240 0x7f2c580b3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:41.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.816+0000 7f2c56ffd700 1 -- 172.21.15.99:0/473639835 >> v1:172.21.15.191:6789/0 conn(0x7f2c580ce810 legacy=0x7f2c580cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:41.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.816+0000 7f2c56ffd700 1 -- 172.21.15.99:0/473639835 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c580cbe60 con 0x7f2c580cc940 2023-10-08T00:16:41.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.816+0000 7f2c56ffd700 1 -- 172.21.15.99:0/473639835 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f2c48001470 con 0x7f2c580cc940 2023-10-08T00:16:41.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.816+0000 7f2c56ffd700 1 -- 172.21.15.99:0/473639835 >> v1:172.21.15.99:6789/0 conn(0x7f2c580cc940 legacy=0x7f2c580ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:41.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.816+0000 7f2c56ffd700 1 --2- 172.21.15.99:0/473639835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c40003a90 0x7f2c40003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:41.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.817+0000 7f2c56ffd700 1 --2- 172.21.15.99:0/473639835 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c5815d240 0x7f2c40004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:41.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.817+0000 7f2c56ffd700 1 --2- 172.21.15.99:0/473639835 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c40004d50 0x7f2c400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:41.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.817+0000 7f2c56ffd700 1 -- 172.21.15.99:0/473639835 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c580cbb80 con 0x7f2c40003a90 2023-10-08T00:16:41.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.817+0000 7f2c56ffd700 1 -- 172.21.15.99:0/473639835 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4000bb70 con 0x7f2c40004d50 2023-10-08T00:16:41.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.817+0000 7f2c56ffd700 1 -- 172.21.15.99:0/473639835 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4000bd30 con 0x7f2c5815d240 2023-10-08T00:16:41.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.817+0000 7f2c57fff700 1 --2- 172.21.15.99:0/473639835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c40003a90 0x7f2c40003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:41.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.817+0000 7f2c577fe700 1 --2- 172.21.15.99:0/473639835 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c5815d240 0x7f2c40004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:41.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.817+0000 7f2c5ca10700 1 --2- 172.21.15.99:0/473639835 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c40004d50 0x7f2c400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:41.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.817+0000 7f2c57fff700 1 -- 172.21.15.99:0/473639835 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c5815d240 msgr2=0x7f2c40004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:41.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.817+0000 7f2c57fff700 1 --2- 172.21.15.99:0/473639835 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c5815d240 0x7f2c40004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:41.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.817+0000 7f2c57fff700 1 -- 172.21.15.99:0/473639835 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c40004d50 msgr2=0x7f2c400051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:41.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.817+0000 7f2c57fff700 1 --2- 172.21.15.99:0/473639835 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c40004d50 0x7f2c400051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:41.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.817+0000 7f2c57fff700 1 -- 172.21.15.99:0/473639835 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c40003570 con 0x7f2c40003a90 2023-10-08T00:16:41.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.817+0000 7f2c577fe700 1 --2- 172.21.15.99:0/473639835 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c5815d240 0x7f2c40004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:41.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.817+0000 7f2c5ca10700 1 --2- 172.21.15.99:0/473639835 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c40004d50 0x7f2c400051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:16:41.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.817+0000 7f2c57fff700 1 --2- 172.21.15.99:0/473639835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c40003a90 0x7f2c40003ea0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c009bf0 tx=0x7f2c4c009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d980fabddf9b2484 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:41.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.817+0000 7f2c56ffd700 1 -- 172.21.15.99:0/473639835 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c005a00 con 0x7f2c40003a90 2023-10-08T00:16:41.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.817+0000 7f2c56ffd700 1 -- 172.21.15.99:0/473639835 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c4c005b60 con 0x7f2c40003a90 2023-10-08T00:16:41.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.817+0000 7f2c56ffd700 1 -- 172.21.15.99:0/473639835 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c00e710 con 0x7f2c40003a90 2023-10-08T00:16:41.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.818+0000 7f2c5d211700 1 -- 172.21.15.99:0/473639835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c40003a90 msgr2=0x7f2c40003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:41.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.818+0000 7f2c5d211700 1 --2- 172.21.15.99:0/473639835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c40003a90 0x7f2c40003ea0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c009bf0 tx=0x7f2c4c009120 comp rx=0 tx=0).stop 2023-10-08T00:16:41.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.818+0000 7f2c5ca10700 1 -- 172.21.15.99:0/473639835 reap_dead start 2023-10-08T00:16:41.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.818+0000 7f2c5d211700 1 -- 172.21.15.99:0/473639835 shutdown_connections 2023-10-08T00:16:41.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.818+0000 7f2c5d211700 1 -- 172.21.15.99:0/473639835 >> 172.21.15.99:0/473639835 conn(0x7f2c58067e40 msgr2=0x7f2c580d0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:41.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.818+0000 7f2c5d211700 1 -- 172.21.15.99:0/473639835 shutdown_connections 2023-10-08T00:16:41.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.818+0000 7f2c5d211700 1 -- 172.21.15.99:0/473639835 wait complete. 2023-10-08T00:16:41.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.819+0000 7f2c5d211700 1 Processor -- start 2023-10-08T00:16:41.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.819+0000 7f2c5d211700 1 -- start start 2023-10-08T00:16:41.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.820+0000 7f2c5d211700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c40003a90 0x7f2c580b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:41.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.820+0000 7f2c5d211700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c5816bb10 0x7f2c580c9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:41.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.820+0000 7f2c5d211700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c580c9bc0 0x7f2c5816ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:41.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.820+0000 7f2c5d211700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c580ca200 con 0x7f2c5816bb10 2023-10-08T00:16:41.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.820+0000 7f2c5d211700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c580ca4e0 con 0x7f2c40003a90 2023-10-08T00:16:41.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.820+0000 7f2c5d211700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5816c0b0 con 0x7f2c580c9bc0 2023-10-08T00:16:41.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.820+0000 7f2c577fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c5816bb10 0x7f2c580c9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:41.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.820+0000 7f2c577fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c5816bb10 0x7f2c580c9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40150/0 (socket says 172.21.15.99:40150) 2023-10-08T00:16:41.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.820+0000 7f2c577fe700 1 -- 172.21.15.99:0/3475416093 learned_addr learned my addr 172.21.15.99:0/3475416093 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:41.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.820+0000 7f2c57fff700 1 --2- 172.21.15.99:0/3475416093 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c40003a90 0x7f2c580b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:41.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.820+0000 7f2c5ca10700 1 --2- 172.21.15.99:0/3475416093 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c580c9bc0 0x7f2c5816ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:41.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.820+0000 7f2c577fe700 1 -- 172.21.15.99:0/3475416093 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c580c9bc0 msgr2=0x7f2c5816ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:41.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.820+0000 7f2c577fe700 1 --2- 172.21.15.99:0/3475416093 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c580c9bc0 0x7f2c5816ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:41.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.820+0000 7f2c577fe700 1 -- 172.21.15.99:0/3475416093 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c40003a90 msgr2=0x7f2c580b2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:41.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.820+0000 7f2c577fe700 1 --2- 172.21.15.99:0/3475416093 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c40003a90 0x7f2c580b2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:41.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.820+0000 7f2c577fe700 1 -- 172.21.15.99:0/3475416093 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c5816c390 con 0x7f2c5816bb10 2023-10-08T00:16:41.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.820+0000 7f2c57fff700 1 --2- 172.21.15.99:0/3475416093 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c40003a90 0x7f2c580b2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:16:41.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.820+0000 7f2c5ca10700 1 --2- 172.21.15.99:0/3475416093 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c580c9bc0 0x7f2c5816ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:16:41.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.820+0000 7f2c577fe700 1 --2- 172.21.15.99:0/3475416093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c5816bb10 0x7f2c580c9680 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f2c48000fd0 tx=0x7f2c48001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:41.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.820+0000 7f2c3ffff700 1 -- 172.21.15.99:0/3475416093 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4800c7b0 con 0x7f2c5816bb10 2023-10-08T00:16:41.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.820+0000 7f2c3ffff700 1 -- 172.21.15.99:0/3475416093 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c4800c910 con 0x7f2c5816bb10 2023-10-08T00:16:41.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.820+0000 7f2c5d211700 1 -- 172.21.15.99:0/3475416093 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c580b34a0 con 0x7f2c5816bb10 2023-10-08T00:16:41.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.820+0000 7f2c3ffff700 1 -- 172.21.15.99:0/3475416093 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4800ccb0 con 0x7f2c5816bb10 2023-10-08T00:16:41.829 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.820+0000 7f2c5d211700 1 -- 172.21.15.99:0/3475416093 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c5816b4b0 con 0x7f2c5816bb10 2023-10-08T00:16:41.829 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.821+0000 7f2c5d211700 1 -- 172.21.15.99:0/3475416093 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c5816b6d0 con 0x7f2c5816bb10 2023-10-08T00:16:41.829 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.822+0000 7f2c3ffff700 1 -- 172.21.15.99:0/3475416093 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2c48013630 con 0x7f2c5816bb10 2023-10-08T00:16:41.829 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.822+0000 7f2c3ffff700 1 --2- 172.21.15.99:0/3475416093 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2c44066f30 0x7f2c440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:41.829 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.822+0000 7f2c3ffff700 1 -- 172.21.15.99:0/3475416093 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5353+0+0 (secure 0 0 0) 0x7f2c48084580 con 0x7f2c5816bb10 2023-10-08T00:16:41.829 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.822+0000 7f2c57fff700 1 --2- 172.21.15.99:0/3475416093 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2c44066f30 0x7f2c440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:41.829 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.823+0000 7f2c57fff700 1 --2- 172.21.15.99:0/3475416093 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2c44066f30 0x7f2c440693e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2c5815bf30 tx=0x7f2c4c019000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:41.830 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:41.827+0000 7f2c3ffff700 1 -- 172.21.15.99:0/3475416093 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2c5816b6d0 con 0x7f2c5816bb10 2023-10-08T00:16:42.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.009+0000 7f2c5d211700 1 -- 172.21.15.99:0/3475416093 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2c580c9fc0 con 0x7f2c5816bb10 2023-10-08T00:16:42.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.684+0000 7f2c3ffff700 1 -- 172.21.15.99:0/3475416093 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v130) v1 ==== 169+0+0 (secure 0 0 0) 0x7f2c580c9fc0 con 0x7f2c5816bb10 2023-10-08T00:16:42.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.687+0000 7f2c5d211700 1 -- 172.21.15.99:0/3475416093 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2c44066f30 msgr2=0x7f2c440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:42.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.687+0000 7f2c5d211700 1 --2- 172.21.15.99:0/3475416093 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2c44066f30 0x7f2c440693e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2c5815bf30 tx=0x7f2c4c019000 comp rx=0 tx=0).stop 2023-10-08T00:16:42.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.687+0000 7f2c5d211700 1 -- 172.21.15.99:0/3475416093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c5816bb10 msgr2=0x7f2c580c9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:42.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.687+0000 7f2c5d211700 1 --2- 172.21.15.99:0/3475416093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c5816bb10 0x7f2c580c9680 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f2c48000fd0 tx=0x7f2c48001e90 comp rx=0 tx=0).stop 2023-10-08T00:16:42.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.687+0000 7f2c5d211700 1 -- 172.21.15.99:0/3475416093 shutdown_connections 2023-10-08T00:16:42.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.687+0000 7f2c5d211700 1 --2- 172.21.15.99:0/3475416093 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2c44066f30 0x7f2c440693e0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:42.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.687+0000 7f2c5d211700 1 --2- 172.21.15.99:0/3475416093 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c40003a90 0x7f2c580b2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:42.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.687+0000 7f2c5d211700 1 --2- 172.21.15.99:0/3475416093 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c580c9bc0 0x7f2c5816ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:42.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.687+0000 7f2c5d211700 1 --2- 172.21.15.99:0/3475416093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c5816bb10 0x7f2c580c9680 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:42.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.687+0000 7f2c5d211700 1 -- 172.21.15.99:0/3475416093 >> 172.21.15.99:0/3475416093 conn(0x7f2c58067e40 msgr2=0x7f2c580cec10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:42.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.687+0000 7f2c5d211700 1 -- 172.21.15.99:0/3475416093 shutdown_connections 2023-10-08T00:16:42.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.687+0000 7f2c5d211700 1 -- 172.21.15.99:0/3475416093 wait complete. 2023-10-08T00:16:42.689 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' removed 2023-10-08T00:16:42.700 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-08T00:16:42.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.817+0000 7f5a1990a700 1 Processor -- start 2023-10-08T00:16:42.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.817+0000 7f5a1990a700 1 -- start start 2023-10-08T00:16:42.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.817+0000 7f5a1990a700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f5a14159e60 0x7f5a1415e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:42.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.817+0000 7f5a1990a700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5a1415a230 0x7f5a140690b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:42.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.817+0000 7f5a1990a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5a140695f0 0x7f5a140af730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:42.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.817+0000 7f5a1990a700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a140d1cf0 con 0x7f5a1415a230 2023-10-08T00:16:42.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.817+0000 7f5a1990a700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a14069a70 con 0x7f5a14159e60 2023-10-08T00:16:42.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.817+0000 7f5a1990a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1415ea90 con 0x7f5a140695f0 2023-10-08T00:16:42.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.818+0000 7f5a0a7fc700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5a1415a230 0x7f5a140690b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:42.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.818+0000 7f5a0a7fc700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5a1415a230 0x7f5a140690b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40164/0 (socket says 172.21.15.99:40164) 2023-10-08T00:16:42.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.818+0000 7f5a0a7fc700 1 -- 172.21.15.99:0/4144447527 learned_addr learned my addr 172.21.15.99:0/4144447527 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:42.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.818+0000 7f5a12ffd700 1 --2- 172.21.15.99:0/4144447527 >> v2:172.21.15.191:3300/0 conn(0x7f5a14159e60 0x7f5a1415e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:42.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.818+0000 7f5a137fe700 1 --2- 172.21.15.99:0/4144447527 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5a140695f0 0x7f5a140af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:42.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.818+0000 7f5a0a7fc700 1 -- 172.21.15.99:0/4144447527 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5a140695f0 msgr2=0x7f5a140af730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:42.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.818+0000 7f5a0a7fc700 1 --2- 172.21.15.99:0/4144447527 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5a140695f0 0x7f5a140af730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:42.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.818+0000 7f5a0a7fc700 1 -- 172.21.15.99:0/4144447527 >> v2:172.21.15.191:3300/0 conn(0x7f5a14159e60 msgr2=0x7f5a1415e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:42.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.818+0000 7f5a0a7fc700 1 --2- 172.21.15.99:0/4144447527 >> v2:172.21.15.191:3300/0 conn(0x7f5a14159e60 0x7f5a1415e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:42.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.818+0000 7f5a0a7fc700 1 -- 172.21.15.99:0/4144447527 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a140b3100 con 0x7f5a1415a230 2023-10-08T00:16:42.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.818+0000 7f5a12ffd700 1 --2- 172.21.15.99:0/4144447527 >> v2:172.21.15.191:3300/0 conn(0x7f5a14159e60 0x7f5a1415e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:42.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.818+0000 7f5a0a7fc700 1 --2- 172.21.15.99:0/4144447527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a1415a230 0x7f5a140690b0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f5a140cdf30 tx=0x7f59fc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daafd96333163212 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:42.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.818+0000 7f5a127fc700 1 -- 172.21.15.99:0/4144447527 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59fc00b820 con 0x7f5a1415a230 2023-10-08T00:16:42.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.818+0000 7f5a127fc700 1 -- 172.21.15.99:0/4144447527 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59fc00b980 con 0x7f5a1415a230 2023-10-08T00:16:42.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.818+0000 7f5a127fc700 1 -- 172.21.15.99:0/4144447527 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59fc00bd20 con 0x7f5a1415a230 2023-10-08T00:16:42.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.819+0000 7f5a1990a700 1 -- 172.21.15.99:0/4144447527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a1415a230 msgr2=0x7f5a140690b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:42.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.819+0000 7f5a1990a700 1 --2- 172.21.15.99:0/4144447527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a1415a230 0x7f5a140690b0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f5a140cdf30 tx=0x7f59fc009250 comp rx=0 tx=0).stop 2023-10-08T00:16:42.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.819+0000 7f5a1990a700 1 -- 172.21.15.99:0/4144447527 shutdown_connections 2023-10-08T00:16:42.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.819+0000 7f5a1990a700 1 --2- 172.21.15.99:0/4144447527 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5a140695f0 0x7f5a140af730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:42.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.819+0000 7f5a1990a700 1 --2- 172.21.15.99:0/4144447527 >> v2:172.21.15.191:3300/0 conn(0x7f5a14159e60 0x7f5a1415e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:42.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.819+0000 7f5a1990a700 1 --2- 172.21.15.99:0/4144447527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a1415a230 0x7f5a140690b0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:42.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.819+0000 7f5a1990a700 1 -- 172.21.15.99:0/4144447527 >> 172.21.15.99:0/4144447527 conn(0x7f5a14070700 msgr2=0x7f5a14070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:42.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.819+0000 7f5a1990a700 1 -- 172.21.15.99:0/4144447527 shutdown_connections 2023-10-08T00:16:42.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.819+0000 7f5a1990a700 1 -- 172.21.15.99:0/4144447527 wait complete. 2023-10-08T00:16:42.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.820+0000 7f5a1990a700 1 Processor -- start 2023-10-08T00:16:42.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.820+0000 7f5a1990a700 1 -- start start 2023-10-08T00:16:42.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.821+0000 7f5a1990a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a140695f0 0x7f5a140c75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:42.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.821+0000 7f5a1990a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5a14159e60 0x7f5a141458a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:42.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.821+0000 7f5a1990a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5a1415a230 0x7f5a140c5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:42.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.821+0000 7f5a1990a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a140c7b00 con 0x7f5a140695f0 2023-10-08T00:16:42.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.821+0000 7f5a1990a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a140ca300 con 0x7f5a1415a230 2023-10-08T00:16:42.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.821+0000 7f5a1990a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a140c5260 con 0x7f5a14159e60 2023-10-08T00:16:42.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.821+0000 7f5a12ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a140695f0 0x7f5a140c75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:42.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.821+0000 7f5a12ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a140695f0 0x7f5a140c75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40176/0 (socket says 172.21.15.99:40176) 2023-10-08T00:16:42.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.821+0000 7f5a12ffd700 1 -- 172.21.15.99:0/1342529128 learned_addr learned my addr 172.21.15.99:0/1342529128 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:42.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.821+0000 7f5a0a7fc700 1 --2- 172.21.15.99:0/1342529128 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5a14159e60 0x7f5a141458a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:42.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.821+0000 7f5a137fe700 1 --2- 172.21.15.99:0/1342529128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5a1415a230 0x7f5a140c5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:42.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.821+0000 7f5a12ffd700 1 -- 172.21.15.99:0/1342529128 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5a14159e60 msgr2=0x7f5a141458a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:42.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.821+0000 7f5a12ffd700 1 --2- 172.21.15.99:0/1342529128 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5a14159e60 0x7f5a141458a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:42.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.821+0000 7f5a12ffd700 1 -- 172.21.15.99:0/1342529128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5a1415a230 msgr2=0x7f5a140c5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:42.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.821+0000 7f5a12ffd700 1 --2- 172.21.15.99:0/1342529128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5a1415a230 0x7f5a140c5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:42.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.821+0000 7f5a12ffd700 1 -- 172.21.15.99:0/1342529128 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59fc0094f0 con 0x7f5a140695f0 2023-10-08T00:16:42.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.821+0000 7f5a137fe700 1 --2- 172.21.15.99:0/1342529128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5a1415a230 0x7f5a140c5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:42.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.821+0000 7f5a12ffd700 1 --2- 172.21.15.99:0/1342529128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a140695f0 0x7f5a140c75c0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f5a0400e8d0 tx=0x7f5a04008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:42.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.821+0000 7f5a0b7fe700 1 -- 172.21.15.99:0/1342529128 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a04013410 con 0x7f5a140695f0 2023-10-08T00:16:42.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.821+0000 7f5a0b7fe700 1 -- 172.21.15.99:0/1342529128 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a04013570 con 0x7f5a140695f0 2023-10-08T00:16:42.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.821+0000 7f5a1990a700 1 -- 172.21.15.99:0/1342529128 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a140c4ca0 con 0x7f5a140695f0 2023-10-08T00:16:42.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.821+0000 7f5a0b7fe700 1 -- 172.21.15.99:0/1342529128 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a04013910 con 0x7f5a140695f0 2023-10-08T00:16:42.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.822+0000 7f5a1990a700 1 -- 172.21.15.99:0/1342529128 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a140c9f90 con 0x7f5a140695f0 2023-10-08T00:16:42.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.822+0000 7f5a1990a700 1 -- 172.21.15.99:0/1342529128 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a140ca5e0 con 0x7f5a140695f0 2023-10-08T00:16:42.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.823+0000 7f5a0b7fe700 1 -- 172.21.15.99:0/1342529128 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5a04013d40 con 0x7f5a140695f0 2023-10-08T00:16:42.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.823+0000 7f5a0b7fe700 1 --2- 172.21.15.99:0/1342529128 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f59f4066f30 0x7f59f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:42.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.823+0000 7f5a0b7fe700 1 -- 172.21.15.99:0/1342529128 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f5a04083f00 con 0x7f5a140695f0 2023-10-08T00:16:42.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.824+0000 7f5a0a7fc700 1 --2- 172.21.15.99:0/1342529128 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f59f4066f30 0x7f59f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:42.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.824+0000 7f5a0a7fc700 1 --2- 172.21.15.99:0/1342529128 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f59f4066f30 0x7f59f40693e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5a140cdf30 tx=0x7f59fc017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:42.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:42.827+0000 7f5a0b7fe700 1 -- 172.21.15.99:0/1342529128 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5a04051470 con 0x7f5a140695f0 2023-10-08T00:16:43.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.011+0000 7f5a1990a700 1 -- 172.21.15.99:0/1342529128 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f5a1404c3c0 con 0x7f5a140695f0 2023-10-08T00:16:43.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.015+0000 7f5a0b7fe700 1 -- 172.21.15.99:0/1342529128 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5a04054a90 con 0x7f5a140695f0 2023-10-08T00:16:43.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.017+0000 7f5a1990a700 1 -- 172.21.15.99:0/1342529128 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f59f4066f30 msgr2=0x7f59f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:43.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.017+0000 7f5a1990a700 1 --2- 172.21.15.99:0/1342529128 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f59f4066f30 0x7f59f40693e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5a140cdf30 tx=0x7f59fc017000 comp rx=0 tx=0).stop 2023-10-08T00:16:43.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.017+0000 7f5a1990a700 1 -- 172.21.15.99:0/1342529128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a140695f0 msgr2=0x7f5a140c75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:43.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.017+0000 7f5a1990a700 1 --2- 172.21.15.99:0/1342529128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a140695f0 0x7f5a140c75c0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f5a0400e8d0 tx=0x7f5a04008040 comp rx=0 tx=0).stop 2023-10-08T00:16:43.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.017+0000 7f5a1990a700 1 -- 172.21.15.99:0/1342529128 shutdown_connections 2023-10-08T00:16:43.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.017+0000 7f5a1990a700 1 --2- 172.21.15.99:0/1342529128 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f59f4066f30 0x7f59f40693e0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:43.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.017+0000 7f5a1990a700 1 --2- 172.21.15.99:0/1342529128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5a1415a230 0x7f5a140c5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:43.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.017+0000 7f5a1990a700 1 --2- 172.21.15.99:0/1342529128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a140695f0 0x7f5a140c75c0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:43.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.017+0000 7f5a1990a700 1 --2- 172.21.15.99:0/1342529128 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5a14159e60 0x7f5a141458a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:43.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.017+0000 7f5a1990a700 1 -- 172.21.15.99:0/1342529128 >> 172.21.15.99:0/1342529128 conn(0x7f5a14070700 msgr2=0x7f5a140cbff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:43.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.017+0000 7f5a1990a700 1 -- 172.21.15.99:0/1342529128 shutdown_connections 2023-10-08T00:16:43.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.017+0000 7f5a1990a700 1 -- 172.21.15.99:0/1342529128 wait complete. 2023-10-08T00:16:43.030 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_barrier' 2023-10-08T00:16:43.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.144+0000 7f27cc079700 1 Processor -- start 2023-10-08T00:16:43.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.145+0000 7f27cc079700 1 -- start start 2023-10-08T00:16:43.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.145+0000 7f27cc079700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27c4152010 0x7f27c415b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:43.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.145+0000 7f27cc079700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f27c41523e0 0x7f27c406c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:43.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.145+0000 7f27cc079700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27c406c010 con 0x7f27c4158e50 2023-10-08T00:16:43.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.145+0000 7f27cc079700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27c406bd10 con 0x7f27c41523e0 2023-10-08T00:16:43.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.145+0000 7f27cc079700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27c415c060 con 0x7f27c4152010 2023-10-08T00:16:43.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.145+0000 7f27c9614700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f27c41523e0 0x7f27c406c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:43.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.146+0000 7f27c9614700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f27c41523e0 0x7f27c406c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40190/0 (socket says 172.21.15.99:40190) 2023-10-08T00:16:43.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.146+0000 7f27c9614700 1 -- 172.21.15.99:0/1819397636 learned_addr learned my addr 172.21.15.99:0/1819397636 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:43.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.146+0000 7f27c9e15700 1 --2- 172.21.15.99:0/1819397636 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27c4152010 0x7f27c415b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:43.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.146+0000 7f27c9614700 1 -- 172.21.15.99:0/1819397636 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27c4152010 msgr2=0x7f27c415b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:43.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.146+0000 7f27c9614700 1 --2- 172.21.15.99:0/1819397636 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27c4152010 0x7f27c415b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:43.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.146+0000 7f27c9614700 1 -- 172.21.15.99:0/1819397636 >> v1:172.21.15.191:6789/0 conn(0x7f27c4158e50 legacy=0x7f27c40cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:43.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.146+0000 7f27c9614700 1 -- 172.21.15.99:0/1819397636 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c415c340 con 0x7f27c41523e0 2023-10-08T00:16:43.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.146+0000 7f27c9e15700 1 --2- 172.21.15.99:0/1819397636 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27c4152010 0x7f27c415b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:16:43.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.146+0000 7f27c9614700 1 --2- 172.21.15.99:0/1819397636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27c41523e0 0x7f27c406c4a0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f27b40099b0 tx=0x7f27b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75d670bc4baecd1b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:43.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.146+0000 7f27c8e13700 1 -- 172.21.15.99:0/1819397636 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b400b820 con 0x7f27c41523e0 2023-10-08T00:16:43.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.146+0000 7f27c8e13700 1 -- 172.21.15.99:0/1819397636 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27b400b980 con 0x7f27c41523e0 2023-10-08T00:16:43.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.146+0000 7f27c8e13700 1 -- 172.21.15.99:0/1819397636 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b400bd20 con 0x7f27c41523e0 2023-10-08T00:16:43.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.147+0000 7f27cc079700 1 -- 172.21.15.99:0/1819397636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27c41523e0 msgr2=0x7f27c406c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:43.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.147+0000 7f27cc079700 1 --2- 172.21.15.99:0/1819397636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27c41523e0 0x7f27c406c4a0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f27b40099b0 tx=0x7f27b4009250 comp rx=0 tx=0).stop 2023-10-08T00:16:43.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.147+0000 7f27cc079700 1 -- 172.21.15.99:0/1819397636 shutdown_connections 2023-10-08T00:16:43.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.147+0000 7f27cc079700 1 --2- 172.21.15.99:0/1819397636 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27c4152010 0x7f27c415b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:43.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.147+0000 7f27cc079700 1 --2- 172.21.15.99:0/1819397636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27c41523e0 0x7f27c406c4a0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:43.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.147+0000 7f27cc079700 1 -- 172.21.15.99:0/1819397636 >> 172.21.15.99:0/1819397636 conn(0x7f27c4069870 msgr2=0x7f27c4069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:43.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.147+0000 7f27cc079700 1 -- 172.21.15.99:0/1819397636 shutdown_connections 2023-10-08T00:16:43.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.147+0000 7f27cc079700 1 -- 172.21.15.99:0/1819397636 wait complete. 2023-10-08T00:16:43.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.148+0000 7f27cc079700 1 Processor -- start 2023-10-08T00:16:43.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.148+0000 7f27cc079700 1 -- start start 2023-10-08T00:16:43.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.149+0000 7f27cc079700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27c4152010 0x7f27c414fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:43.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.149+0000 7f27cc079700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27c41523e0 0x7f27c414eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:43.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.149+0000 7f27cc079700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27c4158e50 0x7f27c414d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:43.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.149+0000 7f27cc079700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c40b2210 con 0x7f27c4152010 2023-10-08T00:16:43.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.149+0000 7f27cc079700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c414f050 con 0x7f27c4158e50 2023-10-08T00:16:43.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.149+0000 7f27cc079700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27c4150090 con 0x7f27c41523e0 2023-10-08T00:16:43.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.149+0000 7f27c9e15700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27c4152010 0x7f27c414fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:43.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.149+0000 7f27c9e15700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27c4152010 0x7f27c414fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40192/0 (socket says 172.21.15.99:40192) 2023-10-08T00:16:43.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.149+0000 7f27c9e15700 1 -- 172.21.15.99:0/3091197798 learned_addr learned my addr 172.21.15.99:0/3091197798 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:43.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.149+0000 7f27c9614700 1 --2- 172.21.15.99:0/3091197798 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27c41523e0 0x7f27c414eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:43.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.149+0000 7f27ca616700 1 --2- 172.21.15.99:0/3091197798 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27c4158e50 0x7f27c414d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:43.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.149+0000 7f27c9e15700 1 -- 172.21.15.99:0/3091197798 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27c41523e0 msgr2=0x7f27c414eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:43.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.149+0000 7f27c9e15700 1 --2- 172.21.15.99:0/3091197798 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27c41523e0 0x7f27c414eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:43.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.149+0000 7f27c9e15700 1 -- 172.21.15.99:0/3091197798 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27c4158e50 msgr2=0x7f27c414d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:43.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.149+0000 7f27c9e15700 1 --2- 172.21.15.99:0/3091197798 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27c4158e50 0x7f27c414d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:43.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.149+0000 7f27c9e15700 1 -- 172.21.15.99:0/3091197798 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b40094f0 con 0x7f27c4152010 2023-10-08T00:16:43.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.149+0000 7f27c9614700 1 --2- 172.21.15.99:0/3091197798 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27c41523e0 0x7f27c414eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:43.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.149+0000 7f27ca616700 1 --2- 172.21.15.99:0/3091197798 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27c4158e50 0x7f27c414d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:43.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.149+0000 7f27c9e15700 1 --2- 172.21.15.99:0/3091197798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27c4152010 0x7f27c414fa20 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f27c0007f90 tx=0x7f27c0007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:43.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.149+0000 7f27b9ffb700 1 -- 172.21.15.99:0/3091197798 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c0016420 con 0x7f27c4152010 2023-10-08T00:16:43.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.149+0000 7f27b9ffb700 1 -- 172.21.15.99:0/3091197798 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27c0016580 con 0x7f27c4152010 2023-10-08T00:16:43.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.149+0000 7f27cc079700 1 -- 172.21.15.99:0/3091197798 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27c40b18e0 con 0x7f27c4152010 2023-10-08T00:16:43.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.149+0000 7f27b9ffb700 1 -- 172.21.15.99:0/3091197798 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c0016940 con 0x7f27c4152010 2023-10-08T00:16:43.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.150+0000 7f27cc079700 1 -- 172.21.15.99:0/3091197798 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27c40b1ea0 con 0x7f27c4152010 2023-10-08T00:16:43.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.150+0000 7f27cc079700 1 -- 172.21.15.99:0/3091197798 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27c40b24f0 con 0x7f27c4152010 2023-10-08T00:16:43.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.151+0000 7f27b9ffb700 1 -- 172.21.15.99:0/3091197798 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f27c0016cf0 con 0x7f27c4152010 2023-10-08T00:16:43.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.151+0000 7f27b9ffb700 1 --2- 172.21.15.99:0/3091197798 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f27a8066f30 0x7f27a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:43.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.151+0000 7f27b9ffb700 1 -- 172.21.15.99:0/3091197798 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f27c0012070 con 0x7f27c4152010 2023-10-08T00:16:43.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.151+0000 7f27c9614700 1 --2- 172.21.15.99:0/3091197798 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f27a8066f30 0x7f27a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:43.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.152+0000 7f27c9614700 1 --2- 172.21.15.99:0/3091197798 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f27a8066f30 0x7f27a80693e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f27b40099b0 tx=0x7f27b4016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:43.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.156+0000 7f27b9ffb700 1 -- 172.21.15.99:0/3091197798 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f27c00521a0 con 0x7f27c4152010 2023-10-08T00:16:43.414 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.413+0000 7f27cc079700 1 -- 172.21.15.99:0/3091197798 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]} v 0) v1 -- 0x7f27c404c3c0 con 0x7f27c4152010 2023-10-08T00:16:43.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.686+0000 7f27b9ffb700 1 -- 172.21.15.99:0/3091197798 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]}]=0 v306) v1 ==== 128+0+0 (secure 0 0 0) 0x7f27c00557c0 con 0x7f27c4152010 2023-10-08T00:16:43.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.688+0000 7f27cc079700 1 -- 172.21.15.99:0/3091197798 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f27a8066f30 msgr2=0x7f27a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:43.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.688+0000 7f27cc079700 1 --2- 172.21.15.99:0/3091197798 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f27a8066f30 0x7f27a80693e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f27b40099b0 tx=0x7f27b4016000 comp rx=0 tx=0).stop 2023-10-08T00:16:43.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.688+0000 7f27cc079700 1 -- 172.21.15.99:0/3091197798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27c4152010 msgr2=0x7f27c414fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:43.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.688+0000 7f27cc079700 1 --2- 172.21.15.99:0/3091197798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27c4152010 0x7f27c414fa20 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f27c0007f90 tx=0x7f27c0007870 comp rx=0 tx=0).stop 2023-10-08T00:16:43.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.688+0000 7f27cc079700 1 -- 172.21.15.99:0/3091197798 shutdown_connections 2023-10-08T00:16:43.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.688+0000 7f27cc079700 1 --2- 172.21.15.99:0/3091197798 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f27a8066f30 0x7f27a80693e0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:43.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.688+0000 7f27cc079700 1 --2- 172.21.15.99:0/3091197798 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27c4158e50 0x7f27c414d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:43.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.688+0000 7f27cc079700 1 --2- 172.21.15.99:0/3091197798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27c4152010 0x7f27c414fa20 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:43.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.688+0000 7f27cc079700 1 --2- 172.21.15.99:0/3091197798 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27c41523e0 0x7f27c414eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:43.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.688+0000 7f27cc079700 1 -- 172.21.15.99:0/3091197798 >> 172.21.15.99:0/3091197798 conn(0x7f27c4069870 msgr2=0x7f27c4069c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:43.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.688+0000 7f27cc079700 1 -- 172.21.15.99:0/3091197798 shutdown_connections 2023-10-08T00:16:43.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.688+0000 7f27cc079700 1 -- 172.21.15.99:0/3091197798 wait complete. 2023-10-08T00:16:43.700 INFO:tasks.cephfs_test_runner:That when an OSD epoch barrier is set on an MDS, subsequently ... ok 2023-10-08T00:16:43.701 INFO:tasks.cephfs_test_runner:Starting test: test_full_different_file (tasks.cephfs.test_full.TestQuotaFull) 2023-10-08T00:16:43.701 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file' 2023-10-08T00:16:43.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.814+0000 7f8392b43700 1 Processor -- start 2023-10-08T00:16:43.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.814+0000 7f8392b43700 1 -- start start 2023-10-08T00:16:43.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.815+0000 7f8392b43700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f838c0cf1c0 0x7f838c15e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:43.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.815+0000 7f8392b43700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f838c15a0c0 0x7f838c068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:43.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.815+0000 7f8392b43700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f838c0cca20 con 0x7f838c159cf0 2023-10-08T00:16:43.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.815+0000 7f8392b43700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f838c15ea40 con 0x7f838c0cf1c0 2023-10-08T00:16:43.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.815+0000 7f8392b43700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f838c067660 con 0x7f838c15a0c0 2023-10-08T00:16:43.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.815+0000 7f8383fff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f838c0cf1c0 0x7f838c15e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:43.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.815+0000 7f83908df700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f838c159cf0 0x7f838c0ceab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:42938/0 (socket says 172.21.15.99:42938) 2023-10-08T00:16:43.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.815+0000 7f83908df700 1 -- 172.21.15.99:0/3264884024 learned_addr learned my addr 172.21.15.99:0/3264884024 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:43.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.816+0000 7f83910e0700 1 --2- 172.21.15.99:0/3264884024 >> v2:172.21.15.191:3300/0 conn(0x7f838c15a0c0 0x7f838c068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:43.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.816+0000 7f8383fff700 1 -- 172.21.15.99:0/3264884024 >> v2:172.21.15.191:3300/0 conn(0x7f838c15a0c0 msgr2=0x7f838c068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:43.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.816+0000 7f8383fff700 1 --2- 172.21.15.99:0/3264884024 >> v2:172.21.15.191:3300/0 conn(0x7f838c15a0c0 0x7f838c068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:43.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.816+0000 7f8383fff700 1 -- 172.21.15.99:0/3264884024 >> v1:172.21.15.99:6789/0 conn(0x7f838c159cf0 legacy=0x7f838c0ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:43.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.816+0000 7f8383fff700 1 -- 172.21.15.99:0/3264884024 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f838c067f00 con 0x7f838c0cf1c0 2023-10-08T00:16:43.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.816+0000 7f83837fe700 1 -- 172.21.15.99:0/3264884024 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2307653201 0 0) 0x7f838c0cca20 con 0x7f838c159cf0 2023-10-08T00:16:43.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.816+0000 7f8383fff700 1 --2- 172.21.15.99:0/3264884024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f838c0cf1c0 0x7f838c15e500 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f838c06f260 tx=0x7f837c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25c0c8a49066d713 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:43.819 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.816+0000 7f83837fe700 1 -- 172.21.15.99:0/3264884024 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837c00b6b0 con 0x7f838c0cf1c0 2023-10-08T00:16:43.819 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.816+0000 7f83837fe700 1 -- 172.21.15.99:0/3264884024 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f837c00b810 con 0x7f838c0cf1c0 2023-10-08T00:16:43.819 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.816+0000 7f83837fe700 1 -- 172.21.15.99:0/3264884024 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837c00bbb0 con 0x7f838c0cf1c0 2023-10-08T00:16:43.819 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.817+0000 7f8392b43700 1 -- 172.21.15.99:0/3264884024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f838c0cf1c0 msgr2=0x7f838c15e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:43.819 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.817+0000 7f8392b43700 1 --2- 172.21.15.99:0/3264884024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f838c0cf1c0 0x7f838c15e500 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f838c06f260 tx=0x7f837c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:16:43.819 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.817+0000 7f8392b43700 1 -- 172.21.15.99:0/3264884024 shutdown_connections 2023-10-08T00:16:43.819 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.817+0000 7f8392b43700 1 --2- 172.21.15.99:0/3264884024 >> v2:172.21.15.191:3300/0 conn(0x7f838c15a0c0 0x7f838c068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:43.819 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.817+0000 7f8392b43700 1 --2- 172.21.15.99:0/3264884024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f838c0cf1c0 0x7f838c15e500 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:43.819 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.817+0000 7f8392b43700 1 -- 172.21.15.99:0/3264884024 >> 172.21.15.99:0/3264884024 conn(0x7f838c06c6e0 msgr2=0x7f838c069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:43.819 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.817+0000 7f8392b43700 1 -- 172.21.15.99:0/3264884024 shutdown_connections 2023-10-08T00:16:43.819 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.817+0000 7f8392b43700 1 -- 172.21.15.99:0/3264884024 wait complete. 2023-10-08T00:16:43.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.818+0000 7f8392b43700 1 Processor -- start 2023-10-08T00:16:43.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.818+0000 7f8392b43700 1 -- start start 2023-10-08T00:16:43.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.818+0000 7f8392b43700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f838c0cf1c0 0x7f838c1555b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:43.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.819+0000 7f8392b43700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f838c159cf0 0x7f838c0acf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:43.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.819+0000 7f8392b43700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f838c15a0c0 0x7f838c0c4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:43.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.819+0000 7f8392b43700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f838c0adb90 con 0x7f838c15a0c0 2023-10-08T00:16:43.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.819+0000 7f8392b43700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f838c0c8c30 con 0x7f838c0cf1c0 2023-10-08T00:16:43.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.819+0000 7f8392b43700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f838c0c8020 con 0x7f838c159cf0 2023-10-08T00:16:43.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.819+0000 7f83910e0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f838c15a0c0 0x7f838c0c4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:43.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.819+0000 7f83910e0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f838c15a0c0 0x7f838c0c4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47274/0 (socket says 172.21.15.99:47274) 2023-10-08T00:16:43.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.819+0000 7f83910e0700 1 -- 172.21.15.99:0/1228957768 learned_addr learned my addr 172.21.15.99:0/1228957768 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:43.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.819+0000 7f8383fff700 1 --2- 172.21.15.99:0/1228957768 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f838c159cf0 0x7f838c0acf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:43.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.819+0000 7f83910e0700 1 -- 172.21.15.99:0/1228957768 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f838c159cf0 msgr2=0x7f838c0acf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:43.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.819+0000 7f83910e0700 1 --2- 172.21.15.99:0/1228957768 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f838c159cf0 0x7f838c0acf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:43.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.819+0000 7f83910e0700 1 -- 172.21.15.99:0/1228957768 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f838c0cf1c0 msgr2=0x7f838c1555b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:43.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.819+0000 7f83910e0700 1 --2- 172.21.15.99:0/1228957768 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f838c0cf1c0 0x7f838c1555b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:43.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.819+0000 7f83910e0700 1 -- 172.21.15.99:0/1228957768 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f837c009420 con 0x7f838c15a0c0 2023-10-08T00:16:43.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.819+0000 7f83910e0700 1 --2- 172.21.15.99:0/1228957768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f838c15a0c0 0x7f838c0c4710 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f838400d890 tx=0x7f8384008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:43.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.819+0000 7f8380ff9700 1 -- 172.21.15.99:0/1228957768 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83840125f0 con 0x7f838c15a0c0 2023-10-08T00:16:43.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.819+0000 7f8380ff9700 1 -- 172.21.15.99:0/1228957768 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8384012770 con 0x7f838c15a0c0 2023-10-08T00:16:43.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.819+0000 7f8380ff9700 1 -- 172.21.15.99:0/1228957768 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8384012b30 con 0x7f838c15a0c0 2023-10-08T00:16:43.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.819+0000 7f8392b43700 1 -- 172.21.15.99:0/1228957768 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f838c0c8300 con 0x7f838c15a0c0 2023-10-08T00:16:43.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.819+0000 7f8392b43700 1 -- 172.21.15.99:0/1228957768 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f838c0c88c0 con 0x7f838c15a0c0 2023-10-08T00:16:43.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.820+0000 7f8392b43700 1 -- 172.21.15.99:0/1228957768 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f838c0692e0 con 0x7f838c15a0c0 2023-10-08T00:16:43.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.821+0000 7f8380ff9700 1 -- 172.21.15.99:0/1228957768 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f838401e460 con 0x7f838c15a0c0 2023-10-08T00:16:43.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.821+0000 7f8380ff9700 1 --2- 172.21.15.99:0/1228957768 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8378066f30 0x7f83780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:43.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.821+0000 7f8380ff9700 1 -- 172.21.15.99:0/1228957768 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f8384084860 con 0x7f838c15a0c0 2023-10-08T00:16:43.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.821+0000 7f83908df700 1 --2- 172.21.15.99:0/1228957768 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8378066f30 0x7f83780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:43.824 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.822+0000 7f83908df700 1 --2- 172.21.15.99:0/1228957768 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8378066f30 0x7f83780693e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f8388001560 tx=0x7f83880016c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:43.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:43.825+0000 7f8380ff9700 1 -- 172.21.15.99:0/1228957768 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8384051dd0 con 0x7f838c15a0c0 2023-10-08T00:16:44.084 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.083+0000 7f8392b43700 1 -- 172.21.15.99:0/1228957768 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]} v 0) v1 -- 0x7f838c04c3c0 con 0x7f838c15a0c0 2023-10-08T00:16:44.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.690+0000 7f8380ff9700 1 -- 172.21.15.99:0/1228957768 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]}]=0 v307) v1 ==== 143+0+0 (secure 0 0 0) 0x7f83840553f0 con 0x7f838c15a0c0 2023-10-08T00:16:44.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.692+0000 7f8392b43700 1 -- 172.21.15.99:0/1228957768 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8378066f30 msgr2=0x7f83780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:44.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.692+0000 7f8392b43700 1 --2- 172.21.15.99:0/1228957768 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8378066f30 0x7f83780693e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f8388001560 tx=0x7f83880016c0 comp rx=0 tx=0).stop 2023-10-08T00:16:44.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.692+0000 7f8392b43700 1 -- 172.21.15.99:0/1228957768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f838c15a0c0 msgr2=0x7f838c0c4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:44.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.692+0000 7f8392b43700 1 --2- 172.21.15.99:0/1228957768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f838c15a0c0 0x7f838c0c4710 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f838400d890 tx=0x7f8384008040 comp rx=0 tx=0).stop 2023-10-08T00:16:44.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.692+0000 7f8392b43700 1 -- 172.21.15.99:0/1228957768 shutdown_connections 2023-10-08T00:16:44.694 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.692+0000 7f8392b43700 1 --2- 172.21.15.99:0/1228957768 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8378066f30 0x7f83780693e0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:44.694 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.692+0000 7f8392b43700 1 --2- 172.21.15.99:0/1228957768 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f838c0cf1c0 0x7f838c1555b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:44.694 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.692+0000 7f8392b43700 1 --2- 172.21.15.99:0/1228957768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f838c15a0c0 0x7f838c0c4710 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:44.694 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.692+0000 7f8392b43700 1 --2- 172.21.15.99:0/1228957768 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f838c159cf0 0x7f838c0acf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:44.694 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.692+0000 7f8392b43700 1 -- 172.21.15.99:0/1228957768 >> 172.21.15.99:0/1228957768 conn(0x7f838c06c6e0 msgr2=0x7f838c068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:44.694 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.693+0000 7f8392b43700 1 -- 172.21.15.99:0/1228957768 shutdown_connections 2023-10-08T00:16:44.694 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.693+0000 7f8392b43700 1 -- 172.21.15.99:0/1228957768 wait complete. 2023-10-08T00:16:44.706 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T00:16:44.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.822+0000 7f7d1926c700 1 Processor -- start 2023-10-08T00:16:44.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.822+0000 7f7d1926c700 1 -- start start 2023-10-08T00:16:44.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.823+0000 7f7d1926c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7d1415b9a0 0x7f7d140caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:44.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.823+0000 7f7d1926c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7d1406d370 0x7f7d1406d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:44.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.823+0000 7f7d1926c700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d1415c8d0 con 0x7f7d14152300 2023-10-08T00:16:44.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.823+0000 7f7d1926c700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d140b1400 con 0x7f7d1406d370 2023-10-08T00:16:44.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.823+0000 7f7d1926c700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d1415be20 con 0x7f7d1415b9a0 2023-10-08T00:16:44.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.823+0000 7f7d12ffd700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f7d14152300 0x7f7d14158df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:42946/0 (socket says 172.21.15.99:42946) 2023-10-08T00:16:44.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.823+0000 7f7d137fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7d1406d370 0x7f7d1406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:44.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.823+0000 7f7d12ffd700 1 -- 172.21.15.99:0/647618769 learned_addr learned my addr 172.21.15.99:0/647618769 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:44.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.824+0000 7f7d137fe700 1 -- 172.21.15.99:0/647618769 >> v2:172.21.15.191:3300/0 conn(0x7f7d1415b9a0 msgr2=0x7f7d140caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:44.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.824+0000 7f7d137fe700 1 --2- 172.21.15.99:0/647618769 >> v2:172.21.15.191:3300/0 conn(0x7f7d1415b9a0 0x7f7d140caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:44.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.824+0000 7f7d137fe700 1 -- 172.21.15.99:0/647618769 >> v1:172.21.15.99:6789/0 conn(0x7f7d14152300 legacy=0x7f7d14158df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:44.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.824+0000 7f7d137fe700 1 -- 172.21.15.99:0/647618769 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d140c9370 con 0x7f7d1406d370 2023-10-08T00:16:44.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.824+0000 7f7d127fc700 1 -- 172.21.15.99:0/647618769 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3951113154 0 0) 0x7f7d1415c8d0 con 0x7f7d14152300 2023-10-08T00:16:44.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.824+0000 7f7d137fe700 1 --2- 172.21.15.99:0/647618769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d1406d370 0x7f7d1406d790 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f7d1406e8d0 tx=0x7f7d0c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb9c39a10ca9cf5a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:44.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.824+0000 7f7d127fc700 1 -- 172.21.15.99:0/647618769 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c00e070 con 0x7f7d1406d370 2023-10-08T00:16:44.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.824+0000 7f7d127fc700 1 -- 172.21.15.99:0/647618769 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d0c00bc30 con 0x7f7d1406d370 2023-10-08T00:16:44.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.824+0000 7f7d127fc700 1 -- 172.21.15.99:0/647618769 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c012430 con 0x7f7d1406d370 2023-10-08T00:16:44.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.825+0000 7f7d1926c700 1 -- 172.21.15.99:0/647618769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d1406d370 msgr2=0x7f7d1406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:44.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.825+0000 7f7d1926c700 1 --2- 172.21.15.99:0/647618769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d1406d370 0x7f7d1406d790 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f7d1406e8d0 tx=0x7f7d0c009780 comp rx=0 tx=0).stop 2023-10-08T00:16:44.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.825+0000 7f7d1926c700 1 -- 172.21.15.99:0/647618769 shutdown_connections 2023-10-08T00:16:44.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.825+0000 7f7d1926c700 1 --2- 172.21.15.99:0/647618769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d1406d370 0x7f7d1406d790 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:44.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.825+0000 7f7d1926c700 1 --2- 172.21.15.99:0/647618769 >> v2:172.21.15.191:3300/0 conn(0x7f7d1415b9a0 0x7f7d140caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:44.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.825+0000 7f7d1926c700 1 -- 172.21.15.99:0/647618769 >> 172.21.15.99:0/647618769 conn(0x7f7d14069870 msgr2=0x7f7d14069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:44.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.825+0000 7f7d1926c700 1 -- 172.21.15.99:0/647618769 shutdown_connections 2023-10-08T00:16:44.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.825+0000 7f7d1926c700 1 -- 172.21.15.99:0/647618769 wait complete. 2023-10-08T00:16:44.830 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.826+0000 7f7d1926c700 1 Processor -- start 2023-10-08T00:16:44.830 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.826+0000 7f7d1926c700 1 -- start start 2023-10-08T00:16:44.830 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.826+0000 7f7d1926c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d1406d370 0x7f7d140b1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:44.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.827+0000 7f7d1926c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7d14152300 0x7f7d14068950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:44.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.827+0000 7f7d1926c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d1415b9a0 0x7f7d141583b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:44.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.827+0000 7f7d1926c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d14067660 con 0x7f7d1406d370 2023-10-08T00:16:44.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.827+0000 7f7d1926c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d140b1f70 con 0x7f7d1415b9a0 2023-10-08T00:16:44.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.827+0000 7f7d1926c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d140678b0 con 0x7f7d14152300 2023-10-08T00:16:44.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.827+0000 7f7d12ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d1406d370 0x7f7d140b1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:44.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.827+0000 7f7d12ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d1406d370 0x7f7d140b1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47298/0 (socket says 172.21.15.99:47298) 2023-10-08T00:16:44.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.827+0000 7f7d12ffd700 1 -- 172.21.15.99:0/998346296 learned_addr learned my addr 172.21.15.99:0/998346296 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:44.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.827+0000 7f7d12ffd700 1 -- 172.21.15.99:0/998346296 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7d14152300 msgr2=0x7f7d14068950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:44.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.827+0000 7f7d12ffd700 1 --2- 172.21.15.99:0/998346296 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7d14152300 0x7f7d14068950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:44.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.827+0000 7f7d12ffd700 1 -- 172.21.15.99:0/998346296 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d1415b9a0 msgr2=0x7f7d141583b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:44.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.827+0000 7f7d12ffd700 1 --2- 172.21.15.99:0/998346296 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d1415b9a0 0x7f7d141583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:44.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.827+0000 7f7d12ffd700 1 -- 172.21.15.99:0/998346296 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d0c009a20 con 0x7f7d1406d370 2023-10-08T00:16:44.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.827+0000 7f7d12ffd700 1 --2- 172.21.15.99:0/998346296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d1406d370 0x7f7d140b1a30 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f7d0400c7e0 tx=0x7f7d04007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:44.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.827+0000 7f7d0b7fe700 1 -- 172.21.15.99:0/998346296 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d04005b30 con 0x7f7d1406d370 2023-10-08T00:16:44.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.827+0000 7f7d0b7fe700 1 -- 172.21.15.99:0/998346296 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d04005c90 con 0x7f7d1406d370 2023-10-08T00:16:44.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.827+0000 7f7d1926c700 1 -- 172.21.15.99:0/998346296 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d14067b90 con 0x7f7d1406d370 2023-10-08T00:16:44.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.827+0000 7f7d0b7fe700 1 -- 172.21.15.99:0/998346296 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d04007300 con 0x7f7d1406d370 2023-10-08T00:16:44.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.827+0000 7f7d1926c700 1 -- 172.21.15.99:0/998346296 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d14067f20 con 0x7f7d1406d370 2023-10-08T00:16:44.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.828+0000 7f7d1926c700 1 -- 172.21.15.99:0/998346296 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d14067d20 con 0x7f7d1406d370 2023-10-08T00:16:44.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.829+0000 7f7d0b7fe700 1 -- 172.21.15.99:0/998346296 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7d0401b400 con 0x7f7d1406d370 2023-10-08T00:16:44.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.829+0000 7f7d0b7fe700 1 --2- 172.21.15.99:0/998346296 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7cf4066f30 0x7f7cf40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:44.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.829+0000 7f7d0b7fe700 1 -- 172.21.15.99:0/998346296 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f7d04083ce0 con 0x7f7d1406d370 2023-10-08T00:16:44.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.829+0000 7f7d0a7fc700 1 --2- 172.21.15.99:0/998346296 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7cf4066f30 0x7f7cf40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:44.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.830+0000 7f7d0a7fc700 1 --2- 172.21.15.99:0/998346296 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7cf4066f30 0x7f7cf40693e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7d1415ad00 tx=0x7f7cfc006d10 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:44.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:44.833+0000 7f7d0b7fe700 1 -- 172.21.15.99:0/998346296 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7d040512d0 con 0x7f7d1406d370 2023-10-08T00:16:45.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.014+0000 7f7d1926c700 1 -- 172.21.15.99:0/998346296 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f7d140cdf10 con 0x7f7d1406d370 2023-10-08T00:16:45.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.018+0000 7f7d0b7fe700 1 -- 172.21.15.99:0/998346296 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7d040548f0 con 0x7f7d1406d370 2023-10-08T00:16:45.021 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.020+0000 7f7d1926c700 1 -- 172.21.15.99:0/998346296 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7cf4066f30 msgr2=0x7f7cf40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:45.021 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.020+0000 7f7d1926c700 1 --2- 172.21.15.99:0/998346296 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7cf4066f30 0x7f7cf40693e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7d1415ad00 tx=0x7f7cfc006d10 comp rx=0 tx=0).stop 2023-10-08T00:16:45.021 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.020+0000 7f7d1926c700 1 -- 172.21.15.99:0/998346296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d1406d370 msgr2=0x7f7d140b1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:45.021 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.020+0000 7f7d1926c700 1 --2- 172.21.15.99:0/998346296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d1406d370 0x7f7d140b1a30 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f7d0400c7e0 tx=0x7f7d04007df0 comp rx=0 tx=0).stop 2023-10-08T00:16:45.021 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.020+0000 7f7d1926c700 1 -- 172.21.15.99:0/998346296 shutdown_connections 2023-10-08T00:16:45.021 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.020+0000 7f7d1926c700 1 --2- 172.21.15.99:0/998346296 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7cf4066f30 0x7f7cf40693e0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:45.022 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.020+0000 7f7d1926c700 1 --2- 172.21.15.99:0/998346296 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d1415b9a0 0x7f7d141583b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:45.022 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.020+0000 7f7d1926c700 1 --2- 172.21.15.99:0/998346296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d1406d370 0x7f7d140b1a30 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:45.022 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.020+0000 7f7d1926c700 1 --2- 172.21.15.99:0/998346296 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7d14152300 0x7f7d14068950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:45.022 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.020+0000 7f7d1926c700 1 -- 172.21.15.99:0/998346296 >> 172.21.15.99:0/998346296 conn(0x7f7d14069870 msgr2=0x7f7d140cae40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:45.022 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.020+0000 7f7d1926c700 1 -- 172.21.15.99:0/998346296 shutdown_connections 2023-10-08T00:16:45.022 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.020+0000 7f7d1926c700 1 -- 172.21.15.99:0/998346296 wait complete. 2023-10-08T00:16:45.032 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T00:16:45.033 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T00:16:45.069 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T00:16:45.115 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T00:16:45.116 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-08T00:16:45.116 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:16:45.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.228+0000 7f01eb59e700 1 Processor -- start 2023-10-08T00:16:45.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.228+0000 7f01eb59e700 1 -- start start 2023-10-08T00:16:45.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.229+0000 7f01eb59e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01ec0cdab0 0x7f01ec15d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:45.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.229+0000 7f01eb59e700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01ec0cb7c0 con 0x7f01ec0cd6e0 2023-10-08T00:16:45.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.229+0000 7f01eb59e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01ec15e290 con 0x7f01ec15dda0 2023-10-08T00:16:45.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.229+0000 7f01eb59e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec15e850 con 0x7f01ec0cdab0 2023-10-08T00:16:45.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.229+0000 7f01ea59c700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f01ec0cd6e0 0x7f01ec0cbc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:42954/0 (socket says 172.21.15.99:42954) 2023-10-08T00:16:45.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.229+0000 7f01ea59c700 1 -- 172.21.15.99:0/3242235821 learned_addr learned my addr 172.21.15.99:0/3242235821 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:45.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.229+0000 7f01e1d9b700 1 --2- 172.21.15.99:0/3242235821 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01ec0cdab0 0x7f01ec15d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:45.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.229+0000 7f01e9d9b700 1 -- 172.21.15.99:0/3242235821 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 43092911 0 0) 0x7f01ec0cb7c0 con 0x7f01ec0cd6e0 2023-10-08T00:16:45.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.229+0000 7f01e9d9b700 1 -- 172.21.15.99:0/3242235821 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01d8003570 con 0x7f01ec0cd6e0 2023-10-08T00:16:45.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.229+0000 7f01e9d9b700 1 -- 172.21.15.99:0/3242235821 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1706526726 0 0) 0x7f01d8003570 con 0x7f01ec0cd6e0 2023-10-08T00:16:45.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.229+0000 7f01e9d9b700 1 -- 172.21.15.99:0/3242235821 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01ec0cdab0 msgr2=0x7f01ec15d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:45.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.229+0000 7f01e9d9b700 1 --2- 172.21.15.99:0/3242235821 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01ec0cdab0 0x7f01ec15d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:45.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.230+0000 7f01e9d9b700 1 -- 172.21.15.99:0/3242235821 >> v1:172.21.15.191:6789/0 conn(0x7f01ec15dda0 legacy=0x7f01ec15cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:45.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.230+0000 7f01e9d9b700 1 -- 172.21.15.99:0/3242235821 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ec15e570 con 0x7f01ec0cd6e0 2023-10-08T00:16:45.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.230+0000 7f01e9d9b700 1 -- 172.21.15.99:0/3242235821 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f01dc001470 con 0x7f01ec0cd6e0 2023-10-08T00:16:45.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.230+0000 7f01e9d9b700 1 -- 172.21.15.99:0/3242235821 >> v1:172.21.15.99:6789/0 conn(0x7f01ec0cd6e0 legacy=0x7f01ec0cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:45.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.230+0000 7f01e9d9b700 1 --2- 172.21.15.99:0/3242235821 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01d8003a90 0x7f01d8003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:45.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.230+0000 7f01e9d9b700 1 --2- 172.21.15.99:0/3242235821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01d8004810 0x7f01d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:45.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.230+0000 7f01e9d9b700 1 --2- 172.21.15.99:0/3242235821 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01ec0cdab0 0x7f01d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:45.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.230+0000 7f01e9d9b700 1 -- 172.21.15.99:0/3242235821 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec15e850 con 0x7f01d8004810 2023-10-08T00:16:45.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.230+0000 7f01e9d9b700 1 -- 172.21.15.99:0/3242235821 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d800bb10 con 0x7f01d8003a90 2023-10-08T00:16:45.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.230+0000 7f01e9d9b700 1 -- 172.21.15.99:0/3242235821 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01d800bcd0 con 0x7f01ec0cdab0 2023-10-08T00:16:45.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.230+0000 7f01e1d9b700 1 --2- 172.21.15.99:0/3242235821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01d8004810 0x7f01d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:45.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.230+0000 7f01ea59c700 1 --2- 172.21.15.99:0/3242235821 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01d8003a90 0x7f01d8003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:45.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.230+0000 7f01ead9d700 1 --2- 172.21.15.99:0/3242235821 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01ec0cdab0 0x7f01d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:45.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.230+0000 7f01e1d9b700 1 -- 172.21.15.99:0/3242235821 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01ec0cdab0 msgr2=0x7f01d80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:45.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.230+0000 7f01e1d9b700 1 --2- 172.21.15.99:0/3242235821 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01ec0cdab0 0x7f01d80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:45.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.230+0000 7f01e1d9b700 1 -- 172.21.15.99:0/3242235821 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01d8003a90 msgr2=0x7f01d8003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:45.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.230+0000 7f01e1d9b700 1 --2- 172.21.15.99:0/3242235821 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01d8003a90 0x7f01d8003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:45.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.230+0000 7f01e1d9b700 1 -- 172.21.15.99:0/3242235821 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d8003570 con 0x7f01d8004810 2023-10-08T00:16:45.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.230+0000 7f01ead9d700 1 --2- 172.21.15.99:0/3242235821 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01ec0cdab0 0x7f01d80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:45.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.230+0000 7f01e1d9b700 1 --2- 172.21.15.99:0/3242235821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01d8004810 0x7f01d8004c80 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f01d4009c00 tx=0x7f01d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60354a9b9496ce61 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:45.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.230+0000 7f01e9d9b700 1 -- 172.21.15.99:0/3242235821 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d400e710 con 0x7f01d8004810 2023-10-08T00:16:45.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.230+0000 7f01e9d9b700 1 -- 172.21.15.99:0/3242235821 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d400e870 con 0x7f01d8004810 2023-10-08T00:16:45.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.230+0000 7f01e9d9b700 1 -- 172.21.15.99:0/3242235821 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d400ec30 con 0x7f01d8004810 2023-10-08T00:16:45.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.231+0000 7f01eb59e700 1 -- 172.21.15.99:0/3242235821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01d8004810 msgr2=0x7f01d8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:45.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.231+0000 7f01eb59e700 1 --2- 172.21.15.99:0/3242235821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01d8004810 0x7f01d8004c80 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f01d4009c00 tx=0x7f01d40091a0 comp rx=0 tx=0).stop 2023-10-08T00:16:45.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.231+0000 7f01ead9d700 1 -- 172.21.15.99:0/3242235821 reap_dead start 2023-10-08T00:16:45.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.231+0000 7f01eb59e700 1 -- 172.21.15.99:0/3242235821 shutdown_connections 2023-10-08T00:16:45.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.231+0000 7f01eb59e700 1 -- 172.21.15.99:0/3242235821 >> 172.21.15.99:0/3242235821 conn(0x7f01ec067660 msgr2=0x7f01ec067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:45.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.231+0000 7f01eb59e700 1 -- 172.21.15.99:0/3242235821 shutdown_connections 2023-10-08T00:16:45.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.231+0000 7f01eb59e700 1 -- 172.21.15.99:0/3242235821 wait complete. 2023-10-08T00:16:45.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.232+0000 7f01eb59e700 1 Processor -- start 2023-10-08T00:16:45.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.232+0000 7f01eb59e700 1 -- start start 2023-10-08T00:16:45.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.233+0000 7f01eb59e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01d8004810 0x7f01ec0c9e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:45.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.233+0000 7f01eb59e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01ec0c87e0 0x7f01ec0c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:45.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.233+0000 7f01eb59e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01ec0b4e00 0x7f01ec0b5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:45.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.233+0000 7f01eb59e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec0cc360 con 0x7f01ec0c87e0 2023-10-08T00:16:45.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.233+0000 7f01eb59e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec0b3d60 con 0x7f01ec0b4e00 2023-10-08T00:16:45.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.233+0000 7f01eb59e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec0b4040 con 0x7f01d8004810 2023-10-08T00:16:45.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.233+0000 7f01e1d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01ec0c87e0 0x7f01ec0c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:45.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.233+0000 7f01e1d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01ec0c87e0 0x7f01ec0c8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47314/0 (socket says 172.21.15.99:47314) 2023-10-08T00:16:45.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.233+0000 7f01e1d9b700 1 -- 172.21.15.99:0/3538564360 learned_addr learned my addr 172.21.15.99:0/3538564360 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:45.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.233+0000 7f01ea59c700 1 --2- 172.21.15.99:0/3538564360 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01d8004810 0x7f01ec0c9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:45.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.233+0000 7f01ead9d700 1 --2- 172.21.15.99:0/3538564360 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01ec0b4e00 0x7f01ec0b5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:45.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.233+0000 7f01e1d9b700 1 -- 172.21.15.99:0/3538564360 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01d8004810 msgr2=0x7f01ec0c9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:45.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.233+0000 7f01e1d9b700 1 --2- 172.21.15.99:0/3538564360 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01d8004810 0x7f01ec0c9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:45.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.233+0000 7f01e1d9b700 1 -- 172.21.15.99:0/3538564360 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01ec0b4e00 msgr2=0x7f01ec0b5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:45.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.233+0000 7f01e1d9b700 1 --2- 172.21.15.99:0/3538564360 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01ec0b4e00 0x7f01ec0b5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:45.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.233+0000 7f01e1d9b700 1 -- 172.21.15.99:0/3538564360 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ec0b4320 con 0x7f01ec0c87e0 2023-10-08T00:16:45.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.233+0000 7f01e1d9b700 1 --2- 172.21.15.99:0/3538564360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01ec0c87e0 0x7f01ec0c8c50 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f01d4013ef0 tx=0x7f01d401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:45.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.234+0000 7f01e2ffd700 1 -- 172.21.15.99:0/3538564360 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d4022070 con 0x7f01ec0c87e0 2023-10-08T00:16:45.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.234+0000 7f01e2ffd700 1 -- 172.21.15.99:0/3538564360 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d400e9e0 con 0x7f01ec0c87e0 2023-10-08T00:16:45.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.234+0000 7f01eb59e700 1 -- 172.21.15.99:0/3538564360 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01ec0b4600 con 0x7f01ec0c87e0 2023-10-08T00:16:45.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.234+0000 7f01e2ffd700 1 -- 172.21.15.99:0/3538564360 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d4027470 con 0x7f01ec0c87e0 2023-10-08T00:16:45.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.234+0000 7f01eb59e700 1 -- 172.21.15.99:0/3538564360 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01ec0b3a10 con 0x7f01ec0c87e0 2023-10-08T00:16:45.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.234+0000 7f01eb59e700 1 -- 172.21.15.99:0/3538564360 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01ec0b4b20 con 0x7f01ec0c87e0 2023-10-08T00:16:45.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.235+0000 7f01e2ffd700 1 -- 172.21.15.99:0/3538564360 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f01d401c450 con 0x7f01ec0c87e0 2023-10-08T00:16:45.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.235+0000 7f01e2ffd700 1 --2- 172.21.15.99:0/3538564360 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f01cc066f30 0x7f01cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:45.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.235+0000 7f01e2ffd700 1 -- 172.21.15.99:0/3538564360 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f01d4085d50 con 0x7f01ec0c87e0 2023-10-08T00:16:45.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.235+0000 7f01ea59c700 1 --2- 172.21.15.99:0/3538564360 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f01cc066f30 0x7f01cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:45.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.236+0000 7f01ea59c700 1 --2- 172.21.15.99:0/3538564360 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f01cc066f30 0x7f01cc0693e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f01dc0015a0 tx=0x7f01dc001e70 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:45.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.239+0000 7f01e2ffd700 1 -- 172.21.15.99:0/3538564360 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f01ec0b4b20 con 0x7f01ec0c87e0 2023-10-08T00:16:45.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.461+0000 7f01eb59e700 1 -- 172.21.15.99:0/3538564360 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f01ec0b4b20 con 0x7f01ec0c87e0 2023-10-08T00:16:45.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.462+0000 7f01e2ffd700 1 -- 172.21.15.99:0/3538564360 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+3535 (secure 0 0 0) 0x7f01d402cde0 con 0x7f01ec0c87e0 2023-10-08T00:16:45.463 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:16:45.463 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":58,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5627,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/2782755379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":2782755379},{"type":"v1","addr":"172.21.15.191:6837","nonce":2782755379}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6166,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/2980541392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2980541392},{"type":"v1","addr":"172.21.15.99:6834","nonce":2980541392}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6184,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/1342206989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1342206989},{"type":"v1","addr":"172.21.15.191:6836","nonce":1342206989}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":7029,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2087721472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":2087721472},{"type":"v1","addr":"172.21.15.99:6835","nonce":2087721472}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58}],"filesystems":[]} 2023-10-08T00:16:45.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.464+0000 7f01eb59e700 1 -- 172.21.15.99:0/3538564360 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f01cc066f30 msgr2=0x7f01cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:45.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.464+0000 7f01eb59e700 1 --2- 172.21.15.99:0/3538564360 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f01cc066f30 0x7f01cc0693e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f01dc0015a0 tx=0x7f01dc001e70 comp rx=0 tx=0).stop 2023-10-08T00:16:45.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.464+0000 7f01eb59e700 1 -- 172.21.15.99:0/3538564360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01ec0c87e0 msgr2=0x7f01ec0c8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:45.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.464+0000 7f01eb59e700 1 --2- 172.21.15.99:0/3538564360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01ec0c87e0 0x7f01ec0c8c50 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f01d4013ef0 tx=0x7f01d401a000 comp rx=0 tx=0).stop 2023-10-08T00:16:45.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.464+0000 7f01eb59e700 1 -- 172.21.15.99:0/3538564360 shutdown_connections 2023-10-08T00:16:45.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.464+0000 7f01eb59e700 1 --2- 172.21.15.99:0/3538564360 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f01cc066f30 0x7f01cc0693e0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:45.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.464+0000 7f01eb59e700 1 --2- 172.21.15.99:0/3538564360 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01ec0b4e00 0x7f01ec0b5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:45.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.464+0000 7f01eb59e700 1 --2- 172.21.15.99:0/3538564360 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01d8004810 0x7f01ec0c9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:45.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.464+0000 7f01eb59e700 1 --2- 172.21.15.99:0/3538564360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01ec0c87e0 0x7f01ec0c8c50 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:45.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.464+0000 7f01eb59e700 1 -- 172.21.15.99:0/3538564360 >> 172.21.15.99:0/3538564360 conn(0x7f01ec067660 msgr2=0x7f01ec0c6e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:45.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.464+0000 7f01eb59e700 1 -- 172.21.15.99:0/3538564360 shutdown_connections 2023-10-08T00:16:45.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.464+0000 7f01eb59e700 1 -- 172.21.15.99:0/3538564360 wait complete. 2023-10-08T00:16:45.466 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 58 2023-10-08T00:16:45.477 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T00:16:45.477 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-08T00:16:45.477 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T00:16:45.477 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:16:45.478 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T00:16:45.478 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-08T00:16:45.478 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T00:16:45.478 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:16:45.478 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T00:16:45.478 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-08T00:16:45.479 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T00:16:45.479 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:16:45.479 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T00:16:45.479 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-08T00:16:45.479 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T00:16:45.479 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:16:45.487 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T00:16:45.487 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T00:16:45.488 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T00:16:45.488 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T00:16:45.492 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T00:16:45.493 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T00:16:45.493 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T00:16:45.493 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T00:16:45.529 INFO:tasks.ceph.mds.a:Started 2023-10-08T00:16:45.529 DEBUG:teuthology.parallel:result is None 2023-10-08T00:16:45.530 INFO:tasks.ceph.mds.c:Started 2023-10-08T00:16:45.530 DEBUG:teuthology.parallel:result is None 2023-10-08T00:16:45.535 INFO:tasks.ceph.mds.d:Started 2023-10-08T00:16:45.535 DEBUG:teuthology.parallel:result is None 2023-10-08T00:16:45.535 INFO:tasks.ceph.mds.b:Started 2023-10-08T00:16:45.536 DEBUG:teuthology.parallel:result is None 2023-10-08T00:16:45.536 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-08T00:16:45.601 INFO:tasks.ceph.mds.c.smithi099.stdout:starting mds.c at 2023-10-08T00:16:45.602 INFO:tasks.ceph.mds.a.smithi099.stdout:starting mds.a at 2023-10-08T00:16:45.614 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2023-10-08T00:16:45.614 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2023-10-08T00:16:45.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.656+0000 7f3ef481e700 1 Processor -- start 2023-10-08T00:16:45.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.656+0000 7f3ef481e700 1 -- start start 2023-10-08T00:16:45.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.657+0000 7f3ef481e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f3ef00b0890 0x7f3ef00cb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:45.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.657+0000 7f3ef481e700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3ef00cbd40 0x7f3ef0067660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:45.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.657+0000 7f3ef481e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3ef006c630 0x7f3ef0067ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:45.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.657+0000 7f3ef481e700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ef00cb370 con 0x7f3ef00cbd40 2023-10-08T00:16:45.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.657+0000 7f3ef481e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ef00b0c60 con 0x7f3ef00b0890 2023-10-08T00:16:45.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.657+0000 7f3ef481e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef006a1e0 con 0x7f3ef006c630 2023-10-08T00:16:45.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.657+0000 7f3eedd9b700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3ef00cbd40 0x7f3ef0067660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:45.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.657+0000 7f3eedd9b700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3ef00cbd40 0x7f3ef0067660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47362/0 (socket says 172.21.15.99:47362) 2023-10-08T00:16:45.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.657+0000 7f3eedd9b700 1 -- 172.21.15.99:0/2384510996 learned_addr learned my addr 172.21.15.99:0/2384510996 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:45.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.657+0000 7f3eeed9d700 1 --2- 172.21.15.99:0/2384510996 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3ef006c630 0x7f3ef0067ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:45.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.657+0000 7f3eee59c700 1 --2- 172.21.15.99:0/2384510996 >> v2:172.21.15.191:3300/0 conn(0x7f3ef00b0890 0x7f3ef00cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:45.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.657+0000 7f3eedd9b700 1 -- 172.21.15.99:0/2384510996 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3ef006c630 msgr2=0x7f3ef0067ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:45.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.657+0000 7f3eedd9b700 1 --2- 172.21.15.99:0/2384510996 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3ef006c630 0x7f3ef0067ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:45.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.657+0000 7f3eedd9b700 1 -- 172.21.15.99:0/2384510996 >> v2:172.21.15.191:3300/0 conn(0x7f3ef00b0890 msgr2=0x7f3ef00cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:45.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.657+0000 7f3eedd9b700 1 --2- 172.21.15.99:0/2384510996 >> v2:172.21.15.191:3300/0 conn(0x7f3ef00b0890 0x7f3ef00cb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:45.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.657+0000 7f3eedd9b700 1 -- 172.21.15.99:0/2384510996 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef006cb80 con 0x7f3ef00cbd40 2023-10-08T00:16:45.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.657+0000 7f3eedd9b700 1 --2- 172.21.15.99:0/2384510996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ef00cbd40 0x7f3ef0067660 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f3ef006aae0 tx=0x7f3ee4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=634fe28c9607103a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:45.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.657+0000 7f3eed59a700 1 -- 172.21.15.99:0/2384510996 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee400b860 con 0x7f3ef00cbd40 2023-10-08T00:16:45.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.657+0000 7f3eed59a700 1 -- 172.21.15.99:0/2384510996 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee400b9c0 con 0x7f3ef00cbd40 2023-10-08T00:16:45.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.657+0000 7f3eed59a700 1 -- 172.21.15.99:0/2384510996 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee400bd60 con 0x7f3ef00cbd40 2023-10-08T00:16:45.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.658+0000 7f3ef481e700 1 -- 172.21.15.99:0/2384510996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ef00cbd40 msgr2=0x7f3ef0067660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:45.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.658+0000 7f3ef481e700 1 --2- 172.21.15.99:0/2384510996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ef00cbd40 0x7f3ef0067660 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f3ef006aae0 tx=0x7f3ee4009250 comp rx=0 tx=0).stop 2023-10-08T00:16:45.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.658+0000 7f3ef481e700 1 -- 172.21.15.99:0/2384510996 shutdown_connections 2023-10-08T00:16:45.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.658+0000 7f3ef481e700 1 --2- 172.21.15.99:0/2384510996 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3ef006c630 0x7f3ef0067ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:45.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.658+0000 7f3ef481e700 1 --2- 172.21.15.99:0/2384510996 >> v2:172.21.15.191:3300/0 conn(0x7f3ef00b0890 0x7f3ef00cb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:45.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.658+0000 7f3ef481e700 1 --2- 172.21.15.99:0/2384510996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ef00cbd40 0x7f3ef0067660 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:45.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.658+0000 7f3ef481e700 1 -- 172.21.15.99:0/2384510996 >> 172.21.15.99:0/2384510996 conn(0x7f3ef006fe50 msgr2=0x7f3ef0069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:45.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.658+0000 7f3ef481e700 1 -- 172.21.15.99:0/2384510996 shutdown_connections 2023-10-08T00:16:45.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.658+0000 7f3ef481e700 1 -- 172.21.15.99:0/2384510996 wait complete. 2023-10-08T00:16:45.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.659+0000 7f3ef481e700 1 Processor -- start 2023-10-08T00:16:45.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.659+0000 7f3ef481e700 1 -- start start 2023-10-08T00:16:45.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.660+0000 7f3ef481e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ef006c630 0x7f3ef00b3330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:45.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.660+0000 7f3ef481e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ef00b0890 0x7f3ef015a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:45.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.660+0000 7f3ef481e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3ef00cbd40 0x7f3ef00c32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:45.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.660+0000 7f3ef481e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef00b3970 con 0x7f3ef00b0890 2023-10-08T00:16:45.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.660+0000 7f3ef481e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef00b3c50 con 0x7f3ef006c630 2023-10-08T00:16:45.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.660+0000 7f3ef481e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef00b3f30 con 0x7f3ef00cbd40 2023-10-08T00:16:45.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.660+0000 7f3eedd9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ef00b0890 0x7f3ef015a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:45.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.660+0000 7f3eedd9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ef00b0890 0x7f3ef015a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47374/0 (socket says 172.21.15.99:47374) 2023-10-08T00:16:45.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.660+0000 7f3eedd9b700 1 -- 172.21.15.99:0/3696370551 learned_addr learned my addr 172.21.15.99:0/3696370551 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:45.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.660+0000 7f3eeed9d700 1 --2- 172.21.15.99:0/3696370551 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3ef00cbd40 0x7f3ef00c32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:45.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.660+0000 7f3eee59c700 1 --2- 172.21.15.99:0/3696370551 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ef006c630 0x7f3ef00b3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:45.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.660+0000 7f3eedd9b700 1 -- 172.21.15.99:0/3696370551 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3ef00cbd40 msgr2=0x7f3ef00c32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:45.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.660+0000 7f3eedd9b700 1 --2- 172.21.15.99:0/3696370551 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3ef00cbd40 0x7f3ef00c32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:45.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.660+0000 7f3eedd9b700 1 -- 172.21.15.99:0/3696370551 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ef006c630 msgr2=0x7f3ef00b3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:45.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.660+0000 7f3eedd9b700 1 --2- 172.21.15.99:0/3696370551 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ef006c630 0x7f3ef00b3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:45.666 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.660+0000 7f3eedd9b700 1 -- 172.21.15.99:0/3696370551 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee40094f0 con 0x7f3ef00b0890 2023-10-08T00:16:45.666 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.660+0000 7f3eedd9b700 1 --2- 172.21.15.99:0/3696370551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ef00b0890 0x7f3ef015a0e0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f3ef006ab60 tx=0x7f3ee4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:45.666 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.661+0000 7f3ede7fc700 1 -- 172.21.15.99:0/3696370551 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4011dc0 con 0x7f3ef00b0890 2023-10-08T00:16:45.666 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.661+0000 7f3ede7fc700 1 -- 172.21.15.99:0/3696370551 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee4024440 con 0x7f3ef00b0890 2023-10-08T00:16:45.666 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.661+0000 7f3ef481e700 1 -- 172.21.15.99:0/3696370551 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ef00b4210 con 0x7f3ef00b0890 2023-10-08T00:16:45.666 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.661+0000 7f3ede7fc700 1 -- 172.21.15.99:0/3696370551 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4024800 con 0x7f3ef00b0890 2023-10-08T00:16:45.666 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.661+0000 7f3ef481e700 1 -- 172.21.15.99:0/3696370551 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ef015a620 con 0x7f3ef00b0890 2023-10-08T00:16:45.666 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.661+0000 7f3ef481e700 1 -- 172.21.15.99:0/3696370551 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ef00b4730 con 0x7f3ef00b0890 2023-10-08T00:16:45.666 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.662+0000 7f3ede7fc700 1 -- 172.21.15.99:0/3696370551 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3ee401a420 con 0x7f3ef00b0890 2023-10-08T00:16:45.666 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.662+0000 7f3ede7fc700 1 --2- 172.21.15.99:0/3696370551 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3ed0066f30 0x7f3ed00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:45.667 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.662+0000 7f3ede7fc700 1 -- 172.21.15.99:0/3696370551 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f3ee40860a0 con 0x7f3ef00b0890 2023-10-08T00:16:45.667 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.662+0000 7f3eee59c700 1 --2- 172.21.15.99:0/3696370551 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3ed0066f30 0x7f3ed00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:45.667 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.663+0000 7f3eee59c700 1 --2- 172.21.15.99:0/3696370551 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3ed0066f30 0x7f3ed00693e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0006fd0 tx=0x7f3ee0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:45.667 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.667+0000 7f3ede7fc700 1 -- 172.21.15.99:0/3696370551 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3ee4053610 con 0x7f3ef00b0890 2023-10-08T00:16:45.866 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:45.865+0000 7f3ef481e700 1 -- 172.21.15.99:0/3696370551 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f3ef004c3c0 con 0x7f3ef00b0890 2023-10-08T00:16:46.702 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.701+0000 7f3ede7fc700 1 -- 172.21.15.99:0/3696370551 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v135) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3ee4056c30 con 0x7f3ef00b0890 2023-10-08T00:16:46.705 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.704+0000 7f3ef481e700 1 -- 172.21.15.99:0/3696370551 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3ed0066f30 msgr2=0x7f3ed00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:46.705 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.704+0000 7f3ef481e700 1 --2- 172.21.15.99:0/3696370551 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3ed0066f30 0x7f3ed00693e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0006fd0 tx=0x7f3ee0008040 comp rx=0 tx=0).stop 2023-10-08T00:16:46.705 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.704+0000 7f3ef481e700 1 -- 172.21.15.99:0/3696370551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ef00b0890 msgr2=0x7f3ef015a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:46.705 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.704+0000 7f3ef481e700 1 --2- 172.21.15.99:0/3696370551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ef00b0890 0x7f3ef015a0e0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f3ef006ab60 tx=0x7f3ee4017000 comp rx=0 tx=0).stop 2023-10-08T00:16:46.705 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.704+0000 7f3ef481e700 1 -- 172.21.15.99:0/3696370551 shutdown_connections 2023-10-08T00:16:46.705 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.704+0000 7f3ef481e700 1 --2- 172.21.15.99:0/3696370551 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3ed0066f30 0x7f3ed00693e0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:46.705 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.704+0000 7f3ef481e700 1 --2- 172.21.15.99:0/3696370551 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ef006c630 0x7f3ef00b3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:46.705 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.704+0000 7f3ef481e700 1 --2- 172.21.15.99:0/3696370551 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3ef00cbd40 0x7f3ef00c32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:46.706 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.704+0000 7f3ef481e700 1 --2- 172.21.15.99:0/3696370551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ef00b0890 0x7f3ef015a0e0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:46.706 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.704+0000 7f3ef481e700 1 -- 172.21.15.99:0/3696370551 >> 172.21.15.99:0/3696370551 conn(0x7f3ef006fe50 msgr2=0x7f3ef015bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:46.706 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.704+0000 7f3ef481e700 1 -- 172.21.15.99:0/3696370551 shutdown_connections 2023-10-08T00:16:46.706 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.704+0000 7f3ef481e700 1 -- 172.21.15.99:0/3696370551 wait complete. 2023-10-08T00:16:46.706 INFO:teuthology.orchestra.run.smithi099.stderr: removed all blocklist entries 2023-10-08T00:16:46.719 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-08T00:16:46.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.833+0000 7f26489a4700 1 Processor -- start 2023-10-08T00:16:46.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.833+0000 7f26489a4700 1 -- start start 2023-10-08T00:16:46.836 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.834+0000 7f26489a4700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f264415a620 0x7f264415d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:46.836 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.834+0000 7f26489a4700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f264415d580 0x7f2644158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:46.836 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.834+0000 7f26489a4700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f264415cbb0 con 0x7f2644157870 2023-10-08T00:16:46.836 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.834+0000 7f26489a4700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f264415bdb0 con 0x7f264415d580 2023-10-08T00:16:46.836 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.834+0000 7f26489a4700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f264415bfa0 con 0x7f264415a620 2023-10-08T00:16:46.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.834+0000 7f2642d9d700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f264415d580 0x7f2644158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:46.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.834+0000 7f2642d9d700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f264415d580 0x7f2644158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47386/0 (socket says 172.21.15.99:47386) 2023-10-08T00:16:46.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.834+0000 7f2642d9d700 1 -- 172.21.15.99:0/347708488 learned_addr learned my addr 172.21.15.99:0/347708488 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:46.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.834+0000 7f2639d9b700 1 --2- 172.21.15.99:0/347708488 >> v2:172.21.15.191:3300/0 conn(0x7f264415a620 0x7f264415d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:46.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.834+0000 7f2642d9d700 1 -- 172.21.15.99:0/347708488 >> v2:172.21.15.191:3300/0 conn(0x7f264415a620 msgr2=0x7f264415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:46.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.834+0000 7f2642d9d700 1 --2- 172.21.15.99:0/347708488 >> v2:172.21.15.191:3300/0 conn(0x7f264415a620 0x7f264415d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:46.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.834+0000 7f2642d9d700 1 -- 172.21.15.99:0/347708488 >> v1:172.21.15.191:6789/0 conn(0x7f2644157870 legacy=0x7f264415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:46.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.835+0000 7f2642d9d700 1 -- 172.21.15.99:0/347708488 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f264415c280 con 0x7f264415d580 2023-10-08T00:16:46.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.835+0000 7f2642d9d700 1 --2- 172.21.15.99:0/347708488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f264415d580 0x7f2644158110 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f264415cb20 tx=0x7f263c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9d0d3719292ff19 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:46.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.835+0000 7f2641d9b700 1 -- 172.21.15.99:0/347708488 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263c00e070 con 0x7f264415d580 2023-10-08T00:16:46.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.835+0000 7f2641d9b700 1 -- 172.21.15.99:0/347708488 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f263c00ba20 con 0x7f264415d580 2023-10-08T00:16:46.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.835+0000 7f2641d9b700 1 -- 172.21.15.99:0/347708488 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263c0092e0 con 0x7f264415d580 2023-10-08T00:16:46.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.835+0000 7f26489a4700 1 -- 172.21.15.99:0/347708488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f264415d580 msgr2=0x7f2644158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:46.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.835+0000 7f26489a4700 1 --2- 172.21.15.99:0/347708488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f264415d580 0x7f2644158110 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f264415cb20 tx=0x7f263c009840 comp rx=0 tx=0).stop 2023-10-08T00:16:46.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.835+0000 7f26489a4700 1 -- 172.21.15.99:0/347708488 shutdown_connections 2023-10-08T00:16:46.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.835+0000 7f26489a4700 1 --2- 172.21.15.99:0/347708488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f264415d580 0x7f2644158110 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:46.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.835+0000 7f26489a4700 1 --2- 172.21.15.99:0/347708488 >> v2:172.21.15.191:3300/0 conn(0x7f264415a620 0x7f264415d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:46.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.835+0000 7f26489a4700 1 -- 172.21.15.99:0/347708488 >> 172.21.15.99:0/347708488 conn(0x7f2644068ce0 msgr2=0x7f26440690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:46.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.835+0000 7f26489a4700 1 -- 172.21.15.99:0/347708488 shutdown_connections 2023-10-08T00:16:46.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.835+0000 7f26489a4700 1 -- 172.21.15.99:0/347708488 wait complete. 2023-10-08T00:16:46.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.837+0000 7f26489a4700 1 Processor -- start 2023-10-08T00:16:46.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.837+0000 7f26489a4700 1 -- start start 2023-10-08T00:16:46.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.837+0000 7f26489a4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2644157870 0x7f264416df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:46.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.837+0000 7f26489a4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f264415a620 0x7f264416af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:46.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.837+0000 7f26489a4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f264415d580 0x7f26440cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:46.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.837+0000 7f26489a4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26440b2400 con 0x7f264415d580 2023-10-08T00:16:46.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.837+0000 7f26489a4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26440b2140 con 0x7f264415a620 2023-10-08T00:16:46.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.837+0000 7f26489a4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f264414d1b0 con 0x7f2644157870 2023-10-08T00:16:46.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.838+0000 7f2642d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f264415d580 0x7f26440cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:46.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.838+0000 7f2642d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f264415d580 0x7f26440cf2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47392/0 (socket says 172.21.15.99:47392) 2023-10-08T00:16:46.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.838+0000 7f2642d9d700 1 -- 172.21.15.99:0/750722765 learned_addr learned my addr 172.21.15.99:0/750722765 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:46.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.838+0000 7f264259c700 1 --2- 172.21.15.99:0/750722765 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2644157870 0x7f264416df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:46.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.838+0000 7f2639d9b700 1 --2- 172.21.15.99:0/750722765 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f264415a620 0x7f264416af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:46.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.838+0000 7f2642d9d700 1 -- 172.21.15.99:0/750722765 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2644157870 msgr2=0x7f264416df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:46.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.838+0000 7f2642d9d700 1 --2- 172.21.15.99:0/750722765 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2644157870 0x7f264416df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:46.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.838+0000 7f2642d9d700 1 -- 172.21.15.99:0/750722765 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f264415a620 msgr2=0x7f264416af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:46.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.838+0000 7f2642d9d700 1 --2- 172.21.15.99:0/750722765 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f264415a620 0x7f264416af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:46.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.838+0000 7f2642d9d700 1 -- 172.21.15.99:0/750722765 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f263c009ae0 con 0x7f264415d580 2023-10-08T00:16:46.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.838+0000 7f2642d9d700 1 --2- 172.21.15.99:0/750722765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f264415d580 0x7f26440cf2b0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f263c009880 tx=0x7f263c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:46.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.838+0000 7f263affd700 1 -- 172.21.15.99:0/750722765 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263c008a80 con 0x7f264415d580 2023-10-08T00:16:46.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.838+0000 7f263affd700 1 -- 172.21.15.99:0/750722765 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f263c008be0 con 0x7f264415d580 2023-10-08T00:16:46.844 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.838+0000 7f26489a4700 1 -- 172.21.15.99:0/750722765 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26440d03a0 con 0x7f264415d580 2023-10-08T00:16:46.844 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.838+0000 7f263affd700 1 -- 172.21.15.99:0/750722765 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263c025460 con 0x7f264415d580 2023-10-08T00:16:46.844 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.838+0000 7f26489a4700 1 -- 172.21.15.99:0/750722765 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26440cfde0 con 0x7f264415d580 2023-10-08T00:16:46.844 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.839+0000 7f26489a4700 1 -- 172.21.15.99:0/750722765 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26440b2570 con 0x7f264415d580 2023-10-08T00:16:46.844 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.840+0000 7f263affd700 1 -- 172.21.15.99:0/750722765 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f263c003040 con 0x7f264415d580 2023-10-08T00:16:46.844 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.840+0000 7f263affd700 1 --2- 172.21.15.99:0/750722765 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2624066f30 0x7f26240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:46.844 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.840+0000 7f263affd700 1 -- 172.21.15.99:0/750722765 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f263c085640 con 0x7f264415d580 2023-10-08T00:16:46.844 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.840+0000 7f264259c700 1 --2- 172.21.15.99:0/750722765 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2624066f30 0x7f26240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:46.844 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.841+0000 7f264259c700 1 --2- 172.21.15.99:0/750722765 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2624066f30 0x7f26240693e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2634003a80 tx=0x7f2634007ec0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:46.845 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:46.844+0000 7f263affd700 1 -- 172.21.15.99:0/750722765 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f26440b2570 con 0x7f264415d580 2023-10-08T00:16:47.105 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.104+0000 7f26489a4700 1 -- 172.21.15.99:0/750722765 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f26440b2540 con 0x7f264415d580 2023-10-08T00:16:47.105 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.105+0000 7f263affd700 1 -- 172.21.15.99:0/750722765 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f263c052cb0 con 0x7f264415d580 2023-10-08T00:16:47.106 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:16:47.106 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:16:47.106 INFO:teuthology.orchestra.run.smithi099.stdout: "auth_dump": [ 2023-10-08T00:16:47.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:47.107 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.a", 2023-10-08T00:16:47.107 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlFlN9AhAAKdbeH0XznSOuyUnReD6RsA==", 2023-10-08T00:16:47.107 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:16:47.107 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:16:47.107 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2023-10-08T00:16:47.108 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2023-10-08T00:16:47.108 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:16:47.108 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:47.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:47.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:47.108 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.b", 2023-10-08T00:16:47.108 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlh2G0DRAA1mYGrUrELHsS9abUxCCG2g==", 2023-10-08T00:16:47.108 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:16:47.108 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:16:47.109 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2023-10-08T00:16:47.109 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2023-10-08T00:16:47.109 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:16:47.109 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:47.109 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:47.109 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:47.109 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.c", 2023-10-08T00:16:47.109 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFl/M1DCRAANrfHJGZO1s0KOyCdUzZWIA==", 2023-10-08T00:16:47.110 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:16:47.110 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:16:47.110 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2023-10-08T00:16:47.110 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2023-10-08T00:16:47.110 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:16:47.110 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:47.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:47.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:47.111 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.d", 2023-10-08T00:16:47.111 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlr5F2FBAAYYeG6EM0/N0FiEAwSbYRyA==", 2023-10-08T00:16:47.111 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:16:47.111 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:16:47.111 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2023-10-08T00:16:47.111 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2023-10-08T00:16:47.112 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:16:47.112 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:47.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:47.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:47.112 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.0", 2023-10-08T00:16:47.112 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB08yFla7rpKxAAWTOutLcrjDeR69JbPZMoLg==", 2023-10-08T00:16:47.112 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:16:47.112 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:16:47.112 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:16:47.112 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:16:47.112 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:47.113 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:47.113 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:47.113 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.1", 2023-10-08T00:16:47.113 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB08yFlw+y2MhAAAvLDIB/tb49IlaireTqTFg==", 2023-10-08T00:16:47.113 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:16:47.113 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:16:47.113 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:16:47.114 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:16:47.114 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:47.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:47.114 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:47.114 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.2", 2023-10-08T00:16:47.114 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB18yFlh4wvABAALXOtJZoEfGmt/pOBxReooQ==", 2023-10-08T00:16:47.114 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:16:47.114 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:16:47.115 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:16:47.115 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:16:47.115 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:47.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:47.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:47.115 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.3", 2023-10-08T00:16:47.115 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB18yFl8cVfCRAAoGXTACMT+f5Avll5mxG9OA==", 2023-10-08T00:16:47.115 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:16:47.115 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:16:47.115 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:16:47.115 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:16:47.115 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:47.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:47.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:47.116 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.4", 2023-10-08T00:16:47.116 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB48yFlft9WGhAA8JZN4MjSX9wbXiMKJnfCYg==", 2023-10-08T00:16:47.116 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:16:47.116 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:16:47.116 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:16:47.116 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:16:47.116 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:47.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:47.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:47.116 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.5", 2023-10-08T00:16:47.116 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB48yFlUf1BIRAA2FMBt1jAcm4ia/vYBDyojw==", 2023-10-08T00:16:47.117 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:16:47.117 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:16:47.117 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:16:47.117 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:16:47.117 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:47.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:47.117 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:47.117 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.6", 2023-10-08T00:16:47.117 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB48yFl0SI6KhAAI6G+Sqfm7OQmGf9nK59x9Q==", 2023-10-08T00:16:47.117 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:16:47.117 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:16:47.118 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:16:47.118 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:16:47.118 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:47.118 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:47.118 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:47.118 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.7", 2023-10-08T00:16:47.118 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB48yFlo5xWMxAA4p9OSLK0JpWhEExsifX+3A==", 2023-10-08T00:16:47.118 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:16:47.118 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:16:47.118 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:16:47.119 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:16:47.119 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:47.119 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:47.119 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:47.119 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.0", 2023-10-08T00:16:47.119 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlmzhQGBAAReslU/Pvcsr0dwE6l1N3/Q==", 2023-10-08T00:16:47.119 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:16:47.119 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:16:47.119 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow r", 2023-10-08T00:16:47.119 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-08T00:16:47.119 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:47.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:47.120 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:47.120 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.1", 2023-10-08T00:16:47.120 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFl2NKsIBAAUg6rXynl1J7qB6BkUizmpA==", 2023-10-08T00:16:47.120 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:16:47.120 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:16:47.120 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2023-10-08T00:16:47.120 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2023-10-08T00:16:47.120 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2023-10-08T00:16:47.120 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:47.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:47.121 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:47.121 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.2", 2023-10-08T00:16:47.121 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlMp6rHRAArVgJfA3mY4n78hOMeVlw1Q==", 2023-10-08T00:16:47.121 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:16:47.121 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:16:47.121 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2023-10-08T00:16:47.121 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2023-10-08T00:16:47.121 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2023-10-08T00:16:47.122 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:47.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:47.122 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:47.122 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.3", 2023-10-08T00:16:47.122 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlH0gNJhAAEgGVGiabVIWknpWUohOnHA==", 2023-10-08T00:16:47.122 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:16:47.122 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:16:47.122 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2023-10-08T00:16:47.123 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2023-10-08T00:16:47.123 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2023-10-08T00:16:47.123 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:47.123 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:47.123 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:47.124 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.admin", 2023-10-08T00:16:47.124 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBw8yFlidUfIxAAIvpvkHbyCx5JFvHo/g5A9g==", 2023-10-08T00:16:47.124 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:16:47.124 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2023-10-08T00:16:47.124 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2023-10-08T00:16:47.124 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2023-10-08T00:16:47.124 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:16:47.125 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:47.125 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:47.125 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:47.125 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mgr.x", 2023-10-08T00:16:47.125 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBw8yFl2kMLOxAAsEyuLBLsjQCUS0ERDOi75A==", 2023-10-08T00:16:47.125 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:16:47.125 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2023-10-08T00:16:47.125 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile mgr", 2023-10-08T00:16:47.126 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:16:47.126 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:47.126 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:16:47.126 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:47.126 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mgr.y", 2023-10-08T00:16:47.126 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBw8yFl6ht5NxAAFzXnLDjklCePD6hw/pSfFg==", 2023-10-08T00:16:47.126 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:16:47.126 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2023-10-08T00:16:47.126 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile mgr", 2023-10-08T00:16:47.126 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:16:47.127 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:47.127 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:47.127 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:16:47.127 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:16:47.129 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.107+0000 7f26489a4700 1 -- 172.21.15.99:0/750722765 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2624066f30 msgr2=0x7f26240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:47.129 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.107+0000 7f26489a4700 1 --2- 172.21.15.99:0/750722765 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2624066f30 0x7f26240693e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2634003a80 tx=0x7f2634007ec0 comp rx=0 tx=0).stop 2023-10-08T00:16:47.129 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.107+0000 7f26489a4700 1 -- 172.21.15.99:0/750722765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f264415d580 msgr2=0x7f26440cf2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:47.129 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.107+0000 7f26489a4700 1 --2- 172.21.15.99:0/750722765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f264415d580 0x7f26440cf2b0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f263c009880 tx=0x7f263c018000 comp rx=0 tx=0).stop 2023-10-08T00:16:47.129 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.107+0000 7f26489a4700 1 -- 172.21.15.99:0/750722765 shutdown_connections 2023-10-08T00:16:47.129 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.107+0000 7f26489a4700 1 --2- 172.21.15.99:0/750722765 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2624066f30 0x7f26240693e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:47.129 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.107+0000 7f26489a4700 1 --2- 172.21.15.99:0/750722765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f264415d580 0x7f26440cf2b0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:47.129 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.107+0000 7f26489a4700 1 --2- 172.21.15.99:0/750722765 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2644157870 0x7f264416df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:47.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.107+0000 7f26489a4700 1 --2- 172.21.15.99:0/750722765 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f264415a620 0x7f264416af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:47.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.107+0000 7f26489a4700 1 -- 172.21.15.99:0/750722765 >> 172.21.15.99:0/750722765 conn(0x7f2644068ce0 msgr2=0x7f26440b5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:47.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.107+0000 7f26489a4700 1 -- 172.21.15.99:0/750722765 shutdown_connections 2023-10-08T00:16:47.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.107+0000 7f26489a4700 1 -- 172.21.15.99:0/750722765 wait complete. 2023-10-08T00:16:47.130 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:16:47.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.243+0000 7f89f68ea700 1 Processor -- start 2023-10-08T00:16:47.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.243+0000 7f89f68ea700 1 -- start start 2023-10-08T00:16:47.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.244+0000 7f89f68ea700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89f0156570 0x7f89f006d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:47.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.244+0000 7f89f68ea700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f89f015a330 0x7f89f006d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:47.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.244+0000 7f89f68ea700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89f0157c30 con 0x7f89f0156940 2023-10-08T00:16:47.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.244+0000 7f89f68ea700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89f01584f0 con 0x7f89f015a330 2023-10-08T00:16:47.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.244+0000 7f89f68ea700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89f0157670 con 0x7f89f0156570 2023-10-08T00:16:47.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.245+0000 7f89effff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89f0156570 0x7f89f006d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:47.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.245+0000 7f89ef7fe700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f89f015a330 0x7f89f006d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:47.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.245+0000 7f89f4e87700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f89f0156940 0x7f89f0156da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:51756/0 (socket says 172.21.15.99:51756) 2023-10-08T00:16:47.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.245+0000 7f89f4e87700 1 -- 172.21.15.99:0/3896798287 learned_addr learned my addr 172.21.15.99:0/3896798287 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:47.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.245+0000 7f89effff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89f0156570 0x7f89f006d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.99:36144/0 (socket says 172.21.15.99:36144) 2023-10-08T00:16:47.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.245+0000 7f89effff700 1 -- 172.21.15.99:0/3896798287 >> v2:172.21.15.191:3300/0 conn(0x7f89f015a330 msgr2=0x7f89f006d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:47.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.245+0000 7f89effff700 1 --2- 172.21.15.99:0/3896798287 >> v2:172.21.15.191:3300/0 conn(0x7f89f015a330 0x7f89f006d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:47.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.245+0000 7f89effff700 1 -- 172.21.15.99:0/3896798287 >> v1:172.21.15.191:6789/0 conn(0x7f89f0156940 legacy=0x7f89f0156da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:47.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.245+0000 7f89effff700 1 -- 172.21.15.99:0/3896798287 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f0157950 con 0x7f89f0156570 2023-10-08T00:16:47.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.245+0000 7f89ef7fe700 1 --2- 172.21.15.99:0/3896798287 >> v2:172.21.15.191:3300/0 conn(0x7f89f015a330 0x7f89f006d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:16:47.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.245+0000 7f89effff700 1 --2- 172.21.15.99:0/3896798287 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89f0156570 0x7f89f006d380 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f89e00098e0 tx=0x7f89e00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72f12de924373e7f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:47.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.245+0000 7f89eeffd700 1 -- 172.21.15.99:0/3896798287 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e000b640 con 0x7f89f0156570 2023-10-08T00:16:47.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.245+0000 7f89eeffd700 1 -- 172.21.15.99:0/3896798287 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e000b7a0 con 0x7f89f0156570 2023-10-08T00:16:47.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.245+0000 7f89eeffd700 1 -- 172.21.15.99:0/3896798287 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e000bb40 con 0x7f89f0156570 2023-10-08T00:16:47.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.246+0000 7f89f68ea700 1 -- 172.21.15.99:0/3896798287 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89f0156570 msgr2=0x7f89f006d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:47.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.246+0000 7f89f68ea700 1 --2- 172.21.15.99:0/3896798287 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89f0156570 0x7f89f006d380 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f89e00098e0 tx=0x7f89e00091a0 comp rx=0 tx=0).stop 2023-10-08T00:16:47.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.246+0000 7f89f68ea700 1 -- 172.21.15.99:0/3896798287 shutdown_connections 2023-10-08T00:16:47.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.246+0000 7f89f68ea700 1 --2- 172.21.15.99:0/3896798287 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89f0156570 0x7f89f006d380 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:47.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.246+0000 7f89f68ea700 1 --2- 172.21.15.99:0/3896798287 >> v2:172.21.15.191:3300/0 conn(0x7f89f015a330 0x7f89f006d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:47.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.246+0000 7f89f68ea700 1 -- 172.21.15.99:0/3896798287 >> 172.21.15.99:0/3896798287 conn(0x7f89f0067660 msgr2=0x7f89f0067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:47.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.246+0000 7f89f68ea700 1 -- 172.21.15.99:0/3896798287 shutdown_connections 2023-10-08T00:16:47.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.246+0000 7f89f68ea700 1 -- 172.21.15.99:0/3896798287 wait complete. 2023-10-08T00:16:47.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.247+0000 7f89f68ea700 1 Processor -- start 2023-10-08T00:16:47.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.247+0000 7f89f68ea700 1 -- start start 2023-10-08T00:16:47.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.248+0000 7f89f68ea700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89f0156570 0x7f89f014afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:47.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.248+0000 7f89f68ea700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89f0156940 0x7f89f0158980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:47.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.248+0000 7f89f68ea700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89f015a330 0x7f89f0158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:47.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.248+0000 7f89f68ea700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f00af6c0 con 0x7f89f0156940 2023-10-08T00:16:47.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.248+0000 7f89f68ea700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f00b0240 con 0x7f89f015a330 2023-10-08T00:16:47.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.248+0000 7f89f68ea700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89f00af910 con 0x7f89f0156570 2023-10-08T00:16:47.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.248+0000 7f89ef7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89f0156940 0x7f89f0158980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:47.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.248+0000 7f89ef7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89f0156940 0x7f89f0158980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47400/0 (socket says 172.21.15.99:47400) 2023-10-08T00:16:47.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.248+0000 7f89ef7fe700 1 -- 172.21.15.99:0/1811635951 learned_addr learned my addr 172.21.15.99:0/1811635951 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:47.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.248+0000 7f89f4e87700 1 --2- 172.21.15.99:0/1811635951 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89f015a330 0x7f89f0158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:47.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.248+0000 7f89effff700 1 --2- 172.21.15.99:0/1811635951 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89f0156570 0x7f89f014afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:47.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.248+0000 7f89ef7fe700 1 -- 172.21.15.99:0/1811635951 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89f0156570 msgr2=0x7f89f014afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:47.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.248+0000 7f89ef7fe700 1 --2- 172.21.15.99:0/1811635951 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89f0156570 0x7f89f014afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:47.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.248+0000 7f89ef7fe700 1 -- 172.21.15.99:0/1811635951 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89f015a330 msgr2=0x7f89f0158ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:47.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.248+0000 7f89ef7fe700 1 --2- 172.21.15.99:0/1811635951 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89f015a330 0x7f89f0158ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:47.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.248+0000 7f89ef7fe700 1 -- 172.21.15.99:0/1811635951 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e400a5d0 con 0x7f89f0156940 2023-10-08T00:16:47.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.248+0000 7f89ef7fe700 1 --2- 172.21.15.99:0/1811635951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89f0156940 0x7f89f0158980 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f89e400eaf0 tx=0x7f89e400a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:47.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.248+0000 7f89d7fff700 1 -- 172.21.15.99:0/1811635951 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e400eed0 con 0x7f89f0156940 2023-10-08T00:16:47.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.249+0000 7f89d7fff700 1 -- 172.21.15.99:0/1811635951 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e400c950 con 0x7f89f0156940 2023-10-08T00:16:47.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.249+0000 7f89f68ea700 1 -- 172.21.15.99:0/1811635951 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89e0009420 con 0x7f89f0156940 2023-10-08T00:16:47.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.249+0000 7f89d7fff700 1 -- 172.21.15.99:0/1811635951 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e400cd10 con 0x7f89f0156940 2023-10-08T00:16:47.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.249+0000 7f89f68ea700 1 -- 172.21.15.99:0/1811635951 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89f00afed0 con 0x7f89f0156940 2023-10-08T00:16:47.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.250+0000 7f89f68ea700 1 -- 172.21.15.99:0/1811635951 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89f004c3c0 con 0x7f89f0156940 2023-10-08T00:16:47.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.250+0000 7f89d7fff700 1 -- 172.21.15.99:0/1811635951 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f89e40113f0 con 0x7f89f0156940 2023-10-08T00:16:47.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.250+0000 7f89d7fff700 1 --2- 172.21.15.99:0/1811635951 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f89dc066f30 0x7f89dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:47.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.250+0000 7f89d7fff700 1 -- 172.21.15.99:0/1811635951 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f89e4015030 con 0x7f89f0156940 2023-10-08T00:16:47.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.250+0000 7f89effff700 1 --2- 172.21.15.99:0/1811635951 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f89dc066f30 0x7f89dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:47.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.251+0000 7f89effff700 1 --2- 172.21.15.99:0/1811635951 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f89dc066f30 0x7f89dc0693e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f89e00093f0 tx=0x7f89e0017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:47.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.254+0000 7f89d7fff700 1 -- 172.21.15.99:0/1811635951 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f89e40518c0 con 0x7f89f0156940 2023-10-08T00:16:47.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.438+0000 7f89f68ea700 1 -- 172.21.15.99:0/1811635951 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f89f00b0380 con 0x7f89f0156940 2023-10-08T00:16:47.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.439+0000 7f89d7fff700 1 -- 172.21.15.99:0/1811635951 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v135) v1 ==== 74+0+12145 (secure 0 0 0) 0x7f89e4054ee0 con 0x7f89f0156940 2023-10-08T00:16:47.440 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:16:47.440 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":135,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:16:46.699945+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:16:47.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.441+0000 7f89f68ea700 1 -- 172.21.15.99:0/1811635951 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f89dc066f30 msgr2=0x7f89dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:47.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.441+0000 7f89f68ea700 1 --2- 172.21.15.99:0/1811635951 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f89dc066f30 0x7f89dc0693e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f89e00093f0 tx=0x7f89e0017000 comp rx=0 tx=0).stop 2023-10-08T00:16:47.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.441+0000 7f89f68ea700 1 -- 172.21.15.99:0/1811635951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89f0156940 msgr2=0x7f89f0158980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:47.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.441+0000 7f89f68ea700 1 --2- 172.21.15.99:0/1811635951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89f0156940 0x7f89f0158980 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f89e400eaf0 tx=0x7f89e400a330 comp rx=0 tx=0).stop 2023-10-08T00:16:47.443 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.441+0000 7f89f68ea700 1 -- 172.21.15.99:0/1811635951 shutdown_connections 2023-10-08T00:16:47.443 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.441+0000 7f89f68ea700 1 --2- 172.21.15.99:0/1811635951 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f89dc066f30 0x7f89dc0693e0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:47.443 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.441+0000 7f89f68ea700 1 --2- 172.21.15.99:0/1811635951 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89f015a330 0x7f89f0158ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:47.443 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.441+0000 7f89f68ea700 1 --2- 172.21.15.99:0/1811635951 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89f0156570 0x7f89f014afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:47.443 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.441+0000 7f89f68ea700 1 --2- 172.21.15.99:0/1811635951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89f0156940 0x7f89f0158980 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:47.443 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.441+0000 7f89f68ea700 1 -- 172.21.15.99:0/1811635951 >> 172.21.15.99:0/1811635951 conn(0x7f89f0067660 msgr2=0x7f89f0159a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:47.443 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.441+0000 7f89f68ea700 1 -- 172.21.15.99:0/1811635951 shutdown_connections 2023-10-08T00:16:47.443 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.441+0000 7f89f68ea700 1 -- 172.21.15.99:0/1811635951 wait complete. 2023-10-08T00:16:47.454 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:10:42.063344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-08T00:16:47.454 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T00:16:47.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.570+0000 7f848f39f700 1 Processor -- start 2023-10-08T00:16:47.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.570+0000 7f848f39f700 1 -- start start 2023-10-08T00:16:47.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.571+0000 7f848f39f700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f84880cce30 0x7f848815aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:47.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.571+0000 7f848f39f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f848806aa00 0x7f84880b5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:47.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.571+0000 7f848f39f700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84880b4d70 con 0x7f84880b6580 2023-10-08T00:16:47.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.571+0000 7f848f39f700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84880cd8d0 con 0x7f84880cce30 2023-10-08T00:16:47.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.571+0000 7f848f39f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84880b4440 con 0x7f848806aa00 2023-10-08T00:16:47.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.571+0000 7f848eb9e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f848806aa00 0x7f84880b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:47.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.571+0000 7f848e39d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f84880cce30 0x7f848815aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:47.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.571+0000 7f848db9c700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f84880b6580 0x7f848806a2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:51764/0 (socket says 172.21.15.99:51764) 2023-10-08T00:16:47.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.571+0000 7f848eb9e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f848806aa00 0x7f84880b5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.99:36166/0 (socket says 172.21.15.99:36166) 2023-10-08T00:16:47.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.571+0000 7f848db9c700 1 -- 172.21.15.99:0/3497716833 learned_addr learned my addr 172.21.15.99:0/3497716833 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:47.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.572+0000 7f848e39d700 1 -- 172.21.15.99:0/3497716833 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f848806aa00 msgr2=0x7f84880b5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:47.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.572+0000 7f848e39d700 1 --2- 172.21.15.99:0/3497716833 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f848806aa00 0x7f84880b5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:47.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.572+0000 7f848e39d700 1 -- 172.21.15.99:0/3497716833 >> v1:172.21.15.191:6789/0 conn(0x7f84880b6580 legacy=0x7f848806a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:47.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.572+0000 7f848e39d700 1 -- 172.21.15.99:0/3497716833 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84880b4720 con 0x7f84880cce30 2023-10-08T00:16:47.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.572+0000 7f848eb9e700 1 --2- 172.21.15.99:0/3497716833 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f848806aa00 0x7f84880b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:16:47.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.572+0000 7f848e39d700 1 --2- 172.21.15.99:0/3497716833 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84880cce30 0x7f848815aca0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f84800098e0 tx=0x7f84800091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5979e1973be7d83 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:47.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.572+0000 7f848d39b700 1 -- 172.21.15.99:0/3497716833 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848000b640 con 0x7f84880cce30 2023-10-08T00:16:47.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.572+0000 7f848d39b700 1 -- 172.21.15.99:0/3497716833 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f848000b7a0 con 0x7f84880cce30 2023-10-08T00:16:47.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.572+0000 7f848d39b700 1 -- 172.21.15.99:0/3497716833 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848000bb40 con 0x7f84880cce30 2023-10-08T00:16:47.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.572+0000 7f848f39f700 1 -- 172.21.15.99:0/3497716833 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84880cce30 msgr2=0x7f848815aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:47.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.572+0000 7f848f39f700 1 --2- 172.21.15.99:0/3497716833 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84880cce30 0x7f848815aca0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f84800098e0 tx=0x7f84800091a0 comp rx=0 tx=0).stop 2023-10-08T00:16:47.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.573+0000 7f848f39f700 1 -- 172.21.15.99:0/3497716833 shutdown_connections 2023-10-08T00:16:47.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.573+0000 7f848f39f700 1 --2- 172.21.15.99:0/3497716833 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f848806aa00 0x7f84880b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:47.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.573+0000 7f848f39f700 1 --2- 172.21.15.99:0/3497716833 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84880cce30 0x7f848815aca0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:47.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.573+0000 7f848f39f700 1 -- 172.21.15.99:0/3497716833 >> 172.21.15.99:0/3497716833 conn(0x7f8488068cb0 msgr2=0x7f84880690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:47.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.573+0000 7f848f39f700 1 -- 172.21.15.99:0/3497716833 shutdown_connections 2023-10-08T00:16:47.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.573+0000 7f848f39f700 1 -- 172.21.15.99:0/3497716833 wait complete. 2023-10-08T00:16:47.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.574+0000 7f848f39f700 1 Processor -- start 2023-10-08T00:16:47.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.574+0000 7f848f39f700 1 -- start start 2023-10-08T00:16:47.580 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.574+0000 7f848f39f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f848806aa00 0x7f84880b2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:47.580 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.574+0000 7f848f39f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84880b6580 0x7f848816b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:47.580 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.574+0000 7f848f39f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84880cce30 0x7f84881676f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:47.580 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.574+0000 7f848f39f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84880cef50 con 0x7f848806aa00 2023-10-08T00:16:47.581 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.574+0000 7f848f39f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84880cfad0 con 0x7f84880cce30 2023-10-08T00:16:47.581 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.574+0000 7f848f39f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84880cf1a0 con 0x7f84880b6580 2023-10-08T00:16:47.581 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.574+0000 7f848e39d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f848806aa00 0x7f84880b2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:47.581 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.574+0000 7f848e39d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f848806aa00 0x7f84880b2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47410/0 (socket says 172.21.15.99:47410) 2023-10-08T00:16:47.581 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.574+0000 7f848e39d700 1 -- 172.21.15.99:0/2698783449 learned_addr learned my addr 172.21.15.99:0/2698783449 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:47.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.575+0000 7f848eb9e700 1 --2- 172.21.15.99:0/2698783449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84880cce30 0x7f84881676f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:47.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.575+0000 7f848db9c700 1 --2- 172.21.15.99:0/2698783449 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84880b6580 0x7f848816b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:47.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.575+0000 7f848e39d700 1 -- 172.21.15.99:0/2698783449 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84880b6580 msgr2=0x7f848816b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:47.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.575+0000 7f848e39d700 1 --2- 172.21.15.99:0/2698783449 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84880b6580 0x7f848816b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:47.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.575+0000 7f848e39d700 1 -- 172.21.15.99:0/2698783449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84880cce30 msgr2=0x7f84881676f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:47.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.575+0000 7f848e39d700 1 --2- 172.21.15.99:0/2698783449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84880cce30 0x7f84881676f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:47.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.575+0000 7f848e39d700 1 -- 172.21.15.99:0/2698783449 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8484009a20 con 0x7f848806aa00 2023-10-08T00:16:47.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.575+0000 7f848e39d700 1 --2- 172.21.15.99:0/2698783449 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f848806aa00 0x7f84880b2c00 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f8480009f80 tx=0x7f8480017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:47.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.575+0000 7f847e7fc700 1 -- 172.21.15.99:0/2698783449 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848001bc10 con 0x7f848806aa00 2023-10-08T00:16:47.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.575+0000 7f847e7fc700 1 -- 172.21.15.99:0/2698783449 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f848001bd70 con 0x7f848806aa00 2023-10-08T00:16:47.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.575+0000 7f848f39f700 1 -- 172.21.15.99:0/2698783449 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8480009420 con 0x7f848806aa00 2023-10-08T00:16:47.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.575+0000 7f847e7fc700 1 -- 172.21.15.99:0/2698783449 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8480011670 con 0x7f848806aa00 2023-10-08T00:16:47.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.575+0000 7f848f39f700 1 -- 172.21.15.99:0/2698783449 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84880cf760 con 0x7f848806aa00 2023-10-08T00:16:47.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.576+0000 7f848f39f700 1 -- 172.21.15.99:0/2698783449 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8488089550 con 0x7f848806aa00 2023-10-08T00:16:47.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.576+0000 7f847e7fc700 1 -- 172.21.15.99:0/2698783449 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8480011a20 con 0x7f848806aa00 2023-10-08T00:16:47.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.577+0000 7f847e7fc700 1 --2- 172.21.15.99:0/2698783449 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f846c066f30 0x7f846c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:47.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.577+0000 7f847e7fc700 1 -- 172.21.15.99:0/2698783449 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f84800854e0 con 0x7f848806aa00 2023-10-08T00:16:47.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.577+0000 7f848db9c700 1 --2- 172.21.15.99:0/2698783449 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f846c066f30 0x7f846c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:47.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.577+0000 7f848db9c700 1 --2- 172.21.15.99:0/2698783449 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f846c066f30 0x7f846c0693e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f848815c010 tx=0x7f8478007eb0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:47.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.581+0000 7f847e7fc700 1 -- 172.21.15.99:0/2698783449 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8480052b60 con 0x7f848806aa00 2023-10-08T00:16:47.761 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.761+0000 7f848f39f700 1 -- 172.21.15.99:0/2698783449 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f84880b4a00 con 0x7f848806aa00 2023-10-08T00:16:47.762 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.761+0000 7f847e7fc700 1 -- 172.21.15.99:0/2698783449 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v135) v1 ==== 93+0+10 (secure 0 0 0) 0x7f84800850a0 con 0x7f848806aa00 2023-10-08T00:16:47.762 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2023-10-08T00:16:47.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.763+0000 7f848f39f700 1 -- 172.21.15.99:0/2698783449 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f846c066f30 msgr2=0x7f846c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:47.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.763+0000 7f848f39f700 1 --2- 172.21.15.99:0/2698783449 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f846c066f30 0x7f846c0693e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f848815c010 tx=0x7f8478007eb0 comp rx=0 tx=0).stop 2023-10-08T00:16:47.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.763+0000 7f848f39f700 1 -- 172.21.15.99:0/2698783449 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f848806aa00 msgr2=0x7f84880b2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:47.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.763+0000 7f848f39f700 1 --2- 172.21.15.99:0/2698783449 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f848806aa00 0x7f84880b2c00 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f8480009f80 tx=0x7f8480017000 comp rx=0 tx=0).stop 2023-10-08T00:16:47.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.763+0000 7f848f39f700 1 -- 172.21.15.99:0/2698783449 shutdown_connections 2023-10-08T00:16:47.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.763+0000 7f848f39f700 1 --2- 172.21.15.99:0/2698783449 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f846c066f30 0x7f846c0693e0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:47.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.763+0000 7f848f39f700 1 --2- 172.21.15.99:0/2698783449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84880cce30 0x7f84881676f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:47.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.763+0000 7f848f39f700 1 --2- 172.21.15.99:0/2698783449 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f848806aa00 0x7f84880b2c00 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:47.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.763+0000 7f848f39f700 1 --2- 172.21.15.99:0/2698783449 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84880b6580 0x7f848816b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:47.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.763+0000 7f848f39f700 1 -- 172.21.15.99:0/2698783449 >> 172.21.15.99:0/2698783449 conn(0x7f8488068cb0 msgr2=0x7f848815c280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:47.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.763+0000 7f848f39f700 1 -- 172.21.15.99:0/2698783449 shutdown_connections 2023-10-08T00:16:47.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.763+0000 7f848f39f700 1 -- 172.21.15.99:0/2698783449 wait complete. 2023-10-08T00:16:47.776 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T00:16:47.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.888+0000 7f8a6b2e2700 1 Processor -- start 2023-10-08T00:16:47.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.888+0000 7f8a6b2e2700 1 -- start start 2023-10-08T00:16:47.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.889+0000 7f8a6b2e2700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8a64157870 0x7f8a6415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:47.898 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.889+0000 7f8a6b2e2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a6415a620 0x7f8a6415d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:47.898 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.889+0000 7f8a6b2e2700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a6415cbb0 con 0x7f8a6415d580 2023-10-08T00:16:47.898 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.889+0000 7f8a6b2e2700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a6415bfa0 con 0x7f8a64157870 2023-10-08T00:16:47.898 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.889+0000 7f8a6b2e2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6415c280 con 0x7f8a6415a620 2023-10-08T00:16:47.898 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.889+0000 7f8a6907e700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8a64157870 0x7f8a6415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:47.898 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.889+0000 7f8a6987f700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f8a6415d580 0x7f8a64158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:42958/0 (socket says 172.21.15.99:42958) 2023-10-08T00:16:47.898 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.889+0000 7f8a6987f700 1 -- 172.21.15.99:0/1297658398 learned_addr learned my addr 172.21.15.99:0/1297658398 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:47.898 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.889+0000 7f8a6887d700 1 --2- 172.21.15.99:0/1297658398 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a6415a620 0x7f8a6415d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:47.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.889+0000 7f8a6907e700 1 -- 172.21.15.99:0/1297658398 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a6415a620 msgr2=0x7f8a6415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:47.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.889+0000 7f8a6907e700 1 --2- 172.21.15.99:0/1297658398 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a6415a620 0x7f8a6415d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:47.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.889+0000 7f8a6907e700 1 -- 172.21.15.99:0/1297658398 >> v1:172.21.15.99:6789/0 conn(0x7f8a6415d580 legacy=0x7f8a64158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:47.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.889+0000 7f8a6907e700 1 -- 172.21.15.99:0/1297658398 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a6415c560 con 0x7f8a64157870 2023-10-08T00:16:47.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.889+0000 7f8a63fff700 1 -- 172.21.15.99:0/1297658398 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 161903105 0 0) 0x7f8a6415cbb0 con 0x7f8a6415d580 2023-10-08T00:16:47.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.890+0000 7f8a6887d700 1 --2- 172.21.15.99:0/1297658398 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a6415a620 0x7f8a6415d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:16:47.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.890+0000 7f8a6907e700 1 --2- 172.21.15.99:0/1297658398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a64157870 0x7f8a6415b6a0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f8a54009e60 tx=0x7f8a54009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=705c19f51be99b51 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:47.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.890+0000 7f8a63fff700 1 -- 172.21.15.99:0/1297658398 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5400e070 con 0x7f8a64157870 2023-10-08T00:16:47.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.890+0000 7f8a63fff700 1 -- 172.21.15.99:0/1297658398 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a5400bc30 con 0x7f8a64157870 2023-10-08T00:16:47.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.890+0000 7f8a63fff700 1 -- 172.21.15.99:0/1297658398 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a54012430 con 0x7f8a64157870 2023-10-08T00:16:47.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.890+0000 7f8a6b2e2700 1 -- 172.21.15.99:0/1297658398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a64157870 msgr2=0x7f8a6415b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:47.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.890+0000 7f8a6b2e2700 1 --2- 172.21.15.99:0/1297658398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a64157870 0x7f8a6415b6a0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f8a54009e60 tx=0x7f8a54009780 comp rx=0 tx=0).stop 2023-10-08T00:16:47.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.890+0000 7f8a6b2e2700 1 -- 172.21.15.99:0/1297658398 shutdown_connections 2023-10-08T00:16:47.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.890+0000 7f8a6b2e2700 1 --2- 172.21.15.99:0/1297658398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a64157870 0x7f8a6415b6a0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:47.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.890+0000 7f8a6b2e2700 1 --2- 172.21.15.99:0/1297658398 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a6415a620 0x7f8a6415d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:47.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.890+0000 7f8a6b2e2700 1 -- 172.21.15.99:0/1297658398 >> 172.21.15.99:0/1297658398 conn(0x7f8a64068ce0 msgr2=0x7f8a640690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:47.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.890+0000 7f8a6b2e2700 1 -- 172.21.15.99:0/1297658398 shutdown_connections 2023-10-08T00:16:47.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.890+0000 7f8a6b2e2700 1 -- 172.21.15.99:0/1297658398 wait complete. 2023-10-08T00:16:47.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.891+0000 7f8a6b2e2700 1 Processor -- start 2023-10-08T00:16:47.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.892+0000 7f8a6b2e2700 1 -- start start 2023-10-08T00:16:47.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.892+0000 7f8a6b2e2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a64157870 0x7f8a640b5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:47.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.892+0000 7f8a6b2e2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a6415a620 0x7f8a6414d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:47.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.892+0000 7f8a6b2e2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a6415d580 0x7f8a6414de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:47.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.892+0000 7f8a6b2e2700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6414e350 con 0x7f8a64157870 2023-10-08T00:16:47.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.892+0000 7f8a6b2e2700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6416edb0 con 0x7f8a6415a620 2023-10-08T00:16:47.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.892+0000 7f8a6b2e2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6416e920 con 0x7f8a6415d580 2023-10-08T00:16:47.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.892+0000 7f8a6907e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a64157870 0x7f8a640b5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:47.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.892+0000 7f8a6907e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a64157870 0x7f8a640b5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47434/0 (socket says 172.21.15.99:47434) 2023-10-08T00:16:47.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.892+0000 7f8a6907e700 1 -- 172.21.15.99:0/3177178013 learned_addr learned my addr 172.21.15.99:0/3177178013 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:47.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.893+0000 7f8a6907e700 1 -- 172.21.15.99:0/3177178013 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a6415d580 msgr2=0x7f8a6414de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:47.903 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.893+0000 7f8a6987f700 1 --2- 172.21.15.99:0/3177178013 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a6415d580 0x7f8a6414de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:47.903 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.893+0000 7f8a6887d700 1 --2- 172.21.15.99:0/3177178013 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a6415a620 0x7f8a6414d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:47.903 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.893+0000 7f8a6907e700 1 --2- 172.21.15.99:0/3177178013 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a6415d580 0x7f8a6414de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:47.903 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.893+0000 7f8a6907e700 1 -- 172.21.15.99:0/3177178013 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a6415a620 msgr2=0x7f8a6414d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:47.903 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.893+0000 7f8a6907e700 1 --2- 172.21.15.99:0/3177178013 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a6415a620 0x7f8a6414d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:47.903 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.893+0000 7f8a6907e700 1 -- 172.21.15.99:0/3177178013 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a54009a20 con 0x7f8a64157870 2023-10-08T00:16:47.903 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.893+0000 7f8a6907e700 1 --2- 172.21.15.99:0/3177178013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a64157870 0x7f8a640b5d60 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f8a54013000 tx=0x7f8a54019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:47.903 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.893+0000 7f8a617fa700 1 -- 172.21.15.99:0/3177178013 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a54012e10 con 0x7f8a64157870 2023-10-08T00:16:47.903 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.893+0000 7f8a617fa700 1 -- 172.21.15.99:0/3177178013 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a5401d630 con 0x7f8a64157870 2023-10-08T00:16:47.903 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.893+0000 7f8a617fa700 1 -- 172.21.15.99:0/3177178013 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5401d9f0 con 0x7f8a64157870 2023-10-08T00:16:47.903 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.893+0000 7f8a6b2e2700 1 -- 172.21.15.99:0/3177178013 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a6416dff0 con 0x7f8a64157870 2023-10-08T00:16:47.904 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.893+0000 7f8a6b2e2700 1 -- 172.21.15.99:0/3177178013 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a6416e5b0 con 0x7f8a64157870 2023-10-08T00:16:47.904 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.894+0000 7f8a6b2e2700 1 -- 172.21.15.99:0/3177178013 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a6404c3c0 con 0x7f8a64157870 2023-10-08T00:16:47.904 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.894+0000 7f8a617fa700 1 -- 172.21.15.99:0/3177178013 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8a54024070 con 0x7f8a64157870 2023-10-08T00:16:47.904 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.895+0000 7f8a617fa700 1 --2- 172.21.15.99:0/3177178013 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8a50066f60 0x7f8a50069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:47.904 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.895+0000 7f8a617fa700 1 -- 172.21.15.99:0/3177178013 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f8a54086900 con 0x7f8a64157870 2023-10-08T00:16:47.904 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.895+0000 7f8a6887d700 1 --2- 172.21.15.99:0/3177178013 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8a50066f60 0x7f8a50069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:47.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.896+0000 7f8a6887d700 1 --2- 172.21.15.99:0/3177178013 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8a50066f60 0x7f8a50069410 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8a6415b530 tx=0x7f8a58008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:47.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:47.899+0000 7f8a617fa700 1 -- 172.21.15.99:0/3177178013 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8a54053ec0 con 0x7f8a64157870 2023-10-08T00:16:48.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.084+0000 7f8a6b2e2700 1 -- 172.21.15.99:0/3177178013 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f8a6415c840 con 0x7f8a64157870 2023-10-08T00:16:48.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.085+0000 7f8a617fa700 1 -- 172.21.15.99:0/3177178013 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v135) v1 ==== 84+0+65 (secure 0 0 0) 0x7f8a5402c060 con 0x7f8a64157870 2023-10-08T00:16:48.085 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:16:48.086 INFO:teuthology.orchestra.run.smithi099.stdout:[ 2023-10-08T00:16:48.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:16:48.086 INFO:teuthology.orchestra.run.smithi099.stdout: "poolnum": 1, 2023-10-08T00:16:48.086 INFO:teuthology.orchestra.run.smithi099.stdout: "poolname": ".mgr" 2023-10-08T00:16:48.086 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:16:48.086 INFO:teuthology.orchestra.run.smithi099.stdout:] 2023-10-08T00:16:48.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.086+0000 7f8a6b2e2700 1 -- 172.21.15.99:0/3177178013 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8a50066f60 msgr2=0x7f8a50069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:48.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.086+0000 7f8a6b2e2700 1 --2- 172.21.15.99:0/3177178013 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8a50066f60 0x7f8a50069410 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8a6415b530 tx=0x7f8a58008040 comp rx=0 tx=0).stop 2023-10-08T00:16:48.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.086+0000 7f8a6b2e2700 1 -- 172.21.15.99:0/3177178013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a64157870 msgr2=0x7f8a640b5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:48.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.086+0000 7f8a6b2e2700 1 --2- 172.21.15.99:0/3177178013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a64157870 0x7f8a640b5d60 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f8a54013000 tx=0x7f8a54019000 comp rx=0 tx=0).stop 2023-10-08T00:16:48.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.086+0000 7f8a6b2e2700 1 -- 172.21.15.99:0/3177178013 shutdown_connections 2023-10-08T00:16:48.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.086+0000 7f8a6b2e2700 1 --2- 172.21.15.99:0/3177178013 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8a50066f60 0x7f8a50069410 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:48.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.086+0000 7f8a6b2e2700 1 --2- 172.21.15.99:0/3177178013 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a6415d580 0x7f8a6414de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:48.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.087+0000 7f8a6b2e2700 1 --2- 172.21.15.99:0/3177178013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a64157870 0x7f8a640b5d60 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:48.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.087+0000 7f8a6b2e2700 1 --2- 172.21.15.99:0/3177178013 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a6415a620 0x7f8a6414d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:48.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.087+0000 7f8a6b2e2700 1 -- 172.21.15.99:0/3177178013 >> 172.21.15.99:0/3177178013 conn(0x7f8a64068ce0 msgr2=0x7f8a64155550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:48.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.087+0000 7f8a6b2e2700 1 -- 172.21.15.99:0/3177178013 shutdown_connections 2023-10-08T00:16:48.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.087+0000 7f8a6b2e2700 1 -- 172.21.15.99:0/3177178013 wait complete. 2023-10-08T00:16:48.099 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T00:16:48.099 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T00:16:48.218 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.217+0000 7f4a23a4a700 1 Processor -- start 2023-10-08T00:16:48.218 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.217+0000 7f4a23a4a700 1 -- start start 2023-10-08T00:16:48.221 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.218+0000 7f4a23a4a700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4a1c159ef0 0x7f4a1c15a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:48.221 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.218+0000 7f4a23a4a700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a1c0b17b0 con 0x7f4a1c0b46f0 2023-10-08T00:16:48.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.218+0000 7f4a23a4a700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a1c0b1a60 con 0x7f4a1c0b6a10 2023-10-08T00:16:48.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.218+0000 7f4a23a4a700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a1c0b1d40 con 0x7f4a1c159ef0 2023-10-08T00:16:48.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.218+0000 7f4a21fe7700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f4a1c0b46f0 0x7f4a1c0b4ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:42968/0 (socket says 172.21.15.99:42968) 2023-10-08T00:16:48.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.218+0000 7f4a21fe7700 1 -- 172.21.15.99:0/2947131062 learned_addr learned my addr 172.21.15.99:0/2947131062 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:48.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.219+0000 7f4a217e6700 1 --2- 172.21.15.99:0/2947131062 >> v2:172.21.15.191:3300/0 conn(0x7f4a1c159ef0 0x7f4a1c15a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:48.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.219+0000 7f4a13fff700 1 -- 172.21.15.99:0/2947131062 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2658533862 0 0) 0x7f4a1c0b17b0 con 0x7f4a1c0b46f0 2023-10-08T00:16:48.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.219+0000 7f4a13fff700 1 -- 172.21.15.99:0/2947131062 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a08003570 con 0x7f4a1c0b46f0 2023-10-08T00:16:48.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.219+0000 7f4a13fff700 1 -- 172.21.15.99:0/2947131062 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 8438477 0 0) 0x7f4a08003570 con 0x7f4a1c0b46f0 2023-10-08T00:16:48.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.219+0000 7f4a13fff700 1 -- 172.21.15.99:0/2947131062 >> v2:172.21.15.191:3300/0 conn(0x7f4a1c159ef0 msgr2=0x7f4a1c15a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:48.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.219+0000 7f4a13fff700 1 --2- 172.21.15.99:0/2947131062 >> v2:172.21.15.191:3300/0 conn(0x7f4a1c159ef0 0x7f4a1c15a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:48.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.219+0000 7f4a13fff700 1 -- 172.21.15.99:0/2947131062 >> v1:172.21.15.191:6789/0 conn(0x7f4a1c0b6a10 legacy=0x7f4a1c0b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:48.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.219+0000 7f4a13fff700 1 -- 172.21.15.99:0/2947131062 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a1c0b2020 con 0x7f4a1c0b46f0 2023-10-08T00:16:48.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.219+0000 7f4a13fff700 1 -- 172.21.15.99:0/2947131062 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f4a140017f0 con 0x7f4a1c0b46f0 2023-10-08T00:16:48.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.219+0000 7f4a13fff700 1 -- 172.21.15.99:0/2947131062 >> v1:172.21.15.99:6789/0 conn(0x7f4a1c0b46f0 legacy=0x7f4a1c0b4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:48.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.219+0000 7f4a13fff700 1 --2- 172.21.15.99:0/2947131062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a08003a60 0x7f4a08003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:48.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.219+0000 7f4a13fff700 1 --2- 172.21.15.99:0/2947131062 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a08004810 0x7f4a08004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:48.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.219+0000 7f4a13fff700 1 --2- 172.21.15.99:0/2947131062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a080051c0 0x7f4a0800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:48.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.219+0000 7f4a13fff700 1 -- 172.21.15.99:0/2947131062 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c0b1d40 con 0x7f4a080051c0 2023-10-08T00:16:48.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.219+0000 7f4a13fff700 1 -- 172.21.15.99:0/2947131062 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0800c0d0 con 0x7f4a08003a60 2023-10-08T00:16:48.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.219+0000 7f4a13fff700 1 -- 172.21.15.99:0/2947131062 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0800c290 con 0x7f4a08004810 2023-10-08T00:16:48.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.219+0000 7f4a21fe7700 1 --2- 172.21.15.99:0/2947131062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a080051c0 0x7f4a0800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:48.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.219+0000 7f4a20fe5700 1 --2- 172.21.15.99:0/2947131062 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a08004810 0x7f4a08004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:48.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.219+0000 7f4a217e6700 1 --2- 172.21.15.99:0/2947131062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a08003a60 0x7f4a08003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:48.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.219+0000 7f4a21fe7700 1 -- 172.21.15.99:0/2947131062 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a08004810 msgr2=0x7f4a08004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:48.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.219+0000 7f4a21fe7700 1 --2- 172.21.15.99:0/2947131062 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a08004810 0x7f4a08004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:48.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.219+0000 7f4a21fe7700 1 -- 172.21.15.99:0/2947131062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a08003a60 msgr2=0x7f4a08003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:48.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.219+0000 7f4a21fe7700 1 --2- 172.21.15.99:0/2947131062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a08003a60 0x7f4a08003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:48.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.219+0000 7f4a21fe7700 1 -- 172.21.15.99:0/2947131062 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a08003570 con 0x7f4a080051c0 2023-10-08T00:16:48.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.219+0000 7f4a217e6700 1 --2- 172.21.15.99:0/2947131062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a08003a60 0x7f4a08003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:48.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.219+0000 7f4a20fe5700 1 --2- 172.21.15.99:0/2947131062 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a08004810 0x7f4a08004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:48.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.219+0000 7f4a21fe7700 1 -- 172.21.15.99:0/2947131062 reap_dead start 2023-10-08T00:16:48.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.220+0000 7f4a21fe7700 1 --2- 172.21.15.99:0/2947131062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a080051c0 0x7f4a0800ba60 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f4a1400a930 tx=0x7f4a140021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a2d3519333ec8f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:48.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.220+0000 7f4a13fff700 1 -- 172.21.15.99:0/2947131062 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a140070d0 con 0x7f4a080051c0 2023-10-08T00:16:48.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.220+0000 7f4a13fff700 1 -- 172.21.15.99:0/2947131062 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a14007230 con 0x7f4a080051c0 2023-10-08T00:16:48.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.220+0000 7f4a13fff700 1 -- 172.21.15.99:0/2947131062 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a140075d0 con 0x7f4a080051c0 2023-10-08T00:16:48.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.220+0000 7f4a23a4a700 1 -- 172.21.15.99:0/2947131062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a080051c0 msgr2=0x7f4a0800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:48.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.220+0000 7f4a23a4a700 1 --2- 172.21.15.99:0/2947131062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a080051c0 0x7f4a0800ba60 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f4a1400a930 tx=0x7f4a140021d0 comp rx=0 tx=0).stop 2023-10-08T00:16:48.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.220+0000 7f4a23a4a700 1 -- 172.21.15.99:0/2947131062 shutdown_connections 2023-10-08T00:16:48.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.220+0000 7f4a23a4a700 1 --2- 172.21.15.99:0/2947131062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a080051c0 0x7f4a0800ba60 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:48.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.220+0000 7f4a23a4a700 1 -- 172.21.15.99:0/2947131062 >> 172.21.15.99:0/2947131062 conn(0x7f4a1c09a0a0 msgr2=0x7f4a1c067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:48.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.220+0000 7f4a23a4a700 1 -- 172.21.15.99:0/2947131062 shutdown_connections 2023-10-08T00:16:48.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.220+0000 7f4a23a4a700 1 -- 172.21.15.99:0/2947131062 wait complete. 2023-10-08T00:16:48.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.221+0000 7f4a23a4a700 1 Processor -- start 2023-10-08T00:16:48.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.222+0000 7f4a23a4a700 1 -- start start 2023-10-08T00:16:48.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.222+0000 7f4a23a4a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a080051c0 0x7f4a1c157cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:48.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.222+0000 7f4a23a4a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a1c15bd80 0x7f4a1c0aea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:48.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.222+0000 7f4a23a4a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a1c15c1d0 0x7f4a1c0aef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:48.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.222+0000 7f4a23a4a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c15d300 con 0x7f4a1c15c1d0 2023-10-08T00:16:48.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.222+0000 7f4a23a4a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c15c6f0 con 0x7f4a080051c0 2023-10-08T00:16:48.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.222+0000 7f4a23a4a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c15c9d0 con 0x7f4a1c15bd80 2023-10-08T00:16:48.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.223+0000 7f4a21fe7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a1c15c1d0 0x7f4a1c0aef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:48.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.223+0000 7f4a21fe7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a1c15c1d0 0x7f4a1c0aef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47448/0 (socket says 172.21.15.99:47448) 2023-10-08T00:16:48.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.223+0000 7f4a21fe7700 1 -- 172.21.15.99:0/782249255 learned_addr learned my addr 172.21.15.99:0/782249255 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:48.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.223+0000 7f4a20fe5700 1 --2- 172.21.15.99:0/782249255 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a1c15bd80 0x7f4a1c0aea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:48.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.223+0000 7f4a217e6700 1 --2- 172.21.15.99:0/782249255 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a080051c0 0x7f4a1c157cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:48.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.223+0000 7f4a21fe7700 1 -- 172.21.15.99:0/782249255 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a1c15bd80 msgr2=0x7f4a1c0aea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:48.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.223+0000 7f4a21fe7700 1 --2- 172.21.15.99:0/782249255 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a1c15bd80 0x7f4a1c0aea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:48.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.223+0000 7f4a21fe7700 1 -- 172.21.15.99:0/782249255 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a080051c0 msgr2=0x7f4a1c157cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:48.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.223+0000 7f4a21fe7700 1 --2- 172.21.15.99:0/782249255 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a080051c0 0x7f4a1c157cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:48.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.223+0000 7f4a21fe7700 1 -- 172.21.15.99:0/782249255 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a1c15ccb0 con 0x7f4a1c15c1d0 2023-10-08T00:16:48.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.223+0000 7f4a21fe7700 1 --2- 172.21.15.99:0/782249255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a1c15c1d0 0x7f4a1c0aef50 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f4a14000f80 tx=0x7f4a14016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:48.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.223+0000 7f4a117fa700 1 -- 172.21.15.99:0/782249255 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a140070d0 con 0x7f4a1c15c1d0 2023-10-08T00:16:48.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.223+0000 7f4a117fa700 1 -- 172.21.15.99:0/782249255 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a1401d630 con 0x7f4a1c15c1d0 2023-10-08T00:16:48.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.223+0000 7f4a117fa700 1 -- 172.21.15.99:0/782249255 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a140017f0 con 0x7f4a1c15c1d0 2023-10-08T00:16:48.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.223+0000 7f4a23a4a700 1 -- 172.21.15.99:0/782249255 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a1c15cf90 con 0x7f4a1c15c1d0 2023-10-08T00:16:48.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.223+0000 7f4a23a4a700 1 -- 172.21.15.99:0/782249255 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a1c15d790 con 0x7f4a1c15c1d0 2023-10-08T00:16:48.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.224+0000 7f4a23a4a700 1 -- 172.21.15.99:0/782249255 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a1c15d4b0 con 0x7f4a1c15c1d0 2023-10-08T00:16:48.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.224+0000 7f4a117fa700 1 -- 172.21.15.99:0/782249255 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4a1401dca0 con 0x7f4a1c15c1d0 2023-10-08T00:16:48.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.225+0000 7f4a117fa700 1 --2- 172.21.15.99:0/782249255 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4a00066f30 0x7f4a000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:48.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.225+0000 7f4a117fa700 1 -- 172.21.15.99:0/782249255 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f4a14086470 con 0x7f4a1c15c1d0 2023-10-08T00:16:48.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.225+0000 7f4a217e6700 1 --2- 172.21.15.99:0/782249255 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4a00066f30 0x7f4a000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:48.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.226+0000 7f4a217e6700 1 --2- 172.21.15.99:0/782249255 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4a00066f30 0x7f4a000693e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4a18006c20 tx=0x7f4a18008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:48.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.229+0000 7f4a117fa700 1 -- 172.21.15.99:0/782249255 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4a1c15d4b0 con 0x7f4a1c15c1d0 2023-10-08T00:16:48.413 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.412+0000 7f4a23a4a700 1 -- 172.21.15.99:0/782249255 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f4a1c04c3c0 con 0x7f4a1c15c1d0 2023-10-08T00:16:48.714 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.713+0000 7f4a117fa700 1 -- 172.21.15.99:0/782249255 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v136) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4a14053a30 con 0x7f4a1c15c1d0 2023-10-08T00:16:48.716 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.716+0000 7f4a23a4a700 1 -- 172.21.15.99:0/782249255 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4a00066f30 msgr2=0x7f4a000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:48.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.716+0000 7f4a23a4a700 1 --2- 172.21.15.99:0/782249255 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4a00066f30 0x7f4a000693e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4a18006c20 tx=0x7f4a18008040 comp rx=0 tx=0).stop 2023-10-08T00:16:48.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.716+0000 7f4a23a4a700 1 -- 172.21.15.99:0/782249255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a1c15c1d0 msgr2=0x7f4a1c0aef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:48.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.716+0000 7f4a23a4a700 1 --2- 172.21.15.99:0/782249255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a1c15c1d0 0x7f4a1c0aef50 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f4a14000f80 tx=0x7f4a14016000 comp rx=0 tx=0).stop 2023-10-08T00:16:48.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.716+0000 7f4a23a4a700 1 -- 172.21.15.99:0/782249255 shutdown_connections 2023-10-08T00:16:48.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.716+0000 7f4a23a4a700 1 --2- 172.21.15.99:0/782249255 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4a00066f30 0x7f4a000693e0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:48.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.716+0000 7f4a23a4a700 1 --2- 172.21.15.99:0/782249255 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a080051c0 0x7f4a1c157cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:48.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.716+0000 7f4a23a4a700 1 --2- 172.21.15.99:0/782249255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a1c15c1d0 0x7f4a1c0aef50 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:48.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.716+0000 7f4a23a4a700 1 --2- 172.21.15.99:0/782249255 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a1c15bd80 0x7f4a1c0aea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:48.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.716+0000 7f4a23a4a700 1 -- 172.21.15.99:0/782249255 >> 172.21.15.99:0/782249255 conn(0x7f4a1c09a0a0 msgr2=0x7f4a1c09ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:48.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.716+0000 7f4a23a4a700 1 -- 172.21.15.99:0/782249255 shutdown_connections 2023-10-08T00:16:48.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.716+0000 7f4a23a4a700 1 -- 172.21.15.99:0/782249255 wait complete. 2023-10-08T00:16:48.718 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' created 2023-10-08T00:16:48.729 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T00:16:48.849 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.849+0000 7f635759e700 1 Processor -- start 2023-10-08T00:16:48.850 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.849+0000 7f635759e700 1 -- start start 2023-10-08T00:16:48.851 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.850+0000 7f635759e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63581542a0 0x7f63580b65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:48.851 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.850+0000 7f635759e700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f63580cd630 0x7f635806a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:48.851 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.850+0000 7f635759e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f635806ac90 con 0x7f635806a830 2023-10-08T00:16:48.851 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.850+0000 7f635759e700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f635815b2e0 con 0x7f63580cd630 2023-10-08T00:16:48.851 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.850+0000 7f635759e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f635806b0a0 con 0x7f63581542a0 2023-10-08T00:16:48.851 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.850+0000 7f6355d9b700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f63580cd630 0x7f635806a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:48.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.850+0000 7f6355d9b700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f63580cd630 0x7f635806a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47464/0 (socket says 172.21.15.99:47464) 2023-10-08T00:16:48.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.850+0000 7f6355d9b700 1 -- 172.21.15.99:0/177294347 learned_addr learned my addr 172.21.15.99:0/177294347 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:48.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.850+0000 7f635659c700 1 --2- 172.21.15.99:0/177294347 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63581542a0 0x7f63580b65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:48.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.850+0000 7f6355d9b700 1 -- 172.21.15.99:0/177294347 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63581542a0 msgr2=0x7f63580b65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:48.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.850+0000 7f6355d9b700 1 --2- 172.21.15.99:0/177294347 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63581542a0 0x7f63580b65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:48.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.850+0000 7f6355d9b700 1 -- 172.21.15.99:0/177294347 >> v1:172.21.15.191:6789/0 conn(0x7f635806a830 legacy=0x7f635815d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:48.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.850+0000 7f6355d9b700 1 -- 172.21.15.99:0/177294347 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f635815ad20 con 0x7f63580cd630 2023-10-08T00:16:48.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.850+0000 7f6355d9b700 1 --2- 172.21.15.99:0/177294347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63580cd630 0x7f635806a2f0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f634c0099b0 tx=0x7f634c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2644fd39dd8649ac server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:48.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.850+0000 7f635559a700 1 -- 172.21.15.99:0/177294347 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634c009ea0 con 0x7f63580cd630 2023-10-08T00:16:48.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.851+0000 7f635559a700 1 -- 172.21.15.99:0/177294347 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f634c00b860 con 0x7f63580cd630 2023-10-08T00:16:48.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.851+0000 7f635559a700 1 -- 172.21.15.99:0/177294347 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634c00bc00 con 0x7f63580cd630 2023-10-08T00:16:48.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.851+0000 7f635759e700 1 -- 172.21.15.99:0/177294347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63580cd630 msgr2=0x7f635806a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:48.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.851+0000 7f635759e700 1 --2- 172.21.15.99:0/177294347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63580cd630 0x7f635806a2f0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f634c0099b0 tx=0x7f634c009250 comp rx=0 tx=0).stop 2023-10-08T00:16:48.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.851+0000 7f635759e700 1 -- 172.21.15.99:0/177294347 shutdown_connections 2023-10-08T00:16:48.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.851+0000 7f635759e700 1 --2- 172.21.15.99:0/177294347 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63581542a0 0x7f63580b65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:48.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.851+0000 7f635759e700 1 --2- 172.21.15.99:0/177294347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63580cd630 0x7f635806a2f0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:48.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.851+0000 7f635759e700 1 -- 172.21.15.99:0/177294347 >> 172.21.15.99:0/177294347 conn(0x7f6358068cb0 msgr2=0x7f63580690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:48.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.851+0000 7f635759e700 1 -- 172.21.15.99:0/177294347 shutdown_connections 2023-10-08T00:16:48.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.851+0000 7f635759e700 1 -- 172.21.15.99:0/177294347 wait complete. 2023-10-08T00:16:48.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.852+0000 7f635759e700 1 Processor -- start 2023-10-08T00:16:48.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.852+0000 7f635759e700 1 -- start start 2023-10-08T00:16:48.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.853+0000 7f635759e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f635806a830 0x7f6358169e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:48.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.853+0000 7f635759e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63580cd630 0x7f6358158d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:48.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.853+0000 7f635759e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63581542a0 0x7f6358159240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:48.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.853+0000 7f635759e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6358156d20 con 0x7f63581542a0 2023-10-08T00:16:48.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.853+0000 7f635759e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6358157000 con 0x7f635806a830 2023-10-08T00:16:48.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.853+0000 7f635759e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63581572e0 con 0x7f63580cd630 2023-10-08T00:16:48.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.853+0000 7f6356d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63581542a0 0x7f6358159240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:48.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.853+0000 7f6356d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63581542a0 0x7f6358159240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47470/0 (socket says 172.21.15.99:47470) 2023-10-08T00:16:48.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.853+0000 7f6356d9d700 1 -- 172.21.15.99:0/1709715518 learned_addr learned my addr 172.21.15.99:0/1709715518 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:48.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.853+0000 7f635659c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f635806a830 0x7f6358169e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:48.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.853+0000 7f6355d9b700 1 --2- 172.21.15.99:0/1709715518 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63580cd630 0x7f6358158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:48.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.853+0000 7f6356d9d700 1 -- 172.21.15.99:0/1709715518 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63580cd630 msgr2=0x7f6358158d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:48.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.853+0000 7f6356d9d700 1 --2- 172.21.15.99:0/1709715518 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63580cd630 0x7f6358158d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:48.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.853+0000 7f6356d9d700 1 -- 172.21.15.99:0/1709715518 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f635806a830 msgr2=0x7f6358169e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:48.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.853+0000 7f6356d9d700 1 --2- 172.21.15.99:0/1709715518 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f635806a830 0x7f6358169e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:48.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.853+0000 7f6356d9d700 1 -- 172.21.15.99:0/1709715518 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f634c0094f0 con 0x7f63581542a0 2023-10-08T00:16:48.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.854+0000 7f6356d9d700 1 --2- 172.21.15.99:0/1709715518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63581542a0 0x7f6358159240 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f635000c8e0 tx=0x7f6350001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:48.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.854+0000 7f63467fc700 1 -- 172.21.15.99:0/1709715518 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6350012630 con 0x7f63581542a0 2023-10-08T00:16:48.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.854+0000 7f63467fc700 1 -- 172.21.15.99:0/1709715518 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6350012790 con 0x7f63581542a0 2023-10-08T00:16:48.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.854+0000 7f635759e700 1 -- 172.21.15.99:0/1709715518 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63581575c0 con 0x7f63581542a0 2023-10-08T00:16:48.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.854+0000 7f63467fc700 1 -- 172.21.15.99:0/1709715518 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6350012b50 con 0x7f63581542a0 2023-10-08T00:16:48.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.854+0000 7f635759e700 1 -- 172.21.15.99:0/1709715518 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f635816a580 con 0x7f63581542a0 2023-10-08T00:16:48.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.855+0000 7f635759e700 1 -- 172.21.15.99:0/1709715518 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f635816a770 con 0x7f63581542a0 2023-10-08T00:16:48.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.855+0000 7f63467fc700 1 -- 172.21.15.99:0/1709715518 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f635001a3a0 con 0x7f63581542a0 2023-10-08T00:16:48.859 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.856+0000 7f63467fc700 1 --2- 172.21.15.99:0/1709715518 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6338066f30 0x7f63380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:48.859 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.856+0000 7f63467fc700 1 -- 172.21.15.99:0/1709715518 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f63500843d0 con 0x7f63581542a0 2023-10-08T00:16:48.859 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.856+0000 7f635659c700 1 --2- 172.21.15.99:0/1709715518 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6338066f30 0x7f63380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:48.859 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.856+0000 7f635659c700 1 --2- 172.21.15.99:0/1709715518 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6338066f30 0x7f63380693e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f6348009d00 tx=0x7f6348009120 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:48.863 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:48.860+0000 7f63467fc700 1 -- 172.21.15.99:0/1709715518 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6350055210 con 0x7f63581542a0 2023-10-08T00:16:49.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.044+0000 7f635759e700 1 -- 172.21.15.99:0/1709715518 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f635806ac90 con 0x7f63581542a0 2023-10-08T00:16:49.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.714+0000 7f63467fc700 1 -- 172.21.15.99:0/1709715518 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v137) v1 ==== 170+0+0 (secure 0 0 0) 0x7f635806ac90 con 0x7f63581542a0 2023-10-08T00:16:49.720 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.720+0000 7f635759e700 1 -- 172.21.15.99:0/1709715518 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6338066f30 msgr2=0x7f63380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:49.720 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.720+0000 7f635759e700 1 --2- 172.21.15.99:0/1709715518 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6338066f30 0x7f63380693e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f6348009d00 tx=0x7f6348009120 comp rx=0 tx=0).stop 2023-10-08T00:16:49.720 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.720+0000 7f635759e700 1 -- 172.21.15.99:0/1709715518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63581542a0 msgr2=0x7f6358159240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:49.721 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.720+0000 7f635759e700 1 --2- 172.21.15.99:0/1709715518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63581542a0 0x7f6358159240 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f635000c8e0 tx=0x7f6350001710 comp rx=0 tx=0).stop 2023-10-08T00:16:49.721 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.720+0000 7f635759e700 1 -- 172.21.15.99:0/1709715518 shutdown_connections 2023-10-08T00:16:49.721 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.720+0000 7f635759e700 1 --2- 172.21.15.99:0/1709715518 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6338066f30 0x7f63380693e0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:49.721 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.720+0000 7f635759e700 1 --2- 172.21.15.99:0/1709715518 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f635806a830 0x7f6358169e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:49.721 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.720+0000 7f635759e700 1 --2- 172.21.15.99:0/1709715518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63581542a0 0x7f6358159240 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:49.721 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.720+0000 7f635759e700 1 --2- 172.21.15.99:0/1709715518 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63580cd630 0x7f6358158d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:49.721 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.720+0000 7f635759e700 1 -- 172.21.15.99:0/1709715518 >> 172.21.15.99:0/1709715518 conn(0x7f6358068cb0 msgr2=0x7f63580b2490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:49.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.721+0000 7f635759e700 1 -- 172.21.15.99:0/1709715518 shutdown_connections 2023-10-08T00:16:49.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.721+0000 7f635759e700 1 -- 172.21.15.99:0/1709715518 wait complete. 2023-10-08T00:16:49.724 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' created 2023-10-08T00:16:49.735 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T00:16:49.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.852+0000 7f70ca63a700 1 Processor -- start 2023-10-08T00:16:49.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.852+0000 7f70ca63a700 1 -- start start 2023-10-08T00:16:49.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.853+0000 7f70ca63a700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f70c4159e80 0x7f70c40b4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:49.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.853+0000 7f70ca63a700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f70c40aada0 0x7f70c40ab240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:49.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.853+0000 7f70ca63a700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70c40b6b10 con 0x7f70c40ab780 2023-10-08T00:16:49.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.853+0000 7f70ca63a700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70c40b50a0 con 0x7f70c40aada0 2023-10-08T00:16:49.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.853+0000 7f70ca63a700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70c40b0e70 con 0x7f70c4159e80 2023-10-08T00:16:49.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.853+0000 7f70c8e37700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f70c40aada0 0x7f70c40ab240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:49.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.853+0000 7f70c9e39700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f70c40ab780 0x7f70c40b2570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:42972/0 (socket says 172.21.15.99:42972) 2023-10-08T00:16:49.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.853+0000 7f70c9e39700 1 -- 172.21.15.99:0/1181046801 learned_addr learned my addr 172.21.15.99:0/1181046801 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:49.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.853+0000 7f70c9638700 1 --2- 172.21.15.99:0/1181046801 >> v2:172.21.15.191:3300/0 conn(0x7f70c4159e80 0x7f70c40b4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:49.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.853+0000 7f70c8e37700 1 -- 172.21.15.99:0/1181046801 >> v2:172.21.15.191:3300/0 conn(0x7f70c4159e80 msgr2=0x7f70c40b4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:49.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.853+0000 7f70c8e37700 1 --2- 172.21.15.99:0/1181046801 >> v2:172.21.15.191:3300/0 conn(0x7f70c4159e80 0x7f70c40b4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:49.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.853+0000 7f70c8e37700 1 -- 172.21.15.99:0/1181046801 >> v1:172.21.15.99:6789/0 conn(0x7f70c40ab780 legacy=0x7f70c40b2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:49.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.853+0000 7f70c8e37700 1 -- 172.21.15.99:0/1181046801 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c40b2310 con 0x7f70c40aada0 2023-10-08T00:16:49.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.853+0000 7f70bbfff700 1 -- 172.21.15.99:0/1181046801 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2080585703 0 0) 0x7f70c40b6b10 con 0x7f70c40ab780 2023-10-08T00:16:49.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.853+0000 7f70c8e37700 1 --2- 172.21.15.99:0/1181046801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70c40aada0 0x7f70c40ab240 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f70c408a5c0 tx=0x7f70b40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a56ba40fe627d0d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:49.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.854+0000 7f70bbfff700 1 -- 172.21.15.99:0/1181046801 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b400b6b0 con 0x7f70c40aada0 2023-10-08T00:16:49.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.854+0000 7f70bbfff700 1 -- 172.21.15.99:0/1181046801 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70b400b810 con 0x7f70c40aada0 2023-10-08T00:16:49.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.854+0000 7f70bbfff700 1 -- 172.21.15.99:0/1181046801 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b400bbb0 con 0x7f70c40aada0 2023-10-08T00:16:49.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.854+0000 7f70ca63a700 1 -- 172.21.15.99:0/1181046801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70c40aada0 msgr2=0x7f70c40ab240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:49.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.854+0000 7f70ca63a700 1 --2- 172.21.15.99:0/1181046801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70c40aada0 0x7f70c40ab240 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f70c408a5c0 tx=0x7f70b40091a0 comp rx=0 tx=0).stop 2023-10-08T00:16:49.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.854+0000 7f70ca63a700 1 -- 172.21.15.99:0/1181046801 shutdown_connections 2023-10-08T00:16:49.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.854+0000 7f70ca63a700 1 --2- 172.21.15.99:0/1181046801 >> v2:172.21.15.191:3300/0 conn(0x7f70c4159e80 0x7f70c40b4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:49.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.854+0000 7f70ca63a700 1 --2- 172.21.15.99:0/1181046801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70c40aada0 0x7f70c40ab240 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:49.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.854+0000 7f70ca63a700 1 -- 172.21.15.99:0/1181046801 >> 172.21.15.99:0/1181046801 conn(0x7f70c4099f50 msgr2=0x7f70c4067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:49.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.854+0000 7f70ca63a700 1 -- 172.21.15.99:0/1181046801 shutdown_connections 2023-10-08T00:16:49.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.854+0000 7f70ca63a700 1 -- 172.21.15.99:0/1181046801 wait complete. 2023-10-08T00:16:49.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.855+0000 7f70ca63a700 1 Processor -- start 2023-10-08T00:16:49.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.855+0000 7f70ca63a700 1 -- start start 2023-10-08T00:16:49.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.856+0000 7f70ca63a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70c40aada0 0x7f70c414dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:49.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.856+0000 7f70ca63a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70c40ab780 0x7f70c4155fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:49.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.856+0000 7f70ca63a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f70c4159e80 0x7f70c40d0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:49.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.856+0000 7f70ca63a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c41577a0 con 0x7f70c40aada0 2023-10-08T00:16:49.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.856+0000 7f70ca63a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c4156b90 con 0x7f70c40ab780 2023-10-08T00:16:49.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.856+0000 7f70ca63a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70c4156e70 con 0x7f70c4159e80 2023-10-08T00:16:49.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.856+0000 7f70c9638700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70c40aada0 0x7f70c414dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:49.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.856+0000 7f70c9638700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70c40aada0 0x7f70c414dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47490/0 (socket says 172.21.15.99:47490) 2023-10-08T00:16:49.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.856+0000 7f70c9638700 1 -- 172.21.15.99:0/3858105543 learned_addr learned my addr 172.21.15.99:0/3858105543 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:49.861 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.856+0000 7f70c8e37700 1 --2- 172.21.15.99:0/3858105543 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70c40ab780 0x7f70c4155fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:49.861 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.856+0000 7f70c9e39700 1 --2- 172.21.15.99:0/3858105543 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f70c4159e80 0x7f70c40d0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:49.861 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.857+0000 7f70c9638700 1 -- 172.21.15.99:0/3858105543 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f70c4159e80 msgr2=0x7f70c40d0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:49.861 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.857+0000 7f70c9638700 1 --2- 172.21.15.99:0/3858105543 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f70c4159e80 0x7f70c40d0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:49.861 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.857+0000 7f70c9638700 1 -- 172.21.15.99:0/3858105543 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70c40ab780 msgr2=0x7f70c4155fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:49.861 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.857+0000 7f70c9638700 1 --2- 172.21.15.99:0/3858105543 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70c40ab780 0x7f70c4155fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:49.861 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.857+0000 7f70c9638700 1 -- 172.21.15.99:0/3858105543 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b4009420 con 0x7f70c40aada0 2023-10-08T00:16:49.861 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.857+0000 7f70c9638700 1 --2- 172.21.15.99:0/3858105543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70c40aada0 0x7f70c414dea0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f70c000d890 tx=0x7f70c0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:49.861 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.857+0000 7f70b97fa700 1 -- 172.21.15.99:0/3858105543 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c00125f0 con 0x7f70c40aada0 2023-10-08T00:16:49.861 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.857+0000 7f70b97fa700 1 -- 172.21.15.99:0/3858105543 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c0012750 con 0x7f70c40aada0 2023-10-08T00:16:49.862 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.857+0000 7f70ca63a700 1 -- 172.21.15.99:0/3858105543 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70c4157150 con 0x7f70c40aada0 2023-10-08T00:16:49.863 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.857+0000 7f70b97fa700 1 -- 172.21.15.99:0/3858105543 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c0012af0 con 0x7f70c40aada0 2023-10-08T00:16:49.863 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.857+0000 7f70ca63a700 1 -- 172.21.15.99:0/3858105543 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70c41564e0 con 0x7f70c40aada0 2023-10-08T00:16:49.863 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.858+0000 7f70ca63a700 1 -- 172.21.15.99:0/3858105543 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70c4089110 con 0x7f70c40aada0 2023-10-08T00:16:49.863 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.858+0000 7f70b97fa700 1 -- 172.21.15.99:0/3858105543 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f70c001e460 con 0x7f70c40aada0 2023-10-08T00:16:49.863 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.859+0000 7f70b97fa700 1 --2- 172.21.15.99:0/3858105543 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f70a8066f30 0x7f70a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:49.863 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.859+0000 7f70b97fa700 1 -- 172.21.15.99:0/3858105543 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f70c0084a40 con 0x7f70c40aada0 2023-10-08T00:16:49.863 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.859+0000 7f70c8e37700 1 --2- 172.21.15.99:0/3858105543 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f70a8066f30 0x7f70a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:49.863 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.859+0000 7f70c8e37700 1 --2- 172.21.15.99:0/3858105543 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f70a8066f30 0x7f70a80693e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f70b40093f0 tx=0x7f70b4017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:49.864 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:49.863+0000 7f70b97fa700 1 -- 172.21.15.99:0/3858105543 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f70c0086e60 con 0x7f70c40aada0 2023-10-08T00:16:50.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.090+0000 7f70ca63a700 1 -- 172.21.15.99:0/3858105543 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f70c40b0310 con 0x7f70c40aada0 2023-10-08T00:16:50.740 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.740+0000 7f70b97fa700 1 -- 172.21.15.99:0/3858105543 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v60) v1 ==== 174+0+0 (secure 0 0 0) 0x7f70c00557a0 con 0x7f70c40aada0 2023-10-08T00:16:50.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.744+0000 7f70ca63a700 1 -- 172.21.15.99:0/3858105543 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f70a8066f30 msgr2=0x7f70a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:50.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.744+0000 7f70ca63a700 1 --2- 172.21.15.99:0/3858105543 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f70a8066f30 0x7f70a80693e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f70b40093f0 tx=0x7f70b4017000 comp rx=0 tx=0).stop 2023-10-08T00:16:50.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.744+0000 7f70ca63a700 1 -- 172.21.15.99:0/3858105543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70c40aada0 msgr2=0x7f70c414dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:50.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.744+0000 7f70ca63a700 1 --2- 172.21.15.99:0/3858105543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70c40aada0 0x7f70c414dea0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f70c000d890 tx=0x7f70c0008040 comp rx=0 tx=0).stop 2023-10-08T00:16:50.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.745+0000 7f70ca63a700 1 -- 172.21.15.99:0/3858105543 shutdown_connections 2023-10-08T00:16:50.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.745+0000 7f70ca63a700 1 --2- 172.21.15.99:0/3858105543 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f70a8066f30 0x7f70a80693e0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:50.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.745+0000 7f70ca63a700 1 --2- 172.21.15.99:0/3858105543 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f70c4159e80 0x7f70c40d0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:50.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.745+0000 7f70ca63a700 1 --2- 172.21.15.99:0/3858105543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70c40aada0 0x7f70c414dea0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:50.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.745+0000 7f70ca63a700 1 --2- 172.21.15.99:0/3858105543 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70c40ab780 0x7f70c4155fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:50.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.745+0000 7f70ca63a700 1 -- 172.21.15.99:0/3858105543 >> 172.21.15.99:0/3858105543 conn(0x7f70c4099f50 msgr2=0x7f70c415bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:50.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.745+0000 7f70ca63a700 1 -- 172.21.15.99:0/3858105543 shutdown_connections 2023-10-08T00:16:50.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.745+0000 7f70ca63a700 1 -- 172.21.15.99:0/3858105543 wait complete. 2023-10-08T00:16:50.747 INFO:teuthology.orchestra.run.smithi099.stderr:new fs with metadata pool 16 and data pool 17 2023-10-08T00:16:50.758 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:16:50.872 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.871+0000 7f6dca84f700 1 Processor -- start 2023-10-08T00:16:50.872 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.871+0000 7f6dca84f700 1 -- start start 2023-10-08T00:16:50.874 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.872+0000 7f6dca84f700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6dc4157870 0x7f6dc415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:50.874 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.872+0000 7f6dca84f700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f6dc4157c40 0x7f6dc4158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:50.875 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.872+0000 7f6dca84f700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dc415cbb0 con 0x7f6dc415a620 2023-10-08T00:16:50.875 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.872+0000 7f6dca84f700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dc415bfa0 con 0x7f6dc4157870 2023-10-08T00:16:50.875 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.872+0000 7f6dca84f700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dc415c280 con 0x7f6dc4157c40 2023-10-08T00:16:50.875 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.872+0000 7f6dc3fff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6dc4157870 0x7f6dc415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:50.875 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.872+0000 7f6dc37fe700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f6dc415a620 0x7f6dc415d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:42974/0 (socket says 172.21.15.99:42974) 2023-10-08T00:16:50.875 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.872+0000 7f6dc37fe700 1 -- 172.21.15.99:0/2323072878 learned_addr learned my addr 172.21.15.99:0/2323072878 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:50.875 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.873+0000 7f6dc2ffd700 1 -- 172.21.15.99:0/2323072878 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 438419780 0 0) 0x7f6dc415cbb0 con 0x7f6dc415a620 2023-10-08T00:16:50.875 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.873+0000 7f6dc8dec700 1 --2- 172.21.15.99:0/2323072878 >> v2:172.21.15.191:3300/0 conn(0x7f6dc4157c40 0x7f6dc4158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:50.875 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.873+0000 7f6dc2ffd700 1 -- 172.21.15.99:0/2323072878 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6dac003570 con 0x7f6dc415a620 2023-10-08T00:16:50.875 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.873+0000 7f6dc3fff700 1 -- 172.21.15.99:0/2323072878 >> v2:172.21.15.191:3300/0 conn(0x7f6dc4157c40 msgr2=0x7f6dc4158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:50.875 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.873+0000 7f6dc3fff700 1 --2- 172.21.15.99:0/2323072878 >> v2:172.21.15.191:3300/0 conn(0x7f6dc4157c40 0x7f6dc4158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:50.875 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.873+0000 7f6dc3fff700 1 -- 172.21.15.99:0/2323072878 >> v1:172.21.15.99:6789/0 conn(0x7f6dc415a620 legacy=0x7f6dc415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:50.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.873+0000 7f6dc3fff700 1 -- 172.21.15.99:0/2323072878 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc415c560 con 0x7f6dc4157870 2023-10-08T00:16:50.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.873+0000 7f6dc8dec700 1 --2- 172.21.15.99:0/2323072878 >> v2:172.21.15.191:3300/0 conn(0x7f6dc4157c40 0x7f6dc4158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:16:50.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.873+0000 7f6dc3fff700 1 --2- 172.21.15.99:0/2323072878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6dc4157870 0x7f6dc415b6a0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f6dc406b4e0 tx=0x7f6db40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36fbdf1a47f9c8dc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:50.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.873+0000 7f6dc2ffd700 1 -- 172.21.15.99:0/2323072878 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db4011630 con 0x7f6dc4157870 2023-10-08T00:16:50.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.873+0000 7f6dc2ffd700 1 -- 172.21.15.99:0/2323072878 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db4011790 con 0x7f6dc4157870 2023-10-08T00:16:50.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.873+0000 7f6dc2ffd700 1 -- 172.21.15.99:0/2323072878 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db4011b30 con 0x7f6dc4157870 2023-10-08T00:16:50.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.873+0000 7f6dca84f700 1 -- 172.21.15.99:0/2323072878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6dc4157870 msgr2=0x7f6dc415b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:50.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.873+0000 7f6dca84f700 1 --2- 172.21.15.99:0/2323072878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6dc4157870 0x7f6dc415b6a0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f6dc406b4e0 tx=0x7f6db40091a0 comp rx=0 tx=0).stop 2023-10-08T00:16:50.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.873+0000 7f6dca84f700 1 -- 172.21.15.99:0/2323072878 shutdown_connections 2023-10-08T00:16:50.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.873+0000 7f6dca84f700 1 --2- 172.21.15.99:0/2323072878 >> v2:172.21.15.191:3300/0 conn(0x7f6dc4157c40 0x7f6dc4158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:50.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.873+0000 7f6dca84f700 1 --2- 172.21.15.99:0/2323072878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6dc4157870 0x7f6dc415b6a0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:50.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.874+0000 7f6dca84f700 1 -- 172.21.15.99:0/2323072878 >> 172.21.15.99:0/2323072878 conn(0x7f6dc4068ce0 msgr2=0x7f6dc40690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:50.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.874+0000 7f6dca84f700 1 -- 172.21.15.99:0/2323072878 shutdown_connections 2023-10-08T00:16:50.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.874+0000 7f6dca84f700 1 -- 172.21.15.99:0/2323072878 wait complete. 2023-10-08T00:16:50.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.875+0000 7f6dca84f700 1 Processor -- start 2023-10-08T00:16:50.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.875+0000 7f6dca84f700 1 -- start start 2023-10-08T00:16:50.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.875+0000 7f6dca84f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6dc4157870 0x7f6dc415e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:50.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.875+0000 7f6dca84f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dc4157c40 0x7f6dc415f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:50.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.875+0000 7f6dca84f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6dc415a620 0x7f6dc40c6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:50.880 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.875+0000 7f6dca84f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc40c6db0 con 0x7f6dc415a620 2023-10-08T00:16:50.880 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.875+0000 7f6dca84f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc415e5f0 con 0x7f6dc4157c40 2023-10-08T00:16:50.880 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.876+0000 7f6dca84f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc415e8d0 con 0x7f6dc4157870 2023-10-08T00:16:50.880 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.876+0000 7f6dc8dec700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6dc415a620 0x7f6dc40c6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:50.880 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.876+0000 7f6dc8dec700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6dc415a620 0x7f6dc40c6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47506/0 (socket says 172.21.15.99:47506) 2023-10-08T00:16:50.880 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.876+0000 7f6dc8dec700 1 -- 172.21.15.99:0/4193010091 learned_addr learned my addr 172.21.15.99:0/4193010091 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:50.880 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.876+0000 7f6dc3fff700 1 --2- 172.21.15.99:0/4193010091 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6dc4157870 0x7f6dc415e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:50.880 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.876+0000 7f6dc8dec700 1 -- 172.21.15.99:0/4193010091 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6dc4157870 msgr2=0x7f6dc415e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:50.880 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.876+0000 7f6dc8dec700 1 --2- 172.21.15.99:0/4193010091 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6dc4157870 0x7f6dc415e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:50.880 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.876+0000 7f6dc8dec700 1 -- 172.21.15.99:0/4193010091 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dc4157c40 msgr2=0x7f6dc415f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:50.880 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.876+0000 7f6dc8dec700 1 --2- 172.21.15.99:0/4193010091 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dc4157c40 0x7f6dc415f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:50.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.876+0000 7f6dc8dec700 1 -- 172.21.15.99:0/4193010091 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db4009420 con 0x7f6dc415a620 2023-10-08T00:16:50.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.876+0000 7f6dc8dec700 1 --2- 172.21.15.99:0/4193010091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6dc415a620 0x7f6dc40c6870 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc00a380 tx=0x7f6dbc0078f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:50.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.876+0000 7f6dabfff700 1 -- 172.21.15.99:0/4193010091 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc00ae10 con 0x7f6dc415a620 2023-10-08T00:16:50.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.876+0000 7f6dabfff700 1 -- 172.21.15.99:0/4193010091 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dbc00d650 con 0x7f6dc415a620 2023-10-08T00:16:50.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.876+0000 7f6dca84f700 1 -- 172.21.15.99:0/4193010091 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dc415ebb0 con 0x7f6dc415a620 2023-10-08T00:16:50.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.876+0000 7f6dabfff700 1 -- 172.21.15.99:0/4193010091 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc00da10 con 0x7f6dc415a620 2023-10-08T00:16:50.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.876+0000 7f6dca84f700 1 -- 172.21.15.99:0/4193010091 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dc415f170 con 0x7f6dc415a620 2023-10-08T00:16:50.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.877+0000 7f6dca84f700 1 -- 172.21.15.99:0/4193010091 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dc404c3c0 con 0x7f6dc415a620 2023-10-08T00:16:50.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.878+0000 7f6dabfff700 1 -- 172.21.15.99:0/4193010091 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6dbc003020 con 0x7f6dc415a620 2023-10-08T00:16:50.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.878+0000 7f6dabfff700 1 --2- 172.21.15.99:0/4193010091 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6db0066f30 0x7f6db00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:50.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.878+0000 7f6dabfff700 1 -- 172.21.15.99:0/4193010091 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6dbc084d30 con 0x7f6dc415a620 2023-10-08T00:16:50.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.878+0000 7f6dc3fff700 1 --2- 172.21.15.99:0/4193010091 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6db0066f30 0x7f6db00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:50.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.879+0000 7f6dc3fff700 1 --2- 172.21.15.99:0/4193010091 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6db0066f30 0x7f6db00693e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f6db4003080 tx=0x7f6db4017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:50.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:50.882+0000 7f6dabfff700 1 -- 172.21.15.99:0/4193010091 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6dbc04d5a0 con 0x7f6dc415a620 2023-10-08T00:16:51.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.065+0000 7f6dca84f700 1 -- 172.21.15.99:0/4193010091 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6dc415c840 con 0x7f6dc415a620 2023-10-08T00:16:51.066 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.066+0000 7f6dabfff700 1 -- 172.21.15.99:0/4193010091 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+15378 (secure 0 0 0) 0x7f6dbc051fa0 con 0x7f6dc415a620 2023-10-08T00:16:51.066 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:16:51.066 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":138,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:16:50.092194+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:16:48.430607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-08T00:16:49.063424+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:16:51.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.068+0000 7f6dca84f700 1 -- 172.21.15.99:0/4193010091 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6db0066f30 msgr2=0x7f6db00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:51.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.068+0000 7f6dca84f700 1 --2- 172.21.15.99:0/4193010091 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6db0066f30 0x7f6db00693e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f6db4003080 tx=0x7f6db4017000 comp rx=0 tx=0).stop 2023-10-08T00:16:51.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.068+0000 7f6dca84f700 1 -- 172.21.15.99:0/4193010091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6dc415a620 msgr2=0x7f6dc40c6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:51.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.068+0000 7f6dca84f700 1 --2- 172.21.15.99:0/4193010091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6dc415a620 0x7f6dc40c6870 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc00a380 tx=0x7f6dbc0078f0 comp rx=0 tx=0).stop 2023-10-08T00:16:51.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.068+0000 7f6dca84f700 1 -- 172.21.15.99:0/4193010091 shutdown_connections 2023-10-08T00:16:51.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.068+0000 7f6dca84f700 1 --2- 172.21.15.99:0/4193010091 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6db0066f30 0x7f6db00693e0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:51.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.068+0000 7f6dca84f700 1 --2- 172.21.15.99:0/4193010091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6dc415a620 0x7f6dc40c6870 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:51.069 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.068+0000 7f6dca84f700 1 --2- 172.21.15.99:0/4193010091 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6dc4157870 0x7f6dc415e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:51.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.068+0000 7f6dca84f700 1 --2- 172.21.15.99:0/4193010091 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dc4157c40 0x7f6dc415f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:51.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.068+0000 7f6dca84f700 1 -- 172.21.15.99:0/4193010091 >> 172.21.15.99:0/4193010091 conn(0x7f6dc4068ce0 msgr2=0x7f6dc40b5840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:51.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.068+0000 7f6dca84f700 1 -- 172.21.15.99:0/4193010091 shutdown_connections 2023-10-08T00:16:51.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.068+0000 7f6dca84f700 1 -- 172.21.15.99:0/4193010091 wait complete. 2023-10-08T00:16:51.081 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:16:51.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.196+0000 7fc7edf96700 1 Processor -- start 2023-10-08T00:16:51.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.196+0000 7fc7edf96700 1 -- start start 2023-10-08T00:16:51.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.197+0000 7fc7edf96700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc7e8156700 0x7fc7e8156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:51.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.197+0000 7fc7edf96700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7e8067d10 0x7fc7e806dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:51.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.197+0000 7fc7edf96700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7e81573c0 con 0x7fc7e8067870 2023-10-08T00:16:51.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.197+0000 7fc7edf96700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7e8068140 con 0x7fc7e8156700 2023-10-08T00:16:51.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.197+0000 7fc7edf96700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e80c7e90 con 0x7fc7e8067d10 2023-10-08T00:16:51.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.197+0000 7fc7ecf94700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc7e8156700 0x7fc7e8156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:51.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.197+0000 7fc7ed795700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7e8067d10 0x7fc7e806dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:51.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.197+0000 7fc7e7fff700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7fc7e8067870 0x7fc7e80c88e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:51790/0 (socket says 172.21.15.99:51790) 2023-10-08T00:16:51.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.197+0000 7fc7e7fff700 1 -- 172.21.15.99:0/3130726538 learned_addr learned my addr 172.21.15.99:0/3130726538 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:51.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.197+0000 7fc7ecf94700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc7e8156700 0x7fc7e8156ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.99:48844/0 (socket says 172.21.15.99:48844) 2023-10-08T00:16:51.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.198+0000 7fc7ed795700 1 -- 172.21.15.99:0/3130726538 >> v2:172.21.15.191:3300/0 conn(0x7fc7e8156700 msgr2=0x7fc7e8156ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:51.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.198+0000 7fc7ed795700 1 --2- 172.21.15.99:0/3130726538 >> v2:172.21.15.191:3300/0 conn(0x7fc7e8156700 0x7fc7e8156ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:51.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.198+0000 7fc7ed795700 1 -- 172.21.15.99:0/3130726538 >> v1:172.21.15.191:6789/0 conn(0x7fc7e8067870 legacy=0x7fc7e80c88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:51.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.198+0000 7fc7ed795700 1 -- 172.21.15.99:0/3130726538 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7e80c8450 con 0x7fc7e8067d10 2023-10-08T00:16:51.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.198+0000 7fc7e77fe700 1 -- 172.21.15.99:0/3130726538 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 228405844 0 0) 0x7fc7e81573c0 con 0x7fc7e8067870 2023-10-08T00:16:51.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.198+0000 7fc7ecf94700 1 --2- 172.21.15.99:0/3130726538 >> v2:172.21.15.191:3300/0 conn(0x7fc7e8156700 0x7fc7e8156ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:16:51.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.198+0000 7fc7ed795700 1 --2- 172.21.15.99:0/3130726538 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7e8067d10 0x7fc7e806dfb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc7e000d0d0 tx=0x7fc7e000b990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ada86f41e3a1ae66 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:51.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.198+0000 7fc7e77fe700 1 -- 172.21.15.99:0/3130726538 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e0010070 con 0x7fc7e8067d10 2023-10-08T00:16:51.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.198+0000 7fc7e77fe700 1 -- 172.21.15.99:0/3130726538 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7e000b2e0 con 0x7fc7e8067d10 2023-10-08T00:16:51.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.198+0000 7fc7e77fe700 1 -- 172.21.15.99:0/3130726538 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e000dd80 con 0x7fc7e8067d10 2023-10-08T00:16:51.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.198+0000 7fc7edf96700 1 -- 172.21.15.99:0/3130726538 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7e8067d10 msgr2=0x7fc7e806dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:51.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.198+0000 7fc7edf96700 1 --2- 172.21.15.99:0/3130726538 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7e8067d10 0x7fc7e806dfb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc7e000d0d0 tx=0x7fc7e000b990 comp rx=0 tx=0).stop 2023-10-08T00:16:51.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.198+0000 7fc7edf96700 1 -- 172.21.15.99:0/3130726538 shutdown_connections 2023-10-08T00:16:51.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.198+0000 7fc7edf96700 1 --2- 172.21.15.99:0/3130726538 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7e8067d10 0x7fc7e806dfb0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:51.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.198+0000 7fc7edf96700 1 --2- 172.21.15.99:0/3130726538 >> v2:172.21.15.191:3300/0 conn(0x7fc7e8156700 0x7fc7e8156ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:51.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.198+0000 7fc7edf96700 1 -- 172.21.15.99:0/3130726538 >> 172.21.15.99:0/3130726538 conn(0x7fc7e80cd8f0 msgr2=0x7fc7e80cdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:51.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.198+0000 7fc7edf96700 1 -- 172.21.15.99:0/3130726538 shutdown_connections 2023-10-08T00:16:51.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.199+0000 7fc7edf96700 1 -- 172.21.15.99:0/3130726538 wait complete. 2023-10-08T00:16:51.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.200+0000 7fc7edf96700 1 Processor -- start 2023-10-08T00:16:51.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.200+0000 7fc7edf96700 1 -- start start 2023-10-08T00:16:51.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.200+0000 7fc7edf96700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7e8067870 0x7fc7e8154750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:51.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.200+0000 7fc7edf96700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7e8067d10 0x7fc7e80a71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:51.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.200+0000 7fc7edf96700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7e8156700 0x7fc7e8157190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:51.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.200+0000 7fc7edf96700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e80a7af0 con 0x7fc7e8067870 2023-10-08T00:16:51.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.200+0000 7fc7edf96700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e80a7dd0 con 0x7fc7e8156700 2023-10-08T00:16:51.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.200+0000 7fc7edf96700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e80a80b0 con 0x7fc7e8067d10 2023-10-08T00:16:51.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.201+0000 7fc7ecf94700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7e8067870 0x7fc7e8154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:51.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.201+0000 7fc7ecf94700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7e8067870 0x7fc7e8154750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47512/0 (socket says 172.21.15.99:47512) 2023-10-08T00:16:51.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.201+0000 7fc7ecf94700 1 -- 172.21.15.99:0/4019136829 learned_addr learned my addr 172.21.15.99:0/4019136829 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:51.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.201+0000 7fc7e7fff700 1 --2- 172.21.15.99:0/4019136829 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7e8067d10 0x7fc7e80a71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:51.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.201+0000 7fc7ed795700 1 --2- 172.21.15.99:0/4019136829 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7e8156700 0x7fc7e8157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:51.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.201+0000 7fc7ecf94700 1 -- 172.21.15.99:0/4019136829 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7e8067d10 msgr2=0x7fc7e80a71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:51.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.201+0000 7fc7ecf94700 1 --2- 172.21.15.99:0/4019136829 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7e8067d10 0x7fc7e80a71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:51.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.201+0000 7fc7ecf94700 1 -- 172.21.15.99:0/4019136829 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7e8156700 msgr2=0x7fc7e8157190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:51.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.201+0000 7fc7ecf94700 1 --2- 172.21.15.99:0/4019136829 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7e8156700 0x7fc7e8157190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:51.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.201+0000 7fc7ecf94700 1 -- 172.21.15.99:0/4019136829 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7dc009420 con 0x7fc7e8067870 2023-10-08T00:16:51.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.201+0000 7fc7e7fff700 1 --2- 172.21.15.99:0/4019136829 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7e8067d10 0x7fc7e80a71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:51.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.201+0000 7fc7ecf94700 1 --2- 172.21.15.99:0/4019136829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7e8067870 0x7fc7e8154750 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc00ec80 tx=0x7fc7dc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:51.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.201+0000 7fc7e4ff9700 1 -- 172.21.15.99:0/4019136829 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7dc00c9d0 con 0x7fc7e8067870 2023-10-08T00:16:51.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.201+0000 7fc7e4ff9700 1 -- 172.21.15.99:0/4019136829 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7dc00cb30 con 0x7fc7e8067870 2023-10-08T00:16:51.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.201+0000 7fc7edf96700 1 -- 172.21.15.99:0/4019136829 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7e000bc30 con 0x7fc7e8067870 2023-10-08T00:16:51.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.201+0000 7fc7e4ff9700 1 -- 172.21.15.99:0/4019136829 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7dc00ced0 con 0x7fc7e8067870 2023-10-08T00:16:51.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.201+0000 7fc7edf96700 1 -- 172.21.15.99:0/4019136829 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7e8166180 con 0x7fc7e8067870 2023-10-08T00:16:51.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.202+0000 7fc7edf96700 1 -- 172.21.15.99:0/4019136829 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7e81664f0 con 0x7fc7e8067870 2023-10-08T00:16:51.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.203+0000 7fc7e4ff9700 1 -- 172.21.15.99:0/4019136829 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc7dc013680 con 0x7fc7e8067870 2023-10-08T00:16:51.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.203+0000 7fc7e4ff9700 1 --2- 172.21.15.99:0/4019136829 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc7cc066f30 0x7fc7cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:51.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.203+0000 7fc7e4ff9700 1 -- 172.21.15.99:0/4019136829 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc7dc0848a0 con 0x7fc7e8067870 2023-10-08T00:16:51.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.203+0000 7fc7e7fff700 1 --2- 172.21.15.99:0/4019136829 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc7cc066f30 0x7fc7cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:51.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.203+0000 7fc7e7fff700 1 --2- 172.21.15.99:0/4019136829 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc7cc066f30 0x7fc7cc0693e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc7e80cacf0 tx=0x7fc7d4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:51.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.207+0000 7fc7e4ff9700 1 -- 172.21.15.99:0/4019136829 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc7dc051b10 con 0x7fc7e8067870 2023-10-08T00:16:51.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.389+0000 7fc7edf96700 1 -- 172.21.15.99:0/4019136829 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc7e806eb70 con 0x7fc7e8067870 2023-10-08T00:16:51.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.390+0000 7fc7e4ff9700 1 -- 172.21.15.99:0/4019136829 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+15378 (secure 0 0 0) 0x7fc7dc055130 con 0x7fc7e8067870 2023-10-08T00:16:51.391 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:16:51.391 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":138,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:16:50.092194+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:16:48.430607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-08T00:16:49.063424+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:16:51.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.392+0000 7fc7edf96700 1 -- 172.21.15.99:0/4019136829 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc7cc066f30 msgr2=0x7fc7cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:51.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.392+0000 7fc7edf96700 1 --2- 172.21.15.99:0/4019136829 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc7cc066f30 0x7fc7cc0693e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc7e80cacf0 tx=0x7fc7d4009040 comp rx=0 tx=0).stop 2023-10-08T00:16:51.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.392+0000 7fc7edf96700 1 -- 172.21.15.99:0/4019136829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7e8067870 msgr2=0x7fc7e8154750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:51.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.392+0000 7fc7edf96700 1 --2- 172.21.15.99:0/4019136829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7e8067870 0x7fc7e8154750 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc00ec80 tx=0x7fc7dc0091a0 comp rx=0 tx=0).stop 2023-10-08T00:16:51.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.392+0000 7fc7edf96700 1 -- 172.21.15.99:0/4019136829 shutdown_connections 2023-10-08T00:16:51.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.392+0000 7fc7edf96700 1 --2- 172.21.15.99:0/4019136829 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc7cc066f30 0x7fc7cc0693e0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:51.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.392+0000 7fc7edf96700 1 --2- 172.21.15.99:0/4019136829 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7e8156700 0x7fc7e8157190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:51.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.392+0000 7fc7edf96700 1 --2- 172.21.15.99:0/4019136829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7e8067870 0x7fc7e8154750 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:51.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.392+0000 7fc7edf96700 1 --2- 172.21.15.99:0/4019136829 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7e8067d10 0x7fc7e80a71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:51.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.392+0000 7fc7edf96700 1 -- 172.21.15.99:0/4019136829 >> 172.21.15.99:0/4019136829 conn(0x7fc7e80cd8f0 msgr2=0x7fc7e80cb5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:51.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.392+0000 7fc7edf96700 1 -- 172.21.15.99:0/4019136829 shutdown_connections 2023-10-08T00:16:51.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.392+0000 7fc7edf96700 1 -- 172.21.15.99:0/4019136829 wait complete. 2023-10-08T00:16:51.405 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T00:16:51.522 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.521+0000 7f7e4cbab700 1 Processor -- start 2023-10-08T00:16:51.522 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.521+0000 7f7e4cbab700 1 -- start start 2023-10-08T00:16:51.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.522+0000 7f7e4cbab700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7e48063ac0 0x7f7e480c7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:51.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.522+0000 7f7e4cbab700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7e48063e90 0x7f7e480cbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:51.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.522+0000 7f7e4cbab700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e480aaaa0 con 0x7f7e480cc270 2023-10-08T00:16:51.526 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.522+0000 7f7e4cbab700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e480a9eb0 con 0x7f7e48063ac0 2023-10-08T00:16:51.526 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.522+0000 7f7e4cbab700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e480aa170 con 0x7f7e48063e90 2023-10-08T00:16:51.526 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.522+0000 7f7e477fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7e48063ac0 0x7f7e480c7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:51.526 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.522+0000 7f7e47fff700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f7e480cc270 0x7f7e48155f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:42982/0 (socket says 172.21.15.99:42982) 2023-10-08T00:16:51.526 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.522+0000 7f7e47fff700 1 -- 172.21.15.99:0/1409467862 learned_addr learned my addr 172.21.15.99:0/1409467862 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:51.526 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.523+0000 7f7e46ffd700 1 --2- 172.21.15.99:0/1409467862 >> v2:172.21.15.191:3300/0 conn(0x7f7e48063e90 0x7f7e480cbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:51.526 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.523+0000 7f7e477fe700 1 -- 172.21.15.99:0/1409467862 >> v2:172.21.15.191:3300/0 conn(0x7f7e48063e90 msgr2=0x7f7e480cbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:51.526 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.523+0000 7f7e477fe700 1 --2- 172.21.15.99:0/1409467862 >> v2:172.21.15.191:3300/0 conn(0x7f7e48063e90 0x7f7e480cbd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:51.526 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.523+0000 7f7e477fe700 1 -- 172.21.15.99:0/1409467862 >> v1:172.21.15.99:6789/0 conn(0x7f7e480cc270 legacy=0x7f7e48155f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:51.527 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.523+0000 7f7e477fe700 1 -- 172.21.15.99:0/1409467862 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e480aa450 con 0x7f7e48063ac0 2023-10-08T00:16:51.527 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.523+0000 7f7e467fc700 1 -- 172.21.15.99:0/1409467862 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 34162166 0 0) 0x7f7e480aaaa0 con 0x7f7e480cc270 2023-10-08T00:16:51.527 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.523+0000 7f7e477fe700 1 --2- 172.21.15.99:0/1409467862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e48063ac0 0x7f7e480c7690 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f7e480686e0 tx=0x7f7e38009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b7098040c586824 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:51.527 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.523+0000 7f7e467fc700 1 -- 172.21.15.99:0/1409467862 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3800e070 con 0x7f7e48063ac0 2023-10-08T00:16:51.527 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.523+0000 7f7e467fc700 1 -- 172.21.15.99:0/1409467862 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e3800bb60 con 0x7f7e48063ac0 2023-10-08T00:16:51.527 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.523+0000 7f7e467fc700 1 -- 172.21.15.99:0/1409467862 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e38012430 con 0x7f7e48063ac0 2023-10-08T00:16:51.527 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.523+0000 7f7e4cbab700 1 -- 172.21.15.99:0/1409467862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e48063ac0 msgr2=0x7f7e480c7690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:51.527 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.523+0000 7f7e4cbab700 1 --2- 172.21.15.99:0/1409467862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e48063ac0 0x7f7e480c7690 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f7e480686e0 tx=0x7f7e38009780 comp rx=0 tx=0).stop 2023-10-08T00:16:51.527 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.524+0000 7f7e4cbab700 1 -- 172.21.15.99:0/1409467862 shutdown_connections 2023-10-08T00:16:51.527 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.524+0000 7f7e4cbab700 1 --2- 172.21.15.99:0/1409467862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e48063ac0 0x7f7e480c7690 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:51.527 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.524+0000 7f7e4cbab700 1 --2- 172.21.15.99:0/1409467862 >> v2:172.21.15.191:3300/0 conn(0x7f7e48063e90 0x7f7e480cbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:51.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.524+0000 7f7e4cbab700 1 -- 172.21.15.99:0/1409467862 >> 172.21.15.99:0/1409467862 conn(0x7f7e480cdbc0 msgr2=0x7f7e480c9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:51.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.524+0000 7f7e4cbab700 1 -- 172.21.15.99:0/1409467862 shutdown_connections 2023-10-08T00:16:51.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.524+0000 7f7e4cbab700 1 -- 172.21.15.99:0/1409467862 wait complete. 2023-10-08T00:16:51.530 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.525+0000 7f7e4cbab700 1 Processor -- start 2023-10-08T00:16:51.530 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.525+0000 7f7e4cbab700 1 -- start start 2023-10-08T00:16:51.530 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.525+0000 7f7e4cbab700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e48063ac0 0x7f7e4809a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:51.530 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.525+0000 7f7e4cbab700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e48063e90 0x7f7e480993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:51.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.526+0000 7f7e4cbab700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e480cc270 0x7f7e480ab890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:51.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.526+0000 7f7e4cbab700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e480ab4a0 con 0x7f7e480cc270 2023-10-08T00:16:51.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.526+0000 7f7e4cbab700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e4809a8c0 con 0x7f7e48063e90 2023-10-08T00:16:51.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.526+0000 7f7e4cbab700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e481679f0 con 0x7f7e48063ac0 2023-10-08T00:16:51.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.526+0000 7f7e47fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e480cc270 0x7f7e480ab890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:51.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.526+0000 7f7e47fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e480cc270 0x7f7e480ab890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47536/0 (socket says 172.21.15.99:47536) 2023-10-08T00:16:51.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.526+0000 7f7e47fff700 1 -- 172.21.15.99:0/426354869 learned_addr learned my addr 172.21.15.99:0/426354869 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:51.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.526+0000 7f7e477fe700 1 --2- 172.21.15.99:0/426354869 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e48063ac0 0x7f7e4809a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:51.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.526+0000 7f7e46ffd700 1 --2- 172.21.15.99:0/426354869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e48063e90 0x7f7e480993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:51.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.526+0000 7f7e47fff700 1 -- 172.21.15.99:0/426354869 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e48063ac0 msgr2=0x7f7e4809a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:51.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.526+0000 7f7e47fff700 1 --2- 172.21.15.99:0/426354869 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e48063ac0 0x7f7e4809a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:51.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.526+0000 7f7e47fff700 1 -- 172.21.15.99:0/426354869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e48063e90 msgr2=0x7f7e480993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:51.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.526+0000 7f7e47fff700 1 --2- 172.21.15.99:0/426354869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e48063e90 0x7f7e480993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:51.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.526+0000 7f7e47fff700 1 -- 172.21.15.99:0/426354869 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e38009a20 con 0x7f7e480cc270 2023-10-08T00:16:51.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.526+0000 7f7e47fff700 1 --2- 172.21.15.99:0/426354869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e480cc270 0x7f7e480ab890 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f7e4000c7e0 tx=0x7f7e40007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:51.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.526+0000 7f7e2f7fe700 1 -- 172.21.15.99:0/426354869 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e40005c00 con 0x7f7e480cc270 2023-10-08T00:16:51.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.526+0000 7f7e2f7fe700 1 -- 172.21.15.99:0/426354869 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e40005d80 con 0x7f7e480cc270 2023-10-08T00:16:51.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.526+0000 7f7e2f7fe700 1 -- 172.21.15.99:0/426354869 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e40007320 con 0x7f7e480cc270 2023-10-08T00:16:51.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.526+0000 7f7e4cbab700 1 -- 172.21.15.99:0/426354869 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e48167bf0 con 0x7f7e480cc270 2023-10-08T00:16:51.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.526+0000 7f7e4cbab700 1 -- 172.21.15.99:0/426354869 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e48167ed0 con 0x7f7e480cc270 2023-10-08T00:16:51.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.527+0000 7f7e4cbab700 1 -- 172.21.15.99:0/426354869 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e480ada30 con 0x7f7e480cc270 2023-10-08T00:16:51.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.528+0000 7f7e2f7fe700 1 -- 172.21.15.99:0/426354869 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7e4001b3b0 con 0x7f7e480cc270 2023-10-08T00:16:51.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.528+0000 7f7e2f7fe700 1 --2- 172.21.15.99:0/426354869 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7e34066f30 0x7f7e340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:51.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.528+0000 7f7e2f7fe700 1 -- 172.21.15.99:0/426354869 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7e40083970 con 0x7f7e480cc270 2023-10-08T00:16:51.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.528+0000 7f7e477fe700 1 --2- 172.21.15.99:0/426354869 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7e34066f30 0x7f7e340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:51.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.528+0000 7f7e477fe700 1 --2- 172.21.15.99:0/426354869 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7e34066f30 0x7f7e340693e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7e380099f0 tx=0x7f7e38019000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:51.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.532+0000 7f7e2f7fe700 1 -- 172.21.15.99:0/426354869 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7e40050be0 con 0x7f7e480cc270 2023-10-08T00:16:51.761 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:51.761+0000 7f7e4cbab700 1 -- 172.21.15.99:0/426354869 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f7e480c87f0 con 0x7f7e480cc270 2023-10-08T00:16:52.740 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.739+0000 7f7e2f7fe700 1 -- 172.21.15.99:0/426354869 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v62) v1 ==== 118+0+0 (secure 0 0 0) 0x7f7e40017020 con 0x7f7e480cc270 2023-10-08T00:16:52.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.742+0000 7f7e4cbab700 1 -- 172.21.15.99:0/426354869 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7e34066f30 msgr2=0x7f7e340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:52.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.742+0000 7f7e4cbab700 1 --2- 172.21.15.99:0/426354869 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7e34066f30 0x7f7e340693e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7e380099f0 tx=0x7f7e38019000 comp rx=0 tx=0).stop 2023-10-08T00:16:52.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.742+0000 7f7e4cbab700 1 -- 172.21.15.99:0/426354869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e480cc270 msgr2=0x7f7e480ab890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:52.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.742+0000 7f7e4cbab700 1 --2- 172.21.15.99:0/426354869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e480cc270 0x7f7e480ab890 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f7e4000c7e0 tx=0x7f7e40007d90 comp rx=0 tx=0).stop 2023-10-08T00:16:52.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.742+0000 7f7e4cbab700 1 -- 172.21.15.99:0/426354869 shutdown_connections 2023-10-08T00:16:52.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.742+0000 7f7e4cbab700 1 --2- 172.21.15.99:0/426354869 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7e34066f30 0x7f7e340693e0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:52.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.742+0000 7f7e4cbab700 1 --2- 172.21.15.99:0/426354869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e480cc270 0x7f7e480ab890 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:52.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.742+0000 7f7e4cbab700 1 --2- 172.21.15.99:0/426354869 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e48063ac0 0x7f7e4809a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:52.744 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.742+0000 7f7e4cbab700 1 --2- 172.21.15.99:0/426354869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e48063e90 0x7f7e480993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:52.744 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.742+0000 7f7e4cbab700 1 -- 172.21.15.99:0/426354869 >> 172.21.15.99:0/426354869 conn(0x7f7e480cdbc0 msgr2=0x7f7e4806a880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:52.744 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.742+0000 7f7e4cbab700 1 -- 172.21.15.99:0/426354869 shutdown_connections 2023-10-08T00:16:52.744 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.742+0000 7f7e4cbab700 1 -- 172.21.15.99:0/426354869 wait complete. 2023-10-08T00:16:52.757 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T00:16:52.875 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.874+0000 7f503f498700 1 Processor -- start 2023-10-08T00:16:52.875 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.875+0000 7f503f498700 1 -- start start 2023-10-08T00:16:52.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.875+0000 7f503f498700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5038159ec0 0x7f50380af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:52.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.875+0000 7f503f498700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f503815a290 0x7f50380afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:52.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.875+0000 7f503f498700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50380b0790 con 0x7f503806a5a0 2023-10-08T00:16:52.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.875+0000 7f503f498700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50380ce1f0 con 0x7f503815a290 2023-10-08T00:16:52.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.875+0000 7f503f498700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50380b0b20 con 0x7f5038159ec0 2023-10-08T00:16:52.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.876+0000 7f503ca33700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f503815a290 0x7f50380afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:52.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.876+0000 7f503ca33700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f503815a290 0x7f50380afe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.99:48890/0 (socket says 172.21.15.99:48890) 2023-10-08T00:16:52.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.876+0000 7f503ca33700 1 -- 172.21.15.99:0/2621768921 learned_addr learned my addr 172.21.15.99:0/2621768921 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:52.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.876+0000 7f503da35700 1 --1- 172.21.15.99:0/2621768921 >> v1:172.21.15.191:6789/0 conn(0x7f503806a5a0 0x7f503815db90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:51802/0 (socket says 172.21.15.99:51802) 2023-10-08T00:16:52.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.876+0000 7f503d234700 1 --2- 172.21.15.99:0/2621768921 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5038159ec0 0x7f50380af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:52.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.876+0000 7f503ca33700 1 -- 172.21.15.99:0/2621768921 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5038159ec0 msgr2=0x7f50380af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:52.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.876+0000 7f503ca33700 1 --2- 172.21.15.99:0/2621768921 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5038159ec0 0x7f50380af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:52.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.876+0000 7f503ca33700 1 -- 172.21.15.99:0/2621768921 >> v1:172.21.15.191:6789/0 conn(0x7f503806a5a0 legacy=0x7f503815db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:52.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.876+0000 7f503ca33700 1 -- 172.21.15.99:0/2621768921 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50380b04b0 con 0x7f503815a290 2023-10-08T00:16:52.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.876+0000 7f503ca33700 1 --2- 172.21.15.99:0/2621768921 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f503815a290 0x7f50380afe80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f50280099b0 tx=0x7f5028009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3d4587fe4ae7ec6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:52.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.876+0000 7f502ffff700 1 -- 172.21.15.99:0/2621768921 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5028009ea0 con 0x7f503815a290 2023-10-08T00:16:52.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.876+0000 7f502ffff700 1 -- 172.21.15.99:0/2621768921 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f502800b860 con 0x7f503815a290 2023-10-08T00:16:52.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.876+0000 7f502ffff700 1 -- 172.21.15.99:0/2621768921 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502800bc00 con 0x7f503815a290 2023-10-08T00:16:52.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.877+0000 7f503f498700 1 -- 172.21.15.99:0/2621768921 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f503815a290 msgr2=0x7f50380afe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:52.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.877+0000 7f503f498700 1 --2- 172.21.15.99:0/2621768921 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f503815a290 0x7f50380afe80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f50280099b0 tx=0x7f5028009250 comp rx=0 tx=0).stop 2023-10-08T00:16:52.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.877+0000 7f503f498700 1 -- 172.21.15.99:0/2621768921 shutdown_connections 2023-10-08T00:16:52.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.877+0000 7f503f498700 1 --2- 172.21.15.99:0/2621768921 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5038159ec0 0x7f50380af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:52.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.877+0000 7f503f498700 1 --2- 172.21.15.99:0/2621768921 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f503815a290 0x7f50380afe80 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:52.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.877+0000 7f503f498700 1 -- 172.21.15.99:0/2621768921 >> 172.21.15.99:0/2621768921 conn(0x7f50380d1880 msgr2=0x7f50380cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:52.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.877+0000 7f503f498700 1 -- 172.21.15.99:0/2621768921 shutdown_connections 2023-10-08T00:16:52.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.877+0000 7f503f498700 1 -- 172.21.15.99:0/2621768921 wait complete. 2023-10-08T00:16:52.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.878+0000 7f503f498700 1 Processor -- start 2023-10-08T00:16:52.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.878+0000 7f503f498700 1 -- start start 2023-10-08T00:16:52.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.879+0000 7f503f498700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f503806a5a0 0x7f503815dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:52.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.879+0000 7f503f498700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5038159ec0 0x7f5038156fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:52.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.879+0000 7f503f498700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f503815a760 0x7f50380b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:52.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.879+0000 7f503f498700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50380b48a0 con 0x7f503815a760 2023-10-08T00:16:52.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.879+0000 7f503f498700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50380b4b80 con 0x7f5038159ec0 2023-10-08T00:16:52.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.879+0000 7f503f498700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50380b4e60 con 0x7f503806a5a0 2023-10-08T00:16:52.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.879+0000 7f503da35700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f503815a760 0x7f50380b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:52.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.879+0000 7f503da35700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f503815a760 0x7f50380b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47546/0 (socket says 172.21.15.99:47546) 2023-10-08T00:16:52.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.879+0000 7f503da35700 1 -- 172.21.15.99:0/3687393487 learned_addr learned my addr 172.21.15.99:0/3687393487 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:52.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.879+0000 7f503d234700 1 --2- 172.21.15.99:0/3687393487 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f503806a5a0 0x7f503815dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:52.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.879+0000 7f503ca33700 1 --2- 172.21.15.99:0/3687393487 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5038159ec0 0x7f5038156fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:52.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.880+0000 7f503da35700 1 -- 172.21.15.99:0/3687393487 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f503806a5a0 msgr2=0x7f503815dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:52.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.880+0000 7f503da35700 1 --2- 172.21.15.99:0/3687393487 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f503806a5a0 0x7f503815dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:52.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.880+0000 7f503da35700 1 -- 172.21.15.99:0/3687393487 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5038159ec0 msgr2=0x7f5038156fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:52.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.880+0000 7f503da35700 1 --2- 172.21.15.99:0/3687393487 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5038159ec0 0x7f5038156fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:52.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.880+0000 7f503da35700 1 -- 172.21.15.99:0/3687393487 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50280094f0 con 0x7f503815a760 2023-10-08T00:16:52.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.880+0000 7f503d234700 1 --2- 172.21.15.99:0/3687393487 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f503806a5a0 0x7f503815dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:52.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.880+0000 7f503da35700 1 --2- 172.21.15.99:0/3687393487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f503815a760 0x7f50380b3f30 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f503000c590 tx=0x7f5030007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:52.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.880+0000 7f502d7fa700 1 -- 172.21.15.99:0/3687393487 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5030005a70 con 0x7f503815a760 2023-10-08T00:16:52.884 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.880+0000 7f502d7fa700 1 -- 172.21.15.99:0/3687393487 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5030005bf0 con 0x7f503815a760 2023-10-08T00:16:52.884 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.880+0000 7f503f498700 1 -- 172.21.15.99:0/3687393487 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50380b5690 con 0x7f503815a760 2023-10-08T00:16:52.884 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.880+0000 7f502d7fa700 1 -- 172.21.15.99:0/3687393487 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50300070d0 con 0x7f503815a760 2023-10-08T00:16:52.884 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.880+0000 7f503f498700 1 -- 172.21.15.99:0/3687393487 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50380b4470 con 0x7f503815a760 2023-10-08T00:16:52.884 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.881+0000 7f503f498700 1 -- 172.21.15.99:0/3687393487 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50380b4660 con 0x7f503815a760 2023-10-08T00:16:52.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.881+0000 7f502d7fa700 1 -- 172.21.15.99:0/3687393487 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f50300074c0 con 0x7f503815a760 2023-10-08T00:16:52.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.882+0000 7f502d7fa700 1 --2- 172.21.15.99:0/3687393487 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f501c066f30 0x7f501c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:52.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.882+0000 7f502d7fa700 1 -- 172.21.15.99:0/3687393487 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5030083a70 con 0x7f503815a760 2023-10-08T00:16:52.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.882+0000 7f503d234700 1 --2- 172.21.15.99:0/3687393487 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f501c066f30 0x7f501c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:52.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.882+0000 7f503d234700 1 --2- 172.21.15.99:0/3687393487 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f501c066f30 0x7f501c0693e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f50340076c0 tx=0x7f503400b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:52.886 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:52.886+0000 7f502d7fa700 1 -- 172.21.15.99:0/3687393487 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f50380b4660 con 0x7f503815a760 2023-10-08T00:16:53.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.109+0000 7f503f498700 1 -- 172.21.15.99:0/3687393487 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f50380b4660 con 0x7f503815a760 2023-10-08T00:16:53.376 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:16:53.379 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:16:53.748 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.748+0000 7f502d7fa700 1 -- 172.21.15.99:0/3687393487 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v63) v1 ==== 122+0+0 (secure 0 0 0) 0x7f50380b4660 con 0x7f503815a760 2023-10-08T00:16:53.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.750+0000 7f503f498700 1 -- 172.21.15.99:0/3687393487 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f501c066f30 msgr2=0x7f501c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:53.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.750+0000 7f503f498700 1 --2- 172.21.15.99:0/3687393487 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f501c066f30 0x7f501c0693e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f50340076c0 tx=0x7f503400b040 comp rx=0 tx=0).stop 2023-10-08T00:16:53.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.750+0000 7f503f498700 1 -- 172.21.15.99:0/3687393487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f503815a760 msgr2=0x7f50380b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:53.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.750+0000 7f503f498700 1 --2- 172.21.15.99:0/3687393487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f503815a760 0x7f50380b3f30 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f503000c590 tx=0x7f5030007ef0 comp rx=0 tx=0).stop 2023-10-08T00:16:53.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.750+0000 7f503f498700 1 -- 172.21.15.99:0/3687393487 shutdown_connections 2023-10-08T00:16:53.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.750+0000 7f503f498700 1 --2- 172.21.15.99:0/3687393487 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f501c066f30 0x7f501c0693e0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:53.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.750+0000 7f503f498700 1 --2- 172.21.15.99:0/3687393487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f503815a760 0x7f50380b3f30 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:53.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.750+0000 7f503f498700 1 --2- 172.21.15.99:0/3687393487 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f503806a5a0 0x7f503815dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:53.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.750+0000 7f503f498700 1 --2- 172.21.15.99:0/3687393487 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5038159ec0 0x7f5038156fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:53.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.750+0000 7f503f498700 1 -- 172.21.15.99:0/3687393487 >> 172.21.15.99:0/3687393487 conn(0x7f50380d1880 msgr2=0x7f5038158f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:53.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.751+0000 7f503f498700 1 -- 172.21.15.99:0/3687393487 shutdown_connections 2023-10-08T00:16:53.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.751+0000 7f503f498700 1 -- 172.21.15.99:0/3687393487 wait complete. 2023-10-08T00:16:53.763 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:16:53.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.877+0000 7f380252a700 1 Processor -- start 2023-10-08T00:16:53.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.877+0000 7f380252a700 1 -- start start 2023-10-08T00:16:53.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.878+0000 7f380252a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f37fc0cd6e0 0x7f37fc0cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:53.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.878+0000 7f380252a700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f37fc15cbd0 0x7f37fc15d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:53.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.878+0000 7f380252a700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37fc0cb7c0 con 0x7f37fc0cdab0 2023-10-08T00:16:53.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.878+0000 7f380252a700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37fc15e850 con 0x7f37fc15cbd0 2023-10-08T00:16:53.880 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.878+0000 7f380252a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37fc0cd030 con 0x7f37fc0cd6e0 2023-10-08T00:16:53.880 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.878+0000 7f3801d29700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f37fc15cbd0 0x7f37fc15d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:53.880 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.878+0000 7f3800d27700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f37fc0cdab0 0x7f37fc15d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:46836/0 (socket says 172.21.15.99:46836) 2023-10-08T00:16:53.880 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.878+0000 7f3800d27700 1 -- 172.21.15.99:0/3268831275 learned_addr learned my addr 172.21.15.99:0/3268831275 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:53.880 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.878+0000 7f3801528700 1 --2- 172.21.15.99:0/3268831275 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f37fc0cd6e0 0x7f37fc0cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:53.880 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.878+0000 7f3801d29700 1 -- 172.21.15.99:0/3268831275 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f37fc0cd6e0 msgr2=0x7f37fc0cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:53.880 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.878+0000 7f3801d29700 1 --2- 172.21.15.99:0/3268831275 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f37fc0cd6e0 0x7f37fc0cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:53.880 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.878+0000 7f3801d29700 1 -- 172.21.15.99:0/3268831275 >> v1:172.21.15.99:6789/0 conn(0x7f37fc0cdab0 legacy=0x7f37fc15d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:53.880 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.878+0000 7f3801d29700 1 -- 172.21.15.99:0/3268831275 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37fc15e290 con 0x7f37fc15cbd0 2023-10-08T00:16:53.880 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.878+0000 7f37f3fff700 1 -- 172.21.15.99:0/3268831275 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2339608181 0 0) 0x7f37fc0cb7c0 con 0x7f37fc0cdab0 2023-10-08T00:16:53.880 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.879+0000 7f3801d29700 1 --2- 172.21.15.99:0/3268831275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37fc15cbd0 0x7f37fc15d020 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f37f40098e0 tx=0x7f37f40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31150e3cf6600a47 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:53.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.879+0000 7f37f3fff700 1 -- 172.21.15.99:0/3268831275 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f400b640 con 0x7f37fc15cbd0 2023-10-08T00:16:53.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.879+0000 7f37f3fff700 1 -- 172.21.15.99:0/3268831275 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f400b7a0 con 0x7f37fc15cbd0 2023-10-08T00:16:53.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.879+0000 7f37f3fff700 1 -- 172.21.15.99:0/3268831275 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f400bb40 con 0x7f37fc15cbd0 2023-10-08T00:16:53.884 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.879+0000 7f380252a700 1 -- 172.21.15.99:0/3268831275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37fc15cbd0 msgr2=0x7f37fc15d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:53.884 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.879+0000 7f380252a700 1 --2- 172.21.15.99:0/3268831275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37fc15cbd0 0x7f37fc15d020 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f37f40098e0 tx=0x7f37f40091a0 comp rx=0 tx=0).stop 2023-10-08T00:16:53.884 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.879+0000 7f380252a700 1 -- 172.21.15.99:0/3268831275 shutdown_connections 2023-10-08T00:16:53.884 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.879+0000 7f380252a700 1 --2- 172.21.15.99:0/3268831275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37fc15cbd0 0x7f37fc15d020 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:53.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.879+0000 7f380252a700 1 --2- 172.21.15.99:0/3268831275 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f37fc0cd6e0 0x7f37fc0cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:53.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.879+0000 7f380252a700 1 -- 172.21.15.99:0/3268831275 >> 172.21.15.99:0/3268831275 conn(0x7f37fc067660 msgr2=0x7f37fc067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:53.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.879+0000 7f380252a700 1 -- 172.21.15.99:0/3268831275 shutdown_connections 2023-10-08T00:16:53.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.879+0000 7f380252a700 1 -- 172.21.15.99:0/3268831275 wait complete. 2023-10-08T00:16:53.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.880+0000 7f380252a700 1 Processor -- start 2023-10-08T00:16:53.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.880+0000 7f380252a700 1 -- start start 2023-10-08T00:16:53.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.881+0000 7f380252a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37fc0cd6e0 0x7f37fc158f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:53.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.881+0000 7f380252a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f37fc0cdab0 0x7f37fc157780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:53.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.881+0000 7f380252a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f37fc15cbd0 0x7f37fc1438f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:53.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.881+0000 7f380252a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37fc15a820 con 0x7f37fc0cd6e0 2023-10-08T00:16:53.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.881+0000 7f380252a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37fc15b640 con 0x7f37fc0cdab0 2023-10-08T00:16:53.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.881+0000 7f380252a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37fc143e30 con 0x7f37fc15cbd0 2023-10-08T00:16:53.886 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.881+0000 7f3801528700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37fc0cd6e0 0x7f37fc158f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:53.886 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.881+0000 7f3801528700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37fc0cd6e0 0x7f37fc158f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55914/0 (socket says 172.21.15.99:55914) 2023-10-08T00:16:53.886 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.881+0000 7f3801528700 1 -- 172.21.15.99:0/2606925608 learned_addr learned my addr 172.21.15.99:0/2606925608 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:53.886 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.882+0000 7f3801d29700 1 --2- 172.21.15.99:0/2606925608 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f37fc15cbd0 0x7f37fc1438f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:53.886 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.882+0000 7f3801528700 1 -- 172.21.15.99:0/2606925608 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f37fc15cbd0 msgr2=0x7f37fc1438f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:53.886 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.882+0000 7f3800d27700 1 --2- 172.21.15.99:0/2606925608 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f37fc0cdab0 0x7f37fc157780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:53.886 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.882+0000 7f3801528700 1 --2- 172.21.15.99:0/2606925608 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f37fc15cbd0 0x7f37fc1438f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:53.886 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.882+0000 7f3801528700 1 -- 172.21.15.99:0/2606925608 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f37fc0cdab0 msgr2=0x7f37fc157780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:53.886 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.882+0000 7f3801528700 1 --2- 172.21.15.99:0/2606925608 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f37fc0cdab0 0x7f37fc157780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:53.886 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.882+0000 7f3801528700 1 -- 172.21.15.99:0/2606925608 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f4009420 con 0x7f37fc0cd6e0 2023-10-08T00:16:53.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.882+0000 7f3801d29700 1 --2- 172.21.15.99:0/2606925608 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f37fc15cbd0 0x7f37fc1438f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:16:53.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.882+0000 7f3800d27700 1 --2- 172.21.15.99:0/2606925608 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f37fc0cdab0 0x7f37fc157780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:16:53.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.882+0000 7f3801528700 1 --2- 172.21.15.99:0/2606925608 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37fc0cd6e0 0x7f37fc158f10 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f37f8009e80 tx=0x7f37f800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:53.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.882+0000 7f37f17fa700 1 -- 172.21.15.99:0/2606925608 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f800cc20 con 0x7f37fc0cd6e0 2023-10-08T00:16:53.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.882+0000 7f37f17fa700 1 -- 172.21.15.99:0/2606925608 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f800f040 con 0x7f37fc0cd6e0 2023-10-08T00:16:53.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.882+0000 7f380252a700 1 -- 172.21.15.99:0/2606925608 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37fc15aa30 con 0x7f37fc0cd6e0 2023-10-08T00:16:53.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.882+0000 7f37f17fa700 1 -- 172.21.15.99:0/2606925608 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f8014480 con 0x7f37fc0cd6e0 2023-10-08T00:16:53.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.882+0000 7f380252a700 1 -- 172.21.15.99:0/2606925608 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37fc15aff0 con 0x7f37fc0cd6e0 2023-10-08T00:16:53.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.883+0000 7f380252a700 1 -- 172.21.15.99:0/2606925608 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37fc15b920 con 0x7f37fc0cd6e0 2023-10-08T00:16:53.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.883+0000 7f37f17fa700 1 -- 172.21.15.99:0/2606925608 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f37f80147f0 con 0x7f37fc0cd6e0 2023-10-08T00:16:53.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.884+0000 7f37f17fa700 1 --2- 172.21.15.99:0/2606925608 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f37e0066f30 0x7f37e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:53.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.884+0000 7f37f17fa700 1 -- 172.21.15.99:0/2606925608 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f37f8085170 con 0x7f37fc0cd6e0 2023-10-08T00:16:53.888 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.884+0000 7f3800d27700 1 --2- 172.21.15.99:0/2606925608 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f37e0066f30 0x7f37e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:53.888 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.884+0000 7f3800d27700 1 --2- 172.21.15.99:0/2606925608 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f37e0066f30 0x7f37e00693e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f37fc157160 tx=0x7f37ec009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:53.888 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:53.888+0000 7f37f17fa700 1 -- 172.21.15.99:0/2606925608 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f37f80523e0 con 0x7f37fc0cd6e0 2023-10-08T00:16:54.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.111+0000 7f380252a700 1 -- 172.21.15.99:0/2606925608 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f37fc04bb80 con 0x7f37fc0cd6e0 2023-10-08T00:16:54.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.111+0000 7f37f17fa700 1 -- 172.21.15.99:0/2606925608 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f37f8055a00 con 0x7f37fc0cd6e0 2023-10-08T00:16:54.112 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:16:54.113 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":63,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5813,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/887374839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":887374839},{"type":"v1","addr":"172.21.15.191:6837","nonce":887374839}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":6445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/2409696901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2409696901},{"type":"v1","addr":"172.21.15.191:6836","nonce":2409696901}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":7095,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/1924096228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1924096228},{"type":"v1","addr":"172.21.15.99:6834","nonce":1924096228}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:16:50.092494+0000","modified":"2023-10-08T00:16:53.744105+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7098":{"gid":7098,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/2522933027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":2522933027},{"type":"v1","addr":"172.21.15.99:6835","nonce":2522933027}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-08T00:16:54.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.113+0000 7f380252a700 1 -- 172.21.15.99:0/2606925608 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f37e0066f30 msgr2=0x7f37e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:54.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.113+0000 7f380252a700 1 --2- 172.21.15.99:0/2606925608 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f37e0066f30 0x7f37e00693e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f37fc157160 tx=0x7f37ec009040 comp rx=0 tx=0).stop 2023-10-08T00:16:54.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.113+0000 7f380252a700 1 -- 172.21.15.99:0/2606925608 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37fc0cd6e0 msgr2=0x7f37fc158f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:54.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.113+0000 7f380252a700 1 --2- 172.21.15.99:0/2606925608 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37fc0cd6e0 0x7f37fc158f10 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f37f8009e80 tx=0x7f37f800b040 comp rx=0 tx=0).stop 2023-10-08T00:16:54.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.113+0000 7f380252a700 1 -- 172.21.15.99:0/2606925608 shutdown_connections 2023-10-08T00:16:54.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.113+0000 7f380252a700 1 --2- 172.21.15.99:0/2606925608 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f37e0066f30 0x7f37e00693e0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:54.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.113+0000 7f380252a700 1 --2- 172.21.15.99:0/2606925608 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f37fc15cbd0 0x7f37fc1438f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:54.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.113+0000 7f380252a700 1 --2- 172.21.15.99:0/2606925608 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37fc0cd6e0 0x7f37fc158f10 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:54.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.113+0000 7f380252a700 1 --2- 172.21.15.99:0/2606925608 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f37fc0cdab0 0x7f37fc157780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:54.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.113+0000 7f380252a700 1 -- 172.21.15.99:0/2606925608 >> 172.21.15.99:0/2606925608 conn(0x7f37fc067660 msgr2=0x7f37fc1565b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:54.116 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.114+0000 7f380252a700 1 -- 172.21.15.99:0/2606925608 shutdown_connections 2023-10-08T00:16:54.116 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.114+0000 7f380252a700 1 -- 172.21.15.99:0/2606925608 wait complete. 2023-10-08T00:16:54.116 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 63 2023-10-08T00:16:54.126 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:16:54.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.240+0000 7f7e99d54700 1 Processor -- start 2023-10-08T00:16:54.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.240+0000 7f7e99d54700 1 -- start start 2023-10-08T00:16:54.243 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.241+0000 7f7e99d54700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7e94157870 0x7f7e9415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:54.243 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.241+0000 7f7e99d54700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e9415a620 0x7f7e9415d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:54.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.241+0000 7f7e99d54700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e9415cbb0 con 0x7f7e9415d580 2023-10-08T00:16:54.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.241+0000 7f7e99d54700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e9415bfa0 con 0x7f7e94157870 2023-10-08T00:16:54.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.241+0000 7f7e99d54700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9415c280 con 0x7f7e9415a620 2023-10-08T00:16:54.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.241+0000 7f7e937fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7e94157870 0x7f7e9415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:54.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.241+0000 7f7e93fff700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f7e9415d580 0x7f7e94158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:46838/0 (socket says 172.21.15.99:46838) 2023-10-08T00:16:54.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.241+0000 7f7e93fff700 1 -- 172.21.15.99:0/1500221732 learned_addr learned my addr 172.21.15.99:0/1500221732 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:54.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.241+0000 7f7e92ffd700 1 --2- 172.21.15.99:0/1500221732 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e9415a620 0x7f7e9415d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:54.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.241+0000 7f7e937fe700 1 -- 172.21.15.99:0/1500221732 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e9415a620 msgr2=0x7f7e9415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:54.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.241+0000 7f7e937fe700 1 --2- 172.21.15.99:0/1500221732 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e9415a620 0x7f7e9415d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:54.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.241+0000 7f7e937fe700 1 -- 172.21.15.99:0/1500221732 >> v1:172.21.15.99:6789/0 conn(0x7f7e9415d580 legacy=0x7f7e94158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:54.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.241+0000 7f7e927fc700 1 -- 172.21.15.99:0/1500221732 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3896624044 0 0) 0x7f7e9415cbb0 con 0x7f7e9415d580 2023-10-08T00:16:54.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.241+0000 7f7e937fe700 1 -- 172.21.15.99:0/1500221732 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e9415c560 con 0x7f7e94157870 2023-10-08T00:16:54.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.241+0000 7f7e937fe700 1 --2- 172.21.15.99:0/1500221732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e94157870 0x7f7e9415b6a0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f7e840098e0 tx=0x7f7e840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=666939d473042638 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:54.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.241+0000 7f7e927fc700 1 -- 172.21.15.99:0/1500221732 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e84009e80 con 0x7f7e94157870 2023-10-08T00:16:54.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.241+0000 7f7e927fc700 1 -- 172.21.15.99:0/1500221732 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e8400b860 con 0x7f7e94157870 2023-10-08T00:16:54.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.241+0000 7f7e927fc700 1 -- 172.21.15.99:0/1500221732 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8400bbe0 con 0x7f7e94157870 2023-10-08T00:16:54.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.242+0000 7f7e99d54700 1 -- 172.21.15.99:0/1500221732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e94157870 msgr2=0x7f7e9415b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:54.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.242+0000 7f7e99d54700 1 --2- 172.21.15.99:0/1500221732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e94157870 0x7f7e9415b6a0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f7e840098e0 tx=0x7f7e840091a0 comp rx=0 tx=0).stop 2023-10-08T00:16:54.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.242+0000 7f7e99d54700 1 -- 172.21.15.99:0/1500221732 shutdown_connections 2023-10-08T00:16:54.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.242+0000 7f7e99d54700 1 --2- 172.21.15.99:0/1500221732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e94157870 0x7f7e9415b6a0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:54.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.242+0000 7f7e99d54700 1 --2- 172.21.15.99:0/1500221732 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e9415a620 0x7f7e9415d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:54.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.242+0000 7f7e99d54700 1 -- 172.21.15.99:0/1500221732 >> 172.21.15.99:0/1500221732 conn(0x7f7e94068ce0 msgr2=0x7f7e940690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:54.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.242+0000 7f7e99d54700 1 -- 172.21.15.99:0/1500221732 shutdown_connections 2023-10-08T00:16:54.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.242+0000 7f7e99d54700 1 -- 172.21.15.99:0/1500221732 wait complete. 2023-10-08T00:16:54.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.243+0000 7f7e99d54700 1 Processor -- start 2023-10-08T00:16:54.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.243+0000 7f7e99d54700 1 -- start start 2023-10-08T00:16:54.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.244+0000 7f7e99d54700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e94157870 0x7f7e940b14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:54.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.244+0000 7f7e99d54700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e9415a620 0x7f7e94151970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:54.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.244+0000 7f7e99d54700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e9415d580 0x7f7e9414c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:54.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.244+0000 7f7e99d54700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e94152d40 con 0x7f7e94157870 2023-10-08T00:16:54.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.244+0000 7f7e99d54700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e94151eb0 con 0x7f7e9415d580 2023-10-08T00:16:54.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.244+0000 7f7e99d54700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e941531d0 con 0x7f7e9415a620 2023-10-08T00:16:54.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.244+0000 7f7e937fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e94157870 0x7f7e940b14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:54.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.244+0000 7f7e937fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e94157870 0x7f7e940b14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55940/0 (socket says 172.21.15.99:55940) 2023-10-08T00:16:54.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.244+0000 7f7e937fe700 1 -- 172.21.15.99:0/1322415221 learned_addr learned my addr 172.21.15.99:0/1322415221 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:54.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.244+0000 7f7e92ffd700 1 --2- 172.21.15.99:0/1322415221 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e9415a620 0x7f7e94151970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:54.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.244+0000 7f7e93fff700 1 --2- 172.21.15.99:0/1322415221 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e9415d580 0x7f7e9414c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:54.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.244+0000 7f7e937fe700 1 -- 172.21.15.99:0/1322415221 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e9415a620 msgr2=0x7f7e94151970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:54.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.244+0000 7f7e937fe700 1 --2- 172.21.15.99:0/1322415221 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e9415a620 0x7f7e94151970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:54.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.244+0000 7f7e937fe700 1 -- 172.21.15.99:0/1322415221 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e9415d580 msgr2=0x7f7e9414c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:54.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.244+0000 7f7e937fe700 1 --2- 172.21.15.99:0/1322415221 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e9415d580 0x7f7e9414c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:54.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.244+0000 7f7e937fe700 1 -- 172.21.15.99:0/1322415221 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e84009420 con 0x7f7e94157870 2023-10-08T00:16:54.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.244+0000 7f7e92ffd700 1 --2- 172.21.15.99:0/1322415221 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e9415a620 0x7f7e94151970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:54.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.244+0000 7f7e937fe700 1 --2- 172.21.15.99:0/1322415221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e94157870 0x7f7e940b14f0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f7e8400b3c0 tx=0x7f7e84017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:54.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.245+0000 7f7e7b7fe700 1 -- 172.21.15.99:0/1322415221 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e84011420 con 0x7f7e94157870 2023-10-08T00:16:54.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.245+0000 7f7e7b7fe700 1 -- 172.21.15.99:0/1322415221 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e84011580 con 0x7f7e94157870 2023-10-08T00:16:54.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.245+0000 7f7e7b7fe700 1 -- 172.21.15.99:0/1322415221 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e84011940 con 0x7f7e94157870 2023-10-08T00:16:54.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.245+0000 7f7e99d54700 1 -- 172.21.15.99:0/1322415221 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e94152130 con 0x7f7e94157870 2023-10-08T00:16:54.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.245+0000 7f7e99d54700 1 -- 172.21.15.99:0/1322415221 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e941526f0 con 0x7f7e94157870 2023-10-08T00:16:54.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.245+0000 7f7e99d54700 1 -- 172.21.15.99:0/1322415221 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e94153020 con 0x7f7e94157870 2023-10-08T00:16:54.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.246+0000 7f7e7b7fe700 1 -- 172.21.15.99:0/1322415221 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7e840253f0 con 0x7f7e94157870 2023-10-08T00:16:54.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.246+0000 7f7e7b7fe700 1 --2- 172.21.15.99:0/1322415221 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7e80066fb0 0x7f7e80069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:54.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.246+0000 7f7e7b7fe700 1 -- 172.21.15.99:0/1322415221 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7e84086040 con 0x7f7e94157870 2023-10-08T00:16:54.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.246+0000 7f7e92ffd700 1 --2- 172.21.15.99:0/1322415221 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7e80066fb0 0x7f7e80069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:54.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.247+0000 7f7e92ffd700 1 --2- 172.21.15.99:0/1322415221 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7e80066fb0 0x7f7e80069460 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f7e940699f0 tx=0x7f7e88008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:54.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.250+0000 7f7e7b7fe700 1 -- 172.21.15.99:0/1322415221 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7e84053210 con 0x7f7e94157870 2023-10-08T00:16:54.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.436+0000 7f7e99d54700 1 -- 172.21.15.99:0/1322415221 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7e9404c3c0 con 0x7f7e94157870 2023-10-08T00:16:54.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.437+0000 7f7e7b7fe700 1 -- 172.21.15.99:0/1322415221 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) v1 ==== 74+0+15365 (secure 0 0 0) 0x7f7e8405ae60 con 0x7f7e94157870 2023-10-08T00:16:54.437 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:16:54.438 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":139,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:16:51.724484+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:16:48.430607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-08T00:16:49.063424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:16:54.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.439+0000 7f7e99d54700 1 -- 172.21.15.99:0/1322415221 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7e80066fb0 msgr2=0x7f7e80069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:54.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.439+0000 7f7e99d54700 1 --2- 172.21.15.99:0/1322415221 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7e80066fb0 0x7f7e80069460 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f7e940699f0 tx=0x7f7e88008040 comp rx=0 tx=0).stop 2023-10-08T00:16:54.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.439+0000 7f7e99d54700 1 -- 172.21.15.99:0/1322415221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e94157870 msgr2=0x7f7e940b14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:54.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.439+0000 7f7e99d54700 1 --2- 172.21.15.99:0/1322415221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e94157870 0x7f7e940b14f0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f7e8400b3c0 tx=0x7f7e84017000 comp rx=0 tx=0).stop 2023-10-08T00:16:54.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.439+0000 7f7e99d54700 1 -- 172.21.15.99:0/1322415221 shutdown_connections 2023-10-08T00:16:54.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.439+0000 7f7e99d54700 1 --2- 172.21.15.99:0/1322415221 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7e80066fb0 0x7f7e80069460 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:54.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.439+0000 7f7e99d54700 1 --2- 172.21.15.99:0/1322415221 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e9415d580 0x7f7e9414c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:54.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.439+0000 7f7e99d54700 1 --2- 172.21.15.99:0/1322415221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e94157870 0x7f7e940b14f0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:54.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.439+0000 7f7e99d54700 1 --2- 172.21.15.99:0/1322415221 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e9415a620 0x7f7e94151970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:54.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.439+0000 7f7e99d54700 1 -- 172.21.15.99:0/1322415221 >> 172.21.15.99:0/1322415221 conn(0x7f7e94068ce0 msgr2=0x7f7e9415aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:54.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.439+0000 7f7e99d54700 1 -- 172.21.15.99:0/1322415221 shutdown_connections 2023-10-08T00:16:54.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.439+0000 7f7e99d54700 1 -- 172.21.15.99:0/1322415221 wait complete. 2023-10-08T00:16:54.452 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T00:16:54.452 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T00:16:54.566 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.565+0000 7fb624caa700 1 Processor -- start 2023-10-08T00:16:54.566 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.565+0000 7fb624caa700 1 -- start start 2023-10-08T00:16:54.568 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.566+0000 7fb624caa700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb6200cdab0 0x7fb62015d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:54.568 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.566+0000 7fb624caa700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6200cb7c0 con 0x7fb6200cd6e0 2023-10-08T00:16:54.568 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.566+0000 7fb624caa700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb62015e290 con 0x7fb62015dda0 2023-10-08T00:16:54.568 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.566+0000 7fb624caa700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb62015e850 con 0x7fb6200cdab0 2023-10-08T00:16:54.568 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.566+0000 7fb61effd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb6200cdab0 0x7fb62015d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:54.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.566+0000 7fb61f7fe700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fb6200cd6e0 0x7fb6200cbc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:46842/0 (socket says 172.21.15.99:46842) 2023-10-08T00:16:54.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.566+0000 7fb61f7fe700 1 -- 172.21.15.99:0/1259200040 learned_addr learned my addr 172.21.15.99:0/1259200040 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:54.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.566+0000 7fb61e7fc700 1 -- 172.21.15.99:0/1259200040 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3461473113 0 0) 0x7fb6200cb7c0 con 0x7fb6200cd6e0 2023-10-08T00:16:54.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.566+0000 7fb61e7fc700 1 -- 172.21.15.99:0/1259200040 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb608003570 con 0x7fb6200cd6e0 2023-10-08T00:16:54.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.566+0000 7fb61effd700 1 -- 172.21.15.99:0/1259200040 >> v1:172.21.15.191:6789/0 conn(0x7fb62015dda0 legacy=0x7fb62015cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:54.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.566+0000 7fb61effd700 1 -- 172.21.15.99:0/1259200040 >> v1:172.21.15.99:6789/0 conn(0x7fb6200cd6e0 legacy=0x7fb6200cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:54.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.566+0000 7fb61effd700 1 -- 172.21.15.99:0/1259200040 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb62015e570 con 0x7fb6200cdab0 2023-10-08T00:16:54.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.567+0000 7fb61effd700 1 --2- 172.21.15.99:0/1259200040 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6200cdab0 0x7fb62015d860 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fb6140098e0 tx=0x7fb6140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=246551dd5e2e6983 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:54.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.567+0000 7fb61e7fc700 1 -- 172.21.15.99:0/1259200040 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61400b640 con 0x7fb6200cdab0 2023-10-08T00:16:54.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.567+0000 7fb61e7fc700 1 -- 172.21.15.99:0/1259200040 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb61400b7a0 con 0x7fb6200cdab0 2023-10-08T00:16:54.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.567+0000 7fb61e7fc700 1 -- 172.21.15.99:0/1259200040 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61400bb40 con 0x7fb6200cdab0 2023-10-08T00:16:54.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.567+0000 7fb624caa700 1 -- 172.21.15.99:0/1259200040 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6200cdab0 msgr2=0x7fb62015d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:54.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.567+0000 7fb624caa700 1 --2- 172.21.15.99:0/1259200040 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6200cdab0 0x7fb62015d860 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fb6140098e0 tx=0x7fb6140091a0 comp rx=0 tx=0).stop 2023-10-08T00:16:54.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.567+0000 7fb624caa700 1 -- 172.21.15.99:0/1259200040 shutdown_connections 2023-10-08T00:16:54.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.567+0000 7fb624caa700 1 --2- 172.21.15.99:0/1259200040 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6200cdab0 0x7fb62015d860 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:54.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.567+0000 7fb624caa700 1 -- 172.21.15.99:0/1259200040 >> 172.21.15.99:0/1259200040 conn(0x7fb620067660 msgr2=0x7fb620067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:54.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.567+0000 7fb624caa700 1 -- 172.21.15.99:0/1259200040 shutdown_connections 2023-10-08T00:16:54.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.567+0000 7fb624caa700 1 -- 172.21.15.99:0/1259200040 wait complete. 2023-10-08T00:16:54.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.568+0000 7fb624caa700 1 Processor -- start 2023-10-08T00:16:54.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.569+0000 7fb624caa700 1 -- start start 2023-10-08T00:16:54.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.569+0000 7fb624caa700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6200cd6e0 0x7fb62014d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:54.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.569+0000 7fb624caa700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb6200cdab0 0x7fb62015a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:54.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.569+0000 7fb624caa700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb62015dda0 0x7fb62015ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:54.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.569+0000 7fb624caa700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6200cc360 con 0x7fb6200cd6e0 2023-10-08T00:16:54.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.569+0000 7fb624caa700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6200c7e40 con 0x7fb62015dda0 2023-10-08T00:16:54.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.569+0000 7fb624caa700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6200c80e0 con 0x7fb6200cdab0 2023-10-08T00:16:54.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.569+0000 7fb61f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6200cd6e0 0x7fb62014d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:54.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.569+0000 7fb61f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6200cd6e0 0x7fb62014d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55948/0 (socket says 172.21.15.99:55948) 2023-10-08T00:16:54.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.569+0000 7fb61f7fe700 1 -- 172.21.15.99:0/273429957 learned_addr learned my addr 172.21.15.99:0/273429957 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:54.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.569+0000 7fb61effd700 1 --2- 172.21.15.99:0/273429957 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb6200cdab0 0x7fb62015a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:54.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.569+0000 7fb61ffff700 1 --2- 172.21.15.99:0/273429957 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb62015dda0 0x7fb62015ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:54.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.570+0000 7fb61f7fe700 1 -- 172.21.15.99:0/273429957 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb6200cdab0 msgr2=0x7fb62015a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:54.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.570+0000 7fb61f7fe700 1 --2- 172.21.15.99:0/273429957 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb6200cdab0 0x7fb62015a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:54.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.570+0000 7fb61f7fe700 1 -- 172.21.15.99:0/273429957 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb62015dda0 msgr2=0x7fb62015ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:54.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.570+0000 7fb61f7fe700 1 --2- 172.21.15.99:0/273429957 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb62015dda0 0x7fb62015ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:54.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.570+0000 7fb61f7fe700 1 -- 172.21.15.99:0/273429957 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb614009420 con 0x7fb6200cd6e0 2023-10-08T00:16:54.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.570+0000 7fb61ffff700 1 --2- 172.21.15.99:0/273429957 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb62015dda0 0x7fb62015ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:54.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.570+0000 7fb61f7fe700 1 --2- 172.21.15.99:0/273429957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6200cd6e0 0x7fb62014d760 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fb61000cd30 tx=0x7fb610001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:54.574 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.570+0000 7fb6077fe700 1 -- 172.21.15.99:0/273429957 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61000ab70 con 0x7fb6200cd6e0 2023-10-08T00:16:54.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.570+0000 7fb6077fe700 1 -- 172.21.15.99:0/273429957 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb61000acf0 con 0x7fb6200cd6e0 2023-10-08T00:16:54.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.570+0000 7fb624caa700 1 -- 172.21.15.99:0/273429957 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6200c83c0 con 0x7fb6200cd6e0 2023-10-08T00:16:54.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.570+0000 7fb6077fe700 1 -- 172.21.15.99:0/273429957 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb610007340 con 0x7fb6200cd6e0 2023-10-08T00:16:54.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.570+0000 7fb624caa700 1 -- 172.21.15.99:0/273429957 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6200c8980 con 0x7fb6200cd6e0 2023-10-08T00:16:54.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.571+0000 7fb624caa700 1 -- 172.21.15.99:0/273429957 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb620089280 con 0x7fb6200cd6e0 2023-10-08T00:16:54.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.571+0000 7fb6077fe700 1 -- 172.21.15.99:0/273429957 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb610007730 con 0x7fb6200cd6e0 2023-10-08T00:16:54.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.572+0000 7fb6077fe700 1 --2- 172.21.15.99:0/273429957 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb60c066f30 0x7fb60c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:54.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.572+0000 7fb6077fe700 1 -- 172.21.15.99:0/273429957 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb610084a30 con 0x7fb6200cd6e0 2023-10-08T00:16:54.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.572+0000 7fb61effd700 1 --2- 172.21.15.99:0/273429957 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb60c066f30 0x7fb60c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:54.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.572+0000 7fb61effd700 1 --2- 172.21.15.99:0/273429957 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb60c066f30 0x7fb60c0693e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fb6140097b0 tx=0x7fb614017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:54.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.576+0000 7fb6077fe700 1 -- 172.21.15.99:0/273429957 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb610051d20 con 0x7fb6200cd6e0 2023-10-08T00:16:54.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.754+0000 7fb624caa700 1 -- 172.21.15.99:0/273429957 --> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb6200b35e0 con 0x7fb60c066f30 2023-10-08T00:16:54.763 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.763+0000 7fb6077fe700 1 -- 172.21.15.99:0/273429957 <== mgr.4099 v2:172.21.15.191:6832/85985 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285712 (secure 0 0 0) 0x7fb6200b35e0 con 0x7fb60c066f30 2023-10-08T00:16:54.767 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:16:54.770 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.766+0000 7fb624caa700 1 -- 172.21.15.99:0/273429957 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb60c066f30 msgr2=0x7fb60c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:54.770 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.766+0000 7fb624caa700 1 --2- 172.21.15.99:0/273429957 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb60c066f30 0x7fb60c0693e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fb6140097b0 tx=0x7fb614017000 comp rx=0 tx=0).stop 2023-10-08T00:16:54.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.766+0000 7fb624caa700 1 -- 172.21.15.99:0/273429957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6200cd6e0 msgr2=0x7fb62014d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:54.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.766+0000 7fb624caa700 1 --2- 172.21.15.99:0/273429957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6200cd6e0 0x7fb62014d760 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fb61000cd30 tx=0x7fb610001b10 comp rx=0 tx=0).stop 2023-10-08T00:16:54.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.766+0000 7fb624caa700 1 -- 172.21.15.99:0/273429957 shutdown_connections 2023-10-08T00:16:54.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.766+0000 7fb624caa700 1 --2- 172.21.15.99:0/273429957 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb60c066f30 0x7fb60c0693e0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:54.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.766+0000 7fb624caa700 1 --2- 172.21.15.99:0/273429957 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb62015dda0 0x7fb62015ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:54.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.766+0000 7fb624caa700 1 --2- 172.21.15.99:0/273429957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6200cd6e0 0x7fb62014d760 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:54.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.766+0000 7fb624caa700 1 --2- 172.21.15.99:0/273429957 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb6200cdab0 0x7fb62015a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:54.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.766+0000 7fb624caa700 1 -- 172.21.15.99:0/273429957 >> 172.21.15.99:0/273429957 conn(0x7fb620067660 msgr2=0x7fb6200b07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:54.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.766+0000 7fb624caa700 1 -- 172.21.15.99:0/273429957 shutdown_connections 2023-10-08T00:16:54.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.766+0000 7fb624caa700 1 -- 172.21.15.99:0/273429957 wait complete. 2023-10-08T00:16:54.773 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2023-10-08T00:16:54.778 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":315,"stamp":"2023-10-08T00:16:54.109263+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":126,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":7153,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555346,"statfs":{"total":1600000000,"available":1592675100,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.396980"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.738670+0000","last_change":"2023-10-08T00:16:50.737905+0000","last_active":"2023-10-08T00:16:50.738670+0000","last_peered":"2023-10-08T00:16:50.738670+0000","last_clean":"2023-10-08T00:16:50.738670+0000","last_became_active":"2023-10-08T00:16:50.736176+0000","last_became_peered":"2023-10-08T00:16:50.736176+0000","last_unstale":"2023-10-08T00:16:50.738670+0000","last_undegraded":"2023-10-08T00:16:50.738670+0000","last_fullsized":"2023-10-08T00:16:50.738670+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:09:43.382366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.735639+0000","last_change":"2023-10-08T00:16:50.735542+0000","last_active":"2023-10-08T00:16:50.735639+0000","last_peered":"2023-10-08T00:16:50.735639+0000","last_clean":"2023-10-08T00:16:50.735639+0000","last_became_active":"2023-10-08T00:16:50.735133+0000","last_became_peered":"2023-10-08T00:16:50.735133+0000","last_unstale":"2023-10-08T00:16:50.735639+0000","last_undegraded":"2023-10-08T00:16:50.735639+0000","last_fullsized":"2023-10-08T00:16:50.735639+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:28:01.427127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.730656+0000","last_change":"2023-10-08T00:16:50.730594+0000","last_active":"2023-10-08T00:16:50.730656+0000","last_peered":"2023-10-08T00:16:50.730656+0000","last_clean":"2023-10-08T00:16:50.730656+0000","last_became_active":"2023-10-08T00:16:50.730375+0000","last_became_peered":"2023-10-08T00:16:50.730375+0000","last_unstale":"2023-10-08T00:16:50.730656+0000","last_undegraded":"2023-10-08T00:16:50.730656+0000","last_fullsized":"2023-10-08T00:16:50.730656+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:49:20.063701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.734354+0000","last_change":"2023-10-08T00:16:50.734293+0000","last_active":"2023-10-08T00:16:50.734354+0000","last_peered":"2023-10-08T00:16:50.734354+0000","last_clean":"2023-10-08T00:16:50.734354+0000","last_became_active":"2023-10-08T00:16:50.734039+0000","last_became_peered":"2023-10-08T00:16:50.734039+0000","last_unstale":"2023-10-08T00:16:50.734354+0000","last_undegraded":"2023-10-08T00:16:50.734354+0000","last_fullsized":"2023-10-08T00:16:50.734354+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:44:32.941862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.748672+0000","last_change":"2023-10-08T00:16:50.748168+0000","last_active":"2023-10-08T00:16:50.748672+0000","last_peered":"2023-10-08T00:16:50.748672+0000","last_clean":"2023-10-08T00:16:50.748672+0000","last_became_active":"2023-10-08T00:16:50.748000+0000","last_became_peered":"2023-10-08T00:16:50.748000+0000","last_unstale":"2023-10-08T00:16:50.748672+0000","last_undegraded":"2023-10-08T00:16:50.748672+0000","last_fullsized":"2023-10-08T00:16:50.748672+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:53:19.351074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.737901+0000","last_change":"2023-10-08T00:16:50.736246+0000","last_active":"2023-10-08T00:16:50.737901+0000","last_peered":"2023-10-08T00:16:50.737901+0000","last_clean":"2023-10-08T00:16:50.737901+0000","last_became_active":"2023-10-08T00:16:50.736052+0000","last_became_peered":"2023-10-08T00:16:50.736052+0000","last_unstale":"2023-10-08T00:16:50.737901+0000","last_undegraded":"2023-10-08T00:16:50.737901+0000","last_fullsized":"2023-10-08T00:16:50.737901+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:17:10.436982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.737565+0000","last_change":"2023-10-08T00:16:50.737453+0000","last_active":"2023-10-08T00:16:50.737565+0000","last_peered":"2023-10-08T00:16:50.737565+0000","last_clean":"2023-10-08T00:16:50.737565+0000","last_became_active":"2023-10-08T00:16:50.737257+0000","last_became_peered":"2023-10-08T00:16:50.737257+0000","last_unstale":"2023-10-08T00:16:50.737565+0000","last_undegraded":"2023-10-08T00:16:50.737565+0000","last_fullsized":"2023-10-08T00:16:50.737565+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:42:38.573355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.733487+0000","last_change":"2023-10-08T00:16:50.733289+0000","last_active":"2023-10-08T00:16:50.733487+0000","last_peered":"2023-10-08T00:16:50.733487+0000","last_clean":"2023-10-08T00:16:50.733487+0000","last_became_active":"2023-10-08T00:16:50.733095+0000","last_became_peered":"2023-10-08T00:16:50.733095+0000","last_unstale":"2023-10-08T00:16:50.733487+0000","last_undegraded":"2023-10-08T00:16:50.733487+0000","last_fullsized":"2023-10-08T00:16:50.733487+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:52:52.729685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.740049+0000","last_change":"2023-10-08T00:16:50.739989+0000","last_active":"2023-10-08T00:16:50.740049+0000","last_peered":"2023-10-08T00:16:50.740049+0000","last_clean":"2023-10-08T00:16:50.740049+0000","last_became_active":"2023-10-08T00:16:50.739634+0000","last_became_peered":"2023-10-08T00:16:50.739634+0000","last_unstale":"2023-10-08T00:16:50.740049+0000","last_undegraded":"2023-10-08T00:16:50.740049+0000","last_fullsized":"2023-10-08T00:16:50.740049+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:38:58.924084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.744458+0000","last_change":"2023-10-08T00:16:50.744404+0000","last_active":"2023-10-08T00:16:50.744458+0000","last_peered":"2023-10-08T00:16:50.744458+0000","last_clean":"2023-10-08T00:16:50.744458+0000","last_became_active":"2023-10-08T00:16:50.744185+0000","last_became_peered":"2023-10-08T00:16:50.744185+0000","last_unstale":"2023-10-08T00:16:50.744458+0000","last_undegraded":"2023-10-08T00:16:50.744458+0000","last_fullsized":"2023-10-08T00:16:50.744458+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:08:56.689278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.737547+0000","last_change":"2023-10-08T00:16:50.737398+0000","last_active":"2023-10-08T00:16:50.737547+0000","last_peered":"2023-10-08T00:16:50.737547+0000","last_clean":"2023-10-08T00:16:50.737547+0000","last_became_active":"2023-10-08T00:16:50.737195+0000","last_became_peered":"2023-10-08T00:16:50.737195+0000","last_unstale":"2023-10-08T00:16:50.737547+0000","last_undegraded":"2023-10-08T00:16:50.737547+0000","last_fullsized":"2023-10-08T00:16:50.737547+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:36:42.997162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.743585+0000","last_change":"2023-10-08T00:16:50.743504+0000","last_active":"2023-10-08T00:16:50.743585+0000","last_peered":"2023-10-08T00:16:50.743585+0000","last_clean":"2023-10-08T00:16:50.743585+0000","last_became_active":"2023-10-08T00:16:50.742979+0000","last_became_peered":"2023-10-08T00:16:50.742979+0000","last_unstale":"2023-10-08T00:16:50.743585+0000","last_undegraded":"2023-10-08T00:16:50.743585+0000","last_fullsized":"2023-10-08T00:16:50.743585+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:49:08.450879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.742360+0000","last_change":"2023-10-08T00:16:50.742306+0000","last_active":"2023-10-08T00:16:50.742360+0000","last_peered":"2023-10-08T00:16:50.742360+0000","last_clean":"2023-10-08T00:16:50.742360+0000","last_became_active":"2023-10-08T00:16:50.742064+0000","last_became_peered":"2023-10-08T00:16:50.742064+0000","last_unstale":"2023-10-08T00:16:50.742360+0000","last_undegraded":"2023-10-08T00:16:50.742360+0000","last_fullsized":"2023-10-08T00:16:50.742360+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:21:02.984633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.739965+0000","last_change":"2023-10-08T00:16:50.739921+0000","last_active":"2023-10-08T00:16:50.739965+0000","last_peered":"2023-10-08T00:16:50.739965+0000","last_clean":"2023-10-08T00:16:50.739965+0000","last_became_active":"2023-10-08T00:16:50.739605+0000","last_became_peered":"2023-10-08T00:16:50.739605+0000","last_unstale":"2023-10-08T00:16:50.739965+0000","last_undegraded":"2023-10-08T00:16:50.739965+0000","last_fullsized":"2023-10-08T00:16:50.739965+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:57:49.104276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.746133+0000","last_change":"2023-10-08T00:16:50.746042+0000","last_active":"2023-10-08T00:16:50.746133+0000","last_peered":"2023-10-08T00:16:50.746133+0000","last_clean":"2023-10-08T00:16:50.746133+0000","last_became_active":"2023-10-08T00:16:50.745655+0000","last_became_peered":"2023-10-08T00:16:50.745655+0000","last_unstale":"2023-10-08T00:16:50.746133+0000","last_undegraded":"2023-10-08T00:16:50.746133+0000","last_fullsized":"2023-10-08T00:16:50.746133+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:17:10.436982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.742338+0000","last_change":"2023-10-08T00:16:50.742281+0000","last_active":"2023-10-08T00:16:50.742338+0000","last_peered":"2023-10-08T00:16:50.742338+0000","last_clean":"2023-10-08T00:16:50.742338+0000","last_became_active":"2023-10-08T00:16:50.741932+0000","last_became_peered":"2023-10-08T00:16:50.741932+0000","last_unstale":"2023-10-08T00:16:50.742338+0000","last_undegraded":"2023-10-08T00:16:50.742338+0000","last_fullsized":"2023-10-08T00:16:50.742338+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:48:03.562982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.737913+0000","last_change":"2023-10-08T00:16:50.737858+0000","last_active":"2023-10-08T00:16:50.737913+0000","last_peered":"2023-10-08T00:16:50.737913+0000","last_clean":"2023-10-08T00:16:50.737913+0000","last_became_active":"2023-10-08T00:16:50.737628+0000","last_became_peered":"2023-10-08T00:16:50.737628+0000","last_unstale":"2023-10-08T00:16:50.737913+0000","last_undegraded":"2023-10-08T00:16:50.737913+0000","last_fullsized":"2023-10-08T00:16:50.737913+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:53:24.578330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.739882+0000","last_change":"2023-10-08T00:16:50.739828+0000","last_active":"2023-10-08T00:16:50.739882+0000","last_peered":"2023-10-08T00:16:50.739882+0000","last_clean":"2023-10-08T00:16:50.739882+0000","last_became_active":"2023-10-08T00:16:50.739634+0000","last_became_peered":"2023-10-08T00:16:50.739634+0000","last_unstale":"2023-10-08T00:16:50.739882+0000","last_undegraded":"2023-10-08T00:16:50.739882+0000","last_fullsized":"2023-10-08T00:16:50.739882+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:47:35.965982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.742032+0000","last_change":"2023-10-08T00:16:50.741598+0000","last_active":"2023-10-08T00:16:50.742032+0000","last_peered":"2023-10-08T00:16:50.742032+0000","last_clean":"2023-10-08T00:16:50.742032+0000","last_became_active":"2023-10-08T00:16:50.741387+0000","last_became_peered":"2023-10-08T00:16:50.741387+0000","last_unstale":"2023-10-08T00:16:50.742032+0000","last_undegraded":"2023-10-08T00:16:50.742032+0000","last_fullsized":"2023-10-08T00:16:50.742032+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:38:38.194551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.736134+0000","last_change":"2023-10-08T00:16:50.736043+0000","last_active":"2023-10-08T00:16:50.736134+0000","last_peered":"2023-10-08T00:16:50.736134+0000","last_clean":"2023-10-08T00:16:50.736134+0000","last_became_active":"2023-10-08T00:16:50.735850+0000","last_became_peered":"2023-10-08T00:16:50.735850+0000","last_unstale":"2023-10-08T00:16:50.736134+0000","last_undegraded":"2023-10-08T00:16:50.736134+0000","last_fullsized":"2023-10-08T00:16:50.736134+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:29:26.667047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.736599+0000","last_change":"2023-10-08T00:16:50.736361+0000","last_active":"2023-10-08T00:16:50.736599+0000","last_peered":"2023-10-08T00:16:50.736599+0000","last_clean":"2023-10-08T00:16:50.736599+0000","last_became_active":"2023-10-08T00:16:50.736134+0000","last_became_peered":"2023-10-08T00:16:50.736134+0000","last_unstale":"2023-10-08T00:16:50.736599+0000","last_undegraded":"2023-10-08T00:16:50.736599+0000","last_fullsized":"2023-10-08T00:16:50.736599+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:35:31.155748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.737870+0000","last_change":"2023-10-08T00:16:50.734528+0000","last_active":"2023-10-08T00:16:50.737870+0000","last_peered":"2023-10-08T00:16:50.737870+0000","last_clean":"2023-10-08T00:16:50.737870+0000","last_became_active":"2023-10-08T00:16:50.734294+0000","last_became_peered":"2023-10-08T00:16:50.734294+0000","last_unstale":"2023-10-08T00:16:50.737870+0000","last_undegraded":"2023-10-08T00:16:50.737870+0000","last_fullsized":"2023-10-08T00:16:50.737870+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:38:58.924084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.739376+0000","last_change":"2023-10-08T00:16:50.739322+0000","last_active":"2023-10-08T00:16:50.739376+0000","last_peered":"2023-10-08T00:16:50.739376+0000","last_clean":"2023-10-08T00:16:50.739376+0000","last_became_active":"2023-10-08T00:16:50.739116+0000","last_became_peered":"2023-10-08T00:16:50.739116+0000","last_unstale":"2023-10-08T00:16:50.739376+0000","last_undegraded":"2023-10-08T00:16:50.739376+0000","last_fullsized":"2023-10-08T00:16:50.739376+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:30:15.845800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.738544+0000","last_change":"2023-10-08T00:16:50.738361+0000","last_active":"2023-10-08T00:16:50.738544+0000","last_peered":"2023-10-08T00:16:50.738544+0000","last_clean":"2023-10-08T00:16:50.738544+0000","last_became_active":"2023-10-08T00:16:50.738013+0000","last_became_peered":"2023-10-08T00:16:50.738013+0000","last_unstale":"2023-10-08T00:16:50.738544+0000","last_undegraded":"2023-10-08T00:16:50.738544+0000","last_fullsized":"2023-10-08T00:16:50.738544+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:16:47.905466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.733723+0000","last_change":"2023-10-08T00:16:50.733524+0000","last_active":"2023-10-08T00:16:50.733723+0000","last_peered":"2023-10-08T00:16:50.733723+0000","last_clean":"2023-10-08T00:16:50.733723+0000","last_became_active":"2023-10-08T00:16:50.733020+0000","last_became_peered":"2023-10-08T00:16:50.733020+0000","last_unstale":"2023-10-08T00:16:50.733723+0000","last_undegraded":"2023-10-08T00:16:50.733723+0000","last_fullsized":"2023-10-08T00:16:50.733723+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:37:43.082386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.729830+0000","last_change":"2023-10-08T00:16:50.729766+0000","last_active":"2023-10-08T00:16:50.729830+0000","last_peered":"2023-10-08T00:16:50.729830+0000","last_clean":"2023-10-08T00:16:50.729830+0000","last_became_active":"2023-10-08T00:16:50.729640+0000","last_became_peered":"2023-10-08T00:16:50.729640+0000","last_unstale":"2023-10-08T00:16:50.729830+0000","last_undegraded":"2023-10-08T00:16:50.729830+0000","last_fullsized":"2023-10-08T00:16:50.729830+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:57:49.104276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.738528+0000","last_change":"2023-10-08T00:16:50.737900+0000","last_active":"2023-10-08T00:16:50.738528+0000","last_peered":"2023-10-08T00:16:50.738528+0000","last_clean":"2023-10-08T00:16:50.738528+0000","last_became_active":"2023-10-08T00:16:50.736680+0000","last_became_peered":"2023-10-08T00:16:50.736680+0000","last_unstale":"2023-10-08T00:16:50.738528+0000","last_undegraded":"2023-10-08T00:16:50.738528+0000","last_fullsized":"2023-10-08T00:16:50.738528+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:41:41.134711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.737851+0000","last_change":"2023-10-08T00:16:50.733336+0000","last_active":"2023-10-08T00:16:50.737851+0000","last_peered":"2023-10-08T00:16:50.737851+0000","last_clean":"2023-10-08T00:16:50.737851+0000","last_became_active":"2023-10-08T00:16:50.733152+0000","last_became_peered":"2023-10-08T00:16:50.733152+0000","last_unstale":"2023-10-08T00:16:50.737851+0000","last_undegraded":"2023-10-08T00:16:50.737851+0000","last_fullsized":"2023-10-08T00:16:50.737851+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:45:12.144679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.739863+0000","last_change":"2023-10-08T00:16:50.739806+0000","last_active":"2023-10-08T00:16:50.739863+0000","last_peered":"2023-10-08T00:16:50.739863+0000","last_clean":"2023-10-08T00:16:50.739863+0000","last_became_active":"2023-10-08T00:16:50.739346+0000","last_became_peered":"2023-10-08T00:16:50.739346+0000","last_unstale":"2023-10-08T00:16:50.739863+0000","last_undegraded":"2023-10-08T00:16:50.739863+0000","last_fullsized":"2023-10-08T00:16:50.739863+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:17:10.436982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.733738+0000","last_change":"2023-10-08T00:16:50.733649+0000","last_active":"2023-10-08T00:16:50.733738+0000","last_peered":"2023-10-08T00:16:50.733738+0000","last_clean":"2023-10-08T00:16:50.733738+0000","last_became_active":"2023-10-08T00:16:50.733308+0000","last_became_peered":"2023-10-08T00:16:50.733308+0000","last_unstale":"2023-10-08T00:16:50.733738+0000","last_undegraded":"2023-10-08T00:16:50.733738+0000","last_fullsized":"2023-10-08T00:16:50.733738+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:16:44.190570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.738689+0000","last_change":"2023-10-08T00:16:50.738244+0000","last_active":"2023-10-08T00:16:50.738689+0000","last_peered":"2023-10-08T00:16:50.738689+0000","last_clean":"2023-10-08T00:16:50.738689+0000","last_became_active":"2023-10-08T00:16:50.736457+0000","last_became_peered":"2023-10-08T00:16:50.736457+0000","last_unstale":"2023-10-08T00:16:50.738689+0000","last_undegraded":"2023-10-08T00:16:50.738689+0000","last_fullsized":"2023-10-08T00:16:50.738689+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:57:49.104276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.743840+0000","last_change":"2023-10-08T00:16:50.743777+0000","last_active":"2023-10-08T00:16:50.743840+0000","last_peered":"2023-10-08T00:16:50.743840+0000","last_clean":"2023-10-08T00:16:50.743840+0000","last_became_active":"2023-10-08T00:16:50.743300+0000","last_became_peered":"2023-10-08T00:16:50.743300+0000","last_unstale":"2023-10-08T00:16:50.743840+0000","last_undegraded":"2023-10-08T00:16:50.743840+0000","last_fullsized":"2023-10-08T00:16:50.743840+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:17:21.037010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.739908+0000","last_change":"2023-10-08T00:16:50.739852+0000","last_active":"2023-10-08T00:16:50.739908+0000","last_peered":"2023-10-08T00:16:50.739908+0000","last_clean":"2023-10-08T00:16:50.739908+0000","last_became_active":"2023-10-08T00:16:50.739316+0000","last_became_peered":"2023-10-08T00:16:50.739316+0000","last_unstale":"2023-10-08T00:16:50.739908+0000","last_undegraded":"2023-10-08T00:16:50.739908+0000","last_fullsized":"2023-10-08T00:16:50.739908+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:53:27.357114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"138'4","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.746348+0000","last_change":"2023-10-08T00:16:49.719279+0000","last_active":"2023-10-08T00:16:50.746348+0000","last_peered":"2023-10-08T00:16:50.746348+0000","last_clean":"2023-10-08T00:16:50.746348+0000","last_became_active":"2023-10-08T00:16:49.719031+0000","last_became_peered":"2023-10-08T00:16:49.719031+0000","last_unstale":"2023-10-08T00:16:50.746348+0000","last_undegraded":"2023-10-08T00:16:50.746348+0000","last_fullsized":"2023-10-08T00:16:50.746348+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:08:55.684301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.738520+0000","last_change":"2023-10-08T00:16:50.738464+0000","last_active":"2023-10-08T00:16:50.738520+0000","last_peered":"2023-10-08T00:16:50.738520+0000","last_clean":"2023-10-08T00:16:50.738520+0000","last_became_active":"2023-10-08T00:16:50.738253+0000","last_became_peered":"2023-10-08T00:16:50.738253+0000","last_unstale":"2023-10-08T00:16:50.738520+0000","last_undegraded":"2023-10-08T00:16:50.738520+0000","last_fullsized":"2023-10-08T00:16:50.738520+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:21:02.984633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"138'1","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.744981+0000","last_change":"2023-10-08T00:16:49.721557+0000","last_active":"2023-10-08T00:16:50.744981+0000","last_peered":"2023-10-08T00:16:50.744981+0000","last_clean":"2023-10-08T00:16:50.744981+0000","last_became_active":"2023-10-08T00:16:49.721319+0000","last_became_peered":"2023-10-08T00:16:49.721319+0000","last_unstale":"2023-10-08T00:16:50.744981+0000","last_undegraded":"2023-10-08T00:16:50.744981+0000","last_fullsized":"2023-10-08T00:16:50.744981+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:48:02.558006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.737886+0000","last_change":"2023-10-08T00:16:50.735836+0000","last_active":"2023-10-08T00:16:50.737886+0000","last_peered":"2023-10-08T00:16:50.737886+0000","last_clean":"2023-10-08T00:16:50.737886+0000","last_became_active":"2023-10-08T00:16:50.735625+0000","last_became_peered":"2023-10-08T00:16:50.735625+0000","last_unstale":"2023-10-08T00:16:50.737886+0000","last_undegraded":"2023-10-08T00:16:50.737886+0000","last_fullsized":"2023-10-08T00:16:50.737886+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:34:23.975157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1d","version":"138'2","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.750483+0000","last_change":"2023-10-08T00:16:49.722772+0000","last_active":"2023-10-08T00:16:50.750483+0000","last_peered":"2023-10-08T00:16:50.750483+0000","last_clean":"2023-10-08T00:16:50.750483+0000","last_became_active":"2023-10-08T00:16:49.721872+0000","last_became_peered":"2023-10-08T00:16:49.721872+0000","last_unstale":"2023-10-08T00:16:50.750483+0000","last_undegraded":"2023-10-08T00:16:50.750483+0000","last_fullsized":"2023-10-08T00:16:50.750483+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:53:23.573354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.743075+0000","last_change":"2023-10-08T00:16:50.742985+0000","last_active":"2023-10-08T00:16:50.743075+0000","last_peered":"2023-10-08T00:16:50.743075+0000","last_clean":"2023-10-08T00:16:50.743075+0000","last_became_active":"2023-10-08T00:16:50.742803+0000","last_became_peered":"2023-10-08T00:16:50.742803+0000","last_unstale":"2023-10-08T00:16:50.743075+0000","last_undegraded":"2023-10-08T00:16:50.743075+0000","last_fullsized":"2023-10-08T00:16:50.743075+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:14:18.374798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"138'2","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.750339+0000","last_change":"2023-10-08T00:16:49.724426+0000","last_active":"2023-10-08T00:16:50.750339+0000","last_peered":"2023-10-08T00:16:50.750339+0000","last_clean":"2023-10-08T00:16:50.750339+0000","last_became_active":"2023-10-08T00:16:49.723645+0000","last_became_peered":"2023-10-08T00:16:49.723645+0000","last_unstale":"2023-10-08T00:16:50.750339+0000","last_undegraded":"2023-10-08T00:16:50.750339+0000","last_fullsized":"2023-10-08T00:16:50.750339+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:14:17.369821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.738503+0000","last_change":"2023-10-08T00:16:50.737642+0000","last_active":"2023-10-08T00:16:50.738503+0000","last_peered":"2023-10-08T00:16:50.738503+0000","last_clean":"2023-10-08T00:16:50.738503+0000","last_became_active":"2023-10-08T00:16:50.737393+0000","last_became_peered":"2023-10-08T00:16:50.737393+0000","last_unstale":"2023-10-08T00:16:50.738503+0000","last_undegraded":"2023-10-08T00:16:50.738503+0000","last_fullsized":"2023-10-08T00:16:50.738503+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:48:03.562982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1b","version":"138'1","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.745606+0000","last_change":"2023-10-08T00:16:49.725193+0000","last_active":"2023-10-08T00:16:50.745606+0000","last_peered":"2023-10-08T00:16:50.745606+0000","last_clean":"2023-10-08T00:16:50.745606+0000","last_became_active":"2023-10-08T00:16:49.724960+0000","last_became_peered":"2023-10-08T00:16:49.724960+0000","last_unstale":"2023-10-08T00:16:50.745606+0000","last_undegraded":"2023-10-08T00:16:50.745606+0000","last_fullsized":"2023-10-08T00:16:50.745606+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:41:22.003276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.742409+0000","last_change":"2023-10-08T00:16:50.742367+0000","last_active":"2023-10-08T00:16:50.742409+0000","last_peered":"2023-10-08T00:16:50.742409+0000","last_clean":"2023-10-08T00:16:50.742409+0000","last_became_active":"2023-10-08T00:16:50.742227+0000","last_became_peered":"2023-10-08T00:16:50.742227+0000","last_unstale":"2023-10-08T00:16:50.742409+0000","last_undegraded":"2023-10-08T00:16:50.742409+0000","last_fullsized":"2023-10-08T00:16:50.742409+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:17:26.761580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1a","version":"138'1","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.750338+0000","last_change":"2023-10-08T00:16:49.722953+0000","last_active":"2023-10-08T00:16:50.750338+0000","last_peered":"2023-10-08T00:16:50.750338+0000","last_clean":"2023-10-08T00:16:50.750338+0000","last_became_active":"2023-10-08T00:16:49.722706+0000","last_became_peered":"2023-10-08T00:16:49.722706+0000","last_unstale":"2023-10-08T00:16:50.750338+0000","last_undegraded":"2023-10-08T00:16:50.750338+0000","last_fullsized":"2023-10-08T00:16:50.750338+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:48:02.558006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.748470+0000","last_change":"2023-10-08T00:16:50.746574+0000","last_active":"2023-10-08T00:16:50.748470+0000","last_peered":"2023-10-08T00:16:50.748470+0000","last_clean":"2023-10-08T00:16:50.748470+0000","last_became_active":"2023-10-08T00:16:50.746008+0000","last_became_peered":"2023-10-08T00:16:50.746008+0000","last_unstale":"2023-10-08T00:16:50.748470+0000","last_undegraded":"2023-10-08T00:16:50.748470+0000","last_fullsized":"2023-10-08T00:16:50.748470+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:16:47.905466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.731355+0000","last_change":"2023-10-08T00:16:49.721439+0000","last_active":"2023-10-08T00:16:50.731355+0000","last_peered":"2023-10-08T00:16:50.731355+0000","last_clean":"2023-10-08T00:16:50.731355+0000","last_became_active":"2023-10-08T00:16:49.721228+0000","last_became_peered":"2023-10-08T00:16:49.721228+0000","last_unstale":"2023-10-08T00:16:50.731355+0000","last_undegraded":"2023-10-08T00:16:50.731355+0000","last_fullsized":"2023-10-08T00:16:50.731355+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:00:03.204002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.733505+0000","last_change":"2023-10-08T00:16:50.733319+0000","last_active":"2023-10-08T00:16:50.733505+0000","last_peered":"2023-10-08T00:16:50.733505+0000","last_clean":"2023-10-08T00:16:50.733505+0000","last_became_active":"2023-10-08T00:16:50.733111+0000","last_became_peered":"2023-10-08T00:16:50.733111+0000","last_unstale":"2023-10-08T00:16:50.733505+0000","last_undegraded":"2023-10-08T00:16:50.733505+0000","last_fullsized":"2023-10-08T00:16:50.733505+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:47:03.175925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.742155+0000","last_change":"2023-10-08T00:16:49.716983+0000","last_active":"2023-10-08T00:16:50.742155+0000","last_peered":"2023-10-08T00:16:50.742155+0000","last_clean":"2023-10-08T00:16:50.742155+0000","last_became_active":"2023-10-08T00:16:49.716090+0000","last_became_peered":"2023-10-08T00:16:49.716090+0000","last_unstale":"2023-10-08T00:16:50.742155+0000","last_undegraded":"2023-10-08T00:16:50.742155+0000","last_fullsized":"2023-10-08T00:16:50.742155+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:49:19.058724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.734418+0000","last_change":"2023-10-08T00:16:50.734358+0000","last_active":"2023-10-08T00:16:50.734418+0000","last_peered":"2023-10-08T00:16:50.734418+0000","last_clean":"2023-10-08T00:16:50.734418+0000","last_became_active":"2023-10-08T00:16:50.734137+0000","last_became_peered":"2023-10-08T00:16:50.734137+0000","last_unstale":"2023-10-08T00:16:50.734418+0000","last_undegraded":"2023-10-08T00:16:50.734418+0000","last_fullsized":"2023-10-08T00:16:50.734418+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:56:08.736930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.734899+0000","last_change":"2023-10-08T00:16:49.724560+0000","last_active":"2023-10-08T00:16:50.734899+0000","last_peered":"2023-10-08T00:16:50.734899+0000","last_clean":"2023-10-08T00:16:50.734899+0000","last_became_active":"2023-10-08T00:16:49.723939+0000","last_became_peered":"2023-10-08T00:16:49.723939+0000","last_unstale":"2023-10-08T00:16:50.734899+0000","last_undegraded":"2023-10-08T00:16:50.734899+0000","last_fullsized":"2023-10-08T00:16:50.734899+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:24:15.757109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.742293+0000","last_change":"2023-10-08T00:16:50.742232+0000","last_active":"2023-10-08T00:16:50.742293+0000","last_peered":"2023-10-08T00:16:50.742293+0000","last_clean":"2023-10-08T00:16:50.742293+0000","last_became_active":"2023-10-08T00:16:50.741995+0000","last_became_peered":"2023-10-08T00:16:50.741995+0000","last_unstale":"2023-10-08T00:16:50.742293+0000","last_undegraded":"2023-10-08T00:16:50.742293+0000","last_fullsized":"2023-10-08T00:16:50.742293+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:19:40.203536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.741013+0000","last_change":"2023-10-08T00:16:49.725331+0000","last_active":"2023-10-08T00:16:50.741013+0000","last_peered":"2023-10-08T00:16:50.741013+0000","last_clean":"2023-10-08T00:16:50.741013+0000","last_became_active":"2023-10-08T00:16:49.725134+0000","last_became_peered":"2023-10-08T00:16:49.725134+0000","last_unstale":"2023-10-08T00:16:50.741013+0000","last_undegraded":"2023-10-08T00:16:50.741013+0000","last_fullsized":"2023-10-08T00:16:50.741013+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:09:50.287365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.732293+0000","last_change":"2023-10-08T00:16:50.732104+0000","last_active":"2023-10-08T00:16:50.732293+0000","last_peered":"2023-10-08T00:16:50.732293+0000","last_clean":"2023-10-08T00:16:50.732293+0000","last_became_active":"2023-10-08T00:16:50.731869+0000","last_became_peered":"2023-10-08T00:16:50.731869+0000","last_unstale":"2023-10-08T00:16:50.732293+0000","last_undegraded":"2023-10-08T00:16:50.732293+0000","last_fullsized":"2023-10-08T00:16:50.732293+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:55:58.689766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"138'3","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.750127+0000","last_change":"2023-10-08T00:16:49.720487+0000","last_active":"2023-10-08T00:16:50.750127+0000","last_peered":"2023-10-08T00:16:50.750127+0000","last_clean":"2023-10-08T00:16:50.750127+0000","last_became_active":"2023-10-08T00:16:49.720290+0000","last_became_peered":"2023-10-08T00:16:49.720290+0000","last_unstale":"2023-10-08T00:16:50.750127+0000","last_undegraded":"2023-10-08T00:16:50.750127+0000","last_fullsized":"2023-10-08T00:16:50.750127+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:47:05.808379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.732596+0000","last_change":"2023-10-08T00:16:50.732318+0000","last_active":"2023-10-08T00:16:50.732596+0000","last_peered":"2023-10-08T00:16:50.732596+0000","last_clean":"2023-10-08T00:16:50.732596+0000","last_became_active":"2023-10-08T00:16:50.732057+0000","last_became_peered":"2023-10-08T00:16:50.732057+0000","last_unstale":"2023-10-08T00:16:50.732596+0000","last_undegraded":"2023-10-08T00:16:50.732596+0000","last_fullsized":"2023-10-08T00:16:50.732596+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:06:08.939435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"138'6","reported_seq":16,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.751411+0000","last_change":"2023-10-08T00:16:49.721242+0000","last_active":"2023-10-08T00:16:50.751411+0000","last_peered":"2023-10-08T00:16:50.751411+0000","last_clean":"2023-10-08T00:16:50.751411+0000","last_became_active":"2023-10-08T00:16:49.720775+0000","last_became_peered":"2023-10-08T00:16:49.720775+0000","last_unstale":"2023-10-08T00:16:50.751411+0000","last_undegraded":"2023-10-08T00:16:50.751411+0000","last_fullsized":"2023-10-08T00:16:50.751411+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:21:27.301699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.742432+0000","last_change":"2023-10-08T00:16:50.742395+0000","last_active":"2023-10-08T00:16:50.742432+0000","last_peered":"2023-10-08T00:16:50.742432+0000","last_clean":"2023-10-08T00:16:50.742432+0000","last_became_active":"2023-10-08T00:16:50.742141+0000","last_became_peered":"2023-10-08T00:16:50.742141+0000","last_unstale":"2023-10-08T00:16:50.742432+0000","last_undegraded":"2023-10-08T00:16:50.742432+0000","last_fullsized":"2023-10-08T00:16:50.742432+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:43:14.175754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"138'2","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.750291+0000","last_change":"2023-10-08T00:16:49.718190+0000","last_active":"2023-10-08T00:16:50.750291+0000","last_peered":"2023-10-08T00:16:50.750291+0000","last_clean":"2023-10-08T00:16:50.750291+0000","last_became_active":"2023-10-08T00:16:49.717794+0000","last_became_peered":"2023-10-08T00:16:49.717794+0000","last_unstale":"2023-10-08T00:16:50.750291+0000","last_undegraded":"2023-10-08T00:16:50.750291+0000","last_fullsized":"2023-10-08T00:16:50.750291+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:56:16.239039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.737577+0000","last_change":"2023-10-08T00:16:50.737418+0000","last_active":"2023-10-08T00:16:50.737577+0000","last_peered":"2023-10-08T00:16:50.737577+0000","last_clean":"2023-10-08T00:16:50.737577+0000","last_became_active":"2023-10-08T00:16:50.737186+0000","last_became_peered":"2023-10-08T00:16:50.737186+0000","last_unstale":"2023-10-08T00:16:50.737577+0000","last_undegraded":"2023-10-08T00:16:50.737577+0000","last_fullsized":"2023-10-08T00:16:50.737577+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:21:33.681095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.728407+0000","last_change":"2023-10-08T00:16:49.724096+0000","last_active":"2023-10-08T00:16:50.728407+0000","last_peered":"2023-10-08T00:16:50.728407+0000","last_clean":"2023-10-08T00:16:50.728407+0000","last_became_active":"2023-10-08T00:16:49.720406+0000","last_became_peered":"2023-10-08T00:16:49.720406+0000","last_unstale":"2023-10-08T00:16:50.728407+0000","last_undegraded":"2023-10-08T00:16:50.728407+0000","last_fullsized":"2023-10-08T00:16:50.728407+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:41:41.931737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.734543+0000","last_change":"2023-10-08T00:16:50.734288+0000","last_active":"2023-10-08T00:16:50.734543+0000","last_peered":"2023-10-08T00:16:50.734543+0000","last_clean":"2023-10-08T00:16:50.734543+0000","last_became_active":"2023-10-08T00:16:50.733473+0000","last_became_peered":"2023-10-08T00:16:50.733473+0000","last_unstale":"2023-10-08T00:16:50.734543+0000","last_undegraded":"2023-10-08T00:16:50.734543+0000","last_fullsized":"2023-10-08T00:16:50.734543+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:19:59.641079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"138'8","reported_seq":18,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.750685+0000","last_change":"2023-10-08T00:16:49.725499+0000","last_active":"2023-10-08T00:16:50.750685+0000","last_peered":"2023-10-08T00:16:50.750685+0000","last_clean":"2023-10-08T00:16:50.750685+0000","last_became_active":"2023-10-08T00:16:49.725129+0000","last_became_peered":"2023-10-08T00:16:49.725129+0000","last_unstale":"2023-10-08T00:16:50.750685+0000","last_undegraded":"2023-10-08T00:16:50.750685+0000","last_fullsized":"2023-10-08T00:16:50.750685+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:43:13.170777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.739803+0000","last_change":"2023-10-08T00:16:50.739723+0000","last_active":"2023-10-08T00:16:50.739803+0000","last_peered":"2023-10-08T00:16:50.739803+0000","last_clean":"2023-10-08T00:16:50.739803+0000","last_became_active":"2023-10-08T00:16:50.739510+0000","last_became_peered":"2023-10-08T00:16:50.739510+0000","last_unstale":"2023-10-08T00:16:50.739803+0000","last_undegraded":"2023-10-08T00:16:50.739803+0000","last_fullsized":"2023-10-08T00:16:50.739803+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:51:20.090559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.f","version":"138'1","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.746397+0000","last_change":"2023-10-08T00:16:49.714785+0000","last_active":"2023-10-08T00:16:50.746397+0000","last_peered":"2023-10-08T00:16:50.746397+0000","last_clean":"2023-10-08T00:16:50.746397+0000","last_became_active":"2023-10-08T00:16:49.714402+0000","last_became_peered":"2023-10-08T00:16:49.714402+0000","last_unstale":"2023-10-08T00:16:50.746397+0000","last_undegraded":"2023-10-08T00:16:50.746397+0000","last_fullsized":"2023-10-08T00:16:50.746397+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:41:40.129734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.737517+0000","last_change":"2023-10-08T00:16:50.737430+0000","last_active":"2023-10-08T00:16:50.737517+0000","last_peered":"2023-10-08T00:16:50.737517+0000","last_clean":"2023-10-08T00:16:50.737517+0000","last_became_active":"2023-10-08T00:16:50.737251+0000","last_became_peered":"2023-10-08T00:16:50.737251+0000","last_unstale":"2023-10-08T00:16:50.737517+0000","last_undegraded":"2023-10-08T00:16:50.737517+0000","last_fullsized":"2023-10-08T00:16:50.737517+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:55:58.689766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"138'2","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.750059+0000","last_change":"2023-10-08T00:16:49.717409+0000","last_active":"2023-10-08T00:16:50.750059+0000","last_peered":"2023-10-08T00:16:50.750059+0000","last_clean":"2023-10-08T00:16:50.750059+0000","last_became_active":"2023-10-08T00:16:49.717194+0000","last_became_peered":"2023-10-08T00:16:49.717194+0000","last_unstale":"2023-10-08T00:16:50.750059+0000","last_undegraded":"2023-10-08T00:16:50.750059+0000","last_fullsized":"2023-10-08T00:16:50.750059+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:41:42.950196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.734513+0000","last_change":"2023-10-08T00:16:50.734437+0000","last_active":"2023-10-08T00:16:50.734513+0000","last_peered":"2023-10-08T00:16:50.734513+0000","last_clean":"2023-10-08T00:16:50.734513+0000","last_became_active":"2023-10-08T00:16:50.734224+0000","last_became_peered":"2023-10-08T00:16:50.734224+0000","last_unstale":"2023-10-08T00:16:50.734513+0000","last_undegraded":"2023-10-08T00:16:50.734513+0000","last_fullsized":"2023-10-08T00:16:50.734513+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:52:51.647697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.d","version":"138'1","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.744283+0000","last_change":"2023-10-08T00:16:49.720369+0000","last_active":"2023-10-08T00:16:50.744283+0000","last_peered":"2023-10-08T00:16:50.744283+0000","last_clean":"2023-10-08T00:16:50.744283+0000","last_became_active":"2023-10-08T00:16:49.720143+0000","last_became_peered":"2023-10-08T00:16:49.720143+0000","last_unstale":"2023-10-08T00:16:50.744283+0000","last_undegraded":"2023-10-08T00:16:50.744283+0000","last_fullsized":"2023-10-08T00:16:50.744283+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:17:20.032034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.736803+0000","last_change":"2023-10-08T00:16:50.736753+0000","last_active":"2023-10-08T00:16:50.736803+0000","last_peered":"2023-10-08T00:16:50.736803+0000","last_clean":"2023-10-08T00:16:50.736803+0000","last_became_active":"2023-10-08T00:16:50.736529+0000","last_became_peered":"2023-10-08T00:16:50.736529+0000","last_unstale":"2023-10-08T00:16:50.736803+0000","last_undegraded":"2023-10-08T00:16:50.736803+0000","last_fullsized":"2023-10-08T00:16:50.736803+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:53:27.357114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.736560+0000","last_change":"2023-10-08T00:16:49.723203+0000","last_active":"2023-10-08T00:16:50.736560+0000","last_peered":"2023-10-08T00:16:50.736560+0000","last_clean":"2023-10-08T00:16:50.736560+0000","last_became_active":"2023-10-08T00:16:49.722933+0000","last_became_peered":"2023-10-08T00:16:49.722933+0000","last_unstale":"2023-10-08T00:16:50.736560+0000","last_undegraded":"2023-10-08T00:16:50.736560+0000","last_fullsized":"2023-10-08T00:16:50.736560+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:53:18.346097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.730742+0000","last_change":"2023-10-08T00:16:50.730670+0000","last_active":"2023-10-08T00:16:50.730742+0000","last_peered":"2023-10-08T00:16:50.730742+0000","last_clean":"2023-10-08T00:16:50.730742+0000","last_became_active":"2023-10-08T00:16:50.730428+0000","last_became_peered":"2023-10-08T00:16:50.730428+0000","last_unstale":"2023-10-08T00:16:50.730742+0000","last_undegraded":"2023-10-08T00:16:50.730742+0000","last_fullsized":"2023-10-08T00:16:50.730742+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:21:28.306676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"138'2","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.748614+0000","last_change":"2023-10-08T00:16:49.725381+0000","last_active":"2023-10-08T00:16:50.748614+0000","last_peered":"2023-10-08T00:16:50.748614+0000","last_clean":"2023-10-08T00:16:50.748614+0000","last_became_active":"2023-10-08T00:16:49.725123+0000","last_became_peered":"2023-10-08T00:16:49.725123+0000","last_unstale":"2023-10-08T00:16:50.748614+0000","last_undegraded":"2023-10-08T00:16:50.748614+0000","last_fullsized":"2023-10-08T00:16:50.748614+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:00:17.187757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.737498+0000","last_change":"2023-10-08T00:16:50.737046+0000","last_active":"2023-10-08T00:16:50.737498+0000","last_peered":"2023-10-08T00:16:50.737498+0000","last_clean":"2023-10-08T00:16:50.737498+0000","last_became_active":"2023-10-08T00:16:50.736660+0000","last_became_peered":"2023-10-08T00:16:50.736660+0000","last_unstale":"2023-10-08T00:16:50.737498+0000","last_undegraded":"2023-10-08T00:16:50.737498+0000","last_fullsized":"2023-10-08T00:16:50.737498+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:34:23.975157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.734442+0000","last_change":"2023-10-08T00:16:49.721861+0000","last_active":"2023-10-08T00:16:50.734442+0000","last_peered":"2023-10-08T00:16:50.734442+0000","last_clean":"2023-10-08T00:16:50.734442+0000","last_became_active":"2023-10-08T00:16:49.721622+0000","last_became_peered":"2023-10-08T00:16:49.721622+0000","last_unstale":"2023-10-08T00:16:50.734442+0000","last_undegraded":"2023-10-08T00:16:50.734442+0000","last_fullsized":"2023-10-08T00:16:50.734442+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:19:58.636102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.738658+0000","last_change":"2023-10-08T00:16:50.738603+0000","last_active":"2023-10-08T00:16:50.738658+0000","last_peered":"2023-10-08T00:16:50.738658+0000","last_clean":"2023-10-08T00:16:50.738658+0000","last_became_active":"2023-10-08T00:16:50.738389+0000","last_became_peered":"2023-10-08T00:16:50.738389+0000","last_unstale":"2023-10-08T00:16:50.738658+0000","last_undegraded":"2023-10-08T00:16:50.738658+0000","last_fullsized":"2023-10-08T00:16:50.738658+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:09:43.382366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.733900+0000","last_change":"2023-10-08T00:16:49.723222+0000","last_active":"2023-10-08T00:16:50.733900+0000","last_peered":"2023-10-08T00:16:50.733900+0000","last_clean":"2023-10-08T00:16:50.733900+0000","last_became_active":"2023-10-08T00:16:49.722490+0000","last_became_peered":"2023-10-08T00:16:49.722490+0000","last_unstale":"2023-10-08T00:16:50.733900+0000","last_undegraded":"2023-10-08T00:16:50.733900+0000","last_fullsized":"2023-10-08T00:16:50.733900+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:38:37.189574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.748486+0000","last_change":"2023-10-08T00:16:50.746715+0000","last_active":"2023-10-08T00:16:50.748486+0000","last_peered":"2023-10-08T00:16:50.748486+0000","last_clean":"2023-10-08T00:16:50.748486+0000","last_became_active":"2023-10-08T00:16:50.746323+0000","last_became_peered":"2023-10-08T00:16:50.746323+0000","last_unstale":"2023-10-08T00:16:50.748486+0000","last_undegraded":"2023-10-08T00:16:50.748486+0000","last_fullsized":"2023-10-08T00:16:50.748486+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:43:29.947771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.734999+0000","last_change":"2023-10-08T00:16:49.724702+0000","last_active":"2023-10-08T00:16:50.734999+0000","last_peered":"2023-10-08T00:16:50.734999+0000","last_clean":"2023-10-08T00:16:50.734999+0000","last_became_active":"2023-10-08T00:16:49.724254+0000","last_became_peered":"2023-10-08T00:16:49.724254+0000","last_unstale":"2023-10-08T00:16:50.734999+0000","last_undegraded":"2023-10-08T00:16:50.734999+0000","last_fullsized":"2023-10-08T00:16:50.734999+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:09:50.287365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.737968+0000","last_change":"2023-10-08T00:16:50.737903+0000","last_active":"2023-10-08T00:16:50.737968+0000","last_peered":"2023-10-08T00:16:50.737968+0000","last_clean":"2023-10-08T00:16:50.737968+0000","last_became_active":"2023-10-08T00:16:50.737683+0000","last_became_peered":"2023-10-08T00:16:50.737683+0000","last_unstale":"2023-10-08T00:16:50.737968+0000","last_undegraded":"2023-10-08T00:16:50.737968+0000","last_fullsized":"2023-10-08T00:16:50.737968+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:55:58.689766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"138'2","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.750614+0000","last_change":"2023-10-08T00:16:49.722975+0000","last_active":"2023-10-08T00:16:50.750614+0000","last_peered":"2023-10-08T00:16:50.750614+0000","last_clean":"2023-10-08T00:16:50.750614+0000","last_became_active":"2023-10-08T00:16:49.722744+0000","last_became_peered":"2023-10-08T00:16:49.722744+0000","last_unstale":"2023-10-08T00:16:50.750614+0000","last_undegraded":"2023-10-08T00:16:50.750614+0000","last_fullsized":"2023-10-08T00:16:50.750614+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:29:25.662070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.731662+0000","last_change":"2023-10-08T00:16:50.730913+0000","last_active":"2023-10-08T00:16:50.731662+0000","last_peered":"2023-10-08T00:16:50.731662+0000","last_clean":"2023-10-08T00:16:50.731662+0000","last_became_active":"2023-10-08T00:16:50.730693+0000","last_became_peered":"2023-10-08T00:16:50.730693+0000","last_unstale":"2023-10-08T00:16:50.731662+0000","last_undegraded":"2023-10-08T00:16:50.731662+0000","last_fullsized":"2023-10-08T00:16:50.731662+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:52:10.559787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.731535+0000","last_change":"2023-10-08T00:16:49.722398+0000","last_active":"2023-10-08T00:16:50.731535+0000","last_peered":"2023-10-08T00:16:50.731535+0000","last_clean":"2023-10-08T00:16:50.731535+0000","last_became_active":"2023-10-08T00:16:49.721743+0000","last_became_peered":"2023-10-08T00:16:49.721743+0000","last_unstale":"2023-10-08T00:16:50.731535+0000","last_undegraded":"2023-10-08T00:16:50.731535+0000","last_fullsized":"2023-10-08T00:16:50.731535+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:33:08.393982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.746115+0000","last_change":"2023-10-08T00:16:50.745889+0000","last_active":"2023-10-08T00:16:50.746115+0000","last_peered":"2023-10-08T00:16:50.746115+0000","last_clean":"2023-10-08T00:16:50.746115+0000","last_became_active":"2023-10-08T00:16:50.745332+0000","last_became_peered":"2023-10-08T00:16:50.745332+0000","last_unstale":"2023-10-08T00:16:50.746115+0000","last_undegraded":"2023-10-08T00:16:50.746115+0000","last_fullsized":"2023-10-08T00:16:50.746115+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:45:12.144679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.733802+0000","last_change":"2023-10-08T00:16:49.723551+0000","last_active":"2023-10-08T00:16:50.733802+0000","last_peered":"2023-10-08T00:16:50.733802+0000","last_clean":"2023-10-08T00:16:50.733802+0000","last_became_active":"2023-10-08T00:16:49.723120+0000","last_became_peered":"2023-10-08T00:16:49.723120+0000","last_unstale":"2023-10-08T00:16:50.733802+0000","last_undegraded":"2023-10-08T00:16:50.733802+0000","last_fullsized":"2023-10-08T00:16:50.733802+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:56:16.239039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.737026+0000","last_change":"2023-10-08T00:16:50.736952+0000","last_active":"2023-10-08T00:16:50.737026+0000","last_peered":"2023-10-08T00:16:50.737026+0000","last_clean":"2023-10-08T00:16:50.737026+0000","last_became_active":"2023-10-08T00:16:50.736785+0000","last_became_peered":"2023-10-08T00:16:50.736785+0000","last_unstale":"2023-10-08T00:16:50.737026+0000","last_undegraded":"2023-10-08T00:16:50.737026+0000","last_fullsized":"2023-10-08T00:16:50.737026+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:51:20.090559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.733998+0000","last_change":"2023-10-08T00:16:49.722921+0000","last_active":"2023-10-08T00:16:50.733998+0000","last_peered":"2023-10-08T00:16:50.733998+0000","last_clean":"2023-10-08T00:16:50.733998+0000","last_became_active":"2023-10-08T00:16:49.722188+0000","last_became_peered":"2023-10-08T00:16:49.722188+0000","last_unstale":"2023-10-08T00:16:50.733998+0000","last_undegraded":"2023-10-08T00:16:50.733998+0000","last_fullsized":"2023-10-08T00:16:50.733998+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:19:03.121126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.734768+0000","last_change":"2023-10-08T00:16:50.734696+0000","last_active":"2023-10-08T00:16:50.734768+0000","last_peered":"2023-10-08T00:16:50.734768+0000","last_clean":"2023-10-08T00:16:50.734768+0000","last_became_active":"2023-10-08T00:16:50.734460+0000","last_became_peered":"2023-10-08T00:16:50.734460+0000","last_unstale":"2023-10-08T00:16:50.734768+0000","last_undegraded":"2023-10-08T00:16:50.734768+0000","last_fullsized":"2023-10-08T00:16:50.734768+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:24:16.762086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.731894+0000","last_change":"2023-10-08T00:16:49.716168+0000","last_active":"2023-10-08T00:16:50.731894+0000","last_peered":"2023-10-08T00:16:50.731894+0000","last_clean":"2023-10-08T00:16:50.731894+0000","last_became_active":"2023-10-08T00:16:49.715935+0000","last_became_peered":"2023-10-08T00:16:49.715935+0000","last_unstale":"2023-10-08T00:16:50.731894+0000","last_undegraded":"2023-10-08T00:16:50.731894+0000","last_fullsized":"2023-10-08T00:16:50.731894+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:08:16.260647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.741800+0000","last_change":"2023-10-08T00:16:50.741598+0000","last_active":"2023-10-08T00:16:50.741800+0000","last_peered":"2023-10-08T00:16:50.741800+0000","last_clean":"2023-10-08T00:16:50.741800+0000","last_became_active":"2023-10-08T00:16:50.741387+0000","last_became_peered":"2023-10-08T00:16:50.741387+0000","last_unstale":"2023-10-08T00:16:50.741800+0000","last_undegraded":"2023-10-08T00:16:50.741800+0000","last_fullsized":"2023-10-08T00:16:50.741800+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:45:12.144679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.736649+0000","last_change":"2023-10-08T00:16:49.721058+0000","last_active":"2023-10-08T00:16:50.736649+0000","last_peered":"2023-10-08T00:16:50.736649+0000","last_clean":"2023-10-08T00:16:50.736649+0000","last_became_active":"2023-10-08T00:16:49.720822+0000","last_became_peered":"2023-10-08T00:16:49.720822+0000","last_unstale":"2023-10-08T00:16:50.736649+0000","last_undegraded":"2023-10-08T00:16:50.736649+0000","last_fullsized":"2023-10-08T00:16:50.736649+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:49:07.445902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.740418+0000","last_change":"2023-10-08T00:16:50.740365+0000","last_active":"2023-10-08T00:16:50.740418+0000","last_peered":"2023-10-08T00:16:50.740418+0000","last_clean":"2023-10-08T00:16:50.740418+0000","last_became_active":"2023-10-08T00:16:50.740176+0000","last_became_peered":"2023-10-08T00:16:50.740176+0000","last_unstale":"2023-10-08T00:16:50.740418+0000","last_undegraded":"2023-10-08T00:16:50.740418+0000","last_fullsized":"2023-10-08T00:16:50.740418+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:52:10.559787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.735646+0000","last_change":"2023-10-08T00:16:49.725348+0000","last_active":"2023-10-08T00:16:50.735646+0000","last_peered":"2023-10-08T00:16:50.735646+0000","last_clean":"2023-10-08T00:16:50.735646+0000","last_became_active":"2023-10-08T00:16:49.724834+0000","last_became_peered":"2023-10-08T00:16:49.724834+0000","last_unstale":"2023-10-08T00:16:50.735646+0000","last_undegraded":"2023-10-08T00:16:50.735646+0000","last_fullsized":"2023-10-08T00:16:50.735646+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:53:18.346097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.737917+0000","last_change":"2023-10-08T00:16:50.737000+0000","last_active":"2023-10-08T00:16:50.737917+0000","last_peered":"2023-10-08T00:16:50.737917+0000","last_clean":"2023-10-08T00:16:50.737917+0000","last_became_active":"2023-10-08T00:16:50.736826+0000","last_became_peered":"2023-10-08T00:16:50.736826+0000","last_unstale":"2023-10-08T00:16:50.737917+0000","last_undegraded":"2023-10-08T00:16:50.737917+0000","last_fullsized":"2023-10-08T00:16:50.737917+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:53:27.357114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.728767+0000","last_change":"2023-10-08T00:16:49.714631+0000","last_active":"2023-10-08T00:16:50.728767+0000","last_peered":"2023-10-08T00:16:50.728767+0000","last_clean":"2023-10-08T00:16:50.728767+0000","last_became_active":"2023-10-08T00:16:49.714103+0000","last_became_peered":"2023-10-08T00:16:49.714103+0000","last_unstale":"2023-10-08T00:16:50.728767+0000","last_undegraded":"2023-10-08T00:16:50.728767+0000","last_fullsized":"2023-10-08T00:16:50.728767+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:21:32.676119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.734141+0000","last_change":"2023-10-08T00:16:50.734075+0000","last_active":"2023-10-08T00:16:50.734141+0000","last_peered":"2023-10-08T00:16:50.734141+0000","last_clean":"2023-10-08T00:16:50.734141+0000","last_became_active":"2023-10-08T00:16:50.733829+0000","last_became_peered":"2023-10-08T00:16:50.733829+0000","last_unstale":"2023-10-08T00:16:50.734141+0000","last_undegraded":"2023-10-08T00:16:50.734141+0000","last_fullsized":"2023-10-08T00:16:50.734141+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:24:49.153516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.727783+0000","last_change":"2023-10-08T00:16:49.715211+0000","last_active":"2023-10-08T00:16:50.727783+0000","last_peered":"2023-10-08T00:16:50.727783+0000","last_clean":"2023-10-08T00:16:50.727783+0000","last_became_active":"2023-10-08T00:16:49.714937+0000","last_became_peered":"2023-10-08T00:16:49.714937+0000","last_unstale":"2023-10-08T00:16:50.727783+0000","last_undegraded":"2023-10-08T00:16:50.727783+0000","last_fullsized":"2023-10-08T00:16:50.727783+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:16:46.900489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":264,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.728981+0000","last_change":"2023-10-08T00:10:43.728490+0000","last_active":"2023-10-08T00:16:50.728981+0000","last_peered":"2023-10-08T00:16:50.728981+0000","last_clean":"2023-10-08T00:16:50.728981+0000","last_became_active":"2023-10-08T00:10:43.728341+0000","last_became_peered":"2023-10-08T00:10:43.728341+0000","last_unstale":"2023-10-08T00:16:50.728981+0000","last_undegraded":"2023-10-08T00:16:50.728981+0000","last_fullsized":"2023-10-08T00:16:50.728981+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_clean_scrub_stamp":"2023-10-08T00:10:42.716096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:06:31.035293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":754,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194558,"statfs":{"total":200000000,"available":199227752,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.53300000000000003,"15min":0.50700000000000001},"min":{"1min":0.39700000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.54300000000000004,"15min":0.54500000000000004},"min":{"1min":0.36099999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.5}]},{"osd":1,"last update":"Sun Oct 8 00:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56100000000000005,"15min":0.53700000000000003},"min":{"1min":0.38,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.72799999999999998,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.58699999999999997,"15min":0.56699999999999995},"min":{"1min":0.42399999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.78400000000000003,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.57099999999999995}]},{"osd":2,"last update":"Sun Oct 8 00:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55100000000000005,"15min":0.54000000000000004},"min":{"1min":0.34599999999999997,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.55200000000000005,"15min":0.52600000000000002},"min":{"1min":0.433,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77200000000000002,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.52200000000000002}]},{"osd":3,"last update":"Sun Oct 8 00:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54500000000000004,"15min":0.52300000000000002},"min":{"1min":0.35599999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.75600000000000001,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56899999999999995,"15min":0.55900000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.46999999999999997}]},{"osd":4,"last update":"Sun Oct 8 00:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.497,"15min":0.51700000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.65100000000000002,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.5,"5min":0.504,"15min":0.52400000000000002},"min":{"1min":0.315,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.75900000000000001,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.60799999999999998}]},{"osd":5,"last update":"Sun Oct 8 00:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.51500000000000001,"15min":0.54500000000000004},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.78000000000000003,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54300000000000004,"15min":0.57999999999999996},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.76900000000000002,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.65900000000000003}]},{"osd":6,"last update":"Sun Oct 8 00:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.57799999999999996,"15min":0.59399999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.74399999999999999,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58899999999999997,"15min":0.625},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.80100000000000005,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.63600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574924,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":746,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194566,"statfs":{"total":200000000,"available":199235700,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.51500000000000001,"15min":0.48699999999999999},"min":{"1min":0.39400000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.71199999999999997,"5min":1.012,"15min":1.012},"last":0.495},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.56999999999999995,"15min":0.66700000000000004},"min":{"1min":0.39900000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.0529999999999999,"5min":5.1230000000000002,"15min":5.1230000000000002},"last":0.45600000000000002}]},{"osd":1,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55200000000000005,"15min":0.52400000000000002},"min":{"1min":0.34799999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.78100000000000003,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57599999999999996,"15min":0.56599999999999995},"min":{"1min":0.42499999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.98999999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.48199999999999998}]},{"osd":2,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.54700000000000004,"15min":0.53200000000000003},"min":{"1min":0.439,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.753,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.39700000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.76900000000000002,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.39200000000000002}]},{"osd":3,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57299999999999995,"15min":0.54300000000000004},"min":{"1min":0.36699999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.84199999999999997,"5min":1.046,"15min":1.046},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56899999999999995,"15min":0.54500000000000004},"min":{"1min":0.39800000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.81000000000000005,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.442}]},{"osd":4,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.47299999999999998,"15min":0.496},"min":{"1min":0.371,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.55200000000000005,"15min":0.56299999999999994},"min":{"1min":0.38100000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.55200000000000005}]},{"osd":5,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.48299999999999998,"15min":0.48699999999999999},"min":{"1min":0.34499999999999997,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.69499999999999995,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.52300000000000002,"15min":0.55200000000000005},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.42599999999999999}]},{"osd":7,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.52700000000000002,"15min":0.54000000000000004},"min":{"1min":0.31,"5min":0.307,"15min":0.307},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.54500000000000004,"15min":0.54800000000000004},"min":{"1min":0.40200000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76000000000000001,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.50600000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1322,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193989,"statfs":{"total":200000000,"available":198645332,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:16:51 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.34200000000000003,"15min":0.312},"min":{"1min":0.215,"5min":0.20300000000000001,"15min":0.19500000000000001},"max":{"1min":0.55900000000000005,"5min":0.56899999999999995,"15min":0.56899999999999995},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.375,"5min":0.39400000000000002,"15min":0.379},"min":{"1min":0.29699999999999999,"5min":0.255,"15min":0.255},"max":{"1min":0.54600000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.316}]},{"osd":2,"last update":"Sun Oct 8 00:16:51 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.35199999999999998,"15min":0.33800000000000002},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.53200000000000003,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.38200000000000001,"15min":0.34599999999999997},"min":{"1min":0.26500000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.57399999999999995,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.35899999999999999}]},{"osd":3,"last update":"Sun Oct 8 00:16:51 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.39200000000000002,"15min":0.373},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.56999999999999995,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.35399999999999998,"15min":0.34399999999999997},"min":{"1min":0.255,"5min":0.222,"15min":0.222},"max":{"1min":0.502,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.32900000000000001}]},{"osd":4,"last update":"Sun Oct 8 00:16:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.56000000000000005,"15min":0.54200000000000004},"min":{"1min":0.34000000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77100000000000002,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.443},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.57799999999999996,"15min":0.56799999999999995},"min":{"1min":0.40899999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.754,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.46400000000000002}]},{"osd":5,"last update":"Sun Oct 8 00:16:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57099999999999995,"15min":0.54200000000000004},"min":{"1min":0.372,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.81000000000000005,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.58799999999999997,"15min":0.57599999999999996},"min":{"1min":0.41999999999999998,"5min":0.38300000000000001,"15min":0.30399999999999999},"max":{"1min":0.84999999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.47599999999999998}]},{"osd":6,"last update":"Sun Oct 8 00:16:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58499999999999996,"15min":0.56899999999999995},"min":{"1min":0.379,"5min":0.317,"15min":0.317},"max":{"1min":0.80400000000000005,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.434},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.58099999999999996,"15min":0.56399999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.80500000000000005,"5min":0.83899999999999997,"15min":0.84999999999999998},"last":0.38600000000000001}]},{"osd":7,"last update":"Sun Oct 8 00:16:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.58399999999999996,"15min":0.56599999999999995},"min":{"1min":0.40100000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.79600000000000004,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.61799999999999999,"15min":0.59399999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81899999999999995,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.52000000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574924,"num_pgs":14,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":746,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194565,"statfs":{"total":200000000,"available":199235076,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 00:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.34100000000000003,"15min":0.33200000000000002},"min":{"1min":0.20699999999999999,"5min":0.193,"15min":0.193},"max":{"1min":0.57599999999999996,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.252},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.35599999999999998,"15min":0.34799999999999998},"min":{"1min":0.218,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.57199999999999995,"5min":0.87,"15min":0.87},"last":0.22600000000000001}]},{"osd":2,"last update":"Sun Oct 8 00:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.38900000000000001,"15min":0.38800000000000001},"min":{"1min":0.27600000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.623,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.379,"5min":0.46400000000000002,"15min":0.72799999999999998},"min":{"1min":0.245,"5min":0.185,"15min":0.185},"max":{"1min":0.65600000000000003,"5min":12.019,"15min":12.019},"last":0.28599999999999998}]},{"osd":3,"last update":"Sun Oct 8 00:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.36799999999999999,"15min":0.36399999999999999},"min":{"1min":0.27500000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.54300000000000004,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.41399999999999998,"15min":0.39900000000000002},"min":{"1min":0.29999999999999999,"5min":0.246,"15min":0.246},"max":{"1min":0.67400000000000004,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.33300000000000002}]},{"osd":4,"last update":"Sun Oct 8 00:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61399999999999999,"15min":0.61699999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.90300000000000002,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.626,"15min":0.626},"min":{"1min":0.439,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.873,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.63700000000000001}]},{"osd":5,"last update":"Sun Oct 8 00:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.69099999999999995,"15min":0.96699999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.751,"5min":11.992000000000001,"15min":11.992000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60099999999999998,"15min":0.58899999999999997},"min":{"1min":0.46800000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.85599999999999998,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.64600000000000002}]},{"osd":6,"last update":"Sun Oct 8 00:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.59899999999999998,"15min":0.58899999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.88100000000000001,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.69599999999999995,"15min":0.94799999999999995},"min":{"1min":0.41299999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84499999999999997,"5min":11.927,"15min":11.927},"last":0.55400000000000005}]},{"osd":7,"last update":"Sun Oct 8 00:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.70799999999999996,"15min":0.96899999999999997},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.84099999999999997,"5min":11.965,"15min":11.965},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.623,"5min":0.626,"15min":0.624},"min":{"1min":0.42699999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81100000000000005,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.59499999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574924,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":753,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194559,"statfs":{"total":200000000,"available":199228802,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.33600000000000002,"15min":0.38400000000000001},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.53400000000000003,"5min":0.72399999999999998,"15min":2.5139999999999998},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.34100000000000003,"15min":0.39400000000000002},"min":{"1min":0.22700000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.55600000000000005,"5min":0.56399999999999995,"15min":2.1309999999999998},"last":0.26000000000000001}]},{"osd":1,"last update":"Sun Oct 8 00:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.34899999999999998,"15min":0.38800000000000001},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":2.2639999999999998},"last":0.23000000000000001},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.36799999999999999,"15min":0.42699999999999999},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":2.5379999999999998},"last":0.32300000000000001}]},{"osd":3,"last update":"Sun Oct 8 00:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.38400000000000001,"15min":0.42399999999999999},"min":{"1min":0.28100000000000003,"5min":0.254,"15min":0.254},"max":{"1min":0.61599999999999999,"5min":0.61899999999999999,"15min":2.2890000000000001},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40100000000000002,"15min":0.45700000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":2.5619999999999998},"last":0.40000000000000002}]},{"osd":4,"last update":"Sun Oct 8 00:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56200000000000006,"15min":0.60999999999999999},"min":{"1min":0.38200000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.76600000000000001,"5min":1.073,"15min":2.488},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58799999999999997,"15min":0.60499999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.85899999999999999,"5min":0.93600000000000005,"15min":2.073},"last":0.42099999999999999}]},{"osd":5,"last update":"Sun Oct 8 00:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.60899999999999999,"15min":0.65100000000000002},"min":{"1min":0.374,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.82199999999999995,"5min":1.2130000000000001,"15min":2.2069999999999999},"last":0.435},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.629,"15min":0.65900000000000003},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.39400000000000002},"max":{"1min":0.94199999999999995,"5min":1.0369999999999999,"15min":2.456},"last":0.54600000000000004}]},{"osd":6,"last update":"Sun Oct 8 00:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59599999999999997,"15min":0.61899999999999999},"min":{"1min":0.39200000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.93000000000000005,"5min":1.05,"15min":2.0339999999999998},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60599999999999998,"15min":0.61299999999999999},"min":{"1min":0.439,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.873,"5min":1.0940000000000001,"15min":2.1699999999999999},"last":0.497}]},{"osd":7,"last update":"Sun Oct 8 00:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.61199999999999999,"15min":0.63300000000000001},"min":{"1min":0.42999999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.95999999999999996,"5min":1.0229999999999999,"15min":2.1030000000000002},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60999999999999999,"15min":0.627},"min":{"1min":0.443,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.90500000000000003,"5min":1.0569999999999999,"15min":2.2349999999999999},"last":0.56299999999999994}]}]},{"osd":3,"up_from":13,"seq":55834574924,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":746,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194566,"statfs":{"total":200000000,"available":199235700,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:16:49 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.32000000000000001,"15min":0.36399999999999999},"min":{"1min":0.218,"5min":0.192,"15min":0.192},"max":{"1min":0.55200000000000005,"5min":0.71199999999999997,"15min":1.46},"last":0.218},{"interface":"front","average":{"1min":0.32000000000000001,"5min":0.33200000000000002,"15min":0.34000000000000002},"min":{"1min":0.26500000000000001,"5min":0.192,"15min":0.192},"max":{"1min":0.42399999999999999,"5min":0.504,"15min":0.54800000000000004},"last":0.314}]},{"osd":1,"last update":"Sun Oct 8 00:16:49 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.34200000000000003,"15min":0.754},"min":{"1min":0.23699999999999999,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.46200000000000002,"5min":0.57599999999999996,"15min":12.351000000000001},"last":0.247},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.36399999999999999,"15min":0.36499999999999999},"min":{"1min":0.248,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.54200000000000004,"5min":0.60199999999999998,"15min":0.71999999999999997},"last":0.34599999999999997}]},{"osd":2,"last update":"Sun Oct 8 00:16:49 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.38100000000000001,"15min":0.41399999999999998},"min":{"1min":0.30099999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.48999999999999999,"5min":0.70499999999999996,"15min":1.5489999999999999},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.37,"5min":0.39000000000000001,"15min":0.38700000000000001},"min":{"1min":0.28299999999999997,"5min":0.26800000000000002,"15min":0.246},"max":{"1min":0.45500000000000002,"5min":0.64700000000000002,"15min":0.753},"last":0.30099999999999999}]},{"osd":4,"last update":"Sun Oct 8 00:16:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59999999999999998,"15min":0.95099999999999996},"min":{"1min":0.39600000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.80000000000000004,"5min":1.024,"15min":11.333},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61599999999999999,"15min":0.97999999999999998},"min":{"1min":0.51000000000000001,"5min":0.376,"15min":0.36099999999999999},"max":{"1min":0.77800000000000002,"5min":0.94899999999999995,"15min":12.321999999999999},"last":0.54400000000000004}]},{"osd":5,"last update":"Sun Oct 8 00:16:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60199999999999998,"15min":0.98799999999999999},"min":{"1min":0.41099999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.73499999999999999,"5min":0.92000000000000004,"15min":12.291},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.622,"15min":0.60499999999999998},"min":{"1min":0.45600000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.73199999999999998,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.59599999999999997}]},{"osd":6,"last update":"Sun Oct 8 00:16:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58099999999999996,"15min":0.91800000000000004},"min":{"1min":0.442,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.83599999999999997,"5min":0.96199999999999997,"15min":11.288},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.59199999999999997,"15min":0.96999999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.73599999999999999,"5min":1.002,"15min":12.253},"last":0.56100000000000005}]},{"osd":7,"last update":"Sun Oct 8 00:16:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.60899999999999999,"15min":0.58199999999999996},"min":{"1min":0.47699999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80800000000000005,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.61199999999999999,"15min":0.95299999999999996},"min":{"1min":0.46200000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82599999999999996,"5min":1.381,"15min":11.363},"last":0.46999999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574924,"num_pgs":33,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":754,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194558,"statfs":{"total":200000000,"available":199227786,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.56899999999999995,"15min":0.55700000000000005},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.78100000000000003,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57399999999999995,"15min":0.58399999999999996},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.72899999999999998,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.58499999999999996}]},{"osd":1,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.52500000000000002,"15min":0.52100000000000002},"min":{"1min":0.33900000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.53100000000000003,"15min":0.53400000000000003},"min":{"1min":0.314,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.80400000000000005,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.44}]},{"osd":2,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.51500000000000001,"15min":0.51900000000000002},"min":{"1min":0.41699999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.68700000000000006,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.52400000000000002,"15min":0.51300000000000001},"min":{"1min":0.374,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.42399999999999999}]},{"osd":3,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.53700000000000003,"15min":0.54000000000000004},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.79800000000000004,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.53500000000000003,"15min":0.53400000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.45500000000000002}]},{"osd":5,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.46600000000000003,"15min":0.49399999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.499},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.495,"15min":0.51000000000000001},"min":{"1min":0.26400000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.48899999999999999}]},{"osd":6,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.47199999999999998,"15min":0.48799999999999999},"min":{"1min":0.32200000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.65000000000000002,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.371},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52200000000000002,"15min":0.55200000000000005},"min":{"1min":0.35399999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.55200000000000005}]},{"osd":7,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.53400000000000003,"15min":0.55100000000000005},"min":{"1min":0.35399999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.79700000000000004,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.56899999999999995,"15min":0.57999999999999996},"min":{"1min":0.45500000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.77500000000000002,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.52700000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574924,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1329,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193983,"statfs":{"total":200000000,"available":198638952,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48599999999999999},"min":{"1min":0.28799999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.66700000000000004,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53800000000000003,"15min":0.55700000000000005},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.70999999999999996,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.44400000000000001}]},{"osd":1,"last update":"Sun Oct 8 00:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.39000000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.71799999999999997,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50700000000000001,"15min":0.52300000000000002},"min":{"1min":0.33100000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70699999999999996,"5min":0.755,"15min":0.755},"last":0.48799999999999999}]},{"osd":2,"last update":"Sun Oct 8 00:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.52300000000000002,"15min":0.52900000000000003},"min":{"1min":0.39500000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.64700000000000002,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.376},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.55000000000000004,"15min":0.56699999999999995},"min":{"1min":0.33700000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.68799999999999994,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.46400000000000002}]},{"osd":3,"last update":"Sun Oct 8 00:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.55100000000000005,"15min":0.54700000000000004},"min":{"1min":0.35499999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.73199999999999998,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.55800000000000005,"15min":0.57599999999999996},"min":{"1min":0.38,"5min":0.371,"15min":0.371},"max":{"1min":0.71999999999999997,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.42699999999999999}]},{"osd":4,"last update":"Sun Oct 8 00:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.46999999999999997,"15min":0.47399999999999998},"min":{"1min":0.312,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.69399999999999995,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.51300000000000001,"15min":0.51500000000000001},"min":{"1min":0.40000000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.68300000000000005,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.57999999999999996}]},{"osd":6,"last update":"Sun Oct 8 00:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.505,"15min":0.501},"min":{"1min":0.36299999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.63200000000000001,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.505,"15min":0.496},"min":{"1min":0.36199999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68300000000000005,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.56499999999999995}]},{"osd":7,"last update":"Sun Oct 8 00:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52600000000000002,"15min":0.51900000000000002},"min":{"1min":0.379,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.66600000000000004,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55600000000000005,"15min":0.55700000000000005},"min":{"1min":0.34499999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.69599999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.59099999999999997}]}]}],"pool_statfs":[]}} 2023-10-08T00:16:54.790 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T00:16:54.931 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.931+0000 7fce2824e700 1 Processor -- start 2023-10-08T00:16:54.931 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.931+0000 7fce2824e700 1 -- start start 2023-10-08T00:16:54.933 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.932+0000 7fce2824e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce20157870 0x7fce2015b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:54.934 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.932+0000 7fce2824e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fce2015a620 0x7fce2015d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:54.934 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.932+0000 7fce2824e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce2015cbb0 con 0x7fce2015d580 2023-10-08T00:16:54.934 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.932+0000 7fce2824e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce2015bfa0 con 0x7fce2015a620 2023-10-08T00:16:54.934 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.932+0000 7fce2824e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce2015c280 con 0x7fce20157870 2023-10-08T00:16:54.934 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.932+0000 7fce257e9700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fce2015a620 0x7fce2015d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:54.934 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.932+0000 7fce25fea700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce20157870 0x7fce2015b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:54.934 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.932+0000 7fce267eb700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7fce2015d580 0x7fce20158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:42542/0 (socket says 172.21.15.99:42542) 2023-10-08T00:16:54.934 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.932+0000 7fce267eb700 1 -- 172.21.15.99:0/710982151 learned_addr learned my addr 172.21.15.99:0/710982151 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:54.934 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.932+0000 7fce257e9700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fce2015a620 0x7fce2015d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.99:38034/0 (socket says 172.21.15.99:38034) 2023-10-08T00:16:54.934 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.932+0000 7fce25fea700 1 -- 172.21.15.99:0/710982151 >> v2:172.21.15.191:3300/0 conn(0x7fce2015a620 msgr2=0x7fce2015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:54.934 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.932+0000 7fce25fea700 1 --2- 172.21.15.99:0/710982151 >> v2:172.21.15.191:3300/0 conn(0x7fce2015a620 0x7fce2015d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:54.935 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.932+0000 7fce25fea700 1 -- 172.21.15.99:0/710982151 >> v1:172.21.15.191:6789/0 conn(0x7fce2015d580 legacy=0x7fce20158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:54.935 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.932+0000 7fce25fea700 1 -- 172.21.15.99:0/710982151 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce2015c560 con 0x7fce20157870 2023-10-08T00:16:54.935 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.933+0000 7fce257e9700 1 --2- 172.21.15.99:0/710982151 >> v2:172.21.15.191:3300/0 conn(0x7fce2015a620 0x7fce2015d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:16:54.935 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.933+0000 7fce25fea700 1 --2- 172.21.15.99:0/710982151 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce20157870 0x7fce2015b6a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fce1c0098e0 tx=0x7fce1c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8ec8f6641c65621 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:54.935 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.933+0000 7fce24fe8700 1 -- 172.21.15.99:0/710982151 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c00b640 con 0x7fce20157870 2023-10-08T00:16:54.935 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.933+0000 7fce24fe8700 1 -- 172.21.15.99:0/710982151 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce1c00b7a0 con 0x7fce20157870 2023-10-08T00:16:54.935 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.933+0000 7fce24fe8700 1 -- 172.21.15.99:0/710982151 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c00bb40 con 0x7fce20157870 2023-10-08T00:16:54.938 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.933+0000 7fce2824e700 1 -- 172.21.15.99:0/710982151 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce20157870 msgr2=0x7fce2015b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:54.939 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.933+0000 7fce2824e700 1 --2- 172.21.15.99:0/710982151 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce20157870 0x7fce2015b6a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fce1c0098e0 tx=0x7fce1c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:16:54.939 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.933+0000 7fce2824e700 1 -- 172.21.15.99:0/710982151 shutdown_connections 2023-10-08T00:16:54.939 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.933+0000 7fce2824e700 1 --2- 172.21.15.99:0/710982151 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce20157870 0x7fce2015b6a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:54.939 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.933+0000 7fce2824e700 1 --2- 172.21.15.99:0/710982151 >> v2:172.21.15.191:3300/0 conn(0x7fce2015a620 0x7fce2015d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:54.939 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.933+0000 7fce2824e700 1 -- 172.21.15.99:0/710982151 >> 172.21.15.99:0/710982151 conn(0x7fce20068ce0 msgr2=0x7fce200690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:54.940 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.934+0000 7fce2824e700 1 -- 172.21.15.99:0/710982151 shutdown_connections 2023-10-08T00:16:54.940 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.934+0000 7fce2824e700 1 -- 172.21.15.99:0/710982151 wait complete. 2023-10-08T00:16:54.940 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.935+0000 7fce2824e700 1 Processor -- start 2023-10-08T00:16:54.940 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.935+0000 7fce2824e700 1 -- start start 2023-10-08T00:16:54.940 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.935+0000 7fce2824e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce20157870 0x7fce2016a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:54.940 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.935+0000 7fce2824e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce2015a620 0x7fce200b2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:54.940 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.935+0000 7fce2824e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce2015d580 0x7fce200b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:54.940 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.935+0000 7fce2824e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce201491c0 con 0x7fce2015a620 2023-10-08T00:16:54.940 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.935+0000 7fce2824e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce200cd7a0 con 0x7fce2015d580 2023-10-08T00:16:54.940 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.935+0000 7fce2824e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce20148b70 con 0x7fce20157870 2023-10-08T00:16:54.940 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.935+0000 7fce257e9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce2015a620 0x7fce200b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:54.940 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.935+0000 7fce257e9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce2015a620 0x7fce200b2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55958/0 (socket says 172.21.15.99:55958) 2023-10-08T00:16:54.941 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.936+0000 7fce257e9700 1 -- 172.21.15.99:0/1755707080 learned_addr learned my addr 172.21.15.99:0/1755707080 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:54.941 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.936+0000 7fce25fea700 1 --2- 172.21.15.99:0/1755707080 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce20157870 0x7fce2016a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:54.941 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.936+0000 7fce267eb700 1 --2- 172.21.15.99:0/1755707080 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce2015d580 0x7fce200b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:54.941 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.936+0000 7fce257e9700 1 -- 172.21.15.99:0/1755707080 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce20157870 msgr2=0x7fce2016a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:54.941 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.936+0000 7fce257e9700 1 --2- 172.21.15.99:0/1755707080 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce20157870 0x7fce2016a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:54.941 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.936+0000 7fce257e9700 1 -- 172.21.15.99:0/1755707080 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce2015d580 msgr2=0x7fce200b2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:54.941 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.936+0000 7fce257e9700 1 --2- 172.21.15.99:0/1755707080 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce2015d580 0x7fce200b2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:54.941 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.936+0000 7fce257e9700 1 -- 172.21.15.99:0/1755707080 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce10009420 con 0x7fce2015a620 2023-10-08T00:16:54.941 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.936+0000 7fce257e9700 1 --2- 172.21.15.99:0/1755707080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce2015a620 0x7fce200b2760 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fce1000ec80 tx=0x7fce100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:54.941 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.936+0000 7fce15ffb700 1 -- 172.21.15.99:0/1755707080 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1000c920 con 0x7fce2015a620 2023-10-08T00:16:54.941 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.936+0000 7fce2824e700 1 -- 172.21.15.99:0/1755707080 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce1c009420 con 0x7fce2015a620 2023-10-08T00:16:54.941 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.936+0000 7fce15ffb700 1 -- 172.21.15.99:0/1755707080 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce1000ca80 con 0x7fce2015a620 2023-10-08T00:16:54.942 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.936+0000 7fce15ffb700 1 -- 172.21.15.99:0/1755707080 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1000ce00 con 0x7fce2015a620 2023-10-08T00:16:54.942 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.936+0000 7fce2824e700 1 -- 172.21.15.99:0/1755707080 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce2016a750 con 0x7fce2015a620 2023-10-08T00:16:54.942 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.937+0000 7fce2824e700 1 -- 172.21.15.99:0/1755707080 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce201489d0 con 0x7fce2015a620 2023-10-08T00:16:54.943 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.938+0000 7fce15ffb700 1 -- 172.21.15.99:0/1755707080 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fce10013680 con 0x7fce2015a620 2023-10-08T00:16:54.943 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.939+0000 7fce15ffb700 1 --2- 172.21.15.99:0/1755707080 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fce04066f30 0x7fce040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:54.943 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.939+0000 7fce15ffb700 1 -- 172.21.15.99:0/1755707080 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fce10085060 con 0x7fce2015a620 2023-10-08T00:16:54.943 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.939+0000 7fce25fea700 1 --2- 172.21.15.99:0/1755707080 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fce04066f30 0x7fce040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:54.943 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.940+0000 7fce25fea700 1 --2- 172.21.15.99:0/1755707080 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fce04066f30 0x7fce040693e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fce1c0093f0 tx=0x7fce1c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:54.943 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:54.942+0000 7fce15ffb700 1 -- 172.21.15.99:0/1755707080 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fce10052230 con 0x7fce2015a620 2023-10-08T00:16:55.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.119+0000 7fce2824e700 1 -- 172.21.15.99:0/1755707080 --> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fce201489d0 con 0x7fce04066f30 2023-10-08T00:16:55.128 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.128+0000 7fce15ffb700 1 -- 172.21.15.99:0/1755707080 <== mgr.4099 v2:172.21.15.191:6832/85985 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285712 (secure 0 0 0) 0x7fce201489d0 con 0x7fce04066f30 2023-10-08T00:16:55.132 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:16:55.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.131+0000 7fce2824e700 1 -- 172.21.15.99:0/1755707080 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fce04066f30 msgr2=0x7fce040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:55.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.131+0000 7fce2824e700 1 --2- 172.21.15.99:0/1755707080 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fce04066f30 0x7fce040693e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fce1c0093f0 tx=0x7fce1c017000 comp rx=0 tx=0).stop 2023-10-08T00:16:55.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.131+0000 7fce2824e700 1 -- 172.21.15.99:0/1755707080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce2015a620 msgr2=0x7fce200b2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:55.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.131+0000 7fce2824e700 1 --2- 172.21.15.99:0/1755707080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce2015a620 0x7fce200b2760 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fce1000ec80 tx=0x7fce100091a0 comp rx=0 tx=0).stop 2023-10-08T00:16:55.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.131+0000 7fce2824e700 1 -- 172.21.15.99:0/1755707080 shutdown_connections 2023-10-08T00:16:55.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.131+0000 7fce2824e700 1 --2- 172.21.15.99:0/1755707080 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fce04066f30 0x7fce040693e0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:55.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.131+0000 7fce2824e700 1 --2- 172.21.15.99:0/1755707080 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce2015d580 0x7fce200b2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:55.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.131+0000 7fce2824e700 1 --2- 172.21.15.99:0/1755707080 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce20157870 0x7fce2016a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:55.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.131+0000 7fce2824e700 1 --2- 172.21.15.99:0/1755707080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce2015a620 0x7fce200b2760 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:55.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.131+0000 7fce2824e700 1 -- 172.21.15.99:0/1755707080 >> 172.21.15.99:0/1755707080 conn(0x7fce20068ce0 msgr2=0x7fce20159d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:55.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.131+0000 7fce2824e700 1 -- 172.21.15.99:0/1755707080 shutdown_connections 2023-10-08T00:16:55.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.131+0000 7fce2824e700 1 -- 172.21.15.99:0/1755707080 wait complete. 2023-10-08T00:16:55.136 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2023-10-08T00:16:55.143 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":315,"stamp":"2023-10-08T00:16:54.109263+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":126,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":7153,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555346,"statfs":{"total":1600000000,"available":1592675100,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.396980"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.738670+0000","last_change":"2023-10-08T00:16:50.737905+0000","last_active":"2023-10-08T00:16:50.738670+0000","last_peered":"2023-10-08T00:16:50.738670+0000","last_clean":"2023-10-08T00:16:50.738670+0000","last_became_active":"2023-10-08T00:16:50.736176+0000","last_became_peered":"2023-10-08T00:16:50.736176+0000","last_unstale":"2023-10-08T00:16:50.738670+0000","last_undegraded":"2023-10-08T00:16:50.738670+0000","last_fullsized":"2023-10-08T00:16:50.738670+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:09:43.382366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.735639+0000","last_change":"2023-10-08T00:16:50.735542+0000","last_active":"2023-10-08T00:16:50.735639+0000","last_peered":"2023-10-08T00:16:50.735639+0000","last_clean":"2023-10-08T00:16:50.735639+0000","last_became_active":"2023-10-08T00:16:50.735133+0000","last_became_peered":"2023-10-08T00:16:50.735133+0000","last_unstale":"2023-10-08T00:16:50.735639+0000","last_undegraded":"2023-10-08T00:16:50.735639+0000","last_fullsized":"2023-10-08T00:16:50.735639+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:28:01.427127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.730656+0000","last_change":"2023-10-08T00:16:50.730594+0000","last_active":"2023-10-08T00:16:50.730656+0000","last_peered":"2023-10-08T00:16:50.730656+0000","last_clean":"2023-10-08T00:16:50.730656+0000","last_became_active":"2023-10-08T00:16:50.730375+0000","last_became_peered":"2023-10-08T00:16:50.730375+0000","last_unstale":"2023-10-08T00:16:50.730656+0000","last_undegraded":"2023-10-08T00:16:50.730656+0000","last_fullsized":"2023-10-08T00:16:50.730656+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:49:20.063701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.734354+0000","last_change":"2023-10-08T00:16:50.734293+0000","last_active":"2023-10-08T00:16:50.734354+0000","last_peered":"2023-10-08T00:16:50.734354+0000","last_clean":"2023-10-08T00:16:50.734354+0000","last_became_active":"2023-10-08T00:16:50.734039+0000","last_became_peered":"2023-10-08T00:16:50.734039+0000","last_unstale":"2023-10-08T00:16:50.734354+0000","last_undegraded":"2023-10-08T00:16:50.734354+0000","last_fullsized":"2023-10-08T00:16:50.734354+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:44:32.941862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.748672+0000","last_change":"2023-10-08T00:16:50.748168+0000","last_active":"2023-10-08T00:16:50.748672+0000","last_peered":"2023-10-08T00:16:50.748672+0000","last_clean":"2023-10-08T00:16:50.748672+0000","last_became_active":"2023-10-08T00:16:50.748000+0000","last_became_peered":"2023-10-08T00:16:50.748000+0000","last_unstale":"2023-10-08T00:16:50.748672+0000","last_undegraded":"2023-10-08T00:16:50.748672+0000","last_fullsized":"2023-10-08T00:16:50.748672+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:53:19.351074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.737901+0000","last_change":"2023-10-08T00:16:50.736246+0000","last_active":"2023-10-08T00:16:50.737901+0000","last_peered":"2023-10-08T00:16:50.737901+0000","last_clean":"2023-10-08T00:16:50.737901+0000","last_became_active":"2023-10-08T00:16:50.736052+0000","last_became_peered":"2023-10-08T00:16:50.736052+0000","last_unstale":"2023-10-08T00:16:50.737901+0000","last_undegraded":"2023-10-08T00:16:50.737901+0000","last_fullsized":"2023-10-08T00:16:50.737901+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:17:10.436982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.737565+0000","last_change":"2023-10-08T00:16:50.737453+0000","last_active":"2023-10-08T00:16:50.737565+0000","last_peered":"2023-10-08T00:16:50.737565+0000","last_clean":"2023-10-08T00:16:50.737565+0000","last_became_active":"2023-10-08T00:16:50.737257+0000","last_became_peered":"2023-10-08T00:16:50.737257+0000","last_unstale":"2023-10-08T00:16:50.737565+0000","last_undegraded":"2023-10-08T00:16:50.737565+0000","last_fullsized":"2023-10-08T00:16:50.737565+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:42:38.573355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.733487+0000","last_change":"2023-10-08T00:16:50.733289+0000","last_active":"2023-10-08T00:16:50.733487+0000","last_peered":"2023-10-08T00:16:50.733487+0000","last_clean":"2023-10-08T00:16:50.733487+0000","last_became_active":"2023-10-08T00:16:50.733095+0000","last_became_peered":"2023-10-08T00:16:50.733095+0000","last_unstale":"2023-10-08T00:16:50.733487+0000","last_undegraded":"2023-10-08T00:16:50.733487+0000","last_fullsized":"2023-10-08T00:16:50.733487+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:52:52.729685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.740049+0000","last_change":"2023-10-08T00:16:50.739989+0000","last_active":"2023-10-08T00:16:50.740049+0000","last_peered":"2023-10-08T00:16:50.740049+0000","last_clean":"2023-10-08T00:16:50.740049+0000","last_became_active":"2023-10-08T00:16:50.739634+0000","last_became_peered":"2023-10-08T00:16:50.739634+0000","last_unstale":"2023-10-08T00:16:50.740049+0000","last_undegraded":"2023-10-08T00:16:50.740049+0000","last_fullsized":"2023-10-08T00:16:50.740049+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:38:58.924084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.744458+0000","last_change":"2023-10-08T00:16:50.744404+0000","last_active":"2023-10-08T00:16:50.744458+0000","last_peered":"2023-10-08T00:16:50.744458+0000","last_clean":"2023-10-08T00:16:50.744458+0000","last_became_active":"2023-10-08T00:16:50.744185+0000","last_became_peered":"2023-10-08T00:16:50.744185+0000","last_unstale":"2023-10-08T00:16:50.744458+0000","last_undegraded":"2023-10-08T00:16:50.744458+0000","last_fullsized":"2023-10-08T00:16:50.744458+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:08:56.689278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.737547+0000","last_change":"2023-10-08T00:16:50.737398+0000","last_active":"2023-10-08T00:16:50.737547+0000","last_peered":"2023-10-08T00:16:50.737547+0000","last_clean":"2023-10-08T00:16:50.737547+0000","last_became_active":"2023-10-08T00:16:50.737195+0000","last_became_peered":"2023-10-08T00:16:50.737195+0000","last_unstale":"2023-10-08T00:16:50.737547+0000","last_undegraded":"2023-10-08T00:16:50.737547+0000","last_fullsized":"2023-10-08T00:16:50.737547+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:36:42.997162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.743585+0000","last_change":"2023-10-08T00:16:50.743504+0000","last_active":"2023-10-08T00:16:50.743585+0000","last_peered":"2023-10-08T00:16:50.743585+0000","last_clean":"2023-10-08T00:16:50.743585+0000","last_became_active":"2023-10-08T00:16:50.742979+0000","last_became_peered":"2023-10-08T00:16:50.742979+0000","last_unstale":"2023-10-08T00:16:50.743585+0000","last_undegraded":"2023-10-08T00:16:50.743585+0000","last_fullsized":"2023-10-08T00:16:50.743585+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:49:08.450879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.742360+0000","last_change":"2023-10-08T00:16:50.742306+0000","last_active":"2023-10-08T00:16:50.742360+0000","last_peered":"2023-10-08T00:16:50.742360+0000","last_clean":"2023-10-08T00:16:50.742360+0000","last_became_active":"2023-10-08T00:16:50.742064+0000","last_became_peered":"2023-10-08T00:16:50.742064+0000","last_unstale":"2023-10-08T00:16:50.742360+0000","last_undegraded":"2023-10-08T00:16:50.742360+0000","last_fullsized":"2023-10-08T00:16:50.742360+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:21:02.984633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.739965+0000","last_change":"2023-10-08T00:16:50.739921+0000","last_active":"2023-10-08T00:16:50.739965+0000","last_peered":"2023-10-08T00:16:50.739965+0000","last_clean":"2023-10-08T00:16:50.739965+0000","last_became_active":"2023-10-08T00:16:50.739605+0000","last_became_peered":"2023-10-08T00:16:50.739605+0000","last_unstale":"2023-10-08T00:16:50.739965+0000","last_undegraded":"2023-10-08T00:16:50.739965+0000","last_fullsized":"2023-10-08T00:16:50.739965+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:57:49.104276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.746133+0000","last_change":"2023-10-08T00:16:50.746042+0000","last_active":"2023-10-08T00:16:50.746133+0000","last_peered":"2023-10-08T00:16:50.746133+0000","last_clean":"2023-10-08T00:16:50.746133+0000","last_became_active":"2023-10-08T00:16:50.745655+0000","last_became_peered":"2023-10-08T00:16:50.745655+0000","last_unstale":"2023-10-08T00:16:50.746133+0000","last_undegraded":"2023-10-08T00:16:50.746133+0000","last_fullsized":"2023-10-08T00:16:50.746133+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:17:10.436982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.742338+0000","last_change":"2023-10-08T00:16:50.742281+0000","last_active":"2023-10-08T00:16:50.742338+0000","last_peered":"2023-10-08T00:16:50.742338+0000","last_clean":"2023-10-08T00:16:50.742338+0000","last_became_active":"2023-10-08T00:16:50.741932+0000","last_became_peered":"2023-10-08T00:16:50.741932+0000","last_unstale":"2023-10-08T00:16:50.742338+0000","last_undegraded":"2023-10-08T00:16:50.742338+0000","last_fullsized":"2023-10-08T00:16:50.742338+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:48:03.562982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.737913+0000","last_change":"2023-10-08T00:16:50.737858+0000","last_active":"2023-10-08T00:16:50.737913+0000","last_peered":"2023-10-08T00:16:50.737913+0000","last_clean":"2023-10-08T00:16:50.737913+0000","last_became_active":"2023-10-08T00:16:50.737628+0000","last_became_peered":"2023-10-08T00:16:50.737628+0000","last_unstale":"2023-10-08T00:16:50.737913+0000","last_undegraded":"2023-10-08T00:16:50.737913+0000","last_fullsized":"2023-10-08T00:16:50.737913+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:53:24.578330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.739882+0000","last_change":"2023-10-08T00:16:50.739828+0000","last_active":"2023-10-08T00:16:50.739882+0000","last_peered":"2023-10-08T00:16:50.739882+0000","last_clean":"2023-10-08T00:16:50.739882+0000","last_became_active":"2023-10-08T00:16:50.739634+0000","last_became_peered":"2023-10-08T00:16:50.739634+0000","last_unstale":"2023-10-08T00:16:50.739882+0000","last_undegraded":"2023-10-08T00:16:50.739882+0000","last_fullsized":"2023-10-08T00:16:50.739882+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:47:35.965982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.742032+0000","last_change":"2023-10-08T00:16:50.741598+0000","last_active":"2023-10-08T00:16:50.742032+0000","last_peered":"2023-10-08T00:16:50.742032+0000","last_clean":"2023-10-08T00:16:50.742032+0000","last_became_active":"2023-10-08T00:16:50.741387+0000","last_became_peered":"2023-10-08T00:16:50.741387+0000","last_unstale":"2023-10-08T00:16:50.742032+0000","last_undegraded":"2023-10-08T00:16:50.742032+0000","last_fullsized":"2023-10-08T00:16:50.742032+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:38:38.194551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.736134+0000","last_change":"2023-10-08T00:16:50.736043+0000","last_active":"2023-10-08T00:16:50.736134+0000","last_peered":"2023-10-08T00:16:50.736134+0000","last_clean":"2023-10-08T00:16:50.736134+0000","last_became_active":"2023-10-08T00:16:50.735850+0000","last_became_peered":"2023-10-08T00:16:50.735850+0000","last_unstale":"2023-10-08T00:16:50.736134+0000","last_undegraded":"2023-10-08T00:16:50.736134+0000","last_fullsized":"2023-10-08T00:16:50.736134+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:29:26.667047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.736599+0000","last_change":"2023-10-08T00:16:50.736361+0000","last_active":"2023-10-08T00:16:50.736599+0000","last_peered":"2023-10-08T00:16:50.736599+0000","last_clean":"2023-10-08T00:16:50.736599+0000","last_became_active":"2023-10-08T00:16:50.736134+0000","last_became_peered":"2023-10-08T00:16:50.736134+0000","last_unstale":"2023-10-08T00:16:50.736599+0000","last_undegraded":"2023-10-08T00:16:50.736599+0000","last_fullsized":"2023-10-08T00:16:50.736599+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:35:31.155748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.737870+0000","last_change":"2023-10-08T00:16:50.734528+0000","last_active":"2023-10-08T00:16:50.737870+0000","last_peered":"2023-10-08T00:16:50.737870+0000","last_clean":"2023-10-08T00:16:50.737870+0000","last_became_active":"2023-10-08T00:16:50.734294+0000","last_became_peered":"2023-10-08T00:16:50.734294+0000","last_unstale":"2023-10-08T00:16:50.737870+0000","last_undegraded":"2023-10-08T00:16:50.737870+0000","last_fullsized":"2023-10-08T00:16:50.737870+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:38:58.924084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.739376+0000","last_change":"2023-10-08T00:16:50.739322+0000","last_active":"2023-10-08T00:16:50.739376+0000","last_peered":"2023-10-08T00:16:50.739376+0000","last_clean":"2023-10-08T00:16:50.739376+0000","last_became_active":"2023-10-08T00:16:50.739116+0000","last_became_peered":"2023-10-08T00:16:50.739116+0000","last_unstale":"2023-10-08T00:16:50.739376+0000","last_undegraded":"2023-10-08T00:16:50.739376+0000","last_fullsized":"2023-10-08T00:16:50.739376+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:30:15.845800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.738544+0000","last_change":"2023-10-08T00:16:50.738361+0000","last_active":"2023-10-08T00:16:50.738544+0000","last_peered":"2023-10-08T00:16:50.738544+0000","last_clean":"2023-10-08T00:16:50.738544+0000","last_became_active":"2023-10-08T00:16:50.738013+0000","last_became_peered":"2023-10-08T00:16:50.738013+0000","last_unstale":"2023-10-08T00:16:50.738544+0000","last_undegraded":"2023-10-08T00:16:50.738544+0000","last_fullsized":"2023-10-08T00:16:50.738544+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:16:47.905466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.733723+0000","last_change":"2023-10-08T00:16:50.733524+0000","last_active":"2023-10-08T00:16:50.733723+0000","last_peered":"2023-10-08T00:16:50.733723+0000","last_clean":"2023-10-08T00:16:50.733723+0000","last_became_active":"2023-10-08T00:16:50.733020+0000","last_became_peered":"2023-10-08T00:16:50.733020+0000","last_unstale":"2023-10-08T00:16:50.733723+0000","last_undegraded":"2023-10-08T00:16:50.733723+0000","last_fullsized":"2023-10-08T00:16:50.733723+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:37:43.082386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.729830+0000","last_change":"2023-10-08T00:16:50.729766+0000","last_active":"2023-10-08T00:16:50.729830+0000","last_peered":"2023-10-08T00:16:50.729830+0000","last_clean":"2023-10-08T00:16:50.729830+0000","last_became_active":"2023-10-08T00:16:50.729640+0000","last_became_peered":"2023-10-08T00:16:50.729640+0000","last_unstale":"2023-10-08T00:16:50.729830+0000","last_undegraded":"2023-10-08T00:16:50.729830+0000","last_fullsized":"2023-10-08T00:16:50.729830+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:57:49.104276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.738528+0000","last_change":"2023-10-08T00:16:50.737900+0000","last_active":"2023-10-08T00:16:50.738528+0000","last_peered":"2023-10-08T00:16:50.738528+0000","last_clean":"2023-10-08T00:16:50.738528+0000","last_became_active":"2023-10-08T00:16:50.736680+0000","last_became_peered":"2023-10-08T00:16:50.736680+0000","last_unstale":"2023-10-08T00:16:50.738528+0000","last_undegraded":"2023-10-08T00:16:50.738528+0000","last_fullsized":"2023-10-08T00:16:50.738528+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:41:41.134711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.737851+0000","last_change":"2023-10-08T00:16:50.733336+0000","last_active":"2023-10-08T00:16:50.737851+0000","last_peered":"2023-10-08T00:16:50.737851+0000","last_clean":"2023-10-08T00:16:50.737851+0000","last_became_active":"2023-10-08T00:16:50.733152+0000","last_became_peered":"2023-10-08T00:16:50.733152+0000","last_unstale":"2023-10-08T00:16:50.737851+0000","last_undegraded":"2023-10-08T00:16:50.737851+0000","last_fullsized":"2023-10-08T00:16:50.737851+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:45:12.144679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.739863+0000","last_change":"2023-10-08T00:16:50.739806+0000","last_active":"2023-10-08T00:16:50.739863+0000","last_peered":"2023-10-08T00:16:50.739863+0000","last_clean":"2023-10-08T00:16:50.739863+0000","last_became_active":"2023-10-08T00:16:50.739346+0000","last_became_peered":"2023-10-08T00:16:50.739346+0000","last_unstale":"2023-10-08T00:16:50.739863+0000","last_undegraded":"2023-10-08T00:16:50.739863+0000","last_fullsized":"2023-10-08T00:16:50.739863+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:17:10.436982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.733738+0000","last_change":"2023-10-08T00:16:50.733649+0000","last_active":"2023-10-08T00:16:50.733738+0000","last_peered":"2023-10-08T00:16:50.733738+0000","last_clean":"2023-10-08T00:16:50.733738+0000","last_became_active":"2023-10-08T00:16:50.733308+0000","last_became_peered":"2023-10-08T00:16:50.733308+0000","last_unstale":"2023-10-08T00:16:50.733738+0000","last_undegraded":"2023-10-08T00:16:50.733738+0000","last_fullsized":"2023-10-08T00:16:50.733738+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:16:44.190570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.738689+0000","last_change":"2023-10-08T00:16:50.738244+0000","last_active":"2023-10-08T00:16:50.738689+0000","last_peered":"2023-10-08T00:16:50.738689+0000","last_clean":"2023-10-08T00:16:50.738689+0000","last_became_active":"2023-10-08T00:16:50.736457+0000","last_became_peered":"2023-10-08T00:16:50.736457+0000","last_unstale":"2023-10-08T00:16:50.738689+0000","last_undegraded":"2023-10-08T00:16:50.738689+0000","last_fullsized":"2023-10-08T00:16:50.738689+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:57:49.104276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.743840+0000","last_change":"2023-10-08T00:16:50.743777+0000","last_active":"2023-10-08T00:16:50.743840+0000","last_peered":"2023-10-08T00:16:50.743840+0000","last_clean":"2023-10-08T00:16:50.743840+0000","last_became_active":"2023-10-08T00:16:50.743300+0000","last_became_peered":"2023-10-08T00:16:50.743300+0000","last_unstale":"2023-10-08T00:16:50.743840+0000","last_undegraded":"2023-10-08T00:16:50.743840+0000","last_fullsized":"2023-10-08T00:16:50.743840+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:17:21.037010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.739908+0000","last_change":"2023-10-08T00:16:50.739852+0000","last_active":"2023-10-08T00:16:50.739908+0000","last_peered":"2023-10-08T00:16:50.739908+0000","last_clean":"2023-10-08T00:16:50.739908+0000","last_became_active":"2023-10-08T00:16:50.739316+0000","last_became_peered":"2023-10-08T00:16:50.739316+0000","last_unstale":"2023-10-08T00:16:50.739908+0000","last_undegraded":"2023-10-08T00:16:50.739908+0000","last_fullsized":"2023-10-08T00:16:50.739908+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:53:27.357114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"138'4","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.746348+0000","last_change":"2023-10-08T00:16:49.719279+0000","last_active":"2023-10-08T00:16:50.746348+0000","last_peered":"2023-10-08T00:16:50.746348+0000","last_clean":"2023-10-08T00:16:50.746348+0000","last_became_active":"2023-10-08T00:16:49.719031+0000","last_became_peered":"2023-10-08T00:16:49.719031+0000","last_unstale":"2023-10-08T00:16:50.746348+0000","last_undegraded":"2023-10-08T00:16:50.746348+0000","last_fullsized":"2023-10-08T00:16:50.746348+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:08:55.684301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.738520+0000","last_change":"2023-10-08T00:16:50.738464+0000","last_active":"2023-10-08T00:16:50.738520+0000","last_peered":"2023-10-08T00:16:50.738520+0000","last_clean":"2023-10-08T00:16:50.738520+0000","last_became_active":"2023-10-08T00:16:50.738253+0000","last_became_peered":"2023-10-08T00:16:50.738253+0000","last_unstale":"2023-10-08T00:16:50.738520+0000","last_undegraded":"2023-10-08T00:16:50.738520+0000","last_fullsized":"2023-10-08T00:16:50.738520+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:21:02.984633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"138'1","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.744981+0000","last_change":"2023-10-08T00:16:49.721557+0000","last_active":"2023-10-08T00:16:50.744981+0000","last_peered":"2023-10-08T00:16:50.744981+0000","last_clean":"2023-10-08T00:16:50.744981+0000","last_became_active":"2023-10-08T00:16:49.721319+0000","last_became_peered":"2023-10-08T00:16:49.721319+0000","last_unstale":"2023-10-08T00:16:50.744981+0000","last_undegraded":"2023-10-08T00:16:50.744981+0000","last_fullsized":"2023-10-08T00:16:50.744981+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:48:02.558006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.737886+0000","last_change":"2023-10-08T00:16:50.735836+0000","last_active":"2023-10-08T00:16:50.737886+0000","last_peered":"2023-10-08T00:16:50.737886+0000","last_clean":"2023-10-08T00:16:50.737886+0000","last_became_active":"2023-10-08T00:16:50.735625+0000","last_became_peered":"2023-10-08T00:16:50.735625+0000","last_unstale":"2023-10-08T00:16:50.737886+0000","last_undegraded":"2023-10-08T00:16:50.737886+0000","last_fullsized":"2023-10-08T00:16:50.737886+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:34:23.975157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1d","version":"138'2","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.750483+0000","last_change":"2023-10-08T00:16:49.722772+0000","last_active":"2023-10-08T00:16:50.750483+0000","last_peered":"2023-10-08T00:16:50.750483+0000","last_clean":"2023-10-08T00:16:50.750483+0000","last_became_active":"2023-10-08T00:16:49.721872+0000","last_became_peered":"2023-10-08T00:16:49.721872+0000","last_unstale":"2023-10-08T00:16:50.750483+0000","last_undegraded":"2023-10-08T00:16:50.750483+0000","last_fullsized":"2023-10-08T00:16:50.750483+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:53:23.573354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.743075+0000","last_change":"2023-10-08T00:16:50.742985+0000","last_active":"2023-10-08T00:16:50.743075+0000","last_peered":"2023-10-08T00:16:50.743075+0000","last_clean":"2023-10-08T00:16:50.743075+0000","last_became_active":"2023-10-08T00:16:50.742803+0000","last_became_peered":"2023-10-08T00:16:50.742803+0000","last_unstale":"2023-10-08T00:16:50.743075+0000","last_undegraded":"2023-10-08T00:16:50.743075+0000","last_fullsized":"2023-10-08T00:16:50.743075+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:14:18.374798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"138'2","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.750339+0000","last_change":"2023-10-08T00:16:49.724426+0000","last_active":"2023-10-08T00:16:50.750339+0000","last_peered":"2023-10-08T00:16:50.750339+0000","last_clean":"2023-10-08T00:16:50.750339+0000","last_became_active":"2023-10-08T00:16:49.723645+0000","last_became_peered":"2023-10-08T00:16:49.723645+0000","last_unstale":"2023-10-08T00:16:50.750339+0000","last_undegraded":"2023-10-08T00:16:50.750339+0000","last_fullsized":"2023-10-08T00:16:50.750339+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:14:17.369821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.738503+0000","last_change":"2023-10-08T00:16:50.737642+0000","last_active":"2023-10-08T00:16:50.738503+0000","last_peered":"2023-10-08T00:16:50.738503+0000","last_clean":"2023-10-08T00:16:50.738503+0000","last_became_active":"2023-10-08T00:16:50.737393+0000","last_became_peered":"2023-10-08T00:16:50.737393+0000","last_unstale":"2023-10-08T00:16:50.738503+0000","last_undegraded":"2023-10-08T00:16:50.738503+0000","last_fullsized":"2023-10-08T00:16:50.738503+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:48:03.562982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1b","version":"138'1","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.745606+0000","last_change":"2023-10-08T00:16:49.725193+0000","last_active":"2023-10-08T00:16:50.745606+0000","last_peered":"2023-10-08T00:16:50.745606+0000","last_clean":"2023-10-08T00:16:50.745606+0000","last_became_active":"2023-10-08T00:16:49.724960+0000","last_became_peered":"2023-10-08T00:16:49.724960+0000","last_unstale":"2023-10-08T00:16:50.745606+0000","last_undegraded":"2023-10-08T00:16:50.745606+0000","last_fullsized":"2023-10-08T00:16:50.745606+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:41:22.003276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.742409+0000","last_change":"2023-10-08T00:16:50.742367+0000","last_active":"2023-10-08T00:16:50.742409+0000","last_peered":"2023-10-08T00:16:50.742409+0000","last_clean":"2023-10-08T00:16:50.742409+0000","last_became_active":"2023-10-08T00:16:50.742227+0000","last_became_peered":"2023-10-08T00:16:50.742227+0000","last_unstale":"2023-10-08T00:16:50.742409+0000","last_undegraded":"2023-10-08T00:16:50.742409+0000","last_fullsized":"2023-10-08T00:16:50.742409+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:17:26.761580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1a","version":"138'1","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.750338+0000","last_change":"2023-10-08T00:16:49.722953+0000","last_active":"2023-10-08T00:16:50.750338+0000","last_peered":"2023-10-08T00:16:50.750338+0000","last_clean":"2023-10-08T00:16:50.750338+0000","last_became_active":"2023-10-08T00:16:49.722706+0000","last_became_peered":"2023-10-08T00:16:49.722706+0000","last_unstale":"2023-10-08T00:16:50.750338+0000","last_undegraded":"2023-10-08T00:16:50.750338+0000","last_fullsized":"2023-10-08T00:16:50.750338+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:48:02.558006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.748470+0000","last_change":"2023-10-08T00:16:50.746574+0000","last_active":"2023-10-08T00:16:50.748470+0000","last_peered":"2023-10-08T00:16:50.748470+0000","last_clean":"2023-10-08T00:16:50.748470+0000","last_became_active":"2023-10-08T00:16:50.746008+0000","last_became_peered":"2023-10-08T00:16:50.746008+0000","last_unstale":"2023-10-08T00:16:50.748470+0000","last_undegraded":"2023-10-08T00:16:50.748470+0000","last_fullsized":"2023-10-08T00:16:50.748470+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:16:47.905466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.731355+0000","last_change":"2023-10-08T00:16:49.721439+0000","last_active":"2023-10-08T00:16:50.731355+0000","last_peered":"2023-10-08T00:16:50.731355+0000","last_clean":"2023-10-08T00:16:50.731355+0000","last_became_active":"2023-10-08T00:16:49.721228+0000","last_became_peered":"2023-10-08T00:16:49.721228+0000","last_unstale":"2023-10-08T00:16:50.731355+0000","last_undegraded":"2023-10-08T00:16:50.731355+0000","last_fullsized":"2023-10-08T00:16:50.731355+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:00:03.204002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.733505+0000","last_change":"2023-10-08T00:16:50.733319+0000","last_active":"2023-10-08T00:16:50.733505+0000","last_peered":"2023-10-08T00:16:50.733505+0000","last_clean":"2023-10-08T00:16:50.733505+0000","last_became_active":"2023-10-08T00:16:50.733111+0000","last_became_peered":"2023-10-08T00:16:50.733111+0000","last_unstale":"2023-10-08T00:16:50.733505+0000","last_undegraded":"2023-10-08T00:16:50.733505+0000","last_fullsized":"2023-10-08T00:16:50.733505+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:47:03.175925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.742155+0000","last_change":"2023-10-08T00:16:49.716983+0000","last_active":"2023-10-08T00:16:50.742155+0000","last_peered":"2023-10-08T00:16:50.742155+0000","last_clean":"2023-10-08T00:16:50.742155+0000","last_became_active":"2023-10-08T00:16:49.716090+0000","last_became_peered":"2023-10-08T00:16:49.716090+0000","last_unstale":"2023-10-08T00:16:50.742155+0000","last_undegraded":"2023-10-08T00:16:50.742155+0000","last_fullsized":"2023-10-08T00:16:50.742155+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:49:19.058724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.734418+0000","last_change":"2023-10-08T00:16:50.734358+0000","last_active":"2023-10-08T00:16:50.734418+0000","last_peered":"2023-10-08T00:16:50.734418+0000","last_clean":"2023-10-08T00:16:50.734418+0000","last_became_active":"2023-10-08T00:16:50.734137+0000","last_became_peered":"2023-10-08T00:16:50.734137+0000","last_unstale":"2023-10-08T00:16:50.734418+0000","last_undegraded":"2023-10-08T00:16:50.734418+0000","last_fullsized":"2023-10-08T00:16:50.734418+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:56:08.736930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.734899+0000","last_change":"2023-10-08T00:16:49.724560+0000","last_active":"2023-10-08T00:16:50.734899+0000","last_peered":"2023-10-08T00:16:50.734899+0000","last_clean":"2023-10-08T00:16:50.734899+0000","last_became_active":"2023-10-08T00:16:49.723939+0000","last_became_peered":"2023-10-08T00:16:49.723939+0000","last_unstale":"2023-10-08T00:16:50.734899+0000","last_undegraded":"2023-10-08T00:16:50.734899+0000","last_fullsized":"2023-10-08T00:16:50.734899+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:24:15.757109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.742293+0000","last_change":"2023-10-08T00:16:50.742232+0000","last_active":"2023-10-08T00:16:50.742293+0000","last_peered":"2023-10-08T00:16:50.742293+0000","last_clean":"2023-10-08T00:16:50.742293+0000","last_became_active":"2023-10-08T00:16:50.741995+0000","last_became_peered":"2023-10-08T00:16:50.741995+0000","last_unstale":"2023-10-08T00:16:50.742293+0000","last_undegraded":"2023-10-08T00:16:50.742293+0000","last_fullsized":"2023-10-08T00:16:50.742293+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:19:40.203536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.741013+0000","last_change":"2023-10-08T00:16:49.725331+0000","last_active":"2023-10-08T00:16:50.741013+0000","last_peered":"2023-10-08T00:16:50.741013+0000","last_clean":"2023-10-08T00:16:50.741013+0000","last_became_active":"2023-10-08T00:16:49.725134+0000","last_became_peered":"2023-10-08T00:16:49.725134+0000","last_unstale":"2023-10-08T00:16:50.741013+0000","last_undegraded":"2023-10-08T00:16:50.741013+0000","last_fullsized":"2023-10-08T00:16:50.741013+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:09:50.287365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.732293+0000","last_change":"2023-10-08T00:16:50.732104+0000","last_active":"2023-10-08T00:16:50.732293+0000","last_peered":"2023-10-08T00:16:50.732293+0000","last_clean":"2023-10-08T00:16:50.732293+0000","last_became_active":"2023-10-08T00:16:50.731869+0000","last_became_peered":"2023-10-08T00:16:50.731869+0000","last_unstale":"2023-10-08T00:16:50.732293+0000","last_undegraded":"2023-10-08T00:16:50.732293+0000","last_fullsized":"2023-10-08T00:16:50.732293+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:55:58.689766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"138'3","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.750127+0000","last_change":"2023-10-08T00:16:49.720487+0000","last_active":"2023-10-08T00:16:50.750127+0000","last_peered":"2023-10-08T00:16:50.750127+0000","last_clean":"2023-10-08T00:16:50.750127+0000","last_became_active":"2023-10-08T00:16:49.720290+0000","last_became_peered":"2023-10-08T00:16:49.720290+0000","last_unstale":"2023-10-08T00:16:50.750127+0000","last_undegraded":"2023-10-08T00:16:50.750127+0000","last_fullsized":"2023-10-08T00:16:50.750127+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:47:05.808379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.732596+0000","last_change":"2023-10-08T00:16:50.732318+0000","last_active":"2023-10-08T00:16:50.732596+0000","last_peered":"2023-10-08T00:16:50.732596+0000","last_clean":"2023-10-08T00:16:50.732596+0000","last_became_active":"2023-10-08T00:16:50.732057+0000","last_became_peered":"2023-10-08T00:16:50.732057+0000","last_unstale":"2023-10-08T00:16:50.732596+0000","last_undegraded":"2023-10-08T00:16:50.732596+0000","last_fullsized":"2023-10-08T00:16:50.732596+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:06:08.939435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"138'6","reported_seq":16,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.751411+0000","last_change":"2023-10-08T00:16:49.721242+0000","last_active":"2023-10-08T00:16:50.751411+0000","last_peered":"2023-10-08T00:16:50.751411+0000","last_clean":"2023-10-08T00:16:50.751411+0000","last_became_active":"2023-10-08T00:16:49.720775+0000","last_became_peered":"2023-10-08T00:16:49.720775+0000","last_unstale":"2023-10-08T00:16:50.751411+0000","last_undegraded":"2023-10-08T00:16:50.751411+0000","last_fullsized":"2023-10-08T00:16:50.751411+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:21:27.301699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.742432+0000","last_change":"2023-10-08T00:16:50.742395+0000","last_active":"2023-10-08T00:16:50.742432+0000","last_peered":"2023-10-08T00:16:50.742432+0000","last_clean":"2023-10-08T00:16:50.742432+0000","last_became_active":"2023-10-08T00:16:50.742141+0000","last_became_peered":"2023-10-08T00:16:50.742141+0000","last_unstale":"2023-10-08T00:16:50.742432+0000","last_undegraded":"2023-10-08T00:16:50.742432+0000","last_fullsized":"2023-10-08T00:16:50.742432+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:43:14.175754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"138'2","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.750291+0000","last_change":"2023-10-08T00:16:49.718190+0000","last_active":"2023-10-08T00:16:50.750291+0000","last_peered":"2023-10-08T00:16:50.750291+0000","last_clean":"2023-10-08T00:16:50.750291+0000","last_became_active":"2023-10-08T00:16:49.717794+0000","last_became_peered":"2023-10-08T00:16:49.717794+0000","last_unstale":"2023-10-08T00:16:50.750291+0000","last_undegraded":"2023-10-08T00:16:50.750291+0000","last_fullsized":"2023-10-08T00:16:50.750291+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:56:16.239039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.737577+0000","last_change":"2023-10-08T00:16:50.737418+0000","last_active":"2023-10-08T00:16:50.737577+0000","last_peered":"2023-10-08T00:16:50.737577+0000","last_clean":"2023-10-08T00:16:50.737577+0000","last_became_active":"2023-10-08T00:16:50.737186+0000","last_became_peered":"2023-10-08T00:16:50.737186+0000","last_unstale":"2023-10-08T00:16:50.737577+0000","last_undegraded":"2023-10-08T00:16:50.737577+0000","last_fullsized":"2023-10-08T00:16:50.737577+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:21:33.681095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.728407+0000","last_change":"2023-10-08T00:16:49.724096+0000","last_active":"2023-10-08T00:16:50.728407+0000","last_peered":"2023-10-08T00:16:50.728407+0000","last_clean":"2023-10-08T00:16:50.728407+0000","last_became_active":"2023-10-08T00:16:49.720406+0000","last_became_peered":"2023-10-08T00:16:49.720406+0000","last_unstale":"2023-10-08T00:16:50.728407+0000","last_undegraded":"2023-10-08T00:16:50.728407+0000","last_fullsized":"2023-10-08T00:16:50.728407+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:41:41.931737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.734543+0000","last_change":"2023-10-08T00:16:50.734288+0000","last_active":"2023-10-08T00:16:50.734543+0000","last_peered":"2023-10-08T00:16:50.734543+0000","last_clean":"2023-10-08T00:16:50.734543+0000","last_became_active":"2023-10-08T00:16:50.733473+0000","last_became_peered":"2023-10-08T00:16:50.733473+0000","last_unstale":"2023-10-08T00:16:50.734543+0000","last_undegraded":"2023-10-08T00:16:50.734543+0000","last_fullsized":"2023-10-08T00:16:50.734543+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:19:59.641079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"138'8","reported_seq":18,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.750685+0000","last_change":"2023-10-08T00:16:49.725499+0000","last_active":"2023-10-08T00:16:50.750685+0000","last_peered":"2023-10-08T00:16:50.750685+0000","last_clean":"2023-10-08T00:16:50.750685+0000","last_became_active":"2023-10-08T00:16:49.725129+0000","last_became_peered":"2023-10-08T00:16:49.725129+0000","last_unstale":"2023-10-08T00:16:50.750685+0000","last_undegraded":"2023-10-08T00:16:50.750685+0000","last_fullsized":"2023-10-08T00:16:50.750685+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:43:13.170777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.739803+0000","last_change":"2023-10-08T00:16:50.739723+0000","last_active":"2023-10-08T00:16:50.739803+0000","last_peered":"2023-10-08T00:16:50.739803+0000","last_clean":"2023-10-08T00:16:50.739803+0000","last_became_active":"2023-10-08T00:16:50.739510+0000","last_became_peered":"2023-10-08T00:16:50.739510+0000","last_unstale":"2023-10-08T00:16:50.739803+0000","last_undegraded":"2023-10-08T00:16:50.739803+0000","last_fullsized":"2023-10-08T00:16:50.739803+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:51:20.090559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.f","version":"138'1","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.746397+0000","last_change":"2023-10-08T00:16:49.714785+0000","last_active":"2023-10-08T00:16:50.746397+0000","last_peered":"2023-10-08T00:16:50.746397+0000","last_clean":"2023-10-08T00:16:50.746397+0000","last_became_active":"2023-10-08T00:16:49.714402+0000","last_became_peered":"2023-10-08T00:16:49.714402+0000","last_unstale":"2023-10-08T00:16:50.746397+0000","last_undegraded":"2023-10-08T00:16:50.746397+0000","last_fullsized":"2023-10-08T00:16:50.746397+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:41:40.129734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.737517+0000","last_change":"2023-10-08T00:16:50.737430+0000","last_active":"2023-10-08T00:16:50.737517+0000","last_peered":"2023-10-08T00:16:50.737517+0000","last_clean":"2023-10-08T00:16:50.737517+0000","last_became_active":"2023-10-08T00:16:50.737251+0000","last_became_peered":"2023-10-08T00:16:50.737251+0000","last_unstale":"2023-10-08T00:16:50.737517+0000","last_undegraded":"2023-10-08T00:16:50.737517+0000","last_fullsized":"2023-10-08T00:16:50.737517+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:55:58.689766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"138'2","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.750059+0000","last_change":"2023-10-08T00:16:49.717409+0000","last_active":"2023-10-08T00:16:50.750059+0000","last_peered":"2023-10-08T00:16:50.750059+0000","last_clean":"2023-10-08T00:16:50.750059+0000","last_became_active":"2023-10-08T00:16:49.717194+0000","last_became_peered":"2023-10-08T00:16:49.717194+0000","last_unstale":"2023-10-08T00:16:50.750059+0000","last_undegraded":"2023-10-08T00:16:50.750059+0000","last_fullsized":"2023-10-08T00:16:50.750059+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:41:42.950196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.734513+0000","last_change":"2023-10-08T00:16:50.734437+0000","last_active":"2023-10-08T00:16:50.734513+0000","last_peered":"2023-10-08T00:16:50.734513+0000","last_clean":"2023-10-08T00:16:50.734513+0000","last_became_active":"2023-10-08T00:16:50.734224+0000","last_became_peered":"2023-10-08T00:16:50.734224+0000","last_unstale":"2023-10-08T00:16:50.734513+0000","last_undegraded":"2023-10-08T00:16:50.734513+0000","last_fullsized":"2023-10-08T00:16:50.734513+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:52:51.647697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.d","version":"138'1","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.744283+0000","last_change":"2023-10-08T00:16:49.720369+0000","last_active":"2023-10-08T00:16:50.744283+0000","last_peered":"2023-10-08T00:16:50.744283+0000","last_clean":"2023-10-08T00:16:50.744283+0000","last_became_active":"2023-10-08T00:16:49.720143+0000","last_became_peered":"2023-10-08T00:16:49.720143+0000","last_unstale":"2023-10-08T00:16:50.744283+0000","last_undegraded":"2023-10-08T00:16:50.744283+0000","last_fullsized":"2023-10-08T00:16:50.744283+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:17:20.032034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.736803+0000","last_change":"2023-10-08T00:16:50.736753+0000","last_active":"2023-10-08T00:16:50.736803+0000","last_peered":"2023-10-08T00:16:50.736803+0000","last_clean":"2023-10-08T00:16:50.736803+0000","last_became_active":"2023-10-08T00:16:50.736529+0000","last_became_peered":"2023-10-08T00:16:50.736529+0000","last_unstale":"2023-10-08T00:16:50.736803+0000","last_undegraded":"2023-10-08T00:16:50.736803+0000","last_fullsized":"2023-10-08T00:16:50.736803+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:53:27.357114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.736560+0000","last_change":"2023-10-08T00:16:49.723203+0000","last_active":"2023-10-08T00:16:50.736560+0000","last_peered":"2023-10-08T00:16:50.736560+0000","last_clean":"2023-10-08T00:16:50.736560+0000","last_became_active":"2023-10-08T00:16:49.722933+0000","last_became_peered":"2023-10-08T00:16:49.722933+0000","last_unstale":"2023-10-08T00:16:50.736560+0000","last_undegraded":"2023-10-08T00:16:50.736560+0000","last_fullsized":"2023-10-08T00:16:50.736560+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:53:18.346097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.730742+0000","last_change":"2023-10-08T00:16:50.730670+0000","last_active":"2023-10-08T00:16:50.730742+0000","last_peered":"2023-10-08T00:16:50.730742+0000","last_clean":"2023-10-08T00:16:50.730742+0000","last_became_active":"2023-10-08T00:16:50.730428+0000","last_became_peered":"2023-10-08T00:16:50.730428+0000","last_unstale":"2023-10-08T00:16:50.730742+0000","last_undegraded":"2023-10-08T00:16:50.730742+0000","last_fullsized":"2023-10-08T00:16:50.730742+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:21:28.306676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"138'2","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.748614+0000","last_change":"2023-10-08T00:16:49.725381+0000","last_active":"2023-10-08T00:16:50.748614+0000","last_peered":"2023-10-08T00:16:50.748614+0000","last_clean":"2023-10-08T00:16:50.748614+0000","last_became_active":"2023-10-08T00:16:49.725123+0000","last_became_peered":"2023-10-08T00:16:49.725123+0000","last_unstale":"2023-10-08T00:16:50.748614+0000","last_undegraded":"2023-10-08T00:16:50.748614+0000","last_fullsized":"2023-10-08T00:16:50.748614+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:00:17.187757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.737498+0000","last_change":"2023-10-08T00:16:50.737046+0000","last_active":"2023-10-08T00:16:50.737498+0000","last_peered":"2023-10-08T00:16:50.737498+0000","last_clean":"2023-10-08T00:16:50.737498+0000","last_became_active":"2023-10-08T00:16:50.736660+0000","last_became_peered":"2023-10-08T00:16:50.736660+0000","last_unstale":"2023-10-08T00:16:50.737498+0000","last_undegraded":"2023-10-08T00:16:50.737498+0000","last_fullsized":"2023-10-08T00:16:50.737498+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:34:23.975157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.734442+0000","last_change":"2023-10-08T00:16:49.721861+0000","last_active":"2023-10-08T00:16:50.734442+0000","last_peered":"2023-10-08T00:16:50.734442+0000","last_clean":"2023-10-08T00:16:50.734442+0000","last_became_active":"2023-10-08T00:16:49.721622+0000","last_became_peered":"2023-10-08T00:16:49.721622+0000","last_unstale":"2023-10-08T00:16:50.734442+0000","last_undegraded":"2023-10-08T00:16:50.734442+0000","last_fullsized":"2023-10-08T00:16:50.734442+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:19:58.636102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.738658+0000","last_change":"2023-10-08T00:16:50.738603+0000","last_active":"2023-10-08T00:16:50.738658+0000","last_peered":"2023-10-08T00:16:50.738658+0000","last_clean":"2023-10-08T00:16:50.738658+0000","last_became_active":"2023-10-08T00:16:50.738389+0000","last_became_peered":"2023-10-08T00:16:50.738389+0000","last_unstale":"2023-10-08T00:16:50.738658+0000","last_undegraded":"2023-10-08T00:16:50.738658+0000","last_fullsized":"2023-10-08T00:16:50.738658+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:09:43.382366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.733900+0000","last_change":"2023-10-08T00:16:49.723222+0000","last_active":"2023-10-08T00:16:50.733900+0000","last_peered":"2023-10-08T00:16:50.733900+0000","last_clean":"2023-10-08T00:16:50.733900+0000","last_became_active":"2023-10-08T00:16:49.722490+0000","last_became_peered":"2023-10-08T00:16:49.722490+0000","last_unstale":"2023-10-08T00:16:50.733900+0000","last_undegraded":"2023-10-08T00:16:50.733900+0000","last_fullsized":"2023-10-08T00:16:50.733900+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:38:37.189574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.748486+0000","last_change":"2023-10-08T00:16:50.746715+0000","last_active":"2023-10-08T00:16:50.748486+0000","last_peered":"2023-10-08T00:16:50.748486+0000","last_clean":"2023-10-08T00:16:50.748486+0000","last_became_active":"2023-10-08T00:16:50.746323+0000","last_became_peered":"2023-10-08T00:16:50.746323+0000","last_unstale":"2023-10-08T00:16:50.748486+0000","last_undegraded":"2023-10-08T00:16:50.748486+0000","last_fullsized":"2023-10-08T00:16:50.748486+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:43:29.947771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.734999+0000","last_change":"2023-10-08T00:16:49.724702+0000","last_active":"2023-10-08T00:16:50.734999+0000","last_peered":"2023-10-08T00:16:50.734999+0000","last_clean":"2023-10-08T00:16:50.734999+0000","last_became_active":"2023-10-08T00:16:49.724254+0000","last_became_peered":"2023-10-08T00:16:49.724254+0000","last_unstale":"2023-10-08T00:16:50.734999+0000","last_undegraded":"2023-10-08T00:16:50.734999+0000","last_fullsized":"2023-10-08T00:16:50.734999+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:09:50.287365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.737968+0000","last_change":"2023-10-08T00:16:50.737903+0000","last_active":"2023-10-08T00:16:50.737968+0000","last_peered":"2023-10-08T00:16:50.737968+0000","last_clean":"2023-10-08T00:16:50.737968+0000","last_became_active":"2023-10-08T00:16:50.737683+0000","last_became_peered":"2023-10-08T00:16:50.737683+0000","last_unstale":"2023-10-08T00:16:50.737968+0000","last_undegraded":"2023-10-08T00:16:50.737968+0000","last_fullsized":"2023-10-08T00:16:50.737968+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:55:58.689766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"138'2","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.750614+0000","last_change":"2023-10-08T00:16:49.722975+0000","last_active":"2023-10-08T00:16:50.750614+0000","last_peered":"2023-10-08T00:16:50.750614+0000","last_clean":"2023-10-08T00:16:50.750614+0000","last_became_active":"2023-10-08T00:16:49.722744+0000","last_became_peered":"2023-10-08T00:16:49.722744+0000","last_unstale":"2023-10-08T00:16:50.750614+0000","last_undegraded":"2023-10-08T00:16:50.750614+0000","last_fullsized":"2023-10-08T00:16:50.750614+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:29:25.662070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.731662+0000","last_change":"2023-10-08T00:16:50.730913+0000","last_active":"2023-10-08T00:16:50.731662+0000","last_peered":"2023-10-08T00:16:50.731662+0000","last_clean":"2023-10-08T00:16:50.731662+0000","last_became_active":"2023-10-08T00:16:50.730693+0000","last_became_peered":"2023-10-08T00:16:50.730693+0000","last_unstale":"2023-10-08T00:16:50.731662+0000","last_undegraded":"2023-10-08T00:16:50.731662+0000","last_fullsized":"2023-10-08T00:16:50.731662+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:52:10.559787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.731535+0000","last_change":"2023-10-08T00:16:49.722398+0000","last_active":"2023-10-08T00:16:50.731535+0000","last_peered":"2023-10-08T00:16:50.731535+0000","last_clean":"2023-10-08T00:16:50.731535+0000","last_became_active":"2023-10-08T00:16:49.721743+0000","last_became_peered":"2023-10-08T00:16:49.721743+0000","last_unstale":"2023-10-08T00:16:50.731535+0000","last_undegraded":"2023-10-08T00:16:50.731535+0000","last_fullsized":"2023-10-08T00:16:50.731535+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:33:08.393982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.746115+0000","last_change":"2023-10-08T00:16:50.745889+0000","last_active":"2023-10-08T00:16:50.746115+0000","last_peered":"2023-10-08T00:16:50.746115+0000","last_clean":"2023-10-08T00:16:50.746115+0000","last_became_active":"2023-10-08T00:16:50.745332+0000","last_became_peered":"2023-10-08T00:16:50.745332+0000","last_unstale":"2023-10-08T00:16:50.746115+0000","last_undegraded":"2023-10-08T00:16:50.746115+0000","last_fullsized":"2023-10-08T00:16:50.746115+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:45:12.144679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.733802+0000","last_change":"2023-10-08T00:16:49.723551+0000","last_active":"2023-10-08T00:16:50.733802+0000","last_peered":"2023-10-08T00:16:50.733802+0000","last_clean":"2023-10-08T00:16:50.733802+0000","last_became_active":"2023-10-08T00:16:49.723120+0000","last_became_peered":"2023-10-08T00:16:49.723120+0000","last_unstale":"2023-10-08T00:16:50.733802+0000","last_undegraded":"2023-10-08T00:16:50.733802+0000","last_fullsized":"2023-10-08T00:16:50.733802+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:56:16.239039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.737026+0000","last_change":"2023-10-08T00:16:50.736952+0000","last_active":"2023-10-08T00:16:50.737026+0000","last_peered":"2023-10-08T00:16:50.737026+0000","last_clean":"2023-10-08T00:16:50.737026+0000","last_became_active":"2023-10-08T00:16:50.736785+0000","last_became_peered":"2023-10-08T00:16:50.736785+0000","last_unstale":"2023-10-08T00:16:50.737026+0000","last_undegraded":"2023-10-08T00:16:50.737026+0000","last_fullsized":"2023-10-08T00:16:50.737026+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:51:20.090559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.733998+0000","last_change":"2023-10-08T00:16:49.722921+0000","last_active":"2023-10-08T00:16:50.733998+0000","last_peered":"2023-10-08T00:16:50.733998+0000","last_clean":"2023-10-08T00:16:50.733998+0000","last_became_active":"2023-10-08T00:16:49.722188+0000","last_became_peered":"2023-10-08T00:16:49.722188+0000","last_unstale":"2023-10-08T00:16:50.733998+0000","last_undegraded":"2023-10-08T00:16:50.733998+0000","last_fullsized":"2023-10-08T00:16:50.733998+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:19:03.121126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.734768+0000","last_change":"2023-10-08T00:16:50.734696+0000","last_active":"2023-10-08T00:16:50.734768+0000","last_peered":"2023-10-08T00:16:50.734768+0000","last_clean":"2023-10-08T00:16:50.734768+0000","last_became_active":"2023-10-08T00:16:50.734460+0000","last_became_peered":"2023-10-08T00:16:50.734460+0000","last_unstale":"2023-10-08T00:16:50.734768+0000","last_undegraded":"2023-10-08T00:16:50.734768+0000","last_fullsized":"2023-10-08T00:16:50.734768+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:24:16.762086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.731894+0000","last_change":"2023-10-08T00:16:49.716168+0000","last_active":"2023-10-08T00:16:50.731894+0000","last_peered":"2023-10-08T00:16:50.731894+0000","last_clean":"2023-10-08T00:16:50.731894+0000","last_became_active":"2023-10-08T00:16:49.715935+0000","last_became_peered":"2023-10-08T00:16:49.715935+0000","last_unstale":"2023-10-08T00:16:50.731894+0000","last_undegraded":"2023-10-08T00:16:50.731894+0000","last_fullsized":"2023-10-08T00:16:50.731894+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:08:16.260647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.741800+0000","last_change":"2023-10-08T00:16:50.741598+0000","last_active":"2023-10-08T00:16:50.741800+0000","last_peered":"2023-10-08T00:16:50.741800+0000","last_clean":"2023-10-08T00:16:50.741800+0000","last_became_active":"2023-10-08T00:16:50.741387+0000","last_became_peered":"2023-10-08T00:16:50.741387+0000","last_unstale":"2023-10-08T00:16:50.741800+0000","last_undegraded":"2023-10-08T00:16:50.741800+0000","last_fullsized":"2023-10-08T00:16:50.741800+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:45:12.144679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.736649+0000","last_change":"2023-10-08T00:16:49.721058+0000","last_active":"2023-10-08T00:16:50.736649+0000","last_peered":"2023-10-08T00:16:50.736649+0000","last_clean":"2023-10-08T00:16:50.736649+0000","last_became_active":"2023-10-08T00:16:49.720822+0000","last_became_peered":"2023-10-08T00:16:49.720822+0000","last_unstale":"2023-10-08T00:16:50.736649+0000","last_undegraded":"2023-10-08T00:16:50.736649+0000","last_fullsized":"2023-10-08T00:16:50.736649+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:49:07.445902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.740418+0000","last_change":"2023-10-08T00:16:50.740365+0000","last_active":"2023-10-08T00:16:50.740418+0000","last_peered":"2023-10-08T00:16:50.740418+0000","last_clean":"2023-10-08T00:16:50.740418+0000","last_became_active":"2023-10-08T00:16:50.740176+0000","last_became_peered":"2023-10-08T00:16:50.740176+0000","last_unstale":"2023-10-08T00:16:50.740418+0000","last_undegraded":"2023-10-08T00:16:50.740418+0000","last_fullsized":"2023-10-08T00:16:50.740418+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:52:10.559787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.735646+0000","last_change":"2023-10-08T00:16:49.725348+0000","last_active":"2023-10-08T00:16:50.735646+0000","last_peered":"2023-10-08T00:16:50.735646+0000","last_clean":"2023-10-08T00:16:50.735646+0000","last_became_active":"2023-10-08T00:16:49.724834+0000","last_became_peered":"2023-10-08T00:16:49.724834+0000","last_unstale":"2023-10-08T00:16:50.735646+0000","last_undegraded":"2023-10-08T00:16:50.735646+0000","last_fullsized":"2023-10-08T00:16:50.735646+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:53:18.346097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.737917+0000","last_change":"2023-10-08T00:16:50.737000+0000","last_active":"2023-10-08T00:16:50.737917+0000","last_peered":"2023-10-08T00:16:50.737917+0000","last_clean":"2023-10-08T00:16:50.737917+0000","last_became_active":"2023-10-08T00:16:50.736826+0000","last_became_peered":"2023-10-08T00:16:50.736826+0000","last_unstale":"2023-10-08T00:16:50.737917+0000","last_undegraded":"2023-10-08T00:16:50.737917+0000","last_fullsized":"2023-10-08T00:16:50.737917+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:53:27.357114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.728767+0000","last_change":"2023-10-08T00:16:49.714631+0000","last_active":"2023-10-08T00:16:50.728767+0000","last_peered":"2023-10-08T00:16:50.728767+0000","last_clean":"2023-10-08T00:16:50.728767+0000","last_became_active":"2023-10-08T00:16:49.714103+0000","last_became_peered":"2023-10-08T00:16:49.714103+0000","last_unstale":"2023-10-08T00:16:50.728767+0000","last_undegraded":"2023-10-08T00:16:50.728767+0000","last_fullsized":"2023-10-08T00:16:50.728767+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:21:32.676119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.734141+0000","last_change":"2023-10-08T00:16:50.734075+0000","last_active":"2023-10-08T00:16:50.734141+0000","last_peered":"2023-10-08T00:16:50.734141+0000","last_clean":"2023-10-08T00:16:50.734141+0000","last_became_active":"2023-10-08T00:16:50.733829+0000","last_became_peered":"2023-10-08T00:16:50.733829+0000","last_unstale":"2023-10-08T00:16:50.734141+0000","last_undegraded":"2023-10-08T00:16:50.734141+0000","last_fullsized":"2023-10-08T00:16:50.734141+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:49.707449+0000","last_clean_scrub_stamp":"2023-10-08T00:16:49.707449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:24:49.153516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.727783+0000","last_change":"2023-10-08T00:16:49.715211+0000","last_active":"2023-10-08T00:16:50.727783+0000","last_peered":"2023-10-08T00:16:50.727783+0000","last_clean":"2023-10-08T00:16:50.727783+0000","last_became_active":"2023-10-08T00:16:49.714937+0000","last_became_peered":"2023-10-08T00:16:49.714937+0000","last_unstale":"2023-10-08T00:16:50.727783+0000","last_undegraded":"2023-10-08T00:16:50.727783+0000","last_fullsized":"2023-10-08T00:16:50.727783+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:16:48.702473+0000","last_clean_scrub_stamp":"2023-10-08T00:16:48.702473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:16:46.900489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":264,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T00:16:50.728981+0000","last_change":"2023-10-08T00:10:43.728490+0000","last_active":"2023-10-08T00:16:50.728981+0000","last_peered":"2023-10-08T00:16:50.728981+0000","last_clean":"2023-10-08T00:16:50.728981+0000","last_became_active":"2023-10-08T00:10:43.728341+0000","last_became_peered":"2023-10-08T00:10:43.728341+0000","last_unstale":"2023-10-08T00:16:50.728981+0000","last_undegraded":"2023-10-08T00:16:50.728981+0000","last_fullsized":"2023-10-08T00:16:50.728981+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_clean_scrub_stamp":"2023-10-08T00:10:42.716096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:06:31.035293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":754,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194558,"statfs":{"total":200000000,"available":199227752,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.53300000000000003,"15min":0.50700000000000001},"min":{"1min":0.39700000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.54300000000000004,"15min":0.54500000000000004},"min":{"1min":0.36099999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.5}]},{"osd":1,"last update":"Sun Oct 8 00:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56100000000000005,"15min":0.53700000000000003},"min":{"1min":0.38,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.72799999999999998,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.58699999999999997,"15min":0.56699999999999995},"min":{"1min":0.42399999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.78400000000000003,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.57099999999999995}]},{"osd":2,"last update":"Sun Oct 8 00:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55100000000000005,"15min":0.54000000000000004},"min":{"1min":0.34599999999999997,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.55200000000000005,"15min":0.52600000000000002},"min":{"1min":0.433,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77200000000000002,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.52200000000000002}]},{"osd":3,"last update":"Sun Oct 8 00:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54500000000000004,"15min":0.52300000000000002},"min":{"1min":0.35599999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.75600000000000001,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56899999999999995,"15min":0.55900000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.46999999999999997}]},{"osd":4,"last update":"Sun Oct 8 00:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.497,"15min":0.51700000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.65100000000000002,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.5,"5min":0.504,"15min":0.52400000000000002},"min":{"1min":0.315,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.75900000000000001,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.60799999999999998}]},{"osd":5,"last update":"Sun Oct 8 00:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.51500000000000001,"15min":0.54500000000000004},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.78000000000000003,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54300000000000004,"15min":0.57999999999999996},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.76900000000000002,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.65900000000000003}]},{"osd":6,"last update":"Sun Oct 8 00:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.57799999999999996,"15min":0.59399999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.74399999999999999,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58899999999999997,"15min":0.625},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.80100000000000005,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.63600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574924,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":746,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194566,"statfs":{"total":200000000,"available":199235700,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.51500000000000001,"15min":0.48699999999999999},"min":{"1min":0.39400000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.71199999999999997,"5min":1.012,"15min":1.012},"last":0.495},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.56999999999999995,"15min":0.66700000000000004},"min":{"1min":0.39900000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.0529999999999999,"5min":5.1230000000000002,"15min":5.1230000000000002},"last":0.45600000000000002}]},{"osd":1,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55200000000000005,"15min":0.52400000000000002},"min":{"1min":0.34799999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.78100000000000003,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57599999999999996,"15min":0.56599999999999995},"min":{"1min":0.42499999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.98999999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.48199999999999998}]},{"osd":2,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.54700000000000004,"15min":0.53200000000000003},"min":{"1min":0.439,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.753,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.39700000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.76900000000000002,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.39200000000000002}]},{"osd":3,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57299999999999995,"15min":0.54300000000000004},"min":{"1min":0.36699999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.84199999999999997,"5min":1.046,"15min":1.046},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56899999999999995,"15min":0.54500000000000004},"min":{"1min":0.39800000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.81000000000000005,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.442}]},{"osd":4,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.47299999999999998,"15min":0.496},"min":{"1min":0.371,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.55200000000000005,"15min":0.56299999999999994},"min":{"1min":0.38100000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.55200000000000005}]},{"osd":5,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.48299999999999998,"15min":0.48699999999999999},"min":{"1min":0.34499999999999997,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.69499999999999995,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.52300000000000002,"15min":0.55200000000000005},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.42599999999999999}]},{"osd":7,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.52700000000000002,"15min":0.54000000000000004},"min":{"1min":0.31,"5min":0.307,"15min":0.307},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.54500000000000004,"15min":0.54800000000000004},"min":{"1min":0.40200000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76000000000000001,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.50600000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1322,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193989,"statfs":{"total":200000000,"available":198645332,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:16:51 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.34200000000000003,"15min":0.312},"min":{"1min":0.215,"5min":0.20300000000000001,"15min":0.19500000000000001},"max":{"1min":0.55900000000000005,"5min":0.56899999999999995,"15min":0.56899999999999995},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.375,"5min":0.39400000000000002,"15min":0.379},"min":{"1min":0.29699999999999999,"5min":0.255,"15min":0.255},"max":{"1min":0.54600000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.316}]},{"osd":2,"last update":"Sun Oct 8 00:16:51 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.35199999999999998,"15min":0.33800000000000002},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.53200000000000003,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.38200000000000001,"15min":0.34599999999999997},"min":{"1min":0.26500000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.57399999999999995,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.35899999999999999}]},{"osd":3,"last update":"Sun Oct 8 00:16:51 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.39200000000000002,"15min":0.373},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.56999999999999995,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.35399999999999998,"15min":0.34399999999999997},"min":{"1min":0.255,"5min":0.222,"15min":0.222},"max":{"1min":0.502,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.32900000000000001}]},{"osd":4,"last update":"Sun Oct 8 00:16:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.56000000000000005,"15min":0.54200000000000004},"min":{"1min":0.34000000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77100000000000002,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.443},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.57799999999999996,"15min":0.56799999999999995},"min":{"1min":0.40899999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.754,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.46400000000000002}]},{"osd":5,"last update":"Sun Oct 8 00:16:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57099999999999995,"15min":0.54200000000000004},"min":{"1min":0.372,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.81000000000000005,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.58799999999999997,"15min":0.57599999999999996},"min":{"1min":0.41999999999999998,"5min":0.38300000000000001,"15min":0.30399999999999999},"max":{"1min":0.84999999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.47599999999999998}]},{"osd":6,"last update":"Sun Oct 8 00:16:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58499999999999996,"15min":0.56899999999999995},"min":{"1min":0.379,"5min":0.317,"15min":0.317},"max":{"1min":0.80400000000000005,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.434},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.58099999999999996,"15min":0.56399999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.80500000000000005,"5min":0.83899999999999997,"15min":0.84999999999999998},"last":0.38600000000000001}]},{"osd":7,"last update":"Sun Oct 8 00:16:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.58399999999999996,"15min":0.56599999999999995},"min":{"1min":0.40100000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.79600000000000004,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.61799999999999999,"15min":0.59399999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81899999999999995,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.52000000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574924,"num_pgs":14,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":746,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194565,"statfs":{"total":200000000,"available":199235076,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 00:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.34100000000000003,"15min":0.33200000000000002},"min":{"1min":0.20699999999999999,"5min":0.193,"15min":0.193},"max":{"1min":0.57599999999999996,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.252},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.35599999999999998,"15min":0.34799999999999998},"min":{"1min":0.218,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.57199999999999995,"5min":0.87,"15min":0.87},"last":0.22600000000000001}]},{"osd":2,"last update":"Sun Oct 8 00:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.38900000000000001,"15min":0.38800000000000001},"min":{"1min":0.27600000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.623,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.379,"5min":0.46400000000000002,"15min":0.72799999999999998},"min":{"1min":0.245,"5min":0.185,"15min":0.185},"max":{"1min":0.65600000000000003,"5min":12.019,"15min":12.019},"last":0.28599999999999998}]},{"osd":3,"last update":"Sun Oct 8 00:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.36799999999999999,"15min":0.36399999999999999},"min":{"1min":0.27500000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.54300000000000004,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.41399999999999998,"15min":0.39900000000000002},"min":{"1min":0.29999999999999999,"5min":0.246,"15min":0.246},"max":{"1min":0.67400000000000004,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.33300000000000002}]},{"osd":4,"last update":"Sun Oct 8 00:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61399999999999999,"15min":0.61699999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.90300000000000002,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.626,"15min":0.626},"min":{"1min":0.439,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.873,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.63700000000000001}]},{"osd":5,"last update":"Sun Oct 8 00:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.69099999999999995,"15min":0.96699999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.751,"5min":11.992000000000001,"15min":11.992000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60099999999999998,"15min":0.58899999999999997},"min":{"1min":0.46800000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.85599999999999998,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.64600000000000002}]},{"osd":6,"last update":"Sun Oct 8 00:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.59899999999999998,"15min":0.58899999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.88100000000000001,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.69599999999999995,"15min":0.94799999999999995},"min":{"1min":0.41299999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84499999999999997,"5min":11.927,"15min":11.927},"last":0.55400000000000005}]},{"osd":7,"last update":"Sun Oct 8 00:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.70799999999999996,"15min":0.96899999999999997},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.84099999999999997,"5min":11.965,"15min":11.965},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.623,"5min":0.626,"15min":0.624},"min":{"1min":0.42699999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81100000000000005,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.59499999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574924,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":753,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194559,"statfs":{"total":200000000,"available":199228802,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.33600000000000002,"15min":0.38400000000000001},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.53400000000000003,"5min":0.72399999999999998,"15min":2.5139999999999998},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.34100000000000003,"15min":0.39400000000000002},"min":{"1min":0.22700000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.55600000000000005,"5min":0.56399999999999995,"15min":2.1309999999999998},"last":0.26000000000000001}]},{"osd":1,"last update":"Sun Oct 8 00:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.34899999999999998,"15min":0.38800000000000001},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":2.2639999999999998},"last":0.23000000000000001},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.36799999999999999,"15min":0.42699999999999999},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":2.5379999999999998},"last":0.32300000000000001}]},{"osd":3,"last update":"Sun Oct 8 00:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.38400000000000001,"15min":0.42399999999999999},"min":{"1min":0.28100000000000003,"5min":0.254,"15min":0.254},"max":{"1min":0.61599999999999999,"5min":0.61899999999999999,"15min":2.2890000000000001},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40100000000000002,"15min":0.45700000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":2.5619999999999998},"last":0.40000000000000002}]},{"osd":4,"last update":"Sun Oct 8 00:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56200000000000006,"15min":0.60999999999999999},"min":{"1min":0.38200000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.76600000000000001,"5min":1.073,"15min":2.488},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58799999999999997,"15min":0.60499999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.85899999999999999,"5min":0.93600000000000005,"15min":2.073},"last":0.42099999999999999}]},{"osd":5,"last update":"Sun Oct 8 00:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.60899999999999999,"15min":0.65100000000000002},"min":{"1min":0.374,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.82199999999999995,"5min":1.2130000000000001,"15min":2.2069999999999999},"last":0.435},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.629,"15min":0.65900000000000003},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.39400000000000002},"max":{"1min":0.94199999999999995,"5min":1.0369999999999999,"15min":2.456},"last":0.54600000000000004}]},{"osd":6,"last update":"Sun Oct 8 00:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59599999999999997,"15min":0.61899999999999999},"min":{"1min":0.39200000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.93000000000000005,"5min":1.05,"15min":2.0339999999999998},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60599999999999998,"15min":0.61299999999999999},"min":{"1min":0.439,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.873,"5min":1.0940000000000001,"15min":2.1699999999999999},"last":0.497}]},{"osd":7,"last update":"Sun Oct 8 00:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.61199999999999999,"15min":0.63300000000000001},"min":{"1min":0.42999999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.95999999999999996,"5min":1.0229999999999999,"15min":2.1030000000000002},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60999999999999999,"15min":0.627},"min":{"1min":0.443,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.90500000000000003,"5min":1.0569999999999999,"15min":2.2349999999999999},"last":0.56299999999999994}]}]},{"osd":3,"up_from":13,"seq":55834574924,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":746,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194566,"statfs":{"total":200000000,"available":199235700,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:16:49 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.32000000000000001,"15min":0.36399999999999999},"min":{"1min":0.218,"5min":0.192,"15min":0.192},"max":{"1min":0.55200000000000005,"5min":0.71199999999999997,"15min":1.46},"last":0.218},{"interface":"front","average":{"1min":0.32000000000000001,"5min":0.33200000000000002,"15min":0.34000000000000002},"min":{"1min":0.26500000000000001,"5min":0.192,"15min":0.192},"max":{"1min":0.42399999999999999,"5min":0.504,"15min":0.54800000000000004},"last":0.314}]},{"osd":1,"last update":"Sun Oct 8 00:16:49 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.34200000000000003,"15min":0.754},"min":{"1min":0.23699999999999999,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.46200000000000002,"5min":0.57599999999999996,"15min":12.351000000000001},"last":0.247},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.36399999999999999,"15min":0.36499999999999999},"min":{"1min":0.248,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.54200000000000004,"5min":0.60199999999999998,"15min":0.71999999999999997},"last":0.34599999999999997}]},{"osd":2,"last update":"Sun Oct 8 00:16:49 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.38100000000000001,"15min":0.41399999999999998},"min":{"1min":0.30099999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.48999999999999999,"5min":0.70499999999999996,"15min":1.5489999999999999},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.37,"5min":0.39000000000000001,"15min":0.38700000000000001},"min":{"1min":0.28299999999999997,"5min":0.26800000000000002,"15min":0.246},"max":{"1min":0.45500000000000002,"5min":0.64700000000000002,"15min":0.753},"last":0.30099999999999999}]},{"osd":4,"last update":"Sun Oct 8 00:16:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59999999999999998,"15min":0.95099999999999996},"min":{"1min":0.39600000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.80000000000000004,"5min":1.024,"15min":11.333},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61599999999999999,"15min":0.97999999999999998},"min":{"1min":0.51000000000000001,"5min":0.376,"15min":0.36099999999999999},"max":{"1min":0.77800000000000002,"5min":0.94899999999999995,"15min":12.321999999999999},"last":0.54400000000000004}]},{"osd":5,"last update":"Sun Oct 8 00:16:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60199999999999998,"15min":0.98799999999999999},"min":{"1min":0.41099999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.73499999999999999,"5min":0.92000000000000004,"15min":12.291},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.622,"15min":0.60499999999999998},"min":{"1min":0.45600000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.73199999999999998,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.59599999999999997}]},{"osd":6,"last update":"Sun Oct 8 00:16:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58099999999999996,"15min":0.91800000000000004},"min":{"1min":0.442,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.83599999999999997,"5min":0.96199999999999997,"15min":11.288},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.59199999999999997,"15min":0.96999999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.73599999999999999,"5min":1.002,"15min":12.253},"last":0.56100000000000005}]},{"osd":7,"last update":"Sun Oct 8 00:16:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.60899999999999999,"15min":0.58199999999999996},"min":{"1min":0.47699999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80800000000000005,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.61199999999999999,"15min":0.95299999999999996},"min":{"1min":0.46200000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82599999999999996,"5min":1.381,"15min":11.363},"last":0.46999999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574924,"num_pgs":33,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":754,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194558,"statfs":{"total":200000000,"available":199227786,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.56899999999999995,"15min":0.55700000000000005},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.78100000000000003,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57399999999999995,"15min":0.58399999999999996},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.72899999999999998,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.58499999999999996}]},{"osd":1,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.52500000000000002,"15min":0.52100000000000002},"min":{"1min":0.33900000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.53100000000000003,"15min":0.53400000000000003},"min":{"1min":0.314,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.80400000000000005,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.44}]},{"osd":2,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.51500000000000001,"15min":0.51900000000000002},"min":{"1min":0.41699999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.68700000000000006,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.52400000000000002,"15min":0.51300000000000001},"min":{"1min":0.374,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.42399999999999999}]},{"osd":3,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.53700000000000003,"15min":0.54000000000000004},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.79800000000000004,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.53500000000000003,"15min":0.53400000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.45500000000000002}]},{"osd":5,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.46600000000000003,"15min":0.49399999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.499},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.495,"15min":0.51000000000000001},"min":{"1min":0.26400000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.48899999999999999}]},{"osd":6,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.47199999999999998,"15min":0.48799999999999999},"min":{"1min":0.32200000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.65000000000000002,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.371},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52200000000000002,"15min":0.55200000000000005},"min":{"1min":0.35399999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.55200000000000005}]},{"osd":7,"last update":"Sun Oct 8 00:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.53400000000000003,"15min":0.55100000000000005},"min":{"1min":0.35399999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.79700000000000004,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.56899999999999995,"15min":0.57999999999999996},"min":{"1min":0.45500000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.77500000000000002,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.52700000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574924,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1329,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193983,"statfs":{"total":200000000,"available":198638952,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48599999999999999},"min":{"1min":0.28799999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.66700000000000004,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53800000000000003,"15min":0.55700000000000005},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.70999999999999996,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.44400000000000001}]},{"osd":1,"last update":"Sun Oct 8 00:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.39000000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.71799999999999997,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50700000000000001,"15min":0.52300000000000002},"min":{"1min":0.33100000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70699999999999996,"5min":0.755,"15min":0.755},"last":0.48799999999999999}]},{"osd":2,"last update":"Sun Oct 8 00:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.52300000000000002,"15min":0.52900000000000003},"min":{"1min":0.39500000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.64700000000000002,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.376},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.55000000000000004,"15min":0.56699999999999995},"min":{"1min":0.33700000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.68799999999999994,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.46400000000000002}]},{"osd":3,"last update":"Sun Oct 8 00:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.55100000000000005,"15min":0.54700000000000004},"min":{"1min":0.35499999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.73199999999999998,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.55800000000000005,"15min":0.57599999999999996},"min":{"1min":0.38,"5min":0.371,"15min":0.371},"max":{"1min":0.71999999999999997,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.42699999999999999}]},{"osd":4,"last update":"Sun Oct 8 00:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.46999999999999997,"15min":0.47399999999999998},"min":{"1min":0.312,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.69399999999999995,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.51300000000000001,"15min":0.51500000000000001},"min":{"1min":0.40000000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.68300000000000005,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.57999999999999996}]},{"osd":6,"last update":"Sun Oct 8 00:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.505,"15min":0.501},"min":{"1min":0.36299999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.63200000000000001,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.505,"15min":0.496},"min":{"1min":0.36199999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68300000000000005,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.56499999999999995}]},{"osd":7,"last update":"Sun Oct 8 00:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52600000000000002,"15min":0.51900000000000002},"min":{"1min":0.379,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.66600000000000004,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55600000000000005,"15min":0.55700000000000005},"min":{"1min":0.34499999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.69599999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.59099999999999997}]}]}],"pool_statfs":[]}} 2023-10-08T00:16:55.152 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T00:16:55.153 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-08T00:16:55.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.305+0000 7fd31cbfb700 1 Processor -- start 2023-10-08T00:16:55.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.305+0000 7fd31cbfb700 1 -- start start 2023-10-08T00:16:55.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.305+0000 7fd31cbfb700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd3180aeea0 0x7fd3180cbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:55.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.305+0000 7fd31cbfb700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd3180af270 0x7fd3180c2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:55.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.305+0000 7fd31cbfb700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3180cba20 con 0x7fd3180c2e40 2023-10-08T00:16:55.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.305+0000 7fd31cbfb700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3180c34c0 con 0x7fd3180aeea0 2023-10-08T00:16:55.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.305+0000 7fd31cbfb700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3180c32e0 con 0x7fd3180af270 2023-10-08T00:16:55.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.306+0000 7fd316d9d700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fd3180c2e40 0x7fd3180c7c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:46850/0 (socket says 172.21.15.99:46850) 2023-10-08T00:16:55.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.306+0000 7fd316d9d700 1 -- 172.21.15.99:0/1957804181 learned_addr learned my addr 172.21.15.99:0/1957804181 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:55.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.306+0000 7fd30dd9b700 1 --2- 172.21.15.99:0/1957804181 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd3180af270 0x7fd3180c2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:55.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.306+0000 7fd31659c700 1 --2- 172.21.15.99:0/1957804181 >> v2:172.21.15.191:3300/0 conn(0x7fd3180aeea0 0x7fd3180cbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:55.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.306+0000 7fd315d9b700 1 -- 172.21.15.99:0/1957804181 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4186910975 0 0) 0x7fd3180cba20 con 0x7fd3180c2e40 2023-10-08T00:16:55.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.306+0000 7fd315d9b700 1 -- 172.21.15.99:0/1957804181 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd304003570 con 0x7fd3180c2e40 2023-10-08T00:16:55.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.306+0000 7fd30dd9b700 1 -- 172.21.15.99:0/1957804181 >> v2:172.21.15.191:3300/0 conn(0x7fd3180aeea0 msgr2=0x7fd3180cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:55.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.306+0000 7fd30dd9b700 1 --2- 172.21.15.99:0/1957804181 >> v2:172.21.15.191:3300/0 conn(0x7fd3180aeea0 0x7fd3180cbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:55.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.306+0000 7fd30dd9b700 1 -- 172.21.15.99:0/1957804181 >> v1:172.21.15.99:6789/0 conn(0x7fd3180c2e40 legacy=0x7fd3180c7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:55.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.306+0000 7fd315d9b700 1 -- 172.21.15.99:0/1957804181 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 969708914 0 0) 0x7fd304003570 con 0x7fd3180c2e40 2023-10-08T00:16:55.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.306+0000 7fd30dd9b700 1 -- 172.21.15.99:0/1957804181 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd31815b690 con 0x7fd3180af270 2023-10-08T00:16:55.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.306+0000 7fd30dd9b700 1 --2- 172.21.15.99:0/1957804181 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd3180af270 0x7fd3180c2900 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd3000098e0 tx=0x7fd3000091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=130fefdeaa5a8705 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:55.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.306+0000 7fd315d9b700 1 -- 172.21.15.99:0/1957804181 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3100017f0 con 0x7fd3180af270 2023-10-08T00:16:55.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.306+0000 7fd315d9b700 1 -- 172.21.15.99:0/1957804181 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd300011420 con 0x7fd3180af270 2023-10-08T00:16:55.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.306+0000 7fd315d9b700 1 -- 172.21.15.99:0/1957804181 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3000117c0 con 0x7fd3180af270 2023-10-08T00:16:55.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.307+0000 7fd31cbfb700 1 -- 172.21.15.99:0/1957804181 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd3180af270 msgr2=0x7fd3180c2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:55.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.307+0000 7fd31cbfb700 1 --2- 172.21.15.99:0/1957804181 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd3180af270 0x7fd3180c2900 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd3000098e0 tx=0x7fd3000091a0 comp rx=0 tx=0).stop 2023-10-08T00:16:55.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.307+0000 7fd31cbfb700 1 -- 172.21.15.99:0/1957804181 shutdown_connections 2023-10-08T00:16:55.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.307+0000 7fd31cbfb700 1 --2- 172.21.15.99:0/1957804181 >> v2:172.21.15.191:3300/0 conn(0x7fd3180aeea0 0x7fd3180cbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:55.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.307+0000 7fd31cbfb700 1 --2- 172.21.15.99:0/1957804181 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd3180af270 0x7fd3180c2900 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:55.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.307+0000 7fd31cbfb700 1 -- 172.21.15.99:0/1957804181 >> 172.21.15.99:0/1957804181 conn(0x7fd318067660 msgr2=0x7fd318067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:55.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.307+0000 7fd31cbfb700 1 -- 172.21.15.99:0/1957804181 shutdown_connections 2023-10-08T00:16:55.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.307+0000 7fd31cbfb700 1 -- 172.21.15.99:0/1957804181 wait complete. 2023-10-08T00:16:55.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.308+0000 7fd31cbfb700 1 Processor -- start 2023-10-08T00:16:55.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.308+0000 7fd31cbfb700 1 -- start start 2023-10-08T00:16:55.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.309+0000 7fd31cbfb700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3180aeea0 0x7fd3180b0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:55.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.309+0000 7fd31cbfb700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd3180af270 0x7fd3181438a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:55.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.309+0000 7fd31cbfb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3180c2e40 0x7fd31814eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:55.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.309+0000 7fd31cbfb700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3181425a0 con 0x7fd3180c2e40 2023-10-08T00:16:55.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.309+0000 7fd31cbfb700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd318141920 con 0x7fd3180aeea0 2023-10-08T00:16:55.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.309+0000 7fd31cbfb700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd318141c00 con 0x7fd3180af270 2023-10-08T00:16:55.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.309+0000 7fd316d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3180c2e40 0x7fd31814eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:55.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.309+0000 7fd316d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3180c2e40 0x7fd31814eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55974/0 (socket says 172.21.15.99:55974) 2023-10-08T00:16:55.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.309+0000 7fd316d9d700 1 -- 172.21.15.99:0/3106599674 learned_addr learned my addr 172.21.15.99:0/3106599674 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:55.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.309+0000 7fd31659c700 1 --2- 172.21.15.99:0/3106599674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3180aeea0 0x7fd3180b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:55.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.309+0000 7fd30dd9b700 1 --2- 172.21.15.99:0/3106599674 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd3180af270 0x7fd3181438a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:55.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.309+0000 7fd316d9d700 1 -- 172.21.15.99:0/3106599674 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd3180af270 msgr2=0x7fd3181438a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:55.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.309+0000 7fd316d9d700 1 --2- 172.21.15.99:0/3106599674 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd3180af270 0x7fd3181438a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:55.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.309+0000 7fd316d9d700 1 -- 172.21.15.99:0/3106599674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3180aeea0 msgr2=0x7fd3180b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:55.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.309+0000 7fd316d9d700 1 --2- 172.21.15.99:0/3106599674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3180aeea0 0x7fd3180b0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:55.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.309+0000 7fd316d9d700 1 -- 172.21.15.99:0/3106599674 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd300009420 con 0x7fd3180c2e40 2023-10-08T00:16:55.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.309+0000 7fd316d9d700 1 --2- 172.21.15.99:0/3106599674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3180c2e40 0x7fd31814eaa0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fd31000cae0 tx=0x7fd310007ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:55.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.310+0000 7fd30effd700 1 -- 172.21.15.99:0/3106599674 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd310004dc0 con 0x7fd3180c2e40 2023-10-08T00:16:55.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.310+0000 7fd30effd700 1 -- 172.21.15.99:0/3106599674 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd31000a5d0 con 0x7fd3180c2e40 2023-10-08T00:16:55.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.310+0000 7fd31cbfb700 1 -- 172.21.15.99:0/3106599674 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd318141ee0 con 0x7fd3180c2e40 2023-10-08T00:16:55.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.310+0000 7fd30effd700 1 -- 172.21.15.99:0/3106599674 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd31000a990 con 0x7fd3180c2e40 2023-10-08T00:16:55.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.310+0000 7fd31cbfb700 1 -- 172.21.15.99:0/3106599674 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd318141540 con 0x7fd3180c2e40 2023-10-08T00:16:55.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.311+0000 7fd31cbfb700 1 -- 172.21.15.99:0/3106599674 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3181426e0 con 0x7fd3180c2e40 2023-10-08T00:16:55.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.311+0000 7fd30effd700 1 -- 172.21.15.99:0/3106599674 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd310007150 con 0x7fd3180c2e40 2023-10-08T00:16:55.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.311+0000 7fd30effd700 1 --2- 172.21.15.99:0/3106599674 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd2f8066f30 0x7fd2f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:55.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.311+0000 7fd30effd700 1 -- 172.21.15.99:0/3106599674 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd310083ec0 con 0x7fd3180c2e40 2023-10-08T00:16:55.321 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.312+0000 7fd31659c700 1 --2- 172.21.15.99:0/3106599674 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd2f8066f30 0x7fd2f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:55.321 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.312+0000 7fd31659c700 1 --2- 172.21.15.99:0/3106599674 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd2f8066f30 0x7fd2f80693e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd308009db0 tx=0x7fd3080091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:55.321 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.315+0000 7fd30effd700 1 -- 172.21.15.99:0/3106599674 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd3181426e0 con 0x7fd3180c2e40 2023-10-08T00:16:55.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.573+0000 7fd31cbfb700 1 -- 172.21.15.99:0/3106599674 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fd3181426e0 con 0x7fd3180c2e40 2023-10-08T00:16:55.577 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.576+0000 7fd30effd700 1 -- 172.21.15.99:0/3106599674 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+0 (secure 0 0 0) 0x7fd3181426e0 con 0x7fd3180c2e40 2023-10-08T00:16:55.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.578+0000 7fd31cbfb700 1 -- 172.21.15.99:0/3106599674 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd2f8066f30 msgr2=0x7fd2f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:55.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.578+0000 7fd31cbfb700 1 --2- 172.21.15.99:0/3106599674 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd2f8066f30 0x7fd2f80693e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd308009db0 tx=0x7fd3080091a0 comp rx=0 tx=0).stop 2023-10-08T00:16:55.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.578+0000 7fd31cbfb700 1 -- 172.21.15.99:0/3106599674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3180c2e40 msgr2=0x7fd31814eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:55.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.578+0000 7fd31cbfb700 1 --2- 172.21.15.99:0/3106599674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3180c2e40 0x7fd31814eaa0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fd31000cae0 tx=0x7fd310007ca0 comp rx=0 tx=0).stop 2023-10-08T00:16:55.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.578+0000 7fd31cbfb700 1 -- 172.21.15.99:0/3106599674 shutdown_connections 2023-10-08T00:16:55.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.578+0000 7fd31cbfb700 1 --2- 172.21.15.99:0/3106599674 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd2f8066f30 0x7fd2f80693e0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:55.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.578+0000 7fd31cbfb700 1 --2- 172.21.15.99:0/3106599674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3180aeea0 0x7fd3180b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:55.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.578+0000 7fd31cbfb700 1 --2- 172.21.15.99:0/3106599674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3180c2e40 0x7fd31814eaa0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:55.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.578+0000 7fd31cbfb700 1 --2- 172.21.15.99:0/3106599674 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd3180af270 0x7fd3181438a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:55.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.579+0000 7fd31cbfb700 1 -- 172.21.15.99:0/3106599674 >> 172.21.15.99:0/3106599674 conn(0x7fd318067660 msgr2=0x7fd3180c7470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:55.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.579+0000 7fd31cbfb700 1 -- 172.21.15.99:0/3106599674 shutdown_connections 2023-10-08T00:16:55.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.579+0000 7fd31cbfb700 1 -- 172.21.15.99:0/3106599674 wait complete. 2023-10-08T00:16:55.583 INFO:teuthology.orchestra.run.smithi099.stderr:updated caps for client.0 2023-10-08T00:16:55.591 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:16:55.705 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.704+0000 7ff9ac8da700 1 Processor -- start 2023-10-08T00:16:55.705 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.704+0000 7ff9ac8da700 1 -- start start 2023-10-08T00:16:55.708 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.705+0000 7ff9ac8da700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9a815a1e0 0x7ff9a80cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:55.708 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.705+0000 7ff9ac8da700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff9a80b3430 0x7ff9a80cdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:55.708 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.705+0000 7ff9ac8da700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9a815db80 con 0x7ff9a8159e10 2023-10-08T00:16:55.708 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.705+0000 7ff9ac8da700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9a815e800 con 0x7ff9a80b3430 2023-10-08T00:16:55.708 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.705+0000 7ff9ac8da700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a815de60 con 0x7ff9a815a1e0 2023-10-08T00:16:55.708 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.705+0000 7ff9a6d9d700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff9a80b3430 0x7ff9a80cdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:55.708 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.705+0000 7ff9a6d9d700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff9a80b3430 0x7ff9a80cdc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55980/0 (socket says 172.21.15.99:55980) 2023-10-08T00:16:55.708 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.706+0000 7ff9a6d9d700 1 -- 172.21.15.99:0/3911774253 learned_addr learned my addr 172.21.15.99:0/3911774253 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:55.708 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.706+0000 7ff9a5d9b700 1 --2- 172.21.15.99:0/3911774253 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9a815a1e0 0x7ff9a80cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:55.709 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.706+0000 7ff9a6d9d700 1 -- 172.21.15.99:0/3911774253 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9a815a1e0 msgr2=0x7ff9a80cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:55.709 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.706+0000 7ff9a6d9d700 1 --2- 172.21.15.99:0/3911774253 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9a815a1e0 0x7ff9a80cd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:55.709 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.706+0000 7ff9a6d9d700 1 -- 172.21.15.99:0/3911774253 >> v1:172.21.15.191:6789/0 conn(0x7ff9a8159e10 legacy=0x7ff9a80b2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:55.709 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.706+0000 7ff9a6d9d700 1 -- 172.21.15.99:0/3911774253 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a815e140 con 0x7ff9a80b3430 2023-10-08T00:16:55.709 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.706+0000 7ff9a5d9b700 1 --2- 172.21.15.99:0/3911774253 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9a815a1e0 0x7ff9a80cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:55.709 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.706+0000 7ff9a6d9d700 1 --2- 172.21.15.99:0/3911774253 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9a80b3430 0x7ff9a80cdc80 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0009f20 tx=0x7ff9a0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eea827df09066a66 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:55.709 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.706+0000 7ff9a559a700 1 -- 172.21.15.99:0/3911774253 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a000e070 con 0x7ff9a80b3430 2023-10-08T00:16:55.709 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.706+0000 7ff9a559a700 1 -- 172.21.15.99:0/3911774253 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a000ba80 con 0x7ff9a80b3430 2023-10-08T00:16:55.709 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.706+0000 7ff9a559a700 1 -- 172.21.15.99:0/3911774253 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a00092e0 con 0x7ff9a80b3430 2023-10-08T00:16:55.709 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.707+0000 7ff9ac8da700 1 -- 172.21.15.99:0/3911774253 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9a80b3430 msgr2=0x7ff9a80cdc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:55.709 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.707+0000 7ff9ac8da700 1 --2- 172.21.15.99:0/3911774253 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9a80b3430 0x7ff9a80cdc80 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0009f20 tx=0x7ff9a0009840 comp rx=0 tx=0).stop 2023-10-08T00:16:55.710 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.707+0000 7ff9ac8da700 1 -- 172.21.15.99:0/3911774253 shutdown_connections 2023-10-08T00:16:55.710 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.707+0000 7ff9ac8da700 1 --2- 172.21.15.99:0/3911774253 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9a80b3430 0x7ff9a80cdc80 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:55.710 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.707+0000 7ff9ac8da700 1 --2- 172.21.15.99:0/3911774253 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9a815a1e0 0x7ff9a80cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:55.710 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.707+0000 7ff9ac8da700 1 -- 172.21.15.99:0/3911774253 >> 172.21.15.99:0/3911774253 conn(0x7ff9a8067660 msgr2=0x7ff9a8067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:55.710 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.707+0000 7ff9ac8da700 1 -- 172.21.15.99:0/3911774253 shutdown_connections 2023-10-08T00:16:55.710 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.707+0000 7ff9ac8da700 1 -- 172.21.15.99:0/3911774253 wait complete. 2023-10-08T00:16:55.713 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.708+0000 7ff9ac8da700 1 Processor -- start 2023-10-08T00:16:55.713 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.708+0000 7ff9ac8da700 1 -- start start 2023-10-08T00:16:55.713 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.708+0000 7ff9ac8da700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9a80b3430 0x7ff9a80c9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:55.713 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.709+0000 7ff9ac8da700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9a8159e10 0x7ff9a80c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:55.713 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.709+0000 7ff9ac8da700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9a815a1e0 0x7ff9a80c4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:55.713 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.709+0000 7ff9ac8da700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a80c6770 con 0x7ff9a815a1e0 2023-10-08T00:16:55.713 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.709+0000 7ff9ac8da700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a80ca0c0 con 0x7ff9a8159e10 2023-10-08T00:16:55.713 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.709+0000 7ff9ac8da700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a8157250 con 0x7ff9a80b3430 2023-10-08T00:16:55.714 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.709+0000 7ff9a6d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9a815a1e0 0x7ff9a80c4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:55.714 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.709+0000 7ff9a6d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9a815a1e0 0x7ff9a80c4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55988/0 (socket says 172.21.15.99:55988) 2023-10-08T00:16:55.714 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.709+0000 7ff9a6d9d700 1 -- 172.21.15.99:0/555486365 learned_addr learned my addr 172.21.15.99:0/555486365 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:55.714 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.709+0000 7ff9a5d9b700 1 --2- 172.21.15.99:0/555486365 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9a8159e10 0x7ff9a80c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:55.714 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.709+0000 7ff9a659c700 1 --2- 172.21.15.99:0/555486365 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9a80b3430 0x7ff9a80c9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:55.714 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.709+0000 7ff9a6d9d700 1 -- 172.21.15.99:0/555486365 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9a80b3430 msgr2=0x7ff9a80c9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:55.714 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.709+0000 7ff9a6d9d700 1 --2- 172.21.15.99:0/555486365 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9a80b3430 0x7ff9a80c9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:55.714 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.709+0000 7ff9a6d9d700 1 -- 172.21.15.99:0/555486365 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9a8159e10 msgr2=0x7ff9a80c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:55.715 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.709+0000 7ff9a6d9d700 1 --2- 172.21.15.99:0/555486365 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9a8159e10 0x7ff9a80c9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:55.715 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.709+0000 7ff9a6d9d700 1 -- 172.21.15.99:0/555486365 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a0009ae0 con 0x7ff9a815a1e0 2023-10-08T00:16:55.715 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.709+0000 7ff9a5d9b700 1 --2- 172.21.15.99:0/555486365 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9a8159e10 0x7ff9a80c9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:55.715 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.709+0000 7ff9a6d9d700 1 --2- 172.21.15.99:0/555486365 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9a815a1e0 0x7ff9a80c4ce0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7ff9a000b6c0 tx=0x7ff9a0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:55.715 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.709+0000 7ff9967fc700 1 -- 172.21.15.99:0/555486365 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a00032f0 con 0x7ff9a815a1e0 2023-10-08T00:16:55.715 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.709+0000 7ff9967fc700 1 -- 172.21.15.99:0/555486365 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a0008e70 con 0x7ff9a815a1e0 2023-10-08T00:16:55.715 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.709+0000 7ff9ac8da700 1 -- 172.21.15.99:0/555486365 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9a80ca320 con 0x7ff9a815a1e0 2023-10-08T00:16:55.715 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.709+0000 7ff9967fc700 1 -- 172.21.15.99:0/555486365 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0026670 con 0x7ff9a815a1e0 2023-10-08T00:16:55.715 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.709+0000 7ff9ac8da700 1 -- 172.21.15.99:0/555486365 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9a80ca8e0 con 0x7ff9a815a1e0 2023-10-08T00:16:55.715 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.710+0000 7ff9ac8da700 1 -- 172.21.15.99:0/555486365 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9a804c3c0 con 0x7ff9a815a1e0 2023-10-08T00:16:55.716 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.711+0000 7ff9967fc700 1 -- 172.21.15.99:0/555486365 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff9a0003040 con 0x7ff9a815a1e0 2023-10-08T00:16:55.716 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.711+0000 7ff9967fc700 1 --2- 172.21.15.99:0/555486365 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff988066f30 0x7ff9880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:55.716 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.711+0000 7ff9967fc700 1 -- 172.21.15.99:0/555486365 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff9a00864b0 con 0x7ff9a815a1e0 2023-10-08T00:16:55.716 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.711+0000 7ff9a659c700 1 --2- 172.21.15.99:0/555486365 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff988066f30 0x7ff9880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:55.716 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.712+0000 7ff9a659c700 1 --2- 172.21.15.99:0/555486365 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff988066f30 0x7ff9880693e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff9a80b08c0 tx=0x7ff998009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:55.716 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.715+0000 7ff9967fc700 1 -- 172.21.15.99:0/555486365 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff9a0053720 con 0x7ff9a815a1e0 2023-10-08T00:16:55.940 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.939+0000 7ff9ac8da700 1 -- 172.21.15.99:0/555486365 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff9a80cb520 con 0x7ff9a815a1e0 2023-10-08T00:16:55.940 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.940+0000 7ff9967fc700 1 -- 172.21.15.99:0/555486365 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4602 (secure 0 0 0) 0x7ff9a0056d40 con 0x7ff9a815a1e0 2023-10-08T00:16:55.941 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:16:55.941 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":63,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5813,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/887374839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":887374839},{"type":"v1","addr":"172.21.15.191:6837","nonce":887374839}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":6445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/2409696901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2409696901},{"type":"v1","addr":"172.21.15.191:6836","nonce":2409696901}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":7095,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/1924096228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1924096228},{"type":"v1","addr":"172.21.15.99:6834","nonce":1924096228}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:16:50.092494+0000","modified":"2023-10-08T00:16:53.744105+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7098":{"gid":7098,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/2522933027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":2522933027},{"type":"v1","addr":"172.21.15.99:6835","nonce":2522933027}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-08T00:16:55.943 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.942+0000 7ff9ac8da700 1 -- 172.21.15.99:0/555486365 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff988066f30 msgr2=0x7ff9880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:55.943 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.942+0000 7ff9ac8da700 1 --2- 172.21.15.99:0/555486365 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff988066f30 0x7ff9880693e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff9a80b08c0 tx=0x7ff998009040 comp rx=0 tx=0).stop 2023-10-08T00:16:55.943 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.942+0000 7ff9ac8da700 1 -- 172.21.15.99:0/555486365 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9a815a1e0 msgr2=0x7ff9a80c4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:55.943 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.942+0000 7ff9ac8da700 1 --2- 172.21.15.99:0/555486365 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9a815a1e0 0x7ff9a80c4ce0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7ff9a000b6c0 tx=0x7ff9a0018000 comp rx=0 tx=0).stop 2023-10-08T00:16:55.944 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.942+0000 7ff9ac8da700 1 -- 172.21.15.99:0/555486365 shutdown_connections 2023-10-08T00:16:55.944 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.942+0000 7ff9ac8da700 1 --2- 172.21.15.99:0/555486365 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff988066f30 0x7ff9880693e0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:55.944 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.942+0000 7ff9ac8da700 1 --2- 172.21.15.99:0/555486365 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9a815a1e0 0x7ff9a80c4ce0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:55.944 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.942+0000 7ff9ac8da700 1 --2- 172.21.15.99:0/555486365 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9a80b3430 0x7ff9a80c9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:55.944 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.942+0000 7ff9ac8da700 1 --2- 172.21.15.99:0/555486365 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9a8159e10 0x7ff9a80c9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:55.944 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.942+0000 7ff9ac8da700 1 -- 172.21.15.99:0/555486365 >> 172.21.15.99:0/555486365 conn(0x7ff9a8067660 msgr2=0x7ff9a815a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:55.944 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.942+0000 7ff9ac8da700 1 -- 172.21.15.99:0/555486365 shutdown_connections 2023-10-08T00:16:55.945 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:55.942+0000 7ff9ac8da700 1 -- 172.21.15.99:0/555486365 wait complete. 2023-10-08T00:16:55.945 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 63 2023-10-08T00:16:56.024 INFO:tasks.ceph.mds.a.smithi099.stderr:2023-10-08T00:16:56.024+0000 7f48ac854700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T00:16:56.357 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 63, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T00:16:50.092494+0000', 'modified': '2023-10-08T00:16:53.744105+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7098}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7098': {'gid': 7098, 'name': 'a', 'rank': 0, 'incarnation': 60, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.99:6835/2522933027', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6833', 'nonce': 2522933027}, {'type': 'v1', 'addr': '172.21.15.99:6835', 'nonce': 2522933027}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-08T00:16:56.358 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T00:16:56.358 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-08T00:16:56.474 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.473+0000 7f1d0b4ac700 1 Processor -- start 2023-10-08T00:16:56.474 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.473+0000 7f1d0b4ac700 1 -- start start 2023-10-08T00:16:56.480 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.474+0000 7f1d0b4ac700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1d040c9af0 0x7f1d0406ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:56.480 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.474+0000 7f1d0b4ac700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1d04070700 0x7f1d04070b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:56.480 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.474+0000 7f1d0b4ac700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d04071080 con 0x7f1d041510f0 2023-10-08T00:16:56.481 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.474+0000 7f1d0b4ac700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d040ca0b0 con 0x7f1d040c9af0 2023-10-08T00:16:56.481 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.474+0000 7f1d0b4ac700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d04068a10 con 0x7f1d04070700 2023-10-08T00:16:56.481 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.475+0000 7f1d08a47700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1d040c9af0 0x7f1d0406ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:56.481 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.475+0000 7f1d09a49700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1d04070700 0x7f1d04070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:56.481 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.475+0000 7f1d08a47700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1d040c9af0 0x7f1d0406ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.99:38064/0 (socket says 172.21.15.99:38064) 2023-10-08T00:16:56.481 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.475+0000 7f1d08a47700 1 -- 172.21.15.99:0/3342061938 learned_addr learned my addr 172.21.15.99:0/3342061938 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:56.481 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.475+0000 7f1d09248700 1 --1- 172.21.15.99:0/3342061938 >> v1:172.21.15.191:6789/0 conn(0x7f1d041510f0 0x7f1d0415dd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:42556/0 (socket says 172.21.15.99:42556) 2023-10-08T00:16:56.481 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.475+0000 7f1d08a47700 1 -- 172.21.15.99:0/3342061938 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1d04070700 msgr2=0x7f1d04070b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:56.481 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.475+0000 7f1d08a47700 1 --2- 172.21.15.99:0/3342061938 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1d04070700 0x7f1d04070b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:56.481 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.475+0000 7f1d08a47700 1 -- 172.21.15.99:0/3342061938 >> v1:172.21.15.191:6789/0 conn(0x7f1d041510f0 legacy=0x7f1d0415dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:56.481 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.475+0000 7f1d08a47700 1 -- 172.21.15.99:0/3342061938 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d04068cf0 con 0x7f1d040c9af0 2023-10-08T00:16:56.481 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.475+0000 7f1d09a49700 1 --2- 172.21.15.99:0/3342061938 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1d04070700 0x7f1d04070b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:16:56.482 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.475+0000 7f1d08a47700 1 --2- 172.21.15.99:0/3342061938 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d040c9af0 0x7f1d0406ecc0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1cf40099b0 tx=0x7f1cf4009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b7692b49d5b7255 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:56.482 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.475+0000 7f1cfbfff700 1 -- 172.21.15.99:0/3342061938 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf4009ea0 con 0x7f1d040c9af0 2023-10-08T00:16:56.482 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.475+0000 7f1cfbfff700 1 -- 172.21.15.99:0/3342061938 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf400b860 con 0x7f1d040c9af0 2023-10-08T00:16:56.482 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.475+0000 7f1cfbfff700 1 -- 172.21.15.99:0/3342061938 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf400bc00 con 0x7f1d040c9af0 2023-10-08T00:16:56.482 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.476+0000 7f1d0b4ac700 1 -- 172.21.15.99:0/3342061938 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d040c9af0 msgr2=0x7f1d0406ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:56.482 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.476+0000 7f1d0b4ac700 1 --2- 172.21.15.99:0/3342061938 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d040c9af0 0x7f1d0406ecc0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1cf40099b0 tx=0x7f1cf4009250 comp rx=0 tx=0).stop 2023-10-08T00:16:56.482 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.476+0000 7f1d0b4ac700 1 -- 172.21.15.99:0/3342061938 shutdown_connections 2023-10-08T00:16:56.482 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.476+0000 7f1d0b4ac700 1 --2- 172.21.15.99:0/3342061938 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1d04070700 0x7f1d04070b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:56.482 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.476+0000 7f1d0b4ac700 1 --2- 172.21.15.99:0/3342061938 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d040c9af0 0x7f1d0406ecc0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:56.482 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.476+0000 7f1d0b4ac700 1 -- 172.21.15.99:0/3342061938 >> 172.21.15.99:0/3342061938 conn(0x7f1d040ca4f0 msgr2=0x7f1d040ca8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:56.482 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.476+0000 7f1d0b4ac700 1 -- 172.21.15.99:0/3342061938 shutdown_connections 2023-10-08T00:16:56.483 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.476+0000 7f1d0b4ac700 1 -- 172.21.15.99:0/3342061938 wait complete. 2023-10-08T00:16:56.483 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.477+0000 7f1d0b4ac700 1 Processor -- start 2023-10-08T00:16:56.483 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.477+0000 7f1d0b4ac700 1 -- start start 2023-10-08T00:16:56.483 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.478+0000 7f1d0b4ac700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d04070700 0x7f1d0414feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:56.483 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.478+0000 7f1d0b4ac700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d040c9af0 0x7f1d04151810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:56.483 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.478+0000 7f1d0b4ac700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1d041510f0 0x7f1d040cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:56.483 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.478+0000 7f1d0b4ac700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d040b04a0 con 0x7f1d040c9af0 2023-10-08T00:16:56.483 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.478+0000 7f1d0b4ac700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d040b36c0 con 0x7f1d04070700 2023-10-08T00:16:56.483 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.478+0000 7f1d0b4ac700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d04155450 con 0x7f1d041510f0 2023-10-08T00:16:56.483 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.478+0000 7f1d08a47700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d040c9af0 0x7f1d04151810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:56.483 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.478+0000 7f1d09248700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d04070700 0x7f1d0414feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:56.483 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.478+0000 7f1d08a47700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d040c9af0 0x7f1d04151810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55992/0 (socket says 172.21.15.99:55992) 2023-10-08T00:16:56.484 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.478+0000 7f1d08a47700 1 -- 172.21.15.99:0/3083412588 learned_addr learned my addr 172.21.15.99:0/3083412588 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:56.484 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.478+0000 7f1d09a49700 1 --2- 172.21.15.99:0/3083412588 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1d041510f0 0x7f1d040cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:56.484 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.479+0000 7f1d08a47700 1 -- 172.21.15.99:0/3083412588 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1d041510f0 msgr2=0x7f1d040cc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:56.484 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.479+0000 7f1d08a47700 1 --2- 172.21.15.99:0/3083412588 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1d041510f0 0x7f1d040cc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:56.484 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.479+0000 7f1d08a47700 1 -- 172.21.15.99:0/3083412588 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d04070700 msgr2=0x7f1d0414feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:56.484 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.479+0000 7f1d08a47700 1 --2- 172.21.15.99:0/3083412588 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d04070700 0x7f1d0414feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:56.484 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.479+0000 7f1d08a47700 1 -- 172.21.15.99:0/3083412588 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cfc009a20 con 0x7f1d040c9af0 2023-10-08T00:16:56.484 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.479+0000 7f1d09248700 1 --2- 172.21.15.99:0/3083412588 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d04070700 0x7f1d0414feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:16:56.484 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.479+0000 7f1d08a47700 1 --2- 172.21.15.99:0/3083412588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d040c9af0 0x7f1d04151810 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4006fd0 tx=0x7f1cf4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:56.484 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.479+0000 7f1cf97fa700 1 -- 172.21.15.99:0/3083412588 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf4000d10 con 0x7f1d040c9af0 2023-10-08T00:16:56.484 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.479+0000 7f1cf97fa700 1 -- 172.21.15.99:0/3083412588 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf400bbe0 con 0x7f1d040c9af0 2023-10-08T00:16:56.484 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.479+0000 7f1cf97fa700 1 -- 172.21.15.99:0/3083412588 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf4011650 con 0x7f1d040c9af0 2023-10-08T00:16:56.485 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.479+0000 7f1d0b4ac700 1 -- 172.21.15.99:0/3083412588 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cf40094f0 con 0x7f1d040c9af0 2023-10-08T00:16:56.485 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.479+0000 7f1d0b4ac700 1 -- 172.21.15.99:0/3083412588 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d041562b0 con 0x7f1d040c9af0 2023-10-08T00:16:56.493 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.480+0000 7f1cf97fa700 1 -- 172.21.15.99:0/3083412588 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1cf4011a40 con 0x7f1d040c9af0 2023-10-08T00:16:56.493 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.481+0000 7f1cf97fa700 1 --2- 172.21.15.99:0/3083412588 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1ce8066f30 0x7f1ce80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:56.493 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.481+0000 7f1cf97fa700 1 -- 172.21.15.99:0/3083412588 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1cf4085df0 con 0x7f1d040c9af0 2023-10-08T00:16:56.493 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.481+0000 7f1d09248700 1 --2- 172.21.15.99:0/3083412588 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1ce8066f30 0x7f1ce80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:56.493 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.482+0000 7f1d09248700 1 --2- 172.21.15.99:0/3083412588 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1ce8066f30 0x7f1ce80693e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1d00007060 tx=0x7f1d00007eb0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:56.494 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.483+0000 7f1d147a4b80 1 Processor -- start 2023-10-08T00:16:56.494 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.483+0000 7f1d147a4b80 1 -- start start 2023-10-08T00:16:56.494 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.483+0000 7f1d147a4b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55cbe95d8a50 0x55cbe95d8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:56.494 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.483+0000 7f1d147a4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbe9622d00 0x55cbe9601a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:56.494 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.483+0000 7f1d147a4b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cbe9601f60 0x55cbe95d4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:56.494 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.483+0000 7f1d147a4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbe952fa70 con 0x55cbe9622d00 2023-10-08T00:16:56.494 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.483+0000 7f1d147a4b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbe952d270 con 0x55cbe9601f60 2023-10-08T00:16:56.494 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.483+0000 7f1d147a4b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cbe9529060 con 0x55cbe95d8a50 2023-10-08T00:16:56.494 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.484+0000 7f1d08a47700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbe9622d00 0x55cbe9601a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:56.494 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.484+0000 7f1d08a47700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbe9622d00 0x55cbe9601a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55994/0 (socket says 172.21.15.99:55994) 2023-10-08T00:16:56.494 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.484+0000 7f1d08a47700 1 -- 172.21.15.99:0/1378094793 learned_addr learned my addr 172.21.15.99:0/1378094793 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:56.495 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.484+0000 7f1d09a49700 1 --2- 172.21.15.99:0/1378094793 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cbe9601f60 0x55cbe95d4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:56.495 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.484+0000 7f1d09248700 1 --2- 172.21.15.99:0/1378094793 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55cbe95d8a50 0x55cbe95d8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:56.495 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.484+0000 7f1d08a47700 1 -- 172.21.15.99:0/1378094793 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55cbe95d8a50 msgr2=0x55cbe95d8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:56.495 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.484+0000 7f1d08a47700 1 --2- 172.21.15.99:0/1378094793 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55cbe95d8a50 0x55cbe95d8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:56.495 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.484+0000 7f1d08a47700 1 -- 172.21.15.99:0/1378094793 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cbe9601f60 msgr2=0x55cbe95d4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:56.495 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.484+0000 7f1d08a47700 1 --2- 172.21.15.99:0/1378094793 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cbe9601f60 0x55cbe95d4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:56.495 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.484+0000 7f1d08a47700 1 -- 172.21.15.99:0/1378094793 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbe93c3270 con 0x55cbe9622d00 2023-10-08T00:16:56.495 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.484+0000 7f1d09a49700 1 --2- 172.21.15.99:0/1378094793 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cbe9601f60 0x55cbe95d4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:56.495 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.484+0000 7f1d08a47700 1 --2- 172.21.15.99:0/1378094793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbe9622d00 0x55cbe9601a20 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4056720 tx=0x7f1cf40841d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd7e7f66c1c5fc76 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:56.495 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.484+0000 7f1cee7fc700 1 -- 172.21.15.99:0/1378094793 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf4059f90 con 0x55cbe9622d00 2023-10-08T00:16:56.495 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.484+0000 7f1cee7fc700 1 -- 172.21.15.99:0/1378094793 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf405dce0 con 0x55cbe9622d00 2023-10-08T00:16:56.495 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.484+0000 7f1cee7fc700 1 -- 172.21.15.99:0/1378094793 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf40550e0 con 0x55cbe9622d00 2023-10-08T00:16:56.496 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.485+0000 7f1d147a4b80 1 -- 172.21.15.99:0/1378094793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbe9622d00 msgr2=0x55cbe9601a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:56.496 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.485+0000 7f1d147a4b80 1 --2- 172.21.15.99:0/1378094793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbe9622d00 0x55cbe9601a20 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4056720 tx=0x7f1cf40841d0 comp rx=0 tx=0).stop 2023-10-08T00:16:56.496 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.485+0000 7f1d147a4b80 1 -- 172.21.15.99:0/1378094793 shutdown_connections 2023-10-08T00:16:56.496 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.485+0000 7f1d147a4b80 1 --2- 172.21.15.99:0/1378094793 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cbe9601f60 0x55cbe95d4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:56.496 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.485+0000 7f1d147a4b80 1 --2- 172.21.15.99:0/1378094793 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55cbe95d8a50 0x55cbe95d8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:56.496 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.485+0000 7f1d147a4b80 1 --2- 172.21.15.99:0/1378094793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbe9622d00 0x55cbe9601a20 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:56.496 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.485+0000 7f1d147a4b80 1 -- 172.21.15.99:0/1378094793 >> 172.21.15.99:0/1378094793 conn(0x55cbe95da370 msgr2=0x55cbe95da740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:56.496 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.485+0000 7f1d147a4b80 1 -- 172.21.15.99:0/1378094793 shutdown_connections 2023-10-08T00:16:56.496 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.485+0000 7f1d147a4b80 1 -- 172.21.15.99:0/1378094793 wait complete. 2023-10-08T00:16:56.496 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.485+0000 7f1d147a4b80 1 -- start start 2023-10-08T00:16:56.496 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.486+0000 7f1d147a4b80 1 Processor -- start 2023-10-08T00:16:56.497 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.486+0000 7f1d147a4b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cbe95d8a50 0x55cbe95fe0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:56.497 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.486+0000 7f1d147a4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbe9601f60 0x55cbe961f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:56.497 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.486+0000 7f1d147a4b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55cbe9622d00 0x55cbe95d3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:56.497 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.486+0000 7f1d147a4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbe94d4890 con 0x55cbe9601f60 2023-10-08T00:16:56.497 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.486+0000 7f1d147a4b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbe94efea0 con 0x55cbe95d8a50 2023-10-08T00:16:56.497 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.486+0000 7f1d147a4b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cbe93abe30 con 0x55cbe9622d00 2023-10-08T00:16:56.497 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.486+0000 7f1d147a4b80 10 client.0 _ll_register_callbacks cb 0x55cbe959c110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T00:16:56.497 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.486+0000 7f1d08a47700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbe9601f60 0x55cbe961f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:56.497 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.486+0000 7f1d08a47700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbe9601f60 0x55cbe961f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56010/0 (socket says 172.21.15.99:56010) 2023-10-08T00:16:56.497 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.486+0000 7f1d08a47700 1 -- 172.21.15.99:0/203729651 learned_addr learned my addr 172.21.15.99:0/203729651 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:56.497 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.486+0000 7f1d09a49700 1 --2- 172.21.15.99:0/203729651 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55cbe9622d00 0x55cbe95d3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:56.497 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.486+0000 7f1d09248700 1 --2- 172.21.15.99:0/203729651 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cbe95d8a50 0x55cbe95fe0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:56.498 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.487+0000 7f1d08a47700 1 -- 172.21.15.99:0/203729651 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55cbe9622d00 msgr2=0x55cbe95d3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:56.498 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.487+0000 7f1d08a47700 1 --2- 172.21.15.99:0/203729651 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55cbe9622d00 0x55cbe95d3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:56.498 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.487+0000 7f1d08a47700 1 -- 172.21.15.99:0/203729651 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cbe95d8a50 msgr2=0x55cbe95fe0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:56.498 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.487+0000 7f1d08a47700 1 --2- 172.21.15.99:0/203729651 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cbe95d8a50 0x55cbe95fe0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:56.498 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.487+0000 7f1d08a47700 1 -- 172.21.15.99:0/203729651 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d041562b0 con 0x55cbe9601f60 2023-10-08T00:16:56.498 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.487+0000 7f1d09248700 1 --2- 172.21.15.99:0/203729651 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cbe95d8a50 0x55cbe95fe0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:56.498 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.487+0000 7f1d09a49700 1 --2- 172.21.15.99:0/203729651 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55cbe9622d00 0x55cbe95d3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:56.498 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.487+0000 7f1d08a47700 1 --2- 172.21.15.99:0/203729651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbe9601f60 0x55cbe961f8e0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4053760 tx=0x7f1cf406f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c05d0cbd386dbb70 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:56.498 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.487+0000 7f1ce37fe700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T00:16:56.498 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.487+0000 7f1ce37fe700 1 -- 172.21.15.99:0/203729651 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf40550e0 con 0x55cbe9601f60 2023-10-08T00:16:56.498 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.487+0000 7f1ce37fe700 1 -- 172.21.15.99:0/203729651 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(139..139 src has 1..139) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1cf4056230 con 0x55cbe9601f60 2023-10-08T00:16:56.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.487+0000 7f1d147a4b80 1 -- 172.21.15.99:0/203729651 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cbe95cdb70 con 0x55cbe9601f60 2023-10-08T00:16:56.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.487+0000 7f1ce37fe700 1 -- 172.21.15.99:0/203729651 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1cf405d850 con 0x55cbe9601f60 2023-10-08T00:16:56.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.487+0000 7f1d147a4b80 10 client.7233 fetch_fsmap learned FSMap version 63 2023-10-08T00:16:56.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.487+0000 7f1d147a4b80 1 -- 172.21.15.99:0/203729651 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x55cbe9365fe0 con 0x55cbe9601f60 2023-10-08T00:16:56.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.487+0000 7f1ce37fe700 1 -- 172.21.15.99:0/203729651 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cbe95cdb70 con 0x55cbe9601f60 2023-10-08T00:16:56.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.487+0000 7f1ce37fe700 1 -- 172.21.15.99:0/203729651 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 63) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f1cf4072b00 con 0x55cbe9601f60 2023-10-08T00:16:56.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.487+0000 7f1d147a4b80 10 client.7233 fetch_fsmap finished waiting for FSMap version 63 2023-10-08T00:16:56.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.487+0000 7f1d147a4b80 10 client.7233 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-08T00:16:56.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.487+0000 7f1d147a4b80 20 client.7233 populate_metadata read hostname 'smithi099' 2023-10-08T00:16:56.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.487+0000 7f1d147a4b80 1 --2- 172.21.15.99:0/203729651 >> [v2:172.21.15.99:6833/2522933027,v1:172.21.15.99:6835/2522933027] conn(0x55cbe961a410 0x55cbe961a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:56.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.487+0000 7f1d147a4b80 4 client.7233 mds_command: new command op to 7098 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-08T00:16:56.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.487+0000 7f1d147a4b80 1 -- 172.21.15.99:0/203729651 --> [v2:172.21.15.99:6833/2522933027,v1:172.21.15.99:6835/2522933027] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cbe95aaaa0 con 0x55cbe961a410 2023-10-08T00:16:56.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.487+0000 7f1d09248700 1 --2- 172.21.15.99:0/203729651 >> [v2:172.21.15.99:6833/2522933027,v1:172.21.15.99:6835/2522933027] conn(0x55cbe961a410 0x55cbe961a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:56.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.488+0000 7f1ce37fe700 10 client.7233 ms_handle_connect on v2:172.21.15.99:6833/2522933027 2023-10-08T00:16:56.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.488+0000 7f1d09248700 1 --2- 172.21.15.99:0/203729651 >> [v2:172.21.15.99:6833/2522933027,v1:172.21.15.99:6835/2522933027] conn(0x55cbe961a410 0x55cbe961a7e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ff32790c98a5dfe server_cookie=ed783d16ac07f1b9 in_seq=0 out_seq=0 2023-10-08T00:16:56.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.488+0000 7f1ce37fe700 1 -- 172.21.15.99:0/203729651 <== mds.0 v2:172.21.15.99:6833/2522933027 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55cbe95aaaa0 con 0x55cbe961a410 2023-10-08T00:16:56.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.488+0000 7f1ce37fe700 10 client.7233 handle_command_reply: tid=0 2023-10-08T00:16:56.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.488+0000 7f1d147a4b80 1 client.7233 shutdown 2023-10-08T00:16:56.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.488+0000 7f1d147a4b80 1 -- 172.21.15.99:0/203729651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbe9601f60 msgr2=0x55cbe961f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:56.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.488+0000 7f1d147a4b80 1 --2- 172.21.15.99:0/203729651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbe9601f60 0x55cbe961f8e0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4053760 tx=0x7f1cf406f000 comp rx=0 tx=0).stop 2023-10-08T00:16:56.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.488+0000 7f1d147a4b80 1 -- 172.21.15.99:0/203729651 shutdown_connections 2023-10-08T00:16:56.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.488+0000 7f1d147a4b80 1 --2- 172.21.15.99:0/203729651 >> [v2:172.21.15.99:6833/2522933027,v1:172.21.15.99:6835/2522933027] conn(0x55cbe961a410 0x55cbe961a7e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:56.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.488+0000 7f1d147a4b80 1 --2- 172.21.15.99:0/203729651 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cbe95d8a50 0x55cbe95fe0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:56.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.488+0000 7f1d147a4b80 1 --2- 172.21.15.99:0/203729651 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55cbe9622d00 0x55cbe95d3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:56.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.488+0000 7f1ce37fe700 0 client.7233 ms_handle_reset on v2:172.21.15.99:6833/2522933027 2023-10-08T00:16:56.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.488+0000 7f1d147a4b80 1 --2- 172.21.15.99:0/203729651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbe9601f60 0x55cbe961f8e0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:56.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.488+0000 7f1d147a4b80 1 -- 172.21.15.99:0/203729651 >> 172.21.15.99:0/203729651 conn(0x55cbe95da370 msgr2=0x55cbe95d5780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:56.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.489+0000 7f1d147a4b80 1 -- 172.21.15.99:0/203729651 shutdown_connections 2023-10-08T00:16:56.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.489+0000 7f1d147a4b80 1 -- 172.21.15.99:0/203729651 wait complete. 2023-10-08T00:16:56.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.489+0000 7f1d147a4b80 20 client.7233 trim_cache size 0 max 16384 2023-10-08T00:16:56.506 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.505+0000 7f1d147a4b80 1 Processor -- start 2023-10-08T00:16:56.506 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.505+0000 7f1d147a4b80 1 -- start start 2023-10-08T00:16:56.513 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.506+0000 7f1d147a4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbe95d8a50 0x55cbe9608a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:56.513 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.506+0000 7f1d147a4b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cbe9601f60 0x55cbe961c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:56.513 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.506+0000 7f1d147a4b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55cbe9622d00 0x55cbe9619ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:56.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.506+0000 7f1d147a4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbe96086b0 con 0x55cbe95d8a50 2023-10-08T00:16:56.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.506+0000 7f1d147a4b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbe95d35a0 con 0x55cbe9601f60 2023-10-08T00:16:56.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.506+0000 7f1d147a4b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cbe96011d0 con 0x55cbe9622d00 2023-10-08T00:16:56.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.506+0000 7f1d09248700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbe95d8a50 0x55cbe9608a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:56.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.506+0000 7f1d09248700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbe95d8a50 0x55cbe9608a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56024/0 (socket says 172.21.15.99:56024) 2023-10-08T00:16:56.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.506+0000 7f1d09248700 1 -- 172.21.15.99:0/594167731 learned_addr learned my addr 172.21.15.99:0/594167731 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:56.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.506+0000 7f1d08a47700 1 --2- 172.21.15.99:0/594167731 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cbe9601f60 0x55cbe961c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:56.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.506+0000 7f1d09a49700 1 --2- 172.21.15.99:0/594167731 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55cbe9622d00 0x55cbe9619ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:56.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.506+0000 7f1d09248700 1 -- 172.21.15.99:0/594167731 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55cbe9622d00 msgr2=0x55cbe9619ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:56.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.506+0000 7f1d09248700 1 --2- 172.21.15.99:0/594167731 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55cbe9622d00 0x55cbe9619ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:56.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.506+0000 7f1d09248700 1 -- 172.21.15.99:0/594167731 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cbe9601f60 msgr2=0x55cbe961c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:56.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.506+0000 7f1d09248700 1 --2- 172.21.15.99:0/594167731 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cbe9601f60 0x55cbe961c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:56.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.506+0000 7f1d09248700 1 -- 172.21.15.99:0/594167731 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbe94f7780 con 0x55cbe95d8a50 2023-10-08T00:16:56.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.506+0000 7f1d09a49700 1 --2- 172.21.15.99:0/594167731 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55cbe9622d00 0x55cbe9619ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:16:56.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.506+0000 7f1d08a47700 1 --2- 172.21.15.99:0/594167731 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cbe9601f60 0x55cbe961c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:16:56.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.506+0000 7f1d09248700 1 --2- 172.21.15.99:0/594167731 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbe95d8a50 0x55cbe9608a60 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f1d00001b90 tx=0x7f1d0000a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eda2b59bf3fb18fc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:56.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.506+0000 7f1ceeffd700 1 -- 172.21.15.99:0/594167731 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d00001d20 con 0x55cbe95d8a50 2023-10-08T00:16:56.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.506+0000 7f1ceeffd700 1 -- 172.21.15.99:0/594167731 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d0000b070 con 0x55cbe95d8a50 2023-10-08T00:16:56.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.506+0000 7f1ceeffd700 1 -- 172.21.15.99:0/594167731 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d00026ce0 con 0x55cbe95d8a50 2023-10-08T00:16:56.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.507+0000 7f1d147a4b80 1 -- 172.21.15.99:0/594167731 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbe95d8a50 msgr2=0x55cbe9608a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:56.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.507+0000 7f1d147a4b80 1 --2- 172.21.15.99:0/594167731 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbe95d8a50 0x55cbe9608a60 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f1d00001b90 tx=0x7f1d0000a000 comp rx=0 tx=0).stop 2023-10-08T00:16:56.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.507+0000 7f1d147a4b80 1 -- 172.21.15.99:0/594167731 shutdown_connections 2023-10-08T00:16:56.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.507+0000 7f1d147a4b80 1 --2- 172.21.15.99:0/594167731 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55cbe9622d00 0x55cbe9619ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:56.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.507+0000 7f1d147a4b80 1 --2- 172.21.15.99:0/594167731 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbe95d8a50 0x55cbe9608a60 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:56.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.507+0000 7f1d147a4b80 1 --2- 172.21.15.99:0/594167731 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cbe9601f60 0x55cbe961c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:56.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.507+0000 7f1d147a4b80 1 -- 172.21.15.99:0/594167731 >> 172.21.15.99:0/594167731 conn(0x55cbe95da370 msgr2=0x55cbe95d5780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:56.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.507+0000 7f1d147a4b80 1 -- 172.21.15.99:0/594167731 shutdown_connections 2023-10-08T00:16:56.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.507+0000 7f1d147a4b80 1 -- 172.21.15.99:0/594167731 wait complete. 2023-10-08T00:16:56.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.507+0000 7f1d147a4b80 1 -- start start 2023-10-08T00:16:56.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.508+0000 7f1d147a4b80 1 Processor -- start 2023-10-08T00:16:56.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.508+0000 7f1d147a4b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cbe95d8a50 0x55cbe9685710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:56.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.508+0000 7f1d147a4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbe9601f60 0x55cbe961d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:56.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.508+0000 7f1d147a4b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55cbe9622d00 0x55cbe9638b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:56.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.508+0000 7f1d147a4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbe95fd2a0 con 0x55cbe9601f60 2023-10-08T00:16:56.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.508+0000 7f1d147a4b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbe9619c70 con 0x55cbe95d8a50 2023-10-08T00:16:56.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.508+0000 7f1d147a4b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cbe9685c50 con 0x55cbe9622d00 2023-10-08T00:16:56.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.508+0000 7f1d147a4b80 10 client.0 _ll_register_callbacks cb 0x55cbe959c110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T00:16:56.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.509+0000 7f1d08a47700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbe9601f60 0x55cbe961d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:56.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.509+0000 7f1d08a47700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbe9601f60 0x55cbe961d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56028/0 (socket says 172.21.15.99:56028) 2023-10-08T00:16:56.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.509+0000 7f1d08a47700 1 -- 172.21.15.99:0/3191304589 learned_addr learned my addr 172.21.15.99:0/3191304589 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:56.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.509+0000 7f1d09248700 1 --2- 172.21.15.99:0/3191304589 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cbe95d8a50 0x55cbe9685710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:56.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.509+0000 7f1d09a49700 1 --2- 172.21.15.99:0/3191304589 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55cbe9622d00 0x55cbe9638b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:56.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.509+0000 7f1d08a47700 1 -- 172.21.15.99:0/3191304589 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55cbe9622d00 msgr2=0x55cbe9638b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:56.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.509+0000 7f1d08a47700 1 --2- 172.21.15.99:0/3191304589 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55cbe9622d00 0x55cbe9638b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:56.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.509+0000 7f1d08a47700 1 -- 172.21.15.99:0/3191304589 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cbe95d8a50 msgr2=0x55cbe9685710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:56.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.509+0000 7f1d08a47700 1 --2- 172.21.15.99:0/3191304589 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cbe95d8a50 0x55cbe9685710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:56.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.509+0000 7f1d08a47700 1 -- 172.21.15.99:0/3191304589 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d04071080 con 0x55cbe9601f60 2023-10-08T00:16:56.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.509+0000 7f1d09248700 1 --2- 172.21.15.99:0/3191304589 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cbe95d8a50 0x55cbe9685710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:56.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.509+0000 7f1d09a49700 1 --2- 172.21.15.99:0/3191304589 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55cbe9622d00 0x55cbe9638b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:56.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.509+0000 7f1d08a47700 1 --2- 172.21.15.99:0/3191304589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbe9601f60 0x55cbe961d820 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4056a40 tx=0x7f1cf4080070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7439d37eaff2df2d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:56.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.509+0000 7f1ce37fe700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T00:16:56.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.509+0000 7f1ce37fe700 1 -- 172.21.15.99:0/3191304589 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf40550e0 con 0x55cbe9601f60 2023-10-08T00:16:56.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.509+0000 7f1ce37fe700 1 -- 172.21.15.99:0/3191304589 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(139..139 src has 1..139) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1cf4056230 con 0x55cbe9601f60 2023-10-08T00:16:56.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.509+0000 7f1d147a4b80 1 -- 172.21.15.99:0/3191304589 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1cf405dce0 con 0x55cbe9601f60 2023-10-08T00:16:56.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.509+0000 7f1ce37fe700 1 -- 172.21.15.99:0/3191304589 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1cf4056b10 con 0x55cbe9601f60 2023-10-08T00:16:56.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.509+0000 7f1d147a4b80 10 client.7239 fetch_fsmap learned FSMap version 63 2023-10-08T00:16:56.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.509+0000 7f1d147a4b80 1 -- 172.21.15.99:0/3191304589 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x55cbe94de470 con 0x55cbe9601f60 2023-10-08T00:16:56.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.509+0000 7f1ce37fe700 1 -- 172.21.15.99:0/3191304589 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf405dce0 con 0x55cbe9601f60 2023-10-08T00:16:56.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.509+0000 7f1ce37fe700 1 -- 172.21.15.99:0/3191304589 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 63) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f1cf407e430 con 0x55cbe9601f60 2023-10-08T00:16:56.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.509+0000 7f1d147a4b80 10 client.7239 fetch_fsmap finished waiting for FSMap version 63 2023-10-08T00:16:56.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.509+0000 7f1d147a4b80 10 client.7239 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-08T00:16:56.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.509+0000 7f1d147a4b80 20 client.7239 populate_metadata read hostname 'smithi099' 2023-10-08T00:16:56.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.509+0000 7f1d147a4b80 1 --2- 172.21.15.99:0/3191304589 >> [v2:172.21.15.99:6833/2522933027,v1:172.21.15.99:6835/2522933027] conn(0x55cbe96840f0 0x55cbe96844c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:56.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.509+0000 7f1d147a4b80 4 client.7239 mds_command: new command op to 7098 tid=0[{"prefix": "status"}] 2023-10-08T00:16:56.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.509+0000 7f1d147a4b80 1 -- 172.21.15.99:0/3191304589 --> [v2:172.21.15.99:6833/2522933027,v1:172.21.15.99:6835/2522933027] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55cbe94cbe00 con 0x55cbe96840f0 2023-10-08T00:16:56.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.510+0000 7f1d09248700 1 --2- 172.21.15.99:0/3191304589 >> [v2:172.21.15.99:6833/2522933027,v1:172.21.15.99:6835/2522933027] conn(0x55cbe96840f0 0x55cbe96844c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:56.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.510+0000 7f1ce37fe700 10 client.7239 ms_handle_connect on v2:172.21.15.99:6833/2522933027 2023-10-08T00:16:56.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.510+0000 7f1d09248700 1 --2- 172.21.15.99:0/3191304589 >> [v2:172.21.15.99:6833/2522933027,v1:172.21.15.99:6835/2522933027] conn(0x55cbe96840f0 0x55cbe96844c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9af5ffa02ced7064 server_cookie=366d8d244ae5352e in_seq=0 out_seq=0 2023-10-08T00:16:56.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.510+0000 7f1ce37fe700 1 -- 172.21.15.99:0/3191304589 <== mds.0 v2:172.21.15.99:6833/2522933027 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55cbe94cbe00 con 0x55cbe96840f0 2023-10-08T00:16:56.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.510+0000 7f1ce37fe700 10 client.7239 handle_command_reply: tid=0 2023-10-08T00:16:56.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.510+0000 7f1d147a4b80 1 client.7239 shutdown 2023-10-08T00:16:56.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.510+0000 7f1d147a4b80 1 -- 172.21.15.99:0/3191304589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbe9601f60 msgr2=0x55cbe961d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:56.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.510+0000 7f1d147a4b80 1 --2- 172.21.15.99:0/3191304589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbe9601f60 0x55cbe961d820 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4056a40 tx=0x7f1cf4080070 comp rx=0 tx=0).stop 2023-10-08T00:16:56.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.510+0000 7f1d147a4b80 1 -- 172.21.15.99:0/3191304589 shutdown_connections 2023-10-08T00:16:56.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.510+0000 7f1d147a4b80 1 --2- 172.21.15.99:0/3191304589 >> [v2:172.21.15.99:6833/2522933027,v1:172.21.15.99:6835/2522933027] conn(0x55cbe96840f0 0x55cbe96844c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:56.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.510+0000 7f1d147a4b80 1 --2- 172.21.15.99:0/3191304589 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cbe95d8a50 0x55cbe9685710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:56.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.510+0000 7f1ce37fe700 0 client.7239 ms_handle_reset on v2:172.21.15.99:6833/2522933027 2023-10-08T00:16:56.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.510+0000 7f1d147a4b80 1 --2- 172.21.15.99:0/3191304589 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55cbe9622d00 0x55cbe9638b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:56.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.510+0000 7f1d147a4b80 1 --2- 172.21.15.99:0/3191304589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbe9601f60 0x55cbe961d820 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:56.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.510+0000 7f1d147a4b80 1 -- 172.21.15.99:0/3191304589 >> 172.21.15.99:0/3191304589 conn(0x55cbe95da370 msgr2=0x55cbe95da820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:56.521 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.510+0000 7f1d147a4b80 1 -- 172.21.15.99:0/3191304589 shutdown_connections 2023-10-08T00:16:56.521 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.510+0000 7f1d147a4b80 1 -- 172.21.15.99:0/3191304589 wait complete. 2023-10-08T00:16:56.521 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.510+0000 7f1d147a4b80 20 client.7239 trim_cache size 0 max 16384 2023-10-08T00:16:56.521 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:16:56.521 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", 2023-10-08T00:16:56.521 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 0, 2023-10-08T00:16:56.521 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 7098, 2023-10-08T00:16:56.521 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2023-10-08T00:16:56.521 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2023-10-08T00:16:56.522 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2023-10-08T00:16:56.522 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 5.7727175480000001, 2023-10-08T00:16:56.522 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 63, 2023-10-08T00:16:56.522 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 139, 2023-10-08T00:16:56.522 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 139, 2023-10-08T00:16:56.522 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 10.907554773999999 2023-10-08T00:16:56.522 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:16:56.523 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.511+0000 7f1d0b4ac700 1 -- 172.21.15.99:0/3083412588 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1ce8066f30 msgr2=0x7f1ce80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:56.524 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.511+0000 7f1d0b4ac700 1 --2- 172.21.15.99:0/3083412588 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1ce8066f30 0x7f1ce80693e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1d00007060 tx=0x7f1d00007eb0 comp rx=0 tx=0).stop 2023-10-08T00:16:56.524 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.511+0000 7f1d0b4ac700 1 -- 172.21.15.99:0/3083412588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d040c9af0 msgr2=0x7f1d04151810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:56.524 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.511+0000 7f1d0b4ac700 1 --2- 172.21.15.99:0/3083412588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d040c9af0 0x7f1d04151810 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4006fd0 tx=0x7f1cf4017000 comp rx=0 tx=0).stop 2023-10-08T00:16:56.524 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.511+0000 7f1d0b4ac700 1 -- 172.21.15.99:0/3083412588 shutdown_connections 2023-10-08T00:16:56.524 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.511+0000 7f1d0b4ac700 1 --2- 172.21.15.99:0/3083412588 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1ce8066f30 0x7f1ce80693e0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:56.524 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.511+0000 7f1d0b4ac700 1 --2- 172.21.15.99:0/3083412588 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d04070700 0x7f1d0414feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:56.524 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.511+0000 7f1d0b4ac700 1 --2- 172.21.15.99:0/3083412588 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1d041510f0 0x7f1d040cc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:56.524 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.511+0000 7f1d0b4ac700 1 --2- 172.21.15.99:0/3083412588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d040c9af0 0x7f1d04151810 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:56.524 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.511+0000 7f1d0b4ac700 1 -- 172.21.15.99:0/3083412588 >> 172.21.15.99:0/3083412588 conn(0x7f1d040ca4f0 msgr2=0x7f1d040ca8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:56.524 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.511+0000 7f1d0b4ac700 1 -- 172.21.15.99:0/3083412588 shutdown_connections 2023-10-08T00:16:56.524 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.511+0000 7f1d0b4ac700 1 -- 172.21.15.99:0/3083412588 wait complete. 2023-10-08T00:16:56.525 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T00:16:56.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.638+0000 7fef82f51700 1 Processor -- start 2023-10-08T00:16:56.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.638+0000 7fef82f51700 1 -- start start 2023-10-08T00:16:56.641 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.639+0000 7fef82f51700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef7c0cd6e0 0x7fef7c0cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:56.641 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.639+0000 7fef82f51700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef7c0cb7c0 con 0x7fef7c0cdab0 2023-10-08T00:16:56.641 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.639+0000 7fef82f51700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef7c15e290 con 0x7fef7c15cbd0 2023-10-08T00:16:56.641 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.639+0000 7fef82f51700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c15e850 con 0x7fef7c0cd6e0 2023-10-08T00:16:56.641 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.639+0000 7fef8174e700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fef7c0cdab0 0x7fef7c15d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:46852/0 (socket says 172.21.15.99:46852) 2023-10-08T00:16:56.642 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.639+0000 7fef8174e700 1 -- 172.21.15.99:0/1845663205 learned_addr learned my addr 172.21.15.99:0/1845663205 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:56.642 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.639+0000 7fef81f4f700 1 --2- 172.21.15.99:0/1845663205 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef7c0cd6e0 0x7fef7c0cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:56.642 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.639+0000 7fef80f4d700 1 -- 172.21.15.99:0/1845663205 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3470594428 0 0) 0x7fef7c0cb7c0 con 0x7fef7c0cdab0 2023-10-08T00:16:56.642 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.639+0000 7fef80f4d700 1 -- 172.21.15.99:0/1845663205 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef68003570 con 0x7fef7c0cdab0 2023-10-08T00:16:56.642 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.640+0000 7fef81f4f700 1 -- 172.21.15.99:0/1845663205 >> v1:172.21.15.191:6789/0 conn(0x7fef7c15cbd0 legacy=0x7fef7c15d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:56.642 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.640+0000 7fef80f4d700 1 -- 172.21.15.99:0/1845663205 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2847450840 0 0) 0x7fef7c15e290 con 0x7fef7c15cbd0 2023-10-08T00:16:56.642 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.640+0000 7fef81f4f700 1 -- 172.21.15.99:0/1845663205 >> v1:172.21.15.99:6789/0 conn(0x7fef7c0cdab0 legacy=0x7fef7c15d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:56.642 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.640+0000 7fef81f4f700 1 -- 172.21.15.99:0/1845663205 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef7c15e570 con 0x7fef7c0cd6e0 2023-10-08T00:16:56.642 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.640+0000 7fef81f4f700 1 --2- 172.21.15.99:0/1845663205 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef7c0cd6e0 0x7fef7c0cbc50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fef78009910 tx=0x7fef780091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a67346111a2fabd7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:56.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.640+0000 7fef80f4d700 1 -- 172.21.15.99:0/1845663205 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7800b680 con 0x7fef7c0cd6e0 2023-10-08T00:16:56.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.640+0000 7fef80f4d700 1 -- 172.21.15.99:0/1845663205 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef7800b7e0 con 0x7fef7c0cd6e0 2023-10-08T00:16:56.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.640+0000 7fef80f4d700 1 -- 172.21.15.99:0/1845663205 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7800bb60 con 0x7fef7c0cd6e0 2023-10-08T00:16:56.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.640+0000 7fef82f51700 1 -- 172.21.15.99:0/1845663205 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef7c0cd6e0 msgr2=0x7fef7c0cbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:56.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.640+0000 7fef82f51700 1 --2- 172.21.15.99:0/1845663205 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef7c0cd6e0 0x7fef7c0cbc50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fef78009910 tx=0x7fef780091a0 comp rx=0 tx=0).stop 2023-10-08T00:16:56.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.641+0000 7fef82f51700 1 -- 172.21.15.99:0/1845663205 shutdown_connections 2023-10-08T00:16:56.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.641+0000 7fef82f51700 1 --2- 172.21.15.99:0/1845663205 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef7c0cd6e0 0x7fef7c0cbc50 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:56.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.641+0000 7fef82f51700 1 -- 172.21.15.99:0/1845663205 >> 172.21.15.99:0/1845663205 conn(0x7fef7c067660 msgr2=0x7fef7c067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:56.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.641+0000 7fef82f51700 1 -- 172.21.15.99:0/1845663205 shutdown_connections 2023-10-08T00:16:56.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.641+0000 7fef82f51700 1 -- 172.21.15.99:0/1845663205 wait complete. 2023-10-08T00:16:56.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.642+0000 7fef82f51700 1 Processor -- start 2023-10-08T00:16:56.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.642+0000 7fef82f51700 1 -- start start 2023-10-08T00:16:56.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.642+0000 7fef82f51700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef7c0cd6e0 0x7fef7c14a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:56.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.642+0000 7fef82f51700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef7c0cdab0 0x7fef7c0c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:56.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.642+0000 7fef82f51700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef7c15cbd0 0x7fef7c0c9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:56.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.642+0000 7fef82f51700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c0cab60 con 0x7fef7c0cd6e0 2023-10-08T00:16:56.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.642+0000 7fef82f51700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c0c9ac0 con 0x7fef7c0cdab0 2023-10-08T00:16:56.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.642+0000 7fef82f51700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c0c9da0 con 0x7fef7c15cbd0 2023-10-08T00:16:56.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.642+0000 7fef81f4f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef7c0cd6e0 0x7fef7c14a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:56.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.642+0000 7fef81f4f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef7c0cd6e0 0x7fef7c14a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56040/0 (socket says 172.21.15.99:56040) 2023-10-08T00:16:56.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.643+0000 7fef81f4f700 1 -- 172.21.15.99:0/1274462345 learned_addr learned my addr 172.21.15.99:0/1274462345 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:56.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.643+0000 7fef8174e700 1 --2- 172.21.15.99:0/1274462345 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef7c0cdab0 0x7fef7c0c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:56.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.643+0000 7fef82750700 1 --2- 172.21.15.99:0/1274462345 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef7c15cbd0 0x7fef7c0c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:56.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.643+0000 7fef81f4f700 1 -- 172.21.15.99:0/1274462345 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef7c15cbd0 msgr2=0x7fef7c0c9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:56.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.643+0000 7fef81f4f700 1 --2- 172.21.15.99:0/1274462345 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef7c15cbd0 0x7fef7c0c9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:56.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.643+0000 7fef81f4f700 1 -- 172.21.15.99:0/1274462345 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef7c0cdab0 msgr2=0x7fef7c0c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:56.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.643+0000 7fef81f4f700 1 --2- 172.21.15.99:0/1274462345 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef7c0cdab0 0x7fef7c0c8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:56.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.643+0000 7fef81f4f700 1 -- 172.21.15.99:0/1274462345 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef78009420 con 0x7fef7c0cd6e0 2023-10-08T00:16:56.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.643+0000 7fef8174e700 1 --2- 172.21.15.99:0/1274462345 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef7c0cdab0 0x7fef7c0c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:56.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.643+0000 7fef81f4f700 1 --2- 172.21.15.99:0/1274462345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef7c0cd6e0 0x7fef7c14a800 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fef7c0c7b10 tx=0x7fef78017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:56.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.643+0000 7fef71ffb700 1 -- 172.21.15.99:0/1274462345 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7801f070 con 0x7fef7c0cd6e0 2023-10-08T00:16:56.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.643+0000 7fef71ffb700 1 -- 172.21.15.99:0/1274462345 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef78000c90 con 0x7fef7c0cd6e0 2023-10-08T00:16:56.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.643+0000 7fef82f51700 1 -- 172.21.15.99:0/1274462345 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef7c0ca080 con 0x7fef7c0cd6e0 2023-10-08T00:16:56.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.643+0000 7fef71ffb700 1 -- 172.21.15.99:0/1274462345 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef78025420 con 0x7fef7c0cd6e0 2023-10-08T00:16:56.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.643+0000 7fef82f51700 1 -- 172.21.15.99:0/1274462345 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef7c0ca640 con 0x7fef7c0cd6e0 2023-10-08T00:16:56.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.644+0000 7fef82f51700 1 -- 172.21.15.99:0/1274462345 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef7c089280 con 0x7fef7c0cd6e0 2023-10-08T00:16:56.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.645+0000 7fef71ffb700 1 -- 172.21.15.99:0/1274462345 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fef78023070 con 0x7fef7c0cd6e0 2023-10-08T00:16:56.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.645+0000 7fef71ffb700 1 --2- 172.21.15.99:0/1274462345 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fef60066f30 0x7fef600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:56.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.645+0000 7fef71ffb700 1 -- 172.21.15.99:0/1274462345 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fef780859d0 con 0x7fef7c0cd6e0 2023-10-08T00:16:56.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.645+0000 7fef8174e700 1 --2- 172.21.15.99:0/1274462345 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fef60066f30 0x7fef600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:56.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.646+0000 7fef8174e700 1 --2- 172.21.15.99:0/1274462345 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fef60066f30 0x7fef600693e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fef7c0b6a10 tx=0x7fef6c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:56.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.649+0000 7fef71ffb700 1 -- 172.21.15.99:0/1274462345 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fef78052c40 con 0x7fef7c0cd6e0 2023-10-08T00:16:56.875 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.875+0000 7fef82f51700 1 -- 172.21.15.99:0/1274462345 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fef7c04bb80 con 0x7fef7c0cd6e0 2023-10-08T00:16:56.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.875+0000 7fef71ffb700 1 -- 172.21.15.99:0/1274462345 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v63) v1 ==== 53+0+73 (secure 0 0 0) 0x7fef78015070 con 0x7fef7c0cd6e0 2023-10-08T00:16:56.876 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T00:16:56.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.877+0000 7fef82f51700 1 -- 172.21.15.99:0/1274462345 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fef60066f30 msgr2=0x7fef600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:56.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.877+0000 7fef82f51700 1 --2- 172.21.15.99:0/1274462345 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fef60066f30 0x7fef600693e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fef7c0b6a10 tx=0x7fef6c009040 comp rx=0 tx=0).stop 2023-10-08T00:16:56.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.877+0000 7fef82f51700 1 -- 172.21.15.99:0/1274462345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef7c0cd6e0 msgr2=0x7fef7c14a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:56.878 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.877+0000 7fef82f51700 1 --2- 172.21.15.99:0/1274462345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef7c0cd6e0 0x7fef7c14a800 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fef7c0c7b10 tx=0x7fef78017000 comp rx=0 tx=0).stop 2023-10-08T00:16:56.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.877+0000 7fef82f51700 1 -- 172.21.15.99:0/1274462345 shutdown_connections 2023-10-08T00:16:56.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.877+0000 7fef82f51700 1 --2- 172.21.15.99:0/1274462345 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fef60066f30 0x7fef600693e0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:56.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.877+0000 7fef82f51700 1 --2- 172.21.15.99:0/1274462345 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef7c15cbd0 0x7fef7c0c9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:56.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.877+0000 7fef82f51700 1 --2- 172.21.15.99:0/1274462345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef7c0cd6e0 0x7fef7c14a800 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:56.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.877+0000 7fef82f51700 1 --2- 172.21.15.99:0/1274462345 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef7c0cdab0 0x7fef7c0c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:56.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.877+0000 7fef82f51700 1 -- 172.21.15.99:0/1274462345 >> 172.21.15.99:0/1274462345 conn(0x7fef7c067660 msgr2=0x7fef7c159d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:56.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.877+0000 7fef82f51700 1 -- 172.21.15.99:0/1274462345 shutdown_connections 2023-10-08T00:16:56.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:56.877+0000 7fef82f51700 1 -- 172.21.15.99:0/1274462345 wait complete. 2023-10-08T00:16:56.889 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T00:16:56.889 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T00:16:56.890 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi099.front.sepia.ceph.com 2023-10-08T00:16:56.890 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T00:16:56.890 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T00:16:56.890 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T00:16:56.890 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T00:16:56.890 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T00:16:56.890 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T00:16:56.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:16:56.893 DEBUG:teuthology.orchestra.run.smithi099:> ip addr 2023-10-08T00:16:56.908 INFO:teuthology.orchestra.run.smithi099.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T00:16:56.908 INFO:teuthology.orchestra.run.smithi099.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T00:16:56.908 INFO:teuthology.orchestra.run.smithi099.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T00:16:56.908 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:16:56.909 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 ::1/128 scope host 2023-10-08T00:16:56.909 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:16:56.909 INFO:teuthology.orchestra.run.smithi099.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:16:56.909 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:10 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:16:56.909 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f0 2023-10-08T00:16:56.909 INFO:teuthology.orchestra.run.smithi099.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:16:56.909 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:11 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:16:56.909 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f1 2023-10-08T00:16:56.909 INFO:teuthology.orchestra.run.smithi099.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:16:56.909 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:02 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:16:56.910 INFO:teuthology.orchestra.run.smithi099.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T00:16:56.910 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:03 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:16:56.910 INFO:teuthology.orchestra.run.smithi099.stdout: inet 172.21.15.99/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T00:16:56.910 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft 42173sec preferred_lft 42173sec 2023-10-08T00:16:56.910 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::ec4:7aff:fe8f:d803/64 scope link 2023-10-08T00:16:56.910 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:16:56.910 INFO:teuthology.orchestra.run.smithi099.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T00:16:56.910 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 22:2c:1c:f6:42:e1 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:16:56.910 INFO:teuthology.orchestra.run.smithi099.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T00:16:56.910 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:16:56.910 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::600d:26ff:fede:5eec/64 scope link 2023-10-08T00:16:56.911 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:16:56.911 INFO:teuthology.orchestra.run.smithi099.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T00:16:56.911 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether ae:2f:b5:94:69:ba brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T00:16:56.911 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::ac2f:b5ff:fe94:69ba/64 scope link 2023-10-08T00:16:56.911 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:16:56.911 INFO:teuthology.orchestra.run.smithi099.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T00:16:56.911 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 22:2c:1c:f6:42:e1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T00:16:56.911 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::202c:1cff:fef6:42e1/64 scope link 2023-10-08T00:16:56.911 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:16:56.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:16:56.912 DEBUG:teuthology.orchestra.run.smithi099:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T00:16:56.971 INFO:teuthology.orchestra.run.smithi099.stdout:1 2023-10-08T00:16:56.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:16:56.973 DEBUG:teuthology.orchestra.run.smithi099:> route 2023-10-08T00:16:57.029 INFO:teuthology.orchestra.run.smithi099.stdout:Kernel IP routing table 2023-10-08T00:16:57.030 INFO:teuthology.orchestra.run.smithi099.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T00:16:57.030 INFO:teuthology.orchestra.run.smithi099.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T00:16:57.030 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T00:16:57.030 INFO:teuthology.orchestra.run.smithi099.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T00:16:57.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:16:57.031 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2023-10-08T00:16:57.031 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2023-10-08T00:16:57.031 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T00:16:57.031 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T00:16:57.031 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T00:16:57.031 DEBUG:teuthology.orchestra.run.smithi099:> ') 2023-10-08T00:16:57.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:16:57.128 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2023-10-08T00:16:57.143 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T00:16:57.143 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T00:16:57.144 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T00:16:57.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:16:57.144 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set brx.0 up 2023-10-08T00:16:57.208 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:16:57.208 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T00:16:57.263 INFO:teuthology.orchestra.run.smithi099.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T00:16:57.263 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:16:57.263 DEBUG:teuthology.orchestra.run.smithi099:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T00:16:57.317 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-08T00:16:57.317 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T00:16:57.318 DEBUG:teuthology.orchestra.run.smithi099:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:16:57.397 INFO:teuthology.orchestra.run.smithi099.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:16:57.397 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:16:57.416 INFO:teuthology.orchestra.run.smithi099.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:16:57.416 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:16:57.416 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: resolved to: "172.21.15.99:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T00:16:57.416 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: trying mount with new device syntax: 0@06962cdb-9a6f-417f-82fd-9e4e3dedbccf.cephfs=/ 2023-10-08T00:16:57.416 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.99:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T00:16:57.416 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T00:16:57.416 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T00:16:57.417 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T00:16:57.438 INFO:teuthology.orchestra.run.smithi099.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:16:57.438 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:16:57.438 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: resolved to: "172.21.15.99:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T00:16:57.438 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: trying mount with new device syntax: 0@06962cdb-9a6f-417f-82fd-9e4e3dedbccf.cephfs=/ 2023-10-08T00:16:57.438 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.99:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T00:16:57.439 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T00:16:57.439 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T00:16:57.439 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T00:16:57.439 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: resolved to: "172.21.15.99:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T00:16:57.439 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.99:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T00:16:57.439 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=06962cdb-9a6f-417f-82fd-9e4e3dedbccf" will pass to kernel 2023-10-08T00:16:57.440 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T00:16:57.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:16:57.441 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T00:16:57.505 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T00:16:57.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.659+0000 7f156fc40700 1 Processor -- start 2023-10-08T00:16:57.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.660+0000 7f156fc40700 1 -- start start 2023-10-08T00:16:57.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.660+0000 7f156fc40700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f156806a530 0x7f15680c8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:57.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.660+0000 7f156fc40700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f15680ad9b0 0x7f15680c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:57.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.660+0000 7f156fc40700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1568067ba0 con 0x7f15681568d0 2023-10-08T00:16:57.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.660+0000 7f156fc40700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f156815aa90 con 0x7f156806a530 2023-10-08T00:16:57.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.660+0000 7f156fc40700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1568067870 con 0x7f15680ad9b0 2023-10-08T00:16:57.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.661+0000 7f156d9dc700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f15681568d0 0x7f1568069e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:46860/0 (socket says 172.21.15.99:46860) 2023-10-08T00:16:57.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.661+0000 7f156d1db700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f156806a530 0x7f15680c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:57.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.661+0000 7f156d9dc700 1 -- 172.21.15.99:0/3219152551 learned_addr learned my addr 172.21.15.99:0/3219152551 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:57.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.661+0000 7f156c9da700 1 -- 172.21.15.99:0/3219152551 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4135754978 0 0) 0x7f1568067ba0 con 0x7f15681568d0 2023-10-08T00:16:57.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.661+0000 7f156e1dd700 1 --2- 172.21.15.99:0/3219152551 >> v2:172.21.15.191:3300/0 conn(0x7f15680ad9b0 0x7f15680c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:57.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.661+0000 7f156c9da700 1 -- 172.21.15.99:0/3219152551 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1554003570 con 0x7f15681568d0 2023-10-08T00:16:57.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.661+0000 7f156d1db700 1 -- 172.21.15.99:0/3219152551 >> v2:172.21.15.191:3300/0 conn(0x7f15680ad9b0 msgr2=0x7f15680c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:57.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.661+0000 7f156d1db700 1 --2- 172.21.15.99:0/3219152551 >> v2:172.21.15.191:3300/0 conn(0x7f15680ad9b0 0x7f15680c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:57.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.661+0000 7f156d1db700 1 -- 172.21.15.99:0/3219152551 >> v1:172.21.15.99:6789/0 conn(0x7f15681568d0 legacy=0x7f1568069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:57.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.661+0000 7f156d1db700 1 -- 172.21.15.99:0/3219152551 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15680c8ca0 con 0x7f156806a530 2023-10-08T00:16:57.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.661+0000 7f156d1db700 1 --2- 172.21.15.99:0/3219152551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f156806a530 0x7f15680c8090 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f15680cead0 tx=0x7f15580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caf01b3c075af3ff server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:57.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.661+0000 7f156c9da700 1 -- 172.21.15.99:0/3219152551 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1558011630 con 0x7f156806a530 2023-10-08T00:16:57.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.661+0000 7f156c9da700 1 -- 172.21.15.99:0/3219152551 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1558011790 con 0x7f156806a530 2023-10-08T00:16:57.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.661+0000 7f156c9da700 1 -- 172.21.15.99:0/3219152551 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1558011b30 con 0x7f156806a530 2023-10-08T00:16:57.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.662+0000 7f156fc40700 1 -- 172.21.15.99:0/3219152551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f156806a530 msgr2=0x7f15680c8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:57.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.662+0000 7f156fc40700 1 --2- 172.21.15.99:0/3219152551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f156806a530 0x7f15680c8090 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f15680cead0 tx=0x7f15580091a0 comp rx=0 tx=0).stop 2023-10-08T00:16:57.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.662+0000 7f156fc40700 1 -- 172.21.15.99:0/3219152551 shutdown_connections 2023-10-08T00:16:57.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.662+0000 7f156fc40700 1 --2- 172.21.15.99:0/3219152551 >> v2:172.21.15.191:3300/0 conn(0x7f15680ad9b0 0x7f15680c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:57.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.662+0000 7f156fc40700 1 --2- 172.21.15.99:0/3219152551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f156806a530 0x7f15680c8090 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:57.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.662+0000 7f156fc40700 1 -- 172.21.15.99:0/3219152551 >> 172.21.15.99:0/3219152551 conn(0x7f15680cda50 msgr2=0x7f15680cde50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:57.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.662+0000 7f156fc40700 1 -- 172.21.15.99:0/3219152551 shutdown_connections 2023-10-08T00:16:57.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.662+0000 7f156fc40700 1 -- 172.21.15.99:0/3219152551 wait complete. 2023-10-08T00:16:57.667 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.663+0000 7f156fc40700 1 Processor -- start 2023-10-08T00:16:57.667 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.663+0000 7f156fc40700 1 -- start start 2023-10-08T00:16:57.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.664+0000 7f156fc40700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f156806a530 0x7f15680ae570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:57.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.664+0000 7f156fc40700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15680ad9b0 0x7f1568157640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:57.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.664+0000 7f156fc40700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15681568d0 0x7f1568157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:57.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.664+0000 7f156fc40700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15681528d0 con 0x7f15680ad9b0 2023-10-08T00:16:57.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.664+0000 7f156fc40700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1568158270 con 0x7f156806a530 2023-10-08T00:16:57.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.664+0000 7f156fc40700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15680b2cd0 con 0x7f15681568d0 2023-10-08T00:16:57.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.664+0000 7f156d1db700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15680ad9b0 0x7f1568157640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:57.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.664+0000 7f156d1db700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15680ad9b0 0x7f1568157640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56052/0 (socket says 172.21.15.99:56052) 2023-10-08T00:16:57.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.664+0000 7f156d1db700 1 -- 172.21.15.99:0/2489445307 learned_addr learned my addr 172.21.15.99:0/2489445307 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:57.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.664+0000 7f156d9dc700 1 --2- 172.21.15.99:0/2489445307 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f156806a530 0x7f15680ae570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:57.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.664+0000 7f156e1dd700 1 --2- 172.21.15.99:0/2489445307 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15681568d0 0x7f1568157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:57.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.664+0000 7f156d1db700 1 -- 172.21.15.99:0/2489445307 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15681568d0 msgr2=0x7f1568157b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:57.669 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.664+0000 7f156d1db700 1 --2- 172.21.15.99:0/2489445307 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15681568d0 0x7f1568157b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:57.669 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.664+0000 7f156d1db700 1 -- 172.21.15.99:0/2489445307 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f156806a530 msgr2=0x7f15680ae570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:57.669 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.664+0000 7f156d1db700 1 --2- 172.21.15.99:0/2489445307 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f156806a530 0x7f15680ae570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:57.669 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.664+0000 7f156d1db700 1 -- 172.21.15.99:0/2489445307 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1558009420 con 0x7f15680ad9b0 2023-10-08T00:16:57.669 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.664+0000 7f156e1dd700 1 --2- 172.21.15.99:0/2489445307 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15681568d0 0x7f1568157b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:57.669 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.664+0000 7f156d1db700 1 --2- 172.21.15.99:0/2489445307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15680ad9b0 0x7f1568157640 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f15680ce880 tx=0x7f1558017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:57.669 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.664+0000 7f155dffb700 1 -- 172.21.15.99:0/2489445307 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1558011b10 con 0x7f15680ad9b0 2023-10-08T00:16:57.669 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.664+0000 7f155dffb700 1 -- 172.21.15.99:0/2489445307 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15580253f0 con 0x7f15680ad9b0 2023-10-08T00:16:57.669 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.664+0000 7f155dffb700 1 -- 172.21.15.99:0/2489445307 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1558025790 con 0x7f15680ad9b0 2023-10-08T00:16:57.669 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.664+0000 7f156fc40700 1 -- 172.21.15.99:0/2489445307 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15680b2ee0 con 0x7f15680ad9b0 2023-10-08T00:16:57.669 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.665+0000 7f156fc40700 1 -- 172.21.15.99:0/2489445307 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15680b34a0 con 0x7f15680ad9b0 2023-10-08T00:16:57.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.665+0000 7f156fc40700 1 -- 172.21.15.99:0/2489445307 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f156804c3c0 con 0x7f15680ad9b0 2023-10-08T00:16:57.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.666+0000 7f155dffb700 1 -- 172.21.15.99:0/2489445307 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1558025b80 con 0x7f15680ad9b0 2023-10-08T00:16:57.673 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.666+0000 7f155dffb700 1 --2- 172.21.15.99:0/2489445307 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f154c066f30 0x7f154c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:57.673 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.666+0000 7f155dffb700 1 -- 172.21.15.99:0/2489445307 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1558085d20 con 0x7f15680ad9b0 2023-10-08T00:16:57.673 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.667+0000 7f156d9dc700 1 --2- 172.21.15.99:0/2489445307 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f154c066f30 0x7f154c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:57.673 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.667+0000 7f156d9dc700 1 --2- 172.21.15.99:0/2489445307 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f154c066f30 0x7f154c0693e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f1564007fd0 tx=0x7f1564009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:57.673 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.670+0000 7f155dffb700 1 -- 172.21.15.99:0/2489445307 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1558052f90 con 0x7f15680ad9b0 2023-10-08T00:16:57.903 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.902+0000 7f156fc40700 1 -- 172.21.15.99:0/2489445307 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f15680c8f80 con 0x7f15680ad9b0 2023-10-08T00:16:57.903 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.903+0000 7f155dffb700 1 -- 172.21.15.99:0/2489445307 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v63) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1558015060 con 0x7f15680ad9b0 2023-10-08T00:16:57.904 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T00:16:57.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.905+0000 7f156fc40700 1 -- 172.21.15.99:0/2489445307 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f154c066f30 msgr2=0x7f154c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:57.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.905+0000 7f156fc40700 1 --2- 172.21.15.99:0/2489445307 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f154c066f30 0x7f154c0693e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f1564007fd0 tx=0x7f1564009040 comp rx=0 tx=0).stop 2023-10-08T00:16:57.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.905+0000 7f156fc40700 1 -- 172.21.15.99:0/2489445307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15680ad9b0 msgr2=0x7f1568157640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:57.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.905+0000 7f156fc40700 1 --2- 172.21.15.99:0/2489445307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15680ad9b0 0x7f1568157640 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f15680ce880 tx=0x7f1558017000 comp rx=0 tx=0).stop 2023-10-08T00:16:57.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.905+0000 7f156fc40700 1 -- 172.21.15.99:0/2489445307 shutdown_connections 2023-10-08T00:16:57.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.905+0000 7f156fc40700 1 --2- 172.21.15.99:0/2489445307 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f154c066f30 0x7f154c0693e0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:57.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.905+0000 7f156fc40700 1 --2- 172.21.15.99:0/2489445307 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f156806a530 0x7f15680ae570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:57.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.905+0000 7f156fc40700 1 --2- 172.21.15.99:0/2489445307 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15681568d0 0x7f1568157b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:57.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.905+0000 7f156fc40700 1 --2- 172.21.15.99:0/2489445307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15680ad9b0 0x7f1568157640 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:57.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.905+0000 7f156fc40700 1 -- 172.21.15.99:0/2489445307 >> 172.21.15.99:0/2489445307 conn(0x7f15680cda50 msgr2=0x7f15680af2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:57.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.905+0000 7f156fc40700 1 -- 172.21.15.99:0/2489445307 shutdown_connections 2023-10-08T00:16:57.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:57.905+0000 7f156fc40700 1 -- 172.21.15.99:0/2489445307 wait complete. 2023-10-08T00:16:57.918 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T00:16:57.918 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T00:16:57.918 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi191.front.sepia.ceph.com 2023-10-08T00:16:57.918 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-08T00:16:57.918 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-08T00:16:57.918 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T00:16:57.918 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T00:16:57.918 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T00:16:57.919 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T00:16:57.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:16:57.919 DEBUG:teuthology.orchestra.run.smithi191:> ip addr 2023-10-08T00:16:57.936 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T00:16:57.937 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T00:16:57.937 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T00:16:57.937 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:16:57.937 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2023-10-08T00:16:57.937 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:16:57.937 INFO:teuthology.orchestra.run.smithi191.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:16:57.937 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f6 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:16:57.937 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f0 2023-10-08T00:16:57.937 INFO:teuthology.orchestra.run.smithi191.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:16:57.938 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f7 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:16:57.938 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f1 2023-10-08T00:16:57.938 INFO:teuthology.orchestra.run.smithi191.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:16:57.938 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:10 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:16:57.938 INFO:teuthology.orchestra.run.smithi191.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T00:16:57.938 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:11 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:16:57.938 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.21.15.191/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T00:16:57.938 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft 42174sec preferred_lft 42174sec 2023-10-08T00:16:57.939 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec4:7aff:fe88:7211/64 scope link 2023-10-08T00:16:57.939 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:16:57.939 INFO:teuthology.orchestra.run.smithi191.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T00:16:57.939 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 9a:bc:39:72:87:3d brd ff:ff:ff:ff:ff:ff 2023-10-08T00:16:57.939 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T00:16:57.939 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:16:57.939 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec77:a7ff:fea4:fbb/64 scope link 2023-10-08T00:16:57.939 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:16:57.939 INFO:teuthology.orchestra.run.smithi191.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T00:16:57.940 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether ea:1c:d9:46:05:b3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-08T00:16:57.941 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::e81c:d9ff:fe46:5b3/64 scope link 2023-10-08T00:16:57.941 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:16:57.944 INFO:teuthology.orchestra.run.smithi191.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T00:16:57.944 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 9a:bc:39:72:87:3d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-08T00:16:57.944 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::98bc:39ff:fe72:873d/64 scope link 2023-10-08T00:16:57.945 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:16:57.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:16:57.945 DEBUG:teuthology.orchestra.run.smithi191:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T00:16:58.002 INFO:teuthology.orchestra.run.smithi191.stdout:1 2023-10-08T00:16:58.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:16:58.004 DEBUG:teuthology.orchestra.run.smithi191:> route 2023-10-08T00:16:58.062 INFO:teuthology.orchestra.run.smithi191.stdout:Kernel IP routing table 2023-10-08T00:16:58.062 INFO:teuthology.orchestra.run.smithi191.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T00:16:58.062 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T00:16:58.062 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T00:16:58.062 INFO:teuthology.orchestra.run.smithi191.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T00:16:58.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:16:58.063 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2023-10-08T00:16:58.063 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2023-10-08T00:16:58.063 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T00:16:58.063 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T00:16:58.063 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T00:16:58.063 DEBUG:teuthology.orchestra.run.smithi191:> ') 2023-10-08T00:16:58.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:16:58.166 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2023-10-08T00:16:58.181 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-08T00:16:58.181 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-08T00:16:58.181 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T00:16:58.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:16:58.181 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set brx.0 up 2023-10-08T00:16:58.246 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:16:58.246 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-08T00:16:58.301 INFO:teuthology.orchestra.run.smithi191.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-08T00:16:58.301 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:16:58.301 DEBUG:teuthology.orchestra.run.smithi191:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-08T00:16:58.356 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-08T00:16:58.356 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T00:16:58.356 DEBUG:teuthology.orchestra.run.smithi191:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:16:58.436 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:16:58.436 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:16:58.455 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:16:58.455 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:16:58.455 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.99:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T00:16:58.455 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@06962cdb-9a6f-417f-82fd-9e4e3dedbccf.cephfs=/ 2023-10-08T00:16:58.455 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.99:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T00:16:58.455 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T00:16:58.456 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T00:16:58.456 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T00:16:58.476 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:16:58.477 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:16:58.477 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.99:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T00:16:58.477 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@06962cdb-9a6f-417f-82fd-9e4e3dedbccf.cephfs=/ 2023-10-08T00:16:58.477 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.99:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T00:16:58.477 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T00:16:58.477 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T00:16:58.477 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T00:16:58.477 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.99:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T00:16:58.477 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.99:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T00:16:58.477 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=06962cdb-9a6f-417f-82fd-9e4e3dedbccf" will pass to kernel 2023-10-08T00:16:58.478 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T00:16:58.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:16:58.479 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-08T00:16:58.543 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:16:58.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.657+0000 7fec1a018700 1 Processor -- start 2023-10-08T00:16:58.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.657+0000 7fec1a018700 1 -- start start 2023-10-08T00:16:58.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.658+0000 7fec1a018700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fec14157c40 0x7fec14158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:58.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.658+0000 7fec1a018700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec1415cbb0 con 0x7fec1415a620 2023-10-08T00:16:58.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.658+0000 7fec1a018700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec1415bfa0 con 0x7fec14157870 2023-10-08T00:16:58.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.658+0000 7fec1a018700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec1415bdb0 con 0x7fec14157c40 2023-10-08T00:16:58.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.658+0000 7fec13fff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fec14157c40 0x7fec14158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:58.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.658+0000 7fec12ffd700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fec1415a620 0x7fec1415d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:46864/0 (socket says 172.21.15.99:46864) 2023-10-08T00:16:58.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.658+0000 7fec12ffd700 1 -- 172.21.15.99:0/3403415092 learned_addr learned my addr 172.21.15.99:0/3403415092 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:16:58.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.658+0000 7fec13fff700 1 --2- 172.21.15.99:0/3403415092 >> v2:172.21.15.99:3300/0 conn(0x7fec14157c40 0x7fec14158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56062/0 (socket says 172.21.15.99:56062) 2023-10-08T00:16:58.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.659+0000 7fec13fff700 1 -- 172.21.15.99:0/3403415092 >> v1:172.21.15.191:6789/0 conn(0x7fec14157870 legacy=0x7fec1415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:58.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.659+0000 7fec13fff700 1 -- 172.21.15.99:0/3403415092 >> v1:172.21.15.99:6789/0 conn(0x7fec1415a620 legacy=0x7fec1415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:58.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.659+0000 7fec13fff700 1 -- 172.21.15.99:0/3403415092 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec1415c280 con 0x7fec14157c40 2023-10-08T00:16:58.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.659+0000 7fec127fc700 1 -- 172.21.15.99:0/3403415092 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3308001355 0 0) 0x7fec1415cbb0 con 0x7fec1415a620 2023-10-08T00:16:58.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.659+0000 7fec13fff700 1 --2- 172.21.15.99:0/3403415092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec14157c40 0x7fec14158110 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fec0c0098e0 tx=0x7fec0c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c8f14b738f7ab8e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:58.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.659+0000 7fec127fc700 1 -- 172.21.15.99:0/3403415092 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0c009e80 con 0x7fec14157c40 2023-10-08T00:16:58.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.659+0000 7fec127fc700 1 -- 172.21.15.99:0/3403415092 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec0c00b860 con 0x7fec14157c40 2023-10-08T00:16:58.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.659+0000 7fec127fc700 1 -- 172.21.15.99:0/3403415092 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0c00bbe0 con 0x7fec14157c40 2023-10-08T00:16:58.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.659+0000 7fec1a018700 1 -- 172.21.15.99:0/3403415092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec14157c40 msgr2=0x7fec14158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:58.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.659+0000 7fec1a018700 1 --2- 172.21.15.99:0/3403415092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec14157c40 0x7fec14158110 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fec0c0098e0 tx=0x7fec0c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:16:58.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.659+0000 7fec1a018700 1 -- 172.21.15.99:0/3403415092 shutdown_connections 2023-10-08T00:16:58.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.659+0000 7fec1a018700 1 --2- 172.21.15.99:0/3403415092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec14157c40 0x7fec14158110 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:58.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.659+0000 7fec1a018700 1 -- 172.21.15.99:0/3403415092 >> 172.21.15.99:0/3403415092 conn(0x7fec14068ce0 msgr2=0x7fec140690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:58.666 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.659+0000 7fec1a018700 1 -- 172.21.15.99:0/3403415092 shutdown_connections 2023-10-08T00:16:58.666 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.659+0000 7fec1a018700 1 -- 172.21.15.99:0/3403415092 wait complete. 2023-10-08T00:16:58.669 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.661+0000 7fec1a018700 1 Processor -- start 2023-10-08T00:16:58.669 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.661+0000 7fec1a018700 1 -- start start 2023-10-08T00:16:58.670 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.661+0000 7fec1a018700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fec14157870 0x7fec14160c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:58.670 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.661+0000 7fec1a018700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec14157c40 0x7fec1415df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:58.670 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.661+0000 7fec1a018700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fec1415a620 0x7fec141486e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:58.670 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.661+0000 7fec1a018700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1415e740 con 0x7fec14157c40 2023-10-08T00:16:58.670 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.661+0000 7fec1a018700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1415ea20 con 0x7fec1415a620 2023-10-08T00:16:58.671 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.661+0000 7fec1a018700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec1415ed00 con 0x7fec14157870 2023-10-08T00:16:58.671 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.661+0000 7fec12ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec14157c40 0x7fec1415df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:58.671 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.661+0000 7fec12ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec14157c40 0x7fec1415df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56070/0 (socket says 172.21.15.99:56070) 2023-10-08T00:16:58.671 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.661+0000 7fec12ffd700 1 -- 172.21.15.99:0/2333959934 learned_addr learned my addr 172.21.15.99:0/2333959934 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:58.671 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.662+0000 7fec13fff700 1 --2- 172.21.15.99:0/2333959934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fec1415a620 0x7fec141486e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:58.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.662+0000 7fec137fe700 1 --2- 172.21.15.99:0/2333959934 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fec14157870 0x7fec14160c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:58.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.662+0000 7fec12ffd700 1 -- 172.21.15.99:0/2333959934 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fec14157870 msgr2=0x7fec14160c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:58.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.662+0000 7fec12ffd700 1 --2- 172.21.15.99:0/2333959934 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fec14157870 0x7fec14160c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:58.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.662+0000 7fec12ffd700 1 -- 172.21.15.99:0/2333959934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fec1415a620 msgr2=0x7fec141486e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:58.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.662+0000 7fec12ffd700 1 --2- 172.21.15.99:0/2333959934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fec1415a620 0x7fec141486e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:58.673 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.662+0000 7fec12ffd700 1 -- 172.21.15.99:0/2333959934 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec0c009420 con 0x7fec14157c40 2023-10-08T00:16:58.673 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.662+0000 7fec137fe700 1 --2- 172.21.15.99:0/2333959934 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fec14157870 0x7fec14160c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:58.673 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.662+0000 7fec12ffd700 1 --2- 172.21.15.99:0/2333959934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec14157c40 0x7fec1415df10 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fec0800c530 tx=0x7fec08007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:58.673 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.662+0000 7febfb7fe700 1 -- 172.21.15.99:0/2333959934 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec08005b30 con 0x7fec14157c40 2023-10-08T00:16:58.673 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.662+0000 7febfb7fe700 1 -- 172.21.15.99:0/2333959934 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec08005c90 con 0x7fec14157c40 2023-10-08T00:16:58.674 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.662+0000 7febfb7fe700 1 -- 172.21.15.99:0/2333959934 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec08007320 con 0x7fec14157c40 2023-10-08T00:16:58.674 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.662+0000 7fec1a018700 1 -- 172.21.15.99:0/2333959934 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec1415efe0 con 0x7fec14157c40 2023-10-08T00:16:58.674 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.662+0000 7fec1a018700 1 -- 172.21.15.99:0/2333959934 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec1415e450 con 0x7fec14157c40 2023-10-08T00:16:58.674 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.663+0000 7fec1a018700 1 -- 172.21.15.99:0/2333959934 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec140d1760 con 0x7fec14157c40 2023-10-08T00:16:58.674 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.663+0000 7febfb7fe700 1 -- 172.21.15.99:0/2333959934 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fec0801b3b0 con 0x7fec14157c40 2023-10-08T00:16:58.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.664+0000 7febfb7fe700 1 --2- 172.21.15.99:0/2333959934 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fec00066f30 0x7fec000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:58.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.664+0000 7febfb7fe700 1 -- 172.21.15.99:0/2333959934 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fec080839a0 con 0x7fec14157c40 2023-10-08T00:16:58.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.664+0000 7fec137fe700 1 --2- 172.21.15.99:0/2333959934 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fec00066f30 0x7fec000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:58.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.664+0000 7fec137fe700 1 --2- 172.21.15.99:0/2333959934 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fec00066f30 0x7fec000693e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fec04007fd0 tx=0x7fec04009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:58.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.668+0000 7febfb7fe700 1 -- 172.21.15.99:0/2333959934 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fec08050c10 con 0x7fec14157c40 2023-10-08T00:16:58.898 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.896+0000 7fec1a018700 1 -- 172.21.15.99:0/2333959934 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec1415f4e0 con 0x7fec14157c40 2023-10-08T00:16:58.898 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.897+0000 7febfb7fe700 1 -- 172.21.15.99:0/2333959934 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4602 (secure 0 0 0) 0x7fec08017090 con 0x7fec14157c40 2023-10-08T00:16:58.898 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:16:58.899 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":63,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5813,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/887374839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":887374839},{"type":"v1","addr":"172.21.15.191:6837","nonce":887374839}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":6445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/2409696901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2409696901},{"type":"v1","addr":"172.21.15.191:6836","nonce":2409696901}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":7095,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/1924096228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1924096228},{"type":"v1","addr":"172.21.15.99:6834","nonce":1924096228}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:16:50.092494+0000","modified":"2023-10-08T00:16:53.744105+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7098":{"gid":7098,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/2522933027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":2522933027},{"type":"v1","addr":"172.21.15.99:6835","nonce":2522933027}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-08T00:16:58.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.899+0000 7fec1a018700 1 -- 172.21.15.99:0/2333959934 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fec00066f30 msgr2=0x7fec000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:58.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.899+0000 7fec1a018700 1 --2- 172.21.15.99:0/2333959934 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fec00066f30 0x7fec000693e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fec04007fd0 tx=0x7fec04009040 comp rx=0 tx=0).stop 2023-10-08T00:16:58.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.899+0000 7fec1a018700 1 -- 172.21.15.99:0/2333959934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec14157c40 msgr2=0x7fec1415df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:58.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.899+0000 7fec1a018700 1 --2- 172.21.15.99:0/2333959934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec14157c40 0x7fec1415df10 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fec0800c530 tx=0x7fec08007f90 comp rx=0 tx=0).stop 2023-10-08T00:16:58.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.899+0000 7fec1a018700 1 -- 172.21.15.99:0/2333959934 shutdown_connections 2023-10-08T00:16:58.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.899+0000 7fec1a018700 1 --2- 172.21.15.99:0/2333959934 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fec00066f30 0x7fec000693e0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:58.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.899+0000 7fec1a018700 1 --2- 172.21.15.99:0/2333959934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fec1415a620 0x7fec141486e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:58.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.899+0000 7fec1a018700 1 --2- 172.21.15.99:0/2333959934 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fec14157870 0x7fec14160c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:58.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.899+0000 7fec1a018700 1 --2- 172.21.15.99:0/2333959934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec14157c40 0x7fec1415df10 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:58.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.899+0000 7fec1a018700 1 -- 172.21.15.99:0/2333959934 >> 172.21.15.99:0/2333959934 conn(0x7fec14068ce0 msgr2=0x7fec141580f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:58.901 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.899+0000 7fec1a018700 1 -- 172.21.15.99:0/2333959934 shutdown_connections 2023-10-08T00:16:58.902 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:58.899+0000 7fec1a018700 1 -- 172.21.15.99:0/2333959934 wait complete. 2023-10-08T00:16:58.902 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 63 2023-10-08T00:16:58.913 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2023-10-08T00:16:59.046 INFO:teuthology.orchestra.run.smithi099.stdout:{"cluster_fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","whoami":0,"id":7098,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":8.2974688560000001,"mdsmap_epoch":63,"osdmap_epoch":139,"osdmap_epoch_barrier":139,"uptime":13.432307975000001} 2023-10-08T00:16:59.046 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", "fs_name": "cephfs", "id": 7098, "mdsmap_epoch": 63, "osdmap_epoch": 139, "osdmap_epoch_barrier": 139, "rank_uptime": 8.297468856, "state": "up:active", "uptime": 13.432307975, "want_state": "up:active", "whoami": 0 } 2023-10-08T00:16:59.046 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2023-10-08T00:16:59.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.163+0000 7fb897774700 1 Processor -- start 2023-10-08T00:16:59.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.163+0000 7fb897774700 1 -- start start 2023-10-08T00:16:59.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.163+0000 7fb897774700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb89015a0d0 0x7fb8900c8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:59.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.163+0000 7fb897774700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb89006f6c0 0x7fb8900cb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:59.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.163+0000 7fb897774700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8900c9690 con 0x7fb8900c8b60 2023-10-08T00:16:59.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.163+0000 7fb897774700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8900cb270 con 0x7fb89006f6c0 2023-10-08T00:16:59.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.163+0000 7fb897774700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8900c90d0 con 0x7fb89015a0d0 2023-10-08T00:16:59.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.163+0000 7fb895d11700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb89006f6c0 0x7fb8900cb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:59.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.164+0000 7fb895d11700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb89006f6c0 0x7fb8900cb700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56080/0 (socket says 172.21.15.99:56080) 2023-10-08T00:16:59.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.164+0000 7fb895d11700 1 -- 172.21.15.99:0/2846990446 learned_addr learned my addr 172.21.15.99:0/2846990446 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:59.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.164+0000 7fb895510700 1 --2- 172.21.15.99:0/2846990446 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb89015a0d0 0x7fb8900c8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:59.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.164+0000 7fb895d11700 1 -- 172.21.15.99:0/2846990446 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb89015a0d0 msgr2=0x7fb8900c8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:16:59.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.164+0000 7fb895d11700 1 --2- 172.21.15.99:0/2846990446 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb89015a0d0 0x7fb8900c8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:59.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.164+0000 7fb895d11700 1 -- 172.21.15.99:0/2846990446 >> v1:172.21.15.191:6789/0 conn(0x7fb8900c8b60 legacy=0x7fb89006efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:59.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.164+0000 7fb895d11700 1 -- 172.21.15.99:0/2846990446 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8900c93b0 con 0x7fb89006f6c0 2023-10-08T00:16:59.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.164+0000 7fb895510700 1 --2- 172.21.15.99:0/2846990446 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb89015a0d0 0x7fb8900c8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:16:59.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.164+0000 7fb895d11700 1 --2- 172.21.15.99:0/2846990446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb89006f6c0 0x7fb8900cb700 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fb88c0099b0 tx=0x7fb88c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31c54df3d9ed1ae2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:59.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.164+0000 7fb887fff700 1 -- 172.21.15.99:0/2846990446 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88c00b860 con 0x7fb89006f6c0 2023-10-08T00:16:59.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.164+0000 7fb887fff700 1 -- 172.21.15.99:0/2846990446 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb88c00b9c0 con 0x7fb89006f6c0 2023-10-08T00:16:59.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.164+0000 7fb887fff700 1 -- 172.21.15.99:0/2846990446 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88c00bd60 con 0x7fb89006f6c0 2023-10-08T00:16:59.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.165+0000 7fb897774700 1 -- 172.21.15.99:0/2846990446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb89006f6c0 msgr2=0x7fb8900cb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:59.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.165+0000 7fb897774700 1 --2- 172.21.15.99:0/2846990446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb89006f6c0 0x7fb8900cb700 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fb88c0099b0 tx=0x7fb88c009250 comp rx=0 tx=0).stop 2023-10-08T00:16:59.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.165+0000 7fb897774700 1 -- 172.21.15.99:0/2846990446 shutdown_connections 2023-10-08T00:16:59.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.165+0000 7fb897774700 1 --2- 172.21.15.99:0/2846990446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb89006f6c0 0x7fb8900cb700 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:59.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.165+0000 7fb897774700 1 --2- 172.21.15.99:0/2846990446 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb89015a0d0 0x7fb8900c8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:59.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.165+0000 7fb897774700 1 -- 172.21.15.99:0/2846990446 >> 172.21.15.99:0/2846990446 conn(0x7fb8900cd230 msgr2=0x7fb8900cd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:59.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.165+0000 7fb897774700 1 -- 172.21.15.99:0/2846990446 shutdown_connections 2023-10-08T00:16:59.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.165+0000 7fb897774700 1 -- 172.21.15.99:0/2846990446 wait complete. 2023-10-08T00:16:59.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.166+0000 7fb897774700 1 Processor -- start 2023-10-08T00:16:59.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.166+0000 7fb897774700 1 -- start start 2023-10-08T00:16:59.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.167+0000 7fb897774700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb89006f6c0 0x7fb8901580b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:59.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.167+0000 7fb897774700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8900c8b60 0x7fb8900c7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:59.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.167+0000 7fb897774700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb89015a0d0 0x7fb8900ae670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:59.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.167+0000 7fb897774700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8900c75c0 con 0x7fb89006f6c0 2023-10-08T00:16:59.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.167+0000 7fb897774700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8900c1080 con 0x7fb8900c8b60 2023-10-08T00:16:59.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.167+0000 7fb897774700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8900ac690 con 0x7fb89015a0d0 2023-10-08T00:16:59.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.167+0000 7fb895510700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb89006f6c0 0x7fb8901580b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:59.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.167+0000 7fb895510700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb89006f6c0 0x7fb8901580b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56084/0 (socket says 172.21.15.99:56084) 2023-10-08T00:16:59.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.167+0000 7fb895510700 1 -- 172.21.15.99:0/4228814879 learned_addr learned my addr 172.21.15.99:0/4228814879 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:16:59.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.167+0000 7fb894d0f700 1 --2- 172.21.15.99:0/4228814879 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8900c8b60 0x7fb8900c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:59.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.167+0000 7fb895d11700 1 --2- 172.21.15.99:0/4228814879 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb89015a0d0 0x7fb8900ae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:59.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.167+0000 7fb895510700 1 -- 172.21.15.99:0/4228814879 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb89015a0d0 msgr2=0x7fb8900ae670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:59.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.167+0000 7fb895510700 1 --2- 172.21.15.99:0/4228814879 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb89015a0d0 0x7fb8900ae670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:59.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.167+0000 7fb895510700 1 -- 172.21.15.99:0/4228814879 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8900c8b60 msgr2=0x7fb8900c7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:59.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.167+0000 7fb895510700 1 --2- 172.21.15.99:0/4228814879 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8900c8b60 0x7fb8900c7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:59.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.167+0000 7fb895510700 1 -- 172.21.15.99:0/4228814879 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb88c0094f0 con 0x7fb89006f6c0 2023-10-08T00:16:59.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.167+0000 7fb895510700 1 --2- 172.21.15.99:0/4228814879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb89006f6c0 0x7fb8901580b0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fb88800d1d0 tx=0x7fb888009a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:59.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.167+0000 7fb8857fa700 1 -- 172.21.15.99:0/4228814879 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88800dd10 con 0x7fb89006f6c0 2023-10-08T00:16:59.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.167+0000 7fb8857fa700 1 -- 172.21.15.99:0/4228814879 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb88800f040 con 0x7fb89006f6c0 2023-10-08T00:16:59.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.167+0000 7fb897774700 1 -- 172.21.15.99:0/4228814879 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8900c66c0 con 0x7fb89006f6c0 2023-10-08T00:16:59.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.167+0000 7fb8857fa700 1 -- 172.21.15.99:0/4228814879 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb888015670 con 0x7fb89006f6c0 2023-10-08T00:16:59.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.168+0000 7fb897774700 1 -- 172.21.15.99:0/4228814879 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8900c6100 con 0x7fb89006f6c0 2023-10-08T00:16:59.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.168+0000 7fb897774700 1 -- 172.21.15.99:0/4228814879 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8900c1360 con 0x7fb89006f6c0 2023-10-08T00:16:59.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.169+0000 7fb8857fa700 1 -- 172.21.15.99:0/4228814879 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb888015a60 con 0x7fb89006f6c0 2023-10-08T00:16:59.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.169+0000 7fb8857fa700 1 --2- 172.21.15.99:0/4228814879 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb874066f30 0x7fb8740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:16:59.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.169+0000 7fb8857fa700 1 -- 172.21.15.99:0/4228814879 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb8880848f0 con 0x7fb89006f6c0 2023-10-08T00:16:59.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.169+0000 7fb894d0f700 1 --2- 172.21.15.99:0/4228814879 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb874066f30 0x7fb8740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:16:59.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.170+0000 7fb894d0f700 1 --2- 172.21.15.99:0/4228814879 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb874066f30 0x7fb8740693e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb89009b020 tx=0x7fb880007ec0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:16:59.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.174+0000 7fb8857fa700 1 -- 172.21.15.99:0/4228814879 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb88804e5a0 con 0x7fb89006f6c0 2023-10-08T00:16:59.359 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.357+0000 7fb897774700 1 -- 172.21.15.99:0/4228814879 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7fb89004c3c0 con 0x7fb89006f6c0 2023-10-08T00:16:59.769 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.768+0000 7fb8857fa700 1 -- 172.21.15.99:0/4228814879 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v140) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb88801a0c0 con 0x7fb89006f6c0 2023-10-08T00:16:59.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.770+0000 7fb897774700 1 -- 172.21.15.99:0/4228814879 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb874066f30 msgr2=0x7fb8740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:59.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.770+0000 7fb897774700 1 --2- 172.21.15.99:0/4228814879 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb874066f30 0x7fb8740693e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb89009b020 tx=0x7fb880007ec0 comp rx=0 tx=0).stop 2023-10-08T00:16:59.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.770+0000 7fb897774700 1 -- 172.21.15.99:0/4228814879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb89006f6c0 msgr2=0x7fb8901580b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:16:59.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.770+0000 7fb897774700 1 --2- 172.21.15.99:0/4228814879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb89006f6c0 0x7fb8901580b0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fb88800d1d0 tx=0x7fb888009a90 comp rx=0 tx=0).stop 2023-10-08T00:16:59.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.770+0000 7fb897774700 1 -- 172.21.15.99:0/4228814879 shutdown_connections 2023-10-08T00:16:59.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.770+0000 7fb897774700 1 --2- 172.21.15.99:0/4228814879 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb874066f30 0x7fb8740693e0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:59.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.770+0000 7fb897774700 1 --2- 172.21.15.99:0/4228814879 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb89015a0d0 0x7fb8900ae670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:59.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.770+0000 7fb897774700 1 --2- 172.21.15.99:0/4228814879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb89006f6c0 0x7fb8901580b0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:59.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.770+0000 7fb897774700 1 --2- 172.21.15.99:0/4228814879 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8900c8b60 0x7fb8900c7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:16:59.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.770+0000 7fb897774700 1 -- 172.21.15.99:0/4228814879 >> 172.21.15.99:0/4228814879 conn(0x7fb8900cd230 msgr2=0x7fb8900aa4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:16:59.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.770+0000 7fb897774700 1 -- 172.21.15.99:0/4228814879 shutdown_connections 2023-10-08T00:16:59.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:16:59.770+0000 7fb897774700 1 -- 172.21.15.99:0/4228814879 wait complete. 2023-10-08T00:16:59.774 INFO:teuthology.orchestra.run.smithi099.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2023-10-08T00:16:59.784 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-10-08T00:16:59.912 INFO:teuthology.orchestra.run.smithi099.stdout:{"osd_mon_report_interval":"5"} 2023-10-08T00:16:59.912 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-10-08T00:16:59.913 INFO:tasks.cephfs.test_full:Writing 32MB should fill this cluster 2023-10-08T00:16:59.913 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:16:59.913 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=16 seek=0) 2023-10-08T00:17:00.031 INFO:teuthology.orchestra.run.smithi099.stderr:16+0 records in 2023-10-08T00:17:00.031 INFO:teuthology.orchestra.run.smithi099.stderr:16+0 records out 2023-10-08T00:17:00.031 INFO:teuthology.orchestra.run.smithi099.stderr:16777216 bytes (17 MB, 16 MiB) copied, 0.0998942 s, 168 MB/s 2023-10-08T00:17:00.031 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:17:00.031 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=17 seek=0) 2023-10-08T00:17:00.139 INFO:teuthology.orchestra.run.smithi099.stderr:17+0 records in 2023-10-08T00:17:00.139 INFO:teuthology.orchestra.run.smithi099.stderr:17+0 records out 2023-10-08T00:17:00.139 INFO:teuthology.orchestra.run.smithi099.stderr:17825792 bytes (18 MB, 17 MiB) copied, 0.0917165 s, 194 MB/s 2023-10-08T00:17:00.140 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2023-10-08T00:17:00.140 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-08T00:17:00.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.253+0000 7f32d8029700 1 Processor -- start 2023-10-08T00:17:00.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.253+0000 7f32d8029700 1 -- start start 2023-10-08T00:17:00.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.254+0000 7f32d8029700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f32d00abae0 0x7f32d00c6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:00.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.254+0000 7f32d8029700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f32d00c8b10 0x7f32d00c73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:00.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.254+0000 7f32d8029700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32d00c9cb0 con 0x7f32d00abeb0 2023-10-08T00:17:00.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.254+0000 7f32d8029700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32d00c90a0 con 0x7f32d00c8b10 2023-10-08T00:17:00.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.254+0000 7f32d8029700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32d00c9380 con 0x7f32d00abae0 2023-10-08T00:17:00.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.254+0000 7f32d65c6700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f32d00c8b10 0x7f32d00c73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:00.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.254+0000 7f32d65c6700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f32d00c8b10 0x7f32d00c73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56094/0 (socket says 172.21.15.99:56094) 2023-10-08T00:17:00.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.254+0000 7f32d65c6700 1 -- 172.21.15.99:0/3879587660 learned_addr learned my addr 172.21.15.99:0/3879587660 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:00.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.254+0000 7f32d5dc5700 1 --2- 172.21.15.99:0/3879587660 >> v2:172.21.15.191:3300/0 conn(0x7f32d00abae0 0x7f32d00c6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:00.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.255+0000 7f32d65c6700 1 -- 172.21.15.99:0/3879587660 >> v2:172.21.15.191:3300/0 conn(0x7f32d00abae0 msgr2=0x7f32d00c6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:00.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.255+0000 7f32d65c6700 1 --2- 172.21.15.99:0/3879587660 >> v2:172.21.15.191:3300/0 conn(0x7f32d00abae0 0x7f32d00c6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:00.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.255+0000 7f32d65c6700 1 -- 172.21.15.99:0/3879587660 >> v1:172.21.15.191:6789/0 conn(0x7f32d00abeb0 legacy=0x7f32d00c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:00.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.255+0000 7f32d65c6700 1 -- 172.21.15.99:0/3879587660 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d00c9660 con 0x7f32d00c8b10 2023-10-08T00:17:00.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.255+0000 7f32d65c6700 1 --2- 172.21.15.99:0/3879587660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32d00c8b10 0x7f32d00c73f0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f32d00cc7f0 tx=0x7f32c800ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61204434963c408c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:00.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.255+0000 7f32d4dc3700 1 -- 172.21.15.99:0/3879587660 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c8010070 con 0x7f32d00c8b10 2023-10-08T00:17:00.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.255+0000 7f32d4dc3700 1 -- 172.21.15.99:0/3879587660 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c800dde0 con 0x7f32d00c8b10 2023-10-08T00:17:00.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.255+0000 7f32d4dc3700 1 -- 172.21.15.99:0/3879587660 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c800b320 con 0x7f32d00c8b10 2023-10-08T00:17:00.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.255+0000 7f32d8029700 1 -- 172.21.15.99:0/3879587660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32d00c8b10 msgr2=0x7f32d00c73f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:00.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.255+0000 7f32d8029700 1 --2- 172.21.15.99:0/3879587660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32d00c8b10 0x7f32d00c73f0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f32d00cc7f0 tx=0x7f32c800ba50 comp rx=0 tx=0).stop 2023-10-08T00:17:00.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.255+0000 7f32d8029700 1 -- 172.21.15.99:0/3879587660 shutdown_connections 2023-10-08T00:17:00.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.255+0000 7f32d8029700 1 --2- 172.21.15.99:0/3879587660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32d00c8b10 0x7f32d00c73f0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:00.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.255+0000 7f32d8029700 1 --2- 172.21.15.99:0/3879587660 >> v2:172.21.15.191:3300/0 conn(0x7f32d00abae0 0x7f32d00c6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:00.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.255+0000 7f32d8029700 1 -- 172.21.15.99:0/3879587660 >> 172.21.15.99:0/3879587660 conn(0x7f32d0067660 msgr2=0x7f32d0067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:00.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.256+0000 7f32d8029700 1 -- 172.21.15.99:0/3879587660 shutdown_connections 2023-10-08T00:17:00.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.256+0000 7f32d8029700 1 -- 172.21.15.99:0/3879587660 wait complete. 2023-10-08T00:17:00.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.257+0000 7f32d8029700 1 Processor -- start 2023-10-08T00:17:00.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.257+0000 7f32d8029700 1 -- start start 2023-10-08T00:17:00.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.257+0000 7f32d8029700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32d00abae0 0x7f32d014de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:00.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.257+0000 7f32d8029700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f32d00abeb0 0x7f32d0156ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:00.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.257+0000 7f32d8029700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32d00c8b10 0x7f32d00c5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:00.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.257+0000 7f32d8029700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d00c02b0 con 0x7f32d00c8b10 2023-10-08T00:17:00.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.257+0000 7f32d8029700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d00b0f50 con 0x7f32d00abae0 2023-10-08T00:17:00.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.257+0000 7f32d8029700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d014e370 con 0x7f32d00abeb0 2023-10-08T00:17:00.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.257+0000 7f32d65c6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32d00c8b10 0x7f32d00c5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:00.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.257+0000 7f32d65c6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32d00c8b10 0x7f32d00c5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56098/0 (socket says 172.21.15.99:56098) 2023-10-08T00:17:00.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.257+0000 7f32d65c6700 1 -- 172.21.15.99:0/3760053088 learned_addr learned my addr 172.21.15.99:0/3760053088 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:00.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.257+0000 7f32d5dc5700 1 --2- 172.21.15.99:0/3760053088 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32d00abae0 0x7f32d014de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:00.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.257+0000 7f32d55c4700 1 --2- 172.21.15.99:0/3760053088 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f32d00abeb0 0x7f32d0156ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:00.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.258+0000 7f32d65c6700 1 -- 172.21.15.99:0/3760053088 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f32d00abeb0 msgr2=0x7f32d0156ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:00.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.258+0000 7f32d65c6700 1 --2- 172.21.15.99:0/3760053088 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f32d00abeb0 0x7f32d0156ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:00.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.258+0000 7f32d65c6700 1 -- 172.21.15.99:0/3760053088 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32d00abae0 msgr2=0x7f32d014de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:00.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.258+0000 7f32d65c6700 1 --2- 172.21.15.99:0/3760053088 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32d00abae0 0x7f32d014de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:00.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.258+0000 7f32d65c6700 1 -- 172.21.15.99:0/3760053088 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c800bcf0 con 0x7f32d00c8b10 2023-10-08T00:17:00.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.258+0000 7f32d55c4700 1 --2- 172.21.15.99:0/3760053088 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f32d00abeb0 0x7f32d0156ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:17:00.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.258+0000 7f32d65c6700 1 --2- 172.21.15.99:0/3760053088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32d00c8b10 0x7f32d00c5b10 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f32d006ba00 tx=0x7f32c801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:00.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.258+0000 7f32c5ffb700 1 -- 172.21.15.99:0/3760053088 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c800b4d0 con 0x7f32d00c8b10 2023-10-08T00:17:00.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.258+0000 7f32c5ffb700 1 -- 172.21.15.99:0/3760053088 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c800ae10 con 0x7f32d00c8b10 2023-10-08T00:17:00.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.258+0000 7f32d8029700 1 -- 172.21.15.99:0/3760053088 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32d00b0340 con 0x7f32d00c8b10 2023-10-08T00:17:00.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.258+0000 7f32c5ffb700 1 -- 172.21.15.99:0/3760053088 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c801f880 con 0x7f32d00c8b10 2023-10-08T00:17:00.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.258+0000 7f32d8029700 1 -- 172.21.15.99:0/3760053088 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32d00b0900 con 0x7f32d00c8b10 2023-10-08T00:17:00.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.259+0000 7f32d8029700 1 -- 172.21.15.99:0/3760053088 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32d00b10c0 con 0x7f32d00c8b10 2023-10-08T00:17:00.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.259+0000 7f32c5ffb700 1 -- 172.21.15.99:0/3760053088 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f32c8003040 con 0x7f32d00c8b10 2023-10-08T00:17:00.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.260+0000 7f32c5ffb700 1 --2- 172.21.15.99:0/3760053088 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f32b4066f30 0x7f32b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:00.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.260+0000 7f32c5ffb700 1 -- 172.21.15.99:0/3760053088 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f32c8088630 con 0x7f32d00c8b10 2023-10-08T00:17:00.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.260+0000 7f32d5dc5700 1 --2- 172.21.15.99:0/3760053088 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f32b4066f30 0x7f32b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:00.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.260+0000 7f32d5dc5700 1 --2- 172.21.15.99:0/3760053088 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f32b4066f30 0x7f32b40693e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f32cc009d10 tx=0x7f32cc0091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:00.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.264+0000 7f32c5ffb700 1 -- 172.21.15.99:0/3760053088 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f32c80558a0 con 0x7f32d00c8b10 2023-10-08T00:17:00.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.450+0000 7f32d8029700 1 -- 172.21.15.99:0/3760053088 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f32d004c3c0 con 0x7f32d00c8b10 2023-10-08T00:17:00.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.451+0000 7f32c5ffb700 1 -- 172.21.15.99:0/3760053088 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v140) v1 ==== 81+0+32282 (secure 0 0 0) 0x7f32c8058ec0 con 0x7f32d00c8b10 2023-10-08T00:17:00.452 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:17:00.452 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:17:00.452 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 140, 2023-10-08T00:17:00.452 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", 2023-10-08T00:17:00.452 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2023-10-08T00:10:36.303601+0000", 2023-10-08T00:17:00.452 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2023-10-08T00:16:59.766183+0000", 2023-10-08T00:17:00.452 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2023-10-08T00:10:41.709171+0000", 2023-10-08T00:17:00.452 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2023-10-08T00:10:38.690404+0000", 2023-10-08T00:17:00.452 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-08T00:17:00.452 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2023-10-08T00:17:00.452 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2023-10-08T00:17:00.453 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2023-10-08T00:17:00.453 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2023-10-08T00:17:00.453 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2023-10-08T00:17:00.453 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2023-10-08T00:17:00.453 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:17:00.453 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2023-10-08T00:17:00.453 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2023-10-08T00:17:00.453 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-08T00:17:00.453 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-08T00:17:00.453 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2023-10-08T00:17:00.454 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 17, 2023-10-08T00:17:00.454 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2023-10-08T00:17:00.454 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2023-10-08T00:17:00.454 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "jewel", 2023-10-08T00:17:00.454 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "quincy", 2023-10-08T00:17:00.454 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2023-10-08T00:17:00.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.454 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2023-10-08T00:17:00.454 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2023-10-08T00:17:00.454 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:10:42.063344+0000", 2023-10-08T00:17:00.454 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:17:00.455 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:17:00.455 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:17:00.455 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:17:00.455 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:17:00.455 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:17:00.455 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:17:00.455 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:17:00.455 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:17:00.455 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:17:00.455 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:17:00.455 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:17:00.456 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2023-10-08T00:17:00.456 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2023-10-08T00:17:00.456 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2023-10-08T00:17:00.456 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2023-10-08T00:17:00.456 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2023-10-08T00:17:00.456 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:17:00.456 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:17:00.456 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:17:00.456 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:17:00.456 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:17:00.456 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:17:00.457 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:17:00.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.457 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "112", 2023-10-08T00:17:00.457 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:17:00.457 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:17:00.457 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:17:00.457 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:17:00.457 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:17:00.457 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:17:00.457 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:17:00.457 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:17:00.457 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:17:00.458 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:17:00.458 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:17:00.458 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:17:00.458 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:17:00.458 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:17:00.458 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:17:00.458 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:17:00.458 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:17:00.458 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:17:00.458 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:17:00.458 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:17:00.458 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:17:00.459 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:17:00.459 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:17:00.459 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:17:00.459 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:17:00.459 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:17:00.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.459 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:17:00.459 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:17:00.459 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:17:00.459 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:17:00.459 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:17:00.460 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:17:00.460 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:17:00.460 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:17:00.460 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:17:00.460 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:17:00.460 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:17:00.460 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:17:00.460 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2023-10-08T00:17:00.460 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2023-10-08T00:17:00.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.460 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:17:00.460 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2023-10-08T00:17:00.461 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:00.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.461 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 16, 2023-10-08T00:17:00.461 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2023-10-08T00:17:00.461 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:16:48.430607+0000", 2023-10-08T00:17:00.461 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:17:00.461 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:17:00.461 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:17:00.461 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:17:00.461 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:17:00.461 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:17:00.462 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:17:00.462 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:17:00.462 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:17:00.462 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:17:00.462 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:17:00.462 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:17:00.462 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2023-10-08T00:17:00.462 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2023-10-08T00:17:00.462 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2023-10-08T00:17:00.462 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2023-10-08T00:17:00.462 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2023-10-08T00:17:00.463 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:17:00.463 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:17:00.463 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:17:00.463 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:17:00.463 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:17:00.463 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:17:00.463 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:17:00.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.463 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "138", 2023-10-08T00:17:00.463 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:17:00.463 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:17:00.463 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:17:00.464 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:17:00.464 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:17:00.464 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:17:00.464 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:17:00.464 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:17:00.464 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:17:00.464 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:17:00.464 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:17:00.464 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:17:00.464 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:17:00.464 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:17:00.465 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:17:00.465 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:17:00.465 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:17:00.465 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:17:00.465 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:17:00.465 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:17:00.465 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:17:00.465 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:17:00.465 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:17:00.465 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:17:00.465 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:17:00.465 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:17:00.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.466 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:17:00.466 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:17:00.466 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:17:00.466 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:17:00.466 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:17:00.466 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:17:00.466 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:17:00.466 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:17:00.466 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:17:00.466 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:17:00.467 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:17:00.467 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:17:00.467 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2023-10-08T00:17:00.467 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2023-10-08T00:17:00.467 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2023-10-08T00:17:00.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.467 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:17:00.467 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:17:00.467 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2023-10-08T00:17:00.467 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:00.467 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:00.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.468 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 17, 2023-10-08T00:17:00.468 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2023-10-08T00:17:00.468 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:16:49.063424+0000", 2023-10-08T00:17:00.468 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:17:00.468 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:17:00.468 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:17:00.468 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:17:00.468 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:17:00.468 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:17:00.468 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:17:00.468 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:17:00.469 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:17:00.469 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:17:00.469 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:17:00.469 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:17:00.469 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2023-10-08T00:17:00.469 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2023-10-08T00:17:00.469 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2023-10-08T00:17:00.469 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2023-10-08T00:17:00.469 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2023-10-08T00:17:00.469 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:17:00.469 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:17:00.469 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:17:00.470 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:17:00.470 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:17:00.470 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:17:00.470 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:17:00.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.470 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "140", 2023-10-08T00:17:00.470 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:17:00.470 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:17:00.470 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:17:00.470 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:17:00.470 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:17:00.471 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:17:00.471 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:17:00.471 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:17:00.471 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:17:00.471 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 33554432, 2023-10-08T00:17:00.471 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:17:00.471 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:17:00.471 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:17:00.471 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:17:00.471 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:17:00.471 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:17:00.472 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:17:00.472 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:17:00.472 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:17:00.472 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:17:00.472 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:17:00.472 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:17:00.472 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:17:00.472 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:17:00.472 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:17:00.472 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:17:00.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.473 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:17:00.473 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:17:00.473 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:17:00.473 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:17:00.473 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:17:00.473 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:17:00.473 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:17:00.473 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:17:00.473 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:17:00.473 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:17:00.473 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:17:00.474 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:17:00.474 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2023-10-08T00:17:00.474 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2023-10-08T00:17:00.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.474 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:17:00.474 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:17:00.474 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2023-10-08T00:17:00.474 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:00.474 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:00.474 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:00.474 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:17:00.474 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2023-10-08T00:17:00.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.475 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:17:00.475 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:00.475 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:00.475 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:00.475 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:00.475 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:00.475 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:00.475 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:00.475 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:00.475 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:00.475 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:00.476 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:00.476 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:00.476 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:00.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.476 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:00.476 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2023-10-08T00:17:00.476 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:00.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.476 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:00.476 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2023-10-08T00:17:00.476 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:00.477 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:00.477 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:00.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.477 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:00.477 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:00.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.477 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:00.477 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2023-10-08T00:17:00.477 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:00.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.477 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:00.478 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2023-10-08T00:17:00.478 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:00.478 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:00.478 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:00.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.478 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:00.478 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:00.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.478 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:00.478 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2023-10-08T00:17:00.478 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:00.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.479 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:00.479 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2023-10-08T00:17:00.479 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:00.479 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:00.479 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:00.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.479 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:00.479 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:00.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.480 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:00.480 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2023-10-08T00:17:00.480 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:00.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.480 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:00.480 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2023-10-08T00:17:00.480 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:00.480 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:00.480 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:00.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.480 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/86365", 2023-10-08T00:17:00.481 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/86365", 2023-10-08T00:17:00.481 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/86365", 2023-10-08T00:17:00.481 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/86365", 2023-10-08T00:17:00.481 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:00.481 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:00.481 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:00.481 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:00.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.481 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:17:00.481 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:00.481 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:00.482 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:00.482 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:00.482 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:00.482 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:00.482 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:00.482 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:00.482 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:00.482 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:00.482 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:00.482 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:00.482 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:00.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.483 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:00.483 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2023-10-08T00:17:00.483 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:00.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.483 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:00.483 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2023-10-08T00:17:00.483 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:00.483 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:00.483 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:00.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.484 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:00.484 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:00.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.484 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:00.484 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2023-10-08T00:17:00.484 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:00.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.484 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:00.484 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2023-10-08T00:17:00.484 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:00.485 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:00.485 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:00.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.485 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:00.485 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:00.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.485 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:00.485 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2023-10-08T00:17:00.485 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:00.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.485 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:00.486 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2023-10-08T00:17:00.486 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:00.486 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:00.486 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:00.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.486 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:00.486 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:00.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.486 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:00.486 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2023-10-08T00:17:00.486 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:00.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2023-10-08T00:17:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/86366", 2023-10-08T00:17:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/86366", 2023-10-08T00:17:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/86366", 2023-10-08T00:17:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/86366", 2023-10-08T00:17:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:00.488 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:00.488 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:00.488 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:00.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.488 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:17:00.488 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:00.488 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:00.488 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:00.488 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:00.488 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:00.489 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:00.489 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:00.489 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:00.489 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:00.489 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:00.489 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:00.489 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:00.489 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:00.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.489 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:00.489 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2023-10-08T00:17:00.489 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:00.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.490 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:00.490 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2023-10-08T00:17:00.490 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:00.490 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:00.490 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:00.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.490 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:00.490 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:00.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.490 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:00.490 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2023-10-08T00:17:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2023-10-08T00:17:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.492 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:00.492 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2023-10-08T00:17:00.492 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:00.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.492 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:00.492 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2023-10-08T00:17:00.492 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:00.492 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:00.492 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:00.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.492 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:00.493 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:00.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.493 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:00.493 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2023-10-08T00:17:00.493 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:00.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.493 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:00.493 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2023-10-08T00:17:00.493 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:00.493 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:00.493 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:00.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6817/86382", 2023-10-08T00:17:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/86382", 2023-10-08T00:17:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/86382", 2023-10-08T00:17:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/86382", 2023-10-08T00:17:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:17:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:00.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.496 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:00.496 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2023-10-08T00:17:00.496 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:00.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.496 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:00.496 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2023-10-08T00:17:00.496 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:00.496 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:00.496 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2023-10-08T00:17:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2023-10-08T00:17:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2023-10-08T00:17:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:00.499 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2023-10-08T00:17:00.499 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:00.499 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:00.499 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:00.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.499 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:00.499 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:00.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.499 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:00.499 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2023-10-08T00:17:00.499 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2023-10-08T00:17:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6820/86393", 2023-10-08T00:17:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/86393", 2023-10-08T00:17:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/86393", 2023-10-08T00:17:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/86393", 2023-10-08T00:17:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:17:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6800", 2023-10-08T00:17:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6801", 2023-10-08T00:17:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6802", 2023-10-08T00:17:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6803", 2023-10-08T00:17:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6806", 2023-10-08T00:17:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6807", 2023-10-08T00:17:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:00.506 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:00.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.506 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:00.506 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6804", 2023-10-08T00:17:00.506 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:00.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.506 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:00.506 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6805", 2023-10-08T00:17:00.506 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:00.506 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:00.506 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6801/86570", 2023-10-08T00:17:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6803/86570", 2023-10-08T00:17:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6807/86570", 2023-10-08T00:17:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6805/86570", 2023-10-08T00:17:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:17:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6816", 2023-10-08T00:17:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6818", 2023-10-08T00:17:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6820", 2023-10-08T00:17:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6821", 2023-10-08T00:17:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6826", 2023-10-08T00:17:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6828", 2023-10-08T00:17:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6823", 2023-10-08T00:17:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:00.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.513 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:00.513 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6825", 2023-10-08T00:17:00.513 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:00.513 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:00.513 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:00.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.513 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6818/86608", 2023-10-08T00:17:00.513 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6821/86608", 2023-10-08T00:17:00.513 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6828/86608", 2023-10-08T00:17:00.514 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6825/86608", 2023-10-08T00:17:00.514 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:00.514 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:00.514 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:00.514 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:00.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.514 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:17:00.514 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:00.514 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:00.515 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:00.515 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:00.515 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:00.515 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:00.515 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:00.515 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:00.515 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:00.515 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:00.515 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:00.515 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:00.516 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:00.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.516 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:00.516 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6808", 2023-10-08T00:17:00.516 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:00.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.516 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:00.516 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6809", 2023-10-08T00:17:00.516 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6810", 2023-10-08T00:17:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6811", 2023-10-08T00:17:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6814", 2023-10-08T00:17:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6815", 2023-10-08T00:17:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6812", 2023-10-08T00:17:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6813", 2023-10-08T00:17:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6809/86571", 2023-10-08T00:17:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6811/86571", 2023-10-08T00:17:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6815/86571", 2023-10-08T00:17:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6813/86571", 2023-10-08T00:17:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:17:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6817", 2023-10-08T00:17:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6819", 2023-10-08T00:17:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6822", 2023-10-08T00:17:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6824", 2023-10-08T00:17:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6830", 2023-10-08T00:17:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6831", 2023-10-08T00:17:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:00.527 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:00.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.527 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:00.527 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:00.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.527 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:00.527 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6827", 2023-10-08T00:17:00.527 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:00.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.527 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:00.528 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6829", 2023-10-08T00:17:00.528 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:00.528 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:00.528 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:00.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.528 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6819/86612", 2023-10-08T00:17:00.528 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6824/86612", 2023-10-08T00:17:00.528 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6831/86612", 2023-10-08T00:17:00.528 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6829/86612", 2023-10-08T00:17:00.528 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:00.528 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:00.529 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:00.529 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:00.529 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:00.529 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:17:00.529 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2023-10-08T00:17:00.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.529 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:17:00.529 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:00.529 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:00.529 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:00.529 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:00.529 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:00.530 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.905615+0000", 2023-10-08T00:17:00.530 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:00.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.530 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:17:00.530 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:00.530 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:00.530 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:00.530 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:00.530 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:00.530 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.880709+0000", 2023-10-08T00:17:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:17:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.874704+0000", 2023-10-08T00:17:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:17:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.899604+0000", 2023-10-08T00:17:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:17:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.844371+0000", 2023-10-08T00:17:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:17:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.794810+0000", 2023-10-08T00:17:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:17:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.859474+0000", 2023-10-08T00:17:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:17:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.893130+0000", 2023-10-08T00:17:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:17:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2023-10-08T00:17:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [], 2023-10-08T00:17:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2023-10-08T00:17:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2023-10-08T00:17:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2023-10-08T00:17:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2023-10-08T00:17:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2023-10-08T00:17:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2023-10-08T00:17:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2023-10-08T00:17:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2023-10-08T00:17:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2023-10-08T00:17:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2023-10-08T00:17:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2023-10-08T00:17:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2023-10-08T00:17:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2023-10-08T00:17:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2023-10-08T00:17:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2023-10-08T00:17:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2023-10-08T00:17:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2023-10-08T00:17:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2023-10-08T00:17:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2023-10-08T00:17:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2023-10-08T00:17:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2023-10-08T00:17:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2023-10-08T00:17:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:00.539 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:17:00.540 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.453+0000 7f32d8029700 1 -- 172.21.15.99:0/3760053088 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f32b4066f30 msgr2=0x7f32b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:00.540 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.453+0000 7f32d8029700 1 --2- 172.21.15.99:0/3760053088 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f32b4066f30 0x7f32b40693e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f32cc009d10 tx=0x7f32cc0091a0 comp rx=0 tx=0).stop 2023-10-08T00:17:00.541 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.453+0000 7f32d8029700 1 -- 172.21.15.99:0/3760053088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32d00c8b10 msgr2=0x7f32d00c5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:00.541 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.453+0000 7f32d8029700 1 --2- 172.21.15.99:0/3760053088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32d00c8b10 0x7f32d00c5b10 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f32d006ba00 tx=0x7f32c801a000 comp rx=0 tx=0).stop 2023-10-08T00:17:00.541 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.453+0000 7f32d8029700 1 -- 172.21.15.99:0/3760053088 shutdown_connections 2023-10-08T00:17:00.541 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.453+0000 7f32d8029700 1 --2- 172.21.15.99:0/3760053088 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f32b4066f30 0x7f32b40693e0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:00.541 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.453+0000 7f32d8029700 1 --2- 172.21.15.99:0/3760053088 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32d00abae0 0x7f32d014de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:00.541 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.453+0000 7f32d8029700 1 --2- 172.21.15.99:0/3760053088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32d00c8b10 0x7f32d00c5b10 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:00.541 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.453+0000 7f32d8029700 1 --2- 172.21.15.99:0/3760053088 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f32d00abeb0 0x7f32d0156ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:00.541 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.453+0000 7f32d8029700 1 -- 172.21.15.99:0/3760053088 >> 172.21.15.99:0/3760053088 conn(0x7f32d0067660 msgr2=0x7f32d00c6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:00.541 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.453+0000 7f32d8029700 1 -- 172.21.15.99:0/3760053088 shutdown_connections 2023-10-08T00:17:00.542 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:00.453+0000 7f32d8029700 1 -- 172.21.15.99:0/3760053088 wait complete. 2023-10-08T00:17:00.542 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-08T00:17:05.543 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-08T00:17:05.667 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.666+0000 7f6bf32cd700 1 Processor -- start 2023-10-08T00:17:05.667 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.667+0000 7f6bf32cd700 1 -- start start 2023-10-08T00:17:05.670 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.667+0000 7f6bf32cd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6bec157870 0x7f6bec15b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:05.670 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.667+0000 7f6bf32cd700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f6bec15a620 0x7f6bec15d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:05.670 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.668+0000 7f6bf32cd700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bec15cbb0 con 0x7f6bec15d580 2023-10-08T00:17:05.670 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.668+0000 7f6bf32cd700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bec15bfa0 con 0x7f6bec15a620 2023-10-08T00:17:05.670 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.668+0000 7f6bf32cd700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bec15c280 con 0x7f6bec157870 2023-10-08T00:17:05.670 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.668+0000 7f6bf186a700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f6bec15d580 0x7f6bec158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:52276/0 (socket says 172.21.15.99:52276) 2023-10-08T00:17:05.670 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.668+0000 7f6bf186a700 1 -- 172.21.15.99:0/847652440 learned_addr learned my addr 172.21.15.99:0/847652440 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:17:05.670 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.668+0000 7f6bf0868700 1 --2- 172.21.15.99:0/847652440 >> v2:172.21.15.191:3300/0 conn(0x7f6bec15a620 0x7f6bec15d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:05.670 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.668+0000 7f6bf1069700 1 --2- 172.21.15.99:0/847652440 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6bec157870 0x7f6bec15b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:05.671 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.668+0000 7f6be3fff700 1 -- 172.21.15.99:0/847652440 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2380893331 0 0) 0x7f6bec15cbb0 con 0x7f6bec15d580 2023-10-08T00:17:05.671 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.668+0000 7f6be3fff700 1 -- 172.21.15.99:0/847652440 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bd8003570 con 0x7f6bec15d580 2023-10-08T00:17:05.671 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.668+0000 7f6bf0868700 1 -- 172.21.15.99:0/847652440 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6bec157870 msgr2=0x7f6bec15b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:05.671 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.668+0000 7f6bf0868700 1 --2- 172.21.15.99:0/847652440 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6bec157870 0x7f6bec15b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:05.671 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.668+0000 7f6bf0868700 1 -- 172.21.15.99:0/847652440 >> v1:172.21.15.99:6789/0 conn(0x7f6bec15d580 legacy=0x7f6bec158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:05.671 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.668+0000 7f6bf0868700 1 -- 172.21.15.99:0/847652440 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bec15c560 con 0x7f6bec15a620 2023-10-08T00:17:05.671 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.668+0000 7f6bf0868700 1 --2- 172.21.15.99:0/847652440 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6bec15a620 0x7f6bec15d040 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc0098e0 tx=0x7f6bdc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d87a1de3909b7ccd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:05.671 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.668+0000 7f6be3fff700 1 -- 172.21.15.99:0/847652440 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc011630 con 0x7f6bec15a620 2023-10-08T00:17:05.671 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.669+0000 7f6be3fff700 1 -- 172.21.15.99:0/847652440 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bdc011790 con 0x7f6bec15a620 2023-10-08T00:17:05.671 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.669+0000 7f6be3fff700 1 -- 172.21.15.99:0/847652440 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc011b30 con 0x7f6bec15a620 2023-10-08T00:17:05.671 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.669+0000 7f6bf32cd700 1 -- 172.21.15.99:0/847652440 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6bec15a620 msgr2=0x7f6bec15d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:05.671 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.669+0000 7f6bf32cd700 1 --2- 172.21.15.99:0/847652440 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6bec15a620 0x7f6bec15d040 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc0098e0 tx=0x7f6bdc0091a0 comp rx=0 tx=0).stop 2023-10-08T00:17:05.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.669+0000 7f6bf32cd700 1 -- 172.21.15.99:0/847652440 shutdown_connections 2023-10-08T00:17:05.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.669+0000 7f6bf32cd700 1 --2- 172.21.15.99:0/847652440 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6bec157870 0x7f6bec15b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:05.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.669+0000 7f6bf32cd700 1 --2- 172.21.15.99:0/847652440 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6bec15a620 0x7f6bec15d040 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:05.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.669+0000 7f6bf32cd700 1 -- 172.21.15.99:0/847652440 >> 172.21.15.99:0/847652440 conn(0x7f6bec068ce0 msgr2=0x7f6bec0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:05.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.669+0000 7f6bf32cd700 1 -- 172.21.15.99:0/847652440 shutdown_connections 2023-10-08T00:17:05.672 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.669+0000 7f6bf32cd700 1 -- 172.21.15.99:0/847652440 wait complete. 2023-10-08T00:17:05.674 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.670+0000 7f6bf32cd700 1 Processor -- start 2023-10-08T00:17:05.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.670+0000 7f6bf32cd700 1 -- start start 2023-10-08T00:17:05.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.671+0000 7f6bf32cd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6bec157870 0x7f6bec0b53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:05.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.671+0000 7f6bf32cd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6bec15a620 0x7f6bec14f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:05.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.671+0000 7f6bf32cd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6bec15d580 0x7f6bec0cfd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:05.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.671+0000 7f6bf32cd700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bec0b58e0 con 0x7f6bec157870 2023-10-08T00:17:05.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.671+0000 7f6bf32cd700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bec0b61c0 con 0x7f6bec15a620 2023-10-08T00:17:05.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.671+0000 7f6bf32cd700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bec0b1df0 con 0x7f6bec15d580 2023-10-08T00:17:05.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.671+0000 7f6bf1069700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6bec157870 0x7f6bec0b53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:05.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.671+0000 7f6bf1069700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6bec157870 0x7f6bec0b53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36562/0 (socket says 172.21.15.99:36562) 2023-10-08T00:17:05.677 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.671+0000 7f6bf1069700 1 -- 172.21.15.99:0/3653999782 learned_addr learned my addr 172.21.15.99:0/3653999782 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:05.677 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.671+0000 7f6bf186a700 1 --2- 172.21.15.99:0/3653999782 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6bec15d580 0x7f6bec0cfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:05.677 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.671+0000 7f6bf0868700 1 --2- 172.21.15.99:0/3653999782 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6bec15a620 0x7f6bec14f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:05.677 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.671+0000 7f6bf1069700 1 -- 172.21.15.99:0/3653999782 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6bec15d580 msgr2=0x7f6bec0cfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:05.677 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.671+0000 7f6bf1069700 1 --2- 172.21.15.99:0/3653999782 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6bec15d580 0x7f6bec0cfd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:05.678 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.671+0000 7f6bf1069700 1 -- 172.21.15.99:0/3653999782 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6bec15a620 msgr2=0x7f6bec14f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:05.678 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.671+0000 7f6bf1069700 1 --2- 172.21.15.99:0/3653999782 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6bec15a620 0x7f6bec14f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:05.678 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.671+0000 7f6bf1069700 1 -- 172.21.15.99:0/3653999782 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bdc009420 con 0x7f6bec157870 2023-10-08T00:17:05.678 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.672+0000 7f6bf1069700 1 --2- 172.21.15.99:0/3653999782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6bec157870 0x7f6bec0b53a0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f6be800ec80 tx=0x7f6be80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:05.678 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.672+0000 7f6be17fa700 1 -- 172.21.15.99:0/3653999782 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be800c9d0 con 0x7f6bec157870 2023-10-08T00:17:05.678 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.672+0000 7f6be17fa700 1 -- 172.21.15.99:0/3653999782 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6be800cb30 con 0x7f6bec157870 2023-10-08T00:17:05.678 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.672+0000 7f6bf32cd700 1 -- 172.21.15.99:0/3653999782 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bec0b20d0 con 0x7f6bec157870 2023-10-08T00:17:05.678 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.672+0000 7f6be17fa700 1 -- 172.21.15.99:0/3653999782 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be8013470 con 0x7f6bec157870 2023-10-08T00:17:05.678 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.672+0000 7f6bf32cd700 1 -- 172.21.15.99:0/3653999782 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bec0b2690 con 0x7f6bec157870 2023-10-08T00:17:05.678 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.673+0000 7f6bf32cd700 1 -- 172.21.15.99:0/3653999782 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bec04c3c0 con 0x7f6bec157870 2023-10-08T00:17:05.679 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.673+0000 7f6be17fa700 1 -- 172.21.15.99:0/3653999782 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6be8013820 con 0x7f6bec157870 2023-10-08T00:17:05.679 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.673+0000 7f6be17fa700 1 --2- 172.21.15.99:0/3653999782 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6bd0066f30 0x7f6bd00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:05.679 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.674+0000 7f6be17fa700 1 -- 172.21.15.99:0/3653999782 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6be8084920 con 0x7f6bec157870 2023-10-08T00:17:05.679 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.674+0000 7f6bf0868700 1 --2- 172.21.15.99:0/3653999782 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6bd0066f30 0x7f6bd00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:05.679 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.674+0000 7f6bf0868700 1 --2- 172.21.15.99:0/3653999782 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6bd0066f30 0x7f6bd00693e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc0091e0 tx=0x7f6bdc017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:05.679 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.678+0000 7f6be17fa700 1 -- 172.21.15.99:0/3653999782 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6be8051b90 con 0x7f6bec157870 2023-10-08T00:17:05.864 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.863+0000 7f6bf32cd700 1 -- 172.21.15.99:0/3653999782 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f6bec15c840 con 0x7f6bec157870 2023-10-08T00:17:05.865 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.864+0000 7f6be17fa700 1 -- 172.21.15.99:0/3653999782 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v140) v1 ==== 81+0+32282 (secure 0 0 0) 0x7f6be8017090 con 0x7f6bec157870 2023-10-08T00:17:05.865 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:17:05.866 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:17:05.866 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 140, 2023-10-08T00:17:05.866 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", 2023-10-08T00:17:05.866 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2023-10-08T00:10:36.303601+0000", 2023-10-08T00:17:05.866 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2023-10-08T00:16:59.766183+0000", 2023-10-08T00:17:05.866 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2023-10-08T00:10:41.709171+0000", 2023-10-08T00:17:05.866 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2023-10-08T00:10:38.690404+0000", 2023-10-08T00:17:05.866 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-08T00:17:05.866 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2023-10-08T00:17:05.866 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2023-10-08T00:17:05.867 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2023-10-08T00:17:05.867 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2023-10-08T00:17:05.867 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2023-10-08T00:17:05.867 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2023-10-08T00:17:05.867 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:17:05.867 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2023-10-08T00:17:05.867 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2023-10-08T00:17:05.867 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-08T00:17:05.867 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-08T00:17:05.867 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2023-10-08T00:17:05.867 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 17, 2023-10-08T00:17:05.868 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2023-10-08T00:17:05.868 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2023-10-08T00:17:05.868 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "jewel", 2023-10-08T00:17:05.868 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "quincy", 2023-10-08T00:17:05.868 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2023-10-08T00:17:05.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.868 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2023-10-08T00:17:05.868 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2023-10-08T00:17:05.868 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:10:42.063344+0000", 2023-10-08T00:17:05.868 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:17:05.868 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:17:05.869 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:17:05.869 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:17:05.869 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:17:05.869 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:17:05.869 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:17:05.869 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:17:05.869 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:17:05.869 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:17:05.869 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:17:05.869 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:17:05.869 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2023-10-08T00:17:05.869 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2023-10-08T00:17:05.870 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2023-10-08T00:17:05.870 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2023-10-08T00:17:05.870 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2023-10-08T00:17:05.870 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:17:05.870 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:17:05.870 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:17:05.870 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:17:05.870 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:17:05.870 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:17:05.870 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:17:05.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.871 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "112", 2023-10-08T00:17:05.871 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:17:05.871 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:17:05.871 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:17:05.871 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:17:05.871 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:17:05.871 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:17:05.871 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:17:05.871 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:17:05.871 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:17:05.871 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:17:05.871 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:17:05.872 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:17:05.872 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:17:05.872 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:17:05.872 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:17:05.872 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:17:05.872 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:17:05.872 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:17:05.872 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:17:05.872 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:17:05.872 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:17:05.872 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:17:05.872 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:17:05.873 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:17:05.873 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:17:05.873 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:17:05.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.873 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:17:05.873 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:17:05.873 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:17:05.873 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:17:05.873 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:17:05.873 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:17:05.873 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:17:05.873 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:17:05.874 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:17:05.874 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:17:05.874 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:17:05.874 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:17:05.874 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2023-10-08T00:17:05.874 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2023-10-08T00:17:05.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.874 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:17:05.874 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2023-10-08T00:17:05.874 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:05.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.875 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 16, 2023-10-08T00:17:05.875 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2023-10-08T00:17:05.875 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:16:48.430607+0000", 2023-10-08T00:17:05.875 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:17:05.875 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:17:05.875 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:17:05.875 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:17:05.875 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:17:05.875 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:17:05.875 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:17:05.876 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:17:05.876 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:17:05.876 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:17:05.876 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:17:05.876 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:17:05.876 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2023-10-08T00:17:05.876 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2023-10-08T00:17:05.876 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2023-10-08T00:17:05.876 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2023-10-08T00:17:05.876 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2023-10-08T00:17:05.876 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:17:05.877 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:17:05.877 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:17:05.877 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:17:05.877 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:17:05.877 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:17:05.877 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:17:05.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.877 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "138", 2023-10-08T00:17:05.877 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:17:05.877 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:17:05.877 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:17:05.878 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:17:05.878 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:17:05.878 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:17:05.878 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:17:05.878 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:17:05.878 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:17:05.878 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:17:05.878 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:17:05.878 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:17:05.878 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:17:05.878 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:17:05.878 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:17:05.879 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:17:05.879 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:17:05.879 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:17:05.879 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:17:05.879 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:17:05.879 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:17:05.879 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:17:05.879 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:17:05.879 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:17:05.879 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:17:05.879 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:17:05.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.880 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:17:05.880 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:17:05.880 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:17:05.880 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:17:05.880 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:17:05.880 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:17:05.880 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:17:05.880 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:17:05.880 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:17:05.880 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:17:05.880 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:17:05.880 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:17:05.881 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2023-10-08T00:17:05.881 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2023-10-08T00:17:05.881 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2023-10-08T00:17:05.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.881 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:17:05.881 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:17:05.881 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2023-10-08T00:17:05.881 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:05.881 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:05.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.881 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 17, 2023-10-08T00:17:05.882 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2023-10-08T00:17:05.882 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:16:49.063424+0000", 2023-10-08T00:17:05.882 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:17:05.882 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:17:05.882 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:17:05.882 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:17:05.882 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:17:05.882 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:17:05.882 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:17:05.882 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:17:05.882 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:17:05.882 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:17:05.883 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:17:05.883 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:17:05.883 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2023-10-08T00:17:05.883 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2023-10-08T00:17:05.883 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2023-10-08T00:17:05.883 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2023-10-08T00:17:05.883 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2023-10-08T00:17:05.883 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:17:05.883 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:17:05.883 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:17:05.883 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:17:05.884 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:17:05.884 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:17:05.884 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:17:05.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.884 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "140", 2023-10-08T00:17:05.884 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:17:05.884 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:17:05.884 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:17:05.884 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:17:05.884 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:17:05.884 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:17:05.885 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:17:05.885 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:17:05.885 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:17:05.885 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 33554432, 2023-10-08T00:17:05.885 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:17:05.885 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:17:05.885 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:17:05.885 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:17:05.885 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:17:05.885 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:17:05.885 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:17:05.885 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:17:05.886 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:17:05.886 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:17:05.886 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:17:05.886 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:17:05.886 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:17:05.886 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:17:05.886 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:17:05.886 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:17:05.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.886 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:17:05.886 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:17:05.887 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:17:05.887 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:17:05.887 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:17:05.887 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:17:05.887 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:17:05.887 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:17:05.887 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:17:05.887 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:17:05.887 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:17:05.887 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:17:05.887 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2023-10-08T00:17:05.887 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2023-10-08T00:17:05.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.888 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:17:05.888 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:17:05.888 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2023-10-08T00:17:05.888 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:05.888 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:05.888 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:05.888 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:17:05.888 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2023-10-08T00:17:05.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.888 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:17:05.888 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:05.889 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:05.889 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:05.889 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:05.889 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:05.889 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:05.889 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:05.889 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:05.889 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:05.889 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:05.889 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:05.889 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:05.889 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:05.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.890 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:05.890 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2023-10-08T00:17:05.890 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:05.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.890 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:05.890 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2023-10-08T00:17:05.890 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:05.890 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:05.890 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:05.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.891 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:05.891 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:05.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.891 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:05.891 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2023-10-08T00:17:05.891 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:05.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.891 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:05.891 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2023-10-08T00:17:05.891 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:05.891 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:05.892 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:05.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.892 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:05.892 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:05.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.892 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:05.892 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2023-10-08T00:17:05.892 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:05.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.892 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:05.892 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2023-10-08T00:17:05.893 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:05.893 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:05.893 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:05.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.893 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:05.893 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:05.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.893 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:05.893 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2023-10-08T00:17:05.893 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:05.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.894 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:05.894 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2023-10-08T00:17:05.894 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:05.894 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:05.894 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:05.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.895 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/86365", 2023-10-08T00:17:05.895 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/86365", 2023-10-08T00:17:05.895 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/86365", 2023-10-08T00:17:05.895 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/86365", 2023-10-08T00:17:05.895 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:05.895 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:05.896 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:05.896 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:05.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.896 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:17:05.896 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:05.896 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:05.896 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:05.896 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:05.896 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:05.897 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:05.897 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:05.897 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:05.897 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:05.897 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:05.897 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:05.897 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:05.897 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:05.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.897 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:05.897 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2023-10-08T00:17:05.898 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:05.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.898 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:05.898 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2023-10-08T00:17:05.898 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:05.898 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:05.898 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:05.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.898 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:05.899 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:05.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.899 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:05.899 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2023-10-08T00:17:05.899 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:05.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.899 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:05.899 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2023-10-08T00:17:05.899 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:05.899 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:05.900 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:05.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.901 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:05.901 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:05.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.901 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:05.901 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2023-10-08T00:17:05.902 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:05.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.902 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:05.902 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2023-10-08T00:17:05.902 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:05.902 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:05.902 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:05.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.902 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:05.902 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:05.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.903 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:05.903 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2023-10-08T00:17:05.903 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:05.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.903 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:05.903 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2023-10-08T00:17:05.903 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:05.903 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:05.903 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:05.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.904 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/86366", 2023-10-08T00:17:05.904 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/86366", 2023-10-08T00:17:05.904 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/86366", 2023-10-08T00:17:05.904 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/86366", 2023-10-08T00:17:05.904 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:05.904 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:05.904 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:05.904 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:05.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.905 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:17:05.905 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:05.905 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:05.905 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:05.905 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:05.905 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:05.905 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:05.905 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:05.905 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:05.905 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:05.905 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:05.906 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:05.906 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:05.906 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:05.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.906 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:05.906 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2023-10-08T00:17:05.906 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:05.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.906 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:05.906 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2023-10-08T00:17:05.907 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:05.907 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:05.907 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:05.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.907 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:05.907 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:05.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.907 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:05.907 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2023-10-08T00:17:05.907 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:05.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.908 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:05.908 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2023-10-08T00:17:05.908 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:05.908 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:05.908 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:05.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.908 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:05.908 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:05.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.909 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:05.909 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2023-10-08T00:17:05.909 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:05.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.909 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:05.909 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2023-10-08T00:17:05.909 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:05.909 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:05.909 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:05.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.909 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:05.910 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:05.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.910 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:05.910 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2023-10-08T00:17:05.910 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:05.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.910 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:05.910 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2023-10-08T00:17:05.910 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:05.910 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:05.911 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:05.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.911 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6817/86382", 2023-10-08T00:17:05.911 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/86382", 2023-10-08T00:17:05.911 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/86382", 2023-10-08T00:17:05.911 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/86382", 2023-10-08T00:17:05.911 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:05.911 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:05.911 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:05.911 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:05.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.912 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:17:05.912 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:05.912 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:05.912 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:05.912 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:05.912 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:05.912 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:05.912 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:05.912 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:05.913 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:05.913 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:05.913 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:05.913 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:05.913 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:05.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.913 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:05.913 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2023-10-08T00:17:05.913 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:05.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.914 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:05.914 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2023-10-08T00:17:05.914 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:05.914 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:05.914 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:05.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.914 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:05.914 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:05.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.914 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:05.914 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2023-10-08T00:17:05.915 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:05.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.915 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:05.915 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2023-10-08T00:17:05.915 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:05.915 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:05.915 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:05.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.915 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:05.915 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:05.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.916 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:05.916 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2023-10-08T00:17:05.916 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:05.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.916 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:05.916 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2023-10-08T00:17:05.916 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:05.916 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:05.916 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:05.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.917 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:05.917 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:05.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.917 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:05.917 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2023-10-08T00:17:05.917 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:05.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.917 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:05.917 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2023-10-08T00:17:05.917 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:05.918 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:05.918 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:05.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.918 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6820/86393", 2023-10-08T00:17:05.918 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/86393", 2023-10-08T00:17:05.918 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/86393", 2023-10-08T00:17:05.918 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/86393", 2023-10-08T00:17:05.918 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:05.918 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:05.918 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:05.918 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:05.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.919 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:17:05.919 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:05.919 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:05.919 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:05.919 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:05.919 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:05.919 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:05.919 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:05.919 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:05.919 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:05.920 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:05.920 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:05.920 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:05.920 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:05.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.920 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:05.920 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6800", 2023-10-08T00:17:05.920 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:05.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.921 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:05.921 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6801", 2023-10-08T00:17:05.921 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:05.921 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:05.921 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:05.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.921 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:05.921 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:05.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.921 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:05.921 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6802", 2023-10-08T00:17:05.922 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:05.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.922 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:05.922 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6803", 2023-10-08T00:17:05.922 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:05.922 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:05.922 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:05.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.923 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:05.923 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:05.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.923 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:05.923 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6806", 2023-10-08T00:17:05.923 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:05.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.923 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:05.923 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6807", 2023-10-08T00:17:05.923 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:05.924 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:05.924 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:05.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.924 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:05.924 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:05.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.924 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:05.924 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6804", 2023-10-08T00:17:05.924 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:05.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.925 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:05.925 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6805", 2023-10-08T00:17:05.925 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:05.925 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:05.925 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:05.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.925 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6801/86570", 2023-10-08T00:17:05.925 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6803/86570", 2023-10-08T00:17:05.925 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6807/86570", 2023-10-08T00:17:05.925 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6805/86570", 2023-10-08T00:17:05.926 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:05.926 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:05.926 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:05.926 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:05.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.926 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:17:05.926 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:05.926 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:05.926 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:05.926 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:05.927 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:05.927 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:05.927 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:05.927 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:05.927 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:05.927 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:05.927 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:05.927 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:05.927 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:05.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.927 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:05.928 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6816", 2023-10-08T00:17:05.928 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:05.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.928 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:05.928 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6818", 2023-10-08T00:17:05.928 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:05.928 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:05.928 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:05.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.928 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:05.929 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:05.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.929 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:05.929 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6820", 2023-10-08T00:17:05.929 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:05.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.929 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:05.929 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6821", 2023-10-08T00:17:05.929 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:05.929 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:05.930 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:05.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.930 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:05.930 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:05.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.930 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:05.930 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6826", 2023-10-08T00:17:05.930 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:05.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.930 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:05.931 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6828", 2023-10-08T00:17:05.931 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:05.931 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:05.931 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:05.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.931 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:05.931 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:05.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.931 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:05.931 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6823", 2023-10-08T00:17:05.931 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:05.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.932 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:05.932 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6825", 2023-10-08T00:17:05.932 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:05.932 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:05.932 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:05.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.932 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6818/86608", 2023-10-08T00:17:05.932 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6821/86608", 2023-10-08T00:17:05.932 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6828/86608", 2023-10-08T00:17:05.932 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6825/86608", 2023-10-08T00:17:05.933 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:05.933 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:05.933 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:05.933 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:05.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.933 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:17:05.933 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:05.933 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:05.933 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:05.933 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:05.933 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:05.934 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:05.934 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:05.934 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:05.934 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:05.934 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:05.934 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:05.934 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:05.934 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:05.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.934 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:05.934 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6808", 2023-10-08T00:17:05.935 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:05.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.935 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:05.935 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6809", 2023-10-08T00:17:05.935 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:05.935 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:05.935 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:05.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.935 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:05.935 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:05.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.936 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:05.936 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6810", 2023-10-08T00:17:05.936 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:05.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.936 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:05.936 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6811", 2023-10-08T00:17:05.936 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:05.936 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:05.936 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:05.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.937 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:05.937 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:05.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.937 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:05.937 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6814", 2023-10-08T00:17:05.937 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:05.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.937 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:05.937 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6815", 2023-10-08T00:17:05.937 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:05.937 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:05.938 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:05.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.938 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:05.938 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:05.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.938 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:05.938 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6812", 2023-10-08T00:17:05.938 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:05.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.938 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:05.938 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6813", 2023-10-08T00:17:05.939 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:05.939 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:05.939 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:05.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.939 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6809/86571", 2023-10-08T00:17:05.939 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6811/86571", 2023-10-08T00:17:05.939 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6815/86571", 2023-10-08T00:17:05.939 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6813/86571", 2023-10-08T00:17:05.939 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:05.939 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:05.939 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:05.939 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:05.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.940 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:17:05.940 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:05.940 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:05.940 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:05.940 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:05.940 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:05.940 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:05.940 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:05.940 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:05.941 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:05.941 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:05.941 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:05.941 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:05.941 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:05.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.941 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:05.941 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6817", 2023-10-08T00:17:05.941 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:05.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.941 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:05.942 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6819", 2023-10-08T00:17:05.942 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:05.942 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:05.942 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:05.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.942 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:05.942 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:05.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.942 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:05.942 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6822", 2023-10-08T00:17:05.942 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:05.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.943 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:05.943 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6824", 2023-10-08T00:17:05.943 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:05.943 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:05.943 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:05.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.943 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:05.943 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:05.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.943 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:05.944 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6830", 2023-10-08T00:17:05.944 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:05.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.944 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:05.944 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6831", 2023-10-08T00:17:05.944 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:05.944 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:05.944 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:05.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.944 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:05.945 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:05.945 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.945 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:05.945 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6827", 2023-10-08T00:17:05.945 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:05.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.945 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.945 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:05.945 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6829", 2023-10-08T00:17:05.945 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:05.945 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:05.945 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:05.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.946 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6819/86612", 2023-10-08T00:17:05.946 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6824/86612", 2023-10-08T00:17:05.946 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6831/86612", 2023-10-08T00:17:05.946 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6829/86612", 2023-10-08T00:17:05.946 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:05.946 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:05.946 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:05.946 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:05.946 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:05.946 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:17:05.946 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2023-10-08T00:17:05.947 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.947 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:17:05.947 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:05.947 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:05.947 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:05.947 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:05.947 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:05.947 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.905615+0000", 2023-10-08T00:17:05.947 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:05.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.947 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.948 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:17:05.948 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:05.948 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:05.948 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:05.948 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:05.948 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:05.948 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.880709+0000", 2023-10-08T00:17:05.948 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:05.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.948 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.948 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:17:05.948 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:05.949 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:05.949 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:05.949 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:05.949 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:05.949 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.874704+0000", 2023-10-08T00:17:05.949 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:05.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.949 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:17:05.949 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:05.949 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:05.949 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:05.950 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:05.950 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:05.950 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.899604+0000", 2023-10-08T00:17:05.950 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:05.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.950 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:17:05.950 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:05.950 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:05.950 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:05.950 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:05.951 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:05.951 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.844371+0000", 2023-10-08T00:17:05.951 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:05.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.951 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:17:05.951 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:05.951 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:05.951 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:05.951 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:05.951 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:05.951 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.794810+0000", 2023-10-08T00:17:05.952 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:05.952 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.952 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.952 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:17:05.952 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:05.952 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:05.952 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:05.952 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:05.952 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:05.952 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.859474+0000", 2023-10-08T00:17:05.952 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:05.953 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.953 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:05.953 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:17:05.953 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:05.953 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:05.953 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:05.953 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:05.953 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:05.953 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.893130+0000", 2023-10-08T00:17:05.953 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:05.953 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:05.953 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:17:05.954 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2023-10-08T00:17:05.954 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [], 2023-10-08T00:17:05.954 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2023-10-08T00:17:05.954 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2023-10-08T00:17:05.954 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2023-10-08T00:17:05.954 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2023-10-08T00:17:05.954 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2023-10-08T00:17:05.954 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2023-10-08T00:17:05.954 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2023-10-08T00:17:05.954 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2023-10-08T00:17:05.954 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2023-10-08T00:17:05.954 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2023-10-08T00:17:05.955 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2023-10-08T00:17:05.955 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:05.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:05.955 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2023-10-08T00:17:05.955 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2023-10-08T00:17:05.955 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2023-10-08T00:17:05.955 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2023-10-08T00:17:05.955 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2023-10-08T00:17:05.955 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2023-10-08T00:17:05.955 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2023-10-08T00:17:05.955 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2023-10-08T00:17:05.955 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2023-10-08T00:17:05.956 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2023-10-08T00:17:05.956 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2023-10-08T00:17:05.956 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:05.956 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:17:05.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.866+0000 7f6bf32cd700 1 -- 172.21.15.99:0/3653999782 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6bd0066f30 msgr2=0x7f6bd00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:05.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.866+0000 7f6bf32cd700 1 --2- 172.21.15.99:0/3653999782 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6bd0066f30 0x7f6bd00693e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc0091e0 tx=0x7f6bdc017000 comp rx=0 tx=0).stop 2023-10-08T00:17:05.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.866+0000 7f6bf32cd700 1 -- 172.21.15.99:0/3653999782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6bec157870 msgr2=0x7f6bec0b53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:05.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.866+0000 7f6bf32cd700 1 --2- 172.21.15.99:0/3653999782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6bec157870 0x7f6bec0b53a0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f6be800ec80 tx=0x7f6be80091a0 comp rx=0 tx=0).stop 2023-10-08T00:17:05.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.867+0000 7f6bf32cd700 1 -- 172.21.15.99:0/3653999782 shutdown_connections 2023-10-08T00:17:05.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.867+0000 7f6bf32cd700 1 --2- 172.21.15.99:0/3653999782 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6bd0066f30 0x7f6bd00693e0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:05.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.867+0000 7f6bf32cd700 1 --2- 172.21.15.99:0/3653999782 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6bec15d580 0x7f6bec0cfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:05.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.867+0000 7f6bf32cd700 1 --2- 172.21.15.99:0/3653999782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6bec157870 0x7f6bec0b53a0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:05.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.867+0000 7f6bf32cd700 1 --2- 172.21.15.99:0/3653999782 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6bec15a620 0x7f6bec14f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:05.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.867+0000 7f6bf32cd700 1 -- 172.21.15.99:0/3653999782 >> 172.21.15.99:0/3653999782 conn(0x7f6bec068ce0 msgr2=0x7f6bec15fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:05.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.867+0000 7f6bf32cd700 1 -- 172.21.15.99:0/3653999782 shutdown_connections 2023-10-08T00:17:05.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:05.867+0000 7f6bf32cd700 1 -- 172.21.15.99:0/3653999782 wait complete. 2023-10-08T00:17:05.959 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-08T00:17:10.960 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-08T00:17:11.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.086+0000 7f708fee1700 1 Processor -- start 2023-10-08T00:17:11.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.086+0000 7f708fee1700 1 -- start start 2023-10-08T00:17:11.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.087+0000 7f708fee1700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f708815a620 0x7f708815d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:11.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.087+0000 7f708fee1700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f708815d580 0x7f7088158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:11.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.087+0000 7f708fee1700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f708815cbb0 con 0x7f7088157870 2023-10-08T00:17:11.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.087+0000 7f708fee1700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f708815bdb0 con 0x7f708815a620 2023-10-08T00:17:11.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.087+0000 7f708fee1700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f708815bfa0 con 0x7f708815d580 2023-10-08T00:17:11.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.087+0000 7f708d47c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f708815a620 0x7f708815d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:11.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.087+0000 7f708d47c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f708815a620 0x7f708815d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36568/0 (socket says 172.21.15.99:36568) 2023-10-08T00:17:11.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.087+0000 7f708d47c700 1 -- 172.21.15.99:0/591727153 learned_addr learned my addr 172.21.15.99:0/591727153 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:11.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.087+0000 7f708e47e700 1 --2- 172.21.15.99:0/591727153 >> v2:172.21.15.191:3300/0 conn(0x7f708815d580 0x7f7088158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:11.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.087+0000 7f708d47c700 1 -- 172.21.15.99:0/591727153 >> v2:172.21.15.191:3300/0 conn(0x7f708815d580 msgr2=0x7f7088158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:11.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.087+0000 7f708d47c700 1 --2- 172.21.15.99:0/591727153 >> v2:172.21.15.191:3300/0 conn(0x7f708815d580 0x7f7088158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:11.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.087+0000 7f708d47c700 1 -- 172.21.15.99:0/591727153 >> v1:172.21.15.191:6789/0 conn(0x7f7088157870 legacy=0x7f708815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:11.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.087+0000 7f708d47c700 1 -- 172.21.15.99:0/591727153 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f708815c280 con 0x7f708815a620 2023-10-08T00:17:11.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.087+0000 7f708d47c700 1 --2- 172.21.15.99:0/591727153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f708815a620 0x7f708815d040 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f708806b4e0 tx=0x7f7078009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=209c63f17459db69 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:11.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.088+0000 7f708cc7b700 1 -- 172.21.15.99:0/591727153 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707800b820 con 0x7f708815a620 2023-10-08T00:17:11.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.088+0000 7f708cc7b700 1 -- 172.21.15.99:0/591727153 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f707800b980 con 0x7f708815a620 2023-10-08T00:17:11.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.088+0000 7f708cc7b700 1 -- 172.21.15.99:0/591727153 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707800bd20 con 0x7f708815a620 2023-10-08T00:17:11.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.088+0000 7f708fee1700 1 -- 172.21.15.99:0/591727153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f708815a620 msgr2=0x7f708815d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:11.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.088+0000 7f708fee1700 1 --2- 172.21.15.99:0/591727153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f708815a620 0x7f708815d040 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f708806b4e0 tx=0x7f7078009250 comp rx=0 tx=0).stop 2023-10-08T00:17:11.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.088+0000 7f708fee1700 1 -- 172.21.15.99:0/591727153 shutdown_connections 2023-10-08T00:17:11.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.088+0000 7f708fee1700 1 --2- 172.21.15.99:0/591727153 >> v2:172.21.15.191:3300/0 conn(0x7f708815d580 0x7f7088158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:11.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.088+0000 7f708fee1700 1 --2- 172.21.15.99:0/591727153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f708815a620 0x7f708815d040 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:11.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.088+0000 7f708fee1700 1 -- 172.21.15.99:0/591727153 >> 172.21.15.99:0/591727153 conn(0x7f7088068ce0 msgr2=0x7f70880690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:11.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.088+0000 7f708fee1700 1 -- 172.21.15.99:0/591727153 shutdown_connections 2023-10-08T00:17:11.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.088+0000 7f708fee1700 1 -- 172.21.15.99:0/591727153 wait complete. 2023-10-08T00:17:11.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.089+0000 7f708fee1700 1 Processor -- start 2023-10-08T00:17:11.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.089+0000 7f708fee1700 1 -- start start 2023-10-08T00:17:11.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.090+0000 7f708fee1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7088157870 0x7f708815f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:11.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.090+0000 7f708fee1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f708815a620 0x7f708814a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:11.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.090+0000 7f708fee1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f708815d580 0x7f708814b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:11.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.090+0000 7f708fee1700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708815ec30 con 0x7f7088157870 2023-10-08T00:17:11.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.090+0000 7f708fee1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708815f1f0 con 0x7f708815a620 2023-10-08T00:17:11.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.090+0000 7f708fee1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f708815fbc0 con 0x7f708815d580 2023-10-08T00:17:11.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.090+0000 7f708dc7d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7088157870 0x7f708815f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:11.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.090+0000 7f708dc7d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7088157870 0x7f708815f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36578/0 (socket says 172.21.15.99:36578) 2023-10-08T00:17:11.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.090+0000 7f708dc7d700 1 -- 172.21.15.99:0/678794056 learned_addr learned my addr 172.21.15.99:0/678794056 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:11.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.090+0000 7f708e47e700 1 --2- 172.21.15.99:0/678794056 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f708815d580 0x7f708814b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:11.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.090+0000 7f708d47c700 1 --2- 172.21.15.99:0/678794056 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f708815a620 0x7f708814a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:11.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.091+0000 7f708dc7d700 1 -- 172.21.15.99:0/678794056 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f708815d580 msgr2=0x7f708814b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:11.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.091+0000 7f708dc7d700 1 --2- 172.21.15.99:0/678794056 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f708815d580 0x7f708814b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:11.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.091+0000 7f708dc7d700 1 -- 172.21.15.99:0/678794056 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f708815a620 msgr2=0x7f708814a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:11.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.091+0000 7f708dc7d700 1 --2- 172.21.15.99:0/678794056 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f708815a620 0x7f708814a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:11.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.091+0000 7f708dc7d700 1 -- 172.21.15.99:0/678794056 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70780094f0 con 0x7f7088157870 2023-10-08T00:17:11.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.091+0000 7f708dc7d700 1 --2- 172.21.15.99:0/678794056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7088157870 0x7f708815f680 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f708400c600 tx=0x7f7084007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:11.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.091+0000 7f707dffb700 1 -- 172.21.15.99:0/678794056 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7084011590 con 0x7f7088157870 2023-10-08T00:17:11.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.091+0000 7f707dffb700 1 -- 172.21.15.99:0/678794056 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7084011710 con 0x7f7088157870 2023-10-08T00:17:11.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.091+0000 7f707dffb700 1 -- 172.21.15.99:0/678794056 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7084011ad0 con 0x7f7088157870 2023-10-08T00:17:11.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.091+0000 7f708fee1700 1 -- 172.21.15.99:0/678794056 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f708814bac0 con 0x7f7088157870 2023-10-08T00:17:11.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.091+0000 7f708fee1700 1 -- 172.21.15.99:0/678794056 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f708815ee80 con 0x7f7088157870 2023-10-08T00:17:11.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.092+0000 7f708fee1700 1 -- 172.21.15.99:0/678794056 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f708815f4d0 con 0x7f7088157870 2023-10-08T00:17:11.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.092+0000 7f707dffb700 1 -- 172.21.15.99:0/678794056 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f708401c460 con 0x7f7088157870 2023-10-08T00:17:11.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.093+0000 7f707dffb700 1 --2- 172.21.15.99:0/678794056 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f706c066f30 0x7f706c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:11.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.093+0000 7f707dffb700 1 -- 172.21.15.99:0/678794056 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7084084170 con 0x7f7088157870 2023-10-08T00:17:11.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.093+0000 7f707dffb700 1 -- 172.21.15.99:0/678794056 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=142}) v3 -- 0x7f706c06ea50 con 0x7f7088157870 2023-10-08T00:17:11.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.093+0000 7f708d47c700 1 --2- 172.21.15.99:0/678794056 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f706c066f30 0x7f706c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:11.097 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.094+0000 7f708d47c700 1 --2- 172.21.15.99:0/678794056 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f706c066f30 0x7f706c0693e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f708806b4e0 tx=0x7f7078016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:11.098 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.097+0000 7f707dffb700 1 -- 172.21.15.99:0/678794056 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f70840513e0 con 0x7f7088157870 2023-10-08T00:17:11.283 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.282+0000 7f708fee1700 1 -- 172.21.15.99:0/678794056 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f708804c3c0 con 0x7f7088157870 2023-10-08T00:17:11.284 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.283+0000 7f707dffb700 1 -- 172.21.15.99:0/678794056 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v141) v1 ==== 81+0+32301 (secure 0 0 0) 0x7f7084054a00 con 0x7f7088157870 2023-10-08T00:17:11.284 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:17:11.285 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:17:11.285 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 141, 2023-10-08T00:17:11.285 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", 2023-10-08T00:17:11.285 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2023-10-08T00:10:36.303601+0000", 2023-10-08T00:17:11.285 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2023-10-08T00:17:06.114261+0000", 2023-10-08T00:17:11.285 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2023-10-08T00:10:41.709171+0000", 2023-10-08T00:17:11.285 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2023-10-08T00:10:38.690404+0000", 2023-10-08T00:17:11.285 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-08T00:17:11.285 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2023-10-08T00:17:11.285 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2023-10-08T00:17:11.286 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2023-10-08T00:17:11.286 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2023-10-08T00:17:11.286 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2023-10-08T00:17:11.286 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2023-10-08T00:17:11.286 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:17:11.286 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2023-10-08T00:17:11.286 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2023-10-08T00:17:11.286 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-08T00:17:11.286 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-08T00:17:11.286 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2023-10-08T00:17:11.286 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 17, 2023-10-08T00:17:11.286 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2023-10-08T00:17:11.287 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2023-10-08T00:17:11.287 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "jewel", 2023-10-08T00:17:11.287 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "quincy", 2023-10-08T00:17:11.287 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2023-10-08T00:17:11.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.287 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2023-10-08T00:17:11.287 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2023-10-08T00:17:11.287 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:10:42.063344+0000", 2023-10-08T00:17:11.287 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:17:11.287 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:17:11.287 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:17:11.287 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:17:11.288 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:17:11.288 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:17:11.288 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:17:11.288 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:17:11.288 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:17:11.288 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:17:11.288 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:17:11.288 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:17:11.288 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2023-10-08T00:17:11.288 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2023-10-08T00:17:11.288 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2023-10-08T00:17:11.288 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2023-10-08T00:17:11.289 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2023-10-08T00:17:11.289 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:17:11.289 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:17:11.289 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:17:11.289 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:17:11.289 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:17:11.289 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:17:11.289 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:17:11.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.289 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "112", 2023-10-08T00:17:11.289 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:17:11.290 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:17:11.290 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:17:11.290 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:17:11.290 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:17:11.290 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:17:11.290 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:17:11.291 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:17:11.291 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:17:11.291 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:17:11.291 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:17:11.291 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:17:11.291 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:17:11.291 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:17:11.291 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:17:11.291 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:17:11.291 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:17:11.291 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:17:11.291 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:17:11.292 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:17:11.292 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:17:11.292 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:17:11.292 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:17:11.292 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:17:11.292 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:17:11.292 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:17:11.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.292 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:17:11.292 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:17:11.292 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:17:11.293 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:17:11.293 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:17:11.293 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:17:11.293 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:17:11.293 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:17:11.293 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:17:11.293 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:17:11.293 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:17:11.293 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:17:11.293 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2023-10-08T00:17:11.293 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2023-10-08T00:17:11.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.294 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:17:11.294 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2023-10-08T00:17:11.294 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.294 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 16, 2023-10-08T00:17:11.294 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2023-10-08T00:17:11.294 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:16:48.430607+0000", 2023-10-08T00:17:11.294 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:17:11.294 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:17:11.294 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:17:11.295 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:17:11.295 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:17:11.295 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:17:11.295 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:17:11.295 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:17:11.295 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:17:11.295 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:17:11.295 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:17:11.295 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:17:11.295 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2023-10-08T00:17:11.295 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2023-10-08T00:17:11.296 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2023-10-08T00:17:11.296 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2023-10-08T00:17:11.296 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2023-10-08T00:17:11.296 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:17:11.296 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:17:11.296 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:17:11.296 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:17:11.296 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:17:11.296 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:17:11.296 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:17:11.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.296 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "138", 2023-10-08T00:17:11.297 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:17:11.297 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:17:11.297 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:17:11.297 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:17:11.297 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:17:11.297 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:17:11.297 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:17:11.297 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:17:11.297 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:17:11.297 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:17:11.297 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:17:11.297 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:17:11.298 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:17:11.298 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:17:11.298 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:17:11.298 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:17:11.298 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:17:11.298 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:17:11.298 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:17:11.298 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:17:11.298 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:17:11.298 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:17:11.298 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:17:11.298 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:17:11.299 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:17:11.299 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:17:11.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.299 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:17:11.299 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:17:11.299 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:17:11.299 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:17:11.299 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:17:11.299 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:17:11.299 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:17:11.299 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:17:11.299 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:17:11.300 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:17:11.300 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:17:11.300 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:17:11.300 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2023-10-08T00:17:11.301 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2023-10-08T00:17:11.301 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2023-10-08T00:17:11.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.301 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:17:11.301 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:17:11.301 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2023-10-08T00:17:11.301 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.301 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.301 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 17, 2023-10-08T00:17:11.301 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2023-10-08T00:17:11.302 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:16:49.063424+0000", 2023-10-08T00:17:11.302 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1027, 2023-10-08T00:17:11.302 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full,full_quota", 2023-10-08T00:17:11.302 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:17:11.302 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:17:11.302 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:17:11.302 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:17:11.302 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:17:11.302 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:17:11.302 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:17:11.302 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:17:11.303 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:17:11.303 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:17:11.303 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2023-10-08T00:17:11.303 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2023-10-08T00:17:11.303 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2023-10-08T00:17:11.303 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2023-10-08T00:17:11.303 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2023-10-08T00:17:11.303 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:17:11.303 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:17:11.303 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:17:11.303 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:17:11.303 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:17:11.304 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:17:11.304 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:17:11.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.304 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "141", 2023-10-08T00:17:11.304 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:17:11.304 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:17:11.304 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:17:11.304 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:17:11.304 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:17:11.304 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:17:11.304 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:17:11.304 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:17:11.305 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:17:11.305 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 33554432, 2023-10-08T00:17:11.305 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:17:11.305 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:17:11.305 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:17:11.305 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:17:11.305 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:17:11.305 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:17:11.305 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:17:11.305 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:17:11.305 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:17:11.305 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:17:11.306 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:17:11.306 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:17:11.306 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:17:11.306 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:17:11.306 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:17:11.306 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:17:11.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.306 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:17:11.306 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:17:11.306 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:17:11.306 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:17:11.307 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:17:11.307 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:17:11.307 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:17:11.307 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:17:11.307 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:17:11.307 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:17:11.307 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:17:11.307 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:17:11.307 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2023-10-08T00:17:11.307 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2023-10-08T00:17:11.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.307 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:17:11.308 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:17:11.308 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2023-10-08T00:17:11.308 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.308 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.308 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.308 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:17:11.308 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2023-10-08T00:17:11.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.308 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:17:11.308 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:11.308 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:11.308 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:11.308 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:11.309 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:11.309 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:11.309 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:11.309 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:11.309 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:11.309 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:11.309 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:11.309 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:11.309 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:11.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.309 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:11.309 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2023-10-08T00:17:11.310 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:11.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.310 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:11.310 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2023-10-08T00:17:11.310 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:11.310 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.310 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.311 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:11.311 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:11.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.311 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:11.311 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2023-10-08T00:17:11.311 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:11.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.312 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:11.312 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2023-10-08T00:17:11.312 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:11.312 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.312 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.312 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:11.312 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:11.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.312 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:11.312 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2023-10-08T00:17:11.312 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:11.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.313 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:11.313 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2023-10-08T00:17:11.313 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:11.313 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.313 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.313 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:11.313 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:11.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.313 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:11.314 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2023-10-08T00:17:11.314 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:11.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.314 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:11.314 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2023-10-08T00:17:11.314 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:11.314 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.314 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.314 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/86365", 2023-10-08T00:17:11.314 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/86365", 2023-10-08T00:17:11.315 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/86365", 2023-10-08T00:17:11.315 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/86365", 2023-10-08T00:17:11.315 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:11.315 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:11.315 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:11.315 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.315 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:17:11.315 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:11.315 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:11.315 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:11.316 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:11.316 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:11.316 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:11.316 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:11.316 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:11.316 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:11.316 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:11.316 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:11.316 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:11.316 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:11.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.316 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:11.317 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2023-10-08T00:17:11.317 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:11.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.317 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:11.317 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2023-10-08T00:17:11.317 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:11.317 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.317 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.317 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:11.317 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:11.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.318 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:11.318 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2023-10-08T00:17:11.318 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:11.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.318 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:11.318 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2023-10-08T00:17:11.318 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:11.318 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.318 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.319 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:11.319 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:11.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.319 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:11.319 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2023-10-08T00:17:11.319 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:11.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.319 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:11.319 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2023-10-08T00:17:11.319 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:11.320 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.320 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.320 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:11.320 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:11.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.320 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:11.320 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2023-10-08T00:17:11.320 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:11.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.321 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:11.321 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2023-10-08T00:17:11.321 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:11.321 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.321 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.322 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/86366", 2023-10-08T00:17:11.322 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/86366", 2023-10-08T00:17:11.322 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/86366", 2023-10-08T00:17:11.322 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/86366", 2023-10-08T00:17:11.322 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:11.322 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:11.322 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:11.322 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.322 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:17:11.323 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:11.323 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:11.323 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:11.323 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:11.323 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:11.323 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:11.323 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:11.323 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:11.323 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:11.323 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:11.323 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:11.323 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:11.324 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:11.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.324 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:11.324 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2023-10-08T00:17:11.324 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:11.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.324 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:11.324 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2023-10-08T00:17:11.324 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:11.324 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.324 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.325 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:11.325 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:11.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.325 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:11.325 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2023-10-08T00:17:11.325 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:11.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.325 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:11.325 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2023-10-08T00:17:11.325 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:11.326 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.326 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.326 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:11.326 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:11.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.326 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:11.326 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2023-10-08T00:17:11.326 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:11.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.326 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:11.327 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2023-10-08T00:17:11.327 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:11.327 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.327 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.327 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:11.327 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:11.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.327 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:11.327 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2023-10-08T00:17:11.327 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:11.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.328 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:11.328 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2023-10-08T00:17:11.328 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:11.328 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.328 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.328 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6817/86382", 2023-10-08T00:17:11.328 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/86382", 2023-10-08T00:17:11.328 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/86382", 2023-10-08T00:17:11.328 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/86382", 2023-10-08T00:17:11.328 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:11.329 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:11.329 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:11.329 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.329 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:17:11.329 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:11.329 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:11.329 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:11.329 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:11.329 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:11.329 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:11.330 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:11.330 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:11.330 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:11.330 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:11.330 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:11.330 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:11.330 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:11.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.330 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:11.330 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2023-10-08T00:17:11.330 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:11.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.331 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:11.331 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2023-10-08T00:17:11.331 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:11.331 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.331 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.331 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:11.331 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:11.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.331 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:11.332 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2023-10-08T00:17:11.332 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:11.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.332 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:11.332 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2023-10-08T00:17:11.332 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:11.332 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.332 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.332 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:11.333 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:11.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.333 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:11.333 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2023-10-08T00:17:11.333 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:11.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.333 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:11.333 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2023-10-08T00:17:11.333 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:11.333 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.333 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.334 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:11.334 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:11.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.334 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:11.334 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2023-10-08T00:17:11.334 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:11.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.334 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:11.334 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2023-10-08T00:17:11.334 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:11.334 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.335 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.335 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6820/86393", 2023-10-08T00:17:11.335 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/86393", 2023-10-08T00:17:11.335 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/86393", 2023-10-08T00:17:11.335 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/86393", 2023-10-08T00:17:11.335 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:11.335 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:11.335 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:11.335 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.336 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:17:11.336 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:11.336 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:11.336 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:11.336 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:11.336 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:11.336 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:11.336 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:11.336 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:11.336 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:11.336 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:11.336 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:11.337 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:11.337 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:11.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.337 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:11.337 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6800", 2023-10-08T00:17:11.337 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:11.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.337 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:11.337 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6801", 2023-10-08T00:17:11.337 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:11.337 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.338 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.338 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:11.338 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:11.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.338 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:11.338 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6802", 2023-10-08T00:17:11.338 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:11.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.339 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:11.339 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6803", 2023-10-08T00:17:11.339 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:11.340 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.340 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.340 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:11.340 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:11.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.340 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:11.340 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6806", 2023-10-08T00:17:11.340 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:11.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.341 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:11.341 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6807", 2023-10-08T00:17:11.341 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:11.341 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.341 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.341 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:11.341 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:11.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.342 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:11.342 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6804", 2023-10-08T00:17:11.342 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:11.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.342 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:11.342 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6805", 2023-10-08T00:17:11.342 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:11.342 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.343 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.343 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6801/86570", 2023-10-08T00:17:11.343 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6803/86570", 2023-10-08T00:17:11.343 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6807/86570", 2023-10-08T00:17:11.343 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6805/86570", 2023-10-08T00:17:11.343 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:11.343 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:11.343 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:11.343 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.344 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:17:11.344 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:11.344 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:11.344 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:11.344 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:11.344 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:11.344 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:11.344 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:11.344 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:11.344 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:11.345 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:11.345 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:11.345 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:11.345 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:11.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.345 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:11.345 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6816", 2023-10-08T00:17:11.345 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:11.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.345 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:11.345 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6818", 2023-10-08T00:17:11.346 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:11.346 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.346 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.346 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:11.346 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:11.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.346 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:11.346 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6820", 2023-10-08T00:17:11.346 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:11.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.347 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:11.347 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6821", 2023-10-08T00:17:11.347 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:11.347 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.347 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.347 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:11.347 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:11.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.347 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:11.348 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6826", 2023-10-08T00:17:11.348 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:11.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.348 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:11.348 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6828", 2023-10-08T00:17:11.348 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:11.348 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.348 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.348 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:11.349 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:11.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.349 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:11.349 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6823", 2023-10-08T00:17:11.349 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:11.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.349 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:11.349 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6825", 2023-10-08T00:17:11.349 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:11.349 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.350 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.350 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6818/86608", 2023-10-08T00:17:11.350 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6821/86608", 2023-10-08T00:17:11.350 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6828/86608", 2023-10-08T00:17:11.350 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6825/86608", 2023-10-08T00:17:11.350 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:11.350 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:11.350 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:11.350 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.351 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:17:11.351 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:11.351 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:11.351 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:11.351 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:11.351 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:11.351 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:11.351 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:11.351 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:11.351 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:11.351 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:11.352 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:11.352 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:11.352 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:11.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.352 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:11.352 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6808", 2023-10-08T00:17:11.352 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:11.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.353 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:11.353 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6809", 2023-10-08T00:17:11.353 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:11.353 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.353 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.353 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:11.353 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:11.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.353 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:11.354 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6810", 2023-10-08T00:17:11.354 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:11.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.354 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:11.354 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6811", 2023-10-08T00:17:11.354 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:11.354 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.354 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.354 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:11.355 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:11.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.355 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:11.355 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6814", 2023-10-08T00:17:11.355 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:11.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.355 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:11.355 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6815", 2023-10-08T00:17:11.355 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:11.355 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.356 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.356 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:11.356 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:11.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.356 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:11.356 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6812", 2023-10-08T00:17:11.356 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:11.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.356 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:11.357 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6813", 2023-10-08T00:17:11.357 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:11.357 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.357 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.357 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6809/86571", 2023-10-08T00:17:11.357 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6811/86571", 2023-10-08T00:17:11.357 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6815/86571", 2023-10-08T00:17:11.357 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6813/86571", 2023-10-08T00:17:11.357 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:11.357 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:11.358 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:11.358 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.358 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:17:11.358 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:11.358 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:11.358 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:11.358 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:11.358 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:11.358 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:11.358 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:11.359 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:11.359 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:11.359 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:11.359 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:11.359 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:11.359 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:11.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.359 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:11.359 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6817", 2023-10-08T00:17:11.359 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:11.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.359 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:11.360 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6819", 2023-10-08T00:17:11.360 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:11.360 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.360 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.360 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:11.360 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:11.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.360 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:11.360 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6822", 2023-10-08T00:17:11.360 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:11.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.361 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:11.361 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6824", 2023-10-08T00:17:11.361 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:11.361 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.361 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.361 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:11.361 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:11.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.361 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:11.361 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6830", 2023-10-08T00:17:11.362 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:11.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.362 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:11.362 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6831", 2023-10-08T00:17:11.362 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:11.362 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.362 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.362 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:11.362 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:11.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.362 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:11.363 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6827", 2023-10-08T00:17:11.363 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:11.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.363 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:11.363 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6829", 2023-10-08T00:17:11.363 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:11.363 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.363 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.364 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6819/86612", 2023-10-08T00:17:11.364 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6824/86612", 2023-10-08T00:17:11.364 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6831/86612", 2023-10-08T00:17:11.364 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6829/86612", 2023-10-08T00:17:11.364 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:11.364 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:11.364 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:11.364 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.364 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.364 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:17:11.364 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2023-10-08T00:17:11.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.365 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:17:11.365 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:11.365 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:11.365 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:11.365 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:11.365 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:11.365 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.905615+0000", 2023-10-08T00:17:11.365 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:11.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.365 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:17:11.365 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:11.366 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:11.366 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:11.366 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:11.366 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:11.366 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.880709+0000", 2023-10-08T00:17:11.366 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:11.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.366 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:17:11.366 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:11.366 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:11.366 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:11.367 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:11.367 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:11.367 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.874704+0000", 2023-10-08T00:17:11.367 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:11.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.367 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:17:11.367 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:11.367 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:11.367 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:11.367 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:11.367 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:11.368 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.899604+0000", 2023-10-08T00:17:11.368 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:11.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.368 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:17:11.368 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:11.368 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:11.368 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:11.368 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:11.368 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:11.368 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.844371+0000", 2023-10-08T00:17:11.368 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:11.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.369 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.369 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:17:11.369 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:11.369 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:11.369 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:11.369 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:11.369 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:11.369 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.794810+0000", 2023-10-08T00:17:11.369 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:11.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.369 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.370 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:17:11.370 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:11.370 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:11.370 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:11.370 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:11.370 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:11.370 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.859474+0000", 2023-10-08T00:17:11.370 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:11.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.370 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:17:11.370 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:11.371 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:11.371 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:11.371 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:11.371 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:11.371 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.893130+0000", 2023-10-08T00:17:11.371 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:11.371 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.371 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:17:11.371 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2023-10-08T00:17:11.371 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [], 2023-10-08T00:17:11.371 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2023-10-08T00:17:11.371 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2023-10-08T00:17:11.372 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2023-10-08T00:17:11.372 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2023-10-08T00:17:11.372 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2023-10-08T00:17:11.372 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2023-10-08T00:17:11.372 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2023-10-08T00:17:11.372 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2023-10-08T00:17:11.372 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2023-10-08T00:17:11.372 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2023-10-08T00:17:11.372 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2023-10-08T00:17:11.372 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.372 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2023-10-08T00:17:11.372 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2023-10-08T00:17:11.373 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2023-10-08T00:17:11.373 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2023-10-08T00:17:11.373 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2023-10-08T00:17:11.373 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2023-10-08T00:17:11.373 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2023-10-08T00:17:11.373 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2023-10-08T00:17:11.373 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2023-10-08T00:17:11.373 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2023-10-08T00:17:11.373 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2023-10-08T00:17:11.373 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.373 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:17:11.375 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.285+0000 7f708fee1700 1 -- 172.21.15.99:0/678794056 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f706c066f30 msgr2=0x7f706c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:11.375 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.285+0000 7f708fee1700 1 --2- 172.21.15.99:0/678794056 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f706c066f30 0x7f706c0693e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f708806b4e0 tx=0x7f7078016000 comp rx=0 tx=0).stop 2023-10-08T00:17:11.375 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.285+0000 7f708fee1700 1 -- 172.21.15.99:0/678794056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7088157870 msgr2=0x7f708815f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:11.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.285+0000 7f708fee1700 1 --2- 172.21.15.99:0/678794056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7088157870 0x7f708815f680 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f708400c600 tx=0x7f7084007f60 comp rx=0 tx=0).stop 2023-10-08T00:17:11.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.286+0000 7f708fee1700 1 -- 172.21.15.99:0/678794056 shutdown_connections 2023-10-08T00:17:11.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.286+0000 7f708fee1700 1 --2- 172.21.15.99:0/678794056 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f706c066f30 0x7f706c0693e0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:11.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.286+0000 7f708fee1700 1 --2- 172.21.15.99:0/678794056 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f708815d580 0x7f708814b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:11.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.286+0000 7f708fee1700 1 --2- 172.21.15.99:0/678794056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7088157870 0x7f708815f680 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:11.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.286+0000 7f708fee1700 1 --2- 172.21.15.99:0/678794056 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f708815a620 0x7f708814a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:11.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.286+0000 7f708fee1700 1 -- 172.21.15.99:0/678794056 >> 172.21.15.99:0/678794056 conn(0x7f7088068ce0 msgr2=0x7f708814fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:11.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.286+0000 7f708fee1700 1 -- 172.21.15.99:0/678794056 shutdown_connections 2023-10-08T00:17:11.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.286+0000 7f708fee1700 1 -- 172.21.15.99:0/678794056 wait complete. 2023-10-08T00:17:11.377 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2023-10-08T00:17:11.377 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:17:11.377 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=16) 2023-10-08T00:17:11.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T00:17:11.620 INFO:teuthology.orchestra.run.smithi099.stderr:dd: fdatasync failed for 'large_file_b': No space left on device 2023-10-08T00:17:11.620 INFO:teuthology.orchestra.run.smithi099.stderr:50+0 records in 2023-10-08T00:17:11.620 INFO:teuthology.orchestra.run.smithi099.stderr:50+0 records out 2023-10-08T00:17:11.620 INFO:teuthology.orchestra.run.smithi099.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.223649 s, 234 MB/s 2023-10-08T00:17:11.620 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-08T00:17:11.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.733+0000 7f9d489be700 1 Processor -- start 2023-10-08T00:17:11.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.733+0000 7f9d489be700 1 -- start start 2023-10-08T00:17:11.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.734+0000 7f9d489be700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9d44157a90 0x7f9d440b1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:11.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.734+0000 7f9d489be700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9d4406fef0 0x7f9d440ce950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:11.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.734+0000 7f9d489be700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d440cb0b0 con 0x7f9d44157e60 2023-10-08T00:17:11.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.734+0000 7f9d489be700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d440b0ad0 con 0x7f9d4406fef0 2023-10-08T00:17:11.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.734+0000 7f9d489be700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d440b4970 con 0x7f9d44157a90 2023-10-08T00:17:11.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.734+0000 7f9d43fff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9d4406fef0 0x7f9d440ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:11.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.734+0000 7f9d42ffd700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f9d44157e60 0x7f9d440b0120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:52278/0 (socket says 172.21.15.99:52278) 2023-10-08T00:17:11.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.734+0000 7f9d42ffd700 1 -- 172.21.15.99:0/3756519538 learned_addr learned my addr 172.21.15.99:0/3756519538 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:17:11.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.734+0000 7f9d437fe700 1 --2- 172.21.15.99:0/3756519538 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9d44157a90 0x7f9d440b1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:11.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.734+0000 7f9d43fff700 1 -- 172.21.15.99:0/3756519538 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9d44157a90 msgr2=0x7f9d440b1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:11.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.734+0000 7f9d43fff700 1 --2- 172.21.15.99:0/3756519538 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9d44157a90 0x7f9d440b1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:11.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.734+0000 7f9d43fff700 1 -- 172.21.15.99:0/3756519538 >> v1:172.21.15.99:6789/0 conn(0x7f9d44157e60 legacy=0x7f9d440b0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:11.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.734+0000 7f9d43fff700 1 -- 172.21.15.99:0/3756519538 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d440b0830 con 0x7f9d4406fef0 2023-10-08T00:17:11.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.734+0000 7f9d427fc700 1 -- 172.21.15.99:0/3756519538 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1193358019 0 0) 0x7f9d440cb0b0 con 0x7f9d44157e60 2023-10-08T00:17:11.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.735+0000 7f9d43fff700 1 --2- 172.21.15.99:0/3756519538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d4406fef0 0x7f9d440ce950 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c0098e0 tx=0x7f9d3c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c3e633e16d20a31 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:11.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.735+0000 7f9d427fc700 1 -- 172.21.15.99:0/3756519538 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c009e80 con 0x7f9d4406fef0 2023-10-08T00:17:11.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.735+0000 7f9d427fc700 1 -- 172.21.15.99:0/3756519538 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d3c00b860 con 0x7f9d4406fef0 2023-10-08T00:17:11.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.735+0000 7f9d427fc700 1 -- 172.21.15.99:0/3756519538 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c00bbe0 con 0x7f9d4406fef0 2023-10-08T00:17:11.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.735+0000 7f9d489be700 1 -- 172.21.15.99:0/3756519538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d4406fef0 msgr2=0x7f9d440ce950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:11.739 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.735+0000 7f9d489be700 1 --2- 172.21.15.99:0/3756519538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d4406fef0 0x7f9d440ce950 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c0098e0 tx=0x7f9d3c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:17:11.739 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.735+0000 7f9d489be700 1 -- 172.21.15.99:0/3756519538 shutdown_connections 2023-10-08T00:17:11.739 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.735+0000 7f9d489be700 1 --2- 172.21.15.99:0/3756519538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d4406fef0 0x7f9d440ce950 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:11.739 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.735+0000 7f9d489be700 1 --2- 172.21.15.99:0/3756519538 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9d44157a90 0x7f9d440b1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:11.739 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.735+0000 7f9d489be700 1 -- 172.21.15.99:0/3756519538 >> 172.21.15.99:0/3756519538 conn(0x7f9d4406bc70 msgr2=0x7f9d4406c070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:11.739 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.735+0000 7f9d489be700 1 -- 172.21.15.99:0/3756519538 shutdown_connections 2023-10-08T00:17:11.739 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.735+0000 7f9d489be700 1 -- 172.21.15.99:0/3756519538 wait complete. 2023-10-08T00:17:11.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.737+0000 7f9d489be700 1 Processor -- start 2023-10-08T00:17:11.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.737+0000 7f9d489be700 1 -- start start 2023-10-08T00:17:11.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.737+0000 7f9d489be700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d4406fef0 0x7f9d441533e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:11.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.737+0000 7f9d489be700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9d44157a90 0x7f9d440c46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:11.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.737+0000 7f9d489be700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d44157e60 0x7f9d440c4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:11.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.737+0000 7f9d489be700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d44153920 con 0x7f9d44157e60 2023-10-08T00:17:11.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.737+0000 7f9d489be700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d440c33f0 con 0x7f9d4406fef0 2023-10-08T00:17:11.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.737+0000 7f9d489be700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d440c36d0 con 0x7f9d44157a90 2023-10-08T00:17:11.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.737+0000 7f9d43fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d44157e60 0x7f9d440c4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:11.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.737+0000 7f9d43fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d44157e60 0x7f9d440c4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36598/0 (socket says 172.21.15.99:36598) 2023-10-08T00:17:11.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.738+0000 7f9d43fff700 1 -- 172.21.15.99:0/3239151666 learned_addr learned my addr 172.21.15.99:0/3239151666 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:11.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.738+0000 7f9d437fe700 1 --2- 172.21.15.99:0/3239151666 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d4406fef0 0x7f9d441533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:11.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.738+0000 7f9d43fff700 1 -- 172.21.15.99:0/3239151666 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9d44157a90 msgr2=0x7f9d440c46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:11.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.738+0000 7f9d42ffd700 1 --2- 172.21.15.99:0/3239151666 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9d44157a90 0x7f9d440c46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:11.743 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.738+0000 7f9d43fff700 1 --2- 172.21.15.99:0/3239151666 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9d44157a90 0x7f9d440c46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:11.744 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.738+0000 7f9d43fff700 1 -- 172.21.15.99:0/3239151666 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d4406fef0 msgr2=0x7f9d441533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:11.744 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.738+0000 7f9d43fff700 1 --2- 172.21.15.99:0/3239151666 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d4406fef0 0x7f9d441533e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:11.744 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.738+0000 7f9d43fff700 1 -- 172.21.15.99:0/3239151666 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d3c009420 con 0x7f9d44157e60 2023-10-08T00:17:11.744 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.738+0000 7f9d437fe700 1 --2- 172.21.15.99:0/3239151666 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d4406fef0 0x7f9d441533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:17:11.744 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.738+0000 7f9d42ffd700 1 --2- 172.21.15.99:0/3239151666 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9d44157a90 0x7f9d440c46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:17:11.744 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.738+0000 7f9d43fff700 1 --2- 172.21.15.99:0/3239151666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d44157e60 0x7f9d440c4c30 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c00b3c0 tx=0x7f9d3c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:11.744 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.738+0000 7f9d2b7fe700 1 -- 172.21.15.99:0/3239151666 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c011630 con 0x7f9d44157e60 2023-10-08T00:17:11.744 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.738+0000 7f9d2b7fe700 1 -- 172.21.15.99:0/3239151666 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d3c0117b0 con 0x7f9d44157e60 2023-10-08T00:17:11.744 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.738+0000 7f9d489be700 1 -- 172.21.15.99:0/3239151666 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d440c39b0 con 0x7f9d44157e60 2023-10-08T00:17:11.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.738+0000 7f9d2b7fe700 1 -- 172.21.15.99:0/3239151666 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c011b70 con 0x7f9d44157e60 2023-10-08T00:17:11.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.738+0000 7f9d489be700 1 -- 172.21.15.99:0/3239151666 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d441679f0 con 0x7f9d44157e60 2023-10-08T00:17:11.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.739+0000 7f9d489be700 1 -- 172.21.15.99:0/3239151666 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d44167c60 con 0x7f9d44157e60 2023-10-08T00:17:11.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.740+0000 7f9d2b7fe700 1 -- 172.21.15.99:0/3239151666 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9d3c02c3b0 con 0x7f9d44157e60 2023-10-08T00:17:11.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.740+0000 7f9d2b7fe700 1 --2- 172.21.15.99:0/3239151666 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9d30066f30 0x7f9d300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:11.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.740+0000 7f9d2b7fe700 1 -- 172.21.15.99:0/3239151666 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9d3c085c70 con 0x7f9d44157e60 2023-10-08T00:17:11.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.740+0000 7f9d437fe700 1 --2- 172.21.15.99:0/3239151666 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9d30066f30 0x7f9d300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:11.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.740+0000 7f9d2b7fe700 1 -- 172.21.15.99:0/3239151666 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=142}) v3 -- 0x7f9d3006ea50 con 0x7f9d44157e60 2023-10-08T00:17:11.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.741+0000 7f9d437fe700 1 --2- 172.21.15.99:0/3239151666 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9d30066f30 0x7f9d300693e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f9d34007540 tx=0x7f9d3400b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:11.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.744+0000 7f9d2b7fe700 1 -- 172.21.15.99:0/3239151666 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9d3c052fa0 con 0x7f9d44157e60 2023-10-08T00:17:11.930 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.929+0000 7f9d489be700 1 -- 172.21.15.99:0/3239151666 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f9d440ca780 con 0x7f9d44157e60 2023-10-08T00:17:11.931 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.930+0000 7f9d2b7fe700 1 -- 172.21.15.99:0/3239151666 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v141) v1 ==== 81+0+32301 (secure 0 0 0) 0x7f9d3c028090 con 0x7f9d44157e60 2023-10-08T00:17:11.931 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:17:11.931 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:17:11.932 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 141, 2023-10-08T00:17:11.932 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", 2023-10-08T00:17:11.932 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2023-10-08T00:10:36.303601+0000", 2023-10-08T00:17:11.932 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2023-10-08T00:17:06.114261+0000", 2023-10-08T00:17:11.932 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2023-10-08T00:10:41.709171+0000", 2023-10-08T00:17:11.932 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2023-10-08T00:10:38.690404+0000", 2023-10-08T00:17:11.932 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-08T00:17:11.932 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2023-10-08T00:17:11.932 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2023-10-08T00:17:11.932 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2023-10-08T00:17:11.932 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2023-10-08T00:17:11.932 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2023-10-08T00:17:11.933 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2023-10-08T00:17:11.933 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:17:11.933 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2023-10-08T00:17:11.933 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2023-10-08T00:17:11.933 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-08T00:17:11.933 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-08T00:17:11.933 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2023-10-08T00:17:11.933 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 17, 2023-10-08T00:17:11.933 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2023-10-08T00:17:11.933 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2023-10-08T00:17:11.933 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "jewel", 2023-10-08T00:17:11.934 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "quincy", 2023-10-08T00:17:11.934 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2023-10-08T00:17:11.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.934 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2023-10-08T00:17:11.934 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2023-10-08T00:17:11.934 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:10:42.063344+0000", 2023-10-08T00:17:11.934 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:17:11.934 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:17:11.934 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:17:11.934 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:17:11.934 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:17:11.935 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:17:11.935 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:17:11.935 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:17:11.935 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:17:11.935 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:17:11.935 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:17:11.935 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:17:11.935 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2023-10-08T00:17:11.936 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2023-10-08T00:17:11.936 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2023-10-08T00:17:11.936 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2023-10-08T00:17:11.936 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2023-10-08T00:17:11.936 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:17:11.936 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:17:11.936 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:17:11.936 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:17:11.936 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:17:11.936 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:17:11.936 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:17:11.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.937 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "112", 2023-10-08T00:17:11.937 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:17:11.937 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:17:11.937 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:17:11.937 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:17:11.937 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:17:11.937 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:17:11.937 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:17:11.937 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:17:11.937 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:17:11.937 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:17:11.937 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:17:11.938 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:17:11.938 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:17:11.938 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:17:11.938 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:17:11.938 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:17:11.938 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:17:11.938 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:17:11.938 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:17:11.938 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:17:11.938 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:17:11.938 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:17:11.939 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:17:11.939 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:17:11.939 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:17:11.939 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:17:11.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.939 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:17:11.939 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:17:11.939 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:17:11.939 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:17:11.939 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:17:11.939 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:17:11.940 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:17:11.940 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:17:11.940 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:17:11.940 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:17:11.940 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:17:11.940 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:17:11.940 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2023-10-08T00:17:11.940 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2023-10-08T00:17:11.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.941 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:17:11.941 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2023-10-08T00:17:11.941 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.941 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 16, 2023-10-08T00:17:11.941 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2023-10-08T00:17:11.941 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:16:48.430607+0000", 2023-10-08T00:17:11.942 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:17:11.942 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:17:11.942 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:17:11.942 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:17:11.943 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:17:11.943 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:17:11.943 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:17:11.943 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:17:11.943 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:17:11.943 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:17:11.943 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:17:11.943 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:17:11.944 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2023-10-08T00:17:11.944 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2023-10-08T00:17:11.944 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2023-10-08T00:17:11.944 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2023-10-08T00:17:11.944 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2023-10-08T00:17:11.944 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:17:11.944 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:17:11.944 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:17:11.944 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:17:11.944 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:17:11.944 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:17:11.944 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:17:11.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.945 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "138", 2023-10-08T00:17:11.945 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:17:11.945 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:17:11.945 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:17:11.945 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:17:11.945 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:17:11.945 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:17:11.946 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:17:11.946 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:17:11.946 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:17:11.946 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:17:11.946 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:17:11.946 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:17:11.946 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:17:11.946 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:17:11.946 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:17:11.946 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:17:11.947 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:17:11.947 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:17:11.947 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:17:11.947 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:17:11.947 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:17:11.947 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:17:11.947 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:17:11.947 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:17:11.947 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:17:11.947 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:17:11.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.947 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:17:11.948 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:17:11.948 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:17:11.948 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:17:11.948 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:17:11.948 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:17:11.948 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:17:11.948 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:17:11.948 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:17:11.948 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:17:11.948 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:17:11.948 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:17:11.948 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2023-10-08T00:17:11.949 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2023-10-08T00:17:11.949 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2023-10-08T00:17:11.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.949 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:17:11.949 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:17:11.949 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2023-10-08T00:17:11.949 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.949 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.950 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 17, 2023-10-08T00:17:11.950 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2023-10-08T00:17:11.950 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:16:49.063424+0000", 2023-10-08T00:17:11.950 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1027, 2023-10-08T00:17:11.950 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full,full_quota", 2023-10-08T00:17:11.950 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:17:11.950 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:17:11.950 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:17:11.950 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:17:11.950 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:17:11.951 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:17:11.951 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:17:11.951 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:17:11.951 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:17:11.951 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:17:11.951 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2023-10-08T00:17:11.951 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2023-10-08T00:17:11.951 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2023-10-08T00:17:11.952 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2023-10-08T00:17:11.952 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2023-10-08T00:17:11.952 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:17:11.952 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:17:11.952 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:17:11.952 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:17:11.952 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:17:11.952 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:17:11.953 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:17:11.953 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.953 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "141", 2023-10-08T00:17:11.953 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:17:11.953 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:17:11.953 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:17:11.953 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:17:11.954 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:17:11.954 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:17:11.954 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:17:11.954 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:17:11.954 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:17:11.954 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 33554432, 2023-10-08T00:17:11.954 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:17:11.954 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:17:11.955 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:17:11.955 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:17:11.955 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:17:11.955 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:17:11.955 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:17:11.955 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:17:11.956 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:17:11.956 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:17:11.956 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:17:11.956 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:17:11.956 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:17:11.956 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:17:11.957 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:17:11.957 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:17:11.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.957 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:17:11.957 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:17:11.957 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:17:11.958 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:17:11.958 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:17:11.958 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:17:11.958 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:17:11.958 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:17:11.958 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:17:11.959 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:17:11.959 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:17:11.959 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:17:11.959 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2023-10-08T00:17:11.959 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2023-10-08T00:17:11.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.959 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:17:11.960 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:17:11.960 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2023-10-08T00:17:11.960 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.960 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.960 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.960 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:17:11.961 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2023-10-08T00:17:11.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.961 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:17:11.961 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:11.961 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:11.961 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:11.962 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:11.962 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:11.962 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:11.962 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:11.962 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:11.962 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:11.963 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:11.963 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:11.963 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:11.963 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:11.963 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.963 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:11.963 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2023-10-08T00:17:11.964 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:11.964 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.964 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.964 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:11.964 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2023-10-08T00:17:11.964 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:11.965 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.965 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.965 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:11.965 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:11.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.966 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:11.966 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2023-10-08T00:17:11.966 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:11.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.966 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:11.967 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2023-10-08T00:17:11.967 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:11.967 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.967 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.967 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:11.968 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:11.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.968 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:11.968 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2023-10-08T00:17:11.968 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:11.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.969 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.969 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:11.969 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2023-10-08T00:17:11.969 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:11.969 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.969 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.970 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.970 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:11.970 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:11.970 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.970 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:11.970 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2023-10-08T00:17:11.971 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:11.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.971 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.971 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:11.971 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2023-10-08T00:17:11.971 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:11.972 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.972 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.972 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/86365", 2023-10-08T00:17:11.972 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/86365", 2023-10-08T00:17:11.972 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/86365", 2023-10-08T00:17:11.973 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/86365", 2023-10-08T00:17:11.973 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:11.973 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:11.973 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:11.973 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.974 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.974 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:17:11.974 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:11.974 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:11.974 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:11.974 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:11.975 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:11.975 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:11.975 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:11.975 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:11.975 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:11.975 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:11.976 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:11.976 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:11.976 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:11.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.976 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:11.976 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2023-10-08T00:17:11.976 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:11.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.977 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:11.977 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2023-10-08T00:17:11.977 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:11.977 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.978 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.978 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:11.978 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:11.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.978 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:11.979 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2023-10-08T00:17:11.979 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:11.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.979 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:11.979 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2023-10-08T00:17:11.980 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:11.980 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.980 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.980 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:11.980 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:11.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.981 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:11.981 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2023-10-08T00:17:11.981 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:11.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.981 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:11.982 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2023-10-08T00:17:11.982 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:11.982 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.982 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.983 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:11.983 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:11.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.983 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:11.983 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2023-10-08T00:17:11.983 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:11.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.984 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:11.984 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2023-10-08T00:17:11.984 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:11.984 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.985 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.985 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/86366", 2023-10-08T00:17:11.985 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/86366", 2023-10-08T00:17:11.985 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/86366", 2023-10-08T00:17:11.985 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/86366", 2023-10-08T00:17:11.985 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:11.985 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:11.986 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:11.986 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.986 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:17:11.986 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:11.986 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:11.987 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:11.987 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:11.987 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:11.987 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:11.987 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:11.987 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:11.987 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:11.987 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:11.987 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:11.988 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:11.988 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:11.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.988 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:11.988 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2023-10-08T00:17:11.988 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:11.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.988 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:11.989 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2023-10-08T00:17:11.989 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:11.989 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.989 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.989 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:11.989 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:11.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.990 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:11.990 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2023-10-08T00:17:11.990 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:11.990 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.990 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.990 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:11.991 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2023-10-08T00:17:11.991 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:11.991 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.991 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.991 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:11.991 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:11.991 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.992 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:11.992 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2023-10-08T00:17:11.992 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:11.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.992 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.992 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:11.992 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2023-10-08T00:17:11.993 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:11.993 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.993 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.993 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:11.993 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:11.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.993 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:11.994 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2023-10-08T00:17:11.994 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:11.994 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.994 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:11.994 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2023-10-08T00:17:11.994 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:11.995 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.995 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.995 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6817/86382", 2023-10-08T00:17:11.995 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/86382", 2023-10-08T00:17:11.995 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/86382", 2023-10-08T00:17:11.995 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/86382", 2023-10-08T00:17:11.995 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:11.996 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:11.996 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:11.996 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.996 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.996 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:17:11.996 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:11.996 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:11.997 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:11.997 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:11.997 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:11.997 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:11.997 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:11.997 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:11.997 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:11.997 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:11.997 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:11.998 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:11.998 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:11.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.998 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:11.998 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2023-10-08T00:17:11.998 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:11.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.999 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:11.999 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2023-10-08T00:17:11.999 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:11.999 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:11.999 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:11.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:11.999 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:11.999 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:11.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:11.999 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:12.000 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2023-10-08T00:17:12.000 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:12.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.000 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.000 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:12.000 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2023-10-08T00:17:12.000 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:12.000 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:12.000 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:12.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.000 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:12.001 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:12.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.001 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:12.001 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2023-10-08T00:17:12.001 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:12.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.001 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:12.001 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2023-10-08T00:17:12.001 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:12.002 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:12.002 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:12.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.002 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:12.002 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:12.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.002 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:12.002 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2023-10-08T00:17:12.002 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:12.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.002 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:12.003 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2023-10-08T00:17:12.003 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:12.003 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:12.003 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:12.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.003 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6820/86393", 2023-10-08T00:17:12.003 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/86393", 2023-10-08T00:17:12.003 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/86393", 2023-10-08T00:17:12.003 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/86393", 2023-10-08T00:17:12.003 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:12.004 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:12.004 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:12.004 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:12.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.004 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:17:12.004 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:12.004 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:12.004 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:12.004 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:12.004 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:12.005 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:12.005 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:12.005 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:12.005 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:12.005 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:12.005 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:12.005 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:12.005 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:12.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.005 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:12.005 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6800", 2023-10-08T00:17:12.006 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:12.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.006 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.006 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:12.006 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6801", 2023-10-08T00:17:12.006 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:12.006 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:12.006 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:12.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.006 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:12.007 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:12.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.007 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:12.007 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6802", 2023-10-08T00:17:12.007 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:12.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.007 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:12.007 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6803", 2023-10-08T00:17:12.008 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:12.008 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:12.008 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:12.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.008 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:12.008 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:12.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.008 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:12.008 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6806", 2023-10-08T00:17:12.008 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:12.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.009 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:12.009 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6807", 2023-10-08T00:17:12.009 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:12.009 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:12.009 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:12.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.009 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:12.009 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:12.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.010 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:12.010 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6804", 2023-10-08T00:17:12.010 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:12.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.010 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:12.010 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6805", 2023-10-08T00:17:12.010 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:12.010 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:12.010 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:12.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.011 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6801/86570", 2023-10-08T00:17:12.011 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6803/86570", 2023-10-08T00:17:12.011 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6807/86570", 2023-10-08T00:17:12.011 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6805/86570", 2023-10-08T00:17:12.011 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:12.011 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:12.011 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:12.011 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:12.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.012 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:17:12.012 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:12.012 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:12.012 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:12.012 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:12.012 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:12.012 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:12.012 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:12.013 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:12.013 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:12.013 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:12.013 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:12.013 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:12.013 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:12.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.013 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:12.013 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6816", 2023-10-08T00:17:12.013 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:12.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.014 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:12.014 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6818", 2023-10-08T00:17:12.014 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:12.014 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:12.014 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:12.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.014 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:12.014 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:12.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.015 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:12.015 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6820", 2023-10-08T00:17:12.015 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:12.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.015 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:12.015 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6821", 2023-10-08T00:17:12.015 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:12.015 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:12.016 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:12.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.016 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:12.016 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:12.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.016 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:12.016 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6826", 2023-10-08T00:17:12.016 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:12.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.017 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:12.017 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6828", 2023-10-08T00:17:12.017 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:12.017 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:12.017 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:12.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.017 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:12.017 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:12.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.017 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:12.018 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6823", 2023-10-08T00:17:12.018 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:12.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.018 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:12.018 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6825", 2023-10-08T00:17:12.018 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:12.018 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:12.018 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:12.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.019 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6818/86608", 2023-10-08T00:17:12.019 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6821/86608", 2023-10-08T00:17:12.019 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6828/86608", 2023-10-08T00:17:12.019 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6825/86608", 2023-10-08T00:17:12.019 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:12.019 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:12.019 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:12.019 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:12.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.020 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:17:12.020 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:12.020 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:12.020 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:12.020 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:12.020 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:12.020 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:12.020 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:12.021 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:12.021 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:12.021 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:12.021 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:12.021 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:12.021 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:12.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.021 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:12.021 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6808", 2023-10-08T00:17:12.021 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:12.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.022 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:12.022 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6809", 2023-10-08T00:17:12.022 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:12.022 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:12.022 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:12.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.022 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:12.022 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:12.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.022 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:12.022 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6810", 2023-10-08T00:17:12.023 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:12.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.023 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:12.023 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6811", 2023-10-08T00:17:12.023 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:12.023 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:12.023 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:12.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.023 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:12.023 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:12.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.024 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:12.024 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6814", 2023-10-08T00:17:12.024 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:12.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.024 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:12.024 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6815", 2023-10-08T00:17:12.024 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:12.024 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:12.024 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:12.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.024 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:12.025 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:12.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.025 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:12.025 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6812", 2023-10-08T00:17:12.025 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:12.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.025 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:12.025 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6813", 2023-10-08T00:17:12.025 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:12.025 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:12.025 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:12.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.026 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6809/86571", 2023-10-08T00:17:12.026 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6811/86571", 2023-10-08T00:17:12.026 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6815/86571", 2023-10-08T00:17:12.026 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6813/86571", 2023-10-08T00:17:12.026 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:12.026 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:12.026 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:12.026 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:12.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.026 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:17:12.027 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:12.027 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:12.027 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:12.027 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:12.027 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:12.027 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:12.027 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:12.027 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:12.027 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:12.027 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:12.027 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:12.027 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:12.028 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:12.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.028 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:12.028 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6817", 2023-10-08T00:17:12.028 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:12.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.028 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:12.028 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6819", 2023-10-08T00:17:12.028 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:12.028 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:12.028 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:12.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.029 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:12.029 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:12.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.029 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:12.029 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6822", 2023-10-08T00:17:12.029 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:12.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.029 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:12.029 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6824", 2023-10-08T00:17:12.029 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:12.030 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:12.030 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:12.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.030 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:12.030 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:12.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.030 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:12.030 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6830", 2023-10-08T00:17:12.030 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:12.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.031 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:12.031 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6831", 2023-10-08T00:17:12.031 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:12.031 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:12.031 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:12.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.031 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:12.031 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:12.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.031 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:12.031 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6827", 2023-10-08T00:17:12.032 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:12.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.032 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:12.032 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6829", 2023-10-08T00:17:12.032 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:12.032 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:12.032 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:12.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.032 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6819/86612", 2023-10-08T00:17:12.032 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6824/86612", 2023-10-08T00:17:12.032 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6831/86612", 2023-10-08T00:17:12.033 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6829/86612", 2023-10-08T00:17:12.033 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:12.033 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:12.033 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:12.033 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:12.033 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:12.033 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:17:12.033 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2023-10-08T00:17:12.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.033 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:17:12.033 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:12.033 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:12.034 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:12.034 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:12.034 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:12.034 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.905615+0000", 2023-10-08T00:17:12.034 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:12.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.034 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:17:12.034 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:12.034 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:12.034 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:12.035 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:12.035 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:12.035 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.880709+0000", 2023-10-08T00:17:12.035 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:12.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.035 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:17:12.035 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:12.035 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:12.035 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:12.035 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:12.035 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:12.036 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.874704+0000", 2023-10-08T00:17:12.036 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:12.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.036 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:17:12.036 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:12.036 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:12.036 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:12.036 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:12.036 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:12.036 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.899604+0000", 2023-10-08T00:17:12.037 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:12.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.037 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:17:12.037 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:12.037 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:12.037 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:12.037 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:12.037 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:12.037 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.844371+0000", 2023-10-08T00:17:12.037 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:12.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.038 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:17:12.038 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:12.038 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:12.038 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:12.038 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:12.038 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:12.038 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.794810+0000", 2023-10-08T00:17:12.038 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:12.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.038 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:17:12.039 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:12.039 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:12.039 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:12.039 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:12.039 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:12.039 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.859474+0000", 2023-10-08T00:17:12.039 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:12.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.039 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:17:12.039 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:12.039 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:12.040 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:12.040 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:12.040 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:12.040 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.893130+0000", 2023-10-08T00:17:12.040 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:12.040 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:12.040 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:17:12.040 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2023-10-08T00:17:12.040 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [], 2023-10-08T00:17:12.040 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2023-10-08T00:17:12.040 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2023-10-08T00:17:12.041 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2023-10-08T00:17:12.041 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2023-10-08T00:17:12.041 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2023-10-08T00:17:12.041 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2023-10-08T00:17:12.041 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2023-10-08T00:17:12.041 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2023-10-08T00:17:12.041 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2023-10-08T00:17:12.041 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2023-10-08T00:17:12.041 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2023-10-08T00:17:12.041 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:12.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.041 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2023-10-08T00:17:12.042 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2023-10-08T00:17:12.042 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2023-10-08T00:17:12.042 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2023-10-08T00:17:12.042 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2023-10-08T00:17:12.042 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2023-10-08T00:17:12.042 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2023-10-08T00:17:12.042 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2023-10-08T00:17:12.042 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2023-10-08T00:17:12.042 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2023-10-08T00:17:12.042 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2023-10-08T00:17:12.042 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:12.042 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:17:12.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.932+0000 7f9d489be700 1 -- 172.21.15.99:0/3239151666 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9d30066f30 msgr2=0x7f9d300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:12.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.933+0000 7f9d489be700 1 --2- 172.21.15.99:0/3239151666 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9d30066f30 0x7f9d300693e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f9d34007540 tx=0x7f9d3400b040 comp rx=0 tx=0).stop 2023-10-08T00:17:12.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.933+0000 7f9d489be700 1 -- 172.21.15.99:0/3239151666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d44157e60 msgr2=0x7f9d440c4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:12.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.933+0000 7f9d489be700 1 --2- 172.21.15.99:0/3239151666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d44157e60 0x7f9d440c4c30 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c00b3c0 tx=0x7f9d3c017000 comp rx=0 tx=0).stop 2023-10-08T00:17:12.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.933+0000 7f9d489be700 1 -- 172.21.15.99:0/3239151666 shutdown_connections 2023-10-08T00:17:12.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.933+0000 7f9d489be700 1 --2- 172.21.15.99:0/3239151666 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9d30066f30 0x7f9d300693e0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:12.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.933+0000 7f9d489be700 1 --2- 172.21.15.99:0/3239151666 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d4406fef0 0x7f9d441533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:12.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.933+0000 7f9d489be700 1 --2- 172.21.15.99:0/3239151666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d44157e60 0x7f9d440c4c30 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:12.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.933+0000 7f9d489be700 1 --2- 172.21.15.99:0/3239151666 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9d44157a90 0x7f9d440c46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:12.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.933+0000 7f9d489be700 1 -- 172.21.15.99:0/3239151666 >> 172.21.15.99:0/3239151666 conn(0x7f9d4406bc70 msgr2=0x7f9d440b0120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:12.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.933+0000 7f9d489be700 1 -- 172.21.15.99:0/3239151666 shutdown_connections 2023-10-08T00:17:12.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:11.933+0000 7f9d489be700 1 -- 172.21.15.99:0/3239151666 wait complete. 2023-10-08T00:17:12.046 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:17:12.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.161+0000 7f6bff031700 1 Processor -- start 2023-10-08T00:17:12.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.161+0000 7f6bff031700 1 -- start start 2023-10-08T00:17:12.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.162+0000 7f6bff031700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f6bf8159e10 0x7f6bf80b2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:12.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.162+0000 7f6bff031700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bf815db80 con 0x7f6bf815a1e0 2023-10-08T00:17:12.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.162+0000 7f6bff031700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bf815de60 con 0x7f6bf80cde50 2023-10-08T00:17:12.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.162+0000 7f6bff031700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bf815e800 con 0x7f6bf8159e10 2023-10-08T00:17:12.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.162+0000 7f6beffff700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f6bf815a1e0 0x7f6bf80cd740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:52292/0 (socket says 172.21.15.99:52292) 2023-10-08T00:17:12.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.162+0000 7f6beffff700 1 -- 172.21.15.99:0/3332514767 learned_addr learned my addr 172.21.15.99:0/3332514767 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:17:12.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.162+0000 7f6bfcdcd700 1 --2- 172.21.15.99:0/3332514767 >> v2:172.21.15.191:3300/0 conn(0x7f6bf8159e10 0x7f6bf80b2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:12.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.162+0000 7f6bef7fe700 1 -- 172.21.15.99:0/3332514767 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 530762594 0 0) 0x7f6bf815db80 con 0x7f6bf815a1e0 2023-10-08T00:17:12.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.162+0000 7f6bef7fe700 1 -- 172.21.15.99:0/3332514767 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6be0003570 con 0x7f6bf815a1e0 2023-10-08T00:17:12.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.163+0000 7f6bef7fe700 1 -- 172.21.15.99:0/3332514767 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3209575908 0 0) 0x7f6be0003570 con 0x7f6bf815a1e0 2023-10-08T00:17:12.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.163+0000 7f6bef7fe700 1 -- 172.21.15.99:0/3332514767 >> v2:172.21.15.191:3300/0 conn(0x7f6bf8159e10 msgr2=0x7f6bf80b2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:12.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.163+0000 7f6bef7fe700 1 --2- 172.21.15.99:0/3332514767 >> v2:172.21.15.191:3300/0 conn(0x7f6bf8159e10 0x7f6bf80b2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:12.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.163+0000 7f6bef7fe700 1 -- 172.21.15.99:0/3332514767 >> v1:172.21.15.191:6789/0 conn(0x7f6bf80cde50 legacy=0x7f6bf80cbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:12.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.163+0000 7f6bef7fe700 1 -- 172.21.15.99:0/3332514767 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bf815e140 con 0x7f6bf815a1e0 2023-10-08T00:17:12.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.163+0000 7f6bef7fe700 1 -- 172.21.15.99:0/3332514767 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f6be8001470 con 0x7f6bf815a1e0 2023-10-08T00:17:12.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.163+0000 7f6bef7fe700 1 -- 172.21.15.99:0/3332514767 >> v1:172.21.15.99:6789/0 conn(0x7f6bf815a1e0 legacy=0x7f6bf80cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:12.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.163+0000 7f6bef7fe700 1 --2- 172.21.15.99:0/3332514767 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6be0003ab0 0x7f6be0003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:12.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.163+0000 7f6bef7fe700 1 --2- 172.21.15.99:0/3332514767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6be0004810 0x7f6be0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:12.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.163+0000 7f6bef7fe700 1 --2- 172.21.15.99:0/3332514767 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6be00052e0 0x7f6be000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:12.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.163+0000 7f6bef7fe700 1 -- 172.21.15.99:0/3332514767 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf815e800 con 0x7f6be0004810 2023-10-08T00:17:12.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.163+0000 7f6bef7fe700 1 -- 172.21.15.99:0/3332514767 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be000c260 con 0x7f6be0003ab0 2023-10-08T00:17:12.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.163+0000 7f6bef7fe700 1 -- 172.21.15.99:0/3332514767 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6be000c420 con 0x7f6be00052e0 2023-10-08T00:17:12.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.163+0000 7f6beffff700 1 --2- 172.21.15.99:0/3332514767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6be0004810 0x7f6be0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:12.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.163+0000 7f6bfcdcd700 1 --2- 172.21.15.99:0/3332514767 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6be0003ab0 0x7f6be0003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:12.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.163+0000 7f6bfd5ce700 1 --2- 172.21.15.99:0/3332514767 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6be00052e0 0x7f6be000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:12.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.163+0000 7f6beffff700 1 -- 172.21.15.99:0/3332514767 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6be00052e0 msgr2=0x7f6be000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:12.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.163+0000 7f6beffff700 1 --2- 172.21.15.99:0/3332514767 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6be00052e0 0x7f6be000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:12.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.163+0000 7f6beffff700 1 -- 172.21.15.99:0/3332514767 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6be0003ab0 msgr2=0x7f6be0003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:12.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.163+0000 7f6beffff700 1 --2- 172.21.15.99:0/3332514767 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6be0003ab0 0x7f6be0003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:12.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.163+0000 7f6beffff700 1 -- 172.21.15.99:0/3332514767 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be0003570 con 0x7f6be0004810 2023-10-08T00:17:12.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.163+0000 7f6bfd5ce700 1 -- 172.21.15.99:0/3332514767 reap_dead start 2023-10-08T00:17:12.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.163+0000 7f6beffff700 1 --2- 172.21.15.99:0/3332514767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6be0004810 0x7f6be0004c80 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f6be800c2a0 tx=0x7f6be8001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd6dfcc08a25673c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:12.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.163+0000 7f6bef7fe700 1 -- 172.21.15.99:0/3332514767 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be800cb70 con 0x7f6be0004810 2023-10-08T00:17:12.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.163+0000 7f6bef7fe700 1 -- 172.21.15.99:0/3332514767 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6be800ccf0 con 0x7f6be0004810 2023-10-08T00:17:12.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.163+0000 7f6bef7fe700 1 -- 172.21.15.99:0/3332514767 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be8005b70 con 0x7f6be0004810 2023-10-08T00:17:12.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.164+0000 7f6bff031700 1 -- 172.21.15.99:0/3332514767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6be0004810 msgr2=0x7f6be0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:12.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.164+0000 7f6bff031700 1 --2- 172.21.15.99:0/3332514767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6be0004810 0x7f6be0004c80 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f6be800c2a0 tx=0x7f6be8001d80 comp rx=0 tx=0).stop 2023-10-08T00:17:12.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.164+0000 7f6bff031700 1 -- 172.21.15.99:0/3332514767 shutdown_connections 2023-10-08T00:17:12.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.164+0000 7f6bff031700 1 --2- 172.21.15.99:0/3332514767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6be0004810 0x7f6be0004c80 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:12.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.164+0000 7f6bff031700 1 -- 172.21.15.99:0/3332514767 >> 172.21.15.99:0/3332514767 conn(0x7f6bf8067660 msgr2=0x7f6bf8067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:12.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.164+0000 7f6bff031700 1 -- 172.21.15.99:0/3332514767 shutdown_connections 2023-10-08T00:17:12.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.164+0000 7f6bff031700 1 -- 172.21.15.99:0/3332514767 wait complete. 2023-10-08T00:17:12.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.165+0000 7f6bff031700 1 Processor -- start 2023-10-08T00:17:12.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.165+0000 7f6bff031700 1 -- start start 2023-10-08T00:17:12.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.166+0000 7f6bff031700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6be0004810 0x7f6bf8158360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:12.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.166+0000 7f6bff031700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6bf815c700 0x7f6bf815cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:12.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.166+0000 7f6bff031700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6bf80c6a70 0x7f6bf80c6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:12.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.166+0000 7f6bff031700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf815d0b0 con 0x7f6bf80c6a70 2023-10-08T00:17:12.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.166+0000 7f6bff031700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf80c9650 con 0x7f6bf815c700 2023-10-08T00:17:12.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.166+0000 7f6bff031700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf80c8a40 con 0x7f6be0004810 2023-10-08T00:17:12.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.166+0000 7f6bfd5ce700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6bf80c6a70 0x7f6bf80c6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:12.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.166+0000 7f6bfd5ce700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6bf80c6a70 0x7f6bf80c6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36610/0 (socket says 172.21.15.99:36610) 2023-10-08T00:17:12.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.166+0000 7f6bfd5ce700 1 -- 172.21.15.99:0/1128659413 learned_addr learned my addr 172.21.15.99:0/1128659413 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:12.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.166+0000 7f6bfcdcd700 1 --2- 172.21.15.99:0/1128659413 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6be0004810 0x7f6bf8158360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:12.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.166+0000 7f6beffff700 1 --2- 172.21.15.99:0/1128659413 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6bf815c700 0x7f6bf815cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:12.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.166+0000 7f6bfd5ce700 1 -- 172.21.15.99:0/1128659413 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6be0004810 msgr2=0x7f6bf8158360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:12.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.166+0000 7f6bfd5ce700 1 --2- 172.21.15.99:0/1128659413 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6be0004810 0x7f6bf8158360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:12.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.166+0000 7f6bfd5ce700 1 -- 172.21.15.99:0/1128659413 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6bf815c700 msgr2=0x7f6bf815cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:12.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.166+0000 7f6bfd5ce700 1 --2- 172.21.15.99:0/1128659413 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6bf815c700 0x7f6bf815cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:12.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.166+0000 7f6bfd5ce700 1 -- 172.21.15.99:0/1128659413 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bf80c8d20 con 0x7f6bf80c6a70 2023-10-08T00:17:12.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.166+0000 7f6bfd5ce700 1 --2- 172.21.15.99:0/1128659413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6bf80c6a70 0x7f6bf80c6f10 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f6bf80c7ce0 tx=0x7f6bf0009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:12.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.167+0000 7f6becff9700 1 -- 172.21.15.99:0/1128659413 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf00032e0 con 0x7f6bf80c6a70 2023-10-08T00:17:12.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.167+0000 7f6becff9700 1 -- 172.21.15.99:0/1128659413 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bf0003440 con 0x7f6bf80c6a70 2023-10-08T00:17:12.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.167+0000 7f6bff031700 1 -- 172.21.15.99:0/1128659413 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bf80c9000 con 0x7f6bf80c6a70 2023-10-08T00:17:12.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.167+0000 7f6becff9700 1 -- 172.21.15.99:0/1128659413 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf00017d0 con 0x7f6bf80c6a70 2023-10-08T00:17:12.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.167+0000 7f6bff031700 1 -- 172.21.15.99:0/1128659413 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bf81588a0 con 0x7f6bf80c6a70 2023-10-08T00:17:12.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.168+0000 7f6bff031700 1 -- 172.21.15.99:0/1128659413 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bf804c3c0 con 0x7f6bf80c6a70 2023-10-08T00:17:12.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.168+0000 7f6becff9700 1 -- 172.21.15.99:0/1128659413 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6bf0001bc0 con 0x7f6bf80c6a70 2023-10-08T00:17:12.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.168+0000 7f6becff9700 1 --2- 172.21.15.99:0/1128659413 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6be4066f30 0x7f6be40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:12.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.168+0000 7f6becff9700 1 -- 172.21.15.99:0/1128659413 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6bf0083dd0 con 0x7f6bf80c6a70 2023-10-08T00:17:12.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.168+0000 7f6becff9700 1 -- 172.21.15.99:0/1128659413 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=142}) v3 -- 0x7f6be406ea50 con 0x7f6bf80c6a70 2023-10-08T00:17:12.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.168+0000 7f6bfcdcd700 1 --2- 172.21.15.99:0/1128659413 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6be4066f30 0x7f6be40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:12.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.169+0000 7f6bfcdcd700 1 --2- 172.21.15.99:0/1128659413 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6be4066f30 0x7f6be40693e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4006fd0 tx=0x7f6bf4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:12.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.172+0000 7f6becff9700 1 -- 172.21.15.99:0/1128659413 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6bf0019060 con 0x7f6bf80c6a70 2023-10-08T00:17:12.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.400+0000 7f6bff031700 1 -- 172.21.15.99:0/1128659413 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6bf815e420 con 0x7f6bf80c6a70 2023-10-08T00:17:12.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.400+0000 7f6becff9700 1 -- 172.21.15.99:0/1128659413 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f6bf0085020 con 0x7f6bf80c6a70 2023-10-08T00:17:12.401 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:17:12.401 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":63,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5813,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/887374839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":887374839},{"type":"v1","addr":"172.21.15.191:6837","nonce":887374839}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":6445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/2409696901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2409696901},{"type":"v1","addr":"172.21.15.191:6836","nonce":2409696901}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":7095,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/1924096228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1924096228},{"type":"v1","addr":"172.21.15.99:6834","nonce":1924096228}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:16:50.092494+0000","modified":"2023-10-08T00:16:53.744105+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7098":{"gid":7098,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/2522933027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":2522933027},{"type":"v1","addr":"172.21.15.99:6835","nonce":2522933027}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-08T00:17:12.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.402+0000 7f6bff031700 1 -- 172.21.15.99:0/1128659413 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6be4066f30 msgr2=0x7f6be40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:12.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.402+0000 7f6bff031700 1 --2- 172.21.15.99:0/1128659413 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6be4066f30 0x7f6be40693e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4006fd0 tx=0x7f6bf4008040 comp rx=0 tx=0).stop 2023-10-08T00:17:12.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.402+0000 7f6bff031700 1 -- 172.21.15.99:0/1128659413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6bf80c6a70 msgr2=0x7f6bf80c6f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:12.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.402+0000 7f6bff031700 1 --2- 172.21.15.99:0/1128659413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6bf80c6a70 0x7f6bf80c6f10 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f6bf80c7ce0 tx=0x7f6bf0009040 comp rx=0 tx=0).stop 2023-10-08T00:17:12.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.402+0000 7f6bff031700 1 -- 172.21.15.99:0/1128659413 shutdown_connections 2023-10-08T00:17:12.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.402+0000 7f6bff031700 1 --2- 172.21.15.99:0/1128659413 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6be4066f30 0x7f6be40693e0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:12.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.402+0000 7f6bff031700 1 --2- 172.21.15.99:0/1128659413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6bf80c6a70 0x7f6bf80c6f10 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:12.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.402+0000 7f6bff031700 1 --2- 172.21.15.99:0/1128659413 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6be0004810 0x7f6bf8158360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:12.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.402+0000 7f6bff031700 1 --2- 172.21.15.99:0/1128659413 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6bf815c700 0x7f6bf815cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:12.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.402+0000 7f6bff031700 1 -- 172.21.15.99:0/1128659413 >> 172.21.15.99:0/1128659413 conn(0x7f6bf8067660 msgr2=0x7f6bf8159180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:12.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.403+0000 7f6bff031700 1 -- 172.21.15.99:0/1128659413 shutdown_connections 2023-10-08T00:17:12.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.403+0000 7f6bff031700 1 -- 172.21.15.99:0/1128659413 wait complete. 2023-10-08T00:17:12.405 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 63 2023-10-08T00:17:12.415 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2023-10-08T00:17:12.547 INFO:teuthology.orchestra.run.smithi099.stdout:{"cluster_fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","whoami":0,"id":7098,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":21.799698813999999,"mdsmap_epoch":63,"osdmap_epoch":141,"osdmap_epoch_barrier":141,"uptime":26.934538352000001} 2023-10-08T00:17:12.548 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", "fs_name": "cephfs", "id": 7098, "mdsmap_epoch": 63, "osdmap_epoch": 141, "osdmap_epoch_barrier": 141, "rank_uptime": 21.799698814, "state": "up:active", "uptime": 26.934538352, "want_state": "up:active", "whoami": 0 } 2023-10-08T00:17:12.548 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-08T00:17:12.548 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:17:12.548 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2023-10-08T00:17:12.564 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:17:12.564 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2023-10-08T00:17:12.622 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-08T00:17:12.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.776+0000 7fcc6bef8700 1 Processor -- start 2023-10-08T00:17:12.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.776+0000 7fcc6bef8700 1 -- start start 2023-10-08T00:17:12.780 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.777+0000 7fcc6bef8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcc6415a620 0x7fcc6415d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:12.781 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.777+0000 7fcc6bef8700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fcc6415d580 0x7fcc64158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:12.781 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.777+0000 7fcc6bef8700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc6415cbb0 con 0x7fcc64157870 2023-10-08T00:17:12.781 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.777+0000 7fcc6bef8700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc6415bdb0 con 0x7fcc6415d580 2023-10-08T00:17:12.781 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.777+0000 7fcc6bef8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6415bfa0 con 0x7fcc6415a620 2023-10-08T00:17:12.781 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.777+0000 7fcc69c94700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fcc64157870 0x7fcc6415b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:52298/0 (socket says 172.21.15.99:52298) 2023-10-08T00:17:12.781 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.777+0000 7fcc69c94700 1 -- 172.21.15.99:0/3194371609 learned_addr learned my addr 172.21.15.99:0/3194371609 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:17:12.781 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.777+0000 7fcc69493700 1 --2- 172.21.15.99:0/3194371609 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcc6415a620 0x7fcc6415d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:12.781 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.777+0000 7fcc6a495700 1 --2- 172.21.15.99:0/3194371609 >> v2:172.21.15.191:3300/0 conn(0x7fcc6415d580 0x7fcc64158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:12.781 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.777+0000 7fcc68c92700 1 -- 172.21.15.99:0/3194371609 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2031924369 0 0) 0x7fcc6415cbb0 con 0x7fcc64157870 2023-10-08T00:17:12.781 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.777+0000 7fcc68c92700 1 -- 172.21.15.99:0/3194371609 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc50003570 con 0x7fcc64157870 2023-10-08T00:17:12.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.778+0000 7fcc68c92700 1 -- 172.21.15.99:0/3194371609 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 896382974 0 0) 0x7fcc50003570 con 0x7fcc64157870 2023-10-08T00:17:12.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.778+0000 7fcc68c92700 1 -- 172.21.15.99:0/3194371609 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcc6415a620 msgr2=0x7fcc6415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:12.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.778+0000 7fcc68c92700 1 --2- 172.21.15.99:0/3194371609 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcc6415a620 0x7fcc6415d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:12.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.778+0000 7fcc68c92700 1 -- 172.21.15.99:0/3194371609 >> v2:172.21.15.191:3300/0 conn(0x7fcc6415d580 msgr2=0x7fcc64158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:12.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.778+0000 7fcc68c92700 1 --2- 172.21.15.99:0/3194371609 >> v2:172.21.15.191:3300/0 conn(0x7fcc6415d580 0x7fcc64158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:12.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.778+0000 7fcc68c92700 1 -- 172.21.15.99:0/3194371609 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc6415c280 con 0x7fcc64157870 2023-10-08T00:17:12.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.778+0000 7fcc68c92700 1 -- 172.21.15.99:0/3194371609 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7fcc600017f0 con 0x7fcc64157870 2023-10-08T00:17:12.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.778+0000 7fcc68c92700 1 -- 172.21.15.99:0/3194371609 >> v1:172.21.15.99:6789/0 conn(0x7fcc64157870 legacy=0x7fcc6415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:12.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.778+0000 7fcc68c92700 1 --2- 172.21.15.99:0/3194371609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc50003a60 0x7fcc50003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:12.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.778+0000 7fcc68c92700 1 --2- 172.21.15.99:0/3194371609 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcc50004810 0x7fcc50004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:12.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.778+0000 7fcc68c92700 1 --2- 172.21.15.99:0/3194371609 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcc6415a620 0x7fcc500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:12.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.778+0000 7fcc68c92700 1 -- 172.21.15.99:0/3194371609 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6415bfa0 con 0x7fcc50003a60 2023-10-08T00:17:12.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.778+0000 7fcc68c92700 1 -- 172.21.15.99:0/3194371609 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5000ba60 con 0x7fcc50004810 2023-10-08T00:17:12.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.778+0000 7fcc68c92700 1 -- 172.21.15.99:0/3194371609 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5000bba0 con 0x7fcc6415a620 2023-10-08T00:17:12.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.778+0000 7fcc69c94700 1 --2- 172.21.15.99:0/3194371609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc50003a60 0x7fcc50003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:12.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.778+0000 7fcc69493700 1 --2- 172.21.15.99:0/3194371609 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcc50004810 0x7fcc50004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:12.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.778+0000 7fcc6a495700 1 --2- 172.21.15.99:0/3194371609 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcc6415a620 0x7fcc500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:12.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.778+0000 7fcc69c94700 1 -- 172.21.15.99:0/3194371609 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcc6415a620 msgr2=0x7fcc500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:12.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.778+0000 7fcc69c94700 1 --2- 172.21.15.99:0/3194371609 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcc6415a620 0x7fcc500051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:12.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.778+0000 7fcc69c94700 1 -- 172.21.15.99:0/3194371609 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcc50004810 msgr2=0x7fcc50004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:12.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.778+0000 7fcc69c94700 1 --2- 172.21.15.99:0/3194371609 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcc50004810 0x7fcc50004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:12.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.778+0000 7fcc69c94700 1 -- 172.21.15.99:0/3194371609 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc50003570 con 0x7fcc50003a60 2023-10-08T00:17:12.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.778+0000 7fcc69c94700 1 --2- 172.21.15.99:0/3194371609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc50003a60 0x7fcc50003e70 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fcc6000a9a0 tx=0x7fcc60002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3f5838f474d8dd4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:12.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.778+0000 7fcc68c92700 1 -- 172.21.15.99:0/3194371609 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc600070d0 con 0x7fcc50003a60 2023-10-08T00:17:12.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.778+0000 7fcc68c92700 1 -- 172.21.15.99:0/3194371609 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc60007230 con 0x7fcc50003a60 2023-10-08T00:17:12.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.778+0000 7fcc68c92700 1 -- 172.21.15.99:0/3194371609 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc600075f0 con 0x7fcc50003a60 2023-10-08T00:17:12.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.779+0000 7fcc6bef8700 1 -- 172.21.15.99:0/3194371609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc50003a60 msgr2=0x7fcc50003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:12.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.779+0000 7fcc6bef8700 1 --2- 172.21.15.99:0/3194371609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc50003a60 0x7fcc50003e70 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fcc6000a9a0 tx=0x7fcc60002210 comp rx=0 tx=0).stop 2023-10-08T00:17:12.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.779+0000 7fcc6a495700 1 -- 172.21.15.99:0/3194371609 reap_dead start 2023-10-08T00:17:12.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.779+0000 7fcc6bef8700 1 -- 172.21.15.99:0/3194371609 shutdown_connections 2023-10-08T00:17:12.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.779+0000 7fcc6bef8700 1 -- 172.21.15.99:0/3194371609 >> 172.21.15.99:0/3194371609 conn(0x7fcc64068ce0 msgr2=0x7fcc640690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:12.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.779+0000 7fcc6bef8700 1 -- 172.21.15.99:0/3194371609 shutdown_connections 2023-10-08T00:17:12.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.779+0000 7fcc6bef8700 1 -- 172.21.15.99:0/3194371609 wait complete. 2023-10-08T00:17:12.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.780+0000 7fcc6bef8700 1 Processor -- start 2023-10-08T00:17:12.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.780+0000 7fcc6bef8700 1 -- start start 2023-10-08T00:17:12.788 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.781+0000 7fcc6bef8700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcc50003a60 0x7fcc640b5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:12.788 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.781+0000 7fcc6bef8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc6414ba70 0x7fcc640b1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:12.788 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.781+0000 7fcc6bef8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcc640b2190 0x7fcc640b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:12.788 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.781+0000 7fcc6bef8700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc640b2600 con 0x7fcc6414ba70 2023-10-08T00:17:12.788 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.781+0000 7fcc6bef8700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc640b6940 con 0x7fcc50003a60 2023-10-08T00:17:12.788 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.781+0000 7fcc6bef8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc640b5d30 con 0x7fcc640b2190 2023-10-08T00:17:12.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.781+0000 7fcc69493700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc6414ba70 0x7fcc640b1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:12.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.781+0000 7fcc69493700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc6414ba70 0x7fcc640b1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36630/0 (socket says 172.21.15.99:36630) 2023-10-08T00:17:12.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.781+0000 7fcc69493700 1 -- 172.21.15.99:0/3258705852 learned_addr learned my addr 172.21.15.99:0/3258705852 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:12.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.781+0000 7fcc69c94700 1 --2- 172.21.15.99:0/3258705852 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcc50003a60 0x7fcc640b5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:12.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.781+0000 7fcc6a495700 1 --2- 172.21.15.99:0/3258705852 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcc640b2190 0x7fcc640b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:12.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.781+0000 7fcc69493700 1 -- 172.21.15.99:0/3258705852 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcc640b2190 msgr2=0x7fcc640b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:12.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.781+0000 7fcc69493700 1 --2- 172.21.15.99:0/3258705852 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcc640b2190 0x7fcc640b06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:12.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.781+0000 7fcc69493700 1 -- 172.21.15.99:0/3258705852 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcc50003a60 msgr2=0x7fcc640b5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:12.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.781+0000 7fcc69493700 1 --2- 172.21.15.99:0/3258705852 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcc50003a60 0x7fcc640b5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:12.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.781+0000 7fcc69493700 1 -- 172.21.15.99:0/3258705852 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc640b6010 con 0x7fcc6414ba70 2023-10-08T00:17:12.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.781+0000 7fcc69493700 1 --2- 172.21.15.99:0/3258705852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc6414ba70 0x7fcc640b1c50 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fcc54011b40 tx=0x7fcc540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:12.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.782+0000 7fcc59ffb700 1 -- 172.21.15.99:0/3258705852 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5400f940 con 0x7fcc6414ba70 2023-10-08T00:17:12.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.782+0000 7fcc59ffb700 1 -- 172.21.15.99:0/3258705852 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc5400faa0 con 0x7fcc6414ba70 2023-10-08T00:17:12.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.782+0000 7fcc6bef8700 1 -- 172.21.15.99:0/3258705852 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc640b62f0 con 0x7fcc6414ba70 2023-10-08T00:17:12.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.782+0000 7fcc59ffb700 1 -- 172.21.15.99:0/3258705852 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5400fe20 con 0x7fcc6414ba70 2023-10-08T00:17:12.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.782+0000 7fcc6bef8700 1 -- 172.21.15.99:0/3258705852 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc640b5750 con 0x7fcc6414ba70 2023-10-08T00:17:12.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.783+0000 7fcc6bef8700 1 -- 172.21.15.99:0/3258705852 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc640b6af0 con 0x7fcc6414ba70 2023-10-08T00:17:12.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.783+0000 7fcc59ffb700 1 -- 172.21.15.99:0/3258705852 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcc54014650 con 0x7fcc6414ba70 2023-10-08T00:17:12.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.783+0000 7fcc59ffb700 1 --2- 172.21.15.99:0/3258705852 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcc48066f30 0x7fcc480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:12.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.784+0000 7fcc59ffb700 1 -- 172.21.15.99:0/3258705852 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcc54018030 con 0x7fcc6414ba70 2023-10-08T00:17:12.791 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.784+0000 7fcc59ffb700 1 -- 172.21.15.99:0/3258705852 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=142}) v3 -- 0x7fcc4806ea50 con 0x7fcc6414ba70 2023-10-08T00:17:12.792 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.784+0000 7fcc69c94700 1 --2- 172.21.15.99:0/3258705852 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcc48066f30 0x7fcc480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:12.792 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.784+0000 7fcc69c94700 1 --2- 172.21.15.99:0/3258705852 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcc48066f30 0x7fcc480693e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fcc6000a940 tx=0x7fcc60015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:12.792 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.788+0000 7fcc59ffb700 1 -- 172.21.15.99:0/3258705852 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcc640b6af0 con 0x7fcc6414ba70 2023-10-08T00:17:12.971 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.970+0000 7fcc6bef8700 1 -- 172.21.15.99:0/3258705852 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fcc640b6ab0 con 0x7fcc6414ba70 2023-10-08T00:17:12.972 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.971+0000 7fcc59ffb700 1 -- 172.21.15.99:0/3258705852 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v141) v1 ==== 81+0+32301 (secure 0 0 0) 0x7fcc54022030 con 0x7fcc6414ba70 2023-10-08T00:17:12.972 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:17:12.973 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:17:12.973 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 141, 2023-10-08T00:17:12.973 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", 2023-10-08T00:17:12.973 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2023-10-08T00:10:36.303601+0000", 2023-10-08T00:17:12.973 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2023-10-08T00:17:06.114261+0000", 2023-10-08T00:17:12.973 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2023-10-08T00:10:41.709171+0000", 2023-10-08T00:17:12.973 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2023-10-08T00:10:38.690404+0000", 2023-10-08T00:17:12.973 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-08T00:17:12.973 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2023-10-08T00:17:12.973 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2023-10-08T00:17:12.973 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2023-10-08T00:17:12.974 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2023-10-08T00:17:12.974 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2023-10-08T00:17:12.974 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2023-10-08T00:17:12.974 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:17:12.974 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2023-10-08T00:17:12.974 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2023-10-08T00:17:12.974 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-08T00:17:12.974 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-08T00:17:12.974 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2023-10-08T00:17:12.974 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 17, 2023-10-08T00:17:12.974 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2023-10-08T00:17:12.974 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2023-10-08T00:17:12.975 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "jewel", 2023-10-08T00:17:12.975 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "quincy", 2023-10-08T00:17:12.975 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2023-10-08T00:17:12.975 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.975 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2023-10-08T00:17:12.975 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2023-10-08T00:17:12.975 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:10:42.063344+0000", 2023-10-08T00:17:12.975 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:17:12.975 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:17:12.975 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:17:12.975 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:17:12.976 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:17:12.976 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:17:12.976 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:17:12.976 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:17:12.976 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:17:12.976 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:17:12.976 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:17:12.976 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:17:12.976 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2023-10-08T00:17:12.976 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2023-10-08T00:17:12.976 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2023-10-08T00:17:12.977 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2023-10-08T00:17:12.977 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2023-10-08T00:17:12.977 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:17:12.977 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:17:12.977 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:17:12.977 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:17:12.977 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:17:12.977 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:17:12.977 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:17:12.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.977 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "112", 2023-10-08T00:17:12.977 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:17:12.978 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:17:12.978 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:17:12.978 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:17:12.978 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:17:12.978 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:17:12.978 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:17:12.978 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:17:12.978 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:17:12.978 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:17:12.978 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:17:12.978 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:17:12.979 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:17:12.979 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:17:12.979 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:17:12.979 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:17:12.979 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:17:12.979 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:17:12.979 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:17:12.979 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:17:12.979 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:17:12.979 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:17:12.979 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:17:12.979 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:17:12.980 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:17:12.980 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:17:12.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.980 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:17:12.980 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:17:12.980 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:17:12.981 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:17:12.981 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:17:12.981 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:17:12.981 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:17:12.981 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:17:12.981 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:17:12.981 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:17:12.981 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:17:12.981 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:17:12.981 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2023-10-08T00:17:12.981 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2023-10-08T00:17:12.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.982 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:17:12.982 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2023-10-08T00:17:12.982 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:12.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.982 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 16, 2023-10-08T00:17:12.982 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2023-10-08T00:17:12.982 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:16:48.430607+0000", 2023-10-08T00:17:12.982 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:17:12.982 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:17:12.982 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:17:12.983 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:17:12.983 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:17:12.983 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:17:12.983 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:17:12.983 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:17:12.983 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:17:12.983 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:17:12.983 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:17:12.983 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:17:12.983 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2023-10-08T00:17:12.983 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2023-10-08T00:17:12.983 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2023-10-08T00:17:12.984 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2023-10-08T00:17:12.984 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2023-10-08T00:17:12.984 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:17:12.984 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:17:12.984 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:17:12.984 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:17:12.984 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:17:12.984 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:17:12.984 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:17:12.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.984 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "138", 2023-10-08T00:17:12.985 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:17:12.985 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:17:12.985 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:17:12.985 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:17:12.985 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:17:12.985 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:17:12.985 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:17:12.985 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:17:12.985 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:17:12.985 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:17:12.985 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:17:12.985 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:17:12.986 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:17:12.986 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:17:12.986 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:17:12.986 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:17:12.986 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:17:12.986 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:17:12.986 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:17:12.986 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:17:12.986 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:17:12.987 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:17:12.987 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:17:12.987 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:17:12.987 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:17:12.987 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:17:12.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.987 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:17:12.987 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:17:12.988 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:17:12.988 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:17:12.988 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:17:12.988 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:17:12.988 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:17:12.988 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:17:12.988 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:17:12.988 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:17:12.988 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:17:12.989 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:17:12.989 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2023-10-08T00:17:12.989 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2023-10-08T00:17:12.989 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2023-10-08T00:17:12.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.989 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:17:12.989 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:17:12.989 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2023-10-08T00:17:12.989 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:12.990 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:12.990 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.990 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.990 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 17, 2023-10-08T00:17:12.990 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2023-10-08T00:17:12.990 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:16:49.063424+0000", 2023-10-08T00:17:12.990 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1027, 2023-10-08T00:17:12.990 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full,full_quota", 2023-10-08T00:17:12.990 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:17:12.990 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:17:12.991 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:17:12.991 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:17:12.991 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:17:12.991 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:17:12.991 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:17:12.991 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:17:12.991 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:17:12.991 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:17:12.991 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2023-10-08T00:17:12.991 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2023-10-08T00:17:12.991 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2023-10-08T00:17:12.991 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2023-10-08T00:17:12.992 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2023-10-08T00:17:12.992 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:17:12.992 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:17:12.992 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:17:12.992 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:17:12.992 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:17:12.992 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:17:12.992 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:17:12.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.992 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "141", 2023-10-08T00:17:12.992 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:17:12.992 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:17:12.993 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:17:12.993 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:17:12.993 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:17:12.993 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:17:12.993 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:17:12.993 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:17:12.993 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:17:12.993 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 33554432, 2023-10-08T00:17:12.993 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:17:12.993 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:17:12.993 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:17:12.993 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:17:12.994 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:17:12.994 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:17:12.994 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:17:12.994 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:17:12.994 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:17:12.994 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:17:12.994 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:17:12.994 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:17:12.994 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:17:12.994 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:17:12.994 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:17:12.995 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:17:12.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.995 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:17:12.995 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:17:12.995 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:17:12.995 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:17:12.995 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:17:12.995 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:17:12.995 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:17:12.995 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:17:12.995 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:17:12.996 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:17:12.996 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:17:12.996 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:17:12.996 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2023-10-08T00:17:12.996 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2023-10-08T00:17:12.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.996 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:17:12.996 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:17:12.996 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2023-10-08T00:17:12.996 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:12.996 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:12.997 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:12.997 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:17:12.997 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2023-10-08T00:17:12.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.997 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:17:12.997 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:12.997 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:12.997 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:12.997 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:12.997 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:12.997 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:12.998 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:12.998 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:12.998 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:12.998 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:12.998 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:12.998 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:12.998 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:12.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.998 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:12.998 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2023-10-08T00:17:12.998 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:12.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:12.999 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:12.999 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2023-10-08T00:17:12.999 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:12.999 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:12.999 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:12.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:12.999 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:12.999 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:12.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.000 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:13.000 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2023-10-08T00:17:13.000 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:13.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.000 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.000 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:13.000 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2023-10-08T00:17:13.000 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:13.000 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:13.000 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:13.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.000 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:13.001 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:13.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.001 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:13.001 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2023-10-08T00:17:13.001 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:13.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.001 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:13.001 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2023-10-08T00:17:13.001 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:13.001 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:13.002 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:13.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.002 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:13.002 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:13.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.002 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:13.002 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2023-10-08T00:17:13.002 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:13.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.002 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:13.003 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2023-10-08T00:17:13.003 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:13.003 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:13.003 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:13.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.003 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/86365", 2023-10-08T00:17:13.003 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/86365", 2023-10-08T00:17:13.003 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/86365", 2023-10-08T00:17:13.003 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/86365", 2023-10-08T00:17:13.003 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:13.003 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:13.003 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:13.004 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:13.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.004 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:17:13.004 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:13.004 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:13.004 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:13.004 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:13.004 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:13.004 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:13.004 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:13.005 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:13.005 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:13.005 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:13.005 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:13.005 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:13.005 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:13.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.005 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:13.005 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2023-10-08T00:17:13.005 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:13.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.006 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:13.006 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2023-10-08T00:17:13.006 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:13.006 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:13.006 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:13.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.006 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:13.006 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:13.006 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.006 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:13.006 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2023-10-08T00:17:13.006 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:13.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.007 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:13.007 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2023-10-08T00:17:13.007 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:13.007 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:13.007 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:13.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.007 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:13.007 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:13.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.007 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:13.008 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2023-10-08T00:17:13.008 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:13.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.008 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:13.008 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2023-10-08T00:17:13.008 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:13.008 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:13.008 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:13.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.008 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:13.008 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:13.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.009 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:13.009 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2023-10-08T00:17:13.009 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:13.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.009 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:13.009 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2023-10-08T00:17:13.009 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:13.009 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:13.009 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:13.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.010 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/86366", 2023-10-08T00:17:13.010 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/86366", 2023-10-08T00:17:13.010 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/86366", 2023-10-08T00:17:13.010 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/86366", 2023-10-08T00:17:13.010 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:13.010 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:13.010 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:13.010 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:13.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.010 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:17:13.011 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:13.011 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:13.011 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:13.011 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:13.011 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:13.011 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:13.011 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:13.011 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:13.011 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:13.011 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:13.011 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:13.011 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:13.011 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:13.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.012 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:13.012 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2023-10-08T00:17:13.012 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:13.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.012 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:13.012 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2023-10-08T00:17:13.012 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:13.012 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:13.012 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:13.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.013 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:13.013 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:13.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.013 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:13.013 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2023-10-08T00:17:13.013 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:13.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.013 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:13.013 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2023-10-08T00:17:13.013 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:13.013 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:13.014 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:13.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.014 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:13.014 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:13.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.014 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:13.014 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2023-10-08T00:17:13.014 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:13.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.014 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:13.014 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2023-10-08T00:17:13.015 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:13.015 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:13.015 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:13.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.015 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:13.015 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:13.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.015 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:13.015 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2023-10-08T00:17:13.015 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:13.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.016 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:13.016 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2023-10-08T00:17:13.016 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:13.016 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:13.016 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:13.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.016 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6817/86382", 2023-10-08T00:17:13.016 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/86382", 2023-10-08T00:17:13.016 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/86382", 2023-10-08T00:17:13.016 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/86382", 2023-10-08T00:17:13.016 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:13.017 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:13.017 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:13.017 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:13.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.017 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:17:13.017 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:13.017 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:13.017 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:13.017 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:13.017 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:13.017 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:13.018 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:13.018 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:13.018 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:13.018 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:13.018 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:13.018 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:13.018 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:13.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.018 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:13.018 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2023-10-08T00:17:13.018 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:13.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.019 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:13.019 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2023-10-08T00:17:13.019 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:13.019 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:13.019 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:13.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.019 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:13.019 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:13.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.020 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:13.020 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2023-10-08T00:17:13.020 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:13.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.020 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:13.020 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2023-10-08T00:17:13.020 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:13.020 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:13.020 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:13.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.020 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:13.021 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:13.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.021 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:13.021 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2023-10-08T00:17:13.021 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:13.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.021 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:13.021 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2023-10-08T00:17:13.021 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:13.021 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:13.022 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:13.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.022 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:13.022 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:13.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.022 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:13.022 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2023-10-08T00:17:13.022 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:13.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.022 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:13.022 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2023-10-08T00:17:13.023 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:13.023 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:13.023 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:13.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.023 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6820/86393", 2023-10-08T00:17:13.023 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/86393", 2023-10-08T00:17:13.023 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/86393", 2023-10-08T00:17:13.023 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/86393", 2023-10-08T00:17:13.023 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:13.023 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:13.023 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:13.023 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:13.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.024 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:17:13.024 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:13.024 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:13.024 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:13.024 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:13.024 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:13.024 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:13.024 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:13.024 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:13.024 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:13.025 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:13.025 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:13.025 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:13.025 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:13.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.025 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:13.025 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6800", 2023-10-08T00:17:13.025 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:13.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.025 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:13.026 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6801", 2023-10-08T00:17:13.026 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:13.026 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:13.026 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:13.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.026 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:13.026 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:13.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.026 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:13.026 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6802", 2023-10-08T00:17:13.026 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:13.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.027 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:13.027 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6803", 2023-10-08T00:17:13.027 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:13.027 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:13.027 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:13.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.027 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:13.027 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:13.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.027 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:13.027 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6806", 2023-10-08T00:17:13.028 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:13.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.028 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:13.028 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6807", 2023-10-08T00:17:13.028 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:13.028 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:13.028 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:13.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.028 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:13.028 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:13.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.029 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:13.029 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6804", 2023-10-08T00:17:13.029 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:13.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.029 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:13.029 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6805", 2023-10-08T00:17:13.029 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:13.029 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:13.029 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:13.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.030 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6801/86570", 2023-10-08T00:17:13.030 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6803/86570", 2023-10-08T00:17:13.030 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6807/86570", 2023-10-08T00:17:13.030 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6805/86570", 2023-10-08T00:17:13.030 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:13.030 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:13.030 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:13.030 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:13.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.030 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:17:13.030 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:13.031 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:13.031 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:13.031 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:13.031 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:13.031 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:13.031 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:13.031 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:13.031 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:13.031 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:13.031 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:13.031 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:13.031 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:13.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.032 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:13.032 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6816", 2023-10-08T00:17:13.032 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:13.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.032 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:13.032 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6818", 2023-10-08T00:17:13.032 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:13.032 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:13.032 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:13.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.033 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:13.033 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:13.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.033 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:13.033 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6820", 2023-10-08T00:17:13.033 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:13.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.033 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:13.033 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6821", 2023-10-08T00:17:13.033 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:13.034 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:13.034 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:13.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.034 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:13.034 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:13.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.034 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:13.034 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6826", 2023-10-08T00:17:13.034 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:13.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.035 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:13.035 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6828", 2023-10-08T00:17:13.035 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:13.035 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:13.035 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:13.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.035 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:13.035 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:13.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.035 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:13.035 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6823", 2023-10-08T00:17:13.035 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:13.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.036 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:13.036 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6825", 2023-10-08T00:17:13.036 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:13.036 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:13.036 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:13.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.036 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6818/86608", 2023-10-08T00:17:13.036 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6821/86608", 2023-10-08T00:17:13.036 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6828/86608", 2023-10-08T00:17:13.037 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6825/86608", 2023-10-08T00:17:13.037 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:13.037 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:13.037 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:13.037 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:13.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.037 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:17:13.037 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:13.037 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:13.037 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:13.037 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:13.038 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:13.038 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:13.038 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:13.038 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:13.038 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:13.038 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:13.038 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:13.038 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:13.038 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:13.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.038 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:13.039 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6808", 2023-10-08T00:17:13.039 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:13.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.039 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:13.039 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6809", 2023-10-08T00:17:13.039 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:13.039 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:13.039 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:13.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.039 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:13.040 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:13.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.040 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:13.040 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6810", 2023-10-08T00:17:13.040 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:13.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.040 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:13.040 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6811", 2023-10-08T00:17:13.040 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:13.040 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:13.040 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:13.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.041 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:13.041 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:13.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.041 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:13.041 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6814", 2023-10-08T00:17:13.041 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:13.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.041 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:13.041 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6815", 2023-10-08T00:17:13.041 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:13.042 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:13.042 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:13.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.042 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:13.042 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:13.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.042 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:13.042 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6812", 2023-10-08T00:17:13.042 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:13.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.042 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:13.043 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6813", 2023-10-08T00:17:13.043 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:13.043 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:13.043 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:13.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.043 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6809/86571", 2023-10-08T00:17:13.043 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6811/86571", 2023-10-08T00:17:13.043 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6815/86571", 2023-10-08T00:17:13.043 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6813/86571", 2023-10-08T00:17:13.043 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:13.043 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:13.043 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:13.044 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:13.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.044 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:17:13.044 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:13.044 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:13.044 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:13.044 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:13.044 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:13.044 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:13.044 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:13.045 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:13.045 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:13.045 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:13.045 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:13.045 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:13.045 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:13.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.045 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:13.045 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6817", 2023-10-08T00:17:13.045 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:13.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.046 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:13.046 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6819", 2023-10-08T00:17:13.046 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:13.046 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:13.046 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:13.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.046 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:13.046 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:13.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.046 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:13.046 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6822", 2023-10-08T00:17:13.047 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:13.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.047 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:13.047 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6824", 2023-10-08T00:17:13.047 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:13.047 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:13.047 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:13.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.047 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:13.047 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:13.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.048 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:13.048 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6830", 2023-10-08T00:17:13.048 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:13.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.048 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:13.048 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6831", 2023-10-08T00:17:13.048 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:13.048 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:13.048 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:13.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.049 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:13.049 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:13.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.049 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:13.049 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6827", 2023-10-08T00:17:13.049 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:13.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.049 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:13.049 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6829", 2023-10-08T00:17:13.049 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:13.049 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:13.050 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:13.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.050 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6819/86612", 2023-10-08T00:17:13.050 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6824/86612", 2023-10-08T00:17:13.050 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6831/86612", 2023-10-08T00:17:13.050 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6829/86612", 2023-10-08T00:17:13.050 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:13.050 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:13.050 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:13.050 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:13.050 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:13.050 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:17:13.051 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2023-10-08T00:17:13.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.051 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:17:13.051 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:13.051 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:13.051 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:13.051 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:13.051 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:13.051 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.905615+0000", 2023-10-08T00:17:13.051 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:13.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.052 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:17:13.052 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:13.052 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:13.052 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:13.052 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:13.052 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:13.052 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.880709+0000", 2023-10-08T00:17:13.052 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:13.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.052 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:17:13.052 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:13.053 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:13.053 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:13.053 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:13.053 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:13.053 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.874704+0000", 2023-10-08T00:17:13.053 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:13.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.053 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:17:13.053 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:13.053 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:13.053 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:13.054 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:13.054 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:13.054 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.899604+0000", 2023-10-08T00:17:13.054 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:13.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.054 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:17:13.054 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:13.054 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:13.054 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:13.054 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:13.054 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:13.055 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.844371+0000", 2023-10-08T00:17:13.055 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:13.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.055 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:17:13.055 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:13.055 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:13.055 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:13.055 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:13.055 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:13.055 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.794810+0000", 2023-10-08T00:17:13.055 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:13.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.056 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:17:13.056 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:13.056 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:13.056 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:13.056 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:13.056 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:13.056 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.859474+0000", 2023-10-08T00:17:13.056 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:13.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:13.057 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:17:13.057 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:13.057 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:13.057 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:13.057 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:13.057 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:13.057 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.893130+0000", 2023-10-08T00:17:13.057 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:13.057 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:13.057 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:17:13.057 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2023-10-08T00:17:13.058 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [], 2023-10-08T00:17:13.058 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2023-10-08T00:17:13.058 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2023-10-08T00:17:13.058 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2023-10-08T00:17:13.058 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2023-10-08T00:17:13.058 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2023-10-08T00:17:13.058 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2023-10-08T00:17:13.058 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2023-10-08T00:17:13.058 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2023-10-08T00:17:13.058 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2023-10-08T00:17:13.058 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2023-10-08T00:17:13.059 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2023-10-08T00:17:13.059 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:13.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:13.059 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2023-10-08T00:17:13.059 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2023-10-08T00:17:13.059 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2023-10-08T00:17:13.059 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2023-10-08T00:17:13.059 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2023-10-08T00:17:13.059 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2023-10-08T00:17:13.059 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2023-10-08T00:17:13.059 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2023-10-08T00:17:13.059 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2023-10-08T00:17:13.060 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2023-10-08T00:17:13.060 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2023-10-08T00:17:13.060 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:13.060 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:17:13.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.974+0000 7fcc6bef8700 1 -- 172.21.15.99:0/3258705852 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcc48066f30 msgr2=0x7fcc480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:13.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.974+0000 7fcc6bef8700 1 --2- 172.21.15.99:0/3258705852 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcc48066f30 0x7fcc480693e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fcc6000a940 tx=0x7fcc60015000 comp rx=0 tx=0).stop 2023-10-08T00:17:13.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.974+0000 7fcc6bef8700 1 -- 172.21.15.99:0/3258705852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc6414ba70 msgr2=0x7fcc640b1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:13.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.974+0000 7fcc6bef8700 1 --2- 172.21.15.99:0/3258705852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc6414ba70 0x7fcc640b1c50 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fcc54011b40 tx=0x7fcc540091a0 comp rx=0 tx=0).stop 2023-10-08T00:17:13.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.974+0000 7fcc6bef8700 1 -- 172.21.15.99:0/3258705852 shutdown_connections 2023-10-08T00:17:13.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.974+0000 7fcc6bef8700 1 --2- 172.21.15.99:0/3258705852 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcc48066f30 0x7fcc480693e0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:13.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.974+0000 7fcc6bef8700 1 --2- 172.21.15.99:0/3258705852 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcc50003a60 0x7fcc640b5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:13.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.974+0000 7fcc6bef8700 1 --2- 172.21.15.99:0/3258705852 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcc640b2190 0x7fcc640b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:13.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.974+0000 7fcc6bef8700 1 --2- 172.21.15.99:0/3258705852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc6414ba70 0x7fcc640b1c50 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:13.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.974+0000 7fcc6bef8700 1 -- 172.21.15.99:0/3258705852 >> 172.21.15.99:0/3258705852 conn(0x7fcc64068ce0 msgr2=0x7fcc6414c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:13.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.974+0000 7fcc6bef8700 1 -- 172.21.15.99:0/3258705852 shutdown_connections 2023-10-08T00:17:13.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:12.974+0000 7fcc6bef8700 1 -- 172.21.15.99:0/3258705852 wait complete. 2023-10-08T00:17:13.063 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-08T00:17:18.064 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-08T00:17:18.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.188+0000 7f7468227700 1 Processor -- start 2023-10-08T00:17:18.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.189+0000 7f7468227700 1 -- start start 2023-10-08T00:17:18.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.189+0000 7f7468227700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7460152010 0x7f746015b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:18.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.189+0000 7f7468227700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7460158e50 0x7f74600cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:18.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.189+0000 7f7468227700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f746006c010 con 0x7f74601523e0 2023-10-08T00:17:18.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.189+0000 7f7468227700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f746006bd10 con 0x7f7460152010 2023-10-08T00:17:18.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.189+0000 7f7468227700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f746015c060 con 0x7f7460158e50 2023-10-08T00:17:18.195 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.189+0000 7f7465fc3700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7460152010 0x7f746015b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:18.195 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.189+0000 7f74657c2700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f74601523e0 0x7f746006c4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45102/0 (socket says 172.21.15.99:45102) 2023-10-08T00:17:18.195 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.189+0000 7f74657c2700 1 -- 172.21.15.99:0/3713999604 learned_addr learned my addr 172.21.15.99:0/3713999604 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:17:18.195 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.190+0000 7f7465fc3700 1 -- 172.21.15.99:0/3713999604 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7460158e50 msgr2=0x7f74600cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:18.195 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.190+0000 7f7465fc3700 1 --2- 172.21.15.99:0/3713999604 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7460158e50 0x7f74600cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:18.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.190+0000 7f7465fc3700 1 -- 172.21.15.99:0/3713999604 >> v1:172.21.15.99:6789/0 conn(0x7f74601523e0 legacy=0x7f746006c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:18.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.190+0000 7f7464fc1700 1 -- 172.21.15.99:0/3713999604 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3823374283 0 0) 0x7f746006c010 con 0x7f74601523e0 2023-10-08T00:17:18.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.190+0000 7f7465fc3700 1 -- 172.21.15.99:0/3713999604 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f746015c340 con 0x7f7460152010 2023-10-08T00:17:18.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.190+0000 7f7465fc3700 1 --2- 172.21.15.99:0/3713999604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7460152010 0x7f746015b810 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f745c0098e0 tx=0x7f745c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3a5a38610929261 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:18.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.190+0000 7f7464fc1700 1 -- 172.21.15.99:0/3713999604 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745c009e80 con 0x7f7460152010 2023-10-08T00:17:18.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.190+0000 7f7464fc1700 1 -- 172.21.15.99:0/3713999604 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f745c00b860 con 0x7f7460152010 2023-10-08T00:17:18.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.190+0000 7f7464fc1700 1 -- 172.21.15.99:0/3713999604 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745c00bbe0 con 0x7f7460152010 2023-10-08T00:17:18.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.191+0000 7f7468227700 1 -- 172.21.15.99:0/3713999604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7460152010 msgr2=0x7f746015b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:18.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.191+0000 7f7468227700 1 --2- 172.21.15.99:0/3713999604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7460152010 0x7f746015b810 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f745c0098e0 tx=0x7f745c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:17:18.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.191+0000 7f7468227700 1 -- 172.21.15.99:0/3713999604 shutdown_connections 2023-10-08T00:17:18.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.191+0000 7f7468227700 1 --2- 172.21.15.99:0/3713999604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7460152010 0x7f746015b810 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:18.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.191+0000 7f7468227700 1 --2- 172.21.15.99:0/3713999604 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7460158e50 0x7f74600cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:18.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.191+0000 7f7468227700 1 -- 172.21.15.99:0/3713999604 >> 172.21.15.99:0/3713999604 conn(0x7f7460069870 msgr2=0x7f7460069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:18.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.191+0000 7f7468227700 1 -- 172.21.15.99:0/3713999604 shutdown_connections 2023-10-08T00:17:18.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.191+0000 7f7468227700 1 -- 172.21.15.99:0/3713999604 wait complete. 2023-10-08T00:17:18.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.192+0000 7f7468227700 1 Processor -- start 2023-10-08T00:17:18.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.192+0000 7f7468227700 1 -- start start 2023-10-08T00:17:18.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.192+0000 7f7468227700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7460152010 0x7f74600b4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:18.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.192+0000 7f7468227700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74601523e0 0x7f746014fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:18.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.192+0000 7f7468227700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7460158e50 0x7f74601502e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:18.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.192+0000 7f7468227700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746016c630 con 0x7f74601523e0 2023-10-08T00:17:18.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.192+0000 7f7468227700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746016bab0 con 0x7f7460152010 2023-10-08T00:17:18.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.192+0000 7f7468227700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f746016bd90 con 0x7f7460158e50 2023-10-08T00:17:18.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.193+0000 7f74657c2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74601523e0 0x7f746014fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:18.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.193+0000 7f74657c2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74601523e0 0x7f746014fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53224/0 (socket says 172.21.15.99:53224) 2023-10-08T00:17:18.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.193+0000 7f74657c2700 1 -- 172.21.15.99:0/741702100 learned_addr learned my addr 172.21.15.99:0/741702100 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:18.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.193+0000 7f7465fc3700 1 --2- 172.21.15.99:0/741702100 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7460152010 0x7f74600b4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:18.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.193+0000 7f74667c4700 1 --2- 172.21.15.99:0/741702100 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7460158e50 0x7f74601502e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:18.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.193+0000 7f74657c2700 1 -- 172.21.15.99:0/741702100 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7460158e50 msgr2=0x7f74601502e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:18.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.193+0000 7f74657c2700 1 --2- 172.21.15.99:0/741702100 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7460158e50 0x7f74601502e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:18.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.193+0000 7f74657c2700 1 -- 172.21.15.99:0/741702100 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7460152010 msgr2=0x7f74600b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:18.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.193+0000 7f74657c2700 1 --2- 172.21.15.99:0/741702100 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7460152010 0x7f74600b4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:18.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.193+0000 7f74657c2700 1 -- 172.21.15.99:0/741702100 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f745c009420 con 0x7f74601523e0 2023-10-08T00:17:18.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.193+0000 7f74657c2700 1 --2- 172.21.15.99:0/741702100 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74601523e0 0x7f746014fda0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f745000c7e0 tx=0x7f7450007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:18.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.193+0000 7f7455ffb700 1 -- 172.21.15.99:0/741702100 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7450005b30 con 0x7f74601523e0 2023-10-08T00:17:18.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.193+0000 7f7455ffb700 1 -- 172.21.15.99:0/741702100 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7450005c90 con 0x7f74601523e0 2023-10-08T00:17:18.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.193+0000 7f7468227700 1 -- 172.21.15.99:0/741702100 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f746016c070 con 0x7f74601523e0 2023-10-08T00:17:18.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.193+0000 7f7455ffb700 1 -- 172.21.15.99:0/741702100 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7450007300 con 0x7f74601523e0 2023-10-08T00:17:18.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.193+0000 7f7468227700 1 -- 172.21.15.99:0/741702100 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f746016b6d0 con 0x7f74601523e0 2023-10-08T00:17:18.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.194+0000 7f7468227700 1 -- 172.21.15.99:0/741702100 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74600b23d0 con 0x7f74601523e0 2023-10-08T00:17:18.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.195+0000 7f7455ffb700 1 -- 172.21.15.99:0/741702100 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f745001b400 con 0x7f74601523e0 2023-10-08T00:17:18.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.195+0000 7f7455ffb700 1 --2- 172.21.15.99:0/741702100 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7444066f30 0x7f74440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:18.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.195+0000 7f7455ffb700 1 -- 172.21.15.99:0/741702100 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7450084020 con 0x7f74601523e0 2023-10-08T00:17:18.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.195+0000 7f7455ffb700 1 -- 172.21.15.99:0/741702100 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=142}) v3 -- 0x7f744406ea50 con 0x7f74601523e0 2023-10-08T00:17:18.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.195+0000 7f7465fc3700 1 --2- 172.21.15.99:0/741702100 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7444066f30 0x7f74440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:18.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.196+0000 7f7465fc3700 1 --2- 172.21.15.99:0/741702100 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7444066f30 0x7f74440693e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f745c0091e0 tx=0x7f745c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:18.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.199+0000 7f7455ffb700 1 -- 172.21.15.99:0/741702100 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f74600b23d0 con 0x7f74601523e0 2023-10-08T00:17:18.384 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.383+0000 7f7468227700 1 -- 172.21.15.99:0/741702100 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f74600b23d0 con 0x7f74601523e0 2023-10-08T00:17:18.385 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.384+0000 7f7455ffb700 1 -- 172.21.15.99:0/741702100 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v141) v1 ==== 81+0+32301 (secure 0 0 0) 0x7f74600b23d0 con 0x7f74601523e0 2023-10-08T00:17:18.385 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:17:18.385 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:17:18.386 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 141, 2023-10-08T00:17:18.386 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", 2023-10-08T00:17:18.386 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2023-10-08T00:10:36.303601+0000", 2023-10-08T00:17:18.386 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2023-10-08T00:17:06.114261+0000", 2023-10-08T00:17:18.386 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2023-10-08T00:10:41.709171+0000", 2023-10-08T00:17:18.386 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2023-10-08T00:10:38.690404+0000", 2023-10-08T00:17:18.386 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-08T00:17:18.386 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2023-10-08T00:17:18.386 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2023-10-08T00:17:18.386 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2023-10-08T00:17:18.386 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2023-10-08T00:17:18.387 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2023-10-08T00:17:18.387 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2023-10-08T00:17:18.387 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:17:18.387 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2023-10-08T00:17:18.387 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2023-10-08T00:17:18.387 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-08T00:17:18.387 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-08T00:17:18.387 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2023-10-08T00:17:18.387 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 17, 2023-10-08T00:17:18.387 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2023-10-08T00:17:18.387 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2023-10-08T00:17:18.387 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "jewel", 2023-10-08T00:17:18.388 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "quincy", 2023-10-08T00:17:18.388 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2023-10-08T00:17:18.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.388 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2023-10-08T00:17:18.388 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2023-10-08T00:17:18.388 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:10:42.063344+0000", 2023-10-08T00:17:18.388 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:17:18.388 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:17:18.388 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:17:18.388 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:17:18.388 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:17:18.388 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:17:18.389 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:17:18.389 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:17:18.389 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:17:18.389 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:17:18.389 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:17:18.389 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:17:18.389 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2023-10-08T00:17:18.389 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2023-10-08T00:17:18.389 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2023-10-08T00:17:18.389 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2023-10-08T00:17:18.389 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2023-10-08T00:17:18.389 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:17:18.390 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:17:18.390 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:17:18.390 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:17:18.390 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:17:18.390 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:17:18.390 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:17:18.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.390 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "112", 2023-10-08T00:17:18.390 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:17:18.390 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:17:18.390 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:17:18.391 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:17:18.391 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:17:18.391 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:17:18.391 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:17:18.391 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:17:18.391 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:17:18.391 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:17:18.391 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:17:18.391 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:17:18.391 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:17:18.391 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:17:18.391 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:17:18.392 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:17:18.392 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:17:18.392 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:17:18.392 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:17:18.392 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:17:18.392 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:17:18.392 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:17:18.392 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:17:18.392 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:17:18.392 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:17:18.392 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:17:18.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.393 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:17:18.393 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:17:18.393 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:17:18.393 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:17:18.393 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:17:18.393 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:17:18.393 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:17:18.393 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:17:18.393 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:17:18.393 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:17:18.393 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:17:18.393 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:17:18.394 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2023-10-08T00:17:18.394 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2023-10-08T00:17:18.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.394 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:17:18.394 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2023-10-08T00:17:18.394 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:18.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.394 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 16, 2023-10-08T00:17:18.394 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2023-10-08T00:17:18.394 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:16:48.430607+0000", 2023-10-08T00:17:18.394 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:17:18.395 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:17:18.395 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:17:18.395 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:17:18.395 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:17:18.395 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:17:18.395 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:17:18.395 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:17:18.395 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:17:18.395 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:17:18.395 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:17:18.395 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:17:18.395 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2023-10-08T00:17:18.396 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2023-10-08T00:17:18.396 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2023-10-08T00:17:18.396 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2023-10-08T00:17:18.396 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2023-10-08T00:17:18.396 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:17:18.396 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:17:18.396 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:17:18.396 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:17:18.396 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:17:18.396 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:17:18.396 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:17:18.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.397 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "138", 2023-10-08T00:17:18.397 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:17:18.397 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:17:18.397 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:17:18.397 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:17:18.397 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:17:18.397 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:17:18.397 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:17:18.397 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:17:18.397 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:17:18.397 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:17:18.397 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:17:18.398 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:17:18.398 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:17:18.398 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:17:18.398 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:17:18.398 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:17:18.398 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:17:18.398 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:17:18.398 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:17:18.398 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:17:18.398 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:17:18.398 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:17:18.399 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:17:18.399 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:17:18.399 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:17:18.399 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:17:18.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.399 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:17:18.399 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:17:18.399 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:17:18.399 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:17:18.399 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:17:18.399 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:17:18.399 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:17:18.400 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:17:18.400 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:17:18.400 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:17:18.400 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:17:18.400 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:17:18.400 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2023-10-08T00:17:18.400 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2023-10-08T00:17:18.400 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2023-10-08T00:17:18.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.400 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:17:18.400 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:17:18.400 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2023-10-08T00:17:18.401 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:18.401 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:18.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.401 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 17, 2023-10-08T00:17:18.401 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2023-10-08T00:17:18.401 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:16:49.063424+0000", 2023-10-08T00:17:18.401 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1027, 2023-10-08T00:17:18.401 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full,full_quota", 2023-10-08T00:17:18.401 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:17:18.401 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:17:18.401 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:17:18.402 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:17:18.402 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:17:18.402 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:17:18.402 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:17:18.402 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:17:18.402 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:17:18.402 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:17:18.402 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2023-10-08T00:17:18.402 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2023-10-08T00:17:18.402 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2023-10-08T00:17:18.402 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2023-10-08T00:17:18.403 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2023-10-08T00:17:18.403 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:17:18.403 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:17:18.403 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:17:18.403 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:17:18.403 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:17:18.403 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:17:18.403 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:17:18.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.403 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "141", 2023-10-08T00:17:18.403 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:17:18.403 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:17:18.404 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:17:18.404 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:17:18.404 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:17:18.404 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:17:18.404 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:17:18.404 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:17:18.404 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:17:18.404 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 33554432, 2023-10-08T00:17:18.404 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:17:18.404 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:17:18.404 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:17:18.404 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:17:18.405 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:17:18.405 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:17:18.405 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:17:18.405 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:17:18.405 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:17:18.405 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:17:18.405 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:17:18.405 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:17:18.405 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:17:18.405 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:17:18.405 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:17:18.406 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:17:18.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.406 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:17:18.406 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:17:18.406 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:17:18.406 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:17:18.406 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:17:18.406 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:17:18.406 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:17:18.406 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:17:18.406 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:17:18.406 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:17:18.407 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:17:18.407 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:17:18.407 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2023-10-08T00:17:18.407 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2023-10-08T00:17:18.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.407 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:17:18.407 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:17:18.407 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2023-10-08T00:17:18.407 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:18.407 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:18.407 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:18.407 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:17:18.408 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2023-10-08T00:17:18.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.408 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:17:18.408 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:18.408 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:18.408 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:18.408 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:18.408 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:18.408 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:18.408 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:18.408 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:18.408 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:18.409 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:18.409 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:18.409 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:18.409 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:18.409 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.409 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:18.409 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2023-10-08T00:17:18.409 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:18.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.409 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.409 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:18.410 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2023-10-08T00:17:18.410 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:18.410 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:18.410 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:18.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.410 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:18.410 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:18.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.410 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:18.410 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2023-10-08T00:17:18.410 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:18.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.411 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:18.411 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2023-10-08T00:17:18.411 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:18.411 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:18.411 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:18.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.411 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:18.411 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:18.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.411 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:18.412 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2023-10-08T00:17:18.412 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:18.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.412 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:18.412 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2023-10-08T00:17:18.412 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:18.412 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:18.412 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:18.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.412 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:18.412 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:18.413 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.413 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:18.413 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2023-10-08T00:17:18.413 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:18.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.413 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.413 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:18.413 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2023-10-08T00:17:18.413 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:18.413 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:18.413 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:18.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.414 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/86365", 2023-10-08T00:17:18.414 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/86365", 2023-10-08T00:17:18.414 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/86365", 2023-10-08T00:17:18.414 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/86365", 2023-10-08T00:17:18.414 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:18.414 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:18.414 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:18.414 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:18.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.414 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.415 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:17:18.415 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:18.415 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:18.415 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:18.415 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:18.415 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:18.415 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:18.415 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:18.415 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:18.415 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:18.415 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:18.415 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:18.416 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:18.416 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:18.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.416 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:18.416 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2023-10-08T00:17:18.416 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:18.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.416 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:18.416 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2023-10-08T00:17:18.416 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:18.416 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:18.417 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:18.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.417 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:18.417 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:18.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.417 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:18.417 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2023-10-08T00:17:18.417 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:18.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.417 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:18.418 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2023-10-08T00:17:18.418 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:18.418 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:18.418 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:18.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.418 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:18.418 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:18.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.418 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:18.418 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2023-10-08T00:17:18.418 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:18.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.419 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:18.419 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2023-10-08T00:17:18.419 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:18.419 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:18.419 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:18.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.421 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:18.422 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:18.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.422 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:18.422 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2023-10-08T00:17:18.422 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:18.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.422 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:18.422 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2023-10-08T00:17:18.422 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:18.422 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:18.423 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:18.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.423 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/86366", 2023-10-08T00:17:18.423 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/86366", 2023-10-08T00:17:18.423 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/86366", 2023-10-08T00:17:18.423 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/86366", 2023-10-08T00:17:18.423 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:18.423 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:18.423 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:18.423 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:18.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.424 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:17:18.424 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:18.424 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:18.424 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:18.424 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:18.424 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:18.424 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:18.424 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:18.424 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:18.424 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:18.424 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:18.424 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:18.425 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:18.425 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:18.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.425 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:18.425 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2023-10-08T00:17:18.425 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:18.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.425 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:18.425 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2023-10-08T00:17:18.425 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:18.425 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:18.426 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:18.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.426 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:18.426 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:18.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.426 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:18.426 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2023-10-08T00:17:18.426 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:18.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.426 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:18.426 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2023-10-08T00:17:18.427 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:18.427 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:18.427 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:18.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.427 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:18.427 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:18.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.427 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:18.427 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2023-10-08T00:17:18.427 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:18.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.428 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:18.428 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2023-10-08T00:17:18.428 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:18.428 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:18.428 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:18.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.428 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:18.428 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:18.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.428 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:18.428 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2023-10-08T00:17:18.428 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:18.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.429 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:18.429 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2023-10-08T00:17:18.429 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:18.429 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:18.429 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:18.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.429 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6817/86382", 2023-10-08T00:17:18.429 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/86382", 2023-10-08T00:17:18.429 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/86382", 2023-10-08T00:17:18.430 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/86382", 2023-10-08T00:17:18.430 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:18.430 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:18.430 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:18.430 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:18.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.430 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:17:18.430 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:18.430 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:18.430 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:18.430 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:18.431 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:18.431 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:18.431 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:18.431 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:18.431 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:18.431 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:18.431 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:18.431 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:18.431 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:18.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.431 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:18.431 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2023-10-08T00:17:18.432 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:18.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.432 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:18.432 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2023-10-08T00:17:18.432 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:18.432 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:18.432 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:18.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.432 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:18.432 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:18.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.433 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:18.433 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2023-10-08T00:17:18.433 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:18.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.433 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:18.433 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2023-10-08T00:17:18.433 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:18.433 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:18.433 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:18.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.433 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:18.434 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:18.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.434 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:18.434 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2023-10-08T00:17:18.434 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:18.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.434 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:18.434 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2023-10-08T00:17:18.434 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:18.434 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:18.435 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:18.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.435 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:18.435 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:18.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.435 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:18.435 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2023-10-08T00:17:18.435 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:18.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.435 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:18.435 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2023-10-08T00:17:18.436 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:18.436 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:18.436 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:18.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.436 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6820/86393", 2023-10-08T00:17:18.436 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/86393", 2023-10-08T00:17:18.436 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/86393", 2023-10-08T00:17:18.436 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/86393", 2023-10-08T00:17:18.436 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:18.436 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:18.436 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:18.436 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:18.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.437 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:17:18.437 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:18.437 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:18.437 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:18.437 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:18.437 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:18.437 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:18.437 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:18.437 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:18.437 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:18.438 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:18.438 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:18.438 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:18.438 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:18.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.438 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:18.438 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6800", 2023-10-08T00:17:18.438 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:18.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.438 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:18.438 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6801", 2023-10-08T00:17:18.439 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:18.439 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:18.439 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:18.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.439 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:18.439 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:18.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.439 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:18.439 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6802", 2023-10-08T00:17:18.439 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:18.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.440 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:18.440 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6803", 2023-10-08T00:17:18.440 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:18.440 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:18.440 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:18.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.440 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:18.440 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:18.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.440 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:18.440 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6806", 2023-10-08T00:17:18.440 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:18.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.441 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:18.441 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6807", 2023-10-08T00:17:18.441 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:18.441 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:18.441 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:18.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.441 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:18.441 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:18.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.441 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:18.442 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6804", 2023-10-08T00:17:18.442 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:18.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.442 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:18.442 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6805", 2023-10-08T00:17:18.442 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:18.442 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:18.442 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:18.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.442 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6801/86570", 2023-10-08T00:17:18.443 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6803/86570", 2023-10-08T00:17:18.443 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6807/86570", 2023-10-08T00:17:18.443 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6805/86570", 2023-10-08T00:17:18.443 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:18.443 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:18.443 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:18.443 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:18.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.443 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:17:18.443 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:18.443 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:18.444 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:18.444 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:18.444 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:18.444 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:18.444 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:18.444 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:18.444 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:18.444 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:18.444 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:18.444 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:18.444 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:18.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.445 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:18.445 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6816", 2023-10-08T00:17:18.445 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:18.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.445 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:18.445 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6818", 2023-10-08T00:17:18.445 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:18.445 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:18.445 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:18.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.445 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:18.446 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:18.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.446 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:18.446 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6820", 2023-10-08T00:17:18.446 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:18.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.446 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:18.446 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6821", 2023-10-08T00:17:18.446 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:18.446 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:18.447 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:18.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.447 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:18.447 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:18.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.447 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:18.447 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6826", 2023-10-08T00:17:18.447 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:18.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.447 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:18.447 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6828", 2023-10-08T00:17:18.448 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:18.448 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:18.448 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:18.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.448 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:18.448 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:18.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.448 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:18.448 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6823", 2023-10-08T00:17:18.448 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:18.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.449 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:18.449 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6825", 2023-10-08T00:17:18.449 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:18.449 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:18.449 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:18.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.449 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6818/86608", 2023-10-08T00:17:18.449 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6821/86608", 2023-10-08T00:17:18.449 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6828/86608", 2023-10-08T00:17:18.449 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6825/86608", 2023-10-08T00:17:18.449 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:18.449 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:18.450 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:18.450 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:18.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.450 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:17:18.450 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:18.450 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:18.450 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:18.450 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:18.450 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:18.450 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:18.450 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:18.451 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:18.451 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:18.451 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:18.451 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:18.451 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:18.451 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:18.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.451 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:18.451 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6808", 2023-10-08T00:17:18.451 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:18.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.452 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:18.452 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6809", 2023-10-08T00:17:18.452 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:18.452 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:18.452 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:18.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.452 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:18.452 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:18.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.452 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:18.452 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6810", 2023-10-08T00:17:18.453 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:18.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.453 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:18.453 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6811", 2023-10-08T00:17:18.453 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:18.453 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:18.453 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:18.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.453 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:18.453 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:18.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.454 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:18.454 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6814", 2023-10-08T00:17:18.454 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:18.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.454 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:18.454 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6815", 2023-10-08T00:17:18.454 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:18.454 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:18.454 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:18.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.455 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:18.455 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:18.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.455 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:18.455 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6812", 2023-10-08T00:17:18.455 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:18.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.455 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:18.455 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6813", 2023-10-08T00:17:18.455 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:18.455 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:18.456 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:18.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.456 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6809/86571", 2023-10-08T00:17:18.456 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6811/86571", 2023-10-08T00:17:18.456 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6815/86571", 2023-10-08T00:17:18.456 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6813/86571", 2023-10-08T00:17:18.456 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:18.456 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:18.456 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:18.456 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:18.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.457 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:17:18.457 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:18.457 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:18.457 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:18.457 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:18.457 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:18.457 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:18.457 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:18.457 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:18.457 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:18.457 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:18.457 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:18.458 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:18.458 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:18.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.458 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:18.458 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6817", 2023-10-08T00:17:18.458 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:18.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.458 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:18.458 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6819", 2023-10-08T00:17:18.458 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:18.458 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:18.459 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:18.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.459 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:18.459 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:18.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.459 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:18.459 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6822", 2023-10-08T00:17:18.459 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:18.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.459 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:18.459 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6824", 2023-10-08T00:17:18.460 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:18.460 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:18.460 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:18.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.460 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:18.460 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:18.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.460 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:18.460 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6830", 2023-10-08T00:17:18.460 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:18.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.461 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:18.461 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6831", 2023-10-08T00:17:18.461 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:18.461 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:18.461 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:18.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.461 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:18.461 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:18.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.461 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:18.461 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6827", 2023-10-08T00:17:18.461 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:18.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.462 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:18.462 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6829", 2023-10-08T00:17:18.462 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:18.462 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:18.462 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:18.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.462 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6819/86612", 2023-10-08T00:17:18.462 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6824/86612", 2023-10-08T00:17:18.462 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6831/86612", 2023-10-08T00:17:18.463 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6829/86612", 2023-10-08T00:17:18.463 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:18.463 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:18.463 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:18.463 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:18.463 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:18.463 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:17:18.463 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2023-10-08T00:17:18.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.463 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:17:18.463 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:18.463 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:18.464 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:18.464 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:18.464 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:18.464 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.905615+0000", 2023-10-08T00:17:18.464 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:18.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.464 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:17:18.464 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:18.464 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:18.464 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:18.464 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:18.465 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:18.465 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.880709+0000", 2023-10-08T00:17:18.465 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:18.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.465 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:17:18.465 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:18.465 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:18.465 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:18.465 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:18.465 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:18.465 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.874704+0000", 2023-10-08T00:17:18.466 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:18.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.466 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:17:18.466 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:18.466 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:18.466 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:18.466 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:18.466 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:18.466 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.899604+0000", 2023-10-08T00:17:18.466 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:18.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.467 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:17:18.467 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:18.467 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:18.467 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:18.467 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:18.467 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:18.467 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.844371+0000", 2023-10-08T00:17:18.467 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:18.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.467 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:17:18.468 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:18.468 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:18.468 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:18.468 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:18.468 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:18.468 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.794810+0000", 2023-10-08T00:17:18.468 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:18.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.468 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:17:18.468 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:18.468 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:18.469 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:18.469 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:18.469 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:18.469 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.859474+0000", 2023-10-08T00:17:18.469 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:18.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:18.469 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:17:18.469 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:18.469 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:18.469 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:18.469 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:18.470 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:18.470 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.893130+0000", 2023-10-08T00:17:18.470 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:18.470 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:18.470 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:17:18.470 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2023-10-08T00:17:18.470 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [], 2023-10-08T00:17:18.470 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2023-10-08T00:17:18.470 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2023-10-08T00:17:18.470 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2023-10-08T00:17:18.470 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2023-10-08T00:17:18.470 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2023-10-08T00:17:18.471 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2023-10-08T00:17:18.471 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2023-10-08T00:17:18.471 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2023-10-08T00:17:18.471 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2023-10-08T00:17:18.471 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2023-10-08T00:17:18.471 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2023-10-08T00:17:18.471 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:18.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:18.471 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2023-10-08T00:17:18.471 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2023-10-08T00:17:18.471 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2023-10-08T00:17:18.471 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2023-10-08T00:17:18.472 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2023-10-08T00:17:18.472 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2023-10-08T00:17:18.472 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2023-10-08T00:17:18.472 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2023-10-08T00:17:18.472 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2023-10-08T00:17:18.472 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2023-10-08T00:17:18.472 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2023-10-08T00:17:18.472 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:18.472 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:17:18.474 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.386+0000 7f7468227700 1 -- 172.21.15.99:0/741702100 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7444066f30 msgr2=0x7f74440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:18.474 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.386+0000 7f7468227700 1 --2- 172.21.15.99:0/741702100 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7444066f30 0x7f74440693e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f745c0091e0 tx=0x7f745c017000 comp rx=0 tx=0).stop 2023-10-08T00:17:18.474 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.386+0000 7f7468227700 1 -- 172.21.15.99:0/741702100 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74601523e0 msgr2=0x7f746014fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:18.474 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.386+0000 7f7468227700 1 --2- 172.21.15.99:0/741702100 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74601523e0 0x7f746014fda0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f745000c7e0 tx=0x7f7450007df0 comp rx=0 tx=0).stop 2023-10-08T00:17:18.474 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.386+0000 7f7468227700 1 -- 172.21.15.99:0/741702100 shutdown_connections 2023-10-08T00:17:18.474 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.387+0000 7f7468227700 1 --2- 172.21.15.99:0/741702100 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7444066f30 0x7f74440693e0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:18.474 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.387+0000 7f7468227700 1 --2- 172.21.15.99:0/741702100 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7460152010 0x7f74600b4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:18.474 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.387+0000 7f7468227700 1 --2- 172.21.15.99:0/741702100 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7460158e50 0x7f74601502e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:18.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.387+0000 7f7468227700 1 --2- 172.21.15.99:0/741702100 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74601523e0 0x7f746014fda0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:18.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.387+0000 7f7468227700 1 -- 172.21.15.99:0/741702100 >> 172.21.15.99:0/741702100 conn(0x7f7460069870 msgr2=0x7f7460158210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:18.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.387+0000 7f7468227700 1 -- 172.21.15.99:0/741702100 shutdown_connections 2023-10-08T00:17:18.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:18.387+0000 7f7468227700 1 -- 172.21.15.99:0/741702100 wait complete. 2023-10-08T00:17:18.475 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-08T00:17:23.443 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:17:23.446 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:17:23.476 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-08T00:17:23.609 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.608+0000 7f5d471f0700 1 Processor -- start 2023-10-08T00:17:23.609 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.608+0000 7f5d471f0700 1 -- start start 2023-10-08T00:17:23.611 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.609+0000 7f5d471f0700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5d4006ab90 0x7f5d400cb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:23.611 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.609+0000 7f5d471f0700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f5d4015abb0 0x7f5d4015afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:23.611 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.609+0000 7f5d471f0700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d400cac20 con 0x7f5d400cc710 2023-10-08T00:17:23.611 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.609+0000 7f5d471f0700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d400640c0 con 0x7f5d4006ab90 2023-10-08T00:17:23.612 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.609+0000 7f5d471f0700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d400cb4c0 con 0x7f5d4015abb0 2023-10-08T00:17:23.612 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.609+0000 7f5d461ee700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5d4006ab90 0x7f5d400cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:23.612 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.609+0000 7f5d459ed700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f5d400cc710 0x7f5d4006af60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50916/0 (socket says 172.21.15.99:50916) 2023-10-08T00:17:23.612 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.609+0000 7f5d459ed700 1 -- 172.21.15.99:0/3644738682 learned_addr learned my addr 172.21.15.99:0/3644738682 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:17:23.612 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.610+0000 7f5d461ee700 1 -- 172.21.15.99:0/3644738682 >> v2:172.21.15.191:3300/0 conn(0x7f5d4015abb0 msgr2=0x7f5d4015afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:23.612 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.610+0000 7f5d461ee700 1 --2- 172.21.15.99:0/3644738682 >> v2:172.21.15.191:3300/0 conn(0x7f5d4015abb0 0x7f5d4015afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:23.612 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.610+0000 7f5d461ee700 1 -- 172.21.15.99:0/3644738682 >> v1:172.21.15.99:6789/0 conn(0x7f5d400cc710 legacy=0x7f5d4006af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:23.612 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.610+0000 7f5d461ee700 1 -- 172.21.15.99:0/3644738682 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d400ccc10 con 0x7f5d4006ab90 2023-10-08T00:17:23.612 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.610+0000 7f5d451ec700 1 -- 172.21.15.99:0/3644738682 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 440760337 0 0) 0x7f5d400cac20 con 0x7f5d400cc710 2023-10-08T00:17:23.612 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.610+0000 7f5d461ee700 1 --2- 172.21.15.99:0/3644738682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d4006ab90 0x7f5d400cb950 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f5d400caba0 tx=0x7f5d3c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eece6bdc28e088af server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:23.612 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.610+0000 7f5d451ec700 1 -- 172.21.15.99:0/3644738682 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c00b6b0 con 0x7f5d4006ab90 2023-10-08T00:17:23.612 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.610+0000 7f5d451ec700 1 -- 172.21.15.99:0/3644738682 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d3c00b810 con 0x7f5d4006ab90 2023-10-08T00:17:23.613 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.610+0000 7f5d451ec700 1 -- 172.21.15.99:0/3644738682 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c00bbb0 con 0x7f5d4006ab90 2023-10-08T00:17:23.613 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.610+0000 7f5d471f0700 1 -- 172.21.15.99:0/3644738682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d4006ab90 msgr2=0x7f5d400cb950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:23.613 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.610+0000 7f5d471f0700 1 --2- 172.21.15.99:0/3644738682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d4006ab90 0x7f5d400cb950 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f5d400caba0 tx=0x7f5d3c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:17:23.613 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.611+0000 7f5d471f0700 1 -- 172.21.15.99:0/3644738682 shutdown_connections 2023-10-08T00:17:23.613 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.611+0000 7f5d471f0700 1 --2- 172.21.15.99:0/3644738682 >> v2:172.21.15.191:3300/0 conn(0x7f5d4015abb0 0x7f5d4015afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:23.613 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.611+0000 7f5d471f0700 1 --2- 172.21.15.99:0/3644738682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d4006ab90 0x7f5d400cb950 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:23.613 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.611+0000 7f5d471f0700 1 -- 172.21.15.99:0/3644738682 >> 172.21.15.99:0/3644738682 conn(0x7f5d400cd050 msgr2=0x7f5d400cd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:23.613 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.611+0000 7f5d471f0700 1 -- 172.21.15.99:0/3644738682 shutdown_connections 2023-10-08T00:17:23.613 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.611+0000 7f5d471f0700 1 -- 172.21.15.99:0/3644738682 wait complete. 2023-10-08T00:17:23.617 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.612+0000 7f5d471f0700 1 Processor -- start 2023-10-08T00:17:23.618 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.612+0000 7f5d471f0700 1 -- start start 2023-10-08T00:17:23.618 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.612+0000 7f5d471f0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5d4006ab90 0x7f5d400af530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:23.618 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.612+0000 7f5d471f0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d400cc710 0x7f5d400afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:23.618 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.612+0000 7f5d471f0700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d4015abb0 0x7f5d4015b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:23.618 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.612+0000 7f5d471f0700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d400b1b90 con 0x7f5d400cc710 2023-10-08T00:17:23.618 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.612+0000 7f5d471f0700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d400b1840 con 0x7f5d4015abb0 2023-10-08T00:17:23.618 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.613+0000 7f5d471f0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d400b0620 con 0x7f5d4006ab90 2023-10-08T00:17:23.618 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.613+0000 7f5d459ed700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d400cc710 0x7f5d400afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:23.618 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.613+0000 7f5d459ed700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d400cc710 0x7f5d400afdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44930/0 (socket says 172.21.15.99:44930) 2023-10-08T00:17:23.619 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.613+0000 7f5d459ed700 1 -- 172.21.15.99:0/2232578975 learned_addr learned my addr 172.21.15.99:0/2232578975 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:23.619 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.613+0000 7f5d461ee700 1 --2- 172.21.15.99:0/2232578975 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5d4006ab90 0x7f5d400af530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:23.619 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.613+0000 7f5d459ed700 1 -- 172.21.15.99:0/2232578975 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5d4006ab90 msgr2=0x7f5d400af530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:23.619 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.613+0000 7f5d469ef700 1 --2- 172.21.15.99:0/2232578975 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d4015abb0 0x7f5d4015b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:23.619 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.613+0000 7f5d459ed700 1 --2- 172.21.15.99:0/2232578975 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5d4006ab90 0x7f5d400af530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:23.619 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.613+0000 7f5d459ed700 1 -- 172.21.15.99:0/2232578975 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d4015abb0 msgr2=0x7f5d4015b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:23.619 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.613+0000 7f5d459ed700 1 --2- 172.21.15.99:0/2232578975 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d4015abb0 0x7f5d4015b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:23.619 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.613+0000 7f5d459ed700 1 -- 172.21.15.99:0/2232578975 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d3c009420 con 0x7f5d400cc710 2023-10-08T00:17:23.619 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.613+0000 7f5d459ed700 1 --2- 172.21.15.99:0/2232578975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d400cc710 0x7f5d400afdf0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f5d3000c810 tx=0x7f5d30007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:23.619 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.613+0000 7f5d367fc700 1 -- 172.21.15.99:0/2232578975 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d30005c00 con 0x7f5d400cc710 2023-10-08T00:17:23.619 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.613+0000 7f5d367fc700 1 -- 172.21.15.99:0/2232578975 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d30005d60 con 0x7f5d400cc710 2023-10-08T00:17:23.620 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.613+0000 7f5d471f0700 1 -- 172.21.15.99:0/2232578975 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d400b0900 con 0x7f5d400cc710 2023-10-08T00:17:23.620 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.613+0000 7f5d367fc700 1 -- 172.21.15.99:0/2232578975 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d30007320 con 0x7f5d400cc710 2023-10-08T00:17:23.620 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.613+0000 7f5d471f0700 1 -- 172.21.15.99:0/2232578975 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d400b0330 con 0x7f5d400cc710 2023-10-08T00:17:23.620 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.614+0000 7f5d471f0700 1 -- 172.21.15.99:0/2232578975 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d400b1d40 con 0x7f5d400cc710 2023-10-08T00:17:23.620 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.615+0000 7f5d367fc700 1 -- 172.21.15.99:0/2232578975 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5d3001b400 con 0x7f5d400cc710 2023-10-08T00:17:23.620 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.615+0000 7f5d367fc700 1 --2- 172.21.15.99:0/2232578975 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5d24066f30 0x7f5d240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:23.620 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.615+0000 7f5d367fc700 1 -- 172.21.15.99:0/2232578975 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5d30083ff0 con 0x7f5d400cc710 2023-10-08T00:17:23.620 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.615+0000 7f5d461ee700 1 --2- 172.21.15.99:0/2232578975 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5d24066f30 0x7f5d240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:23.620 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.616+0000 7f5d461ee700 1 --2- 172.21.15.99:0/2232578975 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5d24066f30 0x7f5d240693e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c0093f0 tx=0x7f5d3c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:23.621 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.619+0000 7f5d367fc700 1 -- 172.21.15.99:0/2232578975 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5d400b1d40 con 0x7f5d400cc710 2023-10-08T00:17:23.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.801+0000 7f5d471f0700 1 -- 172.21.15.99:0/2232578975 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f5d400b1cd0 con 0x7f5d400cc710 2023-10-08T00:17:23.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.802+0000 7f5d367fc700 1 -- 172.21.15.99:0/2232578975 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v142) v1 ==== 81+0+32282 (secure 0 0 0) 0x7f5d300512e0 con 0x7f5d400cc710 2023-10-08T00:17:23.803 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:17:23.803 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:17:23.803 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 142, 2023-10-08T00:17:23.803 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", 2023-10-08T00:17:23.803 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2023-10-08T00:10:36.303601+0000", 2023-10-08T00:17:23.804 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2023-10-08T00:17:21.117117+0000", 2023-10-08T00:17:23.804 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2023-10-08T00:10:41.709171+0000", 2023-10-08T00:17:23.804 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2023-10-08T00:10:38.690404+0000", 2023-10-08T00:17:23.804 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-08T00:17:23.804 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2023-10-08T00:17:23.804 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2023-10-08T00:17:23.805 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2023-10-08T00:17:23.805 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2023-10-08T00:17:23.805 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2023-10-08T00:17:23.805 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2023-10-08T00:17:23.805 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:17:23.805 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2023-10-08T00:17:23.805 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2023-10-08T00:17:23.805 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-08T00:17:23.805 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-08T00:17:23.805 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2023-10-08T00:17:23.805 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 17, 2023-10-08T00:17:23.806 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2023-10-08T00:17:23.806 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2023-10-08T00:17:23.806 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "jewel", 2023-10-08T00:17:23.806 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "quincy", 2023-10-08T00:17:23.806 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2023-10-08T00:17:23.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.806 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2023-10-08T00:17:23.806 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2023-10-08T00:17:23.806 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:10:42.063344+0000", 2023-10-08T00:17:23.806 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:17:23.806 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:17:23.806 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:17:23.807 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:17:23.807 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:17:23.807 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:17:23.807 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:17:23.807 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:17:23.807 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:17:23.807 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:17:23.807 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:17:23.808 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:17:23.808 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2023-10-08T00:17:23.808 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2023-10-08T00:17:23.808 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2023-10-08T00:17:23.808 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2023-10-08T00:17:23.808 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2023-10-08T00:17:23.808 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:17:23.808 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:17:23.808 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:17:23.808 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:17:23.808 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:17:23.809 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:17:23.809 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:17:23.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.809 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "112", 2023-10-08T00:17:23.809 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:17:23.809 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:17:23.809 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:17:23.809 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:17:23.809 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:17:23.809 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:17:23.809 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:17:23.809 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:17:23.810 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:17:23.810 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:17:23.810 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:17:23.810 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:17:23.810 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:17:23.810 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:17:23.810 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:17:23.810 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:17:23.810 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:17:23.811 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:17:23.811 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:17:23.811 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:17:23.811 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:17:23.811 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:17:23.811 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:17:23.811 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:17:23.811 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:17:23.811 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:17:23.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.811 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:17:23.811 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:17:23.812 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:17:23.812 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:17:23.812 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:17:23.812 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:17:23.812 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:17:23.812 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:17:23.812 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:17:23.812 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:17:23.812 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:17:23.812 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:17:23.812 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2023-10-08T00:17:23.812 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2023-10-08T00:17:23.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.813 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:17:23.813 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2023-10-08T00:17:23.813 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:23.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.813 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 16, 2023-10-08T00:17:23.813 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2023-10-08T00:17:23.813 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:16:48.430607+0000", 2023-10-08T00:17:23.813 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:17:23.813 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:17:23.813 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:17:23.814 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:17:23.814 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:17:23.814 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:17:23.814 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:17:23.814 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:17:23.814 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:17:23.814 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:17:23.814 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:17:23.814 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:17:23.814 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2023-10-08T00:17:23.815 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2023-10-08T00:17:23.815 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2023-10-08T00:17:23.815 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2023-10-08T00:17:23.815 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2023-10-08T00:17:23.815 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:17:23.815 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:17:23.815 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:17:23.816 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:17:23.816 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:17:23.816 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:17:23.816 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:17:23.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.816 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "138", 2023-10-08T00:17:23.816 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:17:23.816 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:17:23.816 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:17:23.816 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:17:23.816 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:17:23.817 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:17:23.817 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:17:23.817 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:17:23.817 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:17:23.817 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:17:23.817 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:17:23.817 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:17:23.817 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:17:23.817 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:17:23.817 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:17:23.817 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:17:23.817 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:17:23.817 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:17:23.818 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:17:23.818 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:17:23.818 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:17:23.818 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:17:23.818 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:17:23.818 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:17:23.819 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:17:23.819 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:17:23.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.819 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:17:23.819 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:17:23.819 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:17:23.819 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:17:23.819 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:17:23.819 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:17:23.819 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:17:23.819 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:17:23.819 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:17:23.820 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:17:23.820 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:17:23.820 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:17:23.820 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2023-10-08T00:17:23.820 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2023-10-08T00:17:23.820 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2023-10-08T00:17:23.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.820 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:17:23.820 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:17:23.820 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2023-10-08T00:17:23.820 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:23.820 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:23.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.821 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 17, 2023-10-08T00:17:23.821 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2023-10-08T00:17:23.821 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:16:49.063424+0000", 2023-10-08T00:17:23.821 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:17:23.821 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:17:23.821 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:17:23.821 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:17:23.821 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:17:23.821 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:17:23.821 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:17:23.821 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:17:23.822 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:17:23.822 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:17:23.822 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:17:23.822 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:17:23.822 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2023-10-08T00:17:23.822 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2023-10-08T00:17:23.822 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2023-10-08T00:17:23.822 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2023-10-08T00:17:23.822 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2023-10-08T00:17:23.822 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:17:23.822 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:17:23.822 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:17:23.823 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:17:23.823 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:17:23.823 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:17:23.823 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:17:23.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.823 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "142", 2023-10-08T00:17:23.823 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:17:23.823 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:17:23.823 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:17:23.823 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:17:23.823 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:17:23.823 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:17:23.823 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:17:23.824 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:17:23.824 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:17:23.824 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 33554432, 2023-10-08T00:17:23.824 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:17:23.824 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:17:23.824 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:17:23.824 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:17:23.824 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:17:23.824 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:17:23.824 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:17:23.824 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:17:23.825 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:17:23.825 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:17:23.825 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:17:23.825 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:17:23.826 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:17:23.826 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:17:23.826 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:17:23.826 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:17:23.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.826 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:17:23.826 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:17:23.826 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:17:23.827 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:17:23.827 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:17:23.827 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:17:23.827 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:17:23.827 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:17:23.827 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:17:23.827 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:17:23.827 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:17:23.827 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:17:23.827 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2023-10-08T00:17:23.827 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2023-10-08T00:17:23.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.828 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:17:23.828 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:17:23.828 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2023-10-08T00:17:23.828 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:23.828 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:23.828 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:23.828 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:17:23.828 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2023-10-08T00:17:23.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.828 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:17:23.828 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:23.828 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:23.829 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:23.829 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:23.829 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:23.829 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:23.829 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:23.829 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:23.829 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:23.829 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:23.829 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:23.829 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:23.829 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:23.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.830 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:23.830 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2023-10-08T00:17:23.830 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:23.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.830 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:23.830 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2023-10-08T00:17:23.830 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:23.830 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:23.830 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:23.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.831 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:23.831 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:23.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.831 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:23.831 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2023-10-08T00:17:23.831 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:23.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.831 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:23.831 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2023-10-08T00:17:23.831 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:23.831 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:23.832 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:23.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.832 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:23.832 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:23.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.832 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:23.832 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2023-10-08T00:17:23.832 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:23.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.832 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:23.833 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2023-10-08T00:17:23.833 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:23.833 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:23.833 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:23.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.833 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:23.833 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:23.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.833 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:23.833 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2023-10-08T00:17:23.833 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:23.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.834 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:23.834 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2023-10-08T00:17:23.834 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:23.834 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:23.834 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:23.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.834 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/86365", 2023-10-08T00:17:23.834 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/86365", 2023-10-08T00:17:23.834 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/86365", 2023-10-08T00:17:23.834 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/86365", 2023-10-08T00:17:23.834 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:23.835 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:23.835 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:23.835 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:23.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.835 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:17:23.835 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:23.835 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:23.835 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:23.835 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:23.835 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:23.835 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:23.836 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:23.836 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:23.836 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:23.836 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:23.836 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:23.836 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:23.836 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:23.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.836 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:23.836 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2023-10-08T00:17:23.836 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:23.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.837 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:23.837 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2023-10-08T00:17:23.837 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:23.837 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:23.837 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:23.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.837 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:23.837 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:23.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.837 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:23.838 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2023-10-08T00:17:23.838 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:23.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.838 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:23.838 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2023-10-08T00:17:23.838 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:23.838 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:23.838 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:23.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.838 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:23.839 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:23.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.839 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:23.839 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2023-10-08T00:17:23.839 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:23.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.839 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:23.839 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2023-10-08T00:17:23.839 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:23.839 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:23.839 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:23.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.840 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:23.840 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:23.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.840 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:23.840 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2023-10-08T00:17:23.840 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:23.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.840 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:23.840 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2023-10-08T00:17:23.840 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:23.840 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:23.841 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:23.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.841 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/86366", 2023-10-08T00:17:23.841 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/86366", 2023-10-08T00:17:23.841 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/86366", 2023-10-08T00:17:23.841 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/86366", 2023-10-08T00:17:23.841 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:23.841 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:23.841 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:23.841 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:23.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.842 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:17:23.842 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:23.842 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:23.842 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:23.842 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:23.842 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:23.842 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:23.842 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:23.842 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:23.842 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:23.842 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:23.843 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:23.843 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:23.843 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:23.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.843 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:23.843 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2023-10-08T00:17:23.843 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:23.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.844 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:23.844 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2023-10-08T00:17:23.844 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:23.844 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:23.844 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:23.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.844 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:23.844 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:23.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.844 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:23.844 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2023-10-08T00:17:23.845 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:23.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.845 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:23.845 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2023-10-08T00:17:23.845 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:23.845 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:23.845 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:23.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.845 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:23.845 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:23.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.846 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:23.846 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2023-10-08T00:17:23.846 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:23.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.846 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:23.846 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2023-10-08T00:17:23.846 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:23.847 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:23.847 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:23.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.847 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:23.847 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:23.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.847 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:23.847 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2023-10-08T00:17:23.847 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:23.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.848 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:23.848 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2023-10-08T00:17:23.848 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:23.848 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:23.848 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:23.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.848 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6817/86382", 2023-10-08T00:17:23.848 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/86382", 2023-10-08T00:17:23.848 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/86382", 2023-10-08T00:17:23.848 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/86382", 2023-10-08T00:17:23.848 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:23.849 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:23.849 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:23.849 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:23.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.849 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:17:23.849 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:23.849 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:23.849 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:23.849 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:23.849 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:23.850 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:23.850 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:23.850 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:23.850 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:23.850 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:23.850 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:23.850 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:23.850 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:23.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.850 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:23.850 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2023-10-08T00:17:23.850 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:23.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.851 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:23.851 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2023-10-08T00:17:23.851 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:23.851 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:23.851 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:23.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.851 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:23.851 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:23.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.852 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:23.852 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2023-10-08T00:17:23.852 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:23.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.852 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:23.852 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2023-10-08T00:17:23.852 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:23.852 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:23.852 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:23.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.852 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:23.853 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:23.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.853 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:23.853 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2023-10-08T00:17:23.853 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:23.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.853 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:23.853 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2023-10-08T00:17:23.853 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:23.853 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:23.854 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:23.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.854 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:23.854 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:23.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.854 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:23.854 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2023-10-08T00:17:23.854 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:23.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.854 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:23.854 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2023-10-08T00:17:23.855 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:23.855 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:23.855 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:23.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.855 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6820/86393", 2023-10-08T00:17:23.855 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/86393", 2023-10-08T00:17:23.855 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/86393", 2023-10-08T00:17:23.855 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/86393", 2023-10-08T00:17:23.855 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:23.855 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:23.855 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:23.855 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:23.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.856 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:17:23.856 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:23.856 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:23.856 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:23.856 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:23.856 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:23.856 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:23.856 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:23.856 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:23.857 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:23.857 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:23.857 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:23.857 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:23.857 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:23.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.857 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:23.857 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6800", 2023-10-08T00:17:23.857 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:23.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.857 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:23.858 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6801", 2023-10-08T00:17:23.858 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:23.858 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:23.858 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:23.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.858 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:23.858 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:23.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.858 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:23.858 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6802", 2023-10-08T00:17:23.858 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:23.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.859 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:23.859 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6803", 2023-10-08T00:17:23.859 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:23.859 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:23.859 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:23.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.859 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:23.859 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:23.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.859 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:23.860 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6806", 2023-10-08T00:17:23.860 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:23.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.860 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:23.860 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6807", 2023-10-08T00:17:23.860 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:23.860 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:23.860 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:23.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.860 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:23.860 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:23.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.861 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:23.861 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6804", 2023-10-08T00:17:23.861 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:23.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.861 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:23.861 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6805", 2023-10-08T00:17:23.861 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:23.861 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:23.861 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:23.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.862 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6801/86570", 2023-10-08T00:17:23.862 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6803/86570", 2023-10-08T00:17:23.862 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6807/86570", 2023-10-08T00:17:23.862 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6805/86570", 2023-10-08T00:17:23.862 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:23.862 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:23.862 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:23.862 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:23.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.862 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:17:23.863 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:23.863 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:23.863 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:23.863 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:23.863 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:23.863 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:23.863 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:23.863 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:23.863 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:23.863 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:23.863 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:23.864 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:23.864 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:23.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.864 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:23.864 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6816", 2023-10-08T00:17:23.864 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:23.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.864 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:23.864 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6818", 2023-10-08T00:17:23.864 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:23.864 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:23.865 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:23.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.865 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:23.865 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:23.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.865 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:23.865 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6820", 2023-10-08T00:17:23.865 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:23.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.865 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:23.865 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6821", 2023-10-08T00:17:23.866 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:23.866 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:23.866 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:23.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.866 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:23.866 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:23.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.866 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:23.866 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6826", 2023-10-08T00:17:23.866 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:23.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.867 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:23.867 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6828", 2023-10-08T00:17:23.867 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:23.867 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:23.867 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:23.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.867 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:23.867 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:23.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.867 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:23.867 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6823", 2023-10-08T00:17:23.868 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:23.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.868 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:23.868 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6825", 2023-10-08T00:17:23.868 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:23.868 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:23.868 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:23.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.868 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6818/86608", 2023-10-08T00:17:23.868 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6821/86608", 2023-10-08T00:17:23.868 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6828/86608", 2023-10-08T00:17:23.869 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6825/86608", 2023-10-08T00:17:23.869 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:23.869 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:23.869 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:23.869 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:23.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.869 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:17:23.869 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:23.869 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:23.869 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:23.869 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:23.870 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:23.870 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:23.870 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:23.870 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:23.870 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:23.870 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:23.870 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:23.870 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:23.870 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:23.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.870 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:23.870 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6808", 2023-10-08T00:17:23.871 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:23.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.871 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:23.871 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6809", 2023-10-08T00:17:23.871 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:23.871 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:23.871 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:23.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.871 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:23.871 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:23.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.872 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:23.872 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6810", 2023-10-08T00:17:23.872 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:23.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.872 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:23.872 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6811", 2023-10-08T00:17:23.872 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:23.872 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:23.872 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:23.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.873 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:23.873 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:23.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.873 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:23.873 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6814", 2023-10-08T00:17:23.873 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:23.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.873 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:23.873 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6815", 2023-10-08T00:17:23.873 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:23.874 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:23.874 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:23.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.874 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:23.874 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:23.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.874 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:23.874 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6812", 2023-10-08T00:17:23.874 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:23.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.875 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:23.875 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6813", 2023-10-08T00:17:23.875 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:23.875 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:23.875 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:23.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.875 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6809/86571", 2023-10-08T00:17:23.875 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6811/86571", 2023-10-08T00:17:23.875 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6815/86571", 2023-10-08T00:17:23.875 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6813/86571", 2023-10-08T00:17:23.875 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:23.875 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:23.876 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:23.876 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:23.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.876 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:17:23.876 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:23.876 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:23.876 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:23.876 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:23.876 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:23.876 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:23.876 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:23.877 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:23.877 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:23.877 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:23.877 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:23.877 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:23.877 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:23.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.877 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:23.877 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6817", 2023-10-08T00:17:23.877 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:23.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.878 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:23.878 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6819", 2023-10-08T00:17:23.878 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:23.878 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:23.878 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:23.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.878 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:23.878 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:23.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.878 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:23.878 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6822", 2023-10-08T00:17:23.878 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:23.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.879 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:23.879 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6824", 2023-10-08T00:17:23.879 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:23.879 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:23.879 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:23.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.879 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:23.879 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:23.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.879 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:23.880 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6830", 2023-10-08T00:17:23.880 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:23.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.880 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:23.880 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6831", 2023-10-08T00:17:23.880 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:23.880 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:23.880 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:23.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.881 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:23.881 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:23.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.881 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:23.881 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6827", 2023-10-08T00:17:23.881 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:23.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.881 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:23.881 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6829", 2023-10-08T00:17:23.881 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:23.882 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:23.882 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:23.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.882 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6819/86612", 2023-10-08T00:17:23.882 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6824/86612", 2023-10-08T00:17:23.882 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6831/86612", 2023-10-08T00:17:23.882 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6829/86612", 2023-10-08T00:17:23.882 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:23.882 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:23.882 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:23.882 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:23.882 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:23.883 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:17:23.883 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2023-10-08T00:17:23.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.883 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:17:23.883 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:23.883 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:23.883 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:23.883 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:23.883 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:23.884 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.905615+0000", 2023-10-08T00:17:23.884 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:23.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.886 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:17:23.886 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:23.886 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:23.886 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:23.886 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:23.887 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:23.887 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.880709+0000", 2023-10-08T00:17:23.887 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:23.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.888 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:17:23.888 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:23.888 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:23.888 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:23.888 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:23.889 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:23.889 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.874704+0000", 2023-10-08T00:17:23.889 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:23.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.891 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:17:23.891 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:23.891 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:23.891 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:23.891 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:23.891 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:23.891 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.899604+0000", 2023-10-08T00:17:23.891 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:23.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.892 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:17:23.892 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:23.892 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:23.892 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:23.892 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:23.892 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:23.892 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.844371+0000", 2023-10-08T00:17:23.893 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:23.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.893 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:17:23.893 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:23.893 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:23.893 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:23.893 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:23.893 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:23.893 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.794810+0000", 2023-10-08T00:17:23.893 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:23.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.894 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.894 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:17:23.894 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:23.894 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:23.894 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:23.894 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:23.894 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:23.894 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.859474+0000", 2023-10-08T00:17:23.894 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:23.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.894 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:23.894 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:17:23.895 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:23.895 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:23.895 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:23.895 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:23.895 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:23.895 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.893130+0000", 2023-10-08T00:17:23.895 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:23.895 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:23.895 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:17:23.895 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2023-10-08T00:17:23.895 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [], 2023-10-08T00:17:23.895 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2023-10-08T00:17:23.896 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2023-10-08T00:17:23.896 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2023-10-08T00:17:23.896 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2023-10-08T00:17:23.896 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2023-10-08T00:17:23.896 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2023-10-08T00:17:23.896 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2023-10-08T00:17:23.896 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2023-10-08T00:17:23.896 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2023-10-08T00:17:23.896 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2023-10-08T00:17:23.896 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2023-10-08T00:17:23.896 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:23.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:23.897 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2023-10-08T00:17:23.897 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2023-10-08T00:17:23.897 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2023-10-08T00:17:23.897 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2023-10-08T00:17:23.897 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2023-10-08T00:17:23.897 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2023-10-08T00:17:23.897 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2023-10-08T00:17:23.897 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2023-10-08T00:17:23.897 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2023-10-08T00:17:23.897 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2023-10-08T00:17:23.897 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2023-10-08T00:17:23.897 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:23.898 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:17:23.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.804+0000 7f5d471f0700 1 -- 172.21.15.99:0/2232578975 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5d24066f30 msgr2=0x7f5d240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:23.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.804+0000 7f5d471f0700 1 --2- 172.21.15.99:0/2232578975 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5d24066f30 0x7f5d240693e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c0093f0 tx=0x7f5d3c017000 comp rx=0 tx=0).stop 2023-10-08T00:17:23.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.804+0000 7f5d471f0700 1 -- 172.21.15.99:0/2232578975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d400cc710 msgr2=0x7f5d400afdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:23.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.804+0000 7f5d471f0700 1 --2- 172.21.15.99:0/2232578975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d400cc710 0x7f5d400afdf0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f5d3000c810 tx=0x7f5d30007db0 comp rx=0 tx=0).stop 2023-10-08T00:17:23.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.804+0000 7f5d471f0700 1 -- 172.21.15.99:0/2232578975 shutdown_connections 2023-10-08T00:17:23.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.804+0000 7f5d471f0700 1 --2- 172.21.15.99:0/2232578975 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5d24066f30 0x7f5d240693e0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:23.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.804+0000 7f5d471f0700 1 --2- 172.21.15.99:0/2232578975 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d4015abb0 0x7f5d4015b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:23.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.804+0000 7f5d471f0700 1 --2- 172.21.15.99:0/2232578975 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5d4006ab90 0x7f5d400af530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:23.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.804+0000 7f5d471f0700 1 --2- 172.21.15.99:0/2232578975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d400cc710 0x7f5d400afdf0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:23.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.804+0000 7f5d471f0700 1 -- 172.21.15.99:0/2232578975 >> 172.21.15.99:0/2232578975 conn(0x7f5d400cd050 msgr2=0x7f5d4006dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:23.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.805+0000 7f5d471f0700 1 -- 172.21.15.99:0/2232578975 shutdown_connections 2023-10-08T00:17:23.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:23.805+0000 7f5d471f0700 1 -- 172.21.15.99:0/2232578975 wait complete. 2023-10-08T00:17:23.901 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2023-10-08T00:17:23.901 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-08T00:17:24.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.014+0000 7f452bfff700 1 Processor -- start 2023-10-08T00:17:24.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.014+0000 7f452bfff700 1 -- start start 2023-10-08T00:17:24.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.015+0000 7f452bfff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f452c0cd6e0 0x7f452c0cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:24.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.015+0000 7f452bfff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f452c0cdab0 0x7f452c15d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:24.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.015+0000 7f452bfff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f452c15dda0 0x7f452c15cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:24.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.015+0000 7f452bfff700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f452c15e850 con 0x7f452c0cdab0 2023-10-08T00:17:24.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.015+0000 7f452bfff700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f452c0cd030 con 0x7f452c0cd6e0 2023-10-08T00:17:24.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.015+0000 7f452bfff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f452c0cb7c0 con 0x7f452c15dda0 2023-10-08T00:17:24.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.015+0000 7f45227fc700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f452c0cdab0 0x7f452c15d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:24.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.015+0000 7f45227fc700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f452c0cdab0 0x7f452c15d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44938/0 (socket says 172.21.15.99:44938) 2023-10-08T00:17:24.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.015+0000 7f45227fc700 1 -- 172.21.15.99:0/1863040206 learned_addr learned my addr 172.21.15.99:0/1863040206 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:24.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.015+0000 7f452affd700 1 --2- 172.21.15.99:0/1863040206 >> v2:172.21.15.191:3300/0 conn(0x7f452c0cd6e0 0x7f452c0cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:24.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.015+0000 7f452b7fe700 1 --2- 172.21.15.99:0/1863040206 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f452c15dda0 0x7f452c15cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:24.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.015+0000 7f45227fc700 1 -- 172.21.15.99:0/1863040206 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f452c15dda0 msgr2=0x7f452c15cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:24.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.015+0000 7f45227fc700 1 --2- 172.21.15.99:0/1863040206 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f452c15dda0 0x7f452c15cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:24.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.015+0000 7f45227fc700 1 -- 172.21.15.99:0/1863040206 >> v2:172.21.15.191:3300/0 conn(0x7f452c0cd6e0 msgr2=0x7f452c0cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:24.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.015+0000 7f45227fc700 1 --2- 172.21.15.99:0/1863040206 >> v2:172.21.15.191:3300/0 conn(0x7f452c0cd6e0 0x7f452c0cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:24.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.015+0000 7f45227fc700 1 -- 172.21.15.99:0/1863040206 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f452c15e290 con 0x7f452c0cdab0 2023-10-08T00:17:24.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.015+0000 7f452affd700 1 --2- 172.21.15.99:0/1863040206 >> v2:172.21.15.191:3300/0 conn(0x7f452c0cd6e0 0x7f452c0cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:17:24.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.016+0000 7f45227fc700 1 --2- 172.21.15.99:0/1863040206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f452c0cdab0 0x7f452c15d860 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f452c06b730 tx=0x7f4514009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cf319064415fbea server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:24.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.016+0000 7f452a7fc700 1 -- 172.21.15.99:0/1863040206 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451400b860 con 0x7f452c0cdab0 2023-10-08T00:17:24.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.016+0000 7f452a7fc700 1 -- 172.21.15.99:0/1863040206 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f451400b9c0 con 0x7f452c0cdab0 2023-10-08T00:17:24.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.016+0000 7f452a7fc700 1 -- 172.21.15.99:0/1863040206 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451400bd60 con 0x7f452c0cdab0 2023-10-08T00:17:24.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.016+0000 7f452bfff700 1 -- 172.21.15.99:0/1863040206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f452c0cdab0 msgr2=0x7f452c15d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:24.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.016+0000 7f452bfff700 1 --2- 172.21.15.99:0/1863040206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f452c0cdab0 0x7f452c15d860 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f452c06b730 tx=0x7f4514009250 comp rx=0 tx=0).stop 2023-10-08T00:17:24.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.016+0000 7f452bfff700 1 -- 172.21.15.99:0/1863040206 shutdown_connections 2023-10-08T00:17:24.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.016+0000 7f452bfff700 1 --2- 172.21.15.99:0/1863040206 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f452c15dda0 0x7f452c15cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:24.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.016+0000 7f452bfff700 1 --2- 172.21.15.99:0/1863040206 >> v2:172.21.15.191:3300/0 conn(0x7f452c0cd6e0 0x7f452c0cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:24.020 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.016+0000 7f452bfff700 1 --2- 172.21.15.99:0/1863040206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f452c0cdab0 0x7f452c15d860 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:24.020 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.016+0000 7f452bfff700 1 -- 172.21.15.99:0/1863040206 >> 172.21.15.99:0/1863040206 conn(0x7f452c067660 msgr2=0x7f452c067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:24.020 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.016+0000 7f452bfff700 1 -- 172.21.15.99:0/1863040206 shutdown_connections 2023-10-08T00:17:24.020 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.016+0000 7f452bfff700 1 -- 172.21.15.99:0/1863040206 wait complete. 2023-10-08T00:17:24.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.018+0000 7f452bfff700 1 Processor -- start 2023-10-08T00:17:24.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.018+0000 7f452bfff700 1 -- start start 2023-10-08T00:17:24.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.018+0000 7f452bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f452c0cd6e0 0x7f452c0c8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:24.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.018+0000 7f452bfff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f452c0cdab0 0x7f452c16d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:24.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.018+0000 7f452bfff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f452c15dda0 0x7f452c0b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:24.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.018+0000 7f452bfff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452c15b1c0 con 0x7f452c0cd6e0 2023-10-08T00:17:24.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.018+0000 7f452bfff700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452c16e750 con 0x7f452c15dda0 2023-10-08T00:17:24.024 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.018+0000 7f452bfff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f452c15b3f0 con 0x7f452c0cdab0 2023-10-08T00:17:24.024 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.018+0000 7f452affd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f452c0cd6e0 0x7f452c0c8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:24.024 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.018+0000 7f452affd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f452c0cd6e0 0x7f452c0c8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44944/0 (socket says 172.21.15.99:44944) 2023-10-08T00:17:24.024 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.018+0000 7f452affd700 1 -- 172.21.15.99:0/3924942603 learned_addr learned my addr 172.21.15.99:0/3924942603 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:24.024 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.019+0000 7f452b7fe700 1 --2- 172.21.15.99:0/3924942603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f452c15dda0 0x7f452c0b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:24.024 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.019+0000 7f452affd700 1 -- 172.21.15.99:0/3924942603 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f452c0cdab0 msgr2=0x7f452c16d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:24.024 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.019+0000 7f45227fc700 1 --2- 172.21.15.99:0/3924942603 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f452c0cdab0 0x7f452c16d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:24.024 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.019+0000 7f452affd700 1 --2- 172.21.15.99:0/3924942603 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f452c0cdab0 0x7f452c16d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:24.024 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.019+0000 7f452affd700 1 -- 172.21.15.99:0/3924942603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f452c15dda0 msgr2=0x7f452c0b50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:24.024 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.019+0000 7f452affd700 1 --2- 172.21.15.99:0/3924942603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f452c15dda0 0x7f452c0b50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:24.024 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.019+0000 7f452affd700 1 -- 172.21.15.99:0/3924942603 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45140094f0 con 0x7f452c0cd6e0 2023-10-08T00:17:24.025 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.019+0000 7f452b7fe700 1 --2- 172.21.15.99:0/3924942603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f452c15dda0 0x7f452c0b50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:17:24.025 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.019+0000 7f452affd700 1 --2- 172.21.15.99:0/3924942603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f452c0cd6e0 0x7f452c0c8e50 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f451c00c830 tx=0x7f451c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:24.025 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.019+0000 7f45237fe700 1 -- 172.21.15.99:0/3924942603 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451c011600 con 0x7f452c0cd6e0 2023-10-08T00:17:24.025 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.019+0000 7f45237fe700 1 -- 172.21.15.99:0/3924942603 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f451c011760 con 0x7f452c0cd6e0 2023-10-08T00:17:24.025 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.019+0000 7f452bfff700 1 -- 172.21.15.99:0/3924942603 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f452c16e190 con 0x7f452c0cd6e0 2023-10-08T00:17:24.025 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.019+0000 7f45237fe700 1 -- 172.21.15.99:0/3924942603 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451c011b20 con 0x7f452c0cd6e0 2023-10-08T00:17:24.025 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.019+0000 7f452bfff700 1 -- 172.21.15.99:0/3924942603 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f452c0c9390 con 0x7f452c0cd6e0 2023-10-08T00:17:24.025 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.020+0000 7f452bfff700 1 -- 172.21.15.99:0/3924942603 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f452c16e580 con 0x7f452c0cd6e0 2023-10-08T00:17:24.025 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.020+0000 7f45237fe700 1 -- 172.21.15.99:0/3924942603 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f451c00a3e0 con 0x7f452c0cd6e0 2023-10-08T00:17:24.025 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.021+0000 7f45237fe700 1 --2- 172.21.15.99:0/3924942603 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f450c066fb0 0x7f450c069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:24.025 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.021+0000 7f45237fe700 1 -- 172.21.15.99:0/3924942603 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f451c084a90 con 0x7f452c0cd6e0 2023-10-08T00:17:24.025 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.021+0000 7f45227fc700 1 --2- 172.21.15.99:0/3924942603 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f450c066fb0 0x7f450c069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:24.026 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.022+0000 7f45227fc700 1 --2- 172.21.15.99:0/3924942603 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f450c066fb0 0x7f450c069460 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f452c06b730 tx=0x7f4514017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:24.026 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.025+0000 7f45237fe700 1 -- 172.21.15.99:0/3924942603 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f451c051c60 con 0x7f452c0cd6e0 2023-10-08T00:17:24.212 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.211+0000 7f452bfff700 1 -- 172.21.15.99:0/3924942603 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f452c15e570 con 0x7f452c0cd6e0 2023-10-08T00:17:24.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.212+0000 7f45237fe700 1 -- 172.21.15.99:0/3924942603 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v142) v1 ==== 81+0+32282 (secure 0 0 0) 0x7f451c015090 con 0x7f452c0cd6e0 2023-10-08T00:17:24.213 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:17:24.213 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:17:24.214 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 142, 2023-10-08T00:17:24.214 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", 2023-10-08T00:17:24.214 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2023-10-08T00:10:36.303601+0000", 2023-10-08T00:17:24.214 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2023-10-08T00:17:21.117117+0000", 2023-10-08T00:17:24.214 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2023-10-08T00:10:41.709171+0000", 2023-10-08T00:17:24.214 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2023-10-08T00:10:38.690404+0000", 2023-10-08T00:17:24.214 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-08T00:17:24.214 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2023-10-08T00:17:24.214 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2023-10-08T00:17:24.215 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2023-10-08T00:17:24.215 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2023-10-08T00:17:24.215 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2023-10-08T00:17:24.215 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2023-10-08T00:17:24.215 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:17:24.215 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2023-10-08T00:17:24.215 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2023-10-08T00:17:24.215 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-08T00:17:24.215 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-08T00:17:24.215 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2023-10-08T00:17:24.215 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 17, 2023-10-08T00:17:24.215 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2023-10-08T00:17:24.216 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2023-10-08T00:17:24.216 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "jewel", 2023-10-08T00:17:24.216 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "quincy", 2023-10-08T00:17:24.216 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2023-10-08T00:17:24.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.216 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2023-10-08T00:17:24.216 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2023-10-08T00:17:24.216 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:10:42.063344+0000", 2023-10-08T00:17:24.216 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:17:24.216 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:17:24.216 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:17:24.217 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:17:24.217 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:17:24.217 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:17:24.217 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:17:24.217 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:17:24.217 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:17:24.217 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:17:24.217 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:17:24.217 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:17:24.217 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2023-10-08T00:17:24.217 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2023-10-08T00:17:24.218 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2023-10-08T00:17:24.218 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2023-10-08T00:17:24.218 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2023-10-08T00:17:24.218 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:17:24.218 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:17:24.218 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:17:24.218 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:17:24.218 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:17:24.218 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:17:24.219 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:17:24.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.219 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "112", 2023-10-08T00:17:24.219 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:17:24.219 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:17:24.219 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:17:24.219 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:17:24.219 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:17:24.219 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:17:24.219 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:17:24.220 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:17:24.220 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:17:24.220 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:17:24.220 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:17:24.220 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:17:24.220 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:17:24.220 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:17:24.220 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:17:24.221 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:17:24.221 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:17:24.221 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:17:24.221 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:17:24.221 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:17:24.221 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:17:24.221 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:17:24.221 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:17:24.221 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:17:24.221 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:17:24.221 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:17:24.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.222 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:17:24.222 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:17:24.222 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:17:24.222 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:17:24.222 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:17:24.222 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:17:24.222 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:17:24.222 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:17:24.222 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:17:24.223 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:17:24.223 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:17:24.223 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:17:24.223 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2023-10-08T00:17:24.223 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2023-10-08T00:17:24.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.223 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:17:24.223 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2023-10-08T00:17:24.224 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:24.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.224 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.224 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 16, 2023-10-08T00:17:24.224 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2023-10-08T00:17:24.224 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:16:48.430607+0000", 2023-10-08T00:17:24.224 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:17:24.224 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:17:24.224 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:17:24.225 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:17:24.225 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:17:24.225 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:17:24.225 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:17:24.225 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:17:24.225 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:17:24.225 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:17:24.225 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:17:24.225 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:17:24.225 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2023-10-08T00:17:24.226 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2023-10-08T00:17:24.226 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2023-10-08T00:17:24.226 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2023-10-08T00:17:24.226 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2023-10-08T00:17:24.226 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:17:24.226 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:17:24.226 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:17:24.226 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:17:24.226 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:17:24.227 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:17:24.227 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:17:24.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.227 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "138", 2023-10-08T00:17:24.227 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:17:24.227 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:17:24.227 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:17:24.227 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:17:24.227 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:17:24.228 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:17:24.228 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:17:24.228 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:17:24.228 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:17:24.228 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:17:24.228 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:17:24.228 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:17:24.229 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:17:24.229 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:17:24.229 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:17:24.229 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:17:24.229 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:17:24.230 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:17:24.230 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:17:24.230 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:17:24.230 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:17:24.230 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:17:24.230 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:17:24.230 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:17:24.230 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:17:24.231 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:17:24.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.231 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:17:24.231 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:17:24.231 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:17:24.231 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:17:24.231 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:17:24.231 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:17:24.231 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:17:24.232 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:17:24.232 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:17:24.232 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:17:24.232 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:17:24.232 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:17:24.232 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2023-10-08T00:17:24.232 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2023-10-08T00:17:24.232 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2023-10-08T00:17:24.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.232 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:17:24.233 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:17:24.233 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2023-10-08T00:17:24.233 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:24.233 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:24.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.233 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 17, 2023-10-08T00:17:24.233 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2023-10-08T00:17:24.233 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:16:49.063424+0000", 2023-10-08T00:17:24.233 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:17:24.233 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:17:24.233 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:17:24.234 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:17:24.234 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:17:24.234 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:17:24.234 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:17:24.234 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:17:24.234 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:17:24.234 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:17:24.234 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:17:24.234 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:17:24.234 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2023-10-08T00:17:24.234 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2023-10-08T00:17:24.235 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2023-10-08T00:17:24.235 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2023-10-08T00:17:24.235 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2023-10-08T00:17:24.235 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:17:24.235 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:17:24.235 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:17:24.235 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:17:24.236 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:17:24.236 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:17:24.236 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:17:24.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.236 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "142", 2023-10-08T00:17:24.236 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:17:24.236 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:17:24.237 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:17:24.237 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:17:24.237 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:17:24.237 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:17:24.237 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:17:24.237 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:17:24.238 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:17:24.238 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 33554432, 2023-10-08T00:17:24.238 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:17:24.238 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:17:24.238 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:17:24.238 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:17:24.239 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:17:24.239 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:17:24.239 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:17:24.239 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:17:24.239 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:17:24.239 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:17:24.240 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:17:24.240 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:17:24.240 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:17:24.240 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:17:24.240 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:17:24.240 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:17:24.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.241 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:17:24.241 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:17:24.241 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:17:24.241 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:17:24.241 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:17:24.241 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:17:24.241 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:17:24.242 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:17:24.242 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:17:24.242 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:17:24.242 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:17:24.242 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:17:24.243 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2023-10-08T00:17:24.243 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2023-10-08T00:17:24.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.243 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:17:24.243 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:17:24.243 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2023-10-08T00:17:24.244 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:24.244 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:24.244 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:24.244 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:17:24.244 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2023-10-08T00:17:24.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.244 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:17:24.244 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:24.244 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:24.245 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:24.245 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:24.245 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:24.245 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:24.245 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:24.245 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:24.245 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:24.245 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:24.245 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:24.245 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:24.245 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:24.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.246 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:24.246 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2023-10-08T00:17:24.246 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:24.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.246 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:24.246 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2023-10-08T00:17:24.246 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:24.246 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:24.246 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:24.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.246 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:24.247 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:24.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.247 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:24.247 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2023-10-08T00:17:24.247 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:24.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.247 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:24.247 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2023-10-08T00:17:24.247 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:24.247 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:24.247 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:24.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.248 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:24.248 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:24.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.248 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:24.248 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2023-10-08T00:17:24.248 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:24.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.248 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:24.248 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2023-10-08T00:17:24.248 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:24.249 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:24.249 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:24.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.249 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:24.249 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:24.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.249 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:24.249 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2023-10-08T00:17:24.249 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:24.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.249 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:24.249 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2023-10-08T00:17:24.250 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:17:24.250 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:24.250 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:24.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.250 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/86365", 2023-10-08T00:17:24.250 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/86365", 2023-10-08T00:17:24.250 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/86365", 2023-10-08T00:17:24.250 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/86365", 2023-10-08T00:17:24.250 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:24.250 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:24.250 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:24.250 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:24.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.251 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:17:24.251 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:24.251 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:24.251 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:24.251 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:24.251 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:24.251 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:24.251 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:24.251 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:24.251 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:24.252 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:24.252 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:24.252 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:24.252 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:24.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.252 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:24.253 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2023-10-08T00:17:24.253 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:24.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.253 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:24.254 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2023-10-08T00:17:24.254 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:24.254 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:24.254 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:24.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.254 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:24.254 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:24.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.254 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:24.254 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2023-10-08T00:17:24.255 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:24.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.255 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:24.255 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2023-10-08T00:17:24.255 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:24.256 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:24.256 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:24.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.256 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:24.256 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:24.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.257 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:24.257 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2023-10-08T00:17:24.257 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:24.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.257 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:24.257 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2023-10-08T00:17:24.258 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:24.258 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:24.258 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:24.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.258 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:24.258 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:24.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.259 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:24.259 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2023-10-08T00:17:24.259 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:24.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.260 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:24.260 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2023-10-08T00:17:24.260 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:17:24.260 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:24.260 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:24.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.261 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/86366", 2023-10-08T00:17:24.261 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/86366", 2023-10-08T00:17:24.261 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/86366", 2023-10-08T00:17:24.261 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/86366", 2023-10-08T00:17:24.261 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:24.261 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:24.262 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:24.262 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:24.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.262 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:17:24.262 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:24.263 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:24.263 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:24.263 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:24.263 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:24.263 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:24.264 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:24.264 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:24.264 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:24.264 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:24.264 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:24.265 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:24.265 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:24.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.265 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:24.265 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2023-10-08T00:17:24.266 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:24.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.266 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:24.266 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2023-10-08T00:17:24.267 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:24.267 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:24.267 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:24.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.267 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:24.268 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:24.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.268 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:24.268 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2023-10-08T00:17:24.268 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:24.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.269 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:24.269 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2023-10-08T00:17:24.269 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:24.269 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:24.270 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:24.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.270 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:24.270 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:24.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.270 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:24.271 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2023-10-08T00:17:24.271 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:24.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.271 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:24.271 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2023-10-08T00:17:24.272 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:24.272 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:24.272 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:24.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.272 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:24.272 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:24.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.273 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:24.273 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2023-10-08T00:17:24.273 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:24.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.274 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:24.274 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2023-10-08T00:17:24.274 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:17:24.274 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:24.274 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:24.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.275 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6817/86382", 2023-10-08T00:17:24.275 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/86382", 2023-10-08T00:17:24.275 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/86382", 2023-10-08T00:17:24.275 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/86382", 2023-10-08T00:17:24.275 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:24.276 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:24.276 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:24.276 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:24.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.276 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:17:24.277 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:24.277 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:24.277 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:24.277 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:24.277 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:24.277 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:24.278 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:24.278 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:24.278 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:24.278 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:24.278 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:24.278 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:24.279 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:24.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.279 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:24.279 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2023-10-08T00:17:24.279 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:24.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.280 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:24.280 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2023-10-08T00:17:24.280 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:24.280 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:24.280 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:24.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.281 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:24.281 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:24.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.281 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:24.282 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2023-10-08T00:17:24.282 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:24.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.282 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:24.282 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2023-10-08T00:17:24.283 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:24.283 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:24.283 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:24.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.283 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:24.283 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:24.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.283 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:24.283 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2023-10-08T00:17:24.284 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:24.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.284 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:24.284 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2023-10-08T00:17:24.284 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:24.284 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:24.284 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:24.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.284 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:24.285 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:24.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.285 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:24.285 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2023-10-08T00:17:24.285 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:24.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2023-10-08T00:17:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:17:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6820/86393", 2023-10-08T00:17:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/86393", 2023-10-08T00:17:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/86393", 2023-10-08T00:17:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/86393", 2023-10-08T00:17:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:17:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6800", 2023-10-08T00:17:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6801", 2023-10-08T00:17:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6802", 2023-10-08T00:17:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6803", 2023-10-08T00:17:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6806", 2023-10-08T00:17:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6807", 2023-10-08T00:17:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6804", 2023-10-08T00:17:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6805", 2023-10-08T00:17:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:17:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6801/86570", 2023-10-08T00:17:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6803/86570", 2023-10-08T00:17:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6807/86570", 2023-10-08T00:17:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6805/86570", 2023-10-08T00:17:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:17:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6816", 2023-10-08T00:17:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6818", 2023-10-08T00:17:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6820", 2023-10-08T00:17:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6821", 2023-10-08T00:17:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6826", 2023-10-08T00:17:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6828", 2023-10-08T00:17:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6823", 2023-10-08T00:17:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6825", 2023-10-08T00:17:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:17:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6818/86608", 2023-10-08T00:17:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6821/86608", 2023-10-08T00:17:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6828/86608", 2023-10-08T00:17:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6825/86608", 2023-10-08T00:17:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:17:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:24.313 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:24.313 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:24.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.313 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:24.313 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6808", 2023-10-08T00:17:24.313 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:24.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.314 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:24.314 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6809", 2023-10-08T00:17:24.314 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:24.314 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:24.314 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:24.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.315 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:24.315 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:24.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.315 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:24.315 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6810", 2023-10-08T00:17:24.315 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:24.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.315 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:24.315 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6811", 2023-10-08T00:17:24.315 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:24.315 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:24.316 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:24.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.316 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:24.316 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:24.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.316 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:24.316 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6814", 2023-10-08T00:17:24.316 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:24.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.316 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:24.316 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6815", 2023-10-08T00:17:24.317 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:24.317 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:24.317 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:24.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.317 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:24.317 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:24.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.317 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:24.317 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6812", 2023-10-08T00:17:24.317 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:24.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.318 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:24.318 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6813", 2023-10-08T00:17:24.318 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:17:24.318 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:24.318 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:24.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.318 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6809/86571", 2023-10-08T00:17:24.318 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6811/86571", 2023-10-08T00:17:24.318 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6815/86571", 2023-10-08T00:17:24.318 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6813/86571", 2023-10-08T00:17:24.318 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:24.318 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:24.319 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:24.319 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:24.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.319 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:17:24.319 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:17:24.319 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:17:24.319 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:17:24.319 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:17:24.319 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:17:24.319 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:17:24.319 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:17:24.320 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:17:24.320 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 137, 2023-10-08T00:17:24.320 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:17:24.320 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:17:24.320 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:17:24.320 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:24.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.320 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:24.320 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6817", 2023-10-08T00:17:24.320 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:24.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.321 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:24.321 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6819", 2023-10-08T00:17:24.321 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:24.321 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:24.321 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:24.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.321 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:17:24.321 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:24.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.321 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:24.321 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6822", 2023-10-08T00:17:24.321 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:24.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.322 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:24.322 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6824", 2023-10-08T00:17:24.322 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:24.322 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:24.322 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:24.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.322 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:17:24.322 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:24.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.322 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:24.323 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6830", 2023-10-08T00:17:24.323 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:24.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.323 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:24.323 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6831", 2023-10-08T00:17:24.323 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:24.323 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:24.323 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:24.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.323 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:17:24.323 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:17:24.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.324 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:17:24.324 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6827", 2023-10-08T00:17:24.324 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:24.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.324 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:17:24.324 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6829", 2023-10-08T00:17:24.324 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:17:24.324 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:24.324 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:24.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.325 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6819/86612", 2023-10-08T00:17:24.325 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6824/86612", 2023-10-08T00:17:24.325 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6831/86612", 2023-10-08T00:17:24.325 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6829/86612", 2023-10-08T00:17:24.325 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:17:24.325 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:17:24.325 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:17:24.325 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:24.325 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:24.325 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:17:24.326 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2023-10-08T00:17:24.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.326 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:17:24.326 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:24.326 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:24.326 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:24.326 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:24.326 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:24.326 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.905615+0000", 2023-10-08T00:17:24.326 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:24.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.327 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:17:24.327 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:24.327 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:24.327 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:24.327 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:24.327 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:24.327 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.880709+0000", 2023-10-08T00:17:24.327 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:24.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.328 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:17:24.328 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:24.328 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:24.328 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:24.328 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:24.328 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:24.328 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.874704+0000", 2023-10-08T00:17:24.328 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:24.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.329 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:17:24.329 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:24.329 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:24.329 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:24.329 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:24.329 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:24.329 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.899604+0000", 2023-10-08T00:17:24.329 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:24.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.329 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:17:24.329 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:24.330 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:24.330 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:24.330 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:24.330 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:24.330 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.844371+0000", 2023-10-08T00:17:24.330 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:24.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.330 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:17:24.330 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:24.330 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:24.330 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:24.331 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:24.331 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:24.331 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.794810+0000", 2023-10-08T00:17:24.331 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:24.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.331 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:17:24.331 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:24.331 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:24.331 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:24.331 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:24.331 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:24.332 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.859474+0000", 2023-10-08T00:17:24.332 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:24.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:24.332 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:17:24.332 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:17:24.332 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:17:24.332 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:17:24.332 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:17:24.332 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:17:24.332 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.893130+0000", 2023-10-08T00:17:24.332 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:17:24.333 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:24.333 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:17:24.333 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2023-10-08T00:17:24.333 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [], 2023-10-08T00:17:24.333 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2023-10-08T00:17:24.333 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2023-10-08T00:17:24.333 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2023-10-08T00:17:24.333 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2023-10-08T00:17:24.333 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2023-10-08T00:17:24.333 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2023-10-08T00:17:24.333 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2023-10-08T00:17:24.333 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2023-10-08T00:17:24.333 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2023-10-08T00:17:24.334 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2023-10-08T00:17:24.334 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2023-10-08T00:17:24.334 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:24.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:24.334 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2023-10-08T00:17:24.334 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2023-10-08T00:17:24.334 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2023-10-08T00:17:24.334 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2023-10-08T00:17:24.334 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2023-10-08T00:17:24.334 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2023-10-08T00:17:24.334 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2023-10-08T00:17:24.334 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2023-10-08T00:17:24.335 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2023-10-08T00:17:24.335 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2023-10-08T00:17:24.335 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2023-10-08T00:17:24.335 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:24.335 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:17:24.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.214+0000 7f452bfff700 1 -- 172.21.15.99:0/3924942603 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f450c066fb0 msgr2=0x7f450c069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:24.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.214+0000 7f452bfff700 1 --2- 172.21.15.99:0/3924942603 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f450c066fb0 0x7f450c069460 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f452c06b730 tx=0x7f4514017000 comp rx=0 tx=0).stop 2023-10-08T00:17:24.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.214+0000 7f452bfff700 1 -- 172.21.15.99:0/3924942603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f452c0cd6e0 msgr2=0x7f452c0c8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:24.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.214+0000 7f452bfff700 1 --2- 172.21.15.99:0/3924942603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f452c0cd6e0 0x7f452c0c8e50 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f451c00c830 tx=0x7f451c008040 comp rx=0 tx=0).stop 2023-10-08T00:17:24.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.214+0000 7f452bfff700 1 -- 172.21.15.99:0/3924942603 shutdown_connections 2023-10-08T00:17:24.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.214+0000 7f452bfff700 1 --2- 172.21.15.99:0/3924942603 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f450c066fb0 0x7f450c069460 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:24.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.214+0000 7f452bfff700 1 --2- 172.21.15.99:0/3924942603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f452c15dda0 0x7f452c0b50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:24.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.214+0000 7f452bfff700 1 --2- 172.21.15.99:0/3924942603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f452c0cd6e0 0x7f452c0c8e50 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:24.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.214+0000 7f452bfff700 1 --2- 172.21.15.99:0/3924942603 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f452c0cdab0 0x7f452c16d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:24.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.214+0000 7f452bfff700 1 -- 172.21.15.99:0/3924942603 >> 172.21.15.99:0/3924942603 conn(0x7f452c067660 msgr2=0x7f452c0d1340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:24.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.214+0000 7f452bfff700 1 -- 172.21.15.99:0/3924942603 shutdown_connections 2023-10-08T00:17:24.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.214+0000 7f452bfff700 1 -- 172.21.15.99:0/3924942603 wait complete. 2023-10-08T00:17:24.338 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:17:24.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.452+0000 7fb8b429c700 1 Processor -- start 2023-10-08T00:17:24.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.452+0000 7fb8b429c700 1 -- start start 2023-10-08T00:17:24.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.453+0000 7fb8b429c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb8ac157ba0 0x7fb8ac06a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:24.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.453+0000 7fb8b429c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb8ac15b0d0 0x7fb8ac06aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:24.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.453+0000 7fb8b429c700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8ac15c360 con 0x7fb8ac1577d0 2023-10-08T00:17:24.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.453+0000 7fb8b429c700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8ac15b790 con 0x7fb8ac157ba0 2023-10-08T00:17:24.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.453+0000 7fb8b429c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ac15d800 con 0x7fb8ac15b0d0 2023-10-08T00:17:24.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.454+0000 7fb8b2038700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fb8ac1577d0 0x7fb8ac0cd770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50918/0 (socket says 172.21.15.99:50918) 2023-10-08T00:17:24.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.454+0000 7fb8b2038700 1 -- 172.21.15.99:0/714227407 learned_addr learned my addr 172.21.15.99:0/714227407 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:17:24.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.454+0000 7fb8b1036700 1 -- 172.21.15.99:0/714227407 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1435109482 0 0) 0x7fb8ac15c360 con 0x7fb8ac1577d0 2023-10-08T00:17:24.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.454+0000 7fb8b1036700 1 -- 172.21.15.99:0/714227407 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb898003570 con 0x7fb8ac1577d0 2023-10-08T00:17:24.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.454+0000 7fb8b1837700 1 --2- 172.21.15.99:0/714227407 >> v2:172.21.15.191:3300/0 conn(0x7fb8ac157ba0 0x7fb8ac06a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:24.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.454+0000 7fb8b2839700 1 --2- 172.21.15.99:0/714227407 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb8ac15b0d0 0x7fb8ac06aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:24.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.454+0000 7fb8b1036700 1 -- 172.21.15.99:0/714227407 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3440836190 0 0) 0x7fb898003570 con 0x7fb8ac1577d0 2023-10-08T00:17:24.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.454+0000 7fb8b1036700 1 -- 172.21.15.99:0/714227407 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb8ac15b0d0 msgr2=0x7fb8ac06aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:24.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.454+0000 7fb8b1036700 1 --2- 172.21.15.99:0/714227407 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb8ac15b0d0 0x7fb8ac06aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:24.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.454+0000 7fb8b1036700 1 -- 172.21.15.99:0/714227407 >> v2:172.21.15.191:3300/0 conn(0x7fb8ac157ba0 msgr2=0x7fb8ac06a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:24.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.454+0000 7fb8b1036700 1 --2- 172.21.15.99:0/714227407 >> v2:172.21.15.191:3300/0 conn(0x7fb8ac157ba0 0x7fb8ac06a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:24.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.454+0000 7fb8b1036700 1 -- 172.21.15.99:0/714227407 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8ac15ba30 con 0x7fb8ac1577d0 2023-10-08T00:17:24.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.454+0000 7fb8b1036700 1 -- 172.21.15.99:0/714227407 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7fb8a8001470 con 0x7fb8ac1577d0 2023-10-08T00:17:24.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.454+0000 7fb8b1036700 1 -- 172.21.15.99:0/714227407 >> v1:172.21.15.99:6789/0 conn(0x7fb8ac1577d0 legacy=0x7fb8ac0cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:24.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.454+0000 7fb8b1036700 1 --2- 172.21.15.99:0/714227407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb898003a60 0x7fb898003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:24.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.454+0000 7fb8b1036700 1 --2- 172.21.15.99:0/714227407 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb8ac15b0d0 0x7fb898004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:24.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.454+0000 7fb8b1036700 1 --2- 172.21.15.99:0/714227407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb898004d50 0x7fb8980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:24.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.454+0000 7fb8b1036700 1 -- 172.21.15.99:0/714227407 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ac15b790 con 0x7fb898003a60 2023-10-08T00:17:24.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.454+0000 7fb8b1036700 1 -- 172.21.15.99:0/714227407 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ac15d800 con 0x7fb898004d50 2023-10-08T00:17:24.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.454+0000 7fb8b1036700 1 -- 172.21.15.99:0/714227407 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb89800bbc0 con 0x7fb8ac15b0d0 2023-10-08T00:17:24.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.454+0000 7fb8b2038700 1 --2- 172.21.15.99:0/714227407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb898003a60 0x7fb898003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:24.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.454+0000 7fb8b1837700 1 --2- 172.21.15.99:0/714227407 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb8ac15b0d0 0x7fb898004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:24.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.454+0000 7fb8b2839700 1 --2- 172.21.15.99:0/714227407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb898004d50 0x7fb8980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:24.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.455+0000 7fb8b2038700 1 -- 172.21.15.99:0/714227407 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb8ac15b0d0 msgr2=0x7fb898004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:24.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.455+0000 7fb8b2038700 1 --2- 172.21.15.99:0/714227407 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb8ac15b0d0 0x7fb898004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:24.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.455+0000 7fb8b2038700 1 -- 172.21.15.99:0/714227407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb898004d50 msgr2=0x7fb8980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:24.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.455+0000 7fb8b2038700 1 --2- 172.21.15.99:0/714227407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb898004d50 0x7fb8980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:24.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.455+0000 7fb8b2038700 1 -- 172.21.15.99:0/714227407 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb898003570 con 0x7fb898003a60 2023-10-08T00:17:24.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.455+0000 7fb8b1837700 1 --2- 172.21.15.99:0/714227407 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb8ac15b0d0 0x7fb898004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:17:24.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.455+0000 7fb8b2839700 1 --2- 172.21.15.99:0/714227407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb898004d50 0x7fb8980051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:17:24.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.455+0000 7fb8b2038700 1 --2- 172.21.15.99:0/714227407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb898003a60 0x7fb898003e70 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fb8a800c3c0 tx=0x7fb8a8001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3e8b44f358be547 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:24.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.455+0000 7fb8b1036700 1 -- 172.21.15.99:0/714227407 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a800cc90 con 0x7fb898003a60 2023-10-08T00:17:24.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.455+0000 7fb8b1036700 1 -- 172.21.15.99:0/714227407 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a800ce30 con 0x7fb898003a60 2023-10-08T00:17:24.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.455+0000 7fb8b1036700 1 -- 172.21.15.99:0/714227407 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a8005b70 con 0x7fb898003a60 2023-10-08T00:17:24.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.455+0000 7fb8b429c700 1 -- 172.21.15.99:0/714227407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb898003a60 msgr2=0x7fb898003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:24.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.455+0000 7fb8b429c700 1 --2- 172.21.15.99:0/714227407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb898003a60 0x7fb898003e70 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fb8a800c3c0 tx=0x7fb8a8001e90 comp rx=0 tx=0).stop 2023-10-08T00:17:24.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.455+0000 7fb8b2839700 1 -- 172.21.15.99:0/714227407 reap_dead start 2023-10-08T00:17:24.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.455+0000 7fb8b429c700 1 -- 172.21.15.99:0/714227407 shutdown_connections 2023-10-08T00:17:24.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.455+0000 7fb8b429c700 1 -- 172.21.15.99:0/714227407 >> 172.21.15.99:0/714227407 conn(0x7fb8ac068ce0 msgr2=0x7fb8ac0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:24.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.455+0000 7fb8b429c700 1 -- 172.21.15.99:0/714227407 shutdown_connections 2023-10-08T00:17:24.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.455+0000 7fb8b429c700 1 -- 172.21.15.99:0/714227407 wait complete. 2023-10-08T00:17:24.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.456+0000 7fb8b429c700 1 Processor -- start 2023-10-08T00:17:24.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.456+0000 7fb8b429c700 1 -- start start 2023-10-08T00:17:24.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.457+0000 7fb8b429c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb898003a60 0x7fb8ac153610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:24.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.457+0000 7fb8b429c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8ac16a1f0 0x7fb8ac16a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:24.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.457+0000 7fb8b429c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8ac0b1e50 0x7fb8ac0b22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:24.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.457+0000 7fb8b429c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ac0b2800 con 0x7fb8ac16a1f0 2023-10-08T00:17:24.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.457+0000 7fb8b429c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ac0b19c0 con 0x7fb8ac0b1e50 2023-10-08T00:17:24.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.457+0000 7fb8b429c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ac0b0db0 con 0x7fb898003a60 2023-10-08T00:17:24.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.457+0000 7fb8b1837700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8ac16a1f0 0x7fb8ac16a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:24.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.457+0000 7fb8b1837700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8ac16a1f0 0x7fb8ac16a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44966/0 (socket says 172.21.15.99:44966) 2023-10-08T00:17:24.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.457+0000 7fb8b1837700 1 -- 172.21.15.99:0/400098182 learned_addr learned my addr 172.21.15.99:0/400098182 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:24.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.457+0000 7fb8b2038700 1 --2- 172.21.15.99:0/400098182 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb898003a60 0x7fb8ac153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:24.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.457+0000 7fb8b2839700 1 --2- 172.21.15.99:0/400098182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8ac0b1e50 0x7fb8ac0b22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:24.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.457+0000 7fb8b1837700 1 -- 172.21.15.99:0/400098182 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb898003a60 msgr2=0x7fb8ac153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:24.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.457+0000 7fb8b1837700 1 --2- 172.21.15.99:0/400098182 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb898003a60 0x7fb8ac153610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:24.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.457+0000 7fb8b1837700 1 -- 172.21.15.99:0/400098182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8ac0b1e50 msgr2=0x7fb8ac0b22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:24.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.457+0000 7fb8b1837700 1 --2- 172.21.15.99:0/400098182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8ac0b1e50 0x7fb8ac0b22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:24.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.457+0000 7fb8b1837700 1 -- 172.21.15.99:0/400098182 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8ac0b1090 con 0x7fb8ac16a1f0 2023-10-08T00:17:24.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.457+0000 7fb8b2839700 1 --2- 172.21.15.99:0/400098182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8ac0b1e50 0x7fb8ac0b22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:17:24.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.458+0000 7fb8b1837700 1 --2- 172.21.15.99:0/400098182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8ac16a1f0 0x7fb8ac16a660 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fb89c00f910 tx=0x7fb89c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:24.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.458+0000 7fb8a27fc700 1 -- 172.21.15.99:0/400098182 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89c00fe50 con 0x7fb8ac16a1f0 2023-10-08T00:17:24.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.458+0000 7fb8a27fc700 1 -- 172.21.15.99:0/400098182 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb89c00d860 con 0x7fb8ac16a1f0 2023-10-08T00:17:24.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.458+0000 7fb8b429c700 1 -- 172.21.15.99:0/400098182 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8ac0b1370 con 0x7fb8ac16a1f0 2023-10-08T00:17:24.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.458+0000 7fb8a27fc700 1 -- 172.21.15.99:0/400098182 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89c00dc20 con 0x7fb8ac16a1f0 2023-10-08T00:17:24.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.458+0000 7fb8b429c700 1 -- 172.21.15.99:0/400098182 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8ac153b50 con 0x7fb8ac16a1f0 2023-10-08T00:17:24.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.459+0000 7fb8b429c700 1 -- 172.21.15.99:0/400098182 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8ac04c3c0 con 0x7fb8ac16a1f0 2023-10-08T00:17:24.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.459+0000 7fb8a27fc700 1 -- 172.21.15.99:0/400098182 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb89c014460 con 0x7fb8ac16a1f0 2023-10-08T00:17:24.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.459+0000 7fb8a27fc700 1 --2- 172.21.15.99:0/400098182 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb890066f30 0x7fb8900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:24.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.459+0000 7fb8a27fc700 1 -- 172.21.15.99:0/400098182 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb89c084ea0 con 0x7fb8ac16a1f0 2023-10-08T00:17:24.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.460+0000 7fb8b2038700 1 --2- 172.21.15.99:0/400098182 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb890066f30 0x7fb8900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:24.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.460+0000 7fb8b2038700 1 --2- 172.21.15.99:0/400098182 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb890066f30 0x7fb8900693e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb8a800c3c0 tx=0x7fb8a8015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:24.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.464+0000 7fb8a27fc700 1 -- 172.21.15.99:0/400098182 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb89c0521c0 con 0x7fb8ac16a1f0 2023-10-08T00:17:24.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.686+0000 7fb8b429c700 1 -- 172.21.15.99:0/400098182 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8ac15bd10 con 0x7fb8ac16a1f0 2023-10-08T00:17:24.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.687+0000 7fb8a27fc700 1 -- 172.21.15.99:0/400098182 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4602 (secure 0 0 0) 0x7fb89c0557e0 con 0x7fb8ac16a1f0 2023-10-08T00:17:24.689 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:17:24.689 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":63,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5813,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/887374839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":887374839},{"type":"v1","addr":"172.21.15.191:6837","nonce":887374839}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":6445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/2409696901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2409696901},{"type":"v1","addr":"172.21.15.191:6836","nonce":2409696901}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":7095,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/1924096228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1924096228},{"type":"v1","addr":"172.21.15.99:6834","nonce":1924096228}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:16:50.092494+0000","modified":"2023-10-08T00:16:53.744105+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7098":{"gid":7098,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/2522933027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":2522933027},{"type":"v1","addr":"172.21.15.99:6835","nonce":2522933027}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-08T00:17:24.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.689+0000 7fb8b429c700 1 -- 172.21.15.99:0/400098182 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb890066f30 msgr2=0x7fb8900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:24.691 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.689+0000 7fb8b429c700 1 --2- 172.21.15.99:0/400098182 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb890066f30 0x7fb8900693e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb8a800c3c0 tx=0x7fb8a8015000 comp rx=0 tx=0).stop 2023-10-08T00:17:24.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.689+0000 7fb8b429c700 1 -- 172.21.15.99:0/400098182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8ac16a1f0 msgr2=0x7fb8ac16a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:24.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.689+0000 7fb8b429c700 1 --2- 172.21.15.99:0/400098182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8ac16a1f0 0x7fb8ac16a660 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fb89c00f910 tx=0x7fb89c008040 comp rx=0 tx=0).stop 2023-10-08T00:17:24.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.689+0000 7fb8b429c700 1 -- 172.21.15.99:0/400098182 shutdown_connections 2023-10-08T00:17:24.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.689+0000 7fb8b429c700 1 --2- 172.21.15.99:0/400098182 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb890066f30 0x7fb8900693e0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:24.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.689+0000 7fb8b429c700 1 --2- 172.21.15.99:0/400098182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8ac0b1e50 0x7fb8ac0b22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:24.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.689+0000 7fb8b429c700 1 --2- 172.21.15.99:0/400098182 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb898003a60 0x7fb8ac153610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:24.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.689+0000 7fb8b429c700 1 --2- 172.21.15.99:0/400098182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8ac16a1f0 0x7fb8ac16a660 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:24.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.689+0000 7fb8b429c700 1 -- 172.21.15.99:0/400098182 >> 172.21.15.99:0/400098182 conn(0x7fb8ac068ce0 msgr2=0x7fb8ac155610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:24.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.689+0000 7fb8b429c700 1 -- 172.21.15.99:0/400098182 shutdown_connections 2023-10-08T00:17:24.693 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:24.689+0000 7fb8b429c700 1 -- 172.21.15.99:0/400098182 wait complete. 2023-10-08T00:17:24.693 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 63 2023-10-08T00:17:24.702 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2023-10-08T00:17:24.835 INFO:teuthology.orchestra.run.smithi099.stdout:{"cluster_fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","whoami":0,"id":7098,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":34.087062871000001,"mdsmap_epoch":63,"osdmap_epoch":142,"osdmap_epoch_barrier":141,"uptime":39.221906273999998} 2023-10-08T00:17:24.835 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", "fs_name": "cephfs", "id": 7098, "mdsmap_epoch": 63, "osdmap_epoch": 142, "osdmap_epoch_barrier": 141, "rank_uptime": 34.087062871, "state": "up:active", "uptime": 39.221906274, "want_state": "up:active", "whoami": 0 } 2023-10-08T00:17:24.835 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-08T00:17:24.836 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:17:24.836 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2023-10-08T00:17:25.110 INFO:teuthology.orchestra.run.smithi099.stderr:50+0 records in 2023-10-08T00:17:25.110 INFO:teuthology.orchestra.run.smithi099.stderr:50+0 records out 2023-10-08T00:17:25.110 INFO:teuthology.orchestra.run.smithi099.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.258882 s, 203 MB/s 2023-10-08T00:17:25.111 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T00:17:25.112 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T00:17:25.156 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T00:17:25.197 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T00:17:25.198 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T00:17:25.198 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:17:25.198 DEBUG:teuthology.orchestra.run.smithi099:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-08T00:17:25.268 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2023-10-08T00:17:25.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:17:25.268 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T00:17:25.283 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-08T00:17:25.283 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:17:25.283 DEBUG:teuthology.orchestra.run.smithi191:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-08T00:17:25.326 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2023-10-08T00:17:25.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:17:25.326 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-08T00:17:25.382 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:17:25.496 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.495+0000 7f13fefa4700 1 Processor -- start 2023-10-08T00:17:25.496 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.495+0000 7f13fefa4700 1 -- start start 2023-10-08T00:17:25.498 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.496+0000 7f13fefa4700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f13f8157c40 0x7f13f8158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:25.498 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.496+0000 7f13fefa4700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13f815cbb0 con 0x7f13f815a620 2023-10-08T00:17:25.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.496+0000 7f13fefa4700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13f815bfa0 con 0x7f13f8157870 2023-10-08T00:17:25.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.496+0000 7f13fefa4700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13f815bdb0 con 0x7f13f8157c40 2023-10-08T00:17:25.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.496+0000 7f13f7fff700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f13f815a620 0x7f13f815d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50924/0 (socket says 172.21.15.99:50924) 2023-10-08T00:17:25.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.496+0000 7f13f7fff700 1 -- 172.21.15.99:0/1517419563 learned_addr learned my addr 172.21.15.99:0/1517419563 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:17:25.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.496+0000 7f13fd541700 1 --2- 172.21.15.99:0/1517419563 >> v2:172.21.15.191:3300/0 conn(0x7f13f8157c40 0x7f13f8158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:25.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.496+0000 7f13f77fe700 1 -- 172.21.15.99:0/1517419563 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1163955096 0 0) 0x7f13f815cbb0 con 0x7f13f815a620 2023-10-08T00:17:25.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.497+0000 7f13f77fe700 1 -- 172.21.15.99:0/1517419563 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13e8003570 con 0x7f13f815a620 2023-10-08T00:17:25.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.497+0000 7f13fd541700 1 -- 172.21.15.99:0/1517419563 >> v1:172.21.15.191:6789/0 conn(0x7f13f8157870 legacy=0x7f13f815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:25.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.497+0000 7f13fd541700 1 -- 172.21.15.99:0/1517419563 >> v1:172.21.15.99:6789/0 conn(0x7f13f815a620 legacy=0x7f13f815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:25.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.497+0000 7f13fd541700 1 -- 172.21.15.99:0/1517419563 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f815c280 con 0x7f13f8157c40 2023-10-08T00:17:25.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.497+0000 7f13fd541700 1 --2- 172.21.15.99:0/1517419563 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f13f8157c40 0x7f13f8158110 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f13f0009e60 tx=0x7f13f0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9b6ae3c49fa70bd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:25.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.497+0000 7f13f77fe700 1 -- 172.21.15.99:0/1517419563 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f000e070 con 0x7f13f8157c40 2023-10-08T00:17:25.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.497+0000 7f13f77fe700 1 -- 172.21.15.99:0/1517419563 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13f000bb60 con 0x7f13f8157c40 2023-10-08T00:17:25.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.497+0000 7f13f77fe700 1 -- 172.21.15.99:0/1517419563 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f0012430 con 0x7f13f8157c40 2023-10-08T00:17:25.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.498+0000 7f13fefa4700 1 -- 172.21.15.99:0/1517419563 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f13f8157c40 msgr2=0x7f13f8158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:25.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.498+0000 7f13fefa4700 1 --2- 172.21.15.99:0/1517419563 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f13f8157c40 0x7f13f8158110 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f13f0009e60 tx=0x7f13f0009780 comp rx=0 tx=0).stop 2023-10-08T00:17:25.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.498+0000 7f13fefa4700 1 -- 172.21.15.99:0/1517419563 shutdown_connections 2023-10-08T00:17:25.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.498+0000 7f13fefa4700 1 --2- 172.21.15.99:0/1517419563 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f13f8157c40 0x7f13f8158110 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:25.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.498+0000 7f13fefa4700 1 -- 172.21.15.99:0/1517419563 >> 172.21.15.99:0/1517419563 conn(0x7f13f8068ce0 msgr2=0x7f13f80690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:25.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.498+0000 7f13fefa4700 1 -- 172.21.15.99:0/1517419563 shutdown_connections 2023-10-08T00:17:25.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.498+0000 7f13fefa4700 1 -- 172.21.15.99:0/1517419563 wait complete. 2023-10-08T00:17:25.503 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.499+0000 7f13fefa4700 1 Processor -- start 2023-10-08T00:17:25.503 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.499+0000 7f13fefa4700 1 -- start start 2023-10-08T00:17:25.503 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.500+0000 7f13fefa4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13f8157870 0x7f13f813f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:25.503 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.500+0000 7f13fefa4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f13f8157c40 0x7f13f80b1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:25.503 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.500+0000 7f13fefa4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13f815a620 0x7f13f814d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:25.503 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.500+0000 7f13fefa4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f80b2cf0 con 0x7f13f8157870 2023-10-08T00:17:25.503 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.500+0000 7f13fefa4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f813fcc0 con 0x7f13f8157c40 2023-10-08T00:17:25.503 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.500+0000 7f13fefa4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13f80b20e0 con 0x7f13f815a620 2023-10-08T00:17:25.504 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.500+0000 7f13fcd40700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13f8157870 0x7f13f813f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:25.504 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.500+0000 7f13fcd40700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13f8157870 0x7f13f813f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44978/0 (socket says 172.21.15.99:44978) 2023-10-08T00:17:25.504 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.500+0000 7f13fcd40700 1 -- 172.21.15.99:0/2731662747 learned_addr learned my addr 172.21.15.99:0/2731662747 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:25.504 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.500+0000 7f13f7fff700 1 --2- 172.21.15.99:0/2731662747 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f13f8157c40 0x7f13f80b1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:25.504 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.500+0000 7f13fd541700 1 --2- 172.21.15.99:0/2731662747 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13f815a620 0x7f13f814d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:25.504 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.500+0000 7f13fcd40700 1 -- 172.21.15.99:0/2731662747 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13f815a620 msgr2=0x7f13f814d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:25.504 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.500+0000 7f13fcd40700 1 --2- 172.21.15.99:0/2731662747 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13f815a620 0x7f13f814d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:25.505 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.500+0000 7f13fcd40700 1 -- 172.21.15.99:0/2731662747 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f13f8157c40 msgr2=0x7f13f80b1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:25.505 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.500+0000 7f13fcd40700 1 --2- 172.21.15.99:0/2731662747 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f13f8157c40 0x7f13f80b1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:25.505 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.500+0000 7f13fcd40700 1 -- 172.21.15.99:0/2731662747 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f0009a20 con 0x7f13f8157870 2023-10-08T00:17:25.505 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.500+0000 7f13f7fff700 1 --2- 172.21.15.99:0/2731662747 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f13f8157c40 0x7f13f80b1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:17:25.505 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.500+0000 7f13fcd40700 1 --2- 172.21.15.99:0/2731662747 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13f8157870 0x7f13f813f780 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f13ec00c590 tx=0x7f13ec00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:25.505 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.500+0000 7f13f4ff9700 1 -- 172.21.15.99:0/2731662747 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ec011590 con 0x7f13f8157870 2023-10-08T00:17:25.505 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.500+0000 7f13f4ff9700 1 -- 172.21.15.99:0/2731662747 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13ec0116f0 con 0x7f13f8157870 2023-10-08T00:17:25.505 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.500+0000 7f13fefa4700 1 -- 172.21.15.99:0/2731662747 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13f80b23c0 con 0x7f13f8157870 2023-10-08T00:17:25.505 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.500+0000 7f13f4ff9700 1 -- 172.21.15.99:0/2731662747 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ec011ab0 con 0x7f13f8157870 2023-10-08T00:17:25.506 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.500+0000 7f13fefa4700 1 -- 172.21.15.99:0/2731662747 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13f80b2980 con 0x7f13f8157870 2023-10-08T00:17:25.506 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.501+0000 7f13fefa4700 1 -- 172.21.15.99:0/2731662747 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13f804c3c0 con 0x7f13f8157870 2023-10-08T00:17:25.506 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.501+0000 7f13f4ff9700 1 -- 172.21.15.99:0/2731662747 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f13ec01b3b0 con 0x7f13f8157870 2023-10-08T00:17:25.506 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.502+0000 7f13f4ff9700 1 --2- 172.21.15.99:0/2731662747 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f13dc066f30 0x7f13dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:25.506 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.502+0000 7f13f4ff9700 1 -- 172.21.15.99:0/2731662747 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f13ec083b70 con 0x7f13f8157870 2023-10-08T00:17:25.506 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.502+0000 7f13f7fff700 1 --2- 172.21.15.99:0/2731662747 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f13dc066f30 0x7f13dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:25.506 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.503+0000 7f13f7fff700 1 --2- 172.21.15.99:0/2731662747 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f13dc066f30 0x7f13dc0693e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f13f816d970 tx=0x7f13e4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:25.506 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.506+0000 7f13f4ff9700 1 -- 172.21.15.99:0/2731662747 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f13ec050de0 con 0x7f13f8157870 2023-10-08T00:17:25.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.732+0000 7f13fefa4700 1 -- 172.21.15.99:0/2731662747 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f13f80b2fd0 con 0x7f13f8157870 2023-10-08T00:17:25.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.733+0000 7f13f4ff9700 1 -- 172.21.15.99:0/2731662747 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f13ec054400 con 0x7f13f8157870 2023-10-08T00:17:25.733 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:17:25.733 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":63,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5813,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/887374839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":887374839},{"type":"v1","addr":"172.21.15.191:6837","nonce":887374839}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":6445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/2409696901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2409696901},{"type":"v1","addr":"172.21.15.191:6836","nonce":2409696901}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":7095,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/1924096228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1924096228},{"type":"v1","addr":"172.21.15.99:6834","nonce":1924096228}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:16:50.092494+0000","modified":"2023-10-08T00:16:53.744105+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7098":{"gid":7098,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/2522933027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":2522933027},{"type":"v1","addr":"172.21.15.99:6835","nonce":2522933027}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-08T00:17:25.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.735+0000 7f13fefa4700 1 -- 172.21.15.99:0/2731662747 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f13dc066f30 msgr2=0x7f13dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:25.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.735+0000 7f13fefa4700 1 --2- 172.21.15.99:0/2731662747 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f13dc066f30 0x7f13dc0693e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f13f816d970 tx=0x7f13e4009040 comp rx=0 tx=0).stop 2023-10-08T00:17:25.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.735+0000 7f13fefa4700 1 -- 172.21.15.99:0/2731662747 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13f8157870 msgr2=0x7f13f813f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:25.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.735+0000 7f13fefa4700 1 --2- 172.21.15.99:0/2731662747 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13f8157870 0x7f13f813f780 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f13ec00c590 tx=0x7f13ec00c950 comp rx=0 tx=0).stop 2023-10-08T00:17:25.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.735+0000 7f13fefa4700 1 -- 172.21.15.99:0/2731662747 shutdown_connections 2023-10-08T00:17:25.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.735+0000 7f13fefa4700 1 --2- 172.21.15.99:0/2731662747 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f13dc066f30 0x7f13dc0693e0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:25.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.735+0000 7f13fefa4700 1 --2- 172.21.15.99:0/2731662747 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13f815a620 0x7f13f814d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:25.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.735+0000 7f13fefa4700 1 --2- 172.21.15.99:0/2731662747 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13f8157870 0x7f13f813f780 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:25.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.735+0000 7f13fefa4700 1 --2- 172.21.15.99:0/2731662747 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f13f8157c40 0x7f13f80b1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:25.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.735+0000 7f13fefa4700 1 -- 172.21.15.99:0/2731662747 >> 172.21.15.99:0/2731662747 conn(0x7f13f8068ce0 msgr2=0x7f13f80b57d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:25.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.735+0000 7f13fefa4700 1 -- 172.21.15.99:0/2731662747 shutdown_connections 2023-10-08T00:17:25.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.735+0000 7f13fefa4700 1 -- 172.21.15.99:0/2731662747 wait complete. 2023-10-08T00:17:25.737 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 63 2023-10-08T00:17:25.747 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:17:25.861 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.860+0000 7eff98e1f700 1 Processor -- start 2023-10-08T00:17:25.861 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.860+0000 7eff98e1f700 1 -- start start 2023-10-08T00:17:25.863 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.861+0000 7eff98e1f700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7eff94157870 0x7eff9415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:25.863 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.861+0000 7eff98e1f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7eff9415a620 0x7eff9415d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:25.863 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.861+0000 7eff98e1f700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff9415cbb0 con 0x7eff9415d580 2023-10-08T00:17:25.863 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.861+0000 7eff98e1f700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff9415bfa0 con 0x7eff94157870 2023-10-08T00:17:25.863 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.861+0000 7eff98e1f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff9415c280 con 0x7eff9415a620 2023-10-08T00:17:25.863 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.861+0000 7eff9259c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7eff94157870 0x7eff9415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:25.864 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.861+0000 7eff9259c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7eff94157870 0x7eff9415b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44994/0 (socket says 172.21.15.99:44994) 2023-10-08T00:17:25.864 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.861+0000 7eff9259c700 1 -- 172.21.15.99:0/1436437982 learned_addr learned my addr 172.21.15.99:0/1436437982 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:25.864 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.861+0000 7eff9259c700 1 -- 172.21.15.99:0/1436437982 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7eff9415a620 msgr2=0x7eff9415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:25.864 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.861+0000 7eff89d9b700 1 --2- 172.21.15.99:0/1436437982 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7eff9415a620 0x7eff9415d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:25.864 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.861+0000 7eff9259c700 1 --2- 172.21.15.99:0/1436437982 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7eff9415a620 0x7eff9415d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:25.864 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.861+0000 7eff9259c700 1 -- 172.21.15.99:0/1436437982 >> v1:172.21.15.191:6789/0 conn(0x7eff9415d580 legacy=0x7eff94158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:25.864 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.861+0000 7eff9259c700 1 -- 172.21.15.99:0/1436437982 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff9415c560 con 0x7eff94157870 2023-10-08T00:17:25.864 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.861+0000 7eff9259c700 1 --2- 172.21.15.99:0/1436437982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff94157870 0x7eff9415b6a0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7eff840099b0 tx=0x7eff84009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b38162e119050de server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:25.864 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.862+0000 7eff91d9b700 1 -- 172.21.15.99:0/1436437982 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8400b820 con 0x7eff94157870 2023-10-08T00:17:25.864 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.862+0000 7eff91d9b700 1 -- 172.21.15.99:0/1436437982 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff8400b980 con 0x7eff94157870 2023-10-08T00:17:25.864 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.862+0000 7eff91d9b700 1 -- 172.21.15.99:0/1436437982 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8400bd20 con 0x7eff94157870 2023-10-08T00:17:25.865 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.862+0000 7eff98e1f700 1 -- 172.21.15.99:0/1436437982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff94157870 msgr2=0x7eff9415b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:25.865 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.862+0000 7eff98e1f700 1 --2- 172.21.15.99:0/1436437982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff94157870 0x7eff9415b6a0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7eff840099b0 tx=0x7eff84009250 comp rx=0 tx=0).stop 2023-10-08T00:17:25.865 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.862+0000 7eff98e1f700 1 -- 172.21.15.99:0/1436437982 shutdown_connections 2023-10-08T00:17:25.865 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.862+0000 7eff98e1f700 1 --2- 172.21.15.99:0/1436437982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff94157870 0x7eff9415b6a0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:25.865 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.862+0000 7eff98e1f700 1 --2- 172.21.15.99:0/1436437982 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7eff9415a620 0x7eff9415d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:25.865 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.862+0000 7eff98e1f700 1 -- 172.21.15.99:0/1436437982 >> 172.21.15.99:0/1436437982 conn(0x7eff94068ce0 msgr2=0x7eff940690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:25.865 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.862+0000 7eff98e1f700 1 -- 172.21.15.99:0/1436437982 shutdown_connections 2023-10-08T00:17:25.865 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.862+0000 7eff98e1f700 1 -- 172.21.15.99:0/1436437982 wait complete. 2023-10-08T00:17:25.868 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.863+0000 7eff98e1f700 1 Processor -- start 2023-10-08T00:17:25.868 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.864+0000 7eff98e1f700 1 -- start start 2023-10-08T00:17:25.868 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.864+0000 7eff98e1f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7eff94157870 0x7eff9414fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:25.868 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.864+0000 7eff98e1f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff9415a620 0x7eff940b0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:25.868 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.864+0000 7eff98e1f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7eff9415d580 0x7eff941566a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:25.868 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.864+0000 7eff98e1f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff94151ad0 con 0x7eff9415a620 2023-10-08T00:17:25.868 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.864+0000 7eff98e1f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff940b0f80 con 0x7eff94157870 2023-10-08T00:17:25.868 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.864+0000 7eff98e1f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff940b1260 con 0x7eff9415d580 2023-10-08T00:17:25.869 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.864+0000 7eff89d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff9415a620 0x7eff940b0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:25.869 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.864+0000 7eff89d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff9415a620 0x7eff940b0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44996/0 (socket says 172.21.15.99:44996) 2023-10-08T00:17:25.869 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.864+0000 7eff89d9b700 1 -- 172.21.15.99:0/3131196896 learned_addr learned my addr 172.21.15.99:0/3131196896 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:25.869 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.864+0000 7eff9259c700 1 --2- 172.21.15.99:0/3131196896 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7eff94157870 0x7eff9414fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:25.869 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.864+0000 7eff92d9d700 1 --2- 172.21.15.99:0/3131196896 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7eff9415d580 0x7eff941566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:25.869 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.865+0000 7eff89d9b700 1 -- 172.21.15.99:0/3131196896 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7eff9415d580 msgr2=0x7eff941566a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:25.869 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.865+0000 7eff89d9b700 1 --2- 172.21.15.99:0/3131196896 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7eff9415d580 0x7eff941566a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:25.869 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.865+0000 7eff89d9b700 1 -- 172.21.15.99:0/3131196896 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7eff94157870 msgr2=0x7eff9414fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:25.869 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.865+0000 7eff89d9b700 1 --2- 172.21.15.99:0/3131196896 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7eff94157870 0x7eff9414fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:25.869 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.865+0000 7eff89d9b700 1 -- 172.21.15.99:0/3131196896 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff840094f0 con 0x7eff9415a620 2023-10-08T00:17:25.869 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.865+0000 7eff89d9b700 1 --2- 172.21.15.99:0/3131196896 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff9415a620 0x7eff940b0610 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7eff7c00c880 tx=0x7eff7c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:25.869 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.865+0000 7eff8affd700 1 -- 172.21.15.99:0/3131196896 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7c011600 con 0x7eff9415a620 2023-10-08T00:17:25.870 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.865+0000 7eff8affd700 1 -- 172.21.15.99:0/3131196896 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff7c011760 con 0x7eff9415a620 2023-10-08T00:17:25.870 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.865+0000 7eff98e1f700 1 -- 172.21.15.99:0/3131196896 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff940b1540 con 0x7eff9415a620 2023-10-08T00:17:25.870 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.865+0000 7eff8affd700 1 -- 172.21.15.99:0/3131196896 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7c011b00 con 0x7eff9415a620 2023-10-08T00:17:25.870 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.865+0000 7eff98e1f700 1 -- 172.21.15.99:0/3131196896 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff94150420 con 0x7eff9415a620 2023-10-08T00:17:25.870 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.866+0000 7eff98e1f700 1 -- 172.21.15.99:0/3131196896 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff94151cd0 con 0x7eff9415a620 2023-10-08T00:17:25.870 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.866+0000 7eff8affd700 1 -- 172.21.15.99:0/3131196896 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7eff7c018360 con 0x7eff9415a620 2023-10-08T00:17:25.870 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.867+0000 7eff8affd700 1 --2- 172.21.15.99:0/3131196896 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7eff74066f30 0x7eff740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:25.870 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.867+0000 7eff8affd700 1 -- 172.21.15.99:0/3131196896 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5705+0+0 (secure 0 0 0) 0x7eff7c0843d0 con 0x7eff9415a620 2023-10-08T00:17:25.870 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.867+0000 7eff9259c700 1 --2- 172.21.15.99:0/3131196896 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7eff74066f30 0x7eff740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:25.870 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.867+0000 7eff9259c700 1 --2- 172.21.15.99:0/3131196896 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7eff74066f30 0x7eff740693e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7eff840099b0 tx=0x7eff84016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:25.871 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:25.871+0000 7eff8affd700 1 -- 172.21.15.99:0/3131196896 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7eff94151cd0 con 0x7eff9415a620 2023-10-08T00:17:26.053 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.053+0000 7eff98e1f700 1 -- 172.21.15.99:0/3131196896 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7eff94151cd0 con 0x7eff9415a620 2023-10-08T00:17:26.054 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.054+0000 7eff8affd700 1 -- 172.21.15.99:0/3131196896 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) v1 ==== 74+0+15372 (secure 0 0 0) 0x7eff94151cd0 con 0x7eff9415a620 2023-10-08T00:17:26.055 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:17:26.055 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":142,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:17:21.117117+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:16:48.430607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-08T00:16:49.063424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:17:26.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.056+0000 7eff98e1f700 1 -- 172.21.15.99:0/3131196896 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7eff74066f30 msgr2=0x7eff740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:26.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.056+0000 7eff98e1f700 1 --2- 172.21.15.99:0/3131196896 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7eff74066f30 0x7eff740693e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7eff840099b0 tx=0x7eff84016000 comp rx=0 tx=0).stop 2023-10-08T00:17:26.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.056+0000 7eff98e1f700 1 -- 172.21.15.99:0/3131196896 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff9415a620 msgr2=0x7eff940b0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:26.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.056+0000 7eff98e1f700 1 --2- 172.21.15.99:0/3131196896 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff9415a620 0x7eff940b0610 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7eff7c00c880 tx=0x7eff7c008040 comp rx=0 tx=0).stop 2023-10-08T00:17:26.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.056+0000 7eff98e1f700 1 -- 172.21.15.99:0/3131196896 shutdown_connections 2023-10-08T00:17:26.058 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.056+0000 7eff98e1f700 1 --2- 172.21.15.99:0/3131196896 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7eff74066f30 0x7eff740693e0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:26.058 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.056+0000 7eff98e1f700 1 --2- 172.21.15.99:0/3131196896 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7eff94157870 0x7eff9414fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:26.058 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.056+0000 7eff98e1f700 1 --2- 172.21.15.99:0/3131196896 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7eff9415d580 0x7eff941566a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:26.058 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.056+0000 7eff98e1f700 1 --2- 172.21.15.99:0/3131196896 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff9415a620 0x7eff940b0610 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:26.058 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.056+0000 7eff98e1f700 1 -- 172.21.15.99:0/3131196896 >> 172.21.15.99:0/3131196896 conn(0x7eff94068ce0 msgr2=0x7eff940690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:26.058 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.056+0000 7eff98e1f700 1 -- 172.21.15.99:0/3131196896 shutdown_connections 2023-10-08T00:17:26.058 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.056+0000 7eff98e1f700 1 -- 172.21.15.99:0/3131196896 wait complete. 2023-10-08T00:17:26.069 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:10:42.063344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T00:16:48.430607+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '138', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T00:16:49.063424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '142', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T00:17:26.069 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T00:17:26.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.187+0000 7ff28c70a700 1 Processor -- start 2023-10-08T00:17:26.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.187+0000 7ff28c70a700 1 -- start start 2023-10-08T00:17:26.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.188+0000 7ff28c70a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff284159ec0 0x7ff2840af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:26.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.188+0000 7ff28c70a700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff28415a290 0x7ff2840afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:26.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.188+0000 7ff28c70a700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2840b0790 con 0x7ff28406a5a0 2023-10-08T00:17:26.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.188+0000 7ff28c70a700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2840ce1f0 con 0x7ff28415a290 2023-10-08T00:17:26.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.188+0000 7ff28c70a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2840b0b20 con 0x7ff284159ec0 2023-10-08T00:17:26.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.188+0000 7ff28aca7700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7ff28406a5a0 0x7ff28415db90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50930/0 (socket says 172.21.15.99:50930) 2023-10-08T00:17:26.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.188+0000 7ff28aca7700 1 -- 172.21.15.99:0/8940614 learned_addr learned my addr 172.21.15.99:0/8940614 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:17:26.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.188+0000 7ff289ca5700 1 --2- 172.21.15.99:0/8940614 >> v2:172.21.15.191:3300/0 conn(0x7ff28415a290 0x7ff2840afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:26.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.188+0000 7ff28a4a6700 1 --2- 172.21.15.99:0/8940614 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff284159ec0 0x7ff2840af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:26.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.188+0000 7ff2894a4700 1 -- 172.21.15.99:0/8940614 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2215148946 0 0) 0x7ff2840b0790 con 0x7ff28406a5a0 2023-10-08T00:17:26.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.188+0000 7ff2894a4700 1 -- 172.21.15.99:0/8940614 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff270003570 con 0x7ff28406a5a0 2023-10-08T00:17:26.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.189+0000 7ff289ca5700 1 -- 172.21.15.99:0/8940614 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff284159ec0 msgr2=0x7ff2840af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:26.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.189+0000 7ff289ca5700 1 --2- 172.21.15.99:0/8940614 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff284159ec0 0x7ff2840af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:26.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.189+0000 7ff2894a4700 1 -- 172.21.15.99:0/8940614 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2694336504 0 0) 0x7ff270003570 con 0x7ff28406a5a0 2023-10-08T00:17:26.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.189+0000 7ff289ca5700 1 -- 172.21.15.99:0/8940614 >> v1:172.21.15.99:6789/0 conn(0x7ff28406a5a0 legacy=0x7ff28415db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:26.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.189+0000 7ff289ca5700 1 -- 172.21.15.99:0/8940614 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2840b04b0 con 0x7ff28415a290 2023-10-08T00:17:26.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.189+0000 7ff28a4a6700 1 --2- 172.21.15.99:0/8940614 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff284159ec0 0x7ff2840af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:17:26.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.189+0000 7ff289ca5700 1 --2- 172.21.15.99:0/8940614 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff28415a290 0x7ff2840afe80 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff2740098e0 tx=0x7ff2740091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=247faccfe256cb37 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:26.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.189+0000 7ff2894a4700 1 -- 172.21.15.99:0/8940614 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27c0017f0 con 0x7ff28415a290 2023-10-08T00:17:26.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.189+0000 7ff2894a4700 1 -- 172.21.15.99:0/8940614 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff274011420 con 0x7ff28415a290 2023-10-08T00:17:26.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.189+0000 7ff2894a4700 1 -- 172.21.15.99:0/8940614 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2740117c0 con 0x7ff28415a290 2023-10-08T00:17:26.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.189+0000 7ff28c70a700 1 -- 172.21.15.99:0/8940614 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff28415a290 msgr2=0x7ff2840afe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:26.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.189+0000 7ff28c70a700 1 --2- 172.21.15.99:0/8940614 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff28415a290 0x7ff2840afe80 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff2740098e0 tx=0x7ff2740091a0 comp rx=0 tx=0).stop 2023-10-08T00:17:26.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.189+0000 7ff28c70a700 1 -- 172.21.15.99:0/8940614 shutdown_connections 2023-10-08T00:17:26.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.189+0000 7ff28c70a700 1 --2- 172.21.15.99:0/8940614 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff284159ec0 0x7ff2840af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:26.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.189+0000 7ff28c70a700 1 --2- 172.21.15.99:0/8940614 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff28415a290 0x7ff2840afe80 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:26.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.189+0000 7ff28c70a700 1 -- 172.21.15.99:0/8940614 >> 172.21.15.99:0/8940614 conn(0x7ff2840d1880 msgr2=0x7ff2840cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:26.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.190+0000 7ff28c70a700 1 -- 172.21.15.99:0/8940614 shutdown_connections 2023-10-08T00:17:26.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.190+0000 7ff28c70a700 1 -- 172.21.15.99:0/8940614 wait complete. 2023-10-08T00:17:26.195 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.191+0000 7ff28c70a700 1 Processor -- start 2023-10-08T00:17:26.195 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.191+0000 7ff28c70a700 1 -- start start 2023-10-08T00:17:26.195 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.191+0000 7ff28c70a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff28406a5a0 0x7ff28415f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:26.195 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.191+0000 7ff28c70a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff284159ec0 0x7ff28416a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:26.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.191+0000 7ff28c70a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff28415a290 0x7ff284154bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:26.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.191+0000 7ff28c70a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28416dc30 con 0x7ff28406a5a0 2023-10-08T00:17:26.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.191+0000 7ff28c70a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28416a930 con 0x7ff28415a290 2023-10-08T00:17:26.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.191+0000 7ff28c70a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff28416abb0 con 0x7ff284159ec0 2023-10-08T00:17:26.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.191+0000 7ff28a4a6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff28406a5a0 0x7ff28415f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:26.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.191+0000 7ff28a4a6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff28406a5a0 0x7ff28415f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45006/0 (socket says 172.21.15.99:45006) 2023-10-08T00:17:26.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.191+0000 7ff28a4a6700 1 -- 172.21.15.99:0/1501568320 learned_addr learned my addr 172.21.15.99:0/1501568320 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:26.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.192+0000 7ff289ca5700 1 --2- 172.21.15.99:0/1501568320 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff284159ec0 0x7ff28416a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:26.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.192+0000 7ff28aca7700 1 --2- 172.21.15.99:0/1501568320 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff28415a290 0x7ff284154bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:26.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.192+0000 7ff28a4a6700 1 -- 172.21.15.99:0/1501568320 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff284159ec0 msgr2=0x7ff28416a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:26.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.192+0000 7ff28a4a6700 1 --2- 172.21.15.99:0/1501568320 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff284159ec0 0x7ff28416a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:26.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.192+0000 7ff28a4a6700 1 -- 172.21.15.99:0/1501568320 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff28415a290 msgr2=0x7ff284154bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:26.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.192+0000 7ff28a4a6700 1 --2- 172.21.15.99:0/1501568320 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff28415a290 0x7ff284154bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:26.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.192+0000 7ff28a4a6700 1 -- 172.21.15.99:0/1501568320 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff280009420 con 0x7ff28406a5a0 2023-10-08T00:17:26.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.192+0000 7ff28aca7700 1 --2- 172.21.15.99:0/1501568320 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff28415a290 0x7ff284154bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:17:26.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.192+0000 7ff28a4a6700 1 --2- 172.21.15.99:0/1501568320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff28406a5a0 0x7ff28415f3a0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7ff28000ec80 tx=0x7ff2800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:26.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.192+0000 7ff27a7fc700 1 -- 172.21.15.99:0/1501568320 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28000c8f0 con 0x7ff28406a5a0 2023-10-08T00:17:26.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.192+0000 7ff27a7fc700 1 -- 172.21.15.99:0/1501568320 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff28000ca50 con 0x7ff28406a5a0 2023-10-08T00:17:26.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.192+0000 7ff27a7fc700 1 -- 172.21.15.99:0/1501568320 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28000cdd0 con 0x7ff28406a5a0 2023-10-08T00:17:26.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.192+0000 7ff28c70a700 1 -- 172.21.15.99:0/1501568320 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff274009420 con 0x7ff28406a5a0 2023-10-08T00:17:26.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.192+0000 7ff28c70a700 1 -- 172.21.15.99:0/1501568320 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff28416b170 con 0x7ff28406a5a0 2023-10-08T00:17:26.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.193+0000 7ff28c70a700 1 -- 172.21.15.99:0/1501568320 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff28416da00 con 0x7ff28406a5a0 2023-10-08T00:17:26.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.194+0000 7ff27a7fc700 1 -- 172.21.15.99:0/1501568320 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff280013680 con 0x7ff28406a5a0 2023-10-08T00:17:26.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.194+0000 7ff27a7fc700 1 --2- 172.21.15.99:0/1501568320 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff268066f30 0x7ff2680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:26.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.194+0000 7ff27a7fc700 1 -- 172.21.15.99:0/1501568320 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff280084c00 con 0x7ff28406a5a0 2023-10-08T00:17:26.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.194+0000 7ff289ca5700 1 --2- 172.21.15.99:0/1501568320 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff268066f30 0x7ff2680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:26.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.194+0000 7ff289ca5700 1 --2- 172.21.15.99:0/1501568320 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff268066f30 0x7ff2680693e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff2740030c0 tx=0x7ff274016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:26.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.198+0000 7ff27a7fc700 1 -- 172.21.15.99:0/1501568320 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff28008b080 con 0x7ff28406a5a0 2023-10-08T00:17:26.379 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.378+0000 7ff28c70a700 1 -- 172.21.15.99:0/1501568320 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff28416b360 con 0x7ff28406a5a0 2023-10-08T00:17:26.379 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.379+0000 7ff27a7fc700 1 -- 172.21.15.99:0/1501568320 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v142) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff2800170a0 con 0x7ff28406a5a0 2023-10-08T00:17:26.379 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2023-10-08T00:17:26.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.381+0000 7ff28c70a700 1 -- 172.21.15.99:0/1501568320 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff268066f30 msgr2=0x7ff2680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:26.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.381+0000 7ff28c70a700 1 --2- 172.21.15.99:0/1501568320 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff268066f30 0x7ff2680693e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff2740030c0 tx=0x7ff274016000 comp rx=0 tx=0).stop 2023-10-08T00:17:26.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.381+0000 7ff28c70a700 1 -- 172.21.15.99:0/1501568320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff28406a5a0 msgr2=0x7ff28415f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:26.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.381+0000 7ff28c70a700 1 --2- 172.21.15.99:0/1501568320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff28406a5a0 0x7ff28415f3a0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7ff28000ec80 tx=0x7ff2800091a0 comp rx=0 tx=0).stop 2023-10-08T00:17:26.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.381+0000 7ff28c70a700 1 -- 172.21.15.99:0/1501568320 shutdown_connections 2023-10-08T00:17:26.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.381+0000 7ff28c70a700 1 --2- 172.21.15.99:0/1501568320 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff268066f30 0x7ff2680693e0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:26.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.381+0000 7ff28c70a700 1 --2- 172.21.15.99:0/1501568320 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff28415a290 0x7ff284154bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:26.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.381+0000 7ff28c70a700 1 --2- 172.21.15.99:0/1501568320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff28406a5a0 0x7ff28415f3a0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:26.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.381+0000 7ff28c70a700 1 --2- 172.21.15.99:0/1501568320 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff284159ec0 0x7ff28416a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:26.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.381+0000 7ff28c70a700 1 -- 172.21.15.99:0/1501568320 >> 172.21.15.99:0/1501568320 conn(0x7ff2840d1880 msgr2=0x7ff284155640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:26.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.381+0000 7ff28c70a700 1 -- 172.21.15.99:0/1501568320 shutdown_connections 2023-10-08T00:17:26.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.381+0000 7ff28c70a700 1 -- 172.21.15.99:0/1501568320 wait complete. 2023-10-08T00:17:26.393 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T00:17:26.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.509+0000 7f203635b700 1 Processor -- start 2023-10-08T00:17:26.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.509+0000 7f203635b700 1 -- start start 2023-10-08T00:17:26.512 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.510+0000 7f203635b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2030159ec0 0x7f20300af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:26.512 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.510+0000 7f203635b700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f203006acb0 0x7f20300afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:26.513 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.510+0000 7f203635b700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20300b0790 con 0x7f203015a290 2023-10-08T00:17:26.513 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.510+0000 7f203635b700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20300ce1f0 con 0x7f203006acb0 2023-10-08T00:17:26.513 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.510+0000 7f203635b700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20300b0b20 con 0x7f2030159ec0 2023-10-08T00:17:26.513 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.510+0000 7f20348f8700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f203006acb0 0x7f20300afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:26.513 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.510+0000 7f202f7fe700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f203015a290 0x7f203006a5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50940/0 (socket says 172.21.15.99:50940) 2023-10-08T00:17:26.513 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.510+0000 7f202f7fe700 1 -- 172.21.15.99:0/1277689476 learned_addr learned my addr 172.21.15.99:0/1277689476 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:17:26.513 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.510+0000 7f202ffff700 1 --2- 172.21.15.99:0/1277689476 >> v2:172.21.15.191:3300/0 conn(0x7f2030159ec0 0x7f20300af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:26.513 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.511+0000 7f20348f8700 1 -- 172.21.15.99:0/1277689476 >> v2:172.21.15.191:3300/0 conn(0x7f2030159ec0 msgr2=0x7f20300af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:26.513 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.511+0000 7f20348f8700 1 --2- 172.21.15.99:0/1277689476 >> v2:172.21.15.191:3300/0 conn(0x7f2030159ec0 0x7f20300af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:26.513 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.511+0000 7f20348f8700 1 -- 172.21.15.99:0/1277689476 >> v1:172.21.15.99:6789/0 conn(0x7f203015a290 legacy=0x7f203006a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:26.513 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.511+0000 7f20348f8700 1 -- 172.21.15.99:0/1277689476 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20300b04b0 con 0x7f203006acb0 2023-10-08T00:17:26.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.511+0000 7f202effd700 1 -- 172.21.15.99:0/1277689476 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1651363615 0 0) 0x7f20300b0790 con 0x7f203015a290 2023-10-08T00:17:26.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.511+0000 7f20348f8700 1 --2- 172.21.15.99:0/1277689476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f203006acb0 0x7f20300afe80 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f20300898e0 tx=0x7f2028009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=426d11f303b7d3fa server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:26.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.511+0000 7f202effd700 1 -- 172.21.15.99:0/1277689476 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202800e070 con 0x7f203006acb0 2023-10-08T00:17:26.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.511+0000 7f202effd700 1 -- 172.21.15.99:0/1277689476 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202800bb60 con 0x7f203006acb0 2023-10-08T00:17:26.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.511+0000 7f202effd700 1 -- 172.21.15.99:0/1277689476 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2028012430 con 0x7f203006acb0 2023-10-08T00:17:26.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.511+0000 7f203635b700 1 -- 172.21.15.99:0/1277689476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f203006acb0 msgr2=0x7f20300afe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:26.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.511+0000 7f203635b700 1 --2- 172.21.15.99:0/1277689476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f203006acb0 0x7f20300afe80 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f20300898e0 tx=0x7f2028009780 comp rx=0 tx=0).stop 2023-10-08T00:17:26.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.512+0000 7f203635b700 1 -- 172.21.15.99:0/1277689476 shutdown_connections 2023-10-08T00:17:26.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.512+0000 7f203635b700 1 --2- 172.21.15.99:0/1277689476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f203006acb0 0x7f20300afe80 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:26.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.512+0000 7f203635b700 1 --2- 172.21.15.99:0/1277689476 >> v2:172.21.15.191:3300/0 conn(0x7f2030159ec0 0x7f20300af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:26.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.512+0000 7f203635b700 1 -- 172.21.15.99:0/1277689476 >> 172.21.15.99:0/1277689476 conn(0x7f20300d1880 msgr2=0x7f20300cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:26.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.512+0000 7f203635b700 1 -- 172.21.15.99:0/1277689476 shutdown_connections 2023-10-08T00:17:26.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.512+0000 7f203635b700 1 -- 172.21.15.99:0/1277689476 wait complete. 2023-10-08T00:17:26.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.513+0000 7f203635b700 1 Processor -- start 2023-10-08T00:17:26.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.513+0000 7f203635b700 1 -- start start 2023-10-08T00:17:26.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.514+0000 7f203635b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f203006acb0 0x7f2030155980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:26.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.514+0000 7f203635b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2030159ec0 0x7f20300b5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:26.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.514+0000 7f203635b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f203015a290 0x7f20300b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:26.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.514+0000 7f203635b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20300c8070 con 0x7f203015a290 2023-10-08T00:17:26.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.514+0000 7f203635b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2030155ec0 con 0x7f203006acb0 2023-10-08T00:17:26.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.514+0000 7f203635b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20300cb1f0 con 0x7f2030159ec0 2023-10-08T00:17:26.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.514+0000 7f20348f8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f203015a290 0x7f20300b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:26.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.514+0000 7f20348f8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f203015a290 0x7f20300b2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45028/0 (socket says 172.21.15.99:45028) 2023-10-08T00:17:26.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.514+0000 7f20348f8700 1 -- 172.21.15.99:0/907479031 learned_addr learned my addr 172.21.15.99:0/907479031 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:26.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.514+0000 7f202ffff700 1 --2- 172.21.15.99:0/907479031 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f203006acb0 0x7f2030155980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:26.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.514+0000 7f202f7fe700 1 --2- 172.21.15.99:0/907479031 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2030159ec0 0x7f20300b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:26.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.514+0000 7f20348f8700 1 -- 172.21.15.99:0/907479031 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2030159ec0 msgr2=0x7f20300b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:26.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.514+0000 7f20348f8700 1 --2- 172.21.15.99:0/907479031 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2030159ec0 0x7f20300b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:26.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.514+0000 7f20348f8700 1 -- 172.21.15.99:0/907479031 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f203006acb0 msgr2=0x7f2030155980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:26.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.514+0000 7f20348f8700 1 --2- 172.21.15.99:0/907479031 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f203006acb0 0x7f2030155980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:26.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.514+0000 7f20348f8700 1 -- 172.21.15.99:0/907479031 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2028009a20 con 0x7f203015a290 2023-10-08T00:17:26.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.514+0000 7f20348f8700 1 --2- 172.21.15.99:0/907479031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f203015a290 0x7f20300b2070 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f2028013000 tx=0x7f2028019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:26.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.514+0000 7f2017fff700 1 -- 172.21.15.99:0/907479031 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2028027470 con 0x7f203015a290 2023-10-08T00:17:26.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.514+0000 7f2017fff700 1 -- 172.21.15.99:0/907479031 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20280275d0 con 0x7f203015a290 2023-10-08T00:17:26.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.514+0000 7f203635b700 1 -- 172.21.15.99:0/907479031 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20300cb4d0 con 0x7f203015a290 2023-10-08T00:17:26.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.514+0000 7f2017fff700 1 -- 172.21.15.99:0/907479031 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2028027990 con 0x7f203015a290 2023-10-08T00:17:26.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.514+0000 7f203635b700 1 -- 172.21.15.99:0/907479031 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20300b6650 con 0x7f203015a290 2023-10-08T00:17:26.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.515+0000 7f203635b700 1 -- 172.21.15.99:0/907479031 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f203004c3c0 con 0x7f203015a290 2023-10-08T00:17:26.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.516+0000 7f2017fff700 1 -- 172.21.15.99:0/907479031 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2028024070 con 0x7f203015a290 2023-10-08T00:17:26.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.516+0000 7f2017fff700 1 --2- 172.21.15.99:0/907479031 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f201c066f30 0x7f201c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:26.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.516+0000 7f2017fff700 1 -- 172.21.15.99:0/907479031 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2028086e00 con 0x7f203015a290 2023-10-08T00:17:26.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.516+0000 7f202ffff700 1 --2- 172.21.15.99:0/907479031 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f201c066f30 0x7f201c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:26.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.517+0000 7f202ffff700 1 --2- 172.21.15.99:0/907479031 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f201c066f30 0x7f201c0693e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f2020009db0 tx=0x7f20200091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:26.521 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.520+0000 7f2017fff700 1 -- 172.21.15.99:0/907479031 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2028054070 con 0x7f203015a290 2023-10-08T00:17:26.703 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.702+0000 7f203635b700 1 -- 172.21.15.99:0/907479031 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f20300ceab0 con 0x7f203015a290 2023-10-08T00:17:26.703 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.703+0000 7f2017fff700 1 -- 172.21.15.99:0/907479031 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v142) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2028009880 con 0x7f203015a290 2023-10-08T00:17:26.703 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2023-10-08T00:17:26.706 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.704+0000 7f203635b700 1 -- 172.21.15.99:0/907479031 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f201c066f30 msgr2=0x7f201c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:26.706 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.704+0000 7f203635b700 1 --2- 172.21.15.99:0/907479031 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f201c066f30 0x7f201c0693e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f2020009db0 tx=0x7f20200091a0 comp rx=0 tx=0).stop 2023-10-08T00:17:26.706 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.704+0000 7f203635b700 1 -- 172.21.15.99:0/907479031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f203015a290 msgr2=0x7f20300b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:26.706 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.704+0000 7f203635b700 1 --2- 172.21.15.99:0/907479031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f203015a290 0x7f20300b2070 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f2028013000 tx=0x7f2028019000 comp rx=0 tx=0).stop 2023-10-08T00:17:26.707 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.704+0000 7f203635b700 1 -- 172.21.15.99:0/907479031 shutdown_connections 2023-10-08T00:17:26.707 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.704+0000 7f203635b700 1 --2- 172.21.15.99:0/907479031 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f201c066f30 0x7f201c0693e0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:26.707 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.704+0000 7f203635b700 1 --2- 172.21.15.99:0/907479031 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f203006acb0 0x7f2030155980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:26.707 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.704+0000 7f203635b700 1 --2- 172.21.15.99:0/907479031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f203015a290 0x7f20300b2070 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:26.707 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.704+0000 7f203635b700 1 --2- 172.21.15.99:0/907479031 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2030159ec0 0x7f20300b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:26.708 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.704+0000 7f203635b700 1 -- 172.21.15.99:0/907479031 >> 172.21.15.99:0/907479031 conn(0x7f20300d1880 msgr2=0x7f2030156ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:26.708 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.705+0000 7f203635b700 1 -- 172.21.15.99:0/907479031 shutdown_connections 2023-10-08T00:17:26.708 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.705+0000 7f203635b700 1 -- 172.21.15.99:0/907479031 wait complete. 2023-10-08T00:17:26.717 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T00:17:26.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.833+0000 7f0d96bb4700 1 Processor -- start 2023-10-08T00:17:26.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.833+0000 7f0d96bb4700 1 -- start start 2023-10-08T00:17:26.836 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.834+0000 7f0d96bb4700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f0d9015a290 0x7f0d9006a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:26.836 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.834+0000 7f0d96bb4700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0d9006aae0 0x7f0d900cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:26.836 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.834+0000 7f0d96bb4700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d900b0790 con 0x7f0d90159ec0 2023-10-08T00:17:26.836 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.834+0000 7f0d96bb4700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d9006af20 con 0x7f0d9006aae0 2023-10-08T00:17:26.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.834+0000 7f0d96bb4700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d900ce1f0 con 0x7f0d9015a290 2023-10-08T00:17:26.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.834+0000 7f0d95151700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0d9006aae0 0x7f0d900cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:26.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.834+0000 7f0d94950700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f0d90159ec0 0x7f0d900af940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50944/0 (socket says 172.21.15.99:50944) 2023-10-08T00:17:26.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.834+0000 7f0d94950700 1 -- 172.21.15.99:0/1872156309 learned_addr learned my addr 172.21.15.99:0/1872156309 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:17:26.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.834+0000 7f0d95151700 1 -- 172.21.15.99:0/1872156309 >> v2:172.21.15.191:3300/0 conn(0x7f0d9015a290 msgr2=0x7f0d9006a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:26.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.834+0000 7f0d87fff700 1 --2- 172.21.15.99:0/1872156309 >> v2:172.21.15.191:3300/0 conn(0x7f0d9015a290 0x7f0d9006a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:26.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.834+0000 7f0d877fe700 1 -- 172.21.15.99:0/1872156309 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2178025382 0 0) 0x7f0d900b0790 con 0x7f0d90159ec0 2023-10-08T00:17:26.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.834+0000 7f0d95151700 1 --2- 172.21.15.99:0/1872156309 >> v2:172.21.15.191:3300/0 conn(0x7f0d9015a290 0x7f0d9006a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:26.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.835+0000 7f0d95151700 1 -- 172.21.15.99:0/1872156309 >> v1:172.21.15.99:6789/0 conn(0x7f0d90159ec0 legacy=0x7f0d900af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:26.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.835+0000 7f0d95151700 1 -- 172.21.15.99:0/1872156309 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d900b0b20 con 0x7f0d9006aae0 2023-10-08T00:17:26.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.835+0000 7f0d95151700 1 --2- 172.21.15.99:0/1872156309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d9006aae0 0x7f0d900cf590 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f0d90089910 tx=0x7f0d88009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa3790f3a33f3cad server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:26.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.835+0000 7f0d877fe700 1 -- 172.21.15.99:0/1872156309 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8800e070 con 0x7f0d9006aae0 2023-10-08T00:17:26.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.835+0000 7f0d877fe700 1 -- 172.21.15.99:0/1872156309 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d8800bc30 con 0x7f0d9006aae0 2023-10-08T00:17:26.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.835+0000 7f0d877fe700 1 -- 172.21.15.99:0/1872156309 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d88012430 con 0x7f0d9006aae0 2023-10-08T00:17:26.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.835+0000 7f0d96bb4700 1 -- 172.21.15.99:0/1872156309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d9006aae0 msgr2=0x7f0d900cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:26.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.835+0000 7f0d96bb4700 1 --2- 172.21.15.99:0/1872156309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d9006aae0 0x7f0d900cf590 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f0d90089910 tx=0x7f0d88009780 comp rx=0 tx=0).stop 2023-10-08T00:17:26.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.835+0000 7f0d96bb4700 1 -- 172.21.15.99:0/1872156309 shutdown_connections 2023-10-08T00:17:26.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.835+0000 7f0d96bb4700 1 --2- 172.21.15.99:0/1872156309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d9006aae0 0x7f0d900cf590 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:26.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.835+0000 7f0d96bb4700 1 --2- 172.21.15.99:0/1872156309 >> v2:172.21.15.191:3300/0 conn(0x7f0d9015a290 0x7f0d9006a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:26.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.835+0000 7f0d96bb4700 1 -- 172.21.15.99:0/1872156309 >> 172.21.15.99:0/1872156309 conn(0x7f0d900d1880 msgr2=0x7f0d900cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:26.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.835+0000 7f0d96bb4700 1 -- 172.21.15.99:0/1872156309 shutdown_connections 2023-10-08T00:17:26.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.835+0000 7f0d96bb4700 1 -- 172.21.15.99:0/1872156309 wait complete. 2023-10-08T00:17:26.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.836+0000 7f0d96bb4700 1 Processor -- start 2023-10-08T00:17:26.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.837+0000 7f0d96bb4700 1 -- start start 2023-10-08T00:17:26.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.837+0000 7f0d96bb4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d9006aae0 0x7f0d900b6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:26.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.837+0000 7f0d96bb4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d90159ec0 0x7f0d900b4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:26.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.837+0000 7f0d96bb4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d9015a290 0x7f0d900ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:26.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.837+0000 7f0d96bb4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d900b4aa0 con 0x7f0d9015a290 2023-10-08T00:17:26.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.837+0000 7f0d96bb4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d900ca250 con 0x7f0d9006aae0 2023-10-08T00:17:26.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.837+0000 7f0d96bb4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d900b4cd0 con 0x7f0d90159ec0 2023-10-08T00:17:26.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.837+0000 7f0d95151700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d9015a290 0x7f0d900ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:26.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.837+0000 7f0d95151700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d9015a290 0x7f0d900ca6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45048/0 (socket says 172.21.15.99:45048) 2023-10-08T00:17:26.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.837+0000 7f0d95151700 1 -- 172.21.15.99:0/1706375945 learned_addr learned my addr 172.21.15.99:0/1706375945 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:26.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.838+0000 7f0d94950700 1 --2- 172.21.15.99:0/1706375945 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d9006aae0 0x7f0d900b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:26.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.838+0000 7f0d87fff700 1 --2- 172.21.15.99:0/1706375945 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d90159ec0 0x7f0d900b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:26.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.838+0000 7f0d95151700 1 -- 172.21.15.99:0/1706375945 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d90159ec0 msgr2=0x7f0d900b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:26.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.838+0000 7f0d95151700 1 --2- 172.21.15.99:0/1706375945 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d90159ec0 0x7f0d900b4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:26.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.838+0000 7f0d95151700 1 -- 172.21.15.99:0/1706375945 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d9006aae0 msgr2=0x7f0d900b6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:26.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.838+0000 7f0d95151700 1 --2- 172.21.15.99:0/1706375945 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d9006aae0 0x7f0d900b6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:26.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.838+0000 7f0d95151700 1 -- 172.21.15.99:0/1706375945 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d88009a20 con 0x7f0d9015a290 2023-10-08T00:17:26.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.838+0000 7f0d95151700 1 --2- 172.21.15.99:0/1706375945 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d9015a290 0x7f0d900ca6e0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f0d880097c0 tx=0x7f0d88019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:26.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.838+0000 7f0d84ff9700 1 -- 172.21.15.99:0/1706375945 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8801e630 con 0x7f0d9015a290 2023-10-08T00:17:26.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.838+0000 7f0d84ff9700 1 -- 172.21.15.99:0/1706375945 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d8801e790 con 0x7f0d9015a290 2023-10-08T00:17:26.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.838+0000 7f0d96bb4700 1 -- 172.21.15.99:0/1706375945 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d900c9920 con 0x7f0d9015a290 2023-10-08T00:17:26.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.838+0000 7f0d84ff9700 1 -- 172.21.15.99:0/1706375945 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8801eb50 con 0x7f0d9015a290 2023-10-08T00:17:26.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.838+0000 7f0d96bb4700 1 -- 172.21.15.99:0/1706375945 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d900c9ee0 con 0x7f0d9015a290 2023-10-08T00:17:26.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.839+0000 7f0d96bb4700 1 -- 172.21.15.99:0/1706375945 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d900ca400 con 0x7f0d9015a290 2023-10-08T00:17:26.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.839+0000 7f0d84ff9700 1 -- 172.21.15.99:0/1706375945 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0d88024070 con 0x7f0d9015a290 2023-10-08T00:17:26.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.840+0000 7f0d84ff9700 1 --2- 172.21.15.99:0/1706375945 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0d7c066f30 0x7f0d7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:26.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.840+0000 7f0d84ff9700 1 -- 172.21.15.99:0/1706375945 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0d88086e10 con 0x7f0d9015a290 2023-10-08T00:17:26.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.840+0000 7f0d94950700 1 --2- 172.21.15.99:0/1706375945 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0d7c066f30 0x7f0d7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:26.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.840+0000 7f0d94950700 1 --2- 172.21.15.99:0/1706375945 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0d7c066f30 0x7f0d7c0693e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c0011e0 tx=0x7f0d8c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:26.845 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:26.844+0000 7f0d84ff9700 1 -- 172.21.15.99:0/1706375945 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0d88054080 con 0x7f0d9015a290 2023-10-08T00:17:27.024 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.024+0000 7f0d96bb4700 1 -- 172.21.15.99:0/1706375945 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0d9004c3c0 con 0x7f0d9015a290 2023-10-08T00:17:27.025 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.024+0000 7f0d84ff9700 1 -- 172.21.15.99:0/1706375945 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v142) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0d8802c020 con 0x7f0d9015a290 2023-10-08T00:17:27.025 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2023-10-08T00:17:27.027 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.026+0000 7f0d96bb4700 1 -- 172.21.15.99:0/1706375945 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0d7c066f30 msgr2=0x7f0d7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:27.027 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.026+0000 7f0d96bb4700 1 --2- 172.21.15.99:0/1706375945 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0d7c066f30 0x7f0d7c0693e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c0011e0 tx=0x7f0d8c009040 comp rx=0 tx=0).stop 2023-10-08T00:17:27.027 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.026+0000 7f0d96bb4700 1 -- 172.21.15.99:0/1706375945 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d9015a290 msgr2=0x7f0d900ca6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:27.028 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.026+0000 7f0d96bb4700 1 --2- 172.21.15.99:0/1706375945 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d9015a290 0x7f0d900ca6e0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f0d880097c0 tx=0x7f0d88019000 comp rx=0 tx=0).stop 2023-10-08T00:17:27.028 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.026+0000 7f0d96bb4700 1 -- 172.21.15.99:0/1706375945 shutdown_connections 2023-10-08T00:17:27.028 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.026+0000 7f0d96bb4700 1 --2- 172.21.15.99:0/1706375945 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0d7c066f30 0x7f0d7c0693e0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:27.028 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.026+0000 7f0d96bb4700 1 --2- 172.21.15.99:0/1706375945 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d9006aae0 0x7f0d900b6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:27.028 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.026+0000 7f0d96bb4700 1 --2- 172.21.15.99:0/1706375945 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d9015a290 0x7f0d900ca6e0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:27.028 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.026+0000 7f0d96bb4700 1 --2- 172.21.15.99:0/1706375945 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d90159ec0 0x7f0d900b4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:27.028 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.026+0000 7f0d96bb4700 1 -- 172.21.15.99:0/1706375945 >> 172.21.15.99:0/1706375945 conn(0x7f0d900d1880 msgr2=0x7f0d900c6370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:27.028 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.027+0000 7f0d96bb4700 1 -- 172.21.15.99:0/1706375945 shutdown_connections 2023-10-08T00:17:27.028 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.027+0000 7f0d96bb4700 1 -- 172.21.15.99:0/1706375945 wait complete. 2023-10-08T00:17:27.039 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:17:27.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.152+0000 7f65601f5700 1 Processor -- start 2023-10-08T00:17:27.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.152+0000 7f65601f5700 1 -- start start 2023-10-08T00:17:27.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.153+0000 7f65601f5700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f6558157870 0x7f655815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:27.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.153+0000 7f65601f5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f655815a620 0x7f655815d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:27.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.153+0000 7f65601f5700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f655815cbb0 con 0x7f655815d580 2023-10-08T00:17:27.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.153+0000 7f65601f5700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f655815bfa0 con 0x7f6558157870 2023-10-08T00:17:27.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.153+0000 7f65601f5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f655815c280 con 0x7f655815a620 2023-10-08T00:17:27.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.153+0000 7f655e792700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f655815d580 0x7f6558158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50946/0 (socket says 172.21.15.99:50946) 2023-10-08T00:17:27.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.153+0000 7f655e792700 1 -- 172.21.15.99:0/1418773603 learned_addr learned my addr 172.21.15.99:0/1418773603 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:17:27.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.153+0000 7f655df91700 1 --2- 172.21.15.99:0/1418773603 >> v2:172.21.15.191:3300/0 conn(0x7f6558157870 0x7f655815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:27.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.153+0000 7f655d790700 1 --2- 172.21.15.99:0/1418773603 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f655815a620 0x7f655815d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:27.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.154+0000 7f655cf8f700 1 -- 172.21.15.99:0/1418773603 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1849403536 0 0) 0x7f655815cbb0 con 0x7f655815d580 2023-10-08T00:17:27.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.154+0000 7f655cf8f700 1 -- 172.21.15.99:0/1418773603 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6544003570 con 0x7f655815d580 2023-10-08T00:17:27.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.154+0000 7f655df91700 1 -- 172.21.15.99:0/1418773603 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f655815a620 msgr2=0x7f655815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:27.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.154+0000 7f655cf8f700 1 -- 172.21.15.99:0/1418773603 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2458223309 0 0) 0x7f6544003570 con 0x7f655815d580 2023-10-08T00:17:27.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.154+0000 7f655df91700 1 --2- 172.21.15.99:0/1418773603 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f655815a620 0x7f655815d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:27.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.154+0000 7f655df91700 1 -- 172.21.15.99:0/1418773603 >> v1:172.21.15.99:6789/0 conn(0x7f655815d580 legacy=0x7f6558158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:27.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.154+0000 7f655df91700 1 -- 172.21.15.99:0/1418773603 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f655815c560 con 0x7f6558157870 2023-10-08T00:17:27.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.154+0000 7f655d790700 1 --2- 172.21.15.99:0/1418773603 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f655815a620 0x7f655815d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:17:27.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.154+0000 7f655df91700 1 --2- 172.21.15.99:0/1418773603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6558157870 0x7f655815b6a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6554009e60 tx=0x7f6554009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67fe1d6618816466 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:27.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.154+0000 7f655cf8f700 1 -- 172.21.15.99:0/1418773603 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655400e070 con 0x7f6558157870 2023-10-08T00:17:27.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.154+0000 7f655cf8f700 1 -- 172.21.15.99:0/1418773603 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f655400b7a0 con 0x7f6558157870 2023-10-08T00:17:27.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.154+0000 7f655cf8f700 1 -- 172.21.15.99:0/1418773603 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6550001470 con 0x7f6558157870 2023-10-08T00:17:27.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.155+0000 7f65601f5700 1 -- 172.21.15.99:0/1418773603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6558157870 msgr2=0x7f655815b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:27.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.155+0000 7f65601f5700 1 --2- 172.21.15.99:0/1418773603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6558157870 0x7f655815b6a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6554009e60 tx=0x7f6554009780 comp rx=0 tx=0).stop 2023-10-08T00:17:27.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.155+0000 7f65601f5700 1 -- 172.21.15.99:0/1418773603 shutdown_connections 2023-10-08T00:17:27.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.155+0000 7f65601f5700 1 --2- 172.21.15.99:0/1418773603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6558157870 0x7f655815b6a0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:27.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.155+0000 7f65601f5700 1 --2- 172.21.15.99:0/1418773603 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f655815a620 0x7f655815d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:27.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.155+0000 7f65601f5700 1 -- 172.21.15.99:0/1418773603 >> 172.21.15.99:0/1418773603 conn(0x7f6558068ce0 msgr2=0x7f65580690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:27.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.155+0000 7f65601f5700 1 -- 172.21.15.99:0/1418773603 shutdown_connections 2023-10-08T00:17:27.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.155+0000 7f65601f5700 1 -- 172.21.15.99:0/1418773603 wait complete. 2023-10-08T00:17:27.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.156+0000 7f65601f5700 1 Processor -- start 2023-10-08T00:17:27.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.156+0000 7f65601f5700 1 -- start start 2023-10-08T00:17:27.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.156+0000 7f65601f5700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6558157870 0x7f6558147790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:27.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.157+0000 7f65601f5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f655815a620 0x7f65580b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:27.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.157+0000 7f65601f5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f655815d580 0x7f65580a7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:27.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.157+0000 7f65601f5700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65580b1340 con 0x7f655815d580 2023-10-08T00:17:27.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.157+0000 7f65601f5700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65580b1ba0 con 0x7f6558157870 2023-10-08T00:17:27.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.157+0000 7f65601f5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65580b66d0 con 0x7f655815a620 2023-10-08T00:17:27.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.157+0000 7f655e792700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f655815d580 0x7f65580a7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:27.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.157+0000 7f655e792700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f655815d580 0x7f65580a7e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45064/0 (socket says 172.21.15.99:45064) 2023-10-08T00:17:27.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.157+0000 7f655e792700 1 -- 172.21.15.99:0/1944358587 learned_addr learned my addr 172.21.15.99:0/1944358587 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:27.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.157+0000 7f655e792700 1 -- 172.21.15.99:0/1944358587 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f655815a620 msgr2=0x7f65580b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:27.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.157+0000 7f655d790700 1 --2- 172.21.15.99:0/1944358587 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f655815a620 0x7f65580b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:27.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.157+0000 7f655e792700 1 --2- 172.21.15.99:0/1944358587 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f655815a620 0x7f65580b5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:27.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.157+0000 7f655df91700 1 --2- 172.21.15.99:0/1944358587 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6558157870 0x7f6558147790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:27.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.157+0000 7f655e792700 1 -- 172.21.15.99:0/1944358587 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6558157870 msgr2=0x7f6558147790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:27.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.157+0000 7f655e792700 1 --2- 172.21.15.99:0/1944358587 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6558157870 0x7f6558147790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:27.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.157+0000 7f655e792700 1 -- 172.21.15.99:0/1944358587 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6548009420 con 0x7f655815d580 2023-10-08T00:17:27.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.157+0000 7f655e792700 1 --2- 172.21.15.99:0/1944358587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f655815d580 0x7f65580a7e20 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f655000cb60 tx=0x7f6550001aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:27.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.157+0000 7f654dffb700 1 -- 172.21.15.99:0/1944358587 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6550007150 con 0x7f655815d580 2023-10-08T00:17:27.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.157+0000 7f654dffb700 1 -- 172.21.15.99:0/1944358587 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65500072d0 con 0x7f655815d580 2023-10-08T00:17:27.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.157+0000 7f65601f5700 1 -- 172.21.15.99:0/1944358587 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6554009a20 con 0x7f655815d580 2023-10-08T00:17:27.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.157+0000 7f654dffb700 1 -- 172.21.15.99:0/1944358587 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6550007690 con 0x7f655815d580 2023-10-08T00:17:27.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.157+0000 7f65601f5700 1 -- 172.21.15.99:0/1944358587 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65580b1830 con 0x7f655815d580 2023-10-08T00:17:27.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.158+0000 7f65601f5700 1 -- 172.21.15.99:0/1944358587 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65580b1d50 con 0x7f655815d580 2023-10-08T00:17:27.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.159+0000 7f654dffb700 1 -- 172.21.15.99:0/1944358587 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6550004a40 con 0x7f655815d580 2023-10-08T00:17:27.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.159+0000 7f654dffb700 1 --2- 172.21.15.99:0/1944358587 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f653c066f30 0x7f653c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:27.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.159+0000 7f654dffb700 1 -- 172.21.15.99:0/1944358587 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6550083d40 con 0x7f655815d580 2023-10-08T00:17:27.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.159+0000 7f655df91700 1 --2- 172.21.15.99:0/1944358587 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f653c066f30 0x7f653c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:27.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.160+0000 7f655df91700 1 --2- 172.21.15.99:0/1944358587 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f653c066f30 0x7f653c0693e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f65540099a0 tx=0x7f6554018000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:27.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.163+0000 7f654dffb700 1 -- 172.21.15.99:0/1944358587 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6550051030 con 0x7f655815d580 2023-10-08T00:17:27.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.387+0000 7f65601f5700 1 -- 172.21.15.99:0/1944358587 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f655804c3c0 con 0x7f655815d580 2023-10-08T00:17:27.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.388+0000 7f654dffb700 1 -- 172.21.15.99:0/1944358587 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f6550018020 con 0x7f655815d580 2023-10-08T00:17:27.388 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:17:27.389 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":63,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5813,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/887374839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":887374839},{"type":"v1","addr":"172.21.15.191:6837","nonce":887374839}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":6445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/2409696901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2409696901},{"type":"v1","addr":"172.21.15.191:6836","nonce":2409696901}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":7095,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/1924096228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1924096228},{"type":"v1","addr":"172.21.15.99:6834","nonce":1924096228}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:16:50.092494+0000","modified":"2023-10-08T00:16:53.744105+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7098":{"gid":7098,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/2522933027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":2522933027},{"type":"v1","addr":"172.21.15.99:6835","nonce":2522933027}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-08T00:17:27.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.390+0000 7f65601f5700 1 -- 172.21.15.99:0/1944358587 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f653c066f30 msgr2=0x7f653c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:27.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.390+0000 7f65601f5700 1 --2- 172.21.15.99:0/1944358587 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f653c066f30 0x7f653c0693e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f65540099a0 tx=0x7f6554018000 comp rx=0 tx=0).stop 2023-10-08T00:17:27.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.390+0000 7f65601f5700 1 -- 172.21.15.99:0/1944358587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f655815d580 msgr2=0x7f65580a7e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:27.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.390+0000 7f65601f5700 1 --2- 172.21.15.99:0/1944358587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f655815d580 0x7f65580a7e20 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f655000cb60 tx=0x7f6550001aa0 comp rx=0 tx=0).stop 2023-10-08T00:17:27.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.390+0000 7f65601f5700 1 -- 172.21.15.99:0/1944358587 shutdown_connections 2023-10-08T00:17:27.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.390+0000 7f65601f5700 1 --2- 172.21.15.99:0/1944358587 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f653c066f30 0x7f653c0693e0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:27.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.390+0000 7f65601f5700 1 --2- 172.21.15.99:0/1944358587 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6558157870 0x7f6558147790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:27.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.390+0000 7f65601f5700 1 --2- 172.21.15.99:0/1944358587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f655815d580 0x7f65580a7e20 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:27.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.390+0000 7f65601f5700 1 --2- 172.21.15.99:0/1944358587 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f655815a620 0x7f65580b5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:27.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.390+0000 7f65601f5700 1 -- 172.21.15.99:0/1944358587 >> 172.21.15.99:0/1944358587 conn(0x7f6558068ce0 msgr2=0x7f655815aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:27.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.390+0000 7f65601f5700 1 -- 172.21.15.99:0/1944358587 shutdown_connections 2023-10-08T00:17:27.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.390+0000 7f65601f5700 1 -- 172.21.15.99:0/1944358587 wait complete. 2023-10-08T00:17:27.393 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 63 2023-10-08T00:17:27.402 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:17:27.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.515+0000 7efe44e76700 1 Processor -- start 2023-10-08T00:17:27.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.515+0000 7efe44e76700 1 -- start start 2023-10-08T00:17:27.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.516+0000 7efe44e76700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7efe40152010 0x7efe4015b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:27.518 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.516+0000 7efe44e76700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe40158e50 0x7efe400cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:27.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.516+0000 7efe44e76700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe4006c010 con 0x7efe401523e0 2023-10-08T00:17:27.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.516+0000 7efe44e76700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe4006bd10 con 0x7efe40152010 2023-10-08T00:17:27.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.516+0000 7efe44e76700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe4015c060 con 0x7efe40158e50 2023-10-08T00:17:27.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.516+0000 7efe3e59c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7efe40152010 0x7efe4015b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:27.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.516+0000 7efe3ed9d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe40158e50 0x7efe400cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:27.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.516+0000 7efe35d9b700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7efe401523e0 0x7efe4006c4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:58748/0 (socket says 172.21.15.99:58748) 2023-10-08T00:17:27.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.517+0000 7efe3ed9d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe40158e50 0x7efe400cdf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.99:36892/0 (socket says 172.21.15.99:36892) 2023-10-08T00:17:27.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.517+0000 7efe35d9b700 1 -- 172.21.15.99:0/1029863798 learned_addr learned my addr 172.21.15.99:0/1029863798 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:17:27.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.517+0000 7efe3e59c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7efe40152010 0x7efe4015b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.99:56156/0 (socket says 172.21.15.99:56156) 2023-10-08T00:17:27.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.517+0000 7efe3ed9d700 1 -- 172.21.15.99:0/1029863798 >> v2:172.21.15.191:3300/0 conn(0x7efe40152010 msgr2=0x7efe4015b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:27.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.517+0000 7efe3ed9d700 1 --2- 172.21.15.99:0/1029863798 >> v2:172.21.15.191:3300/0 conn(0x7efe40152010 0x7efe4015b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:27.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.517+0000 7efe3ed9d700 1 -- 172.21.15.99:0/1029863798 >> v1:172.21.15.191:6789/0 conn(0x7efe401523e0 legacy=0x7efe4006c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:27.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.517+0000 7efe3ed9d700 1 -- 172.21.15.99:0/1029863798 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe4015c340 con 0x7efe40158e50 2023-10-08T00:17:27.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.517+0000 7efe3e59c700 1 --2- 172.21.15.99:0/1029863798 >> v2:172.21.15.191:3300/0 conn(0x7efe40152010 0x7efe4015b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:17:27.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.517+0000 7efe3ed9d700 1 --2- 172.21.15.99:0/1029863798 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe40158e50 0x7efe400cdf30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efe38009e60 tx=0x7efe38009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb03bd62f9023b3b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:27.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.517+0000 7efe3dd9b700 1 -- 172.21.15.99:0/1029863798 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3800e070 con 0x7efe40158e50 2023-10-08T00:17:27.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.517+0000 7efe3dd9b700 1 -- 172.21.15.99:0/1029863798 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe3800bb60 con 0x7efe40158e50 2023-10-08T00:17:27.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.517+0000 7efe3dd9b700 1 -- 172.21.15.99:0/1029863798 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe38012430 con 0x7efe40158e50 2023-10-08T00:17:27.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.518+0000 7efe44e76700 1 -- 172.21.15.99:0/1029863798 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe40158e50 msgr2=0x7efe400cdf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:27.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.518+0000 7efe44e76700 1 --2- 172.21.15.99:0/1029863798 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe40158e50 0x7efe400cdf30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efe38009e60 tx=0x7efe38009780 comp rx=0 tx=0).stop 2023-10-08T00:17:27.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.518+0000 7efe44e76700 1 -- 172.21.15.99:0/1029863798 shutdown_connections 2023-10-08T00:17:27.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.518+0000 7efe44e76700 1 --2- 172.21.15.99:0/1029863798 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe40158e50 0x7efe400cdf30 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:27.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.518+0000 7efe44e76700 1 --2- 172.21.15.99:0/1029863798 >> v2:172.21.15.191:3300/0 conn(0x7efe40152010 0x7efe4015b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:27.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.518+0000 7efe44e76700 1 -- 172.21.15.99:0/1029863798 >> 172.21.15.99:0/1029863798 conn(0x7efe40069870 msgr2=0x7efe40069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:27.521 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.518+0000 7efe44e76700 1 -- 172.21.15.99:0/1029863798 shutdown_connections 2023-10-08T00:17:27.521 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.518+0000 7efe44e76700 1 -- 172.21.15.99:0/1029863798 wait complete. 2023-10-08T00:17:27.523 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.519+0000 7efe44e76700 1 Processor -- start 2023-10-08T00:17:27.523 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.519+0000 7efe44e76700 1 -- start start 2023-10-08T00:17:27.523 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.520+0000 7efe44e76700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe40152010 0x7efe40150ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:27.523 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.520+0000 7efe44e76700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe401523e0 0x7efe401573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:27.523 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.520+0000 7efe44e76700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe40158e50 0x7efe400b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:27.524 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.520+0000 7efe44e76700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe400b2860 con 0x7efe401523e0 2023-10-08T00:17:27.524 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.520+0000 7efe44e76700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe40150ff0 con 0x7efe40152010 2023-10-08T00:17:27.524 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.520+0000 7efe44e76700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe400b1c50 con 0x7efe40158e50 2023-10-08T00:17:27.524 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.520+0000 7efe35d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe401523e0 0x7efe401573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:27.524 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.520+0000 7efe35d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe401523e0 0x7efe401573e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45072/0 (socket says 172.21.15.99:45072) 2023-10-08T00:17:27.524 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.520+0000 7efe35d9b700 1 -- 172.21.15.99:0/2896391502 learned_addr learned my addr 172.21.15.99:0/2896391502 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:27.524 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.520+0000 7efe3e59c700 1 --2- 172.21.15.99:0/2896391502 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe40152010 0x7efe40150ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:27.524 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.520+0000 7efe3ed9d700 1 --2- 172.21.15.99:0/2896391502 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe40158e50 0x7efe400b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:27.524 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.520+0000 7efe35d9b700 1 -- 172.21.15.99:0/2896391502 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe40158e50 msgr2=0x7efe400b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:27.524 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.520+0000 7efe35d9b700 1 --2- 172.21.15.99:0/2896391502 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe40158e50 0x7efe400b2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:27.524 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.520+0000 7efe35d9b700 1 -- 172.21.15.99:0/2896391502 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe40152010 msgr2=0x7efe40150ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:27.524 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.520+0000 7efe35d9b700 1 --2- 172.21.15.99:0/2896391502 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe40152010 0x7efe40150ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:27.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.520+0000 7efe35d9b700 1 -- 172.21.15.99:0/2896391502 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe30009420 con 0x7efe401523e0 2023-10-08T00:17:27.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.520+0000 7efe3e59c700 1 --2- 172.21.15.99:0/2896391502 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe40152010 0x7efe40150ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:17:27.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.520+0000 7efe35d9b700 1 --2- 172.21.15.99:0/2896391502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe401523e0 0x7efe401573e0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7efe2800c7c0 tx=0x7efe28007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:27.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.520+0000 7efe36ffd700 1 -- 172.21.15.99:0/2896391502 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe28005b30 con 0x7efe401523e0 2023-10-08T00:17:27.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.521+0000 7efe36ffd700 1 -- 172.21.15.99:0/2896391502 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe28005c90 con 0x7efe401523e0 2023-10-08T00:17:27.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.521+0000 7efe36ffd700 1 -- 172.21.15.99:0/2896391502 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe28004a80 con 0x7efe401523e0 2023-10-08T00:17:27.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.521+0000 7efe44e76700 1 -- 172.21.15.99:0/2896391502 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe38009a20 con 0x7efe401523e0 2023-10-08T00:17:27.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.521+0000 7efe44e76700 1 -- 172.21.15.99:0/2896391502 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe400b2210 con 0x7efe401523e0 2023-10-08T00:17:27.526 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.521+0000 7efe44e76700 1 -- 172.21.15.99:0/2896391502 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe4004c3c0 con 0x7efe401523e0 2023-10-08T00:17:27.526 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.522+0000 7efe36ffd700 1 -- 172.21.15.99:0/2896391502 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efe280070d0 con 0x7efe401523e0 2023-10-08T00:17:27.526 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.522+0000 7efe36ffd700 1 --2- 172.21.15.99:0/2896391502 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efe20066f30 0x7efe200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:27.526 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.522+0000 7efe36ffd700 1 -- 172.21.15.99:0/2896391502 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5705+0+0 (secure 0 0 0) 0x7efe28083820 con 0x7efe401523e0 2023-10-08T00:17:27.526 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.522+0000 7efe3e59c700 1 --2- 172.21.15.99:0/2896391502 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efe20066f30 0x7efe200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:27.526 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.523+0000 7efe3e59c700 1 --2- 172.21.15.99:0/2896391502 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efe20066f30 0x7efe200693e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7efe30011600 tx=0x7efe300091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:27.527 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.527+0000 7efe36ffd700 1 -- 172.21.15.99:0/2896391502 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efe2804b5a0 con 0x7efe401523e0 2023-10-08T00:17:27.708 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.708+0000 7efe44e76700 1 -- 172.21.15.99:0/2896391502 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe400b29a0 con 0x7efe401523e0 2023-10-08T00:17:27.709 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.709+0000 7efe36ffd700 1 -- 172.21.15.99:0/2896391502 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) v1 ==== 74+0+15372 (secure 0 0 0) 0x7efe28050a90 con 0x7efe401523e0 2023-10-08T00:17:27.709 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:17:27.709 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":142,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:17:21.117117+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:16:48.430607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-08T00:16:49.063424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:17:27.711 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.710+0000 7efe44e76700 1 -- 172.21.15.99:0/2896391502 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efe20066f30 msgr2=0x7efe200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:27.712 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.710+0000 7efe44e76700 1 --2- 172.21.15.99:0/2896391502 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efe20066f30 0x7efe200693e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7efe30011600 tx=0x7efe300091a0 comp rx=0 tx=0).stop 2023-10-08T00:17:27.712 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.710+0000 7efe44e76700 1 -- 172.21.15.99:0/2896391502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe401523e0 msgr2=0x7efe401573e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:27.712 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.710+0000 7efe44e76700 1 --2- 172.21.15.99:0/2896391502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe401523e0 0x7efe401573e0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7efe2800c7c0 tx=0x7efe28007f70 comp rx=0 tx=0).stop 2023-10-08T00:17:27.712 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.710+0000 7efe44e76700 1 -- 172.21.15.99:0/2896391502 shutdown_connections 2023-10-08T00:17:27.712 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.710+0000 7efe44e76700 1 --2- 172.21.15.99:0/2896391502 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efe20066f30 0x7efe200693e0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:27.712 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.710+0000 7efe44e76700 1 --2- 172.21.15.99:0/2896391502 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe40152010 0x7efe40150ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:27.712 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.710+0000 7efe44e76700 1 --2- 172.21.15.99:0/2896391502 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe40158e50 0x7efe400b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:27.712 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.710+0000 7efe44e76700 1 --2- 172.21.15.99:0/2896391502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe401523e0 0x7efe401573e0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:27.712 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.710+0000 7efe44e76700 1 -- 172.21.15.99:0/2896391502 >> 172.21.15.99:0/2896391502 conn(0x7efe40069870 msgr2=0x7efe400cc680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:27.712 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.710+0000 7efe44e76700 1 -- 172.21.15.99:0/2896391502 shutdown_connections 2023-10-08T00:17:27.712 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.711+0000 7efe44e76700 1 -- 172.21.15.99:0/2896391502 wait complete. 2023-10-08T00:17:27.723 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T00:17:27.723 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:17:27.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.837+0000 7f3c9a296700 1 Processor -- start 2023-10-08T00:17:27.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.837+0000 7f3c9a296700 1 -- start start 2023-10-08T00:17:27.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.838+0000 7f3c9a296700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c940cdab0 0x7f3c9415d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:27.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.838+0000 7f3c9a296700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f3c9415dda0 0x7f3c9415cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:27.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.838+0000 7f3c9a296700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c940cb7c0 con 0x7f3c940cd6e0 2023-10-08T00:17:27.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.838+0000 7f3c9a296700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c9415e850 con 0x7f3c9415dda0 2023-10-08T00:17:27.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.838+0000 7f3c9a296700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c940cc360 con 0x7f3c940cdab0 2023-10-08T00:17:27.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.838+0000 7f3c99294700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f3c940cd6e0 0x7f3c940cbc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50960/0 (socket says 172.21.15.99:50960) 2023-10-08T00:17:27.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.838+0000 7f3c99294700 1 -- 172.21.15.99:0/1296999842 learned_addr learned my addr 172.21.15.99:0/1296999842 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:17:27.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.838+0000 7f3c99a95700 1 --2- 172.21.15.99:0/1296999842 >> v2:172.21.15.191:3300/0 conn(0x7f3c9415dda0 0x7f3c9415cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:27.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.838+0000 7f3c98a93700 1 --2- 172.21.15.99:0/1296999842 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c940cdab0 0x7f3c9415d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:27.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.838+0000 7f3c93fff700 1 -- 172.21.15.99:0/1296999842 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1116100383 0 0) 0x7f3c940cb7c0 con 0x7f3c940cd6e0 2023-10-08T00:17:27.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.838+0000 7f3c93fff700 1 -- 172.21.15.99:0/1296999842 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c7c003570 con 0x7f3c940cd6e0 2023-10-08T00:17:27.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.838+0000 7f3c98a93700 1 -- 172.21.15.99:0/1296999842 >> v2:172.21.15.191:3300/0 conn(0x7f3c9415dda0 msgr2=0x7f3c9415cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:27.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.838+0000 7f3c98a93700 1 --2- 172.21.15.99:0/1296999842 >> v2:172.21.15.191:3300/0 conn(0x7f3c9415dda0 0x7f3c9415cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:27.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.838+0000 7f3c93fff700 1 -- 172.21.15.99:0/1296999842 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4079088726 0 0) 0x7f3c7c003570 con 0x7f3c940cd6e0 2023-10-08T00:17:27.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.838+0000 7f3c98a93700 1 -- 172.21.15.99:0/1296999842 >> v1:172.21.15.99:6789/0 conn(0x7f3c940cd6e0 legacy=0x7f3c940cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:27.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.838+0000 7f3c98a93700 1 -- 172.21.15.99:0/1296999842 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c9415e290 con 0x7f3c940cdab0 2023-10-08T00:17:27.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.838+0000 7f3c99a95700 1 --2- 172.21.15.99:0/1296999842 >> v2:172.21.15.191:3300/0 conn(0x7f3c9415dda0 0x7f3c9415cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:17:27.843 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.839+0000 7f3c98a93700 1 --2- 172.21.15.99:0/1296999842 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c940cdab0 0x7f3c9415d860 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3c880098e0 tx=0x7f3c880091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cec570a6cdc39ea server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:27.844 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.839+0000 7f3c93fff700 1 -- 172.21.15.99:0/1296999842 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c001470 con 0x7f3c940cdab0 2023-10-08T00:17:27.844 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.839+0000 7f3c93fff700 1 -- 172.21.15.99:0/1296999842 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c88011420 con 0x7f3c940cdab0 2023-10-08T00:17:27.844 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.839+0000 7f3c93fff700 1 -- 172.21.15.99:0/1296999842 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c880117a0 con 0x7f3c940cdab0 2023-10-08T00:17:27.844 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.839+0000 7f3c9a296700 1 -- 172.21.15.99:0/1296999842 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c940cdab0 msgr2=0x7f3c9415d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:27.844 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.839+0000 7f3c9a296700 1 --2- 172.21.15.99:0/1296999842 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c940cdab0 0x7f3c9415d860 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3c880098e0 tx=0x7f3c880091a0 comp rx=0 tx=0).stop 2023-10-08T00:17:27.844 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.839+0000 7f3c9a296700 1 -- 172.21.15.99:0/1296999842 shutdown_connections 2023-10-08T00:17:27.844 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.839+0000 7f3c9a296700 1 --2- 172.21.15.99:0/1296999842 >> v2:172.21.15.191:3300/0 conn(0x7f3c9415dda0 0x7f3c9415cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:27.845 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.839+0000 7f3c9a296700 1 --2- 172.21.15.99:0/1296999842 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c940cdab0 0x7f3c9415d860 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:27.845 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.839+0000 7f3c9a296700 1 -- 172.21.15.99:0/1296999842 >> 172.21.15.99:0/1296999842 conn(0x7f3c94067660 msgr2=0x7f3c94067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:27.845 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.839+0000 7f3c9a296700 1 -- 172.21.15.99:0/1296999842 shutdown_connections 2023-10-08T00:17:27.845 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.840+0000 7f3c9a296700 1 -- 172.21.15.99:0/1296999842 wait complete. 2023-10-08T00:17:27.845 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.841+0000 7f3c9a296700 1 Processor -- start 2023-10-08T00:17:27.845 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.841+0000 7f3c9a296700 1 -- start start 2023-10-08T00:17:27.850 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.841+0000 7f3c9a296700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c940cd6e0 0x7f3c9413e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:27.850 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.841+0000 7f3c9a296700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c940cdab0 0x7f3c940b5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:27.850 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.842+0000 7f3c9a296700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c9415dda0 0x7f3c940b2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:27.851 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.842+0000 7f3c9a296700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c940c7e50 con 0x7f3c940cdab0 2023-10-08T00:17:27.851 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.842+0000 7f3c9a296700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c940b5f60 con 0x7f3c9415dda0 2023-10-08T00:17:27.851 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.842+0000 7f3c9a296700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c940b6240 con 0x7f3c940cd6e0 2023-10-08T00:17:27.851 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.842+0000 7f3c98a93700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c940cdab0 0x7f3c940b5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:27.851 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.842+0000 7f3c98a93700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c940cdab0 0x7f3c940b5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45080/0 (socket says 172.21.15.99:45080) 2023-10-08T00:17:27.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.842+0000 7f3c98a93700 1 -- 172.21.15.99:0/3046608423 learned_addr learned my addr 172.21.15.99:0/3046608423 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:27.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.842+0000 7f3c99294700 1 --2- 172.21.15.99:0/3046608423 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c940cd6e0 0x7f3c9413e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:27.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.842+0000 7f3c99a95700 1 --2- 172.21.15.99:0/3046608423 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c9415dda0 0x7f3c940b2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:27.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.842+0000 7f3c98a93700 1 -- 172.21.15.99:0/3046608423 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c940cd6e0 msgr2=0x7f3c9413e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:27.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.842+0000 7f3c98a93700 1 --2- 172.21.15.99:0/3046608423 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c940cd6e0 0x7f3c9413e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:27.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.842+0000 7f3c98a93700 1 -- 172.21.15.99:0/3046608423 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c9415dda0 msgr2=0x7f3c940b2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:27.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.842+0000 7f3c98a93700 1 --2- 172.21.15.99:0/3046608423 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c9415dda0 0x7f3c940b2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:27.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.842+0000 7f3c98a93700 1 -- 172.21.15.99:0/3046608423 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c84009a20 con 0x7f3c940cdab0 2023-10-08T00:17:27.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.842+0000 7f3c99294700 1 --2- 172.21.15.99:0/3046608423 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c940cd6e0 0x7f3c9413e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:17:27.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.842+0000 7f3c99a95700 1 --2- 172.21.15.99:0/3046608423 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c9415dda0 0x7f3c940b2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:17:27.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.842+0000 7f3c98a93700 1 --2- 172.21.15.99:0/3046608423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c940cdab0 0x7f3c940b5580 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f3c940ccac0 tx=0x7f3c88016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:27.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.842+0000 7f3c917fa700 1 -- 172.21.15.99:0/3046608423 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c8801cba0 con 0x7f3c940cdab0 2023-10-08T00:17:27.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.842+0000 7f3c917fa700 1 -- 172.21.15.99:0/3046608423 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c8801cd00 con 0x7f3c940cdab0 2023-10-08T00:17:27.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.842+0000 7f3c9a296700 1 -- 172.21.15.99:0/3046608423 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c88009420 con 0x7f3c940cdab0 2023-10-08T00:17:27.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.842+0000 7f3c917fa700 1 -- 172.21.15.99:0/3046608423 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c8801a6a0 con 0x7f3c940cdab0 2023-10-08T00:17:27.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.842+0000 7f3c9a296700 1 -- 172.21.15.99:0/3046608423 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c940b6800 con 0x7f3c940cdab0 2023-10-08T00:17:27.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.843+0000 7f3c9a296700 1 -- 172.21.15.99:0/3046608423 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c940c8000 con 0x7f3c940cdab0 2023-10-08T00:17:27.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.844+0000 7f3c917fa700 1 -- 172.21.15.99:0/3046608423 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3c8801aa10 con 0x7f3c940cdab0 2023-10-08T00:17:27.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.844+0000 7f3c917fa700 1 --2- 172.21.15.99:0/3046608423 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3c80066f60 0x7f3c80069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:27.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.844+0000 7f3c917fa700 1 -- 172.21.15.99:0/3046608423 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3c88085c70 con 0x7f3c940cdab0 2023-10-08T00:17:27.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.844+0000 7f3c99294700 1 --2- 172.21.15.99:0/3046608423 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3c80066f60 0x7f3c80069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:27.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.845+0000 7f3c99294700 1 --2- 172.21.15.99:0/3046608423 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3c80066f60 0x7f3c80069410 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c007c40 tx=0x7f3c8c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:27.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:27.848+0000 7f3c917fa700 1 -- 172.21.15.99:0/3046608423 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3c940c8000 con 0x7f3c940cdab0 2023-10-08T00:17:28.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.072+0000 7f3c9a296700 1 -- 172.21.15.99:0/3046608423 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3c940c7f90 con 0x7f3c940cdab0 2023-10-08T00:17:28.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.073+0000 7f3c917fa700 1 -- 172.21.15.99:0/3046608423 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f3c88029090 con 0x7f3c940cdab0 2023-10-08T00:17:28.074 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:17:28.074 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":63,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5813,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/887374839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":887374839},{"type":"v1","addr":"172.21.15.191:6837","nonce":887374839}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":6445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/2409696901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2409696901},{"type":"v1","addr":"172.21.15.191:6836","nonce":2409696901}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":7095,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/1924096228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1924096228},{"type":"v1","addr":"172.21.15.99:6834","nonce":1924096228}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:16:50.092494+0000","modified":"2023-10-08T00:16:53.744105+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7098":{"gid":7098,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/2522933027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":2522933027},{"type":"v1","addr":"172.21.15.99:6835","nonce":2522933027}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-08T00:17:28.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.075+0000 7f3c9a296700 1 -- 172.21.15.99:0/3046608423 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3c80066f60 msgr2=0x7f3c80069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:28.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.075+0000 7f3c9a296700 1 --2- 172.21.15.99:0/3046608423 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3c80066f60 0x7f3c80069410 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c007c40 tx=0x7f3c8c009040 comp rx=0 tx=0).stop 2023-10-08T00:17:28.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.075+0000 7f3c9a296700 1 -- 172.21.15.99:0/3046608423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c940cdab0 msgr2=0x7f3c940b5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:28.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.075+0000 7f3c9a296700 1 --2- 172.21.15.99:0/3046608423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c940cdab0 0x7f3c940b5580 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f3c940ccac0 tx=0x7f3c88016000 comp rx=0 tx=0).stop 2023-10-08T00:17:28.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.075+0000 7f3c9a296700 1 -- 172.21.15.99:0/3046608423 shutdown_connections 2023-10-08T00:17:28.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.075+0000 7f3c9a296700 1 --2- 172.21.15.99:0/3046608423 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3c80066f60 0x7f3c80069410 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:28.077 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.075+0000 7f3c9a296700 1 --2- 172.21.15.99:0/3046608423 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c9415dda0 0x7f3c940b2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:28.077 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.075+0000 7f3c9a296700 1 --2- 172.21.15.99:0/3046608423 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c940cd6e0 0x7f3c9413e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:28.077 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.075+0000 7f3c9a296700 1 --2- 172.21.15.99:0/3046608423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c940cdab0 0x7f3c940b5580 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:28.077 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.075+0000 7f3c9a296700 1 -- 172.21.15.99:0/3046608423 >> 172.21.15.99:0/3046608423 conn(0x7f3c94067660 msgr2=0x7f3c94156ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:28.077 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.075+0000 7f3c9a296700 1 -- 172.21.15.99:0/3046608423 shutdown_connections 2023-10-08T00:17:28.077 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.075+0000 7f3c9a296700 1 -- 172.21.15.99:0/3046608423 wait complete. 2023-10-08T00:17:28.078 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 63 2023-10-08T00:17:28.087 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:17:28.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.202+0000 7fa5b6b8c700 1 Processor -- start 2023-10-08T00:17:28.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.202+0000 7fa5b6b8c700 1 -- start start 2023-10-08T00:17:28.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.203+0000 7fa5b6b8c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa5b00cd6e0 0x7fa5b00cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:28.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.203+0000 7fa5b6b8c700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5b00cb7c0 con 0x7fa5b00cdab0 2023-10-08T00:17:28.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.203+0000 7fa5b6b8c700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5b015e290 con 0x7fa5b015cbd0 2023-10-08T00:17:28.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.203+0000 7fa5b6b8c700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5b015e850 con 0x7fa5b00cd6e0 2023-10-08T00:17:28.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.203+0000 7fa5b5b8a700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa5b00cd6e0 0x7fa5b00cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:28.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.203+0000 7fa5b5389700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fa5b00cdab0 0x7fa5b015d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50970/0 (socket says 172.21.15.99:50970) 2023-10-08T00:17:28.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.203+0000 7fa5b5389700 1 -- 172.21.15.99:0/3950559472 learned_addr learned my addr 172.21.15.99:0/3950559472 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:17:28.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.203+0000 7fa5b5b8a700 1 -- 172.21.15.99:0/3950559472 >> v1:172.21.15.191:6789/0 conn(0x7fa5b015cbd0 legacy=0x7fa5b015d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:28.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.203+0000 7fa5b5b8a700 1 -- 172.21.15.99:0/3950559472 >> v1:172.21.15.99:6789/0 conn(0x7fa5b00cdab0 legacy=0x7fa5b015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:28.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.203+0000 7fa5b5b8a700 1 -- 172.21.15.99:0/3950559472 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b015e570 con 0x7fa5b00cd6e0 2023-10-08T00:17:28.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.203+0000 7fa5b4b88700 1 -- 172.21.15.99:0/3950559472 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3205393042 0 0) 0x7fa5b00cb7c0 con 0x7fa5b00cdab0 2023-10-08T00:17:28.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.203+0000 7fa5b5b8a700 1 --2- 172.21.15.99:0/3950559472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5b00cd6e0 0x7fa5b00cbc50 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0009e60 tx=0x7fa5a0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7648974956681952 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:28.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.204+0000 7fa5b4b88700 1 -- 172.21.15.99:0/3950559472 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a000e070 con 0x7fa5b00cd6e0 2023-10-08T00:17:28.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.204+0000 7fa5b4b88700 1 -- 172.21.15.99:0/3950559472 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5a000bbf0 con 0x7fa5b00cd6e0 2023-10-08T00:17:28.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.204+0000 7fa5b4b88700 1 -- 172.21.15.99:0/3950559472 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a0012430 con 0x7fa5b00cd6e0 2023-10-08T00:17:28.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.204+0000 7fa5b6b8c700 1 -- 172.21.15.99:0/3950559472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5b00cd6e0 msgr2=0x7fa5b00cbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:28.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.204+0000 7fa5b6b8c700 1 --2- 172.21.15.99:0/3950559472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5b00cd6e0 0x7fa5b00cbc50 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0009e60 tx=0x7fa5a0009780 comp rx=0 tx=0).stop 2023-10-08T00:17:28.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.204+0000 7fa5b6b8c700 1 -- 172.21.15.99:0/3950559472 shutdown_connections 2023-10-08T00:17:28.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.204+0000 7fa5b6b8c700 1 --2- 172.21.15.99:0/3950559472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5b00cd6e0 0x7fa5b00cbc50 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:28.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.204+0000 7fa5b6b8c700 1 -- 172.21.15.99:0/3950559472 >> 172.21.15.99:0/3950559472 conn(0x7fa5b0067660 msgr2=0x7fa5b0067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:28.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.204+0000 7fa5b6b8c700 1 -- 172.21.15.99:0/3950559472 shutdown_connections 2023-10-08T00:17:28.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.204+0000 7fa5b6b8c700 1 -- 172.21.15.99:0/3950559472 wait complete. 2023-10-08T00:17:28.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.205+0000 7fa5b6b8c700 1 Processor -- start 2023-10-08T00:17:28.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.205+0000 7fa5b6b8c700 1 -- start start 2023-10-08T00:17:28.211 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.206+0000 7fa5b6b8c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa5b00cd6e0 0x7fa5b0158760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:28.211 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.206+0000 7fa5b6b8c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5b00cdab0 0x7fa5b00afd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:28.211 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.206+0000 7fa5b6b8c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5b015cbd0 0x7fa5b00b53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:28.211 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.206+0000 7fa5b6b8c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b015b870 con 0x7fa5b00cdab0 2023-10-08T00:17:28.211 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.206+0000 7fa5b6b8c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b015c670 con 0x7fa5b015cbd0 2023-10-08T00:17:28.212 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.206+0000 7fa5b6b8c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b0158ca0 con 0x7fa5b00cd6e0 2023-10-08T00:17:28.212 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.206+0000 7fa5b5389700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5b00cdab0 0x7fa5b00afd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:28.212 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.206+0000 7fa5b5389700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5b00cdab0 0x7fa5b00afd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45090/0 (socket says 172.21.15.99:45090) 2023-10-08T00:17:28.212 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.206+0000 7fa5b5389700 1 -- 172.21.15.99:0/3058842170 learned_addr learned my addr 172.21.15.99:0/3058842170 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:28.212 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.206+0000 7fa5b5b8a700 1 --2- 172.21.15.99:0/3058842170 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa5b00cd6e0 0x7fa5b0158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:28.212 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.206+0000 7fa5b638b700 1 --2- 172.21.15.99:0/3058842170 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5b015cbd0 0x7fa5b00b53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:28.212 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.206+0000 7fa5b5389700 1 -- 172.21.15.99:0/3058842170 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa5b00cd6e0 msgr2=0x7fa5b0158760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:28.212 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.206+0000 7fa5b5389700 1 --2- 172.21.15.99:0/3058842170 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa5b00cd6e0 0x7fa5b0158760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:28.212 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.206+0000 7fa5b5389700 1 -- 172.21.15.99:0/3058842170 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5b015cbd0 msgr2=0x7fa5b00b53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:28.212 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.207+0000 7fa5b5389700 1 --2- 172.21.15.99:0/3058842170 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5b015cbd0 0x7fa5b00b53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:28.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.207+0000 7fa5b5389700 1 -- 172.21.15.99:0/3058842170 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a0009a20 con 0x7fa5b00cdab0 2023-10-08T00:17:28.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.207+0000 7fa5b5389700 1 --2- 172.21.15.99:0/3058842170 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5b00cdab0 0x7fa5b00afd40 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fa5a400c840 tx=0x7fa5a4007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:28.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.207+0000 7fa5adffb700 1 -- 172.21.15.99:0/3058842170 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a4005b30 con 0x7fa5b00cdab0 2023-10-08T00:17:28.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.207+0000 7fa5adffb700 1 -- 172.21.15.99:0/3058842170 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5a4005c90 con 0x7fa5b00cdab0 2023-10-08T00:17:28.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.207+0000 7fa5adffb700 1 -- 172.21.15.99:0/3058842170 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a4007300 con 0x7fa5b00cdab0 2023-10-08T00:17:28.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.207+0000 7fa5b6b8c700 1 -- 172.21.15.99:0/3058842170 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5b015ba60 con 0x7fa5b00cdab0 2023-10-08T00:17:28.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.207+0000 7fa5b6b8c700 1 -- 172.21.15.99:0/3058842170 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5b015c020 con 0x7fa5b00cdab0 2023-10-08T00:17:28.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.208+0000 7fa5b6b8c700 1 -- 172.21.15.99:0/3058842170 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5b016de60 con 0x7fa5b00cdab0 2023-10-08T00:17:28.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.208+0000 7fa5adffb700 1 -- 172.21.15.99:0/3058842170 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa5a40076b0 con 0x7fa5b00cdab0 2023-10-08T00:17:28.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.209+0000 7fa5adffb700 1 --2- 172.21.15.99:0/3058842170 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa59c066f30 0x7fa59c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:28.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.209+0000 7fa5adffb700 1 -- 172.21.15.99:0/3058842170 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa5a4084030 con 0x7fa5b00cdab0 2023-10-08T00:17:28.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.209+0000 7fa5b5b8a700 1 --2- 172.21.15.99:0/3058842170 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa59c066f30 0x7fa59c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:28.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.209+0000 7fa5b5b8a700 1 --2- 172.21.15.99:0/3058842170 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa59c066f30 0x7fa59c0693e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa5a00097c0 tx=0x7fa5a0018000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:28.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.213+0000 7fa5adffb700 1 -- 172.21.15.99:0/3058842170 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa5b016de60 con 0x7fa5b00cdab0 2023-10-08T00:17:28.438 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.438+0000 7fa5b6b8c700 1 -- 172.21.15.99:0/3058842170 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa5b016de60 con 0x7fa5b00cdab0 2023-10-08T00:17:28.439 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.438+0000 7fa5adffb700 1 -- 172.21.15.99:0/3058842170 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4602 (secure 0 0 0) 0x7fa5b016de60 con 0x7fa5b00cdab0 2023-10-08T00:17:28.439 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:17:28.439 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":63,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5813,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/887374839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":887374839},{"type":"v1","addr":"172.21.15.191:6837","nonce":887374839}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":6445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/2409696901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2409696901},{"type":"v1","addr":"172.21.15.191:6836","nonce":2409696901}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":7095,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/1924096228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1924096228},{"type":"v1","addr":"172.21.15.99:6834","nonce":1924096228}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:16:50.092494+0000","modified":"2023-10-08T00:16:53.744105+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7098},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7098":{"gid":7098,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/2522933027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":2522933027},{"type":"v1","addr":"172.21.15.99:6835","nonce":2522933027}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-08T00:17:28.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.440+0000 7fa5b6b8c700 1 -- 172.21.15.99:0/3058842170 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa59c066f30 msgr2=0x7fa59c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:28.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.440+0000 7fa5b6b8c700 1 --2- 172.21.15.99:0/3058842170 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa59c066f30 0x7fa59c0693e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa5a00097c0 tx=0x7fa5a0018000 comp rx=0 tx=0).stop 2023-10-08T00:17:28.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.440+0000 7fa5b6b8c700 1 -- 172.21.15.99:0/3058842170 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5b00cdab0 msgr2=0x7fa5b00afd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:28.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.440+0000 7fa5b6b8c700 1 --2- 172.21.15.99:0/3058842170 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5b00cdab0 0x7fa5b00afd40 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fa5a400c840 tx=0x7fa5a4007df0 comp rx=0 tx=0).stop 2023-10-08T00:17:28.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.440+0000 7fa5b6b8c700 1 -- 172.21.15.99:0/3058842170 shutdown_connections 2023-10-08T00:17:28.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.440+0000 7fa5b6b8c700 1 --2- 172.21.15.99:0/3058842170 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa59c066f30 0x7fa59c0693e0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:28.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.440+0000 7fa5b6b8c700 1 --2- 172.21.15.99:0/3058842170 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5b015cbd0 0x7fa5b00b53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:28.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.440+0000 7fa5b6b8c700 1 --2- 172.21.15.99:0/3058842170 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa5b00cd6e0 0x7fa5b0158760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:28.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.440+0000 7fa5b6b8c700 1 --2- 172.21.15.99:0/3058842170 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5b00cdab0 0x7fa5b00afd40 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:28.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.440+0000 7fa5b6b8c700 1 -- 172.21.15.99:0/3058842170 >> 172.21.15.99:0/3058842170 conn(0x7fa5b0067660 msgr2=0x7fa5b00d0810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:28.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.440+0000 7fa5b6b8c700 1 -- 172.21.15.99:0/3058842170 shutdown_connections 2023-10-08T00:17:28.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.440+0000 7fa5b6b8c700 1 -- 172.21.15.99:0/3058842170 wait complete. 2023-10-08T00:17:28.442 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 63 2023-10-08T00:17:28.453 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:17:28.565 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.564+0000 7f5c7f65b700 1 Processor -- start 2023-10-08T00:17:28.565 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.564+0000 7f5c7f65b700 1 -- start start 2023-10-08T00:17:28.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.565+0000 7f5c7f65b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c78157870 0x7f5c7815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:28.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.565+0000 7f5c7f65b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f5c78157c40 0x7f5c78158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:28.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.565+0000 7f5c7f65b700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c7815cbb0 con 0x7f5c7815a620 2023-10-08T00:17:28.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.565+0000 7f5c7f65b700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c7815bfa0 con 0x7f5c78157c40 2023-10-08T00:17:28.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.565+0000 7f5c7f65b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7815c280 con 0x7f5c78157870 2023-10-08T00:17:28.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.565+0000 7f5c7cbf6700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f5c7815a620 0x7f5c7815d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50978/0 (socket says 172.21.15.99:50978) 2023-10-08T00:17:28.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.565+0000 7f5c7cbf6700 1 -- 172.21.15.99:0/2363764129 learned_addr learned my addr 172.21.15.99:0/2363764129 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:17:28.569 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.565+0000 7f5c7dbf8700 1 --2- 172.21.15.99:0/2363764129 >> v2:172.21.15.191:3300/0 conn(0x7f5c78157c40 0x7f5c78158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:28.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.565+0000 7f5c7d3f7700 1 --2- 172.21.15.99:0/2363764129 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c78157870 0x7f5c7815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:28.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.565+0000 7f5c6ffff700 1 -- 172.21.15.99:0/2363764129 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 699075201 0 0) 0x7f5c7815cbb0 con 0x7f5c7815a620 2023-10-08T00:17:28.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.566+0000 7f5c6ffff700 1 -- 172.21.15.99:0/2363764129 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c64003570 con 0x7f5c7815a620 2023-10-08T00:17:28.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.566+0000 7f5c6ffff700 1 -- 172.21.15.99:0/2363764129 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2780667968 0 0) 0x7f5c64003570 con 0x7f5c7815a620 2023-10-08T00:17:28.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.566+0000 7f5c6ffff700 1 -- 172.21.15.99:0/2363764129 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c78157870 msgr2=0x7f5c7815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:28.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.566+0000 7f5c6ffff700 1 --2- 172.21.15.99:0/2363764129 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c78157870 0x7f5c7815b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:28.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.566+0000 7f5c6ffff700 1 -- 172.21.15.99:0/2363764129 >> v2:172.21.15.191:3300/0 conn(0x7f5c78157c40 msgr2=0x7f5c78158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:28.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.566+0000 7f5c6ffff700 1 --2- 172.21.15.99:0/2363764129 >> v2:172.21.15.191:3300/0 conn(0x7f5c78157c40 0x7f5c78158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:28.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.566+0000 7f5c6ffff700 1 -- 172.21.15.99:0/2363764129 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c7815c560 con 0x7f5c7815a620 2023-10-08T00:17:28.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.566+0000 7f5c6ffff700 1 -- 172.21.15.99:0/2363764129 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f5c68001470 con 0x7f5c7815a620 2023-10-08T00:17:28.570 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.566+0000 7f5c6ffff700 1 -- 172.21.15.99:0/2363764129 >> v1:172.21.15.99:6789/0 conn(0x7f5c7815a620 legacy=0x7f5c7815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:28.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.566+0000 7f5c6ffff700 1 --2- 172.21.15.99:0/2363764129 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c78157870 0x7f5c64003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:28.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.566+0000 7f5c6ffff700 1 --2- 172.21.15.99:0/2363764129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c64003fe0 0x7f5c64004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:28.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.566+0000 7f5c6ffff700 1 --2- 172.21.15.99:0/2363764129 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c64004d50 0x7f5c640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:28.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.566+0000 7f5c6ffff700 1 -- 172.21.15.99:0/2363764129 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7815bfa0 con 0x7f5c64003fe0 2023-10-08T00:17:28.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.566+0000 7f5c6ffff700 1 -- 172.21.15.99:0/2363764129 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7815c280 con 0x7f5c64004d50 2023-10-08T00:17:28.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.566+0000 7f5c6ffff700 1 -- 172.21.15.99:0/2363764129 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6400ba90 con 0x7f5c78157870 2023-10-08T00:17:28.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.566+0000 7f5c7cbf6700 1 --2- 172.21.15.99:0/2363764129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c64003fe0 0x7f5c64004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:28.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.566+0000 7f5c7d3f7700 1 --2- 172.21.15.99:0/2363764129 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c78157870 0x7f5c64003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:28.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.566+0000 7f5c7dbf8700 1 --2- 172.21.15.99:0/2363764129 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c64004d50 0x7f5c640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:28.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.566+0000 7f5c7cbf6700 1 -- 172.21.15.99:0/2363764129 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c78157870 msgr2=0x7f5c64003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:28.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.566+0000 7f5c7cbf6700 1 --2- 172.21.15.99:0/2363764129 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c78157870 0x7f5c64003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:28.571 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.566+0000 7f5c7cbf6700 1 -- 172.21.15.99:0/2363764129 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c64004d50 msgr2=0x7f5c640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:28.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.566+0000 7f5c7cbf6700 1 --2- 172.21.15.99:0/2363764129 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c64004d50 0x7f5c640051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:28.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.566+0000 7f5c7cbf6700 1 -- 172.21.15.99:0/2363764129 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c64003570 con 0x7f5c64003fe0 2023-10-08T00:17:28.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.566+0000 7f5c7dbf8700 1 --2- 172.21.15.99:0/2363764129 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c64004d50 0x7f5c640051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:17:28.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.566+0000 7f5c7cbf6700 1 --2- 172.21.15.99:0/2363764129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c64003fe0 0x7f5c64004810 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f5c6800c3c0 tx=0x7f5c68001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3692ebb6f54de0b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:28.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.567+0000 7f5c6ffff700 1 -- 172.21.15.99:0/2363764129 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6800cc30 con 0x7f5c64003fe0 2023-10-08T00:17:28.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.567+0000 7f5c6ffff700 1 -- 172.21.15.99:0/2363764129 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c6800cdb0 con 0x7f5c64003fe0 2023-10-08T00:17:28.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.567+0000 7f5c6ffff700 1 -- 172.21.15.99:0/2363764129 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c68005b70 con 0x7f5c64003fe0 2023-10-08T00:17:28.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.567+0000 7f5c7f65b700 1 -- 172.21.15.99:0/2363764129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c64003fe0 msgr2=0x7f5c64004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:28.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.567+0000 7f5c7f65b700 1 --2- 172.21.15.99:0/2363764129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c64003fe0 0x7f5c64004810 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f5c6800c3c0 tx=0x7f5c68001e90 comp rx=0 tx=0).stop 2023-10-08T00:17:28.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.567+0000 7f5c7dbf8700 1 -- 172.21.15.99:0/2363764129 reap_dead start 2023-10-08T00:17:28.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.567+0000 7f5c7f65b700 1 -- 172.21.15.99:0/2363764129 shutdown_connections 2023-10-08T00:17:28.572 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.567+0000 7f5c7f65b700 1 -- 172.21.15.99:0/2363764129 >> 172.21.15.99:0/2363764129 conn(0x7f5c78068ce0 msgr2=0x7f5c780690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:28.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.567+0000 7f5c7f65b700 1 -- 172.21.15.99:0/2363764129 shutdown_connections 2023-10-08T00:17:28.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.567+0000 7f5c7f65b700 1 -- 172.21.15.99:0/2363764129 wait complete. 2023-10-08T00:17:28.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.568+0000 7f5c7f65b700 1 Processor -- start 2023-10-08T00:17:28.573 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.568+0000 7f5c7f65b700 1 -- start start 2023-10-08T00:17:28.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.569+0000 7f5c7f65b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c64003fe0 0x7f5c7813d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:28.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.569+0000 7f5c7f65b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c7813f3c0 0x7f5c78151c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:28.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.569+0000 7f5c7f65b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c781521d0 0x7f5c780b2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:28.575 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.569+0000 7f5c7f65b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c781527a0 con 0x7f5c7813f3c0 2023-10-08T00:17:28.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.569+0000 7f5c7f65b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7813fe20 con 0x7f5c64003fe0 2023-10-08T00:17:28.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.569+0000 7f5c7f65b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7814f3d0 con 0x7f5c781521d0 2023-10-08T00:17:28.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.569+0000 7f5c7cbf6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c7813f3c0 0x7f5c78151c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:28.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.569+0000 7f5c7cbf6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c7813f3c0 0x7f5c78151c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45108/0 (socket says 172.21.15.99:45108) 2023-10-08T00:17:28.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.569+0000 7f5c7cbf6700 1 -- 172.21.15.99:0/1519629395 learned_addr learned my addr 172.21.15.99:0/1519629395 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:28.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.569+0000 7f5c7d3f7700 1 --2- 172.21.15.99:0/1519629395 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c64003fe0 0x7f5c7813d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:28.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.569+0000 7f5c7dbf8700 1 --2- 172.21.15.99:0/1519629395 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c781521d0 0x7f5c780b2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:28.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.570+0000 7f5c7cbf6700 1 -- 172.21.15.99:0/1519629395 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c781521d0 msgr2=0x7f5c780b2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:28.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.570+0000 7f5c7cbf6700 1 --2- 172.21.15.99:0/1519629395 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c781521d0 0x7f5c780b2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:28.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.570+0000 7f5c7cbf6700 1 -- 172.21.15.99:0/1519629395 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c64003fe0 msgr2=0x7f5c7813d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:28.576 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.570+0000 7f5c7cbf6700 1 --2- 172.21.15.99:0/1519629395 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c64003fe0 0x7f5c7813d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:28.577 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.570+0000 7f5c7cbf6700 1 -- 172.21.15.99:0/1519629395 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c7814f6b0 con 0x7f5c7813f3c0 2023-10-08T00:17:28.577 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.570+0000 7f5c7cbf6700 1 --2- 172.21.15.99:0/1519629395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c7813f3c0 0x7f5c78151c90 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f5c64004730 tx=0x7f5c68015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:28.577 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.570+0000 7f5c6d7fa700 1 -- 172.21.15.99:0/1519629395 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6800cc30 con 0x7f5c7813f3c0 2023-10-08T00:17:28.577 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.570+0000 7f5c6d7fa700 1 -- 172.21.15.99:0/1519629395 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c680072e0 con 0x7f5c7813f3c0 2023-10-08T00:17:28.577 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.570+0000 7f5c7f65b700 1 -- 172.21.15.99:0/1519629395 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c7814f990 con 0x7f5c7813f3c0 2023-10-08T00:17:28.577 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.570+0000 7f5c6d7fa700 1 -- 172.21.15.99:0/1519629395 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c680076a0 con 0x7f5c7813f3c0 2023-10-08T00:17:28.577 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.570+0000 7f5c7f65b700 1 -- 172.21.15.99:0/1519629395 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c7813f830 con 0x7f5c7813f3c0 2023-10-08T00:17:28.577 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.571+0000 7f5c7f65b700 1 -- 172.21.15.99:0/1519629395 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c7804c3c0 con 0x7f5c7813f3c0 2023-10-08T00:17:28.577 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.571+0000 7f5c6d7fa700 1 -- 172.21.15.99:0/1519629395 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5c68019030 con 0x7f5c7813f3c0 2023-10-08T00:17:28.577 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.572+0000 7f5c6d7fa700 1 --2- 172.21.15.99:0/1519629395 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5c5c066f60 0x7f5c5c069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:28.577 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.572+0000 7f5c6d7fa700 1 -- 172.21.15.99:0/1519629395 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5c68085460 con 0x7f5c7813f3c0 2023-10-08T00:17:28.577 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.572+0000 7f5c7d3f7700 1 --2- 172.21.15.99:0/1519629395 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5c5c066f60 0x7f5c5c069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:28.578 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.572+0000 7f5c7d3f7700 1 --2- 172.21.15.99:0/1519629395 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5c5c066f60 0x7f5c5c069410 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f5c74006f00 tx=0x7f5c740091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:28.578 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.576+0000 7f5c6d7fa700 1 -- 172.21.15.99:0/1519629395 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5c68052630 con 0x7f5c7813f3c0 2023-10-08T00:17:28.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.758+0000 7f5c7f65b700 1 -- 172.21.15.99:0/1519629395 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c781527a0 con 0x7f5c7813f3c0 2023-10-08T00:17:28.759 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.758+0000 7f5c6d7fa700 1 -- 172.21.15.99:0/1519629395 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) v1 ==== 74+0+15372 (secure 0 0 0) 0x7f5c68055c50 con 0x7f5c7813f3c0 2023-10-08T00:17:28.759 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:17:28.759 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":142,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:17:21.117117+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:16:48.430607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-08T00:16:49.063424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:17:28.761 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.760+0000 7f5c7f65b700 1 -- 172.21.15.99:0/1519629395 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5c5c066f60 msgr2=0x7f5c5c069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:28.761 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.760+0000 7f5c7f65b700 1 --2- 172.21.15.99:0/1519629395 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5c5c066f60 0x7f5c5c069410 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f5c74006f00 tx=0x7f5c740091a0 comp rx=0 tx=0).stop 2023-10-08T00:17:28.761 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.760+0000 7f5c7f65b700 1 -- 172.21.15.99:0/1519629395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c7813f3c0 msgr2=0x7f5c78151c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:28.761 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.760+0000 7f5c7f65b700 1 --2- 172.21.15.99:0/1519629395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c7813f3c0 0x7f5c78151c90 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f5c64004730 tx=0x7f5c68015000 comp rx=0 tx=0).stop 2023-10-08T00:17:28.762 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.760+0000 7f5c7f65b700 1 -- 172.21.15.99:0/1519629395 shutdown_connections 2023-10-08T00:17:28.762 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.760+0000 7f5c7f65b700 1 --2- 172.21.15.99:0/1519629395 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5c5c066f60 0x7f5c5c069410 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:28.762 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.760+0000 7f5c7f65b700 1 --2- 172.21.15.99:0/1519629395 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c64003fe0 0x7f5c7813d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:28.762 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.760+0000 7f5c7f65b700 1 --2- 172.21.15.99:0/1519629395 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c781521d0 0x7f5c780b2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:28.762 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.760+0000 7f5c7f65b700 1 --2- 172.21.15.99:0/1519629395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c7813f3c0 0x7f5c78151c90 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:28.762 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.760+0000 7f5c7f65b700 1 -- 172.21.15.99:0/1519629395 >> 172.21.15.99:0/1519629395 conn(0x7f5c78068ce0 msgr2=0x7f5c780d0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:28.762 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.761+0000 7f5c7f65b700 1 -- 172.21.15.99:0/1519629395 shutdown_connections 2023-10-08T00:17:28.762 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.761+0000 7f5c7f65b700 1 -- 172.21.15.99:0/1519629395 wait complete. 2023-10-08T00:17:28.773 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T00:17:28.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.886+0000 7f5f22f80700 1 Processor -- start 2023-10-08T00:17:28.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.886+0000 7f5f22f80700 1 -- start start 2023-10-08T00:17:28.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.887+0000 7f5f22f80700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f1c159da0 0x7f5f1c0b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:28.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.887+0000 7f5f22f80700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f5f1c15a170 0x7f5f1c068b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:28.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.887+0000 7f5f22f80700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5f1c069040 0x7f5f1c0af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:28.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.887+0000 7f5f22f80700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f1c0694e0 con 0x7f5f1c069040 2023-10-08T00:17:28.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.887+0000 7f5f22f80700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f1c15ea70 con 0x7f5f1c15a170 2023-10-08T00:17:28.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.887+0000 7f5f22f80700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c0b3990 con 0x7f5f1c159da0 2023-10-08T00:17:28.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.887+0000 7f5f2277f700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5f1c069040 0x7f5f1c0af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:28.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.887+0000 7f5f2277f700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5f1c069040 0x7f5f1c0af4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45116/0 (socket says 172.21.15.99:45116) 2023-10-08T00:17:28.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.887+0000 7f5f2277f700 1 -- 172.21.15.99:0/1089249273 learned_addr learned my addr 172.21.15.99:0/1089249273 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:28.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.887+0000 7f5f21f7e700 1 --2- 172.21.15.99:0/1089249273 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f1c159da0 0x7f5f1c0b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:28.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.887+0000 7f5f2177d700 1 --2- 172.21.15.99:0/1089249273 >> v2:172.21.15.191:3300/0 conn(0x7f5f1c15a170 0x7f5f1c068b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:28.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.887+0000 7f5f2277f700 1 -- 172.21.15.99:0/1089249273 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f1c159da0 msgr2=0x7f5f1c0b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:28.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.887+0000 7f5f2277f700 1 --2- 172.21.15.99:0/1089249273 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f1c159da0 0x7f5f1c0b2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:28.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.887+0000 7f5f2277f700 1 -- 172.21.15.99:0/1089249273 >> v2:172.21.15.191:3300/0 conn(0x7f5f1c15a170 msgr2=0x7f5f1c068b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:28.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.887+0000 7f5f2277f700 1 --2- 172.21.15.99:0/1089249273 >> v2:172.21.15.191:3300/0 conn(0x7f5f1c15a170 0x7f5f1c068b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:28.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.887+0000 7f5f2277f700 1 -- 172.21.15.99:0/1089249273 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f1c0b0b70 con 0x7f5f1c069040 2023-10-08T00:17:28.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.888+0000 7f5f2277f700 1 --2- 172.21.15.99:0/1089249273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f1c069040 0x7f5f1c0af4a0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c06b2c0 tx=0x7f5f14009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a161c9bdb082fc01 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:28.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.888+0000 7f5f20f7c700 1 -- 172.21.15.99:0/1089249273 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1400b860 con 0x7f5f1c069040 2023-10-08T00:17:28.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.888+0000 7f5f20f7c700 1 -- 172.21.15.99:0/1089249273 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1400b9c0 con 0x7f5f1c069040 2023-10-08T00:17:28.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.888+0000 7f5f20f7c700 1 -- 172.21.15.99:0/1089249273 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1400bd80 con 0x7f5f1c069040 2023-10-08T00:17:28.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.888+0000 7f5f22f80700 1 -- 172.21.15.99:0/1089249273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f1c069040 msgr2=0x7f5f1c0af4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:28.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.888+0000 7f5f22f80700 1 --2- 172.21.15.99:0/1089249273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f1c069040 0x7f5f1c0af4a0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c06b2c0 tx=0x7f5f14009250 comp rx=0 tx=0).stop 2023-10-08T00:17:28.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.888+0000 7f5f22f80700 1 -- 172.21.15.99:0/1089249273 shutdown_connections 2023-10-08T00:17:28.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.888+0000 7f5f22f80700 1 --2- 172.21.15.99:0/1089249273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f1c069040 0x7f5f1c0af4a0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:28.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.888+0000 7f5f22f80700 1 --2- 172.21.15.99:0/1089249273 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f1c159da0 0x7f5f1c0b2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:28.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.888+0000 7f5f22f80700 1 --2- 172.21.15.99:0/1089249273 >> v2:172.21.15.191:3300/0 conn(0x7f5f1c15a170 0x7f5f1c068b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:28.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.888+0000 7f5f22f80700 1 -- 172.21.15.99:0/1089249273 >> 172.21.15.99:0/1089249273 conn(0x7f5f1c0d0bc0 msgr2=0x7f5f1c0d0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:28.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.888+0000 7f5f22f80700 1 -- 172.21.15.99:0/1089249273 shutdown_connections 2023-10-08T00:17:28.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.888+0000 7f5f22f80700 1 -- 172.21.15.99:0/1089249273 wait complete. 2023-10-08T00:17:28.894 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.889+0000 7f5f22f80700 1 Processor -- start 2023-10-08T00:17:28.894 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.889+0000 7f5f22f80700 1 -- start start 2023-10-08T00:17:28.894 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.890+0000 7f5f22f80700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f1c069040 0x7f5f1c159160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:28.895 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.890+0000 7f5f22f80700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f1c159da0 0x7f5f1c0c8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:28.895 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.890+0000 7f5f22f80700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f1c15a170 0x7f5f1c0c8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:28.895 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.890+0000 7f5f22f80700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c0c6330 con 0x7f5f1c069040 2023-10-08T00:17:28.895 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.890+0000 7f5f22f80700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c0c9dd0 con 0x7f5f1c15a170 2023-10-08T00:17:28.896 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.890+0000 7f5f22f80700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c1596a0 con 0x7f5f1c159da0 2023-10-08T00:17:28.896 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.890+0000 7f5f21f7e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f1c069040 0x7f5f1c159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:28.896 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.890+0000 7f5f21f7e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f1c069040 0x7f5f1c159160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45118/0 (socket says 172.21.15.99:45118) 2023-10-08T00:17:28.896 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.890+0000 7f5f21f7e700 1 -- 172.21.15.99:0/3935951638 learned_addr learned my addr 172.21.15.99:0/3935951638 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:28.896 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.890+0000 7f5f2177d700 1 --2- 172.21.15.99:0/3935951638 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f1c159da0 0x7f5f1c0c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:28.896 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.890+0000 7f5f2277f700 1 --2- 172.21.15.99:0/3935951638 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f1c15a170 0x7f5f1c0c8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:28.896 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.891+0000 7f5f21f7e700 1 -- 172.21.15.99:0/3935951638 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f1c159da0 msgr2=0x7f5f1c0c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:28.896 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.891+0000 7f5f21f7e700 1 --2- 172.21.15.99:0/3935951638 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f1c159da0 0x7f5f1c0c8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:28.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.891+0000 7f5f21f7e700 1 -- 172.21.15.99:0/3935951638 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f1c15a170 msgr2=0x7f5f1c0c8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:28.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.891+0000 7f5f21f7e700 1 --2- 172.21.15.99:0/3935951638 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f1c15a170 0x7f5f1c0c8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:28.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.891+0000 7f5f21f7e700 1 -- 172.21.15.99:0/3935951638 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f140094f0 con 0x7f5f1c069040 2023-10-08T00:17:28.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.891+0000 7f5f21f7e700 1 --2- 172.21.15.99:0/3935951638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f1c069040 0x7f5f1c159160 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f5f1800e1c0 tx=0x7f5f18009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:28.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.891+0000 7f5f11ffb700 1 -- 172.21.15.99:0/3935951638 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1800ed00 con 0x7f5f1c069040 2023-10-08T00:17:28.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.891+0000 7f5f11ffb700 1 -- 172.21.15.99:0/3935951638 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1800ee60 con 0x7f5f1c069040 2023-10-08T00:17:28.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.891+0000 7f5f22f80700 1 -- 172.21.15.99:0/3935951638 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f1c0c91c0 con 0x7f5f1c069040 2023-10-08T00:17:28.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.891+0000 7f5f11ffb700 1 -- 172.21.15.99:0/3935951638 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f18014670 con 0x7f5f1c069040 2023-10-08T00:17:28.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.891+0000 7f5f22f80700 1 -- 172.21.15.99:0/3935951638 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f1c0c9780 con 0x7f5f1c069040 2023-10-08T00:17:28.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.892+0000 7f5f22f80700 1 -- 172.21.15.99:0/3935951638 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f1c089480 con 0x7f5f1c069040 2023-10-08T00:17:28.898 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.892+0000 7f5f11ffb700 1 -- 172.21.15.99:0/3935951638 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5f18014a60 con 0x7f5f1c069040 2023-10-08T00:17:28.898 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.893+0000 7f5f11ffb700 1 --2- 172.21.15.99:0/3935951638 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5f00066f30 0x7f5f000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:28.898 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.893+0000 7f5f11ffb700 1 -- 172.21.15.99:0/3935951638 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5f18084c10 con 0x7f5f1c069040 2023-10-08T00:17:28.898 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.893+0000 7f5f2177d700 1 --2- 172.21.15.99:0/3935951638 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5f00066f30 0x7f5f000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:28.898 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.893+0000 7f5f2177d700 1 --2- 172.21.15.99:0/3935951638 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5f00066f30 0x7f5f000693e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c0cf690 tx=0x7f5f0c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:28.898 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:28.897+0000 7f5f11ffb700 1 -- 172.21.15.99:0/3935951638 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5f18051e80 con 0x7f5f1c069040 2023-10-08T00:17:29.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.120+0000 7f5f22f80700 1 -- 172.21.15.99:0/3935951638 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f5f1c15def0 con 0x7f5f1c069040 2023-10-08T00:17:29.121 INFO:tasks.ceph.mon.a.smithi099.stderr:2023-10-08T00:17:29.120+0000 7f6c60fa2700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T00:17:29.127 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.127+0000 7f5f11ffb700 1 -- 172.21.15.99:0/3935951638 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v64) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5f18009a60 con 0x7f5f1c069040 2023-10-08T00:17:29.128 INFO:tasks.ceph.mds.a.smithi099.stderr: -7686> 2023-10-08T00:16:56.024+0000 7f48ac854700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T00:17:29.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.129+0000 7f5f22f80700 1 -- 172.21.15.99:0/3935951638 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5f00066f30 msgr2=0x7f5f000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:29.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.129+0000 7f5f22f80700 1 --2- 172.21.15.99:0/3935951638 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5f00066f30 0x7f5f000693e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c0cf690 tx=0x7f5f0c008040 comp rx=0 tx=0).stop 2023-10-08T00:17:29.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.129+0000 7f5f22f80700 1 -- 172.21.15.99:0/3935951638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f1c069040 msgr2=0x7f5f1c159160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:29.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.129+0000 7f5f22f80700 1 --2- 172.21.15.99:0/3935951638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f1c069040 0x7f5f1c159160 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f5f1800e1c0 tx=0x7f5f18009920 comp rx=0 tx=0).stop 2023-10-08T00:17:29.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.129+0000 7f5f22f80700 1 -- 172.21.15.99:0/3935951638 shutdown_connections 2023-10-08T00:17:29.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.129+0000 7f5f22f80700 1 --2- 172.21.15.99:0/3935951638 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5f00066f30 0x7f5f000693e0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:29.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.129+0000 7f5f22f80700 1 --2- 172.21.15.99:0/3935951638 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f1c15a170 0x7f5f1c0c8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:29.131 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.129+0000 7f5f22f80700 1 --2- 172.21.15.99:0/3935951638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f1c069040 0x7f5f1c159160 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:29.131 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.129+0000 7f5f22f80700 1 --2- 172.21.15.99:0/3935951638 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f1c159da0 0x7f5f1c0c8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:29.131 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.129+0000 7f5f22f80700 1 -- 172.21.15.99:0/3935951638 >> 172.21.15.99:0/3935951638 conn(0x7f5f1c0d0bc0 msgr2=0x7f5f1c0d0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:29.131 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.129+0000 7f5f22f80700 1 -- 172.21.15.99:0/3935951638 shutdown_connections 2023-10-08T00:17:29.131 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.129+0000 7f5f22f80700 1 -- 172.21.15.99:0/3935951638 wait complete. 2023-10-08T00:17:29.131 INFO:teuthology.orchestra.run.smithi099.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T00:17:29.142 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T00:17:29.157 INFO:tasks.ceph.mds.a.smithi099.stdout:starting mds.a at 2023-10-08T00:17:29.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.258+0000 7fc1b659f700 1 Processor -- start 2023-10-08T00:17:29.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.258+0000 7fc1b659f700 1 -- start start 2023-10-08T00:17:29.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.259+0000 7fc1b659f700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc1b0157ba0 0x7fc1b0158c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:29.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.259+0000 7fc1b659f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1b015b7e0 0x7fc1b0070700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:29.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.259+0000 7fc1b659f700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc1b0070c40 0x7fc1b00ca4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:29.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.259+0000 7fc1b659f700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1b0071080 con 0x7fc1b0070c40 2023-10-08T00:17:29.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.259+0000 7fc1b659f700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1b0159140 con 0x7fc1b0157ba0 2023-10-08T00:17:29.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.259+0000 7fc1b659f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b015bda0 con 0x7fc1b015b7e0 2023-10-08T00:17:29.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.259+0000 7fc1b4b3c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc1b0070c40 0x7fc1b00ca4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:29.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.259+0000 7fc1b4b3c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc1b0070c40 0x7fc1b00ca4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45138/0 (socket says 172.21.15.99:45138) 2023-10-08T00:17:29.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.259+0000 7fc1b4b3c700 1 -- 172.21.15.99:0/3561896553 learned_addr learned my addr 172.21.15.99:0/3561896553 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:29.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.260+0000 7fc1af7fe700 1 --2- 172.21.15.99:0/3561896553 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1b015b7e0 0x7fc1b0070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:29.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.260+0000 7fc1b4b3c700 1 -- 172.21.15.99:0/3561896553 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1b015b7e0 msgr2=0x7fc1b0070700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:29.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.260+0000 7fc1affff700 1 --2- 172.21.15.99:0/3561896553 >> v2:172.21.15.191:3300/0 conn(0x7fc1b0157ba0 0x7fc1b0158c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:29.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.260+0000 7fc1b4b3c700 1 --2- 172.21.15.99:0/3561896553 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1b015b7e0 0x7fc1b0070700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:29.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.260+0000 7fc1b4b3c700 1 -- 172.21.15.99:0/3561896553 >> v2:172.21.15.191:3300/0 conn(0x7fc1b0157ba0 msgr2=0x7fc1b0158c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:29.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.260+0000 7fc1b4b3c700 1 --2- 172.21.15.99:0/3561896553 >> v2:172.21.15.191:3300/0 conn(0x7fc1b0157ba0 0x7fc1b0158c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:29.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.260+0000 7fc1b4b3c700 1 -- 172.21.15.99:0/3561896553 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b015c080 con 0x7fc1b0070c40 2023-10-08T00:17:29.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.260+0000 7fc1affff700 1 --2- 172.21.15.99:0/3561896553 >> v2:172.21.15.191:3300/0 conn(0x7fc1b0157ba0 0x7fc1b0158c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:17:29.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.260+0000 7fc1b4b3c700 1 --2- 172.21.15.99:0/3561896553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1b0070c40 0x7fc1b00ca4c0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0071480 tx=0x7fc1a8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e85629a6c03bfe15 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:29.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.260+0000 7fc1aeffd700 1 -- 172.21.15.99:0/3561896553 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a800e070 con 0x7fc1b0070c40 2023-10-08T00:17:29.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.260+0000 7fc1aeffd700 1 -- 172.21.15.99:0/3561896553 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1a800ba50 con 0x7fc1b0070c40 2023-10-08T00:17:29.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.260+0000 7fc1aeffd700 1 -- 172.21.15.99:0/3561896553 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a80092e0 con 0x7fc1b0070c40 2023-10-08T00:17:29.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.260+0000 7fc1b659f700 1 -- 172.21.15.99:0/3561896553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1b0070c40 msgr2=0x7fc1b00ca4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:29.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.260+0000 7fc1b659f700 1 --2- 172.21.15.99:0/3561896553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1b0070c40 0x7fc1b00ca4c0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0071480 tx=0x7fc1a8009840 comp rx=0 tx=0).stop 2023-10-08T00:17:29.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.261+0000 7fc1b659f700 1 -- 172.21.15.99:0/3561896553 shutdown_connections 2023-10-08T00:17:29.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.261+0000 7fc1b659f700 1 --2- 172.21.15.99:0/3561896553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1b0070c40 0x7fc1b00ca4c0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:29.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.261+0000 7fc1b659f700 1 --2- 172.21.15.99:0/3561896553 >> v2:172.21.15.191:3300/0 conn(0x7fc1b0157ba0 0x7fc1b0158c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:29.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.261+0000 7fc1b659f700 1 --2- 172.21.15.99:0/3561896553 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1b015b7e0 0x7fc1b0070700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:29.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.261+0000 7fc1b659f700 1 -- 172.21.15.99:0/3561896553 >> 172.21.15.99:0/3561896553 conn(0x7fc1b006be80 msgr2=0x7fc1b006c280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:29.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.261+0000 7fc1b659f700 1 -- 172.21.15.99:0/3561896553 shutdown_connections 2023-10-08T00:17:29.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.261+0000 7fc1b659f700 1 -- 172.21.15.99:0/3561896553 wait complete. 2023-10-08T00:17:29.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.262+0000 7fc1b659f700 1 Processor -- start 2023-10-08T00:17:29.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.262+0000 7fc1b659f700 1 -- start start 2023-10-08T00:17:29.268 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.262+0000 7fc1b659f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1b0070c40 0x7fc1b00c5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:29.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.262+0000 7fc1b659f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1b0157ba0 0x7fc1b0168ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:29.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.263+0000 7fc1b659f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1b015b7e0 0x7fc1b00c4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:29.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.263+0000 7fc1b659f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b00c15c0 con 0x7fc1b015b7e0 2023-10-08T00:17:29.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.263+0000 7fc1b659f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b00c50b0 con 0x7fc1b0157ba0 2023-10-08T00:17:29.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.263+0000 7fc1b659f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b00c5330 con 0x7fc1b0070c40 2023-10-08T00:17:29.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.263+0000 7fc1b4b3c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1b015b7e0 0x7fc1b00c4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:29.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.263+0000 7fc1b4b3c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1b015b7e0 0x7fc1b00c4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45142/0 (socket says 172.21.15.99:45142) 2023-10-08T00:17:29.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.263+0000 7fc1b4b3c700 1 -- 172.21.15.99:0/474300573 learned_addr learned my addr 172.21.15.99:0/474300573 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:29.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.263+0000 7fc1affff700 1 --2- 172.21.15.99:0/474300573 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1b0070c40 0x7fc1b00c5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:29.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.263+0000 7fc1af7fe700 1 --2- 172.21.15.99:0/474300573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1b0157ba0 0x7fc1b0168ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:29.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.263+0000 7fc1b4b3c700 1 -- 172.21.15.99:0/474300573 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1b0070c40 msgr2=0x7fc1b00c5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:29.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.263+0000 7fc1b4b3c700 1 --2- 172.21.15.99:0/474300573 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1b0070c40 0x7fc1b00c5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:29.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.263+0000 7fc1b4b3c700 1 -- 172.21.15.99:0/474300573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1b0157ba0 msgr2=0x7fc1b0168ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:29.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.263+0000 7fc1b4b3c700 1 --2- 172.21.15.99:0/474300573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1b0157ba0 0x7fc1b0168ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:29.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.263+0000 7fc1b4b3c700 1 -- 172.21.15.99:0/474300573 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a8009ae0 con 0x7fc1b015b7e0 2023-10-08T00:17:29.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.263+0000 7fc1b4b3c700 1 --2- 172.21.15.99:0/474300573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1b015b7e0 0x7fc1b00c4b70 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8009fd0 tx=0x7fc1a8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:29.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.263+0000 7fc197fff700 1 -- 172.21.15.99:0/474300573 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a80032f0 con 0x7fc1b015b7e0 2023-10-08T00:17:29.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.263+0000 7fc197fff700 1 -- 172.21.15.99:0/474300573 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1a8008d10 con 0x7fc1b015b7e0 2023-10-08T00:17:29.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.263+0000 7fc197fff700 1 -- 172.21.15.99:0/474300573 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8025690 con 0x7fc1b015b7e0 2023-10-08T00:17:29.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.263+0000 7fc1b659f700 1 -- 172.21.15.99:0/474300573 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1b00c5610 con 0x7fc1b015b7e0 2023-10-08T00:17:29.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.263+0000 7fc1b659f700 1 -- 172.21.15.99:0/474300573 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1b00c1810 con 0x7fc1b015b7e0 2023-10-08T00:17:29.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.264+0000 7fc1b659f700 1 -- 172.21.15.99:0/474300573 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1b004c3c0 con 0x7fc1b015b7e0 2023-10-08T00:17:29.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.264+0000 7fc197fff700 1 -- 172.21.15.99:0/474300573 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc1a8003040 con 0x7fc1b015b7e0 2023-10-08T00:17:29.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.265+0000 7fc197fff700 1 --2- 172.21.15.99:0/474300573 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc19c066f30 0x7fc19c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:29.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.265+0000 7fc197fff700 1 -- 172.21.15.99:0/474300573 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5791+0+0 (secure 0 0 0) 0x7fc1a8085c60 con 0x7fc1b015b7e0 2023-10-08T00:17:29.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.265+0000 7fc1affff700 1 --2- 172.21.15.99:0/474300573 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc19c066f30 0x7fc19c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:29.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.266+0000 7fc1affff700 1 --2- 172.21.15.99:0/474300573 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc19c066f30 0x7fc19c0693e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc1b00c45c0 tx=0x7fc1a0006f90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:29.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.269+0000 7fc197fff700 1 -- 172.21.15.99:0/474300573 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc1a8052f30 con 0x7fc1b015b7e0 2023-10-08T00:17:29.494 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:29.494+0000 7fc1b659f700 1 -- 172.21.15.99:0/474300573 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc1b00ad600 con 0x7fc1b015b7e0 2023-10-08T00:17:30.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.125+0000 7fc197fff700 1 -- 172.21.15.99:0/474300573 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v65) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc1a8056550 con 0x7fc1b015b7e0 2023-10-08T00:17:30.128 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.127+0000 7fc1b659f700 1 -- 172.21.15.99:0/474300573 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc19c066f30 msgr2=0x7fc19c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:30.128 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.127+0000 7fc1b659f700 1 --2- 172.21.15.99:0/474300573 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc19c066f30 0x7fc19c0693e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc1b00c45c0 tx=0x7fc1a0006f90 comp rx=0 tx=0).stop 2023-10-08T00:17:30.128 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.127+0000 7fc1b659f700 1 -- 172.21.15.99:0/474300573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1b015b7e0 msgr2=0x7fc1b00c4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:30.129 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.127+0000 7fc1b659f700 1 --2- 172.21.15.99:0/474300573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1b015b7e0 0x7fc1b00c4b70 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8009fd0 tx=0x7fc1a8018000 comp rx=0 tx=0).stop 2023-10-08T00:17:30.129 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.127+0000 7fc1b659f700 1 -- 172.21.15.99:0/474300573 shutdown_connections 2023-10-08T00:17:30.129 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.127+0000 7fc1b659f700 1 --2- 172.21.15.99:0/474300573 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc19c066f30 0x7fc19c0693e0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:30.129 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.127+0000 7fc1b659f700 1 --2- 172.21.15.99:0/474300573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1b015b7e0 0x7fc1b00c4b70 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:30.129 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.127+0000 7fc1b659f700 1 --2- 172.21.15.99:0/474300573 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1b0070c40 0x7fc1b00c5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:30.129 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.127+0000 7fc1b659f700 1 --2- 172.21.15.99:0/474300573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1b0157ba0 0x7fc1b0168ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:30.129 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.127+0000 7fc1b659f700 1 -- 172.21.15.99:0/474300573 >> 172.21.15.99:0/474300573 conn(0x7fc1b006be80 msgr2=0x7fc1b0154f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:30.129 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.128+0000 7fc1b659f700 1 -- 172.21.15.99:0/474300573 shutdown_connections 2023-10-08T00:17:30.129 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.128+0000 7fc1b659f700 1 -- 172.21.15.99:0/474300573 wait complete. 2023-10-08T00:17:30.141 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T00:17:30.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.300+0000 7f6851bc7700 1 Processor -- start 2023-10-08T00:17:30.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.300+0000 7f6851bc7700 1 -- start start 2023-10-08T00:17:30.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.301+0000 7f6851bc7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f684c159ef0 0x7f684c15a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:30.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.301+0000 7f6851bc7700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f684c0b46f0 0x7f684c0b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:30.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.301+0000 7f6851bc7700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f684c0b5090 con 0x7f684c0b6a10 2023-10-08T00:17:30.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.301+0000 7f6851bc7700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f684c0b17b0 con 0x7f684c0b46f0 2023-10-08T00:17:30.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.301+0000 7f6851bc7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f684c0b1a60 con 0x7f684c159ef0 2023-10-08T00:17:30.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.301+0000 7f684bfff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f684c0b46f0 0x7f684c0b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:30.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.301+0000 7f684b7fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f684c159ef0 0x7f684c15a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:30.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.301+0000 7f684bfff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f684c0b46f0 0x7f684c0b4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.99:56280/0 (socket says 172.21.15.99:56280) 2023-10-08T00:17:30.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.301+0000 7f684b7fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f684c159ef0 0x7f684c15a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.99:37004/0 (socket says 172.21.15.99:37004) 2023-10-08T00:17:30.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.302+0000 7f684bfff700 1 -- 172.21.15.99:0/1198566898 learned_addr learned my addr 172.21.15.99:0/1198566898 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:30.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.301+0000 7f684affd700 1 --1- 172.21.15.99:0/1198566898 >> v1:172.21.15.191:6789/0 conn(0x7f684c0b6a10 0x7f684c0b10a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:58764/0 (socket says 172.21.15.99:58764) 2023-10-08T00:17:30.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.302+0000 7f684bfff700 1 -- 172.21.15.99:0/1198566898 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f684c159ef0 msgr2=0x7f684c15a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:30.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.302+0000 7f684bfff700 1 --2- 172.21.15.99:0/1198566898 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f684c159ef0 0x7f684c15a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:30.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.302+0000 7f684bfff700 1 -- 172.21.15.99:0/1198566898 >> v1:172.21.15.191:6789/0 conn(0x7f684c0b6a10 legacy=0x7f684c0b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:30.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.302+0000 7f684bfff700 1 -- 172.21.15.99:0/1198566898 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f684c0b1d40 con 0x7f684c0b46f0 2023-10-08T00:17:30.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.302+0000 7f684a7fc700 1 -- 172.21.15.99:0/1198566898 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1044623396 0 0) 0x7f6840001110 con 0x7f684c0b6a10 2023-10-08T00:17:30.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.302+0000 7f684bfff700 1 --2- 172.21.15.99:0/1198566898 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f684c0b46f0 0x7f684c0b4b50 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f6844009f20 tx=0x7f6844009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6e5c5fbf2bdf7a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:30.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.302+0000 7f684a7fc700 1 -- 172.21.15.99:0/1198566898 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684400e070 con 0x7f684c0b46f0 2023-10-08T00:17:30.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.302+0000 7f684a7fc700 1 -- 172.21.15.99:0/1198566898 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f684400b9d0 con 0x7f684c0b46f0 2023-10-08T00:17:30.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.302+0000 7f684a7fc700 1 -- 172.21.15.99:0/1198566898 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68440092e0 con 0x7f684c0b46f0 2023-10-08T00:17:30.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.303+0000 7f6851bc7700 1 -- 172.21.15.99:0/1198566898 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f684c0b46f0 msgr2=0x7f684c0b4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:30.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.303+0000 7f6851bc7700 1 --2- 172.21.15.99:0/1198566898 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f684c0b46f0 0x7f684c0b4b50 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f6844009f20 tx=0x7f6844009840 comp rx=0 tx=0).stop 2023-10-08T00:17:30.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.303+0000 7f6851bc7700 1 -- 172.21.15.99:0/1198566898 shutdown_connections 2023-10-08T00:17:30.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.303+0000 7f6851bc7700 1 --2- 172.21.15.99:0/1198566898 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f684c0b46f0 0x7f684c0b4b50 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:30.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.303+0000 7f6851bc7700 1 --2- 172.21.15.99:0/1198566898 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f684c159ef0 0x7f684c15a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:30.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.303+0000 7f6851bc7700 1 -- 172.21.15.99:0/1198566898 >> 172.21.15.99:0/1198566898 conn(0x7f684c09a0a0 msgr2=0x7f684c067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:30.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.303+0000 7f6851bc7700 1 -- 172.21.15.99:0/1198566898 shutdown_connections 2023-10-08T00:17:30.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.303+0000 7f6851bc7700 1 -- 172.21.15.99:0/1198566898 wait complete. 2023-10-08T00:17:30.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.304+0000 7f6851bc7700 1 Processor -- start 2023-10-08T00:17:30.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.304+0000 7f6851bc7700 1 -- start start 2023-10-08T00:17:30.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.305+0000 7f6851bc7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f684c0b46f0 0x7f684c09b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:30.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.305+0000 7f6851bc7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f684c0b6a10 0x7f684c146280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:30.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.305+0000 7f6851bc7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f684c159ef0 0x7f684c09c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:30.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.305+0000 7f6851bc7700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684c09ba60 con 0x7f684c0b6a10 2023-10-08T00:17:30.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.305+0000 7f6851bc7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684c1467c0 con 0x7f684c0b46f0 2023-10-08T00:17:30.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.305+0000 7f6851bc7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f684c0ae0c0 con 0x7f684c159ef0 2023-10-08T00:17:30.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.305+0000 7f684affd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f684c0b6a10 0x7f684c146280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:30.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.305+0000 7f684affd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f684c0b6a10 0x7f684c146280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45150/0 (socket says 172.21.15.99:45150) 2023-10-08T00:17:30.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.305+0000 7f684affd700 1 -- 172.21.15.99:0/767864809 learned_addr learned my addr 172.21.15.99:0/767864809 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:30.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.305+0000 7f684b7fe700 1 --2- 172.21.15.99:0/767864809 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f684c0b46f0 0x7f684c09b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:30.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.305+0000 7f684bfff700 1 --2- 172.21.15.99:0/767864809 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f684c159ef0 0x7f684c09c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:30.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.305+0000 7f684affd700 1 -- 172.21.15.99:0/767864809 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f684c159ef0 msgr2=0x7f684c09c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:30.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.305+0000 7f684affd700 1 --2- 172.21.15.99:0/767864809 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f684c159ef0 0x7f684c09c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:30.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.305+0000 7f684affd700 1 -- 172.21.15.99:0/767864809 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f684c0b46f0 msgr2=0x7f684c09b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:30.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.305+0000 7f684affd700 1 --2- 172.21.15.99:0/767864809 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f684c0b46f0 0x7f684c09b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:30.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.305+0000 7f684affd700 1 -- 172.21.15.99:0/767864809 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6844009ae0 con 0x7f684c0b6a10 2023-10-08T00:17:30.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.306+0000 7f684affd700 1 --2- 172.21.15.99:0/767864809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f684c0b6a10 0x7f684c146280 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f684000c810 tx=0x7f6840007c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:30.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.306+0000 7f68337fe700 1 -- 172.21.15.99:0/767864809 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6840005a70 con 0x7f684c0b6a10 2023-10-08T00:17:30.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.306+0000 7f68337fe700 1 -- 172.21.15.99:0/767864809 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6840005bf0 con 0x7f684c0b6a10 2023-10-08T00:17:30.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.306+0000 7f68337fe700 1 -- 172.21.15.99:0/767864809 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68400070d0 con 0x7f684c0b6a10 2023-10-08T00:17:30.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.306+0000 7f6851bc7700 1 -- 172.21.15.99:0/767864809 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f684c0ae3a0 con 0x7f684c0b6a10 2023-10-08T00:17:30.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.306+0000 7f6851bc7700 1 -- 172.21.15.99:0/767864809 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f684c0ae960 con 0x7f684c0b6a10 2023-10-08T00:17:30.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.307+0000 7f6851bc7700 1 -- 172.21.15.99:0/767864809 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f684c04c3c0 con 0x7f684c0b6a10 2023-10-08T00:17:30.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.307+0000 7f68337fe700 1 -- 172.21.15.99:0/767864809 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f68400074c0 con 0x7f684c0b6a10 2023-10-08T00:17:30.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.307+0000 7f68337fe700 1 --2- 172.21.15.99:0/767864809 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6838066f30 0x7f68380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:30.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.307+0000 7f68337fe700 1 -- 172.21.15.99:0/767864809 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f68400838e0 con 0x7f684c0b6a10 2023-10-08T00:17:30.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.308+0000 7f684b7fe700 1 --2- 172.21.15.99:0/767864809 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6838066f30 0x7f68380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:30.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.308+0000 7f684b7fe700 1 --2- 172.21.15.99:0/767864809 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6838066f30 0x7f68380693e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f683c009d00 tx=0x7f683c009120 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:30.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.312+0000 7f68337fe700 1 -- 172.21.15.99:0/767864809 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6840050b00 con 0x7f684c0b6a10 2023-10-08T00:17:30.494 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:30.493+0000 7f6851bc7700 1 -- 172.21.15.99:0/767864809 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f684c0b5090 con 0x7f684c0b6a10 2023-10-08T00:17:31.122 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.122+0000 7f68337fe700 1 -- 172.21.15.99:0/767864809 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v144) v1 ==== 181+0+0 (secure 0 0 0) 0x7f684c0b5090 con 0x7f684c0b6a10 2023-10-08T00:17:31.126 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.125+0000 7f6851bc7700 1 -- 172.21.15.99:0/767864809 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6838066f30 msgr2=0x7f68380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:31.126 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.125+0000 7f6851bc7700 1 --2- 172.21.15.99:0/767864809 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6838066f30 0x7f68380693e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f683c009d00 tx=0x7f683c009120 comp rx=0 tx=0).stop 2023-10-08T00:17:31.126 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.125+0000 7f6851bc7700 1 -- 172.21.15.99:0/767864809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f684c0b6a10 msgr2=0x7f684c146280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:31.127 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.125+0000 7f6851bc7700 1 --2- 172.21.15.99:0/767864809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f684c0b6a10 0x7f684c146280 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f684000c810 tx=0x7f6840007c50 comp rx=0 tx=0).stop 2023-10-08T00:17:31.127 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.125+0000 7f6851bc7700 1 -- 172.21.15.99:0/767864809 shutdown_connections 2023-10-08T00:17:31.127 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.125+0000 7f6851bc7700 1 --2- 172.21.15.99:0/767864809 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6838066f30 0x7f68380693e0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:31.127 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.125+0000 7f6851bc7700 1 --2- 172.21.15.99:0/767864809 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f684c0b46f0 0x7f684c09b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:31.127 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.125+0000 7f6851bc7700 1 --2- 172.21.15.99:0/767864809 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f684c159ef0 0x7f684c09c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:31.127 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.125+0000 7f6851bc7700 1 --2- 172.21.15.99:0/767864809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f684c0b6a10 0x7f684c146280 secure :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f684000c810 tx=0x7f6840007c50 comp rx=0 tx=0).stop 2023-10-08T00:17:31.127 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.125+0000 7f6851bc7700 1 -- 172.21.15.99:0/767864809 >> 172.21.15.99:0/767864809 conn(0x7f684c09a0a0 msgr2=0x7f684c1590e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:31.127 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.126+0000 7f6851bc7700 1 -- 172.21.15.99:0/767864809 shutdown_connections 2023-10-08T00:17:31.128 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.127+0000 7f6851bc7700 1 -- 172.21.15.99:0/767864809 wait complete. 2023-10-08T00:17:31.128 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' removed 2023-10-08T00:17:31.140 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T00:17:31.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.259+0000 7f0a4759e700 1 Processor -- start 2023-10-08T00:17:31.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.259+0000 7f0a4759e700 1 -- start start 2023-10-08T00:17:31.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.260+0000 7f0a4759e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a4815d240 0x7f0a480b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:31.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.260+0000 7f0a4759e700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0a480ce810 0x7f0a480cec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:31.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.260+0000 7f0a4759e700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a480cd4c0 con 0x7f0a480cc940 2023-10-08T00:17:31.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.260+0000 7f0a4759e700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a480b3f40 con 0x7f0a480ce810 2023-10-08T00:17:31.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.260+0000 7f0a4759e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a480cbb80 con 0x7f0a4815d240 2023-10-08T00:17:31.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.260+0000 7f0a46d9d700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0a480ce810 0x7f0a480cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:31.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.260+0000 7f0a3ffff700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f0a480cc940 0x7f0a480ccdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50990/0 (socket says 172.21.15.99:50990) 2023-10-08T00:17:31.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.260+0000 7f0a3ffff700 1 -- 172.21.15.99:0/404685185 learned_addr learned my addr 172.21.15.99:0/404685185 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:17:31.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.260+0000 7f0a4659c700 1 --2- 172.21.15.99:0/404685185 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a4815d240 0x7f0a480b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:31.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.260+0000 7f0a46d9d700 1 -- 172.21.15.99:0/404685185 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a4815d240 msgr2=0x7f0a480b3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:31.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.260+0000 7f0a46d9d700 1 --2- 172.21.15.99:0/404685185 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a4815d240 0x7f0a480b3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:31.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.260+0000 7f0a46d9d700 1 -- 172.21.15.99:0/404685185 >> v1:172.21.15.99:6789/0 conn(0x7f0a480cc940 legacy=0x7f0a480ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:31.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.260+0000 7f0a46d9d700 1 -- 172.21.15.99:0/404685185 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a480cbe60 con 0x7f0a480ce810 2023-10-08T00:17:31.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.260+0000 7f0a45d9b700 1 -- 172.21.15.99:0/404685185 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3965620925 0 0) 0x7f0a480cd4c0 con 0x7f0a480cc940 2023-10-08T00:17:31.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.261+0000 7f0a46d9d700 1 --2- 172.21.15.99:0/404685185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a480ce810 0x7f0a480cec90 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f0a40009e60 tx=0x7f0a40009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beae205f38e90ae3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:31.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.261+0000 7f0a45d9b700 1 -- 172.21.15.99:0/404685185 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4000e070 con 0x7f0a480ce810 2023-10-08T00:17:31.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.261+0000 7f0a45d9b700 1 -- 172.21.15.99:0/404685185 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a4000bb60 con 0x7f0a480ce810 2023-10-08T00:17:31.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.261+0000 7f0a45d9b700 1 -- 172.21.15.99:0/404685185 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a40012430 con 0x7f0a480ce810 2023-10-08T00:17:31.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.261+0000 7f0a4759e700 1 -- 172.21.15.99:0/404685185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a480ce810 msgr2=0x7f0a480cec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:31.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.261+0000 7f0a4759e700 1 --2- 172.21.15.99:0/404685185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a480ce810 0x7f0a480cec90 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f0a40009e60 tx=0x7f0a40009780 comp rx=0 tx=0).stop 2023-10-08T00:17:31.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.261+0000 7f0a4759e700 1 -- 172.21.15.99:0/404685185 shutdown_connections 2023-10-08T00:17:31.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.261+0000 7f0a4759e700 1 --2- 172.21.15.99:0/404685185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a480ce810 0x7f0a480cec90 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:31.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.261+0000 7f0a4759e700 1 --2- 172.21.15.99:0/404685185 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a4815d240 0x7f0a480b3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:31.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.261+0000 7f0a4759e700 1 -- 172.21.15.99:0/404685185 >> 172.21.15.99:0/404685185 conn(0x7f0a48067e40 msgr2=0x7f0a480d0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:31.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.261+0000 7f0a4759e700 1 -- 172.21.15.99:0/404685185 shutdown_connections 2023-10-08T00:17:31.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.261+0000 7f0a4759e700 1 -- 172.21.15.99:0/404685185 wait complete. 2023-10-08T00:17:31.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.262+0000 7f0a4759e700 1 Processor -- start 2023-10-08T00:17:31.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.262+0000 7f0a4759e700 1 -- start start 2023-10-08T00:17:31.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.263+0000 7f0a4759e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a480cc940 0x7f0a480cac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:31.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.263+0000 7f0a4759e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a480ce810 0x7f0a48069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:31.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.263+0000 7f0a4759e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a4815d240 0x7f0a48069f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:31.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.263+0000 7f0a4759e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a48157f50 con 0x7f0a480ce810 2023-10-08T00:17:31.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.263+0000 7f0a4759e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a48153d20 con 0x7f0a480cc940 2023-10-08T00:17:31.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.263+0000 7f0a4759e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a48157340 con 0x7f0a4815d240 2023-10-08T00:17:31.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.263+0000 7f0a3ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a480ce810 0x7f0a48069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:31.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.263+0000 7f0a3ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a480ce810 0x7f0a48069a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45174/0 (socket says 172.21.15.99:45174) 2023-10-08T00:17:31.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.263+0000 7f0a3ffff700 1 -- 172.21.15.99:0/714394077 learned_addr learned my addr 172.21.15.99:0/714394077 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:31.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.263+0000 7f0a46d9d700 1 --2- 172.21.15.99:0/714394077 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a4815d240 0x7f0a48069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:31.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.263+0000 7f0a4659c700 1 --2- 172.21.15.99:0/714394077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a480cc940 0x7f0a480cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:31.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.263+0000 7f0a3ffff700 1 -- 172.21.15.99:0/714394077 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a4815d240 msgr2=0x7f0a48069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:31.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.263+0000 7f0a3ffff700 1 --2- 172.21.15.99:0/714394077 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a4815d240 0x7f0a48069f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:31.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.263+0000 7f0a3ffff700 1 -- 172.21.15.99:0/714394077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a480cc940 msgr2=0x7f0a480cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:31.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.263+0000 7f0a3ffff700 1 --2- 172.21.15.99:0/714394077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a480cc940 0x7f0a480cac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:31.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.263+0000 7f0a3ffff700 1 -- 172.21.15.99:0/714394077 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a40009a20 con 0x7f0a480ce810 2023-10-08T00:17:31.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.264+0000 7f0a3ffff700 1 --2- 172.21.15.99:0/714394077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a480ce810 0x7f0a48069a10 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f0a3000c560 tx=0x7f0a30000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:31.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.264+0000 7f0a3e7fc700 1 -- 172.21.15.99:0/714394077 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a30005b30 con 0x7f0a480ce810 2023-10-08T00:17:31.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.264+0000 7f0a3e7fc700 1 -- 172.21.15.99:0/714394077 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a30005c90 con 0x7f0a480ce810 2023-10-08T00:17:31.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.264+0000 7f0a3e7fc700 1 -- 172.21.15.99:0/714394077 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a30007320 con 0x7f0a480ce810 2023-10-08T00:17:31.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.264+0000 7f0a4759e700 1 -- 172.21.15.99:0/714394077 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a48157620 con 0x7f0a480ce810 2023-10-08T00:17:31.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.264+0000 7f0a4759e700 1 -- 172.21.15.99:0/714394077 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a48157be0 con 0x7f0a480ce810 2023-10-08T00:17:31.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.265+0000 7f0a4759e700 1 -- 172.21.15.99:0/714394077 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a481580c0 con 0x7f0a480ce810 2023-10-08T00:17:31.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.265+0000 7f0a3e7fc700 1 -- 172.21.15.99:0/714394077 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0a3001b3b0 con 0x7f0a480ce810 2023-10-08T00:17:31.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.266+0000 7f0a3e7fc700 1 --2- 172.21.15.99:0/714394077 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0a28066f30 0x7f0a280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:31.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.266+0000 7f0a3e7fc700 1 -- 172.21.15.99:0/714394077 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5353+0+0 (secure 0 0 0) 0x7f0a300837f0 con 0x7f0a480ce810 2023-10-08T00:17:31.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.266+0000 7f0a4659c700 1 --2- 172.21.15.99:0/714394077 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0a28066f30 0x7f0a280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:31.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.266+0000 7f0a3e7fc700 1 -- 172.21.15.99:0/714394077 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=145}) v3 -- 0x7f0a2806dcf0 con 0x7f0a480ce810 2023-10-08T00:17:31.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.266+0000 7f0a4659c700 1 --2- 172.21.15.99:0/714394077 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0a28066f30 0x7f0a280693e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f0a38009d00 tx=0x7f0a38009120 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:31.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.270+0000 7f0a3e7fc700 1 -- 172.21.15.99:0/714394077 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0a30017060 con 0x7f0a480ce810 2023-10-08T00:17:31.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:31.454+0000 7f0a4759e700 1 -- 172.21.15.99:0/714394077 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0a4806ac60 con 0x7f0a480ce810 2023-10-08T00:17:32.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.189+0000 7f0a3e7fc700 1 -- 172.21.15.99:0/714394077 <== mon.0 v2:172.21.15.99:3300/0 7 ==== osd_map(145..145 src has 1..145) v4 ==== 288+0+0 (secure 0 0 0) 0x7f0a30054f80 con 0x7f0a480ce810 2023-10-08T00:17:32.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.189+0000 7f0a3e7fc700 1 -- 172.21.15.99:0/714394077 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=146}) v3 -- 0x7f0a2806ca80 con 0x7f0a480ce810 2023-10-08T00:17:32.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.195+0000 7f0a3e7fc700 1 -- 172.21.15.99:0/714394077 <== mon.0 v2:172.21.15.99:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v145) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0a30054650 con 0x7f0a480ce810 2023-10-08T00:17:32.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.200+0000 7f0a4759e700 1 -- 172.21.15.99:0/714394077 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0a28066f30 msgr2=0x7f0a280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:32.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.200+0000 7f0a4759e700 1 --2- 172.21.15.99:0/714394077 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0a28066f30 0x7f0a280693e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f0a38009d00 tx=0x7f0a38009120 comp rx=0 tx=0).stop 2023-10-08T00:17:32.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.200+0000 7f0a4759e700 1 -- 172.21.15.99:0/714394077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a480ce810 msgr2=0x7f0a48069a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:32.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.200+0000 7f0a4759e700 1 --2- 172.21.15.99:0/714394077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a480ce810 0x7f0a48069a10 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f0a3000c560 tx=0x7f0a30000fe0 comp rx=0 tx=0).stop 2023-10-08T00:17:32.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.200+0000 7f0a4759e700 1 -- 172.21.15.99:0/714394077 shutdown_connections 2023-10-08T00:17:32.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.200+0000 7f0a4759e700 1 --2- 172.21.15.99:0/714394077 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0a28066f30 0x7f0a280693e0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:32.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.200+0000 7f0a4759e700 1 --2- 172.21.15.99:0/714394077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a480cc940 0x7f0a480cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:32.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.200+0000 7f0a4759e700 1 --2- 172.21.15.99:0/714394077 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a4815d240 0x7f0a48069f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:32.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.200+0000 7f0a4759e700 1 --2- 172.21.15.99:0/714394077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a480ce810 0x7f0a48069a10 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:32.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.200+0000 7f0a4759e700 1 -- 172.21.15.99:0/714394077 >> 172.21.15.99:0/714394077 conn(0x7f0a48067e40 msgr2=0x7f0a480c6b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:32.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.201+0000 7f0a4759e700 1 -- 172.21.15.99:0/714394077 shutdown_connections 2023-10-08T00:17:32.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.201+0000 7f0a4759e700 1 -- 172.21.15.99:0/714394077 wait complete. 2023-10-08T00:17:32.203 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' removed 2023-10-08T00:17:32.216 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-08T00:17:32.333 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.332+0000 7fa537a3f700 1 Processor -- start 2023-10-08T00:17:32.333 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.332+0000 7fa537a3f700 1 -- start start 2023-10-08T00:17:32.335 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.333+0000 7fa537a3f700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa530159ec0 0x7fa5300af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:32.335 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.333+0000 7fa537a3f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa53015a290 0x7fa5300afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:32.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.333+0000 7fa537a3f700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5300b0790 con 0x7fa53006a5a0 2023-10-08T00:17:32.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.333+0000 7fa537a3f700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5300ce1f0 con 0x7fa530159ec0 2023-10-08T00:17:32.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.333+0000 7fa537a3f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5300b0b20 con 0x7fa53015a290 2023-10-08T00:17:32.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.333+0000 7fa5357db700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa530159ec0 0x7fa5300af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:32.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.333+0000 7fa5357db700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa530159ec0 0x7fa5300af940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45184/0 (socket says 172.21.15.99:45184) 2023-10-08T00:17:32.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.333+0000 7fa5357db700 1 -- 172.21.15.99:0/820021177 learned_addr learned my addr 172.21.15.99:0/820021177 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:32.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.333+0000 7fa5357db700 1 -- 172.21.15.99:0/820021177 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa53015a290 msgr2=0x7fa5300afe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:32.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.333+0000 7fa5357db700 1 --2- 172.21.15.99:0/820021177 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa53015a290 0x7fa5300afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:32.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.333+0000 7fa5357db700 1 -- 172.21.15.99:0/820021177 >> v1:172.21.15.191:6789/0 conn(0x7fa53006a5a0 legacy=0x7fa53015db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:32.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.334+0000 7fa5357db700 1 -- 172.21.15.99:0/820021177 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5300b04b0 con 0x7fa530159ec0 2023-10-08T00:17:32.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.334+0000 7fa5357db700 1 --2- 172.21.15.99:0/820021177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa530159ec0 0x7fa5300af940 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fa52c009f20 tx=0x7fa52c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a235846a47a857 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:32.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.334+0000 7fa527fff700 1 -- 172.21.15.99:0/820021177 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c00e070 con 0x7fa530159ec0 2023-10-08T00:17:32.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.334+0000 7fa527fff700 1 -- 172.21.15.99:0/820021177 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa52c00ba40 con 0x7fa530159ec0 2023-10-08T00:17:32.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.334+0000 7fa527fff700 1 -- 172.21.15.99:0/820021177 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c0092e0 con 0x7fa530159ec0 2023-10-08T00:17:32.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.334+0000 7fa537a3f700 1 -- 172.21.15.99:0/820021177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa530159ec0 msgr2=0x7fa5300af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:32.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.334+0000 7fa537a3f700 1 --2- 172.21.15.99:0/820021177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa530159ec0 0x7fa5300af940 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fa52c009f20 tx=0x7fa52c009840 comp rx=0 tx=0).stop 2023-10-08T00:17:32.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.334+0000 7fa537a3f700 1 -- 172.21.15.99:0/820021177 shutdown_connections 2023-10-08T00:17:32.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.334+0000 7fa537a3f700 1 --2- 172.21.15.99:0/820021177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa530159ec0 0x7fa5300af940 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:32.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.334+0000 7fa537a3f700 1 --2- 172.21.15.99:0/820021177 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa53015a290 0x7fa5300afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:32.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.334+0000 7fa537a3f700 1 -- 172.21.15.99:0/820021177 >> 172.21.15.99:0/820021177 conn(0x7fa5300d1880 msgr2=0x7fa5300cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:32.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.335+0000 7fa537a3f700 1 -- 172.21.15.99:0/820021177 shutdown_connections 2023-10-08T00:17:32.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.335+0000 7fa537a3f700 1 -- 172.21.15.99:0/820021177 wait complete. 2023-10-08T00:17:32.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.336+0000 7fa537a3f700 1 Processor -- start 2023-10-08T00:17:32.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.336+0000 7fa537a3f700 1 -- start start 2023-10-08T00:17:32.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.337+0000 7fa537a3f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa53006a5a0 0x7fa530156cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:32.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.337+0000 7fa537a3f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa530159ec0 0x7fa530158b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:32.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.337+0000 7fa537a3f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa53015a290 0x7fa5300c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:32.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.337+0000 7fa537a3f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5300b3be0 con 0x7fa530159ec0 2023-10-08T00:17:32.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.337+0000 7fa537a3f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5301590a0 con 0x7fa53006a5a0 2023-10-08T00:17:32.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.337+0000 7fa537a3f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5301572e0 con 0x7fa53015a290 2023-10-08T00:17:32.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.337+0000 7fa534fda700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa530159ec0 0x7fa530158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:32.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.337+0000 7fa534fda700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa530159ec0 0x7fa530158b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45196/0 (socket says 172.21.15.99:45196) 2023-10-08T00:17:32.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.337+0000 7fa534fda700 1 -- 172.21.15.99:0/2309319868 learned_addr learned my addr 172.21.15.99:0/2309319868 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:32.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.337+0000 7fa535fdc700 1 --2- 172.21.15.99:0/2309319868 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa53015a290 0x7fa5300c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:32.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.337+0000 7fa5357db700 1 --2- 172.21.15.99:0/2309319868 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa53006a5a0 0x7fa530156cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:32.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.337+0000 7fa534fda700 1 -- 172.21.15.99:0/2309319868 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa53015a290 msgr2=0x7fa5300c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:32.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.337+0000 7fa534fda700 1 --2- 172.21.15.99:0/2309319868 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa53015a290 0x7fa5300c8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:32.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.337+0000 7fa534fda700 1 -- 172.21.15.99:0/2309319868 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa53006a5a0 msgr2=0x7fa530156cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:32.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.337+0000 7fa534fda700 1 --2- 172.21.15.99:0/2309319868 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa53006a5a0 0x7fa530156cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:32.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.337+0000 7fa534fda700 1 -- 172.21.15.99:0/2309319868 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa52c009ae0 con 0x7fa530159ec0 2023-10-08T00:17:32.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.337+0000 7fa5357db700 1 --2- 172.21.15.99:0/2309319868 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa53006a5a0 0x7fa530156cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:17:32.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.337+0000 7fa534fda700 1 --2- 172.21.15.99:0/2309319868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa530159ec0 0x7fa530158b60 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fa52000b810 tx=0x7fa520009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:32.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.337+0000 7fa5257fa700 1 -- 172.21.15.99:0/2309319868 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52000f510 con 0x7fa530159ec0 2023-10-08T00:17:32.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.337+0000 7fa5257fa700 1 -- 172.21.15.99:0/2309319868 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa52000f690 con 0x7fa530159ec0 2023-10-08T00:17:32.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.337+0000 7fa537a3f700 1 -- 172.21.15.99:0/2309319868 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5301575c0 con 0x7fa530159ec0 2023-10-08T00:17:32.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.337+0000 7fa5257fa700 1 -- 172.21.15.99:0/2309319868 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52000fa50 con 0x7fa530159ec0 2023-10-08T00:17:32.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.337+0000 7fa537a3f700 1 -- 172.21.15.99:0/2309319868 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa530157b80 con 0x7fa530159ec0 2023-10-08T00:17:32.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.338+0000 7fa537a3f700 1 -- 172.21.15.99:0/2309319868 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5300b3d90 con 0x7fa530159ec0 2023-10-08T00:17:32.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.339+0000 7fa5257fa700 1 -- 172.21.15.99:0/2309319868 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa52001a400 con 0x7fa530159ec0 2023-10-08T00:17:32.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.339+0000 7fa5257fa700 1 --2- 172.21.15.99:0/2309319868 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa514066f30 0x7fa5140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:32.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.339+0000 7fa5257fa700 1 -- 172.21.15.99:0/2309319868 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fa520083d90 con 0x7fa530159ec0 2023-10-08T00:17:32.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.339+0000 7fa5357db700 1 --2- 172.21.15.99:0/2309319868 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa514066f30 0x7fa5140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:32.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.340+0000 7fa5357db700 1 --2- 172.21.15.99:0/2309319868 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa514066f30 0x7fa5140693e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fa52c009880 tx=0x7fa52c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:32.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.343+0000 7fa5257fa700 1 -- 172.21.15.99:0/2309319868 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa5300b3d90 con 0x7fa530159ec0 2023-10-08T00:17:32.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.524+0000 7fa537a3f700 1 -- 172.21.15.99:0/2309319868 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fa53004c3c0 con 0x7fa530159ec0 2023-10-08T00:17:32.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.528+0000 7fa5257fa700 1 -- 172.21.15.99:0/2309319868 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa520051380 con 0x7fa530159ec0 2023-10-08T00:17:32.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.530+0000 7fa537a3f700 1 -- 172.21.15.99:0/2309319868 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa514066f30 msgr2=0x7fa5140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:32.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.530+0000 7fa537a3f700 1 --2- 172.21.15.99:0/2309319868 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa514066f30 0x7fa5140693e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fa52c009880 tx=0x7fa52c017000 comp rx=0 tx=0).stop 2023-10-08T00:17:32.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.530+0000 7fa537a3f700 1 -- 172.21.15.99:0/2309319868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa530159ec0 msgr2=0x7fa530158b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:32.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.530+0000 7fa537a3f700 1 --2- 172.21.15.99:0/2309319868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa530159ec0 0x7fa530158b60 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fa52000b810 tx=0x7fa520009250 comp rx=0 tx=0).stop 2023-10-08T00:17:32.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.530+0000 7fa537a3f700 1 -- 172.21.15.99:0/2309319868 shutdown_connections 2023-10-08T00:17:32.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.530+0000 7fa537a3f700 1 --2- 172.21.15.99:0/2309319868 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa514066f30 0x7fa5140693e0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:32.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.530+0000 7fa537a3f700 1 --2- 172.21.15.99:0/2309319868 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa53006a5a0 0x7fa530156cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:32.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.530+0000 7fa537a3f700 1 --2- 172.21.15.99:0/2309319868 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa53015a290 0x7fa5300c8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:32.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.530+0000 7fa537a3f700 1 --2- 172.21.15.99:0/2309319868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa530159ec0 0x7fa530158b60 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:32.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.530+0000 7fa537a3f700 1 -- 172.21.15.99:0/2309319868 >> 172.21.15.99:0/2309319868 conn(0x7fa5300d1880 msgr2=0x7fa5300b2890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:32.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.530+0000 7fa537a3f700 1 -- 172.21.15.99:0/2309319868 shutdown_connections 2023-10-08T00:17:32.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.530+0000 7fa537a3f700 1 -- 172.21.15.99:0/2309319868 wait complete. 2023-10-08T00:17:32.542 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file' 2023-10-08T00:17:32.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.655+0000 7f93efb26700 1 Processor -- start 2023-10-08T00:17:32.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.656+0000 7f93efb26700 1 -- start start 2023-10-08T00:17:32.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.656+0000 7f93efb26700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f93e815a620 0x7f93e815d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:32.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.656+0000 7f93efb26700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93e815cbb0 con 0x7f93e8157870 2023-10-08T00:17:32.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.656+0000 7f93efb26700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93e815bfa0 con 0x7f93e815d580 2023-10-08T00:17:32.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.656+0000 7f93efb26700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93e815bdb0 con 0x7f93e815a620 2023-10-08T00:17:32.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.656+0000 7f93ed0c1700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f93e815a620 0x7f93e815d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:32.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.656+0000 7f93ed8c2700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f93e8157870 0x7f93e815b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50992/0 (socket says 172.21.15.99:50992) 2023-10-08T00:17:32.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.657+0000 7f93ed8c2700 1 -- 172.21.15.99:0/2658485273 learned_addr learned my addr 172.21.15.99:0/2658485273 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:17:32.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.657+0000 7f93ed0c1700 1 -- 172.21.15.99:0/2658485273 >> v1:172.21.15.191:6789/0 conn(0x7f93e815d580 legacy=0x7f93e8158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:32.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.657+0000 7f93ed0c1700 1 -- 172.21.15.99:0/2658485273 >> v1:172.21.15.99:6789/0 conn(0x7f93e8157870 legacy=0x7f93e815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:32.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.657+0000 7f93ed0c1700 1 -- 172.21.15.99:0/2658485273 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e815c280 con 0x7f93e815a620 2023-10-08T00:17:32.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.657+0000 7f93ed0c1700 1 --2- 172.21.15.99:0/2658485273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93e815a620 0x7f93e815d040 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f93d80098e0 tx=0x7f93d80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=765b7f5c88a0e7e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:32.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.657+0000 7f93ec8c0700 1 -- 172.21.15.99:0/2658485273 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d8009e80 con 0x7f93e815a620 2023-10-08T00:17:32.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.657+0000 7f93ec8c0700 1 -- 172.21.15.99:0/2658485273 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93d800b860 con 0x7f93e815a620 2023-10-08T00:17:32.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.657+0000 7f93ec8c0700 1 -- 172.21.15.99:0/2658485273 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d800bbe0 con 0x7f93e815a620 2023-10-08T00:17:32.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.658+0000 7f93efb26700 1 -- 172.21.15.99:0/2658485273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93e815a620 msgr2=0x7f93e815d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:32.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.658+0000 7f93efb26700 1 --2- 172.21.15.99:0/2658485273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93e815a620 0x7f93e815d040 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f93d80098e0 tx=0x7f93d80091a0 comp rx=0 tx=0).stop 2023-10-08T00:17:32.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.658+0000 7f93efb26700 1 -- 172.21.15.99:0/2658485273 shutdown_connections 2023-10-08T00:17:32.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.658+0000 7f93efb26700 1 --2- 172.21.15.99:0/2658485273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93e815a620 0x7f93e815d040 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:32.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.658+0000 7f93efb26700 1 -- 172.21.15.99:0/2658485273 >> 172.21.15.99:0/2658485273 conn(0x7f93e8068ce0 msgr2=0x7f93e80690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:32.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.658+0000 7f93efb26700 1 -- 172.21.15.99:0/2658485273 shutdown_connections 2023-10-08T00:17:32.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.658+0000 7f93efb26700 1 -- 172.21.15.99:0/2658485273 wait complete. 2023-10-08T00:17:32.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.659+0000 7f93efb26700 1 Processor -- start 2023-10-08T00:17:32.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.659+0000 7f93efb26700 1 -- start start 2023-10-08T00:17:32.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.659+0000 7f93efb26700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f93e8157870 0x7f93e8153550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:32.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.659+0000 7f93efb26700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93e815a620 0x7f93e814c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:32.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.659+0000 7f93efb26700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93e815d580 0x7f93e814ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:32.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.660+0000 7f93efb26700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e814feb0 con 0x7f93e815a620 2023-10-08T00:17:32.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.660+0000 7f93efb26700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e8152d90 con 0x7f93e815d580 2023-10-08T00:17:32.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.660+0000 7f93efb26700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93e81506c0 con 0x7f93e8157870 2023-10-08T00:17:32.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.660+0000 7f93ed0c1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93e815a620 0x7f93e814c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:32.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.660+0000 7f93ed0c1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93e815a620 0x7f93e814c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45208/0 (socket says 172.21.15.99:45208) 2023-10-08T00:17:32.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.660+0000 7f93ed0c1700 1 -- 172.21.15.99:0/2124075866 learned_addr learned my addr 172.21.15.99:0/2124075866 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:32.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.660+0000 7f93ee0c3700 1 --2- 172.21.15.99:0/2124075866 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93e815d580 0x7f93e814ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:32.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.660+0000 7f93ed8c2700 1 --2- 172.21.15.99:0/2124075866 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f93e8157870 0x7f93e8153550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:32.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.660+0000 7f93ed0c1700 1 -- 172.21.15.99:0/2124075866 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f93e8157870 msgr2=0x7f93e8153550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:32.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.660+0000 7f93ed0c1700 1 --2- 172.21.15.99:0/2124075866 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f93e8157870 0x7f93e8153550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:32.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.660+0000 7f93ed0c1700 1 -- 172.21.15.99:0/2124075866 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93e815d580 msgr2=0x7f93e814ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:32.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.660+0000 7f93ed0c1700 1 --2- 172.21.15.99:0/2124075866 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93e815d580 0x7f93e814ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:32.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.660+0000 7f93ed0c1700 1 -- 172.21.15.99:0/2124075866 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93d8009420 con 0x7f93e815a620 2023-10-08T00:17:32.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.660+0000 7f93ed0c1700 1 --2- 172.21.15.99:0/2124075866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93e815a620 0x7f93e814c4e0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f93d8013000 tx=0x7f93d8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:32.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.660+0000 7f93ddffb700 1 -- 172.21.15.99:0/2124075866 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d8011420 con 0x7f93e815a620 2023-10-08T00:17:32.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.660+0000 7f93ddffb700 1 -- 172.21.15.99:0/2124075866 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93d8011580 con 0x7f93e815a620 2023-10-08T00:17:32.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.660+0000 7f93efb26700 1 -- 172.21.15.99:0/2124075866 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93e8150100 con 0x7f93e815a620 2023-10-08T00:17:32.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.660+0000 7f93ddffb700 1 -- 172.21.15.99:0/2124075866 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d8011940 con 0x7f93e815a620 2023-10-08T00:17:32.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.660+0000 7f93efb26700 1 -- 172.21.15.99:0/2124075866 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93e8153030 con 0x7f93e815a620 2023-10-08T00:17:32.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.661+0000 7f93efb26700 1 -- 172.21.15.99:0/2124075866 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93e8153220 con 0x7f93e815a620 2023-10-08T00:17:32.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.662+0000 7f93ddffb700 1 -- 172.21.15.99:0/2124075866 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f93d8011cf0 con 0x7f93e815a620 2023-10-08T00:17:32.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.662+0000 7f93ddffb700 1 --2- 172.21.15.99:0/2124075866 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f93cc066f30 0x7f93cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:32.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.662+0000 7f93ddffb700 1 -- 172.21.15.99:0/2124075866 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f93d8085980 con 0x7f93e815a620 2023-10-08T00:17:32.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.662+0000 7f93ed8c2700 1 --2- 172.21.15.99:0/2124075866 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f93cc066f30 0x7f93cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:32.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.663+0000 7f93ed8c2700 1 --2- 172.21.15.99:0/2124075866 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f93cc066f30 0x7f93cc0693e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f93e4001820 tx=0x7f93e4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:32.667 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.666+0000 7f93ddffb700 1 -- 172.21.15.99:0/2124075866 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f93d8052ef0 con 0x7f93e815a620 2023-10-08T00:17:32.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:32.924+0000 7f93efb26700 1 -- 172.21.15.99:0/2124075866 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]} v 0) v1 -- 0x7f93e804c3c0 con 0x7f93e815a620 2023-10-08T00:17:33.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.188+0000 7f93ddffb700 1 -- 172.21.15.99:0/2124075866 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]}]=0 v346) v1 ==== 140+0+0 (secure 0 0 0) 0x7f93d8056510 con 0x7f93e815a620 2023-10-08T00:17:33.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.190+0000 7f93efb26700 1 -- 172.21.15.99:0/2124075866 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f93cc066f30 msgr2=0x7f93cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:33.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.190+0000 7f93efb26700 1 --2- 172.21.15.99:0/2124075866 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f93cc066f30 0x7f93cc0693e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f93e4001820 tx=0x7f93e4009040 comp rx=0 tx=0).stop 2023-10-08T00:17:33.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.190+0000 7f93efb26700 1 -- 172.21.15.99:0/2124075866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93e815a620 msgr2=0x7f93e814c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:33.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.190+0000 7f93efb26700 1 --2- 172.21.15.99:0/2124075866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93e815a620 0x7f93e814c4e0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f93d8013000 tx=0x7f93d8017000 comp rx=0 tx=0).stop 2023-10-08T00:17:33.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.190+0000 7f93efb26700 1 -- 172.21.15.99:0/2124075866 shutdown_connections 2023-10-08T00:17:33.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.190+0000 7f93efb26700 1 --2- 172.21.15.99:0/2124075866 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f93cc066f30 0x7f93cc0693e0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:33.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.190+0000 7f93efb26700 1 --2- 172.21.15.99:0/2124075866 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93e815d580 0x7f93e814ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:33.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.190+0000 7f93efb26700 1 --2- 172.21.15.99:0/2124075866 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f93e8157870 0x7f93e8153550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:33.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.190+0000 7f93efb26700 1 --2- 172.21.15.99:0/2124075866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93e815a620 0x7f93e814c4e0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:33.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.190+0000 7f93efb26700 1 -- 172.21.15.99:0/2124075866 >> 172.21.15.99:0/2124075866 conn(0x7f93e8068ce0 msgr2=0x7f93e80690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:33.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.190+0000 7f93efb26700 1 -- 172.21.15.99:0/2124075866 shutdown_connections 2023-10-08T00:17:33.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.190+0000 7f93efb26700 1 -- 172.21.15.99:0/2124075866 wait complete. 2023-10-08T00:17:33.203 INFO:tasks.cephfs_test_runner:test_full_different_file (tasks.cephfs.test_full.TestQuotaFull) ... ok 2023-10-08T00:17:33.203 INFO:tasks.cephfs_test_runner:Starting test: test_full_fclose (tasks.cephfs.test_full.TestQuotaFull) 2023-10-08T00:17:33.203 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose' 2023-10-08T00:17:33.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.317+0000 7f413359e700 1 Processor -- start 2023-10-08T00:17:33.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.317+0000 7f413359e700 1 -- start start 2023-10-08T00:17:33.321 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.318+0000 7f413359e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4134143520 0x7f41340cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:33.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.318+0000 7f413359e700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41340d0870 con 0x7f413406ab50 2023-10-08T00:17:33.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.318+0000 7f413359e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41340cdfb0 con 0x7f41341438f0 2023-10-08T00:17:33.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.318+0000 7f413359e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41340cfc60 con 0x7f4134143520 2023-10-08T00:17:33.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.318+0000 7f4132d9d700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f413406ab50 0x7f41340b3ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50998/0 (socket says 172.21.15.99:50998) 2023-10-08T00:17:33.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.318+0000 7f4132d9d700 1 -- 172.21.15.99:0/866089513 learned_addr learned my addr 172.21.15.99:0/866089513 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:17:33.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.318+0000 7f413259c700 1 --2- 172.21.15.99:0/866089513 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4134143520 0x7f41340cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:33.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.318+0000 7f4131d9b700 1 -- 172.21.15.99:0/866089513 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 521014397 0 0) 0x7f41340d0870 con 0x7f413406ab50 2023-10-08T00:17:33.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.318+0000 7f4131d9b700 1 -- 172.21.15.99:0/866089513 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4120003570 con 0x7f413406ab50 2023-10-08T00:17:33.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.319+0000 7f4131d9b700 1 -- 172.21.15.99:0/866089513 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3467390424 0 0) 0x7f4120003570 con 0x7f413406ab50 2023-10-08T00:17:33.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.319+0000 7f4131d9b700 1 -- 172.21.15.99:0/866089513 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4134143520 msgr2=0x7f41340cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:33.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.319+0000 7f4131d9b700 1 --2- 172.21.15.99:0/866089513 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4134143520 0x7f41340cf580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:33.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.319+0000 7f4131d9b700 1 -- 172.21.15.99:0/866089513 >> v1:172.21.15.191:6789/0 conn(0x7f41341438f0 legacy=0x7f413406a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:33.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.319+0000 7f4131d9b700 1 -- 172.21.15.99:0/866089513 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41340cff40 con 0x7f413406ab50 2023-10-08T00:17:33.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.319+0000 7f4131d9b700 1 -- 172.21.15.99:0/866089513 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f412c0017f0 con 0x7f413406ab50 2023-10-08T00:17:33.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.319+0000 7f4131d9b700 1 -- 172.21.15.99:0/866089513 >> v1:172.21.15.99:6789/0 conn(0x7f413406ab50 legacy=0x7f41340b3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:33.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.319+0000 7f4131d9b700 1 --2- 172.21.15.99:0/866089513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4120003a90 0x7f4120003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:33.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.319+0000 7f4131d9b700 1 --2- 172.21.15.99:0/866089513 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4134143520 0x7f4120004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:33.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.319+0000 7f4131d9b700 1 --2- 172.21.15.99:0/866089513 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4120004d50 0x7f41200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:33.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.319+0000 7f4131d9b700 1 -- 172.21.15.99:0/866089513 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41340cfc60 con 0x7f4120003a90 2023-10-08T00:17:33.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.319+0000 7f4131d9b700 1 -- 172.21.15.99:0/866089513 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412000bb10 con 0x7f4120004d50 2023-10-08T00:17:33.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.319+0000 7f4131d9b700 1 -- 172.21.15.99:0/866089513 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f412000bcd0 con 0x7f4134143520 2023-10-08T00:17:33.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.319+0000 7f413259c700 1 --2- 172.21.15.99:0/866089513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4120003a90 0x7f4120003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:33.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.319+0000 7f4129d9b700 1 --2- 172.21.15.99:0/866089513 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4134143520 0x7f4120004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:33.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.319+0000 7f4132d9d700 1 --2- 172.21.15.99:0/866089513 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4120004d50 0x7f41200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:33.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.319+0000 7f413259c700 1 -- 172.21.15.99:0/866089513 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4134143520 msgr2=0x7f4120004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:33.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.319+0000 7f413259c700 1 --2- 172.21.15.99:0/866089513 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4134143520 0x7f4120004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:33.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.319+0000 7f413259c700 1 -- 172.21.15.99:0/866089513 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4120004d50 msgr2=0x7f41200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:33.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.319+0000 7f413259c700 1 --2- 172.21.15.99:0/866089513 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4120004d50 0x7f41200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:33.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.319+0000 7f413259c700 1 -- 172.21.15.99:0/866089513 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4120003570 con 0x7f4120003a90 2023-10-08T00:17:33.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.319+0000 7f413259c700 1 --2- 172.21.15.99:0/866089513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4120003a90 0x7f4120003e60 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f4124009c00 tx=0x7f41240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c07160c8254fa7b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:33.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.319+0000 7f4131d9b700 1 -- 172.21.15.99:0/866089513 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412400e710 con 0x7f4120003a90 2023-10-08T00:17:33.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.319+0000 7f4131d9b700 1 -- 172.21.15.99:0/866089513 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f412400e870 con 0x7f4120003a90 2023-10-08T00:17:33.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.319+0000 7f4131d9b700 1 -- 172.21.15.99:0/866089513 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412400ec30 con 0x7f4120003a90 2023-10-08T00:17:33.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.320+0000 7f413359e700 1 -- 172.21.15.99:0/866089513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4120003a90 msgr2=0x7f4120003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:33.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.320+0000 7f413359e700 1 --2- 172.21.15.99:0/866089513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4120003a90 0x7f4120003e60 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f4124009c00 tx=0x7f41240091a0 comp rx=0 tx=0).stop 2023-10-08T00:17:33.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.320+0000 7f4132d9d700 1 -- 172.21.15.99:0/866089513 reap_dead start 2023-10-08T00:17:33.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.320+0000 7f413359e700 1 -- 172.21.15.99:0/866089513 shutdown_connections 2023-10-08T00:17:33.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.320+0000 7f413359e700 1 -- 172.21.15.99:0/866089513 >> 172.21.15.99:0/866089513 conn(0x7f4134069750 msgr2=0x7f41340b25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:33.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.320+0000 7f413359e700 1 -- 172.21.15.99:0/866089513 shutdown_connections 2023-10-08T00:17:33.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.320+0000 7f413359e700 1 -- 172.21.15.99:0/866089513 wait complete. 2023-10-08T00:17:33.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.321+0000 7f413359e700 1 Processor -- start 2023-10-08T00:17:33.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.321+0000 7f413359e700 1 -- start start 2023-10-08T00:17:33.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.322+0000 7f413359e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4120003a90 0x7f41341586d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:33.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.322+0000 7f413359e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f413415eff0 0x7f413415cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:33.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.322+0000 7f413359e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41340b1c90 0x7f413415ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:33.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.322+0000 7f413359e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41340b0f80 con 0x7f4120003a90 2023-10-08T00:17:33.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.322+0000 7f413359e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f413415a990 con 0x7f413415eff0 2023-10-08T00:17:33.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.322+0000 7f413359e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41340b1800 con 0x7f41340b1c90 2023-10-08T00:17:33.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.322+0000 7f413259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4120003a90 0x7f41341586d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:33.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.322+0000 7f413259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4120003a90 0x7f41341586d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45228/0 (socket says 172.21.15.99:45228) 2023-10-08T00:17:33.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.322+0000 7f413259c700 1 -- 172.21.15.99:0/3723564204 learned_addr learned my addr 172.21.15.99:0/3723564204 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:33.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.322+0000 7f4132d9d700 1 --2- 172.21.15.99:0/3723564204 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41340b1c90 0x7f413415ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:33.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.322+0000 7f4129d9b700 1 --2- 172.21.15.99:0/3723564204 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f413415eff0 0x7f413415cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:33.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.322+0000 7f413259c700 1 -- 172.21.15.99:0/3723564204 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41340b1c90 msgr2=0x7f413415ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:33.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.322+0000 7f413259c700 1 --2- 172.21.15.99:0/3723564204 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41340b1c90 0x7f413415ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:33.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.322+0000 7f413259c700 1 -- 172.21.15.99:0/3723564204 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f413415eff0 msgr2=0x7f413415cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:33.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.322+0000 7f413259c700 1 --2- 172.21.15.99:0/3723564204 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f413415eff0 0x7f413415cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:33.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.322+0000 7f413259c700 1 -- 172.21.15.99:0/3723564204 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f413415f460 con 0x7f4120003a90 2023-10-08T00:17:33.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.322+0000 7f4132d9d700 1 --2- 172.21.15.99:0/3723564204 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41340b1c90 0x7f413415ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:17:33.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.322+0000 7f4129d9b700 1 --2- 172.21.15.99:0/3723564204 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f413415eff0 0x7f413415cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:17:33.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.322+0000 7f413259c700 1 --2- 172.21.15.99:0/3723564204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4120003a90 0x7f41341586d0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f412400e4b0 tx=0x7f412401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:33.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.323+0000 7f412affd700 1 -- 172.21.15.99:0/3723564204 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4124022070 con 0x7f4120003a90 2023-10-08T00:17:33.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.323+0000 7f412affd700 1 -- 172.21.15.99:0/3723564204 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f412400e9e0 con 0x7f4120003a90 2023-10-08T00:17:33.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.323+0000 7f413359e700 1 -- 172.21.15.99:0/3723564204 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41340b11b0 con 0x7f4120003a90 2023-10-08T00:17:33.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.323+0000 7f412affd700 1 -- 172.21.15.99:0/3723564204 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4124027470 con 0x7f4120003a90 2023-10-08T00:17:33.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.323+0000 7f413359e700 1 -- 172.21.15.99:0/3723564204 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4134158c10 con 0x7f4120003a90 2023-10-08T00:17:33.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.323+0000 7f413359e700 1 -- 172.21.15.99:0/3723564204 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4134089030 con 0x7f4120003a90 2023-10-08T00:17:33.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.324+0000 7f412affd700 1 -- 172.21.15.99:0/3723564204 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4124027880 con 0x7f4120003a90 2023-10-08T00:17:33.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.324+0000 7f412affd700 1 --2- 172.21.15.99:0/3723564204 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4114066f30 0x7f41140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:33.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.324+0000 7f412affd700 1 -- 172.21.15.99:0/3723564204 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f4124085ca0 con 0x7f4120003a90 2023-10-08T00:17:33.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.325+0000 7f4129d9b700 1 --2- 172.21.15.99:0/3723564204 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4114066f30 0x7f41140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:33.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.325+0000 7f4129d9b700 1 --2- 172.21.15.99:0/3723564204 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4114066f30 0x7f41140693e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f4134140580 tx=0x7f411c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:33.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.328+0000 7f412affd700 1 -- 172.21.15.99:0/3723564204 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f41240532c0 con 0x7f4120003a90 2023-10-08T00:17:33.586 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:33.586+0000 7f413359e700 1 -- 172.21.15.99:0/3723564204 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]} v 0) v1 -- 0x7f41340d0220 con 0x7f4120003a90 2023-10-08T00:17:34.194 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.193+0000 7f412affd700 1 -- 172.21.15.99:0/3723564204 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]}]=0 v347) v1 ==== 135+0+0 (secure 0 0 0) 0x7f412405ae60 con 0x7f4120003a90 2023-10-08T00:17:34.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.196+0000 7f413359e700 1 -- 172.21.15.99:0/3723564204 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4114066f30 msgr2=0x7f41140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:34.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.196+0000 7f413359e700 1 --2- 172.21.15.99:0/3723564204 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4114066f30 0x7f41140693e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f4134140580 tx=0x7f411c009040 comp rx=0 tx=0).stop 2023-10-08T00:17:34.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.196+0000 7f413359e700 1 -- 172.21.15.99:0/3723564204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4120003a90 msgr2=0x7f41341586d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:34.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.196+0000 7f413359e700 1 --2- 172.21.15.99:0/3723564204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4120003a90 0x7f41341586d0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f412400e4b0 tx=0x7f412401a000 comp rx=0 tx=0).stop 2023-10-08T00:17:34.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.196+0000 7f413359e700 1 -- 172.21.15.99:0/3723564204 shutdown_connections 2023-10-08T00:17:34.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.196+0000 7f413359e700 1 --2- 172.21.15.99:0/3723564204 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4114066f30 0x7f41140693e0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:34.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.196+0000 7f413359e700 1 --2- 172.21.15.99:0/3723564204 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41340b1c90 0x7f413415ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:34.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.196+0000 7f413359e700 1 --2- 172.21.15.99:0/3723564204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4120003a90 0x7f41341586d0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:34.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.196+0000 7f413359e700 1 --2- 172.21.15.99:0/3723564204 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f413415eff0 0x7f413415cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:34.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.196+0000 7f413359e700 1 -- 172.21.15.99:0/3723564204 >> 172.21.15.99:0/3723564204 conn(0x7f4134069750 msgr2=0x7f41340b2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:34.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.196+0000 7f413359e700 1 -- 172.21.15.99:0/3723564204 shutdown_connections 2023-10-08T00:17:34.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.196+0000 7f413359e700 1 -- 172.21.15.99:0/3723564204 wait complete. 2023-10-08T00:17:34.210 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T00:17:34.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.329+0000 7fec3e103700 1 Processor -- start 2023-10-08T00:17:34.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.330+0000 7fec3e103700 1 -- start start 2023-10-08T00:17:34.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.330+0000 7fec3e103700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fec380cce30 0x7fec3815aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:34.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.330+0000 7fec3e103700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fec380b6580 0x7fec3806a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:34.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.330+0000 7fec3e103700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec3806ac90 con 0x7fec3806a830 2023-10-08T00:17:34.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.330+0000 7fec3e103700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec380b4d70 con 0x7fec380b6580 2023-10-08T00:17:34.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.330+0000 7fec3e103700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec380cd8d0 con 0x7fec380cce30 2023-10-08T00:17:34.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.331+0000 7fec3c900700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fec380b6580 0x7fec3806a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:34.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.331+0000 7fec3c900700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fec380b6580 0x7fec3806a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60580/0 (socket says 172.21.15.99:60580) 2023-10-08T00:17:34.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.331+0000 7fec3c900700 1 -- 172.21.15.99:0/4275475707 learned_addr learned my addr 172.21.15.99:0/4275475707 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:34.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.331+0000 7fec3c900700 1 -- 172.21.15.99:0/4275475707 >> v2:172.21.15.191:3300/0 conn(0x7fec380cce30 msgr2=0x7fec3815aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:34.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.331+0000 7fec3d101700 1 --2- 172.21.15.99:0/4275475707 >> v2:172.21.15.191:3300/0 conn(0x7fec380cce30 0x7fec3815aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:34.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.331+0000 7fec3c900700 1 --2- 172.21.15.99:0/4275475707 >> v2:172.21.15.191:3300/0 conn(0x7fec380cce30 0x7fec3815aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:34.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.331+0000 7fec3c900700 1 -- 172.21.15.99:0/4275475707 >> v1:172.21.15.191:6789/0 conn(0x7fec3806a830 legacy=0x7fec3815d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:34.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.331+0000 7fec3c900700 1 -- 172.21.15.99:0/4275475707 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec380b4440 con 0x7fec380b6580 2023-10-08T00:17:34.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.331+0000 7fec3c900700 1 --2- 172.21.15.99:0/4275475707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec380b6580 0x7fec3806a2f0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fec380b3770 tx=0x7fec28009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6d3f183addcd2d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:34.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.331+0000 7fec2ffff700 1 -- 172.21.15.99:0/4275475707 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2800b820 con 0x7fec380b6580 2023-10-08T00:17:34.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.331+0000 7fec2ffff700 1 -- 172.21.15.99:0/4275475707 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec2800b980 con 0x7fec380b6580 2023-10-08T00:17:34.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.331+0000 7fec2ffff700 1 -- 172.21.15.99:0/4275475707 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2800bd20 con 0x7fec380b6580 2023-10-08T00:17:34.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.332+0000 7fec3e103700 1 -- 172.21.15.99:0/4275475707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec380b6580 msgr2=0x7fec3806a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:34.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.332+0000 7fec3e103700 1 --2- 172.21.15.99:0/4275475707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec380b6580 0x7fec3806a2f0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fec380b3770 tx=0x7fec28009250 comp rx=0 tx=0).stop 2023-10-08T00:17:34.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.332+0000 7fec3e103700 1 -- 172.21.15.99:0/4275475707 shutdown_connections 2023-10-08T00:17:34.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.332+0000 7fec3e103700 1 --2- 172.21.15.99:0/4275475707 >> v2:172.21.15.191:3300/0 conn(0x7fec380cce30 0x7fec3815aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:34.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.332+0000 7fec3e103700 1 --2- 172.21.15.99:0/4275475707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec380b6580 0x7fec3806a2f0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:34.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.332+0000 7fec3e103700 1 -- 172.21.15.99:0/4275475707 >> 172.21.15.99:0/4275475707 conn(0x7fec38068cb0 msgr2=0x7fec380690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:34.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.332+0000 7fec3e103700 1 -- 172.21.15.99:0/4275475707 shutdown_connections 2023-10-08T00:17:34.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.332+0000 7fec3e103700 1 -- 172.21.15.99:0/4275475707 wait complete. 2023-10-08T00:17:34.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.333+0000 7fec3e103700 1 Processor -- start 2023-10-08T00:17:34.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.333+0000 7fec3e103700 1 -- start start 2023-10-08T00:17:34.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.334+0000 7fec3e103700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fec3806a830 0x7fec38148cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:34.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.334+0000 7fec3e103700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fec380b6580 0x7fec380b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:34.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.334+0000 7fec3e103700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec380cce30 0x7fec3816d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:34.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.334+0000 7fec3e103700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3815b3c0 con 0x7fec380cce30 2023-10-08T00:17:34.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.334+0000 7fec3e103700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3815a7b0 con 0x7fec380b6580 2023-10-08T00:17:34.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.334+0000 7fec3e103700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec3815aa90 con 0x7fec3806a830 2023-10-08T00:17:34.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.334+0000 7fec3d902700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec380cce30 0x7fec3816d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:34.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.334+0000 7fec3d902700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec380cce30 0x7fec3816d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60590/0 (socket says 172.21.15.99:60590) 2023-10-08T00:17:34.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.334+0000 7fec3d902700 1 -- 172.21.15.99:0/3893632536 learned_addr learned my addr 172.21.15.99:0/3893632536 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:34.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.334+0000 7fec3c900700 1 --2- 172.21.15.99:0/3893632536 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fec380b6580 0x7fec380b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:34.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.334+0000 7fec3d101700 1 --2- 172.21.15.99:0/3893632536 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fec3806a830 0x7fec38148cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:34.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.334+0000 7fec3d902700 1 -- 172.21.15.99:0/3893632536 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fec3806a830 msgr2=0x7fec38148cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:34.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.334+0000 7fec3d902700 1 --2- 172.21.15.99:0/3893632536 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fec3806a830 0x7fec38148cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:34.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.334+0000 7fec3d902700 1 -- 172.21.15.99:0/3893632536 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fec380b6580 msgr2=0x7fec380b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:34.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.334+0000 7fec3d902700 1 --2- 172.21.15.99:0/3893632536 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fec380b6580 0x7fec380b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:34.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.334+0000 7fec3d902700 1 -- 172.21.15.99:0/3893632536 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec280094f0 con 0x7fec380cce30 2023-10-08T00:17:34.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.334+0000 7fec3d902700 1 --2- 172.21.15.99:0/3893632536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec380cce30 0x7fec3816d650 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fec3000bbb0 tx=0x7fec30009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:34.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.335+0000 7fec2d7fa700 1 -- 172.21.15.99:0/3893632536 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec300092e0 con 0x7fec380cce30 2023-10-08T00:17:34.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.335+0000 7fec2d7fa700 1 -- 172.21.15.99:0/3893632536 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec3000c630 con 0x7fec380cce30 2023-10-08T00:17:34.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.335+0000 7fec3e103700 1 -- 172.21.15.99:0/3893632536 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec3815ad70 con 0x7fec380cce30 2023-10-08T00:17:34.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.335+0000 7fec2d7fa700 1 -- 172.21.15.99:0/3893632536 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3000c9f0 con 0x7fec380cce30 2023-10-08T00:17:34.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.335+0000 7fec3e103700 1 -- 172.21.15.99:0/3893632536 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec38149be0 con 0x7fec380cce30 2023-10-08T00:17:34.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.336+0000 7fec3e103700 1 -- 172.21.15.99:0/3893632536 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec3815b530 con 0x7fec380cce30 2023-10-08T00:17:34.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.336+0000 7fec2d7fa700 1 -- 172.21.15.99:0/3893632536 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fec30012070 con 0x7fec380cce30 2023-10-08T00:17:34.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.336+0000 7fec2d7fa700 1 --2- 172.21.15.99:0/3893632536 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fec1c066f30 0x7fec1c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:34.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.337+0000 7fec2d7fa700 1 -- 172.21.15.99:0/3893632536 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fec30083ed0 con 0x7fec380cce30 2023-10-08T00:17:34.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.337+0000 7fec3d101700 1 --2- 172.21.15.99:0/3893632536 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fec1c066f30 0x7fec1c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:34.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.337+0000 7fec3d101700 1 --2- 172.21.15.99:0/3893632536 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fec1c066f30 0x7fec1c0693e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fec34006fd0 tx=0x7fec34008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:34.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.341+0000 7fec2d7fa700 1 -- 172.21.15.99:0/3893632536 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fec300514f0 con 0x7fec380cce30 2023-10-08T00:17:34.522 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.521+0000 7fec3e103700 1 -- 172.21.15.99:0/3893632536 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fec380b4720 con 0x7fec380cce30 2023-10-08T00:17:34.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.525+0000 7fec2d7fa700 1 -- 172.21.15.99:0/3893632536 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7fec30054b10 con 0x7fec380cce30 2023-10-08T00:17:34.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.527+0000 7fec3e103700 1 -- 172.21.15.99:0/3893632536 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fec1c066f30 msgr2=0x7fec1c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:34.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.527+0000 7fec3e103700 1 --2- 172.21.15.99:0/3893632536 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fec1c066f30 0x7fec1c0693e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fec34006fd0 tx=0x7fec34008040 comp rx=0 tx=0).stop 2023-10-08T00:17:34.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.527+0000 7fec3e103700 1 -- 172.21.15.99:0/3893632536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec380cce30 msgr2=0x7fec3816d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:34.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.527+0000 7fec3e103700 1 --2- 172.21.15.99:0/3893632536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec380cce30 0x7fec3816d650 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fec3000bbb0 tx=0x7fec30009840 comp rx=0 tx=0).stop 2023-10-08T00:17:34.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.527+0000 7fec3e103700 1 -- 172.21.15.99:0/3893632536 shutdown_connections 2023-10-08T00:17:34.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.527+0000 7fec3e103700 1 --2- 172.21.15.99:0/3893632536 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fec1c066f30 0x7fec1c0693e0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:34.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.527+0000 7fec3e103700 1 --2- 172.21.15.99:0/3893632536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec380cce30 0x7fec3816d650 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:34.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.527+0000 7fec3e103700 1 --2- 172.21.15.99:0/3893632536 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fec3806a830 0x7fec38148cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:34.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.527+0000 7fec3e103700 1 --2- 172.21.15.99:0/3893632536 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fec380b6580 0x7fec380b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:34.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.527+0000 7fec3e103700 1 -- 172.21.15.99:0/3893632536 >> 172.21.15.99:0/3893632536 conn(0x7fec38068cb0 msgr2=0x7fec3815d810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:34.530 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.527+0000 7fec3e103700 1 -- 172.21.15.99:0/3893632536 shutdown_connections 2023-10-08T00:17:34.530 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.527+0000 7fec3e103700 1 -- 172.21.15.99:0/3893632536 wait complete. 2023-10-08T00:17:34.540 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T00:17:34.541 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T00:17:34.578 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T00:17:34.621 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T00:17:34.621 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-08T00:17:34.621 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:17:34.735 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.734+0000 7ffbcc369700 1 Processor -- start 2023-10-08T00:17:34.735 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.734+0000 7ffbcc369700 1 -- start start 2023-10-08T00:17:34.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.735+0000 7ffbcc369700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ffbc40d1c30 0x7ffbc40ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:34.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.735+0000 7ffbcc369700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ffbc415a680 0x7ffbc40cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:34.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.735+0000 7ffbcc369700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbc40c5d80 con 0x7ffbc415a2b0 2023-10-08T00:17:34.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.735+0000 7ffbcc369700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbc4068470 con 0x7ffbc40d1c30 2023-10-08T00:17:34.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.735+0000 7ffbcc369700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbc40c6060 con 0x7ffbc415a680 2023-10-08T00:17:34.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.736+0000 7ffbc9904700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ffbc40d1c30 0x7ffbc40ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:34.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.736+0000 7ffbca105700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7ffbc415a2b0 0x7ffbc40b3da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:32772/0 (socket says 172.21.15.99:32772) 2023-10-08T00:17:34.739 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.736+0000 7ffbca105700 1 -- 172.21.15.99:0/537714959 learned_addr learned my addr 172.21.15.99:0/537714959 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:17:34.739 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.736+0000 7ffbc9103700 1 -- 172.21.15.99:0/537714959 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3608900618 0 0) 0x7ffbc40c5d80 con 0x7ffbc415a2b0 2023-10-08T00:17:34.739 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.736+0000 7ffbc9103700 1 -- 172.21.15.99:0/537714959 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffbb0003570 con 0x7ffbc415a2b0 2023-10-08T00:17:34.739 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.736+0000 7ffbca906700 1 --2- 172.21.15.99:0/537714959 >> v2:172.21.15.191:3300/0 conn(0x7ffbc415a680 0x7ffbc40cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:34.739 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.736+0000 7ffbc9904700 1 -- 172.21.15.99:0/537714959 >> v2:172.21.15.191:3300/0 conn(0x7ffbc415a680 msgr2=0x7ffbc40cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:34.739 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.736+0000 7ffbc9904700 1 --2- 172.21.15.99:0/537714959 >> v2:172.21.15.191:3300/0 conn(0x7ffbc415a680 0x7ffbc40cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:34.739 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.736+0000 7ffbc9904700 1 -- 172.21.15.99:0/537714959 >> v1:172.21.15.99:6789/0 conn(0x7ffbc415a2b0 legacy=0x7ffbc40b3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:34.739 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.736+0000 7ffbc9904700 1 -- 172.21.15.99:0/537714959 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc40c6340 con 0x7ffbc40d1c30 2023-10-08T00:17:34.739 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.736+0000 7ffbca906700 1 --2- 172.21.15.99:0/537714959 >> v2:172.21.15.191:3300/0 conn(0x7ffbc415a680 0x7ffbc40cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:17:34.739 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.736+0000 7ffbc9904700 1 --2- 172.21.15.99:0/537714959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbc40d1c30 0x7ffbc40ccdd0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4064970 tx=0x7ffbb40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ddc7bb5d50b3b13 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:34.740 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.736+0000 7ffbc9103700 1 -- 172.21.15.99:0/537714959 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4011630 con 0x7ffbc40d1c30 2023-10-08T00:17:34.740 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.736+0000 7ffbc9103700 1 -- 172.21.15.99:0/537714959 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbb4011790 con 0x7ffbc40d1c30 2023-10-08T00:17:34.740 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.736+0000 7ffbc9103700 1 -- 172.21.15.99:0/537714959 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4011b30 con 0x7ffbc40d1c30 2023-10-08T00:17:34.740 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.737+0000 7ffbcc369700 1 -- 172.21.15.99:0/537714959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbc40d1c30 msgr2=0x7ffbc40ccdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:34.740 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.737+0000 7ffbcc369700 1 --2- 172.21.15.99:0/537714959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbc40d1c30 0x7ffbc40ccdd0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4064970 tx=0x7ffbb40091a0 comp rx=0 tx=0).stop 2023-10-08T00:17:34.740 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.737+0000 7ffbcc369700 1 -- 172.21.15.99:0/537714959 shutdown_connections 2023-10-08T00:17:34.740 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.737+0000 7ffbcc369700 1 --2- 172.21.15.99:0/537714959 >> v2:172.21.15.191:3300/0 conn(0x7ffbc415a680 0x7ffbc40cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:34.740 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.737+0000 7ffbcc369700 1 --2- 172.21.15.99:0/537714959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbc40d1c30 0x7ffbc40ccdd0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:34.741 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.737+0000 7ffbcc369700 1 -- 172.21.15.99:0/537714959 >> 172.21.15.99:0/537714959 conn(0x7ffbc4067e40 msgr2=0x7ffbc40d0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:34.741 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.737+0000 7ffbcc369700 1 -- 172.21.15.99:0/537714959 shutdown_connections 2023-10-08T00:17:34.741 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.737+0000 7ffbcc369700 1 -- 172.21.15.99:0/537714959 wait complete. 2023-10-08T00:17:34.744 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.738+0000 7ffbcc369700 1 Processor -- start 2023-10-08T00:17:34.744 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.738+0000 7ffbcc369700 1 -- start start 2023-10-08T00:17:34.744 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.739+0000 7ffbcc369700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbc40d1c30 0x7ffbc40c9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:34.744 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.739+0000 7ffbcc369700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffbc415a2b0 0x7ffbc40c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:34.744 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.739+0000 7ffbcc369700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffbc415a680 0x7ffbc40b5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:34.744 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.739+0000 7ffbcc369700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc416a920 con 0x7ffbc40d1c30 2023-10-08T00:17:34.744 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.739+0000 7ffbcc369700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc40c85d0 con 0x7ffbc415a2b0 2023-10-08T00:17:34.744 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.739+0000 7ffbcc369700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc40c8870 con 0x7ffbc415a680 2023-10-08T00:17:34.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.739+0000 7ffbca105700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbc40d1c30 0x7ffbc40c9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:34.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.739+0000 7ffbca105700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbc40d1c30 0x7ffbc40c9f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60608/0 (socket says 172.21.15.99:60608) 2023-10-08T00:17:34.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.739+0000 7ffbca105700 1 -- 172.21.15.99:0/1685460950 learned_addr learned my addr 172.21.15.99:0/1685460950 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:34.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.739+0000 7ffbc9904700 1 --2- 172.21.15.99:0/1685460950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffbc415a2b0 0x7ffbc40c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:34.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.739+0000 7ffbca906700 1 --2- 172.21.15.99:0/1685460950 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffbc415a680 0x7ffbc40b5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:34.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.739+0000 7ffbca105700 1 -- 172.21.15.99:0/1685460950 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffbc415a680 msgr2=0x7ffbc40b5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:34.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.739+0000 7ffbca105700 1 --2- 172.21.15.99:0/1685460950 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffbc415a680 0x7ffbc40b5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:34.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.739+0000 7ffbca105700 1 -- 172.21.15.99:0/1685460950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffbc415a2b0 msgr2=0x7ffbc40c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:34.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.739+0000 7ffbca105700 1 --2- 172.21.15.99:0/1685460950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffbc415a2b0 0x7ffbc40c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:34.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.739+0000 7ffbca105700 1 -- 172.21.15.99:0/1685460950 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb4009420 con 0x7ffbc40d1c30 2023-10-08T00:17:34.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.739+0000 7ffbca105700 1 --2- 172.21.15.99:0/1685460950 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbc40d1c30 0x7ffbc40c9f30 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7ffbc000c790 tx=0x7ffbc0007c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:34.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.739+0000 7ffbba7fc700 1 -- 172.21.15.99:0/1685460950 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc00072e0 con 0x7ffbc40d1c30 2023-10-08T00:17:34.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.739+0000 7ffbba7fc700 1 -- 172.21.15.99:0/1685460950 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbc0007440 con 0x7ffbc40d1c30 2023-10-08T00:17:34.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.739+0000 7ffbcc369700 1 -- 172.21.15.99:0/1685460950 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbc40c8b50 con 0x7ffbc40d1c30 2023-10-08T00:17:34.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.739+0000 7ffbba7fc700 1 -- 172.21.15.99:0/1685460950 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc00057f0 con 0x7ffbc40d1c30 2023-10-08T00:17:34.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.740+0000 7ffbcc369700 1 -- 172.21.15.99:0/1685460950 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbc416a510 con 0x7ffbc40d1c30 2023-10-08T00:17:34.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.740+0000 7ffbcc369700 1 -- 172.21.15.99:0/1685460950 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbc404c3c0 con 0x7ffbc40d1c30 2023-10-08T00:17:34.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.741+0000 7ffbba7fc700 1 -- 172.21.15.99:0/1685460950 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffbc0005c20 con 0x7ffbc40d1c30 2023-10-08T00:17:34.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.741+0000 7ffbba7fc700 1 --2- 172.21.15.99:0/1685460950 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ffba8066f30 0x7ffba80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:34.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.741+0000 7ffbba7fc700 1 -- 172.21.15.99:0/1685460950 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 4939+0+0 (secure 0 0 0) 0x7ffbc0083610 con 0x7ffbc40d1c30 2023-10-08T00:17:34.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.741+0000 7ffbc9904700 1 --2- 172.21.15.99:0/1685460950 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ffba8066f30 0x7ffba80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:34.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.742+0000 7ffbc9904700 1 --2- 172.21.15.99:0/1685460950 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ffba8066f30 0x7ffba80693e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ffbc40d2000 tx=0x7ffbb4017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:34.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.745+0000 7ffbba7fc700 1 -- 172.21.15.99:0/1685460950 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffbc004b5a0 con 0x7ffbc40d1c30 2023-10-08T00:17:34.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.968+0000 7ffbcc369700 1 -- 172.21.15.99:0/1685460950 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffbc40cafa0 con 0x7ffbc40d1c30 2023-10-08T00:17:34.970 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.969+0000 7ffbba7fc700 1 -- 172.21.15.99:0/1685460950 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+3529 (secure 0 0 0) 0x7ffbc0017090 con 0x7ffbc40d1c30 2023-10-08T00:17:34.970 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:17:34.970 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":65,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5813,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/887374839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":887374839},{"type":"v1","addr":"172.21.15.191:6837","nonce":887374839}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":6445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/2409696901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2409696901},{"type":"v1","addr":"172.21.15.191:6836","nonce":2409696901}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":7095,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/1924096228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1924096228},{"type":"v1","addr":"172.21.15.99:6834","nonce":1924096228}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":7419,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/220516298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":220516298},{"type":"v1","addr":"172.21.15.99:6835","nonce":220516298}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65}],"filesystems":[]} 2023-10-08T00:17:34.972 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.971+0000 7ffbcc369700 1 -- 172.21.15.99:0/1685460950 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ffba8066f30 msgr2=0x7ffba80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:34.972 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.971+0000 7ffbcc369700 1 --2- 172.21.15.99:0/1685460950 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ffba8066f30 0x7ffba80693e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ffbc40d2000 tx=0x7ffbb4017000 comp rx=0 tx=0).stop 2023-10-08T00:17:34.973 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.971+0000 7ffbcc369700 1 -- 172.21.15.99:0/1685460950 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbc40d1c30 msgr2=0x7ffbc40c9f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:34.973 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.971+0000 7ffbcc369700 1 --2- 172.21.15.99:0/1685460950 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbc40d1c30 0x7ffbc40c9f30 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7ffbc000c790 tx=0x7ffbc0007c80 comp rx=0 tx=0).stop 2023-10-08T00:17:34.973 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.971+0000 7ffbcc369700 1 -- 172.21.15.99:0/1685460950 shutdown_connections 2023-10-08T00:17:34.973 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.971+0000 7ffbcc369700 1 --2- 172.21.15.99:0/1685460950 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ffba8066f30 0x7ffba80693e0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:34.973 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.971+0000 7ffbcc369700 1 --2- 172.21.15.99:0/1685460950 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffbc415a680 0x7ffbc40b5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:34.973 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.971+0000 7ffbcc369700 1 --2- 172.21.15.99:0/1685460950 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbc40d1c30 0x7ffbc40c9f30 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:34.973 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.971+0000 7ffbcc369700 1 --2- 172.21.15.99:0/1685460950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffbc415a2b0 0x7ffbc40c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:34.973 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.971+0000 7ffbcc369700 1 -- 172.21.15.99:0/1685460950 >> 172.21.15.99:0/1685460950 conn(0x7ffbc4067e40 msgr2=0x7ffbc40b3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:34.973 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.971+0000 7ffbcc369700 1 -- 172.21.15.99:0/1685460950 shutdown_connections 2023-10-08T00:17:34.973 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:34.971+0000 7ffbcc369700 1 -- 172.21.15.99:0/1685460950 wait complete. 2023-10-08T00:17:34.973 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 65 2023-10-08T00:17:34.984 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T00:17:34.984 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-08T00:17:34.984 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T00:17:34.985 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:17:34.985 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T00:17:34.985 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-08T00:17:34.985 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T00:17:34.985 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:17:34.985 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T00:17:34.985 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-08T00:17:34.986 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T00:17:34.986 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:17:34.986 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T00:17:34.986 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-08T00:17:34.986 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T00:17:34.986 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:17:34.993 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T00:17:34.993 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T00:17:34.994 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T00:17:34.994 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T00:17:34.996 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T00:17:34.997 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T00:17:34.997 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T00:17:34.998 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T00:17:35.036 INFO:tasks.ceph.mds.a:Started 2023-10-08T00:17:35.036 DEBUG:teuthology.parallel:result is None 2023-10-08T00:17:35.037 INFO:tasks.ceph.mds.c:Started 2023-10-08T00:17:35.037 DEBUG:teuthology.parallel:result is None 2023-10-08T00:17:35.039 INFO:tasks.ceph.mds.d:Started 2023-10-08T00:17:35.039 DEBUG:teuthology.parallel:result is None 2023-10-08T00:17:35.040 INFO:tasks.ceph.mds.b:Started 2023-10-08T00:17:35.040 DEBUG:teuthology.parallel:result is None 2023-10-08T00:17:35.040 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-08T00:17:35.109 INFO:tasks.ceph.mds.a.smithi099.stdout:starting mds.a at 2023-10-08T00:17:35.109 INFO:tasks.ceph.mds.c.smithi099.stdout:starting mds.c at 2023-10-08T00:17:35.120 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2023-10-08T00:17:35.121 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2023-10-08T00:17:35.121 INFO:tasks.ceph.mds.d.smithi191.stderr:2023-10-08T00:17:35.121+0000 7f4ff3ad8700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-10-08T00:17:35.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.160+0000 7f1dfb1a4700 1 Processor -- start 2023-10-08T00:17:35.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.160+0000 7f1dfb1a4700 1 -- start start 2023-10-08T00:17:35.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.161+0000 7f1dfb1a4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1df40afff0 0x7f1df415e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:35.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.161+0000 7f1dfb1a4700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1df4069e20 0x7f1df406a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:35.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.161+0000 7f1dfb1a4700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1df40b03c0 con 0x7f1df4070700 2023-10-08T00:17:35.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.161+0000 7f1dfb1a4700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1df406a870 con 0x7f1df4069e20 2023-10-08T00:17:35.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.161+0000 7f1dfb1a4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df406ab50 con 0x7f1df40afff0 2023-10-08T00:17:35.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.162+0000 7f1df8f40700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1df40afff0 0x7f1df415e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:35.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.162+0000 7f1df3fff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1df4069e20 0x7f1df406a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:35.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.162+0000 7f1df9741700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f1df4070700 0x7f1df406ea90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:34398/0 (socket says 172.21.15.99:34398) 2023-10-08T00:17:35.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.162+0000 7f1df9741700 1 -- 172.21.15.99:0/2934351703 learned_addr learned my addr 172.21.15.99:0/2934351703 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:17:35.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.162+0000 7f1df3fff700 1 -- 172.21.15.99:0/2934351703 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1df40afff0 msgr2=0x7f1df415e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:35.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.162+0000 7f1df3fff700 1 --2- 172.21.15.99:0/2934351703 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1df40afff0 0x7f1df415e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:35.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.162+0000 7f1df3fff700 1 -- 172.21.15.99:0/2934351703 >> v1:172.21.15.191:6789/0 conn(0x7f1df4070700 legacy=0x7f1df406ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:35.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.162+0000 7f1df3fff700 1 -- 172.21.15.99:0/2934351703 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df415da60 con 0x7f1df4069e20 2023-10-08T00:17:35.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.162+0000 7f1df37fe700 1 -- 172.21.15.99:0/2934351703 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1832473123 0 0) 0x7f1df40b03c0 con 0x7f1df4070700 2023-10-08T00:17:35.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.162+0000 7f1df8f40700 1 --2- 172.21.15.99:0/2934351703 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1df40afff0 0x7f1df415e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:17:35.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.162+0000 7f1df3fff700 1 --2- 172.21.15.99:0/2934351703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1df4069e20 0x7f1df406a290 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1de00098e0 tx=0x7f1de00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13176b218097716b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:35.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.162+0000 7f1df37fe700 1 -- 172.21.15.99:0/2934351703 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de000b640 con 0x7f1df4069e20 2023-10-08T00:17:35.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.162+0000 7f1df37fe700 1 -- 172.21.15.99:0/2934351703 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de000b7a0 con 0x7f1df4069e20 2023-10-08T00:17:35.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.162+0000 7f1df37fe700 1 -- 172.21.15.99:0/2934351703 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de000bb40 con 0x7f1df4069e20 2023-10-08T00:17:35.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.163+0000 7f1dfb1a4700 1 -- 172.21.15.99:0/2934351703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1df4069e20 msgr2=0x7f1df406a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:35.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.163+0000 7f1dfb1a4700 1 --2- 172.21.15.99:0/2934351703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1df4069e20 0x7f1df406a290 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1de00098e0 tx=0x7f1de00091a0 comp rx=0 tx=0).stop 2023-10-08T00:17:35.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.163+0000 7f1dfb1a4700 1 -- 172.21.15.99:0/2934351703 shutdown_connections 2023-10-08T00:17:35.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.163+0000 7f1dfb1a4700 1 --2- 172.21.15.99:0/2934351703 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1df40afff0 0x7f1df415e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:35.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.163+0000 7f1dfb1a4700 1 --2- 172.21.15.99:0/2934351703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1df4069e20 0x7f1df406a290 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:35.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.163+0000 7f1dfb1a4700 1 -- 172.21.15.99:0/2934351703 >> 172.21.15.99:0/2934351703 conn(0x7f1df40cda50 msgr2=0x7f1df40cde50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:35.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.163+0000 7f1dfb1a4700 1 -- 172.21.15.99:0/2934351703 shutdown_connections 2023-10-08T00:17:35.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.163+0000 7f1dfb1a4700 1 -- 172.21.15.99:0/2934351703 wait complete. 2023-10-08T00:17:35.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.164+0000 7f1dfb1a4700 1 Processor -- start 2023-10-08T00:17:35.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.164+0000 7f1dfb1a4700 1 -- start start 2023-10-08T00:17:35.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.165+0000 7f1dfb1a4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1df4069e20 0x7f1df41584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:35.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.165+0000 7f1dfb1a4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1df4070700 0x7f1df40c6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:35.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.165+0000 7f1dfb1a4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1df40afff0 0x7f1df40c65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:35.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.165+0000 7f1dfb1a4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df41589e0 con 0x7f1df4070700 2023-10-08T00:17:35.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.165+0000 7f1dfb1a4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df40b3520 con 0x7f1df4069e20 2023-10-08T00:17:35.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.165+0000 7f1dfb1a4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df40b2910 con 0x7f1df40afff0 2023-10-08T00:17:35.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.165+0000 7f1df3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1df4070700 0x7f1df40c6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:35.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.165+0000 7f1df3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1df4070700 0x7f1df40c6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60624/0 (socket says 172.21.15.99:60624) 2023-10-08T00:17:35.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.165+0000 7f1df3fff700 1 -- 172.21.15.99:0/3676585388 learned_addr learned my addr 172.21.15.99:0/3676585388 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:35.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.165+0000 7f1df9741700 1 --2- 172.21.15.99:0/3676585388 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1df40afff0 0x7f1df40c65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:35.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.165+0000 7f1df8f40700 1 --2- 172.21.15.99:0/3676585388 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1df4069e20 0x7f1df41584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:35.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.165+0000 7f1df3fff700 1 -- 172.21.15.99:0/3676585388 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1df40afff0 msgr2=0x7f1df40c65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:35.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.165+0000 7f1df3fff700 1 --2- 172.21.15.99:0/3676585388 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1df40afff0 0x7f1df40c65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:35.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.165+0000 7f1df3fff700 1 -- 172.21.15.99:0/3676585388 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1df4069e20 msgr2=0x7f1df41584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:35.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.165+0000 7f1df3fff700 1 --2- 172.21.15.99:0/3676585388 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1df4069e20 0x7f1df41584a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:35.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.165+0000 7f1df3fff700 1 -- 172.21.15.99:0/3676585388 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1de8009420 con 0x7f1df4070700 2023-10-08T00:17:35.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.165+0000 7f1df9741700 1 --2- 172.21.15.99:0/3676585388 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1df40afff0 0x7f1df40c65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:17:35.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.166+0000 7f1df3fff700 1 --2- 172.21.15.99:0/3676585388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1df4070700 0x7f1df40c6080 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f1de00093f0 tx=0x7f1de0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:35.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.166+0000 7f1df0ff9700 1 -- 172.21.15.99:0/3676585388 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de000bb20 con 0x7f1df4070700 2023-10-08T00:17:35.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.166+0000 7f1df0ff9700 1 -- 172.21.15.99:0/3676585388 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de0011420 con 0x7f1df4070700 2023-10-08T00:17:35.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.166+0000 7f1dfb1a4700 1 -- 172.21.15.99:0/3676585388 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1de0009420 con 0x7f1df4070700 2023-10-08T00:17:35.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.166+0000 7f1df0ff9700 1 -- 172.21.15.99:0/3676585388 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de0011800 con 0x7f1df4070700 2023-10-08T00:17:35.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.166+0000 7f1dfb1a4700 1 -- 172.21.15.99:0/3676585388 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1df40b2ed0 con 0x7f1df4070700 2023-10-08T00:17:35.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.167+0000 7f1dfb1a4700 1 -- 172.21.15.99:0/3676585388 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1df404c3c0 con 0x7f1df4070700 2023-10-08T00:17:35.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.167+0000 7f1df0ff9700 1 -- 172.21.15.99:0/3676585388 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1de0011c30 con 0x7f1df4070700 2023-10-08T00:17:35.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.167+0000 7f1df0ff9700 1 --2- 172.21.15.99:0/3676585388 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1dd8066f30 0x7f1dd80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:35.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.167+0000 7f1df0ff9700 1 -- 172.21.15.99:0/3676585388 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f1de0085740 con 0x7f1df4070700 2023-10-08T00:17:35.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.168+0000 7f1df8f40700 1 --2- 172.21.15.99:0/3676585388 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1dd8066f30 0x7f1dd80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:35.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.168+0000 7f1df8f40700 1 --2- 172.21.15.99:0/3676585388 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1dd8066f30 0x7f1dd80693e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f1de80093f0 tx=0x7f1de8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:35.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.171+0000 7f1df0ff9700 1 -- 172.21.15.99:0/3676585388 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1de0052d30 con 0x7f1df4070700 2023-10-08T00:17:35.372 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:35.371+0000 7f1dfb1a4700 1 -- 172.21.15.99:0/3676585388 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f1df40b3660 con 0x7f1df4070700 2023-10-08T00:17:36.221 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.221+0000 7f1df0ff9700 1 -- 172.21.15.99:0/3676585388 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v150) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1de0056350 con 0x7f1df4070700 2023-10-08T00:17:36.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.224+0000 7f1dfb1a4700 1 -- 172.21.15.99:0/3676585388 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1dd8066f30 msgr2=0x7f1dd80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:36.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.224+0000 7f1dfb1a4700 1 --2- 172.21.15.99:0/3676585388 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1dd8066f30 0x7f1dd80693e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f1de80093f0 tx=0x7f1de8008040 comp rx=0 tx=0).stop 2023-10-08T00:17:36.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.224+0000 7f1dfb1a4700 1 -- 172.21.15.99:0/3676585388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1df4070700 msgr2=0x7f1df40c6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:36.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.224+0000 7f1dfb1a4700 1 --2- 172.21.15.99:0/3676585388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1df4070700 0x7f1df40c6080 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f1de00093f0 tx=0x7f1de0017000 comp rx=0 tx=0).stop 2023-10-08T00:17:36.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.224+0000 7f1dfb1a4700 1 -- 172.21.15.99:0/3676585388 shutdown_connections 2023-10-08T00:17:36.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.224+0000 7f1dfb1a4700 1 --2- 172.21.15.99:0/3676585388 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1dd8066f30 0x7f1dd80693e0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:36.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.224+0000 7f1dfb1a4700 1 --2- 172.21.15.99:0/3676585388 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1df4069e20 0x7f1df41584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:36.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.224+0000 7f1dfb1a4700 1 --2- 172.21.15.99:0/3676585388 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1df40afff0 0x7f1df40c65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:36.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.224+0000 7f1dfb1a4700 1 --2- 172.21.15.99:0/3676585388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1df4070700 0x7f1df40c6080 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:36.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.224+0000 7f1dfb1a4700 1 -- 172.21.15.99:0/3676585388 >> 172.21.15.99:0/3676585388 conn(0x7f1df40cda50 msgr2=0x7f1df4070ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:36.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.224+0000 7f1dfb1a4700 1 -- 172.21.15.99:0/3676585388 shutdown_connections 2023-10-08T00:17:36.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.224+0000 7f1dfb1a4700 1 -- 172.21.15.99:0/3676585388 wait complete. 2023-10-08T00:17:36.226 INFO:teuthology.orchestra.run.smithi099.stderr: removed all blocklist entries 2023-10-08T00:17:36.238 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-08T00:17:36.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.353+0000 7ff2ad5ef700 1 Processor -- start 2023-10-08T00:17:36.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.353+0000 7ff2ad5ef700 1 -- start start 2023-10-08T00:17:36.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.354+0000 7ff2ad5ef700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff2a8157ba0 0x7ff2a806a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:36.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.354+0000 7ff2ad5ef700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2a815c360 con 0x7ff2a81577d0 2023-10-08T00:17:36.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.354+0000 7ff2ad5ef700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2a815b790 con 0x7ff2a815b0d0 2023-10-08T00:17:36.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.354+0000 7ff2ad5ef700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2a815ba30 con 0x7ff2a8157ba0 2023-10-08T00:17:36.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.354+0000 7ff2a67fc700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff2a8157ba0 0x7ff2a806a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:36.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.354+0000 7ff2a6ffd700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7ff2a81577d0 0x7ff2a80cd770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:32776/0 (socket says 172.21.15.99:32776) 2023-10-08T00:17:36.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.354+0000 7ff2a6ffd700 1 -- 172.21.15.99:0/583843275 learned_addr learned my addr 172.21.15.99:0/583843275 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:17:36.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.355+0000 7ff2a67fc700 1 -- 172.21.15.99:0/583843275 >> v1:172.21.15.191:6789/0 conn(0x7ff2a815b0d0 legacy=0x7ff2a815c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:36.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.355+0000 7ff2a67fc700 1 -- 172.21.15.99:0/583843275 >> v1:172.21.15.99:6789/0 conn(0x7ff2a81577d0 legacy=0x7ff2a80cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:36.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.355+0000 7ff2a67fc700 1 -- 172.21.15.99:0/583843275 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a815bd10 con 0x7ff2a8157ba0 2023-10-08T00:17:36.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.355+0000 7ff2a5ffb700 1 -- 172.21.15.99:0/583843275 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1031146813 0 0) 0x7ff2a815c360 con 0x7ff2a81577d0 2023-10-08T00:17:36.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.355+0000 7ff2a67fc700 1 --2- 172.21.15.99:0/583843275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2a8157ba0 0x7ff2a806a520 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7ff29c0098e0 tx=0x7ff29c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d359119cabc59604 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:36.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.355+0000 7ff2a5ffb700 1 -- 172.21.15.99:0/583843275 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c009e80 con 0x7ff2a8157ba0 2023-10-08T00:17:36.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.355+0000 7ff2a5ffb700 1 -- 172.21.15.99:0/583843275 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff29c00b820 con 0x7ff2a8157ba0 2023-10-08T00:17:36.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.355+0000 7ff2a5ffb700 1 -- 172.21.15.99:0/583843275 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c00bba0 con 0x7ff2a8157ba0 2023-10-08T00:17:36.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.355+0000 7ff2ad5ef700 1 -- 172.21.15.99:0/583843275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2a8157ba0 msgr2=0x7ff2a806a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:36.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.355+0000 7ff2ad5ef700 1 --2- 172.21.15.99:0/583843275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2a8157ba0 0x7ff2a806a520 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7ff29c0098e0 tx=0x7ff29c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:17:36.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.355+0000 7ff2ad5ef700 1 -- 172.21.15.99:0/583843275 shutdown_connections 2023-10-08T00:17:36.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.355+0000 7ff2ad5ef700 1 --2- 172.21.15.99:0/583843275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2a8157ba0 0x7ff2a806a520 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:36.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.355+0000 7ff2ad5ef700 1 -- 172.21.15.99:0/583843275 >> 172.21.15.99:0/583843275 conn(0x7ff2a8068ce0 msgr2=0x7ff2a80690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:36.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.355+0000 7ff2ad5ef700 1 -- 172.21.15.99:0/583843275 shutdown_connections 2023-10-08T00:17:36.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.356+0000 7ff2ad5ef700 1 -- 172.21.15.99:0/583843275 wait complete. 2023-10-08T00:17:36.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.357+0000 7ff2ad5ef700 1 Processor -- start 2023-10-08T00:17:36.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.357+0000 7ff2ad5ef700 1 -- start start 2023-10-08T00:17:36.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.357+0000 7ff2ad5ef700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2a81577d0 0x7ff2a814cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:36.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.357+0000 7ff2ad5ef700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff2a8157ba0 0x7ff2a8152270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:36.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.357+0000 7ff2ad5ef700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff2a815b0d0 0x7ff2a81527b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:36.362 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.357+0000 7ff2ad5ef700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a816b000 con 0x7ff2a81577d0 2023-10-08T00:17:36.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.357+0000 7ff2ad5ef700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a814fa20 con 0x7ff2a815b0d0 2023-10-08T00:17:36.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.357+0000 7ff2ad5ef700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a816a410 con 0x7ff2a8157ba0 2023-10-08T00:17:36.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.358+0000 7ff2a6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2a81577d0 0x7ff2a814cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:36.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.358+0000 7ff2a6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2a81577d0 0x7ff2a814cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60634/0 (socket says 172.21.15.99:60634) 2023-10-08T00:17:36.363 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.358+0000 7ff2a6ffd700 1 -- 172.21.15.99:0/2876133183 learned_addr learned my addr 172.21.15.99:0/2876133183 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:36.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.358+0000 7ff2a67fc700 1 --2- 172.21.15.99:0/2876133183 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff2a8157ba0 0x7ff2a8152270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:36.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.358+0000 7ff2a77fe700 1 --2- 172.21.15.99:0/2876133183 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff2a815b0d0 0x7ff2a81527b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:36.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.358+0000 7ff2a6ffd700 1 -- 172.21.15.99:0/2876133183 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff2a8157ba0 msgr2=0x7ff2a8152270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:36.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.358+0000 7ff2a6ffd700 1 --2- 172.21.15.99:0/2876133183 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff2a8157ba0 0x7ff2a8152270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:36.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.358+0000 7ff2a6ffd700 1 -- 172.21.15.99:0/2876133183 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff2a815b0d0 msgr2=0x7ff2a81527b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:36.364 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.358+0000 7ff2a6ffd700 1 --2- 172.21.15.99:0/2876133183 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff2a815b0d0 0x7ff2a81527b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:36.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.358+0000 7ff2a6ffd700 1 -- 172.21.15.99:0/2876133183 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff29c009420 con 0x7ff2a81577d0 2023-10-08T00:17:36.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.358+0000 7ff2a77fe700 1 --2- 172.21.15.99:0/2876133183 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff2a815b0d0 0x7ff2a81527b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:17:36.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.358+0000 7ff2a6ffd700 1 --2- 172.21.15.99:0/2876133183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2a81577d0 0x7ff2a814cdb0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7ff29800cdb0 tx=0x7ff298001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:36.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.358+0000 7ff28effd700 1 -- 172.21.15.99:0/2876133183 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29800aad0 con 0x7ff2a81577d0 2023-10-08T00:17:36.365 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.358+0000 7ff28effd700 1 -- 172.21.15.99:0/2876133183 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff29800ac30 con 0x7ff2a81577d0 2023-10-08T00:17:36.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.358+0000 7ff2ad5ef700 1 -- 172.21.15.99:0/2876133183 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2a816a6d0 con 0x7ff2a81577d0 2023-10-08T00:17:36.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.358+0000 7ff28effd700 1 -- 172.21.15.99:0/2876133183 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2980070f0 con 0x7ff2a81577d0 2023-10-08T00:17:36.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.358+0000 7ff2ad5ef700 1 -- 172.21.15.99:0/2876133183 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2a816ac90 con 0x7ff2a81577d0 2023-10-08T00:17:36.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.359+0000 7ff2ad5ef700 1 -- 172.21.15.99:0/2876133183 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2a80cf400 con 0x7ff2a81577d0 2023-10-08T00:17:36.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.360+0000 7ff28effd700 1 -- 172.21.15.99:0/2876133183 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff2980074a0 con 0x7ff2a81577d0 2023-10-08T00:17:36.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.360+0000 7ff28effd700 1 --2- 172.21.15.99:0/2876133183 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff294066f30 0x7ff2940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:36.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.360+0000 7ff28effd700 1 -- 172.21.15.99:0/2876133183 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 4853+0+0 (secure 0 0 0) 0x7ff2980846f0 con 0x7ff2a81577d0 2023-10-08T00:17:36.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.360+0000 7ff2a67fc700 1 --2- 172.21.15.99:0/2876133183 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff294066f30 0x7ff2940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:36.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.361+0000 7ff2a67fc700 1 --2- 172.21.15.99:0/2876133183 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff294066f30 0x7ff2940693e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ff29c0030c0 tx=0x7ff29c016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:36.368 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.364+0000 7ff28effd700 1 -- 172.21.15.99:0/2876133183 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff298051d30 con 0x7ff2a81577d0 2023-10-08T00:17:36.617 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.616+0000 7ff2ad5ef700 1 -- 172.21.15.99:0/2876133183 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff2a804c3c0 con 0x7ff2a81577d0 2023-10-08T00:17:36.617 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.616+0000 7ff28effd700 1 -- 172.21.15.99:0/2876133183 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7ff298055350 con 0x7ff2a81577d0 2023-10-08T00:17:36.617 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:17:36.617 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:17:36.618 INFO:teuthology.orchestra.run.smithi099.stdout: "auth_dump": [ 2023-10-08T00:17:36.618 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:36.618 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.a", 2023-10-08T00:17:36.618 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlFlN9AhAAKdbeH0XznSOuyUnReD6RsA==", 2023-10-08T00:17:36.618 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:17:36.618 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:17:36.618 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2023-10-08T00:17:36.618 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2023-10-08T00:17:36.618 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:17:36.619 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:36.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:36.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:36.619 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.b", 2023-10-08T00:17:36.619 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlh2G0DRAA1mYGrUrELHsS9abUxCCG2g==", 2023-10-08T00:17:36.619 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:17:36.620 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:17:36.620 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2023-10-08T00:17:36.620 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2023-10-08T00:17:36.620 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:17:36.620 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:36.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:36.621 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:36.621 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.c", 2023-10-08T00:17:36.621 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFl/M1DCRAANrfHJGZO1s0KOyCdUzZWIA==", 2023-10-08T00:17:36.621 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:17:36.621 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:17:36.621 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2023-10-08T00:17:36.621 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2023-10-08T00:17:36.622 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:17:36.622 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:36.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:36.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:36.622 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.d", 2023-10-08T00:17:36.622 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlr5F2FBAAYYeG6EM0/N0FiEAwSbYRyA==", 2023-10-08T00:17:36.622 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:17:36.623 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:17:36.623 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2023-10-08T00:17:36.623 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2023-10-08T00:17:36.623 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:17:36.623 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:36.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:36.623 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:36.623 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.0", 2023-10-08T00:17:36.624 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB08yFla7rpKxAAWTOutLcrjDeR69JbPZMoLg==", 2023-10-08T00:17:36.624 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:17:36.624 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:17:36.624 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:17:36.624 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:17:36.624 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:36.624 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:36.625 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:36.625 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.1", 2023-10-08T00:17:36.625 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB08yFlw+y2MhAAAvLDIB/tb49IlaireTqTFg==", 2023-10-08T00:17:36.625 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:17:36.625 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:17:36.625 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:17:36.625 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:17:36.626 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:36.626 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:36.626 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:36.626 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.2", 2023-10-08T00:17:36.626 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB18yFlh4wvABAALXOtJZoEfGmt/pOBxReooQ==", 2023-10-08T00:17:36.626 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:17:36.626 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:17:36.626 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:17:36.626 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:17:36.626 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:36.626 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:36.627 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:36.627 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.3", 2023-10-08T00:17:36.627 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB18yFl8cVfCRAAoGXTACMT+f5Avll5mxG9OA==", 2023-10-08T00:17:36.627 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:17:36.627 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:17:36.627 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:17:36.627 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:17:36.627 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:36.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:36.628 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:36.628 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.4", 2023-10-08T00:17:36.628 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB48yFlft9WGhAA8JZN4MjSX9wbXiMKJnfCYg==", 2023-10-08T00:17:36.628 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:17:36.628 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:17:36.629 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:17:36.629 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:17:36.629 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:36.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:36.629 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:36.629 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.5", 2023-10-08T00:17:36.630 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB48yFlUf1BIRAA2FMBt1jAcm4ia/vYBDyojw==", 2023-10-08T00:17:36.630 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:17:36.630 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:17:36.630 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:17:36.630 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:17:36.630 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:36.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:36.630 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:36.631 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.6", 2023-10-08T00:17:36.631 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB48yFl0SI6KhAAI6G+Sqfm7OQmGf9nK59x9Q==", 2023-10-08T00:17:36.631 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:17:36.631 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:17:36.631 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:17:36.631 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:17:36.631 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:36.632 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:36.632 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:36.632 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.7", 2023-10-08T00:17:36.632 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB48yFlo5xWMxAA4p9OSLK0JpWhEExsifX+3A==", 2023-10-08T00:17:36.632 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:17:36.632 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:17:36.632 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:17:36.632 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:17:36.633 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:36.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:36.633 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:36.633 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.0", 2023-10-08T00:17:36.633 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlmzhQGBAAReslU/Pvcsr0dwE6l1N3/Q==", 2023-10-08T00:17:36.633 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:17:36.633 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:17:36.633 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow r", 2023-10-08T00:17:36.634 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-08T00:17:36.634 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:36.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:36.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:36.634 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.1", 2023-10-08T00:17:36.634 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFl2NKsIBAAUg6rXynl1J7qB6BkUizmpA==", 2023-10-08T00:17:36.634 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:17:36.635 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:17:36.635 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2023-10-08T00:17:36.635 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2023-10-08T00:17:36.635 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2023-10-08T00:17:36.635 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:36.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:36.635 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:36.635 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.2", 2023-10-08T00:17:36.635 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlMp6rHRAArVgJfA3mY4n78hOMeVlw1Q==", 2023-10-08T00:17:36.635 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:17:36.635 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:17:36.635 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2023-10-08T00:17:36.635 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2023-10-08T00:17:36.636 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2023-10-08T00:17:36.636 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:36.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:36.636 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:36.636 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.3", 2023-10-08T00:17:36.636 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlH0gNJhAAEgGVGiabVIWknpWUohOnHA==", 2023-10-08T00:17:36.636 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:17:36.636 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:17:36.636 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2023-10-08T00:17:36.636 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2023-10-08T00:17:36.636 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2023-10-08T00:17:36.637 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:36.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:36.637 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:36.637 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.admin", 2023-10-08T00:17:36.637 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBw8yFlidUfIxAAIvpvkHbyCx5JFvHo/g5A9g==", 2023-10-08T00:17:36.637 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:17:36.637 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2023-10-08T00:17:36.637 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2023-10-08T00:17:36.637 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2023-10-08T00:17:36.637 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:17:36.637 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:36.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:36.638 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:36.638 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mgr.x", 2023-10-08T00:17:36.638 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBw8yFl2kMLOxAAsEyuLBLsjQCUS0ERDOi75A==", 2023-10-08T00:17:36.638 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:17:36.638 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2023-10-08T00:17:36.638 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile mgr", 2023-10-08T00:17:36.638 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:17:36.639 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:36.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:17:36.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:36.639 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mgr.y", 2023-10-08T00:17:36.639 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBw8yFl6ht5NxAAFzXnLDjklCePD6hw/pSfFg==", 2023-10-08T00:17:36.639 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:17:36.639 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2023-10-08T00:17:36.639 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile mgr", 2023-10-08T00:17:36.639 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:17:36.639 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:36.639 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:36.639 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:17:36.640 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:17:36.641 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.618+0000 7ff2ad5ef700 1 -- 172.21.15.99:0/2876133183 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff294066f30 msgr2=0x7ff2940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:36.641 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.618+0000 7ff2ad5ef700 1 --2- 172.21.15.99:0/2876133183 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff294066f30 0x7ff2940693e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ff29c0030c0 tx=0x7ff29c016000 comp rx=0 tx=0).stop 2023-10-08T00:17:36.641 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.619+0000 7ff2ad5ef700 1 -- 172.21.15.99:0/2876133183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2a81577d0 msgr2=0x7ff2a814cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:36.642 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.619+0000 7ff2ad5ef700 1 --2- 172.21.15.99:0/2876133183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2a81577d0 0x7ff2a814cdb0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7ff29800cdb0 tx=0x7ff298001ae0 comp rx=0 tx=0).stop 2023-10-08T00:17:36.642 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.619+0000 7ff2ad5ef700 1 -- 172.21.15.99:0/2876133183 shutdown_connections 2023-10-08T00:17:36.642 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.619+0000 7ff2ad5ef700 1 --2- 172.21.15.99:0/2876133183 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff294066f30 0x7ff2940693e0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:36.642 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.619+0000 7ff2ad5ef700 1 --2- 172.21.15.99:0/2876133183 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff2a815b0d0 0x7ff2a81527b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:36.642 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.619+0000 7ff2ad5ef700 1 --2- 172.21.15.99:0/2876133183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2a81577d0 0x7ff2a814cdb0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:36.642 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.619+0000 7ff2ad5ef700 1 --2- 172.21.15.99:0/2876133183 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff2a8157ba0 0x7ff2a8152270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:36.642 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.619+0000 7ff2ad5ef700 1 -- 172.21.15.99:0/2876133183 >> 172.21.15.99:0/2876133183 conn(0x7ff2a8068ce0 msgr2=0x7ff2a80d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:36.642 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.619+0000 7ff2ad5ef700 1 -- 172.21.15.99:0/2876133183 shutdown_connections 2023-10-08T00:17:36.642 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.619+0000 7ff2ad5ef700 1 -- 172.21.15.99:0/2876133183 wait complete. 2023-10-08T00:17:36.643 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:17:36.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.755+0000 7fa0c9eb4700 1 Processor -- start 2023-10-08T00:17:36.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.755+0000 7fa0c9eb4700 1 -- start start 2023-10-08T00:17:36.762 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.756+0000 7fa0c9eb4700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa0c41510f0 0x7fa0c406ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:36.763 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.756+0000 7fa0c9eb4700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0c40ca0b0 con 0x7fa0c40c9af0 2023-10-08T00:17:36.763 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.756+0000 7fa0c9eb4700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0c4068a10 con 0x7fa0c4070700 2023-10-08T00:17:36.763 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.756+0000 7fa0c9eb4700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0c4068cf0 con 0x7fa0c41510f0 2023-10-08T00:17:36.763 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.756+0000 7fa0c2ffd700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fa0c40c9af0 0x7fa0c415dd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:32788/0 (socket says 172.21.15.99:32788) 2023-10-08T00:17:36.763 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.756+0000 7fa0c2ffd700 1 -- 172.21.15.99:0/1622507805 learned_addr learned my addr 172.21.15.99:0/1622507805 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:17:36.763 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.756+0000 7fa0c27fc700 1 -- 172.21.15.99:0/1622507805 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3778339300 0 0) 0x7fa0c40ca0b0 con 0x7fa0c40c9af0 2023-10-08T00:17:36.763 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.757+0000 7fa0c27fc700 1 -- 172.21.15.99:0/1622507805 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0ac003570 con 0x7fa0c40c9af0 2023-10-08T00:17:36.763 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.757+0000 7fa0c37fe700 1 --2- 172.21.15.99:0/1622507805 >> v2:172.21.15.191:3300/0 conn(0x7fa0c41510f0 0x7fa0c406ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:36.763 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.757+0000 7fa0c27fc700 1 -- 172.21.15.99:0/1622507805 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2438206481 0 0) 0x7fa0ac003570 con 0x7fa0c40c9af0 2023-10-08T00:17:36.763 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.757+0000 7fa0c27fc700 1 -- 172.21.15.99:0/1622507805 >> v2:172.21.15.191:3300/0 conn(0x7fa0c41510f0 msgr2=0x7fa0c406ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:36.763 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.757+0000 7fa0c27fc700 1 --2- 172.21.15.99:0/1622507805 >> v2:172.21.15.191:3300/0 conn(0x7fa0c41510f0 0x7fa0c406ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:36.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.757+0000 7fa0c27fc700 1 -- 172.21.15.99:0/1622507805 >> v1:172.21.15.191:6789/0 conn(0x7fa0c4070700 legacy=0x7fa0c4070b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:36.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.757+0000 7fa0c27fc700 1 -- 172.21.15.99:0/1622507805 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c40cb230 con 0x7fa0c40c9af0 2023-10-08T00:17:36.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.757+0000 7fa0c27fc700 1 -- 172.21.15.99:0/1622507805 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7fa0b8001470 con 0x7fa0c40c9af0 2023-10-08T00:17:36.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.757+0000 7fa0c27fc700 1 -- 172.21.15.99:0/1622507805 >> v1:172.21.15.99:6789/0 conn(0x7fa0c40c9af0 legacy=0x7fa0c415dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:36.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.757+0000 7fa0c27fc700 1 --2- 172.21.15.99:0/1622507805 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0ac003a60 0x7fa0ac003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:36.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.757+0000 7fa0c27fc700 1 --2- 172.21.15.99:0/1622507805 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0ac004810 0x7fa0ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:36.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.757+0000 7fa0c27fc700 1 --2- 172.21.15.99:0/1622507805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0ac0052e0 0x7fa0ac00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:36.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.757+0000 7fa0c27fc700 1 -- 172.21.15.99:0/1622507805 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c4068cf0 con 0x7fa0ac0052e0 2023-10-08T00:17:36.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.757+0000 7fa0c27fc700 1 -- 172.21.15.99:0/1622507805 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ac00c190 con 0x7fa0ac004810 2023-10-08T00:17:36.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.757+0000 7fa0c27fc700 1 -- 172.21.15.99:0/1622507805 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ac00c350 con 0x7fa0ac003a60 2023-10-08T00:17:36.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.757+0000 7fa0c3fff700 1 --2- 172.21.15.99:0/1622507805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0ac0052e0 0x7fa0ac00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:36.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.757+0000 7fa0c37fe700 1 --2- 172.21.15.99:0/1622507805 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0ac003a60 0x7fa0ac003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:36.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.757+0000 7fa0c2ffd700 1 --2- 172.21.15.99:0/1622507805 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0ac004810 0x7fa0ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:36.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.757+0000 7fa0c3fff700 1 -- 172.21.15.99:0/1622507805 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0ac003a60 msgr2=0x7fa0ac003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:36.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.757+0000 7fa0c3fff700 1 --2- 172.21.15.99:0/1622507805 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0ac003a60 0x7fa0ac003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:36.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.757+0000 7fa0c3fff700 1 -- 172.21.15.99:0/1622507805 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0ac004810 msgr2=0x7fa0ac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:36.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.757+0000 7fa0c3fff700 1 --2- 172.21.15.99:0/1622507805 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0ac004810 0x7fa0ac004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:36.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.757+0000 7fa0c3fff700 1 -- 172.21.15.99:0/1622507805 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0ac003570 con 0x7fa0ac0052e0 2023-10-08T00:17:36.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.757+0000 7fa0c3fff700 1 -- 172.21.15.99:0/1622507805 reap_dead start 2023-10-08T00:17:36.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.757+0000 7fa0c3fff700 1 --2- 172.21.15.99:0/1622507805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0ac0052e0 0x7fa0ac00ba60 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc001880 tx=0x7fa0bc007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b65a0616c7264fe3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:36.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.758+0000 7fa0c27fc700 1 -- 172.21.15.99:0/1622507805 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc012420 con 0x7fa0ac0052e0 2023-10-08T00:17:36.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.758+0000 7fa0c27fc700 1 -- 172.21.15.99:0/1622507805 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0bc0125a0 con 0x7fa0ac0052e0 2023-10-08T00:17:36.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.758+0000 7fa0c27fc700 1 -- 172.21.15.99:0/1622507805 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc012960 con 0x7fa0ac0052e0 2023-10-08T00:17:36.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.758+0000 7fa0c9eb4700 1 -- 172.21.15.99:0/1622507805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0ac0052e0 msgr2=0x7fa0ac00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:36.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.758+0000 7fa0c9eb4700 1 --2- 172.21.15.99:0/1622507805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0ac0052e0 0x7fa0ac00ba60 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc001880 tx=0x7fa0bc007d30 comp rx=0 tx=0).stop 2023-10-08T00:17:36.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.758+0000 7fa0c9eb4700 1 -- 172.21.15.99:0/1622507805 shutdown_connections 2023-10-08T00:17:36.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.758+0000 7fa0c9eb4700 1 --2- 172.21.15.99:0/1622507805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0ac0052e0 0x7fa0ac00ba60 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:36.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.758+0000 7fa0c9eb4700 1 -- 172.21.15.99:0/1622507805 >> 172.21.15.99:0/1622507805 conn(0x7fa0c40ca4f0 msgr2=0x7fa0c40ca8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:36.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.758+0000 7fa0c9eb4700 1 -- 172.21.15.99:0/1622507805 shutdown_connections 2023-10-08T00:17:36.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.758+0000 7fa0c9eb4700 1 -- 172.21.15.99:0/1622507805 wait complete. 2023-10-08T00:17:36.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.759+0000 7fa0c9eb4700 1 Processor -- start 2023-10-08T00:17:36.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.759+0000 7fa0c9eb4700 1 -- start start 2023-10-08T00:17:36.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.760+0000 7fa0c9eb4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0ac0052e0 0x7fa0c416b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:36.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.760+0000 7fa0c9eb4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0c41542b0 0x7fa0c41505e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:36.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.760+0000 7fa0c9eb4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0c4152c80 0x7fa0c406a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:36.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.760+0000 7fa0c9eb4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c416b680 con 0x7fa0ac0052e0 2023-10-08T00:17:36.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.760+0000 7fa0c9eb4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c4153e20 con 0x7fa0c4152c80 2023-10-08T00:17:36.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.760+0000 7fa0c9eb4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c40b3440 con 0x7fa0c41542b0 2023-10-08T00:17:36.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.760+0000 7fa0c37fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0ac0052e0 0x7fa0c416b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:36.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.760+0000 7fa0c37fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0ac0052e0 0x7fa0c416b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60646/0 (socket says 172.21.15.99:60646) 2023-10-08T00:17:36.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.760+0000 7fa0c37fe700 1 -- 172.21.15.99:0/391930010 learned_addr learned my addr 172.21.15.99:0/391930010 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:36.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.760+0000 7fa0c3fff700 1 --2- 172.21.15.99:0/391930010 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0c4152c80 0x7fa0c406a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:36.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.760+0000 7fa0c2ffd700 1 --2- 172.21.15.99:0/391930010 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0c41542b0 0x7fa0c41505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:36.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.760+0000 7fa0c37fe700 1 -- 172.21.15.99:0/391930010 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0c41542b0 msgr2=0x7fa0c41505e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:36.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.760+0000 7fa0c37fe700 1 --2- 172.21.15.99:0/391930010 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0c41542b0 0x7fa0c41505e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:36.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.760+0000 7fa0c37fe700 1 -- 172.21.15.99:0/391930010 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0c4152c80 msgr2=0x7fa0c406a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:36.768 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.760+0000 7fa0c37fe700 1 --2- 172.21.15.99:0/391930010 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0c4152c80 0x7fa0c406a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:36.768 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.761+0000 7fa0c37fe700 1 -- 172.21.15.99:0/391930010 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c4150b20 con 0x7fa0ac0052e0 2023-10-08T00:17:36.768 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.761+0000 7fa0c37fe700 1 --2- 172.21.15.99:0/391930010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0ac0052e0 0x7fa0c416b140 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4002300 tx=0x7fa0b4009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:36.768 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.761+0000 7fa0ab7fe700 1 -- 172.21.15.99:0/391930010 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b4015420 con 0x7fa0ac0052e0 2023-10-08T00:17:36.768 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.761+0000 7fa0ab7fe700 1 -- 172.21.15.99:0/391930010 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b4015580 con 0x7fa0ac0052e0 2023-10-08T00:17:36.768 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.761+0000 7fa0c9eb4700 1 -- 172.21.15.99:0/391930010 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c4153210 con 0x7fa0ac0052e0 2023-10-08T00:17:36.768 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.761+0000 7fa0ab7fe700 1 -- 172.21.15.99:0/391930010 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b4015920 con 0x7fa0ac0052e0 2023-10-08T00:17:36.768 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.761+0000 7fa0c9eb4700 1 -- 172.21.15.99:0/391930010 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c41537d0 con 0x7fa0ac0052e0 2023-10-08T00:17:36.768 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.762+0000 7fa0c9eb4700 1 -- 172.21.15.99:0/391930010 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0c404c3c0 con 0x7fa0ac0052e0 2023-10-08T00:17:36.768 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.762+0000 7fa0ab7fe700 1 -- 172.21.15.99:0/391930010 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa0b4003040 con 0x7fa0ac0052e0 2023-10-08T00:17:36.769 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.762+0000 7fa0ab7fe700 1 --2- 172.21.15.99:0/391930010 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa0b0066f30 0x7fa0b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:36.769 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.763+0000 7fa0ab7fe700 1 -- 172.21.15.99:0/391930010 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fa0b4084b90 con 0x7fa0ac0052e0 2023-10-08T00:17:36.769 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.763+0000 7fa0c2ffd700 1 --2- 172.21.15.99:0/391930010 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa0b0066f30 0x7fa0b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:36.770 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.763+0000 7fa0c2ffd700 1 --2- 172.21.15.99:0/391930010 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa0b0066f30 0x7fa0b00693e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8001050 tx=0x7fa0b8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:36.770 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.767+0000 7fa0ab7fe700 1 -- 172.21.15.99:0/391930010 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa0b4052150 con 0x7fa0ac0052e0 2023-10-08T00:17:36.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.948+0000 7fa0c9eb4700 1 -- 172.21.15.99:0/391930010 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa0c406b620 con 0x7fa0ac0052e0 2023-10-08T00:17:36.950 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.949+0000 7fa0ab7fe700 1 -- 172.21.15.99:0/391930010 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) v1 ==== 74+0+12145 (secure 0 0 0) 0x7fa0b4055770 con 0x7fa0ac0052e0 2023-10-08T00:17:36.950 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:17:36.950 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":150,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:17:36.217329+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:17:36.952 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.951+0000 7fa0c9eb4700 1 -- 172.21.15.99:0/391930010 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa0b0066f30 msgr2=0x7fa0b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:36.952 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.951+0000 7fa0c9eb4700 1 --2- 172.21.15.99:0/391930010 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa0b0066f30 0x7fa0b00693e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8001050 tx=0x7fa0b8009040 comp rx=0 tx=0).stop 2023-10-08T00:17:36.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.951+0000 7fa0c9eb4700 1 -- 172.21.15.99:0/391930010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0ac0052e0 msgr2=0x7fa0c416b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:36.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.951+0000 7fa0c9eb4700 1 --2- 172.21.15.99:0/391930010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0ac0052e0 0x7fa0c416b140 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4002300 tx=0x7fa0b4009710 comp rx=0 tx=0).stop 2023-10-08T00:17:36.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.951+0000 7fa0c9eb4700 1 -- 172.21.15.99:0/391930010 shutdown_connections 2023-10-08T00:17:36.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.951+0000 7fa0c9eb4700 1 --2- 172.21.15.99:0/391930010 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa0b0066f30 0x7fa0b00693e0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:36.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.951+0000 7fa0c9eb4700 1 --2- 172.21.15.99:0/391930010 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0c4152c80 0x7fa0c406a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:36.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.951+0000 7fa0c9eb4700 1 --2- 172.21.15.99:0/391930010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0ac0052e0 0x7fa0c416b140 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:36.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.951+0000 7fa0c9eb4700 1 --2- 172.21.15.99:0/391930010 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0c41542b0 0x7fa0c41505e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:36.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.951+0000 7fa0c9eb4700 1 -- 172.21.15.99:0/391930010 >> 172.21.15.99:0/391930010 conn(0x7fa0c40ca4f0 msgr2=0x7fa0c40c2ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:36.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.951+0000 7fa0c9eb4700 1 -- 172.21.15.99:0/391930010 shutdown_connections 2023-10-08T00:17:36.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:36.951+0000 7fa0c9eb4700 1 -- 172.21.15.99:0/391930010 wait complete. 2023-10-08T00:17:36.964 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:10:42.063344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-08T00:17:36.964 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T00:17:37.081 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.081+0000 7f0464a2f700 1 Processor -- start 2023-10-08T00:17:37.081 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.081+0000 7f0464a2f700 1 -- start start 2023-10-08T00:17:37.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.081+0000 7f0464a2f700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f0460159ec0 0x7f04600af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:37.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.081+0000 7f0464a2f700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04600b0790 con 0x7f046015a290 2023-10-08T00:17:37.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.081+0000 7f0464a2f700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04600ce1f0 con 0x7f046006acb0 2023-10-08T00:17:37.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.081+0000 7f0464a2f700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04600b0b20 con 0x7f0460159ec0 2023-10-08T00:17:37.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.082+0000 7f0455d9b700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f046015a290 0x7f046006a5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:32798/0 (socket says 172.21.15.99:32798) 2023-10-08T00:17:37.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.082+0000 7f0455d9b700 1 -- 172.21.15.99:0/540911793 learned_addr learned my addr 172.21.15.99:0/540911793 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:17:37.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.082+0000 7f045dd9b700 1 -- 172.21.15.99:0/540911793 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2365924336 0 0) 0x7f04600b0790 con 0x7f046015a290 2023-10-08T00:17:37.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.082+0000 7f045dd9b700 1 -- 172.21.15.99:0/540911793 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f044c003570 con 0x7f046015a290 2023-10-08T00:17:37.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.082+0000 7f045e59c700 1 --2- 172.21.15.99:0/540911793 >> v2:172.21.15.191:3300/0 conn(0x7f0460159ec0 0x7f04600af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:37.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.082+0000 7f045dd9b700 1 -- 172.21.15.99:0/540911793 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1892590998 0 0) 0x7f044c003570 con 0x7f046015a290 2023-10-08T00:17:37.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.082+0000 7f045dd9b700 1 -- 172.21.15.99:0/540911793 >> v2:172.21.15.191:3300/0 conn(0x7f0460159ec0 msgr2=0x7f04600af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:37.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.082+0000 7f045dd9b700 1 --2- 172.21.15.99:0/540911793 >> v2:172.21.15.191:3300/0 conn(0x7f0460159ec0 0x7f04600af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:37.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.082+0000 7f045dd9b700 1 -- 172.21.15.99:0/540911793 >> v1:172.21.15.191:6789/0 conn(0x7f046006acb0 legacy=0x7f04600cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:37.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.082+0000 7f045dd9b700 1 -- 172.21.15.99:0/540911793 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04600b04b0 con 0x7f046015a290 2023-10-08T00:17:37.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.082+0000 7f045dd9b700 1 -- 172.21.15.99:0/540911793 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f0448001470 con 0x7f046015a290 2023-10-08T00:17:37.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.082+0000 7f045dd9b700 1 -- 172.21.15.99:0/540911793 >> v1:172.21.15.99:6789/0 conn(0x7f046015a290 legacy=0x7f046006a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:37.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.082+0000 7f045dd9b700 1 --2- 172.21.15.99:0/540911793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f044c003a60 0x7f044c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:37.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.082+0000 7f045dd9b700 1 --2- 172.21.15.99:0/540911793 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f044c004810 0x7f044c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:37.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.082+0000 7f045dd9b700 1 --2- 172.21.15.99:0/540911793 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f044c0052e0 0x7f044c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:37.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.082+0000 7f045dd9b700 1 -- 172.21.15.99:0/540911793 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04600b0b20 con 0x7f044c003a60 2023-10-08T00:17:37.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.082+0000 7f045dd9b700 1 -- 172.21.15.99:0/540911793 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f044c00c230 con 0x7f044c004810 2023-10-08T00:17:37.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.082+0000 7f045dd9b700 1 -- 172.21.15.99:0/540911793 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f044c00c3f0 con 0x7f044c0052e0 2023-10-08T00:17:37.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.082+0000 7f045e59c700 1 --2- 172.21.15.99:0/540911793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f044c003a60 0x7f044c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:37.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.082+0000 7f0455d9b700 1 --2- 172.21.15.99:0/540911793 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f044c004810 0x7f044c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:37.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.082+0000 7f045ed9d700 1 --2- 172.21.15.99:0/540911793 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f044c0052e0 0x7f044c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:37.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.083+0000 7f045e59c700 1 -- 172.21.15.99:0/540911793 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f044c0052e0 msgr2=0x7f044c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:37.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.083+0000 7f045e59c700 1 --2- 172.21.15.99:0/540911793 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f044c0052e0 0x7f044c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:37.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.083+0000 7f045e59c700 1 -- 172.21.15.99:0/540911793 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f044c004810 msgr2=0x7f044c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:37.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.083+0000 7f045e59c700 1 --2- 172.21.15.99:0/540911793 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f044c004810 0x7f044c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:37.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.083+0000 7f045e59c700 1 -- 172.21.15.99:0/540911793 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f044c003570 con 0x7f044c003a60 2023-10-08T00:17:37.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.083+0000 7f045ed9d700 1 -- 172.21.15.99:0/540911793 reap_dead start 2023-10-08T00:17:37.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.083+0000 7f045e59c700 1 --2- 172.21.15.99:0/540911793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f044c003a60 0x7f044c003e70 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f045000bb90 tx=0x7f0450008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9060a1a1aea6bcd2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:37.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.083+0000 7f045dd9b700 1 -- 172.21.15.99:0/540911793 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f045000d5d0 con 0x7f044c003a60 2023-10-08T00:17:37.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.083+0000 7f045dd9b700 1 -- 172.21.15.99:0/540911793 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f045000d730 con 0x7f044c003a60 2023-10-08T00:17:37.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.083+0000 7f045dd9b700 1 -- 172.21.15.99:0/540911793 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f045000daf0 con 0x7f044c003a60 2023-10-08T00:17:37.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.083+0000 7f0464a2f700 1 -- 172.21.15.99:0/540911793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f044c003a60 msgr2=0x7f044c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:37.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.083+0000 7f0464a2f700 1 --2- 172.21.15.99:0/540911793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f044c003a60 0x7f044c003e70 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f045000bb90 tx=0x7f0450008040 comp rx=0 tx=0).stop 2023-10-08T00:17:37.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.083+0000 7f0464a2f700 1 -- 172.21.15.99:0/540911793 shutdown_connections 2023-10-08T00:17:37.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.083+0000 7f0464a2f700 1 --2- 172.21.15.99:0/540911793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f044c003a60 0x7f044c003e70 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:37.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.083+0000 7f0464a2f700 1 -- 172.21.15.99:0/540911793 >> 172.21.15.99:0/540911793 conn(0x7f04600d1880 msgr2=0x7f04600cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:37.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.083+0000 7f0464a2f700 1 -- 172.21.15.99:0/540911793 shutdown_connections 2023-10-08T00:17:37.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.083+0000 7f0464a2f700 1 -- 172.21.15.99:0/540911793 wait complete. 2023-10-08T00:17:37.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.084+0000 7f0464a2f700 1 Processor -- start 2023-10-08T00:17:37.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.085+0000 7f0464a2f700 1 -- start start 2023-10-08T00:17:37.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.085+0000 7f0464a2f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f044c003a60 0x7f0460153c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:37.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.085+0000 7f0464a2f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04601541b0 0x7f046016a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:37.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.085+0000 7f0464a2f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f046016a700 0x7f046016ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:37.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.085+0000 7f0464a2f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0460155340 con 0x7f046016a700 2023-10-08T00:17:37.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.085+0000 7f0464a2f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f046016b090 con 0x7f04601541b0 2023-10-08T00:17:37.093 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.085+0000 7f0464a2f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0460154730 con 0x7f044c003a60 2023-10-08T00:17:37.093 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.085+0000 7f045ed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f046016a700 0x7f046016ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:37.093 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.085+0000 7f045ed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f046016a700 0x7f046016ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60670/0 (socket says 172.21.15.99:60670) 2023-10-08T00:17:37.093 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.085+0000 7f045ed9d700 1 -- 172.21.15.99:0/2255681623 learned_addr learned my addr 172.21.15.99:0/2255681623 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:37.093 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.085+0000 7f045e59c700 1 --2- 172.21.15.99:0/2255681623 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f044c003a60 0x7f0460153c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:37.093 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.085+0000 7f0455d9b700 1 --2- 172.21.15.99:0/2255681623 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04601541b0 0x7f046016a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:37.093 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.086+0000 7f045ed9d700 1 -- 172.21.15.99:0/2255681623 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f044c003a60 msgr2=0x7f0460153c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:37.093 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.086+0000 7f045ed9d700 1 --2- 172.21.15.99:0/2255681623 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f044c003a60 0x7f0460153c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:37.093 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.086+0000 7f045ed9d700 1 -- 172.21.15.99:0/2255681623 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04601541b0 msgr2=0x7f046016a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:37.093 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.086+0000 7f045ed9d700 1 --2- 172.21.15.99:0/2255681623 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04601541b0 0x7f046016a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:37.093 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.086+0000 7f045ed9d700 1 -- 172.21.15.99:0/2255681623 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0460154a10 con 0x7f046016a700 2023-10-08T00:17:37.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.086+0000 7f0455d9b700 1 --2- 172.21.15.99:0/2255681623 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04601541b0 0x7f046016a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:17:37.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.086+0000 7f045ed9d700 1 --2- 172.21.15.99:0/2255681623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f046016a700 0x7f046016ab50 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f04600ce900 tx=0x7f0458009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:37.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.086+0000 7f0456ffd700 1 -- 172.21.15.99:0/2255681623 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0458006db0 con 0x7f046016a700 2023-10-08T00:17:37.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.086+0000 7f0456ffd700 1 -- 172.21.15.99:0/2255681623 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0458003030 con 0x7f046016a700 2023-10-08T00:17:37.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.086+0000 7f0464a2f700 1 -- 172.21.15.99:0/2255681623 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0460154cf0 con 0x7f046016a700 2023-10-08T00:17:37.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.086+0000 7f0456ffd700 1 -- 172.21.15.99:0/2255681623 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04580033f0 con 0x7f046016a700 2023-10-08T00:17:37.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.086+0000 7f0464a2f700 1 -- 172.21.15.99:0/2255681623 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f046014efb0 con 0x7f046016a700 2023-10-08T00:17:37.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.087+0000 7f0464a2f700 1 -- 172.21.15.99:0/2255681623 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f046004c3c0 con 0x7f046016a700 2023-10-08T00:17:37.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.087+0000 7f0456ffd700 1 -- 172.21.15.99:0/2255681623 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0458001790 con 0x7f046016a700 2023-10-08T00:17:37.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.088+0000 7f0456ffd700 1 --2- 172.21.15.99:0/2255681623 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0440066f30 0x7f04400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:37.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.088+0000 7f0456ffd700 1 -- 172.21.15.99:0/2255681623 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f04580835b0 con 0x7f046016a700 2023-10-08T00:17:37.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.088+0000 7f045e59c700 1 --2- 172.21.15.99:0/2255681623 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0440066f30 0x7f04400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:37.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.088+0000 7f045e59c700 1 --2- 172.21.15.99:0/2255681623 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0440066f30 0x7f04400693e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f045000bb90 tx=0x7f045001b000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:37.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.092+0000 7f0456ffd700 1 -- 172.21.15.99:0/2255681623 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0458050b70 con 0x7f046016a700 2023-10-08T00:17:37.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.273+0000 7f0464a2f700 1 -- 172.21.15.99:0/2255681623 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0460155480 con 0x7f046016a700 2023-10-08T00:17:37.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.273+0000 7f0456ffd700 1 -- 172.21.15.99:0/2255681623 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v150) v1 ==== 93+0+10 (secure 0 0 0) 0x7f045801b0a0 con 0x7f046016a700 2023-10-08T00:17:37.274 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2023-10-08T00:17:37.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.275+0000 7f0464a2f700 1 -- 172.21.15.99:0/2255681623 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0440066f30 msgr2=0x7f04400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:37.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.275+0000 7f0464a2f700 1 --2- 172.21.15.99:0/2255681623 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0440066f30 0x7f04400693e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f045000bb90 tx=0x7f045001b000 comp rx=0 tx=0).stop 2023-10-08T00:17:37.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.275+0000 7f0464a2f700 1 -- 172.21.15.99:0/2255681623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f046016a700 msgr2=0x7f046016ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:37.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.275+0000 7f0464a2f700 1 --2- 172.21.15.99:0/2255681623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f046016a700 0x7f046016ab50 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f04600ce900 tx=0x7f0458009040 comp rx=0 tx=0).stop 2023-10-08T00:17:37.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.275+0000 7f0464a2f700 1 -- 172.21.15.99:0/2255681623 shutdown_connections 2023-10-08T00:17:37.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.275+0000 7f0464a2f700 1 --2- 172.21.15.99:0/2255681623 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0440066f30 0x7f04400693e0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:37.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.275+0000 7f0464a2f700 1 --2- 172.21.15.99:0/2255681623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f046016a700 0x7f046016ab50 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:37.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.275+0000 7f0464a2f700 1 --2- 172.21.15.99:0/2255681623 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f044c003a60 0x7f0460153c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:37.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.275+0000 7f0464a2f700 1 --2- 172.21.15.99:0/2255681623 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04601541b0 0x7f046016a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:37.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.275+0000 7f0464a2f700 1 -- 172.21.15.99:0/2255681623 >> 172.21.15.99:0/2255681623 conn(0x7f04600d1880 msgr2=0x7f04600b0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:37.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.275+0000 7f0464a2f700 1 -- 172.21.15.99:0/2255681623 shutdown_connections 2023-10-08T00:17:37.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.275+0000 7f0464a2f700 1 -- 172.21.15.99:0/2255681623 wait complete. 2023-10-08T00:17:37.288 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T00:17:37.402 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.401+0000 7f243a451700 1 Processor -- start 2023-10-08T00:17:37.402 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.401+0000 7f243a451700 1 -- start start 2023-10-08T00:17:37.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.402+0000 7f243a451700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2434157c80 0x7f24340ca810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:37.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.402+0000 7f243a451700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f243415bd50 con 0x7f24341578b0 2023-10-08T00:17:37.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.402+0000 7f243a451700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f243406a150 con 0x7f243406fe80 2023-10-08T00:17:37.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.402+0000 7f243a451700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f243406a710 con 0x7f2434157c80 2023-10-08T00:17:37.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.402+0000 7f243944f700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f24341578b0 0x7f243415c1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:32806/0 (socket says 172.21.15.99:32806) 2023-10-08T00:17:37.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.402+0000 7f243944f700 1 -- 172.21.15.99:0/3258262258 learned_addr learned my addr 172.21.15.99:0/3258262258 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:17:37.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.402+0000 7f2439c50700 1 --2- 172.21.15.99:0/3258262258 >> v2:172.21.15.191:3300/0 conn(0x7f2434157c80 0x7f24340ca810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:37.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.402+0000 7f242bfff700 1 -- 172.21.15.99:0/3258262258 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4198438907 0 0) 0x7f243415bd50 con 0x7f24341578b0 2023-10-08T00:17:37.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.402+0000 7f242bfff700 1 -- 172.21.15.99:0/3258262258 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2420003570 con 0x7f24341578b0 2023-10-08T00:17:37.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.403+0000 7f2439c50700 1 -- 172.21.15.99:0/3258262258 >> v1:172.21.15.191:6789/0 conn(0x7f243406fe80 legacy=0x7f2434069870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:37.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.403+0000 7f2439c50700 1 -- 172.21.15.99:0/3258262258 >> v1:172.21.15.99:6789/0 conn(0x7f24341578b0 legacy=0x7f243415c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:37.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.403+0000 7f2439c50700 1 -- 172.21.15.99:0/3258262258 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f243406a430 con 0x7f2434157c80 2023-10-08T00:17:37.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.403+0000 7f2439c50700 1 --2- 172.21.15.99:0/3258262258 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2434157c80 0x7f24340ca810 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f242c009e60 tx=0x7f242c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d066bbffcfe9be42 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:37.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.403+0000 7f242bfff700 1 -- 172.21.15.99:0/3258262258 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242c00e070 con 0x7f2434157c80 2023-10-08T00:17:37.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.403+0000 7f242bfff700 1 -- 172.21.15.99:0/3258262258 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f242c00bb60 con 0x7f2434157c80 2023-10-08T00:17:37.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.403+0000 7f242bfff700 1 -- 172.21.15.99:0/3258262258 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242c012430 con 0x7f2434157c80 2023-10-08T00:17:37.407 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.403+0000 7f243a451700 1 -- 172.21.15.99:0/3258262258 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2434157c80 msgr2=0x7f24340ca810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:37.407 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.403+0000 7f243a451700 1 --2- 172.21.15.99:0/3258262258 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2434157c80 0x7f24340ca810 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f242c009e60 tx=0x7f242c009780 comp rx=0 tx=0).stop 2023-10-08T00:17:37.408 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.403+0000 7f243a451700 1 -- 172.21.15.99:0/3258262258 shutdown_connections 2023-10-08T00:17:37.408 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.403+0000 7f243a451700 1 --2- 172.21.15.99:0/3258262258 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2434157c80 0x7f24340ca810 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:37.408 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.403+0000 7f243a451700 1 -- 172.21.15.99:0/3258262258 >> 172.21.15.99:0/3258262258 conn(0x7f243406bc70 msgr2=0x7f243406c070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:37.408 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.404+0000 7f243a451700 1 -- 172.21.15.99:0/3258262258 shutdown_connections 2023-10-08T00:17:37.408 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.404+0000 7f243a451700 1 -- 172.21.15.99:0/3258262258 wait complete. 2023-10-08T00:17:37.408 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.405+0000 7f243a451700 1 Processor -- start 2023-10-08T00:17:37.408 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.405+0000 7f243a451700 1 -- start start 2023-10-08T00:17:37.408 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.405+0000 7f243a451700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f243406fe80 0x7f24340c52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:37.408 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.405+0000 7f243a451700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24341578b0 0x7f243414bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:37.408 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.405+0000 7f243a451700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2434157c80 0x7f2434067cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:37.408 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.405+0000 7f243a451700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2434069290 con 0x7f24341578b0 2023-10-08T00:17:37.408 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.405+0000 7f243a451700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2434068680 con 0x7f243406fe80 2023-10-08T00:17:37.408 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.405+0000 7f243a451700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2434068960 con 0x7f2434157c80 2023-10-08T00:17:37.409 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.406+0000 7f2438c4e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24341578b0 0x7f243414bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:37.409 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.406+0000 7f2438c4e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24341578b0 0x7f243414bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60684/0 (socket says 172.21.15.99:60684) 2023-10-08T00:17:37.409 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.406+0000 7f2438c4e700 1 -- 172.21.15.99:0/1542697603 learned_addr learned my addr 172.21.15.99:0/1542697603 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:37.409 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.406+0000 7f243944f700 1 --2- 172.21.15.99:0/1542697603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f243406fe80 0x7f24340c52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:37.409 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.406+0000 7f2439c50700 1 --2- 172.21.15.99:0/1542697603 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2434157c80 0x7f2434067cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:37.409 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.406+0000 7f2438c4e700 1 -- 172.21.15.99:0/1542697603 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2434157c80 msgr2=0x7f2434067cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:37.409 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.406+0000 7f2438c4e700 1 --2- 172.21.15.99:0/1542697603 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2434157c80 0x7f2434067cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:37.409 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.406+0000 7f2438c4e700 1 -- 172.21.15.99:0/1542697603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f243406fe80 msgr2=0x7f24340c52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:37.409 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.406+0000 7f2438c4e700 1 --2- 172.21.15.99:0/1542697603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f243406fe80 0x7f24340c52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:37.409 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.406+0000 7f2438c4e700 1 -- 172.21.15.99:0/1542697603 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f242c009a20 con 0x7f24341578b0 2023-10-08T00:17:37.409 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.406+0000 7f2438c4e700 1 --2- 172.21.15.99:0/1542697603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24341578b0 0x7f243414bfd0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f242400c510 tx=0x7f242400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:37.409 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.406+0000 7f24297fa700 1 -- 172.21.15.99:0/1542697603 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2424011590 con 0x7f24341578b0 2023-10-08T00:17:37.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.406+0000 7f24297fa700 1 -- 172.21.15.99:0/1542697603 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2424011710 con 0x7f24341578b0 2023-10-08T00:17:37.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.406+0000 7f243a451700 1 -- 172.21.15.99:0/1542697603 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2434068c40 con 0x7f24341578b0 2023-10-08T00:17:37.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.406+0000 7f24297fa700 1 -- 172.21.15.99:0/1542697603 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2424011ad0 con 0x7f24341578b0 2023-10-08T00:17:37.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.406+0000 7f243a451700 1 -- 172.21.15.99:0/1542697603 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2434068230 con 0x7f24341578b0 2023-10-08T00:17:37.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.407+0000 7f243a451700 1 -- 172.21.15.99:0/1542697603 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2434069620 con 0x7f24341578b0 2023-10-08T00:17:37.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.407+0000 7f24297fa700 1 -- 172.21.15.99:0/1542697603 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f242401b3b0 con 0x7f24341578b0 2023-10-08T00:17:37.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.408+0000 7f24297fa700 1 --2- 172.21.15.99:0/1542697603 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2418066f30 0x7f24180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:37.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.408+0000 7f24297fa700 1 -- 172.21.15.99:0/1542697603 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f2424083760 con 0x7f24341578b0 2023-10-08T00:17:37.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.408+0000 7f243944f700 1 --2- 172.21.15.99:0/1542697603 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2418066f30 0x7f24180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:37.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.408+0000 7f243944f700 1 --2- 172.21.15.99:0/1542697603 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2418066f30 0x7f24180693e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f24300058d0 tx=0x7f2430009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:37.412 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.412+0000 7f24297fa700 1 -- 172.21.15.99:0/1542697603 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2424050d20 con 0x7f24341578b0 2023-10-08T00:17:37.596 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.595+0000 7f243a451700 1 -- 172.21.15.99:0/1542697603 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f24340693d0 con 0x7f24341578b0 2023-10-08T00:17:37.596 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.596+0000 7f24297fa700 1 -- 172.21.15.99:0/1542697603 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v150) v1 ==== 84+0+65 (secure 0 0 0) 0x7f2424017020 con 0x7f24341578b0 2023-10-08T00:17:37.597 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:17:37.597 INFO:teuthology.orchestra.run.smithi099.stdout:[ 2023-10-08T00:17:37.597 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:17:37.597 INFO:teuthology.orchestra.run.smithi099.stdout: "poolnum": 1, 2023-10-08T00:17:37.597 INFO:teuthology.orchestra.run.smithi099.stdout: "poolname": ".mgr" 2023-10-08T00:17:37.597 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:17:37.597 INFO:teuthology.orchestra.run.smithi099.stdout:] 2023-10-08T00:17:37.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.598+0000 7f243a451700 1 -- 172.21.15.99:0/1542697603 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2418066f30 msgr2=0x7f24180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:37.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.598+0000 7f243a451700 1 --2- 172.21.15.99:0/1542697603 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2418066f30 0x7f24180693e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f24300058d0 tx=0x7f2430009040 comp rx=0 tx=0).stop 2023-10-08T00:17:37.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.598+0000 7f243a451700 1 -- 172.21.15.99:0/1542697603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24341578b0 msgr2=0x7f243414bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:37.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.598+0000 7f243a451700 1 --2- 172.21.15.99:0/1542697603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24341578b0 0x7f243414bfd0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f242400c510 tx=0x7f242400c8d0 comp rx=0 tx=0).stop 2023-10-08T00:17:37.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.598+0000 7f243a451700 1 -- 172.21.15.99:0/1542697603 shutdown_connections 2023-10-08T00:17:37.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.598+0000 7f243a451700 1 --2- 172.21.15.99:0/1542697603 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2418066f30 0x7f24180693e0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:37.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.598+0000 7f243a451700 1 --2- 172.21.15.99:0/1542697603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f243406fe80 0x7f24340c52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:37.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.598+0000 7f243a451700 1 --2- 172.21.15.99:0/1542697603 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2434157c80 0x7f2434067cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:37.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.598+0000 7f243a451700 1 --2- 172.21.15.99:0/1542697603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24341578b0 0x7f243414bfd0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:37.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.598+0000 7f243a451700 1 -- 172.21.15.99:0/1542697603 >> 172.21.15.99:0/1542697603 conn(0x7f243406bc70 msgr2=0x7f24340b0670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:37.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.598+0000 7f243a451700 1 -- 172.21.15.99:0/1542697603 shutdown_connections 2023-10-08T00:17:37.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.598+0000 7f243a451700 1 -- 172.21.15.99:0/1542697603 wait complete. 2023-10-08T00:17:37.610 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T00:17:37.611 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T00:17:37.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.728+0000 7eff35379700 1 Processor -- start 2023-10-08T00:17:37.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.728+0000 7eff35379700 1 -- start start 2023-10-08T00:17:37.731 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.728+0000 7eff35379700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7eff30159ef0 0x7eff3015a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:37.731 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.728+0000 7eff35379700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7eff300b6a10 0x7eff300b10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:37.731 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.729+0000 7eff35379700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff300b1a60 con 0x7eff300b15e0 2023-10-08T00:17:37.731 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.729+0000 7eff35379700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff300b1d40 con 0x7eff30159ef0 2023-10-08T00:17:37.731 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.729+0000 7eff35379700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff300b2020 con 0x7eff300b6a10 2023-10-08T00:17:37.731 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.729+0000 7eff2effd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7eff30159ef0 0x7eff3015a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:37.731 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.729+0000 7eff2effd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7eff30159ef0 0x7eff3015a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60690/0 (socket says 172.21.15.99:60690) 2023-10-08T00:17:37.732 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.729+0000 7eff2effd700 1 -- 172.21.15.99:0/3325036315 learned_addr learned my addr 172.21.15.99:0/3325036315 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:37.732 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.729+0000 7eff2e7fc700 1 --2- 172.21.15.99:0/3325036315 >> v2:172.21.15.191:3300/0 conn(0x7eff300b6a10 0x7eff300b10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:37.732 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.729+0000 7eff2effd700 1 -- 172.21.15.99:0/3325036315 >> v2:172.21.15.191:3300/0 conn(0x7eff300b6a10 msgr2=0x7eff300b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:37.732 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.729+0000 7eff2effd700 1 --2- 172.21.15.99:0/3325036315 >> v2:172.21.15.191:3300/0 conn(0x7eff300b6a10 0x7eff300b10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:37.732 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.729+0000 7eff2effd700 1 -- 172.21.15.99:0/3325036315 >> v1:172.21.15.191:6789/0 conn(0x7eff300b15e0 legacy=0x7eff300b46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:37.732 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.729+0000 7eff2effd700 1 -- 172.21.15.99:0/3325036315 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff300b2300 con 0x7eff30159ef0 2023-10-08T00:17:37.732 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.729+0000 7eff2effd700 1 --2- 172.21.15.99:0/3325036315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff30159ef0 0x7eff3015a2c0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7eff30089550 tx=0x7eff20009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de5d9a13acea3a5d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:37.732 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.729+0000 7eff2dffb700 1 -- 172.21.15.99:0/3325036315 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff20005c10 con 0x7eff30159ef0 2023-10-08T00:17:37.732 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.729+0000 7eff2dffb700 1 -- 172.21.15.99:0/3325036315 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff20005d70 con 0x7eff30159ef0 2023-10-08T00:17:37.732 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.729+0000 7eff2dffb700 1 -- 172.21.15.99:0/3325036315 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff20006260 con 0x7eff30159ef0 2023-10-08T00:17:37.732 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.730+0000 7eff35379700 1 -- 172.21.15.99:0/3325036315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff30159ef0 msgr2=0x7eff3015a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:37.732 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.730+0000 7eff35379700 1 --2- 172.21.15.99:0/3325036315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff30159ef0 0x7eff3015a2c0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7eff30089550 tx=0x7eff20009220 comp rx=0 tx=0).stop 2023-10-08T00:17:37.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.730+0000 7eff35379700 1 -- 172.21.15.99:0/3325036315 shutdown_connections 2023-10-08T00:17:37.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.730+0000 7eff35379700 1 --2- 172.21.15.99:0/3325036315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff30159ef0 0x7eff3015a2c0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:37.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.730+0000 7eff35379700 1 --2- 172.21.15.99:0/3325036315 >> v2:172.21.15.191:3300/0 conn(0x7eff300b6a10 0x7eff300b10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:37.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.730+0000 7eff35379700 1 -- 172.21.15.99:0/3325036315 >> 172.21.15.99:0/3325036315 conn(0x7eff3009a0a0 msgr2=0x7eff30067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:37.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.730+0000 7eff35379700 1 -- 172.21.15.99:0/3325036315 shutdown_connections 2023-10-08T00:17:37.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.730+0000 7eff35379700 1 -- 172.21.15.99:0/3325036315 wait complete. 2023-10-08T00:17:37.735 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.731+0000 7eff35379700 1 Processor -- start 2023-10-08T00:17:37.735 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.731+0000 7eff35379700 1 -- start start 2023-10-08T00:17:37.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.732+0000 7eff35379700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7eff300b6a10 0x7eff3014f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:37.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.732+0000 7eff35379700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7eff30159ef0 0x7eff3009e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:37.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.732+0000 7eff35379700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff3014f980 0x7eff3015c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:37.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.732+0000 7eff35379700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff30155560 con 0x7eff3014f980 2023-10-08T00:17:37.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.732+0000 7eff35379700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff30155840 con 0x7eff30159ef0 2023-10-08T00:17:37.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.732+0000 7eff35379700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff30155b20 con 0x7eff300b6a10 2023-10-08T00:17:37.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.732+0000 7eff2f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff3014f980 0x7eff3015c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:37.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.732+0000 7eff2e7fc700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7eff30159ef0 0x7eff3009e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:37.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.732+0000 7eff2f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff3014f980 0x7eff3015c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60698/0 (socket says 172.21.15.99:60698) 2023-10-08T00:17:37.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.732+0000 7eff2f7fe700 1 -- 172.21.15.99:0/1753471377 learned_addr learned my addr 172.21.15.99:0/1753471377 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:37.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.732+0000 7eff2effd700 1 --2- 172.21.15.99:0/1753471377 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7eff300b6a10 0x7eff3014f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:37.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.732+0000 7eff2f7fe700 1 -- 172.21.15.99:0/1753471377 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7eff300b6a10 msgr2=0x7eff3014f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:37.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.732+0000 7eff2f7fe700 1 --2- 172.21.15.99:0/1753471377 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7eff300b6a10 0x7eff3014f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:37.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.732+0000 7eff2f7fe700 1 -- 172.21.15.99:0/1753471377 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7eff30159ef0 msgr2=0x7eff3009e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:37.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.732+0000 7eff2f7fe700 1 --2- 172.21.15.99:0/1753471377 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7eff30159ef0 0x7eff3009e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:37.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.732+0000 7eff2f7fe700 1 -- 172.21.15.99:0/1753471377 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff20009480 con 0x7eff3014f980 2023-10-08T00:17:37.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.732+0000 7eff2f7fe700 1 --2- 172.21.15.99:0/1753471377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff3014f980 0x7eff3015c8a0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7eff2800c8e0 tx=0x7eff28001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:37.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.733+0000 7eff16ffd700 1 -- 172.21.15.99:0/1753471377 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff28012630 con 0x7eff3014f980 2023-10-08T00:17:37.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.733+0000 7eff16ffd700 1 -- 172.21.15.99:0/1753471377 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff28012790 con 0x7eff3014f980 2023-10-08T00:17:37.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.733+0000 7eff35379700 1 -- 172.21.15.99:0/1753471377 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff30155e00 con 0x7eff3014f980 2023-10-08T00:17:37.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.733+0000 7eff16ffd700 1 -- 172.21.15.99:0/1753471377 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff28012b50 con 0x7eff3014f980 2023-10-08T00:17:37.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.733+0000 7eff35379700 1 -- 172.21.15.99:0/1753471377 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff30155080 con 0x7eff3014f980 2023-10-08T00:17:37.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.734+0000 7eff35379700 1 -- 172.21.15.99:0/1753471377 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff301562a0 con 0x7eff3014f980 2023-10-08T00:17:37.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.734+0000 7eff16ffd700 1 -- 172.21.15.99:0/1753471377 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7eff2801a3a0 con 0x7eff3014f980 2023-10-08T00:17:37.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.734+0000 7eff16ffd700 1 --2- 172.21.15.99:0/1753471377 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7eff1c066f30 0x7eff1c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:37.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.734+0000 7eff16ffd700 1 -- 172.21.15.99:0/1753471377 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 4853+0+0 (secure 0 0 0) 0x7eff28084410 con 0x7eff3014f980 2023-10-08T00:17:37.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.735+0000 7eff2effd700 1 --2- 172.21.15.99:0/1753471377 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7eff1c066f30 0x7eff1c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:37.738 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.735+0000 7eff2effd700 1 --2- 172.21.15.99:0/1753471377 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7eff1c066f30 0x7eff1c0693e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7eff20009260 tx=0x7eff20016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:37.739 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.739+0000 7eff16ffd700 1 -- 172.21.15.99:0/1753471377 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7eff301562a0 con 0x7eff3014f980 2023-10-08T00:17:37.919 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:37.918+0000 7eff35379700 1 -- 172.21.15.99:0/1753471377 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7eff301562a0 con 0x7eff3014f980 2023-10-08T00:17:38.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.227+0000 7eff16ffd700 1 -- 172.21.15.99:0/1753471377 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v151) v1 ==== 138+0+0 (secure 0 0 0) 0x7eff301562a0 con 0x7eff3014f980 2023-10-08T00:17:38.231 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.230+0000 7eff35379700 1 -- 172.21.15.99:0/1753471377 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7eff1c066f30 msgr2=0x7eff1c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:38.231 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.230+0000 7eff35379700 1 --2- 172.21.15.99:0/1753471377 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7eff1c066f30 0x7eff1c0693e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7eff20009260 tx=0x7eff20016000 comp rx=0 tx=0).stop 2023-10-08T00:17:38.231 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.230+0000 7eff35379700 1 -- 172.21.15.99:0/1753471377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff3014f980 msgr2=0x7eff3015c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:38.232 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.230+0000 7eff35379700 1 --2- 172.21.15.99:0/1753471377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff3014f980 0x7eff3015c8a0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7eff2800c8e0 tx=0x7eff28001710 comp rx=0 tx=0).stop 2023-10-08T00:17:38.232 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.230+0000 7eff35379700 1 -- 172.21.15.99:0/1753471377 shutdown_connections 2023-10-08T00:17:38.232 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.230+0000 7eff35379700 1 --2- 172.21.15.99:0/1753471377 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7eff1c066f30 0x7eff1c0693e0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:38.232 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.230+0000 7eff35379700 1 --2- 172.21.15.99:0/1753471377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff3014f980 0x7eff3015c8a0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:38.232 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.230+0000 7eff35379700 1 --2- 172.21.15.99:0/1753471377 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7eff300b6a10 0x7eff3014f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:38.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.230+0000 7eff35379700 1 --2- 172.21.15.99:0/1753471377 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7eff30159ef0 0x7eff3009e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:38.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.230+0000 7eff35379700 1 -- 172.21.15.99:0/1753471377 >> 172.21.15.99:0/1753471377 conn(0x7eff3009a0a0 msgr2=0x7eff30152420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:38.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.231+0000 7eff35379700 1 -- 172.21.15.99:0/1753471377 shutdown_connections 2023-10-08T00:17:38.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.231+0000 7eff35379700 1 -- 172.21.15.99:0/1753471377 wait complete. 2023-10-08T00:17:38.234 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' created 2023-10-08T00:17:38.244 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T00:17:38.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.365+0000 7ff0c81d5700 1 Processor -- start 2023-10-08T00:17:38.366 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.366+0000 7ff0c81d5700 1 -- start start 2023-10-08T00:17:38.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.366+0000 7ff0c81d5700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff0c00c9340 0x7ff0c00afa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:38.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.366+0000 7ff0c81d5700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff0c015b960 0x7ff0c01577a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:38.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.366+0000 7ff0c81d5700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0c00b0300 con 0x7ff0c00c9710 2023-10-08T00:17:38.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.366+0000 7ff0c81d5700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0c00cb320 con 0x7ff0c00c9340 2023-10-08T00:17:38.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.366+0000 7ff0c81d5700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0c00b05e0 con 0x7ff0c015b960 2023-10-08T00:17:38.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.367+0000 7ff0c5f71700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff0c00c9340 0x7ff0c00afa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:38.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.367+0000 7ff0c6772700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7ff0c00c9710 0x7ff0c0157ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:32816/0 (socket says 172.21.15.99:32816) 2023-10-08T00:17:38.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.367+0000 7ff0c6772700 1 -- 172.21.15.99:0/4062908152 learned_addr learned my addr 172.21.15.99:0/4062908152 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:17:38.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.367+0000 7ff0c5f71700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff0c00c9340 0x7ff0c00afa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60708/0 (socket says 172.21.15.99:60708) 2023-10-08T00:17:38.369 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.367+0000 7ff0c5f71700 1 -- 172.21.15.99:0/4062908152 >> v2:172.21.15.191:3300/0 conn(0x7ff0c015b960 msgr2=0x7ff0c01577a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:38.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.367+0000 7ff0c5f71700 1 --2- 172.21.15.99:0/4062908152 >> v2:172.21.15.191:3300/0 conn(0x7ff0c015b960 0x7ff0c01577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:38.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.367+0000 7ff0c5f71700 1 -- 172.21.15.99:0/4062908152 >> v1:172.21.15.99:6789/0 conn(0x7ff0c00c9710 legacy=0x7ff0c0157ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:38.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.367+0000 7ff0c5f71700 1 -- 172.21.15.99:0/4062908152 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c00b08c0 con 0x7ff0c00c9340 2023-10-08T00:17:38.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.367+0000 7ff0c4f6f700 1 -- 172.21.15.99:0/4062908152 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1305483134 0 0) 0x7ff0c00b0300 con 0x7ff0c00c9710 2023-10-08T00:17:38.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.367+0000 7ff0c5f71700 1 --2- 172.21.15.99:0/4062908152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0c00c9340 0x7ff0c00afa60 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7ff0c006a870 tx=0x7ff0bc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=286567f01180881c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:38.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.367+0000 7ff0c4f6f700 1 -- 172.21.15.99:0/4062908152 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc00b6b0 con 0x7ff0c00c9340 2023-10-08T00:17:38.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.367+0000 7ff0c4f6f700 1 -- 172.21.15.99:0/4062908152 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0bc00b810 con 0x7ff0c00c9340 2023-10-08T00:17:38.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.367+0000 7ff0c4f6f700 1 -- 172.21.15.99:0/4062908152 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc00bbb0 con 0x7ff0c00c9340 2023-10-08T00:17:38.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.368+0000 7ff0c81d5700 1 -- 172.21.15.99:0/4062908152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0c00c9340 msgr2=0x7ff0c00afa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:38.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.368+0000 7ff0c81d5700 1 --2- 172.21.15.99:0/4062908152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0c00c9340 0x7ff0c00afa60 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7ff0c006a870 tx=0x7ff0bc0091a0 comp rx=0 tx=0).stop 2023-10-08T00:17:38.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.368+0000 7ff0c81d5700 1 -- 172.21.15.99:0/4062908152 shutdown_connections 2023-10-08T00:17:38.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.368+0000 7ff0c81d5700 1 --2- 172.21.15.99:0/4062908152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0c00c9340 0x7ff0c00afa60 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:38.371 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.368+0000 7ff0c81d5700 1 --2- 172.21.15.99:0/4062908152 >> v2:172.21.15.191:3300/0 conn(0x7ff0c015b960 0x7ff0c01577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:38.371 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.368+0000 7ff0c81d5700 1 -- 172.21.15.99:0/4062908152 >> 172.21.15.99:0/4062908152 conn(0x7ff0c0069870 msgr2=0x7ff0c0069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:38.371 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.368+0000 7ff0c81d5700 1 -- 172.21.15.99:0/4062908152 shutdown_connections 2023-10-08T00:17:38.371 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.368+0000 7ff0c81d5700 1 -- 172.21.15.99:0/4062908152 wait complete. 2023-10-08T00:17:38.373 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.369+0000 7ff0c81d5700 1 Processor -- start 2023-10-08T00:17:38.373 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.369+0000 7ff0c81d5700 1 -- start start 2023-10-08T00:17:38.373 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.370+0000 7ff0c81d5700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0c00c9340 0x7ff0c00c4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:38.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.370+0000 7ff0c81d5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0c00c9710 0x7ff0c01525f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:38.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.370+0000 7ff0c81d5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0c015b960 0x7ff0c00c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:38.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.370+0000 7ff0c81d5700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c00b37d0 con 0x7ff0c00c9710 2023-10-08T00:17:38.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.370+0000 7ff0c81d5700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c0152b30 con 0x7ff0c00c9340 2023-10-08T00:17:38.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.370+0000 7ff0c81d5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c00b2bc0 con 0x7ff0c015b960 2023-10-08T00:17:38.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.370+0000 7ff0c5770700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0c00c9710 0x7ff0c01525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:38.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.370+0000 7ff0c5770700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0c00c9710 0x7ff0c01525f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60716/0 (socket says 172.21.15.99:60716) 2023-10-08T00:17:38.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.370+0000 7ff0c5770700 1 -- 172.21.15.99:0/19140843 learned_addr learned my addr 172.21.15.99:0/19140843 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:38.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.370+0000 7ff0c5770700 1 -- 172.21.15.99:0/19140843 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0c015b960 msgr2=0x7ff0c00c82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:38.374 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.370+0000 7ff0c5f71700 1 --2- 172.21.15.99:0/19140843 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0c00c9340 0x7ff0c00c4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:38.375 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.370+0000 7ff0c6772700 1 --2- 172.21.15.99:0/19140843 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0c015b960 0x7ff0c00c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:38.375 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.370+0000 7ff0c5770700 1 --2- 172.21.15.99:0/19140843 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0c015b960 0x7ff0c00c82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:38.375 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.370+0000 7ff0c5770700 1 -- 172.21.15.99:0/19140843 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0c00c9340 msgr2=0x7ff0c00c4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:38.375 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.370+0000 7ff0c5770700 1 --2- 172.21.15.99:0/19140843 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0c00c9340 0x7ff0c00c4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:38.375 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.370+0000 7ff0c5770700 1 -- 172.21.15.99:0/19140843 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0bc009420 con 0x7ff0c00c9710 2023-10-08T00:17:38.375 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.370+0000 7ff0c5770700 1 --2- 172.21.15.99:0/19140843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0c00c9710 0x7ff0c01525f0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7ff0b000b840 tx=0x7ff0b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:38.375 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.370+0000 7ff0b5ffb700 1 -- 172.21.15.99:0/19140843 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b000f590 con 0x7ff0c00c9710 2023-10-08T00:17:38.375 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.370+0000 7ff0b5ffb700 1 -- 172.21.15.99:0/19140843 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b000f6f0 con 0x7ff0c00c9710 2023-10-08T00:17:38.375 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.370+0000 7ff0b5ffb700 1 -- 172.21.15.99:0/19140843 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b000fab0 con 0x7ff0c00c9710 2023-10-08T00:17:38.375 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.370+0000 7ff0c81d5700 1 -- 172.21.15.99:0/19140843 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0c00b2ea0 con 0x7ff0c00c9710 2023-10-08T00:17:38.375 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.370+0000 7ff0c81d5700 1 -- 172.21.15.99:0/19140843 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0c00b3460 con 0x7ff0c00c9710 2023-10-08T00:17:38.375 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.371+0000 7ff0c81d5700 1 -- 172.21.15.99:0/19140843 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0c00b3980 con 0x7ff0c00c9710 2023-10-08T00:17:38.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.372+0000 7ff0b5ffb700 1 -- 172.21.15.99:0/19140843 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff0b001a460 con 0x7ff0c00c9710 2023-10-08T00:17:38.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.372+0000 7ff0b5ffb700 1 --2- 172.21.15.99:0/19140843 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff0a4066f30 0x7ff0a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:38.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.372+0000 7ff0b5ffb700 1 -- 172.21.15.99:0/19140843 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5223+0+0 (secure 0 0 0) 0x7ff0b00840c0 con 0x7ff0c00c9710 2023-10-08T00:17:38.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.372+0000 7ff0c5f71700 1 --2- 172.21.15.99:0/19140843 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff0a4066f30 0x7ff0a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:38.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.373+0000 7ff0c5f71700 1 --2- 172.21.15.99:0/19140843 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff0a4066f30 0x7ff0a40693e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc00c010 tx=0x7ff0bc017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:38.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.377+0000 7ff0b5ffb700 1 -- 172.21.15.99:0/19140843 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff0c00b3980 con 0x7ff0c00c9710 2023-10-08T00:17:38.559 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:38.558+0000 7ff0c81d5700 1 -- 172.21.15.99:0/19140843 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff0c00b3980 con 0x7ff0c00c9710 2023-10-08T00:17:39.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.234+0000 7ff0b5ffb700 1 -- 172.21.15.99:0/19140843 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v152) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff0c00b3980 con 0x7ff0c00c9710 2023-10-08T00:17:39.242 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.241+0000 7ff0c81d5700 1 -- 172.21.15.99:0/19140843 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff0a4066f30 msgr2=0x7ff0a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:39.242 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.241+0000 7ff0c81d5700 1 --2- 172.21.15.99:0/19140843 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff0a4066f30 0x7ff0a40693e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc00c010 tx=0x7ff0bc017000 comp rx=0 tx=0).stop 2023-10-08T00:17:39.242 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.241+0000 7ff0c81d5700 1 -- 172.21.15.99:0/19140843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0c00c9710 msgr2=0x7ff0c01525f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:39.242 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.241+0000 7ff0c81d5700 1 --2- 172.21.15.99:0/19140843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0c00c9710 0x7ff0c01525f0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7ff0b000b840 tx=0x7ff0b0009250 comp rx=0 tx=0).stop 2023-10-08T00:17:39.242 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.241+0000 7ff0c81d5700 1 -- 172.21.15.99:0/19140843 shutdown_connections 2023-10-08T00:17:39.242 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.241+0000 7ff0c81d5700 1 --2- 172.21.15.99:0/19140843 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff0a4066f30 0x7ff0a40693e0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:39.242 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.241+0000 7ff0c81d5700 1 --2- 172.21.15.99:0/19140843 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0c00c9340 0x7ff0c00c4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:39.243 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.241+0000 7ff0c81d5700 1 --2- 172.21.15.99:0/19140843 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0c015b960 0x7ff0c00c82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:39.243 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.241+0000 7ff0c81d5700 1 --2- 172.21.15.99:0/19140843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0c00c9710 0x7ff0c01525f0 secure :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7ff0b000b840 tx=0x7ff0b0009250 comp rx=0 tx=0).stop 2023-10-08T00:17:39.243 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.241+0000 7ff0c81d5700 1 -- 172.21.15.99:0/19140843 >> 172.21.15.99:0/19140843 conn(0x7ff0c0069870 msgr2=0x7ff0c0153440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:39.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.248+0000 7ff0c81d5700 1 -- 172.21.15.99:0/19140843 shutdown_connections 2023-10-08T00:17:39.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.248+0000 7ff0c81d5700 1 -- 172.21.15.99:0/19140843 wait complete. 2023-10-08T00:17:39.249 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' created 2023-10-08T00:17:39.262 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T00:17:39.385 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.384+0000 7f3d6e6b0700 1 Processor -- start 2023-10-08T00:17:39.385 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.384+0000 7f3d6e6b0700 1 -- start start 2023-10-08T00:17:39.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.385+0000 7f3d6e6b0700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f3d680678b0 0x7f3d6815db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:39.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.385+0000 7f3d6e6b0700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3d680cbf10 0x7f3d6815e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:39.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.385+0000 7f3d6e6b0700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d680cae70 con 0x7f3d680c9390 2023-10-08T00:17:39.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.385+0000 7f3d6e6b0700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d680cb150 con 0x7f3d680cbf10 2023-10-08T00:17:39.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.385+0000 7f3d6e6b0700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d680cb430 con 0x7f3d680678b0 2023-10-08T00:17:39.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.385+0000 7f3d677fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3d680cbf10 0x7f3d6815e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:39.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.385+0000 7f3d677fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3d680cbf10 0x7f3d6815e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60732/0 (socket says 172.21.15.99:60732) 2023-10-08T00:17:39.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.385+0000 7f3d677fe700 1 -- 172.21.15.99:0/66424517 learned_addr learned my addr 172.21.15.99:0/66424517 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:39.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.385+0000 7f3d67fff700 1 --2- 172.21.15.99:0/66424517 >> v2:172.21.15.191:3300/0 conn(0x7f3d680678b0 0x7f3d6815db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:39.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.386+0000 7f3d677fe700 1 -- 172.21.15.99:0/66424517 >> v2:172.21.15.191:3300/0 conn(0x7f3d680678b0 msgr2=0x7f3d6815db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:39.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.386+0000 7f3d677fe700 1 --2- 172.21.15.99:0/66424517 >> v2:172.21.15.191:3300/0 conn(0x7f3d680678b0 0x7f3d6815db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:39.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.386+0000 7f3d677fe700 1 -- 172.21.15.99:0/66424517 >> v1:172.21.15.191:6789/0 conn(0x7f3d680c9390 legacy=0x7f3d6806bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:39.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.386+0000 7f3d677fe700 1 -- 172.21.15.99:0/66424517 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d680cb710 con 0x7f3d680cbf10 2023-10-08T00:17:39.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.386+0000 7f3d677fe700 1 --2- 172.21.15.99:0/66424517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d680cbf10 0x7f3d6815e0c0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f3d680c5b10 tx=0x7f3d5c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b7e3d844bfa5cb8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:39.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.386+0000 7f3d66ffd700 1 -- 172.21.15.99:0/66424517 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c00b860 con 0x7f3d680cbf10 2023-10-08T00:17:39.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.386+0000 7f3d66ffd700 1 -- 172.21.15.99:0/66424517 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d5c00b9c0 con 0x7f3d680cbf10 2023-10-08T00:17:39.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.386+0000 7f3d66ffd700 1 -- 172.21.15.99:0/66424517 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c00bd60 con 0x7f3d680cbf10 2023-10-08T00:17:39.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.386+0000 7f3d6e6b0700 1 -- 172.21.15.99:0/66424517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d680cbf10 msgr2=0x7f3d6815e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:39.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.386+0000 7f3d6e6b0700 1 --2- 172.21.15.99:0/66424517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d680cbf10 0x7f3d6815e0c0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f3d680c5b10 tx=0x7f3d5c009250 comp rx=0 tx=0).stop 2023-10-08T00:17:39.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.386+0000 7f3d6e6b0700 1 -- 172.21.15.99:0/66424517 shutdown_connections 2023-10-08T00:17:39.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.386+0000 7f3d6e6b0700 1 --2- 172.21.15.99:0/66424517 >> v2:172.21.15.191:3300/0 conn(0x7f3d680678b0 0x7f3d6815db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:39.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.386+0000 7f3d6e6b0700 1 --2- 172.21.15.99:0/66424517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d680cbf10 0x7f3d6815e0c0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:39.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.386+0000 7f3d6e6b0700 1 -- 172.21.15.99:0/66424517 >> 172.21.15.99:0/66424517 conn(0x7f3d68069870 msgr2=0x7f3d68069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:39.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.387+0000 7f3d6e6b0700 1 -- 172.21.15.99:0/66424517 shutdown_connections 2023-10-08T00:17:39.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.387+0000 7f3d6e6b0700 1 -- 172.21.15.99:0/66424517 wait complete. 2023-10-08T00:17:39.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.388+0000 7f3d6e6b0700 1 Processor -- start 2023-10-08T00:17:39.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.388+0000 7f3d6e6b0700 1 -- start start 2023-10-08T00:17:39.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.388+0000 7f3d6e6b0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d680678b0 0x7f3d6814fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:39.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.389+0000 7f3d6e6b0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d680c9390 0x7f3d680c8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:39.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.389+0000 7f3d6e6b0700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d680cbf10 0x7f3d68155640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:39.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.389+0000 7f3d6e6b0700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6816ad70 con 0x7f3d680678b0 2023-10-08T00:17:39.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.389+0000 7f3d6e6b0700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6816a160 con 0x7f3d680cbf10 2023-10-08T00:17:39.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.389+0000 7f3d6e6b0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6816a440 con 0x7f3d680c9390 2023-10-08T00:17:39.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.389+0000 7f3d67fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d680678b0 0x7f3d6814fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:39.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.389+0000 7f3d67fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d680678b0 0x7f3d6814fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60734/0 (socket says 172.21.15.99:60734) 2023-10-08T00:17:39.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.389+0000 7f3d67fff700 1 -- 172.21.15.99:0/3750720374 learned_addr learned my addr 172.21.15.99:0/3750720374 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:39.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.389+0000 7f3d6cc4d700 1 --2- 172.21.15.99:0/3750720374 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d680cbf10 0x7f3d68155640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:39.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.389+0000 7f3d677fe700 1 --2- 172.21.15.99:0/3750720374 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d680c9390 0x7f3d680c8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:39.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.389+0000 7f3d67fff700 1 -- 172.21.15.99:0/3750720374 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d680c9390 msgr2=0x7f3d680c8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:39.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.389+0000 7f3d67fff700 1 --2- 172.21.15.99:0/3750720374 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d680c9390 0x7f3d680c8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:39.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.389+0000 7f3d67fff700 1 -- 172.21.15.99:0/3750720374 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d680cbf10 msgr2=0x7f3d68155640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:39.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.389+0000 7f3d67fff700 1 --2- 172.21.15.99:0/3750720374 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d680cbf10 0x7f3d68155640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:39.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.389+0000 7f3d67fff700 1 -- 172.21.15.99:0/3750720374 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d5c0094f0 con 0x7f3d680678b0 2023-10-08T00:17:39.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.389+0000 7f3d67fff700 1 --2- 172.21.15.99:0/3750720374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d680678b0 0x7f3d6814fd20 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f3d5800d860 tx=0x7f3d58008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:39.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.389+0000 7f3d4ffff700 1 -- 172.21.15.99:0/3750720374 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d580125f0 con 0x7f3d680678b0 2023-10-08T00:17:39.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.389+0000 7f3d4ffff700 1 -- 172.21.15.99:0/3750720374 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d58012770 con 0x7f3d680678b0 2023-10-08T00:17:39.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.389+0000 7f3d4ffff700 1 -- 172.21.15.99:0/3750720374 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d58012b30 con 0x7f3d680678b0 2023-10-08T00:17:39.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.389+0000 7f3d6e6b0700 1 -- 172.21.15.99:0/3750720374 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d6816a720 con 0x7f3d680678b0 2023-10-08T00:17:39.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.389+0000 7f3d6e6b0700 1 -- 172.21.15.99:0/3750720374 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d680c7090 con 0x7f3d680678b0 2023-10-08T00:17:39.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.390+0000 7f3d6e6b0700 1 -- 172.21.15.99:0/3750720374 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d6816abc0 con 0x7f3d680678b0 2023-10-08T00:17:39.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.391+0000 7f3d4ffff700 1 -- 172.21.15.99:0/3750720374 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3d58013480 con 0x7f3d680678b0 2023-10-08T00:17:39.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.391+0000 7f3d4ffff700 1 --2- 172.21.15.99:0/3750720374 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3d54066f30 0x7f3d540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:39.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.391+0000 7f3d4ffff700 1 -- 172.21.15.99:0/3750720374 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f3d580852f0 con 0x7f3d680678b0 2023-10-08T00:17:39.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.391+0000 7f3d677fe700 1 --2- 172.21.15.99:0/3750720374 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3d54066f30 0x7f3d540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:39.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.392+0000 7f3d677fe700 1 --2- 172.21.15.99:0/3750720374 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3d54066f30 0x7f3d540693e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f3d680c5b10 tx=0x7f3d5c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:39.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.395+0000 7f3d4ffff700 1 -- 172.21.15.99:0/3750720374 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3d580526f0 con 0x7f3d680678b0 2023-10-08T00:17:39.622 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:39.622+0000 7f3d6e6b0700 1 -- 172.21.15.99:0/3750720374 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f3d6816b050 con 0x7f3d680678b0 2023-10-08T00:17:40.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.245+0000 7f3d4ffff700 1 -- 172.21.15.99:0/3750720374 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v67) v1 ==== 174+0+0 (secure 0 0 0) 0x7f3d58055d10 con 0x7f3d680678b0 2023-10-08T00:17:40.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.253+0000 7f3d6e6b0700 1 -- 172.21.15.99:0/3750720374 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3d54066f30 msgr2=0x7f3d540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:40.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.253+0000 7f3d6e6b0700 1 --2- 172.21.15.99:0/3750720374 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3d54066f30 0x7f3d540693e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f3d680c5b10 tx=0x7f3d5c017000 comp rx=0 tx=0).stop 2023-10-08T00:17:40.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.253+0000 7f3d6e6b0700 1 -- 172.21.15.99:0/3750720374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d680678b0 msgr2=0x7f3d6814fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:40.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.253+0000 7f3d6e6b0700 1 --2- 172.21.15.99:0/3750720374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d680678b0 0x7f3d6814fd20 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f3d5800d860 tx=0x7f3d58008040 comp rx=0 tx=0).stop 2023-10-08T00:17:40.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.254+0000 7f3d6e6b0700 1 -- 172.21.15.99:0/3750720374 shutdown_connections 2023-10-08T00:17:40.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.254+0000 7f3d6e6b0700 1 --2- 172.21.15.99:0/3750720374 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3d54066f30 0x7f3d540693e0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:40.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.254+0000 7f3d6e6b0700 1 --2- 172.21.15.99:0/3750720374 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d680cbf10 0x7f3d68155640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:40.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.254+0000 7f3d6e6b0700 1 --2- 172.21.15.99:0/3750720374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d680678b0 0x7f3d6814fd20 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:40.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.254+0000 7f3d6e6b0700 1 --2- 172.21.15.99:0/3750720374 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d680c9390 0x7f3d680c8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:40.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.254+0000 7f3d6e6b0700 1 -- 172.21.15.99:0/3750720374 >> 172.21.15.99:0/3750720374 conn(0x7f3d68069870 msgr2=0x7f3d680c4e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:40.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.254+0000 7f3d6e6b0700 1 -- 172.21.15.99:0/3750720374 shutdown_connections 2023-10-08T00:17:40.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.254+0000 7f3d6e6b0700 1 -- 172.21.15.99:0/3750720374 wait complete. 2023-10-08T00:17:40.258 INFO:teuthology.orchestra.run.smithi099.stderr:new fs with metadata pool 18 and data pool 19 2023-10-08T00:17:40.270 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:17:40.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.385+0000 7f88e188c700 1 Processor -- start 2023-10-08T00:17:40.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.385+0000 7f88e188c700 1 -- start start 2023-10-08T00:17:40.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.386+0000 7f88e188c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f88dc0b0e50 0x7f88dc0c9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:40.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.386+0000 7f88e188c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f88dc0b1220 0x7f88dc06d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:40.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.386+0000 7f88e188c700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88dc15de60 con 0x7f88dc06dad0 2023-10-08T00:17:40.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.386+0000 7f88e188c700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88dc15e140 con 0x7f88dc0b0e50 2023-10-08T00:17:40.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.386+0000 7f88e188c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88dc15e420 con 0x7f88dc0b1220 2023-10-08T00:17:40.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.387+0000 7f88d27fc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f88dc0b1220 0x7f88dc06d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:40.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.387+0000 7f88d27fc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f88dc0b1220 0x7f88dc06d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.99:58794/0 (socket says 172.21.15.99:58794) 2023-10-08T00:17:40.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.387+0000 7f88d27fc700 1 -- 172.21.15.99:0/2426693988 learned_addr learned my addr 172.21.15.99:0/2426693988 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:40.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.387+0000 7f88daffd700 1 --2- 172.21.15.99:0/2426693988 >> v2:172.21.15.191:3300/0 conn(0x7f88dc0b0e50 0x7f88dc0c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:40.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.387+0000 7f88d27fc700 1 -- 172.21.15.99:0/2426693988 >> v2:172.21.15.191:3300/0 conn(0x7f88dc0b0e50 msgr2=0x7f88dc0c9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:40.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.387+0000 7f88d27fc700 1 --2- 172.21.15.99:0/2426693988 >> v2:172.21.15.191:3300/0 conn(0x7f88dc0b0e50 0x7f88dc0c9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:40.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.387+0000 7f88d27fc700 1 -- 172.21.15.99:0/2426693988 >> v1:172.21.15.191:6789/0 conn(0x7f88dc06dad0 legacy=0x7f88dc0cb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:40.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.387+0000 7f88d27fc700 1 -- 172.21.15.99:0/2426693988 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88dc15e700 con 0x7f88dc0b1220 2023-10-08T00:17:40.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.387+0000 7f88d27fc700 1 --2- 172.21.15.99:0/2426693988 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f88dc0b1220 0x7f88dc06d590 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f88c40099e0 tx=0x7f88c4009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9bd416c8933412b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:40.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.387+0000 7f88da7fc700 1 -- 172.21.15.99:0/2426693988 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c400b820 con 0x7f88dc0b1220 2023-10-08T00:17:40.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.388+0000 7f88da7fc700 1 -- 172.21.15.99:0/2426693988 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88c400b980 con 0x7f88dc0b1220 2023-10-08T00:17:40.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.388+0000 7f88da7fc700 1 -- 172.21.15.99:0/2426693988 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c400bd40 con 0x7f88dc0b1220 2023-10-08T00:17:40.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.388+0000 7f88e188c700 1 -- 172.21.15.99:0/2426693988 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f88dc0b1220 msgr2=0x7f88dc06d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:40.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.388+0000 7f88e188c700 1 --2- 172.21.15.99:0/2426693988 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f88dc0b1220 0x7f88dc06d590 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f88c40099e0 tx=0x7f88c4009250 comp rx=0 tx=0).stop 2023-10-08T00:17:40.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.388+0000 7f88e188c700 1 -- 172.21.15.99:0/2426693988 shutdown_connections 2023-10-08T00:17:40.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.388+0000 7f88e188c700 1 --2- 172.21.15.99:0/2426693988 >> v2:172.21.15.191:3300/0 conn(0x7f88dc0b0e50 0x7f88dc0c9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:40.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.388+0000 7f88e188c700 1 --2- 172.21.15.99:0/2426693988 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f88dc0b1220 0x7f88dc06d590 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:40.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.388+0000 7f88e188c700 1 -- 172.21.15.99:0/2426693988 >> 172.21.15.99:0/2426693988 conn(0x7f88dc069870 msgr2=0x7f88dc069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:40.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.388+0000 7f88e188c700 1 -- 172.21.15.99:0/2426693988 shutdown_connections 2023-10-08T00:17:40.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.388+0000 7f88e188c700 1 -- 172.21.15.99:0/2426693988 wait complete. 2023-10-08T00:17:40.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.389+0000 7f88e188c700 1 Processor -- start 2023-10-08T00:17:40.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.389+0000 7f88e188c700 1 -- start start 2023-10-08T00:17:40.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.390+0000 7f88e188c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88dc06dad0 0x7f88dc0b2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:40.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.390+0000 7f88e188c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88dc0b0e50 0x7f88dc0b4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:40.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.390+0000 7f88e188c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f88dc0b1220 0x7f88dc0caa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:40.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.390+0000 7f88e188c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88dc0b23b0 con 0x7f88dc06dad0 2023-10-08T00:17:40.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.390+0000 7f88e188c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88dc0c6850 con 0x7f88dc0b0e50 2023-10-08T00:17:40.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.390+0000 7f88e188c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88dc0b1a80 con 0x7f88dc0b1220 2023-10-08T00:17:40.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.390+0000 7f88daffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88dc06dad0 0x7f88dc0b2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:40.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.390+0000 7f88d27fc700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88dc0b0e50 0x7f88dc0b4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:40.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.390+0000 7f88db7fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f88dc0b1220 0x7f88dc0caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:40.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.390+0000 7f88daffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88dc06dad0 0x7f88dc0b2840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60744/0 (socket says 172.21.15.99:60744) 2023-10-08T00:17:40.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.390+0000 7f88d27fc700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88dc0b0e50 0x7f88dc0b4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.99:43286/0 (socket says 172.21.15.99:43286) 2023-10-08T00:17:40.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.390+0000 7f88daffd700 1 -- 172.21.15.99:0/3016514730 learned_addr learned my addr 172.21.15.99:0/3016514730 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:40.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.391+0000 7f88daffd700 1 -- 172.21.15.99:0/3016514730 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f88dc0b1220 msgr2=0x7f88dc0caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:40.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.391+0000 7f88daffd700 1 --2- 172.21.15.99:0/3016514730 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f88dc0b1220 0x7f88dc0caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:40.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.391+0000 7f88daffd700 1 -- 172.21.15.99:0/3016514730 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88dc0b0e50 msgr2=0x7f88dc0b4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:40.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.391+0000 7f88daffd700 1 --2- 172.21.15.99:0/3016514730 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88dc0b0e50 0x7f88dc0b4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:40.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.391+0000 7f88daffd700 1 -- 172.21.15.99:0/3016514730 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c40094f0 con 0x7f88dc06dad0 2023-10-08T00:17:40.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.391+0000 7f88daffd700 1 --2- 172.21.15.99:0/3016514730 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88dc06dad0 0x7f88dc0b2840 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f88cc00ec80 tx=0x7f88cc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:40.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.391+0000 7f88d37fe700 1 -- 172.21.15.99:0/3016514730 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88cc00c850 con 0x7f88dc06dad0 2023-10-08T00:17:40.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.391+0000 7f88d37fe700 1 -- 172.21.15.99:0/3016514730 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88cc00c9b0 con 0x7f88dc06dad0 2023-10-08T00:17:40.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.391+0000 7f88e188c700 1 -- 172.21.15.99:0/3016514730 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88dc0b1d60 con 0x7f88dc06dad0 2023-10-08T00:17:40.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.391+0000 7f88d37fe700 1 -- 172.21.15.99:0/3016514730 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88cc00cd50 con 0x7f88dc06dad0 2023-10-08T00:17:40.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.391+0000 7f88e188c700 1 -- 172.21.15.99:0/3016514730 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88dc0b4850 con 0x7f88dc06dad0 2023-10-08T00:17:40.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.392+0000 7f88e188c700 1 -- 172.21.15.99:0/3016514730 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88dc069c40 con 0x7f88dc06dad0 2023-10-08T00:17:40.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.392+0000 7f88d37fe700 1 -- 172.21.15.99:0/3016514730 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f88cc013680 con 0x7f88dc06dad0 2023-10-08T00:17:40.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.393+0000 7f88d37fe700 1 --2- 172.21.15.99:0/3016514730 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f88bc066f30 0x7f88bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:40.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.393+0000 7f88d37fe700 1 -- 172.21.15.99:0/3016514730 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f88cc084760 con 0x7f88dc06dad0 2023-10-08T00:17:40.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.393+0000 7f88d27fc700 1 --2- 172.21.15.99:0/3016514730 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f88bc066f30 0x7f88bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:40.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.393+0000 7f88d27fc700 1 --2- 172.21.15.99:0/3016514730 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f88bc066f30 0x7f88bc0693e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f88c40099e0 tx=0x7f88c4017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:40.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.397+0000 7f88d37fe700 1 -- 172.21.15.99:0/3016514730 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f88cc0519d0 con 0x7f88dc06dad0 2023-10-08T00:17:40.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.582+0000 7f88e188c700 1 -- 172.21.15.99:0/3016514730 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f88dc0b2690 con 0x7f88dc06dad0 2023-10-08T00:17:40.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.583+0000 7f88d37fe700 1 -- 172.21.15.99:0/3016514730 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v153) v1 ==== 74+0+15378 (secure 0 0 0) 0x7f88cc054ff0 con 0x7f88dc06dad0 2023-10-08T00:17:40.584 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:17:40.584 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":153,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:17:39.623421+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:17:37.936880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-08T00:17:38.577281+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:17:40.586 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.585+0000 7f88e188c700 1 -- 172.21.15.99:0/3016514730 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f88bc066f30 msgr2=0x7f88bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:40.587 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.585+0000 7f88e188c700 1 --2- 172.21.15.99:0/3016514730 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f88bc066f30 0x7f88bc0693e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f88c40099e0 tx=0x7f88c4017000 comp rx=0 tx=0).stop 2023-10-08T00:17:40.587 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.585+0000 7f88e188c700 1 -- 172.21.15.99:0/3016514730 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88dc06dad0 msgr2=0x7f88dc0b2840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:40.587 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.585+0000 7f88e188c700 1 --2- 172.21.15.99:0/3016514730 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88dc06dad0 0x7f88dc0b2840 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f88cc00ec80 tx=0x7f88cc0091a0 comp rx=0 tx=0).stop 2023-10-08T00:17:40.587 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.585+0000 7f88e188c700 1 -- 172.21.15.99:0/3016514730 shutdown_connections 2023-10-08T00:17:40.587 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.585+0000 7f88e188c700 1 --2- 172.21.15.99:0/3016514730 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f88bc066f30 0x7f88bc0693e0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:40.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.585+0000 7f88e188c700 1 --2- 172.21.15.99:0/3016514730 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f88dc0b1220 0x7f88dc0caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:40.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.585+0000 7f88e188c700 1 --2- 172.21.15.99:0/3016514730 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88dc06dad0 0x7f88dc0b2840 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:40.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.585+0000 7f88e188c700 1 --2- 172.21.15.99:0/3016514730 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88dc0b0e50 0x7f88dc0b4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:40.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.585+0000 7f88e188c700 1 -- 172.21.15.99:0/3016514730 >> 172.21.15.99:0/3016514730 conn(0x7f88dc069870 msgr2=0x7f88dc1514b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:40.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.585+0000 7f88e188c700 1 -- 172.21.15.99:0/3016514730 shutdown_connections 2023-10-08T00:17:40.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.585+0000 7f88e188c700 1 -- 172.21.15.99:0/3016514730 wait complete. 2023-10-08T00:17:40.598 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:17:40.712 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.712+0000 7f7ae2375700 1 Processor -- start 2023-10-08T00:17:40.713 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.712+0000 7f7ae2375700 1 -- start start 2023-10-08T00:17:40.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.712+0000 7f7ae2375700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7adc0b1220 0x7f7adc0c9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:40.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.712+0000 7f7ae2375700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7adc0cb4f0 0x7f7adc0cb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:40.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.712+0000 7f7ae2375700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7adc15de60 con 0x7f7adc0b0e50 2023-10-08T00:17:40.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.712+0000 7f7ae2375700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7adc0cbed0 con 0x7f7adc0b1220 2023-10-08T00:17:40.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.712+0000 7f7ae2375700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7adc15e140 con 0x7f7adc0cb4f0 2023-10-08T00:17:40.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.713+0000 7f7adb7fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7adc0b1220 0x7f7adc0c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:40.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.713+0000 7f7adb7fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7adc0b1220 0x7f7adc0c9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60758/0 (socket says 172.21.15.99:60758) 2023-10-08T00:17:40.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.713+0000 7f7adb7fe700 1 -- 172.21.15.99:0/2460605306 learned_addr learned my addr 172.21.15.99:0/2460605306 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:40.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.713+0000 7f7ae0912700 1 --2- 172.21.15.99:0/2460605306 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7adc0cb4f0 0x7f7adc0cb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:40.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.713+0000 7f7adb7fe700 1 -- 172.21.15.99:0/2460605306 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7adc0cb4f0 msgr2=0x7f7adc0cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:40.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.713+0000 7f7adb7fe700 1 --2- 172.21.15.99:0/2460605306 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7adc0cb4f0 0x7f7adc0cb990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:40.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.713+0000 7f7adb7fe700 1 -- 172.21.15.99:0/2460605306 >> v1:172.21.15.191:6789/0 conn(0x7f7adc0b0e50 legacy=0x7f7adc06d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:40.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.713+0000 7f7adb7fe700 1 -- 172.21.15.99:0/2460605306 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7adc15e420 con 0x7f7adc0b1220 2023-10-08T00:17:40.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.713+0000 7f7adb7fe700 1 --2- 172.21.15.99:0/2460605306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7adc0b1220 0x7f7adc0c9180 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f7ad00099b0 tx=0x7f7ad0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7549d58d2fa0e3f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:40.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.713+0000 7f7adaffd700 1 -- 172.21.15.99:0/2460605306 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad000b820 con 0x7f7adc0b1220 2023-10-08T00:17:40.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.713+0000 7f7adaffd700 1 -- 172.21.15.99:0/2460605306 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad000b980 con 0x7f7adc0b1220 2023-10-08T00:17:40.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.713+0000 7f7adaffd700 1 -- 172.21.15.99:0/2460605306 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad000bd40 con 0x7f7adc0b1220 2023-10-08T00:17:40.724 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.714+0000 7f7ae2375700 1 -- 172.21.15.99:0/2460605306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7adc0b1220 msgr2=0x7f7adc0c9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:40.724 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.714+0000 7f7ae2375700 1 --2- 172.21.15.99:0/2460605306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7adc0b1220 0x7f7adc0c9180 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f7ad00099b0 tx=0x7f7ad0009250 comp rx=0 tx=0).stop 2023-10-08T00:17:40.724 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.714+0000 7f7ae2375700 1 -- 172.21.15.99:0/2460605306 shutdown_connections 2023-10-08T00:17:40.724 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.714+0000 7f7ae2375700 1 --2- 172.21.15.99:0/2460605306 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7adc0cb4f0 0x7f7adc0cb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:40.724 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.714+0000 7f7ae2375700 1 --2- 172.21.15.99:0/2460605306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7adc0b1220 0x7f7adc0c9180 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:40.724 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.714+0000 7f7ae2375700 1 -- 172.21.15.99:0/2460605306 >> 172.21.15.99:0/2460605306 conn(0x7f7adc069870 msgr2=0x7f7adc069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:40.724 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.714+0000 7f7ae2375700 1 -- 172.21.15.99:0/2460605306 shutdown_connections 2023-10-08T00:17:40.724 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.714+0000 7f7ae2375700 1 -- 172.21.15.99:0/2460605306 wait complete. 2023-10-08T00:17:40.724 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.715+0000 7f7ae2375700 1 Processor -- start 2023-10-08T00:17:40.724 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.715+0000 7f7ae2375700 1 -- start start 2023-10-08T00:17:40.724 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.716+0000 7f7ae2375700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7adc0b0e50 0x7f7adc151790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:40.725 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.716+0000 7f7ae2375700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7adc0b1220 0x7f7adc0ca190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:40.725 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.716+0000 7f7ae2375700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7adc0cb4f0 0x7f7adc0ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:40.725 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.716+0000 7f7ae2375700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7adc0b2630 con 0x7f7adc0b1220 2023-10-08T00:17:40.725 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.716+0000 7f7ae2375700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7adc0b4660 con 0x7f7adc0cb4f0 2023-10-08T00:17:40.725 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.716+0000 7f7ae2375700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7adc0b3d30 con 0x7f7adc0b0e50 2023-10-08T00:17:40.725 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.716+0000 7f7adb7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7adc0b1220 0x7f7adc0ca190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:40.725 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.716+0000 7f7adb7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7adc0b1220 0x7f7adc0ca190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60762/0 (socket says 172.21.15.99:60762) 2023-10-08T00:17:40.725 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.716+0000 7f7adb7fe700 1 -- 172.21.15.99:0/3127732231 learned_addr learned my addr 172.21.15.99:0/3127732231 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:40.725 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.716+0000 7f7ae0912700 1 --2- 172.21.15.99:0/3127732231 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7adc0cb4f0 0x7f7adc0ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:40.725 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.716+0000 7f7adbfff700 1 --2- 172.21.15.99:0/3127732231 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7adc0b0e50 0x7f7adc151790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:40.726 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.716+0000 7f7adb7fe700 1 -- 172.21.15.99:0/3127732231 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7adc0b0e50 msgr2=0x7f7adc151790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:40.726 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.716+0000 7f7adb7fe700 1 --2- 172.21.15.99:0/3127732231 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7adc0b0e50 0x7f7adc151790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:40.726 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.716+0000 7f7adb7fe700 1 -- 172.21.15.99:0/3127732231 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7adc0cb4f0 msgr2=0x7f7adc0ca6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:40.726 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.716+0000 7f7adb7fe700 1 --2- 172.21.15.99:0/3127732231 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7adc0cb4f0 0x7f7adc0ca6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:40.726 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.716+0000 7f7adb7fe700 1 -- 172.21.15.99:0/3127732231 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad00094f0 con 0x7f7adc0b1220 2023-10-08T00:17:40.726 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.716+0000 7f7adb7fe700 1 --2- 172.21.15.99:0/3127732231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7adc0b1220 0x7f7adc0ca190 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f7ad00030a0 tx=0x7f7ad0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:40.726 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.716+0000 7f7ac3fff700 1 -- 172.21.15.99:0/3127732231 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad0006be0 con 0x7f7adc0b1220 2023-10-08T00:17:40.726 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.716+0000 7f7ac3fff700 1 -- 172.21.15.99:0/3127732231 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad0020070 con 0x7f7adc0b1220 2023-10-08T00:17:40.726 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.716+0000 7f7ae2375700 1 -- 172.21.15.99:0/3127732231 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7adc0b4010 con 0x7f7adc0b1220 2023-10-08T00:17:40.726 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.716+0000 7f7ac3fff700 1 -- 172.21.15.99:0/3127732231 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad0011d50 con 0x7f7adc0b1220 2023-10-08T00:17:40.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.717+0000 7f7ae2375700 1 -- 172.21.15.99:0/3127732231 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7adc0b3a20 con 0x7f7adc0b1220 2023-10-08T00:17:40.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.717+0000 7f7ae2375700 1 -- 172.21.15.99:0/3127732231 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7adc04c3c0 con 0x7f7adc0b1220 2023-10-08T00:17:40.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.718+0000 7f7ac3fff700 1 -- 172.21.15.99:0/3127732231 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7ad0022030 con 0x7f7adc0b1220 2023-10-08T00:17:40.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.718+0000 7f7ac3fff700 1 --2- 172.21.15.99:0/3127732231 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7ac8066f30 0x7f7ac80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:40.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.718+0000 7f7ac3fff700 1 -- 172.21.15.99:0/3127732231 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7ad00863d0 con 0x7f7adc0b1220 2023-10-08T00:17:40.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.718+0000 7f7adbfff700 1 --2- 172.21.15.99:0/3127732231 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7ac8066f30 0x7f7ac80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:40.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.719+0000 7f7adbfff700 1 --2- 172.21.15.99:0/3127732231 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7ac8066f30 0x7f7ac80693e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7acc009af0 tx=0x7f7acc009060 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:40.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.722+0000 7f7ac3fff700 1 -- 172.21.15.99:0/3127732231 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7ad0053640 con 0x7f7adc0b1220 2023-10-08T00:17:40.904 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.904+0000 7f7ae2375700 1 -- 172.21.15.99:0/3127732231 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7adc04c3c0 con 0x7f7adc0b1220 2023-10-08T00:17:40.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.905+0000 7f7ac3fff700 1 -- 172.21.15.99:0/3127732231 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v153) v1 ==== 74+0+15378 (secure 0 0 0) 0x7f7ad0056c60 con 0x7f7adc0b1220 2023-10-08T00:17:40.905 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:17:40.905 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":153,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:17:39.623421+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:17:37.936880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-08T00:17:38.577281+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:17:40.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.907+0000 7f7ae2375700 1 -- 172.21.15.99:0/3127732231 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7ac8066f30 msgr2=0x7f7ac80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:40.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.907+0000 7f7ae2375700 1 --2- 172.21.15.99:0/3127732231 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7ac8066f30 0x7f7ac80693e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7acc009af0 tx=0x7f7acc009060 comp rx=0 tx=0).stop 2023-10-08T00:17:40.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.907+0000 7f7ae2375700 1 -- 172.21.15.99:0/3127732231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7adc0b1220 msgr2=0x7f7adc0ca190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:40.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.907+0000 7f7ae2375700 1 --2- 172.21.15.99:0/3127732231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7adc0b1220 0x7f7adc0ca190 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f7ad00030a0 tx=0x7f7ad0016000 comp rx=0 tx=0).stop 2023-10-08T00:17:40.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.907+0000 7f7ae2375700 1 -- 172.21.15.99:0/3127732231 shutdown_connections 2023-10-08T00:17:40.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.907+0000 7f7ae2375700 1 --2- 172.21.15.99:0/3127732231 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7ac8066f30 0x7f7ac80693e0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:40.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.907+0000 7f7ae2375700 1 --2- 172.21.15.99:0/3127732231 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7adc0cb4f0 0x7f7adc0ca6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:40.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.907+0000 7f7ae2375700 1 --2- 172.21.15.99:0/3127732231 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7adc0b0e50 0x7f7adc151790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:40.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.907+0000 7f7ae2375700 1 --2- 172.21.15.99:0/3127732231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7adc0b1220 0x7f7adc0ca190 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:40.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.907+0000 7f7ae2375700 1 -- 172.21.15.99:0/3127732231 >> 172.21.15.99:0/3127732231 conn(0x7f7adc069870 msgr2=0x7f7adc0c5030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:40.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.907+0000 7f7ae2375700 1 -- 172.21.15.99:0/3127732231 shutdown_connections 2023-10-08T00:17:40.909 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:40.907+0000 7f7ae2375700 1 -- 172.21.15.99:0/3127732231 wait complete. 2023-10-08T00:17:40.920 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T00:17:41.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.036+0000 7f78019b9700 1 Processor -- start 2023-10-08T00:17:41.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.036+0000 7f78019b9700 1 -- start start 2023-10-08T00:17:41.039 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.037+0000 7f78019b9700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f77fc0cce30 0x7f77fc15aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:41.039 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.037+0000 7f78019b9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77fc06aa00 0x7f77fc0b5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:41.039 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.037+0000 7f78019b9700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77fc0b4d70 con 0x7f77fc0b6580 2023-10-08T00:17:41.039 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.037+0000 7f78019b9700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77fc0cd8d0 con 0x7f77fc0cce30 2023-10-08T00:17:41.039 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.037+0000 7f78019b9700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77fc0b4440 con 0x7f77fc06aa00 2023-10-08T00:17:41.039 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.037+0000 7f78009b7700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f77fc0cce30 0x7f77fc15aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:41.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.037+0000 7f77fbfff700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f77fc0b6580 0x7f77fc06a2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:32830/0 (socket says 172.21.15.99:32830) 2023-10-08T00:17:41.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.037+0000 7f78009b7700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f77fc0cce30 0x7f77fc15aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60776/0 (socket says 172.21.15.99:60776) 2023-10-08T00:17:41.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.037+0000 7f77fbfff700 1 -- 172.21.15.99:0/3252306011 learned_addr learned my addr 172.21.15.99:0/3252306011 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:17:41.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.037+0000 7f78011b8700 1 --2- 172.21.15.99:0/3252306011 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77fc06aa00 0x7f77fc0b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:41.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.037+0000 7f78009b7700 1 -- 172.21.15.99:0/3252306011 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77fc06aa00 msgr2=0x7f77fc0b5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:41.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.037+0000 7f78009b7700 1 --2- 172.21.15.99:0/3252306011 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77fc06aa00 0x7f77fc0b5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:41.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.037+0000 7f78009b7700 1 -- 172.21.15.99:0/3252306011 >> v1:172.21.15.99:6789/0 conn(0x7f77fc0b6580 legacy=0x7f77fc06a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:41.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.037+0000 7f78009b7700 1 -- 172.21.15.99:0/3252306011 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77fc0b4720 con 0x7f77fc0cce30 2023-10-08T00:17:41.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.037+0000 7f78011b8700 1 --2- 172.21.15.99:0/3252306011 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77fc06aa00 0x7f77fc0b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:17:41.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.037+0000 7f78009b7700 1 --2- 172.21.15.99:0/3252306011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77fc0cce30 0x7f77fc15aca0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f77f00098e0 tx=0x7f77f00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e091a265705b047 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:41.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.037+0000 7f77fb7fe700 1 -- 172.21.15.99:0/3252306011 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f0009e80 con 0x7f77fc0cce30 2023-10-08T00:17:41.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.038+0000 7f77fb7fe700 1 -- 172.21.15.99:0/3252306011 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77f000b860 con 0x7f77fc0cce30 2023-10-08T00:17:41.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.038+0000 7f77fb7fe700 1 -- 172.21.15.99:0/3252306011 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f000bbe0 con 0x7f77fc0cce30 2023-10-08T00:17:41.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.038+0000 7f78019b9700 1 -- 172.21.15.99:0/3252306011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77fc0cce30 msgr2=0x7f77fc15aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:41.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.038+0000 7f78019b9700 1 --2- 172.21.15.99:0/3252306011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77fc0cce30 0x7f77fc15aca0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f77f00098e0 tx=0x7f77f00091a0 comp rx=0 tx=0).stop 2023-10-08T00:17:41.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.038+0000 7f78019b9700 1 -- 172.21.15.99:0/3252306011 shutdown_connections 2023-10-08T00:17:41.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.038+0000 7f78019b9700 1 --2- 172.21.15.99:0/3252306011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77fc0cce30 0x7f77fc15aca0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:41.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.038+0000 7f78019b9700 1 --2- 172.21.15.99:0/3252306011 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77fc06aa00 0x7f77fc0b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:41.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.038+0000 7f78019b9700 1 -- 172.21.15.99:0/3252306011 >> 172.21.15.99:0/3252306011 conn(0x7f77fc068cb0 msgr2=0x7f77fc0690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:41.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.038+0000 7f78019b9700 1 -- 172.21.15.99:0/3252306011 shutdown_connections 2023-10-08T00:17:41.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.038+0000 7f78019b9700 1 -- 172.21.15.99:0/3252306011 wait complete. 2023-10-08T00:17:41.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.039+0000 7f78019b9700 1 Processor -- start 2023-10-08T00:17:41.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.039+0000 7f78019b9700 1 -- start start 2023-10-08T00:17:41.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.040+0000 7f78019b9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77fc06aa00 0x7f77fc14ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:41.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.040+0000 7f78019b9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77fc0b6580 0x7f77fc153430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:41.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.040+0000 7f78019b9700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77fc0cce30 0x7f77fc153970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:41.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.040+0000 7f78019b9700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77fc16e140 con 0x7f77fc0b6580 2023-10-08T00:17:41.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.040+0000 7f78019b9700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77fc157630 con 0x7f77fc0cce30 2023-10-08T00:17:41.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.040+0000 7f78019b9700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77fc16d810 con 0x7f77fc06aa00 2023-10-08T00:17:41.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.040+0000 7f77fbfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77fc0b6580 0x7f77fc153430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:41.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.040+0000 7f77fbfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77fc0b6580 0x7f77fc153430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60782/0 (socket says 172.21.15.99:60782) 2023-10-08T00:17:41.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.040+0000 7f77fbfff700 1 -- 172.21.15.99:0/3086630092 learned_addr learned my addr 172.21.15.99:0/3086630092 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:41.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.040+0000 7f78009b7700 1 --2- 172.21.15.99:0/3086630092 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77fc06aa00 0x7f77fc14ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:41.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.040+0000 7f78011b8700 1 --2- 172.21.15.99:0/3086630092 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77fc0cce30 0x7f77fc153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:41.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.040+0000 7f77fbfff700 1 -- 172.21.15.99:0/3086630092 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77fc06aa00 msgr2=0x7f77fc14ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:41.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.040+0000 7f77fbfff700 1 --2- 172.21.15.99:0/3086630092 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77fc06aa00 0x7f77fc14ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:41.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.040+0000 7f77fbfff700 1 -- 172.21.15.99:0/3086630092 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77fc0cce30 msgr2=0x7f77fc153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:41.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.040+0000 7f77fbfff700 1 --2- 172.21.15.99:0/3086630092 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77fc0cce30 0x7f77fc153970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:41.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.040+0000 7f77fbfff700 1 -- 172.21.15.99:0/3086630092 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f0009420 con 0x7f77fc0b6580 2023-10-08T00:17:41.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.040+0000 7f78009b7700 1 --2- 172.21.15.99:0/3086630092 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77fc06aa00 0x7f77fc14ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:17:41.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.041+0000 7f77fbfff700 1 --2- 172.21.15.99:0/3086630092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77fc0b6580 0x7f77fc153430 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f77e800c7c0 tx=0x7f77e8007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:41.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.041+0000 7f77f8ff9700 1 -- 172.21.15.99:0/3086630092 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e8005b30 con 0x7f77fc0b6580 2023-10-08T00:17:41.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.041+0000 7f77f8ff9700 1 -- 172.21.15.99:0/3086630092 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e8005cb0 con 0x7f77fc0b6580 2023-10-08T00:17:41.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.041+0000 7f78019b9700 1 -- 172.21.15.99:0/3086630092 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77fc16daf0 con 0x7f77fc0b6580 2023-10-08T00:17:41.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.041+0000 7f77f8ff9700 1 -- 172.21.15.99:0/3086630092 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e8004a80 con 0x7f77fc0b6580 2023-10-08T00:17:41.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.041+0000 7f78019b9700 1 -- 172.21.15.99:0/3086630092 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77fc16e5d0 con 0x7f77fc0b6580 2023-10-08T00:17:41.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.042+0000 7f78019b9700 1 -- 172.21.15.99:0/3086630092 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77fc16e2f0 con 0x7f77fc0b6580 2023-10-08T00:17:41.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.042+0000 7f77f8ff9700 1 -- 172.21.15.99:0/3086630092 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f77e80070f0 con 0x7f77fc0b6580 2023-10-08T00:17:41.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.043+0000 7f77f8ff9700 1 --2- 172.21.15.99:0/3086630092 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f77e0066f30 0x7f77e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:41.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.043+0000 7f77f8ff9700 1 -- 172.21.15.99:0/3086630092 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f77e80839e0 con 0x7f77fc0b6580 2023-10-08T00:17:41.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.043+0000 7f78009b7700 1 --2- 172.21.15.99:0/3086630092 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f77e0066f30 0x7f77e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:41.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.043+0000 7f78009b7700 1 --2- 172.21.15.99:0/3086630092 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f77e0066f30 0x7f77e00693e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f77f00091e0 tx=0x7f77f0017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:41.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.047+0000 7f77f8ff9700 1 -- 172.21.15.99:0/3086630092 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f77fc16e2f0 con 0x7f77fc0b6580 2023-10-08T00:17:41.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:41.275+0000 7f78019b9700 1 -- 172.21.15.99:0/3086630092 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f77fc16e280 con 0x7f77fc0b6580 2023-10-08T00:17:42.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.253+0000 7f77f8ff9700 1 -- 172.21.15.99:0/3086630092 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v69) v1 ==== 118+0+0 (secure 0 0 0) 0x7f77e8018070 con 0x7f77fc0b6580 2023-10-08T00:17:42.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.256+0000 7f78019b9700 1 -- 172.21.15.99:0/3086630092 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f77e0066f30 msgr2=0x7f77e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:42.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.256+0000 7f78019b9700 1 --2- 172.21.15.99:0/3086630092 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f77e0066f30 0x7f77e00693e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f77f00091e0 tx=0x7f77f0017000 comp rx=0 tx=0).stop 2023-10-08T00:17:42.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.256+0000 7f78019b9700 1 -- 172.21.15.99:0/3086630092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77fc0b6580 msgr2=0x7f77fc153430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:42.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.256+0000 7f78019b9700 1 --2- 172.21.15.99:0/3086630092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77fc0b6580 0x7f77fc153430 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f77e800c7c0 tx=0x7f77e8007f70 comp rx=0 tx=0).stop 2023-10-08T00:17:42.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.256+0000 7f78019b9700 1 -- 172.21.15.99:0/3086630092 shutdown_connections 2023-10-08T00:17:42.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.256+0000 7f78019b9700 1 --2- 172.21.15.99:0/3086630092 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f77e0066f30 0x7f77e00693e0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:42.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.256+0000 7f78019b9700 1 --2- 172.21.15.99:0/3086630092 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77fc0cce30 0x7f77fc153970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:42.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.256+0000 7f78019b9700 1 --2- 172.21.15.99:0/3086630092 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77fc06aa00 0x7f77fc14ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:42.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.256+0000 7f78019b9700 1 --2- 172.21.15.99:0/3086630092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77fc0b6580 0x7f77fc153430 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:42.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.256+0000 7f78019b9700 1 -- 172.21.15.99:0/3086630092 >> 172.21.15.99:0/3086630092 conn(0x7f77fc068cb0 msgr2=0x7f77fc15b280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:42.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.256+0000 7f78019b9700 1 -- 172.21.15.99:0/3086630092 shutdown_connections 2023-10-08T00:17:42.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.256+0000 7f78019b9700 1 -- 172.21.15.99:0/3086630092 wait complete. 2023-10-08T00:17:42.271 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T00:17:42.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.393+0000 7f4565c40700 1 Processor -- start 2023-10-08T00:17:42.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.393+0000 7f4565c40700 1 -- start start 2023-10-08T00:17:42.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.394+0000 7f4565c40700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f45600b3570 0x7f456015a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:42.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.394+0000 7f4565c40700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45600b14f0 con 0x7f45600b4860 2023-10-08T00:17:42.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.394+0000 7f4565c40700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45600b17d0 con 0x7f45600b4c30 2023-10-08T00:17:42.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.394+0000 7f4565c40700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45600b1ab0 con 0x7f45600b3570 2023-10-08T00:17:42.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.394+0000 7f455f7fe700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f45600b4860 0x7f4560159a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:32840/0 (socket says 172.21.15.99:32840) 2023-10-08T00:17:42.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.394+0000 7f455f7fe700 1 -- 172.21.15.99:0/2669901707 learned_addr learned my addr 172.21.15.99:0/2669901707 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:17:42.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.395+0000 7f455effd700 1 --2- 172.21.15.99:0/2669901707 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f45600b3570 0x7f456015a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:42.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.395+0000 7f455e7fc700 1 -- 172.21.15.99:0/2669901707 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2934495038 0 0) 0x7f45600b14f0 con 0x7f45600b4860 2023-10-08T00:17:42.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.395+0000 7f455e7fc700 1 -- 172.21.15.99:0/2669901707 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4548003570 con 0x7f45600b4860 2023-10-08T00:17:42.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.395+0000 7f455e7fc700 1 -- 172.21.15.99:0/2669901707 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1594928515 0 0) 0x7f45600b17d0 con 0x7f45600b4c30 2023-10-08T00:17:42.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.395+0000 7f455e7fc700 1 -- 172.21.15.99:0/2669901707 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45600b14f0 con 0x7f45600b4c30 2023-10-08T00:17:42.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.395+0000 7f455e7fc700 1 -- 172.21.15.99:0/2669901707 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 280915014 0 0) 0x7f4548003570 con 0x7f45600b4860 2023-10-08T00:17:42.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.395+0000 7f455e7fc700 1 -- 172.21.15.99:0/2669901707 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f45600b3570 msgr2=0x7f456015a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:42.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.395+0000 7f455e7fc700 1 --2- 172.21.15.99:0/2669901707 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f45600b3570 0x7f456015a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:42.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.395+0000 7f455e7fc700 1 -- 172.21.15.99:0/2669901707 >> v1:172.21.15.191:6789/0 conn(0x7f45600b4c30 legacy=0x7f45600cfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:42.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.395+0000 7f455e7fc700 1 -- 172.21.15.99:0/2669901707 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45600b1d90 con 0x7f45600b4860 2023-10-08T00:17:42.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.395+0000 7f455e7fc700 1 -- 172.21.15.99:0/2669901707 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f45500015f0 con 0x7f45600b4860 2023-10-08T00:17:42.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.395+0000 7f455effd700 1 --2- 172.21.15.99:0/2669901707 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f45600b3570 0x7f456015a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:17:42.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.395+0000 7f455e7fc700 1 -- 172.21.15.99:0/2669901707 >> v1:172.21.15.99:6789/0 conn(0x7f45600b4860 legacy=0x7f4560159a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:42.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.395+0000 7f455e7fc700 1 --2- 172.21.15.99:0/2669901707 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4548003c20 0x7f4548004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:42.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.395+0000 7f455e7fc700 1 --2- 172.21.15.99:0/2669901707 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f45600b3570 0x7f4548004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:42.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.395+0000 7f455e7fc700 1 --2- 172.21.15.99:0/2669901707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4548004d50 0x7f45480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:42.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.395+0000 7f455e7fc700 1 -- 172.21.15.99:0/2669901707 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45600b1ab0 con 0x7f4548004d50 2023-10-08T00:17:42.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.395+0000 7f455e7fc700 1 -- 172.21.15.99:0/2669901707 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454800bb10 con 0x7f4548003c20 2023-10-08T00:17:42.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.395+0000 7f455e7fc700 1 -- 172.21.15.99:0/2669901707 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45480144e0 con 0x7f45600b3570 2023-10-08T00:17:42.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.395+0000 7f455ffff700 1 --2- 172.21.15.99:0/2669901707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4548004d50 0x7f45480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:42.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.395+0000 7f455f7fe700 1 --2- 172.21.15.99:0/2669901707 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4548003c20 0x7f4548004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:42.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.395+0000 7f455effd700 1 --2- 172.21.15.99:0/2669901707 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f45600b3570 0x7f4548004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:42.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.395+0000 7f455ffff700 1 -- 172.21.15.99:0/2669901707 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f45600b3570 msgr2=0x7f4548004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:42.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.395+0000 7f455ffff700 1 --2- 172.21.15.99:0/2669901707 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f45600b3570 0x7f4548004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:42.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.395+0000 7f455ffff700 1 -- 172.21.15.99:0/2669901707 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4548003c20 msgr2=0x7f4548004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:42.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.395+0000 7f455ffff700 1 --2- 172.21.15.99:0/2669901707 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4548003c20 0x7f4548004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:42.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.395+0000 7f455ffff700 1 -- 172.21.15.99:0/2669901707 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4554009420 con 0x7f4548004d50 2023-10-08T00:17:42.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.396+0000 7f455ffff700 1 --2- 172.21.15.99:0/2669901707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4548004d50 0x7f45480051c0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f4558002120 tx=0x7f45580017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7279b99f61635b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:42.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.396+0000 7f455e7fc700 1 -- 172.21.15.99:0/2669901707 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4558001ab0 con 0x7f4548004d50 2023-10-08T00:17:42.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.396+0000 7f455e7fc700 1 -- 172.21.15.99:0/2669901707 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f455800c980 con 0x7f4548004d50 2023-10-08T00:17:42.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.396+0000 7f455e7fc700 1 -- 172.21.15.99:0/2669901707 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455800cd20 con 0x7f4548004d50 2023-10-08T00:17:42.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.396+0000 7f4565c40700 1 -- 172.21.15.99:0/2669901707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4548004d50 msgr2=0x7f45480051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:42.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.396+0000 7f4565c40700 1 --2- 172.21.15.99:0/2669901707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4548004d50 0x7f45480051c0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f4558002120 tx=0x7f45580017a0 comp rx=0 tx=0).stop 2023-10-08T00:17:42.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.396+0000 7f455ffff700 1 -- 172.21.15.99:0/2669901707 reap_dead start 2023-10-08T00:17:42.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.396+0000 7f4565c40700 1 -- 172.21.15.99:0/2669901707 shutdown_connections 2023-10-08T00:17:42.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.396+0000 7f4565c40700 1 -- 172.21.15.99:0/2669901707 >> 172.21.15.99:0/2669901707 conn(0x7f456009a0f0 msgr2=0x7f4560067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:42.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.396+0000 7f4565c40700 1 -- 172.21.15.99:0/2669901707 shutdown_connections 2023-10-08T00:17:42.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.396+0000 7f4565c40700 1 -- 172.21.15.99:0/2669901707 wait complete. 2023-10-08T00:17:42.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.398+0000 7f4565c40700 1 Processor -- start 2023-10-08T00:17:42.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.398+0000 7f4565c40700 1 -- start start 2023-10-08T00:17:42.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.398+0000 7f4565c40700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4548004d50 0x7f456006c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:42.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.398+0000 7f4565c40700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f456015a220 0x7f4560153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:42.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.398+0000 7f4565c40700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4560154200 0x7f4560067e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:42.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.398+0000 7f4565c40700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456006d4a0 con 0x7f456015a220 2023-10-08T00:17:42.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.398+0000 7f4565c40700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456006d1c0 con 0x7f4560154200 2023-10-08T00:17:42.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.398+0000 7f4565c40700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f456009c670 con 0x7f4548004d50 2023-10-08T00:17:42.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.398+0000 7f455effd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f456015a220 0x7f4560153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:42.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.398+0000 7f455effd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f456015a220 0x7f4560153cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60800/0 (socket says 172.21.15.99:60800) 2023-10-08T00:17:42.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.399+0000 7f455effd700 1 -- 172.21.15.99:0/2079472188 learned_addr learned my addr 172.21.15.99:0/2079472188 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:42.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.399+0000 7f455f7fe700 1 --2- 172.21.15.99:0/2079472188 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4548004d50 0x7f456006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:42.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.399+0000 7f455ffff700 1 --2- 172.21.15.99:0/2079472188 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4560154200 0x7f4560067e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:42.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.399+0000 7f455effd700 1 -- 172.21.15.99:0/2079472188 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4548004d50 msgr2=0x7f456006c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:42.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.399+0000 7f455effd700 1 --2- 172.21.15.99:0/2079472188 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4548004d50 0x7f456006c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:42.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.399+0000 7f455effd700 1 -- 172.21.15.99:0/2079472188 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4560154200 msgr2=0x7f4560067e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:42.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.399+0000 7f455effd700 1 --2- 172.21.15.99:0/2079472188 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4560154200 0x7f4560067e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:42.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.399+0000 7f455effd700 1 -- 172.21.15.99:0/2079472188 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f456009c950 con 0x7f456015a220 2023-10-08T00:17:42.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.399+0000 7f455effd700 1 --2- 172.21.15.99:0/2079472188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f456015a220 0x7f4560153cc0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f4554011d70 tx=0x7f45540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:42.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.399+0000 7f45477fe700 1 -- 172.21.15.99:0/2079472188 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4554011e70 con 0x7f456015a220 2023-10-08T00:17:42.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.399+0000 7f45477fe700 1 -- 172.21.15.99:0/2079472188 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f455400fc20 con 0x7f456015a220 2023-10-08T00:17:42.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.399+0000 7f45477fe700 1 -- 172.21.15.99:0/2079472188 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455401d480 con 0x7f456015a220 2023-10-08T00:17:42.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.399+0000 7f4565c40700 1 -- 172.21.15.99:0/2079472188 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f456009cc30 con 0x7f456015a220 2023-10-08T00:17:42.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.399+0000 7f4565c40700 1 -- 172.21.15.99:0/2079472188 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45601660f0 con 0x7f456015a220 2023-10-08T00:17:42.406 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.400+0000 7f4565c40700 1 -- 172.21.15.99:0/2079472188 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f456004c3c0 con 0x7f456015a220 2023-10-08T00:17:42.406 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.400+0000 7f45477fe700 1 -- 172.21.15.99:0/2079472188 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f455401d870 con 0x7f456015a220 2023-10-08T00:17:42.406 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.401+0000 7f45477fe700 1 --2- 172.21.15.99:0/2079472188 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f454c066f30 0x7f454c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:42.406 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.401+0000 7f45477fe700 1 -- 172.21.15.99:0/2079472188 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4554018070 con 0x7f456015a220 2023-10-08T00:17:42.406 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.401+0000 7f455f7fe700 1 --2- 172.21.15.99:0/2079472188 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f454c066f30 0x7f454c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:42.406 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.401+0000 7f455f7fe700 1 --2- 172.21.15.99:0/2079472188 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f454c066f30 0x7f454c0693e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f4550002e30 tx=0x7f4550009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:42.406 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.405+0000 7f45477fe700 1 -- 172.21.15.99:0/2079472188 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f45540525c0 con 0x7f456015a220 2023-10-08T00:17:42.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:42.628+0000 7f4565c40700 1 -- 172.21.15.99:0/2079472188 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f45600b2070 con 0x7f456015a220 2023-10-08T00:17:43.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.255+0000 7f45477fe700 1 -- 172.21.15.99:0/2079472188 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v70) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4554055be0 con 0x7f456015a220 2023-10-08T00:17:43.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.257+0000 7f4565c40700 1 -- 172.21.15.99:0/2079472188 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f454c066f30 msgr2=0x7f454c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:43.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.257+0000 7f4565c40700 1 --2- 172.21.15.99:0/2079472188 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f454c066f30 0x7f454c0693e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f4550002e30 tx=0x7f4550009040 comp rx=0 tx=0).stop 2023-10-08T00:17:43.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.257+0000 7f4565c40700 1 -- 172.21.15.99:0/2079472188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f456015a220 msgr2=0x7f4560153cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:43.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.257+0000 7f4565c40700 1 --2- 172.21.15.99:0/2079472188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f456015a220 0x7f4560153cc0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f4554011d70 tx=0x7f45540091a0 comp rx=0 tx=0).stop 2023-10-08T00:17:43.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.257+0000 7f4565c40700 1 -- 172.21.15.99:0/2079472188 shutdown_connections 2023-10-08T00:17:43.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.257+0000 7f4565c40700 1 --2- 172.21.15.99:0/2079472188 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f454c066f30 0x7f454c0693e0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:43.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.257+0000 7f4565c40700 1 --2- 172.21.15.99:0/2079472188 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4560154200 0x7f4560067e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:43.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.257+0000 7f4565c40700 1 --2- 172.21.15.99:0/2079472188 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4548004d50 0x7f456006c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:43.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.257+0000 7f4565c40700 1 --2- 172.21.15.99:0/2079472188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f456015a220 0x7f4560153cc0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:43.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.257+0000 7f4565c40700 1 -- 172.21.15.99:0/2079472188 >> 172.21.15.99:0/2079472188 conn(0x7f456009a0f0 msgr2=0x7f456015bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:43.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.257+0000 7f4565c40700 1 -- 172.21.15.99:0/2079472188 shutdown_connections 2023-10-08T00:17:43.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.257+0000 7f4565c40700 1 -- 172.21.15.99:0/2079472188 wait complete. 2023-10-08T00:17:43.271 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:17:43.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.385+0000 7f4c1a554700 1 Processor -- start 2023-10-08T00:17:43.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.385+0000 7f4c1a554700 1 -- start start 2023-10-08T00:17:43.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.385+0000 7f4c1a554700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4c140cf1c0 0x7f4c1415e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:43.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.385+0000 7f4c1a554700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4c1415a0c0 0x7f4c14068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:43.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.385+0000 7f4c1a554700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c140cca20 con 0x7f4c14159cf0 2023-10-08T00:17:43.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.385+0000 7f4c1a554700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c1415ea40 con 0x7f4c1415a0c0 2023-10-08T00:17:43.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.385+0000 7f4c1a554700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c14067660 con 0x7f4c140cf1c0 2023-10-08T00:17:43.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.386+0000 7f4c18af1700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4c1415a0c0 0x7f4c14068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:43.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.386+0000 7f4c18af1700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4c1415a0c0 0x7f4c14068390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52994/0 (socket says 172.21.15.99:52994) 2023-10-08T00:17:43.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.386+0000 7f4c18af1700 1 -- 172.21.15.99:0/1990461894 learned_addr learned my addr 172.21.15.99:0/1990461894 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:43.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.386+0000 7f4c137fe700 1 --2- 172.21.15.99:0/1990461894 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4c140cf1c0 0x7f4c1415e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:43.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.386+0000 7f4c18af1700 1 -- 172.21.15.99:0/1990461894 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4c140cf1c0 msgr2=0x7f4c1415e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:43.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.386+0000 7f4c18af1700 1 --2- 172.21.15.99:0/1990461894 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4c140cf1c0 0x7f4c1415e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:43.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.386+0000 7f4c18af1700 1 -- 172.21.15.99:0/1990461894 >> v1:172.21.15.191:6789/0 conn(0x7f4c14159cf0 legacy=0x7f4c140ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:43.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.386+0000 7f4c18af1700 1 -- 172.21.15.99:0/1990461894 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c14067f00 con 0x7f4c1415a0c0 2023-10-08T00:17:43.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.386+0000 7f4c137fe700 1 --2- 172.21.15.99:0/1990461894 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4c140cf1c0 0x7f4c1415e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:17:43.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.386+0000 7f4c18af1700 1 --2- 172.21.15.99:0/1990461894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c1415a0c0 0x7f4c14068390 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c009f20 tx=0x7f4c0c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a9a99894d4321c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:43.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.386+0000 7f4c12ffd700 1 -- 172.21.15.99:0/1990461894 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c00e070 con 0x7f4c1415a0c0 2023-10-08T00:17:43.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.386+0000 7f4c12ffd700 1 -- 172.21.15.99:0/1990461894 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c0c00ba80 con 0x7f4c1415a0c0 2023-10-08T00:17:43.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.386+0000 7f4c12ffd700 1 -- 172.21.15.99:0/1990461894 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c0092e0 con 0x7f4c1415a0c0 2023-10-08T00:17:43.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.387+0000 7f4c1a554700 1 -- 172.21.15.99:0/1990461894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c1415a0c0 msgr2=0x7f4c14068390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:43.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.387+0000 7f4c1a554700 1 --2- 172.21.15.99:0/1990461894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c1415a0c0 0x7f4c14068390 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c009f20 tx=0x7f4c0c009840 comp rx=0 tx=0).stop 2023-10-08T00:17:43.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.387+0000 7f4c1a554700 1 -- 172.21.15.99:0/1990461894 shutdown_connections 2023-10-08T00:17:43.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.387+0000 7f4c1a554700 1 --2- 172.21.15.99:0/1990461894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c1415a0c0 0x7f4c14068390 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:43.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.387+0000 7f4c1a554700 1 --2- 172.21.15.99:0/1990461894 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4c140cf1c0 0x7f4c1415e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:43.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.387+0000 7f4c1a554700 1 -- 172.21.15.99:0/1990461894 >> 172.21.15.99:0/1990461894 conn(0x7f4c1406c6e0 msgr2=0x7f4c14069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:43.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.387+0000 7f4c1a554700 1 -- 172.21.15.99:0/1990461894 shutdown_connections 2023-10-08T00:17:43.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.387+0000 7f4c1a554700 1 -- 172.21.15.99:0/1990461894 wait complete. 2023-10-08T00:17:43.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.388+0000 7f4c1a554700 1 Processor -- start 2023-10-08T00:17:43.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.388+0000 7f4c1a554700 1 -- start start 2023-10-08T00:17:43.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.389+0000 7f4c1a554700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4c140cf1c0 0x7f4c141556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:43.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.389+0000 7f4c1a554700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c14159cf0 0x7f4c140b00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:43.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.389+0000 7f4c1a554700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c1415a0c0 0x7f4c140ace50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:43.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.389+0000 7f4c1a554700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c140b28c0 con 0x7f4c1415a0c0 2023-10-08T00:17:43.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.389+0000 7f4c1a554700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c140b0990 con 0x7f4c14159cf0 2023-10-08T00:17:43.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.389+0000 7f4c1a554700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c140b0c70 con 0x7f4c140cf1c0 2023-10-08T00:17:43.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.389+0000 7f4c18af1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c1415a0c0 0x7f4c140ace50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:43.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.389+0000 7f4c18af1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c1415a0c0 0x7f4c140ace50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53008/0 (socket says 172.21.15.99:53008) 2023-10-08T00:17:43.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.389+0000 7f4c18af1700 1 -- 172.21.15.99:0/3148189790 learned_addr learned my addr 172.21.15.99:0/3148189790 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:43.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.389+0000 7f4c13fff700 1 --2- 172.21.15.99:0/3148189790 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4c140cf1c0 0x7f4c141556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:43.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.389+0000 7f4c137fe700 1 --2- 172.21.15.99:0/3148189790 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c14159cf0 0x7f4c140b00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:43.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.389+0000 7f4c18af1700 1 -- 172.21.15.99:0/3148189790 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4c140cf1c0 msgr2=0x7f4c141556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:43.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.389+0000 7f4c18af1700 1 --2- 172.21.15.99:0/3148189790 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4c140cf1c0 0x7f4c141556e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:43.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.389+0000 7f4c18af1700 1 -- 172.21.15.99:0/3148189790 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c14159cf0 msgr2=0x7f4c140b00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:43.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.389+0000 7f4c18af1700 1 --2- 172.21.15.99:0/3148189790 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c14159cf0 0x7f4c140b00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:43.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.389+0000 7f4c18af1700 1 -- 172.21.15.99:0/3148189790 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c0c009ae0 con 0x7f4c1415a0c0 2023-10-08T00:17:43.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.389+0000 7f4c18af1700 1 --2- 172.21.15.99:0/3148189790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c1415a0c0 0x7f4c140ace50 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c00b6c0 tx=0x7f4c0c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:43.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.389+0000 7f4bfbfff700 1 -- 172.21.15.99:0/3148189790 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c008d20 con 0x7f4c1415a0c0 2023-10-08T00:17:43.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.389+0000 7f4bfbfff700 1 -- 172.21.15.99:0/3148189790 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c0c008e80 con 0x7f4c1415a0c0 2023-10-08T00:17:43.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.389+0000 7f4bfbfff700 1 -- 172.21.15.99:0/3148189790 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c0266b0 con 0x7f4c1415a0c0 2023-10-08T00:17:43.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.389+0000 7f4c1a554700 1 -- 172.21.15.99:0/3148189790 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c140b2b10 con 0x7f4c1415a0c0 2023-10-08T00:17:43.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.390+0000 7f4c1a554700 1 -- 172.21.15.99:0/3148189790 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c14155c20 con 0x7f4c1415a0c0 2023-10-08T00:17:43.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.390+0000 7f4c1a554700 1 -- 172.21.15.99:0/3148189790 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c1404c3c0 con 0x7f4c1415a0c0 2023-10-08T00:17:43.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.391+0000 7f4bfbfff700 1 -- 172.21.15.99:0/3148189790 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4c0c003040 con 0x7f4c1415a0c0 2023-10-08T00:17:43.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.391+0000 7f4bfbfff700 1 --2- 172.21.15.99:0/3148189790 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4c00066f30 0x7f4c000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:43.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.391+0000 7f4bfbfff700 1 -- 172.21.15.99:0/3148189790 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4c0c086470 con 0x7f4c1415a0c0 2023-10-08T00:17:43.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.391+0000 7f4c13fff700 1 --2- 172.21.15.99:0/3148189790 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4c00066f30 0x7f4c000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:43.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.392+0000 7f4c13fff700 1 --2- 172.21.15.99:0/3148189790 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4c00066f30 0x7f4c000693e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f4c04003a80 tx=0x7f4c04007ec0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:43.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.396+0000 7f4bfbfff700 1 -- 172.21.15.99:0/3148189790 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4c0c0536e0 con 0x7f4c1415a0c0 2023-10-08T00:17:43.620 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.618+0000 7f4c1a554700 1 -- 172.21.15.99:0/3148189790 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c140ce730 con 0x7f4c1415a0c0 2023-10-08T00:17:43.621 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.619+0000 7f4bfbfff700 1 -- 172.21.15.99:0/3148189790 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4599 (secure 0 0 0) 0x7f4c0c056d00 con 0x7f4c1415a0c0 2023-10-08T00:17:43.621 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:17:43.621 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":70,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6056,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/3025449992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3025449992},{"type":"v1","addr":"172.21.15.191:6836","nonce":3025449992}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":7476,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/86350138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":86350138},{"type":"v1","addr":"172.21.15.99:6834","nonce":86350138}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":7479,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1282887036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1282887036},{"type":"v1","addr":"172.21.15.99:6835","nonce":1282887036}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:17:39.623734+0000","modified":"2023-10-08T00:17:43.252633+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7494":{"gid":7494,"name":"d","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/1165162515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":1165162515},{"type":"v1","addr":"172.21.15.191:6837","nonce":1165162515}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-08T00:17:43.623 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.621+0000 7f4c1a554700 1 -- 172.21.15.99:0/3148189790 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4c00066f30 msgr2=0x7f4c000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:43.623 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.621+0000 7f4c1a554700 1 --2- 172.21.15.99:0/3148189790 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4c00066f30 0x7f4c000693e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f4c04003a80 tx=0x7f4c04007ec0 comp rx=0 tx=0).stop 2023-10-08T00:17:43.623 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.621+0000 7f4c1a554700 1 -- 172.21.15.99:0/3148189790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c1415a0c0 msgr2=0x7f4c140ace50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:43.623 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.621+0000 7f4c1a554700 1 --2- 172.21.15.99:0/3148189790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c1415a0c0 0x7f4c140ace50 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c00b6c0 tx=0x7f4c0c018000 comp rx=0 tx=0).stop 2023-10-08T00:17:43.623 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.621+0000 7f4c1a554700 1 -- 172.21.15.99:0/3148189790 shutdown_connections 2023-10-08T00:17:43.623 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.621+0000 7f4c1a554700 1 --2- 172.21.15.99:0/3148189790 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4c00066f30 0x7f4c000693e0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:43.623 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.621+0000 7f4c1a554700 1 --2- 172.21.15.99:0/3148189790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c1415a0c0 0x7f4c140ace50 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:43.624 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.621+0000 7f4c1a554700 1 --2- 172.21.15.99:0/3148189790 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4c140cf1c0 0x7f4c141556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:43.624 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.621+0000 7f4c1a554700 1 --2- 172.21.15.99:0/3148189790 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c14159cf0 0x7f4c140b00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:43.626 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.621+0000 7f4c1a554700 1 -- 172.21.15.99:0/3148189790 >> 172.21.15.99:0/3148189790 conn(0x7f4c1406c6e0 msgr2=0x7f4c140c9d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:43.626 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.622+0000 7f4c1a554700 1 -- 172.21.15.99:0/3148189790 shutdown_connections 2023-10-08T00:17:43.626 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.622+0000 7f4c1a554700 1 -- 172.21.15.99:0/3148189790 wait complete. 2023-10-08T00:17:43.626 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 70 2023-10-08T00:17:43.635 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:17:43.750 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.748+0000 7f61e511b700 1 Processor -- start 2023-10-08T00:17:43.750 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.748+0000 7f61e511b700 1 -- start start 2023-10-08T00:17:43.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.749+0000 7f61e511b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f61e0159ef0 0x7f61e00ce2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:43.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.749+0000 7f61e511b700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61e00c91d0 con 0x7f61e00cbca0 2023-10-08T00:17:43.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.749+0000 7f61e511b700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61e015de70 con 0x7f61e015a2c0 2023-10-08T00:17:43.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.749+0000 7f61e511b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61e015dc60 con 0x7f61e0159ef0 2023-10-08T00:17:43.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.749+0000 7f61df59e700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f61e00cbca0 0x7f61e006be20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:60442/0 (socket says 172.21.15.99:60442) 2023-10-08T00:17:43.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.749+0000 7f61df59e700 1 -- 172.21.15.99:0/2374756688 learned_addr learned my addr 172.21.15.99:0/2374756688 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:17:43.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.749+0000 7f61ded9d700 1 --2- 172.21.15.99:0/2374756688 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f61e0159ef0 0x7f61e00ce2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:43.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.749+0000 7f61de59c700 1 -- 172.21.15.99:0/2374756688 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3545749520 0 0) 0x7f61e00c91d0 con 0x7f61e00cbca0 2023-10-08T00:17:43.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.749+0000 7f61de59c700 1 -- 172.21.15.99:0/2374756688 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61cc003570 con 0x7f61e00cbca0 2023-10-08T00:17:43.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.750+0000 7f61de59c700 1 -- 172.21.15.99:0/2374756688 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1426405293 0 0) 0x7f61e015de70 con 0x7f61e015a2c0 2023-10-08T00:17:43.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.750+0000 7f61de59c700 1 -- 172.21.15.99:0/2374756688 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61e00c91d0 con 0x7f61e015a2c0 2023-10-08T00:17:43.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.750+0000 7f61ded9d700 1 -- 172.21.15.99:0/2374756688 >> v1:172.21.15.191:6789/0 conn(0x7f61e015a2c0 legacy=0x7f61e00cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:43.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.750+0000 7f61ded9d700 1 -- 172.21.15.99:0/2374756688 >> v1:172.21.15.99:6789/0 conn(0x7f61e00cbca0 legacy=0x7f61e006be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:43.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.750+0000 7f61ded9d700 1 -- 172.21.15.99:0/2374756688 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e015e150 con 0x7f61e0159ef0 2023-10-08T00:17:43.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.750+0000 7f61ded9d700 1 --2- 172.21.15.99:0/2374756688 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f61e0159ef0 0x7f61e00ce2e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f61d0009910 tx=0x7f61d00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fd090ec6c734c26 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:43.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.750+0000 7f61de59c700 1 -- 172.21.15.99:0/2374756688 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d0011630 con 0x7f61e0159ef0 2023-10-08T00:17:43.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.750+0000 7f61de59c700 1 -- 172.21.15.99:0/2374756688 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61d0011790 con 0x7f61e0159ef0 2023-10-08T00:17:43.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.750+0000 7f61de59c700 1 -- 172.21.15.99:0/2374756688 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d0011b30 con 0x7f61e0159ef0 2023-10-08T00:17:43.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.750+0000 7f61e511b700 1 -- 172.21.15.99:0/2374756688 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f61e0159ef0 msgr2=0x7f61e00ce2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:43.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.750+0000 7f61e511b700 1 --2- 172.21.15.99:0/2374756688 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f61e0159ef0 0x7f61e00ce2e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f61d0009910 tx=0x7f61d00091a0 comp rx=0 tx=0).stop 2023-10-08T00:17:43.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.751+0000 7f61e511b700 1 -- 172.21.15.99:0/2374756688 shutdown_connections 2023-10-08T00:17:43.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.751+0000 7f61e511b700 1 --2- 172.21.15.99:0/2374756688 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f61e0159ef0 0x7f61e00ce2e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:43.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.751+0000 7f61e511b700 1 -- 172.21.15.99:0/2374756688 >> 172.21.15.99:0/2374756688 conn(0x7f61e0069870 msgr2=0x7f61e0069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:43.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.751+0000 7f61e511b700 1 -- 172.21.15.99:0/2374756688 shutdown_connections 2023-10-08T00:17:43.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.751+0000 7f61e511b700 1 -- 172.21.15.99:0/2374756688 wait complete. 2023-10-08T00:17:43.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.752+0000 7f61e511b700 1 Processor -- start 2023-10-08T00:17:43.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.752+0000 7f61e511b700 1 -- start start 2023-10-08T00:17:43.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.752+0000 7f61e511b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61e00cbca0 0x7f61e00c7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:43.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.752+0000 7f61e511b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f61e0159ef0 0x7f61e00ca800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:43.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.752+0000 7f61e511b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f61e015a2c0 0x7f61e01544c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:43.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.752+0000 7f61e511b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e00c7ce0 con 0x7f61e00cbca0 2023-10-08T00:17:43.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.752+0000 7f61e511b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e00c7fc0 con 0x7f61e0159ef0 2023-10-08T00:17:43.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.752+0000 7f61e511b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61e00c82a0 con 0x7f61e015a2c0 2023-10-08T00:17:43.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.753+0000 7f61ded9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61e00cbca0 0x7f61e00c7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:43.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.753+0000 7f61ded9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61e00cbca0 0x7f61e00c7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53024/0 (socket says 172.21.15.99:53024) 2023-10-08T00:17:43.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.753+0000 7f61ded9d700 1 -- 172.21.15.99:0/3161640086 learned_addr learned my addr 172.21.15.99:0/3161640086 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:43.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.753+0000 7f61ded9d700 1 -- 172.21.15.99:0/3161640086 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f61e015a2c0 msgr2=0x7f61e01544c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:43.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.753+0000 7f61df59e700 1 --2- 172.21.15.99:0/3161640086 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f61e015a2c0 0x7f61e01544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:43.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.753+0000 7f61d659c700 1 --2- 172.21.15.99:0/3161640086 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f61e0159ef0 0x7f61e00ca800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:43.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.753+0000 7f61ded9d700 1 --2- 172.21.15.99:0/3161640086 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f61e015a2c0 0x7f61e01544c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:43.759 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.753+0000 7f61ded9d700 1 -- 172.21.15.99:0/3161640086 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f61e0159ef0 msgr2=0x7f61e00ca800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:43.759 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.753+0000 7f61ded9d700 1 --2- 172.21.15.99:0/3161640086 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f61e0159ef0 0x7f61e00ca800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:43.759 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.753+0000 7f61ded9d700 1 -- 172.21.15.99:0/3161640086 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d0009420 con 0x7f61e00cbca0 2023-10-08T00:17:43.759 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.753+0000 7f61ded9d700 1 --2- 172.21.15.99:0/3161640086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61e00cbca0 0x7f61e00c7490 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f61d0009f00 tx=0x7f61d0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:43.759 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.753+0000 7f61d77fe700 1 -- 172.21.15.99:0/3161640086 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d001f070 con 0x7f61e00cbca0 2023-10-08T00:17:43.759 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.753+0000 7f61d77fe700 1 -- 172.21.15.99:0/3161640086 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61d0000bc0 con 0x7f61e00cbca0 2023-10-08T00:17:43.759 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.753+0000 7f61e511b700 1 -- 172.21.15.99:0/3161640086 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61e00c8580 con 0x7f61e00cbca0 2023-10-08T00:17:43.759 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.753+0000 7f61d77fe700 1 -- 172.21.15.99:0/3161640086 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d0025420 con 0x7f61e00cbca0 2023-10-08T00:17:43.759 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.753+0000 7f61e511b700 1 -- 172.21.15.99:0/3161640086 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61e00cad40 con 0x7f61e00cbca0 2023-10-08T00:17:43.759 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.754+0000 7f61e511b700 1 -- 172.21.15.99:0/3161640086 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61e00b2e70 con 0x7f61e00cbca0 2023-10-08T00:17:43.759 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.754+0000 7f61d77fe700 1 -- 172.21.15.99:0/3161640086 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f61d0023070 con 0x7f61e00cbca0 2023-10-08T00:17:43.759 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.755+0000 7f61d77fe700 1 --2- 172.21.15.99:0/3161640086 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f61c0066f30 0x7f61c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:43.760 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.755+0000 7f61d77fe700 1 -- 172.21.15.99:0/3161640086 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f61d00858f0 con 0x7f61e00cbca0 2023-10-08T00:17:43.760 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.755+0000 7f61d659c700 1 --2- 172.21.15.99:0/3161640086 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f61c0066f30 0x7f61c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:43.760 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.755+0000 7f61d659c700 1 --2- 172.21.15.99:0/3161640086 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f61c0066f30 0x7f61c00693e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f61e00cee90 tx=0x7f61c8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:43.760 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.759+0000 7f61d77fe700 1 -- 172.21.15.99:0/3161640086 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f61d0052b60 con 0x7f61e00cbca0 2023-10-08T00:17:43.945 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.943+0000 7f61e511b700 1 -- 172.21.15.99:0/3161640086 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f61e00caf30 con 0x7f61e00cbca0 2023-10-08T00:17:43.945 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.944+0000 7f61d77fe700 1 -- 172.21.15.99:0/3161640086 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v154) v1 ==== 74+0+15365 (secure 0 0 0) 0x7f61d002a090 con 0x7f61e00cbca0 2023-10-08T00:17:43.946 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:17:43.946 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":154,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:17:41.234166+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:17:37.936880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-08T00:17:38.577281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:17:43.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.946+0000 7f61e511b700 1 -- 172.21.15.99:0/3161640086 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f61c0066f30 msgr2=0x7f61c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:43.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.946+0000 7f61e511b700 1 --2- 172.21.15.99:0/3161640086 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f61c0066f30 0x7f61c00693e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f61e00cee90 tx=0x7f61c8009040 comp rx=0 tx=0).stop 2023-10-08T00:17:43.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.946+0000 7f61e511b700 1 -- 172.21.15.99:0/3161640086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61e00cbca0 msgr2=0x7f61e00c7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:43.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.946+0000 7f61e511b700 1 --2- 172.21.15.99:0/3161640086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61e00cbca0 0x7f61e00c7490 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f61d0009f00 tx=0x7f61d0017000 comp rx=0 tx=0).stop 2023-10-08T00:17:43.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.946+0000 7f61e511b700 1 -- 172.21.15.99:0/3161640086 shutdown_connections 2023-10-08T00:17:43.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.946+0000 7f61e511b700 1 --2- 172.21.15.99:0/3161640086 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f61c0066f30 0x7f61c00693e0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:43.948 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.946+0000 7f61e511b700 1 --2- 172.21.15.99:0/3161640086 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f61e015a2c0 0x7f61e01544c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:43.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.946+0000 7f61e511b700 1 --2- 172.21.15.99:0/3161640086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61e00cbca0 0x7f61e00c7490 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:43.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.946+0000 7f61e511b700 1 --2- 172.21.15.99:0/3161640086 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f61e0159ef0 0x7f61e00ca800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:43.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.946+0000 7f61e511b700 1 -- 172.21.15.99:0/3161640086 >> 172.21.15.99:0/3161640086 conn(0x7f61e0069870 msgr2=0x7f61e0069e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:43.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.946+0000 7f61e511b700 1 -- 172.21.15.99:0/3161640086 shutdown_connections 2023-10-08T00:17:43.949 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:43.946+0000 7f61e511b700 1 -- 172.21.15.99:0/3161640086 wait complete. 2023-10-08T00:17:43.960 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T00:17:43.960 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T00:17:44.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.072+0000 7f667bf1c700 1 Processor -- start 2023-10-08T00:17:44.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.072+0000 7f667bf1c700 1 -- start start 2023-10-08T00:17:44.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.073+0000 7f667bf1c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6674157870 0x7f667415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:44.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.073+0000 7f667bf1c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f6674157c40 0x7f6674158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:44.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.073+0000 7f667bf1c700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f667415cbb0 con 0x7f667415a620 2023-10-08T00:17:44.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.073+0000 7f667bf1c700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f667415bfa0 con 0x7f6674157870 2023-10-08T00:17:44.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.073+0000 7f667bf1c700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f667415c280 con 0x7f6674157c40 2023-10-08T00:17:44.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.073+0000 7f6679cb8700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6674157870 0x7f667415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:44.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.073+0000 7f6679cb8700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6674157870 0x7f667415b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53040/0 (socket says 172.21.15.99:53040) 2023-10-08T00:17:44.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.073+0000 7f6679cb8700 1 -- 172.21.15.99:0/1780414062 learned_addr learned my addr 172.21.15.99:0/1780414062 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:44.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.073+0000 7f6679cb8700 1 -- 172.21.15.99:0/1780414062 >> v2:172.21.15.191:3300/0 conn(0x7f6674157c40 msgr2=0x7f6674158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:44.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.073+0000 7f6679cb8700 1 --2- 172.21.15.99:0/1780414062 >> v2:172.21.15.191:3300/0 conn(0x7f6674157c40 0x7f6674158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:44.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.073+0000 7f6679cb8700 1 -- 172.21.15.99:0/1780414062 >> v1:172.21.15.191:6789/0 conn(0x7f667415a620 legacy=0x7f667415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:44.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.073+0000 7f6679cb8700 1 -- 172.21.15.99:0/1780414062 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f667415c560 con 0x7f6674157870 2023-10-08T00:17:44.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.074+0000 7f6679cb8700 1 --2- 172.21.15.99:0/1780414062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6674157870 0x7f667415b6a0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f667406b4e0 tx=0x7f6670009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a094facca77f1f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:44.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.074+0000 7f6678cb6700 1 -- 172.21.15.99:0/1780414062 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667000e070 con 0x7f6674157870 2023-10-08T00:17:44.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.074+0000 7f6678cb6700 1 -- 172.21.15.99:0/1780414062 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f667000ba10 con 0x7f6674157870 2023-10-08T00:17:44.077 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.074+0000 7f6678cb6700 1 -- 172.21.15.99:0/1780414062 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66700092e0 con 0x7f6674157870 2023-10-08T00:17:44.079 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.074+0000 7f667bf1c700 1 -- 172.21.15.99:0/1780414062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6674157870 msgr2=0x7f667415b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:44.079 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.074+0000 7f667bf1c700 1 --2- 172.21.15.99:0/1780414062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6674157870 0x7f667415b6a0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f667406b4e0 tx=0x7f6670009840 comp rx=0 tx=0).stop 2023-10-08T00:17:44.079 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.074+0000 7f667bf1c700 1 -- 172.21.15.99:0/1780414062 shutdown_connections 2023-10-08T00:17:44.079 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.074+0000 7f667bf1c700 1 --2- 172.21.15.99:0/1780414062 >> v2:172.21.15.191:3300/0 conn(0x7f6674157c40 0x7f6674158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:44.080 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.074+0000 7f667bf1c700 1 --2- 172.21.15.99:0/1780414062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6674157870 0x7f667415b6a0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:44.080 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.074+0000 7f667bf1c700 1 -- 172.21.15.99:0/1780414062 >> 172.21.15.99:0/1780414062 conn(0x7f6674068ce0 msgr2=0x7f66740690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:44.080 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.074+0000 7f667bf1c700 1 -- 172.21.15.99:0/1780414062 shutdown_connections 2023-10-08T00:17:44.080 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.074+0000 7f667bf1c700 1 -- 172.21.15.99:0/1780414062 wait complete. 2023-10-08T00:17:44.080 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.075+0000 7f667bf1c700 1 Processor -- start 2023-10-08T00:17:44.080 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.076+0000 7f667bf1c700 1 -- start start 2023-10-08T00:17:44.080 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.076+0000 7f667bf1c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6674157870 0x7f66740c7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:44.080 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.076+0000 7f667bf1c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6674157c40 0x7f66740c7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:44.080 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.076+0000 7f667bf1c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f667415a620 0x7f66740c5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:44.080 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.076+0000 7f667bf1c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667415dac0 con 0x7f667415a620 2023-10-08T00:17:44.080 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.076+0000 7f667bf1c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66740c6920 con 0x7f6674157c40 2023-10-08T00:17:44.080 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.076+0000 7f667bf1c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66740c6c00 con 0x7f6674157870 2023-10-08T00:17:44.081 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.076+0000 7f667a4b9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f667415a620 0x7f66740c5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:44.081 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.076+0000 7f667a4b9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f667415a620 0x7f66740c5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53046/0 (socket says 172.21.15.99:53046) 2023-10-08T00:17:44.081 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.076+0000 7f667a4b9700 1 -- 172.21.15.99:0/3535658569 learned_addr learned my addr 172.21.15.99:0/3535658569 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:44.081 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.076+0000 7f66794b7700 1 --2- 172.21.15.99:0/3535658569 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6674157c40 0x7f66740c7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:44.081 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.076+0000 7f6679cb8700 1 --2- 172.21.15.99:0/3535658569 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6674157870 0x7f66740c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:44.081 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.077+0000 7f667a4b9700 1 -- 172.21.15.99:0/3535658569 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6674157870 msgr2=0x7f66740c7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:44.081 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.077+0000 7f667a4b9700 1 --2- 172.21.15.99:0/3535658569 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6674157870 0x7f66740c7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:44.081 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.077+0000 7f667a4b9700 1 -- 172.21.15.99:0/3535658569 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6674157c40 msgr2=0x7f66740c7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:44.081 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.077+0000 7f667a4b9700 1 --2- 172.21.15.99:0/3535658569 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6674157c40 0x7f66740c7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:44.081 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.077+0000 7f667a4b9700 1 -- 172.21.15.99:0/3535658569 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6670009ae0 con 0x7f667415a620 2023-10-08T00:17:44.081 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.077+0000 7f667a4b9700 1 --2- 172.21.15.99:0/3535658569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f667415a620 0x7f66740c5bc0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f666c00b600 tx=0x7f666c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:44.081 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.077+0000 7f6669ffb700 1 -- 172.21.15.99:0/3535658569 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f666c00f510 con 0x7f667415a620 2023-10-08T00:17:44.081 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.077+0000 7f6669ffb700 1 -- 172.21.15.99:0/3535658569 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f666c00f690 con 0x7f667415a620 2023-10-08T00:17:44.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.077+0000 7f667bf1c700 1 -- 172.21.15.99:0/3535658569 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66740c6ee0 con 0x7f667415a620 2023-10-08T00:17:44.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.077+0000 7f6669ffb700 1 -- 172.21.15.99:0/3535658569 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f666c00fa50 con 0x7f667415a620 2023-10-08T00:17:44.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.077+0000 7f667bf1c700 1 -- 172.21.15.99:0/3535658569 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66740c9620 con 0x7f667415a620 2023-10-08T00:17:44.083 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.078+0000 7f667bf1c700 1 -- 172.21.15.99:0/3535658569 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f667404c3c0 con 0x7f667415a620 2023-10-08T00:17:44.083 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.078+0000 7f6669ffb700 1 -- 172.21.15.99:0/3535658569 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f666c01a400 con 0x7f667415a620 2023-10-08T00:17:44.083 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.079+0000 7f6669ffb700 1 --2- 172.21.15.99:0/3535658569 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6658066f30 0x7f66580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:44.083 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.079+0000 7f6669ffb700 1 -- 172.21.15.99:0/3535658569 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f666c083f80 con 0x7f667415a620 2023-10-08T00:17:44.083 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.079+0000 7f6679cb8700 1 --2- 172.21.15.99:0/3535658569 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6658066f30 0x7f66580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:44.083 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.080+0000 7f6679cb8700 1 --2- 172.21.15.99:0/3535658569 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6658066f30 0x7f66580693e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6670009880 tx=0x7f6670017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:44.084 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.084+0000 7f6669ffb700 1 -- 172.21.15.99:0/3535658569 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f666c051270 con 0x7f667415a620 2023-10-08T00:17:44.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.260+0000 7f667bf1c700 1 -- 172.21.15.99:0/3535658569 --> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f667415c840 con 0x7f6658066f30 2023-10-08T00:17:44.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.272+0000 7f6669ffb700 1 -- 172.21.15.99:0/3535658569 <== mgr.4099 v2:172.21.15.191:6832/85985 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285840 (secure 0 0 0) 0x7f667415c840 con 0x7f6658066f30 2023-10-08T00:17:44.277 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:17:44.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.275+0000 7f667bf1c700 1 -- 172.21.15.99:0/3535658569 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6658066f30 msgr2=0x7f66580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:44.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.275+0000 7f667bf1c700 1 --2- 172.21.15.99:0/3535658569 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6658066f30 0x7f66580693e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6670009880 tx=0x7f6670017000 comp rx=0 tx=0).stop 2023-10-08T00:17:44.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.275+0000 7f667bf1c700 1 -- 172.21.15.99:0/3535658569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f667415a620 msgr2=0x7f66740c5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:44.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.275+0000 7f667bf1c700 1 --2- 172.21.15.99:0/3535658569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f667415a620 0x7f66740c5bc0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f666c00b600 tx=0x7f666c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:17:44.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.275+0000 7f667bf1c700 1 -- 172.21.15.99:0/3535658569 shutdown_connections 2023-10-08T00:17:44.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.275+0000 7f667bf1c700 1 --2- 172.21.15.99:0/3535658569 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6658066f30 0x7f66580693e0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:44.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.275+0000 7f667bf1c700 1 --2- 172.21.15.99:0/3535658569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f667415a620 0x7f66740c5bc0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:44.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.275+0000 7f667bf1c700 1 --2- 172.21.15.99:0/3535658569 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6674157870 0x7f66740c7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:44.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.275+0000 7f667bf1c700 1 --2- 172.21.15.99:0/3535658569 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6674157c40 0x7f66740c7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:44.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.275+0000 7f667bf1c700 1 -- 172.21.15.99:0/3535658569 >> 172.21.15.99:0/3535658569 conn(0x7f6674068ce0 msgr2=0x7f66740690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:44.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.275+0000 7f667bf1c700 1 -- 172.21.15.99:0/3535658569 shutdown_connections 2023-10-08T00:17:44.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.275+0000 7f667bf1c700 1 -- 172.21.15.99:0/3535658569 wait complete. 2023-10-08T00:17:44.280 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2023-10-08T00:17:44.288 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2023-10-08T00:17:44.122252+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":7856,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1554643,"statfs":{"total":1600000000,"available":1591955228,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.894650"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.257925+0000","last_change":"2023-10-08T00:17:40.257821+0000","last_active":"2023-10-08T00:17:40.257925+0000","last_peered":"2023-10-08T00:17:40.257925+0000","last_clean":"2023-10-08T00:17:40.257925+0000","last_became_active":"2023-10-08T00:17:40.257038+0000","last_became_peered":"2023-10-08T00:17:40.257038+0000","last_unstale":"2023-10-08T00:17:40.257925+0000","last_undegraded":"2023-10-08T00:17:40.257925+0000","last_fullsized":"2023-10-08T00:17:40.257925+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:34:52.462946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.258842+0000","last_change":"2023-10-08T00:17:40.258697+0000","last_active":"2023-10-08T00:17:40.258842+0000","last_peered":"2023-10-08T00:17:40.258842+0000","last_clean":"2023-10-08T00:17:40.258842+0000","last_became_active":"2023-10-08T00:17:40.257477+0000","last_became_peered":"2023-10-08T00:17:40.257477+0000","last_unstale":"2023-10-08T00:17:40.258842+0000","last_undegraded":"2023-10-08T00:17:40.258842+0000","last_fullsized":"2023-10-08T00:17:40.258842+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:30:30.805838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.241699+0000","last_change":"2023-10-08T00:17:40.241468+0000","last_active":"2023-10-08T00:17:40.241699+0000","last_peered":"2023-10-08T00:17:40.241699+0000","last_clean":"2023-10-08T00:17:40.241699+0000","last_became_active":"2023-10-08T00:17:40.241052+0000","last_became_peered":"2023-10-08T00:17:40.241052+0000","last_unstale":"2023-10-08T00:17:40.241699+0000","last_undegraded":"2023-10-08T00:17:40.241699+0000","last_fullsized":"2023-10-08T00:17:40.241699+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:42:27.289223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.244797+0000","last_change":"2023-10-08T00:17:40.244720+0000","last_active":"2023-10-08T00:17:40.244797+0000","last_peered":"2023-10-08T00:17:40.244797+0000","last_clean":"2023-10-08T00:17:40.244797+0000","last_became_active":"2023-10-08T00:17:40.244476+0000","last_became_peered":"2023-10-08T00:17:40.244476+0000","last_unstale":"2023-10-08T00:17:40.244797+0000","last_undegraded":"2023-10-08T00:17:40.244797+0000","last_fullsized":"2023-10-08T00:17:40.244797+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:34:52.462946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.244822+0000","last_change":"2023-10-08T00:17:40.244700+0000","last_active":"2023-10-08T00:17:40.244822+0000","last_peered":"2023-10-08T00:17:40.244822+0000","last_clean":"2023-10-08T00:17:40.244822+0000","last_became_active":"2023-10-08T00:17:40.244351+0000","last_became_peered":"2023-10-08T00:17:40.244351+0000","last_unstale":"2023-10-08T00:17:40.244822+0000","last_undegraded":"2023-10-08T00:17:40.244822+0000","last_fullsized":"2023-10-08T00:17:40.244822+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:24:15.333112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.258402+0000","last_change":"2023-10-08T00:17:40.258335+0000","last_active":"2023-10-08T00:17:40.258402+0000","last_peered":"2023-10-08T00:17:40.258402+0000","last_clean":"2023-10-08T00:17:40.258402+0000","last_became_active":"2023-10-08T00:17:40.258021+0000","last_became_peered":"2023-10-08T00:17:40.258021+0000","last_unstale":"2023-10-08T00:17:40.258402+0000","last_undegraded":"2023-10-08T00:17:40.258402+0000","last_fullsized":"2023-10-08T00:17:40.258402+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:40:24.777787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.239407+0000","last_change":"2023-10-08T00:17:40.239070+0000","last_active":"2023-10-08T00:17:40.239407+0000","last_peered":"2023-10-08T00:17:40.239407+0000","last_clean":"2023-10-08T00:17:40.239407+0000","last_became_active":"2023-10-08T00:17:40.238829+0000","last_became_peered":"2023-10-08T00:17:40.238829+0000","last_unstale":"2023-10-08T00:17:40.239407+0000","last_undegraded":"2023-10-08T00:17:40.239407+0000","last_fullsized":"2023-10-08T00:17:40.239407+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:42:33.653060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.245603+0000","last_change":"2023-10-08T00:17:40.245531+0000","last_active":"2023-10-08T00:17:40.245603+0000","last_peered":"2023-10-08T00:17:40.245603+0000","last_clean":"2023-10-08T00:17:40.245603+0000","last_became_active":"2023-10-08T00:17:40.245306+0000","last_became_peered":"2023-10-08T00:17:40.245306+0000","last_unstale":"2023-10-08T00:17:40.245603+0000","last_undegraded":"2023-10-08T00:17:40.245603+0000","last_fullsized":"2023-10-08T00:17:40.245603+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:43:13.797123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.245086+0000","last_change":"2023-10-08T00:17:40.244989+0000","last_active":"2023-10-08T00:17:40.245086+0000","last_peered":"2023-10-08T00:17:40.245086+0000","last_clean":"2023-10-08T00:17:40.245086+0000","last_became_active":"2023-10-08T00:17:40.244634+0000","last_became_peered":"2023-10-08T00:17:40.244634+0000","last_unstale":"2023-10-08T00:17:40.245086+0000","last_undegraded":"2023-10-08T00:17:40.245086+0000","last_fullsized":"2023-10-08T00:17:40.245086+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:23:21.315613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.254877+0000","last_change":"2023-10-08T00:17:40.254397+0000","last_active":"2023-10-08T00:17:40.254877+0000","last_peered":"2023-10-08T00:17:40.254877+0000","last_clean":"2023-10-08T00:17:40.254877+0000","last_became_active":"2023-10-08T00:17:40.248198+0000","last_became_peered":"2023-10-08T00:17:40.248198+0000","last_unstale":"2023-10-08T00:17:40.254877+0000","last_undegraded":"2023-10-08T00:17:40.254877+0000","last_fullsized":"2023-10-08T00:17:40.254877+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:55:57.192962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.246240+0000","last_change":"2023-10-08T00:17:40.246175+0000","last_active":"2023-10-08T00:17:40.246240+0000","last_peered":"2023-10-08T00:17:40.246240+0000","last_clean":"2023-10-08T00:17:40.246240+0000","last_became_active":"2023-10-08T00:17:40.245940+0000","last_became_peered":"2023-10-08T00:17:40.245940+0000","last_unstale":"2023-10-08T00:17:40.246240+0000","last_undegraded":"2023-10-08T00:17:40.246240+0000","last_fullsized":"2023-10-08T00:17:40.246240+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:22:03.771134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.246886+0000","last_change":"2023-10-08T00:17:40.246490+0000","last_active":"2023-10-08T00:17:40.246886+0000","last_peered":"2023-10-08T00:17:40.246886+0000","last_clean":"2023-10-08T00:17:40.246886+0000","last_became_active":"2023-10-08T00:17:40.244763+0000","last_became_peered":"2023-10-08T00:17:40.244763+0000","last_unstale":"2023-10-08T00:17:40.246886+0000","last_undegraded":"2023-10-08T00:17:40.246886+0000","last_fullsized":"2023-10-08T00:17:40.246886+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:09:08.837995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.253523+0000","last_change":"2023-10-08T00:17:40.253423+0000","last_active":"2023-10-08T00:17:40.253523+0000","last_peered":"2023-10-08T00:17:40.253523+0000","last_clean":"2023-10-08T00:17:40.253523+0000","last_became_active":"2023-10-08T00:17:40.252932+0000","last_became_peered":"2023-10-08T00:17:40.252932+0000","last_unstale":"2023-10-08T00:17:40.253523+0000","last_undegraded":"2023-10-08T00:17:40.253523+0000","last_fullsized":"2023-10-08T00:17:40.253523+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:37:58.966950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.249317+0000","last_change":"2023-10-08T00:17:40.249184+0000","last_active":"2023-10-08T00:17:40.249317+0000","last_peered":"2023-10-08T00:17:40.249317+0000","last_clean":"2023-10-08T00:17:40.249317+0000","last_became_active":"2023-10-08T00:17:40.248663+0000","last_became_peered":"2023-10-08T00:17:40.248663+0000","last_unstale":"2023-10-08T00:17:40.249317+0000","last_undegraded":"2023-10-08T00:17:40.249317+0000","last_fullsized":"2023-10-08T00:17:40.249317+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:24:21.948361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.243906+0000","last_change":"2023-10-08T00:17:40.239467+0000","last_active":"2023-10-08T00:17:40.243906+0000","last_peered":"2023-10-08T00:17:40.243906+0000","last_clean":"2023-10-08T00:17:40.243906+0000","last_became_active":"2023-10-08T00:17:40.238903+0000","last_became_peered":"2023-10-08T00:17:40.238903+0000","last_unstale":"2023-10-08T00:17:40.243906+0000","last_undegraded":"2023-10-08T00:17:40.243906+0000","last_fullsized":"2023-10-08T00:17:40.243906+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:12:58.622892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.247787+0000","last_change":"2023-10-08T00:17:40.247333+0000","last_active":"2023-10-08T00:17:40.247787+0000","last_peered":"2023-10-08T00:17:40.247787+0000","last_clean":"2023-10-08T00:17:40.247787+0000","last_became_active":"2023-10-08T00:17:40.246785+0000","last_became_peered":"2023-10-08T00:17:40.246785+0000","last_unstale":"2023-10-08T00:17:40.247787+0000","last_undegraded":"2023-10-08T00:17:40.247787+0000","last_fullsized":"2023-10-08T00:17:40.247787+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:51:35.954365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.243149+0000","last_change":"2023-10-08T00:17:40.243071+0000","last_active":"2023-10-08T00:17:40.243149+0000","last_peered":"2023-10-08T00:17:40.243149+0000","last_clean":"2023-10-08T00:17:40.243149+0000","last_became_active":"2023-10-08T00:17:40.242841+0000","last_became_peered":"2023-10-08T00:17:40.242841+0000","last_unstale":"2023-10-08T00:17:40.243149+0000","last_undegraded":"2023-10-08T00:17:40.243149+0000","last_fullsized":"2023-10-08T00:17:40.243149+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:05:20.180784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.243942+0000","last_change":"2023-10-08T00:17:40.240463+0000","last_active":"2023-10-08T00:17:40.243942+0000","last_peered":"2023-10-08T00:17:40.243942+0000","last_clean":"2023-10-08T00:17:40.243942+0000","last_became_active":"2023-10-08T00:17:40.240256+0000","last_became_peered":"2023-10-08T00:17:40.240256+0000","last_unstale":"2023-10-08T00:17:40.243942+0000","last_undegraded":"2023-10-08T00:17:40.243942+0000","last_fullsized":"2023-10-08T00:17:40.243942+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:57:32.743438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.248205+0000","last_change":"2023-10-08T00:17:40.247835+0000","last_active":"2023-10-08T00:17:40.248205+0000","last_peered":"2023-10-08T00:17:40.248205+0000","last_clean":"2023-10-08T00:17:40.248205+0000","last_became_active":"2023-10-08T00:17:40.246940+0000","last_became_peered":"2023-10-08T00:17:40.246940+0000","last_unstale":"2023-10-08T00:17:40.248205+0000","last_undegraded":"2023-10-08T00:17:40.248205+0000","last_fullsized":"2023-10-08T00:17:40.248205+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:51:35.954365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.243925+0000","last_change":"2023-10-08T00:17:40.239606+0000","last_active":"2023-10-08T00:17:40.243925+0000","last_peered":"2023-10-08T00:17:40.243925+0000","last_clean":"2023-10-08T00:17:40.243925+0000","last_became_active":"2023-10-08T00:17:40.239240+0000","last_became_peered":"2023-10-08T00:17:40.239240+0000","last_unstale":"2023-10-08T00:17:40.243925+0000","last_undegraded":"2023-10-08T00:17:40.243925+0000","last_fullsized":"2023-10-08T00:17:40.243925+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:57:38.260338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.243287+0000","last_change":"2023-10-08T00:17:40.243222+0000","last_active":"2023-10-08T00:17:40.243287+0000","last_peered":"2023-10-08T00:17:40.243287+0000","last_clean":"2023-10-08T00:17:40.243287+0000","last_became_active":"2023-10-08T00:17:40.242924+0000","last_became_peered":"2023-10-08T00:17:40.242924+0000","last_unstale":"2023-10-08T00:17:40.243287+0000","last_undegraded":"2023-10-08T00:17:40.243287+0000","last_fullsized":"2023-10-08T00:17:40.243287+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:55:38.261584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.258448+0000","last_change":"2023-10-08T00:17:40.258373+0000","last_active":"2023-10-08T00:17:40.258448+0000","last_peered":"2023-10-08T00:17:40.258448+0000","last_clean":"2023-10-08T00:17:40.258448+0000","last_became_active":"2023-10-08T00:17:40.256619+0000","last_became_peered":"2023-10-08T00:17:40.256619+0000","last_unstale":"2023-10-08T00:17:40.258448+0000","last_undegraded":"2023-10-08T00:17:40.258448+0000","last_fullsized":"2023-10-08T00:17:40.258448+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:32:33.029129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.254234+0000","last_change":"2023-10-08T00:17:40.253954+0000","last_active":"2023-10-08T00:17:40.254234+0000","last_peered":"2023-10-08T00:17:40.254234+0000","last_clean":"2023-10-08T00:17:40.254234+0000","last_became_active":"2023-10-08T00:17:40.253387+0000","last_became_peered":"2023-10-08T00:17:40.253387+0000","last_unstale":"2023-10-08T00:17:40.254234+0000","last_undegraded":"2023-10-08T00:17:40.254234+0000","last_fullsized":"2023-10-08T00:17:40.254234+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:36:20.674965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.243990+0000","last_change":"2023-10-08T00:17:40.243816+0000","last_active":"2023-10-08T00:17:40.243990+0000","last_peered":"2023-10-08T00:17:40.243990+0000","last_clean":"2023-10-08T00:17:40.243990+0000","last_became_active":"2023-10-08T00:17:40.243233+0000","last_became_peered":"2023-10-08T00:17:40.243233+0000","last_unstale":"2023-10-08T00:17:40.243990+0000","last_undegraded":"2023-10-08T00:17:40.243990+0000","last_fullsized":"2023-10-08T00:17:40.243990+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:33:51.997828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.253112+0000","last_change":"2023-10-08T00:17:40.250762+0000","last_active":"2023-10-08T00:17:40.253112+0000","last_peered":"2023-10-08T00:17:40.253112+0000","last_clean":"2023-10-08T00:17:40.253112+0000","last_became_active":"2023-10-08T00:17:40.250534+0000","last_became_peered":"2023-10-08T00:17:40.250534+0000","last_unstale":"2023-10-08T00:17:40.253112+0000","last_undegraded":"2023-10-08T00:17:40.253112+0000","last_fullsized":"2023-10-08T00:17:40.253112+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:33:38.865833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.258056+0000","last_change":"2023-10-08T00:17:40.257965+0000","last_active":"2023-10-08T00:17:40.258056+0000","last_peered":"2023-10-08T00:17:40.258056+0000","last_clean":"2023-10-08T00:17:40.258056+0000","last_became_active":"2023-10-08T00:17:40.257289+0000","last_became_peered":"2023-10-08T00:17:40.257289+0000","last_unstale":"2023-10-08T00:17:40.258056+0000","last_undegraded":"2023-10-08T00:17:40.258056+0000","last_fullsized":"2023-10-08T00:17:40.258056+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:58:26.467186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.245895+0000","last_change":"2023-10-08T00:17:40.245675+0000","last_active":"2023-10-08T00:17:40.245895+0000","last_peered":"2023-10-08T00:17:40.245895+0000","last_clean":"2023-10-08T00:17:40.245895+0000","last_became_active":"2023-10-08T00:17:40.245287+0000","last_became_peered":"2023-10-08T00:17:40.245287+0000","last_unstale":"2023-10-08T00:17:40.245895+0000","last_undegraded":"2023-10-08T00:17:40.245895+0000","last_fullsized":"2023-10-08T00:17:40.245895+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:40:39.003917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.252124+0000","last_change":"2023-10-08T00:17:40.251915+0000","last_active":"2023-10-08T00:17:40.252124+0000","last_peered":"2023-10-08T00:17:40.252124+0000","last_clean":"2023-10-08T00:17:40.252124+0000","last_became_active":"2023-10-08T00:17:40.251068+0000","last_became_peered":"2023-10-08T00:17:40.251068+0000","last_unstale":"2023-10-08T00:17:40.252124+0000","last_undegraded":"2023-10-08T00:17:40.252124+0000","last_fullsized":"2023-10-08T00:17:40.252124+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:37:32.516380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.248248+0000","last_change":"2023-10-08T00:17:40.248128+0000","last_active":"2023-10-08T00:17:40.248248+0000","last_peered":"2023-10-08T00:17:40.248248+0000","last_clean":"2023-10-08T00:17:40.248248+0000","last_became_active":"2023-10-08T00:17:40.247616+0000","last_became_peered":"2023-10-08T00:17:40.247616+0000","last_unstale":"2023-10-08T00:17:40.248248+0000","last_undegraded":"2023-10-08T00:17:40.248248+0000","last_fullsized":"2023-10-08T00:17:40.248248+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:54:35.040253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.254848+0000","last_change":"2023-10-08T00:17:40.254124+0000","last_active":"2023-10-08T00:17:40.254848+0000","last_peered":"2023-10-08T00:17:40.254848+0000","last_clean":"2023-10-08T00:17:40.254848+0000","last_became_active":"2023-10-08T00:17:40.247006+0000","last_became_peered":"2023-10-08T00:17:40.247006+0000","last_unstale":"2023-10-08T00:17:40.254848+0000","last_undegraded":"2023-10-08T00:17:40.254848+0000","last_fullsized":"2023-10-08T00:17:40.254848+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:29:56.427939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.250748+0000","last_change":"2023-10-08T00:17:40.250683+0000","last_active":"2023-10-08T00:17:40.250748+0000","last_peered":"2023-10-08T00:17:40.250748+0000","last_clean":"2023-10-08T00:17:40.250748+0000","last_became_active":"2023-10-08T00:17:40.250471+0000","last_became_peered":"2023-10-08T00:17:40.250471+0000","last_unstale":"2023-10-08T00:17:40.250748+0000","last_undegraded":"2023-10-08T00:17:40.250748+0000","last_fullsized":"2023-10-08T00:17:40.250748+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:43:28.092573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.250758+0000","last_change":"2023-10-08T00:17:40.250689+0000","last_active":"2023-10-08T00:17:40.250758+0000","last_peered":"2023-10-08T00:17:40.250758+0000","last_clean":"2023-10-08T00:17:40.250758+0000","last_became_active":"2023-10-08T00:17:40.250479+0000","last_became_peered":"2023-10-08T00:17:40.250479+0000","last_unstale":"2023-10-08T00:17:40.250758+0000","last_undegraded":"2023-10-08T00:17:40.250758+0000","last_fullsized":"2023-10-08T00:17:40.250758+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:05:25.070445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.249299+0000","last_change":"2023-10-08T00:17:40.248404+0000","last_active":"2023-10-08T00:17:40.249299+0000","last_peered":"2023-10-08T00:17:40.249299+0000","last_clean":"2023-10-08T00:17:40.249299+0000","last_became_active":"2023-10-08T00:17:40.248157+0000","last_became_peered":"2023-10-08T00:17:40.248157+0000","last_unstale":"2023-10-08T00:17:40.249299+0000","last_undegraded":"2023-10-08T00:17:40.249299+0000","last_fullsized":"2023-10-08T00:17:40.249299+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:30:30.805838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"153'4","reported_seq":14,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.259166+0000","last_change":"2023-10-08T00:17:39.242536+0000","last_active":"2023-10-08T00:17:40.259166+0000","last_peered":"2023-10-08T00:17:40.259166+0000","last_clean":"2023-10-08T00:17:40.259166+0000","last_became_active":"2023-10-08T00:17:39.242024+0000","last_became_peered":"2023-10-08T00:17:39.242024+0000","last_unstale":"2023-10-08T00:17:40.259166+0000","last_undegraded":"2023-10-08T00:17:40.259166+0000","last_fullsized":"2023-10-08T00:17:40.259166+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:53:34.765782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.242911+0000","last_change":"2023-10-08T00:17:40.242851+0000","last_active":"2023-10-08T00:17:40.242911+0000","last_peered":"2023-10-08T00:17:40.242911+0000","last_clean":"2023-10-08T00:17:40.242911+0000","last_became_active":"2023-10-08T00:17:40.242661+0000","last_became_peered":"2023-10-08T00:17:40.242661+0000","last_unstale":"2023-10-08T00:17:40.242911+0000","last_undegraded":"2023-10-08T00:17:40.242911+0000","last_fullsized":"2023-10-08T00:17:40.242911+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:21:25.611483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"153'1","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.254304+0000","last_change":"2023-10-08T00:17:39.235694+0000","last_active":"2023-10-08T00:17:40.254304+0000","last_peered":"2023-10-08T00:17:40.254304+0000","last_clean":"2023-10-08T00:17:40.254304+0000","last_became_active":"2023-10-08T00:17:39.235491+0000","last_became_peered":"2023-10-08T00:17:39.235491+0000","last_unstale":"2023-10-08T00:17:40.254304+0000","last_undegraded":"2023-10-08T00:17:40.254304+0000","last_fullsized":"2023-10-08T00:17:40.254304+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:26:49.727979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.251816+0000","last_change":"2023-10-08T00:17:40.251412+0000","last_active":"2023-10-08T00:17:40.251816+0000","last_peered":"2023-10-08T00:17:40.251816+0000","last_clean":"2023-10-08T00:17:40.251816+0000","last_became_active":"2023-10-08T00:17:40.251195+0000","last_became_peered":"2023-10-08T00:17:40.251195+0000","last_unstale":"2023-10-08T00:17:40.251816+0000","last_undegraded":"2023-10-08T00:17:40.251816+0000","last_fullsized":"2023-10-08T00:17:40.251816+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:21:32.636851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"153'2","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.253430+0000","last_change":"2023-10-08T00:17:39.240086+0000","last_active":"2023-10-08T00:17:40.253430+0000","last_peered":"2023-10-08T00:17:40.253430+0000","last_clean":"2023-10-08T00:17:40.253430+0000","last_became_active":"2023-10-08T00:17:39.239861+0000","last_became_peered":"2023-10-08T00:17:39.239861+0000","last_unstale":"2023-10-08T00:17:40.253430+0000","last_undegraded":"2023-10-08T00:17:40.253430+0000","last_fullsized":"2023-10-08T00:17:40.253430+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:43:12.792670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.245880+0000","last_change":"2023-10-08T00:17:40.244740+0000","last_active":"2023-10-08T00:17:40.245880+0000","last_peered":"2023-10-08T00:17:40.245880+0000","last_clean":"2023-10-08T00:17:40.245880+0000","last_became_active":"2023-10-08T00:17:40.244542+0000","last_became_peered":"2023-10-08T00:17:40.244542+0000","last_unstale":"2023-10-08T00:17:40.245880+0000","last_undegraded":"2023-10-08T00:17:40.245880+0000","last_fullsized":"2023-10-08T00:17:40.245880+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:45:05.381563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1c","version":"153'2","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.259353+0000","last_change":"2023-10-08T00:17:39.243527+0000","last_active":"2023-10-08T00:17:40.259353+0000","last_peered":"2023-10-08T00:17:40.259353+0000","last_clean":"2023-10-08T00:17:40.259353+0000","last_became_active":"2023-10-08T00:17:39.243275+0000","last_became_peered":"2023-10-08T00:17:39.243275+0000","last_unstale":"2023-10-08T00:17:40.259353+0000","last_undegraded":"2023-10-08T00:17:40.259353+0000","last_fullsized":"2023-10-08T00:17:40.259353+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:34:21.713614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.238189+0000","last_change":"2023-10-08T00:17:40.238121+0000","last_active":"2023-10-08T00:17:40.238189+0000","last_peered":"2023-10-08T00:17:40.238189+0000","last_clean":"2023-10-08T00:17:40.238189+0000","last_became_active":"2023-10-08T00:17:40.237877+0000","last_became_peered":"2023-10-08T00:17:40.237877+0000","last_unstale":"2023-10-08T00:17:40.238189+0000","last_undegraded":"2023-10-08T00:17:40.238189+0000","last_fullsized":"2023-10-08T00:17:40.238189+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:06:11.047879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1b","version":"153'1","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.259689+0000","last_change":"2023-10-08T00:17:39.246860+0000","last_active":"2023-10-08T00:17:40.259689+0000","last_peered":"2023-10-08T00:17:40.259689+0000","last_clean":"2023-10-08T00:17:40.259689+0000","last_became_active":"2023-10-08T00:17:39.245722+0000","last_became_peered":"2023-10-08T00:17:39.245722+0000","last_unstale":"2023-10-08T00:17:40.259689+0000","last_undegraded":"2023-10-08T00:17:40.259689+0000","last_fullsized":"2023-10-08T00:17:40.259689+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:26:05.256919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.245900+0000","last_change":"2023-10-08T00:17:40.244945+0000","last_active":"2023-10-08T00:17:40.245900+0000","last_peered":"2023-10-08T00:17:40.245900+0000","last_clean":"2023-10-08T00:17:40.245900+0000","last_became_active":"2023-10-08T00:17:40.244072+0000","last_became_peered":"2023-10-08T00:17:40.244072+0000","last_unstale":"2023-10-08T00:17:40.245900+0000","last_undegraded":"2023-10-08T00:17:40.245900+0000","last_fullsized":"2023-10-08T00:17:40.245900+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:51:35.954365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"153'1","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.256127+0000","last_change":"2023-10-08T00:17:39.241419+0000","last_active":"2023-10-08T00:17:40.256127+0000","last_peered":"2023-10-08T00:17:40.256127+0000","last_clean":"2023-10-08T00:17:40.256127+0000","last_became_active":"2023-10-08T00:17:39.241171+0000","last_became_peered":"2023-10-08T00:17:39.241171+0000","last_unstale":"2023-10-08T00:17:40.256127+0000","last_undegraded":"2023-10-08T00:17:40.256127+0000","last_fullsized":"2023-10-08T00:17:40.256127+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:09:45.565409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.253944+0000","last_change":"2023-10-08T00:17:40.251476+0000","last_active":"2023-10-08T00:17:40.253944+0000","last_peered":"2023-10-08T00:17:40.253944+0000","last_clean":"2023-10-08T00:17:40.253944+0000","last_became_active":"2023-10-08T00:17:40.251249+0000","last_became_peered":"2023-10-08T00:17:40.251249+0000","last_unstale":"2023-10-08T00:17:40.253944+0000","last_undegraded":"2023-10-08T00:17:40.253944+0000","last_fullsized":"2023-10-08T00:17:40.253944+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:59:35.355008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.239229+0000","last_change":"2023-10-08T00:17:39.242323+0000","last_active":"2023-10-08T00:17:40.239229+0000","last_peered":"2023-10-08T00:17:40.239229+0000","last_clean":"2023-10-08T00:17:40.239229+0000","last_became_active":"2023-10-08T00:17:39.242141+0000","last_became_peered":"2023-10-08T00:17:39.242141+0000","last_unstale":"2023-10-08T00:17:40.239229+0000","last_undegraded":"2023-10-08T00:17:40.239229+0000","last_fullsized":"2023-10-08T00:17:40.239229+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:39:55.887101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.248232+0000","last_change":"2023-10-08T00:17:40.247988+0000","last_active":"2023-10-08T00:17:40.248232+0000","last_peered":"2023-10-08T00:17:40.248232+0000","last_clean":"2023-10-08T00:17:40.248232+0000","last_became_active":"2023-10-08T00:17:40.247312+0000","last_became_peered":"2023-10-08T00:17:40.247312+0000","last_unstale":"2023-10-08T00:17:40.248232+0000","last_undegraded":"2023-10-08T00:17:40.248232+0000","last_fullsized":"2023-10-08T00:17:40.248232+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:57:32.743438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.242602+0000","last_change":"2023-10-08T00:17:39.247556+0000","last_active":"2023-10-08T00:17:40.242602+0000","last_peered":"2023-10-08T00:17:40.242602+0000","last_clean":"2023-10-08T00:17:40.242602+0000","last_became_active":"2023-10-08T00:17:39.246978+0000","last_became_peered":"2023-10-08T00:17:39.246978+0000","last_unstale":"2023-10-08T00:17:40.242602+0000","last_undegraded":"2023-10-08T00:17:40.242602+0000","last_fullsized":"2023-10-08T00:17:40.242602+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:02:20.033189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.258618+0000","last_change":"2023-10-08T00:17:40.258531+0000","last_active":"2023-10-08T00:17:40.258618+0000","last_peered":"2023-10-08T00:17:40.258618+0000","last_clean":"2023-10-08T00:17:40.258618+0000","last_became_active":"2023-10-08T00:17:40.256967+0000","last_became_peered":"2023-10-08T00:17:40.256967+0000","last_unstale":"2023-10-08T00:17:40.258618+0000","last_undegraded":"2023-10-08T00:17:40.258618+0000","last_fullsized":"2023-10-08T00:17:40.258618+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:57:49.413076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.242788+0000","last_change":"2023-10-08T00:17:39.246910+0000","last_active":"2023-10-08T00:17:40.242788+0000","last_peered":"2023-10-08T00:17:40.242788+0000","last_clean":"2023-10-08T00:17:40.242788+0000","last_became_active":"2023-10-08T00:17:39.246350+0000","last_became_peered":"2023-10-08T00:17:39.246350+0000","last_unstale":"2023-10-08T00:17:40.242788+0000","last_undegraded":"2023-10-08T00:17:40.242788+0000","last_fullsized":"2023-10-08T00:17:40.242788+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:33:50.993375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.238214+0000","last_change":"2023-10-08T00:17:40.238153+0000","last_active":"2023-10-08T00:17:40.238214+0000","last_peered":"2023-10-08T00:17:40.238214+0000","last_clean":"2023-10-08T00:17:40.238214+0000","last_became_active":"2023-10-08T00:17:40.237924+0000","last_became_peered":"2023-10-08T00:17:40.237924+0000","last_unstale":"2023-10-08T00:17:40.238214+0000","last_undegraded":"2023-10-08T00:17:40.238214+0000","last_fullsized":"2023-10-08T00:17:40.238214+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:22:40.363368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.237011+0000","last_change":"2023-10-08T00:17:39.245641+0000","last_active":"2023-10-08T00:17:40.237011+0000","last_peered":"2023-10-08T00:17:40.237011+0000","last_clean":"2023-10-08T00:17:40.237011+0000","last_became_active":"2023-10-08T00:17:39.245352+0000","last_became_peered":"2023-10-08T00:17:39.245352+0000","last_unstale":"2023-10-08T00:17:40.237011+0000","last_undegraded":"2023-10-08T00:17:40.237011+0000","last_fullsized":"2023-10-08T00:17:40.237011+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:26:05.256919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.246903+0000","last_change":"2023-10-08T00:17:40.246703+0000","last_active":"2023-10-08T00:17:40.246903+0000","last_peered":"2023-10-08T00:17:40.246903+0000","last_clean":"2023-10-08T00:17:40.246903+0000","last_became_active":"2023-10-08T00:17:40.246141+0000","last_became_peered":"2023-10-08T00:17:40.246141+0000","last_unstale":"2023-10-08T00:17:40.246903+0000","last_undegraded":"2023-10-08T00:17:40.246903+0000","last_fullsized":"2023-10-08T00:17:40.246903+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:34:52.462946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.15","version":"153'3","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.259024+0000","last_change":"2023-10-08T00:17:39.242676+0000","last_active":"2023-10-08T00:17:40.259024+0000","last_peered":"2023-10-08T00:17:40.259024+0000","last_clean":"2023-10-08T00:17:40.259024+0000","last_became_active":"2023-10-08T00:17:39.242315+0000","last_became_peered":"2023-10-08T00:17:39.242315+0000","last_unstale":"2023-10-08T00:17:40.259024+0000","last_undegraded":"2023-10-08T00:17:40.259024+0000","last_fullsized":"2023-10-08T00:17:40.259024+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:21:35.315638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.253269+0000","last_change":"2023-10-08T00:17:40.253208+0000","last_active":"2023-10-08T00:17:40.253269+0000","last_peered":"2023-10-08T00:17:40.253269+0000","last_clean":"2023-10-08T00:17:40.253269+0000","last_became_active":"2023-10-08T00:17:40.251582+0000","last_became_peered":"2023-10-08T00:17:40.251582+0000","last_unstale":"2023-10-08T00:17:40.253269+0000","last_undegraded":"2023-10-08T00:17:40.253269+0000","last_fullsized":"2023-10-08T00:17:40.253269+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:47:58.933828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"153'6","reported_seq":16,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.260501+0000","last_change":"2023-10-08T00:17:39.245646+0000","last_active":"2023-10-08T00:17:40.260501+0000","last_peered":"2023-10-08T00:17:40.260501+0000","last_clean":"2023-10-08T00:17:40.260501+0000","last_became_active":"2023-10-08T00:17:39.245356+0000","last_became_peered":"2023-10-08T00:17:39.245356+0000","last_unstale":"2023-10-08T00:17:40.260501+0000","last_undegraded":"2023-10-08T00:17:40.260501+0000","last_fullsized":"2023-10-08T00:17:40.260501+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:37:01.074871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.249335+0000","last_change":"2023-10-08T00:17:40.249199+0000","last_active":"2023-10-08T00:17:40.249335+0000","last_peered":"2023-10-08T00:17:40.249335+0000","last_clean":"2023-10-08T00:17:40.249335+0000","last_became_active":"2023-10-08T00:17:40.248923+0000","last_became_peered":"2023-10-08T00:17:40.248923+0000","last_unstale":"2023-10-08T00:17:40.249335+0000","last_undegraded":"2023-10-08T00:17:40.249335+0000","last_fullsized":"2023-10-08T00:17:40.249335+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:59:35.355008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.244703+0000","last_change":"2023-10-08T00:17:39.242832+0000","last_active":"2023-10-08T00:17:40.244703+0000","last_peered":"2023-10-08T00:17:40.244703+0000","last_clean":"2023-10-08T00:17:40.244703+0000","last_became_active":"2023-10-08T00:17:39.242603+0000","last_became_peered":"2023-10-08T00:17:39.242603+0000","last_unstale":"2023-10-08T00:17:40.244703+0000","last_undegraded":"2023-10-08T00:17:40.244703+0000","last_fullsized":"2023-10-08T00:17:40.244703+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:07:09.770455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.243062+0000","last_change":"2023-10-08T00:17:40.242863+0000","last_active":"2023-10-08T00:17:40.243062+0000","last_peered":"2023-10-08T00:17:40.243062+0000","last_clean":"2023-10-08T00:17:40.243062+0000","last_became_active":"2023-10-08T00:17:40.242610+0000","last_became_peered":"2023-10-08T00:17:40.242610+0000","last_unstale":"2023-10-08T00:17:40.243062+0000","last_undegraded":"2023-10-08T00:17:40.243062+0000","last_fullsized":"2023-10-08T00:17:40.243062+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:22:03.771134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.235924+0000","last_change":"2023-10-08T00:17:39.236771+0000","last_active":"2023-10-08T00:17:40.235924+0000","last_peered":"2023-10-08T00:17:40.235924+0000","last_clean":"2023-10-08T00:17:40.235924+0000","last_became_active":"2023-10-08T00:17:39.236530+0000","last_became_peered":"2023-10-08T00:17:39.236530+0000","last_unstale":"2023-10-08T00:17:40.235924+0000","last_undegraded":"2023-10-08T00:17:40.235924+0000","last_fullsized":"2023-10-08T00:17:40.235924+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:22:39.358915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.245915+0000","last_change":"2023-10-08T00:17:40.245067+0000","last_active":"2023-10-08T00:17:40.245915+0000","last_peered":"2023-10-08T00:17:40.245915+0000","last_clean":"2023-10-08T00:17:40.245915+0000","last_became_active":"2023-10-08T00:17:40.244498+0000","last_became_peered":"2023-10-08T00:17:40.244498+0000","last_unstale":"2023-10-08T00:17:40.245915+0000","last_undegraded":"2023-10-08T00:17:40.245915+0000","last_fullsized":"2023-10-08T00:17:40.245915+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:40:39.003917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.10","version":"153'8","reported_seq":18,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.255994+0000","last_change":"2023-10-08T00:17:39.245625+0000","last_active":"2023-10-08T00:17:40.255994+0000","last_peered":"2023-10-08T00:17:40.255994+0000","last_clean":"2023-10-08T00:17:40.255994+0000","last_became_active":"2023-10-08T00:17:39.245016+0000","last_became_peered":"2023-10-08T00:17:39.245016+0000","last_unstale":"2023-10-08T00:17:40.255994+0000","last_undegraded":"2023-10-08T00:17:40.255994+0000","last_fullsized":"2023-10-08T00:17:40.255994+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:57:31.738986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.244805+0000","last_change":"2023-10-08T00:17:40.244567+0000","last_active":"2023-10-08T00:17:40.244805+0000","last_peered":"2023-10-08T00:17:40.244805+0000","last_clean":"2023-10-08T00:17:40.244805+0000","last_became_active":"2023-10-08T00:17:40.244034+0000","last_became_peered":"2023-10-08T00:17:40.244034+0000","last_unstale":"2023-10-08T00:17:40.244805+0000","last_undegraded":"2023-10-08T00:17:40.244805+0000","last_fullsized":"2023-10-08T00:17:40.244805+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:58:26.467186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"153'1","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.254012+0000","last_change":"2023-10-08T00:17:39.245330+0000","last_active":"2023-10-08T00:17:40.254012+0000","last_peered":"2023-10-08T00:17:40.254012+0000","last_clean":"2023-10-08T00:17:40.254012+0000","last_became_active":"2023-10-08T00:17:39.244579+0000","last_became_peered":"2023-10-08T00:17:39.244579+0000","last_unstale":"2023-10-08T00:17:40.254012+0000","last_undegraded":"2023-10-08T00:17:40.254012+0000","last_fullsized":"2023-10-08T00:17:40.254012+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:36:23.511902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.238774+0000","last_change":"2023-10-08T00:17:40.238724+0000","last_active":"2023-10-08T00:17:40.238774+0000","last_peered":"2023-10-08T00:17:40.238774+0000","last_clean":"2023-10-08T00:17:40.238774+0000","last_became_active":"2023-10-08T00:17:40.238508+0000","last_became_peered":"2023-10-08T00:17:40.238508+0000","last_unstale":"2023-10-08T00:17:40.238774+0000","last_undegraded":"2023-10-08T00:17:40.238774+0000","last_fullsized":"2023-10-08T00:17:40.238774+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:07:10.774907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"153'2","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.253631+0000","last_change":"2023-10-08T00:17:39.241807+0000","last_active":"2023-10-08T00:17:40.253631+0000","last_peered":"2023-10-08T00:17:40.253631+0000","last_clean":"2023-10-08T00:17:40.253631+0000","last_became_active":"2023-10-08T00:17:39.241530+0000","last_became_peered":"2023-10-08T00:17:39.241530+0000","last_unstale":"2023-10-08T00:17:40.253631+0000","last_undegraded":"2023-10-08T00:17:40.253631+0000","last_fullsized":"2023-10-08T00:17:40.253631+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:58:25.462734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.258964+0000","last_change":"2023-10-08T00:17:40.258872+0000","last_active":"2023-10-08T00:17:40.258964+0000","last_peered":"2023-10-08T00:17:40.258964+0000","last_clean":"2023-10-08T00:17:40.258964+0000","last_became_active":"2023-10-08T00:17:40.257807+0000","last_became_peered":"2023-10-08T00:17:40.257807+0000","last_unstale":"2023-10-08T00:17:40.258964+0000","last_undegraded":"2023-10-08T00:17:40.258964+0000","last_fullsized":"2023-10-08T00:17:40.258964+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:22:03.771134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"153'1","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.253847+0000","last_change":"2023-10-08T00:17:39.242761+0000","last_active":"2023-10-08T00:17:40.253847+0000","last_peered":"2023-10-08T00:17:40.253847+0000","last_clean":"2023-10-08T00:17:40.253847+0000","last_became_active":"2023-10-08T00:17:39.241966+0000","last_became_peered":"2023-10-08T00:17:39.241966+0000","last_unstale":"2023-10-08T00:17:40.253847+0000","last_undegraded":"2023-10-08T00:17:40.253847+0000","last_fullsized":"2023-10-08T00:17:40.253847+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:26:48.239749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.257905+0000","last_change":"2023-10-08T00:17:40.257615+0000","last_active":"2023-10-08T00:17:40.257905+0000","last_peered":"2023-10-08T00:17:40.257905+0000","last_clean":"2023-10-08T00:17:40.257905+0000","last_became_active":"2023-10-08T00:17:40.256003+0000","last_became_peered":"2023-10-08T00:17:40.256003+0000","last_unstale":"2023-10-08T00:17:40.257905+0000","last_undegraded":"2023-10-08T00:17:40.257905+0000","last_fullsized":"2023-10-08T00:17:40.257905+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:22:40.363368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.235362+0000","last_change":"2023-10-08T00:17:39.241610+0000","last_active":"2023-10-08T00:17:40.235362+0000","last_peered":"2023-10-08T00:17:40.235362+0000","last_clean":"2023-10-08T00:17:40.235362+0000","last_became_active":"2023-10-08T00:17:39.239512+0000","last_became_peered":"2023-10-08T00:17:39.239512+0000","last_unstale":"2023-10-08T00:17:40.235362+0000","last_undegraded":"2023-10-08T00:17:40.235362+0000","last_fullsized":"2023-10-08T00:17:40.235362+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:01:21.166679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.256354+0000","last_change":"2023-10-08T00:17:40.256287+0000","last_active":"2023-10-08T00:17:40.256354+0000","last_peered":"2023-10-08T00:17:40.256354+0000","last_clean":"2023-10-08T00:17:40.256354+0000","last_became_active":"2023-10-08T00:17:40.255898+0000","last_became_peered":"2023-10-08T00:17:40.255898+0000","last_unstale":"2023-10-08T00:17:40.256354+0000","last_undegraded":"2023-10-08T00:17:40.256354+0000","last_fullsized":"2023-10-08T00:17:40.256354+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:33:38.865833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"153'2","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.253654+0000","last_change":"2023-10-08T00:17:39.242251+0000","last_active":"2023-10-08T00:17:40.253654+0000","last_peered":"2023-10-08T00:17:40.253654+0000","last_clean":"2023-10-08T00:17:40.253654+0000","last_became_active":"2023-10-08T00:17:39.241975+0000","last_became_peered":"2023-10-08T00:17:39.241975+0000","last_unstale":"2023-10-08T00:17:40.253654+0000","last_undegraded":"2023-10-08T00:17:40.253654+0000","last_fullsized":"2023-10-08T00:17:40.253654+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:18:13.853377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.253168+0000","last_change":"2023-10-08T00:17:40.252101+0000","last_active":"2023-10-08T00:17:40.253168+0000","last_peered":"2023-10-08T00:17:40.253168+0000","last_clean":"2023-10-08T00:17:40.253168+0000","last_became_active":"2023-10-08T00:17:40.251300+0000","last_became_peered":"2023-10-08T00:17:40.251300+0000","last_unstale":"2023-10-08T00:17:40.253168+0000","last_undegraded":"2023-10-08T00:17:40.253168+0000","last_fullsized":"2023-10-08T00:17:40.253168+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:53:42.248903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.235995+0000","last_change":"2023-10-08T00:17:39.243908+0000","last_active":"2023-10-08T00:17:40.235995+0000","last_peered":"2023-10-08T00:17:40.235995+0000","last_clean":"2023-10-08T00:17:40.235995+0000","last_became_active":"2023-10-08T00:17:39.243671+0000","last_became_peered":"2023-10-08T00:17:39.243671+0000","last_unstale":"2023-10-08T00:17:40.235995+0000","last_undegraded":"2023-10-08T00:17:40.235995+0000","last_fullsized":"2023-10-08T00:17:40.235995+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:39:56.549324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.247806+0000","last_change":"2023-10-08T00:17:40.247494+0000","last_active":"2023-10-08T00:17:40.247806+0000","last_peered":"2023-10-08T00:17:40.247806+0000","last_clean":"2023-10-08T00:17:40.247806+0000","last_became_active":"2023-10-08T00:17:40.247106+0000","last_became_peered":"2023-10-08T00:17:40.247106+0000","last_unstale":"2023-10-08T00:17:40.247806+0000","last_undegraded":"2023-10-08T00:17:40.247806+0000","last_fullsized":"2023-10-08T00:17:40.247806+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:29:56.427939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.242280+0000","last_change":"2023-10-08T00:17:39.248289+0000","last_active":"2023-10-08T00:17:40.242280+0000","last_peered":"2023-10-08T00:17:40.242280+0000","last_clean":"2023-10-08T00:17:40.242280+0000","last_became_active":"2023-10-08T00:17:39.248014+0000","last_became_peered":"2023-10-08T00:17:39.248014+0000","last_unstale":"2023-10-08T00:17:40.242280+0000","last_undegraded":"2023-10-08T00:17:40.242280+0000","last_fullsized":"2023-10-08T00:17:40.242280+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:58:25.462734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.245930+0000","last_change":"2023-10-08T00:17:40.245809+0000","last_active":"2023-10-08T00:17:40.245930+0000","last_peered":"2023-10-08T00:17:40.245930+0000","last_clean":"2023-10-08T00:17:40.245930+0000","last_became_active":"2023-10-08T00:17:40.245131+0000","last_became_peered":"2023-10-08T00:17:40.245131+0000","last_unstale":"2023-10-08T00:17:40.245930+0000","last_undegraded":"2023-10-08T00:17:40.245930+0000","last_fullsized":"2023-10-08T00:17:40.245930+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:11:59.235622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.237238+0000","last_change":"2023-10-08T00:17:39.243240+0000","last_active":"2023-10-08T00:17:40.237238+0000","last_peered":"2023-10-08T00:17:40.237238+0000","last_clean":"2023-10-08T00:17:40.237238+0000","last_became_active":"2023-10-08T00:17:39.242906+0000","last_became_peered":"2023-10-08T00:17:39.242906+0000","last_unstale":"2023-10-08T00:17:40.237238+0000","last_undegraded":"2023-10-08T00:17:40.237238+0000","last_fullsized":"2023-10-08T00:17:40.237238+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:42:32.648608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.251861+0000","last_change":"2023-10-08T00:17:40.251800+0000","last_active":"2023-10-08T00:17:40.251861+0000","last_peered":"2023-10-08T00:17:40.251861+0000","last_clean":"2023-10-08T00:17:40.251861+0000","last_became_active":"2023-10-08T00:17:40.250816+0000","last_became_peered":"2023-10-08T00:17:40.250816+0000","last_unstale":"2023-10-08T00:17:40.251861+0000","last_undegraded":"2023-10-08T00:17:40.251861+0000","last_fullsized":"2023-10-08T00:17:40.251861+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:06:58.458653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"153'2","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.254730+0000","last_change":"2023-10-08T00:17:39.244806+0000","last_active":"2023-10-08T00:17:40.254730+0000","last_peered":"2023-10-08T00:17:40.254730+0000","last_clean":"2023-10-08T00:17:40.254730+0000","last_became_active":"2023-10-08T00:17:39.244600+0000","last_became_peered":"2023-10-08T00:17:39.244600+0000","last_unstale":"2023-10-08T00:17:40.254730+0000","last_undegraded":"2023-10-08T00:17:40.254730+0000","last_fullsized":"2023-10-08T00:17:40.254730+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:14:31.718231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.259112+0000","last_change":"2023-10-08T00:17:40.259048+0000","last_active":"2023-10-08T00:17:40.259112+0000","last_peered":"2023-10-08T00:17:40.259112+0000","last_clean":"2023-10-08T00:17:40.259112+0000","last_became_active":"2023-10-08T00:17:40.258119+0000","last_became_peered":"2023-10-08T00:17:40.258119+0000","last_unstale":"2023-10-08T00:17:40.259112+0000","last_undegraded":"2023-10-08T00:17:40.259112+0000","last_fullsized":"2023-10-08T00:17:40.259112+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:37:58.966950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.235657+0000","last_change":"2023-10-08T00:17:39.247178+0000","last_active":"2023-10-08T00:17:40.235657+0000","last_peered":"2023-10-08T00:17:40.235657+0000","last_clean":"2023-10-08T00:17:40.235657+0000","last_became_active":"2023-10-08T00:17:39.246939+0000","last_became_peered":"2023-10-08T00:17:39.246939+0000","last_unstale":"2023-10-08T00:17:40.235657+0000","last_undegraded":"2023-10-08T00:17:40.235657+0000","last_fullsized":"2023-10-08T00:17:40.235657+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:35:11.105844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.254846+0000","last_change":"2023-10-08T00:17:40.254779+0000","last_active":"2023-10-08T00:17:40.254846+0000","last_peered":"2023-10-08T00:17:40.254846+0000","last_clean":"2023-10-08T00:17:40.254846+0000","last_became_active":"2023-10-08T00:17:40.254566+0000","last_became_peered":"2023-10-08T00:17:40.254566+0000","last_unstale":"2023-10-08T00:17:40.254846+0000","last_undegraded":"2023-10-08T00:17:40.254846+0000","last_fullsized":"2023-10-08T00:17:40.254846+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:09:08.837995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.235898+0000","last_change":"2023-10-08T00:17:39.247837+0000","last_active":"2023-10-08T00:17:40.235898+0000","last_peered":"2023-10-08T00:17:40.235898+0000","last_clean":"2023-10-08T00:17:40.235898+0000","last_became_active":"2023-10-08T00:17:39.247595+0000","last_became_peered":"2023-10-08T00:17:39.247595+0000","last_unstale":"2023-10-08T00:17:40.235898+0000","last_undegraded":"2023-10-08T00:17:40.235898+0000","last_fullsized":"2023-10-08T00:17:40.235898+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:34:21.713614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.244907+0000","last_change":"2023-10-08T00:17:40.244842+0000","last_active":"2023-10-08T00:17:40.244907+0000","last_peered":"2023-10-08T00:17:40.244907+0000","last_clean":"2023-10-08T00:17:40.244907+0000","last_became_active":"2023-10-08T00:17:40.244351+0000","last_became_peered":"2023-10-08T00:17:40.244351+0000","last_unstale":"2023-10-08T00:17:40.244907+0000","last_undegraded":"2023-10-08T00:17:40.244907+0000","last_fullsized":"2023-10-08T00:17:40.244907+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:25:09.195876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.242386+0000","last_change":"2023-10-08T00:17:39.243249+0000","last_active":"2023-10-08T00:17:40.242386+0000","last_peered":"2023-10-08T00:17:40.242386+0000","last_clean":"2023-10-08T00:17:40.242386+0000","last_became_active":"2023-10-08T00:17:39.242464+0000","last_became_peered":"2023-10-08T00:17:39.242464+0000","last_unstale":"2023-10-08T00:17:40.242386+0000","last_undegraded":"2023-10-08T00:17:40.242386+0000","last_fullsized":"2023-10-08T00:17:40.242386+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:57:48.408623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.245862+0000","last_change":"2023-10-08T00:17:40.244292+0000","last_active":"2023-10-08T00:17:40.245862+0000","last_peered":"2023-10-08T00:17:40.245862+0000","last_clean":"2023-10-08T00:17:40.245862+0000","last_became_active":"2023-10-08T00:17:40.243716+0000","last_became_peered":"2023-10-08T00:17:40.243716+0000","last_unstale":"2023-10-08T00:17:40.245862+0000","last_undegraded":"2023-10-08T00:17:40.245862+0000","last_fullsized":"2023-10-08T00:17:40.245862+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:24:21.948361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.241028+0000","last_change":"2023-10-08T00:17:39.248190+0000","last_active":"2023-10-08T00:17:40.241028+0000","last_peered":"2023-10-08T00:17:40.241028+0000","last_clean":"2023-10-08T00:17:40.241028+0000","last_became_active":"2023-10-08T00:17:39.247710+0000","last_became_peered":"2023-10-08T00:17:39.247710+0000","last_unstale":"2023-10-08T00:17:40.241028+0000","last_undegraded":"2023-10-08T00:17:40.241028+0000","last_fullsized":"2023-10-08T00:17:40.241028+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:42:45.677981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.246381+0000","last_change":"2023-10-08T00:17:40.246334+0000","last_active":"2023-10-08T00:17:40.246381+0000","last_peered":"2023-10-08T00:17:40.246381+0000","last_clean":"2023-10-08T00:17:40.246381+0000","last_became_active":"2023-10-08T00:17:40.246112+0000","last_became_peered":"2023-10-08T00:17:40.246112+0000","last_unstale":"2023-10-08T00:17:40.246381+0000","last_undegraded":"2023-10-08T00:17:40.246381+0000","last_fullsized":"2023-10-08T00:17:40.246381+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:39:34.388679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.243525+0000","last_change":"2023-10-08T00:17:39.247326+0000","last_active":"2023-10-08T00:17:40.243525+0000","last_peered":"2023-10-08T00:17:40.243525+0000","last_clean":"2023-10-08T00:17:40.243525+0000","last_became_active":"2023-10-08T00:17:39.247081+0000","last_became_peered":"2023-10-08T00:17:39.247081+0000","last_unstale":"2023-10-08T00:17:40.243525+0000","last_undegraded":"2023-10-08T00:17:40.243525+0000","last_fullsized":"2023-10-08T00:17:40.243525+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:22:39.358915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.244248+0000","last_change":"2023-10-08T00:17:40.243986+0000","last_active":"2023-10-08T00:17:40.244248+0000","last_peered":"2023-10-08T00:17:40.244248+0000","last_clean":"2023-10-08T00:17:40.244248+0000","last_became_active":"2023-10-08T00:17:40.243578+0000","last_became_peered":"2023-10-08T00:17:40.243578+0000","last_unstale":"2023-10-08T00:17:40.244248+0000","last_undegraded":"2023-10-08T00:17:40.244248+0000","last_fullsized":"2023-10-08T00:17:40.244248+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:36:24.516355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.237911+0000","last_change":"2023-10-08T00:17:39.244659+0000","last_active":"2023-10-08T00:17:40.237911+0000","last_peered":"2023-10-08T00:17:40.237911+0000","last_clean":"2023-10-08T00:17:40.237911+0000","last_became_active":"2023-10-08T00:17:39.244411+0000","last_became_peered":"2023-10-08T00:17:39.244411+0000","last_unstale":"2023-10-08T00:17:40.237911+0000","last_undegraded":"2023-10-08T00:17:40.237911+0000","last_fullsized":"2023-10-08T00:17:40.237911+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:07:09.770455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.253213+0000","last_change":"2023-10-08T00:17:40.253156+0000","last_active":"2023-10-08T00:17:40.253213+0000","last_peered":"2023-10-08T00:17:40.253213+0000","last_clean":"2023-10-08T00:17:40.253213+0000","last_became_active":"2023-10-08T00:17:40.252770+0000","last_became_peered":"2023-10-08T00:17:40.252770+0000","last_unstale":"2023-10-08T00:17:40.253213+0000","last_undegraded":"2023-10-08T00:17:40.253213+0000","last_fullsized":"2023-10-08T00:17:40.253213+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:55:38.261584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.244676+0000","last_change":"2023-10-08T00:17:39.240855+0000","last_active":"2023-10-08T00:17:40.244676+0000","last_peered":"2023-10-08T00:17:40.244676+0000","last_clean":"2023-10-08T00:17:40.244676+0000","last_became_active":"2023-10-08T00:17:39.240622+0000","last_became_peered":"2023-10-08T00:17:39.240622+0000","last_unstale":"2023-10-08T00:17:40.244676+0000","last_undegraded":"2023-10-08T00:17:40.244676+0000","last_fullsized":"2023-10-08T00:17:40.244676+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:22:39.358915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.242556+0000","last_change":"2023-10-08T00:17:40.242493+0000","last_active":"2023-10-08T00:17:40.242556+0000","last_peered":"2023-10-08T00:17:40.242556+0000","last_clean":"2023-10-08T00:17:40.242556+0000","last_became_active":"2023-10-08T00:17:40.242276+0000","last_became_peered":"2023-10-08T00:17:40.242276+0000","last_unstale":"2023-10-08T00:17:40.242556+0000","last_undegraded":"2023-10-08T00:17:40.242556+0000","last_fullsized":"2023-10-08T00:17:40.242556+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:26:46.315710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"153'2","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.254486+0000","last_change":"2023-10-08T00:17:39.246049+0000","last_active":"2023-10-08T00:17:40.254486+0000","last_peered":"2023-10-08T00:17:40.254486+0000","last_clean":"2023-10-08T00:17:40.254486+0000","last_became_active":"2023-10-08T00:17:39.245601+0000","last_became_peered":"2023-10-08T00:17:39.245601+0000","last_unstale":"2023-10-08T00:17:40.254486+0000","last_undegraded":"2023-10-08T00:17:40.254486+0000","last_fullsized":"2023-10-08T00:17:40.254486+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:57:48.408623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":277,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.243521+0000","last_change":"2023-10-08T00:10:43.728490+0000","last_active":"2023-10-08T00:17:40.243521+0000","last_peered":"2023-10-08T00:17:40.243521+0000","last_clean":"2023-10-08T00:17:40.243521+0000","last_became_active":"2023-10-08T00:10:43.728341+0000","last_became_peered":"2023-10-08T00:10:43.728341+0000","last_unstale":"2023-10-08T00:17:40.243521+0000","last_undegraded":"2023-10-08T00:17:40.243521+0000","last_fullsized":"2023-10-08T00:17:40.243521+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_clean_scrub_stamp":"2023-10-08T00:10:42.716096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:06:31.035293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574934,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":838,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194474,"statfs":{"total":200000000,"available":199141752,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.53500000000000003,"15min":0.50700000000000001},"min":{"1min":0.373,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.72499999999999998,"5min":1.129,"15min":1.129},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53900000000000003,"15min":0.54400000000000004},"min":{"1min":0.40699999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.71999999999999997,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.61199999999999999}]},{"osd":1,"last update":"Sun Oct 8 00:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.56599999999999995,"15min":0.53900000000000003},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.77900000000000003,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.58899999999999997,"15min":0.56699999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.38200000000000001},"max":{"1min":0.76500000000000001,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.55900000000000005}]},{"osd":2,"last update":"Sun Oct 8 00:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.55300000000000005,"15min":0.54100000000000004},"min":{"1min":0.38,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.55300000000000005,"15min":0.52600000000000002},"min":{"1min":0.34499999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.68999999999999995,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.56699999999999995}]},{"osd":3,"last update":"Sun Oct 8 00:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55500000000000005,"15min":0.52600000000000002},"min":{"1min":0.376,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.749,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56899999999999995,"15min":0.55900000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.69999999999999996,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.49099999999999999}]},{"osd":4,"last update":"Sun Oct 8 00:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.496,"15min":0.51700000000000002},"min":{"1min":0.27200000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.50700000000000001,"15min":0.52500000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.28199999999999997},"max":{"1min":0.69699999999999995,"5min":0.76500000000000001,"15min":0.78900000000000003},"last":0.58999999999999997}]},{"osd":5,"last update":"Sun Oct 8 00:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.51400000000000001,"15min":0.54400000000000004},"min":{"1min":0.41699999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.68000000000000005,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.623},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.54000000000000004,"15min":0.57899999999999996},"min":{"1min":0.33100000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.60499999999999998}]},{"osd":6,"last update":"Sun Oct 8 00:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.58499999999999996,"15min":0.59599999999999997},"min":{"1min":0.42599999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.58599999999999997,"15min":0.624},"min":{"1min":0.35999999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.82999999999999996,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.66200000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574934,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":838,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194474,"statfs":{"total":200000000,"available":199141786,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:16:55 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.52100000000000002,"15min":0.48899999999999999},"min":{"1min":0.35299999999999998,"5min":0.34599999999999997,"15min":0.29199999999999998},"max":{"1min":0.64600000000000002,"5min":0.752,"15min":1.012},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.52500000000000002,"15min":0.65200000000000002},"min":{"1min":0.317,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.77000000000000002,"5min":1.2789999999999999,"15min":5.1230000000000002},"last":0.47299999999999998}]},{"osd":1,"last update":"Sun Oct 8 00:16:55 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.55600000000000005,"15min":0.52600000000000002},"min":{"1min":0.29899999999999999,"5min":0.29299999999999998,"15min":0.28899999999999998},"max":{"1min":0.78100000000000003,"5min":0.82499999999999996,"15min":1.1419999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.57299999999999995,"15min":0.56499999999999995},"min":{"1min":0.41099999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73599999999999999,"5min":0.98999999999999999,"15min":1.1539999999999999},"last":0.503}]},{"osd":2,"last update":"Sun Oct 8 00:16:55 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.54200000000000004,"15min":0.53000000000000003},"min":{"1min":0.36299999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.63900000000000001,"5min":0.75700000000000001,"15min":1.3149999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.57399999999999995,"15min":0.56899999999999995},"min":{"1min":0.39200000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":1.3280000000000001},"last":0.443}]},{"osd":3,"last update":"Sun Oct 8 00:16:55 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.56899999999999995,"15min":0.54100000000000004},"min":{"1min":0.38,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.72899999999999998,"5min":0.84199999999999997,"15min":1.046},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.56899999999999995,"15min":0.54500000000000004},"min":{"1min":0.40999999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.69399999999999995,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.61299999999999999}]},{"osd":4,"last update":"Sun Oct 8 00:16:55 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.47299999999999998,"15min":0.496},"min":{"1min":0.30099999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.69699999999999995,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55300000000000005,"15min":0.56299999999999994},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79500000000000004,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.66400000000000003}]},{"osd":5,"last update":"Sun Oct 8 00:16:55 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48699999999999999},"min":{"1min":0.315,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.52000000000000002,"15min":0.55100000000000005},"min":{"1min":0.39200000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.59999999999999998}]},{"osd":7,"last update":"Sun Oct 8 00:16:55 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.53800000000000003},"min":{"1min":0.34699999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.82899999999999996,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.628},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.54000000000000004,"15min":0.54600000000000004},"min":{"1min":0.35299999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.753,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.67600000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1416,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193896,"statfs":{"total":200000000,"available":198549860,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:16:51 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.34200000000000003,"15min":0.312},"min":{"1min":0.215,"5min":0.20300000000000001,"15min":0.19500000000000001},"max":{"1min":0.55900000000000005,"5min":0.56899999999999995,"15min":0.56899999999999995},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.375,"5min":0.39400000000000002,"15min":0.379},"min":{"1min":0.29699999999999999,"5min":0.255,"15min":0.255},"max":{"1min":0.54600000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.55100000000000005}]},{"osd":2,"last update":"Sun Oct 8 00:16:51 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.35199999999999998,"15min":0.33800000000000002},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.53200000000000003,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.38200000000000001,"15min":0.34599999999999997},"min":{"1min":0.26500000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.57399999999999995,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.48399999999999999}]},{"osd":3,"last update":"Sun Oct 8 00:16:51 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.39200000000000002,"15min":0.373},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.56999999999999995,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.35399999999999998,"15min":0.34399999999999997},"min":{"1min":0.255,"5min":0.222,"15min":0.222},"max":{"1min":0.502,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.441}]},{"osd":4,"last update":"Sun Oct 8 00:16:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.56000000000000005,"15min":0.54200000000000004},"min":{"1min":0.34000000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77100000000000002,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.57799999999999996,"15min":0.56799999999999995},"min":{"1min":0.40899999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.754,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.75600000000000001}]},{"osd":5,"last update":"Sun Oct 8 00:16:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57099999999999995,"15min":0.54200000000000004},"min":{"1min":0.372,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.81000000000000005,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.58799999999999997,"15min":0.57599999999999996},"min":{"1min":0.41999999999999998,"5min":0.38300000000000001,"15min":0.30399999999999999},"max":{"1min":0.84999999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.78800000000000003}]},{"osd":6,"last update":"Sun Oct 8 00:16:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58499999999999996,"15min":0.56899999999999995},"min":{"1min":0.379,"5min":0.317,"15min":0.317},"max":{"1min":0.80400000000000005,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.58099999999999996,"15min":0.56399999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.80500000000000005,"5min":0.83899999999999997,"15min":0.84999999999999998},"last":0.64500000000000002}]},{"osd":7,"last update":"Sun Oct 8 00:16:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.58399999999999996,"15min":0.56599999999999995},"min":{"1min":0.40100000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.79600000000000004,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.61799999999999999,"15min":0.59399999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81899999999999995,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.83399999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574934,"num_pgs":32,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":839,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194472,"statfs":{"total":200000000,"available":199140194,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 00:16:55 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.33800000000000002,"15min":0.33100000000000002},"min":{"1min":0.252,"5min":0.20699999999999999,"15min":0.193},"max":{"1min":0.38300000000000001,"5min":0.60499999999999998,"15min":1.0640000000000001},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.35199999999999998,"15min":0.34599999999999997},"min":{"1min":0.217,"5min":0.217,"15min":0.20300000000000001},"max":{"1min":0.47499999999999998,"5min":0.57699999999999996,"15min":0.87},"last":0.28000000000000003}]},{"osd":2,"last update":"Sun Oct 8 00:16:55 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.38500000000000001,"15min":0.38700000000000001},"min":{"1min":0.29899999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.54200000000000004,"5min":0.68000000000000005,"15min":0.93799999999999994},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.35599999999999998,"15min":0.69199999999999995},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.185},"max":{"1min":0.44,"5min":0.65600000000000003,"15min":12.019},"last":0.33600000000000002}]},{"osd":3,"last update":"Sun Oct 8 00:16:55 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.46000000000000002,"5min":0.69999999999999996,"15min":1.1020000000000001},"last":0.308},{"interface":"front","average":{"1min":0.374,"5min":0.41099999999999998,"15min":0.39800000000000002},"min":{"1min":0.28799999999999998,"5min":0.28299999999999997,"15min":0.246},"max":{"1min":0.51700000000000002,"5min":0.72899999999999998,"15min":1.0409999999999999},"last":0.40200000000000002}]},{"osd":4,"last update":"Sun Oct 8 00:16:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60999999999999999,"15min":0.61599999999999999},"min":{"1min":0.46000000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81000000000000005,"5min":0.96399999999999997,"15min":1.1779999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.61799999999999999,"15min":0.623},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.81899999999999995,"5min":0.94999999999999996,"15min":0.96199999999999997},"last":0.621}]},{"osd":5,"last update":"Sun Oct 8 00:16:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58399999999999996,"15min":0.93100000000000005},"min":{"1min":0.39300000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.83299999999999996,"5min":0.85599999999999998,"15min":11.992000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.59799999999999998,"15min":0.58799999999999997},"min":{"1min":0.41599999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.83199999999999996,"5min":0.90300000000000002,"15min":1.1299999999999999},"last":0.70899999999999996}]},{"osd":6,"last update":"Sun Oct 8 00:16:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.59499999999999997,"15min":0.58699999999999997},"min":{"1min":0.40600000000000003,"5min":0.378,"15min":0.378},"max":{"1min":0.86299999999999999,"5min":0.88700000000000001,"15min":0.91200000000000003},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59499999999999997,"15min":0.91400000000000003},"min":{"1min":0.41799999999999998,"5min":0.41199999999999998,"15min":0.38100000000000001},"max":{"1min":0.75800000000000001,"5min":0.96599999999999997,"15min":11.927},"last":0.60599999999999998}]},{"osd":7,"last update":"Sun Oct 8 00:16:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.60099999999999998,"15min":0.93300000000000005},"min":{"1min":0.44,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.79000000000000004,"5min":1.0069999999999999,"15min":11.965},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.62,"15min":0.622},"min":{"1min":0.41499999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.84799999999999998,"5min":0.98299999999999998,"15min":1.1539999999999999},"last":0.57899999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574934,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":838,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194473,"statfs":{"total":200000000,"available":199141142,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.33600000000000002,"15min":0.38400000000000001},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.53400000000000003,"5min":0.72399999999999998,"15min":2.5139999999999998},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.34100000000000003,"15min":0.39400000000000002},"min":{"1min":0.22700000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.55600000000000005,"5min":0.56399999999999995,"15min":2.1309999999999998},"last":0.32000000000000001}]},{"osd":1,"last update":"Sun Oct 8 00:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.34899999999999998,"15min":0.38800000000000001},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":2.2639999999999998},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.36799999999999999,"15min":0.42699999999999999},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":2.5379999999999998},"last":0.33800000000000002}]},{"osd":3,"last update":"Sun Oct 8 00:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.38400000000000001,"15min":0.42399999999999999},"min":{"1min":0.28100000000000003,"5min":0.254,"15min":0.254},"max":{"1min":0.61599999999999999,"5min":0.61899999999999999,"15min":2.2890000000000001},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40100000000000002,"15min":0.45700000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":2.5619999999999998},"last":0.38400000000000001}]},{"osd":4,"last update":"Sun Oct 8 00:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56200000000000006,"15min":0.60999999999999999},"min":{"1min":0.38200000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.76600000000000001,"5min":1.073,"15min":2.488},"last":0.433},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58799999999999997,"15min":0.60499999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.85899999999999999,"5min":0.93600000000000005,"15min":2.073},"last":0.39300000000000002}]},{"osd":5,"last update":"Sun Oct 8 00:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.60899999999999999,"15min":0.65100000000000002},"min":{"1min":0.374,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.82199999999999995,"5min":1.2130000000000001,"15min":2.2069999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.629,"15min":0.65900000000000003},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.39400000000000002},"max":{"1min":0.94199999999999995,"5min":1.0369999999999999,"15min":2.456},"last":0.53000000000000003}]},{"osd":6,"last update":"Sun Oct 8 00:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59599999999999997,"15min":0.61899999999999999},"min":{"1min":0.39200000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.93000000000000005,"5min":1.05,"15min":2.0339999999999998},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60599999999999998,"15min":0.61299999999999999},"min":{"1min":0.439,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.873,"5min":1.0940000000000001,"15min":2.1699999999999999},"last":0.48899999999999999}]},{"osd":7,"last update":"Sun Oct 8 00:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.61199999999999999,"15min":0.63300000000000001},"min":{"1min":0.42999999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.95999999999999996,"5min":1.0229999999999999,"15min":2.1030000000000002},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60999999999999999,"15min":0.627},"min":{"1min":0.443,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.90500000000000003,"5min":1.0569999999999999,"15min":2.2349999999999999},"last":0.45700000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574934,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":832,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194480,"statfs":{"total":200000000,"available":199148004,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:16:49 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.32000000000000001,"15min":0.36399999999999999},"min":{"1min":0.218,"5min":0.192,"15min":0.192},"max":{"1min":0.55200000000000005,"5min":0.71199999999999997,"15min":1.46},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.32000000000000001,"5min":0.33200000000000002,"15min":0.34000000000000002},"min":{"1min":0.26500000000000001,"5min":0.192,"15min":0.192},"max":{"1min":0.42399999999999999,"5min":0.504,"15min":0.54800000000000004},"last":0.27900000000000003}]},{"osd":1,"last update":"Sun Oct 8 00:16:49 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.34200000000000003,"15min":0.754},"min":{"1min":0.23699999999999999,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.46200000000000002,"5min":0.57599999999999996,"15min":12.351000000000001},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.36399999999999999,"15min":0.36499999999999999},"min":{"1min":0.248,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.54200000000000004,"5min":0.60199999999999998,"15min":0.71999999999999997},"last":0.36199999999999999}]},{"osd":2,"last update":"Sun Oct 8 00:16:49 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.38100000000000001,"15min":0.41399999999999998},"min":{"1min":0.30099999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.48999999999999999,"5min":0.70499999999999996,"15min":1.5489999999999999},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.37,"5min":0.39000000000000001,"15min":0.38700000000000001},"min":{"1min":0.28299999999999997,"5min":0.26800000000000002,"15min":0.246},"max":{"1min":0.45500000000000002,"5min":0.64700000000000002,"15min":0.753},"last":0.308}]},{"osd":4,"last update":"Sun Oct 8 00:16:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59999999999999998,"15min":0.95099999999999996},"min":{"1min":0.39600000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.80000000000000004,"5min":1.024,"15min":11.333},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61599999999999999,"15min":0.97999999999999998},"min":{"1min":0.51000000000000001,"5min":0.376,"15min":0.36099999999999999},"max":{"1min":0.77800000000000002,"5min":0.94899999999999995,"15min":12.321999999999999},"last":0.497}]},{"osd":5,"last update":"Sun Oct 8 00:16:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60199999999999998,"15min":0.98799999999999999},"min":{"1min":0.41099999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.73499999999999999,"5min":0.92000000000000004,"15min":12.291},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.622,"15min":0.60499999999999998},"min":{"1min":0.45600000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.73199999999999998,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.46100000000000002}]},{"osd":6,"last update":"Sun Oct 8 00:16:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58099999999999996,"15min":0.91800000000000004},"min":{"1min":0.442,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.83599999999999997,"5min":0.96199999999999997,"15min":11.288},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.59199999999999997,"15min":0.96999999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.73599999999999999,"5min":1.002,"15min":12.253},"last":0.438}]},{"osd":7,"last update":"Sun Oct 8 00:16:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.60899999999999999,"15min":0.58199999999999996},"min":{"1min":0.47699999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80800000000000005,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.61199999999999999,"15min":0.95299999999999996},"min":{"1min":0.46200000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82599999999999996,"5min":1.381,"15min":11.363},"last":0.45100000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574934,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":838,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194474,"statfs":{"total":200000000,"available":199141786,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.56499999999999995,"15min":0.55500000000000005},"min":{"1min":0.36899999999999999,"5min":0.193,"15min":0.193},"max":{"1min":0.67800000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57199999999999995,"15min":0.58299999999999996},"min":{"1min":0.42599999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.60599999999999998}]},{"osd":1,"last update":"Sun Oct 8 00:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.52400000000000002,"15min":0.52000000000000002},"min":{"1min":0.35199999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76400000000000001,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52800000000000002,"15min":0.53300000000000003},"min":{"1min":0.34999999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.64400000000000002,"5min":0.80400000000000005,"15min":0.81299999999999994},"last":0.59099999999999997}]},{"osd":2,"last update":"Sun Oct 8 00:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.51100000000000001,"15min":0.51800000000000002},"min":{"1min":0.39700000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.66600000000000004,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53000000000000003,"15min":0.51500000000000001},"min":{"1min":0.41799999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.67900000000000005,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.47799999999999998}]},{"osd":3,"last update":"Sun Oct 8 00:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.52800000000000002,"15min":0.53700000000000003},"min":{"1min":0.33700000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.71999999999999997,"5min":0.79800000000000004,"15min":0.85399999999999998},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53500000000000003,"15min":0.53400000000000003},"min":{"1min":0.39100000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.505}]},{"osd":5,"last update":"Sun Oct 8 00:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.45800000000000002,"15min":0.49099999999999999},"min":{"1min":0.317,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.65700000000000003,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.503,"5min":0.49199999999999999,"15min":0.50900000000000001},"min":{"1min":0.315,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.73899999999999999,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.45400000000000001}]},{"osd":6,"last update":"Sun Oct 8 00:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.47199999999999998,"15min":0.48799999999999999},"min":{"1min":0.33700000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.69899999999999995,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.51900000000000002,"15min":0.55100000000000005},"min":{"1min":0.438,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.72399999999999998,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.54600000000000004}]},{"osd":7,"last update":"Sun Oct 8 00:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.53200000000000003,"15min":0.55000000000000004},"min":{"1min":0.36899999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.78000000000000003,"5min":0.79700000000000004,"15min":0.90100000000000002},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56699999999999995,"15min":0.57899999999999996},"min":{"1min":0.47099999999999997,"5min":0.376,"15min":0.376},"max":{"1min":0.75600000000000001,"5min":0.78000000000000003,"15min":0.78500000000000003},"last":0.57099999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574934,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1415,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193897,"statfs":{"total":200000000,"available":198550704,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.48699999999999999,"15min":0.48799999999999999},"min":{"1min":0.36099999999999999,"5min":0.249,"15min":0.249},"max":{"1min":0.63100000000000001,"5min":0.68799999999999994,"15min":0.72699999999999998},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53500000000000003,"15min":0.55600000000000005},"min":{"1min":0.39200000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.88600000000000001},"last":0.495}]},{"osd":1,"last update":"Sun Oct 8 00:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.52700000000000002,"15min":0.52500000000000002},"min":{"1min":0.34399999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.82999999999999996},"last":0.623},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.505,"15min":0.52200000000000002},"min":{"1min":0.39900000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.71699999999999997,"5min":0.747,"15min":0.755},"last":0.433}]},{"osd":2,"last update":"Sun Oct 8 00:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52300000000000002,"15min":0.52900000000000003},"min":{"1min":0.34499999999999997,"5min":0.34399999999999997,"15min":0.32400000000000001},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.86199999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54300000000000004,"15min":0.56499999999999995},"min":{"1min":0.41499999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.78500000000000003,"5min":0.79500000000000004,"15min":0.95399999999999996},"last":0.46899999999999997}]},{"osd":3,"last update":"Sun Oct 8 00:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55300000000000005,"15min":0.54800000000000004},"min":{"1min":0.42799999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.69799999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.55500000000000005,"15min":0.57499999999999996},"min":{"1min":0.41999999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.71999999999999997,"5min":0.72299999999999998,"15min":1.0529999999999999},"last":0.58599999999999997}]},{"osd":4,"last update":"Sun Oct 8 00:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.47899999999999998,"15min":0.47699999999999998},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.27700000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.79200000000000004},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.52100000000000002,"15min":0.51700000000000002},"min":{"1min":0.314,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.41599999999999998}]},{"osd":6,"last update":"Sun Oct 8 00:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.51700000000000002,"15min":0.505},"min":{"1min":0.33000000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.76500000000000001,"5min":0.78900000000000003,"15min":0.79700000000000004},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.51700000000000002,"15min":0.5},"min":{"1min":0.34499999999999997,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.71499999999999997,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.60199999999999998}]},{"osd":7,"last update":"Sun Oct 8 00:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.53500000000000003,"15min":0.52200000000000002},"min":{"1min":0.35699999999999998,"5min":0.33300000000000002,"15min":0.29399999999999998},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56000000000000005,"15min":0.55800000000000005},"min":{"1min":0.374,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.65500000000000003}]}]}],"pool_statfs":[]}} 2023-10-08T00:17:44.295 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T00:17:44.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.440+0000 7fcb5fdfb700 1 Processor -- start 2023-10-08T00:17:44.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.441+0000 7fcb5fdfb700 1 -- start start 2023-10-08T00:17:44.444 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.441+0000 7fcb5fdfb700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb5815a620 0x7fcb5815d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:44.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.441+0000 7fcb5fdfb700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb5815cbb0 con 0x7fcb5815d580 2023-10-08T00:17:44.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.441+0000 7fcb5fdfb700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb5815bfa0 con 0x7fcb58157870 2023-10-08T00:17:44.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.441+0000 7fcb5fdfb700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5815bdb0 con 0x7fcb5815a620 2023-10-08T00:17:44.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.442+0000 7fcb5e398700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fcb5815d580 0x7fcb58158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:60446/0 (socket says 172.21.15.99:60446) 2023-10-08T00:17:44.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.442+0000 7fcb5e398700 1 -- 172.21.15.99:0/1031695786 learned_addr learned my addr 172.21.15.99:0/1031695786 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:17:44.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.442+0000 7fcb5d396700 1 --2- 172.21.15.99:0/1031695786 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb5815a620 0x7fcb5815d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:44.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.442+0000 7fcb5cb95700 1 -- 172.21.15.99:0/1031695786 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3637653854 0 0) 0x7fcb5815cbb0 con 0x7fcb5815d580 2023-10-08T00:17:44.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.442+0000 7fcb5cb95700 1 -- 172.21.15.99:0/1031695786 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb44003570 con 0x7fcb5815d580 2023-10-08T00:17:44.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.442+0000 7fcb5cb95700 1 -- 172.21.15.99:0/1031695786 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1927744297 0 0) 0x7fcb5815bfa0 con 0x7fcb58157870 2023-10-08T00:17:44.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.442+0000 7fcb5cb95700 1 -- 172.21.15.99:0/1031695786 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb5815cbb0 con 0x7fcb58157870 2023-10-08T00:17:44.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.442+0000 7fcb5d396700 1 -- 172.21.15.99:0/1031695786 >> v1:172.21.15.191:6789/0 conn(0x7fcb58157870 legacy=0x7fcb5815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:44.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.442+0000 7fcb5cb95700 1 -- 172.21.15.99:0/1031695786 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3609390055 0 0) 0x7fcb44003570 con 0x7fcb5815d580 2023-10-08T00:17:44.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.442+0000 7fcb5d396700 1 -- 172.21.15.99:0/1031695786 >> v1:172.21.15.99:6789/0 conn(0x7fcb5815d580 legacy=0x7fcb58158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:44.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.442+0000 7fcb5d396700 1 -- 172.21.15.99:0/1031695786 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb5815c280 con 0x7fcb5815a620 2023-10-08T00:17:44.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.442+0000 7fcb5d396700 1 --2- 172.21.15.99:0/1031695786 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb5815a620 0x7fcb5815d040 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcb48009910 tx=0x7fcb480091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5509be00110c49d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:44.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.442+0000 7fcb5cb95700 1 -- 172.21.15.99:0/1031695786 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb50001470 con 0x7fcb5815a620 2023-10-08T00:17:44.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.442+0000 7fcb5cb95700 1 -- 172.21.15.99:0/1031695786 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb48011420 con 0x7fcb5815a620 2023-10-08T00:17:44.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.443+0000 7fcb5cb95700 1 -- 172.21.15.99:0/1031695786 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb480117a0 con 0x7fcb5815a620 2023-10-08T00:17:44.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.443+0000 7fcb5fdfb700 1 -- 172.21.15.99:0/1031695786 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb5815a620 msgr2=0x7fcb5815d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:44.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.443+0000 7fcb5fdfb700 1 --2- 172.21.15.99:0/1031695786 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb5815a620 0x7fcb5815d040 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcb48009910 tx=0x7fcb480091a0 comp rx=0 tx=0).stop 2023-10-08T00:17:44.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.443+0000 7fcb5fdfb700 1 -- 172.21.15.99:0/1031695786 shutdown_connections 2023-10-08T00:17:44.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.443+0000 7fcb5fdfb700 1 --2- 172.21.15.99:0/1031695786 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb5815a620 0x7fcb5815d040 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:44.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.443+0000 7fcb5fdfb700 1 -- 172.21.15.99:0/1031695786 >> 172.21.15.99:0/1031695786 conn(0x7fcb58068ce0 msgr2=0x7fcb580690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:44.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.443+0000 7fcb5fdfb700 1 -- 172.21.15.99:0/1031695786 shutdown_connections 2023-10-08T00:17:44.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.443+0000 7fcb5fdfb700 1 -- 172.21.15.99:0/1031695786 wait complete. 2023-10-08T00:17:44.449 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.444+0000 7fcb5fdfb700 1 Processor -- start 2023-10-08T00:17:44.449 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.444+0000 7fcb5fdfb700 1 -- start start 2023-10-08T00:17:44.449 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.445+0000 7fcb5fdfb700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb58157870 0x7fcb5815eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:44.449 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.445+0000 7fcb5fdfb700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb5815a620 0x7fcb5814a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:44.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.445+0000 7fcb5fdfb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb5815d580 0x7fcb5816df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:44.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.445+0000 7fcb5fdfb700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb581581c0 con 0x7fcb5815d580 2023-10-08T00:17:44.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.445+0000 7fcb5fdfb700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb580b2b20 con 0x7fcb5815a620 2023-10-08T00:17:44.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.445+0000 7fcb5fdfb700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb580b2e00 con 0x7fcb58157870 2023-10-08T00:17:44.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.445+0000 7fcb5e398700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb5815d580 0x7fcb5816df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:44.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.445+0000 7fcb5e398700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb5815d580 0x7fcb5816df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53050/0 (socket says 172.21.15.99:53050) 2023-10-08T00:17:44.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.445+0000 7fcb5e398700 1 -- 172.21.15.99:0/2693279767 learned_addr learned my addr 172.21.15.99:0/2693279767 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:44.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.445+0000 7fcb5d396700 1 --2- 172.21.15.99:0/2693279767 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb5815a620 0x7fcb5814a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:44.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.445+0000 7fcb5db97700 1 --2- 172.21.15.99:0/2693279767 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb58157870 0x7fcb5815eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:44.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.445+0000 7fcb5e398700 1 -- 172.21.15.99:0/2693279767 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb58157870 msgr2=0x7fcb5815eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:44.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.445+0000 7fcb5e398700 1 --2- 172.21.15.99:0/2693279767 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb58157870 0x7fcb5815eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:44.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.445+0000 7fcb5e398700 1 -- 172.21.15.99:0/2693279767 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb5815a620 msgr2=0x7fcb5814a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:44.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.445+0000 7fcb5e398700 1 --2- 172.21.15.99:0/2693279767 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb5815a620 0x7fcb5814a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:44.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.445+0000 7fcb5e398700 1 -- 172.21.15.99:0/2693279767 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb48009420 con 0x7fcb5815d580 2023-10-08T00:17:44.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.445+0000 7fcb5e398700 1 --2- 172.21.15.99:0/2693279767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb5815d580 0x7fcb5816df90 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fcb5000c7e0 tx=0x7fcb50001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:44.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.445+0000 7fcb4dffb700 1 -- 172.21.15.99:0/2693279767 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb50007360 con 0x7fcb5815d580 2023-10-08T00:17:44.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.446+0000 7fcb4dffb700 1 -- 172.21.15.99:0/2693279767 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb500074e0 con 0x7fcb5815d580 2023-10-08T00:17:44.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.446+0000 7fcb5fdfb700 1 -- 172.21.15.99:0/2693279767 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb580b30e0 con 0x7fcb5815d580 2023-10-08T00:17:44.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.446+0000 7fcb4dffb700 1 -- 172.21.15.99:0/2693279767 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb50004830 con 0x7fcb5815d580 2023-10-08T00:17:44.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.446+0000 7fcb5fdfb700 1 -- 172.21.15.99:0/2693279767 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb580b2760 con 0x7fcb5815d580 2023-10-08T00:17:44.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.447+0000 7fcb5fdfb700 1 -- 172.21.15.99:0/2693279767 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb580b3580 con 0x7fcb5815d580 2023-10-08T00:17:44.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.447+0000 7fcb4dffb700 1 -- 172.21.15.99:0/2693279767 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcb50004bc0 con 0x7fcb5815d580 2023-10-08T00:17:44.452 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.447+0000 7fcb4dffb700 1 --2- 172.21.15.99:0/2693279767 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcb3c066f30 0x7fcb3c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:44.452 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.447+0000 7fcb4dffb700 1 -- 172.21.15.99:0/2693279767 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcb500843e0 con 0x7fcb5815d580 2023-10-08T00:17:44.452 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.447+0000 7fcb5db97700 1 --2- 172.21.15.99:0/2693279767 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcb3c066f30 0x7fcb3c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:44.452 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.448+0000 7fcb5db97700 1 --2- 172.21.15.99:0/2693279767 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcb3c066f30 0x7fcb3c0693e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fcb54004140 tx=0x7fcb54007f90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:44.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.451+0000 7fcb4dffb700 1 -- 172.21.15.99:0/2693279767 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcb500516d0 con 0x7fcb5815d580 2023-10-08T00:17:44.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.630+0000 7fcb5fdfb700 1 -- 172.21.15.99:0/2693279767 --> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcb5804c3c0 con 0x7fcb3c066f30 2023-10-08T00:17:44.639 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.638+0000 7fcb4dffb700 1 -- 172.21.15.99:0/2693279767 <== mgr.4099 v2:172.21.15.191:6832/85985 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285840 (secure 0 0 0) 0x7fcb5804c3c0 con 0x7fcb3c066f30 2023-10-08T00:17:44.642 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:17:44.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.642+0000 7fcb5fdfb700 1 -- 172.21.15.99:0/2693279767 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcb3c066f30 msgr2=0x7fcb3c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:44.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.642+0000 7fcb5fdfb700 1 --2- 172.21.15.99:0/2693279767 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcb3c066f30 0x7fcb3c0693e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fcb54004140 tx=0x7fcb54007f90 comp rx=0 tx=0).stop 2023-10-08T00:17:44.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.642+0000 7fcb5fdfb700 1 -- 172.21.15.99:0/2693279767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb5815d580 msgr2=0x7fcb5816df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:44.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.642+0000 7fcb5fdfb700 1 --2- 172.21.15.99:0/2693279767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb5815d580 0x7fcb5816df90 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fcb5000c7e0 tx=0x7fcb50001980 comp rx=0 tx=0).stop 2023-10-08T00:17:44.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.642+0000 7fcb5fdfb700 1 -- 172.21.15.99:0/2693279767 shutdown_connections 2023-10-08T00:17:44.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.642+0000 7fcb5fdfb700 1 --2- 172.21.15.99:0/2693279767 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcb3c066f30 0x7fcb3c0693e0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:44.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.642+0000 7fcb5fdfb700 1 --2- 172.21.15.99:0/2693279767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb5815d580 0x7fcb5816df90 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:44.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.642+0000 7fcb5fdfb700 1 --2- 172.21.15.99:0/2693279767 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb58157870 0x7fcb5815eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:44.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.642+0000 7fcb5fdfb700 1 --2- 172.21.15.99:0/2693279767 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb5815a620 0x7fcb5814a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:44.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.642+0000 7fcb5fdfb700 1 -- 172.21.15.99:0/2693279767 >> 172.21.15.99:0/2693279767 conn(0x7fcb58068ce0 msgr2=0x7fcb5815fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:44.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.642+0000 7fcb5fdfb700 1 -- 172.21.15.99:0/2693279767 shutdown_connections 2023-10-08T00:17:44.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.642+0000 7fcb5fdfb700 1 -- 172.21.15.99:0/2693279767 wait complete. 2023-10-08T00:17:44.645 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2023-10-08T00:17:44.655 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2023-10-08T00:17:44.122252+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":7856,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1554643,"statfs":{"total":1600000000,"available":1591955228,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.894650"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.257925+0000","last_change":"2023-10-08T00:17:40.257821+0000","last_active":"2023-10-08T00:17:40.257925+0000","last_peered":"2023-10-08T00:17:40.257925+0000","last_clean":"2023-10-08T00:17:40.257925+0000","last_became_active":"2023-10-08T00:17:40.257038+0000","last_became_peered":"2023-10-08T00:17:40.257038+0000","last_unstale":"2023-10-08T00:17:40.257925+0000","last_undegraded":"2023-10-08T00:17:40.257925+0000","last_fullsized":"2023-10-08T00:17:40.257925+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:34:52.462946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.258842+0000","last_change":"2023-10-08T00:17:40.258697+0000","last_active":"2023-10-08T00:17:40.258842+0000","last_peered":"2023-10-08T00:17:40.258842+0000","last_clean":"2023-10-08T00:17:40.258842+0000","last_became_active":"2023-10-08T00:17:40.257477+0000","last_became_peered":"2023-10-08T00:17:40.257477+0000","last_unstale":"2023-10-08T00:17:40.258842+0000","last_undegraded":"2023-10-08T00:17:40.258842+0000","last_fullsized":"2023-10-08T00:17:40.258842+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:30:30.805838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.241699+0000","last_change":"2023-10-08T00:17:40.241468+0000","last_active":"2023-10-08T00:17:40.241699+0000","last_peered":"2023-10-08T00:17:40.241699+0000","last_clean":"2023-10-08T00:17:40.241699+0000","last_became_active":"2023-10-08T00:17:40.241052+0000","last_became_peered":"2023-10-08T00:17:40.241052+0000","last_unstale":"2023-10-08T00:17:40.241699+0000","last_undegraded":"2023-10-08T00:17:40.241699+0000","last_fullsized":"2023-10-08T00:17:40.241699+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:42:27.289223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.244797+0000","last_change":"2023-10-08T00:17:40.244720+0000","last_active":"2023-10-08T00:17:40.244797+0000","last_peered":"2023-10-08T00:17:40.244797+0000","last_clean":"2023-10-08T00:17:40.244797+0000","last_became_active":"2023-10-08T00:17:40.244476+0000","last_became_peered":"2023-10-08T00:17:40.244476+0000","last_unstale":"2023-10-08T00:17:40.244797+0000","last_undegraded":"2023-10-08T00:17:40.244797+0000","last_fullsized":"2023-10-08T00:17:40.244797+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:34:52.462946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.244822+0000","last_change":"2023-10-08T00:17:40.244700+0000","last_active":"2023-10-08T00:17:40.244822+0000","last_peered":"2023-10-08T00:17:40.244822+0000","last_clean":"2023-10-08T00:17:40.244822+0000","last_became_active":"2023-10-08T00:17:40.244351+0000","last_became_peered":"2023-10-08T00:17:40.244351+0000","last_unstale":"2023-10-08T00:17:40.244822+0000","last_undegraded":"2023-10-08T00:17:40.244822+0000","last_fullsized":"2023-10-08T00:17:40.244822+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:24:15.333112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.258402+0000","last_change":"2023-10-08T00:17:40.258335+0000","last_active":"2023-10-08T00:17:40.258402+0000","last_peered":"2023-10-08T00:17:40.258402+0000","last_clean":"2023-10-08T00:17:40.258402+0000","last_became_active":"2023-10-08T00:17:40.258021+0000","last_became_peered":"2023-10-08T00:17:40.258021+0000","last_unstale":"2023-10-08T00:17:40.258402+0000","last_undegraded":"2023-10-08T00:17:40.258402+0000","last_fullsized":"2023-10-08T00:17:40.258402+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:40:24.777787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.239407+0000","last_change":"2023-10-08T00:17:40.239070+0000","last_active":"2023-10-08T00:17:40.239407+0000","last_peered":"2023-10-08T00:17:40.239407+0000","last_clean":"2023-10-08T00:17:40.239407+0000","last_became_active":"2023-10-08T00:17:40.238829+0000","last_became_peered":"2023-10-08T00:17:40.238829+0000","last_unstale":"2023-10-08T00:17:40.239407+0000","last_undegraded":"2023-10-08T00:17:40.239407+0000","last_fullsized":"2023-10-08T00:17:40.239407+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:42:33.653060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.245603+0000","last_change":"2023-10-08T00:17:40.245531+0000","last_active":"2023-10-08T00:17:40.245603+0000","last_peered":"2023-10-08T00:17:40.245603+0000","last_clean":"2023-10-08T00:17:40.245603+0000","last_became_active":"2023-10-08T00:17:40.245306+0000","last_became_peered":"2023-10-08T00:17:40.245306+0000","last_unstale":"2023-10-08T00:17:40.245603+0000","last_undegraded":"2023-10-08T00:17:40.245603+0000","last_fullsized":"2023-10-08T00:17:40.245603+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:43:13.797123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.245086+0000","last_change":"2023-10-08T00:17:40.244989+0000","last_active":"2023-10-08T00:17:40.245086+0000","last_peered":"2023-10-08T00:17:40.245086+0000","last_clean":"2023-10-08T00:17:40.245086+0000","last_became_active":"2023-10-08T00:17:40.244634+0000","last_became_peered":"2023-10-08T00:17:40.244634+0000","last_unstale":"2023-10-08T00:17:40.245086+0000","last_undegraded":"2023-10-08T00:17:40.245086+0000","last_fullsized":"2023-10-08T00:17:40.245086+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:23:21.315613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.254877+0000","last_change":"2023-10-08T00:17:40.254397+0000","last_active":"2023-10-08T00:17:40.254877+0000","last_peered":"2023-10-08T00:17:40.254877+0000","last_clean":"2023-10-08T00:17:40.254877+0000","last_became_active":"2023-10-08T00:17:40.248198+0000","last_became_peered":"2023-10-08T00:17:40.248198+0000","last_unstale":"2023-10-08T00:17:40.254877+0000","last_undegraded":"2023-10-08T00:17:40.254877+0000","last_fullsized":"2023-10-08T00:17:40.254877+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:55:57.192962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.246240+0000","last_change":"2023-10-08T00:17:40.246175+0000","last_active":"2023-10-08T00:17:40.246240+0000","last_peered":"2023-10-08T00:17:40.246240+0000","last_clean":"2023-10-08T00:17:40.246240+0000","last_became_active":"2023-10-08T00:17:40.245940+0000","last_became_peered":"2023-10-08T00:17:40.245940+0000","last_unstale":"2023-10-08T00:17:40.246240+0000","last_undegraded":"2023-10-08T00:17:40.246240+0000","last_fullsized":"2023-10-08T00:17:40.246240+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:22:03.771134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.246886+0000","last_change":"2023-10-08T00:17:40.246490+0000","last_active":"2023-10-08T00:17:40.246886+0000","last_peered":"2023-10-08T00:17:40.246886+0000","last_clean":"2023-10-08T00:17:40.246886+0000","last_became_active":"2023-10-08T00:17:40.244763+0000","last_became_peered":"2023-10-08T00:17:40.244763+0000","last_unstale":"2023-10-08T00:17:40.246886+0000","last_undegraded":"2023-10-08T00:17:40.246886+0000","last_fullsized":"2023-10-08T00:17:40.246886+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:09:08.837995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.253523+0000","last_change":"2023-10-08T00:17:40.253423+0000","last_active":"2023-10-08T00:17:40.253523+0000","last_peered":"2023-10-08T00:17:40.253523+0000","last_clean":"2023-10-08T00:17:40.253523+0000","last_became_active":"2023-10-08T00:17:40.252932+0000","last_became_peered":"2023-10-08T00:17:40.252932+0000","last_unstale":"2023-10-08T00:17:40.253523+0000","last_undegraded":"2023-10-08T00:17:40.253523+0000","last_fullsized":"2023-10-08T00:17:40.253523+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:37:58.966950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.249317+0000","last_change":"2023-10-08T00:17:40.249184+0000","last_active":"2023-10-08T00:17:40.249317+0000","last_peered":"2023-10-08T00:17:40.249317+0000","last_clean":"2023-10-08T00:17:40.249317+0000","last_became_active":"2023-10-08T00:17:40.248663+0000","last_became_peered":"2023-10-08T00:17:40.248663+0000","last_unstale":"2023-10-08T00:17:40.249317+0000","last_undegraded":"2023-10-08T00:17:40.249317+0000","last_fullsized":"2023-10-08T00:17:40.249317+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:24:21.948361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.243906+0000","last_change":"2023-10-08T00:17:40.239467+0000","last_active":"2023-10-08T00:17:40.243906+0000","last_peered":"2023-10-08T00:17:40.243906+0000","last_clean":"2023-10-08T00:17:40.243906+0000","last_became_active":"2023-10-08T00:17:40.238903+0000","last_became_peered":"2023-10-08T00:17:40.238903+0000","last_unstale":"2023-10-08T00:17:40.243906+0000","last_undegraded":"2023-10-08T00:17:40.243906+0000","last_fullsized":"2023-10-08T00:17:40.243906+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:12:58.622892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.247787+0000","last_change":"2023-10-08T00:17:40.247333+0000","last_active":"2023-10-08T00:17:40.247787+0000","last_peered":"2023-10-08T00:17:40.247787+0000","last_clean":"2023-10-08T00:17:40.247787+0000","last_became_active":"2023-10-08T00:17:40.246785+0000","last_became_peered":"2023-10-08T00:17:40.246785+0000","last_unstale":"2023-10-08T00:17:40.247787+0000","last_undegraded":"2023-10-08T00:17:40.247787+0000","last_fullsized":"2023-10-08T00:17:40.247787+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:51:35.954365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.243149+0000","last_change":"2023-10-08T00:17:40.243071+0000","last_active":"2023-10-08T00:17:40.243149+0000","last_peered":"2023-10-08T00:17:40.243149+0000","last_clean":"2023-10-08T00:17:40.243149+0000","last_became_active":"2023-10-08T00:17:40.242841+0000","last_became_peered":"2023-10-08T00:17:40.242841+0000","last_unstale":"2023-10-08T00:17:40.243149+0000","last_undegraded":"2023-10-08T00:17:40.243149+0000","last_fullsized":"2023-10-08T00:17:40.243149+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:05:20.180784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.243942+0000","last_change":"2023-10-08T00:17:40.240463+0000","last_active":"2023-10-08T00:17:40.243942+0000","last_peered":"2023-10-08T00:17:40.243942+0000","last_clean":"2023-10-08T00:17:40.243942+0000","last_became_active":"2023-10-08T00:17:40.240256+0000","last_became_peered":"2023-10-08T00:17:40.240256+0000","last_unstale":"2023-10-08T00:17:40.243942+0000","last_undegraded":"2023-10-08T00:17:40.243942+0000","last_fullsized":"2023-10-08T00:17:40.243942+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:57:32.743438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.248205+0000","last_change":"2023-10-08T00:17:40.247835+0000","last_active":"2023-10-08T00:17:40.248205+0000","last_peered":"2023-10-08T00:17:40.248205+0000","last_clean":"2023-10-08T00:17:40.248205+0000","last_became_active":"2023-10-08T00:17:40.246940+0000","last_became_peered":"2023-10-08T00:17:40.246940+0000","last_unstale":"2023-10-08T00:17:40.248205+0000","last_undegraded":"2023-10-08T00:17:40.248205+0000","last_fullsized":"2023-10-08T00:17:40.248205+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:51:35.954365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.243925+0000","last_change":"2023-10-08T00:17:40.239606+0000","last_active":"2023-10-08T00:17:40.243925+0000","last_peered":"2023-10-08T00:17:40.243925+0000","last_clean":"2023-10-08T00:17:40.243925+0000","last_became_active":"2023-10-08T00:17:40.239240+0000","last_became_peered":"2023-10-08T00:17:40.239240+0000","last_unstale":"2023-10-08T00:17:40.243925+0000","last_undegraded":"2023-10-08T00:17:40.243925+0000","last_fullsized":"2023-10-08T00:17:40.243925+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:57:38.260338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.243287+0000","last_change":"2023-10-08T00:17:40.243222+0000","last_active":"2023-10-08T00:17:40.243287+0000","last_peered":"2023-10-08T00:17:40.243287+0000","last_clean":"2023-10-08T00:17:40.243287+0000","last_became_active":"2023-10-08T00:17:40.242924+0000","last_became_peered":"2023-10-08T00:17:40.242924+0000","last_unstale":"2023-10-08T00:17:40.243287+0000","last_undegraded":"2023-10-08T00:17:40.243287+0000","last_fullsized":"2023-10-08T00:17:40.243287+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:55:38.261584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.258448+0000","last_change":"2023-10-08T00:17:40.258373+0000","last_active":"2023-10-08T00:17:40.258448+0000","last_peered":"2023-10-08T00:17:40.258448+0000","last_clean":"2023-10-08T00:17:40.258448+0000","last_became_active":"2023-10-08T00:17:40.256619+0000","last_became_peered":"2023-10-08T00:17:40.256619+0000","last_unstale":"2023-10-08T00:17:40.258448+0000","last_undegraded":"2023-10-08T00:17:40.258448+0000","last_fullsized":"2023-10-08T00:17:40.258448+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:32:33.029129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.254234+0000","last_change":"2023-10-08T00:17:40.253954+0000","last_active":"2023-10-08T00:17:40.254234+0000","last_peered":"2023-10-08T00:17:40.254234+0000","last_clean":"2023-10-08T00:17:40.254234+0000","last_became_active":"2023-10-08T00:17:40.253387+0000","last_became_peered":"2023-10-08T00:17:40.253387+0000","last_unstale":"2023-10-08T00:17:40.254234+0000","last_undegraded":"2023-10-08T00:17:40.254234+0000","last_fullsized":"2023-10-08T00:17:40.254234+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:36:20.674965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.243990+0000","last_change":"2023-10-08T00:17:40.243816+0000","last_active":"2023-10-08T00:17:40.243990+0000","last_peered":"2023-10-08T00:17:40.243990+0000","last_clean":"2023-10-08T00:17:40.243990+0000","last_became_active":"2023-10-08T00:17:40.243233+0000","last_became_peered":"2023-10-08T00:17:40.243233+0000","last_unstale":"2023-10-08T00:17:40.243990+0000","last_undegraded":"2023-10-08T00:17:40.243990+0000","last_fullsized":"2023-10-08T00:17:40.243990+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:33:51.997828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.253112+0000","last_change":"2023-10-08T00:17:40.250762+0000","last_active":"2023-10-08T00:17:40.253112+0000","last_peered":"2023-10-08T00:17:40.253112+0000","last_clean":"2023-10-08T00:17:40.253112+0000","last_became_active":"2023-10-08T00:17:40.250534+0000","last_became_peered":"2023-10-08T00:17:40.250534+0000","last_unstale":"2023-10-08T00:17:40.253112+0000","last_undegraded":"2023-10-08T00:17:40.253112+0000","last_fullsized":"2023-10-08T00:17:40.253112+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:33:38.865833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.258056+0000","last_change":"2023-10-08T00:17:40.257965+0000","last_active":"2023-10-08T00:17:40.258056+0000","last_peered":"2023-10-08T00:17:40.258056+0000","last_clean":"2023-10-08T00:17:40.258056+0000","last_became_active":"2023-10-08T00:17:40.257289+0000","last_became_peered":"2023-10-08T00:17:40.257289+0000","last_unstale":"2023-10-08T00:17:40.258056+0000","last_undegraded":"2023-10-08T00:17:40.258056+0000","last_fullsized":"2023-10-08T00:17:40.258056+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:58:26.467186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.245895+0000","last_change":"2023-10-08T00:17:40.245675+0000","last_active":"2023-10-08T00:17:40.245895+0000","last_peered":"2023-10-08T00:17:40.245895+0000","last_clean":"2023-10-08T00:17:40.245895+0000","last_became_active":"2023-10-08T00:17:40.245287+0000","last_became_peered":"2023-10-08T00:17:40.245287+0000","last_unstale":"2023-10-08T00:17:40.245895+0000","last_undegraded":"2023-10-08T00:17:40.245895+0000","last_fullsized":"2023-10-08T00:17:40.245895+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:40:39.003917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.252124+0000","last_change":"2023-10-08T00:17:40.251915+0000","last_active":"2023-10-08T00:17:40.252124+0000","last_peered":"2023-10-08T00:17:40.252124+0000","last_clean":"2023-10-08T00:17:40.252124+0000","last_became_active":"2023-10-08T00:17:40.251068+0000","last_became_peered":"2023-10-08T00:17:40.251068+0000","last_unstale":"2023-10-08T00:17:40.252124+0000","last_undegraded":"2023-10-08T00:17:40.252124+0000","last_fullsized":"2023-10-08T00:17:40.252124+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:37:32.516380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.248248+0000","last_change":"2023-10-08T00:17:40.248128+0000","last_active":"2023-10-08T00:17:40.248248+0000","last_peered":"2023-10-08T00:17:40.248248+0000","last_clean":"2023-10-08T00:17:40.248248+0000","last_became_active":"2023-10-08T00:17:40.247616+0000","last_became_peered":"2023-10-08T00:17:40.247616+0000","last_unstale":"2023-10-08T00:17:40.248248+0000","last_undegraded":"2023-10-08T00:17:40.248248+0000","last_fullsized":"2023-10-08T00:17:40.248248+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:54:35.040253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.254848+0000","last_change":"2023-10-08T00:17:40.254124+0000","last_active":"2023-10-08T00:17:40.254848+0000","last_peered":"2023-10-08T00:17:40.254848+0000","last_clean":"2023-10-08T00:17:40.254848+0000","last_became_active":"2023-10-08T00:17:40.247006+0000","last_became_peered":"2023-10-08T00:17:40.247006+0000","last_unstale":"2023-10-08T00:17:40.254848+0000","last_undegraded":"2023-10-08T00:17:40.254848+0000","last_fullsized":"2023-10-08T00:17:40.254848+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:29:56.427939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.250748+0000","last_change":"2023-10-08T00:17:40.250683+0000","last_active":"2023-10-08T00:17:40.250748+0000","last_peered":"2023-10-08T00:17:40.250748+0000","last_clean":"2023-10-08T00:17:40.250748+0000","last_became_active":"2023-10-08T00:17:40.250471+0000","last_became_peered":"2023-10-08T00:17:40.250471+0000","last_unstale":"2023-10-08T00:17:40.250748+0000","last_undegraded":"2023-10-08T00:17:40.250748+0000","last_fullsized":"2023-10-08T00:17:40.250748+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:43:28.092573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.250758+0000","last_change":"2023-10-08T00:17:40.250689+0000","last_active":"2023-10-08T00:17:40.250758+0000","last_peered":"2023-10-08T00:17:40.250758+0000","last_clean":"2023-10-08T00:17:40.250758+0000","last_became_active":"2023-10-08T00:17:40.250479+0000","last_became_peered":"2023-10-08T00:17:40.250479+0000","last_unstale":"2023-10-08T00:17:40.250758+0000","last_undegraded":"2023-10-08T00:17:40.250758+0000","last_fullsized":"2023-10-08T00:17:40.250758+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:05:25.070445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.249299+0000","last_change":"2023-10-08T00:17:40.248404+0000","last_active":"2023-10-08T00:17:40.249299+0000","last_peered":"2023-10-08T00:17:40.249299+0000","last_clean":"2023-10-08T00:17:40.249299+0000","last_became_active":"2023-10-08T00:17:40.248157+0000","last_became_peered":"2023-10-08T00:17:40.248157+0000","last_unstale":"2023-10-08T00:17:40.249299+0000","last_undegraded":"2023-10-08T00:17:40.249299+0000","last_fullsized":"2023-10-08T00:17:40.249299+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:30:30.805838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"153'4","reported_seq":14,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.259166+0000","last_change":"2023-10-08T00:17:39.242536+0000","last_active":"2023-10-08T00:17:40.259166+0000","last_peered":"2023-10-08T00:17:40.259166+0000","last_clean":"2023-10-08T00:17:40.259166+0000","last_became_active":"2023-10-08T00:17:39.242024+0000","last_became_peered":"2023-10-08T00:17:39.242024+0000","last_unstale":"2023-10-08T00:17:40.259166+0000","last_undegraded":"2023-10-08T00:17:40.259166+0000","last_fullsized":"2023-10-08T00:17:40.259166+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:53:34.765782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.242911+0000","last_change":"2023-10-08T00:17:40.242851+0000","last_active":"2023-10-08T00:17:40.242911+0000","last_peered":"2023-10-08T00:17:40.242911+0000","last_clean":"2023-10-08T00:17:40.242911+0000","last_became_active":"2023-10-08T00:17:40.242661+0000","last_became_peered":"2023-10-08T00:17:40.242661+0000","last_unstale":"2023-10-08T00:17:40.242911+0000","last_undegraded":"2023-10-08T00:17:40.242911+0000","last_fullsized":"2023-10-08T00:17:40.242911+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:21:25.611483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"153'1","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.254304+0000","last_change":"2023-10-08T00:17:39.235694+0000","last_active":"2023-10-08T00:17:40.254304+0000","last_peered":"2023-10-08T00:17:40.254304+0000","last_clean":"2023-10-08T00:17:40.254304+0000","last_became_active":"2023-10-08T00:17:39.235491+0000","last_became_peered":"2023-10-08T00:17:39.235491+0000","last_unstale":"2023-10-08T00:17:40.254304+0000","last_undegraded":"2023-10-08T00:17:40.254304+0000","last_fullsized":"2023-10-08T00:17:40.254304+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:26:49.727979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.251816+0000","last_change":"2023-10-08T00:17:40.251412+0000","last_active":"2023-10-08T00:17:40.251816+0000","last_peered":"2023-10-08T00:17:40.251816+0000","last_clean":"2023-10-08T00:17:40.251816+0000","last_became_active":"2023-10-08T00:17:40.251195+0000","last_became_peered":"2023-10-08T00:17:40.251195+0000","last_unstale":"2023-10-08T00:17:40.251816+0000","last_undegraded":"2023-10-08T00:17:40.251816+0000","last_fullsized":"2023-10-08T00:17:40.251816+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:21:32.636851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"153'2","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.253430+0000","last_change":"2023-10-08T00:17:39.240086+0000","last_active":"2023-10-08T00:17:40.253430+0000","last_peered":"2023-10-08T00:17:40.253430+0000","last_clean":"2023-10-08T00:17:40.253430+0000","last_became_active":"2023-10-08T00:17:39.239861+0000","last_became_peered":"2023-10-08T00:17:39.239861+0000","last_unstale":"2023-10-08T00:17:40.253430+0000","last_undegraded":"2023-10-08T00:17:40.253430+0000","last_fullsized":"2023-10-08T00:17:40.253430+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:43:12.792670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.245880+0000","last_change":"2023-10-08T00:17:40.244740+0000","last_active":"2023-10-08T00:17:40.245880+0000","last_peered":"2023-10-08T00:17:40.245880+0000","last_clean":"2023-10-08T00:17:40.245880+0000","last_became_active":"2023-10-08T00:17:40.244542+0000","last_became_peered":"2023-10-08T00:17:40.244542+0000","last_unstale":"2023-10-08T00:17:40.245880+0000","last_undegraded":"2023-10-08T00:17:40.245880+0000","last_fullsized":"2023-10-08T00:17:40.245880+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:45:05.381563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1c","version":"153'2","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.259353+0000","last_change":"2023-10-08T00:17:39.243527+0000","last_active":"2023-10-08T00:17:40.259353+0000","last_peered":"2023-10-08T00:17:40.259353+0000","last_clean":"2023-10-08T00:17:40.259353+0000","last_became_active":"2023-10-08T00:17:39.243275+0000","last_became_peered":"2023-10-08T00:17:39.243275+0000","last_unstale":"2023-10-08T00:17:40.259353+0000","last_undegraded":"2023-10-08T00:17:40.259353+0000","last_fullsized":"2023-10-08T00:17:40.259353+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:34:21.713614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.238189+0000","last_change":"2023-10-08T00:17:40.238121+0000","last_active":"2023-10-08T00:17:40.238189+0000","last_peered":"2023-10-08T00:17:40.238189+0000","last_clean":"2023-10-08T00:17:40.238189+0000","last_became_active":"2023-10-08T00:17:40.237877+0000","last_became_peered":"2023-10-08T00:17:40.237877+0000","last_unstale":"2023-10-08T00:17:40.238189+0000","last_undegraded":"2023-10-08T00:17:40.238189+0000","last_fullsized":"2023-10-08T00:17:40.238189+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:06:11.047879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1b","version":"153'1","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.259689+0000","last_change":"2023-10-08T00:17:39.246860+0000","last_active":"2023-10-08T00:17:40.259689+0000","last_peered":"2023-10-08T00:17:40.259689+0000","last_clean":"2023-10-08T00:17:40.259689+0000","last_became_active":"2023-10-08T00:17:39.245722+0000","last_became_peered":"2023-10-08T00:17:39.245722+0000","last_unstale":"2023-10-08T00:17:40.259689+0000","last_undegraded":"2023-10-08T00:17:40.259689+0000","last_fullsized":"2023-10-08T00:17:40.259689+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:26:05.256919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.245900+0000","last_change":"2023-10-08T00:17:40.244945+0000","last_active":"2023-10-08T00:17:40.245900+0000","last_peered":"2023-10-08T00:17:40.245900+0000","last_clean":"2023-10-08T00:17:40.245900+0000","last_became_active":"2023-10-08T00:17:40.244072+0000","last_became_peered":"2023-10-08T00:17:40.244072+0000","last_unstale":"2023-10-08T00:17:40.245900+0000","last_undegraded":"2023-10-08T00:17:40.245900+0000","last_fullsized":"2023-10-08T00:17:40.245900+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:51:35.954365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"153'1","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.256127+0000","last_change":"2023-10-08T00:17:39.241419+0000","last_active":"2023-10-08T00:17:40.256127+0000","last_peered":"2023-10-08T00:17:40.256127+0000","last_clean":"2023-10-08T00:17:40.256127+0000","last_became_active":"2023-10-08T00:17:39.241171+0000","last_became_peered":"2023-10-08T00:17:39.241171+0000","last_unstale":"2023-10-08T00:17:40.256127+0000","last_undegraded":"2023-10-08T00:17:40.256127+0000","last_fullsized":"2023-10-08T00:17:40.256127+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:09:45.565409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.253944+0000","last_change":"2023-10-08T00:17:40.251476+0000","last_active":"2023-10-08T00:17:40.253944+0000","last_peered":"2023-10-08T00:17:40.253944+0000","last_clean":"2023-10-08T00:17:40.253944+0000","last_became_active":"2023-10-08T00:17:40.251249+0000","last_became_peered":"2023-10-08T00:17:40.251249+0000","last_unstale":"2023-10-08T00:17:40.253944+0000","last_undegraded":"2023-10-08T00:17:40.253944+0000","last_fullsized":"2023-10-08T00:17:40.253944+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:59:35.355008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.239229+0000","last_change":"2023-10-08T00:17:39.242323+0000","last_active":"2023-10-08T00:17:40.239229+0000","last_peered":"2023-10-08T00:17:40.239229+0000","last_clean":"2023-10-08T00:17:40.239229+0000","last_became_active":"2023-10-08T00:17:39.242141+0000","last_became_peered":"2023-10-08T00:17:39.242141+0000","last_unstale":"2023-10-08T00:17:40.239229+0000","last_undegraded":"2023-10-08T00:17:40.239229+0000","last_fullsized":"2023-10-08T00:17:40.239229+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:39:55.887101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.248232+0000","last_change":"2023-10-08T00:17:40.247988+0000","last_active":"2023-10-08T00:17:40.248232+0000","last_peered":"2023-10-08T00:17:40.248232+0000","last_clean":"2023-10-08T00:17:40.248232+0000","last_became_active":"2023-10-08T00:17:40.247312+0000","last_became_peered":"2023-10-08T00:17:40.247312+0000","last_unstale":"2023-10-08T00:17:40.248232+0000","last_undegraded":"2023-10-08T00:17:40.248232+0000","last_fullsized":"2023-10-08T00:17:40.248232+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:57:32.743438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.242602+0000","last_change":"2023-10-08T00:17:39.247556+0000","last_active":"2023-10-08T00:17:40.242602+0000","last_peered":"2023-10-08T00:17:40.242602+0000","last_clean":"2023-10-08T00:17:40.242602+0000","last_became_active":"2023-10-08T00:17:39.246978+0000","last_became_peered":"2023-10-08T00:17:39.246978+0000","last_unstale":"2023-10-08T00:17:40.242602+0000","last_undegraded":"2023-10-08T00:17:40.242602+0000","last_fullsized":"2023-10-08T00:17:40.242602+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:02:20.033189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.258618+0000","last_change":"2023-10-08T00:17:40.258531+0000","last_active":"2023-10-08T00:17:40.258618+0000","last_peered":"2023-10-08T00:17:40.258618+0000","last_clean":"2023-10-08T00:17:40.258618+0000","last_became_active":"2023-10-08T00:17:40.256967+0000","last_became_peered":"2023-10-08T00:17:40.256967+0000","last_unstale":"2023-10-08T00:17:40.258618+0000","last_undegraded":"2023-10-08T00:17:40.258618+0000","last_fullsized":"2023-10-08T00:17:40.258618+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:57:49.413076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.242788+0000","last_change":"2023-10-08T00:17:39.246910+0000","last_active":"2023-10-08T00:17:40.242788+0000","last_peered":"2023-10-08T00:17:40.242788+0000","last_clean":"2023-10-08T00:17:40.242788+0000","last_became_active":"2023-10-08T00:17:39.246350+0000","last_became_peered":"2023-10-08T00:17:39.246350+0000","last_unstale":"2023-10-08T00:17:40.242788+0000","last_undegraded":"2023-10-08T00:17:40.242788+0000","last_fullsized":"2023-10-08T00:17:40.242788+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:33:50.993375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.238214+0000","last_change":"2023-10-08T00:17:40.238153+0000","last_active":"2023-10-08T00:17:40.238214+0000","last_peered":"2023-10-08T00:17:40.238214+0000","last_clean":"2023-10-08T00:17:40.238214+0000","last_became_active":"2023-10-08T00:17:40.237924+0000","last_became_peered":"2023-10-08T00:17:40.237924+0000","last_unstale":"2023-10-08T00:17:40.238214+0000","last_undegraded":"2023-10-08T00:17:40.238214+0000","last_fullsized":"2023-10-08T00:17:40.238214+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:22:40.363368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.237011+0000","last_change":"2023-10-08T00:17:39.245641+0000","last_active":"2023-10-08T00:17:40.237011+0000","last_peered":"2023-10-08T00:17:40.237011+0000","last_clean":"2023-10-08T00:17:40.237011+0000","last_became_active":"2023-10-08T00:17:39.245352+0000","last_became_peered":"2023-10-08T00:17:39.245352+0000","last_unstale":"2023-10-08T00:17:40.237011+0000","last_undegraded":"2023-10-08T00:17:40.237011+0000","last_fullsized":"2023-10-08T00:17:40.237011+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:26:05.256919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.246903+0000","last_change":"2023-10-08T00:17:40.246703+0000","last_active":"2023-10-08T00:17:40.246903+0000","last_peered":"2023-10-08T00:17:40.246903+0000","last_clean":"2023-10-08T00:17:40.246903+0000","last_became_active":"2023-10-08T00:17:40.246141+0000","last_became_peered":"2023-10-08T00:17:40.246141+0000","last_unstale":"2023-10-08T00:17:40.246903+0000","last_undegraded":"2023-10-08T00:17:40.246903+0000","last_fullsized":"2023-10-08T00:17:40.246903+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:34:52.462946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.15","version":"153'3","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.259024+0000","last_change":"2023-10-08T00:17:39.242676+0000","last_active":"2023-10-08T00:17:40.259024+0000","last_peered":"2023-10-08T00:17:40.259024+0000","last_clean":"2023-10-08T00:17:40.259024+0000","last_became_active":"2023-10-08T00:17:39.242315+0000","last_became_peered":"2023-10-08T00:17:39.242315+0000","last_unstale":"2023-10-08T00:17:40.259024+0000","last_undegraded":"2023-10-08T00:17:40.259024+0000","last_fullsized":"2023-10-08T00:17:40.259024+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:21:35.315638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.253269+0000","last_change":"2023-10-08T00:17:40.253208+0000","last_active":"2023-10-08T00:17:40.253269+0000","last_peered":"2023-10-08T00:17:40.253269+0000","last_clean":"2023-10-08T00:17:40.253269+0000","last_became_active":"2023-10-08T00:17:40.251582+0000","last_became_peered":"2023-10-08T00:17:40.251582+0000","last_unstale":"2023-10-08T00:17:40.253269+0000","last_undegraded":"2023-10-08T00:17:40.253269+0000","last_fullsized":"2023-10-08T00:17:40.253269+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:47:58.933828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"153'6","reported_seq":16,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.260501+0000","last_change":"2023-10-08T00:17:39.245646+0000","last_active":"2023-10-08T00:17:40.260501+0000","last_peered":"2023-10-08T00:17:40.260501+0000","last_clean":"2023-10-08T00:17:40.260501+0000","last_became_active":"2023-10-08T00:17:39.245356+0000","last_became_peered":"2023-10-08T00:17:39.245356+0000","last_unstale":"2023-10-08T00:17:40.260501+0000","last_undegraded":"2023-10-08T00:17:40.260501+0000","last_fullsized":"2023-10-08T00:17:40.260501+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:37:01.074871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.249335+0000","last_change":"2023-10-08T00:17:40.249199+0000","last_active":"2023-10-08T00:17:40.249335+0000","last_peered":"2023-10-08T00:17:40.249335+0000","last_clean":"2023-10-08T00:17:40.249335+0000","last_became_active":"2023-10-08T00:17:40.248923+0000","last_became_peered":"2023-10-08T00:17:40.248923+0000","last_unstale":"2023-10-08T00:17:40.249335+0000","last_undegraded":"2023-10-08T00:17:40.249335+0000","last_fullsized":"2023-10-08T00:17:40.249335+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:59:35.355008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.244703+0000","last_change":"2023-10-08T00:17:39.242832+0000","last_active":"2023-10-08T00:17:40.244703+0000","last_peered":"2023-10-08T00:17:40.244703+0000","last_clean":"2023-10-08T00:17:40.244703+0000","last_became_active":"2023-10-08T00:17:39.242603+0000","last_became_peered":"2023-10-08T00:17:39.242603+0000","last_unstale":"2023-10-08T00:17:40.244703+0000","last_undegraded":"2023-10-08T00:17:40.244703+0000","last_fullsized":"2023-10-08T00:17:40.244703+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:07:09.770455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.243062+0000","last_change":"2023-10-08T00:17:40.242863+0000","last_active":"2023-10-08T00:17:40.243062+0000","last_peered":"2023-10-08T00:17:40.243062+0000","last_clean":"2023-10-08T00:17:40.243062+0000","last_became_active":"2023-10-08T00:17:40.242610+0000","last_became_peered":"2023-10-08T00:17:40.242610+0000","last_unstale":"2023-10-08T00:17:40.243062+0000","last_undegraded":"2023-10-08T00:17:40.243062+0000","last_fullsized":"2023-10-08T00:17:40.243062+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:22:03.771134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.235924+0000","last_change":"2023-10-08T00:17:39.236771+0000","last_active":"2023-10-08T00:17:40.235924+0000","last_peered":"2023-10-08T00:17:40.235924+0000","last_clean":"2023-10-08T00:17:40.235924+0000","last_became_active":"2023-10-08T00:17:39.236530+0000","last_became_peered":"2023-10-08T00:17:39.236530+0000","last_unstale":"2023-10-08T00:17:40.235924+0000","last_undegraded":"2023-10-08T00:17:40.235924+0000","last_fullsized":"2023-10-08T00:17:40.235924+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:22:39.358915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.245915+0000","last_change":"2023-10-08T00:17:40.245067+0000","last_active":"2023-10-08T00:17:40.245915+0000","last_peered":"2023-10-08T00:17:40.245915+0000","last_clean":"2023-10-08T00:17:40.245915+0000","last_became_active":"2023-10-08T00:17:40.244498+0000","last_became_peered":"2023-10-08T00:17:40.244498+0000","last_unstale":"2023-10-08T00:17:40.245915+0000","last_undegraded":"2023-10-08T00:17:40.245915+0000","last_fullsized":"2023-10-08T00:17:40.245915+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:40:39.003917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.10","version":"153'8","reported_seq":18,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.255994+0000","last_change":"2023-10-08T00:17:39.245625+0000","last_active":"2023-10-08T00:17:40.255994+0000","last_peered":"2023-10-08T00:17:40.255994+0000","last_clean":"2023-10-08T00:17:40.255994+0000","last_became_active":"2023-10-08T00:17:39.245016+0000","last_became_peered":"2023-10-08T00:17:39.245016+0000","last_unstale":"2023-10-08T00:17:40.255994+0000","last_undegraded":"2023-10-08T00:17:40.255994+0000","last_fullsized":"2023-10-08T00:17:40.255994+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:57:31.738986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.244805+0000","last_change":"2023-10-08T00:17:40.244567+0000","last_active":"2023-10-08T00:17:40.244805+0000","last_peered":"2023-10-08T00:17:40.244805+0000","last_clean":"2023-10-08T00:17:40.244805+0000","last_became_active":"2023-10-08T00:17:40.244034+0000","last_became_peered":"2023-10-08T00:17:40.244034+0000","last_unstale":"2023-10-08T00:17:40.244805+0000","last_undegraded":"2023-10-08T00:17:40.244805+0000","last_fullsized":"2023-10-08T00:17:40.244805+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:58:26.467186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"153'1","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.254012+0000","last_change":"2023-10-08T00:17:39.245330+0000","last_active":"2023-10-08T00:17:40.254012+0000","last_peered":"2023-10-08T00:17:40.254012+0000","last_clean":"2023-10-08T00:17:40.254012+0000","last_became_active":"2023-10-08T00:17:39.244579+0000","last_became_peered":"2023-10-08T00:17:39.244579+0000","last_unstale":"2023-10-08T00:17:40.254012+0000","last_undegraded":"2023-10-08T00:17:40.254012+0000","last_fullsized":"2023-10-08T00:17:40.254012+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:36:23.511902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.238774+0000","last_change":"2023-10-08T00:17:40.238724+0000","last_active":"2023-10-08T00:17:40.238774+0000","last_peered":"2023-10-08T00:17:40.238774+0000","last_clean":"2023-10-08T00:17:40.238774+0000","last_became_active":"2023-10-08T00:17:40.238508+0000","last_became_peered":"2023-10-08T00:17:40.238508+0000","last_unstale":"2023-10-08T00:17:40.238774+0000","last_undegraded":"2023-10-08T00:17:40.238774+0000","last_fullsized":"2023-10-08T00:17:40.238774+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:07:10.774907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"153'2","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.253631+0000","last_change":"2023-10-08T00:17:39.241807+0000","last_active":"2023-10-08T00:17:40.253631+0000","last_peered":"2023-10-08T00:17:40.253631+0000","last_clean":"2023-10-08T00:17:40.253631+0000","last_became_active":"2023-10-08T00:17:39.241530+0000","last_became_peered":"2023-10-08T00:17:39.241530+0000","last_unstale":"2023-10-08T00:17:40.253631+0000","last_undegraded":"2023-10-08T00:17:40.253631+0000","last_fullsized":"2023-10-08T00:17:40.253631+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:58:25.462734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.258964+0000","last_change":"2023-10-08T00:17:40.258872+0000","last_active":"2023-10-08T00:17:40.258964+0000","last_peered":"2023-10-08T00:17:40.258964+0000","last_clean":"2023-10-08T00:17:40.258964+0000","last_became_active":"2023-10-08T00:17:40.257807+0000","last_became_peered":"2023-10-08T00:17:40.257807+0000","last_unstale":"2023-10-08T00:17:40.258964+0000","last_undegraded":"2023-10-08T00:17:40.258964+0000","last_fullsized":"2023-10-08T00:17:40.258964+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:22:03.771134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"153'1","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.253847+0000","last_change":"2023-10-08T00:17:39.242761+0000","last_active":"2023-10-08T00:17:40.253847+0000","last_peered":"2023-10-08T00:17:40.253847+0000","last_clean":"2023-10-08T00:17:40.253847+0000","last_became_active":"2023-10-08T00:17:39.241966+0000","last_became_peered":"2023-10-08T00:17:39.241966+0000","last_unstale":"2023-10-08T00:17:40.253847+0000","last_undegraded":"2023-10-08T00:17:40.253847+0000","last_fullsized":"2023-10-08T00:17:40.253847+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:26:48.239749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.257905+0000","last_change":"2023-10-08T00:17:40.257615+0000","last_active":"2023-10-08T00:17:40.257905+0000","last_peered":"2023-10-08T00:17:40.257905+0000","last_clean":"2023-10-08T00:17:40.257905+0000","last_became_active":"2023-10-08T00:17:40.256003+0000","last_became_peered":"2023-10-08T00:17:40.256003+0000","last_unstale":"2023-10-08T00:17:40.257905+0000","last_undegraded":"2023-10-08T00:17:40.257905+0000","last_fullsized":"2023-10-08T00:17:40.257905+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:22:40.363368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.235362+0000","last_change":"2023-10-08T00:17:39.241610+0000","last_active":"2023-10-08T00:17:40.235362+0000","last_peered":"2023-10-08T00:17:40.235362+0000","last_clean":"2023-10-08T00:17:40.235362+0000","last_became_active":"2023-10-08T00:17:39.239512+0000","last_became_peered":"2023-10-08T00:17:39.239512+0000","last_unstale":"2023-10-08T00:17:40.235362+0000","last_undegraded":"2023-10-08T00:17:40.235362+0000","last_fullsized":"2023-10-08T00:17:40.235362+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:01:21.166679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.256354+0000","last_change":"2023-10-08T00:17:40.256287+0000","last_active":"2023-10-08T00:17:40.256354+0000","last_peered":"2023-10-08T00:17:40.256354+0000","last_clean":"2023-10-08T00:17:40.256354+0000","last_became_active":"2023-10-08T00:17:40.255898+0000","last_became_peered":"2023-10-08T00:17:40.255898+0000","last_unstale":"2023-10-08T00:17:40.256354+0000","last_undegraded":"2023-10-08T00:17:40.256354+0000","last_fullsized":"2023-10-08T00:17:40.256354+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:33:38.865833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"153'2","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.253654+0000","last_change":"2023-10-08T00:17:39.242251+0000","last_active":"2023-10-08T00:17:40.253654+0000","last_peered":"2023-10-08T00:17:40.253654+0000","last_clean":"2023-10-08T00:17:40.253654+0000","last_became_active":"2023-10-08T00:17:39.241975+0000","last_became_peered":"2023-10-08T00:17:39.241975+0000","last_unstale":"2023-10-08T00:17:40.253654+0000","last_undegraded":"2023-10-08T00:17:40.253654+0000","last_fullsized":"2023-10-08T00:17:40.253654+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:18:13.853377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.253168+0000","last_change":"2023-10-08T00:17:40.252101+0000","last_active":"2023-10-08T00:17:40.253168+0000","last_peered":"2023-10-08T00:17:40.253168+0000","last_clean":"2023-10-08T00:17:40.253168+0000","last_became_active":"2023-10-08T00:17:40.251300+0000","last_became_peered":"2023-10-08T00:17:40.251300+0000","last_unstale":"2023-10-08T00:17:40.253168+0000","last_undegraded":"2023-10-08T00:17:40.253168+0000","last_fullsized":"2023-10-08T00:17:40.253168+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:53:42.248903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.235995+0000","last_change":"2023-10-08T00:17:39.243908+0000","last_active":"2023-10-08T00:17:40.235995+0000","last_peered":"2023-10-08T00:17:40.235995+0000","last_clean":"2023-10-08T00:17:40.235995+0000","last_became_active":"2023-10-08T00:17:39.243671+0000","last_became_peered":"2023-10-08T00:17:39.243671+0000","last_unstale":"2023-10-08T00:17:40.235995+0000","last_undegraded":"2023-10-08T00:17:40.235995+0000","last_fullsized":"2023-10-08T00:17:40.235995+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:39:56.549324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.247806+0000","last_change":"2023-10-08T00:17:40.247494+0000","last_active":"2023-10-08T00:17:40.247806+0000","last_peered":"2023-10-08T00:17:40.247806+0000","last_clean":"2023-10-08T00:17:40.247806+0000","last_became_active":"2023-10-08T00:17:40.247106+0000","last_became_peered":"2023-10-08T00:17:40.247106+0000","last_unstale":"2023-10-08T00:17:40.247806+0000","last_undegraded":"2023-10-08T00:17:40.247806+0000","last_fullsized":"2023-10-08T00:17:40.247806+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:29:56.427939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.242280+0000","last_change":"2023-10-08T00:17:39.248289+0000","last_active":"2023-10-08T00:17:40.242280+0000","last_peered":"2023-10-08T00:17:40.242280+0000","last_clean":"2023-10-08T00:17:40.242280+0000","last_became_active":"2023-10-08T00:17:39.248014+0000","last_became_peered":"2023-10-08T00:17:39.248014+0000","last_unstale":"2023-10-08T00:17:40.242280+0000","last_undegraded":"2023-10-08T00:17:40.242280+0000","last_fullsized":"2023-10-08T00:17:40.242280+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:58:25.462734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.245930+0000","last_change":"2023-10-08T00:17:40.245809+0000","last_active":"2023-10-08T00:17:40.245930+0000","last_peered":"2023-10-08T00:17:40.245930+0000","last_clean":"2023-10-08T00:17:40.245930+0000","last_became_active":"2023-10-08T00:17:40.245131+0000","last_became_peered":"2023-10-08T00:17:40.245131+0000","last_unstale":"2023-10-08T00:17:40.245930+0000","last_undegraded":"2023-10-08T00:17:40.245930+0000","last_fullsized":"2023-10-08T00:17:40.245930+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:11:59.235622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.237238+0000","last_change":"2023-10-08T00:17:39.243240+0000","last_active":"2023-10-08T00:17:40.237238+0000","last_peered":"2023-10-08T00:17:40.237238+0000","last_clean":"2023-10-08T00:17:40.237238+0000","last_became_active":"2023-10-08T00:17:39.242906+0000","last_became_peered":"2023-10-08T00:17:39.242906+0000","last_unstale":"2023-10-08T00:17:40.237238+0000","last_undegraded":"2023-10-08T00:17:40.237238+0000","last_fullsized":"2023-10-08T00:17:40.237238+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:42:32.648608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.251861+0000","last_change":"2023-10-08T00:17:40.251800+0000","last_active":"2023-10-08T00:17:40.251861+0000","last_peered":"2023-10-08T00:17:40.251861+0000","last_clean":"2023-10-08T00:17:40.251861+0000","last_became_active":"2023-10-08T00:17:40.250816+0000","last_became_peered":"2023-10-08T00:17:40.250816+0000","last_unstale":"2023-10-08T00:17:40.251861+0000","last_undegraded":"2023-10-08T00:17:40.251861+0000","last_fullsized":"2023-10-08T00:17:40.251861+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:06:58.458653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"153'2","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.254730+0000","last_change":"2023-10-08T00:17:39.244806+0000","last_active":"2023-10-08T00:17:40.254730+0000","last_peered":"2023-10-08T00:17:40.254730+0000","last_clean":"2023-10-08T00:17:40.254730+0000","last_became_active":"2023-10-08T00:17:39.244600+0000","last_became_peered":"2023-10-08T00:17:39.244600+0000","last_unstale":"2023-10-08T00:17:40.254730+0000","last_undegraded":"2023-10-08T00:17:40.254730+0000","last_fullsized":"2023-10-08T00:17:40.254730+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:14:31.718231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.259112+0000","last_change":"2023-10-08T00:17:40.259048+0000","last_active":"2023-10-08T00:17:40.259112+0000","last_peered":"2023-10-08T00:17:40.259112+0000","last_clean":"2023-10-08T00:17:40.259112+0000","last_became_active":"2023-10-08T00:17:40.258119+0000","last_became_peered":"2023-10-08T00:17:40.258119+0000","last_unstale":"2023-10-08T00:17:40.259112+0000","last_undegraded":"2023-10-08T00:17:40.259112+0000","last_fullsized":"2023-10-08T00:17:40.259112+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:37:58.966950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.235657+0000","last_change":"2023-10-08T00:17:39.247178+0000","last_active":"2023-10-08T00:17:40.235657+0000","last_peered":"2023-10-08T00:17:40.235657+0000","last_clean":"2023-10-08T00:17:40.235657+0000","last_became_active":"2023-10-08T00:17:39.246939+0000","last_became_peered":"2023-10-08T00:17:39.246939+0000","last_unstale":"2023-10-08T00:17:40.235657+0000","last_undegraded":"2023-10-08T00:17:40.235657+0000","last_fullsized":"2023-10-08T00:17:40.235657+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:35:11.105844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.254846+0000","last_change":"2023-10-08T00:17:40.254779+0000","last_active":"2023-10-08T00:17:40.254846+0000","last_peered":"2023-10-08T00:17:40.254846+0000","last_clean":"2023-10-08T00:17:40.254846+0000","last_became_active":"2023-10-08T00:17:40.254566+0000","last_became_peered":"2023-10-08T00:17:40.254566+0000","last_unstale":"2023-10-08T00:17:40.254846+0000","last_undegraded":"2023-10-08T00:17:40.254846+0000","last_fullsized":"2023-10-08T00:17:40.254846+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:09:08.837995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.235898+0000","last_change":"2023-10-08T00:17:39.247837+0000","last_active":"2023-10-08T00:17:40.235898+0000","last_peered":"2023-10-08T00:17:40.235898+0000","last_clean":"2023-10-08T00:17:40.235898+0000","last_became_active":"2023-10-08T00:17:39.247595+0000","last_became_peered":"2023-10-08T00:17:39.247595+0000","last_unstale":"2023-10-08T00:17:40.235898+0000","last_undegraded":"2023-10-08T00:17:40.235898+0000","last_fullsized":"2023-10-08T00:17:40.235898+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:34:21.713614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.244907+0000","last_change":"2023-10-08T00:17:40.244842+0000","last_active":"2023-10-08T00:17:40.244907+0000","last_peered":"2023-10-08T00:17:40.244907+0000","last_clean":"2023-10-08T00:17:40.244907+0000","last_became_active":"2023-10-08T00:17:40.244351+0000","last_became_peered":"2023-10-08T00:17:40.244351+0000","last_unstale":"2023-10-08T00:17:40.244907+0000","last_undegraded":"2023-10-08T00:17:40.244907+0000","last_fullsized":"2023-10-08T00:17:40.244907+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:25:09.195876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.242386+0000","last_change":"2023-10-08T00:17:39.243249+0000","last_active":"2023-10-08T00:17:40.242386+0000","last_peered":"2023-10-08T00:17:40.242386+0000","last_clean":"2023-10-08T00:17:40.242386+0000","last_became_active":"2023-10-08T00:17:39.242464+0000","last_became_peered":"2023-10-08T00:17:39.242464+0000","last_unstale":"2023-10-08T00:17:40.242386+0000","last_undegraded":"2023-10-08T00:17:40.242386+0000","last_fullsized":"2023-10-08T00:17:40.242386+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:57:48.408623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.245862+0000","last_change":"2023-10-08T00:17:40.244292+0000","last_active":"2023-10-08T00:17:40.245862+0000","last_peered":"2023-10-08T00:17:40.245862+0000","last_clean":"2023-10-08T00:17:40.245862+0000","last_became_active":"2023-10-08T00:17:40.243716+0000","last_became_peered":"2023-10-08T00:17:40.243716+0000","last_unstale":"2023-10-08T00:17:40.245862+0000","last_undegraded":"2023-10-08T00:17:40.245862+0000","last_fullsized":"2023-10-08T00:17:40.245862+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:24:21.948361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.241028+0000","last_change":"2023-10-08T00:17:39.248190+0000","last_active":"2023-10-08T00:17:40.241028+0000","last_peered":"2023-10-08T00:17:40.241028+0000","last_clean":"2023-10-08T00:17:40.241028+0000","last_became_active":"2023-10-08T00:17:39.247710+0000","last_became_peered":"2023-10-08T00:17:39.247710+0000","last_unstale":"2023-10-08T00:17:40.241028+0000","last_undegraded":"2023-10-08T00:17:40.241028+0000","last_fullsized":"2023-10-08T00:17:40.241028+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:42:45.677981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.246381+0000","last_change":"2023-10-08T00:17:40.246334+0000","last_active":"2023-10-08T00:17:40.246381+0000","last_peered":"2023-10-08T00:17:40.246381+0000","last_clean":"2023-10-08T00:17:40.246381+0000","last_became_active":"2023-10-08T00:17:40.246112+0000","last_became_peered":"2023-10-08T00:17:40.246112+0000","last_unstale":"2023-10-08T00:17:40.246381+0000","last_undegraded":"2023-10-08T00:17:40.246381+0000","last_fullsized":"2023-10-08T00:17:40.246381+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:39:34.388679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.243525+0000","last_change":"2023-10-08T00:17:39.247326+0000","last_active":"2023-10-08T00:17:40.243525+0000","last_peered":"2023-10-08T00:17:40.243525+0000","last_clean":"2023-10-08T00:17:40.243525+0000","last_became_active":"2023-10-08T00:17:39.247081+0000","last_became_peered":"2023-10-08T00:17:39.247081+0000","last_unstale":"2023-10-08T00:17:40.243525+0000","last_undegraded":"2023-10-08T00:17:40.243525+0000","last_fullsized":"2023-10-08T00:17:40.243525+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:22:39.358915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.244248+0000","last_change":"2023-10-08T00:17:40.243986+0000","last_active":"2023-10-08T00:17:40.244248+0000","last_peered":"2023-10-08T00:17:40.244248+0000","last_clean":"2023-10-08T00:17:40.244248+0000","last_became_active":"2023-10-08T00:17:40.243578+0000","last_became_peered":"2023-10-08T00:17:40.243578+0000","last_unstale":"2023-10-08T00:17:40.244248+0000","last_undegraded":"2023-10-08T00:17:40.244248+0000","last_fullsized":"2023-10-08T00:17:40.244248+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:36:24.516355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.237911+0000","last_change":"2023-10-08T00:17:39.244659+0000","last_active":"2023-10-08T00:17:40.237911+0000","last_peered":"2023-10-08T00:17:40.237911+0000","last_clean":"2023-10-08T00:17:40.237911+0000","last_became_active":"2023-10-08T00:17:39.244411+0000","last_became_peered":"2023-10-08T00:17:39.244411+0000","last_unstale":"2023-10-08T00:17:40.237911+0000","last_undegraded":"2023-10-08T00:17:40.237911+0000","last_fullsized":"2023-10-08T00:17:40.237911+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:07:09.770455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.253213+0000","last_change":"2023-10-08T00:17:40.253156+0000","last_active":"2023-10-08T00:17:40.253213+0000","last_peered":"2023-10-08T00:17:40.253213+0000","last_clean":"2023-10-08T00:17:40.253213+0000","last_became_active":"2023-10-08T00:17:40.252770+0000","last_became_peered":"2023-10-08T00:17:40.252770+0000","last_unstale":"2023-10-08T00:17:40.253213+0000","last_undegraded":"2023-10-08T00:17:40.253213+0000","last_fullsized":"2023-10-08T00:17:40.253213+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:55:38.261584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.244676+0000","last_change":"2023-10-08T00:17:39.240855+0000","last_active":"2023-10-08T00:17:40.244676+0000","last_peered":"2023-10-08T00:17:40.244676+0000","last_clean":"2023-10-08T00:17:40.244676+0000","last_became_active":"2023-10-08T00:17:39.240622+0000","last_became_peered":"2023-10-08T00:17:39.240622+0000","last_unstale":"2023-10-08T00:17:40.244676+0000","last_undegraded":"2023-10-08T00:17:40.244676+0000","last_fullsized":"2023-10-08T00:17:40.244676+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:22:39.358915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.242556+0000","last_change":"2023-10-08T00:17:40.242493+0000","last_active":"2023-10-08T00:17:40.242556+0000","last_peered":"2023-10-08T00:17:40.242556+0000","last_clean":"2023-10-08T00:17:40.242556+0000","last_became_active":"2023-10-08T00:17:40.242276+0000","last_became_peered":"2023-10-08T00:17:40.242276+0000","last_unstale":"2023-10-08T00:17:40.242556+0000","last_undegraded":"2023-10-08T00:17:40.242556+0000","last_fullsized":"2023-10-08T00:17:40.242556+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:39.226667+0000","last_clean_scrub_stamp":"2023-10-08T00:17:39.226667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:26:46.315710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"153'2","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.254486+0000","last_change":"2023-10-08T00:17:39.246049+0000","last_active":"2023-10-08T00:17:40.254486+0000","last_peered":"2023-10-08T00:17:40.254486+0000","last_clean":"2023-10-08T00:17:40.254486+0000","last_became_active":"2023-10-08T00:17:39.245601+0000","last_became_peered":"2023-10-08T00:17:39.245601+0000","last_unstale":"2023-10-08T00:17:40.254486+0000","last_undegraded":"2023-10-08T00:17:40.254486+0000","last_fullsized":"2023-10-08T00:17:40.254486+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:17:38.222215+0000","last_clean_scrub_stamp":"2023-10-08T00:17:38.222215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:57:48.408623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":277,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T00:17:40.243521+0000","last_change":"2023-10-08T00:10:43.728490+0000","last_active":"2023-10-08T00:17:40.243521+0000","last_peered":"2023-10-08T00:17:40.243521+0000","last_clean":"2023-10-08T00:17:40.243521+0000","last_became_active":"2023-10-08T00:10:43.728341+0000","last_became_peered":"2023-10-08T00:10:43.728341+0000","last_unstale":"2023-10-08T00:17:40.243521+0000","last_undegraded":"2023-10-08T00:17:40.243521+0000","last_fullsized":"2023-10-08T00:17:40.243521+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_clean_scrub_stamp":"2023-10-08T00:10:42.716096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:06:31.035293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574934,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":838,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194474,"statfs":{"total":200000000,"available":199141752,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.53500000000000003,"15min":0.50700000000000001},"min":{"1min":0.373,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.72499999999999998,"5min":1.129,"15min":1.129},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53900000000000003,"15min":0.54400000000000004},"min":{"1min":0.40699999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.71999999999999997,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.61199999999999999}]},{"osd":1,"last update":"Sun Oct 8 00:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.56599999999999995,"15min":0.53900000000000003},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.77900000000000003,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.58899999999999997,"15min":0.56699999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.38200000000000001},"max":{"1min":0.76500000000000001,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.55900000000000005}]},{"osd":2,"last update":"Sun Oct 8 00:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.55300000000000005,"15min":0.54100000000000004},"min":{"1min":0.38,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.55300000000000005,"15min":0.52600000000000002},"min":{"1min":0.34499999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.68999999999999995,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.56699999999999995}]},{"osd":3,"last update":"Sun Oct 8 00:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55500000000000005,"15min":0.52600000000000002},"min":{"1min":0.376,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.749,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56899999999999995,"15min":0.55900000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.69999999999999996,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.49099999999999999}]},{"osd":4,"last update":"Sun Oct 8 00:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.496,"15min":0.51700000000000002},"min":{"1min":0.27200000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.50700000000000001,"15min":0.52500000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.28199999999999997},"max":{"1min":0.69699999999999995,"5min":0.76500000000000001,"15min":0.78900000000000003},"last":0.58999999999999997}]},{"osd":5,"last update":"Sun Oct 8 00:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.51400000000000001,"15min":0.54400000000000004},"min":{"1min":0.41699999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.68000000000000005,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.623},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.54000000000000004,"15min":0.57899999999999996},"min":{"1min":0.33100000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.60499999999999998}]},{"osd":6,"last update":"Sun Oct 8 00:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.58499999999999996,"15min":0.59599999999999997},"min":{"1min":0.42599999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.58599999999999997,"15min":0.624},"min":{"1min":0.35999999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.82999999999999996,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.66200000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574934,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":838,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194474,"statfs":{"total":200000000,"available":199141786,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:16:55 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.52100000000000002,"15min":0.48899999999999999},"min":{"1min":0.35299999999999998,"5min":0.34599999999999997,"15min":0.29199999999999998},"max":{"1min":0.64600000000000002,"5min":0.752,"15min":1.012},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.52500000000000002,"15min":0.65200000000000002},"min":{"1min":0.317,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.77000000000000002,"5min":1.2789999999999999,"15min":5.1230000000000002},"last":0.47299999999999998}]},{"osd":1,"last update":"Sun Oct 8 00:16:55 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.55600000000000005,"15min":0.52600000000000002},"min":{"1min":0.29899999999999999,"5min":0.29299999999999998,"15min":0.28899999999999998},"max":{"1min":0.78100000000000003,"5min":0.82499999999999996,"15min":1.1419999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.57299999999999995,"15min":0.56499999999999995},"min":{"1min":0.41099999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73599999999999999,"5min":0.98999999999999999,"15min":1.1539999999999999},"last":0.503}]},{"osd":2,"last update":"Sun Oct 8 00:16:55 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.54200000000000004,"15min":0.53000000000000003},"min":{"1min":0.36299999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.63900000000000001,"5min":0.75700000000000001,"15min":1.3149999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.57399999999999995,"15min":0.56899999999999995},"min":{"1min":0.39200000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":1.3280000000000001},"last":0.443}]},{"osd":3,"last update":"Sun Oct 8 00:16:55 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.56899999999999995,"15min":0.54100000000000004},"min":{"1min":0.38,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.72899999999999998,"5min":0.84199999999999997,"15min":1.046},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.56899999999999995,"15min":0.54500000000000004},"min":{"1min":0.40999999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.69399999999999995,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.61299999999999999}]},{"osd":4,"last update":"Sun Oct 8 00:16:55 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.47299999999999998,"15min":0.496},"min":{"1min":0.30099999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.69699999999999995,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55300000000000005,"15min":0.56299999999999994},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79500000000000004,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.66400000000000003}]},{"osd":5,"last update":"Sun Oct 8 00:16:55 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48699999999999999},"min":{"1min":0.315,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.52000000000000002,"15min":0.55100000000000005},"min":{"1min":0.39200000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.59999999999999998}]},{"osd":7,"last update":"Sun Oct 8 00:16:55 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.53800000000000003},"min":{"1min":0.34699999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.82899999999999996,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.628},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.54000000000000004,"15min":0.54600000000000004},"min":{"1min":0.35299999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.753,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.67600000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1416,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193896,"statfs":{"total":200000000,"available":198549860,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:16:51 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.34200000000000003,"15min":0.312},"min":{"1min":0.215,"5min":0.20300000000000001,"15min":0.19500000000000001},"max":{"1min":0.55900000000000005,"5min":0.56899999999999995,"15min":0.56899999999999995},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.375,"5min":0.39400000000000002,"15min":0.379},"min":{"1min":0.29699999999999999,"5min":0.255,"15min":0.255},"max":{"1min":0.54600000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.55100000000000005}]},{"osd":2,"last update":"Sun Oct 8 00:16:51 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.35199999999999998,"15min":0.33800000000000002},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.53200000000000003,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.38200000000000001,"15min":0.34599999999999997},"min":{"1min":0.26500000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.57399999999999995,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.48399999999999999}]},{"osd":3,"last update":"Sun Oct 8 00:16:51 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.39200000000000002,"15min":0.373},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.56999999999999995,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.35399999999999998,"15min":0.34399999999999997},"min":{"1min":0.255,"5min":0.222,"15min":0.222},"max":{"1min":0.502,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.441}]},{"osd":4,"last update":"Sun Oct 8 00:16:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.56000000000000005,"15min":0.54200000000000004},"min":{"1min":0.34000000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77100000000000002,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.57799999999999996,"15min":0.56799999999999995},"min":{"1min":0.40899999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.754,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.75600000000000001}]},{"osd":5,"last update":"Sun Oct 8 00:16:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57099999999999995,"15min":0.54200000000000004},"min":{"1min":0.372,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.81000000000000005,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.58799999999999997,"15min":0.57599999999999996},"min":{"1min":0.41999999999999998,"5min":0.38300000000000001,"15min":0.30399999999999999},"max":{"1min":0.84999999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.78800000000000003}]},{"osd":6,"last update":"Sun Oct 8 00:16:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58499999999999996,"15min":0.56899999999999995},"min":{"1min":0.379,"5min":0.317,"15min":0.317},"max":{"1min":0.80400000000000005,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.58099999999999996,"15min":0.56399999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.80500000000000005,"5min":0.83899999999999997,"15min":0.84999999999999998},"last":0.64500000000000002}]},{"osd":7,"last update":"Sun Oct 8 00:16:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.58399999999999996,"15min":0.56599999999999995},"min":{"1min":0.40100000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.79600000000000004,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.61799999999999999,"15min":0.59399999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81899999999999995,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.83399999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574934,"num_pgs":32,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":839,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194472,"statfs":{"total":200000000,"available":199140194,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 00:16:55 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.33800000000000002,"15min":0.33100000000000002},"min":{"1min":0.252,"5min":0.20699999999999999,"15min":0.193},"max":{"1min":0.38300000000000001,"5min":0.60499999999999998,"15min":1.0640000000000001},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.35199999999999998,"15min":0.34599999999999997},"min":{"1min":0.217,"5min":0.217,"15min":0.20300000000000001},"max":{"1min":0.47499999999999998,"5min":0.57699999999999996,"15min":0.87},"last":0.28000000000000003}]},{"osd":2,"last update":"Sun Oct 8 00:16:55 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.38500000000000001,"15min":0.38700000000000001},"min":{"1min":0.29899999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.54200000000000004,"5min":0.68000000000000005,"15min":0.93799999999999994},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.35599999999999998,"15min":0.69199999999999995},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.185},"max":{"1min":0.44,"5min":0.65600000000000003,"15min":12.019},"last":0.33600000000000002}]},{"osd":3,"last update":"Sun Oct 8 00:16:55 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.46000000000000002,"5min":0.69999999999999996,"15min":1.1020000000000001},"last":0.308},{"interface":"front","average":{"1min":0.374,"5min":0.41099999999999998,"15min":0.39800000000000002},"min":{"1min":0.28799999999999998,"5min":0.28299999999999997,"15min":0.246},"max":{"1min":0.51700000000000002,"5min":0.72899999999999998,"15min":1.0409999999999999},"last":0.40200000000000002}]},{"osd":4,"last update":"Sun Oct 8 00:16:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60999999999999999,"15min":0.61599999999999999},"min":{"1min":0.46000000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81000000000000005,"5min":0.96399999999999997,"15min":1.1779999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.61799999999999999,"15min":0.623},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.81899999999999995,"5min":0.94999999999999996,"15min":0.96199999999999997},"last":0.621}]},{"osd":5,"last update":"Sun Oct 8 00:16:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58399999999999996,"15min":0.93100000000000005},"min":{"1min":0.39300000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.83299999999999996,"5min":0.85599999999999998,"15min":11.992000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.59799999999999998,"15min":0.58799999999999997},"min":{"1min":0.41599999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.83199999999999996,"5min":0.90300000000000002,"15min":1.1299999999999999},"last":0.70899999999999996}]},{"osd":6,"last update":"Sun Oct 8 00:16:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.59499999999999997,"15min":0.58699999999999997},"min":{"1min":0.40600000000000003,"5min":0.378,"15min":0.378},"max":{"1min":0.86299999999999999,"5min":0.88700000000000001,"15min":0.91200000000000003},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59499999999999997,"15min":0.91400000000000003},"min":{"1min":0.41799999999999998,"5min":0.41199999999999998,"15min":0.38100000000000001},"max":{"1min":0.75800000000000001,"5min":0.96599999999999997,"15min":11.927},"last":0.60599999999999998}]},{"osd":7,"last update":"Sun Oct 8 00:16:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.60099999999999998,"15min":0.93300000000000005},"min":{"1min":0.44,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.79000000000000004,"5min":1.0069999999999999,"15min":11.965},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.62,"15min":0.622},"min":{"1min":0.41499999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.84799999999999998,"5min":0.98299999999999998,"15min":1.1539999999999999},"last":0.57899999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574934,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":838,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194473,"statfs":{"total":200000000,"available":199141142,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.33600000000000002,"15min":0.38400000000000001},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.53400000000000003,"5min":0.72399999999999998,"15min":2.5139999999999998},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.34100000000000003,"15min":0.39400000000000002},"min":{"1min":0.22700000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.55600000000000005,"5min":0.56399999999999995,"15min":2.1309999999999998},"last":0.32000000000000001}]},{"osd":1,"last update":"Sun Oct 8 00:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.34899999999999998,"15min":0.38800000000000001},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":2.2639999999999998},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.36799999999999999,"15min":0.42699999999999999},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":2.5379999999999998},"last":0.33800000000000002}]},{"osd":3,"last update":"Sun Oct 8 00:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.38400000000000001,"15min":0.42399999999999999},"min":{"1min":0.28100000000000003,"5min":0.254,"15min":0.254},"max":{"1min":0.61599999999999999,"5min":0.61899999999999999,"15min":2.2890000000000001},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40100000000000002,"15min":0.45700000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":2.5619999999999998},"last":0.38400000000000001}]},{"osd":4,"last update":"Sun Oct 8 00:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56200000000000006,"15min":0.60999999999999999},"min":{"1min":0.38200000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.76600000000000001,"5min":1.073,"15min":2.488},"last":0.433},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58799999999999997,"15min":0.60499999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.85899999999999999,"5min":0.93600000000000005,"15min":2.073},"last":0.39300000000000002}]},{"osd":5,"last update":"Sun Oct 8 00:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.60899999999999999,"15min":0.65100000000000002},"min":{"1min":0.374,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.82199999999999995,"5min":1.2130000000000001,"15min":2.2069999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.629,"15min":0.65900000000000003},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.39400000000000002},"max":{"1min":0.94199999999999995,"5min":1.0369999999999999,"15min":2.456},"last":0.53000000000000003}]},{"osd":6,"last update":"Sun Oct 8 00:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59599999999999997,"15min":0.61899999999999999},"min":{"1min":0.39200000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.93000000000000005,"5min":1.05,"15min":2.0339999999999998},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60599999999999998,"15min":0.61299999999999999},"min":{"1min":0.439,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.873,"5min":1.0940000000000001,"15min":2.1699999999999999},"last":0.48899999999999999}]},{"osd":7,"last update":"Sun Oct 8 00:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.61199999999999999,"15min":0.63300000000000001},"min":{"1min":0.42999999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.95999999999999996,"5min":1.0229999999999999,"15min":2.1030000000000002},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60999999999999999,"15min":0.627},"min":{"1min":0.443,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.90500000000000003,"5min":1.0569999999999999,"15min":2.2349999999999999},"last":0.45700000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574934,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":832,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194480,"statfs":{"total":200000000,"available":199148004,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:16:49 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.32000000000000001,"15min":0.36399999999999999},"min":{"1min":0.218,"5min":0.192,"15min":0.192},"max":{"1min":0.55200000000000005,"5min":0.71199999999999997,"15min":1.46},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.32000000000000001,"5min":0.33200000000000002,"15min":0.34000000000000002},"min":{"1min":0.26500000000000001,"5min":0.192,"15min":0.192},"max":{"1min":0.42399999999999999,"5min":0.504,"15min":0.54800000000000004},"last":0.27900000000000003}]},{"osd":1,"last update":"Sun Oct 8 00:16:49 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.34200000000000003,"15min":0.754},"min":{"1min":0.23699999999999999,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.46200000000000002,"5min":0.57599999999999996,"15min":12.351000000000001},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.36399999999999999,"15min":0.36499999999999999},"min":{"1min":0.248,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.54200000000000004,"5min":0.60199999999999998,"15min":0.71999999999999997},"last":0.36199999999999999}]},{"osd":2,"last update":"Sun Oct 8 00:16:49 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.38100000000000001,"15min":0.41399999999999998},"min":{"1min":0.30099999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.48999999999999999,"5min":0.70499999999999996,"15min":1.5489999999999999},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.37,"5min":0.39000000000000001,"15min":0.38700000000000001},"min":{"1min":0.28299999999999997,"5min":0.26800000000000002,"15min":0.246},"max":{"1min":0.45500000000000002,"5min":0.64700000000000002,"15min":0.753},"last":0.308}]},{"osd":4,"last update":"Sun Oct 8 00:16:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59999999999999998,"15min":0.95099999999999996},"min":{"1min":0.39600000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.80000000000000004,"5min":1.024,"15min":11.333},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61599999999999999,"15min":0.97999999999999998},"min":{"1min":0.51000000000000001,"5min":0.376,"15min":0.36099999999999999},"max":{"1min":0.77800000000000002,"5min":0.94899999999999995,"15min":12.321999999999999},"last":0.497}]},{"osd":5,"last update":"Sun Oct 8 00:16:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60199999999999998,"15min":0.98799999999999999},"min":{"1min":0.41099999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.73499999999999999,"5min":0.92000000000000004,"15min":12.291},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.622,"15min":0.60499999999999998},"min":{"1min":0.45600000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.73199999999999998,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.46100000000000002}]},{"osd":6,"last update":"Sun Oct 8 00:16:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58099999999999996,"15min":0.91800000000000004},"min":{"1min":0.442,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.83599999999999997,"5min":0.96199999999999997,"15min":11.288},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.59199999999999997,"15min":0.96999999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.73599999999999999,"5min":1.002,"15min":12.253},"last":0.438}]},{"osd":7,"last update":"Sun Oct 8 00:16:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.60899999999999999,"15min":0.58199999999999996},"min":{"1min":0.47699999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80800000000000005,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.61199999999999999,"15min":0.95299999999999996},"min":{"1min":0.46200000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82599999999999996,"5min":1.381,"15min":11.363},"last":0.45100000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574934,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":838,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194474,"statfs":{"total":200000000,"available":199141786,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.56499999999999995,"15min":0.55500000000000005},"min":{"1min":0.36899999999999999,"5min":0.193,"15min":0.193},"max":{"1min":0.67800000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57199999999999995,"15min":0.58299999999999996},"min":{"1min":0.42599999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.60599999999999998}]},{"osd":1,"last update":"Sun Oct 8 00:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.52400000000000002,"15min":0.52000000000000002},"min":{"1min":0.35199999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76400000000000001,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52800000000000002,"15min":0.53300000000000003},"min":{"1min":0.34999999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.64400000000000002,"5min":0.80400000000000005,"15min":0.81299999999999994},"last":0.59099999999999997}]},{"osd":2,"last update":"Sun Oct 8 00:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.51100000000000001,"15min":0.51800000000000002},"min":{"1min":0.39700000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.66600000000000004,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53000000000000003,"15min":0.51500000000000001},"min":{"1min":0.41799999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.67900000000000005,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.47799999999999998}]},{"osd":3,"last update":"Sun Oct 8 00:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.52800000000000002,"15min":0.53700000000000003},"min":{"1min":0.33700000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.71999999999999997,"5min":0.79800000000000004,"15min":0.85399999999999998},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53500000000000003,"15min":0.53400000000000003},"min":{"1min":0.39100000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.505}]},{"osd":5,"last update":"Sun Oct 8 00:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.45800000000000002,"15min":0.49099999999999999},"min":{"1min":0.317,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.65700000000000003,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.503,"5min":0.49199999999999999,"15min":0.50900000000000001},"min":{"1min":0.315,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.73899999999999999,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.45400000000000001}]},{"osd":6,"last update":"Sun Oct 8 00:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.47199999999999998,"15min":0.48799999999999999},"min":{"1min":0.33700000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.69899999999999995,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.51900000000000002,"15min":0.55100000000000005},"min":{"1min":0.438,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.72399999999999998,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.54600000000000004}]},{"osd":7,"last update":"Sun Oct 8 00:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.53200000000000003,"15min":0.55000000000000004},"min":{"1min":0.36899999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.78000000000000003,"5min":0.79700000000000004,"15min":0.90100000000000002},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56699999999999995,"15min":0.57899999999999996},"min":{"1min":0.47099999999999997,"5min":0.376,"15min":0.376},"max":{"1min":0.75600000000000001,"5min":0.78000000000000003,"15min":0.78500000000000003},"last":0.57099999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574934,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1415,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193897,"statfs":{"total":200000000,"available":198550704,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.48699999999999999,"15min":0.48799999999999999},"min":{"1min":0.36099999999999999,"5min":0.249,"15min":0.249},"max":{"1min":0.63100000000000001,"5min":0.68799999999999994,"15min":0.72699999999999998},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53500000000000003,"15min":0.55600000000000005},"min":{"1min":0.39200000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.88600000000000001},"last":0.495}]},{"osd":1,"last update":"Sun Oct 8 00:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.52700000000000002,"15min":0.52500000000000002},"min":{"1min":0.34399999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.82999999999999996},"last":0.623},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.505,"15min":0.52200000000000002},"min":{"1min":0.39900000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.71699999999999997,"5min":0.747,"15min":0.755},"last":0.433}]},{"osd":2,"last update":"Sun Oct 8 00:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52300000000000002,"15min":0.52900000000000003},"min":{"1min":0.34499999999999997,"5min":0.34399999999999997,"15min":0.32400000000000001},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.86199999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54300000000000004,"15min":0.56499999999999995},"min":{"1min":0.41499999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.78500000000000003,"5min":0.79500000000000004,"15min":0.95399999999999996},"last":0.46899999999999997}]},{"osd":3,"last update":"Sun Oct 8 00:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55300000000000005,"15min":0.54800000000000004},"min":{"1min":0.42799999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.69799999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.55500000000000005,"15min":0.57499999999999996},"min":{"1min":0.41999999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.71999999999999997,"5min":0.72299999999999998,"15min":1.0529999999999999},"last":0.58599999999999997}]},{"osd":4,"last update":"Sun Oct 8 00:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.47899999999999998,"15min":0.47699999999999998},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.27700000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.79200000000000004},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.52100000000000002,"15min":0.51700000000000002},"min":{"1min":0.314,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.41599999999999998}]},{"osd":6,"last update":"Sun Oct 8 00:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.51700000000000002,"15min":0.505},"min":{"1min":0.33000000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.76500000000000001,"5min":0.78900000000000003,"15min":0.79700000000000004},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.51700000000000002,"15min":0.5},"min":{"1min":0.34499999999999997,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.71499999999999997,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.60199999999999998}]},{"osd":7,"last update":"Sun Oct 8 00:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.53500000000000003,"15min":0.52200000000000002},"min":{"1min":0.35699999999999998,"5min":0.33300000000000002,"15min":0.29399999999999998},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56000000000000005,"15min":0.55800000000000005},"min":{"1min":0.374,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.65500000000000003}]}]}],"pool_statfs":[]}} 2023-10-08T00:17:44.661 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T00:17:44.661 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-08T00:17:44.816 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.815+0000 7f46d0089700 1 Processor -- start 2023-10-08T00:17:44.816 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.815+0000 7f46d0089700 1 -- start start 2023-10-08T00:17:44.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.816+0000 7f46d0089700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f46c8159ec0 0x7f46c80af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:44.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.816+0000 7f46d0089700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46c80b0790 con 0x7f46c806acb0 2023-10-08T00:17:44.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.816+0000 7f46d0089700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46c80ce1f0 con 0x7f46c815a290 2023-10-08T00:17:44.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.816+0000 7f46d0089700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46c80b0b20 con 0x7f46c8159ec0 2023-10-08T00:17:44.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.816+0000 7f46ce626700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f46c806acb0 0x7f46c80cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:60452/0 (socket says 172.21.15.99:60452) 2023-10-08T00:17:44.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.816+0000 7f46ce626700 1 -- 172.21.15.99:0/3101930439 learned_addr learned my addr 172.21.15.99:0/3101930439 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:17:44.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.817+0000 7f46cde25700 1 --2- 172.21.15.99:0/3101930439 >> v2:172.21.15.191:3300/0 conn(0x7f46c8159ec0 0x7f46c80af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:44.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.817+0000 7f46cce23700 1 -- 172.21.15.99:0/3101930439 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1905920109 0 0) 0x7f46c80b0790 con 0x7f46c806acb0 2023-10-08T00:17:44.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.817+0000 7f46cce23700 1 -- 172.21.15.99:0/3101930439 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46b4003570 con 0x7f46c806acb0 2023-10-08T00:17:44.820 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.817+0000 7f46cce23700 1 -- 172.21.15.99:0/3101930439 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 332831372 0 0) 0x7f46b4003570 con 0x7f46c806acb0 2023-10-08T00:17:44.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.817+0000 7f46cce23700 1 -- 172.21.15.99:0/3101930439 >> v2:172.21.15.191:3300/0 conn(0x7f46c8159ec0 msgr2=0x7f46c80af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:44.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.817+0000 7f46cce23700 1 --2- 172.21.15.99:0/3101930439 >> v2:172.21.15.191:3300/0 conn(0x7f46c8159ec0 0x7f46c80af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:44.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.817+0000 7f46cce23700 1 -- 172.21.15.99:0/3101930439 >> v1:172.21.15.191:6789/0 conn(0x7f46c815a290 legacy=0x7f46c806a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:44.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.817+0000 7f46cce23700 1 -- 172.21.15.99:0/3101930439 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c80b04b0 con 0x7f46c806acb0 2023-10-08T00:17:44.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.817+0000 7f46cce23700 1 -- 172.21.15.99:0/3101930439 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f46c00017f0 con 0x7f46c806acb0 2023-10-08T00:17:44.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.817+0000 7f46cce23700 1 -- 172.21.15.99:0/3101930439 >> v1:172.21.15.99:6789/0 conn(0x7f46c806acb0 legacy=0x7f46c80cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:44.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.817+0000 7f46cce23700 1 --2- 172.21.15.99:0/3101930439 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46b4003a60 0x7f46b4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:44.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.817+0000 7f46cce23700 1 --2- 172.21.15.99:0/3101930439 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46b4004810 0x7f46b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:44.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.817+0000 7f46cce23700 1 --2- 172.21.15.99:0/3101930439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46b40051c0 0x7f46b400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:44.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.817+0000 7f46cce23700 1 -- 172.21.15.99:0/3101930439 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c80b0b20 con 0x7f46b40051c0 2023-10-08T00:17:44.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.817+0000 7f46cce23700 1 -- 172.21.15.99:0/3101930439 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b400c160 con 0x7f46b4003a60 2023-10-08T00:17:44.821 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.817+0000 7f46cce23700 1 -- 172.21.15.99:0/3101930439 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46b400c320 con 0x7f46b4004810 2023-10-08T00:17:44.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.817+0000 7f46ce626700 1 --2- 172.21.15.99:0/3101930439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46b40051c0 0x7f46b400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:44.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.817+0000 7f46cde25700 1 --2- 172.21.15.99:0/3101930439 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46b4003a60 0x7f46b4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:44.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.817+0000 7f46cd624700 1 --2- 172.21.15.99:0/3101930439 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46b4004810 0x7f46b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:44.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.817+0000 7f46ce626700 1 -- 172.21.15.99:0/3101930439 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46b4004810 msgr2=0x7f46b4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:44.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.817+0000 7f46ce626700 1 --2- 172.21.15.99:0/3101930439 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46b4004810 0x7f46b4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:44.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.817+0000 7f46ce626700 1 -- 172.21.15.99:0/3101930439 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46b4003a60 msgr2=0x7f46b4003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:44.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.817+0000 7f46ce626700 1 --2- 172.21.15.99:0/3101930439 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46b4003a60 0x7f46b4003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:44.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.817+0000 7f46ce626700 1 -- 172.21.15.99:0/3101930439 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b4003570 con 0x7f46b40051c0 2023-10-08T00:17:44.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.817+0000 7f46ce626700 1 -- 172.21.15.99:0/3101930439 reap_dead start 2023-10-08T00:17:44.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.818+0000 7f46ce626700 1 --2- 172.21.15.99:0/3101930439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46b40051c0 0x7f46b400ba60 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f46c80b3770 tx=0x7f46c00021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=826c6d5b2f4f7ea1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:44.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.818+0000 7f46cce23700 1 -- 172.21.15.99:0/3101930439 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c00070d0 con 0x7f46b40051c0 2023-10-08T00:17:44.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.818+0000 7f46cce23700 1 -- 172.21.15.99:0/3101930439 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c0007230 con 0x7f46b40051c0 2023-10-08T00:17:44.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.818+0000 7f46cce23700 1 -- 172.21.15.99:0/3101930439 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c00075d0 con 0x7f46b40051c0 2023-10-08T00:17:44.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.818+0000 7f46d0089700 1 -- 172.21.15.99:0/3101930439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46b40051c0 msgr2=0x7f46b400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:44.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.818+0000 7f46d0089700 1 --2- 172.21.15.99:0/3101930439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46b40051c0 0x7f46b400ba60 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f46c80b3770 tx=0x7f46c00021d0 comp rx=0 tx=0).stop 2023-10-08T00:17:44.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.818+0000 7f46d0089700 1 -- 172.21.15.99:0/3101930439 shutdown_connections 2023-10-08T00:17:44.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.818+0000 7f46d0089700 1 --2- 172.21.15.99:0/3101930439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46b40051c0 0x7f46b400ba60 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:44.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.818+0000 7f46d0089700 1 -- 172.21.15.99:0/3101930439 >> 172.21.15.99:0/3101930439 conn(0x7f46c80d1880 msgr2=0x7f46c80cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:44.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.818+0000 7f46d0089700 1 -- 172.21.15.99:0/3101930439 shutdown_connections 2023-10-08T00:17:44.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.818+0000 7f46d0089700 1 -- 172.21.15.99:0/3101930439 wait complete. 2023-10-08T00:17:44.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.819+0000 7f46d0089700 1 Processor -- start 2023-10-08T00:17:44.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.819+0000 7f46d0089700 1 -- start start 2023-10-08T00:17:44.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.820+0000 7f46d0089700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46b40051c0 0x7f46c81535d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:44.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.820+0000 7f46d0089700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46c816a1c0 0x7f46c815c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:44.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.820+0000 7f46d0089700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46c80c7e60 0x7f46c80cf730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:44.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.820+0000 7f46d0089700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c8153b10 con 0x7f46c816a1c0 2023-10-08T00:17:44.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.820+0000 7f46d0089700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c816a730 con 0x7f46b40051c0 2023-10-08T00:17:44.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.820+0000 7f46d0089700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46c816aa10 con 0x7f46c80c7e60 2023-10-08T00:17:44.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.820+0000 7f46cd624700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46c816a1c0 0x7f46c815c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:44.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.820+0000 7f46cd624700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46c816a1c0 0x7f46c815c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53076/0 (socket says 172.21.15.99:53076) 2023-10-08T00:17:44.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.820+0000 7f46cd624700 1 -- 172.21.15.99:0/1642297943 learned_addr learned my addr 172.21.15.99:0/1642297943 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:44.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.820+0000 7f46cde25700 1 --2- 172.21.15.99:0/1642297943 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46b40051c0 0x7f46c81535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:44.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.820+0000 7f46ce626700 1 --2- 172.21.15.99:0/1642297943 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46c80c7e60 0x7f46c80cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:44.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.821+0000 7f46cd624700 1 -- 172.21.15.99:0/1642297943 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46c80c7e60 msgr2=0x7f46c80cf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:44.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.821+0000 7f46cd624700 1 --2- 172.21.15.99:0/1642297943 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46c80c7e60 0x7f46c80cf730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:44.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.821+0000 7f46cd624700 1 -- 172.21.15.99:0/1642297943 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46b40051c0 msgr2=0x7f46c81535d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:44.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.821+0000 7f46cd624700 1 --2- 172.21.15.99:0/1642297943 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46b40051c0 0x7f46c81535d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:44.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.821+0000 7f46cd624700 1 -- 172.21.15.99:0/1642297943 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c816acf0 con 0x7f46c816a1c0 2023-10-08T00:17:44.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.821+0000 7f46cd624700 1 --2- 172.21.15.99:0/1642297943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46c816a1c0 0x7f46c815c8c0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f46b8002950 tx=0x7f46b8007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:44.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.821+0000 7f46bdffb700 1 -- 172.21.15.99:0/1642297943 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b8003390 con 0x7f46c816a1c0 2023-10-08T00:17:44.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.821+0000 7f46bdffb700 1 -- 172.21.15.99:0/1642297943 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46b8006820 con 0x7f46c816a1c0 2023-10-08T00:17:44.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.821+0000 7f46d0089700 1 -- 172.21.15.99:0/1642297943 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46c816afd0 con 0x7f46c816a1c0 2023-10-08T00:17:44.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.821+0000 7f46bdffb700 1 -- 172.21.15.99:0/1642297943 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b8006bc0 con 0x7f46c816a1c0 2023-10-08T00:17:44.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.821+0000 7f46d0089700 1 -- 172.21.15.99:0/1642297943 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46c80c82d0 con 0x7f46c816a1c0 2023-10-08T00:17:44.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.822+0000 7f46d0089700 1 -- 172.21.15.99:0/1642297943 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46c816d780 con 0x7f46c816a1c0 2023-10-08T00:17:44.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.822+0000 7f46bdffb700 1 -- 172.21.15.99:0/1642297943 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f46b8007030 con 0x7f46c816a1c0 2023-10-08T00:17:44.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.823+0000 7f46bdffb700 1 --2- 172.21.15.99:0/1642297943 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f46ac066f30 0x7f46ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:44.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.823+0000 7f46bdffb700 1 -- 172.21.15.99:0/1642297943 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f46b80845c0 con 0x7f46c816a1c0 2023-10-08T00:17:44.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.823+0000 7f46cde25700 1 --2- 172.21.15.99:0/1642297943 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f46ac066f30 0x7f46ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:44.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.823+0000 7f46cde25700 1 --2- 172.21.15.99:0/1642297943 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f46ac066f30 0x7f46ac0693e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f46c815b800 tx=0x7f46c40091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:44.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:44.827+0000 7f46bdffb700 1 -- 172.21.15.99:0/1642297943 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f46b8055340 con 0x7f46c816a1c0 2023-10-08T00:17:45.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.082+0000 7f46d0089700 1 -- 172.21.15.99:0/1642297943 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f46c80c8550 con 0x7f46c816a1c0 2023-10-08T00:17:45.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.085+0000 7f46bdffb700 1 -- 172.21.15.99:0/1642297943 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+0 (secure 0 0 0) 0x7f46c80c8550 con 0x7f46c816a1c0 2023-10-08T00:17:45.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.088+0000 7f46d0089700 1 -- 172.21.15.99:0/1642297943 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f46ac066f30 msgr2=0x7f46ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:45.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.088+0000 7f46d0089700 1 --2- 172.21.15.99:0/1642297943 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f46ac066f30 0x7f46ac0693e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f46c815b800 tx=0x7f46c40091a0 comp rx=0 tx=0).stop 2023-10-08T00:17:45.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.088+0000 7f46d0089700 1 -- 172.21.15.99:0/1642297943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46c816a1c0 msgr2=0x7f46c815c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:45.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.088+0000 7f46d0089700 1 --2- 172.21.15.99:0/1642297943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46c816a1c0 0x7f46c815c8c0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f46b8002950 tx=0x7f46b8007d30 comp rx=0 tx=0).stop 2023-10-08T00:17:45.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.088+0000 7f46d0089700 1 -- 172.21.15.99:0/1642297943 shutdown_connections 2023-10-08T00:17:45.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.088+0000 7f46d0089700 1 --2- 172.21.15.99:0/1642297943 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f46ac066f30 0x7f46ac0693e0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:45.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.088+0000 7f46d0089700 1 --2- 172.21.15.99:0/1642297943 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46b40051c0 0x7f46c81535d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:45.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.088+0000 7f46d0089700 1 --2- 172.21.15.99:0/1642297943 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46c80c7e60 0x7f46c80cf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:45.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.088+0000 7f46d0089700 1 --2- 172.21.15.99:0/1642297943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46c816a1c0 0x7f46c815c8c0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:45.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.088+0000 7f46d0089700 1 -- 172.21.15.99:0/1642297943 >> 172.21.15.99:0/1642297943 conn(0x7f46c80d1880 msgr2=0x7f46c81541f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:45.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.088+0000 7f46d0089700 1 -- 172.21.15.99:0/1642297943 shutdown_connections 2023-10-08T00:17:45.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.088+0000 7f46d0089700 1 -- 172.21.15.99:0/1642297943 wait complete. 2023-10-08T00:17:45.090 INFO:teuthology.orchestra.run.smithi099.stderr:updated caps for client.0 2023-10-08T00:17:45.100 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:17:45.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.214+0000 7fd310d13700 1 Processor -- start 2023-10-08T00:17:45.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.214+0000 7fd310d13700 1 -- start start 2023-10-08T00:17:45.217 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.215+0000 7fd310d13700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd30c157870 0x7fd30c15b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:45.217 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.215+0000 7fd310d13700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd30c15a620 0x7fd30c15d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:45.217 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.215+0000 7fd310d13700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd30c15cbb0 con 0x7fd30c15d580 2023-10-08T00:17:45.217 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.215+0000 7fd310d13700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd30c15bfa0 con 0x7fd30c15a620 2023-10-08T00:17:45.218 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.215+0000 7fd310d13700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd30c15c280 con 0x7fd30c157870 2023-10-08T00:17:45.218 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.215+0000 7fd309d9b700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd30c15a620 0x7fd30c15d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:45.218 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.215+0000 7fd30ad9d700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fd30c15d580 0x7fd30c158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:60466/0 (socket says 172.21.15.99:60466) 2023-10-08T00:17:45.218 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.215+0000 7fd309d9b700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd30c15a620 0x7fd30c15d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53080/0 (socket says 172.21.15.99:53080) 2023-10-08T00:17:45.218 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.215+0000 7fd30ad9d700 1 -- 172.21.15.99:0/388667380 learned_addr learned my addr 172.21.15.99:0/388667380 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:17:45.218 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.215+0000 7fd30a59c700 1 --2- 172.21.15.99:0/388667380 >> v2:172.21.15.191:3300/0 conn(0x7fd30c157870 0x7fd30c15b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:45.218 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.215+0000 7fd309d9b700 1 -- 172.21.15.99:0/388667380 >> v2:172.21.15.191:3300/0 conn(0x7fd30c157870 msgr2=0x7fd30c15b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:45.218 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.215+0000 7fd309d9b700 1 --2- 172.21.15.99:0/388667380 >> v2:172.21.15.191:3300/0 conn(0x7fd30c157870 0x7fd30c15b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:45.218 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.215+0000 7fd309d9b700 1 -- 172.21.15.99:0/388667380 >> v1:172.21.15.99:6789/0 conn(0x7fd30c15d580 legacy=0x7fd30c158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:45.218 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.215+0000 7fd309d9b700 1 -- 172.21.15.99:0/388667380 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd30c15c560 con 0x7fd30c15a620 2023-10-08T00:17:45.218 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.215+0000 7fd30959a700 1 -- 172.21.15.99:0/388667380 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2503729357 0 0) 0x7fd30c15cbb0 con 0x7fd30c15d580 2023-10-08T00:17:45.219 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.215+0000 7fd30a59c700 1 --2- 172.21.15.99:0/388667380 >> v2:172.21.15.191:3300/0 conn(0x7fd30c157870 0x7fd30c15b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:17:45.219 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.215+0000 7fd309d9b700 1 --2- 172.21.15.99:0/388667380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd30c15a620 0x7fd30c15d040 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fd30c069d80 tx=0x7fd3000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=917c515eab954197 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:45.219 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.216+0000 7fd30959a700 1 -- 172.21.15.99:0/388667380 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30000b6b0 con 0x7fd30c15a620 2023-10-08T00:17:45.219 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.216+0000 7fd30959a700 1 -- 172.21.15.99:0/388667380 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd30000b810 con 0x7fd30c15a620 2023-10-08T00:17:45.219 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.216+0000 7fd30959a700 1 -- 172.21.15.99:0/388667380 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30000bbb0 con 0x7fd30c15a620 2023-10-08T00:17:45.219 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.216+0000 7fd310d13700 1 -- 172.21.15.99:0/388667380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd30c15a620 msgr2=0x7fd30c15d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:45.219 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.216+0000 7fd310d13700 1 --2- 172.21.15.99:0/388667380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd30c15a620 0x7fd30c15d040 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fd30c069d80 tx=0x7fd3000091a0 comp rx=0 tx=0).stop 2023-10-08T00:17:45.219 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.216+0000 7fd310d13700 1 -- 172.21.15.99:0/388667380 shutdown_connections 2023-10-08T00:17:45.219 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.216+0000 7fd310d13700 1 --2- 172.21.15.99:0/388667380 >> v2:172.21.15.191:3300/0 conn(0x7fd30c157870 0x7fd30c15b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:45.219 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.216+0000 7fd310d13700 1 --2- 172.21.15.99:0/388667380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd30c15a620 0x7fd30c15d040 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:45.219 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.216+0000 7fd310d13700 1 -- 172.21.15.99:0/388667380 >> 172.21.15.99:0/388667380 conn(0x7fd30c068ce0 msgr2=0x7fd30c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:45.219 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.216+0000 7fd310d13700 1 -- 172.21.15.99:0/388667380 shutdown_connections 2023-10-08T00:17:45.220 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.216+0000 7fd310d13700 1 -- 172.21.15.99:0/388667380 wait complete. 2023-10-08T00:17:45.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.217+0000 7fd310d13700 1 Processor -- start 2023-10-08T00:17:45.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.217+0000 7fd310d13700 1 -- start start 2023-10-08T00:17:45.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.218+0000 7fd310d13700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd30c157870 0x7fd30c14bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:45.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.218+0000 7fd310d13700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd30c15a620 0x7fd30c0b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:45.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.218+0000 7fd310d13700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd30c15d580 0x7fd30c0b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:45.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.218+0000 7fd310d13700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30c0b2140 con 0x7fd30c15a620 2023-10-08T00:17:45.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.218+0000 7fd310d13700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30c0b1cb0 con 0x7fd30c15d580 2023-10-08T00:17:45.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.218+0000 7fd310d13700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd30c14c1d0 con 0x7fd30c157870 2023-10-08T00:17:45.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.218+0000 7fd309d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd30c15a620 0x7fd30c0b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:45.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.218+0000 7fd309d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd30c15a620 0x7fd30c0b06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53094/0 (socket says 172.21.15.99:53094) 2023-10-08T00:17:45.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.218+0000 7fd309d9b700 1 -- 172.21.15.99:0/2964362318 learned_addr learned my addr 172.21.15.99:0/2964362318 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:45.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.218+0000 7fd30a59c700 1 --2- 172.21.15.99:0/2964362318 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd30c157870 0x7fd30c14bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:45.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.218+0000 7fd30ad9d700 1 --2- 172.21.15.99:0/2964362318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd30c15d580 0x7fd30c0b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:45.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.218+0000 7fd309d9b700 1 -- 172.21.15.99:0/2964362318 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd30c157870 msgr2=0x7fd30c14bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:45.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.218+0000 7fd309d9b700 1 --2- 172.21.15.99:0/2964362318 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd30c157870 0x7fd30c14bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:45.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.218+0000 7fd309d9b700 1 -- 172.21.15.99:0/2964362318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd30c15d580 msgr2=0x7fd30c0b29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:45.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.218+0000 7fd309d9b700 1 --2- 172.21.15.99:0/2964362318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd30c15d580 0x7fd30c0b29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:45.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.218+0000 7fd309d9b700 1 -- 172.21.15.99:0/2964362318 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd300009420 con 0x7fd30c15a620 2023-10-08T00:17:45.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.218+0000 7fd30ad9d700 1 --2- 172.21.15.99:0/2964362318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd30c15d580 0x7fd30c0b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:17:45.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.218+0000 7fd309d9b700 1 --2- 172.21.15.99:0/2964362318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd30c15a620 0x7fd30c0b06c0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fd30c06b530 tx=0x7fd300017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:45.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.219+0000 7fd2fa7fc700 1 -- 172.21.15.99:0/2964362318 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30000bb90 con 0x7fd30c15a620 2023-10-08T00:17:45.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.219+0000 7fd2fa7fc700 1 -- 172.21.15.99:0/2964362318 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd300011420 con 0x7fd30c15a620 2023-10-08T00:17:45.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.219+0000 7fd310d13700 1 -- 172.21.15.99:0/2964362318 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd30c0b10a0 con 0x7fd30c15a620 2023-10-08T00:17:45.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.219+0000 7fd2fa7fc700 1 -- 172.21.15.99:0/2964362318 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3000117a0 con 0x7fd30c15a620 2023-10-08T00:17:45.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.219+0000 7fd310d13700 1 -- 172.21.15.99:0/2964362318 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd30c0b1660 con 0x7fd30c15a620 2023-10-08T00:17:45.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.220+0000 7fd310d13700 1 -- 172.21.15.99:0/2964362318 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd30c0b1e40 con 0x7fd30c15a620 2023-10-08T00:17:45.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.220+0000 7fd2fa7fc700 1 -- 172.21.15.99:0/2964362318 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd300011b50 con 0x7fd30c15a620 2023-10-08T00:17:45.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.220+0000 7fd2fa7fc700 1 --2- 172.21.15.99:0/2964362318 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd2ec066f30 0x7fd2ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:45.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.221+0000 7fd2fa7fc700 1 -- 172.21.15.99:0/2964362318 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd300085eb0 con 0x7fd30c15a620 2023-10-08T00:17:45.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.221+0000 7fd30a59c700 1 --2- 172.21.15.99:0/2964362318 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd2ec066f30 0x7fd2ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:45.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.221+0000 7fd30a59c700 1 --2- 172.21.15.99:0/2964362318 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd2ec066f30 0x7fd2ec0693e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc00ad30 tx=0x7fd2fc006f90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:45.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.225+0000 7fd2fa7fc700 1 -- 172.21.15.99:0/2964362318 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd3000531a0 con 0x7fd30c15a620 2023-10-08T00:17:45.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.452+0000 7fd310d13700 1 -- 172.21.15.99:0/2964362318 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd30c04c3c0 con 0x7fd30c15a620 2023-10-08T00:17:45.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.452+0000 7fd2fa7fc700 1 -- 172.21.15.99:0/2964362318 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4599 (secure 0 0 0) 0x7fd3000567c0 con 0x7fd30c15a620 2023-10-08T00:17:45.453 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:17:45.454 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":70,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6056,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/3025449992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3025449992},{"type":"v1","addr":"172.21.15.191:6836","nonce":3025449992}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":7476,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/86350138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":86350138},{"type":"v1","addr":"172.21.15.99:6834","nonce":86350138}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":7479,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1282887036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1282887036},{"type":"v1","addr":"172.21.15.99:6835","nonce":1282887036}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:17:39.623734+0000","modified":"2023-10-08T00:17:43.252633+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7494":{"gid":7494,"name":"d","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/1165162515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":1165162515},{"type":"v1","addr":"172.21.15.191:6837","nonce":1165162515}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-08T00:17:45.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.455+0000 7fd310d13700 1 -- 172.21.15.99:0/2964362318 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd2ec066f30 msgr2=0x7fd2ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:45.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.455+0000 7fd310d13700 1 --2- 172.21.15.99:0/2964362318 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd2ec066f30 0x7fd2ec0693e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc00ad30 tx=0x7fd2fc006f90 comp rx=0 tx=0).stop 2023-10-08T00:17:45.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.455+0000 7fd310d13700 1 -- 172.21.15.99:0/2964362318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd30c15a620 msgr2=0x7fd30c0b06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:45.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.455+0000 7fd310d13700 1 --2- 172.21.15.99:0/2964362318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd30c15a620 0x7fd30c0b06c0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fd30c06b530 tx=0x7fd300017000 comp rx=0 tx=0).stop 2023-10-08T00:17:45.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.455+0000 7fd310d13700 1 -- 172.21.15.99:0/2964362318 shutdown_connections 2023-10-08T00:17:45.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.455+0000 7fd310d13700 1 --2- 172.21.15.99:0/2964362318 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd2ec066f30 0x7fd2ec0693e0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:45.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.455+0000 7fd310d13700 1 --2- 172.21.15.99:0/2964362318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd30c15d580 0x7fd30c0b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:45.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.455+0000 7fd310d13700 1 --2- 172.21.15.99:0/2964362318 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd30c157870 0x7fd30c14bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:45.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.455+0000 7fd310d13700 1 --2- 172.21.15.99:0/2964362318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd30c15a620 0x7fd30c0b06c0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:45.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.455+0000 7fd310d13700 1 -- 172.21.15.99:0/2964362318 >> 172.21.15.99:0/2964362318 conn(0x7fd30c068ce0 msgr2=0x7fd30c153be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:45.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.455+0000 7fd310d13700 1 -- 172.21.15.99:0/2964362318 shutdown_connections 2023-10-08T00:17:45.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.455+0000 7fd310d13700 1 -- 172.21.15.99:0/2964362318 wait complete. 2023-10-08T00:17:45.457 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 70 2023-10-08T00:17:45.532 INFO:tasks.ceph.mds.d.smithi191.stderr:2023-10-08T00:17:45.532+0000 7f4fedacc700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T00:17:45.870 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 70, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T00:17:39.623734+0000', 'modified': '2023-10-08T00:17:43.252633+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7494}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7494': {'gid': 7494, 'name': 'd', 'rank': 0, 'incarnation': 67, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.191:6837/1165162515', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6835', 'nonce': 1165162515}, {'type': 'v1', 'addr': '172.21.15.191:6837', 'nonce': 1165162515}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-08T00:17:45.870 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T00:17:45.870 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-08T00:17:45.986 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.986+0000 7fe534a22700 1 Processor -- start 2023-10-08T00:17:45.987 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.986+0000 7fe534a22700 1 -- start start 2023-10-08T00:17:45.993 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.987+0000 7fe534a22700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe530159e10 0x7fe5300b2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:45.993 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.987+0000 7fe534a22700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe53015db80 con 0x7fe53015a1e0 2023-10-08T00:17:45.993 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.987+0000 7fe534a22700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe53015de60 con 0x7fe5300cde50 2023-10-08T00:17:45.993 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.987+0000 7fe534a22700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe53015e800 con 0x7fe530159e10 2023-10-08T00:17:45.993 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.987+0000 7fe52dd9b700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fe53015a1e0 0x7fe5300cd740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:60480/0 (socket says 172.21.15.99:60480) 2023-10-08T00:17:45.993 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.987+0000 7fe52dd9b700 1 -- 172.21.15.99:0/795711904 learned_addr learned my addr 172.21.15.99:0/795711904 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:17:45.994 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.987+0000 7fe52e59c700 1 --2- 172.21.15.99:0/795711904 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe530159e10 0x7fe5300b2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:45.994 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.987+0000 7fe52d59a700 1 -- 172.21.15.99:0/795711904 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 471035241 0 0) 0x7fe53015db80 con 0x7fe53015a1e0 2023-10-08T00:17:45.994 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.987+0000 7fe52d59a700 1 -- 172.21.15.99:0/795711904 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe51c003570 con 0x7fe53015a1e0 2023-10-08T00:17:45.994 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.987+0000 7fe52d59a700 1 -- 172.21.15.99:0/795711904 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4262819725 0 0) 0x7fe53015de60 con 0x7fe5300cde50 2023-10-08T00:17:45.994 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.987+0000 7fe52d59a700 1 -- 172.21.15.99:0/795711904 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe53015db80 con 0x7fe5300cde50 2023-10-08T00:17:45.994 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.987+0000 7fe52d59a700 1 -- 172.21.15.99:0/795711904 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1584626000 0 0) 0x7fe51c003570 con 0x7fe53015a1e0 2023-10-08T00:17:45.994 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.987+0000 7fe52d59a700 1 -- 172.21.15.99:0/795711904 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe530159e10 msgr2=0x7fe5300b2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:45.994 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.987+0000 7fe52d59a700 1 --2- 172.21.15.99:0/795711904 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe530159e10 0x7fe5300b2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:45.994 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.987+0000 7fe52d59a700 1 -- 172.21.15.99:0/795711904 >> v1:172.21.15.191:6789/0 conn(0x7fe5300cde50 legacy=0x7fe5300cbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:45.994 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.987+0000 7fe52d59a700 1 -- 172.21.15.99:0/795711904 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe53015e140 con 0x7fe53015a1e0 2023-10-08T00:17:45.995 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.987+0000 7fe52d59a700 1 -- 172.21.15.99:0/795711904 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7fe518001490 con 0x7fe53015a1e0 2023-10-08T00:17:45.995 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.987+0000 7fe52e59c700 1 --2- 172.21.15.99:0/795711904 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe530159e10 0x7fe5300b2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:17:45.995 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.988+0000 7fe52d59a700 1 -- 172.21.15.99:0/795711904 >> v1:172.21.15.99:6789/0 conn(0x7fe53015a1e0 legacy=0x7fe5300cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:45.995 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.988+0000 7fe52d59a700 1 --2- 172.21.15.99:0/795711904 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe51c003c20 0x7fe51c004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:45.995 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.988+0000 7fe52d59a700 1 --2- 172.21.15.99:0/795711904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe51c004810 0x7fe51c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:45.995 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.988+0000 7fe52d59a700 1 --2- 172.21.15.99:0/795711904 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe530159e10 0x7fe51c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:45.995 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.988+0000 7fe52d59a700 1 -- 172.21.15.99:0/795711904 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53015e800 con 0x7fe51c004810 2023-10-08T00:17:45.995 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.988+0000 7fe52d59a700 1 -- 172.21.15.99:0/795711904 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51c00bb10 con 0x7fe51c003c20 2023-10-08T00:17:45.995 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.988+0000 7fe52d59a700 1 -- 172.21.15.99:0/795711904 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe51c00bcd0 con 0x7fe530159e10 2023-10-08T00:17:45.995 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.988+0000 7fe52dd9b700 1 --2- 172.21.15.99:0/795711904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe51c004810 0x7fe51c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:45.996 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.988+0000 7fe52e59c700 1 --2- 172.21.15.99:0/795711904 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe51c003c20 0x7fe51c004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:45.996 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.988+0000 7fe52ed9d700 1 --2- 172.21.15.99:0/795711904 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe530159e10 0x7fe51c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:45.996 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.988+0000 7fe52dd9b700 1 -- 172.21.15.99:0/795711904 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe530159e10 msgr2=0x7fe51c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:45.996 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.988+0000 7fe52dd9b700 1 --2- 172.21.15.99:0/795711904 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe530159e10 0x7fe51c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:45.996 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.988+0000 7fe52dd9b700 1 -- 172.21.15.99:0/795711904 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe51c003c20 msgr2=0x7fe51c004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:45.996 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.988+0000 7fe52dd9b700 1 --2- 172.21.15.99:0/795711904 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe51c003c20 0x7fe51c004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:45.996 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.988+0000 7fe52dd9b700 1 -- 172.21.15.99:0/795711904 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe528009a20 con 0x7fe51c004810 2023-10-08T00:17:45.996 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.988+0000 7fe52ed9d700 1 --2- 172.21.15.99:0/795711904 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe530159e10 0x7fe51c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:17:45.996 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.988+0000 7fe52dd9b700 1 --2- 172.21.15.99:0/795711904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe51c004810 0x7fe51c004c80 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fe51800c2a0 tx=0x7fe518001e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9d5733fd1c8abd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:45.996 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.988+0000 7fe52d59a700 1 -- 172.21.15.99:0/795711904 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51800cbd0 con 0x7fe51c004810 2023-10-08T00:17:45.996 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.988+0000 7fe52d59a700 1 -- 172.21.15.99:0/795711904 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe51800cd30 con 0x7fe51c004810 2023-10-08T00:17:45.997 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.988+0000 7fe52d59a700 1 -- 172.21.15.99:0/795711904 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51800cea0 con 0x7fe51c004810 2023-10-08T00:17:45.997 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.989+0000 7fe534a22700 1 -- 172.21.15.99:0/795711904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe51c004810 msgr2=0x7fe51c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:45.997 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.989+0000 7fe534a22700 1 --2- 172.21.15.99:0/795711904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe51c004810 0x7fe51c004c80 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fe51800c2a0 tx=0x7fe518001e30 comp rx=0 tx=0).stop 2023-10-08T00:17:45.997 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.989+0000 7fe52ed9d700 1 -- 172.21.15.99:0/795711904 reap_dead start 2023-10-08T00:17:45.997 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.989+0000 7fe534a22700 1 -- 172.21.15.99:0/795711904 shutdown_connections 2023-10-08T00:17:45.997 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.989+0000 7fe534a22700 1 -- 172.21.15.99:0/795711904 >> 172.21.15.99:0/795711904 conn(0x7fe530067660 msgr2=0x7fe530067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:45.997 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.989+0000 7fe534a22700 1 -- 172.21.15.99:0/795711904 shutdown_connections 2023-10-08T00:17:45.997 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.989+0000 7fe534a22700 1 -- 172.21.15.99:0/795711904 wait complete. 2023-10-08T00:17:45.997 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.990+0000 7fe534a22700 1 Processor -- start 2023-10-08T00:17:45.997 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.990+0000 7fe534a22700 1 -- start start 2023-10-08T00:17:45.997 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.991+0000 7fe534a22700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe51c004810 0x7fe5300c7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:45.998 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.991+0000 7fe534a22700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5300c4760 0x7fe53016b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:45.998 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.991+0000 7fe534a22700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5300d0e20 0x7fe5300c9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:45.998 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.991+0000 7fe534a22700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53016b6b0 con 0x7fe5300c4760 2023-10-08T00:17:45.998 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.991+0000 7fe534a22700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5300c6480 con 0x7fe5300d0e20 2023-10-08T00:17:45.998 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.991+0000 7fe534a22700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5300c6760 con 0x7fe51c004810 2023-10-08T00:17:45.998 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.991+0000 7fe52dd9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5300c4760 0x7fe53016b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:45.998 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.991+0000 7fe52dd9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5300c4760 0x7fe53016b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53112/0 (socket says 172.21.15.99:53112) 2023-10-08T00:17:45.998 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.991+0000 7fe52dd9b700 1 -- 172.21.15.99:0/2731074131 learned_addr learned my addr 172.21.15.99:0/2731074131 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:45.998 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.991+0000 7fe52e59c700 1 --2- 172.21.15.99:0/2731074131 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe51c004810 0x7fe5300c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:45.998 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.991+0000 7fe52ed9d700 1 --2- 172.21.15.99:0/2731074131 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5300d0e20 0x7fe5300c9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:45.999 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.991+0000 7fe52dd9b700 1 -- 172.21.15.99:0/2731074131 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe51c004810 msgr2=0x7fe5300c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:45.999 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.991+0000 7fe52dd9b700 1 --2- 172.21.15.99:0/2731074131 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe51c004810 0x7fe5300c7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:45.999 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.991+0000 7fe52dd9b700 1 -- 172.21.15.99:0/2731074131 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5300d0e20 msgr2=0x7fe5300c9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:45.999 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.991+0000 7fe52dd9b700 1 --2- 172.21.15.99:0/2731074131 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5300d0e20 0x7fe5300c9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:45.999 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.991+0000 7fe52dd9b700 1 -- 172.21.15.99:0/2731074131 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5300c6a40 con 0x7fe5300c4760 2023-10-08T00:17:45.999 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.991+0000 7fe52dd9b700 1 --2- 172.21.15.99:0/2731074131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5300c4760 0x7fe53016b170 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fe51800c890 tx=0x7fe518015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:45.999 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.991+0000 7fe5267fc700 1 -- 172.21.15.99:0/2731074131 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe518000cb0 con 0x7fe5300c4760 2023-10-08T00:17:45.999 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.991+0000 7fe5267fc700 1 -- 172.21.15.99:0/2731074131 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe518004d50 con 0x7fe5300c4760 2023-10-08T00:17:45.999 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.991+0000 7fe534a22700 1 -- 172.21.15.99:0/2731074131 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5300c6d20 con 0x7fe5300c4760 2023-10-08T00:17:45.999 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.991+0000 7fe5267fc700 1 -- 172.21.15.99:0/2731074131 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe518000ed0 con 0x7fe5300c4760 2023-10-08T00:17:46.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.991+0000 7fe534a22700 1 -- 172.21.15.99:0/2731074131 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5300c6150 con 0x7fe5300c4760 2023-10-08T00:17:46.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.993+0000 7fe5267fc700 1 -- 172.21.15.99:0/2731074131 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe518014630 con 0x7fe5300c4760 2023-10-08T00:17:46.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.993+0000 7fe5267fc700 1 --2- 172.21.15.99:0/2731074131 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe510066f60 0x7fe510069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:46.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.993+0000 7fe5267fc700 1 -- 172.21.15.99:0/2731074131 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe518085f60 con 0x7fe5300c4760 2023-10-08T00:17:46.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.993+0000 7fe52e59c700 1 --2- 172.21.15.99:0/2731074131 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe510066f60 0x7fe510069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:46.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.994+0000 7fe52e59c700 1 --2- 172.21.15.99:0/2731074131 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe510066f60 0x7fe510069410 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fe5280094d0 tx=0x7fe52800b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:46.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.995+0000 7fe53dd1ab80 1 Processor -- start 2023-10-08T00:17:46.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.995+0000 7fe53dd1ab80 1 -- start start 2023-10-08T00:17:46.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.996+0000 7fe53dd1ab80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561762433a50 0x561762433e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:46.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.996+0000 7fe53dd1ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56176247dd00 0x56176245ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:46.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.996+0000 7fe53dd1ab80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56176245cf60 0x56176242fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:46.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.996+0000 7fe53dd1ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56176238aa70 con 0x56176247dd00 2023-10-08T00:17:46.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.996+0000 7fe53dd1ab80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x561762388270 con 0x56176245cf60 2023-10-08T00:17:46.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.996+0000 7fe53dd1ab80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x561762384060 con 0x561762433a50 2023-10-08T00:17:46.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.996+0000 7fe52dd9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56176247dd00 0x56176245ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:46.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.996+0000 7fe52dd9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56176247dd00 0x56176245ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53126/0 (socket says 172.21.15.99:53126) 2023-10-08T00:17:46.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.996+0000 7fe52dd9b700 1 -- 172.21.15.99:0/564340369 learned_addr learned my addr 172.21.15.99:0/564340369 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:46.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.996+0000 7fe52e59c700 1 --2- 172.21.15.99:0/564340369 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561762433a50 0x561762433e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:46.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.996+0000 7fe52ed9d700 1 --2- 172.21.15.99:0/564340369 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56176245cf60 0x56176242fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:46.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.996+0000 7fe52dd9b700 1 -- 172.21.15.99:0/564340369 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561762433a50 msgr2=0x561762433e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:46.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.996+0000 7fe52dd9b700 1 --2- 172.21.15.99:0/564340369 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561762433a50 0x561762433e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:46.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.996+0000 7fe52dd9b700 1 -- 172.21.15.99:0/564340369 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56176245cf60 msgr2=0x56176242fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:46.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.996+0000 7fe52dd9b700 1 --2- 172.21.15.99:0/564340369 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56176245cf60 0x56176242fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:46.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.996+0000 7fe52dd9b700 1 -- 172.21.15.99:0/564340369 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56176221e270 con 0x56176247dd00 2023-10-08T00:17:46.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.997+0000 7fe52dd9b700 1 --2- 172.21.15.99:0/564340369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56176247dd00 0x56176245ca20 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fe518084480 tx=0x7fe5180843e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b91032e248bb67fe server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:46.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.997+0000 7fe50f7fe700 1 -- 172.21.15.99:0/564340369 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe518057ea0 con 0x56176247dd00 2023-10-08T00:17:46.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.997+0000 7fe50f7fe700 1 -- 172.21.15.99:0/564340369 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe518056e90 con 0x56176247dd00 2023-10-08T00:17:46.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.997+0000 7fe50f7fe700 1 -- 172.21.15.99:0/564340369 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51805a0e0 con 0x56176247dd00 2023-10-08T00:17:46.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.997+0000 7fe53dd1ab80 1 -- 172.21.15.99:0/564340369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56176247dd00 msgr2=0x56176245ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:46.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.997+0000 7fe53dd1ab80 1 --2- 172.21.15.99:0/564340369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56176247dd00 0x56176245ca20 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fe518084480 tx=0x7fe5180843e0 comp rx=0 tx=0).stop 2023-10-08T00:17:46.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.997+0000 7fe53dd1ab80 1 -- 172.21.15.99:0/564340369 shutdown_connections 2023-10-08T00:17:46.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.997+0000 7fe53dd1ab80 1 --2- 172.21.15.99:0/564340369 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56176245cf60 0x56176242fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:46.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.997+0000 7fe53dd1ab80 1 --2- 172.21.15.99:0/564340369 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561762433a50 0x561762433e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:46.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.997+0000 7fe53dd1ab80 1 --2- 172.21.15.99:0/564340369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56176247dd00 0x56176245ca20 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:46.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.997+0000 7fe53dd1ab80 1 -- 172.21.15.99:0/564340369 >> 172.21.15.99:0/564340369 conn(0x561762435370 msgr2=0x561762435740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:46.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.997+0000 7fe53dd1ab80 1 -- 172.21.15.99:0/564340369 shutdown_connections 2023-10-08T00:17:46.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.997+0000 7fe53dd1ab80 1 -- 172.21.15.99:0/564340369 wait complete. 2023-10-08T00:17:46.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.998+0000 7fe53dd1ab80 1 -- start start 2023-10-08T00:17:46.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.998+0000 7fe53dd1ab80 1 Processor -- start 2023-10-08T00:17:46.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.999+0000 7fe53dd1ab80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561762433a50 0x56176247a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:46.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.999+0000 7fe53dd1ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56176245cf60 0x561762473e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:46.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.999+0000 7fe53dd1ab80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x56176247dd00 0x5617624743b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:46.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.999+0000 7fe53dd1ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56176232f890 con 0x56176245cf60 2023-10-08T00:17:46.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.999+0000 7fe53dd1ab80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x56176234aea0 con 0x561762433a50 2023-10-08T00:17:46.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.999+0000 7fe53dd1ab80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x561762206e30 con 0x56176247dd00 2023-10-08T00:17:46.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.999+0000 7fe53dd1ab80 10 client.0 _ll_register_callbacks cb 0x5617623f7110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T00:17:46.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.999+0000 7fe52dd9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56176245cf60 0x561762473e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:46.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.999+0000 7fe52dd9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56176245cf60 0x561762473e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53142/0 (socket says 172.21.15.99:53142) 2023-10-08T00:17:46.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.999+0000 7fe52dd9b700 1 -- 172.21.15.99:0/1744351594 learned_addr learned my addr 172.21.15.99:0/1744351594 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:46.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.999+0000 7fe52e59c700 1 --2- 172.21.15.99:0/1744351594 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561762433a50 0x56176247a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:46.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.999+0000 7fe52ed9d700 1 --2- 172.21.15.99:0/1744351594 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x56176247dd00 0x5617624743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:46.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.999+0000 7fe52dd9b700 1 -- 172.21.15.99:0/1744351594 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x56176247dd00 msgr2=0x5617624743b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:46.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.999+0000 7fe52dd9b700 1 --2- 172.21.15.99:0/1744351594 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x56176247dd00 0x5617624743b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:46.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.999+0000 7fe52dd9b700 1 -- 172.21.15.99:0/1744351594 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561762433a50 msgr2=0x56176247a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:46.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.999+0000 7fe52dd9b700 1 --2- 172.21.15.99:0/1744351594 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561762433a50 0x56176247a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:46.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:45.999+0000 7fe52dd9b700 1 -- 172.21.15.99:0/1744351594 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5300c6150 con 0x56176245cf60 2023-10-08T00:17:46.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.000+0000 7fe52dd9b700 1 --2- 172.21.15.99:0/1744351594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56176245cf60 0x561762473e70 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fe51800cd00 tx=0x7fe518021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efec814260862cf6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:46.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.000+0000 7fe50cff9700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T00:17:46.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.000+0000 7fe50cff9700 1 -- 172.21.15.99:0/1744351594 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe518057ea0 con 0x56176245cf60 2023-10-08T00:17:46.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.000+0000 7fe50cff9700 1 -- 172.21.15.99:0/1744351594 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(154..154 src has 1..154) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe51805de30 con 0x56176245cf60 2023-10-08T00:17:46.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.000+0000 7fe53dd1ab80 1 -- 172.21.15.99:0/1744351594 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561762428b70 con 0x56176245cf60 2023-10-08T00:17:46.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.000+0000 7fe50cff9700 1 -- 172.21.15.99:0/1744351594 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe51805a0e0 con 0x56176245cf60 2023-10-08T00:17:46.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.000+0000 7fe53dd1ab80 10 client.7620 fetch_fsmap learned FSMap version 70 2023-10-08T00:17:46.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.000+0000 7fe53dd1ab80 1 -- 172.21.15.99:0/1744351594 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) v3 -- 0x5617621c0fe0 con 0x56176245cf60 2023-10-08T00:17:46.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.000+0000 7fe50cff9700 1 -- 172.21.15.99:0/1744351594 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561762428b70 con 0x56176245cf60 2023-10-08T00:17:46.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.000+0000 7fe50cff9700 1 -- 172.21.15.99:0/1744351594 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 70) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fe51805c070 con 0x56176245cf60 2023-10-08T00:17:46.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.000+0000 7fe53dd1ab80 10 client.7620 fetch_fsmap finished waiting for FSMap version 70 2023-10-08T00:17:46.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.000+0000 7fe53dd1ab80 10 client.7620 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-08T00:17:46.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.000+0000 7fe53dd1ab80 20 client.7620 populate_metadata read hostname 'smithi099' 2023-10-08T00:17:46.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.000+0000 7fe53dd1ab80 1 --2- 172.21.15.99:0/1744351594 >> [v2:172.21.15.191:6835/1165162515,v1:172.21.15.191:6837/1165162515] conn(0x56176248a3f0 0x56176248a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:46.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.000+0000 7fe53dd1ab80 4 client.7620 mds_command: new command op to 7494 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-08T00:17:46.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.000+0000 7fe53dd1ab80 1 -- 172.21.15.99:0/1744351594 --> [v2:172.21.15.191:6835/1165162515,v1:172.21.15.191:6837/1165162515] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561762405aa0 con 0x56176248a3f0 2023-10-08T00:17:46.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.000+0000 7fe52e59c700 1 --2- 172.21.15.99:0/1744351594 >> [v2:172.21.15.191:6835/1165162515,v1:172.21.15.191:6837/1165162515] conn(0x56176248a3f0 0x56176248a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:46.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.001+0000 7fe50cff9700 10 client.7620 ms_handle_connect on v2:172.21.15.191:6835/1165162515 2023-10-08T00:17:46.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.001+0000 7fe52e59c700 1 --2- 172.21.15.99:0/1744351594 >> [v2:172.21.15.191:6835/1165162515,v1:172.21.15.191:6837/1165162515] conn(0x56176248a3f0 0x56176248a7c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f730a42ad815451f server_cookie=eb1dd5493ac7c568 in_seq=0 out_seq=0 2023-10-08T00:17:46.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.001+0000 7fe50cff9700 1 -- 172.21.15.99:0/1744351594 <== mds.0 v2:172.21.15.191:6835/1165162515 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x561762405aa0 con 0x56176248a3f0 2023-10-08T00:17:46.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.001+0000 7fe50cff9700 10 client.7620 handle_command_reply: tid=0 2023-10-08T00:17:46.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.001+0000 7fe53dd1ab80 1 client.7620 shutdown 2023-10-08T00:17:46.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.001+0000 7fe53dd1ab80 1 -- 172.21.15.99:0/1744351594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56176245cf60 msgr2=0x561762473e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:46.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.001+0000 7fe53dd1ab80 1 --2- 172.21.15.99:0/1744351594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56176245cf60 0x561762473e70 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fe51800cd00 tx=0x7fe518021000 comp rx=0 tx=0).stop 2023-10-08T00:17:46.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.001+0000 7fe53dd1ab80 1 -- 172.21.15.99:0/1744351594 shutdown_connections 2023-10-08T00:17:46.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.001+0000 7fe53dd1ab80 1 --2- 172.21.15.99:0/1744351594 >> [v2:172.21.15.191:6835/1165162515,v1:172.21.15.191:6837/1165162515] conn(0x56176248a3f0 0x56176248a7c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:46.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.001+0000 7fe53dd1ab80 1 --2- 172.21.15.99:0/1744351594 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561762433a50 0x56176247a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:46.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.001+0000 7fe53dd1ab80 1 --2- 172.21.15.99:0/1744351594 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x56176247dd00 0x5617624743b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:46.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.001+0000 7fe50cff9700 0 client.7620 ms_handle_reset on v2:172.21.15.191:6835/1165162515 2023-10-08T00:17:46.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.001+0000 7fe53dd1ab80 1 --2- 172.21.15.99:0/1744351594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56176245cf60 0x561762473e70 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:46.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.001+0000 7fe53dd1ab80 1 -- 172.21.15.99:0/1744351594 >> 172.21.15.99:0/1744351594 conn(0x561762435370 msgr2=0x561762479520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:46.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.002+0000 7fe53dd1ab80 1 -- 172.21.15.99:0/1744351594 shutdown_connections 2023-10-08T00:17:46.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.002+0000 7fe53dd1ab80 1 -- 172.21.15.99:0/1744351594 wait complete. 2023-10-08T00:17:46.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.002+0000 7fe53dd1ab80 20 client.7620 trim_cache size 0 max 16384 2023-10-08T00:17:46.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.018+0000 7fe53dd1ab80 1 Processor -- start 2023-10-08T00:17:46.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.018+0000 7fe53dd1ab80 1 -- start start 2023-10-08T00:17:46.021 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.019+0000 7fe53dd1ab80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561762433a50 0x5617624639a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:46.021 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.019+0000 7fe53dd1ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56176245cf60 0x56176242edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:46.021 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.019+0000 7fe53dd1ab80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56176247dd00 0x56176248abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:46.021 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.019+0000 7fe53dd1ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561762431270 con 0x56176245cf60 2023-10-08T00:17:46.021 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.019+0000 7fe53dd1ab80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x561762434670 con 0x56176247dd00 2023-10-08T00:17:46.022 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.019+0000 7fe53dd1ab80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x561762433370 con 0x561762433a50 2023-10-08T00:17:46.022 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.019+0000 7fe52dd9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56176245cf60 0x56176242edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:46.022 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.019+0000 7fe52dd9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56176245cf60 0x56176242edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53146/0 (socket says 172.21.15.99:53146) 2023-10-08T00:17:46.022 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.019+0000 7fe52dd9b700 1 -- 172.21.15.99:0/1491597356 learned_addr learned my addr 172.21.15.99:0/1491597356 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:46.022 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.019+0000 7fe52dd9b700 1 -- 172.21.15.99:0/1491597356 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561762433a50 msgr2=0x5617624639a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:46.022 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.019+0000 7fe52dd9b700 1 --2- 172.21.15.99:0/1491597356 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561762433a50 0x5617624639a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:46.022 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.019+0000 7fe52dd9b700 1 -- 172.21.15.99:0/1491597356 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56176247dd00 msgr2=0x56176248abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:46.022 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.019+0000 7fe52dd9b700 1 --2- 172.21.15.99:0/1491597356 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56176247dd00 0x56176248abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:46.022 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.019+0000 7fe52dd9b700 1 -- 172.21.15.99:0/1491597356 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5300cf860 con 0x56176245cf60 2023-10-08T00:17:46.022 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.019+0000 7fe52dd9b700 1 --2- 172.21.15.99:0/1491597356 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56176245cf60 0x56176242edb0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fe518000e50 tx=0x7fe51806f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3314e5b2759574cb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:46.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.019+0000 7fe50ffff700 1 -- 172.21.15.99:0/1491597356 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe518057ea0 con 0x56176245cf60 2023-10-08T00:17:46.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.019+0000 7fe50ffff700 1 -- 172.21.15.99:0/1491597356 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe51805de30 con 0x56176245cf60 2023-10-08T00:17:46.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.019+0000 7fe50ffff700 1 -- 172.21.15.99:0/1491597356 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe518057330 con 0x56176245cf60 2023-10-08T00:17:46.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.020+0000 7fe53dd1ab80 1 -- 172.21.15.99:0/1491597356 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56176245cf60 msgr2=0x56176242edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:46.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.020+0000 7fe53dd1ab80 1 --2- 172.21.15.99:0/1491597356 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56176245cf60 0x56176242edb0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fe518000e50 tx=0x7fe51806f040 comp rx=0 tx=0).stop 2023-10-08T00:17:46.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.020+0000 7fe53dd1ab80 1 -- 172.21.15.99:0/1491597356 shutdown_connections 2023-10-08T00:17:46.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.020+0000 7fe53dd1ab80 1 --2- 172.21.15.99:0/1491597356 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56176247dd00 0x56176248abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:46.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.020+0000 7fe53dd1ab80 1 --2- 172.21.15.99:0/1491597356 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561762433a50 0x5617624639a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:46.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.020+0000 7fe53dd1ab80 1 --2- 172.21.15.99:0/1491597356 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56176245cf60 0x56176242edb0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:46.024 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.020+0000 7fe53dd1ab80 1 -- 172.21.15.99:0/1491597356 >> 172.21.15.99:0/1491597356 conn(0x561762435370 msgr2=0x561762433e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:46.024 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.020+0000 7fe53dd1ab80 1 -- 172.21.15.99:0/1491597356 shutdown_connections 2023-10-08T00:17:46.024 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.020+0000 7fe53dd1ab80 1 -- 172.21.15.99:0/1491597356 wait complete. 2023-10-08T00:17:46.024 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.020+0000 7fe53dd1ab80 1 -- start start 2023-10-08T00:17:46.024 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.021+0000 7fe53dd1ab80 1 Processor -- start 2023-10-08T00:17:46.028 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.021+0000 7fe53dd1ab80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561762433a50 0x561762488bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:46.028 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.021+0000 7fe53dd1ab80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56176245cf60 0x5617624df7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:46.028 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.021+0000 7fe53dd1ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56176247dd00 0x5617624dfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:46.028 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.021+0000 7fe53dd1ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617624e05e0 con 0x56176247dd00 2023-10-08T00:17:46.028 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.021+0000 7fe53dd1ab80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x561762488730 con 0x56176245cf60 2023-10-08T00:17:46.028 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.021+0000 7fe53dd1ab80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x561762433e20 con 0x561762433a50 2023-10-08T00:17:46.028 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.021+0000 7fe53dd1ab80 10 client.0 _ll_register_callbacks cb 0x5617623f7110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T00:17:46.028 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.022+0000 7fe52ed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56176247dd00 0x5617624dfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:46.029 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.022+0000 7fe52dd9b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56176245cf60 0x5617624df7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:46.029 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.022+0000 7fe52ed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56176247dd00 0x5617624dfd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53156/0 (socket says 172.21.15.99:53156) 2023-10-08T00:17:46.029 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.022+0000 7fe52ed9d700 1 -- 172.21.15.99:0/2693865847 learned_addr learned my addr 172.21.15.99:0/2693865847 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:46.029 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.022+0000 7fe52e59c700 1 --2- 172.21.15.99:0/2693865847 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561762433a50 0x561762488bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:46.029 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.022+0000 7fe52ed9d700 1 -- 172.21.15.99:0/2693865847 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561762433a50 msgr2=0x561762488bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:46.029 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.022+0000 7fe52ed9d700 1 --2- 172.21.15.99:0/2693865847 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561762433a50 0x561762488bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:46.029 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.022+0000 7fe52ed9d700 1 -- 172.21.15.99:0/2693865847 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56176245cf60 msgr2=0x5617624df7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:46.029 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.022+0000 7fe52ed9d700 1 --2- 172.21.15.99:0/2693865847 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56176245cf60 0x5617624df7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:46.029 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.022+0000 7fe52ed9d700 1 -- 172.21.15.99:0/2693865847 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56176221e270 con 0x56176247dd00 2023-10-08T00:17:46.029 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.022+0000 7fe52ed9d700 1 --2- 172.21.15.99:0/2693865847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56176247dd00 0x5617624dfd20 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fe5300ce250 tx=0x7fe520009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff1ca0a3647df194 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:46.030 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.022+0000 7fe50cff9700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T00:17:46.030 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.022+0000 7fe50cff9700 1 -- 172.21.15.99:0/2693865847 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe520016ee0 con 0x56176247dd00 2023-10-08T00:17:46.030 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.022+0000 7fe50cff9700 1 -- 172.21.15.99:0/2693865847 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(154..154 src has 1..154) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe52001c020 con 0x56176247dd00 2023-10-08T00:17:46.030 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.022+0000 7fe53dd1ab80 1 -- 172.21.15.99:0/2693865847 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe518056e90 con 0x56176247dd00 2023-10-08T00:17:46.030 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.022+0000 7fe50cff9700 1 -- 172.21.15.99:0/2693865847 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe520006820 con 0x56176247dd00 2023-10-08T00:17:46.030 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.022+0000 7fe53dd1ab80 10 client.7626 fetch_fsmap learned FSMap version 70 2023-10-08T00:17:46.030 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.022+0000 7fe53dd1ab80 1 -- 172.21.15.99:0/2693865847 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) v3 -- 0x561762352780 con 0x56176247dd00 2023-10-08T00:17:46.030 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.022+0000 7fe50cff9700 1 -- 172.21.15.99:0/2693865847 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe518056e90 con 0x56176247dd00 2023-10-08T00:17:46.030 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.022+0000 7fe50cff9700 1 -- 172.21.15.99:0/2693865847 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 70) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fe520006980 con 0x56176247dd00 2023-10-08T00:17:46.031 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.022+0000 7fe53dd1ab80 10 client.7626 fetch_fsmap finished waiting for FSMap version 70 2023-10-08T00:17:46.031 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.022+0000 7fe53dd1ab80 10 client.7626 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-08T00:17:46.031 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.022+0000 7fe53dd1ab80 20 client.7626 populate_metadata read hostname 'smithi099' 2023-10-08T00:17:46.031 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.022+0000 7fe53dd1ab80 1 --2- 172.21.15.99:0/2693865847 >> [v2:172.21.15.191:6835/1165162515,v1:172.21.15.191:6837/1165162515] conn(0x56176247d460 0x5617624771e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:46.031 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.022+0000 7fe53dd1ab80 4 client.7626 mds_command: new command op to 7494 tid=0[{"prefix": "status"}] 2023-10-08T00:17:46.031 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.022+0000 7fe53dd1ab80 1 -- 172.21.15.99:0/2693865847 --> [v2:172.21.15.191:6835/1165162515,v1:172.21.15.191:6837/1165162515] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561762326e00 con 0x56176247d460 2023-10-08T00:17:46.031 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.023+0000 7fe52e59c700 1 --2- 172.21.15.99:0/2693865847 >> [v2:172.21.15.191:6835/1165162515,v1:172.21.15.191:6837/1165162515] conn(0x56176247d460 0x5617624771e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:46.031 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.023+0000 7fe50cff9700 10 client.7626 ms_handle_connect on v2:172.21.15.191:6835/1165162515 2023-10-08T00:17:46.031 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.023+0000 7fe52e59c700 1 --2- 172.21.15.99:0/2693865847 >> [v2:172.21.15.191:6835/1165162515,v1:172.21.15.191:6837/1165162515] conn(0x56176247d460 0x5617624771e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7147392c310186b server_cookie=f45fcfa52af7417f in_seq=0 out_seq=0 2023-10-08T00:17:46.032 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.023+0000 7fe50cff9700 1 -- 172.21.15.99:0/2693865847 <== mds.0 v2:172.21.15.191:6835/1165162515 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x561762326e00 con 0x56176247d460 2023-10-08T00:17:46.032 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.023+0000 7fe50cff9700 10 client.7626 handle_command_reply: tid=0 2023-10-08T00:17:46.032 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.023+0000 7fe53dd1ab80 1 client.7626 shutdown 2023-10-08T00:17:46.032 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.023+0000 7fe53dd1ab80 1 -- 172.21.15.99:0/2693865847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56176247dd00 msgr2=0x5617624dfd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:46.032 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.023+0000 7fe53dd1ab80 1 --2- 172.21.15.99:0/2693865847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56176247dd00 0x5617624dfd20 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fe5300ce250 tx=0x7fe520009040 comp rx=0 tx=0).stop 2023-10-08T00:17:46.032 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.023+0000 7fe53dd1ab80 1 -- 172.21.15.99:0/2693865847 shutdown_connections 2023-10-08T00:17:46.032 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.023+0000 7fe53dd1ab80 1 --2- 172.21.15.99:0/2693865847 >> [v2:172.21.15.191:6835/1165162515,v1:172.21.15.191:6837/1165162515] conn(0x56176247d460 0x5617624771e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:46.032 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.023+0000 7fe53dd1ab80 1 --2- 172.21.15.99:0/2693865847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56176247dd00 0x5617624dfd20 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:46.032 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.023+0000 7fe53dd1ab80 1 --2- 172.21.15.99:0/2693865847 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561762433a50 0x561762488bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:46.032 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.023+0000 7fe53dd1ab80 1 --2- 172.21.15.99:0/2693865847 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56176245cf60 0x5617624df7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:46.033 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.023+0000 7fe50cff9700 0 client.7626 ms_handle_reset on v2:172.21.15.191:6835/1165162515 2023-10-08T00:17:46.033 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.023+0000 7fe53dd1ab80 1 -- 172.21.15.99:0/2693865847 >> 172.21.15.99:0/2693865847 conn(0x561762435370 msgr2=0x56176242ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:46.033 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.023+0000 7fe53dd1ab80 1 -- 172.21.15.99:0/2693865847 shutdown_connections 2023-10-08T00:17:46.033 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.023+0000 7fe53dd1ab80 1 -- 172.21.15.99:0/2693865847 wait complete. 2023-10-08T00:17:46.033 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.024+0000 7fe53dd1ab80 20 client.7626 trim_cache size 0 max 16384 2023-10-08T00:17:46.033 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:17:46.033 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", 2023-10-08T00:17:46.034 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 0, 2023-10-08T00:17:46.034 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 7494, 2023-10-08T00:17:46.034 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2023-10-08T00:17:46.034 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2023-10-08T00:17:46.034 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2023-10-08T00:17:46.034 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 5.7800377840000001, 2023-10-08T00:17:46.034 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 70, 2023-10-08T00:17:46.034 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 154, 2023-10-08T00:17:46.034 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 154, 2023-10-08T00:17:46.034 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 10.901845862 2023-10-08T00:17:46.035 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:17:46.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.024+0000 7fe534a22700 1 -- 172.21.15.99:0/2731074131 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe510066f60 msgr2=0x7fe510069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:46.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.024+0000 7fe534a22700 1 --2- 172.21.15.99:0/2731074131 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe510066f60 0x7fe510069410 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fe5280094d0 tx=0x7fe52800b040 comp rx=0 tx=0).stop 2023-10-08T00:17:46.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.024+0000 7fe534a22700 1 -- 172.21.15.99:0/2731074131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5300c4760 msgr2=0x7fe53016b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:46.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.024+0000 7fe534a22700 1 --2- 172.21.15.99:0/2731074131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5300c4760 0x7fe53016b170 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fe51800c890 tx=0x7fe518015000 comp rx=0 tx=0).stop 2023-10-08T00:17:46.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.024+0000 7fe534a22700 1 -- 172.21.15.99:0/2731074131 shutdown_connections 2023-10-08T00:17:46.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.024+0000 7fe534a22700 1 --2- 172.21.15.99:0/2731074131 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe510066f60 0x7fe510069410 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:46.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.024+0000 7fe534a22700 1 --2- 172.21.15.99:0/2731074131 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5300d0e20 0x7fe5300c9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:46.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.024+0000 7fe534a22700 1 --2- 172.21.15.99:0/2731074131 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe51c004810 0x7fe5300c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:46.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.024+0000 7fe534a22700 1 --2- 172.21.15.99:0/2731074131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5300c4760 0x7fe53016b170 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:46.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.024+0000 7fe534a22700 1 -- 172.21.15.99:0/2731074131 >> 172.21.15.99:0/2731074131 conn(0x7fe530067660 msgr2=0x7fe530159340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:46.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.024+0000 7fe534a22700 1 -- 172.21.15.99:0/2731074131 shutdown_connections 2023-10-08T00:17:46.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.024+0000 7fe534a22700 1 -- 172.21.15.99:0/2731074131 wait complete. 2023-10-08T00:17:46.038 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T00:17:46.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.150+0000 7fd928af5700 1 Processor -- start 2023-10-08T00:17:46.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.151+0000 7fd928af5700 1 -- start start 2023-10-08T00:17:46.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.151+0000 7fd928af5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9240cf1c0 0x7fd92415e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:46.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.151+0000 7fd928af5700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9240cca20 con 0x7fd924159cf0 2023-10-08T00:17:46.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.151+0000 7fd928af5700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd92415ea40 con 0x7fd92415a0c0 2023-10-08T00:17:46.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.151+0000 7fd928af5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd924067660 con 0x7fd9240cf1c0 2023-10-08T00:17:46.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.151+0000 7fd92259c700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fd924159cf0 0x7fd9240ceab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:60482/0 (socket says 172.21.15.99:60482) 2023-10-08T00:17:46.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.151+0000 7fd92259c700 1 -- 172.21.15.99:0/4043910976 learned_addr learned my addr 172.21.15.99:0/4043910976 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:17:46.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.152+0000 7fd919d9b700 1 --2- 172.21.15.99:0/4043910976 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9240cf1c0 0x7fd92415e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:46.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.152+0000 7fd921d9b700 1 -- 172.21.15.99:0/4043910976 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1963423616 0 0) 0x7fd9240cca20 con 0x7fd924159cf0 2023-10-08T00:17:46.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.152+0000 7fd921d9b700 1 -- 172.21.15.99:0/4043910976 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd910003570 con 0x7fd924159cf0 2023-10-08T00:17:46.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.152+0000 7fd921d9b700 1 -- 172.21.15.99:0/4043910976 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1757524026 0 0) 0x7fd91c001490 con 0x7fd92415a0c0 2023-10-08T00:17:46.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.152+0000 7fd921d9b700 1 -- 172.21.15.99:0/4043910976 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9240cca20 con 0x7fd92415a0c0 2023-10-08T00:17:46.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.152+0000 7fd919d9b700 1 -- 172.21.15.99:0/4043910976 >> v1:172.21.15.191:6789/0 conn(0x7fd92415a0c0 legacy=0x7fd924068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:46.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.152+0000 7fd919d9b700 1 -- 172.21.15.99:0/4043910976 >> v1:172.21.15.99:6789/0 conn(0x7fd924159cf0 legacy=0x7fd9240ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:46.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.152+0000 7fd919d9b700 1 -- 172.21.15.99:0/4043910976 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd924067f00 con 0x7fd9240cf1c0 2023-10-08T00:17:46.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.152+0000 7fd919d9b700 1 --2- 172.21.15.99:0/4043910976 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9240cf1c0 0x7fd92415e500 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd90c009910 tx=0x7fd90c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29621498a6e4ab54 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:46.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.152+0000 7fd921d9b700 1 -- 172.21.15.99:0/4043910976 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c011630 con 0x7fd9240cf1c0 2023-10-08T00:17:46.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.152+0000 7fd921d9b700 1 -- 172.21.15.99:0/4043910976 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd90c011790 con 0x7fd9240cf1c0 2023-10-08T00:17:46.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.152+0000 7fd921d9b700 1 -- 172.21.15.99:0/4043910976 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c011b30 con 0x7fd9240cf1c0 2023-10-08T00:17:46.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.153+0000 7fd928af5700 1 -- 172.21.15.99:0/4043910976 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9240cf1c0 msgr2=0x7fd92415e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:46.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.153+0000 7fd928af5700 1 --2- 172.21.15.99:0/4043910976 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9240cf1c0 0x7fd92415e500 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd90c009910 tx=0x7fd90c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:17:46.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.153+0000 7fd928af5700 1 -- 172.21.15.99:0/4043910976 shutdown_connections 2023-10-08T00:17:46.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.153+0000 7fd928af5700 1 --2- 172.21.15.99:0/4043910976 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9240cf1c0 0x7fd92415e500 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:46.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.153+0000 7fd928af5700 1 -- 172.21.15.99:0/4043910976 >> 172.21.15.99:0/4043910976 conn(0x7fd92406c6e0 msgr2=0x7fd924069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:46.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.153+0000 7fd928af5700 1 -- 172.21.15.99:0/4043910976 shutdown_connections 2023-10-08T00:17:46.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.153+0000 7fd928af5700 1 -- 172.21.15.99:0/4043910976 wait complete. 2023-10-08T00:17:46.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.154+0000 7fd928af5700 1 Processor -- start 2023-10-08T00:17:46.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.154+0000 7fd928af5700 1 -- start start 2023-10-08T00:17:46.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.155+0000 7fd928af5700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9240cf1c0 0x7fd9240acab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:46.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.155+0000 7fd928af5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd924159cf0 0x7fd92416a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:46.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.155+0000 7fd928af5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd92415a0c0 0x7fd924154640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:46.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.155+0000 7fd928af5700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9240ad1b0 con 0x7fd924159cf0 2023-10-08T00:17:46.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.155+0000 7fd928af5700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92416ad60 con 0x7fd9240cf1c0 2023-10-08T00:17:46.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.155+0000 7fd928af5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd924154b80 con 0x7fd92415a0c0 2023-10-08T00:17:46.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.155+0000 7fd919d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd924159cf0 0x7fd92416a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:46.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.155+0000 7fd919d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd924159cf0 0x7fd92416a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53170/0 (socket says 172.21.15.99:53170) 2023-10-08T00:17:46.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.155+0000 7fd919d9b700 1 -- 172.21.15.99:0/356050161 learned_addr learned my addr 172.21.15.99:0/356050161 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:46.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.155+0000 7fd922d9d700 1 --2- 172.21.15.99:0/356050161 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd92415a0c0 0x7fd924154640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:46.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.155+0000 7fd92259c700 1 --2- 172.21.15.99:0/356050161 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9240cf1c0 0x7fd9240acab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:46.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.155+0000 7fd919d9b700 1 -- 172.21.15.99:0/356050161 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd92415a0c0 msgr2=0x7fd924154640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:46.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.155+0000 7fd919d9b700 1 --2- 172.21.15.99:0/356050161 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd92415a0c0 0x7fd924154640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:46.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.155+0000 7fd919d9b700 1 -- 172.21.15.99:0/356050161 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9240cf1c0 msgr2=0x7fd9240acab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:46.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.155+0000 7fd919d9b700 1 --2- 172.21.15.99:0/356050161 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9240cf1c0 0x7fd9240acab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:46.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.155+0000 7fd919d9b700 1 -- 172.21.15.99:0/356050161 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd90c009420 con 0x7fd924159cf0 2023-10-08T00:17:46.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.155+0000 7fd922d9d700 1 --2- 172.21.15.99:0/356050161 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd92415a0c0 0x7fd924154640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:17:46.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.155+0000 7fd92259c700 1 --2- 172.21.15.99:0/356050161 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9240cf1c0 0x7fd9240acab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:17:46.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.155+0000 7fd919d9b700 1 --2- 172.21.15.99:0/356050161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd924159cf0 0x7fd92416a820 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fd92406de20 tx=0x7fd90c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:46.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.155+0000 7fd91affd700 1 -- 172.21.15.99:0/356050161 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c01f070 con 0x7fd924159cf0 2023-10-08T00:17:46.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.156+0000 7fd91affd700 1 -- 172.21.15.99:0/356050161 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd90c000bc0 con 0x7fd924159cf0 2023-10-08T00:17:46.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.156+0000 7fd928af5700 1 -- 172.21.15.99:0/356050161 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd924157730 con 0x7fd924159cf0 2023-10-08T00:17:46.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.156+0000 7fd91affd700 1 -- 172.21.15.99:0/356050161 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c011b30 con 0x7fd924159cf0 2023-10-08T00:17:46.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.156+0000 7fd928af5700 1 -- 172.21.15.99:0/356050161 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd924157cf0 con 0x7fd924159cf0 2023-10-08T00:17:46.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.156+0000 7fd928af5700 1 -- 172.21.15.99:0/356050161 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd92404c3c0 con 0x7fd924159cf0 2023-10-08T00:17:46.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.157+0000 7fd91affd700 1 -- 172.21.15.99:0/356050161 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd90c023070 con 0x7fd924159cf0 2023-10-08T00:17:46.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.157+0000 7fd91affd700 1 --2- 172.21.15.99:0/356050161 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd904066f30 0x7fd9040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:46.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.157+0000 7fd91affd700 1 -- 172.21.15.99:0/356050161 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd90c085940 con 0x7fd924159cf0 2023-10-08T00:17:46.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.157+0000 7fd92259c700 1 --2- 172.21.15.99:0/356050161 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd904066f30 0x7fd9040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:46.162 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.158+0000 7fd92259c700 1 --2- 172.21.15.99:0/356050161 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd904066f30 0x7fd9040693e0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd924160600 tx=0x7fd914009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:46.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.162+0000 7fd91affd700 1 -- 172.21.15.99:0/356050161 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd90c052bb0 con 0x7fd924159cf0 2023-10-08T00:17:46.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.389+0000 7fd928af5700 1 -- 172.21.15.99:0/356050161 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd9240ce730 con 0x7fd924159cf0 2023-10-08T00:17:46.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.389+0000 7fd91affd700 1 -- 172.21.15.99:0/356050161 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v70) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd90c02a070 con 0x7fd924159cf0 2023-10-08T00:17:46.390 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T00:17:46.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.391+0000 7fd928af5700 1 -- 172.21.15.99:0/356050161 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd904066f30 msgr2=0x7fd9040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:46.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.391+0000 7fd928af5700 1 --2- 172.21.15.99:0/356050161 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd904066f30 0x7fd9040693e0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd924160600 tx=0x7fd914009040 comp rx=0 tx=0).stop 2023-10-08T00:17:46.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.391+0000 7fd928af5700 1 -- 172.21.15.99:0/356050161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd924159cf0 msgr2=0x7fd92416a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:46.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.391+0000 7fd928af5700 1 --2- 172.21.15.99:0/356050161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd924159cf0 0x7fd92416a820 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fd92406de20 tx=0x7fd90c017000 comp rx=0 tx=0).stop 2023-10-08T00:17:46.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.391+0000 7fd928af5700 1 -- 172.21.15.99:0/356050161 shutdown_connections 2023-10-08T00:17:46.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.391+0000 7fd928af5700 1 --2- 172.21.15.99:0/356050161 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd904066f30 0x7fd9040693e0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:46.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.391+0000 7fd928af5700 1 --2- 172.21.15.99:0/356050161 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9240cf1c0 0x7fd9240acab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:46.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.391+0000 7fd928af5700 1 --2- 172.21.15.99:0/356050161 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd92415a0c0 0x7fd924154640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:46.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.391+0000 7fd928af5700 1 --2- 172.21.15.99:0/356050161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd924159cf0 0x7fd92416a820 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:46.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.391+0000 7fd928af5700 1 -- 172.21.15.99:0/356050161 >> 172.21.15.99:0/356050161 conn(0x7fd92406c6e0 msgr2=0x7fd9240ca000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:46.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.391+0000 7fd928af5700 1 -- 172.21.15.99:0/356050161 shutdown_connections 2023-10-08T00:17:46.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:46.391+0000 7fd928af5700 1 -- 172.21.15.99:0/356050161 wait complete. 2023-10-08T00:17:46.404 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T00:17:46.404 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T00:17:46.404 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi099.front.sepia.ceph.com 2023-10-08T00:17:46.404 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T00:17:46.404 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T00:17:46.404 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T00:17:46.404 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T00:17:46.404 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T00:17:46.405 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T00:17:46.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:17:46.405 DEBUG:teuthology.orchestra.run.smithi099:> ip addr 2023-10-08T00:17:46.420 INFO:teuthology.orchestra.run.smithi099.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T00:17:46.420 INFO:teuthology.orchestra.run.smithi099.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T00:17:46.420 INFO:teuthology.orchestra.run.smithi099.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T00:17:46.421 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:17:46.421 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 ::1/128 scope host 2023-10-08T00:17:46.421 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:17:46.421 INFO:teuthology.orchestra.run.smithi099.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:17:46.421 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:10 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:17:46.421 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f0 2023-10-08T00:17:46.421 INFO:teuthology.orchestra.run.smithi099.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:17:46.421 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:11 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:17:46.421 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f1 2023-10-08T00:17:46.421 INFO:teuthology.orchestra.run.smithi099.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:17:46.421 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:02 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:17:46.421 INFO:teuthology.orchestra.run.smithi099.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T00:17:46.422 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:03 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:17:46.422 INFO:teuthology.orchestra.run.smithi099.stdout: inet 172.21.15.99/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T00:17:46.422 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft 42123sec preferred_lft 42123sec 2023-10-08T00:17:46.422 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::ec4:7aff:fe8f:d803/64 scope link 2023-10-08T00:17:46.422 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:17:46.422 INFO:teuthology.orchestra.run.smithi099.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T00:17:46.422 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 22:2c:1c:f6:42:e1 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:17:46.422 INFO:teuthology.orchestra.run.smithi099.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T00:17:46.422 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:17:46.422 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::600d:26ff:fede:5eec/64 scope link 2023-10-08T00:17:46.423 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:17:46.423 INFO:teuthology.orchestra.run.smithi099.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T00:17:46.423 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether ae:2f:b5:94:69:ba brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T00:17:46.423 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::ac2f:b5ff:fe94:69ba/64 scope link 2023-10-08T00:17:46.423 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:17:46.423 INFO:teuthology.orchestra.run.smithi099.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T00:17:46.423 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 22:2c:1c:f6:42:e1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T00:17:46.423 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::202c:1cff:fef6:42e1/64 scope link 2023-10-08T00:17:46.424 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:17:46.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:17:46.424 DEBUG:teuthology.orchestra.run.smithi099:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T00:17:46.483 INFO:teuthology.orchestra.run.smithi099.stdout:1 2023-10-08T00:17:46.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:17:46.484 DEBUG:teuthology.orchestra.run.smithi099:> route 2023-10-08T00:17:46.541 INFO:teuthology.orchestra.run.smithi099.stdout:Kernel IP routing table 2023-10-08T00:17:46.541 INFO:teuthology.orchestra.run.smithi099.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T00:17:46.541 INFO:teuthology.orchestra.run.smithi099.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T00:17:46.541 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T00:17:46.541 INFO:teuthology.orchestra.run.smithi099.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T00:17:46.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:17:46.542 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2023-10-08T00:17:46.542 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2023-10-08T00:17:46.542 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T00:17:46.542 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T00:17:46.542 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T00:17:46.542 DEBUG:teuthology.orchestra.run.smithi099:> ') 2023-10-08T00:17:46.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:17:46.641 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2023-10-08T00:17:46.656 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T00:17:46.656 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T00:17:46.656 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T00:17:46.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:17:46.657 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set brx.0 up 2023-10-08T00:17:46.721 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:17:46.721 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T00:17:46.776 INFO:teuthology.orchestra.run.smithi099.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T00:17:46.776 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:17:46.777 DEBUG:teuthology.orchestra.run.smithi099:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T00:17:46.831 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-08T00:17:46.831 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T00:17:46.832 DEBUG:teuthology.orchestra.run.smithi099:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:17:46.911 INFO:teuthology.orchestra.run.smithi099.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:17:46.911 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:17:46.930 INFO:teuthology.orchestra.run.smithi099.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:17:46.930 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:17:46.930 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: resolved to: "172.21.15.99:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T00:17:46.930 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: trying mount with new device syntax: 0@06962cdb-9a6f-417f-82fd-9e4e3dedbccf.cephfs=/ 2023-10-08T00:17:46.930 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.99:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T00:17:46.930 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T00:17:46.930 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T00:17:46.930 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T00:17:46.951 INFO:teuthology.orchestra.run.smithi099.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:17:46.952 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:17:46.952 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: resolved to: "172.21.15.99:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T00:17:46.952 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: trying mount with new device syntax: 0@06962cdb-9a6f-417f-82fd-9e4e3dedbccf.cephfs=/ 2023-10-08T00:17:46.952 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.99:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T00:17:46.952 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T00:17:46.952 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T00:17:46.952 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T00:17:46.952 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: resolved to: "172.21.15.99:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T00:17:46.952 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.99:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T00:17:46.952 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=06962cdb-9a6f-417f-82fd-9e4e3dedbccf" will pass to kernel 2023-10-08T00:17:46.954 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T00:17:46.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:17:46.955 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T00:17:47.020 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T00:17:47.176 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.175+0000 7f625ac06700 1 Processor -- start 2023-10-08T00:17:47.176 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.175+0000 7f625ac06700 1 -- start start 2023-10-08T00:17:47.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.176+0000 7f625ac06700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f625415a620 0x7f625415d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:47.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.176+0000 7f625ac06700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f625415d580 0x7f6254158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:47.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.176+0000 7f625ac06700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f625415cbb0 con 0x7f6254157870 2023-10-08T00:17:47.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.176+0000 7f625ac06700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f625415bdb0 con 0x7f625415a620 2023-10-08T00:17:47.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.176+0000 7f625ac06700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f625415bfa0 con 0x7f625415d580 2023-10-08T00:17:47.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.176+0000 7f624bfff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f625415a620 0x7f625415d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:47.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.176+0000 7f624bfff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f625415a620 0x7f625415d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53184/0 (socket says 172.21.15.99:53184) 2023-10-08T00:17:47.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.176+0000 7f624bfff700 1 -- 172.21.15.99:0/3438015184 learned_addr learned my addr 172.21.15.99:0/3438015184 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:47.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.176+0000 7f62591a3700 1 --2- 172.21.15.99:0/3438015184 >> v2:172.21.15.191:3300/0 conn(0x7f625415d580 0x7f6254158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:47.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.176+0000 7f624bfff700 1 -- 172.21.15.99:0/3438015184 >> v2:172.21.15.191:3300/0 conn(0x7f625415d580 msgr2=0x7f6254158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:47.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.176+0000 7f624bfff700 1 --2- 172.21.15.99:0/3438015184 >> v2:172.21.15.191:3300/0 conn(0x7f625415d580 0x7f6254158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:47.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.176+0000 7f624bfff700 1 -- 172.21.15.99:0/3438015184 >> v1:172.21.15.191:6789/0 conn(0x7f6254157870 legacy=0x7f625415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:47.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.176+0000 7f624bfff700 1 -- 172.21.15.99:0/3438015184 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f625415c280 con 0x7f625415a620 2023-10-08T00:17:47.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.177+0000 7f624bfff700 1 --2- 172.21.15.99:0/3438015184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f625415a620 0x7f625415d040 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f625406b4e0 tx=0x7f6244009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=892af248a06987f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:47.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.177+0000 7f624b7fe700 1 -- 172.21.15.99:0/3438015184 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624400b820 con 0x7f625415a620 2023-10-08T00:17:47.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.177+0000 7f624b7fe700 1 -- 172.21.15.99:0/3438015184 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f624400b980 con 0x7f625415a620 2023-10-08T00:17:47.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.177+0000 7f624b7fe700 1 -- 172.21.15.99:0/3438015184 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624400bd20 con 0x7f625415a620 2023-10-08T00:17:47.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.177+0000 7f625ac06700 1 -- 172.21.15.99:0/3438015184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f625415a620 msgr2=0x7f625415d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:47.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.177+0000 7f625ac06700 1 --2- 172.21.15.99:0/3438015184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f625415a620 0x7f625415d040 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f625406b4e0 tx=0x7f6244009250 comp rx=0 tx=0).stop 2023-10-08T00:17:47.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.177+0000 7f625ac06700 1 -- 172.21.15.99:0/3438015184 shutdown_connections 2023-10-08T00:17:47.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.177+0000 7f625ac06700 1 --2- 172.21.15.99:0/3438015184 >> v2:172.21.15.191:3300/0 conn(0x7f625415d580 0x7f6254158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:47.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.177+0000 7f625ac06700 1 --2- 172.21.15.99:0/3438015184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f625415a620 0x7f625415d040 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:47.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.177+0000 7f625ac06700 1 -- 172.21.15.99:0/3438015184 >> 172.21.15.99:0/3438015184 conn(0x7f6254068ce0 msgr2=0x7f62540690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:47.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.177+0000 7f625ac06700 1 -- 172.21.15.99:0/3438015184 shutdown_connections 2023-10-08T00:17:47.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.177+0000 7f625ac06700 1 -- 172.21.15.99:0/3438015184 wait complete. 2023-10-08T00:17:47.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.178+0000 7f625ac06700 1 Processor -- start 2023-10-08T00:17:47.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.179+0000 7f625ac06700 1 -- start start 2023-10-08T00:17:47.184 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.179+0000 7f625ac06700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6254157870 0x7f62540b2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:47.184 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.179+0000 7f625ac06700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f625415a620 0x7f62540b1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:47.184 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.179+0000 7f625ac06700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f625415d580 0x7f62540cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:47.184 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.179+0000 7f625ac06700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62540b06c0 con 0x7f625415d580 2023-10-08T00:17:47.184 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.179+0000 7f625ac06700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62540d09b0 con 0x7f6254157870 2023-10-08T00:17:47.184 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.179+0000 7f625ac06700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62540cfe90 con 0x7f625415a620 2023-10-08T00:17:47.184 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.179+0000 7f62591a3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f625415d580 0x7f62540cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:47.184 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.179+0000 7f62591a3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f625415d580 0x7f62540cf2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53196/0 (socket says 172.21.15.99:53196) 2023-10-08T00:17:47.185 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.179+0000 7f62591a3700 1 -- 172.21.15.99:0/1134943367 learned_addr learned my addr 172.21.15.99:0/1134943367 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:47.185 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.180+0000 7f62589a2700 1 --2- 172.21.15.99:0/1134943367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6254157870 0x7f62540b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:47.185 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.180+0000 7f624bfff700 1 --2- 172.21.15.99:0/1134943367 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f625415a620 0x7f62540b1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:47.185 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.180+0000 7f62591a3700 1 -- 172.21.15.99:0/1134943367 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f625415a620 msgr2=0x7f62540b1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:47.185 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.180+0000 7f62591a3700 1 --2- 172.21.15.99:0/1134943367 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f625415a620 0x7f62540b1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:47.185 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.180+0000 7f62591a3700 1 -- 172.21.15.99:0/1134943367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6254157870 msgr2=0x7f62540b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:47.185 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.180+0000 7f62591a3700 1 --2- 172.21.15.99:0/1134943367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6254157870 0x7f62540b2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:47.185 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.180+0000 7f62591a3700 1 -- 172.21.15.99:0/1134943367 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62440094f0 con 0x7f625415d580 2023-10-08T00:17:47.185 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.180+0000 7f62589a2700 1 --2- 172.21.15.99:0/1134943367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6254157870 0x7f62540b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:17:47.185 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.180+0000 7f62591a3700 1 --2- 172.21.15.99:0/1134943367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f625415d580 0x7f62540cf2b0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f624c00cdb0 tx=0x7f624c0077a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:47.185 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.180+0000 7f6248ff9700 1 -- 172.21.15.99:0/1134943367 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624c00aac0 con 0x7f625415d580 2023-10-08T00:17:47.186 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.180+0000 7f6248ff9700 1 -- 172.21.15.99:0/1134943367 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f624c00ac20 con 0x7f625415d580 2023-10-08T00:17:47.186 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.180+0000 7f6248ff9700 1 -- 172.21.15.99:0/1134943367 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624c017430 con 0x7f625415d580 2023-10-08T00:17:47.186 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.180+0000 7f625ac06700 1 -- 172.21.15.99:0/1134943367 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62540d0170 con 0x7f625415d580 2023-10-08T00:17:47.186 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.180+0000 7f625ac06700 1 -- 172.21.15.99:0/1134943367 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62540b0bf0 con 0x7f625415d580 2023-10-08T00:17:47.186 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.181+0000 7f625ac06700 1 -- 172.21.15.99:0/1134943367 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f625404c3c0 con 0x7f625415d580 2023-10-08T00:17:47.186 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.181+0000 7f6248ff9700 1 -- 172.21.15.99:0/1134943367 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f624c003020 con 0x7f625415d580 2023-10-08T00:17:47.186 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.182+0000 7f6248ff9700 1 --2- 172.21.15.99:0/1134943367 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6240066f30 0x7f62400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:47.186 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.182+0000 7f6248ff9700 1 -- 172.21.15.99:0/1134943367 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f624c013070 con 0x7f625415d580 2023-10-08T00:17:47.186 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.182+0000 7f62589a2700 1 --2- 172.21.15.99:0/1134943367 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6240066f30 0x7f62400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:47.186 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.182+0000 7f62589a2700 1 --2- 172.21.15.99:0/1134943367 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6240066f30 0x7f62400693e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f6250007fd0 tx=0x7f6250009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:47.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.186+0000 7f6248ff9700 1 -- 172.21.15.99:0/1134943367 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f624c051bf0 con 0x7f625415d580 2023-10-08T00:17:47.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.414+0000 7f625ac06700 1 -- 172.21.15.99:0/1134943367 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f625415c560 con 0x7f625415d580 2023-10-08T00:17:47.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.415+0000 7f6248ff9700 1 -- 172.21.15.99:0/1134943367 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v70) v1 ==== 53+0+73 (secure 0 0 0) 0x7f624c01c020 con 0x7f625415d580 2023-10-08T00:17:47.416 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T00:17:47.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.417+0000 7f625ac06700 1 -- 172.21.15.99:0/1134943367 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6240066f30 msgr2=0x7f62400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:47.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.417+0000 7f625ac06700 1 --2- 172.21.15.99:0/1134943367 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6240066f30 0x7f62400693e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f6250007fd0 tx=0x7f6250009040 comp rx=0 tx=0).stop 2023-10-08T00:17:47.418 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.417+0000 7f625ac06700 1 -- 172.21.15.99:0/1134943367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f625415d580 msgr2=0x7f62540cf2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:47.418 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.417+0000 7f625ac06700 1 --2- 172.21.15.99:0/1134943367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f625415d580 0x7f62540cf2b0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f624c00cdb0 tx=0x7f624c0077a0 comp rx=0 tx=0).stop 2023-10-08T00:17:47.418 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.417+0000 7f625ac06700 1 -- 172.21.15.99:0/1134943367 shutdown_connections 2023-10-08T00:17:47.418 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.417+0000 7f625ac06700 1 --2- 172.21.15.99:0/1134943367 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6240066f30 0x7f62400693e0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:47.418 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.417+0000 7f625ac06700 1 --2- 172.21.15.99:0/1134943367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6254157870 0x7f62540b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:47.418 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.417+0000 7f625ac06700 1 --2- 172.21.15.99:0/1134943367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f625415d580 0x7f62540cf2b0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:47.418 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.417+0000 7f625ac06700 1 --2- 172.21.15.99:0/1134943367 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f625415a620 0x7f62540b1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:47.418 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.417+0000 7f625ac06700 1 -- 172.21.15.99:0/1134943367 >> 172.21.15.99:0/1134943367 conn(0x7f6254068ce0 msgr2=0x7f6254158320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:47.418 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.417+0000 7f625ac06700 1 -- 172.21.15.99:0/1134943367 shutdown_connections 2023-10-08T00:17:47.418 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:47.417+0000 7f625ac06700 1 -- 172.21.15.99:0/1134943367 wait complete. 2023-10-08T00:17:47.429 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T00:17:47.429 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T00:17:47.429 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi191.front.sepia.ceph.com 2023-10-08T00:17:47.430 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-08T00:17:47.430 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-08T00:17:47.430 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T00:17:47.430 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T00:17:47.430 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T00:17:47.430 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T00:17:47.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:17:47.430 DEBUG:teuthology.orchestra.run.smithi191:> ip addr 2023-10-08T00:17:47.449 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T00:17:47.449 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T00:17:47.449 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T00:17:47.449 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:17:47.449 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2023-10-08T00:17:47.449 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:17:47.449 INFO:teuthology.orchestra.run.smithi191.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:17:47.450 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f6 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:17:47.450 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f0 2023-10-08T00:17:47.450 INFO:teuthology.orchestra.run.smithi191.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:17:47.450 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f7 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:17:47.450 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f1 2023-10-08T00:17:47.450 INFO:teuthology.orchestra.run.smithi191.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:17:47.450 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:10 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:17:47.450 INFO:teuthology.orchestra.run.smithi191.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T00:17:47.450 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:11 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:17:47.450 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.21.15.191/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T00:17:47.450 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft 42124sec preferred_lft 42124sec 2023-10-08T00:17:47.451 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec4:7aff:fe88:7211/64 scope link 2023-10-08T00:17:47.451 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:17:47.451 INFO:teuthology.orchestra.run.smithi191.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T00:17:47.451 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 9a:bc:39:72:87:3d brd ff:ff:ff:ff:ff:ff 2023-10-08T00:17:47.451 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T00:17:47.451 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:17:47.451 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec77:a7ff:fea4:fbb/64 scope link 2023-10-08T00:17:47.451 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:17:47.451 INFO:teuthology.orchestra.run.smithi191.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T00:17:47.451 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether ea:1c:d9:46:05:b3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-08T00:17:47.451 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::e81c:d9ff:fe46:5b3/64 scope link 2023-10-08T00:17:47.451 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:17:47.452 INFO:teuthology.orchestra.run.smithi191.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T00:17:47.452 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 9a:bc:39:72:87:3d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-08T00:17:47.452 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::98bc:39ff:fe72:873d/64 scope link 2023-10-08T00:17:47.452 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:17:47.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:17:47.452 DEBUG:teuthology.orchestra.run.smithi191:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T00:17:47.518 INFO:teuthology.orchestra.run.smithi191.stdout:1 2023-10-08T00:17:47.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:17:47.519 DEBUG:teuthology.orchestra.run.smithi191:> route 2023-10-08T00:17:47.576 INFO:teuthology.orchestra.run.smithi191.stdout:Kernel IP routing table 2023-10-08T00:17:47.577 INFO:teuthology.orchestra.run.smithi191.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T00:17:47.577 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T00:17:47.577 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T00:17:47.577 INFO:teuthology.orchestra.run.smithi191.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T00:17:47.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:17:47.577 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2023-10-08T00:17:47.577 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2023-10-08T00:17:47.577 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T00:17:47.577 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T00:17:47.578 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T00:17:47.578 DEBUG:teuthology.orchestra.run.smithi191:> ') 2023-10-08T00:17:47.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:17:47.682 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2023-10-08T00:17:47.697 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-08T00:17:47.698 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-08T00:17:47.698 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T00:17:47.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:17:47.698 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set brx.0 up 2023-10-08T00:17:47.764 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:17:47.765 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-08T00:17:47.820 INFO:teuthology.orchestra.run.smithi191.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-08T00:17:47.820 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:17:47.821 DEBUG:teuthology.orchestra.run.smithi191:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-08T00:17:47.875 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-08T00:17:47.875 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T00:17:47.875 DEBUG:teuthology.orchestra.run.smithi191:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:17:47.956 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:17:47.956 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:17:47.975 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:17:47.975 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:17:47.975 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.99:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T00:17:47.975 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@06962cdb-9a6f-417f-82fd-9e4e3dedbccf.cephfs=/ 2023-10-08T00:17:47.975 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.99:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T00:17:47.975 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T00:17:47.976 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T00:17:47.976 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T00:17:47.996 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:17:47.996 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:17:47.996 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.99:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T00:17:47.997 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@06962cdb-9a6f-417f-82fd-9e4e3dedbccf.cephfs=/ 2023-10-08T00:17:47.997 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.99:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T00:17:47.997 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T00:17:47.997 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T00:17:47.997 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T00:17:47.997 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.99:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T00:17:47.997 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.99:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T00:17:47.998 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=06962cdb-9a6f-417f-82fd-9e4e3dedbccf" will pass to kernel 2023-10-08T00:17:47.999 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T00:17:47.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:17:47.999 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-08T00:17:48.063 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:17:48.183 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.182+0000 7f590a66c700 1 Processor -- start 2023-10-08T00:17:48.183 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.182+0000 7f590a66c700 1 -- start start 2023-10-08T00:17:48.186 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.183+0000 7f590a66c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5904157870 0x7f590415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:48.186 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.183+0000 7f590a66c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f590415a620 0x7f590415d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:48.186 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.183+0000 7f590a66c700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f590415cbb0 con 0x7f590415d580 2023-10-08T00:17:48.186 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.183+0000 7f590a66c700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f590415bfa0 con 0x7f590415a620 2023-10-08T00:17:48.186 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.183+0000 7f590a66c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f590415c280 con 0x7f5904157870 2023-10-08T00:17:48.186 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.183+0000 7f59037fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f590415a620 0x7f590415d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:48.186 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.183+0000 7f5908c09700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f590415d580 0x7f5904158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:60490/0 (socket says 172.21.15.99:60490) 2023-10-08T00:17:48.186 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.183+0000 7f5908c09700 1 -- 172.21.15.99:0/3799609120 learned_addr learned my addr 172.21.15.99:0/3799609120 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:17:48.186 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.183+0000 7f5903fff700 1 --2- 172.21.15.99:0/3799609120 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5904157870 0x7f590415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:48.186 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.183+0000 7f59037fe700 1 -- 172.21.15.99:0/3799609120 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5904157870 msgr2=0x7f590415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:48.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.183+0000 7f59037fe700 1 --2- 172.21.15.99:0/3799609120 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5904157870 0x7f590415b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:48.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.183+0000 7f59037fe700 1 -- 172.21.15.99:0/3799609120 >> v1:172.21.15.99:6789/0 conn(0x7f590415d580 legacy=0x7f5904158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:48.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.183+0000 7f59037fe700 1 -- 172.21.15.99:0/3799609120 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f590415c560 con 0x7f590415a620 2023-10-08T00:17:48.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.183+0000 7f5902ffd700 1 -- 172.21.15.99:0/3799609120 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3414923427 0 0) 0x7f590415cbb0 con 0x7f590415d580 2023-10-08T00:17:48.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.183+0000 7f59037fe700 1 --2- 172.21.15.99:0/3799609120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f590415a620 0x7f590415d040 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f58f80098e0 tx=0x7f58f80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2006315f6d0e869b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:48.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.183+0000 7f5902ffd700 1 -- 172.21.15.99:0/3799609120 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f8009e80 con 0x7f590415a620 2023-10-08T00:17:48.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.183+0000 7f5902ffd700 1 -- 172.21.15.99:0/3799609120 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f800b860 con 0x7f590415a620 2023-10-08T00:17:48.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.183+0000 7f5902ffd700 1 -- 172.21.15.99:0/3799609120 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f800bbe0 con 0x7f590415a620 2023-10-08T00:17:48.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.184+0000 7f590a66c700 1 -- 172.21.15.99:0/3799609120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f590415a620 msgr2=0x7f590415d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:48.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.184+0000 7f590a66c700 1 --2- 172.21.15.99:0/3799609120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f590415a620 0x7f590415d040 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f58f80098e0 tx=0x7f58f80091a0 comp rx=0 tx=0).stop 2023-10-08T00:17:48.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.184+0000 7f590a66c700 1 -- 172.21.15.99:0/3799609120 shutdown_connections 2023-10-08T00:17:48.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.184+0000 7f590a66c700 1 --2- 172.21.15.99:0/3799609120 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5904157870 0x7f590415b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:48.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.184+0000 7f590a66c700 1 --2- 172.21.15.99:0/3799609120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f590415a620 0x7f590415d040 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:48.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.184+0000 7f590a66c700 1 -- 172.21.15.99:0/3799609120 >> 172.21.15.99:0/3799609120 conn(0x7f5904068ce0 msgr2=0x7f59040690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:48.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.184+0000 7f590a66c700 1 -- 172.21.15.99:0/3799609120 shutdown_connections 2023-10-08T00:17:48.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.184+0000 7f590a66c700 1 -- 172.21.15.99:0/3799609120 wait complete. 2023-10-08T00:17:48.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.185+0000 7f590a66c700 1 Processor -- start 2023-10-08T00:17:48.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.185+0000 7f590a66c700 1 -- start start 2023-10-08T00:17:48.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.186+0000 7f590a66c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5904157870 0x7f590416d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:48.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.186+0000 7f590a66c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f590415a620 0x7f59040c5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:48.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.186+0000 7f590a66c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f590415d580 0x7f59040c6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:48.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.186+0000 7f590a66c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59040d0f70 con 0x7f590415d580 2023-10-08T00:17:48.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.186+0000 7f590a66c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59040b34c0 con 0x7f590415a620 2023-10-08T00:17:48.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.186+0000 7f590a66c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59040d0360 con 0x7f5904157870 2023-10-08T00:17:48.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.186+0000 7f5908c09700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f590415d580 0x7f59040c6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:48.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.186+0000 7f5903fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5904157870 0x7f590416d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:48.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.186+0000 7f5908c09700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f590415d580 0x7f59040c6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53212/0 (socket says 172.21.15.99:53212) 2023-10-08T00:17:48.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.186+0000 7f5908c09700 1 -- 172.21.15.99:0/2669540179 learned_addr learned my addr 172.21.15.99:0/2669540179 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:48.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.186+0000 7f59037fe700 1 --2- 172.21.15.99:0/2669540179 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f590415a620 0x7f59040c5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:48.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.186+0000 7f5908c09700 1 -- 172.21.15.99:0/2669540179 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5904157870 msgr2=0x7f590416d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:48.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.186+0000 7f5908c09700 1 --2- 172.21.15.99:0/2669540179 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5904157870 0x7f590416d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:48.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.186+0000 7f5908c09700 1 -- 172.21.15.99:0/2669540179 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f590415a620 msgr2=0x7f59040c5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:48.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.186+0000 7f5908c09700 1 --2- 172.21.15.99:0/2669540179 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f590415a620 0x7f59040c5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:48.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.186+0000 7f5908c09700 1 -- 172.21.15.99:0/2669540179 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f8009420 con 0x7f590415d580 2023-10-08T00:17:48.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.187+0000 7f5908c09700 1 --2- 172.21.15.99:0/2669540179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f590415d580 0x7f59040c6050 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f58fc00cbc0 tx=0x7f58fc001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:48.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.187+0000 7f58ebfff700 1 -- 172.21.15.99:0/2669540179 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58fc00a7e0 con 0x7f590415d580 2023-10-08T00:17:48.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.187+0000 7f58ebfff700 1 -- 172.21.15.99:0/2669540179 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58fc00a940 con 0x7f590415d580 2023-10-08T00:17:48.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.187+0000 7f590a66c700 1 -- 172.21.15.99:0/2669540179 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59040d0640 con 0x7f590415d580 2023-10-08T00:17:48.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.187+0000 7f58ebfff700 1 -- 172.21.15.99:0/2669540179 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58fc00ad00 con 0x7f590415d580 2023-10-08T00:17:48.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.187+0000 7f590a66c700 1 -- 172.21.15.99:0/2669540179 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59040d0c00 con 0x7f590415d580 2023-10-08T00:17:48.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.188+0000 7f590a66c700 1 -- 172.21.15.99:0/2669540179 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f590416a690 con 0x7f590415d580 2023-10-08T00:17:48.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.188+0000 7f58ebfff700 1 -- 172.21.15.99:0/2669540179 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f58fc007300 con 0x7f590415d580 2023-10-08T00:17:48.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.189+0000 7f58ebfff700 1 --2- 172.21.15.99:0/2669540179 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f58f0066f30 0x7f58f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:48.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.189+0000 7f58ebfff700 1 -- 172.21.15.99:0/2669540179 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f58fc0847e0 con 0x7f590415d580 2023-10-08T00:17:48.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.189+0000 7f5903fff700 1 --2- 172.21.15.99:0/2669540179 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f58f0066f30 0x7f58f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:48.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.189+0000 7f5903fff700 1 --2- 172.21.15.99:0/2669540179 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f58f0066f30 0x7f58f00693e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f58f4006fd0 tx=0x7f58f4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:48.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.193+0000 7f58ebfff700 1 -- 172.21.15.99:0/2669540179 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f58fc051ad0 con 0x7f590415d580 2023-10-08T00:17:48.423 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.422+0000 7f590a66c700 1 -- 172.21.15.99:0/2669540179 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f590404c3c0 con 0x7f590415d580 2023-10-08T00:17:48.423 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.423+0000 7f58ebfff700 1 -- 172.21.15.99:0/2669540179 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4599 (secure 0 0 0) 0x7f58fc00ae60 con 0x7f590415d580 2023-10-08T00:17:48.423 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:17:48.424 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":70,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6056,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/3025449992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3025449992},{"type":"v1","addr":"172.21.15.191:6836","nonce":3025449992}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":7476,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/86350138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":86350138},{"type":"v1","addr":"172.21.15.99:6834","nonce":86350138}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":7479,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1282887036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1282887036},{"type":"v1","addr":"172.21.15.99:6835","nonce":1282887036}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:17:39.623734+0000","modified":"2023-10-08T00:17:43.252633+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7494":{"gid":7494,"name":"d","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/1165162515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":1165162515},{"type":"v1","addr":"172.21.15.191:6837","nonce":1165162515}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-08T00:17:48.426 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.425+0000 7f590a66c700 1 -- 172.21.15.99:0/2669540179 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f58f0066f30 msgr2=0x7f58f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:48.426 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.425+0000 7f590a66c700 1 --2- 172.21.15.99:0/2669540179 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f58f0066f30 0x7f58f00693e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f58f4006fd0 tx=0x7f58f4008040 comp rx=0 tx=0).stop 2023-10-08T00:17:48.426 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.425+0000 7f590a66c700 1 -- 172.21.15.99:0/2669540179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f590415d580 msgr2=0x7f59040c6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:48.426 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.425+0000 7f590a66c700 1 --2- 172.21.15.99:0/2669540179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f590415d580 0x7f59040c6050 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f58fc00cbc0 tx=0x7f58fc001980 comp rx=0 tx=0).stop 2023-10-08T00:17:48.426 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.425+0000 7f590a66c700 1 -- 172.21.15.99:0/2669540179 shutdown_connections 2023-10-08T00:17:48.426 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.425+0000 7f590a66c700 1 --2- 172.21.15.99:0/2669540179 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f58f0066f30 0x7f58f00693e0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:48.426 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.425+0000 7f590a66c700 1 --2- 172.21.15.99:0/2669540179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f590415d580 0x7f59040c6050 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:48.426 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.425+0000 7f590a66c700 1 --2- 172.21.15.99:0/2669540179 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5904157870 0x7f590416d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:48.426 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.425+0000 7f590a66c700 1 --2- 172.21.15.99:0/2669540179 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f590415a620 0x7f59040c5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:48.426 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.425+0000 7f590a66c700 1 -- 172.21.15.99:0/2669540179 >> 172.21.15.99:0/2669540179 conn(0x7f5904068ce0 msgr2=0x7f5904155710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:48.427 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.425+0000 7f590a66c700 1 -- 172.21.15.99:0/2669540179 shutdown_connections 2023-10-08T00:17:48.427 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.425+0000 7f590a66c700 1 -- 172.21.15.99:0/2669540179 wait complete. 2023-10-08T00:17:48.427 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 70 2023-10-08T00:17:48.438 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json status 2023-10-08T00:17:48.572 INFO:teuthology.orchestra.run.smithi191.stdout:{"cluster_fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","whoami":0,"id":7494,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":8.3181356439999998,"mdsmap_epoch":70,"osdmap_epoch":154,"osdmap_epoch_barrier":154,"uptime":13.439943028} 2023-10-08T00:17:48.572 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", "fs_name": "cephfs", "id": 7494, "mdsmap_epoch": 70, "osdmap_epoch": 154, "osdmap_epoch_barrier": 154, "rank_uptime": 8.318135644, "state": "up:active", "uptime": 13.439943028, "want_state": "up:active", "whoami": 0 } 2023-10-08T00:17:48.573 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2023-10-08T00:17:48.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.691+0000 7efed2e39700 1 Processor -- start 2023-10-08T00:17:48.692 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.691+0000 7efed2e39700 1 -- start start 2023-10-08T00:17:48.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.692+0000 7efed2e39700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7efecc159ef0 0x7efecc15a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:48.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.692+0000 7efed2e39700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efecc0b6a10 0x7efecc0b10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:48.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.692+0000 7efed2e39700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efecc0b1a60 con 0x7efecc0b15e0 2023-10-08T00:17:48.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.692+0000 7efed2e39700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efecc0b1d40 con 0x7efecc159ef0 2023-10-08T00:17:48.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.692+0000 7efed2e39700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efecc0b2020 con 0x7efecc0b6a10 2023-10-08T00:17:48.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.692+0000 7efed13d6700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7efecc0b15e0 0x7efecc0b46f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:60492/0 (socket says 172.21.15.99:60492) 2023-10-08T00:17:48.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.692+0000 7efed13d6700 1 -- 172.21.15.99:0/2161128213 learned_addr learned my addr 172.21.15.99:0/2161128213 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:17:48.696 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.692+0000 7efed0bd5700 1 --2- 172.21.15.99:0/2161128213 >> v2:172.21.15.191:3300/0 conn(0x7efecc159ef0 0x7efecc15a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:48.696 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.692+0000 7efecbfff700 1 --2- 172.21.15.99:0/2161128213 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efecc0b6a10 0x7efecc0b10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:48.696 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.692+0000 7efecb7fe700 1 -- 172.21.15.99:0/2161128213 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2409879046 0 0) 0x7efecc0b1a60 con 0x7efecc0b15e0 2023-10-08T00:17:48.696 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.692+0000 7efecb7fe700 1 -- 172.21.15.99:0/2161128213 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efeb4003570 con 0x7efecc0b15e0 2023-10-08T00:17:48.696 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.692+0000 7efecbfff700 1 -- 172.21.15.99:0/2161128213 >> v2:172.21.15.191:3300/0 conn(0x7efecc159ef0 msgr2=0x7efecc15a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:17:48.696 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.692+0000 7efecbfff700 1 --2- 172.21.15.99:0/2161128213 >> v2:172.21.15.191:3300/0 conn(0x7efecc159ef0 0x7efecc15a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:48.696 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.692+0000 7efecbfff700 1 -- 172.21.15.99:0/2161128213 >> v1:172.21.15.99:6789/0 conn(0x7efecc0b15e0 legacy=0x7efecc0b46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:48.696 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.692+0000 7efecbfff700 1 -- 172.21.15.99:0/2161128213 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efecc0b2300 con 0x7efecc0b6a10 2023-10-08T00:17:48.696 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.692+0000 7efed0bd5700 1 --2- 172.21.15.99:0/2161128213 >> v2:172.21.15.191:3300/0 conn(0x7efecc159ef0 0x7efecc15a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:17:48.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.692+0000 7efecbfff700 1 --2- 172.21.15.99:0/2161128213 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efecc0b6a10 0x7efecc0b10a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efec00098e0 tx=0x7efec00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10f6f3aac613437c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:48.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.693+0000 7efecb7fe700 1 -- 172.21.15.99:0/2161128213 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec0011630 con 0x7efecc0b6a10 2023-10-08T00:17:48.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.693+0000 7efecb7fe700 1 -- 172.21.15.99:0/2161128213 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efec0011790 con 0x7efecc0b6a10 2023-10-08T00:17:48.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.693+0000 7efecb7fe700 1 -- 172.21.15.99:0/2161128213 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec0011b30 con 0x7efecc0b6a10 2023-10-08T00:17:48.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.693+0000 7efed2e39700 1 -- 172.21.15.99:0/2161128213 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efecc0b6a10 msgr2=0x7efecc0b10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:48.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.693+0000 7efed2e39700 1 --2- 172.21.15.99:0/2161128213 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efecc0b6a10 0x7efecc0b10a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efec00098e0 tx=0x7efec00091a0 comp rx=0 tx=0).stop 2023-10-08T00:17:48.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.693+0000 7efed2e39700 1 -- 172.21.15.99:0/2161128213 shutdown_connections 2023-10-08T00:17:48.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.693+0000 7efed2e39700 1 --2- 172.21.15.99:0/2161128213 >> v2:172.21.15.191:3300/0 conn(0x7efecc159ef0 0x7efecc15a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:48.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.693+0000 7efed2e39700 1 --2- 172.21.15.99:0/2161128213 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efecc0b6a10 0x7efecc0b10a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:48.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.693+0000 7efed2e39700 1 -- 172.21.15.99:0/2161128213 >> 172.21.15.99:0/2161128213 conn(0x7efecc09a0a0 msgr2=0x7efecc067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:48.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.693+0000 7efed2e39700 1 -- 172.21.15.99:0/2161128213 shutdown_connections 2023-10-08T00:17:48.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.693+0000 7efed2e39700 1 -- 172.21.15.99:0/2161128213 wait complete. 2023-10-08T00:17:48.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.694+0000 7efed2e39700 1 Processor -- start 2023-10-08T00:17:48.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.695+0000 7efed2e39700 1 -- start start 2023-10-08T00:17:48.701 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.695+0000 7efed2e39700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efecc0b6a10 0x7efecc154110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:48.701 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.695+0000 7efed2e39700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efecc159ef0 0x7efecc0af700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:48.701 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.695+0000 7efed2e39700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efecc150300 0x7efecc150760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:48.701 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.695+0000 7efed2e39700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efecc153c80 con 0x7efecc0b6a10 2023-10-08T00:17:48.701 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.695+0000 7efed2e39700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efecc150ca0 con 0x7efecc159ef0 2023-10-08T00:17:48.701 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.695+0000 7efed2e39700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efecc153070 con 0x7efecc150300 2023-10-08T00:17:48.701 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.695+0000 7efed0bd5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efecc0b6a10 0x7efecc154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:48.702 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.695+0000 7efed0bd5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efecc0b6a10 0x7efecc154110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53228/0 (socket says 172.21.15.99:53228) 2023-10-08T00:17:48.702 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.695+0000 7efed0bd5700 1 -- 172.21.15.99:0/236581597 learned_addr learned my addr 172.21.15.99:0/236581597 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:17:48.702 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.695+0000 7efed13d6700 1 --2- 172.21.15.99:0/236581597 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efecc150300 0x7efecc150760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:48.702 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.695+0000 7efecbfff700 1 --2- 172.21.15.99:0/236581597 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efecc159ef0 0x7efecc0af700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:48.702 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.696+0000 7efed0bd5700 1 -- 172.21.15.99:0/236581597 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efecc150300 msgr2=0x7efecc150760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:48.702 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.696+0000 7efed0bd5700 1 --2- 172.21.15.99:0/236581597 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efecc150300 0x7efecc150760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:48.702 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.696+0000 7efed0bd5700 1 -- 172.21.15.99:0/236581597 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efecc159ef0 msgr2=0x7efecc0af700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:48.702 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.696+0000 7efed0bd5700 1 --2- 172.21.15.99:0/236581597 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efecc159ef0 0x7efecc0af700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:48.702 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.696+0000 7efed0bd5700 1 -- 172.21.15.99:0/236581597 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec4009a20 con 0x7efecc0b6a10 2023-10-08T00:17:48.703 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.696+0000 7efed0bd5700 1 --2- 172.21.15.99:0/236581597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efecc0b6a10 0x7efecc154110 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7efec400c170 tx=0x7efec400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:48.703 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.696+0000 7efec8ff9700 1 -- 172.21.15.99:0/236581597 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec400cc50 con 0x7efecc0b6a10 2023-10-08T00:17:48.703 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.696+0000 7efec8ff9700 1 -- 172.21.15.99:0/236581597 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efec400f040 con 0x7efecc0b6a10 2023-10-08T00:17:48.703 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.696+0000 7efec8ff9700 1 -- 172.21.15.99:0/236581597 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec4014480 con 0x7efecc0b6a10 2023-10-08T00:17:48.703 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.696+0000 7efed2e39700 1 -- 172.21.15.99:0/236581597 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efec0009420 con 0x7efecc0b6a10 2023-10-08T00:17:48.703 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.696+0000 7efed2e39700 1 -- 172.21.15.99:0/236581597 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efecc153630 con 0x7efecc0b6a10 2023-10-08T00:17:48.703 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.697+0000 7efed2e39700 1 -- 172.21.15.99:0/236581597 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efecc0b4d40 con 0x7efecc0b6a10 2023-10-08T00:17:48.703 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.698+0000 7efec8ff9700 1 -- 172.21.15.99:0/236581597 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efec40147f0 con 0x7efecc0b6a10 2023-10-08T00:17:48.703 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.698+0000 7efec8ff9700 1 --2- 172.21.15.99:0/236581597 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efeb8066f30 0x7efeb80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:17:48.703 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.698+0000 7efec8ff9700 1 -- 172.21.15.99:0/236581597 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5705+0+0 (secure 0 0 0) 0x7efec40853b0 con 0x7efecc0b6a10 2023-10-08T00:17:48.704 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.698+0000 7efecbfff700 1 --2- 172.21.15.99:0/236581597 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efeb8066f30 0x7efeb80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:17:48.704 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.698+0000 7efecbfff700 1 --2- 172.21.15.99:0/236581597 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efeb8066f30 0x7efeb80693e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7efec00091e0 tx=0x7efec0017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:17:48.704 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.702+0000 7efec8ff9700 1 -- 172.21.15.99:0/236581597 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efec40560b0 con 0x7efecc0b6a10 2023-10-08T00:17:48.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:48.884+0000 7efed2e39700 1 -- 172.21.15.99:0/236581597 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7efecc04c3c0 con 0x7efecc0b6a10 2023-10-08T00:17:49.286 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:49.285+0000 7efec8ff9700 1 -- 172.21.15.99:0/236581597 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v155) v1 ==== 181+0+0 (secure 0 0 0) 0x7efec4055c40 con 0x7efecc0b6a10 2023-10-08T00:17:49.288 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:49.287+0000 7efed2e39700 1 -- 172.21.15.99:0/236581597 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efeb8066f30 msgr2=0x7efeb80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:49.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:49.287+0000 7efed2e39700 1 --2- 172.21.15.99:0/236581597 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efeb8066f30 0x7efeb80693e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7efec00091e0 tx=0x7efec0017000 comp rx=0 tx=0).stop 2023-10-08T00:17:49.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:49.287+0000 7efed2e39700 1 -- 172.21.15.99:0/236581597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efecc0b6a10 msgr2=0x7efecc154110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:17:49.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:49.287+0000 7efed2e39700 1 --2- 172.21.15.99:0/236581597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efecc0b6a10 0x7efecc154110 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7efec400c170 tx=0x7efec400b040 comp rx=0 tx=0).stop 2023-10-08T00:17:49.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:49.287+0000 7efed2e39700 1 -- 172.21.15.99:0/236581597 shutdown_connections 2023-10-08T00:17:49.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:49.287+0000 7efed2e39700 1 --2- 172.21.15.99:0/236581597 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efeb8066f30 0x7efeb80693e0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:49.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:49.287+0000 7efed2e39700 1 --2- 172.21.15.99:0/236581597 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efecc150300 0x7efecc150760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:49.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:49.287+0000 7efed2e39700 1 --2- 172.21.15.99:0/236581597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efecc0b6a10 0x7efecc154110 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:49.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:49.287+0000 7efed2e39700 1 --2- 172.21.15.99:0/236581597 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efecc159ef0 0x7efecc0af700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:17:49.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:49.287+0000 7efed2e39700 1 -- 172.21.15.99:0/236581597 >> 172.21.15.99:0/236581597 conn(0x7efecc09a0a0 msgr2=0x7efecc15aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:17:49.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:49.288+0000 7efed2e39700 1 -- 172.21.15.99:0/236581597 shutdown_connections 2023-10-08T00:17:49.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:17:49.288+0000 7efed2e39700 1 -- 172.21.15.99:0/236581597 wait complete. 2023-10-08T00:17:49.290 INFO:teuthology.orchestra.run.smithi099.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2023-10-08T00:17:49.301 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-10-08T00:17:49.432 INFO:teuthology.orchestra.run.smithi099.stdout:{"osd_mon_report_interval":"5"} 2023-10-08T00:17:49.432 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-10-08T00:17:49.433 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2023-10-08T00:17:49.556 INFO:teuthology.orchestra.run.smithi099.stdout:{"mon_tick_interval":"5"} 2023-10-08T00:17:49.556 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2023-10-08T00:17:49.557 INFO:tasks.cephfs.test_full:pool capacity 33554432, 32MB should be enough to fill it 2023-10-08T00:17:49.557 DEBUG:teuthology.orchestra.run.smithi099:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-08T00:17:49.557 DEBUG:teuthology.orchestra.run.smithi099:> import time 2023-10-08T00:17:49.557 DEBUG:teuthology.orchestra.run.smithi099:> import datetime 2023-10-08T00:17:49.557 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2023-10-08T00:17:49.557 DEBUG:teuthology.orchestra.run.smithi099:> import os 2023-10-08T00:17:49.557 DEBUG:teuthology.orchestra.run.smithi099:> 2023-10-08T00:17:49.558 DEBUG:teuthology.orchestra.run.smithi099:> # Write some buffered data through before going full, all should be well 2023-10-08T00:17:49.558 DEBUG:teuthology.orchestra.run.smithi099:> print("writing some data through which we expect to succeed") 2023-10-08T00:17:49.558 DEBUG:teuthology.orchestra.run.smithi099:> bytes = 0 2023-10-08T00:17:49.558 DEBUG:teuthology.orchestra.run.smithi099:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2023-10-08T00:17:49.558 DEBUG:teuthology.orchestra.run.smithi099:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2023-10-08T00:17:49.558 DEBUG:teuthology.orchestra.run.smithi099:> bytes += os.write(f, b'"'"'a'"'"' * 512 * 1024) 2023-10-08T00:17:49.558 DEBUG:teuthology.orchestra.run.smithi099:> os.fsync(f) 2023-10-08T00:17:49.558 DEBUG:teuthology.orchestra.run.smithi099:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2023-10-08T00:17:49.558 DEBUG:teuthology.orchestra.run.smithi099:> 2023-10-08T00:17:49.558 DEBUG:teuthology.orchestra.run.smithi099:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2023-10-08T00:17:49.559 DEBUG:teuthology.orchestra.run.smithi099:> # writing until we see an error from fsync. As long as we'"'"'re doing 2023-10-08T00:17:49.559 DEBUG:teuthology.orchestra.run.smithi099:> # buffered IO, the error should always only appear from fsync and not 2023-10-08T00:17:49.559 DEBUG:teuthology.orchestra.run.smithi099:> # from write 2023-10-08T00:17:49.559 DEBUG:teuthology.orchestra.run.smithi099:> full = False 2023-10-08T00:17:49.559 DEBUG:teuthology.orchestra.run.smithi099:> 2023-10-08T00:17:49.559 DEBUG:teuthology.orchestra.run.smithi099:> for n in range(0, int(32 * 0.9)): 2023-10-08T00:17:49.559 DEBUG:teuthology.orchestra.run.smithi099:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2023-10-08T00:17:49.559 DEBUG:teuthology.orchestra.run.smithi099:> print("wrote {0} bytes via buffered write, may repeat".format(bytes)) 2023-10-08T00:17:49.559 DEBUG:teuthology.orchestra.run.smithi099:> print("done writing {0} bytes".format(bytes)) 2023-10-08T00:17:49.559 DEBUG:teuthology.orchestra.run.smithi099:> 2023-10-08T00:17:49.560 DEBUG:teuthology.orchestra.run.smithi099:> # OK, now we should sneak in under the full condition 2023-10-08T00:17:49.560 DEBUG:teuthology.orchestra.run.smithi099:> # due to the time it takes the OSDs to report to the 2023-10-08T00:17:49.560 DEBUG:teuthology.orchestra.run.smithi099:> # mons, and get a successful fsync on our full-making data 2023-10-08T00:17:49.560 DEBUG:teuthology.orchestra.run.smithi099:> os.fsync(f) 2023-10-08T00:17:49.560 DEBUG:teuthology.orchestra.run.smithi099:> print("successfully fsync'"'"'ed prior to getting full state reported") 2023-10-08T00:17:49.560 DEBUG:teuthology.orchestra.run.smithi099:> 2023-10-08T00:17:49.561 DEBUG:teuthology.orchestra.run.smithi099:> # buffered write, add more dirty data to the buffer 2023-10-08T00:17:49.561 DEBUG:teuthology.orchestra.run.smithi099:> print("starting buffered write") 2023-10-08T00:17:49.561 DEBUG:teuthology.orchestra.run.smithi099:> try: 2023-10-08T00:17:49.561 DEBUG:teuthology.orchestra.run.smithi099:> for n in range(0, int(32 * 0.2)): 2023-10-08T00:17:49.561 DEBUG:teuthology.orchestra.run.smithi099:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2023-10-08T00:17:49.562 DEBUG:teuthology.orchestra.run.smithi099:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2023-10-08T00:17:49.562 DEBUG:teuthology.orchestra.run.smithi099:> time.sleep(15.0) 2023-10-08T00:17:49.562 DEBUG:teuthology.orchestra.run.smithi099:> except OSError: 2023-10-08T00:17:49.562 DEBUG:teuthology.orchestra.run.smithi099:> pass; 2023-10-08T00:17:49.562 DEBUG:teuthology.orchestra.run.smithi099:> 2023-10-08T00:17:49.562 DEBUG:teuthology.orchestra.run.smithi099:> print("wrote, now waiting 30s and then doing a close we expect to fail") 2023-10-08T00:17:49.562 DEBUG:teuthology.orchestra.run.smithi099:> 2023-10-08T00:17:49.562 DEBUG:teuthology.orchestra.run.smithi099:> # Wait long enough for a background flush that should fail 2023-10-08T00:17:49.563 DEBUG:teuthology.orchestra.run.smithi099:> time.sleep(30) 2023-10-08T00:17:49.563 DEBUG:teuthology.orchestra.run.smithi099:> 2023-10-08T00:17:49.563 DEBUG:teuthology.orchestra.run.smithi099:> if False: 2023-10-08T00:17:49.563 DEBUG:teuthology.orchestra.run.smithi099:> # ...and check that the failed background flush is reflected in fclose 2023-10-08T00:17:49.563 DEBUG:teuthology.orchestra.run.smithi099:> try: 2023-10-08T00:17:49.563 DEBUG:teuthology.orchestra.run.smithi099:> os.close(f) 2023-10-08T00:17:49.563 DEBUG:teuthology.orchestra.run.smithi099:> except OSError: 2023-10-08T00:17:49.563 DEBUG:teuthology.orchestra.run.smithi099:> print("close() returned an error as expected") 2023-10-08T00:17:49.564 DEBUG:teuthology.orchestra.run.smithi099:> else: 2023-10-08T00:17:49.564 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("close() failed to raise error") 2023-10-08T00:17:49.564 DEBUG:teuthology.orchestra.run.smithi099:> else: 2023-10-08T00:17:49.564 DEBUG:teuthology.orchestra.run.smithi099:> # The kernel cephfs client does not raise errors on fclose 2023-10-08T00:17:49.564 DEBUG:teuthology.orchestra.run.smithi099:> os.close(f) 2023-10-08T00:17:49.564 DEBUG:teuthology.orchestra.run.smithi099:> 2023-10-08T00:17:49.564 DEBUG:teuthology.orchestra.run.smithi099:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2023-10-08T00:17:49.564 DEBUG:teuthology.orchestra.run.smithi099:> ' 2023-10-08T00:17:53.506 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:17:53.509 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:18:23.586 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:18:23.589 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:18:53.644 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:18:53.647 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:19:23.712 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:19:23.715 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:19:50.255 INFO:teuthology.orchestra.run.smithi099.stdout:writing some data through which we expect to succeed 2023-10-08T00:19:50.256 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed data successfully, will now attempt to fill fs 2023-10-08T00:19:50.256 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 1572864 bytes via buffered write, may repeat 2023-10-08T00:19:50.256 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 2621440 bytes via buffered write, may repeat 2023-10-08T00:19:50.256 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 3670016 bytes via buffered write, may repeat 2023-10-08T00:19:50.256 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 4718592 bytes via buffered write, may repeat 2023-10-08T00:19:50.256 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 5767168 bytes via buffered write, may repeat 2023-10-08T00:19:50.256 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 6815744 bytes via buffered write, may repeat 2023-10-08T00:19:50.256 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 7864320 bytes via buffered write, may repeat 2023-10-08T00:19:50.257 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 8912896 bytes via buffered write, may repeat 2023-10-08T00:19:50.257 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 9961472 bytes via buffered write, may repeat 2023-10-08T00:19:50.257 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 11010048 bytes via buffered write, may repeat 2023-10-08T00:19:50.257 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 12058624 bytes via buffered write, may repeat 2023-10-08T00:19:50.257 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 13107200 bytes via buffered write, may repeat 2023-10-08T00:19:50.257 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 14155776 bytes via buffered write, may repeat 2023-10-08T00:19:50.257 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 15204352 bytes via buffered write, may repeat 2023-10-08T00:19:50.257 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 16252928 bytes via buffered write, may repeat 2023-10-08T00:19:50.257 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 17301504 bytes via buffered write, may repeat 2023-10-08T00:19:50.257 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 18350080 bytes via buffered write, may repeat 2023-10-08T00:19:50.257 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 19398656 bytes via buffered write, may repeat 2023-10-08T00:19:50.258 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 20447232 bytes via buffered write, may repeat 2023-10-08T00:19:50.258 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 21495808 bytes via buffered write, may repeat 2023-10-08T00:19:50.258 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 22544384 bytes via buffered write, may repeat 2023-10-08T00:19:50.258 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 23592960 bytes via buffered write, may repeat 2023-10-08T00:19:50.258 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 24641536 bytes via buffered write, may repeat 2023-10-08T00:19:50.258 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 25690112 bytes via buffered write, may repeat 2023-10-08T00:19:50.258 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 26738688 bytes via buffered write, may repeat 2023-10-08T00:19:50.258 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 27787264 bytes via buffered write, may repeat 2023-10-08T00:19:50.258 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 28835840 bytes via buffered write, may repeat 2023-10-08T00:19:50.258 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 29884416 bytes via buffered write, may repeat 2023-10-08T00:19:50.259 INFO:teuthology.orchestra.run.smithi099.stdout:done writing 29884416 bytes 2023-10-08T00:19:50.259 INFO:teuthology.orchestra.run.smithi099.stdout:successfully fsync'ed prior to getting full state reported 2023-10-08T00:19:50.259 INFO:teuthology.orchestra.run.smithi099.stdout:starting buffered write 2023-10-08T00:19:50.259 INFO:teuthology.orchestra.run.smithi099.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-08T00:19:50.259 INFO:teuthology.orchestra.run.smithi099.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-08T00:19:50.259 INFO:teuthology.orchestra.run.smithi099.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-08T00:19:50.259 INFO:teuthology.orchestra.run.smithi099.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-08T00:19:50.259 INFO:teuthology.orchestra.run.smithi099.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-08T00:19:50.259 INFO:teuthology.orchestra.run.smithi099.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-08T00:19:50.259 INFO:teuthology.orchestra.run.smithi099.stdout:wrote, now waiting 30s and then doing a close we expect to fail 2023-10-08T00:19:50.382 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T00:19:50.382 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T00:19:50.426 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T00:19:50.472 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T00:19:50.473 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T00:19:50.473 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:19:50.473 DEBUG:teuthology.orchestra.run.smithi099:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-08T00:19:50.522 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2023-10-08T00:19:50.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:19:50.523 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T00:19:50.540 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-08T00:19:50.540 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:19:50.540 DEBUG:teuthology.orchestra.run.smithi191:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-08T00:19:50.584 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2023-10-08T00:19:50.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:19:50.585 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-08T00:19:50.641 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:19:50.763 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.763+0000 7fa658493700 1 Processor -- start 2023-10-08T00:19:50.763 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.763+0000 7fa658493700 1 -- start start 2023-10-08T00:19:50.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.764+0000 7fa658493700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa650157870 0x7fa65015b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:50.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.764+0000 7fa658493700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa65015a620 0x7fa65015d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:50.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.764+0000 7fa658493700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa65015cbb0 con 0x7fa65015d580 2023-10-08T00:19:50.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.764+0000 7fa658493700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa65015bfa0 con 0x7fa65015a620 2023-10-08T00:19:50.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.764+0000 7fa658493700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa65015c280 con 0x7fa650157870 2023-10-08T00:19:50.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.764+0000 7fa655a2e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa65015a620 0x7fa65015d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:50.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.764+0000 7fa65622f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa650157870 0x7fa65015b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:50.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.764+0000 7fa655a2e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa65015a620 0x7fa65015d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.99:58472/0 (socket says 172.21.15.99:58472) 2023-10-08T00:19:50.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.764+0000 7fa656a30700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7fa65015d580 0x7fa650158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:59872/0 (socket says 172.21.15.99:59872) 2023-10-08T00:19:50.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.764+0000 7fa655a2e700 1 -- 172.21.15.99:0/3520582265 learned_addr learned my addr 172.21.15.99:0/3520582265 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:19:50.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.764+0000 7fa65622f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa650157870 0x7fa65015b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.99:51280/0 (socket says 172.21.15.99:51280) 2023-10-08T00:19:50.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.764+0000 7fa65622f700 1 -- 172.21.15.99:0/3520582265 >> v2:172.21.15.191:3300/0 conn(0x7fa65015a620 msgr2=0x7fa65015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:19:50.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.764+0000 7fa65622f700 1 --2- 172.21.15.99:0/3520582265 >> v2:172.21.15.191:3300/0 conn(0x7fa65015a620 0x7fa65015d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:50.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.764+0000 7fa65622f700 1 -- 172.21.15.99:0/3520582265 >> v1:172.21.15.191:6789/0 conn(0x7fa65015d580 legacy=0x7fa650158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:50.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.764+0000 7fa65622f700 1 -- 172.21.15.99:0/3520582265 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa65015c560 con 0x7fa650157870 2023-10-08T00:19:50.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.764+0000 7fa65522d700 1 -- 172.21.15.99:0/3520582265 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 604881941 0 0) 0x7fa65015cbb0 con 0x7fa65015d580 2023-10-08T00:19:50.768 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.764+0000 7fa655a2e700 1 --2- 172.21.15.99:0/3520582265 >> v2:172.21.15.191:3300/0 conn(0x7fa65015a620 0x7fa65015d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:19:50.768 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.765+0000 7fa65622f700 1 --2- 172.21.15.99:0/3520582265 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa650157870 0x7fa65015b6a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa64c0098e0 tx=0x7fa64c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6f097e0ff46d2e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:50.768 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.765+0000 7fa65522d700 1 -- 172.21.15.99:0/3520582265 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64c00b640 con 0x7fa650157870 2023-10-08T00:19:50.768 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.765+0000 7fa65522d700 1 -- 172.21.15.99:0/3520582265 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa64c00b7a0 con 0x7fa650157870 2023-10-08T00:19:50.768 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.765+0000 7fa65522d700 1 -- 172.21.15.99:0/3520582265 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64c00bb40 con 0x7fa650157870 2023-10-08T00:19:50.768 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.765+0000 7fa658493700 1 -- 172.21.15.99:0/3520582265 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa650157870 msgr2=0x7fa65015b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:50.768 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.765+0000 7fa658493700 1 --2- 172.21.15.99:0/3520582265 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa650157870 0x7fa65015b6a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa64c0098e0 tx=0x7fa64c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:19:50.768 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.765+0000 7fa658493700 1 -- 172.21.15.99:0/3520582265 shutdown_connections 2023-10-08T00:19:50.768 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.765+0000 7fa658493700 1 --2- 172.21.15.99:0/3520582265 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa650157870 0x7fa65015b6a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:50.768 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.765+0000 7fa658493700 1 --2- 172.21.15.99:0/3520582265 >> v2:172.21.15.191:3300/0 conn(0x7fa65015a620 0x7fa65015d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:50.768 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.765+0000 7fa658493700 1 -- 172.21.15.99:0/3520582265 >> 172.21.15.99:0/3520582265 conn(0x7fa650068ce0 msgr2=0x7fa6500690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:19:50.769 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.765+0000 7fa658493700 1 -- 172.21.15.99:0/3520582265 shutdown_connections 2023-10-08T00:19:50.769 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.765+0000 7fa658493700 1 -- 172.21.15.99:0/3520582265 wait complete. 2023-10-08T00:19:50.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.766+0000 7fa658493700 1 Processor -- start 2023-10-08T00:19:50.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.767+0000 7fa658493700 1 -- start start 2023-10-08T00:19:50.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.767+0000 7fa658493700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa650157870 0x7fa65014df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:50.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.767+0000 7fa658493700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa65015a620 0x7fa6500b5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:50.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.767+0000 7fa658493700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa65015d580 0x7fa6500b57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:50.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.767+0000 7fa658493700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6500d0060 con 0x7fa650157870 2023-10-08T00:19:50.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.767+0000 7fa658493700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6500cf450 con 0x7fa65015d580 2023-10-08T00:19:50.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.767+0000 7fa658493700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6500cf730 con 0x7fa65015a620 2023-10-08T00:19:50.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.767+0000 7fa65622f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa650157870 0x7fa65014df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:50.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.767+0000 7fa65622f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa650157870 0x7fa65014df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44236/0 (socket says 172.21.15.99:44236) 2023-10-08T00:19:50.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.767+0000 7fa65622f700 1 -- 172.21.15.99:0/795014758 learned_addr learned my addr 172.21.15.99:0/795014758 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:19:50.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.767+0000 7fa656a30700 1 --2- 172.21.15.99:0/795014758 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa65015d580 0x7fa6500b57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:50.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.767+0000 7fa655a2e700 1 --2- 172.21.15.99:0/795014758 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa65015a620 0x7fa6500b5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:50.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.768+0000 7fa65622f700 1 -- 172.21.15.99:0/795014758 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa65015a620 msgr2=0x7fa6500b5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:50.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.768+0000 7fa65622f700 1 --2- 172.21.15.99:0/795014758 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa65015a620 0x7fa6500b5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:50.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.768+0000 7fa65622f700 1 -- 172.21.15.99:0/795014758 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa65015d580 msgr2=0x7fa6500b57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:50.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.768+0000 7fa65622f700 1 --2- 172.21.15.99:0/795014758 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa65015d580 0x7fa6500b57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:50.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.768+0000 7fa65622f700 1 -- 172.21.15.99:0/795014758 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6400094f0 con 0x7fa650157870 2023-10-08T00:19:50.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.768+0000 7fa65622f700 1 --2- 172.21.15.99:0/795014758 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa650157870 0x7fa65014df60 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fa65015c3c0 tx=0x7fa64c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:50.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.768+0000 7fa6467fc700 1 -- 172.21.15.99:0/795014758 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64c00bb20 con 0x7fa650157870 2023-10-08T00:19:50.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.768+0000 7fa6467fc700 1 -- 172.21.15.99:0/795014758 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa64c000c90 con 0x7fa650157870 2023-10-08T00:19:50.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.768+0000 7fa6467fc700 1 -- 172.21.15.99:0/795014758 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64c011650 con 0x7fa650157870 2023-10-08T00:19:50.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.768+0000 7fa658493700 1 -- 172.21.15.99:0/795014758 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa64c009420 con 0x7fa650157870 2023-10-08T00:19:50.774 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.768+0000 7fa658493700 1 -- 172.21.15.99:0/795014758 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6500cfcf0 con 0x7fa650157870 2023-10-08T00:19:50.774 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.769+0000 7fa658493700 1 -- 172.21.15.99:0/795014758 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa65004c3c0 con 0x7fa650157870 2023-10-08T00:19:50.774 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.769+0000 7fa6467fc700 1 -- 172.21.15.99:0/795014758 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa64c011a00 con 0x7fa650157870 2023-10-08T00:19:50.774 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.770+0000 7fa6467fc700 1 --2- 172.21.15.99:0/795014758 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa634066f30 0x7fa6340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:50.774 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.770+0000 7fa6467fc700 1 -- 172.21.15.99:0/795014758 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa64c085c40 con 0x7fa650157870 2023-10-08T00:19:50.774 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.770+0000 7fa6467fc700 1 -- 172.21.15.99:0/795014758 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=157}) v3 -- 0x7fa63406ea50 con 0x7fa650157870 2023-10-08T00:19:50.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.770+0000 7fa655a2e700 1 --2- 172.21.15.99:0/795014758 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa634066f30 0x7fa6340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:50.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.770+0000 7fa655a2e700 1 --2- 172.21.15.99:0/795014758 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa634066f30 0x7fa6340693e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa6500d1730 tx=0x7fa640009250 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:50.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.774+0000 7fa6467fc700 1 -- 172.21.15.99:0/795014758 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa64c015060 con 0x7fa650157870 2023-10-08T00:19:50.997 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.997+0000 7fa658493700 1 -- 172.21.15.99:0/795014758 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa65015c840 con 0x7fa650157870 2023-10-08T00:19:50.998 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.997+0000 7fa6467fc700 1 -- 172.21.15.99:0/795014758 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4599 (secure 0 0 0) 0x7fa64c029020 con 0x7fa650157870 2023-10-08T00:19:50.998 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:19:50.999 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":70,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6056,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/3025449992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3025449992},{"type":"v1","addr":"172.21.15.191:6836","nonce":3025449992}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":7476,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/86350138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":86350138},{"type":"v1","addr":"172.21.15.99:6834","nonce":86350138}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":7479,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1282887036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1282887036},{"type":"v1","addr":"172.21.15.99:6835","nonce":1282887036}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:17:39.623734+0000","modified":"2023-10-08T00:17:43.252633+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7494":{"gid":7494,"name":"d","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/1165162515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":1165162515},{"type":"v1","addr":"172.21.15.191:6837","nonce":1165162515}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-08T00:19:51.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.999+0000 7fa658493700 1 -- 172.21.15.99:0/795014758 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa634066f30 msgr2=0x7fa6340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:51.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.999+0000 7fa658493700 1 --2- 172.21.15.99:0/795014758 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa634066f30 0x7fa6340693e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa6500d1730 tx=0x7fa640009250 comp rx=0 tx=0).stop 2023-10-08T00:19:51.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.999+0000 7fa658493700 1 -- 172.21.15.99:0/795014758 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa650157870 msgr2=0x7fa65014df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:51.001 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.999+0000 7fa658493700 1 --2- 172.21.15.99:0/795014758 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa650157870 0x7fa65014df60 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fa65015c3c0 tx=0x7fa64c017000 comp rx=0 tx=0).stop 2023-10-08T00:19:51.001 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.999+0000 7fa658493700 1 -- 172.21.15.99:0/795014758 shutdown_connections 2023-10-08T00:19:51.001 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.999+0000 7fa658493700 1 --2- 172.21.15.99:0/795014758 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa634066f30 0x7fa6340693e0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:51.001 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.999+0000 7fa658493700 1 --2- 172.21.15.99:0/795014758 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa65015d580 0x7fa6500b57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:51.001 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.999+0000 7fa658493700 1 --2- 172.21.15.99:0/795014758 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa650157870 0x7fa65014df60 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:51.001 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.999+0000 7fa658493700 1 --2- 172.21.15.99:0/795014758 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa65015a620 0x7fa6500b5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:51.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.999+0000 7fa658493700 1 -- 172.21.15.99:0/795014758 >> 172.21.15.99:0/795014758 conn(0x7fa650068ce0 msgr2=0x7fa650159160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:19:51.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.999+0000 7fa658493700 1 -- 172.21.15.99:0/795014758 shutdown_connections 2023-10-08T00:19:51.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:50.999+0000 7fa658493700 1 -- 172.21.15.99:0/795014758 wait complete. 2023-10-08T00:19:51.002 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 70 2023-10-08T00:19:51.012 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:19:51.127 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.127+0000 7fd907e29700 1 Processor -- start 2023-10-08T00:19:51.128 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.127+0000 7fd907e29700 1 -- start start 2023-10-08T00:19:51.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.128+0000 7fd907e29700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd900157c40 0x7fd900158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:51.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.128+0000 7fd907e29700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd90015cbb0 con 0x7fd900157870 2023-10-08T00:19:51.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.128+0000 7fd907e29700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd90015bfa0 con 0x7fd90015a620 2023-10-08T00:19:51.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.128+0000 7fd907e29700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd90015bdb0 con 0x7fd900157c40 2023-10-08T00:19:51.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.128+0000 7fd9063c6700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd900157c40 0x7fd900158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:51.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.128+0000 7fd905bc5700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fd900157870 0x7fd90015b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:38526/0 (socket says 172.21.15.99:38526) 2023-10-08T00:19:51.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.128+0000 7fd9063c6700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd900157c40 0x7fd900158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44242/0 (socket says 172.21.15.99:44242) 2023-10-08T00:19:51.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.128+0000 7fd905bc5700 1 -- 172.21.15.99:0/589657442 learned_addr learned my addr 172.21.15.99:0/589657442 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:19:51.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.128+0000 7fd9063c6700 1 -- 172.21.15.99:0/589657442 >> v1:172.21.15.191:6789/0 conn(0x7fd90015a620 legacy=0x7fd90015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:51.131 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.128+0000 7fd9063c6700 1 -- 172.21.15.99:0/589657442 >> v1:172.21.15.99:6789/0 conn(0x7fd900157870 legacy=0x7fd90015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:51.131 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.128+0000 7fd9063c6700 1 -- 172.21.15.99:0/589657442 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd90015c280 con 0x7fd900157c40 2023-10-08T00:19:51.131 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.128+0000 7fd904bc3700 1 -- 172.21.15.99:0/589657442 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2829131735 0 0) 0x7fd90015cbb0 con 0x7fd900157870 2023-10-08T00:19:51.131 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.128+0000 7fd9063c6700 1 --2- 172.21.15.99:0/589657442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd900157c40 0x7fd900158110 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fd8f8009e60 tx=0x7fd8f8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=138d42bbeffbd40d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:51.131 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.129+0000 7fd904bc3700 1 -- 172.21.15.99:0/589657442 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f800e070 con 0x7fd900157c40 2023-10-08T00:19:51.131 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.129+0000 7fd904bc3700 1 -- 172.21.15.99:0/589657442 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f800bc30 con 0x7fd900157c40 2023-10-08T00:19:51.131 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.129+0000 7fd904bc3700 1 -- 172.21.15.99:0/589657442 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f8012430 con 0x7fd900157c40 2023-10-08T00:19:51.131 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.129+0000 7fd907e29700 1 -- 172.21.15.99:0/589657442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd900157c40 msgr2=0x7fd900158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:51.131 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.129+0000 7fd907e29700 1 --2- 172.21.15.99:0/589657442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd900157c40 0x7fd900158110 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fd8f8009e60 tx=0x7fd8f8009780 comp rx=0 tx=0).stop 2023-10-08T00:19:51.131 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.129+0000 7fd907e29700 1 -- 172.21.15.99:0/589657442 shutdown_connections 2023-10-08T00:19:51.131 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.129+0000 7fd907e29700 1 --2- 172.21.15.99:0/589657442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd900157c40 0x7fd900158110 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:51.132 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.129+0000 7fd907e29700 1 -- 172.21.15.99:0/589657442 >> 172.21.15.99:0/589657442 conn(0x7fd900068ce0 msgr2=0x7fd9000690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:19:51.132 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.129+0000 7fd907e29700 1 -- 172.21.15.99:0/589657442 shutdown_connections 2023-10-08T00:19:51.132 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.129+0000 7fd907e29700 1 -- 172.21.15.99:0/589657442 wait complete. 2023-10-08T00:19:51.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.130+0000 7fd907e29700 1 Processor -- start 2023-10-08T00:19:51.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.130+0000 7fd907e29700 1 -- start start 2023-10-08T00:19:51.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.131+0000 7fd907e29700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd900157870 0x7fd90013f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:51.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.131+0000 7fd907e29700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd900157c40 0x7fd90016a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:51.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.131+0000 7fd907e29700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd90015a620 0x7fd90016aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:51.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.131+0000 7fd907e29700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd90016af40 con 0x7fd900157870 2023-10-08T00:19:51.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.131+0000 7fd907e29700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9000b1840 con 0x7fd90015a620 2023-10-08T00:19:51.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.131+0000 7fd907e29700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd90013f6e0 con 0x7fd900157c40 2023-10-08T00:19:51.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.131+0000 7fd905bc5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd900157870 0x7fd90013f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:51.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.131+0000 7fd905bc5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd900157870 0x7fd90013f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44256/0 (socket says 172.21.15.99:44256) 2023-10-08T00:19:51.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.131+0000 7fd905bc5700 1 -- 172.21.15.99:0/4012862269 learned_addr learned my addr 172.21.15.99:0/4012862269 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:19:51.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.131+0000 7fd9063c6700 1 --2- 172.21.15.99:0/4012862269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd90015a620 0x7fd90016aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:51.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.131+0000 7fd9053c4700 1 --2- 172.21.15.99:0/4012862269 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd900157c40 0x7fd90016a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:51.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.131+0000 7fd905bc5700 1 -- 172.21.15.99:0/4012862269 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd900157c40 msgr2=0x7fd90016a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:51.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.131+0000 7fd905bc5700 1 --2- 172.21.15.99:0/4012862269 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd900157c40 0x7fd90016a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:51.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.131+0000 7fd905bc5700 1 -- 172.21.15.99:0/4012862269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd90015a620 msgr2=0x7fd90016aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:51.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.131+0000 7fd905bc5700 1 --2- 172.21.15.99:0/4012862269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd90015a620 0x7fd90016aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:51.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.131+0000 7fd905bc5700 1 -- 172.21.15.99:0/4012862269 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f8009a20 con 0x7fd900157870 2023-10-08T00:19:51.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.131+0000 7fd9063c6700 1 --2- 172.21.15.99:0/4012862269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd90015a620 0x7fd90016aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:19:51.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.132+0000 7fd905bc5700 1 --2- 172.21.15.99:0/4012862269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd900157870 0x7fd90013f1a0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc001d10 tx=0x7fd8fc007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:51.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.132+0000 7fd8f5ffb700 1 -- 172.21.15.99:0/4012862269 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc017070 con 0x7fd900157870 2023-10-08T00:19:51.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.132+0000 7fd8f5ffb700 1 -- 172.21.15.99:0/4012862269 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8fc014bf0 con 0x7fd900157870 2023-10-08T00:19:51.137 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.132+0000 7fd907e29700 1 -- 172.21.15.99:0/4012862269 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd900153a80 con 0x7fd900157870 2023-10-08T00:19:51.137 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.132+0000 7fd8f5ffb700 1 -- 172.21.15.99:0/4012862269 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc005900 con 0x7fd900157870 2023-10-08T00:19:51.137 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.132+0000 7fd907e29700 1 -- 172.21.15.99:0/4012862269 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd900154040 con 0x7fd900157870 2023-10-08T00:19:51.137 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.133+0000 7fd907e29700 1 -- 172.21.15.99:0/4012862269 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd90004c3c0 con 0x7fd900157870 2023-10-08T00:19:51.137 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.133+0000 7fd8f5ffb700 1 -- 172.21.15.99:0/4012862269 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd8fc005cb0 con 0x7fd900157870 2023-10-08T00:19:51.137 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.134+0000 7fd8f5ffb700 1 --2- 172.21.15.99:0/4012862269 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd8e4066f30 0x7fd8e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:51.137 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.134+0000 7fd8f5ffb700 1 -- 172.21.15.99:0/4012862269 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd8fc08c830 con 0x7fd900157870 2023-10-08T00:19:51.137 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.134+0000 7fd8f5ffb700 1 -- 172.21.15.99:0/4012862269 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=157}) v3 -- 0x7fd8e406ea50 con 0x7fd900157870 2023-10-08T00:19:51.137 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.134+0000 7fd9053c4700 1 --2- 172.21.15.99:0/4012862269 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd8e4066f30 0x7fd8e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:51.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.134+0000 7fd9053c4700 1 --2- 172.21.15.99:0/4012862269 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd8e4066f30 0x7fd8e40693e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fd900156bd0 tx=0x7fd8f0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:51.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.138+0000 7fd8f5ffb700 1 -- 172.21.15.99:0/4012862269 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd8fc08c070 con 0x7fd900157870 2023-10-08T00:19:51.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.326+0000 7fd907e29700 1 -- 172.21.15.99:0/4012862269 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd90015c560 con 0x7fd900157870 2023-10-08T00:19:51.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.327+0000 7fd8f5ffb700 1 -- 172.21.15.99:0/4012862269 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) v1 ==== 74+0+15391 (secure 0 0 0) 0x7fd8fc059b20 con 0x7fd900157870 2023-10-08T00:19:51.328 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:19:51.328 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":156,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:19:11.138430+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:17:37.936880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-08T00:17:38.577281+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:19:51.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.329+0000 7fd907e29700 1 -- 172.21.15.99:0/4012862269 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd8e4066f30 msgr2=0x7fd8e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:51.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.329+0000 7fd907e29700 1 --2- 172.21.15.99:0/4012862269 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd8e4066f30 0x7fd8e40693e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fd900156bd0 tx=0x7fd8f0009040 comp rx=0 tx=0).stop 2023-10-08T00:19:51.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.329+0000 7fd907e29700 1 -- 172.21.15.99:0/4012862269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd900157870 msgr2=0x7fd90013f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:51.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.329+0000 7fd907e29700 1 --2- 172.21.15.99:0/4012862269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd900157870 0x7fd90013f1a0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc001d10 tx=0x7fd8fc007e80 comp rx=0 tx=0).stop 2023-10-08T00:19:51.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.329+0000 7fd907e29700 1 -- 172.21.15.99:0/4012862269 shutdown_connections 2023-10-08T00:19:51.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.329+0000 7fd907e29700 1 --2- 172.21.15.99:0/4012862269 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd8e4066f30 0x7fd8e40693e0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:51.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.329+0000 7fd907e29700 1 --2- 172.21.15.99:0/4012862269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd90015a620 0x7fd90016aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:51.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.329+0000 7fd907e29700 1 --2- 172.21.15.99:0/4012862269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd900157870 0x7fd90013f1a0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:51.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.329+0000 7fd907e29700 1 --2- 172.21.15.99:0/4012862269 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd900157c40 0x7fd90016a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:51.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.329+0000 7fd907e29700 1 -- 172.21.15.99:0/4012862269 >> 172.21.15.99:0/4012862269 conn(0x7fd900068ce0 msgr2=0x7fd9000690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:19:51.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.329+0000 7fd907e29700 1 -- 172.21.15.99:0/4012862269 shutdown_connections 2023-10-08T00:19:51.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.329+0000 7fd907e29700 1 -- 172.21.15.99:0/4012862269 wait complete. 2023-10-08T00:19:51.342 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:10:42.063344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T00:17:37.936880+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '153', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T00:17:38.577281+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '156', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T00:19:51.342 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T00:19:51.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.457+0000 7f728b59e700 1 Processor -- start 2023-10-08T00:19:51.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.458+0000 7f728b59e700 1 -- start start 2023-10-08T00:19:51.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.458+0000 7f728b59e700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f728415a290 0x7f728406a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:51.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.458+0000 7f728b59e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f728406aae0 0x7f72840cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:51.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.458+0000 7f728b59e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72840b0790 con 0x7f7284159ec0 2023-10-08T00:19:51.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.458+0000 7f728b59e700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f728406af60 con 0x7f728415a290 2023-10-08T00:19:51.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.458+0000 7f728b59e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72840ce1f0 con 0x7f728406aae0 2023-10-08T00:19:51.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.459+0000 7f7288b39700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f728415a290 0x7f728406a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:51.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.459+0000 7f7288b39700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f728415a290 0x7f728406a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44262/0 (socket says 172.21.15.99:44262) 2023-10-08T00:19:51.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.459+0000 7f7288b39700 1 -- 172.21.15.99:0/876595573 learned_addr learned my addr 172.21.15.99:0/876595573 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:19:51.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.459+0000 7f7289b3b700 1 --2- 172.21.15.99:0/876595573 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f728406aae0 0x7f72840cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:51.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.459+0000 7f7288b39700 1 -- 172.21.15.99:0/876595573 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f728406aae0 msgr2=0x7f72840cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:19:51.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.459+0000 7f7288b39700 1 --2- 172.21.15.99:0/876595573 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f728406aae0 0x7f72840cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:51.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.459+0000 7f7288b39700 1 -- 172.21.15.99:0/876595573 >> v1:172.21.15.191:6789/0 conn(0x7f7284159ec0 legacy=0x7f72840af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:51.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.459+0000 7f7288b39700 1 -- 172.21.15.99:0/876595573 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72840b0b20 con 0x7f728415a290 2023-10-08T00:19:51.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.459+0000 7f7289b3b700 1 --2- 172.21.15.99:0/876595573 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f728406aae0 0x7f72840cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:19:51.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.459+0000 7f7288b39700 1 --2- 172.21.15.99:0/876595573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f728415a290 0x7f728406a5a0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f72740099b0 tx=0x7f7274009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb67f298fd33cc5c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:51.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.459+0000 7f727bfff700 1 -- 172.21.15.99:0/876595573 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727400b860 con 0x7f728415a290 2023-10-08T00:19:51.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.459+0000 7f727bfff700 1 -- 172.21.15.99:0/876595573 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f727400b9c0 con 0x7f728415a290 2023-10-08T00:19:51.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.459+0000 7f727bfff700 1 -- 172.21.15.99:0/876595573 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727400bd60 con 0x7f728415a290 2023-10-08T00:19:51.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.460+0000 7f728b59e700 1 -- 172.21.15.99:0/876595573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f728415a290 msgr2=0x7f728406a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:51.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.460+0000 7f728b59e700 1 --2- 172.21.15.99:0/876595573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f728415a290 0x7f728406a5a0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f72740099b0 tx=0x7f7274009250 comp rx=0 tx=0).stop 2023-10-08T00:19:51.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.460+0000 7f728b59e700 1 -- 172.21.15.99:0/876595573 shutdown_connections 2023-10-08T00:19:51.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.460+0000 7f728b59e700 1 --2- 172.21.15.99:0/876595573 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f728406aae0 0x7f72840cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:51.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.460+0000 7f728b59e700 1 --2- 172.21.15.99:0/876595573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f728415a290 0x7f728406a5a0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:51.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.460+0000 7f728b59e700 1 -- 172.21.15.99:0/876595573 >> 172.21.15.99:0/876595573 conn(0x7f72840d1880 msgr2=0x7f72840cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:19:51.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.460+0000 7f728b59e700 1 -- 172.21.15.99:0/876595573 shutdown_connections 2023-10-08T00:19:51.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.460+0000 7f728b59e700 1 -- 172.21.15.99:0/876595573 wait complete. 2023-10-08T00:19:51.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.461+0000 7f728b59e700 1 Processor -- start 2023-10-08T00:19:51.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.461+0000 7f728b59e700 1 -- start start 2023-10-08T00:19:51.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.462+0000 7f728b59e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f728406aae0 0x7f728415dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:51.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.462+0000 7f728b59e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7284159ec0 0x7f728416a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:51.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.462+0000 7f728b59e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f728415a290 0x7f728416a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:51.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.462+0000 7f728b59e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f728415c8c0 con 0x7f7284159ec0 2023-10-08T00:19:51.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.462+0000 7f728b59e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f728415b2a0 con 0x7f728406aae0 2023-10-08T00:19:51.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.462+0000 7f728b59e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f728415bc40 con 0x7f728415a290 2023-10-08T00:19:51.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.462+0000 7f7288b39700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7284159ec0 0x7f728416a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:51.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.462+0000 7f7288b39700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7284159ec0 0x7f728416a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44264/0 (socket says 172.21.15.99:44264) 2023-10-08T00:19:51.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.462+0000 7f7288b39700 1 -- 172.21.15.99:0/1796705935 learned_addr learned my addr 172.21.15.99:0/1796705935 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:19:51.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.462+0000 7f7289b3b700 1 --2- 172.21.15.99:0/1796705935 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f728415a290 0x7f728416a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:51.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.462+0000 7f728933a700 1 --2- 172.21.15.99:0/1796705935 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f728406aae0 0x7f728415dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:51.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.462+0000 7f7288b39700 1 -- 172.21.15.99:0/1796705935 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f728415a290 msgr2=0x7f728416a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:51.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.462+0000 7f7288b39700 1 --2- 172.21.15.99:0/1796705935 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f728415a290 0x7f728416a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:51.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.462+0000 7f7288b39700 1 -- 172.21.15.99:0/1796705935 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f728406aae0 msgr2=0x7f728415dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:51.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.462+0000 7f7288b39700 1 --2- 172.21.15.99:0/1796705935 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f728406aae0 0x7f728415dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:51.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.462+0000 7f7288b39700 1 -- 172.21.15.99:0/1796705935 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72740094f0 con 0x7f7284159ec0 2023-10-08T00:19:51.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.462+0000 7f728933a700 1 --2- 172.21.15.99:0/1796705935 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f728406aae0 0x7f728415dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:19:51.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.462+0000 7f7288b39700 1 --2- 172.21.15.99:0/1796705935 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7284159ec0 0x7f728416a3f0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f7274009290 tx=0x7f7274017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:51.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.462+0000 7f72797fa700 1 -- 172.21.15.99:0/1796705935 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727401a450 con 0x7f7284159ec0 2023-10-08T00:19:51.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.462+0000 7f72797fa700 1 -- 172.21.15.99:0/1796705935 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f727401a5d0 con 0x7f7284159ec0 2023-10-08T00:19:51.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.463+0000 7f728b59e700 1 -- 172.21.15.99:0/1796705935 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f728415bf20 con 0x7f7284159ec0 2023-10-08T00:19:51.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.463+0000 7f72797fa700 1 -- 172.21.15.99:0/1796705935 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727401a990 con 0x7f7284159ec0 2023-10-08T00:19:51.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.463+0000 7f728b59e700 1 -- 172.21.15.99:0/1796705935 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f728415c4e0 con 0x7f7284159ec0 2023-10-08T00:19:51.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.463+0000 7f728b59e700 1 -- 172.21.15.99:0/1796705935 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f728404c3c0 con 0x7f7284159ec0 2023-10-08T00:19:51.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.464+0000 7f72797fa700 1 -- 172.21.15.99:0/1796705935 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f727401ad40 con 0x7f7284159ec0 2023-10-08T00:19:51.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.464+0000 7f72797fa700 1 --2- 172.21.15.99:0/1796705935 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7268066f30 0x7f72680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:51.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.464+0000 7f72797fa700 1 -- 172.21.15.99:0/1796705935 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7274086410 con 0x7f7284159ec0 2023-10-08T00:19:51.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.464+0000 7f72797fa700 1 -- 172.21.15.99:0/1796705935 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=157}) v3 -- 0x7f726806ea50 con 0x7f7284159ec0 2023-10-08T00:19:51.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.464+0000 7f728933a700 1 --2- 172.21.15.99:0/1796705935 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7268066f30 0x7f72680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:51.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.465+0000 7f728933a700 1 --2- 172.21.15.99:0/1796705935 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7268066f30 0x7f72680693e0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7280007fd0 tx=0x7f7280009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:51.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.469+0000 7f72797fa700 1 -- 172.21.15.99:0/1796705935 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7274053700 con 0x7f7284159ec0 2023-10-08T00:19:51.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.652+0000 7f728b59e700 1 -- 172.21.15.99:0/1796705935 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f72840b04b0 con 0x7f7284159ec0 2023-10-08T00:19:51.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.653+0000 7f72797fa700 1 -- 172.21.15.99:0/1796705935 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v156) v1 ==== 93+0+10 (secure 0 0 0) 0x7f72740150a0 con 0x7f7284159ec0 2023-10-08T00:19:51.654 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2023-10-08T00:19:51.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.655+0000 7f728b59e700 1 -- 172.21.15.99:0/1796705935 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7268066f30 msgr2=0x7f72680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:51.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.655+0000 7f728b59e700 1 --2- 172.21.15.99:0/1796705935 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7268066f30 0x7f72680693e0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7280007fd0 tx=0x7f7280009040 comp rx=0 tx=0).stop 2023-10-08T00:19:51.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.655+0000 7f728b59e700 1 -- 172.21.15.99:0/1796705935 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7284159ec0 msgr2=0x7f728416a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:51.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.655+0000 7f728b59e700 1 --2- 172.21.15.99:0/1796705935 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7284159ec0 0x7f728416a3f0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f7274009290 tx=0x7f7274017000 comp rx=0 tx=0).stop 2023-10-08T00:19:51.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.655+0000 7f728b59e700 1 -- 172.21.15.99:0/1796705935 shutdown_connections 2023-10-08T00:19:51.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.655+0000 7f728b59e700 1 --2- 172.21.15.99:0/1796705935 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7268066f30 0x7f72680693e0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:51.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.655+0000 7f728b59e700 1 --2- 172.21.15.99:0/1796705935 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f728406aae0 0x7f728415dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:51.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.655+0000 7f728b59e700 1 --2- 172.21.15.99:0/1796705935 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f728415a290 0x7f728416a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:51.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.655+0000 7f728b59e700 1 --2- 172.21.15.99:0/1796705935 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7284159ec0 0x7f728416a3f0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:51.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.655+0000 7f728b59e700 1 -- 172.21.15.99:0/1796705935 >> 172.21.15.99:0/1796705935 conn(0x7f72840d1880 msgr2=0x7f72840b1630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:19:51.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.655+0000 7f728b59e700 1 -- 172.21.15.99:0/1796705935 shutdown_connections 2023-10-08T00:19:51.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.655+0000 7f728b59e700 1 -- 172.21.15.99:0/1796705935 wait complete. 2023-10-08T00:19:51.668 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T00:19:51.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.784+0000 7f2620306700 1 Processor -- start 2023-10-08T00:19:51.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.784+0000 7f2620306700 1 -- start start 2023-10-08T00:19:51.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.785+0000 7f2620306700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2618159ec0 0x7f26180af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:51.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.785+0000 7f2620306700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f261806acb0 0x7f26180afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:51.788 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.785+0000 7f2620306700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26180b0790 con 0x7f261815a290 2023-10-08T00:19:51.788 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.785+0000 7f2620306700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26180ce1f0 con 0x7f2618159ec0 2023-10-08T00:19:51.788 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.785+0000 7f2620306700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26180b0b20 con 0x7f261806acb0 2023-10-08T00:19:51.788 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.785+0000 7f261e8a3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f261806acb0 0x7f26180afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:51.788 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.785+0000 7f261e0a2700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2618159ec0 0x7f26180af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:51.788 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.785+0000 7f261e8a3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f261806acb0 0x7f26180afe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.99:51312/0 (socket says 172.21.15.99:51312) 2023-10-08T00:19:51.788 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.785+0000 7f261e8a3700 1 -- 172.21.15.99:0/440695543 learned_addr learned my addr 172.21.15.99:0/440695543 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:19:51.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.785+0000 7f261d8a1700 1 --1- 172.21.15.99:0/440695543 >> v1:172.21.15.191:6789/0 conn(0x7f261815a290 0x7f261806a5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:59900/0 (socket says 172.21.15.99:59900) 2023-10-08T00:19:51.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.785+0000 7f261e8a3700 1 -- 172.21.15.99:0/440695543 >> v2:172.21.15.191:3300/0 conn(0x7f2618159ec0 msgr2=0x7f26180af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:19:51.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.785+0000 7f261e8a3700 1 --2- 172.21.15.99:0/440695543 >> v2:172.21.15.191:3300/0 conn(0x7f2618159ec0 0x7f26180af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:51.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.785+0000 7f261e8a3700 1 -- 172.21.15.99:0/440695543 >> v1:172.21.15.191:6789/0 conn(0x7f261815a290 legacy=0x7f261806a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:51.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.785+0000 7f261e8a3700 1 -- 172.21.15.99:0/440695543 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26180b04b0 con 0x7f261806acb0 2023-10-08T00:19:51.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.786+0000 7f261e8a3700 1 --2- 172.21.15.99:0/440695543 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f261806acb0 0x7f26180afe80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f26100099b0 tx=0x7f2610009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26d2c7d2952f2578 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:51.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.786+0000 7f261d0a0700 1 -- 172.21.15.99:0/440695543 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2610009ea0 con 0x7f261806acb0 2023-10-08T00:19:51.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.786+0000 7f261d0a0700 1 -- 172.21.15.99:0/440695543 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f261000b860 con 0x7f261806acb0 2023-10-08T00:19:51.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.786+0000 7f261d0a0700 1 -- 172.21.15.99:0/440695543 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261000bc00 con 0x7f261806acb0 2023-10-08T00:19:51.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.786+0000 7f2620306700 1 -- 172.21.15.99:0/440695543 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f261806acb0 msgr2=0x7f26180afe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:51.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.786+0000 7f2620306700 1 --2- 172.21.15.99:0/440695543 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f261806acb0 0x7f26180afe80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f26100099b0 tx=0x7f2610009250 comp rx=0 tx=0).stop 2023-10-08T00:19:51.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.786+0000 7f2620306700 1 -- 172.21.15.99:0/440695543 shutdown_connections 2023-10-08T00:19:51.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.786+0000 7f2620306700 1 --2- 172.21.15.99:0/440695543 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f261806acb0 0x7f26180afe80 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:51.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.786+0000 7f2620306700 1 --2- 172.21.15.99:0/440695543 >> v2:172.21.15.191:3300/0 conn(0x7f2618159ec0 0x7f26180af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:51.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.786+0000 7f2620306700 1 -- 172.21.15.99:0/440695543 >> 172.21.15.99:0/440695543 conn(0x7f26180d1880 msgr2=0x7f26180cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:19:51.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.786+0000 7f2620306700 1 -- 172.21.15.99:0/440695543 shutdown_connections 2023-10-08T00:19:51.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.786+0000 7f2620306700 1 -- 172.21.15.99:0/440695543 wait complete. 2023-10-08T00:19:51.793 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.787+0000 7f2620306700 1 Processor -- start 2023-10-08T00:19:51.794 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.787+0000 7f2620306700 1 -- start start 2023-10-08T00:19:51.794 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.788+0000 7f2620306700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f261806acb0 0x7f2618155990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:51.794 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.788+0000 7f2620306700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2618159ec0 0x7f261816a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:51.794 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.788+0000 7f2620306700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f261815a290 0x7f26180c9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:51.794 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.788+0000 7f2620306700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2618155ed0 con 0x7f261806acb0 2023-10-08T00:19:51.794 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.788+0000 7f2620306700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26181561f0 con 0x7f261815a290 2023-10-08T00:19:51.794 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.788+0000 7f2620306700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26180aff00 con 0x7f2618159ec0 2023-10-08T00:19:51.795 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.788+0000 7f261e0a2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f261806acb0 0x7f2618155990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:51.795 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.788+0000 7f261e0a2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f261806acb0 0x7f2618155990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44270/0 (socket says 172.21.15.99:44270) 2023-10-08T00:19:51.795 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.788+0000 7f261e0a2700 1 -- 172.21.15.99:0/434601084 learned_addr learned my addr 172.21.15.99:0/434601084 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:19:51.795 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.788+0000 7f261d8a1700 1 --2- 172.21.15.99:0/434601084 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2618159ec0 0x7f261816a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:51.795 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.788+0000 7f261e8a3700 1 --2- 172.21.15.99:0/434601084 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f261815a290 0x7f26180c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:51.795 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.789+0000 7f261e0a2700 1 -- 172.21.15.99:0/434601084 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2618159ec0 msgr2=0x7f261816a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:51.795 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.789+0000 7f261e0a2700 1 --2- 172.21.15.99:0/434601084 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2618159ec0 0x7f261816a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:51.795 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.789+0000 7f261e0a2700 1 -- 172.21.15.99:0/434601084 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f261815a290 msgr2=0x7f26180c9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:51.795 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.789+0000 7f261e0a2700 1 --2- 172.21.15.99:0/434601084 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f261815a290 0x7f26180c9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:51.796 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.789+0000 7f261e0a2700 1 -- 172.21.15.99:0/434601084 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26100094f0 con 0x7f261806acb0 2023-10-08T00:19:51.796 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.789+0000 7f261e0a2700 1 --2- 172.21.15.99:0/434601084 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f261806acb0 0x7f2618155990 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f2614009f20 tx=0x7f261400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:51.796 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.789+0000 7f260e7fc700 1 -- 172.21.15.99:0/434601084 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261400cb70 con 0x7f261806acb0 2023-10-08T00:19:51.796 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.789+0000 7f260e7fc700 1 -- 172.21.15.99:0/434601084 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f261400f040 con 0x7f261806acb0 2023-10-08T00:19:51.796 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.789+0000 7f2620306700 1 -- 172.21.15.99:0/434601084 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26181585a0 con 0x7f261806acb0 2023-10-08T00:19:51.796 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.789+0000 7f260e7fc700 1 -- 172.21.15.99:0/434601084 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2614014480 con 0x7f261806acb0 2023-10-08T00:19:51.796 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.789+0000 7f2620306700 1 -- 172.21.15.99:0/434601084 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26180af940 con 0x7f261806acb0 2023-10-08T00:19:51.796 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.790+0000 7f2620306700 1 -- 172.21.15.99:0/434601084 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26180c7e40 con 0x7f261806acb0 2023-10-08T00:19:51.796 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.790+0000 7f260e7fc700 1 -- 172.21.15.99:0/434601084 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f26140147f0 con 0x7f261806acb0 2023-10-08T00:19:51.796 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.790+0000 7f260e7fc700 1 --2- 172.21.15.99:0/434601084 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f25fc066f30 0x7f25fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:51.797 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.791+0000 7f260e7fc700 1 -- 172.21.15.99:0/434601084 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f26140851d0 con 0x7f261806acb0 2023-10-08T00:19:51.797 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.791+0000 7f260e7fc700 1 -- 172.21.15.99:0/434601084 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=157}) v3 -- 0x7f25fc06ea50 con 0x7f261806acb0 2023-10-08T00:19:51.797 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.791+0000 7f261d8a1700 1 --2- 172.21.15.99:0/434601084 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f25fc066f30 0x7f25fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:51.797 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.791+0000 7f261d8a1700 1 --2- 172.21.15.99:0/434601084 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f25fc066f30 0x7f25fc0693e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f26180ccf50 tx=0x7f2608007eb0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:51.797 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.795+0000 7f260e7fc700 1 -- 172.21.15.99:0/434601084 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2614052440 con 0x7f261806acb0 2023-10-08T00:19:51.977 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.976+0000 7f2620306700 1 -- 172.21.15.99:0/434601084 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f261804c3c0 con 0x7f261806acb0 2023-10-08T00:19:51.977 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.977+0000 7f260e7fc700 1 -- 172.21.15.99:0/434601084 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v156) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2614055a60 con 0x7f261806acb0 2023-10-08T00:19:51.978 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2023-10-08T00:19:51.980 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.979+0000 7f2620306700 1 -- 172.21.15.99:0/434601084 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f25fc066f30 msgr2=0x7f25fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:51.980 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.979+0000 7f2620306700 1 --2- 172.21.15.99:0/434601084 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f25fc066f30 0x7f25fc0693e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f26180ccf50 tx=0x7f2608007eb0 comp rx=0 tx=0).stop 2023-10-08T00:19:51.980 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.979+0000 7f2620306700 1 -- 172.21.15.99:0/434601084 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f261806acb0 msgr2=0x7f2618155990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:51.980 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.979+0000 7f2620306700 1 --2- 172.21.15.99:0/434601084 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f261806acb0 0x7f2618155990 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f2614009f20 tx=0x7f261400b040 comp rx=0 tx=0).stop 2023-10-08T00:19:51.980 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.979+0000 7f2620306700 1 -- 172.21.15.99:0/434601084 shutdown_connections 2023-10-08T00:19:51.980 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.979+0000 7f2620306700 1 --2- 172.21.15.99:0/434601084 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f25fc066f30 0x7f25fc0693e0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:51.980 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.979+0000 7f2620306700 1 --2- 172.21.15.99:0/434601084 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f261815a290 0x7f26180c9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:51.981 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.979+0000 7f2620306700 1 --2- 172.21.15.99:0/434601084 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f261806acb0 0x7f2618155990 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:51.981 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.979+0000 7f2620306700 1 --2- 172.21.15.99:0/434601084 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2618159ec0 0x7f261816a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:51.981 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.979+0000 7f2620306700 1 -- 172.21.15.99:0/434601084 >> 172.21.15.99:0/434601084 conn(0x7f26180d1880 msgr2=0x7f26180b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:19:51.981 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.979+0000 7f2620306700 1 -- 172.21.15.99:0/434601084 shutdown_connections 2023-10-08T00:19:51.981 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:51.979+0000 7f2620306700 1 -- 172.21.15.99:0/434601084 wait complete. 2023-10-08T00:19:51.992 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T00:19:52.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.107+0000 7f6f7ad1a700 1 Processor -- start 2023-10-08T00:19:52.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.107+0000 7f6f7ad1a700 1 -- start start 2023-10-08T00:19:52.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.108+0000 7f6f7ad1a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f740b3570 0x7f6f7415a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:52.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.108+0000 7f6f7ad1a700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6f740b4c30 0x7f6f740b6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:52.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.108+0000 7f6f7ad1a700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f740b14f0 con 0x7f6f740b4860 2023-10-08T00:19:52.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.108+0000 7f6f7ad1a700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f740b17d0 con 0x7f6f740b4c30 2023-10-08T00:19:52.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.108+0000 7f6f7ad1a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f740b1ab0 con 0x7f6f740b3570 2023-10-08T00:19:52.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.108+0000 7f6f792b7700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6f740b4c30 0x7f6f740b6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:52.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.108+0000 7f6f792b7700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6f740b4c30 0x7f6f740b6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44282/0 (socket says 172.21.15.99:44282) 2023-10-08T00:19:52.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.108+0000 7f6f792b7700 1 -- 172.21.15.99:0/2282464833 learned_addr learned my addr 172.21.15.99:0/2282464833 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:19:52.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.108+0000 7f6f73fff700 1 --2- 172.21.15.99:0/2282464833 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f740b3570 0x7f6f7415a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:52.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.108+0000 7f6f792b7700 1 -- 172.21.15.99:0/2282464833 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f740b3570 msgr2=0x7f6f7415a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:19:52.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.108+0000 7f6f792b7700 1 --2- 172.21.15.99:0/2282464833 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f740b3570 0x7f6f7415a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:52.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.108+0000 7f6f792b7700 1 -- 172.21.15.99:0/2282464833 >> v1:172.21.15.191:6789/0 conn(0x7f6f740b4860 legacy=0x7f6f74159a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:19:52.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.108+0000 7f6f792b7700 1 -- 172.21.15.99:0/2282464833 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f740b1d90 con 0x7f6f740b4c30 2023-10-08T00:19:52.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.108+0000 7f6f792b7700 1 --2- 172.21.15.99:0/2282464833 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f740b4c30 0x7f6f740b6700 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f6f640099b0 tx=0x7f6f64009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45f90fe4ad21d3ec server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:52.113 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.109+0000 7f6f737fe700 1 -- 172.21.15.99:0/2282464833 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6400b8b0 con 0x7f6f740b4c30 2023-10-08T00:19:52.113 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.109+0000 7f6f737fe700 1 -- 172.21.15.99:0/2282464833 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f6400ba10 con 0x7f6f740b4c30 2023-10-08T00:19:52.113 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.109+0000 7f6f737fe700 1 -- 172.21.15.99:0/2282464833 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6400bdb0 con 0x7f6f740b4c30 2023-10-08T00:19:52.113 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.109+0000 7f6f7ad1a700 1 -- 172.21.15.99:0/2282464833 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f740b4c30 msgr2=0x7f6f740b6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:52.113 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.109+0000 7f6f7ad1a700 1 --2- 172.21.15.99:0/2282464833 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f740b4c30 0x7f6f740b6700 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f6f640099b0 tx=0x7f6f64009250 comp rx=0 tx=0).stop 2023-10-08T00:19:52.113 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.109+0000 7f6f7ad1a700 1 -- 172.21.15.99:0/2282464833 shutdown_connections 2023-10-08T00:19:52.113 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.109+0000 7f6f7ad1a700 1 --2- 172.21.15.99:0/2282464833 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f740b4c30 0x7f6f740b6700 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:52.113 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.109+0000 7f6f7ad1a700 1 --2- 172.21.15.99:0/2282464833 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f740b3570 0x7f6f7415a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:52.113 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.109+0000 7f6f7ad1a700 1 -- 172.21.15.99:0/2282464833 >> 172.21.15.99:0/2282464833 conn(0x7f6f7409a0f0 msgr2=0x7f6f74067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:19:52.113 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.109+0000 7f6f7ad1a700 1 -- 172.21.15.99:0/2282464833 shutdown_connections 2023-10-08T00:19:52.114 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.109+0000 7f6f7ad1a700 1 -- 172.21.15.99:0/2282464833 wait complete. 2023-10-08T00:19:52.114 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.110+0000 7f6f7ad1a700 1 Processor -- start 2023-10-08T00:19:52.114 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.110+0000 7f6f7ad1a700 1 -- start start 2023-10-08T00:19:52.116 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.111+0000 7f6f7ad1a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f740b3570 0x7f6f740ad120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:52.116 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.111+0000 7f6f7ad1a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f740b4860 0x7f6f740ae960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:52.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.111+0000 7f6f7ad1a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f740b4c30 0x7f6f740aeea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:52.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.111+0000 7f6f7ad1a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f740afd50 con 0x7f6f740b4c30 2023-10-08T00:19:52.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.111+0000 7f6f7ad1a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7409bd60 con 0x7f6f740b4860 2023-10-08T00:19:52.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.111+0000 7f6f7ad1a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f740af4b0 con 0x7f6f740b3570 2023-10-08T00:19:52.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.111+0000 7f6f792b7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f740b4c30 0x7f6f740aeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:52.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.111+0000 7f6f792b7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f740b4c30 0x7f6f740aeea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44290/0 (socket says 172.21.15.99:44290) 2023-10-08T00:19:52.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.111+0000 7f6f792b7700 1 -- 172.21.15.99:0/2060130890 learned_addr learned my addr 172.21.15.99:0/2060130890 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:19:52.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.111+0000 7f6f78ab6700 1 --2- 172.21.15.99:0/2060130890 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f740b3570 0x7f6f740ad120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:52.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.111+0000 7f6f73fff700 1 --2- 172.21.15.99:0/2060130890 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f740b4860 0x7f6f740ae960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:52.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.111+0000 7f6f792b7700 1 -- 172.21.15.99:0/2060130890 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f740b3570 msgr2=0x7f6f740ad120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:52.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.111+0000 7f6f792b7700 1 --2- 172.21.15.99:0/2060130890 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f740b3570 0x7f6f740ad120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:52.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.111+0000 7f6f792b7700 1 -- 172.21.15.99:0/2060130890 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f740b4860 msgr2=0x7f6f740ae960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:52.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.112+0000 7f6f792b7700 1 --2- 172.21.15.99:0/2060130890 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f740b4860 0x7f6f740ae960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:52.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.112+0000 7f6f792b7700 1 -- 172.21.15.99:0/2060130890 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f640094f0 con 0x7f6f740b4c30 2023-10-08T00:19:52.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.112+0000 7f6f792b7700 1 --2- 172.21.15.99:0/2060130890 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f740b4c30 0x7f6f740aeea0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f6f640094c0 tx=0x7f6f64016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:52.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.112+0000 7f6f70ff9700 1 -- 172.21.15.99:0/2060130890 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6400b8b0 con 0x7f6f740b4c30 2023-10-08T00:19:52.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.112+0000 7f6f70ff9700 1 -- 172.21.15.99:0/2060130890 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f64020070 con 0x7f6f740b4c30 2023-10-08T00:19:52.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.112+0000 7f6f7ad1a700 1 -- 172.21.15.99:0/2060130890 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f740af790 con 0x7f6f740b4c30 2023-10-08T00:19:52.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.112+0000 7f6f70ff9700 1 -- 172.21.15.99:0/2060130890 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6401b6a0 con 0x7f6f740b4c30 2023-10-08T00:19:52.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.112+0000 7f6f7ad1a700 1 -- 172.21.15.99:0/2060130890 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f7406a240 con 0x7f6f740b4c30 2023-10-08T00:19:52.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.113+0000 7f6f7ad1a700 1 -- 172.21.15.99:0/2060130890 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f7409bb30 con 0x7f6f740b4c30 2023-10-08T00:19:52.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.113+0000 7f6f70ff9700 1 -- 172.21.15.99:0/2060130890 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6f64022030 con 0x7f6f740b4c30 2023-10-08T00:19:52.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.114+0000 7f6f70ff9700 1 --2- 172.21.15.99:0/2060130890 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6f60066f30 0x7f6f600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:52.119 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.114+0000 7f6f70ff9700 1 -- 172.21.15.99:0/2060130890 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6f640857b0 con 0x7f6f740b4c30 2023-10-08T00:19:52.119 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.114+0000 7f6f70ff9700 1 -- 172.21.15.99:0/2060130890 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=157}) v3 -- 0x7f6f6006ea50 con 0x7f6f740b4c30 2023-10-08T00:19:52.119 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.114+0000 7f6f78ab6700 1 --2- 172.21.15.99:0/2060130890 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6f60066f30 0x7f6f600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:52.119 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.114+0000 7f6f78ab6700 1 --2- 172.21.15.99:0/2060130890 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6f60066f30 0x7f6f600693e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c00f430 tx=0x7f6f6c002ad0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:52.119 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.118+0000 7f6f70ff9700 1 -- 172.21.15.99:0/2060130890 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6f7409bb30 con 0x7f6f740b4c30 2023-10-08T00:19:52.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.302+0000 7f6f7ad1a700 1 -- 172.21.15.99:0/2060130890 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6f7409bb30 con 0x7f6f740b4c30 2023-10-08T00:19:52.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.302+0000 7f6f70ff9700 1 -- 172.21.15.99:0/2060130890 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v156) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6f7409bb30 con 0x7f6f740b4c30 2023-10-08T00:19:52.303 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2023-10-08T00:19:52.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.304+0000 7f6f7ad1a700 1 -- 172.21.15.99:0/2060130890 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6f60066f30 msgr2=0x7f6f600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:52.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.304+0000 7f6f7ad1a700 1 --2- 172.21.15.99:0/2060130890 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6f60066f30 0x7f6f600693e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c00f430 tx=0x7f6f6c002ad0 comp rx=0 tx=0).stop 2023-10-08T00:19:52.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.304+0000 7f6f7ad1a700 1 -- 172.21.15.99:0/2060130890 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f740b4c30 msgr2=0x7f6f740aeea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:52.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.304+0000 7f6f7ad1a700 1 --2- 172.21.15.99:0/2060130890 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f740b4c30 0x7f6f740aeea0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f6f640094c0 tx=0x7f6f64016000 comp rx=0 tx=0).stop 2023-10-08T00:19:52.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.304+0000 7f6f7ad1a700 1 -- 172.21.15.99:0/2060130890 shutdown_connections 2023-10-08T00:19:52.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.304+0000 7f6f7ad1a700 1 --2- 172.21.15.99:0/2060130890 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6f60066f30 0x7f6f600693e0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:52.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.304+0000 7f6f7ad1a700 1 --2- 172.21.15.99:0/2060130890 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f740b4c30 0x7f6f740aeea0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:52.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.304+0000 7f6f7ad1a700 1 --2- 172.21.15.99:0/2060130890 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f740b3570 0x7f6f740ad120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:52.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.304+0000 7f6f7ad1a700 1 --2- 172.21.15.99:0/2060130890 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f740b4860 0x7f6f740ae960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:52.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.304+0000 7f6f7ad1a700 1 -- 172.21.15.99:0/2060130890 >> 172.21.15.99:0/2060130890 conn(0x7f6f7409a0f0 msgr2=0x7f6f74068eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:19:52.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.304+0000 7f6f7ad1a700 1 -- 172.21.15.99:0/2060130890 shutdown_connections 2023-10-08T00:19:52.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.304+0000 7f6f7ad1a700 1 -- 172.21.15.99:0/2060130890 wait complete. 2023-10-08T00:19:52.317 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:19:52.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.431+0000 7fc4cca17700 1 Processor -- start 2023-10-08T00:19:52.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.431+0000 7fc4cca17700 1 -- start start 2023-10-08T00:19:52.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.432+0000 7fc4cca17700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4c80cd6e0 0x7fc4c80cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:52.443 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.432+0000 7fc4cca17700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4c80cb7c0 con 0x7fc4c80cdab0 2023-10-08T00:19:52.443 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.432+0000 7fc4cca17700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4c815e290 con 0x7fc4c815cbd0 2023-10-08T00:19:52.443 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.432+0000 7fc4cca17700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c815e850 con 0x7fc4c80cd6e0 2023-10-08T00:19:52.443 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.432+0000 7fc4c6ffd700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fc4c80cdab0 0x7fc4c815d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:38530/0 (socket says 172.21.15.99:38530) 2023-10-08T00:19:52.443 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.432+0000 7fc4c6ffd700 1 -- 172.21.15.99:0/440299421 learned_addr learned my addr 172.21.15.99:0/440299421 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:19:52.443 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.432+0000 7fc4c67fc700 1 -- 172.21.15.99:0/440299421 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1746165210 0 0) 0x7fc4c80cb7c0 con 0x7fc4c80cdab0 2023-10-08T00:19:52.444 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.432+0000 7fc4c77fe700 1 --2- 172.21.15.99:0/440299421 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4c80cd6e0 0x7fc4c80cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:52.444 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.432+0000 7fc4c67fc700 1 -- 172.21.15.99:0/440299421 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4b0003570 con 0x7fc4c80cdab0 2023-10-08T00:19:52.444 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.433+0000 7fc4c67fc700 1 -- 172.21.15.99:0/440299421 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4014045091 0 0) 0x7fc4b0003570 con 0x7fc4c80cdab0 2023-10-08T00:19:52.444 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.433+0000 7fc4c67fc700 1 -- 172.21.15.99:0/440299421 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4c80cd6e0 msgr2=0x7fc4c80cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:19:52.444 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.433+0000 7fc4c67fc700 1 --2- 172.21.15.99:0/440299421 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4c80cd6e0 0x7fc4c80cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:52.444 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.433+0000 7fc4c67fc700 1 -- 172.21.15.99:0/440299421 >> v1:172.21.15.191:6789/0 conn(0x7fc4c815cbd0 legacy=0x7fc4c815d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:52.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.433+0000 7fc4c67fc700 1 -- 172.21.15.99:0/440299421 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c815e570 con 0x7fc4c80cdab0 2023-10-08T00:19:52.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.433+0000 7fc4c67fc700 1 -- 172.21.15.99:0/440299421 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7fc4bc001470 con 0x7fc4c80cdab0 2023-10-08T00:19:52.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.433+0000 7fc4c67fc700 1 -- 172.21.15.99:0/440299421 >> v1:172.21.15.99:6789/0 conn(0x7fc4c80cdab0 legacy=0x7fc4c815d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:52.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.433+0000 7fc4c67fc700 1 --2- 172.21.15.99:0/440299421 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4b0003a90 0x7fc4b0003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:52.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.433+0000 7fc4c67fc700 1 --2- 172.21.15.99:0/440299421 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4c80cd6e0 0x7fc4b0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:52.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.433+0000 7fc4c67fc700 1 --2- 172.21.15.99:0/440299421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4b0004d50 0x7fc4b00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:52.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.433+0000 7fc4c67fc700 1 -- 172.21.15.99:0/440299421 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c815e850 con 0x7fc4b0004d50 2023-10-08T00:19:52.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.433+0000 7fc4c67fc700 1 -- 172.21.15.99:0/440299421 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b000bb70 con 0x7fc4b0003a90 2023-10-08T00:19:52.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.433+0000 7fc4c67fc700 1 -- 172.21.15.99:0/440299421 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b000bd30 con 0x7fc4c80cd6e0 2023-10-08T00:19:52.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.433+0000 7fc4c7fff700 1 --2- 172.21.15.99:0/440299421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4b0004d50 0x7fc4b00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:52.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.433+0000 7fc4c77fe700 1 --2- 172.21.15.99:0/440299421 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4b0003a90 0x7fc4b0003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:52.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.433+0000 7fc4c6ffd700 1 --2- 172.21.15.99:0/440299421 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4c80cd6e0 0x7fc4b0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:52.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.433+0000 7fc4c7fff700 1 -- 172.21.15.99:0/440299421 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4c80cd6e0 msgr2=0x7fc4b0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:19:52.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.433+0000 7fc4c7fff700 1 --2- 172.21.15.99:0/440299421 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4c80cd6e0 0x7fc4b0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:52.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.433+0000 7fc4c7fff700 1 -- 172.21.15.99:0/440299421 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4b0003a90 msgr2=0x7fc4b0003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:19:52.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.433+0000 7fc4c7fff700 1 --2- 172.21.15.99:0/440299421 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4b0003a90 0x7fc4b0003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:52.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.433+0000 7fc4c7fff700 1 -- 172.21.15.99:0/440299421 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b0003570 con 0x7fc4b0004d50 2023-10-08T00:19:52.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.433+0000 7fc4c7fff700 1 --2- 172.21.15.99:0/440299421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4b0004d50 0x7fc4b00051c0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fc4c00018b0 tx=0x7fc4c0007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=877061e5a897134f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:52.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.433+0000 7fc4c67fc700 1 -- 172.21.15.99:0/440299421 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c0012420 con 0x7fc4b0004d50 2023-10-08T00:19:52.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.434+0000 7fc4c67fc700 1 -- 172.21.15.99:0/440299421 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4c0012580 con 0x7fc4b0004d50 2023-10-08T00:19:52.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.434+0000 7fc4c67fc700 1 -- 172.21.15.99:0/440299421 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c0012920 con 0x7fc4b0004d50 2023-10-08T00:19:52.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.434+0000 7fc4cca17700 1 -- 172.21.15.99:0/440299421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4b0004d50 msgr2=0x7fc4b00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:52.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.434+0000 7fc4cca17700 1 --2- 172.21.15.99:0/440299421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4b0004d50 0x7fc4b00051c0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fc4c00018b0 tx=0x7fc4c0007d30 comp rx=0 tx=0).stop 2023-10-08T00:19:52.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.434+0000 7fc4c7fff700 1 -- 172.21.15.99:0/440299421 reap_dead start 2023-10-08T00:19:52.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.434+0000 7fc4cca17700 1 -- 172.21.15.99:0/440299421 shutdown_connections 2023-10-08T00:19:52.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.434+0000 7fc4cca17700 1 -- 172.21.15.99:0/440299421 >> 172.21.15.99:0/440299421 conn(0x7fc4c8067660 msgr2=0x7fc4c8067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:19:52.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.434+0000 7fc4cca17700 1 -- 172.21.15.99:0/440299421 shutdown_connections 2023-10-08T00:19:52.449 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.434+0000 7fc4cca17700 1 -- 172.21.15.99:0/440299421 wait complete. 2023-10-08T00:19:52.449 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.435+0000 7fc4cca17700 1 Processor -- start 2023-10-08T00:19:52.449 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.435+0000 7fc4cca17700 1 -- start start 2023-10-08T00:19:52.449 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.436+0000 7fc4cca17700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4c8155c20 0x7fc4c80c8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:52.449 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.436+0000 7fc4cca17700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4c80ca9c0 0x7fc4c81586d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:52.449 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.436+0000 7fc4cca17700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4c8158c10 0x7fc4c80c8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:52.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.436+0000 7fc4cca17700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c815a810 con 0x7fc4c80ca9c0 2023-10-08T00:19:52.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.436+0000 7fc4cca17700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c8159060 con 0x7fc4c8158c10 2023-10-08T00:19:52.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.436+0000 7fc4cca17700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c815aa40 con 0x7fc4c8155c20 2023-10-08T00:19:52.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.436+0000 7fc4c6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4c80ca9c0 0x7fc4c81586d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:52.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.436+0000 7fc4c6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4c80ca9c0 0x7fc4c81586d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44308/0 (socket says 172.21.15.99:44308) 2023-10-08T00:19:52.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.436+0000 7fc4c6ffd700 1 -- 172.21.15.99:0/2911517393 learned_addr learned my addr 172.21.15.99:0/2911517393 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:19:52.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.436+0000 7fc4c77fe700 1 --2- 172.21.15.99:0/2911517393 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4c8155c20 0x7fc4c80c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:52.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.436+0000 7fc4c7fff700 1 --2- 172.21.15.99:0/2911517393 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4c8158c10 0x7fc4c80c8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:52.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.436+0000 7fc4c6ffd700 1 -- 172.21.15.99:0/2911517393 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4c8155c20 msgr2=0x7fc4c80c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:52.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.436+0000 7fc4c6ffd700 1 --2- 172.21.15.99:0/2911517393 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4c8155c20 0x7fc4c80c8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:52.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.436+0000 7fc4c6ffd700 1 -- 172.21.15.99:0/2911517393 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4c8158c10 msgr2=0x7fc4c80c8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:52.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.436+0000 7fc4c6ffd700 1 --2- 172.21.15.99:0/2911517393 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4c8158c10 0x7fc4c80c8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:52.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.436+0000 7fc4c6ffd700 1 -- 172.21.15.99:0/2911517393 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c815ad20 con 0x7fc4c80ca9c0 2023-10-08T00:19:52.452 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.436+0000 7fc4c7fff700 1 --2- 172.21.15.99:0/2911517393 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4c8158c10 0x7fc4c80c8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:19:52.452 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.437+0000 7fc4c6ffd700 1 --2- 172.21.15.99:0/2911517393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4c80ca9c0 0x7fc4c81586d0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc000fd0 tx=0x7fc4bc001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:52.452 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.437+0000 7fc4af7fe700 1 -- 172.21.15.99:0/2911517393 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc018030 con 0x7fc4c80ca9c0 2023-10-08T00:19:52.452 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.437+0000 7fc4af7fe700 1 -- 172.21.15.99:0/2911517393 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4bc0077f0 con 0x7fc4c80ca9c0 2023-10-08T00:19:52.452 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.437+0000 7fc4cca17700 1 -- 172.21.15.99:0/2911517393 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4c815b000 con 0x7fc4c80ca9c0 2023-10-08T00:19:52.452 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.437+0000 7fc4af7fe700 1 -- 172.21.15.99:0/2911517393 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc00fb20 con 0x7fc4c80ca9c0 2023-10-08T00:19:52.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.437+0000 7fc4cca17700 1 -- 172.21.15.99:0/2911517393 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4c8155ff0 con 0x7fc4c80ca9c0 2023-10-08T00:19:52.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.438+0000 7fc4cca17700 1 -- 172.21.15.99:0/2911517393 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4c815b800 con 0x7fc4c80ca9c0 2023-10-08T00:19:52.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.438+0000 7fc4af7fe700 1 -- 172.21.15.99:0/2911517393 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc4bc01b070 con 0x7fc4c80ca9c0 2023-10-08T00:19:52.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.438+0000 7fc4af7fe700 1 --2- 172.21.15.99:0/2911517393 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc4b4066f30 0x7fc4b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:52.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.439+0000 7fc4af7fe700 1 -- 172.21.15.99:0/2911517393 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc4bc019070 con 0x7fc4c80ca9c0 2023-10-08T00:19:52.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.439+0000 7fc4af7fe700 1 -- 172.21.15.99:0/2911517393 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=157}) v3 -- 0x7fc4b406ea50 con 0x7fc4c80ca9c0 2023-10-08T00:19:52.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.439+0000 7fc4c77fe700 1 --2- 172.21.15.99:0/2911517393 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc4b4066f30 0x7fc4b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:52.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.439+0000 7fc4c77fe700 1 --2- 172.21.15.99:0/2911517393 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc4b4066f30 0x7fc4b40693e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc4b80094d0 tx=0x7fc4b800b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:52.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.443+0000 7fc4af7fe700 1 -- 172.21.15.99:0/2911517393 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc4c815b800 con 0x7fc4c80ca9c0 2023-10-08T00:19:52.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.664+0000 7fc4cca17700 1 -- 172.21.15.99:0/2911517393 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc4c815b800 con 0x7fc4c80ca9c0 2023-10-08T00:19:52.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.665+0000 7fc4af7fe700 1 -- 172.21.15.99:0/2911517393 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4599 (secure 0 0 0) 0x7fc4c815b800 con 0x7fc4c80ca9c0 2023-10-08T00:19:52.665 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:19:52.666 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":70,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6056,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/3025449992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3025449992},{"type":"v1","addr":"172.21.15.191:6836","nonce":3025449992}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":7476,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/86350138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":86350138},{"type":"v1","addr":"172.21.15.99:6834","nonce":86350138}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":7479,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1282887036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1282887036},{"type":"v1","addr":"172.21.15.99:6835","nonce":1282887036}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:17:39.623734+0000","modified":"2023-10-08T00:17:43.252633+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7494":{"gid":7494,"name":"d","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/1165162515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":1165162515},{"type":"v1","addr":"172.21.15.191:6837","nonce":1165162515}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-08T00:19:52.667 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.666+0000 7fc4cca17700 1 -- 172.21.15.99:0/2911517393 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc4b4066f30 msgr2=0x7fc4b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:52.667 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.666+0000 7fc4cca17700 1 --2- 172.21.15.99:0/2911517393 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc4b4066f30 0x7fc4b40693e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc4b80094d0 tx=0x7fc4b800b040 comp rx=0 tx=0).stop 2023-10-08T00:19:52.667 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.666+0000 7fc4cca17700 1 -- 172.21.15.99:0/2911517393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4c80ca9c0 msgr2=0x7fc4c81586d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:52.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.666+0000 7fc4cca17700 1 --2- 172.21.15.99:0/2911517393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4c80ca9c0 0x7fc4c81586d0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc000fd0 tx=0x7fc4bc001e50 comp rx=0 tx=0).stop 2023-10-08T00:19:52.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.666+0000 7fc4cca17700 1 -- 172.21.15.99:0/2911517393 shutdown_connections 2023-10-08T00:19:52.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.666+0000 7fc4cca17700 1 --2- 172.21.15.99:0/2911517393 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc4b4066f30 0x7fc4b40693e0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:52.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.667+0000 7fc4cca17700 1 --2- 172.21.15.99:0/2911517393 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4c8158c10 0x7fc4c80c8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:52.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.667+0000 7fc4cca17700 1 --2- 172.21.15.99:0/2911517393 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4c8155c20 0x7fc4c80c8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:52.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.667+0000 7fc4cca17700 1 --2- 172.21.15.99:0/2911517393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4c80ca9c0 0x7fc4c81586d0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:52.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.667+0000 7fc4cca17700 1 -- 172.21.15.99:0/2911517393 >> 172.21.15.99:0/2911517393 conn(0x7fc4c8067660 msgr2=0x7fc4c80d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:19:52.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.667+0000 7fc4cca17700 1 -- 172.21.15.99:0/2911517393 shutdown_connections 2023-10-08T00:19:52.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.667+0000 7fc4cca17700 1 -- 172.21.15.99:0/2911517393 wait complete. 2023-10-08T00:19:52.669 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 70 2023-10-08T00:19:52.679 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:19:52.792 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.791+0000 7f68ba2cd700 1 Processor -- start 2023-10-08T00:19:52.792 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.791+0000 7f68ba2cd700 1 -- start start 2023-10-08T00:19:52.800 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.792+0000 7f68ba2cd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f68b4157870 0x7f68b415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:52.800 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.792+0000 7f68ba2cd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f68b415a620 0x7f68b415d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:52.800 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.792+0000 7f68ba2cd700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f68b415d580 0x7f68b4158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:52.800 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.792+0000 7f68ba2cd700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68b415cbb0 con 0x7f68b415a620 2023-10-08T00:19:52.800 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.792+0000 7f68ba2cd700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68b415bfa0 con 0x7f68b415d580 2023-10-08T00:19:52.801 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.792+0000 7f68ba2cd700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68b415c280 con 0x7f68b4157870 2023-10-08T00:19:52.801 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.792+0000 7f68b37fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f68b415a620 0x7f68b415d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:52.801 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.793+0000 7f68b37fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f68b415a620 0x7f68b415d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44316/0 (socket says 172.21.15.99:44316) 2023-10-08T00:19:52.801 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.793+0000 7f68b37fe700 1 -- 172.21.15.99:0/2019670575 learned_addr learned my addr 172.21.15.99:0/2019670575 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:19:52.801 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.793+0000 7f68b886a700 1 --2- 172.21.15.99:0/2019670575 >> v2:172.21.15.191:3300/0 conn(0x7f68b415d580 0x7f68b4158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:52.801 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.793+0000 7f68b3fff700 1 --2- 172.21.15.99:0/2019670575 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f68b4157870 0x7f68b415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:52.801 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.793+0000 7f68b37fe700 1 -- 172.21.15.99:0/2019670575 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f68b4157870 msgr2=0x7f68b415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:19:52.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.793+0000 7f68b37fe700 1 --2- 172.21.15.99:0/2019670575 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f68b4157870 0x7f68b415b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:52.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.793+0000 7f68b37fe700 1 -- 172.21.15.99:0/2019670575 >> v2:172.21.15.191:3300/0 conn(0x7f68b415d580 msgr2=0x7f68b4158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:19:52.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.793+0000 7f68b37fe700 1 --2- 172.21.15.99:0/2019670575 >> v2:172.21.15.191:3300/0 conn(0x7f68b415d580 0x7f68b4158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:52.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.793+0000 7f68b37fe700 1 -- 172.21.15.99:0/2019670575 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b415c560 con 0x7f68b415a620 2023-10-08T00:19:52.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.793+0000 7f68b37fe700 1 --2- 172.21.15.99:0/2019670575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68b415a620 0x7f68b415d040 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f68b40b03b0 tx=0x7f68a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee9dfd83d2699b0e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:52.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.793+0000 7f68b2ffd700 1 -- 172.21.15.99:0/2019670575 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a800b860 con 0x7f68b415a620 2023-10-08T00:19:52.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.793+0000 7f68b2ffd700 1 -- 172.21.15.99:0/2019670575 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68a800b9c0 con 0x7f68b415a620 2023-10-08T00:19:52.803 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.793+0000 7f68b2ffd700 1 -- 172.21.15.99:0/2019670575 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a800bd60 con 0x7f68b415a620 2023-10-08T00:19:52.803 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.794+0000 7f68ba2cd700 1 -- 172.21.15.99:0/2019670575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68b415a620 msgr2=0x7f68b415d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:52.803 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.794+0000 7f68ba2cd700 1 --2- 172.21.15.99:0/2019670575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68b415a620 0x7f68b415d040 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f68b40b03b0 tx=0x7f68a8009250 comp rx=0 tx=0).stop 2023-10-08T00:19:52.803 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.794+0000 7f68ba2cd700 1 -- 172.21.15.99:0/2019670575 shutdown_connections 2023-10-08T00:19:52.803 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.794+0000 7f68ba2cd700 1 --2- 172.21.15.99:0/2019670575 >> v2:172.21.15.191:3300/0 conn(0x7f68b415d580 0x7f68b4158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:52.803 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.794+0000 7f68ba2cd700 1 --2- 172.21.15.99:0/2019670575 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f68b4157870 0x7f68b415b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:52.803 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.794+0000 7f68ba2cd700 1 --2- 172.21.15.99:0/2019670575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68b415a620 0x7f68b415d040 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:52.804 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.794+0000 7f68ba2cd700 1 -- 172.21.15.99:0/2019670575 >> 172.21.15.99:0/2019670575 conn(0x7f68b4068ce0 msgr2=0x7f68b40690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:19:52.804 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.794+0000 7f68ba2cd700 1 -- 172.21.15.99:0/2019670575 shutdown_connections 2023-10-08T00:19:52.804 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.794+0000 7f68ba2cd700 1 -- 172.21.15.99:0/2019670575 wait complete. 2023-10-08T00:19:52.804 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.795+0000 7f68ba2cd700 1 Processor -- start 2023-10-08T00:19:52.804 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.795+0000 7f68ba2cd700 1 -- start start 2023-10-08T00:19:52.804 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.796+0000 7f68ba2cd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f68b4157870 0x7f68b40cf950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:52.804 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.796+0000 7f68ba2cd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f68b415a620 0x7f68b40b0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:52.804 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.796+0000 7f68ba2cd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68b415d580 0x7f68b40b1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:52.805 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.796+0000 7f68ba2cd700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b40d0ba0 con 0x7f68b415d580 2023-10-08T00:19:52.805 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.796+0000 7f68ba2cd700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b40d0020 con 0x7f68b4157870 2023-10-08T00:19:52.805 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.796+0000 7f68ba2cd700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68b40d0300 con 0x7f68b415a620 2023-10-08T00:19:52.805 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.796+0000 7f68b886a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68b415d580 0x7f68b40b1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:52.805 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.796+0000 7f68b886a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68b415d580 0x7f68b40b1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44322/0 (socket says 172.21.15.99:44322) 2023-10-08T00:19:52.805 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.796+0000 7f68b886a700 1 -- 172.21.15.99:0/2135994182 learned_addr learned my addr 172.21.15.99:0/2135994182 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:19:52.805 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.796+0000 7f68b3fff700 1 --2- 172.21.15.99:0/2135994182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f68b4157870 0x7f68b40cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:52.805 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.796+0000 7f68b37fe700 1 --2- 172.21.15.99:0/2135994182 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f68b415a620 0x7f68b40b0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:52.805 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.796+0000 7f68b886a700 1 -- 172.21.15.99:0/2135994182 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f68b415a620 msgr2=0x7f68b40b0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:52.805 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.796+0000 7f68b886a700 1 --2- 172.21.15.99:0/2135994182 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f68b415a620 0x7f68b40b0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:52.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.796+0000 7f68b886a700 1 -- 172.21.15.99:0/2135994182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f68b4157870 msgr2=0x7f68b40cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:52.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.796+0000 7f68b886a700 1 --2- 172.21.15.99:0/2135994182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f68b4157870 0x7f68b40cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:52.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.796+0000 7f68b886a700 1 -- 172.21.15.99:0/2135994182 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a80094f0 con 0x7f68b415d580 2023-10-08T00:19:52.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.796+0000 7f68b3fff700 1 --2- 172.21.15.99:0/2135994182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f68b4157870 0x7f68b40cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:19:52.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.796+0000 7f68b886a700 1 --2- 172.21.15.99:0/2135994182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68b415d580 0x7f68b40b1030 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f68ac007ef0 tx=0x7f68ac009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:52.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.796+0000 7f689bfff700 1 -- 172.21.15.99:0/2135994182 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ac00ec10 con 0x7f68b415d580 2023-10-08T00:19:52.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.796+0000 7f689bfff700 1 -- 172.21.15.99:0/2135994182 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68ac00ed70 con 0x7f68b415d580 2023-10-08T00:19:52.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.796+0000 7f68ba2cd700 1 -- 172.21.15.99:0/2135994182 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68b40d05e0 con 0x7f68b415d580 2023-10-08T00:19:52.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.796+0000 7f689bfff700 1 -- 172.21.15.99:0/2135994182 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ac014670 con 0x7f68b415d580 2023-10-08T00:19:52.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.796+0000 7f68ba2cd700 1 -- 172.21.15.99:0/2135994182 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68b40b1570 con 0x7f68b415d580 2023-10-08T00:19:52.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.797+0000 7f68ba2cd700 1 -- 172.21.15.99:0/2135994182 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68b404c3c0 con 0x7f68b415d580 2023-10-08T00:19:52.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.798+0000 7f689bfff700 1 -- 172.21.15.99:0/2135994182 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f68ac014a60 con 0x7f68b415d580 2023-10-08T00:19:52.808 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.798+0000 7f689bfff700 1 --2- 172.21.15.99:0/2135994182 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f68a0066f30 0x7f68a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:52.808 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.798+0000 7f689bfff700 1 -- 172.21.15.99:0/2135994182 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f68ac084c20 con 0x7f68b415d580 2023-10-08T00:19:52.808 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.798+0000 7f68b3fff700 1 --2- 172.21.15.99:0/2135994182 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f68a0066f30 0x7f68a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:52.808 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.798+0000 7f689bfff700 1 -- 172.21.15.99:0/2135994182 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=157}) v3 -- 0x7f68a006ea50 con 0x7f68b415d580 2023-10-08T00:19:52.808 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.799+0000 7f68b3fff700 1 --2- 172.21.15.99:0/2135994182 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f68a0066f30 0x7f68a00693e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f68a4006fd0 tx=0x7f68a4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:52.809 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.802+0000 7f689bfff700 1 -- 172.21.15.99:0/2135994182 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f68ac051e90 con 0x7f68b415d580 2023-10-08T00:19:52.986 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.986+0000 7f68ba2cd700 1 -- 172.21.15.99:0/2135994182 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f68b415c840 con 0x7f68b415d580 2023-10-08T00:19:52.987 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.987+0000 7f689bfff700 1 -- 172.21.15.99:0/2135994182 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) v1 ==== 74+0+15391 (secure 0 0 0) 0x7f68ac070c30 con 0x7f68b415d580 2023-10-08T00:19:52.987 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:19:52.988 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":156,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:19:11.138430+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:17:37.936880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-08T00:17:38.577281+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:19:52.989 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.989+0000 7f68ba2cd700 1 -- 172.21.15.99:0/2135994182 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f68a0066f30 msgr2=0x7f68a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:52.990 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.989+0000 7f68ba2cd700 1 --2- 172.21.15.99:0/2135994182 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f68a0066f30 0x7f68a00693e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f68a4006fd0 tx=0x7f68a4008040 comp rx=0 tx=0).stop 2023-10-08T00:19:52.990 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.989+0000 7f68ba2cd700 1 -- 172.21.15.99:0/2135994182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68b415d580 msgr2=0x7f68b40b1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:52.990 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.989+0000 7f68ba2cd700 1 --2- 172.21.15.99:0/2135994182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68b415d580 0x7f68b40b1030 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f68ac007ef0 tx=0x7f68ac009840 comp rx=0 tx=0).stop 2023-10-08T00:19:52.990 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.989+0000 7f68ba2cd700 1 -- 172.21.15.99:0/2135994182 shutdown_connections 2023-10-08T00:19:52.990 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.989+0000 7f68ba2cd700 1 --2- 172.21.15.99:0/2135994182 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f68a0066f30 0x7f68a00693e0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:52.990 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.989+0000 7f68ba2cd700 1 --2- 172.21.15.99:0/2135994182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f68b4157870 0x7f68b40cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:52.990 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.989+0000 7f68ba2cd700 1 --2- 172.21.15.99:0/2135994182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68b415d580 0x7f68b40b1030 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:52.990 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.989+0000 7f68ba2cd700 1 --2- 172.21.15.99:0/2135994182 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f68b415a620 0x7f68b40b0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:52.990 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.989+0000 7f68ba2cd700 1 -- 172.21.15.99:0/2135994182 >> 172.21.15.99:0/2135994182 conn(0x7f68b4068ce0 msgr2=0x7f68b415da10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:19:52.991 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.989+0000 7f68ba2cd700 1 -- 172.21.15.99:0/2135994182 shutdown_connections 2023-10-08T00:19:52.991 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:52.989+0000 7f68ba2cd700 1 -- 172.21.15.99:0/2135994182 wait complete. 2023-10-08T00:19:53.001 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T00:19:53.002 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:19:53.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.115+0000 7fcb5f93f700 1 Processor -- start 2023-10-08T00:19:53.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.115+0000 7fcb5f93f700 1 -- start start 2023-10-08T00:19:53.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.116+0000 7fcb5f93f700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fcb58157870 0x7fcb5815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:53.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.116+0000 7fcb5f93f700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fcb5815a620 0x7fcb5815d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:53.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.116+0000 7fcb5f93f700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb5815cbb0 con 0x7fcb5815d580 2023-10-08T00:19:53.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.116+0000 7fcb5f93f700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb5815bfa0 con 0x7fcb5815a620 2023-10-08T00:19:53.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.116+0000 7fcb5f93f700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb5815c280 con 0x7fcb58157870 2023-10-08T00:19:53.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.116+0000 7fcb5ceda700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fcb5815a620 0x7fcb5815d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:53.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.116+0000 7fcb5ceda700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fcb5815a620 0x7fcb5815d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44338/0 (socket says 172.21.15.99:44338) 2023-10-08T00:19:53.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.116+0000 7fcb5ceda700 1 -- 172.21.15.99:0/2870332534 learned_addr learned my addr 172.21.15.99:0/2870332534 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:19:53.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.116+0000 7fcb5d6db700 1 --2- 172.21.15.99:0/2870332534 >> v2:172.21.15.191:3300/0 conn(0x7fcb58157870 0x7fcb5815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:53.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.116+0000 7fcb5ceda700 1 -- 172.21.15.99:0/2870332534 >> v2:172.21.15.191:3300/0 conn(0x7fcb58157870 msgr2=0x7fcb5815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:19:53.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.116+0000 7fcb5ceda700 1 --2- 172.21.15.99:0/2870332534 >> v2:172.21.15.191:3300/0 conn(0x7fcb58157870 0x7fcb5815b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:53.119 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.116+0000 7fcb5ceda700 1 -- 172.21.15.99:0/2870332534 >> v1:172.21.15.191:6789/0 conn(0x7fcb5815d580 legacy=0x7fcb58158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:19:53.119 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.116+0000 7fcb5ceda700 1 -- 172.21.15.99:0/2870332534 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb5815c560 con 0x7fcb5815a620 2023-10-08T00:19:53.119 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.116+0000 7fcb5ceda700 1 --2- 172.21.15.99:0/2870332534 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb5815a620 0x7fcb5815d040 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fcb58069d80 tx=0x7fcb48009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=758dd4160e88512f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:53.119 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.116+0000 7fcb4ffff700 1 -- 172.21.15.99:0/2870332534 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4800b820 con 0x7fcb5815a620 2023-10-08T00:19:53.119 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.116+0000 7fcb4ffff700 1 -- 172.21.15.99:0/2870332534 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb4800b980 con 0x7fcb5815a620 2023-10-08T00:19:53.119 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.116+0000 7fcb4ffff700 1 -- 172.21.15.99:0/2870332534 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4800bd20 con 0x7fcb5815a620 2023-10-08T00:19:53.119 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.117+0000 7fcb5f93f700 1 -- 172.21.15.99:0/2870332534 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb5815a620 msgr2=0x7fcb5815d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:53.119 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.117+0000 7fcb5f93f700 1 --2- 172.21.15.99:0/2870332534 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb5815a620 0x7fcb5815d040 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fcb58069d80 tx=0x7fcb48009250 comp rx=0 tx=0).stop 2023-10-08T00:19:53.119 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.117+0000 7fcb5f93f700 1 -- 172.21.15.99:0/2870332534 shutdown_connections 2023-10-08T00:19:53.119 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.117+0000 7fcb5f93f700 1 --2- 172.21.15.99:0/2870332534 >> v2:172.21.15.191:3300/0 conn(0x7fcb58157870 0x7fcb5815b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:53.119 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.117+0000 7fcb5f93f700 1 --2- 172.21.15.99:0/2870332534 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb5815a620 0x7fcb5815d040 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:53.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.117+0000 7fcb5f93f700 1 -- 172.21.15.99:0/2870332534 >> 172.21.15.99:0/2870332534 conn(0x7fcb58068ce0 msgr2=0x7fcb580690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:19:53.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.117+0000 7fcb5f93f700 1 -- 172.21.15.99:0/2870332534 shutdown_connections 2023-10-08T00:19:53.120 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.117+0000 7fcb5f93f700 1 -- 172.21.15.99:0/2870332534 wait complete. 2023-10-08T00:19:53.123 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.118+0000 7fcb5f93f700 1 Processor -- start 2023-10-08T00:19:53.123 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.118+0000 7fcb5f93f700 1 -- start start 2023-10-08T00:19:53.123 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.119+0000 7fcb5f93f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb58157870 0x7fcb580b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:53.123 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.119+0000 7fcb5f93f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb5815a620 0x7fcb580cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:53.123 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.119+0000 7fcb5f93f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb5815d580 0x7fcb5814e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:53.123 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.119+0000 7fcb5f93f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb580cf7f0 con 0x7fcb58157870 2023-10-08T00:19:53.123 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.119+0000 7fcb5f93f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5814e690 con 0x7fcb5815d580 2023-10-08T00:19:53.123 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.119+0000 7fcb5f93f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb580cfa90 con 0x7fcb5815a620 2023-10-08T00:19:53.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.119+0000 7fcb5d6db700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb58157870 0x7fcb580b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:53.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.119+0000 7fcb5d6db700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb58157870 0x7fcb580b2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44348/0 (socket says 172.21.15.99:44348) 2023-10-08T00:19:53.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.119+0000 7fcb5d6db700 1 -- 172.21.15.99:0/1773555921 learned_addr learned my addr 172.21.15.99:0/1773555921 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:19:53.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.119+0000 7fcb5ceda700 1 --2- 172.21.15.99:0/1773555921 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb5815a620 0x7fcb580cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:53.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.119+0000 7fcb5dedc700 1 --2- 172.21.15.99:0/1773555921 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb5815d580 0x7fcb5814e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:53.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.119+0000 7fcb5d6db700 1 -- 172.21.15.99:0/1773555921 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb5815a620 msgr2=0x7fcb580cf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:53.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.119+0000 7fcb5d6db700 1 --2- 172.21.15.99:0/1773555921 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb5815a620 0x7fcb580cf2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:53.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.119+0000 7fcb5d6db700 1 -- 172.21.15.99:0/1773555921 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb5815d580 msgr2=0x7fcb5814e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:53.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.119+0000 7fcb5d6db700 1 --2- 172.21.15.99:0/1773555921 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb5815d580 0x7fcb5814e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:53.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.119+0000 7fcb5d6db700 1 -- 172.21.15.99:0/1773555921 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb480094f0 con 0x7fcb58157870 2023-10-08T00:19:53.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.119+0000 7fcb5dedc700 1 --2- 172.21.15.99:0/1773555921 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb5815d580 0x7fcb5814e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:19:53.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.119+0000 7fcb5ceda700 1 --2- 172.21.15.99:0/1773555921 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb5815a620 0x7fcb580cf2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:19:53.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.120+0000 7fcb5d6db700 1 --2- 172.21.15.99:0/1773555921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb58157870 0x7fcb580b2e10 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fcb5400c7d0 tx=0x7fcb54008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:53.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.120+0000 7fcb4d7fa700 1 -- 172.21.15.99:0/1773555921 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb54011600 con 0x7fcb58157870 2023-10-08T00:19:53.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.120+0000 7fcb4d7fa700 1 -- 172.21.15.99:0/1773555921 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb54011760 con 0x7fcb58157870 2023-10-08T00:19:53.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.120+0000 7fcb5f93f700 1 -- 172.21.15.99:0/1773555921 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb580cfd70 con 0x7fcb58157870 2023-10-08T00:19:53.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.120+0000 7fcb4d7fa700 1 -- 172.21.15.99:0/1773555921 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb54011b20 con 0x7fcb58157870 2023-10-08T00:19:53.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.120+0000 7fcb5f93f700 1 -- 172.21.15.99:0/1773555921 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb580b0860 con 0x7fcb58157870 2023-10-08T00:19:53.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.121+0000 7fcb5f93f700 1 -- 172.21.15.99:0/1773555921 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb580d0290 con 0x7fcb58157870 2023-10-08T00:19:53.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.121+0000 7fcb4d7fa700 1 -- 172.21.15.99:0/1773555921 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcb540193f0 con 0x7fcb58157870 2023-10-08T00:19:53.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.121+0000 7fcb4d7fa700 1 --2- 172.21.15.99:0/1773555921 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcb3c066f30 0x7fcb3c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:53.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.121+0000 7fcb4d7fa700 1 -- 172.21.15.99:0/1773555921 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcb54084930 con 0x7fcb58157870 2023-10-08T00:19:53.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.121+0000 7fcb5ceda700 1 --2- 172.21.15.99:0/1773555921 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcb3c066f30 0x7fcb3c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:53.126 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.122+0000 7fcb4d7fa700 1 -- 172.21.15.99:0/1773555921 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=157}) v3 -- 0x7fcb3c06ea50 con 0x7fcb58157870 2023-10-08T00:19:53.126 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.122+0000 7fcb5ceda700 1 --2- 172.21.15.99:0/1773555921 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcb3c066f30 0x7fcb3c0693e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fcb58069d80 tx=0x7fcb48016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:53.126 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.125+0000 7fcb4d7fa700 1 -- 172.21.15.99:0/1773555921 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcb580d0290 con 0x7fcb58157870 2023-10-08T00:19:53.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.349+0000 7fcb5f93f700 1 -- 172.21.15.99:0/1773555921 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb580b06c0 con 0x7fcb58157870 2023-10-08T00:19:53.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.349+0000 7fcb4d7fa700 1 -- 172.21.15.99:0/1773555921 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4599 (secure 0 0 0) 0x7fcb580b06c0 con 0x7fcb58157870 2023-10-08T00:19:53.350 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:19:53.350 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":70,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6056,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/3025449992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3025449992},{"type":"v1","addr":"172.21.15.191:6836","nonce":3025449992}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":7476,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/86350138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":86350138},{"type":"v1","addr":"172.21.15.99:6834","nonce":86350138}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":7479,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1282887036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1282887036},{"type":"v1","addr":"172.21.15.99:6835","nonce":1282887036}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:17:39.623734+0000","modified":"2023-10-08T00:17:43.252633+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7494":{"gid":7494,"name":"d","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/1165162515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":1165162515},{"type":"v1","addr":"172.21.15.191:6837","nonce":1165162515}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-08T00:19:53.352 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.351+0000 7fcb5f93f700 1 -- 172.21.15.99:0/1773555921 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcb3c066f30 msgr2=0x7fcb3c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:53.352 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.351+0000 7fcb5f93f700 1 --2- 172.21.15.99:0/1773555921 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcb3c066f30 0x7fcb3c0693e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fcb58069d80 tx=0x7fcb48016000 comp rx=0 tx=0).stop 2023-10-08T00:19:53.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.351+0000 7fcb5f93f700 1 -- 172.21.15.99:0/1773555921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb58157870 msgr2=0x7fcb580b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:53.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.351+0000 7fcb5f93f700 1 --2- 172.21.15.99:0/1773555921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb58157870 0x7fcb580b2e10 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fcb5400c7d0 tx=0x7fcb54008040 comp rx=0 tx=0).stop 2023-10-08T00:19:53.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.352+0000 7fcb5f93f700 1 -- 172.21.15.99:0/1773555921 shutdown_connections 2023-10-08T00:19:53.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.352+0000 7fcb5f93f700 1 --2- 172.21.15.99:0/1773555921 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcb3c066f30 0x7fcb3c0693e0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:53.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.352+0000 7fcb5f93f700 1 --2- 172.21.15.99:0/1773555921 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb5815d580 0x7fcb5814e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:53.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.352+0000 7fcb5f93f700 1 --2- 172.21.15.99:0/1773555921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb58157870 0x7fcb580b2e10 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:53.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.352+0000 7fcb5f93f700 1 --2- 172.21.15.99:0/1773555921 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb5815a620 0x7fcb580cf2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:53.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.352+0000 7fcb5f93f700 1 -- 172.21.15.99:0/1773555921 >> 172.21.15.99:0/1773555921 conn(0x7fcb58068ce0 msgr2=0x7fcb58159330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:19:53.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.352+0000 7fcb5f93f700 1 -- 172.21.15.99:0/1773555921 shutdown_connections 2023-10-08T00:19:53.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.352+0000 7fcb5f93f700 1 -- 172.21.15.99:0/1773555921 wait complete. 2023-10-08T00:19:53.354 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 70 2023-10-08T00:19:53.364 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:19:53.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.477+0000 7f26baaf3700 1 Processor -- start 2023-10-08T00:19:53.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.477+0000 7f26baaf3700 1 -- start start 2023-10-08T00:19:53.480 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.478+0000 7f26baaf3700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f26b40cd6e0 0x7f26b40cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:53.480 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.478+0000 7f26baaf3700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f26b40cdab0 0x7f26b415d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:53.480 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.478+0000 7f26baaf3700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26b40cb7c0 con 0x7f26b415dda0 2023-10-08T00:19:53.480 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.478+0000 7f26baaf3700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26b415e850 con 0x7f26b40cdab0 2023-10-08T00:19:53.480 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.478+0000 7f26baaf3700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26b40cd030 con 0x7f26b40cd6e0 2023-10-08T00:19:53.480 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.478+0000 7f26b92f0700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f26b40cdab0 0x7f26b415d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:53.480 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.478+0000 7f26b92f0700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f26b40cdab0 0x7f26b415d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40622/0 (socket says 172.21.15.99:40622) 2023-10-08T00:19:53.480 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.478+0000 7f26b92f0700 1 -- 172.21.15.99:0/1684327236 learned_addr learned my addr 172.21.15.99:0/1684327236 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:19:53.481 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.478+0000 7f26b9af1700 1 --2- 172.21.15.99:0/1684327236 >> v2:172.21.15.191:3300/0 conn(0x7f26b40cd6e0 0x7f26b40cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:53.481 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.478+0000 7f26b92f0700 1 -- 172.21.15.99:0/1684327236 >> v2:172.21.15.191:3300/0 conn(0x7f26b40cd6e0 msgr2=0x7f26b40cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:19:53.481 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.478+0000 7f26b92f0700 1 --2- 172.21.15.99:0/1684327236 >> v2:172.21.15.191:3300/0 conn(0x7f26b40cd6e0 0x7f26b40cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:53.481 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.478+0000 7f26b92f0700 1 -- 172.21.15.99:0/1684327236 >> v1:172.21.15.191:6789/0 conn(0x7f26b415dda0 legacy=0x7f26b415cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:19:53.481 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.478+0000 7f26b92f0700 1 -- 172.21.15.99:0/1684327236 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b415e290 con 0x7f26b40cdab0 2023-10-08T00:19:53.481 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.478+0000 7f26b92f0700 1 --2- 172.21.15.99:0/1684327236 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26b40cdab0 0x7f26b415d860 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f26b406b730 tx=0x7f26a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90bbefab01fa286f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:53.481 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.479+0000 7f26b8aef700 1 -- 172.21.15.99:0/1684327236 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a800b820 con 0x7f26b40cdab0 2023-10-08T00:19:53.481 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.479+0000 7f26b8aef700 1 -- 172.21.15.99:0/1684327236 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26a800b980 con 0x7f26b40cdab0 2023-10-08T00:19:53.481 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.479+0000 7f26b8aef700 1 -- 172.21.15.99:0/1684327236 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a800bd20 con 0x7f26b40cdab0 2023-10-08T00:19:53.482 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.479+0000 7f26baaf3700 1 -- 172.21.15.99:0/1684327236 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26b40cdab0 msgr2=0x7f26b415d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:53.482 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.479+0000 7f26baaf3700 1 --2- 172.21.15.99:0/1684327236 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26b40cdab0 0x7f26b415d860 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f26b406b730 tx=0x7f26a8009250 comp rx=0 tx=0).stop 2023-10-08T00:19:53.482 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.479+0000 7f26baaf3700 1 -- 172.21.15.99:0/1684327236 shutdown_connections 2023-10-08T00:19:53.482 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.479+0000 7f26baaf3700 1 --2- 172.21.15.99:0/1684327236 >> v2:172.21.15.191:3300/0 conn(0x7f26b40cd6e0 0x7f26b40cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:53.482 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.479+0000 7f26baaf3700 1 --2- 172.21.15.99:0/1684327236 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26b40cdab0 0x7f26b415d860 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:53.482 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.479+0000 7f26baaf3700 1 -- 172.21.15.99:0/1684327236 >> 172.21.15.99:0/1684327236 conn(0x7f26b4067660 msgr2=0x7f26b4067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:19:53.482 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.479+0000 7f26baaf3700 1 -- 172.21.15.99:0/1684327236 shutdown_connections 2023-10-08T00:19:53.482 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.479+0000 7f26baaf3700 1 -- 172.21.15.99:0/1684327236 wait complete. 2023-10-08T00:19:53.485 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.480+0000 7f26baaf3700 1 Processor -- start 2023-10-08T00:19:53.485 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.480+0000 7f26baaf3700 1 -- start start 2023-10-08T00:19:53.485 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.481+0000 7f26baaf3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26b40cd6e0 0x7f26b415b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:53.485 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.481+0000 7f26baaf3700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26b40cdab0 0x7f26b4155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:53.485 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.481+0000 7f26baaf3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26b415dda0 0x7f26b40c98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:53.485 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.481+0000 7f26baaf3700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b4156520 con 0x7f26b415dda0 2023-10-08T00:19:53.486 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.481+0000 7f26baaf3700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b4158c60 con 0x7f26b40cdab0 2023-10-08T00:19:53.486 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.481+0000 7f26baaf3700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26b415c020 con 0x7f26b40cd6e0 2023-10-08T00:19:53.486 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.481+0000 7f26ba2f2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26b415dda0 0x7f26b40c98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:53.486 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.481+0000 7f26ba2f2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26b415dda0 0x7f26b40c98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40626/0 (socket says 172.21.15.99:40626) 2023-10-08T00:19:53.486 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.481+0000 7f26ba2f2700 1 -- 172.21.15.99:0/3329485528 learned_addr learned my addr 172.21.15.99:0/3329485528 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:19:53.486 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.481+0000 7f26b9af1700 1 --2- 172.21.15.99:0/3329485528 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26b40cd6e0 0x7f26b415b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:53.486 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.481+0000 7f26b92f0700 1 --2- 172.21.15.99:0/3329485528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26b40cdab0 0x7f26b4155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:53.486 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.481+0000 7f26ba2f2700 1 -- 172.21.15.99:0/3329485528 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26b40cd6e0 msgr2=0x7f26b415b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:53.486 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.481+0000 7f26ba2f2700 1 --2- 172.21.15.99:0/3329485528 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26b40cd6e0 0x7f26b415b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:53.486 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.481+0000 7f26ba2f2700 1 -- 172.21.15.99:0/3329485528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26b40cdab0 msgr2=0x7f26b4155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:53.486 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.481+0000 7f26ba2f2700 1 --2- 172.21.15.99:0/3329485528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26b40cdab0 0x7f26b4155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:53.486 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.481+0000 7f26ba2f2700 1 -- 172.21.15.99:0/3329485528 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a80094f0 con 0x7f26b415dda0 2023-10-08T00:19:53.487 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.482+0000 7f26ba2f2700 1 --2- 172.21.15.99:0/3329485528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26b415dda0 0x7f26b40c98f0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f26a400bb60 tx=0x7f26a40097f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:53.487 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.482+0000 7f26b1ffb700 1 -- 172.21.15.99:0/3329485528 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a400c7e0 con 0x7f26b415dda0 2023-10-08T00:19:53.487 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.482+0000 7f26b1ffb700 1 -- 172.21.15.99:0/3329485528 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26a400c940 con 0x7f26b415dda0 2023-10-08T00:19:53.487 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.482+0000 7f26baaf3700 1 -- 172.21.15.99:0/3329485528 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26b415c300 con 0x7f26b415dda0 2023-10-08T00:19:53.487 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.482+0000 7f26b1ffb700 1 -- 172.21.15.99:0/3329485528 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a400cd00 con 0x7f26b415dda0 2023-10-08T00:19:53.487 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.482+0000 7f26baaf3700 1 -- 172.21.15.99:0/3329485528 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26b41588f0 con 0x7f26b415dda0 2023-10-08T00:19:53.487 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.483+0000 7f26baaf3700 1 -- 172.21.15.99:0/3329485528 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26b4089280 con 0x7f26b415dda0 2023-10-08T00:19:53.487 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.483+0000 7f26b1ffb700 1 -- 172.21.15.99:0/3329485528 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f26a4014620 con 0x7f26b415dda0 2023-10-08T00:19:53.487 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.484+0000 7f26b1ffb700 1 --2- 172.21.15.99:0/3329485528 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f26a0066fb0 0x7f26a0069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:53.487 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.484+0000 7f26b1ffb700 1 -- 172.21.15.99:0/3329485528 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f26a4084560 con 0x7f26b415dda0 2023-10-08T00:19:53.487 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.484+0000 7f26b9af1700 1 --2- 172.21.15.99:0/3329485528 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f26a0066fb0 0x7f26a0069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:53.487 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.484+0000 7f26b1ffb700 1 -- 172.21.15.99:0/3329485528 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=157}) v3 -- 0x7f26a006ead0 con 0x7f26b415dda0 2023-10-08T00:19:53.488 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.484+0000 7f26b9af1700 1 --2- 172.21.15.99:0/3329485528 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f26a0066fb0 0x7f26a0069460 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f26ac006fd0 tx=0x7f26ac008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:53.488 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.488+0000 7f26b1ffb700 1 -- 172.21.15.99:0/3329485528 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f26a4051730 con 0x7f26b415dda0 2023-10-08T00:19:53.710 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.709+0000 7f26baaf3700 1 -- 172.21.15.99:0/3329485528 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f26b415e570 con 0x7f26b415dda0 2023-10-08T00:19:53.710 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.710+0000 7f26b1ffb700 1 -- 172.21.15.99:0/3329485528 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4599 (secure 0 0 0) 0x7f26a400ce60 con 0x7f26b415dda0 2023-10-08T00:19:53.710 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:19:53.711 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":70,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6056,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/3025449992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3025449992},{"type":"v1","addr":"172.21.15.191:6836","nonce":3025449992}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":7476,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/86350138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":86350138},{"type":"v1","addr":"172.21.15.99:6834","nonce":86350138}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":7479,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1282887036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1282887036},{"type":"v1","addr":"172.21.15.99:6835","nonce":1282887036}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:17:39.623734+0000","modified":"2023-10-08T00:17:43.252633+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7494":{"gid":7494,"name":"d","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/1165162515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":1165162515},{"type":"v1","addr":"172.21.15.191:6837","nonce":1165162515}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-08T00:19:53.713 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.712+0000 7f26baaf3700 1 -- 172.21.15.99:0/3329485528 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f26a0066fb0 msgr2=0x7f26a0069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:53.713 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.712+0000 7f26baaf3700 1 --2- 172.21.15.99:0/3329485528 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f26a0066fb0 0x7f26a0069460 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f26ac006fd0 tx=0x7f26ac008040 comp rx=0 tx=0).stop 2023-10-08T00:19:53.713 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.712+0000 7f26baaf3700 1 -- 172.21.15.99:0/3329485528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26b415dda0 msgr2=0x7f26b40c98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:53.713 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.712+0000 7f26baaf3700 1 --2- 172.21.15.99:0/3329485528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26b415dda0 0x7f26b40c98f0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f26a400bb60 tx=0x7f26a40097f0 comp rx=0 tx=0).stop 2023-10-08T00:19:53.713 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.712+0000 7f26baaf3700 1 -- 172.21.15.99:0/3329485528 shutdown_connections 2023-10-08T00:19:53.713 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.712+0000 7f26baaf3700 1 --2- 172.21.15.99:0/3329485528 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f26a0066fb0 0x7f26a0069460 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:53.713 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.712+0000 7f26baaf3700 1 --2- 172.21.15.99:0/3329485528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26b415dda0 0x7f26b40c98f0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:53.714 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.712+0000 7f26baaf3700 1 --2- 172.21.15.99:0/3329485528 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26b40cd6e0 0x7f26b415b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:53.714 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.712+0000 7f26baaf3700 1 --2- 172.21.15.99:0/3329485528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26b40cdab0 0x7f26b4155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:53.714 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.712+0000 7f26baaf3700 1 -- 172.21.15.99:0/3329485528 >> 172.21.15.99:0/3329485528 conn(0x7f26b4067660 msgr2=0x7f26b40c62b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:19:53.714 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.712+0000 7f26baaf3700 1 -- 172.21.15.99:0/3329485528 shutdown_connections 2023-10-08T00:19:53.714 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.712+0000 7f26baaf3700 1 -- 172.21.15.99:0/3329485528 wait complete. 2023-10-08T00:19:53.714 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 70 2023-10-08T00:19:53.726 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:19:53.800 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:19:53.802 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:19:53.850 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.841+0000 7f65d56d7700 1 Processor -- start 2023-10-08T00:19:53.850 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.841+0000 7f65d56d7700 1 -- start start 2023-10-08T00:19:53.850 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.842+0000 7f65d56d7700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f65d0070c90 0x7f65d0068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:53.850 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.842+0000 7f65d56d7700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65d00710b0 con 0x7f65d00afcf0 2023-10-08T00:19:53.850 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.842+0000 7f65d56d7700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65d015dd90 con 0x7f65d00708c0 2023-10-08T00:19:53.850 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.842+0000 7f65d56d7700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65d006f1b0 con 0x7f65d0070c90 2023-10-08T00:19:53.850 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.842+0000 7f65cf7fe700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f65d00afcf0 0x7f65d006eaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:54484/0 (socket says 172.21.15.99:54484) 2023-10-08T00:19:53.850 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.842+0000 7f65cf7fe700 1 -- 172.21.15.99:0/3570070906 learned_addr learned my addr 172.21.15.99:0/3570070906 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:19:53.851 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.842+0000 7f65ceffd700 1 -- 172.21.15.99:0/3570070906 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2788707195 0 0) 0x7f65d00710b0 con 0x7f65d00afcf0 2023-10-08T00:19:53.851 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.842+0000 7f65ceffd700 1 -- 172.21.15.99:0/3570070906 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65b8003570 con 0x7f65d00afcf0 2023-10-08T00:19:53.851 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.842+0000 7f65d4ed6700 1 --2- 172.21.15.99:0/3570070906 >> v2:172.21.15.191:3300/0 conn(0x7f65d0070c90 0x7f65d0068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:53.851 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.843+0000 7f65ceffd700 1 -- 172.21.15.99:0/3570070906 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4019797997 0 0) 0x7f65b8003570 con 0x7f65d00afcf0 2023-10-08T00:19:53.851 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.843+0000 7f65ceffd700 1 -- 172.21.15.99:0/3570070906 >> v2:172.21.15.191:3300/0 conn(0x7f65d0070c90 msgr2=0x7f65d0068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:19:53.851 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.843+0000 7f65ceffd700 1 --2- 172.21.15.99:0/3570070906 >> v2:172.21.15.191:3300/0 conn(0x7f65d0070c90 0x7f65d0068670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:53.851 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.843+0000 7f65ceffd700 1 -- 172.21.15.99:0/3570070906 >> v1:172.21.15.191:6789/0 conn(0x7f65d00708c0 legacy=0x7f65d015e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:53.851 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.843+0000 7f65ceffd700 1 -- 172.21.15.99:0/3570070906 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d006f960 con 0x7f65d00afcf0 2023-10-08T00:19:53.851 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.843+0000 7f65ceffd700 1 -- 172.21.15.99:0/3570070906 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f65c4001470 con 0x7f65d00afcf0 2023-10-08T00:19:53.851 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.843+0000 7f65ceffd700 1 -- 172.21.15.99:0/3570070906 >> v1:172.21.15.99:6789/0 conn(0x7f65d00afcf0 legacy=0x7f65d006eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:53.851 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.843+0000 7f65ceffd700 1 --2- 172.21.15.99:0/3570070906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65b8003a60 0x7f65b8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:53.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.843+0000 7f65ceffd700 1 --2- 172.21.15.99:0/3570070906 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65b8004810 0x7f65b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:53.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.843+0000 7f65ceffd700 1 --2- 172.21.15.99:0/3570070906 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65b80051c0 0x7f65b800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:53.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.843+0000 7f65ceffd700 1 -- 172.21.15.99:0/3570070906 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b800c130 con 0x7f65b8003a60 2023-10-08T00:19:53.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.843+0000 7f65ceffd700 1 -- 172.21.15.99:0/3570070906 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b800c2a0 con 0x7f65b80051c0 2023-10-08T00:19:53.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.843+0000 7f65ceffd700 1 -- 172.21.15.99:0/3570070906 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65b800c460 con 0x7f65b8004810 2023-10-08T00:19:53.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.843+0000 7f65cffff700 1 --2- 172.21.15.99:0/3570070906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65b8003a60 0x7f65b8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:53.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.843+0000 7f65cf7fe700 1 --2- 172.21.15.99:0/3570070906 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65b8004810 0x7f65b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:53.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.843+0000 7f65d4ed6700 1 --2- 172.21.15.99:0/3570070906 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65b80051c0 0x7f65b800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:53.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.843+0000 7f65cffff700 1 -- 172.21.15.99:0/3570070906 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65b8004810 msgr2=0x7f65b8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:19:53.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.843+0000 7f65cffff700 1 --2- 172.21.15.99:0/3570070906 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65b8004810 0x7f65b8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:53.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.843+0000 7f65cffff700 1 -- 172.21.15.99:0/3570070906 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65b80051c0 msgr2=0x7f65b800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:19:53.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.843+0000 7f65cffff700 1 --2- 172.21.15.99:0/3570070906 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65b80051c0 0x7f65b800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:53.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.843+0000 7f65cffff700 1 -- 172.21.15.99:0/3570070906 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65b8003570 con 0x7f65b8003a60 2023-10-08T00:19:53.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.843+0000 7f65d4ed6700 1 -- 172.21.15.99:0/3570070906 reap_dead start 2023-10-08T00:19:53.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.843+0000 7f65cffff700 1 --2- 172.21.15.99:0/3570070906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65b8003a60 0x7f65b8003e70 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f65c00018e0 tx=0x7f65c0007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfbf3bf108f75e4f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:53.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.843+0000 7f65ceffd700 1 -- 172.21.15.99:0/3570070906 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c0012420 con 0x7f65b8003a60 2023-10-08T00:19:53.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.843+0000 7f65ceffd700 1 -- 172.21.15.99:0/3570070906 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c00125a0 con 0x7f65b8003a60 2023-10-08T00:19:53.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.843+0000 7f65ceffd700 1 -- 172.21.15.99:0/3570070906 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c0012960 con 0x7f65b8003a60 2023-10-08T00:19:53.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.844+0000 7f65d56d7700 1 -- 172.21.15.99:0/3570070906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65b8003a60 msgr2=0x7f65b8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:53.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.844+0000 7f65d56d7700 1 --2- 172.21.15.99:0/3570070906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65b8003a60 0x7f65b8003e70 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f65c00018e0 tx=0x7f65c0007d30 comp rx=0 tx=0).stop 2023-10-08T00:19:53.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.844+0000 7f65d56d7700 1 -- 172.21.15.99:0/3570070906 shutdown_connections 2023-10-08T00:19:53.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.844+0000 7f65d56d7700 1 --2- 172.21.15.99:0/3570070906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65b8003a60 0x7f65b8003e70 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:53.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.844+0000 7f65d56d7700 1 -- 172.21.15.99:0/3570070906 >> 172.21.15.99:0/3570070906 conn(0x7f65d00cd8f0 msgr2=0x7f65d00cdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:19:53.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.844+0000 7f65d56d7700 1 -- 172.21.15.99:0/3570070906 shutdown_connections 2023-10-08T00:19:53.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.844+0000 7f65d56d7700 1 -- 172.21.15.99:0/3570070906 wait complete. 2023-10-08T00:19:53.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.845+0000 7f65d56d7700 1 Processor -- start 2023-10-08T00:19:53.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.845+0000 7f65d56d7700 1 -- start start 2023-10-08T00:19:53.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.846+0000 7f65d56d7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65b8003a60 0x7f65d01589e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:53.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.846+0000 7f65d56d7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65d016a460 0x7f65d00c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:53.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.846+0000 7f65d56d7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65d00c71d0 0x7f65d015a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:53.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.846+0000 7f65d56d7700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d015e700 con 0x7f65b8003a60 2023-10-08T00:19:53.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.846+0000 7f65d56d7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d015ecc0 con 0x7f65d00c71d0 2023-10-08T00:19:53.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.846+0000 7f65d56d7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d00c7750 con 0x7f65d016a460 2023-10-08T00:19:53.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.846+0000 7f65cffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65b8003a60 0x7f65d01589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:53.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.846+0000 7f65cffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65b8003a60 0x7f65d01589e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40656/0 (socket says 172.21.15.99:40656) 2023-10-08T00:19:53.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.846+0000 7f65cffff700 1 -- 172.21.15.99:0/1206828306 learned_addr learned my addr 172.21.15.99:0/1206828306 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:19:53.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.846+0000 7f65d4ed6700 1 --2- 172.21.15.99:0/1206828306 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65d00c71d0 0x7f65d015a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:53.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.846+0000 7f65cf7fe700 1 --2- 172.21.15.99:0/1206828306 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65d016a460 0x7f65d00c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:53.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.846+0000 7f65cffff700 1 -- 172.21.15.99:0/1206828306 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65d016a460 msgr2=0x7f65d00c6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:53.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.846+0000 7f65cffff700 1 --2- 172.21.15.99:0/1206828306 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65d016a460 0x7f65d00c6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:53.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.846+0000 7f65cffff700 1 -- 172.21.15.99:0/1206828306 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65d00c71d0 msgr2=0x7f65d015a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:53.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.846+0000 7f65cffff700 1 --2- 172.21.15.99:0/1206828306 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65d00c71d0 0x7f65d015a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:53.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.846+0000 7f65cffff700 1 -- 172.21.15.99:0/1206828306 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d00c7a30 con 0x7f65b8003a60 2023-10-08T00:19:53.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.846+0000 7f65cffff700 1 --2- 172.21.15.99:0/1206828306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65b8003a60 0x7f65d01589e0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f65c000c0e0 tx=0x7f65c0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:53.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.847+0000 7f65b7fff700 1 -- 172.21.15.99:0/1206828306 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c0001230 con 0x7f65b8003a60 2023-10-08T00:19:53.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.847+0000 7f65b7fff700 1 -- 172.21.15.99:0/1206828306 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c000ca10 con 0x7f65b8003a60 2023-10-08T00:19:53.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.847+0000 7f65d56d7700 1 -- 172.21.15.99:0/1206828306 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65d015e950 con 0x7f65b8003a60 2023-10-08T00:19:53.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.847+0000 7f65b7fff700 1 -- 172.21.15.99:0/1206828306 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c000cdb0 con 0x7f65b8003a60 2023-10-08T00:19:53.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.847+0000 7f65d56d7700 1 -- 172.21.15.99:0/1206828306 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65d00c67f0 con 0x7f65b8003a60 2023-10-08T00:19:53.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.847+0000 7f65d56d7700 1 -- 172.21.15.99:0/1206828306 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65d006d640 con 0x7f65b8003a60 2023-10-08T00:19:53.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.848+0000 7f65b7fff700 1 -- 172.21.15.99:0/1206828306 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f65c0019030 con 0x7f65b8003a60 2023-10-08T00:19:53.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.848+0000 7f65b7fff700 1 --2- 172.21.15.99:0/1206828306 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f65bc066f30 0x7f65bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:53.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.848+0000 7f65b7fff700 1 -- 172.21.15.99:0/1206828306 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f65c0085a50 con 0x7f65b8003a60 2023-10-08T00:19:53.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.848+0000 7f65b7fff700 1 -- 172.21.15.99:0/1206828306 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=157}) v3 -- 0x7f65bc06ea50 con 0x7f65b8003a60 2023-10-08T00:19:53.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.849+0000 7f65cf7fe700 1 --2- 172.21.15.99:0/1206828306 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f65bc066f30 0x7f65bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:53.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.849+0000 7f65cf7fe700 1 --2- 172.21.15.99:0/1206828306 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f65bc066f30 0x7f65bc0693e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f65c40015a0 tx=0x7f65d015dd00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:53.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:53.853+0000 7f65b7fff700 1 -- 172.21.15.99:0/1206828306 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f65c002c020 con 0x7f65b8003a60 2023-10-08T00:19:54.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.036+0000 7f65d56d7700 1 -- 172.21.15.99:0/1206828306 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f65d006f3a0 con 0x7f65b8003a60 2023-10-08T00:19:54.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.037+0000 7f65b7fff700 1 -- 172.21.15.99:0/1206828306 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) v1 ==== 74+0+15391 (secure 0 0 0) 0x7f65c0017070 con 0x7f65b8003a60 2023-10-08T00:19:54.038 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:19:54.038 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":156,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:19:11.138430+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:17:37.936880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-08T00:17:38.577281+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:19:54.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.039+0000 7f65d56d7700 1 -- 172.21.15.99:0/1206828306 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f65bc066f30 msgr2=0x7f65bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:54.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.039+0000 7f65d56d7700 1 --2- 172.21.15.99:0/1206828306 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f65bc066f30 0x7f65bc0693e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f65c40015a0 tx=0x7f65d015dd00 comp rx=0 tx=0).stop 2023-10-08T00:19:54.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.039+0000 7f65d56d7700 1 -- 172.21.15.99:0/1206828306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65b8003a60 msgr2=0x7f65d01589e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:54.040 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.039+0000 7f65d56d7700 1 --2- 172.21.15.99:0/1206828306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65b8003a60 0x7f65d01589e0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f65c000c0e0 tx=0x7f65c0015000 comp rx=0 tx=0).stop 2023-10-08T00:19:54.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.039+0000 7f65d56d7700 1 -- 172.21.15.99:0/1206828306 shutdown_connections 2023-10-08T00:19:54.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.039+0000 7f65d56d7700 1 --2- 172.21.15.99:0/1206828306 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f65bc066f30 0x7f65bc0693e0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:54.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.039+0000 7f65d56d7700 1 --2- 172.21.15.99:0/1206828306 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65d00c71d0 0x7f65d015a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:54.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.039+0000 7f65d56d7700 1 --2- 172.21.15.99:0/1206828306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65b8003a60 0x7f65d01589e0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:54.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.039+0000 7f65d56d7700 1 --2- 172.21.15.99:0/1206828306 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65d016a460 0x7f65d00c6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:54.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.039+0000 7f65d56d7700 1 -- 172.21.15.99:0/1206828306 >> 172.21.15.99:0/1206828306 conn(0x7f65d00cd8f0 msgr2=0x7f65d00cc310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:19:54.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.039+0000 7f65d56d7700 1 -- 172.21.15.99:0/1206828306 shutdown_connections 2023-10-08T00:19:54.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.039+0000 7f65d56d7700 1 -- 172.21.15.99:0/1206828306 wait complete. 2023-10-08T00:19:54.052 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T00:19:54.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.166+0000 7fe8511b5700 1 Processor -- start 2023-10-08T00:19:54.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.166+0000 7fe8511b5700 1 -- start start 2023-10-08T00:19:54.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.167+0000 7fe8511b5700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fe84c159da0 0x7fe84c0b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:54.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.167+0000 7fe8511b5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe84c069210 0x7fe84c0af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:54.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.167+0000 7fe8511b5700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe84c15ea70 con 0x7fe84c15a170 2023-10-08T00:19:54.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.167+0000 7fe8511b5700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe84c0b3990 con 0x7fe84c159da0 2023-10-08T00:19:54.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.167+0000 7fe8511b5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe84c0b0b70 con 0x7fe84c069210 2023-10-08T00:19:54.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.167+0000 7fe8509b4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe84c069210 0x7fe84c0af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:54.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.167+0000 7fe84bfff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fe84c159da0 0x7fe84c0b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:54.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.167+0000 7fe84b7fe700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7fe84c15a170 0x7fe84c068b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:36964/0 (socket says 172.21.15.99:36964) 2023-10-08T00:19:54.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.167+0000 7fe84b7fe700 1 -- 172.21.15.99:0/4123816634 learned_addr learned my addr 172.21.15.99:0/4123816634 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:19:54.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.167+0000 7fe84bfff700 1 -- 172.21.15.99:0/4123816634 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe84c069210 msgr2=0x7fe84c0af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:19:54.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.167+0000 7fe84bfff700 1 --2- 172.21.15.99:0/4123816634 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe84c069210 0x7fe84c0af4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:54.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.167+0000 7fe84bfff700 1 -- 172.21.15.99:0/4123816634 >> v1:172.21.15.191:6789/0 conn(0x7fe84c15a170 legacy=0x7fe84c068b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:54.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.167+0000 7fe84bfff700 1 -- 172.21.15.99:0/4123816634 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe84c15def0 con 0x7fe84c159da0 2023-10-08T00:19:54.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.168+0000 7fe84bfff700 1 --2- 172.21.15.99:0/4123816634 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe84c159da0 0x7fe84c0b2da0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe83c0098e0 tx=0x7fe83c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b33dc906edbd7612 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:54.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.168+0000 7fe84affd700 1 -- 172.21.15.99:0/4123816634 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83c00b640 con 0x7fe84c159da0 2023-10-08T00:19:54.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.168+0000 7fe84affd700 1 -- 172.21.15.99:0/4123816634 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe83c00b7a0 con 0x7fe84c159da0 2023-10-08T00:19:54.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.168+0000 7fe84affd700 1 -- 172.21.15.99:0/4123816634 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83c00bb40 con 0x7fe84c159da0 2023-10-08T00:19:54.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.168+0000 7fe8511b5700 1 -- 172.21.15.99:0/4123816634 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe84c159da0 msgr2=0x7fe84c0b2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:54.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.168+0000 7fe8511b5700 1 --2- 172.21.15.99:0/4123816634 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe84c159da0 0x7fe84c0b2da0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe83c0098e0 tx=0x7fe83c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:19:54.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.168+0000 7fe8511b5700 1 -- 172.21.15.99:0/4123816634 shutdown_connections 2023-10-08T00:19:54.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.168+0000 7fe8511b5700 1 --2- 172.21.15.99:0/4123816634 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe84c069210 0x7fe84c0af4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:54.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.168+0000 7fe8511b5700 1 --2- 172.21.15.99:0/4123816634 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe84c159da0 0x7fe84c0b2da0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:54.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.168+0000 7fe8511b5700 1 -- 172.21.15.99:0/4123816634 >> 172.21.15.99:0/4123816634 conn(0x7fe84c0d0bc0 msgr2=0x7fe84c0d0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:19:54.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.168+0000 7fe8511b5700 1 -- 172.21.15.99:0/4123816634 shutdown_connections 2023-10-08T00:19:54.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.168+0000 7fe8511b5700 1 -- 172.21.15.99:0/4123816634 wait complete. 2023-10-08T00:19:54.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.169+0000 7fe8511b5700 1 Processor -- start 2023-10-08T00:19:54.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.170+0000 7fe8511b5700 1 -- start start 2023-10-08T00:19:54.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.170+0000 7fe8511b5700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe84c069210 0x7fe84c15af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:54.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.171+0000 7fe8511b5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe84c159da0 0x7fe84c0c73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:54.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.171+0000 7fe8511b5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe84c15a170 0x7fe84c0c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:54.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.171+0000 7fe8511b5700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84c15b7e0 con 0x7fe84c15a170 2023-10-08T00:19:54.176 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.171+0000 7fe8511b5700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84c0c7900 con 0x7fe84c069210 2023-10-08T00:19:54.176 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.171+0000 7fe8511b5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe84c0c5e00 con 0x7fe84c159da0 2023-10-08T00:19:54.176 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.171+0000 7fe8509b4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe84c15a170 0x7fe84c0c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:54.176 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.171+0000 7fe8509b4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe84c15a170 0x7fe84c0c7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40672/0 (socket says 172.21.15.99:40672) 2023-10-08T00:19:54.176 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.171+0000 7fe8509b4700 1 -- 172.21.15.99:0/204330645 learned_addr learned my addr 172.21.15.99:0/204330645 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:19:54.176 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.171+0000 7fe84bfff700 1 --2- 172.21.15.99:0/204330645 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe84c069210 0x7fe84c15af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:54.177 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.171+0000 7fe84b7fe700 1 --2- 172.21.15.99:0/204330645 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe84c159da0 0x7fe84c0c73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:54.177 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.171+0000 7fe8509b4700 1 -- 172.21.15.99:0/204330645 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe84c159da0 msgr2=0x7fe84c0c73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:54.177 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.171+0000 7fe8509b4700 1 --2- 172.21.15.99:0/204330645 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe84c159da0 0x7fe84c0c73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:54.177 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.171+0000 7fe8509b4700 1 -- 172.21.15.99:0/204330645 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe84c069210 msgr2=0x7fe84c15af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:54.177 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.171+0000 7fe8509b4700 1 --2- 172.21.15.99:0/204330645 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe84c069210 0x7fe84c15af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:54.177 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.171+0000 7fe8509b4700 1 -- 172.21.15.99:0/204330645 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe83c009420 con 0x7fe84c15a170 2023-10-08T00:19:54.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.171+0000 7fe8509b4700 1 --2- 172.21.15.99:0/204330645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe84c15a170 0x7fe84c0c7bd0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fe844009ec0 tx=0x7fe84400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:54.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.172+0000 7fe833fff700 1 -- 172.21.15.99:0/204330645 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84400ca40 con 0x7fe84c15a170 2023-10-08T00:19:54.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.172+0000 7fe833fff700 1 -- 172.21.15.99:0/204330645 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe84400f040 con 0x7fe84c15a170 2023-10-08T00:19:54.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.172+0000 7fe8511b5700 1 -- 172.21.15.99:0/204330645 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe84c0c8110 con 0x7fe84c15a170 2023-10-08T00:19:54.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.172+0000 7fe833fff700 1 -- 172.21.15.99:0/204330645 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84400cdd0 con 0x7fe84c15a170 2023-10-08T00:19:54.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.172+0000 7fe8511b5700 1 -- 172.21.15.99:0/204330645 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe84c0c57b0 con 0x7fe84c15a170 2023-10-08T00:19:54.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.173+0000 7fe8511b5700 1 -- 172.21.15.99:0/204330645 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe84c0c5f70 con 0x7fe84c15a170 2023-10-08T00:19:54.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.173+0000 7fe833fff700 1 -- 172.21.15.99:0/204330645 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe844014690 con 0x7fe84c15a170 2023-10-08T00:19:54.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.174+0000 7fe833fff700 1 --2- 172.21.15.99:0/204330645 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe838066f30 0x7fe8380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:54.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.174+0000 7fe833fff700 1 -- 172.21.15.99:0/204330645 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe8440851d0 con 0x7fe84c15a170 2023-10-08T00:19:54.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.174+0000 7fe833fff700 1 -- 172.21.15.99:0/204330645 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=157}) v3 -- 0x7fe83806ea50 con 0x7fe84c15a170 2023-10-08T00:19:54.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.174+0000 7fe84bfff700 1 --2- 172.21.15.99:0/204330645 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe838066f30 0x7fe8380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:54.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.174+0000 7fe84bfff700 1 --2- 172.21.15.99:0/204330645 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe838066f30 0x7fe8380693e0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe83c0093f0 tx=0x7fe83c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:54.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.178+0000 7fe833fff700 1 -- 172.21.15.99:0/204330645 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe844018060 con 0x7fe84c15a170 2023-10-08T00:19:54.406 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.405+0000 7fe8511b5700 1 -- 172.21.15.99:0/204330645 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe84c0894e0 con 0x7fe84c15a170 2023-10-08T00:19:54.406 INFO:tasks.ceph.mon.a.smithi099.stderr:2023-10-08T00:19:54.406+0000 7f6c60fa2700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T00:19:54.412 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.411+0000 7fe833fff700 1 -- 172.21.15.99:0/204330645 <== mon.0 v2:172.21.15.99:3300/0 7 ==== osd_map(157..157 src has 1..157) v4 ==== 366+0+0 (secure 0 0 0) 0x7fe844056800 con 0x7fe84c15a170 2023-10-08T00:19:54.412 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.411+0000 7fe833fff700 1 -- 172.21.15.99:0/204330645 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=158}) v3 -- 0x7fe83806f3a0 con 0x7fe84c15a170 2023-10-08T00:19:54.414 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.414+0000 7fe833fff700 1 -- 172.21.15.99:0/204330645 <== mon.0 v2:172.21.15.99:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v71) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe844055ed0 con 0x7fe84c15a170 2023-10-08T00:19:54.415 INFO:tasks.ceph.mds.d.smithi191.stderr: -9999> 2023-10-08T00:17:35.121+0000 7f4ff3ad8700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-10-08T00:19:54.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.416+0000 7fe8511b5700 1 -- 172.21.15.99:0/204330645 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe838066f30 msgr2=0x7fe8380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:54.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.416+0000 7fe8511b5700 1 --2- 172.21.15.99:0/204330645 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe838066f30 0x7fe8380693e0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe83c0093f0 tx=0x7fe83c017000 comp rx=0 tx=0).stop 2023-10-08T00:19:54.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.416+0000 7fe8511b5700 1 -- 172.21.15.99:0/204330645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe84c15a170 msgr2=0x7fe84c0c7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:54.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.416+0000 7fe8511b5700 1 --2- 172.21.15.99:0/204330645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe84c15a170 0x7fe84c0c7bd0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fe844009ec0 tx=0x7fe84400b040 comp rx=0 tx=0).stop 2023-10-08T00:19:54.418 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.416+0000 7fe8511b5700 1 -- 172.21.15.99:0/204330645 shutdown_connections 2023-10-08T00:19:54.418 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.416+0000 7fe8511b5700 1 --2- 172.21.15.99:0/204330645 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe838066f30 0x7fe8380693e0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:54.418 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.416+0000 7fe8511b5700 1 --2- 172.21.15.99:0/204330645 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe84c069210 0x7fe84c15af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:54.418 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.416+0000 7fe8511b5700 1 --2- 172.21.15.99:0/204330645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe84c15a170 0x7fe84c0c7bd0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:54.418 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.416+0000 7fe8511b5700 1 --2- 172.21.15.99:0/204330645 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe84c159da0 0x7fe84c0c73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:54.418 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.416+0000 7fe8511b5700 1 -- 172.21.15.99:0/204330645 >> 172.21.15.99:0/204330645 conn(0x7fe84c0d0bc0 msgr2=0x7fe84c0b3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:19:54.418 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.416+0000 7fe8511b5700 1 -- 172.21.15.99:0/204330645 shutdown_connections 2023-10-08T00:19:54.418 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.416+0000 7fe8511b5700 1 -- 172.21.15.99:0/204330645 wait complete. 2023-10-08T00:19:54.419 INFO:tasks.ceph.mds.d.smithi191.stderr: -8826> 2023-10-08T00:17:45.532+0000 7f4fedacc700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T00:19:54.419 INFO:teuthology.orchestra.run.smithi099.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T00:19:54.429 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T00:19:54.447 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2023-10-08T00:19:54.545 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.544+0000 7fca26a30700 1 Processor -- start 2023-10-08T00:19:54.545 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.544+0000 7fca26a30700 1 -- start start 2023-10-08T00:19:54.547 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.545+0000 7fca26a30700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fca2015a290 0x7fca2006a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:54.547 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.545+0000 7fca26a30700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fca2006aae0 0x7fca200cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:54.547 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.545+0000 7fca26a30700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca200b0790 con 0x7fca20159ec0 2023-10-08T00:19:54.548 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.545+0000 7fca26a30700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca2006af20 con 0x7fca2006aae0 2023-10-08T00:19:54.548 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.545+0000 7fca26a30700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca200ce1f0 con 0x7fca2015a290 2023-10-08T00:19:54.548 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.545+0000 7fca24fcd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fca2006aae0 0x7fca200cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:54.548 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.545+0000 7fca1ffff700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fca20159ec0 0x7fca200af940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:54490/0 (socket says 172.21.15.99:54490) 2023-10-08T00:19:54.548 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.545+0000 7fca1ffff700 1 -- 172.21.15.99:0/3581225472 learned_addr learned my addr 172.21.15.99:0/3581225472 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:19:54.548 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.545+0000 7fca1f7fe700 1 --2- 172.21.15.99:0/3581225472 >> v2:172.21.15.191:3300/0 conn(0x7fca2015a290 0x7fca2006a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:54.548 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.546+0000 7fca24fcd700 1 -- 172.21.15.99:0/3581225472 >> v2:172.21.15.191:3300/0 conn(0x7fca2015a290 msgr2=0x7fca2006a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:19:54.548 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.546+0000 7fca24fcd700 1 --2- 172.21.15.99:0/3581225472 >> v2:172.21.15.191:3300/0 conn(0x7fca2015a290 0x7fca2006a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:54.548 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.546+0000 7fca24fcd700 1 -- 172.21.15.99:0/3581225472 >> v1:172.21.15.99:6789/0 conn(0x7fca20159ec0 legacy=0x7fca200af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:54.548 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.546+0000 7fca24fcd700 1 -- 172.21.15.99:0/3581225472 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca200b0b20 con 0x7fca2006aae0 2023-10-08T00:19:54.548 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.546+0000 7fca1effd700 1 -- 172.21.15.99:0/3581225472 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 585062227 0 0) 0x7fca200b0790 con 0x7fca20159ec0 2023-10-08T00:19:54.549 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.546+0000 7fca24fcd700 1 --2- 172.21.15.99:0/3581225472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca2006aae0 0x7fca200cf590 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fca20089910 tx=0x7fca18009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dec1ab90feca3fe5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:54.549 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.546+0000 7fca1effd700 1 -- 172.21.15.99:0/3581225472 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1800e070 con 0x7fca2006aae0 2023-10-08T00:19:54.549 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.546+0000 7fca1effd700 1 -- 172.21.15.99:0/3581225472 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca1800bc30 con 0x7fca2006aae0 2023-10-08T00:19:54.549 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.546+0000 7fca1effd700 1 -- 172.21.15.99:0/3581225472 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca18012430 con 0x7fca2006aae0 2023-10-08T00:19:54.549 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.546+0000 7fca26a30700 1 -- 172.21.15.99:0/3581225472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca2006aae0 msgr2=0x7fca200cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:54.549 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.546+0000 7fca26a30700 1 --2- 172.21.15.99:0/3581225472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca2006aae0 0x7fca200cf590 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fca20089910 tx=0x7fca18009780 comp rx=0 tx=0).stop 2023-10-08T00:19:54.549 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.547+0000 7fca26a30700 1 -- 172.21.15.99:0/3581225472 shutdown_connections 2023-10-08T00:19:54.549 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.547+0000 7fca26a30700 1 --2- 172.21.15.99:0/3581225472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca2006aae0 0x7fca200cf590 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:54.549 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.547+0000 7fca26a30700 1 --2- 172.21.15.99:0/3581225472 >> v2:172.21.15.191:3300/0 conn(0x7fca2015a290 0x7fca2006a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:54.549 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.547+0000 7fca26a30700 1 -- 172.21.15.99:0/3581225472 >> 172.21.15.99:0/3581225472 conn(0x7fca200d1880 msgr2=0x7fca200cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:19:54.549 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.547+0000 7fca26a30700 1 -- 172.21.15.99:0/3581225472 shutdown_connections 2023-10-08T00:19:54.549 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.547+0000 7fca26a30700 1 -- 172.21.15.99:0/3581225472 wait complete. 2023-10-08T00:19:54.552 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.548+0000 7fca26a30700 1 Processor -- start 2023-10-08T00:19:54.553 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.548+0000 7fca26a30700 1 -- start start 2023-10-08T00:19:54.553 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.548+0000 7fca26a30700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca2006aae0 0x7fca200c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:54.553 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.548+0000 7fca26a30700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca20159ec0 0x7fca200b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:54.553 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.549+0000 7fca26a30700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca2015a290 0x7fca200b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:54.553 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.549+0000 7fca26a30700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca200b4b20 con 0x7fca2006aae0 2023-10-08T00:19:54.553 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.549+0000 7fca26a30700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca200c8720 con 0x7fca20159ec0 2023-10-08T00:19:54.553 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.549+0000 7fca26a30700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca200b4090 con 0x7fca2015a290 2023-10-08T00:19:54.553 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.549+0000 7fca1ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca2006aae0 0x7fca200c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:54.553 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.549+0000 7fca1ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca2006aae0 0x7fca200c81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40692/0 (socket says 172.21.15.99:40692) 2023-10-08T00:19:54.553 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.549+0000 7fca1ffff700 1 -- 172.21.15.99:0/2817415661 learned_addr learned my addr 172.21.15.99:0/2817415661 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:19:54.553 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.549+0000 7fca1f7fe700 1 --2- 172.21.15.99:0/2817415661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca20159ec0 0x7fca200b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:54.554 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.549+0000 7fca24fcd700 1 --2- 172.21.15.99:0/2817415661 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca2015a290 0x7fca200b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:54.554 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.549+0000 7fca1ffff700 1 -- 172.21.15.99:0/2817415661 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca2015a290 msgr2=0x7fca200b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:54.554 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.549+0000 7fca1ffff700 1 --2- 172.21.15.99:0/2817415661 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca2015a290 0x7fca200b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:54.554 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.549+0000 7fca1ffff700 1 -- 172.21.15.99:0/2817415661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca20159ec0 msgr2=0x7fca200b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:54.554 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.549+0000 7fca1ffff700 1 --2- 172.21.15.99:0/2817415661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca20159ec0 0x7fca200b3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:54.554 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.549+0000 7fca1ffff700 1 -- 172.21.15.99:0/2817415661 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca18009a20 con 0x7fca2006aae0 2023-10-08T00:19:54.554 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.549+0000 7fca1f7fe700 1 --2- 172.21.15.99:0/2817415661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca20159ec0 0x7fca200b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:19:54.554 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.549+0000 7fca1ffff700 1 --2- 172.21.15.99:0/2817415661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca2006aae0 0x7fca200c81e0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fca1000c7e0 tx=0x7fca10007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:54.554 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.549+0000 7fca07fff700 1 -- 172.21.15.99:0/2817415661 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca10005b30 con 0x7fca2006aae0 2023-10-08T00:19:54.554 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.549+0000 7fca07fff700 1 -- 172.21.15.99:0/2817415661 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca10005c90 con 0x7fca2006aae0 2023-10-08T00:19:54.554 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.549+0000 7fca26a30700 1 -- 172.21.15.99:0/2817415661 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca200b42e0 con 0x7fca2006aae0 2023-10-08T00:19:54.555 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.549+0000 7fca07fff700 1 -- 172.21.15.99:0/2817415661 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca10007300 con 0x7fca2006aae0 2023-10-08T00:19:54.555 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.549+0000 7fca26a30700 1 -- 172.21.15.99:0/2817415661 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca200b5080 con 0x7fca2006aae0 2023-10-08T00:19:54.555 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.550+0000 7fca26a30700 1 -- 172.21.15.99:0/2817415661 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca200b52a0 con 0x7fca2006aae0 2023-10-08T00:19:54.555 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.551+0000 7fca07fff700 1 -- 172.21.15.99:0/2817415661 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fca1001b400 con 0x7fca2006aae0 2023-10-08T00:19:54.555 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.551+0000 7fca07fff700 1 --2- 172.21.15.99:0/2817415661 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fca0c066f30 0x7fca0c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:54.555 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.551+0000 7fca07fff700 1 -- 172.21.15.99:0/2817415661 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5791+0+0 (secure 0 0 0) 0x7fca10084090 con 0x7fca2006aae0 2023-10-08T00:19:54.555 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.551+0000 7fca1f7fe700 1 --2- 172.21.15.99:0/2817415661 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fca0c066f30 0x7fca0c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:54.555 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.551+0000 7fca07fff700 1 -- 172.21.15.99:0/2817415661 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=158}) v3 -- 0x7fca0c06eaf0 con 0x7fca2006aae0 2023-10-08T00:19:54.555 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.552+0000 7fca1f7fe700 1 --2- 172.21.15.99:0/2817415661 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fca0c066f30 0x7fca0c0693e0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fca201595f0 tx=0x7fca14008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:54.556 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.555+0000 7fca07fff700 1 -- 172.21.15.99:0/2817415661 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fca10051330 con 0x7fca2006aae0 2023-10-08T00:19:54.781 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:54.780+0000 7fca26a30700 1 -- 172.21.15.99:0/2817415661 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fca2004c3c0 con 0x7fca2006aae0 2023-10-08T00:19:55.412 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.411+0000 7fca07fff700 1 -- 172.21.15.99:0/2817415661 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v72) v1 ==== 104+0+0 (secure 0 0 0) 0x7fca10054950 con 0x7fca2006aae0 2023-10-08T00:19:55.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.413+0000 7fca26a30700 1 -- 172.21.15.99:0/2817415661 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fca0c066f30 msgr2=0x7fca0c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:55.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.413+0000 7fca26a30700 1 --2- 172.21.15.99:0/2817415661 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fca0c066f30 0x7fca0c0693e0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fca201595f0 tx=0x7fca14008040 comp rx=0 tx=0).stop 2023-10-08T00:19:55.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.413+0000 7fca26a30700 1 -- 172.21.15.99:0/2817415661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca2006aae0 msgr2=0x7fca200c81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:55.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.413+0000 7fca26a30700 1 --2- 172.21.15.99:0/2817415661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca2006aae0 0x7fca200c81e0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fca1000c7e0 tx=0x7fca10007df0 comp rx=0 tx=0).stop 2023-10-08T00:19:55.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.413+0000 7fca26a30700 1 -- 172.21.15.99:0/2817415661 shutdown_connections 2023-10-08T00:19:55.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.413+0000 7fca26a30700 1 --2- 172.21.15.99:0/2817415661 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fca0c066f30 0x7fca0c0693e0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:55.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.413+0000 7fca26a30700 1 --2- 172.21.15.99:0/2817415661 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca2015a290 0x7fca200b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:55.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.413+0000 7fca26a30700 1 --2- 172.21.15.99:0/2817415661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca2006aae0 0x7fca200c81e0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:55.416 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.413+0000 7fca26a30700 1 --2- 172.21.15.99:0/2817415661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca20159ec0 0x7fca200b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:55.416 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.413+0000 7fca26a30700 1 -- 172.21.15.99:0/2817415661 >> 172.21.15.99:0/2817415661 conn(0x7fca200d1880 msgr2=0x7fca200c63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:19:55.416 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.413+0000 7fca26a30700 1 -- 172.21.15.99:0/2817415661 shutdown_connections 2023-10-08T00:19:55.416 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.413+0000 7fca26a30700 1 -- 172.21.15.99:0/2817415661 wait complete. 2023-10-08T00:19:55.428 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T00:19:55.547 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.545+0000 7ff57bf5d700 1 Processor -- start 2023-10-08T00:19:55.547 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.546+0000 7ff57bf5d700 1 -- start start 2023-10-08T00:19:55.550 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.546+0000 7ff57bf5d700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff5740b6a10 0x7ff57415a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:55.550 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.546+0000 7ff57bf5d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff5740b46f0 0x7ff5740b4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:55.550 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.546+0000 7ff57bf5d700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5740b5070 con 0x7ff574159ef0 2023-10-08T00:19:55.550 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.546+0000 7ff57bf5d700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5740b17b0 con 0x7ff5740b6a10 2023-10-08T00:19:55.550 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.546+0000 7ff57bf5d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5740b1a60 con 0x7ff5740b46f0 2023-10-08T00:19:55.550 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.547+0000 7ff5794f8700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff5740b6a10 0x7ff57415a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:55.550 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.547+0000 7ff579cf9700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7ff574159ef0 0x7ff5740b10a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:54504/0 (socket says 172.21.15.99:54504) 2023-10-08T00:19:55.550 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.547+0000 7ff579cf9700 1 -- 172.21.15.99:0/669455014 learned_addr learned my addr 172.21.15.99:0/669455014 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:19:55.550 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.547+0000 7ff57a4fa700 1 --2- 172.21.15.99:0/669455014 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff5740b46f0 0x7ff5740b4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:55.550 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.547+0000 7ff5794f8700 1 -- 172.21.15.99:0/669455014 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff5740b46f0 msgr2=0x7ff5740b4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:19:55.551 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.547+0000 7ff5794f8700 1 --2- 172.21.15.99:0/669455014 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff5740b46f0 0x7ff5740b4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:55.551 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.547+0000 7ff5794f8700 1 -- 172.21.15.99:0/669455014 >> v1:172.21.15.99:6789/0 conn(0x7ff574159ef0 legacy=0x7ff5740b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:55.551 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.547+0000 7ff578cf7700 1 -- 172.21.15.99:0/669455014 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1728872808 0 0) 0x7ff5740b5070 con 0x7ff574159ef0 2023-10-08T00:19:55.551 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.547+0000 7ff5794f8700 1 -- 172.21.15.99:0/669455014 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5740b1d40 con 0x7ff5740b6a10 2023-10-08T00:19:55.551 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.547+0000 7ff57a4fa700 1 --2- 172.21.15.99:0/669455014 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff5740b46f0 0x7ff5740b4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:19:55.551 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.547+0000 7ff5794f8700 1 --2- 172.21.15.99:0/669455014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5740b6a10 0x7ff57415a2c0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7ff5640098e0 tx=0x7ff5640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0b3e1f580a3aa17 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:55.551 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.547+0000 7ff578cf7700 1 -- 172.21.15.99:0/669455014 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff564009e80 con 0x7ff5740b6a10 2023-10-08T00:19:55.552 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.547+0000 7ff578cf7700 1 -- 172.21.15.99:0/669455014 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff56400b860 con 0x7ff5740b6a10 2023-10-08T00:19:55.552 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.547+0000 7ff578cf7700 1 -- 172.21.15.99:0/669455014 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56400bbe0 con 0x7ff5740b6a10 2023-10-08T00:19:55.552 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.548+0000 7ff57bf5d700 1 -- 172.21.15.99:0/669455014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5740b6a10 msgr2=0x7ff57415a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:55.552 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.548+0000 7ff57bf5d700 1 --2- 172.21.15.99:0/669455014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5740b6a10 0x7ff57415a2c0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7ff5640098e0 tx=0x7ff5640091a0 comp rx=0 tx=0).stop 2023-10-08T00:19:55.552 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.548+0000 7ff57bf5d700 1 -- 172.21.15.99:0/669455014 shutdown_connections 2023-10-08T00:19:55.552 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.548+0000 7ff57bf5d700 1 --2- 172.21.15.99:0/669455014 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff5740b46f0 0x7ff5740b4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:55.552 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.548+0000 7ff57bf5d700 1 --2- 172.21.15.99:0/669455014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5740b6a10 0x7ff57415a2c0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:55.552 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.548+0000 7ff57bf5d700 1 -- 172.21.15.99:0/669455014 >> 172.21.15.99:0/669455014 conn(0x7ff57409a0a0 msgr2=0x7ff574067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:19:55.552 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.548+0000 7ff57bf5d700 1 -- 172.21.15.99:0/669455014 shutdown_connections 2023-10-08T00:19:55.553 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.548+0000 7ff57bf5d700 1 -- 172.21.15.99:0/669455014 wait complete. 2023-10-08T00:19:55.556 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.549+0000 7ff57bf5d700 1 Processor -- start 2023-10-08T00:19:55.556 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.549+0000 7ff57bf5d700 1 -- start start 2023-10-08T00:19:55.556 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.550+0000 7ff57bf5d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff5740b46f0 0x7ff574169910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:55.556 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.550+0000 7ff57bf5d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff5740b6a10 0x7ff574159340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:55.556 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.550+0000 7ff57bf5d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff574159ef0 0x7ff57409df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:55.556 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.550+0000 7ff57bf5d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff57416a960 con 0x7ff574159ef0 2023-10-08T00:19:55.556 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.550+0000 7ff57bf5d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5740ae860 con 0x7ff5740b46f0 2023-10-08T00:19:55.556 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.550+0000 7ff57bf5d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5740afff0 con 0x7ff5740b6a10 2023-10-08T00:19:55.556 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.550+0000 7ff57a4fa700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff574159ef0 0x7ff57409df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:55.557 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.550+0000 7ff57a4fa700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff574159ef0 0x7ff57409df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40718/0 (socket says 172.21.15.99:40718) 2023-10-08T00:19:55.557 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.550+0000 7ff57a4fa700 1 -- 172.21.15.99:0/605020262 learned_addr learned my addr 172.21.15.99:0/605020262 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:19:55.557 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.550+0000 7ff579cf9700 1 --2- 172.21.15.99:0/605020262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff5740b46f0 0x7ff574169910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:55.557 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.550+0000 7ff5794f8700 1 --2- 172.21.15.99:0/605020262 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff5740b6a10 0x7ff574159340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:55.557 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.550+0000 7ff57a4fa700 1 -- 172.21.15.99:0/605020262 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff5740b6a10 msgr2=0x7ff574159340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:55.557 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.550+0000 7ff57a4fa700 1 --2- 172.21.15.99:0/605020262 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff5740b6a10 0x7ff574159340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:55.557 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.550+0000 7ff57a4fa700 1 -- 172.21.15.99:0/605020262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff5740b46f0 msgr2=0x7ff574169910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:55.557 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.550+0000 7ff57a4fa700 1 --2- 172.21.15.99:0/605020262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff5740b46f0 0x7ff574169910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:55.557 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.550+0000 7ff57a4fa700 1 -- 172.21.15.99:0/605020262 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff564009420 con 0x7ff574159ef0 2023-10-08T00:19:55.557 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.550+0000 7ff57a4fa700 1 --2- 172.21.15.99:0/605020262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff574159ef0 0x7ff57409df70 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7ff56c00c850 tx=0x7ff56c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:55.557 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.550+0000 7ff569ffb700 1 -- 172.21.15.99:0/605020262 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56c011600 con 0x7ff574159ef0 2023-10-08T00:19:55.558 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.550+0000 7ff569ffb700 1 -- 172.21.15.99:0/605020262 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff56c011760 con 0x7ff574159ef0 2023-10-08T00:19:55.558 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.550+0000 7ff57bf5d700 1 -- 172.21.15.99:0/605020262 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5740ac090 con 0x7ff574159ef0 2023-10-08T00:19:55.558 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.550+0000 7ff569ffb700 1 -- 172.21.15.99:0/605020262 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56c011b20 con 0x7ff574159ef0 2023-10-08T00:19:55.558 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.551+0000 7ff57bf5d700 1 -- 172.21.15.99:0/605020262 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff57416a140 con 0x7ff574159ef0 2023-10-08T00:19:55.558 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.551+0000 7ff57bf5d700 1 -- 172.21.15.99:0/605020262 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5740ac220 con 0x7ff574159ef0 2023-10-08T00:19:55.558 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.552+0000 7ff569ffb700 1 -- 172.21.15.99:0/605020262 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff56c018350 con 0x7ff574159ef0 2023-10-08T00:19:55.558 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.552+0000 7ff569ffb700 1 --2- 172.21.15.99:0/605020262 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff558066f30 0x7ff5580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:55.558 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.552+0000 7ff569ffb700 1 -- 172.21.15.99:0/605020262 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5791+0+0 (secure 0 0 0) 0x7ff56c0845c0 con 0x7ff574159ef0 2023-10-08T00:19:55.558 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.552+0000 7ff569ffb700 1 -- 172.21.15.99:0/605020262 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=158}) v3 -- 0x7ff55806eaf0 con 0x7ff574159ef0 2023-10-08T00:19:55.558 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.552+0000 7ff579cf9700 1 --2- 172.21.15.99:0/605020262 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff558066f30 0x7ff5580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:55.559 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.553+0000 7ff579cf9700 1 --2- 172.21.15.99:0/605020262 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff558066f30 0x7ff5580693e0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ff570007fd0 tx=0x7ff570009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:55.559 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.556+0000 7ff569ffb700 1 -- 172.21.15.99:0/605020262 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff56c055270 con 0x7ff574159ef0 2023-10-08T00:19:55.740 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:55.739+0000 7ff57bf5d700 1 -- 172.21.15.99:0/605020262 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff574159880 con 0x7ff574159ef0 2023-10-08T00:19:56.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.153+0000 7ff569ffb700 1 -- 172.21.15.99:0/605020262 <== mon.0 v2:172.21.15.99:3300/0 7 ==== osd_map(158..158 src has 1..158) v4 ==== 679+0+0 (secure 0 0 0) 0x7ff56c055ba0 con 0x7ff574159ef0 2023-10-08T00:19:56.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.153+0000 7ff569ffb700 1 -- 172.21.15.99:0/605020262 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=159}) v3 -- 0x7ff55806cf20 con 0x7ff574159ef0 2023-10-08T00:19:56.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.154+0000 7ff569ffb700 1 -- 172.21.15.99:0/605020262 <== mon.0 v2:172.21.15.99:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v158) v1 ==== 181+0+0 (secure 0 0 0) 0x7ff56c01e0d0 con 0x7ff574159ef0 2023-10-08T00:19:56.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.157+0000 7ff57bf5d700 1 -- 172.21.15.99:0/605020262 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff558066f30 msgr2=0x7ff5580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:56.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.157+0000 7ff57bf5d700 1 --2- 172.21.15.99:0/605020262 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff558066f30 0x7ff5580693e0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ff570007fd0 tx=0x7ff570009040 comp rx=0 tx=0).stop 2023-10-08T00:19:56.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.157+0000 7ff57bf5d700 1 -- 172.21.15.99:0/605020262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff574159ef0 msgr2=0x7ff57409df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:56.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.157+0000 7ff57bf5d700 1 --2- 172.21.15.99:0/605020262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff574159ef0 0x7ff57409df70 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7ff56c00c850 tx=0x7ff56c008040 comp rx=0 tx=0).stop 2023-10-08T00:19:56.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.157+0000 7ff57bf5d700 1 -- 172.21.15.99:0/605020262 shutdown_connections 2023-10-08T00:19:56.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.157+0000 7ff57bf5d700 1 --2- 172.21.15.99:0/605020262 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff558066f30 0x7ff5580693e0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:56.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.157+0000 7ff57bf5d700 1 --2- 172.21.15.99:0/605020262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff5740b46f0 0x7ff574169910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:56.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.157+0000 7ff57bf5d700 1 --2- 172.21.15.99:0/605020262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff574159ef0 0x7ff57409df70 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:56.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.157+0000 7ff57bf5d700 1 --2- 172.21.15.99:0/605020262 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff5740b6a10 0x7ff574159340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:56.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.157+0000 7ff57bf5d700 1 -- 172.21.15.99:0/605020262 >> 172.21.15.99:0/605020262 conn(0x7ff57409a0a0 msgr2=0x7ff57415b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:19:56.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.157+0000 7ff57bf5d700 1 -- 172.21.15.99:0/605020262 shutdown_connections 2023-10-08T00:19:56.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.157+0000 7ff57bf5d700 1 -- 172.21.15.99:0/605020262 wait complete. 2023-10-08T00:19:56.160 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' removed 2023-10-08T00:19:56.172 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T00:19:56.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.291+0000 7ff12c9f6700 1 Processor -- start 2023-10-08T00:19:56.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.291+0000 7ff12c9f6700 1 -- start start 2023-10-08T00:19:56.295 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.292+0000 7ff12c9f6700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff128070e10 0x7ff12815d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:56.295 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.292+0000 7ff12c9f6700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff128067d50 con 0x7ff128159b50 2023-10-08T00:19:56.295 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.292+0000 7ff12c9f6700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff12806f560 con 0x7ff128159f20 2023-10-08T00:19:56.295 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.292+0000 7ff12c9f6700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff12806f840 con 0x7ff128070e10 2023-10-08T00:19:56.295 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.292+0000 7ff12759e700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff128070e10 0x7ff12815d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:56.295 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.292+0000 7ff126d9d700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7ff128159b50 0x7ff128068980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:54512/0 (socket says 172.21.15.99:54512) 2023-10-08T00:19:56.295 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.292+0000 7ff126d9d700 1 -- 172.21.15.99:0/834366906 learned_addr learned my addr 172.21.15.99:0/834366906 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:19:56.295 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.292+0000 7ff12759e700 1 -- 172.21.15.99:0/834366906 >> v1:172.21.15.191:6789/0 conn(0x7ff128159f20 legacy=0x7ff128070700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:19:56.295 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.292+0000 7ff12759e700 1 -- 172.21.15.99:0/834366906 >> v1:172.21.15.99:6789/0 conn(0x7ff128159b50 legacy=0x7ff128068980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:56.295 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.292+0000 7ff125d9b700 1 -- 172.21.15.99:0/834366906 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3861920283 0 0) 0x7ff128067d50 con 0x7ff128159b50 2023-10-08T00:19:56.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.292+0000 7ff12759e700 1 -- 172.21.15.99:0/834366906 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff128067700 con 0x7ff128070e10 2023-10-08T00:19:56.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.292+0000 7ff12759e700 1 --2- 172.21.15.99:0/834366906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff128070e10 0x7ff12815d920 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7ff1180098e0 tx=0x7ff1180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82a9cabfa7d6eb6d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:56.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.292+0000 7ff125d9b700 1 -- 172.21.15.99:0/834366906 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff118009e80 con 0x7ff128070e10 2023-10-08T00:19:56.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.292+0000 7ff125d9b700 1 -- 172.21.15.99:0/834366906 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff11800b820 con 0x7ff128070e10 2023-10-08T00:19:56.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.292+0000 7ff125d9b700 1 -- 172.21.15.99:0/834366906 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11800bba0 con 0x7ff128070e10 2023-10-08T00:19:56.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.293+0000 7ff12c9f6700 1 -- 172.21.15.99:0/834366906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff128070e10 msgr2=0x7ff12815d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:56.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.293+0000 7ff12c9f6700 1 --2- 172.21.15.99:0/834366906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff128070e10 0x7ff12815d920 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7ff1180098e0 tx=0x7ff1180091a0 comp rx=0 tx=0).stop 2023-10-08T00:19:56.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.293+0000 7ff12c9f6700 1 -- 172.21.15.99:0/834366906 shutdown_connections 2023-10-08T00:19:56.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.293+0000 7ff12c9f6700 1 --2- 172.21.15.99:0/834366906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff128070e10 0x7ff12815d920 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:56.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.293+0000 7ff12c9f6700 1 -- 172.21.15.99:0/834366906 >> 172.21.15.99:0/834366906 conn(0x7ff1280cd900 msgr2=0x7ff1280cdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:19:56.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.293+0000 7ff12c9f6700 1 -- 172.21.15.99:0/834366906 shutdown_connections 2023-10-08T00:19:56.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.293+0000 7ff12c9f6700 1 -- 172.21.15.99:0/834366906 wait complete. 2023-10-08T00:19:56.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.294+0000 7ff12c9f6700 1 Processor -- start 2023-10-08T00:19:56.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.294+0000 7ff12c9f6700 1 -- start start 2023-10-08T00:19:56.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.295+0000 7ff12c9f6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff128070e10 0x7ff128145410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:56.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.295+0000 7ff12c9f6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff128159b50 0x7ff1280c5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:56.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.295+0000 7ff12c9f6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff128159f20 0x7ff128158110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:56.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.295+0000 7ff12c9f6700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff128145950 con 0x7ff128159f20 2023-10-08T00:19:56.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.295+0000 7ff12c9f6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1280c6ce0 con 0x7ff128159b50 2023-10-08T00:19:56.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.295+0000 7ff12c9f6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1280c8680 con 0x7ff128070e10 2023-10-08T00:19:56.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.295+0000 7ff12759e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff128159f20 0x7ff128158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:56.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.295+0000 7ff12759e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff128159f20 0x7ff128158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40750/0 (socket says 172.21.15.99:40750) 2023-10-08T00:19:56.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.295+0000 7ff12759e700 1 -- 172.21.15.99:0/4291386190 learned_addr learned my addr 172.21.15.99:0/4291386190 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:19:56.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.295+0000 7ff12659c700 1 --2- 172.21.15.99:0/4291386190 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff128159b50 0x7ff1280c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:56.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.295+0000 7ff126d9d700 1 --2- 172.21.15.99:0/4291386190 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff128070e10 0x7ff128145410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:56.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.295+0000 7ff12759e700 1 -- 172.21.15.99:0/4291386190 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff128070e10 msgr2=0x7ff128145410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:56.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.295+0000 7ff12759e700 1 --2- 172.21.15.99:0/4291386190 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff128070e10 0x7ff128145410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:56.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.295+0000 7ff12759e700 1 -- 172.21.15.99:0/4291386190 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff128159b50 msgr2=0x7ff1280c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:56.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.295+0000 7ff12759e700 1 --2- 172.21.15.99:0/4291386190 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff128159b50 0x7ff1280c5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:56.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.295+0000 7ff12759e700 1 -- 172.21.15.99:0/4291386190 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff118009420 con 0x7ff128159f20 2023-10-08T00:19:56.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.295+0000 7ff12759e700 1 --2- 172.21.15.99:0/4291386190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff128159f20 0x7ff128158110 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7ff118002000 tx=0x7ff118016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:56.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.295+0000 7ff10effd700 1 -- 172.21.15.99:0/4291386190 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11800bb60 con 0x7ff128159f20 2023-10-08T00:19:56.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.296+0000 7ff10effd700 1 -- 172.21.15.99:0/4291386190 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff11801c660 con 0x7ff128159f20 2023-10-08T00:19:56.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.296+0000 7ff12c9f6700 1 -- 172.21.15.99:0/4291386190 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1280c6a00 con 0x7ff128159f20 2023-10-08T00:19:56.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.296+0000 7ff10effd700 1 -- 172.21.15.99:0/4291386190 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11801ca00 con 0x7ff128159f20 2023-10-08T00:19:56.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.296+0000 7ff12c9f6700 1 -- 172.21.15.99:0/4291386190 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1280c8310 con 0x7ff128159f20 2023-10-08T00:19:56.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.296+0000 7ff12c9f6700 1 -- 172.21.15.99:0/4291386190 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff12806d660 con 0x7ff128159f20 2023-10-08T00:19:56.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.297+0000 7ff10effd700 1 -- 172.21.15.99:0/4291386190 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff11801b470 con 0x7ff128159f20 2023-10-08T00:19:56.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.297+0000 7ff10effd700 1 --2- 172.21.15.99:0/4291386190 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff114066f30 0x7ff1140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:56.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.297+0000 7ff10effd700 1 -- 172.21.15.99:0/4291386190 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5353+0+0 (secure 0 0 0) 0x7ff1180861f0 con 0x7ff128159f20 2023-10-08T00:19:56.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.297+0000 7ff126d9d700 1 --2- 172.21.15.99:0/4291386190 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff114066f30 0x7ff1140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:56.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.298+0000 7ff126d9d700 1 --2- 172.21.15.99:0/4291386190 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff114066f30 0x7ff1140693e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff120002250 tx=0x7ff120001a20 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:56.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.302+0000 7ff10effd700 1 -- 172.21.15.99:0/4291386190 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff1180535c0 con 0x7ff128159f20 2023-10-08T00:19:56.487 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:56.485+0000 7ff12c9f6700 1 -- 172.21.15.99:0/4291386190 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff1280c7cb0 con 0x7ff128159f20 2023-10-08T00:19:57.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.191+0000 7ff10effd700 1 -- 172.21.15.99:0/4291386190 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v159) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff118056be0 con 0x7ff128159f20 2023-10-08T00:19:57.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.196+0000 7ff12c9f6700 1 -- 172.21.15.99:0/4291386190 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff114066f30 msgr2=0x7ff1140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:57.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.196+0000 7ff12c9f6700 1 --2- 172.21.15.99:0/4291386190 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff114066f30 0x7ff1140693e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff120002250 tx=0x7ff120001a20 comp rx=0 tx=0).stop 2023-10-08T00:19:57.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.196+0000 7ff12c9f6700 1 -- 172.21.15.99:0/4291386190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff128159f20 msgr2=0x7ff128158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:57.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.196+0000 7ff12c9f6700 1 --2- 172.21.15.99:0/4291386190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff128159f20 0x7ff128158110 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7ff118002000 tx=0x7ff118016000 comp rx=0 tx=0).stop 2023-10-08T00:19:57.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.198+0000 7ff12c9f6700 1 -- 172.21.15.99:0/4291386190 shutdown_connections 2023-10-08T00:19:57.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.198+0000 7ff12c9f6700 1 --2- 172.21.15.99:0/4291386190 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff114066f30 0x7ff1140693e0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:57.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.198+0000 7ff12c9f6700 1 --2- 172.21.15.99:0/4291386190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff128159f20 0x7ff128158110 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:57.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.198+0000 7ff12c9f6700 1 --2- 172.21.15.99:0/4291386190 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff128070e10 0x7ff128145410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:57.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.198+0000 7ff12c9f6700 1 --2- 172.21.15.99:0/4291386190 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff128159b50 0x7ff1280c5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:57.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.198+0000 7ff12c9f6700 1 -- 172.21.15.99:0/4291386190 >> 172.21.15.99:0/4291386190 conn(0x7ff1280cd900 msgr2=0x7ff1280afad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:19:57.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.199+0000 7ff12c9f6700 1 -- 172.21.15.99:0/4291386190 shutdown_connections 2023-10-08T00:19:57.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.199+0000 7ff12c9f6700 1 -- 172.21.15.99:0/4291386190 wait complete. 2023-10-08T00:19:57.200 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' removed 2023-10-08T00:19:57.212 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-08T00:19:57.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.329+0000 7fb7ce855700 1 Processor -- start 2023-10-08T00:19:57.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.329+0000 7fb7ce855700 1 -- start start 2023-10-08T00:19:57.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.330+0000 7fb7ce855700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb7c815b9a0 0x7fb7c80caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:57.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.330+0000 7fb7ce855700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7c815c8d0 con 0x7fb7c8152300 2023-10-08T00:19:57.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.330+0000 7fb7ce855700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7c80c9370 con 0x7fb7c806d370 2023-10-08T00:19:57.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.330+0000 7fb7ce855700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7c80b1400 con 0x7fb7c815b9a0 2023-10-08T00:19:57.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.330+0000 7fb7c77fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb7c815b9a0 0x7fb7c80caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:57.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.330+0000 7fb7c7fff700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fb7c8152300 0x7fb7c8158df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:54516/0 (socket says 172.21.15.99:54516) 2023-10-08T00:19:57.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.330+0000 7fb7c7fff700 1 -- 172.21.15.99:0/2209162347 learned_addr learned my addr 172.21.15.99:0/2209162347 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:19:57.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.330+0000 7fb7c77fe700 1 -- 172.21.15.99:0/2209162347 >> v1:172.21.15.191:6789/0 conn(0x7fb7c806d370 legacy=0x7fb7c806d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:19:57.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.330+0000 7fb7c77fe700 1 -- 172.21.15.99:0/2209162347 >> v1:172.21.15.99:6789/0 conn(0x7fb7c8152300 legacy=0x7fb7c8158df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:57.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.330+0000 7fb7c77fe700 1 -- 172.21.15.99:0/2209162347 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c815c030 con 0x7fb7c815b9a0 2023-10-08T00:19:57.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.330+0000 7fb7c6ffd700 1 -- 172.21.15.99:0/2209162347 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2839906993 0 0) 0x7fb7c815c8d0 con 0x7fb7c8152300 2023-10-08T00:19:57.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.331+0000 7fb7c77fe700 1 --2- 172.21.15.99:0/2209162347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7c815b9a0 0x7fb7c80caaf0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc0098e0 tx=0x7fb7bc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17c064b5512656fb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:57.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.331+0000 7fb7c6ffd700 1 -- 172.21.15.99:0/2209162347 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc009e80 con 0x7fb7c815b9a0 2023-10-08T00:19:57.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.331+0000 7fb7c6ffd700 1 -- 172.21.15.99:0/2209162347 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7bc00b820 con 0x7fb7c815b9a0 2023-10-08T00:19:57.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.331+0000 7fb7c6ffd700 1 -- 172.21.15.99:0/2209162347 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc00bba0 con 0x7fb7c815b9a0 2023-10-08T00:19:57.335 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.331+0000 7fb7ce855700 1 -- 172.21.15.99:0/2209162347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7c815b9a0 msgr2=0x7fb7c80caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:57.335 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.331+0000 7fb7ce855700 1 --2- 172.21.15.99:0/2209162347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7c815b9a0 0x7fb7c80caaf0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc0098e0 tx=0x7fb7bc0091a0 comp rx=0 tx=0).stop 2023-10-08T00:19:57.335 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.331+0000 7fb7ce855700 1 -- 172.21.15.99:0/2209162347 shutdown_connections 2023-10-08T00:19:57.335 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.331+0000 7fb7ce855700 1 --2- 172.21.15.99:0/2209162347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7c815b9a0 0x7fb7c80caaf0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:57.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.331+0000 7fb7ce855700 1 -- 172.21.15.99:0/2209162347 >> 172.21.15.99:0/2209162347 conn(0x7fb7c8069870 msgr2=0x7fb7c8069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:19:57.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.331+0000 7fb7ce855700 1 -- 172.21.15.99:0/2209162347 shutdown_connections 2023-10-08T00:19:57.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.331+0000 7fb7ce855700 1 -- 172.21.15.99:0/2209162347 wait complete. 2023-10-08T00:19:57.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.332+0000 7fb7ce855700 1 Processor -- start 2023-10-08T00:19:57.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.333+0000 7fb7ce855700 1 -- start start 2023-10-08T00:19:57.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.333+0000 7fb7ce855700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7c806d370 0x7fb7c8154f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:57.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.333+0000 7fb7ce855700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb7c8152300 0x7fb7c8150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:57.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.333+0000 7fb7ce855700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7c815b9a0 0x7fb7c80c7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:57.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.333+0000 7fb7ce855700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c8168070 con 0x7fb7c815b9a0 2023-10-08T00:19:57.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.333+0000 7fb7ce855700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c8150ea0 con 0x7fb7c806d370 2023-10-08T00:19:57.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.333+0000 7fb7ce855700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c80c8340 con 0x7fb7c8152300 2023-10-08T00:19:57.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.333+0000 7fb7ccdf2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7c815b9a0 0x7fb7c80c7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:57.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.333+0000 7fb7ccdf2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7c815b9a0 0x7fb7c80c7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40772/0 (socket says 172.21.15.99:40772) 2023-10-08T00:19:57.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.333+0000 7fb7ccdf2700 1 -- 172.21.15.99:0/79947737 learned_addr learned my addr 172.21.15.99:0/79947737 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:19:57.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.333+0000 7fb7c7fff700 1 --2- 172.21.15.99:0/79947737 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7c806d370 0x7fb7c8154f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:57.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.333+0000 7fb7c77fe700 1 --2- 172.21.15.99:0/79947737 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb7c8152300 0x7fb7c8150960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:57.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.334+0000 7fb7ccdf2700 1 -- 172.21.15.99:0/79947737 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb7c8152300 msgr2=0x7fb7c8150960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:57.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.334+0000 7fb7ccdf2700 1 --2- 172.21.15.99:0/79947737 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb7c8152300 0x7fb7c8150960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:57.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.334+0000 7fb7ccdf2700 1 -- 172.21.15.99:0/79947737 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7c806d370 msgr2=0x7fb7c8154f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:57.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.334+0000 7fb7ccdf2700 1 --2- 172.21.15.99:0/79947737 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7c806d370 0x7fb7c8154f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:57.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.334+0000 7fb7ccdf2700 1 -- 172.21.15.99:0/79947737 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7bc009420 con 0x7fb7c815b9a0 2023-10-08T00:19:57.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.334+0000 7fb7ccdf2700 1 --2- 172.21.15.99:0/79947737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7c815b9a0 0x7fb7c80c7e00 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fb7c000c930 tx=0x7fb7c00026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:57.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.334+0000 7fb7affff700 1 -- 172.21.15.99:0/79947737 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c000ce70 con 0x7fb7c815b9a0 2023-10-08T00:19:57.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.334+0000 7fb7affff700 1 -- 172.21.15.99:0/79947737 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7c000a7e0 con 0x7fb7c815b9a0 2023-10-08T00:19:57.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.334+0000 7fb7ce855700 1 -- 172.21.15.99:0/79947737 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7c8167d20 con 0x7fb7c815b9a0 2023-10-08T00:19:57.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.334+0000 7fb7affff700 1 -- 172.21.15.99:0/79947737 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c000abc0 con 0x7fb7c815b9a0 2023-10-08T00:19:57.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.334+0000 7fb7ce855700 1 -- 172.21.15.99:0/79947737 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7c80c88c0 con 0x7fb7c815b9a0 2023-10-08T00:19:57.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.335+0000 7fb7ce855700 1 -- 172.21.15.99:0/79947737 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7c804c3c0 con 0x7fb7c815b9a0 2023-10-08T00:19:57.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.335+0000 7fb7affff700 1 -- 172.21.15.99:0/79947737 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb7c0012420 con 0x7fb7c815b9a0 2023-10-08T00:19:57.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.336+0000 7fb7affff700 1 --2- 172.21.15.99:0/79947737 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb7b4066fb0 0x7fb7b4069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:57.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.336+0000 7fb7affff700 1 -- 172.21.15.99:0/79947737 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fb7c0084050 con 0x7fb7c815b9a0 2023-10-08T00:19:57.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.336+0000 7fb7c7fff700 1 --2- 172.21.15.99:0/79947737 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb7b4066fb0 0x7fb7b4069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:57.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.336+0000 7fb7c7fff700 1 --2- 172.21.15.99:0/79947737 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb7b4066fb0 0x7fb7b4069460 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb7b80011e0 tx=0x7fb7b8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:57.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.340+0000 7fb7affff700 1 -- 172.21.15.99:0/79947737 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb7c0051520 con 0x7fb7c815b9a0 2023-10-08T00:19:57.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.520+0000 7fb7ce855700 1 -- 172.21.15.99:0/79947737 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb7c80cdf10 con 0x7fb7c815b9a0 2023-10-08T00:19:57.524 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.523+0000 7fb7affff700 1 -- 172.21.15.99:0/79947737 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb7c00160a0 con 0x7fb7c815b9a0 2023-10-08T00:19:57.527 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.526+0000 7fb7ce855700 1 -- 172.21.15.99:0/79947737 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb7b4066fb0 msgr2=0x7fb7b4069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:57.527 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.526+0000 7fb7ce855700 1 --2- 172.21.15.99:0/79947737 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb7b4066fb0 0x7fb7b4069460 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb7b80011e0 tx=0x7fb7b8009040 comp rx=0 tx=0).stop 2023-10-08T00:19:57.527 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.526+0000 7fb7ce855700 1 -- 172.21.15.99:0/79947737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7c815b9a0 msgr2=0x7fb7c80c7e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:57.527 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.526+0000 7fb7ce855700 1 --2- 172.21.15.99:0/79947737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7c815b9a0 0x7fb7c80c7e00 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fb7c000c930 tx=0x7fb7c00026e0 comp rx=0 tx=0).stop 2023-10-08T00:19:57.527 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.526+0000 7fb7ce855700 1 -- 172.21.15.99:0/79947737 shutdown_connections 2023-10-08T00:19:57.527 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.526+0000 7fb7ce855700 1 --2- 172.21.15.99:0/79947737 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb7b4066fb0 0x7fb7b4069460 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:57.527 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.526+0000 7fb7ce855700 1 --2- 172.21.15.99:0/79947737 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7c806d370 0x7fb7c8154f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:57.527 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.526+0000 7fb7ce855700 1 --2- 172.21.15.99:0/79947737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7c815b9a0 0x7fb7c80c7e00 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:57.527 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.526+0000 7fb7ce855700 1 --2- 172.21.15.99:0/79947737 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb7c8152300 0x7fb7c8150960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:57.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.526+0000 7fb7ce855700 1 -- 172.21.15.99:0/79947737 >> 172.21.15.99:0/79947737 conn(0x7fb7c8069870 msgr2=0x7fb7c80cd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:19:57.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.526+0000 7fb7ce855700 1 -- 172.21.15.99:0/79947737 shutdown_connections 2023-10-08T00:19:57.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.526+0000 7fb7ce855700 1 -- 172.21.15.99:0/79947737 wait complete. 2023-10-08T00:19:57.538 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose' 2023-10-08T00:19:57.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.652+0000 7f447a578700 1 Processor -- start 2023-10-08T00:19:57.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.652+0000 7f447a578700 1 -- start start 2023-10-08T00:19:57.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.653+0000 7f447a578700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4474159be0 0x7f4474068db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:57.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.653+0000 7f447a578700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4474159fb0 0x7f44740ce950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:57.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.653+0000 7f447a578700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4474067800 con 0x7f44740692f0 2023-10-08T00:19:57.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.653+0000 7f447a578700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44740701f0 con 0x7f4474159be0 2023-10-08T00:19:57.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.653+0000 7f447a578700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4474067ae0 con 0x7f4474159fb0 2023-10-08T00:19:57.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.653+0000 7f4479d77700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f44740692f0 0x7f44740b3c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:54520/0 (socket says 172.21.15.99:54520) 2023-10-08T00:19:57.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.653+0000 7f4479d77700 1 -- 172.21.15.99:0/2708133365 learned_addr learned my addr 172.21.15.99:0/2708133365 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:19:57.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.654+0000 7f446bfff700 1 -- 172.21.15.99:0/2708133365 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1817714311 0 0) 0x7f4474067800 con 0x7f44740692f0 2023-10-08T00:19:57.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.654+0000 7f4478d75700 1 --2- 172.21.15.99:0/2708133365 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4474159fb0 0x7f44740ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:57.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.654+0000 7f4479576700 1 --2- 172.21.15.99:0/2708133365 >> v2:172.21.15.191:3300/0 conn(0x7f4474159be0 0x7f4474068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:57.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.654+0000 7f446bfff700 1 -- 172.21.15.99:0/2708133365 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4460003570 con 0x7f44740692f0 2023-10-08T00:19:57.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.654+0000 7f446bfff700 1 -- 172.21.15.99:0/2708133365 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3702988723 0 0) 0x7f4460003570 con 0x7f44740692f0 2023-10-08T00:19:57.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.654+0000 7f446bfff700 1 -- 172.21.15.99:0/2708133365 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4474159fb0 msgr2=0x7f44740ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:19:57.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.654+0000 7f446bfff700 1 --2- 172.21.15.99:0/2708133365 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4474159fb0 0x7f44740ce950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:57.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.654+0000 7f446bfff700 1 -- 172.21.15.99:0/2708133365 >> v2:172.21.15.191:3300/0 conn(0x7f4474159be0 msgr2=0x7f4474068db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:19:57.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.654+0000 7f446bfff700 1 --2- 172.21.15.99:0/2708133365 >> v2:172.21.15.191:3300/0 conn(0x7f4474159be0 0x7f4474068db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:57.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.654+0000 7f446bfff700 1 -- 172.21.15.99:0/2708133365 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4474067dc0 con 0x7f44740692f0 2023-10-08T00:19:57.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.654+0000 7f446bfff700 1 -- 172.21.15.99:0/2708133365 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f446c001470 con 0x7f44740692f0 2023-10-08T00:19:57.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.654+0000 7f446bfff700 1 -- 172.21.15.99:0/2708133365 >> v1:172.21.15.99:6789/0 conn(0x7f44740692f0 legacy=0x7f44740b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:57.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.654+0000 7f446bfff700 1 --2- 172.21.15.99:0/2708133365 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4460003a60 0x7f4460003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:57.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.654+0000 7f446bfff700 1 --2- 172.21.15.99:0/2708133365 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4474159fb0 0x7f4460004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:57.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.654+0000 7f446bfff700 1 --2- 172.21.15.99:0/2708133365 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4460004d50 0x7f44600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:57.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.654+0000 7f446bfff700 1 -- 172.21.15.99:0/2708133365 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44740701f0 con 0x7f4460004d50 2023-10-08T00:19:57.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.654+0000 7f446bfff700 1 -- 172.21.15.99:0/2708133365 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4474067ae0 con 0x7f4460003a60 2023-10-08T00:19:57.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.654+0000 7f446bfff700 1 -- 172.21.15.99:0/2708133365 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f446000ba60 con 0x7f4474159fb0 2023-10-08T00:19:57.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.654+0000 7f4479d77700 1 --2- 172.21.15.99:0/2708133365 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4460004d50 0x7f44600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:57.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.654+0000 7f4479576700 1 --2- 172.21.15.99:0/2708133365 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4460003a60 0x7f4460003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:57.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.654+0000 7f4478d75700 1 --2- 172.21.15.99:0/2708133365 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4474159fb0 0x7f4460004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:57.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.654+0000 7f4479d77700 1 -- 172.21.15.99:0/2708133365 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4474159fb0 msgr2=0x7f4460004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:19:57.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.654+0000 7f4479d77700 1 --2- 172.21.15.99:0/2708133365 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4474159fb0 0x7f4460004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:57.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.654+0000 7f4479d77700 1 -- 172.21.15.99:0/2708133365 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4460003a60 msgr2=0x7f4460003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:19:57.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.654+0000 7f4479d77700 1 --2- 172.21.15.99:0/2708133365 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4460003a60 0x7f4460003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:57.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.654+0000 7f4479d77700 1 -- 172.21.15.99:0/2708133365 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4460003570 con 0x7f4460004d50 2023-10-08T00:19:57.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.655+0000 7f4479d77700 1 --2- 172.21.15.99:0/2708133365 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4460004d50 0x7f44600051c0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f446c00c3c0 tx=0x7f446c001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=785d09a7e6d78645 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:57.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.655+0000 7f446bfff700 1 -- 172.21.15.99:0/2708133365 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c00cc30 con 0x7f4460004d50 2023-10-08T00:19:57.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.655+0000 7f446bfff700 1 -- 172.21.15.99:0/2708133365 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f446c00cdb0 con 0x7f4460004d50 2023-10-08T00:19:57.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.655+0000 7f446bfff700 1 -- 172.21.15.99:0/2708133365 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c005b70 con 0x7f4460004d50 2023-10-08T00:19:57.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.655+0000 7f447a578700 1 -- 172.21.15.99:0/2708133365 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4460004d50 msgr2=0x7f44600051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:57.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.655+0000 7f447a578700 1 --2- 172.21.15.99:0/2708133365 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4460004d50 0x7f44600051c0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f446c00c3c0 tx=0x7f446c001e90 comp rx=0 tx=0).stop 2023-10-08T00:19:57.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.655+0000 7f4479d77700 1 -- 172.21.15.99:0/2708133365 reap_dead start 2023-10-08T00:19:57.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.655+0000 7f447a578700 1 -- 172.21.15.99:0/2708133365 shutdown_connections 2023-10-08T00:19:57.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.655+0000 7f447a578700 1 -- 172.21.15.99:0/2708133365 >> 172.21.15.99:0/2708133365 conn(0x7f447406bc70 msgr2=0x7f447406c070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:19:57.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.655+0000 7f447a578700 1 -- 172.21.15.99:0/2708133365 shutdown_connections 2023-10-08T00:19:57.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.655+0000 7f447a578700 1 -- 172.21.15.99:0/2708133365 wait complete. 2023-10-08T00:19:57.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.656+0000 7f447a578700 1 Processor -- start 2023-10-08T00:19:57.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.656+0000 7f447a578700 1 -- start start 2023-10-08T00:19:57.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.657+0000 7f447a578700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4460004d50 0x7f44740c5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:57.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.657+0000 7f447a578700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f44740c8190 0x7f447416a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:57.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.657+0000 7f447a578700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f447416a5d0 0x7f447416aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:57.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.657+0000 7f447a578700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447416af40 con 0x7f4460004d50 2023-10-08T00:19:57.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.657+0000 7f447a578700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447416b260 con 0x7f44740c8190 2023-10-08T00:19:57.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.657+0000 7f447a578700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44740c8f40 con 0x7f447416a5d0 2023-10-08T00:19:57.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.657+0000 7f4479576700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4460004d50 0x7f44740c5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:57.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.657+0000 7f4479576700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4460004d50 0x7f44740c5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40786/0 (socket says 172.21.15.99:40786) 2023-10-08T00:19:57.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.657+0000 7f4479576700 1 -- 172.21.15.99:0/1113915381 learned_addr learned my addr 172.21.15.99:0/1113915381 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:19:57.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.658+0000 7f4479d77700 1 --2- 172.21.15.99:0/1113915381 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f447416a5d0 0x7f447416aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:57.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.658+0000 7f4478d75700 1 --2- 172.21.15.99:0/1113915381 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f44740c8190 0x7f447416a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:57.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.658+0000 7f4479576700 1 -- 172.21.15.99:0/1113915381 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f447416a5d0 msgr2=0x7f447416aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:57.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.658+0000 7f4479576700 1 --2- 172.21.15.99:0/1113915381 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f447416a5d0 0x7f447416aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:57.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.658+0000 7f4479576700 1 -- 172.21.15.99:0/1113915381 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f44740c8190 msgr2=0x7f447416a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:57.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.658+0000 7f4479576700 1 --2- 172.21.15.99:0/1113915381 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f44740c8190 0x7f447416a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:57.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.658+0000 7f4479576700 1 -- 172.21.15.99:0/1113915381 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44740c8980 con 0x7f4460004d50 2023-10-08T00:19:57.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.658+0000 7f4478d75700 1 --2- 172.21.15.99:0/1113915381 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f44740c8190 0x7f447416a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:19:57.666 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.658+0000 7f4479576700 1 --2- 172.21.15.99:0/1113915381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4460004d50 0x7f44740c5e00 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f4470009fd0 tx=0x7f447000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:57.666 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.658+0000 7f44697fa700 1 -- 172.21.15.99:0/1113915381 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447000e040 con 0x7f4460004d50 2023-10-08T00:19:57.666 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.658+0000 7f44697fa700 1 -- 172.21.15.99:0/1113915381 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f447000f040 con 0x7f4460004d50 2023-10-08T00:19:57.666 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.658+0000 7f447a578700 1 -- 172.21.15.99:0/1113915381 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44740c8c60 con 0x7f4460004d50 2023-10-08T00:19:57.666 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.658+0000 7f44697fa700 1 -- 172.21.15.99:0/1113915381 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4470015430 con 0x7f4460004d50 2023-10-08T00:19:57.666 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.658+0000 7f447a578700 1 -- 172.21.15.99:0/1113915381 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44740c3770 con 0x7f4460004d50 2023-10-08T00:19:57.666 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.659+0000 7f447a578700 1 -- 172.21.15.99:0/1113915381 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f447406e3e0 con 0x7f4460004d50 2023-10-08T00:19:57.666 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.659+0000 7f44697fa700 1 -- 172.21.15.99:0/1113915381 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4470015860 con 0x7f4460004d50 2023-10-08T00:19:57.667 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.660+0000 7f44697fa700 1 --2- 172.21.15.99:0/1113915381 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4458066f30 0x7f44580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:57.667 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.660+0000 7f44697fa700 1 -- 172.21.15.99:0/1113915381 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f4470084fc0 con 0x7f4460004d50 2023-10-08T00:19:57.667 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.660+0000 7f4478d75700 1 --2- 172.21.15.99:0/1113915381 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4458066f30 0x7f44580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:57.667 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.660+0000 7f4478d75700 1 --2- 172.21.15.99:0/1113915381 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4458066f30 0x7f44580693e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f4464006f00 tx=0x7f4464008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:57.667 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.664+0000 7f44697fa700 1 -- 172.21.15.99:0/1113915381 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4470052490 con 0x7f4460004d50 2023-10-08T00:19:57.922 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:57.921+0000 7f447a578700 1 -- 172.21.15.99:0/1113915381 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]} v 0) v1 -- 0x7f44740cf080 con 0x7f4460004d50 2023-10-08T00:19:58.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.201+0000 7f44697fa700 1 -- 172.21.15.99:0/1113915381 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]}]=0 v431) v1 ==== 132+0+0 (secure 0 0 0) 0x7f4470055ab0 con 0x7f4460004d50 2023-10-08T00:19:58.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.203+0000 7f447a578700 1 -- 172.21.15.99:0/1113915381 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4458066f30 msgr2=0x7f44580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:58.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.203+0000 7f447a578700 1 --2- 172.21.15.99:0/1113915381 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4458066f30 0x7f44580693e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f4464006f00 tx=0x7f4464008040 comp rx=0 tx=0).stop 2023-10-08T00:19:58.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.203+0000 7f447a578700 1 -- 172.21.15.99:0/1113915381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4460004d50 msgr2=0x7f44740c5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:58.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.203+0000 7f447a578700 1 --2- 172.21.15.99:0/1113915381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4460004d50 0x7f44740c5e00 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f4470009fd0 tx=0x7f447000b040 comp rx=0 tx=0).stop 2023-10-08T00:19:58.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.203+0000 7f447a578700 1 -- 172.21.15.99:0/1113915381 shutdown_connections 2023-10-08T00:19:58.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.203+0000 7f447a578700 1 --2- 172.21.15.99:0/1113915381 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4458066f30 0x7f44580693e0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:58.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.203+0000 7f447a578700 1 --2- 172.21.15.99:0/1113915381 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f447416a5d0 0x7f447416aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:58.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.203+0000 7f447a578700 1 --2- 172.21.15.99:0/1113915381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4460004d50 0x7f44740c5e00 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:58.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.203+0000 7f447a578700 1 --2- 172.21.15.99:0/1113915381 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f44740c8190 0x7f447416a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:58.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.203+0000 7f447a578700 1 -- 172.21.15.99:0/1113915381 >> 172.21.15.99:0/1113915381 conn(0x7f447406bc70 msgr2=0x7f4474158db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:19:58.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.203+0000 7f447a578700 1 -- 172.21.15.99:0/1113915381 shutdown_connections 2023-10-08T00:19:58.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.203+0000 7f447a578700 1 -- 172.21.15.99:0/1113915381 wait complete. 2023-10-08T00:19:58.215 INFO:tasks.cephfs_test_runner:test_full_fclose (tasks.cephfs.test_full.TestQuotaFull) ... ok 2023-10-08T00:19:58.216 INFO:tasks.cephfs_test_runner:Starting test: test_full_fsync (tasks.cephfs.test_full.TestQuotaFull) That when the full flag is encountered during asynchronous 2023-10-08T00:19:58.216 INFO:tasks.cephfs_test_runner:test_full_fsync (tasks.cephfs.test_full.TestQuotaFull) 2023-10-08T00:19:58.216 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync' 2023-10-08T00:19:58.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.330+0000 7fea24af4700 1 Processor -- start 2023-10-08T00:19:58.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.330+0000 7fea24af4700 1 -- start start 2023-10-08T00:19:58.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.330+0000 7fea24af4700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fea20067d70 0x7fea2015a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:58.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.330+0000 7fea24af4700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea200cb370 con 0x7fea2006c630 2023-10-08T00:19:58.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.330+0000 7fea24af4700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea200b0c60 con 0x7fea200b0890 2023-10-08T00:19:58.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.330+0000 7fea24af4700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea2006a1e0 con 0x7fea20067d70 2023-10-08T00:19:58.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.331+0000 7fea17fff700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fea2006c630 0x7fea20067660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:54524/0 (socket says 172.21.15.99:54524) 2023-10-08T00:19:58.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.331+0000 7fea17fff700 1 -- 172.21.15.99:0/4205373269 learned_addr learned my addr 172.21.15.99:0/4205373269 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:19:58.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.331+0000 7fea1ed9d700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fea20067d70 0x7fea2015a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:58.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.331+0000 7fea1ed9d700 1 -- 172.21.15.99:0/4205373269 >> v1:172.21.15.191:6789/0 conn(0x7fea200b0890 legacy=0x7fea200cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:58.333 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.331+0000 7fea1ed9d700 1 -- 172.21.15.99:0/4205373269 >> v1:172.21.15.99:6789/0 conn(0x7fea2006c630 legacy=0x7fea20067660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:58.333 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.331+0000 7fea1ed9d700 1 -- 172.21.15.99:0/4205373269 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea2006cb80 con 0x7fea20067d70 2023-10-08T00:19:58.333 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.331+0000 7fea1ed9d700 1 --2- 172.21.15.99:0/4205373269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea20067d70 0x7fea2015a7a0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fea18009e60 tx=0x7fea18009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f2732166ff755cb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:58.333 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.331+0000 7fea1dd9b700 1 -- 172.21.15.99:0/4205373269 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1800e070 con 0x7fea20067d70 2023-10-08T00:19:58.333 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.331+0000 7fea1dd9b700 1 -- 172.21.15.99:0/4205373269 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea1800bc30 con 0x7fea20067d70 2023-10-08T00:19:58.333 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.331+0000 7fea1dd9b700 1 -- 172.21.15.99:0/4205373269 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea18012430 con 0x7fea20067d70 2023-10-08T00:19:58.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.332+0000 7fea24af4700 1 -- 172.21.15.99:0/4205373269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea20067d70 msgr2=0x7fea2015a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:58.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.332+0000 7fea24af4700 1 --2- 172.21.15.99:0/4205373269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea20067d70 0x7fea2015a7a0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fea18009e60 tx=0x7fea18009780 comp rx=0 tx=0).stop 2023-10-08T00:19:58.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.332+0000 7fea24af4700 1 -- 172.21.15.99:0/4205373269 shutdown_connections 2023-10-08T00:19:58.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.332+0000 7fea24af4700 1 --2- 172.21.15.99:0/4205373269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea20067d70 0x7fea2015a7a0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:58.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.332+0000 7fea24af4700 1 -- 172.21.15.99:0/4205373269 >> 172.21.15.99:0/4205373269 conn(0x7fea2006fe50 msgr2=0x7fea20069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:19:58.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.332+0000 7fea24af4700 1 -- 172.21.15.99:0/4205373269 shutdown_connections 2023-10-08T00:19:58.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.332+0000 7fea24af4700 1 -- 172.21.15.99:0/4205373269 wait complete. 2023-10-08T00:19:58.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.333+0000 7fea24af4700 1 Processor -- start 2023-10-08T00:19:58.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.333+0000 7fea24af4700 1 -- start start 2023-10-08T00:19:58.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.333+0000 7fea24af4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea20067d70 0x7fea2015c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:58.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.333+0000 7fea24af4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea2006c630 0x7fea2015cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:58.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.333+0000 7fea24af4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea200b0890 0x7fea2015d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:58.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.333+0000 7fea24af4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea200c5b30 con 0x7fea2006c630 2023-10-08T00:19:58.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.333+0000 7fea24af4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea200c5e10 con 0x7fea20067d70 2023-10-08T00:19:58.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.333+0000 7fea24af4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea200c60f0 con 0x7fea200b0890 2023-10-08T00:19:58.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.334+0000 7fea17fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea2006c630 0x7fea2015cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:58.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.334+0000 7fea17fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea2006c630 0x7fea2015cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40800/0 (socket says 172.21.15.99:40800) 2023-10-08T00:19:58.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.334+0000 7fea17fff700 1 -- 172.21.15.99:0/3909718649 learned_addr learned my addr 172.21.15.99:0/3909718649 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:19:58.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.334+0000 7fea17fff700 1 -- 172.21.15.99:0/3909718649 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea200b0890 msgr2=0x7fea2015d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:58.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.334+0000 7fea1e59c700 1 --2- 172.21.15.99:0/3909718649 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea20067d70 0x7fea2015c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:58.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.334+0000 7fea17fff700 1 --2- 172.21.15.99:0/3909718649 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea200b0890 0x7fea2015d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:58.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.334+0000 7fea17fff700 1 -- 172.21.15.99:0/3909718649 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea20067d70 msgr2=0x7fea2015c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:58.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.334+0000 7fea17fff700 1 --2- 172.21.15.99:0/3909718649 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea20067d70 0x7fea2015c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:58.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.334+0000 7fea17fff700 1 -- 172.21.15.99:0/3909718649 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea18009a20 con 0x7fea2006c630 2023-10-08T00:19:58.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.334+0000 7fea17fff700 1 --2- 172.21.15.99:0/3909718649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea2006c630 0x7fea2015cee0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fea0800c790 tx=0x7fea08007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:58.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.334+0000 7fea167fc700 1 -- 172.21.15.99:0/3909718649 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea08005b30 con 0x7fea2006c630 2023-10-08T00:19:58.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.334+0000 7fea167fc700 1 -- 172.21.15.99:0/3909718649 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea08005cb0 con 0x7fea2006c630 2023-10-08T00:19:58.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.334+0000 7fea24af4700 1 -- 172.21.15.99:0/3909718649 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea200c63d0 con 0x7fea2006c630 2023-10-08T00:19:58.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.334+0000 7fea167fc700 1 -- 172.21.15.99:0/3909718649 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea08004a80 con 0x7fea2006c630 2023-10-08T00:19:58.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.334+0000 7fea24af4700 1 -- 172.21.15.99:0/3909718649 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea2016d7e0 con 0x7fea2006c630 2023-10-08T00:19:58.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.335+0000 7fea24af4700 1 -- 172.21.15.99:0/3909718649 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea2004c3c0 con 0x7fea2006c630 2023-10-08T00:19:58.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.335+0000 7fea167fc700 1 -- 172.21.15.99:0/3909718649 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fea08005030 con 0x7fea2006c630 2023-10-08T00:19:58.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.336+0000 7fea167fc700 1 --2- 172.21.15.99:0/3909718649 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fea00066f30 0x7fea000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:58.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.336+0000 7fea167fc700 1 -- 172.21.15.99:0/3909718649 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fea08083bd0 con 0x7fea2006c630 2023-10-08T00:19:58.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.336+0000 7fea1e59c700 1 --2- 172.21.15.99:0/3909718649 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fea00066f30 0x7fea000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:58.339 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.337+0000 7fea1e59c700 1 --2- 172.21.15.99:0/3909718649 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fea00066f30 0x7fea000693e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fea10000c00 tx=0x7fea10009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:58.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.340+0000 7fea167fc700 1 -- 172.21.15.99:0/3909718649 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fea08051140 con 0x7fea2006c630 2023-10-08T00:19:58.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:58.598+0000 7fea24af4700 1 -- 172.21.15.99:0/3909718649 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]} v 0) v1 -- 0x7fea2006cdc0 con 0x7fea2006c630 2023-10-08T00:19:59.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.202+0000 7fea167fc700 1 -- 172.21.15.99:0/3909718649 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]}]=0 v432) v1 ==== 134+0+0 (secure 0 0 0) 0x7fea08054760 con 0x7fea2006c630 2023-10-08T00:19:59.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.205+0000 7fea24af4700 1 -- 172.21.15.99:0/3909718649 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fea00066f30 msgr2=0x7fea000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:59.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.205+0000 7fea24af4700 1 --2- 172.21.15.99:0/3909718649 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fea00066f30 0x7fea000693e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fea10000c00 tx=0x7fea10009040 comp rx=0 tx=0).stop 2023-10-08T00:19:59.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.205+0000 7fea24af4700 1 -- 172.21.15.99:0/3909718649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea2006c630 msgr2=0x7fea2015cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:59.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.205+0000 7fea24af4700 1 --2- 172.21.15.99:0/3909718649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea2006c630 0x7fea2015cee0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fea0800c790 tx=0x7fea08007f40 comp rx=0 tx=0).stop 2023-10-08T00:19:59.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.205+0000 7fea24af4700 1 -- 172.21.15.99:0/3909718649 shutdown_connections 2023-10-08T00:19:59.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.205+0000 7fea24af4700 1 --2- 172.21.15.99:0/3909718649 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fea00066f30 0x7fea000693e0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:59.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.205+0000 7fea24af4700 1 --2- 172.21.15.99:0/3909718649 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea20067d70 0x7fea2015c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:59.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.205+0000 7fea24af4700 1 --2- 172.21.15.99:0/3909718649 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea200b0890 0x7fea2015d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:59.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.205+0000 7fea24af4700 1 --2- 172.21.15.99:0/3909718649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea2006c630 0x7fea2015cee0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:59.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.205+0000 7fea24af4700 1 -- 172.21.15.99:0/3909718649 >> 172.21.15.99:0/3909718649 conn(0x7fea2006fe50 msgr2=0x7fea2015be40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:19:59.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.205+0000 7fea24af4700 1 -- 172.21.15.99:0/3909718649 shutdown_connections 2023-10-08T00:19:59.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.205+0000 7fea24af4700 1 -- 172.21.15.99:0/3909718649 wait complete. 2023-10-08T00:19:59.218 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T00:19:59.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.337+0000 7f047dd48700 1 Processor -- start 2023-10-08T00:19:59.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.337+0000 7f047dd48700 1 -- start start 2023-10-08T00:19:59.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.338+0000 7f047dd48700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f04780ab910 0x7f04780caf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:59.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.338+0000 7f047dd48700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04780c3430 con 0x7f047815a730 2023-10-08T00:19:59.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.338+0000 7f047dd48700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f047815abc0 con 0x7f04780abce0 2023-10-08T00:19:59.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.338+0000 7f047dd48700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04780c9100 con 0x7f04780ab910 2023-10-08T00:19:59.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.338+0000 7f0476ffd700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f047815a730 0x7f04780c2d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:54526/0 (socket says 172.21.15.99:54526) 2023-10-08T00:19:59.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.338+0000 7f0476ffd700 1 -- 172.21.15.99:0/1708569844 learned_addr learned my addr 172.21.15.99:0/1708569844 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:19:59.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.339+0000 7f04767fc700 1 -- 172.21.15.99:0/1708569844 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2072810453 0 0) 0x7f04780c3430 con 0x7f047815a730 2023-10-08T00:19:59.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.339+0000 7f04777fe700 1 --2- 172.21.15.99:0/1708569844 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f04780ab910 0x7f04780caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:59.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.339+0000 7f04767fc700 1 -- 172.21.15.99:0/1708569844 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0460003570 con 0x7f047815a730 2023-10-08T00:19:59.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.339+0000 7f04767fc700 1 -- 172.21.15.99:0/1708569844 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1922570834 0 0) 0x7f0460003570 con 0x7f047815a730 2023-10-08T00:19:59.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.339+0000 7f04767fc700 1 -- 172.21.15.99:0/1708569844 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f04780ab910 msgr2=0x7f04780caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:19:59.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.339+0000 7f04767fc700 1 --2- 172.21.15.99:0/1708569844 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f04780ab910 0x7f04780caf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:59.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.339+0000 7f04767fc700 1 -- 172.21.15.99:0/1708569844 >> v1:172.21.15.191:6789/0 conn(0x7f04780abce0 legacy=0x7f04780cb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:59.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.339+0000 7f04767fc700 1 -- 172.21.15.99:0/1708569844 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f047815aea0 con 0x7f047815a730 2023-10-08T00:19:59.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.339+0000 7f04767fc700 1 -- 172.21.15.99:0/1708569844 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f046c001470 con 0x7f047815a730 2023-10-08T00:19:59.347 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.339+0000 7f04767fc700 1 -- 172.21.15.99:0/1708569844 >> v1:172.21.15.99:6789/0 conn(0x7f047815a730 legacy=0x7f04780c2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:59.347 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.339+0000 7f04767fc700 1 --2- 172.21.15.99:0/1708569844 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f04780ab910 0x7f0460003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:59.347 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.339+0000 7f04767fc700 1 --2- 172.21.15.99:0/1708569844 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0460003fd0 0x7f0460004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:59.347 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.339+0000 7f04767fc700 1 --2- 172.21.15.99:0/1708569844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0460004d50 0x7f04600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:59.347 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.339+0000 7f04767fc700 1 -- 172.21.15.99:0/1708569844 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04780c9100 con 0x7f0460004d50 2023-10-08T00:19:59.347 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.339+0000 7f04767fc700 1 -- 172.21.15.99:0/1708569844 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f046000bb40 con 0x7f0460003fd0 2023-10-08T00:19:59.347 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.339+0000 7f04767fc700 1 -- 172.21.15.99:0/1708569844 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f046000bd00 con 0x7f04780ab910 2023-10-08T00:19:59.347 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.339+0000 7f0477fff700 1 --2- 172.21.15.99:0/1708569844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0460004d50 0x7f04600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:59.347 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.339+0000 7f0476ffd700 1 --2- 172.21.15.99:0/1708569844 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0460003fd0 0x7f0460004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:59.347 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.339+0000 7f04777fe700 1 --2- 172.21.15.99:0/1708569844 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f04780ab910 0x7f0460003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:59.347 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.339+0000 7f0477fff700 1 -- 172.21.15.99:0/1708569844 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f04780ab910 msgr2=0x7f0460003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:19:59.348 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.339+0000 7f0477fff700 1 --2- 172.21.15.99:0/1708569844 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f04780ab910 0x7f0460003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:59.348 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.339+0000 7f0477fff700 1 -- 172.21.15.99:0/1708569844 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0460003fd0 msgr2=0x7f0460004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:19:59.348 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.339+0000 7f0477fff700 1 --2- 172.21.15.99:0/1708569844 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0460003fd0 0x7f0460004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:59.348 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.339+0000 7f0477fff700 1 -- 172.21.15.99:0/1708569844 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0460003570 con 0x7f0460004d50 2023-10-08T00:19:59.348 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.339+0000 7f0477fff700 1 --2- 172.21.15.99:0/1708569844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0460004d50 0x7f04600051c0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f04700020a0 tx=0x7f04700016d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3227f67e72b0b2d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:59.348 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.340+0000 7f04767fc700 1 -- 172.21.15.99:0/1708569844 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04700019e0 con 0x7f0460004d50 2023-10-08T00:19:59.348 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.340+0000 7f04767fc700 1 -- 172.21.15.99:0/1708569844 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f047000c8b0 con 0x7f0460004d50 2023-10-08T00:19:59.348 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.340+0000 7f04767fc700 1 -- 172.21.15.99:0/1708569844 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047000cc50 con 0x7f0460004d50 2023-10-08T00:19:59.348 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.340+0000 7f047dd48700 1 -- 172.21.15.99:0/1708569844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0460004d50 msgr2=0x7f04600051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:59.348 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.340+0000 7f047dd48700 1 --2- 172.21.15.99:0/1708569844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0460004d50 0x7f04600051c0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f04700020a0 tx=0x7f04700016d0 comp rx=0 tx=0).stop 2023-10-08T00:19:59.348 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.340+0000 7f0477fff700 1 -- 172.21.15.99:0/1708569844 reap_dead start 2023-10-08T00:19:59.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.340+0000 7f047dd48700 1 -- 172.21.15.99:0/1708569844 shutdown_connections 2023-10-08T00:19:59.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.340+0000 7f047dd48700 1 -- 172.21.15.99:0/1708569844 >> 172.21.15.99:0/1708569844 conn(0x7f0478067660 msgr2=0x7f0478067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:19:59.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.340+0000 7f047dd48700 1 -- 172.21.15.99:0/1708569844 shutdown_connections 2023-10-08T00:19:59.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.340+0000 7f047dd48700 1 -- 172.21.15.99:0/1708569844 wait complete. 2023-10-08T00:19:59.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.341+0000 7f047dd48700 1 Processor -- start 2023-10-08T00:19:59.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.341+0000 7f047dd48700 1 -- start start 2023-10-08T00:19:59.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.342+0000 7f047dd48700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0460004d50 0x7f04781562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:59.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.342+0000 7f047dd48700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0478149610 0x7f0478159230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:59.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.342+0000 7f047dd48700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0478159770 0x7f04780c17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:59.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.342+0000 7f047dd48700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0478159be0 con 0x7f0478149610 2023-10-08T00:19:59.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.342+0000 7f047dd48700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0478155e30 con 0x7f0478159770 2023-10-08T00:19:59.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.342+0000 7f047dd48700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04780c2520 con 0x7f0460004d50 2023-10-08T00:19:59.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.342+0000 7f0476ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0478149610 0x7f0478159230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:59.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.342+0000 7f0476ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0478149610 0x7f0478159230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40824/0 (socket says 172.21.15.99:40824) 2023-10-08T00:19:59.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.342+0000 7f0476ffd700 1 -- 172.21.15.99:0/1959115923 learned_addr learned my addr 172.21.15.99:0/1959115923 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:19:59.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.342+0000 7f0477fff700 1 --2- 172.21.15.99:0/1959115923 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0478159770 0x7f04780c17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:59.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.342+0000 7f04777fe700 1 --2- 172.21.15.99:0/1959115923 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0460004d50 0x7f04781562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:59.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.342+0000 7f0476ffd700 1 -- 172.21.15.99:0/1959115923 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0460004d50 msgr2=0x7f04781562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:59.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.342+0000 7f0476ffd700 1 --2- 172.21.15.99:0/1959115923 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0460004d50 0x7f04781562c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:59.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.342+0000 7f0476ffd700 1 -- 172.21.15.99:0/1959115923 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0478159770 msgr2=0x7f04780c17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:59.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.342+0000 7f0476ffd700 1 --2- 172.21.15.99:0/1959115923 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0478159770 0x7f04780c17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:59.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.342+0000 7f0476ffd700 1 -- 172.21.15.99:0/1959115923 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0478155500 con 0x7f0478149610 2023-10-08T00:19:59.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.343+0000 7f0476ffd700 1 --2- 172.21.15.99:0/1959115923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0478149610 0x7f0478159230 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f046c0011e0 tx=0x7f046c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:59.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.343+0000 7f045f7fe700 1 -- 172.21.15.99:0/1959115923 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046c018030 con 0x7f0478149610 2023-10-08T00:19:59.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.343+0000 7f045f7fe700 1 -- 172.21.15.99:0/1959115923 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f046c00f840 con 0x7f0478149610 2023-10-08T00:19:59.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.343+0000 7f047dd48700 1 -- 172.21.15.99:0/1959115923 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04781557e0 con 0x7f0478149610 2023-10-08T00:19:59.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.343+0000 7f045f7fe700 1 -- 172.21.15.99:0/1959115923 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046c00fc00 con 0x7f0478149610 2023-10-08T00:19:59.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.343+0000 7f047dd48700 1 -- 172.21.15.99:0/1959115923 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0478149a80 con 0x7f0478149610 2023-10-08T00:19:59.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.344+0000 7f047dd48700 1 -- 172.21.15.99:0/1959115923 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0478155c80 con 0x7f0478149610 2023-10-08T00:19:59.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.344+0000 7f045f7fe700 1 -- 172.21.15.99:0/1959115923 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f046c01b070 con 0x7f0478149610 2023-10-08T00:19:59.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.344+0000 7f045f7fe700 1 --2- 172.21.15.99:0/1959115923 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0464066f30 0x7f04640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:59.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.345+0000 7f045f7fe700 1 -- 172.21.15.99:0/1959115923 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f046c019070 con 0x7f0478149610 2023-10-08T00:19:59.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.345+0000 7f04777fe700 1 --2- 172.21.15.99:0/1959115923 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0464066f30 0x7f04640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:59.352 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.345+0000 7f04777fe700 1 --2- 172.21.15.99:0/1959115923 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0464066f30 0x7f04640693e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f0468006f00 tx=0x7f04680091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:59.352 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.349+0000 7f045f7fe700 1 -- 172.21.15.99:0/1959115923 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f046c051780 con 0x7f0478149610 2023-10-08T00:19:59.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.530+0000 7f047dd48700 1 -- 172.21.15.99:0/1959115923 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0478155fa0 con 0x7f0478149610 2023-10-08T00:19:59.534 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.534+0000 7f045f7fe700 1 -- 172.21.15.99:0/1959115923 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f046c054da0 con 0x7f0478149610 2023-10-08T00:19:59.537 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.536+0000 7f047dd48700 1 -- 172.21.15.99:0/1959115923 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0464066f30 msgr2=0x7f04640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:59.537 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.536+0000 7f047dd48700 1 --2- 172.21.15.99:0/1959115923 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0464066f30 0x7f04640693e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f0468006f00 tx=0x7f04680091a0 comp rx=0 tx=0).stop 2023-10-08T00:19:59.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.536+0000 7f047dd48700 1 -- 172.21.15.99:0/1959115923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0478149610 msgr2=0x7f0478159230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:59.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.536+0000 7f047dd48700 1 --2- 172.21.15.99:0/1959115923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0478149610 0x7f0478159230 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f046c0011e0 tx=0x7f046c001e70 comp rx=0 tx=0).stop 2023-10-08T00:19:59.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.536+0000 7f047dd48700 1 -- 172.21.15.99:0/1959115923 shutdown_connections 2023-10-08T00:19:59.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.536+0000 7f047dd48700 1 --2- 172.21.15.99:0/1959115923 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0464066f30 0x7f04640693e0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:59.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.536+0000 7f047dd48700 1 --2- 172.21.15.99:0/1959115923 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0478159770 0x7f04780c17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:59.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.536+0000 7f047dd48700 1 --2- 172.21.15.99:0/1959115923 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0460004d50 0x7f04781562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:59.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.536+0000 7f047dd48700 1 --2- 172.21.15.99:0/1959115923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0478149610 0x7f0478159230 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:59.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.536+0000 7f047dd48700 1 -- 172.21.15.99:0/1959115923 >> 172.21.15.99:0/1959115923 conn(0x7f0478067660 msgr2=0x7f047806e170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:19:59.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.536+0000 7f047dd48700 1 -- 172.21.15.99:0/1959115923 shutdown_connections 2023-10-08T00:19:59.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.536+0000 7f047dd48700 1 -- 172.21.15.99:0/1959115923 wait complete. 2023-10-08T00:19:59.549 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T00:19:59.549 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T00:19:59.584 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T00:19:59.631 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T00:19:59.631 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-08T00:19:59.631 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:19:59.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.745+0000 7f390acff700 1 Processor -- start 2023-10-08T00:19:59.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.745+0000 7f390acff700 1 -- start start 2023-10-08T00:19:59.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.746+0000 7f390acff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f390415a620 0x7f390415d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:59.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.746+0000 7f390acff700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f390415cbb0 con 0x7f390415d580 2023-10-08T00:19:59.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.746+0000 7f390acff700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f390415bfa0 con 0x7f3904157870 2023-10-08T00:19:59.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.746+0000 7f390acff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f390415bdb0 con 0x7f390415a620 2023-10-08T00:19:59.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.746+0000 7f390929c700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f390415d580 0x7f3904158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:54542/0 (socket says 172.21.15.99:54542) 2023-10-08T00:19:59.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.746+0000 7f390929c700 1 -- 172.21.15.99:0/4228204006 learned_addr learned my addr 172.21.15.99:0/4228204006 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:19:59.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.746+0000 7f38fbfff700 1 --2- 172.21.15.99:0/4228204006 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f390415a620 0x7f390415d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:59.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.746+0000 7f38fb7fe700 1 -- 172.21.15.99:0/4228204006 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3942185878 0 0) 0x7f390415cbb0 con 0x7f390415d580 2023-10-08T00:19:59.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.746+0000 7f38fb7fe700 1 -- 172.21.15.99:0/4228204006 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38ec003570 con 0x7f390415d580 2023-10-08T00:19:59.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.746+0000 7f38fb7fe700 1 -- 172.21.15.99:0/4228204006 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2049536818 0 0) 0x7f38ec003570 con 0x7f390415d580 2023-10-08T00:19:59.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.746+0000 7f38fb7fe700 1 -- 172.21.15.99:0/4228204006 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f390415a620 msgr2=0x7f390415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:19:59.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.746+0000 7f38fb7fe700 1 --2- 172.21.15.99:0/4228204006 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f390415a620 0x7f390415d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:59.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.746+0000 7f38fb7fe700 1 -- 172.21.15.99:0/4228204006 >> v1:172.21.15.191:6789/0 conn(0x7f3904157870 legacy=0x7f390415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:59.759 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.746+0000 7f38fb7fe700 1 -- 172.21.15.99:0/4228204006 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f390415c280 con 0x7f390415d580 2023-10-08T00:19:59.759 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.746+0000 7f38fb7fe700 1 -- 172.21.15.99:0/4228204006 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f38fc0017f0 con 0x7f390415d580 2023-10-08T00:19:59.759 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.746+0000 7f38fb7fe700 1 -- 172.21.15.99:0/4228204006 >> v1:172.21.15.99:6789/0 conn(0x7f390415d580 legacy=0x7f3904158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:59.759 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.746+0000 7f38fb7fe700 1 --2- 172.21.15.99:0/4228204006 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f390415a620 0x7f38ec003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:59.759 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.746+0000 7f38fb7fe700 1 --2- 172.21.15.99:0/4228204006 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38ec003fd0 0x7f38ec004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:59.759 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.746+0000 7f38fb7fe700 1 --2- 172.21.15.99:0/4228204006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38ec004d50 0x7f38ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:59.760 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.746+0000 7f38fb7fe700 1 -- 172.21.15.99:0/4228204006 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec00bb10 con 0x7f38ec004d50 2023-10-08T00:19:59.760 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.746+0000 7f38fb7fe700 1 -- 172.21.15.99:0/4228204006 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec00bc80 con 0x7f38ec003fd0 2023-10-08T00:19:59.760 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.746+0000 7f38fb7fe700 1 -- 172.21.15.99:0/4228204006 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec00be40 con 0x7f390415a620 2023-10-08T00:19:59.760 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.746+0000 7f390929c700 1 --2- 172.21.15.99:0/4228204006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38ec004d50 0x7f38ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:59.760 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.747+0000 7f38fbfff700 1 --2- 172.21.15.99:0/4228204006 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38ec003fd0 0x7f38ec004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:59.760 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.747+0000 7f3908a9b700 1 --2- 172.21.15.99:0/4228204006 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f390415a620 0x7f38ec003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:59.761 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.747+0000 7f390929c700 1 -- 172.21.15.99:0/4228204006 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f390415a620 msgr2=0x7f38ec003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:19:59.761 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.747+0000 7f390929c700 1 --2- 172.21.15.99:0/4228204006 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f390415a620 0x7f38ec003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:59.761 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.747+0000 7f390929c700 1 -- 172.21.15.99:0/4228204006 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38ec003fd0 msgr2=0x7f38ec004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:19:59.761 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.747+0000 7f390929c700 1 --2- 172.21.15.99:0/4228204006 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38ec003fd0 0x7f38ec004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:59.761 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.747+0000 7f390929c700 1 -- 172.21.15.99:0/4228204006 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38ec003570 con 0x7f38ec004d50 2023-10-08T00:19:59.761 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.747+0000 7f390929c700 1 --2- 172.21.15.99:0/4228204006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38ec004d50 0x7f38ec0051c0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f38fc00a920 tx=0x7f38fc0021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4de905e84362c393 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:59.761 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.747+0000 7f38fb7fe700 1 -- 172.21.15.99:0/4228204006 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38fc0070d0 con 0x7f38ec004d50 2023-10-08T00:19:59.762 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.747+0000 7f38fb7fe700 1 -- 172.21.15.99:0/4228204006 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38fc007230 con 0x7f38ec004d50 2023-10-08T00:19:59.762 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.747+0000 7f38fb7fe700 1 -- 172.21.15.99:0/4228204006 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38fc0075f0 con 0x7f38ec004d50 2023-10-08T00:19:59.762 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.747+0000 7f390acff700 1 -- 172.21.15.99:0/4228204006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38ec004d50 msgr2=0x7f38ec0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:59.762 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.748+0000 7f390acff700 1 --2- 172.21.15.99:0/4228204006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38ec004d50 0x7f38ec0051c0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f38fc00a920 tx=0x7f38fc0021f0 comp rx=0 tx=0).stop 2023-10-08T00:19:59.762 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.748+0000 7f390929c700 1 -- 172.21.15.99:0/4228204006 reap_dead start 2023-10-08T00:19:59.762 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.748+0000 7f390acff700 1 -- 172.21.15.99:0/4228204006 shutdown_connections 2023-10-08T00:19:59.762 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.748+0000 7f390acff700 1 -- 172.21.15.99:0/4228204006 >> 172.21.15.99:0/4228204006 conn(0x7f3904068ce0 msgr2=0x7f39040690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:19:59.763 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.748+0000 7f390acff700 1 -- 172.21.15.99:0/4228204006 shutdown_connections 2023-10-08T00:19:59.763 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.748+0000 7f390acff700 1 -- 172.21.15.99:0/4228204006 wait complete. 2023-10-08T00:19:59.763 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.749+0000 7f390acff700 1 Processor -- start 2023-10-08T00:19:59.763 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.749+0000 7f390acff700 1 -- start start 2023-10-08T00:19:59.763 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.749+0000 7f390acff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f390414bd10 0x7f39040b0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:59.763 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.749+0000 7f390acff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f390416b060 0x7f3904157d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:59.763 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.749+0000 7f390acff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3904158280 0x7f39040b2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:59.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.750+0000 7f390acff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390414f470 con 0x7f390416b060 2023-10-08T00:19:59.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.750+0000 7f390acff700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390414c0e0 con 0x7f390414bd10 2023-10-08T00:19:59.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.750+0000 7f390acff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39040b0f20 con 0x7f3904158280 2023-10-08T00:19:59.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.750+0000 7f38fbfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f390416b060 0x7f3904157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:59.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.750+0000 7f38fbfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f390416b060 0x7f3904157d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40846/0 (socket says 172.21.15.99:40846) 2023-10-08T00:19:59.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.750+0000 7f38fbfff700 1 -- 172.21.15.99:0/2084611673 learned_addr learned my addr 172.21.15.99:0/2084611673 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:19:59.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.750+0000 7f3908a9b700 1 --2- 172.21.15.99:0/2084611673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f390414bd10 0x7f39040b0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:59.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.750+0000 7f390929c700 1 --2- 172.21.15.99:0/2084611673 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3904158280 0x7f39040b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:59.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.750+0000 7f38fbfff700 1 -- 172.21.15.99:0/2084611673 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3904158280 msgr2=0x7f39040b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:59.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.750+0000 7f38fbfff700 1 --2- 172.21.15.99:0/2084611673 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3904158280 0x7f39040b2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:59.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.750+0000 7f38fbfff700 1 -- 172.21.15.99:0/2084611673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f390414bd10 msgr2=0x7f39040b0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:59.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.750+0000 7f38fbfff700 1 --2- 172.21.15.99:0/2084611673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f390414bd10 0x7f39040b0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:59.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.750+0000 7f38fbfff700 1 -- 172.21.15.99:0/2084611673 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f390414eb40 con 0x7f390416b060 2023-10-08T00:19:59.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.750+0000 7f390929c700 1 --2- 172.21.15.99:0/2084611673 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3904158280 0x7f39040b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:19:59.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.750+0000 7f38fbfff700 1 --2- 172.21.15.99:0/2084611673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f390416b060 0x7f3904157d40 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f38f4011b40 tx=0x7f38f40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:59.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.750+0000 7f38f8ff9700 1 -- 172.21.15.99:0/2084611673 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f400f940 con 0x7f390416b060 2023-10-08T00:19:59.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.750+0000 7f38f8ff9700 1 -- 172.21.15.99:0/2084611673 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f400faa0 con 0x7f390416b060 2023-10-08T00:19:59.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.750+0000 7f390acff700 1 -- 172.21.15.99:0/2084611673 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f390414ee20 con 0x7f390416b060 2023-10-08T00:19:59.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.750+0000 7f38f8ff9700 1 -- 172.21.15.99:0/2084611673 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f400fe20 con 0x7f390416b060 2023-10-08T00:19:59.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.750+0000 7f390acff700 1 -- 172.21.15.99:0/2084611673 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f390416b4d0 con 0x7f390416b060 2023-10-08T00:19:59.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.751+0000 7f390acff700 1 -- 172.21.15.99:0/2084611673 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f390404c3c0 con 0x7f390416b060 2023-10-08T00:19:59.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.752+0000 7f38f8ff9700 1 -- 172.21.15.99:0/2084611673 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f38f4014650 con 0x7f390416b060 2023-10-08T00:19:59.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.752+0000 7f38f8ff9700 1 --2- 172.21.15.99:0/2084611673 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f38f0066f30 0x7f38f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:19:59.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.752+0000 7f38f8ff9700 1 -- 172.21.15.99:0/2084611673 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f38f4018030 con 0x7f390416b060 2023-10-08T00:19:59.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.752+0000 7f3908a9b700 1 --2- 172.21.15.99:0/2084611673 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f38f0066f30 0x7f38f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:19:59.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.753+0000 7f3908a9b700 1 --2- 172.21.15.99:0/2084611673 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f38f0066f30 0x7f38f00693e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f39040b58f0 tx=0x7f3900007d10 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:19:59.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.756+0000 7f38f8ff9700 1 -- 172.21.15.99:0/2084611673 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f38f4052520 con 0x7f390416b060 2023-10-08T00:19:59.980 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.980+0000 7f390acff700 1 -- 172.21.15.99:0/2084611673 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f390415c560 con 0x7f390416b060 2023-10-08T00:19:59.981 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.980+0000 7f38f8ff9700 1 -- 172.21.15.99:0/2084611673 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+3526 (secure 0 0 0) 0x7f38f4022020 con 0x7f390416b060 2023-10-08T00:19:59.981 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:19:59.981 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":72,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6056,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/3025449992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3025449992},{"type":"v1","addr":"172.21.15.191:6836","nonce":3025449992}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":6212,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/434540621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":434540621},{"type":"v1","addr":"172.21.15.191:6837","nonce":434540621}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":7476,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/86350138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":86350138},{"type":"v1","addr":"172.21.15.99:6834","nonce":86350138}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":7479,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1282887036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1282887036},{"type":"v1","addr":"172.21.15.99:6835","nonce":1282887036}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[]} 2023-10-08T00:19:59.983 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.982+0000 7f390acff700 1 -- 172.21.15.99:0/2084611673 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f38f0066f30 msgr2=0x7f38f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:59.983 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.982+0000 7f390acff700 1 --2- 172.21.15.99:0/2084611673 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f38f0066f30 0x7f38f00693e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f39040b58f0 tx=0x7f3900007d10 comp rx=0 tx=0).stop 2023-10-08T00:19:59.983 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.982+0000 7f390acff700 1 -- 172.21.15.99:0/2084611673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f390416b060 msgr2=0x7f3904157d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:19:59.983 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.982+0000 7f390acff700 1 --2- 172.21.15.99:0/2084611673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f390416b060 0x7f3904157d40 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f38f4011b40 tx=0x7f38f40091a0 comp rx=0 tx=0).stop 2023-10-08T00:19:59.983 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.982+0000 7f390acff700 1 -- 172.21.15.99:0/2084611673 shutdown_connections 2023-10-08T00:19:59.983 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.982+0000 7f390acff700 1 --2- 172.21.15.99:0/2084611673 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f38f0066f30 0x7f38f00693e0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:59.983 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.982+0000 7f390acff700 1 --2- 172.21.15.99:0/2084611673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f390414bd10 0x7f39040b0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:59.984 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.982+0000 7f390acff700 1 --2- 172.21.15.99:0/2084611673 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3904158280 0x7f39040b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:59.984 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.982+0000 7f390acff700 1 --2- 172.21.15.99:0/2084611673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f390416b060 0x7f3904157d40 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:19:59.984 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.982+0000 7f390acff700 1 -- 172.21.15.99:0/2084611673 >> 172.21.15.99:0/2084611673 conn(0x7f3904068ce0 msgr2=0x7f39040d0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:19:59.984 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.982+0000 7f390acff700 1 -- 172.21.15.99:0/2084611673 shutdown_connections 2023-10-08T00:19:59.984 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:19:59.982+0000 7f390acff700 1 -- 172.21.15.99:0/2084611673 wait complete. 2023-10-08T00:19:59.984 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 72 2023-10-08T00:19:59.995 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T00:19:59.996 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-08T00:19:59.996 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T00:19:59.996 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:19:59.996 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T00:19:59.996 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-08T00:19:59.997 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T00:19:59.997 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:19:59.997 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T00:19:59.997 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-08T00:19:59.997 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T00:19:59.998 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:19:59.998 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T00:19:59.998 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-08T00:19:59.998 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T00:19:59.998 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:20:00.004 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T00:20:00.005 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T00:20:00.005 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T00:20:00.006 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T00:20:00.008 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T00:20:00.008 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T00:20:00.009 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T00:20:00.009 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T00:20:00.046 INFO:tasks.ceph.mds.a:Started 2023-10-08T00:20:00.047 DEBUG:teuthology.parallel:result is None 2023-10-08T00:20:00.047 INFO:tasks.ceph.mds.c:Started 2023-10-08T00:20:00.048 DEBUG:teuthology.parallel:result is None 2023-10-08T00:20:00.051 INFO:tasks.ceph.mds.b:Started 2023-10-08T00:20:00.051 DEBUG:teuthology.parallel:result is None 2023-10-08T00:20:00.052 INFO:tasks.ceph.mds.d:Started 2023-10-08T00:20:00.052 DEBUG:teuthology.parallel:result is None 2023-10-08T00:20:00.052 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-08T00:20:00.121 INFO:tasks.ceph.mds.c.smithi099.stdout:starting mds.c at 2023-10-08T00:20:00.122 INFO:tasks.ceph.mds.a.smithi099.stdout:starting mds.a at 2023-10-08T00:20:00.133 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2023-10-08T00:20:00.134 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2023-10-08T00:20:00.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.174+0000 7f67c2b9f700 1 Processor -- start 2023-10-08T00:20:00.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.174+0000 7f67c2b9f700 1 -- start start 2023-10-08T00:20:00.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.175+0000 7f67c2b9f700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f67bc15b820 0x7f67bc0cb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:00.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.175+0000 7f67c2b9f700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67bc0c9450 con 0x7f67bc0b10a0 2023-10-08T00:20:00.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.175+0000 7f67c2b9f700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67bc0cc520 con 0x7f67bc0b0cd0 2023-10-08T00:20:00.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.175+0000 7f67c2b9f700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67bc0ce960 con 0x7f67bc15b820 2023-10-08T00:20:00.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.175+0000 7f67bbfff700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f67bc0b10a0 0x7f67bc0ca930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:54566/0 (socket says 172.21.15.99:54566) 2023-10-08T00:20:00.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.175+0000 7f67bbfff700 1 -- 172.21.15.99:0/877340427 learned_addr learned my addr 172.21.15.99:0/877340427 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:20:00.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.176+0000 7f67c113c700 1 --2- 172.21.15.99:0/877340427 >> v2:172.21.15.191:3300/0 conn(0x7f67bc15b820 0x7f67bc0cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:00.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.176+0000 7f67bb7fe700 1 -- 172.21.15.99:0/877340427 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3924221815 0 0) 0x7f67bc0c9450 con 0x7f67bc0b10a0 2023-10-08T00:20:00.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.176+0000 7f67bb7fe700 1 -- 172.21.15.99:0/877340427 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67ac003570 con 0x7f67bc0b10a0 2023-10-08T00:20:00.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.176+0000 7f67bb7fe700 1 -- 172.21.15.99:0/877340427 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3414716324 0 0) 0x7f67ac003570 con 0x7f67bc0b10a0 2023-10-08T00:20:00.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.176+0000 7f67bb7fe700 1 -- 172.21.15.99:0/877340427 >> v2:172.21.15.191:3300/0 conn(0x7f67bc15b820 msgr2=0x7f67bc0cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:00.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.176+0000 7f67bb7fe700 1 --2- 172.21.15.99:0/877340427 >> v2:172.21.15.191:3300/0 conn(0x7f67bc15b820 0x7f67bc0cb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:00.180 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.176+0000 7f67bb7fe700 1 -- 172.21.15.99:0/877340427 >> v1:172.21.15.191:6789/0 conn(0x7f67bc0b0cd0 legacy=0x7f67bc0ce250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:00.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.176+0000 7f67bb7fe700 1 -- 172.21.15.99:0/877340427 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67bc0cc910 con 0x7f67bc0b10a0 2023-10-08T00:20:00.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.176+0000 7f67bb7fe700 1 -- 172.21.15.99:0/877340427 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f67a8001470 con 0x7f67bc0b10a0 2023-10-08T00:20:00.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.176+0000 7f67bb7fe700 1 -- 172.21.15.99:0/877340427 >> v1:172.21.15.99:6789/0 conn(0x7f67bc0b10a0 legacy=0x7f67bc0ca930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:00.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.176+0000 7f67bb7fe700 1 --2- 172.21.15.99:0/877340427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67ac003a60 0x7f67ac003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:00.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.176+0000 7f67bb7fe700 1 --2- 172.21.15.99:0/877340427 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67ac004810 0x7f67ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:00.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.176+0000 7f67bb7fe700 1 --2- 172.21.15.99:0/877340427 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67ac0051c0 0x7f67ac00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:00.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.176+0000 7f67bb7fe700 1 -- 172.21.15.99:0/877340427 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc0ce960 con 0x7f67ac003a60 2023-10-08T00:20:00.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.176+0000 7f67bb7fe700 1 -- 172.21.15.99:0/877340427 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67ac00c160 con 0x7f67ac004810 2023-10-08T00:20:00.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.176+0000 7f67bb7fe700 1 -- 172.21.15.99:0/877340427 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67ac00c320 con 0x7f67ac0051c0 2023-10-08T00:20:00.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.176+0000 7f67c093b700 1 --2- 172.21.15.99:0/877340427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67ac003a60 0x7f67ac003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:00.182 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.176+0000 7f67bbfff700 1 --2- 172.21.15.99:0/877340427 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67ac004810 0x7f67ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:00.182 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.176+0000 7f67c113c700 1 --2- 172.21.15.99:0/877340427 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67ac0051c0 0x7f67ac00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:00.182 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.176+0000 7f67c093b700 1 -- 172.21.15.99:0/877340427 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67ac0051c0 msgr2=0x7f67ac00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:00.182 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.176+0000 7f67c093b700 1 --2- 172.21.15.99:0/877340427 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67ac0051c0 0x7f67ac00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:00.182 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.176+0000 7f67c093b700 1 -- 172.21.15.99:0/877340427 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67ac004810 msgr2=0x7f67ac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:00.182 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.176+0000 7f67c093b700 1 --2- 172.21.15.99:0/877340427 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67ac004810 0x7f67ac004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:00.182 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.176+0000 7f67c093b700 1 -- 172.21.15.99:0/877340427 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67ac003570 con 0x7f67ac003a60 2023-10-08T00:20:00.182 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.176+0000 7f67c113c700 1 --2- 172.21.15.99:0/877340427 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67ac0051c0 0x7f67ac00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:20:00.182 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.176+0000 7f67c113c700 1 -- 172.21.15.99:0/877340427 reap_dead start 2023-10-08T00:20:00.183 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.177+0000 7f67c093b700 1 --2- 172.21.15.99:0/877340427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67ac003a60 0x7f67ac003e70 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f67b00018b0 tx=0x7f67b0007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=437be81369941a4f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:00.183 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.177+0000 7f67bb7fe700 1 -- 172.21.15.99:0/877340427 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b0012420 con 0x7f67ac003a60 2023-10-08T00:20:00.183 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.177+0000 7f67bb7fe700 1 -- 172.21.15.99:0/877340427 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b00125a0 con 0x7f67ac003a60 2023-10-08T00:20:00.183 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.177+0000 7f67bb7fe700 1 -- 172.21.15.99:0/877340427 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b0012960 con 0x7f67ac003a60 2023-10-08T00:20:00.183 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.177+0000 7f67c2b9f700 1 -- 172.21.15.99:0/877340427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67ac003a60 msgr2=0x7f67ac003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:00.183 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.177+0000 7f67c2b9f700 1 --2- 172.21.15.99:0/877340427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67ac003a60 0x7f67ac003e70 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f67b00018b0 tx=0x7f67b0007d30 comp rx=0 tx=0).stop 2023-10-08T00:20:00.183 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.177+0000 7f67c2b9f700 1 -- 172.21.15.99:0/877340427 shutdown_connections 2023-10-08T00:20:00.183 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.177+0000 7f67c2b9f700 1 --2- 172.21.15.99:0/877340427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67ac003a60 0x7f67ac003e70 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:00.183 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.177+0000 7f67c2b9f700 1 -- 172.21.15.99:0/877340427 >> 172.21.15.99:0/877340427 conn(0x7f67bc06fdc0 msgr2=0x7f67bc06bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:20:00.184 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.177+0000 7f67c2b9f700 1 -- 172.21.15.99:0/877340427 shutdown_connections 2023-10-08T00:20:00.184 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.177+0000 7f67c2b9f700 1 -- 172.21.15.99:0/877340427 wait complete. 2023-10-08T00:20:00.184 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.178+0000 7f67c2b9f700 1 Processor -- start 2023-10-08T00:20:00.184 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.179+0000 7f67c2b9f700 1 -- start start 2023-10-08T00:20:00.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.179+0000 7f67c2b9f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67ac003a60 0x7f67bc0b1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:00.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.179+0000 7f67c2b9f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67bc16be60 0x7f67bc15a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:00.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.179+0000 7f67c2b9f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67bc15ab20 0x7f67bc15c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:00.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.179+0000 7f67c2b9f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc15bce0 con 0x7f67bc16be60 2023-10-08T00:20:00.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.179+0000 7f67c2b9f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc15af90 con 0x7f67ac003a60 2023-10-08T00:20:00.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.179+0000 7f67c2b9f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc16c2b0 con 0x7f67bc15ab20 2023-10-08T00:20:00.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.179+0000 7f67bbfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67bc16be60 0x7f67bc15a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:00.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.179+0000 7f67bbfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67bc16be60 0x7f67bc15a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40878/0 (socket says 172.21.15.99:40878) 2023-10-08T00:20:00.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.179+0000 7f67bbfff700 1 -- 172.21.15.99:0/992158519 learned_addr learned my addr 172.21.15.99:0/992158519 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:20:00.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.180+0000 7f67c113c700 1 --2- 172.21.15.99:0/992158519 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67bc15ab20 0x7f67bc15c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:00.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.180+0000 7f67c093b700 1 --2- 172.21.15.99:0/992158519 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67ac003a60 0x7f67bc0b1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:00.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.180+0000 7f67bbfff700 1 -- 172.21.15.99:0/992158519 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67bc15ab20 msgr2=0x7f67bc15c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:00.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.180+0000 7f67bbfff700 1 --2- 172.21.15.99:0/992158519 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67bc15ab20 0x7f67bc15c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:00.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.180+0000 7f67bbfff700 1 -- 172.21.15.99:0/992158519 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67ac003a60 msgr2=0x7f67bc0b1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:00.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.180+0000 7f67bbfff700 1 --2- 172.21.15.99:0/992158519 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67ac003a60 0x7f67bc0b1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:00.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.180+0000 7f67bbfff700 1 -- 172.21.15.99:0/992158519 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67bc0b1f40 con 0x7f67bc16be60 2023-10-08T00:20:00.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.180+0000 7f67c113c700 1 --2- 172.21.15.99:0/992158519 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67bc15ab20 0x7f67bc15c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:20:00.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.180+0000 7f67c093b700 1 --2- 172.21.15.99:0/992158519 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67ac003a60 0x7f67bc0b1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:20:00.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.180+0000 7f67bbfff700 1 --2- 172.21.15.99:0/992158519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67bc16be60 0x7f67bc15a5e0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f67a80073c0 tx=0x7f67a8001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:00.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.180+0000 7f67b8ff9700 1 -- 172.21.15.99:0/992158519 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a8018030 con 0x7f67bc16be60 2023-10-08T00:20:00.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.180+0000 7f67b8ff9700 1 -- 172.21.15.99:0/992158519 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67a800f8d0 con 0x7f67bc16be60 2023-10-08T00:20:00.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.180+0000 7f67c2b9f700 1 -- 172.21.15.99:0/992158519 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67bc0b2220 con 0x7f67bc16be60 2023-10-08T00:20:00.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.180+0000 7f67b8ff9700 1 -- 172.21.15.99:0/992158519 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a8001bb0 con 0x7f67bc16be60 2023-10-08T00:20:00.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.180+0000 7f67c2b9f700 1 -- 172.21.15.99:0/992158519 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67bc0b27e0 con 0x7f67bc16be60 2023-10-08T00:20:00.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.181+0000 7f67c2b9f700 1 -- 172.21.15.99:0/992158519 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67bc15be20 con 0x7f67bc16be60 2023-10-08T00:20:00.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.181+0000 7f67b8ff9700 1 -- 172.21.15.99:0/992158519 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f67a801b070 con 0x7f67bc16be60 2023-10-08T00:20:00.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.182+0000 7f67b8ff9700 1 --2- 172.21.15.99:0/992158519 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f67a0066f60 0x7f67a0069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:00.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.182+0000 7f67b8ff9700 1 -- 172.21.15.99:0/992158519 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f67a8019070 con 0x7f67bc16be60 2023-10-08T00:20:00.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.182+0000 7f67c093b700 1 --2- 172.21.15.99:0/992158519 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f67a0066f60 0x7f67a0069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:00.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.182+0000 7f67c093b700 1 --2- 172.21.15.99:0/992158519 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f67a0066f60 0x7f67a0069410 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f67b00018b0 tx=0x7f67b0015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:00.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.186+0000 7f67b8ff9700 1 -- 172.21.15.99:0/992158519 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f67bc15be20 con 0x7f67bc16be60 2023-10-08T00:20:00.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:00.383+0000 7f67c2b9f700 1 -- 172.21.15.99:0/992158519 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f67bc15be20 con 0x7f67bc16be60 2023-10-08T00:20:01.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.214+0000 7f67b8ff9700 1 -- 172.21.15.99:0/992158519 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v164) v1 ==== 97+0+0 (secure 0 0 0) 0x7f67bc15be20 con 0x7f67bc16be60 2023-10-08T00:20:01.217 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.216+0000 7f67c2b9f700 1 -- 172.21.15.99:0/992158519 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f67a0066f60 msgr2=0x7f67a0069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:01.217 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.216+0000 7f67c2b9f700 1 --2- 172.21.15.99:0/992158519 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f67a0066f60 0x7f67a0069410 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f67b00018b0 tx=0x7f67b0015000 comp rx=0 tx=0).stop 2023-10-08T00:20:01.218 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.216+0000 7f67c2b9f700 1 -- 172.21.15.99:0/992158519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67bc16be60 msgr2=0x7f67bc15a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:01.218 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.216+0000 7f67c2b9f700 1 --2- 172.21.15.99:0/992158519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67bc16be60 0x7f67bc15a5e0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f67a80073c0 tx=0x7f67a8001e50 comp rx=0 tx=0).stop 2023-10-08T00:20:01.218 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.216+0000 7f67c2b9f700 1 -- 172.21.15.99:0/992158519 shutdown_connections 2023-10-08T00:20:01.218 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.216+0000 7f67c2b9f700 1 --2- 172.21.15.99:0/992158519 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f67a0066f60 0x7f67a0069410 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:01.218 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.216+0000 7f67c2b9f700 1 --2- 172.21.15.99:0/992158519 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67ac003a60 0x7f67bc0b1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:01.218 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.216+0000 7f67c2b9f700 1 --2- 172.21.15.99:0/992158519 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67bc15ab20 0x7f67bc15c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:01.218 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.216+0000 7f67c2b9f700 1 --2- 172.21.15.99:0/992158519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67bc16be60 0x7f67bc15a5e0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:01.218 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.216+0000 7f67c2b9f700 1 -- 172.21.15.99:0/992158519 >> 172.21.15.99:0/992158519 conn(0x7f67bc06fdc0 msgr2=0x7f67bc0c4380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:20:01.218 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.217+0000 7f67c2b9f700 1 -- 172.21.15.99:0/992158519 shutdown_connections 2023-10-08T00:20:01.219 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.217+0000 7f67c2b9f700 1 -- 172.21.15.99:0/992158519 wait complete. 2023-10-08T00:20:01.219 INFO:teuthology.orchestra.run.smithi099.stderr: removed all blocklist entries 2023-10-08T00:20:01.230 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-08T00:20:01.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.346+0000 7fbea36ff700 1 Processor -- start 2023-10-08T00:20:01.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.346+0000 7fbea36ff700 1 -- start start 2023-10-08T00:20:01.348 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.347+0000 7fbea36ff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbe9c157c40 0x7fbe9c158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:01.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.347+0000 7fbea36ff700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe9c15cbb0 con 0x7fbe9c157870 2023-10-08T00:20:01.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.347+0000 7fbea36ff700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe9c15bfa0 con 0x7fbe9c15a620 2023-10-08T00:20:01.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.347+0000 7fbea36ff700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe9c15bdb0 con 0x7fbe9c157c40 2023-10-08T00:20:01.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.347+0000 7fbea1c9c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbe9c157c40 0x7fbe9c158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:01.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.347+0000 7fbea149b700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fbe9c157870 0x7fbe9c15b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:54576/0 (socket says 172.21.15.99:54576) 2023-10-08T00:20:01.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.347+0000 7fbea149b700 1 -- 172.21.15.99:0/2128136680 learned_addr learned my addr 172.21.15.99:0/2128136680 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:20:01.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.347+0000 7fbea1c9c700 1 -- 172.21.15.99:0/2128136680 >> v1:172.21.15.191:6789/0 conn(0x7fbe9c15a620 legacy=0x7fbe9c15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:01.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.347+0000 7fbea1c9c700 1 -- 172.21.15.99:0/2128136680 >> v1:172.21.15.99:6789/0 conn(0x7fbe9c157870 legacy=0x7fbe9c15b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:01.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.347+0000 7fbea1c9c700 1 -- 172.21.15.99:0/2128136680 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe9c15c280 con 0x7fbe9c157c40 2023-10-08T00:20:01.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.347+0000 7fbe93fff700 1 -- 172.21.15.99:0/2128136680 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 837991872 0 0) 0x7fbe9c15cbb0 con 0x7fbe9c157870 2023-10-08T00:20:01.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.347+0000 7fbea1c9c700 1 --2- 172.21.15.99:0/2128136680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe9c157c40 0x7fbe9c158110 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fbe94009e60 tx=0x7fbe94009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60c625015e3a2fb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:01.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.347+0000 7fbe93fff700 1 -- 172.21.15.99:0/2128136680 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9400e070 con 0x7fbe9c157c40 2023-10-08T00:20:01.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.347+0000 7fbe93fff700 1 -- 172.21.15.99:0/2128136680 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe9400bbf0 con 0x7fbe9c157c40 2023-10-08T00:20:01.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.347+0000 7fbe93fff700 1 -- 172.21.15.99:0/2128136680 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe94012430 con 0x7fbe9c157c40 2023-10-08T00:20:01.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.348+0000 7fbea36ff700 1 -- 172.21.15.99:0/2128136680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe9c157c40 msgr2=0x7fbe9c158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:01.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.348+0000 7fbea36ff700 1 --2- 172.21.15.99:0/2128136680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe9c157c40 0x7fbe9c158110 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fbe94009e60 tx=0x7fbe94009780 comp rx=0 tx=0).stop 2023-10-08T00:20:01.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.348+0000 7fbea36ff700 1 -- 172.21.15.99:0/2128136680 shutdown_connections 2023-10-08T00:20:01.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.348+0000 7fbea36ff700 1 --2- 172.21.15.99:0/2128136680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe9c157c40 0x7fbe9c158110 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:01.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.348+0000 7fbea36ff700 1 -- 172.21.15.99:0/2128136680 >> 172.21.15.99:0/2128136680 conn(0x7fbe9c068ce0 msgr2=0x7fbe9c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:20:01.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.348+0000 7fbea36ff700 1 -- 172.21.15.99:0/2128136680 shutdown_connections 2023-10-08T00:20:01.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.348+0000 7fbea36ff700 1 -- 172.21.15.99:0/2128136680 wait complete. 2023-10-08T00:20:01.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.349+0000 7fbea36ff700 1 Processor -- start 2023-10-08T00:20:01.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.349+0000 7fbea36ff700 1 -- start start 2023-10-08T00:20:01.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.350+0000 7fbea36ff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe9c157870 0x7fbe9c0b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:01.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.350+0000 7fbea36ff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbe9c157c40 0x7fbe9c153800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:01.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.350+0000 7fbea36ff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe9c15a620 0x7fbe9c16b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:01.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.350+0000 7fbea36ff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9c0b25d0 con 0x7fbe9c157870 2023-10-08T00:20:01.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.350+0000 7fbea36ff700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9c0b2fa0 con 0x7fbe9c15a620 2023-10-08T00:20:01.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.350+0000 7fbea36ff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9c0b19c0 con 0x7fbe9c157c40 2023-10-08T00:20:01.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.350+0000 7fbea149b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe9c157870 0x7fbe9c0b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:01.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.350+0000 7fbea149b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe9c157870 0x7fbe9c0b2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40900/0 (socket says 172.21.15.99:40900) 2023-10-08T00:20:01.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.350+0000 7fbea149b700 1 -- 172.21.15.99:0/2599346647 learned_addr learned my addr 172.21.15.99:0/2599346647 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:20:01.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.350+0000 7fbea1c9c700 1 --2- 172.21.15.99:0/2599346647 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe9c15a620 0x7fbe9c16b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:01.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.350+0000 7fbea0c9a700 1 --2- 172.21.15.99:0/2599346647 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbe9c157c40 0x7fbe9c153800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:01.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.350+0000 7fbea149b700 1 -- 172.21.15.99:0/2599346647 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbe9c157c40 msgr2=0x7fbe9c153800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:01.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.350+0000 7fbea149b700 1 --2- 172.21.15.99:0/2599346647 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbe9c157c40 0x7fbe9c153800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:01.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.350+0000 7fbea149b700 1 -- 172.21.15.99:0/2599346647 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe9c15a620 msgr2=0x7fbe9c16b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:01.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.350+0000 7fbea149b700 1 --2- 172.21.15.99:0/2599346647 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe9c15a620 0x7fbe9c16b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:01.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.350+0000 7fbea149b700 1 -- 172.21.15.99:0/2599346647 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe94009a20 con 0x7fbe9c157870 2023-10-08T00:20:01.354 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.350+0000 7fbea149b700 1 --2- 172.21.15.99:0/2599346647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe9c157870 0x7fbe9c0b2a60 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fbe9800c7e0 tx=0x7fbe98007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:01.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.351+0000 7fbe917fa700 1 -- 172.21.15.99:0/2599346647 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe98005b30 con 0x7fbe9c157870 2023-10-08T00:20:01.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.351+0000 7fbe917fa700 1 -- 172.21.15.99:0/2599346647 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe98005c90 con 0x7fbe9c157870 2023-10-08T00:20:01.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.351+0000 7fbe917fa700 1 -- 172.21.15.99:0/2599346647 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe98007300 con 0x7fbe9c157870 2023-10-08T00:20:01.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.351+0000 7fbea36ff700 1 -- 172.21.15.99:0/2599346647 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe9c0b1ca0 con 0x7fbe9c157870 2023-10-08T00:20:01.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.351+0000 7fbea36ff700 1 -- 172.21.15.99:0/2599346647 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe9c0b2260 con 0x7fbe9c157870 2023-10-08T00:20:01.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.352+0000 7fbea36ff700 1 -- 172.21.15.99:0/2599346647 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe9c0b28b0 con 0x7fbe9c157870 2023-10-08T00:20:01.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.352+0000 7fbe917fa700 1 -- 172.21.15.99:0/2599346647 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbe9801b400 con 0x7fbe9c157870 2023-10-08T00:20:01.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.352+0000 7fbe917fa700 1 --2- 172.21.15.99:0/2599346647 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbe80066f30 0x7fbe800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:01.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.352+0000 7fbe917fa700 1 -- 172.21.15.99:0/2599346647 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fbe98083c90 con 0x7fbe9c157870 2023-10-08T00:20:01.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.352+0000 7fbea0c9a700 1 --2- 172.21.15.99:0/2599346647 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbe80066f30 0x7fbe800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:01.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.353+0000 7fbea0c9a700 1 --2- 172.21.15.99:0/2599346647 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbe80066f30 0x7fbe800693e0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c006fd0 tx=0x7fbe8c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:01.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.357+0000 7fbe917fa700 1 -- 172.21.15.99:0/2599346647 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbe980512d0 con 0x7fbe9c157870 2023-10-08T00:20:01.614 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.613+0000 7fbea36ff700 1 -- 172.21.15.99:0/2599346647 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fbe9c04c3c0 con 0x7fbe9c157870 2023-10-08T00:20:01.614 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.614+0000 7fbe917fa700 1 -- 172.21.15.99:0/2599346647 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fbe980548f0 con 0x7fbe9c157870 2023-10-08T00:20:01.615 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:20:01.615 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:20:01.615 INFO:teuthology.orchestra.run.smithi099.stdout: "auth_dump": [ 2023-10-08T00:20:01.615 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:20:01.615 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.a", 2023-10-08T00:20:01.615 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlFlN9AhAAKdbeH0XznSOuyUnReD6RsA==", 2023-10-08T00:20:01.615 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:20:01.615 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:20:01.615 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2023-10-08T00:20:01.615 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2023-10-08T00:20:01.616 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:20:01.616 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:20:01.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:20:01.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:20:01.616 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.b", 2023-10-08T00:20:01.616 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlh2G0DRAA1mYGrUrELHsS9abUxCCG2g==", 2023-10-08T00:20:01.616 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:20:01.616 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:20:01.616 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2023-10-08T00:20:01.616 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2023-10-08T00:20:01.616 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:20:01.617 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:20:01.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:20:01.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:20:01.617 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.c", 2023-10-08T00:20:01.617 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFl/M1DCRAANrfHJGZO1s0KOyCdUzZWIA==", 2023-10-08T00:20:01.617 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:20:01.617 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:20:01.617 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2023-10-08T00:20:01.617 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2023-10-08T00:20:01.617 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:20:01.617 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:20:01.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:20:01.618 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:20:01.618 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.d", 2023-10-08T00:20:01.618 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlr5F2FBAAYYeG6EM0/N0FiEAwSbYRyA==", 2023-10-08T00:20:01.618 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:20:01.618 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:20:01.618 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2023-10-08T00:20:01.618 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2023-10-08T00:20:01.618 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:20:01.618 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:20:01.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:20:01.618 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:20:01.619 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.0", 2023-10-08T00:20:01.619 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB08yFla7rpKxAAWTOutLcrjDeR69JbPZMoLg==", 2023-10-08T00:20:01.619 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:20:01.619 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:20:01.619 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:20:01.619 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:20:01.619 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:20:01.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:20:01.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:20:01.619 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.1", 2023-10-08T00:20:01.619 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB08yFlw+y2MhAAAvLDIB/tb49IlaireTqTFg==", 2023-10-08T00:20:01.620 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:20:01.620 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:20:01.620 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:20:01.620 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:20:01.620 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:20:01.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:20:01.620 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:20:01.620 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.2", 2023-10-08T00:20:01.620 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB18yFlh4wvABAALXOtJZoEfGmt/pOBxReooQ==", 2023-10-08T00:20:01.620 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:20:01.620 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:20:01.620 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:20:01.621 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:20:01.621 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:20:01.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:20:01.621 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:20:01.621 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.3", 2023-10-08T00:20:01.621 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB18yFl8cVfCRAAoGXTACMT+f5Avll5mxG9OA==", 2023-10-08T00:20:01.621 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:20:01.621 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:20:01.621 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:20:01.621 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:20:01.621 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:20:01.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:20:01.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:20:01.622 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.4", 2023-10-08T00:20:01.622 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB48yFlft9WGhAA8JZN4MjSX9wbXiMKJnfCYg==", 2023-10-08T00:20:01.622 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:20:01.622 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:20:01.622 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:20:01.622 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:20:01.622 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:20:01.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:20:01.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:20:01.623 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.5", 2023-10-08T00:20:01.623 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB48yFlUf1BIRAA2FMBt1jAcm4ia/vYBDyojw==", 2023-10-08T00:20:01.623 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:20:01.623 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:20:01.623 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:20:01.623 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:20:01.623 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:20:01.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:20:01.623 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:20:01.623 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.6", 2023-10-08T00:20:01.623 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB48yFl0SI6KhAAI6G+Sqfm7OQmGf9nK59x9Q==", 2023-10-08T00:20:01.623 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:20:01.624 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:20:01.624 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:20:01.624 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:20:01.624 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:20:01.624 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:20:01.624 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:20:01.624 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.7", 2023-10-08T00:20:01.624 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB48yFlo5xWMxAA4p9OSLK0JpWhEExsifX+3A==", 2023-10-08T00:20:01.624 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:20:01.624 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:20:01.624 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:20:01.625 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:20:01.625 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:20:01.625 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:20:01.625 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:20:01.625 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.0", 2023-10-08T00:20:01.625 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlmzhQGBAAReslU/Pvcsr0dwE6l1N3/Q==", 2023-10-08T00:20:01.625 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:20:01.625 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:20:01.625 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow r", 2023-10-08T00:20:01.625 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-08T00:20:01.625 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:20:01.626 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:20:01.626 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:20:01.626 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.1", 2023-10-08T00:20:01.626 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFl2NKsIBAAUg6rXynl1J7qB6BkUizmpA==", 2023-10-08T00:20:01.626 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:20:01.626 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:20:01.626 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2023-10-08T00:20:01.626 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2023-10-08T00:20:01.626 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2023-10-08T00:20:01.626 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:20:01.626 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:20:01.627 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:20:01.627 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.2", 2023-10-08T00:20:01.627 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlMp6rHRAArVgJfA3mY4n78hOMeVlw1Q==", 2023-10-08T00:20:01.627 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:20:01.627 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:20:01.627 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2023-10-08T00:20:01.627 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2023-10-08T00:20:01.627 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2023-10-08T00:20:01.627 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:20:01.627 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:20:01.627 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:20:01.628 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.3", 2023-10-08T00:20:01.628 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlH0gNJhAAEgGVGiabVIWknpWUohOnHA==", 2023-10-08T00:20:01.628 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:20:01.628 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:20:01.628 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2023-10-08T00:20:01.628 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2023-10-08T00:20:01.628 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2023-10-08T00:20:01.628 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:20:01.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:20:01.628 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:20:01.628 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.admin", 2023-10-08T00:20:01.628 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBw8yFlidUfIxAAIvpvkHbyCx5JFvHo/g5A9g==", 2023-10-08T00:20:01.629 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:20:01.629 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2023-10-08T00:20:01.629 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2023-10-08T00:20:01.629 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2023-10-08T00:20:01.629 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:20:01.629 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:20:01.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:20:01.629 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:20:01.629 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mgr.x", 2023-10-08T00:20:01.629 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBw8yFl2kMLOxAAsEyuLBLsjQCUS0ERDOi75A==", 2023-10-08T00:20:01.629 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:20:01.630 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2023-10-08T00:20:01.630 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile mgr", 2023-10-08T00:20:01.630 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:20:01.630 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:20:01.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:20:01.630 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:20:01.630 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mgr.y", 2023-10-08T00:20:01.630 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBw8yFl6ht5NxAAFzXnLDjklCePD6hw/pSfFg==", 2023-10-08T00:20:01.630 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:20:01.630 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2023-10-08T00:20:01.630 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile mgr", 2023-10-08T00:20:01.631 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:20:01.631 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:20:01.631 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:20:01.631 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:20:01.631 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:20:01.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.616+0000 7fbea36ff700 1 -- 172.21.15.99:0/2599346647 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbe80066f30 msgr2=0x7fbe800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:01.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.616+0000 7fbea36ff700 1 --2- 172.21.15.99:0/2599346647 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbe80066f30 0x7fbe800693e0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c006fd0 tx=0x7fbe8c008040 comp rx=0 tx=0).stop 2023-10-08T00:20:01.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.616+0000 7fbea36ff700 1 -- 172.21.15.99:0/2599346647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe9c157870 msgr2=0x7fbe9c0b2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:01.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.616+0000 7fbea36ff700 1 --2- 172.21.15.99:0/2599346647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe9c157870 0x7fbe9c0b2a60 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fbe9800c7e0 tx=0x7fbe98007df0 comp rx=0 tx=0).stop 2023-10-08T00:20:01.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.616+0000 7fbea36ff700 1 -- 172.21.15.99:0/2599346647 shutdown_connections 2023-10-08T00:20:01.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.616+0000 7fbea36ff700 1 --2- 172.21.15.99:0/2599346647 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fbe80066f30 0x7fbe800693e0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:01.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.616+0000 7fbea36ff700 1 --2- 172.21.15.99:0/2599346647 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe9c15a620 0x7fbe9c16b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:01.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.616+0000 7fbea36ff700 1 --2- 172.21.15.99:0/2599346647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe9c157870 0x7fbe9c0b2a60 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:01.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.616+0000 7fbea36ff700 1 --2- 172.21.15.99:0/2599346647 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbe9c157c40 0x7fbe9c153800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:01.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.616+0000 7fbea36ff700 1 -- 172.21.15.99:0/2599346647 >> 172.21.15.99:0/2599346647 conn(0x7fbe9c068ce0 msgr2=0x7fbe9c0690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:20:01.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.616+0000 7fbea36ff700 1 -- 172.21.15.99:0/2599346647 shutdown_connections 2023-10-08T00:20:01.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.616+0000 7fbea36ff700 1 -- 172.21.15.99:0/2599346647 wait complete. 2023-10-08T00:20:01.634 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:20:01.748 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.747+0000 7f51cce58700 1 Processor -- start 2023-10-08T00:20:01.748 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.747+0000 7f51cce58700 1 -- start start 2023-10-08T00:20:01.750 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.748+0000 7f51cce58700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f51c81510f0 0x7f51c806ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:01.750 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.748+0000 7f51cce58700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f51c80c9af0 0x7f51c815dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:01.751 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.748+0000 7f51cce58700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f51c8070700 0x7f51c8070b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:01.751 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.748+0000 7f51cce58700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51c8071080 con 0x7f51c81510f0 2023-10-08T00:20:01.751 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.748+0000 7f51cce58700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51c815e280 con 0x7f51c8070700 2023-10-08T00:20:01.751 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.748+0000 7f51cce58700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51c80ca0b0 con 0x7f51c80c9af0 2023-10-08T00:20:01.751 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.748+0000 7f51c659c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f51c81510f0 0x7f51c806ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:01.751 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.748+0000 7f51c659c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f51c81510f0 0x7f51c806ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40912/0 (socket says 172.21.15.99:40912) 2023-10-08T00:20:01.751 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.748+0000 7f51c659c700 1 -- 172.21.15.99:0/708749322 learned_addr learned my addr 172.21.15.99:0/708749322 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:20:01.751 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.748+0000 7f51c6d9d700 1 --2- 172.21.15.99:0/708749322 >> v2:172.21.15.191:3300/0 conn(0x7f51c8070700 0x7f51c8070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:01.751 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.748+0000 7f51bdd9b700 1 --2- 172.21.15.99:0/708749322 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f51c80c9af0 0x7f51c815dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:01.751 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.749+0000 7f51c659c700 1 -- 172.21.15.99:0/708749322 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f51c80c9af0 msgr2=0x7f51c815dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:01.751 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.749+0000 7f51c659c700 1 --2- 172.21.15.99:0/708749322 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f51c80c9af0 0x7f51c815dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:01.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.749+0000 7f51c659c700 1 -- 172.21.15.99:0/708749322 >> v2:172.21.15.191:3300/0 conn(0x7f51c8070700 msgr2=0x7f51c8070b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:01.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.749+0000 7f51c659c700 1 --2- 172.21.15.99:0/708749322 >> v2:172.21.15.191:3300/0 conn(0x7f51c8070700 0x7f51c8070b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:01.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.749+0000 7f51c659c700 1 -- 172.21.15.99:0/708749322 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c8068a10 con 0x7f51c81510f0 2023-10-08T00:20:01.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.749+0000 7f51c6d9d700 1 --2- 172.21.15.99:0/708749322 >> v2:172.21.15.191:3300/0 conn(0x7f51c8070700 0x7f51c8070b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:20:01.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.749+0000 7f51c659c700 1 --2- 172.21.15.99:0/708749322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51c81510f0 0x7f51c806ecc0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f51c80c8050 tx=0x7f51b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd90e0bc8ee4cf56 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:01.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.749+0000 7f51c5d9b700 1 -- 172.21.15.99:0/708749322 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b800b860 con 0x7f51c81510f0 2023-10-08T00:20:01.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.749+0000 7f51c5d9b700 1 -- 172.21.15.99:0/708749322 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b800b9c0 con 0x7f51c81510f0 2023-10-08T00:20:01.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.749+0000 7f51c5d9b700 1 -- 172.21.15.99:0/708749322 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b800bd60 con 0x7f51c81510f0 2023-10-08T00:20:01.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.749+0000 7f51cce58700 1 -- 172.21.15.99:0/708749322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51c81510f0 msgr2=0x7f51c806ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:01.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.749+0000 7f51cce58700 1 --2- 172.21.15.99:0/708749322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51c81510f0 0x7f51c806ecc0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f51c80c8050 tx=0x7f51b8009250 comp rx=0 tx=0).stop 2023-10-08T00:20:01.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.749+0000 7f51cce58700 1 -- 172.21.15.99:0/708749322 shutdown_connections 2023-10-08T00:20:01.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.749+0000 7f51cce58700 1 --2- 172.21.15.99:0/708749322 >> v2:172.21.15.191:3300/0 conn(0x7f51c8070700 0x7f51c8070b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:01.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.749+0000 7f51cce58700 1 --2- 172.21.15.99:0/708749322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51c81510f0 0x7f51c806ecc0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:01.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.749+0000 7f51cce58700 1 --2- 172.21.15.99:0/708749322 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f51c80c9af0 0x7f51c815dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:01.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.749+0000 7f51cce58700 1 -- 172.21.15.99:0/708749322 >> 172.21.15.99:0/708749322 conn(0x7f51c80ca4f0 msgr2=0x7f51c80ca8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:20:01.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.749+0000 7f51cce58700 1 -- 172.21.15.99:0/708749322 shutdown_connections 2023-10-08T00:20:01.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.749+0000 7f51cce58700 1 -- 172.21.15.99:0/708749322 wait complete. 2023-10-08T00:20:01.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.751+0000 7f51cce58700 1 Processor -- start 2023-10-08T00:20:01.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.751+0000 7f51cce58700 1 -- start start 2023-10-08T00:20:01.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.751+0000 7f51cce58700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51c8070700 0x7f51c80cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:01.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.751+0000 7f51cce58700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f51c80c9af0 0x7f51c80c6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:01.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.751+0000 7f51cce58700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f51c81510f0 0x7f51c80c7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:01.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.751+0000 7f51cce58700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c80b1240 con 0x7f51c8070700 2023-10-08T00:20:01.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.751+0000 7f51cce58700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c80c7b10 con 0x7f51c80c9af0 2023-10-08T00:20:01.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.751+0000 7f51cce58700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51c80b14e0 con 0x7f51c81510f0 2023-10-08T00:20:01.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.751+0000 7f51c659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51c8070700 0x7f51c80cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:01.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.751+0000 7f51c659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51c8070700 0x7f51c80cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40928/0 (socket says 172.21.15.99:40928) 2023-10-08T00:20:01.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.751+0000 7f51c659c700 1 -- 172.21.15.99:0/2206098671 learned_addr learned my addr 172.21.15.99:0/2206098671 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:20:01.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.752+0000 7f51bdd9b700 1 --2- 172.21.15.99:0/2206098671 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f51c80c9af0 0x7f51c80c6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:01.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.752+0000 7f51c659c700 1 -- 172.21.15.99:0/2206098671 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f51c81510f0 msgr2=0x7f51c80c7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:01.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.752+0000 7f51c6d9d700 1 --2- 172.21.15.99:0/2206098671 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f51c81510f0 0x7f51c80c7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:01.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.752+0000 7f51c659c700 1 --2- 172.21.15.99:0/2206098671 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f51c81510f0 0x7f51c80c7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:01.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.752+0000 7f51c659c700 1 -- 172.21.15.99:0/2206098671 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f51c80c9af0 msgr2=0x7f51c80c6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:01.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.752+0000 7f51c659c700 1 --2- 172.21.15.99:0/2206098671 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f51c80c9af0 0x7f51c80c6d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:01.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.752+0000 7f51c659c700 1 -- 172.21.15.99:0/2206098671 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b80094f0 con 0x7f51c8070700 2023-10-08T00:20:01.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.752+0000 7f51c659c700 1 --2- 172.21.15.99:0/2206098671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51c8070700 0x7f51c80cceb0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f51b8009290 tx=0x7f51b8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:01.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.752+0000 7f51beffd700 1 -- 172.21.15.99:0/2206098671 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b8026480 con 0x7f51c8070700 2023-10-08T00:20:01.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.752+0000 7f51beffd700 1 -- 172.21.15.99:0/2206098671 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b801f070 con 0x7f51c8070700 2023-10-08T00:20:01.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.752+0000 7f51cce58700 1 -- 172.21.15.99:0/2206098671 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51c80b17c0 con 0x7f51c8070700 2023-10-08T00:20:01.757 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.752+0000 7f51beffd700 1 -- 172.21.15.99:0/2206098671 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b8026830 con 0x7f51c8070700 2023-10-08T00:20:01.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.752+0000 7f51cce58700 1 -- 172.21.15.99:0/2206098671 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51c80c77d0 con 0x7f51c8070700 2023-10-08T00:20:01.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.753+0000 7f51cce58700 1 -- 172.21.15.99:0/2206098671 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51c8070e80 con 0x7f51c8070700 2023-10-08T00:20:01.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.753+0000 7f51beffd700 1 -- 172.21.15.99:0/2206098671 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f51b8026ba0 con 0x7f51c8070700 2023-10-08T00:20:01.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.754+0000 7f51beffd700 1 --2- 172.21.15.99:0/2206098671 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f51a8066f30 0x7f51a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:01.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.754+0000 7f51beffd700 1 -- 172.21.15.99:0/2206098671 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f51b8021070 con 0x7f51c8070700 2023-10-08T00:20:01.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.754+0000 7f51bdd9b700 1 --2- 172.21.15.99:0/2206098671 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f51a8066f30 0x7f51a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:01.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.755+0000 7f51bdd9b700 1 --2- 172.21.15.99:0/2206098671 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f51a8066f30 0x7f51a80693e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f51c80b35c0 tx=0x7f51b0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:01.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.758+0000 7f51beffd700 1 -- 172.21.15.99:0/2206098671 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f51b80536c0 con 0x7f51c8070700 2023-10-08T00:20:01.942 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.941+0000 7f51cce58700 1 -- 172.21.15.99:0/2206098671 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f51c804c3c0 con 0x7f51c8070700 2023-10-08T00:20:01.943 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.942+0000 7f51beffd700 1 -- 172.21.15.99:0/2206098671 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v164) v1 ==== 74+0+12145 (secure 0 0 0) 0x7f51b8056ce0 con 0x7f51c8070700 2023-10-08T00:20:01.943 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:20:01.943 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":164,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:20:01.211084+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:20:01.945 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.944+0000 7f51cce58700 1 -- 172.21.15.99:0/2206098671 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f51a8066f30 msgr2=0x7f51a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:01.945 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.944+0000 7f51cce58700 1 --2- 172.21.15.99:0/2206098671 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f51a8066f30 0x7f51a80693e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f51c80b35c0 tx=0x7f51b0008040 comp rx=0 tx=0).stop 2023-10-08T00:20:01.945 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.944+0000 7f51cce58700 1 -- 172.21.15.99:0/2206098671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51c8070700 msgr2=0x7f51c80cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:01.946 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.944+0000 7f51cce58700 1 --2- 172.21.15.99:0/2206098671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51c8070700 0x7f51c80cceb0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f51b8009290 tx=0x7f51b8017000 comp rx=0 tx=0).stop 2023-10-08T00:20:01.946 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.944+0000 7f51cce58700 1 -- 172.21.15.99:0/2206098671 shutdown_connections 2023-10-08T00:20:01.946 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.944+0000 7f51cce58700 1 --2- 172.21.15.99:0/2206098671 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f51a8066f30 0x7f51a80693e0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:01.946 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.944+0000 7f51cce58700 1 --2- 172.21.15.99:0/2206098671 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f51c81510f0 0x7f51c80c7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:01.946 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.944+0000 7f51cce58700 1 --2- 172.21.15.99:0/2206098671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51c8070700 0x7f51c80cceb0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:01.946 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.944+0000 7f51cce58700 1 --2- 172.21.15.99:0/2206098671 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f51c80c9af0 0x7f51c80c6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:01.946 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.944+0000 7f51cce58700 1 -- 172.21.15.99:0/2206098671 >> 172.21.15.99:0/2206098671 conn(0x7f51c80ca4f0 msgr2=0x7f51c80afb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:20:01.946 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.944+0000 7f51cce58700 1 -- 172.21.15.99:0/2206098671 shutdown_connections 2023-10-08T00:20:01.946 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:01.944+0000 7f51cce58700 1 -- 172.21.15.99:0/2206098671 wait complete. 2023-10-08T00:20:01.957 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:10:42.063344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-08T00:20:01.957 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T00:20:02.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.073+0000 7f296ba24700 1 Processor -- start 2023-10-08T00:20:02.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.073+0000 7f296ba24700 1 -- start start 2023-10-08T00:20:02.081 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.074+0000 7f296ba24700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f296415a730 0x7f29640caf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:02.081 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.074+0000 7f296ba24700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29640c3430 con 0x7f29640abce0 2023-10-08T00:20:02.081 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.074+0000 7f296ba24700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f296415abc0 con 0x7f29640ab910 2023-10-08T00:20:02.081 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.074+0000 7f296ba24700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29640c9100 con 0x7f296415a730 2023-10-08T00:20:02.081 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.074+0000 7f2969fc1700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f29640abce0 0x7f29640cb970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:54592/0 (socket says 172.21.15.99:54592) 2023-10-08T00:20:02.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.074+0000 7f2969fc1700 1 -- 172.21.15.99:0/1069009850 learned_addr learned my addr 172.21.15.99:0/1069009850 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:20:02.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.074+0000 7f295bfff700 1 -- 172.21.15.99:0/1069009850 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 260026888 0 0) 0x7f29640c3430 con 0x7f29640abce0 2023-10-08T00:20:02.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.075+0000 7f295bfff700 1 -- 172.21.15.99:0/1069009850 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2950003570 con 0x7f29640abce0 2023-10-08T00:20:02.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.075+0000 7f2968fbf700 1 --2- 172.21.15.99:0/1069009850 >> v2:172.21.15.191:3300/0 conn(0x7f296415a730 0x7f29640caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:02.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.075+0000 7f295bfff700 1 -- 172.21.15.99:0/1069009850 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1341502044 0 0) 0x7f2950003570 con 0x7f29640abce0 2023-10-08T00:20:02.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.075+0000 7f295bfff700 1 -- 172.21.15.99:0/1069009850 >> v2:172.21.15.191:3300/0 conn(0x7f296415a730 msgr2=0x7f29640caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:02.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.075+0000 7f295bfff700 1 --2- 172.21.15.99:0/1069009850 >> v2:172.21.15.191:3300/0 conn(0x7f296415a730 0x7f29640caf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:02.083 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.075+0000 7f295bfff700 1 -- 172.21.15.99:0/1069009850 >> v1:172.21.15.191:6789/0 conn(0x7f29640ab910 legacy=0x7f29640c2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:02.083 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.075+0000 7f295bfff700 1 -- 172.21.15.99:0/1069009850 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f296415aea0 con 0x7f29640abce0 2023-10-08T00:20:02.083 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.075+0000 7f295bfff700 1 -- 172.21.15.99:0/1069009850 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f295c001470 con 0x7f29640abce0 2023-10-08T00:20:02.083 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.075+0000 7f295bfff700 1 -- 172.21.15.99:0/1069009850 >> v1:172.21.15.99:6789/0 conn(0x7f29640abce0 legacy=0x7f29640cb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:02.083 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.075+0000 7f295bfff700 1 --2- 172.21.15.99:0/1069009850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2950003a60 0x7f2950003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:02.083 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.075+0000 7f295bfff700 1 --2- 172.21.15.99:0/1069009850 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2950004810 0x7f2950004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:02.083 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.075+0000 7f295bfff700 1 --2- 172.21.15.99:0/1069009850 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29500051c0 0x7f295000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:02.083 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.075+0000 7f295bfff700 1 -- 172.21.15.99:0/1069009850 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29640c9100 con 0x7f2950003a60 2023-10-08T00:20:02.084 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.075+0000 7f295bfff700 1 -- 172.21.15.99:0/1069009850 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295000c0d0 con 0x7f29500051c0 2023-10-08T00:20:02.084 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.075+0000 7f295bfff700 1 -- 172.21.15.99:0/1069009850 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f295000c290 con 0x7f2950004810 2023-10-08T00:20:02.084 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.075+0000 7f29697c0700 1 --2- 172.21.15.99:0/1069009850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2950003a60 0x7f2950003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:02.084 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.075+0000 7f2968fbf700 1 --2- 172.21.15.99:0/1069009850 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2950004810 0x7f2950004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:02.084 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.075+0000 7f2969fc1700 1 --2- 172.21.15.99:0/1069009850 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29500051c0 0x7f295000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:02.084 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.075+0000 7f29697c0700 1 -- 172.21.15.99:0/1069009850 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2950004810 msgr2=0x7f2950004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:02.084 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.075+0000 7f29697c0700 1 --2- 172.21.15.99:0/1069009850 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2950004810 0x7f2950004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:02.084 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.075+0000 7f29697c0700 1 -- 172.21.15.99:0/1069009850 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29500051c0 msgr2=0x7f295000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:02.084 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.075+0000 7f29697c0700 1 --2- 172.21.15.99:0/1069009850 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29500051c0 0x7f295000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:02.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.075+0000 7f29697c0700 1 -- 172.21.15.99:0/1069009850 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2950003570 con 0x7f2950003a60 2023-10-08T00:20:02.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.075+0000 7f2969fc1700 1 --2- 172.21.15.99:0/1069009850 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29500051c0 0x7f295000ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:20:02.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.075+0000 7f2969fc1700 1 -- 172.21.15.99:0/1069009850 reap_dead start 2023-10-08T00:20:02.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.075+0000 7f29697c0700 1 --2- 172.21.15.99:0/1069009850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2950003a60 0x7f2950003e70 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f2960001fc0 tx=0x7f2960001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ae81bee8d648249 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:02.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.076+0000 7f295bfff700 1 -- 172.21.15.99:0/1069009850 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2960001950 con 0x7f2950003a60 2023-10-08T00:20:02.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.076+0000 7f295bfff700 1 -- 172.21.15.99:0/1069009850 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f296000c8b0 con 0x7f2950003a60 2023-10-08T00:20:02.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.076+0000 7f295bfff700 1 -- 172.21.15.99:0/1069009850 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296000cc50 con 0x7f2950003a60 2023-10-08T00:20:02.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.076+0000 7f296ba24700 1 -- 172.21.15.99:0/1069009850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2950003a60 msgr2=0x7f2950003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:02.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.076+0000 7f296ba24700 1 --2- 172.21.15.99:0/1069009850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2950003a60 0x7f2950003e70 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f2960001fc0 tx=0x7f2960001640 comp rx=0 tx=0).stop 2023-10-08T00:20:02.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.076+0000 7f296ba24700 1 -- 172.21.15.99:0/1069009850 shutdown_connections 2023-10-08T00:20:02.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.076+0000 7f296ba24700 1 --2- 172.21.15.99:0/1069009850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2950003a60 0x7f2950003e70 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:02.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.076+0000 7f296ba24700 1 -- 172.21.15.99:0/1069009850 >> 172.21.15.99:0/1069009850 conn(0x7f2964067660 msgr2=0x7f2964067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:20:02.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.076+0000 7f296ba24700 1 -- 172.21.15.99:0/1069009850 shutdown_connections 2023-10-08T00:20:02.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.076+0000 7f296ba24700 1 -- 172.21.15.99:0/1069009850 wait complete. 2023-10-08T00:20:02.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.077+0000 7f296ba24700 1 Processor -- start 2023-10-08T00:20:02.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.077+0000 7f296ba24700 1 -- start start 2023-10-08T00:20:02.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.078+0000 7f296ba24700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2950003a60 0x7f296406aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:02.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.078+0000 7f296ba24700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29641582e0 0x7f29640ad660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:02.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.078+0000 7f296ba24700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f296414a000 0x7f296414a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:02.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.078+0000 7f296ba24700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2964159640 con 0x7f2950003a60 2023-10-08T00:20:02.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.078+0000 7f296ba24700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2964158a30 con 0x7f29641582e0 2023-10-08T00:20:02.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.078+0000 7f296ba24700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2964158d10 con 0x7f296414a000 2023-10-08T00:20:02.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.078+0000 7f29697c0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2950003a60 0x7f296406aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:02.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.078+0000 7f29697c0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2950003a60 0x7f296406aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40944/0 (socket says 172.21.15.99:40944) 2023-10-08T00:20:02.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.078+0000 7f29697c0700 1 -- 172.21.15.99:0/3875852177 learned_addr learned my addr 172.21.15.99:0/3875852177 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:20:02.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.078+0000 7f2968fbf700 1 --2- 172.21.15.99:0/3875852177 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29641582e0 0x7f29640ad660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:02.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.078+0000 7f2969fc1700 1 --2- 172.21.15.99:0/3875852177 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f296414a000 0x7f296414a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:02.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.078+0000 7f29697c0700 1 -- 172.21.15.99:0/3875852177 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f296414a000 msgr2=0x7f296414a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:02.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.078+0000 7f29697c0700 1 --2- 172.21.15.99:0/3875852177 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f296414a000 0x7f296414a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:02.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.078+0000 7f29697c0700 1 -- 172.21.15.99:0/3875852177 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29641582e0 msgr2=0x7f29640ad660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:02.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.078+0000 7f29697c0700 1 --2- 172.21.15.99:0/3875852177 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29641582e0 0x7f29640ad660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:02.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.078+0000 7f29697c0700 1 -- 172.21.15.99:0/3875852177 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2964158ff0 con 0x7f2950003a60 2023-10-08T00:20:02.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.078+0000 7f2969fc1700 1 --2- 172.21.15.99:0/3875852177 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f296414a000 0x7f296414a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:20:02.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.079+0000 7f29697c0700 1 --2- 172.21.15.99:0/3875852177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2950003a60 0x7f296406aeb0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f295000c0a0 tx=0x7f2960015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:02.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.079+0000 7f29597fa700 1 -- 172.21.15.99:0/3875852177 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2960001950 con 0x7f2950003a60 2023-10-08T00:20:02.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.079+0000 7f29597fa700 1 -- 172.21.15.99:0/3875852177 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f296000c6d0 con 0x7f2950003a60 2023-10-08T00:20:02.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.079+0000 7f29597fa700 1 -- 172.21.15.99:0/3875852177 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296001e860 con 0x7f2950003a60 2023-10-08T00:20:02.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.079+0000 7f296ba24700 1 -- 172.21.15.99:0/3875852177 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29641592d0 con 0x7f2950003a60 2023-10-08T00:20:02.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.079+0000 7f296ba24700 1 -- 172.21.15.99:0/3875852177 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f296414a970 con 0x7f2950003a60 2023-10-08T00:20:02.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.080+0000 7f296ba24700 1 -- 172.21.15.99:0/3875852177 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f296404c3c0 con 0x7f2950003a60 2023-10-08T00:20:02.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.080+0000 7f29597fa700 1 -- 172.21.15.99:0/3875852177 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2960019030 con 0x7f2950003a60 2023-10-08T00:20:02.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.081+0000 7f29597fa700 1 --2- 172.21.15.99:0/3875852177 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2948066f30 0x7f29480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:02.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.081+0000 7f29597fa700 1 -- 172.21.15.99:0/3875852177 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f2960085f70 con 0x7f2950003a60 2023-10-08T00:20:02.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.081+0000 7f2968fbf700 1 --2- 172.21.15.99:0/3875852177 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2948066f30 0x7f29480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:02.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.081+0000 7f2968fbf700 1 --2- 172.21.15.99:0/3875852177 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2948066f30 0x7f29480693e0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f2954009e00 tx=0x7f2954008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:02.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.085+0000 7f29597fa700 1 -- 172.21.15.99:0/3875852177 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f29600b75a0 con 0x7f2950003a60 2023-10-08T00:20:02.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.266+0000 7f296ba24700 1 -- 172.21.15.99:0/3875852177 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f296415b180 con 0x7f2950003a60 2023-10-08T00:20:02.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.266+0000 7f29597fa700 1 -- 172.21.15.99:0/3875852177 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v164) v1 ==== 93+0+10 (secure 0 0 0) 0x7f296002b020 con 0x7f2950003a60 2023-10-08T00:20:02.267 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2023-10-08T00:20:02.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.268+0000 7f296ba24700 1 -- 172.21.15.99:0/3875852177 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2948066f30 msgr2=0x7f29480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:02.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.268+0000 7f296ba24700 1 --2- 172.21.15.99:0/3875852177 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2948066f30 0x7f29480693e0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f2954009e00 tx=0x7f2954008040 comp rx=0 tx=0).stop 2023-10-08T00:20:02.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.268+0000 7f296ba24700 1 -- 172.21.15.99:0/3875852177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2950003a60 msgr2=0x7f296406aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:02.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.268+0000 7f296ba24700 1 --2- 172.21.15.99:0/3875852177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2950003a60 0x7f296406aeb0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f295000c0a0 tx=0x7f2960015000 comp rx=0 tx=0).stop 2023-10-08T00:20:02.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.268+0000 7f296ba24700 1 -- 172.21.15.99:0/3875852177 shutdown_connections 2023-10-08T00:20:02.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.268+0000 7f296ba24700 1 --2- 172.21.15.99:0/3875852177 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2948066f30 0x7f29480693e0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:02.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.268+0000 7f296ba24700 1 --2- 172.21.15.99:0/3875852177 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f296414a000 0x7f296414a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:02.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.268+0000 7f296ba24700 1 --2- 172.21.15.99:0/3875852177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2950003a60 0x7f296406aeb0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:02.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.268+0000 7f296ba24700 1 --2- 172.21.15.99:0/3875852177 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29641582e0 0x7f29640ad660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:02.269 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.268+0000 7f296ba24700 1 -- 172.21.15.99:0/3875852177 >> 172.21.15.99:0/3875852177 conn(0x7f2964067660 msgr2=0x7f2964154470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:20:02.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.268+0000 7f296ba24700 1 -- 172.21.15.99:0/3875852177 shutdown_connections 2023-10-08T00:20:02.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.268+0000 7f296ba24700 1 -- 172.21.15.99:0/3875852177 wait complete. 2023-10-08T00:20:02.280 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T00:20:02.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.394+0000 7fef12221700 1 Processor -- start 2023-10-08T00:20:02.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.394+0000 7fef12221700 1 -- start start 2023-10-08T00:20:02.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.395+0000 7fef12221700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fef0c157870 0x7fef0c15b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:02.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.395+0000 7fef12221700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef0c15a620 0x7fef0c15d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:02.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.395+0000 7fef12221700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef0c15cbb0 con 0x7fef0c15d580 2023-10-08T00:20:02.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.395+0000 7fef12221700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef0c15bfa0 con 0x7fef0c157870 2023-10-08T00:20:02.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.395+0000 7fef12221700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef0c15c280 con 0x7fef0c15a620 2023-10-08T00:20:02.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.395+0000 7fef0bfff700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fef0c15d580 0x7fef0c158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:54598/0 (socket says 172.21.15.99:54598) 2023-10-08T00:20:02.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.395+0000 7fef0bfff700 1 -- 172.21.15.99:0/571679318 learned_addr learned my addr 172.21.15.99:0/571679318 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:20:02.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.395+0000 7fef0b7fe700 1 --2- 172.21.15.99:0/571679318 >> v2:172.21.15.191:3300/0 conn(0x7fef0c157870 0x7fef0c15b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:02.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.395+0000 7fef0affd700 1 --2- 172.21.15.99:0/571679318 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef0c15a620 0x7fef0c15d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:02.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.395+0000 7fef0a7fc700 1 -- 172.21.15.99:0/571679318 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1520656698 0 0) 0x7fef0c15cbb0 con 0x7fef0c15d580 2023-10-08T00:20:02.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.396+0000 7fef0a7fc700 1 -- 172.21.15.99:0/571679318 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feef4003570 con 0x7fef0c15d580 2023-10-08T00:20:02.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.396+0000 7fef0b7fe700 1 -- 172.21.15.99:0/571679318 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef0c15a620 msgr2=0x7fef0c15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:02.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.396+0000 7fef0b7fe700 1 --2- 172.21.15.99:0/571679318 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef0c15a620 0x7fef0c15d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:02.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.396+0000 7fef0b7fe700 1 -- 172.21.15.99:0/571679318 >> v1:172.21.15.99:6789/0 conn(0x7fef0c15d580 legacy=0x7fef0c158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:02.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.396+0000 7fef0b7fe700 1 -- 172.21.15.99:0/571679318 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef0c15c560 con 0x7fef0c157870 2023-10-08T00:20:02.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.396+0000 7fef0affd700 1 --2- 172.21.15.99:0/571679318 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef0c15a620 0x7fef0c15d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:20:02.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.396+0000 7fef0b7fe700 1 --2- 172.21.15.99:0/571679318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef0c157870 0x7fef0c15b6a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7feefc009e60 tx=0x7feefc009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55fb7da88c00eb00 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:02.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.396+0000 7fef0a7fc700 1 -- 172.21.15.99:0/571679318 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feefc00e070 con 0x7fef0c157870 2023-10-08T00:20:02.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.396+0000 7fef0a7fc700 1 -- 172.21.15.99:0/571679318 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feefc00bab0 con 0x7fef0c157870 2023-10-08T00:20:02.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.396+0000 7fef0a7fc700 1 -- 172.21.15.99:0/571679318 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feefc00be70 con 0x7fef0c157870 2023-10-08T00:20:02.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.397+0000 7fef12221700 1 -- 172.21.15.99:0/571679318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef0c157870 msgr2=0x7fef0c15b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:02.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.397+0000 7fef12221700 1 --2- 172.21.15.99:0/571679318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef0c157870 0x7fef0c15b6a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7feefc009e60 tx=0x7feefc009780 comp rx=0 tx=0).stop 2023-10-08T00:20:02.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.397+0000 7fef12221700 1 -- 172.21.15.99:0/571679318 shutdown_connections 2023-10-08T00:20:02.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.397+0000 7fef12221700 1 --2- 172.21.15.99:0/571679318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef0c157870 0x7fef0c15b6a0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:02.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.397+0000 7fef12221700 1 --2- 172.21.15.99:0/571679318 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef0c15a620 0x7fef0c15d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:02.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.397+0000 7fef12221700 1 -- 172.21.15.99:0/571679318 >> 172.21.15.99:0/571679318 conn(0x7fef0c068ce0 msgr2=0x7fef0c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:20:02.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.397+0000 7fef12221700 1 -- 172.21.15.99:0/571679318 shutdown_connections 2023-10-08T00:20:02.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.397+0000 7fef12221700 1 -- 172.21.15.99:0/571679318 wait complete. 2023-10-08T00:20:02.402 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.398+0000 7fef12221700 1 Processor -- start 2023-10-08T00:20:02.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.398+0000 7fef12221700 1 -- start start 2023-10-08T00:20:02.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.399+0000 7fef12221700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef0c157870 0x7fef0c0b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:02.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.399+0000 7fef12221700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef0c15a620 0x7fef0c0cfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:02.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.399+0000 7fef12221700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef0c15d580 0x7fef0c14e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:02.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.399+0000 7fef12221700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0c16b6c0 con 0x7fef0c15d580 2023-10-08T00:20:02.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.399+0000 7fef12221700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0c0d0200 con 0x7fef0c157870 2023-10-08T00:20:02.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.399+0000 7fef12221700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef0c16aab0 con 0x7fef0c15a620 2023-10-08T00:20:02.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.399+0000 7fef0bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef0c15d580 0x7fef0c14e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:02.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.399+0000 7fef0bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef0c15d580 0x7fef0c14e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40952/0 (socket says 172.21.15.99:40952) 2023-10-08T00:20:02.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.399+0000 7fef0bfff700 1 -- 172.21.15.99:0/1007554152 learned_addr learned my addr 172.21.15.99:0/1007554152 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:20:02.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.399+0000 7fef0affd700 1 --2- 172.21.15.99:0/1007554152 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef0c15a620 0x7fef0c0cfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:02.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.399+0000 7fef0b7fe700 1 --2- 172.21.15.99:0/1007554152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef0c157870 0x7fef0c0b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:02.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.399+0000 7fef0bfff700 1 -- 172.21.15.99:0/1007554152 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef0c15a620 msgr2=0x7fef0c0cfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:02.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.399+0000 7fef0bfff700 1 --2- 172.21.15.99:0/1007554152 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef0c15a620 0x7fef0c0cfcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:02.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.399+0000 7fef0bfff700 1 -- 172.21.15.99:0/1007554152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef0c157870 msgr2=0x7fef0c0b31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:02.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.399+0000 7fef0bfff700 1 --2- 172.21.15.99:0/1007554152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef0c157870 0x7fef0c0b31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:02.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.399+0000 7fef0bfff700 1 -- 172.21.15.99:0/1007554152 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef00009420 con 0x7fef0c15d580 2023-10-08T00:20:02.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.399+0000 7fef0bfff700 1 --2- 172.21.15.99:0/1007554152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef0c15d580 0x7fef0c14e7f0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fef0400c790 tx=0x7fef04007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:02.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.399+0000 7feef37fe700 1 -- 172.21.15.99:0/1007554152 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef04005c00 con 0x7fef0c15d580 2023-10-08T00:20:02.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.399+0000 7feef37fe700 1 -- 172.21.15.99:0/1007554152 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef04005d60 con 0x7fef0c15d580 2023-10-08T00:20:02.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.399+0000 7feef37fe700 1 -- 172.21.15.99:0/1007554152 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef040072e0 con 0x7fef0c15d580 2023-10-08T00:20:02.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.399+0000 7fef12221700 1 -- 172.21.15.99:0/1007554152 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feefc009a20 con 0x7fef0c15d580 2023-10-08T00:20:02.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.399+0000 7fef12221700 1 -- 172.21.15.99:0/1007554152 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef0c16b070 con 0x7fef0c15d580 2023-10-08T00:20:02.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.400+0000 7fef12221700 1 -- 172.21.15.99:0/1007554152 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef0c14cf70 con 0x7fef0c15d580 2023-10-08T00:20:02.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.401+0000 7feef37fe700 1 -- 172.21.15.99:0/1007554152 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fef0401b3b0 con 0x7fef0c15d580 2023-10-08T00:20:02.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.401+0000 7feef37fe700 1 --2- 172.21.15.99:0/1007554152 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7feef8066f30 0x7feef80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:02.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.401+0000 7feef37fe700 1 -- 172.21.15.99:0/1007554152 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fef040835e0 con 0x7fef0c15d580 2023-10-08T00:20:02.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.401+0000 7fef0b7fe700 1 --2- 172.21.15.99:0/1007554152 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7feef8066f30 0x7feef80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:02.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.402+0000 7fef0b7fe700 1 --2- 172.21.15.99:0/1007554152 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7feef8066f30 0x7feef80693e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7feefc0097c0 tx=0x7feefc019000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:02.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.405+0000 7feef37fe700 1 -- 172.21.15.99:0/1007554152 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fef0c14cf70 con 0x7fef0c15d580 2023-10-08T00:20:02.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.590+0000 7fef12221700 1 -- 172.21.15.99:0/1007554152 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fef0c14cf70 con 0x7fef0c15d580 2023-10-08T00:20:02.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.590+0000 7feef37fe700 1 -- 172.21.15.99:0/1007554152 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v164) v1 ==== 84+0+65 (secure 0 0 0) 0x7fef0c14cf70 con 0x7fef0c15d580 2023-10-08T00:20:02.591 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:20:02.591 INFO:teuthology.orchestra.run.smithi099.stdout:[ 2023-10-08T00:20:02.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:20:02.592 INFO:teuthology.orchestra.run.smithi099.stdout: "poolnum": 1, 2023-10-08T00:20:02.592 INFO:teuthology.orchestra.run.smithi099.stdout: "poolname": ".mgr" 2023-10-08T00:20:02.592 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:20:02.592 INFO:teuthology.orchestra.run.smithi099.stdout:] 2023-10-08T00:20:02.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.592+0000 7fef12221700 1 -- 172.21.15.99:0/1007554152 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7feef8066f30 msgr2=0x7feef80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:02.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.592+0000 7fef12221700 1 --2- 172.21.15.99:0/1007554152 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7feef8066f30 0x7feef80693e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7feefc0097c0 tx=0x7feefc019000 comp rx=0 tx=0).stop 2023-10-08T00:20:02.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.592+0000 7fef12221700 1 -- 172.21.15.99:0/1007554152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef0c15d580 msgr2=0x7fef0c14e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:02.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.592+0000 7fef12221700 1 --2- 172.21.15.99:0/1007554152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef0c15d580 0x7fef0c14e7f0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fef0400c790 tx=0x7fef04007d90 comp rx=0 tx=0).stop 2023-10-08T00:20:02.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.592+0000 7fef12221700 1 -- 172.21.15.99:0/1007554152 shutdown_connections 2023-10-08T00:20:02.593 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.592+0000 7fef12221700 1 --2- 172.21.15.99:0/1007554152 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7feef8066f30 0x7feef80693e0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:02.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.592+0000 7fef12221700 1 --2- 172.21.15.99:0/1007554152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef0c157870 0x7fef0c0b31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:02.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.592+0000 7fef12221700 1 --2- 172.21.15.99:0/1007554152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef0c15d580 0x7fef0c14e7f0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:02.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.592+0000 7fef12221700 1 --2- 172.21.15.99:0/1007554152 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef0c15a620 0x7fef0c0cfcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:02.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.592+0000 7fef12221700 1 -- 172.21.15.99:0/1007554152 >> 172.21.15.99:0/1007554152 conn(0x7fef0c068ce0 msgr2=0x7fef0c14f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:20:02.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.592+0000 7fef12221700 1 -- 172.21.15.99:0/1007554152 shutdown_connections 2023-10-08T00:20:02.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.592+0000 7fef12221700 1 -- 172.21.15.99:0/1007554152 wait complete. 2023-10-08T00:20:02.605 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T00:20:02.605 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T00:20:02.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.722+0000 7f4a2b439700 1 Processor -- start 2023-10-08T00:20:02.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.722+0000 7f4a2b439700 1 -- start start 2023-10-08T00:20:02.726 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.723+0000 7f4a2b439700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4a2415d240 0x7f4a240b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:02.726 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.723+0000 7f4a2b439700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a240cc940 0x7f4a240ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:02.726 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.723+0000 7f4a2b439700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a240b3f40 con 0x7f4a240c59e0 2023-10-08T00:20:02.726 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.723+0000 7f4a2b439700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a240cbb80 con 0x7f4a2415d240 2023-10-08T00:20:02.726 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.723+0000 7f4a2b439700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a240cbe60 con 0x7f4a240cc940 2023-10-08T00:20:02.726 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.723+0000 7f4a2ac38700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f4a240c59e0 0x7f4a240c5e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:54614/0 (socket says 172.21.15.99:54614) 2023-10-08T00:20:02.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.723+0000 7f4a2ac38700 1 -- 172.21.15.99:0/2304450690 learned_addr learned my addr 172.21.15.99:0/2304450690 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:20:02.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.723+0000 7f4a2a437700 1 --2- 172.21.15.99:0/2304450690 >> v2:172.21.15.191:3300/0 conn(0x7f4a2415d240 0x7f4a240b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:02.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.723+0000 7f4a29c36700 1 --2- 172.21.15.99:0/2304450690 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a240cc940 0x7f4a240ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:02.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.723+0000 7f4a29435700 1 -- 172.21.15.99:0/2304450690 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 573926883 0 0) 0x7f4a240b3f40 con 0x7f4a240c59e0 2023-10-08T00:20:02.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.723+0000 7f4a29435700 1 -- 172.21.15.99:0/2304450690 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a10003570 con 0x7f4a240c59e0 2023-10-08T00:20:02.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.723+0000 7f4a29435700 1 -- 172.21.15.99:0/2304450690 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2152770953 0 0) 0x7f4a10003570 con 0x7f4a240c59e0 2023-10-08T00:20:02.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.724+0000 7f4a29435700 1 -- 172.21.15.99:0/2304450690 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a240cc940 msgr2=0x7f4a240ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:02.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.724+0000 7f4a29435700 1 --2- 172.21.15.99:0/2304450690 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a240cc940 0x7f4a240ccdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:02.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.724+0000 7f4a29435700 1 -- 172.21.15.99:0/2304450690 >> v2:172.21.15.191:3300/0 conn(0x7f4a2415d240 msgr2=0x7f4a240b3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:02.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.724+0000 7f4a29435700 1 --2- 172.21.15.99:0/2304450690 >> v2:172.21.15.191:3300/0 conn(0x7f4a2415d240 0x7f4a240b3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:02.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.724+0000 7f4a29435700 1 -- 172.21.15.99:0/2304450690 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a240cc140 con 0x7f4a240c59e0 2023-10-08T00:20:02.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.724+0000 7f4a29435700 1 -- 172.21.15.99:0/2304450690 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f4a1c0017f0 con 0x7f4a240c59e0 2023-10-08T00:20:02.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.724+0000 7f4a29435700 1 -- 172.21.15.99:0/2304450690 >> v1:172.21.15.99:6789/0 conn(0x7f4a240c59e0 legacy=0x7f4a240c5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:02.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.724+0000 7f4a29435700 1 --2- 172.21.15.99:0/2304450690 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a10003a60 0x7f4a10003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:02.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.724+0000 7f4a29435700 1 --2- 172.21.15.99:0/2304450690 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a240cc940 0x7f4a10004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:02.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.724+0000 7f4a29435700 1 --2- 172.21.15.99:0/2304450690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a10004d50 0x7f4a100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:02.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.724+0000 7f4a29435700 1 -- 172.21.15.99:0/2304450690 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a240cbb80 con 0x7f4a10004d50 2023-10-08T00:20:02.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.724+0000 7f4a29435700 1 -- 172.21.15.99:0/2304450690 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a240cbe60 con 0x7f4a10003a60 2023-10-08T00:20:02.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.724+0000 7f4a29435700 1 -- 172.21.15.99:0/2304450690 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1000ba60 con 0x7f4a240cc940 2023-10-08T00:20:02.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.724+0000 7f4a2ac38700 1 --2- 172.21.15.99:0/2304450690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a10004d50 0x7f4a100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:02.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.724+0000 7f4a2a437700 1 --2- 172.21.15.99:0/2304450690 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a10003a60 0x7f4a10003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:02.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.724+0000 7f4a29c36700 1 --2- 172.21.15.99:0/2304450690 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a240cc940 0x7f4a10004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:02.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.724+0000 7f4a2ac38700 1 -- 172.21.15.99:0/2304450690 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a240cc940 msgr2=0x7f4a10004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:02.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.724+0000 7f4a2ac38700 1 --2- 172.21.15.99:0/2304450690 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a240cc940 0x7f4a10004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:02.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.724+0000 7f4a2ac38700 1 -- 172.21.15.99:0/2304450690 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a10003a60 msgr2=0x7f4a10003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:02.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.724+0000 7f4a2ac38700 1 --2- 172.21.15.99:0/2304450690 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a10003a60 0x7f4a10003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:02.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.724+0000 7f4a2ac38700 1 -- 172.21.15.99:0/2304450690 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a10003570 con 0x7f4a10004d50 2023-10-08T00:20:02.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.724+0000 7f4a2ac38700 1 --2- 172.21.15.99:0/2304450690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a10004d50 0x7f4a100051c0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c00a970 tx=0x7f4a1c002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae6f8cff0167fed4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:02.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.724+0000 7f4a29435700 1 -- 172.21.15.99:0/2304450690 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c012440 con 0x7f4a10004d50 2023-10-08T00:20:02.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.724+0000 7f4a29435700 1 -- 172.21.15.99:0/2304450690 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a1c0125c0 con 0x7f4a10004d50 2023-10-08T00:20:02.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.724+0000 7f4a29435700 1 -- 172.21.15.99:0/2304450690 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c012980 con 0x7f4a10004d50 2023-10-08T00:20:02.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.725+0000 7f4a2b439700 1 -- 172.21.15.99:0/2304450690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a10004d50 msgr2=0x7f4a100051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:02.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.725+0000 7f4a2b439700 1 --2- 172.21.15.99:0/2304450690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a10004d50 0x7f4a100051c0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c00a970 tx=0x7f4a1c002210 comp rx=0 tx=0).stop 2023-10-08T00:20:02.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.725+0000 7f4a2ac38700 1 -- 172.21.15.99:0/2304450690 reap_dead start 2023-10-08T00:20:02.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.725+0000 7f4a2b439700 1 -- 172.21.15.99:0/2304450690 shutdown_connections 2023-10-08T00:20:02.730 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.725+0000 7f4a2b439700 1 -- 172.21.15.99:0/2304450690 >> 172.21.15.99:0/2304450690 conn(0x7f4a24067e40 msgr2=0x7f4a240d0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:20:02.730 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.725+0000 7f4a2b439700 1 -- 172.21.15.99:0/2304450690 shutdown_connections 2023-10-08T00:20:02.730 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.725+0000 7f4a2b439700 1 -- 172.21.15.99:0/2304450690 wait complete. 2023-10-08T00:20:02.732 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.726+0000 7f4a2b439700 1 Processor -- start 2023-10-08T00:20:02.732 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.726+0000 7f4a2b439700 1 -- start start 2023-10-08T00:20:02.732 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.727+0000 7f4a2b439700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a10004d50 0x7f4a24158c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:02.732 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.727+0000 7f4a2b439700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a2406a2a0 0x7f4a2409c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:02.732 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.727+0000 7f4a2b439700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a2414b910 0x7f4a2409c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:02.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.727+0000 7f4a2b439700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2409e320 con 0x7f4a10004d50 2023-10-08T00:20:02.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.727+0000 7f4a2b439700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2409bda0 con 0x7f4a2406a2a0 2023-10-08T00:20:02.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.727+0000 7f4a2b439700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2409b910 con 0x7f4a2414b910 2023-10-08T00:20:02.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.727+0000 7f4a2a437700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a10004d50 0x7f4a24158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:02.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.727+0000 7f4a2a437700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a10004d50 0x7f4a24158c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40970/0 (socket says 172.21.15.99:40970) 2023-10-08T00:20:02.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.727+0000 7f4a2a437700 1 -- 172.21.15.99:0/1644836767 learned_addr learned my addr 172.21.15.99:0/1644836767 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:20:02.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.727+0000 7f4a2a437700 1 -- 172.21.15.99:0/1644836767 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a2414b910 msgr2=0x7f4a2409c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:02.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.727+0000 7f4a2a437700 1 --2- 172.21.15.99:0/1644836767 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a2414b910 0x7f4a2409c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:02.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.727+0000 7f4a2a437700 1 -- 172.21.15.99:0/1644836767 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a2406a2a0 msgr2=0x7f4a2409c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:02.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.727+0000 7f4a2a437700 1 --2- 172.21.15.99:0/1644836767 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a2406a2a0 0x7f4a2409c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:02.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.727+0000 7f4a2a437700 1 -- 172.21.15.99:0/1644836767 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a240b4680 con 0x7f4a10004d50 2023-10-08T00:20:02.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.727+0000 7f4a2a437700 1 --2- 172.21.15.99:0/1644836767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a10004d50 0x7f4a24158c70 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f4a20011a40 tx=0x7f4a20009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:02.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.727+0000 7f4a1a7fc700 1 -- 172.21.15.99:0/1644836767 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a20005c10 con 0x7f4a10004d50 2023-10-08T00:20:02.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.727+0000 7f4a1a7fc700 1 -- 172.21.15.99:0/1644836767 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a20005d70 con 0x7f4a10004d50 2023-10-08T00:20:02.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.727+0000 7f4a2b439700 1 -- 172.21.15.99:0/1644836767 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a2409e810 con 0x7f4a10004d50 2023-10-08T00:20:02.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.727+0000 7f4a1a7fc700 1 -- 172.21.15.99:0/1644836767 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2000f900 con 0x7f4a10004d50 2023-10-08T00:20:02.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.727+0000 7f4a2b439700 1 -- 172.21.15.99:0/1644836767 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a2409b2c0 con 0x7f4a10004d50 2023-10-08T00:20:02.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.728+0000 7f4a2b439700 1 -- 172.21.15.99:0/1644836767 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a2409bbf0 con 0x7f4a10004d50 2023-10-08T00:20:02.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.729+0000 7f4a1a7fc700 1 -- 172.21.15.99:0/1644836767 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4a2000fd30 con 0x7f4a10004d50 2023-10-08T00:20:02.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.729+0000 7f4a1a7fc700 1 --2- 172.21.15.99:0/1644836767 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4a08066f30 0x7f4a080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:02.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.729+0000 7f4a1a7fc700 1 -- 172.21.15.99:0/1644836767 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f4a20084bb0 con 0x7f4a10004d50 2023-10-08T00:20:02.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.729+0000 7f4a29c36700 1 --2- 172.21.15.99:0/1644836767 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4a08066f30 0x7f4a080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:02.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.730+0000 7f4a29c36700 1 --2- 172.21.15.99:0/1644836767 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4a08066f30 0x7f4a080693e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4a14006eb0 tx=0x7f4a140091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:02.735 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.733+0000 7f4a1a7fc700 1 -- 172.21.15.99:0/1644836767 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4a20052170 con 0x7f4a10004d50 2023-10-08T00:20:02.914 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:02.914+0000 7f4a2b439700 1 -- 172.21.15.99:0/1644836767 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f4a240d1220 con 0x7f4a10004d50 2023-10-08T00:20:03.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.229+0000 7f4a1a7fc700 1 -- 172.21.15.99:0/1644836767 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v165) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4a20055790 con 0x7f4a10004d50 2023-10-08T00:20:03.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.232+0000 7f4a2b439700 1 -- 172.21.15.99:0/1644836767 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4a08066f30 msgr2=0x7f4a080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:03.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.232+0000 7f4a2b439700 1 --2- 172.21.15.99:0/1644836767 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4a08066f30 0x7f4a080693e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4a14006eb0 tx=0x7f4a140091a0 comp rx=0 tx=0).stop 2023-10-08T00:20:03.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.232+0000 7f4a2b439700 1 -- 172.21.15.99:0/1644836767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a10004d50 msgr2=0x7f4a24158c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:03.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.232+0000 7f4a2b439700 1 --2- 172.21.15.99:0/1644836767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a10004d50 0x7f4a24158c70 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f4a20011a40 tx=0x7f4a20009120 comp rx=0 tx=0).stop 2023-10-08T00:20:03.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.232+0000 7f4a2b439700 1 -- 172.21.15.99:0/1644836767 shutdown_connections 2023-10-08T00:20:03.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.232+0000 7f4a2b439700 1 --2- 172.21.15.99:0/1644836767 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4a08066f30 0x7f4a080693e0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:03.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.232+0000 7f4a2b439700 1 --2- 172.21.15.99:0/1644836767 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a2414b910 0x7f4a2409c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:03.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.232+0000 7f4a2b439700 1 --2- 172.21.15.99:0/1644836767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a10004d50 0x7f4a24158c70 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:03.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.232+0000 7f4a2b439700 1 --2- 172.21.15.99:0/1644836767 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a2406a2a0 0x7f4a2409c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:03.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.232+0000 7f4a2b439700 1 -- 172.21.15.99:0/1644836767 >> 172.21.15.99:0/1644836767 conn(0x7f4a24067e40 msgr2=0x7f4a2415c010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:20:03.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.232+0000 7f4a2b439700 1 -- 172.21.15.99:0/1644836767 shutdown_connections 2023-10-08T00:20:03.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.232+0000 7f4a2b439700 1 -- 172.21.15.99:0/1644836767 wait complete. 2023-10-08T00:20:03.234 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' created 2023-10-08T00:20:03.246 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T00:20:03.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.366+0000 7f8bbd940700 1 Processor -- start 2023-10-08T00:20:03.367 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.366+0000 7f8bbd940700 1 -- start start 2023-10-08T00:20:03.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.367+0000 7f8bbd940700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8bb80711c0 0x7f8bb806c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:03.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.367+0000 7f8bbd940700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8bb806c8c0 0x7f8bb815a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:03.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.367+0000 7f8bbd940700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bb8070d30 con 0x7f8bb8159c30 2023-10-08T00:20:03.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.367+0000 7f8bbd940700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bb806cd40 con 0x7f8bb80711c0 2023-10-08T00:20:03.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.367+0000 7f8bbd940700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bb8070700 con 0x7f8bb806c8c0 2023-10-08T00:20:03.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.367+0000 7f8bae7fc700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8bb80711c0 0x7f8bb806c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:03.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.367+0000 7f8bb6ffd700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f8bb8159c30 0x7f8bb806ecc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56066/0 (socket says 172.21.15.99:56066) 2023-10-08T00:20:03.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.367+0000 7f8bae7fc700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8bb80711c0 0x7f8bb806c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60786/0 (socket says 172.21.15.99:60786) 2023-10-08T00:20:03.370 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.367+0000 7f8bb6ffd700 1 -- 172.21.15.99:0/1147644975 learned_addr learned my addr 172.21.15.99:0/1147644975 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:20:03.371 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.368+0000 7f8bb77fe700 1 --2- 172.21.15.99:0/1147644975 >> v2:172.21.15.191:3300/0 conn(0x7f8bb806c8c0 0x7f8bb815a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:03.371 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.368+0000 7f8bae7fc700 1 -- 172.21.15.99:0/1147644975 >> v2:172.21.15.191:3300/0 conn(0x7f8bb806c8c0 msgr2=0x7f8bb815a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:03.371 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.368+0000 7f8bae7fc700 1 --2- 172.21.15.99:0/1147644975 >> v2:172.21.15.191:3300/0 conn(0x7f8bb806c8c0 0x7f8bb815a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:03.371 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.368+0000 7f8bae7fc700 1 -- 172.21.15.99:0/1147644975 >> v1:172.21.15.99:6789/0 conn(0x7f8bb8159c30 legacy=0x7f8bb806ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:03.371 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.368+0000 7f8bae7fc700 1 -- 172.21.15.99:0/1147644975 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb806f710 con 0x7f8bb80711c0 2023-10-08T00:20:03.371 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.368+0000 7f8bb67fc700 1 -- 172.21.15.99:0/1147644975 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4168479228 0 0) 0x7f8bb8070d30 con 0x7f8bb8159c30 2023-10-08T00:20:03.371 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.368+0000 7f8bae7fc700 1 --2- 172.21.15.99:0/1147644975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bb80711c0 0x7f8bb806c380 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8070ca0 tx=0x7f8ba00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5e434241aa8ea4b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:03.371 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.368+0000 7f8bb67fc700 1 -- 172.21.15.99:0/1147644975 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba000b6b0 con 0x7f8bb80711c0 2023-10-08T00:20:03.371 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.368+0000 7f8bb67fc700 1 -- 172.21.15.99:0/1147644975 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba000b810 con 0x7f8bb80711c0 2023-10-08T00:20:03.372 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.368+0000 7f8bb67fc700 1 -- 172.21.15.99:0/1147644975 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba000bbb0 con 0x7f8bb80711c0 2023-10-08T00:20:03.372 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.368+0000 7f8bbd940700 1 -- 172.21.15.99:0/1147644975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bb80711c0 msgr2=0x7f8bb806c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:03.372 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.368+0000 7f8bbd940700 1 --2- 172.21.15.99:0/1147644975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bb80711c0 0x7f8bb806c380 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8070ca0 tx=0x7f8ba00091a0 comp rx=0 tx=0).stop 2023-10-08T00:20:03.372 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.369+0000 7f8bbd940700 1 -- 172.21.15.99:0/1147644975 shutdown_connections 2023-10-08T00:20:03.372 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.369+0000 7f8bbd940700 1 --2- 172.21.15.99:0/1147644975 >> v2:172.21.15.191:3300/0 conn(0x7f8bb806c8c0 0x7f8bb815a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:03.372 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.369+0000 7f8bbd940700 1 --2- 172.21.15.99:0/1147644975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bb80711c0 0x7f8bb806c380 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:03.372 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.369+0000 7f8bbd940700 1 -- 172.21.15.99:0/1147644975 >> 172.21.15.99:0/1147644975 conn(0x7f8bb80cb030 msgr2=0x7f8bb806bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:20:03.372 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.369+0000 7f8bbd940700 1 -- 172.21.15.99:0/1147644975 shutdown_connections 2023-10-08T00:20:03.372 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.369+0000 7f8bbd940700 1 -- 172.21.15.99:0/1147644975 wait complete. 2023-10-08T00:20:03.375 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.370+0000 7f8bbd940700 1 Processor -- start 2023-10-08T00:20:03.375 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.370+0000 7f8bbd940700 1 -- start start 2023-10-08T00:20:03.375 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.370+0000 7f8bbd940700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bb806c8c0 0x7f8bb81461e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:03.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.371+0000 7f8bbd940700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bb80711c0 0x7f8bb8146af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:03.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.371+0000 7f8bbd940700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bb8159c30 0x7f8bb80af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:03.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.371+0000 7f8bbd940700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb815af60 con 0x7f8bb80711c0 2023-10-08T00:20:03.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.371+0000 7f8bbd940700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb815a350 con 0x7f8bb8159c30 2023-10-08T00:20:03.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.371+0000 7f8bbd940700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb815a630 con 0x7f8bb806c8c0 2023-10-08T00:20:03.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.371+0000 7f8bae7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bb80711c0 0x7f8bb8146af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:03.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.371+0000 7f8bae7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bb80711c0 0x7f8bb8146af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60788/0 (socket says 172.21.15.99:60788) 2023-10-08T00:20:03.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.371+0000 7f8bae7fc700 1 -- 172.21.15.99:0/3268279724 learned_addr learned my addr 172.21.15.99:0/3268279724 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:20:03.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.371+0000 7f8bb77fe700 1 --2- 172.21.15.99:0/3268279724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bb8159c30 0x7f8bb80af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:03.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.371+0000 7f8bb6ffd700 1 --2- 172.21.15.99:0/3268279724 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bb806c8c0 0x7f8bb81461e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:03.376 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.371+0000 7f8bae7fc700 1 -- 172.21.15.99:0/3268279724 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bb806c8c0 msgr2=0x7f8bb81461e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:03.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.371+0000 7f8bae7fc700 1 --2- 172.21.15.99:0/3268279724 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bb806c8c0 0x7f8bb81461e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:03.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.371+0000 7f8bae7fc700 1 -- 172.21.15.99:0/3268279724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bb8159c30 msgr2=0x7f8bb80af6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:03.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.371+0000 7f8bae7fc700 1 --2- 172.21.15.99:0/3268279724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bb8159c30 0x7f8bb80af6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:03.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.371+0000 7f8bae7fc700 1 -- 172.21.15.99:0/3268279724 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba0009420 con 0x7f8bb80711c0 2023-10-08T00:20:03.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.371+0000 7f8bb77fe700 1 --2- 172.21.15.99:0/3268279724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bb8159c30 0x7f8bb80af6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:20:03.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.371+0000 7f8bae7fc700 1 --2- 172.21.15.99:0/3268279724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bb80711c0 0x7f8bb8146af0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8072020 tx=0x7f8ba0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:03.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.371+0000 7f8baf7fe700 1 -- 172.21.15.99:0/3268279724 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba0011420 con 0x7f8bb80711c0 2023-10-08T00:20:03.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.371+0000 7f8baf7fe700 1 -- 172.21.15.99:0/3268279724 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba0011580 con 0x7f8bb80711c0 2023-10-08T00:20:03.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.371+0000 7f8bbd940700 1 -- 172.21.15.99:0/3268279724 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bb815a910 con 0x7f8bb80711c0 2023-10-08T00:20:03.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.371+0000 7f8baf7fe700 1 -- 172.21.15.99:0/3268279724 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba0011940 con 0x7f8bb80711c0 2023-10-08T00:20:03.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.371+0000 7f8bbd940700 1 -- 172.21.15.99:0/3268279724 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bb815a000 con 0x7f8bb80711c0 2023-10-08T00:20:03.377 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.372+0000 7f8bbd940700 1 -- 172.21.15.99:0/3268279724 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bb804c3c0 con 0x7f8bb80711c0 2023-10-08T00:20:03.378 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.373+0000 7f8baf7fe700 1 -- 172.21.15.99:0/3268279724 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8ba0011d30 con 0x7f8bb80711c0 2023-10-08T00:20:03.378 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.373+0000 7f8baf7fe700 1 --2- 172.21.15.99:0/3268279724 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8b98066f30 0x7f8b980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:03.378 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.373+0000 7f8baf7fe700 1 -- 172.21.15.99:0/3268279724 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f8ba0085da0 con 0x7f8bb80711c0 2023-10-08T00:20:03.378 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.373+0000 7f8bb6ffd700 1 --2- 172.21.15.99:0/3268279724 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8b98066f30 0x7f8b980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:03.378 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.374+0000 7f8bb6ffd700 1 --2- 172.21.15.99:0/3268279724 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8b98066f30 0x7f8b980693e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f8ba800c8f0 tx=0x7f8ba8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:03.378 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.378+0000 7f8baf7fe700 1 -- 172.21.15.99:0/3268279724 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8ba0053270 con 0x7f8bb80711c0 2023-10-08T00:20:03.560 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:03.559+0000 7f8bbd940700 1 -- 172.21.15.99:0/3268279724 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f8bb806f9f0 con 0x7f8bb80711c0 2023-10-08T00:20:04.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.235+0000 7f8baf7fe700 1 -- 172.21.15.99:0/3268279724 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v166) v1 ==== 170+0+0 (secure 0 0 0) 0x7f8ba0056890 con 0x7f8bb80711c0 2023-10-08T00:20:04.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.240+0000 7f8bbd940700 1 -- 172.21.15.99:0/3268279724 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8b98066f30 msgr2=0x7f8b980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:04.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.240+0000 7f8bbd940700 1 --2- 172.21.15.99:0/3268279724 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8b98066f30 0x7f8b980693e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f8ba800c8f0 tx=0x7f8ba8009040 comp rx=0 tx=0).stop 2023-10-08T00:20:04.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.240+0000 7f8bbd940700 1 -- 172.21.15.99:0/3268279724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bb80711c0 msgr2=0x7f8bb8146af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:04.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.240+0000 7f8bbd940700 1 --2- 172.21.15.99:0/3268279724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bb80711c0 0x7f8bb8146af0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8072020 tx=0x7f8ba0017000 comp rx=0 tx=0).stop 2023-10-08T00:20:04.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.240+0000 7f8bbd940700 1 -- 172.21.15.99:0/3268279724 shutdown_connections 2023-10-08T00:20:04.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.241+0000 7f8bbd940700 1 --2- 172.21.15.99:0/3268279724 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8b98066f30 0x7f8b980693e0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:04.242 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.241+0000 7f8bbd940700 1 --2- 172.21.15.99:0/3268279724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bb8159c30 0x7f8bb80af6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:04.242 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.241+0000 7f8bbd940700 1 --2- 172.21.15.99:0/3268279724 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bb806c8c0 0x7f8bb81461e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:04.242 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.241+0000 7f8bbd940700 1 --2- 172.21.15.99:0/3268279724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bb80711c0 0x7f8bb8146af0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:04.242 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.241+0000 7f8bbd940700 1 -- 172.21.15.99:0/3268279724 >> 172.21.15.99:0/3268279724 conn(0x7f8bb80cb030 msgr2=0x7f8bb80aaa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:20:04.242 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.241+0000 7f8bbd940700 1 -- 172.21.15.99:0/3268279724 shutdown_connections 2023-10-08T00:20:04.242 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.241+0000 7f8bbd940700 1 -- 172.21.15.99:0/3268279724 wait complete. 2023-10-08T00:20:04.248 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' created 2023-10-08T00:20:04.260 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T00:20:04.378 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.378+0000 7f66c5f2e700 1 Processor -- start 2023-10-08T00:20:04.379 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.378+0000 7f66c5f2e700 1 -- start start 2023-10-08T00:20:04.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.379+0000 7f66c5f2e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f66c0159ec0 0x7f66c00af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:04.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.379+0000 7f66c5f2e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f66c006acb0 0x7f66c00afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:04.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.379+0000 7f66c5f2e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66c00b0790 con 0x7f66c015a290 2023-10-08T00:20:04.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.379+0000 7f66c5f2e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66c00ce1f0 con 0x7f66c0159ec0 2023-10-08T00:20:04.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.379+0000 7f66c5f2e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66c00b0b20 con 0x7f66c006acb0 2023-10-08T00:20:04.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.379+0000 7f66bffff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f66c006acb0 0x7f66c00afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:04.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.379+0000 7f66bf7fe700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f66c0159ec0 0x7f66c00af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:04.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.379+0000 7f66bffff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f66c006acb0 0x7f66c00afe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.99:52992/0 (socket says 172.21.15.99:52992) 2023-10-08T00:20:04.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.379+0000 7f66bffff700 1 -- 172.21.15.99:0/1705233722 learned_addr learned my addr 172.21.15.99:0/1705233722 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:20:04.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.379+0000 7f66beffd700 1 --1- 172.21.15.99:0/1705233722 >> v1:172.21.15.191:6789/0 conn(0x7f66c015a290 0x7f66c006a5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:36000/0 (socket says 172.21.15.99:36000) 2023-10-08T00:20:04.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.379+0000 7f66bf7fe700 1 -- 172.21.15.99:0/1705233722 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f66c006acb0 msgr2=0x7f66c00afe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:04.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.379+0000 7f66bf7fe700 1 --2- 172.21.15.99:0/1705233722 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f66c006acb0 0x7f66c00afe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:04.382 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.379+0000 7f66bf7fe700 1 -- 172.21.15.99:0/1705233722 >> v1:172.21.15.191:6789/0 conn(0x7f66c015a290 legacy=0x7f66c006a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:04.385 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.379+0000 7f66bf7fe700 1 -- 172.21.15.99:0/1705233722 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c00b04b0 con 0x7f66c0159ec0 2023-10-08T00:20:04.385 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.379+0000 7f66be7fc700 1 -- 172.21.15.99:0/1705233722 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3123193207 0 0) 0x7f66c00b0790 con 0x7f66c015a290 2023-10-08T00:20:04.385 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.379+0000 7f66bffff700 1 --2- 172.21.15.99:0/1705233722 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f66c006acb0 0x7f66c00afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:20:04.385 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.380+0000 7f66bf7fe700 1 --2- 172.21.15.99:0/1705233722 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66c0159ec0 0x7f66c00af940 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f66b0009e60 tx=0x7f66b0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d55f09914f78f9f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:04.385 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.380+0000 7f66be7fc700 1 -- 172.21.15.99:0/1705233722 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b000e070 con 0x7f66c0159ec0 2023-10-08T00:20:04.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.380+0000 7f66be7fc700 1 -- 172.21.15.99:0/1705233722 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66b000bb60 con 0x7f66c0159ec0 2023-10-08T00:20:04.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.380+0000 7f66be7fc700 1 -- 172.21.15.99:0/1705233722 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b0012430 con 0x7f66c0159ec0 2023-10-08T00:20:04.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.380+0000 7f66c5f2e700 1 -- 172.21.15.99:0/1705233722 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66c0159ec0 msgr2=0x7f66c00af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:04.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.380+0000 7f66c5f2e700 1 --2- 172.21.15.99:0/1705233722 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66c0159ec0 0x7f66c00af940 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f66b0009e60 tx=0x7f66b0009780 comp rx=0 tx=0).stop 2023-10-08T00:20:04.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.380+0000 7f66c5f2e700 1 -- 172.21.15.99:0/1705233722 shutdown_connections 2023-10-08T00:20:04.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.380+0000 7f66c5f2e700 1 --2- 172.21.15.99:0/1705233722 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f66c006acb0 0x7f66c00afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:04.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.380+0000 7f66c5f2e700 1 --2- 172.21.15.99:0/1705233722 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66c0159ec0 0x7f66c00af940 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:04.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.380+0000 7f66c5f2e700 1 -- 172.21.15.99:0/1705233722 >> 172.21.15.99:0/1705233722 conn(0x7f66c00d1880 msgr2=0x7f66c00cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:20:04.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.380+0000 7f66c5f2e700 1 -- 172.21.15.99:0/1705233722 shutdown_connections 2023-10-08T00:20:04.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.380+0000 7f66c5f2e700 1 -- 172.21.15.99:0/1705233722 wait complete. 2023-10-08T00:20:04.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.381+0000 7f66c5f2e700 1 Processor -- start 2023-10-08T00:20:04.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.382+0000 7f66c5f2e700 1 -- start start 2023-10-08T00:20:04.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.382+0000 7f66c5f2e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f66c006acb0 0x7f66c015e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:04.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.382+0000 7f66c5f2e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66c0159ec0 0x7f66c016b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:04.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.382+0000 7f66c5f2e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66c015a290 0x7f66c00c9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:04.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.382+0000 7f66c5f2e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c0156280 con 0x7f66c0159ec0 2023-10-08T00:20:04.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.382+0000 7f66c5f2e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c0156e00 con 0x7f66c015a290 2023-10-08T00:20:04.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.382+0000 7f66c5f2e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66c01564d0 con 0x7f66c006acb0 2023-10-08T00:20:04.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.382+0000 7f66beffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66c0159ec0 0x7f66c016b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:04.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.382+0000 7f66beffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66c0159ec0 0x7f66c016b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60790/0 (socket says 172.21.15.99:60790) 2023-10-08T00:20:04.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.382+0000 7f66beffd700 1 -- 172.21.15.99:0/373748919 learned_addr learned my addr 172.21.15.99:0/373748919 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:20:04.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.382+0000 7f66bffff700 1 --2- 172.21.15.99:0/373748919 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66c015a290 0x7f66c00c9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:04.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.383+0000 7f66bf7fe700 1 --2- 172.21.15.99:0/373748919 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f66c006acb0 0x7f66c015e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:04.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.383+0000 7f66bffff700 1 -- 172.21.15.99:0/373748919 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f66c006acb0 msgr2=0x7f66c015e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:04.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.383+0000 7f66bffff700 1 --2- 172.21.15.99:0/373748919 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f66c006acb0 0x7f66c015e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:04.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.383+0000 7f66bffff700 1 -- 172.21.15.99:0/373748919 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66c0159ec0 msgr2=0x7f66c016b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:04.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.383+0000 7f66bffff700 1 --2- 172.21.15.99:0/373748919 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66c0159ec0 0x7f66c016b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:04.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.383+0000 7f66bffff700 1 -- 172.21.15.99:0/373748919 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b80094f0 con 0x7f66c015a290 2023-10-08T00:20:04.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.383+0000 7f66beffd700 1 --2- 172.21.15.99:0/373748919 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66c0159ec0 0x7f66c016b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:20:04.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.383+0000 7f66bffff700 1 --2- 172.21.15.99:0/373748919 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66c015a290 0x7f66c00c9830 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f66c006b0b0 tx=0x7f66b8009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:04.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.383+0000 7f66a77fe700 1 -- 172.21.15.99:0/373748919 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b8013680 con 0x7f66c015a290 2023-10-08T00:20:04.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.383+0000 7f66a77fe700 1 -- 172.21.15.99:0/373748919 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66b80137e0 con 0x7f66c015a290 2023-10-08T00:20:04.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.383+0000 7f66c5f2e700 1 -- 172.21.15.99:0/373748919 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66b0009a20 con 0x7f66c015a290 2023-10-08T00:20:04.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.383+0000 7f66a77fe700 1 -- 172.21.15.99:0/373748919 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b8013ba0 con 0x7f66c015a290 2023-10-08T00:20:04.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.383+0000 7f66c5f2e700 1 -- 172.21.15.99:0/373748919 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66c0156a90 con 0x7f66c015a290 2023-10-08T00:20:04.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.384+0000 7f66c5f2e700 1 -- 172.21.15.99:0/373748919 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66c004c3c0 con 0x7f66c015a290 2023-10-08T00:20:04.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.385+0000 7f66a77fe700 1 -- 172.21.15.99:0/373748919 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f66b8005a00 con 0x7f66c015a290 2023-10-08T00:20:04.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.385+0000 7f66a77fe700 1 --2- 172.21.15.99:0/373748919 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f66ac066f30 0x7f66ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:04.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.385+0000 7f66a77fe700 1 -- 172.21.15.99:0/373748919 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f66b8084830 con 0x7f66c015a290 2023-10-08T00:20:04.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.385+0000 7f66bf7fe700 1 --2- 172.21.15.99:0/373748919 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f66ac066f30 0x7f66ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:04.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.386+0000 7f66bf7fe700 1 --2- 172.21.15.99:0/373748919 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f66ac066f30 0x7f66ac0693e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f66c0157f60 tx=0x7f66b0019000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:04.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.389+0000 7f66a77fe700 1 -- 172.21.15.99:0/373748919 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f66b8051b00 con 0x7f66c015a290 2023-10-08T00:20:04.612 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:04.611+0000 7f66c5f2e700 1 -- 172.21.15.99:0/373748919 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f66c00ceab0 con 0x7f66c015a290 2023-10-08T00:20:05.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.238+0000 7f66a77fe700 1 -- 172.21.15.99:0/373748919 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 20 and data pool 21 v74) v1 ==== 174+0+0 (secure 0 0 0) 0x7f66b80170c0 con 0x7f66c015a290 2023-10-08T00:20:05.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.247+0000 7f66c5f2e700 1 -- 172.21.15.99:0/373748919 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f66ac066f30 msgr2=0x7f66ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:05.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.247+0000 7f66c5f2e700 1 --2- 172.21.15.99:0/373748919 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f66ac066f30 0x7f66ac0693e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f66c0157f60 tx=0x7f66b0019000 comp rx=0 tx=0).stop 2023-10-08T00:20:05.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.247+0000 7f66c5f2e700 1 -- 172.21.15.99:0/373748919 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66c015a290 msgr2=0x7f66c00c9830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:05.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.247+0000 7f66c5f2e700 1 --2- 172.21.15.99:0/373748919 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66c015a290 0x7f66c00c9830 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f66c006b0b0 tx=0x7f66b8009250 comp rx=0 tx=0).stop 2023-10-08T00:20:05.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.250+0000 7f66c5f2e700 1 -- 172.21.15.99:0/373748919 shutdown_connections 2023-10-08T00:20:05.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.250+0000 7f66c5f2e700 1 --2- 172.21.15.99:0/373748919 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f66ac066f30 0x7f66ac0693e0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:05.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.250+0000 7f66c5f2e700 1 --2- 172.21.15.99:0/373748919 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66c015a290 0x7f66c00c9830 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:05.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.250+0000 7f66c5f2e700 1 --2- 172.21.15.99:0/373748919 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f66c006acb0 0x7f66c015e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:05.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.250+0000 7f66c5f2e700 1 --2- 172.21.15.99:0/373748919 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66c0159ec0 0x7f66c016b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:05.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.250+0000 7f66c5f2e700 1 -- 172.21.15.99:0/373748919 >> 172.21.15.99:0/373748919 conn(0x7f66c00d1880 msgr2=0x7f66c00c5b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:20:05.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.253+0000 7f66c5f2e700 1 -- 172.21.15.99:0/373748919 shutdown_connections 2023-10-08T00:20:05.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.254+0000 7f66c5f2e700 1 -- 172.21.15.99:0/373748919 wait complete. 2023-10-08T00:20:05.258 INFO:teuthology.orchestra.run.smithi099.stderr:new fs with metadata pool 20 and data pool 21 2023-10-08T00:20:05.271 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:20:05.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.386+0000 7f720a1e6700 1 Processor -- start 2023-10-08T00:20:05.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.386+0000 7f720a1e6700 1 -- start start 2023-10-08T00:20:05.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.387+0000 7f720a1e6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72040cc3c0 0x7f720406a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:05.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.387+0000 7f720a1e6700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f720406a9f0 0x7f72040af850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:05.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.387+0000 7f720a1e6700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f720406ae60 con 0x7f72040afd90 2023-10-08T00:20:05.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.387+0000 7f720a1e6700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72040b0580 con 0x7f720406a9f0 2023-10-08T00:20:05.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.387+0000 7f720a1e6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f720415b0c0 con 0x7f72040cc3c0 2023-10-08T00:20:05.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.387+0000 7f72037fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72040cc3c0 0x7f720406a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:05.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.387+0000 7f7202ffd700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f720406a9f0 0x7f72040af850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:05.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.387+0000 7f7203fff700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f72040afd90 0x7f72040ce8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:36006/0 (socket says 172.21.15.99:36006) 2023-10-08T00:20:05.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.387+0000 7f7203fff700 1 -- 172.21.15.99:0/3779678254 learned_addr learned my addr 172.21.15.99:0/3779678254 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:20:05.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.388+0000 7f72037fe700 1 -- 172.21.15.99:0/3779678254 >> v2:172.21.15.191:3300/0 conn(0x7f720406a9f0 msgr2=0x7f72040af850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:05.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.388+0000 7f72037fe700 1 --2- 172.21.15.99:0/3779678254 >> v2:172.21.15.191:3300/0 conn(0x7f720406a9f0 0x7f72040af850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:05.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.388+0000 7f72037fe700 1 -- 172.21.15.99:0/3779678254 >> v1:172.21.15.191:6789/0 conn(0x7f72040afd90 legacy=0x7f72040ce8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:05.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.388+0000 7f72037fe700 1 -- 172.21.15.99:0/3779678254 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f720415b550 con 0x7f72040cc3c0 2023-10-08T00:20:05.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.388+0000 7f72027fc700 1 -- 172.21.15.99:0/3779678254 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 366588556 0 0) 0x7f720406ae60 con 0x7f72040afd90 2023-10-08T00:20:05.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.388+0000 7f7202ffd700 1 --2- 172.21.15.99:0/3779678254 >> v2:172.21.15.191:3300/0 conn(0x7f720406a9f0 0x7f72040af850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:20:05.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.388+0000 7f72037fe700 1 --2- 172.21.15.99:0/3779678254 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72040cc3c0 0x7f720406a4b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f71f40098e0 tx=0x7f71f40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=beaa01b01c066ae server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:05.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.388+0000 7f72027fc700 1 -- 172.21.15.99:0/3779678254 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f400b640 con 0x7f72040cc3c0 2023-10-08T00:20:05.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.388+0000 7f72027fc700 1 -- 172.21.15.99:0/3779678254 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f400b7a0 con 0x7f72040cc3c0 2023-10-08T00:20:05.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.388+0000 7f72027fc700 1 -- 172.21.15.99:0/3779678254 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f400bb40 con 0x7f72040cc3c0 2023-10-08T00:20:05.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.389+0000 7f720a1e6700 1 -- 172.21.15.99:0/3779678254 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72040cc3c0 msgr2=0x7f720406a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:05.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.389+0000 7f720a1e6700 1 --2- 172.21.15.99:0/3779678254 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72040cc3c0 0x7f720406a4b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f71f40098e0 tx=0x7f71f40091a0 comp rx=0 tx=0).stop 2023-10-08T00:20:05.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.389+0000 7f720a1e6700 1 -- 172.21.15.99:0/3779678254 shutdown_connections 2023-10-08T00:20:05.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.389+0000 7f720a1e6700 1 --2- 172.21.15.99:0/3779678254 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72040cc3c0 0x7f720406a4b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:05.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.389+0000 7f720a1e6700 1 --2- 172.21.15.99:0/3779678254 >> v2:172.21.15.191:3300/0 conn(0x7f720406a9f0 0x7f72040af850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:05.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.389+0000 7f720a1e6700 1 -- 172.21.15.99:0/3779678254 >> 172.21.15.99:0/3779678254 conn(0x7f7204069800 msgr2=0x7f72040d0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:20:05.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.389+0000 7f720a1e6700 1 -- 172.21.15.99:0/3779678254 shutdown_connections 2023-10-08T00:20:05.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.389+0000 7f720a1e6700 1 -- 172.21.15.99:0/3779678254 wait complete. 2023-10-08T00:20:05.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.390+0000 7f720a1e6700 1 Processor -- start 2023-10-08T00:20:05.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.390+0000 7f720a1e6700 1 -- start start 2023-10-08T00:20:05.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.391+0000 7f720a1e6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f720406a9f0 0x7f72040aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:05.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.391+0000 7f720a1e6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72040afd90 0x7f720416abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:05.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.391+0000 7f720a1e6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72040cc3c0 0x7f720415c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:05.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.391+0000 7f720a1e6700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72040b46c0 con 0x7f72040afd90 2023-10-08T00:20:05.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.391+0000 7f720a1e6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72040b3ab0 con 0x7f720406a9f0 2023-10-08T00:20:05.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.391+0000 7f720a1e6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72040b3d90 con 0x7f72040cc3c0 2023-10-08T00:20:05.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.391+0000 7f7202ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72040afd90 0x7f720416abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:05.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.391+0000 7f7202ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72040afd90 0x7f720416abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60798/0 (socket says 172.21.15.99:60798) 2023-10-08T00:20:05.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.391+0000 7f7202ffd700 1 -- 172.21.15.99:0/1803814904 learned_addr learned my addr 172.21.15.99:0/1803814904 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:20:05.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.391+0000 7f72037fe700 1 --2- 172.21.15.99:0/1803814904 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f720406a9f0 0x7f72040aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:05.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.391+0000 7f7203fff700 1 --2- 172.21.15.99:0/1803814904 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72040cc3c0 0x7f720415c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:05.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.391+0000 7f7202ffd700 1 -- 172.21.15.99:0/1803814904 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72040cc3c0 msgr2=0x7f720415c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:05.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.391+0000 7f7202ffd700 1 --2- 172.21.15.99:0/1803814904 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72040cc3c0 0x7f720415c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:05.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.391+0000 7f7202ffd700 1 -- 172.21.15.99:0/1803814904 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f720406a9f0 msgr2=0x7f72040aecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:05.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.391+0000 7f7202ffd700 1 --2- 172.21.15.99:0/1803814904 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f720406a9f0 0x7f72040aecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:05.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.391+0000 7f7202ffd700 1 -- 172.21.15.99:0/1803814904 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f8009420 con 0x7f72040afd90 2023-10-08T00:20:05.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.391+0000 7f7202ffd700 1 --2- 172.21.15.99:0/1803814904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72040afd90 0x7f720416abf0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f71f800ec80 tx=0x7f71f80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:05.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.391+0000 7f71eb7fe700 1 -- 172.21.15.99:0/1803814904 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f800c920 con 0x7f72040afd90 2023-10-08T00:20:05.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.391+0000 7f71eb7fe700 1 -- 172.21.15.99:0/1803814904 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f800ca80 con 0x7f72040afd90 2023-10-08T00:20:05.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.391+0000 7f720a1e6700 1 -- 172.21.15.99:0/1803814904 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71f4009420 con 0x7f72040afd90 2023-10-08T00:20:05.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.392+0000 7f71eb7fe700 1 -- 172.21.15.99:0/1803814904 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f800ce00 con 0x7f72040afd90 2023-10-08T00:20:05.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.392+0000 7f720a1e6700 1 -- 172.21.15.99:0/1803814904 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72040b4350 con 0x7f72040afd90 2023-10-08T00:20:05.396 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.392+0000 7f720a1e6700 1 -- 172.21.15.99:0/1803814904 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7204089110 con 0x7f72040afd90 2023-10-08T00:20:05.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.393+0000 7f71eb7fe700 1 -- 172.21.15.99:0/1803814904 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f71f8013680 con 0x7f72040afd90 2023-10-08T00:20:05.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.393+0000 7f71eb7fe700 1 --2- 172.21.15.99:0/1803814904 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f71f0066f30 0x7f71f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:05.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.393+0000 7f71eb7fe700 1 -- 172.21.15.99:0/1803814904 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f71f8084900 con 0x7f72040afd90 2023-10-08T00:20:05.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.394+0000 7f72037fe700 1 --2- 172.21.15.99:0/1803814904 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f71f0066f30 0x7f71f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:05.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.394+0000 7f72037fe700 1 --2- 172.21.15.99:0/1803814904 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f71f0066f30 0x7f71f00693e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f71f40091e0 tx=0x7f71f4017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:05.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.398+0000 7f71eb7fe700 1 -- 172.21.15.99:0/1803814904 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f71f8051b70 con 0x7f72040afd90 2023-10-08T00:20:05.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.584+0000 7f720a1e6700 1 -- 172.21.15.99:0/1803814904 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f720404c3c0 con 0x7f72040afd90 2023-10-08T00:20:05.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.585+0000 7f71eb7fe700 1 -- 172.21.15.99:0/1803814904 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v167) v1 ==== 74+0+15378 (secure 0 0 0) 0x7f71f8055190 con 0x7f72040afd90 2023-10-08T00:20:05.586 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:20:05.586 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":167,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:20:04.613676+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:20:02.932463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-08T00:20:03.578010+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:20:05.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.587+0000 7f720a1e6700 1 -- 172.21.15.99:0/1803814904 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f71f0066f30 msgr2=0x7f71f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:05.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.587+0000 7f720a1e6700 1 --2- 172.21.15.99:0/1803814904 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f71f0066f30 0x7f71f00693e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f71f40091e0 tx=0x7f71f4017000 comp rx=0 tx=0).stop 2023-10-08T00:20:05.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.587+0000 7f720a1e6700 1 -- 172.21.15.99:0/1803814904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72040afd90 msgr2=0x7f720416abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:05.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.587+0000 7f720a1e6700 1 --2- 172.21.15.99:0/1803814904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72040afd90 0x7f720416abf0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f71f800ec80 tx=0x7f71f80091a0 comp rx=0 tx=0).stop 2023-10-08T00:20:05.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.587+0000 7f720a1e6700 1 -- 172.21.15.99:0/1803814904 shutdown_connections 2023-10-08T00:20:05.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.587+0000 7f720a1e6700 1 --2- 172.21.15.99:0/1803814904 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f71f0066f30 0x7f71f00693e0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:05.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.587+0000 7f720a1e6700 1 --2- 172.21.15.99:0/1803814904 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f720406a9f0 0x7f72040aecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:05.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.587+0000 7f720a1e6700 1 --2- 172.21.15.99:0/1803814904 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72040cc3c0 0x7f720415c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:05.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.587+0000 7f720a1e6700 1 --2- 172.21.15.99:0/1803814904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72040afd90 0x7f720416abf0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:05.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.587+0000 7f720a1e6700 1 -- 172.21.15.99:0/1803814904 >> 172.21.15.99:0/1803814904 conn(0x7f7204069800 msgr2=0x7f7204158310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:20:05.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.587+0000 7f720a1e6700 1 -- 172.21.15.99:0/1803814904 shutdown_connections 2023-10-08T00:20:05.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.587+0000 7f720a1e6700 1 -- 172.21.15.99:0/1803814904 wait complete. 2023-10-08T00:20:05.600 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:20:05.714 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.713+0000 7f89cab4c700 1 Processor -- start 2023-10-08T00:20:05.714 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.713+0000 7f89cab4c700 1 -- start start 2023-10-08T00:20:05.716 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.714+0000 7f89cab4c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f89c406ab50 0x7f89c40cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:05.716 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.714+0000 7f89cab4c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f89c41438f0 0x7f89c40b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:05.716 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.714+0000 7f89cab4c700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89c40d0870 con 0x7f89c4143520 2023-10-08T00:20:05.716 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.714+0000 7f89cab4c700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89c40cdfb0 con 0x7f89c406ab50 2023-10-08T00:20:05.716 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.714+0000 7f89cab4c700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89c40cfc60 con 0x7f89c41438f0 2023-10-08T00:20:05.716 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.714+0000 7f89c9349700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f89c406ab50 0x7f89c40cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:05.716 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.714+0000 7f89c9349700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f89c406ab50 0x7f89c40cf580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60806/0 (socket says 172.21.15.99:60806) 2023-10-08T00:20:05.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.714+0000 7f89c9349700 1 -- 172.21.15.99:0/3989049633 learned_addr learned my addr 172.21.15.99:0/3989049633 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:20:05.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.714+0000 7f89ca34b700 1 --2- 172.21.15.99:0/3989049633 >> v2:172.21.15.191:3300/0 conn(0x7f89c41438f0 0x7f89c40b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:05.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.714+0000 7f89c9349700 1 -- 172.21.15.99:0/3989049633 >> v2:172.21.15.191:3300/0 conn(0x7f89c41438f0 msgr2=0x7f89c40b3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:05.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.714+0000 7f89c9349700 1 --2- 172.21.15.99:0/3989049633 >> v2:172.21.15.191:3300/0 conn(0x7f89c41438f0 0x7f89c40b3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:05.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.714+0000 7f89c9349700 1 -- 172.21.15.99:0/3989049633 >> v1:172.21.15.191:6789/0 conn(0x7f89c4143520 legacy=0x7f89c406a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:05.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.714+0000 7f89c9349700 1 -- 172.21.15.99:0/3989049633 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89c40cff40 con 0x7f89c406ab50 2023-10-08T00:20:05.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.715+0000 7f89c9349700 1 --2- 172.21.15.99:0/3989049633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89c406ab50 0x7f89c40cf580 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f89c4063e50 tx=0x7f89b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aecfbbe3d534bc69 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:05.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.715+0000 7f89c8b48700 1 -- 172.21.15.99:0/3989049633 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b000b860 con 0x7f89c406ab50 2023-10-08T00:20:05.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.715+0000 7f89c8b48700 1 -- 172.21.15.99:0/3989049633 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89b000b9c0 con 0x7f89c406ab50 2023-10-08T00:20:05.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.715+0000 7f89c8b48700 1 -- 172.21.15.99:0/3989049633 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b000bd60 con 0x7f89c406ab50 2023-10-08T00:20:05.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.715+0000 7f89cab4c700 1 -- 172.21.15.99:0/3989049633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89c406ab50 msgr2=0x7f89c40cf580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:05.718 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.715+0000 7f89cab4c700 1 --2- 172.21.15.99:0/3989049633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89c406ab50 0x7f89c40cf580 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f89c4063e50 tx=0x7f89b0009250 comp rx=0 tx=0).stop 2023-10-08T00:20:05.718 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.715+0000 7f89cab4c700 1 -- 172.21.15.99:0/3989049633 shutdown_connections 2023-10-08T00:20:05.718 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.715+0000 7f89cab4c700 1 --2- 172.21.15.99:0/3989049633 >> v2:172.21.15.191:3300/0 conn(0x7f89c41438f0 0x7f89c40b3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:05.718 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.715+0000 7f89cab4c700 1 --2- 172.21.15.99:0/3989049633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89c406ab50 0x7f89c40cf580 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:05.718 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.715+0000 7f89cab4c700 1 -- 172.21.15.99:0/3989049633 >> 172.21.15.99:0/3989049633 conn(0x7f89c4069750 msgr2=0x7f89c40b25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:20:05.718 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.715+0000 7f89cab4c700 1 -- 172.21.15.99:0/3989049633 shutdown_connections 2023-10-08T00:20:05.718 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.715+0000 7f89cab4c700 1 -- 172.21.15.99:0/3989049633 wait complete. 2023-10-08T00:20:05.721 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.716+0000 7f89cab4c700 1 Processor -- start 2023-10-08T00:20:05.721 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.716+0000 7f89cab4c700 1 -- start start 2023-10-08T00:20:05.721 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.717+0000 7f89cab4c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89c406ab50 0x7f89c413dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:05.721 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.717+0000 7f89cab4c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89c4143520 0x7f89c4139350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:05.721 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.717+0000 7f89cab4c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89c41438f0 0x7f89c413aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:05.721 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.717+0000 7f89cab4c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c413a5f0 con 0x7f89c41438f0 2023-10-08T00:20:05.721 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.717+0000 7f89cab4c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c413b330 con 0x7f89c406ab50 2023-10-08T00:20:05.721 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.717+0000 7f89cab4c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89c41399e0 con 0x7f89c4143520 2023-10-08T00:20:05.721 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.717+0000 7f89ca34b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89c41438f0 0x7f89c413aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:05.721 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.717+0000 7f89ca34b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89c41438f0 0x7f89c413aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60820/0 (socket says 172.21.15.99:60820) 2023-10-08T00:20:05.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.717+0000 7f89ca34b700 1 -- 172.21.15.99:0/1031287479 learned_addr learned my addr 172.21.15.99:0/1031287479 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:20:05.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.717+0000 7f89c9b4a700 1 --2- 172.21.15.99:0/1031287479 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89c406ab50 0x7f89c413dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:05.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.717+0000 7f89c9349700 1 --2- 172.21.15.99:0/1031287479 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89c4143520 0x7f89c4139350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:05.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.718+0000 7f89ca34b700 1 -- 172.21.15.99:0/1031287479 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89c4143520 msgr2=0x7f89c4139350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:05.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.718+0000 7f89ca34b700 1 --2- 172.21.15.99:0/1031287479 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89c4143520 0x7f89c4139350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:05.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.718+0000 7f89ca34b700 1 -- 172.21.15.99:0/1031287479 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89c406ab50 msgr2=0x7f89c413dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:05.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.718+0000 7f89ca34b700 1 --2- 172.21.15.99:0/1031287479 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89c406ab50 0x7f89c413dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:05.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.718+0000 7f89ca34b700 1 -- 172.21.15.99:0/1031287479 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b00094f0 con 0x7f89c41438f0 2023-10-08T00:20:05.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.718+0000 7f89ca34b700 1 --2- 172.21.15.99:0/1031287479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89c41438f0 0x7f89c413aa80 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f89bc007f50 tx=0x7f89bc009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:05.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.718+0000 7f89c1ffb700 1 -- 172.21.15.99:0/1031287479 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89bc00eb70 con 0x7f89c41438f0 2023-10-08T00:20:05.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.718+0000 7f89c1ffb700 1 -- 172.21.15.99:0/1031287479 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89bc00ecf0 con 0x7f89c41438f0 2023-10-08T00:20:05.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.718+0000 7f89cab4c700 1 -- 172.21.15.99:0/1031287479 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89c4139cc0 con 0x7f89c41438f0 2023-10-08T00:20:05.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.718+0000 7f89c1ffb700 1 -- 172.21.15.99:0/1031287479 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89bc014670 con 0x7f89c41438f0 2023-10-08T00:20:05.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.718+0000 7f89cab4c700 1 -- 172.21.15.99:0/1031287479 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89c413a280 con 0x7f89c41438f0 2023-10-08T00:20:05.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.719+0000 7f89cab4c700 1 -- 172.21.15.99:0/1031287479 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89c413a7a0 con 0x7f89c41438f0 2023-10-08T00:20:05.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.719+0000 7f89c1ffb700 1 -- 172.21.15.99:0/1031287479 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f89bc0149e0 con 0x7f89c41438f0 2023-10-08T00:20:05.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.720+0000 7f89c1ffb700 1 --2- 172.21.15.99:0/1031287479 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f89a8066f30 0x7f89a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:05.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.720+0000 7f89c1ffb700 1 -- 172.21.15.99:0/1031287479 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f89bc084ac0 con 0x7f89c41438f0 2023-10-08T00:20:05.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.720+0000 7f89c9b4a700 1 --2- 172.21.15.99:0/1031287479 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f89a8066f30 0x7f89a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:05.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.720+0000 7f89c9b4a700 1 --2- 172.21.15.99:0/1031287479 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f89a8066f30 0x7f89a80693e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f89b8003a80 tx=0x7f89b8007ec0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:05.724 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.724+0000 7f89c1ffb700 1 -- 172.21.15.99:0/1031287479 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f89c413a7a0 con 0x7f89c41438f0 2023-10-08T00:20:05.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.905+0000 7f89cab4c700 1 -- 172.21.15.99:0/1031287479 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f89c413a7a0 con 0x7f89c41438f0 2023-10-08T00:20:05.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.906+0000 7f89c1ffb700 1 -- 172.21.15.99:0/1031287479 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v167) v1 ==== 74+0+15378 (secure 0 0 0) 0x7f89c413a7a0 con 0x7f89c41438f0 2023-10-08T00:20:05.906 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:20:05.907 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":167,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:20:04.613676+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:20:02.932463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-08T00:20:03.578010+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:20:05.910 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.908+0000 7f89cab4c700 1 -- 172.21.15.99:0/1031287479 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f89a8066f30 msgr2=0x7f89a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:05.911 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.908+0000 7f89cab4c700 1 --2- 172.21.15.99:0/1031287479 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f89a8066f30 0x7f89a80693e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f89b8003a80 tx=0x7f89b8007ec0 comp rx=0 tx=0).stop 2023-10-08T00:20:05.911 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.908+0000 7f89cab4c700 1 -- 172.21.15.99:0/1031287479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89c41438f0 msgr2=0x7f89c413aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:05.911 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.908+0000 7f89cab4c700 1 --2- 172.21.15.99:0/1031287479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89c41438f0 0x7f89c413aa80 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f89bc007f50 tx=0x7f89bc009860 comp rx=0 tx=0).stop 2023-10-08T00:20:05.911 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.908+0000 7f89cab4c700 1 -- 172.21.15.99:0/1031287479 shutdown_connections 2023-10-08T00:20:05.911 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.908+0000 7f89cab4c700 1 --2- 172.21.15.99:0/1031287479 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f89a8066f30 0x7f89a80693e0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:05.911 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.908+0000 7f89cab4c700 1 --2- 172.21.15.99:0/1031287479 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89c406ab50 0x7f89c413dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:05.911 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.908+0000 7f89cab4c700 1 --2- 172.21.15.99:0/1031287479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89c41438f0 0x7f89c413aa80 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:05.911 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.908+0000 7f89cab4c700 1 --2- 172.21.15.99:0/1031287479 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89c4143520 0x7f89c4139350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:05.911 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.908+0000 7f89cab4c700 1 -- 172.21.15.99:0/1031287479 >> 172.21.15.99:0/1031287479 conn(0x7f89c4069750 msgr2=0x7f89c4137320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:20:05.911 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.908+0000 7f89cab4c700 1 -- 172.21.15.99:0/1031287479 shutdown_connections 2023-10-08T00:20:05.911 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:05.908+0000 7f89cab4c700 1 -- 172.21.15.99:0/1031287479 wait complete. 2023-10-08T00:20:05.921 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T00:20:06.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.036+0000 7f0f6ec96700 1 Processor -- start 2023-10-08T00:20:06.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.036+0000 7f0f6ec96700 1 -- start start 2023-10-08T00:20:06.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.037+0000 7f0f6ec96700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0f68159ec0 0x7f0f680af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:06.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.037+0000 7f0f6ec96700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f680b0790 con 0x7f0f6815a290 2023-10-08T00:20:06.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.037+0000 7f0f6ec96700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f680ce1f0 con 0x7f0f6806acb0 2023-10-08T00:20:06.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.037+0000 7f0f6ec96700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f680b0b20 con 0x7f0f68159ec0 2023-10-08T00:20:06.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.037+0000 7f0f5ffff700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f0f6815a290 0x7f0f6806a5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56072/0 (socket says 172.21.15.99:56072) 2023-10-08T00:20:06.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.037+0000 7f0f5ffff700 1 -- 172.21.15.99:0/598161609 learned_addr learned my addr 172.21.15.99:0/598161609 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:20:06.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.038+0000 7f0f6ca32700 1 --2- 172.21.15.99:0/598161609 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0f68159ec0 0x7f0f680af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:06.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.038+0000 7f0f5f7fe700 1 -- 172.21.15.99:0/598161609 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4082855584 0 0) 0x7f0f680b0790 con 0x7f0f6815a290 2023-10-08T00:20:06.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.038+0000 7f0f5f7fe700 1 -- 172.21.15.99:0/598161609 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f50003570 con 0x7f0f6815a290 2023-10-08T00:20:06.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.038+0000 7f0f5f7fe700 1 -- 172.21.15.99:0/598161609 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1140310310 0 0) 0x7f0f680ce1f0 con 0x7f0f6806acb0 2023-10-08T00:20:06.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.038+0000 7f0f5f7fe700 1 -- 172.21.15.99:0/598161609 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f680b0790 con 0x7f0f6806acb0 2023-10-08T00:20:06.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.038+0000 7f0f5f7fe700 1 -- 172.21.15.99:0/598161609 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2425424535 0 0) 0x7f0f50003570 con 0x7f0f6815a290 2023-10-08T00:20:06.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.038+0000 7f0f5f7fe700 1 -- 172.21.15.99:0/598161609 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0f68159ec0 msgr2=0x7f0f680af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:06.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.038+0000 7f0f5f7fe700 1 --2- 172.21.15.99:0/598161609 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0f68159ec0 0x7f0f680af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:06.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.038+0000 7f0f5f7fe700 1 -- 172.21.15.99:0/598161609 >> v1:172.21.15.191:6789/0 conn(0x7f0f6806acb0 legacy=0x7f0f680cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:06.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.038+0000 7f0f5f7fe700 1 -- 172.21.15.99:0/598161609 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f680b04b0 con 0x7f0f6815a290 2023-10-08T00:20:06.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.038+0000 7f0f5f7fe700 1 -- 172.21.15.99:0/598161609 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f0f58001470 con 0x7f0f6815a290 2023-10-08T00:20:06.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.038+0000 7f0f5f7fe700 1 -- 172.21.15.99:0/598161609 >> v1:172.21.15.99:6789/0 conn(0x7f0f6815a290 legacy=0x7f0f6806a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:06.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.038+0000 7f0f5f7fe700 1 --2- 172.21.15.99:0/598161609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f50003c20 0x7f0f50004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:06.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.038+0000 7f0f5f7fe700 1 --2- 172.21.15.99:0/598161609 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0f68159ec0 0x7f0f50004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:06.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.038+0000 7f0f5f7fe700 1 --2- 172.21.15.99:0/598161609 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0f50004d50 0x7f0f500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:06.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.038+0000 7f0f5f7fe700 1 -- 172.21.15.99:0/598161609 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f680b0b20 con 0x7f0f50003c20 2023-10-08T00:20:06.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.038+0000 7f0f5f7fe700 1 -- 172.21.15.99:0/598161609 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5000bbf0 con 0x7f0f50004d50 2023-10-08T00:20:06.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.038+0000 7f0f5f7fe700 1 -- 172.21.15.99:0/598161609 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5000bd60 con 0x7f0f68159ec0 2023-10-08T00:20:06.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.038+0000 7f0f6ca32700 1 --2- 172.21.15.99:0/598161609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f50003c20 0x7f0f50004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:06.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.038+0000 7f0f5ffff700 1 --2- 172.21.15.99:0/598161609 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0f68159ec0 0x7f0f50004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:06.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.038+0000 7f0f6d233700 1 --2- 172.21.15.99:0/598161609 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0f50004d50 0x7f0f500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:06.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.038+0000 7f0f6ca32700 1 -- 172.21.15.99:0/598161609 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0f68159ec0 msgr2=0x7f0f50004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:06.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.038+0000 7f0f6ca32700 1 --2- 172.21.15.99:0/598161609 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0f68159ec0 0x7f0f50004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:06.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.038+0000 7f0f6ca32700 1 -- 172.21.15.99:0/598161609 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0f50004d50 msgr2=0x7f0f500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:06.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.038+0000 7f0f6ca32700 1 --2- 172.21.15.99:0/598161609 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0f50004d50 0x7f0f500051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:06.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.038+0000 7f0f6ca32700 1 -- 172.21.15.99:0/598161609 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f50003570 con 0x7f0f50003c20 2023-10-08T00:20:06.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.039+0000 7f0f6ca32700 1 --2- 172.21.15.99:0/598161609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f50003c20 0x7f0f50004010 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f0f640094d0 tx=0x7f0f6400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e9c736aa883946 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:06.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.039+0000 7f0f5f7fe700 1 -- 172.21.15.99:0/598161609 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6400ea90 con 0x7f0f50003c20 2023-10-08T00:20:06.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.039+0000 7f0f5f7fe700 1 -- 172.21.15.99:0/598161609 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f6400f040 con 0x7f0f50003c20 2023-10-08T00:20:06.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.039+0000 7f0f5f7fe700 1 -- 172.21.15.99:0/598161609 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6400ee40 con 0x7f0f50003c20 2023-10-08T00:20:06.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.039+0000 7f0f6ec96700 1 -- 172.21.15.99:0/598161609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f50003c20 msgr2=0x7f0f50004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:06.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.039+0000 7f0f6ec96700 1 --2- 172.21.15.99:0/598161609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f50003c20 0x7f0f50004010 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f0f640094d0 tx=0x7f0f6400b040 comp rx=0 tx=0).stop 2023-10-08T00:20:06.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.039+0000 7f0f6d233700 1 -- 172.21.15.99:0/598161609 reap_dead start 2023-10-08T00:20:06.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.039+0000 7f0f6ec96700 1 -- 172.21.15.99:0/598161609 shutdown_connections 2023-10-08T00:20:06.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.039+0000 7f0f6ec96700 1 -- 172.21.15.99:0/598161609 >> 172.21.15.99:0/598161609 conn(0x7f0f680d1880 msgr2=0x7f0f680cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:20:06.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.039+0000 7f0f6ec96700 1 -- 172.21.15.99:0/598161609 shutdown_connections 2023-10-08T00:20:06.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.039+0000 7f0f6ec96700 1 -- 172.21.15.99:0/598161609 wait complete. 2023-10-08T00:20:06.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.040+0000 7f0f6ec96700 1 Processor -- start 2023-10-08T00:20:06.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.041+0000 7f0f6ec96700 1 -- start start 2023-10-08T00:20:06.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.041+0000 7f0f6ec96700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f50003c20 0x7f0f6814d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:06.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.041+0000 7f0f6ec96700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0f680b3b50 0x7f0f680b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:06.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.041+0000 7f0f6ec96700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0f680b5150 0x7f0f680b55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:06.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.041+0000 7f0f6ec96700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6815b2e0 con 0x7f0f50003c20 2023-10-08T00:20:06.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.041+0000 7f0f6ec96700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f680b45f0 con 0x7f0f680b3b50 2023-10-08T00:20:06.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.041+0000 7f0f6ec96700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f680b48d0 con 0x7f0f680b5150 2023-10-08T00:20:06.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.041+0000 7f0f6ca32700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f50003c20 0x7f0f6814d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:06.048 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.041+0000 7f0f6ca32700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f50003c20 0x7f0f6814d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60826/0 (socket says 172.21.15.99:60826) 2023-10-08T00:20:06.048 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.041+0000 7f0f6ca32700 1 -- 172.21.15.99:0/3253690870 learned_addr learned my addr 172.21.15.99:0/3253690870 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:20:06.048 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.041+0000 7f0f6d233700 1 --2- 172.21.15.99:0/3253690870 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0f680b5150 0x7f0f680b55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:06.048 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.041+0000 7f0f5ffff700 1 --2- 172.21.15.99:0/3253690870 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0f680b3b50 0x7f0f680b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:06.048 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.042+0000 7f0f6ca32700 1 -- 172.21.15.99:0/3253690870 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0f680b5150 msgr2=0x7f0f680b55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:06.048 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.042+0000 7f0f6ca32700 1 --2- 172.21.15.99:0/3253690870 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0f680b5150 0x7f0f680b55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:06.048 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.042+0000 7f0f6ca32700 1 -- 172.21.15.99:0/3253690870 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0f680b3b50 msgr2=0x7f0f680b3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:06.048 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.042+0000 7f0f6ca32700 1 --2- 172.21.15.99:0/3253690870 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0f680b3b50 0x7f0f680b3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:06.048 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.042+0000 7f0f6ca32700 1 -- 172.21.15.99:0/3253690870 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f6815af70 con 0x7f0f50003c20 2023-10-08T00:20:06.048 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.042+0000 7f0f6ca32700 1 --2- 172.21.15.99:0/3253690870 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f50003c20 0x7f0f6814d810 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f0f500057a0 tx=0x7f0f6401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:06.048 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.042+0000 7f0f5cff9700 1 -- 172.21.15.99:0/3253690870 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6401d420 con 0x7f0f50003c20 2023-10-08T00:20:06.049 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.042+0000 7f0f5cff9700 1 -- 172.21.15.99:0/3253690870 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f6401d5a0 con 0x7f0f50003c20 2023-10-08T00:20:06.049 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.042+0000 7f0f6ec96700 1 -- 172.21.15.99:0/3253690870 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f6816ad10 con 0x7f0f50003c20 2023-10-08T00:20:06.049 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.042+0000 7f0f5cff9700 1 -- 172.21.15.99:0/3253690870 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6401d960 con 0x7f0f50003c20 2023-10-08T00:20:06.049 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.042+0000 7f0f6ec96700 1 -- 172.21.15.99:0/3253690870 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f6815b770 con 0x7f0f50003c20 2023-10-08T00:20:06.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.043+0000 7f0f6ec96700 1 -- 172.21.15.99:0/3253690870 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f680b4b10 con 0x7f0f50003c20 2023-10-08T00:20:06.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.043+0000 7f0f5cff9700 1 -- 172.21.15.99:0/3253690870 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0f64003040 con 0x7f0f50003c20 2023-10-08T00:20:06.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.044+0000 7f0f5cff9700 1 --2- 172.21.15.99:0/3253690870 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0f54066fb0 0x7f0f54069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:06.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.044+0000 7f0f5cff9700 1 -- 172.21.15.99:0/3253690870 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0f64086a00 con 0x7f0f50003c20 2023-10-08T00:20:06.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.044+0000 7f0f5ffff700 1 --2- 172.21.15.99:0/3253690870 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0f54066fb0 0x7f0f54069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:06.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.044+0000 7f0f5ffff700 1 --2- 172.21.15.99:0/3253690870 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0f54066fb0 0x7f0f54069460 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f0f681597e0 tx=0x7f0f58009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:06.051 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.048+0000 7f0f5cff9700 1 -- 172.21.15.99:0/3253690870 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0f680b4b10 con 0x7f0f50003c20 2023-10-08T00:20:06.275 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:06.275+0000 7f0f6ec96700 1 -- 172.21.15.99:0/3253690870 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f0f680b4b10 con 0x7f0f50003c20 2023-10-08T00:20:07.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.255+0000 7f0f5cff9700 1 -- 172.21.15.99:0/3253690870 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v76) v1 ==== 118+0+0 (secure 0 0 0) 0x7f0f680b4b10 con 0x7f0f50003c20 2023-10-08T00:20:07.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.257+0000 7f0f6ec96700 1 -- 172.21.15.99:0/3253690870 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0f54066fb0 msgr2=0x7f0f54069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:07.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.257+0000 7f0f6ec96700 1 --2- 172.21.15.99:0/3253690870 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0f54066fb0 0x7f0f54069460 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f0f681597e0 tx=0x7f0f58009040 comp rx=0 tx=0).stop 2023-10-08T00:20:07.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.257+0000 7f0f6ec96700 1 -- 172.21.15.99:0/3253690870 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f50003c20 msgr2=0x7f0f6814d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:07.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.257+0000 7f0f6ec96700 1 --2- 172.21.15.99:0/3253690870 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f50003c20 0x7f0f6814d810 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f0f500057a0 tx=0x7f0f6401a000 comp rx=0 tx=0).stop 2023-10-08T00:20:07.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.257+0000 7f0f6ec96700 1 -- 172.21.15.99:0/3253690870 shutdown_connections 2023-10-08T00:20:07.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.257+0000 7f0f6ec96700 1 --2- 172.21.15.99:0/3253690870 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0f54066fb0 0x7f0f54069460 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:07.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.257+0000 7f0f6ec96700 1 --2- 172.21.15.99:0/3253690870 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0f680b5150 0x7f0f680b55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:07.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.257+0000 7f0f6ec96700 1 --2- 172.21.15.99:0/3253690870 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f50003c20 0x7f0f6814d810 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:07.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.257+0000 7f0f6ec96700 1 --2- 172.21.15.99:0/3253690870 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0f680b3b50 0x7f0f680b3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:07.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.257+0000 7f0f6ec96700 1 -- 172.21.15.99:0/3253690870 >> 172.21.15.99:0/3253690870 conn(0x7f0f680d1880 msgr2=0x7f0f680c6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:20:07.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.258+0000 7f0f6ec96700 1 -- 172.21.15.99:0/3253690870 shutdown_connections 2023-10-08T00:20:07.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.258+0000 7f0f6ec96700 1 -- 172.21.15.99:0/3253690870 wait complete. 2023-10-08T00:20:07.271 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T00:20:07.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.389+0000 7f875a25b700 1 Processor -- start 2023-10-08T00:20:07.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.389+0000 7f875a25b700 1 -- start start 2023-10-08T00:20:07.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.389+0000 7f875a25b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87540cd300 0x7f875415afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:07.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.389+0000 7f875a25b700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f87540b6760 0x7f87540d0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:07.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.389+0000 7f875a25b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f87540d12a0 0x7f875406a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:07.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.389+0000 7f875a25b700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f875415e7a0 con 0x7f87540b6760 2023-10-08T00:20:07.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.390+0000 7f875a25b700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87541513f0 con 0x7f87540d12a0 2023-10-08T00:20:07.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.390+0000 7f875a25b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87540d1700 con 0x7f87540cd300 2023-10-08T00:20:07.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.390+0000 7f8752ffd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f87540b6760 0x7f87540d0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:07.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.390+0000 7f8752ffd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f87540b6760 0x7f87540d0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60830/0 (socket says 172.21.15.99:60830) 2023-10-08T00:20:07.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.390+0000 7f8752ffd700 1 -- 172.21.15.99:0/647950707 learned_addr learned my addr 172.21.15.99:0/647950707 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:20:07.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.390+0000 7f87537fe700 1 --2- 172.21.15.99:0/647950707 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87540cd300 0x7f875415afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:07.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.390+0000 7f8753fff700 1 --2- 172.21.15.99:0/647950707 >> v2:172.21.15.191:3300/0 conn(0x7f87540d12a0 0x7f875406a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:07.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.390+0000 7f8752ffd700 1 -- 172.21.15.99:0/647950707 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87540cd300 msgr2=0x7f875415afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:07.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.390+0000 7f8752ffd700 1 --2- 172.21.15.99:0/647950707 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87540cd300 0x7f875415afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:07.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.390+0000 7f8752ffd700 1 -- 172.21.15.99:0/647950707 >> v2:172.21.15.191:3300/0 conn(0x7f87540d12a0 msgr2=0x7f875406a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:07.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.390+0000 7f8752ffd700 1 --2- 172.21.15.99:0/647950707 >> v2:172.21.15.191:3300/0 conn(0x7f87540d12a0 0x7f875406a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:07.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.390+0000 7f8752ffd700 1 -- 172.21.15.99:0/647950707 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f875415db20 con 0x7f87540b6760 2023-10-08T00:20:07.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.390+0000 7f8753fff700 1 --2- 172.21.15.99:0/647950707 >> v2:172.21.15.191:3300/0 conn(0x7f87540d12a0 0x7f875406a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:20:07.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.390+0000 7f8752ffd700 1 --2- 172.21.15.99:0/647950707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87540b6760 0x7f87540d0d60 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f875408a160 tx=0x7f8748009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fcb2e370962d958 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:07.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.390+0000 7f87527fc700 1 -- 172.21.15.99:0/647950707 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8748009e70 con 0x7f87540b6760 2023-10-08T00:20:07.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.390+0000 7f87527fc700 1 -- 172.21.15.99:0/647950707 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f874800b860 con 0x7f87540b6760 2023-10-08T00:20:07.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.390+0000 7f87527fc700 1 -- 172.21.15.99:0/647950707 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874800bc00 con 0x7f87540b6760 2023-10-08T00:20:07.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.391+0000 7f875a25b700 1 -- 172.21.15.99:0/647950707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87540b6760 msgr2=0x7f87540d0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:07.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.391+0000 7f875a25b700 1 --2- 172.21.15.99:0/647950707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87540b6760 0x7f87540d0d60 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f875408a160 tx=0x7f8748009250 comp rx=0 tx=0).stop 2023-10-08T00:20:07.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.391+0000 7f875a25b700 1 -- 172.21.15.99:0/647950707 shutdown_connections 2023-10-08T00:20:07.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.391+0000 7f875a25b700 1 --2- 172.21.15.99:0/647950707 >> v2:172.21.15.191:3300/0 conn(0x7f87540d12a0 0x7f875406a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:07.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.391+0000 7f875a25b700 1 --2- 172.21.15.99:0/647950707 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87540cd300 0x7f875415afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:07.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.391+0000 7f875a25b700 1 --2- 172.21.15.99:0/647950707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87540b6760 0x7f87540d0d60 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:07.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.391+0000 7f875a25b700 1 -- 172.21.15.99:0/647950707 >> 172.21.15.99:0/647950707 conn(0x7f8754068d90 msgr2=0x7f8754069190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:20:07.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.391+0000 7f875a25b700 1 -- 172.21.15.99:0/647950707 shutdown_connections 2023-10-08T00:20:07.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.391+0000 7f875a25b700 1 -- 172.21.15.99:0/647950707 wait complete. 2023-10-08T00:20:07.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.392+0000 7f875a25b700 1 Processor -- start 2023-10-08T00:20:07.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.392+0000 7f875a25b700 1 -- start start 2023-10-08T00:20:07.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.393+0000 7f875a25b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87540b6760 0x7f875416a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:07.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.393+0000 7f875a25b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87540cd300 0x7f875416a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:07.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.393+0000 7f875a25b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87540d12a0 0x7f875406a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:07.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.393+0000 7f875a25b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87540b1560 con 0x7f87540cd300 2023-10-08T00:20:07.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.393+0000 7f875a25b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87540b0670 con 0x7f87540d12a0 2023-10-08T00:20:07.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.393+0000 7f875a25b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87540b0950 con 0x7f87540b6760 2023-10-08T00:20:07.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.393+0000 7f8752ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87540cd300 0x7f875416a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:07.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.393+0000 7f8752ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87540cd300 0x7f875416a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60844/0 (socket says 172.21.15.99:60844) 2023-10-08T00:20:07.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.393+0000 7f8752ffd700 1 -- 172.21.15.99:0/468202831 learned_addr learned my addr 172.21.15.99:0/468202831 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:20:07.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.393+0000 7f87537fe700 1 --2- 172.21.15.99:0/468202831 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87540b6760 0x7f875416a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:07.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.393+0000 7f8753fff700 1 --2- 172.21.15.99:0/468202831 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87540d12a0 0x7f875406a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:07.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.393+0000 7f8752ffd700 1 -- 172.21.15.99:0/468202831 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87540b6760 msgr2=0x7f875416a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:07.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.393+0000 7f8752ffd700 1 --2- 172.21.15.99:0/468202831 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87540b6760 0x7f875416a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:07.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.393+0000 7f8752ffd700 1 -- 172.21.15.99:0/468202831 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87540d12a0 msgr2=0x7f875406a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:07.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.393+0000 7f8752ffd700 1 --2- 172.21.15.99:0/468202831 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87540d12a0 0x7f875406a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:07.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.393+0000 7f8752ffd700 1 -- 172.21.15.99:0/468202831 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87480094f0 con 0x7f87540cd300 2023-10-08T00:20:07.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.393+0000 7f8752ffd700 1 --2- 172.21.15.99:0/468202831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87540cd300 0x7f875416a7c0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f8748009290 tx=0x7f8748017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:07.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.393+0000 7f873b7fe700 1 -- 172.21.15.99:0/468202831 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8748000d10 con 0x7f87540cd300 2023-10-08T00:20:07.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.393+0000 7f873b7fe700 1 -- 172.21.15.99:0/468202831 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f874801f070 con 0x7f87540cd300 2023-10-08T00:20:07.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.393+0000 7f875a25b700 1 -- 172.21.15.99:0/468202831 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87540b0c30 con 0x7f87540cd300 2023-10-08T00:20:07.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.394+0000 7f873b7fe700 1 -- 172.21.15.99:0/468202831 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87480266d0 con 0x7f87540cd300 2023-10-08T00:20:07.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.394+0000 7f875a25b700 1 -- 172.21.15.99:0/468202831 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87540b11f0 con 0x7f87540cd300 2023-10-08T00:20:07.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.394+0000 7f875a25b700 1 -- 172.21.15.99:0/468202831 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f875404c3c0 con 0x7f87540cd300 2023-10-08T00:20:07.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.395+0000 7f873b7fe700 1 -- 172.21.15.99:0/468202831 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8748026a80 con 0x7f87540cd300 2023-10-08T00:20:07.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.395+0000 7f873b7fe700 1 --2- 172.21.15.99:0/468202831 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8740066f30 0x7f87400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:07.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.395+0000 7f873b7fe700 1 -- 172.21.15.99:0/468202831 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8748021070 con 0x7f87540cd300 2023-10-08T00:20:07.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.395+0000 7f87537fe700 1 --2- 172.21.15.99:0/468202831 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8740066f30 0x7f87400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:07.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.396+0000 7f87537fe700 1 --2- 172.21.15.99:0/468202831 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8740066f30 0x7f87400693e0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f8744009d10 tx=0x7f87440091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:07.401 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.399+0000 7f873b7fe700 1 -- 172.21.15.99:0/468202831 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8748053440 con 0x7f87540cd300 2023-10-08T00:20:07.628 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:07.627+0000 7f875a25b700 1 -- 172.21.15.99:0/468202831 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f87541554e0 con 0x7f87540cd300 2023-10-08T00:20:08.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.260+0000 7f873b7fe700 1 -- 172.21.15.99:0/468202831 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v77) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8748056a60 con 0x7f87540cd300 2023-10-08T00:20:08.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.262+0000 7f875a25b700 1 -- 172.21.15.99:0/468202831 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8740066f30 msgr2=0x7f87400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:08.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.263+0000 7f875a25b700 1 --2- 172.21.15.99:0/468202831 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8740066f30 0x7f87400693e0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f8744009d10 tx=0x7f87440091a0 comp rx=0 tx=0).stop 2023-10-08T00:20:08.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.263+0000 7f875a25b700 1 -- 172.21.15.99:0/468202831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87540cd300 msgr2=0x7f875416a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:08.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.263+0000 7f875a25b700 1 --2- 172.21.15.99:0/468202831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87540cd300 0x7f875416a7c0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f8748009290 tx=0x7f8748017000 comp rx=0 tx=0).stop 2023-10-08T00:20:08.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.263+0000 7f875a25b700 1 -- 172.21.15.99:0/468202831 shutdown_connections 2023-10-08T00:20:08.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.263+0000 7f875a25b700 1 --2- 172.21.15.99:0/468202831 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8740066f30 0x7f87400693e0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:08.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.263+0000 7f875a25b700 1 --2- 172.21.15.99:0/468202831 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87540d12a0 0x7f875406a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:08.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.263+0000 7f875a25b700 1 --2- 172.21.15.99:0/468202831 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87540b6760 0x7f875416a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:08.264 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.263+0000 7f875a25b700 1 --2- 172.21.15.99:0/468202831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87540cd300 0x7f875416a7c0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:08.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.263+0000 7f875a25b700 1 -- 172.21.15.99:0/468202831 >> 172.21.15.99:0/468202831 conn(0x7f8754068d90 msgr2=0x7f875414f960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:20:08.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.263+0000 7f875a25b700 1 -- 172.21.15.99:0/468202831 shutdown_connections 2023-10-08T00:20:08.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.263+0000 7f875a25b700 1 -- 172.21.15.99:0/468202831 wait complete. 2023-10-08T00:20:08.278 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:20:08.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.394+0000 7f0010285700 1 Processor -- start 2023-10-08T00:20:08.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.394+0000 7f0010285700 1 -- start start 2023-10-08T00:20:08.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.395+0000 7f0010285700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f00080afd90 0x7f00080c9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:08.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.395+0000 7f0010285700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f000806f1a0 0x7f000815b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:08.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.395+0000 7f0010285700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00080ca270 con 0x7f00080b0160 2023-10-08T00:20:08.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.395+0000 7f0010285700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00080c87c0 con 0x7f00080afd90 2023-10-08T00:20:08.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.395+0000 7f0010285700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f000815b770 con 0x7f000806f1a0 2023-10-08T00:20:08.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.395+0000 7f000e021700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f00080afd90 0x7f00080c9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:08.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.395+0000 7f000e822700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f00080b0160 0x7f0008070700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56086/0 (socket says 172.21.15.99:56086) 2023-10-08T00:20:08.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.395+0000 7f000e021700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f00080afd90 0x7f00080c9c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60858/0 (socket says 172.21.15.99:60858) 2023-10-08T00:20:08.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.395+0000 7f000e822700 1 -- 172.21.15.99:0/3653152406 learned_addr learned my addr 172.21.15.99:0/3653152406 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:20:08.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.395+0000 7f000e021700 1 -- 172.21.15.99:0/3653152406 >> v2:172.21.15.191:3300/0 conn(0x7f000806f1a0 msgr2=0x7f000815b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:08.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.395+0000 7f000d820700 1 --2- 172.21.15.99:0/3653152406 >> v2:172.21.15.191:3300/0 conn(0x7f000806f1a0 0x7f000815b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:08.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.396+0000 7f000e021700 1 --2- 172.21.15.99:0/3653152406 >> v2:172.21.15.191:3300/0 conn(0x7f000806f1a0 0x7f000815b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:08.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.396+0000 7f000d01f700 1 -- 172.21.15.99:0/3653152406 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1424264263 0 0) 0x7f00080ca270 con 0x7f00080b0160 2023-10-08T00:20:08.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.396+0000 7f000e021700 1 -- 172.21.15.99:0/3653152406 >> v1:172.21.15.99:6789/0 conn(0x7f00080b0160 legacy=0x7f0008070700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:08.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.396+0000 7f000e021700 1 -- 172.21.15.99:0/3653152406 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f000806e3e0 con 0x7f00080afd90 2023-10-08T00:20:08.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.396+0000 7f000d820700 1 --2- 172.21.15.99:0/3653152406 >> v2:172.21.15.191:3300/0 conn(0x7f000806f1a0 0x7f000815b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:20:08.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.396+0000 7f000e021700 1 --2- 172.21.15.99:0/3653152406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00080afd90 0x7f00080c9c20 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f000806d3f0 tx=0x7efff8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0b5c01f75dd1e97 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:08.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.396+0000 7f000d01f700 1 -- 172.21.15.99:0/3653152406 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff800e070 con 0x7f00080afd90 2023-10-08T00:20:08.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.396+0000 7f000d01f700 1 -- 172.21.15.99:0/3653152406 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff800bc30 con 0x7f00080afd90 2023-10-08T00:20:08.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.396+0000 7f000d01f700 1 -- 172.21.15.99:0/3653152406 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff8012430 con 0x7f00080afd90 2023-10-08T00:20:08.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.396+0000 7f0010285700 1 -- 172.21.15.99:0/3653152406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00080afd90 msgr2=0x7f00080c9c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:08.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.396+0000 7f0010285700 1 --2- 172.21.15.99:0/3653152406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00080afd90 0x7f00080c9c20 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f000806d3f0 tx=0x7efff8009780 comp rx=0 tx=0).stop 2023-10-08T00:20:08.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.396+0000 7f0010285700 1 -- 172.21.15.99:0/3653152406 shutdown_connections 2023-10-08T00:20:08.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.396+0000 7f0010285700 1 --2- 172.21.15.99:0/3653152406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00080afd90 0x7f00080c9c20 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:08.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.396+0000 7f0010285700 1 --2- 172.21.15.99:0/3653152406 >> v2:172.21.15.191:3300/0 conn(0x7f000806f1a0 0x7f000815b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:08.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.396+0000 7f0010285700 1 -- 172.21.15.99:0/3653152406 >> 172.21.15.99:0/3653152406 conn(0x7f00080ca4f0 msgr2=0x7f00080ca8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:20:08.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.396+0000 7f0010285700 1 -- 172.21.15.99:0/3653152406 shutdown_connections 2023-10-08T00:20:08.400 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.396+0000 7f0010285700 1 -- 172.21.15.99:0/3653152406 wait complete. 2023-10-08T00:20:08.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.397+0000 7f0010285700 1 Processor -- start 2023-10-08T00:20:08.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.398+0000 7f0010285700 1 -- start start 2023-10-08T00:20:08.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.398+0000 7f0010285700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f000806f1a0 0x7f00080c68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:08.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.398+0000 7f0010285700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00080afd90 0x7f0008156a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:08.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.398+0000 7f0010285700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00080b0160 0x7f00080b2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:08.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.398+0000 7f0010285700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00080c73e0 con 0x7f00080afd90 2023-10-08T00:20:08.403 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.398+0000 7f0010285700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00080b1540 con 0x7f000806f1a0 2023-10-08T00:20:08.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.398+0000 7f0010285700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00080b1dc0 con 0x7f00080b0160 2023-10-08T00:20:08.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.398+0000 7f000d820700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00080afd90 0x7f0008156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:08.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.398+0000 7f000d820700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00080afd90 0x7f0008156a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60874/0 (socket says 172.21.15.99:60874) 2023-10-08T00:20:08.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.398+0000 7f000d820700 1 -- 172.21.15.99:0/396675300 learned_addr learned my addr 172.21.15.99:0/396675300 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:20:08.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.399+0000 7f000e021700 1 --2- 172.21.15.99:0/396675300 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f000806f1a0 0x7f00080c68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:08.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.399+0000 7f000e822700 1 --2- 172.21.15.99:0/396675300 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00080b0160 0x7f00080b2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:08.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.399+0000 7f000d820700 1 -- 172.21.15.99:0/396675300 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00080b0160 msgr2=0x7f00080b2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:08.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.399+0000 7f000d820700 1 --2- 172.21.15.99:0/396675300 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00080b0160 0x7f00080b2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:08.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.399+0000 7f000d820700 1 -- 172.21.15.99:0/396675300 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f000806f1a0 msgr2=0x7f00080c68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:08.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.399+0000 7f000d820700 1 --2- 172.21.15.99:0/396675300 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f000806f1a0 0x7f00080c68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:08.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.399+0000 7f000d820700 1 -- 172.21.15.99:0/396675300 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efff8009a20 con 0x7f00080afd90 2023-10-08T00:20:08.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.399+0000 7f000e021700 1 --2- 172.21.15.99:0/396675300 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f000806f1a0 0x7f00080c68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:20:08.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.399+0000 7f000e822700 1 --2- 172.21.15.99:0/396675300 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00080b0160 0x7f00080b2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:20:08.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.399+0000 7f000d820700 1 --2- 172.21.15.99:0/396675300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00080afd90 0x7f0008156a60 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7efffc00c850 tx=0x7efffc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:08.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.399+0000 7f00067fc700 1 -- 172.21.15.99:0/396675300 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efffc011600 con 0x7f00080afd90 2023-10-08T00:20:08.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.399+0000 7f00067fc700 1 -- 172.21.15.99:0/396675300 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efffc011760 con 0x7f00080afd90 2023-10-08T00:20:08.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.399+0000 7f0010285700 1 -- 172.21.15.99:0/396675300 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00080c7100 con 0x7f00080afd90 2023-10-08T00:20:08.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.399+0000 7f00067fc700 1 -- 172.21.15.99:0/396675300 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efffc011b00 con 0x7f00080afd90 2023-10-08T00:20:08.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.399+0000 7f0010285700 1 -- 172.21.15.99:0/396675300 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00080b1a50 con 0x7f00080afd90 2023-10-08T00:20:08.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.400+0000 7f0010285700 1 -- 172.21.15.99:0/396675300 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00080c6eb0 con 0x7f00080afd90 2023-10-08T00:20:08.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.400+0000 7f00067fc700 1 -- 172.21.15.99:0/396675300 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efffc010350 con 0x7f00080afd90 2023-10-08T00:20:08.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.401+0000 7f00067fc700 1 --2- 172.21.15.99:0/396675300 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efff4066f30 0x7efff40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:08.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.401+0000 7f00067fc700 1 -- 172.21.15.99:0/396675300 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5705+0+0 (secure 0 0 0) 0x7efffc0844f0 con 0x7f00080afd90 2023-10-08T00:20:08.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.401+0000 7f000e021700 1 --2- 172.21.15.99:0/396675300 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efff4066f30 0x7efff40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:08.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.401+0000 7f000e021700 1 --2- 172.21.15.99:0/396675300 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efff4066f30 0x7efff40693e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f000806d3f0 tx=0x7efff8019000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:08.406 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.405+0000 7f00067fc700 1 -- 172.21.15.99:0/396675300 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efffc051760 con 0x7f00080afd90 2023-10-08T00:20:08.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.630+0000 7f0010285700 1 -- 172.21.15.99:0/396675300 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f000804c3c0 con 0x7f00080afd90 2023-10-08T00:20:08.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.630+0000 7f00067fc700 1 -- 172.21.15.99:0/396675300 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4601 (secure 0 0 0) 0x7efffc054d80 con 0x7f00080afd90 2023-10-08T00:20:08.631 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:20:08.631 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":77,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6257,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/661016677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":661016677},{"type":"v1","addr":"172.21.15.191:6837","nonce":661016677}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":7012,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/4102051559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":4102051559},{"type":"v1","addr":"172.21.15.99:6835","nonce":4102051559}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":7021,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/658854389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":658854389},{"type":"v1","addr":"172.21.15.191:6836","nonce":658854389}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:20:04.613994+0000","modified":"2023-10-08T00:20:08.255350+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7791},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7791":{"gid":7791,"name":"c","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6834/3880334948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3880334948},{"type":"v1","addr":"172.21.15.99:6834","nonce":3880334948}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-08T00:20:08.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.632+0000 7f0010285700 1 -- 172.21.15.99:0/396675300 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efff4066f30 msgr2=0x7efff40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:08.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.632+0000 7f0010285700 1 --2- 172.21.15.99:0/396675300 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efff4066f30 0x7efff40693e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f000806d3f0 tx=0x7efff8019000 comp rx=0 tx=0).stop 2023-10-08T00:20:08.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.632+0000 7f0010285700 1 -- 172.21.15.99:0/396675300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00080afd90 msgr2=0x7f0008156a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:08.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.632+0000 7f0010285700 1 --2- 172.21.15.99:0/396675300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00080afd90 0x7f0008156a60 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7efffc00c850 tx=0x7efffc008040 comp rx=0 tx=0).stop 2023-10-08T00:20:08.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.632+0000 7f0010285700 1 -- 172.21.15.99:0/396675300 shutdown_connections 2023-10-08T00:20:08.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.632+0000 7f0010285700 1 --2- 172.21.15.99:0/396675300 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efff4066f30 0x7efff40693e0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:08.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.632+0000 7f0010285700 1 --2- 172.21.15.99:0/396675300 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f000806f1a0 0x7f00080c68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:08.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.632+0000 7f0010285700 1 --2- 172.21.15.99:0/396675300 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00080b0160 0x7f00080b2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:08.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.632+0000 7f0010285700 1 --2- 172.21.15.99:0/396675300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00080afd90 0x7f0008156a60 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:08.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.632+0000 7f0010285700 1 -- 172.21.15.99:0/396675300 >> 172.21.15.99:0/396675300 conn(0x7f00080ca4f0 msgr2=0x7f00081573f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:20:08.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.632+0000 7f0010285700 1 -- 172.21.15.99:0/396675300 shutdown_connections 2023-10-08T00:20:08.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.632+0000 7f0010285700 1 -- 172.21.15.99:0/396675300 wait complete. 2023-10-08T00:20:08.634 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 77 2023-10-08T00:20:08.644 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:20:08.759 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.758+0000 7f5f50518700 1 Processor -- start 2023-10-08T00:20:08.759 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.758+0000 7f5f50518700 1 -- start start 2023-10-08T00:20:08.763 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.759+0000 7f5f50518700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f5f480b0e50 0x7f5f480c9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:08.763 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.759+0000 7f5f50518700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f480b1220 0x7f5f4806d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:08.763 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.759+0000 7f5f50518700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f4815de60 con 0x7f5f4806dad0 2023-10-08T00:20:08.763 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.759+0000 7f5f50518700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f4815e140 con 0x7f5f480b0e50 2023-10-08T00:20:08.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.759+0000 7f5f50518700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4815e420 con 0x7f5f480b1220 2023-10-08T00:20:08.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.760+0000 7f5f4eab5700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f5f4806dad0 0x7f5f480cb4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56096/0 (socket says 172.21.15.99:56096) 2023-10-08T00:20:08.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.760+0000 7f5f4eab5700 1 -- 172.21.15.99:0/1099052791 learned_addr learned my addr 172.21.15.99:0/1099052791 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:20:08.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.760+0000 7f5f4dab3700 1 --2- 172.21.15.99:0/1099052791 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f480b1220 0x7f5f4806d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:08.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.760+0000 7f5f4e2b4700 1 --2- 172.21.15.99:0/1099052791 >> v2:172.21.15.191:3300/0 conn(0x7f5f480b0e50 0x7f5f480c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:08.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.760+0000 7f5f4d2b2700 1 -- 172.21.15.99:0/1099052791 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2140944972 0 0) 0x7f5f4815de60 con 0x7f5f4806dad0 2023-10-08T00:20:08.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.760+0000 7f5f4d2b2700 1 -- 172.21.15.99:0/1099052791 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f34003570 con 0x7f5f4806dad0 2023-10-08T00:20:08.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.760+0000 7f5f4d2b2700 1 -- 172.21.15.99:0/1099052791 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4203916969 0 0) 0x7f5f34003570 con 0x7f5f4806dad0 2023-10-08T00:20:08.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.760+0000 7f5f4d2b2700 1 -- 172.21.15.99:0/1099052791 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f480b1220 msgr2=0x7f5f4806d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:08.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.760+0000 7f5f4d2b2700 1 --2- 172.21.15.99:0/1099052791 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f480b1220 0x7f5f4806d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:08.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.760+0000 7f5f4d2b2700 1 -- 172.21.15.99:0/1099052791 >> v2:172.21.15.191:3300/0 conn(0x7f5f480b0e50 msgr2=0x7f5f480c9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:08.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.760+0000 7f5f4d2b2700 1 --2- 172.21.15.99:0/1099052791 >> v2:172.21.15.191:3300/0 conn(0x7f5f480b0e50 0x7f5f480c9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:08.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.760+0000 7f5f4d2b2700 1 -- 172.21.15.99:0/1099052791 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f4815e700 con 0x7f5f4806dad0 2023-10-08T00:20:08.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.760+0000 7f5f4d2b2700 1 -- 172.21.15.99:0/1099052791 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f5f40001470 con 0x7f5f4806dad0 2023-10-08T00:20:08.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.760+0000 7f5f4d2b2700 1 -- 172.21.15.99:0/1099052791 >> v1:172.21.15.99:6789/0 conn(0x7f5f4806dad0 legacy=0x7f5f480cb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:08.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.760+0000 7f5f4d2b2700 1 --2- 172.21.15.99:0/1099052791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f34003a60 0x7f5f34003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:08.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.760+0000 7f5f4d2b2700 1 --2- 172.21.15.99:0/1099052791 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f34004810 0x7f5f34004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:08.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.760+0000 7f5f4d2b2700 1 --2- 172.21.15.99:0/1099052791 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f340051c0 0x7f5f3400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:08.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.760+0000 7f5f4d2b2700 1 -- 172.21.15.99:0/1099052791 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4815e140 con 0x7f5f34003a60 2023-10-08T00:20:08.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.760+0000 7f5f4d2b2700 1 -- 172.21.15.99:0/1099052791 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4815e420 con 0x7f5f340051c0 2023-10-08T00:20:08.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.760+0000 7f5f4d2b2700 1 -- 172.21.15.99:0/1099052791 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3400c050 con 0x7f5f34004810 2023-10-08T00:20:08.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.760+0000 7f5f4dab3700 1 --2- 172.21.15.99:0/1099052791 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f480b1220 0x7f5f4806d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:20:08.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.760+0000 7f5f4e2b4700 1 --2- 172.21.15.99:0/1099052791 >> v2:172.21.15.191:3300/0 conn(0x7f5f480b0e50 0x7f5f480c9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:20:08.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.760+0000 7f5f4e2b4700 1 --2- 172.21.15.99:0/1099052791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f34003a60 0x7f5f34003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:08.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.760+0000 7f5f4eab5700 1 --2- 172.21.15.99:0/1099052791 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f340051c0 0x7f5f3400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:08.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.760+0000 7f5f4dab3700 1 --2- 172.21.15.99:0/1099052791 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f34004810 0x7f5f34004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:08.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.761+0000 7f5f4e2b4700 1 -- 172.21.15.99:0/1099052791 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f34004810 msgr2=0x7f5f34004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:08.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.761+0000 7f5f4e2b4700 1 --2- 172.21.15.99:0/1099052791 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f34004810 0x7f5f34004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:08.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.761+0000 7f5f4e2b4700 1 -- 172.21.15.99:0/1099052791 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f340051c0 msgr2=0x7f5f3400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:08.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.761+0000 7f5f4e2b4700 1 --2- 172.21.15.99:0/1099052791 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f340051c0 0x7f5f3400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:08.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.761+0000 7f5f4e2b4700 1 -- 172.21.15.99:0/1099052791 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f34003570 con 0x7f5f34003a60 2023-10-08T00:20:08.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.761+0000 7f5f4dab3700 1 --2- 172.21.15.99:0/1099052791 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f34004810 0x7f5f34004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:20:08.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.761+0000 7f5f4e2b4700 1 --2- 172.21.15.99:0/1099052791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f34003a60 0x7f5f34003e70 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f5f4400e3c0 tx=0x7f5f4400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9866b1d69aee11d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:08.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.761+0000 7f5f4d2b2700 1 -- 172.21.15.99:0/1099052791 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4400ec70 con 0x7f5f34003a60 2023-10-08T00:20:08.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.761+0000 7f5f4d2b2700 1 -- 172.21.15.99:0/1099052791 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f4400f040 con 0x7f5f34003a60 2023-10-08T00:20:08.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.761+0000 7f5f4d2b2700 1 -- 172.21.15.99:0/1099052791 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f44014480 con 0x7f5f34003a60 2023-10-08T00:20:08.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.761+0000 7f5f50518700 1 -- 172.21.15.99:0/1099052791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f34003a60 msgr2=0x7f5f34003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:08.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.761+0000 7f5f50518700 1 --2- 172.21.15.99:0/1099052791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f34003a60 0x7f5f34003e70 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f5f4400e3c0 tx=0x7f5f4400b040 comp rx=0 tx=0).stop 2023-10-08T00:20:08.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.761+0000 7f5f4eab5700 1 -- 172.21.15.99:0/1099052791 reap_dead start 2023-10-08T00:20:08.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.762+0000 7f5f50518700 1 -- 172.21.15.99:0/1099052791 shutdown_connections 2023-10-08T00:20:08.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.762+0000 7f5f50518700 1 -- 172.21.15.99:0/1099052791 >> 172.21.15.99:0/1099052791 conn(0x7f5f48069870 msgr2=0x7f5f48069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:20:08.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.762+0000 7f5f50518700 1 -- 172.21.15.99:0/1099052791 shutdown_connections 2023-10-08T00:20:08.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.762+0000 7f5f50518700 1 -- 172.21.15.99:0/1099052791 wait complete. 2023-10-08T00:20:08.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.763+0000 7f5f50518700 1 Processor -- start 2023-10-08T00:20:08.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.763+0000 7f5f50518700 1 -- start start 2023-10-08T00:20:08.770 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.763+0000 7f5f50518700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f34003a60 0x7f5f4815ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:08.770 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.763+0000 7f5f50518700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f48155990 0x7f5f480b2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:08.770 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.763+0000 7f5f50518700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f4815c470 0x7f5f4815c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:08.770 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.763+0000 7f5f50518700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4815ce20 con 0x7f5f34003a60 2023-10-08T00:20:08.770 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.763+0000 7f5f50518700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f480b2690 con 0x7f5f4815c470 2023-10-08T00:20:08.770 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.763+0000 7f5f50518700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f480b1a80 con 0x7f5f48155990 2023-10-08T00:20:08.770 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.763+0000 7f5f4e2b4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f34003a60 0x7f5f4815ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:08.770 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.763+0000 7f5f4e2b4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f34003a60 0x7f5f4815ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60890/0 (socket says 172.21.15.99:60890) 2023-10-08T00:20:08.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.763+0000 7f5f4e2b4700 1 -- 172.21.15.99:0/1448329557 learned_addr learned my addr 172.21.15.99:0/1448329557 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:20:08.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.764+0000 7f5f4eab5700 1 --2- 172.21.15.99:0/1448329557 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f4815c470 0x7f5f4815c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:08.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.764+0000 7f5f4dab3700 1 --2- 172.21.15.99:0/1448329557 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f48155990 0x7f5f480b2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:08.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.764+0000 7f5f4e2b4700 1 -- 172.21.15.99:0/1448329557 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f48155990 msgr2=0x7f5f480b2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:08.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.764+0000 7f5f4e2b4700 1 --2- 172.21.15.99:0/1448329557 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f48155990 0x7f5f480b2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:08.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.764+0000 7f5f4e2b4700 1 -- 172.21.15.99:0/1448329557 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f4815c470 msgr2=0x7f5f4815c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:08.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.764+0000 7f5f4e2b4700 1 --2- 172.21.15.99:0/1448329557 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f4815c470 0x7f5f4815c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:08.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.764+0000 7f5f4e2b4700 1 -- 172.21.15.99:0/1448329557 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f480b1d60 con 0x7f5f34003a60 2023-10-08T00:20:08.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.764+0000 7f5f4e2b4700 1 --2- 172.21.15.99:0/1448329557 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f34003a60 0x7f5f4815ae10 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f5f440094a0 tx=0x7f5f44019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:08.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.764+0000 7f5f3e7fc700 1 -- 172.21.15.99:0/1448329557 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4401f6e0 con 0x7f5f34003a60 2023-10-08T00:20:08.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.764+0000 7f5f3e7fc700 1 -- 172.21.15.99:0/1448329557 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f4401f840 con 0x7f5f34003a60 2023-10-08T00:20:08.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.764+0000 7f5f50518700 1 -- 172.21.15.99:0/1448329557 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f480b2040 con 0x7f5f34003a60 2023-10-08T00:20:08.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.764+0000 7f5f3e7fc700 1 -- 172.21.15.99:0/1448329557 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4401fbe0 con 0x7f5f34003a60 2023-10-08T00:20:08.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.764+0000 7f5f50518700 1 -- 172.21.15.99:0/1448329557 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f4815b350 con 0x7f5f34003a60 2023-10-08T00:20:08.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.765+0000 7f5f50518700 1 -- 172.21.15.99:0/1448329557 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f480b2970 con 0x7f5f34003a60 2023-10-08T00:20:08.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.765+0000 7f5f3e7fc700 1 -- 172.21.15.99:0/1448329557 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5f4401d440 con 0x7f5f34003a60 2023-10-08T00:20:08.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.766+0000 7f5f3e7fc700 1 --2- 172.21.15.99:0/1448329557 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5f2c066f30 0x7f5f2c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:08.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.766+0000 7f5f3e7fc700 1 -- 172.21.15.99:0/1448329557 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5f44087030 con 0x7f5f34003a60 2023-10-08T00:20:08.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.766+0000 7f5f4dab3700 1 --2- 172.21.15.99:0/1448329557 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5f2c066f30 0x7f5f2c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:08.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.766+0000 7f5f4dab3700 1 --2- 172.21.15.99:0/1448329557 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5f2c066f30 0x7f5f2c0693e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f5f380093f0 tx=0x7f5f38008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:08.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.770+0000 7f5f3e7fc700 1 -- 172.21.15.99:0/1448329557 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5f440542a0 con 0x7f5f34003a60 2023-10-08T00:20:08.953 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.952+0000 7f5f50518700 1 -- 172.21.15.99:0/1448329557 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5f4815d9e0 con 0x7f5f34003a60 2023-10-08T00:20:08.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.953+0000 7f5f3e7fc700 1 -- 172.21.15.99:0/1448329557 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v168) v1 ==== 74+0+15365 (secure 0 0 0) 0x7f5f440578c0 con 0x7f5f34003a60 2023-10-08T00:20:08.954 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:20:08.954 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":168,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:20:06.235453+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:20:02.932463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-08T00:20:03.578010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:20:08.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.955+0000 7f5f50518700 1 -- 172.21.15.99:0/1448329557 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5f2c066f30 msgr2=0x7f5f2c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:08.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.955+0000 7f5f50518700 1 --2- 172.21.15.99:0/1448329557 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5f2c066f30 0x7f5f2c0693e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f5f380093f0 tx=0x7f5f38008040 comp rx=0 tx=0).stop 2023-10-08T00:20:08.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.955+0000 7f5f50518700 1 -- 172.21.15.99:0/1448329557 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f34003a60 msgr2=0x7f5f4815ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:08.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.955+0000 7f5f50518700 1 --2- 172.21.15.99:0/1448329557 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f34003a60 0x7f5f4815ae10 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f5f440094a0 tx=0x7f5f44019000 comp rx=0 tx=0).stop 2023-10-08T00:20:08.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.955+0000 7f5f50518700 1 -- 172.21.15.99:0/1448329557 shutdown_connections 2023-10-08T00:20:08.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.955+0000 7f5f50518700 1 --2- 172.21.15.99:0/1448329557 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5f2c066f30 0x7f5f2c0693e0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:08.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.955+0000 7f5f50518700 1 --2- 172.21.15.99:0/1448329557 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f4815c470 0x7f5f4815c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:08.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.955+0000 7f5f50518700 1 --2- 172.21.15.99:0/1448329557 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f34003a60 0x7f5f4815ae10 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:08.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.955+0000 7f5f50518700 1 --2- 172.21.15.99:0/1448329557 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f48155990 0x7f5f480b2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:08.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.955+0000 7f5f50518700 1 -- 172.21.15.99:0/1448329557 >> 172.21.15.99:0/1448329557 conn(0x7f5f48069870 msgr2=0x7f5f480c3af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:20:08.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.955+0000 7f5f50518700 1 -- 172.21.15.99:0/1448329557 shutdown_connections 2023-10-08T00:20:08.957 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:08.955+0000 7f5f50518700 1 -- 172.21.15.99:0/1448329557 wait complete. 2023-10-08T00:20:08.968 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T00:20:08.968 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T00:20:09.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.082+0000 7f0bb5407700 1 Processor -- start 2023-10-08T00:20:09.083 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.082+0000 7f0bb5407700 1 -- start start 2023-10-08T00:20:09.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.083+0000 7f0bb5407700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0bb00cd6e0 0x7f0bb00cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:09.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.083+0000 7f0bb5407700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0bb00cdab0 0x7f0bb015d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:09.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.083+0000 7f0bb5407700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bb00cb7c0 con 0x7f0bb015dda0 2023-10-08T00:20:09.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.083+0000 7f0bb5407700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bb015e850 con 0x7f0bb00cdab0 2023-10-08T00:20:09.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.083+0000 7f0bb5407700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb00cd030 con 0x7f0bb00cd6e0 2023-10-08T00:20:09.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.083+0000 7f0baf7fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0bb00cdab0 0x7f0bb015d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:09.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.083+0000 7f0bb4c06700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f0bb015dda0 0x7f0bb015cbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56098/0 (socket says 172.21.15.99:56098) 2023-10-08T00:20:09.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.083+0000 7f0baf7fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0bb00cdab0 0x7f0bb015d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60904/0 (socket says 172.21.15.99:60904) 2023-10-08T00:20:09.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.083+0000 7f0bb4c06700 1 -- 172.21.15.99:0/1304600616 learned_addr learned my addr 172.21.15.99:0/1304600616 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:20:09.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.083+0000 7f0baffff700 1 --2- 172.21.15.99:0/1304600616 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0bb00cd6e0 0x7f0bb00cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:09.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.083+0000 7f0baf7fe700 1 -- 172.21.15.99:0/1304600616 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0bb00cd6e0 msgr2=0x7f0bb00cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:09.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.083+0000 7f0baf7fe700 1 --2- 172.21.15.99:0/1304600616 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0bb00cd6e0 0x7f0bb00cbc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:09.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.083+0000 7f0baf7fe700 1 -- 172.21.15.99:0/1304600616 >> v1:172.21.15.99:6789/0 conn(0x7f0bb015dda0 legacy=0x7f0bb015cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:09.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.083+0000 7f0baf7fe700 1 -- 172.21.15.99:0/1304600616 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb015e290 con 0x7f0bb00cdab0 2023-10-08T00:20:09.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.083+0000 7f0baeffd700 1 -- 172.21.15.99:0/1304600616 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1667365752 0 0) 0x7f0bb00cb7c0 con 0x7f0bb015dda0 2023-10-08T00:20:09.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.084+0000 7f0baf7fe700 1 --2- 172.21.15.99:0/1304600616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bb00cdab0 0x7f0bb015d860 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f0ba40098e0 tx=0x7f0ba40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d307dabb853695a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:09.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.084+0000 7f0baeffd700 1 -- 172.21.15.99:0/1304600616 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba4009e80 con 0x7f0bb00cdab0 2023-10-08T00:20:09.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.084+0000 7f0baeffd700 1 -- 172.21.15.99:0/1304600616 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba400b860 con 0x7f0bb00cdab0 2023-10-08T00:20:09.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.084+0000 7f0baeffd700 1 -- 172.21.15.99:0/1304600616 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba400bbe0 con 0x7f0bb00cdab0 2023-10-08T00:20:09.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.084+0000 7f0bb5407700 1 -- 172.21.15.99:0/1304600616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bb00cdab0 msgr2=0x7f0bb015d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:09.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.084+0000 7f0bb5407700 1 --2- 172.21.15.99:0/1304600616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bb00cdab0 0x7f0bb015d860 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f0ba40098e0 tx=0x7f0ba40091a0 comp rx=0 tx=0).stop 2023-10-08T00:20:09.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.084+0000 7f0bb5407700 1 -- 172.21.15.99:0/1304600616 shutdown_connections 2023-10-08T00:20:09.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.084+0000 7f0bb5407700 1 --2- 172.21.15.99:0/1304600616 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0bb00cd6e0 0x7f0bb00cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:09.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.084+0000 7f0bb5407700 1 --2- 172.21.15.99:0/1304600616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bb00cdab0 0x7f0bb015d860 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:09.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.084+0000 7f0bb5407700 1 -- 172.21.15.99:0/1304600616 >> 172.21.15.99:0/1304600616 conn(0x7f0bb0067660 msgr2=0x7f0bb0067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:20:09.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.084+0000 7f0bb5407700 1 -- 172.21.15.99:0/1304600616 shutdown_connections 2023-10-08T00:20:09.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.084+0000 7f0bb5407700 1 -- 172.21.15.99:0/1304600616 wait complete. 2023-10-08T00:20:09.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.085+0000 7f0bb5407700 1 Processor -- start 2023-10-08T00:20:09.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.086+0000 7f0bb5407700 1 -- start start 2023-10-08T00:20:09.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.086+0000 7f0bb5407700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bb00cd6e0 0x7f0bb014ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:09.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.086+0000 7f0bb5407700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0bb00cdab0 0x7f0bb00c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:09.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.086+0000 7f0bb5407700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0bb015dda0 0x7f0bb00c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:09.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.086+0000 7f0bb5407700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb00b2930 con 0x7f0bb00cd6e0 2023-10-08T00:20:09.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.086+0000 7f0bb5407700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb014d3c0 con 0x7f0bb015dda0 2023-10-08T00:20:09.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.086+0000 7f0bb5407700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb016de60 con 0x7f0bb00cdab0 2023-10-08T00:20:09.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.086+0000 7f0baffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bb00cd6e0 0x7f0bb014ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:09.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.086+0000 7f0baffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bb00cd6e0 0x7f0bb014ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60920/0 (socket says 172.21.15.99:60920) 2023-10-08T00:20:09.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.086+0000 7f0baffff700 1 -- 172.21.15.99:0/3123383952 learned_addr learned my addr 172.21.15.99:0/3123383952 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:20:09.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.087+0000 7f0bb4c06700 1 --2- 172.21.15.99:0/3123383952 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0bb015dda0 0x7f0bb00c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:09.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.087+0000 7f0baf7fe700 1 --2- 172.21.15.99:0/3123383952 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0bb00cdab0 0x7f0bb00c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:09.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.087+0000 7f0baffff700 1 -- 172.21.15.99:0/3123383952 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0bb00cdab0 msgr2=0x7f0bb00c7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:09.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.087+0000 7f0baffff700 1 --2- 172.21.15.99:0/3123383952 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0bb00cdab0 0x7f0bb00c7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:09.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.087+0000 7f0baffff700 1 -- 172.21.15.99:0/3123383952 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0bb015dda0 msgr2=0x7f0bb00c8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:09.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.087+0000 7f0baffff700 1 --2- 172.21.15.99:0/3123383952 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0bb015dda0 0x7f0bb00c8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:09.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.087+0000 7f0baffff700 1 -- 172.21.15.99:0/3123383952 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba4009420 con 0x7f0bb00cd6e0 2023-10-08T00:20:09.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.087+0000 7f0baffff700 1 --2- 172.21.15.99:0/3123383952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bb00cd6e0 0x7f0bb014ce80 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f0ba000c880 tx=0x7f0ba0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:09.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.087+0000 7f0b97fff700 1 -- 172.21.15.99:0/3123383952 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba0011600 con 0x7f0bb00cd6e0 2023-10-08T00:20:09.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.087+0000 7f0b97fff700 1 -- 172.21.15.99:0/3123383952 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba0011780 con 0x7f0bb00cd6e0 2023-10-08T00:20:09.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.087+0000 7f0bb5407700 1 -- 172.21.15.99:0/3123383952 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bb016e140 con 0x7f0bb00cd6e0 2023-10-08T00:20:09.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.087+0000 7f0b97fff700 1 -- 172.21.15.99:0/3123383952 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba0011b40 con 0x7f0bb00cd6e0 2023-10-08T00:20:09.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.087+0000 7f0bb5407700 1 -- 172.21.15.99:0/3123383952 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bb016e700 con 0x7f0bb00cd6e0 2023-10-08T00:20:09.092 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.088+0000 7f0bb5407700 1 -- 172.21.15.99:0/3123383952 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bb0089280 con 0x7f0bb00cd6e0 2023-10-08T00:20:09.093 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.088+0000 7f0b97fff700 1 -- 172.21.15.99:0/3123383952 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0ba0018360 con 0x7f0bb00cd6e0 2023-10-08T00:20:09.093 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.089+0000 7f0b97fff700 1 --2- 172.21.15.99:0/3123383952 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0b9c066f60 0x7f0b9c069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:09.093 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.089+0000 7f0b97fff700 1 -- 172.21.15.99:0/3123383952 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0ba0084210 con 0x7f0bb00cd6e0 2023-10-08T00:20:09.093 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.089+0000 7f0baf7fe700 1 --2- 172.21.15.99:0/3123383952 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0b9c066f60 0x7f0b9c069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:09.093 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.089+0000 7f0baf7fe700 1 --2- 172.21.15.99:0/3123383952 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0b9c066f60 0x7f0b9c069410 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f0ba40091e0 tx=0x7f0ba4017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:09.093 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.093+0000 7f0b97fff700 1 -- 172.21.15.99:0/3123383952 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0ba0054f10 con 0x7f0bb00cd6e0 2023-10-08T00:20:09.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.272+0000 7f0bb5407700 1 -- 172.21.15.99:0/3123383952 --> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0bb015e570 con 0x7f0b9c066f60 2023-10-08T00:20:09.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.281+0000 7f0b97fff700 1 -- 172.21.15.99:0/3123383952 <== mgr.4099 v2:172.21.15.191:6832/85985 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285556 (secure 0 0 0) 0x7f0bb015e570 con 0x7f0b9c066f60 2023-10-08T00:20:09.288 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:20:09.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.285+0000 7f0bb5407700 1 -- 172.21.15.99:0/3123383952 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0b9c066f60 msgr2=0x7f0b9c069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:09.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.285+0000 7f0bb5407700 1 --2- 172.21.15.99:0/3123383952 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0b9c066f60 0x7f0b9c069410 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f0ba40091e0 tx=0x7f0ba4017000 comp rx=0 tx=0).stop 2023-10-08T00:20:09.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.285+0000 7f0bb5407700 1 -- 172.21.15.99:0/3123383952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bb00cd6e0 msgr2=0x7f0bb014ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:09.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.285+0000 7f0bb5407700 1 --2- 172.21.15.99:0/3123383952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bb00cd6e0 0x7f0bb014ce80 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f0ba000c880 tx=0x7f0ba0008040 comp rx=0 tx=0).stop 2023-10-08T00:20:09.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.285+0000 7f0bb5407700 1 -- 172.21.15.99:0/3123383952 shutdown_connections 2023-10-08T00:20:09.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.285+0000 7f0bb5407700 1 --2- 172.21.15.99:0/3123383952 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0b9c066f60 0x7f0b9c069410 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:09.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.285+0000 7f0bb5407700 1 --2- 172.21.15.99:0/3123383952 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0bb015dda0 0x7f0bb00c8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:09.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.285+0000 7f0bb5407700 1 --2- 172.21.15.99:0/3123383952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bb00cd6e0 0x7f0bb014ce80 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:09.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.285+0000 7f0bb5407700 1 --2- 172.21.15.99:0/3123383952 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0bb00cdab0 0x7f0bb00c7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:09.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.285+0000 7f0bb5407700 1 -- 172.21.15.99:0/3123383952 >> 172.21.15.99:0/3123383952 conn(0x7f0bb0067660 msgr2=0x7f0bb00d1620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:20:09.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.285+0000 7f0bb5407700 1 -- 172.21.15.99:0/3123383952 shutdown_connections 2023-10-08T00:20:09.294 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.285+0000 7f0bb5407700 1 -- 172.21.15.99:0/3123383952 wait complete. 2023-10-08T00:20:09.294 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2023-10-08T00:20:09.298 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":441,"stamp":"2023-10-08T00:20:08.156351+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":161,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":8497,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1554002,"statfs":{"total":1600000000,"available":1591298236,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.920357"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.251793+0000","last_change":"2023-10-08T00:20:05.249821+0000","last_active":"2023-10-08T00:20:05.251793+0000","last_peered":"2023-10-08T00:20:05.251793+0000","last_clean":"2023-10-08T00:20:05.251793+0000","last_became_active":"2023-10-08T00:20:05.249607+0000","last_became_peered":"2023-10-08T00:20:05.249607+0000","last_unstale":"2023-10-08T00:20:05.251793+0000","last_undegraded":"2023-10-08T00:20:05.251793+0000","last_fullsized":"2023-10-08T00:20:05.251793+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:54:14.710613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.246458+0000","last_change":"2023-10-08T00:20:05.246248+0000","last_active":"2023-10-08T00:20:05.246458+0000","last_peered":"2023-10-08T00:20:05.246458+0000","last_clean":"2023-10-08T00:20:05.246458+0000","last_became_active":"2023-10-08T00:20:05.245992+0000","last_became_peered":"2023-10-08T00:20:05.245992+0000","last_unstale":"2023-10-08T00:20:05.246458+0000","last_undegraded":"2023-10-08T00:20:05.246458+0000","last_fullsized":"2023-10-08T00:20:05.246458+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:24:49.076952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.245864+0000","last_change":"2023-10-08T00:20:05.245791+0000","last_active":"2023-10-08T00:20:05.245864+0000","last_peered":"2023-10-08T00:20:05.245864+0000","last_clean":"2023-10-08T00:20:05.245864+0000","last_became_active":"2023-10-08T00:20:05.245250+0000","last_became_peered":"2023-10-08T00:20:05.245250+0000","last_unstale":"2023-10-08T00:20:05.245864+0000","last_undegraded":"2023-10-08T00:20:05.245864+0000","last_fullsized":"2023-10-08T00:20:05.245864+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:43:26.454384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.247225+0000","last_change":"2023-10-08T00:20:05.245007+0000","last_active":"2023-10-08T00:20:05.247225+0000","last_peered":"2023-10-08T00:20:05.247225+0000","last_clean":"2023-10-08T00:20:05.247225+0000","last_became_active":"2023-10-08T00:20:05.244000+0000","last_became_peered":"2023-10-08T00:20:05.244000+0000","last_unstale":"2023-10-08T00:20:05.247225+0000","last_undegraded":"2023-10-08T00:20:05.247225+0000","last_fullsized":"2023-10-08T00:20:05.247225+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:18:31.037964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.243605+0000","last_change":"2023-10-08T00:20:05.243520+0000","last_active":"2023-10-08T00:20:05.243605+0000","last_peered":"2023-10-08T00:20:05.243605+0000","last_clean":"2023-10-08T00:20:05.243605+0000","last_became_active":"2023-10-08T00:20:05.243299+0000","last_became_peered":"2023-10-08T00:20:05.243299+0000","last_unstale":"2023-10-08T00:20:05.243605+0000","last_undegraded":"2023-10-08T00:20:05.243605+0000","last_fullsized":"2023-10-08T00:20:05.243605+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:51:39.795897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.249393+0000","last_change":"2023-10-08T00:20:05.249299+0000","last_active":"2023-10-08T00:20:05.249393+0000","last_peered":"2023-10-08T00:20:05.249393+0000","last_clean":"2023-10-08T00:20:05.249393+0000","last_became_active":"2023-10-08T00:20:05.249074+0000","last_became_peered":"2023-10-08T00:20:05.249074+0000","last_unstale":"2023-10-08T00:20:05.249393+0000","last_undegraded":"2023-10-08T00:20:05.249393+0000","last_fullsized":"2023-10-08T00:20:05.249393+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:49:32.653899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.242476+0000","last_change":"2023-10-08T00:20:05.241833+0000","last_active":"2023-10-08T00:20:05.242476+0000","last_peered":"2023-10-08T00:20:05.242476+0000","last_clean":"2023-10-08T00:20:05.242476+0000","last_became_active":"2023-10-08T00:20:05.241605+0000","last_became_peered":"2023-10-08T00:20:05.241605+0000","last_unstale":"2023-10-08T00:20:05.242476+0000","last_undegraded":"2023-10-08T00:20:05.242476+0000","last_fullsized":"2023-10-08T00:20:05.242476+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:52:47.931876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.254471+0000","last_change":"2023-10-08T00:20:05.252822+0000","last_active":"2023-10-08T00:20:05.254471+0000","last_peered":"2023-10-08T00:20:05.254471+0000","last_clean":"2023-10-08T00:20:05.254471+0000","last_became_active":"2023-10-08T00:20:05.252594+0000","last_became_peered":"2023-10-08T00:20:05.252594+0000","last_unstale":"2023-10-08T00:20:05.254471+0000","last_undegraded":"2023-10-08T00:20:05.254471+0000","last_fullsized":"2023-10-08T00:20:05.254471+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:49:32.653899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.245665+0000","last_change":"2023-10-08T00:20:05.245584+0000","last_active":"2023-10-08T00:20:05.245665+0000","last_peered":"2023-10-08T00:20:05.245665+0000","last_clean":"2023-10-08T00:20:05.245665+0000","last_became_active":"2023-10-08T00:20:05.245322+0000","last_became_peered":"2023-10-08T00:20:05.245322+0000","last_unstale":"2023-10-08T00:20:05.245665+0000","last_undegraded":"2023-10-08T00:20:05.245665+0000","last_fullsized":"2023-10-08T00:20:05.245665+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:34:06.226802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.258231+0000","last_change":"2023-10-08T00:20:05.258154+0000","last_active":"2023-10-08T00:20:05.258231+0000","last_peered":"2023-10-08T00:20:05.258231+0000","last_clean":"2023-10-08T00:20:05.258231+0000","last_became_active":"2023-10-08T00:20:05.257941+0000","last_became_peered":"2023-10-08T00:20:05.257941+0000","last_unstale":"2023-10-08T00:20:05.258231+0000","last_undegraded":"2023-10-08T00:20:05.258231+0000","last_fullsized":"2023-10-08T00:20:05.258231+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:22:41.037733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.246728+0000","last_change":"2023-10-08T00:20:05.245663+0000","last_active":"2023-10-08T00:20:05.246728+0000","last_peered":"2023-10-08T00:20:05.246728+0000","last_clean":"2023-10-08T00:20:05.246728+0000","last_became_active":"2023-10-08T00:20:05.245433+0000","last_became_peered":"2023-10-08T00:20:05.245433+0000","last_unstale":"2023-10-08T00:20:05.246728+0000","last_undegraded":"2023-10-08T00:20:05.246728+0000","last_fullsized":"2023-10-08T00:20:05.246728+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:55:15.548941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.259588+0000","last_change":"2023-10-08T00:20:05.259500+0000","last_active":"2023-10-08T00:20:05.259588+0000","last_peered":"2023-10-08T00:20:05.259588+0000","last_clean":"2023-10-08T00:20:05.259588+0000","last_became_active":"2023-10-08T00:20:05.259056+0000","last_became_peered":"2023-10-08T00:20:05.259056+0000","last_unstale":"2023-10-08T00:20:05.259588+0000","last_undegraded":"2023-10-08T00:20:05.259588+0000","last_fullsized":"2023-10-08T00:20:05.259588+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:44:29.905006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.254911+0000","last_change":"2023-10-08T00:20:05.250458+0000","last_active":"2023-10-08T00:20:05.254911+0000","last_peered":"2023-10-08T00:20:05.254911+0000","last_clean":"2023-10-08T00:20:05.254911+0000","last_became_active":"2023-10-08T00:20:05.249644+0000","last_became_peered":"2023-10-08T00:20:05.249644+0000","last_unstale":"2023-10-08T00:20:05.254911+0000","last_undegraded":"2023-10-08T00:20:05.254911+0000","last_fullsized":"2023-10-08T00:20:05.254911+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:45:25.075955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.250931+0000","last_change":"2023-10-08T00:20:05.250870+0000","last_active":"2023-10-08T00:20:05.250931+0000","last_peered":"2023-10-08T00:20:05.250931+0000","last_clean":"2023-10-08T00:20:05.250931+0000","last_became_active":"2023-10-08T00:20:05.249895+0000","last_became_peered":"2023-10-08T00:20:05.249895+0000","last_unstale":"2023-10-08T00:20:05.250931+0000","last_undegraded":"2023-10-08T00:20:05.250931+0000","last_fullsized":"2023-10-08T00:20:05.250931+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:49:32.653899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.254949+0000","last_change":"2023-10-08T00:20:05.251645+0000","last_active":"2023-10-08T00:20:05.254949+0000","last_peered":"2023-10-08T00:20:05.254949+0000","last_clean":"2023-10-08T00:20:05.254949+0000","last_became_active":"2023-10-08T00:20:05.251016+0000","last_became_peered":"2023-10-08T00:20:05.251016+0000","last_unstale":"2023-10-08T00:20:05.254949+0000","last_undegraded":"2023-10-08T00:20:05.254949+0000","last_fullsized":"2023-10-08T00:20:05.254949+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:59:01.329001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.242444+0000","last_change":"2023-10-08T00:20:05.241158+0000","last_active":"2023-10-08T00:20:05.242444+0000","last_peered":"2023-10-08T00:20:05.242444+0000","last_clean":"2023-10-08T00:20:05.242444+0000","last_became_active":"2023-10-08T00:20:05.240931+0000","last_became_peered":"2023-10-08T00:20:05.240931+0000","last_unstale":"2023-10-08T00:20:05.242444+0000","last_undegraded":"2023-10-08T00:20:05.242444+0000","last_fullsized":"2023-10-08T00:20:05.242444+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:00:59.451262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.257677+0000","last_change":"2023-10-08T00:20:05.257391+0000","last_active":"2023-10-08T00:20:05.257677+0000","last_peered":"2023-10-08T00:20:05.257677+0000","last_clean":"2023-10-08T00:20:05.257677+0000","last_became_active":"2023-10-08T00:20:05.256940+0000","last_became_peered":"2023-10-08T00:20:05.256940+0000","last_unstale":"2023-10-08T00:20:05.257677+0000","last_undegraded":"2023-10-08T00:20:05.257677+0000","last_fullsized":"2023-10-08T00:20:05.257677+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:52:38.144789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.254875+0000","last_change":"2023-10-08T00:20:05.249866+0000","last_active":"2023-10-08T00:20:05.254875+0000","last_peered":"2023-10-08T00:20:05.254875+0000","last_clean":"2023-10-08T00:20:05.254875+0000","last_became_active":"2023-10-08T00:20:05.249355+0000","last_became_peered":"2023-10-08T00:20:05.249355+0000","last_unstale":"2023-10-08T00:20:05.254875+0000","last_undegraded":"2023-10-08T00:20:05.254875+0000","last_fullsized":"2023-10-08T00:20:05.254875+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:23:44.978067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.246765+0000","last_change":"2023-10-08T00:20:05.246656+0000","last_active":"2023-10-08T00:20:05.246765+0000","last_peered":"2023-10-08T00:20:05.246765+0000","last_clean":"2023-10-08T00:20:05.246765+0000","last_became_active":"2023-10-08T00:20:05.246029+0000","last_became_peered":"2023-10-08T00:20:05.246029+0000","last_unstale":"2023-10-08T00:20:05.246765+0000","last_undegraded":"2023-10-08T00:20:05.246765+0000","last_fullsized":"2023-10-08T00:20:05.246765+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:18:31.037964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.247582+0000","last_change":"2023-10-08T00:20:05.247514+0000","last_active":"2023-10-08T00:20:05.247582+0000","last_peered":"2023-10-08T00:20:05.247582+0000","last_clean":"2023-10-08T00:20:05.247582+0000","last_became_active":"2023-10-08T00:20:05.246966+0000","last_became_peered":"2023-10-08T00:20:05.246966+0000","last_unstale":"2023-10-08T00:20:05.247582+0000","last_undegraded":"2023-10-08T00:20:05.247582+0000","last_fullsized":"2023-10-08T00:20:05.247582+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:55:15.548941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.243502+0000","last_change":"2023-10-08T00:20:05.243396+0000","last_active":"2023-10-08T00:20:05.243502+0000","last_peered":"2023-10-08T00:20:05.243502+0000","last_clean":"2023-10-08T00:20:05.243502+0000","last_became_active":"2023-10-08T00:20:05.243185+0000","last_became_peered":"2023-10-08T00:20:05.243185+0000","last_unstale":"2023-10-08T00:20:05.243502+0000","last_undegraded":"2023-10-08T00:20:05.243502+0000","last_fullsized":"2023-10-08T00:20:05.243502+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:19:47.969487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.250807+0000","last_change":"2023-10-08T00:20:05.250721+0000","last_active":"2023-10-08T00:20:05.250807+0000","last_peered":"2023-10-08T00:20:05.250807+0000","last_clean":"2023-10-08T00:20:05.250807+0000","last_became_active":"2023-10-08T00:20:05.250482+0000","last_became_peered":"2023-10-08T00:20:05.250482+0000","last_unstale":"2023-10-08T00:20:05.250807+0000","last_undegraded":"2023-10-08T00:20:05.250807+0000","last_fullsized":"2023-10-08T00:20:05.250807+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:54:14.710613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.247250+0000","last_change":"2023-10-08T00:20:05.245436+0000","last_active":"2023-10-08T00:20:05.247250+0000","last_peered":"2023-10-08T00:20:05.247250+0000","last_clean":"2023-10-08T00:20:05.247250+0000","last_became_active":"2023-10-08T00:20:05.245218+0000","last_became_peered":"2023-10-08T00:20:05.245218+0000","last_unstale":"2023-10-08T00:20:05.247250+0000","last_undegraded":"2023-10-08T00:20:05.247250+0000","last_fullsized":"2023-10-08T00:20:05.247250+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:33:07.925661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.254974+0000","last_change":"2023-10-08T00:20:05.252069+0000","last_active":"2023-10-08T00:20:05.254974+0000","last_peered":"2023-10-08T00:20:05.254974+0000","last_clean":"2023-10-08T00:20:05.254974+0000","last_became_active":"2023-10-08T00:20:05.251834+0000","last_became_peered":"2023-10-08T00:20:05.251834+0000","last_unstale":"2023-10-08T00:20:05.254974+0000","last_undegraded":"2023-10-08T00:20:05.254974+0000","last_fullsized":"2023-10-08T00:20:05.254974+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:30:53.757333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.241659+0000","last_change":"2023-10-08T00:20:05.241595+0000","last_active":"2023-10-08T00:20:05.241659+0000","last_peered":"2023-10-08T00:20:05.241659+0000","last_clean":"2023-10-08T00:20:05.241659+0000","last_became_active":"2023-10-08T00:20:05.241403+0000","last_became_peered":"2023-10-08T00:20:05.241403+0000","last_unstale":"2023-10-08T00:20:05.241659+0000","last_undegraded":"2023-10-08T00:20:05.241659+0000","last_fullsized":"2023-10-08T00:20:05.241659+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:40:58.162452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.247601+0000","last_change":"2023-10-08T00:20:05.247534+0000","last_active":"2023-10-08T00:20:05.247601+0000","last_peered":"2023-10-08T00:20:05.247601+0000","last_clean":"2023-10-08T00:20:05.247601+0000","last_became_active":"2023-10-08T00:20:05.247303+0000","last_became_peered":"2023-10-08T00:20:05.247303+0000","last_unstale":"2023-10-08T00:20:05.247601+0000","last_undegraded":"2023-10-08T00:20:05.247601+0000","last_fullsized":"2023-10-08T00:20:05.247601+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:20:20.153731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.245700+0000","last_change":"2023-10-08T00:20:05.244175+0000","last_active":"2023-10-08T00:20:05.245700+0000","last_peered":"2023-10-08T00:20:05.245700+0000","last_clean":"2023-10-08T00:20:05.245700+0000","last_became_active":"2023-10-08T00:20:05.243945+0000","last_became_peered":"2023-10-08T00:20:05.243945+0000","last_unstale":"2023-10-08T00:20:05.245700+0000","last_undegraded":"2023-10-08T00:20:05.245700+0000","last_fullsized":"2023-10-08T00:20:05.245700+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:56:49.026616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.243303+0000","last_change":"2023-10-08T00:20:05.242928+0000","last_active":"2023-10-08T00:20:05.243303+0000","last_peered":"2023-10-08T00:20:05.243303+0000","last_clean":"2023-10-08T00:20:05.243303+0000","last_became_active":"2023-10-08T00:20:05.242699+0000","last_became_peered":"2023-10-08T00:20:05.242699+0000","last_unstale":"2023-10-08T00:20:05.243303+0000","last_undegraded":"2023-10-08T00:20:05.243303+0000","last_fullsized":"2023-10-08T00:20:05.243303+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:50:09.923397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.254927+0000","last_change":"2023-10-08T00:20:05.253693+0000","last_active":"2023-10-08T00:20:05.254927+0000","last_peered":"2023-10-08T00:20:05.254927+0000","last_clean":"2023-10-08T00:20:05.254927+0000","last_became_active":"2023-10-08T00:20:05.252558+0000","last_became_peered":"2023-10-08T00:20:05.252558+0000","last_unstale":"2023-10-08T00:20:05.254927+0000","last_undegraded":"2023-10-08T00:20:05.254927+0000","last_fullsized":"2023-10-08T00:20:05.254927+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:44:29.905006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.249304+0000","last_change":"2023-10-08T00:20:05.248336+0000","last_active":"2023-10-08T00:20:05.249304+0000","last_peered":"2023-10-08T00:20:05.249304+0000","last_clean":"2023-10-08T00:20:05.249304+0000","last_became_active":"2023-10-08T00:20:05.248149+0000","last_became_peered":"2023-10-08T00:20:05.248149+0000","last_unstale":"2023-10-08T00:20:05.249304+0000","last_undegraded":"2023-10-08T00:20:05.249304+0000","last_fullsized":"2023-10-08T00:20:05.249304+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:18:31.037964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.243331+0000","last_change":"2023-10-08T00:20:05.243215+0000","last_active":"2023-10-08T00:20:05.243331+0000","last_peered":"2023-10-08T00:20:05.243331+0000","last_clean":"2023-10-08T00:20:05.243331+0000","last_became_active":"2023-10-08T00:20:05.242968+0000","last_became_peered":"2023-10-08T00:20:05.242968+0000","last_unstale":"2023-10-08T00:20:05.243331+0000","last_undegraded":"2023-10-08T00:20:05.243331+0000","last_fullsized":"2023-10-08T00:20:05.243331+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:05:59.854455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.250552+0000","last_change":"2023-10-08T00:20:05.250496+0000","last_active":"2023-10-08T00:20:05.250552+0000","last_peered":"2023-10-08T00:20:05.250552+0000","last_clean":"2023-10-08T00:20:05.250552+0000","last_became_active":"2023-10-08T00:20:05.250266+0000","last_became_peered":"2023-10-08T00:20:05.250266+0000","last_unstale":"2023-10-08T00:20:05.250552+0000","last_undegraded":"2023-10-08T00:20:05.250552+0000","last_fullsized":"2023-10-08T00:20:05.250552+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:03:50.626847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.254894+0000","last_change":"2023-10-08T00:20:05.250340+0000","last_active":"2023-10-08T00:20:05.254894+0000","last_peered":"2023-10-08T00:20:05.254894+0000","last_clean":"2023-10-08T00:20:05.254894+0000","last_became_active":"2023-10-08T00:20:05.250176+0000","last_became_peered":"2023-10-08T00:20:05.250176+0000","last_unstale":"2023-10-08T00:20:05.254894+0000","last_undegraded":"2023-10-08T00:20:05.254894+0000","last_fullsized":"2023-10-08T00:20:05.254894+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:22:48.035127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"167'4","reported_seq":14,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.256976+0000","last_change":"2023-10-08T00:20:04.238738+0000","last_active":"2023-10-08T00:20:05.256976+0000","last_peered":"2023-10-08T00:20:05.256976+0000","last_clean":"2023-10-08T00:20:05.256976+0000","last_became_active":"2023-10-08T00:20:04.238514+0000","last_became_peered":"2023-10-08T00:20:04.238514+0000","last_unstale":"2023-10-08T00:20:05.256976+0000","last_undegraded":"2023-10-08T00:20:05.256976+0000","last_fullsized":"2023-10-08T00:20:05.256976+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:27:28.789777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.242634+0000","last_change":"2023-10-08T00:20:05.242568+0000","last_active":"2023-10-08T00:20:05.242634+0000","last_peered":"2023-10-08T00:20:05.242634+0000","last_clean":"2023-10-08T00:20:05.242634+0000","last_became_active":"2023-10-08T00:20:05.242310+0000","last_became_peered":"2023-10-08T00:20:05.242310+0000","last_unstale":"2023-10-08T00:20:05.242634+0000","last_undegraded":"2023-10-08T00:20:05.242634+0000","last_fullsized":"2023-10-08T00:20:05.242634+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:07:54.014634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.1e","version":"167'1","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.256471+0000","last_change":"2023-10-08T00:20:04.246114+0000","last_active":"2023-10-08T00:20:05.256471+0000","last_peered":"2023-10-08T00:20:05.256471+0000","last_clean":"2023-10-08T00:20:05.256471+0000","last_became_active":"2023-10-08T00:20:04.245605+0000","last_became_peered":"2023-10-08T00:20:04.245605+0000","last_unstale":"2023-10-08T00:20:05.256471+0000","last_undegraded":"2023-10-08T00:20:05.256471+0000","last_fullsized":"2023-10-08T00:20:05.256471+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:43:35.304179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.256884+0000","last_change":"2023-10-08T00:20:05.256838+0000","last_active":"2023-10-08T00:20:05.256884+0000","last_peered":"2023-10-08T00:20:05.256884+0000","last_clean":"2023-10-08T00:20:05.256884+0000","last_became_active":"2023-10-08T00:20:05.256529+0000","last_became_peered":"2023-10-08T00:20:05.256529+0000","last_unstale":"2023-10-08T00:20:05.256884+0000","last_undegraded":"2023-10-08T00:20:05.256884+0000","last_fullsized":"2023-10-08T00:20:05.256884+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:34:08.648121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.1d","version":"167'2","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.254097+0000","last_change":"2023-10-08T00:20:04.248454+0000","last_active":"2023-10-08T00:20:05.254097+0000","last_peered":"2023-10-08T00:20:05.254097+0000","last_clean":"2023-10-08T00:20:05.254097+0000","last_became_active":"2023-10-08T00:20:04.246408+0000","last_became_peered":"2023-10-08T00:20:04.246408+0000","last_unstale":"2023-10-08T00:20:05.254097+0000","last_undegraded":"2023-10-08T00:20:05.254097+0000","last_fullsized":"2023-10-08T00:20:05.254097+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:50.024701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.244320+0000","last_change":"2023-10-08T00:20:05.244255+0000","last_active":"2023-10-08T00:20:05.244320+0000","last_peered":"2023-10-08T00:20:05.244320+0000","last_clean":"2023-10-08T00:20:05.244320+0000","last_became_active":"2023-10-08T00:20:05.243929+0000","last_became_peered":"2023-10-08T00:20:05.243929+0000","last_unstale":"2023-10-08T00:20:05.244320+0000","last_undegraded":"2023-10-08T00:20:05.244320+0000","last_fullsized":"2023-10-08T00:20:05.244320+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:34:06.226802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1c","version":"167'2","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.256526+0000","last_change":"2023-10-08T00:20:04.247073+0000","last_active":"2023-10-08T00:20:05.256526+0000","last_peered":"2023-10-08T00:20:05.256526+0000","last_clean":"2023-10-08T00:20:05.256526+0000","last_became_active":"2023-10-08T00:20:04.246803+0000","last_became_peered":"2023-10-08T00:20:04.246803+0000","last_unstale":"2023-10-08T00:20:05.256526+0000","last_undegraded":"2023-10-08T00:20:05.256526+0000","last_fullsized":"2023-10-08T00:20:05.256526+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:26:57.425301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.245777+0000","last_change":"2023-10-08T00:20:05.245591+0000","last_active":"2023-10-08T00:20:05.245777+0000","last_peered":"2023-10-08T00:20:05.245777+0000","last_clean":"2023-10-08T00:20:05.245777+0000","last_became_active":"2023-10-08T00:20:05.245372+0000","last_became_peered":"2023-10-08T00:20:05.245372+0000","last_unstale":"2023-10-08T00:20:05.245777+0000","last_undegraded":"2023-10-08T00:20:05.245777+0000","last_fullsized":"2023-10-08T00:20:05.245777+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:39:43.013423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"167'1","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.258319+0000","last_change":"2023-10-08T00:20:04.244497+0000","last_active":"2023-10-08T00:20:05.258319+0000","last_peered":"2023-10-08T00:20:05.258319+0000","last_clean":"2023-10-08T00:20:05.258319+0000","last_became_active":"2023-10-08T00:20:04.244226+0000","last_became_peered":"2023-10-08T00:20:04.244226+0000","last_unstale":"2023-10-08T00:20:05.258319+0000","last_undegraded":"2023-10-08T00:20:05.258319+0000","last_fullsized":"2023-10-08T00:20:05.258319+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:40:57.155039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.255428+0000","last_change":"2023-10-08T00:20:05.254308+0000","last_active":"2023-10-08T00:20:05.255428+0000","last_peered":"2023-10-08T00:20:05.255428+0000","last_clean":"2023-10-08T00:20:05.255428+0000","last_became_active":"2023-10-08T00:20:05.253303+0000","last_became_peered":"2023-10-08T00:20:05.253303+0000","last_unstale":"2023-10-08T00:20:05.255428+0000","last_undegraded":"2023-10-08T00:20:05.255428+0000","last_fullsized":"2023-10-08T00:20:05.255428+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:34:08.648121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1a","version":"167'1","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.255040+0000","last_change":"2023-10-08T00:20:04.246127+0000","last_active":"2023-10-08T00:20:05.255040+0000","last_peered":"2023-10-08T00:20:05.255040+0000","last_clean":"2023-10-08T00:20:05.255040+0000","last_became_active":"2023-10-08T00:20:04.245880+0000","last_became_peered":"2023-10-08T00:20:04.245880+0000","last_unstale":"2023-10-08T00:20:05.255040+0000","last_undegraded":"2023-10-08T00:20:05.255040+0000","last_fullsized":"2023-10-08T00:20:05.255040+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:01:16.185368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.247042+0000","last_change":"2023-10-08T00:20:05.246932+0000","last_active":"2023-10-08T00:20:05.247042+0000","last_peered":"2023-10-08T00:20:05.247042+0000","last_clean":"2023-10-08T00:20:05.247042+0000","last_became_active":"2023-10-08T00:20:05.246709+0000","last_became_peered":"2023-10-08T00:20:05.246709+0000","last_unstale":"2023-10-08T00:20:05.247042+0000","last_undegraded":"2023-10-08T00:20:05.247042+0000","last_fullsized":"2023-10-08T00:20:05.247042+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:09:43.710079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.240904+0000","last_change":"2023-10-08T00:20:04.246710+0000","last_active":"2023-10-08T00:20:05.240904+0000","last_peered":"2023-10-08T00:20:05.240904+0000","last_clean":"2023-10-08T00:20:05.240904+0000","last_became_active":"2023-10-08T00:20:04.246477+0000","last_became_peered":"2023-10-08T00:20:04.246477+0000","last_unstale":"2023-10-08T00:20:05.240904+0000","last_undegraded":"2023-10-08T00:20:05.240904+0000","last_fullsized":"2023-10-08T00:20:05.240904+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:05:58.847042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.249776+0000","last_change":"2023-10-08T00:20:05.249568+0000","last_active":"2023-10-08T00:20:05.249776+0000","last_peered":"2023-10-08T00:20:05.249776+0000","last_clean":"2023-10-08T00:20:05.249776+0000","last_became_active":"2023-10-08T00:20:05.249354+0000","last_became_peered":"2023-10-08T00:20:05.249354+0000","last_unstale":"2023-10-08T00:20:05.249776+0000","last_undegraded":"2023-10-08T00:20:05.249776+0000","last_fullsized":"2023-10-08T00:20:05.249776+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:49:32.653899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.242523+0000","last_change":"2023-10-08T00:20:04.248590+0000","last_active":"2023-10-08T00:20:05.242523+0000","last_peered":"2023-10-08T00:20:05.242523+0000","last_clean":"2023-10-08T00:20:05.242523+0000","last_became_active":"2023-10-08T00:20:04.246982+0000","last_became_peered":"2023-10-08T00:20:04.246982+0000","last_unstale":"2023-10-08T00:20:05.242523+0000","last_undegraded":"2023-10-08T00:20:05.242523+0000","last_fullsized":"2023-10-08T00:20:05.242523+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:01:07.906425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.255507+0000","last_change":"2023-10-08T00:20:05.254574+0000","last_active":"2023-10-08T00:20:05.255507+0000","last_peered":"2023-10-08T00:20:05.255507+0000","last_clean":"2023-10-08T00:20:05.255507+0000","last_became_active":"2023-10-08T00:20:05.253902+0000","last_became_peered":"2023-10-08T00:20:05.253902+0000","last_unstale":"2023-10-08T00:20:05.255507+0000","last_undegraded":"2023-10-08T00:20:05.255507+0000","last_fullsized":"2023-10-08T00:20:05.255507+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:52:47.931876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.242247+0000","last_change":"2023-10-08T00:20:04.245003+0000","last_active":"2023-10-08T00:20:05.242247+0000","last_peered":"2023-10-08T00:20:05.242247+0000","last_clean":"2023-10-08T00:20:05.242247+0000","last_became_active":"2023-10-08T00:20:04.244739+0000","last_became_peered":"2023-10-08T00:20:04.244739+0000","last_unstale":"2023-10-08T00:20:05.242247+0000","last_undegraded":"2023-10-08T00:20:05.242247+0000","last_fullsized":"2023-10-08T00:20:05.242247+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:00:58.443848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.254685+0000","last_change":"2023-10-08T00:20:05.254453+0000","last_active":"2023-10-08T00:20:05.254685+0000","last_peered":"2023-10-08T00:20:05.254685+0000","last_clean":"2023-10-08T00:20:05.254685+0000","last_became_active":"2023-10-08T00:20:05.252916+0000","last_became_peered":"2023-10-08T00:20:05.252916+0000","last_unstale":"2023-10-08T00:20:05.254685+0000","last_undegraded":"2023-10-08T00:20:05.254685+0000","last_fullsized":"2023-10-08T00:20:05.254685+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:34:06.226802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.241337+0000","last_change":"2023-10-08T00:20:04.249414+0000","last_active":"2023-10-08T00:20:05.241337+0000","last_peered":"2023-10-08T00:20:05.241337+0000","last_clean":"2023-10-08T00:20:05.241337+0000","last_became_active":"2023-10-08T00:20:04.249191+0000","last_became_peered":"2023-10-08T00:20:04.249191+0000","last_unstale":"2023-10-08T00:20:05.241337+0000","last_undegraded":"2023-10-08T00:20:05.241337+0000","last_fullsized":"2023-10-08T00:20:05.241337+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:52:05.971558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.243591+0000","last_change":"2023-10-08T00:20:05.243491+0000","last_active":"2023-10-08T00:20:05.243591+0000","last_peered":"2023-10-08T00:20:05.243591+0000","last_clean":"2023-10-08T00:20:05.243591+0000","last_became_active":"2023-10-08T00:20:05.243266+0000","last_became_peered":"2023-10-08T00:20:05.243266+0000","last_unstale":"2023-10-08T00:20:05.243591+0000","last_undegraded":"2023-10-08T00:20:05.243591+0000","last_fullsized":"2023-10-08T00:20:05.243591+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:43:36.311593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.14","version":"167'6","reported_seq":16,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.259480+0000","last_change":"2023-10-08T00:20:04.251024+0000","last_active":"2023-10-08T00:20:05.259480+0000","last_peered":"2023-10-08T00:20:05.259480+0000","last_clean":"2023-10-08T00:20:05.259480+0000","last_became_active":"2023-10-08T00:20:04.250509+0000","last_became_peered":"2023-10-08T00:20:04.250509+0000","last_unstale":"2023-10-08T00:20:05.259480+0000","last_undegraded":"2023-10-08T00:20:05.259480+0000","last_fullsized":"2023-10-08T00:20:05.259480+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:37:16.406481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.252153+0000","last_change":"2023-10-08T00:20:05.252090+0000","last_active":"2023-10-08T00:20:05.252153+0000","last_peered":"2023-10-08T00:20:05.252153+0000","last_clean":"2023-10-08T00:20:05.252153+0000","last_became_active":"2023-10-08T00:20:05.251701+0000","last_became_peered":"2023-10-08T00:20:05.251701+0000","last_unstale":"2023-10-08T00:20:05.252153+0000","last_undegraded":"2023-10-08T00:20:05.252153+0000","last_fullsized":"2023-10-08T00:20:05.252153+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:07:54.014634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.13","version":"167'2","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.254265+0000","last_change":"2023-10-08T00:20:04.248311+0000","last_active":"2023-10-08T00:20:05.254265+0000","last_peered":"2023-10-08T00:20:05.254265+0000","last_clean":"2023-10-08T00:20:05.254265+0000","last_became_active":"2023-10-08T00:20:04.246106+0000","last_became_peered":"2023-10-08T00:20:04.246106+0000","last_unstale":"2023-10-08T00:20:05.254265+0000","last_undegraded":"2023-10-08T00:20:05.254265+0000","last_fullsized":"2023-10-08T00:20:05.254265+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:37:23.216226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.258649+0000","last_change":"2023-10-08T00:20:05.258587+0000","last_active":"2023-10-08T00:20:05.258649+0000","last_peered":"2023-10-08T00:20:05.258649+0000","last_clean":"2023-10-08T00:20:05.258649+0000","last_became_active":"2023-10-08T00:20:05.258394+0000","last_became_peered":"2023-10-08T00:20:05.258394+0000","last_unstale":"2023-10-08T00:20:05.258649+0000","last_undegraded":"2023-10-08T00:20:05.258649+0000","last_fullsized":"2023-10-08T00:20:05.258649+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:34:08.648121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.248788+0000","last_change":"2023-10-08T00:20:04.242275+0000","last_active":"2023-10-08T00:20:05.248788+0000","last_peered":"2023-10-08T00:20:05.248788+0000","last_clean":"2023-10-08T00:20:05.248788+0000","last_became_active":"2023-10-08T00:20:04.241872+0000","last_became_peered":"2023-10-08T00:20:04.241872+0000","last_unstale":"2023-10-08T00:20:05.248788+0000","last_undegraded":"2023-10-08T00:20:05.248788+0000","last_fullsized":"2023-10-08T00:20:05.248788+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:18:39.105501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.247210+0000","last_change":"2023-10-08T00:20:05.244976+0000","last_active":"2023-10-08T00:20:05.247210+0000","last_peered":"2023-10-08T00:20:05.247210+0000","last_clean":"2023-10-08T00:20:05.247210+0000","last_became_active":"2023-10-08T00:20:05.244533+0000","last_became_peered":"2023-10-08T00:20:05.244533+0000","last_unstale":"2023-10-08T00:20:05.247210+0000","last_undegraded":"2023-10-08T00:20:05.247210+0000","last_fullsized":"2023-10-08T00:20:05.247210+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:44:29.905006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.238940+0000","last_change":"2023-10-08T00:20:04.246262+0000","last_active":"2023-10-08T00:20:05.238940+0000","last_peered":"2023-10-08T00:20:05.238940+0000","last_clean":"2023-10-08T00:20:05.238940+0000","last_became_active":"2023-10-08T00:20:04.244141+0000","last_became_peered":"2023-10-08T00:20:04.244141+0000","last_unstale":"2023-10-08T00:20:05.238940+0000","last_undegraded":"2023-10-08T00:20:05.238940+0000","last_fullsized":"2023-10-08T00:20:05.238940+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:24:31.008196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.255472+0000","last_change":"2023-10-08T00:20:05.254444+0000","last_active":"2023-10-08T00:20:05.255472+0000","last_peered":"2023-10-08T00:20:05.255472+0000","last_clean":"2023-10-08T00:20:05.255472+0000","last_became_active":"2023-10-08T00:20:05.253619+0000","last_became_peered":"2023-10-08T00:20:05.253619+0000","last_unstale":"2023-10-08T00:20:05.255472+0000","last_undegraded":"2023-10-08T00:20:05.255472+0000","last_fullsized":"2023-10-08T00:20:05.255472+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:00:59.451262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.10","version":"167'8","reported_seq":18,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.256509+0000","last_change":"2023-10-08T00:20:04.246644+0000","last_active":"2023-10-08T00:20:05.256509+0000","last_peered":"2023-10-08T00:20:05.256509+0000","last_clean":"2023-10-08T00:20:05.256509+0000","last_became_active":"2023-10-08T00:20:04.246456+0000","last_became_peered":"2023-10-08T00:20:04.246456+0000","last_unstale":"2023-10-08T00:20:05.256509+0000","last_undegraded":"2023-10-08T00:20:05.256509+0000","last_fullsized":"2023-10-08T00:20:05.256509+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:24:48.069538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.258776+0000","last_change":"2023-10-08T00:20:05.258594+0000","last_active":"2023-10-08T00:20:05.258776+0000","last_peered":"2023-10-08T00:20:05.258776+0000","last_clean":"2023-10-08T00:20:05.258776+0000","last_became_active":"2023-10-08T00:20:05.258316+0000","last_became_peered":"2023-10-08T00:20:05.258316+0000","last_unstale":"2023-10-08T00:20:05.258776+0000","last_undegraded":"2023-10-08T00:20:05.258776+0000","last_fullsized":"2023-10-08T00:20:05.258776+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:45:00.428167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.f","version":"167'1","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.255792+0000","last_change":"2023-10-08T00:20:04.241270+0000","last_active":"2023-10-08T00:20:05.255792+0000","last_peered":"2023-10-08T00:20:05.255792+0000","last_clean":"2023-10-08T00:20:05.255792+0000","last_became_active":"2023-10-08T00:20:04.241025+0000","last_became_peered":"2023-10-08T00:20:04.241025+0000","last_unstale":"2023-10-08T00:20:05.255792+0000","last_undegraded":"2023-10-08T00:20:05.255792+0000","last_fullsized":"2023-10-08T00:20:05.255792+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:27:07.999816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.247024+0000","last_change":"2023-10-08T00:20:05.246917+0000","last_active":"2023-10-08T00:20:05.247024+0000","last_peered":"2023-10-08T00:20:05.247024+0000","last_clean":"2023-10-08T00:20:05.247024+0000","last_became_active":"2023-10-08T00:20:05.246691+0000","last_became_peered":"2023-10-08T00:20:05.246691+0000","last_unstale":"2023-10-08T00:20:05.247024+0000","last_undegraded":"2023-10-08T00:20:05.247024+0000","last_fullsized":"2023-10-08T00:20:05.247024+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:37:17.413894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.e","version":"167'2","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.256655+0000","last_change":"2023-10-08T00:20:04.245210+0000","last_active":"2023-10-08T00:20:05.256655+0000","last_peered":"2023-10-08T00:20:05.256655+0000","last_clean":"2023-10-08T00:20:05.256655+0000","last_became_active":"2023-10-08T00:20:04.244971+0000","last_became_peered":"2023-10-08T00:20:04.244971+0000","last_unstale":"2023-10-08T00:20:05.256655+0000","last_undegraded":"2023-10-08T00:20:05.256655+0000","last_fullsized":"2023-10-08T00:20:05.256655+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:39:42.006010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.257928+0000","last_change":"2023-10-08T00:20:05.257818+0000","last_active":"2023-10-08T00:20:05.257928+0000","last_peered":"2023-10-08T00:20:05.257928+0000","last_clean":"2023-10-08T00:20:05.257928+0000","last_became_active":"2023-10-08T00:20:05.257127+0000","last_became_peered":"2023-10-08T00:20:05.257127+0000","last_unstale":"2023-10-08T00:20:05.257928+0000","last_undegraded":"2023-10-08T00:20:05.257928+0000","last_fullsized":"2023-10-08T00:20:05.257928+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:42:22.115875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"167'1","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.255243+0000","last_change":"2023-10-08T00:20:04.244040+0000","last_active":"2023-10-08T00:20:05.255243+0000","last_peered":"2023-10-08T00:20:05.255243+0000","last_clean":"2023-10-08T00:20:05.255243+0000","last_became_active":"2023-10-08T00:20:04.242376+0000","last_became_peered":"2023-10-08T00:20:04.242376+0000","last_unstale":"2023-10-08T00:20:05.255243+0000","last_undegraded":"2023-10-08T00:20:05.255243+0000","last_fullsized":"2023-10-08T00:20:05.255243+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:09:42.702666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.256772+0000","last_change":"2023-10-08T00:20:05.256706+0000","last_active":"2023-10-08T00:20:05.256772+0000","last_peered":"2023-10-08T00:20:05.256772+0000","last_clean":"2023-10-08T00:20:05.256772+0000","last_became_active":"2023-10-08T00:20:05.256225+0000","last_became_peered":"2023-10-08T00:20:05.256225+0000","last_unstale":"2023-10-08T00:20:05.256772+0000","last_undegraded":"2023-10-08T00:20:05.256772+0000","last_fullsized":"2023-10-08T00:20:05.256772+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:44:29.905006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.239029+0000","last_change":"2023-10-08T00:20:04.246634+0000","last_active":"2023-10-08T00:20:05.239029+0000","last_peered":"2023-10-08T00:20:05.239029+0000","last_clean":"2023-10-08T00:20:05.239029+0000","last_became_active":"2023-10-08T00:20:04.246030+0000","last_became_peered":"2023-10-08T00:20:04.246030+0000","last_unstale":"2023-10-08T00:20:05.239029+0000","last_undegraded":"2023-10-08T00:20:05.239029+0000","last_fullsized":"2023-10-08T00:20:05.239029+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:52:19.278093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.251066+0000","last_change":"2023-10-08T00:20:05.251010+0000","last_active":"2023-10-08T00:20:05.251066+0000","last_peered":"2023-10-08T00:20:05.251066+0000","last_clean":"2023-10-08T00:20:05.251066+0000","last_became_active":"2023-10-08T00:20:05.250656+0000","last_became_peered":"2023-10-08T00:20:05.250656+0000","last_unstale":"2023-10-08T00:20:05.251066+0000","last_undegraded":"2023-10-08T00:20:05.251066+0000","last_fullsized":"2023-10-08T00:20:05.251066+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:55:15.548941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.b","version":"167'2","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.257948+0000","last_change":"2023-10-08T00:20:04.249629+0000","last_active":"2023-10-08T00:20:05.257948+0000","last_peered":"2023-10-08T00:20:05.257948+0000","last_clean":"2023-10-08T00:20:05.257948+0000","last_became_active":"2023-10-08T00:20:04.249401+0000","last_became_peered":"2023-10-08T00:20:04.249401+0000","last_unstale":"2023-10-08T00:20:05.257948+0000","last_undegraded":"2023-10-08T00:20:05.257948+0000","last_fullsized":"2023-10-08T00:20:05.257948+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:24:48.069538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.245751+0000","last_change":"2023-10-08T00:20:05.243906+0000","last_active":"2023-10-08T00:20:05.245751+0000","last_peered":"2023-10-08T00:20:05.245751+0000","last_clean":"2023-10-08T00:20:05.245751+0000","last_became_active":"2023-10-08T00:20:05.243615+0000","last_became_peered":"2023-10-08T00:20:05.243615+0000","last_unstale":"2023-10-08T00:20:05.245751+0000","last_undegraded":"2023-10-08T00:20:05.245751+0000","last_fullsized":"2023-10-08T00:20:05.245751+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:08:07.447036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.237626+0000","last_change":"2023-10-08T00:20:04.247748+0000","last_active":"2023-10-08T00:20:05.237626+0000","last_peered":"2023-10-08T00:20:05.237626+0000","last_clean":"2023-10-08T00:20:05.237626+0000","last_became_active":"2023-10-08T00:20:04.247530+0000","last_became_peered":"2023-10-08T00:20:04.247530+0000","last_unstale":"2023-10-08T00:20:05.237626+0000","last_undegraded":"2023-10-08T00:20:05.237626+0000","last_fullsized":"2023-10-08T00:20:05.237626+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:54:04.355034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.254668+0000","last_change":"2023-10-08T00:20:05.254316+0000","last_active":"2023-10-08T00:20:05.254668+0000","last_peered":"2023-10-08T00:20:05.254668+0000","last_clean":"2023-10-08T00:20:05.254668+0000","last_became_active":"2023-10-08T00:20:05.252638+0000","last_became_peered":"2023-10-08T00:20:05.252638+0000","last_unstale":"2023-10-08T00:20:05.254668+0000","last_undegraded":"2023-10-08T00:20:05.254668+0000","last_fullsized":"2023-10-08T00:20:05.254668+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:34:08.648121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.237342+0000","last_change":"2023-10-08T00:20:04.245345+0000","last_active":"2023-10-08T00:20:05.237342+0000","last_peered":"2023-10-08T00:20:05.237342+0000","last_clean":"2023-10-08T00:20:05.237342+0000","last_became_active":"2023-10-08T00:20:04.245173+0000","last_became_peered":"2023-10-08T00:20:04.245173+0000","last_unstale":"2023-10-08T00:20:05.237342+0000","last_undegraded":"2023-10-08T00:20:05.237342+0000","last_fullsized":"2023-10-08T00:20:05.237342+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:01:36.620681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.245730+0000","last_change":"2023-10-08T00:20:05.244414+0000","last_active":"2023-10-08T00:20:05.245730+0000","last_peered":"2023-10-08T00:20:05.245730+0000","last_clean":"2023-10-08T00:20:05.245730+0000","last_became_active":"2023-10-08T00:20:05.243718+0000","last_became_peered":"2023-10-08T00:20:05.243718+0000","last_unstale":"2023-10-08T00:20:05.245730+0000","last_undegraded":"2023-10-08T00:20:05.245730+0000","last_fullsized":"2023-10-08T00:20:05.245730+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:43:41.968090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.237627+0000","last_change":"2023-10-08T00:20:04.242970+0000","last_active":"2023-10-08T00:20:05.237627+0000","last_peered":"2023-10-08T00:20:05.237627+0000","last_clean":"2023-10-08T00:20:05.237627+0000","last_became_active":"2023-10-08T00:20:04.242709+0000","last_became_peered":"2023-10-08T00:20:04.242709+0000","last_unstale":"2023-10-08T00:20:05.237627+0000","last_undegraded":"2023-10-08T00:20:05.237627+0000","last_fullsized":"2023-10-08T00:20:05.237627+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:04:36.794999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.247187+0000","last_change":"2023-10-08T00:20:05.244829+0000","last_active":"2023-10-08T00:20:05.247187+0000","last_peered":"2023-10-08T00:20:05.247187+0000","last_clean":"2023-10-08T00:20:05.247187+0000","last_became_active":"2023-10-08T00:20:05.244323+0000","last_became_peered":"2023-10-08T00:20:05.244323+0000","last_unstale":"2023-10-08T00:20:05.247187+0000","last_undegraded":"2023-10-08T00:20:05.247187+0000","last_fullsized":"2023-10-08T00:20:05.247187+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:20:20.153731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.7","version":"167'2","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.258473+0000","last_change":"2023-10-08T00:20:04.244594+0000","last_active":"2023-10-08T00:20:05.258473+0000","last_peered":"2023-10-08T00:20:05.258473+0000","last_clean":"2023-10-08T00:20:05.258473+0000","last_became_active":"2023-10-08T00:20:04.244342+0000","last_became_peered":"2023-10-08T00:20:04.244342+0000","last_unstale":"2023-10-08T00:20:05.258473+0000","last_undegraded":"2023-10-08T00:20:05.258473+0000","last_fullsized":"2023-10-08T00:20:05.258473+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:36:39.544482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.255372+0000","last_change":"2023-10-08T00:20:05.253093+0000","last_active":"2023-10-08T00:20:05.255372+0000","last_peered":"2023-10-08T00:20:05.255372+0000","last_clean":"2023-10-08T00:20:05.255372+0000","last_became_active":"2023-10-08T00:20:05.243661+0000","last_became_peered":"2023-10-08T00:20:05.243661+0000","last_unstale":"2023-10-08T00:20:05.255372+0000","last_undegraded":"2023-10-08T00:20:05.255372+0000","last_fullsized":"2023-10-08T00:20:05.255372+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:10:26.392889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.237710+0000","last_change":"2023-10-08T00:20:04.247009+0000","last_active":"2023-10-08T00:20:05.237710+0000","last_peered":"2023-10-08T00:20:05.237710+0000","last_clean":"2023-10-08T00:20:05.237710+0000","last_became_active":"2023-10-08T00:20:04.246663+0000","last_became_peered":"2023-10-08T00:20:04.246663+0000","last_unstale":"2023-10-08T00:20:05.237710+0000","last_undegraded":"2023-10-08T00:20:05.237710+0000","last_fullsized":"2023-10-08T00:20:05.237710+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:45:17.133123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.247754+0000","last_change":"2023-10-08T00:20:05.247665+0000","last_active":"2023-10-08T00:20:05.247754+0000","last_peered":"2023-10-08T00:20:05.247754+0000","last_clean":"2023-10-08T00:20:05.247754+0000","last_became_active":"2023-10-08T00:20:05.247280+0000","last_became_peered":"2023-10-08T00:20:05.247280+0000","last_unstale":"2023-10-08T00:20:05.247754+0000","last_undegraded":"2023-10-08T00:20:05.247754+0000","last_fullsized":"2023-10-08T00:20:05.247754+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:33:07.925661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.238912+0000","last_change":"2023-10-08T00:20:04.251109+0000","last_active":"2023-10-08T00:20:05.238912+0000","last_peered":"2023-10-08T00:20:05.238912+0000","last_clean":"2023-10-08T00:20:05.238912+0000","last_became_active":"2023-10-08T00:20:04.250736+0000","last_became_peered":"2023-10-08T00:20:04.250736+0000","last_unstale":"2023-10-08T00:20:05.238912+0000","last_undegraded":"2023-10-08T00:20:05.238912+0000","last_fullsized":"2023-10-08T00:20:05.238912+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:03:49.619434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.251815+0000","last_change":"2023-10-08T00:20:05.249949+0000","last_active":"2023-10-08T00:20:05.251815+0000","last_peered":"2023-10-08T00:20:05.251815+0000","last_clean":"2023-10-08T00:20:05.251815+0000","last_became_active":"2023-10-08T00:20:05.249727+0000","last_became_peered":"2023-10-08T00:20:05.249727+0000","last_unstale":"2023-10-08T00:20:05.251815+0000","last_undegraded":"2023-10-08T00:20:05.251815+0000","last_fullsized":"2023-10-08T00:20:05.251815+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:34:06.226802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.237409+0000","last_change":"2023-10-08T00:20:04.248488+0000","last_active":"2023-10-08T00:20:05.237409+0000","last_peered":"2023-10-08T00:20:05.237409+0000","last_clean":"2023-10-08T00:20:05.237409+0000","last_became_active":"2023-10-08T00:20:04.248203+0000","last_became_peered":"2023-10-08T00:20:04.248203+0000","last_unstale":"2023-10-08T00:20:05.237409+0000","last_undegraded":"2023-10-08T00:20:05.237409+0000","last_fullsized":"2023-10-08T00:20:05.237409+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:56:17.020837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.250876+0000","last_change":"2023-10-08T00:20:05.250831+0000","last_active":"2023-10-08T00:20:05.250876+0000","last_peered":"2023-10-08T00:20:05.250876+0000","last_clean":"2023-10-08T00:20:05.250876+0000","last_became_active":"2023-10-08T00:20:05.250672+0000","last_became_peered":"2023-10-08T00:20:05.250672+0000","last_unstale":"2023-10-08T00:20:05.250876+0000","last_undegraded":"2023-10-08T00:20:05.250876+0000","last_fullsized":"2023-10-08T00:20:05.250876+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:48:04.097734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.237682+0000","last_change":"2023-10-08T00:20:04.247524+0000","last_active":"2023-10-08T00:20:05.237682+0000","last_peered":"2023-10-08T00:20:05.237682+0000","last_clean":"2023-10-08T00:20:05.237682+0000","last_became_active":"2023-10-08T00:20:04.240348+0000","last_became_peered":"2023-10-08T00:20:04.240348+0000","last_unstale":"2023-10-08T00:20:05.237682+0000","last_undegraded":"2023-10-08T00:20:05.237682+0000","last_fullsized":"2023-10-08T00:20:05.237682+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:37:41.683109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.248946+0000","last_change":"2023-10-08T00:20:05.248871+0000","last_active":"2023-10-08T00:20:05.248946+0000","last_peered":"2023-10-08T00:20:05.248946+0000","last_clean":"2023-10-08T00:20:05.248946+0000","last_became_active":"2023-10-08T00:20:05.248573+0000","last_became_peered":"2023-10-08T00:20:05.248573+0000","last_unstale":"2023-10-08T00:20:05.248946+0000","last_undegraded":"2023-10-08T00:20:05.248946+0000","last_fullsized":"2023-10-08T00:20:05.248946+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:48:04.097734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.242379+0000","last_change":"2023-10-08T00:20:04.243813+0000","last_active":"2023-10-08T00:20:05.242379+0000","last_peered":"2023-10-08T00:20:05.242379+0000","last_clean":"2023-10-08T00:20:05.242379+0000","last_became_active":"2023-10-08T00:20:04.243560+0000","last_became_peered":"2023-10-08T00:20:04.243560+0000","last_unstale":"2023-10-08T00:20:05.242379+0000","last_undegraded":"2023-10-08T00:20:05.242379+0000","last_fullsized":"2023-10-08T00:20:05.242379+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:56:50.470059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.247268+0000","last_change":"2023-10-08T00:20:05.245479+0000","last_active":"2023-10-08T00:20:05.247268+0000","last_peered":"2023-10-08T00:20:05.247268+0000","last_clean":"2023-10-08T00:20:05.247268+0000","last_became_active":"2023-10-08T00:20:05.245227+0000","last_became_peered":"2023-10-08T00:20:05.245227+0000","last_unstale":"2023-10-08T00:20:05.247268+0000","last_undegraded":"2023-10-08T00:20:05.247268+0000","last_fullsized":"2023-10-08T00:20:05.247268+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:55:15.548941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.241393+0000","last_change":"2023-10-08T00:20:04.246657+0000","last_active":"2023-10-08T00:20:05.241393+0000","last_peered":"2023-10-08T00:20:05.241393+0000","last_clean":"2023-10-08T00:20:05.241393+0000","last_became_active":"2023-10-08T00:20:04.246319+0000","last_became_peered":"2023-10-08T00:20:04.246319+0000","last_unstale":"2023-10-08T00:20:05.241393+0000","last_undegraded":"2023-10-08T00:20:05.241393+0000","last_fullsized":"2023-10-08T00:20:05.241393+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:50:08.915984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.247712+0000","last_change":"2023-10-08T00:20:05.247649+0000","last_active":"2023-10-08T00:20:05.247712+0000","last_peered":"2023-10-08T00:20:05.247712+0000","last_clean":"2023-10-08T00:20:05.247712+0000","last_became_active":"2023-10-08T00:20:05.247466+0000","last_became_peered":"2023-10-08T00:20:05.247466+0000","last_unstale":"2023-10-08T00:20:05.247712+0000","last_undegraded":"2023-10-08T00:20:05.247712+0000","last_fullsized":"2023-10-08T00:20:05.247712+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:33:38.066405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.240999+0000","last_change":"2023-10-08T00:20:04.246258+0000","last_active":"2023-10-08T00:20:05.240999+0000","last_peered":"2023-10-08T00:20:05.240999+0000","last_clean":"2023-10-08T00:20:05.240999+0000","last_became_active":"2023-10-08T00:20:04.245892+0000","last_became_peered":"2023-10-08T00:20:04.245892+0000","last_unstale":"2023-10-08T00:20:05.240999+0000","last_undegraded":"2023-10-08T00:20:05.240999+0000","last_fullsized":"2023-10-08T00:20:05.240999+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:50:08.915984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.244288+0000","last_change":"2023-10-08T00:20:05.243876+0000","last_active":"2023-10-08T00:20:05.244288+0000","last_peered":"2023-10-08T00:20:05.244288+0000","last_clean":"2023-10-08T00:20:05.244288+0000","last_became_active":"2023-10-08T00:20:05.243662+0000","last_became_peered":"2023-10-08T00:20:05.243662+0000","last_unstale":"2023-10-08T00:20:05.244288+0000","last_undegraded":"2023-10-08T00:20:05.244288+0000","last_fullsized":"2023-10-08T00:20:05.244288+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:05:59.854455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.15","version":"167'3","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.259385+0000","last_change":"2023-10-08T00:20:04.246528+0000","last_active":"2023-10-08T00:20:05.259385+0000","last_peered":"2023-10-08T00:20:05.259385+0000","last_clean":"2023-10-08T00:20:05.259385+0000","last_became_active":"2023-10-08T00:20:04.246029+0000","last_became_peered":"2023-10-08T00:20:04.246029+0000","last_unstale":"2023-10-08T00:20:05.259385+0000","last_undegraded":"2023-10-08T00:20:05.259385+0000","last_fullsized":"2023-10-08T00:20:05.259385+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:05:58.847042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":289,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.244557+0000","last_change":"2023-10-08T00:10:43.728490+0000","last_active":"2023-10-08T00:20:05.244557+0000","last_peered":"2023-10-08T00:20:05.244557+0000","last_clean":"2023-10-08T00:20:05.244557+0000","last_became_active":"2023-10-08T00:10:43.728341+0000","last_became_peered":"2023-10-08T00:10:43.728341+0000","last_unstale":"2023-10-08T00:20:05.244557+0000","last_undegraded":"2023-10-08T00:20:05.244557+0000","last_fullsized":"2023-10-08T00:20:05.244557+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_clean_scrub_stamp":"2023-10-08T00:10:42.716096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:06:31.035293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574963,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":917,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194394,"statfs":{"total":200000000,"available":199060448,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:19:58 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.53800000000000003,"15min":0.51600000000000001},"min":{"1min":0.33900000000000002,"5min":0.27800000000000002,"15min":0.26900000000000002},"max":{"1min":0.63,"5min":1.129,"15min":1.129},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.55600000000000005,"15min":0.54600000000000004},"min":{"1min":0.41899999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.70599999999999996,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.45000000000000001}]},{"osd":1,"last update":"Sun Oct 8 00:19:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57699999999999996,"15min":0.55100000000000005},"min":{"1min":0.32300000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72299999999999998,"5min":0.77900000000000003,"15min":0.82999999999999996},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.59099999999999997,"15min":0.57499999999999996},"min":{"1min":0.498,"5min":0.39300000000000002,"15min":0.38200000000000001},"max":{"1min":0.76600000000000001,"5min":0.79400000000000004,"15min":0.80100000000000005},"last":0.52400000000000002}]},{"osd":2,"last update":"Sun Oct 8 00:19:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56200000000000006,"15min":0.54800000000000004},"min":{"1min":0.44600000000000001,"5min":0.34599999999999997,"15min":0.34200000000000003},"max":{"1min":0.74299999999999999,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.55800000000000005,"15min":0.53500000000000003},"min":{"1min":0.39000000000000001,"5min":0.34499999999999997,"15min":0.32900000000000001},"max":{"1min":0.69899999999999995,"5min":0.77200000000000002,"15min":0.78500000000000003},"last":0.59199999999999997}]},{"osd":3,"last update":"Sun Oct 8 00:19:58 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.55600000000000005,"15min":0.53400000000000003},"min":{"1min":0.40500000000000003,"5min":0.35599999999999998,"15min":0.33700000000000002},"max":{"1min":0.66400000000000003,"5min":0.75600000000000001,"15min":0.79300000000000004},"last":0.496},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.56899999999999995,"15min":0.56299999999999994},"min":{"1min":0.433,"5min":0.371,"15min":0.371},"max":{"1min":0.745,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.50900000000000001}]},{"osd":4,"last update":"Sun Oct 8 00:19:58 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.47699999999999998,"15min":0.505},"min":{"1min":0.30399999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.67900000000000005,"5min":0.755,"15min":0.755},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.51600000000000001,"15min":0.52000000000000002},"min":{"1min":0.38400000000000001,"5min":0.29199999999999998,"15min":0.28199999999999997},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.78900000000000003},"last":0.379}]},{"osd":5,"last update":"Sun Oct 8 00:19:58 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.51100000000000001,"15min":0.53200000000000003},"min":{"1min":0.27800000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.66700000000000004,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.497,"5min":0.54300000000000004,"15min":0.56499999999999995},"min":{"1min":0.32400000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.71299999999999997,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.41499999999999998}]},{"osd":6,"last update":"Sun Oct 8 00:19:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58799999999999997,"15min":0.59199999999999997},"min":{"1min":0.41599999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.754,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.59099999999999997,"15min":0.61199999999999999},"min":{"1min":0.34300000000000003,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.69499999999999995,"5min":0.82999999999999996,"15min":0.91500000000000004},"last":0.56299999999999994}]}]},{"osd":6,"up_from":13,"seq":55834574963,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":917,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194394,"statfs":{"total":200000000,"available":199060448,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.52700000000000002,"15min":0.497},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.29199999999999998},"max":{"1min":0.69799999999999995,"5min":0.71199999999999997,"15min":1.012},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.51800000000000002,"15min":0.627},"min":{"1min":0.36799999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.65800000000000003,"5min":1.0529999999999999,"15min":5.1230000000000002},"last":0.42899999999999999}]},{"osd":1,"last update":"Sun Oct 8 00:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.54700000000000004,"15min":0.53000000000000003},"min":{"1min":0.39300000000000002,"5min":0.29899999999999999,"15min":0.28899999999999998},"max":{"1min":0.68600000000000005,"5min":0.78100000000000003,"15min":1.1419999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.57699999999999996,"15min":0.56799999999999995},"min":{"1min":0.435,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.70199999999999996,"5min":0.98999999999999999,"15min":1.1539999999999999},"last":0.51600000000000001}]},{"osd":2,"last update":"Sun Oct 8 00:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54200000000000004,"15min":0.53400000000000003},"min":{"1min":0.40400000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.70699999999999996,"5min":0.76500000000000001,"15min":1.3149999999999999},"last":0.373},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57599999999999996,"15min":0.56999999999999995},"min":{"1min":0.45200000000000001,"5min":0.38900000000000001,"15min":0.29399999999999998},"max":{"1min":0.72699999999999998,"5min":0.81699999999999995,"15min":1.3280000000000001},"last":0.54700000000000004}]},{"osd":3,"last update":"Sun Oct 8 00:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.57099999999999995,"15min":0.54800000000000004},"min":{"1min":0.41699999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.72799999999999998,"5min":0.84199999999999997,"15min":1.046},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56499999999999995,"15min":0.55100000000000005},"min":{"1min":0.38600000000000001,"5min":0.35099999999999998,"15min":0.32900000000000001},"max":{"1min":0.71399999999999997,"5min":0.81000000000000005,"15min":1.4390000000000001},"last":0.53200000000000003}]},{"osd":4,"last update":"Sun Oct 8 00:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.46000000000000002,"15min":0.48599999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.52300000000000002,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.54600000000000004,"15min":0.55700000000000005},"min":{"1min":0.375,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.621,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.56599999999999995}]},{"osd":5,"last update":"Sun Oct 8 00:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.46999999999999997,"15min":0.48199999999999998},"min":{"1min":0.312,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.51900000000000002,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.502,"15min":0.53700000000000003},"min":{"1min":0.313,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.51900000000000002,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.624}]},{"osd":7,"last update":"Sun Oct 8 00:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.50600000000000001,"15min":0.52900000000000003},"min":{"1min":0.32800000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.57699999999999996,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.52800000000000002,"15min":0.54100000000000004},"min":{"1min":0.35399999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.56699999999999995,"5min":0.76000000000000001,"15min":0.79900000000000004},"last":0.57999999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574963,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1494,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193818,"statfs":{"total":200000000,"available":198470010,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.35599999999999998,"15min":0.32600000000000001},"min":{"1min":0.216,"5min":0.20300000000000001,"15min":0.19500000000000001},"max":{"1min":0.54400000000000004,"5min":0.55900000000000005,"15min":0.56899999999999995},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.41499999999999998,"15min":0.39000000000000001},"min":{"1min":0.308,"5min":0.27300000000000002,"15min":0.255},"max":{"1min":0.58799999999999997,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.28100000000000003}]},{"osd":2,"last update":"Sun Oct 8 00:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.374,"15min":0.34799999999999998},"min":{"1min":0.25900000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39600000000000002,"15min":0.36199999999999999},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.24299999999999999}]},{"osd":3,"last update":"Sun Oct 8 00:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.41599999999999998,"15min":0.38700000000000001},"min":{"1min":0.27600000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.60399999999999998,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.307},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.36899999999999999,"15min":0.35199999999999998},"min":{"1min":0.26200000000000001,"5min":0.249,"15min":0.222},"max":{"1min":0.56100000000000005,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.26400000000000001}]},{"osd":4,"last update":"Sun Oct 8 00:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.56299999999999994,"15min":0.54900000000000004},"min":{"1min":0.35999999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.80900000000000005,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.58899999999999997,"15min":0.57499999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.77100000000000002,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.495}]},{"osd":5,"last update":"Sun Oct 8 00:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.58099999999999996,"15min":0.55300000000000005},"min":{"1min":0.35099999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.73399999999999999,"5min":0.86499999999999999,"15min":0.88200000000000001},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.59899999999999998,"15min":0.58399999999999996},"min":{"1min":0.42699999999999999,"5min":0.38300000000000001,"15min":0.30399999999999999},"max":{"1min":0.83099999999999996,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.55300000000000005}]},{"osd":6,"last update":"Sun Oct 8 00:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58899999999999997,"15min":0.57599999999999996},"min":{"1min":0.38600000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.745,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58799999999999997,"15min":0.57199999999999995},"min":{"1min":0.377,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77900000000000003,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.503}]},{"osd":7,"last update":"Sun Oct 8 00:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.58699999999999997,"15min":0.57299999999999995},"min":{"1min":0.39400000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.67000000000000004,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.625,"15min":0.60499999999999998},"min":{"1min":0.45800000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.78200000000000003,"5min":0.89500000000000002,"15min":0.96699999999999997},"last":0.53100000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574963,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":917,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194394,"statfs":{"total":200000000,"available":199060358,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 00:20:05 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34499999999999997,"15min":0.33600000000000002},"min":{"1min":0.23899999999999999,"5min":0.20399999999999999,"15min":0.193},"max":{"1min":0.504,"5min":0.57599999999999996,"15min":1.0640000000000001},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.36899999999999999,"15min":0.35499999999999998},"min":{"1min":0.216,"5min":0.216,"15min":0.20300000000000001},"max":{"1min":0.45900000000000002,"5min":0.66500000000000004,"15min":0.87},"last":0.29799999999999999}]},{"osd":2,"last update":"Sun Oct 8 00:20:05 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.39600000000000002,"15min":0.38900000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.24099999999999999},"max":{"1min":0.56499999999999995,"5min":0.623,"15min":0.93799999999999994},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.36299999999999999,"15min":0.59399999999999997},"min":{"1min":0.23499999999999999,"5min":0.23000000000000001,"15min":0.185},"max":{"1min":0.47399999999999998,"5min":0.65600000000000003,"15min":12.019},"last":0.28499999999999998}]},{"osd":3,"last update":"Sun Oct 8 00:20:05 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.38400000000000001,"15min":0.37},"min":{"1min":0.30199999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.55800000000000005,"5min":0.628,"15min":1.1020000000000001},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.42599999999999999,"15min":0.40699999999999997},"min":{"1min":0.28699999999999998,"5min":0.26700000000000002,"15min":0.246},"max":{"1min":0.58299999999999996,"5min":0.67400000000000004,"15min":1.0409999999999999},"last":0.374}]},{"osd":4,"last update":"Sun Oct 8 00:20:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.626,"15min":0.62},"min":{"1min":0.44500000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.875,"5min":0.90300000000000002,"15min":1.1779999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.628,"15min":0.626},"min":{"1min":0.41599999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.88700000000000001,"5min":0.93899999999999995,"15min":0.96199999999999997},"last":0.82499999999999996}]},{"osd":5,"last update":"Sun Oct 8 00:20:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.59099999999999997,"15min":0.83099999999999996},"min":{"1min":0.378,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":11.992000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60999999999999999,"15min":0.59599999999999997},"min":{"1min":0.38400000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.84099999999999997,"5min":0.91600000000000004,"15min":1.1299999999999999},"last":0.76900000000000002}]},{"osd":6,"last update":"Sun Oct 8 00:20:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.60499999999999998,"15min":0.59399999999999997},"min":{"1min":0.41799999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.755},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60999999999999999,"15min":0.82499999999999996},"min":{"1min":0.439,"5min":0.39600000000000002,"15min":0.38100000000000001},"max":{"1min":0.85599999999999998,"5min":0.91200000000000003,"15min":11.927},"last":0.84499999999999997}]},{"osd":7,"last update":"Sun Oct 8 00:20:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.61299999999999999,"15min":0.83799999999999997},"min":{"1min":0.443,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":11.965},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63500000000000001,"15min":0.628},"min":{"1min":0.48399999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.86399999999999999,"5min":0.88,"15min":1.1539999999999999},"last":0.80300000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574963,"num_pgs":31,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":917,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194394,"statfs":{"total":200000000,"available":199060426,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.33700000000000002,"15min":0.371},"min":{"1min":0.245,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.46899999999999997,"5min":0.54000000000000004,"15min":2.5139999999999998},"last":0.309},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.34699999999999998,"15min":0.38100000000000001},"min":{"1min":0.26400000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":2.1309999999999998},"last":0.33700000000000002}]},{"osd":1,"last update":"Sun Oct 8 00:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.36299999999999999,"15min":0.38},"min":{"1min":0.26800000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.53900000000000003,"5min":0.67300000000000004,"15min":2.2639999999999998},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.379,"15min":0.41499999999999998},"min":{"1min":0.30399999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.57199999999999995,"5min":0.64400000000000002,"15min":2.5379999999999998},"last":0.42799999999999999}]},{"osd":3,"last update":"Sun Oct 8 00:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.40100000000000002,"15min":0.41899999999999998},"min":{"1min":0.34399999999999997,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":2.2890000000000001},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.41899999999999998,"15min":0.44600000000000001},"min":{"1min":0.34200000000000003,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.63400000000000001,"5min":0.68799999999999994,"15min":2.5619999999999998},"last":0.39400000000000002}]},{"osd":4,"last update":"Sun Oct 8 00:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.55600000000000005,"15min":0.59499999999999997},"min":{"1min":0.42299999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79100000000000004,"5min":0.81499999999999995,"15min":2.488},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.59599999999999997,"15min":0.60199999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.753,"5min":0.92400000000000004,"15min":2.073},"last":0.80300000000000005}]},{"osd":5,"last update":"Sun Oct 8 00:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.61599999999999999,"15min":0.64300000000000002},"min":{"1min":0.44700000000000001,"5min":0.36899999999999999,"15min":0.36099999999999999},"max":{"1min":0.874,"5min":0.874,"15min":2.2069999999999999},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.628,"15min":0.65100000000000002},"min":{"1min":0.47199999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.85399999999999998,"5min":0.94199999999999995,"15min":2.456},"last":0.88}]},{"osd":6,"last update":"Sun Oct 8 00:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.59899999999999998,"15min":0.61499999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83499999999999996,"5min":0.93000000000000005,"15min":2.0339999999999998},"last":0.749},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.61199999999999999,"15min":0.61099999999999999},"min":{"1min":0.45300000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.78200000000000003,"5min":0.94099999999999995,"15min":2.1699999999999999},"last":0.59199999999999997}]},{"osd":7,"last update":"Sun Oct 8 00:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.629},"min":{"1min":0.46300000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.73899999999999999,"5min":0.95999999999999996,"15min":2.1030000000000002},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61799999999999999,"15min":0.624},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40300000000000002},"max":{"1min":0.83799999999999997,"5min":0.90500000000000003,"15min":2.2349999999999999},"last":1.0169999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574963,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":918,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194394,"statfs":{"total":200000000,"available":199059860,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.33800000000000002,"15min":0.35499999999999998},"min":{"1min":0.245,"5min":0.192,"15min":0.192},"max":{"1min":0.46400000000000002,"5min":0.71199999999999997,"15min":1.46},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.35299999999999998,"15min":0.34399999999999997},"min":{"1min":0.28100000000000003,"5min":0.20399999999999999,"15min":0.192},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.32100000000000001}]},{"osd":1,"last update":"Sun Oct 8 00:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.36099999999999999,"15min":0.63700000000000001},"min":{"1min":0.27900000000000003,"5min":0.22700000000000001,"15min":0.20000000000000001},"max":{"1min":0.53500000000000003,"5min":0.57499999999999996,"15min":12.351000000000001},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.38,"15min":0.37},"min":{"1min":0.27100000000000002,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.55100000000000005,"5min":0.60199999999999998,"15min":0.71999999999999997},"last":0.30499999999999999}]},{"osd":2,"last update":"Sun Oct 8 00:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.40500000000000003,"15min":0.41199999999999998},"min":{"1min":0.307,"5min":0.26800000000000002,"15min":0.23499999999999999},"max":{"1min":0.59699999999999998,"5min":0.60899999999999999,"15min":1.5489999999999999},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.41299999999999998,"15min":0.39500000000000002},"min":{"1min":0.34300000000000003,"5min":0.25700000000000001,"15min":0.246},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.753},"last":0.34999999999999998}]},{"osd":4,"last update":"Sun Oct 8 00:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.63100000000000001,"15min":0.85199999999999998},"min":{"1min":0.377,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.95499999999999996,"5min":1.024,"15min":11.333},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.63300000000000001,"15min":0.874},"min":{"1min":0.41599999999999998,"5min":0.38100000000000001,"15min":0.36099999999999999},"max":{"1min":0.85199999999999998,"5min":0.97699999999999998,"15min":12.321999999999999},"last":0.51300000000000001}]},{"osd":5,"last update":"Sun Oct 8 00:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.622,"15min":0.876},"min":{"1min":0.44800000000000001,"5min":0.38300000000000001,"15min":0.35499999999999998},"max":{"1min":0.90200000000000002,"5min":0.93500000000000005,"15min":12.291},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.64800000000000002,"15min":0.61499999999999999},"min":{"1min":0.42299999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.36799999999999999}]},{"osd":6,"last update":"Sun Oct 8 00:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.60499999999999998,"15min":0.82299999999999995},"min":{"1min":0.46000000000000002,"5min":0.35299999999999998,"15min":0.33700000000000002},"max":{"1min":0.81799999999999995,"5min":0.91500000000000004,"15min":11.288},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.61799999999999999,"15min":0.86299999999999999},"min":{"1min":0.432,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.94599999999999995,"5min":1.002,"15min":12.253},"last":0.42299999999999999}]},{"osd":7,"last update":"Sun Oct 8 00:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.628,"15min":0.59499999999999997},"min":{"1min":0.44500000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.3109999999999999},"last":0.505},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.61899999999999999,"15min":0.85199999999999998},"min":{"1min":0.436,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.89300000000000002,"5min":1.0549999999999999,"15min":11.363},"last":0.434}]}]},{"osd":4,"up_from":13,"seq":55834574963,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":919,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194392,"statfs":{"total":200000000,"available":199058336,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.58399999999999996,"15min":0.56499999999999995},"min":{"1min":0.45800000000000002,"5min":0.193,"15min":0.193},"max":{"1min":0.76800000000000002,"5min":0.84799999999999998,"15min":0.93000000000000005},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60499999999999998,"15min":0.58999999999999997},"min":{"1min":0.438,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.79100000000000004,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.52900000000000003}]},{"osd":1,"last update":"Sun Oct 8 00:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55800000000000005,"15min":0.53100000000000003},"min":{"1min":0.42299999999999999,"5min":0.33900000000000002,"15min":0.33700000000000002},"max":{"1min":0.751,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56999999999999995,"15min":0.54400000000000004},"min":{"1min":0.39500000000000002,"5min":0.314,"15min":0.30199999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.81299999999999994},"last":0.499}]},{"osd":2,"last update":"Sun Oct 8 00:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.55200000000000005,"15min":0.52800000000000002},"min":{"1min":0.436,"5min":0.32500000000000001,"15min":0.32100000000000001},"max":{"1min":0.84799999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56799999999999995,"15min":0.53100000000000003},"min":{"1min":0.46800000000000003,"5min":0.35399999999999998,"15min":0.32600000000000001},"max":{"1min":0.73199999999999998,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.435}]},{"osd":3,"last update":"Sun Oct 8 00:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.57699999999999996,"15min":0.54900000000000004},"min":{"1min":0.441,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.72699999999999998,"5min":0.82299999999999995,"15min":0.85399999999999998},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.54200000000000004},"min":{"1min":0.40899999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.74099999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.38100000000000001}]},{"osd":5,"last update":"Sun Oct 8 00:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47799999999999998,"15min":0.48599999999999999},"min":{"1min":0.33600000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.51100000000000001,"15min":0.50700000000000001},"min":{"1min":0.33000000000000002,"5min":0.26400000000000001,"15min":0.24399999999999999},"max":{"1min":0.79100000000000004,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.317}]},{"osd":6,"last update":"Sun Oct 8 00:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48999999999999999,"15min":0.48699999999999999},"min":{"1min":0.34699999999999998,"5min":0.32200000000000001,"15min":0.26400000000000001},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.53700000000000003,"15min":0.54300000000000004},"min":{"1min":0.34999999999999998,"5min":0.34799999999999998,"15min":0.29799999999999999},"max":{"1min":0.76200000000000001,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.46300000000000002}]},{"osd":7,"last update":"Sun Oct 8 00:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.54500000000000004,"15min":0.54600000000000004},"min":{"1min":0.37,"5min":0.33200000000000002,"15min":0.27900000000000003},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.90100000000000002},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57999999999999996,"15min":0.57799999999999996},"min":{"1min":0.39500000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.51400000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574963,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1495,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193816,"statfs":{"total":200000000,"available":198468350,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.53400000000000003,"15min":0.502},"min":{"1min":0.44500000000000001,"5min":0.28799999999999998,"15min":0.249},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.54400000000000004,"15min":0.55200000000000005},"min":{"1min":0.38400000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.88600000000000001},"last":0.47699999999999998}]},{"osd":1,"last update":"Sun Oct 8 00:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.55200000000000005,"15min":0.53300000000000003},"min":{"1min":0.45800000000000002,"5min":0.34399999999999997,"15min":0.32500000000000001},"max":{"1min":0.72599999999999998,"5min":0.80600000000000005,"15min":0.82999999999999996},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.52900000000000003,"15min":0.52400000000000002},"min":{"1min":0.41199999999999998,"5min":0.33100000000000002,"15min":0.33000000000000002},"max":{"1min":0.67900000000000005,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.495}]},{"osd":2,"last update":"Sun Oct 8 00:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.54400000000000004,"15min":0.53400000000000003},"min":{"1min":0.40799999999999997,"5min":0.34499999999999997,"15min":0.32400000000000001},"max":{"1min":0.70699999999999996,"5min":0.749,"15min":0.86199999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.55100000000000005,"15min":0.56200000000000006},"min":{"1min":0.41899999999999998,"5min":0.33400000000000002,"15min":0.29599999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.95399999999999996},"last":0.45700000000000002}]},{"osd":3,"last update":"Sun Oct 8 00:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.56299999999999994,"15min":0.55400000000000005},"min":{"1min":0.42199999999999999,"5min":0.35499999999999998,"15min":0.316},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.622},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55900000000000005,"15min":0.57099999999999995},"min":{"1min":0.45600000000000002,"5min":0.376,"15min":0.371},"max":{"1min":0.69999999999999996,"5min":0.73499999999999999,"15min":1.0529999999999999},"last":0.55900000000000005}]},{"osd":4,"last update":"Sun Oct 8 00:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.47799999999999998,"15min":0.47399999999999998},"min":{"1min":0.307,"5min":0.28399999999999997,"15min":0.27700000000000002},"max":{"1min":0.60499999999999998,"5min":0.78600000000000003,"15min":0.79200000000000004},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.504,"5min":0.52600000000000002,"15min":0.51700000000000002},"min":{"1min":0.371,"5min":0.314,"15min":0.28899999999999998},"max":{"1min":0.77500000000000002,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.52700000000000002}]},{"osd":6,"last update":"Sun Oct 8 00:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.50800000000000001,"15min":0.505},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.27400000000000002},"max":{"1min":0.66900000000000004,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.52600000000000002,"15min":0.50600000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.30499999999999999},"max":{"1min":0.66700000000000004,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.64400000000000002}]},{"osd":7,"last update":"Sun Oct 8 00:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.54000000000000004,"15min":0.52600000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.29399999999999998},"max":{"1min":0.72299999999999998,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.56399999999999995,"15min":0.56000000000000005},"min":{"1min":0.35599999999999998,"5min":0.34499999999999997,"15min":0.32500000000000001},"max":{"1min":0.68000000000000005,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.63200000000000001}]}]}],"pool_statfs":[]}} 2023-10-08T00:20:09.307 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T00:20:09.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.452+0000 7f05b0886700 1 Processor -- start 2023-10-08T00:20:09.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.452+0000 7f05b0886700 1 -- start start 2023-10-08T00:20:09.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.453+0000 7f05b0886700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f05ac0cd6e0 0x7f05ac0cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:09.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.453+0000 7f05b0886700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f05ac0cdab0 0x7f05ac15d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:09.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.453+0000 7f05b0886700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05ac0cb7c0 con 0x7f05ac15dda0 2023-10-08T00:20:09.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.453+0000 7f05b0886700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05ac15e850 con 0x7f05ac0cdab0 2023-10-08T00:20:09.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.453+0000 7f05b0886700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05ac0cd030 con 0x7f05ac0cd6e0 2023-10-08T00:20:09.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.453+0000 7f05a259c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f05ac0cdab0 0x7f05ac15d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:09.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.453+0000 7f05a259c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f05ac0cdab0 0x7f05ac15d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60922/0 (socket says 172.21.15.99:60922) 2023-10-08T00:20:09.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.453+0000 7f05a259c700 1 -- 172.21.15.99:0/4290721049 learned_addr learned my addr 172.21.15.99:0/4290721049 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:20:09.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.453+0000 7f05aad9d700 1 --2- 172.21.15.99:0/4290721049 >> v2:172.21.15.191:3300/0 conn(0x7f05ac0cd6e0 0x7f05ac0cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:09.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.453+0000 7f05a259c700 1 -- 172.21.15.99:0/4290721049 >> v2:172.21.15.191:3300/0 conn(0x7f05ac0cd6e0 msgr2=0x7f05ac0cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:09.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.453+0000 7f05a259c700 1 --2- 172.21.15.99:0/4290721049 >> v2:172.21.15.191:3300/0 conn(0x7f05ac0cd6e0 0x7f05ac0cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:09.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.453+0000 7f05a259c700 1 -- 172.21.15.99:0/4290721049 >> v1:172.21.15.191:6789/0 conn(0x7f05ac15dda0 legacy=0x7f05ac15cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:09.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.453+0000 7f05a259c700 1 -- 172.21.15.99:0/4290721049 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ac15e290 con 0x7f05ac0cdab0 2023-10-08T00:20:09.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.453+0000 7f05a259c700 1 --2- 172.21.15.99:0/4290721049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05ac0cdab0 0x7f05ac15d860 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f05ac06b730 tx=0x7f0594009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1a2e04ef1e0ab5c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:09.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.454+0000 7f05aa59c700 1 -- 172.21.15.99:0/4290721049 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059400b860 con 0x7f05ac0cdab0 2023-10-08T00:20:09.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.454+0000 7f05aa59c700 1 -- 172.21.15.99:0/4290721049 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f059400b9c0 con 0x7f05ac0cdab0 2023-10-08T00:20:09.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.454+0000 7f05aa59c700 1 -- 172.21.15.99:0/4290721049 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059400bd60 con 0x7f05ac0cdab0 2023-10-08T00:20:09.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.454+0000 7f05b0886700 1 -- 172.21.15.99:0/4290721049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05ac0cdab0 msgr2=0x7f05ac15d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:09.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.454+0000 7f05b0886700 1 --2- 172.21.15.99:0/4290721049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05ac0cdab0 0x7f05ac15d860 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f05ac06b730 tx=0x7f0594009250 comp rx=0 tx=0).stop 2023-10-08T00:20:09.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.454+0000 7f05b0886700 1 -- 172.21.15.99:0/4290721049 shutdown_connections 2023-10-08T00:20:09.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.454+0000 7f05b0886700 1 --2- 172.21.15.99:0/4290721049 >> v2:172.21.15.191:3300/0 conn(0x7f05ac0cd6e0 0x7f05ac0cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:09.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.454+0000 7f05b0886700 1 --2- 172.21.15.99:0/4290721049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05ac0cdab0 0x7f05ac15d860 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:09.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.454+0000 7f05b0886700 1 -- 172.21.15.99:0/4290721049 >> 172.21.15.99:0/4290721049 conn(0x7f05ac067660 msgr2=0x7f05ac067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:20:09.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.454+0000 7f05b0886700 1 -- 172.21.15.99:0/4290721049 shutdown_connections 2023-10-08T00:20:09.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.454+0000 7f05b0886700 1 -- 172.21.15.99:0/4290721049 wait complete. 2023-10-08T00:20:09.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.455+0000 7f05b0886700 1 Processor -- start 2023-10-08T00:20:09.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.455+0000 7f05b0886700 1 -- start start 2023-10-08T00:20:09.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.456+0000 7f05b0886700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05ac0cd6e0 0x7f05ac0b3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:09.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.456+0000 7f05b0886700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05ac0cdab0 0x7f05ac0b41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:09.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.456+0000 7f05b0886700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05ac15dda0 0x7f05ac0b4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:09.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.456+0000 7f05b0886700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac16add0 con 0x7f05ac0cdab0 2023-10-08T00:20:09.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.456+0000 7f05b0886700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac0b12f0 con 0x7f05ac0cd6e0 2023-10-08T00:20:09.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.456+0000 7f05b0886700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac14ad80 con 0x7f05ac15dda0 2023-10-08T00:20:09.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.456+0000 7f05a259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05ac0cdab0 0x7f05ac0b41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:09.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.456+0000 7f05a259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05ac0cdab0 0x7f05ac0b41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60936/0 (socket says 172.21.15.99:60936) 2023-10-08T00:20:09.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.456+0000 7f05a259c700 1 -- 172.21.15.99:0/788120944 learned_addr learned my addr 172.21.15.99:0/788120944 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:20:09.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.456+0000 7f05ab59e700 1 --2- 172.21.15.99:0/788120944 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05ac15dda0 0x7f05ac0b4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:09.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.456+0000 7f05aad9d700 1 --2- 172.21.15.99:0/788120944 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05ac0cd6e0 0x7f05ac0b3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:09.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.456+0000 7f05a259c700 1 -- 172.21.15.99:0/788120944 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05ac15dda0 msgr2=0x7f05ac0b4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:09.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.456+0000 7f05a259c700 1 --2- 172.21.15.99:0/788120944 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05ac15dda0 0x7f05ac0b4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:09.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.457+0000 7f05a259c700 1 -- 172.21.15.99:0/788120944 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05ac0cd6e0 msgr2=0x7f05ac0b3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:09.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.457+0000 7f05a259c700 1 --2- 172.21.15.99:0/788120944 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05ac0cd6e0 0x7f05ac0b3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:09.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.457+0000 7f05a259c700 1 -- 172.21.15.99:0/788120944 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05940094f0 con 0x7f05ac0cdab0 2023-10-08T00:20:09.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.457+0000 7f05a259c700 1 --2- 172.21.15.99:0/788120944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05ac0cdab0 0x7f05ac0b41f0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f05ac0c7b10 tx=0x7f0594017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:09.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.457+0000 7f05a37fe700 1 -- 172.21.15.99:0/788120944 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0594011e00 con 0x7f05ac0cdab0 2023-10-08T00:20:09.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.457+0000 7f05a37fe700 1 -- 172.21.15.99:0/788120944 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0594026450 con 0x7f05ac0cdab0 2023-10-08T00:20:09.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.457+0000 7f05b0886700 1 -- 172.21.15.99:0/788120944 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05ac0c80e0 con 0x7f05ac0cdab0 2023-10-08T00:20:09.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.457+0000 7f05a37fe700 1 -- 172.21.15.99:0/788120944 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05940267f0 con 0x7f05ac0cdab0 2023-10-08T00:20:09.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.457+0000 7f05b0886700 1 -- 172.21.15.99:0/788120944 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05ac0b1840 con 0x7f05ac0cdab0 2023-10-08T00:20:09.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.458+0000 7f05b0886700 1 -- 172.21.15.99:0/788120944 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05ac0d00c0 con 0x7f05ac0cdab0 2023-10-08T00:20:09.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.458+0000 7f05a37fe700 1 -- 172.21.15.99:0/788120944 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0594026c20 con 0x7f05ac0cdab0 2023-10-08T00:20:09.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.458+0000 7f05a37fe700 1 --2- 172.21.15.99:0/788120944 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f058c066f30 0x7f058c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:09.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.458+0000 7f05a37fe700 1 -- 172.21.15.99:0/788120944 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0594086250 con 0x7f05ac0cdab0 2023-10-08T00:20:09.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.459+0000 7f05aad9d700 1 --2- 172.21.15.99:0/788120944 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f058c066f30 0x7f058c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:09.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.459+0000 7f05aad9d700 1 --2- 172.21.15.99:0/788120944 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f058c066f30 0x7f058c0693e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f059c006fd0 tx=0x7f059c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:09.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.463+0000 7f05a37fe700 1 -- 172.21.15.99:0/788120944 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0594053540 con 0x7f05ac0cdab0 2023-10-08T00:20:09.640 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.640+0000 7f05b0886700 1 -- 172.21.15.99:0/788120944 --> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f05ac15e570 con 0x7f058c066f30 2023-10-08T00:20:09.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.648+0000 7f05a37fe700 1 -- 172.21.15.99:0/788120944 <== mgr.4099 v2:172.21.15.191:6832/85985 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285556 (secure 0 0 0) 0x7f05ac15e570 con 0x7f058c066f30 2023-10-08T00:20:09.653 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:20:09.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.651+0000 7f05b0886700 1 -- 172.21.15.99:0/788120944 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f058c066f30 msgr2=0x7f058c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:09.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.651+0000 7f05b0886700 1 --2- 172.21.15.99:0/788120944 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f058c066f30 0x7f058c0693e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f059c006fd0 tx=0x7f059c008040 comp rx=0 tx=0).stop 2023-10-08T00:20:09.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.651+0000 7f05b0886700 1 -- 172.21.15.99:0/788120944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05ac0cdab0 msgr2=0x7f05ac0b41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:09.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.651+0000 7f05b0886700 1 --2- 172.21.15.99:0/788120944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05ac0cdab0 0x7f05ac0b41f0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f05ac0c7b10 tx=0x7f0594017000 comp rx=0 tx=0).stop 2023-10-08T00:20:09.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.651+0000 7f05b0886700 1 -- 172.21.15.99:0/788120944 shutdown_connections 2023-10-08T00:20:09.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.651+0000 7f05b0886700 1 --2- 172.21.15.99:0/788120944 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f058c066f30 0x7f058c0693e0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:09.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.651+0000 7f05b0886700 1 --2- 172.21.15.99:0/788120944 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05ac0cd6e0 0x7f05ac0b3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:09.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.651+0000 7f05b0886700 1 --2- 172.21.15.99:0/788120944 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05ac15dda0 0x7f05ac0b4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:09.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.651+0000 7f05b0886700 1 --2- 172.21.15.99:0/788120944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05ac0cdab0 0x7f05ac0b41f0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:09.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.651+0000 7f05b0886700 1 -- 172.21.15.99:0/788120944 >> 172.21.15.99:0/788120944 conn(0x7f05ac067660 msgr2=0x7f05ac0d1500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:20:09.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.651+0000 7f05b0886700 1 -- 172.21.15.99:0/788120944 shutdown_connections 2023-10-08T00:20:09.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.651+0000 7f05b0886700 1 -- 172.21.15.99:0/788120944 wait complete. 2023-10-08T00:20:09.656 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2023-10-08T00:20:09.664 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":441,"stamp":"2023-10-08T00:20:08.156351+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":161,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":8497,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1554002,"statfs":{"total":1600000000,"available":1591298236,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.920357"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.251793+0000","last_change":"2023-10-08T00:20:05.249821+0000","last_active":"2023-10-08T00:20:05.251793+0000","last_peered":"2023-10-08T00:20:05.251793+0000","last_clean":"2023-10-08T00:20:05.251793+0000","last_became_active":"2023-10-08T00:20:05.249607+0000","last_became_peered":"2023-10-08T00:20:05.249607+0000","last_unstale":"2023-10-08T00:20:05.251793+0000","last_undegraded":"2023-10-08T00:20:05.251793+0000","last_fullsized":"2023-10-08T00:20:05.251793+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:54:14.710613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.246458+0000","last_change":"2023-10-08T00:20:05.246248+0000","last_active":"2023-10-08T00:20:05.246458+0000","last_peered":"2023-10-08T00:20:05.246458+0000","last_clean":"2023-10-08T00:20:05.246458+0000","last_became_active":"2023-10-08T00:20:05.245992+0000","last_became_peered":"2023-10-08T00:20:05.245992+0000","last_unstale":"2023-10-08T00:20:05.246458+0000","last_undegraded":"2023-10-08T00:20:05.246458+0000","last_fullsized":"2023-10-08T00:20:05.246458+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:24:49.076952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.245864+0000","last_change":"2023-10-08T00:20:05.245791+0000","last_active":"2023-10-08T00:20:05.245864+0000","last_peered":"2023-10-08T00:20:05.245864+0000","last_clean":"2023-10-08T00:20:05.245864+0000","last_became_active":"2023-10-08T00:20:05.245250+0000","last_became_peered":"2023-10-08T00:20:05.245250+0000","last_unstale":"2023-10-08T00:20:05.245864+0000","last_undegraded":"2023-10-08T00:20:05.245864+0000","last_fullsized":"2023-10-08T00:20:05.245864+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:43:26.454384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.247225+0000","last_change":"2023-10-08T00:20:05.245007+0000","last_active":"2023-10-08T00:20:05.247225+0000","last_peered":"2023-10-08T00:20:05.247225+0000","last_clean":"2023-10-08T00:20:05.247225+0000","last_became_active":"2023-10-08T00:20:05.244000+0000","last_became_peered":"2023-10-08T00:20:05.244000+0000","last_unstale":"2023-10-08T00:20:05.247225+0000","last_undegraded":"2023-10-08T00:20:05.247225+0000","last_fullsized":"2023-10-08T00:20:05.247225+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:18:31.037964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.243605+0000","last_change":"2023-10-08T00:20:05.243520+0000","last_active":"2023-10-08T00:20:05.243605+0000","last_peered":"2023-10-08T00:20:05.243605+0000","last_clean":"2023-10-08T00:20:05.243605+0000","last_became_active":"2023-10-08T00:20:05.243299+0000","last_became_peered":"2023-10-08T00:20:05.243299+0000","last_unstale":"2023-10-08T00:20:05.243605+0000","last_undegraded":"2023-10-08T00:20:05.243605+0000","last_fullsized":"2023-10-08T00:20:05.243605+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:51:39.795897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.249393+0000","last_change":"2023-10-08T00:20:05.249299+0000","last_active":"2023-10-08T00:20:05.249393+0000","last_peered":"2023-10-08T00:20:05.249393+0000","last_clean":"2023-10-08T00:20:05.249393+0000","last_became_active":"2023-10-08T00:20:05.249074+0000","last_became_peered":"2023-10-08T00:20:05.249074+0000","last_unstale":"2023-10-08T00:20:05.249393+0000","last_undegraded":"2023-10-08T00:20:05.249393+0000","last_fullsized":"2023-10-08T00:20:05.249393+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:49:32.653899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.242476+0000","last_change":"2023-10-08T00:20:05.241833+0000","last_active":"2023-10-08T00:20:05.242476+0000","last_peered":"2023-10-08T00:20:05.242476+0000","last_clean":"2023-10-08T00:20:05.242476+0000","last_became_active":"2023-10-08T00:20:05.241605+0000","last_became_peered":"2023-10-08T00:20:05.241605+0000","last_unstale":"2023-10-08T00:20:05.242476+0000","last_undegraded":"2023-10-08T00:20:05.242476+0000","last_fullsized":"2023-10-08T00:20:05.242476+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:52:47.931876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.254471+0000","last_change":"2023-10-08T00:20:05.252822+0000","last_active":"2023-10-08T00:20:05.254471+0000","last_peered":"2023-10-08T00:20:05.254471+0000","last_clean":"2023-10-08T00:20:05.254471+0000","last_became_active":"2023-10-08T00:20:05.252594+0000","last_became_peered":"2023-10-08T00:20:05.252594+0000","last_unstale":"2023-10-08T00:20:05.254471+0000","last_undegraded":"2023-10-08T00:20:05.254471+0000","last_fullsized":"2023-10-08T00:20:05.254471+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:49:32.653899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.245665+0000","last_change":"2023-10-08T00:20:05.245584+0000","last_active":"2023-10-08T00:20:05.245665+0000","last_peered":"2023-10-08T00:20:05.245665+0000","last_clean":"2023-10-08T00:20:05.245665+0000","last_became_active":"2023-10-08T00:20:05.245322+0000","last_became_peered":"2023-10-08T00:20:05.245322+0000","last_unstale":"2023-10-08T00:20:05.245665+0000","last_undegraded":"2023-10-08T00:20:05.245665+0000","last_fullsized":"2023-10-08T00:20:05.245665+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:34:06.226802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.258231+0000","last_change":"2023-10-08T00:20:05.258154+0000","last_active":"2023-10-08T00:20:05.258231+0000","last_peered":"2023-10-08T00:20:05.258231+0000","last_clean":"2023-10-08T00:20:05.258231+0000","last_became_active":"2023-10-08T00:20:05.257941+0000","last_became_peered":"2023-10-08T00:20:05.257941+0000","last_unstale":"2023-10-08T00:20:05.258231+0000","last_undegraded":"2023-10-08T00:20:05.258231+0000","last_fullsized":"2023-10-08T00:20:05.258231+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:22:41.037733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.246728+0000","last_change":"2023-10-08T00:20:05.245663+0000","last_active":"2023-10-08T00:20:05.246728+0000","last_peered":"2023-10-08T00:20:05.246728+0000","last_clean":"2023-10-08T00:20:05.246728+0000","last_became_active":"2023-10-08T00:20:05.245433+0000","last_became_peered":"2023-10-08T00:20:05.245433+0000","last_unstale":"2023-10-08T00:20:05.246728+0000","last_undegraded":"2023-10-08T00:20:05.246728+0000","last_fullsized":"2023-10-08T00:20:05.246728+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:55:15.548941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.259588+0000","last_change":"2023-10-08T00:20:05.259500+0000","last_active":"2023-10-08T00:20:05.259588+0000","last_peered":"2023-10-08T00:20:05.259588+0000","last_clean":"2023-10-08T00:20:05.259588+0000","last_became_active":"2023-10-08T00:20:05.259056+0000","last_became_peered":"2023-10-08T00:20:05.259056+0000","last_unstale":"2023-10-08T00:20:05.259588+0000","last_undegraded":"2023-10-08T00:20:05.259588+0000","last_fullsized":"2023-10-08T00:20:05.259588+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:44:29.905006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.254911+0000","last_change":"2023-10-08T00:20:05.250458+0000","last_active":"2023-10-08T00:20:05.254911+0000","last_peered":"2023-10-08T00:20:05.254911+0000","last_clean":"2023-10-08T00:20:05.254911+0000","last_became_active":"2023-10-08T00:20:05.249644+0000","last_became_peered":"2023-10-08T00:20:05.249644+0000","last_unstale":"2023-10-08T00:20:05.254911+0000","last_undegraded":"2023-10-08T00:20:05.254911+0000","last_fullsized":"2023-10-08T00:20:05.254911+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:45:25.075955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.250931+0000","last_change":"2023-10-08T00:20:05.250870+0000","last_active":"2023-10-08T00:20:05.250931+0000","last_peered":"2023-10-08T00:20:05.250931+0000","last_clean":"2023-10-08T00:20:05.250931+0000","last_became_active":"2023-10-08T00:20:05.249895+0000","last_became_peered":"2023-10-08T00:20:05.249895+0000","last_unstale":"2023-10-08T00:20:05.250931+0000","last_undegraded":"2023-10-08T00:20:05.250931+0000","last_fullsized":"2023-10-08T00:20:05.250931+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:49:32.653899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.254949+0000","last_change":"2023-10-08T00:20:05.251645+0000","last_active":"2023-10-08T00:20:05.254949+0000","last_peered":"2023-10-08T00:20:05.254949+0000","last_clean":"2023-10-08T00:20:05.254949+0000","last_became_active":"2023-10-08T00:20:05.251016+0000","last_became_peered":"2023-10-08T00:20:05.251016+0000","last_unstale":"2023-10-08T00:20:05.254949+0000","last_undegraded":"2023-10-08T00:20:05.254949+0000","last_fullsized":"2023-10-08T00:20:05.254949+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:59:01.329001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.242444+0000","last_change":"2023-10-08T00:20:05.241158+0000","last_active":"2023-10-08T00:20:05.242444+0000","last_peered":"2023-10-08T00:20:05.242444+0000","last_clean":"2023-10-08T00:20:05.242444+0000","last_became_active":"2023-10-08T00:20:05.240931+0000","last_became_peered":"2023-10-08T00:20:05.240931+0000","last_unstale":"2023-10-08T00:20:05.242444+0000","last_undegraded":"2023-10-08T00:20:05.242444+0000","last_fullsized":"2023-10-08T00:20:05.242444+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:00:59.451262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.257677+0000","last_change":"2023-10-08T00:20:05.257391+0000","last_active":"2023-10-08T00:20:05.257677+0000","last_peered":"2023-10-08T00:20:05.257677+0000","last_clean":"2023-10-08T00:20:05.257677+0000","last_became_active":"2023-10-08T00:20:05.256940+0000","last_became_peered":"2023-10-08T00:20:05.256940+0000","last_unstale":"2023-10-08T00:20:05.257677+0000","last_undegraded":"2023-10-08T00:20:05.257677+0000","last_fullsized":"2023-10-08T00:20:05.257677+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:52:38.144789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.254875+0000","last_change":"2023-10-08T00:20:05.249866+0000","last_active":"2023-10-08T00:20:05.254875+0000","last_peered":"2023-10-08T00:20:05.254875+0000","last_clean":"2023-10-08T00:20:05.254875+0000","last_became_active":"2023-10-08T00:20:05.249355+0000","last_became_peered":"2023-10-08T00:20:05.249355+0000","last_unstale":"2023-10-08T00:20:05.254875+0000","last_undegraded":"2023-10-08T00:20:05.254875+0000","last_fullsized":"2023-10-08T00:20:05.254875+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:23:44.978067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.246765+0000","last_change":"2023-10-08T00:20:05.246656+0000","last_active":"2023-10-08T00:20:05.246765+0000","last_peered":"2023-10-08T00:20:05.246765+0000","last_clean":"2023-10-08T00:20:05.246765+0000","last_became_active":"2023-10-08T00:20:05.246029+0000","last_became_peered":"2023-10-08T00:20:05.246029+0000","last_unstale":"2023-10-08T00:20:05.246765+0000","last_undegraded":"2023-10-08T00:20:05.246765+0000","last_fullsized":"2023-10-08T00:20:05.246765+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:18:31.037964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.247582+0000","last_change":"2023-10-08T00:20:05.247514+0000","last_active":"2023-10-08T00:20:05.247582+0000","last_peered":"2023-10-08T00:20:05.247582+0000","last_clean":"2023-10-08T00:20:05.247582+0000","last_became_active":"2023-10-08T00:20:05.246966+0000","last_became_peered":"2023-10-08T00:20:05.246966+0000","last_unstale":"2023-10-08T00:20:05.247582+0000","last_undegraded":"2023-10-08T00:20:05.247582+0000","last_fullsized":"2023-10-08T00:20:05.247582+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:55:15.548941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.243502+0000","last_change":"2023-10-08T00:20:05.243396+0000","last_active":"2023-10-08T00:20:05.243502+0000","last_peered":"2023-10-08T00:20:05.243502+0000","last_clean":"2023-10-08T00:20:05.243502+0000","last_became_active":"2023-10-08T00:20:05.243185+0000","last_became_peered":"2023-10-08T00:20:05.243185+0000","last_unstale":"2023-10-08T00:20:05.243502+0000","last_undegraded":"2023-10-08T00:20:05.243502+0000","last_fullsized":"2023-10-08T00:20:05.243502+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:19:47.969487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.250807+0000","last_change":"2023-10-08T00:20:05.250721+0000","last_active":"2023-10-08T00:20:05.250807+0000","last_peered":"2023-10-08T00:20:05.250807+0000","last_clean":"2023-10-08T00:20:05.250807+0000","last_became_active":"2023-10-08T00:20:05.250482+0000","last_became_peered":"2023-10-08T00:20:05.250482+0000","last_unstale":"2023-10-08T00:20:05.250807+0000","last_undegraded":"2023-10-08T00:20:05.250807+0000","last_fullsized":"2023-10-08T00:20:05.250807+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:54:14.710613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.247250+0000","last_change":"2023-10-08T00:20:05.245436+0000","last_active":"2023-10-08T00:20:05.247250+0000","last_peered":"2023-10-08T00:20:05.247250+0000","last_clean":"2023-10-08T00:20:05.247250+0000","last_became_active":"2023-10-08T00:20:05.245218+0000","last_became_peered":"2023-10-08T00:20:05.245218+0000","last_unstale":"2023-10-08T00:20:05.247250+0000","last_undegraded":"2023-10-08T00:20:05.247250+0000","last_fullsized":"2023-10-08T00:20:05.247250+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:33:07.925661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.254974+0000","last_change":"2023-10-08T00:20:05.252069+0000","last_active":"2023-10-08T00:20:05.254974+0000","last_peered":"2023-10-08T00:20:05.254974+0000","last_clean":"2023-10-08T00:20:05.254974+0000","last_became_active":"2023-10-08T00:20:05.251834+0000","last_became_peered":"2023-10-08T00:20:05.251834+0000","last_unstale":"2023-10-08T00:20:05.254974+0000","last_undegraded":"2023-10-08T00:20:05.254974+0000","last_fullsized":"2023-10-08T00:20:05.254974+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:30:53.757333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.241659+0000","last_change":"2023-10-08T00:20:05.241595+0000","last_active":"2023-10-08T00:20:05.241659+0000","last_peered":"2023-10-08T00:20:05.241659+0000","last_clean":"2023-10-08T00:20:05.241659+0000","last_became_active":"2023-10-08T00:20:05.241403+0000","last_became_peered":"2023-10-08T00:20:05.241403+0000","last_unstale":"2023-10-08T00:20:05.241659+0000","last_undegraded":"2023-10-08T00:20:05.241659+0000","last_fullsized":"2023-10-08T00:20:05.241659+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:40:58.162452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.247601+0000","last_change":"2023-10-08T00:20:05.247534+0000","last_active":"2023-10-08T00:20:05.247601+0000","last_peered":"2023-10-08T00:20:05.247601+0000","last_clean":"2023-10-08T00:20:05.247601+0000","last_became_active":"2023-10-08T00:20:05.247303+0000","last_became_peered":"2023-10-08T00:20:05.247303+0000","last_unstale":"2023-10-08T00:20:05.247601+0000","last_undegraded":"2023-10-08T00:20:05.247601+0000","last_fullsized":"2023-10-08T00:20:05.247601+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:20:20.153731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.245700+0000","last_change":"2023-10-08T00:20:05.244175+0000","last_active":"2023-10-08T00:20:05.245700+0000","last_peered":"2023-10-08T00:20:05.245700+0000","last_clean":"2023-10-08T00:20:05.245700+0000","last_became_active":"2023-10-08T00:20:05.243945+0000","last_became_peered":"2023-10-08T00:20:05.243945+0000","last_unstale":"2023-10-08T00:20:05.245700+0000","last_undegraded":"2023-10-08T00:20:05.245700+0000","last_fullsized":"2023-10-08T00:20:05.245700+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:56:49.026616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.243303+0000","last_change":"2023-10-08T00:20:05.242928+0000","last_active":"2023-10-08T00:20:05.243303+0000","last_peered":"2023-10-08T00:20:05.243303+0000","last_clean":"2023-10-08T00:20:05.243303+0000","last_became_active":"2023-10-08T00:20:05.242699+0000","last_became_peered":"2023-10-08T00:20:05.242699+0000","last_unstale":"2023-10-08T00:20:05.243303+0000","last_undegraded":"2023-10-08T00:20:05.243303+0000","last_fullsized":"2023-10-08T00:20:05.243303+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:50:09.923397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.254927+0000","last_change":"2023-10-08T00:20:05.253693+0000","last_active":"2023-10-08T00:20:05.254927+0000","last_peered":"2023-10-08T00:20:05.254927+0000","last_clean":"2023-10-08T00:20:05.254927+0000","last_became_active":"2023-10-08T00:20:05.252558+0000","last_became_peered":"2023-10-08T00:20:05.252558+0000","last_unstale":"2023-10-08T00:20:05.254927+0000","last_undegraded":"2023-10-08T00:20:05.254927+0000","last_fullsized":"2023-10-08T00:20:05.254927+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:44:29.905006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.249304+0000","last_change":"2023-10-08T00:20:05.248336+0000","last_active":"2023-10-08T00:20:05.249304+0000","last_peered":"2023-10-08T00:20:05.249304+0000","last_clean":"2023-10-08T00:20:05.249304+0000","last_became_active":"2023-10-08T00:20:05.248149+0000","last_became_peered":"2023-10-08T00:20:05.248149+0000","last_unstale":"2023-10-08T00:20:05.249304+0000","last_undegraded":"2023-10-08T00:20:05.249304+0000","last_fullsized":"2023-10-08T00:20:05.249304+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:18:31.037964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.243331+0000","last_change":"2023-10-08T00:20:05.243215+0000","last_active":"2023-10-08T00:20:05.243331+0000","last_peered":"2023-10-08T00:20:05.243331+0000","last_clean":"2023-10-08T00:20:05.243331+0000","last_became_active":"2023-10-08T00:20:05.242968+0000","last_became_peered":"2023-10-08T00:20:05.242968+0000","last_unstale":"2023-10-08T00:20:05.243331+0000","last_undegraded":"2023-10-08T00:20:05.243331+0000","last_fullsized":"2023-10-08T00:20:05.243331+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:05:59.854455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.250552+0000","last_change":"2023-10-08T00:20:05.250496+0000","last_active":"2023-10-08T00:20:05.250552+0000","last_peered":"2023-10-08T00:20:05.250552+0000","last_clean":"2023-10-08T00:20:05.250552+0000","last_became_active":"2023-10-08T00:20:05.250266+0000","last_became_peered":"2023-10-08T00:20:05.250266+0000","last_unstale":"2023-10-08T00:20:05.250552+0000","last_undegraded":"2023-10-08T00:20:05.250552+0000","last_fullsized":"2023-10-08T00:20:05.250552+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:03:50.626847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.254894+0000","last_change":"2023-10-08T00:20:05.250340+0000","last_active":"2023-10-08T00:20:05.254894+0000","last_peered":"2023-10-08T00:20:05.254894+0000","last_clean":"2023-10-08T00:20:05.254894+0000","last_became_active":"2023-10-08T00:20:05.250176+0000","last_became_peered":"2023-10-08T00:20:05.250176+0000","last_unstale":"2023-10-08T00:20:05.254894+0000","last_undegraded":"2023-10-08T00:20:05.254894+0000","last_fullsized":"2023-10-08T00:20:05.254894+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:22:48.035127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"167'4","reported_seq":14,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.256976+0000","last_change":"2023-10-08T00:20:04.238738+0000","last_active":"2023-10-08T00:20:05.256976+0000","last_peered":"2023-10-08T00:20:05.256976+0000","last_clean":"2023-10-08T00:20:05.256976+0000","last_became_active":"2023-10-08T00:20:04.238514+0000","last_became_peered":"2023-10-08T00:20:04.238514+0000","last_unstale":"2023-10-08T00:20:05.256976+0000","last_undegraded":"2023-10-08T00:20:05.256976+0000","last_fullsized":"2023-10-08T00:20:05.256976+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:27:28.789777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.242634+0000","last_change":"2023-10-08T00:20:05.242568+0000","last_active":"2023-10-08T00:20:05.242634+0000","last_peered":"2023-10-08T00:20:05.242634+0000","last_clean":"2023-10-08T00:20:05.242634+0000","last_became_active":"2023-10-08T00:20:05.242310+0000","last_became_peered":"2023-10-08T00:20:05.242310+0000","last_unstale":"2023-10-08T00:20:05.242634+0000","last_undegraded":"2023-10-08T00:20:05.242634+0000","last_fullsized":"2023-10-08T00:20:05.242634+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:07:54.014634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.1e","version":"167'1","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.256471+0000","last_change":"2023-10-08T00:20:04.246114+0000","last_active":"2023-10-08T00:20:05.256471+0000","last_peered":"2023-10-08T00:20:05.256471+0000","last_clean":"2023-10-08T00:20:05.256471+0000","last_became_active":"2023-10-08T00:20:04.245605+0000","last_became_peered":"2023-10-08T00:20:04.245605+0000","last_unstale":"2023-10-08T00:20:05.256471+0000","last_undegraded":"2023-10-08T00:20:05.256471+0000","last_fullsized":"2023-10-08T00:20:05.256471+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:43:35.304179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.256884+0000","last_change":"2023-10-08T00:20:05.256838+0000","last_active":"2023-10-08T00:20:05.256884+0000","last_peered":"2023-10-08T00:20:05.256884+0000","last_clean":"2023-10-08T00:20:05.256884+0000","last_became_active":"2023-10-08T00:20:05.256529+0000","last_became_peered":"2023-10-08T00:20:05.256529+0000","last_unstale":"2023-10-08T00:20:05.256884+0000","last_undegraded":"2023-10-08T00:20:05.256884+0000","last_fullsized":"2023-10-08T00:20:05.256884+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:34:08.648121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.1d","version":"167'2","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.254097+0000","last_change":"2023-10-08T00:20:04.248454+0000","last_active":"2023-10-08T00:20:05.254097+0000","last_peered":"2023-10-08T00:20:05.254097+0000","last_clean":"2023-10-08T00:20:05.254097+0000","last_became_active":"2023-10-08T00:20:04.246408+0000","last_became_peered":"2023-10-08T00:20:04.246408+0000","last_unstale":"2023-10-08T00:20:05.254097+0000","last_undegraded":"2023-10-08T00:20:05.254097+0000","last_fullsized":"2023-10-08T00:20:05.254097+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:50.024701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.244320+0000","last_change":"2023-10-08T00:20:05.244255+0000","last_active":"2023-10-08T00:20:05.244320+0000","last_peered":"2023-10-08T00:20:05.244320+0000","last_clean":"2023-10-08T00:20:05.244320+0000","last_became_active":"2023-10-08T00:20:05.243929+0000","last_became_peered":"2023-10-08T00:20:05.243929+0000","last_unstale":"2023-10-08T00:20:05.244320+0000","last_undegraded":"2023-10-08T00:20:05.244320+0000","last_fullsized":"2023-10-08T00:20:05.244320+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:34:06.226802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1c","version":"167'2","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.256526+0000","last_change":"2023-10-08T00:20:04.247073+0000","last_active":"2023-10-08T00:20:05.256526+0000","last_peered":"2023-10-08T00:20:05.256526+0000","last_clean":"2023-10-08T00:20:05.256526+0000","last_became_active":"2023-10-08T00:20:04.246803+0000","last_became_peered":"2023-10-08T00:20:04.246803+0000","last_unstale":"2023-10-08T00:20:05.256526+0000","last_undegraded":"2023-10-08T00:20:05.256526+0000","last_fullsized":"2023-10-08T00:20:05.256526+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:26:57.425301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.245777+0000","last_change":"2023-10-08T00:20:05.245591+0000","last_active":"2023-10-08T00:20:05.245777+0000","last_peered":"2023-10-08T00:20:05.245777+0000","last_clean":"2023-10-08T00:20:05.245777+0000","last_became_active":"2023-10-08T00:20:05.245372+0000","last_became_peered":"2023-10-08T00:20:05.245372+0000","last_unstale":"2023-10-08T00:20:05.245777+0000","last_undegraded":"2023-10-08T00:20:05.245777+0000","last_fullsized":"2023-10-08T00:20:05.245777+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:39:43.013423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"167'1","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.258319+0000","last_change":"2023-10-08T00:20:04.244497+0000","last_active":"2023-10-08T00:20:05.258319+0000","last_peered":"2023-10-08T00:20:05.258319+0000","last_clean":"2023-10-08T00:20:05.258319+0000","last_became_active":"2023-10-08T00:20:04.244226+0000","last_became_peered":"2023-10-08T00:20:04.244226+0000","last_unstale":"2023-10-08T00:20:05.258319+0000","last_undegraded":"2023-10-08T00:20:05.258319+0000","last_fullsized":"2023-10-08T00:20:05.258319+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:40:57.155039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.255428+0000","last_change":"2023-10-08T00:20:05.254308+0000","last_active":"2023-10-08T00:20:05.255428+0000","last_peered":"2023-10-08T00:20:05.255428+0000","last_clean":"2023-10-08T00:20:05.255428+0000","last_became_active":"2023-10-08T00:20:05.253303+0000","last_became_peered":"2023-10-08T00:20:05.253303+0000","last_unstale":"2023-10-08T00:20:05.255428+0000","last_undegraded":"2023-10-08T00:20:05.255428+0000","last_fullsized":"2023-10-08T00:20:05.255428+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:34:08.648121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1a","version":"167'1","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.255040+0000","last_change":"2023-10-08T00:20:04.246127+0000","last_active":"2023-10-08T00:20:05.255040+0000","last_peered":"2023-10-08T00:20:05.255040+0000","last_clean":"2023-10-08T00:20:05.255040+0000","last_became_active":"2023-10-08T00:20:04.245880+0000","last_became_peered":"2023-10-08T00:20:04.245880+0000","last_unstale":"2023-10-08T00:20:05.255040+0000","last_undegraded":"2023-10-08T00:20:05.255040+0000","last_fullsized":"2023-10-08T00:20:05.255040+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:01:16.185368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.247042+0000","last_change":"2023-10-08T00:20:05.246932+0000","last_active":"2023-10-08T00:20:05.247042+0000","last_peered":"2023-10-08T00:20:05.247042+0000","last_clean":"2023-10-08T00:20:05.247042+0000","last_became_active":"2023-10-08T00:20:05.246709+0000","last_became_peered":"2023-10-08T00:20:05.246709+0000","last_unstale":"2023-10-08T00:20:05.247042+0000","last_undegraded":"2023-10-08T00:20:05.247042+0000","last_fullsized":"2023-10-08T00:20:05.247042+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:09:43.710079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.240904+0000","last_change":"2023-10-08T00:20:04.246710+0000","last_active":"2023-10-08T00:20:05.240904+0000","last_peered":"2023-10-08T00:20:05.240904+0000","last_clean":"2023-10-08T00:20:05.240904+0000","last_became_active":"2023-10-08T00:20:04.246477+0000","last_became_peered":"2023-10-08T00:20:04.246477+0000","last_unstale":"2023-10-08T00:20:05.240904+0000","last_undegraded":"2023-10-08T00:20:05.240904+0000","last_fullsized":"2023-10-08T00:20:05.240904+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:05:58.847042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.249776+0000","last_change":"2023-10-08T00:20:05.249568+0000","last_active":"2023-10-08T00:20:05.249776+0000","last_peered":"2023-10-08T00:20:05.249776+0000","last_clean":"2023-10-08T00:20:05.249776+0000","last_became_active":"2023-10-08T00:20:05.249354+0000","last_became_peered":"2023-10-08T00:20:05.249354+0000","last_unstale":"2023-10-08T00:20:05.249776+0000","last_undegraded":"2023-10-08T00:20:05.249776+0000","last_fullsized":"2023-10-08T00:20:05.249776+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:49:32.653899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.242523+0000","last_change":"2023-10-08T00:20:04.248590+0000","last_active":"2023-10-08T00:20:05.242523+0000","last_peered":"2023-10-08T00:20:05.242523+0000","last_clean":"2023-10-08T00:20:05.242523+0000","last_became_active":"2023-10-08T00:20:04.246982+0000","last_became_peered":"2023-10-08T00:20:04.246982+0000","last_unstale":"2023-10-08T00:20:05.242523+0000","last_undegraded":"2023-10-08T00:20:05.242523+0000","last_fullsized":"2023-10-08T00:20:05.242523+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:01:07.906425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.255507+0000","last_change":"2023-10-08T00:20:05.254574+0000","last_active":"2023-10-08T00:20:05.255507+0000","last_peered":"2023-10-08T00:20:05.255507+0000","last_clean":"2023-10-08T00:20:05.255507+0000","last_became_active":"2023-10-08T00:20:05.253902+0000","last_became_peered":"2023-10-08T00:20:05.253902+0000","last_unstale":"2023-10-08T00:20:05.255507+0000","last_undegraded":"2023-10-08T00:20:05.255507+0000","last_fullsized":"2023-10-08T00:20:05.255507+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:52:47.931876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.242247+0000","last_change":"2023-10-08T00:20:04.245003+0000","last_active":"2023-10-08T00:20:05.242247+0000","last_peered":"2023-10-08T00:20:05.242247+0000","last_clean":"2023-10-08T00:20:05.242247+0000","last_became_active":"2023-10-08T00:20:04.244739+0000","last_became_peered":"2023-10-08T00:20:04.244739+0000","last_unstale":"2023-10-08T00:20:05.242247+0000","last_undegraded":"2023-10-08T00:20:05.242247+0000","last_fullsized":"2023-10-08T00:20:05.242247+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:00:58.443848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.254685+0000","last_change":"2023-10-08T00:20:05.254453+0000","last_active":"2023-10-08T00:20:05.254685+0000","last_peered":"2023-10-08T00:20:05.254685+0000","last_clean":"2023-10-08T00:20:05.254685+0000","last_became_active":"2023-10-08T00:20:05.252916+0000","last_became_peered":"2023-10-08T00:20:05.252916+0000","last_unstale":"2023-10-08T00:20:05.254685+0000","last_undegraded":"2023-10-08T00:20:05.254685+0000","last_fullsized":"2023-10-08T00:20:05.254685+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:34:06.226802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.241337+0000","last_change":"2023-10-08T00:20:04.249414+0000","last_active":"2023-10-08T00:20:05.241337+0000","last_peered":"2023-10-08T00:20:05.241337+0000","last_clean":"2023-10-08T00:20:05.241337+0000","last_became_active":"2023-10-08T00:20:04.249191+0000","last_became_peered":"2023-10-08T00:20:04.249191+0000","last_unstale":"2023-10-08T00:20:05.241337+0000","last_undegraded":"2023-10-08T00:20:05.241337+0000","last_fullsized":"2023-10-08T00:20:05.241337+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:52:05.971558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.243591+0000","last_change":"2023-10-08T00:20:05.243491+0000","last_active":"2023-10-08T00:20:05.243591+0000","last_peered":"2023-10-08T00:20:05.243591+0000","last_clean":"2023-10-08T00:20:05.243591+0000","last_became_active":"2023-10-08T00:20:05.243266+0000","last_became_peered":"2023-10-08T00:20:05.243266+0000","last_unstale":"2023-10-08T00:20:05.243591+0000","last_undegraded":"2023-10-08T00:20:05.243591+0000","last_fullsized":"2023-10-08T00:20:05.243591+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:43:36.311593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.14","version":"167'6","reported_seq":16,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.259480+0000","last_change":"2023-10-08T00:20:04.251024+0000","last_active":"2023-10-08T00:20:05.259480+0000","last_peered":"2023-10-08T00:20:05.259480+0000","last_clean":"2023-10-08T00:20:05.259480+0000","last_became_active":"2023-10-08T00:20:04.250509+0000","last_became_peered":"2023-10-08T00:20:04.250509+0000","last_unstale":"2023-10-08T00:20:05.259480+0000","last_undegraded":"2023-10-08T00:20:05.259480+0000","last_fullsized":"2023-10-08T00:20:05.259480+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:37:16.406481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.252153+0000","last_change":"2023-10-08T00:20:05.252090+0000","last_active":"2023-10-08T00:20:05.252153+0000","last_peered":"2023-10-08T00:20:05.252153+0000","last_clean":"2023-10-08T00:20:05.252153+0000","last_became_active":"2023-10-08T00:20:05.251701+0000","last_became_peered":"2023-10-08T00:20:05.251701+0000","last_unstale":"2023-10-08T00:20:05.252153+0000","last_undegraded":"2023-10-08T00:20:05.252153+0000","last_fullsized":"2023-10-08T00:20:05.252153+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:07:54.014634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.13","version":"167'2","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.254265+0000","last_change":"2023-10-08T00:20:04.248311+0000","last_active":"2023-10-08T00:20:05.254265+0000","last_peered":"2023-10-08T00:20:05.254265+0000","last_clean":"2023-10-08T00:20:05.254265+0000","last_became_active":"2023-10-08T00:20:04.246106+0000","last_became_peered":"2023-10-08T00:20:04.246106+0000","last_unstale":"2023-10-08T00:20:05.254265+0000","last_undegraded":"2023-10-08T00:20:05.254265+0000","last_fullsized":"2023-10-08T00:20:05.254265+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:37:23.216226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.258649+0000","last_change":"2023-10-08T00:20:05.258587+0000","last_active":"2023-10-08T00:20:05.258649+0000","last_peered":"2023-10-08T00:20:05.258649+0000","last_clean":"2023-10-08T00:20:05.258649+0000","last_became_active":"2023-10-08T00:20:05.258394+0000","last_became_peered":"2023-10-08T00:20:05.258394+0000","last_unstale":"2023-10-08T00:20:05.258649+0000","last_undegraded":"2023-10-08T00:20:05.258649+0000","last_fullsized":"2023-10-08T00:20:05.258649+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:34:08.648121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.248788+0000","last_change":"2023-10-08T00:20:04.242275+0000","last_active":"2023-10-08T00:20:05.248788+0000","last_peered":"2023-10-08T00:20:05.248788+0000","last_clean":"2023-10-08T00:20:05.248788+0000","last_became_active":"2023-10-08T00:20:04.241872+0000","last_became_peered":"2023-10-08T00:20:04.241872+0000","last_unstale":"2023-10-08T00:20:05.248788+0000","last_undegraded":"2023-10-08T00:20:05.248788+0000","last_fullsized":"2023-10-08T00:20:05.248788+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:18:39.105501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.247210+0000","last_change":"2023-10-08T00:20:05.244976+0000","last_active":"2023-10-08T00:20:05.247210+0000","last_peered":"2023-10-08T00:20:05.247210+0000","last_clean":"2023-10-08T00:20:05.247210+0000","last_became_active":"2023-10-08T00:20:05.244533+0000","last_became_peered":"2023-10-08T00:20:05.244533+0000","last_unstale":"2023-10-08T00:20:05.247210+0000","last_undegraded":"2023-10-08T00:20:05.247210+0000","last_fullsized":"2023-10-08T00:20:05.247210+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:44:29.905006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.238940+0000","last_change":"2023-10-08T00:20:04.246262+0000","last_active":"2023-10-08T00:20:05.238940+0000","last_peered":"2023-10-08T00:20:05.238940+0000","last_clean":"2023-10-08T00:20:05.238940+0000","last_became_active":"2023-10-08T00:20:04.244141+0000","last_became_peered":"2023-10-08T00:20:04.244141+0000","last_unstale":"2023-10-08T00:20:05.238940+0000","last_undegraded":"2023-10-08T00:20:05.238940+0000","last_fullsized":"2023-10-08T00:20:05.238940+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:24:31.008196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.255472+0000","last_change":"2023-10-08T00:20:05.254444+0000","last_active":"2023-10-08T00:20:05.255472+0000","last_peered":"2023-10-08T00:20:05.255472+0000","last_clean":"2023-10-08T00:20:05.255472+0000","last_became_active":"2023-10-08T00:20:05.253619+0000","last_became_peered":"2023-10-08T00:20:05.253619+0000","last_unstale":"2023-10-08T00:20:05.255472+0000","last_undegraded":"2023-10-08T00:20:05.255472+0000","last_fullsized":"2023-10-08T00:20:05.255472+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:00:59.451262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.10","version":"167'8","reported_seq":18,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.256509+0000","last_change":"2023-10-08T00:20:04.246644+0000","last_active":"2023-10-08T00:20:05.256509+0000","last_peered":"2023-10-08T00:20:05.256509+0000","last_clean":"2023-10-08T00:20:05.256509+0000","last_became_active":"2023-10-08T00:20:04.246456+0000","last_became_peered":"2023-10-08T00:20:04.246456+0000","last_unstale":"2023-10-08T00:20:05.256509+0000","last_undegraded":"2023-10-08T00:20:05.256509+0000","last_fullsized":"2023-10-08T00:20:05.256509+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:24:48.069538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.258776+0000","last_change":"2023-10-08T00:20:05.258594+0000","last_active":"2023-10-08T00:20:05.258776+0000","last_peered":"2023-10-08T00:20:05.258776+0000","last_clean":"2023-10-08T00:20:05.258776+0000","last_became_active":"2023-10-08T00:20:05.258316+0000","last_became_peered":"2023-10-08T00:20:05.258316+0000","last_unstale":"2023-10-08T00:20:05.258776+0000","last_undegraded":"2023-10-08T00:20:05.258776+0000","last_fullsized":"2023-10-08T00:20:05.258776+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:45:00.428167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.f","version":"167'1","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.255792+0000","last_change":"2023-10-08T00:20:04.241270+0000","last_active":"2023-10-08T00:20:05.255792+0000","last_peered":"2023-10-08T00:20:05.255792+0000","last_clean":"2023-10-08T00:20:05.255792+0000","last_became_active":"2023-10-08T00:20:04.241025+0000","last_became_peered":"2023-10-08T00:20:04.241025+0000","last_unstale":"2023-10-08T00:20:05.255792+0000","last_undegraded":"2023-10-08T00:20:05.255792+0000","last_fullsized":"2023-10-08T00:20:05.255792+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:27:07.999816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.247024+0000","last_change":"2023-10-08T00:20:05.246917+0000","last_active":"2023-10-08T00:20:05.247024+0000","last_peered":"2023-10-08T00:20:05.247024+0000","last_clean":"2023-10-08T00:20:05.247024+0000","last_became_active":"2023-10-08T00:20:05.246691+0000","last_became_peered":"2023-10-08T00:20:05.246691+0000","last_unstale":"2023-10-08T00:20:05.247024+0000","last_undegraded":"2023-10-08T00:20:05.247024+0000","last_fullsized":"2023-10-08T00:20:05.247024+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:37:17.413894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.e","version":"167'2","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.256655+0000","last_change":"2023-10-08T00:20:04.245210+0000","last_active":"2023-10-08T00:20:05.256655+0000","last_peered":"2023-10-08T00:20:05.256655+0000","last_clean":"2023-10-08T00:20:05.256655+0000","last_became_active":"2023-10-08T00:20:04.244971+0000","last_became_peered":"2023-10-08T00:20:04.244971+0000","last_unstale":"2023-10-08T00:20:05.256655+0000","last_undegraded":"2023-10-08T00:20:05.256655+0000","last_fullsized":"2023-10-08T00:20:05.256655+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:39:42.006010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.257928+0000","last_change":"2023-10-08T00:20:05.257818+0000","last_active":"2023-10-08T00:20:05.257928+0000","last_peered":"2023-10-08T00:20:05.257928+0000","last_clean":"2023-10-08T00:20:05.257928+0000","last_became_active":"2023-10-08T00:20:05.257127+0000","last_became_peered":"2023-10-08T00:20:05.257127+0000","last_unstale":"2023-10-08T00:20:05.257928+0000","last_undegraded":"2023-10-08T00:20:05.257928+0000","last_fullsized":"2023-10-08T00:20:05.257928+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:42:22.115875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"167'1","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.255243+0000","last_change":"2023-10-08T00:20:04.244040+0000","last_active":"2023-10-08T00:20:05.255243+0000","last_peered":"2023-10-08T00:20:05.255243+0000","last_clean":"2023-10-08T00:20:05.255243+0000","last_became_active":"2023-10-08T00:20:04.242376+0000","last_became_peered":"2023-10-08T00:20:04.242376+0000","last_unstale":"2023-10-08T00:20:05.255243+0000","last_undegraded":"2023-10-08T00:20:05.255243+0000","last_fullsized":"2023-10-08T00:20:05.255243+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:09:42.702666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.256772+0000","last_change":"2023-10-08T00:20:05.256706+0000","last_active":"2023-10-08T00:20:05.256772+0000","last_peered":"2023-10-08T00:20:05.256772+0000","last_clean":"2023-10-08T00:20:05.256772+0000","last_became_active":"2023-10-08T00:20:05.256225+0000","last_became_peered":"2023-10-08T00:20:05.256225+0000","last_unstale":"2023-10-08T00:20:05.256772+0000","last_undegraded":"2023-10-08T00:20:05.256772+0000","last_fullsized":"2023-10-08T00:20:05.256772+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:44:29.905006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.239029+0000","last_change":"2023-10-08T00:20:04.246634+0000","last_active":"2023-10-08T00:20:05.239029+0000","last_peered":"2023-10-08T00:20:05.239029+0000","last_clean":"2023-10-08T00:20:05.239029+0000","last_became_active":"2023-10-08T00:20:04.246030+0000","last_became_peered":"2023-10-08T00:20:04.246030+0000","last_unstale":"2023-10-08T00:20:05.239029+0000","last_undegraded":"2023-10-08T00:20:05.239029+0000","last_fullsized":"2023-10-08T00:20:05.239029+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:52:19.278093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.251066+0000","last_change":"2023-10-08T00:20:05.251010+0000","last_active":"2023-10-08T00:20:05.251066+0000","last_peered":"2023-10-08T00:20:05.251066+0000","last_clean":"2023-10-08T00:20:05.251066+0000","last_became_active":"2023-10-08T00:20:05.250656+0000","last_became_peered":"2023-10-08T00:20:05.250656+0000","last_unstale":"2023-10-08T00:20:05.251066+0000","last_undegraded":"2023-10-08T00:20:05.251066+0000","last_fullsized":"2023-10-08T00:20:05.251066+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:55:15.548941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.b","version":"167'2","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.257948+0000","last_change":"2023-10-08T00:20:04.249629+0000","last_active":"2023-10-08T00:20:05.257948+0000","last_peered":"2023-10-08T00:20:05.257948+0000","last_clean":"2023-10-08T00:20:05.257948+0000","last_became_active":"2023-10-08T00:20:04.249401+0000","last_became_peered":"2023-10-08T00:20:04.249401+0000","last_unstale":"2023-10-08T00:20:05.257948+0000","last_undegraded":"2023-10-08T00:20:05.257948+0000","last_fullsized":"2023-10-08T00:20:05.257948+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:24:48.069538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.245751+0000","last_change":"2023-10-08T00:20:05.243906+0000","last_active":"2023-10-08T00:20:05.245751+0000","last_peered":"2023-10-08T00:20:05.245751+0000","last_clean":"2023-10-08T00:20:05.245751+0000","last_became_active":"2023-10-08T00:20:05.243615+0000","last_became_peered":"2023-10-08T00:20:05.243615+0000","last_unstale":"2023-10-08T00:20:05.245751+0000","last_undegraded":"2023-10-08T00:20:05.245751+0000","last_fullsized":"2023-10-08T00:20:05.245751+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:08:07.447036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.237626+0000","last_change":"2023-10-08T00:20:04.247748+0000","last_active":"2023-10-08T00:20:05.237626+0000","last_peered":"2023-10-08T00:20:05.237626+0000","last_clean":"2023-10-08T00:20:05.237626+0000","last_became_active":"2023-10-08T00:20:04.247530+0000","last_became_peered":"2023-10-08T00:20:04.247530+0000","last_unstale":"2023-10-08T00:20:05.237626+0000","last_undegraded":"2023-10-08T00:20:05.237626+0000","last_fullsized":"2023-10-08T00:20:05.237626+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:54:04.355034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.254668+0000","last_change":"2023-10-08T00:20:05.254316+0000","last_active":"2023-10-08T00:20:05.254668+0000","last_peered":"2023-10-08T00:20:05.254668+0000","last_clean":"2023-10-08T00:20:05.254668+0000","last_became_active":"2023-10-08T00:20:05.252638+0000","last_became_peered":"2023-10-08T00:20:05.252638+0000","last_unstale":"2023-10-08T00:20:05.254668+0000","last_undegraded":"2023-10-08T00:20:05.254668+0000","last_fullsized":"2023-10-08T00:20:05.254668+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:34:08.648121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.237342+0000","last_change":"2023-10-08T00:20:04.245345+0000","last_active":"2023-10-08T00:20:05.237342+0000","last_peered":"2023-10-08T00:20:05.237342+0000","last_clean":"2023-10-08T00:20:05.237342+0000","last_became_active":"2023-10-08T00:20:04.245173+0000","last_became_peered":"2023-10-08T00:20:04.245173+0000","last_unstale":"2023-10-08T00:20:05.237342+0000","last_undegraded":"2023-10-08T00:20:05.237342+0000","last_fullsized":"2023-10-08T00:20:05.237342+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:01:36.620681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.245730+0000","last_change":"2023-10-08T00:20:05.244414+0000","last_active":"2023-10-08T00:20:05.245730+0000","last_peered":"2023-10-08T00:20:05.245730+0000","last_clean":"2023-10-08T00:20:05.245730+0000","last_became_active":"2023-10-08T00:20:05.243718+0000","last_became_peered":"2023-10-08T00:20:05.243718+0000","last_unstale":"2023-10-08T00:20:05.245730+0000","last_undegraded":"2023-10-08T00:20:05.245730+0000","last_fullsized":"2023-10-08T00:20:05.245730+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:43:41.968090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.237627+0000","last_change":"2023-10-08T00:20:04.242970+0000","last_active":"2023-10-08T00:20:05.237627+0000","last_peered":"2023-10-08T00:20:05.237627+0000","last_clean":"2023-10-08T00:20:05.237627+0000","last_became_active":"2023-10-08T00:20:04.242709+0000","last_became_peered":"2023-10-08T00:20:04.242709+0000","last_unstale":"2023-10-08T00:20:05.237627+0000","last_undegraded":"2023-10-08T00:20:05.237627+0000","last_fullsized":"2023-10-08T00:20:05.237627+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:04:36.794999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.247187+0000","last_change":"2023-10-08T00:20:05.244829+0000","last_active":"2023-10-08T00:20:05.247187+0000","last_peered":"2023-10-08T00:20:05.247187+0000","last_clean":"2023-10-08T00:20:05.247187+0000","last_became_active":"2023-10-08T00:20:05.244323+0000","last_became_peered":"2023-10-08T00:20:05.244323+0000","last_unstale":"2023-10-08T00:20:05.247187+0000","last_undegraded":"2023-10-08T00:20:05.247187+0000","last_fullsized":"2023-10-08T00:20:05.247187+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:20:20.153731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.7","version":"167'2","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.258473+0000","last_change":"2023-10-08T00:20:04.244594+0000","last_active":"2023-10-08T00:20:05.258473+0000","last_peered":"2023-10-08T00:20:05.258473+0000","last_clean":"2023-10-08T00:20:05.258473+0000","last_became_active":"2023-10-08T00:20:04.244342+0000","last_became_peered":"2023-10-08T00:20:04.244342+0000","last_unstale":"2023-10-08T00:20:05.258473+0000","last_undegraded":"2023-10-08T00:20:05.258473+0000","last_fullsized":"2023-10-08T00:20:05.258473+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:36:39.544482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.255372+0000","last_change":"2023-10-08T00:20:05.253093+0000","last_active":"2023-10-08T00:20:05.255372+0000","last_peered":"2023-10-08T00:20:05.255372+0000","last_clean":"2023-10-08T00:20:05.255372+0000","last_became_active":"2023-10-08T00:20:05.243661+0000","last_became_peered":"2023-10-08T00:20:05.243661+0000","last_unstale":"2023-10-08T00:20:05.255372+0000","last_undegraded":"2023-10-08T00:20:05.255372+0000","last_fullsized":"2023-10-08T00:20:05.255372+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:10:26.392889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.237710+0000","last_change":"2023-10-08T00:20:04.247009+0000","last_active":"2023-10-08T00:20:05.237710+0000","last_peered":"2023-10-08T00:20:05.237710+0000","last_clean":"2023-10-08T00:20:05.237710+0000","last_became_active":"2023-10-08T00:20:04.246663+0000","last_became_peered":"2023-10-08T00:20:04.246663+0000","last_unstale":"2023-10-08T00:20:05.237710+0000","last_undegraded":"2023-10-08T00:20:05.237710+0000","last_fullsized":"2023-10-08T00:20:05.237710+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:45:17.133123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.247754+0000","last_change":"2023-10-08T00:20:05.247665+0000","last_active":"2023-10-08T00:20:05.247754+0000","last_peered":"2023-10-08T00:20:05.247754+0000","last_clean":"2023-10-08T00:20:05.247754+0000","last_became_active":"2023-10-08T00:20:05.247280+0000","last_became_peered":"2023-10-08T00:20:05.247280+0000","last_unstale":"2023-10-08T00:20:05.247754+0000","last_undegraded":"2023-10-08T00:20:05.247754+0000","last_fullsized":"2023-10-08T00:20:05.247754+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:33:07.925661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.238912+0000","last_change":"2023-10-08T00:20:04.251109+0000","last_active":"2023-10-08T00:20:05.238912+0000","last_peered":"2023-10-08T00:20:05.238912+0000","last_clean":"2023-10-08T00:20:05.238912+0000","last_became_active":"2023-10-08T00:20:04.250736+0000","last_became_peered":"2023-10-08T00:20:04.250736+0000","last_unstale":"2023-10-08T00:20:05.238912+0000","last_undegraded":"2023-10-08T00:20:05.238912+0000","last_fullsized":"2023-10-08T00:20:05.238912+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:03:49.619434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.251815+0000","last_change":"2023-10-08T00:20:05.249949+0000","last_active":"2023-10-08T00:20:05.251815+0000","last_peered":"2023-10-08T00:20:05.251815+0000","last_clean":"2023-10-08T00:20:05.251815+0000","last_became_active":"2023-10-08T00:20:05.249727+0000","last_became_peered":"2023-10-08T00:20:05.249727+0000","last_unstale":"2023-10-08T00:20:05.251815+0000","last_undegraded":"2023-10-08T00:20:05.251815+0000","last_fullsized":"2023-10-08T00:20:05.251815+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:34:06.226802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.237409+0000","last_change":"2023-10-08T00:20:04.248488+0000","last_active":"2023-10-08T00:20:05.237409+0000","last_peered":"2023-10-08T00:20:05.237409+0000","last_clean":"2023-10-08T00:20:05.237409+0000","last_became_active":"2023-10-08T00:20:04.248203+0000","last_became_peered":"2023-10-08T00:20:04.248203+0000","last_unstale":"2023-10-08T00:20:05.237409+0000","last_undegraded":"2023-10-08T00:20:05.237409+0000","last_fullsized":"2023-10-08T00:20:05.237409+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:56:17.020837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.250876+0000","last_change":"2023-10-08T00:20:05.250831+0000","last_active":"2023-10-08T00:20:05.250876+0000","last_peered":"2023-10-08T00:20:05.250876+0000","last_clean":"2023-10-08T00:20:05.250876+0000","last_became_active":"2023-10-08T00:20:05.250672+0000","last_became_peered":"2023-10-08T00:20:05.250672+0000","last_unstale":"2023-10-08T00:20:05.250876+0000","last_undegraded":"2023-10-08T00:20:05.250876+0000","last_fullsized":"2023-10-08T00:20:05.250876+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:48:04.097734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.237682+0000","last_change":"2023-10-08T00:20:04.247524+0000","last_active":"2023-10-08T00:20:05.237682+0000","last_peered":"2023-10-08T00:20:05.237682+0000","last_clean":"2023-10-08T00:20:05.237682+0000","last_became_active":"2023-10-08T00:20:04.240348+0000","last_became_peered":"2023-10-08T00:20:04.240348+0000","last_unstale":"2023-10-08T00:20:05.237682+0000","last_undegraded":"2023-10-08T00:20:05.237682+0000","last_fullsized":"2023-10-08T00:20:05.237682+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:37:41.683109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.248946+0000","last_change":"2023-10-08T00:20:05.248871+0000","last_active":"2023-10-08T00:20:05.248946+0000","last_peered":"2023-10-08T00:20:05.248946+0000","last_clean":"2023-10-08T00:20:05.248946+0000","last_became_active":"2023-10-08T00:20:05.248573+0000","last_became_peered":"2023-10-08T00:20:05.248573+0000","last_unstale":"2023-10-08T00:20:05.248946+0000","last_undegraded":"2023-10-08T00:20:05.248946+0000","last_fullsized":"2023-10-08T00:20:05.248946+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:48:04.097734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.242379+0000","last_change":"2023-10-08T00:20:04.243813+0000","last_active":"2023-10-08T00:20:05.242379+0000","last_peered":"2023-10-08T00:20:05.242379+0000","last_clean":"2023-10-08T00:20:05.242379+0000","last_became_active":"2023-10-08T00:20:04.243560+0000","last_became_peered":"2023-10-08T00:20:04.243560+0000","last_unstale":"2023-10-08T00:20:05.242379+0000","last_undegraded":"2023-10-08T00:20:05.242379+0000","last_fullsized":"2023-10-08T00:20:05.242379+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:56:50.470059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.247268+0000","last_change":"2023-10-08T00:20:05.245479+0000","last_active":"2023-10-08T00:20:05.247268+0000","last_peered":"2023-10-08T00:20:05.247268+0000","last_clean":"2023-10-08T00:20:05.247268+0000","last_became_active":"2023-10-08T00:20:05.245227+0000","last_became_peered":"2023-10-08T00:20:05.245227+0000","last_unstale":"2023-10-08T00:20:05.247268+0000","last_undegraded":"2023-10-08T00:20:05.247268+0000","last_fullsized":"2023-10-08T00:20:05.247268+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:55:15.548941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.241393+0000","last_change":"2023-10-08T00:20:04.246657+0000","last_active":"2023-10-08T00:20:05.241393+0000","last_peered":"2023-10-08T00:20:05.241393+0000","last_clean":"2023-10-08T00:20:05.241393+0000","last_became_active":"2023-10-08T00:20:04.246319+0000","last_became_peered":"2023-10-08T00:20:04.246319+0000","last_unstale":"2023-10-08T00:20:05.241393+0000","last_undegraded":"2023-10-08T00:20:05.241393+0000","last_fullsized":"2023-10-08T00:20:05.241393+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:50:08.915984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.247712+0000","last_change":"2023-10-08T00:20:05.247649+0000","last_active":"2023-10-08T00:20:05.247712+0000","last_peered":"2023-10-08T00:20:05.247712+0000","last_clean":"2023-10-08T00:20:05.247712+0000","last_became_active":"2023-10-08T00:20:05.247466+0000","last_became_peered":"2023-10-08T00:20:05.247466+0000","last_unstale":"2023-10-08T00:20:05.247712+0000","last_undegraded":"2023-10-08T00:20:05.247712+0000","last_fullsized":"2023-10-08T00:20:05.247712+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:33:38.066405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.240999+0000","last_change":"2023-10-08T00:20:04.246258+0000","last_active":"2023-10-08T00:20:05.240999+0000","last_peered":"2023-10-08T00:20:05.240999+0000","last_clean":"2023-10-08T00:20:05.240999+0000","last_became_active":"2023-10-08T00:20:04.245892+0000","last_became_peered":"2023-10-08T00:20:04.245892+0000","last_unstale":"2023-10-08T00:20:05.240999+0000","last_undegraded":"2023-10-08T00:20:05.240999+0000","last_fullsized":"2023-10-08T00:20:05.240999+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:50:08.915984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.244288+0000","last_change":"2023-10-08T00:20:05.243876+0000","last_active":"2023-10-08T00:20:05.244288+0000","last_peered":"2023-10-08T00:20:05.244288+0000","last_clean":"2023-10-08T00:20:05.244288+0000","last_became_active":"2023-10-08T00:20:05.243662+0000","last_became_peered":"2023-10-08T00:20:05.243662+0000","last_unstale":"2023-10-08T00:20:05.244288+0000","last_undegraded":"2023-10-08T00:20:05.244288+0000","last_fullsized":"2023-10-08T00:20:05.244288+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:04.229489+0000","last_clean_scrub_stamp":"2023-10-08T00:20:04.229489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:05:59.854455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.15","version":"167'3","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.259385+0000","last_change":"2023-10-08T00:20:04.246528+0000","last_active":"2023-10-08T00:20:05.259385+0000","last_peered":"2023-10-08T00:20:05.259385+0000","last_clean":"2023-10-08T00:20:05.259385+0000","last_became_active":"2023-10-08T00:20:04.246029+0000","last_became_peered":"2023-10-08T00:20:04.246029+0000","last_unstale":"2023-10-08T00:20:05.259385+0000","last_undegraded":"2023-10-08T00:20:05.259385+0000","last_fullsized":"2023-10-08T00:20:05.259385+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:20:03.222076+0000","last_clean_scrub_stamp":"2023-10-08T00:20:03.222076+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:05:58.847042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":289,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T00:20:05.244557+0000","last_change":"2023-10-08T00:10:43.728490+0000","last_active":"2023-10-08T00:20:05.244557+0000","last_peered":"2023-10-08T00:20:05.244557+0000","last_clean":"2023-10-08T00:20:05.244557+0000","last_became_active":"2023-10-08T00:10:43.728341+0000","last_became_peered":"2023-10-08T00:10:43.728341+0000","last_unstale":"2023-10-08T00:20:05.244557+0000","last_undegraded":"2023-10-08T00:20:05.244557+0000","last_fullsized":"2023-10-08T00:20:05.244557+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_clean_scrub_stamp":"2023-10-08T00:10:42.716096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:06:31.035293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574963,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":917,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194394,"statfs":{"total":200000000,"available":199060448,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:19:58 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.53800000000000003,"15min":0.51600000000000001},"min":{"1min":0.33900000000000002,"5min":0.27800000000000002,"15min":0.26900000000000002},"max":{"1min":0.63,"5min":1.129,"15min":1.129},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.55600000000000005,"15min":0.54600000000000004},"min":{"1min":0.41899999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.70599999999999996,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.45000000000000001}]},{"osd":1,"last update":"Sun Oct 8 00:19:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57699999999999996,"15min":0.55100000000000005},"min":{"1min":0.32300000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72299999999999998,"5min":0.77900000000000003,"15min":0.82999999999999996},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.59099999999999997,"15min":0.57499999999999996},"min":{"1min":0.498,"5min":0.39300000000000002,"15min":0.38200000000000001},"max":{"1min":0.76600000000000001,"5min":0.79400000000000004,"15min":0.80100000000000005},"last":0.52400000000000002}]},{"osd":2,"last update":"Sun Oct 8 00:19:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56200000000000006,"15min":0.54800000000000004},"min":{"1min":0.44600000000000001,"5min":0.34599999999999997,"15min":0.34200000000000003},"max":{"1min":0.74299999999999999,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.55800000000000005,"15min":0.53500000000000003},"min":{"1min":0.39000000000000001,"5min":0.34499999999999997,"15min":0.32900000000000001},"max":{"1min":0.69899999999999995,"5min":0.77200000000000002,"15min":0.78500000000000003},"last":0.59199999999999997}]},{"osd":3,"last update":"Sun Oct 8 00:19:58 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.55600000000000005,"15min":0.53400000000000003},"min":{"1min":0.40500000000000003,"5min":0.35599999999999998,"15min":0.33700000000000002},"max":{"1min":0.66400000000000003,"5min":0.75600000000000001,"15min":0.79300000000000004},"last":0.496},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.56899999999999995,"15min":0.56299999999999994},"min":{"1min":0.433,"5min":0.371,"15min":0.371},"max":{"1min":0.745,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.50900000000000001}]},{"osd":4,"last update":"Sun Oct 8 00:19:58 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.47699999999999998,"15min":0.505},"min":{"1min":0.30399999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.67900000000000005,"5min":0.755,"15min":0.755},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.51600000000000001,"15min":0.52000000000000002},"min":{"1min":0.38400000000000001,"5min":0.29199999999999998,"15min":0.28199999999999997},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.78900000000000003},"last":0.379}]},{"osd":5,"last update":"Sun Oct 8 00:19:58 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.51100000000000001,"15min":0.53200000000000003},"min":{"1min":0.27800000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.66700000000000004,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.497,"5min":0.54300000000000004,"15min":0.56499999999999995},"min":{"1min":0.32400000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.71299999999999997,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.41499999999999998}]},{"osd":6,"last update":"Sun Oct 8 00:19:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58799999999999997,"15min":0.59199999999999997},"min":{"1min":0.41599999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.754,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.59099999999999997,"15min":0.61199999999999999},"min":{"1min":0.34300000000000003,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.69499999999999995,"5min":0.82999999999999996,"15min":0.91500000000000004},"last":0.56299999999999994}]}]},{"osd":6,"up_from":13,"seq":55834574963,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":917,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194394,"statfs":{"total":200000000,"available":199060448,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.52700000000000002,"15min":0.497},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.29199999999999998},"max":{"1min":0.69799999999999995,"5min":0.71199999999999997,"15min":1.012},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.51800000000000002,"15min":0.627},"min":{"1min":0.36799999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.65800000000000003,"5min":1.0529999999999999,"15min":5.1230000000000002},"last":0.42899999999999999}]},{"osd":1,"last update":"Sun Oct 8 00:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.54700000000000004,"15min":0.53000000000000003},"min":{"1min":0.39300000000000002,"5min":0.29899999999999999,"15min":0.28899999999999998},"max":{"1min":0.68600000000000005,"5min":0.78100000000000003,"15min":1.1419999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.57699999999999996,"15min":0.56799999999999995},"min":{"1min":0.435,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.70199999999999996,"5min":0.98999999999999999,"15min":1.1539999999999999},"last":0.51600000000000001}]},{"osd":2,"last update":"Sun Oct 8 00:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54200000000000004,"15min":0.53400000000000003},"min":{"1min":0.40400000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.70699999999999996,"5min":0.76500000000000001,"15min":1.3149999999999999},"last":0.373},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57599999999999996,"15min":0.56999999999999995},"min":{"1min":0.45200000000000001,"5min":0.38900000000000001,"15min":0.29399999999999998},"max":{"1min":0.72699999999999998,"5min":0.81699999999999995,"15min":1.3280000000000001},"last":0.54700000000000004}]},{"osd":3,"last update":"Sun Oct 8 00:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.57099999999999995,"15min":0.54800000000000004},"min":{"1min":0.41699999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.72799999999999998,"5min":0.84199999999999997,"15min":1.046},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56499999999999995,"15min":0.55100000000000005},"min":{"1min":0.38600000000000001,"5min":0.35099999999999998,"15min":0.32900000000000001},"max":{"1min":0.71399999999999997,"5min":0.81000000000000005,"15min":1.4390000000000001},"last":0.53200000000000003}]},{"osd":4,"last update":"Sun Oct 8 00:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.46000000000000002,"15min":0.48599999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.52300000000000002,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.54600000000000004,"15min":0.55700000000000005},"min":{"1min":0.375,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.621,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.56599999999999995}]},{"osd":5,"last update":"Sun Oct 8 00:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.46999999999999997,"15min":0.48199999999999998},"min":{"1min":0.312,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.51900000000000002,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.502,"15min":0.53700000000000003},"min":{"1min":0.313,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.51900000000000002,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.624}]},{"osd":7,"last update":"Sun Oct 8 00:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.50600000000000001,"15min":0.52900000000000003},"min":{"1min":0.32800000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.57699999999999996,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.52800000000000002,"15min":0.54100000000000004},"min":{"1min":0.35399999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.56699999999999995,"5min":0.76000000000000001,"15min":0.79900000000000004},"last":0.57999999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574963,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1494,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193818,"statfs":{"total":200000000,"available":198470010,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.35599999999999998,"15min":0.32600000000000001},"min":{"1min":0.216,"5min":0.20300000000000001,"15min":0.19500000000000001},"max":{"1min":0.54400000000000004,"5min":0.55900000000000005,"15min":0.56899999999999995},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.41499999999999998,"15min":0.39000000000000001},"min":{"1min":0.308,"5min":0.27300000000000002,"15min":0.255},"max":{"1min":0.58799999999999997,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.28100000000000003}]},{"osd":2,"last update":"Sun Oct 8 00:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.374,"15min":0.34799999999999998},"min":{"1min":0.25900000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39600000000000002,"15min":0.36199999999999999},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.24299999999999999}]},{"osd":3,"last update":"Sun Oct 8 00:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.41599999999999998,"15min":0.38700000000000001},"min":{"1min":0.27600000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.60399999999999998,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.307},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.36899999999999999,"15min":0.35199999999999998},"min":{"1min":0.26200000000000001,"5min":0.249,"15min":0.222},"max":{"1min":0.56100000000000005,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.26400000000000001}]},{"osd":4,"last update":"Sun Oct 8 00:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.56299999999999994,"15min":0.54900000000000004},"min":{"1min":0.35999999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.80900000000000005,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.58899999999999997,"15min":0.57499999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.77100000000000002,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.495}]},{"osd":5,"last update":"Sun Oct 8 00:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.58099999999999996,"15min":0.55300000000000005},"min":{"1min":0.35099999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.73399999999999999,"5min":0.86499999999999999,"15min":0.88200000000000001},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.59899999999999998,"15min":0.58399999999999996},"min":{"1min":0.42699999999999999,"5min":0.38300000000000001,"15min":0.30399999999999999},"max":{"1min":0.83099999999999996,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.55300000000000005}]},{"osd":6,"last update":"Sun Oct 8 00:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58899999999999997,"15min":0.57599999999999996},"min":{"1min":0.38600000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.745,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58799999999999997,"15min":0.57199999999999995},"min":{"1min":0.377,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77900000000000003,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.503}]},{"osd":7,"last update":"Sun Oct 8 00:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.58699999999999997,"15min":0.57299999999999995},"min":{"1min":0.39400000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.67000000000000004,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.625,"15min":0.60499999999999998},"min":{"1min":0.45800000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.78200000000000003,"5min":0.89500000000000002,"15min":0.96699999999999997},"last":0.53100000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574963,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":917,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194394,"statfs":{"total":200000000,"available":199060358,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 00:20:05 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34499999999999997,"15min":0.33600000000000002},"min":{"1min":0.23899999999999999,"5min":0.20399999999999999,"15min":0.193},"max":{"1min":0.504,"5min":0.57599999999999996,"15min":1.0640000000000001},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.36899999999999999,"15min":0.35499999999999998},"min":{"1min":0.216,"5min":0.216,"15min":0.20300000000000001},"max":{"1min":0.45900000000000002,"5min":0.66500000000000004,"15min":0.87},"last":0.29799999999999999}]},{"osd":2,"last update":"Sun Oct 8 00:20:05 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.39600000000000002,"15min":0.38900000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.24099999999999999},"max":{"1min":0.56499999999999995,"5min":0.623,"15min":0.93799999999999994},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.36299999999999999,"15min":0.59399999999999997},"min":{"1min":0.23499999999999999,"5min":0.23000000000000001,"15min":0.185},"max":{"1min":0.47399999999999998,"5min":0.65600000000000003,"15min":12.019},"last":0.28499999999999998}]},{"osd":3,"last update":"Sun Oct 8 00:20:05 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.38400000000000001,"15min":0.37},"min":{"1min":0.30199999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.55800000000000005,"5min":0.628,"15min":1.1020000000000001},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.42599999999999999,"15min":0.40699999999999997},"min":{"1min":0.28699999999999998,"5min":0.26700000000000002,"15min":0.246},"max":{"1min":0.58299999999999996,"5min":0.67400000000000004,"15min":1.0409999999999999},"last":0.374}]},{"osd":4,"last update":"Sun Oct 8 00:20:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.626,"15min":0.62},"min":{"1min":0.44500000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.875,"5min":0.90300000000000002,"15min":1.1779999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.628,"15min":0.626},"min":{"1min":0.41599999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.88700000000000001,"5min":0.93899999999999995,"15min":0.96199999999999997},"last":0.82499999999999996}]},{"osd":5,"last update":"Sun Oct 8 00:20:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.59099999999999997,"15min":0.83099999999999996},"min":{"1min":0.378,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":11.992000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60999999999999999,"15min":0.59599999999999997},"min":{"1min":0.38400000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.84099999999999997,"5min":0.91600000000000004,"15min":1.1299999999999999},"last":0.76900000000000002}]},{"osd":6,"last update":"Sun Oct 8 00:20:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.60499999999999998,"15min":0.59399999999999997},"min":{"1min":0.41799999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.755},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60999999999999999,"15min":0.82499999999999996},"min":{"1min":0.439,"5min":0.39600000000000002,"15min":0.38100000000000001},"max":{"1min":0.85599999999999998,"5min":0.91200000000000003,"15min":11.927},"last":0.84499999999999997}]},{"osd":7,"last update":"Sun Oct 8 00:20:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.61299999999999999,"15min":0.83799999999999997},"min":{"1min":0.443,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":11.965},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63500000000000001,"15min":0.628},"min":{"1min":0.48399999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.86399999999999999,"5min":0.88,"15min":1.1539999999999999},"last":0.80300000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574963,"num_pgs":31,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":917,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194394,"statfs":{"total":200000000,"available":199060426,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.33700000000000002,"15min":0.371},"min":{"1min":0.245,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.46899999999999997,"5min":0.54000000000000004,"15min":2.5139999999999998},"last":0.309},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.34699999999999998,"15min":0.38100000000000001},"min":{"1min":0.26400000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":2.1309999999999998},"last":0.33700000000000002}]},{"osd":1,"last update":"Sun Oct 8 00:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.36299999999999999,"15min":0.38},"min":{"1min":0.26800000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.53900000000000003,"5min":0.67300000000000004,"15min":2.2639999999999998},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.379,"15min":0.41499999999999998},"min":{"1min":0.30399999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.57199999999999995,"5min":0.64400000000000002,"15min":2.5379999999999998},"last":0.42799999999999999}]},{"osd":3,"last update":"Sun Oct 8 00:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.40100000000000002,"15min":0.41899999999999998},"min":{"1min":0.34399999999999997,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":2.2890000000000001},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.41899999999999998,"15min":0.44600000000000001},"min":{"1min":0.34200000000000003,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.63400000000000001,"5min":0.68799999999999994,"15min":2.5619999999999998},"last":0.39400000000000002}]},{"osd":4,"last update":"Sun Oct 8 00:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.55600000000000005,"15min":0.59499999999999997},"min":{"1min":0.42299999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79100000000000004,"5min":0.81499999999999995,"15min":2.488},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.59599999999999997,"15min":0.60199999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.753,"5min":0.92400000000000004,"15min":2.073},"last":0.80300000000000005}]},{"osd":5,"last update":"Sun Oct 8 00:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.61599999999999999,"15min":0.64300000000000002},"min":{"1min":0.44700000000000001,"5min":0.36899999999999999,"15min":0.36099999999999999},"max":{"1min":0.874,"5min":0.874,"15min":2.2069999999999999},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.628,"15min":0.65100000000000002},"min":{"1min":0.47199999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.85399999999999998,"5min":0.94199999999999995,"15min":2.456},"last":0.88}]},{"osd":6,"last update":"Sun Oct 8 00:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.59899999999999998,"15min":0.61499999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83499999999999996,"5min":0.93000000000000005,"15min":2.0339999999999998},"last":0.749},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.61199999999999999,"15min":0.61099999999999999},"min":{"1min":0.45300000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.78200000000000003,"5min":0.94099999999999995,"15min":2.1699999999999999},"last":0.59199999999999997}]},{"osd":7,"last update":"Sun Oct 8 00:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.629},"min":{"1min":0.46300000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.73899999999999999,"5min":0.95999999999999996,"15min":2.1030000000000002},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61799999999999999,"15min":0.624},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40300000000000002},"max":{"1min":0.83799999999999997,"5min":0.90500000000000003,"15min":2.2349999999999999},"last":1.0169999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574963,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":918,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194394,"statfs":{"total":200000000,"available":199059860,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.33800000000000002,"15min":0.35499999999999998},"min":{"1min":0.245,"5min":0.192,"15min":0.192},"max":{"1min":0.46400000000000002,"5min":0.71199999999999997,"15min":1.46},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.35299999999999998,"15min":0.34399999999999997},"min":{"1min":0.28100000000000003,"5min":0.20399999999999999,"15min":0.192},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.32100000000000001}]},{"osd":1,"last update":"Sun Oct 8 00:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.36099999999999999,"15min":0.63700000000000001},"min":{"1min":0.27900000000000003,"5min":0.22700000000000001,"15min":0.20000000000000001},"max":{"1min":0.53500000000000003,"5min":0.57499999999999996,"15min":12.351000000000001},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.38,"15min":0.37},"min":{"1min":0.27100000000000002,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.55100000000000005,"5min":0.60199999999999998,"15min":0.71999999999999997},"last":0.30499999999999999}]},{"osd":2,"last update":"Sun Oct 8 00:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.40500000000000003,"15min":0.41199999999999998},"min":{"1min":0.307,"5min":0.26800000000000002,"15min":0.23499999999999999},"max":{"1min":0.59699999999999998,"5min":0.60899999999999999,"15min":1.5489999999999999},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.41299999999999998,"15min":0.39500000000000002},"min":{"1min":0.34300000000000003,"5min":0.25700000000000001,"15min":0.246},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.753},"last":0.34999999999999998}]},{"osd":4,"last update":"Sun Oct 8 00:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.63100000000000001,"15min":0.85199999999999998},"min":{"1min":0.377,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.95499999999999996,"5min":1.024,"15min":11.333},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.63300000000000001,"15min":0.874},"min":{"1min":0.41599999999999998,"5min":0.38100000000000001,"15min":0.36099999999999999},"max":{"1min":0.85199999999999998,"5min":0.97699999999999998,"15min":12.321999999999999},"last":0.51300000000000001}]},{"osd":5,"last update":"Sun Oct 8 00:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.622,"15min":0.876},"min":{"1min":0.44800000000000001,"5min":0.38300000000000001,"15min":0.35499999999999998},"max":{"1min":0.90200000000000002,"5min":0.93500000000000005,"15min":12.291},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.64800000000000002,"15min":0.61499999999999999},"min":{"1min":0.42299999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.36799999999999999}]},{"osd":6,"last update":"Sun Oct 8 00:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.60499999999999998,"15min":0.82299999999999995},"min":{"1min":0.46000000000000002,"5min":0.35299999999999998,"15min":0.33700000000000002},"max":{"1min":0.81799999999999995,"5min":0.91500000000000004,"15min":11.288},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.61799999999999999,"15min":0.86299999999999999},"min":{"1min":0.432,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.94599999999999995,"5min":1.002,"15min":12.253},"last":0.42299999999999999}]},{"osd":7,"last update":"Sun Oct 8 00:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.628,"15min":0.59499999999999997},"min":{"1min":0.44500000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.3109999999999999},"last":0.505},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.61899999999999999,"15min":0.85199999999999998},"min":{"1min":0.436,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.89300000000000002,"5min":1.0549999999999999,"15min":11.363},"last":0.434}]}]},{"osd":4,"up_from":13,"seq":55834574963,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":919,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194392,"statfs":{"total":200000000,"available":199058336,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.58399999999999996,"15min":0.56499999999999995},"min":{"1min":0.45800000000000002,"5min":0.193,"15min":0.193},"max":{"1min":0.76800000000000002,"5min":0.84799999999999998,"15min":0.93000000000000005},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60499999999999998,"15min":0.58999999999999997},"min":{"1min":0.438,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.79100000000000004,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.52900000000000003}]},{"osd":1,"last update":"Sun Oct 8 00:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55800000000000005,"15min":0.53100000000000003},"min":{"1min":0.42299999999999999,"5min":0.33900000000000002,"15min":0.33700000000000002},"max":{"1min":0.751,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56999999999999995,"15min":0.54400000000000004},"min":{"1min":0.39500000000000002,"5min":0.314,"15min":0.30199999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.81299999999999994},"last":0.499}]},{"osd":2,"last update":"Sun Oct 8 00:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.55200000000000005,"15min":0.52800000000000002},"min":{"1min":0.436,"5min":0.32500000000000001,"15min":0.32100000000000001},"max":{"1min":0.84799999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56799999999999995,"15min":0.53100000000000003},"min":{"1min":0.46800000000000003,"5min":0.35399999999999998,"15min":0.32600000000000001},"max":{"1min":0.73199999999999998,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.435}]},{"osd":3,"last update":"Sun Oct 8 00:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.57699999999999996,"15min":0.54900000000000004},"min":{"1min":0.441,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.72699999999999998,"5min":0.82299999999999995,"15min":0.85399999999999998},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.54200000000000004},"min":{"1min":0.40899999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.74099999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.38100000000000001}]},{"osd":5,"last update":"Sun Oct 8 00:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47799999999999998,"15min":0.48599999999999999},"min":{"1min":0.33600000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.51100000000000001,"15min":0.50700000000000001},"min":{"1min":0.33000000000000002,"5min":0.26400000000000001,"15min":0.24399999999999999},"max":{"1min":0.79100000000000004,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.317}]},{"osd":6,"last update":"Sun Oct 8 00:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48999999999999999,"15min":0.48699999999999999},"min":{"1min":0.34699999999999998,"5min":0.32200000000000001,"15min":0.26400000000000001},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.53700000000000003,"15min":0.54300000000000004},"min":{"1min":0.34999999999999998,"5min":0.34799999999999998,"15min":0.29799999999999999},"max":{"1min":0.76200000000000001,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.46300000000000002}]},{"osd":7,"last update":"Sun Oct 8 00:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.54500000000000004,"15min":0.54600000000000004},"min":{"1min":0.37,"5min":0.33200000000000002,"15min":0.27900000000000003},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.90100000000000002},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57999999999999996,"15min":0.57799999999999996},"min":{"1min":0.39500000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.51400000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574963,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1495,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193816,"statfs":{"total":200000000,"available":198468350,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.53400000000000003,"15min":0.502},"min":{"1min":0.44500000000000001,"5min":0.28799999999999998,"15min":0.249},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.54400000000000004,"15min":0.55200000000000005},"min":{"1min":0.38400000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.88600000000000001},"last":0.47699999999999998}]},{"osd":1,"last update":"Sun Oct 8 00:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.55200000000000005,"15min":0.53300000000000003},"min":{"1min":0.45800000000000002,"5min":0.34399999999999997,"15min":0.32500000000000001},"max":{"1min":0.72599999999999998,"5min":0.80600000000000005,"15min":0.82999999999999996},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.52900000000000003,"15min":0.52400000000000002},"min":{"1min":0.41199999999999998,"5min":0.33100000000000002,"15min":0.33000000000000002},"max":{"1min":0.67900000000000005,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.495}]},{"osd":2,"last update":"Sun Oct 8 00:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.54400000000000004,"15min":0.53400000000000003},"min":{"1min":0.40799999999999997,"5min":0.34499999999999997,"15min":0.32400000000000001},"max":{"1min":0.70699999999999996,"5min":0.749,"15min":0.86199999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.55100000000000005,"15min":0.56200000000000006},"min":{"1min":0.41899999999999998,"5min":0.33400000000000002,"15min":0.29599999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.95399999999999996},"last":0.45700000000000002}]},{"osd":3,"last update":"Sun Oct 8 00:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.56299999999999994,"15min":0.55400000000000005},"min":{"1min":0.42199999999999999,"5min":0.35499999999999998,"15min":0.316},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.622},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55900000000000005,"15min":0.57099999999999995},"min":{"1min":0.45600000000000002,"5min":0.376,"15min":0.371},"max":{"1min":0.69999999999999996,"5min":0.73499999999999999,"15min":1.0529999999999999},"last":0.55900000000000005}]},{"osd":4,"last update":"Sun Oct 8 00:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.47799999999999998,"15min":0.47399999999999998},"min":{"1min":0.307,"5min":0.28399999999999997,"15min":0.27700000000000002},"max":{"1min":0.60499999999999998,"5min":0.78600000000000003,"15min":0.79200000000000004},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.504,"5min":0.52600000000000002,"15min":0.51700000000000002},"min":{"1min":0.371,"5min":0.314,"15min":0.28899999999999998},"max":{"1min":0.77500000000000002,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.52700000000000002}]},{"osd":6,"last update":"Sun Oct 8 00:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.50800000000000001,"15min":0.505},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.27400000000000002},"max":{"1min":0.66900000000000004,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.52600000000000002,"15min":0.50600000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.30499999999999999},"max":{"1min":0.66700000000000004,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.64400000000000002}]},{"osd":7,"last update":"Sun Oct 8 00:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.54000000000000004,"15min":0.52600000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.29399999999999998},"max":{"1min":0.72299999999999998,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.56399999999999995,"15min":0.56000000000000005},"min":{"1min":0.35599999999999998,"5min":0.34499999999999997,"15min":0.32500000000000001},"max":{"1min":0.68000000000000005,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.63200000000000001}]}]}],"pool_statfs":[]}} 2023-10-08T00:20:09.670 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T00:20:09.670 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-08T00:20:09.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.825+0000 7fa793c44700 1 Processor -- start 2023-10-08T00:20:09.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.825+0000 7fa793c44700 1 -- start start 2023-10-08T00:20:09.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.826+0000 7fa793c44700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa78c159ec0 0x7fa78c0af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:09.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.826+0000 7fa793c44700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa78c06acb0 0x7fa78c0afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:09.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.826+0000 7fa793c44700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa78c0b0790 con 0x7fa78c15a290 2023-10-08T00:20:09.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.826+0000 7fa793c44700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa78c0ce1f0 con 0x7fa78c06acb0 2023-10-08T00:20:09.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.826+0000 7fa793c44700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa78c0b0b20 con 0x7fa78c159ec0 2023-10-08T00:20:09.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.826+0000 7fa7921e1700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa78c06acb0 0x7fa78c0afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:09.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.826+0000 7fa7921e1700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa78c06acb0 0x7fa78c0afe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60944/0 (socket says 172.21.15.99:60944) 2023-10-08T00:20:09.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.826+0000 7fa7921e1700 1 -- 172.21.15.99:0/2621851878 learned_addr learned my addr 172.21.15.99:0/2621851878 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:20:09.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.826+0000 7fa7919e0700 1 --2- 172.21.15.99:0/2621851878 >> v2:172.21.15.191:3300/0 conn(0x7fa78c159ec0 0x7fa78c0af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:09.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.826+0000 7fa7921e1700 1 -- 172.21.15.99:0/2621851878 >> v2:172.21.15.191:3300/0 conn(0x7fa78c159ec0 msgr2=0x7fa78c0af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:09.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.826+0000 7fa7921e1700 1 --2- 172.21.15.99:0/2621851878 >> v2:172.21.15.191:3300/0 conn(0x7fa78c159ec0 0x7fa78c0af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:09.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.826+0000 7fa7921e1700 1 -- 172.21.15.99:0/2621851878 >> v1:172.21.15.191:6789/0 conn(0x7fa78c15a290 legacy=0x7fa78c06a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:09.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.826+0000 7fa7921e1700 1 -- 172.21.15.99:0/2621851878 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa78c0b04b0 con 0x7fa78c06acb0 2023-10-08T00:20:09.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.827+0000 7fa7921e1700 1 --2- 172.21.15.99:0/2621851878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa78c06acb0 0x7fa78c0afe80 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fa78c0898e0 tx=0x7fa77c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62a48744b4c8d4ba server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:09.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.827+0000 7fa7909de700 1 -- 172.21.15.99:0/2621851878 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77c00e070 con 0x7fa78c06acb0 2023-10-08T00:20:09.828 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.827+0000 7fa7909de700 1 -- 172.21.15.99:0/2621851878 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa77c00ba10 con 0x7fa78c06acb0 2023-10-08T00:20:09.829 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.827+0000 7fa7909de700 1 -- 172.21.15.99:0/2621851878 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77c0092e0 con 0x7fa78c06acb0 2023-10-08T00:20:09.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.827+0000 7fa793c44700 1 -- 172.21.15.99:0/2621851878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa78c06acb0 msgr2=0x7fa78c0afe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:09.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.827+0000 7fa793c44700 1 --2- 172.21.15.99:0/2621851878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa78c06acb0 0x7fa78c0afe80 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fa78c0898e0 tx=0x7fa77c009840 comp rx=0 tx=0).stop 2023-10-08T00:20:09.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.827+0000 7fa793c44700 1 -- 172.21.15.99:0/2621851878 shutdown_connections 2023-10-08T00:20:09.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.827+0000 7fa793c44700 1 --2- 172.21.15.99:0/2621851878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa78c06acb0 0x7fa78c0afe80 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:09.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.827+0000 7fa793c44700 1 --2- 172.21.15.99:0/2621851878 >> v2:172.21.15.191:3300/0 conn(0x7fa78c159ec0 0x7fa78c0af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:09.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.827+0000 7fa793c44700 1 -- 172.21.15.99:0/2621851878 >> 172.21.15.99:0/2621851878 conn(0x7fa78c0d1880 msgr2=0x7fa78c0cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:20:09.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.827+0000 7fa793c44700 1 -- 172.21.15.99:0/2621851878 shutdown_connections 2023-10-08T00:20:09.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.827+0000 7fa793c44700 1 -- 172.21.15.99:0/2621851878 wait complete. 2023-10-08T00:20:09.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.828+0000 7fa793c44700 1 Processor -- start 2023-10-08T00:20:09.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.828+0000 7fa793c44700 1 -- start start 2023-10-08T00:20:09.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.829+0000 7fa793c44700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa78c06acb0 0x7fa78c154920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:09.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.829+0000 7fa793c44700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa78c159ec0 0x7fa78c0b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:09.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.829+0000 7fa793c44700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa78c15a290 0x7fa78c0b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:09.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.829+0000 7fa793c44700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa78c0c7e60 con 0x7fa78c06acb0 2023-10-08T00:20:09.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.829+0000 7fa793c44700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa78c0c8420 con 0x7fa78c159ec0 2023-10-08T00:20:09.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.829+0000 7fa793c44700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa78c0cb4c0 con 0x7fa78c15a290 2023-10-08T00:20:09.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.829+0000 7fa7919e0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa78c06acb0 0x7fa78c154920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:09.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.829+0000 7fa7919e0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa78c06acb0 0x7fa78c154920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60960/0 (socket says 172.21.15.99:60960) 2023-10-08T00:20:09.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.829+0000 7fa7919e0700 1 -- 172.21.15.99:0/3676888021 learned_addr learned my addr 172.21.15.99:0/3676888021 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:20:09.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.829+0000 7fa7911df700 1 --2- 172.21.15.99:0/3676888021 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa78c159ec0 0x7fa78c0b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:09.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.829+0000 7fa7921e1700 1 --2- 172.21.15.99:0/3676888021 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa78c15a290 0x7fa78c0b25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:09.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.830+0000 7fa7919e0700 1 -- 172.21.15.99:0/3676888021 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa78c15a290 msgr2=0x7fa78c0b25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:09.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.830+0000 7fa7919e0700 1 --2- 172.21.15.99:0/3676888021 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa78c15a290 0x7fa78c0b25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:09.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.830+0000 7fa7919e0700 1 -- 172.21.15.99:0/3676888021 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa78c159ec0 msgr2=0x7fa78c0b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:09.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.830+0000 7fa7919e0700 1 --2- 172.21.15.99:0/3676888021 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa78c159ec0 0x7fa78c0b20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:09.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.830+0000 7fa7919e0700 1 -- 172.21.15.99:0/3676888021 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa77c009ae0 con 0x7fa78c06acb0 2023-10-08T00:20:09.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.830+0000 7fa7921e1700 1 --2- 172.21.15.99:0/3676888021 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa78c15a290 0x7fa78c0b25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:20:09.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.830+0000 7fa7919e0700 1 --2- 172.21.15.99:0/3676888021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa78c06acb0 0x7fa78c154920 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fa78400d890 tx=0x7fa784008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:09.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.830+0000 7fa789ffb700 1 -- 172.21.15.99:0/3676888021 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7840125f0 con 0x7fa78c06acb0 2023-10-08T00:20:09.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.830+0000 7fa789ffb700 1 -- 172.21.15.99:0/3676888021 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa784012750 con 0x7fa78c06acb0 2023-10-08T00:20:09.835 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.830+0000 7fa793c44700 1 -- 172.21.15.99:0/3676888021 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa78c0cb7a0 con 0x7fa78c06acb0 2023-10-08T00:20:09.835 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.830+0000 7fa789ffb700 1 -- 172.21.15.99:0/3676888021 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa784012ad0 con 0x7fa78c06acb0 2023-10-08T00:20:09.835 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.830+0000 7fa793c44700 1 -- 172.21.15.99:0/3676888021 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa78c154e60 con 0x7fa78c06acb0 2023-10-08T00:20:09.835 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.831+0000 7fa793c44700 1 -- 172.21.15.99:0/3676888021 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa78c04c3c0 con 0x7fa78c06acb0 2023-10-08T00:20:09.835 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.831+0000 7fa789ffb700 1 -- 172.21.15.99:0/3676888021 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa78401e460 con 0x7fa78c06acb0 2023-10-08T00:20:09.836 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.831+0000 7fa789ffb700 1 --2- 172.21.15.99:0/3676888021 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa778066f30 0x7fa7780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:09.836 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.832+0000 7fa789ffb700 1 -- 172.21.15.99:0/3676888021 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa784084b50 con 0x7fa78c06acb0 2023-10-08T00:20:09.836 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.832+0000 7fa7911df700 1 --2- 172.21.15.99:0/3676888021 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa778066f30 0x7fa7780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:09.836 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.832+0000 7fa7911df700 1 --2- 172.21.15.99:0/3676888021 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa778066f30 0x7fa7780693e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fa78c0cd460 tx=0x7fa780009060 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:09.836 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:09.836+0000 7fa789ffb700 1 -- 172.21.15.99:0/3676888021 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa784051dc0 con 0x7fa78c06acb0 2023-10-08T00:20:10.094 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.093+0000 7fa793c44700 1 -- 172.21.15.99:0/3676888021 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fa78c0ceab0 con 0x7fa78c06acb0 2023-10-08T00:20:10.097 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.096+0000 7fa789ffb700 1 -- 172.21.15.99:0/3676888021 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+0 (secure 0 0 0) 0x7fa7840553e0 con 0x7fa78c06acb0 2023-10-08T00:20:10.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.098+0000 7fa793c44700 1 -- 172.21.15.99:0/3676888021 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa778066f30 msgr2=0x7fa7780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:10.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.098+0000 7fa793c44700 1 --2- 172.21.15.99:0/3676888021 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa778066f30 0x7fa7780693e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fa78c0cd460 tx=0x7fa780009060 comp rx=0 tx=0).stop 2023-10-08T00:20:10.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.098+0000 7fa793c44700 1 -- 172.21.15.99:0/3676888021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa78c06acb0 msgr2=0x7fa78c154920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:10.100 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.098+0000 7fa793c44700 1 --2- 172.21.15.99:0/3676888021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa78c06acb0 0x7fa78c154920 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fa78400d890 tx=0x7fa784008040 comp rx=0 tx=0).stop 2023-10-08T00:20:10.100 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.099+0000 7fa793c44700 1 -- 172.21.15.99:0/3676888021 shutdown_connections 2023-10-08T00:20:10.100 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.099+0000 7fa793c44700 1 --2- 172.21.15.99:0/3676888021 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa778066f30 0x7fa7780693e0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:10.100 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.099+0000 7fa793c44700 1 --2- 172.21.15.99:0/3676888021 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa78c15a290 0x7fa78c0b25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:10.100 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.099+0000 7fa793c44700 1 --2- 172.21.15.99:0/3676888021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa78c06acb0 0x7fa78c154920 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:10.100 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.099+0000 7fa793c44700 1 --2- 172.21.15.99:0/3676888021 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa78c159ec0 0x7fa78c0b20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:10.100 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.099+0000 7fa793c44700 1 -- 172.21.15.99:0/3676888021 >> 172.21.15.99:0/3676888021 conn(0x7fa78c0d1880 msgr2=0x7fa78c1574c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:20:10.100 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.099+0000 7fa793c44700 1 -- 172.21.15.99:0/3676888021 shutdown_connections 2023-10-08T00:20:10.100 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.099+0000 7fa793c44700 1 -- 172.21.15.99:0/3676888021 wait complete. 2023-10-08T00:20:10.101 INFO:teuthology.orchestra.run.smithi099.stderr:updated caps for client.0 2023-10-08T00:20:10.111 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:20:10.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.225+0000 7ff335bcf700 1 Processor -- start 2023-10-08T00:20:10.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.225+0000 7ff335bcf700 1 -- start start 2023-10-08T00:20:10.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.226+0000 7ff335bcf700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff33015cbd0 0x7ff33015d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:10.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.226+0000 7ff335bcf700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3300cb7c0 con 0x7ff3300cdab0 2023-10-08T00:20:10.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.226+0000 7ff335bcf700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff33015e290 con 0x7ff3300cd6e0 2023-10-08T00:20:10.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.226+0000 7ff335bcf700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff33015e850 con 0x7ff33015cbd0 2023-10-08T00:20:10.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.226+0000 7ff3353ce700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff33015cbd0 0x7ff33015d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:10.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.226+0000 7ff32ffff700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7ff3300cdab0 0x7ff33015d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56108/0 (socket says 172.21.15.99:56108) 2023-10-08T00:20:10.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.226+0000 7ff3353ce700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff33015cbd0 0x7ff33015d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60964/0 (socket says 172.21.15.99:60964) 2023-10-08T00:20:10.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.226+0000 7ff32ffff700 1 -- 172.21.15.99:0/889436197 learned_addr learned my addr 172.21.15.99:0/889436197 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:20:10.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.226+0000 7ff3353ce700 1 -- 172.21.15.99:0/889436197 >> v1:172.21.15.191:6789/0 conn(0x7ff3300cd6e0 legacy=0x7ff3300cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:10.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.226+0000 7ff3353ce700 1 -- 172.21.15.99:0/889436197 >> v1:172.21.15.99:6789/0 conn(0x7ff3300cdab0 legacy=0x7ff33015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:10.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.226+0000 7ff3353ce700 1 -- 172.21.15.99:0/889436197 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff33015e570 con 0x7ff33015cbd0 2023-10-08T00:20:10.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.226+0000 7ff32f7fe700 1 -- 172.21.15.99:0/889436197 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1610162003 0 0) 0x7ff3300cb7c0 con 0x7ff3300cdab0 2023-10-08T00:20:10.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.226+0000 7ff3353ce700 1 --2- 172.21.15.99:0/889436197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff33015cbd0 0x7ff33015d050 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7ff3180098e0 tx=0x7ff3180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7faabde9c68b6ed7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:10.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.226+0000 7ff32f7fe700 1 -- 172.21.15.99:0/889436197 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff318009e80 con 0x7ff33015cbd0 2023-10-08T00:20:10.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.226+0000 7ff32f7fe700 1 -- 172.21.15.99:0/889436197 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff31800b820 con 0x7ff33015cbd0 2023-10-08T00:20:10.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.226+0000 7ff32f7fe700 1 -- 172.21.15.99:0/889436197 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31800bba0 con 0x7ff33015cbd0 2023-10-08T00:20:10.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.227+0000 7ff335bcf700 1 -- 172.21.15.99:0/889436197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff33015cbd0 msgr2=0x7ff33015d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:10.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.227+0000 7ff335bcf700 1 --2- 172.21.15.99:0/889436197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff33015cbd0 0x7ff33015d050 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7ff3180098e0 tx=0x7ff3180091a0 comp rx=0 tx=0).stop 2023-10-08T00:20:10.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.227+0000 7ff335bcf700 1 -- 172.21.15.99:0/889436197 shutdown_connections 2023-10-08T00:20:10.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.227+0000 7ff335bcf700 1 --2- 172.21.15.99:0/889436197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff33015cbd0 0x7ff33015d050 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:10.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.227+0000 7ff335bcf700 1 -- 172.21.15.99:0/889436197 >> 172.21.15.99:0/889436197 conn(0x7ff330067660 msgr2=0x7ff330067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:20:10.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.227+0000 7ff335bcf700 1 -- 172.21.15.99:0/889436197 shutdown_connections 2023-10-08T00:20:10.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.227+0000 7ff335bcf700 1 -- 172.21.15.99:0/889436197 wait complete. 2023-10-08T00:20:10.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.228+0000 7ff335bcf700 1 Processor -- start 2023-10-08T00:20:10.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.228+0000 7ff335bcf700 1 -- start start 2023-10-08T00:20:10.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.229+0000 7ff335bcf700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3300cd6e0 0x7ff3300c9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:10.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.229+0000 7ff335bcf700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3300cdab0 0x7ff33016a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:10.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.229+0000 7ff335bcf700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff33015cbd0 0x7ff3300c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:10.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.229+0000 7ff335bcf700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3300cc360 con 0x7ff3300cd6e0 2023-10-08T00:20:10.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.229+0000 7ff335bcf700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33016a640 con 0x7ff3300cdab0 2023-10-08T00:20:10.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.229+0000 7ff335bcf700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff33016a910 con 0x7ff33015cbd0 2023-10-08T00:20:10.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.229+0000 7ff334bcd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3300cd6e0 0x7ff3300c9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:10.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.229+0000 7ff334bcd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3300cd6e0 0x7ff3300c9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60966/0 (socket says 172.21.15.99:60966) 2023-10-08T00:20:10.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.229+0000 7ff334bcd700 1 -- 172.21.15.99:0/967451540 learned_addr learned my addr 172.21.15.99:0/967451540 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:20:10.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.230+0000 7ff3353ce700 1 --2- 172.21.15.99:0/967451540 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff33015cbd0 0x7ff3300c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:10.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.230+0000 7ff32ffff700 1 --2- 172.21.15.99:0/967451540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3300cdab0 0x7ff33016a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:10.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.230+0000 7ff334bcd700 1 -- 172.21.15.99:0/967451540 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff33015cbd0 msgr2=0x7ff3300c7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:10.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.230+0000 7ff334bcd700 1 --2- 172.21.15.99:0/967451540 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff33015cbd0 0x7ff3300c7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:10.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.230+0000 7ff334bcd700 1 -- 172.21.15.99:0/967451540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3300cdab0 msgr2=0x7ff33016a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:10.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.230+0000 7ff334bcd700 1 --2- 172.21.15.99:0/967451540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3300cdab0 0x7ff33016a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:10.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.230+0000 7ff334bcd700 1 -- 172.21.15.99:0/967451540 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff318009420 con 0x7ff3300cd6e0 2023-10-08T00:20:10.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.230+0000 7ff32ffff700 1 --2- 172.21.15.99:0/967451540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3300cdab0 0x7ff33016a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:20:10.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.230+0000 7ff334bcd700 1 --2- 172.21.15.99:0/967451540 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3300cd6e0 0x7ff3300c9af0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7ff32800bbb0 tx=0x7ff328009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:10.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.230+0000 7ff32cff9700 1 -- 172.21.15.99:0/967451540 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3280092e0 con 0x7ff3300cd6e0 2023-10-08T00:20:10.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.230+0000 7ff32cff9700 1 -- 172.21.15.99:0/967451540 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff32800c7a0 con 0x7ff3300cd6e0 2023-10-08T00:20:10.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.230+0000 7ff335bcf700 1 -- 172.21.15.99:0/967451540 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff33016abf0 con 0x7ff3300cd6e0 2023-10-08T00:20:10.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.230+0000 7ff32cff9700 1 -- 172.21.15.99:0/967451540 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32800cb40 con 0x7ff3300cd6e0 2023-10-08T00:20:10.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.230+0000 7ff335bcf700 1 -- 172.21.15.99:0/967451540 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3300b3bb0 con 0x7ff3300cd6e0 2023-10-08T00:20:10.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.231+0000 7ff335bcf700 1 -- 172.21.15.99:0/967451540 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3300b3a10 con 0x7ff3300cd6e0 2023-10-08T00:20:10.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.231+0000 7ff32cff9700 1 -- 172.21.15.99:0/967451540 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff328012070 con 0x7ff3300cd6e0 2023-10-08T00:20:10.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.232+0000 7ff32cff9700 1 --2- 172.21.15.99:0/967451540 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff310066f30 0x7ff3100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:10.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.232+0000 7ff32cff9700 1 -- 172.21.15.99:0/967451540 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff328084210 con 0x7ff3300cd6e0 2023-10-08T00:20:10.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.232+0000 7ff32ffff700 1 --2- 172.21.15.99:0/967451540 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff310066f30 0x7ff3100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:10.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.232+0000 7ff32ffff700 1 --2- 172.21.15.99:0/967451540 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff310066f30 0x7ff3100693e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff3200011e0 tx=0x7ff320009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:10.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.236+0000 7ff32cff9700 1 -- 172.21.15.99:0/967451540 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff3300b3a10 con 0x7ff3300cd6e0 2023-10-08T00:20:10.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.459+0000 7ff335bcf700 1 -- 172.21.15.99:0/967451540 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff3300b3a10 con 0x7ff3300cd6e0 2023-10-08T00:20:10.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.460+0000 7ff32cff9700 1 -- 172.21.15.99:0/967451540 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4601 (secure 0 0 0) 0x7ff3300b3a10 con 0x7ff3300cd6e0 2023-10-08T00:20:10.461 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:20:10.461 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":77,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6257,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/661016677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":661016677},{"type":"v1","addr":"172.21.15.191:6837","nonce":661016677}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":7012,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/4102051559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":4102051559},{"type":"v1","addr":"172.21.15.99:6835","nonce":4102051559}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":7021,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/658854389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":658854389},{"type":"v1","addr":"172.21.15.191:6836","nonce":658854389}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:20:04.613994+0000","modified":"2023-10-08T00:20:08.255350+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7791},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7791":{"gid":7791,"name":"c","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6834/3880334948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3880334948},{"type":"v1","addr":"172.21.15.99:6834","nonce":3880334948}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-08T00:20:10.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.462+0000 7ff335bcf700 1 -- 172.21.15.99:0/967451540 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff310066f30 msgr2=0x7ff3100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:10.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.462+0000 7ff335bcf700 1 --2- 172.21.15.99:0/967451540 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff310066f30 0x7ff3100693e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff3200011e0 tx=0x7ff320009040 comp rx=0 tx=0).stop 2023-10-08T00:20:10.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.462+0000 7ff335bcf700 1 -- 172.21.15.99:0/967451540 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3300cd6e0 msgr2=0x7ff3300c9af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:10.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.462+0000 7ff335bcf700 1 --2- 172.21.15.99:0/967451540 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3300cd6e0 0x7ff3300c9af0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7ff32800bbb0 tx=0x7ff328009840 comp rx=0 tx=0).stop 2023-10-08T00:20:10.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.462+0000 7ff335bcf700 1 -- 172.21.15.99:0/967451540 shutdown_connections 2023-10-08T00:20:10.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.462+0000 7ff335bcf700 1 --2- 172.21.15.99:0/967451540 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff310066f30 0x7ff3100693e0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:10.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.462+0000 7ff335bcf700 1 --2- 172.21.15.99:0/967451540 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff33015cbd0 0x7ff3300c7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:10.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.462+0000 7ff335bcf700 1 --2- 172.21.15.99:0/967451540 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3300cd6e0 0x7ff3300c9af0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:10.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.462+0000 7ff335bcf700 1 --2- 172.21.15.99:0/967451540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3300cdab0 0x7ff33016a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:10.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.462+0000 7ff335bcf700 1 -- 172.21.15.99:0/967451540 >> 172.21.15.99:0/967451540 conn(0x7ff330067660 msgr2=0x7ff3300b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:20:10.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.462+0000 7ff335bcf700 1 -- 172.21.15.99:0/967451540 shutdown_connections 2023-10-08T00:20:10.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:10.462+0000 7ff335bcf700 1 -- 172.21.15.99:0/967451540 wait complete. 2023-10-08T00:20:10.464 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 77 2023-10-08T00:20:10.541 INFO:tasks.ceph.mds.c.smithi099.stderr:2023-10-08T00:20:10.540+0000 7f61d0334700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T00:20:10.877 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 77, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T00:20:04.613994+0000', 'modified': '2023-10-08T00:20:08.255350+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7791}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7791': {'gid': 7791, 'name': 'c', 'rank': 0, 'incarnation': 74, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.99:6834/3880334948', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6832', 'nonce': 3880334948}, {'type': 'v1', 'addr': '172.21.15.99:6834', 'nonce': 3880334948}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-08T00:20:10.877 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T00:20:10.877 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-08T00:20:11.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.001+0000 7efc4e221700 1 Processor -- start 2023-10-08T00:20:11.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.001+0000 7efc4e221700 1 -- start start 2023-10-08T00:20:11.005 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.002+0000 7efc4e221700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc480aed30 0x7efc480c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:11.005 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.002+0000 7efc4e221700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc480c7ef0 con 0x7efc480af5f0 2023-10-08T00:20:11.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.002+0000 7efc4e221700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc48156560 con 0x7efc480682c0 2023-10-08T00:20:11.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.002+0000 7efc4e221700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc480c72e0 con 0x7efc480aed30 2023-10-08T00:20:11.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.002+0000 7efc4da20700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7efc480af5f0 0x7efc48068d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56122/0 (socket says 172.21.15.99:56122) 2023-10-08T00:20:11.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.002+0000 7efc4da20700 1 -- 172.21.15.99:0/2006615519 learned_addr learned my addr 172.21.15.99:0/2006615519 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:20:11.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.002+0000 7efc4d21f700 1 --2- 172.21.15.99:0/2006615519 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc480aed30 0x7efc480c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:11.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.002+0000 7efc3ffff700 1 -- 172.21.15.99:0/2006615519 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1672831358 0 0) 0x7efc480c7ef0 con 0x7efc480af5f0 2023-10-08T00:20:11.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.002+0000 7efc3ffff700 1 -- 172.21.15.99:0/2006615519 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc34004680 con 0x7efc480af5f0 2023-10-08T00:20:11.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.003+0000 7efc3ffff700 1 -- 172.21.15.99:0/2006615519 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1735928658 0 0) 0x7efc34004680 con 0x7efc480af5f0 2023-10-08T00:20:11.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.003+0000 7efc3ffff700 1 -- 172.21.15.99:0/2006615519 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc480aed30 msgr2=0x7efc480c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:11.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.003+0000 7efc3ffff700 1 --2- 172.21.15.99:0/2006615519 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc480aed30 0x7efc480c6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:11.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.003+0000 7efc3ffff700 1 -- 172.21.15.99:0/2006615519 >> v1:172.21.15.191:6789/0 conn(0x7efc480682c0 legacy=0x7efc480c8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:11.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.003+0000 7efc3ffff700 1 -- 172.21.15.99:0/2006615519 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc480c75c0 con 0x7efc480af5f0 2023-10-08T00:20:11.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.003+0000 7efc3ffff700 1 -- 172.21.15.99:0/2006615519 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7efc400017f0 con 0x7efc480af5f0 2023-10-08T00:20:11.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.003+0000 7efc3ffff700 1 -- 172.21.15.99:0/2006615519 >> v1:172.21.15.99:6789/0 conn(0x7efc480af5f0 legacy=0x7efc48068d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:11.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.003+0000 7efc3ffff700 1 --2- 172.21.15.99:0/2006615519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc34004ba0 0x7efc34004fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:11.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.003+0000 7efc3ffff700 1 --2- 172.21.15.99:0/2006615519 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc480aed30 0x7efc34005920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:11.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.003+0000 7efc3ffff700 1 --2- 172.21.15.99:0/2006615519 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efc34005e60 0x7efc340062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:11.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.003+0000 7efc3ffff700 1 -- 172.21.15.99:0/2006615519 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc480c72e0 con 0x7efc34004ba0 2023-10-08T00:20:11.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.003+0000 7efc3ffff700 1 -- 172.21.15.99:0/2006615519 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3400cc80 con 0x7efc34005e60 2023-10-08T00:20:11.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.003+0000 7efc3ffff700 1 -- 172.21.15.99:0/2006615519 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc340035f0 con 0x7efc480aed30 2023-10-08T00:20:11.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.003+0000 7efc4d21f700 1 --2- 172.21.15.99:0/2006615519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc34004ba0 0x7efc34004fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:11.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.003+0000 7efc4ca1e700 1 --2- 172.21.15.99:0/2006615519 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc480aed30 0x7efc34005920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:11.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.003+0000 7efc4da20700 1 --2- 172.21.15.99:0/2006615519 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efc34005e60 0x7efc340062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:11.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.003+0000 7efc4d21f700 1 -- 172.21.15.99:0/2006615519 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc480aed30 msgr2=0x7efc34005920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:11.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.003+0000 7efc4d21f700 1 --2- 172.21.15.99:0/2006615519 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc480aed30 0x7efc34005920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:11.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.003+0000 7efc4d21f700 1 -- 172.21.15.99:0/2006615519 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efc34005e60 msgr2=0x7efc340062d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:11.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.003+0000 7efc4d21f700 1 --2- 172.21.15.99:0/2006615519 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efc34005e60 0x7efc340062d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:11.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.003+0000 7efc4d21f700 1 -- 172.21.15.99:0/2006615519 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc34004680 con 0x7efc34004ba0 2023-10-08T00:20:11.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.003+0000 7efc4da20700 1 --2- 172.21.15.99:0/2006615519 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efc34005e60 0x7efc340062d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:20:11.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.003+0000 7efc4d21f700 1 --2- 172.21.15.99:0/2006615519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc34004ba0 0x7efc34004fb0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7efc44009c00 tx=0x7efc440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=864cf3a3b2ed6d10 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:11.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.003+0000 7efc3ffff700 1 -- 172.21.15.99:0/2006615519 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4400e710 con 0x7efc34004ba0 2023-10-08T00:20:11.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.003+0000 7efc3ffff700 1 -- 172.21.15.99:0/2006615519 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc4400e870 con 0x7efc34004ba0 2023-10-08T00:20:11.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.003+0000 7efc3ffff700 1 -- 172.21.15.99:0/2006615519 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4400ec30 con 0x7efc34004ba0 2023-10-08T00:20:11.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.004+0000 7efc4e221700 1 -- 172.21.15.99:0/2006615519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc34004ba0 msgr2=0x7efc34004fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:11.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.004+0000 7efc4e221700 1 --2- 172.21.15.99:0/2006615519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc34004ba0 0x7efc34004fb0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7efc44009c00 tx=0x7efc440091a0 comp rx=0 tx=0).stop 2023-10-08T00:20:11.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.004+0000 7efc4da20700 1 -- 172.21.15.99:0/2006615519 reap_dead start 2023-10-08T00:20:11.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.004+0000 7efc4e221700 1 -- 172.21.15.99:0/2006615519 shutdown_connections 2023-10-08T00:20:11.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.004+0000 7efc4e221700 1 -- 172.21.15.99:0/2006615519 >> 172.21.15.99:0/2006615519 conn(0x7efc480cc760 msgr2=0x7efc480ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:20:11.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.004+0000 7efc4e221700 1 -- 172.21.15.99:0/2006615519 shutdown_connections 2023-10-08T00:20:11.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.004+0000 7efc4e221700 1 -- 172.21.15.99:0/2006615519 wait complete. 2023-10-08T00:20:11.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.005+0000 7efc4e221700 1 Processor -- start 2023-10-08T00:20:11.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.005+0000 7efc4e221700 1 -- start start 2023-10-08T00:20:11.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.006+0000 7efc4e221700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efc34004ba0 0x7efc48168cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:11.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.006+0000 7efc4e221700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc480c5af0 0x7efc480c5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:11.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.006+0000 7efc4e221700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc4815a1c0 0x7efc4815a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:11.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.006+0000 7efc4e221700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc48169210 con 0x7efc480c5af0 2023-10-08T00:20:11.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.006+0000 7efc4e221700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc48155770 con 0x7efc34004ba0 2023-10-08T00:20:11.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.006+0000 7efc4e221700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc480c3dc0 con 0x7efc4815a1c0 2023-10-08T00:20:11.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.006+0000 7efc4ca1e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc480c5af0 0x7efc480c5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:11.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.006+0000 7efc4ca1e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc480c5af0 0x7efc480c5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60986/0 (socket says 172.21.15.99:60986) 2023-10-08T00:20:11.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.006+0000 7efc4ca1e700 1 -- 172.21.15.99:0/3197029458 learned_addr learned my addr 172.21.15.99:0/3197029458 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:20:11.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.006+0000 7efc4da20700 1 --2- 172.21.15.99:0/3197029458 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc4815a1c0 0x7efc4815a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:11.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.006+0000 7efc4d21f700 1 --2- 172.21.15.99:0/3197029458 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efc34004ba0 0x7efc48168cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:11.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.006+0000 7efc4ca1e700 1 -- 172.21.15.99:0/3197029458 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc4815a1c0 msgr2=0x7efc4815a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:11.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.006+0000 7efc4ca1e700 1 --2- 172.21.15.99:0/3197029458 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc4815a1c0 0x7efc4815a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:11.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.006+0000 7efc4ca1e700 1 -- 172.21.15.99:0/3197029458 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efc34004ba0 msgr2=0x7efc48168cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:11.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.006+0000 7efc4ca1e700 1 --2- 172.21.15.99:0/3197029458 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efc34004ba0 0x7efc48168cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:11.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.006+0000 7efc4ca1e700 1 -- 172.21.15.99:0/3197029458 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc480c64a0 con 0x7efc480c5af0 2023-10-08T00:20:11.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.007+0000 7efc4ca1e700 1 --2- 172.21.15.99:0/3197029458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc480c5af0 0x7efc480c5f60 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7efc38002980 tx=0x7efc38007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:11.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.007+0000 7efc3d7fa700 1 -- 172.21.15.99:0/3197029458 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3800f8d0 con 0x7efc480c5af0 2023-10-08T00:20:11.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.007+0000 7efc3d7fa700 1 -- 172.21.15.99:0/3197029458 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc3800fa30 con 0x7efc480c5af0 2023-10-08T00:20:11.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.007+0000 7efc3d7fa700 1 -- 172.21.15.99:0/3197029458 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3800fdf0 con 0x7efc480c5af0 2023-10-08T00:20:11.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.007+0000 7efc4e221700 1 -- 172.21.15.99:0/3197029458 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc4815aba0 con 0x7efc480c5af0 2023-10-08T00:20:11.015 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.007+0000 7efc4e221700 1 -- 172.21.15.99:0/3197029458 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc4815ae10 con 0x7efc480c5af0 2023-10-08T00:20:11.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.008+0000 7efc3d7fa700 1 -- 172.21.15.99:0/3197029458 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efc38006a30 con 0x7efc480c5af0 2023-10-08T00:20:11.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.008+0000 7efc3d7fa700 1 --2- 172.21.15.99:0/3197029458 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efc2c066f30 0x7efc2c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:11.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.008+0000 7efc3d7fa700 1 -- 172.21.15.99:0/3197029458 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5705+0+0 (secure 0 0 0) 0x7efc38084590 con 0x7efc480c5af0 2023-10-08T00:20:11.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.009+0000 7efc4d21f700 1 --2- 172.21.15.99:0/3197029458 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efc2c066f30 0x7efc2c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:11.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.009+0000 7efc4d21f700 1 --2- 172.21.15.99:0/3197029458 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efc2c066f30 0x7efc2c0693e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7efc440091e0 tx=0x7efc4401a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:11.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.010+0000 7efc5877bb80 1 Processor -- start 2023-10-08T00:20:11.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.010+0000 7efc5877bb80 1 -- start start 2023-10-08T00:20:11.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.011+0000 7efc5877bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574e4f05480 0x5574e4ee47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:11.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.011+0000 7efc5877bb80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5574e4eb85f0 0x5574e4f05850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:11.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.011+0000 7efc5877bb80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574e4ebab80 0x5574e4ebaff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:11.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.011+0000 7efc5877bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574e4e11a70 con 0x5574e4f05480 2023-10-08T00:20:11.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.011+0000 7efc5877bb80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574e4e0f270 con 0x5574e4ebab80 2023-10-08T00:20:11.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.011+0000 7efc5877bb80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x5574e4e0b060 con 0x5574e4eb85f0 2023-10-08T00:20:11.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.011+0000 7efc4d21f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574e4f05480 0x5574e4ee47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:11.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.011+0000 7efc4d21f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574e4f05480 0x5574e4ee47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60998/0 (socket says 172.21.15.99:60998) 2023-10-08T00:20:11.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.011+0000 7efc4d21f700 1 -- 172.21.15.99:0/3084830832 learned_addr learned my addr 172.21.15.99:0/3084830832 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:20:11.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.011+0000 7efc4da20700 1 --2- 172.21.15.99:0/3084830832 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574e4ebab80 0x5574e4ebaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:11.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.011+0000 7efc4ca1e700 1 --2- 172.21.15.99:0/3084830832 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5574e4eb85f0 0x5574e4f05850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:11.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.011+0000 7efc4d21f700 1 -- 172.21.15.99:0/3084830832 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5574e4eb85f0 msgr2=0x5574e4f05850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:11.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.011+0000 7efc4d21f700 1 --2- 172.21.15.99:0/3084830832 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5574e4eb85f0 0x5574e4f05850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:11.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.011+0000 7efc4d21f700 1 -- 172.21.15.99:0/3084830832 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574e4ebab80 msgr2=0x5574e4ebaff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:11.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.011+0000 7efc4d21f700 1 --2- 172.21.15.99:0/3084830832 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574e4ebab80 0x5574e4ebaff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:11.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.011+0000 7efc4d21f700 1 -- 172.21.15.99:0/3084830832 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574e4ca5270 con 0x5574e4f05480 2023-10-08T00:20:11.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.011+0000 7efc4ca1e700 1 --2- 172.21.15.99:0/3084830832 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5574e4eb85f0 0x5574e4f05850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:20:11.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.012+0000 7efc4d21f700 1 --2- 172.21.15.99:0/3084830832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574e4f05480 0x5574e4ee47d0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7efc4400e4b0 tx=0x7efc4401f050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de79a2878d580802 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:11.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.012+0000 7efc327fc700 1 -- 172.21.15.99:0/3084830832 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc44030070 con 0x5574e4f05480 2023-10-08T00:20:11.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.012+0000 7efc327fc700 1 -- 172.21.15.99:0/3084830832 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc44023040 con 0x5574e4f05480 2023-10-08T00:20:11.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.012+0000 7efc327fc700 1 -- 172.21.15.99:0/3084830832 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc44013470 con 0x5574e4f05480 2023-10-08T00:20:11.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.012+0000 7efc5877bb80 1 -- 172.21.15.99:0/3084830832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574e4f05480 msgr2=0x5574e4ee47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:11.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.012+0000 7efc5877bb80 1 --2- 172.21.15.99:0/3084830832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574e4f05480 0x5574e4ee47d0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7efc4400e4b0 tx=0x7efc4401f050 comp rx=0 tx=0).stop 2023-10-08T00:20:11.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.012+0000 7efc5877bb80 1 -- 172.21.15.99:0/3084830832 shutdown_connections 2023-10-08T00:20:11.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.012+0000 7efc5877bb80 1 --2- 172.21.15.99:0/3084830832 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574e4ebab80 0x5574e4ebaff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:11.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.012+0000 7efc5877bb80 1 --2- 172.21.15.99:0/3084830832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574e4f05480 0x5574e4ee47d0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:11.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.012+0000 7efc5877bb80 1 --2- 172.21.15.99:0/3084830832 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5574e4eb85f0 0x5574e4f05850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:11.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.012+0000 7efc5877bb80 1 -- 172.21.15.99:0/3084830832 >> 172.21.15.99:0/3084830832 conn(0x5574e4ebc0c0 msgr2=0x5574e4ebc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:20:11.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.012+0000 7efc5877bb80 1 -- 172.21.15.99:0/3084830832 shutdown_connections 2023-10-08T00:20:11.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.012+0000 7efc5877bb80 1 -- 172.21.15.99:0/3084830832 wait complete. 2023-10-08T00:20:11.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.013+0000 7efc5877bb80 1 -- start start 2023-10-08T00:20:11.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.013+0000 7efc5877bb80 1 Processor -- start 2023-10-08T00:20:11.022 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.014+0000 7efc5877bb80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5574e4eb85f0 0x5574e4f14e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:11.022 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.014+0000 7efc5877bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574e4ebab80 0x5574e4edebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:11.022 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.014+0000 7efc5877bb80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574e4f05480 0x5574e4ebc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:11.022 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.014+0000 7efc5877bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574e4db6890 con 0x5574e4ebab80 2023-10-08T00:20:11.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.014+0000 7efc5877bb80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574e4dd1ea0 con 0x5574e4f05480 2023-10-08T00:20:11.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.014+0000 7efc5877bb80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x5574e4c8de30 con 0x5574e4eb85f0 2023-10-08T00:20:11.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.014+0000 7efc5877bb80 10 client.0 _ll_register_callbacks cb 0x5574e4e7e2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T00:20:11.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.014+0000 7efc4ca1e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574e4ebab80 0x5574e4edebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:11.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.014+0000 7efc4ca1e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574e4ebab80 0x5574e4edebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:32782/0 (socket says 172.21.15.99:32782) 2023-10-08T00:20:11.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.014+0000 7efc4ca1e700 1 -- 172.21.15.99:0/1618508556 learned_addr learned my addr 172.21.15.99:0/1618508556 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:20:11.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.014+0000 7efc4da20700 1 --2- 172.21.15.99:0/1618508556 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574e4f05480 0x5574e4ebc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:11.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.014+0000 7efc4d21f700 1 --2- 172.21.15.99:0/1618508556 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5574e4eb85f0 0x5574e4f14e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:11.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.014+0000 7efc4ca1e700 1 -- 172.21.15.99:0/1618508556 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5574e4eb85f0 msgr2=0x5574e4f14e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:11.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.014+0000 7efc4ca1e700 1 --2- 172.21.15.99:0/1618508556 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5574e4eb85f0 0x5574e4f14e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:11.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.014+0000 7efc4ca1e700 1 -- 172.21.15.99:0/1618508556 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574e4f05480 msgr2=0x5574e4ebc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:11.024 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.014+0000 7efc4ca1e700 1 --2- 172.21.15.99:0/1618508556 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574e4f05480 0x5574e4ebc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:11.024 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.014+0000 7efc4ca1e700 1 -- 172.21.15.99:0/1618508556 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc34004680 con 0x5574e4ebab80 2023-10-08T00:20:11.024 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.014+0000 7efc4ca1e700 1 --2- 172.21.15.99:0/1618508556 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574e4ebab80 0x5574e4edebf0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7efc380551c0 tx=0x7efc38082a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0501378d11efbf6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:11.024 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.014+0000 7efc277fe700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T00:20:11.024 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.015+0000 7efc277fe700 1 -- 172.21.15.99:0/1618508556 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc380554c0 con 0x5574e4ebab80 2023-10-08T00:20:11.024 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.015+0000 7efc277fe700 1 -- 172.21.15.99:0/1618508556 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(168..168 src has 1..168) v4 ==== 5705+0+0 (secure 0 0 0) 0x7efc38058710 con 0x5574e4ebab80 2023-10-08T00:20:11.024 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.015+0000 7efc5877bb80 1 -- 172.21.15.99:0/1618508556 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5574e4eafcd0 con 0x5574e4ebab80 2023-10-08T00:20:11.024 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.015+0000 7efc277fe700 1 -- 172.21.15.99:0/1618508556 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc3805c460 con 0x5574e4ebab80 2023-10-08T00:20:11.024 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.015+0000 7efc5877bb80 10 client.7938 fetch_fsmap learned FSMap version 77 2023-10-08T00:20:11.024 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.015+0000 7efc5877bb80 1 -- 172.21.15.99:0/1618508556 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) v3 -- 0x5574e4c47fe0 con 0x5574e4ebab80 2023-10-08T00:20:11.024 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.015+0000 7efc277fe700 1 -- 172.21.15.99:0/1618508556 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5574e4eafcd0 con 0x5574e4ebab80 2023-10-08T00:20:11.024 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.015+0000 7efc277fe700 1 -- 172.21.15.99:0/1618508556 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 77) v1 ==== 3100+0+0 (secure 0 0 0) 0x7efc380549b0 con 0x5574e4ebab80 2023-10-08T00:20:11.024 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.015+0000 7efc5877bb80 10 client.7938 fetch_fsmap finished waiting for FSMap version 77 2023-10-08T00:20:11.025 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.015+0000 7efc5877bb80 10 client.7938 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-08T00:20:11.025 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.015+0000 7efc5877bb80 20 client.7938 populate_metadata read hostname 'smithi099' 2023-10-08T00:20:11.025 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.015+0000 7efc5877bb80 1 --2- 172.21.15.99:0/1618508556 >> [v2:172.21.15.99:6832/3880334948,v1:172.21.15.99:6834/3880334948] conn(0x5574e4f047b0 0x5574e4f04b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:11.025 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.015+0000 7efc5877bb80 4 client.7938 mds_command: new command op to 7791 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-08T00:20:11.025 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.015+0000 7efc5877bb80 1 -- 172.21.15.99:0/1618508556 --> [v2:172.21.15.99:6832/3880334948,v1:172.21.15.99:6834/3880334948] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5574e4e8caa0 con 0x5574e4f047b0 2023-10-08T00:20:11.025 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.015+0000 7efc4d21f700 1 --2- 172.21.15.99:0/1618508556 >> [v2:172.21.15.99:6832/3880334948,v1:172.21.15.99:6834/3880334948] conn(0x5574e4f047b0 0x5574e4f04b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:11.025 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.015+0000 7efc277fe700 10 client.7938 ms_handle_connect on v2:172.21.15.99:6832/3880334948 2023-10-08T00:20:11.025 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.015+0000 7efc4d21f700 1 --2- 172.21.15.99:0/1618508556 >> [v2:172.21.15.99:6832/3880334948,v1:172.21.15.99:6834/3880334948] conn(0x5574e4f047b0 0x5574e4f04b80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e6640d8fcbf20be server_cookie=de0dc8033d590fc6 in_seq=0 out_seq=0 2023-10-08T00:20:11.025 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.016+0000 7efc277fe700 1 -- 172.21.15.99:0/1618508556 <== mds.0 v2:172.21.15.99:6832/3880334948 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5574e4e8caa0 con 0x5574e4f047b0 2023-10-08T00:20:11.025 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.016+0000 7efc277fe700 10 client.7938 handle_command_reply: tid=0 2023-10-08T00:20:11.025 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.016+0000 7efc5877bb80 1 client.7938 shutdown 2023-10-08T00:20:11.025 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.016+0000 7efc5877bb80 1 -- 172.21.15.99:0/1618508556 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574e4ebab80 msgr2=0x5574e4edebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:11.025 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.016+0000 7efc5877bb80 1 --2- 172.21.15.99:0/1618508556 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574e4ebab80 0x5574e4edebf0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7efc380551c0 tx=0x7efc38082a10 comp rx=0 tx=0).stop 2023-10-08T00:20:11.026 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.016+0000 7efc5877bb80 1 -- 172.21.15.99:0/1618508556 shutdown_connections 2023-10-08T00:20:11.026 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.016+0000 7efc5877bb80 1 --2- 172.21.15.99:0/1618508556 >> [v2:172.21.15.99:6832/3880334948,v1:172.21.15.99:6834/3880334948] conn(0x5574e4f047b0 0x5574e4f04b80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:11.026 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.016+0000 7efc5877bb80 1 --2- 172.21.15.99:0/1618508556 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574e4f05480 0x5574e4ebc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:11.026 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.016+0000 7efc277fe700 0 client.7938 ms_handle_reset on v2:172.21.15.99:6832/3880334948 2023-10-08T00:20:11.026 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.016+0000 7efc5877bb80 1 --2- 172.21.15.99:0/1618508556 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5574e4eb85f0 0x5574e4f14e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:11.026 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.016+0000 7efc5877bb80 1 --2- 172.21.15.99:0/1618508556 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574e4ebab80 0x5574e4edebf0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:11.026 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.016+0000 7efc5877bb80 1 -- 172.21.15.99:0/1618508556 >> 172.21.15.99:0/1618508556 conn(0x5574e4ebc0c0 msgr2=0x5574e4ebaff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:20:11.026 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.016+0000 7efc5877bb80 1 -- 172.21.15.99:0/1618508556 shutdown_connections 2023-10-08T00:20:11.026 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.016+0000 7efc5877bb80 1 -- 172.21.15.99:0/1618508556 wait complete. 2023-10-08T00:20:11.026 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.016+0000 7efc5877bb80 20 client.7938 trim_cache size 0 max 16384 2023-10-08T00:20:11.033 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.033+0000 7efc5877bb80 1 Processor -- start 2023-10-08T00:20:11.033 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.033+0000 7efc5877bb80 1 -- start start 2023-10-08T00:20:11.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.033+0000 7efc5877bb80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5574e4eb85f0 0x5574e4f14440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:11.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.033+0000 7efc5877bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574e4ebab80 0x5574e4f14980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:11.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.033+0000 7efc5877bb80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574e4f05480 0x5574e4f13590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:11.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.033+0000 7efc5877bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574e4ebdf20 con 0x5574e4ebab80 2023-10-08T00:20:11.036 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.033+0000 7efc5877bb80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574e4eb9d40 con 0x5574e4f05480 2023-10-08T00:20:11.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.033+0000 7efc5877bb80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x5574e4f15130 con 0x5574e4eb85f0 2023-10-08T00:20:11.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.034+0000 7efc4ca1e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574e4ebab80 0x5574e4f14980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:11.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.034+0000 7efc4ca1e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574e4ebab80 0x5574e4f14980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:32798/0 (socket says 172.21.15.99:32798) 2023-10-08T00:20:11.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.034+0000 7efc4ca1e700 1 -- 172.21.15.99:0/3527372305 learned_addr learned my addr 172.21.15.99:0/3527372305 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:20:11.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.034+0000 7efc4da20700 1 --2- 172.21.15.99:0/3527372305 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574e4f05480 0x5574e4f13590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:11.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.034+0000 7efc4d21f700 1 --2- 172.21.15.99:0/3527372305 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5574e4eb85f0 0x5574e4f14440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:11.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.034+0000 7efc4ca1e700 1 -- 172.21.15.99:0/3527372305 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5574e4eb85f0 msgr2=0x5574e4f14440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:11.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.034+0000 7efc4ca1e700 1 --2- 172.21.15.99:0/3527372305 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5574e4eb85f0 0x5574e4f14440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:11.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.034+0000 7efc4ca1e700 1 -- 172.21.15.99:0/3527372305 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574e4f05480 msgr2=0x5574e4f13590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:11.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.034+0000 7efc4ca1e700 1 --2- 172.21.15.99:0/3527372305 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574e4f05480 0x5574e4f13590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:11.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.034+0000 7efc4ca1e700 1 -- 172.21.15.99:0/3527372305 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574e4dd9780 con 0x5574e4ebab80 2023-10-08T00:20:11.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.034+0000 7efc4ca1e700 1 --2- 172.21.15.99:0/3527372305 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574e4ebab80 0x5574e4f14980 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7efc38055450 tx=0x7efc3806a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e55dee595e22f32 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:11.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.034+0000 7efc32ffd700 1 -- 172.21.15.99:0/3527372305 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc380554c0 con 0x5574e4ebab80 2023-10-08T00:20:11.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.034+0000 7efc32ffd700 1 -- 172.21.15.99:0/3527372305 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc38051800 con 0x5574e4ebab80 2023-10-08T00:20:11.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.034+0000 7efc32ffd700 1 -- 172.21.15.99:0/3527372305 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc38058710 con 0x5574e4ebab80 2023-10-08T00:20:11.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.035+0000 7efc5877bb80 1 -- 172.21.15.99:0/3527372305 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574e4ebab80 msgr2=0x5574e4f14980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:11.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.035+0000 7efc5877bb80 1 --2- 172.21.15.99:0/3527372305 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574e4ebab80 0x5574e4f14980 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7efc38055450 tx=0x7efc3806a000 comp rx=0 tx=0).stop 2023-10-08T00:20:11.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.035+0000 7efc5877bb80 1 -- 172.21.15.99:0/3527372305 shutdown_connections 2023-10-08T00:20:11.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.035+0000 7efc5877bb80 1 --2- 172.21.15.99:0/3527372305 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574e4f05480 0x5574e4f13590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:11.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.035+0000 7efc5877bb80 1 --2- 172.21.15.99:0/3527372305 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5574e4eb85f0 0x5574e4f14440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:11.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.035+0000 7efc5877bb80 1 --2- 172.21.15.99:0/3527372305 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574e4ebab80 0x5574e4f14980 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:11.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.035+0000 7efc5877bb80 1 -- 172.21.15.99:0/3527372305 >> 172.21.15.99:0/3527372305 conn(0x5574e4ebc0c0 msgr2=0x5574e4f04b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:20:11.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.035+0000 7efc5877bb80 1 -- 172.21.15.99:0/3527372305 shutdown_connections 2023-10-08T00:20:11.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.035+0000 7efc5877bb80 1 -- 172.21.15.99:0/3527372305 wait complete. 2023-10-08T00:20:11.039 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.035+0000 7efc5877bb80 1 -- start start 2023-10-08T00:20:11.039 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.035+0000 7efc5877bb80 1 Processor -- start 2023-10-08T00:20:11.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.036+0000 7efc5877bb80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574e4eb85f0 0x5574e4ee03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:11.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.036+0000 7efc5877bb80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5574e4ebab80 0x5574e4f664f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:11.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.036+0000 7efc5877bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574e4f05480 0x5574e4f14440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:11.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.036+0000 7efc5877bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574e4f68250 con 0x5574e4f05480 2023-10-08T00:20:11.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.036+0000 7efc5877bb80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574e4f00d40 con 0x5574e4eb85f0 2023-10-08T00:20:11.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.036+0000 7efc5877bb80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x5574e4f66a30 con 0x5574e4ebab80 2023-10-08T00:20:11.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.036+0000 7efc5877bb80 10 client.0 _ll_register_callbacks cb 0x7efc38055000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T00:20:11.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.036+0000 7efc4da20700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574e4f05480 0x5574e4f14440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:11.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.036+0000 7efc4da20700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574e4f05480 0x5574e4f14440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:32808/0 (socket says 172.21.15.99:32808) 2023-10-08T00:20:11.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.036+0000 7efc4da20700 1 -- 172.21.15.99:0/1859374519 learned_addr learned my addr 172.21.15.99:0/1859374519 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:20:11.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.036+0000 7efc4d21f700 1 --2- 172.21.15.99:0/1859374519 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574e4eb85f0 0x5574e4ee03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:11.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.036+0000 7efc4ca1e700 1 --2- 172.21.15.99:0/1859374519 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5574e4ebab80 0x5574e4f664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:11.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.036+0000 7efc4da20700 1 -- 172.21.15.99:0/1859374519 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5574e4ebab80 msgr2=0x5574e4f664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:11.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.036+0000 7efc4da20700 1 --2- 172.21.15.99:0/1859374519 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5574e4ebab80 0x5574e4f664f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:11.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.036+0000 7efc4da20700 1 -- 172.21.15.99:0/1859374519 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574e4eb85f0 msgr2=0x5574e4ee03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:11.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.036+0000 7efc4da20700 1 --2- 172.21.15.99:0/1859374519 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574e4eb85f0 0x5574e4ee03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:11.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.036+0000 7efc4da20700 1 -- 172.21.15.99:0/1859374519 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5574e4c47fe0 con 0x5574e4f05480 2023-10-08T00:20:11.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.037+0000 7efc4da20700 1 --2- 172.21.15.99:0/1859374519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574e4f05480 0x5574e4f14440 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7efc480c7e60 tx=0x7efc40009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62374c91b02c438c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:11.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.037+0000 7efc277fe700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T00:20:11.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.037+0000 7efc277fe700 1 -- 172.21.15.99:0/1859374519 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc400153b0 con 0x5574e4f05480 2023-10-08T00:20:11.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.037+0000 7efc277fe700 1 -- 172.21.15.99:0/1859374519 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(168..168 src has 1..168) v4 ==== 5705+0+0 (secure 0 0 0) 0x7efc40014550 con 0x5574e4f05480 2023-10-08T00:20:11.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.037+0000 7efc5877bb80 1 -- 172.21.15.99:0/1859374519 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efc44023040 con 0x5574e4f05480 2023-10-08T00:20:11.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.037+0000 7efc277fe700 1 -- 172.21.15.99:0/1859374519 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc40014920 con 0x5574e4f05480 2023-10-08T00:20:11.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.037+0000 7efc5877bb80 10 client.7944 fetch_fsmap learned FSMap version 77 2023-10-08T00:20:11.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.037+0000 7efc5877bb80 1 -- 172.21.15.99:0/1859374519 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) v3 -- 0x5574e4dc0470 con 0x5574e4f05480 2023-10-08T00:20:11.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.037+0000 7efc277fe700 1 -- 172.21.15.99:0/1859374519 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc44023040 con 0x5574e4f05480 2023-10-08T00:20:11.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.037+0000 7efc277fe700 1 -- 172.21.15.99:0/1859374519 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 77) v1 ==== 3100+0+0 (secure 0 0 0) 0x7efc40014a80 con 0x5574e4f05480 2023-10-08T00:20:11.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.037+0000 7efc5877bb80 10 client.7944 fetch_fsmap finished waiting for FSMap version 77 2023-10-08T00:20:11.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.037+0000 7efc5877bb80 10 client.7944 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-08T00:20:11.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.037+0000 7efc5877bb80 20 client.7944 populate_metadata read hostname 'smithi099' 2023-10-08T00:20:11.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.037+0000 7efc5877bb80 1 --2- 172.21.15.99:0/1859374519 >> [v2:172.21.15.99:6832/3880334948,v1:172.21.15.99:6834/3880334948] conn(0x5574e4f12b40 0x5574e4f12f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:11.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.037+0000 7efc5877bb80 4 client.7944 mds_command: new command op to 7791 tid=0[{"prefix": "status"}] 2023-10-08T00:20:11.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.037+0000 7efc5877bb80 1 -- 172.21.15.99:0/1859374519 --> [v2:172.21.15.99:6832/3880334948,v1:172.21.15.99:6834/3880334948] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5574e4dade00 con 0x5574e4f12b40 2023-10-08T00:20:11.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.037+0000 7efc4d21f700 1 --2- 172.21.15.99:0/1859374519 >> [v2:172.21.15.99:6832/3880334948,v1:172.21.15.99:6834/3880334948] conn(0x5574e4f12b40 0x5574e4f12f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:11.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.038+0000 7efc277fe700 10 client.7944 ms_handle_connect on v2:172.21.15.99:6832/3880334948 2023-10-08T00:20:11.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.038+0000 7efc4d21f700 1 --2- 172.21.15.99:0/1859374519 >> [v2:172.21.15.99:6832/3880334948,v1:172.21.15.99:6834/3880334948] conn(0x5574e4f12b40 0x5574e4f12f10 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e00a4e4f8911096d server_cookie=11f0beca61391a17 in_seq=0 out_seq=0 2023-10-08T00:20:11.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.038+0000 7efc277fe700 1 -- 172.21.15.99:0/1859374519 <== mds.0 v2:172.21.15.99:6832/3880334948 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5574e4dade00 con 0x5574e4f12b40 2023-10-08T00:20:11.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.038+0000 7efc277fe700 10 client.7944 handle_command_reply: tid=0 2023-10-08T00:20:11.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.038+0000 7efc5877bb80 1 client.7944 shutdown 2023-10-08T00:20:11.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.038+0000 7efc5877bb80 1 -- 172.21.15.99:0/1859374519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574e4f05480 msgr2=0x5574e4f14440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:11.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.038+0000 7efc5877bb80 1 --2- 172.21.15.99:0/1859374519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574e4f05480 0x5574e4f14440 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7efc480c7e60 tx=0x7efc40009040 comp rx=0 tx=0).stop 2023-10-08T00:20:11.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.038+0000 7efc5877bb80 1 -- 172.21.15.99:0/1859374519 shutdown_connections 2023-10-08T00:20:11.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.038+0000 7efc5877bb80 1 --2- 172.21.15.99:0/1859374519 >> [v2:172.21.15.99:6832/3880334948,v1:172.21.15.99:6834/3880334948] conn(0x5574e4f12b40 0x5574e4f12f10 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:11.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.038+0000 7efc5877bb80 1 --2- 172.21.15.99:0/1859374519 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574e4eb85f0 0x5574e4ee03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:11.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.038+0000 7efc277fe700 0 client.7944 ms_handle_reset on v2:172.21.15.99:6832/3880334948 2023-10-08T00:20:11.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.038+0000 7efc5877bb80 1 --2- 172.21.15.99:0/1859374519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574e4f05480 0x5574e4f14440 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:11.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.038+0000 7efc5877bb80 1 --2- 172.21.15.99:0/1859374519 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5574e4ebab80 0x5574e4f664f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:11.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.038+0000 7efc5877bb80 1 -- 172.21.15.99:0/1859374519 >> 172.21.15.99:0/1859374519 conn(0x5574e4ebc0c0 msgr2=0x5574e4ebc490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:20:11.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.038+0000 7efc5877bb80 1 -- 172.21.15.99:0/1859374519 shutdown_connections 2023-10-08T00:20:11.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.038+0000 7efc5877bb80 1 -- 172.21.15.99:0/1859374519 wait complete. 2023-10-08T00:20:11.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.038+0000 7efc5877bb80 20 client.7944 trim_cache size 0 max 16384 2023-10-08T00:20:11.047 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:20:11.047 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", 2023-10-08T00:20:11.047 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 0, 2023-10-08T00:20:11.047 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 7791, 2023-10-08T00:20:11.047 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2023-10-08T00:20:11.047 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2023-10-08T00:20:11.047 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2023-10-08T00:20:11.047 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 5.8032459210000003, 2023-10-08T00:20:11.048 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 77, 2023-10-08T00:20:11.048 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 168, 2023-10-08T00:20:11.048 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 168, 2023-10-08T00:20:11.048 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 10.915914158 2023-10-08T00:20:11.048 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:20:11.049 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.038+0000 7efc4e221700 1 -- 172.21.15.99:0/3197029458 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efc2c066f30 msgr2=0x7efc2c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:11.049 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.039+0000 7efc4e221700 1 --2- 172.21.15.99:0/3197029458 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efc2c066f30 0x7efc2c0693e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7efc440091e0 tx=0x7efc4401a000 comp rx=0 tx=0).stop 2023-10-08T00:20:11.049 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.039+0000 7efc4e221700 1 -- 172.21.15.99:0/3197029458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc480c5af0 msgr2=0x7efc480c5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:11.049 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.039+0000 7efc4e221700 1 --2- 172.21.15.99:0/3197029458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc480c5af0 0x7efc480c5f60 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7efc38002980 tx=0x7efc38007d30 comp rx=0 tx=0).stop 2023-10-08T00:20:11.049 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.039+0000 7efc4e221700 1 -- 172.21.15.99:0/3197029458 shutdown_connections 2023-10-08T00:20:11.049 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.039+0000 7efc4e221700 1 --2- 172.21.15.99:0/3197029458 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efc2c066f30 0x7efc2c0693e0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:11.049 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.039+0000 7efc4e221700 1 --2- 172.21.15.99:0/3197029458 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efc34004ba0 0x7efc48168cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:11.049 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.039+0000 7efc4e221700 1 --2- 172.21.15.99:0/3197029458 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc4815a1c0 0x7efc4815a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:11.049 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.039+0000 7efc4e221700 1 --2- 172.21.15.99:0/3197029458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc480c5af0 0x7efc480c5f60 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:11.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.039+0000 7efc4e221700 1 -- 172.21.15.99:0/3197029458 >> 172.21.15.99:0/3197029458 conn(0x7efc480cc760 msgr2=0x7efc4806cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:20:11.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.039+0000 7efc4e221700 1 -- 172.21.15.99:0/3197029458 shutdown_connections 2023-10-08T00:20:11.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.039+0000 7efc4e221700 1 -- 172.21.15.99:0/3197029458 wait complete. 2023-10-08T00:20:11.051 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T00:20:11.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.165+0000 7fa037c8b700 1 Processor -- start 2023-10-08T00:20:11.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.166+0000 7fa037c8b700 1 -- start start 2023-10-08T00:20:11.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.166+0000 7fa037c8b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0301577d0 0x7fa0300cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:11.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.167+0000 7fa037c8b700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa03015c360 con 0x7fa030157ba0 2023-10-08T00:20:11.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.167+0000 7fa037c8b700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa03015b790 con 0x7fa03006ac30 2023-10-08T00:20:11.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.167+0000 7fa037c8b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa03015ba30 con 0x7fa0301577d0 2023-10-08T00:20:11.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.167+0000 7fa035226700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fa030157ba0 0x7fa03006a520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56132/0 (socket says 172.21.15.99:56132) 2023-10-08T00:20:11.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.167+0000 7fa035226700 1 -- 172.21.15.99:0/2863795433 learned_addr learned my addr 172.21.15.99:0/2863795433 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:20:11.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.167+0000 7fa034a25700 1 -- 172.21.15.99:0/2863795433 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 737825429 0 0) 0x7fa03015c360 con 0x7fa030157ba0 2023-10-08T00:20:11.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.167+0000 7fa034a25700 1 -- 172.21.15.99:0/2863795433 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa01c003570 con 0x7fa030157ba0 2023-10-08T00:20:11.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.167+0000 7fa035a27700 1 --2- 172.21.15.99:0/2863795433 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0301577d0 0x7fa0300cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:11.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.167+0000 7fa034a25700 1 -- 172.21.15.99:0/2863795433 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 871306338 0 0) 0x7fa01c003570 con 0x7fa030157ba0 2023-10-08T00:20:11.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.167+0000 7fa034a25700 1 -- 172.21.15.99:0/2863795433 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0301577d0 msgr2=0x7fa0300cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:11.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.167+0000 7fa034a25700 1 --2- 172.21.15.99:0/2863795433 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0301577d0 0x7fa0300cd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:11.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.167+0000 7fa034a25700 1 -- 172.21.15.99:0/2863795433 >> v1:172.21.15.191:6789/0 conn(0x7fa03006ac30 legacy=0x7fa03015c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:11.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.167+0000 7fa034a25700 1 -- 172.21.15.99:0/2863795433 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa03015bd10 con 0x7fa030157ba0 2023-10-08T00:20:11.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.167+0000 7fa034a25700 1 -- 172.21.15.99:0/2863795433 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7fa020001470 con 0x7fa030157ba0 2023-10-08T00:20:11.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.167+0000 7fa034a25700 1 -- 172.21.15.99:0/2863795433 >> v1:172.21.15.99:6789/0 conn(0x7fa030157ba0 legacy=0x7fa03006a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:11.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.167+0000 7fa034a25700 1 --2- 172.21.15.99:0/2863795433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa01c003a90 0x7fa01c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:11.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.167+0000 7fa034a25700 1 --2- 172.21.15.99:0/2863795433 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa01c004810 0x7fa01c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:11.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.167+0000 7fa034a25700 1 --2- 172.21.15.99:0/2863795433 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0301577d0 0x7fa01c0051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:11.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.167+0000 7fa034a25700 1 -- 172.21.15.99:0/2863795433 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa03015ba30 con 0x7fa01c003a90 2023-10-08T00:20:11.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.167+0000 7fa034a25700 1 -- 172.21.15.99:0/2863795433 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c00bb60 con 0x7fa01c004810 2023-10-08T00:20:11.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.167+0000 7fa034a25700 1 -- 172.21.15.99:0/2863795433 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c00bd20 con 0x7fa0301577d0 2023-10-08T00:20:11.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.167+0000 7fa035a27700 1 --2- 172.21.15.99:0/2863795433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa01c003a90 0x7fa01c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:11.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.167+0000 7fa035226700 1 --2- 172.21.15.99:0/2863795433 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa01c004810 0x7fa01c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:11.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.167+0000 7fa036228700 1 --2- 172.21.15.99:0/2863795433 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0301577d0 0x7fa01c0051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:11.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.168+0000 7fa035a27700 1 -- 172.21.15.99:0/2863795433 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0301577d0 msgr2=0x7fa01c0051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:11.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.168+0000 7fa035a27700 1 --2- 172.21.15.99:0/2863795433 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0301577d0 0x7fa01c0051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:11.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.168+0000 7fa035a27700 1 -- 172.21.15.99:0/2863795433 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa01c004810 msgr2=0x7fa01c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:11.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.168+0000 7fa035a27700 1 --2- 172.21.15.99:0/2863795433 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa01c004810 0x7fa01c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:11.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.168+0000 7fa035a27700 1 -- 172.21.15.99:0/2863795433 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa01c003570 con 0x7fa01c003a90 2023-10-08T00:20:11.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.168+0000 7fa035a27700 1 --2- 172.21.15.99:0/2863795433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa01c003a90 0x7fa01c003ea0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fa02c009c00 tx=0x7fa02c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d63908081b9bd68a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:11.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.168+0000 7fa034a25700 1 -- 172.21.15.99:0/2863795433 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02c00e710 con 0x7fa01c003a90 2023-10-08T00:20:11.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.168+0000 7fa034a25700 1 -- 172.21.15.99:0/2863795433 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa02c00e870 con 0x7fa01c003a90 2023-10-08T00:20:11.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.168+0000 7fa034a25700 1 -- 172.21.15.99:0/2863795433 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02c00ec30 con 0x7fa01c003a90 2023-10-08T00:20:11.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.168+0000 7fa037c8b700 1 -- 172.21.15.99:0/2863795433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa01c003a90 msgr2=0x7fa01c003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:11.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.168+0000 7fa037c8b700 1 --2- 172.21.15.99:0/2863795433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa01c003a90 0x7fa01c003ea0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fa02c009c00 tx=0x7fa02c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:20:11.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.168+0000 7fa036228700 1 -- 172.21.15.99:0/2863795433 reap_dead start 2023-10-08T00:20:11.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.168+0000 7fa037c8b700 1 -- 172.21.15.99:0/2863795433 shutdown_connections 2023-10-08T00:20:11.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.168+0000 7fa037c8b700 1 -- 172.21.15.99:0/2863795433 >> 172.21.15.99:0/2863795433 conn(0x7fa030068ce0 msgr2=0x7fa0300690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:20:11.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.169+0000 7fa037c8b700 1 -- 172.21.15.99:0/2863795433 shutdown_connections 2023-10-08T00:20:11.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.169+0000 7fa037c8b700 1 -- 172.21.15.99:0/2863795433 wait complete. 2023-10-08T00:20:11.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.170+0000 7fa037c8b700 1 Processor -- start 2023-10-08T00:20:11.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.170+0000 7fa037c8b700 1 -- start start 2023-10-08T00:20:11.176 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.170+0000 7fa037c8b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa01c003a90 0x7fa030151b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:11.176 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.170+0000 7fa037c8b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa030154840 0x7fa0300b2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:11.176 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.170+0000 7fa037c8b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa03014e2c0 0x7fa03014e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:11.176 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.170+0000 7fa037c8b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa030150fe0 con 0x7fa01c003a90 2023-10-08T00:20:11.177 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.170+0000 7fa037c8b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa03014fef0 con 0x7fa030154840 2023-10-08T00:20:11.177 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.170+0000 7fa037c8b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0301501d0 con 0x7fa03014e2c0 2023-10-08T00:20:11.177 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.170+0000 7fa035a27700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa01c003a90 0x7fa030151b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:11.177 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.170+0000 7fa035a27700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa01c003a90 0x7fa030151b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:32816/0 (socket says 172.21.15.99:32816) 2023-10-08T00:20:11.177 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.170+0000 7fa035a27700 1 -- 172.21.15.99:0/1638410921 learned_addr learned my addr 172.21.15.99:0/1638410921 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:20:11.177 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.171+0000 7fa036228700 1 --2- 172.21.15.99:0/1638410921 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa03014e2c0 0x7fa03014e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:11.177 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.171+0000 7fa035226700 1 --2- 172.21.15.99:0/1638410921 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa030154840 0x7fa0300b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:11.177 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.171+0000 7fa035a27700 1 -- 172.21.15.99:0/1638410921 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa03014e2c0 msgr2=0x7fa03014e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:11.177 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.171+0000 7fa035a27700 1 --2- 172.21.15.99:0/1638410921 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa03014e2c0 0x7fa03014e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:11.177 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.171+0000 7fa035a27700 1 -- 172.21.15.99:0/1638410921 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa030154840 msgr2=0x7fa0300b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:11.177 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.171+0000 7fa035a27700 1 --2- 172.21.15.99:0/1638410921 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa030154840 0x7fa0300b2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:11.177 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.171+0000 7fa035a27700 1 -- 172.21.15.99:0/1638410921 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0301504b0 con 0x7fa01c003a90 2023-10-08T00:20:11.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.171+0000 7fa035a27700 1 --2- 172.21.15.99:0/1638410921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa01c003a90 0x7fa030151b10 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fa01c00bb30 tx=0x7fa02c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:11.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.171+0000 7fa025ffb700 1 -- 172.21.15.99:0/1638410921 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02c022070 con 0x7fa01c003a90 2023-10-08T00:20:11.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.171+0000 7fa025ffb700 1 -- 172.21.15.99:0/1638410921 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa02c00ebf0 con 0x7fa01c003a90 2023-10-08T00:20:11.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.171+0000 7fa025ffb700 1 -- 172.21.15.99:0/1638410921 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02c01c6a0 con 0x7fa01c003a90 2023-10-08T00:20:11.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.171+0000 7fa037c8b700 1 -- 172.21.15.99:0/1638410921 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa030150c70 con 0x7fa01c003a90 2023-10-08T00:20:11.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.171+0000 7fa037c8b700 1 -- 172.21.15.99:0/1638410921 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0300b2ea0 con 0x7fa01c003a90 2023-10-08T00:20:11.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.172+0000 7fa037c8b700 1 -- 172.21.15.99:0/1638410921 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0300b30c0 con 0x7fa01c003a90 2023-10-08T00:20:11.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.173+0000 7fa025ffb700 1 -- 172.21.15.99:0/1638410921 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa02c01ca90 con 0x7fa01c003a90 2023-10-08T00:20:11.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.173+0000 7fa025ffb700 1 --2- 172.21.15.99:0/1638410921 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa014066f30 0x7fa0140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:11.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.173+0000 7fa025ffb700 1 -- 172.21.15.99:0/1638410921 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa02c085c30 con 0x7fa01c003a90 2023-10-08T00:20:11.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.173+0000 7fa035226700 1 --2- 172.21.15.99:0/1638410921 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa014066f30 0x7fa0140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:11.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.174+0000 7fa035226700 1 --2- 172.21.15.99:0/1638410921 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa014066f30 0x7fa0140693e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fa0200015a0 tx=0x7fa020009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:11.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.177+0000 7fa025ffb700 1 -- 172.21.15.99:0/1638410921 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa02c052ea0 con 0x7fa01c003a90 2023-10-08T00:20:11.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.403+0000 7fa037c8b700 1 -- 172.21.15.99:0/1638410921 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa030150ad0 con 0x7fa01c003a90 2023-10-08T00:20:11.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.403+0000 7fa025ffb700 1 -- 172.21.15.99:0/1638410921 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v77) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa030150ad0 con 0x7fa01c003a90 2023-10-08T00:20:11.404 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T00:20:11.406 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.405+0000 7fa037c8b700 1 -- 172.21.15.99:0/1638410921 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa014066f30 msgr2=0x7fa0140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:11.407 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.405+0000 7fa037c8b700 1 --2- 172.21.15.99:0/1638410921 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa014066f30 0x7fa0140693e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fa0200015a0 tx=0x7fa020009040 comp rx=0 tx=0).stop 2023-10-08T00:20:11.407 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.405+0000 7fa037c8b700 1 -- 172.21.15.99:0/1638410921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa01c003a90 msgr2=0x7fa030151b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:11.407 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.405+0000 7fa037c8b700 1 --2- 172.21.15.99:0/1638410921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa01c003a90 0x7fa030151b10 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fa01c00bb30 tx=0x7fa02c01a000 comp rx=0 tx=0).stop 2023-10-08T00:20:11.407 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.405+0000 7fa037c8b700 1 -- 172.21.15.99:0/1638410921 shutdown_connections 2023-10-08T00:20:11.407 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.405+0000 7fa037c8b700 1 --2- 172.21.15.99:0/1638410921 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa014066f30 0x7fa0140693e0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:11.407 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.405+0000 7fa037c8b700 1 --2- 172.21.15.99:0/1638410921 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa03014e2c0 0x7fa03014e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:11.407 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.405+0000 7fa037c8b700 1 --2- 172.21.15.99:0/1638410921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa01c003a90 0x7fa030151b10 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:11.407 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.405+0000 7fa037c8b700 1 --2- 172.21.15.99:0/1638410921 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa030154840 0x7fa0300b2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:11.407 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.406+0000 7fa037c8b700 1 -- 172.21.15.99:0/1638410921 >> 172.21.15.99:0/1638410921 conn(0x7fa030068ce0 msgr2=0x7fa0300cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:20:11.407 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.406+0000 7fa037c8b700 1 -- 172.21.15.99:0/1638410921 shutdown_connections 2023-10-08T00:20:11.407 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:11.406+0000 7fa037c8b700 1 -- 172.21.15.99:0/1638410921 wait complete. 2023-10-08T00:20:11.418 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T00:20:11.418 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T00:20:11.418 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi099.front.sepia.ceph.com 2023-10-08T00:20:11.418 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T00:20:11.418 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T00:20:11.418 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T00:20:11.418 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T00:20:11.418 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T00:20:11.418 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T00:20:11.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:20:11.419 DEBUG:teuthology.orchestra.run.smithi099:> ip addr 2023-10-08T00:20:11.433 INFO:teuthology.orchestra.run.smithi099.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T00:20:11.434 INFO:teuthology.orchestra.run.smithi099.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T00:20:11.434 INFO:teuthology.orchestra.run.smithi099.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T00:20:11.434 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:20:11.434 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 ::1/128 scope host 2023-10-08T00:20:11.434 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:20:11.434 INFO:teuthology.orchestra.run.smithi099.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:20:11.434 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:10 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:20:11.434 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f0 2023-10-08T00:20:11.434 INFO:teuthology.orchestra.run.smithi099.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:20:11.434 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:11 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:20:11.435 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f1 2023-10-08T00:20:11.435 INFO:teuthology.orchestra.run.smithi099.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:20:11.435 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:02 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:20:11.435 INFO:teuthology.orchestra.run.smithi099.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T00:20:11.435 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:03 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:20:11.435 INFO:teuthology.orchestra.run.smithi099.stdout: inet 172.21.15.99/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T00:20:11.435 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft 41978sec preferred_lft 41978sec 2023-10-08T00:20:11.435 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::ec4:7aff:fe8f:d803/64 scope link 2023-10-08T00:20:11.435 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:20:11.435 INFO:teuthology.orchestra.run.smithi099.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T00:20:11.435 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 22:2c:1c:f6:42:e1 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:20:11.435 INFO:teuthology.orchestra.run.smithi099.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T00:20:11.435 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:20:11.436 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::600d:26ff:fede:5eec/64 scope link 2023-10-08T00:20:11.436 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:20:11.436 INFO:teuthology.orchestra.run.smithi099.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T00:20:11.436 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether ae:2f:b5:94:69:ba brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T00:20:11.436 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::ac2f:b5ff:fe94:69ba/64 scope link 2023-10-08T00:20:11.436 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:20:11.436 INFO:teuthology.orchestra.run.smithi099.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T00:20:11.436 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 22:2c:1c:f6:42:e1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T00:20:11.436 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::202c:1cff:fef6:42e1/64 scope link 2023-10-08T00:20:11.436 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:20:11.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:20:11.437 DEBUG:teuthology.orchestra.run.smithi099:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T00:20:11.497 INFO:teuthology.orchestra.run.smithi099.stdout:1 2023-10-08T00:20:11.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:20:11.498 DEBUG:teuthology.orchestra.run.smithi099:> route 2023-10-08T00:20:11.554 INFO:teuthology.orchestra.run.smithi099.stdout:Kernel IP routing table 2023-10-08T00:20:11.557 INFO:teuthology.orchestra.run.smithi099.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T00:20:11.557 INFO:teuthology.orchestra.run.smithi099.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T00:20:11.557 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T00:20:11.557 INFO:teuthology.orchestra.run.smithi099.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T00:20:11.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:20:11.558 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2023-10-08T00:20:11.558 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2023-10-08T00:20:11.558 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T00:20:11.558 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T00:20:11.558 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T00:20:11.558 DEBUG:teuthology.orchestra.run.smithi099:> ') 2023-10-08T00:20:11.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:20:11.654 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2023-10-08T00:20:11.669 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T00:20:11.669 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T00:20:11.669 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T00:20:11.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:20:11.670 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set brx.0 up 2023-10-08T00:20:11.734 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:20:11.734 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T00:20:11.789 INFO:teuthology.orchestra.run.smithi099.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T00:20:11.789 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:20:11.789 DEBUG:teuthology.orchestra.run.smithi099:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T00:20:11.843 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-08T00:20:11.843 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T00:20:11.843 DEBUG:teuthology.orchestra.run.smithi099:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:20:11.923 INFO:teuthology.orchestra.run.smithi099.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:20:11.923 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:20:11.942 INFO:teuthology.orchestra.run.smithi099.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:20:11.942 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:20:11.942 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: resolved to: "172.21.15.99:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T00:20:11.942 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: trying mount with new device syntax: 0@06962cdb-9a6f-417f-82fd-9e4e3dedbccf.cephfs=/ 2023-10-08T00:20:11.942 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.99:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T00:20:11.942 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T00:20:11.942 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T00:20:11.943 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T00:20:11.964 INFO:teuthology.orchestra.run.smithi099.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:20:11.964 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:20:11.964 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: resolved to: "172.21.15.99:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T00:20:11.964 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: trying mount with new device syntax: 0@06962cdb-9a6f-417f-82fd-9e4e3dedbccf.cephfs=/ 2023-10-08T00:20:11.964 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.99:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T00:20:11.964 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T00:20:11.964 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T00:20:11.964 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T00:20:11.964 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: resolved to: "172.21.15.99:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T00:20:11.965 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.99:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T00:20:11.965 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=06962cdb-9a6f-417f-82fd-9e4e3dedbccf" will pass to kernel 2023-10-08T00:20:11.966 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T00:20:11.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:20:11.966 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T00:20:12.031 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T00:20:12.186 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.185+0000 7f457ee94700 1 Processor -- start 2023-10-08T00:20:12.186 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.185+0000 7f457ee94700 1 -- start start 2023-10-08T00:20:12.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.186+0000 7f457ee94700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f45780c75c0 0x7f45780c8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:12.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.186+0000 7f457ee94700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f45780abeb0 0x7f45780c8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:12.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.186+0000 7f457ee94700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45780c9cb0 con 0x7f45780abae0 2023-10-08T00:20:12.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.186+0000 7f457ee94700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45780c8e80 con 0x7f45780c75c0 2023-10-08T00:20:12.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.186+0000 7f457ee94700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45780c90a0 con 0x7f45780abeb0 2023-10-08T00:20:12.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.186+0000 7f4577fff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f45780c75c0 0x7f45780c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:12.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.186+0000 7f4577fff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f45780c75c0 0x7f45780c8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:32822/0 (socket says 172.21.15.99:32822) 2023-10-08T00:20:12.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.186+0000 7f4577fff700 1 -- 172.21.15.99:0/3702063683 learned_addr learned my addr 172.21.15.99:0/3702063683 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:20:12.188 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.186+0000 7f457d431700 1 --2- 172.21.15.99:0/3702063683 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f45780abeb0 0x7f45780c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:12.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.186+0000 7f4577fff700 1 -- 172.21.15.99:0/3702063683 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f45780abeb0 msgr2=0x7f45780c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:12.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.186+0000 7f4577fff700 1 --2- 172.21.15.99:0/3702063683 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f45780abeb0 0x7f45780c8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:12.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.186+0000 7f4577fff700 1 -- 172.21.15.99:0/3702063683 >> v1:172.21.15.191:6789/0 conn(0x7f45780abae0 legacy=0x7f45780c6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:12.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.186+0000 7f4577fff700 1 -- 172.21.15.99:0/3702063683 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45780c9380 con 0x7f45780c75c0 2023-10-08T00:20:12.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.186+0000 7f4577fff700 1 --2- 172.21.15.99:0/3702063683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45780c75c0 0x7f45780c8400 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f45680099b0 tx=0x7f4568009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d0c6f544292d890 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:12.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.187+0000 7f45777fe700 1 -- 172.21.15.99:0/3702063683 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456800b860 con 0x7f45780c75c0 2023-10-08T00:20:12.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.187+0000 7f45777fe700 1 -- 172.21.15.99:0/3702063683 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f456800b9c0 con 0x7f45780c75c0 2023-10-08T00:20:12.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.187+0000 7f45777fe700 1 -- 172.21.15.99:0/3702063683 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456800bd60 con 0x7f45780c75c0 2023-10-08T00:20:12.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.187+0000 7f457ee94700 1 -- 172.21.15.99:0/3702063683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45780c75c0 msgr2=0x7f45780c8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:12.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.187+0000 7f457ee94700 1 --2- 172.21.15.99:0/3702063683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45780c75c0 0x7f45780c8400 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f45680099b0 tx=0x7f4568009250 comp rx=0 tx=0).stop 2023-10-08T00:20:12.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.187+0000 7f457ee94700 1 -- 172.21.15.99:0/3702063683 shutdown_connections 2023-10-08T00:20:12.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.187+0000 7f457ee94700 1 --2- 172.21.15.99:0/3702063683 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f45780abeb0 0x7f45780c8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:12.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.187+0000 7f457ee94700 1 --2- 172.21.15.99:0/3702063683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45780c75c0 0x7f45780c8400 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:12.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.187+0000 7f457ee94700 1 -- 172.21.15.99:0/3702063683 >> 172.21.15.99:0/3702063683 conn(0x7f4578067660 msgr2=0x7f4578067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:20:12.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.187+0000 7f457ee94700 1 -- 172.21.15.99:0/3702063683 shutdown_connections 2023-10-08T00:20:12.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.187+0000 7f457ee94700 1 -- 172.21.15.99:0/3702063683 wait complete. 2023-10-08T00:20:12.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.188+0000 7f457ee94700 1 Processor -- start 2023-10-08T00:20:12.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.188+0000 7f457ee94700 1 -- start start 2023-10-08T00:20:12.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.189+0000 7f457ee94700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f45780abae0 0x7f45780ae4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:12.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.189+0000 7f457ee94700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f45780abeb0 0x7f45781577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:12.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.189+0000 7f457ee94700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45780c75c0 0x7f45780aa880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:12.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.189+0000 7f457ee94700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4578157350 con 0x7f45780c75c0 2023-10-08T00:20:12.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.189+0000 7f457ee94700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4578157d20 con 0x7f45780abeb0 2023-10-08T00:20:12.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.189+0000 7f457ee94700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4578158a70 con 0x7f45780abae0 2023-10-08T00:20:12.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.189+0000 7f457d431700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45780c75c0 0x7f45780aa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:12.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.189+0000 7f457d431700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45780c75c0 0x7f45780aa880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:32828/0 (socket says 172.21.15.99:32828) 2023-10-08T00:20:12.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.189+0000 7f457d431700 1 -- 172.21.15.99:0/3105507080 learned_addr learned my addr 172.21.15.99:0/3105507080 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:20:12.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.189+0000 7f4577fff700 1 --2- 172.21.15.99:0/3105507080 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f45780abeb0 0x7f45781577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:12.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.189+0000 7f457cc30700 1 --2- 172.21.15.99:0/3105507080 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f45780abae0 0x7f45780ae4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:12.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.190+0000 7f457d431700 1 -- 172.21.15.99:0/3105507080 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f45780abae0 msgr2=0x7f45780ae4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:12.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.190+0000 7f457d431700 1 --2- 172.21.15.99:0/3105507080 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f45780abae0 0x7f45780ae4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:12.193 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.190+0000 7f457d431700 1 -- 172.21.15.99:0/3105507080 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f45780abeb0 msgr2=0x7f45781577e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:12.194 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.190+0000 7f457d431700 1 --2- 172.21.15.99:0/3105507080 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f45780abeb0 0x7f45781577e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:12.194 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.190+0000 7f457d431700 1 -- 172.21.15.99:0/3105507080 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45680094f0 con 0x7f45780c75c0 2023-10-08T00:20:12.194 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.190+0000 7f457cc30700 1 --2- 172.21.15.99:0/3105507080 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f45780abae0 0x7f45780ae4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:20:12.194 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.190+0000 7f457d431700 1 --2- 172.21.15.99:0/3105507080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45780c75c0 0x7f45780aa880 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f456000d890 tx=0x7f4560008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:12.194 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.190+0000 7f4574ff9700 1 -- 172.21.15.99:0/3105507080 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45600125f0 con 0x7f45780c75c0 2023-10-08T00:20:12.194 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.190+0000 7f4574ff9700 1 -- 172.21.15.99:0/3105507080 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4560012770 con 0x7f45780c75c0 2023-10-08T00:20:12.194 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.190+0000 7f457ee94700 1 -- 172.21.15.99:0/3105507080 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4578158d50 con 0x7f45780c75c0 2023-10-08T00:20:12.194 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.190+0000 7f4574ff9700 1 -- 172.21.15.99:0/3105507080 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4560012b30 con 0x7f45780c75c0 2023-10-08T00:20:12.194 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.190+0000 7f457ee94700 1 -- 172.21.15.99:0/3105507080 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4578156fe0 con 0x7f45780c75c0 2023-10-08T00:20:12.194 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.191+0000 7f457ee94700 1 -- 172.21.15.99:0/3105507080 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4578158f40 con 0x7f45780c75c0 2023-10-08T00:20:12.194 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.191+0000 7f4574ff9700 1 -- 172.21.15.99:0/3105507080 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f456001e460 con 0x7f45780c75c0 2023-10-08T00:20:12.194 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.191+0000 7f4574ff9700 1 --2- 172.21.15.99:0/3105507080 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4558066f30 0x7f45580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:12.195 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.192+0000 7f4574ff9700 1 -- 172.21.15.99:0/3105507080 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4560084c90 con 0x7f45780c75c0 2023-10-08T00:20:12.195 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.192+0000 7f457cc30700 1 --2- 172.21.15.99:0/3105507080 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4558066f30 0x7f45580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:12.195 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.192+0000 7f457cc30700 1 --2- 172.21.15.99:0/3105507080 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4558066f30 0x7f45580693e0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f4570007fd0 tx=0x7f4570009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:12.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.196+0000 7f4574ff9700 1 -- 172.21.15.99:0/3105507080 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4578158f40 con 0x7f45780c75c0 2023-10-08T00:20:12.424 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.424+0000 7f457ee94700 1 -- 172.21.15.99:0/3105507080 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4578158f40 con 0x7f45780c75c0 2023-10-08T00:20:12.425 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.424+0000 7f4574ff9700 1 -- 172.21.15.99:0/3105507080 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v77) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4578158f40 con 0x7f45780c75c0 2023-10-08T00:20:12.425 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T00:20:12.427 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.426+0000 7f457ee94700 1 -- 172.21.15.99:0/3105507080 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4558066f30 msgr2=0x7f45580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:12.427 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.426+0000 7f457ee94700 1 --2- 172.21.15.99:0/3105507080 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4558066f30 0x7f45580693e0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f4570007fd0 tx=0x7f4570009040 comp rx=0 tx=0).stop 2023-10-08T00:20:12.427 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.426+0000 7f457ee94700 1 -- 172.21.15.99:0/3105507080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45780c75c0 msgr2=0x7f45780aa880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:12.427 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.426+0000 7f457ee94700 1 --2- 172.21.15.99:0/3105507080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45780c75c0 0x7f45780aa880 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f456000d890 tx=0x7f4560008040 comp rx=0 tx=0).stop 2023-10-08T00:20:12.427 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.426+0000 7f457ee94700 1 -- 172.21.15.99:0/3105507080 shutdown_connections 2023-10-08T00:20:12.427 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.426+0000 7f457ee94700 1 --2- 172.21.15.99:0/3105507080 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4558066f30 0x7f45580693e0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:12.427 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.426+0000 7f457ee94700 1 --2- 172.21.15.99:0/3105507080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45780c75c0 0x7f45780aa880 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:12.428 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.426+0000 7f457ee94700 1 --2- 172.21.15.99:0/3105507080 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f45780abae0 0x7f45780ae4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:12.428 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.426+0000 7f457ee94700 1 --2- 172.21.15.99:0/3105507080 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f45780abeb0 0x7f45781577e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:12.428 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.426+0000 7f457ee94700 1 -- 172.21.15.99:0/3105507080 >> 172.21.15.99:0/3105507080 conn(0x7f4578067660 msgr2=0x7f45780c8640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:20:12.428 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.426+0000 7f457ee94700 1 -- 172.21.15.99:0/3105507080 shutdown_connections 2023-10-08T00:20:12.428 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:12.426+0000 7f457ee94700 1 -- 172.21.15.99:0/3105507080 wait complete. 2023-10-08T00:20:12.438 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T00:20:12.438 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T00:20:12.439 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi191.front.sepia.ceph.com 2023-10-08T00:20:12.439 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-08T00:20:12.439 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-08T00:20:12.439 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T00:20:12.439 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T00:20:12.439 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T00:20:12.439 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T00:20:12.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:20:12.440 DEBUG:teuthology.orchestra.run.smithi191:> ip addr 2023-10-08T00:20:12.458 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T00:20:12.459 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T00:20:12.459 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T00:20:12.459 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:20:12.459 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2023-10-08T00:20:12.459 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:20:12.459 INFO:teuthology.orchestra.run.smithi191.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:20:12.460 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f6 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:20:12.460 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f0 2023-10-08T00:20:12.460 INFO:teuthology.orchestra.run.smithi191.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:20:12.460 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f7 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:20:12.460 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f1 2023-10-08T00:20:12.460 INFO:teuthology.orchestra.run.smithi191.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:20:12.461 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:10 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:20:12.461 INFO:teuthology.orchestra.run.smithi191.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T00:20:12.461 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:11 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:20:12.461 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.21.15.191/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T00:20:12.462 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft 41979sec preferred_lft 41979sec 2023-10-08T00:20:12.462 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec4:7aff:fe88:7211/64 scope link 2023-10-08T00:20:12.462 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:20:12.462 INFO:teuthology.orchestra.run.smithi191.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T00:20:12.462 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 9a:bc:39:72:87:3d brd ff:ff:ff:ff:ff:ff 2023-10-08T00:20:12.462 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T00:20:12.462 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:20:12.462 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec77:a7ff:fea4:fbb/64 scope link 2023-10-08T00:20:12.462 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:20:12.462 INFO:teuthology.orchestra.run.smithi191.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T00:20:12.462 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether ea:1c:d9:46:05:b3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-08T00:20:12.463 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::e81c:d9ff:fe46:5b3/64 scope link 2023-10-08T00:20:12.463 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:20:12.463 INFO:teuthology.orchestra.run.smithi191.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T00:20:12.463 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 9a:bc:39:72:87:3d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-08T00:20:12.463 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::98bc:39ff:fe72:873d/64 scope link 2023-10-08T00:20:12.463 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:20:12.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:20:12.463 DEBUG:teuthology.orchestra.run.smithi191:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T00:20:12.526 INFO:teuthology.orchestra.run.smithi191.stdout:1 2023-10-08T00:20:12.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:20:12.528 DEBUG:teuthology.orchestra.run.smithi191:> route 2023-10-08T00:20:12.587 INFO:teuthology.orchestra.run.smithi191.stdout:Kernel IP routing table 2023-10-08T00:20:12.587 INFO:teuthology.orchestra.run.smithi191.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T00:20:12.587 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T00:20:12.587 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T00:20:12.587 INFO:teuthology.orchestra.run.smithi191.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T00:20:12.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:20:12.588 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2023-10-08T00:20:12.588 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2023-10-08T00:20:12.588 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T00:20:12.588 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T00:20:12.588 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T00:20:12.588 DEBUG:teuthology.orchestra.run.smithi191:> ') 2023-10-08T00:20:12.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:20:12.693 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2023-10-08T00:20:12.708 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-08T00:20:12.709 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-08T00:20:12.709 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T00:20:12.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:20:12.709 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set brx.0 up 2023-10-08T00:20:12.773 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:20:12.774 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-08T00:20:12.829 INFO:teuthology.orchestra.run.smithi191.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-08T00:20:12.829 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:20:12.829 DEBUG:teuthology.orchestra.run.smithi191:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-08T00:20:12.884 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-08T00:20:12.884 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T00:20:12.885 DEBUG:teuthology.orchestra.run.smithi191:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:20:12.965 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:20:12.965 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:20:12.984 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:20:12.984 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:20:12.984 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.99:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T00:20:12.984 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@06962cdb-9a6f-417f-82fd-9e4e3dedbccf.cephfs=/ 2023-10-08T00:20:12.984 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.99:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T00:20:12.984 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T00:20:12.984 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T00:20:12.984 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T00:20:13.006 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:20:13.006 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:20:13.006 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.99:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T00:20:13.006 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@06962cdb-9a6f-417f-82fd-9e4e3dedbccf.cephfs=/ 2023-10-08T00:20:13.006 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.99:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T00:20:13.006 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T00:20:13.006 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T00:20:13.006 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T00:20:13.006 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.99:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T00:20:13.006 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.99:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T00:20:13.007 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=06962cdb-9a6f-417f-82fd-9e4e3dedbccf" will pass to kernel 2023-10-08T00:20:13.008 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T00:20:13.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:20:13.008 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-08T00:20:13.072 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:20:13.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.189+0000 7f2eeb1ba700 1 Processor -- start 2023-10-08T00:20:13.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.189+0000 7f2eeb1ba700 1 -- start start 2023-10-08T00:20:13.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.190+0000 7f2eeb1ba700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2ee415a620 0x7f2ee415d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:13.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.190+0000 7f2eeb1ba700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ee415cbb0 con 0x7f2ee4157870 2023-10-08T00:20:13.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.190+0000 7f2eeb1ba700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ee415bfa0 con 0x7f2ee415d580 2023-10-08T00:20:13.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.190+0000 7f2eeb1ba700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ee415bdb0 con 0x7f2ee415a620 2023-10-08T00:20:13.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.190+0000 7f2ee8f56700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f2ee4157870 0x7f2ee415b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56148/0 (socket says 172.21.15.99:56148) 2023-10-08T00:20:13.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.190+0000 7f2ee8f56700 1 -- 172.21.15.99:0/400815194 learned_addr learned my addr 172.21.15.99:0/400815194 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:20:13.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.191+0000 7f2edbfff700 1 --2- 172.21.15.99:0/400815194 >> v2:172.21.15.191:3300/0 conn(0x7f2ee415a620 0x7f2ee415d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:13.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.191+0000 7f2edb7fe700 1 -- 172.21.15.99:0/400815194 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4190110499 0 0) 0x7f2ee415cbb0 con 0x7f2ee4157870 2023-10-08T00:20:13.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.191+0000 7f2edb7fe700 1 -- 172.21.15.99:0/400815194 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ecc003570 con 0x7f2ee4157870 2023-10-08T00:20:13.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.191+0000 7f2edb7fe700 1 -- 172.21.15.99:0/400815194 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4124446725 0 0) 0x7f2ee415bfa0 con 0x7f2ee415d580 2023-10-08T00:20:13.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.191+0000 7f2edb7fe700 1 -- 172.21.15.99:0/400815194 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ee415cbb0 con 0x7f2ee415d580 2023-10-08T00:20:13.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.191+0000 7f2edb7fe700 1 -- 172.21.15.99:0/400815194 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2324797008 0 0) 0x7f2ecc003570 con 0x7f2ee4157870 2023-10-08T00:20:13.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.191+0000 7f2edb7fe700 1 -- 172.21.15.99:0/400815194 >> v2:172.21.15.191:3300/0 conn(0x7f2ee415a620 msgr2=0x7f2ee415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:13.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.191+0000 7f2edb7fe700 1 --2- 172.21.15.99:0/400815194 >> v2:172.21.15.191:3300/0 conn(0x7f2ee415a620 0x7f2ee415d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:13.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.191+0000 7f2edb7fe700 1 -- 172.21.15.99:0/400815194 >> v1:172.21.15.191:6789/0 conn(0x7f2ee415d580 legacy=0x7f2ee4158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:13.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.191+0000 7f2edb7fe700 1 -- 172.21.15.99:0/400815194 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee415c280 con 0x7f2ee4157870 2023-10-08T00:20:13.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.191+0000 7f2edb7fe700 1 -- 172.21.15.99:0/400815194 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f2ee00017f0 con 0x7f2ee4157870 2023-10-08T00:20:13.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.191+0000 7f2edbfff700 1 --2- 172.21.15.99:0/400815194 >> v2:172.21.15.191:3300/0 conn(0x7f2ee415a620 0x7f2ee415d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:20:13.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.191+0000 7f2edb7fe700 1 -- 172.21.15.99:0/400815194 >> v1:172.21.15.99:6789/0 conn(0x7f2ee4157870 legacy=0x7f2ee415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:13.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.191+0000 7f2edb7fe700 1 --2- 172.21.15.99:0/400815194 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ecc003bf0 0x7f2ecc003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:13.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.191+0000 7f2edb7fe700 1 --2- 172.21.15.99:0/400815194 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ecc004810 0x7f2ecc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:13.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.191+0000 7f2edb7fe700 1 --2- 172.21.15.99:0/400815194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ecc0051c0 0x7f2ecc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:13.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.191+0000 7f2edb7fe700 1 -- 172.21.15.99:0/400815194 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ecc00c130 con 0x7f2ecc0051c0 2023-10-08T00:20:13.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.191+0000 7f2edb7fe700 1 -- 172.21.15.99:0/400815194 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ecc00c2a0 con 0x7f2ecc004810 2023-10-08T00:20:13.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.191+0000 7f2edb7fe700 1 -- 172.21.15.99:0/400815194 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ecc00c460 con 0x7f2ecc003bf0 2023-10-08T00:20:13.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.191+0000 7f2ee9757700 1 --2- 172.21.15.99:0/400815194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ecc0051c0 0x7f2ecc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:13.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.191+0000 7f2edbfff700 1 --2- 172.21.15.99:0/400815194 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ecc004810 0x7f2ecc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:13.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.191+0000 7f2ee8f56700 1 --2- 172.21.15.99:0/400815194 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ecc003bf0 0x7f2ecc003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:13.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.191+0000 7f2ee9757700 1 -- 172.21.15.99:0/400815194 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ecc003bf0 msgr2=0x7f2ecc003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:13.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.191+0000 7f2ee9757700 1 --2- 172.21.15.99:0/400815194 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ecc003bf0 0x7f2ecc003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:13.200 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.191+0000 7f2ee9757700 1 -- 172.21.15.99:0/400815194 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ecc004810 msgr2=0x7f2ecc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:13.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.191+0000 7f2ee9757700 1 --2- 172.21.15.99:0/400815194 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ecc004810 0x7f2ecc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:13.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.191+0000 7f2ee9757700 1 -- 172.21.15.99:0/400815194 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ecc003570 con 0x7f2ecc0051c0 2023-10-08T00:20:13.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.191+0000 7f2ee9757700 1 -- 172.21.15.99:0/400815194 reap_dead start 2023-10-08T00:20:13.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.192+0000 7f2ee9757700 1 --2- 172.21.15.99:0/400815194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ecc0051c0 0x7f2ecc00ba60 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f2ee415d9e0 tx=0x7f2edc007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fda762a179defd81 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:13.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.192+0000 7f2edb7fe700 1 -- 172.21.15.99:0/400815194 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2edc00c5d0 con 0x7f2ecc0051c0 2023-10-08T00:20:13.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.192+0000 7f2edb7fe700 1 -- 172.21.15.99:0/400815194 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2edc00c730 con 0x7f2ecc0051c0 2023-10-08T00:20:13.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.192+0000 7f2edb7fe700 1 -- 172.21.15.99:0/400815194 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2edc00cab0 con 0x7f2ecc0051c0 2023-10-08T00:20:13.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.192+0000 7f2eeb1ba700 1 -- 172.21.15.99:0/400815194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ecc0051c0 msgr2=0x7f2ecc00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:13.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.192+0000 7f2eeb1ba700 1 --2- 172.21.15.99:0/400815194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ecc0051c0 0x7f2ecc00ba60 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f2ee415d9e0 tx=0x7f2edc007d30 comp rx=0 tx=0).stop 2023-10-08T00:20:13.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.192+0000 7f2eeb1ba700 1 -- 172.21.15.99:0/400815194 shutdown_connections 2023-10-08T00:20:13.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.192+0000 7f2eeb1ba700 1 --2- 172.21.15.99:0/400815194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ecc0051c0 0x7f2ecc00ba60 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:13.201 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.192+0000 7f2eeb1ba700 1 -- 172.21.15.99:0/400815194 >> 172.21.15.99:0/400815194 conn(0x7f2ee4068ce0 msgr2=0x7f2ee40690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:20:13.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.192+0000 7f2eeb1ba700 1 -- 172.21.15.99:0/400815194 shutdown_connections 2023-10-08T00:20:13.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.192+0000 7f2eeb1ba700 1 -- 172.21.15.99:0/400815194 wait complete. 2023-10-08T00:20:13.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.193+0000 7f2eeb1ba700 1 Processor -- start 2023-10-08T00:20:13.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.194+0000 7f2eeb1ba700 1 -- start start 2023-10-08T00:20:13.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.194+0000 7f2eeb1ba700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ecc0051c0 0x7f2ee4160160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:13.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.194+0000 7f2eeb1ba700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ee40b0610 0x7f2ee40b1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:13.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.194+0000 7f2eeb1ba700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ee40cf2b0 0x7f2ee40cf720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:13.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.194+0000 7f2eeb1ba700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee415db90 con 0x7f2ecc0051c0 2023-10-08T00:20:13.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.194+0000 7f2eeb1ba700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee415fcd0 con 0x7f2ee40cf2b0 2023-10-08T00:20:13.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.194+0000 7f2eeb1ba700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee41606a0 con 0x7f2ee40b0610 2023-10-08T00:20:13.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.194+0000 7f2ee8f56700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ecc0051c0 0x7f2ee4160160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:13.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.194+0000 7f2ee8f56700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ecc0051c0 0x7f2ee4160160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:32836/0 (socket says 172.21.15.99:32836) 2023-10-08T00:20:13.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.194+0000 7f2ee8f56700 1 -- 172.21.15.99:0/3057156094 learned_addr learned my addr 172.21.15.99:0/3057156094 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:20:13.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.195+0000 7f2edbfff700 1 --2- 172.21.15.99:0/3057156094 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ee40b0610 0x7f2ee40b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:13.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.195+0000 7f2ee8f56700 1 -- 172.21.15.99:0/3057156094 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ee40b0610 msgr2=0x7f2ee40b1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:13.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.195+0000 7f2ee8f56700 1 --2- 172.21.15.99:0/3057156094 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ee40b0610 0x7f2ee40b1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:13.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.195+0000 7f2ee9757700 1 --2- 172.21.15.99:0/3057156094 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ee40cf2b0 0x7f2ee40cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:13.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.195+0000 7f2ee8f56700 1 -- 172.21.15.99:0/3057156094 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ee40cf2b0 msgr2=0x7f2ee40cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:13.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.195+0000 7f2ee8f56700 1 --2- 172.21.15.99:0/3057156094 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ee40cf2b0 0x7f2ee40cf720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:13.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.195+0000 7f2ee8f56700 1 -- 172.21.15.99:0/3057156094 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee415f100 con 0x7f2ecc0051c0 2023-10-08T00:20:13.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.195+0000 7f2edbfff700 1 --2- 172.21.15.99:0/3057156094 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ee40b0610 0x7f2ee40b1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:20:13.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.195+0000 7f2ee8f56700 1 --2- 172.21.15.99:0/3057156094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ecc0051c0 0x7f2ee4160160 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0001190 tx=0x7f2ee0002170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:13.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.195+0000 7f2ed8ff9700 1 -- 172.21.15.99:0/3057156094 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee0007b40 con 0x7f2ecc0051c0 2023-10-08T00:20:13.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.195+0000 7f2ed8ff9700 1 -- 172.21.15.99:0/3057156094 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ee000ad70 con 0x7f2ecc0051c0 2023-10-08T00:20:13.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.195+0000 7f2eeb1ba700 1 -- 172.21.15.99:0/3057156094 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ee415f3a0 con 0x7f2ecc0051c0 2023-10-08T00:20:13.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.195+0000 7f2ed8ff9700 1 -- 172.21.15.99:0/3057156094 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee0013640 con 0x7f2ecc0051c0 2023-10-08T00:20:13.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.195+0000 7f2eeb1ba700 1 -- 172.21.15.99:0/3057156094 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ee415f960 con 0x7f2ecc0051c0 2023-10-08T00:20:13.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.196+0000 7f2eeb1ba700 1 -- 172.21.15.99:0/3057156094 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ee404c3c0 con 0x7f2ecc0051c0 2023-10-08T00:20:13.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.196+0000 7f2ed8ff9700 1 -- 172.21.15.99:0/3057156094 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2ee0013a30 con 0x7f2ecc0051c0 2023-10-08T00:20:13.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.197+0000 7f2ed8ff9700 1 --2- 172.21.15.99:0/3057156094 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2ed0066f30 0x7f2ed00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:13.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.197+0000 7f2ed8ff9700 1 -- 172.21.15.99:0/3057156094 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2ee0085310 con 0x7f2ecc0051c0 2023-10-08T00:20:13.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.197+0000 7f2edbfff700 1 --2- 172.21.15.99:0/3057156094 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2ed0066f30 0x7f2ed00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:13.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.197+0000 7f2edbfff700 1 --2- 172.21.15.99:0/3057156094 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2ed0066f30 0x7f2ed00693e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f2ee40d0ef0 tx=0x7f2ed4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:13.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.201+0000 7f2ed8ff9700 1 -- 172.21.15.99:0/3057156094 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2ee0052580 con 0x7f2ecc0051c0 2023-10-08T00:20:13.430 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.429+0000 7f2eeb1ba700 1 -- 172.21.15.99:0/3057156094 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ee415c560 con 0x7f2ecc0051c0 2023-10-08T00:20:13.430 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.430+0000 7f2ed8ff9700 1 -- 172.21.15.99:0/3057156094 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4601 (secure 0 0 0) 0x7f2ee0055ba0 con 0x7f2ecc0051c0 2023-10-08T00:20:13.431 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:20:13.431 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":77,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6257,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/661016677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":661016677},{"type":"v1","addr":"172.21.15.191:6837","nonce":661016677}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":7012,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/4102051559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":4102051559},{"type":"v1","addr":"172.21.15.99:6835","nonce":4102051559}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":7021,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/658854389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":658854389},{"type":"v1","addr":"172.21.15.191:6836","nonce":658854389}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:20:04.613994+0000","modified":"2023-10-08T00:20:08.255350+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7791},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7791":{"gid":7791,"name":"c","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6834/3880334948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3880334948},{"type":"v1","addr":"172.21.15.99:6834","nonce":3880334948}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-08T00:20:13.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.432+0000 7f2eeb1ba700 1 -- 172.21.15.99:0/3057156094 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2ed0066f30 msgr2=0x7f2ed00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:13.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.432+0000 7f2eeb1ba700 1 --2- 172.21.15.99:0/3057156094 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2ed0066f30 0x7f2ed00693e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f2ee40d0ef0 tx=0x7f2ed4008040 comp rx=0 tx=0).stop 2023-10-08T00:20:13.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.432+0000 7f2eeb1ba700 1 -- 172.21.15.99:0/3057156094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ecc0051c0 msgr2=0x7f2ee4160160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:13.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.432+0000 7f2eeb1ba700 1 --2- 172.21.15.99:0/3057156094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ecc0051c0 0x7f2ee4160160 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0001190 tx=0x7f2ee0002170 comp rx=0 tx=0).stop 2023-10-08T00:20:13.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.432+0000 7f2eeb1ba700 1 -- 172.21.15.99:0/3057156094 shutdown_connections 2023-10-08T00:20:13.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.432+0000 7f2eeb1ba700 1 --2- 172.21.15.99:0/3057156094 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2ed0066f30 0x7f2ed00693e0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:13.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.432+0000 7f2eeb1ba700 1 --2- 172.21.15.99:0/3057156094 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ee40cf2b0 0x7f2ee40cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:13.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.432+0000 7f2eeb1ba700 1 --2- 172.21.15.99:0/3057156094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ecc0051c0 0x7f2ee4160160 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:13.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.432+0000 7f2eeb1ba700 1 --2- 172.21.15.99:0/3057156094 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ee40b0610 0x7f2ee40b1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:13.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.432+0000 7f2eeb1ba700 1 -- 172.21.15.99:0/3057156094 >> 172.21.15.99:0/3057156094 conn(0x7f2ee4068ce0 msgr2=0x7f2ee4157c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:20:13.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.432+0000 7f2eeb1ba700 1 -- 172.21.15.99:0/3057156094 shutdown_connections 2023-10-08T00:20:13.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.432+0000 7f2eeb1ba700 1 -- 172.21.15.99:0/3057156094 wait complete. 2023-10-08T00:20:13.434 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 77 2023-10-08T00:20:13.445 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-10-08T00:20:13.577 INFO:teuthology.orchestra.run.smithi099.stdout:{"cluster_fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","whoami":0,"id":7791,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":8.3313191639999999,"mdsmap_epoch":77,"osdmap_epoch":168,"osdmap_epoch_barrier":168,"uptime":13.443989412000001} 2023-10-08T00:20:13.578 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", "fs_name": "cephfs", "id": 7791, "mdsmap_epoch": 77, "osdmap_epoch": 168, "osdmap_epoch_barrier": 168, "rank_uptime": 8.331319164, "state": "up:active", "uptime": 13.443989412, "want_state": "up:active", "whoami": 0 } 2023-10-08T00:20:13.578 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2023-10-08T00:20:13.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.695+0000 7fe63571b700 1 Processor -- start 2023-10-08T00:20:13.696 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.695+0000 7fe63571b700 1 -- start start 2023-10-08T00:20:13.699 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.696+0000 7fe63571b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fe6300cc940 0x7fe6300ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:13.699 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.696+0000 7fe63571b700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6300cbb80 con 0x7fe6300ce810 2023-10-08T00:20:13.699 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.696+0000 7fe63571b700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6300cbe60 con 0x7fe63015d240 2023-10-08T00:20:13.700 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.696+0000 7fe63571b700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6300cc140 con 0x7fe6300cc940 2023-10-08T00:20:13.700 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.696+0000 7fe634f1a700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fe6300ce810 0x7fe6300cec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:48816/0 (socket says 172.21.15.99:48816) 2023-10-08T00:20:13.700 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.696+0000 7fe634f1a700 1 -- 172.21.15.99:0/2436769 learned_addr learned my addr 172.21.15.99:0/2436769 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:20:13.700 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.696+0000 7fe62f7fe700 1 --2- 172.21.15.99:0/2436769 >> v2:172.21.15.191:3300/0 conn(0x7fe6300cc940 0x7fe6300ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:13.700 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.696+0000 7fe62effd700 1 -- 172.21.15.99:0/2436769 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3137339708 0 0) 0x7fe6300cbb80 con 0x7fe6300ce810 2023-10-08T00:20:13.700 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.696+0000 7fe62effd700 1 -- 172.21.15.99:0/2436769 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe618003570 con 0x7fe6300ce810 2023-10-08T00:20:13.700 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.696+0000 7fe62effd700 1 -- 172.21.15.99:0/2436769 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 529320632 0 0) 0x7fe618003570 con 0x7fe6300ce810 2023-10-08T00:20:13.700 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.696+0000 7fe62effd700 1 -- 172.21.15.99:0/2436769 >> v2:172.21.15.191:3300/0 conn(0x7fe6300cc940 msgr2=0x7fe6300ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:13.700 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.696+0000 7fe62effd700 1 --2- 172.21.15.99:0/2436769 >> v2:172.21.15.191:3300/0 conn(0x7fe6300cc940 0x7fe6300ccde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:13.701 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.696+0000 7fe62effd700 1 -- 172.21.15.99:0/2436769 >> v1:172.21.15.191:6789/0 conn(0x7fe63015d240 legacy=0x7fe6300b3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:13.701 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.696+0000 7fe62effd700 1 -- 172.21.15.99:0/2436769 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6300cc420 con 0x7fe6300ce810 2023-10-08T00:20:13.701 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.696+0000 7fe62effd700 1 -- 172.21.15.99:0/2436769 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7fe6280017f0 con 0x7fe6300ce810 2023-10-08T00:20:13.702 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.696+0000 7fe62effd700 1 -- 172.21.15.99:0/2436769 >> v1:172.21.15.99:6789/0 conn(0x7fe6300ce810 legacy=0x7fe6300cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:13.702 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.697+0000 7fe62effd700 1 --2- 172.21.15.99:0/2436769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe618003a60 0x7fe618003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:13.702 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.697+0000 7fe62effd700 1 --2- 172.21.15.99:0/2436769 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe618004810 0x7fe618004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:13.702 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.697+0000 7fe62effd700 1 --2- 172.21.15.99:0/2436769 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6180051c0 0x7fe61800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:13.702 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.697+0000 7fe62effd700 1 -- 172.21.15.99:0/2436769 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6300cc140 con 0x7fe618003a60 2023-10-08T00:20:13.702 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.697+0000 7fe62effd700 1 -- 172.21.15.99:0/2436769 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61800c0e0 con 0x7fe618004810 2023-10-08T00:20:13.702 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.697+0000 7fe62effd700 1 -- 172.21.15.99:0/2436769 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe61800c2a0 con 0x7fe6180051c0 2023-10-08T00:20:13.703 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.697+0000 7fe62ffff700 1 --2- 172.21.15.99:0/2436769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe618003a60 0x7fe618003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:13.703 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.697+0000 7fe62f7fe700 1 --2- 172.21.15.99:0/2436769 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe618004810 0x7fe618004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:13.703 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.697+0000 7fe634f1a700 1 --2- 172.21.15.99:0/2436769 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6180051c0 0x7fe61800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:13.703 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.697+0000 7fe62ffff700 1 -- 172.21.15.99:0/2436769 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6180051c0 msgr2=0x7fe61800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:13.703 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.697+0000 7fe62ffff700 1 --2- 172.21.15.99:0/2436769 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6180051c0 0x7fe61800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:13.703 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.697+0000 7fe62ffff700 1 -- 172.21.15.99:0/2436769 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe618004810 msgr2=0x7fe618004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:20:13.703 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.697+0000 7fe62ffff700 1 --2- 172.21.15.99:0/2436769 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe618004810 0x7fe618004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:13.703 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.697+0000 7fe62ffff700 1 -- 172.21.15.99:0/2436769 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe618003570 con 0x7fe618003a60 2023-10-08T00:20:13.703 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.697+0000 7fe634f1a700 1 -- 172.21.15.99:0/2436769 reap_dead start 2023-10-08T00:20:13.703 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.697+0000 7fe62ffff700 1 --2- 172.21.15.99:0/2436769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe618003a60 0x7fe618003e30 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fe6200019c0 tx=0x7fe620007d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c62d6e6b08a607b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:13.703 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.697+0000 7fe62effd700 1 -- 172.21.15.99:0/2436769 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe620001330 con 0x7fe618003a60 2023-10-08T00:20:13.704 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.697+0000 7fe62effd700 1 -- 172.21.15.99:0/2436769 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe62000c5d0 con 0x7fe618003a60 2023-10-08T00:20:13.704 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.697+0000 7fe62effd700 1 -- 172.21.15.99:0/2436769 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62000c970 con 0x7fe618003a60 2023-10-08T00:20:13.704 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.698+0000 7fe63571b700 1 -- 172.21.15.99:0/2436769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe618003a60 msgr2=0x7fe618003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:13.704 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.698+0000 7fe63571b700 1 --2- 172.21.15.99:0/2436769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe618003a60 0x7fe618003e30 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fe6200019c0 tx=0x7fe620007d60 comp rx=0 tx=0).stop 2023-10-08T00:20:13.704 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.698+0000 7fe63571b700 1 -- 172.21.15.99:0/2436769 shutdown_connections 2023-10-08T00:20:13.704 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.698+0000 7fe63571b700 1 --2- 172.21.15.99:0/2436769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe618003a60 0x7fe618003e30 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:13.704 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.698+0000 7fe63571b700 1 -- 172.21.15.99:0/2436769 >> 172.21.15.99:0/2436769 conn(0x7fe630067e40 msgr2=0x7fe6300d0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:20:13.704 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.698+0000 7fe63571b700 1 -- 172.21.15.99:0/2436769 shutdown_connections 2023-10-08T00:20:13.704 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.698+0000 7fe63571b700 1 -- 172.21.15.99:0/2436769 wait complete. 2023-10-08T00:20:13.704 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.699+0000 7fe63571b700 1 Processor -- start 2023-10-08T00:20:13.704 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.699+0000 7fe63571b700 1 -- start start 2023-10-08T00:20:13.708 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.700+0000 7fe63571b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe618003a60 0x7fe6301567f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:13.708 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.700+0000 7fe63571b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe630169e90 0x7fe63014a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:13.708 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.700+0000 7fe63571b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe63015c190 0x7fe6300c59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:13.708 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.700+0000 7fe63571b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe63014acc0 con 0x7fe630169e90 2023-10-08T00:20:13.708 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.700+0000 7fe63571b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6300c98d0 con 0x7fe618003a60 2023-10-08T00:20:13.709 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.700+0000 7fe63571b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe630156ff0 con 0x7fe63015c190 2023-10-08T00:20:13.709 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.700+0000 7fe62f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe630169e90 0x7fe63014a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:13.709 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.700+0000 7fe62f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe630169e90 0x7fe63014a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53338/0 (socket says 172.21.15.99:53338) 2023-10-08T00:20:13.709 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.700+0000 7fe62f7fe700 1 -- 172.21.15.99:0/1867783676 learned_addr learned my addr 172.21.15.99:0/1867783676 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:20:13.709 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.700+0000 7fe634f1a700 1 --2- 172.21.15.99:0/1867783676 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe63015c190 0x7fe6300c59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:13.709 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.700+0000 7fe62ffff700 1 --2- 172.21.15.99:0/1867783676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe618003a60 0x7fe6301567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:13.709 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.700+0000 7fe62f7fe700 1 -- 172.21.15.99:0/1867783676 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe63015c190 msgr2=0x7fe6300c59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:13.709 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.700+0000 7fe62f7fe700 1 --2- 172.21.15.99:0/1867783676 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe63015c190 0x7fe6300c59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:13.709 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.700+0000 7fe62f7fe700 1 -- 172.21.15.99:0/1867783676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe618003a60 msgr2=0x7fe6301567f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:13.709 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.700+0000 7fe62f7fe700 1 --2- 172.21.15.99:0/1867783676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe618003a60 0x7fe6301567f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:13.709 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.700+0000 7fe62f7fe700 1 -- 172.21.15.99:0/1867783676 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe63015c920 con 0x7fe630169e90 2023-10-08T00:20:13.710 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.700+0000 7fe634f1a700 1 --2- 172.21.15.99:0/1867783676 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe63015c190 0x7fe6300c59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:20:13.710 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.700+0000 7fe62f7fe700 1 --2- 172.21.15.99:0/1867783676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe630169e90 0x7fe63014a780 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fe624011d40 tx=0x7fe6240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:13.710 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.700+0000 7fe617fff700 1 -- 172.21.15.99:0/1867783676 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62400fa60 con 0x7fe630169e90 2023-10-08T00:20:13.710 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.700+0000 7fe617fff700 1 -- 172.21.15.99:0/1867783676 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe62400fbc0 con 0x7fe630169e90 2023-10-08T00:20:13.710 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.700+0000 7fe63571b700 1 -- 172.21.15.99:0/1867783676 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6300c8fa0 con 0x7fe630169e90 2023-10-08T00:20:13.710 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.700+0000 7fe617fff700 1 -- 172.21.15.99:0/1867783676 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe624014440 con 0x7fe630169e90 2023-10-08T00:20:13.710 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.700+0000 7fe63571b700 1 -- 172.21.15.99:0/1867783676 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6300c9560 con 0x7fe630169e90 2023-10-08T00:20:13.710 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.701+0000 7fe63571b700 1 -- 172.21.15.99:0/1867783676 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe63006ac60 con 0x7fe630169e90 2023-10-08T00:20:13.710 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.702+0000 7fe617fff700 1 -- 172.21.15.99:0/1867783676 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe624014a80 con 0x7fe630169e90 2023-10-08T00:20:13.710 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.702+0000 7fe617fff700 1 --2- 172.21.15.99:0/1867783676 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe61c066f30 0x7fe61c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:20:13.710 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.702+0000 7fe617fff700 1 -- 172.21.15.99:0/1867783676 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe624018030 con 0x7fe630169e90 2023-10-08T00:20:13.711 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.702+0000 7fe62ffff700 1 --2- 172.21.15.99:0/1867783676 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe61c066f30 0x7fe61c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:20:13.711 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.703+0000 7fe62ffff700 1 --2- 172.21.15.99:0/1867783676 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe61c066f30 0x7fe61c0693e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fe620001780 tx=0x7fe620015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:20:13.711 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.706+0000 7fe617fff700 1 -- 172.21.15.99:0/1867783676 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe624052430 con 0x7fe630169e90 2023-10-08T00:20:13.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:13.886+0000 7fe63571b700 1 -- 172.21.15.99:0/1867783676 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7fe6300c9a10 con 0x7fe630169e90 2023-10-08T00:20:14.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:14.275+0000 7fe617fff700 1 -- 172.21.15.99:0/1867783676 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v169) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe6240220c0 con 0x7fe630169e90 2023-10-08T00:20:14.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:14.278+0000 7fe63571b700 1 -- 172.21.15.99:0/1867783676 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe61c066f30 msgr2=0x7fe61c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:14.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:14.278+0000 7fe63571b700 1 --2- 172.21.15.99:0/1867783676 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe61c066f30 0x7fe61c0693e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fe620001780 tx=0x7fe620015000 comp rx=0 tx=0).stop 2023-10-08T00:20:14.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:14.278+0000 7fe63571b700 1 -- 172.21.15.99:0/1867783676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe630169e90 msgr2=0x7fe63014a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:20:14.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:14.278+0000 7fe63571b700 1 --2- 172.21.15.99:0/1867783676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe630169e90 0x7fe63014a780 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fe624011d40 tx=0x7fe6240091a0 comp rx=0 tx=0).stop 2023-10-08T00:20:14.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:14.278+0000 7fe63571b700 1 -- 172.21.15.99:0/1867783676 shutdown_connections 2023-10-08T00:20:14.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:14.278+0000 7fe63571b700 1 --2- 172.21.15.99:0/1867783676 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe61c066f30 0x7fe61c0693e0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:14.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:14.278+0000 7fe63571b700 1 --2- 172.21.15.99:0/1867783676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe618003a60 0x7fe6301567f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:14.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:14.278+0000 7fe63571b700 1 --2- 172.21.15.99:0/1867783676 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe63015c190 0x7fe6300c59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:14.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:14.278+0000 7fe63571b700 1 --2- 172.21.15.99:0/1867783676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe630169e90 0x7fe63014a780 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:20:14.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:14.278+0000 7fe63571b700 1 -- 172.21.15.99:0/1867783676 >> 172.21.15.99:0/1867783676 conn(0x7fe630067e40 msgr2=0x7fe630069270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:20:14.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:14.278+0000 7fe63571b700 1 -- 172.21.15.99:0/1867783676 shutdown_connections 2023-10-08T00:20:14.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:20:14.278+0000 7fe63571b700 1 -- 172.21.15.99:0/1867783676 wait complete. 2023-10-08T00:20:14.280 INFO:teuthology.orchestra.run.smithi099.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2023-10-08T00:20:14.291 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-10-08T00:20:14.420 INFO:teuthology.orchestra.run.smithi099.stdout:{"osd_mon_report_interval":"5"} 2023-10-08T00:20:14.421 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-10-08T00:20:14.421 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2023-10-08T00:20:14.543 INFO:teuthology.orchestra.run.smithi099.stdout:{"mon_tick_interval":"5"} 2023-10-08T00:20:14.543 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2023-10-08T00:20:14.543 INFO:tasks.cephfs.test_full:pool capacity 33554432, 32MB should be enough to fill it 2023-10-08T00:20:14.543 DEBUG:teuthology.orchestra.run.smithi099:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-08T00:20:14.543 DEBUG:teuthology.orchestra.run.smithi099:> import time 2023-10-08T00:20:14.544 DEBUG:teuthology.orchestra.run.smithi099:> import datetime 2023-10-08T00:20:14.544 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2023-10-08T00:20:14.544 DEBUG:teuthology.orchestra.run.smithi099:> import os 2023-10-08T00:20:14.544 DEBUG:teuthology.orchestra.run.smithi099:> 2023-10-08T00:20:14.544 DEBUG:teuthology.orchestra.run.smithi099:> # Write some buffered data through before going full, all should be well 2023-10-08T00:20:14.544 DEBUG:teuthology.orchestra.run.smithi099:> print("writing some data through which we expect to succeed") 2023-10-08T00:20:14.544 DEBUG:teuthology.orchestra.run.smithi099:> bytes = 0 2023-10-08T00:20:14.544 DEBUG:teuthology.orchestra.run.smithi099:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2023-10-08T00:20:14.544 DEBUG:teuthology.orchestra.run.smithi099:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2023-10-08T00:20:14.544 DEBUG:teuthology.orchestra.run.smithi099:> bytes += os.write(f, b'"'"'a'"'"' * 4096) 2023-10-08T00:20:14.544 DEBUG:teuthology.orchestra.run.smithi099:> os.fsync(f) 2023-10-08T00:20:14.545 DEBUG:teuthology.orchestra.run.smithi099:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2023-10-08T00:20:14.545 DEBUG:teuthology.orchestra.run.smithi099:> 2023-10-08T00:20:14.545 DEBUG:teuthology.orchestra.run.smithi099:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2023-10-08T00:20:14.545 DEBUG:teuthology.orchestra.run.smithi099:> # writing until we see an error from fsync. As long as we'"'"'re doing 2023-10-08T00:20:14.545 DEBUG:teuthology.orchestra.run.smithi099:> # buffered IO, the error should always only appear from fsync and not 2023-10-08T00:20:14.545 DEBUG:teuthology.orchestra.run.smithi099:> # from write 2023-10-08T00:20:14.545 DEBUG:teuthology.orchestra.run.smithi099:> full = False 2023-10-08T00:20:14.545 DEBUG:teuthology.orchestra.run.smithi099:> 2023-10-08T00:20:14.545 DEBUG:teuthology.orchestra.run.smithi099:> for n in range(0, int(32 * 1.1)): 2023-10-08T00:20:14.545 DEBUG:teuthology.orchestra.run.smithi099:> try: 2023-10-08T00:20:14.546 DEBUG:teuthology.orchestra.run.smithi099:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2023-10-08T00:20:14.546 DEBUG:teuthology.orchestra.run.smithi099:> print("wrote bytes via buffered write, moving on to fsync") 2023-10-08T00:20:14.546 DEBUG:teuthology.orchestra.run.smithi099:> except OSError as e: 2023-10-08T00:20:14.546 DEBUG:teuthology.orchestra.run.smithi099:> if False: 2023-10-08T00:20:14.546 DEBUG:teuthology.orchestra.run.smithi099:> print("Unexpected error %s from write() instead of fsync()" % e) 2023-10-08T00:20:14.546 DEBUG:teuthology.orchestra.run.smithi099:> raise 2023-10-08T00:20:14.546 DEBUG:teuthology.orchestra.run.smithi099:> else: 2023-10-08T00:20:14.546 DEBUG:teuthology.orchestra.run.smithi099:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2023-10-08T00:20:14.546 DEBUG:teuthology.orchestra.run.smithi099:> full = True 2023-10-08T00:20:14.546 DEBUG:teuthology.orchestra.run.smithi099:> break 2023-10-08T00:20:14.547 DEBUG:teuthology.orchestra.run.smithi099:> 2023-10-08T00:20:14.547 DEBUG:teuthology.orchestra.run.smithi099:> try: 2023-10-08T00:20:14.547 DEBUG:teuthology.orchestra.run.smithi099:> os.fsync(f) 2023-10-08T00:20:14.547 DEBUG:teuthology.orchestra.run.smithi099:> print("fsync'"'"'ed successfully") 2023-10-08T00:20:14.547 DEBUG:teuthology.orchestra.run.smithi099:> except OSError as e: 2023-10-08T00:20:14.547 DEBUG:teuthology.orchestra.run.smithi099:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2023-10-08T00:20:14.547 DEBUG:teuthology.orchestra.run.smithi099:> full = True 2023-10-08T00:20:14.547 DEBUG:teuthology.orchestra.run.smithi099:> break 2023-10-08T00:20:14.547 DEBUG:teuthology.orchestra.run.smithi099:> else: 2023-10-08T00:20:14.547 DEBUG:teuthology.orchestra.run.smithi099:> print("Not full yet after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2023-10-08T00:20:14.547 DEBUG:teuthology.orchestra.run.smithi099:> 2023-10-08T00:20:14.548 DEBUG:teuthology.orchestra.run.smithi099:> if n > 32 * 0.9: 2023-10-08T00:20:14.548 DEBUG:teuthology.orchestra.run.smithi099:> # Be cautious in the last region where we expect to hit 2023-10-08T00:20:14.548 DEBUG:teuthology.orchestra.run.smithi099:> # the full condition, so that we don'"'"'t overshoot too dramatically 2023-10-08T00:20:14.548 DEBUG:teuthology.orchestra.run.smithi099:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2023-10-08T00:20:14.548 DEBUG:teuthology.orchestra.run.smithi099:> time.sleep(15.0) 2023-10-08T00:20:14.548 DEBUG:teuthology.orchestra.run.smithi099:> 2023-10-08T00:20:14.548 DEBUG:teuthology.orchestra.run.smithi099:> if not full: 2023-10-08T00:20:14.548 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Failed to reach fullness after writing %d bytes" % bytes) 2023-10-08T00:20:14.548 DEBUG:teuthology.orchestra.run.smithi099:> 2023-10-08T00:20:14.548 DEBUG:teuthology.orchestra.run.smithi099:> # close() should not raise an error because we already caught it in 2023-10-08T00:20:14.549 DEBUG:teuthology.orchestra.run.smithi099:> # fsync. There shouldn'"'"'t have been any more writeback errors 2023-10-08T00:20:14.549 DEBUG:teuthology.orchestra.run.smithi099:> # since then because all IOs got cancelled on the full flag. 2023-10-08T00:20:14.549 DEBUG:teuthology.orchestra.run.smithi099:> print("calling close") 2023-10-08T00:20:14.549 DEBUG:teuthology.orchestra.run.smithi099:> os.close(f) 2023-10-08T00:20:14.549 DEBUG:teuthology.orchestra.run.smithi099:> print("close() did not raise error") 2023-10-08T00:20:14.549 DEBUG:teuthology.orchestra.run.smithi099:> 2023-10-08T00:20:14.549 DEBUG:teuthology.orchestra.run.smithi099:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2023-10-08T00:20:14.549 DEBUG:teuthology.orchestra.run.smithi099:> ' 2023-10-08T00:20:23.863 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:20:24.051 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:20:54.129 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:20:54.132 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:21:15.263 INFO:teuthology.orchestra.run.smithi099.stdout:writing some data through which we expect to succeed 2023-10-08T00:21:15.264 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed data successfully, will now attempt to fill fs 2023-10-08T00:21:15.264 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:21:15.264 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:21:15.264 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 1.00 MB 2023-10-08T00:21:15.264 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:21:15.264 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:21:15.264 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 2.00 MB 2023-10-08T00:21:15.264 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:21:15.265 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:21:15.265 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 3.00 MB 2023-10-08T00:21:15.265 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:21:15.265 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:21:15.265 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 4.00 MB 2023-10-08T00:21:15.265 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:21:15.265 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:21:15.265 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 5.00 MB 2023-10-08T00:21:15.265 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:21:15.265 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:21:15.265 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 6.00 MB 2023-10-08T00:21:15.265 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:21:15.266 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:21:15.266 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 7.00 MB 2023-10-08T00:21:15.266 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:21:15.266 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:21:15.266 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 8.00 MB 2023-10-08T00:21:15.266 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:21:15.266 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:21:15.266 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 9.00 MB 2023-10-08T00:21:15.266 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:21:15.266 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:21:15.266 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 10.00 MB 2023-10-08T00:21:15.266 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:21:15.267 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:21:15.267 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 11.00 MB 2023-10-08T00:21:15.267 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:21:15.267 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:21:15.267 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 12.00 MB 2023-10-08T00:21:15.267 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:21:15.267 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:21:15.267 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 13.00 MB 2023-10-08T00:21:15.267 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:21:15.267 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:21:15.267 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 14.00 MB 2023-10-08T00:21:15.267 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:21:15.268 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:21:15.268 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 15.00 MB 2023-10-08T00:21:15.268 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:21:15.268 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:21:15.268 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 16.00 MB 2023-10-08T00:21:15.268 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:21:15.268 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:21:15.268 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 17.00 MB 2023-10-08T00:21:15.268 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:21:15.268 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:21:15.268 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 18.00 MB 2023-10-08T00:21:15.268 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:21:15.269 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:21:15.269 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 19.00 MB 2023-10-08T00:21:15.269 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:21:15.269 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:21:15.269 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 20.00 MB 2023-10-08T00:21:15.269 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:21:15.269 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:21:15.269 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 21.00 MB 2023-10-08T00:21:15.269 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:21:15.270 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:21:15.270 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 22.00 MB 2023-10-08T00:21:15.270 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:21:15.270 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:21:15.270 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 23.00 MB 2023-10-08T00:21:15.270 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:21:15.270 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:21:15.270 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 24.00 MB 2023-10-08T00:21:15.270 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:21:15.270 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:21:15.270 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 25.00 MB 2023-10-08T00:21:15.271 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:21:15.271 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:21:15.271 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 26.00 MB 2023-10-08T00:21:15.271 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:21:15.271 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:21:15.271 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 27.00 MB 2023-10-08T00:21:15.271 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:21:15.271 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:21:15.271 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 28.00 MB 2023-10-08T00:21:15.271 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:21:15.271 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:21:15.271 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 29.00 MB 2023-10-08T00:21:15.272 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:21:15.272 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:21:15.272 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 30.00 MB 2023-10-08T00:21:15.272 INFO:teuthology.orchestra.run.smithi099.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-08T00:21:15.272 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:21:15.272 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:21:15.272 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 31.00 MB 2023-10-08T00:21:15.272 INFO:teuthology.orchestra.run.smithi099.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-08T00:21:15.272 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:21:15.272 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:21:15.272 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 32.00 MB 2023-10-08T00:21:15.272 INFO:teuthology.orchestra.run.smithi099.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-08T00:21:15.273 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-08T00:21:15.273 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2023-10-08T00:21:15.273 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 33.00 MB 2023-10-08T00:21:15.273 INFO:teuthology.orchestra.run.smithi099.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-08T00:21:15.273 INFO:teuthology.orchestra.run.smithi099.stdout:Reached fullness after 33.00 MB 2023-10-08T00:21:15.273 INFO:teuthology.orchestra.run.smithi099.stdout:calling close 2023-10-08T00:21:15.273 INFO:teuthology.orchestra.run.smithi099.stdout:close() did not raise error 2023-10-08T00:21:15.285 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T00:21:15.286 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T00:21:15.331 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T00:21:15.376 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T00:21:15.377 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T00:21:15.377 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:21:15.377 DEBUG:teuthology.orchestra.run.smithi099:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-08T00:21:15.424 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2023-10-08T00:21:15.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:21:15.425 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T00:21:15.440 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-08T00:21:15.440 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:21:15.441 DEBUG:teuthology.orchestra.run.smithi191:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-08T00:21:15.488 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2023-10-08T00:21:15.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:21:15.489 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-08T00:21:15.505 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:21:15.622 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.622+0000 7f481e021700 1 Processor -- start 2023-10-08T00:21:15.622 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.622+0000 7f481e021700 1 -- start start 2023-10-08T00:21:15.624 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.622+0000 7f481e021700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4818157870 0x7f481815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:15.625 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.622+0000 7f481e021700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4818157c40 0x7f4818158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:15.625 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.622+0000 7f481e021700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f481815cbb0 con 0x7f481815a620 2023-10-08T00:21:15.625 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.622+0000 7f481e021700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f481815bfa0 con 0x7f4818157870 2023-10-08T00:21:15.625 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.622+0000 7f481e021700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f481815c280 con 0x7f4818157c40 2023-10-08T00:21:15.625 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.623+0000 7f48177fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4818157870 0x7f481815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:15.625 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.623+0000 7f4816ffd700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f481815a620 0x7f481815d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:46690/0 (socket says 172.21.15.99:46690) 2023-10-08T00:21:15.625 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.623+0000 7f4816ffd700 1 -- 172.21.15.99:0/1204091822 learned_addr learned my addr 172.21.15.99:0/1204091822 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:21:15.625 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.623+0000 7f48177fe700 1 --2- 172.21.15.99:0/1204091822 >> v2:172.21.15.99:3300/0 conn(0x7f4818157870 0x7f481815b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57282/0 (socket says 172.21.15.99:57282) 2023-10-08T00:21:15.625 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.623+0000 7f48177fe700 1 -- 172.21.15.99:0/1204091822 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4818157c40 msgr2=0x7f4818158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:15.625 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.623+0000 7f48177fe700 1 --2- 172.21.15.99:0/1204091822 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4818157c40 0x7f4818158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:15.625 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.623+0000 7f48177fe700 1 -- 172.21.15.99:0/1204091822 >> v1:172.21.15.99:6789/0 conn(0x7f481815a620 legacy=0x7f481815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:15.626 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.623+0000 7f48177fe700 1 -- 172.21.15.99:0/1204091822 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f481815c560 con 0x7f4818157870 2023-10-08T00:21:15.626 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.623+0000 7f48167fc700 1 -- 172.21.15.99:0/1204091822 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2406919064 0 0) 0x7f481815cbb0 con 0x7f481815a620 2023-10-08T00:21:15.626 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.623+0000 7f48177fe700 1 --2- 172.21.15.99:0/1204091822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4818157870 0x7f481815b6a0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f4808009e60 tx=0x7f4808009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1833dd0189160169 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:15.626 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.623+0000 7f48167fc700 1 -- 172.21.15.99:0/1204091822 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480800e070 con 0x7f4818157870 2023-10-08T00:21:15.626 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.623+0000 7f48167fc700 1 -- 172.21.15.99:0/1204091822 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f480800bc30 con 0x7f4818157870 2023-10-08T00:21:15.626 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.623+0000 7f48167fc700 1 -- 172.21.15.99:0/1204091822 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4808012430 con 0x7f4818157870 2023-10-08T00:21:15.626 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.624+0000 7f481e021700 1 -- 172.21.15.99:0/1204091822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4818157870 msgr2=0x7f481815b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:15.626 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.624+0000 7f481e021700 1 --2- 172.21.15.99:0/1204091822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4818157870 0x7f481815b6a0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f4808009e60 tx=0x7f4808009780 comp rx=0 tx=0).stop 2023-10-08T00:21:15.626 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.624+0000 7f481e021700 1 -- 172.21.15.99:0/1204091822 shutdown_connections 2023-10-08T00:21:15.626 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.624+0000 7f481e021700 1 --2- 172.21.15.99:0/1204091822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4818157870 0x7f481815b6a0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:15.626 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.624+0000 7f481e021700 1 --2- 172.21.15.99:0/1204091822 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4818157c40 0x7f4818158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:15.626 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.624+0000 7f481e021700 1 -- 172.21.15.99:0/1204091822 >> 172.21.15.99:0/1204091822 conn(0x7f4818068ce0 msgr2=0x7f48180690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:15.627 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.624+0000 7f481e021700 1 -- 172.21.15.99:0/1204091822 shutdown_connections 2023-10-08T00:21:15.627 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.624+0000 7f481e021700 1 -- 172.21.15.99:0/1204091822 wait complete. 2023-10-08T00:21:15.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.625+0000 7f481e021700 1 Processor -- start 2023-10-08T00:21:15.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.625+0000 7f481e021700 1 -- start start 2023-10-08T00:21:15.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.626+0000 7f481e021700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4818157870 0x7f4818152230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:15.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.626+0000 7f481e021700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4818157c40 0x7f48180b2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:15.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.626+0000 7f481e021700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f481815a620 0x7f4818150700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:15.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.626+0000 7f481e021700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481814d630 con 0x7f4818157870 2023-10-08T00:21:15.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.626+0000 7f481e021700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4818152810 con 0x7f481815a620 2023-10-08T00:21:15.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.626+0000 7f481e021700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4818152af0 con 0x7f4818157c40 2023-10-08T00:21:15.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.626+0000 7f48177fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4818157870 0x7f4818152230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:15.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.626+0000 7f48177fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4818157870 0x7f4818152230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57294/0 (socket says 172.21.15.99:57294) 2023-10-08T00:21:15.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.626+0000 7f48177fe700 1 -- 172.21.15.99:0/3289925478 learned_addr learned my addr 172.21.15.99:0/3289925478 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:15.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.626+0000 7f4816ffd700 1 --2- 172.21.15.99:0/3289925478 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4818157c40 0x7f48180b2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:15.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.626+0000 7f48177fe700 1 -- 172.21.15.99:0/3289925478 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4818157c40 msgr2=0x7f48180b2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:15.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.626+0000 7f4817fff700 1 --2- 172.21.15.99:0/3289925478 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f481815a620 0x7f4818150700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:15.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.626+0000 7f48177fe700 1 --2- 172.21.15.99:0/3289925478 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4818157c40 0x7f48180b2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:15.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.626+0000 7f48177fe700 1 -- 172.21.15.99:0/3289925478 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f481815a620 msgr2=0x7f4818150700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:15.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.626+0000 7f48177fe700 1 --2- 172.21.15.99:0/3289925478 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f481815a620 0x7f4818150700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:15.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.626+0000 7f48177fe700 1 -- 172.21.15.99:0/3289925478 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4808009a20 con 0x7f4818157870 2023-10-08T00:21:15.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.626+0000 7f4816ffd700 1 --2- 172.21.15.99:0/3289925478 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4818157c40 0x7f48180b2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:21:15.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.626+0000 7f4817fff700 1 --2- 172.21.15.99:0/3289925478 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f481815a620 0x7f4818150700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:21:15.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.626+0000 7f48177fe700 1 --2- 172.21.15.99:0/3289925478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4818157870 0x7f4818152230 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f48080097c0 tx=0x7f4808019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:15.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.627+0000 7f47ff7fe700 1 -- 172.21.15.99:0/3289925478 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4808027470 con 0x7f4818157870 2023-10-08T00:21:15.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.627+0000 7f47ff7fe700 1 -- 172.21.15.99:0/3289925478 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48080275d0 con 0x7f4818157870 2023-10-08T00:21:15.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.627+0000 7f481e021700 1 -- 172.21.15.99:0/3289925478 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4818152dd0 con 0x7f4818157870 2023-10-08T00:21:15.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.627+0000 7f47ff7fe700 1 -- 172.21.15.99:0/3289925478 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4808027970 con 0x7f4818157870 2023-10-08T00:21:15.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.627+0000 7f481e021700 1 -- 172.21.15.99:0/3289925478 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4818153160 con 0x7f4818157870 2023-10-08T00:21:15.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.628+0000 7f481e021700 1 -- 172.21.15.99:0/3289925478 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48180b3580 con 0x7f4818157870 2023-10-08T00:21:15.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.628+0000 7f47ff7fe700 1 -- 172.21.15.99:0/3289925478 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4808024070 con 0x7f4818157870 2023-10-08T00:21:15.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.628+0000 7f47ff7fe700 1 --2- 172.21.15.99:0/3289925478 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4804066f30 0x7f48040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:15.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.628+0000 7f47ff7fe700 1 -- 172.21.15.99:0/3289925478 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f48080870e0 con 0x7f4818157870 2023-10-08T00:21:15.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.628+0000 7f47ff7fe700 1 -- 172.21.15.99:0/3289925478 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=173}) v3 -- 0x7f480406eb10 con 0x7f4818157870 2023-10-08T00:21:15.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.628+0000 7f4816ffd700 1 --2- 172.21.15.99:0/3289925478 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4804066f30 0x7f48040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:15.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.629+0000 7f4816ffd700 1 --2- 172.21.15.99:0/3289925478 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4804066f30 0x7f48040693e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f48180afa70 tx=0x7f480c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:15.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.633+0000 7f47ff7fe700 1 -- 172.21.15.99:0/3289925478 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4808054310 con 0x7f4818157870 2023-10-08T00:21:15.857 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.857+0000 7f481e021700 1 -- 172.21.15.99:0/3289925478 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f481804c3c0 con 0x7f4818157870 2023-10-08T00:21:15.858 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.857+0000 7f47ff7fe700 1 -- 172.21.15.99:0/3289925478 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4601 (secure 0 0 0) 0x7f4808057930 con 0x7f4818157870 2023-10-08T00:21:15.858 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:21:15.858 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":77,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6257,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/661016677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":661016677},{"type":"v1","addr":"172.21.15.191:6837","nonce":661016677}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":7012,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/4102051559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":4102051559},{"type":"v1","addr":"172.21.15.99:6835","nonce":4102051559}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":7021,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/658854389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":658854389},{"type":"v1","addr":"172.21.15.191:6836","nonce":658854389}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:20:04.613994+0000","modified":"2023-10-08T00:20:08.255350+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7791},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7791":{"gid":7791,"name":"c","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6834/3880334948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3880334948},{"type":"v1","addr":"172.21.15.99:6834","nonce":3880334948}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-08T00:21:15.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.859+0000 7f481e021700 1 -- 172.21.15.99:0/3289925478 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4804066f30 msgr2=0x7f48040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:15.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.859+0000 7f481e021700 1 --2- 172.21.15.99:0/3289925478 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4804066f30 0x7f48040693e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f48180afa70 tx=0x7f480c009040 comp rx=0 tx=0).stop 2023-10-08T00:21:15.861 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.859+0000 7f481e021700 1 -- 172.21.15.99:0/3289925478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4818157870 msgr2=0x7f4818152230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:15.861 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.859+0000 7f481e021700 1 --2- 172.21.15.99:0/3289925478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4818157870 0x7f4818152230 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f48080097c0 tx=0x7f4808019000 comp rx=0 tx=0).stop 2023-10-08T00:21:15.861 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.859+0000 7f481e021700 1 -- 172.21.15.99:0/3289925478 shutdown_connections 2023-10-08T00:21:15.861 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.859+0000 7f481e021700 1 --2- 172.21.15.99:0/3289925478 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4804066f30 0x7f48040693e0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:15.861 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.859+0000 7f481e021700 1 --2- 172.21.15.99:0/3289925478 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f481815a620 0x7f4818150700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:15.861 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.860+0000 7f481e021700 1 --2- 172.21.15.99:0/3289925478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4818157870 0x7f4818152230 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:15.861 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.860+0000 7f481e021700 1 --2- 172.21.15.99:0/3289925478 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4818157c40 0x7f48180b2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:15.861 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.860+0000 7f481e021700 1 -- 172.21.15.99:0/3289925478 >> 172.21.15.99:0/3289925478 conn(0x7f4818068ce0 msgr2=0x7f48180b47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:15.861 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.860+0000 7f481e021700 1 -- 172.21.15.99:0/3289925478 shutdown_connections 2023-10-08T00:21:15.861 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.860+0000 7f481e021700 1 -- 172.21.15.99:0/3289925478 wait complete. 2023-10-08T00:21:15.862 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 77 2023-10-08T00:21:15.872 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:21:15.985 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.985+0000 7f138cf32700 1 Processor -- start 2023-10-08T00:21:15.985 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.985+0000 7f138cf32700 1 -- start start 2023-10-08T00:21:15.988 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.986+0000 7f138cf32700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f138815a2b0 0x7f13880b3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:15.988 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.986+0000 7f138cf32700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13880c5d80 con 0x7f13880d1c30 2023-10-08T00:21:15.988 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.986+0000 7f138cf32700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13880c6060 con 0x7f13880cfe20 2023-10-08T00:21:15.988 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.986+0000 7f138cf32700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1388068470 con 0x7f138815a2b0 2023-10-08T00:21:15.988 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.986+0000 7f138659c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f138815a2b0 0x7f13880b3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:15.988 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.986+0000 7f1385d9b700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f13880d1c30 0x7f13880cf710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:46702/0 (socket says 172.21.15.99:46702) 2023-10-08T00:21:15.988 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.986+0000 7f1385d9b700 1 -- 172.21.15.99:0/97649561 learned_addr learned my addr 172.21.15.99:0/97649561 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:21:15.988 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.986+0000 7f138659c700 1 -- 172.21.15.99:0/97649561 >> v1:172.21.15.191:6789/0 conn(0x7f13880cfe20 legacy=0x7f13880c7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:15.988 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.986+0000 7f138659c700 1 -- 172.21.15.99:0/97649561 >> v1:172.21.15.99:6789/0 conn(0x7f13880d1c30 legacy=0x7f13880cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:15.988 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.986+0000 7f138659c700 1 -- 172.21.15.99:0/97649561 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13880c6340 con 0x7f138815a2b0 2023-10-08T00:21:15.989 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.986+0000 7f138559a700 1 -- 172.21.15.99:0/97649561 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3023231269 0 0) 0x7f13880c5d80 con 0x7f13880d1c30 2023-10-08T00:21:15.989 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.986+0000 7f138659c700 1 --2- 172.21.15.99:0/97649561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f138815a2b0 0x7f13880b3da0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f13780098d0 tx=0x7f1378009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b721451ff6002449 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:15.989 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.986+0000 7f138559a700 1 -- 172.21.15.99:0/97649561 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1378009e70 con 0x7f138815a2b0 2023-10-08T00:21:15.989 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.986+0000 7f138559a700 1 -- 172.21.15.99:0/97649561 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1378005c10 con 0x7f138815a2b0 2023-10-08T00:21:15.990 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.986+0000 7f138559a700 1 -- 172.21.15.99:0/97649561 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137800b650 con 0x7f138815a2b0 2023-10-08T00:21:15.990 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.987+0000 7f138cf32700 1 -- 172.21.15.99:0/97649561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f138815a2b0 msgr2=0x7f13880b3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:15.990 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.987+0000 7f138cf32700 1 --2- 172.21.15.99:0/97649561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f138815a2b0 0x7f13880b3da0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f13780098d0 tx=0x7f1378009120 comp rx=0 tx=0).stop 2023-10-08T00:21:15.990 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.987+0000 7f138cf32700 1 -- 172.21.15.99:0/97649561 shutdown_connections 2023-10-08T00:21:15.990 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.987+0000 7f138cf32700 1 --2- 172.21.15.99:0/97649561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f138815a2b0 0x7f13880b3da0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:15.990 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.987+0000 7f138cf32700 1 -- 172.21.15.99:0/97649561 >> 172.21.15.99:0/97649561 conn(0x7f1388067e40 msgr2=0x7f13880d0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:15.990 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.987+0000 7f138cf32700 1 -- 172.21.15.99:0/97649561 shutdown_connections 2023-10-08T00:21:15.990 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.987+0000 7f138cf32700 1 -- 172.21.15.99:0/97649561 wait complete. 2023-10-08T00:21:15.993 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.988+0000 7f138cf32700 1 Processor -- start 2023-10-08T00:21:15.994 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.988+0000 7f138cf32700 1 -- start start 2023-10-08T00:21:15.994 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.989+0000 7f138cf32700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13880cfe20 0x7f138815e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:15.994 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.989+0000 7f138cf32700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13880d1c30 0x7f13880c8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:15.994 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.989+0000 7f138cf32700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f138815a2b0 0x7f1388153ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:15.994 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.989+0000 7f138cf32700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1388154420 con 0x7f13880cfe20 2023-10-08T00:21:15.994 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.989+0000 7f138cf32700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13880c8a10 con 0x7f138815a2b0 2023-10-08T00:21:15.994 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.989+0000 7f138cf32700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13880c8cf0 con 0x7f13880d1c30 2023-10-08T00:21:15.994 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.989+0000 7f138659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13880cfe20 0x7f138815e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:15.994 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.989+0000 7f138659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13880cfe20 0x7f138815e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57322/0 (socket says 172.21.15.99:57322) 2023-10-08T00:21:15.994 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.989+0000 7f138659c700 1 -- 172.21.15.99:0/956822797 learned_addr learned my addr 172.21.15.99:0/956822797 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:15.994 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.989+0000 7f1386d9d700 1 --2- 172.21.15.99:0/956822797 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f138815a2b0 0x7f1388153ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:15.994 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.989+0000 7f1385d9b700 1 --2- 172.21.15.99:0/956822797 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13880d1c30 0x7f13880c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:15.995 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.989+0000 7f138659c700 1 -- 172.21.15.99:0/956822797 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13880d1c30 msgr2=0x7f13880c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:15.995 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.989+0000 7f138659c700 1 --2- 172.21.15.99:0/956822797 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13880d1c30 0x7f13880c8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:15.995 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.989+0000 7f138659c700 1 -- 172.21.15.99:0/956822797 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f138815a2b0 msgr2=0x7f1388153ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:15.995 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.989+0000 7f138659c700 1 --2- 172.21.15.99:0/956822797 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f138815a2b0 0x7f1388153ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:15.995 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.989+0000 7f138659c700 1 -- 172.21.15.99:0/956822797 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13780093c0 con 0x7f13880cfe20 2023-10-08T00:21:15.995 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.989+0000 7f1385d9b700 1 --2- 172.21.15.99:0/956822797 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13880d1c30 0x7f13880c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:21:15.995 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.989+0000 7f138659c700 1 --2- 172.21.15.99:0/956822797 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13880cfe20 0x7f138815e020 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f1378009390 tx=0x7f1378016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:15.995 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.990+0000 7f13767fc700 1 -- 172.21.15.99:0/956822797 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13780064a0 con 0x7f13880cfe20 2023-10-08T00:21:15.995 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.990+0000 7f13767fc700 1 -- 172.21.15.99:0/956822797 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1378006620 con 0x7f13880cfe20 2023-10-08T00:21:15.995 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.990+0000 7f138cf32700 1 -- 172.21.15.99:0/956822797 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13880c8fd0 con 0x7f13880cfe20 2023-10-08T00:21:15.995 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.990+0000 7f13767fc700 1 -- 172.21.15.99:0/956822797 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137801b480 con 0x7f13880cfe20 2023-10-08T00:21:15.995 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.990+0000 7f138cf32700 1 -- 172.21.15.99:0/956822797 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13880c9590 con 0x7f13880cfe20 2023-10-08T00:21:15.996 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.991+0000 7f138cf32700 1 -- 172.21.15.99:0/956822797 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f138804c3c0 con 0x7f13880cfe20 2023-10-08T00:21:15.996 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.991+0000 7f13767fc700 1 -- 172.21.15.99:0/956822797 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f137801b7f0 con 0x7f13880cfe20 2023-10-08T00:21:15.996 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.991+0000 7f13767fc700 1 --2- 172.21.15.99:0/956822797 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1368066f30 0x7f13680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:15.996 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.991+0000 7f13767fc700 1 -- 172.21.15.99:0/956822797 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f13780867a0 con 0x7f13880cfe20 2023-10-08T00:21:15.996 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.992+0000 7f13767fc700 1 -- 172.21.15.99:0/956822797 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=173}) v3 -- 0x7f136806eb10 con 0x7f13880cfe20 2023-10-08T00:21:15.996 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.992+0000 7f1385d9b700 1 --2- 172.21.15.99:0/956822797 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1368066f30 0x7f13680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:15.996 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.992+0000 7f1385d9b700 1 --2- 172.21.15.99:0/956822797 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1368066f30 0x7f13680693e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f13880cae70 tx=0x7f137c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:15.996 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:15.996+0000 7f13767fc700 1 -- 172.21.15.99:0/956822797 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f13780539d0 con 0x7f13880cfe20 2023-10-08T00:21:16.178 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.178+0000 7f138cf32700 1 -- 172.21.15.99:0/956822797 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f13880b2ed0 con 0x7f13880cfe20 2023-10-08T00:21:16.179 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.179+0000 7f13767fc700 1 -- 172.21.15.99:0/956822797 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v172) v1 ==== 74+0+15488 (secure 0 0 0) 0x7f1378056ff0 con 0x7f13880cfe20 2023-10-08T00:21:16.179 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:21:16.179 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":172,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:20:51.158163+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:20:02.932463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-08T00:20:03.578010+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.6","mappings":[{"from":2,"to":5}]},{"pgid":"21.10","mappings":[{"from":2,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:21:16.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.181+0000 7f138cf32700 1 -- 172.21.15.99:0/956822797 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1368066f30 msgr2=0x7f13680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:16.182 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.181+0000 7f138cf32700 1 --2- 172.21.15.99:0/956822797 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1368066f30 0x7f13680693e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f13880cae70 tx=0x7f137c009040 comp rx=0 tx=0).stop 2023-10-08T00:21:16.182 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.181+0000 7f138cf32700 1 -- 172.21.15.99:0/956822797 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13880cfe20 msgr2=0x7f138815e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:16.182 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.181+0000 7f138cf32700 1 --2- 172.21.15.99:0/956822797 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13880cfe20 0x7f138815e020 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f1378009390 tx=0x7f1378016000 comp rx=0 tx=0).stop 2023-10-08T00:21:16.182 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.181+0000 7f138cf32700 1 -- 172.21.15.99:0/956822797 shutdown_connections 2023-10-08T00:21:16.182 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.181+0000 7f138cf32700 1 --2- 172.21.15.99:0/956822797 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f1368066f30 0x7f13680693e0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:16.182 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.181+0000 7f138cf32700 1 --2- 172.21.15.99:0/956822797 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f138815a2b0 0x7f1388153ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:16.182 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.181+0000 7f138cf32700 1 --2- 172.21.15.99:0/956822797 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13880cfe20 0x7f138815e020 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:16.182 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.181+0000 7f138cf32700 1 --2- 172.21.15.99:0/956822797 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13880d1c30 0x7f13880c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:16.182 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.181+0000 7f138cf32700 1 -- 172.21.15.99:0/956822797 >> 172.21.15.99:0/956822797 conn(0x7f1388067e40 msgr2=0x7f13880b3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:16.182 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.181+0000 7f138cf32700 1 -- 172.21.15.99:0/956822797 shutdown_connections 2023-10-08T00:21:16.183 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.181+0000 7f138cf32700 1 -- 172.21.15.99:0/956822797 wait complete. 2023-10-08T00:21:16.193 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:10:42.063344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T00:20:02.932463+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '167', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T00:20:03.578010+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '172', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T00:21:16.194 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T00:21:16.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.315+0000 7fd9ce5aa700 1 Processor -- start 2023-10-08T00:21:16.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.315+0000 7fd9ce5aa700 1 -- start start 2023-10-08T00:21:16.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.316+0000 7fd9ce5aa700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9c8156b90 0x7fd9c8157b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:16.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.316+0000 7fd9ce5aa700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd9c806c380 0x7fd9c806c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:16.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.316+0000 7fd9ce5aa700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9c806cd30 con 0x7fd9c815a740 2023-10-08T00:21:16.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.316+0000 7fd9ce5aa700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9c806f330 con 0x7fd9c806c380 2023-10-08T00:21:16.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.316+0000 7fd9ce5aa700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c81580d0 con 0x7fd9c8156b90 2023-10-08T00:21:16.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.316+0000 7fd9c7fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9c8156b90 0x7fd9c8157b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:16.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.316+0000 7fd9c77fe700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd9c806c380 0x7fd9c806c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:16.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.316+0000 7fd9c7fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9c8156b90 0x7fd9c8157b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.99:34458/0 (socket says 172.21.15.99:34458) 2023-10-08T00:21:16.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.317+0000 7fd9c77fe700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd9c806c380 0x7fd9c806c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.99:36358/0 (socket says 172.21.15.99:36358) 2023-10-08T00:21:16.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.317+0000 7fd9c7fff700 1 -- 172.21.15.99:0/3966837198 learned_addr learned my addr 172.21.15.99:0/3966837198 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:16.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.317+0000 7fd9c77fe700 1 -- 172.21.15.99:0/3966837198 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9c8156b90 msgr2=0x7fd9c8157b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:16.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.317+0000 7fd9c77fe700 1 --2- 172.21.15.99:0/3966837198 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9c8156b90 0x7fd9c8157b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:16.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.317+0000 7fd9c77fe700 1 -- 172.21.15.99:0/3966837198 >> v1:172.21.15.191:6789/0 conn(0x7fd9c815a740 legacy=0x7fd9c815ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:16.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.317+0000 7fd9c77fe700 1 -- 172.21.15.99:0/3966837198 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c806e720 con 0x7fd9c806c380 2023-10-08T00:21:16.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.317+0000 7fd9c7fff700 1 --2- 172.21.15.99:0/3966837198 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9c8156b90 0x7fd9c8157b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:21:16.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.317+0000 7fd9c77fe700 1 --2- 172.21.15.99:0/3966837198 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9c806c380 0x7fd9c806c7f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc0098e0 tx=0x7fd9bc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ec3b3798969b07a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:16.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.317+0000 7fd9c6ffd700 1 -- 172.21.15.99:0/3966837198 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc00b640 con 0x7fd9c806c380 2023-10-08T00:21:16.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.317+0000 7fd9c6ffd700 1 -- 172.21.15.99:0/3966837198 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9bc00b7a0 con 0x7fd9c806c380 2023-10-08T00:21:16.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.317+0000 7fd9c6ffd700 1 -- 172.21.15.99:0/3966837198 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc00bb40 con 0x7fd9c806c380 2023-10-08T00:21:16.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.318+0000 7fd9ce5aa700 1 -- 172.21.15.99:0/3966837198 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9c806c380 msgr2=0x7fd9c806c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:16.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.318+0000 7fd9ce5aa700 1 --2- 172.21.15.99:0/3966837198 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9c806c380 0x7fd9c806c7f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc0098e0 tx=0x7fd9bc0091a0 comp rx=0 tx=0).stop 2023-10-08T00:21:16.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.318+0000 7fd9ce5aa700 1 -- 172.21.15.99:0/3966837198 shutdown_connections 2023-10-08T00:21:16.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.318+0000 7fd9ce5aa700 1 --2- 172.21.15.99:0/3966837198 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9c8156b90 0x7fd9c8157b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:16.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.318+0000 7fd9ce5aa700 1 --2- 172.21.15.99:0/3966837198 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9c806c380 0x7fd9c806c7f0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:16.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.318+0000 7fd9ce5aa700 1 -- 172.21.15.99:0/3966837198 >> 172.21.15.99:0/3966837198 conn(0x7fd9c80cb030 msgr2=0x7fd9c806bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:16.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.318+0000 7fd9ce5aa700 1 -- 172.21.15.99:0/3966837198 shutdown_connections 2023-10-08T00:21:16.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.318+0000 7fd9ce5aa700 1 -- 172.21.15.99:0/3966837198 wait complete. 2023-10-08T00:21:16.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.319+0000 7fd9ce5aa700 1 Processor -- start 2023-10-08T00:21:16.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.319+0000 7fd9ce5aa700 1 -- start start 2023-10-08T00:21:16.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.320+0000 7fd9ce5aa700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9c806c380 0x7fd9c8153e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:16.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.320+0000 7fd9ce5aa700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9c8156b90 0x7fd9c81672e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:16.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.320+0000 7fd9ce5aa700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9c815a740 0x7fd9c80c5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:16.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.320+0000 7fd9ce5aa700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c80b2770 con 0x7fd9c806c380 2023-10-08T00:21:16.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.320+0000 7fd9ce5aa700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c81543d0 con 0x7fd9c815a740 2023-10-08T00:21:16.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.320+0000 7fd9ce5aa700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c80c64a0 con 0x7fd9c8156b90 2023-10-08T00:21:16.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.320+0000 7fd9c7fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9c806c380 0x7fd9c8153e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:16.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.320+0000 7fd9c7fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9c806c380 0x7fd9c8153e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57332/0 (socket says 172.21.15.99:57332) 2023-10-08T00:21:16.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.320+0000 7fd9c7fff700 1 -- 172.21.15.99:0/2244528958 learned_addr learned my addr 172.21.15.99:0/2244528958 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:16.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.320+0000 7fd9ccb47700 1 --2- 172.21.15.99:0/2244528958 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9c815a740 0x7fd9c80c5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:16.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.320+0000 7fd9c77fe700 1 --2- 172.21.15.99:0/2244528958 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9c8156b90 0x7fd9c81672e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:16.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.320+0000 7fd9c7fff700 1 -- 172.21.15.99:0/2244528958 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9c8156b90 msgr2=0x7fd9c81672e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:16.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.320+0000 7fd9c7fff700 1 --2- 172.21.15.99:0/2244528958 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9c8156b90 0x7fd9c81672e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:16.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.320+0000 7fd9c7fff700 1 -- 172.21.15.99:0/2244528958 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9c815a740 msgr2=0x7fd9c80c5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:16.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.320+0000 7fd9c7fff700 1 --2- 172.21.15.99:0/2244528958 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9c815a740 0x7fd9c80c5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:16.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.320+0000 7fd9c7fff700 1 -- 172.21.15.99:0/2244528958 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b80094f0 con 0x7fd9c806c380 2023-10-08T00:21:16.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.320+0000 7fd9c77fe700 1 --2- 172.21.15.99:0/2244528958 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9c8156b90 0x7fd9c81672e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:21:16.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.320+0000 7fd9c7fff700 1 --2- 172.21.15.99:0/2244528958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9c806c380 0x7fd9c8153e90 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fd9b800ec50 tx=0x7fd9b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:16.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.321+0000 7fd9affff700 1 -- 172.21.15.99:0/2244528958 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b800c9d0 con 0x7fd9c806c380 2023-10-08T00:21:16.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.321+0000 7fd9affff700 1 -- 172.21.15.99:0/2244528958 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b800cb30 con 0x7fd9c806c380 2023-10-08T00:21:16.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.321+0000 7fd9ce5aa700 1 -- 172.21.15.99:0/2244528958 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9bc009420 con 0x7fd9c806c380 2023-10-08T00:21:16.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.321+0000 7fd9affff700 1 -- 172.21.15.99:0/2244528958 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b8013470 con 0x7fd9c806c380 2023-10-08T00:21:16.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.321+0000 7fd9ce5aa700 1 -- 172.21.15.99:0/2244528958 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9c80b2120 con 0x7fd9c806c380 2023-10-08T00:21:16.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.321+0000 7fd9ce5aa700 1 -- 172.21.15.99:0/2244528958 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9c80b2a50 con 0x7fd9c806c380 2023-10-08T00:21:16.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.322+0000 7fd9affff700 1 -- 172.21.15.99:0/2244528958 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd9b8013860 con 0x7fd9c806c380 2023-10-08T00:21:16.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.322+0000 7fd9affff700 1 --2- 172.21.15.99:0/2244528958 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd9b4066f30 0x7fd9b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:16.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.322+0000 7fd9affff700 1 -- 172.21.15.99:0/2244528958 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5763+0+0 (secure 0 0 0) 0x7fd9b8084db0 con 0x7fd9c806c380 2023-10-08T00:21:16.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.323+0000 7fd9c77fe700 1 --2- 172.21.15.99:0/2244528958 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd9b4066f30 0x7fd9b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:16.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.323+0000 7fd9affff700 1 -- 172.21.15.99:0/2244528958 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=173}) v3 -- 0x7fd9b406eb10 con 0x7fd9c806c380 2023-10-08T00:21:16.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.323+0000 7fd9c77fe700 1 --2- 172.21.15.99:0/2244528958 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd9b4066f30 0x7fd9b40693e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8159930 tx=0x7fd9bc017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:16.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.327+0000 7fd9affff700 1 -- 172.21.15.99:0/2244528958 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd9b8051fe0 con 0x7fd9c806c380 2023-10-08T00:21:16.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.510+0000 7fd9ce5aa700 1 -- 172.21.15.99:0/2244528958 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd9c804c3c0 con 0x7fd9c806c380 2023-10-08T00:21:16.511 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.510+0000 7fd9affff700 1 -- 172.21.15.99:0/2244528958 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v172) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd9b8021020 con 0x7fd9c806c380 2023-10-08T00:21:16.511 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2023-10-08T00:21:16.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.512+0000 7fd9ce5aa700 1 -- 172.21.15.99:0/2244528958 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd9b4066f30 msgr2=0x7fd9b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:16.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.512+0000 7fd9ce5aa700 1 --2- 172.21.15.99:0/2244528958 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd9b4066f30 0x7fd9b40693e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8159930 tx=0x7fd9bc017000 comp rx=0 tx=0).stop 2023-10-08T00:21:16.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.512+0000 7fd9ce5aa700 1 -- 172.21.15.99:0/2244528958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9c806c380 msgr2=0x7fd9c8153e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:16.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.512+0000 7fd9ce5aa700 1 --2- 172.21.15.99:0/2244528958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9c806c380 0x7fd9c8153e90 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fd9b800ec50 tx=0x7fd9b8009250 comp rx=0 tx=0).stop 2023-10-08T00:21:16.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.512+0000 7fd9ce5aa700 1 -- 172.21.15.99:0/2244528958 shutdown_connections 2023-10-08T00:21:16.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.512+0000 7fd9ce5aa700 1 --2- 172.21.15.99:0/2244528958 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd9b4066f30 0x7fd9b40693e0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:16.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.512+0000 7fd9ce5aa700 1 --2- 172.21.15.99:0/2244528958 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9c815a740 0x7fd9c80c5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:16.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.512+0000 7fd9ce5aa700 1 --2- 172.21.15.99:0/2244528958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9c806c380 0x7fd9c8153e90 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:16.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.512+0000 7fd9ce5aa700 1 --2- 172.21.15.99:0/2244528958 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9c8156b90 0x7fd9c81672e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:16.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.512+0000 7fd9ce5aa700 1 -- 172.21.15.99:0/2244528958 >> 172.21.15.99:0/2244528958 conn(0x7fd9c80cb030 msgr2=0x7fd9c806a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:16.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.512+0000 7fd9ce5aa700 1 -- 172.21.15.99:0/2244528958 shutdown_connections 2023-10-08T00:21:16.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.512+0000 7fd9ce5aa700 1 -- 172.21.15.99:0/2244528958 wait complete. 2023-10-08T00:21:16.524 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T00:21:16.642 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.641+0000 7f770759e700 1 Processor -- start 2023-10-08T00:21:16.642 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.641+0000 7f770759e700 1 -- start start 2023-10-08T00:21:16.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.642+0000 7f770759e700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f77080b6580 0x7f770806a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:16.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.642+0000 7f770759e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f770806a830 0x7f77080b5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:16.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.642+0000 7f770759e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f770806ac90 con 0x7f77080cce30 2023-10-08T00:21:16.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.642+0000 7f770759e700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77080b4d70 con 0x7f77080b6580 2023-10-08T00:21:16.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.642+0000 7f770759e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f770815b3b0 con 0x7f770806a830 2023-10-08T00:21:16.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.642+0000 7f76fffff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f77080b6580 0x7f770806a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:16.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.642+0000 7f76fffff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f77080b6580 0x7f770806a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57348/0 (socket says 172.21.15.99:57348) 2023-10-08T00:21:16.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.642+0000 7f76fffff700 1 -- 172.21.15.99:0/4027869375 learned_addr learned my addr 172.21.15.99:0/4027869375 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:16.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.642+0000 7f7706d9d700 1 --2- 172.21.15.99:0/4027869375 >> v2:172.21.15.191:3300/0 conn(0x7f770806a830 0x7f77080b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:16.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.642+0000 7f76fffff700 1 -- 172.21.15.99:0/4027869375 >> v2:172.21.15.191:3300/0 conn(0x7f770806a830 msgr2=0x7f77080b5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:16.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.642+0000 7f76fffff700 1 --2- 172.21.15.99:0/4027869375 >> v2:172.21.15.191:3300/0 conn(0x7f770806a830 0x7f77080b5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:16.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.642+0000 7f76fffff700 1 -- 172.21.15.99:0/4027869375 >> v1:172.21.15.191:6789/0 conn(0x7f77080cce30 legacy=0x7f770815aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:16.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.643+0000 7f76fffff700 1 -- 172.21.15.99:0/4027869375 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77080cd8d0 con 0x7f77080b6580 2023-10-08T00:21:16.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.643+0000 7f76fffff700 1 --2- 172.21.15.99:0/4027869375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77080b6580 0x7f770806a2f0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f77080b4ce0 tx=0x7f76f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b244fb3854814be2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:16.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.643+0000 7f7705d9b700 1 -- 172.21.15.99:0/4027869375 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f000b820 con 0x7f77080b6580 2023-10-08T00:21:16.645 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.643+0000 7f7705d9b700 1 -- 172.21.15.99:0/4027869375 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f000b980 con 0x7f77080b6580 2023-10-08T00:21:16.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.643+0000 7f7705d9b700 1 -- 172.21.15.99:0/4027869375 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f000bd20 con 0x7f77080b6580 2023-10-08T00:21:16.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.643+0000 7f770759e700 1 -- 172.21.15.99:0/4027869375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77080b6580 msgr2=0x7f770806a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:16.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.643+0000 7f770759e700 1 --2- 172.21.15.99:0/4027869375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77080b6580 0x7f770806a2f0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f77080b4ce0 tx=0x7f76f0009250 comp rx=0 tx=0).stop 2023-10-08T00:21:16.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.643+0000 7f770759e700 1 -- 172.21.15.99:0/4027869375 shutdown_connections 2023-10-08T00:21:16.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.643+0000 7f770759e700 1 --2- 172.21.15.99:0/4027869375 >> v2:172.21.15.191:3300/0 conn(0x7f770806a830 0x7f77080b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:16.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.643+0000 7f770759e700 1 --2- 172.21.15.99:0/4027869375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77080b6580 0x7f770806a2f0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:16.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.643+0000 7f770759e700 1 -- 172.21.15.99:0/4027869375 >> 172.21.15.99:0/4027869375 conn(0x7f7708068cb0 msgr2=0x7f77080690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:16.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.643+0000 7f770759e700 1 -- 172.21.15.99:0/4027869375 shutdown_connections 2023-10-08T00:21:16.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.643+0000 7f770759e700 1 -- 172.21.15.99:0/4027869375 wait complete. 2023-10-08T00:21:16.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.644+0000 7f770759e700 1 Processor -- start 2023-10-08T00:21:16.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.645+0000 7f770759e700 1 -- start start 2023-10-08T00:21:16.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.645+0000 7f770759e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f770806a830 0x7f770814d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:16.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.645+0000 7f770759e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77080b6580 0x7f77080b0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:16.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.645+0000 7f770759e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77080cce30 0x7f770816b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:16.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.645+0000 7f770759e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77080b09c0 con 0x7f77080b6580 2023-10-08T00:21:16.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.645+0000 7f770759e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7708169f30 con 0x7f770806a830 2023-10-08T00:21:16.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.645+0000 7f770759e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f770816a1e0 con 0x7f77080cce30 2023-10-08T00:21:16.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.645+0000 7f76fffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77080b6580 0x7f77080b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:16.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.645+0000 7f76fffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77080b6580 0x7f77080b0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57362/0 (socket says 172.21.15.99:57362) 2023-10-08T00:21:16.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.645+0000 7f76fffff700 1 -- 172.21.15.99:0/1169784970 learned_addr learned my addr 172.21.15.99:0/1169784970 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:16.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.645+0000 7f770659c700 1 --2- 172.21.15.99:0/1169784970 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f770806a830 0x7f770814d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:16.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.646+0000 7f7706d9d700 1 --2- 172.21.15.99:0/1169784970 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77080cce30 0x7f770816b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:16.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.646+0000 7f76fffff700 1 -- 172.21.15.99:0/1169784970 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77080cce30 msgr2=0x7f770816b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:16.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.646+0000 7f76fffff700 1 --2- 172.21.15.99:0/1169784970 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77080cce30 0x7f770816b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:16.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.646+0000 7f76fffff700 1 -- 172.21.15.99:0/1169784970 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f770806a830 msgr2=0x7f770814d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:16.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.646+0000 7f76fffff700 1 --2- 172.21.15.99:0/1169784970 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f770806a830 0x7f770814d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:16.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.646+0000 7f76fffff700 1 -- 172.21.15.99:0/1169784970 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76f00094f0 con 0x7f77080b6580 2023-10-08T00:21:16.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.646+0000 7f770659c700 1 --2- 172.21.15.99:0/1169784970 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f770806a830 0x7f770814d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:21:16.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.646+0000 7f76fffff700 1 --2- 172.21.15.99:0/1169784970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77080b6580 0x7f77080b0480 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f770806b300 tx=0x7f76f0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:16.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.646+0000 7f76fe7fc700 1 -- 172.21.15.99:0/1169784970 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f0011e20 con 0x7f77080b6580 2023-10-08T00:21:16.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.646+0000 7f76fe7fc700 1 -- 172.21.15.99:0/1169784970 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f001b680 con 0x7f77080b6580 2023-10-08T00:21:16.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.646+0000 7f770759e700 1 -- 172.21.15.99:0/1169784970 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77080b2790 con 0x7f77080b6580 2023-10-08T00:21:16.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.646+0000 7f76fe7fc700 1 -- 172.21.15.99:0/1169784970 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f001ba40 con 0x7f77080b6580 2023-10-08T00:21:16.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.646+0000 7f770759e700 1 -- 172.21.15.99:0/1169784970 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77080b0eb0 con 0x7f77080b6580 2023-10-08T00:21:16.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.647+0000 7f770759e700 1 -- 172.21.15.99:0/1169784970 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7708089550 con 0x7f77080b6580 2023-10-08T00:21:16.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.647+0000 7f76fe7fc700 1 -- 172.21.15.99:0/1169784970 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f76f001c450 con 0x7f77080b6580 2023-10-08T00:21:16.656 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.648+0000 7f76fe7fc700 1 --2- 172.21.15.99:0/1169784970 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f76e8066f30 0x7f76e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:16.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.648+0000 7f76fe7fc700 1 -- 172.21.15.99:0/1169784970 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f76f0086140 con 0x7f77080b6580 2023-10-08T00:21:16.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.648+0000 7f770659c700 1 --2- 172.21.15.99:0/1169784970 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f76e8066f30 0x7f76e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:16.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.648+0000 7f76fe7fc700 1 -- 172.21.15.99:0/1169784970 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=173}) v3 -- 0x7f76e806eb10 con 0x7f77080b6580 2023-10-08T00:21:16.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.648+0000 7f770659c700 1 --2- 172.21.15.99:0/1169784970 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f76e8066f30 0x7f76e80693e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f76f8006fd0 tx=0x7f76f8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:16.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.652+0000 7f76fe7fc700 1 -- 172.21.15.99:0/1169784970 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f76f0053370 con 0x7f77080b6580 2023-10-08T00:21:16.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.831+0000 7f770759e700 1 -- 172.21.15.99:0/1169784970 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f770806ac90 con 0x7f77080b6580 2023-10-08T00:21:16.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.831+0000 7f76fe7fc700 1 -- 172.21.15.99:0/1169784970 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v172) v1 ==== 104+0+11 (secure 0 0 0) 0x7f770806ac90 con 0x7f77080b6580 2023-10-08T00:21:16.832 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2023-10-08T00:21:16.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.833+0000 7f770759e700 1 -- 172.21.15.99:0/1169784970 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f76e8066f30 msgr2=0x7f76e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:16.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.833+0000 7f770759e700 1 --2- 172.21.15.99:0/1169784970 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f76e8066f30 0x7f76e80693e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f76f8006fd0 tx=0x7f76f8008040 comp rx=0 tx=0).stop 2023-10-08T00:21:16.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.833+0000 7f770759e700 1 -- 172.21.15.99:0/1169784970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77080b6580 msgr2=0x7f77080b0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:16.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.833+0000 7f770759e700 1 --2- 172.21.15.99:0/1169784970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77080b6580 0x7f77080b0480 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f770806b300 tx=0x7f76f0016000 comp rx=0 tx=0).stop 2023-10-08T00:21:16.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.833+0000 7f770759e700 1 -- 172.21.15.99:0/1169784970 shutdown_connections 2023-10-08T00:21:16.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.833+0000 7f770759e700 1 --2- 172.21.15.99:0/1169784970 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f76e8066f30 0x7f76e80693e0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:16.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.834+0000 7f770759e700 1 --2- 172.21.15.99:0/1169784970 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f770806a830 0x7f770814d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:16.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.834+0000 7f770759e700 1 --2- 172.21.15.99:0/1169784970 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77080cce30 0x7f770816b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:16.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.834+0000 7f770759e700 1 --2- 172.21.15.99:0/1169784970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77080b6580 0x7f77080b0480 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:16.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.834+0000 7f770759e700 1 -- 172.21.15.99:0/1169784970 >> 172.21.15.99:0/1169784970 conn(0x7f7708068cb0 msgr2=0x7f7708150590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:16.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.834+0000 7f770759e700 1 -- 172.21.15.99:0/1169784970 shutdown_connections 2023-10-08T00:21:16.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.834+0000 7f770759e700 1 -- 172.21.15.99:0/1169784970 wait complete. 2023-10-08T00:21:16.846 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T00:21:16.963 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.963+0000 7f7672fd6700 1 Processor -- start 2023-10-08T00:21:16.964 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.963+0000 7f7672fd6700 1 -- start start 2023-10-08T00:21:16.965 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.964+0000 7f7672fd6700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f766c0b6580 0x7f766c06a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:16.966 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.964+0000 7f7672fd6700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f766c06ac90 con 0x7f766c06a830 2023-10-08T00:21:16.966 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.964+0000 7f7672fd6700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f766c0b4d70 con 0x7f766c0cce30 2023-10-08T00:21:16.966 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.964+0000 7f7672fd6700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f766c15b3b0 con 0x7f766c0b6580 2023-10-08T00:21:16.966 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.964+0000 7f76727d5700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f766c06a830 0x7f766c0b5200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:46708/0 (socket says 172.21.15.99:46708) 2023-10-08T00:21:16.966 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.964+0000 7f76727d5700 1 -- 172.21.15.99:0/3951466315 learned_addr learned my addr 172.21.15.99:0/3951466315 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:21:16.966 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.964+0000 7f76717d3700 1 --2- 172.21.15.99:0/3951466315 >> v2:172.21.15.191:3300/0 conn(0x7f766c0b6580 0x7f766c06a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:16.966 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.964+0000 7f7670fd2700 1 -- 172.21.15.99:0/3951466315 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2857767415 0 0) 0x7f7664001490 con 0x7f766c06a830 2023-10-08T00:21:16.966 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.964+0000 7f7670fd2700 1 -- 172.21.15.99:0/3951466315 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7658003570 con 0x7f766c06a830 2023-10-08T00:21:16.966 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.964+0000 7f76717d3700 1 -- 172.21.15.99:0/3951466315 >> v1:172.21.15.191:6789/0 conn(0x7f766c0cce30 legacy=0x7f766c15aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:16.966 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.964+0000 7f76717d3700 1 -- 172.21.15.99:0/3951466315 >> v1:172.21.15.99:6789/0 conn(0x7f766c06a830 legacy=0x7f766c0b5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:16.966 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.964+0000 7f76717d3700 1 -- 172.21.15.99:0/3951466315 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f766c0cd8d0 con 0x7f766c0b6580 2023-10-08T00:21:16.967 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.965+0000 7f76717d3700 1 --2- 172.21.15.99:0/3951466315 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f766c0b6580 0x7f766c06a2f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f765c0098e0 tx=0x7f765c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26d0cf86128e39d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:16.967 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.965+0000 7f7670fd2700 1 -- 172.21.15.99:0/3951466315 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765c00b640 con 0x7f766c0b6580 2023-10-08T00:21:16.967 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.965+0000 7f7670fd2700 1 -- 172.21.15.99:0/3951466315 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f765c00b7a0 con 0x7f766c0b6580 2023-10-08T00:21:16.967 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.965+0000 7f7670fd2700 1 -- 172.21.15.99:0/3951466315 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765c00bb40 con 0x7f766c0b6580 2023-10-08T00:21:16.970 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.965+0000 7f7672fd6700 1 -- 172.21.15.99:0/3951466315 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f766c0b6580 msgr2=0x7f766c06a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:16.971 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.965+0000 7f7672fd6700 1 --2- 172.21.15.99:0/3951466315 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f766c0b6580 0x7f766c06a2f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f765c0098e0 tx=0x7f765c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:21:16.971 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.965+0000 7f7672fd6700 1 -- 172.21.15.99:0/3951466315 shutdown_connections 2023-10-08T00:21:16.971 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.965+0000 7f7672fd6700 1 --2- 172.21.15.99:0/3951466315 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f766c0b6580 0x7f766c06a2f0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:16.971 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.965+0000 7f7672fd6700 1 -- 172.21.15.99:0/3951466315 >> 172.21.15.99:0/3951466315 conn(0x7f766c068cb0 msgr2=0x7f766c0690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:16.971 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.965+0000 7f7672fd6700 1 -- 172.21.15.99:0/3951466315 shutdown_connections 2023-10-08T00:21:16.971 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.965+0000 7f7672fd6700 1 -- 172.21.15.99:0/3951466315 wait complete. 2023-10-08T00:21:16.971 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.966+0000 7f7672fd6700 1 Processor -- start 2023-10-08T00:21:16.971 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.967+0000 7f7672fd6700 1 -- start start 2023-10-08T00:21:16.971 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.967+0000 7f7672fd6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f766c06a830 0x7f766c169e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:16.971 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.967+0000 7f7672fd6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f766c0b6580 0x7f766c0cf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:16.971 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.967+0000 7f7672fd6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f766c0cce30 0x7f766c153920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:16.971 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.967+0000 7f7672fd6700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766c16a470 con 0x7f766c06a830 2023-10-08T00:21:16.972 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.967+0000 7f7672fd6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766c16a750 con 0x7f766c0cce30 2023-10-08T00:21:16.972 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.967+0000 7f7672fd6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f766c16aa30 con 0x7f766c0b6580 2023-10-08T00:21:16.972 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.967+0000 7f7671fd4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f766c06a830 0x7f766c169e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:16.972 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.967+0000 7f7671fd4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f766c06a830 0x7f766c169e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57376/0 (socket says 172.21.15.99:57376) 2023-10-08T00:21:16.972 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.967+0000 7f7671fd4700 1 -- 172.21.15.99:0/1716179609 learned_addr learned my addr 172.21.15.99:0/1716179609 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:16.972 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.968+0000 7f76727d5700 1 --2- 172.21.15.99:0/1716179609 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f766c0cce30 0x7f766c153920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:16.972 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.968+0000 7f76717d3700 1 --2- 172.21.15.99:0/1716179609 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f766c0b6580 0x7f766c0cf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:16.972 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.968+0000 7f7671fd4700 1 -- 172.21.15.99:0/1716179609 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f766c0b6580 msgr2=0x7f766c0cf6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:16.972 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.968+0000 7f7671fd4700 1 --2- 172.21.15.99:0/1716179609 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f766c0b6580 0x7f766c0cf6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:16.973 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.968+0000 7f7671fd4700 1 -- 172.21.15.99:0/1716179609 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f766c0cce30 msgr2=0x7f766c153920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:16.973 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.968+0000 7f7671fd4700 1 --2- 172.21.15.99:0/1716179609 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f766c0cce30 0x7f766c153920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:16.973 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.968+0000 7f7671fd4700 1 -- 172.21.15.99:0/1716179609 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f765c009420 con 0x7f766c06a830 2023-10-08T00:21:16.973 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.968+0000 7f76727d5700 1 --2- 172.21.15.99:0/1716179609 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f766c0cce30 0x7f766c153920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:21:16.973 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.968+0000 7f7671fd4700 1 --2- 172.21.15.99:0/1716179609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f766c06a830 0x7f766c169e20 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f766800c4e0 tx=0x7f766800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:16.973 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.968+0000 7f7661ffb700 1 -- 172.21.15.99:0/1716179609 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7668011590 con 0x7f766c06a830 2023-10-08T00:21:16.973 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.968+0000 7f7661ffb700 1 -- 172.21.15.99:0/1716179609 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7668011710 con 0x7f766c06a830 2023-10-08T00:21:16.973 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.968+0000 7f7672fd6700 1 -- 172.21.15.99:0/1716179609 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f766c16ad10 con 0x7f766c06a830 2023-10-08T00:21:16.973 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.968+0000 7f7661ffb700 1 -- 172.21.15.99:0/1716179609 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7668011ad0 con 0x7f766c06a830 2023-10-08T00:21:16.973 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.968+0000 7f7672fd6700 1 -- 172.21.15.99:0/1716179609 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f766c0cfc20 con 0x7f766c06a830 2023-10-08T00:21:16.973 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.969+0000 7f7672fd6700 1 -- 172.21.15.99:0/1716179609 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f766c089550 con 0x7f766c06a830 2023-10-08T00:21:16.974 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.969+0000 7f7661ffb700 1 -- 172.21.15.99:0/1716179609 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f766801b3b0 con 0x7f766c06a830 2023-10-08T00:21:16.974 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.970+0000 7f7661ffb700 1 --2- 172.21.15.99:0/1716179609 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7650066f30 0x7f76500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:16.975 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.970+0000 7f7661ffb700 1 -- 172.21.15.99:0/1716179609 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f7668083b40 con 0x7f766c06a830 2023-10-08T00:21:16.975 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.970+0000 7f76717d3700 1 --2- 172.21.15.99:0/1716179609 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7650066f30 0x7f76500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:16.975 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.970+0000 7f7661ffb700 1 -- 172.21.15.99:0/1716179609 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=173}) v3 -- 0x7f765006eb10 con 0x7f766c06a830 2023-10-08T00:21:16.975 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.970+0000 7f76717d3700 1 --2- 172.21.15.99:0/1716179609 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7650066f30 0x7f76500693e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f765c0093f0 tx=0x7f765c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:16.975 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:16.974+0000 7f7661ffb700 1 -- 172.21.15.99:0/1716179609 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7668017070 con 0x7f766c06a830 2023-10-08T00:21:17.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.154+0000 7f7672fd6700 1 -- 172.21.15.99:0/1716179609 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f766c0b4440 con 0x7f766c06a830 2023-10-08T00:21:17.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.154+0000 7f7661ffb700 1 -- 172.21.15.99:0/1716179609 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v172) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7668050d70 con 0x7f766c06a830 2023-10-08T00:21:17.155 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2023-10-08T00:21:17.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.156+0000 7f7672fd6700 1 -- 172.21.15.99:0/1716179609 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7650066f30 msgr2=0x7f76500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:17.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.156+0000 7f7672fd6700 1 --2- 172.21.15.99:0/1716179609 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7650066f30 0x7f76500693e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f765c0093f0 tx=0x7f765c017000 comp rx=0 tx=0).stop 2023-10-08T00:21:17.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.156+0000 7f7672fd6700 1 -- 172.21.15.99:0/1716179609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f766c06a830 msgr2=0x7f766c169e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:17.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.156+0000 7f7672fd6700 1 --2- 172.21.15.99:0/1716179609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f766c06a830 0x7f766c169e20 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f766800c4e0 tx=0x7f766800c8a0 comp rx=0 tx=0).stop 2023-10-08T00:21:17.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.156+0000 7f7672fd6700 1 -- 172.21.15.99:0/1716179609 shutdown_connections 2023-10-08T00:21:17.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.156+0000 7f7672fd6700 1 --2- 172.21.15.99:0/1716179609 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7650066f30 0x7f76500693e0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:17.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.156+0000 7f7672fd6700 1 --2- 172.21.15.99:0/1716179609 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f766c0cce30 0x7f766c153920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:17.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.156+0000 7f7672fd6700 1 --2- 172.21.15.99:0/1716179609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f766c06a830 0x7f766c169e20 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:17.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.156+0000 7f7672fd6700 1 --2- 172.21.15.99:0/1716179609 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f766c0b6580 0x7f766c0cf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:17.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.156+0000 7f7672fd6700 1 -- 172.21.15.99:0/1716179609 >> 172.21.15.99:0/1716179609 conn(0x7f766c068cb0 msgr2=0x7f766c0cff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:17.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.157+0000 7f7672fd6700 1 -- 172.21.15.99:0/1716179609 shutdown_connections 2023-10-08T00:21:17.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.157+0000 7f7672fd6700 1 -- 172.21.15.99:0/1716179609 wait complete. 2023-10-08T00:21:17.169 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:21:17.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.288+0000 7f711c7a8700 1 Processor -- start 2023-10-08T00:21:17.289 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.289+0000 7f711c7a8700 1 -- start start 2023-10-08T00:21:17.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.290+0000 7f711c7a8700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7114157870 0x7f711415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:17.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.290+0000 7f711c7a8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f711415a620 0x7f711415d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:17.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.290+0000 7f711c7a8700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f711415cbb0 con 0x7f711415d580 2023-10-08T00:21:17.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.290+0000 7f711c7a8700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f711415bfa0 con 0x7f7114157870 2023-10-08T00:21:17.292 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.290+0000 7f711c7a8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f711415c280 con 0x7f711415a620 2023-10-08T00:21:17.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.290+0000 7f7119d43700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f711415a620 0x7f711415d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:17.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.290+0000 7f711a544700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7114157870 0x7f711415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:17.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.290+0000 7f7119d43700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f711415a620 0x7f711415d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.99:34506/0 (socket says 172.21.15.99:34506) 2023-10-08T00:21:17.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.290+0000 7f711a544700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7114157870 0x7f711415b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.99:36408/0 (socket says 172.21.15.99:36408) 2023-10-08T00:21:17.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.290+0000 7f711ad45700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f711415d580 0x7f7114158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:57356/0 (socket says 172.21.15.99:57356) 2023-10-08T00:21:17.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.290+0000 7f7119d43700 1 -- 172.21.15.99:0/4273873536 learned_addr learned my addr 172.21.15.99:0/4273873536 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:17.293 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.290+0000 7f7119d43700 1 -- 172.21.15.99:0/4273873536 >> v2:172.21.15.191:3300/0 conn(0x7f7114157870 msgr2=0x7f711415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:17.294 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.290+0000 7f7119d43700 1 --2- 172.21.15.99:0/4273873536 >> v2:172.21.15.191:3300/0 conn(0x7f7114157870 0x7f711415b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:17.294 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.290+0000 7f7119d43700 1 -- 172.21.15.99:0/4273873536 >> v1:172.21.15.191:6789/0 conn(0x7f711415d580 legacy=0x7f7114158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:17.294 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.290+0000 7f7119d43700 1 -- 172.21.15.99:0/4273873536 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f711415c560 con 0x7f711415a620 2023-10-08T00:21:17.294 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.290+0000 7f7119d43700 1 --2- 172.21.15.99:0/4273873536 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f711415a620 0x7f711415d040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f71040099b0 tx=0x7f7104009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79505473ac718a37 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:17.294 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.291+0000 7f7119542700 1 -- 172.21.15.99:0/4273873536 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7104009ea0 con 0x7f711415a620 2023-10-08T00:21:17.294 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.291+0000 7f7119542700 1 -- 172.21.15.99:0/4273873536 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f710400b860 con 0x7f711415a620 2023-10-08T00:21:17.295 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.291+0000 7f7119542700 1 -- 172.21.15.99:0/4273873536 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710400bc00 con 0x7f711415a620 2023-10-08T00:21:17.295 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.291+0000 7f711c7a8700 1 -- 172.21.15.99:0/4273873536 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f711415a620 msgr2=0x7f711415d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:17.295 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.291+0000 7f711c7a8700 1 --2- 172.21.15.99:0/4273873536 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f711415a620 0x7f711415d040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f71040099b0 tx=0x7f7104009250 comp rx=0 tx=0).stop 2023-10-08T00:21:17.295 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.291+0000 7f711c7a8700 1 -- 172.21.15.99:0/4273873536 shutdown_connections 2023-10-08T00:21:17.295 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.291+0000 7f711c7a8700 1 --2- 172.21.15.99:0/4273873536 >> v2:172.21.15.191:3300/0 conn(0x7f7114157870 0x7f711415b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:17.295 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.291+0000 7f711c7a8700 1 --2- 172.21.15.99:0/4273873536 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f711415a620 0x7f711415d040 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:17.295 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.291+0000 7f711c7a8700 1 -- 172.21.15.99:0/4273873536 >> 172.21.15.99:0/4273873536 conn(0x7f7114068ce0 msgr2=0x7f71140690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:17.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.291+0000 7f711c7a8700 1 -- 172.21.15.99:0/4273873536 shutdown_connections 2023-10-08T00:21:17.296 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.291+0000 7f711c7a8700 1 -- 172.21.15.99:0/4273873536 wait complete. 2023-10-08T00:21:17.298 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.292+0000 7f711c7a8700 1 Processor -- start 2023-10-08T00:21:17.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.292+0000 7f711c7a8700 1 -- start start 2023-10-08T00:21:17.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.293+0000 7f711c7a8700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7114157870 0x7f71140b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:17.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.293+0000 7f711c7a8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f711415a620 0x7f7114151780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:17.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.293+0000 7f711c7a8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f711415d580 0x7f7114152730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:17.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.293+0000 7f711c7a8700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71140b20d0 con 0x7f711415a620 2023-10-08T00:21:17.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.293+0000 7f711c7a8700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711416b620 con 0x7f7114157870 2023-10-08T00:21:17.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.293+0000 7f711c7a8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f711416aa10 con 0x7f711415d580 2023-10-08T00:21:17.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.293+0000 7f7119d43700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f711415a620 0x7f7114151780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:17.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.293+0000 7f711a544700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7114157870 0x7f71140b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:17.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.293+0000 7f7119d43700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f711415a620 0x7f7114151780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57386/0 (socket says 172.21.15.99:57386) 2023-10-08T00:21:17.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.293+0000 7f711a544700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7114157870 0x7f71140b1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.99:36412/0 (socket says 172.21.15.99:36412) 2023-10-08T00:21:17.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.293+0000 7f7119d43700 1 -- 172.21.15.99:0/1977953648 learned_addr learned my addr 172.21.15.99:0/1977953648 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:17.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.293+0000 7f711ad45700 1 --2- 172.21.15.99:0/1977953648 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f711415d580 0x7f7114152730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:17.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.293+0000 7f7119d43700 1 -- 172.21.15.99:0/1977953648 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f711415d580 msgr2=0x7f7114152730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:17.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.293+0000 7f7119d43700 1 --2- 172.21.15.99:0/1977953648 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f711415d580 0x7f7114152730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:17.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.293+0000 7f7119d43700 1 -- 172.21.15.99:0/1977953648 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7114157870 msgr2=0x7f71140b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:17.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.293+0000 7f7119d43700 1 --2- 172.21.15.99:0/1977953648 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7114157870 0x7f71140b1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:17.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.293+0000 7f7119d43700 1 -- 172.21.15.99:0/1977953648 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71040094f0 con 0x7f711415a620 2023-10-08T00:21:17.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.293+0000 7f711a544700 1 --2- 172.21.15.99:0/1977953648 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7114157870 0x7f71140b1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:21:17.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.294+0000 7f7119d43700 1 --2- 172.21.15.99:0/1977953648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f711415a620 0x7f7114151780 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f711415c3c0 tx=0x7f7104017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:17.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.294+0000 7f710a7fc700 1 -- 172.21.15.99:0/1977953648 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7104000d10 con 0x7f711415a620 2023-10-08T00:21:17.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.294+0000 7f710a7fc700 1 -- 172.21.15.99:0/1977953648 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f710400bbe0 con 0x7f711415a620 2023-10-08T00:21:17.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.294+0000 7f711c7a8700 1 -- 172.21.15.99:0/1977953648 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f711416acf0 con 0x7f711415a620 2023-10-08T00:21:17.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.294+0000 7f710a7fc700 1 -- 172.21.15.99:0/1977953648 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7104011670 con 0x7f711415a620 2023-10-08T00:21:17.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.294+0000 7f711c7a8700 1 -- 172.21.15.99:0/1977953648 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f711416b2b0 con 0x7f711415a620 2023-10-08T00:21:17.302 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.295+0000 7f711c7a8700 1 -- 172.21.15.99:0/1977953648 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f711416dca0 con 0x7f711415a620 2023-10-08T00:21:17.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.295+0000 7f710a7fc700 1 -- 172.21.15.99:0/1977953648 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7104011aa0 con 0x7f711415a620 2023-10-08T00:21:17.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.295+0000 7f710a7fc700 1 --2- 172.21.15.99:0/1977953648 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f70f8066f30 0x7f70f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:17.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.296+0000 7f710a7fc700 1 -- 172.21.15.99:0/1977953648 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f7104085dd0 con 0x7f711415a620 2023-10-08T00:21:17.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.296+0000 7f711a544700 1 --2- 172.21.15.99:0/1977953648 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f70f8066f30 0x7f70f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:17.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.296+0000 7f710a7fc700 1 -- 172.21.15.99:0/1977953648 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=173}) v3 -- 0x7f70f806eb10 con 0x7f711415a620 2023-10-08T00:21:17.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.296+0000 7f711a544700 1 --2- 172.21.15.99:0/1977953648 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f70f8066f30 0x7f70f80693e0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f7110011480 tx=0x7f71100091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:17.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.300+0000 7f710a7fc700 1 -- 172.21.15.99:0/1977953648 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f711416dca0 con 0x7f711415a620 2023-10-08T00:21:17.524 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.524+0000 7f711c7a8700 1 -- 172.21.15.99:0/1977953648 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f711416dca0 con 0x7f711415a620 2023-10-08T00:21:17.525 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.524+0000 7f710a7fc700 1 -- 172.21.15.99:0/1977953648 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4601 (secure 0 0 0) 0x7f711416dca0 con 0x7f711415a620 2023-10-08T00:21:17.525 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:21:17.525 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":77,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6257,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/661016677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":661016677},{"type":"v1","addr":"172.21.15.191:6837","nonce":661016677}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":7012,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/4102051559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":4102051559},{"type":"v1","addr":"172.21.15.99:6835","nonce":4102051559}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":7021,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/658854389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":658854389},{"type":"v1","addr":"172.21.15.191:6836","nonce":658854389}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:20:04.613994+0000","modified":"2023-10-08T00:20:08.255350+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7791},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7791":{"gid":7791,"name":"c","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6834/3880334948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3880334948},{"type":"v1","addr":"172.21.15.99:6834","nonce":3880334948}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-08T00:21:17.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.526+0000 7f711c7a8700 1 -- 172.21.15.99:0/1977953648 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f70f8066f30 msgr2=0x7f70f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:17.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.526+0000 7f711c7a8700 1 --2- 172.21.15.99:0/1977953648 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f70f8066f30 0x7f70f80693e0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f7110011480 tx=0x7f71100091a0 comp rx=0 tx=0).stop 2023-10-08T00:21:17.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.526+0000 7f711c7a8700 1 -- 172.21.15.99:0/1977953648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f711415a620 msgr2=0x7f7114151780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:17.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.526+0000 7f711c7a8700 1 --2- 172.21.15.99:0/1977953648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f711415a620 0x7f7114151780 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f711415c3c0 tx=0x7f7104017000 comp rx=0 tx=0).stop 2023-10-08T00:21:17.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.526+0000 7f711c7a8700 1 -- 172.21.15.99:0/1977953648 shutdown_connections 2023-10-08T00:21:17.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.526+0000 7f711c7a8700 1 --2- 172.21.15.99:0/1977953648 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f70f8066f30 0x7f70f80693e0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:17.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.526+0000 7f711c7a8700 1 --2- 172.21.15.99:0/1977953648 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7114157870 0x7f71140b1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:17.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.526+0000 7f711c7a8700 1 --2- 172.21.15.99:0/1977953648 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f711415d580 0x7f7114152730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:17.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.526+0000 7f711c7a8700 1 --2- 172.21.15.99:0/1977953648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f711415a620 0x7f7114151780 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:17.530 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.526+0000 7f711c7a8700 1 -- 172.21.15.99:0/1977953648 >> 172.21.15.99:0/1977953648 conn(0x7f7114068ce0 msgr2=0x7f7114153320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:17.530 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.526+0000 7f711c7a8700 1 -- 172.21.15.99:0/1977953648 shutdown_connections 2023-10-08T00:21:17.530 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.526+0000 7f711c7a8700 1 -- 172.21.15.99:0/1977953648 wait complete. 2023-10-08T00:21:17.530 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 77 2023-10-08T00:21:17.539 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:21:17.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.657+0000 7f330bb9b700 1 Processor -- start 2023-10-08T00:21:17.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.657+0000 7f330bb9b700 1 -- start start 2023-10-08T00:21:17.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.658+0000 7f330bb9b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f330415a2b0 0x7f33040b3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:17.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.658+0000 7f330bb9b700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33040c5d80 con 0x7f33040d1c30 2023-10-08T00:21:17.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.658+0000 7f330bb9b700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33040c6060 con 0x7f33040cfe20 2023-10-08T00:21:17.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.658+0000 7f330bb9b700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3304068470 con 0x7f330415a2b0 2023-10-08T00:21:17.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.658+0000 7f3309136700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f33040d1c30 0x7f33040cf710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:46722/0 (socket says 172.21.15.99:46722) 2023-10-08T00:21:17.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.658+0000 7f3309136700 1 -- 172.21.15.99:0/3141088953 learned_addr learned my addr 172.21.15.99:0/3141088953 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:21:17.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.658+0000 7f3309937700 1 --2- 172.21.15.99:0/3141088953 >> v2:172.21.15.191:3300/0 conn(0x7f330415a2b0 0x7f33040b3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:17.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.658+0000 7f3308935700 1 -- 172.21.15.99:0/3141088953 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1089626732 0 0) 0x7f33040c5d80 con 0x7f33040d1c30 2023-10-08T00:21:17.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.658+0000 7f3308935700 1 -- 172.21.15.99:0/3141088953 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32f0003570 con 0x7f33040d1c30 2023-10-08T00:21:17.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.658+0000 7f3309937700 1 -- 172.21.15.99:0/3141088953 >> v1:172.21.15.191:6789/0 conn(0x7f33040cfe20 legacy=0x7f33040c7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:17.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.658+0000 7f3309937700 1 -- 172.21.15.99:0/3141088953 >> v1:172.21.15.99:6789/0 conn(0x7f33040d1c30 legacy=0x7f33040cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:17.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.658+0000 7f3309937700 1 -- 172.21.15.99:0/3141088953 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33040c6340 con 0x7f330415a2b0 2023-10-08T00:21:17.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.659+0000 7f3309937700 1 --2- 172.21.15.99:0/3141088953 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f330415a2b0 0x7f33040b3da0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f33000098d0 tx=0x7f3300009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=371c2b5fd8576594 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:17.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.659+0000 7f3308935700 1 -- 172.21.15.99:0/3141088953 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3300005c10 con 0x7f330415a2b0 2023-10-08T00:21:17.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.659+0000 7f3308935700 1 -- 172.21.15.99:0/3141088953 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3300005d70 con 0x7f330415a2b0 2023-10-08T00:21:17.661 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.659+0000 7f3308935700 1 -- 172.21.15.99:0/3141088953 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3300006280 con 0x7f330415a2b0 2023-10-08T00:21:17.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.659+0000 7f330bb9b700 1 -- 172.21.15.99:0/3141088953 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f330415a2b0 msgr2=0x7f33040b3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:17.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.659+0000 7f330bb9b700 1 --2- 172.21.15.99:0/3141088953 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f330415a2b0 0x7f33040b3da0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f33000098d0 tx=0x7f3300009120 comp rx=0 tx=0).stop 2023-10-08T00:21:17.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.659+0000 7f330bb9b700 1 -- 172.21.15.99:0/3141088953 shutdown_connections 2023-10-08T00:21:17.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.659+0000 7f330bb9b700 1 --2- 172.21.15.99:0/3141088953 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f330415a2b0 0x7f33040b3da0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:17.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.659+0000 7f330bb9b700 1 -- 172.21.15.99:0/3141088953 >> 172.21.15.99:0/3141088953 conn(0x7f3304067e40 msgr2=0x7f33040d0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:17.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.659+0000 7f330bb9b700 1 -- 172.21.15.99:0/3141088953 shutdown_connections 2023-10-08T00:21:17.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.659+0000 7f330bb9b700 1 -- 172.21.15.99:0/3141088953 wait complete. 2023-10-08T00:21:17.666 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.660+0000 7f330bb9b700 1 Processor -- start 2023-10-08T00:21:17.666 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.661+0000 7f330bb9b700 1 -- start start 2023-10-08T00:21:17.666 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.661+0000 7f330bb9b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33040cfe20 0x7f3304158ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:17.666 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.661+0000 7f330bb9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33040d1c30 0x7f330415c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:17.666 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.661+0000 7f330bb9b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f330415a2b0 0x7f330415c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:17.666 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.661+0000 7f330bb9b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330415ba30 con 0x7f33040d1c30 2023-10-08T00:21:17.666 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.661+0000 7f330bb9b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330415e530 con 0x7f330415a2b0 2023-10-08T00:21:17.666 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.661+0000 7f330bb9b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f330415b470 con 0x7f33040cfe20 2023-10-08T00:21:17.667 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.661+0000 7f3309136700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33040d1c30 0x7f330415c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:17.667 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.661+0000 7f3309136700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33040d1c30 0x7f330415c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57388/0 (socket says 172.21.15.99:57388) 2023-10-08T00:21:17.667 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.661+0000 7f3309136700 1 -- 172.21.15.99:0/183797389 learned_addr learned my addr 172.21.15.99:0/183797389 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:17.667 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.662+0000 7f330a138700 1 --2- 172.21.15.99:0/183797389 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f330415a2b0 0x7f330415c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:17.667 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.662+0000 7f3309136700 1 -- 172.21.15.99:0/183797389 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33040cfe20 msgr2=0x7f3304158ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:17.667 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.662+0000 7f3309136700 1 --2- 172.21.15.99:0/183797389 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33040cfe20 0x7f3304158ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:17.667 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.662+0000 7f3309136700 1 -- 172.21.15.99:0/183797389 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f330415a2b0 msgr2=0x7f330415c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:17.667 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.662+0000 7f3309136700 1 --2- 172.21.15.99:0/183797389 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f330415a2b0 0x7f330415c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:17.667 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.662+0000 7f3309136700 1 -- 172.21.15.99:0/183797389 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33000093c0 con 0x7f33040d1c30 2023-10-08T00:21:17.667 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.662+0000 7f330a138700 1 --2- 172.21.15.99:0/183797389 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f330415a2b0 0x7f330415c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:21:17.667 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.662+0000 7f3309136700 1 --2- 172.21.15.99:0/183797389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33040d1c30 0x7f330415c230 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f32f400c590 tx=0x7f32f4007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:17.667 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.662+0000 7f32f9ffb700 1 -- 172.21.15.99:0/183797389 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f4005c00 con 0x7f33040d1c30 2023-10-08T00:21:17.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.662+0000 7f32f9ffb700 1 -- 172.21.15.99:0/183797389 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32f4005d60 con 0x7f33040d1c30 2023-10-08T00:21:17.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.662+0000 7f330bb9b700 1 -- 172.21.15.99:0/183797389 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f330415b750 con 0x7f33040d1c30 2023-10-08T00:21:17.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.662+0000 7f32f9ffb700 1 -- 172.21.15.99:0/183797389 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f4007300 con 0x7f33040d1c30 2023-10-08T00:21:17.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.662+0000 7f330bb9b700 1 -- 172.21.15.99:0/183797389 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f330415b040 con 0x7f33040d1c30 2023-10-08T00:21:17.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.663+0000 7f330bb9b700 1 -- 172.21.15.99:0/183797389 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f330415e020 con 0x7f33040d1c30 2023-10-08T00:21:17.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.663+0000 7f32f9ffb700 1 -- 172.21.15.99:0/183797389 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f32f401b3b0 con 0x7f33040d1c30 2023-10-08T00:21:17.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.664+0000 7f32f9ffb700 1 --2- 172.21.15.99:0/183797389 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f32e8066f30 0x7f32e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:17.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.664+0000 7f32f9ffb700 1 -- 172.21.15.99:0/183797389 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f32f4083a70 con 0x7f33040d1c30 2023-10-08T00:21:17.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.664+0000 7f32f9ffb700 1 -- 172.21.15.99:0/183797389 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=173}) v3 -- 0x7f32e806eb10 con 0x7f33040d1c30 2023-10-08T00:21:17.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.664+0000 7f3309937700 1 --2- 172.21.15.99:0/183797389 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f32e8066f30 0x7f32e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:17.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.664+0000 7f3309937700 1 --2- 172.21.15.99:0/183797389 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f32e8066f30 0x7f32e80693e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f3304160a20 tx=0x7f3300016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:17.669 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.668+0000 7f32f9ffb700 1 -- 172.21.15.99:0/183797389 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f32f4017020 con 0x7f33040d1c30 2023-10-08T00:21:17.851 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.850+0000 7f330bb9b700 1 -- 172.21.15.99:0/183797389 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f33040b3260 con 0x7f33040d1c30 2023-10-08T00:21:17.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.851+0000 7f32f9ffb700 1 -- 172.21.15.99:0/183797389 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v172) v1 ==== 74+0+15488 (secure 0 0 0) 0x7f330415e020 con 0x7f33040d1c30 2023-10-08T00:21:17.852 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:21:17.852 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":172,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:20:51.158163+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:20:02.932463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-08T00:20:03.578010+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.6","mappings":[{"from":2,"to":5}]},{"pgid":"21.10","mappings":[{"from":2,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:21:17.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.853+0000 7f330bb9b700 1 -- 172.21.15.99:0/183797389 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f32e8066f30 msgr2=0x7f32e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:17.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.853+0000 7f330bb9b700 1 --2- 172.21.15.99:0/183797389 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f32e8066f30 0x7f32e80693e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f3304160a20 tx=0x7f3300016000 comp rx=0 tx=0).stop 2023-10-08T00:21:17.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.853+0000 7f330bb9b700 1 -- 172.21.15.99:0/183797389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33040d1c30 msgr2=0x7f330415c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:17.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.853+0000 7f330bb9b700 1 --2- 172.21.15.99:0/183797389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33040d1c30 0x7f330415c230 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f32f400c590 tx=0x7f32f4007dd0 comp rx=0 tx=0).stop 2023-10-08T00:21:17.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.853+0000 7f330bb9b700 1 -- 172.21.15.99:0/183797389 shutdown_connections 2023-10-08T00:21:17.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.853+0000 7f330bb9b700 1 --2- 172.21.15.99:0/183797389 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f32e8066f30 0x7f32e80693e0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:17.855 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.853+0000 7f330bb9b700 1 --2- 172.21.15.99:0/183797389 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f330415a2b0 0x7f330415c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:17.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.853+0000 7f330bb9b700 1 --2- 172.21.15.99:0/183797389 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33040cfe20 0x7f3304158ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:17.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.853+0000 7f330bb9b700 1 --2- 172.21.15.99:0/183797389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33040d1c30 0x7f330415c230 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:17.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.853+0000 7f330bb9b700 1 -- 172.21.15.99:0/183797389 >> 172.21.15.99:0/183797389 conn(0x7f3304067e40 msgr2=0x7f33040cbd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:17.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.853+0000 7f330bb9b700 1 -- 172.21.15.99:0/183797389 shutdown_connections 2023-10-08T00:21:17.856 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.853+0000 7f330bb9b700 1 -- 172.21.15.99:0/183797389 wait complete. 2023-10-08T00:21:17.867 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T00:21:17.867 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:21:17.981 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.980+0000 7f463517b700 1 Processor -- start 2023-10-08T00:21:17.981 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.980+0000 7f463517b700 1 -- start start 2023-10-08T00:21:17.984 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.981+0000 7f463517b700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4630157870 0x7f463015b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:17.984 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.981+0000 7f463517b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f463015a620 0x7f463015d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:17.984 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.981+0000 7f463517b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f463015d580 0x7f4630158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:17.985 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.981+0000 7f463517b700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f463015cbb0 con 0x7f4630157870 2023-10-08T00:21:17.985 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.981+0000 7f463517b700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f463015bfa0 con 0x7f463015d580 2023-10-08T00:21:17.985 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.981+0000 7f463517b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f463015c280 con 0x7f463015a620 2023-10-08T00:21:17.985 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.981+0000 7f462ed9d700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4630157870 0x7f463015b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:17.985 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.981+0000 7f462ed9d700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4630157870 0x7f463015b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57392/0 (socket says 172.21.15.99:57392) 2023-10-08T00:21:17.985 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.981+0000 7f462ed9d700 1 -- 172.21.15.99:0/1880758512 learned_addr learned my addr 172.21.15.99:0/1880758512 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:17.985 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.982+0000 7f462ed9d700 1 -- 172.21.15.99:0/1880758512 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f463015a620 msgr2=0x7f463015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:17.986 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.982+0000 7f462ed9d700 1 --2- 172.21.15.99:0/1880758512 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f463015a620 0x7f463015d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:17.986 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.982+0000 7f462f59e700 1 --2- 172.21.15.99:0/1880758512 >> v2:172.21.15.191:3300/0 conn(0x7f463015d580 0x7f4630158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:17.986 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.982+0000 7f462ed9d700 1 -- 172.21.15.99:0/1880758512 >> v2:172.21.15.191:3300/0 conn(0x7f463015d580 msgr2=0x7f4630158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:17.986 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.982+0000 7f462ed9d700 1 --2- 172.21.15.99:0/1880758512 >> v2:172.21.15.191:3300/0 conn(0x7f463015d580 0x7f4630158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:17.986 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.982+0000 7f462ed9d700 1 -- 172.21.15.99:0/1880758512 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f463015c560 con 0x7f4630157870 2023-10-08T00:21:17.986 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.982+0000 7f462ed9d700 1 --2- 172.21.15.99:0/1880758512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4630157870 0x7f463015b6a0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f46300b03b0 tx=0x7f4620009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7c97723b836a61e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:17.986 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.982+0000 7f462e59c700 1 -- 172.21.15.99:0/1880758512 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462000b860 con 0x7f4630157870 2023-10-08T00:21:17.986 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.982+0000 7f462e59c700 1 -- 172.21.15.99:0/1880758512 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462000b9c0 con 0x7f4630157870 2023-10-08T00:21:17.986 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.982+0000 7f462e59c700 1 -- 172.21.15.99:0/1880758512 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462000bd60 con 0x7f4630157870 2023-10-08T00:21:17.986 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.982+0000 7f463517b700 1 -- 172.21.15.99:0/1880758512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4630157870 msgr2=0x7f463015b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:17.986 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.982+0000 7f463517b700 1 --2- 172.21.15.99:0/1880758512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4630157870 0x7f463015b6a0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f46300b03b0 tx=0x7f4620009250 comp rx=0 tx=0).stop 2023-10-08T00:21:17.987 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.982+0000 7f463517b700 1 -- 172.21.15.99:0/1880758512 shutdown_connections 2023-10-08T00:21:17.987 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.982+0000 7f463517b700 1 --2- 172.21.15.99:0/1880758512 >> v2:172.21.15.191:3300/0 conn(0x7f463015d580 0x7f4630158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:17.987 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.982+0000 7f463517b700 1 --2- 172.21.15.99:0/1880758512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4630157870 0x7f463015b6a0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:17.987 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.982+0000 7f463517b700 1 --2- 172.21.15.99:0/1880758512 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f463015a620 0x7f463015d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:17.987 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.982+0000 7f463517b700 1 -- 172.21.15.99:0/1880758512 >> 172.21.15.99:0/1880758512 conn(0x7f4630068ce0 msgr2=0x7f46300690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:17.987 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.982+0000 7f463517b700 1 -- 172.21.15.99:0/1880758512 shutdown_connections 2023-10-08T00:21:17.987 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.982+0000 7f463517b700 1 -- 172.21.15.99:0/1880758512 wait complete. 2023-10-08T00:21:17.987 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.983+0000 7f463517b700 1 Processor -- start 2023-10-08T00:21:17.987 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.984+0000 7f463517b700 1 -- start start 2023-10-08T00:21:17.990 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.984+0000 7f463517b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4630157870 0x7f4630153a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:17.991 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.984+0000 7f463517b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f463015a620 0x7f46300b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:17.991 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.984+0000 7f463517b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f463015d580 0x7f46300c6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:17.991 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.984+0000 7f463517b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46300b22c0 con 0x7f463015a620 2023-10-08T00:21:17.991 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.984+0000 7f463517b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46300b2b60 con 0x7f4630157870 2023-10-08T00:21:17.991 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.984+0000 7f463517b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46300c6fe0 con 0x7f463015d580 2023-10-08T00:21:17.991 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.984+0000 7f4627fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f463015a620 0x7f46300b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:17.991 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.984+0000 7f4627fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f463015a620 0x7f46300b2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57406/0 (socket says 172.21.15.99:57406) 2023-10-08T00:21:17.991 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.984+0000 7f4627fff700 1 -- 172.21.15.99:0/1926677955 learned_addr learned my addr 172.21.15.99:0/1926677955 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:17.992 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.984+0000 7f462ed9d700 1 --2- 172.21.15.99:0/1926677955 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4630157870 0x7f4630153a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:17.992 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.985+0000 7f462f59e700 1 --2- 172.21.15.99:0/1926677955 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f463015d580 0x7f46300c6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:17.992 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.985+0000 7f4627fff700 1 -- 172.21.15.99:0/1926677955 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f463015d580 msgr2=0x7f46300c6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:17.992 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.985+0000 7f4627fff700 1 --2- 172.21.15.99:0/1926677955 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f463015d580 0x7f46300c6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:17.992 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.985+0000 7f4627fff700 1 -- 172.21.15.99:0/1926677955 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4630157870 msgr2=0x7f4630153a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:17.992 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.985+0000 7f4627fff700 1 --2- 172.21.15.99:0/1926677955 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4630157870 0x7f4630153a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:17.992 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.985+0000 7f4627fff700 1 -- 172.21.15.99:0/1926677955 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46200094f0 con 0x7f463015a620 2023-10-08T00:21:17.992 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.985+0000 7f4627fff700 1 --2- 172.21.15.99:0/1926677955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f463015a620 0x7f46300b2ff0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f461800b840 tx=0x7f4618009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:17.993 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.985+0000 7f4626ffd700 1 -- 172.21.15.99:0/1926677955 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461800f510 con 0x7f463015a620 2023-10-08T00:21:17.993 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.985+0000 7f4626ffd700 1 -- 172.21.15.99:0/1926677955 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f461800f670 con 0x7f463015a620 2023-10-08T00:21:17.993 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.985+0000 7f463517b700 1 -- 172.21.15.99:0/1926677955 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46300b2510 con 0x7f463015a620 2023-10-08T00:21:17.993 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.985+0000 7f4626ffd700 1 -- 172.21.15.99:0/1926677955 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461800fa30 con 0x7f463015a620 2023-10-08T00:21:17.993 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.985+0000 7f463517b700 1 -- 172.21.15.99:0/1926677955 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46300b3530 con 0x7f463015a620 2023-10-08T00:21:17.993 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.986+0000 7f463517b700 1 -- 172.21.15.99:0/1926677955 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46300b2cf0 con 0x7f463015a620 2023-10-08T00:21:17.993 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.986+0000 7f4626ffd700 1 -- 172.21.15.99:0/1926677955 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f461801a400 con 0x7f463015a620 2023-10-08T00:21:17.993 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.987+0000 7f4626ffd700 1 --2- 172.21.15.99:0/1926677955 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4610066f30 0x7f46100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:17.994 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.987+0000 7f4626ffd700 1 -- 172.21.15.99:0/1926677955 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f46180840d0 con 0x7f463015a620 2023-10-08T00:21:17.994 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.987+0000 7f4626ffd700 1 -- 172.21.15.99:0/1926677955 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=173}) v3 -- 0x7f461006eb10 con 0x7f463015a620 2023-10-08T00:21:17.994 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.987+0000 7f462ed9d700 1 --2- 172.21.15.99:0/1926677955 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4610066f30 0x7f46100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:17.994 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.987+0000 7f462ed9d700 1 --2- 172.21.15.99:0/1926677955 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4610066f30 0x7f46100693e0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f46300b03b0 tx=0x7f4620017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:17.994 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:17.991+0000 7f4626ffd700 1 -- 172.21.15.99:0/1926677955 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4618051380 con 0x7f463015a620 2023-10-08T00:21:18.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.213+0000 7f463517b700 1 -- 172.21.15.99:0/1926677955 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f463004c3c0 con 0x7f463015a620 2023-10-08T00:21:18.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.213+0000 7f4626ffd700 1 -- 172.21.15.99:0/1926677955 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4601 (secure 0 0 0) 0x7f46180549a0 con 0x7f463015a620 2023-10-08T00:21:18.214 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:21:18.214 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":77,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6257,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/661016677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":661016677},{"type":"v1","addr":"172.21.15.191:6837","nonce":661016677}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":7012,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/4102051559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":4102051559},{"type":"v1","addr":"172.21.15.99:6835","nonce":4102051559}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":7021,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/658854389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":658854389},{"type":"v1","addr":"172.21.15.191:6836","nonce":658854389}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:20:04.613994+0000","modified":"2023-10-08T00:20:08.255350+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7791},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7791":{"gid":7791,"name":"c","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6834/3880334948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3880334948},{"type":"v1","addr":"172.21.15.99:6834","nonce":3880334948}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-08T00:21:18.216 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.215+0000 7f463517b700 1 -- 172.21.15.99:0/1926677955 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4610066f30 msgr2=0x7f46100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:18.216 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.215+0000 7f463517b700 1 --2- 172.21.15.99:0/1926677955 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4610066f30 0x7f46100693e0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f46300b03b0 tx=0x7f4620017000 comp rx=0 tx=0).stop 2023-10-08T00:21:18.216 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.215+0000 7f463517b700 1 -- 172.21.15.99:0/1926677955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f463015a620 msgr2=0x7f46300b2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:18.216 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.215+0000 7f463517b700 1 --2- 172.21.15.99:0/1926677955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f463015a620 0x7f46300b2ff0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f461800b840 tx=0x7f4618009220 comp rx=0 tx=0).stop 2023-10-08T00:21:18.216 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.215+0000 7f463517b700 1 -- 172.21.15.99:0/1926677955 shutdown_connections 2023-10-08T00:21:18.217 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.215+0000 7f463517b700 1 --2- 172.21.15.99:0/1926677955 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4610066f30 0x7f46100693e0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:18.217 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.215+0000 7f463517b700 1 --2- 172.21.15.99:0/1926677955 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4630157870 0x7f4630153a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:18.217 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.215+0000 7f463517b700 1 --2- 172.21.15.99:0/1926677955 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f463015d580 0x7f46300c6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:18.217 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.215+0000 7f463517b700 1 --2- 172.21.15.99:0/1926677955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f463015a620 0x7f46300b2ff0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:18.217 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.215+0000 7f463517b700 1 -- 172.21.15.99:0/1926677955 >> 172.21.15.99:0/1926677955 conn(0x7f4630068ce0 msgr2=0x7f4630155730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:18.217 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.215+0000 7f463517b700 1 -- 172.21.15.99:0/1926677955 shutdown_connections 2023-10-08T00:21:18.217 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.215+0000 7f463517b700 1 -- 172.21.15.99:0/1926677955 wait complete. 2023-10-08T00:21:18.217 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 77 2023-10-08T00:21:18.228 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:21:18.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.341+0000 7ff8f2f71700 1 Processor -- start 2023-10-08T00:21:18.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.341+0000 7ff8f2f71700 1 -- start start 2023-10-08T00:21:18.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.342+0000 7ff8f2f71700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff8ec143520 0x7ff8ec0cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:18.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.342+0000 7ff8f2f71700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8ec1438f0 0x7ff8ec06a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:18.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.342+0000 7ff8f2f71700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8ec0d0870 con 0x7ff8ec0ccf00 2023-10-08T00:21:18.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.342+0000 7ff8f2f71700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8ec0cdfb0 con 0x7ff8ec143520 2023-10-08T00:21:18.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.342+0000 7ff8f2f71700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ec0cfc60 con 0x7ff8ec1438f0 2023-10-08T00:21:18.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.342+0000 7ff8f1f6f700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff8ec143520 0x7ff8ec0cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:18.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.342+0000 7ff8f176e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8ec1438f0 0x7ff8ec06a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:18.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.342+0000 7ff8f2770700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7ff8ec0ccf00 0x7ff8ec15da60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:57368/0 (socket says 172.21.15.99:57368) 2023-10-08T00:21:18.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.342+0000 7ff8f2770700 1 -- 172.21.15.99:0/254715487 learned_addr learned my addr 172.21.15.99:0/254715487 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:21:18.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.343+0000 7ff8f1f6f700 1 -- 172.21.15.99:0/254715487 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8ec1438f0 msgr2=0x7ff8ec06a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:18.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.343+0000 7ff8f1f6f700 1 --2- 172.21.15.99:0/254715487 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8ec1438f0 0x7ff8ec06a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:18.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.343+0000 7ff8f1f6f700 1 -- 172.21.15.99:0/254715487 >> v1:172.21.15.191:6789/0 conn(0x7ff8ec0ccf00 legacy=0x7ff8ec15da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:18.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.343+0000 7ff8f1f6f700 1 -- 172.21.15.99:0/254715487 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8ec0cff40 con 0x7ff8ec143520 2023-10-08T00:21:18.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.343+0000 7ff8f0f6d700 1 -- 172.21.15.99:0/254715487 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1957098650 0 0) 0x7ff8ec0d0870 con 0x7ff8ec0ccf00 2023-10-08T00:21:18.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.343+0000 7ff8f176e700 1 --2- 172.21.15.99:0/254715487 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8ec1438f0 0x7ff8ec06a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:21:18.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.343+0000 7ff8f1f6f700 1 --2- 172.21.15.99:0/254715487 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8ec143520 0x7ff8ec0cf580 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8009e60 tx=0x7ff8e8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0ea1da090d90844 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:18.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.343+0000 7ff8f0f6d700 1 -- 172.21.15.99:0/254715487 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e800e070 con 0x7ff8ec143520 2023-10-08T00:21:18.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.343+0000 7ff8f0f6d700 1 -- 172.21.15.99:0/254715487 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e800bb60 con 0x7ff8ec143520 2023-10-08T00:21:18.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.343+0000 7ff8f0f6d700 1 -- 172.21.15.99:0/254715487 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e8012430 con 0x7ff8ec143520 2023-10-08T00:21:18.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.344+0000 7ff8f2f71700 1 -- 172.21.15.99:0/254715487 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8ec143520 msgr2=0x7ff8ec0cf580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:18.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.344+0000 7ff8f2f71700 1 --2- 172.21.15.99:0/254715487 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8ec143520 0x7ff8ec0cf580 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8009e60 tx=0x7ff8e8009780 comp rx=0 tx=0).stop 2023-10-08T00:21:18.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.344+0000 7ff8f2f71700 1 -- 172.21.15.99:0/254715487 shutdown_connections 2023-10-08T00:21:18.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.344+0000 7ff8f2f71700 1 --2- 172.21.15.99:0/254715487 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8ec143520 0x7ff8ec0cf580 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:18.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.344+0000 7ff8f2f71700 1 --2- 172.21.15.99:0/254715487 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8ec1438f0 0x7ff8ec06a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:18.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.344+0000 7ff8f2f71700 1 -- 172.21.15.99:0/254715487 >> 172.21.15.99:0/254715487 conn(0x7ff8ec069750 msgr2=0x7ff8ec0b25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:18.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.344+0000 7ff8f2f71700 1 -- 172.21.15.99:0/254715487 shutdown_connections 2023-10-08T00:21:18.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.344+0000 7ff8f2f71700 1 -- 172.21.15.99:0/254715487 wait complete. 2023-10-08T00:21:18.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.345+0000 7ff8f2f71700 1 Processor -- start 2023-10-08T00:21:18.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.345+0000 7ff8f2f71700 1 -- start start 2023-10-08T00:21:18.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.345+0000 7ff8f2f71700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8ec0ccf00 0x7ff8ec0c8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:18.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.345+0000 7ff8f2f71700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8ec143520 0x7ff8ec159500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:18.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.345+0000 7ff8f2f71700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff8ec1438f0 0x7ff8ec16daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:18.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.345+0000 7ff8f2f71700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ec15aae0 con 0x7ff8ec1438f0 2023-10-08T00:21:18.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.345+0000 7ff8f2f71700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ec15b0a0 con 0x7ff8ec0ccf00 2023-10-08T00:21:18.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.345+0000 7ff8f2f71700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ec0c8a80 con 0x7ff8ec143520 2023-10-08T00:21:18.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.346+0000 7ff8f2770700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff8ec1438f0 0x7ff8ec16daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:18.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.346+0000 7ff8f2770700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff8ec1438f0 0x7ff8ec16daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57410/0 (socket says 172.21.15.99:57410) 2023-10-08T00:21:18.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.346+0000 7ff8f2770700 1 -- 172.21.15.99:0/2163402800 learned_addr learned my addr 172.21.15.99:0/2163402800 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:18.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.346+0000 7ff8f176e700 1 --2- 172.21.15.99:0/2163402800 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8ec143520 0x7ff8ec159500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:18.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.346+0000 7ff8f1f6f700 1 --2- 172.21.15.99:0/2163402800 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8ec0ccf00 0x7ff8ec0c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:18.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.346+0000 7ff8f2770700 1 -- 172.21.15.99:0/2163402800 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8ec143520 msgr2=0x7ff8ec159500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:18.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.346+0000 7ff8f2770700 1 --2- 172.21.15.99:0/2163402800 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8ec143520 0x7ff8ec159500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:18.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.346+0000 7ff8f2770700 1 -- 172.21.15.99:0/2163402800 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8ec0ccf00 msgr2=0x7ff8ec0c8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:18.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.346+0000 7ff8f2770700 1 --2- 172.21.15.99:0/2163402800 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8ec0ccf00 0x7ff8ec0c8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:18.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.346+0000 7ff8f2770700 1 -- 172.21.15.99:0/2163402800 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8dc009420 con 0x7ff8ec1438f0 2023-10-08T00:21:18.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.346+0000 7ff8f176e700 1 --2- 172.21.15.99:0/2163402800 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8ec143520 0x7ff8ec159500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:21:18.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.346+0000 7ff8f2770700 1 --2- 172.21.15.99:0/2163402800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff8ec1438f0 0x7ff8ec16daf0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7ff8e400c7e0 tx=0x7ff8e4007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:18.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.346+0000 7ff8e1ffb700 1 -- 172.21.15.99:0/2163402800 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4005c00 con 0x7ff8ec1438f0 2023-10-08T00:21:18.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.346+0000 7ff8e1ffb700 1 -- 172.21.15.99:0/2163402800 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e4005d80 con 0x7ff8ec1438f0 2023-10-08T00:21:18.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.346+0000 7ff8f2f71700 1 -- 172.21.15.99:0/2163402800 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8e8009a20 con 0x7ff8ec1438f0 2023-10-08T00:21:18.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.346+0000 7ff8e1ffb700 1 -- 172.21.15.99:0/2163402800 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4007320 con 0x7ff8ec1438f0 2023-10-08T00:21:18.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.346+0000 7ff8f2f71700 1 -- 172.21.15.99:0/2163402800 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8ec15ad30 con 0x7ff8ec1438f0 2023-10-08T00:21:18.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.348+0000 7ff8f2f71700 1 -- 172.21.15.99:0/2163402800 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8ec089030 con 0x7ff8ec1438f0 2023-10-08T00:21:18.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.348+0000 7ff8e1ffb700 1 -- 172.21.15.99:0/2163402800 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff8e401b3b0 con 0x7ff8ec1438f0 2023-10-08T00:21:18.352 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.348+0000 7ff8e1ffb700 1 --2- 172.21.15.99:0/2163402800 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff8d0066f30 0x7ff8d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:18.352 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.348+0000 7ff8e1ffb700 1 -- 172.21.15.99:0/2163402800 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5763+0+0 (secure 0 0 0) 0x7ff8e4083b00 con 0x7ff8ec1438f0 2023-10-08T00:21:18.352 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.348+0000 7ff8e1ffb700 1 -- 172.21.15.99:0/2163402800 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=173}) v3 -- 0x7ff8d006eb10 con 0x7ff8ec1438f0 2023-10-08T00:21:18.352 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.348+0000 7ff8f1f6f700 1 --2- 172.21.15.99:0/2163402800 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff8d0066f30 0x7ff8d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:18.352 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.349+0000 7ff8f1f6f700 1 --2- 172.21.15.99:0/2163402800 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff8d0066f30 0x7ff8d00693e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ff8e80097c0 tx=0x7ff8e8019000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:18.353 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.352+0000 7ff8e1ffb700 1 -- 172.21.15.99:0/2163402800 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff8e4017070 con 0x7ff8ec1438f0 2023-10-08T00:21:18.579 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.579+0000 7ff8f2f71700 1 -- 172.21.15.99:0/2163402800 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8ec15b210 con 0x7ff8ec1438f0 2023-10-08T00:21:18.580 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.579+0000 7ff8e1ffb700 1 -- 172.21.15.99:0/2163402800 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4601 (secure 0 0 0) 0x7ff8e4050d30 con 0x7ff8ec1438f0 2023-10-08T00:21:18.581 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:21:18.581 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":77,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6257,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/661016677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":661016677},{"type":"v1","addr":"172.21.15.191:6837","nonce":661016677}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":7012,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/4102051559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":4102051559},{"type":"v1","addr":"172.21.15.99:6835","nonce":4102051559}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":7021,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/658854389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":658854389},{"type":"v1","addr":"172.21.15.191:6836","nonce":658854389}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:20:04.613994+0000","modified":"2023-10-08T00:20:08.255350+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7791},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7791":{"gid":7791,"name":"c","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6834/3880334948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3880334948},{"type":"v1","addr":"172.21.15.99:6834","nonce":3880334948}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-08T00:21:18.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.581+0000 7ff8f2f71700 1 -- 172.21.15.99:0/2163402800 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff8d0066f30 msgr2=0x7ff8d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:18.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.581+0000 7ff8f2f71700 1 --2- 172.21.15.99:0/2163402800 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff8d0066f30 0x7ff8d00693e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ff8e80097c0 tx=0x7ff8e8019000 comp rx=0 tx=0).stop 2023-10-08T00:21:18.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.581+0000 7ff8f2f71700 1 -- 172.21.15.99:0/2163402800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff8ec1438f0 msgr2=0x7ff8ec16daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:18.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.581+0000 7ff8f2f71700 1 --2- 172.21.15.99:0/2163402800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff8ec1438f0 0x7ff8ec16daf0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7ff8e400c7e0 tx=0x7ff8e4007d90 comp rx=0 tx=0).stop 2023-10-08T00:21:18.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.581+0000 7ff8f2f71700 1 -- 172.21.15.99:0/2163402800 shutdown_connections 2023-10-08T00:21:18.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.581+0000 7ff8f2f71700 1 --2- 172.21.15.99:0/2163402800 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff8d0066f30 0x7ff8d00693e0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:18.586 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.581+0000 7ff8f2f71700 1 --2- 172.21.15.99:0/2163402800 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8ec0ccf00 0x7ff8ec0c8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:18.586 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.581+0000 7ff8f2f71700 1 --2- 172.21.15.99:0/2163402800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff8ec1438f0 0x7ff8ec16daf0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:18.587 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.581+0000 7ff8f2f71700 1 --2- 172.21.15.99:0/2163402800 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8ec143520 0x7ff8ec159500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:18.587 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.581+0000 7ff8f2f71700 1 -- 172.21.15.99:0/2163402800 >> 172.21.15.99:0/2163402800 conn(0x7ff8ec069750 msgr2=0x7ff8ec140080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:18.587 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.581+0000 7ff8f2f71700 1 -- 172.21.15.99:0/2163402800 shutdown_connections 2023-10-08T00:21:18.587 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.581+0000 7ff8f2f71700 1 -- 172.21.15.99:0/2163402800 wait complete. 2023-10-08T00:21:18.587 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 77 2023-10-08T00:21:18.594 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:21:18.710 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.707+0000 7f6aaefa5700 1 Processor -- start 2023-10-08T00:21:18.710 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.707+0000 7f6aaefa5700 1 -- start start 2023-10-08T00:21:18.710 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.708+0000 7f6aaefa5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6aa80afff0 0x7f6aa815e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:18.711 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.708+0000 7f6aaefa5700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6aa8069e20 0x7f6aa806a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:18.711 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.708+0000 7f6aaefa5700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aa80b03c0 con 0x7f6aa8070700 2023-10-08T00:21:18.711 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.708+0000 7f6aaefa5700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6aa806a870 con 0x7f6aa8069e20 2023-10-08T00:21:18.711 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.708+0000 7f6aaefa5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa806ab50 con 0x7f6aa80afff0 2023-10-08T00:21:18.711 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.708+0000 7f6a9ffff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6aa8069e20 0x7f6aa806a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:18.711 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.708+0000 7f6aad542700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f6aa8070700 0x7f6aa806ea90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:46730/0 (socket says 172.21.15.99:46730) 2023-10-08T00:21:18.711 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.708+0000 7f6aad542700 1 -- 172.21.15.99:0/3245129765 learned_addr learned my addr 172.21.15.99:0/3245129765 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:21:18.711 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.708+0000 7f6aacd41700 1 --2- 172.21.15.99:0/3245129765 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6aa80afff0 0x7f6aa815e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:18.711 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.709+0000 7f6a9ffff700 1 -- 172.21.15.99:0/3245129765 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6aa80afff0 msgr2=0x7f6aa815e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:18.711 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.709+0000 7f6a9ffff700 1 --2- 172.21.15.99:0/3245129765 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6aa80afff0 0x7f6aa815e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:18.711 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.709+0000 7f6a9ffff700 1 -- 172.21.15.99:0/3245129765 >> v1:172.21.15.99:6789/0 conn(0x7f6aa8070700 legacy=0x7f6aa806ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:18.712 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.709+0000 7f6a9f7fe700 1 -- 172.21.15.99:0/3245129765 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1869753928 0 0) 0x7f6aa80b03c0 con 0x7f6aa8070700 2023-10-08T00:21:18.712 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.709+0000 7f6a9ffff700 1 -- 172.21.15.99:0/3245129765 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa815da60 con 0x7f6aa8069e20 2023-10-08T00:21:18.712 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.709+0000 7f6a9ffff700 1 --2- 172.21.15.99:0/3245129765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6aa8069e20 0x7f6aa806a290 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f6a980098e0 tx=0x7f6a980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17deed28f9cce2fd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:18.712 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.709+0000 7f6a9f7fe700 1 -- 172.21.15.99:0/3245129765 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9800b640 con 0x7f6aa8069e20 2023-10-08T00:21:18.712 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.709+0000 7f6a9f7fe700 1 -- 172.21.15.99:0/3245129765 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a9800b7a0 con 0x7f6aa8069e20 2023-10-08T00:21:18.712 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.709+0000 7f6a9f7fe700 1 -- 172.21.15.99:0/3245129765 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9800bb40 con 0x7f6aa8069e20 2023-10-08T00:21:18.712 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.709+0000 7f6aaefa5700 1 -- 172.21.15.99:0/3245129765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6aa8069e20 msgr2=0x7f6aa806a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:18.712 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.709+0000 7f6aaefa5700 1 --2- 172.21.15.99:0/3245129765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6aa8069e20 0x7f6aa806a290 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f6a980098e0 tx=0x7f6a980091a0 comp rx=0 tx=0).stop 2023-10-08T00:21:18.712 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.709+0000 7f6aaefa5700 1 -- 172.21.15.99:0/3245129765 shutdown_connections 2023-10-08T00:21:18.712 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.709+0000 7f6aaefa5700 1 --2- 172.21.15.99:0/3245129765 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6aa80afff0 0x7f6aa815e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:18.712 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.709+0000 7f6aaefa5700 1 --2- 172.21.15.99:0/3245129765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6aa8069e20 0x7f6aa806a290 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:18.712 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.710+0000 7f6aaefa5700 1 -- 172.21.15.99:0/3245129765 >> 172.21.15.99:0/3245129765 conn(0x7f6aa80cda50 msgr2=0x7f6aa80cde50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:18.713 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.710+0000 7f6aaefa5700 1 -- 172.21.15.99:0/3245129765 shutdown_connections 2023-10-08T00:21:18.713 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.710+0000 7f6aaefa5700 1 -- 172.21.15.99:0/3245129765 wait complete. 2023-10-08T00:21:18.716 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.711+0000 7f6aaefa5700 1 Processor -- start 2023-10-08T00:21:18.716 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.711+0000 7f6aaefa5700 1 -- start start 2023-10-08T00:21:18.716 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.711+0000 7f6aaefa5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6aa8069e20 0x7f6aa81584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:18.716 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.711+0000 7f6aaefa5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6aa8070700 0x7f6aa80c6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:18.716 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.711+0000 7f6aaefa5700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6aa80afff0 0x7f6aa80c65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:18.716 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.711+0000 7f6aaefa5700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa81589e0 con 0x7f6aa8070700 2023-10-08T00:21:18.716 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.711+0000 7f6aaefa5700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa80b3520 con 0x7f6aa80afff0 2023-10-08T00:21:18.716 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.711+0000 7f6aaefa5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa80b2910 con 0x7f6aa8069e20 2023-10-08T00:21:18.716 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.712+0000 7f6a9ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6aa8070700 0x7f6aa80c6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:18.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.712+0000 7f6a9ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6aa8070700 0x7f6aa80c6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57420/0 (socket says 172.21.15.99:57420) 2023-10-08T00:21:18.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.712+0000 7f6a9ffff700 1 -- 172.21.15.99:0/196417971 learned_addr learned my addr 172.21.15.99:0/196417971 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:18.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.712+0000 7f6aad542700 1 --2- 172.21.15.99:0/196417971 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6aa80afff0 0x7f6aa80c65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:18.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.712+0000 7f6aacd41700 1 --2- 172.21.15.99:0/196417971 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6aa8069e20 0x7f6aa81584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:18.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.712+0000 7f6a9ffff700 1 -- 172.21.15.99:0/196417971 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6aa8069e20 msgr2=0x7f6aa81584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:18.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.712+0000 7f6a9ffff700 1 --2- 172.21.15.99:0/196417971 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6aa8069e20 0x7f6aa81584a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:18.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.712+0000 7f6a9ffff700 1 -- 172.21.15.99:0/196417971 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6aa80afff0 msgr2=0x7f6aa80c65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:18.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.712+0000 7f6a9ffff700 1 --2- 172.21.15.99:0/196417971 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6aa80afff0 0x7f6aa80c65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:18.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.712+0000 7f6a9ffff700 1 -- 172.21.15.99:0/196417971 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a98009420 con 0x7f6aa8070700 2023-10-08T00:21:18.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.712+0000 7f6aacd41700 1 --2- 172.21.15.99:0/196417971 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6aa8069e20 0x7f6aa81584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:21:18.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.712+0000 7f6a9ffff700 1 --2- 172.21.15.99:0/196417971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6aa8070700 0x7f6aa80c6080 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f6a980093f0 tx=0x7f6a98017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:18.717 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.712+0000 7f6a9cff9700 1 -- 172.21.15.99:0/196417971 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9800bb20 con 0x7f6aa8070700 2023-10-08T00:21:18.718 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.712+0000 7f6a9cff9700 1 -- 172.21.15.99:0/196417971 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a98011420 con 0x7f6aa8070700 2023-10-08T00:21:18.718 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.712+0000 7f6a9cff9700 1 -- 172.21.15.99:0/196417971 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a98011800 con 0x7f6aa8070700 2023-10-08T00:21:18.718 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.712+0000 7f6aaefa5700 1 -- 172.21.15.99:0/196417971 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aa80b2bf0 con 0x7f6aa8070700 2023-10-08T00:21:18.718 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.712+0000 7f6aaefa5700 1 -- 172.21.15.99:0/196417971 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aa80b31b0 con 0x7f6aa8070700 2023-10-08T00:21:18.718 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.713+0000 7f6aaefa5700 1 -- 172.21.15.99:0/196417971 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aa804c3c0 con 0x7f6aa8070700 2023-10-08T00:21:18.718 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.713+0000 7f6a9cff9700 1 -- 172.21.15.99:0/196417971 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6a98011c30 con 0x7f6aa8070700 2023-10-08T00:21:18.718 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.714+0000 7f6a9cff9700 1 --2- 172.21.15.99:0/196417971 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6a94066f30 0x7f6a940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:18.719 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.714+0000 7f6a9cff9700 1 -- 172.21.15.99:0/196417971 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f6a98085aa0 con 0x7f6aa8070700 2023-10-08T00:21:18.719 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.714+0000 7f6aacd41700 1 --2- 172.21.15.99:0/196417971 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6a94066f30 0x7f6a940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:18.719 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.714+0000 7f6a9cff9700 1 -- 172.21.15.99:0/196417971 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=173}) v3 -- 0x7f6a9406eb10 con 0x7f6aa8070700 2023-10-08T00:21:18.719 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.714+0000 7f6aacd41700 1 --2- 172.21.15.99:0/196417971 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6a94066f30 0x7f6a940693e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f6aa40076c0 tx=0x7f6aa400b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:18.719 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.718+0000 7f6a9cff9700 1 -- 172.21.15.99:0/196417971 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6a98052cd0 con 0x7f6aa8070700 2023-10-08T00:21:18.903 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.902+0000 7f6aaefa5700 1 -- 172.21.15.99:0/196417971 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6aa80b3660 con 0x7f6aa8070700 2023-10-08T00:21:18.903 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.903+0000 7f6a9cff9700 1 -- 172.21.15.99:0/196417971 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v172) v1 ==== 74+0+15488 (secure 0 0 0) 0x7f6a98028070 con 0x7f6aa8070700 2023-10-08T00:21:18.904 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:21:18.904 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":172,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:20:51.158163+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:20:02.932463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-08T00:20:03.578010+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.6","mappings":[{"from":2,"to":5}]},{"pgid":"21.10","mappings":[{"from":2,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:21:18.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.905+0000 7f6aaefa5700 1 -- 172.21.15.99:0/196417971 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6a94066f30 msgr2=0x7f6a940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:18.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.905+0000 7f6aaefa5700 1 --2- 172.21.15.99:0/196417971 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6a94066f30 0x7f6a940693e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f6aa40076c0 tx=0x7f6aa400b040 comp rx=0 tx=0).stop 2023-10-08T00:21:18.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.905+0000 7f6aaefa5700 1 -- 172.21.15.99:0/196417971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6aa8070700 msgr2=0x7f6aa80c6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:18.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.905+0000 7f6aaefa5700 1 --2- 172.21.15.99:0/196417971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6aa8070700 0x7f6aa80c6080 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f6a980093f0 tx=0x7f6a98017000 comp rx=0 tx=0).stop 2023-10-08T00:21:18.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.905+0000 7f6aaefa5700 1 -- 172.21.15.99:0/196417971 shutdown_connections 2023-10-08T00:21:18.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.905+0000 7f6aaefa5700 1 --2- 172.21.15.99:0/196417971 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6a94066f30 0x7f6a940693e0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:18.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.905+0000 7f6aaefa5700 1 --2- 172.21.15.99:0/196417971 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6aa80afff0 0x7f6aa80c65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:18.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.905+0000 7f6aaefa5700 1 --2- 172.21.15.99:0/196417971 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6aa8069e20 0x7f6aa81584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:18.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.905+0000 7f6aaefa5700 1 --2- 172.21.15.99:0/196417971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6aa8070700 0x7f6aa80c6080 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:18.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.905+0000 7f6aaefa5700 1 -- 172.21.15.99:0/196417971 >> 172.21.15.99:0/196417971 conn(0x7f6aa80cda50 msgr2=0x7f6aa8070ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:18.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.905+0000 7f6aaefa5700 1 -- 172.21.15.99:0/196417971 shutdown_connections 2023-10-08T00:21:18.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:18.905+0000 7f6aaefa5700 1 -- 172.21.15.99:0/196417971 wait complete. 2023-10-08T00:21:18.918 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T00:21:19.033 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.032+0000 7fb5fb0b1700 1 Processor -- start 2023-10-08T00:21:19.033 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.032+0000 7fb5fb0b1700 1 -- start start 2023-10-08T00:21:19.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.033+0000 7fb5fb0b1700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb5f4157870 0x7fb5f415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:19.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.033+0000 7fb5fb0b1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb5f415a620 0x7fb5f415d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:19.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.033+0000 7fb5fb0b1700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb5f415d580 0x7fb5f4158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:19.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.033+0000 7fb5fb0b1700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5f415cbb0 con 0x7fb5f4157870 2023-10-08T00:21:19.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.033+0000 7fb5fb0b1700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5f415bfa0 con 0x7fb5f415d580 2023-10-08T00:21:19.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.033+0000 7fb5fb0b1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f415c280 con 0x7fb5f415a620 2023-10-08T00:21:19.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.033+0000 7fb5f8e4d700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb5f4157870 0x7fb5f415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:19.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.033+0000 7fb5f8e4d700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb5f4157870 0x7fb5f415b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57430/0 (socket says 172.21.15.99:57430) 2023-10-08T00:21:19.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.033+0000 7fb5f8e4d700 1 -- 172.21.15.99:0/3290773932 learned_addr learned my addr 172.21.15.99:0/3290773932 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:19.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.033+0000 7fb5f964e700 1 --2- 172.21.15.99:0/3290773932 >> v2:172.21.15.191:3300/0 conn(0x7fb5f415d580 0x7fb5f4158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:19.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.033+0000 7fb5ebfff700 1 --2- 172.21.15.99:0/3290773932 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb5f415a620 0x7fb5f415d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:19.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.033+0000 7fb5f8e4d700 1 -- 172.21.15.99:0/3290773932 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb5f415a620 msgr2=0x7fb5f415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:19.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.033+0000 7fb5f8e4d700 1 --2- 172.21.15.99:0/3290773932 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb5f415a620 0x7fb5f415d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:19.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.033+0000 7fb5f8e4d700 1 -- 172.21.15.99:0/3290773932 >> v2:172.21.15.191:3300/0 conn(0x7fb5f415d580 msgr2=0x7fb5f4158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:19.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.033+0000 7fb5f8e4d700 1 --2- 172.21.15.99:0/3290773932 >> v2:172.21.15.191:3300/0 conn(0x7fb5f415d580 0x7fb5f4158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:19.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.033+0000 7fb5f8e4d700 1 -- 172.21.15.99:0/3290773932 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f415c560 con 0x7fb5f4157870 2023-10-08T00:21:19.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.034+0000 7fb5f8e4d700 1 --2- 172.21.15.99:0/3290773932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5f4157870 0x7fb5f415b6a0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fb5f40b03b0 tx=0x7fb5f0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4b299e480642646 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:19.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.034+0000 7fb5eb7fe700 1 -- 172.21.15.99:0/3290773932 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f000e070 con 0x7fb5f4157870 2023-10-08T00:21:19.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.034+0000 7fb5eb7fe700 1 -- 172.21.15.99:0/3290773932 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f000ba50 con 0x7fb5f4157870 2023-10-08T00:21:19.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.034+0000 7fb5eb7fe700 1 -- 172.21.15.99:0/3290773932 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f00092e0 con 0x7fb5f4157870 2023-10-08T00:21:19.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.034+0000 7fb5fb0b1700 1 -- 172.21.15.99:0/3290773932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5f4157870 msgr2=0x7fb5f415b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:19.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.034+0000 7fb5fb0b1700 1 --2- 172.21.15.99:0/3290773932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5f4157870 0x7fb5f415b6a0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fb5f40b03b0 tx=0x7fb5f0009840 comp rx=0 tx=0).stop 2023-10-08T00:21:19.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.034+0000 7fb5fb0b1700 1 -- 172.21.15.99:0/3290773932 shutdown_connections 2023-10-08T00:21:19.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.034+0000 7fb5fb0b1700 1 --2- 172.21.15.99:0/3290773932 >> v2:172.21.15.191:3300/0 conn(0x7fb5f415d580 0x7fb5f4158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:19.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.034+0000 7fb5fb0b1700 1 --2- 172.21.15.99:0/3290773932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5f4157870 0x7fb5f415b6a0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:19.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.034+0000 7fb5fb0b1700 1 --2- 172.21.15.99:0/3290773932 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb5f415a620 0x7fb5f415d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:19.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.034+0000 7fb5fb0b1700 1 -- 172.21.15.99:0/3290773932 >> 172.21.15.99:0/3290773932 conn(0x7fb5f4068ce0 msgr2=0x7fb5f40690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:19.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.034+0000 7fb5fb0b1700 1 -- 172.21.15.99:0/3290773932 shutdown_connections 2023-10-08T00:21:19.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.034+0000 7fb5fb0b1700 1 -- 172.21.15.99:0/3290773932 wait complete. 2023-10-08T00:21:19.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.035+0000 7fb5fb0b1700 1 Processor -- start 2023-10-08T00:21:19.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.036+0000 7fb5fb0b1700 1 -- start start 2023-10-08T00:21:19.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.036+0000 7fb5fb0b1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5f4157870 0x7fb5f4149890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:19.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.036+0000 7fb5fb0b1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5f415a620 0x7fb5f40b6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:19.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.036+0000 7fb5fb0b1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb5f415d580 0x7fb5f414a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:19.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.036+0000 7fb5fb0b1700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f4154940 con 0x7fb5f415a620 2023-10-08T00:21:19.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.036+0000 7fb5fb0b1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f4153d70 con 0x7fb5f4157870 2023-10-08T00:21:19.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.036+0000 7fb5fb0b1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f4154010 con 0x7fb5f415d580 2023-10-08T00:21:19.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.036+0000 7fb5ebfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5f415a620 0x7fb5f40b6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:19.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.037+0000 7fb5ebfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5f415a620 0x7fb5f40b6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57440/0 (socket says 172.21.15.99:57440) 2023-10-08T00:21:19.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.037+0000 7fb5ebfff700 1 -- 172.21.15.99:0/1248550987 learned_addr learned my addr 172.21.15.99:0/1248550987 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:19.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.037+0000 7fb5f964e700 1 --2- 172.21.15.99:0/1248550987 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb5f415d580 0x7fb5f414a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:19.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.037+0000 7fb5f8e4d700 1 --2- 172.21.15.99:0/1248550987 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5f4157870 0x7fb5f4149890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:19.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.037+0000 7fb5ebfff700 1 -- 172.21.15.99:0/1248550987 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb5f415d580 msgr2=0x7fb5f414a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:19.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.037+0000 7fb5ebfff700 1 --2- 172.21.15.99:0/1248550987 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb5f415d580 0x7fb5f414a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:19.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.037+0000 7fb5ebfff700 1 -- 172.21.15.99:0/1248550987 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5f4157870 msgr2=0x7fb5f4149890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:19.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.037+0000 7fb5ebfff700 1 --2- 172.21.15.99:0/1248550987 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5f4157870 0x7fb5f4149890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:19.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.037+0000 7fb5ebfff700 1 -- 172.21.15.99:0/1248550987 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f0009ae0 con 0x7fb5f415a620 2023-10-08T00:21:19.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.037+0000 7fb5ebfff700 1 --2- 172.21.15.99:0/1248550987 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5f415a620 0x7fb5f40b6550 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fb5e400d860 tx=0x7fb5e4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:19.047 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.037+0000 7fb5e8ff9700 1 -- 172.21.15.99:0/1248550987 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e40125f0 con 0x7fb5f415a620 2023-10-08T00:21:19.048 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.037+0000 7fb5e8ff9700 1 -- 172.21.15.99:0/1248550987 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e4012750 con 0x7fb5f415a620 2023-10-08T00:21:19.048 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.037+0000 7fb5fb0b1700 1 -- 172.21.15.99:0/1248550987 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5f41542f0 con 0x7fb5f415a620 2023-10-08T00:21:19.048 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.037+0000 7fb5e8ff9700 1 -- 172.21.15.99:0/1248550987 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e4012af0 con 0x7fb5f415a620 2023-10-08T00:21:19.048 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.037+0000 7fb5fb0b1700 1 -- 172.21.15.99:0/1248550987 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5f4149dd0 con 0x7fb5f415a620 2023-10-08T00:21:19.048 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.038+0000 7fb5fb0b1700 1 -- 172.21.15.99:0/1248550987 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5f404c3c0 con 0x7fb5f415a620 2023-10-08T00:21:19.048 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.038+0000 7fb5e8ff9700 1 -- 172.21.15.99:0/1248550987 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb5e4013480 con 0x7fb5f415a620 2023-10-08T00:21:19.048 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.039+0000 7fb5e8ff9700 1 --2- 172.21.15.99:0/1248550987 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb5e0066f30 0x7fb5e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:19.048 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.039+0000 7fb5e8ff9700 1 -- 172.21.15.99:0/1248550987 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5763+0+0 (secure 0 0 0) 0x7fb5e40848f0 con 0x7fb5f415a620 2023-10-08T00:21:19.048 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.039+0000 7fb5e8ff9700 1 -- 172.21.15.99:0/1248550987 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=173}) v3 -- 0x7fb5e006eb10 con 0x7fb5f415a620 2023-10-08T00:21:19.048 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.039+0000 7fb5f8e4d700 1 --2- 172.21.15.99:0/1248550987 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb5e0066f30 0x7fb5e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:19.048 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.039+0000 7fb5f8e4d700 1 --2- 172.21.15.99:0/1248550987 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb5e0066f30 0x7fb5e00693e0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0009880 tx=0x7fb5f0018000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:19.049 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.043+0000 7fb5e8ff9700 1 -- 172.21.15.99:0/1248550987 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb5e40555b0 con 0x7fb5f415a620 2023-10-08T00:21:19.267 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.266+0000 7fb5fb0b1700 1 -- 172.21.15.99:0/1248550987 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb5f415c840 con 0x7fb5f415a620 2023-10-08T00:21:19.267 INFO:tasks.ceph.mon.a.smithi099.stderr:2023-10-08T00:21:19.267+0000 7f6c60fa2700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T00:21:19.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.271+0000 7fb5e8ff9700 1 -- 172.21.15.99:0/1248550987 <== mon.0 v2:172.21.15.99:3300/0 7 ==== osd_map(173..173 src has 1..173) v4 ==== 366+0+0 (secure 0 0 0) 0x7fb5e4012ea0 con 0x7fb5f415a620 2023-10-08T00:21:19.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.271+0000 7fb5e8ff9700 1 -- 172.21.15.99:0/1248550987 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=174}) v3 -- 0x7fb5e006f400 con 0x7fb5f415a620 2023-10-08T00:21:19.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.273+0000 7fb5e8ff9700 1 -- 172.21.15.99:0/1248550987 <== mon.0 v2:172.21.15.99:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v78) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb5e4055140 con 0x7fb5f415a620 2023-10-08T00:21:19.275 INFO:tasks.ceph.mds.c.smithi099.stderr: -8361> 2023-10-08T00:20:10.540+0000 7f61d0334700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T00:21:19.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.276+0000 7fb5fb0b1700 1 -- 172.21.15.99:0/1248550987 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb5e0066f30 msgr2=0x7fb5e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:19.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.276+0000 7fb5fb0b1700 1 --2- 172.21.15.99:0/1248550987 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb5e0066f30 0x7fb5e00693e0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0009880 tx=0x7fb5f0018000 comp rx=0 tx=0).stop 2023-10-08T00:21:19.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.276+0000 7fb5fb0b1700 1 -- 172.21.15.99:0/1248550987 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5f415a620 msgr2=0x7fb5f40b6550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:19.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.276+0000 7fb5fb0b1700 1 --2- 172.21.15.99:0/1248550987 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5f415a620 0x7fb5f40b6550 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fb5e400d860 tx=0x7fb5e4008040 comp rx=0 tx=0).stop 2023-10-08T00:21:19.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.276+0000 7fb5fb0b1700 1 -- 172.21.15.99:0/1248550987 shutdown_connections 2023-10-08T00:21:19.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.276+0000 7fb5fb0b1700 1 --2- 172.21.15.99:0/1248550987 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb5e0066f30 0x7fb5e00693e0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:19.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.276+0000 7fb5fb0b1700 1 --2- 172.21.15.99:0/1248550987 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5f4157870 0x7fb5f4149890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:19.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.276+0000 7fb5fb0b1700 1 --2- 172.21.15.99:0/1248550987 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb5f415d580 0x7fb5f414a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:19.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.276+0000 7fb5fb0b1700 1 --2- 172.21.15.99:0/1248550987 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5f415a620 0x7fb5f40b6550 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:19.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.276+0000 7fb5fb0b1700 1 -- 172.21.15.99:0/1248550987 >> 172.21.15.99:0/1248550987 conn(0x7fb5f4068ce0 msgr2=0x7fb5f4159b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:19.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.276+0000 7fb5fb0b1700 1 -- 172.21.15.99:0/1248550987 shutdown_connections 2023-10-08T00:21:19.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.276+0000 7fb5fb0b1700 1 -- 172.21.15.99:0/1248550987 wait complete. 2023-10-08T00:21:19.278 INFO:teuthology.orchestra.run.smithi099.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T00:21:19.289 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T00:21:19.304 INFO:tasks.ceph.mds.c.smithi099.stdout:starting mds.c at 2023-10-08T00:21:19.406 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.406+0000 7fcee7001700 1 Processor -- start 2023-10-08T00:21:19.407 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.406+0000 7fcee7001700 1 -- start start 2023-10-08T00:21:19.409 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.407+0000 7fcee7001700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fcee00cce30 0x7fcee015aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:19.409 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.407+0000 7fcee7001700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fcee006aa00 0x7fcee00b5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:19.409 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.407+0000 7fcee7001700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcee00b4d70 con 0x7fcee00b6580 2023-10-08T00:21:19.409 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.407+0000 7fcee7001700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcee00cd8d0 con 0x7fcee006aa00 2023-10-08T00:21:19.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.407+0000 7fcee7001700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcee00b4440 con 0x7fcee00cce30 2023-10-08T00:21:19.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.407+0000 7fcee6800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fcee006aa00 0x7fcee00b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:19.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.407+0000 7fcee6800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fcee006aa00 0x7fcee00b5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57478/0 (socket says 172.21.15.99:57478) 2023-10-08T00:21:19.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.407+0000 7fcee6800700 1 -- 172.21.15.99:0/376134110 learned_addr learned my addr 172.21.15.99:0/376134110 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:19.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.407+0000 7fcee5fff700 1 --2- 172.21.15.99:0/376134110 >> v2:172.21.15.191:3300/0 conn(0x7fcee00cce30 0x7fcee015aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:19.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.407+0000 7fcee6800700 1 -- 172.21.15.99:0/376134110 >> v2:172.21.15.191:3300/0 conn(0x7fcee00cce30 msgr2=0x7fcee015aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:19.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.407+0000 7fcee6800700 1 --2- 172.21.15.99:0/376134110 >> v2:172.21.15.191:3300/0 conn(0x7fcee00cce30 0x7fcee015aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:19.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.407+0000 7fcee6800700 1 -- 172.21.15.99:0/376134110 >> v1:172.21.15.191:6789/0 conn(0x7fcee00b6580 legacy=0x7fcee006a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:19.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.407+0000 7fcee6800700 1 -- 172.21.15.99:0/376134110 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee00b4720 con 0x7fcee006aa00 2023-10-08T00:21:19.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.408+0000 7fcee6800700 1 --2- 172.21.15.99:0/376134110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcee006aa00 0x7fcee00b5200 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fcee00b3770 tx=0x7fced8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6669e1d5c92d399 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:19.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.408+0000 7fcee4ffd700 1 -- 172.21.15.99:0/376134110 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced800e070 con 0x7fcee006aa00 2023-10-08T00:21:19.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.408+0000 7fcee4ffd700 1 -- 172.21.15.99:0/376134110 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fced800ba50 con 0x7fcee006aa00 2023-10-08T00:21:19.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.408+0000 7fcee4ffd700 1 -- 172.21.15.99:0/376134110 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced80092e0 con 0x7fcee006aa00 2023-10-08T00:21:19.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.408+0000 7fcee7001700 1 -- 172.21.15.99:0/376134110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcee006aa00 msgr2=0x7fcee00b5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:19.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.408+0000 7fcee7001700 1 --2- 172.21.15.99:0/376134110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcee006aa00 0x7fcee00b5200 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fcee00b3770 tx=0x7fced8009840 comp rx=0 tx=0).stop 2023-10-08T00:21:19.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.408+0000 7fcee7001700 1 -- 172.21.15.99:0/376134110 shutdown_connections 2023-10-08T00:21:19.412 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.408+0000 7fcee7001700 1 --2- 172.21.15.99:0/376134110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcee006aa00 0x7fcee00b5200 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:19.412 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.408+0000 7fcee7001700 1 --2- 172.21.15.99:0/376134110 >> v2:172.21.15.191:3300/0 conn(0x7fcee00cce30 0x7fcee015aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:19.412 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.408+0000 7fcee7001700 1 -- 172.21.15.99:0/376134110 >> 172.21.15.99:0/376134110 conn(0x7fcee0068cb0 msgr2=0x7fcee00690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:19.412 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.408+0000 7fcee7001700 1 -- 172.21.15.99:0/376134110 shutdown_connections 2023-10-08T00:21:19.412 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.409+0000 7fcee7001700 1 -- 172.21.15.99:0/376134110 wait complete. 2023-10-08T00:21:19.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.410+0000 7fcee7001700 1 Processor -- start 2023-10-08T00:21:19.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.410+0000 7fcee7001700 1 -- start start 2023-10-08T00:21:19.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.410+0000 7fcee7001700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcee006aa00 0x7fcee0153b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:19.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.410+0000 7fcee7001700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcee00b6580 0x7fcee014fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:19.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.410+0000 7fcee7001700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcee00cce30 0x7fcee00b1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:19.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.410+0000 7fcee7001700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee01578c0 con 0x7fcee006aa00 2023-10-08T00:21:19.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.410+0000 7fcee7001700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee0151770 con 0x7fcee00cce30 2023-10-08T00:21:19.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.410+0000 7fcee7001700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcee0151a50 con 0x7fcee00b6580 2023-10-08T00:21:19.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.410+0000 7fcee5fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcee006aa00 0x7fcee0153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:19.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.410+0000 7fcee5fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcee006aa00 0x7fcee0153b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57486/0 (socket says 172.21.15.99:57486) 2023-10-08T00:21:19.416 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.410+0000 7fcee5fff700 1 -- 172.21.15.99:0/3381522618 learned_addr learned my addr 172.21.15.99:0/3381522618 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:19.416 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.411+0000 7fcee6800700 1 --2- 172.21.15.99:0/3381522618 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcee00cce30 0x7fcee00b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:19.416 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.411+0000 7fcee57fe700 1 --2- 172.21.15.99:0/3381522618 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcee00b6580 0x7fcee014fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:19.416 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.411+0000 7fcee5fff700 1 -- 172.21.15.99:0/3381522618 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcee00b6580 msgr2=0x7fcee014fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:19.416 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.411+0000 7fcee5fff700 1 --2- 172.21.15.99:0/3381522618 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcee00b6580 0x7fcee014fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:19.416 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.411+0000 7fcee5fff700 1 -- 172.21.15.99:0/3381522618 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcee00cce30 msgr2=0x7fcee00b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:19.416 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.411+0000 7fcee5fff700 1 --2- 172.21.15.99:0/3381522618 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcee00cce30 0x7fcee00b1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:19.416 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.411+0000 7fcee5fff700 1 -- 172.21.15.99:0/3381522618 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced8009ae0 con 0x7fcee006aa00 2023-10-08T00:21:19.416 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.411+0000 7fcee6800700 1 --2- 172.21.15.99:0/3381522618 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcee00cce30 0x7fcee00b1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:21:19.416 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.411+0000 7fcee5fff700 1 --2- 172.21.15.99:0/3381522618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcee006aa00 0x7fcee0153b20 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fcedc00d860 tx=0x7fcedc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:19.416 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.411+0000 7fced5ffb700 1 -- 172.21.15.99:0/3381522618 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcedc009610 con 0x7fcee006aa00 2023-10-08T00:21:19.416 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.411+0000 7fced5ffb700 1 -- 172.21.15.99:0/3381522618 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcedc009770 con 0x7fcee006aa00 2023-10-08T00:21:19.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.411+0000 7fcee7001700 1 -- 172.21.15.99:0/3381522618 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcee0151d30 con 0x7fcee006aa00 2023-10-08T00:21:19.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.411+0000 7fced5ffb700 1 -- 172.21.15.99:0/3381522618 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcedc009b30 con 0x7fcee006aa00 2023-10-08T00:21:19.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.411+0000 7fcee7001700 1 -- 172.21.15.99:0/3381522618 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcee0154060 con 0x7fcee006aa00 2023-10-08T00:21:19.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.412+0000 7fcee7001700 1 -- 172.21.15.99:0/3381522618 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcee0089550 con 0x7fcee006aa00 2023-10-08T00:21:19.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.413+0000 7fced5ffb700 1 -- 172.21.15.99:0/3381522618 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcedc01e460 con 0x7fcee006aa00 2023-10-08T00:21:19.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.413+0000 7fced5ffb700 1 --2- 172.21.15.99:0/3381522618 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcec4066f30 0x7fcec40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:19.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.413+0000 7fced5ffb700 1 -- 172.21.15.99:0/3381522618 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5849+0+0 (secure 0 0 0) 0x7fcedc084bb0 con 0x7fcee006aa00 2023-10-08T00:21:19.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.413+0000 7fced5ffb700 1 -- 172.21.15.99:0/3381522618 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=174}) v3 -- 0x7fcec406ebb0 con 0x7fcee006aa00 2023-10-08T00:21:19.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.413+0000 7fcee57fe700 1 --2- 172.21.15.99:0/3381522618 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcec4066f30 0x7fcec40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:19.417 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.414+0000 7fcee57fe700 1 --2- 172.21.15.99:0/3381522618 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcec4066f30 0x7fcec40693e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fcee006a6e0 tx=0x7fced0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:19.418 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.417+0000 7fced5ffb700 1 -- 172.21.15.99:0/3381522618 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcedc051d90 con 0x7fcee006aa00 2023-10-08T00:21:19.641 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:19.641+0000 7fcee7001700 1 -- 172.21.15.99:0/3381522618 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fcee00b4a00 con 0x7fcee006aa00 2023-10-08T00:21:20.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.273+0000 7fced5ffb700 1 -- 172.21.15.99:0/3381522618 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v79) v1 ==== 104+0+0 (secure 0 0 0) 0x7fcedc0553b0 con 0x7fcee006aa00 2023-10-08T00:21:20.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.275+0000 7fcee7001700 1 -- 172.21.15.99:0/3381522618 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcec4066f30 msgr2=0x7fcec40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:20.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.275+0000 7fcee7001700 1 --2- 172.21.15.99:0/3381522618 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcec4066f30 0x7fcec40693e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fcee006a6e0 tx=0x7fced0009040 comp rx=0 tx=0).stop 2023-10-08T00:21:20.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.275+0000 7fcee7001700 1 -- 172.21.15.99:0/3381522618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcee006aa00 msgr2=0x7fcee0153b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:20.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.275+0000 7fcee7001700 1 --2- 172.21.15.99:0/3381522618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcee006aa00 0x7fcee0153b20 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fcedc00d860 tx=0x7fcedc008040 comp rx=0 tx=0).stop 2023-10-08T00:21:20.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.275+0000 7fcee7001700 1 -- 172.21.15.99:0/3381522618 shutdown_connections 2023-10-08T00:21:20.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.275+0000 7fcee7001700 1 --2- 172.21.15.99:0/3381522618 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcec4066f30 0x7fcec40693e0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:20.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.275+0000 7fcee7001700 1 --2- 172.21.15.99:0/3381522618 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcee00cce30 0x7fcee00b1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:20.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.275+0000 7fcee7001700 1 --2- 172.21.15.99:0/3381522618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcee006aa00 0x7fcee0153b20 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:20.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.275+0000 7fcee7001700 1 --2- 172.21.15.99:0/3381522618 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcee00b6580 0x7fcee014fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:20.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.275+0000 7fcee7001700 1 -- 172.21.15.99:0/3381522618 >> 172.21.15.99:0/3381522618 conn(0x7fcee0068cb0 msgr2=0x7fcee0150730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:20.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.276+0000 7fcee7001700 1 -- 172.21.15.99:0/3381522618 shutdown_connections 2023-10-08T00:21:20.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.276+0000 7fcee7001700 1 -- 172.21.15.99:0/3381522618 wait complete. 2023-10-08T00:21:20.290 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T00:21:20.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.450+0000 7f51631a2700 1 Processor -- start 2023-10-08T00:21:20.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.450+0000 7f51631a2700 1 -- start start 2023-10-08T00:21:20.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.451+0000 7f51631a2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f515c159ef0 0x7f515c15a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:20.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.451+0000 7f51631a2700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f515c0b46f0 0x7f515c0b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:20.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.451+0000 7f51631a2700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f515c0b5090 con 0x7f515c0b6a10 2023-10-08T00:21:20.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.451+0000 7f51631a2700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f515c0b17b0 con 0x7f515c0b46f0 2023-10-08T00:21:20.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.451+0000 7f51631a2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f515c0b1a60 con 0x7f515c159ef0 2023-10-08T00:21:20.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.451+0000 7f516173f700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f515c0b46f0 0x7f515c0b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:20.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.451+0000 7f516173f700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f515c0b46f0 0x7f515c0b4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57488/0 (socket says 172.21.15.99:57488) 2023-10-08T00:21:20.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.451+0000 7f516173f700 1 -- 172.21.15.99:0/2180333369 learned_addr learned my addr 172.21.15.99:0/2180333369 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:20.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.451+0000 7f5160f3e700 1 --2- 172.21.15.99:0/2180333369 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f515c159ef0 0x7f515c15a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:20.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.451+0000 7f516173f700 1 -- 172.21.15.99:0/2180333369 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f515c159ef0 msgr2=0x7f515c15a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:20.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.451+0000 7f516173f700 1 --2- 172.21.15.99:0/2180333369 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f515c159ef0 0x7f515c15a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:20.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.451+0000 7f516173f700 1 -- 172.21.15.99:0/2180333369 >> v1:172.21.15.191:6789/0 conn(0x7f515c0b6a10 legacy=0x7f515c0b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:20.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.451+0000 7f516173f700 1 -- 172.21.15.99:0/2180333369 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f515c0b1d40 con 0x7f515c0b46f0 2023-10-08T00:21:20.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.451+0000 7f516173f700 1 --2- 172.21.15.99:0/2180333369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f515c0b46f0 0x7f515c0b4b50 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f514c0099b0 tx=0x7f514c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c713ab9a1c82c535 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:20.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.451+0000 7f515b7fe700 1 -- 172.21.15.99:0/2180333369 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514c009ea0 con 0x7f515c0b46f0 2023-10-08T00:21:20.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.451+0000 7f515b7fe700 1 -- 172.21.15.99:0/2180333369 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f514c00b860 con 0x7f515c0b46f0 2023-10-08T00:21:20.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.451+0000 7f515b7fe700 1 -- 172.21.15.99:0/2180333369 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514c00bc00 con 0x7f515c0b46f0 2023-10-08T00:21:20.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.452+0000 7f51631a2700 1 -- 172.21.15.99:0/2180333369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f515c0b46f0 msgr2=0x7f515c0b4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:20.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.452+0000 7f51631a2700 1 --2- 172.21.15.99:0/2180333369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f515c0b46f0 0x7f515c0b4b50 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f514c0099b0 tx=0x7f514c009250 comp rx=0 tx=0).stop 2023-10-08T00:21:20.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.452+0000 7f51631a2700 1 -- 172.21.15.99:0/2180333369 shutdown_connections 2023-10-08T00:21:20.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.452+0000 7f51631a2700 1 --2- 172.21.15.99:0/2180333369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f515c0b46f0 0x7f515c0b4b50 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:20.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.452+0000 7f51631a2700 1 --2- 172.21.15.99:0/2180333369 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f515c159ef0 0x7f515c15a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:20.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.452+0000 7f51631a2700 1 -- 172.21.15.99:0/2180333369 >> 172.21.15.99:0/2180333369 conn(0x7f515c09a0a0 msgr2=0x7f515c067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:20.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.452+0000 7f51631a2700 1 -- 172.21.15.99:0/2180333369 shutdown_connections 2023-10-08T00:21:20.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.452+0000 7f51631a2700 1 -- 172.21.15.99:0/2180333369 wait complete. 2023-10-08T00:21:20.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.453+0000 7f51631a2700 1 Processor -- start 2023-10-08T00:21:20.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.453+0000 7f51631a2700 1 -- start start 2023-10-08T00:21:20.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.454+0000 7f51631a2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f515c0b46f0 0x7f515c0ae710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:20.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.454+0000 7f51631a2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f515c0b6a10 0x7f515c0af120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:20.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.454+0000 7f51631a2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f515c159ef0 0x7f515c1562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:20.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.454+0000 7f51631a2700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f515c09b440 con 0x7f515c159ef0 2023-10-08T00:21:20.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.454+0000 7f51631a2700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f515c0afb70 con 0x7f515c0b6a10 2023-10-08T00:21:20.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.454+0000 7f51631a2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f515c09ab10 con 0x7f515c0b46f0 2023-10-08T00:21:20.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.454+0000 7f516173f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f515c159ef0 0x7f515c1562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:20.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.454+0000 7f516173f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f515c159ef0 0x7f515c1562c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57492/0 (socket says 172.21.15.99:57492) 2023-10-08T00:21:20.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.454+0000 7f5160f3e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f515c0b46f0 0x7f515c0ae710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:20.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.454+0000 7f516173f700 1 -- 172.21.15.99:0/2667594182 learned_addr learned my addr 172.21.15.99:0/2667594182 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:20.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.454+0000 7f515bfff700 1 --2- 172.21.15.99:0/2667594182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f515c0b6a10 0x7f515c0af120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:20.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.454+0000 7f516173f700 1 -- 172.21.15.99:0/2667594182 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f515c0b46f0 msgr2=0x7f515c0ae710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:20.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.454+0000 7f516173f700 1 --2- 172.21.15.99:0/2667594182 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f515c0b46f0 0x7f515c0ae710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:20.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.454+0000 7f516173f700 1 -- 172.21.15.99:0/2667594182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f515c0b6a10 msgr2=0x7f515c0af120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:20.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.454+0000 7f516173f700 1 --2- 172.21.15.99:0/2667594182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f515c0b6a10 0x7f515c0af120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:20.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.454+0000 7f516173f700 1 -- 172.21.15.99:0/2667594182 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f514c0094f0 con 0x7f515c159ef0 2023-10-08T00:21:20.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.455+0000 7f516173f700 1 --2- 172.21.15.99:0/2667594182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f515c159ef0 0x7f515c1562c0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f515c071f80 tx=0x7f514c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:20.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.455+0000 7f5158ff9700 1 -- 172.21.15.99:0/2667594182 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514c00baa0 con 0x7f515c159ef0 2023-10-08T00:21:20.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.455+0000 7f5158ff9700 1 -- 172.21.15.99:0/2667594182 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f514c025420 con 0x7f515c159ef0 2023-10-08T00:21:20.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.455+0000 7f51631a2700 1 -- 172.21.15.99:0/2667594182 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f515c09adf0 con 0x7f515c159ef0 2023-10-08T00:21:20.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.455+0000 7f5158ff9700 1 -- 172.21.15.99:0/2667594182 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514c00bc00 con 0x7f515c159ef0 2023-10-08T00:21:20.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.455+0000 7f51631a2700 1 -- 172.21.15.99:0/2667594182 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f515c0d1910 con 0x7f515c159ef0 2023-10-08T00:21:20.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.456+0000 7f51631a2700 1 -- 172.21.15.99:0/2667594182 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f515c09b5b0 con 0x7f515c159ef0 2023-10-08T00:21:20.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.456+0000 7f5158ff9700 1 -- 172.21.15.99:0/2667594182 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f514c023070 con 0x7f515c159ef0 2023-10-08T00:21:20.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.457+0000 7f5158ff9700 1 --2- 172.21.15.99:0/2667594182 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5148066f30 0x7f51480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:20.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.457+0000 7f5158ff9700 1 -- 172.21.15.99:0/2667594182 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5849+0+0 (secure 0 0 0) 0x7f514c085be0 con 0x7f515c159ef0 2023-10-08T00:21:20.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.457+0000 7f5158ff9700 1 -- 172.21.15.99:0/2667594182 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=174}) v3 -- 0x7f514806ebb0 con 0x7f515c159ef0 2023-10-08T00:21:20.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.457+0000 7f5160f3e700 1 --2- 172.21.15.99:0/2667594182 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5148066f30 0x7f51480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:20.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.457+0000 7f5160f3e700 1 --2- 172.21.15.99:0/2667594182 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5148066f30 0x7f51480693e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f515400f510 tx=0x7f515400b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:20.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.461+0000 7f5158ff9700 1 -- 172.21.15.99:0/2667594182 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f514c052dc0 con 0x7f515c159ef0 2023-10-08T00:21:20.641 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:20.641+0000 7f51631a2700 1 -- 172.21.15.99:0/2667594182 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f515c04c3c0 con 0x7f515c159ef0 2023-10-08T00:21:21.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.164+0000 7f5158ff9700 1 -- 172.21.15.99:0/2667594182 <== mon.0 v2:172.21.15.99:3300/0 7 ==== osd_map(174..174 src has 1..174) v4 ==== 679+0+0 (secure 0 0 0) 0x7f514c057180 con 0x7f515c159ef0 2023-10-08T00:21:21.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.165+0000 7f5158ff9700 1 -- 172.21.15.99:0/2667594182 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=175}) v3 -- 0x7f514806cfe0 con 0x7f515c159ef0 2023-10-08T00:21:21.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.167+0000 7f5158ff9700 1 -- 172.21.15.99:0/2667594182 <== mon.0 v2:172.21.15.99:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v174) v1 ==== 181+0+0 (secure 0 0 0) 0x7f514c015070 con 0x7f515c159ef0 2023-10-08T00:21:21.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.171+0000 7f51631a2700 1 -- 172.21.15.99:0/2667594182 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5148066f30 msgr2=0x7f51480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:21.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.171+0000 7f51631a2700 1 --2- 172.21.15.99:0/2667594182 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5148066f30 0x7f51480693e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f515400f510 tx=0x7f515400b040 comp rx=0 tx=0).stop 2023-10-08T00:21:21.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.171+0000 7f51631a2700 1 -- 172.21.15.99:0/2667594182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f515c159ef0 msgr2=0x7f515c1562c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:21.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.171+0000 7f51631a2700 1 --2- 172.21.15.99:0/2667594182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f515c159ef0 0x7f515c1562c0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f515c071f80 tx=0x7f514c017000 comp rx=0 tx=0).stop 2023-10-08T00:21:21.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.171+0000 7f51631a2700 1 -- 172.21.15.99:0/2667594182 shutdown_connections 2023-10-08T00:21:21.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.171+0000 7f51631a2700 1 --2- 172.21.15.99:0/2667594182 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5148066f30 0x7f51480693e0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:21.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.171+0000 7f51631a2700 1 --2- 172.21.15.99:0/2667594182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f515c159ef0 0x7f515c1562c0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:21.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.171+0000 7f51631a2700 1 --2- 172.21.15.99:0/2667594182 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f515c0b46f0 0x7f515c0ae710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:21.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.171+0000 7f51631a2700 1 --2- 172.21.15.99:0/2667594182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f515c0b6a10 0x7f515c0af120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:21.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.171+0000 7f51631a2700 1 -- 172.21.15.99:0/2667594182 >> 172.21.15.99:0/2667594182 conn(0x7f515c09a0a0 msgr2=0x7f515c15c350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:21.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.172+0000 7f51631a2700 1 -- 172.21.15.99:0/2667594182 shutdown_connections 2023-10-08T00:21:21.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.172+0000 7f51631a2700 1 -- 172.21.15.99:0/2667594182 wait complete. 2023-10-08T00:21:21.175 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' removed 2023-10-08T00:21:21.185 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T00:21:21.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.304+0000 7fa70bfff700 1 Processor -- start 2023-10-08T00:21:21.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.304+0000 7fa70bfff700 1 -- start start 2023-10-08T00:21:21.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.305+0000 7fa70bfff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa70c0cc940 0x7fa70c0ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:21.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.305+0000 7fa70bfff700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa70c0cbb80 con 0x7fa70c0ce810 2023-10-08T00:21:21.311 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.305+0000 7fa70bfff700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa70c0cbe60 con 0x7fa70c15d240 2023-10-08T00:21:21.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.305+0000 7fa70bfff700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa70c0cc140 con 0x7fa70c0cc940 2023-10-08T00:21:21.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.305+0000 7fa7027fc700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa70c0cc940 0x7fa70c0ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:21.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.305+0000 7fa7027fc700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa70c0cc940 0x7fa70c0ccde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57500/0 (socket says 172.21.15.99:57500) 2023-10-08T00:21:21.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.305+0000 7fa70b7fe700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fa70c0ce810 0x7fa70c0cec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:46746/0 (socket says 172.21.15.99:46746) 2023-10-08T00:21:21.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.305+0000 7fa7027fc700 1 -- 172.21.15.99:0/1911797323 learned_addr learned my addr 172.21.15.99:0/1911797323 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:21.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.305+0000 7fa7027fc700 1 -- 172.21.15.99:0/1911797323 >> v1:172.21.15.191:6789/0 conn(0x7fa70c15d240 legacy=0x7fa70c0b3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:21.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.305+0000 7fa7027fc700 1 -- 172.21.15.99:0/1911797323 >> v1:172.21.15.99:6789/0 conn(0x7fa70c0ce810 legacy=0x7fa70c0cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:21.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.305+0000 7fa7027fc700 1 -- 172.21.15.99:0/1911797323 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa70c0cc420 con 0x7fa70c0cc940 2023-10-08T00:21:21.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.305+0000 7fa70a7fc700 1 -- 172.21.15.99:0/1911797323 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2261608267 0 0) 0x7fa70c0cbb80 con 0x7fa70c0ce810 2023-10-08T00:21:21.312 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.306+0000 7fa7027fc700 1 --2- 172.21.15.99:0/1911797323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa70c0cc940 0x7fa70c0ccde0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fa6f40099b0 tx=0x7fa6f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ae0f249e10451cd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:21.313 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.306+0000 7fa70a7fc700 1 -- 172.21.15.99:0/1911797323 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f400b820 con 0x7fa70c0cc940 2023-10-08T00:21:21.313 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.306+0000 7fa70a7fc700 1 -- 172.21.15.99:0/1911797323 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f400b980 con 0x7fa70c0cc940 2023-10-08T00:21:21.313 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.306+0000 7fa70a7fc700 1 -- 172.21.15.99:0/1911797323 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f400bd20 con 0x7fa70c0cc940 2023-10-08T00:21:21.313 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.306+0000 7fa70bfff700 1 -- 172.21.15.99:0/1911797323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa70c0cc940 msgr2=0x7fa70c0ccde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:21.313 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.306+0000 7fa70bfff700 1 --2- 172.21.15.99:0/1911797323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa70c0cc940 0x7fa70c0ccde0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fa6f40099b0 tx=0x7fa6f4009250 comp rx=0 tx=0).stop 2023-10-08T00:21:21.313 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.306+0000 7fa70bfff700 1 -- 172.21.15.99:0/1911797323 shutdown_connections 2023-10-08T00:21:21.313 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.306+0000 7fa70bfff700 1 --2- 172.21.15.99:0/1911797323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa70c0cc940 0x7fa70c0ccde0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:21.313 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.306+0000 7fa70bfff700 1 -- 172.21.15.99:0/1911797323 >> 172.21.15.99:0/1911797323 conn(0x7fa70c067e40 msgr2=0x7fa70c0d0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:21.313 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.306+0000 7fa70bfff700 1 -- 172.21.15.99:0/1911797323 shutdown_connections 2023-10-08T00:21:21.313 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.306+0000 7fa70bfff700 1 -- 172.21.15.99:0/1911797323 wait complete. 2023-10-08T00:21:21.314 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.307+0000 7fa70bfff700 1 Processor -- start 2023-10-08T00:21:21.314 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.308+0000 7fa70bfff700 1 -- start start 2023-10-08T00:21:21.314 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.308+0000 7fa70bfff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa70c0cc940 0x7fa70c0ca6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:21.314 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.308+0000 7fa70bfff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa70c0ce810 0x7fa70c0c9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:21.314 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.308+0000 7fa70bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa70c15d240 0x7fa70c0b5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:21.314 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.308+0000 7fa70bfff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa70c0ca0a0 con 0x7fa70c15d240 2023-10-08T00:21:21.314 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.308+0000 7fa70bfff700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa70c157610 con 0x7fa70c0cc940 2023-10-08T00:21:21.314 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.308+0000 7fa70bfff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa70c0c96d0 con 0x7fa70c0ce810 2023-10-08T00:21:21.314 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.308+0000 7fa70b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa70c15d240 0x7fa70c0b5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:21.314 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.308+0000 7fa70b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa70c15d240 0x7fa70c0b5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57510/0 (socket says 172.21.15.99:57510) 2023-10-08T00:21:21.314 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.308+0000 7fa70b7fe700 1 -- 172.21.15.99:0/3663191285 learned_addr learned my addr 172.21.15.99:0/3663191285 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:21.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.308+0000 7fa70affd700 1 --2- 172.21.15.99:0/3663191285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa70c0cc940 0x7fa70c0ca6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:21.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.309+0000 7fa7027fc700 1 --2- 172.21.15.99:0/3663191285 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa70c0ce810 0x7fa70c0c9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:21.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.309+0000 7fa70b7fe700 1 -- 172.21.15.99:0/3663191285 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa70c0ce810 msgr2=0x7fa70c0c9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:21.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.309+0000 7fa70b7fe700 1 --2- 172.21.15.99:0/3663191285 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa70c0ce810 0x7fa70c0c9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:21.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.309+0000 7fa70b7fe700 1 -- 172.21.15.99:0/3663191285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa70c0cc940 msgr2=0x7fa70c0ca6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:21.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.309+0000 7fa70b7fe700 1 --2- 172.21.15.99:0/3663191285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa70c0cc940 0x7fa70c0ca6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:21.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.309+0000 7fa70b7fe700 1 -- 172.21.15.99:0/3663191285 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f40094f0 con 0x7fa70c15d240 2023-10-08T00:21:21.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.309+0000 7fa70b7fe700 1 --2- 172.21.15.99:0/3663191285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa70c15d240 0x7fa70c0b5660 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fa70400ccc0 tx=0x7fa704001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:21.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.309+0000 7fa7037fe700 1 -- 172.21.15.99:0/3663191285 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa704005df0 con 0x7fa70c15d240 2023-10-08T00:21:21.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.309+0000 7fa7037fe700 1 -- 172.21.15.99:0/3663191285 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa70400a600 con 0x7fa70c15d240 2023-10-08T00:21:21.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.309+0000 7fa70bfff700 1 -- 172.21.15.99:0/3663191285 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa70c0cadc0 con 0x7fa70c15d240 2023-10-08T00:21:21.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.309+0000 7fa7037fe700 1 -- 172.21.15.99:0/3663191285 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70400a9c0 con 0x7fa70c15d240 2023-10-08T00:21:21.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.309+0000 7fa70bfff700 1 -- 172.21.15.99:0/3663191285 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa70c0c9080 con 0x7fa70c15d240 2023-10-08T00:21:21.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.310+0000 7fa70bfff700 1 -- 172.21.15.99:0/3663191285 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa70c0c9840 con 0x7fa70c15d240 2023-10-08T00:21:21.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.310+0000 7fa7037fe700 1 -- 172.21.15.99:0/3663191285 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa7040070f0 con 0x7fa70c15d240 2023-10-08T00:21:21.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.311+0000 7fa7037fe700 1 --2- 172.21.15.99:0/3663191285 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa6ec066f30 0x7fa6ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:21.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.311+0000 7fa7037fe700 1 -- 172.21.15.99:0/3663191285 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 5411+0+0 (secure 0 0 0) 0x7fa704084440 con 0x7fa70c15d240 2023-10-08T00:21:21.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.311+0000 7fa70affd700 1 --2- 172.21.15.99:0/3663191285 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa6ec066f30 0x7fa6ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:21.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.311+0000 7fa70affd700 1 --2- 172.21.15.99:0/3663191285 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa6ec066f30 0x7fa6ec0693e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc006fd0 tx=0x7fa6fc009060 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:21.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.315+0000 7fa7037fe700 1 -- 172.21.15.99:0/3663191285 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa704051900 con 0x7fa70c15d240 2023-10-08T00:21:21.496 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:21.495+0000 7fa70bfff700 1 -- 172.21.15.99:0/3663191285 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa70c06ac60 con 0x7fa70c15d240 2023-10-08T00:21:22.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.257+0000 7fa7037fe700 1 -- 172.21.15.99:0/3663191285 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v175) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa704054f20 con 0x7fa70c15d240 2023-10-08T00:21:22.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.264+0000 7fa70bfff700 1 -- 172.21.15.99:0/3663191285 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa6ec066f30 msgr2=0x7fa6ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:22.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.264+0000 7fa70bfff700 1 --2- 172.21.15.99:0/3663191285 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa6ec066f30 0x7fa6ec0693e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc006fd0 tx=0x7fa6fc009060 comp rx=0 tx=0).stop 2023-10-08T00:21:22.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.264+0000 7fa70bfff700 1 -- 172.21.15.99:0/3663191285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa70c15d240 msgr2=0x7fa70c0b5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:22.265 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.264+0000 7fa70bfff700 1 --2- 172.21.15.99:0/3663191285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa70c15d240 0x7fa70c0b5660 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fa70400ccc0 tx=0x7fa704001a70 comp rx=0 tx=0).stop 2023-10-08T00:21:22.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.264+0000 7fa70bfff700 1 -- 172.21.15.99:0/3663191285 shutdown_connections 2023-10-08T00:21:22.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.264+0000 7fa70bfff700 1 --2- 172.21.15.99:0/3663191285 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa6ec066f30 0x7fa6ec0693e0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:22.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.264+0000 7fa70bfff700 1 --2- 172.21.15.99:0/3663191285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa70c0cc940 0x7fa70c0ca6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:22.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.264+0000 7fa70bfff700 1 --2- 172.21.15.99:0/3663191285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa70c15d240 0x7fa70c0b5660 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:22.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.264+0000 7fa70bfff700 1 --2- 172.21.15.99:0/3663191285 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa70c0ce810 0x7fa70c0c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:22.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.264+0000 7fa70bfff700 1 -- 172.21.15.99:0/3663191285 >> 172.21.15.99:0/3663191285 conn(0x7fa70c067e40 msgr2=0x7fa70c0c59e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:22.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.264+0000 7fa70bfff700 1 -- 172.21.15.99:0/3663191285 shutdown_connections 2023-10-08T00:21:22.266 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.264+0000 7fa70bfff700 1 -- 172.21.15.99:0/3663191285 wait complete. 2023-10-08T00:21:22.267 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' removed 2023-10-08T00:21:22.278 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-08T00:21:22.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.396+0000 7fc1d1b14700 1 Processor -- start 2023-10-08T00:21:22.397 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.396+0000 7fc1d1b14700 1 -- start start 2023-10-08T00:21:22.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.397+0000 7fc1d1b14700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1cc069f80 0x7fc1cc0b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:22.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.397+0000 7fc1d1b14700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1cc0afb50 con 0x7fc1cc157cc0 2023-10-08T00:21:22.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.397+0000 7fc1d1b14700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1cc0afe30 con 0x7fc1cc0cb0f0 2023-10-08T00:21:22.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.397+0000 7fc1d1b14700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1cc0b0110 con 0x7fc1cc069f80 2023-10-08T00:21:22.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.397+0000 7fc1cb7fe700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fc1cc157cc0 0x7fc1cc0ca9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:46750/0 (socket says 172.21.15.99:46750) 2023-10-08T00:21:22.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.397+0000 7fc1cb7fe700 1 -- 172.21.15.99:0/1623916572 learned_addr learned my addr 172.21.15.99:0/1623916572 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:21:22.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.397+0000 7fc1cbfff700 1 --2- 172.21.15.99:0/1623916572 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1cc069f80 0x7fc1cc0b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:22.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.397+0000 7fc1ca7fc700 1 -- 172.21.15.99:0/1623916572 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3145597099 0 0) 0x7fc1cc0afb50 con 0x7fc1cc157cc0 2023-10-08T00:21:22.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.397+0000 7fc1ca7fc700 1 -- 172.21.15.99:0/1623916572 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1b4003570 con 0x7fc1cc157cc0 2023-10-08T00:21:22.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.397+0000 7fc1ca7fc700 1 -- 172.21.15.99:0/1623916572 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1270275862 0 0) 0x7fc1b4003570 con 0x7fc1cc157cc0 2023-10-08T00:21:22.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.397+0000 7fc1ca7fc700 1 -- 172.21.15.99:0/1623916572 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1cc069f80 msgr2=0x7fc1cc0b0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:22.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.397+0000 7fc1ca7fc700 1 --2- 172.21.15.99:0/1623916572 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1cc069f80 0x7fc1cc0b0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:22.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.397+0000 7fc1ca7fc700 1 -- 172.21.15.99:0/1623916572 >> v1:172.21.15.191:6789/0 conn(0x7fc1cc0cb0f0 legacy=0x7fc1cc069870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:22.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.397+0000 7fc1ca7fc700 1 -- 172.21.15.99:0/1623916572 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1cc0b03f0 con 0x7fc1cc157cc0 2023-10-08T00:21:22.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.397+0000 7fc1ca7fc700 1 -- 172.21.15.99:0/1623916572 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7fc1bc0017f0 con 0x7fc1cc157cc0 2023-10-08T00:21:22.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.397+0000 7fc1ca7fc700 1 -- 172.21.15.99:0/1623916572 >> v1:172.21.15.99:6789/0 conn(0x7fc1cc157cc0 legacy=0x7fc1cc0ca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:22.405 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.397+0000 7fc1ca7fc700 1 --2- 172.21.15.99:0/1623916572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1b4003a90 0x7fc1b4003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:22.406 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.397+0000 7fc1ca7fc700 1 --2- 172.21.15.99:0/1623916572 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1b4004810 0x7fc1b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:22.406 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.397+0000 7fc1ca7fc700 1 --2- 172.21.15.99:0/1623916572 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1cc069f80 0x7fc1b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:22.406 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.397+0000 7fc1ca7fc700 1 -- 172.21.15.99:0/1623916572 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1cc0b0110 con 0x7fc1b4003a90 2023-10-08T00:21:22.406 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.397+0000 7fc1ca7fc700 1 -- 172.21.15.99:0/1623916572 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b400bb10 con 0x7fc1b4004810 2023-10-08T00:21:22.406 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.397+0000 7fc1ca7fc700 1 -- 172.21.15.99:0/1623916572 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b400bcd0 con 0x7fc1cc069f80 2023-10-08T00:21:22.406 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.397+0000 7fc1cb7fe700 1 --2- 172.21.15.99:0/1623916572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1b4003a90 0x7fc1b4003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:22.406 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.397+0000 7fc1cbfff700 1 --2- 172.21.15.99:0/1623916572 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1cc069f80 0x7fc1b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:22.406 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.397+0000 7fc1caffd700 1 --2- 172.21.15.99:0/1623916572 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1b4004810 0x7fc1b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:22.406 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.397+0000 7fc1cb7fe700 1 -- 172.21.15.99:0/1623916572 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1cc069f80 msgr2=0x7fc1b40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:22.406 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.397+0000 7fc1cb7fe700 1 --2- 172.21.15.99:0/1623916572 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1cc069f80 0x7fc1b40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:22.406 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.397+0000 7fc1cb7fe700 1 -- 172.21.15.99:0/1623916572 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1b4004810 msgr2=0x7fc1b4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:22.407 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.397+0000 7fc1cb7fe700 1 --2- 172.21.15.99:0/1623916572 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1b4004810 0x7fc1b4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:22.407 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.397+0000 7fc1cb7fe700 1 -- 172.21.15.99:0/1623916572 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b4003570 con 0x7fc1b4003a90 2023-10-08T00:21:22.407 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.397+0000 7fc1cbfff700 1 --2- 172.21.15.99:0/1623916572 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1cc069f80 0x7fc1b40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:21:22.407 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.397+0000 7fc1caffd700 1 --2- 172.21.15.99:0/1623916572 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1b4004810 0x7fc1b4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:21:22.407 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.397+0000 7fc1cb7fe700 1 --2- 172.21.15.99:0/1623916572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1b4003a90 0x7fc1b4003e60 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc00a950 tx=0x7fc1bc0021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28f044004840554a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:22.407 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.398+0000 7fc1ca7fc700 1 -- 172.21.15.99:0/1623916572 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc0070d0 con 0x7fc1b4003a90 2023-10-08T00:21:22.407 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.398+0000 7fc1ca7fc700 1 -- 172.21.15.99:0/1623916572 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1bc007250 con 0x7fc1b4003a90 2023-10-08T00:21:22.407 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.398+0000 7fc1ca7fc700 1 -- 172.21.15.99:0/1623916572 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc007610 con 0x7fc1b4003a90 2023-10-08T00:21:22.407 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.398+0000 7fc1d1b14700 1 -- 172.21.15.99:0/1623916572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1b4003a90 msgr2=0x7fc1b4003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:22.407 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.398+0000 7fc1d1b14700 1 --2- 172.21.15.99:0/1623916572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1b4003a90 0x7fc1b4003e60 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc00a950 tx=0x7fc1bc0021f0 comp rx=0 tx=0).stop 2023-10-08T00:21:22.408 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.398+0000 7fc1cbfff700 1 -- 172.21.15.99:0/1623916572 reap_dead start 2023-10-08T00:21:22.408 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.398+0000 7fc1d1b14700 1 -- 172.21.15.99:0/1623916572 shutdown_connections 2023-10-08T00:21:22.408 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.398+0000 7fc1d1b14700 1 -- 172.21.15.99:0/1623916572 >> 172.21.15.99:0/1623916572 conn(0x7fc1cc0cb5c0 msgr2=0x7fc1cc0cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:22.408 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.398+0000 7fc1d1b14700 1 -- 172.21.15.99:0/1623916572 shutdown_connections 2023-10-08T00:21:22.408 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.398+0000 7fc1d1b14700 1 -- 172.21.15.99:0/1623916572 wait complete. 2023-10-08T00:21:22.408 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.399+0000 7fc1d1b14700 1 Processor -- start 2023-10-08T00:21:22.408 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.399+0000 7fc1d1b14700 1 -- start start 2023-10-08T00:21:22.408 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.400+0000 7fc1d1b14700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1b4003a90 0x7fc1cc159d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:22.408 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.400+0000 7fc1d1b14700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1cc14be00 0x7fc1cc0b36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:22.408 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.400+0000 7fc1d1b14700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1cc14c270 0x7fc1cc0b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:22.408 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.400+0000 7fc1d1b14700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1cc15aa80 con 0x7fc1cc14be00 2023-10-08T00:21:22.408 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.400+0000 7fc1d1b14700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1cc158450 con 0x7fc1cc14c270 2023-10-08T00:21:22.409 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.400+0000 7fc1d1b14700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1cc158730 con 0x7fc1b4003a90 2023-10-08T00:21:22.409 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.400+0000 7fc1caffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1cc14be00 0x7fc1cc0b36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:22.409 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.400+0000 7fc1caffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1cc14be00 0x7fc1cc0b36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57534/0 (socket says 172.21.15.99:57534) 2023-10-08T00:21:22.409 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.400+0000 7fc1caffd700 1 -- 172.21.15.99:0/2775097191 learned_addr learned my addr 172.21.15.99:0/2775097191 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:22.409 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.400+0000 7fc1cb7fe700 1 --2- 172.21.15.99:0/2775097191 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1b4003a90 0x7fc1cc159d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:22.409 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.400+0000 7fc1cbfff700 1 --2- 172.21.15.99:0/2775097191 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1cc14c270 0x7fc1cc0b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:22.409 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.401+0000 7fc1caffd700 1 -- 172.21.15.99:0/2775097191 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1b4003a90 msgr2=0x7fc1cc159d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:22.409 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.401+0000 7fc1caffd700 1 --2- 172.21.15.99:0/2775097191 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1b4003a90 0x7fc1cc159d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:22.409 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.401+0000 7fc1caffd700 1 -- 172.21.15.99:0/2775097191 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1cc14c270 msgr2=0x7fc1cc0b3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:22.409 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.401+0000 7fc1caffd700 1 --2- 172.21.15.99:0/2775097191 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1cc14c270 0x7fc1cc0b3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:22.409 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.401+0000 7fc1caffd700 1 -- 172.21.15.99:0/2775097191 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1cc158a10 con 0x7fc1cc14be00 2023-10-08T00:21:22.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.401+0000 7fc1caffd700 1 --2- 172.21.15.99:0/2775097191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1cc14be00 0x7fc1cc0b36a0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0007a60 tx=0x7fc1c0009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:22.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.401+0000 7fc1b37fe700 1 -- 172.21.15.99:0/2775097191 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c0010520 con 0x7fc1cc14be00 2023-10-08T00:21:22.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.401+0000 7fc1b37fe700 1 -- 172.21.15.99:0/2775097191 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1c0010680 con 0x7fc1cc14be00 2023-10-08T00:21:22.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.401+0000 7fc1d1b14700 1 -- 172.21.15.99:0/2775097191 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1cc158cf0 con 0x7fc1cc14be00 2023-10-08T00:21:22.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.401+0000 7fc1b37fe700 1 -- 172.21.15.99:0/2775097191 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c0010a20 con 0x7fc1cc14be00 2023-10-08T00:21:22.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.401+0000 7fc1d1b14700 1 -- 172.21.15.99:0/2775097191 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1cc158090 con 0x7fc1cc14be00 2023-10-08T00:21:22.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.402+0000 7fc1d1b14700 1 -- 172.21.15.99:0/2775097191 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1cc04c3c0 con 0x7fc1cc14be00 2023-10-08T00:21:22.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.402+0000 7fc1b37fe700 1 -- 172.21.15.99:0/2775097191 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc1c0014480 con 0x7fc1cc14be00 2023-10-08T00:21:22.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.402+0000 7fc1b37fe700 1 --2- 172.21.15.99:0/2775097191 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc1b8066f30 0x7fc1b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:22.410 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.403+0000 7fc1b37fe700 1 -- 172.21.15.99:0/2775097191 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fc1c0083c10 con 0x7fc1cc14be00 2023-10-08T00:21:22.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.403+0000 7fc1cb7fe700 1 --2- 172.21.15.99:0/2775097191 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc1b8066f30 0x7fc1b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:22.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.403+0000 7fc1cb7fe700 1 --2- 172.21.15.99:0/2775097191 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc1b8066f30 0x7fc1b80693e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc002250 tx=0x7fc1bc015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:22.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.407+0000 7fc1b37fe700 1 -- 172.21.15.99:0/2775097191 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc1c004d5a0 con 0x7fc1cc14be00 2023-10-08T00:21:22.590 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.589+0000 7fc1d1b14700 1 -- 172.21.15.99:0/2775097191 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fc1cc0b06d0 con 0x7fc1cc14be00 2023-10-08T00:21:22.594 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.593+0000 7fc1b37fe700 1 -- 172.21.15.99:0/2775097191 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc1c00190a0 con 0x7fc1cc14be00 2023-10-08T00:21:22.596 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.594+0000 7fc1d1b14700 1 -- 172.21.15.99:0/2775097191 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc1b8066f30 msgr2=0x7fc1b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:22.596 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.594+0000 7fc1d1b14700 1 --2- 172.21.15.99:0/2775097191 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc1b8066f30 0x7fc1b80693e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc002250 tx=0x7fc1bc015000 comp rx=0 tx=0).stop 2023-10-08T00:21:22.597 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.594+0000 7fc1d1b14700 1 -- 172.21.15.99:0/2775097191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1cc14be00 msgr2=0x7fc1cc0b36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:22.597 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.594+0000 7fc1d1b14700 1 --2- 172.21.15.99:0/2775097191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1cc14be00 0x7fc1cc0b36a0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0007a60 tx=0x7fc1c0009040 comp rx=0 tx=0).stop 2023-10-08T00:21:22.597 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.595+0000 7fc1d1b14700 1 -- 172.21.15.99:0/2775097191 shutdown_connections 2023-10-08T00:21:22.597 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.595+0000 7fc1d1b14700 1 --2- 172.21.15.99:0/2775097191 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc1b8066f30 0x7fc1b80693e0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:22.597 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.595+0000 7fc1d1b14700 1 --2- 172.21.15.99:0/2775097191 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1cc14c270 0x7fc1cc0b3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:22.597 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.595+0000 7fc1d1b14700 1 --2- 172.21.15.99:0/2775097191 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1b4003a90 0x7fc1cc159d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:22.597 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.595+0000 7fc1d1b14700 1 --2- 172.21.15.99:0/2775097191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1cc14be00 0x7fc1cc0b36a0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:22.597 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.595+0000 7fc1d1b14700 1 -- 172.21.15.99:0/2775097191 >> 172.21.15.99:0/2775097191 conn(0x7fc1cc0cb5c0 msgr2=0x7fc1cc157020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:22.597 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.595+0000 7fc1d1b14700 1 -- 172.21.15.99:0/2775097191 shutdown_connections 2023-10-08T00:21:22.597 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.595+0000 7fc1d1b14700 1 -- 172.21.15.99:0/2775097191 wait complete. 2023-10-08T00:21:22.608 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync' 2023-10-08T00:21:22.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.720+0000 7f21433f1700 1 Processor -- start 2023-10-08T00:21:22.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.720+0000 7f21433f1700 1 -- start start 2023-10-08T00:21:22.726 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.721+0000 7f21433f1700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f213c0b1220 0x7f213c0c9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:22.726 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.721+0000 7f21433f1700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f213c15de60 con 0x7f213c0b0e50 2023-10-08T00:21:22.726 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.721+0000 7f21433f1700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f213c15e140 con 0x7f213c0cb4f0 2023-10-08T00:21:22.726 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.721+0000 7f21433f1700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f213c15e420 con 0x7f213c0b1220 2023-10-08T00:21:22.726 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.721+0000 7f214118d700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f213c0b0e50 0x7f213c06d590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:46764/0 (socket says 172.21.15.99:46764) 2023-10-08T00:21:22.726 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.721+0000 7f214118d700 1 -- 172.21.15.99:0/1511443429 learned_addr learned my addr 172.21.15.99:0/1511443429 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:21:22.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.722+0000 7f214098c700 1 --2- 172.21.15.99:0/1511443429 >> v2:172.21.15.191:3300/0 conn(0x7f213c0b1220 0x7f213c0c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:22.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.722+0000 7f2133fff700 1 -- 172.21.15.99:0/1511443429 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3071272373 0 0) 0x7f213c15de60 con 0x7f213c0b0e50 2023-10-08T00:21:22.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.722+0000 7f2133fff700 1 -- 172.21.15.99:0/1511443429 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2128003570 con 0x7f213c0b0e50 2023-10-08T00:21:22.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.722+0000 7f2133fff700 1 -- 172.21.15.99:0/1511443429 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 408609574 0 0) 0x7f2128003570 con 0x7f213c0b0e50 2023-10-08T00:21:22.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.722+0000 7f2133fff700 1 -- 172.21.15.99:0/1511443429 >> v2:172.21.15.191:3300/0 conn(0x7f213c0b1220 msgr2=0x7f213c0c9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:22.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.722+0000 7f2133fff700 1 --2- 172.21.15.99:0/1511443429 >> v2:172.21.15.191:3300/0 conn(0x7f213c0b1220 0x7f213c0c9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:22.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.722+0000 7f2133fff700 1 -- 172.21.15.99:0/1511443429 >> v1:172.21.15.191:6789/0 conn(0x7f213c0cb4f0 legacy=0x7f213c0679d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:22.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.722+0000 7f2133fff700 1 -- 172.21.15.99:0/1511443429 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f213c15e700 con 0x7f213c0b0e50 2023-10-08T00:21:22.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.722+0000 7f2133fff700 1 -- 172.21.15.99:0/1511443429 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f2138001470 con 0x7f213c0b0e50 2023-10-08T00:21:22.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.722+0000 7f2133fff700 1 -- 172.21.15.99:0/1511443429 >> v1:172.21.15.99:6789/0 conn(0x7f213c0b0e50 legacy=0x7f213c06d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:22.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.722+0000 7f2133fff700 1 --2- 172.21.15.99:0/1511443429 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2128003a60 0x7f2128003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:22.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.722+0000 7f2133fff700 1 --2- 172.21.15.99:0/1511443429 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2128004810 0x7f2128004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:22.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.722+0000 7f2133fff700 1 --2- 172.21.15.99:0/1511443429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21280051c0 0x7f212800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:22.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.722+0000 7f2133fff700 1 -- 172.21.15.99:0/1511443429 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213c15e420 con 0x7f21280051c0 2023-10-08T00:21:22.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.722+0000 7f2133fff700 1 -- 172.21.15.99:0/1511443429 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f212800c1e0 con 0x7f2128004810 2023-10-08T00:21:22.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.722+0000 7f2133fff700 1 -- 172.21.15.99:0/1511443429 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f212800c3a0 con 0x7f2128003a60 2023-10-08T00:21:22.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.722+0000 7f214198e700 1 --2- 172.21.15.99:0/1511443429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21280051c0 0x7f212800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:22.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.722+0000 7f214098c700 1 --2- 172.21.15.99:0/1511443429 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2128004810 0x7f2128004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:22.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.722+0000 7f214118d700 1 --2- 172.21.15.99:0/1511443429 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2128003a60 0x7f2128003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:22.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.722+0000 7f214198e700 1 -- 172.21.15.99:0/1511443429 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2128003a60 msgr2=0x7f2128003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:22.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.722+0000 7f214198e700 1 --2- 172.21.15.99:0/1511443429 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2128003a60 0x7f2128003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:22.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.722+0000 7f214198e700 1 -- 172.21.15.99:0/1511443429 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2128004810 msgr2=0x7f2128004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:22.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.722+0000 7f214198e700 1 --2- 172.21.15.99:0/1511443429 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2128004810 0x7f2128004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:22.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.722+0000 7f214198e700 1 -- 172.21.15.99:0/1511443429 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2128003570 con 0x7f21280051c0 2023-10-08T00:21:22.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.722+0000 7f214098c700 1 --2- 172.21.15.99:0/1511443429 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2128004810 0x7f2128004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:21:22.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.723+0000 7f214198e700 1 -- 172.21.15.99:0/1511443429 reap_dead start 2023-10-08T00:21:22.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.723+0000 7f214198e700 1 --2- 172.21.15.99:0/1511443429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21280051c0 0x7f212800ba60 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f2134002070 tx=0x7f21340016d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39c3e8e6f2d04653 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:22.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.723+0000 7f2133fff700 1 -- 172.21.15.99:0/1511443429 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21340019e0 con 0x7f21280051c0 2023-10-08T00:21:22.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.723+0000 7f2133fff700 1 -- 172.21.15.99:0/1511443429 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f213400c8b0 con 0x7f21280051c0 2023-10-08T00:21:22.730 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.723+0000 7f2133fff700 1 -- 172.21.15.99:0/1511443429 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213400cc30 con 0x7f21280051c0 2023-10-08T00:21:22.730 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.723+0000 7f21433f1700 1 -- 172.21.15.99:0/1511443429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21280051c0 msgr2=0x7f212800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:22.730 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.723+0000 7f21433f1700 1 --2- 172.21.15.99:0/1511443429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21280051c0 0x7f212800ba60 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f2134002070 tx=0x7f21340016d0 comp rx=0 tx=0).stop 2023-10-08T00:21:22.730 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.723+0000 7f21433f1700 1 -- 172.21.15.99:0/1511443429 shutdown_connections 2023-10-08T00:21:22.730 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.723+0000 7f21433f1700 1 --2- 172.21.15.99:0/1511443429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21280051c0 0x7f212800ba60 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:22.730 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.723+0000 7f21433f1700 1 -- 172.21.15.99:0/1511443429 >> 172.21.15.99:0/1511443429 conn(0x7f213c069870 msgr2=0x7f213c069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:22.730 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.723+0000 7f21433f1700 1 -- 172.21.15.99:0/1511443429 shutdown_connections 2023-10-08T00:21:22.730 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.723+0000 7f21433f1700 1 -- 172.21.15.99:0/1511443429 wait complete. 2023-10-08T00:21:22.730 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.724+0000 7f21433f1700 1 Processor -- start 2023-10-08T00:21:22.730 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.725+0000 7f21433f1700 1 -- start start 2023-10-08T00:21:22.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.725+0000 7f21433f1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f21280051c0 0x7f213c148700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:22.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.725+0000 7f21433f1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f213c15b560 0x7f213c15b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:22.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.725+0000 7f21433f1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f213c0b22a0 0x7f213c0b2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:22.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.725+0000 7f21433f1700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213c15b0d0 con 0x7f213c15b560 2023-10-08T00:21:22.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.725+0000 7f21433f1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213c15a500 con 0x7f21280051c0 2023-10-08T00:21:22.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.725+0000 7f21433f1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f213c15a7a0 con 0x7f213c0b22a0 2023-10-08T00:21:22.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.726+0000 7f214098c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f213c15b560 0x7f213c15b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:22.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.726+0000 7f214098c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f213c15b560 0x7f213c15b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57554/0 (socket says 172.21.15.99:57554) 2023-10-08T00:21:22.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.726+0000 7f214098c700 1 -- 172.21.15.99:0/3994048093 learned_addr learned my addr 172.21.15.99:0/3994048093 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:22.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.726+0000 7f214118d700 1 --2- 172.21.15.99:0/3994048093 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f21280051c0 0x7f213c148700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:22.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.726+0000 7f214198e700 1 --2- 172.21.15.99:0/3994048093 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f213c0b22a0 0x7f213c0b2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:22.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.726+0000 7f214098c700 1 -- 172.21.15.99:0/3994048093 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f213c0b22a0 msgr2=0x7f213c0b2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:22.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.726+0000 7f214098c700 1 --2- 172.21.15.99:0/3994048093 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f213c0b22a0 0x7f213c0b2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:22.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.726+0000 7f214098c700 1 -- 172.21.15.99:0/3994048093 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f21280051c0 msgr2=0x7f213c148700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:22.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.726+0000 7f214098c700 1 --2- 172.21.15.99:0/3994048093 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f21280051c0 0x7f213c148700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:22.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.726+0000 7f214098c700 1 -- 172.21.15.99:0/3994048093 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f213c15aa80 con 0x7f213c15b560 2023-10-08T00:21:22.735 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.726+0000 7f214118d700 1 --2- 172.21.15.99:0/3994048093 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f21280051c0 0x7f213c148700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:21:22.735 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.726+0000 7f214198e700 1 --2- 172.21.15.99:0/3994048093 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f213c0b22a0 0x7f213c0b2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:21:22.735 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.726+0000 7f214098c700 1 --2- 172.21.15.99:0/3994048093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f213c15b560 0x7f213c15b9d0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f212c006c80 tx=0x7f212c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:22.735 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.726+0000 7f21317fa700 1 -- 172.21.15.99:0/3994048093 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212c011e70 con 0x7f213c15b560 2023-10-08T00:21:22.735 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.726+0000 7f21317fa700 1 -- 172.21.15.99:0/3994048093 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f212c00fcd0 con 0x7f213c15b560 2023-10-08T00:21:22.735 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.726+0000 7f21433f1700 1 -- 172.21.15.99:0/3994048093 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f213c15ad60 con 0x7f213c15b560 2023-10-08T00:21:22.735 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.726+0000 7f21317fa700 1 -- 172.21.15.99:0/3994048093 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212c01d6d0 con 0x7f213c15b560 2023-10-08T00:21:22.735 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.726+0000 7f21433f1700 1 -- 172.21.15.99:0/3994048093 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f213c16a160 con 0x7f213c15b560 2023-10-08T00:21:22.735 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.727+0000 7f21433f1700 1 -- 172.21.15.99:0/3994048093 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f213c15b240 con 0x7f213c15b560 2023-10-08T00:21:22.735 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.727+0000 7f21317fa700 1 -- 172.21.15.99:0/3994048093 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f212c01da40 con 0x7f213c15b560 2023-10-08T00:21:22.735 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.728+0000 7f21317fa700 1 --2- 172.21.15.99:0/3994048093 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2120066f30 0x7f21200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:22.735 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.728+0000 7f21317fa700 1 -- 172.21.15.99:0/3994048093 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f212c018070 con 0x7f213c15b560 2023-10-08T00:21:22.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.728+0000 7f214118d700 1 --2- 172.21.15.99:0/3994048093 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2120066f30 0x7f21200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:22.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.729+0000 7f214118d700 1 --2- 172.21.15.99:0/3994048093 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2120066f30 0x7f21200693e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f2138001200 tx=0x7f2138009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:22.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.732+0000 7f21317fa700 1 -- 172.21.15.99:0/3994048093 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f212c052770 con 0x7f213c15b560 2023-10-08T00:21:22.991 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:22.990+0000 7f21433f1700 1 -- 172.21.15.99:0/3994048093 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]} v 0) v1 -- 0x7f213c04c3c0 con 0x7f213c15b560 2023-10-08T00:21:23.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.253+0000 7f21317fa700 1 -- 172.21.15.99:0/3994048093 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]}]=0 v487) v1 ==== 131+0+0 (secure 0 0 0) 0x7f212c055d90 con 0x7f213c15b560 2023-10-08T00:21:23.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.255+0000 7f21433f1700 1 -- 172.21.15.99:0/3994048093 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2120066f30 msgr2=0x7f21200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:23.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.255+0000 7f21433f1700 1 --2- 172.21.15.99:0/3994048093 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2120066f30 0x7f21200693e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f2138001200 tx=0x7f2138009040 comp rx=0 tx=0).stop 2023-10-08T00:21:23.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.255+0000 7f21433f1700 1 -- 172.21.15.99:0/3994048093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f213c15b560 msgr2=0x7f213c15b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:23.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.255+0000 7f21433f1700 1 --2- 172.21.15.99:0/3994048093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f213c15b560 0x7f213c15b9d0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f212c006c80 tx=0x7f212c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:21:23.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.255+0000 7f21433f1700 1 -- 172.21.15.99:0/3994048093 shutdown_connections 2023-10-08T00:21:23.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.255+0000 7f21433f1700 1 --2- 172.21.15.99:0/3994048093 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2120066f30 0x7f21200693e0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:23.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.255+0000 7f21433f1700 1 --2- 172.21.15.99:0/3994048093 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f21280051c0 0x7f213c148700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:23.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.255+0000 7f21433f1700 1 --2- 172.21.15.99:0/3994048093 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f213c0b22a0 0x7f213c0b2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:23.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.255+0000 7f21433f1700 1 --2- 172.21.15.99:0/3994048093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f213c15b560 0x7f213c15b9d0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:23.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.255+0000 7f21433f1700 1 -- 172.21.15.99:0/3994048093 >> 172.21.15.99:0/3994048093 conn(0x7f213c069870 msgr2=0x7f213c0c38e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:23.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.255+0000 7f21433f1700 1 -- 172.21.15.99:0/3994048093 shutdown_connections 2023-10-08T00:21:23.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.255+0000 7f21433f1700 1 -- 172.21.15.99:0/3994048093 wait complete. 2023-10-08T00:21:23.269 INFO:tasks.cephfs_test_runner:That when the full flag is encountered during asynchronous ... ok 2023-10-08T00:21:23.269 INFO:tasks.cephfs_test_runner:Starting test: test_full_same_file (tasks.cephfs.test_full.TestQuotaFull) 2023-10-08T00:21:23.269 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file' 2023-10-08T00:21:23.385 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.382+0000 7f00c1ded700 1 Processor -- start 2023-10-08T00:21:23.385 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.382+0000 7f00c1ded700 1 -- start start 2023-10-08T00:21:23.385 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.382+0000 7f00c1ded700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f00bc156570 0x7f00bc06d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:23.385 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.382+0000 7f00c1ded700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f00bc157050 0x7f00bc06d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:23.385 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.382+0000 7f00c1ded700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00bc157c30 con 0x7f00bc15a330 2023-10-08T00:21:23.385 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.382+0000 7f00c1ded700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00bc157490 con 0x7f00bc157050 2023-10-08T00:21:23.385 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.382+0000 7f00c1ded700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00bc1584f0 con 0x7f00bc156570 2023-10-08T00:21:23.385 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.382+0000 7f00bbfff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f00bc157050 0x7f00bc06d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:23.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.382+0000 7f00bbfff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f00bc157050 0x7f00bc06d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53546/0 (socket says 172.21.15.99:53546) 2023-10-08T00:21:23.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.382+0000 7f00bbfff700 1 -- 172.21.15.99:0/202156528 learned_addr learned my addr 172.21.15.99:0/202156528 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:23.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.382+0000 7f00bb7fe700 1 --2- 172.21.15.99:0/202156528 >> v2:172.21.15.191:3300/0 conn(0x7f00bc156570 0x7f00bc06d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:23.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.382+0000 7f00bbfff700 1 -- 172.21.15.99:0/202156528 >> v2:172.21.15.191:3300/0 conn(0x7f00bc156570 msgr2=0x7f00bc06d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:23.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.382+0000 7f00bbfff700 1 --2- 172.21.15.99:0/202156528 >> v2:172.21.15.191:3300/0 conn(0x7f00bc156570 0x7f00bc06d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:23.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.382+0000 7f00bbfff700 1 -- 172.21.15.99:0/202156528 >> v1:172.21.15.191:6789/0 conn(0x7f00bc15a330 legacy=0x7f00bc156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:23.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.382+0000 7f00bbfff700 1 -- 172.21.15.99:0/202156528 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00bc157670 con 0x7f00bc157050 2023-10-08T00:21:23.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.382+0000 7f00bbfff700 1 --2- 172.21.15.99:0/202156528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00bc157050 0x7f00bc06d8c0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f00bc0c9e60 tx=0x7f00b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4521e62254cbb13 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:23.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.382+0000 7f00ba7fc700 1 -- 172.21.15.99:0/202156528 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b4009e70 con 0x7f00bc157050 2023-10-08T00:21:23.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.382+0000 7f00ba7fc700 1 -- 172.21.15.99:0/202156528 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00b400b860 con 0x7f00bc157050 2023-10-08T00:21:23.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.382+0000 7f00ba7fc700 1 -- 172.21.15.99:0/202156528 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b400bc20 con 0x7f00bc157050 2023-10-08T00:21:23.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.383+0000 7f00c1ded700 1 -- 172.21.15.99:0/202156528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00bc157050 msgr2=0x7f00bc06d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:23.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.383+0000 7f00c1ded700 1 --2- 172.21.15.99:0/202156528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00bc157050 0x7f00bc06d8c0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f00bc0c9e60 tx=0x7f00b4009250 comp rx=0 tx=0).stop 2023-10-08T00:21:23.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.383+0000 7f00c1ded700 1 -- 172.21.15.99:0/202156528 shutdown_connections 2023-10-08T00:21:23.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.383+0000 7f00c1ded700 1 --2- 172.21.15.99:0/202156528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00bc157050 0x7f00bc06d8c0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:23.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.383+0000 7f00c1ded700 1 --2- 172.21.15.99:0/202156528 >> v2:172.21.15.191:3300/0 conn(0x7f00bc156570 0x7f00bc06d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:23.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.383+0000 7f00c1ded700 1 -- 172.21.15.99:0/202156528 >> 172.21.15.99:0/202156528 conn(0x7f00bc067660 msgr2=0x7f00bc067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:23.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.383+0000 7f00c1ded700 1 -- 172.21.15.99:0/202156528 shutdown_connections 2023-10-08T00:21:23.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.383+0000 7f00c1ded700 1 -- 172.21.15.99:0/202156528 wait complete. 2023-10-08T00:21:23.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.384+0000 7f00c1ded700 1 Processor -- start 2023-10-08T00:21:23.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.384+0000 7f00c1ded700 1 -- start start 2023-10-08T00:21:23.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.385+0000 7f00c1ded700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00bc156570 0x7f00bc0c25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:23.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.385+0000 7f00c1ded700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00bc157050 0x7f00bc0c2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:23.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.385+0000 7f00c1ded700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00bc15a330 0x7f00bc0c5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:23.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.385+0000 7f00c1ded700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00bc0c52a0 con 0x7f00bc157050 2023-10-08T00:21:23.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.385+0000 7f00c1ded700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00bc14f290 con 0x7f00bc156570 2023-10-08T00:21:23.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.385+0000 7f00c1ded700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00bc0c10f0 con 0x7f00bc15a330 2023-10-08T00:21:23.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.385+0000 7f00baffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00bc157050 0x7f00bc0c2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:23.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.385+0000 7f00baffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00bc157050 0x7f00bc0c2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53560/0 (socket says 172.21.15.99:53560) 2023-10-08T00:21:23.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.385+0000 7f00baffd700 1 -- 172.21.15.99:0/1969706441 learned_addr learned my addr 172.21.15.99:0/1969706441 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:23.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.385+0000 7f00bb7fe700 1 --2- 172.21.15.99:0/1969706441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00bc156570 0x7f00bc0c25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:23.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.385+0000 7f00bbfff700 1 --2- 172.21.15.99:0/1969706441 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00bc15a330 0x7f00bc0c5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:23.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.385+0000 7f00baffd700 1 -- 172.21.15.99:0/1969706441 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00bc15a330 msgr2=0x7f00bc0c5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:23.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.385+0000 7f00baffd700 1 --2- 172.21.15.99:0/1969706441 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00bc15a330 0x7f00bc0c5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:23.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.385+0000 7f00baffd700 1 -- 172.21.15.99:0/1969706441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00bc156570 msgr2=0x7f00bc0c25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:23.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.385+0000 7f00baffd700 1 --2- 172.21.15.99:0/1969706441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00bc156570 0x7f00bc0c25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:23.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.385+0000 7f00baffd700 1 -- 172.21.15.99:0/1969706441 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00b40094f0 con 0x7f00bc157050 2023-10-08T00:21:23.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.385+0000 7f00bb7fe700 1 --2- 172.21.15.99:0/1969706441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00bc156570 0x7f00bc0c25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:21:23.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.385+0000 7f00baffd700 1 --2- 172.21.15.99:0/1969706441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00bc157050 0x7f00bc0c2b10 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f00a400c7f0 tx=0x7f00a4007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:23.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.385+0000 7f00b37fe700 1 -- 172.21.15.99:0/1969706441 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a4011590 con 0x7f00bc157050 2023-10-08T00:21:23.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.385+0000 7f00b37fe700 1 -- 172.21.15.99:0/1969706441 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00a40116f0 con 0x7f00bc157050 2023-10-08T00:21:23.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.385+0000 7f00b37fe700 1 -- 172.21.15.99:0/1969706441 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a4011ab0 con 0x7f00bc157050 2023-10-08T00:21:23.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.385+0000 7f00c1ded700 1 -- 172.21.15.99:0/1969706441 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00bc0c13d0 con 0x7f00bc157050 2023-10-08T00:21:23.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.386+0000 7f00c1ded700 1 -- 172.21.15.99:0/1969706441 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00bc0c1990 con 0x7f00bc157050 2023-10-08T00:21:23.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.386+0000 7f00c1ded700 1 -- 172.21.15.99:0/1969706441 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00bc0c5070 con 0x7f00bc157050 2023-10-08T00:21:23.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.387+0000 7f00b37fe700 1 -- 172.21.15.99:0/1969706441 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f00a401b3b0 con 0x7f00bc157050 2023-10-08T00:21:23.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.387+0000 7f00b37fe700 1 --2- 172.21.15.99:0/1969706441 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f009c066f30 0x7f009c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:23.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.387+0000 7f00b37fe700 1 -- 172.21.15.99:0/1969706441 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f00a4083750 con 0x7f00bc157050 2023-10-08T00:21:23.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.387+0000 7f00bb7fe700 1 --2- 172.21.15.99:0/1969706441 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f009c066f30 0x7f009c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:23.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.388+0000 7f00bb7fe700 1 --2- 172.21.15.99:0/1969706441 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f009c066f30 0x7f009c0693e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f00bc0af710 tx=0x7f00ac00b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:23.393 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.392+0000 7f00b37fe700 1 -- 172.21.15.99:0/1969706441 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f00bc0c5070 con 0x7f00bc157050 2023-10-08T00:21:23.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:23.648+0000 7f00c1ded700 1 -- 172.21.15.99:0/1969706441 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]} v 0) v1 -- 0x7f00bc0c5070 con 0x7f00bc157050 2023-10-08T00:21:24.198 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:21:24.200 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:21:24.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.257+0000 7f00b37fe700 1 -- 172.21.15.99:0/1969706441 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]}]=0 v488) v1 ==== 138+0+0 (secure 0 0 0) 0x7f00bc0c5070 con 0x7f00bc157050 2023-10-08T00:21:24.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.259+0000 7f00c1ded700 1 -- 172.21.15.99:0/1969706441 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f009c066f30 msgr2=0x7f009c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:24.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.259+0000 7f00c1ded700 1 --2- 172.21.15.99:0/1969706441 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f009c066f30 0x7f009c0693e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f00bc0af710 tx=0x7f00ac00b040 comp rx=0 tx=0).stop 2023-10-08T00:21:24.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.259+0000 7f00c1ded700 1 -- 172.21.15.99:0/1969706441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00bc157050 msgr2=0x7f00bc0c2b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:24.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.259+0000 7f00c1ded700 1 --2- 172.21.15.99:0/1969706441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00bc157050 0x7f00bc0c2b10 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f00a400c7f0 tx=0x7f00a4007f50 comp rx=0 tx=0).stop 2023-10-08T00:21:24.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.259+0000 7f00c1ded700 1 -- 172.21.15.99:0/1969706441 shutdown_connections 2023-10-08T00:21:24.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.259+0000 7f00c1ded700 1 --2- 172.21.15.99:0/1969706441 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f009c066f30 0x7f009c0693e0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:24.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.259+0000 7f00c1ded700 1 --2- 172.21.15.99:0/1969706441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00bc156570 0x7f00bc0c25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:24.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.259+0000 7f00c1ded700 1 --2- 172.21.15.99:0/1969706441 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00bc15a330 0x7f00bc0c5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:24.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.259+0000 7f00c1ded700 1 --2- 172.21.15.99:0/1969706441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00bc157050 0x7f00bc0c2b10 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:24.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.259+0000 7f00c1ded700 1 -- 172.21.15.99:0/1969706441 >> 172.21.15.99:0/1969706441 conn(0x7f00bc067660 msgr2=0x7f00bc0c8b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:24.262 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.259+0000 7f00c1ded700 1 -- 172.21.15.99:0/1969706441 shutdown_connections 2023-10-08T00:21:24.263 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.259+0000 7f00c1ded700 1 -- 172.21.15.99:0/1969706441 wait complete. 2023-10-08T00:21:24.273 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T00:21:24.431 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.430+0000 7f6d9ffff700 1 Processor -- start 2023-10-08T00:21:24.431 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.430+0000 7f6d9ffff700 1 -- start start 2023-10-08T00:21:24.434 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.431+0000 7f6d9ffff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f6da00b06b0 0x7f6da00b0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:24.434 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.431+0000 7f6d9ffff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6da00cf5a0 0x7f6da00b1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:24.434 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.431+0000 7f6d9ffff700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6da0069680 con 0x7f6da0069240 2023-10-08T00:21:24.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.431+0000 7f6d9ffff700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6da00ce7e0 con 0x7f6da00cf5a0 2023-10-08T00:21:24.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.431+0000 7f6d9ffff700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6da00ceac0 con 0x7f6da00b06b0 2023-10-08T00:21:24.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.431+0000 7f6d967fc700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6da00cf5a0 0x7f6da00b1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:24.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.431+0000 7f6d967fc700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6da00cf5a0 0x7f6da00b1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53568/0 (socket says 172.21.15.99:53568) 2023-10-08T00:21:24.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.431+0000 7f6d967fc700 1 -- 172.21.15.99:0/3797689125 learned_addr learned my addr 172.21.15.99:0/3797689125 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:24.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.431+0000 7f6d9effd700 1 --2- 172.21.15.99:0/3797689125 >> v2:172.21.15.191:3300/0 conn(0x7f6da00b06b0 0x7f6da00b0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:24.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.431+0000 7f6d967fc700 1 -- 172.21.15.99:0/3797689125 >> v2:172.21.15.191:3300/0 conn(0x7f6da00b06b0 msgr2=0x7f6da00b0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:24.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.431+0000 7f6d967fc700 1 --2- 172.21.15.99:0/3797689125 >> v2:172.21.15.191:3300/0 conn(0x7f6da00b06b0 0x7f6da00b0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:24.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.431+0000 7f6d967fc700 1 -- 172.21.15.99:0/3797689125 >> v1:172.21.15.191:6789/0 conn(0x7f6da0069240 legacy=0x7f6da01597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:24.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.431+0000 7f6d967fc700 1 -- 172.21.15.99:0/3797689125 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da00ceda0 con 0x7f6da00cf5a0 2023-10-08T00:21:24.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.432+0000 7f6d967fc700 1 --2- 172.21.15.99:0/3797689125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6da00cf5a0 0x7f6da00b1850 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f6da006e700 tx=0x7f6d88009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db3ed5d7879a882c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:24.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.432+0000 7f6d9e7fc700 1 -- 172.21.15.99:0/3797689125 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d88009e70 con 0x7f6da00cf5a0 2023-10-08T00:21:24.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.432+0000 7f6d9e7fc700 1 -- 172.21.15.99:0/3797689125 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d8800b860 con 0x7f6da00cf5a0 2023-10-08T00:21:24.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.432+0000 7f6d9e7fc700 1 -- 172.21.15.99:0/3797689125 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8800bc00 con 0x7f6da00cf5a0 2023-10-08T00:21:24.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.432+0000 7f6d9ffff700 1 -- 172.21.15.99:0/3797689125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6da00cf5a0 msgr2=0x7f6da00b1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:24.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.432+0000 7f6d9ffff700 1 --2- 172.21.15.99:0/3797689125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6da00cf5a0 0x7f6da00b1850 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f6da006e700 tx=0x7f6d88009250 comp rx=0 tx=0).stop 2023-10-08T00:21:24.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.432+0000 7f6d9ffff700 1 -- 172.21.15.99:0/3797689125 shutdown_connections 2023-10-08T00:21:24.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.432+0000 7f6d9ffff700 1 --2- 172.21.15.99:0/3797689125 >> v2:172.21.15.191:3300/0 conn(0x7f6da00b06b0 0x7f6da00b0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:24.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.432+0000 7f6d9ffff700 1 --2- 172.21.15.99:0/3797689125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6da00cf5a0 0x7f6da00b1850 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:24.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.432+0000 7f6d9ffff700 1 -- 172.21.15.99:0/3797689125 >> 172.21.15.99:0/3797689125 conn(0x7f6da006c700 msgr2=0x7f6da0069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:24.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.432+0000 7f6d9ffff700 1 -- 172.21.15.99:0/3797689125 shutdown_connections 2023-10-08T00:21:24.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.432+0000 7f6d9ffff700 1 -- 172.21.15.99:0/3797689125 wait complete. 2023-10-08T00:21:24.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.433+0000 7f6d9ffff700 1 Processor -- start 2023-10-08T00:21:24.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.434+0000 7f6d9ffff700 1 -- start start 2023-10-08T00:21:24.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.434+0000 7f6d9ffff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6da0069240 0x7f6da0146390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:24.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.434+0000 7f6d9ffff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6da00b06b0 0x7f6da015d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:24.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.434+0000 7f6d9ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6da00cf5a0 0x7f6da015dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:24.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.434+0000 7f6d9ffff700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da016a570 con 0x7f6da00cf5a0 2023-10-08T00:21:24.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.434+0000 7f6d9ffff700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da00ce220 con 0x7f6da0069240 2023-10-08T00:21:24.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.434+0000 7f6d9ffff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6da016eb10 con 0x7f6da00b06b0 2023-10-08T00:21:24.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.434+0000 7f6d9f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6da00cf5a0 0x7f6da015dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:24.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.434+0000 7f6d9f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6da00cf5a0 0x7f6da015dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53584/0 (socket says 172.21.15.99:53584) 2023-10-08T00:21:24.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.434+0000 7f6d9f7fe700 1 -- 172.21.15.99:0/1198551392 learned_addr learned my addr 172.21.15.99:0/1198551392 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:24.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.434+0000 7f6d9effd700 1 --2- 172.21.15.99:0/1198551392 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6da0069240 0x7f6da0146390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:24.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.434+0000 7f6d967fc700 1 --2- 172.21.15.99:0/1198551392 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6da00b06b0 0x7f6da015d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:24.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.435+0000 7f6d9f7fe700 1 -- 172.21.15.99:0/1198551392 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6da00b06b0 msgr2=0x7f6da015d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:24.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.435+0000 7f6d9f7fe700 1 --2- 172.21.15.99:0/1198551392 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6da00b06b0 0x7f6da015d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:24.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.435+0000 7f6d9f7fe700 1 -- 172.21.15.99:0/1198551392 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6da0069240 msgr2=0x7f6da0146390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:24.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.435+0000 7f6d9f7fe700 1 --2- 172.21.15.99:0/1198551392 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6da0069240 0x7f6da0146390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:24.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.435+0000 7f6d9f7fe700 1 -- 172.21.15.99:0/1198551392 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d880094f0 con 0x7f6da00cf5a0 2023-10-08T00:21:24.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.435+0000 7f6d967fc700 1 --2- 172.21.15.99:0/1198551392 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6da00b06b0 0x7f6da015d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:21:24.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.435+0000 7f6d9f7fe700 1 --2- 172.21.15.99:0/1198551392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6da00cf5a0 0x7f6da015dca0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f6d9800c8e0 tx=0x7f6d98001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:24.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.435+0000 7f6d977fe700 1 -- 172.21.15.99:0/1198551392 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d98012630 con 0x7f6da00cf5a0 2023-10-08T00:21:24.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.435+0000 7f6d977fe700 1 -- 172.21.15.99:0/1198551392 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d98012790 con 0x7f6da00cf5a0 2023-10-08T00:21:24.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.435+0000 7f6d9ffff700 1 -- 172.21.15.99:0/1198551392 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6da016ada0 con 0x7f6da00cf5a0 2023-10-08T00:21:24.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.435+0000 7f6d977fe700 1 -- 172.21.15.99:0/1198551392 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d98012b50 con 0x7f6da00cf5a0 2023-10-08T00:21:24.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.435+0000 7f6d9ffff700 1 -- 172.21.15.99:0/1198551392 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6da016a7d0 con 0x7f6da00cf5a0 2023-10-08T00:21:24.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.436+0000 7f6d9ffff700 1 -- 172.21.15.99:0/1198551392 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6da004c3c0 con 0x7f6da00cf5a0 2023-10-08T00:21:24.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.436+0000 7f6d977fe700 1 -- 172.21.15.99:0/1198551392 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6d9800a3b0 con 0x7f6da00cf5a0 2023-10-08T00:21:24.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.437+0000 7f6d977fe700 1 --2- 172.21.15.99:0/1198551392 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6d80066f30 0x7f6d800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:24.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.437+0000 7f6d977fe700 1 -- 172.21.15.99:0/1198551392 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f6d980844c0 con 0x7f6da00cf5a0 2023-10-08T00:21:24.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.437+0000 7f6d9effd700 1 --2- 172.21.15.99:0/1198551392 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6d80066f30 0x7f6d800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:24.443 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.437+0000 7f6d9effd700 1 --2- 172.21.15.99:0/1198551392 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6d80066f30 0x7f6d800693e0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6da00afb80 tx=0x7f6d900091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:24.443 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.441+0000 7f6d977fe700 1 -- 172.21.15.99:0/1198551392 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6d980554c0 con 0x7f6da00cf5a0 2023-10-08T00:21:24.623 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.623+0000 7f6d9ffff700 1 -- 172.21.15.99:0/1198551392 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f6da016aaf0 con 0x7f6da00cf5a0 2023-10-08T00:21:24.627 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.626+0000 7f6d977fe700 1 -- 172.21.15.99:0/1198551392 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6d98055050 con 0x7f6da00cf5a0 2023-10-08T00:21:24.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.628+0000 7f6d9ffff700 1 -- 172.21.15.99:0/1198551392 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6d80066f30 msgr2=0x7f6d800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:24.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.628+0000 7f6d9ffff700 1 --2- 172.21.15.99:0/1198551392 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6d80066f30 0x7f6d800693e0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6da00afb80 tx=0x7f6d900091a0 comp rx=0 tx=0).stop 2023-10-08T00:21:24.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.628+0000 7f6d9ffff700 1 -- 172.21.15.99:0/1198551392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6da00cf5a0 msgr2=0x7f6da015dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:24.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.628+0000 7f6d9ffff700 1 --2- 172.21.15.99:0/1198551392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6da00cf5a0 0x7f6da015dca0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f6d9800c8e0 tx=0x7f6d98001710 comp rx=0 tx=0).stop 2023-10-08T00:21:24.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.628+0000 7f6d9ffff700 1 -- 172.21.15.99:0/1198551392 shutdown_connections 2023-10-08T00:21:24.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.628+0000 7f6d9ffff700 1 --2- 172.21.15.99:0/1198551392 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6d80066f30 0x7f6d800693e0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:24.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.628+0000 7f6d9ffff700 1 --2- 172.21.15.99:0/1198551392 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6da0069240 0x7f6da0146390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:24.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.628+0000 7f6d9ffff700 1 --2- 172.21.15.99:0/1198551392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6da00cf5a0 0x7f6da015dca0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:24.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.628+0000 7f6d9ffff700 1 --2- 172.21.15.99:0/1198551392 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6da00b06b0 0x7f6da015d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:24.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.628+0000 7f6d9ffff700 1 -- 172.21.15.99:0/1198551392 >> 172.21.15.99:0/1198551392 conn(0x7f6da006c700 msgr2=0x7f6da0067df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:24.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.628+0000 7f6d9ffff700 1 -- 172.21.15.99:0/1198551392 shutdown_connections 2023-10-08T00:21:24.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.628+0000 7f6d9ffff700 1 -- 172.21.15.99:0/1198551392 wait complete. 2023-10-08T00:21:24.641 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T00:21:24.641 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T00:21:24.680 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T00:21:24.725 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T00:21:24.725 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-08T00:21:24.725 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:21:24.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.839+0000 7f5982795700 1 Processor -- start 2023-10-08T00:21:24.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.839+0000 7f5982795700 1 -- start start 2023-10-08T00:21:24.845 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.840+0000 7f5982795700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f597c157870 0x7f597c15b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:24.845 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.840+0000 7f5982795700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f597c15a620 0x7f597c15d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:24.845 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.840+0000 7f5982795700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f597c15cbb0 con 0x7f597c15d580 2023-10-08T00:21:24.845 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.840+0000 7f5982795700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f597c15bfa0 con 0x7f597c157870 2023-10-08T00:21:24.845 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.840+0000 7f5982795700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f597c15c280 con 0x7f597c15a620 2023-10-08T00:21:24.845 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.840+0000 7f597bfff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f597c157870 0x7f597c15b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:24.845 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.840+0000 7f5980d32700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f597c15d580 0x7f597c158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:53466/0 (socket says 172.21.15.99:53466) 2023-10-08T00:21:24.846 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.840+0000 7f597bfff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f597c157870 0x7f597c15b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53598/0 (socket says 172.21.15.99:53598) 2023-10-08T00:21:24.846 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.840+0000 7f5980d32700 1 -- 172.21.15.99:0/1989577157 learned_addr learned my addr 172.21.15.99:0/1989577157 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:21:24.846 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.840+0000 7f597b7fe700 1 --2- 172.21.15.99:0/1989577157 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f597c15a620 0x7f597c15d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:24.846 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.840+0000 7f597bfff700 1 -- 172.21.15.99:0/1989577157 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f597c15a620 msgr2=0x7f597c15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:24.846 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.840+0000 7f597bfff700 1 --2- 172.21.15.99:0/1989577157 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f597c15a620 0x7f597c15d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:24.846 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.840+0000 7f597bfff700 1 -- 172.21.15.99:0/1989577157 >> v1:172.21.15.99:6789/0 conn(0x7f597c15d580 legacy=0x7f597c158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:24.846 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.840+0000 7f597bfff700 1 -- 172.21.15.99:0/1989577157 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f597c15c560 con 0x7f597c157870 2023-10-08T00:21:24.846 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.840+0000 7f597b7fe700 1 --2- 172.21.15.99:0/1989577157 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f597c15a620 0x7f597c15d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:21:24.846 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.841+0000 7f597bfff700 1 --2- 172.21.15.99:0/1989577157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f597c157870 0x7f597c15b6a0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f596c0098e0 tx=0x7f596c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f334926a344370f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:24.846 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.841+0000 7f597affd700 1 -- 172.21.15.99:0/1989577157 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596c009e80 con 0x7f597c157870 2023-10-08T00:21:24.847 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.841+0000 7f597affd700 1 -- 172.21.15.99:0/1989577157 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f596c00b860 con 0x7f597c157870 2023-10-08T00:21:24.847 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.841+0000 7f597affd700 1 -- 172.21.15.99:0/1989577157 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596c00bbe0 con 0x7f597c157870 2023-10-08T00:21:24.847 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.841+0000 7f5982795700 1 -- 172.21.15.99:0/1989577157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f597c157870 msgr2=0x7f597c15b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:24.847 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.841+0000 7f5982795700 1 --2- 172.21.15.99:0/1989577157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f597c157870 0x7f597c15b6a0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f596c0098e0 tx=0x7f596c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:21:24.847 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.841+0000 7f5982795700 1 -- 172.21.15.99:0/1989577157 shutdown_connections 2023-10-08T00:21:24.847 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.841+0000 7f5982795700 1 --2- 172.21.15.99:0/1989577157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f597c157870 0x7f597c15b6a0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:24.847 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.841+0000 7f5982795700 1 --2- 172.21.15.99:0/1989577157 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f597c15a620 0x7f597c15d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:24.847 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.841+0000 7f5982795700 1 -- 172.21.15.99:0/1989577157 >> 172.21.15.99:0/1989577157 conn(0x7f597c068ce0 msgr2=0x7f597c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:24.847 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.841+0000 7f5982795700 1 -- 172.21.15.99:0/1989577157 shutdown_connections 2023-10-08T00:21:24.848 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.841+0000 7f5982795700 1 -- 172.21.15.99:0/1989577157 wait complete. 2023-10-08T00:21:24.848 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.842+0000 7f5982795700 1 Processor -- start 2023-10-08T00:21:24.848 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.842+0000 7f5982795700 1 -- start start 2023-10-08T00:21:24.848 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.843+0000 7f5982795700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f597c157870 0x7f597c14b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:24.848 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.843+0000 7f5982795700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f597c15a620 0x7f597c0d0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:24.849 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.843+0000 7f5982795700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f597c15d580 0x7f597c1507e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:24.849 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.843+0000 7f5982795700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597c0b1570 con 0x7f597c15d580 2023-10-08T00:21:24.849 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.843+0000 7f5982795700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597c14bdf0 con 0x7f597c15a620 2023-10-08T00:21:24.849 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.843+0000 7f5982795700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f597c0b0960 con 0x7f597c157870 2023-10-08T00:21:24.849 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.843+0000 7f5980d32700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f597c15d580 0x7f597c1507e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:24.849 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.843+0000 7f5980d32700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f597c15d580 0x7f597c1507e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53602/0 (socket says 172.21.15.99:53602) 2023-10-08T00:21:24.850 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.843+0000 7f5980d32700 1 -- 172.21.15.99:0/409761330 learned_addr learned my addr 172.21.15.99:0/409761330 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:24.850 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.843+0000 7f597bfff700 1 --2- 172.21.15.99:0/409761330 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f597c157870 0x7f597c14b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:24.850 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.843+0000 7f597b7fe700 1 --2- 172.21.15.99:0/409761330 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f597c15a620 0x7f597c0d0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:24.850 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.844+0000 7f5980d32700 1 -- 172.21.15.99:0/409761330 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f597c157870 msgr2=0x7f597c14b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:24.850 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.844+0000 7f5980d32700 1 --2- 172.21.15.99:0/409761330 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f597c157870 0x7f597c14b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:24.851 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.844+0000 7f5980d32700 1 -- 172.21.15.99:0/409761330 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f597c15a620 msgr2=0x7f597c0d0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:24.851 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.844+0000 7f5980d32700 1 --2- 172.21.15.99:0/409761330 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f597c15a620 0x7f597c0d0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:24.851 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.844+0000 7f5980d32700 1 -- 172.21.15.99:0/409761330 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f596c009420 con 0x7f597c15d580 2023-10-08T00:21:24.851 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.844+0000 7f5980d32700 1 --2- 172.21.15.99:0/409761330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f597c15d580 0x7f597c1507e0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f597400cbc0 tx=0x7f5974001990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:24.851 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.844+0000 7f5963fff700 1 -- 172.21.15.99:0/409761330 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597400a850 con 0x7f597c15d580 2023-10-08T00:21:24.851 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.844+0000 7f5963fff700 1 -- 172.21.15.99:0/409761330 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f597400a9b0 con 0x7f597c15d580 2023-10-08T00:21:24.851 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.844+0000 7f5963fff700 1 -- 172.21.15.99:0/409761330 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597400ad30 con 0x7f597c15d580 2023-10-08T00:21:24.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.844+0000 7f5982795700 1 -- 172.21.15.99:0/409761330 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f597c0b0c40 con 0x7f597c15d580 2023-10-08T00:21:24.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.844+0000 7f5982795700 1 -- 172.21.15.99:0/409761330 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f597c0b1200 con 0x7f597c15d580 2023-10-08T00:21:24.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.845+0000 7f5982795700 1 -- 172.21.15.99:0/409761330 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f597c0b16e0 con 0x7f597c15d580 2023-10-08T00:21:24.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.845+0000 7f5963fff700 1 -- 172.21.15.99:0/409761330 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5974007300 con 0x7f597c15d580 2023-10-08T00:21:24.853 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.845+0000 7f5963fff700 1 --2- 172.21.15.99:0/409761330 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5968066f30 0x7f59680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:24.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.846+0000 7f5963fff700 1 -- 172.21.15.99:0/409761330 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f5974084500 con 0x7f597c15d580 2023-10-08T00:21:24.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.846+0000 7f597bfff700 1 --2- 172.21.15.99:0/409761330 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5968066f30 0x7f59680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:24.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.846+0000 7f597bfff700 1 --2- 172.21.15.99:0/409761330 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5968066f30 0x7f59680693e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f596c0091e0 tx=0x7f596c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:24.854 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:24.850+0000 7f5963fff700 1 -- 172.21.15.99:0/409761330 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5974051a60 con 0x7f597c15d580 2023-10-08T00:21:25.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.075+0000 7f5982795700 1 -- 172.21.15.99:0/409761330 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f597c04c3c0 con 0x7f597c15d580 2023-10-08T00:21:25.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.075+0000 7f5963fff700 1 -- 172.21.15.99:0/409761330 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+3529 (secure 0 0 0) 0x7f5974055080 con 0x7f597c15d580 2023-10-08T00:21:25.076 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:21:25.077 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":79,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6257,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/661016677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":661016677},{"type":"v1","addr":"172.21.15.191:6837","nonce":661016677}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":7012,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/4102051559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":4102051559},{"type":"v1","addr":"172.21.15.99:6835","nonce":4102051559}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":7021,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/658854389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":658854389},{"type":"v1","addr":"172.21.15.191:6836","nonce":658854389}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":8043,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/2616564319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2616564319},{"type":"v1","addr":"172.21.15.99:6834","nonce":2616564319}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79}],"filesystems":[]} 2023-10-08T00:21:25.079 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.078+0000 7f5982795700 1 -- 172.21.15.99:0/409761330 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5968066f30 msgr2=0x7f59680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:25.079 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.078+0000 7f5982795700 1 --2- 172.21.15.99:0/409761330 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5968066f30 0x7f59680693e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f596c0091e0 tx=0x7f596c017000 comp rx=0 tx=0).stop 2023-10-08T00:21:25.079 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.078+0000 7f5982795700 1 -- 172.21.15.99:0/409761330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f597c15d580 msgr2=0x7f597c1507e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:25.079 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.078+0000 7f5982795700 1 --2- 172.21.15.99:0/409761330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f597c15d580 0x7f597c1507e0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f597400cbc0 tx=0x7f5974001990 comp rx=0 tx=0).stop 2023-10-08T00:21:25.079 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.078+0000 7f5982795700 1 -- 172.21.15.99:0/409761330 shutdown_connections 2023-10-08T00:21:25.079 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.078+0000 7f5982795700 1 --2- 172.21.15.99:0/409761330 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5968066f30 0x7f59680693e0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:25.080 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.078+0000 7f5982795700 1 --2- 172.21.15.99:0/409761330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f597c15d580 0x7f597c1507e0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:25.080 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.078+0000 7f5982795700 1 --2- 172.21.15.99:0/409761330 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f597c157870 0x7f597c14b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:25.080 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.078+0000 7f5982795700 1 --2- 172.21.15.99:0/409761330 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f597c15a620 0x7f597c0d0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:25.080 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.078+0000 7f5982795700 1 -- 172.21.15.99:0/409761330 >> 172.21.15.99:0/409761330 conn(0x7f597c068ce0 msgr2=0x7f597c156aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:25.080 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.078+0000 7f5982795700 1 -- 172.21.15.99:0/409761330 shutdown_connections 2023-10-08T00:21:25.080 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.078+0000 7f5982795700 1 -- 172.21.15.99:0/409761330 wait complete. 2023-10-08T00:21:25.080 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 79 2023-10-08T00:21:25.091 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T00:21:25.091 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-08T00:21:25.091 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T00:21:25.091 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:21:25.091 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T00:21:25.092 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-08T00:21:25.092 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T00:21:25.092 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:21:25.092 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T00:21:25.092 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-08T00:21:25.093 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T00:21:25.093 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:21:25.093 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T00:21:25.093 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-08T00:21:25.093 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T00:21:25.093 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:21:25.099 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T00:21:25.099 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T00:21:25.100 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T00:21:25.100 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T00:21:25.103 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T00:21:25.104 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T00:21:25.104 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T00:21:25.104 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T00:21:25.142 INFO:tasks.ceph.mds.a:Started 2023-10-08T00:21:25.142 DEBUG:teuthology.parallel:result is None 2023-10-08T00:21:25.143 INFO:tasks.ceph.mds.c:Started 2023-10-08T00:21:25.143 DEBUG:teuthology.parallel:result is None 2023-10-08T00:21:25.146 INFO:tasks.ceph.mds.d:Started 2023-10-08T00:21:25.146 DEBUG:teuthology.parallel:result is None 2023-10-08T00:21:25.146 INFO:tasks.ceph.mds.b:Started 2023-10-08T00:21:25.147 DEBUG:teuthology.parallel:result is None 2023-10-08T00:21:25.147 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-08T00:21:25.214 INFO:tasks.ceph.mds.a.smithi099.stdout:starting mds.a at 2023-10-08T00:21:25.214 INFO:tasks.ceph.mds.c.smithi099.stdout:starting mds.c at 2023-10-08T00:21:25.225 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2023-10-08T00:21:25.225 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2023-10-08T00:21:25.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.269+0000 7f170d43d700 1 Processor -- start 2023-10-08T00:21:25.270 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.269+0000 7f170d43d700 1 -- start start 2023-10-08T00:21:25.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.270+0000 7f170d43d700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1708156a80 0x7f17080c8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:25.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.270+0000 7f170d43d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f170806a0e0 0x7f170806a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:25.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.270+0000 7f170d43d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f170806e1e0 0x7f170806e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:25.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.270+0000 7f170d43d700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f170806aa90 con 0x7f1708156a80 2023-10-08T00:21:25.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.270+0000 7f170d43d700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f170806ef50 con 0x7f170806e1e0 2023-10-08T00:21:25.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.270+0000 7f170d43d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17080c7e20 con 0x7f170806a0e0 2023-10-08T00:21:25.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.270+0000 7f1707fff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1708156a80 0x7f17080c8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:25.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.270+0000 7f1707fff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1708156a80 0x7f17080c8be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53638/0 (socket says 172.21.15.99:53638) 2023-10-08T00:21:25.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.270+0000 7f1707fff700 1 -- 172.21.15.99:0/1019673597 learned_addr learned my addr 172.21.15.99:0/1019673597 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:25.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.270+0000 7f17077fe700 1 --2- 172.21.15.99:0/1019673597 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f170806a0e0 0x7f170806a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:25.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.270+0000 7f170cc3c700 1 --2- 172.21.15.99:0/1019673597 >> v2:172.21.15.191:3300/0 conn(0x7f170806e1e0 0x7f170806e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:25.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.270+0000 7f1707fff700 1 -- 172.21.15.99:0/1019673597 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f170806a0e0 msgr2=0x7f170806a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:25.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.270+0000 7f1707fff700 1 --2- 172.21.15.99:0/1019673597 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f170806a0e0 0x7f170806a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:25.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.270+0000 7f1707fff700 1 -- 172.21.15.99:0/1019673597 >> v2:172.21.15.191:3300/0 conn(0x7f170806e1e0 msgr2=0x7f170806e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:25.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.270+0000 7f1707fff700 1 --2- 172.21.15.99:0/1019673597 >> v2:172.21.15.191:3300/0 conn(0x7f170806e1e0 0x7f170806e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:25.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.270+0000 7f1707fff700 1 -- 172.21.15.99:0/1019673597 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17080c8100 con 0x7f1708156a80 2023-10-08T00:21:25.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.270+0000 7f1707fff700 1 --2- 172.21.15.99:0/1019673597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1708156a80 0x7f17080c8be0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f17080ce8f0 tx=0x7f16f800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecb2d67fef7945ee server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:25.275 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.272+0000 7f1706ffd700 1 -- 172.21.15.99:0/1019673597 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f800db40 con 0x7f1708156a80 2023-10-08T00:21:25.275 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.272+0000 7f1706ffd700 1 -- 172.21.15.99:0/1019673597 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16f800dca0 con 0x7f1708156a80 2023-10-08T00:21:25.275 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.272+0000 7f1706ffd700 1 -- 172.21.15.99:0/1019673597 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f80196b0 con 0x7f1708156a80 2023-10-08T00:21:25.275 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.273+0000 7f170d43d700 1 -- 172.21.15.99:0/1019673597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1708156a80 msgr2=0x7f17080c8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:25.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.273+0000 7f170d43d700 1 --2- 172.21.15.99:0/1019673597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1708156a80 0x7f17080c8be0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f17080ce8f0 tx=0x7f16f800b4f0 comp rx=0 tx=0).stop 2023-10-08T00:21:25.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.273+0000 7f170d43d700 1 -- 172.21.15.99:0/1019673597 shutdown_connections 2023-10-08T00:21:25.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.273+0000 7f170d43d700 1 --2- 172.21.15.99:0/1019673597 >> v2:172.21.15.191:3300/0 conn(0x7f170806e1e0 0x7f170806e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:25.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.273+0000 7f170d43d700 1 --2- 172.21.15.99:0/1019673597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1708156a80 0x7f17080c8be0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:25.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.273+0000 7f170d43d700 1 --2- 172.21.15.99:0/1019673597 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f170806a0e0 0x7f170806a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:25.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.273+0000 7f170d43d700 1 -- 172.21.15.99:0/1019673597 >> 172.21.15.99:0/1019673597 conn(0x7f17080cd880 msgr2=0x7f17080cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:25.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.273+0000 7f170d43d700 1 -- 172.21.15.99:0/1019673597 shutdown_connections 2023-10-08T00:21:25.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.273+0000 7f170d43d700 1 -- 172.21.15.99:0/1019673597 wait complete. 2023-10-08T00:21:25.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.274+0000 7f170d43d700 1 Processor -- start 2023-10-08T00:21:25.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.274+0000 7f170d43d700 1 -- start start 2023-10-08T00:21:25.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.275+0000 7f170d43d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f170806a0e0 0x7f17080a6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:25.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.275+0000 7f170d43d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f170806e1e0 0x7f170816a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:25.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.275+0000 7f170d43d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1708156a80 0x7f17081457b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:25.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.275+0000 7f170d43d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17080a7660 con 0x7f170806a0e0 2023-10-08T00:21:25.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.275+0000 7f170d43d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170816a770 con 0x7f1708156a80 2023-10-08T00:21:25.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.275+0000 7f170d43d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17080a6a50 con 0x7f170806e1e0 2023-10-08T00:21:25.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.275+0000 7f1707fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f170806a0e0 0x7f17080a6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:25.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.275+0000 7f1707fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f170806a0e0 0x7f17080a6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53650/0 (socket says 172.21.15.99:53650) 2023-10-08T00:21:25.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.275+0000 7f1707fff700 1 -- 172.21.15.99:0/2612368678 learned_addr learned my addr 172.21.15.99:0/2612368678 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:25.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.275+0000 7f17077fe700 1 --2- 172.21.15.99:0/2612368678 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f170806e1e0 0x7f170816a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:25.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.275+0000 7f170cc3c700 1 --2- 172.21.15.99:0/2612368678 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1708156a80 0x7f17081457b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:25.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.275+0000 7f1707fff700 1 -- 172.21.15.99:0/2612368678 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f170806e1e0 msgr2=0x7f170816a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:25.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.275+0000 7f1707fff700 1 --2- 172.21.15.99:0/2612368678 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f170806e1e0 0x7f170816a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:25.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.275+0000 7f1707fff700 1 -- 172.21.15.99:0/2612368678 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1708156a80 msgr2=0x7f17081457b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:25.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.275+0000 7f1707fff700 1 --2- 172.21.15.99:0/2612368678 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1708156a80 0x7f17081457b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:25.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.275+0000 7f1707fff700 1 -- 172.21.15.99:0/2612368678 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16f800b790 con 0x7f170806a0e0 2023-10-08T00:21:25.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.276+0000 7f1707fff700 1 --2- 172.21.15.99:0/2612368678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f170806a0e0 0x7f17080a6470 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f16f800b530 tx=0x7f16f801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:25.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.276+0000 7f16effff700 1 -- 172.21.15.99:0/2612368678 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f8010680 con 0x7f170806a0e0 2023-10-08T00:21:25.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.276+0000 7f16effff700 1 -- 172.21.15.99:0/2612368678 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16f801e050 con 0x7f170806a0e0 2023-10-08T00:21:25.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.276+0000 7f170d43d700 1 -- 172.21.15.99:0/2612368678 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17080a6d30 con 0x7f170806a0e0 2023-10-08T00:21:25.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.276+0000 7f16effff700 1 -- 172.21.15.99:0/2612368678 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f8010a30 con 0x7f170806a0e0 2023-10-08T00:21:25.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.276+0000 7f170d43d700 1 -- 172.21.15.99:0/2612368678 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17080a72f0 con 0x7f170806a0e0 2023-10-08T00:21:25.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.277+0000 7f170d43d700 1 -- 172.21.15.99:0/2612368678 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f170806d5d0 con 0x7f170806a0e0 2023-10-08T00:21:25.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.277+0000 7f16effff700 1 -- 172.21.15.99:0/2612368678 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f16f8024470 con 0x7f170806a0e0 2023-10-08T00:21:25.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.277+0000 7f16effff700 1 --2- 172.21.15.99:0/2612368678 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f16f4066f30 0x7f16f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:25.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.277+0000 7f16effff700 1 -- 172.21.15.99:0/2612368678 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f16f8014070 con 0x7f170806a0e0 2023-10-08T00:21:25.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.278+0000 7f17077fe700 1 --2- 172.21.15.99:0/2612368678 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f16f4066f30 0x7f16f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:25.281 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.278+0000 7f17077fe700 1 --2- 172.21.15.99:0/2612368678 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f16f4066f30 0x7f16f40693e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f17080c6930 tx=0x7f16fc0091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:25.282 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.282+0000 7f16effff700 1 -- 172.21.15.99:0/2612368678 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f16f8055f80 con 0x7f170806a0e0 2023-10-08T00:21:25.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:25.477+0000 7f170d43d700 1 -- 172.21.15.99:0/2612368678 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f17080c83e0 con 0x7f170806a0e0 2023-10-08T00:21:26.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.273+0000 7f16effff700 1 -- 172.21.15.99:0/2612368678 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v180) v1 ==== 97+0+0 (secure 0 0 0) 0x7f16f80595a0 con 0x7f170806a0e0 2023-10-08T00:21:26.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.275+0000 7f170d43d700 1 -- 172.21.15.99:0/2612368678 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f16f4066f30 msgr2=0x7f16f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:26.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.275+0000 7f170d43d700 1 --2- 172.21.15.99:0/2612368678 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f16f4066f30 0x7f16f40693e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f17080c6930 tx=0x7f16fc0091a0 comp rx=0 tx=0).stop 2023-10-08T00:21:26.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.275+0000 7f170d43d700 1 -- 172.21.15.99:0/2612368678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f170806a0e0 msgr2=0x7f17080a6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:26.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.275+0000 7f170d43d700 1 --2- 172.21.15.99:0/2612368678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f170806a0e0 0x7f17080a6470 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f16f800b530 tx=0x7f16f801b000 comp rx=0 tx=0).stop 2023-10-08T00:21:26.276 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.275+0000 7f170d43d700 1 -- 172.21.15.99:0/2612368678 shutdown_connections 2023-10-08T00:21:26.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.275+0000 7f170d43d700 1 --2- 172.21.15.99:0/2612368678 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f16f4066f30 0x7f16f40693e0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:26.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.275+0000 7f170d43d700 1 --2- 172.21.15.99:0/2612368678 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1708156a80 0x7f17081457b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:26.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.275+0000 7f170d43d700 1 --2- 172.21.15.99:0/2612368678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f170806a0e0 0x7f17080a6470 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:26.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.275+0000 7f170d43d700 1 --2- 172.21.15.99:0/2612368678 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f170806e1e0 0x7f170816a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:26.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.275+0000 7f170d43d700 1 -- 172.21.15.99:0/2612368678 >> 172.21.15.99:0/2612368678 conn(0x7f17080cd880 msgr2=0x7f1708070d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:26.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.275+0000 7f170d43d700 1 -- 172.21.15.99:0/2612368678 shutdown_connections 2023-10-08T00:21:26.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.275+0000 7f170d43d700 1 -- 172.21.15.99:0/2612368678 wait complete. 2023-10-08T00:21:26.277 INFO:teuthology.orchestra.run.smithi099.stderr: removed all blocklist entries 2023-10-08T00:21:26.289 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-08T00:21:26.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.403+0000 7fcf5f677700 1 Processor -- start 2023-10-08T00:21:26.404 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.404+0000 7fcf5f677700 1 -- start start 2023-10-08T00:21:26.406 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.404+0000 7fcf5f677700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcf581523e0 0x7fcf5806c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:26.407 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.404+0000 7fcf5f677700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fcf58158e50 0x7fcf580cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:26.407 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.404+0000 7fcf5f677700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf5806c010 con 0x7fcf58152010 2023-10-08T00:21:26.407 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.404+0000 7fcf5f677700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf5806bd10 con 0x7fcf58158e50 2023-10-08T00:21:26.407 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.404+0000 7fcf5f677700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5815c060 con 0x7fcf581523e0 2023-10-08T00:21:26.407 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.405+0000 7fcf5d413700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fcf58152010 0x7fcf5815b810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:53484/0 (socket says 172.21.15.99:53484) 2023-10-08T00:21:26.407 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.405+0000 7fcf5d413700 1 -- 172.21.15.99:0/408212959 learned_addr learned my addr 172.21.15.99:0/408212959 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:21:26.407 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.405+0000 7fcf5dc14700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fcf58158e50 0x7fcf580cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:26.407 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.405+0000 7fcf5cc12700 1 --2- 172.21.15.99:0/408212959 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcf581523e0 0x7fcf5806c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:26.407 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.405+0000 7fcf4ffff700 1 -- 172.21.15.99:0/408212959 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1165695931 0 0) 0x7fcf5806c010 con 0x7fcf58152010 2023-10-08T00:21:26.407 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.405+0000 7fcf4ffff700 1 -- 172.21.15.99:0/408212959 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf44003570 con 0x7fcf58152010 2023-10-08T00:21:26.407 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.405+0000 7fcf5dc14700 1 -- 172.21.15.99:0/408212959 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcf581523e0 msgr2=0x7fcf5806c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:26.408 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.405+0000 7fcf5dc14700 1 --2- 172.21.15.99:0/408212959 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcf581523e0 0x7fcf5806c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:26.408 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.405+0000 7fcf5dc14700 1 -- 172.21.15.99:0/408212959 >> v1:172.21.15.99:6789/0 conn(0x7fcf58152010 legacy=0x7fcf5815b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:26.408 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.405+0000 7fcf5dc14700 1 -- 172.21.15.99:0/408212959 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf5815c340 con 0x7fcf58158e50 2023-10-08T00:21:26.408 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.405+0000 7fcf5dc14700 1 --2- 172.21.15.99:0/408212959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf58158e50 0x7fcf580cdf30 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fcf50009e60 tx=0x7fcf50009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac39cde3dd13edc2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:26.408 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.405+0000 7fcf4ffff700 1 -- 172.21.15.99:0/408212959 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5000e070 con 0x7fcf58158e50 2023-10-08T00:21:26.408 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.405+0000 7fcf4ffff700 1 -- 172.21.15.99:0/408212959 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf5000bb60 con 0x7fcf58158e50 2023-10-08T00:21:26.408 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.405+0000 7fcf4ffff700 1 -- 172.21.15.99:0/408212959 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf50012430 con 0x7fcf58158e50 2023-10-08T00:21:26.408 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.406+0000 7fcf5f677700 1 -- 172.21.15.99:0/408212959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf58158e50 msgr2=0x7fcf580cdf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:26.408 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.406+0000 7fcf5f677700 1 --2- 172.21.15.99:0/408212959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf58158e50 0x7fcf580cdf30 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fcf50009e60 tx=0x7fcf50009780 comp rx=0 tx=0).stop 2023-10-08T00:21:26.408 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.406+0000 7fcf5f677700 1 -- 172.21.15.99:0/408212959 shutdown_connections 2023-10-08T00:21:26.408 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.406+0000 7fcf5f677700 1 --2- 172.21.15.99:0/408212959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf58158e50 0x7fcf580cdf30 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:26.408 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.406+0000 7fcf5f677700 1 --2- 172.21.15.99:0/408212959 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcf581523e0 0x7fcf5806c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:26.409 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.406+0000 7fcf5f677700 1 -- 172.21.15.99:0/408212959 >> 172.21.15.99:0/408212959 conn(0x7fcf58069870 msgr2=0x7fcf58069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:26.409 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.406+0000 7fcf5f677700 1 -- 172.21.15.99:0/408212959 shutdown_connections 2023-10-08T00:21:26.409 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.406+0000 7fcf5f677700 1 -- 172.21.15.99:0/408212959 wait complete. 2023-10-08T00:21:26.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.407+0000 7fcf5f677700 1 Processor -- start 2023-10-08T00:21:26.411 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.407+0000 7fcf5f677700 1 -- start start 2023-10-08T00:21:26.412 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.408+0000 7fcf5f677700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcf58152010 0x7fcf580c8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:26.412 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.408+0000 7fcf5f677700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf581523e0 0x7fcf580c42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:26.412 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.408+0000 7fcf5f677700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf58158e50 0x7fcf580c6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:26.412 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.408+0000 7fcf5f677700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf580c8a80 con 0x7fcf581523e0 2023-10-08T00:21:26.412 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.408+0000 7fcf5f677700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5814d7f0 con 0x7fcf58158e50 2023-10-08T00:21:26.412 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.408+0000 7fcf5f677700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf580c4800 con 0x7fcf58152010 2023-10-08T00:21:26.412 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.408+0000 7fcf5cc12700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf581523e0 0x7fcf580c42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:26.412 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.408+0000 7fcf5cc12700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf581523e0 0x7fcf580c42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53670/0 (socket says 172.21.15.99:53670) 2023-10-08T00:21:26.412 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.408+0000 7fcf5cc12700 1 -- 172.21.15.99:0/1403976639 learned_addr learned my addr 172.21.15.99:0/1403976639 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:26.412 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.408+0000 7fcf5d413700 1 --2- 172.21.15.99:0/1403976639 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcf58152010 0x7fcf580c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:26.412 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.408+0000 7fcf5dc14700 1 --2- 172.21.15.99:0/1403976639 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf58158e50 0x7fcf580c6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:26.413 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.408+0000 7fcf5cc12700 1 -- 172.21.15.99:0/1403976639 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcf58152010 msgr2=0x7fcf580c8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:26.413 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.408+0000 7fcf5cc12700 1 --2- 172.21.15.99:0/1403976639 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcf58152010 0x7fcf580c8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:26.413 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.408+0000 7fcf5cc12700 1 -- 172.21.15.99:0/1403976639 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf58158e50 msgr2=0x7fcf580c6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:26.413 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.408+0000 7fcf5cc12700 1 --2- 172.21.15.99:0/1403976639 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf58158e50 0x7fcf580c6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:26.413 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.408+0000 7fcf5cc12700 1 -- 172.21.15.99:0/1403976639 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf50009a20 con 0x7fcf581523e0 2023-10-08T00:21:26.413 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.408+0000 7fcf5dc14700 1 --2- 172.21.15.99:0/1403976639 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf58158e50 0x7fcf580c6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:21:26.413 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.408+0000 7fcf5cc12700 1 --2- 172.21.15.99:0/1403976639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf581523e0 0x7fcf580c42c0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fcf4800d860 tx=0x7fcf48008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:26.413 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.408+0000 7fcf4d7fa700 1 -- 172.21.15.99:0/1403976639 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf480125f0 con 0x7fcf581523e0 2023-10-08T00:21:26.413 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.408+0000 7fcf4d7fa700 1 -- 172.21.15.99:0/1403976639 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf48012750 con 0x7fcf581523e0 2023-10-08T00:21:26.413 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.408+0000 7fcf4d7fa700 1 -- 172.21.15.99:0/1403976639 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf48012af0 con 0x7fcf581523e0 2023-10-08T00:21:26.413 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.408+0000 7fcf5f677700 1 -- 172.21.15.99:0/1403976639 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf580c4a60 con 0x7fcf581523e0 2023-10-08T00:21:26.413 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.409+0000 7fcf5f677700 1 -- 172.21.15.99:0/1403976639 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf580c5020 con 0x7fcf581523e0 2023-10-08T00:21:26.414 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.409+0000 7fcf5f677700 1 -- 172.21.15.99:0/1403976639 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf5814d5c0 con 0x7fcf581523e0 2023-10-08T00:21:26.414 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.410+0000 7fcf4d7fa700 1 -- 172.21.15.99:0/1403976639 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcf4801e460 con 0x7fcf581523e0 2023-10-08T00:21:26.414 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.410+0000 7fcf4d7fa700 1 --2- 172.21.15.99:0/1403976639 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcf3c066f30 0x7fcf3c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:26.414 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.410+0000 7fcf4d7fa700 1 -- 172.21.15.99:0/1403976639 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fcf480847a0 con 0x7fcf581523e0 2023-10-08T00:21:26.414 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.411+0000 7fcf5d413700 1 --2- 172.21.15.99:0/1403976639 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcf3c066f30 0x7fcf3c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:26.414 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.411+0000 7fcf5d413700 1 --2- 172.21.15.99:0/1403976639 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcf3c066f30 0x7fcf3c0693e0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fcf54007fd0 tx=0x7fcf54009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:26.415 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.415+0000 7fcf4d7fa700 1 -- 172.21.15.99:0/1403976639 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcf480557f0 con 0x7fcf581523e0 2023-10-08T00:21:26.668 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.668+0000 7fcf5f677700 1 -- 172.21.15.99:0/1403976639 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fcf5814d960 con 0x7fcf581523e0 2023-10-08T00:21:26.669 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.668+0000 7fcf4d7fa700 1 -- 172.21.15.99:0/1403976639 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fcf48055380 con 0x7fcf581523e0 2023-10-08T00:21:26.669 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:21:26.669 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:21:26.669 INFO:teuthology.orchestra.run.smithi099.stdout: "auth_dump": [ 2023-10-08T00:21:26.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:26.669 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.a", 2023-10-08T00:21:26.669 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlFlN9AhAAKdbeH0XznSOuyUnReD6RsA==", 2023-10-08T00:21:26.669 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:21:26.669 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:21:26.670 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2023-10-08T00:21:26.670 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2023-10-08T00:21:26.670 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:21:26.670 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:26.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:26.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:26.670 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.b", 2023-10-08T00:21:26.670 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlh2G0DRAA1mYGrUrELHsS9abUxCCG2g==", 2023-10-08T00:21:26.670 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:21:26.670 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:21:26.670 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2023-10-08T00:21:26.670 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2023-10-08T00:21:26.671 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:21:26.671 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:26.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:26.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:26.671 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.c", 2023-10-08T00:21:26.671 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFl/M1DCRAANrfHJGZO1s0KOyCdUzZWIA==", 2023-10-08T00:21:26.671 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:21:26.671 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:21:26.671 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2023-10-08T00:21:26.671 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2023-10-08T00:21:26.671 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:21:26.671 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:26.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:26.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:26.672 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.d", 2023-10-08T00:21:26.672 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlr5F2FBAAYYeG6EM0/N0FiEAwSbYRyA==", 2023-10-08T00:21:26.672 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:21:26.672 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:21:26.672 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2023-10-08T00:21:26.672 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2023-10-08T00:21:26.672 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:21:26.672 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:26.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:26.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:26.672 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.0", 2023-10-08T00:21:26.673 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB08yFla7rpKxAAWTOutLcrjDeR69JbPZMoLg==", 2023-10-08T00:21:26.673 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:21:26.673 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:21:26.673 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:21:26.673 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:21:26.673 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:26.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:26.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:26.674 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.1", 2023-10-08T00:21:26.674 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB08yFlw+y2MhAAAvLDIB/tb49IlaireTqTFg==", 2023-10-08T00:21:26.674 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:21:26.674 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:21:26.674 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:21:26.674 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:21:26.674 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:26.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:26.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:26.674 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.2", 2023-10-08T00:21:26.674 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB18yFlh4wvABAALXOtJZoEfGmt/pOBxReooQ==", 2023-10-08T00:21:26.675 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:21:26.675 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:21:26.675 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:21:26.675 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:21:26.675 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:26.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:26.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:26.675 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.3", 2023-10-08T00:21:26.675 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB18yFl8cVfCRAAoGXTACMT+f5Avll5mxG9OA==", 2023-10-08T00:21:26.675 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:21:26.675 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:21:26.675 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:21:26.676 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:21:26.676 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:26.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:26.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:26.676 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.4", 2023-10-08T00:21:26.676 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB48yFlft9WGhAA8JZN4MjSX9wbXiMKJnfCYg==", 2023-10-08T00:21:26.676 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:21:26.676 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:21:26.676 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:21:26.676 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:21:26.677 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:26.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:26.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:26.677 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.5", 2023-10-08T00:21:26.677 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB48yFlUf1BIRAA2FMBt1jAcm4ia/vYBDyojw==", 2023-10-08T00:21:26.678 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:21:26.678 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:21:26.678 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:21:26.678 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:21:26.678 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:26.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:26.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:26.678 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.6", 2023-10-08T00:21:26.678 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB48yFl0SI6KhAAI6G+Sqfm7OQmGf9nK59x9Q==", 2023-10-08T00:21:26.678 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:21:26.678 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:21:26.678 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:21:26.679 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:21:26.679 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:26.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:26.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:26.679 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.7", 2023-10-08T00:21:26.679 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQB48yFlo5xWMxAA4p9OSLK0JpWhEExsifX+3A==", 2023-10-08T00:21:26.679 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:21:26.679 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2023-10-08T00:21:26.679 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2023-10-08T00:21:26.679 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:21:26.679 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:26.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:26.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:26.680 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.0", 2023-10-08T00:21:26.680 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlmzhQGBAAReslU/Pvcsr0dwE6l1N3/Q==", 2023-10-08T00:21:26.680 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:21:26.680 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:21:26.680 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow r", 2023-10-08T00:21:26.680 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-08T00:21:26.680 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:26.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:26.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:26.680 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.1", 2023-10-08T00:21:26.680 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFl2NKsIBAAUg6rXynl1J7qB6BkUizmpA==", 2023-10-08T00:21:26.681 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:21:26.681 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:21:26.681 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2023-10-08T00:21:26.681 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2023-10-08T00:21:26.681 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2023-10-08T00:21:26.681 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:26.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:26.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:26.681 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.2", 2023-10-08T00:21:26.681 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlMp6rHRAArVgJfA3mY4n78hOMeVlw1Q==", 2023-10-08T00:21:26.681 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:21:26.681 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:21:26.681 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2023-10-08T00:21:26.682 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2023-10-08T00:21:26.682 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2023-10-08T00:21:26.682 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:26.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:26.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:26.682 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.3", 2023-10-08T00:21:26.682 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBx8yFlH0gNJhAAEgGVGiabVIWknpWUohOnHA==", 2023-10-08T00:21:26.682 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:21:26.682 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2023-10-08T00:21:26.682 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2023-10-08T00:21:26.682 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2023-10-08T00:21:26.682 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2023-10-08T00:21:26.683 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:26.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:26.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:26.683 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.admin", 2023-10-08T00:21:26.683 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBw8yFlidUfIxAAIvpvkHbyCx5JFvHo/g5A9g==", 2023-10-08T00:21:26.683 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:21:26.683 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2023-10-08T00:21:26.683 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2023-10-08T00:21:26.683 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2023-10-08T00:21:26.683 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:21:26.683 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:26.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:26.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:26.684 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mgr.x", 2023-10-08T00:21:26.684 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBw8yFl2kMLOxAAsEyuLBLsjQCUS0ERDOi75A==", 2023-10-08T00:21:26.684 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:21:26.684 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2023-10-08T00:21:26.684 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile mgr", 2023-10-08T00:21:26.684 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:21:26.684 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:26.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:26.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:26.684 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mgr.y", 2023-10-08T00:21:26.684 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQBw8yFl6ht5NxAAFzXnLDjklCePD6hw/pSfFg==", 2023-10-08T00:21:26.684 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2023-10-08T00:21:26.685 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2023-10-08T00:21:26.685 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile mgr", 2023-10-08T00:21:26.685 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2023-10-08T00:21:26.685 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:26.685 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:26.685 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:26.685 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:21:26.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.670+0000 7fcf5f677700 1 -- 172.21.15.99:0/1403976639 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcf3c066f30 msgr2=0x7fcf3c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:26.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.670+0000 7fcf5f677700 1 --2- 172.21.15.99:0/1403976639 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcf3c066f30 0x7fcf3c0693e0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fcf54007fd0 tx=0x7fcf54009040 comp rx=0 tx=0).stop 2023-10-08T00:21:26.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.670+0000 7fcf5f677700 1 -- 172.21.15.99:0/1403976639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf581523e0 msgr2=0x7fcf580c42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:26.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.670+0000 7fcf5f677700 1 --2- 172.21.15.99:0/1403976639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf581523e0 0x7fcf580c42c0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fcf4800d860 tx=0x7fcf48008040 comp rx=0 tx=0).stop 2023-10-08T00:21:26.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.670+0000 7fcf5f677700 1 -- 172.21.15.99:0/1403976639 shutdown_connections 2023-10-08T00:21:26.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.670+0000 7fcf5f677700 1 --2- 172.21.15.99:0/1403976639 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcf3c066f30 0x7fcf3c0693e0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:26.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.670+0000 7fcf5f677700 1 --2- 172.21.15.99:0/1403976639 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf58158e50 0x7fcf580c6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:26.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.670+0000 7fcf5f677700 1 --2- 172.21.15.99:0/1403976639 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcf58152010 0x7fcf580c8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:26.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.670+0000 7fcf5f677700 1 --2- 172.21.15.99:0/1403976639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf581523e0 0x7fcf580c42c0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:26.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.670+0000 7fcf5f677700 1 -- 172.21.15.99:0/1403976639 >> 172.21.15.99:0/1403976639 conn(0x7fcf58069870 msgr2=0x7fcf580ca700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:26.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.670+0000 7fcf5f677700 1 -- 172.21.15.99:0/1403976639 shutdown_connections 2023-10-08T00:21:26.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.670+0000 7fcf5f677700 1 -- 172.21.15.99:0/1403976639 wait complete. 2023-10-08T00:21:26.688 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:21:26.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.801+0000 7f9a531c4700 1 Processor -- start 2023-10-08T00:21:26.802 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.801+0000 7f9a531c4700 1 -- start start 2023-10-08T00:21:26.805 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.802+0000 7f9a531c4700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9a4c0cbc00 0x7f9a4c0c9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:26.805 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.802+0000 7f9a531c4700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a4c15de60 con 0x7f9a4c0b1220 2023-10-08T00:21:26.805 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.802+0000 7f9a531c4700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a4c15e140 con 0x7f9a4c0b0e50 2023-10-08T00:21:26.805 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.802+0000 7f9a531c4700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a4c15e420 con 0x7f9a4c0cbc00 2023-10-08T00:21:26.805 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.802+0000 7f9a51761700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9a4c0cbc00 0x7f9a4c0c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:26.805 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.802+0000 7f9a43fff700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f9a4c0b1220 0x7f9a4c0cb4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:53486/0 (socket says 172.21.15.99:53486) 2023-10-08T00:21:26.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.802+0000 7f9a43fff700 1 -- 172.21.15.99:0/3279597698 learned_addr learned my addr 172.21.15.99:0/3279597698 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:21:26.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.802+0000 7f9a51761700 1 --2- 172.21.15.99:0/3279597698 >> v2:172.21.15.99:3300/0 conn(0x7f9a4c0cbc00 0x7f9a4c0c9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53686/0 (socket says 172.21.15.99:53686) 2023-10-08T00:21:26.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.803+0000 7f9a51761700 1 -- 172.21.15.99:0/3279597698 >> v1:172.21.15.191:6789/0 conn(0x7f9a4c0b0e50 legacy=0x7f9a4c06d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:26.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.803+0000 7f9a51761700 1 -- 172.21.15.99:0/3279597698 >> v1:172.21.15.99:6789/0 conn(0x7f9a4c0b1220 legacy=0x7f9a4c0cb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:26.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.803+0000 7f9a51761700 1 -- 172.21.15.99:0/3279597698 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a4c15e700 con 0x7f9a4c0cbc00 2023-10-08T00:21:26.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.803+0000 7f9a437fe700 1 -- 172.21.15.99:0/3279597698 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 408950314 0 0) 0x7f9a4c15de60 con 0x7f9a4c0b1220 2023-10-08T00:21:26.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.803+0000 7f9a51761700 1 --2- 172.21.15.99:0/3279597698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a4c0cbc00 0x7f9a4c0c9180 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f9a440098e0 tx=0x7f9a440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9980647712c3953 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:26.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.803+0000 7f9a437fe700 1 -- 172.21.15.99:0/3279597698 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a44009e80 con 0x7f9a4c0cbc00 2023-10-08T00:21:26.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.803+0000 7f9a437fe700 1 -- 172.21.15.99:0/3279597698 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a4400b820 con 0x7f9a4c0cbc00 2023-10-08T00:21:26.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.803+0000 7f9a437fe700 1 -- 172.21.15.99:0/3279597698 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4400bba0 con 0x7f9a4c0cbc00 2023-10-08T00:21:26.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.804+0000 7f9a531c4700 1 -- 172.21.15.99:0/3279597698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a4c0cbc00 msgr2=0x7f9a4c0c9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:26.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.804+0000 7f9a531c4700 1 --2- 172.21.15.99:0/3279597698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a4c0cbc00 0x7f9a4c0c9180 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f9a440098e0 tx=0x7f9a440091a0 comp rx=0 tx=0).stop 2023-10-08T00:21:26.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.804+0000 7f9a531c4700 1 -- 172.21.15.99:0/3279597698 shutdown_connections 2023-10-08T00:21:26.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.804+0000 7f9a531c4700 1 --2- 172.21.15.99:0/3279597698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a4c0cbc00 0x7f9a4c0c9180 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:26.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.804+0000 7f9a531c4700 1 -- 172.21.15.99:0/3279597698 >> 172.21.15.99:0/3279597698 conn(0x7f9a4c069870 msgr2=0x7f9a4c069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:26.808 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.804+0000 7f9a531c4700 1 -- 172.21.15.99:0/3279597698 shutdown_connections 2023-10-08T00:21:26.808 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.804+0000 7f9a531c4700 1 -- 172.21.15.99:0/3279597698 wait complete. 2023-10-08T00:21:26.808 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.805+0000 7f9a531c4700 1 Processor -- start 2023-10-08T00:21:26.812 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.805+0000 7f9a531c4700 1 -- start start 2023-10-08T00:21:26.812 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.806+0000 7f9a531c4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a4c0b0e50 0x7f9a4c1586e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:26.812 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.806+0000 7f9a531c4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a4c0b1220 0x7f9a4c06bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:26.812 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.806+0000 7f9a531c4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a4c0cbc00 0x7f9a4c06c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:26.812 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.806+0000 7f9a531c4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4c0c7380 con 0x7f9a4c0b0e50 2023-10-08T00:21:26.813 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.806+0000 7f9a531c4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4c0b4660 con 0x7f9a4c0cbc00 2023-10-08T00:21:26.813 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.806+0000 7f9a531c4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4c16a290 con 0x7f9a4c0b1220 2023-10-08T00:21:26.813 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.806+0000 7f9a50f60700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a4c0b0e50 0x7f9a4c1586e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:26.813 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.806+0000 7f9a50f60700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a4c0b0e50 0x7f9a4c1586e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53698/0 (socket says 172.21.15.99:53698) 2023-10-08T00:21:26.813 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.806+0000 7f9a50f60700 1 -- 172.21.15.99:0/2148693357 learned_addr learned my addr 172.21.15.99:0/2148693357 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:26.813 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.806+0000 7f9a43fff700 1 --2- 172.21.15.99:0/2148693357 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a4c0b1220 0x7f9a4c06bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:26.813 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.806+0000 7f9a51761700 1 --2- 172.21.15.99:0/2148693357 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a4c0cbc00 0x7f9a4c06c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:26.813 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.806+0000 7f9a50f60700 1 -- 172.21.15.99:0/2148693357 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a4c0b1220 msgr2=0x7f9a4c06bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:26.813 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.806+0000 7f9a50f60700 1 --2- 172.21.15.99:0/2148693357 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a4c0b1220 0x7f9a4c06bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:26.813 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.806+0000 7f9a50f60700 1 -- 172.21.15.99:0/2148693357 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a4c0cbc00 msgr2=0x7f9a4c06c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:26.814 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.806+0000 7f9a50f60700 1 --2- 172.21.15.99:0/2148693357 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a4c0cbc00 0x7f9a4c06c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:26.814 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.806+0000 7f9a50f60700 1 -- 172.21.15.99:0/2148693357 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a44009420 con 0x7f9a4c0b0e50 2023-10-08T00:21:26.814 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.806+0000 7f9a51761700 1 --2- 172.21.15.99:0/2148693357 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a4c0cbc00 0x7f9a4c06c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:21:26.814 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.806+0000 7f9a43fff700 1 --2- 172.21.15.99:0/2148693357 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a4c0b1220 0x7f9a4c06bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:21:26.814 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.806+0000 7f9a50f60700 1 --2- 172.21.15.99:0/2148693357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a4c0b0e50 0x7f9a4c1586e0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f9a4800c950 tx=0x7f9a48002760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:26.814 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.806+0000 7f9a40ff9700 1 -- 172.21.15.99:0/2148693357 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4800a7e0 con 0x7f9a4c0b0e50 2023-10-08T00:21:26.814 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.806+0000 7f9a40ff9700 1 -- 172.21.15.99:0/2148693357 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a4800a940 con 0x7f9a4c0b0e50 2023-10-08T00:21:26.814 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.806+0000 7f9a531c4700 1 -- 172.21.15.99:0/2148693357 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a4c0c6800 con 0x7f9a4c0b0e50 2023-10-08T00:21:26.814 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.806+0000 7f9a40ff9700 1 -- 172.21.15.99:0/2148693357 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4800ad00 con 0x7f9a4c0b0e50 2023-10-08T00:21:26.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.806+0000 7f9a531c4700 1 -- 172.21.15.99:0/2148693357 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a4c0c6dc0 con 0x7f9a4c0b0e50 2023-10-08T00:21:26.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.807+0000 7f9a531c4700 1 -- 172.21.15.99:0/2148693357 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a4c04c3c0 con 0x7f9a4c0b0e50 2023-10-08T00:21:26.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.808+0000 7f9a40ff9700 1 -- 172.21.15.99:0/2148693357 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9a48012630 con 0x7f9a4c0b0e50 2023-10-08T00:21:26.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.808+0000 7f9a40ff9700 1 --2- 172.21.15.99:0/2148693357 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9a38066f30 0x7f9a380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:26.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.808+0000 7f9a40ff9700 1 -- 172.21.15.99:0/2148693357 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f9a48084740 con 0x7f9a4c0b0e50 2023-10-08T00:21:26.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.808+0000 7f9a43fff700 1 --2- 172.21.15.99:0/2148693357 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9a38066f30 0x7f9a380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:26.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.809+0000 7f9a43fff700 1 --2- 172.21.15.99:0/2148693357 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9a38066f30 0x7f9a380693e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c0cac00 tx=0x7f9a3c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:26.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.812+0000 7f9a40ff9700 1 -- 172.21.15.99:0/2148693357 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9a48051db0 con 0x7f9a4c0b0e50 2023-10-08T00:21:26.996 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.996+0000 7f9a531c4700 1 -- 172.21.15.99:0/2148693357 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9a4c0cef10 con 0x7f9a4c0b0e50 2023-10-08T00:21:26.997 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.997+0000 7f9a40ff9700 1 -- 172.21.15.99:0/2148693357 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v180) v1 ==== 74+0+12145 (secure 0 0 0) 0x7f9a4800ae60 con 0x7f9a4c0b0e50 2023-10-08T00:21:26.997 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:21:26.998 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":180,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:21:26.271682+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:21:27.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.998+0000 7f9a531c4700 1 -- 172.21.15.99:0/2148693357 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9a38066f30 msgr2=0x7f9a380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:27.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.998+0000 7f9a531c4700 1 --2- 172.21.15.99:0/2148693357 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9a38066f30 0x7f9a380693e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c0cac00 tx=0x7f9a3c009040 comp rx=0 tx=0).stop 2023-10-08T00:21:27.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.998+0000 7f9a531c4700 1 -- 172.21.15.99:0/2148693357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a4c0b0e50 msgr2=0x7f9a4c1586e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:27.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.999+0000 7f9a531c4700 1 --2- 172.21.15.99:0/2148693357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a4c0b0e50 0x7f9a4c1586e0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f9a4800c950 tx=0x7f9a48002760 comp rx=0 tx=0).stop 2023-10-08T00:21:27.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.999+0000 7f9a531c4700 1 -- 172.21.15.99:0/2148693357 shutdown_connections 2023-10-08T00:21:27.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.999+0000 7f9a531c4700 1 --2- 172.21.15.99:0/2148693357 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9a38066f30 0x7f9a380693e0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:27.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.999+0000 7f9a531c4700 1 --2- 172.21.15.99:0/2148693357 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a4c0cbc00 0x7f9a4c06c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:27.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.999+0000 7f9a531c4700 1 --2- 172.21.15.99:0/2148693357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a4c0b0e50 0x7f9a4c1586e0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:27.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.999+0000 7f9a531c4700 1 --2- 172.21.15.99:0/2148693357 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a4c0b1220 0x7f9a4c06bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:27.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.999+0000 7f9a531c4700 1 -- 172.21.15.99:0/2148693357 >> 172.21.15.99:0/2148693357 conn(0x7f9a4c069870 msgr2=0x7f9a4c159630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:27.001 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.999+0000 7f9a531c4700 1 -- 172.21.15.99:0/2148693357 shutdown_connections 2023-10-08T00:21:27.001 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:26.999+0000 7f9a531c4700 1 -- 172.21.15.99:0/2148693357 wait complete. 2023-10-08T00:21:27.011 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:10:42.063344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-08T00:21:27.012 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T00:21:27.128 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.127+0000 7f48f1f12700 1 Processor -- start 2023-10-08T00:21:27.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.127+0000 7f48f1f12700 1 -- start start 2023-10-08T00:21:27.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.128+0000 7f48f1f12700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f48ec159ec0 0x7f48ec0af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:27.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.128+0000 7f48f1f12700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f48ec06acb0 0x7f48ec0afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:27.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.128+0000 7f48f1f12700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48ec0b0790 con 0x7f48ec15a290 2023-10-08T00:21:27.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.128+0000 7f48f1f12700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48ec0ce1f0 con 0x7f48ec159ec0 2023-10-08T00:21:27.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.128+0000 7f48f1f12700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48ec0b0b20 con 0x7f48ec06acb0 2023-10-08T00:21:27.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.128+0000 7f48eb7fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f48ec159ec0 0x7f48ec0af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:27.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.128+0000 7f48eaffd700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f48ec15a290 0x7f48ec06a5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:53496/0 (socket says 172.21.15.99:53496) 2023-10-08T00:21:27.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.128+0000 7f48eb7fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f48ec159ec0 0x7f48ec0af940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53702/0 (socket says 172.21.15.99:53702) 2023-10-08T00:21:27.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.128+0000 7f48eaffd700 1 -- 172.21.15.99:0/1261233280 learned_addr learned my addr 172.21.15.99:0/1261233280 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:21:27.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.128+0000 7f48ebfff700 1 --2- 172.21.15.99:0/1261233280 >> v2:172.21.15.191:3300/0 conn(0x7f48ec06acb0 0x7f48ec0afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:27.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.128+0000 7f48eb7fe700 1 -- 172.21.15.99:0/1261233280 >> v2:172.21.15.191:3300/0 conn(0x7f48ec06acb0 msgr2=0x7f48ec0afe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:27.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.128+0000 7f48eb7fe700 1 --2- 172.21.15.99:0/1261233280 >> v2:172.21.15.191:3300/0 conn(0x7f48ec06acb0 0x7f48ec0afe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:27.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.128+0000 7f48eb7fe700 1 -- 172.21.15.99:0/1261233280 >> v1:172.21.15.99:6789/0 conn(0x7f48ec15a290 legacy=0x7f48ec06a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:27.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.128+0000 7f48eb7fe700 1 -- 172.21.15.99:0/1261233280 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48ec0b04b0 con 0x7f48ec159ec0 2023-10-08T00:21:27.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.128+0000 7f48ea7fc700 1 -- 172.21.15.99:0/1261233280 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1693690717 0 0) 0x7f48ec0b0790 con 0x7f48ec15a290 2023-10-08T00:21:27.164 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.128+0000 7f48eb7fe700 1 --2- 172.21.15.99:0/1261233280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48ec159ec0 0x7f48ec0af940 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f48ec0711f0 tx=0x7f48dc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee263920e64c2cf5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:27.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.129+0000 7f48ea7fc700 1 -- 172.21.15.99:0/1261233280 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48dc00e070 con 0x7f48ec159ec0 2023-10-08T00:21:27.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.129+0000 7f48ea7fc700 1 -- 172.21.15.99:0/1261233280 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48dc00bc30 con 0x7f48ec159ec0 2023-10-08T00:21:27.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.129+0000 7f48ea7fc700 1 -- 172.21.15.99:0/1261233280 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48dc012430 con 0x7f48ec159ec0 2023-10-08T00:21:27.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.129+0000 7f48f1f12700 1 -- 172.21.15.99:0/1261233280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48ec159ec0 msgr2=0x7f48ec0af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:27.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.129+0000 7f48f1f12700 1 --2- 172.21.15.99:0/1261233280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48ec159ec0 0x7f48ec0af940 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f48ec0711f0 tx=0x7f48dc009780 comp rx=0 tx=0).stop 2023-10-08T00:21:27.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.129+0000 7f48f1f12700 1 -- 172.21.15.99:0/1261233280 shutdown_connections 2023-10-08T00:21:27.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.129+0000 7f48f1f12700 1 --2- 172.21.15.99:0/1261233280 >> v2:172.21.15.191:3300/0 conn(0x7f48ec06acb0 0x7f48ec0afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:27.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.129+0000 7f48f1f12700 1 --2- 172.21.15.99:0/1261233280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48ec159ec0 0x7f48ec0af940 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:27.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.129+0000 7f48f1f12700 1 -- 172.21.15.99:0/1261233280 >> 172.21.15.99:0/1261233280 conn(0x7f48ec0d1880 msgr2=0x7f48ec0cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:27.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.129+0000 7f48f1f12700 1 -- 172.21.15.99:0/1261233280 shutdown_connections 2023-10-08T00:21:27.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.129+0000 7f48f1f12700 1 -- 172.21.15.99:0/1261233280 wait complete. 2023-10-08T00:21:27.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.130+0000 7f48f1f12700 1 Processor -- start 2023-10-08T00:21:27.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.130+0000 7f48f1f12700 1 -- start start 2023-10-08T00:21:27.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.131+0000 7f48f1f12700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48ec06acb0 0x7f48ec0c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:27.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.131+0000 7f48f1f12700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48ec159ec0 0x7f48ec0c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:27.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.131+0000 7f48f1f12700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f48ec15a290 0x7f48ec0c9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:27.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.131+0000 7f48f1f12700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec16e290 con 0x7f48ec159ec0 2023-10-08T00:21:27.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.131+0000 7f48f1f12700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec159620 con 0x7f48ec06acb0 2023-10-08T00:21:27.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.131+0000 7f48f1f12700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec16d680 con 0x7f48ec15a290 2023-10-08T00:21:27.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.131+0000 7f48eaffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48ec159ec0 0x7f48ec0c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:27.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.131+0000 7f48eaffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48ec159ec0 0x7f48ec0c8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53712/0 (socket says 172.21.15.99:53712) 2023-10-08T00:21:27.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.131+0000 7f48eaffd700 1 -- 172.21.15.99:0/3357416146 learned_addr learned my addr 172.21.15.99:0/3357416146 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:27.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.131+0000 7f48eb7fe700 1 --2- 172.21.15.99:0/3357416146 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48ec06acb0 0x7f48ec0c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:27.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.131+0000 7f48ebfff700 1 --2- 172.21.15.99:0/3357416146 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f48ec15a290 0x7f48ec0c9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:27.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.132+0000 7f48eaffd700 1 -- 172.21.15.99:0/3357416146 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f48ec15a290 msgr2=0x7f48ec0c9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:27.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.132+0000 7f48eaffd700 1 --2- 172.21.15.99:0/3357416146 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f48ec15a290 0x7f48ec0c9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:27.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.132+0000 7f48eaffd700 1 -- 172.21.15.99:0/3357416146 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48ec06acb0 msgr2=0x7f48ec0c7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:27.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.132+0000 7f48eaffd700 1 --2- 172.21.15.99:0/3357416146 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48ec06acb0 0x7f48ec0c7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:27.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.132+0000 7f48eaffd700 1 -- 172.21.15.99:0/3357416146 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48dc009a20 con 0x7f48ec159ec0 2023-10-08T00:21:27.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.132+0000 7f48ebfff700 1 --2- 172.21.15.99:0/3357416146 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f48ec15a290 0x7f48ec0c9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:21:27.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.132+0000 7f48eaffd700 1 --2- 172.21.15.99:0/3357416146 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48ec159ec0 0x7f48ec0c8b00 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f48e000c5b0 tx=0x7f48e0001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:27.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.132+0000 7f48d37fe700 1 -- 172.21.15.99:0/3357416146 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e0005b30 con 0x7f48ec159ec0 2023-10-08T00:21:27.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.132+0000 7f48d37fe700 1 -- 172.21.15.99:0/3357416146 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48e0005c90 con 0x7f48ec159ec0 2023-10-08T00:21:27.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.132+0000 7f48f1f12700 1 -- 172.21.15.99:0/3357416146 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48ec16d960 con 0x7f48ec159ec0 2023-10-08T00:21:27.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.132+0000 7f48d37fe700 1 -- 172.21.15.99:0/3357416146 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e0007300 con 0x7f48ec159ec0 2023-10-08T00:21:27.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.132+0000 7f48f1f12700 1 -- 172.21.15.99:0/3357416146 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48ec16df20 con 0x7f48ec159ec0 2023-10-08T00:21:27.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.133+0000 7f48f1f12700 1 -- 172.21.15.99:0/3357416146 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48ec04c3c0 con 0x7f48ec159ec0 2023-10-08T00:21:27.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.133+0000 7f48d37fe700 1 -- 172.21.15.99:0/3357416146 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f48e001b3b0 con 0x7f48ec159ec0 2023-10-08T00:21:27.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.134+0000 7f48d37fe700 1 --2- 172.21.15.99:0/3357416146 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f48d8066f30 0x7f48d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:27.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.134+0000 7f48d37fe700 1 -- 172.21.15.99:0/3357416146 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f48e00836c0 con 0x7f48ec159ec0 2023-10-08T00:21:27.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.134+0000 7f48eb7fe700 1 --2- 172.21.15.99:0/3357416146 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f48d8066f30 0x7f48d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:27.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.134+0000 7f48eb7fe700 1 --2- 172.21.15.99:0/3357416146 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f48d8066f30 0x7f48d80693e0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f48ec0711f0 tx=0x7f48dc019000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:27.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.138+0000 7f48d37fe700 1 -- 172.21.15.99:0/3357416146 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f48e0050c80 con 0x7f48ec159ec0 2023-10-08T00:21:27.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.319+0000 7f48f1f12700 1 -- 172.21.15.99:0/3357416146 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f48ec0ceab0 con 0x7f48ec159ec0 2023-10-08T00:21:27.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.319+0000 7f48d37fe700 1 -- 172.21.15.99:0/3357416146 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v180) v1 ==== 93+0+10 (secure 0 0 0) 0x7f48e00170a0 con 0x7f48ec159ec0 2023-10-08T00:21:27.320 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2023-10-08T00:21:27.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.322+0000 7f48f1f12700 1 -- 172.21.15.99:0/3357416146 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f48d8066f30 msgr2=0x7f48d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:27.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.322+0000 7f48f1f12700 1 --2- 172.21.15.99:0/3357416146 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f48d8066f30 0x7f48d80693e0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f48ec0711f0 tx=0x7f48dc019000 comp rx=0 tx=0).stop 2023-10-08T00:21:27.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.322+0000 7f48f1f12700 1 -- 172.21.15.99:0/3357416146 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48ec159ec0 msgr2=0x7f48ec0c8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:27.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.322+0000 7f48f1f12700 1 --2- 172.21.15.99:0/3357416146 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48ec159ec0 0x7f48ec0c8b00 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f48e000c5b0 tx=0x7f48e0001070 comp rx=0 tx=0).stop 2023-10-08T00:21:27.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.322+0000 7f48f1f12700 1 -- 172.21.15.99:0/3357416146 shutdown_connections 2023-10-08T00:21:27.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.322+0000 7f48f1f12700 1 --2- 172.21.15.99:0/3357416146 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f48d8066f30 0x7f48d80693e0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:27.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.322+0000 7f48f1f12700 1 --2- 172.21.15.99:0/3357416146 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48ec06acb0 0x7f48ec0c7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:27.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.322+0000 7f48f1f12700 1 --2- 172.21.15.99:0/3357416146 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f48ec15a290 0x7f48ec0c9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:27.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.322+0000 7f48f1f12700 1 --2- 172.21.15.99:0/3357416146 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48ec159ec0 0x7f48ec0c8b00 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:27.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.322+0000 7f48f1f12700 1 -- 172.21.15.99:0/3357416146 >> 172.21.15.99:0/3357416146 conn(0x7f48ec0d1880 msgr2=0x7f48ec0b1780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:27.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.322+0000 7f48f1f12700 1 -- 172.21.15.99:0/3357416146 shutdown_connections 2023-10-08T00:21:27.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.322+0000 7f48f1f12700 1 -- 172.21.15.99:0/3357416146 wait complete. 2023-10-08T00:21:27.335 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T00:21:27.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.447+0000 7f98ceccc700 1 Processor -- start 2023-10-08T00:21:27.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.447+0000 7f98ceccc700 1 -- start start 2023-10-08T00:21:27.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.448+0000 7f98ceccc700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f98c8157ba0 0x7f98c806a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:27.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.448+0000 7f98ceccc700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98c815c360 con 0x7f98c81577d0 2023-10-08T00:21:27.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.448+0000 7f98ceccc700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98c815b790 con 0x7f98c815b0d0 2023-10-08T00:21:27.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.448+0000 7f98ceccc700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98c815ba30 con 0x7f98c8157ba0 2023-10-08T00:21:27.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.448+0000 7f98bffff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f98c8157ba0 0x7f98c806a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:27.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.448+0000 7f98bffff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f98c8157ba0 0x7f98c806a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53726/0 (socket says 172.21.15.99:53726) 2023-10-08T00:21:27.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.448+0000 7f98bffff700 1 -- 172.21.15.99:0/3702053914 learned_addr learned my addr 172.21.15.99:0/3702053914 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:27.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.448+0000 7f98cca68700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f98c81577d0 0x7f98c80cd770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:53506/0 (socket says 172.21.15.99:53506) 2023-10-08T00:21:27.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.449+0000 7f98bffff700 1 -- 172.21.15.99:0/3702053914 >> v1:172.21.15.191:6789/0 conn(0x7f98c815b0d0 legacy=0x7f98c815c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:27.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.449+0000 7f98bffff700 1 -- 172.21.15.99:0/3702053914 >> v1:172.21.15.99:6789/0 conn(0x7f98c81577d0 legacy=0x7f98c80cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:27.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.449+0000 7f98bf7fe700 1 -- 172.21.15.99:0/3702053914 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 636647765 0 0) 0x7f98c815c360 con 0x7f98c81577d0 2023-10-08T00:21:27.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.449+0000 7f98bffff700 1 -- 172.21.15.99:0/3702053914 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c815bd10 con 0x7f98c8157ba0 2023-10-08T00:21:27.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.449+0000 7f98bffff700 1 --2- 172.21.15.99:0/3702053914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98c8157ba0 0x7f98c806a520 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f98b80099b0 tx=0x7f98b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f008311b93a87d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:27.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.449+0000 7f98bf7fe700 1 -- 172.21.15.99:0/3702053914 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b800b820 con 0x7f98c8157ba0 2023-10-08T00:21:27.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.449+0000 7f98bf7fe700 1 -- 172.21.15.99:0/3702053914 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98b800b980 con 0x7f98c8157ba0 2023-10-08T00:21:27.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.449+0000 7f98bf7fe700 1 -- 172.21.15.99:0/3702053914 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b800bd20 con 0x7f98c8157ba0 2023-10-08T00:21:27.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.449+0000 7f98ceccc700 1 -- 172.21.15.99:0/3702053914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98c8157ba0 msgr2=0x7f98c806a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:27.452 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.449+0000 7f98ceccc700 1 --2- 172.21.15.99:0/3702053914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98c8157ba0 0x7f98c806a520 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f98b80099b0 tx=0x7f98b8009250 comp rx=0 tx=0).stop 2023-10-08T00:21:27.452 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.449+0000 7f98ceccc700 1 -- 172.21.15.99:0/3702053914 shutdown_connections 2023-10-08T00:21:27.452 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.449+0000 7f98ceccc700 1 --2- 172.21.15.99:0/3702053914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98c8157ba0 0x7f98c806a520 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:27.452 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.449+0000 7f98ceccc700 1 -- 172.21.15.99:0/3702053914 >> 172.21.15.99:0/3702053914 conn(0x7f98c8068ce0 msgr2=0x7f98c80690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:27.452 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.450+0000 7f98ceccc700 1 -- 172.21.15.99:0/3702053914 shutdown_connections 2023-10-08T00:21:27.452 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.450+0000 7f98ceccc700 1 -- 172.21.15.99:0/3702053914 wait complete. 2023-10-08T00:21:27.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.451+0000 7f98ceccc700 1 Processor -- start 2023-10-08T00:21:27.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.451+0000 7f98ceccc700 1 -- start start 2023-10-08T00:21:27.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.451+0000 7f98ceccc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98c81577d0 0x7f98c8150d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:27.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.451+0000 7f98ceccc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98c8157ba0 0x7f98c80a9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:27.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.451+0000 7f98ceccc700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98c815b0d0 0x7f98c80ae4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:27.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.451+0000 7f98ceccc700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c80ab140 con 0x7f98c8157ba0 2023-10-08T00:21:27.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.451+0000 7f98ceccc700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c81512b0 con 0x7f98c815b0d0 2023-10-08T00:21:27.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.451+0000 7f98ceccc700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98c80aa530 con 0x7f98c81577d0 2023-10-08T00:21:27.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.452+0000 7f98bffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98c8157ba0 0x7f98c80a9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:27.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.452+0000 7f98bffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98c8157ba0 0x7f98c80a9b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53730/0 (socket says 172.21.15.99:53730) 2023-10-08T00:21:27.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.452+0000 7f98bffff700 1 -- 172.21.15.99:0/915925955 learned_addr learned my addr 172.21.15.99:0/915925955 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:27.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.452+0000 7f98cd269700 1 --2- 172.21.15.99:0/915925955 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98c815b0d0 0x7f98c80ae4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:27.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.452+0000 7f98cca68700 1 --2- 172.21.15.99:0/915925955 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98c81577d0 0x7f98c8150d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:27.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.452+0000 7f98bffff700 1 -- 172.21.15.99:0/915925955 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98c81577d0 msgr2=0x7f98c8150d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:27.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.452+0000 7f98bffff700 1 --2- 172.21.15.99:0/915925955 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98c81577d0 0x7f98c8150d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:27.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.452+0000 7f98bffff700 1 -- 172.21.15.99:0/915925955 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98c815b0d0 msgr2=0x7f98c80ae4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:27.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.452+0000 7f98bffff700 1 --2- 172.21.15.99:0/915925955 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98c815b0d0 0x7f98c80ae4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:27.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.452+0000 7f98bffff700 1 -- 172.21.15.99:0/915925955 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b80094f0 con 0x7f98c8157ba0 2023-10-08T00:21:27.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.452+0000 7f98cca68700 1 --2- 172.21.15.99:0/915925955 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98c81577d0 0x7f98c8150d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:21:27.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.452+0000 7f98bffff700 1 --2- 172.21.15.99:0/915925955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98c8157ba0 0x7f98c80a9b50 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f98b80094c0 tx=0x7f98b8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:27.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.452+0000 7f98bcff9700 1 -- 172.21.15.99:0/915925955 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b800bcc0 con 0x7f98c8157ba0 2023-10-08T00:21:27.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.452+0000 7f98bcff9700 1 -- 172.21.15.99:0/915925955 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98b801b680 con 0x7f98c8157ba0 2023-10-08T00:21:27.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.452+0000 7f98ceccc700 1 -- 172.21.15.99:0/915925955 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98c80aa810 con 0x7f98c8157ba0 2023-10-08T00:21:27.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.452+0000 7f98bcff9700 1 -- 172.21.15.99:0/915925955 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b801ba40 con 0x7f98c8157ba0 2023-10-08T00:21:27.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.452+0000 7f98ceccc700 1 -- 172.21.15.99:0/915925955 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98c80aadd0 con 0x7f98c8157ba0 2023-10-08T00:21:27.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.453+0000 7f98ceccc700 1 -- 172.21.15.99:0/915925955 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98c80ab2b0 con 0x7f98c8157ba0 2023-10-08T00:21:27.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.453+0000 7f98bcff9700 1 -- 172.21.15.99:0/915925955 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f98b801c450 con 0x7f98c8157ba0 2023-10-08T00:21:27.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.454+0000 7f98bcff9700 1 --2- 172.21.15.99:0/915925955 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f98b4066f30 0x7f98b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:27.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.454+0000 7f98bcff9700 1 -- 172.21.15.99:0/915925955 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f98b8085fc0 con 0x7f98c8157ba0 2023-10-08T00:21:27.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.454+0000 7f98cca68700 1 --2- 172.21.15.99:0/915925955 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f98b4066f30 0x7f98b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:27.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.454+0000 7f98cca68700 1 --2- 172.21.15.99:0/915925955 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f98b4066f30 0x7f98b40693e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f98c40018d0 tx=0x7f98c4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:27.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.458+0000 7f98bcff9700 1 -- 172.21.15.99:0/915925955 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f98b80b75a0 con 0x7f98c8157ba0 2023-10-08T00:21:27.641 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.640+0000 7f98ceccc700 1 -- 172.21.15.99:0/915925955 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f98c804c3c0 con 0x7f98c8157ba0 2023-10-08T00:21:27.641 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.640+0000 7f98bcff9700 1 -- 172.21.15.99:0/915925955 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v180) v1 ==== 84+0+65 (secure 0 0 0) 0x7f98b802a090 con 0x7f98c8157ba0 2023-10-08T00:21:27.641 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:21:27.641 INFO:teuthology.orchestra.run.smithi099.stdout:[ 2023-10-08T00:21:27.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:27.641 INFO:teuthology.orchestra.run.smithi099.stdout: "poolnum": 1, 2023-10-08T00:21:27.641 INFO:teuthology.orchestra.run.smithi099.stdout: "poolname": ".mgr" 2023-10-08T00:21:27.642 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:27.642 INFO:teuthology.orchestra.run.smithi099.stdout:] 2023-10-08T00:21:27.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.642+0000 7f98ceccc700 1 -- 172.21.15.99:0/915925955 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f98b4066f30 msgr2=0x7f98b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:27.643 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.642+0000 7f98ceccc700 1 --2- 172.21.15.99:0/915925955 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f98b4066f30 0x7f98b40693e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f98c40018d0 tx=0x7f98c4009040 comp rx=0 tx=0).stop 2023-10-08T00:21:27.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.642+0000 7f98ceccc700 1 -- 172.21.15.99:0/915925955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98c8157ba0 msgr2=0x7f98c80a9b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:27.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.642+0000 7f98ceccc700 1 --2- 172.21.15.99:0/915925955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98c8157ba0 0x7f98c80a9b50 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f98b80094c0 tx=0x7f98b8016000 comp rx=0 tx=0).stop 2023-10-08T00:21:27.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.643+0000 7f98ceccc700 1 -- 172.21.15.99:0/915925955 shutdown_connections 2023-10-08T00:21:27.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.643+0000 7f98ceccc700 1 --2- 172.21.15.99:0/915925955 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f98b4066f30 0x7f98b40693e0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:27.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.643+0000 7f98ceccc700 1 --2- 172.21.15.99:0/915925955 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98c815b0d0 0x7f98c80ae4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:27.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.643+0000 7f98ceccc700 1 --2- 172.21.15.99:0/915925955 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98c81577d0 0x7f98c8150d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:27.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.643+0000 7f98ceccc700 1 --2- 172.21.15.99:0/915925955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98c8157ba0 0x7f98c80a9b50 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:27.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.643+0000 7f98ceccc700 1 -- 172.21.15.99:0/915925955 >> 172.21.15.99:0/915925955 conn(0x7f98c8068ce0 msgr2=0x7f98c80cf4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:27.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.643+0000 7f98ceccc700 1 -- 172.21.15.99:0/915925955 shutdown_connections 2023-10-08T00:21:27.644 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.643+0000 7f98ceccc700 1 -- 172.21.15.99:0/915925955 wait complete. 2023-10-08T00:21:27.655 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T00:21:27.655 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T00:21:27.774 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.773+0000 7f027c916700 1 Processor -- start 2023-10-08T00:21:28.750 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.773+0000 7f027c916700 1 -- start start 2023-10-08T00:21:28.761 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.774+0000 7f027c916700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f02780aadd0 0x7f0278159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:28.761 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.774+0000 7f027c916700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02780c7f50 con 0x7f0278155f00 2023-10-08T00:21:28.761 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.774+0000 7f027c916700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f027815a690 con 0x7f027815a230 2023-10-08T00:21:28.761 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.774+0000 7f027c916700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02780c8db0 con 0x7f02780aadd0 2023-10-08T00:21:28.762 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.774+0000 7f026e59c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f02780aadd0 0x7f0278159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:28.762 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.774+0000 7f0276d9d700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f0278155f00 0x7f0278069330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:53522/0 (socket says 172.21.15.99:53522) 2023-10-08T00:21:28.762 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.774+0000 7f0276d9d700 1 -- 172.21.15.99:0/1948004957 learned_addr learned my addr 172.21.15.99:0/1948004957 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:21:28.762 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.774+0000 7f026e59c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f02780aadd0 0x7f0278159cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53744/0 (socket says 172.21.15.99:53744) 2023-10-08T00:21:28.762 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.774+0000 7f026e59c700 1 -- 172.21.15.99:0/1948004957 >> v1:172.21.15.191:6789/0 conn(0x7f027815a230 legacy=0x7f02780aef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:28.762 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.774+0000 7f026e59c700 1 -- 172.21.15.99:0/1948004957 >> v1:172.21.15.99:6789/0 conn(0x7f0278155f00 legacy=0x7f0278069330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:28.763 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.774+0000 7f026e59c700 1 -- 172.21.15.99:0/1948004957 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f027815ab10 con 0x7f02780aadd0 2023-10-08T00:21:28.763 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.774+0000 7f027659c700 1 -- 172.21.15.99:0/1948004957 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4050767989 0 0) 0x7f0268001110 con 0x7f0278155f00 2023-10-08T00:21:28.763 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.774+0000 7f026e59c700 1 --2- 172.21.15.99:0/1948004957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02780aadd0 0x7f0278159cf0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f026000bb90 tx=0x7f026000b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73b438f4ab3f531a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:28.763 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.774+0000 7f027659c700 1 -- 172.21.15.99:0/1948004957 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026000da50 con 0x7f02780aadd0 2023-10-08T00:21:28.763 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.774+0000 7f027659c700 1 -- 172.21.15.99:0/1948004957 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f026000dbb0 con 0x7f02780aadd0 2023-10-08T00:21:28.763 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.774+0000 7f027659c700 1 -- 172.21.15.99:0/1948004957 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0260017410 con 0x7f02780aadd0 2023-10-08T00:21:28.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.775+0000 7f027c916700 1 -- 172.21.15.99:0/1948004957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02780aadd0 msgr2=0x7f0278159cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:28.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.775+0000 7f027c916700 1 --2- 172.21.15.99:0/1948004957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02780aadd0 0x7f0278159cf0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f026000bb90 tx=0x7f026000b430 comp rx=0 tx=0).stop 2023-10-08T00:21:28.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.775+0000 7f027c916700 1 -- 172.21.15.99:0/1948004957 shutdown_connections 2023-10-08T00:21:28.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.775+0000 7f027c916700 1 --2- 172.21.15.99:0/1948004957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02780aadd0 0x7f0278159cf0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:28.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.775+0000 7f027c916700 1 -- 172.21.15.99:0/1948004957 >> 172.21.15.99:0/1948004957 conn(0x7f0278068130 msgr2=0x7f02780cd060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:28.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.775+0000 7f027c916700 1 -- 172.21.15.99:0/1948004957 shutdown_connections 2023-10-08T00:21:28.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.775+0000 7f027c916700 1 -- 172.21.15.99:0/1948004957 wait complete. 2023-10-08T00:21:28.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.776+0000 7f027c916700 1 Processor -- start 2023-10-08T00:21:28.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.776+0000 7f027c916700 1 -- start start 2023-10-08T00:21:28.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.777+0000 7f027c916700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02780aadd0 0x7f027809a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:28.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.777+0000 7f027c916700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0278155f00 0x7f0278097a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:28.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.777+0000 7f027c916700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f027815a230 0x7f0278097f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:28.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.777+0000 7f027c916700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02780c0250 con 0x7f0278155f00 2023-10-08T00:21:28.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.777+0000 7f027c916700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0278156ca0 con 0x7f027815a230 2023-10-08T00:21:28.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.777+0000 7f027c916700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0278098610 con 0x7f02780aadd0 2023-10-08T00:21:28.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.777+0000 7f026e59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0278155f00 0x7f0278097a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:28.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.777+0000 7f026e59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0278155f00 0x7f0278097a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53756/0 (socket says 172.21.15.99:53756) 2023-10-08T00:21:28.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.777+0000 7f026e59c700 1 -- 172.21.15.99:0/1798360829 learned_addr learned my addr 172.21.15.99:0/1798360829 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:28.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.777+0000 7f0276d9d700 1 --2- 172.21.15.99:0/1798360829 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02780aadd0 0x7f027809a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:28.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.777+0000 7f027759e700 1 --2- 172.21.15.99:0/1798360829 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f027815a230 0x7f0278097f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:28.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.777+0000 7f026e59c700 1 -- 172.21.15.99:0/1798360829 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02780aadd0 msgr2=0x7f027809a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:28.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.777+0000 7f026e59c700 1 --2- 172.21.15.99:0/1798360829 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02780aadd0 0x7f027809a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:28.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.777+0000 7f026e59c700 1 -- 172.21.15.99:0/1798360829 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f027815a230 msgr2=0x7f0278097f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:28.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.777+0000 7f026e59c700 1 --2- 172.21.15.99:0/1798360829 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f027815a230 0x7f0278097f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:28.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.777+0000 7f026e59c700 1 -- 172.21.15.99:0/1798360829 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f026000b6d0 con 0x7f0278155f00 2023-10-08T00:21:28.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.777+0000 7f0276d9d700 1 --2- 172.21.15.99:0/1798360829 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02780aadd0 0x7f027809a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:21:28.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.778+0000 7f026e59c700 1 --2- 172.21.15.99:0/1798360829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0278155f00 0x7f0278097a00 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f0260003080 tx=0x7f0260019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:28.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.778+0000 7f026f7fe700 1 -- 172.21.15.99:0/1798360829 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0260009040 con 0x7f0278155f00 2023-10-08T00:21:28.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.778+0000 7f026f7fe700 1 -- 172.21.15.99:0/1798360829 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f026001f5e0 con 0x7f0278155f00 2023-10-08T00:21:28.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.778+0000 7f026f7fe700 1 -- 172.21.15.99:0/1798360829 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026001f9e0 con 0x7f0278155f00 2023-10-08T00:21:28.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.778+0000 7f027c916700 1 -- 172.21.15.99:0/1798360829 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0278156ef0 con 0x7f0278155f00 2023-10-08T00:21:28.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.778+0000 7f027c916700 1 -- 172.21.15.99:0/1798360829 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02781574b0 con 0x7f0278155f00 2023-10-08T00:21:28.909 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.779+0000 7f027c916700 1 -- 172.21.15.99:0/1798360829 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f027806cd50 con 0x7f0278155f00 2023-10-08T00:21:28.909 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.779+0000 7f026f7fe700 1 -- 172.21.15.99:0/1798360829 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0260015070 con 0x7f0278155f00 2023-10-08T00:21:28.909 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.779+0000 7f026f7fe700 1 --2- 172.21.15.99:0/1798360829 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0258066f30 0x7f02580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:28.909 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.780+0000 7f026f7fe700 1 -- 172.21.15.99:0/1798360829 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f0260085110 con 0x7f0278155f00 2023-10-08T00:21:28.909 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.780+0000 7f0276d9d700 1 --2- 172.21.15.99:0/1798360829 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0258066f30 0x7f02580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:28.909 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.780+0000 7f0276d9d700 1 --2- 172.21.15.99:0/1798360829 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0258066f30 0x7f02580693e0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f02680018a0 tx=0x7f0268009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:28.909 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.784+0000 7f026f7fe700 1 -- 172.21.15.99:0/1798360829 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0260052800 con 0x7f0278155f00 2023-10-08T00:21:28.909 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:27.966+0000 7f027c916700 1 -- 172.21.15.99:0/1798360829 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f027815a690 con 0x7f0278155f00 2023-10-08T00:21:28.909 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:28.282+0000 7f026f7fe700 1 -- 172.21.15.99:0/1798360829 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v181) v1 ==== 138+0+0 (secure 0 0 0) 0x7f027815a690 con 0x7f0278155f00 2023-10-08T00:21:28.909 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:28.286+0000 7f027c916700 1 -- 172.21.15.99:0/1798360829 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0258066f30 msgr2=0x7f02580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:28.909 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:28.286+0000 7f027c916700 1 --2- 172.21.15.99:0/1798360829 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0258066f30 0x7f02580693e0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f02680018a0 tx=0x7f0268009040 comp rx=0 tx=0).stop 2023-10-08T00:21:28.910 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:28.286+0000 7f027c916700 1 -- 172.21.15.99:0/1798360829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0278155f00 msgr2=0x7f0278097a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:28.910 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:28.286+0000 7f027c916700 1 --2- 172.21.15.99:0/1798360829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0278155f00 0x7f0278097a00 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f0260003080 tx=0x7f0260019000 comp rx=0 tx=0).stop 2023-10-08T00:21:28.910 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:28.286+0000 7f027c916700 1 -- 172.21.15.99:0/1798360829 shutdown_connections 2023-10-08T00:21:28.910 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:28.286+0000 7f027c916700 1 --2- 172.21.15.99:0/1798360829 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0258066f30 0x7f02580693e0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:28.910 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:28.286+0000 7f027c916700 1 --2- 172.21.15.99:0/1798360829 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f027815a230 0x7f0278097f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:28.910 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:28.286+0000 7f027c916700 1 --2- 172.21.15.99:0/1798360829 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02780aadd0 0x7f027809a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:28.910 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:28.286+0000 7f027c916700 1 --2- 172.21.15.99:0/1798360829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0278155f00 0x7f0278097a00 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:28.910 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:28.286+0000 7f027c916700 1 -- 172.21.15.99:0/1798360829 >> 172.21.15.99:0/1798360829 conn(0x7f0278068130 msgr2=0x7f02780c9ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:28.910 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:28.286+0000 7f027c916700 1 -- 172.21.15.99:0/1798360829 shutdown_connections 2023-10-08T00:21:28.910 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:28.286+0000 7f027c916700 1 -- 172.21.15.99:0/1798360829 wait complete. 2023-10-08T00:21:28.910 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' created 2023-10-08T00:21:28.911 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T00:21:29.035 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.034+0000 7f4f6999d700 1 Processor -- start 2023-10-08T00:21:29.035 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.034+0000 7f4f6999d700 1 -- start start 2023-10-08T00:21:29.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.035+0000 7f4f6999d700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4f64159e60 0x7f4f6415e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:29.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.035+0000 7f4f6999d700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f6415ea90 con 0x7f4f6415a230 2023-10-08T00:21:29.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.035+0000 7f4f6999d700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f640b3100 con 0x7f4f640697c0 2023-10-08T00:21:29.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.035+0000 7f4f6999d700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f640d1cf0 con 0x7f4f64159e60 2023-10-08T00:21:29.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.035+0000 7f4f62ffd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4f64159e60 0x7f4f6415e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:29.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.035+0000 7f4f627fc700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f4f6415a230 0x7f4f640690b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:53534/0 (socket says 172.21.15.99:53534) 2023-10-08T00:21:29.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.035+0000 7f4f627fc700 1 -- 172.21.15.99:0/818119756 learned_addr learned my addr 172.21.15.99:0/818119756 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:21:29.037 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.035+0000 7f4f62ffd700 1 -- 172.21.15.99:0/818119756 >> v1:172.21.15.191:6789/0 conn(0x7f4f640697c0 legacy=0x7f4f640af730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:29.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.035+0000 7f4f62ffd700 1 -- 172.21.15.99:0/818119756 >> v1:172.21.15.99:6789/0 conn(0x7f4f6415a230 legacy=0x7f4f640690b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:29.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.035+0000 7f4f62ffd700 1 -- 172.21.15.99:0/818119756 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f640d10e0 con 0x7f4f64159e60 2023-10-08T00:21:29.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.035+0000 7f4f61ffb700 1 -- 172.21.15.99:0/818119756 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1430067147 0 0) 0x7f4f6415ea90 con 0x7f4f6415a230 2023-10-08T00:21:29.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.035+0000 7f4f62ffd700 1 --2- 172.21.15.99:0/818119756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f64159e60 0x7f4f6415e550 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f4f540098e0 tx=0x7f4f540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26e2ab2abb1ce4f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:29.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.035+0000 7f4f61ffb700 1 -- 172.21.15.99:0/818119756 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f54009e80 con 0x7f4f64159e60 2023-10-08T00:21:29.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.035+0000 7f4f61ffb700 1 -- 172.21.15.99:0/818119756 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f5400b820 con 0x7f4f64159e60 2023-10-08T00:21:29.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.035+0000 7f4f61ffb700 1 -- 172.21.15.99:0/818119756 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5400bba0 con 0x7f4f64159e60 2023-10-08T00:21:29.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.036+0000 7f4f6999d700 1 -- 172.21.15.99:0/818119756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f64159e60 msgr2=0x7f4f6415e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:29.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.036+0000 7f4f6999d700 1 --2- 172.21.15.99:0/818119756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f64159e60 0x7f4f6415e550 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f4f540098e0 tx=0x7f4f540091a0 comp rx=0 tx=0).stop 2023-10-08T00:21:29.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.036+0000 7f4f6999d700 1 -- 172.21.15.99:0/818119756 shutdown_connections 2023-10-08T00:21:29.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.036+0000 7f4f6999d700 1 --2- 172.21.15.99:0/818119756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f64159e60 0x7f4f6415e550 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:29.038 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.036+0000 7f4f6999d700 1 -- 172.21.15.99:0/818119756 >> 172.21.15.99:0/818119756 conn(0x7f4f64070700 msgr2=0x7f4f64070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:29.039 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.036+0000 7f4f6999d700 1 -- 172.21.15.99:0/818119756 shutdown_connections 2023-10-08T00:21:29.039 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.036+0000 7f4f6999d700 1 -- 172.21.15.99:0/818119756 wait complete. 2023-10-08T00:21:29.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.037+0000 7f4f6999d700 1 Processor -- start 2023-10-08T00:21:29.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.037+0000 7f4f6999d700 1 -- start start 2023-10-08T00:21:29.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.038+0000 7f4f6999d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f640697c0 0x7f4f6415e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:29.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.038+0000 7f4f6999d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f64159e60 0x7f4f640c9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:29.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.038+0000 7f4f6999d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f6415a230 0x7f4f6415bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:29.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.038+0000 7f4f6999d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6415eae0 con 0x7f4f640697c0 2023-10-08T00:21:29.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.038+0000 7f4f6999d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6416a9d0 con 0x7f4f6415a230 2023-10-08T00:21:29.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.038+0000 7f4f6999d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6416b230 con 0x7f4f64159e60 2023-10-08T00:21:29.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.038+0000 7f4f62ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f640697c0 0x7f4f6415e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:29.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.038+0000 7f4f62ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f640697c0 0x7f4f6415e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53774/0 (socket says 172.21.15.99:53774) 2023-10-08T00:21:29.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.038+0000 7f4f62ffd700 1 -- 172.21.15.99:0/677082973 learned_addr learned my addr 172.21.15.99:0/677082973 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:29.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.038+0000 7f4f627fc700 1 --2- 172.21.15.99:0/677082973 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f64159e60 0x7f4f640c9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:29.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.038+0000 7f4f637fe700 1 --2- 172.21.15.99:0/677082973 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f6415a230 0x7f4f6415bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:29.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.038+0000 7f4f62ffd700 1 -- 172.21.15.99:0/677082973 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f64159e60 msgr2=0x7f4f640c9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:29.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.038+0000 7f4f62ffd700 1 --2- 172.21.15.99:0/677082973 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f64159e60 0x7f4f640c9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:29.042 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.038+0000 7f4f62ffd700 1 -- 172.21.15.99:0/677082973 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f6415a230 msgr2=0x7f4f6415bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:29.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.038+0000 7f4f62ffd700 1 --2- 172.21.15.99:0/677082973 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f6415a230 0x7f4f6415bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:29.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.038+0000 7f4f62ffd700 1 -- 172.21.15.99:0/677082973 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f54009420 con 0x7f4f640697c0 2023-10-08T00:21:29.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.038+0000 7f4f627fc700 1 --2- 172.21.15.99:0/677082973 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f64159e60 0x7f4f640c9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:21:29.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.039+0000 7f4f62ffd700 1 --2- 172.21.15.99:0/677082973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f640697c0 0x7f4f6415e5a0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f4f540030c0 tx=0x7f4f54016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:29.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.039+0000 7f4f4affd700 1 -- 172.21.15.99:0/677082973 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5400ba10 con 0x7f4f640697c0 2023-10-08T00:21:29.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.039+0000 7f4f4affd700 1 -- 172.21.15.99:0/677082973 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f54020070 con 0x7f4f640697c0 2023-10-08T00:21:29.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.039+0000 7f4f6999d700 1 -- 172.21.15.99:0/677082973 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f640ca9f0 con 0x7f4f640697c0 2023-10-08T00:21:29.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.039+0000 7f4f4affd700 1 -- 172.21.15.99:0/677082973 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5401b6a0 con 0x7f4f640697c0 2023-10-08T00:21:29.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.039+0000 7f4f6999d700 1 -- 172.21.15.99:0/677082973 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f6416aec0 con 0x7f4f640697c0 2023-10-08T00:21:29.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.040+0000 7f4f6999d700 1 -- 172.21.15.99:0/677082973 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f6404c3c0 con 0x7f4f640697c0 2023-10-08T00:21:29.043 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.040+0000 7f4f4affd700 1 -- 172.21.15.99:0/677082973 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4f54022030 con 0x7f4f640697c0 2023-10-08T00:21:29.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.040+0000 7f4f4affd700 1 --2- 172.21.15.99:0/677082973 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4f50066f30 0x7f4f500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:29.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.041+0000 7f4f4affd700 1 -- 172.21.15.99:0/677082973 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f4f54086430 con 0x7f4f640697c0 2023-10-08T00:21:29.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.041+0000 7f4f627fc700 1 --2- 172.21.15.99:0/677082973 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4f50066f30 0x7f4f500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:29.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.041+0000 7f4f627fc700 1 --2- 172.21.15.99:0/677082973 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4f50066f30 0x7f4f500693e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4f6415dae0 tx=0x7f4f58009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:29.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.045+0000 7f4f4affd700 1 -- 172.21.15.99:0/677082973 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4f54053880 con 0x7f4f640697c0 2023-10-08T00:21:29.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.226+0000 7f4f6999d700 1 -- 172.21.15.99:0/677082973 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f4f6404c3c0 con 0x7f4f640697c0 2023-10-08T00:21:29.285 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.284+0000 7f4f4affd700 1 -- 172.21.15.99:0/677082973 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v182) v1 ==== 170+0+0 (secure 0 0 0) 0x7f4f54056ea0 con 0x7f4f640697c0 2023-10-08T00:21:29.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.289+0000 7f4f6999d700 1 -- 172.21.15.99:0/677082973 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4f50066f30 msgr2=0x7f4f500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:29.290 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.289+0000 7f4f6999d700 1 --2- 172.21.15.99:0/677082973 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4f50066f30 0x7f4f500693e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4f6415dae0 tx=0x7f4f58009040 comp rx=0 tx=0).stop 2023-10-08T00:21:29.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.289+0000 7f4f6999d700 1 -- 172.21.15.99:0/677082973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f640697c0 msgr2=0x7f4f6415e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:29.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.289+0000 7f4f6999d700 1 --2- 172.21.15.99:0/677082973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f640697c0 0x7f4f6415e5a0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f4f540030c0 tx=0x7f4f54016000 comp rx=0 tx=0).stop 2023-10-08T00:21:29.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.290+0000 7f4f6999d700 1 -- 172.21.15.99:0/677082973 shutdown_connections 2023-10-08T00:21:29.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.290+0000 7f4f6999d700 1 --2- 172.21.15.99:0/677082973 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4f50066f30 0x7f4f500693e0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:29.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.290+0000 7f4f6999d700 1 --2- 172.21.15.99:0/677082973 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f6415a230 0x7f4f6415bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:29.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.290+0000 7f4f6999d700 1 --2- 172.21.15.99:0/677082973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f640697c0 0x7f4f6415e5a0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:29.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.290+0000 7f4f6999d700 1 --2- 172.21.15.99:0/677082973 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f64159e60 0x7f4f640c9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:29.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.290+0000 7f4f6999d700 1 -- 172.21.15.99:0/677082973 >> 172.21.15.99:0/677082973 conn(0x7f4f64070700 msgr2=0x7f4f640ccb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:29.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.290+0000 7f4f6999d700 1 -- 172.21.15.99:0/677082973 shutdown_connections 2023-10-08T00:21:29.291 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.290+0000 7f4f6999d700 1 -- 172.21.15.99:0/677082973 wait complete. 2023-10-08T00:21:29.292 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' created 2023-10-08T00:21:29.306 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T00:21:29.423 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.422+0000 7fe5edeb4700 1 Processor -- start 2023-10-08T00:21:29.423 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.422+0000 7fe5edeb4700 1 -- start start 2023-10-08T00:21:29.426 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.423+0000 7fe5edeb4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5e8159e60 0x7fe5e815e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:29.426 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.423+0000 7fe5edeb4700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe5e815a230 0x7fe5e80690b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:29.426 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.423+0000 7fe5edeb4700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5e815ea90 con 0x7fe5e80695f0 2023-10-08T00:21:29.426 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.423+0000 7fe5edeb4700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5e80d1cf0 con 0x7fe5e815a230 2023-10-08T00:21:29.426 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.423+0000 7fe5edeb4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e8069a90 con 0x7fe5e8159e60 2023-10-08T00:21:29.426 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.423+0000 7fe5e6ffd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe5e815a230 0x7fe5e80690b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:29.427 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.423+0000 7fe5e6ffd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe5e815a230 0x7fe5e80690b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53788/0 (socket says 172.21.15.99:53788) 2023-10-08T00:21:29.427 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.423+0000 7fe5e6ffd700 1 -- 172.21.15.99:0/2775700004 learned_addr learned my addr 172.21.15.99:0/2775700004 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:29.427 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.424+0000 7fe5e77fe700 1 --2- 172.21.15.99:0/2775700004 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5e8159e60 0x7fe5e815e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:29.427 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.424+0000 7fe5e6ffd700 1 -- 172.21.15.99:0/2775700004 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5e8159e60 msgr2=0x7fe5e815e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:29.427 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.424+0000 7fe5e6ffd700 1 --2- 172.21.15.99:0/2775700004 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5e8159e60 0x7fe5e815e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:29.427 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.424+0000 7fe5e6ffd700 1 -- 172.21.15.99:0/2775700004 >> v1:172.21.15.191:6789/0 conn(0x7fe5e80695f0 legacy=0x7fe5e80af730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:29.427 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.424+0000 7fe5e6ffd700 1 -- 172.21.15.99:0/2775700004 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e80b3100 con 0x7fe5e815a230 2023-10-08T00:21:29.428 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.424+0000 7fe5e6ffd700 1 --2- 172.21.15.99:0/2775700004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5e815a230 0x7fe5e80690b0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc0099b0 tx=0x7fe5dc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99269b6306aefc41 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:29.428 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.424+0000 7fe5e67fc700 1 -- 172.21.15.99:0/2775700004 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc00b820 con 0x7fe5e815a230 2023-10-08T00:21:29.428 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.424+0000 7fe5e67fc700 1 -- 172.21.15.99:0/2775700004 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5dc00b980 con 0x7fe5e815a230 2023-10-08T00:21:29.428 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.424+0000 7fe5e67fc700 1 -- 172.21.15.99:0/2775700004 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc00bd20 con 0x7fe5e815a230 2023-10-08T00:21:29.428 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.424+0000 7fe5edeb4700 1 -- 172.21.15.99:0/2775700004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5e815a230 msgr2=0x7fe5e80690b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:29.428 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.424+0000 7fe5edeb4700 1 --2- 172.21.15.99:0/2775700004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5e815a230 0x7fe5e80690b0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc0099b0 tx=0x7fe5dc009250 comp rx=0 tx=0).stop 2023-10-08T00:21:29.428 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.425+0000 7fe5edeb4700 1 -- 172.21.15.99:0/2775700004 shutdown_connections 2023-10-08T00:21:29.429 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.425+0000 7fe5edeb4700 1 --2- 172.21.15.99:0/2775700004 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5e8159e60 0x7fe5e815e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:29.429 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.425+0000 7fe5edeb4700 1 --2- 172.21.15.99:0/2775700004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5e815a230 0x7fe5e80690b0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:29.429 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.425+0000 7fe5edeb4700 1 -- 172.21.15.99:0/2775700004 >> 172.21.15.99:0/2775700004 conn(0x7fe5e8070700 msgr2=0x7fe5e8070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:29.429 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.425+0000 7fe5edeb4700 1 -- 172.21.15.99:0/2775700004 shutdown_connections 2023-10-08T00:21:29.429 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.425+0000 7fe5edeb4700 1 -- 172.21.15.99:0/2775700004 wait complete. 2023-10-08T00:21:29.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.426+0000 7fe5edeb4700 1 Processor -- start 2023-10-08T00:21:29.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.426+0000 7fe5edeb4700 1 -- start start 2023-10-08T00:21:29.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.427+0000 7fe5edeb4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5e80695f0 0x7fe5e81586d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:29.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.427+0000 7fe5edeb4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5e8159e60 0x7fe5e815e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:29.432 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.427+0000 7fe5edeb4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5e815a230 0x7fe5e815b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:29.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.427+0000 7fe5edeb4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e8158c10 con 0x7fe5e80695f0 2023-10-08T00:21:29.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.427+0000 7fe5edeb4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e80c9ee0 con 0x7fe5e8159e60 2023-10-08T00:21:29.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.427+0000 7fe5edeb4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e80ca1c0 con 0x7fe5e815a230 2023-10-08T00:21:29.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.427+0000 7fe5e77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5e80695f0 0x7fe5e81586d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:29.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.427+0000 7fe5e77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5e80695f0 0x7fe5e81586d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53790/0 (socket says 172.21.15.99:53790) 2023-10-08T00:21:29.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.427+0000 7fe5e77fe700 1 -- 172.21.15.99:0/3328466008 learned_addr learned my addr 172.21.15.99:0/3328466008 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:29.433 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.427+0000 7fe5e7fff700 1 --2- 172.21.15.99:0/3328466008 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5e815a230 0x7fe5e815b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:29.434 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.427+0000 7fe5e6ffd700 1 --2- 172.21.15.99:0/3328466008 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5e8159e60 0x7fe5e815e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:29.434 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.427+0000 7fe5e77fe700 1 -- 172.21.15.99:0/3328466008 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5e815a230 msgr2=0x7fe5e815b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:29.434 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.427+0000 7fe5e77fe700 1 --2- 172.21.15.99:0/3328466008 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5e815a230 0x7fe5e815b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:29.434 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.427+0000 7fe5e77fe700 1 -- 172.21.15.99:0/3328466008 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5e8159e60 msgr2=0x7fe5e815e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:29.434 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.427+0000 7fe5e77fe700 1 --2- 172.21.15.99:0/3328466008 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5e8159e60 0x7fe5e815e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:29.434 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.427+0000 7fe5e77fe700 1 -- 172.21.15.99:0/3328466008 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5dc0094f0 con 0x7fe5e80695f0 2023-10-08T00:21:29.434 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.427+0000 7fe5e6ffd700 1 --2- 172.21.15.99:0/3328466008 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5e8159e60 0x7fe5e815e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:21:29.434 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.427+0000 7fe5e7fff700 1 --2- 172.21.15.99:0/3328466008 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5e815a230 0x7fe5e815b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:21:29.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.427+0000 7fe5e77fe700 1 --2- 172.21.15.99:0/3328466008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5e80695f0 0x7fe5e81586d0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fe5d800cfc0 tx=0x7fe5d80078b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:29.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.427+0000 7fe5cf7fe700 1 -- 172.21.15.99:0/3328466008 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d8016430 con 0x7fe5e80695f0 2023-10-08T00:21:29.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.427+0000 7fe5cf7fe700 1 -- 172.21.15.99:0/3328466008 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d8016590 con 0x7fe5e80695f0 2023-10-08T00:21:29.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.427+0000 7fe5edeb4700 1 -- 172.21.15.99:0/3328466008 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5e80c7da0 con 0x7fe5e80695f0 2023-10-08T00:21:29.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.427+0000 7fe5cf7fe700 1 -- 172.21.15.99:0/3328466008 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d8016930 con 0x7fe5e80695f0 2023-10-08T00:21:29.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.427+0000 7fe5edeb4700 1 -- 172.21.15.99:0/3328466008 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5e80c8360 con 0x7fe5e80695f0 2023-10-08T00:21:29.435 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.428+0000 7fe5edeb4700 1 -- 172.21.15.99:0/3328466008 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5e804c3c0 con 0x7fe5e80695f0 2023-10-08T00:21:29.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.429+0000 7fe5cf7fe700 1 -- 172.21.15.99:0/3328466008 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe5d8003020 con 0x7fe5e80695f0 2023-10-08T00:21:29.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.429+0000 7fe5cf7fe700 1 --2- 172.21.15.99:0/3328466008 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe5d4066f30 0x7fe5d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:29.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.429+0000 7fe5cf7fe700 1 -- 172.21.15.99:0/3328466008 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5605+0+0 (secure 0 0 0) 0x7fe5d8012070 con 0x7fe5e80695f0 2023-10-08T00:21:29.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.429+0000 7fe5e6ffd700 1 --2- 172.21.15.99:0/3328466008 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe5d4066f30 0x7fe5d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:29.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.430+0000 7fe5e6ffd700 1 --2- 172.21.15.99:0/3328466008 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe5d4066f30 0x7fe5d40693e0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc0099b0 tx=0x7fe5dc016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:29.436 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.434+0000 7fe5cf7fe700 1 -- 172.21.15.99:0/3328466008 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe5d8052720 con 0x7fe5e80695f0 2023-10-08T00:21:29.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:29.657+0000 7fe5edeb4700 1 -- 172.21.15.99:0/3328466008 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe5e80cb590 con 0x7fe5e80695f0 2023-10-08T00:21:30.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.303+0000 7fe5cf7fe700 1 -- 172.21.15.99:0/3328466008 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 22 and data pool 23 v81) v1 ==== 174+0+0 (secure 0 0 0) 0x7fe5d8055d40 con 0x7fe5e80695f0 2023-10-08T00:21:30.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.306+0000 7fe5edeb4700 1 -- 172.21.15.99:0/3328466008 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe5d4066f30 msgr2=0x7fe5d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:30.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.306+0000 7fe5edeb4700 1 --2- 172.21.15.99:0/3328466008 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe5d4066f30 0x7fe5d40693e0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc0099b0 tx=0x7fe5dc016000 comp rx=0 tx=0).stop 2023-10-08T00:21:30.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.306+0000 7fe5edeb4700 1 -- 172.21.15.99:0/3328466008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5e80695f0 msgr2=0x7fe5e81586d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:30.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.306+0000 7fe5edeb4700 1 --2- 172.21.15.99:0/3328466008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5e80695f0 0x7fe5e81586d0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fe5d800cfc0 tx=0x7fe5d80078b0 comp rx=0 tx=0).stop 2023-10-08T00:21:30.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.306+0000 7fe5edeb4700 1 -- 172.21.15.99:0/3328466008 shutdown_connections 2023-10-08T00:21:30.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.306+0000 7fe5edeb4700 1 --2- 172.21.15.99:0/3328466008 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fe5d4066f30 0x7fe5d40693e0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:30.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.306+0000 7fe5edeb4700 1 --2- 172.21.15.99:0/3328466008 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5e815a230 0x7fe5e815b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:30.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.306+0000 7fe5edeb4700 1 --2- 172.21.15.99:0/3328466008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5e80695f0 0x7fe5e81586d0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:30.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.306+0000 7fe5edeb4700 1 --2- 172.21.15.99:0/3328466008 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5e8159e60 0x7fe5e815e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:30.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.306+0000 7fe5edeb4700 1 -- 172.21.15.99:0/3328466008 >> 172.21.15.99:0/3328466008 conn(0x7fe5e8070700 msgr2=0x7fe5e81594b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:30.308 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.306+0000 7fe5edeb4700 1 -- 172.21.15.99:0/3328466008 shutdown_connections 2023-10-08T00:21:30.310 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.309+0000 7fe5edeb4700 1 -- 172.21.15.99:0/3328466008 wait complete. 2023-10-08T00:21:30.311 INFO:teuthology.orchestra.run.smithi099.stderr:new fs with metadata pool 22 and data pool 23 2023-10-08T00:21:30.323 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:21:30.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.436+0000 7fdadb211700 1 Processor -- start 2023-10-08T00:21:30.437 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.436+0000 7fdadb211700 1 -- start start 2023-10-08T00:21:30.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.437+0000 7fdadb211700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fdad4159cf0 0x7fdad415e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:30.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.437+0000 7fdadb211700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdad415a0c0 0x7fdad40ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:30.440 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.437+0000 7fdadb211700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdad40cca20 con 0x7fdad40ce1a0 2023-10-08T00:21:30.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.437+0000 7fdadb211700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdad415ea40 con 0x7fdad415a0c0 2023-10-08T00:21:30.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.437+0000 7fdadb211700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdad4067660 con 0x7fdad4159cf0 2023-10-08T00:21:30.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.437+0000 7fdacbfff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdad415a0c0 0x7fdad40ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:30.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.437+0000 7fdacbfff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdad415a0c0 0x7fdad40ceab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53792/0 (socket says 172.21.15.99:53792) 2023-10-08T00:21:30.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.437+0000 7fdad97ae700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fdad40ce1a0 0x7fdad40cb530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:53546/0 (socket says 172.21.15.99:53546) 2023-10-08T00:21:30.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.437+0000 7fdacbfff700 1 -- 172.21.15.99:0/2504972973 learned_addr learned my addr 172.21.15.99:0/2504972973 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:30.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.438+0000 7fdad8fad700 1 --2- 172.21.15.99:0/2504972973 >> v2:172.21.15.191:3300/0 conn(0x7fdad4159cf0 0x7fdad415e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:30.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.438+0000 7fdacbfff700 1 -- 172.21.15.99:0/2504972973 >> v2:172.21.15.191:3300/0 conn(0x7fdad4159cf0 msgr2=0x7fdad415e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:30.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.438+0000 7fdacbfff700 1 --2- 172.21.15.99:0/2504972973 >> v2:172.21.15.191:3300/0 conn(0x7fdad4159cf0 0x7fdad415e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:30.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.438+0000 7fdacbfff700 1 -- 172.21.15.99:0/2504972973 >> v1:172.21.15.99:6789/0 conn(0x7fdad40ce1a0 legacy=0x7fdad40cb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:30.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.438+0000 7fdacbfff700 1 -- 172.21.15.99:0/2504972973 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad4067f00 con 0x7fdad415a0c0 2023-10-08T00:21:30.441 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.438+0000 7fdacb7fe700 1 -- 172.21.15.99:0/2504972973 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 706611858 0 0) 0x7fdad40cca20 con 0x7fdad40ce1a0 2023-10-08T00:21:30.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.438+0000 7fdacbfff700 1 --2- 172.21.15.99:0/2504972973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdad415a0c0 0x7fdad40ceab0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fdad40777e0 tx=0x7fdac4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca799d5ac2c6fe9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:30.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.438+0000 7fdacb7fe700 1 -- 172.21.15.99:0/2504972973 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac4009e70 con 0x7fdad415a0c0 2023-10-08T00:21:30.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.438+0000 7fdacb7fe700 1 -- 172.21.15.99:0/2504972973 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac400b860 con 0x7fdad415a0c0 2023-10-08T00:21:30.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.438+0000 7fdacb7fe700 1 -- 172.21.15.99:0/2504972973 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac400bc00 con 0x7fdad415a0c0 2023-10-08T00:21:30.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.439+0000 7fdadb211700 1 -- 172.21.15.99:0/2504972973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdad415a0c0 msgr2=0x7fdad40ceab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:30.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.439+0000 7fdadb211700 1 --2- 172.21.15.99:0/2504972973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdad415a0c0 0x7fdad40ceab0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fdad40777e0 tx=0x7fdac4009250 comp rx=0 tx=0).stop 2023-10-08T00:21:30.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.439+0000 7fdadb211700 1 -- 172.21.15.99:0/2504972973 shutdown_connections 2023-10-08T00:21:30.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.439+0000 7fdadb211700 1 --2- 172.21.15.99:0/2504972973 >> v2:172.21.15.191:3300/0 conn(0x7fdad4159cf0 0x7fdad415e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:30.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.439+0000 7fdadb211700 1 --2- 172.21.15.99:0/2504972973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdad415a0c0 0x7fdad40ceab0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:30.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.439+0000 7fdadb211700 1 -- 172.21.15.99:0/2504972973 >> 172.21.15.99:0/2504972973 conn(0x7fdad406c6e0 msgr2=0x7fdad4069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:30.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.439+0000 7fdadb211700 1 -- 172.21.15.99:0/2504972973 shutdown_connections 2023-10-08T00:21:30.443 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.439+0000 7fdadb211700 1 -- 172.21.15.99:0/2504972973 wait complete. 2023-10-08T00:21:30.443 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.440+0000 7fdadb211700 1 Processor -- start 2023-10-08T00:21:30.443 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.440+0000 7fdadb211700 1 -- start start 2023-10-08T00:21:30.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.441+0000 7fdadb211700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdad40ce1a0 0x7fdad41573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:30.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.441+0000 7fdadb211700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdad4159cf0 0x7fdad41530e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:30.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.441+0000 7fdadb211700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdad415a0c0 0x7fdad40c6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:30.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.441+0000 7fdadb211700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad40b3500 con 0x7fdad4159cf0 2023-10-08T00:21:30.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.441+0000 7fdadb211700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad4157920 con 0x7fdad415a0c0 2023-10-08T00:21:30.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.441+0000 7fdadb211700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdad40c7540 con 0x7fdad40ce1a0 2023-10-08T00:21:30.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.441+0000 7fdacbfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdad4159cf0 0x7fdad41530e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:30.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.441+0000 7fdacbfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdad4159cf0 0x7fdad41530e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53800/0 (socket says 172.21.15.99:53800) 2023-10-08T00:21:30.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.441+0000 7fdacbfff700 1 -- 172.21.15.99:0/1712536707 learned_addr learned my addr 172.21.15.99:0/1712536707 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:30.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.441+0000 7fdad8fad700 1 --2- 172.21.15.99:0/1712536707 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdad40ce1a0 0x7fdad41573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:30.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.441+0000 7fdad97ae700 1 --2- 172.21.15.99:0/1712536707 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdad415a0c0 0x7fdad40c6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:30.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.441+0000 7fdacbfff700 1 -- 172.21.15.99:0/1712536707 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdad40ce1a0 msgr2=0x7fdad41573e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:30.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.441+0000 7fdacbfff700 1 --2- 172.21.15.99:0/1712536707 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdad40ce1a0 0x7fdad41573e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:30.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.441+0000 7fdacbfff700 1 -- 172.21.15.99:0/1712536707 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdad415a0c0 msgr2=0x7fdad40c6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:30.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.441+0000 7fdacbfff700 1 --2- 172.21.15.99:0/1712536707 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdad415a0c0 0x7fdad40c6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:30.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.441+0000 7fdacbfff700 1 -- 172.21.15.99:0/1712536707 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac40094f0 con 0x7fdad4159cf0 2023-10-08T00:21:30.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.441+0000 7fdad97ae700 1 --2- 172.21.15.99:0/1712536707 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdad415a0c0 0x7fdad40c6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:21:30.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.441+0000 7fdacbfff700 1 --2- 172.21.15.99:0/1712536707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdad4159cf0 0x7fdad41530e0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fdad406de20 tx=0x7fdac4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:30.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.441+0000 7fdac8ff9700 1 -- 172.21.15.99:0/1712536707 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac400bbe0 con 0x7fdad4159cf0 2023-10-08T00:21:30.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.441+0000 7fdac8ff9700 1 -- 172.21.15.99:0/1712536707 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac401a450 con 0x7fdad4159cf0 2023-10-08T00:21:30.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.441+0000 7fdadb211700 1 -- 172.21.15.99:0/1712536707 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdad40c7820 con 0x7fdad4159cf0 2023-10-08T00:21:30.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.441+0000 7fdac8ff9700 1 -- 172.21.15.99:0/1712536707 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac401a7f0 con 0x7fdad4159cf0 2023-10-08T00:21:30.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.441+0000 7fdadb211700 1 -- 172.21.15.99:0/1712536707 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdad40b3750 con 0x7fdad4159cf0 2023-10-08T00:21:30.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.442+0000 7fdadb211700 1 -- 172.21.15.99:0/1712536707 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdad40b3940 con 0x7fdad4159cf0 2023-10-08T00:21:30.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.443+0000 7fdac8ff9700 1 -- 172.21.15.99:0/1712536707 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdac401aba0 con 0x7fdad4159cf0 2023-10-08T00:21:30.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.443+0000 7fdac8ff9700 1 --2- 172.21.15.99:0/1712536707 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdac0066f30 0x7fdac00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:30.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.443+0000 7fdac8ff9700 1 -- 172.21.15.99:0/1712536707 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdac4085fe0 con 0x7fdad4159cf0 2023-10-08T00:21:30.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.443+0000 7fdad8fad700 1 --2- 172.21.15.99:0/1712536707 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdac0066f30 0x7fdac00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:30.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.444+0000 7fdad8fad700 1 --2- 172.21.15.99:0/1712536707 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdac0066f30 0x7fdac00693e0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fdad00076a0 tx=0x7fdad000b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:30.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.447+0000 7fdac8ff9700 1 -- 172.21.15.99:0/1712536707 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdac4053250 con 0x7fdad4159cf0 2023-10-08T00:21:30.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.629+0000 7fdadb211700 1 -- 172.21.15.99:0/1712536707 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdad404c3c0 con 0x7fdad4159cf0 2023-10-08T00:21:30.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.630+0000 7fdac8ff9700 1 -- 172.21.15.99:0/1712536707 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v183) v1 ==== 74+0+15378 (secure 0 0 0) 0x7fdac4056870 con 0x7fdad4159cf0 2023-10-08T00:21:30.630 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:21:30.630 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":183,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:21:29.659079+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:21:27.984574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-08T00:21:29.244620+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:21:30.632 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.632+0000 7fdadb211700 1 -- 172.21.15.99:0/1712536707 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdac0066f30 msgr2=0x7fdac00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:30.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.632+0000 7fdadb211700 1 --2- 172.21.15.99:0/1712536707 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdac0066f30 0x7fdac00693e0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fdad00076a0 tx=0x7fdad000b040 comp rx=0 tx=0).stop 2023-10-08T00:21:30.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.632+0000 7fdadb211700 1 -- 172.21.15.99:0/1712536707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdad4159cf0 msgr2=0x7fdad41530e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:30.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.632+0000 7fdadb211700 1 --2- 172.21.15.99:0/1712536707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdad4159cf0 0x7fdad41530e0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fdad406de20 tx=0x7fdac4017000 comp rx=0 tx=0).stop 2023-10-08T00:21:30.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.632+0000 7fdadb211700 1 -- 172.21.15.99:0/1712536707 shutdown_connections 2023-10-08T00:21:30.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.632+0000 7fdadb211700 1 --2- 172.21.15.99:0/1712536707 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdac0066f30 0x7fdac00693e0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:30.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.632+0000 7fdadb211700 1 --2- 172.21.15.99:0/1712536707 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdad415a0c0 0x7fdad40c6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:30.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.632+0000 7fdadb211700 1 --2- 172.21.15.99:0/1712536707 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdad40ce1a0 0x7fdad41573e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:30.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.632+0000 7fdadb211700 1 --2- 172.21.15.99:0/1712536707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdad4159cf0 0x7fdad41530e0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:30.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.632+0000 7fdadb211700 1 -- 172.21.15.99:0/1712536707 >> 172.21.15.99:0/1712536707 conn(0x7fdad406c6e0 msgr2=0x7fdad4068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:30.633 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.632+0000 7fdadb211700 1 -- 172.21.15.99:0/1712536707 shutdown_connections 2023-10-08T00:21:30.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.632+0000 7fdadb211700 1 -- 172.21.15.99:0/1712536707 wait complete. 2023-10-08T00:21:30.645 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:21:30.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.757+0000 7f6c029e4700 1 Processor -- start 2023-10-08T00:21:30.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.757+0000 7f6c029e4700 1 -- start start 2023-10-08T00:21:30.760 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.758+0000 7f6c029e4700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f6bfc0d1c30 0x7f6bfc0ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:30.761 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.758+0000 7f6c029e4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6bfc15a680 0x7f6bfc0cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:30.761 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.758+0000 7f6c029e4700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bfc0c5d80 con 0x7f6bfc15a2b0 2023-10-08T00:21:30.761 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.758+0000 7f6c029e4700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bfc068470 con 0x7f6bfc0d1c30 2023-10-08T00:21:30.761 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.758+0000 7f6c029e4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc0c6060 con 0x7f6bfc15a680 2023-10-08T00:21:30.761 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.758+0000 7f6c00f81700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6bfc15a680 0x7f6bfc0cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:30.761 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.758+0000 7f6bfb7fe700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f6bfc0d1c30 0x7f6bfc0ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:30.761 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.758+0000 7f6bfb7fe700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f6bfc0d1c30 0x7f6bfc0ccdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.99:32908/0 (socket says 172.21.15.99:32908) 2023-10-08T00:21:30.761 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.758+0000 7f6c00f81700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6bfc15a680 0x7f6bfc0cf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.99:39352/0 (socket says 172.21.15.99:39352) 2023-10-08T00:21:30.761 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.758+0000 7f6bfbfff700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f6bfc15a2b0 0x7f6bfc0b3da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:44474/0 (socket says 172.21.15.99:44474) 2023-10-08T00:21:30.762 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.758+0000 7f6bfb7fe700 1 -- 172.21.15.99:0/1704434858 learned_addr learned my addr 172.21.15.99:0/1704434858 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:30.762 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.759+0000 7f6bfb7fe700 1 -- 172.21.15.99:0/1704434858 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6bfc15a680 msgr2=0x7f6bfc0cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:30.762 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.759+0000 7f6bfb7fe700 1 --2- 172.21.15.99:0/1704434858 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6bfc15a680 0x7f6bfc0cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:30.762 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.759+0000 7f6bfb7fe700 1 -- 172.21.15.99:0/1704434858 >> v1:172.21.15.191:6789/0 conn(0x7f6bfc15a2b0 legacy=0x7f6bfc0b3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:30.762 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.759+0000 7f6bfb7fe700 1 -- 172.21.15.99:0/1704434858 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bfc0c6340 con 0x7f6bfc0d1c30 2023-10-08T00:21:30.762 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.759+0000 7f6c00f81700 1 --2- 172.21.15.99:0/1704434858 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6bfc15a680 0x7f6bfc0cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:21:30.762 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.759+0000 7f6bfb7fe700 1 --2- 172.21.15.99:0/1704434858 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6bfc0d1c30 0x7f6bfc0ccdd0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f6bf00099b0 tx=0x7f6bf0009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dacc6921d8e63ac0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:30.763 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.759+0000 7f6bfaffd700 1 -- 172.21.15.99:0/1704434858 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf0009ea0 con 0x7f6bfc0d1c30 2023-10-08T00:21:30.763 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.759+0000 7f6bfaffd700 1 -- 172.21.15.99:0/1704434858 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bf000b860 con 0x7f6bfc0d1c30 2023-10-08T00:21:30.763 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.759+0000 7f6bfaffd700 1 -- 172.21.15.99:0/1704434858 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf000bc00 con 0x7f6bfc0d1c30 2023-10-08T00:21:30.763 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.760+0000 7f6c029e4700 1 -- 172.21.15.99:0/1704434858 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6bfc0d1c30 msgr2=0x7f6bfc0ccdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:30.763 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.760+0000 7f6c029e4700 1 --2- 172.21.15.99:0/1704434858 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6bfc0d1c30 0x7f6bfc0ccdd0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f6bf00099b0 tx=0x7f6bf0009250 comp rx=0 tx=0).stop 2023-10-08T00:21:30.763 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.760+0000 7f6c029e4700 1 -- 172.21.15.99:0/1704434858 shutdown_connections 2023-10-08T00:21:30.763 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.760+0000 7f6c029e4700 1 --2- 172.21.15.99:0/1704434858 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6bfc15a680 0x7f6bfc0cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:30.763 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.760+0000 7f6c029e4700 1 --2- 172.21.15.99:0/1704434858 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6bfc0d1c30 0x7f6bfc0ccdd0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:30.763 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.760+0000 7f6c029e4700 1 -- 172.21.15.99:0/1704434858 >> 172.21.15.99:0/1704434858 conn(0x7f6bfc067e40 msgr2=0x7f6bfc0d0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:30.763 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.760+0000 7f6c029e4700 1 -- 172.21.15.99:0/1704434858 shutdown_connections 2023-10-08T00:21:30.763 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.760+0000 7f6c029e4700 1 -- 172.21.15.99:0/1704434858 wait complete. 2023-10-08T00:21:30.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.761+0000 7f6c029e4700 1 Processor -- start 2023-10-08T00:21:30.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.761+0000 7f6c029e4700 1 -- start start 2023-10-08T00:21:30.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.762+0000 7f6c029e4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6bfc0d1c30 0x7f6bfc15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:30.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.762+0000 7f6c029e4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6bfc15a2b0 0x7f6bfc154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:30.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.762+0000 7f6c029e4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6bfc15a680 0x7f6bfc16a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:30.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.762+0000 7f6c029e4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc0c94c0 con 0x7f6bfc15a2b0 2023-10-08T00:21:30.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.762+0000 7f6c029e4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc0c97a0 con 0x7f6bfc0d1c30 2023-10-08T00:21:30.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.762+0000 7f6c029e4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc0c9a80 con 0x7f6bfc15a680 2023-10-08T00:21:30.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.762+0000 7f6bfb7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6bfc15a2b0 0x7f6bfc154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:30.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.762+0000 7f6bfb7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6bfc15a2b0 0x7f6bfc154e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53810/0 (socket says 172.21.15.99:53810) 2023-10-08T00:21:30.768 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.762+0000 7f6bfb7fe700 1 -- 172.21.15.99:0/1659403009 learned_addr learned my addr 172.21.15.99:0/1659403009 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:30.768 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.762+0000 7f6bfbfff700 1 --2- 172.21.15.99:0/1659403009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6bfc0d1c30 0x7f6bfc15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:30.768 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.762+0000 7f6c00f81700 1 --2- 172.21.15.99:0/1659403009 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6bfc15a680 0x7f6bfc16a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:30.768 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.762+0000 7f6bfb7fe700 1 -- 172.21.15.99:0/1659403009 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6bfc15a680 msgr2=0x7f6bfc16a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:30.768 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.762+0000 7f6bfb7fe700 1 --2- 172.21.15.99:0/1659403009 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6bfc15a680 0x7f6bfc16a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:30.768 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.762+0000 7f6bfb7fe700 1 -- 172.21.15.99:0/1659403009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6bfc0d1c30 msgr2=0x7f6bfc15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:30.768 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.762+0000 7f6bfb7fe700 1 --2- 172.21.15.99:0/1659403009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6bfc0d1c30 0x7f6bfc15bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:30.768 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.762+0000 7f6bfb7fe700 1 -- 172.21.15.99:0/1659403009 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bf4009a20 con 0x7f6bfc15a2b0 2023-10-08T00:21:30.768 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.762+0000 7f6bfb7fe700 1 --2- 172.21.15.99:0/1659403009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6bfc15a2b0 0x7f6bfc154e60 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc089120 tx=0x7f6bf0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:30.768 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.762+0000 7f6be3fff700 1 -- 172.21.15.99:0/1659403009 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf000bbe0 con 0x7f6bfc15a2b0 2023-10-08T00:21:30.769 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.762+0000 7f6be3fff700 1 -- 172.21.15.99:0/1659403009 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bf0011420 con 0x7f6bfc15a2b0 2023-10-08T00:21:30.769 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.762+0000 7f6c029e4700 1 -- 172.21.15.99:0/1659403009 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bf00094f0 con 0x7f6bfc15a2b0 2023-10-08T00:21:30.769 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.763+0000 7f6be3fff700 1 -- 172.21.15.99:0/1659403009 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf00117c0 con 0x7f6bfc15a2b0 2023-10-08T00:21:30.769 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.763+0000 7f6c029e4700 1 -- 172.21.15.99:0/1659403009 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bfc0ca040 con 0x7f6bfc15a2b0 2023-10-08T00:21:30.769 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.763+0000 7f6c029e4700 1 -- 172.21.15.99:0/1659403009 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bfc04c3c0 con 0x7f6bfc15a2b0 2023-10-08T00:21:30.769 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.764+0000 7f6be3fff700 1 -- 172.21.15.99:0/1659403009 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6bf0011bf0 con 0x7f6bfc15a2b0 2023-10-08T00:21:30.769 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.764+0000 7f6be3fff700 1 --2- 172.21.15.99:0/1659403009 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6be8066f60 0x7f6be8069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:30.769 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.764+0000 7f6be3fff700 1 -- 172.21.15.99:0/1659403009 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6bf0085c00 con 0x7f6bfc15a2b0 2023-10-08T00:21:30.769 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.764+0000 7f6bfbfff700 1 --2- 172.21.15.99:0/1659403009 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6be8066f60 0x7f6be8069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:30.769 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.765+0000 7f6bfbfff700 1 --2- 172.21.15.99:0/1659403009 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6be8066f60 0x7f6be8069410 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f6bec000c00 tx=0x7f6bec007ec0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:30.770 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.768+0000 7f6be3fff700 1 -- 172.21.15.99:0/1659403009 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6bf008c080 con 0x7f6bfc15a2b0 2023-10-08T00:21:30.954 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.954+0000 7f6c029e4700 1 -- 172.21.15.99:0/1659403009 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6bfc1586f0 con 0x7f6bfc15a2b0 2023-10-08T00:21:30.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.955+0000 7f6be3fff700 1 -- 172.21.15.99:0/1659403009 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v183) v1 ==== 74+0+15378 (secure 0 0 0) 0x7f6bf0052f30 con 0x7f6bfc15a2b0 2023-10-08T00:21:30.955 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:21:30.956 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":183,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:21:29.659079+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:21:27.984574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-08T00:21:29.244620+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:21:30.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.957+0000 7f6c029e4700 1 -- 172.21.15.99:0/1659403009 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6be8066f60 msgr2=0x7f6be8069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:30.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.957+0000 7f6c029e4700 1 --2- 172.21.15.99:0/1659403009 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6be8066f60 0x7f6be8069410 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f6bec000c00 tx=0x7f6bec007ec0 comp rx=0 tx=0).stop 2023-10-08T00:21:30.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.957+0000 7f6c029e4700 1 -- 172.21.15.99:0/1659403009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6bfc15a2b0 msgr2=0x7f6bfc154e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:30.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.957+0000 7f6c029e4700 1 --2- 172.21.15.99:0/1659403009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6bfc15a2b0 0x7f6bfc154e60 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc089120 tx=0x7f6bf0017000 comp rx=0 tx=0).stop 2023-10-08T00:21:30.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.957+0000 7f6c029e4700 1 -- 172.21.15.99:0/1659403009 shutdown_connections 2023-10-08T00:21:30.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.957+0000 7f6c029e4700 1 --2- 172.21.15.99:0/1659403009 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6be8066f60 0x7f6be8069410 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:30.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.957+0000 7f6c029e4700 1 --2- 172.21.15.99:0/1659403009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6bfc0d1c30 0x7f6bfc15bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:30.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.957+0000 7f6c029e4700 1 --2- 172.21.15.99:0/1659403009 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6bfc15a680 0x7f6bfc16a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:30.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.957+0000 7f6c029e4700 1 --2- 172.21.15.99:0/1659403009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6bfc15a2b0 0x7f6bfc154e60 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:30.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.957+0000 7f6c029e4700 1 -- 172.21.15.99:0/1659403009 >> 172.21.15.99:0/1659403009 conn(0x7f6bfc067e40 msgr2=0x7f6bfc158980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:30.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.957+0000 7f6c029e4700 1 -- 172.21.15.99:0/1659403009 shutdown_connections 2023-10-08T00:21:30.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:30.957+0000 7f6c029e4700 1 -- 172.21.15.99:0/1659403009 wait complete. 2023-10-08T00:21:30.970 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T00:21:31.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.086+0000 7f966b475700 1 Processor -- start 2023-10-08T00:21:31.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.086+0000 7f966b475700 1 -- start start 2023-10-08T00:21:31.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.087+0000 7f966b475700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f966415a290 0x7f966406a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:31.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.087+0000 7f966b475700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f966406aae0 0x7f96640cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:31.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.087+0000 7f966b475700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96640b0790 con 0x7f9664159ec0 2023-10-08T00:21:31.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.087+0000 7f966b475700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f966406af60 con 0x7f966415a290 2023-10-08T00:21:31.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.087+0000 7f966b475700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96640ce1f0 con 0x7f966406aae0 2023-10-08T00:21:31.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.087+0000 7f9668a10700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f966415a290 0x7f966406a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:31.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.087+0000 7f9668a10700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f966415a290 0x7f966406a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53824/0 (socket says 172.21.15.99:53824) 2023-10-08T00:21:31.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.087+0000 7f9669211700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f9664159ec0 0x7f96640af940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:53558/0 (socket says 172.21.15.99:53558) 2023-10-08T00:21:31.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.087+0000 7f9668a10700 1 -- 172.21.15.99:0/1865944053 learned_addr learned my addr 172.21.15.99:0/1865944053 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:31.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.087+0000 7f9669a12700 1 --2- 172.21.15.99:0/1865944053 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f966406aae0 0x7f96640cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:31.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.087+0000 7f9668a10700 1 -- 172.21.15.99:0/1865944053 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f966406aae0 msgr2=0x7f96640cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:31.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.087+0000 7f9668a10700 1 --2- 172.21.15.99:0/1865944053 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f966406aae0 0x7f96640cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:31.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.087+0000 7f9668a10700 1 -- 172.21.15.99:0/1865944053 >> v1:172.21.15.99:6789/0 conn(0x7f9664159ec0 legacy=0x7f96640af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:31.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.087+0000 7f9668a10700 1 -- 172.21.15.99:0/1865944053 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96640b0b20 con 0x7f966415a290 2023-10-08T00:21:31.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.087+0000 7f965bfff700 1 -- 172.21.15.99:0/1865944053 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3347566934 0 0) 0x7f96640b0790 con 0x7f9664159ec0 2023-10-08T00:21:31.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.087+0000 7f9669a12700 1 --2- 172.21.15.99:0/1865944053 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f966406aae0 0x7f96640cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:21:31.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.087+0000 7f9668a10700 1 --2- 172.21.15.99:0/1865944053 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f966415a290 0x7f966406a5a0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f96540099b0 tx=0x7f9654009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94f2f7ba63026fbd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:31.090 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.088+0000 7f965bfff700 1 -- 172.21.15.99:0/1865944053 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965400b860 con 0x7f966415a290 2023-10-08T00:21:31.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.088+0000 7f965bfff700 1 -- 172.21.15.99:0/1865944053 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f965400b9c0 con 0x7f966415a290 2023-10-08T00:21:31.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.088+0000 7f965bfff700 1 -- 172.21.15.99:0/1865944053 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965400bd60 con 0x7f966415a290 2023-10-08T00:21:31.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.088+0000 7f966b475700 1 -- 172.21.15.99:0/1865944053 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f966415a290 msgr2=0x7f966406a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:31.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.088+0000 7f966b475700 1 --2- 172.21.15.99:0/1865944053 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f966415a290 0x7f966406a5a0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f96540099b0 tx=0x7f9654009250 comp rx=0 tx=0).stop 2023-10-08T00:21:31.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.088+0000 7f966b475700 1 -- 172.21.15.99:0/1865944053 shutdown_connections 2023-10-08T00:21:31.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.088+0000 7f966b475700 1 --2- 172.21.15.99:0/1865944053 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f966406aae0 0x7f96640cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:31.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.088+0000 7f966b475700 1 --2- 172.21.15.99:0/1865944053 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f966415a290 0x7f966406a5a0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:31.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.088+0000 7f966b475700 1 -- 172.21.15.99:0/1865944053 >> 172.21.15.99:0/1865944053 conn(0x7f96640d1880 msgr2=0x7f96640cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:31.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.088+0000 7f966b475700 1 -- 172.21.15.99:0/1865944053 shutdown_connections 2023-10-08T00:21:31.091 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.088+0000 7f966b475700 1 -- 172.21.15.99:0/1865944053 wait complete. 2023-10-08T00:21:31.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.089+0000 7f966b475700 1 Processor -- start 2023-10-08T00:21:31.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.089+0000 7f966b475700 1 -- start start 2023-10-08T00:21:31.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.090+0000 7f966b475700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f966406aae0 0x7f96640b4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:31.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.090+0000 7f966b475700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9664159ec0 0x7f96640c9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:31.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.090+0000 7f966b475700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f966415a290 0x7f9664169bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:31.095 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.090+0000 7f966b475700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96640c9890 con 0x7f966406aae0 2023-10-08T00:21:31.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.090+0000 7f966b475700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96640c82b0 con 0x7f966415a290 2023-10-08T00:21:31.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.090+0000 7f966b475700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96640c8590 con 0x7f9664159ec0 2023-10-08T00:21:31.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.090+0000 7f9669211700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f966406aae0 0x7f96640b4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:31.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.090+0000 7f9669211700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f966406aae0 0x7f96640b4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53836/0 (socket says 172.21.15.99:53836) 2023-10-08T00:21:31.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.090+0000 7f9669211700 1 -- 172.21.15.99:0/465878952 learned_addr learned my addr 172.21.15.99:0/465878952 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:31.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.091+0000 7f9669a12700 1 --2- 172.21.15.99:0/465878952 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f966415a290 0x7f9664169bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:31.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.091+0000 7f9668a10700 1 --2- 172.21.15.99:0/465878952 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9664159ec0 0x7f96640c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:31.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.091+0000 7f9669211700 1 -- 172.21.15.99:0/465878952 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9664159ec0 msgr2=0x7f96640c9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:31.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.091+0000 7f9669211700 1 --2- 172.21.15.99:0/465878952 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9664159ec0 0x7f96640c9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:31.096 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.091+0000 7f9669211700 1 -- 172.21.15.99:0/465878952 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f966415a290 msgr2=0x7f9664169bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:31.097 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.091+0000 7f9669211700 1 --2- 172.21.15.99:0/465878952 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f966415a290 0x7f9664169bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:31.097 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.091+0000 7f9669211700 1 -- 172.21.15.99:0/465878952 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96540094f0 con 0x7f966406aae0 2023-10-08T00:21:31.097 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.091+0000 7f9669211700 1 --2- 172.21.15.99:0/465878952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f966406aae0 0x7f96640b4a20 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f966000cdd0 tx=0x7f9660001bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:31.097 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.091+0000 7f96597fa700 1 -- 172.21.15.99:0/465878952 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9660005df0 con 0x7f966406aae0 2023-10-08T00:21:31.097 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.091+0000 7f96597fa700 1 -- 172.21.15.99:0/465878952 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f966000a960 con 0x7f966406aae0 2023-10-08T00:21:31.097 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.091+0000 7f966b475700 1 -- 172.21.15.99:0/465878952 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96640c8870 con 0x7f966406aae0 2023-10-08T00:21:31.097 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.091+0000 7f96597fa700 1 -- 172.21.15.99:0/465878952 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f966000ad20 con 0x7f966406aae0 2023-10-08T00:21:31.098 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.091+0000 7f966b475700 1 -- 172.21.15.99:0/465878952 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96640c8e30 con 0x7f966406aae0 2023-10-08T00:21:31.098 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.092+0000 7f966b475700 1 -- 172.21.15.99:0/465878952 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96640c9020 con 0x7f966406aae0 2023-10-08T00:21:31.098 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.092+0000 7f96597fa700 1 -- 172.21.15.99:0/465878952 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9660007300 con 0x7f966406aae0 2023-10-08T00:21:31.098 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.093+0000 7f96597fa700 1 --2- 172.21.15.99:0/465878952 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9648066f30 0x7f96480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:31.098 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.093+0000 7f96597fa700 1 -- 172.21.15.99:0/465878952 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9660084790 con 0x7f966406aae0 2023-10-08T00:21:31.098 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.093+0000 7f9668a10700 1 --2- 172.21.15.99:0/465878952 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9648066f30 0x7f96480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:31.098 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.093+0000 7f9668a10700 1 --2- 172.21.15.99:0/465878952 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9648066f30 0x7f96480693e0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f96540099b0 tx=0x7f9654017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:31.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.097+0000 7f96597fa700 1 -- 172.21.15.99:0/465878952 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9660051a80 con 0x7f966406aae0 2023-10-08T00:21:31.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:31.324+0000 7f966b475700 1 -- 172.21.15.99:0/465878952 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f966404c3c0 con 0x7f966406aae0 2023-10-08T00:21:32.313 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.312+0000 7f96597fa700 1 -- 172.21.15.99:0/465878952 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v83) v1 ==== 118+0+0 (secure 0 0 0) 0x7f96600550a0 con 0x7f966406aae0 2023-10-08T00:21:32.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.315+0000 7f966b475700 1 -- 172.21.15.99:0/465878952 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9648066f30 msgr2=0x7f96480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:32.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.315+0000 7f966b475700 1 --2- 172.21.15.99:0/465878952 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9648066f30 0x7f96480693e0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f96540099b0 tx=0x7f9654017000 comp rx=0 tx=0).stop 2023-10-08T00:21:32.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.315+0000 7f966b475700 1 -- 172.21.15.99:0/465878952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f966406aae0 msgr2=0x7f96640b4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:32.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.315+0000 7f966b475700 1 --2- 172.21.15.99:0/465878952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f966406aae0 0x7f96640b4a20 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f966000cdd0 tx=0x7f9660001bc0 comp rx=0 tx=0).stop 2023-10-08T00:21:32.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.315+0000 7f966b475700 1 -- 172.21.15.99:0/465878952 shutdown_connections 2023-10-08T00:21:32.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.315+0000 7f966b475700 1 --2- 172.21.15.99:0/465878952 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9648066f30 0x7f96480693e0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:32.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.315+0000 7f966b475700 1 --2- 172.21.15.99:0/465878952 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f966415a290 0x7f9664169bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:32.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.315+0000 7f966b475700 1 --2- 172.21.15.99:0/465878952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f966406aae0 0x7f96640b4a20 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:32.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.315+0000 7f966b475700 1 --2- 172.21.15.99:0/465878952 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9664159ec0 0x7f96640c9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:32.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.315+0000 7f966b475700 1 -- 172.21.15.99:0/465878952 >> 172.21.15.99:0/465878952 conn(0x7f96640d1880 msgr2=0x7f96640c5b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:32.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.315+0000 7f966b475700 1 -- 172.21.15.99:0/465878952 shutdown_connections 2023-10-08T00:21:32.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.315+0000 7f966b475700 1 -- 172.21.15.99:0/465878952 wait complete. 2023-10-08T00:21:32.329 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T00:21:32.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.446+0000 7f7177cc7700 1 Processor -- start 2023-10-08T00:21:32.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.447+0000 7f7177cc7700 1 -- start start 2023-10-08T00:21:32.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.447+0000 7f7177cc7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7170154440 0x7f717015da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:32.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.447+0000 7f7177cc7700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7170067d70 0x7f71700cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:32.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.447+0000 7f7177cc7700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f717015e1e0 con 0x7f717015af30 2023-10-08T00:21:32.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.447+0000 7f7177cc7700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f717015dfd0 con 0x7f7170067d70 2023-10-08T00:21:32.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.447+0000 7f7177cc7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f717015e4c0 con 0x7f7170154440 2023-10-08T00:21:32.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.448+0000 7f7176264700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7170067d70 0x7f71700cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:32.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.448+0000 7f7175262700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f717015af30 0x7f7170067660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:53572/0 (socket says 172.21.15.99:53572) 2023-10-08T00:21:32.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.448+0000 7f7175262700 1 -- 172.21.15.99:0/4111871008 learned_addr learned my addr 172.21.15.99:0/4111871008 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:21:32.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.448+0000 7f7176264700 1 -- 172.21.15.99:0/4111871008 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7170154440 msgr2=0x7f717015da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:32.450 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.448+0000 7f7175a63700 1 --2- 172.21.15.99:0/4111871008 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7170154440 0x7f717015da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:32.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.448+0000 7f7174a61700 1 -- 172.21.15.99:0/4111871008 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4288270719 0 0) 0x7f717015e1e0 con 0x7f717015af30 2023-10-08T00:21:32.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.448+0000 7f7176264700 1 --2- 172.21.15.99:0/4111871008 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7170154440 0x7f717015da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:32.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.448+0000 7f7176264700 1 -- 172.21.15.99:0/4111871008 >> v1:172.21.15.99:6789/0 conn(0x7f717015af30 legacy=0x7f7170067660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:32.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.448+0000 7f7176264700 1 -- 172.21.15.99:0/4111871008 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f717015e7a0 con 0x7f7170067d70 2023-10-08T00:21:32.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.448+0000 7f7175a63700 1 --2- 172.21.15.99:0/4111871008 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7170154440 0x7f717015da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:21:32.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.448+0000 7f7176264700 1 --2- 172.21.15.99:0/4111871008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7170067d70 0x7f71700cfa20 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f71680098e0 tx=0x7f71680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16be54cd786a04e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:32.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.448+0000 7f7174a61700 1 -- 172.21.15.99:0/4111871008 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7168009e80 con 0x7f7170067d70 2023-10-08T00:21:32.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.448+0000 7f7174a61700 1 -- 172.21.15.99:0/4111871008 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f716800b860 con 0x7f7170067d70 2023-10-08T00:21:32.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.448+0000 7f7174a61700 1 -- 172.21.15.99:0/4111871008 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716800bbe0 con 0x7f7170067d70 2023-10-08T00:21:32.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.449+0000 7f7177cc7700 1 -- 172.21.15.99:0/4111871008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7170067d70 msgr2=0x7f71700cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:32.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.449+0000 7f7177cc7700 1 --2- 172.21.15.99:0/4111871008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7170067d70 0x7f71700cfa20 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f71680098e0 tx=0x7f71680091a0 comp rx=0 tx=0).stop 2023-10-08T00:21:32.451 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.449+0000 7f7177cc7700 1 -- 172.21.15.99:0/4111871008 shutdown_connections 2023-10-08T00:21:32.452 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.449+0000 7f7177cc7700 1 --2- 172.21.15.99:0/4111871008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7170067d70 0x7f71700cfa20 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:32.452 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.449+0000 7f7177cc7700 1 --2- 172.21.15.99:0/4111871008 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7170154440 0x7f717015da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:32.452 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.449+0000 7f7177cc7700 1 -- 172.21.15.99:0/4111871008 >> 172.21.15.99:0/4111871008 conn(0x7f71700d0dd0 msgr2=0x7f71700d11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:32.452 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.449+0000 7f7177cc7700 1 -- 172.21.15.99:0/4111871008 shutdown_connections 2023-10-08T00:21:32.452 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.449+0000 7f7177cc7700 1 -- 172.21.15.99:0/4111871008 wait complete. 2023-10-08T00:21:32.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.450+0000 7f7177cc7700 1 Processor -- start 2023-10-08T00:21:32.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.450+0000 7f7177cc7700 1 -- start start 2023-10-08T00:21:32.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.451+0000 7f7177cc7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7170067d70 0x7f7170158360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:32.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.451+0000 7f7177cc7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7170154440 0x7f717016c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:32.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.451+0000 7f7177cc7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f717015af30 0x7f717016c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:32.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.451+0000 7f7177cc7700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71700b31d0 con 0x7f7170067d70 2023-10-08T00:21:32.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.451+0000 7f7177cc7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71700b25c0 con 0x7f7170154440 2023-10-08T00:21:32.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.451+0000 7f7177cc7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71700b28a0 con 0x7f717015af30 2023-10-08T00:21:32.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.451+0000 7f7175a63700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7170067d70 0x7f7170158360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:32.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.451+0000 7f7175a63700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7170067d70 0x7f7170158360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53854/0 (socket says 172.21.15.99:53854) 2023-10-08T00:21:32.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.451+0000 7f7175a63700 1 -- 172.21.15.99:0/851010437 learned_addr learned my addr 172.21.15.99:0/851010437 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:32.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.451+0000 7f7175a63700 1 -- 172.21.15.99:0/851010437 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f717015af30 msgr2=0x7f717016c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:32.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.451+0000 7f7175a63700 1 --2- 172.21.15.99:0/851010437 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f717015af30 0x7f717016c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:32.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.451+0000 7f7175a63700 1 -- 172.21.15.99:0/851010437 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7170154440 msgr2=0x7f717016c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:32.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.451+0000 7f7175a63700 1 --2- 172.21.15.99:0/851010437 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7170154440 0x7f717016c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:32.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.451+0000 7f7175a63700 1 -- 172.21.15.99:0/851010437 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7168009420 con 0x7f7170067d70 2023-10-08T00:21:32.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.451+0000 7f7175a63700 1 --2- 172.21.15.99:0/851010437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7170067d70 0x7f7170158360 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f716c00a380 tx=0x7f716c007910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:32.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.451+0000 7f7165ffb700 1 -- 172.21.15.99:0/851010437 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716c00ae50 con 0x7f7170067d70 2023-10-08T00:21:32.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.451+0000 7f7165ffb700 1 -- 172.21.15.99:0/851010437 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f716c00d650 con 0x7f7170067d70 2023-10-08T00:21:32.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.451+0000 7f7165ffb700 1 -- 172.21.15.99:0/851010437 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716c00d9f0 con 0x7f7170067d70 2023-10-08T00:21:32.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.451+0000 7f7177cc7700 1 -- 172.21.15.99:0/851010437 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71700b2b80 con 0x7f7170067d70 2023-10-08T00:21:32.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.452+0000 7f7177cc7700 1 -- 172.21.15.99:0/851010437 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71701588a0 con 0x7f7170067d70 2023-10-08T00:21:32.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.452+0000 7f7177cc7700 1 -- 172.21.15.99:0/851010437 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f717004c3c0 con 0x7f7170067d70 2023-10-08T00:21:32.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.453+0000 7f7165ffb700 1 -- 172.21.15.99:0/851010437 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f716c015440 con 0x7f7170067d70 2023-10-08T00:21:32.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.453+0000 7f7165ffb700 1 --2- 172.21.15.99:0/851010437 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7154066f30 0x7f71540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:32.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.453+0000 7f7165ffb700 1 -- 172.21.15.99:0/851010437 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f716c084df0 con 0x7f7170067d70 2023-10-08T00:21:32.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.453+0000 7f7175262700 1 --2- 172.21.15.99:0/851010437 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7154066f30 0x7f71540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:32.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.454+0000 7f7175262700 1 --2- 172.21.15.99:0/851010437 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7154066f30 0x7f71540693e0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f71700b3560 tx=0x7f7160007b60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:32.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.457+0000 7f7165ffb700 1 -- 172.21.15.99:0/851010437 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f716c04e5a0 con 0x7f7170067d70 2023-10-08T00:21:32.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:32.685+0000 7f7177cc7700 1 -- 172.21.15.99:0/851010437 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f71700cf500 con 0x7f7170067d70 2023-10-08T00:21:33.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.315+0000 7f7165ffb700 1 -- 172.21.15.99:0/851010437 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v84) v1 ==== 122+0+0 (secure 0 0 0) 0x7f716c01a0c0 con 0x7f7170067d70 2023-10-08T00:21:33.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.317+0000 7f7177cc7700 1 -- 172.21.15.99:0/851010437 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7154066f30 msgr2=0x7f71540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:33.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.317+0000 7f7177cc7700 1 --2- 172.21.15.99:0/851010437 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7154066f30 0x7f71540693e0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f71700b3560 tx=0x7f7160007b60 comp rx=0 tx=0).stop 2023-10-08T00:21:33.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.317+0000 7f7177cc7700 1 -- 172.21.15.99:0/851010437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7170067d70 msgr2=0x7f7170158360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:33.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.317+0000 7f7177cc7700 1 --2- 172.21.15.99:0/851010437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7170067d70 0x7f7170158360 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f716c00a380 tx=0x7f716c007910 comp rx=0 tx=0).stop 2023-10-08T00:21:33.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.317+0000 7f7177cc7700 1 -- 172.21.15.99:0/851010437 shutdown_connections 2023-10-08T00:21:33.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.317+0000 7f7177cc7700 1 --2- 172.21.15.99:0/851010437 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7154066f30 0x7f71540693e0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:33.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.317+0000 7f7177cc7700 1 --2- 172.21.15.99:0/851010437 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f717015af30 0x7f717016c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:33.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.317+0000 7f7177cc7700 1 --2- 172.21.15.99:0/851010437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7170067d70 0x7f7170158360 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:33.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.317+0000 7f7177cc7700 1 --2- 172.21.15.99:0/851010437 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7170154440 0x7f717016c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:33.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.317+0000 7f7177cc7700 1 -- 172.21.15.99:0/851010437 >> 172.21.15.99:0/851010437 conn(0x7f71700d0dd0 msgr2=0x7f717015ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:33.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.317+0000 7f7177cc7700 1 -- 172.21.15.99:0/851010437 shutdown_connections 2023-10-08T00:21:33.319 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.317+0000 7f7177cc7700 1 -- 172.21.15.99:0/851010437 wait complete. 2023-10-08T00:21:33.331 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:21:33.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.446+0000 7f2a3ef5a700 1 Processor -- start 2023-10-08T00:21:33.447 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.446+0000 7f2a3ef5a700 1 -- start start 2023-10-08T00:21:33.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.447+0000 7f2a3ef5a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a38159e10 0x7f2a380b2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:33.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.447+0000 7f2a3ef5a700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a3815db80 con 0x7f2a3815a1e0 2023-10-08T00:21:33.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.447+0000 7f2a3ef5a700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a3815de60 con 0x7f2a380cde50 2023-10-08T00:21:33.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.447+0000 7f2a3ef5a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3815e800 con 0x7f2a38159e10 2023-10-08T00:21:33.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.447+0000 7f2a37fff700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f2a3815a1e0 0x7f2a380cd740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:57630/0 (socket says 172.21.15.99:57630) 2023-10-08T00:21:33.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.447+0000 7f2a37fff700 1 -- 172.21.15.99:0/3493013177 learned_addr learned my addr 172.21.15.99:0/3493013177 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:21:33.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.448+0000 7f2a3ccf6700 1 --2- 172.21.15.99:0/3493013177 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a38159e10 0x7f2a380b2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:33.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.448+0000 7f2a377fe700 1 -- 172.21.15.99:0/3493013177 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3294747369 0 0) 0x7f2a3815db80 con 0x7f2a3815a1e0 2023-10-08T00:21:33.453 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.448+0000 7f2a377fe700 1 -- 172.21.15.99:0/3493013177 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a24003570 con 0x7f2a3815a1e0 2023-10-08T00:21:33.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.448+0000 7f2a377fe700 1 -- 172.21.15.99:0/3493013177 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2652517715 0 0) 0x7f2a3815de60 con 0x7f2a380cde50 2023-10-08T00:21:33.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.448+0000 7f2a377fe700 1 -- 172.21.15.99:0/3493013177 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a3815db80 con 0x7f2a380cde50 2023-10-08T00:21:33.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.448+0000 7f2a377fe700 1 -- 172.21.15.99:0/3493013177 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4032094484 0 0) 0x7f2a24003570 con 0x7f2a3815a1e0 2023-10-08T00:21:33.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.448+0000 7f2a377fe700 1 -- 172.21.15.99:0/3493013177 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a38159e10 msgr2=0x7f2a380b2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:33.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.448+0000 7f2a377fe700 1 --2- 172.21.15.99:0/3493013177 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a38159e10 0x7f2a380b2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:33.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.448+0000 7f2a377fe700 1 -- 172.21.15.99:0/3493013177 >> v1:172.21.15.191:6789/0 conn(0x7f2a380cde50 legacy=0x7f2a380cbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:33.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.448+0000 7f2a377fe700 1 -- 172.21.15.99:0/3493013177 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a3815e140 con 0x7f2a3815a1e0 2023-10-08T00:21:33.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.448+0000 7f2a377fe700 1 -- 172.21.15.99:0/3493013177 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f2a28001490 con 0x7f2a3815a1e0 2023-10-08T00:21:33.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.448+0000 7f2a377fe700 1 -- 172.21.15.99:0/3493013177 >> v1:172.21.15.99:6789/0 conn(0x7f2a3815a1e0 legacy=0x7f2a380cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:33.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.448+0000 7f2a377fe700 1 --2- 172.21.15.99:0/3493013177 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a24003c20 0x7f2a24004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:33.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.448+0000 7f2a377fe700 1 --2- 172.21.15.99:0/3493013177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a24004810 0x7f2a24004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:33.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.448+0000 7f2a377fe700 1 --2- 172.21.15.99:0/3493013177 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a38159e10 0x7f2a24005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:33.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.448+0000 7f2a377fe700 1 -- 172.21.15.99:0/3493013177 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3815e800 con 0x7f2a24004810 2023-10-08T00:21:33.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.448+0000 7f2a377fe700 1 -- 172.21.15.99:0/3493013177 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2400bbf0 con 0x7f2a24003c20 2023-10-08T00:21:33.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.448+0000 7f2a377fe700 1 -- 172.21.15.99:0/3493013177 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2400bdb0 con 0x7f2a38159e10 2023-10-08T00:21:33.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.448+0000 7f2a37fff700 1 --2- 172.21.15.99:0/3493013177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a24004810 0x7f2a24004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:33.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.448+0000 7f2a3ccf6700 1 --2- 172.21.15.99:0/3493013177 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a24003c20 0x7f2a24004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:33.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.448+0000 7f2a3d4f7700 1 --2- 172.21.15.99:0/3493013177 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a38159e10 0x7f2a24005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:33.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.448+0000 7f2a37fff700 1 -- 172.21.15.99:0/3493013177 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a38159e10 msgr2=0x7f2a24005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:33.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.448+0000 7f2a37fff700 1 --2- 172.21.15.99:0/3493013177 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a38159e10 0x7f2a24005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:33.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.448+0000 7f2a37fff700 1 -- 172.21.15.99:0/3493013177 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a24003c20 msgr2=0x7f2a24004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:33.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.448+0000 7f2a37fff700 1 --2- 172.21.15.99:0/3493013177 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a24003c20 0x7f2a24004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:33.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.448+0000 7f2a37fff700 1 -- 172.21.15.99:0/3493013177 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a24003570 con 0x7f2a24004810 2023-10-08T00:21:33.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.448+0000 7f2a3d4f7700 1 --2- 172.21.15.99:0/3493013177 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a38159e10 0x7f2a24005220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:21:33.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.449+0000 7f2a37fff700 1 --2- 172.21.15.99:0/3493013177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a24004810 0x7f2a24004c80 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f2a2800c1e0 tx=0x7f2a28001d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d5cb7cb6f134f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:33.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.449+0000 7f2a377fe700 1 -- 172.21.15.99:0/3493013177 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2800c9e0 con 0x7f2a24004810 2023-10-08T00:21:33.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.449+0000 7f2a377fe700 1 -- 172.21.15.99:0/3493013177 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a2800cb40 con 0x7f2a24004810 2023-10-08T00:21:33.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.449+0000 7f2a377fe700 1 -- 172.21.15.99:0/3493013177 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a280048f0 con 0x7f2a24004810 2023-10-08T00:21:33.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.449+0000 7f2a3ef5a700 1 -- 172.21.15.99:0/3493013177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a24004810 msgr2=0x7f2a24004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:33.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.449+0000 7f2a3ef5a700 1 --2- 172.21.15.99:0/3493013177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a24004810 0x7f2a24004c80 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f2a2800c1e0 tx=0x7f2a28001d60 comp rx=0 tx=0).stop 2023-10-08T00:21:33.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.449+0000 7f2a3d4f7700 1 -- 172.21.15.99:0/3493013177 reap_dead start 2023-10-08T00:21:33.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.449+0000 7f2a3ef5a700 1 -- 172.21.15.99:0/3493013177 shutdown_connections 2023-10-08T00:21:33.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.449+0000 7f2a3ef5a700 1 -- 172.21.15.99:0/3493013177 >> 172.21.15.99:0/3493013177 conn(0x7f2a38067660 msgr2=0x7f2a38067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:33.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.449+0000 7f2a3ef5a700 1 -- 172.21.15.99:0/3493013177 shutdown_connections 2023-10-08T00:21:33.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.449+0000 7f2a3ef5a700 1 -- 172.21.15.99:0/3493013177 wait complete. 2023-10-08T00:21:33.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.450+0000 7f2a3ef5a700 1 Processor -- start 2023-10-08T00:21:33.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.450+0000 7f2a3ef5a700 1 -- start start 2023-10-08T00:21:33.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.451+0000 7f2a3ef5a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a24004810 0x7f2a380c6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:33.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.451+0000 7f2a3ef5a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a38158680 0x7f2a3816d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:33.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.451+0000 7f2a3ef5a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a380b4fa0 0x7f2a380b5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:33.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.451+0000 7f2a3ef5a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a380c63c0 con 0x7f2a380b4fa0 2023-10-08T00:21:33.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.451+0000 7f2a3ef5a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a380b5950 con 0x7f2a38158680 2023-10-08T00:21:33.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.451+0000 7f2a3ef5a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3816e640 con 0x7f2a24004810 2023-10-08T00:21:33.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.451+0000 7f2a3d4f7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a380b4fa0 0x7f2a380b5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:33.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.451+0000 7f2a3d4f7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a380b4fa0 0x7f2a380b5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51870/0 (socket says 172.21.15.99:51870) 2023-10-08T00:21:33.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.451+0000 7f2a3d4f7700 1 -- 172.21.15.99:0/3638169865 learned_addr learned my addr 172.21.15.99:0/3638169865 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:33.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.451+0000 7f2a3ccf6700 1 --2- 172.21.15.99:0/3638169865 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a24004810 0x7f2a380c6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:33.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.451+0000 7f2a37fff700 1 --2- 172.21.15.99:0/3638169865 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a38158680 0x7f2a3816d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:33.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.452+0000 7f2a3d4f7700 1 -- 172.21.15.99:0/3638169865 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a24004810 msgr2=0x7f2a380c6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:33.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.452+0000 7f2a3d4f7700 1 --2- 172.21.15.99:0/3638169865 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a24004810 0x7f2a380c6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:33.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.452+0000 7f2a3d4f7700 1 -- 172.21.15.99:0/3638169865 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a38158680 msgr2=0x7f2a3816d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:33.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.452+0000 7f2a3d4f7700 1 --2- 172.21.15.99:0/3638169865 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a38158680 0x7f2a3816d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:33.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.452+0000 7f2a3d4f7700 1 -- 172.21.15.99:0/3638169865 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a380cea30 con 0x7f2a380b4fa0 2023-10-08T00:21:33.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.452+0000 7f2a3d4f7700 1 --2- 172.21.15.99:0/3638169865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a380b4fa0 0x7f2a380b5410 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f2a380b6660 tx=0x7f2a20009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:33.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.452+0000 7f2a34ff9700 1 -- 172.21.15.99:0/3638169865 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a20018070 con 0x7f2a380b4fa0 2023-10-08T00:21:33.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.452+0000 7f2a34ff9700 1 -- 172.21.15.99:0/3638169865 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a20006820 con 0x7f2a380b4fa0 2023-10-08T00:21:33.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.452+0000 7f2a3ef5a700 1 -- 172.21.15.99:0/3638169865 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a3816dda0 con 0x7f2a380b4fa0 2023-10-08T00:21:33.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.452+0000 7f2a34ff9700 1 -- 172.21.15.99:0/3638169865 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a20006bc0 con 0x7f2a380b4fa0 2023-10-08T00:21:33.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.452+0000 7f2a3ef5a700 1 -- 172.21.15.99:0/3638169865 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a3816e360 con 0x7f2a380b4fa0 2023-10-08T00:21:33.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.453+0000 7f2a3ef5a700 1 -- 172.21.15.99:0/3638169865 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a380cebc0 con 0x7f2a380b4fa0 2023-10-08T00:21:33.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.453+0000 7f2a34ff9700 1 -- 172.21.15.99:0/3638169865 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2a20007030 con 0x7f2a380b4fa0 2023-10-08T00:21:33.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.454+0000 7f2a34ff9700 1 --2- 172.21.15.99:0/3638169865 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2a18066f30 0x7f2a180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:33.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.454+0000 7f2a34ff9700 1 -- 172.21.15.99:0/3638169865 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2a20083eb0 con 0x7f2a380b4fa0 2023-10-08T00:21:33.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.454+0000 7f2a3ccf6700 1 --2- 172.21.15.99:0/3638169865 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2a18066f30 0x7f2a180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:33.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.454+0000 7f2a3ccf6700 1 --2- 172.21.15.99:0/3638169865 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2a18066f30 0x7f2a180693e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f2a300094d0 tx=0x7f2a3000b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:33.461 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.458+0000 7f2a34ff9700 1 -- 172.21.15.99:0/3638169865 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2a20051120 con 0x7f2a380b4fa0 2023-10-08T00:21:33.680 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.679+0000 7f2a3ef5a700 1 -- 172.21.15.99:0/3638169865 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a3804c3c0 con 0x7f2a380b4fa0 2023-10-08T00:21:33.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.680+0000 7f2a34ff9700 1 -- 172.21.15.99:0/3638169865 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+4604 (secure 0 0 0) 0x7f2a20054740 con 0x7f2a380b4fa0 2023-10-08T00:21:33.681 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:21:33.682 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":84,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/3862684847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3862684847},{"type":"v1","addr":"172.21.15.191:6836","nonce":3862684847}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":8106,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/359623206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":359623206},{"type":"v1","addr":"172.21.15.99:6834","nonce":359623206}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":8109,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3064101777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":3064101777},{"type":"v1","addr":"172.21.15.99:6835","nonce":3064101777}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:21:29.659394+0000","modified":"2023-10-08T00:21:33.311802+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8124},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8124":{"gid":8124,"name":"d","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/2466421758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":2466421758},{"type":"v1","addr":"172.21.15.191:6837","nonce":2466421758}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-08T00:21:33.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.682+0000 7f2a3ef5a700 1 -- 172.21.15.99:0/3638169865 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2a18066f30 msgr2=0x7f2a180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:33.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.682+0000 7f2a3ef5a700 1 --2- 172.21.15.99:0/3638169865 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2a18066f30 0x7f2a180693e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f2a300094d0 tx=0x7f2a3000b040 comp rx=0 tx=0).stop 2023-10-08T00:21:33.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.682+0000 7f2a3ef5a700 1 -- 172.21.15.99:0/3638169865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a380b4fa0 msgr2=0x7f2a380b5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:33.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.682+0000 7f2a3ef5a700 1 --2- 172.21.15.99:0/3638169865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a380b4fa0 0x7f2a380b5410 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f2a380b6660 tx=0x7f2a20009040 comp rx=0 tx=0).stop 2023-10-08T00:21:33.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.682+0000 7f2a3ef5a700 1 -- 172.21.15.99:0/3638169865 shutdown_connections 2023-10-08T00:21:33.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.682+0000 7f2a3ef5a700 1 --2- 172.21.15.99:0/3638169865 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2a18066f30 0x7f2a180693e0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:33.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.682+0000 7f2a3ef5a700 1 --2- 172.21.15.99:0/3638169865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a380b4fa0 0x7f2a380b5410 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:33.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.682+0000 7f2a3ef5a700 1 --2- 172.21.15.99:0/3638169865 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a24004810 0x7f2a380c6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:33.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.682+0000 7f2a3ef5a700 1 --2- 172.21.15.99:0/3638169865 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a38158680 0x7f2a3816d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:33.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.682+0000 7f2a3ef5a700 1 -- 172.21.15.99:0/3638169865 >> 172.21.15.99:0/3638169865 conn(0x7f2a38067660 msgr2=0x7f2a38159210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:33.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.682+0000 7f2a3ef5a700 1 -- 172.21.15.99:0/3638169865 shutdown_connections 2023-10-08T00:21:33.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.682+0000 7f2a3ef5a700 1 -- 172.21.15.99:0/3638169865 wait complete. 2023-10-08T00:21:33.686 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 84 2023-10-08T00:21:33.695 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:21:33.808 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.807+0000 7fba6c37f700 1 Processor -- start 2023-10-08T00:21:33.808 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.807+0000 7fba6c37f700 1 -- start start 2023-10-08T00:21:33.810 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.808+0000 7fba6c37f700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fba64157870 0x7fba6415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:33.810 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.808+0000 7fba6c37f700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fba64157c40 0x7fba64158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:33.811 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.808+0000 7fba6c37f700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba6415cbb0 con 0x7fba6415a620 2023-10-08T00:21:33.811 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.808+0000 7fba6c37f700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba6415bfa0 con 0x7fba64157c40 2023-10-08T00:21:33.811 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.808+0000 7fba6c37f700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba6415c280 con 0x7fba64157870 2023-10-08T00:21:33.811 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.809+0000 7fba6a91c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fba64157c40 0x7fba64158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:33.811 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.809+0000 7fba6a91c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fba64157c40 0x7fba64158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51872/0 (socket says 172.21.15.99:51872) 2023-10-08T00:21:33.811 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.809+0000 7fba6a91c700 1 -- 172.21.15.99:0/3656927895 learned_addr learned my addr 172.21.15.99:0/3656927895 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:33.811 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.809+0000 7fba6a11b700 1 --2- 172.21.15.99:0/3656927895 >> v2:172.21.15.191:3300/0 conn(0x7fba64157870 0x7fba6415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:33.811 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.809+0000 7fba6a91c700 1 -- 172.21.15.99:0/3656927895 >> v2:172.21.15.191:3300/0 conn(0x7fba64157870 msgr2=0x7fba6415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:33.811 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.809+0000 7fba6a91c700 1 --2- 172.21.15.99:0/3656927895 >> v2:172.21.15.191:3300/0 conn(0x7fba64157870 0x7fba6415b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:33.811 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.809+0000 7fba6a91c700 1 -- 172.21.15.99:0/3656927895 >> v1:172.21.15.191:6789/0 conn(0x7fba6415a620 legacy=0x7fba6415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:33.811 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.809+0000 7fba6a91c700 1 -- 172.21.15.99:0/3656927895 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba6415c560 con 0x7fba64157c40 2023-10-08T00:21:33.812 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.809+0000 7fba6a91c700 1 --2- 172.21.15.99:0/3656927895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba64157c40 0x7fba64158110 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fba64069d80 tx=0x7fba5c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da64646f006d4597 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:33.812 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.809+0000 7fba69119700 1 -- 172.21.15.99:0/3656927895 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5c009e70 con 0x7fba64157c40 2023-10-08T00:21:33.812 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.809+0000 7fba69119700 1 -- 172.21.15.99:0/3656927895 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba5c00b860 con 0x7fba64157c40 2023-10-08T00:21:33.812 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.809+0000 7fba69119700 1 -- 172.21.15.99:0/3656927895 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5c00bc00 con 0x7fba64157c40 2023-10-08T00:21:33.812 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.810+0000 7fba6c37f700 1 -- 172.21.15.99:0/3656927895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba64157c40 msgr2=0x7fba64158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:33.812 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.810+0000 7fba6c37f700 1 --2- 172.21.15.99:0/3656927895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba64157c40 0x7fba64158110 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fba64069d80 tx=0x7fba5c009250 comp rx=0 tx=0).stop 2023-10-08T00:21:33.812 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.810+0000 7fba6c37f700 1 -- 172.21.15.99:0/3656927895 shutdown_connections 2023-10-08T00:21:33.812 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.810+0000 7fba6c37f700 1 --2- 172.21.15.99:0/3656927895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba64157c40 0x7fba64158110 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:33.812 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.810+0000 7fba6c37f700 1 --2- 172.21.15.99:0/3656927895 >> v2:172.21.15.191:3300/0 conn(0x7fba64157870 0x7fba6415b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:33.812 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.810+0000 7fba6c37f700 1 -- 172.21.15.99:0/3656927895 >> 172.21.15.99:0/3656927895 conn(0x7fba64068ce0 msgr2=0x7fba640690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:33.813 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.810+0000 7fba6c37f700 1 -- 172.21.15.99:0/3656927895 shutdown_connections 2023-10-08T00:21:33.813 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.810+0000 7fba6c37f700 1 -- 172.21.15.99:0/3656927895 wait complete. 2023-10-08T00:21:33.816 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.811+0000 7fba6c37f700 1 Processor -- start 2023-10-08T00:21:33.816 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.811+0000 7fba6c37f700 1 -- start start 2023-10-08T00:21:33.816 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.812+0000 7fba6c37f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba64157870 0x7fba6414db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:33.816 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.812+0000 7fba6c37f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba64157c40 0x7fba640b2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:33.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.812+0000 7fba6c37f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba6415a620 0x7fba640b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:33.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.812+0000 7fba6c37f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba6414e070 con 0x7fba64157870 2023-10-08T00:21:33.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.812+0000 7fba6c37f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba6414efb0 con 0x7fba64157c40 2023-10-08T00:21:33.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.812+0000 7fba6c37f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba6414e6c0 con 0x7fba6415a620 2023-10-08T00:21:33.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.812+0000 7fba6a11b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba64157870 0x7fba6414db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:33.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.812+0000 7fba6a11b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba64157870 0x7fba6414db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51878/0 (socket says 172.21.15.99:51878) 2023-10-08T00:21:33.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.812+0000 7fba6a11b700 1 -- 172.21.15.99:0/120779609 learned_addr learned my addr 172.21.15.99:0/120779609 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:33.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.812+0000 7fba6991a700 1 --2- 172.21.15.99:0/120779609 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba64157c40 0x7fba640b2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:33.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.812+0000 7fba6a91c700 1 --2- 172.21.15.99:0/120779609 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba6415a620 0x7fba640b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:33.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.812+0000 7fba6a11b700 1 -- 172.21.15.99:0/120779609 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba6415a620 msgr2=0x7fba640b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:33.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.812+0000 7fba6a11b700 1 --2- 172.21.15.99:0/120779609 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba6415a620 0x7fba640b2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:33.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.812+0000 7fba6a11b700 1 -- 172.21.15.99:0/120779609 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba64157c40 msgr2=0x7fba640b2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:33.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.812+0000 7fba6a11b700 1 --2- 172.21.15.99:0/120779609 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba64157c40 0x7fba640b2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:33.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.812+0000 7fba6a11b700 1 -- 172.21.15.99:0/120779609 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba5c0094f0 con 0x7fba64157870 2023-10-08T00:21:33.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.812+0000 7fba6a11b700 1 --2- 172.21.15.99:0/120779609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba64157870 0x7fba6414db30 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fba60009ef0 tx=0x7fba6000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:33.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.812+0000 7fba5a7fc700 1 -- 172.21.15.99:0/120779609 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6000cb70 con 0x7fba64157870 2023-10-08T00:21:33.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.812+0000 7fba5a7fc700 1 -- 172.21.15.99:0/120779609 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba6000f040 con 0x7fba64157870 2023-10-08T00:21:33.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.812+0000 7fba5a7fc700 1 -- 172.21.15.99:0/120779609 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba60014480 con 0x7fba64157870 2023-10-08T00:21:33.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.812+0000 7fba6c37f700 1 -- 172.21.15.99:0/120779609 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba6414e960 con 0x7fba64157870 2023-10-08T00:21:33.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.812+0000 7fba6c37f700 1 -- 172.21.15.99:0/120779609 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba6416b570 con 0x7fba64157870 2023-10-08T00:21:33.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.813+0000 7fba6c37f700 1 -- 172.21.15.99:0/120779609 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba6404c3c0 con 0x7fba64157870 2023-10-08T00:21:33.819 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.814+0000 7fba5a7fc700 1 -- 172.21.15.99:0/120779609 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fba600147f0 con 0x7fba64157870 2023-10-08T00:21:33.819 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.814+0000 7fba5a7fc700 1 --2- 172.21.15.99:0/120779609 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fba48066f30 0x7fba480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:33.819 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.814+0000 7fba5a7fc700 1 -- 172.21.15.99:0/120779609 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fba600850e0 con 0x7fba64157870 2023-10-08T00:21:33.819 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.814+0000 7fba6991a700 1 --2- 172.21.15.99:0/120779609 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fba48066f30 0x7fba480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:33.819 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.815+0000 7fba6991a700 1 --2- 172.21.15.99:0/120779609 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fba48066f30 0x7fba480693e0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fba6415af00 tx=0x7fba54007ea0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:33.819 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:33.818+0000 7fba5a7fc700 1 -- 172.21.15.99:0/120779609 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fba60052350 con 0x7fba64157870 2023-10-08T00:21:34.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.002+0000 7fba6c37f700 1 -- 172.21.15.99:0/120779609 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fba6415c840 con 0x7fba64157870 2023-10-08T00:21:34.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.002+0000 7fba5a7fc700 1 -- 172.21.15.99:0/120779609 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v184) v1 ==== 74+0+15365 (secure 0 0 0) 0x7fba60055970 con 0x7fba64157870 2023-10-08T00:21:34.003 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:21:34.003 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":184,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:21:31.285769+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:21:27.984574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-08T00:21:29.244620+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:21:34.005 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.004+0000 7fba6c37f700 1 -- 172.21.15.99:0/120779609 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fba48066f30 msgr2=0x7fba480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:34.005 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.004+0000 7fba6c37f700 1 --2- 172.21.15.99:0/120779609 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fba48066f30 0x7fba480693e0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fba6415af00 tx=0x7fba54007ea0 comp rx=0 tx=0).stop 2023-10-08T00:21:34.005 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.004+0000 7fba6c37f700 1 -- 172.21.15.99:0/120779609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba64157870 msgr2=0x7fba6414db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:34.005 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.004+0000 7fba6c37f700 1 --2- 172.21.15.99:0/120779609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba64157870 0x7fba6414db30 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fba60009ef0 tx=0x7fba6000b040 comp rx=0 tx=0).stop 2023-10-08T00:21:34.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.004+0000 7fba6c37f700 1 -- 172.21.15.99:0/120779609 shutdown_connections 2023-10-08T00:21:34.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.004+0000 7fba6c37f700 1 --2- 172.21.15.99:0/120779609 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fba48066f30 0x7fba480693e0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:34.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.004+0000 7fba6c37f700 1 --2- 172.21.15.99:0/120779609 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba6415a620 0x7fba640b2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:34.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.004+0000 7fba6c37f700 1 --2- 172.21.15.99:0/120779609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba64157870 0x7fba6414db30 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:34.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.004+0000 7fba6c37f700 1 --2- 172.21.15.99:0/120779609 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba64157c40 0x7fba640b2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:34.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.004+0000 7fba6c37f700 1 -- 172.21.15.99:0/120779609 >> 172.21.15.99:0/120779609 conn(0x7fba64068ce0 msgr2=0x7fba64159860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:34.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.004+0000 7fba6c37f700 1 -- 172.21.15.99:0/120779609 shutdown_connections 2023-10-08T00:21:34.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.004+0000 7fba6c37f700 1 -- 172.21.15.99:0/120779609 wait complete. 2023-10-08T00:21:34.017 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T00:21:34.018 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T00:21:34.131 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.130+0000 7fb3f7c1b700 1 Processor -- start 2023-10-08T00:21:34.131 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.130+0000 7fb3f7c1b700 1 -- start start 2023-10-08T00:21:34.133 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.131+0000 7fb3f7c1b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb3f015e540 0x7fb3f0070700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:34.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.131+0000 7fb3f7c1b700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb3f0070c40 0x7fb3f006ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:34.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.131+0000 7fb3f7c1b700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3f0071080 con 0x7fb3f00afff0 2023-10-08T00:21:34.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.131+0000 7fb3f7c1b700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3f015e9b0 con 0x7fb3f0070c40 2023-10-08T00:21:34.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.131+0000 7fb3f7c1b700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3f00b03c0 con 0x7fb3f015e540 2023-10-08T00:21:34.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.131+0000 7fb3f61b8700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb3f0070c40 0x7fb3f006ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:34.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.131+0000 7fb3f59b7700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fb3f00afff0 0x7fb3f0069e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:57634/0 (socket says 172.21.15.99:57634) 2023-10-08T00:21:34.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.131+0000 7fb3f59b7700 1 -- 172.21.15.99:0/3023090799 learned_addr learned my addr 172.21.15.99:0/3023090799 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:21:34.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.131+0000 7fb3f51b6700 1 --2- 172.21.15.99:0/3023090799 >> v2:172.21.15.191:3300/0 conn(0x7fb3f015e540 0x7fb3f0070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:34.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.132+0000 7fb3f49b5700 1 -- 172.21.15.99:0/3023090799 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 623947859 0 0) 0x7fb3f0071080 con 0x7fb3f00afff0 2023-10-08T00:21:34.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.132+0000 7fb3f49b5700 1 -- 172.21.15.99:0/3023090799 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3d8003570 con 0x7fb3f00afff0 2023-10-08T00:21:34.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.132+0000 7fb3f61b8700 1 -- 172.21.15.99:0/3023090799 >> v2:172.21.15.191:3300/0 conn(0x7fb3f015e540 msgr2=0x7fb3f0070700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:34.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.132+0000 7fb3f61b8700 1 --2- 172.21.15.99:0/3023090799 >> v2:172.21.15.191:3300/0 conn(0x7fb3f015e540 0x7fb3f0070700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:34.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.132+0000 7fb3f61b8700 1 -- 172.21.15.99:0/3023090799 >> v1:172.21.15.99:6789/0 conn(0x7fb3f00afff0 legacy=0x7fb3f0069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:34.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.132+0000 7fb3f61b8700 1 -- 172.21.15.99:0/3023090799 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f006a870 con 0x7fb3f0070c40 2023-10-08T00:21:34.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.132+0000 7fb3f61b8700 1 --2- 172.21.15.99:0/3023090799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3f0070c40 0x7fb3f006ea90 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fb3f00ceb60 tx=0x7fb3e80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=825b945c0b6647da server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:34.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.132+0000 7fb3f49b5700 1 -- 172.21.15.99:0/3023090799 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e8011630 con 0x7fb3f0070c40 2023-10-08T00:21:34.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.132+0000 7fb3f49b5700 1 -- 172.21.15.99:0/3023090799 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3e80117b0 con 0x7fb3f0070c40 2023-10-08T00:21:34.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.132+0000 7fb3f49b5700 1 -- 172.21.15.99:0/3023090799 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e8011b70 con 0x7fb3f0070c40 2023-10-08T00:21:34.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.132+0000 7fb3f7c1b700 1 -- 172.21.15.99:0/3023090799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3f0070c40 msgr2=0x7fb3f006ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:34.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.132+0000 7fb3f7c1b700 1 --2- 172.21.15.99:0/3023090799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3f0070c40 0x7fb3f006ea90 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fb3f00ceb60 tx=0x7fb3e80091a0 comp rx=0 tx=0).stop 2023-10-08T00:21:34.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.132+0000 7fb3f7c1b700 1 -- 172.21.15.99:0/3023090799 shutdown_connections 2023-10-08T00:21:34.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.132+0000 7fb3f7c1b700 1 --2- 172.21.15.99:0/3023090799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3f0070c40 0x7fb3f006ea90 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:34.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.132+0000 7fb3f7c1b700 1 --2- 172.21.15.99:0/3023090799 >> v2:172.21.15.191:3300/0 conn(0x7fb3f015e540 0x7fb3f0070700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:34.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.132+0000 7fb3f7c1b700 1 -- 172.21.15.99:0/3023090799 >> 172.21.15.99:0/3023090799 conn(0x7fb3f00cda50 msgr2=0x7fb3f00cde50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:34.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.133+0000 7fb3f7c1b700 1 -- 172.21.15.99:0/3023090799 shutdown_connections 2023-10-08T00:21:34.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.133+0000 7fb3f7c1b700 1 -- 172.21.15.99:0/3023090799 wait complete. 2023-10-08T00:21:34.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.134+0000 7fb3f7c1b700 1 Processor -- start 2023-10-08T00:21:34.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.134+0000 7fb3f7c1b700 1 -- start start 2023-10-08T00:21:34.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.134+0000 7fb3f7c1b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3f0070c40 0x7fb3f0158030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:34.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.134+0000 7fb3f7c1b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3f00afff0 0x7fb3f00b0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:34.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.134+0000 7fb3f7c1b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3f015e540 0x7fb3f00b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:34.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.134+0000 7fb3f7c1b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f00c42c0 con 0x7fb3f015e540 2023-10-08T00:21:34.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.134+0000 7fb3f7c1b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f00b3930 con 0x7fb3f0070c40 2023-10-08T00:21:34.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.134+0000 7fb3f7c1b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f00b34a0 con 0x7fb3f00afff0 2023-10-08T00:21:34.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.134+0000 7fb3f61b8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3f015e540 0x7fb3f00b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:34.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.134+0000 7fb3f61b8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3f015e540 0x7fb3f00b0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51904/0 (socket says 172.21.15.99:51904) 2023-10-08T00:21:34.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.134+0000 7fb3f61b8700 1 -- 172.21.15.99:0/4110279798 learned_addr learned my addr 172.21.15.99:0/4110279798 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:34.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.135+0000 7fb3f61b8700 1 -- 172.21.15.99:0/4110279798 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3f00afff0 msgr2=0x7fb3f00b0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:34.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.135+0000 7fb3f61b8700 1 --2- 172.21.15.99:0/4110279798 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3f00afff0 0x7fb3f00b0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:34.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.135+0000 7fb3f61b8700 1 -- 172.21.15.99:0/4110279798 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3f0070c40 msgr2=0x7fb3f0158030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:34.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.135+0000 7fb3f61b8700 1 --2- 172.21.15.99:0/4110279798 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3f0070c40 0x7fb3f0158030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:34.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.135+0000 7fb3f61b8700 1 -- 172.21.15.99:0/4110279798 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e8009420 con 0x7fb3f015e540 2023-10-08T00:21:34.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.135+0000 7fb3f61b8700 1 --2- 172.21.15.99:0/4110279798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3f015e540 0x7fb3f00b0e40 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8013000 tx=0x7fb3e8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:34.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.135+0000 7fb3edffb700 1 -- 172.21.15.99:0/4110279798 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e801f070 con 0x7fb3f015e540 2023-10-08T00:21:34.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.135+0000 7fb3edffb700 1 -- 172.21.15.99:0/4110279798 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3e8011920 con 0x7fb3f015e540 2023-10-08T00:21:34.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.135+0000 7fb3f7c1b700 1 -- 172.21.15.99:0/4110279798 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3f00b2890 con 0x7fb3f015e540 2023-10-08T00:21:34.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.135+0000 7fb3edffb700 1 -- 172.21.15.99:0/4110279798 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e8025420 con 0x7fb3f015e540 2023-10-08T00:21:34.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.135+0000 7fb3f7c1b700 1 -- 172.21.15.99:0/4110279798 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3f00b2e50 con 0x7fb3f015e540 2023-10-08T00:21:34.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.136+0000 7fb3f7c1b700 1 -- 172.21.15.99:0/4110279798 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3f004c3c0 con 0x7fb3f015e540 2023-10-08T00:21:34.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.137+0000 7fb3edffb700 1 -- 172.21.15.99:0/4110279798 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb3e8023070 con 0x7fb3f015e540 2023-10-08T00:21:34.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.137+0000 7fb3edffb700 1 --2- 172.21.15.99:0/4110279798 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb3dc066f30 0x7fb3dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:34.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.137+0000 7fb3edffb700 1 -- 172.21.15.99:0/4110279798 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb3e8085b40 con 0x7fb3f015e540 2023-10-08T00:21:34.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.137+0000 7fb3f59b7700 1 --2- 172.21.15.99:0/4110279798 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb3dc066f30 0x7fb3dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:34.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.138+0000 7fb3f59b7700 1 --2- 172.21.15.99:0/4110279798 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb3dc066f30 0x7fb3dc0693e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0005c50 tx=0x7fb3e00016e0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:34.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.141+0000 7fb3edffb700 1 -- 172.21.15.99:0/4110279798 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb3e8052db0 con 0x7fb3f015e540 2023-10-08T00:21:34.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.320+0000 7fb3f7c1b700 1 -- 172.21.15.99:0/4110279798 --> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb3f00b3610 con 0x7fb3dc066f30 2023-10-08T00:21:34.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.329+0000 7fb3edffb700 1 -- 172.21.15.99:0/4110279798 <== mgr.4099 v2:172.21.15.191:6832/85985 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285561 (secure 0 0 0) 0x7fb3f00b3610 con 0x7fb3dc066f30 2023-10-08T00:21:34.333 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:21:34.335 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.333+0000 7fb3f7c1b700 1 -- 172.21.15.99:0/4110279798 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb3dc066f30 msgr2=0x7fb3dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:34.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.333+0000 7fb3f7c1b700 1 --2- 172.21.15.99:0/4110279798 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb3dc066f30 0x7fb3dc0693e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0005c50 tx=0x7fb3e00016e0 comp rx=0 tx=0).stop 2023-10-08T00:21:34.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.333+0000 7fb3f7c1b700 1 -- 172.21.15.99:0/4110279798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3f015e540 msgr2=0x7fb3f00b0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:34.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.333+0000 7fb3f7c1b700 1 --2- 172.21.15.99:0/4110279798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3f015e540 0x7fb3f00b0e40 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8013000 tx=0x7fb3e8017000 comp rx=0 tx=0).stop 2023-10-08T00:21:34.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.333+0000 7fb3f7c1b700 1 -- 172.21.15.99:0/4110279798 shutdown_connections 2023-10-08T00:21:34.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.333+0000 7fb3f7c1b700 1 --2- 172.21.15.99:0/4110279798 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb3dc066f30 0x7fb3dc0693e0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:34.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.333+0000 7fb3f7c1b700 1 --2- 172.21.15.99:0/4110279798 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3f0070c40 0x7fb3f0158030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:34.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.333+0000 7fb3f7c1b700 1 --2- 172.21.15.99:0/4110279798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3f015e540 0x7fb3f00b0e40 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:34.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.333+0000 7fb3f7c1b700 1 --2- 172.21.15.99:0/4110279798 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3f00afff0 0x7fb3f00b0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:34.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.333+0000 7fb3f7c1b700 1 -- 172.21.15.99:0/4110279798 >> 172.21.15.99:0/4110279798 conn(0x7fb3f00cda50 msgr2=0x7fb3f006ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:34.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.333+0000 7fb3f7c1b700 1 -- 172.21.15.99:0/4110279798 shutdown_connections 2023-10-08T00:21:34.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.333+0000 7fb3f7c1b700 1 -- 172.21.15.99:0/4110279798 wait complete. 2023-10-08T00:21:34.336 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2023-10-08T00:21:34.345 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":500,"stamp":"2023-10-08T00:21:34.176301+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":174,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9228,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1553271,"statfs":{"total":1600000000,"available":1590550060,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.895534"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.301479+0000","last_change":"2023-10-08T00:21:30.301213+0000","last_active":"2023-10-08T00:21:30.301479+0000","last_peered":"2023-10-08T00:21:30.301479+0000","last_clean":"2023-10-08T00:21:30.301479+0000","last_became_active":"2023-10-08T00:21:30.300255+0000","last_became_peered":"2023-10-08T00:21:30.300255+0000","last_unstale":"2023-10-08T00:21:30.301479+0000","last_undegraded":"2023-10-08T00:21:30.301479+0000","last_fullsized":"2023-10-08T00:21:30.301479+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:58:05.480523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.300383+0000","last_change":"2023-10-08T00:21:30.298767+0000","last_active":"2023-10-08T00:21:30.300383+0000","last_peered":"2023-10-08T00:21:30.300383+0000","last_clean":"2023-10-08T00:21:30.300383+0000","last_became_active":"2023-10-08T00:21:30.298534+0000","last_became_peered":"2023-10-08T00:21:30.298534+0000","last_unstale":"2023-10-08T00:21:30.300383+0000","last_undegraded":"2023-10-08T00:21:30.300383+0000","last_fullsized":"2023-10-08T00:21:30.300383+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:54:08.470147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.298015+0000","last_change":"2023-10-08T00:21:30.297676+0000","last_active":"2023-10-08T00:21:30.298015+0000","last_peered":"2023-10-08T00:21:30.298015+0000","last_clean":"2023-10-08T00:21:30.298015+0000","last_became_active":"2023-10-08T00:21:30.296962+0000","last_became_peered":"2023-10-08T00:21:30.296962+0000","last_unstale":"2023-10-08T00:21:30.298015+0000","last_undegraded":"2023-10-08T00:21:30.298015+0000","last_fullsized":"2023-10-08T00:21:30.298015+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:35:52.651092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.294926+0000","last_change":"2023-10-08T00:21:30.294697+0000","last_active":"2023-10-08T00:21:30.294926+0000","last_peered":"2023-10-08T00:21:30.294926+0000","last_clean":"2023-10-08T00:21:30.294926+0000","last_became_active":"2023-10-08T00:21:30.294198+0000","last_became_peered":"2023-10-08T00:21:30.294198+0000","last_unstale":"2023-10-08T00:21:30.294926+0000","last_undegraded":"2023-10-08T00:21:30.294926+0000","last_fullsized":"2023-10-08T00:21:30.294926+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:23:46.629646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.300708+0000","last_change":"2023-10-08T00:21:30.300583+0000","last_active":"2023-10-08T00:21:30.300708+0000","last_peered":"2023-10-08T00:21:30.300708+0000","last_clean":"2023-10-08T00:21:30.300708+0000","last_became_active":"2023-10-08T00:21:30.299505+0000","last_became_peered":"2023-10-08T00:21:30.299505+0000","last_unstale":"2023-10-08T00:21:30.300708+0000","last_undegraded":"2023-10-08T00:21:30.300708+0000","last_fullsized":"2023-10-08T00:21:30.300708+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:19:07.335627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.291203+0000","last_change":"2023-10-08T00:21:30.291130+0000","last_active":"2023-10-08T00:21:30.291203+0000","last_peered":"2023-10-08T00:21:30.291203+0000","last_clean":"2023-10-08T00:21:30.291203+0000","last_became_active":"2023-10-08T00:21:30.290758+0000","last_became_peered":"2023-10-08T00:21:30.290758+0000","last_unstale":"2023-10-08T00:21:30.291203+0000","last_undegraded":"2023-10-08T00:21:30.291203+0000","last_fullsized":"2023-10-08T00:21:30.291203+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:52:23.286803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.300075+0000","last_change":"2023-10-08T00:21:30.300030+0000","last_active":"2023-10-08T00:21:30.300075+0000","last_peered":"2023-10-08T00:21:30.300075+0000","last_clean":"2023-10-08T00:21:30.300075+0000","last_became_active":"2023-10-08T00:21:30.299622+0000","last_became_peered":"2023-10-08T00:21:30.299622+0000","last_unstale":"2023-10-08T00:21:30.300075+0000","last_undegraded":"2023-10-08T00:21:30.300075+0000","last_fullsized":"2023-10-08T00:21:30.300075+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:22:50.229264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.296060+0000","last_change":"2023-10-08T00:21:30.295676+0000","last_active":"2023-10-08T00:21:30.296060+0000","last_peered":"2023-10-08T00:21:30.296060+0000","last_clean":"2023-10-08T00:21:30.296060+0000","last_became_active":"2023-10-08T00:21:30.295426+0000","last_became_peered":"2023-10-08T00:21:30.295426+0000","last_unstale":"2023-10-08T00:21:30.296060+0000","last_undegraded":"2023-10-08T00:21:30.296060+0000","last_fullsized":"2023-10-08T00:21:30.296060+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:23:46.629646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.301071+0000","last_change":"2023-10-08T00:21:30.300677+0000","last_active":"2023-10-08T00:21:30.301071+0000","last_peered":"2023-10-08T00:21:30.301071+0000","last_clean":"2023-10-08T00:21:30.301071+0000","last_became_active":"2023-10-08T00:21:30.300290+0000","last_became_peered":"2023-10-08T00:21:30.300290+0000","last_unstale":"2023-10-08T00:21:30.301071+0000","last_undegraded":"2023-10-08T00:21:30.301071+0000","last_fullsized":"2023-10-08T00:21:30.301071+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:58:05.480523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.301087+0000","last_change":"2023-10-08T00:21:30.300861+0000","last_active":"2023-10-08T00:21:30.301087+0000","last_peered":"2023-10-08T00:21:30.301087+0000","last_clean":"2023-10-08T00:21:30.301087+0000","last_became_active":"2023-10-08T00:21:30.300258+0000","last_became_peered":"2023-10-08T00:21:30.300258+0000","last_unstale":"2023-10-08T00:21:30.301087+0000","last_undegraded":"2023-10-08T00:21:30.301087+0000","last_fullsized":"2023-10-08T00:21:30.301087+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:37:52.274322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.300511+0000","last_change":"2023-10-08T00:21:30.300452+0000","last_active":"2023-10-08T00:21:30.300511+0000","last_peered":"2023-10-08T00:21:30.300511+0000","last_clean":"2023-10-08T00:21:30.300511+0000","last_became_active":"2023-10-08T00:21:30.300241+0000","last_became_peered":"2023-10-08T00:21:30.300241+0000","last_unstale":"2023-10-08T00:21:30.300511+0000","last_undegraded":"2023-10-08T00:21:30.300511+0000","last_fullsized":"2023-10-08T00:21:30.300511+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:28:01.264300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.297770+0000","last_change":"2023-10-08T00:21:30.297575+0000","last_active":"2023-10-08T00:21:30.297770+0000","last_peered":"2023-10-08T00:21:30.297770+0000","last_clean":"2023-10-08T00:21:30.297770+0000","last_became_active":"2023-10-08T00:21:30.297366+0000","last_became_peered":"2023-10-08T00:21:30.297366+0000","last_unstale":"2023-10-08T00:21:30.297770+0000","last_undegraded":"2023-10-08T00:21:30.297770+0000","last_fullsized":"2023-10-08T00:21:30.297770+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:18:35.800650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.294244+0000","last_change":"2023-10-08T00:21:30.294177+0000","last_active":"2023-10-08T00:21:30.294244+0000","last_peered":"2023-10-08T00:21:30.294244+0000","last_clean":"2023-10-08T00:21:30.294244+0000","last_became_active":"2023-10-08T00:21:30.293676+0000","last_became_peered":"2023-10-08T00:21:30.293676+0000","last_unstale":"2023-10-08T00:21:30.294244+0000","last_undegraded":"2023-10-08T00:21:30.294244+0000","last_fullsized":"2023-10-08T00:21:30.294244+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:00:46.701778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.299097+0000","last_change":"2023-10-08T00:21:30.299036+0000","last_active":"2023-10-08T00:21:30.299097+0000","last_peered":"2023-10-08T00:21:30.299097+0000","last_clean":"2023-10-08T00:21:30.299097+0000","last_became_active":"2023-10-08T00:21:30.298590+0000","last_became_peered":"2023-10-08T00:21:30.298590+0000","last_unstale":"2023-10-08T00:21:30.299097+0000","last_undegraded":"2023-10-08T00:21:30.299097+0000","last_fullsized":"2023-10-08T00:21:30.299097+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:50:06.095667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.301101+0000","last_change":"2023-10-08T00:21:30.300996+0000","last_active":"2023-10-08T00:21:30.301101+0000","last_peered":"2023-10-08T00:21:30.301101+0000","last_clean":"2023-10-08T00:21:30.301101+0000","last_became_active":"2023-10-08T00:21:30.300543+0000","last_became_peered":"2023-10-08T00:21:30.300543+0000","last_unstale":"2023-10-08T00:21:30.301101+0000","last_undegraded":"2023-10-08T00:21:30.301101+0000","last_fullsized":"2023-10-08T00:21:30.301101+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:03:08.518439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.301726+0000","last_change":"2023-10-08T00:21:30.301665+0000","last_active":"2023-10-08T00:21:30.301726+0000","last_peered":"2023-10-08T00:21:30.301726+0000","last_clean":"2023-10-08T00:21:30.301726+0000","last_became_active":"2023-10-08T00:21:30.301105+0000","last_became_peered":"2023-10-08T00:21:30.301105+0000","last_unstale":"2023-10-08T00:21:30.301726+0000","last_undegraded":"2023-10-08T00:21:30.301726+0000","last_fullsized":"2023-10-08T00:21:30.301726+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:57:33.670768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.295610+0000","last_change":"2023-10-08T00:21:30.295547+0000","last_active":"2023-10-08T00:21:30.295610+0000","last_peered":"2023-10-08T00:21:30.295610+0000","last_clean":"2023-10-08T00:21:30.295610+0000","last_became_active":"2023-10-08T00:21:30.295307+0000","last_became_peered":"2023-10-08T00:21:30.295307+0000","last_unstale":"2023-10-08T00:21:30.295610+0000","last_undegraded":"2023-10-08T00:21:30.295610+0000","last_fullsized":"2023-10-08T00:21:30.295610+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:07:17.295368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.295325+0000","last_change":"2023-10-08T00:21:30.295227+0000","last_active":"2023-10-08T00:21:30.295325+0000","last_peered":"2023-10-08T00:21:30.295325+0000","last_clean":"2023-10-08T00:21:30.295325+0000","last_became_active":"2023-10-08T00:21:30.295014+0000","last_became_peered":"2023-10-08T00:21:30.295014+0000","last_unstale":"2023-10-08T00:21:30.295325+0000","last_undegraded":"2023-10-08T00:21:30.295325+0000","last_fullsized":"2023-10-08T00:21:30.295325+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:02:23.529002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.302574+0000","last_change":"2023-10-08T00:21:30.302299+0000","last_active":"2023-10-08T00:21:30.302574+0000","last_peered":"2023-10-08T00:21:30.302574+0000","last_clean":"2023-10-08T00:21:30.302574+0000","last_became_active":"2023-10-08T00:21:30.302040+0000","last_became_peered":"2023-10-08T00:21:30.302040+0000","last_unstale":"2023-10-08T00:21:30.302574+0000","last_undegraded":"2023-10-08T00:21:30.302574+0000","last_fullsized":"2023-10-08T00:21:30.302574+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:23:46.629646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.299851+0000","last_change":"2023-10-08T00:21:30.299794+0000","last_active":"2023-10-08T00:21:30.299851+0000","last_peered":"2023-10-08T00:21:30.299851+0000","last_clean":"2023-10-08T00:21:30.299851+0000","last_became_active":"2023-10-08T00:21:30.299513+0000","last_became_peered":"2023-10-08T00:21:30.299513+0000","last_unstale":"2023-10-08T00:21:30.299851+0000","last_undegraded":"2023-10-08T00:21:30.299851+0000","last_fullsized":"2023-10-08T00:21:30.299851+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:50:10.281370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.298911+0000","last_change":"2023-10-08T00:21:30.298696+0000","last_active":"2023-10-08T00:21:30.298911+0000","last_peered":"2023-10-08T00:21:30.298911+0000","last_clean":"2023-10-08T00:21:30.298911+0000","last_became_active":"2023-10-08T00:21:30.297906+0000","last_became_peered":"2023-10-08T00:21:30.297906+0000","last_unstale":"2023-10-08T00:21:30.298911+0000","last_undegraded":"2023-10-08T00:21:30.298911+0000","last_fullsized":"2023-10-08T00:21:30.298911+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:30:32.148405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.296539+0000","last_change":"2023-10-08T00:21:30.296493+0000","last_active":"2023-10-08T00:21:30.296539+0000","last_peered":"2023-10-08T00:21:30.296539+0000","last_clean":"2023-10-08T00:21:30.296539+0000","last_became_active":"2023-10-08T00:21:30.296289+0000","last_became_peered":"2023-10-08T00:21:30.296289+0000","last_unstale":"2023-10-08T00:21:30.296539+0000","last_undegraded":"2023-10-08T00:21:30.296539+0000","last_fullsized":"2023-10-08T00:21:30.296539+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:13:04.243694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.298738+0000","last_change":"2023-10-08T00:21:30.298359+0000","last_active":"2023-10-08T00:21:30.298738+0000","last_peered":"2023-10-08T00:21:30.298738+0000","last_clean":"2023-10-08T00:21:30.298738+0000","last_became_active":"2023-10-08T00:21:30.297873+0000","last_became_peered":"2023-10-08T00:21:30.297873+0000","last_unstale":"2023-10-08T00:21:30.298738+0000","last_undegraded":"2023-10-08T00:21:30.298738+0000","last_fullsized":"2023-10-08T00:21:30.298738+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:03:08.518439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.296205+0000","last_change":"2023-10-08T00:21:30.295993+0000","last_active":"2023-10-08T00:21:30.296205+0000","last_peered":"2023-10-08T00:21:30.296205+0000","last_clean":"2023-10-08T00:21:30.296205+0000","last_became_active":"2023-10-08T00:21:30.295770+0000","last_became_peered":"2023-10-08T00:21:30.295770+0000","last_unstale":"2023-10-08T00:21:30.296205+0000","last_undegraded":"2023-10-08T00:21:30.296205+0000","last_fullsized":"2023-10-08T00:21:30.296205+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:49:32.812313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.301446+0000","last_change":"2023-10-08T00:21:30.301389+0000","last_active":"2023-10-08T00:21:30.301446+0000","last_peered":"2023-10-08T00:21:30.301446+0000","last_clean":"2023-10-08T00:21:30.301446+0000","last_became_active":"2023-10-08T00:21:30.301134+0000","last_became_peered":"2023-10-08T00:21:30.301134+0000","last_unstale":"2023-10-08T00:21:30.301446+0000","last_undegraded":"2023-10-08T00:21:30.301446+0000","last_fullsized":"2023-10-08T00:21:30.301446+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:27:00.784747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.290852+0000","last_change":"2023-10-08T00:21:30.290053+0000","last_active":"2023-10-08T00:21:30.290852+0000","last_peered":"2023-10-08T00:21:30.290852+0000","last_clean":"2023-10-08T00:21:30.290852+0000","last_became_active":"2023-10-08T00:21:30.289833+0000","last_became_peered":"2023-10-08T00:21:30.289833+0000","last_unstale":"2023-10-08T00:21:30.290852+0000","last_undegraded":"2023-10-08T00:21:30.290852+0000","last_fullsized":"2023-10-08T00:21:30.290852+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:58:05.480523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.297985+0000","last_change":"2023-10-08T00:21:30.297919+0000","last_active":"2023-10-08T00:21:30.297985+0000","last_peered":"2023-10-08T00:21:30.297985+0000","last_clean":"2023-10-08T00:21:30.297985+0000","last_became_active":"2023-10-08T00:21:30.297688+0000","last_became_peered":"2023-10-08T00:21:30.297688+0000","last_unstale":"2023-10-08T00:21:30.297985+0000","last_undegraded":"2023-10-08T00:21:30.297985+0000","last_fullsized":"2023-10-08T00:21:30.297985+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:11:22.573459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.300555+0000","last_change":"2023-10-08T00:21:30.300492+0000","last_active":"2023-10-08T00:21:30.300555+0000","last_peered":"2023-10-08T00:21:30.300555+0000","last_clean":"2023-10-08T00:21:30.300555+0000","last_became_active":"2023-10-08T00:21:30.300255+0000","last_became_peered":"2023-10-08T00:21:30.300255+0000","last_unstale":"2023-10-08T00:21:30.300555+0000","last_undegraded":"2023-10-08T00:21:30.300555+0000","last_fullsized":"2023-10-08T00:21:30.300555+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:50:06.095667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.297998+0000","last_change":"2023-10-08T00:21:30.297029+0000","last_active":"2023-10-08T00:21:30.297998+0000","last_peered":"2023-10-08T00:21:30.297998+0000","last_clean":"2023-10-08T00:21:30.297998+0000","last_became_active":"2023-10-08T00:21:30.296798+0000","last_became_peered":"2023-10-08T00:21:30.296798+0000","last_unstale":"2023-10-08T00:21:30.297998+0000","last_undegraded":"2023-10-08T00:21:30.297998+0000","last_fullsized":"2023-10-08T00:21:30.297998+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:39:05.416412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.297885+0000","last_change":"2023-10-08T00:21:30.297774+0000","last_active":"2023-10-08T00:21:30.297885+0000","last_peered":"2023-10-08T00:21:30.297885+0000","last_clean":"2023-10-08T00:21:30.297885+0000","last_became_active":"2023-10-08T00:21:30.297378+0000","last_became_peered":"2023-10-08T00:21:30.297378+0000","last_unstale":"2023-10-08T00:21:30.297885+0000","last_undegraded":"2023-10-08T00:21:30.297885+0000","last_fullsized":"2023-10-08T00:21:30.297885+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:57:33.670768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.301460+0000","last_change":"2023-10-08T00:21:30.301052+0000","last_active":"2023-10-08T00:21:30.301460+0000","last_peered":"2023-10-08T00:21:30.301460+0000","last_clean":"2023-10-08T00:21:30.301460+0000","last_became_active":"2023-10-08T00:21:30.299794+0000","last_became_peered":"2023-10-08T00:21:30.299794+0000","last_unstale":"2023-10-08T00:21:30.301460+0000","last_undegraded":"2023-10-08T00:21:30.301460+0000","last_fullsized":"2023-10-08T00:21:30.301460+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:32:18.516618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.294301+0000","last_change":"2023-10-08T00:21:30.294049+0000","last_active":"2023-10-08T00:21:30.294301+0000","last_peered":"2023-10-08T00:21:30.294301+0000","last_clean":"2023-10-08T00:21:30.294301+0000","last_became_active":"2023-10-08T00:21:30.293372+0000","last_became_peered":"2023-10-08T00:21:30.293372+0000","last_unstale":"2023-10-08T00:21:30.294301+0000","last_undegraded":"2023-10-08T00:21:30.294301+0000","last_fullsized":"2023-10-08T00:21:30.294301+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:57:54.792143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.294943+0000","last_change":"2023-10-08T00:21:30.294867+0000","last_active":"2023-10-08T00:21:30.294943+0000","last_peered":"2023-10-08T00:21:30.294943+0000","last_clean":"2023-10-08T00:21:30.294943+0000","last_became_active":"2023-10-08T00:21:30.294476+0000","last_became_peered":"2023-10-08T00:21:30.294476+0000","last_unstale":"2023-10-08T00:21:30.294943+0000","last_undegraded":"2023-10-08T00:21:30.294943+0000","last_fullsized":"2023-10-08T00:21:30.294943+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:58:05.480523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1f","version":"183'4","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.311301+0000","last_change":"2023-10-08T00:21:29.297829+0000","last_active":"2023-10-08T00:21:30.311301+0000","last_peered":"2023-10-08T00:21:30.311301+0000","last_clean":"2023-10-08T00:21:30.311301+0000","last_became_active":"2023-10-08T00:21:29.297497+0000","last_became_peered":"2023-10-08T00:21:29.297497+0000","last_unstale":"2023-10-08T00:21:30.311301+0000","last_undegraded":"2023-10-08T00:21:30.311301+0000","last_fullsized":"2023-10-08T00:21:30.311301+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:20:48.787420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.301510+0000","last_change":"2023-10-08T00:21:30.301405+0000","last_active":"2023-10-08T00:21:30.301510+0000","last_peered":"2023-10-08T00:21:30.301510+0000","last_clean":"2023-10-08T00:21:30.301510+0000","last_became_active":"2023-10-08T00:21:30.301116+0000","last_became_peered":"2023-10-08T00:21:30.301116+0000","last_unstale":"2023-10-08T00:21:30.301510+0000","last_undegraded":"2023-10-08T00:21:30.301510+0000","last_fullsized":"2023-10-08T00:21:30.301510+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:36:42.424132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1e","version":"183'1","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.310359+0000","last_change":"2023-10-08T00:21:29.297676+0000","last_active":"2023-10-08T00:21:30.310359+0000","last_peered":"2023-10-08T00:21:30.310359+0000","last_clean":"2023-10-08T00:21:30.310359+0000","last_became_active":"2023-10-08T00:21:29.297452+0000","last_became_peered":"2023-10-08T00:21:29.297452+0000","last_unstale":"2023-10-08T00:21:30.310359+0000","last_undegraded":"2023-10-08T00:21:30.310359+0000","last_fullsized":"2023-10-08T00:21:30.310359+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:38:37.634900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.298891+0000","last_change":"2023-10-08T00:21:30.298562+0000","last_active":"2023-10-08T00:21:30.298891+0000","last_peered":"2023-10-08T00:21:30.298891+0000","last_clean":"2023-10-08T00:21:30.298891+0000","last_became_active":"2023-10-08T00:21:30.297599+0000","last_became_peered":"2023-10-08T00:21:30.297599+0000","last_unstale":"2023-10-08T00:21:30.298891+0000","last_undegraded":"2023-10-08T00:21:30.298891+0000","last_fullsized":"2023-10-08T00:21:30.298891+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:53:37.006424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1d","version":"183'2","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.311359+0000","last_change":"2023-10-08T00:21:29.292594+0000","last_active":"2023-10-08T00:21:30.311359+0000","last_peered":"2023-10-08T00:21:30.311359+0000","last_clean":"2023-10-08T00:21:30.311359+0000","last_became_active":"2023-10-08T00:21:29.292296+0000","last_became_peered":"2023-10-08T00:21:29.292296+0000","last_unstale":"2023-10-08T00:21:30.311359+0000","last_undegraded":"2023-10-08T00:21:30.311359+0000","last_fullsized":"2023-10-08T00:21:30.311359+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:20:31.873834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.297793+0000","last_change":"2023-10-08T00:21:30.297650+0000","last_active":"2023-10-08T00:21:30.297793+0000","last_peered":"2023-10-08T00:21:30.297793+0000","last_clean":"2023-10-08T00:21:30.297793+0000","last_became_active":"2023-10-08T00:21:30.297091+0000","last_became_peered":"2023-10-08T00:21:30.297091+0000","last_unstale":"2023-10-08T00:21:30.297793+0000","last_undegraded":"2023-10-08T00:21:30.297793+0000","last_fullsized":"2023-10-08T00:21:30.297793+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:33:45.741583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1c","version":"183'2","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.310855+0000","last_change":"2023-10-08T00:21:29.296156+0000","last_active":"2023-10-08T00:21:30.310855+0000","last_peered":"2023-10-08T00:21:30.310855+0000","last_clean":"2023-10-08T00:21:30.310855+0000","last_became_active":"2023-10-08T00:21:29.295887+0000","last_became_peered":"2023-10-08T00:21:29.295887+0000","last_unstale":"2023-10-08T00:21:30.310855+0000","last_undegraded":"2023-10-08T00:21:30.310855+0000","last_fullsized":"2023-10-08T00:21:30.310855+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:53:36.001232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.294453+0000","last_change":"2023-10-08T00:21:30.294381+0000","last_active":"2023-10-08T00:21:30.294453+0000","last_peered":"2023-10-08T00:21:30.294453+0000","last_clean":"2023-10-08T00:21:30.294453+0000","last_became_active":"2023-10-08T00:21:30.294133+0000","last_became_peered":"2023-10-08T00:21:30.294133+0000","last_unstale":"2023-10-08T00:21:30.294453+0000","last_undegraded":"2023-10-08T00:21:30.294453+0000","last_fullsized":"2023-10-08T00:21:30.294453+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:17:21.382612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1b","version":"183'1","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.309034+0000","last_change":"2023-10-08T00:21:29.295395+0000","last_active":"2023-10-08T00:21:30.309034+0000","last_peered":"2023-10-08T00:21:30.309034+0000","last_clean":"2023-10-08T00:21:30.309034+0000","last_became_active":"2023-10-08T00:21:29.295143+0000","last_became_peered":"2023-10-08T00:21:29.295143+0000","last_unstale":"2023-10-08T00:21:30.309034+0000","last_undegraded":"2023-10-08T00:21:30.309034+0000","last_fullsized":"2023-10-08T00:21:30.309034+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:30:31.143213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.294316+0000","last_change":"2023-10-08T00:21:30.294179+0000","last_active":"2023-10-08T00:21:30.294316+0000","last_peered":"2023-10-08T00:21:30.294316+0000","last_clean":"2023-10-08T00:21:30.294316+0000","last_became_active":"2023-10-08T00:21:30.293680+0000","last_became_peered":"2023-10-08T00:21:30.293680+0000","last_unstale":"2023-10-08T00:21:30.294316+0000","last_undegraded":"2023-10-08T00:21:30.294316+0000","last_fullsized":"2023-10-08T00:21:30.294316+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:22:07.112692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1a","version":"183'1","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.309769+0000","last_change":"2023-10-08T00:21:29.297064+0000","last_active":"2023-10-08T00:21:30.309769+0000","last_peered":"2023-10-08T00:21:30.309769+0000","last_clean":"2023-10-08T00:21:30.309769+0000","last_became_active":"2023-10-08T00:21:29.296879+0000","last_became_peered":"2023-10-08T00:21:29.296879+0000","last_unstale":"2023-10-08T00:21:30.309769+0000","last_undegraded":"2023-10-08T00:21:30.309769+0000","last_fullsized":"2023-10-08T00:21:30.309769+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:53:36.001232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.297446+0000","last_change":"2023-10-08T00:21:30.297319+0000","last_active":"2023-10-08T00:21:30.297446+0000","last_peered":"2023-10-08T00:21:30.297446+0000","last_clean":"2023-10-08T00:21:30.297446+0000","last_became_active":"2023-10-08T00:21:30.297046+0000","last_became_peered":"2023-10-08T00:21:30.297046+0000","last_unstale":"2023-10-08T00:21:30.297446+0000","last_undegraded":"2023-10-08T00:21:30.297446+0000","last_fullsized":"2023-10-08T00:21:30.297446+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:37:52.274322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.291722+0000","last_change":"2023-10-08T00:21:29.295415+0000","last_active":"2023-10-08T00:21:30.291722+0000","last_peered":"2023-10-08T00:21:30.291722+0000","last_clean":"2023-10-08T00:21:30.291722+0000","last_became_active":"2023-10-08T00:21:29.295148+0000","last_became_peered":"2023-10-08T00:21:29.295148+0000","last_unstale":"2023-10-08T00:21:30.291722+0000","last_undegraded":"2023-10-08T00:21:30.291722+0000","last_fullsized":"2023-10-08T00:21:30.291722+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:57:53.786951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.299156+0000","last_change":"2023-10-08T00:21:30.299041+0000","last_active":"2023-10-08T00:21:30.299156+0000","last_peered":"2023-10-08T00:21:30.299156+0000","last_clean":"2023-10-08T00:21:30.299156+0000","last_became_active":"2023-10-08T00:21:30.297008+0000","last_became_peered":"2023-10-08T00:21:30.297008+0000","last_unstale":"2023-10-08T00:21:30.299156+0000","last_undegraded":"2023-10-08T00:21:30.299156+0000","last_fullsized":"2023-10-08T00:21:30.299156+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:50:06.095667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.289152+0000","last_change":"2023-10-08T00:21:29.297815+0000","last_active":"2023-10-08T00:21:30.289152+0000","last_peered":"2023-10-08T00:21:30.289152+0000","last_clean":"2023-10-08T00:21:30.289152+0000","last_became_active":"2023-10-08T00:21:29.297569+0000","last_became_peered":"2023-10-08T00:21:29.297569+0000","last_unstale":"2023-10-08T00:21:30.289152+0000","last_undegraded":"2023-10-08T00:21:30.289152+0000","last_fullsized":"2023-10-08T00:21:30.289152+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:52:22.281611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.301421+0000","last_change":"2023-10-08T00:21:30.301354+0000","last_active":"2023-10-08T00:21:30.301421+0000","last_peered":"2023-10-08T00:21:30.301421+0000","last_clean":"2023-10-08T00:21:30.301421+0000","last_became_active":"2023-10-08T00:21:30.300360+0000","last_became_peered":"2023-10-08T00:21:30.300360+0000","last_unstale":"2023-10-08T00:21:30.301421+0000","last_undegraded":"2023-10-08T00:21:30.301421+0000","last_fullsized":"2023-10-08T00:21:30.301421+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:37:52.274322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.288991+0000","last_change":"2023-10-08T00:21:29.289499+0000","last_active":"2023-10-08T00:21:30.288991+0000","last_peered":"2023-10-08T00:21:30.288991+0000","last_clean":"2023-10-08T00:21:30.288991+0000","last_became_active":"2023-10-08T00:21:29.289122+0000","last_became_peered":"2023-10-08T00:21:29.289122+0000","last_unstale":"2023-10-08T00:21:30.288991+0000","last_undegraded":"2023-10-08T00:21:30.288991+0000","last_fullsized":"2023-10-08T00:21:30.288991+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:03:44.571431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.301024+0000","last_change":"2023-10-08T00:21:30.300939+0000","last_active":"2023-10-08T00:21:30.301024+0000","last_peered":"2023-10-08T00:21:30.301024+0000","last_clean":"2023-10-08T00:21:30.301024+0000","last_became_active":"2023-10-08T00:21:30.300559+0000","last_became_peered":"2023-10-08T00:21:30.300559+0000","last_unstale":"2023-10-08T00:21:30.301024+0000","last_undegraded":"2023-10-08T00:21:30.301024+0000","last_fullsized":"2023-10-08T00:21:30.301024+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:28:39.097586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.15","version":"183'3","reported_seq":13,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.311637+0000","last_change":"2023-10-08T00:21:29.294220+0000","last_active":"2023-10-08T00:21:30.311637+0000","last_peered":"2023-10-08T00:21:30.311637+0000","last_clean":"2023-10-08T00:21:30.311637+0000","last_became_active":"2023-10-08T00:21:29.293317+0000","last_became_peered":"2023-10-08T00:21:29.293317+0000","last_unstale":"2023-10-08T00:21:30.311637+0000","last_undegraded":"2023-10-08T00:21:30.311637+0000","last_fullsized":"2023-10-08T00:21:30.311637+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:30:31.143213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.298030+0000","last_change":"2023-10-08T00:21:30.297806+0000","last_active":"2023-10-08T00:21:30.298030+0000","last_peered":"2023-10-08T00:21:30.298030+0000","last_clean":"2023-10-08T00:21:30.298030+0000","last_became_active":"2023-10-08T00:21:30.297213+0000","last_became_peered":"2023-10-08T00:21:30.297213+0000","last_unstale":"2023-10-08T00:21:30.298030+0000","last_undegraded":"2023-10-08T00:21:30.298030+0000","last_fullsized":"2023-10-08T00:21:30.298030+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:26:51.687638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.14","version":"183'6","reported_seq":16,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.312148+0000","last_change":"2023-10-08T00:21:29.295488+0000","last_active":"2023-10-08T00:21:30.312148+0000","last_peered":"2023-10-08T00:21:30.312148+0000","last_clean":"2023-10-08T00:21:30.312148+0000","last_became_active":"2023-10-08T00:21:29.295261+0000","last_became_peered":"2023-10-08T00:21:29.295261+0000","last_unstale":"2023-10-08T00:21:30.312148+0000","last_undegraded":"2023-10-08T00:21:30.312148+0000","last_fullsized":"2023-10-08T00:21:30.312148+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:28:14.454425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.301657+0000","last_change":"2023-10-08T00:21:30.301516+0000","last_active":"2023-10-08T00:21:30.301657+0000","last_peered":"2023-10-08T00:21:30.301657+0000","last_clean":"2023-10-08T00:21:30.301657+0000","last_became_active":"2023-10-08T00:21:30.300667+0000","last_became_peered":"2023-10-08T00:21:30.300667+0000","last_unstale":"2023-10-08T00:21:30.301657+0000","last_undegraded":"2023-10-08T00:21:30.301657+0000","last_fullsized":"2023-10-08T00:21:30.301657+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:54:48.645640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"183'2","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.310210+0000","last_change":"2023-10-08T00:21:29.292595+0000","last_active":"2023-10-08T00:21:30.310210+0000","last_peered":"2023-10-08T00:21:30.310210+0000","last_clean":"2023-10-08T00:21:30.310210+0000","last_became_active":"2023-10-08T00:21:29.291651+0000","last_became_peered":"2023-10-08T00:21:29.291651+0000","last_unstale":"2023-10-08T00:21:30.310210+0000","last_undegraded":"2023-10-08T00:21:30.310210+0000","last_fullsized":"2023-10-08T00:21:30.310210+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:38:37.634900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.302603+0000","last_change":"2023-10-08T00:21:30.302539+0000","last_active":"2023-10-08T00:21:30.302603+0000","last_peered":"2023-10-08T00:21:30.302603+0000","last_clean":"2023-10-08T00:21:30.302603+0000","last_became_active":"2023-10-08T00:21:30.301646+0000","last_became_peered":"2023-10-08T00:21:30.301646+0000","last_unstale":"2023-10-08T00:21:30.302603+0000","last_undegraded":"2023-10-08T00:21:30.302603+0000","last_fullsized":"2023-10-08T00:21:30.302603+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:50:06.095667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.288423+0000","last_change":"2023-10-08T00:21:29.294183+0000","last_active":"2023-10-08T00:21:30.288423+0000","last_peered":"2023-10-08T00:21:30.288423+0000","last_clean":"2023-10-08T00:21:30.288423+0000","last_became_active":"2023-10-08T00:21:29.293884+0000","last_became_peered":"2023-10-08T00:21:29.293884+0000","last_unstale":"2023-10-08T00:21:30.288423+0000","last_undegraded":"2023-10-08T00:21:30.288423+0000","last_fullsized":"2023-10-08T00:21:30.288423+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:33:45.707571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.297045+0000","last_change":"2023-10-08T00:21:30.296980+0000","last_active":"2023-10-08T00:21:30.297045+0000","last_peered":"2023-10-08T00:21:30.297045+0000","last_clean":"2023-10-08T00:21:30.297045+0000","last_became_active":"2023-10-08T00:21:30.296750+0000","last_became_peered":"2023-10-08T00:21:30.296750+0000","last_unstale":"2023-10-08T00:21:30.297045+0000","last_undegraded":"2023-10-08T00:21:30.297045+0000","last_fullsized":"2023-10-08T00:21:30.297045+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:20:49.792611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.293820+0000","last_change":"2023-10-08T00:21:29.298869+0000","last_active":"2023-10-08T00:21:30.293820+0000","last_peered":"2023-10-08T00:21:30.293820+0000","last_clean":"2023-10-08T00:21:30.293820+0000","last_became_active":"2023-10-08T00:21:29.298202+0000","last_became_peered":"2023-10-08T00:21:29.298202+0000","last_unstale":"2023-10-08T00:21:30.293820+0000","last_undegraded":"2023-10-08T00:21:30.293820+0000","last_fullsized":"2023-10-08T00:21:30.293820+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:19:05.281936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.297776+0000","last_change":"2023-10-08T00:21:30.297591+0000","last_active":"2023-10-08T00:21:30.297776+0000","last_peered":"2023-10-08T00:21:30.297776+0000","last_clean":"2023-10-08T00:21:30.297776+0000","last_became_active":"2023-10-08T00:21:30.297276+0000","last_became_peered":"2023-10-08T00:21:30.297276+0000","last_unstale":"2023-10-08T00:21:30.297776+0000","last_undegraded":"2023-10-08T00:21:30.297776+0000","last_fullsized":"2023-10-08T00:21:30.297776+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:54:48.645640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.10","version":"183'8","reported_seq":18,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.311573+0000","last_change":"2023-10-08T00:21:29.294644+0000","last_active":"2023-10-08T00:21:30.311573+0000","last_peered":"2023-10-08T00:21:30.311573+0000","last_clean":"2023-10-08T00:21:30.311573+0000","last_became_active":"2023-10-08T00:21:29.294431+0000","last_became_peered":"2023-10-08T00:21:29.294431+0000","last_unstale":"2023-10-08T00:21:30.311573+0000","last_undegraded":"2023-10-08T00:21:30.311573+0000","last_fullsized":"2023-10-08T00:21:30.311573+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:17:38.121868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.301362+0000","last_change":"2023-10-08T00:21:30.301296+0000","last_active":"2023-10-08T00:21:30.301362+0000","last_peered":"2023-10-08T00:21:30.301362+0000","last_clean":"2023-10-08T00:21:30.301362+0000","last_became_active":"2023-10-08T00:21:30.300732+0000","last_became_peered":"2023-10-08T00:21:30.300732+0000","last_unstale":"2023-10-08T00:21:30.301362+0000","last_undegraded":"2023-10-08T00:21:30.301362+0000","last_fullsized":"2023-10-08T00:21:30.301362+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:33:45.741583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.f","version":"183'1","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.310865+0000","last_change":"2023-10-08T00:21:29.293776+0000","last_active":"2023-10-08T00:21:30.310865+0000","last_peered":"2023-10-08T00:21:30.310865+0000","last_clean":"2023-10-08T00:21:30.310865+0000","last_became_active":"2023-10-08T00:21:29.293558+0000","last_became_peered":"2023-10-08T00:21:29.293558+0000","last_unstale":"2023-10-08T00:21:30.310865+0000","last_undegraded":"2023-10-08T00:21:30.310865+0000","last_fullsized":"2023-10-08T00:21:30.310865+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:52:58.669407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.299733+0000","last_change":"2023-10-08T00:21:30.299678+0000","last_active":"2023-10-08T00:21:30.299733+0000","last_peered":"2023-10-08T00:21:30.299733+0000","last_clean":"2023-10-08T00:21:30.299733+0000","last_became_active":"2023-10-08T00:21:30.299306+0000","last_became_peered":"2023-10-08T00:21:30.299306+0000","last_unstale":"2023-10-08T00:21:30.299733+0000","last_undegraded":"2023-10-08T00:21:30.299733+0000","last_fullsized":"2023-10-08T00:21:30.299733+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:40:21.777719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.e","version":"183'2","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.311273+0000","last_change":"2023-10-08T00:21:29.292720+0000","last_active":"2023-10-08T00:21:30.311273+0000","last_peered":"2023-10-08T00:21:30.311273+0000","last_clean":"2023-10-08T00:21:30.311273+0000","last_became_active":"2023-10-08T00:21:29.292302+0000","last_became_peered":"2023-10-08T00:21:29.292302+0000","last_unstale":"2023-10-08T00:21:30.311273+0000","last_undegraded":"2023-10-08T00:21:30.311273+0000","last_fullsized":"2023-10-08T00:21:30.311273+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:17:05.951588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.302912+0000","last_change":"2023-10-08T00:21:30.302743+0000","last_active":"2023-10-08T00:21:30.302912+0000","last_peered":"2023-10-08T00:21:30.302912+0000","last_clean":"2023-10-08T00:21:30.302912+0000","last_became_active":"2023-10-08T00:21:30.302174+0000","last_became_peered":"2023-10-08T00:21:30.302174+0000","last_unstale":"2023-10-08T00:21:30.302912+0000","last_undegraded":"2023-10-08T00:21:30.302912+0000","last_fullsized":"2023-10-08T00:21:30.302912+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:03:08.518439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.d","version":"183'1","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.310070+0000","last_change":"2023-10-08T00:21:29.292311+0000","last_active":"2023-10-08T00:21:30.310070+0000","last_peered":"2023-10-08T00:21:30.310070+0000","last_clean":"2023-10-08T00:21:30.310070+0000","last_became_active":"2023-10-08T00:21:29.292012+0000","last_became_peered":"2023-10-08T00:21:29.292012+0000","last_unstale":"2023-10-08T00:21:30.310070+0000","last_undegraded":"2023-10-08T00:21:30.310070+0000","last_fullsized":"2023-10-08T00:21:30.310070+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:58:24.760650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.298768+0000","last_change":"2023-10-08T00:21:30.298598+0000","last_active":"2023-10-08T00:21:30.298768+0000","last_peered":"2023-10-08T00:21:30.298768+0000","last_clean":"2023-10-08T00:21:30.298768+0000","last_became_active":"2023-10-08T00:21:30.297864+0000","last_became_peered":"2023-10-08T00:21:30.297864+0000","last_unstale":"2023-10-08T00:21:30.298768+0000","last_undegraded":"2023-10-08T00:21:30.298768+0000","last_fullsized":"2023-10-08T00:21:30.298768+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:23:46.629646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.293948+0000","last_change":"2023-10-08T00:21:29.295959+0000","last_active":"2023-10-08T00:21:30.293948+0000","last_peered":"2023-10-08T00:21:30.293948+0000","last_clean":"2023-10-08T00:21:30.293948+0000","last_became_active":"2023-10-08T00:21:29.295743+0000","last_became_peered":"2023-10-08T00:21:29.295743+0000","last_unstale":"2023-10-08T00:21:30.293948+0000","last_undegraded":"2023-10-08T00:21:30.293948+0000","last_fullsized":"2023-10-08T00:21:30.293948+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:45:30.775065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.301681+0000","last_change":"2023-10-08T00:21:30.301603+0000","last_active":"2023-10-08T00:21:30.301681+0000","last_peered":"2023-10-08T00:21:30.301681+0000","last_clean":"2023-10-08T00:21:30.301681+0000","last_became_active":"2023-10-08T00:21:30.301299+0000","last_became_peered":"2023-10-08T00:21:30.301299+0000","last_unstale":"2023-10-08T00:21:30.301681+0000","last_undegraded":"2023-10-08T00:21:30.301681+0000","last_fullsized":"2023-10-08T00:21:30.301681+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:04:05.010208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.b","version":"183'2","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.310692+0000","last_change":"2023-10-08T00:21:29.297803+0000","last_active":"2023-10-08T00:21:30.310692+0000","last_peered":"2023-10-08T00:21:30.310692+0000","last_clean":"2023-10-08T00:21:30.310692+0000","last_became_active":"2023-10-08T00:21:29.297560+0000","last_became_peered":"2023-10-08T00:21:29.297560+0000","last_unstale":"2023-10-08T00:21:30.310692+0000","last_undegraded":"2023-10-08T00:21:30.310692+0000","last_fullsized":"2023-10-08T00:21:30.310692+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:17:05.951588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.298426+0000","last_change":"2023-10-08T00:21:30.298192+0000","last_active":"2023-10-08T00:21:30.298426+0000","last_peered":"2023-10-08T00:21:30.298426+0000","last_clean":"2023-10-08T00:21:30.298426+0000","last_became_active":"2023-10-08T00:21:30.297166+0000","last_became_peered":"2023-10-08T00:21:30.297166+0000","last_unstale":"2023-10-08T00:21:30.298426+0000","last_undegraded":"2023-10-08T00:21:30.298426+0000","last_fullsized":"2023-10-08T00:21:30.298426+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:02:23.529002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.291935+0000","last_change":"2023-10-08T00:21:29.292681+0000","last_active":"2023-10-08T00:21:30.291935+0000","last_peered":"2023-10-08T00:21:30.291935+0000","last_clean":"2023-10-08T00:21:30.291935+0000","last_became_active":"2023-10-08T00:21:29.291478+0000","last_became_peered":"2023-10-08T00:21:29.291478+0000","last_unstale":"2023-10-08T00:21:30.291935+0000","last_undegraded":"2023-10-08T00:21:30.291935+0000","last_fullsized":"2023-10-08T00:21:30.291935+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:36:41.035968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.301495+0000","last_change":"2023-10-08T00:21:30.301347+0000","last_active":"2023-10-08T00:21:30.301495+0000","last_peered":"2023-10-08T00:21:30.301495+0000","last_clean":"2023-10-08T00:21:30.301495+0000","last_became_active":"2023-10-08T00:21:30.300552+0000","last_became_peered":"2023-10-08T00:21:30.300552+0000","last_unstale":"2023-10-08T00:21:30.301495+0000","last_undegraded":"2023-10-08T00:21:30.301495+0000","last_fullsized":"2023-10-08T00:21:30.301495+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:22:50.229264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.290336+0000","last_change":"2023-10-08T00:21:29.292747+0000","last_active":"2023-10-08T00:21:30.290336+0000","last_peered":"2023-10-08T00:21:30.290336+0000","last_clean":"2023-10-08T00:21:30.290336+0000","last_became_active":"2023-10-08T00:21:29.292039+0000","last_became_peered":"2023-10-08T00:21:29.292039+0000","last_unstale":"2023-10-08T00:21:30.290336+0000","last_undegraded":"2023-10-08T00:21:30.290336+0000","last_fullsized":"2023-10-08T00:21:30.290336+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:17:05.951588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.301612+0000","last_change":"2023-10-08T00:21:30.301419+0000","last_active":"2023-10-08T00:21:30.301612+0000","last_peered":"2023-10-08T00:21:30.301612+0000","last_clean":"2023-10-08T00:21:30.301612+0000","last_became_active":"2023-10-08T00:21:30.301056+0000","last_became_peered":"2023-10-08T00:21:30.301056+0000","last_unstale":"2023-10-08T00:21:30.301612+0000","last_undegraded":"2023-10-08T00:21:30.301612+0000","last_fullsized":"2023-10-08T00:21:30.301612+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:25:58.325029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.293597+0000","last_change":"2023-10-08T00:21:29.293623+0000","last_active":"2023-10-08T00:21:30.293597+0000","last_peered":"2023-10-08T00:21:30.293597+0000","last_clean":"2023-10-08T00:21:30.293597+0000","last_became_active":"2023-10-08T00:21:29.293216+0000","last_became_peered":"2023-10-08T00:21:29.293216+0000","last_unstale":"2023-10-08T00:21:30.293597+0000","last_undegraded":"2023-10-08T00:21:30.293597+0000","last_fullsized":"2023-10-08T00:21:30.293597+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:26:38.276733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.292110+0000","last_change":"2023-10-08T00:21:30.292044+0000","last_active":"2023-10-08T00:21:30.292110+0000","last_peered":"2023-10-08T00:21:30.292110+0000","last_clean":"2023-10-08T00:21:30.292110+0000","last_became_active":"2023-10-08T00:21:30.291731+0000","last_became_peered":"2023-10-08T00:21:30.291731+0000","last_unstale":"2023-10-08T00:21:30.292110+0000","last_undegraded":"2023-10-08T00:21:30.292110+0000","last_fullsized":"2023-10-08T00:21:30.292110+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:33:45.741583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.7","version":"183'2","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.309329+0000","last_change":"2023-10-08T00:21:29.289352+0000","last_active":"2023-10-08T00:21:30.309329+0000","last_peered":"2023-10-08T00:21:30.309329+0000","last_clean":"2023-10-08T00:21:30.309329+0000","last_became_active":"2023-10-08T00:21:29.288764+0000","last_became_peered":"2023-10-08T00:21:29.288764+0000","last_unstale":"2023-10-08T00:21:30.309329+0000","last_undegraded":"2023-10-08T00:21:30.309329+0000","last_fullsized":"2023-10-08T00:21:30.309329+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:42:06.385422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.302689+0000","last_change":"2023-10-08T00:21:30.302628+0000","last_active":"2023-10-08T00:21:30.302689+0000","last_peered":"2023-10-08T00:21:30.302689+0000","last_clean":"2023-10-08T00:21:30.302689+0000","last_became_active":"2023-10-08T00:21:30.298293+0000","last_became_peered":"2023-10-08T00:21:30.298293+0000","last_unstale":"2023-10-08T00:21:30.302689+0000","last_undegraded":"2023-10-08T00:21:30.302689+0000","last_fullsized":"2023-10-08T00:21:30.302689+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:33:45.741583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.293442+0000","last_change":"2023-10-08T00:21:29.297973+0000","last_active":"2023-10-08T00:21:30.293442+0000","last_peered":"2023-10-08T00:21:30.293442+0000","last_clean":"2023-10-08T00:21:30.293442+0000","last_became_active":"2023-10-08T00:21:29.297747+0000","last_became_peered":"2023-10-08T00:21:29.297747+0000","last_unstale":"2023-10-08T00:21:30.293442+0000","last_undegraded":"2023-10-08T00:21:30.293442+0000","last_fullsized":"2023-10-08T00:21:30.293442+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:09:10.597389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.298875+0000","last_change":"2023-10-08T00:21:30.297642+0000","last_active":"2023-10-08T00:21:30.298875+0000","last_peered":"2023-10-08T00:21:30.298875+0000","last_clean":"2023-10-08T00:21:30.298875+0000","last_became_active":"2023-10-08T00:21:30.297410+0000","last_became_peered":"2023-10-08T00:21:30.297410+0000","last_unstale":"2023-10-08T00:21:30.298875+0000","last_undegraded":"2023-10-08T00:21:30.298875+0000","last_fullsized":"2023-10-08T00:21:30.298875+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:54:48.645640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.291816+0000","last_change":"2023-10-08T00:21:29.290597+0000","last_active":"2023-10-08T00:21:30.291816+0000","last_peered":"2023-10-08T00:21:30.291816+0000","last_clean":"2023-10-08T00:21:30.291816+0000","last_became_active":"2023-10-08T00:21:29.290091+0000","last_became_peered":"2023-10-08T00:21:29.290091+0000","last_unstale":"2023-10-08T00:21:30.291816+0000","last_undegraded":"2023-10-08T00:21:30.291816+0000","last_fullsized":"2023-10-08T00:21:30.291816+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:22:06.107501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.300190+0000","last_change":"2023-10-08T00:21:30.300125+0000","last_active":"2023-10-08T00:21:30.300190+0000","last_peered":"2023-10-08T00:21:30.300190+0000","last_clean":"2023-10-08T00:21:30.300190+0000","last_became_active":"2023-10-08T00:21:30.299898+0000","last_became_peered":"2023-10-08T00:21:30.299898+0000","last_unstale":"2023-10-08T00:21:30.300190+0000","last_undegraded":"2023-10-08T00:21:30.300190+0000","last_fullsized":"2023-10-08T00:21:30.300190+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:22:50.229264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.289248+0000","last_change":"2023-10-08T00:21:29.295528+0000","last_active":"2023-10-08T00:21:30.289248+0000","last_peered":"2023-10-08T00:21:30.289248+0000","last_clean":"2023-10-08T00:21:30.289248+0000","last_became_active":"2023-10-08T00:21:29.295272+0000","last_became_peered":"2023-10-08T00:21:29.295272+0000","last_unstale":"2023-10-08T00:21:30.289248+0000","last_undegraded":"2023-10-08T00:21:30.289248+0000","last_fullsized":"2023-10-08T00:21:30.289248+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:20:48.787420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.294281+0000","last_change":"2023-10-08T00:21:30.293908+0000","last_active":"2023-10-08T00:21:30.294281+0000","last_peered":"2023-10-08T00:21:30.294281+0000","last_clean":"2023-10-08T00:21:30.294281+0000","last_became_active":"2023-10-08T00:21:30.293049+0000","last_became_peered":"2023-10-08T00:21:30.293049+0000","last_unstale":"2023-10-08T00:21:30.294281+0000","last_undegraded":"2023-10-08T00:21:30.294281+0000","last_fullsized":"2023-10-08T00:21:30.294281+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:17:06.956780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.296061+0000","last_change":"2023-10-08T00:21:29.292200+0000","last_active":"2023-10-08T00:21:30.296061+0000","last_peered":"2023-10-08T00:21:30.296061+0000","last_clean":"2023-10-08T00:21:30.296061+0000","last_became_active":"2023-10-08T00:21:29.291715+0000","last_became_peered":"2023-10-08T00:21:29.291715+0000","last_unstale":"2023-10-08T00:21:30.296061+0000","last_undegraded":"2023-10-08T00:21:30.296061+0000","last_fullsized":"2023-10-08T00:21:30.296061+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:57:21.248136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.302308+0000","last_change":"2023-10-08T00:21:30.302254+0000","last_active":"2023-10-08T00:21:30.302308+0000","last_peered":"2023-10-08T00:21:30.302308+0000","last_clean":"2023-10-08T00:21:30.302308+0000","last_became_active":"2023-10-08T00:21:30.302082+0000","last_became_peered":"2023-10-08T00:21:30.302082+0000","last_unstale":"2023-10-08T00:21:30.302308+0000","last_undegraded":"2023-10-08T00:21:30.302308+0000","last_fullsized":"2023-10-08T00:21:30.302308+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:02:23.529002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.293919+0000","last_change":"2023-10-08T00:21:29.298981+0000","last_active":"2023-10-08T00:21:30.293919+0000","last_peered":"2023-10-08T00:21:30.293919+0000","last_clean":"2023-10-08T00:21:30.293919+0000","last_became_active":"2023-10-08T00:21:29.298583+0000","last_became_peered":"2023-10-08T00:21:29.298583+0000","last_unstale":"2023-10-08T00:21:30.293919+0000","last_undegraded":"2023-10-08T00:21:30.293919+0000","last_fullsized":"2023-10-08T00:21:30.293919+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:44:14.951665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.300866+0000","last_change":"2023-10-08T00:21:30.300769+0000","last_active":"2023-10-08T00:21:30.300866+0000","last_peered":"2023-10-08T00:21:30.300866+0000","last_clean":"2023-10-08T00:21:30.300866+0000","last_became_active":"2023-10-08T00:21:30.300283+0000","last_became_peered":"2023-10-08T00:21:30.300283+0000","last_unstale":"2023-10-08T00:21:30.300866+0000","last_undegraded":"2023-10-08T00:21:30.300866+0000","last_fullsized":"2023-10-08T00:21:30.300866+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:02:23.529002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.294158+0000","last_change":"2023-10-08T00:21:29.297715+0000","last_active":"2023-10-08T00:21:30.294158+0000","last_peered":"2023-10-08T00:21:30.294158+0000","last_clean":"2023-10-08T00:21:30.294158+0000","last_became_active":"2023-10-08T00:21:29.297200+0000","last_became_peered":"2023-10-08T00:21:29.297200+0000","last_unstale":"2023-10-08T00:21:30.294158+0000","last_undegraded":"2023-10-08T00:21:30.294158+0000","last_fullsized":"2023-10-08T00:21:30.294158+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:17:05.951588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.298043+0000","last_change":"2023-10-08T00:21:30.297937+0000","last_active":"2023-10-08T00:21:30.298043+0000","last_peered":"2023-10-08T00:21:30.298043+0000","last_clean":"2023-10-08T00:21:30.298043+0000","last_became_active":"2023-10-08T00:21:30.297478+0000","last_became_peered":"2023-10-08T00:21:30.297478+0000","last_unstale":"2023-10-08T00:21:30.298043+0000","last_undegraded":"2023-10-08T00:21:30.298043+0000","last_fullsized":"2023-10-08T00:21:30.298043+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:18:43.290407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.291900+0000","last_change":"2023-10-08T00:21:29.294934+0000","last_active":"2023-10-08T00:21:30.291900+0000","last_peered":"2023-10-08T00:21:30.291900+0000","last_clean":"2023-10-08T00:21:30.291900+0000","last_became_active":"2023-10-08T00:21:29.290398+0000","last_became_peered":"2023-10-08T00:21:29.290398+0000","last_unstale":"2023-10-08T00:21:30.291900+0000","last_undegraded":"2023-10-08T00:21:30.291900+0000","last_fullsized":"2023-10-08T00:21:30.291900+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:07:16.290177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.299947+0000","last_change":"2023-10-08T00:21:30.299891+0000","last_active":"2023-10-08T00:21:30.299947+0000","last_peered":"2023-10-08T00:21:30.299947+0000","last_clean":"2023-10-08T00:21:30.299947+0000","last_became_active":"2023-10-08T00:21:30.298860+0000","last_became_peered":"2023-10-08T00:21:30.298860+0000","last_unstale":"2023-10-08T00:21:30.299947+0000","last_undegraded":"2023-10-08T00:21:30.299947+0000","last_fullsized":"2023-10-08T00:21:30.299947+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:40:21.777719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.294097+0000","last_change":"2023-10-08T00:21:29.298156+0000","last_active":"2023-10-08T00:21:30.294097+0000","last_peered":"2023-10-08T00:21:30.294097+0000","last_clean":"2023-10-08T00:21:30.294097+0000","last_became_active":"2023-10-08T00:21:29.297947+0000","last_became_peered":"2023-10-08T00:21:29.297947+0000","last_unstale":"2023-10-08T00:21:30.294097+0000","last_undegraded":"2023-10-08T00:21:30.294097+0000","last_fullsized":"2023-10-08T00:21:30.294097+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:34:18.411604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":304,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.293003+0000","last_change":"2023-10-08T00:10:43.728490+0000","last_active":"2023-10-08T00:21:30.293003+0000","last_peered":"2023-10-08T00:21:30.293003+0000","last_clean":"2023-10-08T00:21:30.293003+0000","last_became_active":"2023-10-08T00:10:43.728341+0000","last_became_peered":"2023-10-08T00:10:43.728341+0000","last_unstale":"2023-10-08T00:21:30.293003+0000","last_undegraded":"2023-10-08T00:21:30.293003+0000","last_fullsized":"2023-10-08T00:21:30.293003+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_clean_scrub_stamp":"2023-10-08T00:10:42.716096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:06:31.035293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574980,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1008,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194303,"statfs":{"total":200000000,"available":198966926,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52700000000000002,"15min":0.51800000000000002},"min":{"1min":0.373,"5min":0.27800000000000002,"15min":0.26900000000000002},"max":{"1min":0.69899999999999995,"5min":0.77600000000000002,"15min":1.129},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55100000000000005,"15min":0.54700000000000004},"min":{"1min":0.35199999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.754,"5min":0.754,"15min":1.0269999999999999},"last":0.73799999999999999}]},{"osd":1,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57599999999999996,"15min":0.55400000000000005},"min":{"1min":0.42799999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.73799999999999999,"5min":0.77900000000000003,"15min":0.82999999999999996},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.58999999999999997,"15min":0.57799999999999996},"min":{"1min":0.44,"5min":0.39300000000000002,"15min":0.38200000000000001},"max":{"1min":0.76200000000000001,"5min":0.79400000000000004,"15min":0.80100000000000005},"last":0.62}]},{"osd":2,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56299999999999994,"15min":0.54900000000000004},"min":{"1min":0.41999999999999998,"5min":0.38,"15min":0.34200000000000003},"max":{"1min":0.72599999999999998,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54700000000000004,"15min":0.53700000000000003},"min":{"1min":0.38,"5min":0.34499999999999997,"15min":0.32900000000000001},"max":{"1min":0.748,"5min":0.748,"15min":0.78500000000000003},"last":0.437}]},{"osd":3,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55500000000000005,"15min":0.53700000000000003},"min":{"1min":0.373,"5min":0.36699999999999999,"15min":0.33700000000000002},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.79300000000000004},"last":0.753},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56399999999999995},"min":{"1min":0.46999999999999997,"5min":0.371,"15min":0.371},"max":{"1min":0.71199999999999997,"5min":0.77600000000000002,"15min":0.82099999999999995},"last":0.65000000000000002}]},{"osd":4,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.47299999999999998,"15min":0.499},"min":{"1min":0.307,"5min":0.27200000000000002,"15min":0.26400000000000001},"max":{"1min":0.69999999999999996,"5min":0.755,"15min":0.755},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.50900000000000001,"15min":0.51500000000000001},"min":{"1min":0.33500000000000002,"5min":0.29199999999999998,"15min":0.28199999999999997},"max":{"1min":0.71599999999999997,"5min":0.77400000000000002,"15min":0.78900000000000003},"last":0.69899999999999995}]},{"osd":5,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.51000000000000001,"15min":0.52800000000000002},"min":{"1min":0.35699999999999998,"5min":0.27800000000000002,"15min":0.254},"max":{"1min":0.73499999999999999,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.54200000000000004,"15min":0.56100000000000005},"min":{"1min":0.38900000000000001,"5min":0.32400000000000001,"15min":0.29099999999999998},"max":{"1min":0.748,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.71599999999999997}]},{"osd":6,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.59399999999999997,"15min":0.59099999999999997},"min":{"1min":0.42099999999999999,"5min":0.39600000000000002,"15min":0.33100000000000002},"max":{"1min":0.81299999999999994,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.58699999999999997,"15min":0.60599999999999998},"min":{"1min":0.40899999999999997,"5min":0.33300000000000002,"15min":0.27800000000000002},"max":{"1min":0.78800000000000003,"5min":0.82999999999999996,"15min":0.91500000000000004},"last":0.82399999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574980,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1008,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194303,"statfs":{"total":200000000,"available":198966892,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53100000000000003,"15min":0.50600000000000001},"min":{"1min":0.42399999999999999,"5min":0.32300000000000001,"15min":0.29199999999999998},"max":{"1min":0.74299999999999999,"5min":0.76300000000000001,"15min":1.012},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.53500000000000003,"15min":0.60699999999999998},"min":{"1min":0.38400000000000001,"5min":0.317,"15min":0.29199999999999998},"max":{"1min":0.65800000000000003,"5min":0.77800000000000002,"15min":5.1230000000000002},"last":0.499}]},{"osd":1,"last update":"Sun Oct 8 00:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.55700000000000005,"15min":0.54000000000000004},"min":{"1min":0.36499999999999999,"5min":0.29899999999999999,"15min":0.28899999999999998},"max":{"1min":0.754,"5min":0.78100000000000003,"15min":1.1419999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.59399999999999997,"15min":0.57699999999999996},"min":{"1min":0.51300000000000001,"5min":0.41099999999999998,"15min":0.32100000000000001},"max":{"1min":0.81000000000000005,"5min":0.81799999999999995,"15min":1.1539999999999999},"last":0.51600000000000001}]},{"osd":2,"last update":"Sun Oct 8 00:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.55400000000000005,"15min":0.54200000000000004},"min":{"1min":0.47199999999999998,"5min":0.36299999999999999,"15min":0.32900000000000001},"max":{"1min":0.74099999999999999,"5min":0.76500000000000001,"15min":1.3149999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58299999999999996,"15min":0.57599999999999996},"min":{"1min":0.441,"5min":0.374,"15min":0.29399999999999998},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":1.3280000000000001},"last":0.64700000000000002}]},{"osd":3,"last update":"Sun Oct 8 00:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.56100000000000005,"15min":0.55400000000000005},"min":{"1min":0.35499999999999998,"5min":0.32000000000000001,"15min":0.25700000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":1.046},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.58099999999999996,"15min":0.56100000000000005},"min":{"1min":0.501,"5min":0.38600000000000001,"15min":0.32900000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":1.4390000000000001},"last":0.66300000000000003}]},{"osd":4,"last update":"Sun Oct 8 00:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46200000000000002,"15min":0.48099999999999998},"min":{"1min":0.30499999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.622,"5min":0.70099999999999996,"15min":0.80100000000000005},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55000000000000004,"15min":0.55700000000000005},"min":{"1min":0.41999999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.77600000000000002,"5min":0.79500000000000004,"15min":0.89100000000000001},"last":0.47499999999999998}]},{"osd":5,"last update":"Sun Oct 8 00:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.47699999999999998,"15min":0.48299999999999998},"min":{"1min":0.36099999999999999,"5min":0.29299999999999998,"15min":0.29099999999999998},"max":{"1min":0.73499999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.505,"15min":0.53200000000000003},"min":{"1min":0.35299999999999998,"5min":0.313,"15min":0.28699999999999998},"max":{"1min":0.70099999999999996,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.42699999999999999}]},{"osd":7,"last update":"Sun Oct 8 00:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.50600000000000001,"15min":0.52600000000000002},"min":{"1min":0.38100000000000001,"5min":0.32800000000000001,"15min":0.307},"max":{"1min":0.753,"5min":0.82899999999999996,"15min":0.91500000000000004},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53200000000000003,"15min":0.54200000000000004},"min":{"1min":0.39000000000000001,"5min":0.34899999999999998,"15min":0.32400000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.79900000000000004},"last":0.45700000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574980,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1587,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193725,"statfs":{"total":200000000,"available":198374840,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:20:54 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.35599999999999998,"15min":0.32900000000000001},"min":{"1min":0.24199999999999999,"5min":0.215,"15min":0.19500000000000001},"max":{"1min":0.495,"5min":0.55900000000000005,"15min":0.56899999999999995},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.41399999999999998,"15min":0.39300000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.255},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.67300000000000004},"last":0.34200000000000003}]},{"osd":2,"last update":"Sun Oct 8 00:20:54 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.378,"15min":0.35199999999999998},"min":{"1min":0.28100000000000003,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.58499999999999996,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.38900000000000001,"15min":0.36399999999999999},"min":{"1min":0.24299999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.55100000000000005,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.28999999999999998}]},{"osd":3,"last update":"Sun Oct 8 00:20:54 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.41699999999999998,"15min":0.39000000000000001},"min":{"1min":0.29799999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.63,"5min":0.63,"15min":0.70999999999999996},"last":0.372},{"interface":"front","average":{"1min":0.375,"5min":0.371,"15min":0.35499999999999998},"min":{"1min":0.26400000000000001,"5min":0.249,"15min":0.222},"max":{"1min":0.51900000000000002,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.47899999999999998}]},{"osd":4,"last update":"Sun Oct 8 00:20:54 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.56399999999999995,"15min":0.55200000000000005},"min":{"1min":0.39300000000000002,"5min":0.34000000000000002,"15min":0.32600000000000001},"max":{"1min":0.81000000000000005,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.58899999999999997,"15min":0.57799999999999996},"min":{"1min":0.442,"5min":0.37,"15min":0.37},"max":{"1min":0.82799999999999996,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.48799999999999999}]},{"osd":5,"last update":"Sun Oct 8 00:20:54 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58199999999999996,"15min":0.55700000000000005},"min":{"1min":0.40200000000000002,"5min":0.35099999999999998,"15min":0.33500000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.58599999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.30399999999999999},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.41799999999999998}]},{"osd":6,"last update":"Sun Oct 8 00:20:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.58399999999999996,"15min":0.57699999999999996},"min":{"1min":0.40600000000000003,"5min":0.37,"15min":0.317},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58599999999999997,"15min":0.57399999999999995},"min":{"1min":0.42699999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84099999999999997,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.40799999999999997}]},{"osd":7,"last update":"Sun Oct 8 00:20:54 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58399999999999996,"15min":0.57499999999999996},"min":{"1min":0.438,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.502},{"interface":"front","average":{"1min":0.624,"5min":0.626,"15min":0.60799999999999998},"min":{"1min":0.42299999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.85499999999999998,"5min":0.872,"15min":0.96699999999999997},"last":0.495}]}]},{"osd":0,"up_from":13,"seq":55834574980,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1008,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194303,"statfs":{"total":200000000,"available":198966856,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 00:21:06 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.33800000000000002,"15min":0.33500000000000002},"min":{"1min":0.23400000000000001,"5min":0.20399999999999999,"15min":0.193},"max":{"1min":0.45600000000000002,"5min":0.54900000000000004,"15min":1.0640000000000001},"last":0.253},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.36199999999999999,"15min":0.35399999999999998},"min":{"1min":0.251,"5min":0.216,"15min":0.20300000000000001},"max":{"1min":0.433,"5min":0.66500000000000004,"15min":0.87},"last":0.375}]},{"osd":2,"last update":"Sun Oct 8 00:21:06 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.38500000000000001,"15min":0.38700000000000001},"min":{"1min":0.29399999999999998,"5min":0.249,"15min":0.24099999999999999},"max":{"1min":0.53200000000000003,"5min":0.57199999999999995,"15min":0.93799999999999994},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.35899999999999999,"15min":0.56100000000000005},"min":{"1min":0.25600000000000001,"5min":0.23000000000000001,"15min":0.185},"max":{"1min":0.54800000000000004,"5min":0.59899999999999998,"15min":12.019},"last":0.27600000000000002}]},{"osd":3,"last update":"Sun Oct 8 00:21:06 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.38200000000000001,"15min":0.371},"min":{"1min":0.25700000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.56200000000000006,"5min":0.628,"15min":1.1020000000000001},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.41599999999999998,"15min":0.40699999999999997},"min":{"1min":0.31900000000000001,"5min":0.26700000000000002,"15min":0.246},"max":{"1min":0.51200000000000001,"5min":0.66100000000000003,"15min":1.0409999999999999},"last":0.40000000000000002}]},{"osd":4,"last update":"Sun Oct 8 00:21:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.61899999999999999,"15min":0.61699999999999999},"min":{"1min":0.39500000000000002,"5min":0.377,"15min":0.35899999999999999},"max":{"1min":0.872,"5min":0.89700000000000002,"15min":1.1779999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.621,"15min":0.624},"min":{"1min":0.44900000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89300000000000002,"5min":0.93899999999999995,"15min":0.96199999999999997},"last":0.56899999999999995}]},{"osd":5,"last update":"Sun Oct 8 00:21:06 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.58799999999999997,"15min":0.79400000000000004},"min":{"1min":0.42399999999999999,"5min":0.378,"15min":0.32000000000000001},"max":{"1min":0.878,"5min":0.90700000000000003,"15min":11.992000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.60599999999999998,"15min":0.59599999999999997},"min":{"1min":0.40999999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.80600000000000005,"5min":0.91600000000000004,"15min":1.1299999999999999},"last":0.443}]},{"osd":6,"last update":"Sun Oct 8 00:21:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60099999999999998,"15min":0.59399999999999997},"min":{"1min":0.38300000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.85699999999999998,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.497},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.60899999999999999,"15min":0.79300000000000004},"min":{"1min":0.42399999999999999,"5min":0.39600000000000002,"15min":0.38100000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":11.927},"last":0.48499999999999999}]},{"osd":7,"last update":"Sun Oct 8 00:21:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.60099999999999998,"15min":0.80300000000000005},"min":{"1min":0.39900000000000002,"5min":0.38600000000000001,"15min":0.35699999999999998},"max":{"1min":0.84399999999999997,"5min":0.95099999999999996,"15min":11.965},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.63500000000000001,"15min":0.628},"min":{"1min":0.45800000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":1.1539999999999999},"last":0.46999999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574980,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1009,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194303,"statfs":{"total":200000000,"available":198966372,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.29899999999999999,"5min":0.33200000000000002,"15min":0.36399999999999999},"min":{"1min":0.17299999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":2.5139999999999998},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.34699999999999998,"15min":0.375},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.20599999999999999},"max":{"1min":0.45800000000000002,"5min":0.59099999999999997,"15min":2.1309999999999998},"last":0.27900000000000003}]},{"osd":1,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.36099999999999999,"15min":0.376},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.49099999999999999,"5min":0.67300000000000004,"15min":2.2639999999999998},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.379,"15min":0.40799999999999997},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.48599999999999999,"5min":0.64400000000000002,"15min":2.5379999999999998},"last":0.254}]},{"osd":3,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.40300000000000002,"15min":0.41499999999999998},"min":{"1min":0.24199999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.58299999999999996,"5min":0.66900000000000004,"15min":2.2890000000000001},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.42299999999999999,"15min":0.442},"min":{"1min":0.32000000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.64300000000000002,"5min":0.68799999999999994,"15min":2.5619999999999998},"last":0.31900000000000001}]},{"osd":4,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.54700000000000004,"15min":0.58699999999999997},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.68500000000000005,"5min":0.80800000000000005,"15min":2.488},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.58499999999999996,"15min":0.59699999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.80300000000000005,"5min":0.92400000000000004,"15min":2.073},"last":0.48599999999999999}]},{"osd":5,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.60999999999999999,"15min":0.63600000000000001},"min":{"1min":0.45900000000000002,"5min":0.374,"15min":0.36099999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":2.2069999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.61899999999999999,"15min":0.64500000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88,"5min":0.94199999999999995,"15min":2.456},"last":0.5}]},{"osd":6,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.59299999999999997,"15min":0.60999999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.77300000000000002,"5min":0.93000000000000005,"15min":2.0339999999999998},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.59899999999999998,"15min":0.60799999999999998},"min":{"1min":0.42799999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.70399999999999996,"5min":0.94099999999999995,"15min":2.1699999999999999},"last":0.432}]},{"osd":7,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.60499999999999998,"15min":0.623},"min":{"1min":0.40799999999999997,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.751,"5min":0.95999999999999996,"15min":2.1030000000000002},"last":0.441},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.61199999999999999,"15min":0.621},"min":{"1min":0.44900000000000001,"5min":0.40699999999999997,"15min":0.40300000000000002},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":2.2349999999999999},"last":0.39500000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574980,"num_pgs":14,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1010,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194302,"statfs":{"total":200000000,"available":198965368,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:20:54 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33600000000000002,"15min":0.35199999999999998},"min":{"1min":0.24399999999999999,"5min":0.21299999999999999,"15min":0.192},"max":{"1min":0.48399999999999999,"5min":0.59299999999999997,"15min":1.46},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.34999999999999998,"15min":0.34399999999999997},"min":{"1min":0.26500000000000001,"5min":0.23799999999999999,"15min":0.192},"max":{"1min":0.46600000000000003,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.34200000000000003}]},{"osd":1,"last update":"Sun Oct 8 00:20:54 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.35899999999999999,"15min":0.59599999999999997},"min":{"1min":0.247,"5min":0.23699999999999999,"15min":0.20000000000000001},"max":{"1min":0.44800000000000001,"5min":0.57499999999999996,"15min":12.351000000000001},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38200000000000001,"15min":0.371},"min":{"1min":0.25700000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.51500000000000001,"5min":0.58199999999999996,"15min":0.71999999999999997},"last":0.29599999999999999}]},{"osd":2,"last update":"Sun Oct 8 00:20:54 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40500000000000003,"15min":0.40999999999999998},"min":{"1min":0.33300000000000002,"5min":0.26800000000000002,"15min":0.23499999999999999},"max":{"1min":0.53400000000000003,"5min":0.59699999999999998,"15min":1.5489999999999999},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41299999999999998,"15min":0.39700000000000002},"min":{"1min":0.34999999999999998,"5min":0.25700000000000001,"15min":0.246},"max":{"1min":0.54900000000000004,"5min":0.64900000000000002,"15min":0.753},"last":0.38300000000000001}]},{"osd":4,"last update":"Sun Oct 8 00:20:54 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.81899999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.34899999999999998},"max":{"1min":0.90300000000000002,"5min":0.96699999999999997,"15min":11.333},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.63,"15min":0.83899999999999997},"min":{"1min":0.46999999999999997,"5min":0.38100000000000001,"15min":0.36099999999999999},"max":{"1min":0.88500000000000001,"5min":0.97699999999999998,"15min":12.321999999999999},"last":0.47399999999999998}]},{"osd":5,"last update":"Sun Oct 8 00:20:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.61199999999999999,"15min":0.83799999999999997},"min":{"1min":0.44400000000000001,"5min":0.40600000000000003,"15min":0.35499999999999998},"max":{"1min":0.81000000000000005,"5min":0.93500000000000005,"15min":12.291},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64000000000000001,"15min":0.61799999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.89800000000000002,"5min":1.143,"15min":1.143},"last":0.51100000000000001}]},{"osd":6,"last update":"Sun Oct 8 00:20:54 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60299999999999998,"15min":0.79000000000000004},"min":{"1min":0.41599999999999998,"5min":0.40100000000000002,"15min":0.33700000000000002},"max":{"1min":0.91300000000000003,"5min":0.91500000000000004,"15min":11.288},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60799999999999998,"15min":0.82699999999999996},"min":{"1min":0.42299999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.83599999999999997,"5min":0.94599999999999995,"15min":12.253},"last":0.44700000000000001}]},{"osd":7,"last update":"Sun Oct 8 00:20:54 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.627,"15min":0.59899999999999998},"min":{"1min":0.443,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.88,"5min":1.0920000000000001,"15min":1.3109999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.621,"5min":0.60899999999999999,"15min":0.81799999999999995},"min":{"1min":0.434,"5min":0.39700000000000002,"15min":0.38600000000000001},"max":{"1min":0.86199999999999999,"5min":0.89300000000000002,"15min":11.363},"last":0.55600000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574980,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1008,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194303,"statfs":{"total":200000000,"available":198966892,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.56999999999999995,"15min":0.56299999999999994},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.193},"max":{"1min":0.67500000000000004,"5min":0.84799999999999998,"15min":0.93000000000000005},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.60299999999999998,"15min":0.58799999999999997},"min":{"1min":0.434,"5min":0.40600000000000003,"15min":0.30399999999999999},"max":{"1min":0.71099999999999997,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.48399999999999999}]},{"osd":1,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.54600000000000004,"15min":0.53000000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.33700000000000002},"max":{"1min":0.67200000000000004,"5min":0.76400000000000001,"15min":0.78100000000000003},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.56299999999999994,"15min":0.54300000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.30199999999999999},"max":{"1min":0.67600000000000005,"5min":0.80600000000000005,"15min":0.81299999999999994},"last":0.46899999999999997}]},{"osd":2,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.54100000000000004,"15min":0.52600000000000002},"min":{"1min":0.33800000000000002,"5min":0.32500000000000001,"15min":0.32100000000000001},"max":{"1min":0.64700000000000002,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.56399999999999995,"15min":0.53200000000000003},"min":{"1min":0.432,"5min":0.35399999999999998,"15min":0.32600000000000001},"max":{"1min":0.65400000000000003,"5min":0.79600000000000004,"15min":0.81699999999999995},"last":0.53600000000000003}]},{"osd":3,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.56799999999999995,"15min":0.54900000000000004},"min":{"1min":0.41099999999999998,"5min":0.33700000000000002,"15min":0.28799999999999998},"max":{"1min":0.71099999999999997,"5min":0.82299999999999995,"15min":0.85399999999999998},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.55400000000000005,"15min":0.54100000000000004},"min":{"1min":0.38100000000000001,"5min":0.37,"15min":0.36199999999999999},"max":{"1min":0.64700000000000002,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.44600000000000001}]},{"osd":5,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.47499999999999998,"15min":0.48299999999999998},"min":{"1min":0.33600000000000002,"5min":0.314,"15min":0.27500000000000002},"max":{"1min":0.65500000000000003,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.501,"15min":0.504},"min":{"1min":0.317,"5min":0.315,"15min":0.24399999999999999},"max":{"1min":0.64400000000000002,"5min":0.79100000000000004,"15min":0.94899999999999995},"last":0.52400000000000002}]},{"osd":6,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.26400000000000001},"max":{"1min":0.66500000000000004,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.53000000000000003,"15min":0.53900000000000003},"min":{"1min":0.35499999999999998,"5min":0.34799999999999998,"15min":0.29799999999999999},"max":{"1min":0.65600000000000003,"5min":0.76200000000000001,"15min":0.80100000000000005},"last":0.42799999999999999}]},{"osd":7,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.53500000000000003,"15min":0.54300000000000004},"min":{"1min":0.41299999999999998,"5min":0.33200000000000002,"15min":0.27900000000000003},"max":{"1min":0.67500000000000004,"5min":0.83599999999999997,"15min":0.90100000000000002},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.57199999999999995,"15min":0.57499999999999996},"min":{"1min":0.38,"5min":0.315,"15min":0.315},"max":{"1min":0.69199999999999995,"5min":0.88,"15min":0.88},"last":0.55000000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574980,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1586,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193726,"statfs":{"total":200000000,"available":198375914,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.54200000000000004,"15min":0.504},"min":{"1min":0.38,"5min":0.36099999999999999,"15min":0.249},"max":{"1min":0.78600000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.54100000000000004,"15min":0.54900000000000004},"min":{"1min":0.40000000000000002,"5min":0.29799999999999999,"15min":0.29399999999999998},"max":{"1min":0.65900000000000003,"5min":0.81699999999999995,"15min":0.88600000000000001},"last":0.57499999999999996}]},{"osd":1,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55700000000000005,"15min":0.53500000000000003},"min":{"1min":0.436,"5min":0.34399999999999997,"15min":0.32500000000000001},"max":{"1min":0.69199999999999995,"5min":0.80600000000000005,"15min":0.82999999999999996},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.53900000000000003,"15min":0.52500000000000002},"min":{"1min":0.46200000000000002,"5min":0.34499999999999997,"15min":0.33000000000000002},"max":{"1min":0.72099999999999997,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.55900000000000005}]},{"osd":2,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55300000000000005,"15min":0.53600000000000003},"min":{"1min":0.47499999999999998,"5min":0.34499999999999997,"15min":0.32400000000000001},"max":{"1min":0.67700000000000005,"5min":0.749,"15min":0.86199999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.55400000000000005,"15min":0.56000000000000005},"min":{"1min":0.41799999999999998,"5min":0.33400000000000002,"15min":0.29599999999999999},"max":{"1min":0.74399999999999999,"5min":0.80000000000000004,"15min":0.95399999999999996},"last":0.59299999999999997}]},{"osd":3,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56999999999999995,"15min":0.55500000000000005},"min":{"1min":0.437,"5min":0.42199999999999999,"15min":0.316},"max":{"1min":0.65300000000000002,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56699999999999995,"15min":0.56999999999999995},"min":{"1min":0.39500000000000002,"5min":0.376,"15min":0.371},"max":{"1min":0.73299999999999998,"5min":0.73499999999999999,"15min":1.0529999999999999},"last":0.50600000000000001}]},{"osd":4,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47399999999999998},"min":{"1min":0.30399999999999999,"5min":0.28399999999999997,"15min":0.27700000000000002},"max":{"1min":0.68700000000000006,"5min":0.78600000000000003,"15min":0.79200000000000004},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52500000000000002,"15min":0.51800000000000002},"min":{"1min":0.317,"5min":0.314,"15min":0.28899999999999998},"max":{"1min":0.76000000000000001,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.48799999999999999}]},{"osd":6,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51700000000000002,"15min":0.50700000000000001},"min":{"1min":0.372,"5min":0.28199999999999997,"15min":0.27400000000000002},"max":{"1min":0.71099999999999997,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53000000000000003,"15min":0.50900000000000001},"min":{"1min":0.35199999999999998,"5min":0.33800000000000002,"15min":0.30499999999999999},"max":{"1min":0.73799999999999999,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.47499999999999998}]},{"osd":7,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54500000000000004,"15min":0.52900000000000003},"min":{"1min":0.36199999999999999,"5min":0.318,"15min":0.29399999999999998},"max":{"1min":0.80200000000000005,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56899999999999995,"15min":0.56100000000000005},"min":{"1min":0.41599999999999998,"5min":0.35599999999999998,"15min":0.32500000000000001},"max":{"1min":0.78400000000000003,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.54300000000000004}]}]}],"pool_statfs":[]}} 2023-10-08T00:21:34.352 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T00:21:34.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.499+0000 7faa215f0700 1 Processor -- start 2023-10-08T00:21:34.500 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.499+0000 7faa215f0700 1 -- start start 2023-10-08T00:21:34.502 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.500+0000 7faa215f0700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7faa1c0b1220 0x7faa1c0c9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:34.502 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.500+0000 7faa215f0700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa1c15de60 con 0x7faa1c0cb4f0 2023-10-08T00:21:34.502 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.500+0000 7faa215f0700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa1c15e140 con 0x7faa1c0b0e50 2023-10-08T00:21:34.502 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.500+0000 7faa215f0700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa1c15e420 con 0x7faa1c0b1220 2023-10-08T00:21:34.502 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.500+0000 7faa1a7fc700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7faa1c0b1220 0x7faa1c0c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:34.502 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.500+0000 7faa1b7fe700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7faa1c0cb4f0 0x7faa1c0679d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:57650/0 (socket says 172.21.15.99:57650) 2023-10-08T00:21:34.502 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.500+0000 7faa1b7fe700 1 -- 172.21.15.99:0/1313360368 learned_addr learned my addr 172.21.15.99:0/1313360368 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:21:34.503 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.500+0000 7faa1a7fc700 1 -- 172.21.15.99:0/1313360368 >> v1:172.21.15.191:6789/0 conn(0x7faa1c0b0e50 legacy=0x7faa1c06d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:34.503 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.500+0000 7faa1a7fc700 1 -- 172.21.15.99:0/1313360368 >> v1:172.21.15.99:6789/0 conn(0x7faa1c0cb4f0 legacy=0x7faa1c0679d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:34.503 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.500+0000 7faa1a7fc700 1 -- 172.21.15.99:0/1313360368 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa1c15e700 con 0x7faa1c0b1220 2023-10-08T00:21:34.503 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.500+0000 7faa19ffb700 1 -- 172.21.15.99:0/1313360368 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1702141780 0 0) 0x7faa1c15de60 con 0x7faa1c0cb4f0 2023-10-08T00:21:34.503 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.501+0000 7faa1a7fc700 1 --2- 172.21.15.99:0/1313360368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa1c0b1220 0x7faa1c0c9180 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7faa100098e0 tx=0x7faa100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca131a81b0194db3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:34.503 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.501+0000 7faa19ffb700 1 -- 172.21.15.99:0/1313360368 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa10009e80 con 0x7faa1c0b1220 2023-10-08T00:21:34.503 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.501+0000 7faa19ffb700 1 -- 172.21.15.99:0/1313360368 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa1000b820 con 0x7faa1c0b1220 2023-10-08T00:21:34.503 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.501+0000 7faa19ffb700 1 -- 172.21.15.99:0/1313360368 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1000bba0 con 0x7faa1c0b1220 2023-10-08T00:21:34.503 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.501+0000 7faa215f0700 1 -- 172.21.15.99:0/1313360368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa1c0b1220 msgr2=0x7faa1c0c9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:34.503 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.501+0000 7faa215f0700 1 --2- 172.21.15.99:0/1313360368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa1c0b1220 0x7faa1c0c9180 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7faa100098e0 tx=0x7faa100091a0 comp rx=0 tx=0).stop 2023-10-08T00:21:34.504 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.501+0000 7faa215f0700 1 -- 172.21.15.99:0/1313360368 shutdown_connections 2023-10-08T00:21:34.504 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.501+0000 7faa215f0700 1 --2- 172.21.15.99:0/1313360368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa1c0b1220 0x7faa1c0c9180 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:34.504 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.501+0000 7faa215f0700 1 -- 172.21.15.99:0/1313360368 >> 172.21.15.99:0/1313360368 conn(0x7faa1c069870 msgr2=0x7faa1c069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:34.504 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.501+0000 7faa215f0700 1 -- 172.21.15.99:0/1313360368 shutdown_connections 2023-10-08T00:21:34.504 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.501+0000 7faa215f0700 1 -- 172.21.15.99:0/1313360368 wait complete. 2023-10-08T00:21:34.506 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.503+0000 7faa215f0700 1 Processor -- start 2023-10-08T00:21:34.506 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.503+0000 7faa215f0700 1 -- start start 2023-10-08T00:21:34.506 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.503+0000 7faa215f0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa1c0b0e50 0x7faa1c14cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:34.507 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.503+0000 7faa215f0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa1c0b1220 0x7faa1c0c5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:34.507 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.503+0000 7faa215f0700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa1c0cb4f0 0x7faa1c0c62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:34.507 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.503+0000 7faa215f0700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c15b110 con 0x7faa1c0b1220 2023-10-08T00:21:34.507 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.503+0000 7faa215f0700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c15a500 con 0x7faa1c0cb4f0 2023-10-08T00:21:34.507 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.503+0000 7faa215f0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c15a7e0 con 0x7faa1c0b0e50 2023-10-08T00:21:34.507 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.503+0000 7faa1a7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa1c0b1220 0x7faa1c0c5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:34.507 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.503+0000 7faa1a7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa1c0b1220 0x7faa1c0c5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51916/0 (socket says 172.21.15.99:51916) 2023-10-08T00:21:34.507 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.503+0000 7faa1a7fc700 1 -- 172.21.15.99:0/3670412186 learned_addr learned my addr 172.21.15.99:0/3670412186 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:34.508 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.504+0000 7faa1affd700 1 --2- 172.21.15.99:0/3670412186 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa1c0b0e50 0x7faa1c14cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:34.508 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.504+0000 7faa1b7fe700 1 --2- 172.21.15.99:0/3670412186 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa1c0cb4f0 0x7faa1c0c62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:34.508 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.504+0000 7faa1a7fc700 1 -- 172.21.15.99:0/3670412186 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa1c0b0e50 msgr2=0x7faa1c14cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:34.508 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.504+0000 7faa1a7fc700 1 --2- 172.21.15.99:0/3670412186 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa1c0b0e50 0x7faa1c14cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:34.508 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.504+0000 7faa1a7fc700 1 -- 172.21.15.99:0/3670412186 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa1c0cb4f0 msgr2=0x7faa1c0c62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:34.508 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.504+0000 7faa1a7fc700 1 --2- 172.21.15.99:0/3670412186 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa1c0cb4f0 0x7faa1c0c62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:34.508 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.504+0000 7faa1a7fc700 1 -- 172.21.15.99:0/3670412186 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa10009420 con 0x7faa1c0b1220 2023-10-08T00:21:34.508 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.504+0000 7faa1a7fc700 1 --2- 172.21.15.99:0/3670412186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa1c0b1220 0x7faa1c0c5d80 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7faa100030c0 tx=0x7faa10016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:34.508 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.504+0000 7faa02ffd700 1 -- 172.21.15.99:0/3670412186 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1000b990 con 0x7faa1c0b1220 2023-10-08T00:21:34.508 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.504+0000 7faa02ffd700 1 -- 172.21.15.99:0/3670412186 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa10020070 con 0x7faa1c0b1220 2023-10-08T00:21:34.508 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.504+0000 7faa215f0700 1 -- 172.21.15.99:0/3670412186 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa1c15aac0 con 0x7faa1c0b1220 2023-10-08T00:21:34.509 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.504+0000 7faa02ffd700 1 -- 172.21.15.99:0/3670412186 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1001b450 con 0x7faa1c0b1220 2023-10-08T00:21:34.509 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.504+0000 7faa215f0700 1 -- 172.21.15.99:0/3670412186 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa1c14d510 con 0x7faa1c0b1220 2023-10-08T00:21:34.509 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.505+0000 7faa215f0700 1 -- 172.21.15.99:0/3670412186 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa1c15b280 con 0x7faa1c0b1220 2023-10-08T00:21:34.509 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.505+0000 7faa02ffd700 1 -- 172.21.15.99:0/3670412186 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faa10022030 con 0x7faa1c0b1220 2023-10-08T00:21:34.509 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.506+0000 7faa02ffd700 1 --2- 172.21.15.99:0/3670412186 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7faa08066f30 0x7faa080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:34.509 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.506+0000 7faa02ffd700 1 -- 172.21.15.99:0/3670412186 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5705+0+0 (secure 0 0 0) 0x7faa10086500 con 0x7faa1c0b1220 2023-10-08T00:21:34.509 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.506+0000 7faa1affd700 1 --2- 172.21.15.99:0/3670412186 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7faa08066f30 0x7faa080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:34.509 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.506+0000 7faa1affd700 1 --2- 172.21.15.99:0/3670412186 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7faa08066f30 0x7faa080693e0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7faa0c007fd0 tx=0x7faa0c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:34.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.510+0000 7faa02ffd700 1 -- 172.21.15.99:0/3670412186 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faa10053770 con 0x7faa1c0b1220 2023-10-08T00:21:34.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.686+0000 7faa215f0700 1 -- 172.21.15.99:0/3670412186 --> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faa1c15b250 con 0x7faa08066f30 2023-10-08T00:21:34.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.694+0000 7faa02ffd700 1 -- 172.21.15.99:0/3670412186 <== mgr.4099 v2:172.21.15.191:6832/85985 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285561 (secure 0 0 0) 0x7faa1c15b250 con 0x7faa08066f30 2023-10-08T00:21:34.698 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:21:34.700 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.697+0000 7faa215f0700 1 -- 172.21.15.99:0/3670412186 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7faa08066f30 msgr2=0x7faa080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:34.700 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.697+0000 7faa215f0700 1 --2- 172.21.15.99:0/3670412186 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7faa08066f30 0x7faa080693e0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7faa0c007fd0 tx=0x7faa0c009040 comp rx=0 tx=0).stop 2023-10-08T00:21:34.700 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.697+0000 7faa215f0700 1 -- 172.21.15.99:0/3670412186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa1c0b1220 msgr2=0x7faa1c0c5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:34.700 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.697+0000 7faa215f0700 1 --2- 172.21.15.99:0/3670412186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa1c0b1220 0x7faa1c0c5d80 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7faa100030c0 tx=0x7faa10016000 comp rx=0 tx=0).stop 2023-10-08T00:21:34.701 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.697+0000 7faa215f0700 1 -- 172.21.15.99:0/3670412186 shutdown_connections 2023-10-08T00:21:34.701 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.697+0000 7faa215f0700 1 --2- 172.21.15.99:0/3670412186 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7faa08066f30 0x7faa080693e0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:34.701 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.697+0000 7faa215f0700 1 --2- 172.21.15.99:0/3670412186 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa1c0cb4f0 0x7faa1c0c62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:34.701 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.697+0000 7faa215f0700 1 --2- 172.21.15.99:0/3670412186 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa1c0b0e50 0x7faa1c14cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:34.701 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.697+0000 7faa215f0700 1 --2- 172.21.15.99:0/3670412186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa1c0b1220 0x7faa1c0c5d80 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:34.701 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.697+0000 7faa215f0700 1 -- 172.21.15.99:0/3670412186 >> 172.21.15.99:0/3670412186 conn(0x7faa1c069870 msgr2=0x7faa1c14dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:34.701 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.697+0000 7faa215f0700 1 -- 172.21.15.99:0/3670412186 shutdown_connections 2023-10-08T00:21:34.701 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.697+0000 7faa215f0700 1 -- 172.21.15.99:0/3670412186 wait complete. 2023-10-08T00:21:34.701 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2023-10-08T00:21:34.710 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":500,"stamp":"2023-10-08T00:21:34.176301+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":174,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9228,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1553271,"statfs":{"total":1600000000,"available":1590550060,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.895534"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.301479+0000","last_change":"2023-10-08T00:21:30.301213+0000","last_active":"2023-10-08T00:21:30.301479+0000","last_peered":"2023-10-08T00:21:30.301479+0000","last_clean":"2023-10-08T00:21:30.301479+0000","last_became_active":"2023-10-08T00:21:30.300255+0000","last_became_peered":"2023-10-08T00:21:30.300255+0000","last_unstale":"2023-10-08T00:21:30.301479+0000","last_undegraded":"2023-10-08T00:21:30.301479+0000","last_fullsized":"2023-10-08T00:21:30.301479+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:58:05.480523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.300383+0000","last_change":"2023-10-08T00:21:30.298767+0000","last_active":"2023-10-08T00:21:30.300383+0000","last_peered":"2023-10-08T00:21:30.300383+0000","last_clean":"2023-10-08T00:21:30.300383+0000","last_became_active":"2023-10-08T00:21:30.298534+0000","last_became_peered":"2023-10-08T00:21:30.298534+0000","last_unstale":"2023-10-08T00:21:30.300383+0000","last_undegraded":"2023-10-08T00:21:30.300383+0000","last_fullsized":"2023-10-08T00:21:30.300383+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:54:08.470147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.298015+0000","last_change":"2023-10-08T00:21:30.297676+0000","last_active":"2023-10-08T00:21:30.298015+0000","last_peered":"2023-10-08T00:21:30.298015+0000","last_clean":"2023-10-08T00:21:30.298015+0000","last_became_active":"2023-10-08T00:21:30.296962+0000","last_became_peered":"2023-10-08T00:21:30.296962+0000","last_unstale":"2023-10-08T00:21:30.298015+0000","last_undegraded":"2023-10-08T00:21:30.298015+0000","last_fullsized":"2023-10-08T00:21:30.298015+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:35:52.651092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.294926+0000","last_change":"2023-10-08T00:21:30.294697+0000","last_active":"2023-10-08T00:21:30.294926+0000","last_peered":"2023-10-08T00:21:30.294926+0000","last_clean":"2023-10-08T00:21:30.294926+0000","last_became_active":"2023-10-08T00:21:30.294198+0000","last_became_peered":"2023-10-08T00:21:30.294198+0000","last_unstale":"2023-10-08T00:21:30.294926+0000","last_undegraded":"2023-10-08T00:21:30.294926+0000","last_fullsized":"2023-10-08T00:21:30.294926+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:23:46.629646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.300708+0000","last_change":"2023-10-08T00:21:30.300583+0000","last_active":"2023-10-08T00:21:30.300708+0000","last_peered":"2023-10-08T00:21:30.300708+0000","last_clean":"2023-10-08T00:21:30.300708+0000","last_became_active":"2023-10-08T00:21:30.299505+0000","last_became_peered":"2023-10-08T00:21:30.299505+0000","last_unstale":"2023-10-08T00:21:30.300708+0000","last_undegraded":"2023-10-08T00:21:30.300708+0000","last_fullsized":"2023-10-08T00:21:30.300708+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:19:07.335627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.291203+0000","last_change":"2023-10-08T00:21:30.291130+0000","last_active":"2023-10-08T00:21:30.291203+0000","last_peered":"2023-10-08T00:21:30.291203+0000","last_clean":"2023-10-08T00:21:30.291203+0000","last_became_active":"2023-10-08T00:21:30.290758+0000","last_became_peered":"2023-10-08T00:21:30.290758+0000","last_unstale":"2023-10-08T00:21:30.291203+0000","last_undegraded":"2023-10-08T00:21:30.291203+0000","last_fullsized":"2023-10-08T00:21:30.291203+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:52:23.286803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.300075+0000","last_change":"2023-10-08T00:21:30.300030+0000","last_active":"2023-10-08T00:21:30.300075+0000","last_peered":"2023-10-08T00:21:30.300075+0000","last_clean":"2023-10-08T00:21:30.300075+0000","last_became_active":"2023-10-08T00:21:30.299622+0000","last_became_peered":"2023-10-08T00:21:30.299622+0000","last_unstale":"2023-10-08T00:21:30.300075+0000","last_undegraded":"2023-10-08T00:21:30.300075+0000","last_fullsized":"2023-10-08T00:21:30.300075+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:22:50.229264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.296060+0000","last_change":"2023-10-08T00:21:30.295676+0000","last_active":"2023-10-08T00:21:30.296060+0000","last_peered":"2023-10-08T00:21:30.296060+0000","last_clean":"2023-10-08T00:21:30.296060+0000","last_became_active":"2023-10-08T00:21:30.295426+0000","last_became_peered":"2023-10-08T00:21:30.295426+0000","last_unstale":"2023-10-08T00:21:30.296060+0000","last_undegraded":"2023-10-08T00:21:30.296060+0000","last_fullsized":"2023-10-08T00:21:30.296060+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:23:46.629646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.301071+0000","last_change":"2023-10-08T00:21:30.300677+0000","last_active":"2023-10-08T00:21:30.301071+0000","last_peered":"2023-10-08T00:21:30.301071+0000","last_clean":"2023-10-08T00:21:30.301071+0000","last_became_active":"2023-10-08T00:21:30.300290+0000","last_became_peered":"2023-10-08T00:21:30.300290+0000","last_unstale":"2023-10-08T00:21:30.301071+0000","last_undegraded":"2023-10-08T00:21:30.301071+0000","last_fullsized":"2023-10-08T00:21:30.301071+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:58:05.480523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.301087+0000","last_change":"2023-10-08T00:21:30.300861+0000","last_active":"2023-10-08T00:21:30.301087+0000","last_peered":"2023-10-08T00:21:30.301087+0000","last_clean":"2023-10-08T00:21:30.301087+0000","last_became_active":"2023-10-08T00:21:30.300258+0000","last_became_peered":"2023-10-08T00:21:30.300258+0000","last_unstale":"2023-10-08T00:21:30.301087+0000","last_undegraded":"2023-10-08T00:21:30.301087+0000","last_fullsized":"2023-10-08T00:21:30.301087+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:37:52.274322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.300511+0000","last_change":"2023-10-08T00:21:30.300452+0000","last_active":"2023-10-08T00:21:30.300511+0000","last_peered":"2023-10-08T00:21:30.300511+0000","last_clean":"2023-10-08T00:21:30.300511+0000","last_became_active":"2023-10-08T00:21:30.300241+0000","last_became_peered":"2023-10-08T00:21:30.300241+0000","last_unstale":"2023-10-08T00:21:30.300511+0000","last_undegraded":"2023-10-08T00:21:30.300511+0000","last_fullsized":"2023-10-08T00:21:30.300511+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:28:01.264300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.297770+0000","last_change":"2023-10-08T00:21:30.297575+0000","last_active":"2023-10-08T00:21:30.297770+0000","last_peered":"2023-10-08T00:21:30.297770+0000","last_clean":"2023-10-08T00:21:30.297770+0000","last_became_active":"2023-10-08T00:21:30.297366+0000","last_became_peered":"2023-10-08T00:21:30.297366+0000","last_unstale":"2023-10-08T00:21:30.297770+0000","last_undegraded":"2023-10-08T00:21:30.297770+0000","last_fullsized":"2023-10-08T00:21:30.297770+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:18:35.800650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.294244+0000","last_change":"2023-10-08T00:21:30.294177+0000","last_active":"2023-10-08T00:21:30.294244+0000","last_peered":"2023-10-08T00:21:30.294244+0000","last_clean":"2023-10-08T00:21:30.294244+0000","last_became_active":"2023-10-08T00:21:30.293676+0000","last_became_peered":"2023-10-08T00:21:30.293676+0000","last_unstale":"2023-10-08T00:21:30.294244+0000","last_undegraded":"2023-10-08T00:21:30.294244+0000","last_fullsized":"2023-10-08T00:21:30.294244+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:00:46.701778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.299097+0000","last_change":"2023-10-08T00:21:30.299036+0000","last_active":"2023-10-08T00:21:30.299097+0000","last_peered":"2023-10-08T00:21:30.299097+0000","last_clean":"2023-10-08T00:21:30.299097+0000","last_became_active":"2023-10-08T00:21:30.298590+0000","last_became_peered":"2023-10-08T00:21:30.298590+0000","last_unstale":"2023-10-08T00:21:30.299097+0000","last_undegraded":"2023-10-08T00:21:30.299097+0000","last_fullsized":"2023-10-08T00:21:30.299097+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:50:06.095667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.301101+0000","last_change":"2023-10-08T00:21:30.300996+0000","last_active":"2023-10-08T00:21:30.301101+0000","last_peered":"2023-10-08T00:21:30.301101+0000","last_clean":"2023-10-08T00:21:30.301101+0000","last_became_active":"2023-10-08T00:21:30.300543+0000","last_became_peered":"2023-10-08T00:21:30.300543+0000","last_unstale":"2023-10-08T00:21:30.301101+0000","last_undegraded":"2023-10-08T00:21:30.301101+0000","last_fullsized":"2023-10-08T00:21:30.301101+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:03:08.518439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.301726+0000","last_change":"2023-10-08T00:21:30.301665+0000","last_active":"2023-10-08T00:21:30.301726+0000","last_peered":"2023-10-08T00:21:30.301726+0000","last_clean":"2023-10-08T00:21:30.301726+0000","last_became_active":"2023-10-08T00:21:30.301105+0000","last_became_peered":"2023-10-08T00:21:30.301105+0000","last_unstale":"2023-10-08T00:21:30.301726+0000","last_undegraded":"2023-10-08T00:21:30.301726+0000","last_fullsized":"2023-10-08T00:21:30.301726+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:57:33.670768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.295610+0000","last_change":"2023-10-08T00:21:30.295547+0000","last_active":"2023-10-08T00:21:30.295610+0000","last_peered":"2023-10-08T00:21:30.295610+0000","last_clean":"2023-10-08T00:21:30.295610+0000","last_became_active":"2023-10-08T00:21:30.295307+0000","last_became_peered":"2023-10-08T00:21:30.295307+0000","last_unstale":"2023-10-08T00:21:30.295610+0000","last_undegraded":"2023-10-08T00:21:30.295610+0000","last_fullsized":"2023-10-08T00:21:30.295610+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:07:17.295368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.295325+0000","last_change":"2023-10-08T00:21:30.295227+0000","last_active":"2023-10-08T00:21:30.295325+0000","last_peered":"2023-10-08T00:21:30.295325+0000","last_clean":"2023-10-08T00:21:30.295325+0000","last_became_active":"2023-10-08T00:21:30.295014+0000","last_became_peered":"2023-10-08T00:21:30.295014+0000","last_unstale":"2023-10-08T00:21:30.295325+0000","last_undegraded":"2023-10-08T00:21:30.295325+0000","last_fullsized":"2023-10-08T00:21:30.295325+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:02:23.529002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.302574+0000","last_change":"2023-10-08T00:21:30.302299+0000","last_active":"2023-10-08T00:21:30.302574+0000","last_peered":"2023-10-08T00:21:30.302574+0000","last_clean":"2023-10-08T00:21:30.302574+0000","last_became_active":"2023-10-08T00:21:30.302040+0000","last_became_peered":"2023-10-08T00:21:30.302040+0000","last_unstale":"2023-10-08T00:21:30.302574+0000","last_undegraded":"2023-10-08T00:21:30.302574+0000","last_fullsized":"2023-10-08T00:21:30.302574+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:23:46.629646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.299851+0000","last_change":"2023-10-08T00:21:30.299794+0000","last_active":"2023-10-08T00:21:30.299851+0000","last_peered":"2023-10-08T00:21:30.299851+0000","last_clean":"2023-10-08T00:21:30.299851+0000","last_became_active":"2023-10-08T00:21:30.299513+0000","last_became_peered":"2023-10-08T00:21:30.299513+0000","last_unstale":"2023-10-08T00:21:30.299851+0000","last_undegraded":"2023-10-08T00:21:30.299851+0000","last_fullsized":"2023-10-08T00:21:30.299851+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:50:10.281370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.298911+0000","last_change":"2023-10-08T00:21:30.298696+0000","last_active":"2023-10-08T00:21:30.298911+0000","last_peered":"2023-10-08T00:21:30.298911+0000","last_clean":"2023-10-08T00:21:30.298911+0000","last_became_active":"2023-10-08T00:21:30.297906+0000","last_became_peered":"2023-10-08T00:21:30.297906+0000","last_unstale":"2023-10-08T00:21:30.298911+0000","last_undegraded":"2023-10-08T00:21:30.298911+0000","last_fullsized":"2023-10-08T00:21:30.298911+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:30:32.148405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.296539+0000","last_change":"2023-10-08T00:21:30.296493+0000","last_active":"2023-10-08T00:21:30.296539+0000","last_peered":"2023-10-08T00:21:30.296539+0000","last_clean":"2023-10-08T00:21:30.296539+0000","last_became_active":"2023-10-08T00:21:30.296289+0000","last_became_peered":"2023-10-08T00:21:30.296289+0000","last_unstale":"2023-10-08T00:21:30.296539+0000","last_undegraded":"2023-10-08T00:21:30.296539+0000","last_fullsized":"2023-10-08T00:21:30.296539+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:13:04.243694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.298738+0000","last_change":"2023-10-08T00:21:30.298359+0000","last_active":"2023-10-08T00:21:30.298738+0000","last_peered":"2023-10-08T00:21:30.298738+0000","last_clean":"2023-10-08T00:21:30.298738+0000","last_became_active":"2023-10-08T00:21:30.297873+0000","last_became_peered":"2023-10-08T00:21:30.297873+0000","last_unstale":"2023-10-08T00:21:30.298738+0000","last_undegraded":"2023-10-08T00:21:30.298738+0000","last_fullsized":"2023-10-08T00:21:30.298738+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:03:08.518439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.296205+0000","last_change":"2023-10-08T00:21:30.295993+0000","last_active":"2023-10-08T00:21:30.296205+0000","last_peered":"2023-10-08T00:21:30.296205+0000","last_clean":"2023-10-08T00:21:30.296205+0000","last_became_active":"2023-10-08T00:21:30.295770+0000","last_became_peered":"2023-10-08T00:21:30.295770+0000","last_unstale":"2023-10-08T00:21:30.296205+0000","last_undegraded":"2023-10-08T00:21:30.296205+0000","last_fullsized":"2023-10-08T00:21:30.296205+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:49:32.812313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.301446+0000","last_change":"2023-10-08T00:21:30.301389+0000","last_active":"2023-10-08T00:21:30.301446+0000","last_peered":"2023-10-08T00:21:30.301446+0000","last_clean":"2023-10-08T00:21:30.301446+0000","last_became_active":"2023-10-08T00:21:30.301134+0000","last_became_peered":"2023-10-08T00:21:30.301134+0000","last_unstale":"2023-10-08T00:21:30.301446+0000","last_undegraded":"2023-10-08T00:21:30.301446+0000","last_fullsized":"2023-10-08T00:21:30.301446+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:27:00.784747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.290852+0000","last_change":"2023-10-08T00:21:30.290053+0000","last_active":"2023-10-08T00:21:30.290852+0000","last_peered":"2023-10-08T00:21:30.290852+0000","last_clean":"2023-10-08T00:21:30.290852+0000","last_became_active":"2023-10-08T00:21:30.289833+0000","last_became_peered":"2023-10-08T00:21:30.289833+0000","last_unstale":"2023-10-08T00:21:30.290852+0000","last_undegraded":"2023-10-08T00:21:30.290852+0000","last_fullsized":"2023-10-08T00:21:30.290852+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:58:05.480523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.297985+0000","last_change":"2023-10-08T00:21:30.297919+0000","last_active":"2023-10-08T00:21:30.297985+0000","last_peered":"2023-10-08T00:21:30.297985+0000","last_clean":"2023-10-08T00:21:30.297985+0000","last_became_active":"2023-10-08T00:21:30.297688+0000","last_became_peered":"2023-10-08T00:21:30.297688+0000","last_unstale":"2023-10-08T00:21:30.297985+0000","last_undegraded":"2023-10-08T00:21:30.297985+0000","last_fullsized":"2023-10-08T00:21:30.297985+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:11:22.573459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.300555+0000","last_change":"2023-10-08T00:21:30.300492+0000","last_active":"2023-10-08T00:21:30.300555+0000","last_peered":"2023-10-08T00:21:30.300555+0000","last_clean":"2023-10-08T00:21:30.300555+0000","last_became_active":"2023-10-08T00:21:30.300255+0000","last_became_peered":"2023-10-08T00:21:30.300255+0000","last_unstale":"2023-10-08T00:21:30.300555+0000","last_undegraded":"2023-10-08T00:21:30.300555+0000","last_fullsized":"2023-10-08T00:21:30.300555+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:50:06.095667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.297998+0000","last_change":"2023-10-08T00:21:30.297029+0000","last_active":"2023-10-08T00:21:30.297998+0000","last_peered":"2023-10-08T00:21:30.297998+0000","last_clean":"2023-10-08T00:21:30.297998+0000","last_became_active":"2023-10-08T00:21:30.296798+0000","last_became_peered":"2023-10-08T00:21:30.296798+0000","last_unstale":"2023-10-08T00:21:30.297998+0000","last_undegraded":"2023-10-08T00:21:30.297998+0000","last_fullsized":"2023-10-08T00:21:30.297998+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:39:05.416412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.297885+0000","last_change":"2023-10-08T00:21:30.297774+0000","last_active":"2023-10-08T00:21:30.297885+0000","last_peered":"2023-10-08T00:21:30.297885+0000","last_clean":"2023-10-08T00:21:30.297885+0000","last_became_active":"2023-10-08T00:21:30.297378+0000","last_became_peered":"2023-10-08T00:21:30.297378+0000","last_unstale":"2023-10-08T00:21:30.297885+0000","last_undegraded":"2023-10-08T00:21:30.297885+0000","last_fullsized":"2023-10-08T00:21:30.297885+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:57:33.670768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.301460+0000","last_change":"2023-10-08T00:21:30.301052+0000","last_active":"2023-10-08T00:21:30.301460+0000","last_peered":"2023-10-08T00:21:30.301460+0000","last_clean":"2023-10-08T00:21:30.301460+0000","last_became_active":"2023-10-08T00:21:30.299794+0000","last_became_peered":"2023-10-08T00:21:30.299794+0000","last_unstale":"2023-10-08T00:21:30.301460+0000","last_undegraded":"2023-10-08T00:21:30.301460+0000","last_fullsized":"2023-10-08T00:21:30.301460+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:32:18.516618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.294301+0000","last_change":"2023-10-08T00:21:30.294049+0000","last_active":"2023-10-08T00:21:30.294301+0000","last_peered":"2023-10-08T00:21:30.294301+0000","last_clean":"2023-10-08T00:21:30.294301+0000","last_became_active":"2023-10-08T00:21:30.293372+0000","last_became_peered":"2023-10-08T00:21:30.293372+0000","last_unstale":"2023-10-08T00:21:30.294301+0000","last_undegraded":"2023-10-08T00:21:30.294301+0000","last_fullsized":"2023-10-08T00:21:30.294301+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:57:54.792143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.294943+0000","last_change":"2023-10-08T00:21:30.294867+0000","last_active":"2023-10-08T00:21:30.294943+0000","last_peered":"2023-10-08T00:21:30.294943+0000","last_clean":"2023-10-08T00:21:30.294943+0000","last_became_active":"2023-10-08T00:21:30.294476+0000","last_became_peered":"2023-10-08T00:21:30.294476+0000","last_unstale":"2023-10-08T00:21:30.294943+0000","last_undegraded":"2023-10-08T00:21:30.294943+0000","last_fullsized":"2023-10-08T00:21:30.294943+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:58:05.480523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1f","version":"183'4","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.311301+0000","last_change":"2023-10-08T00:21:29.297829+0000","last_active":"2023-10-08T00:21:30.311301+0000","last_peered":"2023-10-08T00:21:30.311301+0000","last_clean":"2023-10-08T00:21:30.311301+0000","last_became_active":"2023-10-08T00:21:29.297497+0000","last_became_peered":"2023-10-08T00:21:29.297497+0000","last_unstale":"2023-10-08T00:21:30.311301+0000","last_undegraded":"2023-10-08T00:21:30.311301+0000","last_fullsized":"2023-10-08T00:21:30.311301+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:20:48.787420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.301510+0000","last_change":"2023-10-08T00:21:30.301405+0000","last_active":"2023-10-08T00:21:30.301510+0000","last_peered":"2023-10-08T00:21:30.301510+0000","last_clean":"2023-10-08T00:21:30.301510+0000","last_became_active":"2023-10-08T00:21:30.301116+0000","last_became_peered":"2023-10-08T00:21:30.301116+0000","last_unstale":"2023-10-08T00:21:30.301510+0000","last_undegraded":"2023-10-08T00:21:30.301510+0000","last_fullsized":"2023-10-08T00:21:30.301510+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:36:42.424132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1e","version":"183'1","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.310359+0000","last_change":"2023-10-08T00:21:29.297676+0000","last_active":"2023-10-08T00:21:30.310359+0000","last_peered":"2023-10-08T00:21:30.310359+0000","last_clean":"2023-10-08T00:21:30.310359+0000","last_became_active":"2023-10-08T00:21:29.297452+0000","last_became_peered":"2023-10-08T00:21:29.297452+0000","last_unstale":"2023-10-08T00:21:30.310359+0000","last_undegraded":"2023-10-08T00:21:30.310359+0000","last_fullsized":"2023-10-08T00:21:30.310359+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:38:37.634900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.298891+0000","last_change":"2023-10-08T00:21:30.298562+0000","last_active":"2023-10-08T00:21:30.298891+0000","last_peered":"2023-10-08T00:21:30.298891+0000","last_clean":"2023-10-08T00:21:30.298891+0000","last_became_active":"2023-10-08T00:21:30.297599+0000","last_became_peered":"2023-10-08T00:21:30.297599+0000","last_unstale":"2023-10-08T00:21:30.298891+0000","last_undegraded":"2023-10-08T00:21:30.298891+0000","last_fullsized":"2023-10-08T00:21:30.298891+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:53:37.006424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1d","version":"183'2","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.311359+0000","last_change":"2023-10-08T00:21:29.292594+0000","last_active":"2023-10-08T00:21:30.311359+0000","last_peered":"2023-10-08T00:21:30.311359+0000","last_clean":"2023-10-08T00:21:30.311359+0000","last_became_active":"2023-10-08T00:21:29.292296+0000","last_became_peered":"2023-10-08T00:21:29.292296+0000","last_unstale":"2023-10-08T00:21:30.311359+0000","last_undegraded":"2023-10-08T00:21:30.311359+0000","last_fullsized":"2023-10-08T00:21:30.311359+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:20:31.873834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.297793+0000","last_change":"2023-10-08T00:21:30.297650+0000","last_active":"2023-10-08T00:21:30.297793+0000","last_peered":"2023-10-08T00:21:30.297793+0000","last_clean":"2023-10-08T00:21:30.297793+0000","last_became_active":"2023-10-08T00:21:30.297091+0000","last_became_peered":"2023-10-08T00:21:30.297091+0000","last_unstale":"2023-10-08T00:21:30.297793+0000","last_undegraded":"2023-10-08T00:21:30.297793+0000","last_fullsized":"2023-10-08T00:21:30.297793+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:33:45.741583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1c","version":"183'2","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.310855+0000","last_change":"2023-10-08T00:21:29.296156+0000","last_active":"2023-10-08T00:21:30.310855+0000","last_peered":"2023-10-08T00:21:30.310855+0000","last_clean":"2023-10-08T00:21:30.310855+0000","last_became_active":"2023-10-08T00:21:29.295887+0000","last_became_peered":"2023-10-08T00:21:29.295887+0000","last_unstale":"2023-10-08T00:21:30.310855+0000","last_undegraded":"2023-10-08T00:21:30.310855+0000","last_fullsized":"2023-10-08T00:21:30.310855+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:53:36.001232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.294453+0000","last_change":"2023-10-08T00:21:30.294381+0000","last_active":"2023-10-08T00:21:30.294453+0000","last_peered":"2023-10-08T00:21:30.294453+0000","last_clean":"2023-10-08T00:21:30.294453+0000","last_became_active":"2023-10-08T00:21:30.294133+0000","last_became_peered":"2023-10-08T00:21:30.294133+0000","last_unstale":"2023-10-08T00:21:30.294453+0000","last_undegraded":"2023-10-08T00:21:30.294453+0000","last_fullsized":"2023-10-08T00:21:30.294453+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:17:21.382612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1b","version":"183'1","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.309034+0000","last_change":"2023-10-08T00:21:29.295395+0000","last_active":"2023-10-08T00:21:30.309034+0000","last_peered":"2023-10-08T00:21:30.309034+0000","last_clean":"2023-10-08T00:21:30.309034+0000","last_became_active":"2023-10-08T00:21:29.295143+0000","last_became_peered":"2023-10-08T00:21:29.295143+0000","last_unstale":"2023-10-08T00:21:30.309034+0000","last_undegraded":"2023-10-08T00:21:30.309034+0000","last_fullsized":"2023-10-08T00:21:30.309034+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:30:31.143213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.294316+0000","last_change":"2023-10-08T00:21:30.294179+0000","last_active":"2023-10-08T00:21:30.294316+0000","last_peered":"2023-10-08T00:21:30.294316+0000","last_clean":"2023-10-08T00:21:30.294316+0000","last_became_active":"2023-10-08T00:21:30.293680+0000","last_became_peered":"2023-10-08T00:21:30.293680+0000","last_unstale":"2023-10-08T00:21:30.294316+0000","last_undegraded":"2023-10-08T00:21:30.294316+0000","last_fullsized":"2023-10-08T00:21:30.294316+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:22:07.112692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1a","version":"183'1","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.309769+0000","last_change":"2023-10-08T00:21:29.297064+0000","last_active":"2023-10-08T00:21:30.309769+0000","last_peered":"2023-10-08T00:21:30.309769+0000","last_clean":"2023-10-08T00:21:30.309769+0000","last_became_active":"2023-10-08T00:21:29.296879+0000","last_became_peered":"2023-10-08T00:21:29.296879+0000","last_unstale":"2023-10-08T00:21:30.309769+0000","last_undegraded":"2023-10-08T00:21:30.309769+0000","last_fullsized":"2023-10-08T00:21:30.309769+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:53:36.001232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.297446+0000","last_change":"2023-10-08T00:21:30.297319+0000","last_active":"2023-10-08T00:21:30.297446+0000","last_peered":"2023-10-08T00:21:30.297446+0000","last_clean":"2023-10-08T00:21:30.297446+0000","last_became_active":"2023-10-08T00:21:30.297046+0000","last_became_peered":"2023-10-08T00:21:30.297046+0000","last_unstale":"2023-10-08T00:21:30.297446+0000","last_undegraded":"2023-10-08T00:21:30.297446+0000","last_fullsized":"2023-10-08T00:21:30.297446+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:37:52.274322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.291722+0000","last_change":"2023-10-08T00:21:29.295415+0000","last_active":"2023-10-08T00:21:30.291722+0000","last_peered":"2023-10-08T00:21:30.291722+0000","last_clean":"2023-10-08T00:21:30.291722+0000","last_became_active":"2023-10-08T00:21:29.295148+0000","last_became_peered":"2023-10-08T00:21:29.295148+0000","last_unstale":"2023-10-08T00:21:30.291722+0000","last_undegraded":"2023-10-08T00:21:30.291722+0000","last_fullsized":"2023-10-08T00:21:30.291722+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:57:53.786951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.299156+0000","last_change":"2023-10-08T00:21:30.299041+0000","last_active":"2023-10-08T00:21:30.299156+0000","last_peered":"2023-10-08T00:21:30.299156+0000","last_clean":"2023-10-08T00:21:30.299156+0000","last_became_active":"2023-10-08T00:21:30.297008+0000","last_became_peered":"2023-10-08T00:21:30.297008+0000","last_unstale":"2023-10-08T00:21:30.299156+0000","last_undegraded":"2023-10-08T00:21:30.299156+0000","last_fullsized":"2023-10-08T00:21:30.299156+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:50:06.095667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.289152+0000","last_change":"2023-10-08T00:21:29.297815+0000","last_active":"2023-10-08T00:21:30.289152+0000","last_peered":"2023-10-08T00:21:30.289152+0000","last_clean":"2023-10-08T00:21:30.289152+0000","last_became_active":"2023-10-08T00:21:29.297569+0000","last_became_peered":"2023-10-08T00:21:29.297569+0000","last_unstale":"2023-10-08T00:21:30.289152+0000","last_undegraded":"2023-10-08T00:21:30.289152+0000","last_fullsized":"2023-10-08T00:21:30.289152+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:52:22.281611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.301421+0000","last_change":"2023-10-08T00:21:30.301354+0000","last_active":"2023-10-08T00:21:30.301421+0000","last_peered":"2023-10-08T00:21:30.301421+0000","last_clean":"2023-10-08T00:21:30.301421+0000","last_became_active":"2023-10-08T00:21:30.300360+0000","last_became_peered":"2023-10-08T00:21:30.300360+0000","last_unstale":"2023-10-08T00:21:30.301421+0000","last_undegraded":"2023-10-08T00:21:30.301421+0000","last_fullsized":"2023-10-08T00:21:30.301421+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:37:52.274322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.288991+0000","last_change":"2023-10-08T00:21:29.289499+0000","last_active":"2023-10-08T00:21:30.288991+0000","last_peered":"2023-10-08T00:21:30.288991+0000","last_clean":"2023-10-08T00:21:30.288991+0000","last_became_active":"2023-10-08T00:21:29.289122+0000","last_became_peered":"2023-10-08T00:21:29.289122+0000","last_unstale":"2023-10-08T00:21:30.288991+0000","last_undegraded":"2023-10-08T00:21:30.288991+0000","last_fullsized":"2023-10-08T00:21:30.288991+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:03:44.571431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.301024+0000","last_change":"2023-10-08T00:21:30.300939+0000","last_active":"2023-10-08T00:21:30.301024+0000","last_peered":"2023-10-08T00:21:30.301024+0000","last_clean":"2023-10-08T00:21:30.301024+0000","last_became_active":"2023-10-08T00:21:30.300559+0000","last_became_peered":"2023-10-08T00:21:30.300559+0000","last_unstale":"2023-10-08T00:21:30.301024+0000","last_undegraded":"2023-10-08T00:21:30.301024+0000","last_fullsized":"2023-10-08T00:21:30.301024+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:28:39.097586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.15","version":"183'3","reported_seq":13,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.311637+0000","last_change":"2023-10-08T00:21:29.294220+0000","last_active":"2023-10-08T00:21:30.311637+0000","last_peered":"2023-10-08T00:21:30.311637+0000","last_clean":"2023-10-08T00:21:30.311637+0000","last_became_active":"2023-10-08T00:21:29.293317+0000","last_became_peered":"2023-10-08T00:21:29.293317+0000","last_unstale":"2023-10-08T00:21:30.311637+0000","last_undegraded":"2023-10-08T00:21:30.311637+0000","last_fullsized":"2023-10-08T00:21:30.311637+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:30:31.143213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.298030+0000","last_change":"2023-10-08T00:21:30.297806+0000","last_active":"2023-10-08T00:21:30.298030+0000","last_peered":"2023-10-08T00:21:30.298030+0000","last_clean":"2023-10-08T00:21:30.298030+0000","last_became_active":"2023-10-08T00:21:30.297213+0000","last_became_peered":"2023-10-08T00:21:30.297213+0000","last_unstale":"2023-10-08T00:21:30.298030+0000","last_undegraded":"2023-10-08T00:21:30.298030+0000","last_fullsized":"2023-10-08T00:21:30.298030+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:26:51.687638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.14","version":"183'6","reported_seq":16,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.312148+0000","last_change":"2023-10-08T00:21:29.295488+0000","last_active":"2023-10-08T00:21:30.312148+0000","last_peered":"2023-10-08T00:21:30.312148+0000","last_clean":"2023-10-08T00:21:30.312148+0000","last_became_active":"2023-10-08T00:21:29.295261+0000","last_became_peered":"2023-10-08T00:21:29.295261+0000","last_unstale":"2023-10-08T00:21:30.312148+0000","last_undegraded":"2023-10-08T00:21:30.312148+0000","last_fullsized":"2023-10-08T00:21:30.312148+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:28:14.454425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.301657+0000","last_change":"2023-10-08T00:21:30.301516+0000","last_active":"2023-10-08T00:21:30.301657+0000","last_peered":"2023-10-08T00:21:30.301657+0000","last_clean":"2023-10-08T00:21:30.301657+0000","last_became_active":"2023-10-08T00:21:30.300667+0000","last_became_peered":"2023-10-08T00:21:30.300667+0000","last_unstale":"2023-10-08T00:21:30.301657+0000","last_undegraded":"2023-10-08T00:21:30.301657+0000","last_fullsized":"2023-10-08T00:21:30.301657+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:54:48.645640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"183'2","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.310210+0000","last_change":"2023-10-08T00:21:29.292595+0000","last_active":"2023-10-08T00:21:30.310210+0000","last_peered":"2023-10-08T00:21:30.310210+0000","last_clean":"2023-10-08T00:21:30.310210+0000","last_became_active":"2023-10-08T00:21:29.291651+0000","last_became_peered":"2023-10-08T00:21:29.291651+0000","last_unstale":"2023-10-08T00:21:30.310210+0000","last_undegraded":"2023-10-08T00:21:30.310210+0000","last_fullsized":"2023-10-08T00:21:30.310210+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:38:37.634900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.302603+0000","last_change":"2023-10-08T00:21:30.302539+0000","last_active":"2023-10-08T00:21:30.302603+0000","last_peered":"2023-10-08T00:21:30.302603+0000","last_clean":"2023-10-08T00:21:30.302603+0000","last_became_active":"2023-10-08T00:21:30.301646+0000","last_became_peered":"2023-10-08T00:21:30.301646+0000","last_unstale":"2023-10-08T00:21:30.302603+0000","last_undegraded":"2023-10-08T00:21:30.302603+0000","last_fullsized":"2023-10-08T00:21:30.302603+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:50:06.095667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.288423+0000","last_change":"2023-10-08T00:21:29.294183+0000","last_active":"2023-10-08T00:21:30.288423+0000","last_peered":"2023-10-08T00:21:30.288423+0000","last_clean":"2023-10-08T00:21:30.288423+0000","last_became_active":"2023-10-08T00:21:29.293884+0000","last_became_peered":"2023-10-08T00:21:29.293884+0000","last_unstale":"2023-10-08T00:21:30.288423+0000","last_undegraded":"2023-10-08T00:21:30.288423+0000","last_fullsized":"2023-10-08T00:21:30.288423+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:33:45.707571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.297045+0000","last_change":"2023-10-08T00:21:30.296980+0000","last_active":"2023-10-08T00:21:30.297045+0000","last_peered":"2023-10-08T00:21:30.297045+0000","last_clean":"2023-10-08T00:21:30.297045+0000","last_became_active":"2023-10-08T00:21:30.296750+0000","last_became_peered":"2023-10-08T00:21:30.296750+0000","last_unstale":"2023-10-08T00:21:30.297045+0000","last_undegraded":"2023-10-08T00:21:30.297045+0000","last_fullsized":"2023-10-08T00:21:30.297045+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:20:49.792611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.293820+0000","last_change":"2023-10-08T00:21:29.298869+0000","last_active":"2023-10-08T00:21:30.293820+0000","last_peered":"2023-10-08T00:21:30.293820+0000","last_clean":"2023-10-08T00:21:30.293820+0000","last_became_active":"2023-10-08T00:21:29.298202+0000","last_became_peered":"2023-10-08T00:21:29.298202+0000","last_unstale":"2023-10-08T00:21:30.293820+0000","last_undegraded":"2023-10-08T00:21:30.293820+0000","last_fullsized":"2023-10-08T00:21:30.293820+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:19:05.281936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.297776+0000","last_change":"2023-10-08T00:21:30.297591+0000","last_active":"2023-10-08T00:21:30.297776+0000","last_peered":"2023-10-08T00:21:30.297776+0000","last_clean":"2023-10-08T00:21:30.297776+0000","last_became_active":"2023-10-08T00:21:30.297276+0000","last_became_peered":"2023-10-08T00:21:30.297276+0000","last_unstale":"2023-10-08T00:21:30.297776+0000","last_undegraded":"2023-10-08T00:21:30.297776+0000","last_fullsized":"2023-10-08T00:21:30.297776+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:54:48.645640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.10","version":"183'8","reported_seq":18,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.311573+0000","last_change":"2023-10-08T00:21:29.294644+0000","last_active":"2023-10-08T00:21:30.311573+0000","last_peered":"2023-10-08T00:21:30.311573+0000","last_clean":"2023-10-08T00:21:30.311573+0000","last_became_active":"2023-10-08T00:21:29.294431+0000","last_became_peered":"2023-10-08T00:21:29.294431+0000","last_unstale":"2023-10-08T00:21:30.311573+0000","last_undegraded":"2023-10-08T00:21:30.311573+0000","last_fullsized":"2023-10-08T00:21:30.311573+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:17:38.121868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.301362+0000","last_change":"2023-10-08T00:21:30.301296+0000","last_active":"2023-10-08T00:21:30.301362+0000","last_peered":"2023-10-08T00:21:30.301362+0000","last_clean":"2023-10-08T00:21:30.301362+0000","last_became_active":"2023-10-08T00:21:30.300732+0000","last_became_peered":"2023-10-08T00:21:30.300732+0000","last_unstale":"2023-10-08T00:21:30.301362+0000","last_undegraded":"2023-10-08T00:21:30.301362+0000","last_fullsized":"2023-10-08T00:21:30.301362+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:33:45.741583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.f","version":"183'1","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.310865+0000","last_change":"2023-10-08T00:21:29.293776+0000","last_active":"2023-10-08T00:21:30.310865+0000","last_peered":"2023-10-08T00:21:30.310865+0000","last_clean":"2023-10-08T00:21:30.310865+0000","last_became_active":"2023-10-08T00:21:29.293558+0000","last_became_peered":"2023-10-08T00:21:29.293558+0000","last_unstale":"2023-10-08T00:21:30.310865+0000","last_undegraded":"2023-10-08T00:21:30.310865+0000","last_fullsized":"2023-10-08T00:21:30.310865+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:52:58.669407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.299733+0000","last_change":"2023-10-08T00:21:30.299678+0000","last_active":"2023-10-08T00:21:30.299733+0000","last_peered":"2023-10-08T00:21:30.299733+0000","last_clean":"2023-10-08T00:21:30.299733+0000","last_became_active":"2023-10-08T00:21:30.299306+0000","last_became_peered":"2023-10-08T00:21:30.299306+0000","last_unstale":"2023-10-08T00:21:30.299733+0000","last_undegraded":"2023-10-08T00:21:30.299733+0000","last_fullsized":"2023-10-08T00:21:30.299733+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:40:21.777719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.e","version":"183'2","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.311273+0000","last_change":"2023-10-08T00:21:29.292720+0000","last_active":"2023-10-08T00:21:30.311273+0000","last_peered":"2023-10-08T00:21:30.311273+0000","last_clean":"2023-10-08T00:21:30.311273+0000","last_became_active":"2023-10-08T00:21:29.292302+0000","last_became_peered":"2023-10-08T00:21:29.292302+0000","last_unstale":"2023-10-08T00:21:30.311273+0000","last_undegraded":"2023-10-08T00:21:30.311273+0000","last_fullsized":"2023-10-08T00:21:30.311273+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:17:05.951588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.302912+0000","last_change":"2023-10-08T00:21:30.302743+0000","last_active":"2023-10-08T00:21:30.302912+0000","last_peered":"2023-10-08T00:21:30.302912+0000","last_clean":"2023-10-08T00:21:30.302912+0000","last_became_active":"2023-10-08T00:21:30.302174+0000","last_became_peered":"2023-10-08T00:21:30.302174+0000","last_unstale":"2023-10-08T00:21:30.302912+0000","last_undegraded":"2023-10-08T00:21:30.302912+0000","last_fullsized":"2023-10-08T00:21:30.302912+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:03:08.518439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.d","version":"183'1","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.310070+0000","last_change":"2023-10-08T00:21:29.292311+0000","last_active":"2023-10-08T00:21:30.310070+0000","last_peered":"2023-10-08T00:21:30.310070+0000","last_clean":"2023-10-08T00:21:30.310070+0000","last_became_active":"2023-10-08T00:21:29.292012+0000","last_became_peered":"2023-10-08T00:21:29.292012+0000","last_unstale":"2023-10-08T00:21:30.310070+0000","last_undegraded":"2023-10-08T00:21:30.310070+0000","last_fullsized":"2023-10-08T00:21:30.310070+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:58:24.760650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.298768+0000","last_change":"2023-10-08T00:21:30.298598+0000","last_active":"2023-10-08T00:21:30.298768+0000","last_peered":"2023-10-08T00:21:30.298768+0000","last_clean":"2023-10-08T00:21:30.298768+0000","last_became_active":"2023-10-08T00:21:30.297864+0000","last_became_peered":"2023-10-08T00:21:30.297864+0000","last_unstale":"2023-10-08T00:21:30.298768+0000","last_undegraded":"2023-10-08T00:21:30.298768+0000","last_fullsized":"2023-10-08T00:21:30.298768+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:23:46.629646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.293948+0000","last_change":"2023-10-08T00:21:29.295959+0000","last_active":"2023-10-08T00:21:30.293948+0000","last_peered":"2023-10-08T00:21:30.293948+0000","last_clean":"2023-10-08T00:21:30.293948+0000","last_became_active":"2023-10-08T00:21:29.295743+0000","last_became_peered":"2023-10-08T00:21:29.295743+0000","last_unstale":"2023-10-08T00:21:30.293948+0000","last_undegraded":"2023-10-08T00:21:30.293948+0000","last_fullsized":"2023-10-08T00:21:30.293948+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:45:30.775065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.301681+0000","last_change":"2023-10-08T00:21:30.301603+0000","last_active":"2023-10-08T00:21:30.301681+0000","last_peered":"2023-10-08T00:21:30.301681+0000","last_clean":"2023-10-08T00:21:30.301681+0000","last_became_active":"2023-10-08T00:21:30.301299+0000","last_became_peered":"2023-10-08T00:21:30.301299+0000","last_unstale":"2023-10-08T00:21:30.301681+0000","last_undegraded":"2023-10-08T00:21:30.301681+0000","last_fullsized":"2023-10-08T00:21:30.301681+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:04:05.010208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.b","version":"183'2","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.310692+0000","last_change":"2023-10-08T00:21:29.297803+0000","last_active":"2023-10-08T00:21:30.310692+0000","last_peered":"2023-10-08T00:21:30.310692+0000","last_clean":"2023-10-08T00:21:30.310692+0000","last_became_active":"2023-10-08T00:21:29.297560+0000","last_became_peered":"2023-10-08T00:21:29.297560+0000","last_unstale":"2023-10-08T00:21:30.310692+0000","last_undegraded":"2023-10-08T00:21:30.310692+0000","last_fullsized":"2023-10-08T00:21:30.310692+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:17:05.951588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.298426+0000","last_change":"2023-10-08T00:21:30.298192+0000","last_active":"2023-10-08T00:21:30.298426+0000","last_peered":"2023-10-08T00:21:30.298426+0000","last_clean":"2023-10-08T00:21:30.298426+0000","last_became_active":"2023-10-08T00:21:30.297166+0000","last_became_peered":"2023-10-08T00:21:30.297166+0000","last_unstale":"2023-10-08T00:21:30.298426+0000","last_undegraded":"2023-10-08T00:21:30.298426+0000","last_fullsized":"2023-10-08T00:21:30.298426+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:02:23.529002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.291935+0000","last_change":"2023-10-08T00:21:29.292681+0000","last_active":"2023-10-08T00:21:30.291935+0000","last_peered":"2023-10-08T00:21:30.291935+0000","last_clean":"2023-10-08T00:21:30.291935+0000","last_became_active":"2023-10-08T00:21:29.291478+0000","last_became_peered":"2023-10-08T00:21:29.291478+0000","last_unstale":"2023-10-08T00:21:30.291935+0000","last_undegraded":"2023-10-08T00:21:30.291935+0000","last_fullsized":"2023-10-08T00:21:30.291935+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:36:41.035968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.301495+0000","last_change":"2023-10-08T00:21:30.301347+0000","last_active":"2023-10-08T00:21:30.301495+0000","last_peered":"2023-10-08T00:21:30.301495+0000","last_clean":"2023-10-08T00:21:30.301495+0000","last_became_active":"2023-10-08T00:21:30.300552+0000","last_became_peered":"2023-10-08T00:21:30.300552+0000","last_unstale":"2023-10-08T00:21:30.301495+0000","last_undegraded":"2023-10-08T00:21:30.301495+0000","last_fullsized":"2023-10-08T00:21:30.301495+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:22:50.229264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.290336+0000","last_change":"2023-10-08T00:21:29.292747+0000","last_active":"2023-10-08T00:21:30.290336+0000","last_peered":"2023-10-08T00:21:30.290336+0000","last_clean":"2023-10-08T00:21:30.290336+0000","last_became_active":"2023-10-08T00:21:29.292039+0000","last_became_peered":"2023-10-08T00:21:29.292039+0000","last_unstale":"2023-10-08T00:21:30.290336+0000","last_undegraded":"2023-10-08T00:21:30.290336+0000","last_fullsized":"2023-10-08T00:21:30.290336+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:17:05.951588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.301612+0000","last_change":"2023-10-08T00:21:30.301419+0000","last_active":"2023-10-08T00:21:30.301612+0000","last_peered":"2023-10-08T00:21:30.301612+0000","last_clean":"2023-10-08T00:21:30.301612+0000","last_became_active":"2023-10-08T00:21:30.301056+0000","last_became_peered":"2023-10-08T00:21:30.301056+0000","last_unstale":"2023-10-08T00:21:30.301612+0000","last_undegraded":"2023-10-08T00:21:30.301612+0000","last_fullsized":"2023-10-08T00:21:30.301612+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:25:58.325029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.293597+0000","last_change":"2023-10-08T00:21:29.293623+0000","last_active":"2023-10-08T00:21:30.293597+0000","last_peered":"2023-10-08T00:21:30.293597+0000","last_clean":"2023-10-08T00:21:30.293597+0000","last_became_active":"2023-10-08T00:21:29.293216+0000","last_became_peered":"2023-10-08T00:21:29.293216+0000","last_unstale":"2023-10-08T00:21:30.293597+0000","last_undegraded":"2023-10-08T00:21:30.293597+0000","last_fullsized":"2023-10-08T00:21:30.293597+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:26:38.276733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.292110+0000","last_change":"2023-10-08T00:21:30.292044+0000","last_active":"2023-10-08T00:21:30.292110+0000","last_peered":"2023-10-08T00:21:30.292110+0000","last_clean":"2023-10-08T00:21:30.292110+0000","last_became_active":"2023-10-08T00:21:30.291731+0000","last_became_peered":"2023-10-08T00:21:30.291731+0000","last_unstale":"2023-10-08T00:21:30.292110+0000","last_undegraded":"2023-10-08T00:21:30.292110+0000","last_fullsized":"2023-10-08T00:21:30.292110+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:33:45.741583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.7","version":"183'2","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.309329+0000","last_change":"2023-10-08T00:21:29.289352+0000","last_active":"2023-10-08T00:21:30.309329+0000","last_peered":"2023-10-08T00:21:30.309329+0000","last_clean":"2023-10-08T00:21:30.309329+0000","last_became_active":"2023-10-08T00:21:29.288764+0000","last_became_peered":"2023-10-08T00:21:29.288764+0000","last_unstale":"2023-10-08T00:21:30.309329+0000","last_undegraded":"2023-10-08T00:21:30.309329+0000","last_fullsized":"2023-10-08T00:21:30.309329+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:42:06.385422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.302689+0000","last_change":"2023-10-08T00:21:30.302628+0000","last_active":"2023-10-08T00:21:30.302689+0000","last_peered":"2023-10-08T00:21:30.302689+0000","last_clean":"2023-10-08T00:21:30.302689+0000","last_became_active":"2023-10-08T00:21:30.298293+0000","last_became_peered":"2023-10-08T00:21:30.298293+0000","last_unstale":"2023-10-08T00:21:30.302689+0000","last_undegraded":"2023-10-08T00:21:30.302689+0000","last_fullsized":"2023-10-08T00:21:30.302689+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:33:45.741583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.293442+0000","last_change":"2023-10-08T00:21:29.297973+0000","last_active":"2023-10-08T00:21:30.293442+0000","last_peered":"2023-10-08T00:21:30.293442+0000","last_clean":"2023-10-08T00:21:30.293442+0000","last_became_active":"2023-10-08T00:21:29.297747+0000","last_became_peered":"2023-10-08T00:21:29.297747+0000","last_unstale":"2023-10-08T00:21:30.293442+0000","last_undegraded":"2023-10-08T00:21:30.293442+0000","last_fullsized":"2023-10-08T00:21:30.293442+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:09:10.597389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.298875+0000","last_change":"2023-10-08T00:21:30.297642+0000","last_active":"2023-10-08T00:21:30.298875+0000","last_peered":"2023-10-08T00:21:30.298875+0000","last_clean":"2023-10-08T00:21:30.298875+0000","last_became_active":"2023-10-08T00:21:30.297410+0000","last_became_peered":"2023-10-08T00:21:30.297410+0000","last_unstale":"2023-10-08T00:21:30.298875+0000","last_undegraded":"2023-10-08T00:21:30.298875+0000","last_fullsized":"2023-10-08T00:21:30.298875+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:54:48.645640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.291816+0000","last_change":"2023-10-08T00:21:29.290597+0000","last_active":"2023-10-08T00:21:30.291816+0000","last_peered":"2023-10-08T00:21:30.291816+0000","last_clean":"2023-10-08T00:21:30.291816+0000","last_became_active":"2023-10-08T00:21:29.290091+0000","last_became_peered":"2023-10-08T00:21:29.290091+0000","last_unstale":"2023-10-08T00:21:30.291816+0000","last_undegraded":"2023-10-08T00:21:30.291816+0000","last_fullsized":"2023-10-08T00:21:30.291816+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:22:06.107501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.300190+0000","last_change":"2023-10-08T00:21:30.300125+0000","last_active":"2023-10-08T00:21:30.300190+0000","last_peered":"2023-10-08T00:21:30.300190+0000","last_clean":"2023-10-08T00:21:30.300190+0000","last_became_active":"2023-10-08T00:21:30.299898+0000","last_became_peered":"2023-10-08T00:21:30.299898+0000","last_unstale":"2023-10-08T00:21:30.300190+0000","last_undegraded":"2023-10-08T00:21:30.300190+0000","last_fullsized":"2023-10-08T00:21:30.300190+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:22:50.229264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.289248+0000","last_change":"2023-10-08T00:21:29.295528+0000","last_active":"2023-10-08T00:21:30.289248+0000","last_peered":"2023-10-08T00:21:30.289248+0000","last_clean":"2023-10-08T00:21:30.289248+0000","last_became_active":"2023-10-08T00:21:29.295272+0000","last_became_peered":"2023-10-08T00:21:29.295272+0000","last_unstale":"2023-10-08T00:21:30.289248+0000","last_undegraded":"2023-10-08T00:21:30.289248+0000","last_fullsized":"2023-10-08T00:21:30.289248+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:20:48.787420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.294281+0000","last_change":"2023-10-08T00:21:30.293908+0000","last_active":"2023-10-08T00:21:30.294281+0000","last_peered":"2023-10-08T00:21:30.294281+0000","last_clean":"2023-10-08T00:21:30.294281+0000","last_became_active":"2023-10-08T00:21:30.293049+0000","last_became_peered":"2023-10-08T00:21:30.293049+0000","last_unstale":"2023-10-08T00:21:30.294281+0000","last_undegraded":"2023-10-08T00:21:30.294281+0000","last_fullsized":"2023-10-08T00:21:30.294281+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:17:06.956780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.296061+0000","last_change":"2023-10-08T00:21:29.292200+0000","last_active":"2023-10-08T00:21:30.296061+0000","last_peered":"2023-10-08T00:21:30.296061+0000","last_clean":"2023-10-08T00:21:30.296061+0000","last_became_active":"2023-10-08T00:21:29.291715+0000","last_became_peered":"2023-10-08T00:21:29.291715+0000","last_unstale":"2023-10-08T00:21:30.296061+0000","last_undegraded":"2023-10-08T00:21:30.296061+0000","last_fullsized":"2023-10-08T00:21:30.296061+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:57:21.248136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.302308+0000","last_change":"2023-10-08T00:21:30.302254+0000","last_active":"2023-10-08T00:21:30.302308+0000","last_peered":"2023-10-08T00:21:30.302308+0000","last_clean":"2023-10-08T00:21:30.302308+0000","last_became_active":"2023-10-08T00:21:30.302082+0000","last_became_peered":"2023-10-08T00:21:30.302082+0000","last_unstale":"2023-10-08T00:21:30.302308+0000","last_undegraded":"2023-10-08T00:21:30.302308+0000","last_fullsized":"2023-10-08T00:21:30.302308+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:02:23.529002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.293919+0000","last_change":"2023-10-08T00:21:29.298981+0000","last_active":"2023-10-08T00:21:30.293919+0000","last_peered":"2023-10-08T00:21:30.293919+0000","last_clean":"2023-10-08T00:21:30.293919+0000","last_became_active":"2023-10-08T00:21:29.298583+0000","last_became_peered":"2023-10-08T00:21:29.298583+0000","last_unstale":"2023-10-08T00:21:30.293919+0000","last_undegraded":"2023-10-08T00:21:30.293919+0000","last_fullsized":"2023-10-08T00:21:30.293919+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:44:14.951665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.300866+0000","last_change":"2023-10-08T00:21:30.300769+0000","last_active":"2023-10-08T00:21:30.300866+0000","last_peered":"2023-10-08T00:21:30.300866+0000","last_clean":"2023-10-08T00:21:30.300866+0000","last_became_active":"2023-10-08T00:21:30.300283+0000","last_became_peered":"2023-10-08T00:21:30.300283+0000","last_unstale":"2023-10-08T00:21:30.300866+0000","last_undegraded":"2023-10-08T00:21:30.300866+0000","last_fullsized":"2023-10-08T00:21:30.300866+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:02:23.529002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.294158+0000","last_change":"2023-10-08T00:21:29.297715+0000","last_active":"2023-10-08T00:21:30.294158+0000","last_peered":"2023-10-08T00:21:30.294158+0000","last_clean":"2023-10-08T00:21:30.294158+0000","last_became_active":"2023-10-08T00:21:29.297200+0000","last_became_peered":"2023-10-08T00:21:29.297200+0000","last_unstale":"2023-10-08T00:21:30.294158+0000","last_undegraded":"2023-10-08T00:21:30.294158+0000","last_fullsized":"2023-10-08T00:21:30.294158+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:17:05.951588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.298043+0000","last_change":"2023-10-08T00:21:30.297937+0000","last_active":"2023-10-08T00:21:30.298043+0000","last_peered":"2023-10-08T00:21:30.298043+0000","last_clean":"2023-10-08T00:21:30.298043+0000","last_became_active":"2023-10-08T00:21:30.297478+0000","last_became_peered":"2023-10-08T00:21:30.297478+0000","last_unstale":"2023-10-08T00:21:30.298043+0000","last_undegraded":"2023-10-08T00:21:30.298043+0000","last_fullsized":"2023-10-08T00:21:30.298043+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:18:43.290407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.291900+0000","last_change":"2023-10-08T00:21:29.294934+0000","last_active":"2023-10-08T00:21:30.291900+0000","last_peered":"2023-10-08T00:21:30.291900+0000","last_clean":"2023-10-08T00:21:30.291900+0000","last_became_active":"2023-10-08T00:21:29.290398+0000","last_became_peered":"2023-10-08T00:21:29.290398+0000","last_unstale":"2023-10-08T00:21:30.291900+0000","last_undegraded":"2023-10-08T00:21:30.291900+0000","last_fullsized":"2023-10-08T00:21:30.291900+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:07:16.290177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.299947+0000","last_change":"2023-10-08T00:21:30.299891+0000","last_active":"2023-10-08T00:21:30.299947+0000","last_peered":"2023-10-08T00:21:30.299947+0000","last_clean":"2023-10-08T00:21:30.299947+0000","last_became_active":"2023-10-08T00:21:30.298860+0000","last_became_peered":"2023-10-08T00:21:30.298860+0000","last_unstale":"2023-10-08T00:21:30.299947+0000","last_undegraded":"2023-10-08T00:21:30.299947+0000","last_fullsized":"2023-10-08T00:21:30.299947+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:29.279407+0000","last_clean_scrub_stamp":"2023-10-08T00:21:29.279407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:40:21.777719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.294097+0000","last_change":"2023-10-08T00:21:29.298156+0000","last_active":"2023-10-08T00:21:30.294097+0000","last_peered":"2023-10-08T00:21:30.294097+0000","last_clean":"2023-10-08T00:21:30.294097+0000","last_became_active":"2023-10-08T00:21:29.297947+0000","last_became_peered":"2023-10-08T00:21:29.297947+0000","last_unstale":"2023-10-08T00:21:30.294097+0000","last_undegraded":"2023-10-08T00:21:30.294097+0000","last_fullsized":"2023-10-08T00:21:30.294097+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:21:28.274215+0000","last_clean_scrub_stamp":"2023-10-08T00:21:28.274215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:34:18.411604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":304,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T00:21:30.293003+0000","last_change":"2023-10-08T00:10:43.728490+0000","last_active":"2023-10-08T00:21:30.293003+0000","last_peered":"2023-10-08T00:21:30.293003+0000","last_clean":"2023-10-08T00:21:30.293003+0000","last_became_active":"2023-10-08T00:10:43.728341+0000","last_became_peered":"2023-10-08T00:10:43.728341+0000","last_unstale":"2023-10-08T00:21:30.293003+0000","last_undegraded":"2023-10-08T00:21:30.293003+0000","last_fullsized":"2023-10-08T00:21:30.293003+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_clean_scrub_stamp":"2023-10-08T00:10:42.716096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:06:31.035293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574980,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1008,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194303,"statfs":{"total":200000000,"available":198966926,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52700000000000002,"15min":0.51800000000000002},"min":{"1min":0.373,"5min":0.27800000000000002,"15min":0.26900000000000002},"max":{"1min":0.69899999999999995,"5min":0.77600000000000002,"15min":1.129},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55100000000000005,"15min":0.54700000000000004},"min":{"1min":0.35199999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.754,"5min":0.754,"15min":1.0269999999999999},"last":0.73799999999999999}]},{"osd":1,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57599999999999996,"15min":0.55400000000000005},"min":{"1min":0.42799999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.73799999999999999,"5min":0.77900000000000003,"15min":0.82999999999999996},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.58999999999999997,"15min":0.57799999999999996},"min":{"1min":0.44,"5min":0.39300000000000002,"15min":0.38200000000000001},"max":{"1min":0.76200000000000001,"5min":0.79400000000000004,"15min":0.80100000000000005},"last":0.62}]},{"osd":2,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56299999999999994,"15min":0.54900000000000004},"min":{"1min":0.41999999999999998,"5min":0.38,"15min":0.34200000000000003},"max":{"1min":0.72599999999999998,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54700000000000004,"15min":0.53700000000000003},"min":{"1min":0.38,"5min":0.34499999999999997,"15min":0.32900000000000001},"max":{"1min":0.748,"5min":0.748,"15min":0.78500000000000003},"last":0.437}]},{"osd":3,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55500000000000005,"15min":0.53700000000000003},"min":{"1min":0.373,"5min":0.36699999999999999,"15min":0.33700000000000002},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.79300000000000004},"last":0.753},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56399999999999995},"min":{"1min":0.46999999999999997,"5min":0.371,"15min":0.371},"max":{"1min":0.71199999999999997,"5min":0.77600000000000002,"15min":0.82099999999999995},"last":0.65000000000000002}]},{"osd":4,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.47299999999999998,"15min":0.499},"min":{"1min":0.307,"5min":0.27200000000000002,"15min":0.26400000000000001},"max":{"1min":0.69999999999999996,"5min":0.755,"15min":0.755},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.50900000000000001,"15min":0.51500000000000001},"min":{"1min":0.33500000000000002,"5min":0.29199999999999998,"15min":0.28199999999999997},"max":{"1min":0.71599999999999997,"5min":0.77400000000000002,"15min":0.78900000000000003},"last":0.69899999999999995}]},{"osd":5,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.51000000000000001,"15min":0.52800000000000002},"min":{"1min":0.35699999999999998,"5min":0.27800000000000002,"15min":0.254},"max":{"1min":0.73499999999999999,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.54200000000000004,"15min":0.56100000000000005},"min":{"1min":0.38900000000000001,"5min":0.32400000000000001,"15min":0.29099999999999998},"max":{"1min":0.748,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.71599999999999997}]},{"osd":6,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.59399999999999997,"15min":0.59099999999999997},"min":{"1min":0.42099999999999999,"5min":0.39600000000000002,"15min":0.33100000000000002},"max":{"1min":0.81299999999999994,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.58699999999999997,"15min":0.60599999999999998},"min":{"1min":0.40899999999999997,"5min":0.33300000000000002,"15min":0.27800000000000002},"max":{"1min":0.78800000000000003,"5min":0.82999999999999996,"15min":0.91500000000000004},"last":0.82399999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574980,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1008,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194303,"statfs":{"total":200000000,"available":198966892,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53100000000000003,"15min":0.50600000000000001},"min":{"1min":0.42399999999999999,"5min":0.32300000000000001,"15min":0.29199999999999998},"max":{"1min":0.74299999999999999,"5min":0.76300000000000001,"15min":1.012},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.53500000000000003,"15min":0.60699999999999998},"min":{"1min":0.38400000000000001,"5min":0.317,"15min":0.29199999999999998},"max":{"1min":0.65800000000000003,"5min":0.77800000000000002,"15min":5.1230000000000002},"last":0.499}]},{"osd":1,"last update":"Sun Oct 8 00:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.55700000000000005,"15min":0.54000000000000004},"min":{"1min":0.36499999999999999,"5min":0.29899999999999999,"15min":0.28899999999999998},"max":{"1min":0.754,"5min":0.78100000000000003,"15min":1.1419999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.59399999999999997,"15min":0.57699999999999996},"min":{"1min":0.51300000000000001,"5min":0.41099999999999998,"15min":0.32100000000000001},"max":{"1min":0.81000000000000005,"5min":0.81799999999999995,"15min":1.1539999999999999},"last":0.51600000000000001}]},{"osd":2,"last update":"Sun Oct 8 00:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.55400000000000005,"15min":0.54200000000000004},"min":{"1min":0.47199999999999998,"5min":0.36299999999999999,"15min":0.32900000000000001},"max":{"1min":0.74099999999999999,"5min":0.76500000000000001,"15min":1.3149999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58299999999999996,"15min":0.57599999999999996},"min":{"1min":0.441,"5min":0.374,"15min":0.29399999999999998},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":1.3280000000000001},"last":0.64700000000000002}]},{"osd":3,"last update":"Sun Oct 8 00:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.56100000000000005,"15min":0.55400000000000005},"min":{"1min":0.35499999999999998,"5min":0.32000000000000001,"15min":0.25700000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":1.046},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.58099999999999996,"15min":0.56100000000000005},"min":{"1min":0.501,"5min":0.38600000000000001,"15min":0.32900000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":1.4390000000000001},"last":0.66300000000000003}]},{"osd":4,"last update":"Sun Oct 8 00:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46200000000000002,"15min":0.48099999999999998},"min":{"1min":0.30499999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.622,"5min":0.70099999999999996,"15min":0.80100000000000005},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55000000000000004,"15min":0.55700000000000005},"min":{"1min":0.41999999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.77600000000000002,"5min":0.79500000000000004,"15min":0.89100000000000001},"last":0.47499999999999998}]},{"osd":5,"last update":"Sun Oct 8 00:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.47699999999999998,"15min":0.48299999999999998},"min":{"1min":0.36099999999999999,"5min":0.29299999999999998,"15min":0.29099999999999998},"max":{"1min":0.73499999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.505,"15min":0.53200000000000003},"min":{"1min":0.35299999999999998,"5min":0.313,"15min":0.28699999999999998},"max":{"1min":0.70099999999999996,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.42699999999999999}]},{"osd":7,"last update":"Sun Oct 8 00:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.50600000000000001,"15min":0.52600000000000002},"min":{"1min":0.38100000000000001,"5min":0.32800000000000001,"15min":0.307},"max":{"1min":0.753,"5min":0.82899999999999996,"15min":0.91500000000000004},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53200000000000003,"15min":0.54200000000000004},"min":{"1min":0.39000000000000001,"5min":0.34899999999999998,"15min":0.32400000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.79900000000000004},"last":0.45700000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574980,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1587,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193725,"statfs":{"total":200000000,"available":198374840,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:20:54 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.35599999999999998,"15min":0.32900000000000001},"min":{"1min":0.24199999999999999,"5min":0.215,"15min":0.19500000000000001},"max":{"1min":0.495,"5min":0.55900000000000005,"15min":0.56899999999999995},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.41399999999999998,"15min":0.39300000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.255},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.67300000000000004},"last":0.34200000000000003}]},{"osd":2,"last update":"Sun Oct 8 00:20:54 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.378,"15min":0.35199999999999998},"min":{"1min":0.28100000000000003,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.58499999999999996,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.38900000000000001,"15min":0.36399999999999999},"min":{"1min":0.24299999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.55100000000000005,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.28999999999999998}]},{"osd":3,"last update":"Sun Oct 8 00:20:54 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.41699999999999998,"15min":0.39000000000000001},"min":{"1min":0.29799999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.63,"5min":0.63,"15min":0.70999999999999996},"last":0.372},{"interface":"front","average":{"1min":0.375,"5min":0.371,"15min":0.35499999999999998},"min":{"1min":0.26400000000000001,"5min":0.249,"15min":0.222},"max":{"1min":0.51900000000000002,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.47899999999999998}]},{"osd":4,"last update":"Sun Oct 8 00:20:54 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.56399999999999995,"15min":0.55200000000000005},"min":{"1min":0.39300000000000002,"5min":0.34000000000000002,"15min":0.32600000000000001},"max":{"1min":0.81000000000000005,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.58899999999999997,"15min":0.57799999999999996},"min":{"1min":0.442,"5min":0.37,"15min":0.37},"max":{"1min":0.82799999999999996,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.48799999999999999}]},{"osd":5,"last update":"Sun Oct 8 00:20:54 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58199999999999996,"15min":0.55700000000000005},"min":{"1min":0.40200000000000002,"5min":0.35099999999999998,"15min":0.33500000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.58599999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.30399999999999999},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.41799999999999998}]},{"osd":6,"last update":"Sun Oct 8 00:20:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.58399999999999996,"15min":0.57699999999999996},"min":{"1min":0.40600000000000003,"5min":0.37,"15min":0.317},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58599999999999997,"15min":0.57399999999999995},"min":{"1min":0.42699999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84099999999999997,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.40799999999999997}]},{"osd":7,"last update":"Sun Oct 8 00:20:54 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58399999999999996,"15min":0.57499999999999996},"min":{"1min":0.438,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.502},{"interface":"front","average":{"1min":0.624,"5min":0.626,"15min":0.60799999999999998},"min":{"1min":0.42299999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.85499999999999998,"5min":0.872,"15min":0.96699999999999997},"last":0.495}]}]},{"osd":0,"up_from":13,"seq":55834574980,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1008,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194303,"statfs":{"total":200000000,"available":198966856,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 00:21:06 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.33800000000000002,"15min":0.33500000000000002},"min":{"1min":0.23400000000000001,"5min":0.20399999999999999,"15min":0.193},"max":{"1min":0.45600000000000002,"5min":0.54900000000000004,"15min":1.0640000000000001},"last":0.253},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.36199999999999999,"15min":0.35399999999999998},"min":{"1min":0.251,"5min":0.216,"15min":0.20300000000000001},"max":{"1min":0.433,"5min":0.66500000000000004,"15min":0.87},"last":0.375}]},{"osd":2,"last update":"Sun Oct 8 00:21:06 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.38500000000000001,"15min":0.38700000000000001},"min":{"1min":0.29399999999999998,"5min":0.249,"15min":0.24099999999999999},"max":{"1min":0.53200000000000003,"5min":0.57199999999999995,"15min":0.93799999999999994},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.35899999999999999,"15min":0.56100000000000005},"min":{"1min":0.25600000000000001,"5min":0.23000000000000001,"15min":0.185},"max":{"1min":0.54800000000000004,"5min":0.59899999999999998,"15min":12.019},"last":0.27600000000000002}]},{"osd":3,"last update":"Sun Oct 8 00:21:06 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.38200000000000001,"15min":0.371},"min":{"1min":0.25700000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.56200000000000006,"5min":0.628,"15min":1.1020000000000001},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.41599999999999998,"15min":0.40699999999999997},"min":{"1min":0.31900000000000001,"5min":0.26700000000000002,"15min":0.246},"max":{"1min":0.51200000000000001,"5min":0.66100000000000003,"15min":1.0409999999999999},"last":0.40000000000000002}]},{"osd":4,"last update":"Sun Oct 8 00:21:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.61899999999999999,"15min":0.61699999999999999},"min":{"1min":0.39500000000000002,"5min":0.377,"15min":0.35899999999999999},"max":{"1min":0.872,"5min":0.89700000000000002,"15min":1.1779999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.621,"15min":0.624},"min":{"1min":0.44900000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89300000000000002,"5min":0.93899999999999995,"15min":0.96199999999999997},"last":0.56899999999999995}]},{"osd":5,"last update":"Sun Oct 8 00:21:06 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.58799999999999997,"15min":0.79400000000000004},"min":{"1min":0.42399999999999999,"5min":0.378,"15min":0.32000000000000001},"max":{"1min":0.878,"5min":0.90700000000000003,"15min":11.992000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.60599999999999998,"15min":0.59599999999999997},"min":{"1min":0.40999999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.80600000000000005,"5min":0.91600000000000004,"15min":1.1299999999999999},"last":0.443}]},{"osd":6,"last update":"Sun Oct 8 00:21:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60099999999999998,"15min":0.59399999999999997},"min":{"1min":0.38300000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.85699999999999998,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.497},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.60899999999999999,"15min":0.79300000000000004},"min":{"1min":0.42399999999999999,"5min":0.39600000000000002,"15min":0.38100000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":11.927},"last":0.48499999999999999}]},{"osd":7,"last update":"Sun Oct 8 00:21:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.60099999999999998,"15min":0.80300000000000005},"min":{"1min":0.39900000000000002,"5min":0.38600000000000001,"15min":0.35699999999999998},"max":{"1min":0.84399999999999997,"5min":0.95099999999999996,"15min":11.965},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.63500000000000001,"15min":0.628},"min":{"1min":0.45800000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":1.1539999999999999},"last":0.46999999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574980,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1009,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194303,"statfs":{"total":200000000,"available":198966372,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.29899999999999999,"5min":0.33200000000000002,"15min":0.36399999999999999},"min":{"1min":0.17299999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":2.5139999999999998},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.34699999999999998,"15min":0.375},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.20599999999999999},"max":{"1min":0.45800000000000002,"5min":0.59099999999999997,"15min":2.1309999999999998},"last":0.27900000000000003}]},{"osd":1,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.36099999999999999,"15min":0.376},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.49099999999999999,"5min":0.67300000000000004,"15min":2.2639999999999998},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.379,"15min":0.40799999999999997},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.48599999999999999,"5min":0.64400000000000002,"15min":2.5379999999999998},"last":0.254}]},{"osd":3,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.40300000000000002,"15min":0.41499999999999998},"min":{"1min":0.24199999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.58299999999999996,"5min":0.66900000000000004,"15min":2.2890000000000001},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.42299999999999999,"15min":0.442},"min":{"1min":0.32000000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.64300000000000002,"5min":0.68799999999999994,"15min":2.5619999999999998},"last":0.31900000000000001}]},{"osd":4,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.54700000000000004,"15min":0.58699999999999997},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.68500000000000005,"5min":0.80800000000000005,"15min":2.488},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.58499999999999996,"15min":0.59699999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.80300000000000005,"5min":0.92400000000000004,"15min":2.073},"last":0.48599999999999999}]},{"osd":5,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.60999999999999999,"15min":0.63600000000000001},"min":{"1min":0.45900000000000002,"5min":0.374,"15min":0.36099999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":2.2069999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.61899999999999999,"15min":0.64500000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88,"5min":0.94199999999999995,"15min":2.456},"last":0.5}]},{"osd":6,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.59299999999999997,"15min":0.60999999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.77300000000000002,"5min":0.93000000000000005,"15min":2.0339999999999998},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.59899999999999998,"15min":0.60799999999999998},"min":{"1min":0.42799999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.70399999999999996,"5min":0.94099999999999995,"15min":2.1699999999999999},"last":0.432}]},{"osd":7,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.60499999999999998,"15min":0.623},"min":{"1min":0.40799999999999997,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.751,"5min":0.95999999999999996,"15min":2.1030000000000002},"last":0.441},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.61199999999999999,"15min":0.621},"min":{"1min":0.44900000000000001,"5min":0.40699999999999997,"15min":0.40300000000000002},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":2.2349999999999999},"last":0.39500000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574980,"num_pgs":14,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1010,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194302,"statfs":{"total":200000000,"available":198965368,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:20:54 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33600000000000002,"15min":0.35199999999999998},"min":{"1min":0.24399999999999999,"5min":0.21299999999999999,"15min":0.192},"max":{"1min":0.48399999999999999,"5min":0.59299999999999997,"15min":1.46},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.34999999999999998,"15min":0.34399999999999997},"min":{"1min":0.26500000000000001,"5min":0.23799999999999999,"15min":0.192},"max":{"1min":0.46600000000000003,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.34200000000000003}]},{"osd":1,"last update":"Sun Oct 8 00:20:54 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.35899999999999999,"15min":0.59599999999999997},"min":{"1min":0.247,"5min":0.23699999999999999,"15min":0.20000000000000001},"max":{"1min":0.44800000000000001,"5min":0.57499999999999996,"15min":12.351000000000001},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38200000000000001,"15min":0.371},"min":{"1min":0.25700000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.51500000000000001,"5min":0.58199999999999996,"15min":0.71999999999999997},"last":0.29599999999999999}]},{"osd":2,"last update":"Sun Oct 8 00:20:54 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40500000000000003,"15min":0.40999999999999998},"min":{"1min":0.33300000000000002,"5min":0.26800000000000002,"15min":0.23499999999999999},"max":{"1min":0.53400000000000003,"5min":0.59699999999999998,"15min":1.5489999999999999},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41299999999999998,"15min":0.39700000000000002},"min":{"1min":0.34999999999999998,"5min":0.25700000000000001,"15min":0.246},"max":{"1min":0.54900000000000004,"5min":0.64900000000000002,"15min":0.753},"last":0.38300000000000001}]},{"osd":4,"last update":"Sun Oct 8 00:20:54 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.81899999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.34899999999999998},"max":{"1min":0.90300000000000002,"5min":0.96699999999999997,"15min":11.333},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.63,"15min":0.83899999999999997},"min":{"1min":0.46999999999999997,"5min":0.38100000000000001,"15min":0.36099999999999999},"max":{"1min":0.88500000000000001,"5min":0.97699999999999998,"15min":12.321999999999999},"last":0.47399999999999998}]},{"osd":5,"last update":"Sun Oct 8 00:20:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.61199999999999999,"15min":0.83799999999999997},"min":{"1min":0.44400000000000001,"5min":0.40600000000000003,"15min":0.35499999999999998},"max":{"1min":0.81000000000000005,"5min":0.93500000000000005,"15min":12.291},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64000000000000001,"15min":0.61799999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.89800000000000002,"5min":1.143,"15min":1.143},"last":0.51100000000000001}]},{"osd":6,"last update":"Sun Oct 8 00:20:54 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60299999999999998,"15min":0.79000000000000004},"min":{"1min":0.41599999999999998,"5min":0.40100000000000002,"15min":0.33700000000000002},"max":{"1min":0.91300000000000003,"5min":0.91500000000000004,"15min":11.288},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60799999999999998,"15min":0.82699999999999996},"min":{"1min":0.42299999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.83599999999999997,"5min":0.94599999999999995,"15min":12.253},"last":0.44700000000000001}]},{"osd":7,"last update":"Sun Oct 8 00:20:54 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.627,"15min":0.59899999999999998},"min":{"1min":0.443,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.88,"5min":1.0920000000000001,"15min":1.3109999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.621,"5min":0.60899999999999999,"15min":0.81799999999999995},"min":{"1min":0.434,"5min":0.39700000000000002,"15min":0.38600000000000001},"max":{"1min":0.86199999999999999,"5min":0.89300000000000002,"15min":11.363},"last":0.55600000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574980,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1008,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194303,"statfs":{"total":200000000,"available":198966892,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.56999999999999995,"15min":0.56299999999999994},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.193},"max":{"1min":0.67500000000000004,"5min":0.84799999999999998,"15min":0.93000000000000005},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.60299999999999998,"15min":0.58799999999999997},"min":{"1min":0.434,"5min":0.40600000000000003,"15min":0.30399999999999999},"max":{"1min":0.71099999999999997,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.48399999999999999}]},{"osd":1,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.54600000000000004,"15min":0.53000000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.33700000000000002},"max":{"1min":0.67200000000000004,"5min":0.76400000000000001,"15min":0.78100000000000003},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.56299999999999994,"15min":0.54300000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.30199999999999999},"max":{"1min":0.67600000000000005,"5min":0.80600000000000005,"15min":0.81299999999999994},"last":0.46899999999999997}]},{"osd":2,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.54100000000000004,"15min":0.52600000000000002},"min":{"1min":0.33800000000000002,"5min":0.32500000000000001,"15min":0.32100000000000001},"max":{"1min":0.64700000000000002,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.56399999999999995,"15min":0.53200000000000003},"min":{"1min":0.432,"5min":0.35399999999999998,"15min":0.32600000000000001},"max":{"1min":0.65400000000000003,"5min":0.79600000000000004,"15min":0.81699999999999995},"last":0.53600000000000003}]},{"osd":3,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.56799999999999995,"15min":0.54900000000000004},"min":{"1min":0.41099999999999998,"5min":0.33700000000000002,"15min":0.28799999999999998},"max":{"1min":0.71099999999999997,"5min":0.82299999999999995,"15min":0.85399999999999998},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.55400000000000005,"15min":0.54100000000000004},"min":{"1min":0.38100000000000001,"5min":0.37,"15min":0.36199999999999999},"max":{"1min":0.64700000000000002,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.44600000000000001}]},{"osd":5,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.47499999999999998,"15min":0.48299999999999998},"min":{"1min":0.33600000000000002,"5min":0.314,"15min":0.27500000000000002},"max":{"1min":0.65500000000000003,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.501,"15min":0.504},"min":{"1min":0.317,"5min":0.315,"15min":0.24399999999999999},"max":{"1min":0.64400000000000002,"5min":0.79100000000000004,"15min":0.94899999999999995},"last":0.52400000000000002}]},{"osd":6,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.26400000000000001},"max":{"1min":0.66500000000000004,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.53000000000000003,"15min":0.53900000000000003},"min":{"1min":0.35499999999999998,"5min":0.34799999999999998,"15min":0.29799999999999999},"max":{"1min":0.65600000000000003,"5min":0.76200000000000001,"15min":0.80100000000000005},"last":0.42799999999999999}]},{"osd":7,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.53500000000000003,"15min":0.54300000000000004},"min":{"1min":0.41299999999999998,"5min":0.33200000000000002,"15min":0.27900000000000003},"max":{"1min":0.67500000000000004,"5min":0.83599999999999997,"15min":0.90100000000000002},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.57199999999999995,"15min":0.57499999999999996},"min":{"1min":0.38,"5min":0.315,"15min":0.315},"max":{"1min":0.69199999999999995,"5min":0.88,"15min":0.88},"last":0.55000000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574980,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1586,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193726,"statfs":{"total":200000000,"available":198375914,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.54200000000000004,"15min":0.504},"min":{"1min":0.38,"5min":0.36099999999999999,"15min":0.249},"max":{"1min":0.78600000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.54100000000000004,"15min":0.54900000000000004},"min":{"1min":0.40000000000000002,"5min":0.29799999999999999,"15min":0.29399999999999998},"max":{"1min":0.65900000000000003,"5min":0.81699999999999995,"15min":0.88600000000000001},"last":0.57499999999999996}]},{"osd":1,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55700000000000005,"15min":0.53500000000000003},"min":{"1min":0.436,"5min":0.34399999999999997,"15min":0.32500000000000001},"max":{"1min":0.69199999999999995,"5min":0.80600000000000005,"15min":0.82999999999999996},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.53900000000000003,"15min":0.52500000000000002},"min":{"1min":0.46200000000000002,"5min":0.34499999999999997,"15min":0.33000000000000002},"max":{"1min":0.72099999999999997,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.55900000000000005}]},{"osd":2,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55300000000000005,"15min":0.53600000000000003},"min":{"1min":0.47499999999999998,"5min":0.34499999999999997,"15min":0.32400000000000001},"max":{"1min":0.67700000000000005,"5min":0.749,"15min":0.86199999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.55400000000000005,"15min":0.56000000000000005},"min":{"1min":0.41799999999999998,"5min":0.33400000000000002,"15min":0.29599999999999999},"max":{"1min":0.74399999999999999,"5min":0.80000000000000004,"15min":0.95399999999999996},"last":0.59299999999999997}]},{"osd":3,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56999999999999995,"15min":0.55500000000000005},"min":{"1min":0.437,"5min":0.42199999999999999,"15min":0.316},"max":{"1min":0.65300000000000002,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56699999999999995,"15min":0.56999999999999995},"min":{"1min":0.39500000000000002,"5min":0.376,"15min":0.371},"max":{"1min":0.73299999999999998,"5min":0.73499999999999999,"15min":1.0529999999999999},"last":0.50600000000000001}]},{"osd":4,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47399999999999998},"min":{"1min":0.30399999999999999,"5min":0.28399999999999997,"15min":0.27700000000000002},"max":{"1min":0.68700000000000006,"5min":0.78600000000000003,"15min":0.79200000000000004},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52500000000000002,"15min":0.51800000000000002},"min":{"1min":0.317,"5min":0.314,"15min":0.28899999999999998},"max":{"1min":0.76000000000000001,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.48799999999999999}]},{"osd":6,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51700000000000002,"15min":0.50700000000000001},"min":{"1min":0.372,"5min":0.28199999999999997,"15min":0.27400000000000002},"max":{"1min":0.71099999999999997,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53000000000000003,"15min":0.50900000000000001},"min":{"1min":0.35199999999999998,"5min":0.33800000000000002,"15min":0.30499999999999999},"max":{"1min":0.73799999999999999,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.47499999999999998}]},{"osd":7,"last update":"Sun Oct 8 00:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54500000000000004,"15min":0.52900000000000003},"min":{"1min":0.36199999999999999,"5min":0.318,"15min":0.29399999999999998},"max":{"1min":0.80200000000000005,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56899999999999995,"15min":0.56100000000000005},"min":{"1min":0.41599999999999998,"5min":0.35599999999999998,"15min":0.32500000000000001},"max":{"1min":0.78400000000000003,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.54300000000000004}]}]}],"pool_statfs":[]}} 2023-10-08T00:21:34.716 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T00:21:34.717 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-08T00:21:34.871 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.870+0000 7f28de49d700 1 Processor -- start 2023-10-08T00:21:34.871 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.870+0000 7f28de49d700 1 -- start start 2023-10-08T00:21:34.874 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.871+0000 7f28de49d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28d815a290 0x7f28d806a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:34.874 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.871+0000 7f28de49d700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28d80b0790 con 0x7f28d806aae0 2023-10-08T00:21:34.874 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.871+0000 7f28de49d700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28d80ce1f0 con 0x7f28d8159ec0 2023-10-08T00:21:34.874 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.871+0000 7f28de49d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28d806af60 con 0x7f28d815a290 2023-10-08T00:21:34.874 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.871+0000 7f28dca3a700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f28d806aae0 0x7f28d80cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:57666/0 (socket says 172.21.15.99:57666) 2023-10-08T00:21:34.875 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.871+0000 7f28dca3a700 1 -- 172.21.15.99:0/324944497 learned_addr learned my addr 172.21.15.99:0/324944497 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:21:34.875 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.871+0000 7f28d77fe700 1 --2- 172.21.15.99:0/324944497 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28d815a290 0x7f28d806a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:34.875 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.872+0000 7f28d6ffd700 1 -- 172.21.15.99:0/324944497 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3182772722 0 0) 0x7f28d80b0790 con 0x7f28d806aae0 2023-10-08T00:21:34.875 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.872+0000 7f28d6ffd700 1 -- 172.21.15.99:0/324944497 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28c0003570 con 0x7f28d806aae0 2023-10-08T00:21:34.875 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.872+0000 7f28d6ffd700 1 -- 172.21.15.99:0/324944497 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2701855037 0 0) 0x7f28c0003570 con 0x7f28d806aae0 2023-10-08T00:21:34.875 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.872+0000 7f28d6ffd700 1 -- 172.21.15.99:0/324944497 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28d815a290 msgr2=0x7f28d806a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:34.875 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.872+0000 7f28d6ffd700 1 --2- 172.21.15.99:0/324944497 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28d815a290 0x7f28d806a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:34.875 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.872+0000 7f28d6ffd700 1 -- 172.21.15.99:0/324944497 >> v1:172.21.15.191:6789/0 conn(0x7f28d8159ec0 legacy=0x7f28d80af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:34.875 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.872+0000 7f28d6ffd700 1 -- 172.21.15.99:0/324944497 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d80b0b20 con 0x7f28d806aae0 2023-10-08T00:21:34.875 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.872+0000 7f28d6ffd700 1 -- 172.21.15.99:0/324944497 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f28d00017f0 con 0x7f28d806aae0 2023-10-08T00:21:34.875 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.872+0000 7f28d6ffd700 1 -- 172.21.15.99:0/324944497 >> v1:172.21.15.99:6789/0 conn(0x7f28d806aae0 legacy=0x7f28d80cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:34.875 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.872+0000 7f28d6ffd700 1 --2- 172.21.15.99:0/324944497 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28c0003a90 0x7f28c0003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:34.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.872+0000 7f28d6ffd700 1 --2- 172.21.15.99:0/324944497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28c0004810 0x7f28c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:34.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.872+0000 7f28d6ffd700 1 --2- 172.21.15.99:0/324944497 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28d815a290 0x7f28c00051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:34.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.872+0000 7f28d6ffd700 1 -- 172.21.15.99:0/324944497 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d806af60 con 0x7f28c0004810 2023-10-08T00:21:34.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.872+0000 7f28d6ffd700 1 -- 172.21.15.99:0/324944497 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c000bb60 con 0x7f28c0003a90 2023-10-08T00:21:34.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.872+0000 7f28d6ffd700 1 -- 172.21.15.99:0/324944497 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c000bd20 con 0x7f28d815a290 2023-10-08T00:21:34.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.872+0000 7f28d77fe700 1 --2- 172.21.15.99:0/324944497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28c0004810 0x7f28c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:34.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.872+0000 7f28d7fff700 1 --2- 172.21.15.99:0/324944497 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28c0003a90 0x7f28c0003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:34.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.872+0000 7f28dca3a700 1 --2- 172.21.15.99:0/324944497 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28d815a290 0x7f28c00051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:34.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.872+0000 7f28d77fe700 1 -- 172.21.15.99:0/324944497 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28d815a290 msgr2=0x7f28c00051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:34.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.872+0000 7f28d77fe700 1 --2- 172.21.15.99:0/324944497 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28d815a290 0x7f28c00051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:34.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.872+0000 7f28d77fe700 1 -- 172.21.15.99:0/324944497 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28c0003a90 msgr2=0x7f28c0003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:34.876 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.872+0000 7f28d77fe700 1 --2- 172.21.15.99:0/324944497 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28c0003a90 0x7f28c0003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:34.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.872+0000 7f28d77fe700 1 -- 172.21.15.99:0/324944497 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c0003570 con 0x7f28c0004810 2023-10-08T00:21:34.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.872+0000 7f28d77fe700 1 --2- 172.21.15.99:0/324944497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28c0004810 0x7f28c0004c80 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f28cc009c00 tx=0x7f28cc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8e7b53fc4bd937b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:34.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.872+0000 7f28d6ffd700 1 -- 172.21.15.99:0/324944497 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28cc00e710 con 0x7f28c0004810 2023-10-08T00:21:34.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.872+0000 7f28d6ffd700 1 -- 172.21.15.99:0/324944497 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28cc00e870 con 0x7f28c0004810 2023-10-08T00:21:34.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.872+0000 7f28d6ffd700 1 -- 172.21.15.99:0/324944497 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28cc00ec30 con 0x7f28c0004810 2023-10-08T00:21:34.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.873+0000 7f28de49d700 1 -- 172.21.15.99:0/324944497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28c0004810 msgr2=0x7f28c0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:34.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.873+0000 7f28de49d700 1 --2- 172.21.15.99:0/324944497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28c0004810 0x7f28c0004c80 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f28cc009c00 tx=0x7f28cc0091a0 comp rx=0 tx=0).stop 2023-10-08T00:21:34.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.873+0000 7f28dca3a700 1 -- 172.21.15.99:0/324944497 reap_dead start 2023-10-08T00:21:34.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.873+0000 7f28de49d700 1 -- 172.21.15.99:0/324944497 shutdown_connections 2023-10-08T00:21:34.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.873+0000 7f28de49d700 1 -- 172.21.15.99:0/324944497 >> 172.21.15.99:0/324944497 conn(0x7f28d80d1880 msgr2=0x7f28d80cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:34.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.873+0000 7f28de49d700 1 -- 172.21.15.99:0/324944497 shutdown_connections 2023-10-08T00:21:34.877 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.873+0000 7f28de49d700 1 -- 172.21.15.99:0/324944497 wait complete. 2023-10-08T00:21:34.880 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.874+0000 7f28de49d700 1 Processor -- start 2023-10-08T00:21:34.880 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.874+0000 7f28de49d700 1 -- start start 2023-10-08T00:21:34.880 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.875+0000 7f28de49d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28c0004810 0x7f28d80c84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:34.880 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.875+0000 7f28de49d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28d815a660 0x7f28d80b4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:34.880 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.875+0000 7f28de49d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28d814e8f0 0x7f28d80b5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:34.880 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.875+0000 7f28de49d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d815b6e0 con 0x7f28c0004810 2023-10-08T00:21:34.880 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.875+0000 7f28de49d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d815aad0 con 0x7f28d815a660 2023-10-08T00:21:34.880 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.875+0000 7f28de49d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28d815adb0 con 0x7f28d814e8f0 2023-10-08T00:21:34.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.875+0000 7f28d7fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28c0004810 0x7f28d80c84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:34.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.875+0000 7f28d7fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28c0004810 0x7f28d80c84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51946/0 (socket says 172.21.15.99:51946) 2023-10-08T00:21:34.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.875+0000 7f28d7fff700 1 -- 172.21.15.99:0/3712940587 learned_addr learned my addr 172.21.15.99:0/3712940587 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:34.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.875+0000 7f28d77fe700 1 --2- 172.21.15.99:0/3712940587 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28d815a660 0x7f28d80b4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:34.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.875+0000 7f28dca3a700 1 --2- 172.21.15.99:0/3712940587 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28d814e8f0 0x7f28d80b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:34.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.875+0000 7f28d7fff700 1 -- 172.21.15.99:0/3712940587 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28d814e8f0 msgr2=0x7f28d80b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:34.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.875+0000 7f28d7fff700 1 --2- 172.21.15.99:0/3712940587 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28d814e8f0 0x7f28d80b5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:34.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.875+0000 7f28d7fff700 1 -- 172.21.15.99:0/3712940587 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28d815a660 msgr2=0x7f28d80b4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:34.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.875+0000 7f28d7fff700 1 --2- 172.21.15.99:0/3712940587 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28d815a660 0x7f28d80b4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:34.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.875+0000 7f28d7fff700 1 -- 172.21.15.99:0/3712940587 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d815b090 con 0x7f28c0004810 2023-10-08T00:21:34.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.875+0000 7f28d7fff700 1 --2- 172.21.15.99:0/3712940587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28c0004810 0x7f28d80c84a0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f28c800e810 tx=0x7f28c8009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:34.881 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.876+0000 7f28bffff700 1 -- 172.21.15.99:0/3712940587 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c8012590 con 0x7f28c0004810 2023-10-08T00:21:34.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.876+0000 7f28bffff700 1 -- 172.21.15.99:0/3712940587 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c80126f0 con 0x7f28c0004810 2023-10-08T00:21:34.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.876+0000 7f28de49d700 1 -- 172.21.15.99:0/3712940587 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28d815b370 con 0x7f28c0004810 2023-10-08T00:21:34.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.876+0000 7f28bffff700 1 -- 172.21.15.99:0/3712940587 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c8012a90 con 0x7f28c0004810 2023-10-08T00:21:34.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.876+0000 7f28de49d700 1 -- 172.21.15.99:0/3712940587 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28d80b5e10 con 0x7f28c0004810 2023-10-08T00:21:34.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.877+0000 7f28de49d700 1 -- 172.21.15.99:0/3712940587 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28d80b6000 con 0x7f28c0004810 2023-10-08T00:21:34.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.877+0000 7f28bffff700 1 -- 172.21.15.99:0/3712940587 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f28c801d460 con 0x7f28c0004810 2023-10-08T00:21:34.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.877+0000 7f28bffff700 1 --2- 172.21.15.99:0/3712940587 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f28c4066f30 0x7f28c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:34.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.877+0000 7f28bffff700 1 -- 172.21.15.99:0/3712940587 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f28c8084580 con 0x7f28c0004810 2023-10-08T00:21:34.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.877+0000 7f28d77fe700 1 --2- 172.21.15.99:0/3712940587 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f28c4066f30 0x7f28c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:34.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.878+0000 7f28d77fe700 1 --2- 172.21.15.99:0/3712940587 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f28c4066f30 0x7f28c40693e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f28cc0091e0 tx=0x7f28cc01a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:34.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:34.881+0000 7f28bffff700 1 -- 172.21.15.99:0/3712940587 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f28d80b6000 con 0x7f28c0004810 2023-10-08T00:21:35.137 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.136+0000 7f28de49d700 1 -- 172.21.15.99:0/3712940587 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f28d80b6000 con 0x7f28c0004810 2023-10-08T00:21:35.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.139+0000 7f28bffff700 1 -- 172.21.15.99:0/3712940587 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) v1 ==== 190+0+0 (secure 0 0 0) 0x7f28d80b6000 con 0x7f28c0004810 2023-10-08T00:21:35.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.142+0000 7f28de49d700 1 -- 172.21.15.99:0/3712940587 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f28c4066f30 msgr2=0x7f28c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:35.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.142+0000 7f28de49d700 1 --2- 172.21.15.99:0/3712940587 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f28c4066f30 0x7f28c40693e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f28cc0091e0 tx=0x7f28cc01a000 comp rx=0 tx=0).stop 2023-10-08T00:21:35.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.142+0000 7f28de49d700 1 -- 172.21.15.99:0/3712940587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28c0004810 msgr2=0x7f28d80c84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:35.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.142+0000 7f28de49d700 1 --2- 172.21.15.99:0/3712940587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28c0004810 0x7f28d80c84a0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f28c800e810 tx=0x7f28c8009040 comp rx=0 tx=0).stop 2023-10-08T00:21:35.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.142+0000 7f28de49d700 1 -- 172.21.15.99:0/3712940587 shutdown_connections 2023-10-08T00:21:35.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.142+0000 7f28de49d700 1 --2- 172.21.15.99:0/3712940587 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f28c4066f30 0x7f28c40693e0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:35.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.142+0000 7f28de49d700 1 --2- 172.21.15.99:0/3712940587 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28d814e8f0 0x7f28d80b5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:35.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.142+0000 7f28de49d700 1 --2- 172.21.15.99:0/3712940587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28c0004810 0x7f28d80c84a0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:35.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.142+0000 7f28de49d700 1 --2- 172.21.15.99:0/3712940587 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28d815a660 0x7f28d80b4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:35.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.142+0000 7f28de49d700 1 -- 172.21.15.99:0/3712940587 >> 172.21.15.99:0/3712940587 conn(0x7f28d80d1880 msgr2=0x7f28d80b2830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:35.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.142+0000 7f28de49d700 1 -- 172.21.15.99:0/3712940587 shutdown_connections 2023-10-08T00:21:35.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.142+0000 7f28de49d700 1 -- 172.21.15.99:0/3712940587 wait complete. 2023-10-08T00:21:35.144 INFO:teuthology.orchestra.run.smithi099.stderr:updated caps for client.0 2023-10-08T00:21:35.154 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:21:35.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.268+0000 7fecbc464700 1 Processor -- start 2023-10-08T00:21:35.271 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.268+0000 7fecbc464700 1 -- start start 2023-10-08T00:21:35.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.269+0000 7fecbc464700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fecb4157870 0x7fecb415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:35.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.269+0000 7fecbc464700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecb415cbb0 con 0x7fecb415a620 2023-10-08T00:21:35.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.269+0000 7fecbc464700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecb415bfa0 con 0x7fecb4157c40 2023-10-08T00:21:35.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.269+0000 7fecbc464700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecb415c280 con 0x7fecb4157870 2023-10-08T00:21:35.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.269+0000 7fecb99ff700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fecb415a620 0x7fecb415d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:57674/0 (socket says 172.21.15.99:57674) 2023-10-08T00:21:35.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.269+0000 7fecb99ff700 1 -- 172.21.15.99:0/811908919 learned_addr learned my addr 172.21.15.99:0/811908919 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:21:35.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.269+0000 7fecba200700 1 --2- 172.21.15.99:0/811908919 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fecb4157870 0x7fecb415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:35.272 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.269+0000 7fecb91fe700 1 -- 172.21.15.99:0/811908919 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1568872854 0 0) 0x7fecb415cbb0 con 0x7fecb415a620 2023-10-08T00:21:35.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.269+0000 7fecb91fe700 1 -- 172.21.15.99:0/811908919 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feca0003570 con 0x7fecb415a620 2023-10-08T00:21:35.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.269+0000 7fecb91fe700 1 -- 172.21.15.99:0/811908919 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2500964333 0 0) 0x7fecb415bfa0 con 0x7fecb4157c40 2023-10-08T00:21:35.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.269+0000 7fecb91fe700 1 -- 172.21.15.99:0/811908919 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fecb415cbb0 con 0x7fecb4157c40 2023-10-08T00:21:35.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.269+0000 7fecba200700 1 -- 172.21.15.99:0/811908919 >> v1:172.21.15.191:6789/0 conn(0x7fecb4157c40 legacy=0x7fecb4158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:35.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.269+0000 7fecba200700 1 -- 172.21.15.99:0/811908919 >> v1:172.21.15.99:6789/0 conn(0x7fecb415a620 legacy=0x7fecb415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:35.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.269+0000 7fecba200700 1 -- 172.21.15.99:0/811908919 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb415c560 con 0x7fecb4157870 2023-10-08T00:21:35.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.270+0000 7fecba200700 1 --2- 172.21.15.99:0/811908919 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fecb4157870 0x7fecb415b6a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fecb0009910 tx=0x7fecb00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4ff67fba40d1bc2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:35.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.270+0000 7fecb91fe700 1 -- 172.21.15.99:0/811908919 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb0011630 con 0x7fecb4157870 2023-10-08T00:21:35.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.270+0000 7fecb91fe700 1 -- 172.21.15.99:0/811908919 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecb0011790 con 0x7fecb4157870 2023-10-08T00:21:35.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.270+0000 7fecb91fe700 1 -- 172.21.15.99:0/811908919 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb0011b10 con 0x7fecb4157870 2023-10-08T00:21:35.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.270+0000 7fecbc464700 1 -- 172.21.15.99:0/811908919 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fecb4157870 msgr2=0x7fecb415b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:35.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.270+0000 7fecbc464700 1 --2- 172.21.15.99:0/811908919 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fecb4157870 0x7fecb415b6a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fecb0009910 tx=0x7fecb00091a0 comp rx=0 tx=0).stop 2023-10-08T00:21:35.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.270+0000 7fecbc464700 1 -- 172.21.15.99:0/811908919 shutdown_connections 2023-10-08T00:21:35.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.270+0000 7fecbc464700 1 --2- 172.21.15.99:0/811908919 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fecb4157870 0x7fecb415b6a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:35.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.270+0000 7fecbc464700 1 -- 172.21.15.99:0/811908919 >> 172.21.15.99:0/811908919 conn(0x7fecb4068ce0 msgr2=0x7fecb40690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:35.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.270+0000 7fecbc464700 1 -- 172.21.15.99:0/811908919 shutdown_connections 2023-10-08T00:21:35.274 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.270+0000 7fecbc464700 1 -- 172.21.15.99:0/811908919 wait complete. 2023-10-08T00:21:35.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.272+0000 7fecbc464700 1 Processor -- start 2023-10-08T00:21:35.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.272+0000 7fecbc464700 1 -- start start 2023-10-08T00:21:35.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.272+0000 7fecbc464700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fecb4157870 0x7fecb4153d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:35.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.272+0000 7fecbc464700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fecb4157c40 0x7fecb413f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:35.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.272+0000 7fecbc464700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecb415a620 0x7fecb4151060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:35.277 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.272+0000 7fecbc464700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb413efd0 con 0x7fecb415a620 2023-10-08T00:21:35.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.272+0000 7fecbc464700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb414e950 con 0x7fecb4157c40 2023-10-08T00:21:35.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.272+0000 7fecbc464700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecb416e570 con 0x7fecb4157870 2023-10-08T00:21:35.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.272+0000 7fecbaa01700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecb415a620 0x7fecb4151060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:35.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.272+0000 7fecbaa01700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecb415a620 0x7fecb4151060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51952/0 (socket says 172.21.15.99:51952) 2023-10-08T00:21:35.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.272+0000 7fecbaa01700 1 -- 172.21.15.99:0/2311895017 learned_addr learned my addr 172.21.15.99:0/2311895017 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:35.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.272+0000 7fecba200700 1 --2- 172.21.15.99:0/2311895017 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fecb4157870 0x7fecb4153d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:35.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.273+0000 7fecb99ff700 1 --2- 172.21.15.99:0/2311895017 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fecb4157c40 0x7fecb413f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:35.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.273+0000 7fecbaa01700 1 -- 172.21.15.99:0/2311895017 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fecb4157870 msgr2=0x7fecb4153d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:35.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.273+0000 7fecbaa01700 1 --2- 172.21.15.99:0/2311895017 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fecb4157870 0x7fecb4153d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:35.278 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.273+0000 7fecbaa01700 1 -- 172.21.15.99:0/2311895017 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fecb4157c40 msgr2=0x7fecb413f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:35.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.273+0000 7fecbaa01700 1 --2- 172.21.15.99:0/2311895017 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fecb4157c40 0x7fecb413f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:35.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.273+0000 7fecbaa01700 1 -- 172.21.15.99:0/2311895017 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb0009420 con 0x7fecb415a620 2023-10-08T00:21:35.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.273+0000 7fecbaa01700 1 --2- 172.21.15.99:0/2311895017 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecb415a620 0x7fecb4151060 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fecac00c910 tx=0x7fecac0027a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:35.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.273+0000 7fecaa7fc700 1 -- 172.21.15.99:0/2311895017 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecac012630 con 0x7fecb415a620 2023-10-08T00:21:35.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.273+0000 7fecaa7fc700 1 -- 172.21.15.99:0/2311895017 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecac0127b0 con 0x7fecb415a620 2023-10-08T00:21:35.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.273+0000 7fecbc464700 1 -- 172.21.15.99:0/2311895017 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecb414eba0 con 0x7fecb415a620 2023-10-08T00:21:35.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.273+0000 7fecaa7fc700 1 -- 172.21.15.99:0/2311895017 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecac012b70 con 0x7fecb415a620 2023-10-08T00:21:35.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.273+0000 7fecbc464700 1 -- 172.21.15.99:0/2311895017 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecb4151d70 con 0x7fecb415a620 2023-10-08T00:21:35.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.274+0000 7fecbc464700 1 -- 172.21.15.99:0/2311895017 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecb414f0c0 con 0x7fecb415a620 2023-10-08T00:21:35.279 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.274+0000 7fecaa7fc700 1 -- 172.21.15.99:0/2311895017 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fecac01b400 con 0x7fecb415a620 2023-10-08T00:21:35.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.275+0000 7fecaa7fc700 1 --2- 172.21.15.99:0/2311895017 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fec98066f30 0x7fec980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:35.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.275+0000 7fecaa7fc700 1 -- 172.21.15.99:0/2311895017 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fecac084820 con 0x7fecb415a620 2023-10-08T00:21:35.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.275+0000 7fecba200700 1 --2- 172.21.15.99:0/2311895017 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fec98066f30 0x7fec980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:35.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.275+0000 7fecba200700 1 --2- 172.21.15.99:0/2311895017 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fec98066f30 0x7fec980693e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fecb0009860 tx=0x7fecb0017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:35.280 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.279+0000 7fecaa7fc700 1 -- 172.21.15.99:0/2311895017 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fecb414f0c0 con 0x7fecb415a620 2023-10-08T00:21:35.506 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.505+0000 7fecbc464700 1 -- 172.21.15.99:0/2311895017 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fecb414f0c0 con 0x7fecb415a620 2023-10-08T00:21:35.506 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.506+0000 7fecaa7fc700 1 -- 172.21.15.99:0/2311895017 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+4604 (secure 0 0 0) 0x7fecb414f0c0 con 0x7fecb415a620 2023-10-08T00:21:35.507 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:21:35.507 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":84,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/3862684847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3862684847},{"type":"v1","addr":"172.21.15.191:6836","nonce":3862684847}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":8106,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/359623206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":359623206},{"type":"v1","addr":"172.21.15.99:6834","nonce":359623206}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":8109,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3064101777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":3064101777},{"type":"v1","addr":"172.21.15.99:6835","nonce":3064101777}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:21:29.659394+0000","modified":"2023-10-08T00:21:33.311802+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8124},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8124":{"gid":8124,"name":"d","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/2466421758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":2466421758},{"type":"v1","addr":"172.21.15.191:6837","nonce":2466421758}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-08T00:21:35.509 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.508+0000 7fecbc464700 1 -- 172.21.15.99:0/2311895017 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fec98066f30 msgr2=0x7fec980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:35.509 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.508+0000 7fecbc464700 1 --2- 172.21.15.99:0/2311895017 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fec98066f30 0x7fec980693e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fecb0009860 tx=0x7fecb0017000 comp rx=0 tx=0).stop 2023-10-08T00:21:35.509 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.508+0000 7fecbc464700 1 -- 172.21.15.99:0/2311895017 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecb415a620 msgr2=0x7fecb4151060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:35.509 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.508+0000 7fecbc464700 1 --2- 172.21.15.99:0/2311895017 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecb415a620 0x7fecb4151060 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fecac00c910 tx=0x7fecac0027a0 comp rx=0 tx=0).stop 2023-10-08T00:21:35.509 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.508+0000 7fecbc464700 1 -- 172.21.15.99:0/2311895017 shutdown_connections 2023-10-08T00:21:35.509 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.508+0000 7fecbc464700 1 --2- 172.21.15.99:0/2311895017 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fec98066f30 0x7fec980693e0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:35.509 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.508+0000 7fecbc464700 1 --2- 172.21.15.99:0/2311895017 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecb415a620 0x7fecb4151060 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:35.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.508+0000 7fecbc464700 1 --2- 172.21.15.99:0/2311895017 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fecb4157870 0x7fecb4153d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:35.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.508+0000 7fecbc464700 1 --2- 172.21.15.99:0/2311895017 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fecb4157c40 0x7fecb413f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:35.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.508+0000 7fecbc464700 1 -- 172.21.15.99:0/2311895017 >> 172.21.15.99:0/2311895017 conn(0x7fecb4068ce0 msgr2=0x7fecb40b4810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:35.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.508+0000 7fecbc464700 1 -- 172.21.15.99:0/2311895017 shutdown_connections 2023-10-08T00:21:35.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:35.508+0000 7fecbc464700 1 -- 172.21.15.99:0/2311895017 wait complete. 2023-10-08T00:21:35.510 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 84 2023-10-08T00:21:35.632 INFO:tasks.ceph.mds.d.smithi191.stderr:2023-10-08T00:21:35.632+0000 7fd59d2d7700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T00:21:35.923 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 84, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T00:21:29.659394+0000', 'modified': '2023-10-08T00:21:33.311802+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8124}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8124': {'gid': 8124, 'name': 'd', 'rank': 0, 'incarnation': 81, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.191:6837/2466421758', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6835', 'nonce': 2466421758}, {'type': 'v1', 'addr': '172.21.15.191:6837', 'nonce': 2466421758}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [23], 'metadata_pool': 22, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-08T00:21:35.924 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T00:21:35.924 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-08T00:21:36.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.041+0000 7f4562625700 1 Processor -- start 2023-10-08T00:21:36.041 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.041+0000 7f4562625700 1 -- start start 2023-10-08T00:21:36.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.041+0000 7f4562625700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f455c157870 0x7f455c15b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:36.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.041+0000 7f4562625700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f455c157c40 0x7f455c158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:36.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.042+0000 7f4562625700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f455c15cbb0 con 0x7f455c15a620 2023-10-08T00:21:36.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.042+0000 7f4562625700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f455c15bfa0 con 0x7f455c157870 2023-10-08T00:21:36.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.042+0000 7f4562625700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f455c15c280 con 0x7f455c157c40 2023-10-08T00:21:36.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.042+0000 7f455bfff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f455c157870 0x7f455c15b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:36.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.042+0000 7f455b7fe700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f455c15a620 0x7f455c15d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:57682/0 (socket says 172.21.15.99:57682) 2023-10-08T00:21:36.044 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.042+0000 7f455bfff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f455c157870 0x7f455c15b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51960/0 (socket says 172.21.15.99:51960) 2023-10-08T00:21:36.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.042+0000 7f455b7fe700 1 -- 172.21.15.99:0/2805050726 learned_addr learned my addr 172.21.15.99:0/2805050726 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:21:36.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.042+0000 7f4560bc2700 1 --2- 172.21.15.99:0/2805050726 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f455c157c40 0x7f455c158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:36.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.042+0000 7f455bfff700 1 -- 172.21.15.99:0/2805050726 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f455c157c40 msgr2=0x7f455c158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:36.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.042+0000 7f455bfff700 1 --2- 172.21.15.99:0/2805050726 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f455c157c40 0x7f455c158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:36.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.042+0000 7f455bfff700 1 -- 172.21.15.99:0/2805050726 >> v1:172.21.15.99:6789/0 conn(0x7f455c15a620 legacy=0x7f455c15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:36.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.042+0000 7f455bfff700 1 -- 172.21.15.99:0/2805050726 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f455c15c560 con 0x7f455c157870 2023-10-08T00:21:36.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.042+0000 7f455bfff700 1 --2- 172.21.15.99:0/2805050726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f455c157870 0x7f455c15b6a0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f454c0098e0 tx=0x7f454c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57ca373c5ab94da3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:36.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.042+0000 7f455affd700 1 -- 172.21.15.99:0/2805050726 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454c009e80 con 0x7f455c157870 2023-10-08T00:21:36.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.042+0000 7f455affd700 1 -- 172.21.15.99:0/2805050726 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f454c00b860 con 0x7f455c157870 2023-10-08T00:21:36.045 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.042+0000 7f455affd700 1 -- 172.21.15.99:0/2805050726 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454c00bbe0 con 0x7f455c157870 2023-10-08T00:21:36.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.043+0000 7f4562625700 1 -- 172.21.15.99:0/2805050726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f455c157870 msgr2=0x7f455c15b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:36.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.043+0000 7f4562625700 1 --2- 172.21.15.99:0/2805050726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f455c157870 0x7f455c15b6a0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f454c0098e0 tx=0x7f454c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:21:36.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.043+0000 7f4562625700 1 -- 172.21.15.99:0/2805050726 shutdown_connections 2023-10-08T00:21:36.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.043+0000 7f4562625700 1 --2- 172.21.15.99:0/2805050726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f455c157870 0x7f455c15b6a0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:36.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.043+0000 7f4562625700 1 --2- 172.21.15.99:0/2805050726 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f455c157c40 0x7f455c158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:36.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.043+0000 7f4562625700 1 -- 172.21.15.99:0/2805050726 >> 172.21.15.99:0/2805050726 conn(0x7f455c068ce0 msgr2=0x7f455c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:36.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.043+0000 7f4562625700 1 -- 172.21.15.99:0/2805050726 shutdown_connections 2023-10-08T00:21:36.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.043+0000 7f4562625700 1 -- 172.21.15.99:0/2805050726 wait complete. 2023-10-08T00:21:36.049 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.044+0000 7f4562625700 1 Processor -- start 2023-10-08T00:21:36.049 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.044+0000 7f4562625700 1 -- start start 2023-10-08T00:21:36.049 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.045+0000 7f4562625700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f455c157870 0x7f455c151080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:36.049 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.045+0000 7f4562625700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f455c157c40 0x7f455c16a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:36.049 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.045+0000 7f4562625700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f455c15a620 0x7f455c16a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:36.049 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.045+0000 7f4562625700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455c1543f0 con 0x7f455c157c40 2023-10-08T00:21:36.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.045+0000 7f4562625700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455c0b17a0 con 0x7f455c15a620 2023-10-08T00:21:36.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.045+0000 7f4562625700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f455c153b50 con 0x7f455c157870 2023-10-08T00:21:36.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.045+0000 7f455b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f455c157c40 0x7f455c16a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:36.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.045+0000 7f455b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f455c157c40 0x7f455c16a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51974/0 (socket says 172.21.15.99:51974) 2023-10-08T00:21:36.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.045+0000 7f455b7fe700 1 -- 172.21.15.99:0/2177907816 learned_addr learned my addr 172.21.15.99:0/2177907816 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:36.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.045+0000 7f455bfff700 1 --2- 172.21.15.99:0/2177907816 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f455c157870 0x7f455c151080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:36.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.045+0000 7f4560bc2700 1 --2- 172.21.15.99:0/2177907816 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f455c15a620 0x7f455c16a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:36.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.045+0000 7f455b7fe700 1 -- 172.21.15.99:0/2177907816 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f455c157870 msgr2=0x7f455c151080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:36.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.045+0000 7f455b7fe700 1 --2- 172.21.15.99:0/2177907816 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f455c157870 0x7f455c151080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:36.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.045+0000 7f455b7fe700 1 -- 172.21.15.99:0/2177907816 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f455c15a620 msgr2=0x7f455c16a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:36.051 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.045+0000 7f455b7fe700 1 --2- 172.21.15.99:0/2177907816 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f455c15a620 0x7f455c16a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:36.051 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.045+0000 7f455b7fe700 1 -- 172.21.15.99:0/2177907816 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f454c009420 con 0x7f455c157c40 2023-10-08T00:21:36.051 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.045+0000 7f455b7fe700 1 --2- 172.21.15.99:0/2177907816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f455c157c40 0x7f455c16a360 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f455000c590 tx=0x7f4550001020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:36.051 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.045+0000 7f4543fff700 1 -- 172.21.15.99:0/2177907816 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4550005b30 con 0x7f455c157c40 2023-10-08T00:21:36.051 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.045+0000 7f4543fff700 1 -- 172.21.15.99:0/2177907816 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4550005cb0 con 0x7f455c157c40 2023-10-08T00:21:36.051 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.045+0000 7f4543fff700 1 -- 172.21.15.99:0/2177907816 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4550007320 con 0x7f455c157c40 2023-10-08T00:21:36.051 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.045+0000 7f4562625700 1 -- 172.21.15.99:0/2177907816 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f455c153e30 con 0x7f455c157c40 2023-10-08T00:21:36.051 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.046+0000 7f4562625700 1 -- 172.21.15.99:0/2177907816 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f455c153860 con 0x7f455c157c40 2023-10-08T00:21:36.051 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.047+0000 7f4543fff700 1 -- 172.21.15.99:0/2177907816 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f455001b3b0 con 0x7f455c157c40 2023-10-08T00:21:36.051 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.048+0000 7f4543fff700 1 --2- 172.21.15.99:0/2177907816 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4548066f30 0x7f45480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:36.051 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.048+0000 7f4543fff700 1 -- 172.21.15.99:0/2177907816 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4550083b00 con 0x7f455c157c40 2023-10-08T00:21:36.052 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.048+0000 7f455bfff700 1 --2- 172.21.15.99:0/2177907816 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4548066f30 0x7f45480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:36.052 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.048+0000 7f455bfff700 1 --2- 172.21.15.99:0/2177907816 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4548066f30 0x7f45480693e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f454c0091e0 tx=0x7f454c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:36.058 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.049+0000 7f456b91db80 1 Processor -- start 2023-10-08T00:21:36.058 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.049+0000 7f456b91db80 1 -- start start 2023-10-08T00:21:36.058 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.050+0000 7f456b91db80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5614764eda50 0x5614764ede20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:36.058 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.050+0000 7f456b91db80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561476537d00 0x561476516a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:36.058 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.050+0000 7f456b91db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561476516f60 0x5614764e9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:36.058 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.050+0000 7f456b91db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561476444a70 con 0x561476516f60 2023-10-08T00:21:36.058 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.050+0000 7f456b91db80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x561476442270 con 0x561476537d00 2023-10-08T00:21:36.058 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.050+0000 7f456b91db80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x56147643e060 con 0x5614764eda50 2023-10-08T00:21:36.058 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.050+0000 7f4560bc2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561476516f60 0x5614764e9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:36.058 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.050+0000 7f4560bc2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561476516f60 0x5614764e9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51976/0 (socket says 172.21.15.99:51976) 2023-10-08T00:21:36.059 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.050+0000 7f4560bc2700 1 -- 172.21.15.99:0/2740776330 learned_addr learned my addr 172.21.15.99:0/2740776330 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:36.059 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.050+0000 7f455bfff700 1 --2- 172.21.15.99:0/2740776330 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5614764eda50 0x5614764ede20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:36.059 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.050+0000 7f455b7fe700 1 --2- 172.21.15.99:0/2740776330 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561476537d00 0x561476516a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:36.059 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.051+0000 7f4560bc2700 1 -- 172.21.15.99:0/2740776330 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5614764eda50 msgr2=0x5614764ede20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:36.059 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.051+0000 7f4560bc2700 1 --2- 172.21.15.99:0/2740776330 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5614764eda50 0x5614764ede20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:36.059 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.051+0000 7f4560bc2700 1 -- 172.21.15.99:0/2740776330 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561476537d00 msgr2=0x561476516a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:36.059 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.051+0000 7f4560bc2700 1 --2- 172.21.15.99:0/2740776330 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561476537d00 0x561476516a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:36.059 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.051+0000 7f4560bc2700 1 -- 172.21.15.99:0/2740776330 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614762d8270 con 0x561476516f60 2023-10-08T00:21:36.059 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.051+0000 7f4560bc2700 1 --2- 172.21.15.99:0/2740776330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561476516f60 0x5614764e9cf0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f4554007450 tx=0x7f455400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b22651ff1b0bf4a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:36.059 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.051+0000 7f45417fa700 1 -- 172.21.15.99:0/2740776330 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4554015470 con 0x561476516f60 2023-10-08T00:21:36.060 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.051+0000 7f45417fa700 1 -- 172.21.15.99:0/2740776330 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f455400f050 con 0x561476516f60 2023-10-08T00:21:36.060 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.051+0000 7f45417fa700 1 -- 172.21.15.99:0/2740776330 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4554015800 con 0x561476516f60 2023-10-08T00:21:36.060 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.051+0000 7f456b91db80 1 -- 172.21.15.99:0/2740776330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561476516f60 msgr2=0x5614764e9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:36.060 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.051+0000 7f456b91db80 1 --2- 172.21.15.99:0/2740776330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561476516f60 0x5614764e9cf0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f4554007450 tx=0x7f455400b040 comp rx=0 tx=0).stop 2023-10-08T00:21:36.060 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.052+0000 7f456b91db80 1 -- 172.21.15.99:0/2740776330 shutdown_connections 2023-10-08T00:21:36.060 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.052+0000 7f456b91db80 1 --2- 172.21.15.99:0/2740776330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561476516f60 0x5614764e9cf0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:36.060 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.052+0000 7f456b91db80 1 --2- 172.21.15.99:0/2740776330 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5614764eda50 0x5614764ede20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:36.060 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.052+0000 7f456b91db80 1 --2- 172.21.15.99:0/2740776330 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561476537d00 0x561476516a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:36.060 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.052+0000 7f456b91db80 1 -- 172.21.15.99:0/2740776330 >> 172.21.15.99:0/2740776330 conn(0x5614764ef370 msgr2=0x5614764ef740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:36.060 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.052+0000 7f456b91db80 1 -- 172.21.15.99:0/2740776330 shutdown_connections 2023-10-08T00:21:36.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.052+0000 7f456b91db80 1 -- 172.21.15.99:0/2740776330 wait complete. 2023-10-08T00:21:36.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.052+0000 7f456b91db80 1 -- start start 2023-10-08T00:21:36.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.052+0000 7f456b91db80 1 Processor -- start 2023-10-08T00:21:36.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.053+0000 7f456b91db80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5614764eda50 0x5614765449d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:36.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.053+0000 7f456b91db80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561476516f60 0x56147652de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:36.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.053+0000 7f456b91db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561476537d00 0x56147652e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:36.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.053+0000 7f456b91db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614763e9890 con 0x561476537d00 2023-10-08T00:21:36.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.053+0000 7f456b91db80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x561476404ea0 con 0x561476516f60 2023-10-08T00:21:36.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.053+0000 7f456b91db80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x5614762c0e30 con 0x5614764eda50 2023-10-08T00:21:36.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.053+0000 7f456b91db80 10 client.0 _ll_register_callbacks cb 0x5614764b1110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T00:21:36.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.053+0000 7f4560bc2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561476537d00 0x56147652e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:36.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.053+0000 7f4560bc2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561476537d00 0x56147652e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51990/0 (socket says 172.21.15.99:51990) 2023-10-08T00:21:36.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.053+0000 7f4560bc2700 1 -- 172.21.15.99:0/677952433 learned_addr learned my addr 172.21.15.99:0/677952433 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:36.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.053+0000 7f455b7fe700 1 --2- 172.21.15.99:0/677952433 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561476516f60 0x56147652de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:36.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.053+0000 7f455bfff700 1 --2- 172.21.15.99:0/677952433 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5614764eda50 0x5614765449d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:36.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.053+0000 7f4560bc2700 1 -- 172.21.15.99:0/677952433 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5614764eda50 msgr2=0x5614765449d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:36.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.053+0000 7f4560bc2700 1 --2- 172.21.15.99:0/677952433 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5614764eda50 0x5614765449d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:36.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.053+0000 7f4560bc2700 1 -- 172.21.15.99:0/677952433 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561476516f60 msgr2=0x56147652de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:36.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.053+0000 7f4560bc2700 1 --2- 172.21.15.99:0/677952433 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561476516f60 0x56147652de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:36.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.053+0000 7f4560bc2700 1 -- 172.21.15.99:0/677952433 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4554009d60 con 0x561476537d00 2023-10-08T00:21:36.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.054+0000 7f4560bc2700 1 --2- 172.21.15.99:0/677952433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561476537d00 0x56147652e3b0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f4554007a40 tx=0x7f455401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3286ff8aafca74a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:36.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.054+0000 7f453a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T00:21:36.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.054+0000 7f453a7fc700 1 -- 172.21.15.99:0/677952433 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455401e400 con 0x561476537d00 2023-10-08T00:21:36.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.054+0000 7f453a7fc700 1 -- 172.21.15.99:0/677952433 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(184..184 src has 1..184) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f455401e560 con 0x561476537d00 2023-10-08T00:21:36.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.054+0000 7f456b91db80 1 -- 172.21.15.99:0/677952433 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5614764e2b70 con 0x561476537d00 2023-10-08T00:21:36.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.054+0000 7f453a7fc700 1 -- 172.21.15.99:0/677952433 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f455401e930 con 0x561476537d00 2023-10-08T00:21:36.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.054+0000 7f456b91db80 10 client.8277 fetch_fsmap learned FSMap version 84 2023-10-08T00:21:36.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.054+0000 7f456b91db80 1 -- 172.21.15.99:0/677952433 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=2+}) v3 -- 0x56147627afe0 con 0x561476537d00 2023-10-08T00:21:36.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.054+0000 7f453a7fc700 1 -- 172.21.15.99:0/677952433 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5614764e2b70 con 0x561476537d00 2023-10-08T00:21:36.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.054+0000 7f453a7fc700 1 -- 172.21.15.99:0/677952433 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 84) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f455401ea90 con 0x561476537d00 2023-10-08T00:21:36.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.054+0000 7f456b91db80 10 client.8277 fetch_fsmap finished waiting for FSMap version 84 2023-10-08T00:21:36.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.054+0000 7f456b91db80 10 client.8277 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-08T00:21:36.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.054+0000 7f456b91db80 20 client.8277 populate_metadata read hostname 'smithi099' 2023-10-08T00:21:36.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.054+0000 7f456b91db80 1 --2- 172.21.15.99:0/677952433 >> [v2:172.21.15.191:6835/2466421758,v1:172.21.15.191:6837/2466421758] conn(0x561476537260 0x561476537630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:36.064 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.054+0000 7f456b91db80 4 client.8277 mds_command: new command op to 8124 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-08T00:21:36.064 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.054+0000 7f456b91db80 1 -- 172.21.15.99:0/677952433 --> [v2:172.21.15.191:6835/2466421758,v1:172.21.15.191:6837/2466421758] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5614764bfaa0 con 0x561476537260 2023-10-08T00:21:36.064 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.054+0000 7f455bfff700 1 --2- 172.21.15.99:0/677952433 >> [v2:172.21.15.191:6835/2466421758,v1:172.21.15.191:6837/2466421758] conn(0x561476537260 0x561476537630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:36.064 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.055+0000 7f453a7fc700 10 client.8277 ms_handle_connect on v2:172.21.15.191:6835/2466421758 2023-10-08T00:21:36.064 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.055+0000 7f455bfff700 1 --2- 172.21.15.99:0/677952433 >> [v2:172.21.15.191:6835/2466421758,v1:172.21.15.191:6837/2466421758] conn(0x561476537260 0x561476537630 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c15764199b8538a server_cookie=49699c72c397227f in_seq=0 out_seq=0 2023-10-08T00:21:36.064 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.055+0000 7f453a7fc700 1 -- 172.21.15.99:0/677952433 <== mds.0 v2:172.21.15.191:6835/2466421758 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5614764bfaa0 con 0x561476537260 2023-10-08T00:21:36.064 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.055+0000 7f453a7fc700 10 client.8277 handle_command_reply: tid=0 2023-10-08T00:21:36.064 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.055+0000 7f456b91db80 1 client.8277 shutdown 2023-10-08T00:21:36.064 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.056+0000 7f456b91db80 1 -- 172.21.15.99:0/677952433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561476537d00 msgr2=0x56147652e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:36.064 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.056+0000 7f456b91db80 1 --2- 172.21.15.99:0/677952433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561476537d00 0x56147652e3b0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f4554007a40 tx=0x7f455401c000 comp rx=0 tx=0).stop 2023-10-08T00:21:36.064 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.056+0000 7f456b91db80 1 -- 172.21.15.99:0/677952433 shutdown_connections 2023-10-08T00:21:36.064 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.056+0000 7f456b91db80 1 --2- 172.21.15.99:0/677952433 >> [v2:172.21.15.191:6835/2466421758,v1:172.21.15.191:6837/2466421758] conn(0x561476537260 0x561476537630 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:36.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.056+0000 7f456b91db80 1 --2- 172.21.15.99:0/677952433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561476537d00 0x56147652e3b0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:36.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.056+0000 7f456b91db80 1 --2- 172.21.15.99:0/677952433 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5614764eda50 0x5614765449d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:36.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.056+0000 7f453a7fc700 0 client.8277 ms_handle_reset on v2:172.21.15.191:6835/2466421758 2023-10-08T00:21:36.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.056+0000 7f456b91db80 1 --2- 172.21.15.99:0/677952433 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561476516f60 0x56147652de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:36.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.056+0000 7f456b91db80 1 -- 172.21.15.99:0/677952433 >> 172.21.15.99:0/677952433 conn(0x5614764ef370 msgr2=0x5614764ea850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:36.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.056+0000 7f456b91db80 1 -- 172.21.15.99:0/677952433 shutdown_connections 2023-10-08T00:21:36.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.056+0000 7f456b91db80 1 -- 172.21.15.99:0/677952433 wait complete. 2023-10-08T00:21:36.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.056+0000 7f456b91db80 20 client.8277 trim_cache size 0 max 16384 2023-10-08T00:21:36.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.072+0000 7f456b91db80 1 Processor -- start 2023-10-08T00:21:36.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.072+0000 7f456b91db80 1 -- start start 2023-10-08T00:21:36.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.073+0000 7f456b91db80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5614764eda50 0x5614764ee220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:36.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.073+0000 7f456b91db80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561476516f60 0x561476533480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:36.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.073+0000 7f456b91db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561476537d00 0x5614765339c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:36.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.073+0000 7f456b91db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561476530720 con 0x561476537d00 2023-10-08T00:21:36.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.073+0000 7f456b91db80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x561476534690 con 0x561476516f60 2023-10-08T00:21:36.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.073+0000 7f456b91db80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x5614764e85a0 con 0x5614764eda50 2023-10-08T00:21:36.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.073+0000 7f4560bc2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561476537d00 0x5614765339c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:36.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.073+0000 7f4560bc2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561476537d00 0x5614765339c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51998/0 (socket says 172.21.15.99:51998) 2023-10-08T00:21:36.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.073+0000 7f4560bc2700 1 -- 172.21.15.99:0/125973614 learned_addr learned my addr 172.21.15.99:0/125973614 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:36.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.073+0000 7f455bfff700 1 --2- 172.21.15.99:0/125973614 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5614764eda50 0x5614764ee220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:36.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.073+0000 7f455b7fe700 1 --2- 172.21.15.99:0/125973614 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561476516f60 0x561476533480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:36.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.074+0000 7f4560bc2700 1 -- 172.21.15.99:0/125973614 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5614764eda50 msgr2=0x5614764ee220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:36.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.074+0000 7f4560bc2700 1 --2- 172.21.15.99:0/125973614 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5614764eda50 0x5614764ee220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:36.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.074+0000 7f4560bc2700 1 -- 172.21.15.99:0/125973614 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561476516f60 msgr2=0x561476533480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:36.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.074+0000 7f4560bc2700 1 --2- 172.21.15.99:0/125973614 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561476516f60 0x561476533480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:36.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.074+0000 7f4560bc2700 1 -- 172.21.15.99:0/125973614 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56147640c780 con 0x561476537d00 2023-10-08T00:21:36.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.074+0000 7f4560bc2700 1 --2- 172.21.15.99:0/125973614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561476537d00 0x5614765339c0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f4554007340 tx=0x7f455400b070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3804111045e91d07 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:36.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.074+0000 7f4541ffb700 1 -- 172.21.15.99:0/125973614 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455401eeb0 con 0x561476537d00 2023-10-08T00:21:36.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.074+0000 7f4541ffb700 1 -- 172.21.15.99:0/125973614 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4554007d80 con 0x561476537d00 2023-10-08T00:21:36.077 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.074+0000 7f4541ffb700 1 -- 172.21.15.99:0/125973614 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455401e400 con 0x561476537d00 2023-10-08T00:21:36.077 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.074+0000 7f456b91db80 1 -- 172.21.15.99:0/125973614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561476537d00 msgr2=0x5614765339c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:36.077 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.074+0000 7f456b91db80 1 --2- 172.21.15.99:0/125973614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561476537d00 0x5614765339c0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f4554007340 tx=0x7f455400b070 comp rx=0 tx=0).stop 2023-10-08T00:21:36.077 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.074+0000 7f456b91db80 1 -- 172.21.15.99:0/125973614 shutdown_connections 2023-10-08T00:21:36.077 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.074+0000 7f456b91db80 1 --2- 172.21.15.99:0/125973614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561476537d00 0x5614765339c0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:36.077 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.074+0000 7f456b91db80 1 --2- 172.21.15.99:0/125973614 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5614764eda50 0x5614764ee220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:36.077 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.074+0000 7f456b91db80 1 --2- 172.21.15.99:0/125973614 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561476516f60 0x561476533480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:36.077 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.074+0000 7f456b91db80 1 -- 172.21.15.99:0/125973614 >> 172.21.15.99:0/125973614 conn(0x5614764ef370 msgr2=0x5614764e64b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:36.077 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.075+0000 7f456b91db80 1 -- 172.21.15.99:0/125973614 shutdown_connections 2023-10-08T00:21:36.077 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.075+0000 7f456b91db80 1 -- 172.21.15.99:0/125973614 wait complete. 2023-10-08T00:21:36.081 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.075+0000 7f456b91db80 1 -- start start 2023-10-08T00:21:36.081 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.075+0000 7f456b91db80 1 Processor -- start 2023-10-08T00:21:36.081 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.076+0000 7f456b91db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614764eda50 0x5614765a2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:36.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.076+0000 7f456b91db80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561476516f60 0x561476533480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:36.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.076+0000 7f456b91db80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561476537d00 0x5614765339c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:36.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.076+0000 7f456b91db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561476554270 con 0x5614764eda50 2023-10-08T00:21:36.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.076+0000 7f456b91db80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614764e9610 con 0x561476516f60 2023-10-08T00:21:36.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.076+0000 7f456b91db80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x5614765304a0 con 0x561476537d00 2023-10-08T00:21:36.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.076+0000 7f456b91db80 10 client.0 _ll_register_callbacks cb 0x5614764b1110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T00:21:36.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.076+0000 7f455bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614764eda50 0x5614765a2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:36.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.076+0000 7f455bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614764eda50 0x5614765a2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52006/0 (socket says 172.21.15.99:52006) 2023-10-08T00:21:36.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.076+0000 7f455bfff700 1 -- 172.21.15.99:0/3566778827 learned_addr learned my addr 172.21.15.99:0/3566778827 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:36.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.076+0000 7f455b7fe700 1 --2- 172.21.15.99:0/3566778827 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561476516f60 0x561476533480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:36.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.076+0000 7f455bfff700 1 -- 172.21.15.99:0/3566778827 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561476537d00 msgr2=0x5614765339c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:36.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.076+0000 7f455bfff700 1 --2- 172.21.15.99:0/3566778827 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561476537d00 0x5614765339c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:36.083 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.076+0000 7f455bfff700 1 -- 172.21.15.99:0/3566778827 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561476516f60 msgr2=0x561476533480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:36.083 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.076+0000 7f455bfff700 1 --2- 172.21.15.99:0/3566778827 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561476516f60 0x561476533480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:36.083 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.076+0000 7f455bfff700 1 -- 172.21.15.99:0/3566778827 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56147627afe0 con 0x5614764eda50 2023-10-08T00:21:36.083 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.076+0000 7f455bfff700 1 --2- 172.21.15.99:0/3566778827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614764eda50 0x5614765a2060 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f454c0030a0 tx=0x7f454c02a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b4a9ebf72cf35af server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:36.083 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.076+0000 7f453a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T00:21:36.083 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.077+0000 7f453a7fc700 1 -- 172.21.15.99:0/3566778827 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454c020070 con 0x5614764eda50 2023-10-08T00:21:36.083 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.077+0000 7f453a7fc700 1 -- 172.21.15.99:0/3566778827 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(184..184 src has 1..184) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f454c01a4f0 con 0x5614764eda50 2023-10-08T00:21:36.083 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.077+0000 7f456b91db80 1 -- 172.21.15.99:0/3566778827 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f455400f050 con 0x5614764eda50 2023-10-08T00:21:36.083 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.077+0000 7f453a7fc700 1 -- 172.21.15.99:0/3566778827 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f454c01a8c0 con 0x5614764eda50 2023-10-08T00:21:36.083 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.077+0000 7f456b91db80 10 client.8283 fetch_fsmap learned FSMap version 84 2023-10-08T00:21:36.083 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.077+0000 7f456b91db80 1 -- 172.21.15.99:0/3566778827 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=2+}) v3 -- 0x5614763f3470 con 0x5614764eda50 2023-10-08T00:21:36.083 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.077+0000 7f453a7fc700 1 -- 172.21.15.99:0/3566778827 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f455400f050 con 0x5614764eda50 2023-10-08T00:21:36.084 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.077+0000 7f453a7fc700 1 -- 172.21.15.99:0/3566778827 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 84) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f454c01aa20 con 0x5614764eda50 2023-10-08T00:21:36.084 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.077+0000 7f456b91db80 10 client.8283 fetch_fsmap finished waiting for FSMap version 84 2023-10-08T00:21:36.084 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.077+0000 7f456b91db80 10 client.8283 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-08T00:21:36.084 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.077+0000 7f456b91db80 20 client.8283 populate_metadata read hostname 'smithi099' 2023-10-08T00:21:36.084 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.077+0000 7f456b91db80 1 --2- 172.21.15.99:0/3566778827 >> [v2:172.21.15.191:6835/2466421758,v1:172.21.15.191:6837/2466421758] conn(0x5614764e9b60 0x5614764e9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:36.084 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.077+0000 7f456b91db80 4 client.8283 mds_command: new command op to 8124 tid=0[{"prefix": "status"}] 2023-10-08T00:21:36.084 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.077+0000 7f456b91db80 1 -- 172.21.15.99:0/3566778827 --> [v2:172.21.15.191:6835/2466421758,v1:172.21.15.191:6837/2466421758] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5614763e0e00 con 0x5614764e9b60 2023-10-08T00:21:36.084 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.077+0000 7f455b7fe700 1 --2- 172.21.15.99:0/3566778827 >> [v2:172.21.15.191:6835/2466421758,v1:172.21.15.191:6837/2466421758] conn(0x5614764e9b60 0x5614764e9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:36.084 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.078+0000 7f453a7fc700 10 client.8283 ms_handle_connect on v2:172.21.15.191:6835/2466421758 2023-10-08T00:21:36.084 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.078+0000 7f455b7fe700 1 --2- 172.21.15.99:0/3566778827 >> [v2:172.21.15.191:6835/2466421758,v1:172.21.15.191:6837/2466421758] conn(0x5614764e9b60 0x5614764e9f30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d85b970c68c87d88 server_cookie=ae63f2330dd6dbd0 in_seq=0 out_seq=0 2023-10-08T00:21:36.084 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.078+0000 7f453a7fc700 1 -- 172.21.15.99:0/3566778827 <== mds.0 v2:172.21.15.191:6835/2466421758 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5614763e0e00 con 0x5614764e9b60 2023-10-08T00:21:36.084 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.078+0000 7f453a7fc700 10 client.8283 handle_command_reply: tid=0 2023-10-08T00:21:36.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.078+0000 7f456b91db80 1 client.8283 shutdown 2023-10-08T00:21:36.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.078+0000 7f456b91db80 1 -- 172.21.15.99:0/3566778827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614764eda50 msgr2=0x5614765a2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:36.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.078+0000 7f456b91db80 1 --2- 172.21.15.99:0/3566778827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614764eda50 0x5614765a2060 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f454c0030a0 tx=0x7f454c02a040 comp rx=0 tx=0).stop 2023-10-08T00:21:36.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.078+0000 7f456b91db80 1 -- 172.21.15.99:0/3566778827 shutdown_connections 2023-10-08T00:21:36.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.078+0000 7f456b91db80 1 --2- 172.21.15.99:0/3566778827 >> [v2:172.21.15.191:6835/2466421758,v1:172.21.15.191:6837/2466421758] conn(0x5614764e9b60 0x5614764e9f30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:36.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.078+0000 7f456b91db80 1 --2- 172.21.15.99:0/3566778827 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561476537d00 0x5614765339c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:36.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.078+0000 7f453a7fc700 0 client.8283 ms_handle_reset on v2:172.21.15.191:6835/2466421758 2023-10-08T00:21:36.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.078+0000 7f456b91db80 1 --2- 172.21.15.99:0/3566778827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614764eda50 0x5614765a2060 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:36.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.078+0000 7f456b91db80 1 --2- 172.21.15.99:0/3566778827 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561476516f60 0x561476533480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:36.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.078+0000 7f456b91db80 1 -- 172.21.15.99:0/3566778827 >> 172.21.15.99:0/3566778827 conn(0x5614764ef370 msgr2=0x561476531dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:36.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.078+0000 7f456b91db80 1 -- 172.21.15.99:0/3566778827 shutdown_connections 2023-10-08T00:21:36.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.078+0000 7f456b91db80 1 -- 172.21.15.99:0/3566778827 wait complete. 2023-10-08T00:21:36.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.078+0000 7f456b91db80 20 client.8283 trim_cache size 0 max 16384 2023-10-08T00:21:36.086 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:21:36.086 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", 2023-10-08T00:21:36.086 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 0, 2023-10-08T00:21:36.086 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 8124, 2023-10-08T00:21:36.086 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2023-10-08T00:21:36.086 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2023-10-08T00:21:36.086 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2023-10-08T00:21:36.086 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 5.7758410170000003, 2023-10-08T00:21:36.087 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 84, 2023-10-08T00:21:36.087 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 184, 2023-10-08T00:21:36.087 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 184, 2023-10-08T00:21:36.087 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 10.851479345 2023-10-08T00:21:36.087 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:21:36.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.078+0000 7f4562625700 1 -- 172.21.15.99:0/2177907816 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4548066f30 msgr2=0x7f45480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:36.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.078+0000 7f4562625700 1 --2- 172.21.15.99:0/2177907816 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4548066f30 0x7f45480693e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f454c0091e0 tx=0x7f454c017000 comp rx=0 tx=0).stop 2023-10-08T00:21:36.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.079+0000 7f4562625700 1 -- 172.21.15.99:0/2177907816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f455c157c40 msgr2=0x7f455c16a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:36.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.079+0000 7f4562625700 1 --2- 172.21.15.99:0/2177907816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f455c157c40 0x7f455c16a360 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f455000c590 tx=0x7f4550001020 comp rx=0 tx=0).stop 2023-10-08T00:21:36.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.079+0000 7f4562625700 1 -- 172.21.15.99:0/2177907816 shutdown_connections 2023-10-08T00:21:36.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.079+0000 7f4562625700 1 --2- 172.21.15.99:0/2177907816 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4548066f30 0x7f45480693e0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:36.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.079+0000 7f4562625700 1 --2- 172.21.15.99:0/2177907816 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f455c15a620 0x7f455c16a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:36.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.079+0000 7f4562625700 1 --2- 172.21.15.99:0/2177907816 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f455c157870 0x7f455c151080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:36.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.079+0000 7f4562625700 1 --2- 172.21.15.99:0/2177907816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f455c157c40 0x7f455c16a360 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:36.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.079+0000 7f4562625700 1 -- 172.21.15.99:0/2177907816 >> 172.21.15.99:0/2177907816 conn(0x7f455c068ce0 msgr2=0x7f455c0b46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:36.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.079+0000 7f4562625700 1 -- 172.21.15.99:0/2177907816 shutdown_connections 2023-10-08T00:21:36.089 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.079+0000 7f4562625700 1 -- 172.21.15.99:0/2177907816 wait complete. 2023-10-08T00:21:36.091 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T00:21:36.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.205+0000 7fb823063700 1 Processor -- start 2023-10-08T00:21:36.206 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.205+0000 7fb823063700 1 -- start start 2023-10-08T00:21:36.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.206+0000 7fb823063700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb81c159e10 0x7fb81c0b2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:36.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.206+0000 7fb823063700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb81c15a1e0 0x7fb81c0b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:36.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.206+0000 7fb823063700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb81c15db80 con 0x7fb81c0b1610 2023-10-08T00:21:36.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.206+0000 7fb823063700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb81c15e800 con 0x7fb81c159e10 2023-10-08T00:21:36.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.206+0000 7fb823063700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb81c15de60 con 0x7fb81c15a1e0 2023-10-08T00:21:36.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.206+0000 7fb820dff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb81c159e10 0x7fb81c0b2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:36.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.206+0000 7fb820dff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb81c159e10 0x7fb81c0b2d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52010/0 (socket says 172.21.15.99:52010) 2023-10-08T00:21:36.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.206+0000 7fb820dff700 1 -- 172.21.15.99:0/2331120678 learned_addr learned my addr 172.21.15.99:0/2331120678 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:36.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.206+0000 7fb821600700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fb81c0b1610 0x7fb81c0cd740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:57698/0 (socket says 172.21.15.99:57698) 2023-10-08T00:21:36.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.206+0000 7fb820dff700 1 -- 172.21.15.99:0/2331120678 >> v2:172.21.15.191:3300/0 conn(0x7fb81c15a1e0 msgr2=0x7fb81c0b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:36.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.206+0000 7fb81bfff700 1 --2- 172.21.15.99:0/2331120678 >> v2:172.21.15.191:3300/0 conn(0x7fb81c15a1e0 0x7fb81c0b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:36.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.206+0000 7fb820dff700 1 --2- 172.21.15.99:0/2331120678 >> v2:172.21.15.191:3300/0 conn(0x7fb81c15a1e0 0x7fb81c0b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:36.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.206+0000 7fb820dff700 1 -- 172.21.15.99:0/2331120678 >> v1:172.21.15.99:6789/0 conn(0x7fb81c0b1610 legacy=0x7fb81c0cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:36.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.206+0000 7fb81b7fe700 1 -- 172.21.15.99:0/2331120678 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3738882259 0 0) 0x7fb81c15db80 con 0x7fb81c0b1610 2023-10-08T00:21:36.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.206+0000 7fb820dff700 1 -- 172.21.15.99:0/2331120678 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb81c15e140 con 0x7fb81c159e10 2023-10-08T00:21:36.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.206+0000 7fb81bfff700 1 --2- 172.21.15.99:0/2331120678 >> v2:172.21.15.191:3300/0 conn(0x7fb81c15a1e0 0x7fb81c0b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:21:36.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.206+0000 7fb820dff700 1 --2- 172.21.15.99:0/2331120678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb81c159e10 0x7fb81c0b2d20 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fb81c068580 tx=0x7fb810009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf1e2655ac5c028 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:36.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.207+0000 7fb81b7fe700 1 -- 172.21.15.99:0/2331120678 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81000b860 con 0x7fb81c159e10 2023-10-08T00:21:36.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.207+0000 7fb81b7fe700 1 -- 172.21.15.99:0/2331120678 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb81000b9c0 con 0x7fb81c159e10 2023-10-08T00:21:36.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.207+0000 7fb81b7fe700 1 -- 172.21.15.99:0/2331120678 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81000bd60 con 0x7fb81c159e10 2023-10-08T00:21:36.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.207+0000 7fb823063700 1 -- 172.21.15.99:0/2331120678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb81c159e10 msgr2=0x7fb81c0b2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:36.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.207+0000 7fb823063700 1 --2- 172.21.15.99:0/2331120678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb81c159e10 0x7fb81c0b2d20 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fb81c068580 tx=0x7fb810009250 comp rx=0 tx=0).stop 2023-10-08T00:21:36.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.207+0000 7fb823063700 1 -- 172.21.15.99:0/2331120678 shutdown_connections 2023-10-08T00:21:36.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.207+0000 7fb823063700 1 --2- 172.21.15.99:0/2331120678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb81c159e10 0x7fb81c0b2d20 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:36.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.207+0000 7fb823063700 1 --2- 172.21.15.99:0/2331120678 >> v2:172.21.15.191:3300/0 conn(0x7fb81c15a1e0 0x7fb81c0b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:36.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.207+0000 7fb823063700 1 -- 172.21.15.99:0/2331120678 >> 172.21.15.99:0/2331120678 conn(0x7fb81c067660 msgr2=0x7fb81c067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:36.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.207+0000 7fb823063700 1 -- 172.21.15.99:0/2331120678 shutdown_connections 2023-10-08T00:21:36.211 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.207+0000 7fb823063700 1 -- 172.21.15.99:0/2331120678 wait complete. 2023-10-08T00:21:36.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.208+0000 7fb823063700 1 Processor -- start 2023-10-08T00:21:36.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.209+0000 7fb823063700 1 -- start start 2023-10-08T00:21:36.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.209+0000 7fb823063700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb81c0b1610 0x7fb81c0c6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:36.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.209+0000 7fb823063700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb81c159e10 0x7fb81c0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:36.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.209+0000 7fb823063700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb81c15a1e0 0x7fb81c157120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:36.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.209+0000 7fb823063700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c0d0b00 con 0x7fb81c0b1610 2023-10-08T00:21:36.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.209+0000 7fb823063700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c0d0670 con 0x7fb81c15a1e0 2023-10-08T00:21:36.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.209+0000 7fb823063700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c0cf7c0 con 0x7fb81c159e10 2023-10-08T00:21:36.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.209+0000 7fb820dff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb81c0b1610 0x7fb81c0c6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:36.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.209+0000 7fb820dff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb81c0b1610 0x7fb81c0c6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52026/0 (socket says 172.21.15.99:52026) 2023-10-08T00:21:36.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.209+0000 7fb820dff700 1 -- 172.21.15.99:0/2047138219 learned_addr learned my addr 172.21.15.99:0/2047138219 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:36.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.210+0000 7fb821600700 1 --2- 172.21.15.99:0/2047138219 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb81c15a1e0 0x7fb81c157120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:36.214 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.210+0000 7fb820dff700 1 -- 172.21.15.99:0/2047138219 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb81c159e10 msgr2=0x7fb81c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:36.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.210+0000 7fb820dff700 1 --2- 172.21.15.99:0/2047138219 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb81c159e10 0x7fb81c0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:36.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.210+0000 7fb820dff700 1 -- 172.21.15.99:0/2047138219 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb81c15a1e0 msgr2=0x7fb81c157120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:36.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.210+0000 7fb820dff700 1 --2- 172.21.15.99:0/2047138219 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb81c15a1e0 0x7fb81c157120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:36.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.210+0000 7fb820dff700 1 -- 172.21.15.99:0/2047138219 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8100094f0 con 0x7fb81c0b1610 2023-10-08T00:21:36.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.210+0000 7fb821600700 1 --2- 172.21.15.99:0/2047138219 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb81c15a1e0 0x7fb81c157120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:21:36.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.210+0000 7fb820dff700 1 --2- 172.21.15.99:0/2047138219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb81c0b1610 0x7fb81c0c6c60 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fb81c089c80 tx=0x7fb810017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:36.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.210+0000 7fb818ff9700 1 -- 172.21.15.99:0/2047138219 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb810024440 con 0x7fb81c0b1610 2023-10-08T00:21:36.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.210+0000 7fb818ff9700 1 -- 172.21.15.99:0/2047138219 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8100245e0 con 0x7fb81c0b1610 2023-10-08T00:21:36.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.210+0000 7fb823063700 1 -- 172.21.15.99:0/2047138219 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb81c0cfa60 con 0x7fb81c0b1610 2023-10-08T00:21:36.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.210+0000 7fb818ff9700 1 -- 172.21.15.99:0/2047138219 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8100249a0 con 0x7fb81c0b1610 2023-10-08T00:21:36.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.210+0000 7fb823063700 1 -- 172.21.15.99:0/2047138219 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb81c0d0020 con 0x7fb81c0b1610 2023-10-08T00:21:36.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.211+0000 7fb823063700 1 -- 172.21.15.99:0/2047138219 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb81c04c3c0 con 0x7fb81c0b1610 2023-10-08T00:21:36.216 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.211+0000 7fb818ff9700 1 -- 172.21.15.99:0/2047138219 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb810024d50 con 0x7fb81c0b1610 2023-10-08T00:21:36.216 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.212+0000 7fb818ff9700 1 --2- 172.21.15.99:0/2047138219 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb800066f30 0x7fb8000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:36.216 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.212+0000 7fb818ff9700 1 -- 172.21.15.99:0/2047138219 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb8100861b0 con 0x7fb81c0b1610 2023-10-08T00:21:36.216 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.212+0000 7fb81bfff700 1 --2- 172.21.15.99:0/2047138219 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb800066f30 0x7fb8000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:36.216 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.212+0000 7fb81bfff700 1 --2- 172.21.15.99:0/2047138219 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb800066f30 0x7fb8000693e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb808006fd0 tx=0x7fb808008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:36.216 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.216+0000 7fb818ff9700 1 -- 172.21.15.99:0/2047138219 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb810053420 con 0x7fb81c0b1610 2023-10-08T00:21:36.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.441+0000 7fb823063700 1 -- 172.21.15.99:0/2047138219 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb81c15e420 con 0x7fb81c0b1610 2023-10-08T00:21:36.442 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.442+0000 7fb818ff9700 1 -- 172.21.15.99:0/2047138219 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v84) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb81002b340 con 0x7fb81c0b1610 2023-10-08T00:21:36.442 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T00:21:36.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.444+0000 7fb823063700 1 -- 172.21.15.99:0/2047138219 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb800066f30 msgr2=0x7fb8000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:36.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.444+0000 7fb823063700 1 --2- 172.21.15.99:0/2047138219 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb800066f30 0x7fb8000693e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb808006fd0 tx=0x7fb808008040 comp rx=0 tx=0).stop 2023-10-08T00:21:36.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.444+0000 7fb823063700 1 -- 172.21.15.99:0/2047138219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb81c0b1610 msgr2=0x7fb81c0c6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:36.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.444+0000 7fb823063700 1 --2- 172.21.15.99:0/2047138219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb81c0b1610 0x7fb81c0c6c60 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fb81c089c80 tx=0x7fb810017000 comp rx=0 tx=0).stop 2023-10-08T00:21:36.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.444+0000 7fb823063700 1 -- 172.21.15.99:0/2047138219 shutdown_connections 2023-10-08T00:21:36.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.444+0000 7fb823063700 1 --2- 172.21.15.99:0/2047138219 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb800066f30 0x7fb8000693e0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:36.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.444+0000 7fb823063700 1 --2- 172.21.15.99:0/2047138219 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb81c15a1e0 0x7fb81c157120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:36.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.444+0000 7fb823063700 1 --2- 172.21.15.99:0/2047138219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb81c0b1610 0x7fb81c0c6c60 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:36.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.444+0000 7fb823063700 1 --2- 172.21.15.99:0/2047138219 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb81c159e10 0x7fb81c0cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:36.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.444+0000 7fb823063700 1 -- 172.21.15.99:0/2047138219 >> 172.21.15.99:0/2047138219 conn(0x7fb81c067660 msgr2=0x7fb81c0cbb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:36.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.444+0000 7fb823063700 1 -- 172.21.15.99:0/2047138219 shutdown_connections 2023-10-08T00:21:36.446 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:36.444+0000 7fb823063700 1 -- 172.21.15.99:0/2047138219 wait complete. 2023-10-08T00:21:36.457 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T00:21:36.457 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T00:21:36.457 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi099.front.sepia.ceph.com 2023-10-08T00:21:36.457 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T00:21:36.457 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T00:21:36.457 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T00:21:36.457 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T00:21:36.457 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T00:21:36.457 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T00:21:36.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:21:36.458 DEBUG:teuthology.orchestra.run.smithi099:> ip addr 2023-10-08T00:21:36.472 INFO:teuthology.orchestra.run.smithi099.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T00:21:36.473 INFO:teuthology.orchestra.run.smithi099.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T00:21:36.473 INFO:teuthology.orchestra.run.smithi099.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T00:21:36.473 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:21:36.473 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 ::1/128 scope host 2023-10-08T00:21:36.473 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:21:36.473 INFO:teuthology.orchestra.run.smithi099.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:21:36.473 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:10 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:21:36.473 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f0 2023-10-08T00:21:36.473 INFO:teuthology.orchestra.run.smithi099.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:21:36.473 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:11 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:21:36.473 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f1 2023-10-08T00:21:36.474 INFO:teuthology.orchestra.run.smithi099.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:21:36.474 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:02 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:21:36.474 INFO:teuthology.orchestra.run.smithi099.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T00:21:36.474 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:03 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:21:36.474 INFO:teuthology.orchestra.run.smithi099.stdout: inet 172.21.15.99/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T00:21:36.474 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft 41893sec preferred_lft 41893sec 2023-10-08T00:21:36.474 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::ec4:7aff:fe8f:d803/64 scope link 2023-10-08T00:21:36.474 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:21:36.474 INFO:teuthology.orchestra.run.smithi099.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T00:21:36.474 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 22:2c:1c:f6:42:e1 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:21:36.475 INFO:teuthology.orchestra.run.smithi099.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T00:21:36.475 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:21:36.475 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::600d:26ff:fede:5eec/64 scope link 2023-10-08T00:21:36.475 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:21:36.475 INFO:teuthology.orchestra.run.smithi099.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T00:21:36.475 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether ae:2f:b5:94:69:ba brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T00:21:36.475 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::ac2f:b5ff:fe94:69ba/64 scope link 2023-10-08T00:21:36.475 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:21:36.475 INFO:teuthology.orchestra.run.smithi099.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T00:21:36.475 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 22:2c:1c:f6:42:e1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T00:21:36.475 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::202c:1cff:fef6:42e1/64 scope link 2023-10-08T00:21:36.475 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:21:36.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:21:36.476 DEBUG:teuthology.orchestra.run.smithi099:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T00:21:36.535 INFO:teuthology.orchestra.run.smithi099.stdout:1 2023-10-08T00:21:36.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:21:36.536 DEBUG:teuthology.orchestra.run.smithi099:> route 2023-10-08T00:21:36.592 INFO:teuthology.orchestra.run.smithi099.stdout:Kernel IP routing table 2023-10-08T00:21:36.592 INFO:teuthology.orchestra.run.smithi099.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T00:21:36.592 INFO:teuthology.orchestra.run.smithi099.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T00:21:36.593 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T00:21:36.593 INFO:teuthology.orchestra.run.smithi099.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T00:21:36.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:21:36.593 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2023-10-08T00:21:36.593 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2023-10-08T00:21:36.593 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T00:21:36.593 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T00:21:36.594 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T00:21:36.594 DEBUG:teuthology.orchestra.run.smithi099:> ') 2023-10-08T00:21:36.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:21:36.695 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2023-10-08T00:21:36.710 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T00:21:36.710 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T00:21:36.711 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T00:21:36.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:21:36.711 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set brx.0 up 2023-10-08T00:21:36.774 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:21:36.775 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T00:21:36.829 INFO:teuthology.orchestra.run.smithi099.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T00:21:36.829 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:21:36.830 DEBUG:teuthology.orchestra.run.smithi099:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T00:21:36.883 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-08T00:21:36.884 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T00:21:36.884 DEBUG:teuthology.orchestra.run.smithi099:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:21:36.963 INFO:teuthology.orchestra.run.smithi099.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:21:36.964 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:21:36.982 INFO:teuthology.orchestra.run.smithi099.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:21:36.982 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:21:36.982 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: resolved to: "172.21.15.99:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T00:21:36.983 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: trying mount with new device syntax: 0@06962cdb-9a6f-417f-82fd-9e4e3dedbccf.cephfs=/ 2023-10-08T00:21:36.983 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.99:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T00:21:36.983 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T00:21:36.983 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T00:21:36.983 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T00:21:37.004 INFO:teuthology.orchestra.run.smithi099.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:21:37.004 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:21:37.004 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: resolved to: "172.21.15.99:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T00:21:37.004 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: trying mount with new device syntax: 0@06962cdb-9a6f-417f-82fd-9e4e3dedbccf.cephfs=/ 2023-10-08T00:21:37.004 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.99:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T00:21:37.004 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T00:21:37.004 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T00:21:37.005 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T00:21:37.005 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: resolved to: "172.21.15.99:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T00:21:37.005 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.99:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T00:21:37.005 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=06962cdb-9a6f-417f-82fd-9e4e3dedbccf" will pass to kernel 2023-10-08T00:21:37.006 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T00:21:37.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:21:37.006 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T00:21:37.071 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T00:21:37.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.226+0000 7fa3fa8f3700 1 Processor -- start 2023-10-08T00:21:37.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.226+0000 7fa3fa8f3700 1 -- start start 2023-10-08T00:21:37.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.227+0000 7fa3fa8f3700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa3f415cbd0 0x7fa3f415d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:37.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.227+0000 7fa3fa8f3700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3f40cb7c0 con 0x7fa3f40cd6e0 2023-10-08T00:21:37.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.227+0000 7fa3fa8f3700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3f415e290 con 0x7fa3f40cdab0 2023-10-08T00:21:37.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.227+0000 7fa3fa8f3700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3f415e850 con 0x7fa3f415cbd0 2023-10-08T00:21:37.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.227+0000 7fa3f98f1700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fa3f40cd6e0 0x7fa3f40cbc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:57708/0 (socket says 172.21.15.99:57708) 2023-10-08T00:21:37.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.227+0000 7fa3f98f1700 1 -- 172.21.15.99:0/795160772 learned_addr learned my addr 172.21.15.99:0/795160772 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:21:37.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.227+0000 7fa3fa0f2700 1 --2- 172.21.15.99:0/795160772 >> v2:172.21.15.191:3300/0 conn(0x7fa3f415cbd0 0x7fa3f415d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:37.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.227+0000 7fa3f88ef700 1 -- 172.21.15.99:0/795160772 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2939051730 0 0) 0x7fa3f40cb7c0 con 0x7fa3f40cd6e0 2023-10-08T00:21:37.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.227+0000 7fa3f88ef700 1 -- 172.21.15.99:0/795160772 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3e0003570 con 0x7fa3f40cd6e0 2023-10-08T00:21:37.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.228+0000 7fa3fa0f2700 1 -- 172.21.15.99:0/795160772 >> v1:172.21.15.191:6789/0 conn(0x7fa3f40cdab0 legacy=0x7fa3f415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:37.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.228+0000 7fa3f88ef700 1 -- 172.21.15.99:0/795160772 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1410224969 0 0) 0x7fa3f415e290 con 0x7fa3f40cdab0 2023-10-08T00:21:37.229 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.228+0000 7fa3fa0f2700 1 -- 172.21.15.99:0/795160772 >> v1:172.21.15.99:6789/0 conn(0x7fa3f40cd6e0 legacy=0x7fa3f40cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:37.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.228+0000 7fa3fa0f2700 1 -- 172.21.15.99:0/795160772 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f415e570 con 0x7fa3f415cbd0 2023-10-08T00:21:37.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.228+0000 7fa3fa0f2700 1 --2- 172.21.15.99:0/795160772 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3f415cbd0 0x7fa3f415d050 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec0098e0 tx=0x7fa3ec0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=354665296c1e3471 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:37.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.228+0000 7fa3f88ef700 1 -- 172.21.15.99:0/795160772 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f00017f0 con 0x7fa3f415cbd0 2023-10-08T00:21:37.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.228+0000 7fa3f88ef700 1 -- 172.21.15.99:0/795160772 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3ec011420 con 0x7fa3f415cbd0 2023-10-08T00:21:37.230 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.228+0000 7fa3f88ef700 1 -- 172.21.15.99:0/795160772 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec0117c0 con 0x7fa3f415cbd0 2023-10-08T00:21:37.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.228+0000 7fa3fa8f3700 1 -- 172.21.15.99:0/795160772 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3f415cbd0 msgr2=0x7fa3f415d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:37.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.228+0000 7fa3fa8f3700 1 --2- 172.21.15.99:0/795160772 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3f415cbd0 0x7fa3f415d050 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec0098e0 tx=0x7fa3ec0091a0 comp rx=0 tx=0).stop 2023-10-08T00:21:37.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.229+0000 7fa3fa8f3700 1 -- 172.21.15.99:0/795160772 shutdown_connections 2023-10-08T00:21:37.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.229+0000 7fa3fa8f3700 1 --2- 172.21.15.99:0/795160772 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3f415cbd0 0x7fa3f415d050 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:37.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.229+0000 7fa3fa8f3700 1 -- 172.21.15.99:0/795160772 >> 172.21.15.99:0/795160772 conn(0x7fa3f4067660 msgr2=0x7fa3f4067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:37.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.229+0000 7fa3fa8f3700 1 -- 172.21.15.99:0/795160772 shutdown_connections 2023-10-08T00:21:37.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.229+0000 7fa3fa8f3700 1 -- 172.21.15.99:0/795160772 wait complete. 2023-10-08T00:21:37.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.230+0000 7fa3fa8f3700 1 Processor -- start 2023-10-08T00:21:37.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.230+0000 7fa3fa8f3700 1 -- start start 2023-10-08T00:21:37.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.230+0000 7fa3fa8f3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3f40cd6e0 0x7fa3f414a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:37.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.230+0000 7fa3fa8f3700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3f40cdab0 0x7fa3f415b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:37.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.230+0000 7fa3fa8f3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3f415cbd0 0x7fa3f415be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:37.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.230+0000 7fa3fa8f3700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f40cc360 con 0x7fa3f40cd6e0 2023-10-08T00:21:37.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.230+0000 7fa3fa8f3700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f415ac30 con 0x7fa3f40cdab0 2023-10-08T00:21:37.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.230+0000 7fa3fa8f3700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f4155d10 con 0x7fa3f415cbd0 2023-10-08T00:21:37.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.230+0000 7fa3f98f1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3f40cd6e0 0x7fa3f414a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:37.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.230+0000 7fa3f98f1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3f40cd6e0 0x7fa3f414a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52040/0 (socket says 172.21.15.99:52040) 2023-10-08T00:21:37.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.230+0000 7fa3f98f1700 1 -- 172.21.15.99:0/3271612861 learned_addr learned my addr 172.21.15.99:0/3271612861 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:37.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.231+0000 7fa3fa0f2700 1 --2- 172.21.15.99:0/3271612861 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3f415cbd0 0x7fa3f415be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:37.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.231+0000 7fa3f90f0700 1 --2- 172.21.15.99:0/3271612861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3f40cdab0 0x7fa3f415b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:37.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.231+0000 7fa3f98f1700 1 -- 172.21.15.99:0/3271612861 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3f415cbd0 msgr2=0x7fa3f415be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:37.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.231+0000 7fa3f98f1700 1 --2- 172.21.15.99:0/3271612861 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3f415cbd0 0x7fa3f415be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:37.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.231+0000 7fa3f98f1700 1 -- 172.21.15.99:0/3271612861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3f40cdab0 msgr2=0x7fa3f415b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:37.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.231+0000 7fa3f98f1700 1 --2- 172.21.15.99:0/3271612861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3f40cdab0 0x7fa3f415b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:37.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.231+0000 7fa3f98f1700 1 -- 172.21.15.99:0/3271612861 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3ec009420 con 0x7fa3f40cd6e0 2023-10-08T00:21:37.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.231+0000 7fa3fa0f2700 1 --2- 172.21.15.99:0/3271612861 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3f415cbd0 0x7fa3f415be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:21:37.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.231+0000 7fa3f98f1700 1 --2- 172.21.15.99:0/3271612861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3f40cd6e0 0x7fa3f414a830 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0001560 tx=0x7fa3f0007e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:37.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.231+0000 7fa3e9ffb700 1 -- 172.21.15.99:0/3271612861 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f0004dc0 con 0x7fa3f40cd6e0 2023-10-08T00:21:37.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.231+0000 7fa3e9ffb700 1 -- 172.21.15.99:0/3271612861 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3f000a930 con 0x7fa3f40cd6e0 2023-10-08T00:21:37.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.231+0000 7fa3fa8f3700 1 -- 172.21.15.99:0/3271612861 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3f414ad70 con 0x7fa3f40cd6e0 2023-10-08T00:21:37.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.231+0000 7fa3e9ffb700 1 -- 172.21.15.99:0/3271612861 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f000acf0 con 0x7fa3f40cd6e0 2023-10-08T00:21:37.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.231+0000 7fa3fa8f3700 1 -- 172.21.15.99:0/3271612861 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3f415b120 con 0x7fa3f40cd6e0 2023-10-08T00:21:37.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.232+0000 7fa3fa8f3700 1 -- 172.21.15.99:0/3271612861 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3f4089280 con 0x7fa3f40cd6e0 2023-10-08T00:21:37.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.232+0000 7fa3e9ffb700 1 -- 172.21.15.99:0/3271612861 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa3f0007360 con 0x7fa3f40cd6e0 2023-10-08T00:21:37.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.233+0000 7fa3e9ffb700 1 --2- 172.21.15.99:0/3271612861 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa3d8066f30 0x7fa3d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:37.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.233+0000 7fa3e9ffb700 1 -- 172.21.15.99:0/3271612861 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa3f0084090 con 0x7fa3f40cd6e0 2023-10-08T00:21:37.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.233+0000 7fa3f90f0700 1 --2- 172.21.15.99:0/3271612861 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa3d8066f30 0x7fa3d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:37.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.234+0000 7fa3f90f0700 1 --2- 172.21.15.99:0/3271612861 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa3d8066f30 0x7fa3d80693e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa3e40011e0 tx=0x7fa3e4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:37.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.237+0000 7fa3e9ffb700 1 -- 172.21.15.99:0/3271612861 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa3f0051380 con 0x7fa3f40cd6e0 2023-10-08T00:21:37.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.466+0000 7fa3fa8f3700 1 -- 172.21.15.99:0/3271612861 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa3f404bb80 con 0x7fa3f40cd6e0 2023-10-08T00:21:37.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.466+0000 7fa3e9ffb700 1 -- 172.21.15.99:0/3271612861 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v84) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa3f0018060 con 0x7fa3f40cd6e0 2023-10-08T00:21:37.468 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T00:21:37.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.468+0000 7fa3fa8f3700 1 -- 172.21.15.99:0/3271612861 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa3d8066f30 msgr2=0x7fa3d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:37.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.468+0000 7fa3fa8f3700 1 --2- 172.21.15.99:0/3271612861 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa3d8066f30 0x7fa3d80693e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa3e40011e0 tx=0x7fa3e4009040 comp rx=0 tx=0).stop 2023-10-08T00:21:37.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.469+0000 7fa3fa8f3700 1 -- 172.21.15.99:0/3271612861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3f40cd6e0 msgr2=0x7fa3f414a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:37.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.469+0000 7fa3fa8f3700 1 --2- 172.21.15.99:0/3271612861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3f40cd6e0 0x7fa3f414a830 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0001560 tx=0x7fa3f0007e90 comp rx=0 tx=0).stop 2023-10-08T00:21:37.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.469+0000 7fa3fa8f3700 1 -- 172.21.15.99:0/3271612861 shutdown_connections 2023-10-08T00:21:37.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.469+0000 7fa3fa8f3700 1 --2- 172.21.15.99:0/3271612861 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa3d8066f30 0x7fa3d80693e0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:37.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.469+0000 7fa3fa8f3700 1 --2- 172.21.15.99:0/3271612861 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3f415cbd0 0x7fa3f415be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:37.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.469+0000 7fa3fa8f3700 1 --2- 172.21.15.99:0/3271612861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3f40cd6e0 0x7fa3f414a830 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:37.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.469+0000 7fa3fa8f3700 1 --2- 172.21.15.99:0/3271612861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3f40cdab0 0x7fa3f415b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:37.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.469+0000 7fa3fa8f3700 1 -- 172.21.15.99:0/3271612861 >> 172.21.15.99:0/3271612861 conn(0x7fa3f4067660 msgr2=0x7fa3f40d1630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:37.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.469+0000 7fa3fa8f3700 1 -- 172.21.15.99:0/3271612861 shutdown_connections 2023-10-08T00:21:37.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:37.469+0000 7fa3fa8f3700 1 -- 172.21.15.99:0/3271612861 wait complete. 2023-10-08T00:21:37.483 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T00:21:37.484 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T00:21:37.484 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi191.front.sepia.ceph.com 2023-10-08T00:21:37.484 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-08T00:21:37.484 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-08T00:21:37.484 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T00:21:37.484 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T00:21:37.484 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T00:21:37.484 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T00:21:37.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:21:37.485 DEBUG:teuthology.orchestra.run.smithi191:> ip addr 2023-10-08T00:21:37.504 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T00:21:37.504 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T00:21:37.504 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T00:21:37.504 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:21:37.504 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2023-10-08T00:21:37.504 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:21:37.504 INFO:teuthology.orchestra.run.smithi191.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:21:37.505 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f6 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:21:37.505 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f0 2023-10-08T00:21:37.505 INFO:teuthology.orchestra.run.smithi191.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:21:37.505 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f7 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:21:37.505 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f1 2023-10-08T00:21:37.505 INFO:teuthology.orchestra.run.smithi191.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:21:37.505 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:10 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:21:37.505 INFO:teuthology.orchestra.run.smithi191.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T00:21:37.505 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:11 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:21:37.505 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.21.15.191/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T00:21:37.505 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft 41894sec preferred_lft 41894sec 2023-10-08T00:21:37.506 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec4:7aff:fe88:7211/64 scope link 2023-10-08T00:21:37.506 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:21:37.506 INFO:teuthology.orchestra.run.smithi191.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T00:21:37.506 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 9a:bc:39:72:87:3d brd ff:ff:ff:ff:ff:ff 2023-10-08T00:21:37.506 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T00:21:37.506 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:21:37.506 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec77:a7ff:fea4:fbb/64 scope link 2023-10-08T00:21:37.506 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:21:37.506 INFO:teuthology.orchestra.run.smithi191.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T00:21:37.506 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether ea:1c:d9:46:05:b3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-08T00:21:37.506 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::e81c:d9ff:fe46:5b3/64 scope link 2023-10-08T00:21:37.507 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:21:37.507 INFO:teuthology.orchestra.run.smithi191.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T00:21:37.507 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 9a:bc:39:72:87:3d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-08T00:21:37.507 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::98bc:39ff:fe72:873d/64 scope link 2023-10-08T00:21:37.507 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:21:37.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:21:37.507 DEBUG:teuthology.orchestra.run.smithi191:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T00:21:37.571 INFO:teuthology.orchestra.run.smithi191.stdout:1 2023-10-08T00:21:37.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:21:37.573 DEBUG:teuthology.orchestra.run.smithi191:> route 2023-10-08T00:21:37.631 INFO:teuthology.orchestra.run.smithi191.stdout:Kernel IP routing table 2023-10-08T00:21:37.631 INFO:teuthology.orchestra.run.smithi191.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T00:21:37.631 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T00:21:37.632 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T00:21:37.632 INFO:teuthology.orchestra.run.smithi191.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T00:21:37.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:21:37.632 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2023-10-08T00:21:37.632 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2023-10-08T00:21:37.632 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T00:21:37.632 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T00:21:37.632 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T00:21:37.633 DEBUG:teuthology.orchestra.run.smithi191:> ') 2023-10-08T00:21:37.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:21:37.738 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2023-10-08T00:21:37.753 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-08T00:21:37.753 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-08T00:21:37.753 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T00:21:37.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:21:37.754 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set brx.0 up 2023-10-08T00:21:37.819 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:21:37.821 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-08T00:21:37.875 INFO:teuthology.orchestra.run.smithi191.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-08T00:21:37.875 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:21:37.875 DEBUG:teuthology.orchestra.run.smithi191:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-08T00:21:37.929 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-08T00:21:37.930 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T00:21:37.930 DEBUG:teuthology.orchestra.run.smithi191:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:21:38.009 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:21:38.009 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:21:38.028 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:21:38.028 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:21:38.028 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.99:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T00:21:38.028 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@06962cdb-9a6f-417f-82fd-9e4e3dedbccf.cephfs=/ 2023-10-08T00:21:38.028 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.99:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T00:21:38.029 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T00:21:38.029 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T00:21:38.029 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T00:21:38.050 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:21:38.050 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:21:38.050 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.99:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T00:21:38.051 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@06962cdb-9a6f-417f-82fd-9e4e3dedbccf.cephfs=/ 2023-10-08T00:21:38.051 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.99:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T00:21:38.051 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T00:21:38.051 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T00:21:38.051 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T00:21:38.051 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.99:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T00:21:38.051 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.99:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T00:21:38.051 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=06962cdb-9a6f-417f-82fd-9e4e3dedbccf" will pass to kernel 2023-10-08T00:21:38.052 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T00:21:38.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:21:38.053 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-08T00:21:38.117 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:21:38.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.235+0000 7f3b0c9a3700 1 Processor -- start 2023-10-08T00:21:38.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.235+0000 7f3b0c9a3700 1 -- start start 2023-10-08T00:21:38.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.236+0000 7f3b0c9a3700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3b08157870 0x7f3b0815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:38.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.236+0000 7f3b0c9a3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b0815a620 0x7f3b0815d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:38.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.236+0000 7f3b0c9a3700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b0815cbb0 con 0x7f3b0815d580 2023-10-08T00:21:38.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.236+0000 7f3b0c9a3700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b0815bfa0 con 0x7f3b08157870 2023-10-08T00:21:38.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.236+0000 7f3b0c9a3700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0815c280 con 0x7f3b0815a620 2023-10-08T00:21:38.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.236+0000 7f3b0659c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3b08157870 0x7f3b0815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:38.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.236+0000 7f3b0659c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3b08157870 0x7f3b0815b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52044/0 (socket says 172.21.15.99:52044) 2023-10-08T00:21:38.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.236+0000 7f3b06d9d700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f3b0815d580 0x7f3b08158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:57710/0 (socket says 172.21.15.99:57710) 2023-10-08T00:21:38.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.236+0000 7f3b0659c700 1 -- 172.21.15.99:0/822264561 learned_addr learned my addr 172.21.15.99:0/822264561 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:38.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.236+0000 7f3b05d9b700 1 --2- 172.21.15.99:0/822264561 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b0815a620 0x7f3b0815d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:38.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.236+0000 7f3b0659c700 1 -- 172.21.15.99:0/822264561 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b0815a620 msgr2=0x7f3b0815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:38.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.236+0000 7f3b0659c700 1 --2- 172.21.15.99:0/822264561 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b0815a620 0x7f3b0815d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:38.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.236+0000 7f3b0659c700 1 -- 172.21.15.99:0/822264561 >> v1:172.21.15.99:6789/0 conn(0x7f3b0815d580 legacy=0x7f3b08158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:38.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.236+0000 7f3b0659c700 1 -- 172.21.15.99:0/822264561 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b0815c560 con 0x7f3b08157870 2023-10-08T00:21:38.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.236+0000 7f3b0559a700 1 -- 172.21.15.99:0/822264561 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3772596863 0 0) 0x7f3b0815cbb0 con 0x7f3b0815d580 2023-10-08T00:21:38.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.236+0000 7f3b0659c700 1 --2- 172.21.15.99:0/822264561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b08157870 0x7f3b0815b6a0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f3b00009f20 tx=0x7f3b00009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2ca04c18ed82f9c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:38.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.237+0000 7f3b0559a700 1 -- 172.21.15.99:0/822264561 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0000e070 con 0x7f3b08157870 2023-10-08T00:21:38.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.237+0000 7f3b0559a700 1 -- 172.21.15.99:0/822264561 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b0000ba80 con 0x7f3b08157870 2023-10-08T00:21:38.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.237+0000 7f3b0559a700 1 -- 172.21.15.99:0/822264561 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b000092e0 con 0x7f3b08157870 2023-10-08T00:21:38.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.237+0000 7f3b0c9a3700 1 -- 172.21.15.99:0/822264561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b08157870 msgr2=0x7f3b0815b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:38.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.237+0000 7f3b0c9a3700 1 --2- 172.21.15.99:0/822264561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b08157870 0x7f3b0815b6a0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f3b00009f20 tx=0x7f3b00009840 comp rx=0 tx=0).stop 2023-10-08T00:21:38.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.237+0000 7f3b0c9a3700 1 -- 172.21.15.99:0/822264561 shutdown_connections 2023-10-08T00:21:38.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.237+0000 7f3b0c9a3700 1 --2- 172.21.15.99:0/822264561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b08157870 0x7f3b0815b6a0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:38.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.237+0000 7f3b0c9a3700 1 --2- 172.21.15.99:0/822264561 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b0815a620 0x7f3b0815d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:38.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.237+0000 7f3b0c9a3700 1 -- 172.21.15.99:0/822264561 >> 172.21.15.99:0/822264561 conn(0x7f3b08068ce0 msgr2=0x7f3b080690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:38.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.237+0000 7f3b0c9a3700 1 -- 172.21.15.99:0/822264561 shutdown_connections 2023-10-08T00:21:38.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.237+0000 7f3b0c9a3700 1 -- 172.21.15.99:0/822264561 wait complete. 2023-10-08T00:21:38.243 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.238+0000 7f3b0c9a3700 1 Processor -- start 2023-10-08T00:21:38.243 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.238+0000 7f3b0c9a3700 1 -- start start 2023-10-08T00:21:38.243 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.239+0000 7f3b0c9a3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b08157870 0x7f3b080b14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:38.243 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.239+0000 7f3b0c9a3700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b0815a620 0x7f3b08151970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:38.243 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.239+0000 7f3b0c9a3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b0815d580 0x7f3b0814c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:38.243 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.239+0000 7f3b0c9a3700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b08152d40 con 0x7f3b08157870 2023-10-08T00:21:38.243 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.239+0000 7f3b0c9a3700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b08151eb0 con 0x7f3b0815a620 2023-10-08T00:21:38.243 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.239+0000 7f3b0c9a3700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b081531d0 con 0x7f3b0815d580 2023-10-08T00:21:38.243 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.239+0000 7f3b0659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b08157870 0x7f3b080b14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:38.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.239+0000 7f3b0659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b08157870 0x7f3b080b14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52054/0 (socket says 172.21.15.99:52054) 2023-10-08T00:21:38.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.239+0000 7f3b0659c700 1 -- 172.21.15.99:0/2121519143 learned_addr learned my addr 172.21.15.99:0/2121519143 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:38.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.239+0000 7f3b06d9d700 1 --2- 172.21.15.99:0/2121519143 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b0815d580 0x7f3b0814c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:38.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.239+0000 7f3b05d9b700 1 --2- 172.21.15.99:0/2121519143 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b0815a620 0x7f3b08151970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:38.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.239+0000 7f3b0659c700 1 -- 172.21.15.99:0/2121519143 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b0815d580 msgr2=0x7f3b0814c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:38.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.239+0000 7f3b0659c700 1 --2- 172.21.15.99:0/2121519143 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b0815d580 0x7f3b0814c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:38.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.239+0000 7f3b0659c700 1 -- 172.21.15.99:0/2121519143 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b0815a620 msgr2=0x7f3b08151970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:38.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.239+0000 7f3b0659c700 1 --2- 172.21.15.99:0/2121519143 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b0815a620 0x7f3b08151970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:38.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.239+0000 7f3b0659c700 1 -- 172.21.15.99:0/2121519143 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b00009ae0 con 0x7f3b08157870 2023-10-08T00:21:38.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.240+0000 7f3b0659c700 1 --2- 172.21.15.99:0/2121519143 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b08157870 0x7f3b080b14f0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f3b0000b6c0 tx=0x7f3b00018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:38.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.240+0000 7f3afe7fc700 1 -- 172.21.15.99:0/2121519143 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b00008b70 con 0x7f3b08157870 2023-10-08T00:21:38.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.240+0000 7f3afe7fc700 1 -- 172.21.15.99:0/2121519143 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b00008cf0 con 0x7f3b08157870 2023-10-08T00:21:38.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.240+0000 7f3b0c9a3700 1 -- 172.21.15.99:0/2121519143 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b08152130 con 0x7f3b08157870 2023-10-08T00:21:38.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.240+0000 7f3afe7fc700 1 -- 172.21.15.99:0/2121519143 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b000266b0 con 0x7f3b08157870 2023-10-08T00:21:38.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.240+0000 7f3b0c9a3700 1 -- 172.21.15.99:0/2121519143 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b081526f0 con 0x7f3b08157870 2023-10-08T00:21:38.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.241+0000 7f3b0c9a3700 1 -- 172.21.15.99:0/2121519143 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b08153020 con 0x7f3b08157870 2023-10-08T00:21:38.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.241+0000 7f3afe7fc700 1 -- 172.21.15.99:0/2121519143 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3b00003040 con 0x7f3b08157870 2023-10-08T00:21:38.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.241+0000 7f3afe7fc700 1 --2- 172.21.15.99:0/2121519143 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3ae8066f30 0x7f3ae80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:38.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.241+0000 7f3afe7fc700 1 -- 172.21.15.99:0/2121519143 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3b00086480 con 0x7f3b08157870 2023-10-08T00:21:38.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.242+0000 7f3b05d9b700 1 --2- 172.21.15.99:0/2121519143 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3ae8066f30 0x7f3ae80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:38.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.242+0000 7f3b05d9b700 1 --2- 172.21.15.99:0/2121519143 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3ae8066f30 0x7f3ae80693e0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f3b080699f0 tx=0x7f3af0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:38.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.246+0000 7f3afe7fc700 1 -- 172.21.15.99:0/2121519143 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3b000537a0 con 0x7f3b08157870 2023-10-08T00:21:38.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.474+0000 7f3b0c9a3700 1 -- 172.21.15.99:0/2121519143 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3b0804c3c0 con 0x7f3b08157870 2023-10-08T00:21:38.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.475+0000 7f3afe7fc700 1 -- 172.21.15.99:0/2121519143 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+4604 (secure 0 0 0) 0x7f3b00056dc0 con 0x7f3b08157870 2023-10-08T00:21:38.476 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:21:38.476 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":84,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/3862684847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3862684847},{"type":"v1","addr":"172.21.15.191:6836","nonce":3862684847}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":8106,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/359623206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":359623206},{"type":"v1","addr":"172.21.15.99:6834","nonce":359623206}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":8109,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3064101777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":3064101777},{"type":"v1","addr":"172.21.15.99:6835","nonce":3064101777}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:21:29.659394+0000","modified":"2023-10-08T00:21:33.311802+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8124},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8124":{"gid":8124,"name":"d","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/2466421758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":2466421758},{"type":"v1","addr":"172.21.15.191:6837","nonce":2466421758}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-08T00:21:38.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.477+0000 7f3b0c9a3700 1 -- 172.21.15.99:0/2121519143 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3ae8066f30 msgr2=0x7f3ae80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:38.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.477+0000 7f3b0c9a3700 1 --2- 172.21.15.99:0/2121519143 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3ae8066f30 0x7f3ae80693e0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f3b080699f0 tx=0x7f3af0008040 comp rx=0 tx=0).stop 2023-10-08T00:21:38.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.477+0000 7f3b0c9a3700 1 -- 172.21.15.99:0/2121519143 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b08157870 msgr2=0x7f3b080b14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:38.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.477+0000 7f3b0c9a3700 1 --2- 172.21.15.99:0/2121519143 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b08157870 0x7f3b080b14f0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f3b0000b6c0 tx=0x7f3b00018000 comp rx=0 tx=0).stop 2023-10-08T00:21:38.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.477+0000 7f3b0c9a3700 1 -- 172.21.15.99:0/2121519143 shutdown_connections 2023-10-08T00:21:38.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.477+0000 7f3b0c9a3700 1 --2- 172.21.15.99:0/2121519143 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f3ae8066f30 0x7f3ae80693e0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:38.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.477+0000 7f3b0c9a3700 1 --2- 172.21.15.99:0/2121519143 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b0815d580 0x7f3b0814c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:38.479 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.477+0000 7f3b0c9a3700 1 --2- 172.21.15.99:0/2121519143 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b08157870 0x7f3b080b14f0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:38.479 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.477+0000 7f3b0c9a3700 1 --2- 172.21.15.99:0/2121519143 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b0815a620 0x7f3b08151970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:38.479 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.477+0000 7f3b0c9a3700 1 -- 172.21.15.99:0/2121519143 >> 172.21.15.99:0/2121519143 conn(0x7f3b08068ce0 msgr2=0x7f3b0815aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:38.479 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.477+0000 7f3b0c9a3700 1 -- 172.21.15.99:0/2121519143 shutdown_connections 2023-10-08T00:21:38.479 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.477+0000 7f3b0c9a3700 1 -- 172.21.15.99:0/2121519143 wait complete. 2023-10-08T00:21:38.479 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 84 2023-10-08T00:21:38.490 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json status 2023-10-08T00:21:38.622 INFO:teuthology.orchestra.run.smithi191.stdout:{"cluster_fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","whoami":0,"id":8124,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":8.3104147239999993,"mdsmap_epoch":84,"osdmap_epoch":184,"osdmap_epoch_barrier":184,"uptime":13.386054120000001} 2023-10-08T00:21:38.623 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", "fs_name": "cephfs", "id": 8124, "mdsmap_epoch": 84, "osdmap_epoch": 184, "osdmap_epoch_barrier": 184, "rank_uptime": 8.310414724, "state": "up:active", "uptime": 13.38605412, "want_state": "up:active", "whoami": 0 } 2023-10-08T00:21:38.623 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2023-10-08T00:21:38.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.741+0000 7f9c5dee4700 1 Processor -- start 2023-10-08T00:21:38.742 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.741+0000 7f9c5dee4700 1 -- start start 2023-10-08T00:21:38.744 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.742+0000 7f9c5dee4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c580b11f0 0x7f9c580b1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:38.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.742+0000 7f9c5dee4700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9c58068cb0 0x7f9c58069130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:38.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.742+0000 7f9c5dee4700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c580b0d60 con 0x7f9c58159e50 2023-10-08T00:21:38.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.742+0000 7f9c5dee4700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c58069670 con 0x7f9c58068cb0 2023-10-08T00:21:38.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.742+0000 7f9c5dee4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c580b3540 con 0x7f9c580b11f0 2023-10-08T00:21:38.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.742+0000 7f9c57fff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9c58068cb0 0x7f9c58069130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:38.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.742+0000 7f9c577fe700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f9c58159e50 0x7f9c580b2e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:57722/0 (socket says 172.21.15.99:57722) 2023-10-08T00:21:38.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.742+0000 7f9c577fe700 1 -- 172.21.15.99:0/2642204906 learned_addr learned my addr 172.21.15.99:0/2642204906 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:21:38.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.742+0000 7f9c56ffd700 1 --2- 172.21.15.99:0/2642204906 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c580b11f0 0x7f9c580b1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:38.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.742+0000 7f9c57fff700 1 -- 172.21.15.99:0/2642204906 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c580b11f0 msgr2=0x7f9c580b1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:38.745 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.742+0000 7f9c57fff700 1 --2- 172.21.15.99:0/2642204906 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c580b11f0 0x7f9c580b1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:38.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.742+0000 7f9c57fff700 1 -- 172.21.15.99:0/2642204906 >> v1:172.21.15.99:6789/0 conn(0x7f9c58159e50 legacy=0x7f9c580b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:38.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.742+0000 7f9c57fff700 1 -- 172.21.15.99:0/2642204906 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c580d15b0 con 0x7f9c58068cb0 2023-10-08T00:21:38.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.742+0000 7f9c567fc700 1 -- 172.21.15.99:0/2642204906 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 686545316 0 0) 0x7f9c580b0d60 con 0x7f9c58159e50 2023-10-08T00:21:38.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.743+0000 7f9c57fff700 1 --2- 172.21.15.99:0/2642204906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c58068cb0 0x7f9c58069130 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f9c50009e60 tx=0x7f9c50009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8837979b52a87b12 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:38.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.743+0000 7f9c567fc700 1 -- 172.21.15.99:0/2642204906 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5000e070 con 0x7f9c58068cb0 2023-10-08T00:21:38.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.743+0000 7f9c567fc700 1 -- 172.21.15.99:0/2642204906 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c5000bc30 con 0x7f9c58068cb0 2023-10-08T00:21:38.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.743+0000 7f9c567fc700 1 -- 172.21.15.99:0/2642204906 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c50012430 con 0x7f9c58068cb0 2023-10-08T00:21:38.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.743+0000 7f9c5dee4700 1 -- 172.21.15.99:0/2642204906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c58068cb0 msgr2=0x7f9c58069130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:38.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.743+0000 7f9c5dee4700 1 --2- 172.21.15.99:0/2642204906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c58068cb0 0x7f9c58069130 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f9c50009e60 tx=0x7f9c50009780 comp rx=0 tx=0).stop 2023-10-08T00:21:38.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.743+0000 7f9c5dee4700 1 -- 172.21.15.99:0/2642204906 shutdown_connections 2023-10-08T00:21:38.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.743+0000 7f9c5dee4700 1 --2- 172.21.15.99:0/2642204906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c58068cb0 0x7f9c58069130 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:38.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.743+0000 7f9c5dee4700 1 --2- 172.21.15.99:0/2642204906 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c580b11f0 0x7f9c580b1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:38.746 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.743+0000 7f9c5dee4700 1 -- 172.21.15.99:0/2642204906 >> 172.21.15.99:0/2642204906 conn(0x7f9c58068130 msgr2=0x7f9c580d0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:38.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.743+0000 7f9c5dee4700 1 -- 172.21.15.99:0/2642204906 shutdown_connections 2023-10-08T00:21:38.747 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.743+0000 7f9c5dee4700 1 -- 172.21.15.99:0/2642204906 wait complete. 2023-10-08T00:21:38.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.745+0000 7f9c5dee4700 1 Processor -- start 2023-10-08T00:21:38.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.745+0000 7f9c5dee4700 1 -- start start 2023-10-08T00:21:38.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.745+0000 7f9c5dee4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c58068cb0 0x7f9c580c71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:38.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.745+0000 7f9c5dee4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c580b11f0 0x7f9c580c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:38.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.745+0000 7f9c5dee4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c58159e50 0x7f9c580c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:38.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.745+0000 7f9c5dee4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c58156980 con 0x7f9c58159e50 2023-10-08T00:21:38.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.745+0000 7f9c5dee4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c581560c0 con 0x7f9c580b11f0 2023-10-08T00:21:38.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.745+0000 7f9c5dee4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c58156330 con 0x7f9c58068cb0 2023-10-08T00:21:38.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.745+0000 7f9c57fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c58159e50 0x7f9c580c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:38.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.745+0000 7f9c57fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c58159e50 0x7f9c580c82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52078/0 (socket says 172.21.15.99:52078) 2023-10-08T00:21:38.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.745+0000 7f9c57fff700 1 -- 172.21.15.99:0/2775313868 learned_addr learned my addr 172.21.15.99:0/2775313868 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:38.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.746+0000 7f9c577fe700 1 --2- 172.21.15.99:0/2775313868 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c58068cb0 0x7f9c580c71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:38.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.746+0000 7f9c56ffd700 1 --2- 172.21.15.99:0/2775313868 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c580b11f0 0x7f9c580c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:38.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.746+0000 7f9c57fff700 1 -- 172.21.15.99:0/2775313868 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c58068cb0 msgr2=0x7f9c580c71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:38.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.746+0000 7f9c57fff700 1 --2- 172.21.15.99:0/2775313868 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c58068cb0 0x7f9c580c71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:38.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.746+0000 7f9c57fff700 1 -- 172.21.15.99:0/2775313868 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c580b11f0 msgr2=0x7f9c580c7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:38.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.746+0000 7f9c57fff700 1 --2- 172.21.15.99:0/2775313868 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c580b11f0 0x7f9c580c7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:38.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.746+0000 7f9c57fff700 1 -- 172.21.15.99:0/2775313868 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c50009a20 con 0x7f9c58159e50 2023-10-08T00:21:38.754 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.746+0000 7f9c577fe700 1 --2- 172.21.15.99:0/2775313868 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c58068cb0 0x7f9c580c71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:21:38.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.746+0000 7f9c57fff700 1 --2- 172.21.15.99:0/2775313868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c58159e50 0x7f9c580c82c0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f9c50013000 tx=0x7f9c50019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:38.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.746+0000 7f9c3f7fe700 1 -- 172.21.15.99:0/2775313868 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c50012e50 con 0x7f9c58159e50 2023-10-08T00:21:38.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.746+0000 7f9c3f7fe700 1 -- 172.21.15.99:0/2775313868 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c5001e630 con 0x7f9c58159e50 2023-10-08T00:21:38.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.746+0000 7f9c5dee4700 1 -- 172.21.15.99:0/2775313868 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c58156610 con 0x7f9c58159e50 2023-10-08T00:21:38.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.746+0000 7f9c3f7fe700 1 -- 172.21.15.99:0/2775313868 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5001ea10 con 0x7f9c58159e50 2023-10-08T00:21:38.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.746+0000 7f9c5dee4700 1 -- 172.21.15.99:0/2775313868 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c580c76f0 con 0x7f9c58159e50 2023-10-08T00:21:38.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.747+0000 7f9c5dee4700 1 -- 172.21.15.99:0/2775313868 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c58156b30 con 0x7f9c58159e50 2023-10-08T00:21:38.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.747+0000 7f9c3f7fe700 1 -- 172.21.15.99:0/2775313868 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9c50024070 con 0x7f9c58159e50 2023-10-08T00:21:38.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.748+0000 7f9c3f7fe700 1 --2- 172.21.15.99:0/2775313868 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9c44066f30 0x7f9c440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:38.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.748+0000 7f9c3f7fe700 1 -- 172.21.15.99:0/2775313868 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9c50086d40 con 0x7f9c58159e50 2023-10-08T00:21:38.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.748+0000 7f9c577fe700 1 --2- 172.21.15.99:0/2775313868 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9c44066f30 0x7f9c440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:38.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.748+0000 7f9c577fe700 1 --2- 172.21.15.99:0/2775313868 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9c44066f30 0x7f9c440693e0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f9c480011e0 tx=0x7f9c48009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:38.756 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.752+0000 7f9c3f7fe700 1 -- 172.21.15.99:0/2775313868 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9c58156b30 con 0x7f9c58159e50 2023-10-08T00:21:38.935 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:38.935+0000 7f9c5dee4700 1 -- 172.21.15.99:0/2775313868 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7f9c5804c3c0 con 0x7f9c58159e50 2023-10-08T00:21:39.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.332+0000 7f9c3f7fe700 1 -- 172.21.15.99:0/2775313868 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v185) v1 ==== 181+0+0 (secure 0 0 0) 0x7f9c50053fb0 con 0x7f9c58159e50 2023-10-08T00:21:39.335 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.334+0000 7f9c5dee4700 1 -- 172.21.15.99:0/2775313868 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9c44066f30 msgr2=0x7f9c440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:39.335 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.334+0000 7f9c5dee4700 1 --2- 172.21.15.99:0/2775313868 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9c44066f30 0x7f9c440693e0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f9c480011e0 tx=0x7f9c48009040 comp rx=0 tx=0).stop 2023-10-08T00:21:39.335 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.334+0000 7f9c5dee4700 1 -- 172.21.15.99:0/2775313868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c58159e50 msgr2=0x7f9c580c82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:39.335 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.334+0000 7f9c5dee4700 1 --2- 172.21.15.99:0/2775313868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c58159e50 0x7f9c580c82c0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f9c50013000 tx=0x7f9c50019000 comp rx=0 tx=0).stop 2023-10-08T00:21:39.335 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.334+0000 7f9c5dee4700 1 -- 172.21.15.99:0/2775313868 shutdown_connections 2023-10-08T00:21:39.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.334+0000 7f9c5dee4700 1 --2- 172.21.15.99:0/2775313868 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9c44066f30 0x7f9c440693e0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:39.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.334+0000 7f9c5dee4700 1 --2- 172.21.15.99:0/2775313868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c58159e50 0x7f9c580c82c0 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:39.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.334+0000 7f9c5dee4700 1 --2- 172.21.15.99:0/2775313868 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c58068cb0 0x7f9c580c71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:39.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.334+0000 7f9c5dee4700 1 --2- 172.21.15.99:0/2775313868 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c580b11f0 0x7f9c580c7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:39.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.334+0000 7f9c5dee4700 1 -- 172.21.15.99:0/2775313868 >> 172.21.15.99:0/2775313868 conn(0x7f9c58068130 msgr2=0x7f9c580b4210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:39.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.334+0000 7f9c5dee4700 1 -- 172.21.15.99:0/2775313868 shutdown_connections 2023-10-08T00:21:39.336 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.334+0000 7f9c5dee4700 1 -- 172.21.15.99:0/2775313868 wait complete. 2023-10-08T00:21:39.336 INFO:teuthology.orchestra.run.smithi099.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2023-10-08T00:21:39.348 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-10-08T00:21:39.479 INFO:teuthology.orchestra.run.smithi099.stdout:{"osd_mon_report_interval":"5"} 2023-10-08T00:21:39.480 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-10-08T00:21:39.480 INFO:tasks.cephfs.test_full:Writing 32MB should fill this cluster 2023-10-08T00:21:39.480 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:21:39.480 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=16 seek=0) 2023-10-08T00:21:39.598 INFO:teuthology.orchestra.run.smithi099.stderr:16+0 records in 2023-10-08T00:21:39.598 INFO:teuthology.orchestra.run.smithi099.stderr:16+0 records out 2023-10-08T00:21:39.598 INFO:teuthology.orchestra.run.smithi099.stderr:16777216 bytes (17 MB, 16 MiB) copied, 0.100062 s, 168 MB/s 2023-10-08T00:21:39.599 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:21:39.600 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=17 seek=0) 2023-10-08T00:21:39.712 INFO:teuthology.orchestra.run.smithi099.stderr:17+0 records in 2023-10-08T00:21:39.712 INFO:teuthology.orchestra.run.smithi099.stderr:17+0 records out 2023-10-08T00:21:39.712 INFO:teuthology.orchestra.run.smithi099.stderr:17825792 bytes (18 MB, 17 MiB) copied, 0.0958692 s, 186 MB/s 2023-10-08T00:21:39.712 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2023-10-08T00:21:39.713 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-08T00:21:39.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.826+0000 7f549e249700 1 Processor -- start 2023-10-08T00:21:39.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.826+0000 7f549e249700 1 -- start start 2023-10-08T00:21:39.830 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.827+0000 7f549e249700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f549815cbd0 0x7f549815d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:39.830 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.827+0000 7f549e249700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54980cb7c0 con 0x7f54980cd6e0 2023-10-08T00:21:39.830 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.827+0000 7f549e249700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f549815e290 con 0x7f54980cdab0 2023-10-08T00:21:39.830 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.827+0000 7f549e249700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f549815e850 con 0x7f549815cbd0 2023-10-08T00:21:39.830 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.827+0000 7f549d247700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f54980cd6e0 0x7f54980cbc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:57734/0 (socket says 172.21.15.99:57734) 2023-10-08T00:21:39.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.827+0000 7f549d247700 1 -- 172.21.15.99:0/4077006893 learned_addr learned my addr 172.21.15.99:0/4077006893 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:21:39.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.827+0000 7f549da48700 1 --2- 172.21.15.99:0/4077006893 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f549815cbd0 0x7f549815d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:39.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.827+0000 7f548ffff700 1 -- 172.21.15.99:0/4077006893 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1720058876 0 0) 0x7f54980cb7c0 con 0x7f54980cd6e0 2023-10-08T00:21:39.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.827+0000 7f548ffff700 1 -- 172.21.15.99:0/4077006893 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5484003570 con 0x7f54980cd6e0 2023-10-08T00:21:39.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.828+0000 7f548ffff700 1 -- 172.21.15.99:0/4077006893 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1284521075 0 0) 0x7f549815e290 con 0x7f54980cdab0 2023-10-08T00:21:39.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.828+0000 7f548ffff700 1 -- 172.21.15.99:0/4077006893 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54980cb7c0 con 0x7f54980cdab0 2023-10-08T00:21:39.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.828+0000 7f548ffff700 1 -- 172.21.15.99:0/4077006893 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3194281037 0 0) 0x7f5484003570 con 0x7f54980cd6e0 2023-10-08T00:21:39.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.828+0000 7f548ffff700 1 -- 172.21.15.99:0/4077006893 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f549815cbd0 msgr2=0x7f549815d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:39.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.828+0000 7f548ffff700 1 --2- 172.21.15.99:0/4077006893 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f549815cbd0 0x7f549815d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:39.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.828+0000 7f548ffff700 1 -- 172.21.15.99:0/4077006893 >> v1:172.21.15.191:6789/0 conn(0x7f54980cdab0 legacy=0x7f549815d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:39.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.828+0000 7f548ffff700 1 -- 172.21.15.99:0/4077006893 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f549815e570 con 0x7f54980cd6e0 2023-10-08T00:21:39.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.828+0000 7f548ffff700 1 -- 172.21.15.99:0/4077006893 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f54940017f0 con 0x7f54980cd6e0 2023-10-08T00:21:39.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.828+0000 7f548ffff700 1 -- 172.21.15.99:0/4077006893 >> v1:172.21.15.99:6789/0 conn(0x7f54980cd6e0 legacy=0x7f54980cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:39.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.828+0000 7f548ffff700 1 --2- 172.21.15.99:0/4077006893 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5484003c20 0x7f5484004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:39.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.828+0000 7f548ffff700 1 --2- 172.21.15.99:0/4077006893 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5484004810 0x7f5484004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:39.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.828+0000 7f548ffff700 1 --2- 172.21.15.99:0/4077006893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54840051c0 0x7f548400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:39.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.828+0000 7f548ffff700 1 -- 172.21.15.99:0/4077006893 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549815e850 con 0x7f54840051c0 2023-10-08T00:21:39.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.828+0000 7f548ffff700 1 -- 172.21.15.99:0/4077006893 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548400c020 con 0x7f5484004810 2023-10-08T00:21:39.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.828+0000 7f548ffff700 1 -- 172.21.15.99:0/4077006893 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f548400c1e0 con 0x7f5484003c20 2023-10-08T00:21:39.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.828+0000 7f549da48700 1 --2- 172.21.15.99:0/4077006893 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f549815cbd0 0x7f549815d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:21:39.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.828+0000 7f549d247700 1 --2- 172.21.15.99:0/4077006893 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5484003c20 0x7f5484004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:39.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.828+0000 7f549ca46700 1 --2- 172.21.15.99:0/4077006893 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5484004810 0x7f5484004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:39.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.828+0000 7f549da48700 1 --2- 172.21.15.99:0/4077006893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54840051c0 0x7f548400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:39.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.828+0000 7f549da48700 1 -- 172.21.15.99:0/4077006893 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5484003c20 msgr2=0x7f5484004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:39.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.828+0000 7f549da48700 1 --2- 172.21.15.99:0/4077006893 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5484003c20 0x7f5484004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:39.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.828+0000 7f549da48700 1 -- 172.21.15.99:0/4077006893 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5484004810 msgr2=0x7f5484004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:39.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.828+0000 7f549da48700 1 --2- 172.21.15.99:0/4077006893 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5484004810 0x7f5484004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:39.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.828+0000 7f549da48700 1 -- 172.21.15.99:0/4077006893 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5484003570 con 0x7f54840051c0 2023-10-08T00:21:39.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.828+0000 7f549da48700 1 --2- 172.21.15.99:0/4077006893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54840051c0 0x7f548400ba60 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f5498089af0 tx=0x7f54900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=504fd74f92434475 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:39.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.828+0000 7f548ffff700 1 -- 172.21.15.99:0/4077006893 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549000e710 con 0x7f54840051c0 2023-10-08T00:21:39.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.828+0000 7f548ffff700 1 -- 172.21.15.99:0/4077006893 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f549000e870 con 0x7f54840051c0 2023-10-08T00:21:39.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.828+0000 7f548ffff700 1 -- 172.21.15.99:0/4077006893 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549000ec10 con 0x7f54840051c0 2023-10-08T00:21:39.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.829+0000 7f549e249700 1 -- 172.21.15.99:0/4077006893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54840051c0 msgr2=0x7f548400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:39.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.829+0000 7f549e249700 1 --2- 172.21.15.99:0/4077006893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54840051c0 0x7f548400ba60 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f5498089af0 tx=0x7f54900091a0 comp rx=0 tx=0).stop 2023-10-08T00:21:39.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.829+0000 7f549da48700 1 -- 172.21.15.99:0/4077006893 reap_dead start 2023-10-08T00:21:39.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.829+0000 7f549e249700 1 -- 172.21.15.99:0/4077006893 shutdown_connections 2023-10-08T00:21:39.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.829+0000 7f549e249700 1 -- 172.21.15.99:0/4077006893 >> 172.21.15.99:0/4077006893 conn(0x7f5498067660 msgr2=0x7f5498067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:39.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.829+0000 7f549e249700 1 -- 172.21.15.99:0/4077006893 shutdown_connections 2023-10-08T00:21:39.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.829+0000 7f549e249700 1 -- 172.21.15.99:0/4077006893 wait complete. 2023-10-08T00:21:39.836 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.830+0000 7f549e249700 1 Processor -- start 2023-10-08T00:21:39.836 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.830+0000 7f549e249700 1 -- start start 2023-10-08T00:21:39.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.831+0000 7f549e249700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f54840051c0 0x7f54980b6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:39.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.831+0000 7f549e249700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f549814d640 0x7f549806a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:39.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.831+0000 7f549e249700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f549814dab0 0x7f549806ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:39.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.831+0000 7f549e249700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54980cc360 con 0x7f549814d640 2023-10-08T00:21:39.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.831+0000 7f549e249700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5498152df0 con 0x7f549814dab0 2023-10-08T00:21:39.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.831+0000 7f549e249700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54981530d0 con 0x7f54840051c0 2023-10-08T00:21:39.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.831+0000 7f549ca46700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f549814d640 0x7f549806a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:39.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.831+0000 7f549ca46700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f549814d640 0x7f549806a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52098/0 (socket says 172.21.15.99:52098) 2023-10-08T00:21:39.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.831+0000 7f549ca46700 1 -- 172.21.15.99:0/1379879643 learned_addr learned my addr 172.21.15.99:0/1379879643 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:39.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.831+0000 7f549d247700 1 --2- 172.21.15.99:0/1379879643 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f54840051c0 0x7f54980b6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:39.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.831+0000 7f549da48700 1 --2- 172.21.15.99:0/1379879643 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f549814dab0 0x7f549806ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:39.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.831+0000 7f549ca46700 1 -- 172.21.15.99:0/1379879643 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f54840051c0 msgr2=0x7f54980b6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:39.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.831+0000 7f549ca46700 1 --2- 172.21.15.99:0/1379879643 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f54840051c0 0x7f54980b6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:39.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.831+0000 7f549ca46700 1 -- 172.21.15.99:0/1379879643 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f549814dab0 msgr2=0x7f549806ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:39.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.831+0000 7f549ca46700 1 --2- 172.21.15.99:0/1379879643 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f549814dab0 0x7f549806ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:39.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.831+0000 7f549ca46700 1 -- 172.21.15.99:0/1379879643 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54981533b0 con 0x7f549814d640 2023-10-08T00:21:39.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.832+0000 7f549ca46700 1 --2- 172.21.15.99:0/1379879643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f549814d640 0x7f549806a630 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f548800f880 tx=0x7f5488007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:39.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.832+0000 7f548d7fa700 1 -- 172.21.15.99:0/1379879643 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5488013600 con 0x7f549814d640 2023-10-08T00:21:39.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.832+0000 7f548d7fa700 1 -- 172.21.15.99:0/1379879643 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5488013760 con 0x7f549814d640 2023-10-08T00:21:39.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.832+0000 7f549e249700 1 -- 172.21.15.99:0/1379879643 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5498153690 con 0x7f549814d640 2023-10-08T00:21:39.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.832+0000 7f548d7fa700 1 -- 172.21.15.99:0/1379879643 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5488013b20 con 0x7f549814d640 2023-10-08T00:21:39.838 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.832+0000 7f549e249700 1 -- 172.21.15.99:0/1379879643 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5498153e90 con 0x7f549814d640 2023-10-08T00:21:39.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.833+0000 7f549e249700 1 -- 172.21.15.99:0/1379879643 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5498089280 con 0x7f549814d640 2023-10-08T00:21:39.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.833+0000 7f548d7fa700 1 -- 172.21.15.99:0/1379879643 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f548801c400 con 0x7f549814d640 2023-10-08T00:21:39.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.833+0000 7f548d7fa700 1 --2- 172.21.15.99:0/1379879643 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f547c066f30 0x7f547c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:39.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.833+0000 7f548d7fa700 1 -- 172.21.15.99:0/1379879643 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5488084740 con 0x7f549814d640 2023-10-08T00:21:39.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.834+0000 7f549d247700 1 --2- 172.21.15.99:0/1379879643 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f547c066f30 0x7f547c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:39.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.834+0000 7f549d247700 1 --2- 172.21.15.99:0/1379879643 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f547c066f30 0x7f547c0693e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f5494001920 tx=0x7f5494002190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:39.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:39.837+0000 7f548d7fa700 1 -- 172.21.15.99:0/1379879643 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f54880519b0 con 0x7f549814d640 2023-10-08T00:21:40.021 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:40.021+0000 7f549e249700 1 -- 172.21.15.99:0/1379879643 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f5498157ec0 con 0x7f549814d640 2023-10-08T00:21:40.022 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:40.022+0000 7f548d7fa700 1 -- 172.21.15.99:0/1379879643 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v185) v1 ==== 81+0+32282 (secure 0 0 0) 0x7f5488054fd0 con 0x7f549814d640 2023-10-08T00:21:40.023 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:21:40.023 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:21:40.023 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 185, 2023-10-08T00:21:40.023 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", 2023-10-08T00:21:40.023 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2023-10-08T00:10:36.303601+0000", 2023-10-08T00:21:40.023 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2023-10-08T00:21:39.327067+0000", 2023-10-08T00:21:40.023 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2023-10-08T00:10:41.709171+0000", 2023-10-08T00:21:40.023 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2023-10-08T00:10:38.690404+0000", 2023-10-08T00:21:40.024 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-08T00:21:40.024 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2023-10-08T00:21:40.024 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2023-10-08T00:21:40.024 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2023-10-08T00:21:40.024 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2023-10-08T00:21:40.024 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2023-10-08T00:21:40.024 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2023-10-08T00:21:40.024 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:21:40.024 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2023-10-08T00:21:40.024 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2023-10-08T00:21:40.024 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-08T00:21:40.025 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-08T00:21:40.025 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2023-10-08T00:21:40.025 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 23, 2023-10-08T00:21:40.025 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2023-10-08T00:21:40.025 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2023-10-08T00:21:40.025 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "jewel", 2023-10-08T00:21:40.025 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "quincy", 2023-10-08T00:21:40.025 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2023-10-08T00:21:40.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.026 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2023-10-08T00:21:40.026 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2023-10-08T00:21:40.026 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:10:42.063344+0000", 2023-10-08T00:21:40.026 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:21:40.026 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:21:40.026 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:21:40.026 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:21:40.026 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:21:40.026 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:21:40.026 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:21:40.027 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:21:40.027 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:21:40.027 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:21:40.027 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:21:40.027 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:21:40.027 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2023-10-08T00:21:40.027 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2023-10-08T00:21:40.027 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2023-10-08T00:21:40.027 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2023-10-08T00:21:40.027 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2023-10-08T00:21:40.027 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:21:40.027 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:21:40.027 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:21:40.028 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:21:40.028 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:21:40.028 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:21:40.028 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:21:40.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.028 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "112", 2023-10-08T00:21:40.028 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:21:40.028 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:21:40.028 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:21:40.028 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:21:40.028 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:21:40.029 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:21:40.029 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:21:40.029 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:21:40.029 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:21:40.029 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:21:40.029 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:21:40.029 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:21:40.030 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:21:40.030 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:21:40.030 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:21:40.030 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:21:40.030 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:21:40.030 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:21:40.031 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:21:40.031 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:21:40.031 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:21:40.031 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:21:40.031 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:21:40.031 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:21:40.031 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:21:40.031 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:21:40.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.031 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:21:40.031 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:21:40.031 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:21:40.032 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:21:40.032 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:21:40.032 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:21:40.032 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:21:40.032 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:21:40.032 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:21:40.032 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:21:40.032 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:21:40.032 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:21:40.032 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2023-10-08T00:21:40.032 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2023-10-08T00:21:40.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.033 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:21:40.033 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2023-10-08T00:21:40.033 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:40.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.033 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 22, 2023-10-08T00:21:40.033 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2023-10-08T00:21:40.033 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:21:27.984574+0000", 2023-10-08T00:21:40.033 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:21:40.034 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:21:40.034 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:21:40.034 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:21:40.034 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:21:40.034 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:21:40.034 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:21:40.034 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:21:40.034 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:21:40.034 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:21:40.035 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:21:40.035 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:21:40.035 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2023-10-08T00:21:40.035 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2023-10-08T00:21:40.035 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2023-10-08T00:21:40.035 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2023-10-08T00:21:40.035 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2023-10-08T00:21:40.035 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:21:40.035 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:21:40.035 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:21:40.035 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:21:40.036 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:21:40.036 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:21:40.036 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:21:40.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.036 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "183", 2023-10-08T00:21:40.036 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:21:40.036 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:21:40.036 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:21:40.036 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:21:40.036 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:21:40.037 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:21:40.037 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:21:40.037 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:21:40.037 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:21:40.037 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:21:40.037 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:21:40.037 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:21:40.037 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:21:40.038 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:21:40.038 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:21:40.038 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:21:40.038 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:21:40.038 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:21:40.038 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:21:40.038 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:21:40.038 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:21:40.038 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:21:40.038 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:21:40.038 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:21:40.038 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:21:40.039 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:21:40.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.039 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:21:40.039 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:21:40.039 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:21:40.039 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:21:40.039 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:21:40.039 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:21:40.039 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:21:40.039 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:21:40.039 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:21:40.039 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:21:40.040 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:21:40.040 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:21:40.040 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2023-10-08T00:21:40.040 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2023-10-08T00:21:40.040 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2023-10-08T00:21:40.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.040 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:21:40.040 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:21:40.040 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2023-10-08T00:21:40.040 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:40.040 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:40.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.041 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 23, 2023-10-08T00:21:40.041 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2023-10-08T00:21:40.041 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:21:29.244620+0000", 2023-10-08T00:21:40.041 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:21:40.041 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:21:40.041 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:21:40.041 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:21:40.041 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:21:40.041 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:21:40.041 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:21:40.042 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:21:40.042 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:21:40.042 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:21:40.042 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:21:40.042 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:21:40.042 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2023-10-08T00:21:40.042 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2023-10-08T00:21:40.042 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2023-10-08T00:21:40.042 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2023-10-08T00:21:40.042 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2023-10-08T00:21:40.042 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:21:40.043 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:21:40.043 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:21:40.043 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:21:40.043 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:21:40.043 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:21:40.043 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:21:40.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.043 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "185", 2023-10-08T00:21:40.043 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:21:40.043 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:21:40.043 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:21:40.044 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:21:40.044 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:21:40.044 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:21:40.044 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:21:40.044 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:21:40.044 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:21:40.044 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 33554432, 2023-10-08T00:21:40.044 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:21:40.044 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:21:40.044 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:21:40.045 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:21:40.045 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:21:40.045 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:21:40.045 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:21:40.045 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:21:40.045 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:21:40.045 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:21:40.045 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:21:40.045 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:21:40.046 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:21:40.046 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:21:40.046 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:21:40.046 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:21:40.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.046 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:21:40.046 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:21:40.046 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:21:40.046 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:21:40.046 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:21:40.047 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:21:40.047 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:21:40.047 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:21:40.047 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:21:40.047 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:21:40.047 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:21:40.047 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:21:40.047 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2023-10-08T00:21:40.047 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2023-10-08T00:21:40.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.048 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:21:40.048 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:21:40.048 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2023-10-08T00:21:40.048 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:40.048 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:40.048 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:40.048 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:21:40.048 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2023-10-08T00:21:40.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.048 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:21:40.048 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:21:40.048 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:21:40.049 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:21:40.049 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:21:40.049 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:21:40.049 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:21:40.049 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:21:40.049 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:21:40.049 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:21:40.049 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:21:40.049 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:21:40.049 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:21:40.049 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:40.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.050 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:40.050 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2023-10-08T00:21:40.050 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:21:40.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.050 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:40.050 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2023-10-08T00:21:40.050 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:21:40.050 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:40.050 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:40.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.051 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:21:40.051 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:40.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.051 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:40.051 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2023-10-08T00:21:40.051 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:21:40.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.051 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:40.051 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2023-10-08T00:21:40.051 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:21:40.052 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:40.052 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:40.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.052 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:21:40.052 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:40.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.052 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:40.052 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2023-10-08T00:21:40.052 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:21:40.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.052 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:40.053 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2023-10-08T00:21:40.053 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:21:40.053 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:40.053 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:40.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.053 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:21:40.053 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:40.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.053 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:40.053 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2023-10-08T00:21:40.053 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:21:40.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.054 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:40.054 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2023-10-08T00:21:40.054 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:21:40.054 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:40.054 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:40.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.054 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/86365", 2023-10-08T00:21:40.054 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/86365", 2023-10-08T00:21:40.054 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/86365", 2023-10-08T00:21:40.054 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/86365", 2023-10-08T00:21:40.055 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:21:40.055 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:21:40.055 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:21:40.055 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:40.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.055 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:21:40.055 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:21:40.055 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:21:40.056 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:21:40.056 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:21:40.056 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:21:40.056 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:21:40.056 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:21:40.056 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:21:40.056 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:21:40.056 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:21:40.056 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:21:40.056 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:21:40.056 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:40.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.057 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:40.057 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2023-10-08T00:21:40.057 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:21:40.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.057 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:40.057 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2023-10-08T00:21:40.057 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:21:40.057 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:40.057 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:40.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.057 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:21:40.058 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:40.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.058 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:40.058 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2023-10-08T00:21:40.058 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:21:40.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.058 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:40.058 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2023-10-08T00:21:40.058 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:21:40.058 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:40.059 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:40.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.059 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:21:40.059 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:40.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.059 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:40.059 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2023-10-08T00:21:40.059 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:21:40.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.060 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:40.060 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2023-10-08T00:21:40.060 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:21:40.060 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:40.060 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:40.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.060 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:21:40.060 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:40.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.060 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:40.060 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2023-10-08T00:21:40.061 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:21:40.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.061 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:40.061 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2023-10-08T00:21:40.061 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:21:40.061 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:40.061 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:40.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.061 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/86366", 2023-10-08T00:21:40.061 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/86366", 2023-10-08T00:21:40.061 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/86366", 2023-10-08T00:21:40.062 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/86366", 2023-10-08T00:21:40.062 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:21:40.062 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:21:40.062 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:21:40.062 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:40.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.062 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:21:40.062 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:21:40.062 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:21:40.062 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:21:40.062 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:21:40.063 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:21:40.063 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:21:40.063 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:21:40.063 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:21:40.063 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:21:40.063 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:21:40.063 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:21:40.063 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:21:40.063 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:40.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.063 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:40.063 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2023-10-08T00:21:40.063 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:21:40.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.064 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:40.064 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2023-10-08T00:21:40.064 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:21:40.064 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:40.064 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:40.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.064 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:21:40.064 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:40.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.064 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:40.065 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2023-10-08T00:21:40.065 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:21:40.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.065 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:40.065 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2023-10-08T00:21:40.065 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:21:40.065 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:40.065 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:40.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.065 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:21:40.065 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:40.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.066 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:40.066 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2023-10-08T00:21:40.066 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:21:40.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.066 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:40.066 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2023-10-08T00:21:40.066 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:21:40.066 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:40.066 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:40.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.067 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:21:40.067 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:40.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.067 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:40.067 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2023-10-08T00:21:40.067 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:21:40.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.067 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:40.068 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2023-10-08T00:21:40.068 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:21:40.068 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:40.068 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:40.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.068 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6817/86382", 2023-10-08T00:21:40.068 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/86382", 2023-10-08T00:21:40.068 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/86382", 2023-10-08T00:21:40.068 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/86382", 2023-10-08T00:21:40.068 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:21:40.069 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:21:40.069 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:21:40.069 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:40.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.069 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:21:40.069 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:21:40.069 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:21:40.070 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:21:40.070 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:21:40.070 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:21:40.070 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:21:40.070 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:21:40.070 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:21:40.070 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:21:40.070 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:21:40.070 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:21:40.070 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:21:40.070 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:40.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.071 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:40.071 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2023-10-08T00:21:40.071 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:21:40.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.071 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:40.071 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2023-10-08T00:21:40.071 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:21:40.071 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:40.071 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:40.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.072 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:21:40.072 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:40.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.072 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:40.072 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2023-10-08T00:21:40.072 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:21:40.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.072 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:40.072 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2023-10-08T00:21:40.073 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:21:40.073 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:40.073 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:40.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.073 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:21:40.073 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:40.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.073 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:40.073 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2023-10-08T00:21:40.073 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:21:40.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.074 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:40.074 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2023-10-08T00:21:40.074 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:21:40.074 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:40.074 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:40.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.074 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:21:40.074 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:40.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.074 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:40.074 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2023-10-08T00:21:40.075 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:21:40.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.075 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:40.075 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2023-10-08T00:21:40.075 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:21:40.075 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:40.075 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:40.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.075 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6820/86393", 2023-10-08T00:21:40.076 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/86393", 2023-10-08T00:21:40.076 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/86393", 2023-10-08T00:21:40.076 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/86393", 2023-10-08T00:21:40.076 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:21:40.076 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:21:40.076 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:21:40.076 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:40.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.076 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:21:40.076 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:21:40.076 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:21:40.076 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:21:40.077 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:21:40.077 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:21:40.077 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:21:40.077 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:21:40.077 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:21:40.077 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:21:40.077 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:21:40.077 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:21:40.077 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:21:40.077 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:40.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.077 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:40.078 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6800", 2023-10-08T00:21:40.078 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:21:40.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.078 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:40.078 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6801", 2023-10-08T00:21:40.078 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:21:40.078 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:40.078 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:40.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.078 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:21:40.079 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:40.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.079 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:40.079 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6802", 2023-10-08T00:21:40.079 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:21:40.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.079 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:40.079 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6803", 2023-10-08T00:21:40.079 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:21:40.079 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:40.079 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:40.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.080 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:21:40.080 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:40.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.080 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:40.080 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6806", 2023-10-08T00:21:40.080 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:21:40.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.080 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:40.081 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6807", 2023-10-08T00:21:40.081 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:21:40.081 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:40.081 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:40.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.081 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:21:40.081 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:40.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.081 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:40.081 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6804", 2023-10-08T00:21:40.081 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:21:40.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.082 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:40.082 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6805", 2023-10-08T00:21:40.082 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:21:40.082 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:40.082 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:40.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.082 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6801/86570", 2023-10-08T00:21:40.082 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6803/86570", 2023-10-08T00:21:40.082 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6807/86570", 2023-10-08T00:21:40.083 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6805/86570", 2023-10-08T00:21:40.083 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:21:40.083 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:21:40.083 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:21:40.083 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:40.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.083 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:21:40.083 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:21:40.083 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:21:40.083 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:21:40.083 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:21:40.084 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:21:40.084 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:21:40.084 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:21:40.084 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:21:40.084 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:21:40.084 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:21:40.084 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:21:40.084 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:21:40.084 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:40.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.084 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:40.084 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6816", 2023-10-08T00:21:40.085 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:21:40.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.085 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:40.085 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6818", 2023-10-08T00:21:40.085 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:21:40.085 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:40.085 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:40.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.085 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:21:40.085 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:40.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.086 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:40.086 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6820", 2023-10-08T00:21:40.086 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:21:40.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.086 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:40.086 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6821", 2023-10-08T00:21:40.086 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:21:40.086 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:40.086 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:40.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.087 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:21:40.087 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:40.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.087 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:40.087 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6826", 2023-10-08T00:21:40.087 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:21:40.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.087 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:40.087 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6828", 2023-10-08T00:21:40.087 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:21:40.087 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:40.088 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:40.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.088 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:21:40.088 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:40.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.088 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:40.088 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6823", 2023-10-08T00:21:40.088 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:21:40.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.088 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:40.088 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6825", 2023-10-08T00:21:40.089 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:21:40.089 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:40.089 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:40.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.089 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6818/86608", 2023-10-08T00:21:40.089 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6821/86608", 2023-10-08T00:21:40.089 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6828/86608", 2023-10-08T00:21:40.089 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6825/86608", 2023-10-08T00:21:40.089 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:21:40.089 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:21:40.089 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:21:40.090 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:40.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.090 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:21:40.090 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:21:40.090 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:21:40.090 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:21:40.090 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:21:40.090 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:21:40.090 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:21:40.090 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:21:40.090 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:21:40.091 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:21:40.091 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:21:40.091 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:21:40.091 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:21:40.091 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:40.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.091 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:40.091 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6808", 2023-10-08T00:21:40.091 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:21:40.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.092 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:40.092 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6809", 2023-10-08T00:21:40.092 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:21:40.092 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:40.092 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:40.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.092 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:21:40.092 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:40.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.092 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:40.092 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6810", 2023-10-08T00:21:40.092 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:21:40.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.093 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:40.093 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6811", 2023-10-08T00:21:40.093 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:21:40.093 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:40.093 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:40.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.093 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:21:40.093 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:40.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.093 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:40.094 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6814", 2023-10-08T00:21:40.094 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:21:40.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.094 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:40.094 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6815", 2023-10-08T00:21:40.094 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:21:40.094 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:40.094 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:40.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.094 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:21:40.094 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:40.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.095 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:40.095 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6812", 2023-10-08T00:21:40.095 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:21:40.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.095 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:40.095 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6813", 2023-10-08T00:21:40.095 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:21:40.095 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:40.095 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:40.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.096 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6809/86571", 2023-10-08T00:21:40.096 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6811/86571", 2023-10-08T00:21:40.096 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6815/86571", 2023-10-08T00:21:40.096 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6813/86571", 2023-10-08T00:21:40.096 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:21:40.096 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:21:40.096 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:21:40.096 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:40.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.096 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:21:40.097 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:21:40.097 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:21:40.097 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:21:40.098 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:21:40.098 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:21:40.098 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:21:40.099 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:21:40.099 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:21:40.099 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:21:40.099 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:21:40.099 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:21:40.099 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:21:40.099 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:40.099 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.099 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:40.099 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6817", 2023-10-08T00:21:40.099 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:21:40.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.100 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:40.100 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6819", 2023-10-08T00:21:40.100 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:21:40.100 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:40.100 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:40.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.100 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:21:40.100 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:40.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.101 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:40.101 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6822", 2023-10-08T00:21:40.101 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:21:40.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.101 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:40.101 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6824", 2023-10-08T00:21:40.101 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:21:40.101 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:40.101 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:40.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.101 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:21:40.102 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:40.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.102 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:40.102 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6830", 2023-10-08T00:21:40.102 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:21:40.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.102 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:40.102 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6831", 2023-10-08T00:21:40.102 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:21:40.102 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:40.103 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:40.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.103 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:21:40.103 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:40.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.103 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:40.103 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6827", 2023-10-08T00:21:40.103 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:21:40.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.104 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.104 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:40.104 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6829", 2023-10-08T00:21:40.104 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:21:40.104 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:40.104 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:40.104 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.104 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6819/86612", 2023-10-08T00:21:40.104 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6824/86612", 2023-10-08T00:21:40.105 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6831/86612", 2023-10-08T00:21:40.105 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6829/86612", 2023-10-08T00:21:40.105 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:21:40.105 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:21:40.105 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:21:40.105 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:40.105 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:40.105 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:21:40.105 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2023-10-08T00:21:40.105 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.106 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:21:40.106 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:21:40.106 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:21:40.106 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:21:40.106 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:21:40.106 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:21:40.106 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.905615+0000", 2023-10-08T00:21:40.106 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:21:40.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.107 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:21:40.107 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:21:40.107 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:21:40.107 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:21:40.107 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:21:40.107 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:21:40.107 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.880709+0000", 2023-10-08T00:21:40.107 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:21:40.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.108 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:21:40.108 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:21:40.108 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:21:40.108 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:21:40.108 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:21:40.108 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:21:40.108 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.874704+0000", 2023-10-08T00:21:40.108 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:21:40.109 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.109 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.109 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:21:40.109 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:21:40.109 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:21:40.109 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:21:40.109 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:21:40.109 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:21:40.109 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.899604+0000", 2023-10-08T00:21:40.109 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:21:40.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.110 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:21:40.110 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:21:40.110 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:21:40.110 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:21:40.110 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:21:40.110 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:21:40.110 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.844371+0000", 2023-10-08T00:21:40.110 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:21:40.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.111 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:21:40.111 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:21:40.111 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:21:40.111 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:21:40.111 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:21:40.111 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:21:40.111 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.794810+0000", 2023-10-08T00:21:40.111 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:21:40.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.112 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:21:40.112 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:21:40.112 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:21:40.112 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:21:40.112 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:21:40.112 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:21:40.112 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.859474+0000", 2023-10-08T00:21:40.112 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:21:40.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.113 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:40.113 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:21:40.113 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:21:40.113 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:21:40.113 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:21:40.113 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:21:40.113 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:21:40.113 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.893130+0000", 2023-10-08T00:21:40.113 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:21:40.113 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:40.113 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:21:40.113 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2023-10-08T00:21:40.114 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [], 2023-10-08T00:21:40.114 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2023-10-08T00:21:40.114 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2023-10-08T00:21:40.114 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2023-10-08T00:21:40.114 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2023-10-08T00:21:40.114 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2023-10-08T00:21:40.114 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2023-10-08T00:21:40.114 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2023-10-08T00:21:40.114 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2023-10-08T00:21:40.114 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2023-10-08T00:21:40.114 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2023-10-08T00:21:40.115 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2023-10-08T00:21:40.115 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:40.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:40.115 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2023-10-08T00:21:40.115 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2023-10-08T00:21:40.115 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2023-10-08T00:21:40.115 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2023-10-08T00:21:40.115 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2023-10-08T00:21:40.115 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2023-10-08T00:21:40.115 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2023-10-08T00:21:40.115 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2023-10-08T00:21:40.116 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2023-10-08T00:21:40.116 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2023-10-08T00:21:40.116 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2023-10-08T00:21:40.116 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:40.116 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:21:40.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:40.024+0000 7f549e249700 1 -- 172.21.15.99:0/1379879643 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f547c066f30 msgr2=0x7f547c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:40.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:40.024+0000 7f549e249700 1 --2- 172.21.15.99:0/1379879643 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f547c066f30 0x7f547c0693e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f5494001920 tx=0x7f5494002190 comp rx=0 tx=0).stop 2023-10-08T00:21:40.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:40.024+0000 7f549e249700 1 -- 172.21.15.99:0/1379879643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f549814d640 msgr2=0x7f549806a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:40.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:40.024+0000 7f549e249700 1 --2- 172.21.15.99:0/1379879643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f549814d640 0x7f549806a630 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f548800f880 tx=0x7f5488007d10 comp rx=0 tx=0).stop 2023-10-08T00:21:40.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:40.024+0000 7f549e249700 1 -- 172.21.15.99:0/1379879643 shutdown_connections 2023-10-08T00:21:40.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:40.024+0000 7f549e249700 1 --2- 172.21.15.99:0/1379879643 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f547c066f30 0x7f547c0693e0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:40.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:40.024+0000 7f549e249700 1 --2- 172.21.15.99:0/1379879643 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f549814dab0 0x7f549806ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:40.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:40.024+0000 7f549e249700 1 --2- 172.21.15.99:0/1379879643 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f54840051c0 0x7f54980b6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:40.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:40.024+0000 7f549e249700 1 --2- 172.21.15.99:0/1379879643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f549814d640 0x7f549806a630 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:40.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:40.024+0000 7f549e249700 1 -- 172.21.15.99:0/1379879643 >> 172.21.15.99:0/1379879643 conn(0x7f5498067660 msgr2=0x7f54980b09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:40.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:40.024+0000 7f549e249700 1 -- 172.21.15.99:0/1379879643 shutdown_connections 2023-10-08T00:21:40.119 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:40.024+0000 7f549e249700 1 -- 172.21.15.99:0/1379879643 wait complete. 2023-10-08T00:21:40.119 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-08T00:21:45.120 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-08T00:21:45.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.246+0000 7f2c2a8ca700 1 Processor -- start 2023-10-08T00:21:45.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.246+0000 7f2c2a8ca700 1 -- start start 2023-10-08T00:21:45.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.247+0000 7f2c2a8ca700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c24157c40 0x7f2c24158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:45.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.247+0000 7f2c2a8ca700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c2415cbb0 con 0x7f2c24157870 2023-10-08T00:21:45.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.247+0000 7f2c2a8ca700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c2415bfa0 con 0x7f2c2415a620 2023-10-08T00:21:45.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.247+0000 7f2c2a8ca700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2415bdb0 con 0x7f2c24157c40 2023-10-08T00:21:45.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.247+0000 7f2c23fff700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f2c24157870 0x7f2c2415b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59970/0 (socket says 172.21.15.99:59970) 2023-10-08T00:21:45.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.247+0000 7f2c23fff700 1 -- 172.21.15.99:0/1351597335 learned_addr learned my addr 172.21.15.99:0/1351597335 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:21:45.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.248+0000 7f2c28e67700 1 --2- 172.21.15.99:0/1351597335 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c24157c40 0x7f2c24158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:45.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.248+0000 7f2c22ffd700 1 -- 172.21.15.99:0/1351597335 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2276744946 0 0) 0x7f2c2415cbb0 con 0x7f2c24157870 2023-10-08T00:21:45.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.248+0000 7f2c22ffd700 1 -- 172.21.15.99:0/1351597335 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c0c003570 con 0x7f2c24157870 2023-10-08T00:21:45.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.248+0000 7f2c22ffd700 1 -- 172.21.15.99:0/1351597335 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 82887631 0 0) 0x7f2c2415bfa0 con 0x7f2c2415a620 2023-10-08T00:21:45.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.248+0000 7f2c22ffd700 1 -- 172.21.15.99:0/1351597335 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c2415cbb0 con 0x7f2c2415a620 2023-10-08T00:21:45.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.248+0000 7f2c22ffd700 1 -- 172.21.15.99:0/1351597335 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 259528781 0 0) 0x7f2c0c003570 con 0x7f2c24157870 2023-10-08T00:21:45.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.248+0000 7f2c22ffd700 1 -- 172.21.15.99:0/1351597335 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c24157c40 msgr2=0x7f2c24158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:45.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.248+0000 7f2c22ffd700 1 --2- 172.21.15.99:0/1351597335 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c24157c40 0x7f2c24158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:45.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.248+0000 7f2c22ffd700 1 -- 172.21.15.99:0/1351597335 >> v1:172.21.15.191:6789/0 conn(0x7f2c2415a620 legacy=0x7f2c2415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:45.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.248+0000 7f2c22ffd700 1 -- 172.21.15.99:0/1351597335 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c2415c280 con 0x7f2c24157870 2023-10-08T00:21:45.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.248+0000 7f2c22ffd700 1 -- 172.21.15.99:0/1351597335 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f2c14001470 con 0x7f2c24157870 2023-10-08T00:21:45.252 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.248+0000 7f2c22ffd700 1 -- 172.21.15.99:0/1351597335 >> v1:172.21.15.99:6789/0 conn(0x7f2c24157870 legacy=0x7f2c2415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:45.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.248+0000 7f2c22ffd700 1 --2- 172.21.15.99:0/1351597335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c0c003c20 0x7f2c0c004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:45.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.248+0000 7f2c22ffd700 1 --2- 172.21.15.99:0/1351597335 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c24157c40 0x7f2c0c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:45.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.248+0000 7f2c22ffd700 1 --2- 172.21.15.99:0/1351597335 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c0c004d50 0x7f2c0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:45.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.248+0000 7f2c22ffd700 1 -- 172.21.15.99:0/1351597335 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0c00bbf0 con 0x7f2c0c003c20 2023-10-08T00:21:45.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.248+0000 7f2c22ffd700 1 -- 172.21.15.99:0/1351597335 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0c00bd30 con 0x7f2c0c004d50 2023-10-08T00:21:45.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.248+0000 7f2c22ffd700 1 -- 172.21.15.99:0/1351597335 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0c00bea0 con 0x7f2c24157c40 2023-10-08T00:21:45.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.248+0000 7f2c23fff700 1 --2- 172.21.15.99:0/1351597335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c0c003c20 0x7f2c0c004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:45.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.248+0000 7f2c237fe700 1 --2- 172.21.15.99:0/1351597335 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c24157c40 0x7f2c0c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:45.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.248+0000 7f2c28e67700 1 --2- 172.21.15.99:0/1351597335 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c0c004d50 0x7f2c0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:45.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.248+0000 7f2c23fff700 1 -- 172.21.15.99:0/1351597335 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c24157c40 msgr2=0x7f2c0c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:45.253 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.248+0000 7f2c23fff700 1 --2- 172.21.15.99:0/1351597335 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c24157c40 0x7f2c0c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:45.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.248+0000 7f2c23fff700 1 -- 172.21.15.99:0/1351597335 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c0c004d50 msgr2=0x7f2c0c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:45.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.248+0000 7f2c23fff700 1 --2- 172.21.15.99:0/1351597335 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c0c004d50 0x7f2c0c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:45.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.248+0000 7f2c23fff700 1 -- 172.21.15.99:0/1351597335 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c0c003570 con 0x7f2c0c003c20 2023-10-08T00:21:45.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.249+0000 7f2c23fff700 1 --2- 172.21.15.99:0/1351597335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c0c003c20 0x7f2c0c004010 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f2c14007fd0 tx=0x7f2c14001d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26ef2b9824720d2c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:45.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.249+0000 7f2c22ffd700 1 -- 172.21.15.99:0/1351597335 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1400cae0 con 0x7f2c0c003c20 2023-10-08T00:21:45.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.249+0000 7f2c22ffd700 1 -- 172.21.15.99:0/1351597335 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c1400cc60 con 0x7f2c0c003c20 2023-10-08T00:21:45.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.249+0000 7f2c22ffd700 1 -- 172.21.15.99:0/1351597335 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c14005920 con 0x7f2c0c003c20 2023-10-08T00:21:45.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.249+0000 7f2c2a8ca700 1 -- 172.21.15.99:0/1351597335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c0c003c20 msgr2=0x7f2c0c004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:45.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.249+0000 7f2c2a8ca700 1 --2- 172.21.15.99:0/1351597335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c0c003c20 0x7f2c0c004010 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f2c14007fd0 tx=0x7f2c14001d40 comp rx=0 tx=0).stop 2023-10-08T00:21:45.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.249+0000 7f2c28e67700 1 -- 172.21.15.99:0/1351597335 reap_dead start 2023-10-08T00:21:45.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.249+0000 7f2c2a8ca700 1 -- 172.21.15.99:0/1351597335 shutdown_connections 2023-10-08T00:21:45.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.249+0000 7f2c2a8ca700 1 -- 172.21.15.99:0/1351597335 >> 172.21.15.99:0/1351597335 conn(0x7f2c24068ce0 msgr2=0x7f2c240690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:45.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.249+0000 7f2c2a8ca700 1 -- 172.21.15.99:0/1351597335 shutdown_connections 2023-10-08T00:21:45.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.249+0000 7f2c2a8ca700 1 -- 172.21.15.99:0/1351597335 wait complete. 2023-10-08T00:21:45.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.250+0000 7f2c2a8ca700 1 Processor -- start 2023-10-08T00:21:45.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.251+0000 7f2c2a8ca700 1 -- start start 2023-10-08T00:21:45.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.251+0000 7f2c2a8ca700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c0c003c20 0x7f2c24153fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:45.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.251+0000 7f2c2a8ca700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c24151160 0x7f2c241515d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:45.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.251+0000 7f2c2a8ca700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c240b1820 0x7f2c240b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:45.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.251+0000 7f2c2a8ca700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c24151b10 con 0x7f2c240b1820 2023-10-08T00:21:45.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.251+0000 7f2c2a8ca700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c24154520 con 0x7f2c24151160 2023-10-08T00:21:45.258 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.251+0000 7f2c2a8ca700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c240b2550 con 0x7f2c0c003c20 2023-10-08T00:21:45.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.251+0000 7f2c28e67700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c240b1820 0x7f2c240b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:45.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.251+0000 7f2c28e67700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c240b1820 0x7f2c240b1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41720/0 (socket says 172.21.15.99:41720) 2023-10-08T00:21:45.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.251+0000 7f2c28e67700 1 -- 172.21.15.99:0/2651218974 learned_addr learned my addr 172.21.15.99:0/2651218974 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:45.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.252+0000 7f2c237fe700 1 --2- 172.21.15.99:0/2651218974 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c24151160 0x7f2c241515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:45.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.252+0000 7f2c23fff700 1 --2- 172.21.15.99:0/2651218974 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c0c003c20 0x7f2c24153fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:45.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.252+0000 7f2c28e67700 1 -- 172.21.15.99:0/2651218974 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c0c003c20 msgr2=0x7f2c24153fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:45.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.252+0000 7f2c28e67700 1 --2- 172.21.15.99:0/2651218974 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c0c003c20 0x7f2c24153fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:45.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.252+0000 7f2c28e67700 1 -- 172.21.15.99:0/2651218974 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c24151160 msgr2=0x7f2c241515d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:45.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.252+0000 7f2c28e67700 1 --2- 172.21.15.99:0/2651218974 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c24151160 0x7f2c241515d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:45.259 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.252+0000 7f2c28e67700 1 -- 172.21.15.99:0/2651218974 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c240b2830 con 0x7f2c240b1820 2023-10-08T00:21:45.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.252+0000 7f2c23fff700 1 --2- 172.21.15.99:0/2651218974 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c0c003c20 0x7f2c24153fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:21:45.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.252+0000 7f2c28e67700 1 --2- 172.21.15.99:0/2651218974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c240b1820 0x7f2c240b1c70 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c0094d0 tx=0x7f2c1c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:45.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.252+0000 7f2c0bfff700 1 -- 172.21.15.99:0/2651218974 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c00e040 con 0x7f2c240b1820 2023-10-08T00:21:45.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.252+0000 7f2c0bfff700 1 -- 172.21.15.99:0/2651218974 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c1c00f040 con 0x7f2c240b1820 2023-10-08T00:21:45.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.252+0000 7f2c2a8ca700 1 -- 172.21.15.99:0/2651218974 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c240b2b10 con 0x7f2c240b1820 2023-10-08T00:21:45.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.252+0000 7f2c0bfff700 1 -- 172.21.15.99:0/2651218974 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c00fe20 con 0x7f2c240b1820 2023-10-08T00:21:45.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.252+0000 7f2c2a8ca700 1 -- 172.21.15.99:0/2651218974 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c240b21b0 con 0x7f2c240b1820 2023-10-08T00:21:45.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.253+0000 7f2c2a8ca700 1 -- 172.21.15.99:0/2651218974 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c2404c3c0 con 0x7f2c240b1820 2023-10-08T00:21:45.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.253+0000 7f2c0bfff700 1 -- 172.21.15.99:0/2651218974 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2c1c015640 con 0x7f2c240b1820 2023-10-08T00:21:45.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.254+0000 7f2c0bfff700 1 --2- 172.21.15.99:0/2651218974 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2c10066f30 0x7f2c100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:45.260 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.254+0000 7f2c0bfff700 1 -- 172.21.15.99:0/2651218974 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5734+0+0 (secure 0 0 0) 0x7f2c1c084da0 con 0x7f2c240b1820 2023-10-08T00:21:45.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.254+0000 7f2c23fff700 1 --2- 172.21.15.99:0/2651218974 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2c10066f30 0x7f2c100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:45.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.254+0000 7f2c23fff700 1 --2- 172.21.15.99:0/2651218974 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2c10066f30 0x7f2c100693e0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f2c14007ee0 tx=0x7f2c14015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:45.261 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.258+0000 7f2c0bfff700 1 -- 172.21.15.99:0/2651218974 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2c1c051ff0 con 0x7f2c240b1820 2023-10-08T00:21:45.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.447+0000 7f2c2a8ca700 1 -- 172.21.15.99:0/2651218974 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f2c2415c560 con 0x7f2c240b1820 2023-10-08T00:21:45.449 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.448+0000 7f2c0bfff700 1 -- 172.21.15.99:0/2651218974 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v187) v1 ==== 81+0+32474 (secure 0 0 0) 0x7f2c1c055610 con 0x7f2c240b1820 2023-10-08T00:21:45.449 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:21:45.449 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:21:45.450 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 187, 2023-10-08T00:21:45.450 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", 2023-10-08T00:21:45.450 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2023-10-08T00:10:36.303601+0000", 2023-10-08T00:21:45.450 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2023-10-08T00:21:44.347011+0000", 2023-10-08T00:21:45.450 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2023-10-08T00:10:41.709171+0000", 2023-10-08T00:21:45.450 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2023-10-08T00:10:38.690404+0000", 2023-10-08T00:21:45.450 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-08T00:21:45.450 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2023-10-08T00:21:45.450 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2023-10-08T00:21:45.450 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2023-10-08T00:21:45.450 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2023-10-08T00:21:45.450 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2023-10-08T00:21:45.451 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2023-10-08T00:21:45.451 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:21:45.451 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2023-10-08T00:21:45.451 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2023-10-08T00:21:45.451 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-08T00:21:45.451 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-08T00:21:45.451 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2023-10-08T00:21:45.451 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 23, 2023-10-08T00:21:45.451 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2023-10-08T00:21:45.451 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2023-10-08T00:21:45.451 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "luminous", 2023-10-08T00:21:45.451 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "quincy", 2023-10-08T00:21:45.452 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2023-10-08T00:21:45.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.452 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2023-10-08T00:21:45.452 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2023-10-08T00:21:45.452 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:10:42.063344+0000", 2023-10-08T00:21:45.452 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:21:45.452 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:21:45.452 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:21:45.452 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:21:45.452 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:21:45.452 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:21:45.453 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:21:45.453 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:21:45.453 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:21:45.453 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:21:45.453 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:21:45.453 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:21:45.453 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2023-10-08T00:21:45.453 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2023-10-08T00:21:45.453 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2023-10-08T00:21:45.453 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2023-10-08T00:21:45.453 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2023-10-08T00:21:45.453 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:21:45.453 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:21:45.454 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:21:45.454 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:21:45.454 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:21:45.454 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:21:45.454 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:21:45.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.454 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "112", 2023-10-08T00:21:45.454 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:21:45.454 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:21:45.454 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:21:45.454 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:21:45.454 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:21:45.455 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:21:45.455 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:21:45.455 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:21:45.455 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:21:45.455 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:21:45.455 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:21:45.455 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:21:45.455 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:21:45.455 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:21:45.455 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:21:45.455 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:21:45.455 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:21:45.456 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:21:45.456 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:21:45.456 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:21:45.456 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:21:45.456 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:21:45.456 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:21:45.456 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:21:45.456 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:21:45.456 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:21:45.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.456 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:21:45.456 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:21:45.456 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:21:45.457 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:21:45.457 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:21:45.457 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:21:45.457 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:21:45.457 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:21:45.457 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:21:45.457 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:21:45.457 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:21:45.457 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:21:45.457 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2023-10-08T00:21:45.457 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2023-10-08T00:21:45.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.458 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:21:45.458 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2023-10-08T00:21:45.458 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:45.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.458 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 22, 2023-10-08T00:21:45.458 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2023-10-08T00:21:45.458 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:21:27.984574+0000", 2023-10-08T00:21:45.458 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:21:45.458 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:21:45.458 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:21:45.459 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:21:45.459 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:21:45.459 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:21:45.459 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:21:45.459 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:21:45.459 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:21:45.459 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:21:45.459 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:21:45.459 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:21:45.459 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2023-10-08T00:21:45.459 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2023-10-08T00:21:45.459 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2023-10-08T00:21:45.460 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2023-10-08T00:21:45.460 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2023-10-08T00:21:45.460 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:21:45.460 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:21:45.460 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:21:45.460 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:21:45.460 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:21:45.460 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:21:45.460 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:21:45.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.460 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "183", 2023-10-08T00:21:45.460 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:21:45.460 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:21:45.461 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:21:45.461 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:21:45.461 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:21:45.461 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:21:45.461 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:21:45.461 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:21:45.461 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:21:45.461 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:21:45.461 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:21:45.461 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:21:45.461 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:21:45.461 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:21:45.462 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:21:45.462 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:21:45.462 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:21:45.462 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:21:45.462 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:21:45.462 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:21:45.462 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:21:45.462 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:21:45.462 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:21:45.462 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:21:45.462 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:21:45.462 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:21:45.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.463 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:21:45.463 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:21:45.463 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:21:45.463 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:21:45.463 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:21:45.463 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:21:45.463 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:21:45.463 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:21:45.463 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:21:45.463 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:21:45.463 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:21:45.464 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:21:45.464 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2023-10-08T00:21:45.464 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2023-10-08T00:21:45.464 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2023-10-08T00:21:45.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.464 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:21:45.464 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:21:45.464 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2023-10-08T00:21:45.464 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:45.464 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:45.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.465 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 23, 2023-10-08T00:21:45.465 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2023-10-08T00:21:45.465 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:21:29.244620+0000", 2023-10-08T00:21:45.465 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:21:45.465 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:21:45.465 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:21:45.465 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:21:45.465 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:21:45.465 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:21:45.465 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:21:45.465 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:21:45.466 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:21:45.466 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:21:45.466 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:21:45.466 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:21:45.466 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2023-10-08T00:21:45.466 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2023-10-08T00:21:45.466 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2023-10-08T00:21:45.466 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2023-10-08T00:21:45.466 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2023-10-08T00:21:45.466 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:21:45.466 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:21:45.466 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:21:45.467 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:21:45.467 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:21:45.467 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:21:45.467 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:21:45.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.467 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "185", 2023-10-08T00:21:45.467 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:21:45.467 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:21:45.467 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:21:45.467 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:21:45.467 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:21:45.467 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:21:45.468 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:21:45.468 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:21:45.468 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:21:45.468 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 33554432, 2023-10-08T00:21:45.468 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:21:45.468 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:21:45.468 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:21:45.468 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:21:45.468 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:21:45.468 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:21:45.468 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:21:45.468 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:21:45.469 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:21:45.469 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:21:45.469 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:21:45.469 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:21:45.469 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:21:45.469 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:21:45.469 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:21:45.469 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:21:45.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.469 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:21:45.469 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:21:45.469 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:21:45.470 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:21:45.470 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:21:45.470 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:21:45.470 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:21:45.470 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:21:45.470 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:21:45.470 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:21:45.470 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:21:45.470 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:21:45.470 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2023-10-08T00:21:45.470 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2023-10-08T00:21:45.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.471 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:21:45.471 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:21:45.471 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2023-10-08T00:21:45.471 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:45.471 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:45.471 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:45.471 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:21:45.471 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2023-10-08T00:21:45.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.471 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:21:45.471 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:21:45.471 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:21:45.472 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:21:45.472 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:21:45.472 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:21:45.472 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:21:45.472 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:21:45.472 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:21:45.472 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:21:45.472 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:21:45.472 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:21:45.472 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:21:45.472 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:45.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.473 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:45.473 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2023-10-08T00:21:45.473 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:21:45.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.473 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:45.473 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2023-10-08T00:21:45.473 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:21:45.473 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:45.473 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:45.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.473 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:21:45.474 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:45.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.474 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:45.474 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2023-10-08T00:21:45.474 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:21:45.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.474 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:45.474 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2023-10-08T00:21:45.474 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:21:45.474 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:45.474 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:45.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.475 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:21:45.475 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:45.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.475 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:45.475 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2023-10-08T00:21:45.475 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:21:45.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.475 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:45.475 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2023-10-08T00:21:45.475 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:21:45.475 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:45.476 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:45.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.476 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:21:45.476 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:45.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.476 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:45.476 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2023-10-08T00:21:45.476 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:21:45.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.476 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:45.476 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2023-10-08T00:21:45.477 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:21:45.477 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:45.477 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:45.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.477 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/86365", 2023-10-08T00:21:45.477 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/86365", 2023-10-08T00:21:45.477 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/86365", 2023-10-08T00:21:45.477 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/86365", 2023-10-08T00:21:45.477 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:21:45.477 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:21:45.477 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:21:45.477 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:45.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.478 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:21:45.478 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:21:45.478 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:21:45.478 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:21:45.478 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:21:45.478 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:21:45.478 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:21:45.478 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:21:45.478 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:21:45.478 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:21:45.478 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:21:45.479 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:21:45.479 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:21:45.479 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:45.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.479 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:45.479 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2023-10-08T00:21:45.479 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:21:45.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.479 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:45.479 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2023-10-08T00:21:45.479 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:21:45.480 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:45.480 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:45.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.480 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:21:45.480 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:45.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.480 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:45.480 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2023-10-08T00:21:45.480 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:21:45.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.480 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:45.480 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2023-10-08T00:21:45.481 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:21:45.481 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:45.481 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:45.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.481 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:21:45.481 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:45.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.481 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:45.481 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2023-10-08T00:21:45.481 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:21:45.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.482 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:45.482 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2023-10-08T00:21:45.482 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:21:45.482 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:45.482 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:45.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.482 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:21:45.482 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:45.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.482 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:45.482 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2023-10-08T00:21:45.482 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:21:45.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.483 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:45.483 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2023-10-08T00:21:45.483 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:21:45.483 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:45.483 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:45.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.483 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/86366", 2023-10-08T00:21:45.483 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/86366", 2023-10-08T00:21:45.483 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/86366", 2023-10-08T00:21:45.483 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/86366", 2023-10-08T00:21:45.483 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:21:45.484 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:21:45.484 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:21:45.484 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:45.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.484 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:21:45.484 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:21:45.484 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:21:45.484 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:21:45.484 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:21:45.484 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:21:45.484 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:21:45.485 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:21:45.485 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:21:45.485 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:21:45.485 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:21:45.485 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:21:45.485 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:21:45.485 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:45.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.485 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:45.485 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2023-10-08T00:21:45.485 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:21:45.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.486 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:45.486 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2023-10-08T00:21:45.486 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:21:45.486 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:45.486 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:45.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.486 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:21:45.486 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:45.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.486 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:45.486 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2023-10-08T00:21:45.487 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:21:45.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.487 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:45.487 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2023-10-08T00:21:45.487 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:21:45.487 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:45.487 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:45.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.487 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:21:45.487 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:45.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.488 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:45.488 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2023-10-08T00:21:45.488 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:21:45.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.488 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:45.488 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2023-10-08T00:21:45.488 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:21:45.488 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:45.488 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:45.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.489 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:21:45.489 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:45.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.489 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:45.489 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2023-10-08T00:21:45.489 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:21:45.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.489 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:45.489 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2023-10-08T00:21:45.489 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:21:45.489 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:45.490 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:45.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.490 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6817/86382", 2023-10-08T00:21:45.490 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/86382", 2023-10-08T00:21:45.490 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/86382", 2023-10-08T00:21:45.490 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/86382", 2023-10-08T00:21:45.490 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:21:45.490 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:21:45.490 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:21:45.490 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:45.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.491 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:21:45.491 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:21:45.491 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:21:45.491 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:21:45.491 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:21:45.491 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:21:45.491 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:21:45.491 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:21:45.491 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:21:45.491 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 186, 2023-10-08T00:21:45.491 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:21:45.492 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:21:45.492 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:21:45.492 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:45.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.492 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:45.492 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2023-10-08T00:21:45.492 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:21:45.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.492 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:45.492 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2023-10-08T00:21:45.492 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:21:45.493 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:45.493 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:45.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.493 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:21:45.493 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:45.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.493 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:45.493 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2023-10-08T00:21:45.493 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:21:45.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.493 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:45.494 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2023-10-08T00:21:45.494 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:21:45.494 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:45.494 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:45.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.494 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:21:45.494 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:45.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.494 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:45.494 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2023-10-08T00:21:45.494 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:21:45.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.495 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:45.495 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2023-10-08T00:21:45.495 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:21:45.495 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:45.495 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:45.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.495 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:21:45.495 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:45.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.495 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:45.496 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2023-10-08T00:21:45.496 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:21:45.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.496 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:45.496 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2023-10-08T00:21:45.496 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:21:45.496 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:45.496 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:45.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.496 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6820/86393", 2023-10-08T00:21:45.496 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/86393", 2023-10-08T00:21:45.496 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/86393", 2023-10-08T00:21:45.497 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/86393", 2023-10-08T00:21:45.497 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:21:45.497 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:21:45.497 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:21:45.497 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:45.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.497 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:21:45.497 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:21:45.497 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:21:45.497 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:21:45.497 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:21:45.498 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:21:45.498 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:21:45.498 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:21:45.498 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:21:45.498 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:21:45.498 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:21:45.498 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:21:45.498 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:21:45.498 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:45.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.498 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:45.498 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6800", 2023-10-08T00:21:45.499 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:21:45.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.499 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:45.499 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6801", 2023-10-08T00:21:45.499 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:21:45.499 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:45.499 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:45.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.499 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:21:45.499 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:45.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.500 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:45.500 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6802", 2023-10-08T00:21:45.500 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:21:45.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.500 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:45.500 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6803", 2023-10-08T00:21:45.500 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:21:45.500 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:45.500 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:45.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.500 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:21:45.500 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:45.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.501 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:45.501 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6806", 2023-10-08T00:21:45.501 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:21:45.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.501 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:45.501 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6807", 2023-10-08T00:21:45.501 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:21:45.501 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:45.501 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:45.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.502 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:21:45.502 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:45.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.502 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:45.502 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6804", 2023-10-08T00:21:45.502 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:21:45.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.502 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:45.502 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6805", 2023-10-08T00:21:45.502 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:21:45.502 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:45.503 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:45.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.503 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6801/86570", 2023-10-08T00:21:45.503 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6803/86570", 2023-10-08T00:21:45.503 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6807/86570", 2023-10-08T00:21:45.503 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6805/86570", 2023-10-08T00:21:45.503 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:21:45.503 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:21:45.503 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:21:45.503 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:45.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.504 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:21:45.504 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:21:45.504 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:21:45.504 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:21:45.504 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:21:45.504 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:21:45.504 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:21:45.504 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:21:45.504 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:21:45.504 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:21:45.504 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:21:45.504 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:21:45.504 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:21:45.505 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:45.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.505 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:45.505 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6816", 2023-10-08T00:21:45.505 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:21:45.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.505 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:45.505 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6818", 2023-10-08T00:21:45.505 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:21:45.505 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:45.505 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:45.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.506 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:21:45.506 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:45.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.506 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:45.506 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6820", 2023-10-08T00:21:45.506 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:21:45.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.506 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:45.506 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6821", 2023-10-08T00:21:45.506 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:21:45.507 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:45.507 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:45.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.507 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:21:45.507 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:45.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.507 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:45.507 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6826", 2023-10-08T00:21:45.507 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:21:45.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.507 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:45.508 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6828", 2023-10-08T00:21:45.509 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:21:45.509 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:45.509 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:45.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.509 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:21:45.509 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:45.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.510 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:45.510 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6823", 2023-10-08T00:21:45.510 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:21:45.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.510 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:45.510 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6825", 2023-10-08T00:21:45.510 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:21:45.510 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:45.510 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:45.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.510 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6818/86608", 2023-10-08T00:21:45.511 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6821/86608", 2023-10-08T00:21:45.511 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6828/86608", 2023-10-08T00:21:45.511 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6825/86608", 2023-10-08T00:21:45.511 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:21:45.511 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:21:45.511 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:21:45.511 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:45.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.511 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:21:45.511 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:21:45.511 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:21:45.512 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:21:45.512 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:21:45.512 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:21:45.512 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:21:45.512 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:21:45.512 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:21:45.512 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:21:45.512 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:21:45.512 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:21:45.512 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:21:45.512 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:45.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.512 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:45.513 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6808", 2023-10-08T00:21:45.513 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:21:45.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.513 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:45.513 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6809", 2023-10-08T00:21:45.513 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:21:45.513 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:45.513 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:45.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.513 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:21:45.514 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:45.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.514 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:45.514 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6810", 2023-10-08T00:21:45.514 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:21:45.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.514 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:45.514 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6811", 2023-10-08T00:21:45.514 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:21:45.514 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:45.514 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:45.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.515 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:21:45.515 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:45.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.515 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:45.515 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6814", 2023-10-08T00:21:45.515 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:21:45.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.515 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:45.515 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6815", 2023-10-08T00:21:45.515 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:21:45.515 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:45.516 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:45.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.516 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:21:45.516 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:45.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.516 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:45.516 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6812", 2023-10-08T00:21:45.516 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:21:45.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.516 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:45.516 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6813", 2023-10-08T00:21:45.517 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:21:45.517 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:45.517 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:45.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.517 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6809/86571", 2023-10-08T00:21:45.517 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6811/86571", 2023-10-08T00:21:45.517 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6815/86571", 2023-10-08T00:21:45.517 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6813/86571", 2023-10-08T00:21:45.517 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:21:45.517 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:21:45.517 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:21:45.517 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:45.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.518 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:21:45.518 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:21:45.518 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:21:45.518 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:21:45.518 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:21:45.518 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:21:45.518 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:21:45.518 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:21:45.518 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:21:45.518 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:21:45.519 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:21:45.519 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:21:45.519 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:21:45.519 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:45.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.519 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:45.519 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6817", 2023-10-08T00:21:45.519 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:21:45.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.519 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:45.519 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6819", 2023-10-08T00:21:45.519 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:21:45.520 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:45.520 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:45.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.520 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:21:45.520 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:45.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.520 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:45.520 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6822", 2023-10-08T00:21:45.520 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:21:45.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.520 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:45.521 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6824", 2023-10-08T00:21:45.521 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:21:45.521 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:45.521 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:45.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.521 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:21:45.521 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:45.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.521 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:45.521 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6830", 2023-10-08T00:21:45.521 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:21:45.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.522 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:45.522 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6831", 2023-10-08T00:21:45.522 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:21:45.522 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:45.522 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:45.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.522 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:21:45.522 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:45.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.522 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:45.522 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6827", 2023-10-08T00:21:45.522 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:21:45.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.523 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:45.523 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6829", 2023-10-08T00:21:45.523 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:21:45.523 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:45.523 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:45.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.523 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6819/86612", 2023-10-08T00:21:45.523 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6824/86612", 2023-10-08T00:21:45.523 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6831/86612", 2023-10-08T00:21:45.523 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6829/86612", 2023-10-08T00:21:45.524 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:21:45.524 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:21:45.524 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:21:45.524 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:45.524 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:45.524 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:21:45.524 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2023-10-08T00:21:45.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.524 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:21:45.524 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:21:45.524 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:21:45.524 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:21:45.524 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:21:45.525 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:21:45.525 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.905615+0000", 2023-10-08T00:21:45.525 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:21:45.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.525 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:21:45.525 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:21:45.525 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:21:45.525 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:21:45.526 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:21:45.526 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:21:45.526 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.880709+0000", 2023-10-08T00:21:45.526 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:21:45.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.526 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:21:45.526 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:21:45.526 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:21:45.526 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:21:45.526 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:21:45.526 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:21:45.527 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.874704+0000", 2023-10-08T00:21:45.527 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:21:45.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.527 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:21:45.527 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:21:45.527 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:21:45.527 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:21:45.527 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:21:45.527 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:21:45.527 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.899604+0000", 2023-10-08T00:21:45.527 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:21:45.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.528 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:21:45.528 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:21:45.528 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:21:45.528 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:21:45.528 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:21:45.528 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:21:45.528 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.844371+0000", 2023-10-08T00:21:45.528 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:21:45.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.529 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:21:45.529 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:21:45.529 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:21:45.529 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:21:45.529 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:21:45.529 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:21:45.529 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.794810+0000", 2023-10-08T00:21:45.529 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:21:45.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.529 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:21:45.529 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:21:45.529 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:21:45.530 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:21:45.530 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:21:45.530 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:21:45.530 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.859474+0000", 2023-10-08T00:21:45.530 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:21:45.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.530 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:21:45.530 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:21:45.530 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:21:45.530 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:21:45.530 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:21:45.531 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:21:45.531 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.893130+0000", 2023-10-08T00:21:45.531 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:21:45.531 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:45.531 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:21:45.531 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2023-10-08T00:21:45.531 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [ 2023-10-08T00:21:45.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.531 INFO:teuthology.orchestra.run.smithi099.stdout: "pgid": "23.18", 2023-10-08T00:21:45.531 INFO:teuthology.orchestra.run.smithi099.stdout: "mappings": [ 2023-10-08T00:21:45.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:45.531 INFO:teuthology.orchestra.run.smithi099.stdout: "from": 5, 2023-10-08T00:21:45.532 INFO:teuthology.orchestra.run.smithi099.stdout: "to": 3 2023-10-08T00:21:45.532 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:45.532 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:45.532 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:45.532 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:21:45.532 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2023-10-08T00:21:45.532 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2023-10-08T00:21:45.532 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2023-10-08T00:21:45.532 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2023-10-08T00:21:45.532 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2023-10-08T00:21:45.532 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2023-10-08T00:21:45.532 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2023-10-08T00:21:45.532 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2023-10-08T00:21:45.533 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2023-10-08T00:21:45.533 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2023-10-08T00:21:45.533 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2023-10-08T00:21:45.533 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:45.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:45.533 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2023-10-08T00:21:45.533 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2023-10-08T00:21:45.533 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2023-10-08T00:21:45.533 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2023-10-08T00:21:45.533 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2023-10-08T00:21:45.533 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2023-10-08T00:21:45.533 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2023-10-08T00:21:45.534 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2023-10-08T00:21:45.534 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2023-10-08T00:21:45.534 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2023-10-08T00:21:45.534 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2023-10-08T00:21:45.534 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:45.534 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:21:45.536 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.451+0000 7f2c2a8ca700 1 -- 172.21.15.99:0/2651218974 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2c10066f30 msgr2=0x7f2c100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:45.536 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.451+0000 7f2c2a8ca700 1 --2- 172.21.15.99:0/2651218974 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2c10066f30 0x7f2c100693e0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f2c14007ee0 tx=0x7f2c14015000 comp rx=0 tx=0).stop 2023-10-08T00:21:45.536 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.451+0000 7f2c2a8ca700 1 -- 172.21.15.99:0/2651218974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c240b1820 msgr2=0x7f2c240b1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:45.536 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.451+0000 7f2c2a8ca700 1 --2- 172.21.15.99:0/2651218974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c240b1820 0x7f2c240b1c70 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c0094d0 tx=0x7f2c1c00b040 comp rx=0 tx=0).stop 2023-10-08T00:21:45.536 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.451+0000 7f2c2a8ca700 1 -- 172.21.15.99:0/2651218974 shutdown_connections 2023-10-08T00:21:45.536 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.451+0000 7f2c2a8ca700 1 --2- 172.21.15.99:0/2651218974 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2c10066f30 0x7f2c100693e0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:45.536 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.451+0000 7f2c2a8ca700 1 --2- 172.21.15.99:0/2651218974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c240b1820 0x7f2c240b1c70 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:45.536 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.451+0000 7f2c2a8ca700 1 --2- 172.21.15.99:0/2651218974 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c0c003c20 0x7f2c24153fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:45.536 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.451+0000 7f2c2a8ca700 1 --2- 172.21.15.99:0/2651218974 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c24151160 0x7f2c241515d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:45.536 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.451+0000 7f2c2a8ca700 1 -- 172.21.15.99:0/2651218974 >> 172.21.15.99:0/2651218974 conn(0x7f2c24068ce0 msgr2=0x7f2c240b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:45.536 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.451+0000 7f2c2a8ca700 1 -- 172.21.15.99:0/2651218974 shutdown_connections 2023-10-08T00:21:45.537 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:45.451+0000 7f2c2a8ca700 1 -- 172.21.15.99:0/2651218974 wait complete. 2023-10-08T00:21:45.538 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-08T00:21:50.539 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-08T00:21:50.666 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.665+0000 7f3716b15700 1 Processor -- start 2023-10-08T00:21:50.666 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.665+0000 7f3716b15700 1 -- start start 2023-10-08T00:21:50.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.665+0000 7f3716b15700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f37100cffa0 0x7f371006a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:50.676 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.665+0000 7f3716b15700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f371015a230 0x7f371006abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:50.676 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.665+0000 7f3716b15700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37100b3d40 con 0x7f3710159e60 2023-10-08T00:21:50.676 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.665+0000 7f3716b15700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3710069d70 con 0x7f37100cffa0 2023-10-08T00:21:50.676 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.666+0000 7f3716b15700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37100b4300 con 0x7f371015a230 2023-10-08T00:21:50.676 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.666+0000 7f3707fff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f37100cffa0 0x7f371006a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:50.677 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.666+0000 7f37148b1700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f3710159e60 0x7f37100b6890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59974/0 (socket says 172.21.15.99:59974) 2023-10-08T00:21:50.677 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.666+0000 7f37148b1700 1 -- 172.21.15.99:0/1156050345 learned_addr learned my addr 172.21.15.99:0/1156050345 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:21:50.677 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.666+0000 7f3707fff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f37100cffa0 0x7f371006a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41722/0 (socket says 172.21.15.99:41722) 2023-10-08T00:21:50.677 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.666+0000 7f3707fff700 1 -- 172.21.15.99:0/1156050345 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f371015a230 msgr2=0x7f371006abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:50.677 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.666+0000 7f37150b2700 1 --2- 172.21.15.99:0/1156050345 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f371015a230 0x7f371006abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:50.678 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.666+0000 7f3707fff700 1 --2- 172.21.15.99:0/1156050345 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f371015a230 0x7f371006abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:50.678 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.666+0000 7f37077fe700 1 -- 172.21.15.99:0/1156050345 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 93246847 0 0) 0x7f37100b3d40 con 0x7f3710159e60 2023-10-08T00:21:50.678 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.666+0000 7f3707fff700 1 -- 172.21.15.99:0/1156050345 >> v1:172.21.15.99:6789/0 conn(0x7f3710159e60 legacy=0x7f37100b6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:50.679 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.666+0000 7f3707fff700 1 -- 172.21.15.99:0/1156050345 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37100d0480 con 0x7f37100cffa0 2023-10-08T00:21:50.679 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.666+0000 7f37150b2700 1 --2- 172.21.15.99:0/1156050345 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f371015a230 0x7f371006abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:21:50.679 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.666+0000 7f3707fff700 1 --2- 172.21.15.99:0/1156050345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37100cffa0 0x7f371006a6a0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f37000098e0 tx=0x7f37000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23b8f8ce7aca3f22 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:50.680 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.666+0000 7f37077fe700 1 -- 172.21.15.99:0/1156050345 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3700009e80 con 0x7f37100cffa0 2023-10-08T00:21:50.680 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.666+0000 7f37077fe700 1 -- 172.21.15.99:0/1156050345 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f370000b860 con 0x7f37100cffa0 2023-10-08T00:21:50.680 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.666+0000 7f37077fe700 1 -- 172.21.15.99:0/1156050345 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f370000bbe0 con 0x7f37100cffa0 2023-10-08T00:21:50.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.667+0000 7f3716b15700 1 -- 172.21.15.99:0/1156050345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37100cffa0 msgr2=0x7f371006a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:50.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.667+0000 7f3716b15700 1 --2- 172.21.15.99:0/1156050345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37100cffa0 0x7f371006a6a0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f37000098e0 tx=0x7f37000091a0 comp rx=0 tx=0).stop 2023-10-08T00:21:50.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.667+0000 7f3716b15700 1 -- 172.21.15.99:0/1156050345 shutdown_connections 2023-10-08T00:21:50.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.667+0000 7f3716b15700 1 --2- 172.21.15.99:0/1156050345 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f371015a230 0x7f371006abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:50.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.667+0000 7f3716b15700 1 --2- 172.21.15.99:0/1156050345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37100cffa0 0x7f371006a6a0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:50.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.667+0000 7f3716b15700 1 -- 172.21.15.99:0/1156050345 >> 172.21.15.99:0/1156050345 conn(0x7f37100d1a50 msgr2=0x7f3710068f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:50.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.667+0000 7f3716b15700 1 -- 172.21.15.99:0/1156050345 shutdown_connections 2023-10-08T00:21:50.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.667+0000 7f3716b15700 1 -- 172.21.15.99:0/1156050345 wait complete. 2023-10-08T00:21:50.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.668+0000 7f3716b15700 1 Processor -- start 2023-10-08T00:21:50.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.668+0000 7f3716b15700 1 -- start start 2023-10-08T00:21:50.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.669+0000 7f3716b15700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f37100cffa0 0x7f37100c8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:50.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.669+0000 7f3716b15700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3710159e60 0x7f371016b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:50.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.669+0000 7f3716b15700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f371015a230 0x7f371016de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:50.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.669+0000 7f3716b15700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37101553d0 con 0x7f3710159e60 2023-10-08T00:21:50.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.669+0000 7f3716b15700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f371016ecf0 con 0x7f371015a230 2023-10-08T00:21:50.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.669+0000 7f3716b15700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f371016ad60 con 0x7f37100cffa0 2023-10-08T00:21:50.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.669+0000 7f3707fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3710159e60 0x7f371016b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:50.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.669+0000 7f3707fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3710159e60 0x7f371016b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41734/0 (socket says 172.21.15.99:41734) 2023-10-08T00:21:50.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.669+0000 7f3707fff700 1 -- 172.21.15.99:0/3402763285 learned_addr learned my addr 172.21.15.99:0/3402763285 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:50.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.669+0000 7f37148b1700 1 --2- 172.21.15.99:0/3402763285 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f37100cffa0 0x7f37100c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:50.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.669+0000 7f37150b2700 1 --2- 172.21.15.99:0/3402763285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f371015a230 0x7f371016de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:50.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.669+0000 7f3707fff700 1 -- 172.21.15.99:0/3402763285 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f37100cffa0 msgr2=0x7f37100c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:50.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.669+0000 7f3707fff700 1 --2- 172.21.15.99:0/3402763285 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f37100cffa0 0x7f37100c8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:50.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.669+0000 7f3707fff700 1 -- 172.21.15.99:0/3402763285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f371015a230 msgr2=0x7f371016de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:50.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.669+0000 7f3707fff700 1 --2- 172.21.15.99:0/3402763285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f371015a230 0x7f371016de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:50.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.669+0000 7f3707fff700 1 -- 172.21.15.99:0/3402763285 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3700009420 con 0x7f3710159e60 2023-10-08T00:21:50.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.669+0000 7f37148b1700 1 --2- 172.21.15.99:0/3402763285 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f37100cffa0 0x7f37100c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:21:50.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.669+0000 7f37150b2700 1 --2- 172.21.15.99:0/3402763285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f371015a230 0x7f371016de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:21:50.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.669+0000 7f3707fff700 1 --2- 172.21.15.99:0/3402763285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3710159e60 0x7f371016b1f0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f3710089730 tx=0x7f3700017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:50.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.670+0000 7f3704ff9700 1 -- 172.21.15.99:0/3402763285 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3700011420 con 0x7f3710159e60 2023-10-08T00:21:50.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.670+0000 7f3704ff9700 1 -- 172.21.15.99:0/3402763285 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37000115a0 con 0x7f3710159e60 2023-10-08T00:21:50.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.670+0000 7f3716b15700 1 -- 172.21.15.99:0/3402763285 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37100c8d70 con 0x7f3710159e60 2023-10-08T00:21:50.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.670+0000 7f3704ff9700 1 -- 172.21.15.99:0/3402763285 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3700011960 con 0x7f3710159e60 2023-10-08T00:21:50.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.670+0000 7f3716b15700 1 -- 172.21.15.99:0/3402763285 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f371016ea10 con 0x7f3710159e60 2023-10-08T00:21:50.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.670+0000 7f3716b15700 1 -- 172.21.15.99:0/3402763285 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f371004c3c0 con 0x7f3710159e60 2023-10-08T00:21:50.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.671+0000 7f3704ff9700 1 -- 172.21.15.99:0/3402763285 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3700011d50 con 0x7f3710159e60 2023-10-08T00:21:50.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.671+0000 7f3704ff9700 1 --2- 172.21.15.99:0/3402763285 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f36fc066f30 0x7f36fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:50.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.671+0000 7f3704ff9700 1 -- 172.21.15.99:0/3402763285 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5734+0+0 (secure 0 0 0) 0x7f3700086080 con 0x7f3710159e60 2023-10-08T00:21:50.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.671+0000 7f3704ff9700 1 -- 172.21.15.99:0/3402763285 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=189}) v3 -- 0x7f36fc06eab0 con 0x7f3710159e60 2023-10-08T00:21:50.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.672+0000 7f37148b1700 1 --2- 172.21.15.99:0/3402763285 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f36fc066f30 0x7f36fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:50.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.672+0000 7f37148b1700 1 --2- 172.21.15.99:0/3402763285 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f36fc066f30 0x7f36fc0693e0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f370c001840 tx=0x7f370c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:50.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.676+0000 7f3704ff9700 1 -- 172.21.15.99:0/3402763285 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f37000532d0 con 0x7f3710159e60 2023-10-08T00:21:50.859 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.859+0000 7f3716b15700 1 -- 172.21.15.99:0/3402763285 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f371015f9f0 con 0x7f3710159e60 2023-10-08T00:21:50.860 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.860+0000 7f3704ff9700 1 -- 172.21.15.99:0/3402763285 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v188) v1 ==== 81+0+32493 (secure 0 0 0) 0x7f3700015020 con 0x7f3710159e60 2023-10-08T00:21:50.861 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:21:50.861 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:21:50.861 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 188, 2023-10-08T00:21:50.861 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", 2023-10-08T00:21:50.861 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2023-10-08T00:10:36.303601+0000", 2023-10-08T00:21:50.861 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2023-10-08T00:21:46.168808+0000", 2023-10-08T00:21:50.861 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2023-10-08T00:10:41.709171+0000", 2023-10-08T00:21:50.861 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2023-10-08T00:10:38.690404+0000", 2023-10-08T00:21:50.862 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-08T00:21:50.862 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2023-10-08T00:21:50.862 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2023-10-08T00:21:50.862 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2023-10-08T00:21:50.862 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2023-10-08T00:21:50.862 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2023-10-08T00:21:50.862 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2023-10-08T00:21:50.862 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:21:50.862 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2023-10-08T00:21:50.862 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2023-10-08T00:21:50.862 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-08T00:21:50.862 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-08T00:21:50.863 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2023-10-08T00:21:50.863 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 23, 2023-10-08T00:21:50.863 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2023-10-08T00:21:50.863 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2023-10-08T00:21:50.863 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "luminous", 2023-10-08T00:21:50.863 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "quincy", 2023-10-08T00:21:50.863 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2023-10-08T00:21:50.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.863 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2023-10-08T00:21:50.863 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2023-10-08T00:21:50.863 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:10:42.063344+0000", 2023-10-08T00:21:50.864 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:21:50.864 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:21:50.864 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:21:50.864 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:21:50.864 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:21:50.864 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:21:50.864 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:21:50.864 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:21:50.864 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:21:50.864 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:21:50.864 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:21:50.864 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:21:50.865 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2023-10-08T00:21:50.867 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2023-10-08T00:21:50.867 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2023-10-08T00:21:50.867 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2023-10-08T00:21:50.867 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2023-10-08T00:21:50.867 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:21:50.867 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:21:50.867 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:21:50.867 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:21:50.867 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:21:50.868 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:21:50.868 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:21:50.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.868 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "112", 2023-10-08T00:21:50.868 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:21:50.868 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:21:50.868 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:21:50.868 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:21:50.868 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:21:50.869 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:21:50.869 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:21:50.869 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:21:50.869 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:21:50.869 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:21:50.869 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:21:50.869 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:21:50.869 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:21:50.869 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:21:50.869 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:21:50.870 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:21:50.870 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:21:50.870 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:21:50.870 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:21:50.870 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:21:50.870 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:21:50.870 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:21:50.870 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:21:50.870 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:21:50.870 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:21:50.870 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:21:50.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.871 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:21:50.871 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:21:50.871 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:21:50.871 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:21:50.871 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:21:50.871 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:21:50.871 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:21:50.871 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:21:50.871 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:21:50.871 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:21:50.871 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:21:50.871 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:21:50.872 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2023-10-08T00:21:50.872 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2023-10-08T00:21:50.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.872 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:21:50.872 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2023-10-08T00:21:50.872 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:50.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.872 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 22, 2023-10-08T00:21:50.872 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2023-10-08T00:21:50.872 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:21:27.984574+0000", 2023-10-08T00:21:50.873 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:21:50.873 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:21:50.873 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:21:50.873 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:21:50.873 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:21:50.873 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:21:50.873 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:21:50.873 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:21:50.873 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:21:50.873 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:21:50.873 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:21:50.873 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:21:50.873 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2023-10-08T00:21:50.874 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2023-10-08T00:21:50.874 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2023-10-08T00:21:50.874 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2023-10-08T00:21:50.874 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2023-10-08T00:21:50.874 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:21:50.874 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:21:50.874 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:21:50.874 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:21:50.874 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:21:50.874 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:21:50.874 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:21:50.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.875 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "183", 2023-10-08T00:21:50.875 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:21:50.875 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:21:50.875 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:21:50.875 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:21:50.875 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:21:50.875 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:21:50.875 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:21:50.875 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:21:50.875 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:21:50.875 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:21:50.876 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:21:50.876 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:21:50.876 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:21:50.876 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:21:50.876 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:21:50.876 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:21:50.876 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:21:50.876 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:21:50.876 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:21:50.876 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:21:50.876 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:21:50.876 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:21:50.876 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:21:50.877 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:21:50.877 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:21:50.877 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:21:50.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.877 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:21:50.877 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:21:50.877 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:21:50.877 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:21:50.877 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:21:50.877 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:21:50.877 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:21:50.877 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:21:50.878 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:21:50.878 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:21:50.878 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:21:50.878 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:21:50.878 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2023-10-08T00:21:50.878 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2023-10-08T00:21:50.878 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2023-10-08T00:21:50.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.878 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:21:50.878 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:21:50.878 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2023-10-08T00:21:50.879 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:50.879 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:50.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.879 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 23, 2023-10-08T00:21:50.879 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2023-10-08T00:21:50.879 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:21:29.244620+0000", 2023-10-08T00:21:50.879 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1027, 2023-10-08T00:21:50.879 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full,full_quota", 2023-10-08T00:21:50.879 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:21:50.879 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:21:50.879 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:21:50.880 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:21:50.880 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:21:50.880 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:21:50.880 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:21:50.880 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:21:50.880 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:21:50.880 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:21:50.880 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2023-10-08T00:21:50.880 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2023-10-08T00:21:50.880 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2023-10-08T00:21:50.880 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2023-10-08T00:21:50.880 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2023-10-08T00:21:50.881 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:21:50.881 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:21:50.881 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:21:50.881 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:21:50.881 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:21:50.881 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:21:50.881 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:21:50.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.881 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "188", 2023-10-08T00:21:50.881 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:21:50.881 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:21:50.881 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:21:50.882 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:21:50.882 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:21:50.882 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:21:50.882 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:21:50.882 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:21:50.882 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:21:50.882 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 33554432, 2023-10-08T00:21:50.882 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:21:50.882 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:21:50.882 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:21:50.882 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:21:50.882 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:21:50.883 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:21:50.883 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:21:50.883 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:21:50.883 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:21:50.883 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:21:50.883 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:21:50.883 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:21:50.883 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:21:50.883 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:21:50.883 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:21:50.883 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:21:50.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.884 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:21:50.884 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:21:50.884 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:21:50.884 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:21:50.884 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:21:50.884 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:21:50.884 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:21:50.884 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:21:50.884 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:21:50.884 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:21:50.884 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:21:50.884 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:21:50.885 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2023-10-08T00:21:50.885 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2023-10-08T00:21:50.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.885 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:21:50.885 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:21:50.885 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2023-10-08T00:21:50.885 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:50.885 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:50.885 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:50.885 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:21:50.885 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2023-10-08T00:21:50.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.886 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:21:50.886 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:21:50.886 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:21:50.886 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:21:50.886 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:21:50.886 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:21:50.886 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:21:50.886 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:21:50.886 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:21:50.886 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:21:50.886 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:21:50.886 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:21:50.887 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:21:50.887 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:50.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.887 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:50.887 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2023-10-08T00:21:50.887 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:21:50.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.887 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:50.887 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2023-10-08T00:21:50.887 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:21:50.887 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:50.888 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:50.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.888 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:21:50.888 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:50.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.888 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:50.888 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2023-10-08T00:21:50.888 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:21:50.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.888 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:50.888 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2023-10-08T00:21:50.888 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:21:50.889 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:50.889 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:50.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.889 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:21:50.889 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:50.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.889 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:50.889 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2023-10-08T00:21:50.889 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:21:50.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.889 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:50.890 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2023-10-08T00:21:50.890 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:21:50.890 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:50.890 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:50.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.890 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:21:50.890 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:50.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.890 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:50.890 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2023-10-08T00:21:50.890 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:21:50.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.891 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:50.891 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2023-10-08T00:21:50.891 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:21:50.891 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:50.891 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:50.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.891 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/86365", 2023-10-08T00:21:50.891 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/86365", 2023-10-08T00:21:50.891 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/86365", 2023-10-08T00:21:50.891 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/86365", 2023-10-08T00:21:50.891 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:21:50.892 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:21:50.892 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:21:50.892 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:50.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.892 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:21:50.892 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:21:50.892 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:21:50.892 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:21:50.892 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:21:50.892 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:21:50.892 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:21:50.893 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:21:50.893 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:21:50.893 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:21:50.893 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:21:50.893 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:21:50.893 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:21:50.893 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:50.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.893 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:50.893 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2023-10-08T00:21:50.893 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:21:50.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.894 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.894 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:50.894 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2023-10-08T00:21:50.894 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:21:50.894 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:50.894 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:50.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.894 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:21:50.894 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:50.894 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.894 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:50.894 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2023-10-08T00:21:50.895 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:21:50.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.895 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:50.895 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2023-10-08T00:21:50.895 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:21:50.895 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:50.895 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:50.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.895 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:21:50.895 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:50.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.896 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:50.896 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2023-10-08T00:21:50.896 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:21:50.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.896 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:50.896 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2023-10-08T00:21:50.896 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:21:50.896 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:50.896 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:50.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.896 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:21:50.896 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:50.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.897 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:50.897 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2023-10-08T00:21:50.897 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:21:50.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.897 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:50.897 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2023-10-08T00:21:50.897 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:21:50.897 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:50.897 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:50.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.898 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/86366", 2023-10-08T00:21:50.898 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/86366", 2023-10-08T00:21:50.898 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/86366", 2023-10-08T00:21:50.898 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/86366", 2023-10-08T00:21:50.898 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:21:50.898 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:21:50.898 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:21:50.898 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:50.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.898 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:21:50.898 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:21:50.899 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:21:50.899 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:21:50.899 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:21:50.899 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:21:50.899 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:21:50.899 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:21:50.899 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:21:50.899 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:21:50.899 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:21:50.899 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:21:50.899 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:21:50.899 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:50.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.900 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:50.900 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2023-10-08T00:21:50.900 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:21:50.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.900 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:50.900 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2023-10-08T00:21:50.900 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:21:50.900 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:50.900 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:50.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.900 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:21:50.901 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:50.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.901 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:50.901 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2023-10-08T00:21:50.901 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:21:50.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.901 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:50.901 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2023-10-08T00:21:50.901 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:21:50.901 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:50.901 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:50.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.902 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:21:50.902 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:50.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.902 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:50.902 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2023-10-08T00:21:50.902 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:21:50.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.902 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:50.902 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2023-10-08T00:21:50.903 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:21:50.903 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:50.903 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:50.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.903 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:21:50.903 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:50.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.903 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:50.903 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2023-10-08T00:21:50.903 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:21:50.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.904 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:50.904 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2023-10-08T00:21:50.904 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:21:50.904 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:50.904 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:50.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.904 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6817/86382", 2023-10-08T00:21:50.904 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/86382", 2023-10-08T00:21:50.904 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/86382", 2023-10-08T00:21:50.904 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/86382", 2023-10-08T00:21:50.904 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:21:50.904 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:21:50.904 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:21:50.905 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:50.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.905 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:21:50.905 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:21:50.905 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:21:50.905 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:21:50.905 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:21:50.905 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:21:50.905 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:21:50.905 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:21:50.905 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:21:50.906 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 186, 2023-10-08T00:21:50.906 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:21:50.906 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:21:50.906 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:21:50.906 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:50.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.906 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:50.906 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2023-10-08T00:21:50.906 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:21:50.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.906 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:50.907 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2023-10-08T00:21:50.907 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:21:50.907 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:50.907 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:50.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.907 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:21:50.907 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:50.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.907 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:50.907 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2023-10-08T00:21:50.907 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:21:50.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.908 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:50.908 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2023-10-08T00:21:50.908 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:21:50.908 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:50.908 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:50.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.908 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:21:50.908 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:50.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.908 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:50.908 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2023-10-08T00:21:50.909 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:21:50.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.909 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:50.909 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2023-10-08T00:21:50.909 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:21:50.909 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:50.909 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:50.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.909 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:21:50.909 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:50.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.910 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:50.910 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2023-10-08T00:21:50.910 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:21:50.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.910 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:50.910 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2023-10-08T00:21:50.910 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:21:50.910 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:50.910 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:50.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.910 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6820/86393", 2023-10-08T00:21:50.911 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/86393", 2023-10-08T00:21:50.911 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/86393", 2023-10-08T00:21:50.911 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/86393", 2023-10-08T00:21:50.911 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:21:50.911 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:21:50.911 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:21:50.911 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:50.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.911 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:21:50.911 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:21:50.911 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:21:50.912 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:21:50.912 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:21:50.912 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:21:50.912 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:21:50.912 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:21:50.912 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:21:50.912 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:21:50.912 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:21:50.912 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:21:50.912 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:21:50.912 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:50.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.913 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:50.913 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6800", 2023-10-08T00:21:50.913 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:21:50.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.913 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:50.913 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6801", 2023-10-08T00:21:50.913 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:21:50.913 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:50.913 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:50.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.913 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:21:50.914 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:50.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.914 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:50.914 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6802", 2023-10-08T00:21:50.914 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:21:50.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.914 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:50.914 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6803", 2023-10-08T00:21:50.914 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:21:50.914 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:50.914 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:50.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.915 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:21:50.915 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:50.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.915 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:50.915 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6806", 2023-10-08T00:21:50.915 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:21:50.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.915 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:50.915 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6807", 2023-10-08T00:21:50.915 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:21:50.916 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:50.916 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:50.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.916 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:21:50.916 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:50.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.916 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:50.916 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6804", 2023-10-08T00:21:50.916 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:21:50.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.916 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:50.917 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6805", 2023-10-08T00:21:50.917 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:21:50.917 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:50.917 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:50.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.917 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6801/86570", 2023-10-08T00:21:50.917 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6803/86570", 2023-10-08T00:21:50.917 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6807/86570", 2023-10-08T00:21:50.917 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6805/86570", 2023-10-08T00:21:50.917 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:21:50.917 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:21:50.917 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:21:50.917 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:50.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.918 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:21:50.918 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:21:50.918 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:21:50.918 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:21:50.918 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:21:50.918 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:21:50.918 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:21:50.918 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:21:50.918 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:21:50.919 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:21:50.919 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:21:50.919 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:21:50.919 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:21:50.919 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:50.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.919 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:50.919 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6816", 2023-10-08T00:21:50.919 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:21:50.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.919 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:50.919 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6818", 2023-10-08T00:21:50.920 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:21:50.920 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:50.920 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:50.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.920 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:21:50.920 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:50.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.920 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:50.920 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6820", 2023-10-08T00:21:50.920 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:21:50.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.921 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:50.921 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6821", 2023-10-08T00:21:50.921 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:21:50.921 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:50.921 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:50.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.921 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:21:50.921 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:50.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.921 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:50.921 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6826", 2023-10-08T00:21:50.921 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:21:50.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.922 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:50.922 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6828", 2023-10-08T00:21:50.922 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:21:50.922 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:50.922 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:50.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.922 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:21:50.922 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:50.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.922 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:50.923 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6823", 2023-10-08T00:21:50.923 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:21:50.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.923 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:50.923 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6825", 2023-10-08T00:21:50.923 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:21:50.923 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:50.923 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:50.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.923 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6818/86608", 2023-10-08T00:21:50.923 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6821/86608", 2023-10-08T00:21:50.924 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6828/86608", 2023-10-08T00:21:50.924 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6825/86608", 2023-10-08T00:21:50.924 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:21:50.924 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:21:50.924 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:21:50.924 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:50.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.924 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:21:50.924 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:21:50.924 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:21:50.924 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:21:50.924 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:21:50.925 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:21:50.925 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:21:50.925 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:21:50.925 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:21:50.925 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:21:50.925 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:21:50.925 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:21:50.925 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:21:50.925 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:50.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.925 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:50.926 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6808", 2023-10-08T00:21:50.926 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:21:50.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.926 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:50.926 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6809", 2023-10-08T00:21:50.926 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:21:50.926 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:50.926 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:50.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.926 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:21:50.926 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:50.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.927 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:50.927 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6810", 2023-10-08T00:21:50.927 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:21:50.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.927 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:50.927 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6811", 2023-10-08T00:21:50.927 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:21:50.927 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:50.927 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:50.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.928 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:21:50.928 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:50.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.928 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:50.928 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6814", 2023-10-08T00:21:50.928 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:21:50.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.928 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:50.928 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6815", 2023-10-08T00:21:50.928 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:21:50.929 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:50.929 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:50.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.929 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:21:50.929 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:50.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.929 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:50.929 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6812", 2023-10-08T00:21:50.929 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:21:50.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.929 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:50.930 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6813", 2023-10-08T00:21:50.930 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:21:50.930 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:50.930 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:50.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.930 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6809/86571", 2023-10-08T00:21:50.930 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6811/86571", 2023-10-08T00:21:50.930 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6815/86571", 2023-10-08T00:21:50.930 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6813/86571", 2023-10-08T00:21:50.930 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:21:50.930 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:21:50.930 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:21:50.931 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:50.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.931 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:21:50.931 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:21:50.931 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:21:50.931 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:21:50.931 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:21:50.931 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:21:50.931 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:21:50.931 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:21:50.931 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:21:50.932 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:21:50.932 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:21:50.932 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:21:50.932 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:21:50.932 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:50.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.932 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:50.932 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6817", 2023-10-08T00:21:50.932 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:21:50.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.932 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:50.933 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6819", 2023-10-08T00:21:50.933 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:21:50.933 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:50.933 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:50.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.933 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:21:50.933 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:50.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.933 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:50.933 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6822", 2023-10-08T00:21:50.933 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:21:50.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.934 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:50.934 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6824", 2023-10-08T00:21:50.934 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:21:50.934 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:50.934 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:50.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.934 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:21:50.934 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:50.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.934 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:50.934 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6830", 2023-10-08T00:21:50.935 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:21:50.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.935 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:50.935 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6831", 2023-10-08T00:21:50.935 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:21:50.935 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:50.935 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:50.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.935 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:21:50.935 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:50.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.936 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:50.936 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6827", 2023-10-08T00:21:50.936 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:21:50.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.936 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:50.936 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6829", 2023-10-08T00:21:50.936 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:21:50.936 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:50.936 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:50.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.936 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6819/86612", 2023-10-08T00:21:50.937 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6824/86612", 2023-10-08T00:21:50.937 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6831/86612", 2023-10-08T00:21:50.937 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6829/86612", 2023-10-08T00:21:50.937 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:21:50.937 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:21:50.937 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:21:50.937 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:50.937 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:50.937 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:21:50.937 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2023-10-08T00:21:50.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.937 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:21:50.938 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:21:50.938 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:21:50.938 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:21:50.938 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:21:50.938 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:21:50.938 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.905615+0000", 2023-10-08T00:21:50.938 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:21:50.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.938 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:21:50.938 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:21:50.938 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:21:50.939 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:21:50.939 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:21:50.939 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:21:50.939 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.880709+0000", 2023-10-08T00:21:50.939 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:21:50.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.939 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:21:50.939 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:21:50.939 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:21:50.939 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:21:50.939 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:21:50.940 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:21:50.940 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.874704+0000", 2023-10-08T00:21:50.940 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:21:50.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.940 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:21:50.940 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:21:50.940 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:21:50.940 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:21:50.940 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:21:50.940 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:21:50.940 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.899604+0000", 2023-10-08T00:21:50.941 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:21:50.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.941 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:21:50.941 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:21:50.941 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:21:50.941 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:21:50.941 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:21:50.941 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:21:50.941 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.844371+0000", 2023-10-08T00:21:50.941 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:21:50.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.942 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:21:50.942 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:21:50.942 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:21:50.942 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:21:50.942 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:21:50.942 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:21:50.942 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.794810+0000", 2023-10-08T00:21:50.942 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:21:50.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.942 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:21:50.943 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:21:50.943 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:21:50.943 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:21:50.943 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:21:50.943 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:21:50.943 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.859474+0000", 2023-10-08T00:21:50.943 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:21:50.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.943 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:21:50.943 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:21:50.943 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:21:50.944 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:21:50.944 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:21:50.944 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:21:50.944 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.893130+0000", 2023-10-08T00:21:50.944 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:21:50.944 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:50.944 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:21:50.944 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2023-10-08T00:21:50.944 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [ 2023-10-08T00:21:50.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.944 INFO:teuthology.orchestra.run.smithi099.stdout: "pgid": "23.18", 2023-10-08T00:21:50.944 INFO:teuthology.orchestra.run.smithi099.stdout: "mappings": [ 2023-10-08T00:21:50.945 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:50.945 INFO:teuthology.orchestra.run.smithi099.stdout: "from": 5, 2023-10-08T00:21:50.945 INFO:teuthology.orchestra.run.smithi099.stdout: "to": 3 2023-10-08T00:21:50.945 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:50.945 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:50.945 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:50.945 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:21:50.945 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2023-10-08T00:21:50.946 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2023-10-08T00:21:50.946 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2023-10-08T00:21:50.946 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2023-10-08T00:21:50.946 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2023-10-08T00:21:50.946 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2023-10-08T00:21:50.946 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2023-10-08T00:21:50.946 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2023-10-08T00:21:50.946 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2023-10-08T00:21:50.947 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2023-10-08T00:21:50.947 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2023-10-08T00:21:50.947 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:50.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:50.947 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2023-10-08T00:21:50.947 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2023-10-08T00:21:50.947 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2023-10-08T00:21:50.947 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2023-10-08T00:21:50.947 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2023-10-08T00:21:50.947 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2023-10-08T00:21:50.947 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2023-10-08T00:21:50.948 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2023-10-08T00:21:50.948 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2023-10-08T00:21:50.948 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2023-10-08T00:21:50.948 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2023-10-08T00:21:50.948 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:50.948 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:21:50.950 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.862+0000 7f3716b15700 1 -- 172.21.15.99:0/3402763285 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f36fc066f30 msgr2=0x7f36fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:50.950 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.862+0000 7f3716b15700 1 --2- 172.21.15.99:0/3402763285 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f36fc066f30 0x7f36fc0693e0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f370c001840 tx=0x7f370c009040 comp rx=0 tx=0).stop 2023-10-08T00:21:50.950 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.862+0000 7f3716b15700 1 -- 172.21.15.99:0/3402763285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3710159e60 msgr2=0x7f371016b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:50.950 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.862+0000 7f3716b15700 1 --2- 172.21.15.99:0/3402763285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3710159e60 0x7f371016b1f0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f3710089730 tx=0x7f3700017000 comp rx=0 tx=0).stop 2023-10-08T00:21:50.950 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.862+0000 7f3716b15700 1 -- 172.21.15.99:0/3402763285 shutdown_connections 2023-10-08T00:21:50.950 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.862+0000 7f3716b15700 1 --2- 172.21.15.99:0/3402763285 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f36fc066f30 0x7f36fc0693e0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:50.951 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.862+0000 7f3716b15700 1 --2- 172.21.15.99:0/3402763285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f371015a230 0x7f371016de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:50.951 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.862+0000 7f3716b15700 1 --2- 172.21.15.99:0/3402763285 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f37100cffa0 0x7f37100c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:50.951 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.862+0000 7f3716b15700 1 --2- 172.21.15.99:0/3402763285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3710159e60 0x7f371016b1f0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:50.951 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.862+0000 7f3716b15700 1 -- 172.21.15.99:0/3402763285 >> 172.21.15.99:0/3402763285 conn(0x7f37100d1a50 msgr2=0x7f371015cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:50.951 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.862+0000 7f3716b15700 1 -- 172.21.15.99:0/3402763285 shutdown_connections 2023-10-08T00:21:50.951 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:50.862+0000 7f3716b15700 1 -- 172.21.15.99:0/3402763285 wait complete. 2023-10-08T00:21:50.952 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2023-10-08T00:21:50.952 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:21:50.952 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=16) 2023-10-08T00:21:51.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T00:21:51.200 INFO:teuthology.orchestra.run.smithi099.stderr:dd: fdatasync failed for 'large_file_b': No space left on device 2023-10-08T00:21:51.201 INFO:teuthology.orchestra.run.smithi099.stderr:50+0 records in 2023-10-08T00:21:51.201 INFO:teuthology.orchestra.run.smithi099.stderr:50+0 records out 2023-10-08T00:21:51.201 INFO:teuthology.orchestra.run.smithi099.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.228708 s, 229 MB/s 2023-10-08T00:21:51.201 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-08T00:21:51.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.317+0000 7ff8c46d5700 1 Processor -- start 2023-10-08T00:21:51.318 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.317+0000 7ff8c46d5700 1 -- start start 2023-10-08T00:21:51.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.318+0000 7ff8c46d5700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff8bc1568d0 0x7ff8bc069e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:51.320 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.318+0000 7ff8c46d5700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff8bc0c87a0 0x7ff8bc06a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:51.321 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.318+0000 7ff8c46d5700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8bc067ba0 con 0x7ff8bc0ad9b0 2023-10-08T00:21:51.321 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.318+0000 7ff8c46d5700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8bc15aa90 con 0x7ff8bc1568d0 2023-10-08T00:21:51.321 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.318+0000 7ff8c46d5700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8bc067870 con 0x7ff8bc0c87a0 2023-10-08T00:21:51.321 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.318+0000 7ff8c2471700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff8bc1568d0 0x7ff8bc069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:51.321 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.318+0000 7ff8c2471700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff8bc1568d0 0x7ff8bc069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41748/0 (socket says 172.21.15.99:41748) 2023-10-08T00:21:51.321 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.318+0000 7ff8c2471700 1 -- 172.21.15.99:0/1487956949 learned_addr learned my addr 172.21.15.99:0/1487956949 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:51.321 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.319+0000 7ff8c2c72700 1 --2- 172.21.15.99:0/1487956949 >> v2:172.21.15.191:3300/0 conn(0x7ff8bc0c87a0 0x7ff8bc06a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:51.321 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.319+0000 7ff8c2471700 1 -- 172.21.15.99:0/1487956949 >> v2:172.21.15.191:3300/0 conn(0x7ff8bc0c87a0 msgr2=0x7ff8bc06a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:51.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.319+0000 7ff8c2471700 1 --2- 172.21.15.99:0/1487956949 >> v2:172.21.15.191:3300/0 conn(0x7ff8bc0c87a0 0x7ff8bc06a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:51.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.319+0000 7ff8c2471700 1 -- 172.21.15.99:0/1487956949 >> v1:172.21.15.191:6789/0 conn(0x7ff8bc0ad9b0 legacy=0x7ff8bc0c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:51.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.319+0000 7ff8c2471700 1 -- 172.21.15.99:0/1487956949 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8bc0c8ca0 con 0x7ff8bc1568d0 2023-10-08T00:21:51.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.319+0000 7ff8c2471700 1 --2- 172.21.15.99:0/1487956949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff8bc1568d0 0x7ff8bc069e20 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc0b0370 tx=0x7ff8ac009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fea07147345c2c7d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:51.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.319+0000 7ff8c146f700 1 -- 172.21.15.99:0/1487956949 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac00e070 con 0x7ff8bc1568d0 2023-10-08T00:21:51.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.319+0000 7ff8c146f700 1 -- 172.21.15.99:0/1487956949 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8ac00ba50 con 0x7ff8bc1568d0 2023-10-08T00:21:51.322 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.319+0000 7ff8c146f700 1 -- 172.21.15.99:0/1487956949 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac0092e0 con 0x7ff8bc1568d0 2023-10-08T00:21:51.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.319+0000 7ff8c46d5700 1 -- 172.21.15.99:0/1487956949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff8bc1568d0 msgr2=0x7ff8bc069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:51.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.320+0000 7ff8c46d5700 1 --2- 172.21.15.99:0/1487956949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff8bc1568d0 0x7ff8bc069e20 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc0b0370 tx=0x7ff8ac009840 comp rx=0 tx=0).stop 2023-10-08T00:21:51.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.320+0000 7ff8c46d5700 1 -- 172.21.15.99:0/1487956949 shutdown_connections 2023-10-08T00:21:51.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.320+0000 7ff8c46d5700 1 --2- 172.21.15.99:0/1487956949 >> v2:172.21.15.191:3300/0 conn(0x7ff8bc0c87a0 0x7ff8bc06a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:51.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.320+0000 7ff8c46d5700 1 --2- 172.21.15.99:0/1487956949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff8bc1568d0 0x7ff8bc069e20 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:51.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.320+0000 7ff8c46d5700 1 -- 172.21.15.99:0/1487956949 >> 172.21.15.99:0/1487956949 conn(0x7ff8bc0cda50 msgr2=0x7ff8bc0cde50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:51.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.320+0000 7ff8c46d5700 1 -- 172.21.15.99:0/1487956949 shutdown_connections 2023-10-08T00:21:51.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.320+0000 7ff8c46d5700 1 -- 172.21.15.99:0/1487956949 wait complete. 2023-10-08T00:21:51.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.321+0000 7ff8c46d5700 1 Processor -- start 2023-10-08T00:21:51.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.321+0000 7ff8c46d5700 1 -- start start 2023-10-08T00:21:51.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.321+0000 7ff8c46d5700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8bc0ad9b0 0x7ff8bc0c1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:51.329 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.321+0000 7ff8c46d5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff8bc0c87a0 0x7ff8bc154990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:51.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.321+0000 7ff8c46d5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8bc1568d0 0x7ff8bc0cc820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:51.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.321+0000 7ff8c46d5700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8bc0c2230 con 0x7ff8bc0c87a0 2023-10-08T00:21:51.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.321+0000 7ff8c46d5700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8bc0c6390 con 0x7ff8bc0ad9b0 2023-10-08T00:21:51.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.321+0000 7ff8c46d5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8bc0c6670 con 0x7ff8bc1568d0 2023-10-08T00:21:51.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.322+0000 7ff8c1c70700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff8bc0c87a0 0x7ff8bc154990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:51.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.322+0000 7ff8c1c70700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff8bc0c87a0 0x7ff8bc154990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41752/0 (socket says 172.21.15.99:41752) 2023-10-08T00:21:51.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.322+0000 7ff8c1c70700 1 -- 172.21.15.99:0/1180740382 learned_addr learned my addr 172.21.15.99:0/1180740382 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:51.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.322+0000 7ff8c2471700 1 --2- 172.21.15.99:0/1180740382 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8bc0ad9b0 0x7ff8bc0c1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:51.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.322+0000 7ff8c2c72700 1 --2- 172.21.15.99:0/1180740382 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8bc1568d0 0x7ff8bc0cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:51.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.322+0000 7ff8c1c70700 1 -- 172.21.15.99:0/1180740382 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8bc1568d0 msgr2=0x7ff8bc0cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:51.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.322+0000 7ff8c1c70700 1 --2- 172.21.15.99:0/1180740382 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8bc1568d0 0x7ff8bc0cc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:51.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.322+0000 7ff8c1c70700 1 -- 172.21.15.99:0/1180740382 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8bc0ad9b0 msgr2=0x7ff8bc0c1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:51.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.322+0000 7ff8c1c70700 1 --2- 172.21.15.99:0/1180740382 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8bc0ad9b0 0x7ff8bc0c1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:51.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.322+0000 7ff8c1c70700 1 -- 172.21.15.99:0/1180740382 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8ac009ae0 con 0x7ff8bc0c87a0 2023-10-08T00:21:51.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.322+0000 7ff8c2471700 1 --2- 172.21.15.99:0/1180740382 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8bc0ad9b0 0x7ff8bc0c1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:21:51.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.322+0000 7ff8c1c70700 1 --2- 172.21.15.99:0/1180740382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff8bc0c87a0 0x7ff8bc154990 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7ff8b000c590 tx=0x7ff8b0007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:51.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.322+0000 7ff8ba7fc700 1 -- 172.21.15.99:0/1180740382 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b0011590 con 0x7ff8bc0c87a0 2023-10-08T00:21:51.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.322+0000 7ff8ba7fc700 1 -- 172.21.15.99:0/1180740382 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8b0011710 con 0x7ff8bc0c87a0 2023-10-08T00:21:51.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.322+0000 7ff8ba7fc700 1 -- 172.21.15.99:0/1180740382 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b0011ad0 con 0x7ff8bc0c87a0 2023-10-08T00:21:51.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.322+0000 7ff8c46d5700 1 -- 172.21.15.99:0/1180740382 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8bc0c6950 con 0x7ff8bc0c87a0 2023-10-08T00:21:51.332 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.322+0000 7ff8c46d5700 1 -- 172.21.15.99:0/1180740382 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8bc0c6f10 con 0x7ff8bc0c87a0 2023-10-08T00:21:51.333 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.323+0000 7ff8c46d5700 1 -- 172.21.15.99:0/1180740382 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8bc04c3c0 con 0x7ff8bc0c87a0 2023-10-08T00:21:51.333 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.324+0000 7ff8ba7fc700 1 -- 172.21.15.99:0/1180740382 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff8b001a360 con 0x7ff8bc0c87a0 2023-10-08T00:21:51.333 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.324+0000 7ff8ba7fc700 1 --2- 172.21.15.99:0/1180740382 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff8a8066f30 0x7ff8a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:51.333 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.324+0000 7ff8ba7fc700 1 -- 172.21.15.99:0/1180740382 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5734+0+0 (secure 0 0 0) 0x7ff8b0083550 con 0x7ff8bc0c87a0 2023-10-08T00:21:51.333 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.324+0000 7ff8ba7fc700 1 -- 172.21.15.99:0/1180740382 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=189}) v3 -- 0x7ff8a806eab0 con 0x7ff8bc0c87a0 2023-10-08T00:21:51.333 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.324+0000 7ff8c2471700 1 --2- 172.21.15.99:0/1180740382 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff8a8066f30 0x7ff8a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:51.333 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.325+0000 7ff8c2471700 1 --2- 172.21.15.99:0/1180740382 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff8a8066f30 0x7ff8a80693e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac009880 tx=0x7ff8ac018000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:51.334 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.328+0000 7ff8ba7fc700 1 -- 172.21.15.99:0/1180740382 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff8b00507a0 con 0x7ff8bc0c87a0 2023-10-08T00:21:51.513 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.512+0000 7ff8c46d5700 1 -- 172.21.15.99:0/1180740382 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7ff8bc0c8f80 con 0x7ff8bc0c87a0 2023-10-08T00:21:51.513 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.513+0000 7ff8ba7fc700 1 -- 172.21.15.99:0/1180740382 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v188) v1 ==== 81+0+32493 (secure 0 0 0) 0x7ff8b0017020 con 0x7ff8bc0c87a0 2023-10-08T00:21:51.514 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:21:51.514 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:21:51.514 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 188, 2023-10-08T00:21:51.514 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", 2023-10-08T00:21:51.514 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2023-10-08T00:10:36.303601+0000", 2023-10-08T00:21:51.514 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2023-10-08T00:21:46.168808+0000", 2023-10-08T00:21:51.515 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2023-10-08T00:10:41.709171+0000", 2023-10-08T00:21:51.515 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2023-10-08T00:10:38.690404+0000", 2023-10-08T00:21:51.515 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-08T00:21:51.515 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2023-10-08T00:21:51.515 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2023-10-08T00:21:51.515 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2023-10-08T00:21:51.515 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2023-10-08T00:21:51.515 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2023-10-08T00:21:51.515 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2023-10-08T00:21:51.515 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:21:51.515 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2023-10-08T00:21:51.516 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2023-10-08T00:21:51.516 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-08T00:21:51.516 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-08T00:21:51.516 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2023-10-08T00:21:51.516 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 23, 2023-10-08T00:21:51.516 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2023-10-08T00:21:51.516 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2023-10-08T00:21:51.516 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "luminous", 2023-10-08T00:21:51.516 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "quincy", 2023-10-08T00:21:51.516 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2023-10-08T00:21:51.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.516 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2023-10-08T00:21:51.517 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2023-10-08T00:21:51.517 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:10:42.063344+0000", 2023-10-08T00:21:51.517 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:21:51.517 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:21:51.517 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:21:51.517 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:21:51.517 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:21:51.517 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:21:51.517 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:21:51.517 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:21:51.517 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:21:51.517 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:21:51.518 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:21:51.518 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:21:51.518 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2023-10-08T00:21:51.518 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2023-10-08T00:21:51.518 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2023-10-08T00:21:51.518 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2023-10-08T00:21:51.518 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2023-10-08T00:21:51.518 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:21:51.518 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:21:51.518 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:21:51.518 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:21:51.519 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:21:51.519 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:21:51.519 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:21:51.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.519 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "112", 2023-10-08T00:21:51.519 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:21:51.519 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:21:51.519 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:21:51.519 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:21:51.519 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:21:51.519 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:21:51.520 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:21:51.520 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:21:51.520 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:21:51.520 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:21:51.520 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:21:51.520 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:21:51.520 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:21:51.520 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:21:51.520 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:21:51.520 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:21:51.520 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:21:51.521 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:21:51.521 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:21:51.521 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:21:51.521 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:21:51.521 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:21:51.521 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:21:51.521 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:21:51.521 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:21:51.521 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:21:51.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.521 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:21:51.521 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:21:51.522 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:21:51.522 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:21:51.522 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:21:51.522 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:21:51.522 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:21:51.522 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:21:51.523 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:21:51.523 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:21:51.523 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:21:51.523 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:21:51.523 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2023-10-08T00:21:51.523 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2023-10-08T00:21:51.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.523 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:21:51.523 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2023-10-08T00:21:51.523 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:51.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.524 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 22, 2023-10-08T00:21:51.524 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2023-10-08T00:21:51.524 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:21:27.984574+0000", 2023-10-08T00:21:51.524 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:21:51.524 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:21:51.524 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:21:51.524 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:21:51.524 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:21:51.524 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:21:51.524 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:21:51.524 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:21:51.525 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:21:51.525 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:21:51.525 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:21:51.525 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:21:51.525 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2023-10-08T00:21:51.525 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2023-10-08T00:21:51.525 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2023-10-08T00:21:51.525 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2023-10-08T00:21:51.525 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2023-10-08T00:21:51.525 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:21:51.525 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:21:51.525 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:21:51.526 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:21:51.526 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:21:51.526 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:21:51.526 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:21:51.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.526 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "183", 2023-10-08T00:21:51.526 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:21:51.526 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:21:51.526 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:21:51.526 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:21:51.527 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:21:51.527 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:21:51.527 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:21:51.527 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:21:51.527 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:21:51.527 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:21:51.527 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:21:51.527 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:21:51.527 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:21:51.527 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:21:51.527 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:21:51.527 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:21:51.528 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:21:51.528 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:21:51.528 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:21:51.528 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:21:51.528 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:21:51.528 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:21:51.528 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:21:51.528 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:21:51.528 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:21:51.528 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:21:51.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.528 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:21:51.529 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:21:51.529 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:21:51.529 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:21:51.529 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:21:51.529 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:21:51.529 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:21:51.529 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:21:51.529 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:21:51.529 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:21:51.529 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:21:51.529 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:21:51.529 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2023-10-08T00:21:51.530 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2023-10-08T00:21:51.530 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2023-10-08T00:21:51.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.530 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:21:51.530 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:21:51.530 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2023-10-08T00:21:51.530 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:51.530 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:51.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.530 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 23, 2023-10-08T00:21:51.531 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2023-10-08T00:21:51.531 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:21:29.244620+0000", 2023-10-08T00:21:51.531 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1027, 2023-10-08T00:21:51.531 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full,full_quota", 2023-10-08T00:21:51.531 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:21:51.531 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:21:51.531 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:21:51.531 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:21:51.531 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:21:51.531 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:21:51.531 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:21:51.531 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:21:51.532 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:21:51.532 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:21:51.532 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2023-10-08T00:21:51.532 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2023-10-08T00:21:51.532 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2023-10-08T00:21:51.532 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2023-10-08T00:21:51.532 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2023-10-08T00:21:51.532 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:21:51.532 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:21:51.532 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:21:51.532 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:21:51.533 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:21:51.533 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:21:51.533 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:21:51.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.533 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "188", 2023-10-08T00:21:51.533 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:21:51.533 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:21:51.533 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:21:51.533 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:21:51.533 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:21:51.533 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:21:51.534 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:21:51.534 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:21:51.534 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:21:51.534 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 33554432, 2023-10-08T00:21:51.534 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:21:51.534 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:21:51.534 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:21:51.534 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:21:51.534 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:21:51.534 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:21:51.534 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:21:51.534 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:21:51.535 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:21:51.535 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:21:51.535 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:21:51.535 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:21:51.535 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:21:51.535 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:21:51.535 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:21:51.535 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:21:51.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.535 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:21:51.535 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:21:51.536 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:21:51.536 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:21:51.536 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:21:51.536 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:21:51.536 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:21:51.536 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:21:51.536 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:21:51.536 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:21:51.536 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:21:51.536 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:21:51.536 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2023-10-08T00:21:51.537 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2023-10-08T00:21:51.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.537 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:21:51.537 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:21:51.537 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2023-10-08T00:21:51.537 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:51.537 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:51.537 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:51.537 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:21:51.537 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2023-10-08T00:21:51.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.538 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:21:51.538 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:21:51.538 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:21:51.538 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:21:51.538 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:21:51.538 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:21:51.538 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:21:51.538 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:21:51.538 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:21:51.538 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:21:51.538 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:21:51.539 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:21:51.539 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:21:51.539 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:51.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.539 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:51.539 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2023-10-08T00:21:51.539 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:21:51.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.539 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:51.539 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2023-10-08T00:21:51.540 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:21:51.540 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:51.540 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:51.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.540 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:21:51.540 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:51.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.540 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:51.540 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2023-10-08T00:21:51.540 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:21:51.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.541 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:51.541 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2023-10-08T00:21:51.541 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:21:51.541 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:51.541 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:51.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.541 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:21:51.541 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:51.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.541 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:51.541 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2023-10-08T00:21:51.541 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:21:51.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.542 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:51.542 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2023-10-08T00:21:51.542 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:21:51.542 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:51.542 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:51.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.542 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:21:51.542 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:51.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.543 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:51.543 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2023-10-08T00:21:51.543 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:21:51.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.543 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:51.543 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2023-10-08T00:21:51.543 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:21:51.543 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:51.543 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:51.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.544 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/86365", 2023-10-08T00:21:51.544 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/86365", 2023-10-08T00:21:51.544 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/86365", 2023-10-08T00:21:51.544 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/86365", 2023-10-08T00:21:51.544 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:21:51.544 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:21:51.544 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:21:51.544 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:51.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.544 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:21:51.544 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:21:51.545 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:21:51.545 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:21:51.545 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:21:51.545 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:21:51.545 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:21:51.545 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:21:51.545 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:21:51.545 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:21:51.545 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:21:51.545 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:21:51.545 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:21:51.546 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:51.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.546 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:51.546 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2023-10-08T00:21:51.546 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:21:51.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.546 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:51.546 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2023-10-08T00:21:51.546 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:21:51.546 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:51.546 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:51.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.547 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:21:51.547 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:51.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.547 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:51.547 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2023-10-08T00:21:51.547 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:21:51.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.547 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:51.547 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2023-10-08T00:21:51.547 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:21:51.548 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:51.548 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:51.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.548 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:21:51.548 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:51.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.548 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:51.548 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2023-10-08T00:21:51.548 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:21:51.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.548 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:51.549 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2023-10-08T00:21:51.549 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:21:51.549 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:51.549 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:51.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.549 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:21:51.549 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:51.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.549 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:51.549 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2023-10-08T00:21:51.549 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:21:51.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.550 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:51.550 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2023-10-08T00:21:51.550 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:21:51.550 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:51.550 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:51.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.550 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/86366", 2023-10-08T00:21:51.550 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/86366", 2023-10-08T00:21:51.550 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/86366", 2023-10-08T00:21:51.550 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/86366", 2023-10-08T00:21:51.551 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:21:51.551 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:21:51.551 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:21:51.551 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:51.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.551 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:21:51.551 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:21:51.551 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:21:51.551 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:21:51.551 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:21:51.551 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:21:51.552 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:21:51.552 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:21:51.552 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:21:51.552 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:21:51.552 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:21:51.552 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:21:51.552 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:21:51.552 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:51.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.552 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:51.552 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2023-10-08T00:21:51.552 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:21:51.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.553 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:51.553 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2023-10-08T00:21:51.553 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:21:51.553 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:51.553 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:51.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.553 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:21:51.553 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:51.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.553 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:51.554 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2023-10-08T00:21:51.554 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:21:51.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.554 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:51.554 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2023-10-08T00:21:51.554 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:21:51.554 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:51.554 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:51.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.554 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:21:51.555 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:51.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.555 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:51.555 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2023-10-08T00:21:51.555 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:21:51.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.555 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:51.555 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2023-10-08T00:21:51.555 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:21:51.555 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:51.555 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:51.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.556 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:21:51.556 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:51.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.556 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:51.556 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2023-10-08T00:21:51.556 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:21:51.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.556 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:51.556 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2023-10-08T00:21:51.556 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:21:51.557 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:51.557 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:51.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.557 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6817/86382", 2023-10-08T00:21:51.557 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/86382", 2023-10-08T00:21:51.557 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/86382", 2023-10-08T00:21:51.557 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/86382", 2023-10-08T00:21:51.557 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:21:51.557 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:21:51.557 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:21:51.557 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:51.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.558 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:21:51.558 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:21:51.558 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:21:51.558 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:21:51.558 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:21:51.558 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:21:51.558 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:21:51.558 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:21:51.558 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:21:51.559 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 186, 2023-10-08T00:21:51.559 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:21:51.559 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:21:51.559 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:21:51.559 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:51.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.559 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:51.559 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2023-10-08T00:21:51.559 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:21:51.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.559 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:51.560 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2023-10-08T00:21:51.560 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:21:51.560 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:51.560 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:51.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.560 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:21:51.560 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:51.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.560 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:51.560 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2023-10-08T00:21:51.560 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:21:51.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.561 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:51.561 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2023-10-08T00:21:51.561 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:21:51.561 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:51.561 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:51.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.561 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:21:51.561 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:51.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.561 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:51.562 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2023-10-08T00:21:51.562 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:21:51.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.562 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:51.562 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2023-10-08T00:21:51.562 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:21:51.562 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:51.562 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:51.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.562 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:21:51.563 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:51.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.563 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:51.563 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2023-10-08T00:21:51.563 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:21:51.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.563 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:51.563 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2023-10-08T00:21:51.563 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:21:51.563 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:51.563 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:51.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.564 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6820/86393", 2023-10-08T00:21:51.564 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/86393", 2023-10-08T00:21:51.564 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/86393", 2023-10-08T00:21:51.564 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/86393", 2023-10-08T00:21:51.564 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:21:51.564 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:21:51.564 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:21:51.564 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:51.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.564 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:21:51.565 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:21:51.565 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:21:51.565 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:21:51.565 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:21:51.565 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:21:51.565 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:21:51.565 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:21:51.565 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:21:51.565 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:21:51.565 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:21:51.565 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:21:51.566 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:21:51.566 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:51.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.566 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:51.566 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6800", 2023-10-08T00:21:51.566 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:21:51.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.566 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:51.566 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6801", 2023-10-08T00:21:51.566 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:21:51.567 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:51.567 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:51.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.567 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:21:51.567 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:51.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.567 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:51.567 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6802", 2023-10-08T00:21:51.567 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:21:51.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.567 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:51.568 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6803", 2023-10-08T00:21:51.568 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:21:51.568 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:51.568 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:51.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.568 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:21:51.568 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:51.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.568 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:51.568 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6806", 2023-10-08T00:21:51.568 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:21:51.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.569 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:51.569 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6807", 2023-10-08T00:21:51.569 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:21:51.569 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:51.569 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:51.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.569 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:21:51.569 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:51.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.569 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:51.570 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6804", 2023-10-08T00:21:51.570 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:21:51.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.570 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:51.570 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6805", 2023-10-08T00:21:51.570 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:21:51.570 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:51.570 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:51.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.570 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6801/86570", 2023-10-08T00:21:51.571 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6803/86570", 2023-10-08T00:21:51.571 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6807/86570", 2023-10-08T00:21:51.571 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6805/86570", 2023-10-08T00:21:51.571 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:21:51.571 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:21:51.571 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:21:51.571 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:51.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.571 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:21:51.571 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:21:51.572 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:21:51.572 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:21:51.572 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:21:51.572 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:21:51.572 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:21:51.572 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:21:51.572 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:21:51.572 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:21:51.572 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:21:51.572 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:21:51.572 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:21:51.572 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:51.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.573 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:51.573 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6816", 2023-10-08T00:21:51.573 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:21:51.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.573 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:51.573 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6818", 2023-10-08T00:21:51.573 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:21:51.573 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:51.573 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:51.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.574 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:21:51.574 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:51.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.574 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:51.574 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6820", 2023-10-08T00:21:51.574 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:21:51.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.574 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:51.574 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6821", 2023-10-08T00:21:51.574 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:21:51.575 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:51.575 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:51.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.575 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:21:51.575 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:51.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.575 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:51.575 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6826", 2023-10-08T00:21:51.575 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:21:51.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.575 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:51.576 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6828", 2023-10-08T00:21:51.576 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:21:51.576 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:51.576 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:51.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.576 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:21:51.576 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:51.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.576 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:51.576 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6823", 2023-10-08T00:21:51.576 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:21:51.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.577 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:51.577 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6825", 2023-10-08T00:21:51.577 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:21:51.577 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:51.577 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:51.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.577 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6818/86608", 2023-10-08T00:21:51.577 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6821/86608", 2023-10-08T00:21:51.577 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6828/86608", 2023-10-08T00:21:51.577 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6825/86608", 2023-10-08T00:21:51.577 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:21:51.578 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:21:51.578 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:21:51.578 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:51.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.578 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:21:51.578 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:21:51.578 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:21:51.578 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:21:51.578 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:21:51.578 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:21:51.579 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:21:51.579 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:21:51.579 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:21:51.579 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:21:51.579 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:21:51.579 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:21:51.579 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:21:51.579 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:51.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.579 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:51.579 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6808", 2023-10-08T00:21:51.579 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:21:51.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.580 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:51.580 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6809", 2023-10-08T00:21:51.580 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:21:51.580 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:51.580 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:51.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.580 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:21:51.580 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:51.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.580 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:51.581 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6810", 2023-10-08T00:21:51.581 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:21:51.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.581 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:51.581 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6811", 2023-10-08T00:21:51.581 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:21:51.581 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:51.581 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:51.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.581 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:21:51.582 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:51.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.582 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:51.582 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6814", 2023-10-08T00:21:51.582 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:21:51.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.582 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:51.582 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6815", 2023-10-08T00:21:51.582 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:21:51.582 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:51.582 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:51.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.583 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:21:51.583 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:51.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.583 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:51.583 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6812", 2023-10-08T00:21:51.583 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:21:51.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.583 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:51.583 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6813", 2023-10-08T00:21:51.584 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:21:51.584 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:51.584 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:51.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.584 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6809/86571", 2023-10-08T00:21:51.584 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6811/86571", 2023-10-08T00:21:51.584 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6815/86571", 2023-10-08T00:21:51.584 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6813/86571", 2023-10-08T00:21:51.584 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:21:51.584 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:21:51.584 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:21:51.584 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:51.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.585 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:21:51.585 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:21:51.585 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:21:51.585 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:21:51.585 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:21:51.585 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:21:51.585 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:21:51.585 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:21:51.585 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:21:51.586 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:21:51.586 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:21:51.586 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:21:51.586 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:21:51.586 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:51.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.586 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:51.586 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6817", 2023-10-08T00:21:51.586 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:21:51.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.586 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:51.587 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6819", 2023-10-08T00:21:51.587 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:21:51.587 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:51.587 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:51.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.587 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:21:51.587 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:51.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.587 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:51.587 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6822", 2023-10-08T00:21:51.587 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:21:51.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.588 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:51.588 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6824", 2023-10-08T00:21:51.588 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:21:51.588 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:51.588 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:51.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.588 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:21:51.588 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:51.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.588 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:51.589 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6830", 2023-10-08T00:21:51.589 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:21:51.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.589 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:51.589 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6831", 2023-10-08T00:21:51.589 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:21:51.589 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:51.589 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:51.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.589 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:21:51.589 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:51.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.590 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:51.590 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6827", 2023-10-08T00:21:51.590 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:21:51.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.590 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:51.590 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6829", 2023-10-08T00:21:51.590 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:21:51.590 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:51.590 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:51.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.591 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6819/86612", 2023-10-08T00:21:51.591 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6824/86612", 2023-10-08T00:21:51.591 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6831/86612", 2023-10-08T00:21:51.591 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6829/86612", 2023-10-08T00:21:51.591 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:21:51.591 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:21:51.591 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:21:51.591 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:51.591 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:51.591 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:21:51.591 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2023-10-08T00:21:51.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.592 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:21:51.592 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:21:51.592 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:21:51.592 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:21:51.592 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:21:51.592 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:21:51.592 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.905615+0000", 2023-10-08T00:21:51.592 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:21:51.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.593 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:21:51.593 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:21:51.593 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:21:51.593 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:21:51.593 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:21:51.593 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:21:51.593 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.880709+0000", 2023-10-08T00:21:51.593 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:21:51.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.593 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.593 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:21:51.593 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:21:51.594 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:21:51.594 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:21:51.594 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:21:51.594 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:21:51.594 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.874704+0000", 2023-10-08T00:21:51.594 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:21:51.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.594 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:21:51.594 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:21:51.594 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:21:51.594 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:21:51.595 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:21:51.595 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:21:51.595 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.899604+0000", 2023-10-08T00:21:51.595 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:21:51.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.595 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:21:51.595 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:21:51.595 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:21:51.595 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:21:51.595 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:21:51.595 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:21:51.596 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.844371+0000", 2023-10-08T00:21:51.596 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:21:51.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.596 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:21:51.596 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:21:51.596 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:21:51.596 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:21:51.596 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:21:51.596 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:21:51.596 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.794810+0000", 2023-10-08T00:21:51.596 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:21:51.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.597 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.597 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:21:51.597 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:21:51.597 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:21:51.597 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:21:51.597 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:21:51.597 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:21:51.597 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.859474+0000", 2023-10-08T00:21:51.597 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:21:51.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.597 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.598 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:21:51.598 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:21:51.598 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:21:51.598 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:21:51.598 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:21:51.598 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:21:51.598 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.893130+0000", 2023-10-08T00:21:51.598 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:21:51.598 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:51.598 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:21:51.598 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2023-10-08T00:21:51.598 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [ 2023-10-08T00:21:51.599 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.599 INFO:teuthology.orchestra.run.smithi099.stdout: "pgid": "23.18", 2023-10-08T00:21:51.599 INFO:teuthology.orchestra.run.smithi099.stdout: "mappings": [ 2023-10-08T00:21:51.599 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:51.599 INFO:teuthology.orchestra.run.smithi099.stdout: "from": 5, 2023-10-08T00:21:51.599 INFO:teuthology.orchestra.run.smithi099.stdout: "to": 3 2023-10-08T00:21:51.599 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:51.599 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:51.599 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:51.599 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:21:51.599 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2023-10-08T00:21:51.599 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2023-10-08T00:21:51.600 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2023-10-08T00:21:51.600 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2023-10-08T00:21:51.600 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2023-10-08T00:21:51.600 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2023-10-08T00:21:51.600 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2023-10-08T00:21:51.600 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2023-10-08T00:21:51.600 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2023-10-08T00:21:51.600 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2023-10-08T00:21:51.600 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2023-10-08T00:21:51.600 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:51.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:51.600 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2023-10-08T00:21:51.601 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2023-10-08T00:21:51.601 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2023-10-08T00:21:51.601 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2023-10-08T00:21:51.601 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2023-10-08T00:21:51.601 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2023-10-08T00:21:51.601 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2023-10-08T00:21:51.601 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2023-10-08T00:21:51.601 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2023-10-08T00:21:51.601 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2023-10-08T00:21:51.601 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2023-10-08T00:21:51.601 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:51.601 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:21:51.603 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.515+0000 7ff8c46d5700 1 -- 172.21.15.99:0/1180740382 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff8a8066f30 msgr2=0x7ff8a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:51.603 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.515+0000 7ff8c46d5700 1 --2- 172.21.15.99:0/1180740382 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff8a8066f30 0x7ff8a80693e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac009880 tx=0x7ff8ac018000 comp rx=0 tx=0).stop 2023-10-08T00:21:51.603 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.515+0000 7ff8c46d5700 1 -- 172.21.15.99:0/1180740382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff8bc0c87a0 msgr2=0x7ff8bc154990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:51.604 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.515+0000 7ff8c46d5700 1 --2- 172.21.15.99:0/1180740382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff8bc0c87a0 0x7ff8bc154990 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7ff8b000c590 tx=0x7ff8b0007f50 comp rx=0 tx=0).stop 2023-10-08T00:21:51.604 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.515+0000 7ff8c46d5700 1 -- 172.21.15.99:0/1180740382 shutdown_connections 2023-10-08T00:21:51.604 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.515+0000 7ff8c46d5700 1 --2- 172.21.15.99:0/1180740382 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff8a8066f30 0x7ff8a80693e0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:51.604 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.515+0000 7ff8c46d5700 1 --2- 172.21.15.99:0/1180740382 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8bc0ad9b0 0x7ff8bc0c1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:51.604 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.515+0000 7ff8c46d5700 1 --2- 172.21.15.99:0/1180740382 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8bc1568d0 0x7ff8bc0cc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:51.604 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.515+0000 7ff8c46d5700 1 --2- 172.21.15.99:0/1180740382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff8bc0c87a0 0x7ff8bc154990 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:51.604 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.515+0000 7ff8c46d5700 1 -- 172.21.15.99:0/1180740382 >> 172.21.15.99:0/1180740382 conn(0x7ff8bc0cda50 msgr2=0x7ff8bc0cb540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:51.604 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.515+0000 7ff8c46d5700 1 -- 172.21.15.99:0/1180740382 shutdown_connections 2023-10-08T00:21:51.604 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.515+0000 7ff8c46d5700 1 -- 172.21.15.99:0/1180740382 wait complete. 2023-10-08T00:21:51.605 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:21:51.719 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.718+0000 7f50e4cee700 1 Processor -- start 2023-10-08T00:21:51.719 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.718+0000 7f50e4cee700 1 -- start start 2023-10-08T00:21:51.721 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.719+0000 7f50e4cee700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f50e0157870 0x7f50e015b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:51.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.719+0000 7f50e4cee700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f50e0157c40 0x7f50e0158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:51.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.719+0000 7f50e4cee700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50e015cbb0 con 0x7f50e015a620 2023-10-08T00:21:51.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.719+0000 7f50e4cee700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50e015bfa0 con 0x7f50e0157c40 2023-10-08T00:21:51.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.719+0000 7f50e4cee700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50e015c280 con 0x7f50e0157870 2023-10-08T00:21:51.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.719+0000 7f50ddd9b700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f50e015a620 0x7f50e015d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59980/0 (socket says 172.21.15.99:59980) 2023-10-08T00:21:51.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.719+0000 7f50ddd9b700 1 -- 172.21.15.99:0/4075676575 learned_addr learned my addr 172.21.15.99:0/4075676575 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:21:51.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.719+0000 7f50ded9d700 1 --2- 172.21.15.99:0/4075676575 >> v2:172.21.15.191:3300/0 conn(0x7f50e0157c40 0x7f50e0158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:51.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.719+0000 7f50de59c700 1 --2- 172.21.15.99:0/4075676575 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f50e0157870 0x7f50e015b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:51.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.720+0000 7f50dd59a700 1 -- 172.21.15.99:0/4075676575 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2523002626 0 0) 0x7f50e015cbb0 con 0x7f50e015a620 2023-10-08T00:21:51.722 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.720+0000 7f50dd59a700 1 -- 172.21.15.99:0/4075676575 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50c8003570 con 0x7f50e015a620 2023-10-08T00:21:51.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.720+0000 7f50de59c700 1 -- 172.21.15.99:0/4075676575 >> v2:172.21.15.191:3300/0 conn(0x7f50e0157c40 msgr2=0x7f50e0158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:51.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.720+0000 7f50de59c700 1 --2- 172.21.15.99:0/4075676575 >> v2:172.21.15.191:3300/0 conn(0x7f50e0157c40 0x7f50e0158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:51.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.720+0000 7f50de59c700 1 -- 172.21.15.99:0/4075676575 >> v1:172.21.15.99:6789/0 conn(0x7f50e015a620 legacy=0x7f50e015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:51.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.720+0000 7f50de59c700 1 -- 172.21.15.99:0/4075676575 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e015c560 con 0x7f50e0157870 2023-10-08T00:21:51.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.720+0000 7f50ded9d700 1 --2- 172.21.15.99:0/4075676575 >> v2:172.21.15.191:3300/0 conn(0x7f50e0157c40 0x7f50e0158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:21:51.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.720+0000 7f50de59c700 1 --2- 172.21.15.99:0/4075676575 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f50e0157870 0x7f50e015b6a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f50d0009e60 tx=0x7f50d0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b9fc32fbd12670d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:51.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.720+0000 7f50dd59a700 1 -- 172.21.15.99:0/4075676575 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d000e070 con 0x7f50e0157870 2023-10-08T00:21:51.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.720+0000 7f50dd59a700 1 -- 172.21.15.99:0/4075676575 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50d000bab0 con 0x7f50e0157870 2023-10-08T00:21:51.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.720+0000 7f50dd59a700 1 -- 172.21.15.99:0/4075676575 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d000be70 con 0x7f50e0157870 2023-10-08T00:21:51.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.721+0000 7f50e4cee700 1 -- 172.21.15.99:0/4075676575 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f50e0157870 msgr2=0x7f50e015b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:51.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.721+0000 7f50e4cee700 1 --2- 172.21.15.99:0/4075676575 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f50e0157870 0x7f50e015b6a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f50d0009e60 tx=0x7f50d0009780 comp rx=0 tx=0).stop 2023-10-08T00:21:51.723 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.721+0000 7f50e4cee700 1 -- 172.21.15.99:0/4075676575 shutdown_connections 2023-10-08T00:21:51.724 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.721+0000 7f50e4cee700 1 --2- 172.21.15.99:0/4075676575 >> v2:172.21.15.191:3300/0 conn(0x7f50e0157c40 0x7f50e0158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:51.724 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.721+0000 7f50e4cee700 1 --2- 172.21.15.99:0/4075676575 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f50e0157870 0x7f50e015b6a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:51.724 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.721+0000 7f50e4cee700 1 -- 172.21.15.99:0/4075676575 >> 172.21.15.99:0/4075676575 conn(0x7f50e0068ce0 msgr2=0x7f50e00690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:51.724 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.721+0000 7f50e4cee700 1 -- 172.21.15.99:0/4075676575 shutdown_connections 2023-10-08T00:21:51.724 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.721+0000 7f50e4cee700 1 -- 172.21.15.99:0/4075676575 wait complete. 2023-10-08T00:21:51.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.722+0000 7f50e4cee700 1 Processor -- start 2023-10-08T00:21:51.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.722+0000 7f50e4cee700 1 -- start start 2023-10-08T00:21:51.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.723+0000 7f50e4cee700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f50e0157870 0x7f50e0151b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:51.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.723+0000 7f50e4cee700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f50e0157c40 0x7f50e014d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:51.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.723+0000 7f50e4cee700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50e015a620 0x7f50e01544d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:51.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.723+0000 7f50e4cee700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e014deb0 con 0x7f50e015a620 2023-10-08T00:21:51.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.723+0000 7f50e4cee700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e0152190 con 0x7f50e0157c40 2023-10-08T00:21:51.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.723+0000 7f50e4cee700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50e0152470 con 0x7f50e0157870 2023-10-08T00:21:51.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.723+0000 7f50ded9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50e015a620 0x7f50e01544d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:51.727 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.723+0000 7f50ded9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50e015a620 0x7f50e01544d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41754/0 (socket says 172.21.15.99:41754) 2023-10-08T00:21:51.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.723+0000 7f50ded9d700 1 -- 172.21.15.99:0/1225744859 learned_addr learned my addr 172.21.15.99:0/1225744859 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:51.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.723+0000 7f50de59c700 1 --2- 172.21.15.99:0/1225744859 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f50e0157870 0x7f50e0151b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:51.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.723+0000 7f50ddd9b700 1 --2- 172.21.15.99:0/1225744859 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f50e0157c40 0x7f50e014d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:51.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.723+0000 7f50ded9d700 1 -- 172.21.15.99:0/1225744859 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f50e0157870 msgr2=0x7f50e0151b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:51.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.723+0000 7f50ded9d700 1 --2- 172.21.15.99:0/1225744859 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f50e0157870 0x7f50e0151b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:51.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.723+0000 7f50ded9d700 1 -- 172.21.15.99:0/1225744859 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f50e0157c40 msgr2=0x7f50e014d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:51.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.723+0000 7f50ded9d700 1 --2- 172.21.15.99:0/1225744859 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f50e0157c40 0x7f50e014d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:51.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.723+0000 7f50ded9d700 1 -- 172.21.15.99:0/1225744859 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d8009420 con 0x7f50e015a620 2023-10-08T00:21:51.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.723+0000 7f50ded9d700 1 --2- 172.21.15.99:0/1225744859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50e015a620 0x7f50e01544d0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f50d800ec80 tx=0x7f50d80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:51.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.723+0000 7f50ce7fc700 1 -- 172.21.15.99:0/1225744859 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d8009720 con 0x7f50e015a620 2023-10-08T00:21:51.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.723+0000 7f50ce7fc700 1 -- 172.21.15.99:0/1225744859 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50d800c850 con 0x7f50e015a620 2023-10-08T00:21:51.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.723+0000 7f50e4cee700 1 -- 172.21.15.99:0/1225744859 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50d0009a20 con 0x7f50e015a620 2023-10-08T00:21:51.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.723+0000 7f50ce7fc700 1 -- 172.21.15.99:0/1225744859 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d800cc10 con 0x7f50e015a620 2023-10-08T00:21:51.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.723+0000 7f50e4cee700 1 -- 172.21.15.99:0/1225744859 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50e0152a30 con 0x7f50e015a620 2023-10-08T00:21:51.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.724+0000 7f50e4cee700 1 -- 172.21.15.99:0/1225744859 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50e004c3c0 con 0x7f50e015a620 2023-10-08T00:21:51.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.725+0000 7f50ce7fc700 1 -- 172.21.15.99:0/1225744859 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f50d8013470 con 0x7f50e015a620 2023-10-08T00:21:51.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.725+0000 7f50ce7fc700 1 --2- 172.21.15.99:0/1225744859 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f50c0066f30 0x7f50c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:51.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.725+0000 7f50ce7fc700 1 -- 172.21.15.99:0/1225744859 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5734+0+0 (secure 0 0 0) 0x7f50d8084570 con 0x7f50e015a620 2023-10-08T00:21:51.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.725+0000 7f50ce7fc700 1 -- 172.21.15.99:0/1225744859 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=189}) v3 -- 0x7f50c006eab0 con 0x7f50e015a620 2023-10-08T00:21:51.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.725+0000 7f50de59c700 1 --2- 172.21.15.99:0/1225744859 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f50c0066f30 0x7f50c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:51.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.726+0000 7f50de59c700 1 --2- 172.21.15.99:0/1225744859 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f50c0066f30 0x7f50c00693e0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f50d00097c0 tx=0x7f50d0019000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:51.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.729+0000 7f50ce7fc700 1 -- 172.21.15.99:0/1225744859 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f50d8051720 con 0x7f50e015a620 2023-10-08T00:21:51.952 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.951+0000 7f50e4cee700 1 -- 172.21.15.99:0/1225744859 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f50e015c840 con 0x7f50e015a620 2023-10-08T00:21:51.952 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.952+0000 7f50ce7fc700 1 -- 172.21.15.99:0/1225744859 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+4604 (secure 0 0 0) 0x7f50d8020020 con 0x7f50e015a620 2023-10-08T00:21:51.953 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:21:51.953 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":84,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/3862684847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3862684847},{"type":"v1","addr":"172.21.15.191:6836","nonce":3862684847}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":8106,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/359623206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":359623206},{"type":"v1","addr":"172.21.15.99:6834","nonce":359623206}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":8109,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3064101777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":3064101777},{"type":"v1","addr":"172.21.15.99:6835","nonce":3064101777}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:21:29.659394+0000","modified":"2023-10-08T00:21:33.311802+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8124},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8124":{"gid":8124,"name":"d","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/2466421758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":2466421758},{"type":"v1","addr":"172.21.15.191:6837","nonce":2466421758}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-08T00:21:51.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.954+0000 7f50e4cee700 1 -- 172.21.15.99:0/1225744859 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f50c0066f30 msgr2=0x7f50c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:51.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.954+0000 7f50e4cee700 1 --2- 172.21.15.99:0/1225744859 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f50c0066f30 0x7f50c00693e0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f50d00097c0 tx=0x7f50d0019000 comp rx=0 tx=0).stop 2023-10-08T00:21:51.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.954+0000 7f50e4cee700 1 -- 172.21.15.99:0/1225744859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50e015a620 msgr2=0x7f50e01544d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:51.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.954+0000 7f50e4cee700 1 --2- 172.21.15.99:0/1225744859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50e015a620 0x7f50e01544d0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f50d800ec80 tx=0x7f50d80091a0 comp rx=0 tx=0).stop 2023-10-08T00:21:51.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.954+0000 7f50e4cee700 1 -- 172.21.15.99:0/1225744859 shutdown_connections 2023-10-08T00:21:51.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.954+0000 7f50e4cee700 1 --2- 172.21.15.99:0/1225744859 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f50c0066f30 0x7f50c00693e0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:51.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.954+0000 7f50e4cee700 1 --2- 172.21.15.99:0/1225744859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50e015a620 0x7f50e01544d0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:51.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.954+0000 7f50e4cee700 1 --2- 172.21.15.99:0/1225744859 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f50e0157870 0x7f50e0151b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:51.955 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.954+0000 7f50e4cee700 1 --2- 172.21.15.99:0/1225744859 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f50e0157c40 0x7f50e014d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:51.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.954+0000 7f50e4cee700 1 -- 172.21.15.99:0/1225744859 >> 172.21.15.99:0/1225744859 conn(0x7f50e0068ce0 msgr2=0x7f50e0159910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:51.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.954+0000 7f50e4cee700 1 -- 172.21.15.99:0/1225744859 shutdown_connections 2023-10-08T00:21:51.956 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:51.954+0000 7f50e4cee700 1 -- 172.21.15.99:0/1225744859 wait complete. 2023-10-08T00:21:51.956 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 84 2023-10-08T00:21:51.967 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json status 2023-10-08T00:21:52.111 INFO:teuthology.orchestra.run.smithi191.stdout:{"cluster_fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","whoami":0,"id":8124,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":21.798201896999998,"mdsmap_epoch":84,"osdmap_epoch":188,"osdmap_epoch_barrier":188,"uptime":26.873842656000001} 2023-10-08T00:21:52.111 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", "fs_name": "cephfs", "id": 8124, "mdsmap_epoch": 84, "osdmap_epoch": 188, "osdmap_epoch_barrier": 188, "rank_uptime": 21.798201897, "state": "up:active", "uptime": 26.873842656, "want_state": "up:active", "whoami": 0 } 2023-10-08T00:21:52.111 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-08T00:21:52.111 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:21:52.111 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2023-10-08T00:21:52.128 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:21:52.128 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2023-10-08T00:21:52.190 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-08T00:21:52.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.343+0000 7fd5a54e7700 1 Processor -- start 2023-10-08T00:21:52.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.343+0000 7fd5a54e7700 1 -- start start 2023-10-08T00:21:52.347 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.344+0000 7fd5a54e7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd5a00b0e50 0x7fd5a00c9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:52.347 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.344+0000 7fd5a54e7700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd5a00cb4f0 0x7fd5a00cb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:52.348 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.344+0000 7fd5a54e7700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5a015de60 con 0x7fd5a00b1220 2023-10-08T00:21:52.348 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.344+0000 7fd5a54e7700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5a00cbed0 con 0x7fd5a00cb4f0 2023-10-08T00:21:52.348 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.344+0000 7fd5a54e7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a015e140 con 0x7fd5a00b0e50 2023-10-08T00:21:52.348 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.344+0000 7fd59effd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd5a00b0e50 0x7fd5a00c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:52.348 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.345+0000 7fd59f7fe700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd5a00cb4f0 0x7fd5a00cb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:52.348 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.345+0000 7fd59f7fe700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd5a00cb4f0 0x7fd5a00cb990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.99:34556/0 (socket says 172.21.15.99:34556) 2023-10-08T00:21:52.348 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.345+0000 7fd59effd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd5a00b0e50 0x7fd5a00c9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.99:42620/0 (socket says 172.21.15.99:42620) 2023-10-08T00:21:52.348 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.345+0000 7fd59f7fe700 1 -- 172.21.15.99:0/2446436988 learned_addr learned my addr 172.21.15.99:0/2446436988 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:52.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.345+0000 7fd59e7fc700 1 --1- 172.21.15.99:0/2446436988 >> v1:172.21.15.191:6789/0 conn(0x7fd5a00b1220 0x7fd5a006d590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:45670/0 (socket says 172.21.15.99:45670) 2023-10-08T00:21:52.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.345+0000 7fd59effd700 1 -- 172.21.15.99:0/2446436988 >> v2:172.21.15.191:3300/0 conn(0x7fd5a00cb4f0 msgr2=0x7fd5a00cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:52.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.345+0000 7fd59effd700 1 --2- 172.21.15.99:0/2446436988 >> v2:172.21.15.191:3300/0 conn(0x7fd5a00cb4f0 0x7fd5a00cb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:52.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.345+0000 7fd59effd700 1 -- 172.21.15.99:0/2446436988 >> v1:172.21.15.191:6789/0 conn(0x7fd5a00b1220 legacy=0x7fd5a006d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:52.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.345+0000 7fd59effd700 1 -- 172.21.15.99:0/2446436988 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a015e420 con 0x7fd5a00b0e50 2023-10-08T00:21:52.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.345+0000 7fd59effd700 1 --2- 172.21.15.99:0/2446436988 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd5a00b0e50 0x7fd5a00c9180 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd590009e60 tx=0x7fd590009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4e1659be0299df2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:52.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.345+0000 7fd59dffb700 1 -- 172.21.15.99:0/2446436988 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59000e070 con 0x7fd5a00b0e50 2023-10-08T00:21:52.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.345+0000 7fd59dffb700 1 -- 172.21.15.99:0/2446436988 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd59000bb60 con 0x7fd5a00b0e50 2023-10-08T00:21:52.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.345+0000 7fd59dffb700 1 -- 172.21.15.99:0/2446436988 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd590012430 con 0x7fd5a00b0e50 2023-10-08T00:21:52.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.346+0000 7fd5a54e7700 1 -- 172.21.15.99:0/2446436988 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd5a00b0e50 msgr2=0x7fd5a00c9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:52.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.346+0000 7fd5a54e7700 1 --2- 172.21.15.99:0/2446436988 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd5a00b0e50 0x7fd5a00c9180 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd590009e60 tx=0x7fd590009780 comp rx=0 tx=0).stop 2023-10-08T00:21:52.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.346+0000 7fd5a54e7700 1 -- 172.21.15.99:0/2446436988 shutdown_connections 2023-10-08T00:21:52.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.346+0000 7fd5a54e7700 1 --2- 172.21.15.99:0/2446436988 >> v2:172.21.15.191:3300/0 conn(0x7fd5a00cb4f0 0x7fd5a00cb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:52.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.346+0000 7fd5a54e7700 1 --2- 172.21.15.99:0/2446436988 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd5a00b0e50 0x7fd5a00c9180 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:52.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.346+0000 7fd5a54e7700 1 -- 172.21.15.99:0/2446436988 >> 172.21.15.99:0/2446436988 conn(0x7fd5a0069870 msgr2=0x7fd5a0069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:52.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.346+0000 7fd5a54e7700 1 -- 172.21.15.99:0/2446436988 shutdown_connections 2023-10-08T00:21:52.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.346+0000 7fd5a54e7700 1 -- 172.21.15.99:0/2446436988 wait complete. 2023-10-08T00:21:52.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.347+0000 7fd5a54e7700 1 Processor -- start 2023-10-08T00:21:52.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.347+0000 7fd5a54e7700 1 -- start start 2023-10-08T00:21:52.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.348+0000 7fd5a54e7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd5a00b0e50 0x7fd5a00c4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:52.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.348+0000 7fd5a54e7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5a00b1220 0x7fd5a0151210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:52.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.348+0000 7fd5a54e7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd5a00cb4f0 0x7fd5a0151750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:52.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.348+0000 7fd5a54e7700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a00b26b0 con 0x7fd5a00b1220 2023-10-08T00:21:52.355 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.348+0000 7fd5a54e7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a0151c90 con 0x7fd5a00b0e50 2023-10-08T00:21:52.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.348+0000 7fd5a54e7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a00b1aa0 con 0x7fd5a00cb4f0 2023-10-08T00:21:52.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.348+0000 7fd59e7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5a00b1220 0x7fd5a0151210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:52.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.348+0000 7fd59e7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5a00b1220 0x7fd5a0151210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41770/0 (socket says 172.21.15.99:41770) 2023-10-08T00:21:52.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.348+0000 7fd59e7fc700 1 -- 172.21.15.99:0/4276785444 learned_addr learned my addr 172.21.15.99:0/4276785444 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:52.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.348+0000 7fd59effd700 1 --2- 172.21.15.99:0/4276785444 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd5a00b0e50 0x7fd5a00c4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:52.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.348+0000 7fd59f7fe700 1 --2- 172.21.15.99:0/4276785444 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd5a00cb4f0 0x7fd5a0151750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:52.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.348+0000 7fd59e7fc700 1 -- 172.21.15.99:0/4276785444 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd5a00cb4f0 msgr2=0x7fd5a0151750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:52.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.348+0000 7fd59e7fc700 1 --2- 172.21.15.99:0/4276785444 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd5a00cb4f0 0x7fd5a0151750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:52.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.348+0000 7fd59e7fc700 1 -- 172.21.15.99:0/4276785444 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd5a00b0e50 msgr2=0x7fd5a00c4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:52.356 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.348+0000 7fd59e7fc700 1 --2- 172.21.15.99:0/4276785444 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd5a00b0e50 0x7fd5a00c4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:52.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.348+0000 7fd59e7fc700 1 -- 172.21.15.99:0/4276785444 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd590009a20 con 0x7fd5a00b1220 2023-10-08T00:21:52.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.348+0000 7fd59e7fc700 1 --2- 172.21.15.99:0/4276785444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5a00b1220 0x7fd5a0151210 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fd59400c7f0 tx=0x7fd594007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:52.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.348+0000 7fd586ffd700 1 -- 172.21.15.99:0/4276785444 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd594005a70 con 0x7fd5a00b1220 2023-10-08T00:21:52.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.348+0000 7fd586ffd700 1 -- 172.21.15.99:0/4276785444 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd594005bf0 con 0x7fd5a00b1220 2023-10-08T00:21:52.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.348+0000 7fd5a54e7700 1 -- 172.21.15.99:0/4276785444 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5a00b1d80 con 0x7fd5a00b1220 2023-10-08T00:21:52.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.348+0000 7fd586ffd700 1 -- 172.21.15.99:0/4276785444 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5940070d0 con 0x7fd5a00b1220 2023-10-08T00:21:52.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.348+0000 7fd5a54e7700 1 -- 172.21.15.99:0/4276785444 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5a00b2340 con 0x7fd5a00b1220 2023-10-08T00:21:52.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.349+0000 7fd5a54e7700 1 -- 172.21.15.99:0/4276785444 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5a00b2860 con 0x7fd5a00b1220 2023-10-08T00:21:52.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.350+0000 7fd586ffd700 1 -- 172.21.15.99:0/4276785444 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd594007480 con 0x7fd5a00b1220 2023-10-08T00:21:52.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.350+0000 7fd586ffd700 1 --2- 172.21.15.99:0/4276785444 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd58c066f30 0x7fd58c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:52.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.350+0000 7fd586ffd700 1 -- 172.21.15.99:0/4276785444 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5734+0+0 (secure 0 0 0) 0x7fd594083950 con 0x7fd5a00b1220 2023-10-08T00:21:52.357 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.350+0000 7fd59effd700 1 --2- 172.21.15.99:0/4276785444 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd58c066f30 0x7fd58c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:52.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.350+0000 7fd586ffd700 1 -- 172.21.15.99:0/4276785444 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=189}) v3 -- 0x7fd58c06eab0 con 0x7fd5a00b1220 2023-10-08T00:21:52.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.351+0000 7fd59effd700 1 --2- 172.21.15.99:0/4276785444 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd58c066f30 0x7fd58c0693e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fd590009e60 tx=0x7fd590019000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:52.358 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.354+0000 7fd586ffd700 1 -- 172.21.15.99:0/4276785444 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd594017020 con 0x7fd5a00b1220 2023-10-08T00:21:52.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.537+0000 7fd5a54e7700 1 -- 172.21.15.99:0/4276785444 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fd5a004c3c0 con 0x7fd5a00b1220 2023-10-08T00:21:52.539 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.538+0000 7fd586ffd700 1 -- 172.21.15.99:0/4276785444 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v188) v1 ==== 81+0+32493 (secure 0 0 0) 0x7fd5a00b2860 con 0x7fd5a00b1220 2023-10-08T00:21:52.540 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:21:52.540 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:21:52.540 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 188, 2023-10-08T00:21:52.540 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", 2023-10-08T00:21:52.541 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2023-10-08T00:10:36.303601+0000", 2023-10-08T00:21:52.541 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2023-10-08T00:21:46.168808+0000", 2023-10-08T00:21:52.541 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2023-10-08T00:10:41.709171+0000", 2023-10-08T00:21:52.541 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2023-10-08T00:10:38.690404+0000", 2023-10-08T00:21:52.541 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-08T00:21:52.542 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2023-10-08T00:21:52.542 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2023-10-08T00:21:52.542 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2023-10-08T00:21:52.542 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2023-10-08T00:21:52.542 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2023-10-08T00:21:52.543 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2023-10-08T00:21:52.543 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:21:52.543 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2023-10-08T00:21:52.543 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2023-10-08T00:21:52.544 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-08T00:21:52.544 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-08T00:21:52.544 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2023-10-08T00:21:52.544 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 23, 2023-10-08T00:21:52.545 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2023-10-08T00:21:52.545 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2023-10-08T00:21:52.545 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "luminous", 2023-10-08T00:21:52.545 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "quincy", 2023-10-08T00:21:52.546 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2023-10-08T00:21:52.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.546 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2023-10-08T00:21:52.546 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2023-10-08T00:21:52.546 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:10:42.063344+0000", 2023-10-08T00:21:52.546 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:21:52.546 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:21:52.546 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:21:52.547 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:21:52.547 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:21:52.547 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:21:52.547 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:21:52.547 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:21:52.547 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:21:52.547 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:21:52.547 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:21:52.547 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:21:52.547 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2023-10-08T00:21:52.547 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2023-10-08T00:21:52.548 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2023-10-08T00:21:52.548 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2023-10-08T00:21:52.548 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2023-10-08T00:21:52.548 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:21:52.548 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:21:52.548 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:21:52.548 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:21:52.548 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:21:52.548 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:21:52.548 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:21:52.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.548 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "112", 2023-10-08T00:21:52.549 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:21:52.549 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:21:52.549 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:21:52.549 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:21:52.549 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:21:52.549 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:21:52.549 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:21:52.549 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:21:52.549 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:21:52.549 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:21:52.549 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:21:52.549 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:21:52.550 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:21:52.550 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:21:52.550 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:21:52.550 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:21:52.550 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:21:52.550 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:21:52.550 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:21:52.550 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:21:52.550 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:21:52.550 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:21:52.550 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:21:52.550 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:21:52.551 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:21:52.551 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:21:52.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.551 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:21:52.551 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:21:52.551 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:21:52.551 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:21:52.551 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:21:52.551 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:21:52.551 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:21:52.551 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:21:52.551 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:21:52.552 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:21:52.552 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:21:52.552 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:21:52.552 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2023-10-08T00:21:52.552 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2023-10-08T00:21:52.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.552 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:21:52.552 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2023-10-08T00:21:52.552 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:52.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.552 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 22, 2023-10-08T00:21:52.552 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2023-10-08T00:21:52.553 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:21:27.984574+0000", 2023-10-08T00:21:52.553 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:21:52.553 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:21:52.553 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:21:52.553 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:21:52.553 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:21:52.553 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:21:52.554 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:21:52.554 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:21:52.554 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:21:52.555 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:21:52.555 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:21:52.555 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:21:52.555 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2023-10-08T00:21:52.555 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2023-10-08T00:21:52.555 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2023-10-08T00:21:52.555 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2023-10-08T00:21:52.555 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2023-10-08T00:21:52.555 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:21:52.555 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:21:52.555 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:21:52.555 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:21:52.556 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:21:52.556 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:21:52.556 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:21:52.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.556 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "183", 2023-10-08T00:21:52.556 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:21:52.556 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:21:52.556 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:21:52.556 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:21:52.556 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:21:52.556 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:21:52.556 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:21:52.557 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:21:52.557 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:21:52.557 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:21:52.557 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:21:52.557 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:21:52.557 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:21:52.557 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:21:52.557 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:21:52.557 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:21:52.557 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:21:52.557 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:21:52.557 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:21:52.558 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:21:52.558 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:21:52.558 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:21:52.558 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:21:52.558 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:21:52.558 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:21:52.558 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:21:52.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.558 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:21:52.558 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:21:52.558 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:21:52.558 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:21:52.559 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:21:52.559 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:21:52.559 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:21:52.559 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:21:52.559 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:21:52.559 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:21:52.559 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:21:52.559 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:21:52.559 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2023-10-08T00:21:52.559 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2023-10-08T00:21:52.559 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2023-10-08T00:21:52.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.560 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:21:52.560 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:21:52.560 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2023-10-08T00:21:52.560 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:52.560 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:52.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.560 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 23, 2023-10-08T00:21:52.560 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2023-10-08T00:21:52.560 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:21:29.244620+0000", 2023-10-08T00:21:52.560 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1027, 2023-10-08T00:21:52.560 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full,full_quota", 2023-10-08T00:21:52.561 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:21:52.561 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:21:52.561 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:21:52.561 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:21:52.561 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:21:52.561 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:21:52.561 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:21:52.561 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:21:52.561 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:21:52.561 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:21:52.561 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2023-10-08T00:21:52.561 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2023-10-08T00:21:52.562 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2023-10-08T00:21:52.562 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2023-10-08T00:21:52.562 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2023-10-08T00:21:52.562 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:21:52.562 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:21:52.562 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:21:52.562 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:21:52.562 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:21:52.562 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:21:52.562 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:21:52.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.562 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "188", 2023-10-08T00:21:52.563 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:21:52.563 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:21:52.563 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:21:52.563 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:21:52.563 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:21:52.563 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:21:52.563 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:21:52.563 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:21:52.563 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:21:52.563 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 33554432, 2023-10-08T00:21:52.563 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:21:52.563 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:21:52.564 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:21:52.564 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:21:52.564 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:21:52.564 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:21:52.564 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:21:52.564 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:21:52.564 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:21:52.564 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:21:52.564 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:21:52.564 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:21:52.564 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:21:52.565 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:21:52.565 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:21:52.565 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:21:52.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.565 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:21:52.565 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:21:52.565 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:21:52.565 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:21:52.565 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:21:52.565 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:21:52.565 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:21:52.566 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:21:52.566 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:21:52.566 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:21:52.566 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:21:52.566 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:21:52.566 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2023-10-08T00:21:52.566 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2023-10-08T00:21:52.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.566 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:21:52.566 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:21:52.566 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2023-10-08T00:21:52.566 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:52.567 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:52.567 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:52.567 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:21:52.567 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2023-10-08T00:21:52.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.567 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:21:52.567 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:21:52.567 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:21:52.568 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:21:52.568 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:21:52.568 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:21:52.568 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:21:52.569 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:21:52.569 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:21:52.569 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:21:52.569 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:21:52.570 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:21:52.570 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:21:52.570 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:52.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.571 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:52.571 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2023-10-08T00:21:52.571 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:21:52.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.572 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:52.572 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2023-10-08T00:21:52.572 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:21:52.572 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:52.572 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:52.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.572 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:21:52.573 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:52.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.573 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:52.573 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2023-10-08T00:21:52.573 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:21:52.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.573 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:52.573 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2023-10-08T00:21:52.573 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:21:52.574 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:52.574 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:52.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.574 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:21:52.574 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:52.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.574 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:52.574 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2023-10-08T00:21:52.574 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:21:52.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.575 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:52.575 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2023-10-08T00:21:52.575 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:21:52.575 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:52.575 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:52.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.575 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:21:52.575 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:52.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.575 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:52.575 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2023-10-08T00:21:52.576 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:21:52.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.576 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:52.576 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2023-10-08T00:21:52.576 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:21:52.576 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:52.576 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:52.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.576 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/86365", 2023-10-08T00:21:52.576 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/86365", 2023-10-08T00:21:52.577 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/86365", 2023-10-08T00:21:52.577 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/86365", 2023-10-08T00:21:52.577 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:21:52.577 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:21:52.577 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:21:52.577 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:52.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.577 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:21:52.577 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:21:52.577 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:21:52.577 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:21:52.578 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:21:52.578 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:21:52.578 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:21:52.578 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:21:52.578 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:21:52.578 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:21:52.578 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:21:52.578 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:21:52.578 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:21:52.578 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:52.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.579 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:52.579 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2023-10-08T00:21:52.579 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:21:52.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.579 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:52.579 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2023-10-08T00:21:52.579 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:21:52.579 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:52.579 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:52.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.579 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:21:52.580 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:52.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.580 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:52.580 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2023-10-08T00:21:52.580 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:21:52.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.580 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:52.580 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2023-10-08T00:21:52.580 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:21:52.580 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:52.580 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:52.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.581 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:21:52.581 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:52.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.581 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:52.581 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2023-10-08T00:21:52.581 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:21:52.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.581 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:52.581 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2023-10-08T00:21:52.582 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:21:52.582 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:52.582 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:52.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.582 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:21:52.582 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:52.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.582 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:52.582 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2023-10-08T00:21:52.582 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:21:52.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.583 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:52.583 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2023-10-08T00:21:52.583 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:21:52.583 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:52.583 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:52.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.583 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/86366", 2023-10-08T00:21:52.583 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/86366", 2023-10-08T00:21:52.583 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/86366", 2023-10-08T00:21:52.583 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/86366", 2023-10-08T00:21:52.583 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:21:52.583 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:21:52.584 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:21:52.584 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:52.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.584 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:21:52.584 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:21:52.584 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:21:52.584 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:21:52.584 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:21:52.584 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:21:52.584 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:21:52.584 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:21:52.585 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:21:52.585 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:21:52.585 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:21:52.585 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:21:52.585 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:21:52.585 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:52.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.585 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:52.585 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2023-10-08T00:21:52.585 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:21:52.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.586 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:52.586 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2023-10-08T00:21:52.586 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:21:52.586 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:52.586 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:52.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.586 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:21:52.586 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:52.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.586 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:52.586 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2023-10-08T00:21:52.586 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:21:52.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.587 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:52.587 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2023-10-08T00:21:52.587 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:21:52.587 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:52.587 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:52.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.587 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:21:52.587 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:52.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.587 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:52.588 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2023-10-08T00:21:52.588 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:21:52.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.588 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:52.588 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2023-10-08T00:21:52.588 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:21:52.588 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:52.588 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:52.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.588 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:21:52.588 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:52.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.589 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:52.589 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2023-10-08T00:21:52.589 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:21:52.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.589 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:52.589 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2023-10-08T00:21:52.590 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:21:52.590 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:52.590 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:52.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.590 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6817/86382", 2023-10-08T00:21:52.590 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/86382", 2023-10-08T00:21:52.590 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/86382", 2023-10-08T00:21:52.590 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/86382", 2023-10-08T00:21:52.590 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:21:52.591 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:21:52.591 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:21:52.591 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:52.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.591 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:21:52.591 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:21:52.591 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:21:52.591 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:21:52.591 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:21:52.591 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:21:52.592 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:21:52.592 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:21:52.592 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:21:52.592 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 186, 2023-10-08T00:21:52.592 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:21:52.592 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:21:52.592 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:21:52.592 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:52.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.592 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:52.592 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2023-10-08T00:21:52.592 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:21:52.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.593 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.593 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:52.593 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2023-10-08T00:21:52.593 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:21:52.593 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:52.593 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:52.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.593 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:21:52.593 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:52.593 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.594 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:52.594 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2023-10-08T00:21:52.594 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:21:52.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.594 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:52.594 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2023-10-08T00:21:52.594 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:21:52.594 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:52.595 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:52.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.595 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:21:52.595 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:52.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.595 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:52.595 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2023-10-08T00:21:52.595 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:21:52.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.595 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:52.596 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2023-10-08T00:21:52.596 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:21:52.596 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:52.596 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:52.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.596 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:21:52.596 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:52.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.596 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:52.596 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2023-10-08T00:21:52.597 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:21:52.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.597 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.597 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:52.597 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2023-10-08T00:21:52.597 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:21:52.597 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:52.597 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:52.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.597 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6820/86393", 2023-10-08T00:21:52.597 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/86393", 2023-10-08T00:21:52.598 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/86393", 2023-10-08T00:21:52.598 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/86393", 2023-10-08T00:21:52.598 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:21:52.598 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:21:52.598 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:21:52.598 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:52.598 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.598 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.598 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:21:52.598 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:21:52.599 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:21:52.599 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:21:52.599 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:21:52.599 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:21:52.599 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:21:52.599 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:21:52.599 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:21:52.599 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:21:52.599 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:21:52.599 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:21:52.599 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:21:52.600 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:52.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.600 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:52.600 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6800", 2023-10-08T00:21:52.600 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:21:52.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.600 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:52.600 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6801", 2023-10-08T00:21:52.600 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:21:52.600 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:52.600 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:52.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.601 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:21:52.601 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:52.601 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.601 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:52.601 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6802", 2023-10-08T00:21:52.601 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:21:52.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.601 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.601 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:52.601 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6803", 2023-10-08T00:21:52.601 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:21:52.602 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:52.602 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:52.602 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.602 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:21:52.602 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:52.602 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.602 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:52.602 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6806", 2023-10-08T00:21:52.602 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:21:52.602 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.602 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.602 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:52.603 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6807", 2023-10-08T00:21:52.603 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:21:52.603 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:52.603 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:52.603 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.603 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:21:52.603 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:52.603 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.604 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:52.604 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6804", 2023-10-08T00:21:52.604 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:21:52.604 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.604 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.604 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:52.604 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6805", 2023-10-08T00:21:52.604 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:21:52.604 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:52.604 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:52.604 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.604 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6801/86570", 2023-10-08T00:21:52.605 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6803/86570", 2023-10-08T00:21:52.605 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6807/86570", 2023-10-08T00:21:52.605 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6805/86570", 2023-10-08T00:21:52.605 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:21:52.605 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:21:52.605 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:21:52.605 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:52.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.605 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:21:52.605 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:21:52.605 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:21:52.606 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:21:52.606 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:21:52.606 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:21:52.606 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:21:52.606 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:21:52.606 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:21:52.606 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:21:52.606 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:21:52.606 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:21:52.606 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:21:52.606 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:52.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.607 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:52.607 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6816", 2023-10-08T00:21:52.607 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:21:52.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.607 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:52.607 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6818", 2023-10-08T00:21:52.607 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:21:52.607 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:52.607 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:52.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.607 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:21:52.608 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:52.608 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.608 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:52.608 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6820", 2023-10-08T00:21:52.608 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:21:52.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.608 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.608 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:52.608 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6821", 2023-10-08T00:21:52.608 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:21:52.608 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:52.608 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:52.609 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.609 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:21:52.609 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:52.609 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.609 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:52.609 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6826", 2023-10-08T00:21:52.609 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:21:52.609 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.609 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.609 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:52.609 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6828", 2023-10-08T00:21:52.609 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:21:52.609 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:52.610 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:52.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.610 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:21:52.610 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:52.610 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.610 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:52.610 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6823", 2023-10-08T00:21:52.610 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:21:52.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.610 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.610 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:52.610 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6825", 2023-10-08T00:21:52.611 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:21:52.611 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:52.611 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:52.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.611 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6818/86608", 2023-10-08T00:21:52.611 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6821/86608", 2023-10-08T00:21:52.611 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6828/86608", 2023-10-08T00:21:52.611 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6825/86608", 2023-10-08T00:21:52.611 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:21:52.611 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:21:52.611 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:21:52.611 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:52.612 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.612 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.612 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:21:52.612 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:21:52.612 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:21:52.612 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:21:52.612 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:21:52.612 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:21:52.612 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:21:52.612 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:21:52.612 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:21:52.612 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:21:52.612 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:21:52.613 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:21:52.613 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:21:52.613 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:52.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.613 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:52.613 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6808", 2023-10-08T00:21:52.613 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:21:52.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.613 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:52.613 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6809", 2023-10-08T00:21:52.614 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:21:52.614 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:52.614 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:52.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.614 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:21:52.614 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:52.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.614 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:52.614 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6810", 2023-10-08T00:21:52.614 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:21:52.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.615 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:52.615 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6811", 2023-10-08T00:21:52.615 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:21:52.615 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:52.615 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:52.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.615 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:21:52.615 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:52.615 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.615 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:52.615 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6814", 2023-10-08T00:21:52.615 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:21:52.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.616 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:52.616 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6815", 2023-10-08T00:21:52.616 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:21:52.616 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:52.616 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:52.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.616 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:21:52.616 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:52.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.616 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:52.617 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6812", 2023-10-08T00:21:52.617 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:21:52.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.617 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:52.617 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6813", 2023-10-08T00:21:52.617 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:21:52.617 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:52.617 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:52.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.617 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6809/86571", 2023-10-08T00:21:52.617 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6811/86571", 2023-10-08T00:21:52.617 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6815/86571", 2023-10-08T00:21:52.618 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6813/86571", 2023-10-08T00:21:52.618 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:21:52.618 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:21:52.618 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:21:52.618 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:52.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.618 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.618 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:21:52.618 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:21:52.618 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:21:52.618 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:21:52.619 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:21:52.619 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:21:52.619 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:21:52.619 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:21:52.619 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:21:52.619 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:21:52.619 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:21:52.619 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:21:52.619 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:21:52.619 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:52.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.619 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:52.620 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6817", 2023-10-08T00:21:52.620 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:21:52.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.620 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.620 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:52.620 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6819", 2023-10-08T00:21:52.620 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:21:52.620 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:52.620 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:52.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.620 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:21:52.620 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:52.621 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.621 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:52.621 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6822", 2023-10-08T00:21:52.621 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:21:52.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.621 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.621 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:52.621 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6824", 2023-10-08T00:21:52.621 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:21:52.621 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:52.621 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:52.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.622 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:21:52.622 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:52.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.622 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:52.622 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6830", 2023-10-08T00:21:52.622 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:21:52.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.622 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:52.622 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6831", 2023-10-08T00:21:52.622 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:21:52.622 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:52.623 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:52.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.623 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:21:52.623 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:52.623 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.623 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:52.623 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6827", 2023-10-08T00:21:52.623 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:21:52.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.623 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.623 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:52.623 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6829", 2023-10-08T00:21:52.624 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:21:52.624 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:52.624 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:52.624 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.624 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6819/86612", 2023-10-08T00:21:52.624 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6824/86612", 2023-10-08T00:21:52.624 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6831/86612", 2023-10-08T00:21:52.624 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6829/86612", 2023-10-08T00:21:52.624 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:21:52.624 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:21:52.625 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:21:52.625 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:52.625 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:52.625 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:21:52.625 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2023-10-08T00:21:52.625 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.625 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:21:52.625 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:21:52.625 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:21:52.625 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:21:52.626 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:21:52.626 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:21:52.626 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.905615+0000", 2023-10-08T00:21:52.626 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:21:52.626 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.626 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.626 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:21:52.626 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:21:52.626 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:21:52.626 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:21:52.626 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:21:52.627 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:21:52.627 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.880709+0000", 2023-10-08T00:21:52.627 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:21:52.627 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.627 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.627 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:21:52.627 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:21:52.627 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:21:52.627 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:21:52.627 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:21:52.627 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:21:52.628 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.874704+0000", 2023-10-08T00:21:52.628 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:21:52.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.628 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.628 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:21:52.628 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:21:52.628 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:21:52.628 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:21:52.628 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:21:52.628 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:21:52.628 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.899604+0000", 2023-10-08T00:21:52.628 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:21:52.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.629 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.629 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:21:52.629 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:21:52.629 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:21:52.629 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:21:52.629 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:21:52.629 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:21:52.629 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.844371+0000", 2023-10-08T00:21:52.629 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:21:52.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.629 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.630 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:21:52.630 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:21:52.630 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:21:52.630 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:21:52.630 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:21:52.630 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:21:52.630 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.794810+0000", 2023-10-08T00:21:52.630 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:21:52.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.630 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.630 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:21:52.630 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:21:52.631 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:21:52.631 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:21:52.631 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:21:52.631 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:21:52.631 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.859474+0000", 2023-10-08T00:21:52.631 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:21:52.631 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.631 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.631 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:21:52.631 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:21:52.631 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:21:52.632 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:21:52.632 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:21:52.632 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:21:52.632 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.893130+0000", 2023-10-08T00:21:52.632 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:21:52.632 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:52.632 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:21:52.632 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2023-10-08T00:21:52.632 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [ 2023-10-08T00:21:52.633 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.633 INFO:teuthology.orchestra.run.smithi099.stdout: "pgid": "23.18", 2023-10-08T00:21:52.633 INFO:teuthology.orchestra.run.smithi099.stdout: "mappings": [ 2023-10-08T00:21:52.633 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:52.633 INFO:teuthology.orchestra.run.smithi099.stdout: "from": 5, 2023-10-08T00:21:52.633 INFO:teuthology.orchestra.run.smithi099.stdout: "to": 3 2023-10-08T00:21:52.633 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:52.633 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:52.633 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:52.633 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:21:52.633 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2023-10-08T00:21:52.633 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2023-10-08T00:21:52.634 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2023-10-08T00:21:52.634 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2023-10-08T00:21:52.634 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2023-10-08T00:21:52.634 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2023-10-08T00:21:52.634 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2023-10-08T00:21:52.634 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2023-10-08T00:21:52.634 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2023-10-08T00:21:52.634 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2023-10-08T00:21:52.634 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2023-10-08T00:21:52.634 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:52.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:52.634 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2023-10-08T00:21:52.635 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2023-10-08T00:21:52.635 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2023-10-08T00:21:52.635 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2023-10-08T00:21:52.635 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2023-10-08T00:21:52.635 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2023-10-08T00:21:52.635 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2023-10-08T00:21:52.635 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2023-10-08T00:21:52.635 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2023-10-08T00:21:52.635 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2023-10-08T00:21:52.635 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2023-10-08T00:21:52.635 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:52.635 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:21:52.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.540+0000 7fd5a54e7700 1 -- 172.21.15.99:0/4276785444 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd58c066f30 msgr2=0x7fd58c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:52.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.540+0000 7fd5a54e7700 1 --2- 172.21.15.99:0/4276785444 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd58c066f30 0x7fd58c0693e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fd590009e60 tx=0x7fd590019000 comp rx=0 tx=0).stop 2023-10-08T00:21:52.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.540+0000 7fd5a54e7700 1 -- 172.21.15.99:0/4276785444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5a00b1220 msgr2=0x7fd5a0151210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:52.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.540+0000 7fd5a54e7700 1 --2- 172.21.15.99:0/4276785444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5a00b1220 0x7fd5a0151210 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fd59400c7f0 tx=0x7fd594007f70 comp rx=0 tx=0).stop 2023-10-08T00:21:52.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.540+0000 7fd5a54e7700 1 -- 172.21.15.99:0/4276785444 shutdown_connections 2023-10-08T00:21:52.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.540+0000 7fd5a54e7700 1 --2- 172.21.15.99:0/4276785444 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fd58c066f30 0x7fd58c0693e0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:52.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.540+0000 7fd5a54e7700 1 --2- 172.21.15.99:0/4276785444 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd5a00b0e50 0x7fd5a00c4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:52.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.540+0000 7fd5a54e7700 1 --2- 172.21.15.99:0/4276785444 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd5a00cb4f0 0x7fd5a0151750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:52.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.540+0000 7fd5a54e7700 1 --2- 172.21.15.99:0/4276785444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5a00b1220 0x7fd5a0151210 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:52.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.540+0000 7fd5a54e7700 1 -- 172.21.15.99:0/4276785444 >> 172.21.15.99:0/4276785444 conn(0x7fd5a0069870 msgr2=0x7fd5a006cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:52.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.540+0000 7fd5a54e7700 1 -- 172.21.15.99:0/4276785444 shutdown_connections 2023-10-08T00:21:52.638 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:52.540+0000 7fd5a54e7700 1 -- 172.21.15.99:0/4276785444 wait complete. 2023-10-08T00:21:52.639 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-08T00:21:54.262 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:21:54.266 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:21:57.639 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-08T00:21:57.766 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.766+0000 7fc3f56a3700 1 Processor -- start 2023-10-08T00:21:57.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.766+0000 7fc3f56a3700 1 -- start start 2023-10-08T00:21:57.770 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.767+0000 7fc3f56a3700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc3f0157870 0x7fc3f015b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:57.770 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.767+0000 7fc3f56a3700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3f015cbb0 con 0x7fc3f0157c40 2023-10-08T00:21:57.770 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.767+0000 7fc3f56a3700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3f015bfa0 con 0x7fc3f015a620 2023-10-08T00:21:57.770 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.767+0000 7fc3f56a3700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3f015c280 con 0x7fc3f0157870 2023-10-08T00:21:57.770 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.767+0000 7fc3ef7fe700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fc3f0157c40 0x7fc3f0158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:44244/0 (socket says 172.21.15.99:44244) 2023-10-08T00:21:57.770 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.767+0000 7fc3ef7fe700 1 -- 172.21.15.99:0/3443984212 learned_addr learned my addr 172.21.15.99:0/3443984212 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:21:57.770 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.767+0000 7fc3eeffd700 1 --2- 172.21.15.99:0/3443984212 >> v2:172.21.15.191:3300/0 conn(0x7fc3f0157870 0x7fc3f015b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:57.770 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.767+0000 7fc3edffb700 1 -- 172.21.15.99:0/3443984212 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1346714098 0 0) 0x7fc3f015cbb0 con 0x7fc3f0157c40 2023-10-08T00:21:57.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.767+0000 7fc3edffb700 1 -- 172.21.15.99:0/3443984212 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3dc003570 con 0x7fc3f0157c40 2023-10-08T00:21:57.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.767+0000 7fc3edffb700 1 -- 172.21.15.99:0/3443984212 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 518667112 0 0) 0x7fc3dc003570 con 0x7fc3f0157c40 2023-10-08T00:21:57.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.767+0000 7fc3edffb700 1 -- 172.21.15.99:0/3443984212 >> v2:172.21.15.191:3300/0 conn(0x7fc3f0157870 msgr2=0x7fc3f015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:57.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.767+0000 7fc3edffb700 1 --2- 172.21.15.99:0/3443984212 >> v2:172.21.15.191:3300/0 conn(0x7fc3f0157870 0x7fc3f015b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:57.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.767+0000 7fc3edffb700 1 -- 172.21.15.99:0/3443984212 >> v1:172.21.15.191:6789/0 conn(0x7fc3f015a620 legacy=0x7fc3f015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:57.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.767+0000 7fc3edffb700 1 -- 172.21.15.99:0/3443984212 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f015c560 con 0x7fc3f0157c40 2023-10-08T00:21:57.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.767+0000 7fc3edffb700 1 -- 172.21.15.99:0/3443984212 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7fc3e0001470 con 0x7fc3f0157c40 2023-10-08T00:21:57.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.767+0000 7fc3edffb700 1 -- 172.21.15.99:0/3443984212 >> v1:172.21.15.99:6789/0 conn(0x7fc3f0157c40 legacy=0x7fc3f0158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:57.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.767+0000 7fc3edffb700 1 --2- 172.21.15.99:0/3443984212 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3dc003a60 0x7fc3dc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:57.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.767+0000 7fc3edffb700 1 --2- 172.21.15.99:0/3443984212 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc3dc004810 0x7fc3dc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:57.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.767+0000 7fc3edffb700 1 --2- 172.21.15.99:0/3443984212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3dc0052e0 0x7fc3dc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:57.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.767+0000 7fc3edffb700 1 -- 172.21.15.99:0/3443984212 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f015c280 con 0x7fc3dc0052e0 2023-10-08T00:21:57.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.767+0000 7fc3edffb700 1 -- 172.21.15.99:0/3443984212 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3dc00c230 con 0x7fc3dc003a60 2023-10-08T00:21:57.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.767+0000 7fc3edffb700 1 -- 172.21.15.99:0/3443984212 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3dc00c3f0 con 0x7fc3dc004810 2023-10-08T00:21:57.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.767+0000 7fc3ef7fe700 1 --2- 172.21.15.99:0/3443984212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3dc0052e0 0x7fc3dc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:57.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.767+0000 7fc3eeffd700 1 --2- 172.21.15.99:0/3443984212 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3dc003a60 0x7fc3dc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:57.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.768+0000 7fc3ee7fc700 1 --2- 172.21.15.99:0/3443984212 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc3dc004810 0x7fc3dc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:57.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.768+0000 7fc3ef7fe700 1 -- 172.21.15.99:0/3443984212 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc3dc004810 msgr2=0x7fc3dc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:57.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.768+0000 7fc3ef7fe700 1 --2- 172.21.15.99:0/3443984212 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc3dc004810 0x7fc3dc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:57.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.768+0000 7fc3ef7fe700 1 -- 172.21.15.99:0/3443984212 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3dc003a60 msgr2=0x7fc3dc003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:21:57.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.768+0000 7fc3ef7fe700 1 --2- 172.21.15.99:0/3443984212 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3dc003a60 0x7fc3dc003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:57.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.768+0000 7fc3ef7fe700 1 -- 172.21.15.99:0/3443984212 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3dc003570 con 0x7fc3dc0052e0 2023-10-08T00:21:57.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.768+0000 7fc3ef7fe700 1 -- 172.21.15.99:0/3443984212 reap_dead start 2023-10-08T00:21:57.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.768+0000 7fc3ef7fe700 1 --2- 172.21.15.99:0/3443984212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3dc0052e0 0x7fc3dc00ba60 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fc3e000c390 tx=0x7fc3e0001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b1e03c2b29ccd6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:57.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.768+0000 7fc3edffb700 1 -- 172.21.15.99:0/3443984212 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e000cc60 con 0x7fc3dc0052e0 2023-10-08T00:21:57.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.768+0000 7fc3edffb700 1 -- 172.21.15.99:0/3443984212 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3e000cdc0 con 0x7fc3dc0052e0 2023-10-08T00:21:57.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.768+0000 7fc3edffb700 1 -- 172.21.15.99:0/3443984212 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e0005b50 con 0x7fc3dc0052e0 2023-10-08T00:21:57.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.768+0000 7fc3f56a3700 1 -- 172.21.15.99:0/3443984212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3dc0052e0 msgr2=0x7fc3dc00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:57.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.769+0000 7fc3f56a3700 1 --2- 172.21.15.99:0/3443984212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3dc0052e0 0x7fc3dc00ba60 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fc3e000c390 tx=0x7fc3e0001e50 comp rx=0 tx=0).stop 2023-10-08T00:21:57.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.769+0000 7fc3f56a3700 1 -- 172.21.15.99:0/3443984212 shutdown_connections 2023-10-08T00:21:57.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.769+0000 7fc3f56a3700 1 --2- 172.21.15.99:0/3443984212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3dc0052e0 0x7fc3dc00ba60 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:57.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.769+0000 7fc3f56a3700 1 -- 172.21.15.99:0/3443984212 >> 172.21.15.99:0/3443984212 conn(0x7fc3f0068ce0 msgr2=0x7fc3f00690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:57.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.769+0000 7fc3f56a3700 1 -- 172.21.15.99:0/3443984212 shutdown_connections 2023-10-08T00:21:57.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.769+0000 7fc3f56a3700 1 -- 172.21.15.99:0/3443984212 wait complete. 2023-10-08T00:21:57.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.770+0000 7fc3f56a3700 1 Processor -- start 2023-10-08T00:21:57.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.770+0000 7fc3f56a3700 1 -- start start 2023-10-08T00:21:57.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.771+0000 7fc3f56a3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3dc0052e0 0x7fc3f013f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:57.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.771+0000 7fc3f56a3700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3f00b2a50 0x7fc3f00b2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:57.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.771+0000 7fc3f56a3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc3f016a570 0x7fc3f016aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:57.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.771+0000 7fc3f56a3700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f00b19b0 con 0x7fc3dc0052e0 2023-10-08T00:21:57.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.771+0000 7fc3f56a3700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f00b1c90 con 0x7fc3f00b2a50 2023-10-08T00:21:57.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.771+0000 7fc3f56a3700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f00b1f70 con 0x7fc3f016a570 2023-10-08T00:21:57.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.771+0000 7fc3eeffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3dc0052e0 0x7fc3f013f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:57.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.771+0000 7fc3eeffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3dc0052e0 0x7fc3f013f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60354/0 (socket says 172.21.15.99:60354) 2023-10-08T00:21:57.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.771+0000 7fc3eeffd700 1 -- 172.21.15.99:0/1264859006 learned_addr learned my addr 172.21.15.99:0/1264859006 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:21:57.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.771+0000 7fc3ef7fe700 1 --2- 172.21.15.99:0/1264859006 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc3f016a570 0x7fc3f016aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:57.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.771+0000 7fc3ee7fc700 1 --2- 172.21.15.99:0/1264859006 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3f00b2a50 0x7fc3f00b2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:57.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.771+0000 7fc3eeffd700 1 -- 172.21.15.99:0/1264859006 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc3f016a570 msgr2=0x7fc3f016aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:57.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.771+0000 7fc3eeffd700 1 --2- 172.21.15.99:0/1264859006 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc3f016a570 0x7fc3f016aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:57.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.771+0000 7fc3eeffd700 1 -- 172.21.15.99:0/1264859006 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3f00b2a50 msgr2=0x7fc3f00b2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:57.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.771+0000 7fc3eeffd700 1 --2- 172.21.15.99:0/1264859006 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3f00b2a50 0x7fc3f00b2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:57.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.771+0000 7fc3eeffd700 1 -- 172.21.15.99:0/1264859006 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f00b2250 con 0x7fc3dc0052e0 2023-10-08T00:21:57.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.771+0000 7fc3ef7fe700 1 --2- 172.21.15.99:0/1264859006 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc3f016a570 0x7fc3f016aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:21:57.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.771+0000 7fc3ee7fc700 1 --2- 172.21.15.99:0/1264859006 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3f00b2a50 0x7fc3f00b2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:21:57.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.771+0000 7fc3eeffd700 1 --2- 172.21.15.99:0/1264859006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3dc0052e0 0x7fc3f013f6f0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fc3e80070b0 tx=0x7fc3e8009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:57.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.771+0000 7fc3e6ffd700 1 -- 172.21.15.99:0/1264859006 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8015420 con 0x7fc3dc0052e0 2023-10-08T00:21:57.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.771+0000 7fc3e6ffd700 1 -- 172.21.15.99:0/1264859006 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3e8015580 con 0x7fc3dc0052e0 2023-10-08T00:21:57.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.771+0000 7fc3e6ffd700 1 -- 172.21.15.99:0/1264859006 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8015920 con 0x7fc3dc0052e0 2023-10-08T00:21:57.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.771+0000 7fc3f56a3700 1 -- 172.21.15.99:0/1264859006 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3f00b2530 con 0x7fc3dc0052e0 2023-10-08T00:21:57.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.771+0000 7fc3f56a3700 1 -- 172.21.15.99:0/1264859006 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3f0153440 con 0x7fc3dc0052e0 2023-10-08T00:21:57.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.772+0000 7fc3f56a3700 1 -- 172.21.15.99:0/1264859006 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3f00b28a0 con 0x7fc3dc0052e0 2023-10-08T00:21:57.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.773+0000 7fc3e6ffd700 1 -- 172.21.15.99:0/1264859006 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc3e8003040 con 0x7fc3dc0052e0 2023-10-08T00:21:57.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.773+0000 7fc3e6ffd700 1 --2- 172.21.15.99:0/1264859006 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc3d0066f30 0x7fc3d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:21:57.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.773+0000 7fc3e6ffd700 1 -- 172.21.15.99:0/1264859006 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5734+0+0 (secure 0 0 0) 0x7fc3e80851c0 con 0x7fc3dc0052e0 2023-10-08T00:21:57.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.773+0000 7fc3ee7fc700 1 --2- 172.21.15.99:0/1264859006 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc3d0066f30 0x7fc3d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:21:57.779 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.773+0000 7fc3e6ffd700 1 -- 172.21.15.99:0/1264859006 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=189}) v3 -- 0x7fc3d006eab0 con 0x7fc3dc0052e0 2023-10-08T00:21:57.779 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.774+0000 7fc3ee7fc700 1 --2- 172.21.15.99:0/1264859006 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc3d0066f30 0x7fc3d00693e0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8000f60 tx=0x7fc3d8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:21:57.779 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.777+0000 7fc3e6ffd700 1 -- 172.21.15.99:0/1264859006 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc3e8052410 con 0x7fc3dc0052e0 2023-10-08T00:21:57.958 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.958+0000 7fc3f56a3700 1 -- 172.21.15.99:0/1264859006 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fc3f004c3c0 con 0x7fc3dc0052e0 2023-10-08T00:21:57.959 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.959+0000 7fc3e6ffd700 1 -- 172.21.15.99:0/1264859006 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v188) v1 ==== 81+0+32493 (secure 0 0 0) 0x7fc3e8055a30 con 0x7fc3dc0052e0 2023-10-08T00:21:57.959 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:21:57.960 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:21:57.960 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 188, 2023-10-08T00:21:57.960 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", 2023-10-08T00:21:57.960 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2023-10-08T00:10:36.303601+0000", 2023-10-08T00:21:57.960 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2023-10-08T00:21:46.168808+0000", 2023-10-08T00:21:57.960 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2023-10-08T00:10:41.709171+0000", 2023-10-08T00:21:57.960 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2023-10-08T00:10:38.690404+0000", 2023-10-08T00:21:57.960 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-08T00:21:57.960 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2023-10-08T00:21:57.960 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2023-10-08T00:21:57.960 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2023-10-08T00:21:57.961 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2023-10-08T00:21:57.961 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2023-10-08T00:21:57.961 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2023-10-08T00:21:57.961 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:21:57.961 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2023-10-08T00:21:57.961 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2023-10-08T00:21:57.961 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-08T00:21:57.961 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-08T00:21:57.961 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2023-10-08T00:21:57.961 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 23, 2023-10-08T00:21:57.961 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2023-10-08T00:21:57.961 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2023-10-08T00:21:57.962 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "luminous", 2023-10-08T00:21:57.962 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "quincy", 2023-10-08T00:21:57.962 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2023-10-08T00:21:57.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:57.962 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2023-10-08T00:21:57.962 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2023-10-08T00:21:57.962 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:10:42.063344+0000", 2023-10-08T00:21:57.962 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:21:57.962 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:21:57.962 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:21:57.962 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:21:57.963 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:21:57.963 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:21:57.963 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:21:57.963 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:21:57.963 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:21:57.963 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:21:57.963 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:21:57.963 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:21:57.963 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2023-10-08T00:21:57.963 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2023-10-08T00:21:57.963 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2023-10-08T00:21:57.964 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2023-10-08T00:21:57.964 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2023-10-08T00:21:57.964 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:21:57.964 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:21:57.964 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:21:57.964 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:21:57.964 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:21:57.964 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:21:57.964 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:21:57.964 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:57.964 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "112", 2023-10-08T00:21:57.964 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:21:57.964 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:21:57.965 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:21:57.965 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:21:57.965 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:21:57.965 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:21:57.965 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:21:57.965 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:21:57.965 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:21:57.965 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:21:57.965 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:21:57.965 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:21:57.965 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:21:57.965 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:21:57.966 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:21:57.966 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:21:57.966 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:21:57.966 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:21:57.966 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:21:57.966 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:21:57.966 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:21:57.966 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:21:57.966 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:21:57.966 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:21:57.966 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:21:57.966 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:21:57.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:57.967 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:21:57.967 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:21:57.967 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:21:57.967 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:21:57.967 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:21:57.967 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:21:57.967 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:21:57.967 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:21:57.967 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:21:57.967 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:21:57.967 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:21:57.968 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:21:57.968 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2023-10-08T00:21:57.968 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2023-10-08T00:21:57.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:57.968 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:21:57.968 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2023-10-08T00:21:57.968 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:57.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:57.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:57.968 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 22, 2023-10-08T00:21:57.968 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2023-10-08T00:21:57.968 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:21:27.984574+0000", 2023-10-08T00:21:57.969 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:21:57.969 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:21:57.969 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:21:57.969 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:21:57.969 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:21:57.969 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:21:57.969 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:21:57.969 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:21:57.969 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:21:57.969 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:21:57.969 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:21:57.969 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:21:57.970 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2023-10-08T00:21:57.970 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2023-10-08T00:21:57.970 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2023-10-08T00:21:57.970 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2023-10-08T00:21:57.970 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2023-10-08T00:21:57.970 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:21:57.970 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:21:57.970 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:21:57.970 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:21:57.970 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:21:57.970 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:21:57.970 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:21:57.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:57.971 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "183", 2023-10-08T00:21:57.971 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:21:57.971 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:21:57.971 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:21:57.971 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:21:57.971 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:21:57.971 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:21:57.971 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:21:57.971 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:21:57.971 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:21:57.971 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:21:57.972 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:21:57.972 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:21:57.972 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:21:57.972 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:21:57.972 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:21:57.972 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:21:57.972 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:21:57.972 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:21:57.972 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:21:57.972 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:21:57.972 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:21:57.972 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:21:57.972 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:21:57.973 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:21:57.973 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:21:57.973 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:21:57.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:57.973 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:21:57.973 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:21:57.973 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:21:57.973 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:21:57.973 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:21:57.973 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:21:57.973 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:21:57.973 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:21:57.974 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:21:57.974 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:21:57.974 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:21:57.974 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:21:57.974 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2023-10-08T00:21:57.974 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2023-10-08T00:21:57.974 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2023-10-08T00:21:57.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:57.974 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:21:57.974 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:21:57.974 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2023-10-08T00:21:57.975 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:57.975 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:57.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:57.975 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:57.975 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 23, 2023-10-08T00:21:57.975 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2023-10-08T00:21:57.975 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:21:29.244620+0000", 2023-10-08T00:21:57.975 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1027, 2023-10-08T00:21:57.975 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full,full_quota", 2023-10-08T00:21:57.975 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:21:57.975 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:21:57.975 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:21:57.976 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:21:57.976 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:21:57.976 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:21:57.976 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:21:57.976 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:21:57.976 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:21:57.976 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:21:57.976 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2023-10-08T00:21:57.976 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2023-10-08T00:21:57.976 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2023-10-08T00:21:57.976 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2023-10-08T00:21:57.976 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2023-10-08T00:21:57.977 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:21:57.977 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:21:57.977 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:21:57.977 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:21:57.977 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:21:57.977 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:21:57.977 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:21:57.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:57.977 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "188", 2023-10-08T00:21:57.977 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:21:57.977 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:21:57.977 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:21:57.977 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:21:57.978 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:21:57.978 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:21:57.978 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:21:57.978 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:21:57.978 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:21:57.978 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 33554432, 2023-10-08T00:21:57.978 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:21:57.978 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:21:57.978 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:21:57.978 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:21:57.978 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:21:57.979 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:21:57.979 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:21:57.979 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:21:57.979 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:21:57.979 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:21:57.979 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:21:57.979 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:21:57.979 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:21:57.979 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:21:57.979 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:21:57.979 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:21:57.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:57.979 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:21:57.979 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:21:57.980 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:21:57.980 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:21:57.982 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:21:57.982 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:21:57.982 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:21:57.982 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:21:57.982 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:21:57.982 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:21:57.982 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:21:57.982 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:21:57.982 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2023-10-08T00:21:57.983 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2023-10-08T00:21:57.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:57.983 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:21:57.983 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:21:57.983 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2023-10-08T00:21:57.983 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:57.983 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:57.983 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:57.983 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:21:57.983 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2023-10-08T00:21:57.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:57.983 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:21:57.984 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:21:57.984 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:21:57.984 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:21:57.984 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:21:57.984 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:21:57.984 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:21:57.984 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:21:57.984 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:21:57.984 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:21:57.984 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:21:57.984 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:21:57.984 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:21:57.984 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:57.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:57.985 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:57.985 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2023-10-08T00:21:57.985 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:21:57.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:57.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:57.985 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:57.985 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2023-10-08T00:21:57.985 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:21:57.985 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:57.985 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:57.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:57.986 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:21:57.986 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:57.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:57.986 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:57.986 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2023-10-08T00:21:57.986 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:21:57.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:57.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:57.986 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:57.986 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2023-10-08T00:21:57.986 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:21:57.986 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:57.987 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:57.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:57.987 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:21:57.987 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:57.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:57.987 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:57.987 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2023-10-08T00:21:57.987 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:21:57.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:57.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:57.987 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:57.987 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2023-10-08T00:21:57.987 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:21:57.988 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:57.988 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:57.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:57.988 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:21:57.988 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:57.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:57.988 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:57.988 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2023-10-08T00:21:57.988 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:21:57.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:57.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:57.988 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:57.989 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2023-10-08T00:21:57.989 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:21:57.989 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:57.989 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:57.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:57.989 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/86365", 2023-10-08T00:21:57.989 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/86365", 2023-10-08T00:21:57.989 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/86365", 2023-10-08T00:21:57.989 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/86365", 2023-10-08T00:21:57.989 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:21:57.989 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:21:57.989 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:21:57.989 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:57.990 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:57.990 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:57.990 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:21:57.990 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:21:57.990 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:21:57.990 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:21:57.990 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:21:57.990 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:21:57.990 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:21:57.990 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:21:57.990 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:21:57.991 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:21:57.991 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:21:57.991 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:21:57.991 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:21:57.991 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:57.991 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:57.991 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:57.991 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2023-10-08T00:21:57.991 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:21:57.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:57.991 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:57.991 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:57.991 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2023-10-08T00:21:57.992 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:21:57.992 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:57.992 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:57.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:57.992 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:21:57.992 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:57.992 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:57.992 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:57.992 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2023-10-08T00:21:57.992 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:21:57.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:57.992 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:57.993 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:57.993 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2023-10-08T00:21:57.993 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:21:57.993 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:57.993 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:57.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:57.993 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:21:57.993 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:57.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:57.993 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:57.993 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2023-10-08T00:21:57.993 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:21:57.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:57.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:57.994 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:57.994 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2023-10-08T00:21:57.994 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:21:57.994 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:57.994 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:57.994 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:57.994 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:21:57.994 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:57.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:57.994 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:57.994 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2023-10-08T00:21:57.995 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:21:57.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:57.995 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:57.995 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:57.995 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2023-10-08T00:21:57.995 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:21:57.995 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:57.995 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:57.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:57.995 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/86366", 2023-10-08T00:21:57.995 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/86366", 2023-10-08T00:21:57.995 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/86366", 2023-10-08T00:21:57.996 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/86366", 2023-10-08T00:21:57.996 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:21:57.996 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:21:57.996 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:21:57.996 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:57.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:57.996 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:57.996 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:21:57.996 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:21:57.996 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:21:57.996 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:21:57.996 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:21:57.997 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:21:57.997 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:21:57.997 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:21:57.997 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:21:57.997 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:21:57.997 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:21:57.997 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:21:57.997 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:21:57.997 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:57.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:57.997 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:57.997 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2023-10-08T00:21:57.998 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:21:57.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:57.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:57.998 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:57.998 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2023-10-08T00:21:57.998 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:21:57.998 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:57.998 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:57.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:57.998 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:21:57.998 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:57.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:57.999 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:57.999 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2023-10-08T00:21:57.999 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:21:57.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:57.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:57.999 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:57.999 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2023-10-08T00:21:57.999 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:21:57.999 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:57.999 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:57.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:57.999 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:21:57.999 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:58.000 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.000 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:58.000 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2023-10-08T00:21:58.000 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:21:58.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.000 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.000 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:58.000 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2023-10-08T00:21:58.000 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:21:58.000 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:58.000 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:58.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.001 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:21:58.001 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:58.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.001 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:58.001 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2023-10-08T00:21:58.001 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:21:58.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.001 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:58.001 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2023-10-08T00:21:58.001 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:21:58.002 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:58.002 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:58.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.002 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6817/86382", 2023-10-08T00:21:58.002 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/86382", 2023-10-08T00:21:58.002 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/86382", 2023-10-08T00:21:58.002 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/86382", 2023-10-08T00:21:58.002 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:21:58.002 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:21:58.002 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:21:58.002 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:58.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.003 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:21:58.003 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:21:58.003 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:21:58.003 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:21:58.003 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:21:58.003 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:21:58.003 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:21:58.003 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:21:58.003 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:21:58.003 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 186, 2023-10-08T00:21:58.004 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:21:58.004 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:21:58.004 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:21:58.004 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:58.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.004 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:58.004 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2023-10-08T00:21:58.004 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:21:58.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.004 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:58.004 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2023-10-08T00:21:58.005 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:21:58.005 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:58.005 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:58.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.005 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:21:58.005 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:58.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.005 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:58.005 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2023-10-08T00:21:58.005 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:21:58.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.005 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:58.006 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2023-10-08T00:21:58.006 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:21:58.006 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:58.006 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:58.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.006 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:21:58.006 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:58.006 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.006 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:58.006 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2023-10-08T00:21:58.006 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:21:58.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.007 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:58.007 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2023-10-08T00:21:58.007 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:21:58.007 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:58.007 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:58.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.007 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:21:58.007 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:58.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.007 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:58.008 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2023-10-08T00:21:58.008 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:21:58.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.008 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:58.008 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2023-10-08T00:21:58.008 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:21:58.008 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:58.008 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:58.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.008 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6820/86393", 2023-10-08T00:21:58.009 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/86393", 2023-10-08T00:21:58.009 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/86393", 2023-10-08T00:21:58.009 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/86393", 2023-10-08T00:21:58.009 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:21:58.009 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:21:58.009 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:21:58.009 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:58.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.009 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:21:58.009 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:21:58.009 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:21:58.010 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:21:58.010 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:21:58.010 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:21:58.010 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:21:58.010 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:21:58.010 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:21:58.010 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:21:58.010 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:21:58.010 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:21:58.010 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:21:58.011 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:58.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.011 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:58.011 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6800", 2023-10-08T00:21:58.011 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:21:58.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.011 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:58.011 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6801", 2023-10-08T00:21:58.011 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:21:58.011 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:58.011 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:58.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.012 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:21:58.012 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:58.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.012 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:58.012 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6802", 2023-10-08T00:21:58.012 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:21:58.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.012 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:58.012 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6803", 2023-10-08T00:21:58.012 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:21:58.013 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:58.013 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:58.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.013 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:21:58.013 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:58.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.013 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:58.013 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6806", 2023-10-08T00:21:58.013 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:21:58.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.014 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:58.014 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6807", 2023-10-08T00:21:58.014 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:21:58.014 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:58.014 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:58.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.014 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:21:58.014 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:58.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.014 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:58.066 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6804", 2023-10-08T00:21:58.067 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:21:58.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.067 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:58.067 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6805", 2023-10-08T00:21:58.067 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:21:58.067 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:58.067 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:58.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.068 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6801/86570", 2023-10-08T00:21:58.068 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6803/86570", 2023-10-08T00:21:58.068 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6807/86570", 2023-10-08T00:21:58.068 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6805/86570", 2023-10-08T00:21:58.068 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:21:58.068 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:21:58.068 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:21:58.068 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:58.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.069 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:21:58.069 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:21:58.069 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:21:58.069 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:21:58.069 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:21:58.069 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:21:58.069 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:21:58.069 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:21:58.069 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:21:58.070 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:21:58.070 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:21:58.070 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:21:58.070 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:21:58.070 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:58.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.070 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:58.070 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6816", 2023-10-08T00:21:58.070 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:21:58.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.071 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:58.071 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6818", 2023-10-08T00:21:58.071 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:21:58.071 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:58.071 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:58.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.071 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:21:58.071 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:58.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.071 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:58.071 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6820", 2023-10-08T00:21:58.072 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:21:58.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.072 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:58.072 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6821", 2023-10-08T00:21:58.072 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:21:58.072 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:58.072 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:58.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.072 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:21:58.072 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:58.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.073 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:58.073 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6826", 2023-10-08T00:21:58.073 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:21:58.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.073 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:58.073 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6828", 2023-10-08T00:21:58.073 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:21:58.073 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:58.073 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:58.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.074 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:21:58.074 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:58.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.074 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:58.074 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6823", 2023-10-08T00:21:58.074 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:21:58.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.074 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:58.074 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6825", 2023-10-08T00:21:58.074 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:21:58.075 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:58.075 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:58.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.075 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6818/86608", 2023-10-08T00:21:58.075 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6821/86608", 2023-10-08T00:21:58.075 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6828/86608", 2023-10-08T00:21:58.075 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6825/86608", 2023-10-08T00:21:58.075 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:21:58.075 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:21:58.075 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:21:58.075 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:58.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.076 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:21:58.076 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:21:58.076 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:21:58.076 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:21:58.076 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:21:58.076 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:21:58.076 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:21:58.076 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:21:58.076 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:21:58.076 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:21:58.076 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:21:58.077 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:21:58.077 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:21:58.077 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:58.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.077 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:58.077 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6808", 2023-10-08T00:21:58.077 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:21:58.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.077 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:58.077 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6809", 2023-10-08T00:21:58.077 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:21:58.078 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:58.078 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:58.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.078 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:21:58.078 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:58.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.078 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:58.078 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6810", 2023-10-08T00:21:58.078 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:21:58.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.078 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:58.079 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6811", 2023-10-08T00:21:58.079 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:21:58.079 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:58.079 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:58.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.079 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:21:58.079 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:58.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.079 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:58.079 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6814", 2023-10-08T00:21:58.079 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:21:58.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.080 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:58.080 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6815", 2023-10-08T00:21:58.080 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:21:58.080 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:58.080 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:58.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.080 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:21:58.080 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:58.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.080 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:58.081 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6812", 2023-10-08T00:21:58.081 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:21:58.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.081 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:58.081 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6813", 2023-10-08T00:21:58.081 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:21:58.081 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:58.081 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:58.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.082 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6809/86571", 2023-10-08T00:21:58.082 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6811/86571", 2023-10-08T00:21:58.082 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6815/86571", 2023-10-08T00:21:58.082 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6813/86571", 2023-10-08T00:21:58.082 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:21:58.082 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:21:58.082 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:21:58.082 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:58.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.083 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:21:58.083 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:21:58.083 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:21:58.083 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:21:58.083 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:21:58.083 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:21:58.083 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:21:58.083 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:21:58.083 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:21:58.083 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:21:58.083 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:21:58.084 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:21:58.084 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:21:58.084 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:58.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.084 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:58.084 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6817", 2023-10-08T00:21:58.084 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:21:58.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.084 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:58.084 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6819", 2023-10-08T00:21:58.084 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:21:58.084 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:58.085 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:58.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.085 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:21:58.085 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:58.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.085 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:58.085 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6822", 2023-10-08T00:21:58.085 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:21:58.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.086 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:58.086 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6824", 2023-10-08T00:21:58.086 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:21:58.086 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:58.086 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:58.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.086 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:21:58.086 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:58.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.086 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:58.087 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6830", 2023-10-08T00:21:58.087 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:21:58.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.087 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:58.087 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6831", 2023-10-08T00:21:58.087 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:21:58.087 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:58.087 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:58.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.087 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:21:58.087 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:21:58.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.088 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:21:58.088 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6827", 2023-10-08T00:21:58.088 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:21:58.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.088 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:21:58.088 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6829", 2023-10-08T00:21:58.088 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:21:58.088 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:58.088 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:58.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.089 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6819/86612", 2023-10-08T00:21:58.089 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6824/86612", 2023-10-08T00:21:58.089 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6831/86612", 2023-10-08T00:21:58.089 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6829/86612", 2023-10-08T00:21:58.089 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:21:58.089 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:21:58.089 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:21:58.089 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:58.089 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:58.089 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:21:58.089 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2023-10-08T00:21:58.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.090 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:21:58.090 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:21:58.090 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:21:58.090 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:21:58.090 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:21:58.090 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:21:58.090 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.905615+0000", 2023-10-08T00:21:58.090 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:21:58.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.090 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:21:58.090 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:21:58.091 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:21:58.091 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:21:58.091 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:21:58.091 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:21:58.091 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.880709+0000", 2023-10-08T00:21:58.091 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:21:58.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.091 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:21:58.091 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:21:58.091 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:21:58.091 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:21:58.091 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:21:58.092 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:21:58.092 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.874704+0000", 2023-10-08T00:21:58.092 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:21:58.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.092 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:21:58.092 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:21:58.092 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:21:58.092 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:21:58.092 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:21:58.092 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:21:58.092 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.899604+0000", 2023-10-08T00:21:58.093 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:21:58.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.093 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:21:58.093 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:21:58.093 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:21:58.093 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:21:58.093 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:21:58.093 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:21:58.093 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.844371+0000", 2023-10-08T00:21:58.093 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:21:58.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.094 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:21:58.094 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:21:58.094 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:21:58.094 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:21:58.094 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:21:58.094 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:21:58.094 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.794810+0000", 2023-10-08T00:21:58.094 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:21:58.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.094 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:21:58.094 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:21:58.095 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:21:58.095 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:21:58.095 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:21:58.095 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:21:58.095 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.859474+0000", 2023-10-08T00:21:58.095 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:21:58.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.095 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:21:58.095 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:21:58.095 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:21:58.095 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:21:58.096 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:21:58.096 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:21:58.096 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.893130+0000", 2023-10-08T00:21:58.096 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:21:58.096 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:58.096 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:21:58.096 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2023-10-08T00:21:58.096 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [ 2023-10-08T00:21:58.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.096 INFO:teuthology.orchestra.run.smithi099.stdout: "pgid": "23.18", 2023-10-08T00:21:58.096 INFO:teuthology.orchestra.run.smithi099.stdout: "mappings": [ 2023-10-08T00:21:58.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:21:58.097 INFO:teuthology.orchestra.run.smithi099.stdout: "from": 5, 2023-10-08T00:21:58.097 INFO:teuthology.orchestra.run.smithi099.stdout: "to": 3 2023-10-08T00:21:58.097 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:58.097 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:21:58.097 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:58.097 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:21:58.097 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2023-10-08T00:21:58.097 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2023-10-08T00:21:58.097 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2023-10-08T00:21:58.097 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2023-10-08T00:21:58.097 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2023-10-08T00:21:58.098 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2023-10-08T00:21:58.098 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2023-10-08T00:21:58.098 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2023-10-08T00:21:58.098 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2023-10-08T00:21:58.098 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2023-10-08T00:21:58.098 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2023-10-08T00:21:58.098 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:58.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:21:58.098 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2023-10-08T00:21:58.098 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2023-10-08T00:21:58.098 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2023-10-08T00:21:58.098 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2023-10-08T00:21:58.099 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2023-10-08T00:21:58.099 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2023-10-08T00:21:58.099 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2023-10-08T00:21:58.099 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2023-10-08T00:21:58.099 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2023-10-08T00:21:58.099 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2023-10-08T00:21:58.099 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2023-10-08T00:21:58.099 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:21:58.099 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:21:58.101 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.961+0000 7fc3f56a3700 1 -- 172.21.15.99:0/1264859006 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc3d0066f30 msgr2=0x7fc3d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:58.101 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.961+0000 7fc3f56a3700 1 --2- 172.21.15.99:0/1264859006 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc3d0066f30 0x7fc3d00693e0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8000f60 tx=0x7fc3d8009040 comp rx=0 tx=0).stop 2023-10-08T00:21:58.101 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.961+0000 7fc3f56a3700 1 -- 172.21.15.99:0/1264859006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3dc0052e0 msgr2=0x7fc3f013f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:21:58.101 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.961+0000 7fc3f56a3700 1 --2- 172.21.15.99:0/1264859006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3dc0052e0 0x7fc3f013f6f0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fc3e80070b0 tx=0x7fc3e8009710 comp rx=0 tx=0).stop 2023-10-08T00:21:58.101 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.961+0000 7fc3f56a3700 1 -- 172.21.15.99:0/1264859006 shutdown_connections 2023-10-08T00:21:58.101 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.961+0000 7fc3f56a3700 1 --2- 172.21.15.99:0/1264859006 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc3d0066f30 0x7fc3d00693e0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:58.101 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.961+0000 7fc3f56a3700 1 --2- 172.21.15.99:0/1264859006 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc3f016a570 0x7fc3f016aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:58.101 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.961+0000 7fc3f56a3700 1 --2- 172.21.15.99:0/1264859006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3dc0052e0 0x7fc3f013f6f0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:58.102 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.961+0000 7fc3f56a3700 1 --2- 172.21.15.99:0/1264859006 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3f00b2a50 0x7fc3f00b2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:21:58.102 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.961+0000 7fc3f56a3700 1 -- 172.21.15.99:0/1264859006 >> 172.21.15.99:0/1264859006 conn(0x7fc3f0068ce0 msgr2=0x7fc3f00b5350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:21:58.102 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.961+0000 7fc3f56a3700 1 -- 172.21.15.99:0/1264859006 shutdown_connections 2023-10-08T00:21:58.102 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:21:57.961+0000 7fc3f56a3700 1 -- 172.21.15.99:0/1264859006 wait complete. 2023-10-08T00:21:58.102 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-08T00:22:03.103 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-08T00:22:03.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.227+0000 7fd01cd19700 1 Processor -- start 2023-10-08T00:22:03.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.227+0000 7fd01cd19700 1 -- start start 2023-10-08T00:22:03.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.228+0000 7fd01cd19700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd018157c40 0x7fd018158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:03.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.228+0000 7fd01cd19700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd01815cbb0 con 0x7fd018157870 2023-10-08T00:22:03.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.228+0000 7fd01cd19700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd01815bfa0 con 0x7fd01815a620 2023-10-08T00:22:03.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.228+0000 7fd01cd19700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd01815bdb0 con 0x7fd018157c40 2023-10-08T00:22:03.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.228+0000 7fd01659c700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fd018157870 0x7fd01815b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:44250/0 (socket says 172.21.15.99:44250) 2023-10-08T00:22:03.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.228+0000 7fd01659c700 1 -- 172.21.15.99:0/4071376548 learned_addr learned my addr 172.21.15.99:0/4071376548 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:22:03.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.228+0000 7fd016d9d700 1 --2- 172.21.15.99:0/4071376548 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd018157c40 0x7fd018158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:03.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.228+0000 7fd015d9b700 1 -- 172.21.15.99:0/4071376548 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1324420327 0 0) 0x7fd01815cbb0 con 0x7fd018157870 2023-10-08T00:22:03.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.228+0000 7fd015d9b700 1 -- 172.21.15.99:0/4071376548 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd004003570 con 0x7fd018157870 2023-10-08T00:22:03.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.229+0000 7fd015d9b700 1 -- 172.21.15.99:0/4071376548 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4054743666 0 0) 0x7fd01815bfa0 con 0x7fd01815a620 2023-10-08T00:22:03.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.229+0000 7fd015d9b700 1 -- 172.21.15.99:0/4071376548 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd01815cbb0 con 0x7fd01815a620 2023-10-08T00:22:03.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.229+0000 7fd015d9b700 1 -- 172.21.15.99:0/4071376548 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1955051009 0 0) 0x7fd004003570 con 0x7fd018157870 2023-10-08T00:22:03.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.229+0000 7fd015d9b700 1 -- 172.21.15.99:0/4071376548 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd018157c40 msgr2=0x7fd018158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:03.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.229+0000 7fd015d9b700 1 --2- 172.21.15.99:0/4071376548 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd018157c40 0x7fd018158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:03.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.229+0000 7fd015d9b700 1 -- 172.21.15.99:0/4071376548 >> v1:172.21.15.191:6789/0 conn(0x7fd01815a620 legacy=0x7fd01815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:03.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.229+0000 7fd015d9b700 1 -- 172.21.15.99:0/4071376548 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd01815c280 con 0x7fd018157870 2023-10-08T00:22:03.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.229+0000 7fd015d9b700 1 -- 172.21.15.99:0/4071376548 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7fd008001470 con 0x7fd018157870 2023-10-08T00:22:03.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.229+0000 7fd015d9b700 1 -- 172.21.15.99:0/4071376548 >> v1:172.21.15.99:6789/0 conn(0x7fd018157870 legacy=0x7fd01815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:03.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.229+0000 7fd015d9b700 1 --2- 172.21.15.99:0/4071376548 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd004003c20 0x7fd004004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:03.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.229+0000 7fd015d9b700 1 --2- 172.21.15.99:0/4071376548 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd004004810 0x7fd004004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:03.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.229+0000 7fd015d9b700 1 --2- 172.21.15.99:0/4071376548 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd018157c40 0x7fd0040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:03.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.229+0000 7fd015d9b700 1 -- 172.21.15.99:0/4071376548 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00400bb10 con 0x7fd004004810 2023-10-08T00:22:03.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.229+0000 7fd015d9b700 1 -- 172.21.15.99:0/4071376548 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00400bc80 con 0x7fd004003c20 2023-10-08T00:22:03.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.229+0000 7fd015d9b700 1 -- 172.21.15.99:0/4071376548 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd00400be40 con 0x7fd018157c40 2023-10-08T00:22:03.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.229+0000 7fd00dd9b700 1 --2- 172.21.15.99:0/4071376548 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd004004810 0x7fd004004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:03.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.229+0000 7fd01659c700 1 --2- 172.21.15.99:0/4071376548 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd004003c20 0x7fd004004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:03.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.229+0000 7fd016d9d700 1 --2- 172.21.15.99:0/4071376548 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd018157c40 0x7fd0040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:03.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.229+0000 7fd00dd9b700 1 -- 172.21.15.99:0/4071376548 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd018157c40 msgr2=0x7fd0040051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:03.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.229+0000 7fd00dd9b700 1 --2- 172.21.15.99:0/4071376548 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd018157c40 0x7fd0040051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:03.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.229+0000 7fd00dd9b700 1 -- 172.21.15.99:0/4071376548 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd004003c20 msgr2=0x7fd004004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:03.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.229+0000 7fd00dd9b700 1 --2- 172.21.15.99:0/4071376548 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd004003c20 0x7fd004004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:03.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.229+0000 7fd00dd9b700 1 -- 172.21.15.99:0/4071376548 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd004003570 con 0x7fd004004810 2023-10-08T00:22:03.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.229+0000 7fd00dd9b700 1 --2- 172.21.15.99:0/4071376548 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd004004810 0x7fd004004c80 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fd0000019e0 tx=0x7fd000007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c78e4dd1810e5fb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:03.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.229+0000 7fd015d9b700 1 -- 172.21.15.99:0/4071376548 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd00000c5d0 con 0x7fd004004810 2023-10-08T00:22:03.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.230+0000 7fd015d9b700 1 -- 172.21.15.99:0/4071376548 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd00000c730 con 0x7fd004004810 2023-10-08T00:22:03.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.230+0000 7fd015d9b700 1 -- 172.21.15.99:0/4071376548 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd00000cad0 con 0x7fd004004810 2023-10-08T00:22:03.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.230+0000 7fd01cd19700 1 -- 172.21.15.99:0/4071376548 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd004004810 msgr2=0x7fd004004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:03.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.230+0000 7fd01cd19700 1 --2- 172.21.15.99:0/4071376548 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd004004810 0x7fd004004c80 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fd0000019e0 tx=0x7fd000007d30 comp rx=0 tx=0).stop 2023-10-08T00:22:03.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.230+0000 7fd016d9d700 1 -- 172.21.15.99:0/4071376548 reap_dead start 2023-10-08T00:22:03.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.230+0000 7fd01cd19700 1 -- 172.21.15.99:0/4071376548 shutdown_connections 2023-10-08T00:22:03.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.230+0000 7fd01cd19700 1 -- 172.21.15.99:0/4071376548 >> 172.21.15.99:0/4071376548 conn(0x7fd018068ce0 msgr2=0x7fd0180690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:03.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.230+0000 7fd01cd19700 1 -- 172.21.15.99:0/4071376548 shutdown_connections 2023-10-08T00:22:03.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.230+0000 7fd01cd19700 1 -- 172.21.15.99:0/4071376548 wait complete. 2023-10-08T00:22:03.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.231+0000 7fd01cd19700 1 Processor -- start 2023-10-08T00:22:03.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.231+0000 7fd01cd19700 1 -- start start 2023-10-08T00:22:03.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.232+0000 7fd01cd19700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd004004810 0x7fd01816a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:03.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.232+0000 7fd01cd19700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd018155d40 0x7fd0180c64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:03.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.232+0000 7fd01cd19700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0180c6a10 0x7fd0180c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:03.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.232+0000 7fd01cd19700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd01816ad20 con 0x7fd0180c6a10 2023-10-08T00:22:03.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.232+0000 7fd01cd19700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd01815dff0 con 0x7fd018155d40 2023-10-08T00:22:03.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.232+0000 7fd01cd19700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0180c6fe0 con 0x7fd004004810 2023-10-08T00:22:03.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.232+0000 7fd016d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0180c6a10 0x7fd0180c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:03.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.232+0000 7fd016d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0180c6a10 0x7fd0180c78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60372/0 (socket says 172.21.15.99:60372) 2023-10-08T00:22:03.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.232+0000 7fd016d9d700 1 -- 172.21.15.99:0/2232534545 learned_addr learned my addr 172.21.15.99:0/2232534545 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:03.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.232+0000 7fd01659c700 1 --2- 172.21.15.99:0/2232534545 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd004004810 0x7fd01816a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:03.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.232+0000 7fd00dd9b700 1 --2- 172.21.15.99:0/2232534545 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd018155d40 0x7fd0180c64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:03.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.232+0000 7fd016d9d700 1 -- 172.21.15.99:0/2232534545 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd004004810 msgr2=0x7fd01816a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:03.239 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.232+0000 7fd016d9d700 1 --2- 172.21.15.99:0/2232534545 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd004004810 0x7fd01816a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:03.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.232+0000 7fd016d9d700 1 -- 172.21.15.99:0/2232534545 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd018155d40 msgr2=0x7fd0180c64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:03.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.232+0000 7fd016d9d700 1 --2- 172.21.15.99:0/2232534545 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd018155d40 0x7fd0180c64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:03.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.232+0000 7fd016d9d700 1 -- 172.21.15.99:0/2232534545 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd01815d6c0 con 0x7fd0180c6a10 2023-10-08T00:22:03.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.233+0000 7fd016d9d700 1 --2- 172.21.15.99:0/2232534545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0180c6a10 0x7fd0180c78a0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fd0100094d0 tx=0x7fd01000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:03.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.233+0000 7fd00effd700 1 -- 172.21.15.99:0/2232534545 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01000e040 con 0x7fd0180c6a10 2023-10-08T00:22:03.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.233+0000 7fd00effd700 1 -- 172.21.15.99:0/2232534545 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd01000f040 con 0x7fd0180c6a10 2023-10-08T00:22:03.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.233+0000 7fd01cd19700 1 -- 172.21.15.99:0/2232534545 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd01815d9a0 con 0x7fd0180c6a10 2023-10-08T00:22:03.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.233+0000 7fd00effd700 1 -- 172.21.15.99:0/2232534545 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01000fd70 con 0x7fd0180c6a10 2023-10-08T00:22:03.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.233+0000 7fd01cd19700 1 -- 172.21.15.99:0/2232534545 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0180cd520 con 0x7fd0180c6a10 2023-10-08T00:22:03.240 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.234+0000 7fd01cd19700 1 -- 172.21.15.99:0/2232534545 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd01815aa50 con 0x7fd0180c6a10 2023-10-08T00:22:03.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.234+0000 7fd00effd700 1 -- 172.21.15.99:0/2232534545 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd010015640 con 0x7fd0180c6a10 2023-10-08T00:22:03.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.235+0000 7fd00effd700 1 --2- 172.21.15.99:0/2232534545 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcff8066f30 0x7fcff80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:03.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.235+0000 7fd00effd700 1 -- 172.21.15.99:0/2232534545 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5734+0+0 (secure 0 0 0) 0x7fd010084ca0 con 0x7fd0180c6a10 2023-10-08T00:22:03.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.235+0000 7fd01659c700 1 --2- 172.21.15.99:0/2232534545 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcff8066f30 0x7fcff80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:03.241 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.235+0000 7fd01659c700 1 --2- 172.21.15.99:0/2232534545 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcff8066f30 0x7fcff80693e0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd008001200 tx=0x7fd008001df0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:03.242 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.239+0000 7fd00effd700 1 -- 172.21.15.99:0/2232534545 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd010051ef0 con 0x7fd0180c6a10 2023-10-08T00:22:03.422 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.421+0000 7fd01cd19700 1 -- 172.21.15.99:0/2232534545 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fd01804c3c0 con 0x7fd0180c6a10 2023-10-08T00:22:03.423 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.422+0000 7fd00effd700 1 -- 172.21.15.99:0/2232534545 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v189) v1 ==== 81+0+32474 (secure 0 0 0) 0x7fd010055510 con 0x7fd0180c6a10 2023-10-08T00:22:03.423 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:22:03.424 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:22:03.424 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 189, 2023-10-08T00:22:03.424 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", 2023-10-08T00:22:03.424 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2023-10-08T00:10:36.303601+0000", 2023-10-08T00:22:03.424 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2023-10-08T00:22:01.171728+0000", 2023-10-08T00:22:03.424 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2023-10-08T00:10:41.709171+0000", 2023-10-08T00:22:03.424 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2023-10-08T00:10:38.690404+0000", 2023-10-08T00:22:03.425 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-08T00:22:03.425 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2023-10-08T00:22:03.425 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2023-10-08T00:22:03.425 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2023-10-08T00:22:03.425 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2023-10-08T00:22:03.425 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2023-10-08T00:22:03.425 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2023-10-08T00:22:03.426 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:22:03.426 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2023-10-08T00:22:03.426 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2023-10-08T00:22:03.426 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-08T00:22:03.426 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-08T00:22:03.426 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2023-10-08T00:22:03.426 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 23, 2023-10-08T00:22:03.426 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2023-10-08T00:22:03.427 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2023-10-08T00:22:03.427 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "luminous", 2023-10-08T00:22:03.427 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "quincy", 2023-10-08T00:22:03.427 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2023-10-08T00:22:03.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.427 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2023-10-08T00:22:03.427 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2023-10-08T00:22:03.427 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:10:42.063344+0000", 2023-10-08T00:22:03.427 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:22:03.427 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:22:03.428 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:22:03.428 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:22:03.428 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:22:03.428 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:22:03.428 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:22:03.428 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:22:03.428 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:22:03.428 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:22:03.428 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:22:03.428 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:22:03.429 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2023-10-08T00:22:03.429 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2023-10-08T00:22:03.429 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2023-10-08T00:22:03.429 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2023-10-08T00:22:03.429 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2023-10-08T00:22:03.429 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:22:03.429 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:22:03.429 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:22:03.429 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:22:03.429 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:22:03.430 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:22:03.430 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:22:03.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.430 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "112", 2023-10-08T00:22:03.430 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:22:03.430 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:22:03.430 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:22:03.430 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:22:03.430 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:22:03.431 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:22:03.431 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:22:03.431 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:22:03.431 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:22:03.431 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:22:03.431 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:22:03.431 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:22:03.431 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:22:03.431 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:22:03.432 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:22:03.432 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:22:03.432 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:22:03.432 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:22:03.432 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:22:03.432 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:22:03.432 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:22:03.432 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:22:03.432 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:22:03.432 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:22:03.433 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:22:03.433 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:22:03.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.433 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:22:03.433 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:22:03.433 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:22:03.433 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:22:03.433 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:22:03.433 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:22:03.434 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:22:03.434 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:22:03.434 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:22:03.434 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:22:03.434 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:22:03.434 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:22:03.434 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2023-10-08T00:22:03.434 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2023-10-08T00:22:03.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.435 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:22:03.435 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2023-10-08T00:22:03.435 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.435 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 22, 2023-10-08T00:22:03.435 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2023-10-08T00:22:03.435 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:21:27.984574+0000", 2023-10-08T00:22:03.435 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:22:03.436 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:22:03.436 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:22:03.436 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:22:03.436 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:22:03.436 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:22:03.436 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:22:03.436 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:22:03.436 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:22:03.437 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:22:03.437 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:22:03.437 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:22:03.437 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2023-10-08T00:22:03.437 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2023-10-08T00:22:03.437 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2023-10-08T00:22:03.437 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2023-10-08T00:22:03.437 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2023-10-08T00:22:03.437 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:22:03.438 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:22:03.438 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:22:03.438 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:22:03.438 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:22:03.438 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:22:03.438 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:22:03.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.438 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "183", 2023-10-08T00:22:03.438 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:22:03.439 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:22:03.439 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:22:03.439 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:22:03.439 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:22:03.439 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:22:03.439 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:22:03.439 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:22:03.439 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:22:03.440 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:22:03.440 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:22:03.440 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:22:03.440 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:22:03.440 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:22:03.440 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:22:03.440 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:22:03.440 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:22:03.440 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:22:03.440 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:22:03.441 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:22:03.441 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:22:03.441 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:22:03.441 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:22:03.441 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:22:03.441 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:22:03.441 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:22:03.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.441 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:22:03.441 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:22:03.442 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:22:03.442 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:22:03.442 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:22:03.442 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:22:03.442 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:22:03.442 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:22:03.442 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:22:03.442 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:22:03.442 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:22:03.443 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:22:03.443 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2023-10-08T00:22:03.443 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2023-10-08T00:22:03.443 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2023-10-08T00:22:03.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.443 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:22:03.443 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:22:03.443 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2023-10-08T00:22:03.443 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.443 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.444 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 23, 2023-10-08T00:22:03.444 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2023-10-08T00:22:03.444 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:21:29.244620+0000", 2023-10-08T00:22:03.444 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:22:03.444 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:22:03.444 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:22:03.444 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:22:03.445 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:22:03.445 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:22:03.445 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:22:03.445 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:22:03.445 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:22:03.445 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:22:03.445 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:22:03.445 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:22:03.445 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2023-10-08T00:22:03.446 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2023-10-08T00:22:03.446 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2023-10-08T00:22:03.446 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2023-10-08T00:22:03.446 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2023-10-08T00:22:03.446 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:22:03.446 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:22:03.446 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:22:03.447 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:22:03.447 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:22:03.447 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:22:03.447 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:22:03.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.447 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "189", 2023-10-08T00:22:03.447 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:22:03.447 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:22:03.447 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:22:03.448 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:22:03.448 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:22:03.448 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:22:03.448 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:22:03.448 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:22:03.448 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:22:03.448 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 33554432, 2023-10-08T00:22:03.448 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:22:03.449 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:22:03.449 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:22:03.449 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:22:03.449 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:22:03.449 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:22:03.449 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:22:03.449 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:22:03.449 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:22:03.450 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:22:03.450 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:22:03.450 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:22:03.450 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:22:03.450 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:22:03.450 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:22:03.450 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:22:03.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.451 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:22:03.451 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:22:03.451 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:22:03.451 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:22:03.451 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:22:03.451 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:22:03.451 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:22:03.451 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:22:03.452 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:22:03.452 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:22:03.452 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:22:03.452 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:22:03.452 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2023-10-08T00:22:03.452 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2023-10-08T00:22:03.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.452 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:22:03.452 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:22:03.453 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2023-10-08T00:22:03.453 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.453 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.453 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.453 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:22:03.453 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2023-10-08T00:22:03.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.453 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:22:03.453 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:22:03.453 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:22:03.454 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:22:03.454 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:22:03.454 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:22:03.454 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:22:03.454 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:22:03.454 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:22:03.454 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:22:03.454 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:22:03.454 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:22:03.455 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:22:03.455 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.455 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.455 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2023-10-08T00:22:03.455 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:22:03.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.455 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.455 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2023-10-08T00:22:03.456 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:22:03.456 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.456 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.456 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:22:03.456 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.456 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.456 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2023-10-08T00:22:03.457 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:22:03.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.457 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.457 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2023-10-08T00:22:03.457 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:22:03.457 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.457 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.457 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:22:03.458 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.458 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.458 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2023-10-08T00:22:03.458 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:22:03.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.458 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.458 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2023-10-08T00:22:03.459 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:22:03.459 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.459 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.459 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:22:03.459 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.459 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.459 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2023-10-08T00:22:03.460 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:22:03.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.460 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.460 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2023-10-08T00:22:03.460 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:22:03.460 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.460 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.461 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/86365", 2023-10-08T00:22:03.461 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/86365", 2023-10-08T00:22:03.461 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/86365", 2023-10-08T00:22:03.461 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/86365", 2023-10-08T00:22:03.461 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:22:03.461 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:22:03.461 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:22:03.461 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.462 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:22:03.462 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:22:03.462 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:22:03.462 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:22:03.462 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:22:03.463 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:22:03.463 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:22:03.463 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:22:03.463 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:22:03.463 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:22:03.463 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:22:03.463 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:22:03.463 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:22:03.463 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.464 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.464 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2023-10-08T00:22:03.464 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:22:03.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.464 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.464 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2023-10-08T00:22:03.464 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:22:03.464 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.465 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.465 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:22:03.465 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.465 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.465 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2023-10-08T00:22:03.465 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:22:03.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.466 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.466 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2023-10-08T00:22:03.466 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:22:03.466 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.466 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.466 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:22:03.466 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.467 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.467 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2023-10-08T00:22:03.467 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:22:03.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.467 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.467 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2023-10-08T00:22:03.468 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:22:03.468 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.468 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.468 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:22:03.468 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.468 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.469 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2023-10-08T00:22:03.469 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:22:03.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.469 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.469 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2023-10-08T00:22:03.469 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:22:03.469 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.470 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.470 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/86366", 2023-10-08T00:22:03.470 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/86366", 2023-10-08T00:22:03.470 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/86366", 2023-10-08T00:22:03.470 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/86366", 2023-10-08T00:22:03.470 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:22:03.470 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:22:03.471 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:22:03.471 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.471 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:22:03.471 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:22:03.471 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:22:03.471 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:22:03.472 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:22:03.472 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:22:03.472 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:22:03.472 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:22:03.472 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:22:03.472 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:22:03.472 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:22:03.472 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:22:03.472 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:22:03.472 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.473 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.473 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2023-10-08T00:22:03.473 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:22:03.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.473 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.473 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2023-10-08T00:22:03.473 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:22:03.473 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.474 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.474 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:22:03.474 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.474 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.474 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2023-10-08T00:22:03.474 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:22:03.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.475 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.475 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2023-10-08T00:22:03.475 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:22:03.475 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.475 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.475 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:22:03.475 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.475 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.476 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2023-10-08T00:22:03.476 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:22:03.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.476 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.476 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2023-10-08T00:22:03.476 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:22:03.476 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.477 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.477 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:22:03.477 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.477 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.477 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2023-10-08T00:22:03.477 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:22:03.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.478 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.478 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2023-10-08T00:22:03.478 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:22:03.478 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.478 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.478 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6817/86382", 2023-10-08T00:22:03.478 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/86382", 2023-10-08T00:22:03.478 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/86382", 2023-10-08T00:22:03.478 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/86382", 2023-10-08T00:22:03.479 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:22:03.479 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:22:03.479 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:22:03.479 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.479 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:22:03.479 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:22:03.480 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:22:03.480 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:22:03.480 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:22:03.480 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:22:03.480 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:22:03.480 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:22:03.480 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:22:03.480 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 186, 2023-10-08T00:22:03.480 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:22:03.481 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:22:03.481 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:22:03.481 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.481 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.481 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2023-10-08T00:22:03.481 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:22:03.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.482 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.482 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2023-10-08T00:22:03.482 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:22:03.482 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.482 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.482 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:22:03.482 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.483 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.483 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2023-10-08T00:22:03.483 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:22:03.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.483 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.483 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2023-10-08T00:22:03.484 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:22:03.484 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.484 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.484 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:22:03.484 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.484 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.485 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2023-10-08T00:22:03.485 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:22:03.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.485 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.485 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2023-10-08T00:22:03.485 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:22:03.485 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.485 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.486 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:22:03.486 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.486 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.486 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2023-10-08T00:22:03.486 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:22:03.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.487 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.487 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2023-10-08T00:22:03.487 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:22:03.487 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.487 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.487 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6820/86393", 2023-10-08T00:22:03.487 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/86393", 2023-10-08T00:22:03.487 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/86393", 2023-10-08T00:22:03.488 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/86393", 2023-10-08T00:22:03.488 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:22:03.488 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:22:03.488 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:22:03.488 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.488 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:22:03.488 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:22:03.488 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:22:03.489 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:22:03.489 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:22:03.489 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:22:03.489 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:22:03.489 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:22:03.489 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:22:03.489 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:22:03.489 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:22:03.489 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:22:03.489 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:22:03.490 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.490 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.490 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6800", 2023-10-08T00:22:03.490 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:22:03.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.491 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.491 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6801", 2023-10-08T00:22:03.491 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:22:03.491 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.491 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.491 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:22:03.491 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.492 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.492 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6802", 2023-10-08T00:22:03.492 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:22:03.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.492 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.493 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6803", 2023-10-08T00:22:03.493 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:22:03.493 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.493 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.493 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:22:03.493 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.494 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.494 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6806", 2023-10-08T00:22:03.494 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:22:03.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.494 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.494 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6807", 2023-10-08T00:22:03.494 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:22:03.494 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.494 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.495 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:22:03.495 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.495 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.495 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6804", 2023-10-08T00:22:03.495 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:22:03.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.495 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.496 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6805", 2023-10-08T00:22:03.496 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:22:03.496 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.496 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.496 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6801/86570", 2023-10-08T00:22:03.496 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6803/86570", 2023-10-08T00:22:03.496 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6807/86570", 2023-10-08T00:22:03.496 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6805/86570", 2023-10-08T00:22:03.497 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:22:03.497 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:22:03.497 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:22:03.497 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.497 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:22:03.497 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:22:03.497 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:22:03.498 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:22:03.498 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:22:03.498 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:22:03.498 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:22:03.498 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:22:03.498 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:22:03.498 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:22:03.498 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:22:03.498 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:22:03.498 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:22:03.499 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.499 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.499 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6816", 2023-10-08T00:22:03.499 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:22:03.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.499 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.500 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6818", 2023-10-08T00:22:03.500 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:22:03.500 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.500 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.500 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:22:03.500 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.501 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.501 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6820", 2023-10-08T00:22:03.501 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:22:03.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.501 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.501 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6821", 2023-10-08T00:22:03.501 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:22:03.501 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.501 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.502 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:22:03.502 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.502 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.502 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6826", 2023-10-08T00:22:03.502 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:22:03.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.503 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.503 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6828", 2023-10-08T00:22:03.503 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:22:03.503 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.503 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.503 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:22:03.503 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.504 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.504 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6823", 2023-10-08T00:22:03.504 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:22:03.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.504 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.504 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6825", 2023-10-08T00:22:03.504 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:22:03.504 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.504 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.505 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6818/86608", 2023-10-08T00:22:03.505 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6821/86608", 2023-10-08T00:22:03.505 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6828/86608", 2023-10-08T00:22:03.505 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6825/86608", 2023-10-08T00:22:03.505 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:22:03.505 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:22:03.505 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:22:03.505 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.506 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:22:03.506 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:22:03.506 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:22:03.506 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:22:03.506 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:22:03.506 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:22:03.506 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:22:03.506 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:22:03.506 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:22:03.506 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:22:03.507 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:22:03.507 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:22:03.507 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:22:03.507 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.507 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.507 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6808", 2023-10-08T00:22:03.507 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:22:03.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.508 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.508 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6809", 2023-10-08T00:22:03.508 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:22:03.508 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.508 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.508 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:22:03.508 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.509 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.509 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6810", 2023-10-08T00:22:03.509 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:22:03.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.509 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.509 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6811", 2023-10-08T00:22:03.509 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:22:03.509 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.509 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.510 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:22:03.510 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.510 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.510 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6814", 2023-10-08T00:22:03.510 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:22:03.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.510 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.510 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6815", 2023-10-08T00:22:03.511 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:22:03.511 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.511 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.511 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:22:03.511 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.511 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.511 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6812", 2023-10-08T00:22:03.511 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:22:03.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.512 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.512 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6813", 2023-10-08T00:22:03.512 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:22:03.512 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.512 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.512 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6809/86571", 2023-10-08T00:22:03.512 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6811/86571", 2023-10-08T00:22:03.513 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6815/86571", 2023-10-08T00:22:03.513 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6813/86571", 2023-10-08T00:22:03.513 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:22:03.513 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:22:03.513 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:22:03.513 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.513 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:22:03.513 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:22:03.513 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:22:03.514 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:22:03.514 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:22:03.514 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:22:03.514 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:22:03.514 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:22:03.514 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:22:03.514 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:22:03.514 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:22:03.514 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:22:03.514 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:22:03.515 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.515 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.515 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6817", 2023-10-08T00:22:03.515 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:22:03.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.515 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.515 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6819", 2023-10-08T00:22:03.516 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:22:03.516 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.516 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.516 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:22:03.516 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.516 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.516 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6822", 2023-10-08T00:22:03.516 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:22:03.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.517 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.517 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6824", 2023-10-08T00:22:03.517 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:22:03.517 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.517 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.517 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:22:03.517 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.518 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.518 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6830", 2023-10-08T00:22:03.518 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:22:03.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.518 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.518 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6831", 2023-10-08T00:22:03.518 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:22:03.519 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.519 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.519 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:22:03.519 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.519 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.519 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6827", 2023-10-08T00:22:03.520 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:22:03.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.520 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.520 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6829", 2023-10-08T00:22:03.520 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:22:03.520 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.521 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.521 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6819/86612", 2023-10-08T00:22:03.521 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6824/86612", 2023-10-08T00:22:03.521 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6831/86612", 2023-10-08T00:22:03.521 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6829/86612", 2023-10-08T00:22:03.521 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:22:03.522 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:22:03.522 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:22:03.522 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.522 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.522 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:22:03.522 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2023-10-08T00:22:03.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.522 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:22:03.523 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:22:03.523 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:22:03.523 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:22:03.523 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:22:03.523 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:22:03.523 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.905615+0000", 2023-10-08T00:22:03.523 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:22:03.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.524 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:22:03.524 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:22:03.524 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:22:03.524 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:22:03.524 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:22:03.524 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:22:03.524 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.880709+0000", 2023-10-08T00:22:03.524 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:22:03.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.525 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:22:03.525 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:22:03.525 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:22:03.525 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:22:03.525 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:22:03.525 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:22:03.525 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.874704+0000", 2023-10-08T00:22:03.525 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:22:03.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.526 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:22:03.526 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:22:03.526 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:22:03.526 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:22:03.526 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:22:03.526 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:22:03.526 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.899604+0000", 2023-10-08T00:22:03.526 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:22:03.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.527 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:22:03.527 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:22:03.527 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:22:03.527 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:22:03.527 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:22:03.527 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:22:03.528 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.844371+0000", 2023-10-08T00:22:03.528 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:22:03.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.528 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:22:03.528 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:22:03.528 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:22:03.528 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:22:03.528 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:22:03.528 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:22:03.529 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.794810+0000", 2023-10-08T00:22:03.529 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:22:03.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.529 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:22:03.529 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:22:03.529 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:22:03.529 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:22:03.529 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:22:03.529 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:22:03.530 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.859474+0000", 2023-10-08T00:22:03.530 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:22:03.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.530 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:22:03.530 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:22:03.530 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:22:03.530 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:22:03.530 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:22:03.530 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:22:03.531 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.893130+0000", 2023-10-08T00:22:03.531 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:22:03.531 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.531 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:22:03.531 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2023-10-08T00:22:03.531 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [ 2023-10-08T00:22:03.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.531 INFO:teuthology.orchestra.run.smithi099.stdout: "pgid": "23.18", 2023-10-08T00:22:03.531 INFO:teuthology.orchestra.run.smithi099.stdout: "mappings": [ 2023-10-08T00:22:03.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.532 INFO:teuthology.orchestra.run.smithi099.stdout: "from": 5, 2023-10-08T00:22:03.532 INFO:teuthology.orchestra.run.smithi099.stdout: "to": 3 2023-10-08T00:22:03.532 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.532 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.532 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.532 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:22:03.532 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2023-10-08T00:22:03.532 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2023-10-08T00:22:03.532 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2023-10-08T00:22:03.532 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2023-10-08T00:22:03.533 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2023-10-08T00:22:03.533 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2023-10-08T00:22:03.533 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2023-10-08T00:22:03.533 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2023-10-08T00:22:03.533 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2023-10-08T00:22:03.533 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2023-10-08T00:22:03.533 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2023-10-08T00:22:03.533 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.533 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2023-10-08T00:22:03.534 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2023-10-08T00:22:03.534 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2023-10-08T00:22:03.534 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2023-10-08T00:22:03.534 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2023-10-08T00:22:03.534 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2023-10-08T00:22:03.534 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2023-10-08T00:22:03.534 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2023-10-08T00:22:03.534 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2023-10-08T00:22:03.534 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2023-10-08T00:22:03.535 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2023-10-08T00:22:03.535 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.535 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:22:03.537 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.424+0000 7fd01cd19700 1 -- 172.21.15.99:0/2232534545 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcff8066f30 msgr2=0x7fcff80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:03.537 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.424+0000 7fd01cd19700 1 --2- 172.21.15.99:0/2232534545 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcff8066f30 0x7fcff80693e0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd008001200 tx=0x7fd008001df0 comp rx=0 tx=0).stop 2023-10-08T00:22:03.537 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.424+0000 7fd01cd19700 1 -- 172.21.15.99:0/2232534545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0180c6a10 msgr2=0x7fd0180c78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:03.537 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.424+0000 7fd01cd19700 1 --2- 172.21.15.99:0/2232534545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0180c6a10 0x7fd0180c78a0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fd0100094d0 tx=0x7fd01000b040 comp rx=0 tx=0).stop 2023-10-08T00:22:03.537 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.425+0000 7fd01cd19700 1 -- 172.21.15.99:0/2232534545 shutdown_connections 2023-10-08T00:22:03.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.425+0000 7fd01cd19700 1 --2- 172.21.15.99:0/2232534545 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fcff8066f30 0x7fcff80693e0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:03.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.425+0000 7fd01cd19700 1 --2- 172.21.15.99:0/2232534545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0180c6a10 0x7fd0180c78a0 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:03.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.425+0000 7fd01cd19700 1 --2- 172.21.15.99:0/2232534545 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd004004810 0x7fd01816a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:03.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.425+0000 7fd01cd19700 1 --2- 172.21.15.99:0/2232534545 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd018155d40 0x7fd0180c64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:03.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.425+0000 7fd01cd19700 1 -- 172.21.15.99:0/2232534545 >> 172.21.15.99:0/2232534545 conn(0x7fd018068ce0 msgr2=0x7fd0180d0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:03.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.425+0000 7fd01cd19700 1 -- 172.21.15.99:0/2232534545 shutdown_connections 2023-10-08T00:22:03.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.425+0000 7fd01cd19700 1 -- 172.21.15.99:0/2232534545 wait complete. 2023-10-08T00:22:03.539 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2023-10-08T00:22:03.539 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-08T00:22:03.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.653+0000 7f5bf8986700 1 Processor -- start 2023-10-08T00:22:03.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.653+0000 7f5bf8986700 1 -- start start 2023-10-08T00:22:03.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.654+0000 7f5bf8986700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f5bf40b0890 0x7f5bf40cb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:03.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.654+0000 7f5bf8986700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bf40cb370 con 0x7f5bf4067d70 2023-10-08T00:22:03.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.654+0000 7f5bf8986700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bf40b0c60 con 0x7f5bf40cbd40 2023-10-08T00:22:03.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.654+0000 7f5bf8986700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bf406a1e0 con 0x7f5bf40b0890 2023-10-08T00:22:03.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.654+0000 7f5bf2d9d700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f5bf4067d70 0x7f5bf4159790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56776/0 (socket says 172.21.15.99:56776) 2023-10-08T00:22:03.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.654+0000 7f5bf2d9d700 1 -- 172.21.15.99:0/2726227674 learned_addr learned my addr 172.21.15.99:0/2726227674 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:22:03.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.654+0000 7f5bf259c700 1 --2- 172.21.15.99:0/2726227674 >> v2:172.21.15.191:3300/0 conn(0x7f5bf40b0890 0x7f5bf40cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:03.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.655+0000 7f5bf1d9b700 1 -- 172.21.15.99:0/2726227674 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 42742058 0 0) 0x7f5bf40cb370 con 0x7f5bf4067d70 2023-10-08T00:22:03.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.655+0000 7f5bf1d9b700 1 -- 172.21.15.99:0/2726227674 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5be0003570 con 0x7f5bf4067d70 2023-10-08T00:22:03.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.655+0000 7f5bf259c700 1 -- 172.21.15.99:0/2726227674 >> v1:172.21.15.191:6789/0 conn(0x7f5bf40cbd40 legacy=0x7f5bf4067660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:03.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.655+0000 7f5bf1d9b700 1 -- 172.21.15.99:0/2726227674 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3825846883 0 0) 0x7f5bf40b0c60 con 0x7f5bf40cbd40 2023-10-08T00:22:03.658 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.655+0000 7f5bf259c700 1 -- 172.21.15.99:0/2726227674 >> v1:172.21.15.99:6789/0 conn(0x7f5bf4067d70 legacy=0x7f5bf4159790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:03.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.655+0000 7f5bf259c700 1 -- 172.21.15.99:0/2726227674 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bf406cb80 con 0x7f5bf40b0890 2023-10-08T00:22:03.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.655+0000 7f5bf259c700 1 --2- 172.21.15.99:0/2726227674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5bf40b0890 0x7f5bf40cb800 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f5be40098e0 tx=0x7f5be40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e5497caee2218a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:03.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.655+0000 7f5bf1d9b700 1 -- 172.21.15.99:0/2726227674 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be400b640 con 0x7f5bf40b0890 2023-10-08T00:22:03.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.655+0000 7f5bf1d9b700 1 -- 172.21.15.99:0/2726227674 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5be400b7a0 con 0x7f5bf40b0890 2023-10-08T00:22:03.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.655+0000 7f5bf1d9b700 1 -- 172.21.15.99:0/2726227674 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be400bb40 con 0x7f5bf40b0890 2023-10-08T00:22:03.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.655+0000 7f5bf8986700 1 -- 172.21.15.99:0/2726227674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5bf40b0890 msgr2=0x7f5bf40cb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:03.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.655+0000 7f5bf8986700 1 --2- 172.21.15.99:0/2726227674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5bf40b0890 0x7f5bf40cb800 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f5be40098e0 tx=0x7f5be40091a0 comp rx=0 tx=0).stop 2023-10-08T00:22:03.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.656+0000 7f5bf8986700 1 -- 172.21.15.99:0/2726227674 shutdown_connections 2023-10-08T00:22:03.659 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.656+0000 7f5bf8986700 1 --2- 172.21.15.99:0/2726227674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5bf40b0890 0x7f5bf40cb800 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:03.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.656+0000 7f5bf8986700 1 -- 172.21.15.99:0/2726227674 >> 172.21.15.99:0/2726227674 conn(0x7f5bf406fe50 msgr2=0x7f5bf4069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:03.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.656+0000 7f5bf8986700 1 -- 172.21.15.99:0/2726227674 shutdown_connections 2023-10-08T00:22:03.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.656+0000 7f5bf8986700 1 -- 172.21.15.99:0/2726227674 wait complete. 2023-10-08T00:22:03.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.657+0000 7f5bf8986700 1 Processor -- start 2023-10-08T00:22:03.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.657+0000 7f5bf8986700 1 -- start start 2023-10-08T00:22:03.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.657+0000 7f5bf8986700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5bf4067d70 0x7f5bf4149b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:03.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.657+0000 7f5bf8986700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5bf40b0890 0x7f5bf40c7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:03.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.657+0000 7f5bf8986700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5bf40cbd40 0x7f5bf406bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:03.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.657+0000 7f5bf8986700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf416b720 con 0x7f5bf40b0890 2023-10-08T00:22:03.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.657+0000 7f5bf8986700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf416ba00 con 0x7f5bf40cbd40 2023-10-08T00:22:03.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.657+0000 7f5bf8986700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf40c8260 con 0x7f5bf4067d70 2023-10-08T00:22:03.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.658+0000 7f5be9d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5bf40b0890 0x7f5bf40c7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:03.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.658+0000 7f5be9d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5bf40b0890 0x7f5bf40c7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:32844/0 (socket says 172.21.15.99:32844) 2023-10-08T00:22:03.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.658+0000 7f5be9d9b700 1 -- 172.21.15.99:0/3659770010 learned_addr learned my addr 172.21.15.99:0/3659770010 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:03.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.658+0000 7f5bf2d9d700 1 --2- 172.21.15.99:0/3659770010 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5bf40cbd40 0x7f5bf406bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:03.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.658+0000 7f5bf259c700 1 --2- 172.21.15.99:0/3659770010 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5bf4067d70 0x7f5bf4149b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:03.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.658+0000 7f5be9d9b700 1 -- 172.21.15.99:0/3659770010 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5bf4067d70 msgr2=0x7f5bf4149b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:03.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.658+0000 7f5be9d9b700 1 --2- 172.21.15.99:0/3659770010 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5bf4067d70 0x7f5bf4149b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:03.664 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.658+0000 7f5be9d9b700 1 -- 172.21.15.99:0/3659770010 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5bf40cbd40 msgr2=0x7f5bf406bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:03.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.658+0000 7f5be9d9b700 1 --2- 172.21.15.99:0/3659770010 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5bf40cbd40 0x7f5bf406bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:03.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.658+0000 7f5be9d9b700 1 -- 172.21.15.99:0/3659770010 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be4009420 con 0x7f5bf40b0890 2023-10-08T00:22:03.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.658+0000 7f5bf2d9d700 1 --2- 172.21.15.99:0/3659770010 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5bf40cbd40 0x7f5bf406bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:22:03.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.658+0000 7f5be9d9b700 1 --2- 172.21.15.99:0/3659770010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5bf40b0890 0x7f5bf40c7830 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc00c810 tx=0x7f5bdc007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:03.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.658+0000 7f5beaffd700 1 -- 172.21.15.99:0/3659770010 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc012700 con 0x7f5bf40b0890 2023-10-08T00:22:03.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.658+0000 7f5beaffd700 1 -- 172.21.15.99:0/3659770010 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bdc0128a0 con 0x7f5bf40b0890 2023-10-08T00:22:03.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.658+0000 7f5beaffd700 1 -- 172.21.15.99:0/3659770010 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc012c60 con 0x7f5bf40b0890 2023-10-08T00:22:03.665 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.658+0000 7f5bf8986700 1 -- 172.21.15.99:0/3659770010 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bf40c8540 con 0x7f5bf40b0890 2023-10-08T00:22:03.666 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.658+0000 7f5bf8986700 1 -- 172.21.15.99:0/3659770010 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bf416b2e0 con 0x7f5bf40b0890 2023-10-08T00:22:03.666 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.659+0000 7f5bf8986700 1 -- 172.21.15.99:0/3659770010 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bf404c3c0 con 0x7f5bf40b0890 2023-10-08T00:22:03.666 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.660+0000 7f5beaffd700 1 -- 172.21.15.99:0/3659770010 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5bdc01b400 con 0x7f5bf40b0890 2023-10-08T00:22:03.666 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.660+0000 7f5beaffd700 1 --2- 172.21.15.99:0/3659770010 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5bd4066f30 0x7f5bd40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:03.666 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.660+0000 7f5beaffd700 1 -- 172.21.15.99:0/3659770010 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5734+0+0 (secure 0 0 0) 0x7f5bdc0841c0 con 0x7f5bf40b0890 2023-10-08T00:22:03.666 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.660+0000 7f5bf259c700 1 --2- 172.21.15.99:0/3659770010 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5bd4066f30 0x7f5bd40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:03.666 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.661+0000 7f5bf259c700 1 --2- 172.21.15.99:0/3659770010 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5bd4066f30 0x7f5bd40693e0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f5be40093f0 tx=0x7f5be4017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:03.667 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.664+0000 7f5beaffd700 1 -- 172.21.15.99:0/3659770010 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5bdc051410 con 0x7f5bf40b0890 2023-10-08T00:22:03.846 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.846+0000 7f5bf8986700 1 -- 172.21.15.99:0/3659770010 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f5bf416bb40 con 0x7f5bf40b0890 2023-10-08T00:22:03.847 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.847+0000 7f5beaffd700 1 -- 172.21.15.99:0/3659770010 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v189) v1 ==== 81+0+32474 (secure 0 0 0) 0x7f5bdc054a30 con 0x7f5bf40b0890 2023-10-08T00:22:03.848 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:22:03.848 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:22:03.848 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 189, 2023-10-08T00:22:03.848 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", 2023-10-08T00:22:03.848 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2023-10-08T00:10:36.303601+0000", 2023-10-08T00:22:03.848 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2023-10-08T00:22:01.171728+0000", 2023-10-08T00:22:03.848 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2023-10-08T00:10:41.709171+0000", 2023-10-08T00:22:03.848 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2023-10-08T00:10:38.690404+0000", 2023-10-08T00:22:03.848 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-08T00:22:03.849 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2023-10-08T00:22:03.849 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2023-10-08T00:22:03.849 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2023-10-08T00:22:03.849 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2023-10-08T00:22:03.849 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2023-10-08T00:22:03.849 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2023-10-08T00:22:03.849 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:22:03.849 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2023-10-08T00:22:03.850 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2023-10-08T00:22:03.850 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-08T00:22:03.850 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-08T00:22:03.850 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2023-10-08T00:22:03.850 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 23, 2023-10-08T00:22:03.850 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2023-10-08T00:22:03.851 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2023-10-08T00:22:03.851 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "luminous", 2023-10-08T00:22:03.851 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "quincy", 2023-10-08T00:22:03.851 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2023-10-08T00:22:03.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.851 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2023-10-08T00:22:03.851 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2023-10-08T00:22:03.851 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:10:42.063344+0000", 2023-10-08T00:22:03.851 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:22:03.852 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:22:03.852 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:22:03.852 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:22:03.852 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:22:03.852 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:22:03.852 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:22:03.852 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:22:03.852 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:22:03.852 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:22:03.852 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:22:03.853 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:22:03.853 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2023-10-08T00:22:03.853 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2023-10-08T00:22:03.853 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2023-10-08T00:22:03.853 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2023-10-08T00:22:03.853 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2023-10-08T00:22:03.853 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:22:03.853 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:22:03.853 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:22:03.853 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:22:03.854 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:22:03.854 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:22:03.854 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:22:03.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.854 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "112", 2023-10-08T00:22:03.854 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:22:03.854 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:22:03.854 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:22:03.854 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:22:03.855 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:22:03.855 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:22:03.855 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:22:03.855 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:22:03.855 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:22:03.855 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:22:03.855 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:22:03.855 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:22:03.856 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:22:03.856 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:22:03.856 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:22:03.856 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:22:03.856 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:22:03.856 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:22:03.856 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:22:03.856 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:22:03.856 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:22:03.857 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:22:03.857 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:22:03.857 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:22:03.857 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:22:03.857 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:22:03.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.857 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:22:03.857 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:22:03.857 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:22:03.858 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:22:03.858 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:22:03.858 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:22:03.858 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:22:03.858 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:22:03.858 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:22:03.858 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:22:03.858 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:22:03.858 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:22:03.859 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2023-10-08T00:22:03.859 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2023-10-08T00:22:03.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.859 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:22:03.859 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2023-10-08T00:22:03.859 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.859 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 22, 2023-10-08T00:22:03.859 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2023-10-08T00:22:03.859 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:21:27.984574+0000", 2023-10-08T00:22:03.860 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:22:03.860 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:22:03.860 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:22:03.860 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:22:03.860 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:22:03.860 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:22:03.860 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:22:03.860 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:22:03.860 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:22:03.860 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:22:03.860 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:22:03.861 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:22:03.861 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2023-10-08T00:22:03.861 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2023-10-08T00:22:03.861 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2023-10-08T00:22:03.861 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2023-10-08T00:22:03.861 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2023-10-08T00:22:03.861 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:22:03.861 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:22:03.861 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:22:03.861 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:22:03.862 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:22:03.862 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:22:03.862 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:22:03.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.862 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "183", 2023-10-08T00:22:03.863 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:22:03.863 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:22:03.863 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:22:03.863 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:22:03.864 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:22:03.864 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:22:03.864 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:22:03.864 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:22:03.864 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:22:03.865 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2023-10-08T00:22:03.865 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:22:03.865 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:22:03.865 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:22:03.866 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:22:03.866 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:22:03.866 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:22:03.866 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:22:03.866 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:22:03.867 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:22:03.867 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:22:03.867 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:22:03.867 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:22:03.867 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:22:03.868 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:22:03.868 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:22:03.868 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:22:03.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.868 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:22:03.869 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:22:03.869 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:22:03.869 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:22:03.869 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:22:03.870 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:22:03.870 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:22:03.870 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:22:03.870 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:22:03.870 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:22:03.870 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:22:03.870 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:22:03.870 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2023-10-08T00:22:03.870 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2023-10-08T00:22:03.871 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2023-10-08T00:22:03.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.871 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:22:03.871 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:22:03.871 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2023-10-08T00:22:03.871 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.871 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.871 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 23, 2023-10-08T00:22:03.871 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2023-10-08T00:22:03.872 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2023-10-08T00:21:29.244620+0000", 2023-10-08T00:22:03.872 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2023-10-08T00:22:03.872 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2023-10-08T00:22:03.872 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2023-10-08T00:22:03.872 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2023-10-08T00:22:03.872 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2023-10-08T00:22:03.872 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2023-10-08T00:22:03.872 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2023-10-08T00:22:03.872 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2023-10-08T00:22:03.872 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2023-10-08T00:22:03.872 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-08T00:22:03.872 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2023-10-08T00:22:03.873 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2023-10-08T00:22:03.873 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2023-10-08T00:22:03.873 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2023-10-08T00:22:03.873 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2023-10-08T00:22:03.873 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2023-10-08T00:22:03.873 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2023-10-08T00:22:03.873 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2023-10-08T00:22:03.873 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2023-10-08T00:22:03.873 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2023-10-08T00:22:03.873 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2023-10-08T00:22:03.873 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2023-10-08T00:22:03.873 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2023-10-08T00:22:03.874 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2023-10-08T00:22:03.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.874 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "189", 2023-10-08T00:22:03.874 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2023-10-08T00:22:03.874 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-08T00:22:03.874 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2023-10-08T00:22:03.874 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2023-10-08T00:22:03.874 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2023-10-08T00:22:03.874 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2023-10-08T00:22:03.874 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2023-10-08T00:22:03.874 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2023-10-08T00:22:03.875 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2023-10-08T00:22:03.875 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 33554432, 2023-10-08T00:22:03.875 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2023-10-08T00:22:03.875 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2023-10-08T00:22:03.875 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2023-10-08T00:22:03.875 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2023-10-08T00:22:03.875 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2023-10-08T00:22:03.875 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2023-10-08T00:22:03.875 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2023-10-08T00:22:03.875 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2023-10-08T00:22:03.875 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-08T00:22:03.875 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-08T00:22:03.876 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-08T00:22:03.876 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2023-10-08T00:22:03.876 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2023-10-08T00:22:03.876 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2023-10-08T00:22:03.876 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2023-10-08T00:22:03.876 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2023-10-08T00:22:03.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.876 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2023-10-08T00:22:03.876 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2023-10-08T00:22:03.876 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2023-10-08T00:22:03.876 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2023-10-08T00:22:03.876 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2023-10-08T00:22:03.877 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2023-10-08T00:22:03.877 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2023-10-08T00:22:03.877 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2023-10-08T00:22:03.877 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2023-10-08T00:22:03.877 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2023-10-08T00:22:03.877 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2023-10-08T00:22:03.877 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2023-10-08T00:22:03.877 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2023-10-08T00:22:03.877 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2023-10-08T00:22:03.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.877 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2023-10-08T00:22:03.877 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2023-10-08T00:22:03.878 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2023-10-08T00:22:03.878 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.878 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.878 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.878 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:22:03.878 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2023-10-08T00:22:03.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.878 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:22:03.878 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:22:03.878 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:22:03.878 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:22:03.878 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:22:03.879 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:22:03.879 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:22:03.879 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:22:03.879 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:22:03.879 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:22:03.879 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:22:03.879 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:22:03.879 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:22:03.879 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.880 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.880 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2023-10-08T00:22:03.880 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:22:03.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.880 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.880 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2023-10-08T00:22:03.880 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:22:03.880 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.880 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.880 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:22:03.881 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.881 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.881 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2023-10-08T00:22:03.881 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:22:03.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.881 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.881 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2023-10-08T00:22:03.881 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:22:03.881 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.882 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.882 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:22:03.882 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.882 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.882 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2023-10-08T00:22:03.882 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:22:03.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.883 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.883 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2023-10-08T00:22:03.883 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:22:03.883 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.883 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.883 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:22:03.883 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.883 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.883 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2023-10-08T00:22:03.883 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:22:03.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.884 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.884 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2023-10-08T00:22:03.884 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86365 2023-10-08T00:22:03.884 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.884 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.884 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/86365", 2023-10-08T00:22:03.884 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/86365", 2023-10-08T00:22:03.884 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/86365", 2023-10-08T00:22:03.884 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/86365", 2023-10-08T00:22:03.885 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:22:03.885 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:22:03.885 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:22:03.885 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.885 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:22:03.885 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:22:03.885 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:22:03.885 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:22:03.885 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:22:03.885 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:22:03.886 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:22:03.886 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:22:03.886 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:22:03.886 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:22:03.886 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:22:03.886 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:22:03.886 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:22:03.886 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.886 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.886 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.887 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2023-10-08T00:22:03.887 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:22:03.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.887 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.887 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2023-10-08T00:22:03.887 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:22:03.887 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.887 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.887 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:22:03.888 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.888 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.888 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2023-10-08T00:22:03.888 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:22:03.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.888 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.888 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2023-10-08T00:22:03.888 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:22:03.888 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.889 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.889 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:22:03.889 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.889 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.889 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2023-10-08T00:22:03.889 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:22:03.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.889 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.889 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2023-10-08T00:22:03.890 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:22:03.890 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.890 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.890 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:22:03.890 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.890 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.890 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2023-10-08T00:22:03.890 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:22:03.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.891 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.891 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2023-10-08T00:22:03.891 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86366 2023-10-08T00:22:03.891 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.891 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.891 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/86366", 2023-10-08T00:22:03.891 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/86366", 2023-10-08T00:22:03.891 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/86366", 2023-10-08T00:22:03.891 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/86366", 2023-10-08T00:22:03.892 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:22:03.892 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:22:03.892 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:22:03.892 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.892 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:22:03.892 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:22:03.892 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:22:03.892 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:22:03.893 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:22:03.893 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:22:03.893 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:22:03.893 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:22:03.893 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:22:03.893 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:22:03.893 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:22:03.894 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:22:03.894 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:22:03.894 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.894 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.894 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.894 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2023-10-08T00:22:03.894 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:22:03.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.895 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.895 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2023-10-08T00:22:03.895 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:22:03.895 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.895 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.895 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:22:03.896 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.896 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.896 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2023-10-08T00:22:03.896 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:22:03.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.897 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.898 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2023-10-08T00:22:03.898 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:22:03.898 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.898 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.898 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:22:03.898 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.898 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.898 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2023-10-08T00:22:03.899 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:22:03.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.899 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.899 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2023-10-08T00:22:03.899 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:22:03.899 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.899 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.899 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:22:03.899 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.900 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.900 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2023-10-08T00:22:03.900 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:22:03.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.900 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.900 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2023-10-08T00:22:03.900 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86382 2023-10-08T00:22:03.900 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.900 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.901 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6817/86382", 2023-10-08T00:22:03.901 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/86382", 2023-10-08T00:22:03.901 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/86382", 2023-10-08T00:22:03.901 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/86382", 2023-10-08T00:22:03.901 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:22:03.901 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:22:03.901 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:22:03.901 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.901 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:22:03.902 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:22:03.902 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:22:03.902 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:22:03.902 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:22:03.902 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:22:03.902 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:22:03.902 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:22:03.902 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:22:03.902 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 186, 2023-10-08T00:22:03.902 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:22:03.902 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:22:03.902 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:22:03.903 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.903 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.903 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2023-10-08T00:22:03.903 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:22:03.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.903 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.903 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2023-10-08T00:22:03.903 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:22:03.903 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.903 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.904 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:22:03.904 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.904 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.904 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2023-10-08T00:22:03.904 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:22:03.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.904 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.904 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2023-10-08T00:22:03.905 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:22:03.905 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.905 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.905 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:22:03.905 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.905 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.905 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2023-10-08T00:22:03.905 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:22:03.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.906 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.906 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2023-10-08T00:22:03.906 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:22:03.906 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.906 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.906 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:22:03.906 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.906 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.907 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2023-10-08T00:22:03.907 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:22:03.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.907 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.907 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2023-10-08T00:22:03.907 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86393 2023-10-08T00:22:03.907 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.907 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.907 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6820/86393", 2023-10-08T00:22:03.907 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/86393", 2023-10-08T00:22:03.908 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/86393", 2023-10-08T00:22:03.908 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/86393", 2023-10-08T00:22:03.908 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:22:03.908 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:22:03.908 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:22:03.908 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.909 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:22:03.909 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:22:03.909 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:22:03.909 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:22:03.909 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:22:03.909 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:22:03.909 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:22:03.909 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:22:03.909 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:22:03.909 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:22:03.909 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:22:03.909 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:22:03.910 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:22:03.910 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.910 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.910 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6800", 2023-10-08T00:22:03.910 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:22:03.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.910 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.910 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6801", 2023-10-08T00:22:03.910 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:22:03.910 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.911 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.911 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:22:03.911 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.911 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.911 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6802", 2023-10-08T00:22:03.911 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:22:03.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.911 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.911 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6803", 2023-10-08T00:22:03.912 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:22:03.912 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.912 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.912 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:22:03.912 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.912 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.912 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6806", 2023-10-08T00:22:03.912 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:22:03.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.913 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.913 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6807", 2023-10-08T00:22:03.913 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:22:03.913 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.913 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.913 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:22:03.913 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.913 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.913 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6804", 2023-10-08T00:22:03.913 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:22:03.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.914 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.914 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6805", 2023-10-08T00:22:03.914 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86570 2023-10-08T00:22:03.914 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.914 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.914 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6801/86570", 2023-10-08T00:22:03.914 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6803/86570", 2023-10-08T00:22:03.914 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6807/86570", 2023-10-08T00:22:03.914 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6805/86570", 2023-10-08T00:22:03.915 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:22:03.915 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:22:03.915 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:22:03.915 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.915 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:22:03.915 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:22:03.915 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:22:03.915 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:22:03.915 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:22:03.915 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:22:03.916 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:22:03.916 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:22:03.916 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:22:03.916 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:22:03.916 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:22:03.916 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:22:03.916 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:22:03.916 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.916 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.916 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6816", 2023-10-08T00:22:03.916 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:22:03.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.917 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.917 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6818", 2023-10-08T00:22:03.917 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:22:03.917 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.917 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.917 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:22:03.917 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.917 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.918 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6820", 2023-10-08T00:22:03.918 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:22:03.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.918 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.918 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6821", 2023-10-08T00:22:03.918 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:22:03.918 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.918 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.918 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:22:03.918 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.919 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.919 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6826", 2023-10-08T00:22:03.919 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:22:03.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.919 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.919 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6828", 2023-10-08T00:22:03.919 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:22:03.919 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.919 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.920 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:22:03.920 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.920 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.920 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6823", 2023-10-08T00:22:03.920 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:22:03.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.920 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.920 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6825", 2023-10-08T00:22:03.920 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86608 2023-10-08T00:22:03.920 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.921 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.921 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6818/86608", 2023-10-08T00:22:03.921 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6821/86608", 2023-10-08T00:22:03.921 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6828/86608", 2023-10-08T00:22:03.921 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6825/86608", 2023-10-08T00:22:03.921 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:22:03.921 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:22:03.921 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:22:03.921 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.922 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:22:03.922 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:22:03.922 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:22:03.922 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:22:03.922 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:22:03.922 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:22:03.922 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:22:03.922 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:22:03.922 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:22:03.922 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:22:03.922 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:22:03.922 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:22:03.923 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:22:03.923 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.923 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.923 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6808", 2023-10-08T00:22:03.923 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:22:03.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.923 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.923 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6809", 2023-10-08T00:22:03.923 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:22:03.923 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.924 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.924 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:22:03.924 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.924 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.924 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6810", 2023-10-08T00:22:03.924 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:22:03.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.924 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.924 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6811", 2023-10-08T00:22:03.925 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:22:03.925 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.925 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.925 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:22:03.925 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.925 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.925 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6814", 2023-10-08T00:22:03.925 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:22:03.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.926 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.926 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6815", 2023-10-08T00:22:03.926 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:22:03.926 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.926 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.926 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:22:03.926 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.926 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.926 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6812", 2023-10-08T00:22:03.926 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:22:03.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.927 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.927 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6813", 2023-10-08T00:22:03.927 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86571 2023-10-08T00:22:03.927 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.927 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.927 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6809/86571", 2023-10-08T00:22:03.927 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6811/86571", 2023-10-08T00:22:03.927 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6815/86571", 2023-10-08T00:22:03.927 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6813/86571", 2023-10-08T00:22:03.928 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:22:03.928 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:22:03.928 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:22:03.928 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.928 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:22:03.928 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-08T00:22:03.928 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2023-10-08T00:22:03.928 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2023-10-08T00:22:03.928 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2023-10-08T00:22:03.928 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2023-10-08T00:22:03.929 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2023-10-08T00:22:03.929 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2023-10-08T00:22:03.929 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2023-10-08T00:22:03.929 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 182, 2023-10-08T00:22:03.929 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2023-10-08T00:22:03.929 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2023-10-08T00:22:03.929 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2023-10-08T00:22:03.929 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.929 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.929 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6817", 2023-10-08T00:22:03.929 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:22:03.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.930 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.930 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6819", 2023-10-08T00:22:03.930 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:22:03.930 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.930 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.930 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2023-10-08T00:22:03.930 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.930 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.931 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6822", 2023-10-08T00:22:03.931 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:22:03.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.931 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.931 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6824", 2023-10-08T00:22:03.931 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:22:03.931 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.931 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.931 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2023-10-08T00:22:03.931 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.932 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.932 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6830", 2023-10-08T00:22:03.932 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:22:03.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.932 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.932 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6831", 2023-10-08T00:22:03.932 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:22:03.932 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.932 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.933 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2023-10-08T00:22:03.933 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2023-10-08T00:22:03.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.933 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2023-10-08T00:22:03.933 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6827", 2023-10-08T00:22:03.933 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:22:03.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.933 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-08T00:22:03.933 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.191:6829", 2023-10-08T00:22:03.933 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 86612 2023-10-08T00:22:03.933 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.933 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.934 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.191:6819/86612", 2023-10-08T00:22:03.934 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.191:6824/86612", 2023-10-08T00:22:03.934 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.191:6831/86612", 2023-10-08T00:22:03.934 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.191:6829/86612", 2023-10-08T00:22:03.934 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2023-10-08T00:22:03.934 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2023-10-08T00:22:03.934 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2023-10-08T00:22:03.934 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.934 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.934 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:22:03.934 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2023-10-08T00:22:03.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.935 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2023-10-08T00:22:03.935 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:22:03.935 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:22:03.935 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:22:03.935 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:22:03.935 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:22:03.935 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.905615+0000", 2023-10-08T00:22:03.935 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:22:03.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.935 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2023-10-08T00:22:03.936 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:22:03.936 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:22:03.936 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:22:03.936 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:22:03.936 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:22:03.936 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.880709+0000", 2023-10-08T00:22:03.936 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:22:03.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.936 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2023-10-08T00:22:03.936 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:22:03.936 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:22:03.937 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:22:03.937 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:22:03.937 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:22:03.937 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.874704+0000", 2023-10-08T00:22:03.937 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:22:03.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.937 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2023-10-08T00:22:03.937 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:22:03.938 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:22:03.938 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:22:03.938 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:22:03.938 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:22:03.938 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.899604+0000", 2023-10-08T00:22:03.938 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:22:03.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.938 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2023-10-08T00:22:03.938 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:22:03.938 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:22:03.938 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:22:03.939 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:22:03.939 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:22:03.939 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.844371+0000", 2023-10-08T00:22:03.939 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:22:03.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.939 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2023-10-08T00:22:03.939 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:22:03.939 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:22:03.939 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:22:03.939 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:22:03.939 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:22:03.940 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.794810+0000", 2023-10-08T00:22:03.940 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:22:03.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.940 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2023-10-08T00:22:03.940 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:22:03.940 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:22:03.940 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:22:03.940 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:22:03.940 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:22:03.940 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.859474+0000", 2023-10-08T00:22:03.940 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:22:03.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.941 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2023-10-08T00:22:03.941 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2023-10-08T00:22:03.941 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2023-10-08T00:22:03.941 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2023-10-08T00:22:03.941 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540138320759226367, 2023-10-08T00:22:03.941 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2023-10-08T00:22:03.941 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2023-10-08T00:10:39.893130+0000", 2023-10-08T00:22:03.941 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2023-10-08T00:22:03.941 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.941 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:22:03.941 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2023-10-08T00:22:03.942 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [ 2023-10-08T00:22:03.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.942 INFO:teuthology.orchestra.run.smithi099.stdout: "pgid": "23.18", 2023-10-08T00:22:03.942 INFO:teuthology.orchestra.run.smithi099.stdout: "mappings": [ 2023-10-08T00:22:03.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2023-10-08T00:22:03.942 INFO:teuthology.orchestra.run.smithi099.stdout: "from": 5, 2023-10-08T00:22:03.942 INFO:teuthology.orchestra.run.smithi099.stdout: "to": 3 2023-10-08T00:22:03.942 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.942 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T00:22:03.942 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.942 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2023-10-08T00:22:03.942 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2023-10-08T00:22:03.943 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2023-10-08T00:22:03.943 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2023-10-08T00:22:03.943 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2023-10-08T00:22:03.943 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2023-10-08T00:22:03.943 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2023-10-08T00:22:03.943 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2023-10-08T00:22:03.943 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2023-10-08T00:22:03.943 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2023-10-08T00:22:03.943 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2023-10-08T00:22:03.943 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2023-10-08T00:22:03.943 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T00:22:03.944 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2023-10-08T00:22:03.944 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2023-10-08T00:22:03.944 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2023-10-08T00:22:03.944 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2023-10-08T00:22:03.944 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2023-10-08T00:22:03.944 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2023-10-08T00:22:03.944 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2023-10-08T00:22:03.944 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2023-10-08T00:22:03.944 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2023-10-08T00:22:03.944 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2023-10-08T00:22:03.944 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2023-10-08T00:22:03.944 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T00:22:03.944 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:22:03.946 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.849+0000 7f5bf8986700 1 -- 172.21.15.99:0/3659770010 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5bd4066f30 msgr2=0x7f5bd40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:03.946 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.849+0000 7f5bf8986700 1 --2- 172.21.15.99:0/3659770010 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5bd4066f30 0x7f5bd40693e0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f5be40093f0 tx=0x7f5be4017000 comp rx=0 tx=0).stop 2023-10-08T00:22:03.946 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.849+0000 7f5bf8986700 1 -- 172.21.15.99:0/3659770010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5bf40b0890 msgr2=0x7f5bf40c7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:03.946 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.849+0000 7f5bf8986700 1 --2- 172.21.15.99:0/3659770010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5bf40b0890 0x7f5bf40c7830 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc00c810 tx=0x7f5bdc007df0 comp rx=0 tx=0).stop 2023-10-08T00:22:03.946 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.849+0000 7f5bf8986700 1 -- 172.21.15.99:0/3659770010 shutdown_connections 2023-10-08T00:22:03.947 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.849+0000 7f5bf8986700 1 --2- 172.21.15.99:0/3659770010 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f5bd4066f30 0x7f5bd40693e0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:03.947 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.849+0000 7f5bf8986700 1 --2- 172.21.15.99:0/3659770010 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5bf40cbd40 0x7f5bf406bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:03.947 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.849+0000 7f5bf8986700 1 --2- 172.21.15.99:0/3659770010 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5bf4067d70 0x7f5bf4149b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:03.947 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.849+0000 7f5bf8986700 1 --2- 172.21.15.99:0/3659770010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5bf40b0890 0x7f5bf40c7830 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:03.947 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.849+0000 7f5bf8986700 1 -- 172.21.15.99:0/3659770010 >> 172.21.15.99:0/3659770010 conn(0x7f5bf406fe50 msgr2=0x7f5bf415c040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:03.947 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.849+0000 7f5bf8986700 1 -- 172.21.15.99:0/3659770010 shutdown_connections 2023-10-08T00:22:03.947 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:03.849+0000 7f5bf8986700 1 -- 172.21.15.99:0/3659770010 wait complete. 2023-10-08T00:22:03.948 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:22:04.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.062+0000 7f66c896c700 1 Processor -- start 2023-10-08T00:22:04.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.062+0000 7f66c896c700 1 -- start start 2023-10-08T00:22:04.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.063+0000 7f66c896c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f66c40cd6e0 0x7f66c40cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:04.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.063+0000 7f66c896c700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66c40cb7c0 con 0x7f66c40cdab0 2023-10-08T00:22:04.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.063+0000 7f66c896c700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66c415e290 con 0x7f66c415cbd0 2023-10-08T00:22:04.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.063+0000 7f66c896c700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66c415e850 con 0x7f66c40cd6e0 2023-10-08T00:22:04.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.063+0000 7f66c37fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f66c40cd6e0 0x7f66c40cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:04.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.063+0000 7f66c2ffd700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f66c40cdab0 0x7f66c415d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56780/0 (socket says 172.21.15.99:56780) 2023-10-08T00:22:04.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.063+0000 7f66c2ffd700 1 -- 172.21.15.99:0/3527039165 learned_addr learned my addr 172.21.15.99:0/3527039165 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:22:04.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.063+0000 7f66c37fe700 1 -- 172.21.15.99:0/3527039165 >> v1:172.21.15.191:6789/0 conn(0x7f66c415cbd0 legacy=0x7f66c415d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:04.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.063+0000 7f66c37fe700 1 -- 172.21.15.99:0/3527039165 >> v1:172.21.15.99:6789/0 conn(0x7f66c40cdab0 legacy=0x7f66c415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:04.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.063+0000 7f66c37fe700 1 -- 172.21.15.99:0/3527039165 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c415e570 con 0x7f66c40cd6e0 2023-10-08T00:22:04.066 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.064+0000 7f66c37fe700 1 --2- 172.21.15.99:0/3527039165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66c40cd6e0 0x7f66c40cbc50 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f66b4009e60 tx=0x7f66b4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38b5c12b6663d68b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:04.066 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.064+0000 7f66c27fc700 1 -- 172.21.15.99:0/3527039165 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b400e070 con 0x7f66c40cd6e0 2023-10-08T00:22:04.066 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.064+0000 7f66c27fc700 1 -- 172.21.15.99:0/3527039165 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66b400bbf0 con 0x7f66c40cd6e0 2023-10-08T00:22:04.066 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.064+0000 7f66c27fc700 1 -- 172.21.15.99:0/3527039165 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b4012430 con 0x7f66c40cd6e0 2023-10-08T00:22:04.066 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.064+0000 7f66c896c700 1 -- 172.21.15.99:0/3527039165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66c40cd6e0 msgr2=0x7f66c40cbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:04.066 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.064+0000 7f66c896c700 1 --2- 172.21.15.99:0/3527039165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66c40cd6e0 0x7f66c40cbc50 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f66b4009e60 tx=0x7f66b4009780 comp rx=0 tx=0).stop 2023-10-08T00:22:04.066 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.064+0000 7f66c896c700 1 -- 172.21.15.99:0/3527039165 shutdown_connections 2023-10-08T00:22:04.066 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.064+0000 7f66c896c700 1 --2- 172.21.15.99:0/3527039165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66c40cd6e0 0x7f66c40cbc50 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:04.067 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.064+0000 7f66c896c700 1 -- 172.21.15.99:0/3527039165 >> 172.21.15.99:0/3527039165 conn(0x7f66c4067660 msgr2=0x7f66c4067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:04.067 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.064+0000 7f66c896c700 1 -- 172.21.15.99:0/3527039165 shutdown_connections 2023-10-08T00:22:04.067 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.064+0000 7f66c896c700 1 -- 172.21.15.99:0/3527039165 wait complete. 2023-10-08T00:22:04.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.065+0000 7f66c896c700 1 Processor -- start 2023-10-08T00:22:04.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.065+0000 7f66c896c700 1 -- start start 2023-10-08T00:22:04.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.066+0000 7f66c896c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66c40cd6e0 0x7f66c415bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:04.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.066+0000 7f66c896c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66c40cdab0 0x7f66c40c6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:04.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.066+0000 7f66c896c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f66c415cbd0 0x7f66c40c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:04.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.066+0000 7f66c896c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c40ca740 con 0x7f66c40cdab0 2023-10-08T00:22:04.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.066+0000 7f66c896c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c40c9ac0 con 0x7f66c40cd6e0 2023-10-08T00:22:04.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.066+0000 7f66c896c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66c40c9da0 con 0x7f66c415cbd0 2023-10-08T00:22:04.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.066+0000 7f66c2ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66c40cdab0 0x7f66c40c6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:04.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.066+0000 7f66c2ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66c40cdab0 0x7f66c40c6530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:32874/0 (socket says 172.21.15.99:32874) 2023-10-08T00:22:04.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.066+0000 7f66c2ffd700 1 -- 172.21.15.99:0/709198164 learned_addr learned my addr 172.21.15.99:0/709198164 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:04.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.066+0000 7f66c2ffd700 1 -- 172.21.15.99:0/709198164 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f66c415cbd0 msgr2=0x7f66c40c7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:04.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.066+0000 7f66c2ffd700 1 --2- 172.21.15.99:0/709198164 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f66c415cbd0 0x7f66c40c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:04.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.066+0000 7f66c2ffd700 1 -- 172.21.15.99:0/709198164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66c40cd6e0 msgr2=0x7f66c415bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:04.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.066+0000 7f66c37fe700 1 --2- 172.21.15.99:0/709198164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66c40cd6e0 0x7f66c415bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:04.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.067+0000 7f66c2ffd700 1 --2- 172.21.15.99:0/709198164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66c40cd6e0 0x7f66c415bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:04.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.067+0000 7f66c2ffd700 1 -- 172.21.15.99:0/709198164 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b4009a20 con 0x7f66c40cdab0 2023-10-08T00:22:04.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.067+0000 7f66c2ffd700 1 --2- 172.21.15.99:0/709198164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66c40cdab0 0x7f66c40c6530 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f66b800c7c0 tx=0x7f66b8007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:04.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.067+0000 7f66ab7fe700 1 -- 172.21.15.99:0/709198164 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b8005b30 con 0x7f66c40cdab0 2023-10-08T00:22:04.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.067+0000 7f66ab7fe700 1 -- 172.21.15.99:0/709198164 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66b8005c90 con 0x7f66c40cdab0 2023-10-08T00:22:04.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.067+0000 7f66c896c700 1 -- 172.21.15.99:0/709198164 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66c40ca080 con 0x7f66c40cdab0 2023-10-08T00:22:04.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.067+0000 7f66ab7fe700 1 -- 172.21.15.99:0/709198164 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b8007320 con 0x7f66c40cdab0 2023-10-08T00:22:04.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.067+0000 7f66c896c700 1 -- 172.21.15.99:0/709198164 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66c40ca410 con 0x7f66c40cdab0 2023-10-08T00:22:04.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.068+0000 7f66c896c700 1 -- 172.21.15.99:0/709198164 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66c4089280 con 0x7f66c40cdab0 2023-10-08T00:22:04.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.068+0000 7f66ab7fe700 1 -- 172.21.15.99:0/709198164 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f66b8007690 con 0x7f66c40cdab0 2023-10-08T00:22:04.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.068+0000 7f66ab7fe700 1 --2- 172.21.15.99:0/709198164 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f66b0066f30 0x7f66b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:04.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.069+0000 7f66ab7fe700 1 -- 172.21.15.99:0/709198164 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5734+0+0 (secure 0 0 0) 0x7f66b8083a70 con 0x7f66c40cdab0 2023-10-08T00:22:04.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.069+0000 7f66c37fe700 1 --2- 172.21.15.99:0/709198164 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f66b0066f30 0x7f66b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:04.072 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.069+0000 7f66c37fe700 1 --2- 172.21.15.99:0/709198164 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f66b0066f30 0x7f66b00693e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f66b40099f0 tx=0x7f66b4018000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:04.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.073+0000 7f66ab7fe700 1 -- 172.21.15.99:0/709198164 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f66b8050cc0 con 0x7f66c40cdab0 2023-10-08T00:22:04.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.299+0000 7f66c896c700 1 -- 172.21.15.99:0/709198164 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66c404bb80 con 0x7f66c40cdab0 2023-10-08T00:22:04.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.300+0000 7f66ab7fe700 1 -- 172.21.15.99:0/709198164 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+4604 (secure 0 0 0) 0x7f66b8017070 con 0x7f66c40cdab0 2023-10-08T00:22:04.301 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:22:04.301 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":84,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/3862684847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3862684847},{"type":"v1","addr":"172.21.15.191:6836","nonce":3862684847}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":8106,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/359623206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":359623206},{"type":"v1","addr":"172.21.15.99:6834","nonce":359623206}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":8109,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3064101777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":3064101777},{"type":"v1","addr":"172.21.15.99:6835","nonce":3064101777}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:21:29.659394+0000","modified":"2023-10-08T00:21:33.311802+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8124},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8124":{"gid":8124,"name":"d","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/2466421758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":2466421758},{"type":"v1","addr":"172.21.15.191:6837","nonce":2466421758}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-08T00:22:04.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.302+0000 7f66c896c700 1 -- 172.21.15.99:0/709198164 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f66b0066f30 msgr2=0x7f66b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:04.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.302+0000 7f66c896c700 1 --2- 172.21.15.99:0/709198164 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f66b0066f30 0x7f66b00693e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f66b40099f0 tx=0x7f66b4018000 comp rx=0 tx=0).stop 2023-10-08T00:22:04.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.302+0000 7f66c896c700 1 -- 172.21.15.99:0/709198164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66c40cdab0 msgr2=0x7f66c40c6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:04.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.302+0000 7f66c896c700 1 --2- 172.21.15.99:0/709198164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66c40cdab0 0x7f66c40c6530 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f66b800c7c0 tx=0x7f66b8007f30 comp rx=0 tx=0).stop 2023-10-08T00:22:04.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.302+0000 7f66c896c700 1 -- 172.21.15.99:0/709198164 shutdown_connections 2023-10-08T00:22:04.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.302+0000 7f66c896c700 1 --2- 172.21.15.99:0/709198164 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f66b0066f30 0x7f66b00693e0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:04.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.302+0000 7f66c896c700 1 --2- 172.21.15.99:0/709198164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66c40cd6e0 0x7f66c415bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:04.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.302+0000 7f66c896c700 1 --2- 172.21.15.99:0/709198164 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f66c415cbd0 0x7f66c40c7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:04.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.302+0000 7f66c896c700 1 --2- 172.21.15.99:0/709198164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66c40cdab0 0x7f66c40c6530 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:04.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.302+0000 7f66c896c700 1 -- 172.21.15.99:0/709198164 >> 172.21.15.99:0/709198164 conn(0x7f66c4067660 msgr2=0x7f66c40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:04.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.302+0000 7f66c896c700 1 -- 172.21.15.99:0/709198164 shutdown_connections 2023-10-08T00:22:04.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:04.302+0000 7f66c896c700 1 -- 172.21.15.99:0/709198164 wait complete. 2023-10-08T00:22:04.304 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 84 2023-10-08T00:22:04.315 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json status 2023-10-08T00:22:04.461 INFO:teuthology.orchestra.run.smithi191.stdout:{"cluster_fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","whoami":0,"id":8124,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":34.148900941000001,"mdsmap_epoch":84,"osdmap_epoch":189,"osdmap_epoch_barrier":188,"uptime":39.224540990000001} 2023-10-08T00:22:04.462 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "06962cdb-9a6f-417f-82fd-9e4e3dedbccf", "fs_name": "cephfs", "id": 8124, "mdsmap_epoch": 84, "osdmap_epoch": 189, "osdmap_epoch_barrier": 188, "rank_uptime": 34.148900941, "state": "up:active", "uptime": 39.22454099, "want_state": "up:active", "whoami": 0 } 2023-10-08T00:22:04.462 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-08T00:22:04.462 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:22:04.462 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2023-10-08T00:22:04.747 INFO:teuthology.orchestra.run.smithi099.stderr:50+0 records in 2023-10-08T00:22:04.747 INFO:teuthology.orchestra.run.smithi099.stderr:50+0 records out 2023-10-08T00:22:04.747 INFO:teuthology.orchestra.run.smithi099.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.267609 s, 196 MB/s 2023-10-08T00:22:04.747 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T00:22:04.748 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T00:22:04.784 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T00:22:04.822 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T00:22:04.822 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T00:22:04.822 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:22:04.823 DEBUG:teuthology.orchestra.run.smithi099:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-08T00:22:04.885 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2023-10-08T00:22:04.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:22:04.886 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T00:22:04.902 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-08T00:22:04.902 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:22:04.902 DEBUG:teuthology.orchestra.run.smithi191:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-08T00:22:04.944 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2023-10-08T00:22:04.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:22:04.944 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-08T00:22:04.999 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:22:05.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.111+0000 7f75d459c700 1 Processor -- start 2023-10-08T00:22:05.112 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.111+0000 7f75d459c700 1 -- start start 2023-10-08T00:22:05.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.112+0000 7f75d459c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f75cc1577d0 0x7f75cc0cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:05.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.112+0000 7f75d459c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f75cc157ba0 0x7f75cc06a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:05.115 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.112+0000 7f75d459c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f75cc15b0d0 0x7f75cc06aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:05.116 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.112+0000 7f75d459c700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75cc15c360 con 0x7f75cc1577d0 2023-10-08T00:22:05.116 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.112+0000 7f75d459c700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75cc15b790 con 0x7f75cc157ba0 2023-10-08T00:22:05.116 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.112+0000 7f75d459c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc15ba30 con 0x7f75cc15b0d0 2023-10-08T00:22:05.116 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.112+0000 7f75d2338700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f75cc1577d0 0x7f75cc0cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:05.116 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.112+0000 7f75d2338700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f75cc1577d0 0x7f75cc0cd770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:32876/0 (socket says 172.21.15.99:32876) 2023-10-08T00:22:05.116 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.112+0000 7f75d2338700 1 -- 172.21.15.99:0/31870779 learned_addr learned my addr 172.21.15.99:0/31870779 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:05.116 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.113+0000 7f75d2b39700 1 --2- 172.21.15.99:0/31870779 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f75cc15b0d0 0x7f75cc06aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:05.116 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.113+0000 7f75d1b37700 1 --2- 172.21.15.99:0/31870779 >> v2:172.21.15.191:3300/0 conn(0x7f75cc157ba0 0x7f75cc06a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:05.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.113+0000 7f75d2338700 1 -- 172.21.15.99:0/31870779 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f75cc15b0d0 msgr2=0x7f75cc06aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:05.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.113+0000 7f75d2338700 1 --2- 172.21.15.99:0/31870779 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f75cc15b0d0 0x7f75cc06aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:05.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.113+0000 7f75d2338700 1 -- 172.21.15.99:0/31870779 >> v2:172.21.15.191:3300/0 conn(0x7f75cc157ba0 msgr2=0x7f75cc06a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:05.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.113+0000 7f75d2338700 1 --2- 172.21.15.99:0/31870779 >> v2:172.21.15.191:3300/0 conn(0x7f75cc157ba0 0x7f75cc06a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:05.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.113+0000 7f75d2338700 1 -- 172.21.15.99:0/31870779 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75cc15bd10 con 0x7f75cc1577d0 2023-10-08T00:22:05.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.113+0000 7f75d2338700 1 --2- 172.21.15.99:0/31870779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75cc1577d0 0x7f75cc0cd770 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f75cc0b38b0 tx=0x7f75c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca8369e84931b547 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:05.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.113+0000 7f75d1336700 1 -- 172.21.15.99:0/31870779 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c800b860 con 0x7f75cc1577d0 2023-10-08T00:22:05.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.113+0000 7f75d1336700 1 -- 172.21.15.99:0/31870779 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75c800b9c0 con 0x7f75cc1577d0 2023-10-08T00:22:05.117 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.113+0000 7f75d1336700 1 -- 172.21.15.99:0/31870779 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c800bd60 con 0x7f75cc1577d0 2023-10-08T00:22:05.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.113+0000 7f75d459c700 1 -- 172.21.15.99:0/31870779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75cc1577d0 msgr2=0x7f75cc0cd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:05.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.113+0000 7f75d459c700 1 --2- 172.21.15.99:0/31870779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75cc1577d0 0x7f75cc0cd770 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f75cc0b38b0 tx=0x7f75c8009250 comp rx=0 tx=0).stop 2023-10-08T00:22:05.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.114+0000 7f75d459c700 1 -- 172.21.15.99:0/31870779 shutdown_connections 2023-10-08T00:22:05.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.114+0000 7f75d459c700 1 --2- 172.21.15.99:0/31870779 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f75cc15b0d0 0x7f75cc06aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:05.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.114+0000 7f75d459c700 1 --2- 172.21.15.99:0/31870779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75cc1577d0 0x7f75cc0cd770 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:05.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.114+0000 7f75d459c700 1 --2- 172.21.15.99:0/31870779 >> v2:172.21.15.191:3300/0 conn(0x7f75cc157ba0 0x7f75cc06a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:05.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.114+0000 7f75d459c700 1 -- 172.21.15.99:0/31870779 >> 172.21.15.99:0/31870779 conn(0x7f75cc068ce0 msgr2=0x7f75cc0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:05.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.114+0000 7f75d459c700 1 -- 172.21.15.99:0/31870779 shutdown_connections 2023-10-08T00:22:05.118 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.114+0000 7f75d459c700 1 -- 172.21.15.99:0/31870779 wait complete. 2023-10-08T00:22:05.119 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.115+0000 7f75d459c700 1 Processor -- start 2023-10-08T00:22:05.119 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.115+0000 7f75d459c700 1 -- start start 2023-10-08T00:22:05.122 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.115+0000 7f75d459c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75cc1577d0 0x7f75cc0a9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:05.122 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.115+0000 7f75d459c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f75cc157ba0 0x7f75cc0af000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:05.122 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.115+0000 7f75d459c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f75cc15b0d0 0x7f75cc155c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:05.122 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.115+0000 7f75d459c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc0af540 con 0x7f75cc1577d0 2023-10-08T00:22:05.122 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.115+0000 7f75d459c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc0a7b20 con 0x7f75cc157ba0 2023-10-08T00:22:05.122 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.115+0000 7f75d459c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc0a9840 con 0x7f75cc15b0d0 2023-10-08T00:22:05.123 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.115+0000 7f75d2338700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75cc1577d0 0x7f75cc0a9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:05.123 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.116+0000 7f75d2338700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75cc1577d0 0x7f75cc0a9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:32892/0 (socket says 172.21.15.99:32892) 2023-10-08T00:22:05.123 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.116+0000 7f75d2338700 1 -- 172.21.15.99:0/1029281144 learned_addr learned my addr 172.21.15.99:0/1029281144 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:05.123 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.116+0000 7f75d1b37700 1 --2- 172.21.15.99:0/1029281144 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f75cc157ba0 0x7f75cc0af000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:05.123 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.116+0000 7f75d2b39700 1 --2- 172.21.15.99:0/1029281144 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f75cc15b0d0 0x7f75cc155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:05.123 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.116+0000 7f75d2338700 1 -- 172.21.15.99:0/1029281144 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f75cc15b0d0 msgr2=0x7f75cc155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:05.123 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.116+0000 7f75d2338700 1 --2- 172.21.15.99:0/1029281144 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f75cc15b0d0 0x7f75cc155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:05.123 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.116+0000 7f75d2338700 1 -- 172.21.15.99:0/1029281144 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f75cc157ba0 msgr2=0x7f75cc0af000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:05.123 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.116+0000 7f75d2338700 1 --2- 172.21.15.99:0/1029281144 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f75cc157ba0 0x7f75cc0af000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:05.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.116+0000 7f75d2338700 1 -- 172.21.15.99:0/1029281144 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c80094f0 con 0x7f75cc1577d0 2023-10-08T00:22:05.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.116+0000 7f75d1b37700 1 --2- 172.21.15.99:0/1029281144 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f75cc157ba0 0x7f75cc0af000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:22:05.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.116+0000 7f75d2338700 1 --2- 172.21.15.99:0/1029281144 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75cc1577d0 0x7f75cc0a9160 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f75cc06b580 tx=0x7f75c8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:05.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.116+0000 7f75c27fc700 1 -- 172.21.15.99:0/1029281144 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c8024440 con 0x7f75cc1577d0 2023-10-08T00:22:05.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.116+0000 7f75c27fc700 1 -- 172.21.15.99:0/1029281144 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75c80245a0 con 0x7f75cc1577d0 2023-10-08T00:22:05.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.116+0000 7f75d459c700 1 -- 172.21.15.99:0/1029281144 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75cc0a9b20 con 0x7f75cc1577d0 2023-10-08T00:22:05.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.116+0000 7f75c27fc700 1 -- 172.21.15.99:0/1029281144 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c8024960 con 0x7f75cc1577d0 2023-10-08T00:22:05.124 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.116+0000 7f75d459c700 1 -- 172.21.15.99:0/1029281144 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75cc156150 con 0x7f75cc1577d0 2023-10-08T00:22:05.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.117+0000 7f75d459c700 1 -- 172.21.15.99:0/1029281144 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75cc0a7610 con 0x7f75cc1577d0 2023-10-08T00:22:05.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.118+0000 7f75c27fc700 1 -- 172.21.15.99:0/1029281144 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f75c801a420 con 0x7f75cc1577d0 2023-10-08T00:22:05.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.118+0000 7f75c27fc700 1 --2- 172.21.15.99:0/1029281144 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f75b0066f30 0x7f75b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:05.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.118+0000 7f75c27fc700 1 -- 172.21.15.99:0/1029281144 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5734+0+0 (secure 0 0 0) 0x7f75c8086310 con 0x7f75cc1577d0 2023-10-08T00:22:05.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.118+0000 7f75d1b37700 1 --2- 172.21.15.99:0/1029281144 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f75b0066f30 0x7f75b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:05.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.119+0000 7f75d1b37700 1 --2- 172.21.15.99:0/1029281144 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f75b0066f30 0x7f75b00693e0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f75cc15af40 tx=0x7f75bc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:05.125 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.122+0000 7f75c27fc700 1 -- 172.21.15.99:0/1029281144 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f75cc0a7610 con 0x7f75cc1577d0 2023-10-08T00:22:05.347 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.346+0000 7f75d459c700 1 -- 172.21.15.99:0/1029281144 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f75cc0a7970 con 0x7f75cc1577d0 2023-10-08T00:22:05.347 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.347+0000 7f75c27fc700 1 -- 172.21.15.99:0/1029281144 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+4604 (secure 0 0 0) 0x7f75c80535e0 con 0x7f75cc1577d0 2023-10-08T00:22:05.348 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:22:05.348 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":84,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/3862684847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3862684847},{"type":"v1","addr":"172.21.15.191:6836","nonce":3862684847}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":8106,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/359623206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":359623206},{"type":"v1","addr":"172.21.15.99:6834","nonce":359623206}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":8109,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3064101777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":3064101777},{"type":"v1","addr":"172.21.15.99:6835","nonce":3064101777}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:21:29.659394+0000","modified":"2023-10-08T00:21:33.311802+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8124},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8124":{"gid":8124,"name":"d","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/2466421758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":2466421758},{"type":"v1","addr":"172.21.15.191:6837","nonce":2466421758}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-08T00:22:05.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.349+0000 7f75d459c700 1 -- 172.21.15.99:0/1029281144 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f75b0066f30 msgr2=0x7f75b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:05.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.349+0000 7f75d459c700 1 --2- 172.21.15.99:0/1029281144 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f75b0066f30 0x7f75b00693e0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f75cc15af40 tx=0x7f75bc008040 comp rx=0 tx=0).stop 2023-10-08T00:22:05.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.349+0000 7f75d459c700 1 -- 172.21.15.99:0/1029281144 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75cc1577d0 msgr2=0x7f75cc0a9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:05.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.349+0000 7f75d459c700 1 --2- 172.21.15.99:0/1029281144 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75cc1577d0 0x7f75cc0a9160 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f75cc06b580 tx=0x7f75c8017000 comp rx=0 tx=0).stop 2023-10-08T00:22:05.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.349+0000 7f75d459c700 1 -- 172.21.15.99:0/1029281144 shutdown_connections 2023-10-08T00:22:05.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.349+0000 7f75d459c700 1 --2- 172.21.15.99:0/1029281144 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f75b0066f30 0x7f75b00693e0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:05.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.349+0000 7f75d459c700 1 --2- 172.21.15.99:0/1029281144 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f75cc15b0d0 0x7f75cc155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:05.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.349+0000 7f75d459c700 1 --2- 172.21.15.99:0/1029281144 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75cc1577d0 0x7f75cc0a9160 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:05.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.349+0000 7f75d459c700 1 --2- 172.21.15.99:0/1029281144 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f75cc157ba0 0x7f75cc0af000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:05.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.349+0000 7f75d459c700 1 -- 172.21.15.99:0/1029281144 >> 172.21.15.99:0/1029281144 conn(0x7f75cc068ce0 msgr2=0x7f75cc0cfb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:05.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.349+0000 7f75d459c700 1 -- 172.21.15.99:0/1029281144 shutdown_connections 2023-10-08T00:22:05.352 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.349+0000 7f75d459c700 1 -- 172.21.15.99:0/1029281144 wait complete. 2023-10-08T00:22:05.352 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 84 2023-10-08T00:22:05.362 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:22:05.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.476+0000 7f489f950700 1 Processor -- start 2023-10-08T00:22:05.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.477+0000 7f489f950700 1 -- start start 2023-10-08T00:22:05.480 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.477+0000 7f489f950700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f48981577d0 0x7f48980cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:05.480 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.477+0000 7f489f950700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4898157ba0 0x7f489806a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:05.480 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.477+0000 7f489f950700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f489815b0d0 0x7f489806aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:05.480 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.477+0000 7f489f950700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f489815c360 con 0x7f48981577d0 2023-10-08T00:22:05.480 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.477+0000 7f489f950700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f489815b790 con 0x7f4898157ba0 2023-10-08T00:22:05.480 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.477+0000 7f489f950700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f489815ba30 con 0x7f489815b0d0 2023-10-08T00:22:05.480 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.478+0000 7f489d6ec700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f48981577d0 0x7f48980cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:05.480 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.478+0000 7f489d6ec700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f48981577d0 0x7f48980cd770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:32896/0 (socket says 172.21.15.99:32896) 2023-10-08T00:22:05.481 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.478+0000 7f489d6ec700 1 -- 172.21.15.99:0/3355078685 learned_addr learned my addr 172.21.15.99:0/3355078685 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:05.481 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.478+0000 7f489deed700 1 --2- 172.21.15.99:0/3355078685 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f489815b0d0 0x7f489806aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:05.481 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.478+0000 7f489ceeb700 1 --2- 172.21.15.99:0/3355078685 >> v2:172.21.15.191:3300/0 conn(0x7f4898157ba0 0x7f489806a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:05.481 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.478+0000 7f489d6ec700 1 -- 172.21.15.99:0/3355078685 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f489815b0d0 msgr2=0x7f489806aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:05.481 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.478+0000 7f489d6ec700 1 --2- 172.21.15.99:0/3355078685 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f489815b0d0 0x7f489806aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:05.481 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.478+0000 7f489d6ec700 1 -- 172.21.15.99:0/3355078685 >> v2:172.21.15.191:3300/0 conn(0x7f4898157ba0 msgr2=0x7f489806a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:05.481 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.478+0000 7f489d6ec700 1 --2- 172.21.15.99:0/3355078685 >> v2:172.21.15.191:3300/0 conn(0x7f4898157ba0 0x7f489806a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:05.481 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.478+0000 7f489d6ec700 1 -- 172.21.15.99:0/3355078685 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f489815bd10 con 0x7f48981577d0 2023-10-08T00:22:05.481 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.478+0000 7f489deed700 1 --2- 172.21.15.99:0/3355078685 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f489815b0d0 0x7f489806aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:22:05.481 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.478+0000 7f489d6ec700 1 --2- 172.21.15.99:0/3355078685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48981577d0 0x7f48980cd770 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f48980b38b0 tx=0x7f4894009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de4ef75d2b6bce0d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:05.481 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.478+0000 7f488ffff700 1 -- 172.21.15.99:0/3355078685 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489400e070 con 0x7f48981577d0 2023-10-08T00:22:05.481 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.478+0000 7f488ffff700 1 -- 172.21.15.99:0/3355078685 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f489400ba50 con 0x7f48981577d0 2023-10-08T00:22:05.482 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.478+0000 7f488ffff700 1 -- 172.21.15.99:0/3355078685 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48940092e0 con 0x7f48981577d0 2023-10-08T00:22:05.482 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.479+0000 7f489f950700 1 -- 172.21.15.99:0/3355078685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48981577d0 msgr2=0x7f48980cd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:05.482 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.479+0000 7f489f950700 1 --2- 172.21.15.99:0/3355078685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48981577d0 0x7f48980cd770 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f48980b38b0 tx=0x7f4894009840 comp rx=0 tx=0).stop 2023-10-08T00:22:05.482 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.479+0000 7f489f950700 1 -- 172.21.15.99:0/3355078685 shutdown_connections 2023-10-08T00:22:05.482 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.479+0000 7f489f950700 1 --2- 172.21.15.99:0/3355078685 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f489815b0d0 0x7f489806aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:05.482 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.479+0000 7f489f950700 1 --2- 172.21.15.99:0/3355078685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48981577d0 0x7f48980cd770 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:05.482 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.479+0000 7f489f950700 1 --2- 172.21.15.99:0/3355078685 >> v2:172.21.15.191:3300/0 conn(0x7f4898157ba0 0x7f489806a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:05.482 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.479+0000 7f489f950700 1 -- 172.21.15.99:0/3355078685 >> 172.21.15.99:0/3355078685 conn(0x7f4898068ce0 msgr2=0x7f48980690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:05.482 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.479+0000 7f489f950700 1 -- 172.21.15.99:0/3355078685 shutdown_connections 2023-10-08T00:22:05.482 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.479+0000 7f489f950700 1 -- 172.21.15.99:0/3355078685 wait complete. 2023-10-08T00:22:05.485 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.480+0000 7f489f950700 1 Processor -- start 2023-10-08T00:22:05.485 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.480+0000 7f489f950700 1 -- start start 2023-10-08T00:22:05.485 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.481+0000 7f489f950700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48981577d0 0x7f48980ac240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:05.485 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.481+0000 7f489f950700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4898157ba0 0x7f48980acd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:05.485 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.481+0000 7f489f950700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f489815b0d0 0x7f48980ae910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:05.485 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.481+0000 7f489f950700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48980ac780 con 0x7f4898157ba0 2023-10-08T00:22:05.485 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.481+0000 7f489f950700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48980a9f90 con 0x7f48981577d0 2023-10-08T00:22:05.485 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.481+0000 7f489f950700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48980a8590 con 0x7f489815b0d0 2023-10-08T00:22:05.485 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.481+0000 7f489ceeb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4898157ba0 0x7f48980acd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:05.486 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.481+0000 7f489ceeb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4898157ba0 0x7f48980acd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:32900/0 (socket says 172.21.15.99:32900) 2023-10-08T00:22:05.486 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.481+0000 7f489d6ec700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48981577d0 0x7f48980ac240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:05.486 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.481+0000 7f489ceeb700 1 -- 172.21.15.99:0/3654694033 learned_addr learned my addr 172.21.15.99:0/3654694033 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:05.486 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.481+0000 7f489deed700 1 --2- 172.21.15.99:0/3654694033 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f489815b0d0 0x7f48980ae910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:05.486 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.481+0000 7f489ceeb700 1 -- 172.21.15.99:0/3654694033 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f489815b0d0 msgr2=0x7f48980ae910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:05.486 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.481+0000 7f489ceeb700 1 --2- 172.21.15.99:0/3654694033 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f489815b0d0 0x7f48980ae910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:05.487 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.481+0000 7f489ceeb700 1 -- 172.21.15.99:0/3654694033 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48981577d0 msgr2=0x7f48980ac240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:05.487 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.481+0000 7f489ceeb700 1 --2- 172.21.15.99:0/3654694033 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48981577d0 0x7f48980ac240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:05.487 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.481+0000 7f489ceeb700 1 -- 172.21.15.99:0/3654694033 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4894009ae0 con 0x7f4898157ba0 2023-10-08T00:22:05.487 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.481+0000 7f489ceeb700 1 --2- 172.21.15.99:0/3654694033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4898157ba0 0x7f48980acd80 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f488800d890 tx=0x7f4888008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:05.487 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.482+0000 7f488d7fa700 1 -- 172.21.15.99:0/3654694033 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48880125f0 con 0x7f4898157ba0 2023-10-08T00:22:05.487 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.482+0000 7f488d7fa700 1 -- 172.21.15.99:0/3654694033 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4888012750 con 0x7f4898157ba0 2023-10-08T00:22:05.487 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.482+0000 7f489f950700 1 -- 172.21.15.99:0/3654694033 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48980a8870 con 0x7f4898157ba0 2023-10-08T00:22:05.487 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.482+0000 7f488d7fa700 1 -- 172.21.15.99:0/3654694033 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4888012af0 con 0x7f4898157ba0 2023-10-08T00:22:05.487 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.482+0000 7f489f950700 1 -- 172.21.15.99:0/3654694033 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48980a8e30 con 0x7f4898157ba0 2023-10-08T00:22:05.487 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.483+0000 7f489f950700 1 -- 172.21.15.99:0/3654694033 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48980aa140 con 0x7f4898157ba0 2023-10-08T00:22:05.487 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.483+0000 7f488d7fa700 1 -- 172.21.15.99:0/3654694033 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f488801e460 con 0x7f4898157ba0 2023-10-08T00:22:05.488 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.483+0000 7f488d7fa700 1 --2- 172.21.15.99:0/3654694033 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f487c066f30 0x7f487c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:05.488 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.484+0000 7f488d7fa700 1 -- 172.21.15.99:0/3654694033 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5734+0+0 (secure 0 0 0) 0x7f4888084a10 con 0x7f4898157ba0 2023-10-08T00:22:05.488 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.484+0000 7f489d6ec700 1 --2- 172.21.15.99:0/3654694033 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f487c066f30 0x7f487c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:05.488 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.484+0000 7f489d6ec700 1 --2- 172.21.15.99:0/3654694033 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f487c066f30 0x7f487c0693e0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4894009880 tx=0x7f4894018000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:05.488 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.488+0000 7f488d7fa700 1 -- 172.21.15.99:0/3654694033 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f48980aa140 con 0x7f4898157ba0 2023-10-08T00:22:05.670 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.670+0000 7f489f950700 1 -- 172.21.15.99:0/3654694033 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f48980aa100 con 0x7f4898157ba0 2023-10-08T00:22:05.671 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.671+0000 7f488d7fa700 1 -- 172.21.15.99:0/3654694033 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v189) v1 ==== 74+0+15422 (secure 0 0 0) 0x7f48880556f0 con 0x7f4898157ba0 2023-10-08T00:22:05.671 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:22:05.671 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":189,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:22:01.171728+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:21:27.984574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-08T00:21:29.244620+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"189","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"23.18","mappings":[{"from":5,"to":3}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:22:05.674 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.673+0000 7f489f950700 1 -- 172.21.15.99:0/3654694033 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f487c066f30 msgr2=0x7f487c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:05.674 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.673+0000 7f489f950700 1 --2- 172.21.15.99:0/3654694033 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f487c066f30 0x7f487c0693e0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4894009880 tx=0x7f4894018000 comp rx=0 tx=0).stop 2023-10-08T00:22:05.674 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.673+0000 7f489f950700 1 -- 172.21.15.99:0/3654694033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4898157ba0 msgr2=0x7f48980acd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:05.674 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.673+0000 7f489f950700 1 --2- 172.21.15.99:0/3654694033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4898157ba0 0x7f48980acd80 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f488800d890 tx=0x7f4888008040 comp rx=0 tx=0).stop 2023-10-08T00:22:05.674 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.673+0000 7f489f950700 1 -- 172.21.15.99:0/3654694033 shutdown_connections 2023-10-08T00:22:05.674 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.673+0000 7f489f950700 1 --2- 172.21.15.99:0/3654694033 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f487c066f30 0x7f487c0693e0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:05.674 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.673+0000 7f489f950700 1 --2- 172.21.15.99:0/3654694033 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48981577d0 0x7f48980ac240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:05.674 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.673+0000 7f489f950700 1 --2- 172.21.15.99:0/3654694033 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f489815b0d0 0x7f48980ae910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:05.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.673+0000 7f489f950700 1 --2- 172.21.15.99:0/3654694033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4898157ba0 0x7f48980acd80 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:05.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.673+0000 7f489f950700 1 -- 172.21.15.99:0/3654694033 >> 172.21.15.99:0/3654694033 conn(0x7f4898068ce0 msgr2=0x7f489806a730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:05.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.673+0000 7f489f950700 1 -- 172.21.15.99:0/3654694033 shutdown_connections 2023-10-08T00:22:05.675 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.673+0000 7f489f950700 1 -- 172.21.15.99:0/3654694033 wait complete. 2023-10-08T00:22:05.685 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:10:42.063344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T00:21:27.984574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '183', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T00:21:29.244620+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '189', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T00:22:05.686 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T00:22:05.804 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.803+0000 7ff1d2cc8700 1 Processor -- start 2023-10-08T00:22:05.804 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.803+0000 7ff1d2cc8700 1 -- start start 2023-10-08T00:22:05.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.804+0000 7ff1d2cc8700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff1cc070700 0x7ff1cc15c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:05.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.804+0000 7ff1d2cc8700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff1cc067be0 0x7ff1cc068020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:05.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.804+0000 7ff1d2cc8700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1cc067700 con 0x7ff1cc070ad0 2023-10-08T00:22:05.806 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.804+0000 7ff1d2cc8700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1cc15c770 con 0x7ff1cc070700 2023-10-08T00:22:05.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.804+0000 7ff1d2cc8700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1cc1586a0 con 0x7ff1cc067be0 2023-10-08T00:22:05.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.804+0000 7ff1d0a64700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff1cc070700 0x7ff1cc15c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:05.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.804+0000 7ff1d0a64700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff1cc070700 0x7ff1cc15c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:32908/0 (socket says 172.21.15.99:32908) 2023-10-08T00:22:05.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.804+0000 7ff1d0a64700 1 -- 172.21.15.99:0/3842881674 learned_addr learned my addr 172.21.15.99:0/3842881674 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:05.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.804+0000 7ff1d1265700 1 --2- 172.21.15.99:0/3842881674 >> v2:172.21.15.191:3300/0 conn(0x7ff1cc067be0 0x7ff1cc068020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:05.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.804+0000 7ff1d0a64700 1 -- 172.21.15.99:0/3842881674 >> v2:172.21.15.191:3300/0 conn(0x7ff1cc067be0 msgr2=0x7ff1cc068020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:05.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.804+0000 7ff1d0a64700 1 --2- 172.21.15.99:0/3842881674 >> v2:172.21.15.191:3300/0 conn(0x7ff1cc067be0 0x7ff1cc068020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:05.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.804+0000 7ff1d0a64700 1 -- 172.21.15.99:0/3842881674 >> v1:172.21.15.191:6789/0 conn(0x7ff1cc070ad0 legacy=0x7ff1cc157f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:05.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.804+0000 7ff1d0a64700 1 -- 172.21.15.99:0/3842881674 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1cc158940 con 0x7ff1cc070700 2023-10-08T00:22:05.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.805+0000 7ff1d0a64700 1 --2- 172.21.15.99:0/3842881674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1cc070700 0x7ff1cc15c230 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc15b7b0 tx=0x7ff1c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71626d8e2484d00 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:05.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.805+0000 7ff1c37fe700 1 -- 172.21.15.99:0/3842881674 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c8009e70 con 0x7ff1cc070700 2023-10-08T00:22:05.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.805+0000 7ff1c37fe700 1 -- 172.21.15.99:0/3842881674 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1c800b860 con 0x7ff1cc070700 2023-10-08T00:22:05.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.805+0000 7ff1c37fe700 1 -- 172.21.15.99:0/3842881674 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c800bc00 con 0x7ff1cc070700 2023-10-08T00:22:05.808 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.805+0000 7ff1d2cc8700 1 -- 172.21.15.99:0/3842881674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1cc070700 msgr2=0x7ff1cc15c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:05.808 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.805+0000 7ff1d2cc8700 1 --2- 172.21.15.99:0/3842881674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1cc070700 0x7ff1cc15c230 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc15b7b0 tx=0x7ff1c8009250 comp rx=0 tx=0).stop 2023-10-08T00:22:05.808 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.805+0000 7ff1d2cc8700 1 -- 172.21.15.99:0/3842881674 shutdown_connections 2023-10-08T00:22:05.808 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.805+0000 7ff1d2cc8700 1 --2- 172.21.15.99:0/3842881674 >> v2:172.21.15.191:3300/0 conn(0x7ff1cc067be0 0x7ff1cc068020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:05.808 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.805+0000 7ff1d2cc8700 1 --2- 172.21.15.99:0/3842881674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1cc070700 0x7ff1cc15c230 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:05.808 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.805+0000 7ff1d2cc8700 1 -- 172.21.15.99:0/3842881674 >> 172.21.15.99:0/3842881674 conn(0x7ff1cc0ce4d0 msgr2=0x7ff1cc0ca4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:05.808 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.805+0000 7ff1d2cc8700 1 -- 172.21.15.99:0/3842881674 shutdown_connections 2023-10-08T00:22:05.808 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.805+0000 7ff1d2cc8700 1 -- 172.21.15.99:0/3842881674 wait complete. 2023-10-08T00:22:05.812 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.806+0000 7ff1d2cc8700 1 Processor -- start 2023-10-08T00:22:05.812 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.807+0000 7ff1d2cc8700 1 -- start start 2023-10-08T00:22:05.812 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.807+0000 7ff1d2cc8700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff1cc067be0 0x7ff1cc0c2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:05.812 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.807+0000 7ff1d2cc8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff1cc070700 0x7ff1cc154c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:05.812 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.807+0000 7ff1d2cc8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1cc070ad0 0x7ff1cc0b2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:05.812 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.807+0000 7ff1d2cc8700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc0c7460 con 0x7ff1cc070ad0 2023-10-08T00:22:05.812 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.807+0000 7ff1d2cc8700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc0b1070 con 0x7ff1cc067be0 2023-10-08T00:22:05.812 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.807+0000 7ff1d2cc8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc0c27a0 con 0x7ff1cc070700 2023-10-08T00:22:05.813 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.807+0000 7ff1d1265700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1cc070ad0 0x7ff1cc0b2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:05.813 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.807+0000 7ff1d1265700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1cc070ad0 0x7ff1cc0b2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:32924/0 (socket says 172.21.15.99:32924) 2023-10-08T00:22:05.813 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.807+0000 7ff1d1265700 1 -- 172.21.15.99:0/2941790912 learned_addr learned my addr 172.21.15.99:0/2941790912 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:05.813 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.808+0000 7ff1c3fff700 1 --2- 172.21.15.99:0/2941790912 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff1cc070700 0x7ff1cc154c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:05.813 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.808+0000 7ff1d0a64700 1 --2- 172.21.15.99:0/2941790912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff1cc067be0 0x7ff1cc0c2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:05.813 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.808+0000 7ff1d1265700 1 -- 172.21.15.99:0/2941790912 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff1cc070700 msgr2=0x7ff1cc154c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:05.813 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.808+0000 7ff1d1265700 1 --2- 172.21.15.99:0/2941790912 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff1cc070700 0x7ff1cc154c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:05.813 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.808+0000 7ff1d1265700 1 -- 172.21.15.99:0/2941790912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff1cc067be0 msgr2=0x7ff1cc0c2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:05.813 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.808+0000 7ff1d1265700 1 --2- 172.21.15.99:0/2941790912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff1cc067be0 0x7ff1cc0c2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:05.813 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.808+0000 7ff1d1265700 1 -- 172.21.15.99:0/2941790912 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c80094f0 con 0x7ff1cc070ad0 2023-10-08T00:22:05.813 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.808+0000 7ff1d1265700 1 --2- 172.21.15.99:0/2941790912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1cc070ad0 0x7ff1cc0b2d30 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4009e90 tx=0x7ff1c400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:05.813 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.808+0000 7ff1c0ff9700 1 -- 172.21.15.99:0/2941790912 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c400cc20 con 0x7ff1cc070ad0 2023-10-08T00:22:05.814 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.808+0000 7ff1c0ff9700 1 -- 172.21.15.99:0/2941790912 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1c400f040 con 0x7ff1cc070ad0 2023-10-08T00:22:05.814 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.808+0000 7ff1c0ff9700 1 -- 172.21.15.99:0/2941790912 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4014480 con 0x7ff1cc070ad0 2023-10-08T00:22:05.814 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.808+0000 7ff1d2cc8700 1 -- 172.21.15.99:0/2941790912 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1cc0c7180 con 0x7ff1cc070ad0 2023-10-08T00:22:05.814 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.808+0000 7ff1d2cc8700 1 -- 172.21.15.99:0/2941790912 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1cc0b0d00 con 0x7ff1cc070ad0 2023-10-08T00:22:05.814 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.809+0000 7ff1d2cc8700 1 -- 172.21.15.99:0/2941790912 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1cc0b0880 con 0x7ff1cc070ad0 2023-10-08T00:22:05.814 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.809+0000 7ff1c0ff9700 1 -- 172.21.15.99:0/2941790912 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff1c4014830 con 0x7ff1cc070ad0 2023-10-08T00:22:05.814 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.810+0000 7ff1c0ff9700 1 --2- 172.21.15.99:0/2941790912 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff1b8066f30 0x7ff1b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:05.814 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.810+0000 7ff1c0ff9700 1 -- 172.21.15.99:0/2941790912 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5734+0+0 (secure 0 0 0) 0x7ff1c4084f70 con 0x7ff1cc070ad0 2023-10-08T00:22:05.814 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.810+0000 7ff1d0a64700 1 --2- 172.21.15.99:0/2941790912 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff1b8066f30 0x7ff1b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:05.814 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.810+0000 7ff1d0a64700 1 --2- 172.21.15.99:0/2941790912 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff1b8066f30 0x7ff1b80693e0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc15b7b0 tx=0x7ff1c8017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:05.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.814+0000 7ff1c0ff9700 1 -- 172.21.15.99:0/2941790912 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff1cc0b0880 con 0x7ff1cc070ad0 2023-10-08T00:22:05.997 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.996+0000 7ff1d2cc8700 1 -- 172.21.15.99:0/2941790912 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff1cc0b0880 con 0x7ff1cc070ad0 2023-10-08T00:22:05.997 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.996+0000 7ff1c0ff9700 1 -- 172.21.15.99:0/2941790912 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v189) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff1c4018020 con 0x7ff1cc070ad0 2023-10-08T00:22:05.997 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2023-10-08T00:22:05.999 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.998+0000 7ff1d2cc8700 1 -- 172.21.15.99:0/2941790912 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff1b8066f30 msgr2=0x7ff1b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:05.999 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.998+0000 7ff1d2cc8700 1 --2- 172.21.15.99:0/2941790912 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff1b8066f30 0x7ff1b80693e0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc15b7b0 tx=0x7ff1c8017000 comp rx=0 tx=0).stop 2023-10-08T00:22:05.999 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.998+0000 7ff1d2cc8700 1 -- 172.21.15.99:0/2941790912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1cc070ad0 msgr2=0x7ff1cc0b2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:06.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.998+0000 7ff1d2cc8700 1 --2- 172.21.15.99:0/2941790912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1cc070ad0 0x7ff1cc0b2d30 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4009e90 tx=0x7ff1c400b040 comp rx=0 tx=0).stop 2023-10-08T00:22:06.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.998+0000 7ff1d2cc8700 1 -- 172.21.15.99:0/2941790912 shutdown_connections 2023-10-08T00:22:06.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.998+0000 7ff1d2cc8700 1 --2- 172.21.15.99:0/2941790912 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff1b8066f30 0x7ff1b80693e0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:06.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.998+0000 7ff1d2cc8700 1 --2- 172.21.15.99:0/2941790912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff1cc067be0 0x7ff1cc0c2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:06.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.998+0000 7ff1d2cc8700 1 --2- 172.21.15.99:0/2941790912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1cc070ad0 0x7ff1cc0b2d30 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:06.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.998+0000 7ff1d2cc8700 1 --2- 172.21.15.99:0/2941790912 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff1cc070700 0x7ff1cc154c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:06.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.998+0000 7ff1d2cc8700 1 -- 172.21.15.99:0/2941790912 >> 172.21.15.99:0/2941790912 conn(0x7ff1cc0ce4d0 msgr2=0x7ff1cc155c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:06.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.999+0000 7ff1d2cc8700 1 -- 172.21.15.99:0/2941790912 shutdown_connections 2023-10-08T00:22:06.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:05.999+0000 7ff1d2cc8700 1 -- 172.21.15.99:0/2941790912 wait complete. 2023-10-08T00:22:06.011 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T00:22:06.128 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.127+0000 7f4c87361700 1 Processor -- start 2023-10-08T00:22:06.128 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.127+0000 7f4c87361700 1 -- start start 2023-10-08T00:22:06.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.128+0000 7f4c87361700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4c8015a290 0x7f4c8006a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:06.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.128+0000 7f4c87361700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4c8006aae0 0x7f4c800cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:06.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.128+0000 7f4c87361700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c800b0790 con 0x7f4c80159ec0 2023-10-08T00:22:06.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.128+0000 7f4c87361700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c8006af60 con 0x7f4c8006aae0 2023-10-08T00:22:06.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.128+0000 7f4c87361700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c800ce1f0 con 0x7f4c8015a290 2023-10-08T00:22:06.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.128+0000 7f4c858fe700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4c8006aae0 0x7f4c800cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:06.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.128+0000 7f4c848fc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4c8015a290 0x7f4c8006a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:06.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.128+0000 7f4c858fe700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4c8006aae0 0x7f4c800cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.99:51548/0 (socket says 172.21.15.99:51548) 2023-10-08T00:22:06.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.128+0000 7f4c858fe700 1 -- 172.21.15.99:0/178252959 learned_addr learned my addr 172.21.15.99:0/178252959 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:06.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.129+0000 7f4c848fc700 1 -- 172.21.15.99:0/178252959 >> v2:172.21.15.191:3300/0 conn(0x7f4c8006aae0 msgr2=0x7f4c800cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:06.130 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.129+0000 7f4c848fc700 1 --2- 172.21.15.99:0/178252959 >> v2:172.21.15.191:3300/0 conn(0x7f4c8006aae0 0x7f4c800cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:06.131 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.129+0000 7f4c848fc700 1 -- 172.21.15.99:0/178252959 >> v1:172.21.15.191:6789/0 conn(0x7f4c80159ec0 legacy=0x7f4c800af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:06.131 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.129+0000 7f4c848fc700 1 -- 172.21.15.99:0/178252959 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c800b0b20 con 0x7f4c8015a290 2023-10-08T00:22:06.131 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.129+0000 7f4c858fe700 1 --2- 172.21.15.99:0/178252959 >> v2:172.21.15.191:3300/0 conn(0x7f4c8006aae0 0x7f4c800cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:22:06.131 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.129+0000 7f4c848fc700 1 --2- 172.21.15.99:0/178252959 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4c8015a290 0x7f4c8006a5a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4c700098e0 tx=0x7f4c700091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92926e3eccc07d1a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:06.131 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.129+0000 7f4c77fff700 1 -- 172.21.15.99:0/178252959 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c7000b640 con 0x7f4c8015a290 2023-10-08T00:22:06.131 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.129+0000 7f4c77fff700 1 -- 172.21.15.99:0/178252959 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c7000b7a0 con 0x7f4c8015a290 2023-10-08T00:22:06.131 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.129+0000 7f4c77fff700 1 -- 172.21.15.99:0/178252959 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c7000bb40 con 0x7f4c8015a290 2023-10-08T00:22:06.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.129+0000 7f4c87361700 1 -- 172.21.15.99:0/178252959 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4c8015a290 msgr2=0x7f4c8006a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:06.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.129+0000 7f4c87361700 1 --2- 172.21.15.99:0/178252959 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4c8015a290 0x7f4c8006a5a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4c700098e0 tx=0x7f4c700091a0 comp rx=0 tx=0).stop 2023-10-08T00:22:06.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.129+0000 7f4c87361700 1 -- 172.21.15.99:0/178252959 shutdown_connections 2023-10-08T00:22:06.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.129+0000 7f4c87361700 1 --2- 172.21.15.99:0/178252959 >> v2:172.21.15.191:3300/0 conn(0x7f4c8006aae0 0x7f4c800cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:06.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.129+0000 7f4c87361700 1 --2- 172.21.15.99:0/178252959 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4c8015a290 0x7f4c8006a5a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:06.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.129+0000 7f4c87361700 1 -- 172.21.15.99:0/178252959 >> 172.21.15.99:0/178252959 conn(0x7f4c800d1880 msgr2=0x7f4c800cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:06.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.129+0000 7f4c87361700 1 -- 172.21.15.99:0/178252959 shutdown_connections 2023-10-08T00:22:06.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.129+0000 7f4c87361700 1 -- 172.21.15.99:0/178252959 wait complete. 2023-10-08T00:22:06.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.130+0000 7f4c87361700 1 Processor -- start 2023-10-08T00:22:06.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.130+0000 7f4c87361700 1 -- start start 2023-10-08T00:22:06.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.131+0000 7f4c87361700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c8006aae0 0x7f4c80156310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:06.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.131+0000 7f4c87361700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c80159ec0 0x7f4c80156850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:06.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.131+0000 7f4c87361700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4c8015a290 0x7f4c8016afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:06.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.131+0000 7f4c87361700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c800b01b0 con 0x7f4c8006aae0 2023-10-08T00:22:06.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.131+0000 7f4c87361700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8015cf70 con 0x7f4c80159ec0 2023-10-08T00:22:06.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.131+0000 7f4c87361700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8015c3f0 con 0x7f4c8015a290 2023-10-08T00:22:06.135 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.131+0000 7f4c850fd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c8006aae0 0x7f4c80156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:06.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.131+0000 7f4c850fd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c8006aae0 0x7f4c80156310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:32936/0 (socket says 172.21.15.99:32936) 2023-10-08T00:22:06.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.131+0000 7f4c850fd700 1 -- 172.21.15.99:0/2016355164 learned_addr learned my addr 172.21.15.99:0/2016355164 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:06.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.131+0000 7f4c848fc700 1 --2- 172.21.15.99:0/2016355164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c80159ec0 0x7f4c80156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:06.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.131+0000 7f4c858fe700 1 --2- 172.21.15.99:0/2016355164 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4c8015a290 0x7f4c8016afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:06.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.131+0000 7f4c850fd700 1 -- 172.21.15.99:0/2016355164 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4c8015a290 msgr2=0x7f4c8016afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:06.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.131+0000 7f4c850fd700 1 --2- 172.21.15.99:0/2016355164 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4c8015a290 0x7f4c8016afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:06.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.131+0000 7f4c850fd700 1 -- 172.21.15.99:0/2016355164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c80159ec0 msgr2=0x7f4c80156850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:06.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.131+0000 7f4c850fd700 1 --2- 172.21.15.99:0/2016355164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c80159ec0 0x7f4c80156850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:06.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.131+0000 7f4c850fd700 1 -- 172.21.15.99:0/2016355164 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c780094f0 con 0x7f4c8006aae0 2023-10-08T00:22:06.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.131+0000 7f4c850fd700 1 --2- 172.21.15.99:0/2016355164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c8006aae0 0x7f4c80156310 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c00cb10 tx=0x7f4c7c0017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:06.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.131+0000 7f4c757fa700 1 -- 172.21.15.99:0/2016355164 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c00ce20 con 0x7f4c8006aae0 2023-10-08T00:22:06.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.131+0000 7f4c757fa700 1 -- 172.21.15.99:0/2016355164 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c7c00a7e0 con 0x7f4c8006aae0 2023-10-08T00:22:06.137 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.131+0000 7f4c757fa700 1 -- 172.21.15.99:0/2016355164 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c00aba0 con 0x7f4c8006aae0 2023-10-08T00:22:06.137 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.131+0000 7f4c87361700 1 -- 172.21.15.99:0/2016355164 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c70009420 con 0x7f4c8006aae0 2023-10-08T00:22:06.137 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.131+0000 7f4c87361700 1 -- 172.21.15.99:0/2016355164 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c8015cc90 con 0x7f4c8006aae0 2023-10-08T00:22:06.137 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.132+0000 7f4c87361700 1 -- 172.21.15.99:0/2016355164 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c8015c7e0 con 0x7f4c8006aae0 2023-10-08T00:22:06.137 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.133+0000 7f4c757fa700 1 -- 172.21.15.99:0/2016355164 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4c7c003030 con 0x7f4c8006aae0 2023-10-08T00:22:06.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.133+0000 7f4c757fa700 1 --2- 172.21.15.99:0/2016355164 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4c64066f30 0x7f4c640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:06.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.133+0000 7f4c757fa700 1 -- 172.21.15.99:0/2016355164 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5734+0+0 (secure 0 0 0) 0x7f4c7c0844e0 con 0x7f4c8006aae0 2023-10-08T00:22:06.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.133+0000 7f4c848fc700 1 --2- 172.21.15.99:0/2016355164 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4c64066f30 0x7f4c640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:06.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.134+0000 7f4c848fc700 1 --2- 172.21.15.99:0/2016355164 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4c64066f30 0x7f4c640693e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f4c700093f0 tx=0x7f4c70017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:06.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.137+0000 7f4c757fa700 1 -- 172.21.15.99:0/2016355164 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4c7c051730 con 0x7f4c8006aae0 2023-10-08T00:22:06.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.321+0000 7f4c87361700 1 -- 172.21.15.99:0/2016355164 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4c8004c3f0 con 0x7f4c8006aae0 2023-10-08T00:22:06.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.322+0000 7f4c757fa700 1 -- 172.21.15.99:0/2016355164 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v189) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4c7c054d50 con 0x7f4c8006aae0 2023-10-08T00:22:06.323 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2023-10-08T00:22:06.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.324+0000 7f4c87361700 1 -- 172.21.15.99:0/2016355164 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4c64066f30 msgr2=0x7f4c640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:06.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.324+0000 7f4c87361700 1 --2- 172.21.15.99:0/2016355164 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4c64066f30 0x7f4c640693e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f4c700093f0 tx=0x7f4c70017000 comp rx=0 tx=0).stop 2023-10-08T00:22:06.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.324+0000 7f4c87361700 1 -- 172.21.15.99:0/2016355164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c8006aae0 msgr2=0x7f4c80156310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:06.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.324+0000 7f4c87361700 1 --2- 172.21.15.99:0/2016355164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c8006aae0 0x7f4c80156310 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c00cb10 tx=0x7f4c7c0017b0 comp rx=0 tx=0).stop 2023-10-08T00:22:06.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.324+0000 7f4c87361700 1 -- 172.21.15.99:0/2016355164 shutdown_connections 2023-10-08T00:22:06.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.324+0000 7f4c87361700 1 --2- 172.21.15.99:0/2016355164 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4c64066f30 0x7f4c640693e0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:06.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.324+0000 7f4c87361700 1 --2- 172.21.15.99:0/2016355164 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4c8015a290 0x7f4c8016afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:06.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.324+0000 7f4c87361700 1 --2- 172.21.15.99:0/2016355164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c8006aae0 0x7f4c80156310 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:06.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.324+0000 7f4c87361700 1 --2- 172.21.15.99:0/2016355164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c80159ec0 0x7f4c80156850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:06.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.324+0000 7f4c87361700 1 -- 172.21.15.99:0/2016355164 >> 172.21.15.99:0/2016355164 conn(0x7f4c800d1880 msgr2=0x7f4c80155990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:06.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.324+0000 7f4c87361700 1 -- 172.21.15.99:0/2016355164 shutdown_connections 2023-10-08T00:22:06.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.324+0000 7f4c87361700 1 -- 172.21.15.99:0/2016355164 wait complete. 2023-10-08T00:22:06.337 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T00:22:06.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.452+0000 7f6484eb2700 1 Processor -- start 2023-10-08T00:22:06.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.452+0000 7f6484eb2700 1 -- start start 2023-10-08T00:22:06.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.453+0000 7f6484eb2700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f648015b9a0 0x7f64800caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:06.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.453+0000 7f6484eb2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f648006d370 0x7f648006d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:06.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.453+0000 7f6484eb2700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f648015c8d0 con 0x7f6480152300 2023-10-08T00:22:06.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.453+0000 7f6484eb2700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64800b1400 con 0x7f648015b9a0 2023-10-08T00:22:06.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.453+0000 7f6484eb2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f648015be20 con 0x7f648006d370 2023-10-08T00:22:06.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.453+0000 7f647e59c700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f6480152300 0x7f6480158df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56786/0 (socket says 172.21.15.99:56786) 2023-10-08T00:22:06.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.453+0000 7f647e59c700 1 -- 172.21.15.99:0/3021112733 learned_addr learned my addr 172.21.15.99:0/3021112733 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:22:06.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.454+0000 7f647d59a700 1 -- 172.21.15.99:0/3021112733 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2910427565 0 0) 0x7f648015c8d0 con 0x7f6480152300 2023-10-08T00:22:06.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.454+0000 7f647d59a700 1 -- 172.21.15.99:0/3021112733 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6468003570 con 0x7f6480152300 2023-10-08T00:22:06.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.454+0000 7f647ed9d700 1 --2- 172.21.15.99:0/3021112733 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f648006d370 0x7f648006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:06.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.454+0000 7f647dd9b700 1 --2- 172.21.15.99:0/3021112733 >> v2:172.21.15.191:3300/0 conn(0x7f648015b9a0 0x7f64800caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:06.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.454+0000 7f647d59a700 1 -- 172.21.15.99:0/3021112733 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1732783468 0 0) 0x7f6468003570 con 0x7f6480152300 2023-10-08T00:22:06.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.454+0000 7f647d59a700 1 -- 172.21.15.99:0/3021112733 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f648006d370 msgr2=0x7f648006d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:06.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.454+0000 7f647d59a700 1 --2- 172.21.15.99:0/3021112733 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f648006d370 0x7f648006d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:06.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.454+0000 7f647d59a700 1 -- 172.21.15.99:0/3021112733 >> v2:172.21.15.191:3300/0 conn(0x7f648015b9a0 msgr2=0x7f64800caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:06.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.454+0000 7f647d59a700 1 --2- 172.21.15.99:0/3021112733 >> v2:172.21.15.191:3300/0 conn(0x7f648015b9a0 0x7f64800caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:06.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.454+0000 7f647d59a700 1 -- 172.21.15.99:0/3021112733 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64800c9370 con 0x7f6480152300 2023-10-08T00:22:06.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.454+0000 7f647d59a700 1 -- 172.21.15.99:0/3021112733 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f64700017f0 con 0x7f6480152300 2023-10-08T00:22:06.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.454+0000 7f647d59a700 1 -- 172.21.15.99:0/3021112733 >> v1:172.21.15.99:6789/0 conn(0x7f6480152300 legacy=0x7f6480158df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:06.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.454+0000 7f647d59a700 1 --2- 172.21.15.99:0/3021112733 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f648006d370 0x7f6468003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:06.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.454+0000 7f647d59a700 1 --2- 172.21.15.99:0/3021112733 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6468003fe0 0x7f6468004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:06.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.454+0000 7f647d59a700 1 --2- 172.21.15.99:0/3021112733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6468004d50 0x7f64680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:06.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.454+0000 7f647d59a700 1 -- 172.21.15.99:0/3021112733 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f648015be20 con 0x7f6468004d50 2023-10-08T00:22:06.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.454+0000 7f647d59a700 1 -- 172.21.15.99:0/3021112733 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646800bac0 con 0x7f6468003fe0 2023-10-08T00:22:06.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.454+0000 7f647d59a700 1 -- 172.21.15.99:0/3021112733 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f646800bc80 con 0x7f648006d370 2023-10-08T00:22:06.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.454+0000 7f647dd9b700 1 --2- 172.21.15.99:0/3021112733 >> v2:172.21.15.191:3300/0 conn(0x7f648015b9a0 0x7f64800caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:22:06.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.454+0000 7f647ed9d700 1 --2- 172.21.15.99:0/3021112733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6468004d50 0x7f64680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:06.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.454+0000 7f647e59c700 1 --2- 172.21.15.99:0/3021112733 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f648006d370 0x7f6468003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:06.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.454+0000 7f647dd9b700 1 --2- 172.21.15.99:0/3021112733 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6468003fe0 0x7f6468004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:06.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.454+0000 7f647ed9d700 1 -- 172.21.15.99:0/3021112733 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f648006d370 msgr2=0x7f6468003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:06.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.454+0000 7f647ed9d700 1 --2- 172.21.15.99:0/3021112733 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f648006d370 0x7f6468003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:06.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.454+0000 7f647ed9d700 1 -- 172.21.15.99:0/3021112733 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6468003fe0 msgr2=0x7f6468004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:06.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.454+0000 7f647ed9d700 1 --2- 172.21.15.99:0/3021112733 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6468003fe0 0x7f6468004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:06.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.454+0000 7f647ed9d700 1 -- 172.21.15.99:0/3021112733 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6468003570 con 0x7f6468004d50 2023-10-08T00:22:06.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.455+0000 7f647ed9d700 1 --2- 172.21.15.99:0/3021112733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6468004d50 0x7f64680051c0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f6478006f00 tx=0x7f64780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0cae94bf00d1d19 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:06.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.455+0000 7f647d59a700 1 -- 172.21.15.99:0/3021112733 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647800e710 con 0x7f6468004d50 2023-10-08T00:22:06.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.455+0000 7f647d59a700 1 -- 172.21.15.99:0/3021112733 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f647800e870 con 0x7f6468004d50 2023-10-08T00:22:06.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.455+0000 7f647d59a700 1 -- 172.21.15.99:0/3021112733 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647800ec30 con 0x7f6468004d50 2023-10-08T00:22:06.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.455+0000 7f6484eb2700 1 -- 172.21.15.99:0/3021112733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6468004d50 msgr2=0x7f64680051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:06.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.455+0000 7f6484eb2700 1 --2- 172.21.15.99:0/3021112733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6468004d50 0x7f64680051c0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f6478006f00 tx=0x7f64780091a0 comp rx=0 tx=0).stop 2023-10-08T00:22:06.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.455+0000 7f647ed9d700 1 -- 172.21.15.99:0/3021112733 reap_dead start 2023-10-08T00:22:06.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.455+0000 7f6484eb2700 1 -- 172.21.15.99:0/3021112733 shutdown_connections 2023-10-08T00:22:06.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.455+0000 7f6484eb2700 1 -- 172.21.15.99:0/3021112733 >> 172.21.15.99:0/3021112733 conn(0x7f6480069870 msgr2=0x7f6480069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:06.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.455+0000 7f6484eb2700 1 -- 172.21.15.99:0/3021112733 shutdown_connections 2023-10-08T00:22:06.474 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.455+0000 7f6484eb2700 1 -- 172.21.15.99:0/3021112733 wait complete. 2023-10-08T00:22:06.474 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.456+0000 7f6484eb2700 1 Processor -- start 2023-10-08T00:22:06.474 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.456+0000 7f6484eb2700 1 -- start start 2023-10-08T00:22:06.474 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.457+0000 7f6484eb2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6468004d50 0x7f648014fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:06.474 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.457+0000 7f6484eb2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6480151390 0x7f64800c6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:06.474 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.457+0000 7f6484eb2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f648014f150 0x7f64800b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:06.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.457+0000 7f6484eb2700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64800c7320 con 0x7f6468004d50 2023-10-08T00:22:06.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.457+0000 7f6484eb2700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6480150f00 con 0x7f6480151390 2023-10-08T00:22:06.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.457+0000 7f6484eb2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6480151800 con 0x7f648014f150 2023-10-08T00:22:06.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.457+0000 7f647e59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6468004d50 0x7f648014fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:06.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.457+0000 7f647e59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6468004d50 0x7f648014fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:32958/0 (socket says 172.21.15.99:32958) 2023-10-08T00:22:06.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.457+0000 7f647e59c700 1 -- 172.21.15.99:0/1680222563 learned_addr learned my addr 172.21.15.99:0/1680222563 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:06.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.457+0000 7f647ed9d700 1 --2- 172.21.15.99:0/1680222563 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f648014f150 0x7f64800b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:06.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.457+0000 7f647dd9b700 1 --2- 172.21.15.99:0/1680222563 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6480151390 0x7f64800c6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:06.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.457+0000 7f647e59c700 1 -- 172.21.15.99:0/1680222563 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f648014f150 msgr2=0x7f64800b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:06.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.457+0000 7f647e59c700 1 --2- 172.21.15.99:0/1680222563 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f648014f150 0x7f64800b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:06.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.457+0000 7f647e59c700 1 -- 172.21.15.99:0/1680222563 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6480151390 msgr2=0x7f64800c6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:06.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.457+0000 7f647e59c700 1 --2- 172.21.15.99:0/1680222563 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6480151390 0x7f64800c6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:06.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.458+0000 7f647e59c700 1 -- 172.21.15.99:0/1680222563 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64801502f0 con 0x7f6468004d50 2023-10-08T00:22:06.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.458+0000 7f647dd9b700 1 --2- 172.21.15.99:0/1680222563 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6480151390 0x7f64800c6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:22:06.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.458+0000 7f647e59c700 1 --2- 172.21.15.99:0/1680222563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6468004d50 0x7f648014fc60 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f6470007b40 tx=0x7f64800cdcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:06.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.458+0000 7f646e7fc700 1 -- 172.21.15.99:0/1680222563 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647000adb0 con 0x7f6468004d50 2023-10-08T00:22:06.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.458+0000 7f646e7fc700 1 -- 172.21.15.99:0/1680222563 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64700133f0 con 0x7f6468004d50 2023-10-08T00:22:06.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.458+0000 7f6484eb2700 1 -- 172.21.15.99:0/1680222563 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64801505d0 con 0x7f6468004d50 2023-10-08T00:22:06.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.458+0000 7f646e7fc700 1 -- 172.21.15.99:0/1680222563 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64700137b0 con 0x7f6468004d50 2023-10-08T00:22:06.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.458+0000 7f6484eb2700 1 -- 172.21.15.99:0/1680222563 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6480150b90 con 0x7f6468004d50 2023-10-08T00:22:06.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.459+0000 7f6484eb2700 1 -- 172.21.15.99:0/1680222563 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f648004c3c0 con 0x7f6468004d50 2023-10-08T00:22:06.478 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.459+0000 7f646e7fc700 1 -- 172.21.15.99:0/1680222563 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6470013b60 con 0x7f6468004d50 2023-10-08T00:22:06.479 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.460+0000 7f646e7fc700 1 --2- 172.21.15.99:0/1680222563 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6460066f30 0x7f64600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:06.479 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.460+0000 7f646e7fc700 1 -- 172.21.15.99:0/1680222563 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5734+0+0 (secure 0 0 0) 0x7f6470085370 con 0x7f6468004d50 2023-10-08T00:22:06.479 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.460+0000 7f647dd9b700 1 --2- 172.21.15.99:0/1680222563 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6460066f30 0x7f64600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:06.479 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.460+0000 7f647dd9b700 1 --2- 172.21.15.99:0/1680222563 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6460066f30 0x7f64600693e0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f6474009310 tx=0x7f647400b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:06.480 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.464+0000 7f646e7fc700 1 -- 172.21.15.99:0/1680222563 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f64700525c0 con 0x7f6468004d50 2023-10-08T00:22:06.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.644+0000 7f6484eb2700 1 -- 172.21.15.99:0/1680222563 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f64800cdf10 con 0x7f6468004d50 2023-10-08T00:22:06.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.645+0000 7f646e7fc700 1 -- 172.21.15.99:0/1680222563 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v189) v1 ==== 100+0+11 (secure 0 0 0) 0x7f64700170a0 con 0x7f6468004d50 2023-10-08T00:22:06.647 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2023-10-08T00:22:06.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.647+0000 7f6484eb2700 1 -- 172.21.15.99:0/1680222563 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6460066f30 msgr2=0x7f64600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:06.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.647+0000 7f6484eb2700 1 --2- 172.21.15.99:0/1680222563 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6460066f30 0x7f64600693e0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f6474009310 tx=0x7f647400b040 comp rx=0 tx=0).stop 2023-10-08T00:22:06.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.647+0000 7f6484eb2700 1 -- 172.21.15.99:0/1680222563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6468004d50 msgr2=0x7f648014fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:06.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.647+0000 7f6484eb2700 1 --2- 172.21.15.99:0/1680222563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6468004d50 0x7f648014fc60 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f6470007b40 tx=0x7f64800cdcf0 comp rx=0 tx=0).stop 2023-10-08T00:22:06.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.647+0000 7f6484eb2700 1 -- 172.21.15.99:0/1680222563 shutdown_connections 2023-10-08T00:22:06.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.647+0000 7f6484eb2700 1 --2- 172.21.15.99:0/1680222563 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6460066f30 0x7f64600693e0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:06.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.647+0000 7f6484eb2700 1 --2- 172.21.15.99:0/1680222563 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f648014f150 0x7f64800b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:06.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.647+0000 7f6484eb2700 1 --2- 172.21.15.99:0/1680222563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6468004d50 0x7f648014fc60 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:06.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.647+0000 7f6484eb2700 1 --2- 172.21.15.99:0/1680222563 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6480151390 0x7f64800c6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:06.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.647+0000 7f6484eb2700 1 -- 172.21.15.99:0/1680222563 >> 172.21.15.99:0/1680222563 conn(0x7f6480069870 msgr2=0x7f64800c39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:06.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.647+0000 7f6484eb2700 1 -- 172.21.15.99:0/1680222563 shutdown_connections 2023-10-08T00:22:06.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.647+0000 7f6484eb2700 1 -- 172.21.15.99:0/1680222563 wait complete. 2023-10-08T00:22:06.661 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:22:06.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.773+0000 7f6cdad06700 1 Processor -- start 2023-10-08T00:22:06.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.773+0000 7f6cdad06700 1 -- start start 2023-10-08T00:22:06.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.774+0000 7f6cdad06700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6cd4157780 0x7f6cd4158c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:06.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.774+0000 7f6cdad06700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f6cd406f2d0 0x7f6cd4157b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:06.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.774+0000 7f6cdad06700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cd415b8a0 con 0x7f6cd4070700 2023-10-08T00:22:06.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.774+0000 7f6cdad06700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cd4159150 con 0x7f6cd4157780 2023-10-08T00:22:06.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.774+0000 7f6cdad06700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cd4158090 con 0x7f6cd406f2d0 2023-10-08T00:22:06.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.774+0000 7f6cd9d04700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6cd4157780 0x7f6cd4158c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:06.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.774+0000 7f6cd9d04700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6cd4157780 0x7f6cd4158c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:32964/0 (socket says 172.21.15.99:32964) 2023-10-08T00:22:06.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.774+0000 7f6cd9d04700 1 -- 172.21.15.99:0/2434610428 learned_addr learned my addr 172.21.15.99:0/2434610428 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:06.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.775+0000 7f6cd9503700 1 --2- 172.21.15.99:0/2434610428 >> v2:172.21.15.191:3300/0 conn(0x7f6cd406f2d0 0x7f6cd4157b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:06.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.775+0000 7f6cd9d04700 1 -- 172.21.15.99:0/2434610428 >> v2:172.21.15.191:3300/0 conn(0x7f6cd406f2d0 msgr2=0x7f6cd4157b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:06.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.775+0000 7f6cd9d04700 1 --2- 172.21.15.99:0/2434610428 >> v2:172.21.15.191:3300/0 conn(0x7f6cd406f2d0 0x7f6cd4157b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:06.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.775+0000 7f6cd9d04700 1 -- 172.21.15.99:0/2434610428 >> v1:172.21.15.191:6789/0 conn(0x7f6cd4070700 legacy=0x7f6cd406a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:06.779 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.775+0000 7f6cd9d04700 1 -- 172.21.15.99:0/2434610428 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd415bb80 con 0x7f6cd4157780 2023-10-08T00:22:06.779 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.775+0000 7f6cd9d04700 1 --2- 172.21.15.99:0/2434610428 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cd4157780 0x7f6cd4158c10 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f6cd40ce960 tx=0x7f6cc4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=963e361043a877ed server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:06.779 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.775+0000 7f6cd8d02700 1 -- 172.21.15.99:0/2434610428 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc400e070 con 0x7f6cd4157780 2023-10-08T00:22:06.779 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.775+0000 7f6cd8d02700 1 -- 172.21.15.99:0/2434610428 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cc400ba20 con 0x7f6cd4157780 2023-10-08T00:22:06.779 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.775+0000 7f6cd8d02700 1 -- 172.21.15.99:0/2434610428 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc40092e0 con 0x7f6cd4157780 2023-10-08T00:22:06.779 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.776+0000 7f6cdad06700 1 -- 172.21.15.99:0/2434610428 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cd4157780 msgr2=0x7f6cd4158c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:06.779 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.776+0000 7f6cdad06700 1 --2- 172.21.15.99:0/2434610428 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cd4157780 0x7f6cd4158c10 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f6cd40ce960 tx=0x7f6cc4009840 comp rx=0 tx=0).stop 2023-10-08T00:22:06.779 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.776+0000 7f6cdad06700 1 -- 172.21.15.99:0/2434610428 shutdown_connections 2023-10-08T00:22:06.779 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.776+0000 7f6cdad06700 1 --2- 172.21.15.99:0/2434610428 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cd4157780 0x7f6cd4158c10 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:06.779 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.776+0000 7f6cdad06700 1 --2- 172.21.15.99:0/2434610428 >> v2:172.21.15.191:3300/0 conn(0x7f6cd406f2d0 0x7f6cd4157b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:06.780 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.776+0000 7f6cdad06700 1 -- 172.21.15.99:0/2434610428 >> 172.21.15.99:0/2434610428 conn(0x7f6cd40cd8f0 msgr2=0x7f6cd40cdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:06.780 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.776+0000 7f6cdad06700 1 -- 172.21.15.99:0/2434610428 shutdown_connections 2023-10-08T00:22:06.780 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.776+0000 7f6cdad06700 1 -- 172.21.15.99:0/2434610428 wait complete. 2023-10-08T00:22:06.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.777+0000 7f6cdad06700 1 Processor -- start 2023-10-08T00:22:06.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.777+0000 7f6cdad06700 1 -- start start 2023-10-08T00:22:06.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.777+0000 7f6cdad06700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6cd406f2d0 0x7f6cd4149000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:06.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.777+0000 7f6cdad06700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6cd4070700 0x7f6cd40c1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:06.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.777+0000 7f6cdad06700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cd4157780 0x7f6cd4167fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:06.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.777+0000 7f6cdad06700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd40c1370 con 0x7f6cd4157780 2023-10-08T00:22:06.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.777+0000 7f6cdad06700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd4168980 con 0x7f6cd4070700 2023-10-08T00:22:06.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.777+0000 7f6cdad06700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd4168c60 con 0x7f6cd406f2d0 2023-10-08T00:22:06.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.778+0000 7f6cda505700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cd4157780 0x7f6cd4167fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:06.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.778+0000 7f6cda505700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cd4157780 0x7f6cd4167fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:32974/0 (socket says 172.21.15.99:32974) 2023-10-08T00:22:06.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.778+0000 7f6cda505700 1 -- 172.21.15.99:0/1792900704 learned_addr learned my addr 172.21.15.99:0/1792900704 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:06.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.778+0000 7f6cd9d04700 1 --2- 172.21.15.99:0/1792900704 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6cd406f2d0 0x7f6cd4149000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:06.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.778+0000 7f6cd9503700 1 --2- 172.21.15.99:0/1792900704 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6cd4070700 0x7f6cd40c1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:06.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.778+0000 7f6cda505700 1 -- 172.21.15.99:0/1792900704 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6cd406f2d0 msgr2=0x7f6cd4149000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:06.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.778+0000 7f6cda505700 1 --2- 172.21.15.99:0/1792900704 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6cd406f2d0 0x7f6cd4149000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:06.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.778+0000 7f6cda505700 1 -- 172.21.15.99:0/1792900704 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6cd4070700 msgr2=0x7f6cd40c1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:06.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.778+0000 7f6cda505700 1 --2- 172.21.15.99:0/1792900704 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6cd4070700 0x7f6cd40c1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:06.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.778+0000 7f6cda505700 1 -- 172.21.15.99:0/1792900704 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc4009ae0 con 0x7f6cd4157780 2023-10-08T00:22:06.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.778+0000 7f6cda505700 1 --2- 172.21.15.99:0/1792900704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cd4157780 0x7f6cd4167fd0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc00c560 tx=0x7f6ccc007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:06.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.778+0000 7f6cd1ffb700 1 -- 172.21.15.99:0/1792900704 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc001120 con 0x7f6cd4157780 2023-10-08T00:22:06.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.778+0000 7f6cd1ffb700 1 -- 172.21.15.99:0/1792900704 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ccc011590 con 0x7f6cd4157780 2023-10-08T00:22:06.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.778+0000 7f6cdad06700 1 -- 172.21.15.99:0/1792900704 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cd40c1610 con 0x7f6cd4157780 2023-10-08T00:22:06.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.778+0000 7f6cd1ffb700 1 -- 172.21.15.99:0/1792900704 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc011950 con 0x7f6cd4157780 2023-10-08T00:22:06.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.778+0000 7f6cdad06700 1 -- 172.21.15.99:0/1792900704 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cd40c2070 con 0x7f6cd4157780 2023-10-08T00:22:06.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.779+0000 7f6cdad06700 1 -- 172.21.15.99:0/1792900704 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cd40c2260 con 0x7f6cd4157780 2023-10-08T00:22:06.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.779+0000 7f6cd1ffb700 1 -- 172.21.15.99:0/1792900704 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6ccc011d00 con 0x7f6cd4157780 2023-10-08T00:22:06.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.780+0000 7f6cd1ffb700 1 --2- 172.21.15.99:0/1792900704 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6cc0066f30 0x7f6cc00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:06.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.780+0000 7f6cd1ffb700 1 -- 172.21.15.99:0/1792900704 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5734+0+0 (secure 0 0 0) 0x7f6ccc083870 con 0x7f6cd4157780 2023-10-08T00:22:06.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.780+0000 7f6cd9d04700 1 --2- 172.21.15.99:0/1792900704 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6cc0066f30 0x7f6cc00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:06.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.780+0000 7f6cd9d04700 1 --2- 172.21.15.99:0/1792900704 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6cc0066f30 0x7f6cc00693e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4009880 tx=0x7f6cc4018000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:06.786 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:06.784+0000 7f6cd1ffb700 1 -- 172.21.15.99:0/1792900704 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6ccc050ac0 con 0x7f6cd4157780 2023-10-08T00:22:07.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.007+0000 7f6cdad06700 1 -- 172.21.15.99:0/1792900704 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6cd415be60 con 0x7f6cd4157780 2023-10-08T00:22:07.009 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.008+0000 7f6cd1ffb700 1 -- 172.21.15.99:0/1792900704 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+4604 (secure 0 0 0) 0x7f6ccc017070 con 0x7f6cd4157780 2023-10-08T00:22:07.010 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:22:07.010 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":84,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/3862684847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3862684847},{"type":"v1","addr":"172.21.15.191:6836","nonce":3862684847}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":8106,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/359623206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":359623206},{"type":"v1","addr":"172.21.15.99:6834","nonce":359623206}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":8109,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3064101777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":3064101777},{"type":"v1","addr":"172.21.15.99:6835","nonce":3064101777}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:21:29.659394+0000","modified":"2023-10-08T00:21:33.311802+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8124},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8124":{"gid":8124,"name":"d","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/2466421758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":2466421758},{"type":"v1","addr":"172.21.15.191:6837","nonce":2466421758}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-08T00:22:07.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.010+0000 7f6cdad06700 1 -- 172.21.15.99:0/1792900704 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6cc0066f30 msgr2=0x7f6cc00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:07.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.010+0000 7f6cdad06700 1 --2- 172.21.15.99:0/1792900704 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6cc0066f30 0x7f6cc00693e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4009880 tx=0x7f6cc4018000 comp rx=0 tx=0).stop 2023-10-08T00:22:07.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.010+0000 7f6cdad06700 1 -- 172.21.15.99:0/1792900704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cd4157780 msgr2=0x7f6cd4167fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:07.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.010+0000 7f6cdad06700 1 --2- 172.21.15.99:0/1792900704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cd4157780 0x7f6cd4167fd0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc00c560 tx=0x7f6ccc007ef0 comp rx=0 tx=0).stop 2023-10-08T00:22:07.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.010+0000 7f6cdad06700 1 -- 172.21.15.99:0/1792900704 shutdown_connections 2023-10-08T00:22:07.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.010+0000 7f6cdad06700 1 --2- 172.21.15.99:0/1792900704 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f6cc0066f30 0x7f6cc00693e0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:07.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.010+0000 7f6cdad06700 1 --2- 172.21.15.99:0/1792900704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cd4157780 0x7f6cd4167fd0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:07.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.010+0000 7f6cdad06700 1 --2- 172.21.15.99:0/1792900704 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6cd406f2d0 0x7f6cd4149000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:07.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.010+0000 7f6cdad06700 1 --2- 172.21.15.99:0/1792900704 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6cd4070700 0x7f6cd40c1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:07.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.010+0000 7f6cdad06700 1 -- 172.21.15.99:0/1792900704 >> 172.21.15.99:0/1792900704 conn(0x7f6cd40cd8f0 msgr2=0x7f6cd4154db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:07.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.010+0000 7f6cdad06700 1 -- 172.21.15.99:0/1792900704 shutdown_connections 2023-10-08T00:22:07.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.010+0000 7f6cdad06700 1 -- 172.21.15.99:0/1792900704 wait complete. 2023-10-08T00:22:07.013 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 84 2023-10-08T00:22:07.023 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:22:07.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.135+0000 7fa189f36700 1 Processor -- start 2023-10-08T00:22:07.137 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.135+0000 7fa189f36700 1 -- start start 2023-10-08T00:22:07.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.136+0000 7fa189f36700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa184157870 0x7fa18415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:07.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.136+0000 7fa189f36700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa184157c40 0x7fa184158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:07.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.136+0000 7fa189f36700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa18415cbb0 con 0x7fa18415a620 2023-10-08T00:22:07.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.136+0000 7fa189f36700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa18415bfa0 con 0x7fa184157c40 2023-10-08T00:22:07.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.136+0000 7fa189f36700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa18415c280 con 0x7fa184157870 2023-10-08T00:22:07.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.136+0000 7fa17affd700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fa18415a620 0x7fa18415d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56790/0 (socket says 172.21.15.99:56790) 2023-10-08T00:22:07.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.136+0000 7fa17affd700 1 -- 172.21.15.99:0/908679036 learned_addr learned my addr 172.21.15.99:0/908679036 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:22:07.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.136+0000 7fa183fff700 1 --2- 172.21.15.99:0/908679036 >> v2:172.21.15.191:3300/0 conn(0x7fa184157c40 0x7fa184158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:07.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.136+0000 7fa1837fe700 1 --2- 172.21.15.99:0/908679036 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa184157870 0x7fa18415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:07.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.136+0000 7fa182ffd700 1 -- 172.21.15.99:0/908679036 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1038506234 0 0) 0x7fa18415cbb0 con 0x7fa18415a620 2023-10-08T00:22:07.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.136+0000 7fa182ffd700 1 -- 172.21.15.99:0/908679036 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa170003570 con 0x7fa18415a620 2023-10-08T00:22:07.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.136+0000 7fa1837fe700 1 -- 172.21.15.99:0/908679036 >> v2:172.21.15.191:3300/0 conn(0x7fa184157c40 msgr2=0x7fa184158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:07.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.136+0000 7fa1837fe700 1 --2- 172.21.15.99:0/908679036 >> v2:172.21.15.191:3300/0 conn(0x7fa184157c40 0x7fa184158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:07.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.136+0000 7fa1837fe700 1 -- 172.21.15.99:0/908679036 >> v1:172.21.15.99:6789/0 conn(0x7fa18415a620 legacy=0x7fa18415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:07.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.136+0000 7fa1837fe700 1 -- 172.21.15.99:0/908679036 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa18415c560 con 0x7fa184157870 2023-10-08T00:22:07.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.136+0000 7fa1837fe700 1 --2- 172.21.15.99:0/908679036 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa184157870 0x7fa18415b6a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa1740098e0 tx=0x7fa1740091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8030f8ca1de381 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:07.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.137+0000 7fa182ffd700 1 -- 172.21.15.99:0/908679036 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa174011630 con 0x7fa184157870 2023-10-08T00:22:07.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.137+0000 7fa182ffd700 1 -- 172.21.15.99:0/908679036 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa174011790 con 0x7fa184157870 2023-10-08T00:22:07.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.137+0000 7fa182ffd700 1 -- 172.21.15.99:0/908679036 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa174011b30 con 0x7fa184157870 2023-10-08T00:22:07.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.137+0000 7fa189f36700 1 -- 172.21.15.99:0/908679036 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa184157870 msgr2=0x7fa18415b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:07.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.137+0000 7fa189f36700 1 --2- 172.21.15.99:0/908679036 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa184157870 0x7fa18415b6a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa1740098e0 tx=0x7fa1740091a0 comp rx=0 tx=0).stop 2023-10-08T00:22:07.141 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.137+0000 7fa189f36700 1 -- 172.21.15.99:0/908679036 shutdown_connections 2023-10-08T00:22:07.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.137+0000 7fa189f36700 1 --2- 172.21.15.99:0/908679036 >> v2:172.21.15.191:3300/0 conn(0x7fa184157c40 0x7fa184158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:07.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.137+0000 7fa189f36700 1 --2- 172.21.15.99:0/908679036 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa184157870 0x7fa18415b6a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:07.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.137+0000 7fa189f36700 1 -- 172.21.15.99:0/908679036 >> 172.21.15.99:0/908679036 conn(0x7fa184068ce0 msgr2=0x7fa1840690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:07.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.137+0000 7fa189f36700 1 -- 172.21.15.99:0/908679036 shutdown_connections 2023-10-08T00:22:07.142 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.137+0000 7fa189f36700 1 -- 172.21.15.99:0/908679036 wait complete. 2023-10-08T00:22:07.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.138+0000 7fa189f36700 1 Processor -- start 2023-10-08T00:22:07.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.138+0000 7fa189f36700 1 -- start start 2023-10-08T00:22:07.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.139+0000 7fa189f36700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa184157870 0x7fa18413f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:07.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.139+0000 7fa189f36700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa184157c40 0x7fa18415d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:07.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.139+0000 7fa189f36700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa18415a620 0x7fa18415d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:07.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.139+0000 7fa189f36700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1841526d0 con 0x7fa184157c40 2023-10-08T00:22:07.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.139+0000 7fa189f36700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1840b2e70 con 0x7fa18415a620 2023-10-08T00:22:07.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.139+0000 7fa189f36700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa184151da0 con 0x7fa184157870 2023-10-08T00:22:07.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.139+0000 7fa17affd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa184157c40 0x7fa18415d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:07.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.139+0000 7fa17affd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa184157c40 0x7fa18415d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:32980/0 (socket says 172.21.15.99:32980) 2023-10-08T00:22:07.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.139+0000 7fa17affd700 1 -- 172.21.15.99:0/3225168755 learned_addr learned my addr 172.21.15.99:0/3225168755 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:07.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.139+0000 7fa1837fe700 1 --2- 172.21.15.99:0/3225168755 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa184157870 0x7fa18413f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:07.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.139+0000 7fa183fff700 1 --2- 172.21.15.99:0/3225168755 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa18415a620 0x7fa18415d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:07.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.139+0000 7fa17affd700 1 -- 172.21.15.99:0/3225168755 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa184157870 msgr2=0x7fa18413f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:07.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.140+0000 7fa17affd700 1 --2- 172.21.15.99:0/3225168755 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa184157870 0x7fa18413f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:07.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.140+0000 7fa17affd700 1 -- 172.21.15.99:0/3225168755 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa18415a620 msgr2=0x7fa18415d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:07.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.140+0000 7fa17affd700 1 --2- 172.21.15.99:0/3225168755 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa18415a620 0x7fa18415d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:07.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.140+0000 7fa17affd700 1 -- 172.21.15.99:0/3225168755 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa174009420 con 0x7fa184157c40 2023-10-08T00:22:07.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.140+0000 7fa17affd700 1 --2- 172.21.15.99:0/3225168755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa184157c40 0x7fa18415d040 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fa16c00c590 tx=0x7fa16c007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:07.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.140+0000 7fa17bfff700 1 -- 172.21.15.99:0/3225168755 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16c005b30 con 0x7fa184157c40 2023-10-08T00:22:07.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.140+0000 7fa17bfff700 1 -- 172.21.15.99:0/3225168755 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa16c005cb0 con 0x7fa184157c40 2023-10-08T00:22:07.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.140+0000 7fa189f36700 1 -- 172.21.15.99:0/3225168755 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa184152080 con 0x7fa184157c40 2023-10-08T00:22:07.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.140+0000 7fa17bfff700 1 -- 172.21.15.99:0/3225168755 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16c007320 con 0x7fa184157c40 2023-10-08T00:22:07.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.140+0000 7fa189f36700 1 -- 172.21.15.99:0/3225168755 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa18413fcc0 con 0x7fa184157c40 2023-10-08T00:22:07.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.141+0000 7fa189f36700 1 -- 172.21.15.99:0/3225168755 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa184152880 con 0x7fa184157c40 2023-10-08T00:22:07.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.141+0000 7fa17bfff700 1 -- 172.21.15.99:0/3225168755 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa16c01b3b0 con 0x7fa184157c40 2023-10-08T00:22:07.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.142+0000 7fa17bfff700 1 --2- 172.21.15.99:0/3225168755 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa164066f30 0x7fa1640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:07.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.142+0000 7fa17bfff700 1 -- 172.21.15.99:0/3225168755 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5734+0+0 (secure 0 0 0) 0x7fa16c083c10 con 0x7fa184157c40 2023-10-08T00:22:07.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.142+0000 7fa1837fe700 1 --2- 172.21.15.99:0/3225168755 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa164066f30 0x7fa1640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:07.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.142+0000 7fa1837fe700 1 --2- 172.21.15.99:0/3225168755 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa164066f30 0x7fa1640693e0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fa1740091e0 tx=0x7fa174017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:07.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.146+0000 7fa17bfff700 1 -- 172.21.15.99:0/3225168755 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa184152880 con 0x7fa184157c40 2023-10-08T00:22:07.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.329+0000 7fa189f36700 1 -- 172.21.15.99:0/3225168755 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa18404c3c0 con 0x7fa184157c40 2023-10-08T00:22:07.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.330+0000 7fa17bfff700 1 -- 172.21.15.99:0/3225168755 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v189) v1 ==== 74+0+15422 (secure 0 0 0) 0x7fa16c017020 con 0x7fa184157c40 2023-10-08T00:22:07.332 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:22:07.332 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":189,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:22:01.171728+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:21:27.984574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-08T00:21:29.244620+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"189","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"23.18","mappings":[{"from":5,"to":3}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:22:07.334 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.332+0000 7fa189f36700 1 -- 172.21.15.99:0/3225168755 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa164066f30 msgr2=0x7fa1640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:07.334 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.332+0000 7fa189f36700 1 --2- 172.21.15.99:0/3225168755 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa164066f30 0x7fa1640693e0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fa1740091e0 tx=0x7fa174017000 comp rx=0 tx=0).stop 2023-10-08T00:22:07.334 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.332+0000 7fa189f36700 1 -- 172.21.15.99:0/3225168755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa184157c40 msgr2=0x7fa18415d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:07.334 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.332+0000 7fa189f36700 1 --2- 172.21.15.99:0/3225168755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa184157c40 0x7fa18415d040 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fa16c00c590 tx=0x7fa16c007e00 comp rx=0 tx=0).stop 2023-10-08T00:22:07.334 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.332+0000 7fa189f36700 1 -- 172.21.15.99:0/3225168755 shutdown_connections 2023-10-08T00:22:07.334 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.332+0000 7fa189f36700 1 --2- 172.21.15.99:0/3225168755 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa164066f30 0x7fa1640693e0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:07.334 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.332+0000 7fa189f36700 1 --2- 172.21.15.99:0/3225168755 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa18415a620 0x7fa18415d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:07.334 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.332+0000 7fa189f36700 1 --2- 172.21.15.99:0/3225168755 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa184157870 0x7fa18413f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:07.334 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.332+0000 7fa189f36700 1 --2- 172.21.15.99:0/3225168755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa184157c40 0x7fa18415d040 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:07.334 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.332+0000 7fa189f36700 1 -- 172.21.15.99:0/3225168755 >> 172.21.15.99:0/3225168755 conn(0x7fa184068ce0 msgr2=0x7fa184158c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:07.335 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.332+0000 7fa189f36700 1 -- 172.21.15.99:0/3225168755 shutdown_connections 2023-10-08T00:22:07.335 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.332+0000 7fa189f36700 1 -- 172.21.15.99:0/3225168755 wait complete. 2023-10-08T00:22:07.346 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T00:22:07.346 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:22:07.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.458+0000 7f10fe92b700 1 Processor -- start 2023-10-08T00:22:07.460 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.458+0000 7f10fe92b700 1 -- start start 2023-10-08T00:22:07.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.459+0000 7f10fe92b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f10f815a620 0x7f10f815d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:07.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.459+0000 7f10fe92b700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10f815cbb0 con 0x7f10f815d580 2023-10-08T00:22:07.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.459+0000 7f10fe92b700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10f815bfa0 con 0x7f10f8157870 2023-10-08T00:22:07.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.459+0000 7f10fe92b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f815bdb0 con 0x7f10f815a620 2023-10-08T00:22:07.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.459+0000 7f10fcec8700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f10f815d580 0x7f10f8158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56802/0 (socket says 172.21.15.99:56802) 2023-10-08T00:22:07.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.459+0000 7f10fcec8700 1 -- 172.21.15.99:0/118600792 learned_addr learned my addr 172.21.15.99:0/118600792 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:22:07.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.459+0000 7f10f77fe700 1 --2- 172.21.15.99:0/118600792 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f10f815a620 0x7f10f815d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:07.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.459+0000 7f10f6ffd700 1 -- 172.21.15.99:0/118600792 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3230858495 0 0) 0x7f10f815cbb0 con 0x7f10f815d580 2023-10-08T00:22:07.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.459+0000 7f10f6ffd700 1 -- 172.21.15.99:0/118600792 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10e0003570 con 0x7f10f815d580 2023-10-08T00:22:07.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.459+0000 7f10f6ffd700 1 -- 172.21.15.99:0/118600792 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1984611679 0 0) 0x7f10e0003570 con 0x7f10f815d580 2023-10-08T00:22:07.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.459+0000 7f10f6ffd700 1 -- 172.21.15.99:0/118600792 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f10f815a620 msgr2=0x7f10f815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:07.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.459+0000 7f10f6ffd700 1 --2- 172.21.15.99:0/118600792 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f10f815a620 0x7f10f815d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:07.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.459+0000 7f10f6ffd700 1 -- 172.21.15.99:0/118600792 >> v1:172.21.15.191:6789/0 conn(0x7f10f8157870 legacy=0x7f10f815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:07.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.459+0000 7f10f6ffd700 1 -- 172.21.15.99:0/118600792 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f815c280 con 0x7f10f815d580 2023-10-08T00:22:07.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.459+0000 7f10f6ffd700 1 -- 172.21.15.99:0/118600792 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f10f00017f0 con 0x7f10f815d580 2023-10-08T00:22:07.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.459+0000 7f10f6ffd700 1 -- 172.21.15.99:0/118600792 >> v1:172.21.15.99:6789/0 conn(0x7f10f815d580 legacy=0x7f10f8158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:07.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.459+0000 7f10f6ffd700 1 --2- 172.21.15.99:0/118600792 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f10f815a620 0x7f10e0003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:07.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.459+0000 7f10f6ffd700 1 --2- 172.21.15.99:0/118600792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10e0003fd0 0x7f10e0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:07.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.459+0000 7f10f6ffd700 1 --2- 172.21.15.99:0/118600792 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f10e0004d50 0x7f10e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:07.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.459+0000 7f10f6ffd700 1 -- 172.21.15.99:0/118600792 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e000bb10 con 0x7f10e0003fd0 2023-10-08T00:22:07.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.459+0000 7f10f6ffd700 1 -- 172.21.15.99:0/118600792 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e000bc80 con 0x7f10e0004d50 2023-10-08T00:22:07.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.460+0000 7f10f6ffd700 1 -- 172.21.15.99:0/118600792 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10e000be40 con 0x7f10f815a620 2023-10-08T00:22:07.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.460+0000 7f10f77fe700 1 --2- 172.21.15.99:0/118600792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10e0003fd0 0x7f10e0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:07.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.460+0000 7f10f7fff700 1 --2- 172.21.15.99:0/118600792 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f10f815a620 0x7f10e0003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:07.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.460+0000 7f10fcec8700 1 --2- 172.21.15.99:0/118600792 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f10e0004d50 0x7f10e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:07.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.460+0000 7f10f77fe700 1 -- 172.21.15.99:0/118600792 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f10f815a620 msgr2=0x7f10e0003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:07.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.460+0000 7f10f77fe700 1 --2- 172.21.15.99:0/118600792 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f10f815a620 0x7f10e0003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:07.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.460+0000 7f10f77fe700 1 -- 172.21.15.99:0/118600792 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f10e0004d50 msgr2=0x7f10e00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:07.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.460+0000 7f10f77fe700 1 --2- 172.21.15.99:0/118600792 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f10e0004d50 0x7f10e00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:07.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.460+0000 7f10f77fe700 1 -- 172.21.15.99:0/118600792 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e0003570 con 0x7f10e0003fd0 2023-10-08T00:22:07.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.460+0000 7f10fcec8700 1 --2- 172.21.15.99:0/118600792 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f10e0004d50 0x7f10e00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:22:07.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.460+0000 7f10f77fe700 1 --2- 172.21.15.99:0/118600792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10e0003fd0 0x7f10e0004810 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f10f815aa20 tx=0x7f10ec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8a642c0743a97bd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:07.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.460+0000 7f10f6ffd700 1 -- 172.21.15.99:0/118600792 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ec00e710 con 0x7f10e0003fd0 2023-10-08T00:22:07.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.460+0000 7f10f6ffd700 1 -- 172.21.15.99:0/118600792 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10ec00e870 con 0x7f10e0003fd0 2023-10-08T00:22:07.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.460+0000 7f10f6ffd700 1 -- 172.21.15.99:0/118600792 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ec00ec30 con 0x7f10e0003fd0 2023-10-08T00:22:07.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.461+0000 7f10fe92b700 1 -- 172.21.15.99:0/118600792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10e0003fd0 msgr2=0x7f10e0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:07.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.461+0000 7f10fe92b700 1 --2- 172.21.15.99:0/118600792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10e0003fd0 0x7f10e0004810 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f10f815aa20 tx=0x7f10ec0091a0 comp rx=0 tx=0).stop 2023-10-08T00:22:07.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.461+0000 7f10fcec8700 1 -- 172.21.15.99:0/118600792 reap_dead start 2023-10-08T00:22:07.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.461+0000 7f10fe92b700 1 -- 172.21.15.99:0/118600792 shutdown_connections 2023-10-08T00:22:07.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.461+0000 7f10fe92b700 1 -- 172.21.15.99:0/118600792 >> 172.21.15.99:0/118600792 conn(0x7f10f8068ce0 msgr2=0x7f10f80690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:07.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.461+0000 7f10fe92b700 1 -- 172.21.15.99:0/118600792 shutdown_connections 2023-10-08T00:22:07.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.461+0000 7f10fe92b700 1 -- 172.21.15.99:0/118600792 wait complete. 2023-10-08T00:22:07.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.463+0000 7f10fe92b700 1 Processor -- start 2023-10-08T00:22:07.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.463+0000 7f10fe92b700 1 -- start start 2023-10-08T00:22:07.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.464+0000 7f10fe92b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f10e0003fd0 0x7f10f80b0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:07.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.464+0000 7f10fe92b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f10f8160330 0x7f10f814b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:07.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.464+0000 7f10fe92b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10f8155cd0 0x7f10f8156140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:07.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.464+0000 7f10fe92b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f8156680 con 0x7f10f8155cd0 2023-10-08T00:22:07.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.464+0000 7f10fe92b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f80b0bf0 con 0x7f10e0003fd0 2023-10-08T00:22:07.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.464+0000 7f10fe92b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f80b0ed0 con 0x7f10f8160330 2023-10-08T00:22:07.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.464+0000 7f10fcec8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10f8155cd0 0x7f10f8156140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:07.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.464+0000 7f10fcec8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10f8155cd0 0x7f10f8156140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33010/0 (socket says 172.21.15.99:33010) 2023-10-08T00:22:07.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.464+0000 7f10fcec8700 1 -- 172.21.15.99:0/2304833230 learned_addr learned my addr 172.21.15.99:0/2304833230 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:07.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.464+0000 7f10f77fe700 1 --2- 172.21.15.99:0/2304833230 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f10f8160330 0x7f10f814b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:07.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.464+0000 7f10f7fff700 1 --2- 172.21.15.99:0/2304833230 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f10e0003fd0 0x7f10f80b0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:07.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.464+0000 7f10fcec8700 1 -- 172.21.15.99:0/2304833230 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f10f8160330 msgr2=0x7f10f814b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:07.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.464+0000 7f10fcec8700 1 --2- 172.21.15.99:0/2304833230 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f10f8160330 0x7f10f814b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:07.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.464+0000 7f10fcec8700 1 -- 172.21.15.99:0/2304833230 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f10e0003fd0 msgr2=0x7f10f80b0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:07.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.464+0000 7f10fcec8700 1 --2- 172.21.15.99:0/2304833230 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f10e0003fd0 0x7f10f80b0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:07.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.464+0000 7f10fcec8700 1 -- 172.21.15.99:0/2304833230 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f80b11b0 con 0x7f10f8155cd0 2023-10-08T00:22:07.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.464+0000 7f10fcec8700 1 --2- 172.21.15.99:0/2304833230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10f8155cd0 0x7f10f8156140 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f10f0002250 tx=0x7f10f00013f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:07.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.464+0000 7f10dffff700 1 -- 172.21.15.99:0/2304833230 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f000eb60 con 0x7f10f8155cd0 2023-10-08T00:22:07.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.464+0000 7f10dffff700 1 -- 172.21.15.99:0/2304833230 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10f000ecc0 con 0x7f10f8155cd0 2023-10-08T00:22:07.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.464+0000 7f10dffff700 1 -- 172.21.15.99:0/2304833230 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f0013420 con 0x7f10f8155cd0 2023-10-08T00:22:07.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.464+0000 7f10fe92b700 1 -- 172.21.15.99:0/2304833230 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10f80b1490 con 0x7f10f8155cd0 2023-10-08T00:22:07.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.464+0000 7f10fe92b700 1 -- 172.21.15.99:0/2304833230 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10f81607a0 con 0x7f10f8155cd0 2023-10-08T00:22:07.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.464+0000 7f10fe92b700 1 -- 172.21.15.99:0/2304833230 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10f804c3c0 con 0x7f10f8155cd0 2023-10-08T00:22:07.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.465+0000 7f10dffff700 1 -- 172.21.15.99:0/2304833230 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f10f0013810 con 0x7f10f8155cd0 2023-10-08T00:22:07.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.465+0000 7f10dffff700 1 --2- 172.21.15.99:0/2304833230 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f10e4066f30 0x7f10e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:07.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.465+0000 7f10dffff700 1 -- 172.21.15.99:0/2304833230 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5734+0+0 (secure 0 0 0) 0x7f10f0084a60 con 0x7f10f8155cd0 2023-10-08T00:22:07.471 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.465+0000 7f10f7fff700 1 --2- 172.21.15.99:0/2304833230 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f10e4066f30 0x7f10e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:07.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.466+0000 7f10f7fff700 1 --2- 172.21.15.99:0/2304833230 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f10e4066f30 0x7f10e40693e0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f10e8000c00 tx=0x7f10e8007d10 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:07.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.470+0000 7f10dffff700 1 -- 172.21.15.99:0/2304833230 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f10f0051d70 con 0x7f10f8155cd0 2023-10-08T00:22:07.694 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.693+0000 7f10fe92b700 1 -- 172.21.15.99:0/2304833230 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f10f814d6c0 con 0x7f10f8155cd0 2023-10-08T00:22:07.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.693+0000 7f10dffff700 1 -- 172.21.15.99:0/2304833230 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+4604 (secure 0 0 0) 0x7f10f0055390 con 0x7f10f8155cd0 2023-10-08T00:22:07.695 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:22:07.695 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":84,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/3862684847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3862684847},{"type":"v1","addr":"172.21.15.191:6836","nonce":3862684847}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":8106,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/359623206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":359623206},{"type":"v1","addr":"172.21.15.99:6834","nonce":359623206}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":8109,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3064101777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":3064101777},{"type":"v1","addr":"172.21.15.99:6835","nonce":3064101777}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:21:29.659394+0000","modified":"2023-10-08T00:21:33.311802+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8124},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8124":{"gid":8124,"name":"d","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/2466421758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":2466421758},{"type":"v1","addr":"172.21.15.191:6837","nonce":2466421758}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-08T00:22:07.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.695+0000 7f10fe92b700 1 -- 172.21.15.99:0/2304833230 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f10e4066f30 msgr2=0x7f10e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:07.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.695+0000 7f10fe92b700 1 --2- 172.21.15.99:0/2304833230 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f10e4066f30 0x7f10e40693e0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f10e8000c00 tx=0x7f10e8007d10 comp rx=0 tx=0).stop 2023-10-08T00:22:07.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.695+0000 7f10fe92b700 1 -- 172.21.15.99:0/2304833230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10f8155cd0 msgr2=0x7f10f8156140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:07.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.695+0000 7f10fe92b700 1 --2- 172.21.15.99:0/2304833230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10f8155cd0 0x7f10f8156140 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f10f0002250 tx=0x7f10f00013f0 comp rx=0 tx=0).stop 2023-10-08T00:22:07.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.695+0000 7f10fe92b700 1 -- 172.21.15.99:0/2304833230 shutdown_connections 2023-10-08T00:22:07.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.695+0000 7f10fe92b700 1 --2- 172.21.15.99:0/2304833230 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f10e4066f30 0x7f10e40693e0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:07.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.695+0000 7f10fe92b700 1 --2- 172.21.15.99:0/2304833230 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f10e0003fd0 0x7f10f80b0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:07.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.695+0000 7f10fe92b700 1 --2- 172.21.15.99:0/2304833230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10f8155cd0 0x7f10f8156140 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:07.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.695+0000 7f10fe92b700 1 --2- 172.21.15.99:0/2304833230 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f10f8160330 0x7f10f814b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:07.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.695+0000 7f10fe92b700 1 -- 172.21.15.99:0/2304833230 >> 172.21.15.99:0/2304833230 conn(0x7f10f8068ce0 msgr2=0x7f10f80d0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:07.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.695+0000 7f10fe92b700 1 -- 172.21.15.99:0/2304833230 shutdown_connections 2023-10-08T00:22:07.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.695+0000 7f10fe92b700 1 -- 172.21.15.99:0/2304833230 wait complete. 2023-10-08T00:22:07.698 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 84 2023-10-08T00:22:07.709 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:22:07.822 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.821+0000 7fb17741a700 1 Processor -- start 2023-10-08T00:22:07.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.821+0000 7fb17741a700 1 -- start start 2023-10-08T00:22:07.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.822+0000 7fb17741a700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb1700cd420 0x7fb1700cb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:07.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.822+0000 7fb17741a700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb170156d50 0x7fb1700cbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:07.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.822+0000 7fb17741a700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1700ad5c0 con 0x7fb170156980 2023-10-08T00:22:07.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.822+0000 7fb17741a700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1700ac9b0 con 0x7fb1700cd420 2023-10-08T00:22:07.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.822+0000 7fb17741a700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1700acc90 con 0x7fb170156d50 2023-10-08T00:22:07.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.822+0000 7fb175c17700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb1700cd420 0x7fb1700cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:07.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.822+0000 7fb175c17700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb1700cd420 0x7fb1700cb800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33014/0 (socket says 172.21.15.99:33014) 2023-10-08T00:22:07.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.822+0000 7fb175c17700 1 -- 172.21.15.99:0/2248945591 learned_addr learned my addr 172.21.15.99:0/2248945591 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:07.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.822+0000 7fb176c19700 1 --2- 172.21.15.99:0/2248945591 >> v2:172.21.15.191:3300/0 conn(0x7fb170156d50 0x7fb1700cbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:07.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.822+0000 7fb175c17700 1 -- 172.21.15.99:0/2248945591 >> v2:172.21.15.191:3300/0 conn(0x7fb170156d50 msgr2=0x7fb1700cbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:07.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.822+0000 7fb175c17700 1 --2- 172.21.15.99:0/2248945591 >> v2:172.21.15.191:3300/0 conn(0x7fb170156d50 0x7fb1700cbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:07.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.822+0000 7fb175c17700 1 -- 172.21.15.99:0/2248945591 >> v1:172.21.15.191:6789/0 conn(0x7fb170156980 legacy=0x7fb1700ac0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:07.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.822+0000 7fb175c17700 1 -- 172.21.15.99:0/2248945591 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1700acf70 con 0x7fb1700cd420 2023-10-08T00:22:07.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.823+0000 7fb175c17700 1 --2- 172.21.15.99:0/2248945591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1700cd420 0x7fb1700cb800 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fb1700cc7a0 tx=0x7fb164009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1960a8e7cc5fe6bf server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:07.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.823+0000 7fb175416700 1 -- 172.21.15.99:0/2248945591 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16400b820 con 0x7fb1700cd420 2023-10-08T00:22:07.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.823+0000 7fb175416700 1 -- 172.21.15.99:0/2248945591 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb16400b980 con 0x7fb1700cd420 2023-10-08T00:22:07.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.823+0000 7fb175416700 1 -- 172.21.15.99:0/2248945591 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16400bd20 con 0x7fb1700cd420 2023-10-08T00:22:07.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.823+0000 7fb17741a700 1 -- 172.21.15.99:0/2248945591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1700cd420 msgr2=0x7fb1700cb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:07.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.823+0000 7fb17741a700 1 --2- 172.21.15.99:0/2248945591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1700cd420 0x7fb1700cb800 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fb1700cc7a0 tx=0x7fb164009250 comp rx=0 tx=0).stop 2023-10-08T00:22:07.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.823+0000 7fb17741a700 1 -- 172.21.15.99:0/2248945591 shutdown_connections 2023-10-08T00:22:07.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.823+0000 7fb17741a700 1 --2- 172.21.15.99:0/2248945591 >> v2:172.21.15.191:3300/0 conn(0x7fb170156d50 0x7fb1700cbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:07.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.823+0000 7fb17741a700 1 --2- 172.21.15.99:0/2248945591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1700cd420 0x7fb1700cb800 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:07.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.823+0000 7fb17741a700 1 -- 172.21.15.99:0/2248945591 >> 172.21.15.99:0/2248945591 conn(0x7fb1700680e0 msgr2=0x7fb1700cc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:07.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.823+0000 7fb17741a700 1 -- 172.21.15.99:0/2248945591 shutdown_connections 2023-10-08T00:22:07.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.823+0000 7fb17741a700 1 -- 172.21.15.99:0/2248945591 wait complete. 2023-10-08T00:22:07.830 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.824+0000 7fb17741a700 1 Processor -- start 2023-10-08T00:22:07.830 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.824+0000 7fb17741a700 1 -- start start 2023-10-08T00:22:07.830 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.825+0000 7fb17741a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1700cd420 0x7fb170152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:07.830 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.825+0000 7fb17741a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb170156980 0x7fb170147b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:07.830 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.825+0000 7fb17741a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb170156d50 0x7fb1701432f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:07.830 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.825+0000 7fb17741a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1701496b0 con 0x7fb170156d50 2023-10-08T00:22:07.830 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.825+0000 7fb17741a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb170148610 con 0x7fb1700cd420 2023-10-08T00:22:07.830 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.825+0000 7fb17741a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1701488f0 con 0x7fb170156980 2023-10-08T00:22:07.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.825+0000 7fb176c19700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb170156d50 0x7fb1701432f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:07.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.825+0000 7fb176c19700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb170156d50 0x7fb1701432f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33026/0 (socket says 172.21.15.99:33026) 2023-10-08T00:22:07.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.825+0000 7fb176c19700 1 -- 172.21.15.99:0/1603974026 learned_addr learned my addr 172.21.15.99:0/1603974026 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:07.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.825+0000 7fb176418700 1 --2- 172.21.15.99:0/1603974026 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1700cd420 0x7fb170152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:07.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.826+0000 7fb175c17700 1 --2- 172.21.15.99:0/1603974026 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb170156980 0x7fb170147b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:07.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.826+0000 7fb176c19700 1 -- 172.21.15.99:0/1603974026 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb170156980 msgr2=0x7fb170147b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:07.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.826+0000 7fb176c19700 1 --2- 172.21.15.99:0/1603974026 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb170156980 0x7fb170147b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:07.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.826+0000 7fb176c19700 1 -- 172.21.15.99:0/1603974026 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1700cd420 msgr2=0x7fb170152620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:07.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.826+0000 7fb176c19700 1 --2- 172.21.15.99:0/1603974026 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1700cd420 0x7fb170152620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:07.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.826+0000 7fb176c19700 1 -- 172.21.15.99:0/1603974026 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1640094f0 con 0x7fb170156d50 2023-10-08T00:22:07.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.826+0000 7fb175c17700 1 --2- 172.21.15.99:0/1603974026 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb170156980 0x7fb170147b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:22:07.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.826+0000 7fb176c19700 1 --2- 172.21.15.99:0/1603974026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb170156d50 0x7fb1701432f0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fb16000d860 tx=0x7fb160008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:07.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.826+0000 7fb16e7fc700 1 -- 172.21.15.99:0/1603974026 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1600125f0 con 0x7fb170156d50 2023-10-08T00:22:07.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.826+0000 7fb16e7fc700 1 -- 172.21.15.99:0/1603974026 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb160012750 con 0x7fb170156d50 2023-10-08T00:22:07.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.826+0000 7fb16e7fc700 1 -- 172.21.15.99:0/1603974026 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb160012b10 con 0x7fb170156d50 2023-10-08T00:22:07.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.826+0000 7fb17741a700 1 -- 172.21.15.99:0/1603974026 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb170148bd0 con 0x7fb170156d50 2023-10-08T00:22:07.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.826+0000 7fb17741a700 1 -- 172.21.15.99:0/1603974026 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb170149190 con 0x7fb170156d50 2023-10-08T00:22:07.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.827+0000 7fb17741a700 1 -- 172.21.15.99:0/1603974026 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb17006bc70 con 0x7fb170156d50 2023-10-08T00:22:07.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.827+0000 7fb16e7fc700 1 -- 172.21.15.99:0/1603974026 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb16001e460 con 0x7fb170156d50 2023-10-08T00:22:07.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.828+0000 7fb16e7fc700 1 --2- 172.21.15.99:0/1603974026 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb15c066f30 0x7fb15c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:07.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.828+0000 7fb16e7fc700 1 -- 172.21.15.99:0/1603974026 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5734+0+0 (secure 0 0 0) 0x7fb160084b90 con 0x7fb170156d50 2023-10-08T00:22:07.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.828+0000 7fb176418700 1 --2- 172.21.15.99:0/1603974026 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb15c066f30 0x7fb15c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:07.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.828+0000 7fb176418700 1 --2- 172.21.15.99:0/1603974026 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb15c066f30 0x7fb15c0693e0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fb168007fd0 tx=0x7fb168009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:07.834 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:07.833+0000 7fb16e7fc700 1 -- 172.21.15.99:0/1603974026 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb160051de0 con 0x7fb170156d50 2023-10-08T00:22:08.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.056+0000 7fb17741a700 1 -- 172.21.15.99:0/1603974026 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb17004bb80 con 0x7fb170156d50 2023-10-08T00:22:08.058 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.056+0000 7fb16e7fc700 1 -- 172.21.15.99:0/1603974026 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+4604 (secure 0 0 0) 0x7fb160055400 con 0x7fb170156d50 2023-10-08T00:22:08.058 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:22:08.058 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":84,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/3862684847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3862684847},{"type":"v1","addr":"172.21.15.191:6836","nonce":3862684847}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":8106,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/359623206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":359623206},{"type":"v1","addr":"172.21.15.99:6834","nonce":359623206}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":8109,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3064101777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":3064101777},{"type":"v1","addr":"172.21.15.99:6835","nonce":3064101777}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:21:29.659394+0000","modified":"2023-10-08T00:21:33.311802+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8124},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8124":{"gid":8124,"name":"d","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/2466421758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":2466421758},{"type":"v1","addr":"172.21.15.191:6837","nonce":2466421758}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-08T00:22:08.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.058+0000 7fb17741a700 1 -- 172.21.15.99:0/1603974026 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb15c066f30 msgr2=0x7fb15c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:08.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.058+0000 7fb17741a700 1 --2- 172.21.15.99:0/1603974026 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb15c066f30 0x7fb15c0693e0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fb168007fd0 tx=0x7fb168009040 comp rx=0 tx=0).stop 2023-10-08T00:22:08.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.058+0000 7fb17741a700 1 -- 172.21.15.99:0/1603974026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb170156d50 msgr2=0x7fb1701432f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:08.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.058+0000 7fb17741a700 1 --2- 172.21.15.99:0/1603974026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb170156d50 0x7fb1701432f0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fb16000d860 tx=0x7fb160008040 comp rx=0 tx=0).stop 2023-10-08T00:22:08.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.058+0000 7fb17741a700 1 -- 172.21.15.99:0/1603974026 shutdown_connections 2023-10-08T00:22:08.064 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.058+0000 7fb17741a700 1 --2- 172.21.15.99:0/1603974026 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fb15c066f30 0x7fb15c0693e0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:08.064 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.058+0000 7fb17741a700 1 --2- 172.21.15.99:0/1603974026 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1700cd420 0x7fb170152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:08.064 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.058+0000 7fb17741a700 1 --2- 172.21.15.99:0/1603974026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb170156d50 0x7fb1701432f0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:08.064 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.058+0000 7fb17741a700 1 --2- 172.21.15.99:0/1603974026 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb170156980 0x7fb170147b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:08.064 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.058+0000 7fb17741a700 1 -- 172.21.15.99:0/1603974026 >> 172.21.15.99:0/1603974026 conn(0x7fb1700680e0 msgr2=0x7fb170068cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:08.064 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.060+0000 7fb17741a700 1 -- 172.21.15.99:0/1603974026 shutdown_connections 2023-10-08T00:22:08.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.060+0000 7fb17741a700 1 -- 172.21.15.99:0/1603974026 wait complete. 2023-10-08T00:22:08.065 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 84 2023-10-08T00:22:08.072 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:22:08.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.186+0000 7faffcb3e700 1 Processor -- start 2023-10-08T00:22:08.187 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.186+0000 7faffcb3e700 1 -- start start 2023-10-08T00:22:08.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.187+0000 7faffcb3e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7faff8157870 0x7faff815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:08.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.187+0000 7faffcb3e700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7faff8157c40 0x7faff8158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:08.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.187+0000 7faffcb3e700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faff815cbb0 con 0x7faff815a620 2023-10-08T00:22:08.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.187+0000 7faffcb3e700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faff815bfa0 con 0x7faff8157c40 2023-10-08T00:22:08.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.187+0000 7faffcb3e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faff815c280 con 0x7faff8157870 2023-10-08T00:22:08.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.187+0000 7faff6d9d700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7faff8157c40 0x7faff8158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:08.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.187+0000 7fafedd9b700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7faff815a620 0x7faff815d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56814/0 (socket says 172.21.15.99:56814) 2023-10-08T00:22:08.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.187+0000 7fafedd9b700 1 -- 172.21.15.99:0/1825823904 learned_addr learned my addr 172.21.15.99:0/1825823904 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:22:08.189 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.187+0000 7faff6d9d700 1 --2- 172.21.15.99:0/1825823904 >> v2:172.21.15.99:3300/0 conn(0x7faff8157c40 0x7faff8158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33032/0 (socket says 172.21.15.99:33032) 2023-10-08T00:22:08.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.187+0000 7faff659c700 1 --2- 172.21.15.99:0/1825823904 >> v2:172.21.15.191:3300/0 conn(0x7faff8157870 0x7faff815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:08.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.187+0000 7faff6d9d700 1 -- 172.21.15.99:0/1825823904 >> v2:172.21.15.191:3300/0 conn(0x7faff8157870 msgr2=0x7faff815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:08.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.187+0000 7faff6d9d700 1 --2- 172.21.15.99:0/1825823904 >> v2:172.21.15.191:3300/0 conn(0x7faff8157870 0x7faff815b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:08.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.187+0000 7faff6d9d700 1 -- 172.21.15.99:0/1825823904 >> v1:172.21.15.99:6789/0 conn(0x7faff815a620 legacy=0x7faff815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:08.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.187+0000 7faff6d9d700 1 -- 172.21.15.99:0/1825823904 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff815c560 con 0x7faff8157c40 2023-10-08T00:22:08.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.187+0000 7faff5d9b700 1 -- 172.21.15.99:0/1825823904 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2510143773 0 0) 0x7faff815cbb0 con 0x7faff815a620 2023-10-08T00:22:08.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.187+0000 7faff6d9d700 1 --2- 172.21.15.99:0/1825823904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faff8157c40 0x7faff8158110 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7faff8069d80 tx=0x7faff0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52420d0f13c7ce4f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:08.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.187+0000 7faff5d9b700 1 -- 172.21.15.99:0/1825823904 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff000e070 con 0x7faff8157c40 2023-10-08T00:22:08.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.188+0000 7faff5d9b700 1 -- 172.21.15.99:0/1825823904 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faff000bc30 con 0x7faff8157c40 2023-10-08T00:22:08.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.188+0000 7faff5d9b700 1 -- 172.21.15.99:0/1825823904 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff0012430 con 0x7faff8157c40 2023-10-08T00:22:08.190 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.188+0000 7faffcb3e700 1 -- 172.21.15.99:0/1825823904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faff8157c40 msgr2=0x7faff8158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:08.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.188+0000 7faffcb3e700 1 --2- 172.21.15.99:0/1825823904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faff8157c40 0x7faff8158110 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7faff8069d80 tx=0x7faff0009780 comp rx=0 tx=0).stop 2023-10-08T00:22:08.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.188+0000 7faffcb3e700 1 -- 172.21.15.99:0/1825823904 shutdown_connections 2023-10-08T00:22:08.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.188+0000 7faffcb3e700 1 --2- 172.21.15.99:0/1825823904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faff8157c40 0x7faff8158110 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:08.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.188+0000 7faffcb3e700 1 --2- 172.21.15.99:0/1825823904 >> v2:172.21.15.191:3300/0 conn(0x7faff8157870 0x7faff815b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:08.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.188+0000 7faffcb3e700 1 -- 172.21.15.99:0/1825823904 >> 172.21.15.99:0/1825823904 conn(0x7faff8068ce0 msgr2=0x7faff80690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:08.191 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.188+0000 7faffcb3e700 1 -- 172.21.15.99:0/1825823904 shutdown_connections 2023-10-08T00:22:08.192 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.188+0000 7faffcb3e700 1 -- 172.21.15.99:0/1825823904 wait complete. 2023-10-08T00:22:08.195 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.189+0000 7faffcb3e700 1 Processor -- start 2023-10-08T00:22:08.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.189+0000 7faffcb3e700 1 -- start start 2023-10-08T00:22:08.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.190+0000 7faffcb3e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faff8157870 0x7faff814c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:08.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.190+0000 7faffcb3e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faff8157c40 0x7faff81522d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:08.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.190+0000 7faffcb3e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faff815a620 0x7faff8153690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:08.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.190+0000 7faffcb3e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff80b19c0 con 0x7faff8157c40 2023-10-08T00:22:08.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.190+0000 7faffcb3e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff8151e40 con 0x7faff815a620 2023-10-08T00:22:08.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.190+0000 7faffcb3e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faff8151230 con 0x7faff8157870 2023-10-08T00:22:08.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.190+0000 7fafedd9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faff8157c40 0x7faff81522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:08.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.190+0000 7fafedd9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faff8157c40 0x7faff81522d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33048/0 (socket says 172.21.15.99:33048) 2023-10-08T00:22:08.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.190+0000 7fafedd9b700 1 -- 172.21.15.99:0/738032370 learned_addr learned my addr 172.21.15.99:0/738032370 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:08.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.190+0000 7faff6d9d700 1 --2- 172.21.15.99:0/738032370 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faff815a620 0x7faff8153690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:08.196 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.190+0000 7faff659c700 1 --2- 172.21.15.99:0/738032370 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faff8157870 0x7faff814c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:08.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.190+0000 7fafedd9b700 1 -- 172.21.15.99:0/738032370 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faff8157870 msgr2=0x7faff814c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:08.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.190+0000 7fafedd9b700 1 --2- 172.21.15.99:0/738032370 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faff8157870 0x7faff814c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:08.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.190+0000 7fafedd9b700 1 -- 172.21.15.99:0/738032370 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faff815a620 msgr2=0x7faff8153690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:08.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.190+0000 7fafedd9b700 1 --2- 172.21.15.99:0/738032370 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faff815a620 0x7faff8153690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:08.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.190+0000 7fafedd9b700 1 -- 172.21.15.99:0/738032370 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff0009a20 con 0x7faff8157c40 2023-10-08T00:22:08.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.191+0000 7fafedd9b700 1 --2- 172.21.15.99:0/738032370 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faff8157c40 0x7faff81522d0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fafe000c5e0 tx=0x7fafe0000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:08.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.191+0000 7fafeeffd700 1 -- 172.21.15.99:0/738032370 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe0005b30 con 0x7faff8157c40 2023-10-08T00:22:08.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.191+0000 7fafeeffd700 1 -- 172.21.15.99:0/738032370 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafe0005c90 con 0x7faff8157c40 2023-10-08T00:22:08.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.191+0000 7faffcb3e700 1 -- 172.21.15.99:0/738032370 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faff8151510 con 0x7faff8157c40 2023-10-08T00:22:08.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.191+0000 7fafeeffd700 1 -- 172.21.15.99:0/738032370 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe0007300 con 0x7faff8157c40 2023-10-08T00:22:08.197 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.191+0000 7faffcb3e700 1 -- 172.21.15.99:0/738032370 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faff8151ad0 con 0x7faff8157c40 2023-10-08T00:22:08.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.192+0000 7faffcb3e700 1 -- 172.21.15.99:0/738032370 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faff8151fb0 con 0x7faff8157c40 2023-10-08T00:22:08.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.192+0000 7fafeeffd700 1 -- 172.21.15.99:0/738032370 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fafe0007670 con 0x7faff8157c40 2023-10-08T00:22:08.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.192+0000 7fafeeffd700 1 --2- 172.21.15.99:0/738032370 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fafd8066f30 0x7fafd80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:08.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.192+0000 7fafeeffd700 1 -- 172.21.15.99:0/738032370 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5734+0+0 (secure 0 0 0) 0x7fafe0083aa0 con 0x7faff8157c40 2023-10-08T00:22:08.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.193+0000 7faff659c700 1 --2- 172.21.15.99:0/738032370 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fafd8066f30 0x7fafd80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:08.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.193+0000 7faff659c700 1 --2- 172.21.15.99:0/738032370 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fafd8066f30 0x7fafd80693e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fafe80089b0 tx=0x7fafe8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:08.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.196+0000 7fafeeffd700 1 -- 172.21.15.99:0/738032370 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fafe0050cf0 con 0x7faff8157c40 2023-10-08T00:22:08.380 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.380+0000 7faffcb3e700 1 -- 172.21.15.99:0/738032370 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faff804c3c0 con 0x7faff8157c40 2023-10-08T00:22:08.381 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.380+0000 7fafeeffd700 1 -- 172.21.15.99:0/738032370 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v189) v1 ==== 74+0+15422 (secure 0 0 0) 0x7fafe0017020 con 0x7faff8157c40 2023-10-08T00:22:08.381 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:22:08.381 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":189,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:22:01.171728+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:21:27.984574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-08T00:21:29.244620+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"189","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"23.18","mappings":[{"from":5,"to":3}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:22:08.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.382+0000 7faffcb3e700 1 -- 172.21.15.99:0/738032370 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fafd8066f30 msgr2=0x7fafd80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:08.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.382+0000 7faffcb3e700 1 --2- 172.21.15.99:0/738032370 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fafd8066f30 0x7fafd80693e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fafe80089b0 tx=0x7fafe8008040 comp rx=0 tx=0).stop 2023-10-08T00:22:08.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.382+0000 7faffcb3e700 1 -- 172.21.15.99:0/738032370 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faff8157c40 msgr2=0x7faff81522d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:08.383 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.382+0000 7faffcb3e700 1 --2- 172.21.15.99:0/738032370 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faff8157c40 0x7faff81522d0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fafe000c5e0 tx=0x7fafe0000fe0 comp rx=0 tx=0).stop 2023-10-08T00:22:08.384 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.382+0000 7faffcb3e700 1 -- 172.21.15.99:0/738032370 shutdown_connections 2023-10-08T00:22:08.384 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.382+0000 7faffcb3e700 1 --2- 172.21.15.99:0/738032370 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fafd8066f30 0x7fafd80693e0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:08.384 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.382+0000 7faffcb3e700 1 --2- 172.21.15.99:0/738032370 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faff815a620 0x7faff8153690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:08.384 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.382+0000 7faffcb3e700 1 --2- 172.21.15.99:0/738032370 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faff8157870 0x7faff814c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:08.384 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.382+0000 7faffcb3e700 1 --2- 172.21.15.99:0/738032370 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faff8157c40 0x7faff81522d0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:08.384 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.382+0000 7faffcb3e700 1 -- 172.21.15.99:0/738032370 >> 172.21.15.99:0/738032370 conn(0x7faff8068ce0 msgr2=0x7faff80b5000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:08.384 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.382+0000 7faffcb3e700 1 -- 172.21.15.99:0/738032370 shutdown_connections 2023-10-08T00:22:08.384 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.382+0000 7faffcb3e700 1 -- 172.21.15.99:0/738032370 wait complete. 2023-10-08T00:22:08.395 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T00:22:08.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.509+0000 7f20f5086700 1 Processor -- start 2023-10-08T00:22:08.510 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.509+0000 7f20f5086700 1 -- start start 2023-10-08T00:22:08.513 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.510+0000 7f20f5086700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f20f0157c40 0x7f20f0158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:08.513 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.510+0000 7f20f5086700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20f015cbb0 con 0x7f20f0157870 2023-10-08T00:22:08.513 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.510+0000 7f20f5086700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20f015bfa0 con 0x7f20f015a620 2023-10-08T00:22:08.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.510+0000 7f20f5086700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20f015bdb0 con 0x7f20f0157c40 2023-10-08T00:22:08.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.510+0000 7f20eed9d700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f20f0157870 0x7f20f015b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56816/0 (socket says 172.21.15.99:56816) 2023-10-08T00:22:08.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.510+0000 7f20eed9d700 1 -- 172.21.15.99:0/1520237031 learned_addr learned my addr 172.21.15.99:0/1520237031 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:22:08.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.510+0000 7f20ee59c700 1 -- 172.21.15.99:0/1520237031 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 628981506 0 0) 0x7f20f015cbb0 con 0x7f20f0157870 2023-10-08T00:22:08.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.510+0000 7f20ef59e700 1 --2- 172.21.15.99:0/1520237031 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f20f0157c40 0x7f20f0158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:08.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.510+0000 7f20ee59c700 1 -- 172.21.15.99:0/1520237031 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20dc003570 con 0x7f20f0157870 2023-10-08T00:22:08.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.510+0000 7f20ee59c700 1 -- 172.21.15.99:0/1520237031 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 927380192 0 0) 0x7f20dc003570 con 0x7f20f0157870 2023-10-08T00:22:08.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.511+0000 7f20ee59c700 1 -- 172.21.15.99:0/1520237031 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f20f0157c40 msgr2=0x7f20f0158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:08.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.511+0000 7f20ee59c700 1 --2- 172.21.15.99:0/1520237031 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f20f0157c40 0x7f20f0158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:08.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.511+0000 7f20ee59c700 1 -- 172.21.15.99:0/1520237031 >> v1:172.21.15.191:6789/0 conn(0x7f20f015a620 legacy=0x7f20f015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:08.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.511+0000 7f20ee59c700 1 -- 172.21.15.99:0/1520237031 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f015c280 con 0x7f20f0157870 2023-10-08T00:22:08.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.511+0000 7f20ee59c700 1 -- 172.21.15.99:0/1520237031 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7f20e0001470 con 0x7f20f0157870 2023-10-08T00:22:08.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.511+0000 7f20ee59c700 1 -- 172.21.15.99:0/1520237031 >> v1:172.21.15.99:6789/0 conn(0x7f20f0157870 legacy=0x7f20f015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:08.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.511+0000 7f20ee59c700 1 --2- 172.21.15.99:0/1520237031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20dc003a90 0x7f20dc003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:08.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.511+0000 7f20ee59c700 1 --2- 172.21.15.99:0/1520237031 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20dc004810 0x7f20dc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:08.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.511+0000 7f20ee59c700 1 --2- 172.21.15.99:0/1520237031 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f20f0157c40 0x7f20dc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:08.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.511+0000 7f20ee59c700 1 -- 172.21.15.99:0/1520237031 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20dc00bb10 con 0x7f20dc003a90 2023-10-08T00:22:08.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.511+0000 7f20ee59c700 1 -- 172.21.15.99:0/1520237031 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20dc00bc50 con 0x7f20dc004810 2023-10-08T00:22:08.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.511+0000 7f20ee59c700 1 -- 172.21.15.99:0/1520237031 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20dc00be10 con 0x7f20f0157c40 2023-10-08T00:22:08.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.511+0000 7f20eed9d700 1 --2- 172.21.15.99:0/1520237031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20dc003a90 0x7f20dc003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:08.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.511+0000 7f20e659c700 1 --2- 172.21.15.99:0/1520237031 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20dc004810 0x7f20dc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:08.515 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.511+0000 7f20ef59e700 1 --2- 172.21.15.99:0/1520237031 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f20f0157c40 0x7f20dc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:08.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.511+0000 7f20eed9d700 1 -- 172.21.15.99:0/1520237031 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f20f0157c40 msgr2=0x7f20dc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:08.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.511+0000 7f20eed9d700 1 --2- 172.21.15.99:0/1520237031 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f20f0157c40 0x7f20dc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:08.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.511+0000 7f20eed9d700 1 -- 172.21.15.99:0/1520237031 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20dc004810 msgr2=0x7f20dc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:08.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.511+0000 7f20eed9d700 1 --2- 172.21.15.99:0/1520237031 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20dc004810 0x7f20dc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:08.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.511+0000 7f20eed9d700 1 -- 172.21.15.99:0/1520237031 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20dc003570 con 0x7f20dc003a90 2023-10-08T00:22:08.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.511+0000 7f20ef59e700 1 --2- 172.21.15.99:0/1520237031 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f20f0157c40 0x7f20dc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:22:08.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.511+0000 7f20eed9d700 1 --2- 172.21.15.99:0/1520237031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20dc003a90 0x7f20dc003e60 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f20e000c3c0 tx=0x7f20e0001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c01d51480914ac5d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:08.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.511+0000 7f20ee59c700 1 -- 172.21.15.99:0/1520237031 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e000cc90 con 0x7f20dc003a90 2023-10-08T00:22:08.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.511+0000 7f20ee59c700 1 -- 172.21.15.99:0/1520237031 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e000ce10 con 0x7f20dc003a90 2023-10-08T00:22:08.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.511+0000 7f20ee59c700 1 -- 172.21.15.99:0/1520237031 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e0005b70 con 0x7f20dc003a90 2023-10-08T00:22:08.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.512+0000 7f20f5086700 1 -- 172.21.15.99:0/1520237031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20dc003a90 msgr2=0x7f20dc003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:08.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.512+0000 7f20f5086700 1 --2- 172.21.15.99:0/1520237031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20dc003a90 0x7f20dc003e60 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f20e000c3c0 tx=0x7f20e0001e70 comp rx=0 tx=0).stop 2023-10-08T00:22:08.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.512+0000 7f20ef59e700 1 -- 172.21.15.99:0/1520237031 reap_dead start 2023-10-08T00:22:08.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.512+0000 7f20f5086700 1 -- 172.21.15.99:0/1520237031 shutdown_connections 2023-10-08T00:22:08.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.512+0000 7f20f5086700 1 -- 172.21.15.99:0/1520237031 >> 172.21.15.99:0/1520237031 conn(0x7f20f0068ce0 msgr2=0x7f20f00690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:08.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.512+0000 7f20f5086700 1 -- 172.21.15.99:0/1520237031 shutdown_connections 2023-10-08T00:22:08.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.512+0000 7f20f5086700 1 -- 172.21.15.99:0/1520237031 wait complete. 2023-10-08T00:22:08.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.513+0000 7f20f5086700 1 Processor -- start 2023-10-08T00:22:08.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.513+0000 7f20f5086700 1 -- start start 2023-10-08T00:22:08.519 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.514+0000 7f20f5086700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20dc003a90 0x7f20f0169ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:08.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.514+0000 7f20f5086700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f20f0150ed0 0x7f20f0151340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:08.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.514+0000 7f20f5086700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20f00b16d0 0x7f20f00b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:08.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.514+0000 7f20f5086700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f0151880 con 0x7f20f00b16d0 2023-10-08T00:22:08.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.514+0000 7f20f5086700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f00b3010 con 0x7f20dc003a90 2023-10-08T00:22:08.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.514+0000 7f20f5086700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20f00b2400 con 0x7f20f0150ed0 2023-10-08T00:22:08.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.514+0000 7f20ef59e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20f00b16d0 0x7f20f00b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:08.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.514+0000 7f20ef59e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20f00b16d0 0x7f20f00b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33066/0 (socket says 172.21.15.99:33066) 2023-10-08T00:22:08.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.514+0000 7f20ef59e700 1 -- 172.21.15.99:0/2288192134 learned_addr learned my addr 172.21.15.99:0/2288192134 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:08.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.514+0000 7f20e659c700 1 --2- 172.21.15.99:0/2288192134 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f20f0150ed0 0x7f20f0151340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:08.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.514+0000 7f20eed9d700 1 --2- 172.21.15.99:0/2288192134 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20dc003a90 0x7f20f0169ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:08.520 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.514+0000 7f20ef59e700 1 -- 172.21.15.99:0/2288192134 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f20f0150ed0 msgr2=0x7f20f0151340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:08.521 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.514+0000 7f20ef59e700 1 --2- 172.21.15.99:0/2288192134 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f20f0150ed0 0x7f20f0151340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:08.521 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.514+0000 7f20ef59e700 1 -- 172.21.15.99:0/2288192134 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20dc003a90 msgr2=0x7f20f0169ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:08.521 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.514+0000 7f20ef59e700 1 --2- 172.21.15.99:0/2288192134 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20dc003a90 0x7f20f0169ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:08.521 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.514+0000 7f20ef59e700 1 -- 172.21.15.99:0/2288192134 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f00b26e0 con 0x7f20f00b16d0 2023-10-08T00:22:08.521 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.514+0000 7f20e659c700 1 --2- 172.21.15.99:0/2288192134 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f20f0150ed0 0x7f20f0151340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:22:08.521 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.514+0000 7f20ef59e700 1 --2- 172.21.15.99:0/2288192134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20f00b16d0 0x7f20f00b1b40 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f20f00b3830 tx=0x7f20e800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:08.521 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.514+0000 7f20e77fe700 1 -- 172.21.15.99:0/2288192134 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e801b070 con 0x7f20f00b16d0 2023-10-08T00:22:08.521 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.514+0000 7f20e77fe700 1 -- 172.21.15.99:0/2288192134 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e800f040 con 0x7f20f00b16d0 2023-10-08T00:22:08.521 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.514+0000 7f20f5086700 1 -- 172.21.15.99:0/2288192134 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20f00b29c0 con 0x7f20f00b16d0 2023-10-08T00:22:08.521 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.514+0000 7f20e77fe700 1 -- 172.21.15.99:0/2288192134 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e800e050 con 0x7f20f00b16d0 2023-10-08T00:22:08.521 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.514+0000 7f20f5086700 1 -- 172.21.15.99:0/2288192134 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20f00b2080 con 0x7f20f00b16d0 2023-10-08T00:22:08.522 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.515+0000 7f20f5086700 1 -- 172.21.15.99:0/2288192134 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20f004c3c0 con 0x7f20f00b16d0 2023-10-08T00:22:08.522 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.516+0000 7f20e77fe700 1 -- 172.21.15.99:0/2288192134 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f20e8003020 con 0x7f20f00b16d0 2023-10-08T00:22:08.522 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.516+0000 7f20e77fe700 1 --2- 172.21.15.99:0/2288192134 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f20d0066f30 0x7f20d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:08.522 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.516+0000 7f20e77fe700 1 -- 172.21.15.99:0/2288192134 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5734+0+0 (secure 0 0 0) 0x7f20e8084a40 con 0x7f20f00b16d0 2023-10-08T00:22:08.522 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.516+0000 7f20eed9d700 1 --2- 172.21.15.99:0/2288192134 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f20d0066f30 0x7f20d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:08.522 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.517+0000 7f20eed9d700 1 --2- 172.21.15.99:0/2288192134 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f20d0066f30 0x7f20d00693e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f20e0001ed0 tx=0x7f20e0015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:08.522 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.520+0000 7f20e77fe700 1 -- 172.21.15.99:0/2288192134 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f20e8086e60 con 0x7f20f00b16d0 2023-10-08T00:22:08.744 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.743+0000 7f20f5086700 1 -- 172.21.15.99:0/2288192134 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f20f015c560 con 0x7f20f00b16d0 2023-10-08T00:22:08.744 INFO:tasks.ceph.mon.a.smithi099.stderr:2023-10-08T00:22:08.744+0000 7f6c60fa2700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T00:22:08.748 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.748+0000 7f20e77fe700 1 -- 172.21.15.99:0/2288192134 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v85) v1 ==== 161+0+0 (secure 0 0 0) 0x7f20e8051c90 con 0x7f20f00b16d0 2023-10-08T00:22:08.751 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.750+0000 7f20f5086700 1 -- 172.21.15.99:0/2288192134 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f20d0066f30 msgr2=0x7f20d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:08.751 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.750+0000 7f20f5086700 1 --2- 172.21.15.99:0/2288192134 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f20d0066f30 0x7f20d00693e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f20e0001ed0 tx=0x7f20e0015000 comp rx=0 tx=0).stop 2023-10-08T00:22:08.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.750+0000 7f20f5086700 1 -- 172.21.15.99:0/2288192134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20f00b16d0 msgr2=0x7f20f00b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:08.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.750+0000 7f20f5086700 1 --2- 172.21.15.99:0/2288192134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20f00b16d0 0x7f20f00b1b40 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f20f00b3830 tx=0x7f20e800b040 comp rx=0 tx=0).stop 2023-10-08T00:22:08.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.750+0000 7f20f5086700 1 -- 172.21.15.99:0/2288192134 shutdown_connections 2023-10-08T00:22:08.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.750+0000 7f20f5086700 1 --2- 172.21.15.99:0/2288192134 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f20d0066f30 0x7f20d00693e0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:08.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.750+0000 7f20f5086700 1 --2- 172.21.15.99:0/2288192134 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20dc003a90 0x7f20f0169ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:08.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.750+0000 7f20f5086700 1 --2- 172.21.15.99:0/2288192134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20f00b16d0 0x7f20f00b1b40 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:08.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.750+0000 7f20f5086700 1 --2- 172.21.15.99:0/2288192134 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f20f0150ed0 0x7f20f0151340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:08.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.750+0000 7f20f5086700 1 -- 172.21.15.99:0/2288192134 >> 172.21.15.99:0/2288192134 conn(0x7f20f0068ce0 msgr2=0x7f20f00d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:08.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.751+0000 7f20f5086700 1 -- 172.21.15.99:0/2288192134 shutdown_connections 2023-10-08T00:22:08.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.751+0000 7f20f5086700 1 -- 172.21.15.99:0/2288192134 wait complete. 2023-10-08T00:22:08.753 INFO:teuthology.orchestra.run.smithi099.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T00:22:08.754 INFO:tasks.ceph.mds.d.smithi191.stderr: -7700> 2023-10-08T00:21:35.632+0000 7fd59d2d7700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T00:22:08.763 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T00:22:08.784 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2023-10-08T00:22:08.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.878+0000 7fca933ed700 1 Processor -- start 2023-10-08T00:22:08.879 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.878+0000 7fca933ed700 1 -- start start 2023-10-08T00:22:08.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.879+0000 7fca933ed700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fca8c06aa00 0x7fca8c164c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:08.888 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.879+0000 7fca933ed700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca8c0b4d70 con 0x7fca8c0cce30 2023-10-08T00:22:08.888 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.879+0000 7fca933ed700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca8c15b3b0 con 0x7fca8c0b6580 2023-10-08T00:22:08.888 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.879+0000 7fca933ed700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca8c0cd8d0 con 0x7fca8c06aa00 2023-10-08T00:22:08.888 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.879+0000 7fca923eb700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fca8c0cce30 0x7fca8c15aca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56832/0 (socket says 172.21.15.99:56832) 2023-10-08T00:22:08.888 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.879+0000 7fca923eb700 1 -- 172.21.15.99:0/3920822869 learned_addr learned my addr 172.21.15.99:0/3920822869 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:22:08.888 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.879+0000 7fca92bec700 1 --2- 172.21.15.99:0/3920822869 >> v2:172.21.15.191:3300/0 conn(0x7fca8c06aa00 0x7fca8c164c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:08.888 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.879+0000 7fca913e9700 1 -- 172.21.15.99:0/3920822869 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1862358219 0 0) 0x7fca8c0b4d70 con 0x7fca8c0cce30 2023-10-08T00:22:08.888 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.879+0000 7fca913e9700 1 -- 172.21.15.99:0/3920822869 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca74003570 con 0x7fca8c0cce30 2023-10-08T00:22:08.888 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.880+0000 7fca913e9700 1 -- 172.21.15.99:0/3920822869 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 116554092 0 0) 0x7fca8c15b3b0 con 0x7fca8c0b6580 2023-10-08T00:22:08.888 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.880+0000 7fca913e9700 1 -- 172.21.15.99:0/3920822869 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca8c0b4d70 con 0x7fca8c0b6580 2023-10-08T00:22:08.888 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.880+0000 7fca913e9700 1 -- 172.21.15.99:0/3920822869 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2712751198 0 0) 0x7fca74003570 con 0x7fca8c0cce30 2023-10-08T00:22:08.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.880+0000 7fca92bec700 1 -- 172.21.15.99:0/3920822869 >> v1:172.21.15.191:6789/0 conn(0x7fca8c0b6580 legacy=0x7fca8c06a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:08.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.880+0000 7fca92bec700 1 -- 172.21.15.99:0/3920822869 >> v1:172.21.15.99:6789/0 conn(0x7fca8c0cce30 legacy=0x7fca8c15aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:08.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.880+0000 7fca92bec700 1 -- 172.21.15.99:0/3920822869 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca8c0b4440 con 0x7fca8c06aa00 2023-10-08T00:22:08.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.880+0000 7fca92bec700 1 --2- 172.21.15.99:0/3920822869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca8c06aa00 0x7fca8c164c20 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fca840098e0 tx=0x7fca840091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9dc02880ee4825a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:08.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.880+0000 7fca913e9700 1 -- 172.21.15.99:0/3920822869 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7c0017f0 con 0x7fca8c06aa00 2023-10-08T00:22:08.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.880+0000 7fca913e9700 1 -- 172.21.15.99:0/3920822869 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca84011420 con 0x7fca8c06aa00 2023-10-08T00:22:08.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.880+0000 7fca913e9700 1 -- 172.21.15.99:0/3920822869 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca840117c0 con 0x7fca8c06aa00 2023-10-08T00:22:08.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.881+0000 7fca933ed700 1 -- 172.21.15.99:0/3920822869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca8c06aa00 msgr2=0x7fca8c164c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:08.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.881+0000 7fca933ed700 1 --2- 172.21.15.99:0/3920822869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca8c06aa00 0x7fca8c164c20 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fca840098e0 tx=0x7fca840091a0 comp rx=0 tx=0).stop 2023-10-08T00:22:08.889 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.881+0000 7fca933ed700 1 -- 172.21.15.99:0/3920822869 shutdown_connections 2023-10-08T00:22:08.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.881+0000 7fca933ed700 1 --2- 172.21.15.99:0/3920822869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca8c06aa00 0x7fca8c164c20 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:08.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.881+0000 7fca933ed700 1 -- 172.21.15.99:0/3920822869 >> 172.21.15.99:0/3920822869 conn(0x7fca8c068cb0 msgr2=0x7fca8c0690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:08.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.881+0000 7fca933ed700 1 -- 172.21.15.99:0/3920822869 shutdown_connections 2023-10-08T00:22:08.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.881+0000 7fca933ed700 1 -- 172.21.15.99:0/3920822869 wait complete. 2023-10-08T00:22:08.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.882+0000 7fca933ed700 1 Processor -- start 2023-10-08T00:22:08.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.882+0000 7fca933ed700 1 -- start start 2023-10-08T00:22:08.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.883+0000 7fca933ed700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca8c06aa00 0x7fca8c0b2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:08.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.883+0000 7fca933ed700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca8c0b6580 0x7fca8c158b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:08.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.883+0000 7fca933ed700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca8c0cce30 0x7fca8c143560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:08.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.883+0000 7fca933ed700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c159330 con 0x7fca8c0cce30 2023-10-08T00:22:08.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.883+0000 7fca933ed700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c159610 con 0x7fca8c0b6580 2023-10-08T00:22:08.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.883+0000 7fca933ed700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c151480 con 0x7fca8c06aa00 2023-10-08T00:22:08.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.883+0000 7fca92bec700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca8c0cce30 0x7fca8c143560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:08.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.883+0000 7fca92bec700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca8c0cce30 0x7fca8c143560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33074/0 (socket says 172.21.15.99:33074) 2023-10-08T00:22:08.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.883+0000 7fca92bec700 1 -- 172.21.15.99:0/382205422 learned_addr learned my addr 172.21.15.99:0/382205422 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:08.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.883+0000 7fca92bec700 1 -- 172.21.15.99:0/382205422 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca8c06aa00 msgr2=0x7fca8c0b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:08.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.883+0000 7fca92bec700 1 --2- 172.21.15.99:0/382205422 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca8c06aa00 0x7fca8c0b2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:08.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.883+0000 7fca92bec700 1 -- 172.21.15.99:0/382205422 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca8c0b6580 msgr2=0x7fca8c158b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:08.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.883+0000 7fca91bea700 1 --2- 172.21.15.99:0/382205422 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca8c0b6580 0x7fca8c158b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:08.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.883+0000 7fca92bec700 1 --2- 172.21.15.99:0/382205422 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca8c0b6580 0x7fca8c158b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:08.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.883+0000 7fca92bec700 1 -- 172.21.15.99:0/382205422 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca84009420 con 0x7fca8c0cce30 2023-10-08T00:22:08.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.883+0000 7fca92bec700 1 --2- 172.21.15.99:0/382205422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca8c0cce30 0x7fca8c143560 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fca840093f0 tx=0x7fca84016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:08.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.883+0000 7fca8a7fc700 1 -- 172.21.15.99:0/382205422 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8401ce00 con 0x7fca8c0cce30 2023-10-08T00:22:08.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.883+0000 7fca8a7fc700 1 -- 172.21.15.99:0/382205422 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca8401a450 con 0x7fca8c0cce30 2023-10-08T00:22:08.893 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.883+0000 7fca8a7fc700 1 -- 172.21.15.99:0/382205422 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8401a810 con 0x7fca8c0cce30 2023-10-08T00:22:08.893 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.883+0000 7fca933ed700 1 -- 172.21.15.99:0/382205422 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca8c151760 con 0x7fca8c0cce30 2023-10-08T00:22:08.893 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.883+0000 7fca933ed700 1 -- 172.21.15.99:0/382205422 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca8c143aa0 con 0x7fca8c0cce30 2023-10-08T00:22:08.893 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.884+0000 7fca933ed700 1 -- 172.21.15.99:0/382205422 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca8c1512e0 con 0x7fca8c0cce30 2023-10-08T00:22:08.893 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.885+0000 7fca8a7fc700 1 -- 172.21.15.99:0/382205422 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fca8401ab80 con 0x7fca8c0cce30 2023-10-08T00:22:08.893 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.885+0000 7fca8a7fc700 1 --2- 172.21.15.99:0/382205422 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fca78066f30 0x7fca780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:08.893 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.885+0000 7fca8a7fc700 1 -- 172.21.15.99:0/382205422 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5820+0+0 (secure 0 0 0) 0x7fca840863c0 con 0x7fca8c0cce30 2023-10-08T00:22:08.893 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.885+0000 7fca923eb700 1 --2- 172.21.15.99:0/382205422 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fca78066f30 0x7fca780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:08.893 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.886+0000 7fca923eb700 1 --2- 172.21.15.99:0/382205422 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fca78066f30 0x7fca780693e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fca7c001560 tx=0x7fca7c0016c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:08.894 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:08.889+0000 7fca8a7fc700 1 -- 172.21.15.99:0/382205422 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fca8c1512e0 con 0x7fca8c0cce30 2023-10-08T00:22:09.114 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.114+0000 7fca933ed700 1 -- 172.21.15.99:0/382205422 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fca8c1512e0 con 0x7fca8c0cce30 2023-10-08T00:22:09.749 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.748+0000 7fca8a7fc700 1 -- 172.21.15.99:0/382205422 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v86) v1 ==== 104+0+0 (secure 0 0 0) 0x7fca8c1512e0 con 0x7fca8c0cce30 2023-10-08T00:22:09.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.751+0000 7fca933ed700 1 -- 172.21.15.99:0/382205422 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fca78066f30 msgr2=0x7fca780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:09.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.751+0000 7fca933ed700 1 --2- 172.21.15.99:0/382205422 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fca78066f30 0x7fca780693e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fca7c001560 tx=0x7fca7c0016c0 comp rx=0 tx=0).stop 2023-10-08T00:22:09.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.751+0000 7fca933ed700 1 -- 172.21.15.99:0/382205422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca8c0cce30 msgr2=0x7fca8c143560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:09.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.751+0000 7fca933ed700 1 --2- 172.21.15.99:0/382205422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca8c0cce30 0x7fca8c143560 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fca840093f0 tx=0x7fca84016000 comp rx=0 tx=0).stop 2023-10-08T00:22:09.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.751+0000 7fca933ed700 1 -- 172.21.15.99:0/382205422 shutdown_connections 2023-10-08T00:22:09.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.751+0000 7fca933ed700 1 --2- 172.21.15.99:0/382205422 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fca78066f30 0x7fca780693e0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:09.752 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.751+0000 7fca933ed700 1 --2- 172.21.15.99:0/382205422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca8c0cce30 0x7fca8c143560 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:09.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.751+0000 7fca933ed700 1 --2- 172.21.15.99:0/382205422 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca8c06aa00 0x7fca8c0b2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:09.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.751+0000 7fca933ed700 1 --2- 172.21.15.99:0/382205422 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca8c0b6580 0x7fca8c158b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:09.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.751+0000 7fca933ed700 1 -- 172.21.15.99:0/382205422 >> 172.21.15.99:0/382205422 conn(0x7fca8c068cb0 msgr2=0x7fca8c150360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:09.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.751+0000 7fca933ed700 1 -- 172.21.15.99:0/382205422 shutdown_connections 2023-10-08T00:22:09.753 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.751+0000 7fca933ed700 1 -- 172.21.15.99:0/382205422 wait complete. 2023-10-08T00:22:09.764 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T00:22:09.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.882+0000 7f38f1f11700 1 Processor -- start 2023-10-08T00:22:09.883 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.882+0000 7f38f1f11700 1 -- start start 2023-10-08T00:22:09.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.883+0000 7f38f1f11700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f38ec159ef0 0x7f38ec15a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:09.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.883+0000 7f38f1f11700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f38ec0b46f0 0x7f38ec0b4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:09.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.883+0000 7f38f1f11700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38ec0b5070 con 0x7f38ec0b6a10 2023-10-08T00:22:09.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.883+0000 7f38f1f11700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38ec0b17b0 con 0x7f38ec159ef0 2023-10-08T00:22:09.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.883+0000 7f38f1f11700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec0b1a60 con 0x7f38ec0b46f0 2023-10-08T00:22:09.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.883+0000 7f38eb7fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f38ec159ef0 0x7f38ec15a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:09.886 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.883+0000 7f38eb7fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f38ec159ef0 0x7f38ec15a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33082/0 (socket says 172.21.15.99:33082) 2023-10-08T00:22:09.886 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.883+0000 7f38eb7fe700 1 -- 172.21.15.99:0/451503138 learned_addr learned my addr 172.21.15.99:0/451503138 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:09.886 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.883+0000 7f38eb7fe700 1 -- 172.21.15.99:0/451503138 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f38ec0b46f0 msgr2=0x7f38ec0b4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:09.886 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.883+0000 7f38eb7fe700 1 --2- 172.21.15.99:0/451503138 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f38ec0b46f0 0x7f38ec0b4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:09.886 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.883+0000 7f38eb7fe700 1 -- 172.21.15.99:0/451503138 >> v1:172.21.15.191:6789/0 conn(0x7f38ec0b6a10 legacy=0x7f38ec0b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:09.886 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.883+0000 7f38eb7fe700 1 -- 172.21.15.99:0/451503138 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38ec0b1d40 con 0x7f38ec159ef0 2023-10-08T00:22:09.886 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.884+0000 7f38eb7fe700 1 --2- 172.21.15.99:0/451503138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38ec159ef0 0x7f38ec15a2c0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f38dc009990 tx=0x7f38dc009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63bfa200e54cc434 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:09.886 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.884+0000 7f38ea7fc700 1 -- 172.21.15.99:0/451503138 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38dc005c10 con 0x7f38ec159ef0 2023-10-08T00:22:09.886 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.884+0000 7f38ea7fc700 1 -- 172.21.15.99:0/451503138 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38dc005d70 con 0x7f38ec159ef0 2023-10-08T00:22:09.886 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.884+0000 7f38ea7fc700 1 -- 172.21.15.99:0/451503138 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38dc00b840 con 0x7f38ec159ef0 2023-10-08T00:22:09.886 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.884+0000 7f38f1f11700 1 -- 172.21.15.99:0/451503138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38ec159ef0 msgr2=0x7f38ec15a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:09.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.884+0000 7f38f1f11700 1 --2- 172.21.15.99:0/451503138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38ec159ef0 0x7f38ec15a2c0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f38dc009990 tx=0x7f38dc009220 comp rx=0 tx=0).stop 2023-10-08T00:22:09.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.884+0000 7f38f1f11700 1 -- 172.21.15.99:0/451503138 shutdown_connections 2023-10-08T00:22:09.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.884+0000 7f38f1f11700 1 --2- 172.21.15.99:0/451503138 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f38ec0b46f0 0x7f38ec0b4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:09.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.884+0000 7f38f1f11700 1 --2- 172.21.15.99:0/451503138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38ec159ef0 0x7f38ec15a2c0 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:09.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.884+0000 7f38f1f11700 1 -- 172.21.15.99:0/451503138 >> 172.21.15.99:0/451503138 conn(0x7f38ec09a0a0 msgr2=0x7f38ec067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:09.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.884+0000 7f38f1f11700 1 -- 172.21.15.99:0/451503138 shutdown_connections 2023-10-08T00:22:09.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.884+0000 7f38f1f11700 1 -- 172.21.15.99:0/451503138 wait complete. 2023-10-08T00:22:09.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.886+0000 7f38f1f11700 1 Processor -- start 2023-10-08T00:22:09.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.886+0000 7f38f1f11700 1 -- start start 2023-10-08T00:22:09.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.886+0000 7f38f1f11700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38ec0b46f0 0x7f38ec16a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:09.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.886+0000 7f38f1f11700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38ec0b6a10 0x7f38ec16d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:09.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.886+0000 7f38f1f11700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f38ec159ef0 0x7f38ec0abe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:09.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.886+0000 7f38f1f11700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec16d5c0 con 0x7f38ec0b6a10 2023-10-08T00:22:09.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.886+0000 7f38f1f11700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec0acec0 con 0x7f38ec0b46f0 2023-10-08T00:22:09.890 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.886+0000 7f38f1f11700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec0afd40 con 0x7f38ec159ef0 2023-10-08T00:22:09.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.886+0000 7f38eaffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38ec0b6a10 0x7f38ec16d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:09.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.886+0000 7f38eaffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38ec0b6a10 0x7f38ec16d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33084/0 (socket says 172.21.15.99:33084) 2023-10-08T00:22:09.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.886+0000 7f38eaffd700 1 -- 172.21.15.99:0/1179715657 learned_addr learned my addr 172.21.15.99:0/1179715657 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:09.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.886+0000 7f38eb7fe700 1 --2- 172.21.15.99:0/1179715657 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38ec0b46f0 0x7f38ec16a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:09.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.886+0000 7f38ebfff700 1 --2- 172.21.15.99:0/1179715657 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f38ec159ef0 0x7f38ec0abe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:09.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.887+0000 7f38eaffd700 1 -- 172.21.15.99:0/1179715657 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f38ec159ef0 msgr2=0x7f38ec0abe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:09.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.887+0000 7f38eaffd700 1 --2- 172.21.15.99:0/1179715657 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f38ec159ef0 0x7f38ec0abe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:09.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.887+0000 7f38eaffd700 1 -- 172.21.15.99:0/1179715657 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38ec0b46f0 msgr2=0x7f38ec16a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:09.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.887+0000 7f38eaffd700 1 --2- 172.21.15.99:0/1179715657 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38ec0b46f0 0x7f38ec16a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:09.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.887+0000 7f38eaffd700 1 -- 172.21.15.99:0/1179715657 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38dc009480 con 0x7f38ec0b6a10 2023-10-08T00:22:09.891 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.887+0000 7f38ebfff700 1 --2- 172.21.15.99:0/1179715657 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f38ec159ef0 0x7f38ec0abe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:22:09.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.887+0000 7f38eaffd700 1 --2- 172.21.15.99:0/1179715657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38ec0b6a10 0x7f38ec16d080 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f38e000b7e0 tx=0x7f38e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:09.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.887+0000 7f38d37fe700 1 -- 172.21.15.99:0/1179715657 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e000f510 con 0x7f38ec0b6a10 2023-10-08T00:22:09.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.887+0000 7f38d37fe700 1 -- 172.21.15.99:0/1179715657 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e000f670 con 0x7f38ec0b6a10 2023-10-08T00:22:09.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.887+0000 7f38d37fe700 1 -- 172.21.15.99:0/1179715657 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e000fa10 con 0x7f38ec0b6a10 2023-10-08T00:22:09.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.887+0000 7f38f1f11700 1 -- 172.21.15.99:0/1179715657 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38ec0ac900 con 0x7f38ec0b6a10 2023-10-08T00:22:09.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.887+0000 7f38f1f11700 1 -- 172.21.15.99:0/1179715657 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38ec0adc70 con 0x7f38ec0b6a10 2023-10-08T00:22:09.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.888+0000 7f38f1f11700 1 -- 172.21.15.99:0/1179715657 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38ec04c3c0 con 0x7f38ec0b6a10 2023-10-08T00:22:09.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.888+0000 7f38d37fe700 1 -- 172.21.15.99:0/1179715657 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f38e001a400 con 0x7f38ec0b6a10 2023-10-08T00:22:09.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.889+0000 7f38d37fe700 1 --2- 172.21.15.99:0/1179715657 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f38d8066f30 0x7f38d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:09.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.889+0000 7f38d37fe700 1 -- 172.21.15.99:0/1179715657 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5820+0+0 (secure 0 0 0) 0x7f38e0084000 con 0x7f38ec0b6a10 2023-10-08T00:22:09.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.889+0000 7f38eb7fe700 1 --2- 172.21.15.99:0/1179715657 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f38d8066f30 0x7f38d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:09.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.889+0000 7f38eb7fe700 1 --2- 172.21.15.99:0/1179715657 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f38d8066f30 0x7f38d80693e0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f38dc003080 tx=0x7f38dc015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:09.893 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:09.893+0000 7f38d37fe700 1 -- 172.21.15.99:0/1179715657 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f38e0051280 con 0x7f38ec0b6a10 2023-10-08T00:22:10.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.073+0000 7f38f1f11700 1 -- 172.21.15.99:0/1179715657 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f38ec0690e0 con 0x7f38ec0b6a10 2023-10-08T00:22:10.755 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.754+0000 7f38d37fe700 1 -- 172.21.15.99:0/1179715657 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v191) v1 ==== 181+0+0 (secure 0 0 0) 0x7f38e00548a0 con 0x7f38ec0b6a10 2023-10-08T00:22:10.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.758+0000 7f38f1f11700 1 -- 172.21.15.99:0/1179715657 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f38d8066f30 msgr2=0x7f38d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:10.758 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.758+0000 7f38f1f11700 1 --2- 172.21.15.99:0/1179715657 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f38d8066f30 0x7f38d80693e0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f38dc003080 tx=0x7f38dc015000 comp rx=0 tx=0).stop 2023-10-08T00:22:10.759 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.758+0000 7f38f1f11700 1 -- 172.21.15.99:0/1179715657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38ec0b6a10 msgr2=0x7f38ec16d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:10.759 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.758+0000 7f38f1f11700 1 --2- 172.21.15.99:0/1179715657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38ec0b6a10 0x7f38ec16d080 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f38e000b7e0 tx=0x7f38e00091a0 comp rx=0 tx=0).stop 2023-10-08T00:22:10.759 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.758+0000 7f38f1f11700 1 -- 172.21.15.99:0/1179715657 shutdown_connections 2023-10-08T00:22:10.759 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.758+0000 7f38f1f11700 1 --2- 172.21.15.99:0/1179715657 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f38d8066f30 0x7f38d80693e0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:10.759 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.758+0000 7f38f1f11700 1 --2- 172.21.15.99:0/1179715657 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38ec0b46f0 0x7f38ec16a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:10.759 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.758+0000 7f38f1f11700 1 --2- 172.21.15.99:0/1179715657 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f38ec159ef0 0x7f38ec0abe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:10.759 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.758+0000 7f38f1f11700 1 --2- 172.21.15.99:0/1179715657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38ec0b6a10 0x7f38ec16d080 secure :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f38e000b7e0 tx=0x7f38e00091a0 comp rx=0 tx=0).stop 2023-10-08T00:22:10.759 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.758+0000 7f38f1f11700 1 -- 172.21.15.99:0/1179715657 >> 172.21.15.99:0/1179715657 conn(0x7f38ec09a0a0 msgr2=0x7f38ec0b0670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:10.760 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.758+0000 7f38f1f11700 1 -- 172.21.15.99:0/1179715657 shutdown_connections 2023-10-08T00:22:10.760 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.758+0000 7f38f1f11700 1 -- 172.21.15.99:0/1179715657 wait complete. 2023-10-08T00:22:10.760 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' removed 2023-10-08T00:22:10.773 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T00:22:10.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.891+0000 7ff718734700 1 Processor -- start 2023-10-08T00:22:10.892 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.891+0000 7ff718734700 1 -- start start 2023-10-08T00:22:10.896 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.892+0000 7ff718734700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff7101416f0 0x7ff710070700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:10.896 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.892+0000 7ff718734700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff710070c40 0x7ff710067940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:10.896 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.892+0000 7ff718734700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff71015c850 con 0x7ff710141320 2023-10-08T00:22:10.896 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.892+0000 7ff718734700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7100710c0 con 0x7ff7101416f0 2023-10-08T00:22:10.896 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.892+0000 7ff718734700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff71015bc40 con 0x7ff710070c40 2023-10-08T00:22:10.896 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.892+0000 7ff7164d0700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7ff710141320 0x7ff7100c9a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56848/0 (socket says 172.21.15.99:56848) 2023-10-08T00:22:10.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.892+0000 7ff7164d0700 1 -- 172.21.15.99:0/1754814241 learned_addr learned my addr 172.21.15.99:0/1754814241 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:22:10.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.893+0000 7ff715ccf700 1 --2- 172.21.15.99:0/1754814241 >> v2:172.21.15.191:3300/0 conn(0x7ff7101416f0 0x7ff710070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:10.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.893+0000 7ff716cd1700 1 --2- 172.21.15.99:0/1754814241 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff710070c40 0x7ff710067940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:10.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.893+0000 7ff7154ce700 1 -- 172.21.15.99:0/1754814241 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3439329152 0 0) 0x7ff71015c850 con 0x7ff710141320 2023-10-08T00:22:10.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.893+0000 7ff7154ce700 1 -- 172.21.15.99:0/1754814241 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6fc003570 con 0x7ff710141320 2023-10-08T00:22:10.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.893+0000 7ff7154ce700 1 -- 172.21.15.99:0/1754814241 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 166172209 0 0) 0x7ff6fc003570 con 0x7ff710141320 2023-10-08T00:22:10.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.893+0000 7ff7154ce700 1 -- 172.21.15.99:0/1754814241 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff710070c40 msgr2=0x7ff710067940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:10.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.893+0000 7ff7154ce700 1 --2- 172.21.15.99:0/1754814241 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff710070c40 0x7ff710067940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:10.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.893+0000 7ff7154ce700 1 -- 172.21.15.99:0/1754814241 >> v2:172.21.15.191:3300/0 conn(0x7ff7101416f0 msgr2=0x7ff710070700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:10.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.893+0000 7ff7154ce700 1 --2- 172.21.15.99:0/1754814241 >> v2:172.21.15.191:3300/0 conn(0x7ff7101416f0 0x7ff710070700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:10.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.893+0000 7ff7154ce700 1 -- 172.21.15.99:0/1754814241 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff71015bf20 con 0x7ff710141320 2023-10-08T00:22:10.898 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.893+0000 7ff7154ce700 1 -- 172.21.15.99:0/1754814241 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7ff70c0017f0 con 0x7ff710141320 2023-10-08T00:22:10.898 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.893+0000 7ff7154ce700 1 -- 172.21.15.99:0/1754814241 >> v1:172.21.15.99:6789/0 conn(0x7ff710141320 legacy=0x7ff7100c9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:10.898 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.893+0000 7ff7154ce700 1 --2- 172.21.15.99:0/1754814241 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6fc003a60 0x7ff6fc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:10.898 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.893+0000 7ff7154ce700 1 --2- 172.21.15.99:0/1754814241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6fc004810 0x7ff6fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:10.898 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.893+0000 7ff7154ce700 1 --2- 172.21.15.99:0/1754814241 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff710070c40 0x7ff6fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:10.898 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.893+0000 7ff7154ce700 1 -- 172.21.15.99:0/1754814241 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7100710c0 con 0x7ff6fc004810 2023-10-08T00:22:10.898 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.893+0000 7ff7154ce700 1 -- 172.21.15.99:0/1754814241 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff71015bc40 con 0x7ff6fc003a60 2023-10-08T00:22:10.898 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.893+0000 7ff7154ce700 1 -- 172.21.15.99:0/1754814241 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6fc00ba60 con 0x7ff710070c40 2023-10-08T00:22:10.898 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.893+0000 7ff715ccf700 1 --2- 172.21.15.99:0/1754814241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6fc004810 0x7ff6fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:10.898 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.893+0000 7ff716cd1700 1 --2- 172.21.15.99:0/1754814241 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff710070c40 0x7ff6fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:10.898 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.893+0000 7ff7164d0700 1 --2- 172.21.15.99:0/1754814241 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6fc003a60 0x7ff6fc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:10.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.893+0000 7ff715ccf700 1 -- 172.21.15.99:0/1754814241 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff710070c40 msgr2=0x7ff6fc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:10.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.893+0000 7ff715ccf700 1 --2- 172.21.15.99:0/1754814241 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff710070c40 0x7ff6fc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:10.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.893+0000 7ff715ccf700 1 -- 172.21.15.99:0/1754814241 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6fc003a60 msgr2=0x7ff6fc003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:10.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.893+0000 7ff715ccf700 1 --2- 172.21.15.99:0/1754814241 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6fc003a60 0x7ff6fc003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:10.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.893+0000 7ff715ccf700 1 -- 172.21.15.99:0/1754814241 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6fc003570 con 0x7ff6fc004810 2023-10-08T00:22:10.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.894+0000 7ff715ccf700 1 --2- 172.21.15.99:0/1754814241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6fc004810 0x7ff6fc004c80 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7ff700009c30 tx=0x7ff7000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c71bd38a4b4e209d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:10.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.894+0000 7ff7154ce700 1 -- 172.21.15.99:0/1754814241 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70000e710 con 0x7ff6fc004810 2023-10-08T00:22:10.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.894+0000 7ff7154ce700 1 -- 172.21.15.99:0/1754814241 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff70000e870 con 0x7ff6fc004810 2023-10-08T00:22:10.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.894+0000 7ff7154ce700 1 -- 172.21.15.99:0/1754814241 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70000ec30 con 0x7ff6fc004810 2023-10-08T00:22:10.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.894+0000 7ff718734700 1 -- 172.21.15.99:0/1754814241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6fc004810 msgr2=0x7ff6fc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:10.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.894+0000 7ff718734700 1 --2- 172.21.15.99:0/1754814241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6fc004810 0x7ff6fc004c80 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7ff700009c30 tx=0x7ff7000091a0 comp rx=0 tx=0).stop 2023-10-08T00:22:10.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.894+0000 7ff716cd1700 1 -- 172.21.15.99:0/1754814241 reap_dead start 2023-10-08T00:22:10.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.894+0000 7ff718734700 1 -- 172.21.15.99:0/1754814241 shutdown_connections 2023-10-08T00:22:10.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.894+0000 7ff718734700 1 -- 172.21.15.99:0/1754814241 >> 172.21.15.99:0/1754814241 conn(0x7ff7100cdae0 msgr2=0x7ff7100cdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:10.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.894+0000 7ff718734700 1 -- 172.21.15.99:0/1754814241 shutdown_connections 2023-10-08T00:22:10.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.894+0000 7ff718734700 1 -- 172.21.15.99:0/1754814241 wait complete. 2023-10-08T00:22:10.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.895+0000 7ff718734700 1 Processor -- start 2023-10-08T00:22:10.900 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.895+0000 7ff718734700 1 -- start start 2023-10-08T00:22:10.904 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.896+0000 7ff718734700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6fc004810 0x7ff7101405f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:10.905 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.896+0000 7ff718734700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7100b0160 0x7ff7100ad7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:10.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.896+0000 7ff718734700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7100c6ce0 0x7ff7100c7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:10.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.896+0000 7ff718734700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7100c7690 con 0x7ff7100c6ce0 2023-10-08T00:22:10.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.896+0000 7ff718734700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff71013e400 con 0x7ff7100b0160 2023-10-08T00:22:10.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.896+0000 7ff718734700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff71013d7f0 con 0x7ff6fc004810 2023-10-08T00:22:10.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.896+0000 7ff716cd1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7100c6ce0 0x7ff7100c7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:10.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.896+0000 7ff716cd1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7100c6ce0 0x7ff7100c7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33110/0 (socket says 172.21.15.99:33110) 2023-10-08T00:22:10.906 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.896+0000 7ff716cd1700 1 -- 172.21.15.99:0/874421986 learned_addr learned my addr 172.21.15.99:0/874421986 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:10.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.896+0000 7ff7164d0700 1 --2- 172.21.15.99:0/874421986 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6fc004810 0x7ff7101405f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:10.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.896+0000 7ff715ccf700 1 --2- 172.21.15.99:0/874421986 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7100b0160 0x7ff7100ad7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:10.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.896+0000 7ff716cd1700 1 -- 172.21.15.99:0/874421986 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6fc004810 msgr2=0x7ff7101405f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:10.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.896+0000 7ff716cd1700 1 --2- 172.21.15.99:0/874421986 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6fc004810 0x7ff7101405f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:10.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.896+0000 7ff716cd1700 1 -- 172.21.15.99:0/874421986 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7100b0160 msgr2=0x7ff7100ad7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:10.907 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.896+0000 7ff716cd1700 1 --2- 172.21.15.99:0/874421986 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7100b0160 0x7ff7100ad7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:10.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.896+0000 7ff716cd1700 1 -- 172.21.15.99:0/874421986 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff71013dad0 con 0x7ff7100c6ce0 2023-10-08T00:22:10.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.897+0000 7ff715ccf700 1 --2- 172.21.15.99:0/874421986 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7100b0160 0x7ff7100ad7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:22:10.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.897+0000 7ff716cd1700 1 --2- 172.21.15.99:0/874421986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7100c6ce0 0x7ff7100c7150 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7ff71006dd90 tx=0x7ff7080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:10.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.897+0000 7ff7067fc700 1 -- 172.21.15.99:0/874421986 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70800f910 con 0x7ff7100c6ce0 2023-10-08T00:22:10.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.897+0000 7ff7067fc700 1 -- 172.21.15.99:0/874421986 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff70800fa70 con 0x7ff7100c6ce0 2023-10-08T00:22:10.908 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.897+0000 7ff7067fc700 1 -- 172.21.15.99:0/874421986 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70800fe10 con 0x7ff7100c6ce0 2023-10-08T00:22:10.909 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.897+0000 7ff718734700 1 -- 172.21.15.99:0/874421986 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff71013ddb0 con 0x7ff7100c6ce0 2023-10-08T00:22:10.909 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.897+0000 7ff718734700 1 -- 172.21.15.99:0/874421986 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff710140b30 con 0x7ff7100c6ce0 2023-10-08T00:22:10.909 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.898+0000 7ff718734700 1 -- 172.21.15.99:0/874421986 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff71013e250 con 0x7ff7100c6ce0 2023-10-08T00:22:10.909 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.898+0000 7ff7067fc700 1 -- 172.21.15.99:0/874421986 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff708014650 con 0x7ff7100c6ce0 2023-10-08T00:22:10.909 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.898+0000 7ff7067fc700 1 --2- 172.21.15.99:0/874421986 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff6f4066f60 0x7ff6f4069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:10.910 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.899+0000 7ff7067fc700 1 -- 172.21.15.99:0/874421986 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5382+0+0 (secure 0 0 0) 0x7ff708084960 con 0x7ff7100c6ce0 2023-10-08T00:22:10.910 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.899+0000 7ff7164d0700 1 --2- 172.21.15.99:0/874421986 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff6f4066f60 0x7ff6f4069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:10.910 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.899+0000 7ff7164d0700 1 --2- 172.21.15.99:0/874421986 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff6f4066f60 0x7ff6f4069410 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7ff70c0020a0 tx=0x7ff70c006040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:10.910 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:10.903+0000 7ff7067fc700 1 -- 172.21.15.99:0/874421986 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff708051d90 con 0x7ff7100c6ce0 2023-10-08T00:22:11.088 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.088+0000 7ff718734700 1 -- 172.21.15.99:0/874421986 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff71013e590 con 0x7ff7100c6ce0 2023-10-08T00:22:11.181 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.180+0000 7ff7067fc700 1 -- 172.21.15.99:0/874421986 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v192) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff7080220d0 con 0x7ff7100c6ce0 2023-10-08T00:22:11.184 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.184+0000 7ff718734700 1 -- 172.21.15.99:0/874421986 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff6f4066f60 msgr2=0x7ff6f4069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:11.184 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.184+0000 7ff718734700 1 --2- 172.21.15.99:0/874421986 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff6f4066f60 0x7ff6f4069410 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7ff70c0020a0 tx=0x7ff70c006040 comp rx=0 tx=0).stop 2023-10-08T00:22:11.184 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.184+0000 7ff718734700 1 -- 172.21.15.99:0/874421986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7100c6ce0 msgr2=0x7ff7100c7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:11.184 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.184+0000 7ff718734700 1 --2- 172.21.15.99:0/874421986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7100c6ce0 0x7ff7100c7150 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7ff71006dd90 tx=0x7ff7080091a0 comp rx=0 tx=0).stop 2023-10-08T00:22:11.185 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.184+0000 7ff718734700 1 -- 172.21.15.99:0/874421986 shutdown_connections 2023-10-08T00:22:11.185 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.184+0000 7ff718734700 1 --2- 172.21.15.99:0/874421986 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff6f4066f60 0x7ff6f4069410 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:11.185 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.184+0000 7ff718734700 1 --2- 172.21.15.99:0/874421986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7100c6ce0 0x7ff7100c7150 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:11.185 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.184+0000 7ff718734700 1 --2- 172.21.15.99:0/874421986 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6fc004810 0x7ff7101405f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:11.185 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.184+0000 7ff718734700 1 --2- 172.21.15.99:0/874421986 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7100b0160 0x7ff7100ad7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:11.185 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.184+0000 7ff718734700 1 -- 172.21.15.99:0/874421986 >> 172.21.15.99:0/874421986 conn(0x7ff7100cdae0 msgr2=0x7ff71015b490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:11.185 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.184+0000 7ff718734700 1 -- 172.21.15.99:0/874421986 shutdown_connections 2023-10-08T00:22:11.185 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.184+0000 7ff718734700 1 -- 172.21.15.99:0/874421986 wait complete. 2023-10-08T00:22:11.186 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' removed 2023-10-08T00:22:11.197 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-08T00:22:11.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.315+0000 7f37f9c64700 1 Processor -- start 2023-10-08T00:22:11.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.315+0000 7f37f9c64700 1 -- start start 2023-10-08T00:22:11.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.316+0000 7f37f9c64700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f37f40cce30 0x7f37f415aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:11.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.316+0000 7f37f9c64700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f37f406aa00 0x7f37f40b5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:11.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.316+0000 7f37f9c64700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37f40b4d70 con 0x7f37f40b6580 2023-10-08T00:22:11.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.316+0000 7f37f9c64700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37f40cd8d0 con 0x7f37f40cce30 2023-10-08T00:22:11.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.316+0000 7f37f9c64700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37f40b4440 con 0x7f37f406aa00 2023-10-08T00:22:11.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.316+0000 7f37f8c62700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f37f40cce30 0x7f37f415aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:11.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.316+0000 7f37f9463700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f37f406aa00 0x7f37f40b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:11.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.316+0000 7f37f3fff700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f37f40b6580 0x7f37f406a2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:35242/0 (socket says 172.21.15.99:35242) 2023-10-08T00:22:11.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.316+0000 7f37f3fff700 1 -- 172.21.15.99:0/189271772 learned_addr learned my addr 172.21.15.99:0/189271772 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:22:11.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.317+0000 7f37f9463700 1 -- 172.21.15.99:0/189271772 >> v2:172.21.15.191:3300/0 conn(0x7f37f40cce30 msgr2=0x7f37f415aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:11.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.317+0000 7f37f9463700 1 --2- 172.21.15.99:0/189271772 >> v2:172.21.15.191:3300/0 conn(0x7f37f40cce30 0x7f37f415aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:11.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.317+0000 7f37f9463700 1 -- 172.21.15.99:0/189271772 >> v1:172.21.15.191:6789/0 conn(0x7f37f40b6580 legacy=0x7f37f406a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:11.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.317+0000 7f37f9463700 1 -- 172.21.15.99:0/189271772 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f40b4720 con 0x7f37f406aa00 2023-10-08T00:22:11.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.317+0000 7f37f8c62700 1 --2- 172.21.15.99:0/189271772 >> v2:172.21.15.191:3300/0 conn(0x7f37f40cce30 0x7f37f415aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:22:11.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.317+0000 7f37f9463700 1 --2- 172.21.15.99:0/189271772 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f37f406aa00 0x7f37f40b5200 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f37ec0098e0 tx=0x7f37ec0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3dec5acfd8cb72f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:11.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.317+0000 7f37f37fe700 1 -- 172.21.15.99:0/189271772 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ec00b640 con 0x7f37f406aa00 2023-10-08T00:22:11.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.317+0000 7f37f37fe700 1 -- 172.21.15.99:0/189271772 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37ec00b7a0 con 0x7f37f406aa00 2023-10-08T00:22:11.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.317+0000 7f37f37fe700 1 -- 172.21.15.99:0/189271772 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ec00bb40 con 0x7f37f406aa00 2023-10-08T00:22:11.324 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.318+0000 7f37f9c64700 1 -- 172.21.15.99:0/189271772 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f37f406aa00 msgr2=0x7f37f40b5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:11.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.318+0000 7f37f9c64700 1 --2- 172.21.15.99:0/189271772 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f37f406aa00 0x7f37f40b5200 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f37ec0098e0 tx=0x7f37ec0091a0 comp rx=0 tx=0).stop 2023-10-08T00:22:11.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.318+0000 7f37f9c64700 1 -- 172.21.15.99:0/189271772 shutdown_connections 2023-10-08T00:22:11.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.318+0000 7f37f9c64700 1 --2- 172.21.15.99:0/189271772 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f37f406aa00 0x7f37f40b5200 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:11.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.318+0000 7f37f9c64700 1 --2- 172.21.15.99:0/189271772 >> v2:172.21.15.191:3300/0 conn(0x7f37f40cce30 0x7f37f415aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:11.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.318+0000 7f37f9c64700 1 -- 172.21.15.99:0/189271772 >> 172.21.15.99:0/189271772 conn(0x7f37f4068cb0 msgr2=0x7f37f40690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:11.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.318+0000 7f37f9c64700 1 -- 172.21.15.99:0/189271772 shutdown_connections 2023-10-08T00:22:11.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.318+0000 7f37f9c64700 1 -- 172.21.15.99:0/189271772 wait complete. 2023-10-08T00:22:11.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.319+0000 7f37f9c64700 1 Processor -- start 2023-10-08T00:22:11.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.319+0000 7f37f9c64700 1 -- start start 2023-10-08T00:22:11.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.319+0000 7f37f9c64700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f37f406aa00 0x7f37f414d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:11.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.319+0000 7f37f9c64700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f37f40b6580 0x7f37f414e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:11.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.319+0000 7f37f9c64700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37f40cce30 0x7f37f414e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:11.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.320+0000 7f37f9c64700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f41512e0 con 0x7f37f40cce30 2023-10-08T00:22:11.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.320+0000 7f37f9c64700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f40b1010 con 0x7f37f406aa00 2023-10-08T00:22:11.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.320+0000 7f37f9c64700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37f40b12f0 con 0x7f37f40b6580 2023-10-08T00:22:11.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.320+0000 7f37f9463700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37f40cce30 0x7f37f414e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:11.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.320+0000 7f37f9463700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37f40cce30 0x7f37f414e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33114/0 (socket says 172.21.15.99:33114) 2023-10-08T00:22:11.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.320+0000 7f37f9463700 1 -- 172.21.15.99:0/1521345383 learned_addr learned my addr 172.21.15.99:0/1521345383 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:11.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.320+0000 7f37f3fff700 1 --2- 172.21.15.99:0/1521345383 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f37f40b6580 0x7f37f414e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:11.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.320+0000 7f37f8c62700 1 --2- 172.21.15.99:0/1521345383 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f37f406aa00 0x7f37f414d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:11.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.320+0000 7f37f9463700 1 -- 172.21.15.99:0/1521345383 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f37f40b6580 msgr2=0x7f37f414e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:11.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.320+0000 7f37f9463700 1 --2- 172.21.15.99:0/1521345383 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f37f40b6580 0x7f37f414e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:11.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.320+0000 7f37f9463700 1 -- 172.21.15.99:0/1521345383 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f37f406aa00 msgr2=0x7f37f414d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:11.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.320+0000 7f37f9463700 1 --2- 172.21.15.99:0/1521345383 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f37f406aa00 0x7f37f414d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:11.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.320+0000 7f37f9463700 1 -- 172.21.15.99:0/1521345383 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e4009a20 con 0x7f37f40cce30 2023-10-08T00:22:11.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.320+0000 7f37f9463700 1 --2- 172.21.15.99:0/1521345383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37f40cce30 0x7f37f414e6f0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f37ec013000 tx=0x7f37ec017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:11.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.320+0000 7f37f0ff9700 1 -- 172.21.15.99:0/1521345383 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ec00b640 con 0x7f37f40cce30 2023-10-08T00:22:11.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.320+0000 7f37f0ff9700 1 -- 172.21.15.99:0/1521345383 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37ec01be40 con 0x7f37f40cce30 2023-10-08T00:22:11.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.320+0000 7f37f9c64700 1 -- 172.21.15.99:0/1521345383 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37ec009420 con 0x7f37f40cce30 2023-10-08T00:22:11.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.320+0000 7f37f0ff9700 1 -- 172.21.15.99:0/1521345383 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ec011860 con 0x7f37f40cce30 2023-10-08T00:22:11.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.320+0000 7f37f9c64700 1 -- 172.21.15.99:0/1521345383 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37f40b18b0 con 0x7f37f40cce30 2023-10-08T00:22:11.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.321+0000 7f37f9c64700 1 -- 172.21.15.99:0/1521345383 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37f41514b0 con 0x7f37f40cce30 2023-10-08T00:22:11.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.322+0000 7f37f0ff9700 1 -- 172.21.15.99:0/1521345383 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f37ec011bd0 con 0x7f37f40cce30 2023-10-08T00:22:11.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.322+0000 7f37f0ff9700 1 --2- 172.21.15.99:0/1521345383 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f37d4066f30 0x7f37d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:11.327 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.322+0000 7f37f0ff9700 1 -- 172.21.15.99:0/1521345383 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f37ec085d70 con 0x7f37f40cce30 2023-10-08T00:22:11.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.322+0000 7f37f8c62700 1 --2- 172.21.15.99:0/1521345383 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f37d4066f30 0x7f37d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:11.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.323+0000 7f37f8c62700 1 --2- 172.21.15.99:0/1521345383 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f37d4066f30 0x7f37d40693e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f37e40099f0 tx=0x7f37e400b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:11.328 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.326+0000 7f37f0ff9700 1 -- 172.21.15.99:0/1521345383 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f37ec0532e0 con 0x7f37f40cce30 2023-10-08T00:22:11.511 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.510+0000 7f37f9c64700 1 -- 172.21.15.99:0/1521345383 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f37f40b4a00 con 0x7f37f40cce30 2023-10-08T00:22:11.514 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.514+0000 7f37f0ff9700 1 -- 172.21.15.99:0/1521345383 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7f37ec056900 con 0x7f37f40cce30 2023-10-08T00:22:11.516 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.516+0000 7f37f9c64700 1 -- 172.21.15.99:0/1521345383 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f37d4066f30 msgr2=0x7f37d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:11.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.516+0000 7f37f9c64700 1 --2- 172.21.15.99:0/1521345383 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f37d4066f30 0x7f37d40693e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f37e40099f0 tx=0x7f37e400b040 comp rx=0 tx=0).stop 2023-10-08T00:22:11.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.516+0000 7f37f9c64700 1 -- 172.21.15.99:0/1521345383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37f40cce30 msgr2=0x7f37f414e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:11.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.516+0000 7f37f9c64700 1 --2- 172.21.15.99:0/1521345383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37f40cce30 0x7f37f414e6f0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f37ec013000 tx=0x7f37ec017000 comp rx=0 tx=0).stop 2023-10-08T00:22:11.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.516+0000 7f37f9c64700 1 -- 172.21.15.99:0/1521345383 shutdown_connections 2023-10-08T00:22:11.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.516+0000 7f37f9c64700 1 --2- 172.21.15.99:0/1521345383 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f37d4066f30 0x7f37d40693e0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:11.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.516+0000 7f37f9c64700 1 --2- 172.21.15.99:0/1521345383 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f37f406aa00 0x7f37f414d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:11.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.516+0000 7f37f9c64700 1 --2- 172.21.15.99:0/1521345383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37f40cce30 0x7f37f414e6f0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:11.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.516+0000 7f37f9c64700 1 --2- 172.21.15.99:0/1521345383 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f37f40b6580 0x7f37f414e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:11.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.516+0000 7f37f9c64700 1 -- 172.21.15.99:0/1521345383 >> 172.21.15.99:0/1521345383 conn(0x7f37f4068cb0 msgr2=0x7f37f415f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:11.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.516+0000 7f37f9c64700 1 -- 172.21.15.99:0/1521345383 shutdown_connections 2023-10-08T00:22:11.517 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.516+0000 7f37f9c64700 1 -- 172.21.15.99:0/1521345383 wait complete. 2023-10-08T00:22:11.529 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file' 2023-10-08T00:22:11.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.643+0000 7faa04b1c700 1 Processor -- start 2023-10-08T00:22:11.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.643+0000 7faa04b1c700 1 -- start start 2023-10-08T00:22:11.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.644+0000 7faa04b1c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa00157870 0x7faa0015b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:11.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.644+0000 7faa04b1c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7faa0015a620 0x7faa0015d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:11.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.644+0000 7faa04b1c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7faa0015d580 0x7faa00158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:11.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.644+0000 7faa04b1c700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa0015cbb0 con 0x7faa0015a620 2023-10-08T00:22:11.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.644+0000 7faa04b1c700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa0015bfa0 con 0x7faa0015d580 2023-10-08T00:22:11.647 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.644+0000 7faa04b1c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa0015c280 con 0x7faa00157870 2023-10-08T00:22:11.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.644+0000 7fa9f5d9b700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7faa0015a620 0x7faa0015d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:11.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.644+0000 7fa9f5d9b700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7faa0015a620 0x7faa0015d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33130/0 (socket says 172.21.15.99:33130) 2023-10-08T00:22:11.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.644+0000 7fa9f5d9b700 1 -- 172.21.15.99:0/709226177 learned_addr learned my addr 172.21.15.99:0/709226177 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:11.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.644+0000 7fa9fe59c700 1 --2- 172.21.15.99:0/709226177 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa00157870 0x7faa0015b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:11.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.644+0000 7fa9fed9d700 1 --2- 172.21.15.99:0/709226177 >> v2:172.21.15.191:3300/0 conn(0x7faa0015d580 0x7faa00158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:11.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.644+0000 7fa9f5d9b700 1 -- 172.21.15.99:0/709226177 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa00157870 msgr2=0x7faa0015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:11.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.644+0000 7fa9f5d9b700 1 --2- 172.21.15.99:0/709226177 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa00157870 0x7faa0015b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:11.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.644+0000 7fa9f5d9b700 1 -- 172.21.15.99:0/709226177 >> v2:172.21.15.191:3300/0 conn(0x7faa0015d580 msgr2=0x7faa00158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:11.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.644+0000 7fa9f5d9b700 1 --2- 172.21.15.99:0/709226177 >> v2:172.21.15.191:3300/0 conn(0x7faa0015d580 0x7faa00158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:11.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.644+0000 7fa9f5d9b700 1 -- 172.21.15.99:0/709226177 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa0015c560 con 0x7faa0015a620 2023-10-08T00:22:11.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.644+0000 7fa9fe59c700 1 --2- 172.21.15.99:0/709226177 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa00157870 0x7faa0015b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:22:11.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.644+0000 7fa9fed9d700 1 --2- 172.21.15.99:0/709226177 >> v2:172.21.15.191:3300/0 conn(0x7faa0015d580 0x7faa00158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:22:11.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.644+0000 7fa9f5d9b700 1 --2- 172.21.15.99:0/709226177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa0015a620 0x7faa0015d040 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7faa000b03b0 tx=0x7fa9e8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df517b44bfa56681 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:11.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.645+0000 7fa9fdd9b700 1 -- 172.21.15.99:0/709226177 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e800b860 con 0x7faa0015a620 2023-10-08T00:22:11.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.645+0000 7fa9fdd9b700 1 -- 172.21.15.99:0/709226177 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9e800b9c0 con 0x7faa0015a620 2023-10-08T00:22:11.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.645+0000 7fa9fdd9b700 1 -- 172.21.15.99:0/709226177 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e800bd60 con 0x7faa0015a620 2023-10-08T00:22:11.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.645+0000 7faa04b1c700 1 -- 172.21.15.99:0/709226177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa0015a620 msgr2=0x7faa0015d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:11.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.645+0000 7faa04b1c700 1 --2- 172.21.15.99:0/709226177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa0015a620 0x7faa0015d040 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7faa000b03b0 tx=0x7fa9e8009250 comp rx=0 tx=0).stop 2023-10-08T00:22:11.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.645+0000 7faa04b1c700 1 -- 172.21.15.99:0/709226177 shutdown_connections 2023-10-08T00:22:11.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.645+0000 7faa04b1c700 1 --2- 172.21.15.99:0/709226177 >> v2:172.21.15.191:3300/0 conn(0x7faa0015d580 0x7faa00158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:11.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.645+0000 7faa04b1c700 1 --2- 172.21.15.99:0/709226177 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa00157870 0x7faa0015b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:11.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.645+0000 7faa04b1c700 1 --2- 172.21.15.99:0/709226177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa0015a620 0x7faa0015d040 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:11.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.645+0000 7faa04b1c700 1 -- 172.21.15.99:0/709226177 >> 172.21.15.99:0/709226177 conn(0x7faa00068ce0 msgr2=0x7faa000690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:11.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.645+0000 7faa04b1c700 1 -- 172.21.15.99:0/709226177 shutdown_connections 2023-10-08T00:22:11.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.645+0000 7faa04b1c700 1 -- 172.21.15.99:0/709226177 wait complete. 2023-10-08T00:22:11.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.646+0000 7faa04b1c700 1 Processor -- start 2023-10-08T00:22:11.650 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.647+0000 7faa04b1c700 1 -- start start 2023-10-08T00:22:11.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.647+0000 7faa04b1c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa00157870 0x7faa0016b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:11.652 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.647+0000 7faa04b1c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa0015a620 0x7faa00155530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:11.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.647+0000 7faa04b1c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa0015d580 0x7faa00143380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:11.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.647+0000 7faa04b1c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa00155d80 con 0x7faa0015d580 2023-10-08T00:22:11.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.647+0000 7faa04b1c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa00154ad0 con 0x7faa00157870 2023-10-08T00:22:11.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.647+0000 7faa04b1c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa00154db0 con 0x7faa0015a620 2023-10-08T00:22:11.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.647+0000 7fa9fed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa0015d580 0x7faa00143380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:11.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.647+0000 7fa9fed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa0015d580 0x7faa00143380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33142/0 (socket says 172.21.15.99:33142) 2023-10-08T00:22:11.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.647+0000 7fa9fed9d700 1 -- 172.21.15.99:0/2535724568 learned_addr learned my addr 172.21.15.99:0/2535724568 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:11.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.647+0000 7fa9fe59c700 1 --2- 172.21.15.99:0/2535724568 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa00157870 0x7faa0016b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:11.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.647+0000 7fa9f5d9b700 1 --2- 172.21.15.99:0/2535724568 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa0015a620 0x7faa00155530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:11.653 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.648+0000 7fa9fed9d700 1 -- 172.21.15.99:0/2535724568 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa0015a620 msgr2=0x7faa00155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:11.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.648+0000 7fa9fed9d700 1 --2- 172.21.15.99:0/2535724568 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa0015a620 0x7faa00155530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:11.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.648+0000 7fa9fed9d700 1 -- 172.21.15.99:0/2535724568 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa00157870 msgr2=0x7faa0016b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:11.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.648+0000 7fa9fed9d700 1 --2- 172.21.15.99:0/2535724568 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa00157870 0x7faa0016b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:11.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.648+0000 7fa9fed9d700 1 -- 172.21.15.99:0/2535724568 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e80094f0 con 0x7faa0015d580 2023-10-08T00:22:11.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.648+0000 7fa9fed9d700 1 --2- 172.21.15.99:0/2535724568 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa0015d580 0x7faa00143380 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8007f10 tx=0x7fa9f8009930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:11.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.648+0000 7fa9f6ffd700 1 -- 172.21.15.99:0/2535724568 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f800db80 con 0x7faa0015d580 2023-10-08T00:22:11.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.648+0000 7fa9f6ffd700 1 -- 172.21.15.99:0/2535724568 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9f800f040 con 0x7faa0015d580 2023-10-08T00:22:11.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.648+0000 7faa04b1c700 1 -- 172.21.15.99:0/2535724568 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa00155090 con 0x7faa0015d580 2023-10-08T00:22:11.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.648+0000 7fa9f6ffd700 1 -- 172.21.15.99:0/2535724568 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8015400 con 0x7faa0015d580 2023-10-08T00:22:11.654 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.648+0000 7faa04b1c700 1 -- 172.21.15.99:0/2535724568 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa00155a70 con 0x7faa0015d580 2023-10-08T00:22:11.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.649+0000 7faa04b1c700 1 -- 172.21.15.99:0/2535724568 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa0004c3c0 con 0x7faa0015d580 2023-10-08T00:22:11.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.649+0000 7fa9f6ffd700 1 -- 172.21.15.99:0/2535724568 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa9f80157b0 con 0x7faa0015d580 2023-10-08T00:22:11.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.650+0000 7fa9f6ffd700 1 --2- 172.21.15.99:0/2535724568 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa9e0066f30 0x7fa9e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:11.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.650+0000 7fa9f6ffd700 1 -- 172.21.15.99:0/2535724568 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fa9f8084880 con 0x7faa0015d580 2023-10-08T00:22:11.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.650+0000 7fa9fe59c700 1 --2- 172.21.15.99:0/2535724568 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa9e0066f30 0x7fa9e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:11.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.651+0000 7fa9fe59c700 1 --2- 172.21.15.99:0/2535724568 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa9e0066f30 0x7fa9e00693e0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0006f00 tx=0x7fa9f0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:11.655 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.654+0000 7fa9f6ffd700 1 -- 172.21.15.99:0/2535724568 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa9f8051ea0 con 0x7faa0015d580 2023-10-08T00:22:11.910 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:11.910+0000 7faa04b1c700 1 -- 172.21.15.99:0/2535724568 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]} v 0) v1 -- 0x7faa0015c840 con 0x7faa0015d580 2023-10-08T00:22:12.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:12.212+0000 7fa9f6ffd700 1 -- 172.21.15.99:0/2535724568 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]}]=0 v529) v1 ==== 135+0+0 (secure 0 0 0) 0x7fa9f80554c0 con 0x7faa0015d580 2023-10-08T00:22:12.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:12.214+0000 7faa04b1c700 1 -- 172.21.15.99:0/2535724568 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa9e0066f30 msgr2=0x7fa9e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:12.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:12.214+0000 7faa04b1c700 1 --2- 172.21.15.99:0/2535724568 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa9e0066f30 0x7fa9e00693e0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0006f00 tx=0x7fa9f0008040 comp rx=0 tx=0).stop 2023-10-08T00:22:12.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:12.214+0000 7faa04b1c700 1 -- 172.21.15.99:0/2535724568 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa0015d580 msgr2=0x7faa00143380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:12.215 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:12.214+0000 7faa04b1c700 1 --2- 172.21.15.99:0/2535724568 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa0015d580 0x7faa00143380 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8007f10 tx=0x7fa9f8009930 comp rx=0 tx=0).stop 2023-10-08T00:22:12.216 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:12.214+0000 7faa04b1c700 1 -- 172.21.15.99:0/2535724568 shutdown_connections 2023-10-08T00:22:12.216 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:12.214+0000 7faa04b1c700 1 --2- 172.21.15.99:0/2535724568 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa9e0066f30 0x7fa9e00693e0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:12.216 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:12.214+0000 7faa04b1c700 1 --2- 172.21.15.99:0/2535724568 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa00157870 0x7faa0016b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:12.216 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:12.214+0000 7faa04b1c700 1 --2- 172.21.15.99:0/2535724568 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa0015d580 0x7faa00143380 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:12.216 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:12.214+0000 7faa04b1c700 1 --2- 172.21.15.99:0/2535724568 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa0015a620 0x7faa00155530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:12.216 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:12.214+0000 7faa04b1c700 1 -- 172.21.15.99:0/2535724568 >> 172.21.15.99:0/2535724568 conn(0x7faa00068ce0 msgr2=0x7faa0014c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:12.216 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:12.214+0000 7faa04b1c700 1 -- 172.21.15.99:0/2535724568 shutdown_connections 2023-10-08T00:22:12.217 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:12.214+0000 7faa04b1c700 1 -- 172.21.15.99:0/2535724568 wait complete. 2023-10-08T00:22:12.227 INFO:tasks.cephfs_test_runner:test_full_same_file (tasks.cephfs.test_full.TestQuotaFull) ... ok 2023-10-08T00:22:12.227 INFO:tasks.cephfs_test_runner: 2023-10-08T00:22:12.227 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-08T00:22:12.227 INFO:tasks.cephfs_test_runner:Ran 10 tests in 647.717s 2023-10-08T00:22:12.228 INFO:tasks.cephfs_test_runner: 2023-10-08T00:22:12.228 INFO:tasks.cephfs_test_runner:OK 2023-10-08T00:22:12.228 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-10-08T00:22:12.240 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-10-08T00:22:12.265 INFO:tasks.kclient:Unmounting kernel clients... 2023-10-08T00:22:12.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:22:12.266 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2023-10-08T00:22:12.281 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T00:22:12.281 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T00:22:12.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:22:12.282 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T00:22:12.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:22:12.366 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T00:22:12.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:22:12.417 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link delete ceph-brx 2023-10-08T00:22:12.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:22:12.473 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2023-10-08T00:22:12.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:22:12.488 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link delete ceph-brx 2023-10-08T00:22:12.556 INFO:teuthology.orchestra.run.smithi099.stderr:Cannot find device "ceph-brx" 2023-10-08T00:22:12.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T00:22:12.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:22:12.558 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2023-10-08T00:22:12.576 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-08T00:22:12.576 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-08T00:22:12.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:22:12.577 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-08T00:22:12.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:22:12.656 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-08T00:22:12.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:22:12.738 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link delete ceph-brx 2023-10-08T00:22:12.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:22:12.836 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2023-10-08T00:22:12.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:22:12.852 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link delete ceph-brx 2023-10-08T00:22:12.920 INFO:teuthology.orchestra.run.smithi191.stderr:Cannot find device "ceph-brx" 2023-10-08T00:22:12.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T00:22:12.921 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-10-08T00:22:12.931 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2023-10-08T00:22:12.931 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:22:13.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.045+0000 7f44e4630700 1 Processor -- start 2023-10-08T00:22:13.046 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.045+0000 7f44e4630700 1 -- start start 2023-10-08T00:22:13.048 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.046+0000 7f44e4630700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f44dc156570 0x7f44dc06d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:13.048 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.046+0000 7f44e4630700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f44dc157050 0x7f44dc06d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:13.048 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.046+0000 7f44e4630700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44dc157c30 con 0x7f44dc15a330 2023-10-08T00:22:13.048 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.046+0000 7f44e4630700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44dc1584f0 con 0x7f44dc157050 2023-10-08T00:22:13.049 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.046+0000 7f44e4630700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44dc157670 con 0x7f44dc156570 2023-10-08T00:22:13.049 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.046+0000 7f44e2bcd700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f44dc157050 0x7f44dc06d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:13.049 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.046+0000 7f44e23cc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f44dc156570 0x7f44dc06d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:13.049 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.046+0000 7f44e1bcb700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f44dc15a330 0x7f44dc156940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:35248/0 (socket says 172.21.15.99:35248) 2023-10-08T00:22:13.049 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.046+0000 7f44e1bcb700 1 -- 172.21.15.99:0/1552757823 learned_addr learned my addr 172.21.15.99:0/1552757823 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:22:13.049 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.047+0000 7f44e23cc700 1 -- 172.21.15.99:0/1552757823 >> v2:172.21.15.191:3300/0 conn(0x7f44dc157050 msgr2=0x7f44dc06d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:13.049 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.047+0000 7f44e23cc700 1 --2- 172.21.15.99:0/1552757823 >> v2:172.21.15.191:3300/0 conn(0x7f44dc157050 0x7f44dc06d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:13.049 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.047+0000 7f44e23cc700 1 -- 172.21.15.99:0/1552757823 >> v1:172.21.15.191:6789/0 conn(0x7f44dc15a330 legacy=0x7f44dc156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:13.049 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.047+0000 7f44e23cc700 1 -- 172.21.15.99:0/1552757823 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44dc157950 con 0x7f44dc156570 2023-10-08T00:22:13.049 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.047+0000 7f44e23cc700 1 --2- 172.21.15.99:0/1552757823 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f44dc156570 0x7f44dc06d380 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f44d8009e60 tx=0x7f44d8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12712f153fcff149 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:13.049 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.047+0000 7f44e13ca700 1 -- 172.21.15.99:0/1552757823 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d800e070 con 0x7f44dc156570 2023-10-08T00:22:13.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.047+0000 7f44e13ca700 1 -- 172.21.15.99:0/1552757823 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44d800bb60 con 0x7f44dc156570 2023-10-08T00:22:13.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.047+0000 7f44e13ca700 1 -- 172.21.15.99:0/1552757823 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d8012430 con 0x7f44dc156570 2023-10-08T00:22:13.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.047+0000 7f44e4630700 1 -- 172.21.15.99:0/1552757823 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f44dc156570 msgr2=0x7f44dc06d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:13.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.047+0000 7f44e4630700 1 --2- 172.21.15.99:0/1552757823 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f44dc156570 0x7f44dc06d380 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f44d8009e60 tx=0x7f44d8009780 comp rx=0 tx=0).stop 2023-10-08T00:22:13.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.048+0000 7f44e4630700 1 -- 172.21.15.99:0/1552757823 shutdown_connections 2023-10-08T00:22:13.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.048+0000 7f44e4630700 1 --2- 172.21.15.99:0/1552757823 >> v2:172.21.15.191:3300/0 conn(0x7f44dc157050 0x7f44dc06d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:13.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.048+0000 7f44e4630700 1 --2- 172.21.15.99:0/1552757823 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f44dc156570 0x7f44dc06d380 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:13.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.048+0000 7f44e4630700 1 -- 172.21.15.99:0/1552757823 >> 172.21.15.99:0/1552757823 conn(0x7f44dc067660 msgr2=0x7f44dc067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:13.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.048+0000 7f44e4630700 1 -- 172.21.15.99:0/1552757823 shutdown_connections 2023-10-08T00:22:13.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.048+0000 7f44e4630700 1 -- 172.21.15.99:0/1552757823 wait complete. 2023-10-08T00:22:13.053 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.049+0000 7f44e4630700 1 Processor -- start 2023-10-08T00:22:13.053 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.049+0000 7f44e4630700 1 -- start start 2023-10-08T00:22:13.053 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.050+0000 7f44e4630700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44dc156570 0x7f44dc0b1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:13.053 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.050+0000 7f44e4630700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f44dc157050 0x7f44dc0c5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:13.053 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.050+0000 7f44e4630700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f44dc15a330 0x7f44dc0af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:13.053 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.050+0000 7f44e4630700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44dc0c28b0 con 0x7f44dc156570 2023-10-08T00:22:13.053 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.050+0000 7f44e4630700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44dc0c1c30 con 0x7f44dc157050 2023-10-08T00:22:13.053 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.050+0000 7f44e4630700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44dc0c1f10 con 0x7f44dc15a330 2023-10-08T00:22:13.053 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.050+0000 7f44e23cc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44dc156570 0x7f44dc0b1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:13.054 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.050+0000 7f44e23cc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44dc156570 0x7f44dc0b1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33150/0 (socket says 172.21.15.99:33150) 2023-10-08T00:22:13.054 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.050+0000 7f44e23cc700 1 -- 172.21.15.99:0/1347680717 learned_addr learned my addr 172.21.15.99:0/1347680717 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:13.054 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.050+0000 7f44e1bcb700 1 --2- 172.21.15.99:0/1347680717 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f44dc157050 0x7f44dc0c5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:13.054 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.050+0000 7f44e2bcd700 1 --2- 172.21.15.99:0/1347680717 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f44dc15a330 0x7f44dc0af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:13.054 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.050+0000 7f44e23cc700 1 -- 172.21.15.99:0/1347680717 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f44dc15a330 msgr2=0x7f44dc0af6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:13.054 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.050+0000 7f44e23cc700 1 --2- 172.21.15.99:0/1347680717 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f44dc15a330 0x7f44dc0af6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:13.054 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.050+0000 7f44e23cc700 1 -- 172.21.15.99:0/1347680717 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f44dc157050 msgr2=0x7f44dc0c5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:13.054 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.050+0000 7f44e23cc700 1 --2- 172.21.15.99:0/1347680717 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f44dc157050 0x7f44dc0c5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:13.054 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.050+0000 7f44e23cc700 1 -- 172.21.15.99:0/1347680717 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d8009a20 con 0x7f44dc156570 2023-10-08T00:22:13.054 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.050+0000 7f44e1bcb700 1 --2- 172.21.15.99:0/1347680717 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f44dc157050 0x7f44dc0c5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:22:13.054 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.050+0000 7f44e23cc700 1 --2- 172.21.15.99:0/1347680717 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44dc156570 0x7f44dc0b1e60 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f44d800b6c0 tx=0x7f44d8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:13.054 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.050+0000 7f44d27fc700 1 -- 172.21.15.99:0/1347680717 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d8012c20 con 0x7f44dc156570 2023-10-08T00:22:13.055 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.050+0000 7f44d27fc700 1 -- 172.21.15.99:0/1347680717 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44d8012da0 con 0x7f44dc156570 2023-10-08T00:22:13.055 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.050+0000 7f44e4630700 1 -- 172.21.15.99:0/1347680717 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44dc0c21f0 con 0x7f44dc156570 2023-10-08T00:22:13.055 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.050+0000 7f44d27fc700 1 -- 172.21.15.99:0/1347680717 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d8026640 con 0x7f44dc156570 2023-10-08T00:22:13.055 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.050+0000 7f44e4630700 1 -- 172.21.15.99:0/1347680717 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44dc0b23a0 con 0x7f44dc156570 2023-10-08T00:22:13.055 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.051+0000 7f44e4630700 1 -- 172.21.15.99:0/1347680717 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44dc04c3c0 con 0x7f44dc156570 2023-10-08T00:22:13.055 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.052+0000 7f44d27fc700 1 -- 172.21.15.99:0/1347680717 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f44d8026a70 con 0x7f44dc156570 2023-10-08T00:22:13.055 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.052+0000 7f44d27fc700 1 --2- 172.21.15.99:0/1347680717 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f44c0066f30 0x7f44c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:13.055 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.052+0000 7f44d27fc700 1 -- 172.21.15.99:0/1347680717 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f44d8085f70 con 0x7f44dc156570 2023-10-08T00:22:13.055 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.052+0000 7f44e1bcb700 1 --2- 172.21.15.99:0/1347680717 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f44c0066f30 0x7f44c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:13.056 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.053+0000 7f44e1bcb700 1 --2- 172.21.15.99:0/1347680717 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f44c0066f30 0x7f44c00693e0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f44dc14e970 tx=0x7f44cc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:13.057 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.057+0000 7f44d27fc700 1 -- 172.21.15.99:0/1347680717 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f44d80b75a0 con 0x7f44dc156570 2023-10-08T00:22:13.243 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.243+0000 7f44e4630700 1 -- 172.21.15.99:0/1347680717 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f44dc0ca130 con 0x7f44dc156570 2023-10-08T00:22:13.244 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.243+0000 7f44d27fc700 1 -- 172.21.15.99:0/1347680717 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v192) v1 ==== 74+0+12276 (secure 0 0 0) 0x7f44d80534e0 con 0x7f44dc156570 2023-10-08T00:22:13.244 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:22:13.244 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":192,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:22:11.174008+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.191:6837/2466421758":"2023-10-09T00:22:08.745176+0000","172.21.15.191:6835/2466421758":"2023-10-09T00:22:08.745176+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:22:13.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.245+0000 7f44e4630700 1 -- 172.21.15.99:0/1347680717 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f44c0066f30 msgr2=0x7f44c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:13.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.245+0000 7f44e4630700 1 --2- 172.21.15.99:0/1347680717 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f44c0066f30 0x7f44c00693e0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f44dc14e970 tx=0x7f44cc009040 comp rx=0 tx=0).stop 2023-10-08T00:22:13.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.245+0000 7f44e4630700 1 -- 172.21.15.99:0/1347680717 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44dc156570 msgr2=0x7f44dc0b1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:13.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.245+0000 7f44e4630700 1 --2- 172.21.15.99:0/1347680717 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44dc156570 0x7f44dc0b1e60 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f44d800b6c0 tx=0x7f44d8019000 comp rx=0 tx=0).stop 2023-10-08T00:22:13.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.245+0000 7f44e4630700 1 -- 172.21.15.99:0/1347680717 shutdown_connections 2023-10-08T00:22:13.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.245+0000 7f44e4630700 1 --2- 172.21.15.99:0/1347680717 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f44c0066f30 0x7f44c00693e0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:13.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.245+0000 7f44e4630700 1 --2- 172.21.15.99:0/1347680717 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f44dc15a330 0x7f44dc0af6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:13.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.245+0000 7f44e4630700 1 --2- 172.21.15.99:0/1347680717 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44dc156570 0x7f44dc0b1e60 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:13.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.245+0000 7f44e4630700 1 --2- 172.21.15.99:0/1347680717 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f44dc157050 0x7f44dc0c5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:13.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.246+0000 7f44e4630700 1 -- 172.21.15.99:0/1347680717 >> 172.21.15.99:0/1347680717 conn(0x7f44dc067660 msgr2=0x7f44dc069ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:13.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.246+0000 7f44e4630700 1 -- 172.21.15.99:0/1347680717 shutdown_connections 2023-10-08T00:22:13.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.246+0000 7f44e4630700 1 -- 172.21.15.99:0/1347680717 wait complete. 2023-10-08T00:22:13.258 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-08T00:22:13.259 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T00:22:13.373 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.372+0000 7ff03e072700 1 Processor -- start 2023-10-08T00:22:13.373 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.372+0000 7ff03e072700 1 -- start start 2023-10-08T00:22:13.384 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.373+0000 7ff03e072700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff0380cd6e0 0x7ff0380cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:13.385 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.373+0000 7ff03e072700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff0380cdab0 0x7ff03815d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:13.385 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.373+0000 7ff03e072700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff03815dda0 0x7ff03815cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:13.385 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.373+0000 7ff03e072700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff03815e850 con 0x7ff0380cd6e0 2023-10-08T00:22:13.385 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.373+0000 7ff03e072700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0380cd030 con 0x7ff0380cdab0 2023-10-08T00:22:13.385 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.373+0000 7ff03e072700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0380cb7c0 con 0x7ff03815dda0 2023-10-08T00:22:13.385 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.373+0000 7ff03d070700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff0380cd6e0 0x7ff0380cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:13.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.373+0000 7ff03d070700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff0380cd6e0 0x7ff0380cbc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33228/0 (socket says 172.21.15.99:33228) 2023-10-08T00:22:13.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.373+0000 7ff03d070700 1 -- 172.21.15.99:0/1101387208 learned_addr learned my addr 172.21.15.99:0/1101387208 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:13.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.373+0000 7ff03d871700 1 --2- 172.21.15.99:0/1101387208 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff03815dda0 0x7ff03815cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:13.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.373+0000 7ff03c86f700 1 --2- 172.21.15.99:0/1101387208 >> v2:172.21.15.191:3300/0 conn(0x7ff0380cdab0 0x7ff03815d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:13.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.373+0000 7ff03d070700 1 -- 172.21.15.99:0/1101387208 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff03815dda0 msgr2=0x7ff03815cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:13.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.373+0000 7ff03d070700 1 --2- 172.21.15.99:0/1101387208 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff03815dda0 0x7ff03815cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:13.386 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.373+0000 7ff03d070700 1 -- 172.21.15.99:0/1101387208 >> v2:172.21.15.191:3300/0 conn(0x7ff0380cdab0 msgr2=0x7ff03815d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:13.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.373+0000 7ff03d070700 1 --2- 172.21.15.99:0/1101387208 >> v2:172.21.15.191:3300/0 conn(0x7ff0380cdab0 0x7ff03815d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:13.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.374+0000 7ff03d070700 1 -- 172.21.15.99:0/1101387208 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff03815e290 con 0x7ff0380cd6e0 2023-10-08T00:22:13.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.374+0000 7ff03d070700 1 --2- 172.21.15.99:0/1101387208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0380cd6e0 0x7ff0380cbc50 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7ff0380b6c70 tx=0x7ff034009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6075c6c2a6c2a6e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:13.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.374+0000 7ff02ffff700 1 -- 172.21.15.99:0/1101387208 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03400b860 con 0x7ff0380cd6e0 2023-10-08T00:22:13.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.374+0000 7ff02ffff700 1 -- 172.21.15.99:0/1101387208 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff03400b9c0 con 0x7ff0380cd6e0 2023-10-08T00:22:13.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.374+0000 7ff02ffff700 1 -- 172.21.15.99:0/1101387208 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03400bd60 con 0x7ff0380cd6e0 2023-10-08T00:22:13.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.374+0000 7ff03e072700 1 -- 172.21.15.99:0/1101387208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0380cd6e0 msgr2=0x7ff0380cbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:13.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.374+0000 7ff03e072700 1 --2- 172.21.15.99:0/1101387208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0380cd6e0 0x7ff0380cbc50 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7ff0380b6c70 tx=0x7ff034009250 comp rx=0 tx=0).stop 2023-10-08T00:22:13.387 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.374+0000 7ff03e072700 1 -- 172.21.15.99:0/1101387208 shutdown_connections 2023-10-08T00:22:13.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.374+0000 7ff03e072700 1 --2- 172.21.15.99:0/1101387208 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff03815dda0 0x7ff03815cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:13.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.374+0000 7ff03e072700 1 --2- 172.21.15.99:0/1101387208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0380cd6e0 0x7ff0380cbc50 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:13.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.374+0000 7ff03e072700 1 --2- 172.21.15.99:0/1101387208 >> v2:172.21.15.191:3300/0 conn(0x7ff0380cdab0 0x7ff03815d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:13.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.374+0000 7ff03e072700 1 -- 172.21.15.99:0/1101387208 >> 172.21.15.99:0/1101387208 conn(0x7ff038067660 msgr2=0x7ff038067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:13.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.374+0000 7ff03e072700 1 -- 172.21.15.99:0/1101387208 shutdown_connections 2023-10-08T00:22:13.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.374+0000 7ff03e072700 1 -- 172.21.15.99:0/1101387208 wait complete. 2023-10-08T00:22:13.388 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.375+0000 7ff03e072700 1 Processor -- start 2023-10-08T00:22:13.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.375+0000 7ff03e072700 1 -- start start 2023-10-08T00:22:13.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.376+0000 7ff03e072700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0380cd6e0 0x7ff0380c8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:13.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.376+0000 7ff03e072700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0380cdab0 0x7ff03815c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:13.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.376+0000 7ff03e072700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff03815dda0 0x7ff038159180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:13.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.376+0000 7ff03e072700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03815bb90 con 0x7ff0380cd6e0 2023-10-08T00:22:13.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.376+0000 7ff03e072700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03816b0b0 con 0x7ff03815dda0 2023-10-08T00:22:13.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.376+0000 7ff03e072700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff03815b260 con 0x7ff0380cdab0 2023-10-08T00:22:13.389 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.376+0000 7ff03d070700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0380cd6e0 0x7ff0380c8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:13.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.376+0000 7ff03d070700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0380cd6e0 0x7ff0380c8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33242/0 (socket says 172.21.15.99:33242) 2023-10-08T00:22:13.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.376+0000 7ff03d070700 1 -- 172.21.15.99:0/1253815994 learned_addr learned my addr 172.21.15.99:0/1253815994 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:13.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.377+0000 7ff03d070700 1 -- 172.21.15.99:0/1253815994 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0380cdab0 msgr2=0x7ff03815c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:13.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.377+0000 7ff03d070700 1 --2- 172.21.15.99:0/1253815994 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0380cdab0 0x7ff03815c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:13.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.377+0000 7ff03d070700 1 -- 172.21.15.99:0/1253815994 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff03815dda0 msgr2=0x7ff038159180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:13.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.377+0000 7ff03d871700 1 --2- 172.21.15.99:0/1253815994 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff03815dda0 0x7ff038159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:13.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.377+0000 7ff03d070700 1 --2- 172.21.15.99:0/1253815994 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff03815dda0 0x7ff038159180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:13.390 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.377+0000 7ff03d070700 1 -- 172.21.15.99:0/1253815994 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0340094f0 con 0x7ff0380cd6e0 2023-10-08T00:22:13.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.377+0000 7ff03d070700 1 --2- 172.21.15.99:0/1253815994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0380cd6e0 0x7ff0380c8d20 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7ff0380c7b10 tx=0x7ff034017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:13.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.377+0000 7ff02d7fa700 1 -- 172.21.15.99:0/1253815994 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff034024440 con 0x7ff0380cd6e0 2023-10-08T00:22:13.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.377+0000 7ff02d7fa700 1 -- 172.21.15.99:0/1253815994 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0340245a0 con 0x7ff0380cd6e0 2023-10-08T00:22:13.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.377+0000 7ff03e072700 1 -- 172.21.15.99:0/1253815994 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff03815b540 con 0x7ff0380cd6e0 2023-10-08T00:22:13.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.377+0000 7ff02d7fa700 1 -- 172.21.15.99:0/1253815994 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff034024960 con 0x7ff0380cd6e0 2023-10-08T00:22:13.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.377+0000 7ff03e072700 1 -- 172.21.15.99:0/1253815994 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff038155e00 con 0x7ff0380cd6e0 2023-10-08T00:22:13.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.378+0000 7ff03e072700 1 -- 172.21.15.99:0/1253815994 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff038089280 con 0x7ff0380cd6e0 2023-10-08T00:22:13.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.378+0000 7ff02d7fa700 1 -- 172.21.15.99:0/1253815994 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff03401a420 con 0x7ff0380cd6e0 2023-10-08T00:22:13.391 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.379+0000 7ff02d7fa700 1 --2- 172.21.15.99:0/1253815994 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff01c066f30 0x7ff01c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:13.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.379+0000 7ff02d7fa700 1 -- 172.21.15.99:0/1253815994 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 4939+0+0 (secure 0 0 0) 0x7ff034086050 con 0x7ff0380cd6e0 2023-10-08T00:22:13.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.379+0000 7ff03c86f700 1 --2- 172.21.15.99:0/1253815994 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff01c066f30 0x7ff01c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:13.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.380+0000 7ff03c86f700 1 --2- 172.21.15.99:0/1253815994 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff01c066f30 0x7ff01c0693e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff03806b560 tx=0x7ff028008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:13.392 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.383+0000 7ff02d7fa700 1 -- 172.21.15.99:0/1253815994 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff0340535c0 con 0x7ff0380cd6e0 2023-10-08T00:22:13.561 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.561+0000 7ff03e072700 1 -- 172.21.15.99:0/1253815994 --> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff03815bcd0 con 0x7ff01c066f30 2023-10-08T00:22:13.563 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.563+0000 7ff02d7fa700 1 -- 172.21.15.99:0/1253815994 <== mgr.4099 v2:172.21.15.191:6832/85985 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50119 (secure 0 0 0) 0x7ff028009640 con 0x7ff01c066f30 2023-10-08T00:22:13.564 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:22:13.566 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.565+0000 7ff03e072700 1 -- 172.21.15.99:0/1253815994 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff01c066f30 msgr2=0x7ff01c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:13.566 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.565+0000 7ff03e072700 1 --2- 172.21.15.99:0/1253815994 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff01c066f30 0x7ff01c0693e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff03806b560 tx=0x7ff028008040 comp rx=0 tx=0).stop 2023-10-08T00:22:13.566 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.565+0000 7ff03e072700 1 -- 172.21.15.99:0/1253815994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0380cd6e0 msgr2=0x7ff0380c8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:13.566 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.565+0000 7ff03e072700 1 --2- 172.21.15.99:0/1253815994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0380cd6e0 0x7ff0380c8d20 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7ff0380c7b10 tx=0x7ff034017000 comp rx=0 tx=0).stop 2023-10-08T00:22:13.567 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.565+0000 7ff03e072700 1 -- 172.21.15.99:0/1253815994 shutdown_connections 2023-10-08T00:22:13.567 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.565+0000 7ff03e072700 1 --2- 172.21.15.99:0/1253815994 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff01c066f30 0x7ff01c0693e0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:13.567 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.565+0000 7ff03e072700 1 --2- 172.21.15.99:0/1253815994 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff03815dda0 0x7ff038159180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:13.567 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.565+0000 7ff03e072700 1 --2- 172.21.15.99:0/1253815994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0380cd6e0 0x7ff0380c8d20 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:13.567 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.565+0000 7ff03e072700 1 --2- 172.21.15.99:0/1253815994 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0380cdab0 0x7ff03815c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:13.567 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.565+0000 7ff03e072700 1 -- 172.21.15.99:0/1253815994 >> 172.21.15.99:0/1253815994 conn(0x7ff038067660 msgr2=0x7ff038156970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:13.567 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.565+0000 7ff03e072700 1 -- 172.21.15.99:0/1253815994 shutdown_connections 2023-10-08T00:22:13.567 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.565+0000 7ff03e072700 1 -- 172.21.15.99:0/1253815994 wait complete. 2023-10-08T00:22:13.568 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2023-10-08T00:22:13.577 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":527,"stamp":"2023-10-08T00:22:12.185095+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":111982,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1450517,"statfs":{"total":1600000000,"available":1485329706,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-44025,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-97,"num_write_kb":-33860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001346"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":309,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-08T00:21:46.908404+0000","last_change":"2023-10-08T00:10:43.728490+0000","last_active":"2023-10-08T00:21:46.908404+0000","last_peered":"2023-10-08T00:21:46.908404+0000","last_clean":"2023-10-08T00:21:46.908404+0000","last_became_active":"2023-10-08T00:10:43.728341+0000","last_became_peered":"2023-10-08T00:10:43.728341+0000","last_unstale":"2023-10-08T00:21:46.908404+0000","last_undegraded":"2023-10-08T00:21:46.908404+0000","last_fullsized":"2023-10-08T00:21:46.908404+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_clean_scrub_stamp":"2023-10-08T00:10:42.716096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:06:31.035293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574987,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":5134,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":190177,"statfs":{"total":200000000,"available":194741811,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.53000000000000003,"15min":0.52000000000000002},"min":{"1min":0.377,"5min":0.27800000000000002,"15min":0.26900000000000002},"max":{"1min":0.67500000000000004,"5min":0.77600000000000002,"15min":1.129},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.54900000000000004,"15min":0.54500000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.73799999999999999,"5min":0.754,"15min":1.0269999999999999},"last":0.46200000000000002}]},{"osd":1,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.57199999999999995,"15min":0.55500000000000005},"min":{"1min":0.372,"5min":0.32300000000000001,"15min":0.32000000000000001},"max":{"1min":0.70399999999999996,"5min":0.74299999999999999,"15min":0.82999999999999996},"last":0.504},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.59799999999999998,"15min":0.58099999999999996},"min":{"1min":0.40500000000000003,"5min":0.39900000000000002,"15min":0.38200000000000001},"max":{"1min":0.746,"5min":0.79400000000000004,"15min":0.80100000000000005},"last":0.495}]},{"osd":2,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56699999999999995,"15min":0.55100000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.34200000000000003},"max":{"1min":0.748,"5min":0.76400000000000001,"15min":0.79900000000000004},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.54900000000000004,"15min":0.53800000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.72599999999999998,"5min":0.748,"15min":0.78500000000000003},"last":0.40400000000000003}]},{"osd":3,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.54000000000000004},"min":{"1min":0.42999999999999999,"5min":0.36699999999999999,"15min":0.33700000000000002},"max":{"1min":0.753,"5min":0.77400000000000002,"15min":0.79300000000000004},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57099999999999995,"15min":0.56499999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.75800000000000001,"5min":0.77600000000000002,"15min":0.82099999999999995},"last":0.51300000000000001}]},{"osd":4,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.46600000000000003,"15min":0.496},"min":{"1min":0.36599999999999999,"5min":0.30399999999999999,"15min":0.26400000000000001},"max":{"1min":0.66800000000000004,"5min":0.71499999999999997,"15min":0.755},"last":0.376},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.51300000000000001,"15min":0.51800000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.41599999999999998}]},{"osd":5,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.503,"15min":0.52500000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.73499999999999999,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.54100000000000004,"15min":0.56000000000000005},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.76500000000000001,"5min":0.80600000000000005,"15min":0.82799999999999996},"last":0.435}]},{"osd":6,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.41499999999999998,"5min":0.39600000000000002,"15min":0.33100000000000002},"max":{"1min":0.80400000000000005,"5min":0.84799999999999998,"15min":0.85399999999999998},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.58299999999999996,"15min":0.60399999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.27800000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.91500000000000004},"last":0.54300000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574987,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":17423,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":177889,"statfs":{"total":200000000,"available":182158663,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53100000000000003,"15min":0.50600000000000001},"min":{"1min":0.42399999999999999,"5min":0.32300000000000001,"15min":0.29199999999999998},"max":{"1min":0.74299999999999999,"5min":0.76300000000000001,"15min":1.012},"last":0.495},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.53500000000000003,"15min":0.60699999999999998},"min":{"1min":0.38400000000000001,"5min":0.317,"15min":0.29199999999999998},"max":{"1min":0.65800000000000003,"5min":0.77800000000000002,"15min":5.1230000000000002},"last":0.52300000000000002}]},{"osd":1,"last update":"Sun Oct 8 00:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.55700000000000005,"15min":0.54000000000000004},"min":{"1min":0.36499999999999999,"5min":0.29899999999999999,"15min":0.28899999999999998},"max":{"1min":0.754,"5min":0.78100000000000003,"15min":1.1419999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.59399999999999997,"15min":0.57699999999999996},"min":{"1min":0.51300000000000001,"5min":0.41099999999999998,"15min":0.32100000000000001},"max":{"1min":0.81000000000000005,"5min":0.81799999999999995,"15min":1.1539999999999999},"last":0.57599999999999996}]},{"osd":2,"last update":"Sun Oct 8 00:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.55400000000000005,"15min":0.54200000000000004},"min":{"1min":0.47199999999999998,"5min":0.36299999999999999,"15min":0.32900000000000001},"max":{"1min":0.74099999999999999,"5min":0.76500000000000001,"15min":1.3149999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58299999999999996,"15min":0.57599999999999996},"min":{"1min":0.441,"5min":0.374,"15min":0.29399999999999998},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":1.3280000000000001},"last":0.626}]},{"osd":3,"last update":"Sun Oct 8 00:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.56100000000000005,"15min":0.55400000000000005},"min":{"1min":0.35499999999999998,"5min":0.32000000000000001,"15min":0.25700000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":1.046},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.58099999999999996,"15min":0.56100000000000005},"min":{"1min":0.501,"5min":0.38600000000000001,"15min":0.32900000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":1.4390000000000001},"last":0.64000000000000001}]},{"osd":4,"last update":"Sun Oct 8 00:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46200000000000002,"15min":0.48099999999999998},"min":{"1min":0.30499999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.622,"5min":0.70099999999999996,"15min":0.80100000000000005},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55000000000000004,"15min":0.55700000000000005},"min":{"1min":0.41999999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.77600000000000002,"5min":0.79500000000000004,"15min":0.89100000000000001},"last":0.66600000000000004}]},{"osd":5,"last update":"Sun Oct 8 00:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.47699999999999998,"15min":0.48299999999999998},"min":{"1min":0.36099999999999999,"5min":0.29299999999999998,"15min":0.29099999999999998},"max":{"1min":0.73499999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.505,"15min":0.53200000000000003},"min":{"1min":0.35299999999999998,"5min":0.313,"15min":0.28699999999999998},"max":{"1min":0.70099999999999996,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.65400000000000003}]},{"osd":7,"last update":"Sun Oct 8 00:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.50600000000000001,"15min":0.52600000000000002},"min":{"1min":0.38100000000000001,"5min":0.32800000000000001,"15min":0.307},"max":{"1min":0.753,"5min":0.82899999999999996,"15min":0.91500000000000004},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53200000000000003,"15min":0.54200000000000004},"min":{"1min":0.39000000000000001,"5min":0.34899999999999998,"15min":0.32400000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.79900000000000004},"last":0.68000000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574987,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":22153,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":173158,"statfs":{"total":200000000,"available":177314578,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.35999999999999999,"15min":0.33300000000000002},"min":{"1min":0.223,"5min":0.216,"15min":0.19500000000000001},"max":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.41999999999999998,"15min":0.39500000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.67300000000000004},"last":0.34999999999999998}]},{"osd":2,"last update":"Sun Oct 8 00:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.38200000000000001,"15min":0.35499999999999998},"min":{"1min":0.27400000000000002,"5min":0.25900000000000001,"15min":0.22600000000000001},"max":{"1min":0.56999999999999995,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.313},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39500000000000002,"15min":0.36899999999999999},"min":{"1min":0.28499999999999998,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.53400000000000003,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.32500000000000001}]},{"osd":3,"last update":"Sun Oct 8 00:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.42499999999999999,"15min":0.39300000000000002},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.58799999999999997,"5min":0.63,"15min":0.70999999999999996},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.375,"15min":0.35599999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.222},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.29899999999999999}]},{"osd":4,"last update":"Sun Oct 8 00:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57699999999999996,"15min":0.55600000000000005},"min":{"1min":0.42999999999999999,"5min":0.34999999999999998,"15min":0.32600000000000001},"max":{"1min":0.82199999999999995,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59899999999999998,"15min":0.57999999999999996},"min":{"1min":0.42199999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.76000000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.41999999999999998}]},{"osd":5,"last update":"Sun Oct 8 00:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57799999999999996,"15min":0.55900000000000005},"min":{"1min":0.39900000000000002,"5min":0.35099999999999998,"15min":0.33500000000000002},"max":{"1min":0.71799999999999997,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60299999999999998,"15min":0.58699999999999997},"min":{"1min":0.41799999999999998,"5min":0.39500000000000002,"15min":0.30399999999999999},"max":{"1min":0.76900000000000002,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.48699999999999999}]},{"osd":6,"last update":"Sun Oct 8 00:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.58299999999999996,"15min":0.57599999999999996},"min":{"1min":0.41499999999999998,"5min":0.37,"15min":0.317},"max":{"1min":0.755,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59299999999999997,"15min":0.57599999999999996},"min":{"1min":0.40799999999999997,"5min":0.377,"15min":0.33200000000000002},"max":{"1min":0.748,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.42799999999999999}]},{"osd":7,"last update":"Sun Oct 8 00:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58699999999999997,"15min":0.57599999999999996},"min":{"1min":0.439,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.746,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.63200000000000001,"15min":0.61099999999999999},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.39100000000000001},"max":{"1min":0.79400000000000004,"5min":0.872,"15min":0.96699999999999997},"last":0.54800000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574987,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":19471,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":175841,"statfs":{"total":200000000,"available":180061677,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 00:21:06 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.33800000000000002,"15min":0.33500000000000002},"min":{"1min":0.23400000000000001,"5min":0.20399999999999999,"15min":0.193},"max":{"1min":0.45600000000000002,"5min":0.54900000000000004,"15min":1.0640000000000001},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.36199999999999999,"15min":0.35399999999999998},"min":{"1min":0.251,"5min":0.216,"15min":0.20300000000000001},"max":{"1min":0.433,"5min":0.66500000000000004,"15min":0.87},"last":0.26700000000000002}]},{"osd":2,"last update":"Sun Oct 8 00:21:06 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.38500000000000001,"15min":0.38700000000000001},"min":{"1min":0.29399999999999998,"5min":0.249,"15min":0.24099999999999999},"max":{"1min":0.53200000000000003,"5min":0.57199999999999995,"15min":0.93799999999999994},"last":0.378},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.35899999999999999,"15min":0.56100000000000005},"min":{"1min":0.25600000000000001,"5min":0.23000000000000001,"15min":0.185},"max":{"1min":0.54800000000000004,"5min":0.59899999999999998,"15min":12.019},"last":0.309}]},{"osd":3,"last update":"Sun Oct 8 00:21:06 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.38200000000000001,"15min":0.371},"min":{"1min":0.25700000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.56200000000000006,"5min":0.628,"15min":1.1020000000000001},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.41599999999999998,"15min":0.40699999999999997},"min":{"1min":0.31900000000000001,"5min":0.26700000000000002,"15min":0.246},"max":{"1min":0.51200000000000001,"5min":0.66100000000000003,"15min":1.0409999999999999},"last":0.36099999999999999}]},{"osd":4,"last update":"Sun Oct 8 00:21:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.61899999999999999,"15min":0.61699999999999999},"min":{"1min":0.39500000000000002,"5min":0.377,"15min":0.35899999999999999},"max":{"1min":0.872,"5min":0.89700000000000002,"15min":1.1779999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.621,"15min":0.624},"min":{"1min":0.44900000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89300000000000002,"5min":0.93899999999999995,"15min":0.96199999999999997},"last":0.66500000000000004}]},{"osd":5,"last update":"Sun Oct 8 00:21:06 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.58799999999999997,"15min":0.79400000000000004},"min":{"1min":0.42399999999999999,"5min":0.378,"15min":0.32000000000000001},"max":{"1min":0.878,"5min":0.90700000000000003,"15min":11.992000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.60599999999999998,"15min":0.59599999999999997},"min":{"1min":0.40999999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.80600000000000005,"5min":0.91600000000000004,"15min":1.1299999999999999},"last":0.58799999999999997}]},{"osd":6,"last update":"Sun Oct 8 00:21:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60099999999999998,"15min":0.59399999999999997},"min":{"1min":0.38300000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.85699999999999998,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.60899999999999999,"15min":0.79300000000000004},"min":{"1min":0.42399999999999999,"5min":0.39600000000000002,"15min":0.38100000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":11.927},"last":0.67600000000000005}]},{"osd":7,"last update":"Sun Oct 8 00:21:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.60099999999999998,"15min":0.80300000000000005},"min":{"1min":0.39900000000000002,"5min":0.38600000000000001,"15min":0.35699999999999998},"max":{"1min":0.84399999999999997,"5min":0.95099999999999996,"15min":11.965},"last":0.621},{"interface":"front","average":{"1min":0.622,"5min":0.63500000000000001,"15min":0.628},"min":{"1min":0.45800000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":1.1539999999999999},"last":0.60799999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574987,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":5135,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":190176,"statfs":{"total":200000000,"available":194741055,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.32900000000000001,"15min":0.35799999999999998},"min":{"1min":0.20100000000000001,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.47599999999999998,"5min":0.56899999999999995,"15min":2.5139999999999998},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.34699999999999998,"15min":0.36899999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.20599999999999999},"max":{"1min":0.51200000000000001,"5min":0.59099999999999997,"15min":2.1309999999999998},"last":0.29599999999999999}]},{"osd":1,"last update":"Sun Oct 8 00:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.35499999999999998,"15min":0.37},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":2.2639999999999998},"last":0.312},{"interface":"front","average":{"1min":0.374,"5min":0.38400000000000001,"15min":0.40300000000000002},"min":{"1min":0.254,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.54400000000000004,"5min":0.57199999999999995,"15min":2.5379999999999998},"last":0.35699999999999998}]},{"osd":3,"last update":"Sun Oct 8 00:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.40400000000000003,"15min":0.41099999999999998},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.64400000000000002,"5min":0.66900000000000004,"15min":2.2890000000000001},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.41899999999999998,"15min":0.435},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.55900000000000005,"5min":0.64300000000000002,"15min":2.5619999999999998},"last":0.38200000000000001}]},{"osd":4,"last update":"Sun Oct 8 00:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54400000000000004,"15min":0.57999999999999996},"min":{"1min":0.31,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.77300000000000002,"5min":0.80800000000000005,"15min":2.488},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58599999999999997,"15min":0.59599999999999997},"min":{"1min":0.42999999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.77300000000000002,"5min":0.92400000000000004,"15min":2.073},"last":0.45900000000000002}]},{"osd":5,"last update":"Sun Oct 8 00:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.60899999999999999,"15min":0.63},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.76300000000000001,"5min":0.94299999999999995,"15min":2.2069999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.61699999999999999,"15min":0.64000000000000001},"min":{"1min":0.42199999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77300000000000002,"5min":0.91000000000000003,"15min":2.456},"last":0.439}]},{"osd":6,"last update":"Sun Oct 8 00:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59599999999999997,"15min":0.60699999999999998},"min":{"1min":0.39100000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.80200000000000005,"5min":0.872,"15min":2.0339999999999998},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59399999999999997,"15min":0.60499999999999998},"min":{"1min":0.40799999999999997,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.75,"5min":0.94099999999999995,"15min":2.1699999999999999},"last":0.48399999999999999}]},{"osd":7,"last update":"Sun Oct 8 00:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60099999999999998,"15min":0.61899999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82199999999999995,"5min":0.91500000000000004,"15min":2.1030000000000002},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60699999999999998,"15min":0.61799999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.77700000000000002,"5min":1.0169999999999999,"15min":2.2349999999999999},"last":0.39400000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574987,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":11337,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":183975,"statfs":{"total":200000000,"available":188390866,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.33600000000000002,"15min":0.34699999999999998},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.192},"max":{"1min":0.58899999999999997,"5min":0.59299999999999997,"15min":1.46},"last":0.318},{"interface":"front","average":{"1min":0.31900000000000001,"5min":0.34999999999999998,"15min":0.34200000000000003},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.192},"max":{"1min":0.56000000000000005,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.34300000000000003}]},{"osd":1,"last update":"Sun Oct 8 00:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.36299999999999999,"15min":0.55500000000000005},"min":{"1min":0.25,"5min":0.247,"15min":0.20000000000000001},"max":{"1min":0.52700000000000002,"5min":0.57499999999999996,"15min":12.351000000000001},"last":0.221},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38700000000000001,"15min":0.372},"min":{"1min":0.23200000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.54500000000000004,"5min":0.58199999999999996,"15min":0.71999999999999997},"last":0.38}]},{"osd":2,"last update":"Sun Oct 8 00:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.40799999999999997,"15min":0.40600000000000003},"min":{"1min":0.248,"5min":0.248,"15min":0.23499999999999999},"max":{"1min":0.58099999999999996,"5min":0.59699999999999998,"15min":1.5489999999999999},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.41999999999999998,"15min":0.39800000000000002},"min":{"1min":0.27100000000000002,"5min":0.25700000000000001,"15min":0.246},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.753},"last":0.35799999999999998}]},{"osd":4,"last update":"Sun Oct 8 00:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.625,"15min":0.78400000000000003},"min":{"1min":0.39800000000000002,"5min":0.36899999999999999,"15min":0.34899999999999998},"max":{"1min":0.81399999999999995,"5min":0.96699999999999997,"15min":11.333},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.63400000000000001,"15min":0.80400000000000005},"min":{"1min":0.39100000000000001,"5min":0.38100000000000001,"15min":0.36099999999999999},"max":{"1min":0.89500000000000002,"5min":0.97699999999999998,"15min":12.321999999999999},"last":0.67900000000000005}]},{"osd":5,"last update":"Sun Oct 8 00:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.61199999999999999,"15min":0.79800000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35499999999999998},"max":{"1min":0.86299999999999999,"5min":0.93500000000000005,"15min":12.291},"last":0.628},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.64400000000000002,"15min":0.621},"min":{"1min":0.44800000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80500000000000005,"5min":1.143,"15min":1.143},"last":0.63800000000000001}]},{"osd":6,"last update":"Sun Oct 8 00:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.59899999999999998,"15min":0.755},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.33700000000000002},"max":{"1min":0.91200000000000003,"5min":0.91500000000000004,"15min":11.288},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.62,"15min":0.79100000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.36599999999999999},"max":{"1min":0.84599999999999997,"5min":0.94599999999999995,"15min":12.253},"last":0.65700000000000003}]},{"osd":7,"last update":"Sun Oct 8 00:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63200000000000001,"15min":0.60299999999999998},"min":{"1min":0.45700000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86399999999999999,"5min":1.0920000000000001,"15min":1.3109999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61599999999999999,"15min":0.78400000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.38600000000000001},"max":{"1min":0.879,"5min":0.89300000000000002,"15min":11.363},"last":0.58299999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574987,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9230,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":186081,"statfs":{"total":200000000,"available":190547473,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.56699999999999995,"15min":0.56100000000000005},"min":{"1min":0.36299999999999999,"5min":0.32300000000000001,"15min":0.193},"max":{"1min":0.80900000000000005,"5min":0.84799999999999998,"15min":0.93000000000000005},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.60099999999999998,"15min":0.58699999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.30399999999999999},"max":{"1min":0.70199999999999996,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.41999999999999998}]},{"osd":1,"last update":"Sun Oct 8 00:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.54300000000000004,"15min":0.52900000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.33700000000000002},"max":{"1min":0.749,"5min":0.751,"15min":0.78100000000000003},"last":0.373},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57299999999999995,"15min":0.54600000000000004},"min":{"1min":0.46899999999999997,"5min":0.34300000000000003,"15min":0.30199999999999999},"max":{"1min":0.71799999999999997,"5min":0.80600000000000005,"15min":0.81299999999999994},"last":0.34399999999999997}]},{"osd":2,"last update":"Sun Oct 8 00:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.54800000000000004,"15min":0.52700000000000002},"min":{"1min":0.35999999999999999,"5min":0.32500000000000001,"15min":0.32100000000000001},"max":{"1min":0.78400000000000003,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.56499999999999995,"15min":0.53400000000000003},"min":{"1min":0.41599999999999998,"5min":0.35399999999999998,"15min":0.32600000000000001},"max":{"1min":0.65300000000000002,"5min":0.79600000000000004,"15min":0.81699999999999995},"last":0.436}]},{"osd":3,"last update":"Sun Oct 8 00:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.57999999999999996,"15min":0.55000000000000004},"min":{"1min":0.41099999999999998,"5min":0.35199999999999998,"15min":0.28799999999999998},"max":{"1min":0.77100000000000002,"5min":0.82299999999999995,"15min":0.85399999999999998},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.55200000000000005,"15min":0.54000000000000004},"min":{"1min":0.433,"5min":0.37,"15min":0.36199999999999999},"max":{"1min":0.68400000000000005,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.29499999999999998}]},{"osd":5,"last update":"Sun Oct 8 00:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.47299999999999998,"15min":0.47899999999999998},"min":{"1min":0.33900000000000002,"5min":0.314,"15min":0.27500000000000002},"max":{"1min":0.68600000000000005,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.501,"15min":0.504},"min":{"1min":0.34100000000000003,"5min":0.317,"15min":0.24399999999999999},"max":{"1min":0.72399999999999998,"5min":0.79100000000000004,"15min":0.94899999999999995},"last":0.42699999999999999}]},{"osd":6,"last update":"Sun Oct 8 00:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.48899999999999999,"15min":0.48599999999999999},"min":{"1min":0.371,"5min":0.32400000000000001,"15min":0.26400000000000001},"max":{"1min":0.70899999999999996,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52500000000000002,"15min":0.53700000000000003},"min":{"1min":0.36299999999999999,"5min":0.34799999999999998,"15min":0.29799999999999999},"max":{"1min":0.73699999999999999,"5min":0.76200000000000001,"15min":0.80100000000000005},"last":0.312}]},{"osd":7,"last update":"Sun Oct 8 00:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.53900000000000003,"15min":0.54300000000000004},"min":{"1min":0.34999999999999998,"5min":0.33200000000000002,"15min":0.27900000000000003},"max":{"1min":0.76200000000000001,"5min":0.83599999999999997,"15min":0.90100000000000002},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56999999999999995,"15min":0.57399999999999995},"min":{"1min":0.42999999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.751,"5min":0.88,"15min":0.88},"last":0.40999999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574987,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":22096,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":173216,"statfs":{"total":200000000,"available":177373583,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.54400000000000004,"15min":0.50600000000000001},"min":{"1min":0.38,"5min":0.36399999999999999,"15min":0.249},"max":{"1min":0.71599999999999997,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54100000000000004,"15min":0.54800000000000004},"min":{"1min":0.39000000000000001,"5min":0.29799999999999999,"15min":0.29399999999999998},"max":{"1min":0.75700000000000001,"5min":0.81699999999999995,"15min":0.88600000000000001},"last":0.57999999999999996}]},{"osd":1,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.55600000000000005,"15min":0.53600000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.32500000000000001},"max":{"1min":0.68300000000000005,"5min":0.72599999999999998,"15min":0.82999999999999996},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.53800000000000003,"15min":0.52500000000000002},"min":{"1min":0.34599999999999997,"5min":0.34499999999999997,"15min":0.33000000000000002},"max":{"1min":0.68700000000000006,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.59899999999999998}]},{"osd":2,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55700000000000005,"15min":0.53700000000000003},"min":{"1min":0.42999999999999999,"5min":0.39600000000000002,"15min":0.32400000000000001},"max":{"1min":0.71599999999999997,"5min":0.749,"15min":0.86199999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56100000000000005,"15min":0.56000000000000005},"min":{"1min":0.38100000000000001,"5min":0.33400000000000002,"15min":0.29599999999999999},"max":{"1min":0.749,"5min":0.80000000000000004,"15min":0.95399999999999996},"last":0.63600000000000001}]},{"osd":3,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57299999999999995,"15min":0.55700000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.316},"max":{"1min":0.71999999999999997,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56599999999999995,"15min":0.56899999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.72399999999999998,"5min":0.73499999999999999,"15min":1.0529999999999999},"last":0.56699999999999995}]},{"osd":4,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.47299999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.27700000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.504,"5min":0.51400000000000001,"15min":0.51700000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.28899999999999998},"max":{"1min":0.747,"5min":0.77500000000000002,"15min":0.91500000000000004},"last":0.45800000000000002}]},{"osd":6,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.33600000000000002,"5min":0.28199999999999997,"15min":0.27400000000000002},"max":{"1min":0.79400000000000004,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.443},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52300000000000002,"15min":0.51100000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.30499999999999999},"max":{"1min":0.81499999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.54800000000000004}]},{"osd":7,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.54200000000000004,"15min":0.53100000000000003},"min":{"1min":0.33500000000000002,"5min":0.318,"15min":0.29399999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81999999999999995},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56899999999999995,"15min":0.56200000000000006},"min":{"1min":0.38300000000000001,"5min":0.35599999999999998,"15min":0.32500000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.49099999999999999}]}]}],"pool_statfs":[]}} 2023-10-08T00:22:13.580 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T00:22:13.694 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.693+0000 7efc42d58700 1 Processor -- start 2023-10-08T00:22:13.694 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.693+0000 7efc42d58700 1 -- start start 2023-10-08T00:22:13.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.694+0000 7efc42d58700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7efc3c06ab80 0x7efc3c06e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:13.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.694+0000 7efc42d58700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc3c15a3e0 con 0x7efc3c0cd820 2023-10-08T00:22:13.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.694+0000 7efc42d58700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc3c0ae840 con 0x7efc3c06f310 2023-10-08T00:22:13.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.694+0000 7efc42d58700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc3c0adf10 con 0x7efc3c06ab80 2023-10-08T00:22:13.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.694+0000 7efc41555700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7efc3c0cd820 0x7efc3c06ec00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:37324/0 (socket says 172.21.15.99:37324) 2023-10-08T00:22:13.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.694+0000 7efc41555700 1 -- 172.21.15.99:0/543066384 learned_addr learned my addr 172.21.15.99:0/543066384 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:22:13.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.694+0000 7efc41d56700 1 --2- 172.21.15.99:0/543066384 >> v2:172.21.15.191:3300/0 conn(0x7efc3c06ab80 0x7efc3c06e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:13.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.694+0000 7efc40d54700 1 -- 172.21.15.99:0/543066384 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2334007818 0 0) 0x7efc3c15a3e0 con 0x7efc3c0cd820 2023-10-08T00:22:13.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.694+0000 7efc40d54700 1 -- 172.21.15.99:0/543066384 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc28003570 con 0x7efc3c0cd820 2023-10-08T00:22:13.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.695+0000 7efc40d54700 1 -- 172.21.15.99:0/543066384 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3992249369 0 0) 0x7efc28003570 con 0x7efc3c0cd820 2023-10-08T00:22:13.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.695+0000 7efc40d54700 1 -- 172.21.15.99:0/543066384 >> v2:172.21.15.191:3300/0 conn(0x7efc3c06ab80 msgr2=0x7efc3c06e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:13.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.695+0000 7efc40d54700 1 --2- 172.21.15.99:0/543066384 >> v2:172.21.15.191:3300/0 conn(0x7efc3c06ab80 0x7efc3c06e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:13.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.695+0000 7efc40d54700 1 -- 172.21.15.99:0/543066384 >> v1:172.21.15.191:6789/0 conn(0x7efc3c06f310 legacy=0x7efc3c0a7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:13.699 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.695+0000 7efc40d54700 1 -- 172.21.15.99:0/543066384 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc3c0ae1f0 con 0x7efc3c0cd820 2023-10-08T00:22:13.699 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.695+0000 7efc40d54700 1 -- 172.21.15.99:0/543066384 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7efc2c001470 con 0x7efc3c0cd820 2023-10-08T00:22:13.699 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.695+0000 7efc40d54700 1 -- 172.21.15.99:0/543066384 >> v1:172.21.15.99:6789/0 conn(0x7efc3c0cd820 legacy=0x7efc3c06ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:13.699 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.695+0000 7efc40d54700 1 --2- 172.21.15.99:0/543066384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc28003a60 0x7efc28003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:13.699 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.695+0000 7efc40d54700 1 --2- 172.21.15.99:0/543066384 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc28004810 0x7efc28004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:13.699 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.695+0000 7efc40d54700 1 --2- 172.21.15.99:0/543066384 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efc280051c0 0x7efc2800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:13.699 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.695+0000 7efc40d54700 1 -- 172.21.15.99:0/543066384 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c0adf10 con 0x7efc28003a60 2023-10-08T00:22:13.699 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.695+0000 7efc40d54700 1 -- 172.21.15.99:0/543066384 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2800c050 con 0x7efc280051c0 2023-10-08T00:22:13.699 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.695+0000 7efc40d54700 1 -- 172.21.15.99:0/543066384 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc2800c210 con 0x7efc28004810 2023-10-08T00:22:13.700 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.695+0000 7efc41d56700 1 --2- 172.21.15.99:0/543066384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc28003a60 0x7efc28003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:13.700 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.695+0000 7efc42557700 1 --2- 172.21.15.99:0/543066384 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efc280051c0 0x7efc2800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:13.700 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.695+0000 7efc41555700 1 --2- 172.21.15.99:0/543066384 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc28004810 0x7efc28004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:13.700 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.695+0000 7efc41d56700 1 -- 172.21.15.99:0/543066384 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc28004810 msgr2=0x7efc28004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:13.700 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.695+0000 7efc41d56700 1 --2- 172.21.15.99:0/543066384 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc28004810 0x7efc28004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:13.700 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.695+0000 7efc41d56700 1 -- 172.21.15.99:0/543066384 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efc280051c0 msgr2=0x7efc2800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:13.700 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.695+0000 7efc41d56700 1 --2- 172.21.15.99:0/543066384 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efc280051c0 0x7efc2800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:13.700 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.695+0000 7efc41d56700 1 -- 172.21.15.99:0/543066384 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc28003570 con 0x7efc28003a60 2023-10-08T00:22:13.700 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.695+0000 7efc42557700 1 -- 172.21.15.99:0/543066384 reap_dead start 2023-10-08T00:22:13.701 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.695+0000 7efc41d56700 1 --2- 172.21.15.99:0/543066384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc28003a60 0x7efc28003e30 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7efc38009c00 tx=0x7efc380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e5100feb6f1c53 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:13.701 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.695+0000 7efc40d54700 1 -- 172.21.15.99:0/543066384 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3800e710 con 0x7efc28003a60 2023-10-08T00:22:13.701 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.695+0000 7efc40d54700 1 -- 172.21.15.99:0/543066384 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc3800e870 con 0x7efc28003a60 2023-10-08T00:22:13.701 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.695+0000 7efc40d54700 1 -- 172.21.15.99:0/543066384 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3800ec30 con 0x7efc28003a60 2023-10-08T00:22:13.701 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.696+0000 7efc42d58700 1 -- 172.21.15.99:0/543066384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc28003a60 msgr2=0x7efc28003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:13.701 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.696+0000 7efc42d58700 1 --2- 172.21.15.99:0/543066384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc28003a60 0x7efc28003e30 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7efc38009c00 tx=0x7efc380091a0 comp rx=0 tx=0).stop 2023-10-08T00:22:13.701 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.696+0000 7efc42d58700 1 -- 172.21.15.99:0/543066384 shutdown_connections 2023-10-08T00:22:13.701 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.696+0000 7efc42d58700 1 --2- 172.21.15.99:0/543066384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc28003a60 0x7efc28003e30 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:13.701 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.696+0000 7efc42d58700 1 -- 172.21.15.99:0/543066384 >> 172.21.15.99:0/543066384 conn(0x7efc3c068130 msgr2=0x7efc3c0cd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:13.701 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.696+0000 7efc42d58700 1 -- 172.21.15.99:0/543066384 shutdown_connections 2023-10-08T00:22:13.701 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.696+0000 7efc42d58700 1 -- 172.21.15.99:0/543066384 wait complete. 2023-10-08T00:22:13.704 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.697+0000 7efc42d58700 1 Processor -- start 2023-10-08T00:22:13.704 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.697+0000 7efc42d58700 1 -- start start 2023-10-08T00:22:13.704 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.698+0000 7efc42d58700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efc28003a60 0x7efc3c09a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:13.704 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.698+0000 7efc42d58700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc3c097a00 0x7efc3c098f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:13.704 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.698+0000 7efc42d58700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc3c099470 0x7efc3c16d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:13.704 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.698+0000 7efc42d58700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c098aa0 con 0x7efc3c099470 2023-10-08T00:22:13.704 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.698+0000 7efc42d58700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c097e90 con 0x7efc28003a60 2023-10-08T00:22:13.704 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.698+0000 7efc42d58700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c098170 con 0x7efc3c097a00 2023-10-08T00:22:13.705 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.698+0000 7efc42557700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc3c099470 0x7efc3c16d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:13.705 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.698+0000 7efc42557700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc3c099470 0x7efc3c16d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33266/0 (socket says 172.21.15.99:33266) 2023-10-08T00:22:13.705 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.698+0000 7efc42557700 1 -- 172.21.15.99:0/3696961413 learned_addr learned my addr 172.21.15.99:0/3696961413 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:13.705 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.698+0000 7efc41555700 1 --2- 172.21.15.99:0/3696961413 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc3c097a00 0x7efc3c098f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:13.705 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.698+0000 7efc41d56700 1 --2- 172.21.15.99:0/3696961413 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efc28003a60 0x7efc3c09a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:13.705 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.698+0000 7efc42557700 1 -- 172.21.15.99:0/3696961413 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc3c097a00 msgr2=0x7efc3c098f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:13.705 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.698+0000 7efc42557700 1 --2- 172.21.15.99:0/3696961413 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc3c097a00 0x7efc3c098f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:13.705 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.698+0000 7efc42557700 1 -- 172.21.15.99:0/3696961413 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efc28003a60 msgr2=0x7efc3c09a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:13.705 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.698+0000 7efc42557700 1 --2- 172.21.15.99:0/3696961413 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efc28003a60 0x7efc3c09a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:13.705 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.698+0000 7efc42557700 1 -- 172.21.15.99:0/3696961413 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc3c098450 con 0x7efc3c099470 2023-10-08T00:22:13.705 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.698+0000 7efc42557700 1 --2- 172.21.15.99:0/3696961413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc3c099470 0x7efc3c16d630 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7efc3c0c9ae0 tx=0x7efc34001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:13.706 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.698+0000 7efc31ffb700 1 -- 172.21.15.99:0/3696961413 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc34006e80 con 0x7efc3c099470 2023-10-08T00:22:13.706 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.698+0000 7efc31ffb700 1 -- 172.21.15.99:0/3696961413 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc34014430 con 0x7efc3c099470 2023-10-08T00:22:13.706 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.698+0000 7efc42d58700 1 -- 172.21.15.99:0/3696961413 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc3c098730 con 0x7efc3c099470 2023-10-08T00:22:13.706 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.699+0000 7efc31ffb700 1 -- 172.21.15.99:0/3696961413 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc340147f0 con 0x7efc3c099470 2023-10-08T00:22:13.706 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.699+0000 7efc42d58700 1 -- 172.21.15.99:0/3696961413 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc3c09aa40 con 0x7efc3c099470 2023-10-08T00:22:13.706 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.700+0000 7efc42d58700 1 -- 172.21.15.99:0/3696961413 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc3c098c50 con 0x7efc3c099470 2023-10-08T00:22:13.706 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.700+0000 7efc31ffb700 1 -- 172.21.15.99:0/3696961413 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efc34014be0 con 0x7efc3c099470 2023-10-08T00:22:13.706 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.700+0000 7efc31ffb700 1 --2- 172.21.15.99:0/3696961413 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efc20066f30 0x7efc200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:13.706 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.701+0000 7efc31ffb700 1 -- 172.21.15.99:0/3696961413 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 4939+0+0 (secure 0 0 0) 0x7efc340838d0 con 0x7efc3c099470 2023-10-08T00:22:13.706 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.701+0000 7efc41d56700 1 --2- 172.21.15.99:0/3696961413 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efc20066f30 0x7efc200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:13.706 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.701+0000 7efc41d56700 1 --2- 172.21.15.99:0/3696961413 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efc20066f30 0x7efc200693e0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7efc380091e0 tx=0x7efc3801a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:13.707 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.705+0000 7efc31ffb700 1 -- 172.21.15.99:0/3696961413 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efc3404c5a0 con 0x7efc3c099470 2023-10-08T00:22:13.882 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.882+0000 7efc42d58700 1 -- 172.21.15.99:0/3696961413 --> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efc3c09ac60 con 0x7efc20066f30 2023-10-08T00:22:13.885 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.884+0000 7efc31ffb700 1 -- 172.21.15.99:0/3696961413 <== mgr.4099 v2:172.21.15.191:6832/85985 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50119 (secure 0 0 0) 0x7efc38023040 con 0x7efc20066f30 2023-10-08T00:22:13.885 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:22:13.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.886+0000 7efc42d58700 1 -- 172.21.15.99:0/3696961413 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efc20066f30 msgr2=0x7efc200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:13.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.886+0000 7efc42d58700 1 --2- 172.21.15.99:0/3696961413 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efc20066f30 0x7efc200693e0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7efc380091e0 tx=0x7efc3801a000 comp rx=0 tx=0).stop 2023-10-08T00:22:13.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.886+0000 7efc42d58700 1 -- 172.21.15.99:0/3696961413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc3c099470 msgr2=0x7efc3c16d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:13.887 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.886+0000 7efc42d58700 1 --2- 172.21.15.99:0/3696961413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc3c099470 0x7efc3c16d630 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7efc3c0c9ae0 tx=0x7efc34001640 comp rx=0 tx=0).stop 2023-10-08T00:22:13.888 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.887+0000 7efc42d58700 1 -- 172.21.15.99:0/3696961413 shutdown_connections 2023-10-08T00:22:13.888 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.887+0000 7efc42d58700 1 --2- 172.21.15.99:0/3696961413 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efc20066f30 0x7efc200693e0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:13.888 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.887+0000 7efc42d58700 1 --2- 172.21.15.99:0/3696961413 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efc28003a60 0x7efc3c09a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:13.888 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.887+0000 7efc42d58700 1 --2- 172.21.15.99:0/3696961413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc3c099470 0x7efc3c16d630 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:13.888 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.887+0000 7efc42d58700 1 --2- 172.21.15.99:0/3696961413 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc3c097a00 0x7efc3c098f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:13.888 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.887+0000 7efc42d58700 1 -- 172.21.15.99:0/3696961413 >> 172.21.15.99:0/3696961413 conn(0x7efc3c068130 msgr2=0x7efc3c0cb710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:13.888 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.887+0000 7efc42d58700 1 -- 172.21.15.99:0/3696961413 shutdown_connections 2023-10-08T00:22:13.888 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:13.887+0000 7efc42d58700 1 -- 172.21.15.99:0/3696961413 wait complete. 2023-10-08T00:22:13.889 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2023-10-08T00:22:13.899 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":527,"stamp":"2023-10-08T00:22:12.185095+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":111982,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1450517,"statfs":{"total":1600000000,"available":1485329706,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-44025,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-97,"num_write_kb":-33860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001346"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":309,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-08T00:21:46.908404+0000","last_change":"2023-10-08T00:10:43.728490+0000","last_active":"2023-10-08T00:21:46.908404+0000","last_peered":"2023-10-08T00:21:46.908404+0000","last_clean":"2023-10-08T00:21:46.908404+0000","last_became_active":"2023-10-08T00:10:43.728341+0000","last_became_peered":"2023-10-08T00:10:43.728341+0000","last_unstale":"2023-10-08T00:21:46.908404+0000","last_undegraded":"2023-10-08T00:21:46.908404+0000","last_fullsized":"2023-10-08T00:21:46.908404+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_clean_scrub_stamp":"2023-10-08T00:10:42.716096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:06:31.035293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574987,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":5134,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":190177,"statfs":{"total":200000000,"available":194741811,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.53000000000000003,"15min":0.52000000000000002},"min":{"1min":0.377,"5min":0.27800000000000002,"15min":0.26900000000000002},"max":{"1min":0.67500000000000004,"5min":0.77600000000000002,"15min":1.129},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.54900000000000004,"15min":0.54500000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.73799999999999999,"5min":0.754,"15min":1.0269999999999999},"last":0.46200000000000002}]},{"osd":1,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.57199999999999995,"15min":0.55500000000000005},"min":{"1min":0.372,"5min":0.32300000000000001,"15min":0.32000000000000001},"max":{"1min":0.70399999999999996,"5min":0.74299999999999999,"15min":0.82999999999999996},"last":0.504},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.59799999999999998,"15min":0.58099999999999996},"min":{"1min":0.40500000000000003,"5min":0.39900000000000002,"15min":0.38200000000000001},"max":{"1min":0.746,"5min":0.79400000000000004,"15min":0.80100000000000005},"last":0.495}]},{"osd":2,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56699999999999995,"15min":0.55100000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.34200000000000003},"max":{"1min":0.748,"5min":0.76400000000000001,"15min":0.79900000000000004},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.54900000000000004,"15min":0.53800000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.72599999999999998,"5min":0.748,"15min":0.78500000000000003},"last":0.40400000000000003}]},{"osd":3,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.54000000000000004},"min":{"1min":0.42999999999999999,"5min":0.36699999999999999,"15min":0.33700000000000002},"max":{"1min":0.753,"5min":0.77400000000000002,"15min":0.79300000000000004},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57099999999999995,"15min":0.56499999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.75800000000000001,"5min":0.77600000000000002,"15min":0.82099999999999995},"last":0.51300000000000001}]},{"osd":4,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.46600000000000003,"15min":0.496},"min":{"1min":0.36599999999999999,"5min":0.30399999999999999,"15min":0.26400000000000001},"max":{"1min":0.66800000000000004,"5min":0.71499999999999997,"15min":0.755},"last":0.376},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.51300000000000001,"15min":0.51800000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.41599999999999998}]},{"osd":5,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.503,"15min":0.52500000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.73499999999999999,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.54100000000000004,"15min":0.56000000000000005},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.76500000000000001,"5min":0.80600000000000005,"15min":0.82799999999999996},"last":0.435}]},{"osd":6,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.41499999999999998,"5min":0.39600000000000002,"15min":0.33100000000000002},"max":{"1min":0.80400000000000005,"5min":0.84799999999999998,"15min":0.85399999999999998},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.58299999999999996,"15min":0.60399999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.27800000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.91500000000000004},"last":0.54300000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574987,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":17423,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":177889,"statfs":{"total":200000000,"available":182158663,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53100000000000003,"15min":0.50600000000000001},"min":{"1min":0.42399999999999999,"5min":0.32300000000000001,"15min":0.29199999999999998},"max":{"1min":0.74299999999999999,"5min":0.76300000000000001,"15min":1.012},"last":0.495},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.53500000000000003,"15min":0.60699999999999998},"min":{"1min":0.38400000000000001,"5min":0.317,"15min":0.29199999999999998},"max":{"1min":0.65800000000000003,"5min":0.77800000000000002,"15min":5.1230000000000002},"last":0.52300000000000002}]},{"osd":1,"last update":"Sun Oct 8 00:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.55700000000000005,"15min":0.54000000000000004},"min":{"1min":0.36499999999999999,"5min":0.29899999999999999,"15min":0.28899999999999998},"max":{"1min":0.754,"5min":0.78100000000000003,"15min":1.1419999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.59399999999999997,"15min":0.57699999999999996},"min":{"1min":0.51300000000000001,"5min":0.41099999999999998,"15min":0.32100000000000001},"max":{"1min":0.81000000000000005,"5min":0.81799999999999995,"15min":1.1539999999999999},"last":0.57599999999999996}]},{"osd":2,"last update":"Sun Oct 8 00:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.55400000000000005,"15min":0.54200000000000004},"min":{"1min":0.47199999999999998,"5min":0.36299999999999999,"15min":0.32900000000000001},"max":{"1min":0.74099999999999999,"5min":0.76500000000000001,"15min":1.3149999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58299999999999996,"15min":0.57599999999999996},"min":{"1min":0.441,"5min":0.374,"15min":0.29399999999999998},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":1.3280000000000001},"last":0.626}]},{"osd":3,"last update":"Sun Oct 8 00:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.56100000000000005,"15min":0.55400000000000005},"min":{"1min":0.35499999999999998,"5min":0.32000000000000001,"15min":0.25700000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":1.046},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.58099999999999996,"15min":0.56100000000000005},"min":{"1min":0.501,"5min":0.38600000000000001,"15min":0.32900000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":1.4390000000000001},"last":0.64000000000000001}]},{"osd":4,"last update":"Sun Oct 8 00:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46200000000000002,"15min":0.48099999999999998},"min":{"1min":0.30499999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.622,"5min":0.70099999999999996,"15min":0.80100000000000005},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55000000000000004,"15min":0.55700000000000005},"min":{"1min":0.41999999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.77600000000000002,"5min":0.79500000000000004,"15min":0.89100000000000001},"last":0.66600000000000004}]},{"osd":5,"last update":"Sun Oct 8 00:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.47699999999999998,"15min":0.48299999999999998},"min":{"1min":0.36099999999999999,"5min":0.29299999999999998,"15min":0.29099999999999998},"max":{"1min":0.73499999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.505,"15min":0.53200000000000003},"min":{"1min":0.35299999999999998,"5min":0.313,"15min":0.28699999999999998},"max":{"1min":0.70099999999999996,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.65400000000000003}]},{"osd":7,"last update":"Sun Oct 8 00:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.50600000000000001,"15min":0.52600000000000002},"min":{"1min":0.38100000000000001,"5min":0.32800000000000001,"15min":0.307},"max":{"1min":0.753,"5min":0.82899999999999996,"15min":0.91500000000000004},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53200000000000003,"15min":0.54200000000000004},"min":{"1min":0.39000000000000001,"5min":0.34899999999999998,"15min":0.32400000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.79900000000000004},"last":0.68000000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574987,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":22153,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":173158,"statfs":{"total":200000000,"available":177314578,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.35999999999999999,"15min":0.33300000000000002},"min":{"1min":0.223,"5min":0.216,"15min":0.19500000000000001},"max":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.41999999999999998,"15min":0.39500000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.67300000000000004},"last":0.34999999999999998}]},{"osd":2,"last update":"Sun Oct 8 00:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.38200000000000001,"15min":0.35499999999999998},"min":{"1min":0.27400000000000002,"5min":0.25900000000000001,"15min":0.22600000000000001},"max":{"1min":0.56999999999999995,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.313},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39500000000000002,"15min":0.36899999999999999},"min":{"1min":0.28499999999999998,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.53400000000000003,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.32500000000000001}]},{"osd":3,"last update":"Sun Oct 8 00:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.42499999999999999,"15min":0.39300000000000002},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.58799999999999997,"5min":0.63,"15min":0.70999999999999996},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.375,"15min":0.35599999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.222},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.29899999999999999}]},{"osd":4,"last update":"Sun Oct 8 00:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57699999999999996,"15min":0.55600000000000005},"min":{"1min":0.42999999999999999,"5min":0.34999999999999998,"15min":0.32600000000000001},"max":{"1min":0.82199999999999995,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59899999999999998,"15min":0.57999999999999996},"min":{"1min":0.42199999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.76000000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.41999999999999998}]},{"osd":5,"last update":"Sun Oct 8 00:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57799999999999996,"15min":0.55900000000000005},"min":{"1min":0.39900000000000002,"5min":0.35099999999999998,"15min":0.33500000000000002},"max":{"1min":0.71799999999999997,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60299999999999998,"15min":0.58699999999999997},"min":{"1min":0.41799999999999998,"5min":0.39500000000000002,"15min":0.30399999999999999},"max":{"1min":0.76900000000000002,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.48699999999999999}]},{"osd":6,"last update":"Sun Oct 8 00:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.58299999999999996,"15min":0.57599999999999996},"min":{"1min":0.41499999999999998,"5min":0.37,"15min":0.317},"max":{"1min":0.755,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59299999999999997,"15min":0.57599999999999996},"min":{"1min":0.40799999999999997,"5min":0.377,"15min":0.33200000000000002},"max":{"1min":0.748,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.42799999999999999}]},{"osd":7,"last update":"Sun Oct 8 00:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58699999999999997,"15min":0.57599999999999996},"min":{"1min":0.439,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.746,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.63200000000000001,"15min":0.61099999999999999},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.39100000000000001},"max":{"1min":0.79400000000000004,"5min":0.872,"15min":0.96699999999999997},"last":0.54800000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574987,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":19471,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":175841,"statfs":{"total":200000000,"available":180061677,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 00:21:06 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.33800000000000002,"15min":0.33500000000000002},"min":{"1min":0.23400000000000001,"5min":0.20399999999999999,"15min":0.193},"max":{"1min":0.45600000000000002,"5min":0.54900000000000004,"15min":1.0640000000000001},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.36199999999999999,"15min":0.35399999999999998},"min":{"1min":0.251,"5min":0.216,"15min":0.20300000000000001},"max":{"1min":0.433,"5min":0.66500000000000004,"15min":0.87},"last":0.26700000000000002}]},{"osd":2,"last update":"Sun Oct 8 00:21:06 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.38500000000000001,"15min":0.38700000000000001},"min":{"1min":0.29399999999999998,"5min":0.249,"15min":0.24099999999999999},"max":{"1min":0.53200000000000003,"5min":0.57199999999999995,"15min":0.93799999999999994},"last":0.378},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.35899999999999999,"15min":0.56100000000000005},"min":{"1min":0.25600000000000001,"5min":0.23000000000000001,"15min":0.185},"max":{"1min":0.54800000000000004,"5min":0.59899999999999998,"15min":12.019},"last":0.309}]},{"osd":3,"last update":"Sun Oct 8 00:21:06 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.38200000000000001,"15min":0.371},"min":{"1min":0.25700000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.56200000000000006,"5min":0.628,"15min":1.1020000000000001},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.41599999999999998,"15min":0.40699999999999997},"min":{"1min":0.31900000000000001,"5min":0.26700000000000002,"15min":0.246},"max":{"1min":0.51200000000000001,"5min":0.66100000000000003,"15min":1.0409999999999999},"last":0.36099999999999999}]},{"osd":4,"last update":"Sun Oct 8 00:21:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.61899999999999999,"15min":0.61699999999999999},"min":{"1min":0.39500000000000002,"5min":0.377,"15min":0.35899999999999999},"max":{"1min":0.872,"5min":0.89700000000000002,"15min":1.1779999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.621,"15min":0.624},"min":{"1min":0.44900000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89300000000000002,"5min":0.93899999999999995,"15min":0.96199999999999997},"last":0.66500000000000004}]},{"osd":5,"last update":"Sun Oct 8 00:21:06 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.58799999999999997,"15min":0.79400000000000004},"min":{"1min":0.42399999999999999,"5min":0.378,"15min":0.32000000000000001},"max":{"1min":0.878,"5min":0.90700000000000003,"15min":11.992000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.60599999999999998,"15min":0.59599999999999997},"min":{"1min":0.40999999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.80600000000000005,"5min":0.91600000000000004,"15min":1.1299999999999999},"last":0.58799999999999997}]},{"osd":6,"last update":"Sun Oct 8 00:21:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60099999999999998,"15min":0.59399999999999997},"min":{"1min":0.38300000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.85699999999999998,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.60899999999999999,"15min":0.79300000000000004},"min":{"1min":0.42399999999999999,"5min":0.39600000000000002,"15min":0.38100000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":11.927},"last":0.67600000000000005}]},{"osd":7,"last update":"Sun Oct 8 00:21:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.60099999999999998,"15min":0.80300000000000005},"min":{"1min":0.39900000000000002,"5min":0.38600000000000001,"15min":0.35699999999999998},"max":{"1min":0.84399999999999997,"5min":0.95099999999999996,"15min":11.965},"last":0.621},{"interface":"front","average":{"1min":0.622,"5min":0.63500000000000001,"15min":0.628},"min":{"1min":0.45800000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":1.1539999999999999},"last":0.60799999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574987,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":5135,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":190176,"statfs":{"total":200000000,"available":194741055,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.32900000000000001,"15min":0.35799999999999998},"min":{"1min":0.20100000000000001,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.47599999999999998,"5min":0.56899999999999995,"15min":2.5139999999999998},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.34699999999999998,"15min":0.36899999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.20599999999999999},"max":{"1min":0.51200000000000001,"5min":0.59099999999999997,"15min":2.1309999999999998},"last":0.29599999999999999}]},{"osd":1,"last update":"Sun Oct 8 00:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.35499999999999998,"15min":0.37},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":2.2639999999999998},"last":0.312},{"interface":"front","average":{"1min":0.374,"5min":0.38400000000000001,"15min":0.40300000000000002},"min":{"1min":0.254,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.54400000000000004,"5min":0.57199999999999995,"15min":2.5379999999999998},"last":0.35699999999999998}]},{"osd":3,"last update":"Sun Oct 8 00:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.40400000000000003,"15min":0.41099999999999998},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.64400000000000002,"5min":0.66900000000000004,"15min":2.2890000000000001},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.41899999999999998,"15min":0.435},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.55900000000000005,"5min":0.64300000000000002,"15min":2.5619999999999998},"last":0.38200000000000001}]},{"osd":4,"last update":"Sun Oct 8 00:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54400000000000004,"15min":0.57999999999999996},"min":{"1min":0.31,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.77300000000000002,"5min":0.80800000000000005,"15min":2.488},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58599999999999997,"15min":0.59599999999999997},"min":{"1min":0.42999999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.77300000000000002,"5min":0.92400000000000004,"15min":2.073},"last":0.45900000000000002}]},{"osd":5,"last update":"Sun Oct 8 00:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.60899999999999999,"15min":0.63},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.76300000000000001,"5min":0.94299999999999995,"15min":2.2069999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.61699999999999999,"15min":0.64000000000000001},"min":{"1min":0.42199999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77300000000000002,"5min":0.91000000000000003,"15min":2.456},"last":0.439}]},{"osd":6,"last update":"Sun Oct 8 00:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59599999999999997,"15min":0.60699999999999998},"min":{"1min":0.39100000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.80200000000000005,"5min":0.872,"15min":2.0339999999999998},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59399999999999997,"15min":0.60499999999999998},"min":{"1min":0.40799999999999997,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.75,"5min":0.94099999999999995,"15min":2.1699999999999999},"last":0.48399999999999999}]},{"osd":7,"last update":"Sun Oct 8 00:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60099999999999998,"15min":0.61899999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82199999999999995,"5min":0.91500000000000004,"15min":2.1030000000000002},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60699999999999998,"15min":0.61799999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.77700000000000002,"5min":1.0169999999999999,"15min":2.2349999999999999},"last":0.39400000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574987,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":11337,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":183975,"statfs":{"total":200000000,"available":188390866,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.33600000000000002,"15min":0.34699999999999998},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.192},"max":{"1min":0.58899999999999997,"5min":0.59299999999999997,"15min":1.46},"last":0.318},{"interface":"front","average":{"1min":0.31900000000000001,"5min":0.34999999999999998,"15min":0.34200000000000003},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.192},"max":{"1min":0.56000000000000005,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.34300000000000003}]},{"osd":1,"last update":"Sun Oct 8 00:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.36299999999999999,"15min":0.55500000000000005},"min":{"1min":0.25,"5min":0.247,"15min":0.20000000000000001},"max":{"1min":0.52700000000000002,"5min":0.57499999999999996,"15min":12.351000000000001},"last":0.221},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38700000000000001,"15min":0.372},"min":{"1min":0.23200000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.54500000000000004,"5min":0.58199999999999996,"15min":0.71999999999999997},"last":0.38}]},{"osd":2,"last update":"Sun Oct 8 00:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.40799999999999997,"15min":0.40600000000000003},"min":{"1min":0.248,"5min":0.248,"15min":0.23499999999999999},"max":{"1min":0.58099999999999996,"5min":0.59699999999999998,"15min":1.5489999999999999},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.41999999999999998,"15min":0.39800000000000002},"min":{"1min":0.27100000000000002,"5min":0.25700000000000001,"15min":0.246},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.753},"last":0.35799999999999998}]},{"osd":4,"last update":"Sun Oct 8 00:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.625,"15min":0.78400000000000003},"min":{"1min":0.39800000000000002,"5min":0.36899999999999999,"15min":0.34899999999999998},"max":{"1min":0.81399999999999995,"5min":0.96699999999999997,"15min":11.333},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.63400000000000001,"15min":0.80400000000000005},"min":{"1min":0.39100000000000001,"5min":0.38100000000000001,"15min":0.36099999999999999},"max":{"1min":0.89500000000000002,"5min":0.97699999999999998,"15min":12.321999999999999},"last":0.67900000000000005}]},{"osd":5,"last update":"Sun Oct 8 00:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.61199999999999999,"15min":0.79800000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35499999999999998},"max":{"1min":0.86299999999999999,"5min":0.93500000000000005,"15min":12.291},"last":0.628},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.64400000000000002,"15min":0.621},"min":{"1min":0.44800000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80500000000000005,"5min":1.143,"15min":1.143},"last":0.63800000000000001}]},{"osd":6,"last update":"Sun Oct 8 00:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.59899999999999998,"15min":0.755},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.33700000000000002},"max":{"1min":0.91200000000000003,"5min":0.91500000000000004,"15min":11.288},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.62,"15min":0.79100000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.36599999999999999},"max":{"1min":0.84599999999999997,"5min":0.94599999999999995,"15min":12.253},"last":0.65700000000000003}]},{"osd":7,"last update":"Sun Oct 8 00:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63200000000000001,"15min":0.60299999999999998},"min":{"1min":0.45700000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86399999999999999,"5min":1.0920000000000001,"15min":1.3109999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61599999999999999,"15min":0.78400000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.38600000000000001},"max":{"1min":0.879,"5min":0.89300000000000002,"15min":11.363},"last":0.58299999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574987,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9230,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":186081,"statfs":{"total":200000000,"available":190547473,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.56699999999999995,"15min":0.56100000000000005},"min":{"1min":0.36299999999999999,"5min":0.32300000000000001,"15min":0.193},"max":{"1min":0.80900000000000005,"5min":0.84799999999999998,"15min":0.93000000000000005},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.60099999999999998,"15min":0.58699999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.30399999999999999},"max":{"1min":0.70199999999999996,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.41999999999999998}]},{"osd":1,"last update":"Sun Oct 8 00:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.54300000000000004,"15min":0.52900000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.33700000000000002},"max":{"1min":0.749,"5min":0.751,"15min":0.78100000000000003},"last":0.373},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57299999999999995,"15min":0.54600000000000004},"min":{"1min":0.46899999999999997,"5min":0.34300000000000003,"15min":0.30199999999999999},"max":{"1min":0.71799999999999997,"5min":0.80600000000000005,"15min":0.81299999999999994},"last":0.34399999999999997}]},{"osd":2,"last update":"Sun Oct 8 00:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.54800000000000004,"15min":0.52700000000000002},"min":{"1min":0.35999999999999999,"5min":0.32500000000000001,"15min":0.32100000000000001},"max":{"1min":0.78400000000000003,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.56499999999999995,"15min":0.53400000000000003},"min":{"1min":0.41599999999999998,"5min":0.35399999999999998,"15min":0.32600000000000001},"max":{"1min":0.65300000000000002,"5min":0.79600000000000004,"15min":0.81699999999999995},"last":0.436}]},{"osd":3,"last update":"Sun Oct 8 00:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.57999999999999996,"15min":0.55000000000000004},"min":{"1min":0.41099999999999998,"5min":0.35199999999999998,"15min":0.28799999999999998},"max":{"1min":0.77100000000000002,"5min":0.82299999999999995,"15min":0.85399999999999998},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.55200000000000005,"15min":0.54000000000000004},"min":{"1min":0.433,"5min":0.37,"15min":0.36199999999999999},"max":{"1min":0.68400000000000005,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.29499999999999998}]},{"osd":5,"last update":"Sun Oct 8 00:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.47299999999999998,"15min":0.47899999999999998},"min":{"1min":0.33900000000000002,"5min":0.314,"15min":0.27500000000000002},"max":{"1min":0.68600000000000005,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.501,"15min":0.504},"min":{"1min":0.34100000000000003,"5min":0.317,"15min":0.24399999999999999},"max":{"1min":0.72399999999999998,"5min":0.79100000000000004,"15min":0.94899999999999995},"last":0.42699999999999999}]},{"osd":6,"last update":"Sun Oct 8 00:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.48899999999999999,"15min":0.48599999999999999},"min":{"1min":0.371,"5min":0.32400000000000001,"15min":0.26400000000000001},"max":{"1min":0.70899999999999996,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52500000000000002,"15min":0.53700000000000003},"min":{"1min":0.36299999999999999,"5min":0.34799999999999998,"15min":0.29799999999999999},"max":{"1min":0.73699999999999999,"5min":0.76200000000000001,"15min":0.80100000000000005},"last":0.312}]},{"osd":7,"last update":"Sun Oct 8 00:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.53900000000000003,"15min":0.54300000000000004},"min":{"1min":0.34999999999999998,"5min":0.33200000000000002,"15min":0.27900000000000003},"max":{"1min":0.76200000000000001,"5min":0.83599999999999997,"15min":0.90100000000000002},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56999999999999995,"15min":0.57399999999999995},"min":{"1min":0.42999999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.751,"5min":0.88,"15min":0.88},"last":0.40999999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574987,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":22096,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":173216,"statfs":{"total":200000000,"available":177373583,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.54400000000000004,"15min":0.50600000000000001},"min":{"1min":0.38,"5min":0.36399999999999999,"15min":0.249},"max":{"1min":0.71599999999999997,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54100000000000004,"15min":0.54800000000000004},"min":{"1min":0.39000000000000001,"5min":0.29799999999999999,"15min":0.29399999999999998},"max":{"1min":0.75700000000000001,"5min":0.81699999999999995,"15min":0.88600000000000001},"last":0.57999999999999996}]},{"osd":1,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.55600000000000005,"15min":0.53600000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.32500000000000001},"max":{"1min":0.68300000000000005,"5min":0.72599999999999998,"15min":0.82999999999999996},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.53800000000000003,"15min":0.52500000000000002},"min":{"1min":0.34599999999999997,"5min":0.34499999999999997,"15min":0.33000000000000002},"max":{"1min":0.68700000000000006,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.59899999999999998}]},{"osd":2,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55700000000000005,"15min":0.53700000000000003},"min":{"1min":0.42999999999999999,"5min":0.39600000000000002,"15min":0.32400000000000001},"max":{"1min":0.71599999999999997,"5min":0.749,"15min":0.86199999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56100000000000005,"15min":0.56000000000000005},"min":{"1min":0.38100000000000001,"5min":0.33400000000000002,"15min":0.29599999999999999},"max":{"1min":0.749,"5min":0.80000000000000004,"15min":0.95399999999999996},"last":0.63600000000000001}]},{"osd":3,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57299999999999995,"15min":0.55700000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.316},"max":{"1min":0.71999999999999997,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56599999999999995,"15min":0.56899999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.72399999999999998,"5min":0.73499999999999999,"15min":1.0529999999999999},"last":0.56699999999999995}]},{"osd":4,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.47299999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.27700000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.504,"5min":0.51400000000000001,"15min":0.51700000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.28899999999999998},"max":{"1min":0.747,"5min":0.77500000000000002,"15min":0.91500000000000004},"last":0.45800000000000002}]},{"osd":6,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.33600000000000002,"5min":0.28199999999999997,"15min":0.27400000000000002},"max":{"1min":0.79400000000000004,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.443},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52300000000000002,"15min":0.51100000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.30499999999999999},"max":{"1min":0.81499999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.54800000000000004}]},{"osd":7,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.54200000000000004,"15min":0.53100000000000003},"min":{"1min":0.33500000000000002,"5min":0.318,"15min":0.29399999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81999999999999995},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56899999999999995,"15min":0.56200000000000006},"min":{"1min":0.38300000000000001,"5min":0.35599999999999998,"15min":0.32500000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.49099999999999999}]}]}],"pool_statfs":[]}} 2023-10-08T00:22:13.900 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-08T00:22:13.901 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T00:22:14.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.013+0000 7f2f7f953700 1 Processor -- start 2023-10-08T00:22:14.014 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.013+0000 7f2f7f953700 1 -- start start 2023-10-08T00:22:14.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.014+0000 7f2f7f953700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2f78157870 0x7f2f7815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:14.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.014+0000 7f2f7f953700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2f7815a620 0x7f2f7815d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:14.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.014+0000 7f2f7f953700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f7815cbb0 con 0x7f2f7815d580 2023-10-08T00:22:14.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.014+0000 7f2f7f953700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f7815bfa0 con 0x7f2f78157870 2023-10-08T00:22:14.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.014+0000 7f2f7f953700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f7815c280 con 0x7f2f7815a620 2023-10-08T00:22:14.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.014+0000 7f2f7d6ef700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2f78157870 0x7f2f7815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:14.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.014+0000 7f2f7def0700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f2f7815d580 0x7f2f78158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:37338/0 (socket says 172.21.15.99:37338) 2023-10-08T00:22:14.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.014+0000 7f2f7d6ef700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2f78157870 0x7f2f7815b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33276/0 (socket says 172.21.15.99:33276) 2023-10-08T00:22:14.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.014+0000 7f2f7def0700 1 -- 172.21.15.99:0/1260087695 learned_addr learned my addr 172.21.15.99:0/1260087695 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:22:14.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.015+0000 7f2f7d6ef700 1 -- 172.21.15.99:0/1260087695 >> v2:172.21.15.191:3300/0 conn(0x7f2f7815a620 msgr2=0x7f2f7815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:14.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.015+0000 7f2f7ceee700 1 --2- 172.21.15.99:0/1260087695 >> v2:172.21.15.191:3300/0 conn(0x7f2f7815a620 0x7f2f7815d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:14.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.015+0000 7f2f7d6ef700 1 --2- 172.21.15.99:0/1260087695 >> v2:172.21.15.191:3300/0 conn(0x7f2f7815a620 0x7f2f7815d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:14.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.015+0000 7f2f7d6ef700 1 -- 172.21.15.99:0/1260087695 >> v1:172.21.15.99:6789/0 conn(0x7f2f7815d580 legacy=0x7f2f78158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:14.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.015+0000 7f2f6ffff700 1 -- 172.21.15.99:0/1260087695 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3421297266 0 0) 0x7f2f7815cbb0 con 0x7f2f7815d580 2023-10-08T00:22:14.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.015+0000 7f2f7d6ef700 1 -- 172.21.15.99:0/1260087695 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f7815c560 con 0x7f2f78157870 2023-10-08T00:22:14.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.015+0000 7f2f7d6ef700 1 --2- 172.21.15.99:0/1260087695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f78157870 0x7f2f7815b6a0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f2f7815cb20 tx=0x7f2f74009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d43b3ee12a50ba39 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:14.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.015+0000 7f2f6ffff700 1 -- 172.21.15.99:0/1260087695 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7400e070 con 0x7f2f78157870 2023-10-08T00:22:14.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.015+0000 7f2f6ffff700 1 -- 172.21.15.99:0/1260087695 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f7400bc30 con 0x7f2f78157870 2023-10-08T00:22:14.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.015+0000 7f2f6ffff700 1 -- 172.21.15.99:0/1260087695 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f74012430 con 0x7f2f78157870 2023-10-08T00:22:14.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.015+0000 7f2f7f953700 1 -- 172.21.15.99:0/1260087695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f78157870 msgr2=0x7f2f7815b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:14.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.015+0000 7f2f7f953700 1 --2- 172.21.15.99:0/1260087695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f78157870 0x7f2f7815b6a0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f2f7815cb20 tx=0x7f2f74009780 comp rx=0 tx=0).stop 2023-10-08T00:22:14.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.015+0000 7f2f7f953700 1 -- 172.21.15.99:0/1260087695 shutdown_connections 2023-10-08T00:22:14.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.015+0000 7f2f7f953700 1 --2- 172.21.15.99:0/1260087695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f78157870 0x7f2f7815b6a0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:14.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.015+0000 7f2f7f953700 1 --2- 172.21.15.99:0/1260087695 >> v2:172.21.15.191:3300/0 conn(0x7f2f7815a620 0x7f2f7815d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:14.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.015+0000 7f2f7f953700 1 -- 172.21.15.99:0/1260087695 >> 172.21.15.99:0/1260087695 conn(0x7f2f78068ce0 msgr2=0x7f2f780690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:14.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.016+0000 7f2f7f953700 1 -- 172.21.15.99:0/1260087695 shutdown_connections 2023-10-08T00:22:14.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.016+0000 7f2f7f953700 1 -- 172.21.15.99:0/1260087695 wait complete. 2023-10-08T00:22:14.021 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.017+0000 7f2f7f953700 1 Processor -- start 2023-10-08T00:22:14.021 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.017+0000 7f2f7f953700 1 -- start start 2023-10-08T00:22:14.021 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.017+0000 7f2f7f953700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f78157870 0x7f2f7814bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:14.022 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.017+0000 7f2f7f953700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f7815a620 0x7f2f780b0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:14.022 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.017+0000 7f2f7f953700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f7815d580 0x7f2f78154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:14.022 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.017+0000 7f2f7f953700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f78150760 con 0x7f2f7815a620 2023-10-08T00:22:14.022 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.017+0000 7f2f7f953700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f780b1060 con 0x7f2f7815d580 2023-10-08T00:22:14.022 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.017+0000 7f2f7f953700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f780b1340 con 0x7f2f78157870 2023-10-08T00:22:14.022 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.018+0000 7f2f7ceee700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f7815a620 0x7f2f780b0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:14.022 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.018+0000 7f2f7ceee700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f7815a620 0x7f2f780b0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33282/0 (socket says 172.21.15.99:33282) 2023-10-08T00:22:14.022 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.018+0000 7f2f7ceee700 1 -- 172.21.15.99:0/469050247 learned_addr learned my addr 172.21.15.99:0/469050247 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:14.022 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.018+0000 7f2f7def0700 1 --2- 172.21.15.99:0/469050247 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f7815d580 0x7f2f78154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:14.022 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.018+0000 7f2f7d6ef700 1 --2- 172.21.15.99:0/469050247 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f78157870 0x7f2f7814bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:14.022 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.018+0000 7f2f7ceee700 1 -- 172.21.15.99:0/469050247 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f78157870 msgr2=0x7f2f7814bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:14.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.018+0000 7f2f7ceee700 1 --2- 172.21.15.99:0/469050247 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f78157870 0x7f2f7814bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:14.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.018+0000 7f2f7ceee700 1 -- 172.21.15.99:0/469050247 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f7815d580 msgr2=0x7f2f78154ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:14.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.018+0000 7f2f7ceee700 1 --2- 172.21.15.99:0/469050247 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f7815d580 0x7f2f78154ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:14.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.018+0000 7f2f7ceee700 1 -- 172.21.15.99:0/469050247 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f74009a20 con 0x7f2f7815a620 2023-10-08T00:22:14.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.018+0000 7f2f7ceee700 1 --2- 172.21.15.99:0/469050247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f7815a620 0x7f2f780b0610 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f2f6800c880 tx=0x7f2f68008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:14.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.018+0000 7f2f6d7fa700 1 -- 172.21.15.99:0/469050247 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f68011600 con 0x7f2f7815a620 2023-10-08T00:22:14.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.018+0000 7f2f6d7fa700 1 -- 172.21.15.99:0/469050247 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f68011760 con 0x7f2f7815a620 2023-10-08T00:22:14.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.018+0000 7f2f7f953700 1 -- 172.21.15.99:0/469050247 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f780b1620 con 0x7f2f7815a620 2023-10-08T00:22:14.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.018+0000 7f2f6d7fa700 1 -- 172.21.15.99:0/469050247 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f68011b00 con 0x7f2f7815a620 2023-10-08T00:22:14.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.018+0000 7f2f7f953700 1 -- 172.21.15.99:0/469050247 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f780cf590 con 0x7f2f7815a620 2023-10-08T00:22:14.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.019+0000 7f2f7f953700 1 -- 172.21.15.99:0/469050247 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f780cf7b0 con 0x7f2f7815a620 2023-10-08T00:22:14.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.020+0000 7f2f6d7fa700 1 -- 172.21.15.99:0/469050247 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2f68018360 con 0x7f2f7815a620 2023-10-08T00:22:14.023 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.020+0000 7f2f6d7fa700 1 --2- 172.21.15.99:0/469050247 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2f5c066f30 0x7f2f5c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:14.024 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.020+0000 7f2f6d7fa700 1 -- 172.21.15.99:0/469050247 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f2f680840d0 con 0x7f2f7815a620 2023-10-08T00:22:14.024 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.020+0000 7f2f7d6ef700 1 --2- 172.21.15.99:0/469050247 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2f5c066f30 0x7f2f5c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:14.024 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.021+0000 7f2f7d6ef700 1 --2- 172.21.15.99:0/469050247 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2f5c066f30 0x7f2f5c0693e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f2f7815cb20 tx=0x7f2f74019000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:14.025 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.024+0000 7f2f6d7fa700 1 -- 172.21.15.99:0/469050247 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2f68051640 con 0x7f2f7815a620 2023-10-08T00:22:14.207 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.207+0000 7f2f7f953700 1 -- 172.21.15.99:0/469050247 --> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2f7804c3c0 con 0x7f2f5c066f30 2023-10-08T00:22:14.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.209+0000 7f2f6d7fa700 1 -- 172.21.15.99:0/469050247 <== mgr.4099 v2:172.21.15.191:6832/85985 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50239 (secure 0 0 0) 0x7f2f7804c3c0 con 0x7f2f5c066f30 2023-10-08T00:22:14.211 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:22:14.212 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.212+0000 7f2f7f953700 1 -- 172.21.15.99:0/469050247 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2f5c066f30 msgr2=0x7f2f5c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:14.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.212+0000 7f2f7f953700 1 --2- 172.21.15.99:0/469050247 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2f5c066f30 0x7f2f5c0693e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f2f7815cb20 tx=0x7f2f74019000 comp rx=0 tx=0).stop 2023-10-08T00:22:14.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.212+0000 7f2f7f953700 1 -- 172.21.15.99:0/469050247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f7815a620 msgr2=0x7f2f780b0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:14.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.212+0000 7f2f7f953700 1 --2- 172.21.15.99:0/469050247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f7815a620 0x7f2f780b0610 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f2f6800c880 tx=0x7f2f68008040 comp rx=0 tx=0).stop 2023-10-08T00:22:14.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.212+0000 7f2f7f953700 1 -- 172.21.15.99:0/469050247 shutdown_connections 2023-10-08T00:22:14.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.212+0000 7f2f7f953700 1 --2- 172.21.15.99:0/469050247 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2f5c066f30 0x7f2f5c0693e0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:14.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.212+0000 7f2f7f953700 1 --2- 172.21.15.99:0/469050247 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f7815d580 0x7f2f78154ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:14.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.212+0000 7f2f7f953700 1 --2- 172.21.15.99:0/469050247 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f78157870 0x7f2f7814bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:14.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.212+0000 7f2f7f953700 1 --2- 172.21.15.99:0/469050247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f7815a620 0x7f2f780b0610 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:14.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.212+0000 7f2f7f953700 1 -- 172.21.15.99:0/469050247 >> 172.21.15.99:0/469050247 conn(0x7f2f78068ce0 msgr2=0x7f2f7815b650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:14.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.212+0000 7f2f7f953700 1 -- 172.21.15.99:0/469050247 shutdown_connections 2023-10-08T00:22:14.213 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.212+0000 7f2f7f953700 1 -- 172.21.15.99:0/469050247 wait complete. 2023-10-08T00:22:14.214 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2023-10-08T00:22:14.224 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":528,"stamp":"2023-10-08T00:22:14.185407+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":117,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9605,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1552894,"statfs":{"total":1600000000,"available":1590163552,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52489239,"num_objects":-36,"num_object_clones":0,"num_object_copies":-72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-36,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-122,"num_write_kb":-85081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001240"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":312,"reported_epoch":191,"state":"active+clean","last_fresh":"2023-10-08T00:22:11.248832+0000","last_change":"2023-10-08T00:10:43.728490+0000","last_active":"2023-10-08T00:22:11.248832+0000","last_peered":"2023-10-08T00:22:11.248832+0000","last_clean":"2023-10-08T00:22:11.248832+0000","last_became_active":"2023-10-08T00:10:43.728341+0000","last_became_peered":"2023-10-08T00:10:43.728341+0000","last_unstale":"2023-10-08T00:22:11.248832+0000","last_undegraded":"2023-10-08T00:22:11.248832+0000","last_fullsized":"2023-10-08T00:22:11.248832+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_clean_scrub_stamp":"2023-10-08T00:10:42.716096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:06:31.035293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574988,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1056,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194255,"statfs":{"total":200000000,"available":198918036,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.53000000000000003,"15min":0.52000000000000002},"min":{"1min":0.377,"5min":0.27800000000000002,"15min":0.26900000000000002},"max":{"1min":0.67500000000000004,"5min":0.77600000000000002,"15min":1.129},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.54900000000000004,"15min":0.54500000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.73799999999999999,"5min":0.754,"15min":1.0269999999999999},"last":0.48799999999999999}]},{"osd":1,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.57199999999999995,"15min":0.55500000000000005},"min":{"1min":0.372,"5min":0.32300000000000001,"15min":0.32000000000000001},"max":{"1min":0.70399999999999996,"5min":0.74299999999999999,"15min":0.82999999999999996},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.59799999999999998,"15min":0.58099999999999996},"min":{"1min":0.40500000000000003,"5min":0.39900000000000002,"15min":0.38200000000000001},"max":{"1min":0.746,"5min":0.79400000000000004,"15min":0.80100000000000005},"last":0.44700000000000001}]},{"osd":2,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56699999999999995,"15min":0.55100000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.34200000000000003},"max":{"1min":0.748,"5min":0.76400000000000001,"15min":0.79900000000000004},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.54900000000000004,"15min":0.53800000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.72599999999999998,"5min":0.748,"15min":0.78500000000000003},"last":0.56799999999999995}]},{"osd":3,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.54000000000000004},"min":{"1min":0.42999999999999999,"5min":0.36699999999999999,"15min":0.33700000000000002},"max":{"1min":0.753,"5min":0.77400000000000002,"15min":0.79300000000000004},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57099999999999995,"15min":0.56499999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.75800000000000001,"5min":0.77600000000000002,"15min":0.82099999999999995},"last":0.628}]},{"osd":4,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.46600000000000003,"15min":0.496},"min":{"1min":0.36599999999999999,"5min":0.30399999999999999,"15min":0.26400000000000001},"max":{"1min":0.66800000000000004,"5min":0.71499999999999997,"15min":0.755},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.51300000000000001,"15min":0.51800000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.61599999999999999}]},{"osd":5,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.503,"15min":0.52500000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.73499999999999999,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.54100000000000004,"15min":0.56000000000000005},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.76500000000000001,"5min":0.80600000000000005,"15min":0.82799999999999996},"last":0.80500000000000005}]},{"osd":6,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.41499999999999998,"5min":0.39600000000000002,"15min":0.33100000000000002},"max":{"1min":0.80400000000000005,"5min":0.84799999999999998,"15min":0.85399999999999998},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.58299999999999996,"15min":0.60399999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.27800000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.91500000000000004},"last":0.71999999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574988,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1056,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194255,"statfs":{"total":200000000,"available":198918036,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53100000000000003,"15min":0.50600000000000001},"min":{"1min":0.42399999999999999,"5min":0.32300000000000001,"15min":0.29199999999999998},"max":{"1min":0.74299999999999999,"5min":0.76300000000000001,"15min":1.012},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.53500000000000003,"15min":0.60699999999999998},"min":{"1min":0.38400000000000001,"5min":0.317,"15min":0.29199999999999998},"max":{"1min":0.65800000000000003,"5min":0.77800000000000002,"15min":5.1230000000000002},"last":0.55800000000000005}]},{"osd":1,"last update":"Sun Oct 8 00:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.55700000000000005,"15min":0.54000000000000004},"min":{"1min":0.36499999999999999,"5min":0.29899999999999999,"15min":0.28899999999999998},"max":{"1min":0.754,"5min":0.78100000000000003,"15min":1.1419999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.59399999999999997,"15min":0.57699999999999996},"min":{"1min":0.51300000000000001,"5min":0.41099999999999998,"15min":0.32100000000000001},"max":{"1min":0.81000000000000005,"5min":0.81799999999999995,"15min":1.1539999999999999},"last":0.42899999999999999}]},{"osd":2,"last update":"Sun Oct 8 00:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.55400000000000005,"15min":0.54200000000000004},"min":{"1min":0.47199999999999998,"5min":0.36299999999999999,"15min":0.32900000000000001},"max":{"1min":0.74099999999999999,"5min":0.76500000000000001,"15min":1.3149999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58299999999999996,"15min":0.57599999999999996},"min":{"1min":0.441,"5min":0.374,"15min":0.29399999999999998},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":1.3280000000000001},"last":0.65100000000000002}]},{"osd":3,"last update":"Sun Oct 8 00:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.56100000000000005,"15min":0.55400000000000005},"min":{"1min":0.35499999999999998,"5min":0.32000000000000001,"15min":0.25700000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":1.046},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.58099999999999996,"15min":0.56100000000000005},"min":{"1min":0.501,"5min":0.38600000000000001,"15min":0.32900000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":1.4390000000000001},"last":0.51200000000000001}]},{"osd":4,"last update":"Sun Oct 8 00:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46200000000000002,"15min":0.48099999999999998},"min":{"1min":0.30499999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.622,"5min":0.70099999999999996,"15min":0.80100000000000005},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55000000000000004,"15min":0.55700000000000005},"min":{"1min":0.41999999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.77600000000000002,"5min":0.79500000000000004,"15min":0.89100000000000001},"last":0.63200000000000001}]},{"osd":5,"last update":"Sun Oct 8 00:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.47699999999999998,"15min":0.48299999999999998},"min":{"1min":0.36099999999999999,"5min":0.29299999999999998,"15min":0.29099999999999998},"max":{"1min":0.73499999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.62},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.505,"15min":0.53200000000000003},"min":{"1min":0.35299999999999998,"5min":0.313,"15min":0.28699999999999998},"max":{"1min":0.70099999999999996,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.70299999999999996}]},{"osd":7,"last update":"Sun Oct 8 00:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.50600000000000001,"15min":0.52600000000000002},"min":{"1min":0.38100000000000001,"5min":0.32800000000000001,"15min":0.307},"max":{"1min":0.753,"5min":0.82899999999999996,"15min":0.91500000000000004},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53200000000000003,"15min":0.54200000000000004},"min":{"1min":0.39000000000000001,"5min":0.34899999999999998,"15min":0.32400000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.79900000000000004},"last":0.66400000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574988,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1633,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193679,"statfs":{"total":200000000,"available":198327668,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.35999999999999999,"15min":0.33300000000000002},"min":{"1min":0.223,"5min":0.216,"15min":0.19500000000000001},"max":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.41999999999999998,"15min":0.39500000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.67300000000000004},"last":0.38300000000000001}]},{"osd":2,"last update":"Sun Oct 8 00:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.38200000000000001,"15min":0.35499999999999998},"min":{"1min":0.27400000000000002,"5min":0.25900000000000001,"15min":0.22600000000000001},"max":{"1min":0.56999999999999995,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39500000000000002,"15min":0.36899999999999999},"min":{"1min":0.28499999999999998,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.53400000000000003,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.44600000000000001}]},{"osd":3,"last update":"Sun Oct 8 00:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.42499999999999999,"15min":0.39300000000000002},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.58799999999999997,"5min":0.63,"15min":0.70999999999999996},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.375,"15min":0.35599999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.222},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.34899999999999998}]},{"osd":4,"last update":"Sun Oct 8 00:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57699999999999996,"15min":0.55600000000000005},"min":{"1min":0.42999999999999999,"5min":0.34999999999999998,"15min":0.32600000000000001},"max":{"1min":0.82199999999999995,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59899999999999998,"15min":0.57999999999999996},"min":{"1min":0.42199999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.76000000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.498}]},{"osd":5,"last update":"Sun Oct 8 00:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57799999999999996,"15min":0.55900000000000005},"min":{"1min":0.39900000000000002,"5min":0.35099999999999998,"15min":0.33500000000000002},"max":{"1min":0.71799999999999997,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60299999999999998,"15min":0.58699999999999997},"min":{"1min":0.41799999999999998,"5min":0.39500000000000002,"15min":0.30399999999999999},"max":{"1min":0.76900000000000002,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.53400000000000003}]},{"osd":6,"last update":"Sun Oct 8 00:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.58299999999999996,"15min":0.57599999999999996},"min":{"1min":0.41499999999999998,"5min":0.37,"15min":0.317},"max":{"1min":0.755,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59299999999999997,"15min":0.57599999999999996},"min":{"1min":0.40799999999999997,"5min":0.377,"15min":0.33200000000000002},"max":{"1min":0.748,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.58099999999999996}]},{"osd":7,"last update":"Sun Oct 8 00:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58699999999999997,"15min":0.57599999999999996},"min":{"1min":0.439,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.746,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.626},{"interface":"front","average":{"1min":0.623,"5min":0.63200000000000001,"15min":0.61099999999999999},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.39100000000000001},"max":{"1min":0.79400000000000004,"5min":0.872,"15min":0.96699999999999997},"last":0.56799999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574988,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1056,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194255,"statfs":{"total":200000000,"available":198918036,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 00:22:08 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.33900000000000002,"15min":0.33500000000000002},"min":{"1min":0.218,"5min":0.20399999999999999,"15min":0.193},"max":{"1min":0.501,"5min":0.54900000000000004,"15min":1.0640000000000001},"last":0.22800000000000001},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.36899999999999999,"15min":0.35499999999999998},"min":{"1min":0.23499999999999999,"5min":0.216,"15min":0.20300000000000001},"max":{"1min":0.48999999999999999,"5min":0.66500000000000004,"15min":0.87},"last":0.33800000000000002}]},{"osd":2,"last update":"Sun Oct 8 00:22:08 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38900000000000001,"15min":0.38700000000000001},"min":{"1min":0.25800000000000001,"5min":0.249,"15min":0.24099999999999999},"max":{"1min":0.51600000000000001,"5min":0.57199999999999995,"15min":0.93799999999999994},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.35899999999999999,"15min":0.52500000000000002},"min":{"1min":0.23599999999999999,"5min":0.23499999999999999,"15min":0.185},"max":{"1min":0.45700000000000002,"5min":0.59899999999999998,"15min":12.019},"last":0.26300000000000001}]},{"osd":3,"last update":"Sun Oct 8 00:22:08 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.39000000000000001,"15min":0.371},"min":{"1min":0.27300000000000002,"5min":0.25700000000000001,"15min":0.22500000000000001},"max":{"1min":0.46700000000000003,"5min":0.628,"15min":1.1020000000000001},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.42099999999999999,"15min":0.40799999999999997},"min":{"1min":0.28499999999999998,"5min":0.26700000000000002,"15min":0.246},"max":{"1min":0.52000000000000002,"5min":0.66100000000000003,"15min":1.0409999999999999},"last":0.28000000000000003}]},{"osd":4,"last update":"Sun Oct 8 00:22:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.432,"5min":0.377,"15min":0.35899999999999999},"max":{"1min":0.72499999999999998,"5min":0.89700000000000002,"15min":1.1779999999999999},"last":0.505},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.624,"15min":0.622},"min":{"1min":0.46899999999999997,"5min":0.41099999999999998,"15min":0.40300000000000002},"max":{"1min":0.78000000000000003,"5min":0.93899999999999995,"15min":0.96199999999999997},"last":0.49199999999999999}]},{"osd":5,"last update":"Sun Oct 8 00:22:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58999999999999997,"15min":0.75900000000000001},"min":{"1min":0.44700000000000001,"5min":0.378,"15min":0.32000000000000001},"max":{"1min":0.83299999999999996,"5min":0.90700000000000003,"15min":11.992000000000001},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.60799999999999998,"15min":0.59599999999999997},"min":{"1min":0.443,"5min":0.379,"15min":0.379},"max":{"1min":0.77100000000000002,"5min":0.91600000000000004,"15min":1.1299999999999999},"last":0.45000000000000001}]},{"osd":6,"last update":"Sun Oct 8 00:22:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.60099999999999998,"15min":0.59299999999999997},"min":{"1min":0.442,"5min":0.372,"15min":0.372},"max":{"1min":0.73899999999999999,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.61299999999999999,"15min":0.76100000000000001},"min":{"1min":0.46899999999999997,"5min":0.39600000000000002,"15min":0.38100000000000001},"max":{"1min":0.85799999999999998,"5min":0.93999999999999995,"15min":11.927},"last":0.371}]},{"osd":7,"last update":"Sun Oct 8 00:22:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.60099999999999998,"15min":0.76700000000000002},"min":{"1min":0.42199999999999999,"5min":0.38600000000000001,"15min":0.35699999999999998},"max":{"1min":0.76400000000000001,"5min":0.95099999999999996,"15min":11.965},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.63700000000000001,"15min":0.626},"min":{"1min":0.42399999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.79900000000000004,"5min":0.89800000000000002,"15min":1.1539999999999999},"last":0.48399999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574988,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1056,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194255,"statfs":{"total":200000000,"available":198918036,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.32900000000000001,"15min":0.35799999999999998},"min":{"1min":0.20100000000000001,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.47599999999999998,"5min":0.56899999999999995,"15min":2.5139999999999998},"last":0.375},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.34699999999999998,"15min":0.36899999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.20599999999999999},"max":{"1min":0.51200000000000001,"5min":0.59099999999999997,"15min":2.1309999999999998},"last":0.36199999999999999}]},{"osd":1,"last update":"Sun Oct 8 00:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.35499999999999998,"15min":0.37},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":2.2639999999999998},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.374,"5min":0.38400000000000001,"15min":0.40300000000000002},"min":{"1min":0.254,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.54400000000000004,"5min":0.57199999999999995,"15min":2.5379999999999998},"last":0.443}]},{"osd":3,"last update":"Sun Oct 8 00:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.40400000000000003,"15min":0.41099999999999998},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.64400000000000002,"5min":0.66900000000000004,"15min":2.2890000000000001},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.41899999999999998,"15min":0.435},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.55900000000000005,"5min":0.64300000000000002,"15min":2.5619999999999998},"last":0.40999999999999998}]},{"osd":4,"last update":"Sun Oct 8 00:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54400000000000004,"15min":0.57999999999999996},"min":{"1min":0.31,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.77300000000000002,"5min":0.80800000000000005,"15min":2.488},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58599999999999997,"15min":0.59599999999999997},"min":{"1min":0.42999999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.77300000000000002,"5min":0.92400000000000004,"15min":2.073},"last":0.60799999999999998}]},{"osd":5,"last update":"Sun Oct 8 00:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.60899999999999999,"15min":0.63},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.76300000000000001,"5min":0.94299999999999995,"15min":2.2069999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.61699999999999999,"15min":0.64000000000000001},"min":{"1min":0.42199999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77300000000000002,"5min":0.91000000000000003,"15min":2.456},"last":0.70799999999999996}]},{"osd":6,"last update":"Sun Oct 8 00:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59599999999999997,"15min":0.60699999999999998},"min":{"1min":0.39100000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.80200000000000005,"5min":0.872,"15min":2.0339999999999998},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59399999999999997,"15min":0.60499999999999998},"min":{"1min":0.40799999999999997,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.75,"5min":0.94099999999999995,"15min":2.1699999999999999},"last":0.57999999999999996}]},{"osd":7,"last update":"Sun Oct 8 00:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60099999999999998,"15min":0.61899999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82199999999999995,"5min":0.91500000000000004,"15min":2.1030000000000002},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60699999999999998,"15min":0.61799999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.77700000000000002,"5min":1.0169999999999999,"15min":2.2349999999999999},"last":0.71899999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574988,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1056,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194255,"statfs":{"total":200000000,"available":198918036,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.33600000000000002,"15min":0.34699999999999998},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.192},"max":{"1min":0.58899999999999997,"5min":0.59299999999999997,"15min":1.46},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.31900000000000001,"5min":0.34999999999999998,"15min":0.34200000000000003},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.192},"max":{"1min":0.56000000000000005,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.29699999999999999}]},{"osd":1,"last update":"Sun Oct 8 00:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.36299999999999999,"15min":0.55500000000000005},"min":{"1min":0.25,"5min":0.247,"15min":0.20000000000000001},"max":{"1min":0.52700000000000002,"5min":0.57499999999999996,"15min":12.351000000000001},"last":0.252},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38700000000000001,"15min":0.372},"min":{"1min":0.23200000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.54500000000000004,"5min":0.58199999999999996,"15min":0.71999999999999997},"last":0.314}]},{"osd":2,"last update":"Sun Oct 8 00:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.40799999999999997,"15min":0.40600000000000003},"min":{"1min":0.248,"5min":0.248,"15min":0.23499999999999999},"max":{"1min":0.58099999999999996,"5min":0.59699999999999998,"15min":1.5489999999999999},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.41999999999999998,"15min":0.39800000000000002},"min":{"1min":0.27100000000000002,"5min":0.25700000000000001,"15min":0.246},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.753},"last":0.35599999999999998}]},{"osd":4,"last update":"Sun Oct 8 00:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.625,"15min":0.78400000000000003},"min":{"1min":0.39800000000000002,"5min":0.36899999999999999,"15min":0.34899999999999998},"max":{"1min":0.81399999999999995,"5min":0.96699999999999997,"15min":11.333},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.63400000000000001,"15min":0.80400000000000005},"min":{"1min":0.39100000000000001,"5min":0.38100000000000001,"15min":0.36099999999999999},"max":{"1min":0.89500000000000002,"5min":0.97699999999999998,"15min":12.321999999999999},"last":0.503}]},{"osd":5,"last update":"Sun Oct 8 00:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.61199999999999999,"15min":0.79800000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35499999999999998},"max":{"1min":0.86299999999999999,"5min":0.93500000000000005,"15min":12.291},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.64400000000000002,"15min":0.621},"min":{"1min":0.44800000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80500000000000005,"5min":1.143,"15min":1.143},"last":0.55400000000000005}]},{"osd":6,"last update":"Sun Oct 8 00:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.59899999999999998,"15min":0.755},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.33700000000000002},"max":{"1min":0.91200000000000003,"5min":0.91500000000000004,"15min":11.288},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.62,"15min":0.79100000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.36599999999999999},"max":{"1min":0.84599999999999997,"5min":0.94599999999999995,"15min":12.253},"last":0.56799999999999995}]},{"osd":7,"last update":"Sun Oct 8 00:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63200000000000001,"15min":0.60299999999999998},"min":{"1min":0.45700000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86399999999999999,"5min":1.0920000000000001,"15min":1.3109999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61599999999999999,"15min":0.78400000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.38600000000000001},"max":{"1min":0.879,"5min":0.89300000000000002,"15min":11.363},"last":0.45800000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574988,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1056,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194255,"statfs":{"total":200000000,"available":198918036,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.56699999999999995,"15min":0.56100000000000005},"min":{"1min":0.36299999999999999,"5min":0.32300000000000001,"15min":0.193},"max":{"1min":0.80900000000000005,"5min":0.84799999999999998,"15min":0.93000000000000005},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.60099999999999998,"15min":0.58699999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.30399999999999999},"max":{"1min":0.70199999999999996,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.52600000000000002}]},{"osd":1,"last update":"Sun Oct 8 00:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.54300000000000004,"15min":0.52900000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.33700000000000002},"max":{"1min":0.749,"5min":0.751,"15min":0.78100000000000003},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57299999999999995,"15min":0.54600000000000004},"min":{"1min":0.46899999999999997,"5min":0.34300000000000003,"15min":0.30199999999999999},"max":{"1min":0.71799999999999997,"5min":0.80600000000000005,"15min":0.81299999999999994},"last":0.48099999999999998}]},{"osd":2,"last update":"Sun Oct 8 00:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.54800000000000004,"15min":0.52700000000000002},"min":{"1min":0.35999999999999999,"5min":0.32500000000000001,"15min":0.32100000000000001},"max":{"1min":0.78400000000000003,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.56499999999999995,"15min":0.53400000000000003},"min":{"1min":0.41599999999999998,"5min":0.35399999999999998,"15min":0.32600000000000001},"max":{"1min":0.65300000000000002,"5min":0.79600000000000004,"15min":0.81699999999999995},"last":0.67500000000000004}]},{"osd":3,"last update":"Sun Oct 8 00:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.57999999999999996,"15min":0.55000000000000004},"min":{"1min":0.41099999999999998,"5min":0.35199999999999998,"15min":0.28799999999999998},"max":{"1min":0.77100000000000002,"5min":0.82299999999999995,"15min":0.85399999999999998},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.55200000000000005,"15min":0.54000000000000004},"min":{"1min":0.433,"5min":0.37,"15min":0.36199999999999999},"max":{"1min":0.68400000000000005,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.66100000000000003}]},{"osd":5,"last update":"Sun Oct 8 00:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.47299999999999998,"15min":0.47899999999999998},"min":{"1min":0.33900000000000002,"5min":0.314,"15min":0.27500000000000002},"max":{"1min":0.68600000000000005,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.501,"15min":0.504},"min":{"1min":0.34100000000000003,"5min":0.317,"15min":0.24399999999999999},"max":{"1min":0.72399999999999998,"5min":0.79100000000000004,"15min":0.94899999999999995},"last":0.753}]},{"osd":6,"last update":"Sun Oct 8 00:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.48899999999999999,"15min":0.48599999999999999},"min":{"1min":0.371,"5min":0.32400000000000001,"15min":0.26400000000000001},"max":{"1min":0.70899999999999996,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52500000000000002,"15min":0.53700000000000003},"min":{"1min":0.36299999999999999,"5min":0.34799999999999998,"15min":0.29799999999999999},"max":{"1min":0.73699999999999999,"5min":0.76200000000000001,"15min":0.80100000000000005},"last":0.77500000000000002}]},{"osd":7,"last update":"Sun Oct 8 00:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.53900000000000003,"15min":0.54300000000000004},"min":{"1min":0.34999999999999998,"5min":0.33200000000000002,"15min":0.27900000000000003},"max":{"1min":0.76200000000000001,"5min":0.83599999999999997,"15min":0.90100000000000002},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56999999999999995,"15min":0.57399999999999995},"min":{"1min":0.42999999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.751,"5min":0.88,"15min":0.88},"last":0.84499999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574988,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1633,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193679,"statfs":{"total":200000000,"available":198327668,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.54400000000000004,"15min":0.50600000000000001},"min":{"1min":0.38,"5min":0.36399999999999999,"15min":0.249},"max":{"1min":0.71599999999999997,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54100000000000004,"15min":0.54800000000000004},"min":{"1min":0.39000000000000001,"5min":0.29799999999999999,"15min":0.29399999999999998},"max":{"1min":0.75700000000000001,"5min":0.81699999999999995,"15min":0.88600000000000001},"last":0.38900000000000001}]},{"osd":1,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.55600000000000005,"15min":0.53600000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.32500000000000001},"max":{"1min":0.68300000000000005,"5min":0.72599999999999998,"15min":0.82999999999999996},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.53800000000000003,"15min":0.52500000000000002},"min":{"1min":0.34599999999999997,"5min":0.34499999999999997,"15min":0.33000000000000002},"max":{"1min":0.68700000000000006,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.42999999999999999}]},{"osd":2,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55700000000000005,"15min":0.53700000000000003},"min":{"1min":0.42999999999999999,"5min":0.39600000000000002,"15min":0.32400000000000001},"max":{"1min":0.71599999999999997,"5min":0.749,"15min":0.86199999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56100000000000005,"15min":0.56000000000000005},"min":{"1min":0.38100000000000001,"5min":0.33400000000000002,"15min":0.29599999999999999},"max":{"1min":0.749,"5min":0.80000000000000004,"15min":0.95399999999999996},"last":0.41499999999999998}]},{"osd":3,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57299999999999995,"15min":0.55700000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.316},"max":{"1min":0.71999999999999997,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56599999999999995,"15min":0.56899999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.72399999999999998,"5min":0.73499999999999999,"15min":1.0529999999999999},"last":0.50600000000000001}]},{"osd":4,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.47299999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.27700000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.504,"5min":0.51400000000000001,"15min":0.51700000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.28899999999999998},"max":{"1min":0.747,"5min":0.77500000000000002,"15min":0.91500000000000004},"last":0.53500000000000003}]},{"osd":6,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.33600000000000002,"5min":0.28199999999999997,"15min":0.27400000000000002},"max":{"1min":0.79400000000000004,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52300000000000002,"15min":0.51100000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.30499999999999999},"max":{"1min":0.81499999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.64300000000000002}]},{"osd":7,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.54200000000000004,"15min":0.53100000000000003},"min":{"1min":0.33500000000000002,"5min":0.318,"15min":0.29399999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81999999999999995},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56899999999999995,"15min":0.56200000000000006},"min":{"1min":0.38300000000000001,"5min":0.35599999999999998,"15min":0.32500000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.59199999999999997}]}]}],"pool_statfs":[]}} 2023-10-08T00:22:14.226 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:22:14.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.340+0000 7efe35fec700 1 Processor -- start 2023-10-08T00:22:14.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.340+0000 7efe35fec700 1 -- start start 2023-10-08T00:22:14.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.341+0000 7efe35fec700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7efe300b15e0 0x7efe30165540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:14.343 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.341+0000 7efe35fec700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe300b26d0 con 0x7efe300a79f0 2023-10-08T00:22:14.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.341+0000 7efe35fec700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe300aeab0 con 0x7efe3006f1c0 2023-10-08T00:22:14.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.341+0000 7efe35fec700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe300b2b60 con 0x7efe300b15e0 2023-10-08T00:22:14.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.341+0000 7efe2ffff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7efe300b15e0 0x7efe30165540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:14.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.341+0000 7efe2f7fe700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7efe300a79f0 0x7efe3006eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:37354/0 (socket says 172.21.15.99:37354) 2023-10-08T00:22:14.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.341+0000 7efe2f7fe700 1 -- 172.21.15.99:0/1341993695 learned_addr learned my addr 172.21.15.99:0/1341993695 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:22:14.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.342+0000 7efe2ffff700 1 -- 172.21.15.99:0/1341993695 >> v1:172.21.15.191:6789/0 conn(0x7efe3006f1c0 legacy=0x7efe30161e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:14.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.342+0000 7efe2ffff700 1 -- 172.21.15.99:0/1341993695 >> v1:172.21.15.99:6789/0 conn(0x7efe300a79f0 legacy=0x7efe3006eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:14.344 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.342+0000 7efe2ffff700 1 -- 172.21.15.99:0/1341993695 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe300b1ac0 con 0x7efe300b15e0 2023-10-08T00:22:14.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.342+0000 7efe2e7fc700 1 -- 172.21.15.99:0/1341993695 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1912293197 0 0) 0x7efe300b26d0 con 0x7efe300a79f0 2023-10-08T00:22:14.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.342+0000 7efe2ffff700 1 --2- 172.21.15.99:0/1341993695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe300b15e0 0x7efe30165540 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7efe28009e60 tx=0x7efe28009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dac20ad987a7e040 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:14.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.342+0000 7efe2e7fc700 1 -- 172.21.15.99:0/1341993695 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2800e070 con 0x7efe300b15e0 2023-10-08T00:22:14.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.342+0000 7efe2e7fc700 1 -- 172.21.15.99:0/1341993695 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe2800bbf0 con 0x7efe300b15e0 2023-10-08T00:22:14.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.342+0000 7efe2e7fc700 1 -- 172.21.15.99:0/1341993695 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe28012430 con 0x7efe300b15e0 2023-10-08T00:22:14.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.342+0000 7efe35fec700 1 -- 172.21.15.99:0/1341993695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe300b15e0 msgr2=0x7efe30165540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:14.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.342+0000 7efe35fec700 1 --2- 172.21.15.99:0/1341993695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe300b15e0 0x7efe30165540 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7efe28009e60 tx=0x7efe28009780 comp rx=0 tx=0).stop 2023-10-08T00:22:14.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.342+0000 7efe35fec700 1 -- 172.21.15.99:0/1341993695 shutdown_connections 2023-10-08T00:22:14.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.342+0000 7efe35fec700 1 --2- 172.21.15.99:0/1341993695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe300b15e0 0x7efe30165540 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:14.345 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.342+0000 7efe35fec700 1 -- 172.21.15.99:0/1341993695 >> 172.21.15.99:0/1341993695 conn(0x7efe300cd1e0 msgr2=0x7efe300cd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:14.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.342+0000 7efe35fec700 1 -- 172.21.15.99:0/1341993695 shutdown_connections 2023-10-08T00:22:14.346 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.342+0000 7efe35fec700 1 -- 172.21.15.99:0/1341993695 wait complete. 2023-10-08T00:22:14.348 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.344+0000 7efe35fec700 1 Processor -- start 2023-10-08T00:22:14.348 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.344+0000 7efe35fec700 1 -- start start 2023-10-08T00:22:14.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.344+0000 7efe35fec700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe3006f1c0 0x7efe3014c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:14.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.344+0000 7efe35fec700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe300a79f0 0x7efe300c2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:14.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.344+0000 7efe35fec700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe300b15e0 0x7efe300c2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:14.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.344+0000 7efe35fec700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3015d430 con 0x7efe300a79f0 2023-10-08T00:22:14.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.344+0000 7efe35fec700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe30099280 con 0x7efe3006f1c0 2023-10-08T00:22:14.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.344+0000 7efe35fec700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe30099560 con 0x7efe300b15e0 2023-10-08T00:22:14.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.344+0000 7efe2effd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe300a79f0 0x7efe300c2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:14.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.344+0000 7efe2effd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe300a79f0 0x7efe300c2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33298/0 (socket says 172.21.15.99:33298) 2023-10-08T00:22:14.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.344+0000 7efe2effd700 1 -- 172.21.15.99:0/3828893580 learned_addr learned my addr 172.21.15.99:0/3828893580 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:14.349 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.345+0000 7efe2f7fe700 1 --2- 172.21.15.99:0/3828893580 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe3006f1c0 0x7efe3014c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:14.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.345+0000 7efe2ffff700 1 --2- 172.21.15.99:0/3828893580 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe300b15e0 0x7efe300c2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:14.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.345+0000 7efe2effd700 1 -- 172.21.15.99:0/3828893580 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe300b15e0 msgr2=0x7efe300c2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:14.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.345+0000 7efe2effd700 1 --2- 172.21.15.99:0/3828893580 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe300b15e0 0x7efe300c2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:14.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.345+0000 7efe2effd700 1 -- 172.21.15.99:0/3828893580 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe3006f1c0 msgr2=0x7efe3014c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:14.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.345+0000 7efe2effd700 1 --2- 172.21.15.99:0/3828893580 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe3006f1c0 0x7efe3014c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:14.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.345+0000 7efe2effd700 1 -- 172.21.15.99:0/3828893580 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe28009a20 con 0x7efe300a79f0 2023-10-08T00:22:14.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.345+0000 7efe2effd700 1 --2- 172.21.15.99:0/3828893580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe300a79f0 0x7efe300c2990 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7efe2400b5e0 tx=0x7efe240090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:14.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.345+0000 7efe177fe700 1 -- 172.21.15.99:0/3828893580 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2400f510 con 0x7efe300a79f0 2023-10-08T00:22:14.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.345+0000 7efe177fe700 1 -- 172.21.15.99:0/3828893580 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe2400f6b0 con 0x7efe300a79f0 2023-10-08T00:22:14.350 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.345+0000 7efe177fe700 1 -- 172.21.15.99:0/3828893580 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2400fa70 con 0x7efe300a79f0 2023-10-08T00:22:14.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.345+0000 7efe35fec700 1 -- 172.21.15.99:0/3828893580 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe30099840 con 0x7efe300a79f0 2023-10-08T00:22:14.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.345+0000 7efe35fec700 1 -- 172.21.15.99:0/3828893580 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe3014cac0 con 0x7efe300a79f0 2023-10-08T00:22:14.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.346+0000 7efe35fec700 1 -- 172.21.15.99:0/3828893580 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe30099d60 con 0x7efe300a79f0 2023-10-08T00:22:14.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.346+0000 7efe177fe700 1 -- 172.21.15.99:0/3828893580 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efe2401a400 con 0x7efe300a79f0 2023-10-08T00:22:14.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.347+0000 7efe177fe700 1 --2- 172.21.15.99:0/3828893580 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efe1c066f30 0x7efe1c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:14.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.347+0000 7efe177fe700 1 -- 172.21.15.99:0/3828893580 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 4939+0+0 (secure 0 0 0) 0x7efe24083d80 con 0x7efe300a79f0 2023-10-08T00:22:14.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.347+0000 7efe2f7fe700 1 --2- 172.21.15.99:0/3828893580 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efe1c066f30 0x7efe1c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:14.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.348+0000 7efe2f7fe700 1 --2- 172.21.15.99:0/3828893580 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efe1c066f30 0x7efe1c0693e0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7efe200011e0 tx=0x7efe20009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:14.351 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.351+0000 7efe177fe700 1 -- 172.21.15.99:0/3828893580 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efe30099d60 con 0x7efe300a79f0 2023-10-08T00:22:14.536 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.536+0000 7efe35fec700 1 -- 172.21.15.99:0/3828893580 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe30099d60 con 0x7efe300a79f0 2023-10-08T00:22:14.537 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.536+0000 7efe177fe700 1 -- 172.21.15.99:0/3828893580 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v192) v1 ==== 74+0+12276 (secure 0 0 0) 0x7efe30099d60 con 0x7efe300a79f0 2023-10-08T00:22:14.537 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:22:14.537 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":192,"fsid":"06962cdb-9a6f-417f-82fd-9e4e3dedbccf","created":"2023-10-08T00:10:36.303601+0000","modified":"2023-10-08T00:22:11.174008+0000","last_up_change":"2023-10-08T00:10:41.709171+0000","last_in_change":"2023-10-08T00:10:38.690404+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:10:42.063344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":86365},{"type":"v1","addr":"172.21.15.99:6801","nonce":86365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":86365},{"type":"v1","addr":"172.21.15.99:6803","nonce":86365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":86365},{"type":"v1","addr":"172.21.15.99:6807","nonce":86365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":86365},{"type":"v1","addr":"172.21.15.99:6805","nonce":86365}]},"public_addr":"172.21.15.99:6801/86365","cluster_addr":"172.21.15.99:6803/86365","heartbeat_back_addr":"172.21.15.99:6807/86365","heartbeat_front_addr":"172.21.15.99:6805/86365","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":86366},{"type":"v1","addr":"172.21.15.99:6809","nonce":86366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":86366},{"type":"v1","addr":"172.21.15.99:6811","nonce":86366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":86366},{"type":"v1","addr":"172.21.15.99:6815","nonce":86366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":86366},{"type":"v1","addr":"172.21.15.99:6813","nonce":86366}]},"public_addr":"172.21.15.99:6809/86366","cluster_addr":"172.21.15.99:6811/86366","heartbeat_back_addr":"172.21.15.99:6815/86366","heartbeat_front_addr":"172.21.15.99:6813/86366","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":86382},{"type":"v1","addr":"172.21.15.99:6817","nonce":86382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6819","nonce":86382},{"type":"v1","addr":"172.21.15.99:6821","nonce":86382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":86382},{"type":"v1","addr":"172.21.15.99:6829","nonce":86382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":86382},{"type":"v1","addr":"172.21.15.99:6826","nonce":86382}]},"public_addr":"172.21.15.99:6817/86382","cluster_addr":"172.21.15.99:6821/86382","heartbeat_back_addr":"172.21.15.99:6829/86382","heartbeat_front_addr":"172.21.15.99:6826/86382","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":86393},{"type":"v1","addr":"172.21.15.99:6820","nonce":86393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":86393},{"type":"v1","addr":"172.21.15.99:6823","nonce":86393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":86393},{"type":"v1","addr":"172.21.15.99:6831","nonce":86393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":86393},{"type":"v1","addr":"172.21.15.99:6828","nonce":86393}]},"public_addr":"172.21.15.99:6820/86393","cluster_addr":"172.21.15.99:6823/86393","heartbeat_back_addr":"172.21.15.99:6831/86393","heartbeat_front_addr":"172.21.15.99:6828/86393","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":86570},{"type":"v1","addr":"172.21.15.191:6801","nonce":86570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":86570},{"type":"v1","addr":"172.21.15.191:6803","nonce":86570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":86570},{"type":"v1","addr":"172.21.15.191:6807","nonce":86570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":86570},{"type":"v1","addr":"172.21.15.191:6805","nonce":86570}]},"public_addr":"172.21.15.191:6801/86570","cluster_addr":"172.21.15.191:6803/86570","heartbeat_back_addr":"172.21.15.191:6807/86570","heartbeat_front_addr":"172.21.15.191:6805/86570","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":86608},{"type":"v1","addr":"172.21.15.191:6818","nonce":86608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":86608},{"type":"v1","addr":"172.21.15.191:6821","nonce":86608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":86608},{"type":"v1","addr":"172.21.15.191:6828","nonce":86608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6823","nonce":86608},{"type":"v1","addr":"172.21.15.191:6825","nonce":86608}]},"public_addr":"172.21.15.191:6818/86608","cluster_addr":"172.21.15.191:6821/86608","heartbeat_back_addr":"172.21.15.191:6828/86608","heartbeat_front_addr":"172.21.15.191:6825/86608","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":86571},{"type":"v1","addr":"172.21.15.191:6809","nonce":86571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":86571},{"type":"v1","addr":"172.21.15.191:6811","nonce":86571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":86571},{"type":"v1","addr":"172.21.15.191:6815","nonce":86571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":86571},{"type":"v1","addr":"172.21.15.191:6813","nonce":86571}]},"public_addr":"172.21.15.191:6809/86571","cluster_addr":"172.21.15.191:6811/86571","heartbeat_back_addr":"172.21.15.191:6815/86571","heartbeat_front_addr":"172.21.15.191:6813/86571","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":86612},{"type":"v1","addr":"172.21.15.191:6819","nonce":86612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":86612},{"type":"v1","addr":"172.21.15.191:6824","nonce":86612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":86612},{"type":"v1","addr":"172.21.15.191:6831","nonce":86612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6827","nonce":86612},{"type":"v1","addr":"172.21.15.191:6829","nonce":86612}]},"public_addr":"172.21.15.191:6819/86612","cluster_addr":"172.21.15.191:6824/86612","heartbeat_back_addr":"172.21.15.191:6831/86612","heartbeat_front_addr":"172.21.15.191:6829/86612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.905615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.880709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.874704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.899604+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.844371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.794810+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.859474+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:10:39.893130+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.191:6837/2466421758":"2023-10-09T00:22:08.745176+0000","172.21.15.191:6835/2466421758":"2023-10-09T00:22:08.745176+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:22:14.539 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.538+0000 7efe35fec700 1 -- 172.21.15.99:0/3828893580 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efe1c066f30 msgr2=0x7efe1c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:14.539 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.538+0000 7efe35fec700 1 --2- 172.21.15.99:0/3828893580 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efe1c066f30 0x7efe1c0693e0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7efe200011e0 tx=0x7efe20009040 comp rx=0 tx=0).stop 2023-10-08T00:22:14.539 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.538+0000 7efe35fec700 1 -- 172.21.15.99:0/3828893580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe300a79f0 msgr2=0x7efe300c2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:14.540 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.538+0000 7efe35fec700 1 --2- 172.21.15.99:0/3828893580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe300a79f0 0x7efe300c2990 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7efe2400b5e0 tx=0x7efe240090f0 comp rx=0 tx=0).stop 2023-10-08T00:22:14.540 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.538+0000 7efe35fec700 1 -- 172.21.15.99:0/3828893580 shutdown_connections 2023-10-08T00:22:14.540 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.538+0000 7efe35fec700 1 --2- 172.21.15.99:0/3828893580 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7efe1c066f30 0x7efe1c0693e0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:14.540 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.538+0000 7efe35fec700 1 --2- 172.21.15.99:0/3828893580 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe3006f1c0 0x7efe3014c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:14.540 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.538+0000 7efe35fec700 1 --2- 172.21.15.99:0/3828893580 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe300b15e0 0x7efe300c2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:14.540 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.538+0000 7efe35fec700 1 --2- 172.21.15.99:0/3828893580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe300a79f0 0x7efe300c2990 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:14.540 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.538+0000 7efe35fec700 1 -- 172.21.15.99:0/3828893580 >> 172.21.15.99:0/3828893580 conn(0x7efe300cd1e0 msgr2=0x7efe30069e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:14.540 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.538+0000 7efe35fec700 1 -- 172.21.15.99:0/3828893580 shutdown_connections 2023-10-08T00:22:14.541 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:14.539+0000 7efe35fec700 1 -- 172.21.15.99:0/3828893580 wait complete. 2023-10-08T00:22:15.552 INFO:tasks.ceph:Scrubbing osd.0 2023-10-08T00:22:15.552 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-10-08T00:22:15.676 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.675+0000 7fa8413d6700 1 Processor -- start 2023-10-08T00:22:15.676 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.675+0000 7fa8413d6700 1 -- start start 2023-10-08T00:22:15.679 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.676+0000 7fa8413d6700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa83c159e50 0x7fa83c06ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:15.679 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.676+0000 7fa8413d6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa83c068250 0x7fa83c06f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:15.680 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.676+0000 7fa8413d6700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa83c067dc0 con 0x7fa83c070700 2023-10-08T00:22:15.680 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.676+0000 7fa8413d6700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa83c0cb9e0 con 0x7fa83c159e50 2023-10-08T00:22:15.680 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.676+0000 7fa8413d6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa83c06fa70 con 0x7fa83c068250 2023-10-08T00:22:15.680 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.676+0000 7fa8327fc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa83c068250 0x7fa83c06f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:15.680 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.676+0000 7fa83affd700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa83c159e50 0x7fa83c06ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:15.680 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.676+0000 7fa83affd700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa83c159e50 0x7fa83c06ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.99:46316/0 (socket says 172.21.15.99:46316) 2023-10-08T00:22:15.680 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.676+0000 7fa83b7fe700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7fa83c070700 0x7fa83c06bc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:54868/0 (socket says 172.21.15.99:54868) 2023-10-08T00:22:15.680 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.676+0000 7fa8327fc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa83c068250 0x7fa83c06f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.99:44024/0 (socket says 172.21.15.99:44024) 2023-10-08T00:22:15.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.677+0000 7fa83affd700 1 -- 172.21.15.99:0/673102336 learned_addr learned my addr 172.21.15.99:0/673102336 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:15.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.677+0000 7fa83affd700 1 -- 172.21.15.99:0/673102336 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa83c068250 msgr2=0x7fa83c06f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:15.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.677+0000 7fa83affd700 1 --2- 172.21.15.99:0/673102336 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa83c068250 0x7fa83c06f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:15.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.677+0000 7fa83affd700 1 -- 172.21.15.99:0/673102336 >> v1:172.21.15.191:6789/0 conn(0x7fa83c070700 legacy=0x7fa83c06bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:15.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.677+0000 7fa83affd700 1 -- 172.21.15.99:0/673102336 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa83c06f790 con 0x7fa83c159e50 2023-10-08T00:22:15.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.677+0000 7fa8327fc700 1 --2- 172.21.15.99:0/673102336 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa83c068250 0x7fa83c06f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:22:15.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.677+0000 7fa83a7fc700 1 -- 172.21.15.99:0/673102336 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4273218576 0 0) 0x7fa83c067dc0 con 0x7fa83c070700 2023-10-08T00:22:15.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.677+0000 7fa83affd700 1 --2- 172.21.15.99:0/673102336 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa83c159e50 0x7fa83c06ebb0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa82c0099b0 tx=0x7fa82c009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd95d545997599f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:15.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.677+0000 7fa83a7fc700 1 -- 172.21.15.99:0/673102336 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82c009ea0 con 0x7fa83c159e50 2023-10-08T00:22:15.681 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.677+0000 7fa83a7fc700 1 -- 172.21.15.99:0/673102336 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa82c00b860 con 0x7fa83c159e50 2023-10-08T00:22:15.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.677+0000 7fa83a7fc700 1 -- 172.21.15.99:0/673102336 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82c00bc00 con 0x7fa83c159e50 2023-10-08T00:22:15.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.678+0000 7fa8413d6700 1 -- 172.21.15.99:0/673102336 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa83c159e50 msgr2=0x7fa83c06ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:15.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.678+0000 7fa8413d6700 1 --2- 172.21.15.99:0/673102336 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa83c159e50 0x7fa83c06ebb0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa82c0099b0 tx=0x7fa82c009250 comp rx=0 tx=0).stop 2023-10-08T00:22:15.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.678+0000 7fa8413d6700 1 -- 172.21.15.99:0/673102336 shutdown_connections 2023-10-08T00:22:15.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.678+0000 7fa8413d6700 1 --2- 172.21.15.99:0/673102336 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa83c159e50 0x7fa83c06ebb0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:15.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.678+0000 7fa8413d6700 1 --2- 172.21.15.99:0/673102336 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa83c068250 0x7fa83c06f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:15.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.678+0000 7fa8413d6700 1 -- 172.21.15.99:0/673102336 >> 172.21.15.99:0/673102336 conn(0x7fa83c0cdae0 msgr2=0x7fa83c0cdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:15.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.678+0000 7fa8413d6700 1 -- 172.21.15.99:0/673102336 shutdown_connections 2023-10-08T00:22:15.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.678+0000 7fa8413d6700 1 -- 172.21.15.99:0/673102336 wait complete. 2023-10-08T00:22:15.682 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.679+0000 7fa8413d6700 1 Processor -- start 2023-10-08T00:22:15.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.679+0000 7fa8413d6700 1 -- start start 2023-10-08T00:22:15.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.680+0000 7fa8413d6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa83c068250 0x7fa83c152f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:15.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.680+0000 7fa8413d6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa83c070700 0x7fa83c154650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:15.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.680+0000 7fa8413d6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa83c159e50 0x7fa83c158be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:15.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.680+0000 7fa8413d6700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa83c0ac870 con 0x7fa83c070700 2023-10-08T00:22:15.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.680+0000 7fa8413d6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa83c154b90 con 0x7fa83c159e50 2023-10-08T00:22:15.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.680+0000 7fa8413d6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa83c1541c0 con 0x7fa83c068250 2023-10-08T00:22:15.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.680+0000 7fa8327fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa83c070700 0x7fa83c154650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:15.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.680+0000 7fa8327fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa83c070700 0x7fa83c154650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33304/0 (socket says 172.21.15.99:33304) 2023-10-08T00:22:15.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.680+0000 7fa8327fc700 1 -- 172.21.15.99:0/4096231195 learned_addr learned my addr 172.21.15.99:0/4096231195 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:15.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.680+0000 7fa83affd700 1 --2- 172.21.15.99:0/4096231195 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa83c068250 0x7fa83c152f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:15.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.680+0000 7fa83b7fe700 1 --2- 172.21.15.99:0/4096231195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa83c159e50 0x7fa83c158be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:15.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.680+0000 7fa8327fc700 1 -- 172.21.15.99:0/4096231195 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa83c068250 msgr2=0x7fa83c152f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:15.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.680+0000 7fa8327fc700 1 --2- 172.21.15.99:0/4096231195 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa83c068250 0x7fa83c152f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:15.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.680+0000 7fa8327fc700 1 -- 172.21.15.99:0/4096231195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa83c159e50 msgr2=0x7fa83c158be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:15.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.680+0000 7fa8327fc700 1 --2- 172.21.15.99:0/4096231195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa83c159e50 0x7fa83c158be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:15.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.680+0000 7fa8327fc700 1 -- 172.21.15.99:0/4096231195 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa824009420 con 0x7fa83c070700 2023-10-08T00:22:15.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.680+0000 7fa8327fc700 1 --2- 172.21.15.99:0/4096231195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa83c070700 0x7fa83c154650 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fa82400ec80 tx=0x7fa8240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:15.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.680+0000 7fa8337fe700 1 -- 172.21.15.99:0/4096231195 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82400c8c0 con 0x7fa83c070700 2023-10-08T00:22:15.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.680+0000 7fa8337fe700 1 -- 172.21.15.99:0/4096231195 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa82400ca20 con 0x7fa83c070700 2023-10-08T00:22:15.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.680+0000 7fa8413d6700 1 -- 172.21.15.99:0/4096231195 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa82c0094f0 con 0x7fa83c070700 2023-10-08T00:22:15.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.681+0000 7fa8337fe700 1 -- 172.21.15.99:0/4096231195 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82400cda0 con 0x7fa83c070700 2023-10-08T00:22:15.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.681+0000 7fa8413d6700 1 -- 172.21.15.99:0/4096231195 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa83c153890 con 0x7fa83c070700 2023-10-08T00:22:15.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.681+0000 7fa8413d6700 1 -- 172.21.15.99:0/4096231195 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa83c04c3c0 con 0x7fa83c070700 2023-10-08T00:22:15.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.682+0000 7fa8337fe700 1 -- 172.21.15.99:0/4096231195 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa824013680 con 0x7fa83c070700 2023-10-08T00:22:15.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.682+0000 7fa8337fe700 1 --2- 172.21.15.99:0/4096231195 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa81c066f30 0x7fa81c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:15.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.682+0000 7fa8337fe700 1 -- 172.21.15.99:0/4096231195 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fa8240848f0 con 0x7fa83c070700 2023-10-08T00:22:15.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.682+0000 7fa8337fe700 1 --2- 172.21.15.99:0/4096231195 >> [v2:172.21.15.99:6800/86365,v1:172.21.15.99:6801/86365] conn(0x7fa81c06cd80 0x7fa81c06f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:15.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.682+0000 7fa8337fe700 1 -- 172.21.15.99:0/4096231195 --> [v2:172.21.15.99:6800/86365,v1:172.21.15.99:6801/86365] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa81c06f860 con 0x7fa81c06cd80 2023-10-08T00:22:15.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.682+0000 7fa8337fe700 1 -- 172.21.15.99:0/4096231195 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=192) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa824084cc0 con 0x7fa83c070700 2023-10-08T00:22:15.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.682+0000 7fa83affd700 1 --2- 172.21.15.99:0/4096231195 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa81c066f30 0x7fa81c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:15.688 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.683+0000 7fa83b7fe700 1 --2- 172.21.15.99:0/4096231195 >> [v2:172.21.15.99:6800/86365,v1:172.21.15.99:6801/86365] conn(0x7fa81c06cd80 0x7fa81c06f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:15.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.683+0000 7fa83b7fe700 1 --2- 172.21.15.99:0/4096231195 >> [v2:172.21.15.99:6800/86365,v1:172.21.15.99:6801/86365] conn(0x7fa81c06cd80 0x7fa81c06f190 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:15.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.683+0000 7fa83affd700 1 --2- 172.21.15.99:0/4096231195 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa81c066f30 0x7fa81c0693e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fa82c0099b0 tx=0x7fa82c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:15.689 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.683+0000 7fa8337fe700 1 -- 172.21.15.99:0/4096231195 <== osd.0 v2:172.21.15.99:6800/86365 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22340 (crc 0 0 0) 0x7fa81c06f860 con 0x7fa81c06cd80 2023-10-08T00:22:15.696 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.696+0000 7fa8413d6700 1 -- 172.21.15.99:0/4096231195 --> [v2:172.21.15.99:6800/86365,v1:172.21.15.99:6801/86365] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa83c067770 con 0x7fa81c06cd80 2023-10-08T00:22:15.696 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.696+0000 7fa8337fe700 1 -- 172.21.15.99:0/4096231195 <== osd.0 v2:172.21.15.99:6800/86365 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+682 (crc 0 0 0) 0x7fa83c067770 con 0x7fa81c06cd80 2023-10-08T00:22:15.697 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:22:15.697 INFO:teuthology.orchestra.run.smithi099.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2023-10-08T00:22:15.697 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:22:15.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.696+0000 7fa8413d6700 1 -- 172.21.15.99:0/4096231195 >> [v2:172.21.15.99:6800/86365,v1:172.21.15.99:6801/86365] conn(0x7fa81c06cd80 msgr2=0x7fa81c06f190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:15.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.696+0000 7fa8413d6700 1 --2- 172.21.15.99:0/4096231195 >> [v2:172.21.15.99:6800/86365,v1:172.21.15.99:6801/86365] conn(0x7fa81c06cd80 0x7fa81c06f190 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:15.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.696+0000 7fa8413d6700 1 -- 172.21.15.99:0/4096231195 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa81c066f30 msgr2=0x7fa81c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:15.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.696+0000 7fa8413d6700 1 --2- 172.21.15.99:0/4096231195 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa81c066f30 0x7fa81c0693e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fa82c0099b0 tx=0x7fa82c017000 comp rx=0 tx=0).stop 2023-10-08T00:22:15.699 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.696+0000 7fa8413d6700 1 -- 172.21.15.99:0/4096231195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa83c070700 msgr2=0x7fa83c154650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:15.699 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.696+0000 7fa8413d6700 1 --2- 172.21.15.99:0/4096231195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa83c070700 0x7fa83c154650 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fa82400ec80 tx=0x7fa8240091a0 comp rx=0 tx=0).stop 2023-10-08T00:22:15.699 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.696+0000 7fa83b7fe700 1 -- 172.21.15.99:0/4096231195 reap_dead start 2023-10-08T00:22:15.699 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.696+0000 7fa8413d6700 1 -- 172.21.15.99:0/4096231195 shutdown_connections 2023-10-08T00:22:15.699 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.696+0000 7fa8413d6700 1 -- 172.21.15.99:0/4096231195 >> 172.21.15.99:0/4096231195 conn(0x7fa83c0cdae0 msgr2=0x7fa83c0c2880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:15.699 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.696+0000 7fa8413d6700 1 -- 172.21.15.99:0/4096231195 shutdown_connections 2023-10-08T00:22:15.699 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.696+0000 7fa8413d6700 1 -- 172.21.15.99:0/4096231195 wait complete. 2023-10-08T00:22:15.708 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-10-08T00:22:15.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.822+0000 7fdfee871700 1 Processor -- start 2023-10-08T00:22:15.823 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.822+0000 7fdfee871700 1 -- start start 2023-10-08T00:22:15.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.823+0000 7fdfee871700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdfe80af280 0x7fdfe8063d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:15.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.823+0000 7fdfee871700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfe80b2150 con 0x7fdfe80af650 2023-10-08T00:22:15.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.823+0000 7fdfee871700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfe80b1540 con 0x7fdfe806aa20 2023-10-08T00:22:15.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.823+0000 7fdfee871700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfe815a2c0 con 0x7fdfe80af280 2023-10-08T00:22:15.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.823+0000 7fdfed86f700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdfe80af280 0x7fdfe8063d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:15.825 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.823+0000 7fdfed06e700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fdfe80af650 0x7fdfe806a310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:37368/0 (socket says 172.21.15.99:37368) 2023-10-08T00:22:15.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.823+0000 7fdfed06e700 1 -- 172.21.15.99:0/645569175 learned_addr learned my addr 172.21.15.99:0/645569175 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:22:15.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.824+0000 7fdfed86f700 1 -- 172.21.15.99:0/645569175 >> v1:172.21.15.191:6789/0 conn(0x7fdfe806aa20 legacy=0x7fdfe80ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:15.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.824+0000 7fdfed86f700 1 -- 172.21.15.99:0/645569175 >> v1:172.21.15.99:6789/0 conn(0x7fdfe80af650 legacy=0x7fdfe806a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:15.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.824+0000 7fdfed86f700 1 -- 172.21.15.99:0/645569175 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe80b1820 con 0x7fdfe80af280 2023-10-08T00:22:15.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.824+0000 7fdfec86d700 1 -- 172.21.15.99:0/645569175 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2873459235 0 0) 0x7fdfe80b2150 con 0x7fdfe80af650 2023-10-08T00:22:15.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.824+0000 7fdfed86f700 1 --2- 172.21.15.99:0/645569175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfe80af280 0x7fdfe8063d90 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4009e60 tx=0x7fdfe4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d04383673fc55889 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:15.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.824+0000 7fdfec86d700 1 -- 172.21.15.99:0/645569175 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe400e070 con 0x7fdfe80af280 2023-10-08T00:22:15.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.824+0000 7fdfec86d700 1 -- 172.21.15.99:0/645569175 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe400bbf0 con 0x7fdfe80af280 2023-10-08T00:22:15.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.824+0000 7fdfec86d700 1 -- 172.21.15.99:0/645569175 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe4012430 con 0x7fdfe80af280 2023-10-08T00:22:15.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.824+0000 7fdfee871700 1 -- 172.21.15.99:0/645569175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfe80af280 msgr2=0x7fdfe8063d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:15.826 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.824+0000 7fdfee871700 1 --2- 172.21.15.99:0/645569175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfe80af280 0x7fdfe8063d90 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4009e60 tx=0x7fdfe4009780 comp rx=0 tx=0).stop 2023-10-08T00:22:15.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.824+0000 7fdfee871700 1 -- 172.21.15.99:0/645569175 shutdown_connections 2023-10-08T00:22:15.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.824+0000 7fdfee871700 1 --2- 172.21.15.99:0/645569175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfe80af280 0x7fdfe8063d90 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:15.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.824+0000 7fdfee871700 1 -- 172.21.15.99:0/645569175 >> 172.21.15.99:0/645569175 conn(0x7fdfe80cd010 msgr2=0x7fdfe80cd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:15.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.825+0000 7fdfee871700 1 -- 172.21.15.99:0/645569175 shutdown_connections 2023-10-08T00:22:15.827 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.825+0000 7fdfee871700 1 -- 172.21.15.99:0/645569175 wait complete. 2023-10-08T00:22:15.830 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.826+0000 7fdfee871700 1 Processor -- start 2023-10-08T00:22:15.830 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.826+0000 7fdfee871700 1 -- start start 2023-10-08T00:22:15.830 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.826+0000 7fdfee871700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdfe806aa20 0x7fdfe8150820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:15.830 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.826+0000 7fdfee871700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdfe80af280 0x7fdfe8156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:15.830 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.826+0000 7fdfee871700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfe80af650 0x7fdfe8157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:15.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.826+0000 7fdfee871700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe81554a0 con 0x7fdfe80af650 2023-10-08T00:22:15.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.826+0000 7fdfee871700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe8155780 con 0x7fdfe806aa20 2023-10-08T00:22:15.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.826+0000 7fdfee871700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe8155a60 con 0x7fdfe80af280 2023-10-08T00:22:15.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.826+0000 7fdfee070700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfe80af650 0x7fdfe8157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:15.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.826+0000 7fdfee070700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfe80af650 0x7fdfe8157470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33316/0 (socket says 172.21.15.99:33316) 2023-10-08T00:22:15.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.826+0000 7fdfee070700 1 -- 172.21.15.99:0/2021694490 learned_addr learned my addr 172.21.15.99:0/2021694490 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:15.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.827+0000 7fdfed86f700 1 --2- 172.21.15.99:0/2021694490 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdfe806aa20 0x7fdfe8150820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:15.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.827+0000 7fdfed06e700 1 --2- 172.21.15.99:0/2021694490 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdfe80af280 0x7fdfe8156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:15.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.827+0000 7fdfee070700 1 -- 172.21.15.99:0/2021694490 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdfe80af280 msgr2=0x7fdfe8156f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:15.831 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.827+0000 7fdfee070700 1 --2- 172.21.15.99:0/2021694490 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdfe80af280 0x7fdfe8156f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:15.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.827+0000 7fdfee070700 1 -- 172.21.15.99:0/2021694490 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdfe806aa20 msgr2=0x7fdfe8150820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:15.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.827+0000 7fdfee070700 1 --2- 172.21.15.99:0/2021694490 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdfe806aa20 0x7fdfe8150820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:15.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.827+0000 7fdfee070700 1 -- 172.21.15.99:0/2021694490 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe4009a20 con 0x7fdfe80af650 2023-10-08T00:22:15.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.827+0000 7fdfee070700 1 --2- 172.21.15.99:0/2021694490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfe80af650 0x7fdfe8157470 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fdfe000c510 tx=0x7fdfe0007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:15.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.827+0000 7fdfddffb700 1 -- 172.21.15.99:0/2021694490 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe0011590 con 0x7fdfe80af650 2023-10-08T00:22:15.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.827+0000 7fdfddffb700 1 -- 172.21.15.99:0/2021694490 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe00116f0 con 0x7fdfe80af650 2023-10-08T00:22:15.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.827+0000 7fdfee871700 1 -- 172.21.15.99:0/2021694490 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfe8155d40 con 0x7fdfe80af650 2023-10-08T00:22:15.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.827+0000 7fdfddffb700 1 -- 172.21.15.99:0/2021694490 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe0011ab0 con 0x7fdfe80af650 2023-10-08T00:22:15.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.827+0000 7fdfee871700 1 -- 172.21.15.99:0/2021694490 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfe8098230 con 0x7fdfe80af650 2023-10-08T00:22:15.832 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.828+0000 7fdfee871700 1 -- 172.21.15.99:0/2021694490 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfe80984e0 con 0x7fdfe80af650 2023-10-08T00:22:15.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.828+0000 7fdfddffb700 1 -- 172.21.15.99:0/2021694490 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdfe001c460 con 0x7fdfe80af650 2023-10-08T00:22:15.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.829+0000 7fdfddffb700 1 --2- 172.21.15.99:0/2021694490 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdfcc066f30 0x7fdfcc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:15.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.829+0000 7fdfddffb700 1 -- 172.21.15.99:0/2021694490 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fdfe0083f10 con 0x7fdfe80af650 2023-10-08T00:22:15.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.829+0000 7fdfed86f700 1 --2- 172.21.15.99:0/2021694490 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdfcc066f30 0x7fdfcc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:15.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.829+0000 7fdfed86f700 1 --2- 172.21.15.99:0/2021694490 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdfcc066f30 0x7fdfcc0693e0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fdfe40097c0 tx=0x7fdfe4018000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:15.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:15.833+0000 7fdfddffb700 1 -- 172.21.15.99:0/2021694490 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdfe80984e0 con 0x7fdfe80af650 2023-10-08T00:22:16.016 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.016+0000 7fdfee871700 1 -- 172.21.15.99:0/2021694490 --> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fdfe80984e0 con 0x7fdfcc066f30 2023-10-08T00:22:16.017 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.016+0000 7fdfddffb700 1 -- 172.21.15.99:0/2021694490 <== mgr.4099 v2:172.21.15.191:6832/85985 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fdfe80984e0 con 0x7fdfcc066f30 2023-10-08T00:22:16.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.018+0000 7fdfee871700 1 -- 172.21.15.99:0/2021694490 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdfcc066f30 msgr2=0x7fdfcc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:16.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.018+0000 7fdfee871700 1 --2- 172.21.15.99:0/2021694490 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdfcc066f30 0x7fdfcc0693e0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fdfe40097c0 tx=0x7fdfe4018000 comp rx=0 tx=0).stop 2023-10-08T00:22:16.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.018+0000 7fdfee871700 1 -- 172.21.15.99:0/2021694490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfe80af650 msgr2=0x7fdfe8157470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:16.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.018+0000 7fdfee871700 1 --2- 172.21.15.99:0/2021694490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfe80af650 0x7fdfe8157470 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fdfe000c510 tx=0x7fdfe0007f30 comp rx=0 tx=0).stop 2023-10-08T00:22:16.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.018+0000 7fdfee871700 1 -- 172.21.15.99:0/2021694490 shutdown_connections 2023-10-08T00:22:16.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.018+0000 7fdfee871700 1 --2- 172.21.15.99:0/2021694490 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fdfcc066f30 0x7fdfcc0693e0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:16.020 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.018+0000 7fdfee871700 1 --2- 172.21.15.99:0/2021694490 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdfe806aa20 0x7fdfe8150820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:16.020 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.018+0000 7fdfee871700 1 --2- 172.21.15.99:0/2021694490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfe80af650 0x7fdfe8157470 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:16.020 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.018+0000 7fdfee871700 1 --2- 172.21.15.99:0/2021694490 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdfe80af280 0x7fdfe8156f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:16.020 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.018+0000 7fdfee871700 1 -- 172.21.15.99:0/2021694490 >> 172.21.15.99:0/2021694490 conn(0x7fdfe80cd010 msgr2=0x7fdfe81595d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:16.020 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.018+0000 7fdfee871700 1 -- 172.21.15.99:0/2021694490 shutdown_connections 2023-10-08T00:22:16.020 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.018+0000 7fdfee871700 1 -- 172.21.15.99:0/2021694490 wait complete. 2023-10-08T00:22:16.020 INFO:teuthology.orchestra.run.smithi099.stderr:instructed osd(s) 0 to deep-scrub 2023-10-08T00:22:16.030 INFO:tasks.ceph:Scrubbing osd.1 2023-10-08T00:22:16.031 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-10-08T00:22:16.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.148+0000 7fa8989dc700 1 Processor -- start 2023-10-08T00:22:16.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.148+0000 7fa8989dc700 1 -- start start 2023-10-08T00:22:16.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.149+0000 7fa8989dc700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa89415d240 0x7fa8940b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:16.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.149+0000 7fa8989dc700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa8940cc940 0x7fa8940ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:16.151 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.149+0000 7fa8989dc700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8940b3f40 con 0x7fa8940c59e0 2023-10-08T00:22:16.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.149+0000 7fa8989dc700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8940cbb80 con 0x7fa8940cc940 2023-10-08T00:22:16.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.149+0000 7fa8989dc700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8940cbe60 con 0x7fa89415d240 2023-10-08T00:22:16.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.149+0000 7fa892ffd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa8940cc940 0x7fa8940ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:16.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.149+0000 7fa892ffd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa8940cc940 0x7fa8940ccde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33330/0 (socket says 172.21.15.99:33330) 2023-10-08T00:22:16.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.149+0000 7fa892ffd700 1 -- 172.21.15.99:0/571086596 learned_addr learned my addr 172.21.15.99:0/571086596 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:16.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.149+0000 7fa8937fe700 1 --2- 172.21.15.99:0/571086596 >> v2:172.21.15.191:3300/0 conn(0x7fa89415d240 0x7fa8940b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:16.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.149+0000 7fa892ffd700 1 -- 172.21.15.99:0/571086596 >> v2:172.21.15.191:3300/0 conn(0x7fa89415d240 msgr2=0x7fa8940b3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:16.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.149+0000 7fa892ffd700 1 --2- 172.21.15.99:0/571086596 >> v2:172.21.15.191:3300/0 conn(0x7fa89415d240 0x7fa8940b3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:16.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.149+0000 7fa892ffd700 1 -- 172.21.15.99:0/571086596 >> v1:172.21.15.191:6789/0 conn(0x7fa8940c59e0 legacy=0x7fa8940c5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:16.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.149+0000 7fa892ffd700 1 -- 172.21.15.99:0/571086596 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8940cc140 con 0x7fa8940cc940 2023-10-08T00:22:16.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.150+0000 7fa892ffd700 1 --2- 172.21.15.99:0/571086596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8940cc940 0x7fa8940ccde0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fa89406b540 tx=0x7fa888009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e4eda765c9befa server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:16.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.150+0000 7fa8927fc700 1 -- 172.21.15.99:0/571086596 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa888009e70 con 0x7fa8940cc940 2023-10-08T00:22:16.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.150+0000 7fa8927fc700 1 -- 172.21.15.99:0/571086596 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa88800b860 con 0x7fa8940cc940 2023-10-08T00:22:16.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.150+0000 7fa8927fc700 1 -- 172.21.15.99:0/571086596 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88800bc00 con 0x7fa8940cc940 2023-10-08T00:22:16.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.150+0000 7fa8989dc700 1 -- 172.21.15.99:0/571086596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8940cc940 msgr2=0x7fa8940ccde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:16.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.150+0000 7fa8989dc700 1 --2- 172.21.15.99:0/571086596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8940cc940 0x7fa8940ccde0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fa89406b540 tx=0x7fa888009250 comp rx=0 tx=0).stop 2023-10-08T00:22:16.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.150+0000 7fa8989dc700 1 -- 172.21.15.99:0/571086596 shutdown_connections 2023-10-08T00:22:16.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.150+0000 7fa8989dc700 1 --2- 172.21.15.99:0/571086596 >> v2:172.21.15.191:3300/0 conn(0x7fa89415d240 0x7fa8940b3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:16.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.150+0000 7fa8989dc700 1 --2- 172.21.15.99:0/571086596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8940cc940 0x7fa8940ccde0 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:16.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.150+0000 7fa8989dc700 1 -- 172.21.15.99:0/571086596 >> 172.21.15.99:0/571086596 conn(0x7fa894067e40 msgr2=0x7fa8940d0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:16.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.150+0000 7fa8989dc700 1 -- 172.21.15.99:0/571086596 shutdown_connections 2023-10-08T00:22:16.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.150+0000 7fa8989dc700 1 -- 172.21.15.99:0/571086596 wait complete. 2023-10-08T00:22:16.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.152+0000 7fa8989dc700 1 Processor -- start 2023-10-08T00:22:16.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.152+0000 7fa8989dc700 1 -- start start 2023-10-08T00:22:16.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.152+0000 7fa8989dc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8940c59e0 0x7fa89409b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:16.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.152+0000 7fa8989dc700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa8940cc940 0x7fa89416a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:16.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.152+0000 7fa8989dc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa89415d240 0x7fa894156fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:16.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.152+0000 7fa8989dc700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89409cef0 con 0x7fa8940c59e0 2023-10-08T00:22:16.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.152+0000 7fa8989dc700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89416a8f0 con 0x7fa8940cc940 2023-10-08T00:22:16.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.152+0000 7fa8989dc700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa89416abd0 con 0x7fa89415d240 2023-10-08T00:22:16.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.152+0000 7fa8937fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8940c59e0 0x7fa89409b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:16.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.152+0000 7fa8937fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8940c59e0 0x7fa89409b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33342/0 (socket says 172.21.15.99:33342) 2023-10-08T00:22:16.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.152+0000 7fa8937fe700 1 -- 172.21.15.99:0/1321342922 learned_addr learned my addr 172.21.15.99:0/1321342922 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:16.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.153+0000 7fa892ffd700 1 --2- 172.21.15.99:0/1321342922 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa8940cc940 0x7fa89416a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:16.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.153+0000 7fa893fff700 1 --2- 172.21.15.99:0/1321342922 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa89415d240 0x7fa894156fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:16.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.153+0000 7fa8937fe700 1 -- 172.21.15.99:0/1321342922 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa89415d240 msgr2=0x7fa894156fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:16.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.153+0000 7fa8937fe700 1 --2- 172.21.15.99:0/1321342922 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa89415d240 0x7fa894156fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:16.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.153+0000 7fa8937fe700 1 -- 172.21.15.99:0/1321342922 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa8940cc940 msgr2=0x7fa89416a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:16.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.153+0000 7fa8937fe700 1 --2- 172.21.15.99:0/1321342922 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa8940cc940 0x7fa89416a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:16.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.153+0000 7fa8937fe700 1 -- 172.21.15.99:0/1321342922 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8880094f0 con 0x7fa8940c59e0 2023-10-08T00:22:16.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.153+0000 7fa8937fe700 1 --2- 172.21.15.99:0/1321342922 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8940c59e0 0x7fa89409b240 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fa88400ea00 tx=0x7fa884009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:16.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.153+0000 7fa87b7fe700 1 -- 172.21.15.99:0/1321342922 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88400c6a0 con 0x7fa8940c59e0 2023-10-08T00:22:16.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.153+0000 7fa87b7fe700 1 -- 172.21.15.99:0/1321342922 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa88400c800 con 0x7fa8940c59e0 2023-10-08T00:22:16.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.153+0000 7fa8989dc700 1 -- 172.21.15.99:0/1321342922 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa89416aeb0 con 0x7fa8940c59e0 2023-10-08T00:22:16.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.153+0000 7fa87b7fe700 1 -- 172.21.15.99:0/1321342922 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88400cb80 con 0x7fa8940c59e0 2023-10-08T00:22:16.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.153+0000 7fa8989dc700 1 -- 172.21.15.99:0/1321342922 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa89409cb10 con 0x7fa8940c59e0 2023-10-08T00:22:16.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.154+0000 7fa8989dc700 1 -- 172.21.15.99:0/1321342922 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa89406ac60 con 0x7fa8940c59e0 2023-10-08T00:22:16.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.154+0000 7fa87b7fe700 1 -- 172.21.15.99:0/1321342922 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa884013470 con 0x7fa8940c59e0 2023-10-08T00:22:16.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.155+0000 7fa87b7fe700 1 --2- 172.21.15.99:0/1321342922 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa880066f30 0x7fa8800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:16.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.155+0000 7fa87b7fe700 1 -- 172.21.15.99:0/1321342922 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fa884084db0 con 0x7fa8940c59e0 2023-10-08T00:22:16.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.155+0000 7fa87b7fe700 1 --2- 172.21.15.99:0/1321342922 >> [v2:172.21.15.99:6808/86366,v1:172.21.15.99:6809/86366] conn(0x7fa88006cd80 0x7fa88006f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:16.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.155+0000 7fa87b7fe700 1 -- 172.21.15.99:0/1321342922 --> [v2:172.21.15.99:6808/86366,v1:172.21.15.99:6809/86366] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa88006f860 con 0x7fa88006cd80 2023-10-08T00:22:16.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.155+0000 7fa87b7fe700 1 -- 172.21.15.99:0/1321342922 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=192) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa884085180 con 0x7fa8940c59e0 2023-10-08T00:22:16.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.155+0000 7fa892ffd700 1 --2- 172.21.15.99:0/1321342922 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa880066f30 0x7fa8800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:16.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.155+0000 7fa893fff700 1 --2- 172.21.15.99:0/1321342922 >> [v2:172.21.15.99:6808/86366,v1:172.21.15.99:6809/86366] conn(0x7fa88006cd80 0x7fa88006f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:16.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.155+0000 7fa893fff700 1 --2- 172.21.15.99:0/1321342922 >> [v2:172.21.15.99:6808/86366,v1:172.21.15.99:6809/86366] conn(0x7fa88006cd80 0x7fa88006f190 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:16.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.155+0000 7fa892ffd700 1 --2- 172.21.15.99:0/1321342922 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa880066f30 0x7fa8800693e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fa89406b540 tx=0x7fa888017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:16.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.156+0000 7fa87b7fe700 1 -- 172.21.15.99:0/1321342922 <== osd.1 v2:172.21.15.99:6808/86366 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22340 (crc 0 0 0) 0x7fa88006f860 con 0x7fa88006cd80 2023-10-08T00:22:16.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.168+0000 7fa8989dc700 1 -- 172.21.15.99:0/1321342922 --> [v2:172.21.15.99:6808/86366,v1:172.21.15.99:6809/86366] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa8940cc420 con 0x7fa88006cd80 2023-10-08T00:22:16.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.168+0000 7fa87b7fe700 1 -- 172.21.15.99:0/1321342922 <== osd.1 v2:172.21.15.99:6808/86366 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+682 (crc 0 0 0) 0x7fa8940cc420 con 0x7fa88006cd80 2023-10-08T00:22:16.169 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:22:16.169 INFO:teuthology.orchestra.run.smithi099.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2023-10-08T00:22:16.170 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:22:16.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.169+0000 7fa8989dc700 1 -- 172.21.15.99:0/1321342922 >> [v2:172.21.15.99:6808/86366,v1:172.21.15.99:6809/86366] conn(0x7fa88006cd80 msgr2=0x7fa88006f190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:16.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.169+0000 7fa8989dc700 1 --2- 172.21.15.99:0/1321342922 >> [v2:172.21.15.99:6808/86366,v1:172.21.15.99:6809/86366] conn(0x7fa88006cd80 0x7fa88006f190 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:16.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.169+0000 7fa8989dc700 1 -- 172.21.15.99:0/1321342922 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa880066f30 msgr2=0x7fa8800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:16.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.169+0000 7fa8989dc700 1 --2- 172.21.15.99:0/1321342922 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fa880066f30 0x7fa8800693e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fa89406b540 tx=0x7fa888017000 comp rx=0 tx=0).stop 2023-10-08T00:22:16.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.169+0000 7fa8989dc700 1 -- 172.21.15.99:0/1321342922 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8940c59e0 msgr2=0x7fa89409b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:16.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.169+0000 7fa8989dc700 1 --2- 172.21.15.99:0/1321342922 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8940c59e0 0x7fa89409b240 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fa88400ea00 tx=0x7fa884009120 comp rx=0 tx=0).stop 2023-10-08T00:22:16.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.169+0000 7fa893fff700 1 -- 172.21.15.99:0/1321342922 reap_dead start 2023-10-08T00:22:16.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.169+0000 7fa8989dc700 1 -- 172.21.15.99:0/1321342922 shutdown_connections 2023-10-08T00:22:16.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.169+0000 7fa8989dc700 1 -- 172.21.15.99:0/1321342922 >> 172.21.15.99:0/1321342922 conn(0x7fa894067e40 msgr2=0x7fa894068c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:16.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.169+0000 7fa8989dc700 1 -- 172.21.15.99:0/1321342922 shutdown_connections 2023-10-08T00:22:16.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.169+0000 7fa8989dc700 1 -- 172.21.15.99:0/1321342922 wait complete. 2023-10-08T00:22:16.182 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-10-08T00:22:16.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.297+0000 7fc3122e0700 1 Processor -- start 2023-10-08T00:22:16.297 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.297+0000 7fc3122e0700 1 -- start start 2023-10-08T00:22:16.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.297+0000 7fc3122e0700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc30c0abae0 0x7fc30c0c6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:16.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.297+0000 7fc3122e0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc30c0c8b10 0x7fc30c0c73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:16.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.297+0000 7fc3122e0700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc30c0c9cb0 con 0x7fc30c0abeb0 2023-10-08T00:22:16.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.297+0000 7fc3122e0700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc30c0c90a0 con 0x7fc30c0abae0 2023-10-08T00:22:16.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.297+0000 7fc3122e0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc30c0c9380 con 0x7fc30c0c8b10 2023-10-08T00:22:16.299 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.298+0000 7fc30bfff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc30c0abae0 0x7fc30c0c6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:16.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.298+0000 7fc31087d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc30c0c8b10 0x7fc30c0c73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:16.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.298+0000 7fc31087d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc30c0c8b10 0x7fc30c0c73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.99:44046/0 (socket says 172.21.15.99:44046) 2023-10-08T00:22:16.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.298+0000 7fc30bfff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc30c0abae0 0x7fc30c0c6eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.99:46350/0 (socket says 172.21.15.99:46350) 2023-10-08T00:22:16.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.298+0000 7fc31087d700 1 -- 172.21.15.99:0/1734464258 learned_addr learned my addr 172.21.15.99:0/1734464258 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:16.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.298+0000 7fc31087d700 1 -- 172.21.15.99:0/1734464258 >> v2:172.21.15.191:3300/0 conn(0x7fc30c0abae0 msgr2=0x7fc30c0c6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:16.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.298+0000 7fc31087d700 1 --2- 172.21.15.99:0/1734464258 >> v2:172.21.15.191:3300/0 conn(0x7fc30c0abae0 0x7fc30c0c6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:16.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.298+0000 7fc31087d700 1 -- 172.21.15.99:0/1734464258 >> v1:172.21.15.191:6789/0 conn(0x7fc30c0abeb0 legacy=0x7fc30c0c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:16.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.298+0000 7fc31087d700 1 -- 172.21.15.99:0/1734464258 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc30c0c9660 con 0x7fc30c0c8b10 2023-10-08T00:22:16.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.298+0000 7fc31087d700 1 --2- 172.21.15.99:0/1734464258 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc30c0c8b10 0x7fc30c0c73f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc304009f20 tx=0x7fc304009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26538db955ad4d16 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:16.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.298+0000 7fc30affd700 1 -- 172.21.15.99:0/1734464258 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30400e070 con 0x7fc30c0c8b10 2023-10-08T00:22:16.300 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.299+0000 7fc30affd700 1 -- 172.21.15.99:0/1734464258 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc30400b9d0 con 0x7fc30c0c8b10 2023-10-08T00:22:16.301 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.299+0000 7fc30affd700 1 -- 172.21.15.99:0/1734464258 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3040092e0 con 0x7fc30c0c8b10 2023-10-08T00:22:16.303 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.299+0000 7fc3122e0700 1 -- 172.21.15.99:0/1734464258 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc30c0c8b10 msgr2=0x7fc30c0c73f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:16.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.299+0000 7fc3122e0700 1 --2- 172.21.15.99:0/1734464258 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc30c0c8b10 0x7fc30c0c73f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc304009f20 tx=0x7fc304009840 comp rx=0 tx=0).stop 2023-10-08T00:22:16.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.299+0000 7fc3122e0700 1 -- 172.21.15.99:0/1734464258 shutdown_connections 2023-10-08T00:22:16.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.299+0000 7fc3122e0700 1 --2- 172.21.15.99:0/1734464258 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc30c0c8b10 0x7fc30c0c73f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:16.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.299+0000 7fc3122e0700 1 --2- 172.21.15.99:0/1734464258 >> v2:172.21.15.191:3300/0 conn(0x7fc30c0abae0 0x7fc30c0c6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:16.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.299+0000 7fc3122e0700 1 -- 172.21.15.99:0/1734464258 >> 172.21.15.99:0/1734464258 conn(0x7fc30c067660 msgr2=0x7fc30c067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:16.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.299+0000 7fc3122e0700 1 -- 172.21.15.99:0/1734464258 shutdown_connections 2023-10-08T00:22:16.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.299+0000 7fc3122e0700 1 -- 172.21.15.99:0/1734464258 wait complete. 2023-10-08T00:22:16.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.300+0000 7fc3122e0700 1 Processor -- start 2023-10-08T00:22:16.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.300+0000 7fc3122e0700 1 -- start start 2023-10-08T00:22:16.304 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.301+0000 7fc3122e0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc30c0abae0 0x7fc30c16e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:16.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.301+0000 7fc3122e0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc30c0abeb0 0x7fc30c1523e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:16.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.301+0000 7fc3122e0700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc30c0c8b10 0x7fc30c0aa880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:16.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.301+0000 7fc3122e0700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30c0c0df0 con 0x7fc30c0abeb0 2023-10-08T00:22:16.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.301+0000 7fc3122e0700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30c16e6d0 con 0x7fc30c0c8b10 2023-10-08T00:22:16.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.301+0000 7fc3122e0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc30c0c04e0 con 0x7fc30c0abae0 2023-10-08T00:22:16.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.301+0000 7fc30bfff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc30c0abae0 0x7fc30c16e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:16.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.301+0000 7fc30b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc30c0abeb0 0x7fc30c1523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:16.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.301+0000 7fc30bfff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc30c0abae0 0x7fc30c16e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.99:44052/0 (socket says 172.21.15.99:44052) 2023-10-08T00:22:16.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.301+0000 7fc30bfff700 1 -- 172.21.15.99:0/2555936532 learned_addr learned my addr 172.21.15.99:0/2555936532 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:16.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.301+0000 7fc31087d700 1 --2- 172.21.15.99:0/2555936532 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc30c0c8b10 0x7fc30c0aa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:16.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.302+0000 7fc30bfff700 1 -- 172.21.15.99:0/2555936532 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc30c0c8b10 msgr2=0x7fc30c0aa880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:16.305 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.302+0000 7fc30bfff700 1 --2- 172.21.15.99:0/2555936532 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc30c0c8b10 0x7fc30c0aa880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:16.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.302+0000 7fc30bfff700 1 -- 172.21.15.99:0/2555936532 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc30c0abeb0 msgr2=0x7fc30c1523e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:16.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.302+0000 7fc30bfff700 1 --2- 172.21.15.99:0/2555936532 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc30c0abeb0 0x7fc30c1523e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:16.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.302+0000 7fc30bfff700 1 -- 172.21.15.99:0/2555936532 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc304009ae0 con 0x7fc30c0abae0 2023-10-08T00:22:16.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.302+0000 7fc30bfff700 1 --2- 172.21.15.99:0/2555936532 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc30c0abae0 0x7fc30c16e190 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc00e960 tx=0x7fc2fc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:16.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.302+0000 7fc2f3fff700 1 -- 172.21.15.99:0/2555936532 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc013680 con 0x7fc30c0abae0 2023-10-08T00:22:16.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.302+0000 7fc2f3fff700 1 -- 172.21.15.99:0/2555936532 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2fc0137e0 con 0x7fc30c0abae0 2023-10-08T00:22:16.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.302+0000 7fc3122e0700 1 -- 172.21.15.99:0/2555936532 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc30c16b010 con 0x7fc30c0abae0 2023-10-08T00:22:16.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.302+0000 7fc2f3fff700 1 -- 172.21.15.99:0/2555936532 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc013b80 con 0x7fc30c0abae0 2023-10-08T00:22:16.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.302+0000 7fc3122e0700 1 -- 172.21.15.99:0/2555936532 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc30c0c0a80 con 0x7fc30c0abae0 2023-10-08T00:22:16.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.303+0000 7fc3122e0700 1 -- 172.21.15.99:0/2555936532 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc30c0c10d0 con 0x7fc30c0abae0 2023-10-08T00:22:16.306 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.303+0000 7fc2f3fff700 1 -- 172.21.15.99:0/2555936532 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc2fc0123d0 con 0x7fc30c0abae0 2023-10-08T00:22:16.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.304+0000 7fc2f3fff700 1 --2- 172.21.15.99:0/2555936532 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc2f8066f30 0x7fc2f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:16.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.304+0000 7fc2f3fff700 1 -- 172.21.15.99:0/2555936532 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fc2fc084370 con 0x7fc30c0abae0 2023-10-08T00:22:16.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.304+0000 7fc30b7fe700 1 --2- 172.21.15.99:0/2555936532 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc2f8066f30 0x7fc2f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:16.307 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.304+0000 7fc30b7fe700 1 --2- 172.21.15.99:0/2555936532 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc2f8066f30 0x7fc2f80693e0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fc30c152370 tx=0x7fc300000bc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:16.309 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.308+0000 7fc2f3fff700 1 -- 172.21.15.99:0/2555936532 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc2fc0518e0 con 0x7fc30c0abae0 2023-10-08T00:22:16.492 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.491+0000 7fc3122e0700 1 -- 172.21.15.99:0/2555936532 --> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc30c0c0f30 con 0x7fc2f8066f30 2023-10-08T00:22:16.492 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.492+0000 7fc2f3fff700 1 -- 172.21.15.99:0/2555936532 <== mgr.4099 v2:172.21.15.191:6832/85985 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc300002660 con 0x7fc2f8066f30 2023-10-08T00:22:16.495 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.494+0000 7fc3122e0700 1 -- 172.21.15.99:0/2555936532 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc2f8066f30 msgr2=0x7fc2f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:16.495 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.494+0000 7fc3122e0700 1 --2- 172.21.15.99:0/2555936532 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc2f8066f30 0x7fc2f80693e0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fc30c152370 tx=0x7fc300000bc0 comp rx=0 tx=0).stop 2023-10-08T00:22:16.495 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.494+0000 7fc3122e0700 1 -- 172.21.15.99:0/2555936532 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc30c0abae0 msgr2=0x7fc30c16e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:16.495 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.494+0000 7fc3122e0700 1 --2- 172.21.15.99:0/2555936532 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc30c0abae0 0x7fc30c16e190 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc00e960 tx=0x7fc2fc0091a0 comp rx=0 tx=0).stop 2023-10-08T00:22:16.495 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.494+0000 7fc3122e0700 1 -- 172.21.15.99:0/2555936532 shutdown_connections 2023-10-08T00:22:16.496 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.494+0000 7fc3122e0700 1 --2- 172.21.15.99:0/2555936532 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fc2f8066f30 0x7fc2f80693e0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:16.496 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.494+0000 7fc3122e0700 1 --2- 172.21.15.99:0/2555936532 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc30c0c8b10 0x7fc30c0aa880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:16.496 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.494+0000 7fc3122e0700 1 --2- 172.21.15.99:0/2555936532 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc30c0abae0 0x7fc30c16e190 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:16.496 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.494+0000 7fc3122e0700 1 --2- 172.21.15.99:0/2555936532 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc30c0abeb0 0x7fc30c1523e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:16.496 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.494+0000 7fc3122e0700 1 -- 172.21.15.99:0/2555936532 >> 172.21.15.99:0/2555936532 conn(0x7fc30c067660 msgr2=0x7fc30c06d090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:16.496 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.494+0000 7fc3122e0700 1 -- 172.21.15.99:0/2555936532 shutdown_connections 2023-10-08T00:22:16.496 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.494+0000 7fc3122e0700 1 -- 172.21.15.99:0/2555936532 wait complete. 2023-10-08T00:22:16.496 INFO:teuthology.orchestra.run.smithi099.stderr:instructed osd(s) 1 to deep-scrub 2023-10-08T00:22:16.506 INFO:tasks.ceph:Scrubbing osd.2 2023-10-08T00:22:16.507 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-10-08T00:22:16.626 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.625+0000 7f2bcec2e700 1 Processor -- start 2023-10-08T00:22:16.626 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.625+0000 7f2bcec2e700 1 -- start start 2023-10-08T00:22:16.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.626+0000 7f2bcec2e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2bc8159ef0 0x7f2bc815a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:16.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.626+0000 7f2bcec2e700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2bc80b46f0 0x7f2bc80b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:16.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.626+0000 7f2bcec2e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bc80b17b0 con 0x7f2bc80b6a10 2023-10-08T00:22:16.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.626+0000 7f2bcec2e700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bc80b5020 con 0x7f2bc80b46f0 2023-10-08T00:22:16.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.626+0000 7f2bcec2e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bc80b1a60 con 0x7f2bc8159ef0 2023-10-08T00:22:16.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.626+0000 7f2bcd1cb700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2bc80b46f0 0x7f2bc80b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:16.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.626+0000 7f2bcd1cb700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2bc80b46f0 0x7f2bc80b4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33370/0 (socket says 172.21.15.99:33370) 2023-10-08T00:22:16.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.626+0000 7f2bcd1cb700 1 -- 172.21.15.99:0/1631596280 learned_addr learned my addr 172.21.15.99:0/1631596280 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:16.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.626+0000 7f2bcc9ca700 1 --2- 172.21.15.99:0/1631596280 >> v2:172.21.15.191:3300/0 conn(0x7f2bc8159ef0 0x7f2bc815a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:16.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.626+0000 7f2bcd1cb700 1 -- 172.21.15.99:0/1631596280 >> v2:172.21.15.191:3300/0 conn(0x7f2bc8159ef0 msgr2=0x7f2bc815a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:16.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.627+0000 7f2bcd1cb700 1 --2- 172.21.15.99:0/1631596280 >> v2:172.21.15.191:3300/0 conn(0x7f2bc8159ef0 0x7f2bc815a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:16.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.627+0000 7f2bcd1cb700 1 -- 172.21.15.99:0/1631596280 >> v1:172.21.15.191:6789/0 conn(0x7f2bc80b6a10 legacy=0x7f2bc80b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:16.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.627+0000 7f2bcd1cb700 1 -- 172.21.15.99:0/1631596280 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc80b1d40 con 0x7f2bc80b46f0 2023-10-08T00:22:16.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.627+0000 7f2bcd1cb700 1 --2- 172.21.15.99:0/1631596280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bc80b46f0 0x7f2bc80b4ae0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f2bc808a710 tx=0x7f2bc0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ee08f44f63a6c54 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:16.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.627+0000 7f2bbf7fe700 1 -- 172.21.15.99:0/1631596280 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc000e070 con 0x7f2bc80b46f0 2023-10-08T00:22:16.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.627+0000 7f2bbf7fe700 1 -- 172.21.15.99:0/1631596280 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc000ba50 con 0x7f2bc80b46f0 2023-10-08T00:22:16.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.627+0000 7f2bbf7fe700 1 -- 172.21.15.99:0/1631596280 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc00092e0 con 0x7f2bc80b46f0 2023-10-08T00:22:16.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.627+0000 7f2bcec2e700 1 -- 172.21.15.99:0/1631596280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bc80b46f0 msgr2=0x7f2bc80b4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:16.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.627+0000 7f2bcec2e700 1 --2- 172.21.15.99:0/1631596280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bc80b46f0 0x7f2bc80b4ae0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f2bc808a710 tx=0x7f2bc0009840 comp rx=0 tx=0).stop 2023-10-08T00:22:16.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.627+0000 7f2bcec2e700 1 -- 172.21.15.99:0/1631596280 shutdown_connections 2023-10-08T00:22:16.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.627+0000 7f2bcec2e700 1 --2- 172.21.15.99:0/1631596280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bc80b46f0 0x7f2bc80b4ae0 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:16.630 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.627+0000 7f2bcec2e700 1 --2- 172.21.15.99:0/1631596280 >> v2:172.21.15.191:3300/0 conn(0x7f2bc8159ef0 0x7f2bc815a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:16.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.627+0000 7f2bcec2e700 1 -- 172.21.15.99:0/1631596280 >> 172.21.15.99:0/1631596280 conn(0x7f2bc809a0a0 msgr2=0x7f2bc8067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:16.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.627+0000 7f2bcec2e700 1 -- 172.21.15.99:0/1631596280 shutdown_connections 2023-10-08T00:22:16.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.627+0000 7f2bcec2e700 1 -- 172.21.15.99:0/1631596280 wait complete. 2023-10-08T00:22:16.631 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.628+0000 7f2bcec2e700 1 Processor -- start 2023-10-08T00:22:16.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.629+0000 7f2bcec2e700 1 -- start start 2023-10-08T00:22:16.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.629+0000 7f2bcec2e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2bc80b46f0 0x7f2bc81696e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:16.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.629+0000 7f2bcec2e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2bc80b6a10 0x7f2bc80c6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:16.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.629+0000 7f2bcec2e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bc8159ef0 0x7f2bc806a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:16.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.629+0000 7f2bcec2e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc8147ad0 con 0x7f2bc8159ef0 2023-10-08T00:22:16.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.629+0000 7f2bcec2e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc816d630 con 0x7f2bc80b46f0 2023-10-08T00:22:16.634 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.629+0000 7f2bcec2e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc816d910 con 0x7f2bc80b6a10 2023-10-08T00:22:16.635 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.630+0000 7f2bcd1cb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bc8159ef0 0x7f2bc806a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:16.635 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.630+0000 7f2bcd1cb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bc8159ef0 0x7f2bc806a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33374/0 (socket says 172.21.15.99:33374) 2023-10-08T00:22:16.635 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.630+0000 7f2bcd1cb700 1 -- 172.21.15.99:0/2296240736 learned_addr learned my addr 172.21.15.99:0/2296240736 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:16.635 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.630+0000 7f2bbffff700 1 --2- 172.21.15.99:0/2296240736 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2bc80b6a10 0x7f2bc80c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:16.635 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.630+0000 7f2bcc9ca700 1 --2- 172.21.15.99:0/2296240736 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2bc80b46f0 0x7f2bc81696e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:16.635 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.630+0000 7f2bcd1cb700 1 -- 172.21.15.99:0/2296240736 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2bc80b6a10 msgr2=0x7f2bc80c6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:16.635 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.630+0000 7f2bcd1cb700 1 --2- 172.21.15.99:0/2296240736 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2bc80b6a10 0x7f2bc80c6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:16.635 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.630+0000 7f2bcd1cb700 1 -- 172.21.15.99:0/2296240736 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2bc80b46f0 msgr2=0x7f2bc81696e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:16.635 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.630+0000 7f2bcd1cb700 1 --2- 172.21.15.99:0/2296240736 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2bc80b46f0 0x7f2bc81696e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:16.635 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.630+0000 7f2bcd1cb700 1 -- 172.21.15.99:0/2296240736 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc0009ae0 con 0x7f2bc8159ef0 2023-10-08T00:22:16.635 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.630+0000 7f2bcd1cb700 1 --2- 172.21.15.99:0/2296240736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bc8159ef0 0x7f2bc806a070 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8071f80 tx=0x7f2bc0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:16.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.630+0000 7f2bbcff9700 1 -- 172.21.15.99:0/2296240736 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc000be60 con 0x7f2bc8159ef0 2023-10-08T00:22:16.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.630+0000 7f2bbcff9700 1 -- 172.21.15.99:0/2296240736 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc0008d20 con 0x7f2bc8159ef0 2023-10-08T00:22:16.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.630+0000 7f2bcec2e700 1 -- 172.21.15.99:0/2296240736 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bc816dbf0 con 0x7f2bc8159ef0 2023-10-08T00:22:16.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.630+0000 7f2bbcff9700 1 -- 172.21.15.99:0/2296240736 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc00266b0 con 0x7f2bc8159ef0 2023-10-08T00:22:16.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.630+0000 7f2bcec2e700 1 -- 172.21.15.99:0/2296240736 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bc816e1b0 con 0x7f2bc8159ef0 2023-10-08T00:22:16.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.631+0000 7f2bcec2e700 1 -- 172.21.15.99:0/2296240736 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2bc804c3c0 con 0x7f2bc8159ef0 2023-10-08T00:22:16.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.632+0000 7f2bbcff9700 1 -- 172.21.15.99:0/2296240736 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2bc0003040 con 0x7f2bc8159ef0 2023-10-08T00:22:16.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.632+0000 7f2bbcff9700 1 --2- 172.21.15.99:0/2296240736 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2bb4066f30 0x7f2bb40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:16.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.632+0000 7f2bbcff9700 1 -- 172.21.15.99:0/2296240736 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f2bc00860f0 con 0x7f2bc8159ef0 2023-10-08T00:22:16.636 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.632+0000 7f2bbcff9700 1 --2- 172.21.15.99:0/2296240736 >> [v2:172.21.15.99:6816/86382,v1:172.21.15.99:6817/86382] conn(0x7f2bb406cd80 0x7f2bb406f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:16.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.632+0000 7f2bbcff9700 1 -- 172.21.15.99:0/2296240736 --> [v2:172.21.15.99:6816/86382,v1:172.21.15.99:6817/86382] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2bb406f860 con 0x7f2bb406cd80 2023-10-08T00:22:16.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.632+0000 7f2bbcff9700 1 -- 172.21.15.99:0/2296240736 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=192) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2bc00886d0 con 0x7f2bc8159ef0 2023-10-08T00:22:16.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.632+0000 7f2bcc9ca700 1 --2- 172.21.15.99:0/2296240736 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2bb4066f30 0x7f2bb40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:16.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.632+0000 7f2bbffff700 1 --2- 172.21.15.99:0/2296240736 >> [v2:172.21.15.99:6816/86382,v1:172.21.15.99:6817/86382] conn(0x7f2bb406cd80 0x7f2bb406f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:16.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.632+0000 7f2bbffff700 1 --2- 172.21.15.99:0/2296240736 >> [v2:172.21.15.99:6816/86382,v1:172.21.15.99:6817/86382] conn(0x7f2bb406cd80 0x7f2bb406f190 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:16.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.632+0000 7f2bcc9ca700 1 --2- 172.21.15.99:0/2296240736 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2bb4066f30 0x7f2bb40693e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f2bc40061f0 tx=0x7f2bc4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:16.637 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.633+0000 7f2bbcff9700 1 -- 172.21.15.99:0/2296240736 <== osd.2 v2:172.21.15.99:6816/86382 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22340 (crc 0 0 0) 0x7f2bb406f860 con 0x7f2bb406cd80 2023-10-08T00:22:16.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.645+0000 7f2bcec2e700 1 -- 172.21.15.99:0/2296240736 --> [v2:172.21.15.99:6816/86382,v1:172.21.15.99:6817/86382] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2bc80c5ea0 con 0x7f2bb406cd80 2023-10-08T00:22:16.646 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.645+0000 7f2bbcff9700 1 -- 172.21.15.99:0/2296240736 <== osd.2 v2:172.21.15.99:6816/86382 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+682 (crc 0 0 0) 0x7f2bc80c5ea0 con 0x7f2bb406cd80 2023-10-08T00:22:16.646 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:22:16.647 INFO:teuthology.orchestra.run.smithi099.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2023-10-08T00:22:16.647 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:22:16.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.646+0000 7f2bcec2e700 1 -- 172.21.15.99:0/2296240736 >> [v2:172.21.15.99:6816/86382,v1:172.21.15.99:6817/86382] conn(0x7f2bb406cd80 msgr2=0x7f2bb406f190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:16.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.646+0000 7f2bcec2e700 1 --2- 172.21.15.99:0/2296240736 >> [v2:172.21.15.99:6816/86382,v1:172.21.15.99:6817/86382] conn(0x7f2bb406cd80 0x7f2bb406f190 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:16.648 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.646+0000 7f2bcec2e700 1 -- 172.21.15.99:0/2296240736 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2bb4066f30 msgr2=0x7f2bb40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:16.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.646+0000 7f2bcec2e700 1 --2- 172.21.15.99:0/2296240736 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f2bb4066f30 0x7f2bb40693e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f2bc40061f0 tx=0x7f2bc4008040 comp rx=0 tx=0).stop 2023-10-08T00:22:16.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.646+0000 7f2bcec2e700 1 -- 172.21.15.99:0/2296240736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bc8159ef0 msgr2=0x7f2bc806a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:16.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.646+0000 7f2bcec2e700 1 --2- 172.21.15.99:0/2296240736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bc8159ef0 0x7f2bc806a070 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8071f80 tx=0x7f2bc0018000 comp rx=0 tx=0).stop 2023-10-08T00:22:16.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.646+0000 7f2bcd1cb700 1 -- 172.21.15.99:0/2296240736 reap_dead start 2023-10-08T00:22:16.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.646+0000 7f2bcec2e700 1 -- 172.21.15.99:0/2296240736 shutdown_connections 2023-10-08T00:22:16.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.646+0000 7f2bcec2e700 1 -- 172.21.15.99:0/2296240736 >> 172.21.15.99:0/2296240736 conn(0x7f2bc809a0a0 msgr2=0x7f2bc815aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:16.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.646+0000 7f2bcec2e700 1 -- 172.21.15.99:0/2296240736 shutdown_connections 2023-10-08T00:22:16.649 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.646+0000 7f2bcec2e700 1 -- 172.21.15.99:0/2296240736 wait complete. 2023-10-08T00:22:16.658 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-10-08T00:22:16.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.772+0000 7effaef2f700 1 Processor -- start 2023-10-08T00:22:16.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.772+0000 7effaef2f700 1 -- start start 2023-10-08T00:22:16.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.773+0000 7effaef2f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7effa80b0e50 0x7effa80c9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:16.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.773+0000 7effaef2f700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7effa80b1220 0x7effa806d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:16.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.773+0000 7effaef2f700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effa815de60 con 0x7effa806dad0 2023-10-08T00:22:16.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.773+0000 7effaef2f700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effa815e140 con 0x7effa80b1220 2023-10-08T00:22:16.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.773+0000 7effaef2f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effa815e420 con 0x7effa80b0e50 2023-10-08T00:22:16.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.773+0000 7effa7fff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7effa80b1220 0x7effa806d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:16.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.773+0000 7effad4cc700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7effa806dad0 0x7effa80cb4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:37376/0 (socket says 172.21.15.99:37376) 2023-10-08T00:22:16.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.773+0000 7effad4cc700 1 -- 172.21.15.99:0/1131499222 learned_addr learned my addr 172.21.15.99:0/1131499222 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:22:16.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.774+0000 7effacccb700 1 --2- 172.21.15.99:0/1131499222 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7effa80b0e50 0x7effa80c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:16.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.774+0000 7effa7fff700 1 -- 172.21.15.99:0/1131499222 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7effa80b0e50 msgr2=0x7effa80c9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:16.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.774+0000 7effa7fff700 1 --2- 172.21.15.99:0/1131499222 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7effa80b0e50 0x7effa80c9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:16.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.774+0000 7effa7fff700 1 -- 172.21.15.99:0/1131499222 >> v1:172.21.15.99:6789/0 conn(0x7effa806dad0 legacy=0x7effa80cb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:16.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.774+0000 7effa77fe700 1 -- 172.21.15.99:0/1131499222 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1702641334 0 0) 0x7effa815de60 con 0x7effa806dad0 2023-10-08T00:22:16.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.774+0000 7effa7fff700 1 -- 172.21.15.99:0/1131499222 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa815e700 con 0x7effa80b1220 2023-10-08T00:22:16.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.774+0000 7effa7fff700 1 --2- 172.21.15.99:0/1131499222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effa80b1220 0x7effa806d590 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7eff9c009910 tx=0x7eff9c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b797d1dfc25d8b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:16.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.774+0000 7effa77fe700 1 -- 172.21.15.99:0/1131499222 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9c00b640 con 0x7effa80b1220 2023-10-08T00:22:16.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.774+0000 7effa77fe700 1 -- 172.21.15.99:0/1131499222 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff9c00b7a0 con 0x7effa80b1220 2023-10-08T00:22:16.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.774+0000 7effa77fe700 1 -- 172.21.15.99:0/1131499222 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9c00bb60 con 0x7effa80b1220 2023-10-08T00:22:16.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.774+0000 7effaef2f700 1 -- 172.21.15.99:0/1131499222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effa80b1220 msgr2=0x7effa806d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:16.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.775+0000 7effaef2f700 1 --2- 172.21.15.99:0/1131499222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effa80b1220 0x7effa806d590 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7eff9c009910 tx=0x7eff9c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:22:16.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.775+0000 7effaef2f700 1 -- 172.21.15.99:0/1131499222 shutdown_connections 2023-10-08T00:22:16.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.775+0000 7effaef2f700 1 --2- 172.21.15.99:0/1131499222 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7effa80b0e50 0x7effa80c9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:16.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.775+0000 7effaef2f700 1 --2- 172.21.15.99:0/1131499222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effa80b1220 0x7effa806d590 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:16.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.775+0000 7effaef2f700 1 -- 172.21.15.99:0/1131499222 >> 172.21.15.99:0/1131499222 conn(0x7effa8069870 msgr2=0x7effa8069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:16.779 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.775+0000 7effaef2f700 1 -- 172.21.15.99:0/1131499222 shutdown_connections 2023-10-08T00:22:16.779 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.775+0000 7effaef2f700 1 -- 172.21.15.99:0/1131499222 wait complete. 2023-10-08T00:22:16.781 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.776+0000 7effaef2f700 1 Processor -- start 2023-10-08T00:22:16.781 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.776+0000 7effaef2f700 1 -- start start 2023-10-08T00:22:16.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.777+0000 7effaef2f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effa806dad0 0x7effa80c6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:16.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.777+0000 7effaef2f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7effa80b0e50 0x7effa815a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:16.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.777+0000 7effaef2f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7effa80b1220 0x7effa8158f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:16.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.777+0000 7effaef2f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa8153840 con 0x7effa806dad0 2023-10-08T00:22:16.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.777+0000 7effaef2f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa80c8a60 con 0x7effa80b0e50 2023-10-08T00:22:16.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.777+0000 7effaef2f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effa80c6d70 con 0x7effa80b1220 2023-10-08T00:22:16.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.777+0000 7effacccb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effa806dad0 0x7effa80c6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:16.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.777+0000 7effacccb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effa806dad0 0x7effa80c6600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33394/0 (socket says 172.21.15.99:33394) 2023-10-08T00:22:16.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.777+0000 7effacccb700 1 -- 172.21.15.99:0/3011398134 learned_addr learned my addr 172.21.15.99:0/3011398134 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:16.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.777+0000 7effad4cc700 1 --2- 172.21.15.99:0/3011398134 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7effa80b1220 0x7effa8158f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:16.782 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.777+0000 7effa7fff700 1 --2- 172.21.15.99:0/3011398134 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7effa80b0e50 0x7effa815a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:16.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.777+0000 7effacccb700 1 -- 172.21.15.99:0/3011398134 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7effa80b1220 msgr2=0x7effa8158f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:16.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.777+0000 7effacccb700 1 --2- 172.21.15.99:0/3011398134 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7effa80b1220 0x7effa8158f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:16.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.777+0000 7effacccb700 1 -- 172.21.15.99:0/3011398134 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7effa80b0e50 msgr2=0x7effa815a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:16.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.777+0000 7effacccb700 1 --2- 172.21.15.99:0/3011398134 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7effa80b0e50 0x7effa815a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:16.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.777+0000 7effacccb700 1 -- 172.21.15.99:0/3011398134 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff9c009420 con 0x7effa806dad0 2023-10-08T00:22:16.783 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.777+0000 7effacccb700 1 --2- 172.21.15.99:0/3011398134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effa806dad0 0x7effa80c6600 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7eff98009f10 tx=0x7eff9800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:16.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.777+0000 7effa4ff9700 1 -- 172.21.15.99:0/3011398134 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9800cc80 con 0x7effa806dad0 2023-10-08T00:22:16.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.777+0000 7effa4ff9700 1 -- 172.21.15.99:0/3011398134 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff9800f040 con 0x7effa806dad0 2023-10-08T00:22:16.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.777+0000 7effaef2f700 1 -- 172.21.15.99:0/3011398134 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effa80c6b40 con 0x7effa806dad0 2023-10-08T00:22:16.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.777+0000 7effa4ff9700 1 -- 172.21.15.99:0/3011398134 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff98014480 con 0x7effa806dad0 2023-10-08T00:22:16.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.777+0000 7effaef2f700 1 -- 172.21.15.99:0/3011398134 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effa8154be0 con 0x7effa806dad0 2023-10-08T00:22:16.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.778+0000 7effaef2f700 1 -- 172.21.15.99:0/3011398134 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effa804c3c0 con 0x7effa806dad0 2023-10-08T00:22:16.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.779+0000 7effa4ff9700 1 -- 172.21.15.99:0/3011398134 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7eff98014830 con 0x7effa806dad0 2023-10-08T00:22:16.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.779+0000 7effa4ff9700 1 --2- 172.21.15.99:0/3011398134 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7eff94066f30 0x7eff940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:16.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.779+0000 7effa4ff9700 1 -- 172.21.15.99:0/3011398134 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 4939+0+0 (secure 0 0 0) 0x7eff98084cd0 con 0x7effa806dad0 2023-10-08T00:22:16.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.779+0000 7effa7fff700 1 --2- 172.21.15.99:0/3011398134 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7eff94066f30 0x7eff940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:16.784 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.780+0000 7effa7fff700 1 --2- 172.21.15.99:0/3011398134 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7eff94066f30 0x7eff940693e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7effa80c5450 tx=0x7eff9c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:16.785 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.783+0000 7effa4ff9700 1 -- 172.21.15.99:0/3011398134 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7eff98052240 con 0x7effa806dad0 2023-10-08T00:22:16.966 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.965+0000 7effaef2f700 1 -- 172.21.15.99:0/3011398134 --> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7effa815a9a0 con 0x7eff94066f30 2023-10-08T00:22:16.966 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.966+0000 7effa4ff9700 1 -- 172.21.15.99:0/3011398134 <== mgr.4099 v2:172.21.15.191:6832/85985 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7effa815a9a0 con 0x7eff94066f30 2023-10-08T00:22:16.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.968+0000 7effaef2f700 1 -- 172.21.15.99:0/3011398134 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7eff94066f30 msgr2=0x7eff940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:16.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.968+0000 7effaef2f700 1 --2- 172.21.15.99:0/3011398134 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7eff94066f30 0x7eff940693e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7effa80c5450 tx=0x7eff9c017000 comp rx=0 tx=0).stop 2023-10-08T00:22:16.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.968+0000 7effaef2f700 1 -- 172.21.15.99:0/3011398134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effa806dad0 msgr2=0x7effa80c6600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:16.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.968+0000 7effaef2f700 1 --2- 172.21.15.99:0/3011398134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effa806dad0 0x7effa80c6600 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7eff98009f10 tx=0x7eff9800b040 comp rx=0 tx=0).stop 2023-10-08T00:22:16.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.968+0000 7effaef2f700 1 -- 172.21.15.99:0/3011398134 shutdown_connections 2023-10-08T00:22:16.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.968+0000 7effaef2f700 1 --2- 172.21.15.99:0/3011398134 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7eff94066f30 0x7eff940693e0 secure :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7effa80c5450 tx=0x7eff9c017000 comp rx=0 tx=0).stop 2023-10-08T00:22:16.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.968+0000 7effaef2f700 1 --2- 172.21.15.99:0/3011398134 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7effa80b1220 0x7effa8158f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:16.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.968+0000 7effaef2f700 1 --2- 172.21.15.99:0/3011398134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effa806dad0 0x7effa80c6600 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:16.969 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.968+0000 7effaef2f700 1 --2- 172.21.15.99:0/3011398134 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7effa80b0e50 0x7effa815a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:16.970 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.968+0000 7effaef2f700 1 -- 172.21.15.99:0/3011398134 >> 172.21.15.99:0/3011398134 conn(0x7effa8069870 msgr2=0x7effa8069c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:16.970 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.968+0000 7effaef2f700 1 -- 172.21.15.99:0/3011398134 shutdown_connections 2023-10-08T00:22:16.970 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:16.968+0000 7effaef2f700 1 -- 172.21.15.99:0/3011398134 wait complete. 2023-10-08T00:22:16.970 INFO:teuthology.orchestra.run.smithi099.stderr:instructed osd(s) 2 to deep-scrub 2023-10-08T00:22:16.980 INFO:tasks.ceph:Scrubbing osd.3 2023-10-08T00:22:16.981 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-10-08T00:22:17.098 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.098+0000 7effd1168700 1 Processor -- start 2023-10-08T00:22:17.099 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.098+0000 7effd1168700 1 -- start start 2023-10-08T00:22:17.101 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.099+0000 7effd1168700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7effcc15d240 0x7effcc0b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:17.101 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.099+0000 7effd1168700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7effcc0cc940 0x7effcc0ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:17.101 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.099+0000 7effd1168700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effcc0b3f40 con 0x7effcc0c59e0 2023-10-08T00:22:17.101 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.099+0000 7effd1168700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effcc0cbb80 con 0x7effcc0cc940 2023-10-08T00:22:17.102 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.099+0000 7effd1168700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effcc0cbe60 con 0x7effcc15d240 2023-10-08T00:22:17.102 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.099+0000 7effcb7fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7effcc0cc940 0x7effcc0ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:17.102 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.099+0000 7effcb7fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7effcc0cc940 0x7effcc0ccde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33408/0 (socket says 172.21.15.99:33408) 2023-10-08T00:22:17.102 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.099+0000 7effcb7fe700 1 -- 172.21.15.99:0/672123606 learned_addr learned my addr 172.21.15.99:0/672123606 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:17.102 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.099+0000 7effcbfff700 1 --2- 172.21.15.99:0/672123606 >> v2:172.21.15.191:3300/0 conn(0x7effcc15d240 0x7effcc0b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:17.102 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.099+0000 7effcb7fe700 1 -- 172.21.15.99:0/672123606 >> v2:172.21.15.191:3300/0 conn(0x7effcc15d240 msgr2=0x7effcc0b3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:17.102 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.099+0000 7effcb7fe700 1 --2- 172.21.15.99:0/672123606 >> v2:172.21.15.191:3300/0 conn(0x7effcc15d240 0x7effcc0b3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:17.102 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.099+0000 7effcb7fe700 1 -- 172.21.15.99:0/672123606 >> v1:172.21.15.191:6789/0 conn(0x7effcc0c59e0 legacy=0x7effcc0c5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:17.102 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.099+0000 7effcb7fe700 1 -- 172.21.15.99:0/672123606 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effcc0cc140 con 0x7effcc0cc940 2023-10-08T00:22:17.102 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.099+0000 7effcb7fe700 1 --2- 172.21.15.99:0/672123606 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effcc0cc940 0x7effcc0ccde0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7effcc06b540 tx=0x7effc0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66cefaef223e87c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:17.102 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.100+0000 7effcaffd700 1 -- 172.21.15.99:0/672123606 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc0009e70 con 0x7effcc0cc940 2023-10-08T00:22:17.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.100+0000 7effcaffd700 1 -- 172.21.15.99:0/672123606 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc000b860 con 0x7effcc0cc940 2023-10-08T00:22:17.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.100+0000 7effcaffd700 1 -- 172.21.15.99:0/672123606 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc000bc00 con 0x7effcc0cc940 2023-10-08T00:22:17.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.100+0000 7effd1168700 1 -- 172.21.15.99:0/672123606 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effcc0cc940 msgr2=0x7effcc0ccde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:17.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.100+0000 7effd1168700 1 --2- 172.21.15.99:0/672123606 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effcc0cc940 0x7effcc0ccde0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7effcc06b540 tx=0x7effc0009250 comp rx=0 tx=0).stop 2023-10-08T00:22:17.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.100+0000 7effd1168700 1 -- 172.21.15.99:0/672123606 shutdown_connections 2023-10-08T00:22:17.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.100+0000 7effd1168700 1 --2- 172.21.15.99:0/672123606 >> v2:172.21.15.191:3300/0 conn(0x7effcc15d240 0x7effcc0b3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:17.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.100+0000 7effd1168700 1 --2- 172.21.15.99:0/672123606 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effcc0cc940 0x7effcc0ccde0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:17.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.100+0000 7effd1168700 1 -- 172.21.15.99:0/672123606 >> 172.21.15.99:0/672123606 conn(0x7effcc067e40 msgr2=0x7effcc0d0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:17.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.100+0000 7effd1168700 1 -- 172.21.15.99:0/672123606 shutdown_connections 2023-10-08T00:22:17.103 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.100+0000 7effd1168700 1 -- 172.21.15.99:0/672123606 wait complete. 2023-10-08T00:22:17.107 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.101+0000 7effd1168700 1 Processor -- start 2023-10-08T00:22:17.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.101+0000 7effd1168700 1 -- start start 2023-10-08T00:22:17.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.102+0000 7effd1168700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7effcc0c59e0 0x7effcc16a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:17.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.102+0000 7effd1168700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effcc0cc940 0x7effcc0ca600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:17.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.102+0000 7effd1168700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7effcc15d240 0x7effcc0cab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:17.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.102+0000 7effd1168700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effcc0b5aa0 con 0x7effcc0cc940 2023-10-08T00:22:17.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.102+0000 7effd1168700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effcc0b4e90 con 0x7effcc15d240 2023-10-08T00:22:17.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.102+0000 7effd1168700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effcc0b5170 con 0x7effcc0c59e0 2023-10-08T00:22:17.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.102+0000 7effcb7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effcc0cc940 0x7effcc0ca600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:17.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.102+0000 7effcb7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effcc0cc940 0x7effcc0ca600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33418/0 (socket says 172.21.15.99:33418) 2023-10-08T00:22:17.108 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.102+0000 7effcb7fe700 1 -- 172.21.15.99:0/2385665891 learned_addr learned my addr 172.21.15.99:0/2385665891 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:17.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.102+0000 7effcbfff700 1 --2- 172.21.15.99:0/2385665891 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7effcc0c59e0 0x7effcc16a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:17.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.102+0000 7effd0967700 1 --2- 172.21.15.99:0/2385665891 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7effcc15d240 0x7effcc0cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:17.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.103+0000 7effcb7fe700 1 -- 172.21.15.99:0/2385665891 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7effcc0c59e0 msgr2=0x7effcc16a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:17.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.103+0000 7effcb7fe700 1 --2- 172.21.15.99:0/2385665891 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7effcc0c59e0 0x7effcc16a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:17.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.103+0000 7effcb7fe700 1 -- 172.21.15.99:0/2385665891 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7effcc15d240 msgr2=0x7effcc0cab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:17.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.103+0000 7effcb7fe700 1 --2- 172.21.15.99:0/2385665891 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7effcc15d240 0x7effcc0cab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:17.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.103+0000 7effcb7fe700 1 -- 172.21.15.99:0/2385665891 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc00094f0 con 0x7effcc0cc940 2023-10-08T00:22:17.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.103+0000 7effcb7fe700 1 --2- 172.21.15.99:0/2385665891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effcc0cc940 0x7effcc0ca600 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7effcc0688d0 tx=0x7effc0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:17.109 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.103+0000 7effb3fff700 1 -- 172.21.15.99:0/2385665891 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc000bbe0 con 0x7effcc0cc940 2023-10-08T00:22:17.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.103+0000 7effb3fff700 1 -- 172.21.15.99:0/2385665891 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc0011420 con 0x7effcc0cc940 2023-10-08T00:22:17.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.103+0000 7effb3fff700 1 -- 172.21.15.99:0/2385665891 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc00117c0 con 0x7effcc0cc940 2023-10-08T00:22:17.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.103+0000 7effd1168700 1 -- 172.21.15.99:0/2385665891 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effcc0b5450 con 0x7effcc0cc940 2023-10-08T00:22:17.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.103+0000 7effd1168700 1 -- 172.21.15.99:0/2385665891 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effcc16aec0 con 0x7effcc0cc940 2023-10-08T00:22:17.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.104+0000 7effd1168700 1 -- 172.21.15.99:0/2385665891 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7effcc06ac60 con 0x7effcc0cc940 2023-10-08T00:22:17.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.104+0000 7effb3fff700 1 -- 172.21.15.99:0/2385665891 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7effc0011bf0 con 0x7effcc0cc940 2023-10-08T00:22:17.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.105+0000 7effb3fff700 1 --2- 172.21.15.99:0/2385665891 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7effb8066f30 0x7effb80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:17.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.105+0000 7effb3fff700 1 -- 172.21.15.99:0/2385665891 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 4939+0+0 (secure 0 0 0) 0x7effc0085d80 con 0x7effcc0cc940 2023-10-08T00:22:17.110 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.105+0000 7effb3fff700 1 --2- 172.21.15.99:0/2385665891 >> [v2:172.21.15.99:6818/86393,v1:172.21.15.99:6820/86393] conn(0x7effb806cd80 0x7effb806f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:17.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.105+0000 7effb3fff700 1 -- 172.21.15.99:0/2385665891 --> [v2:172.21.15.99:6818/86393,v1:172.21.15.99:6820/86393] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7effb806f860 con 0x7effb806cd80 2023-10-08T00:22:17.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.105+0000 7effb3fff700 1 -- 172.21.15.99:0/2385665891 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=192) v2 ==== 24+0+0 (secure 0 0 0) 0x7effc0086150 con 0x7effcc0cc940 2023-10-08T00:22:17.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.105+0000 7effcbfff700 1 --2- 172.21.15.99:0/2385665891 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7effb8066f30 0x7effb80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:17.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.105+0000 7effd0967700 1 --2- 172.21.15.99:0/2385665891 >> [v2:172.21.15.99:6818/86393,v1:172.21.15.99:6820/86393] conn(0x7effb806cd80 0x7effb806f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:17.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.105+0000 7effd0967700 1 --2- 172.21.15.99:0/2385665891 >> [v2:172.21.15.99:6818/86393,v1:172.21.15.99:6820/86393] conn(0x7effb806cd80 0x7effb806f190 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:17.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.105+0000 7effcbfff700 1 --2- 172.21.15.99:0/2385665891 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7effb8066f30 0x7effb80693e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7effcc0b6510 tx=0x7effbc009120 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:17.111 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.106+0000 7effb3fff700 1 -- 172.21.15.99:0/2385665891 <== osd.3 v2:172.21.15.99:6818/86393 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22340 (crc 0 0 0) 0x7effb806f860 con 0x7effb806cd80 2023-10-08T00:22:17.119 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.118+0000 7effd1168700 1 -- 172.21.15.99:0/2385665891 --> [v2:172.21.15.99:6818/86393,v1:172.21.15.99:6820/86393] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7effcc0cc420 con 0x7effb806cd80 2023-10-08T00:22:17.119 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.119+0000 7effb3fff700 1 -- 172.21.15.99:0/2385665891 <== osd.3 v2:172.21.15.99:6818/86393 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+682 (crc 0 0 0) 0x7effcc0cc420 con 0x7effb806cd80 2023-10-08T00:22:17.119 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:22:17.120 INFO:teuthology.orchestra.run.smithi099.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2023-10-08T00:22:17.120 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:22:17.121 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.119+0000 7effd1168700 1 -- 172.21.15.99:0/2385665891 >> [v2:172.21.15.99:6818/86393,v1:172.21.15.99:6820/86393] conn(0x7effb806cd80 msgr2=0x7effb806f190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:17.121 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.119+0000 7effd1168700 1 --2- 172.21.15.99:0/2385665891 >> [v2:172.21.15.99:6818/86393,v1:172.21.15.99:6820/86393] conn(0x7effb806cd80 0x7effb806f190 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:17.121 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.119+0000 7effd1168700 1 -- 172.21.15.99:0/2385665891 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7effb8066f30 msgr2=0x7effb80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:17.121 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.119+0000 7effd1168700 1 --2- 172.21.15.99:0/2385665891 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7effb8066f30 0x7effb80693e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7effcc0b6510 tx=0x7effbc009120 comp rx=0 tx=0).stop 2023-10-08T00:22:17.121 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.119+0000 7effd1168700 1 -- 172.21.15.99:0/2385665891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effcc0cc940 msgr2=0x7effcc0ca600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:17.121 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.119+0000 7effd1168700 1 --2- 172.21.15.99:0/2385665891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effcc0cc940 0x7effcc0ca600 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7effcc0688d0 tx=0x7effc0017000 comp rx=0 tx=0).stop 2023-10-08T00:22:17.121 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.119+0000 7effd0967700 1 -- 172.21.15.99:0/2385665891 reap_dead start 2023-10-08T00:22:17.121 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.119+0000 7effd1168700 1 -- 172.21.15.99:0/2385665891 shutdown_connections 2023-10-08T00:22:17.121 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.119+0000 7effd1168700 1 -- 172.21.15.99:0/2385665891 >> 172.21.15.99:0/2385665891 conn(0x7effcc067e40 msgr2=0x7effcc15bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:17.122 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.119+0000 7effd1168700 1 -- 172.21.15.99:0/2385665891 shutdown_connections 2023-10-08T00:22:17.122 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.119+0000 7effd1168700 1 -- 172.21.15.99:0/2385665891 wait complete. 2023-10-08T00:22:17.131 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-10-08T00:22:17.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.246+0000 7fecb22fb700 1 Processor -- start 2023-10-08T00:22:17.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.246+0000 7fecb22fb700 1 -- start start 2023-10-08T00:22:17.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.246+0000 7fecb22fb700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fecac157870 0x7fecac15b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:17.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.246+0000 7fecb22fb700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fecac15a620 0x7fecac15d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:17.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.246+0000 7fecb22fb700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecac15cbb0 con 0x7fecac15d580 2023-10-08T00:22:17.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.246+0000 7fecb22fb700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecac15bfa0 con 0x7fecac15a620 2023-10-08T00:22:17.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.246+0000 7fecb22fb700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecac15c280 con 0x7fecac157870 2023-10-08T00:22:17.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.247+0000 7fecab7fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fecac15a620 0x7fecac15d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:17.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.247+0000 7fecb0898700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fecac15d580 0x7fecac158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:37386/0 (socket says 172.21.15.99:37386) 2023-10-08T00:22:17.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.247+0000 7fecb0898700 1 -- 172.21.15.99:0/220966812 learned_addr learned my addr 172.21.15.99:0/220966812 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:22:17.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.247+0000 7fecabfff700 1 --2- 172.21.15.99:0/220966812 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fecac157870 0x7fecac15b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:17.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.247+0000 7fecab7fe700 1 -- 172.21.15.99:0/220966812 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fecac157870 msgr2=0x7fecac15b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:17.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.247+0000 7fecab7fe700 1 --2- 172.21.15.99:0/220966812 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fecac157870 0x7fecac15b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:17.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.247+0000 7fecab7fe700 1 -- 172.21.15.99:0/220966812 >> v1:172.21.15.99:6789/0 conn(0x7fecac15d580 legacy=0x7fecac158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:17.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.247+0000 7fecab7fe700 1 -- 172.21.15.99:0/220966812 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecac15c560 con 0x7fecac15a620 2023-10-08T00:22:17.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.247+0000 7fecaaffd700 1 -- 172.21.15.99:0/220966812 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2751523302 0 0) 0x7fecac15cbb0 con 0x7fecac15d580 2023-10-08T00:22:17.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.247+0000 7fecabfff700 1 --2- 172.21.15.99:0/220966812 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fecac157870 0x7fecac15b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:22:17.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.247+0000 7fecab7fe700 1 --2- 172.21.15.99:0/220966812 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecac15a620 0x7fecac15d040 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7feca00098e0 tx=0x7feca00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f76c54aa74a788b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:17.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.247+0000 7fecaaffd700 1 -- 172.21.15.99:0/220966812 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca0009e80 con 0x7fecac15a620 2023-10-08T00:22:17.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.247+0000 7fecaaffd700 1 -- 172.21.15.99:0/220966812 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feca000b860 con 0x7fecac15a620 2023-10-08T00:22:17.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.247+0000 7fecaaffd700 1 -- 172.21.15.99:0/220966812 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca000bbe0 con 0x7fecac15a620 2023-10-08T00:22:17.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.248+0000 7fecb22fb700 1 -- 172.21.15.99:0/220966812 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecac15a620 msgr2=0x7fecac15d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:17.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.248+0000 7fecb22fb700 1 --2- 172.21.15.99:0/220966812 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecac15a620 0x7fecac15d040 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7feca00098e0 tx=0x7feca00091a0 comp rx=0 tx=0).stop 2023-10-08T00:22:17.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.248+0000 7fecb22fb700 1 -- 172.21.15.99:0/220966812 shutdown_connections 2023-10-08T00:22:17.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.248+0000 7fecb22fb700 1 --2- 172.21.15.99:0/220966812 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fecac157870 0x7fecac15b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:17.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.248+0000 7fecb22fb700 1 --2- 172.21.15.99:0/220966812 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecac15a620 0x7fecac15d040 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:17.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.248+0000 7fecb22fb700 1 -- 172.21.15.99:0/220966812 >> 172.21.15.99:0/220966812 conn(0x7fecac068ce0 msgr2=0x7fecac0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:17.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.248+0000 7fecb22fb700 1 -- 172.21.15.99:0/220966812 shutdown_connections 2023-10-08T00:22:17.251 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.248+0000 7fecb22fb700 1 -- 172.21.15.99:0/220966812 wait complete. 2023-10-08T00:22:17.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.249+0000 7fecb22fb700 1 Processor -- start 2023-10-08T00:22:17.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.249+0000 7fecb22fb700 1 -- start start 2023-10-08T00:22:17.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.250+0000 7fecb22fb700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fecac157870 0x7fecac15f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:17.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.250+0000 7fecb22fb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecac15a620 0x7fecac16d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:17.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.250+0000 7fecb22fb700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fecac15d580 0x7fecac15ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:17.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.250+0000 7fecb22fb700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecac1520f0 con 0x7fecac15a620 2023-10-08T00:22:17.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.250+0000 7fecb22fb700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecac1513c0 con 0x7fecac157870 2023-10-08T00:22:17.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.250+0000 7fecb22fb700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecac15efb0 con 0x7fecac15d580 2023-10-08T00:22:17.254 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.250+0000 7fecab7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecac15a620 0x7fecac16d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:17.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.250+0000 7fecab7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecac15a620 0x7fecac16d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33438/0 (socket says 172.21.15.99:33438) 2023-10-08T00:22:17.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.250+0000 7fecab7fe700 1 -- 172.21.15.99:0/805781902 learned_addr learned my addr 172.21.15.99:0/805781902 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:17.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.250+0000 7fecabfff700 1 --2- 172.21.15.99:0/805781902 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fecac157870 0x7fecac15f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:17.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.250+0000 7fecb0898700 1 --2- 172.21.15.99:0/805781902 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fecac15d580 0x7fecac15ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:17.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.250+0000 7fecab7fe700 1 -- 172.21.15.99:0/805781902 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fecac15d580 msgr2=0x7fecac15ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:17.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.250+0000 7fecab7fe700 1 --2- 172.21.15.99:0/805781902 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fecac15d580 0x7fecac15ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:17.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.250+0000 7fecab7fe700 1 -- 172.21.15.99:0/805781902 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fecac157870 msgr2=0x7fecac15f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:17.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.250+0000 7fecab7fe700 1 --2- 172.21.15.99:0/805781902 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fecac157870 0x7fecac15f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:17.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.250+0000 7fecab7fe700 1 -- 172.21.15.99:0/805781902 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feca0009420 con 0x7fecac15a620 2023-10-08T00:22:17.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.250+0000 7fecab7fe700 1 --2- 172.21.15.99:0/805781902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecac15a620 0x7fecac16d380 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fecac06b530 tx=0x7feca0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:17.255 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.251+0000 7fec93fff700 1 -- 172.21.15.99:0/805781902 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca000b9d0 con 0x7fecac15a620 2023-10-08T00:22:17.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.251+0000 7fec93fff700 1 -- 172.21.15.99:0/805781902 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feca000bb30 con 0x7fecac15a620 2023-10-08T00:22:17.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.251+0000 7fecb22fb700 1 -- 172.21.15.99:0/805781902 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecac15e3a0 con 0x7fecac15a620 2023-10-08T00:22:17.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.251+0000 7fec93fff700 1 -- 172.21.15.99:0/805781902 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca0025650 con 0x7fecac15a620 2023-10-08T00:22:17.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.251+0000 7fecb22fb700 1 -- 172.21.15.99:0/805781902 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecac15ec40 con 0x7fecac15a620 2023-10-08T00:22:17.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.251+0000 7fecb22fb700 1 -- 172.21.15.99:0/805781902 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecac15f120 con 0x7fecac15a620 2023-10-08T00:22:17.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.252+0000 7fec93fff700 1 -- 172.21.15.99:0/805781902 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7feca0023070 con 0x7fecac15a620 2023-10-08T00:22:17.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.252+0000 7fec93fff700 1 --2- 172.21.15.99:0/805781902 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fec98066f30 0x7fec980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:17.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.252+0000 7fec93fff700 1 -- 172.21.15.99:0/805781902 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 4939+0+0 (secure 0 0 0) 0x7feca0085820 con 0x7fecac15a620 2023-10-08T00:22:17.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.252+0000 7fecabfff700 1 --2- 172.21.15.99:0/805781902 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fec98066f30 0x7fec980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:17.256 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.253+0000 7fecabfff700 1 --2- 172.21.15.99:0/805781902 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fec98066f30 0x7fec980693e0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fec9c006f00 tx=0x7fec9c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:17.257 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.256+0000 7fec93fff700 1 -- 172.21.15.99:0/805781902 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feca0052d90 con 0x7fecac15a620 2023-10-08T00:22:17.444 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.444+0000 7fecb22fb700 1 -- 172.21.15.99:0/805781902 --> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fecac04c3c0 con 0x7fec98066f30 2023-10-08T00:22:17.445 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.444+0000 7fec93fff700 1 -- 172.21.15.99:0/805781902 <== mgr.4099 v2:172.21.15.191:6832/85985 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fecac04c3c0 con 0x7fec98066f30 2023-10-08T00:22:17.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.446+0000 7fecb22fb700 1 -- 172.21.15.99:0/805781902 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fec98066f30 msgr2=0x7fec980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:17.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.446+0000 7fecb22fb700 1 --2- 172.21.15.99:0/805781902 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fec98066f30 0x7fec980693e0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fec9c006f00 tx=0x7fec9c008040 comp rx=0 tx=0).stop 2023-10-08T00:22:17.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.446+0000 7fecb22fb700 1 -- 172.21.15.99:0/805781902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecac15a620 msgr2=0x7fecac16d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:17.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.446+0000 7fecb22fb700 1 --2- 172.21.15.99:0/805781902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecac15a620 0x7fecac16d380 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fecac06b530 tx=0x7feca0017000 comp rx=0 tx=0).stop 2023-10-08T00:22:17.448 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.446+0000 7fecb22fb700 1 -- 172.21.15.99:0/805781902 shutdown_connections 2023-10-08T00:22:17.449 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.446+0000 7fecb22fb700 1 --2- 172.21.15.99:0/805781902 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fec98066f30 0x7fec980693e0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:17.449 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.446+0000 7fecb22fb700 1 --2- 172.21.15.99:0/805781902 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fecac157870 0x7fecac15f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:17.449 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.446+0000 7fecb22fb700 1 --2- 172.21.15.99:0/805781902 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fecac15d580 0x7fecac15ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:17.449 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.446+0000 7fecb22fb700 1 --2- 172.21.15.99:0/805781902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecac15a620 0x7fecac16d380 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:17.449 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.446+0000 7fecb22fb700 1 -- 172.21.15.99:0/805781902 >> 172.21.15.99:0/805781902 conn(0x7fecac068ce0 msgr2=0x7fecac0cf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:17.449 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.446+0000 7fecb22fb700 1 -- 172.21.15.99:0/805781902 shutdown_connections 2023-10-08T00:22:17.449 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.447+0000 7fecb22fb700 1 -- 172.21.15.99:0/805781902 wait complete. 2023-10-08T00:22:17.449 INFO:teuthology.orchestra.run.smithi099.stderr:instructed osd(s) 3 to deep-scrub 2023-10-08T00:22:17.459 INFO:tasks.ceph:Scrubbing osd.4 2023-10-08T00:22:17.459 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-10-08T00:22:17.578 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.577+0000 7faeb5c74700 1 Processor -- start 2023-10-08T00:22:17.578 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.577+0000 7faeb5c74700 1 -- start start 2023-10-08T00:22:17.581 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.578+0000 7faeb5c74700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faeb00cf820 0x7faeb0164a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:17.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.578+0000 7faeb5c74700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faeb00cd4f0 con 0x7faeb015d240 2023-10-08T00:22:17.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.578+0000 7faeb5c74700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faeb00cbb80 con 0x7faeb00cc940 2023-10-08T00:22:17.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.578+0000 7faeb5c74700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeb00cbe60 con 0x7faeb00cf820 2023-10-08T00:22:17.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.578+0000 7faeb4c72700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7faeb015d240 0x7faeb00b3a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:37392/0 (socket says 172.21.15.99:37392) 2023-10-08T00:22:17.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.578+0000 7faeb4c72700 1 -- 172.21.15.99:0/4011197308 learned_addr learned my addr 172.21.15.99:0/4011197308 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:22:17.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.578+0000 7faeb5473700 1 --2- 172.21.15.99:0/4011197308 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faeb00cf820 0x7faeb0164a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:17.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.578+0000 7faeaf7fe700 1 -- 172.21.15.99:0/4011197308 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3543716275 0 0) 0x7faeb00cd4f0 con 0x7faeb015d240 2023-10-08T00:22:17.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.578+0000 7faeaf7fe700 1 -- 172.21.15.99:0/4011197308 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae9c003570 con 0x7faeb015d240 2023-10-08T00:22:17.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.578+0000 7faeaf7fe700 1 -- 172.21.15.99:0/4011197308 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 975197775 0 0) 0x7fae9c003570 con 0x7faeb015d240 2023-10-08T00:22:17.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.578+0000 7faeaf7fe700 1 -- 172.21.15.99:0/4011197308 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faeb00cf820 msgr2=0x7faeb0164a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:17.582 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.578+0000 7faeaf7fe700 1 --2- 172.21.15.99:0/4011197308 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faeb00cf820 0x7faeb0164a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:17.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.578+0000 7faeaf7fe700 1 -- 172.21.15.99:0/4011197308 >> v1:172.21.15.191:6789/0 conn(0x7faeb00cc940 legacy=0x7faeb00ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:17.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.578+0000 7faeaf7fe700 1 -- 172.21.15.99:0/4011197308 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb00cc140 con 0x7faeb015d240 2023-10-08T00:22:17.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.578+0000 7faeaf7fe700 1 -- 172.21.15.99:0/4011197308 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1078567439 0 0) 0x7faea00015f0 con 0x7faeb015d240 2023-10-08T00:22:17.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.579+0000 7faeaf7fe700 1 -- 172.21.15.99:0/4011197308 >> v1:172.21.15.99:6789/0 conn(0x7faeb015d240 legacy=0x7faeb00b3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:17.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.579+0000 7faeaf7fe700 1 --2- 172.21.15.99:0/4011197308 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fae9c003a90 0x7fae9c003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:17.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.579+0000 7faeaf7fe700 1 --2- 172.21.15.99:0/4011197308 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faeb00cf820 0x7fae9c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:17.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.579+0000 7faeaf7fe700 1 --2- 172.21.15.99:0/4011197308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae9c004d50 0x7fae9c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:17.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.579+0000 7faeaf7fe700 1 -- 172.21.15.99:0/4011197308 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb00cbe60 con 0x7fae9c004d50 2023-10-08T00:22:17.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.579+0000 7faeaf7fe700 1 -- 172.21.15.99:0/4011197308 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c00bba0 con 0x7fae9c003a90 2023-10-08T00:22:17.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.579+0000 7faeaf7fe700 1 -- 172.21.15.99:0/4011197308 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c00bd60 con 0x7faeb00cf820 2023-10-08T00:22:17.583 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.579+0000 7faeb5473700 1 --2- 172.21.15.99:0/4011197308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae9c004d50 0x7fae9c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:17.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.579+0000 7faeb4c72700 1 --2- 172.21.15.99:0/4011197308 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fae9c003a90 0x7fae9c003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:17.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.579+0000 7faeaffff700 1 --2- 172.21.15.99:0/4011197308 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faeb00cf820 0x7fae9c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:17.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.579+0000 7faeb5473700 1 -- 172.21.15.99:0/4011197308 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faeb00cf820 msgr2=0x7fae9c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:17.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.579+0000 7faeb5473700 1 --2- 172.21.15.99:0/4011197308 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faeb00cf820 0x7fae9c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:17.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.579+0000 7faeb5473700 1 -- 172.21.15.99:0/4011197308 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fae9c003a90 msgr2=0x7fae9c003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:17.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.579+0000 7faeb5473700 1 --2- 172.21.15.99:0/4011197308 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fae9c003a90 0x7fae9c003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:17.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.579+0000 7faeb5473700 1 -- 172.21.15.99:0/4011197308 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae9c003570 con 0x7fae9c004d50 2023-10-08T00:22:17.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.579+0000 7faeb5473700 1 --2- 172.21.15.99:0/4011197308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae9c004d50 0x7fae9c0051c0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7faea8009c00 tx=0x7faea80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b03f644562a9aaa server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:17.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.579+0000 7faeaf7fe700 1 -- 172.21.15.99:0/4011197308 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea800e710 con 0x7fae9c004d50 2023-10-08T00:22:17.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.579+0000 7faeaf7fe700 1 -- 172.21.15.99:0/4011197308 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faea800e870 con 0x7fae9c004d50 2023-10-08T00:22:17.584 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.579+0000 7faeaf7fe700 1 -- 172.21.15.99:0/4011197308 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea800ec30 con 0x7fae9c004d50 2023-10-08T00:22:17.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.580+0000 7faeb5c74700 1 -- 172.21.15.99:0/4011197308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae9c004d50 msgr2=0x7fae9c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:17.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.580+0000 7faeb5c74700 1 --2- 172.21.15.99:0/4011197308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae9c004d50 0x7fae9c0051c0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7faea8009c00 tx=0x7faea80091a0 comp rx=0 tx=0).stop 2023-10-08T00:22:17.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.580+0000 7faeb5473700 1 -- 172.21.15.99:0/4011197308 reap_dead start 2023-10-08T00:22:17.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.580+0000 7faeb5c74700 1 -- 172.21.15.99:0/4011197308 shutdown_connections 2023-10-08T00:22:17.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.580+0000 7faeb5c74700 1 -- 172.21.15.99:0/4011197308 >> 172.21.15.99:0/4011197308 conn(0x7faeb0067e40 msgr2=0x7faeb00d0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:17.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.580+0000 7faeb5c74700 1 -- 172.21.15.99:0/4011197308 shutdown_connections 2023-10-08T00:22:17.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.580+0000 7faeb5c74700 1 -- 172.21.15.99:0/4011197308 wait complete. 2023-10-08T00:22:17.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.581+0000 7faeb5c74700 1 Processor -- start 2023-10-08T00:22:17.585 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.581+0000 7faeb5c74700 1 -- start start 2023-10-08T00:22:17.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.582+0000 7faeb5c74700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae9c004d50 0x7faeb00ca4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:17.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.582+0000 7faeb5c74700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faeb00ca9e0 0x7faeb016da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:17.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.582+0000 7faeb5c74700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faeb016dfd0 0x7faeb009bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:17.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.582+0000 7faeb5c74700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb015a340 con 0x7faeb00ca9e0 2023-10-08T00:22:17.588 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.582+0000 7faeb5c74700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb00c8700 con 0x7faeb016dfd0 2023-10-08T00:22:17.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.582+0000 7faeb5c74700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeb00c89a0 con 0x7fae9c004d50 2023-10-08T00:22:17.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.582+0000 7faeaffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faeb00ca9e0 0x7faeb016da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:17.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.582+0000 7faeaffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faeb00ca9e0 0x7faeb016da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33458/0 (socket says 172.21.15.99:33458) 2023-10-08T00:22:17.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.582+0000 7faeaffff700 1 -- 172.21.15.99:0/3695208230 learned_addr learned my addr 172.21.15.99:0/3695208230 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:17.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.582+0000 7faeb4c72700 1 --2- 172.21.15.99:0/3695208230 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae9c004d50 0x7faeb00ca4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:17.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.582+0000 7faeb5473700 1 --2- 172.21.15.99:0/3695208230 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faeb016dfd0 0x7faeb009bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:17.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.582+0000 7faeaffff700 1 -- 172.21.15.99:0/3695208230 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae9c004d50 msgr2=0x7faeb00ca4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:17.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.582+0000 7faeaffff700 1 --2- 172.21.15.99:0/3695208230 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae9c004d50 0x7faeb00ca4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:17.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.582+0000 7faeaffff700 1 -- 172.21.15.99:0/3695208230 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faeb016dfd0 msgr2=0x7faeb009bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:17.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.582+0000 7faeaffff700 1 --2- 172.21.15.99:0/3695208230 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faeb016dfd0 0x7faeb009bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:17.589 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.582+0000 7faeaffff700 1 -- 172.21.15.99:0/3695208230 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb00c8c80 con 0x7faeb00ca9e0 2023-10-08T00:22:17.590 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.582+0000 7faeaffff700 1 --2- 172.21.15.99:0/3695208230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faeb00ca9e0 0x7faeb016da90 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fae98007de0 tx=0x7fae980016e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:17.590 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.582+0000 7faeacff9700 1 -- 172.21.15.99:0/3695208230 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9800ee70 con 0x7faeb00ca9e0 2023-10-08T00:22:17.590 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.582+0000 7faeacff9700 1 -- 172.21.15.99:0/3695208230 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae9800c7e0 con 0x7faeb00ca9e0 2023-10-08T00:22:17.590 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.582+0000 7faeb5c74700 1 -- 172.21.15.99:0/3695208230 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeb00c8f60 con 0x7faeb00ca9e0 2023-10-08T00:22:17.590 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.583+0000 7faeacff9700 1 -- 172.21.15.99:0/3695208230 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9800cba0 con 0x7faeb00ca9e0 2023-10-08T00:22:17.590 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.583+0000 7faeb5c74700 1 -- 172.21.15.99:0/3695208230 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeb00c9520 con 0x7faeb00ca9e0 2023-10-08T00:22:17.590 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.583+0000 7faeb5c74700 1 -- 172.21.15.99:0/3695208230 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7faeb006ac60 con 0x7faeb00ca9e0 2023-10-08T00:22:17.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.584+0000 7faeacff9700 1 -- 172.21.15.99:0/3695208230 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fae9801f460 con 0x7faeb00ca9e0 2023-10-08T00:22:17.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.584+0000 7faeacff9700 1 --2- 172.21.15.99:0/3695208230 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fae90066f30 0x7fae900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:17.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.584+0000 7faeacff9700 1 -- 172.21.15.99:0/3695208230 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fae980848d0 con 0x7faeb00ca9e0 2023-10-08T00:22:17.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.584+0000 7faeacff9700 1 --2- 172.21.15.99:0/3695208230 >> [v2:172.21.15.191:6800/86570,v1:172.21.15.191:6801/86570] conn(0x7fae9006cd80 0x7fae9006f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:17.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.584+0000 7faeacff9700 1 -- 172.21.15.99:0/3695208230 --> [v2:172.21.15.191:6800/86570,v1:172.21.15.191:6801/86570] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fae9006f860 con 0x7fae9006cd80 2023-10-08T00:22:17.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.584+0000 7faeacff9700 1 -- 172.21.15.99:0/3695208230 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=192) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae98089240 con 0x7faeb00ca9e0 2023-10-08T00:22:17.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.584+0000 7faeb4c72700 1 --2- 172.21.15.99:0/3695208230 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fae90066f30 0x7fae900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:17.591 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.584+0000 7faeb5473700 1 --2- 172.21.15.99:0/3695208230 >> [v2:172.21.15.191:6800/86570,v1:172.21.15.191:6801/86570] conn(0x7fae9006cd80 0x7fae9006f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:17.592 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.585+0000 7faeb4c72700 1 --2- 172.21.15.99:0/3695208230 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fae90066f30 0x7fae900693e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7faea0001260 tx=0x7faea0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:17.592 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.585+0000 7faeb5473700 1 --2- 172.21.15.99:0/3695208230 >> [v2:172.21.15.191:6800/86570,v1:172.21.15.191:6801/86570] conn(0x7fae9006cd80 0x7fae9006f190 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:17.592 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.586+0000 7faeacff9700 1 -- 172.21.15.99:0/3695208230 <== osd.4 v2:172.21.15.191:6800/86570 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22340 (crc 0 0 0) 0x7fae9006f860 con 0x7fae9006cd80 2023-10-08T00:22:17.598 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.598+0000 7faeb5c74700 1 -- 172.21.15.99:0/3695208230 --> [v2:172.21.15.191:6800/86570,v1:172.21.15.191:6801/86570] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7faeb00cc420 con 0x7fae9006cd80 2023-10-08T00:22:17.599 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.598+0000 7faeacff9700 1 -- 172.21.15.99:0/3695208230 <== osd.4 v2:172.21.15.191:6800/86570 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+682 (crc 0 0 0) 0x7faeb00cc420 con 0x7fae9006cd80 2023-10-08T00:22:17.599 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:22:17.599 INFO:teuthology.orchestra.run.smithi099.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2023-10-08T00:22:17.599 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:22:17.600 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.599+0000 7faeb5c74700 1 -- 172.21.15.99:0/3695208230 >> [v2:172.21.15.191:6800/86570,v1:172.21.15.191:6801/86570] conn(0x7fae9006cd80 msgr2=0x7fae9006f190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:17.601 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.599+0000 7faeb5c74700 1 --2- 172.21.15.99:0/3695208230 >> [v2:172.21.15.191:6800/86570,v1:172.21.15.191:6801/86570] conn(0x7fae9006cd80 0x7fae9006f190 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:17.601 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.599+0000 7faeb5c74700 1 -- 172.21.15.99:0/3695208230 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fae90066f30 msgr2=0x7fae900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:17.601 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.599+0000 7faeb5c74700 1 --2- 172.21.15.99:0/3695208230 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7fae90066f30 0x7fae900693e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7faea0001260 tx=0x7faea0009040 comp rx=0 tx=0).stop 2023-10-08T00:22:17.601 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.599+0000 7faeb5c74700 1 -- 172.21.15.99:0/3695208230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faeb00ca9e0 msgr2=0x7faeb016da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:17.601 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.599+0000 7faeb5c74700 1 --2- 172.21.15.99:0/3695208230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faeb00ca9e0 0x7faeb016da90 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fae98007de0 tx=0x7fae980016e0 comp rx=0 tx=0).stop 2023-10-08T00:22:17.601 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.599+0000 7faeb5473700 1 -- 172.21.15.99:0/3695208230 reap_dead start 2023-10-08T00:22:17.601 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.599+0000 7faeb5c74700 1 -- 172.21.15.99:0/3695208230 shutdown_connections 2023-10-08T00:22:17.601 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.599+0000 7faeb5c74700 1 -- 172.21.15.99:0/3695208230 >> 172.21.15.99:0/3695208230 conn(0x7faeb0067e40 msgr2=0x7faeb00ce170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:17.601 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.599+0000 7faeb5c74700 1 -- 172.21.15.99:0/3695208230 shutdown_connections 2023-10-08T00:22:17.602 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.599+0000 7faeb5c74700 1 -- 172.21.15.99:0/3695208230 wait complete. 2023-10-08T00:22:17.611 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-10-08T00:22:17.725 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.724+0000 7f4ed5b95700 1 Processor -- start 2023-10-08T00:22:17.726 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.724+0000 7f4ed5b95700 1 -- start start 2023-10-08T00:22:17.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.725+0000 7f4ed5b95700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4ed015a2b0 0x7f4ed00b3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:17.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.725+0000 7f4ed5b95700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4ed00d1c30 0x7f4ed00cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:17.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.725+0000 7f4ed5b95700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ed00c5d80 con 0x7f4ed015a680 2023-10-08T00:22:17.728 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.725+0000 7f4ed5b95700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ed0068470 con 0x7f4ed015a2b0 2023-10-08T00:22:17.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.725+0000 7f4ed5b95700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed00c6060 con 0x7f4ed00d1c30 2023-10-08T00:22:17.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.726+0000 7f4ecf7fe700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4ed015a2b0 0x7f4ed00b3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:17.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.726+0000 7f4eceffd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4ed00d1c30 0x7f4ed00cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:17.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.726+0000 7f4ecffff700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f4ed015a680 0x7f4ed00c7100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:54906/0 (socket says 172.21.15.99:54906) 2023-10-08T00:22:17.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.726+0000 7f4ecf7fe700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4ed015a2b0 0x7f4ed00b3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.99:46412/0 (socket says 172.21.15.99:46412) 2023-10-08T00:22:17.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.726+0000 7f4ecffff700 1 -- 172.21.15.99:0/3972399205 learned_addr learned my addr 172.21.15.99:0/3972399205 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:22:17.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.726+0000 7f4eceffd700 1 -- 172.21.15.99:0/3972399205 >> v2:172.21.15.191:3300/0 conn(0x7f4ed015a2b0 msgr2=0x7f4ed00b3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:17.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.726+0000 7f4eceffd700 1 --2- 172.21.15.99:0/3972399205 >> v2:172.21.15.191:3300/0 conn(0x7f4ed015a2b0 0x7f4ed00b3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:17.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.726+0000 7f4eceffd700 1 -- 172.21.15.99:0/3972399205 >> v1:172.21.15.191:6789/0 conn(0x7f4ed015a680 legacy=0x7f4ed00c7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:17.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.726+0000 7f4eceffd700 1 -- 172.21.15.99:0/3972399205 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed00c6340 con 0x7f4ed00d1c30 2023-10-08T00:22:17.729 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.726+0000 7f4ecf7fe700 1 --2- 172.21.15.99:0/3972399205 >> v2:172.21.15.191:3300/0 conn(0x7f4ed015a2b0 0x7f4ed00b3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:22:17.730 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.726+0000 7f4eceffd700 1 --2- 172.21.15.99:0/3972399205 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4ed00d1c30 0x7f4ed00cf710 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4ec40098e0 tx=0x7f4ec40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3541b9ac070369a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:17.730 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.726+0000 7f4ece7fc700 1 -- 172.21.15.99:0/3972399205 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec400b640 con 0x7f4ed00d1c30 2023-10-08T00:22:17.730 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.726+0000 7f4ece7fc700 1 -- 172.21.15.99:0/3972399205 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec400b7a0 con 0x7f4ed00d1c30 2023-10-08T00:22:17.730 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.726+0000 7f4ece7fc700 1 -- 172.21.15.99:0/3972399205 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec400bb40 con 0x7f4ed00d1c30 2023-10-08T00:22:17.730 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.727+0000 7f4ed5b95700 1 -- 172.21.15.99:0/3972399205 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4ed00d1c30 msgr2=0x7f4ed00cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:17.730 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.727+0000 7f4ed5b95700 1 --2- 172.21.15.99:0/3972399205 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4ed00d1c30 0x7f4ed00cf710 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4ec40098e0 tx=0x7f4ec40091a0 comp rx=0 tx=0).stop 2023-10-08T00:22:17.730 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.727+0000 7f4ed5b95700 1 -- 172.21.15.99:0/3972399205 shutdown_connections 2023-10-08T00:22:17.730 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.727+0000 7f4ed5b95700 1 --2- 172.21.15.99:0/3972399205 >> v2:172.21.15.191:3300/0 conn(0x7f4ed015a2b0 0x7f4ed00b3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:17.730 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.727+0000 7f4ed5b95700 1 --2- 172.21.15.99:0/3972399205 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4ed00d1c30 0x7f4ed00cf710 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:17.730 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.727+0000 7f4ed5b95700 1 -- 172.21.15.99:0/3972399205 >> 172.21.15.99:0/3972399205 conn(0x7f4ed0067e40 msgr2=0x7f4ed00d0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:17.730 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.727+0000 7f4ed5b95700 1 -- 172.21.15.99:0/3972399205 shutdown_connections 2023-10-08T00:22:17.730 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.727+0000 7f4ed5b95700 1 -- 172.21.15.99:0/3972399205 wait complete. 2023-10-08T00:22:17.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.728+0000 7f4ed5b95700 1 Processor -- start 2023-10-08T00:22:17.733 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.728+0000 7f4ed5b95700 1 -- start start 2023-10-08T00:22:17.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.729+0000 7f4ed5b95700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4ed00d1c30 0x7f4ed015ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:17.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.729+0000 7f4ed5b95700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ed015a2b0 0x7f4ed014aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:17.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.729+0000 7f4ed5b95700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ed015a680 0x7f4ed015c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:17.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.729+0000 7f4ed5b95700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed014afe0 con 0x7f4ed015a2b0 2023-10-08T00:22:17.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.729+0000 7f4ed5b95700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed0160700 con 0x7f4ed015a680 2023-10-08T00:22:17.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.729+0000 7f4ed5b95700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed01609e0 con 0x7f4ed00d1c30 2023-10-08T00:22:17.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.729+0000 7f4eceffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ed015a2b0 0x7f4ed014aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:17.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.729+0000 7f4ecf7fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4ed00d1c30 0x7f4ed015ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:17.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.729+0000 7f4eceffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ed015a2b0 0x7f4ed014aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33466/0 (socket says 172.21.15.99:33466) 2023-10-08T00:22:17.734 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.729+0000 7f4eceffd700 1 -- 172.21.15.99:0/4111363631 learned_addr learned my addr 172.21.15.99:0/4111363631 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:17.735 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.729+0000 7f4ecffff700 1 --2- 172.21.15.99:0/4111363631 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ed015a680 0x7f4ed015c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:17.735 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.729+0000 7f4eceffd700 1 -- 172.21.15.99:0/4111363631 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4ed00d1c30 msgr2=0x7f4ed015ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:17.735 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.729+0000 7f4eceffd700 1 --2- 172.21.15.99:0/4111363631 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4ed00d1c30 0x7f4ed015ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:17.735 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.729+0000 7f4eceffd700 1 -- 172.21.15.99:0/4111363631 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ed015a680 msgr2=0x7f4ed015c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:17.735 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.729+0000 7f4eceffd700 1 --2- 172.21.15.99:0/4111363631 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ed015a680 0x7f4ed015c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:17.735 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.729+0000 7f4eceffd700 1 -- 172.21.15.99:0/4111363631 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec00093c0 con 0x7f4ed015a2b0 2023-10-08T00:22:17.735 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.729+0000 7f4ecf7fe700 1 --2- 172.21.15.99:0/4111363631 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4ed00d1c30 0x7f4ed015ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:22:17.735 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.729+0000 7f4eceffd700 1 --2- 172.21.15.99:0/4111363631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ed015a2b0 0x7f4ed014aaa0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f4ec400a010 tx=0x7f4ec4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:17.735 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.730+0000 7f4eb77fe700 1 -- 172.21.15.99:0/4111363631 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec401f070 con 0x7f4ed015a2b0 2023-10-08T00:22:17.735 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.730+0000 7f4eb77fe700 1 -- 172.21.15.99:0/4111363631 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec401a950 con 0x7f4ed015a2b0 2023-10-08T00:22:17.735 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.730+0000 7f4ed5b95700 1 -- 172.21.15.99:0/4111363631 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ec4009420 con 0x7f4ed015a2b0 2023-10-08T00:22:17.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.730+0000 7f4eb77fe700 1 -- 172.21.15.99:0/4111363631 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec401acf0 con 0x7f4ed015a2b0 2023-10-08T00:22:17.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.730+0000 7f4ed5b95700 1 -- 172.21.15.99:0/4111363631 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ed00b4770 con 0x7f4ed015a2b0 2023-10-08T00:22:17.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.730+0000 7f4ed5b95700 1 -- 172.21.15.99:0/4111363631 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ed004c3c0 con 0x7f4ed015a2b0 2023-10-08T00:22:17.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.731+0000 7f4eb77fe700 1 -- 172.21.15.99:0/4111363631 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4ec4023070 con 0x7f4ed015a2b0 2023-10-08T00:22:17.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.731+0000 7f4eb77fe700 1 --2- 172.21.15.99:0/4111363631 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4ebc066f30 0x7f4ebc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:17.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.731+0000 7f4eb77fe700 1 -- 172.21.15.99:0/4111363631 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f4ec4085520 con 0x7f4ed015a2b0 2023-10-08T00:22:17.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.732+0000 7f4ecf7fe700 1 --2- 172.21.15.99:0/4111363631 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4ebc066f30 0x7f4ebc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:17.736 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.732+0000 7f4ecf7fe700 1 --2- 172.21.15.99:0/4111363631 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4ebc066f30 0x7f4ebc0693e0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0011560 tx=0x7f4ec0009120 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:17.737 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.735+0000 7f4eb77fe700 1 -- 172.21.15.99:0/4111363631 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4ec4052a90 con 0x7f4ed015a2b0 2023-10-08T00:22:17.917 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.916+0000 7f4ed5b95700 1 -- 172.21.15.99:0/4111363631 --> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ed0064a00 con 0x7f4ebc066f30 2023-10-08T00:22:17.917 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.917+0000 7f4eb77fe700 1 -- 172.21.15.99:0/4111363631 <== mgr.4099 v2:172.21.15.191:6832/85985 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4ed0064a00 con 0x7f4ebc066f30 2023-10-08T00:22:17.920 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.919+0000 7f4ed5b95700 1 -- 172.21.15.99:0/4111363631 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4ebc066f30 msgr2=0x7f4ebc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:17.920 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.919+0000 7f4ed5b95700 1 --2- 172.21.15.99:0/4111363631 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4ebc066f30 0x7f4ebc0693e0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0011560 tx=0x7f4ec0009120 comp rx=0 tx=0).stop 2023-10-08T00:22:17.921 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.919+0000 7f4ed5b95700 1 -- 172.21.15.99:0/4111363631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ed015a2b0 msgr2=0x7f4ed014aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:17.921 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.919+0000 7f4ed5b95700 1 --2- 172.21.15.99:0/4111363631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ed015a2b0 0x7f4ed014aaa0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f4ec400a010 tx=0x7f4ec4017000 comp rx=0 tx=0).stop 2023-10-08T00:22:17.921 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.919+0000 7f4ed5b95700 1 -- 172.21.15.99:0/4111363631 shutdown_connections 2023-10-08T00:22:17.921 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.919+0000 7f4ed5b95700 1 --2- 172.21.15.99:0/4111363631 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4ebc066f30 0x7f4ebc0693e0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:17.921 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.919+0000 7f4ed5b95700 1 --2- 172.21.15.99:0/4111363631 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ed015a680 0x7f4ed015c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:17.921 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.919+0000 7f4ed5b95700 1 --2- 172.21.15.99:0/4111363631 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4ed00d1c30 0x7f4ed015ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:17.921 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.919+0000 7f4ed5b95700 1 --2- 172.21.15.99:0/4111363631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ed015a2b0 0x7f4ed014aaa0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:17.921 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.919+0000 7f4ed5b95700 1 -- 172.21.15.99:0/4111363631 >> 172.21.15.99:0/4111363631 conn(0x7f4ed0067e40 msgr2=0x7f4ed00cf710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:17.921 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.919+0000 7f4ed5b95700 1 -- 172.21.15.99:0/4111363631 shutdown_connections 2023-10-08T00:22:17.922 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:17.919+0000 7f4ed5b95700 1 -- 172.21.15.99:0/4111363631 wait complete. 2023-10-08T00:22:17.922 INFO:teuthology.orchestra.run.smithi099.stderr:instructed osd(s) 4 to deep-scrub 2023-10-08T00:22:17.932 INFO:tasks.ceph:Scrubbing osd.5 2023-10-08T00:22:17.932 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-10-08T00:22:18.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.049+0000 7f7532c80700 1 Processor -- start 2023-10-08T00:22:18.050 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.049+0000 7f7532c80700 1 -- start start 2023-10-08T00:22:18.059 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.050+0000 7f7532c80700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f752c156a10 0x7f752c15b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:18.059 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.050+0000 7f7532c80700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f752c156de0 0x7f752c0cc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:18.059 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.050+0000 7f7532c80700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f752c06e320 0x7f752c0cc650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:18.059 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.050+0000 7f7532c80700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f752c06f4f0 con 0x7f752c06e320 2023-10-08T00:22:18.059 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.050+0000 7f7532c80700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f752c15b700 con 0x7f752c156de0 2023-10-08T00:22:18.059 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.050+0000 7f7532c80700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f752c06e8e0 con 0x7f752c156a10 2023-10-08T00:22:18.059 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.050+0000 7f753121d700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f752c06e320 0x7f752c0cc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:18.060 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.050+0000 7f753121d700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f752c06e320 0x7f752c0cc650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33478/0 (socket says 172.21.15.99:33478) 2023-10-08T00:22:18.060 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.050+0000 7f753121d700 1 -- 172.21.15.99:0/1577479085 learned_addr learned my addr 172.21.15.99:0/1577479085 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:18.060 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.050+0000 7f7530a1c700 1 --2- 172.21.15.99:0/1577479085 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f752c156a10 0x7f752c15b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:18.060 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.050+0000 7f752bfff700 1 --2- 172.21.15.99:0/1577479085 >> v2:172.21.15.191:3300/0 conn(0x7f752c156de0 0x7f752c0cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:18.060 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.051+0000 7f753121d700 1 -- 172.21.15.99:0/1577479085 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f752c156a10 msgr2=0x7f752c15b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:18.060 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.051+0000 7f753121d700 1 --2- 172.21.15.99:0/1577479085 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f752c156a10 0x7f752c15b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:18.060 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.051+0000 7f753121d700 1 -- 172.21.15.99:0/1577479085 >> v2:172.21.15.191:3300/0 conn(0x7f752c156de0 msgr2=0x7f752c0cc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:18.060 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.051+0000 7f753121d700 1 --2- 172.21.15.99:0/1577479085 >> v2:172.21.15.191:3300/0 conn(0x7f752c156de0 0x7f752c0cc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:18.060 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.051+0000 7f753121d700 1 -- 172.21.15.99:0/1577479085 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f752c06ebc0 con 0x7f752c06e320 2023-10-08T00:22:18.060 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.051+0000 7f752bfff700 1 --2- 172.21.15.99:0/1577479085 >> v2:172.21.15.191:3300/0 conn(0x7f752c156de0 0x7f752c0cc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:22:18.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.051+0000 7f7530a1c700 1 --2- 172.21.15.99:0/1577479085 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f752c156a10 0x7f752c15b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:22:18.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.051+0000 7f753121d700 1 --2- 172.21.15.99:0/1577479085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f752c06e320 0x7f752c0cc650 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f752c072050 tx=0x7f7524009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26bf74e6ac2a692c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:18.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.051+0000 7f752b7fe700 1 -- 172.21.15.99:0/1577479085 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752400e070 con 0x7f752c06e320 2023-10-08T00:22:18.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.051+0000 7f752b7fe700 1 -- 172.21.15.99:0/1577479085 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f752400ba50 con 0x7f752c06e320 2023-10-08T00:22:18.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.051+0000 7f752b7fe700 1 -- 172.21.15.99:0/1577479085 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75240092e0 con 0x7f752c06e320 2023-10-08T00:22:18.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.051+0000 7f7532c80700 1 -- 172.21.15.99:0/1577479085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f752c06e320 msgr2=0x7f752c0cc650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:18.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.051+0000 7f7532c80700 1 --2- 172.21.15.99:0/1577479085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f752c06e320 0x7f752c0cc650 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f752c072050 tx=0x7f7524009840 comp rx=0 tx=0).stop 2023-10-08T00:22:18.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.051+0000 7f7532c80700 1 -- 172.21.15.99:0/1577479085 shutdown_connections 2023-10-08T00:22:18.061 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.051+0000 7f7532c80700 1 --2- 172.21.15.99:0/1577479085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f752c06e320 0x7f752c0cc650 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:18.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.051+0000 7f7532c80700 1 --2- 172.21.15.99:0/1577479085 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f752c156a10 0x7f752c15b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:18.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.051+0000 7f7532c80700 1 --2- 172.21.15.99:0/1577479085 >> v2:172.21.15.191:3300/0 conn(0x7f752c156de0 0x7f752c0cc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:18.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.051+0000 7f7532c80700 1 -- 172.21.15.99:0/1577479085 >> 172.21.15.99:0/1577479085 conn(0x7f752c0ce5c0 msgr2=0x7f752c0ca5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:18.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.051+0000 7f7532c80700 1 -- 172.21.15.99:0/1577479085 shutdown_connections 2023-10-08T00:22:18.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.051+0000 7f7532c80700 1 -- 172.21.15.99:0/1577479085 wait complete. 2023-10-08T00:22:18.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.052+0000 7f7532c80700 1 Processor -- start 2023-10-08T00:22:18.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.053+0000 7f7532c80700 1 -- start start 2023-10-08T00:22:18.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.053+0000 7f7532c80700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f752c06e320 0x7f752c0b3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:18.062 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.053+0000 7f7532c80700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f752c156a10 0x7f752c143f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:18.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.053+0000 7f7532c80700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f752c156de0 0x7f752c144460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:18.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.053+0000 7f7532c80700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752c169090 con 0x7f752c156a10 2023-10-08T00:22:18.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.053+0000 7f7532c80700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752c0b3000 con 0x7f752c06e320 2023-10-08T00:22:18.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.053+0000 7f7532c80700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f752c168480 con 0x7f752c156de0 2023-10-08T00:22:18.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.053+0000 7f752bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f752c156a10 0x7f752c143f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:18.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.053+0000 7f752bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f752c156a10 0x7f752c143f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33490/0 (socket says 172.21.15.99:33490) 2023-10-08T00:22:18.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.054+0000 7f752bfff700 1 -- 172.21.15.99:0/3792757980 learned_addr learned my addr 172.21.15.99:0/3792757980 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:18.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.054+0000 7f7530a1c700 1 --2- 172.21.15.99:0/3792757980 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f752c06e320 0x7f752c0b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:18.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.054+0000 7f753121d700 1 --2- 172.21.15.99:0/3792757980 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f752c156de0 0x7f752c144460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:18.063 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.054+0000 7f752bfff700 1 -- 172.21.15.99:0/3792757980 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f752c156de0 msgr2=0x7f752c144460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:18.064 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.054+0000 7f752bfff700 1 --2- 172.21.15.99:0/3792757980 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f752c156de0 0x7f752c144460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:18.064 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.054+0000 7f752bfff700 1 -- 172.21.15.99:0/3792757980 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f752c06e320 msgr2=0x7f752c0b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:18.064 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.054+0000 7f752bfff700 1 --2- 172.21.15.99:0/3792757980 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f752c06e320 0x7f752c0b3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:18.064 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.054+0000 7f752bfff700 1 -- 172.21.15.99:0/3792757980 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7524009ae0 con 0x7f752c156a10 2023-10-08T00:22:18.064 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.054+0000 7f752bfff700 1 --2- 172.21.15.99:0/3792757980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f752c156a10 0x7f752c143f20 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f751800e880 tx=0x7f7518008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:18.064 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.054+0000 7f7528ff9700 1 -- 172.21.15.99:0/3792757980 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7518013410 con 0x7f752c156a10 2023-10-08T00:22:18.064 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.054+0000 7f7528ff9700 1 -- 172.21.15.99:0/3792757980 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7518013570 con 0x7f752c156a10 2023-10-08T00:22:18.064 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.054+0000 7f7532c80700 1 -- 172.21.15.99:0/3792757980 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f752c168760 con 0x7f752c156a10 2023-10-08T00:22:18.064 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.054+0000 7f7528ff9700 1 -- 172.21.15.99:0/3792757980 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7518013910 con 0x7f752c156a10 2023-10-08T00:22:18.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.054+0000 7f7532c80700 1 -- 172.21.15.99:0/3792757980 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f752c168d20 con 0x7f752c156a10 2023-10-08T00:22:18.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.055+0000 7f7532c80700 1 -- 172.21.15.99:0/3792757980 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f752c04c3c0 con 0x7f752c156a10 2023-10-08T00:22:18.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.055+0000 7f7528ff9700 1 -- 172.21.15.99:0/3792757980 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7518013cc0 con 0x7f752c156a10 2023-10-08T00:22:18.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.056+0000 7f7528ff9700 1 --2- 172.21.15.99:0/3792757980 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7510066f30 0x7f75100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:18.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.056+0000 7f7528ff9700 1 -- 172.21.15.99:0/3792757980 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f7518083ff0 con 0x7f752c156a10 2023-10-08T00:22:18.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.056+0000 7f7528ff9700 1 --2- 172.21.15.99:0/3792757980 >> [v2:172.21.15.191:6816/86608,v1:172.21.15.191:6818/86608] conn(0x7f751006cd80 0x7f751006f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:18.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.056+0000 7f7528ff9700 1 -- 172.21.15.99:0/3792757980 --> [v2:172.21.15.191:6816/86608,v1:172.21.15.191:6818/86608] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f751006f860 con 0x7f751006cd80 2023-10-08T00:22:18.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.056+0000 7f7528ff9700 1 -- 172.21.15.99:0/3792757980 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=192) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7518086630 con 0x7f752c156a10 2023-10-08T00:22:18.065 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.056+0000 7f7530a1c700 1 --2- 172.21.15.99:0/3792757980 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7510066f30 0x7f75100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:18.066 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.056+0000 7f753121d700 1 --2- 172.21.15.99:0/3792757980 >> [v2:172.21.15.191:6816/86608,v1:172.21.15.191:6818/86608] conn(0x7f751006cd80 0x7f751006f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:18.066 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.056+0000 7f7530a1c700 1 --2- 172.21.15.99:0/3792757980 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7510066f30 0x7f75100693e0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7520006f00 tx=0x7f7520008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:18.066 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.057+0000 7f753121d700 1 --2- 172.21.15.99:0/3792757980 >> [v2:172.21.15.191:6816/86608,v1:172.21.15.191:6818/86608] conn(0x7f751006cd80 0x7f751006f190 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:18.066 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.057+0000 7f7528ff9700 1 -- 172.21.15.99:0/3792757980 <== osd.5 v2:172.21.15.191:6816/86608 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22340 (crc 0 0 0) 0x7f751006f860 con 0x7f751006cd80 2023-10-08T00:22:18.070 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.070+0000 7f7532c80700 1 -- 172.21.15.99:0/3792757980 --> [v2:172.21.15.191:6816/86608,v1:172.21.15.191:6818/86608] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f752c06eea0 con 0x7f751006cd80 2023-10-08T00:22:18.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.070+0000 7f7528ff9700 1 -- 172.21.15.99:0/3792757980 <== osd.5 v2:172.21.15.191:6816/86608 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+682 (crc 0 0 0) 0x7f752c06eea0 con 0x7f751006cd80 2023-10-08T00:22:18.071 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:22:18.071 INFO:teuthology.orchestra.run.smithi099.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2023-10-08T00:22:18.072 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:22:18.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.070+0000 7f7532c80700 1 -- 172.21.15.99:0/3792757980 >> [v2:172.21.15.191:6816/86608,v1:172.21.15.191:6818/86608] conn(0x7f751006cd80 msgr2=0x7f751006f190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:18.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.070+0000 7f7532c80700 1 --2- 172.21.15.99:0/3792757980 >> [v2:172.21.15.191:6816/86608,v1:172.21.15.191:6818/86608] conn(0x7f751006cd80 0x7f751006f190 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:18.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.071+0000 7f7532c80700 1 -- 172.21.15.99:0/3792757980 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7510066f30 msgr2=0x7f75100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:18.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.071+0000 7f7532c80700 1 --2- 172.21.15.99:0/3792757980 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f7510066f30 0x7f75100693e0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7520006f00 tx=0x7f7520008040 comp rx=0 tx=0).stop 2023-10-08T00:22:18.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.071+0000 7f7532c80700 1 -- 172.21.15.99:0/3792757980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f752c156a10 msgr2=0x7f752c143f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:18.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.071+0000 7f7532c80700 1 --2- 172.21.15.99:0/3792757980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f752c156a10 0x7f752c143f20 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f751800e880 tx=0x7f7518008040 comp rx=0 tx=0).stop 2023-10-08T00:22:18.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.071+0000 7f753121d700 1 -- 172.21.15.99:0/3792757980 reap_dead start 2023-10-08T00:22:18.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.071+0000 7f7532c80700 1 -- 172.21.15.99:0/3792757980 shutdown_connections 2023-10-08T00:22:18.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.071+0000 7f7532c80700 1 -- 172.21.15.99:0/3792757980 >> 172.21.15.99:0/3792757980 conn(0x7f752c0ce5c0 msgr2=0x7f752c0c9840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:18.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.071+0000 7f7532c80700 1 -- 172.21.15.99:0/3792757980 shutdown_connections 2023-10-08T00:22:18.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.071+0000 7f7532c80700 1 -- 172.21.15.99:0/3792757980 wait complete. 2023-10-08T00:22:18.083 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-10-08T00:22:18.198 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.198+0000 7f9059969700 1 Processor -- start 2023-10-08T00:22:18.199 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.198+0000 7f9059969700 1 -- start start 2023-10-08T00:22:18.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.198+0000 7f9059969700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f905406f1c0 0x7f90540c8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:18.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.198+0000 7f9059969700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f9054157f30 0x7f90540c9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:18.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.198+0000 7f9059969700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90540b0ef0 con 0x7f9054157b60 2023-10-08T00:22:18.202 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.198+0000 7f9059969700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90540b02e0 con 0x7f9054157f30 2023-10-08T00:22:18.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.198+0000 7f9059969700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90540b05c0 con 0x7f905406f1c0 2023-10-08T00:22:18.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.199+0000 7f9053fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f905406f1c0 0x7f90540c8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:18.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.199+0000 7f9059168700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f9054157f30 0x7f90540c9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:18.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.199+0000 7f9053fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f905406f1c0 0x7f90540c8f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.99:44166/0 (socket says 172.21.15.99:44166) 2023-10-08T00:22:18.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.199+0000 7f9053fff700 1 -- 172.21.15.99:0/3057345882 learned_addr learned my addr 172.21.15.99:0/3057345882 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:18.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.199+0000 7f9058967700 1 --1- 172.21.15.99:0/3057345882 >> v1:172.21.15.191:6789/0 conn(0x7f9054157b60 0x7f90540af870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.99:54912/0 (socket says 172.21.15.99:54912) 2023-10-08T00:22:18.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.199+0000 7f9053fff700 1 -- 172.21.15.99:0/3057345882 >> v2:172.21.15.191:3300/0 conn(0x7f9054157f30 msgr2=0x7f90540c9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:18.203 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.199+0000 7f9053fff700 1 --2- 172.21.15.99:0/3057345882 >> v2:172.21.15.191:3300/0 conn(0x7f9054157f30 0x7f90540c9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:18.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.199+0000 7f9053fff700 1 -- 172.21.15.99:0/3057345882 >> v1:172.21.15.191:6789/0 conn(0x7f9054157b60 legacy=0x7f90540af870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:18.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.199+0000 7f9053fff700 1 -- 172.21.15.99:0/3057345882 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90540b08a0 con 0x7f905406f1c0 2023-10-08T00:22:18.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.199+0000 7f9059168700 1 --2- 172.21.15.99:0/3057345882 >> v2:172.21.15.191:3300/0 conn(0x7f9054157f30 0x7f90540c9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:22:18.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.199+0000 7f9053fff700 1 --2- 172.21.15.99:0/3057345882 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f905406f1c0 0x7f90540c8f00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f90440099b0 tx=0x7f9044009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a806154d2eab48cc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:18.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.199+0000 7f90537fe700 1 -- 172.21.15.99:0/3057345882 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9044009ea0 con 0x7f905406f1c0 2023-10-08T00:22:18.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.200+0000 7f90537fe700 1 -- 172.21.15.99:0/3057345882 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f904400b8f0 con 0x7f905406f1c0 2023-10-08T00:22:18.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.200+0000 7f90537fe700 1 -- 172.21.15.99:0/3057345882 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904400bc90 con 0x7f905406f1c0 2023-10-08T00:22:18.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.200+0000 7f9059969700 1 -- 172.21.15.99:0/3057345882 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f905406f1c0 msgr2=0x7f90540c8f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:18.204 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.200+0000 7f9059969700 1 --2- 172.21.15.99:0/3057345882 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f905406f1c0 0x7f90540c8f00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f90440099b0 tx=0x7f9044009250 comp rx=0 tx=0).stop 2023-10-08T00:22:18.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.200+0000 7f9059969700 1 -- 172.21.15.99:0/3057345882 shutdown_connections 2023-10-08T00:22:18.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.200+0000 7f9059969700 1 --2- 172.21.15.99:0/3057345882 >> v2:172.21.15.191:3300/0 conn(0x7f9054157f30 0x7f90540c9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:18.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.200+0000 7f9059969700 1 --2- 172.21.15.99:0/3057345882 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f905406f1c0 0x7f90540c8f00 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:18.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.200+0000 7f9059969700 1 -- 172.21.15.99:0/3057345882 >> 172.21.15.99:0/3057345882 conn(0x7f9054069870 msgr2=0x7f9054069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:18.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.200+0000 7f9059969700 1 -- 172.21.15.99:0/3057345882 shutdown_connections 2023-10-08T00:22:18.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.200+0000 7f9059969700 1 -- 172.21.15.99:0/3057345882 wait complete. 2023-10-08T00:22:18.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.201+0000 7f9059969700 1 Processor -- start 2023-10-08T00:22:18.205 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.201+0000 7f9059969700 1 -- start start 2023-10-08T00:22:18.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.202+0000 7f9059969700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f905406f1c0 0x7f90540adf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:18.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.202+0000 7f9059969700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9054157b60 0x7f905416b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:18.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.202+0000 7f9059969700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9054157f30 0x7f9054158d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:18.208 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.202+0000 7f9059969700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9054155960 con 0x7f905406f1c0 2023-10-08T00:22:18.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.202+0000 7f9059969700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905416ba90 con 0x7f9054157f30 2023-10-08T00:22:18.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.202+0000 7f9059969700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9054158680 con 0x7f9054157b60 2023-10-08T00:22:18.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.202+0000 7f9058967700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f905406f1c0 0x7f90540adf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:18.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.202+0000 7f9058967700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f905406f1c0 0x7f90540adf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33506/0 (socket says 172.21.15.99:33506) 2023-10-08T00:22:18.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.202+0000 7f9058967700 1 -- 172.21.15.99:0/3234480376 learned_addr learned my addr 172.21.15.99:0/3234480376 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:18.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.202+0000 7f9053fff700 1 --2- 172.21.15.99:0/3234480376 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9054157b60 0x7f905416b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:18.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.202+0000 7f9059168700 1 --2- 172.21.15.99:0/3234480376 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9054157f30 0x7f9054158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:18.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.203+0000 7f9058967700 1 -- 172.21.15.99:0/3234480376 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9054157b60 msgr2=0x7f905416b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:18.209 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.203+0000 7f9058967700 1 --2- 172.21.15.99:0/3234480376 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9054157b60 0x7f905416b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:18.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.203+0000 7f9058967700 1 -- 172.21.15.99:0/3234480376 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9054157f30 msgr2=0x7f9054158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:18.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.203+0000 7f9058967700 1 --2- 172.21.15.99:0/3234480376 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9054157f30 0x7f9054158d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:18.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.203+0000 7f9058967700 1 -- 172.21.15.99:0/3234480376 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f903c009420 con 0x7f905406f1c0 2023-10-08T00:22:18.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.203+0000 7f9058967700 1 --2- 172.21.15.99:0/3234480376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f905406f1c0 0x7f90540adf60 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f904c00cd80 tx=0x7f904c001b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:18.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.203+0000 7f9050ff9700 1 -- 172.21.15.99:0/3234480376 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904c005df0 con 0x7f905406f1c0 2023-10-08T00:22:18.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.203+0000 7f9050ff9700 1 -- 172.21.15.99:0/3234480376 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f904c00a960 con 0x7f905406f1c0 2023-10-08T00:22:18.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.203+0000 7f9059969700 1 -- 172.21.15.99:0/3234480376 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90440094f0 con 0x7f905406f1c0 2023-10-08T00:22:18.210 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.203+0000 7f9050ff9700 1 -- 172.21.15.99:0/3234480376 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904c00ad20 con 0x7f905406f1c0 2023-10-08T00:22:18.211 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.203+0000 7f9059969700 1 -- 172.21.15.99:0/3234480376 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f905415b030 con 0x7f905406f1c0 2023-10-08T00:22:18.211 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.204+0000 7f9059969700 1 -- 172.21.15.99:0/3234480376 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90540c3e40 con 0x7f905406f1c0 2023-10-08T00:22:18.211 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.204+0000 7f9050ff9700 1 -- 172.21.15.99:0/3234480376 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f904c007300 con 0x7f905406f1c0 2023-10-08T00:22:18.211 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.205+0000 7f9050ff9700 1 --2- 172.21.15.99:0/3234480376 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9034066f30 0x7f90340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:18.211 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.205+0000 7f9050ff9700 1 -- 172.21.15.99:0/3234480376 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f904c084310 con 0x7f905406f1c0 2023-10-08T00:22:18.211 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.205+0000 7f9053fff700 1 --2- 172.21.15.99:0/3234480376 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9034066f30 0x7f90340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:18.211 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.206+0000 7f9053fff700 1 --2- 172.21.15.99:0/3234480376 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9034066f30 0x7f90340693e0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f90440099b0 tx=0x7f9044017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:18.212 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.209+0000 7f9050ff9700 1 -- 172.21.15.99:0/3234480376 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f904c0519c0 con 0x7f905406f1c0 2023-10-08T00:22:18.394 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.394+0000 7f9059969700 1 -- 172.21.15.99:0/3234480376 --> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f905406de20 con 0x7f9034066f30 2023-10-08T00:22:18.395 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.394+0000 7f9050ff9700 1 -- 172.21.15.99:0/3234480376 <== mgr.4099 v2:172.21.15.191:6832/85985 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f905406de20 con 0x7f9034066f30 2023-10-08T00:22:18.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.397+0000 7f9059969700 1 -- 172.21.15.99:0/3234480376 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9034066f30 msgr2=0x7f90340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:18.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.397+0000 7f9059969700 1 --2- 172.21.15.99:0/3234480376 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9034066f30 0x7f90340693e0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f90440099b0 tx=0x7f9044017000 comp rx=0 tx=0).stop 2023-10-08T00:22:18.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.397+0000 7f9059969700 1 -- 172.21.15.99:0/3234480376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f905406f1c0 msgr2=0x7f90540adf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:18.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.397+0000 7f9059969700 1 --2- 172.21.15.99:0/3234480376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f905406f1c0 0x7f90540adf60 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f904c00cd80 tx=0x7f904c001b90 comp rx=0 tx=0).stop 2023-10-08T00:22:18.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.397+0000 7f9059969700 1 -- 172.21.15.99:0/3234480376 shutdown_connections 2023-10-08T00:22:18.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.397+0000 7f9059969700 1 --2- 172.21.15.99:0/3234480376 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9034066f30 0x7f90340693e0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:18.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.397+0000 7f9059969700 1 --2- 172.21.15.99:0/3234480376 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9054157f30 0x7f9054158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:18.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.397+0000 7f9059969700 1 --2- 172.21.15.99:0/3234480376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f905406f1c0 0x7f90540adf60 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:18.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.397+0000 7f9059969700 1 --2- 172.21.15.99:0/3234480376 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9054157b60 0x7f905416b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:18.398 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.397+0000 7f9059969700 1 -- 172.21.15.99:0/3234480376 >> 172.21.15.99:0/3234480376 conn(0x7f9054069870 msgr2=0x7f90540b3720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:18.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.397+0000 7f9059969700 1 -- 172.21.15.99:0/3234480376 shutdown_connections 2023-10-08T00:22:18.399 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.397+0000 7f9059969700 1 -- 172.21.15.99:0/3234480376 wait complete. 2023-10-08T00:22:18.399 INFO:teuthology.orchestra.run.smithi099.stderr:instructed osd(s) 5 to deep-scrub 2023-10-08T00:22:18.409 INFO:tasks.ceph:Scrubbing osd.6 2023-10-08T00:22:18.410 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-10-08T00:22:18.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.527+0000 7ff97f1a0700 1 Processor -- start 2023-10-08T00:22:18.528 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.527+0000 7ff97f1a0700 1 -- start start 2023-10-08T00:22:18.530 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.528+0000 7ff97f1a0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff978159ef0 0x7ff97815a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:18.530 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.528+0000 7ff97f1a0700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff9780b46f0 0x7ff9780b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:18.530 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.528+0000 7ff97f1a0700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9780b5090 con 0x7ff9780b6a10 2023-10-08T00:22:18.530 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.528+0000 7ff97f1a0700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9780b17b0 con 0x7ff9780b46f0 2023-10-08T00:22:18.530 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.528+0000 7ff97f1a0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9780b1a60 con 0x7ff978159ef0 2023-10-08T00:22:18.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.528+0000 7ff97d73d700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff9780b46f0 0x7ff9780b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:18.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.528+0000 7ff97d73d700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff9780b46f0 0x7ff9780b4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33518/0 (socket says 172.21.15.99:33518) 2023-10-08T00:22:18.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.528+0000 7ff97d73d700 1 -- 172.21.15.99:0/3981128176 learned_addr learned my addr 172.21.15.99:0/3981128176 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:18.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.528+0000 7ff97cf3c700 1 --2- 172.21.15.99:0/3981128176 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff978159ef0 0x7ff97815a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:18.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.528+0000 7ff97d73d700 1 -- 172.21.15.99:0/3981128176 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff978159ef0 msgr2=0x7ff97815a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:18.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.529+0000 7ff97d73d700 1 --2- 172.21.15.99:0/3981128176 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff978159ef0 0x7ff97815a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:18.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.529+0000 7ff97d73d700 1 -- 172.21.15.99:0/3981128176 >> v1:172.21.15.191:6789/0 conn(0x7ff9780b6a10 legacy=0x7ff9780b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:18.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.529+0000 7ff97d73d700 1 -- 172.21.15.99:0/3981128176 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9780b1d40 con 0x7ff9780b46f0 2023-10-08T00:22:18.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.529+0000 7ff97d73d700 1 --2- 172.21.15.99:0/3981128176 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9780b46f0 0x7ff9780b4b50 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7ff9700099b0 tx=0x7ff970009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b51d2668c5179c52 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:18.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.529+0000 7ff96f7fe700 1 -- 172.21.15.99:0/3981128176 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97000b820 con 0x7ff9780b46f0 2023-10-08T00:22:18.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.529+0000 7ff96f7fe700 1 -- 172.21.15.99:0/3981128176 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff97000b980 con 0x7ff9780b46f0 2023-10-08T00:22:18.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.529+0000 7ff96f7fe700 1 -- 172.21.15.99:0/3981128176 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97000bd20 con 0x7ff9780b46f0 2023-10-08T00:22:18.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.529+0000 7ff97f1a0700 1 -- 172.21.15.99:0/3981128176 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9780b46f0 msgr2=0x7ff9780b4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:18.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.529+0000 7ff97f1a0700 1 --2- 172.21.15.99:0/3981128176 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9780b46f0 0x7ff9780b4b50 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7ff9700099b0 tx=0x7ff970009250 comp rx=0 tx=0).stop 2023-10-08T00:22:18.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.529+0000 7ff97f1a0700 1 -- 172.21.15.99:0/3981128176 shutdown_connections 2023-10-08T00:22:18.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.529+0000 7ff97f1a0700 1 --2- 172.21.15.99:0/3981128176 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9780b46f0 0x7ff9780b4b50 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:18.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.529+0000 7ff97f1a0700 1 --2- 172.21.15.99:0/3981128176 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff978159ef0 0x7ff97815a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:18.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.529+0000 7ff97f1a0700 1 -- 172.21.15.99:0/3981128176 >> 172.21.15.99:0/3981128176 conn(0x7ff97809a0a0 msgr2=0x7ff978067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:18.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.529+0000 7ff97f1a0700 1 -- 172.21.15.99:0/3981128176 shutdown_connections 2023-10-08T00:22:18.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.529+0000 7ff97f1a0700 1 -- 172.21.15.99:0/3981128176 wait complete. 2023-10-08T00:22:18.537 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.530+0000 7ff97f1a0700 1 Processor -- start 2023-10-08T00:22:18.537 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.531+0000 7ff97f1a0700 1 -- start start 2023-10-08T00:22:18.537 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.531+0000 7ff97f1a0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9780b46f0 0x7ff9780d13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:18.537 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.531+0000 7ff97f1a0700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9780b6a10 0x7ff9780ac870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:18.537 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.531+0000 7ff97f1a0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff978159ef0 0x7ff9780b0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:18.537 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.531+0000 7ff97f1a0700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9780adb50 con 0x7ff9780b46f0 2023-10-08T00:22:18.537 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.531+0000 7ff97f1a0700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9780ade30 con 0x7ff9780b6a10 2023-10-08T00:22:18.537 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.531+0000 7ff97f1a0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9780ae110 con 0x7ff978159ef0 2023-10-08T00:22:18.537 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.531+0000 7ff97cf3c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9780b46f0 0x7ff9780d13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:18.537 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.531+0000 7ff97cf3c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9780b46f0 0x7ff9780d13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33528/0 (socket says 172.21.15.99:33528) 2023-10-08T00:22:18.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.531+0000 7ff97cf3c700 1 -- 172.21.15.99:0/3944858087 learned_addr learned my addr 172.21.15.99:0/3944858087 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:18.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.532+0000 7ff97d73d700 1 --2- 172.21.15.99:0/3944858087 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff978159ef0 0x7ff9780b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:18.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.532+0000 7ff97cf3c700 1 -- 172.21.15.99:0/3944858087 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff978159ef0 msgr2=0x7ff9780b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:18.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.532+0000 7ff96ffff700 1 --2- 172.21.15.99:0/3944858087 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9780b6a10 0x7ff9780ac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:18.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.532+0000 7ff97cf3c700 1 --2- 172.21.15.99:0/3944858087 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff978159ef0 0x7ff9780b0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:18.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.532+0000 7ff97cf3c700 1 -- 172.21.15.99:0/3944858087 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9780b6a10 msgr2=0x7ff9780ac870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:18.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.532+0000 7ff97cf3c700 1 --2- 172.21.15.99:0/3944858087 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9780b6a10 0x7ff9780ac870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:18.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.532+0000 7ff97cf3c700 1 -- 172.21.15.99:0/3944858087 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9700094f0 con 0x7ff9780b46f0 2023-10-08T00:22:18.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.532+0000 7ff97d73d700 1 --2- 172.21.15.99:0/3944858087 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff978159ef0 0x7ff9780b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:22:18.539 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.532+0000 7ff96ffff700 1 --2- 172.21.15.99:0/3944858087 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9780b6a10 0x7ff9780ac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:22:18.539 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.532+0000 7ff97cf3c700 1 --2- 172.21.15.99:0/3944858087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9780b46f0 0x7ff9780d13f0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7ff97400e0e0 tx=0x7ff9740098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:18.539 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.532+0000 7ff96cff9700 1 -- 172.21.15.99:0/3944858087 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97400ed80 con 0x7ff9780b46f0 2023-10-08T00:22:18.539 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.532+0000 7ff96cff9700 1 -- 172.21.15.99:0/3944858087 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff974014420 con 0x7ff9780b46f0 2023-10-08T00:22:18.539 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.532+0000 7ff97f1a0700 1 -- 172.21.15.99:0/3944858087 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9780ae3f0 con 0x7ff9780b46f0 2023-10-08T00:22:18.539 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.532+0000 7ff96cff9700 1 -- 172.21.15.99:0/3944858087 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9740147e0 con 0x7ff9780b46f0 2023-10-08T00:22:18.539 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.532+0000 7ff97f1a0700 1 -- 172.21.15.99:0/3944858087 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9780d1930 con 0x7ff9780b46f0 2023-10-08T00:22:18.539 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.533+0000 7ff97f1a0700 1 -- 172.21.15.99:0/3944858087 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff97804c3c0 con 0x7ff9780b46f0 2023-10-08T00:22:18.539 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.533+0000 7ff96cff9700 1 -- 172.21.15.99:0/3944858087 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff974014b50 con 0x7ff9780b46f0 2023-10-08T00:22:18.539 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.534+0000 7ff96cff9700 1 --2- 172.21.15.99:0/3944858087 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff964066f30 0x7ff9640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:18.539 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.534+0000 7ff96cff9700 1 -- 172.21.15.99:0/3944858087 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 4939+0+0 (secure 0 0 0) 0x7ff974084a30 con 0x7ff9780b46f0 2023-10-08T00:22:18.539 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.534+0000 7ff96cff9700 1 --2- 172.21.15.99:0/3944858087 >> [v2:172.21.15.191:6808/86571,v1:172.21.15.191:6809/86571] conn(0x7ff96406cd80 0x7ff96406f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:18.540 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.534+0000 7ff96cff9700 1 -- 172.21.15.99:0/3944858087 --> [v2:172.21.15.191:6808/86571,v1:172.21.15.191:6809/86571] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff96406f860 con 0x7ff96406cd80 2023-10-08T00:22:18.540 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.534+0000 7ff96cff9700 1 -- 172.21.15.99:0/3944858087 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=192) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff974089240 con 0x7ff9780b46f0 2023-10-08T00:22:18.540 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.534+0000 7ff96ffff700 1 --2- 172.21.15.99:0/3944858087 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff964066f30 0x7ff9640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:18.540 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.534+0000 7ff97d73d700 1 --2- 172.21.15.99:0/3944858087 >> [v2:172.21.15.191:6808/86571,v1:172.21.15.191:6809/86571] conn(0x7ff96406cd80 0x7ff96406f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:18.540 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.535+0000 7ff96ffff700 1 --2- 172.21.15.99:0/3944858087 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff964066f30 0x7ff9640693e0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7ff97809e1b0 tx=0x7ff968009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:18.540 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.535+0000 7ff97d73d700 1 --2- 172.21.15.99:0/3944858087 >> [v2:172.21.15.191:6808/86571,v1:172.21.15.191:6809/86571] conn(0x7ff96406cd80 0x7ff96406f190 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:18.540 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.535+0000 7ff96cff9700 1 -- 172.21.15.99:0/3944858087 <== osd.6 v2:172.21.15.191:6808/86571 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22340 (crc 0 0 0) 0x7ff96406f860 con 0x7ff96406cd80 2023-10-08T00:22:18.548 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.548+0000 7ff97f1a0700 1 -- 172.21.15.99:0/3944858087 --> [v2:172.21.15.191:6808/86571,v1:172.21.15.191:6809/86571] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff9780acdb0 con 0x7ff96406cd80 2023-10-08T00:22:18.549 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.548+0000 7ff96cff9700 1 -- 172.21.15.99:0/3944858087 <== osd.6 v2:172.21.15.191:6808/86571 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+682 (crc 0 0 0) 0x7ff9780acdb0 con 0x7ff96406cd80 2023-10-08T00:22:18.549 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:22:18.549 INFO:teuthology.orchestra.run.smithi099.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2023-10-08T00:22:18.549 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:22:18.550 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.549+0000 7ff97f1a0700 1 -- 172.21.15.99:0/3944858087 >> [v2:172.21.15.191:6808/86571,v1:172.21.15.191:6809/86571] conn(0x7ff96406cd80 msgr2=0x7ff96406f190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:18.550 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.549+0000 7ff97f1a0700 1 --2- 172.21.15.99:0/3944858087 >> [v2:172.21.15.191:6808/86571,v1:172.21.15.191:6809/86571] conn(0x7ff96406cd80 0x7ff96406f190 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:18.550 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.549+0000 7ff97f1a0700 1 -- 172.21.15.99:0/3944858087 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff964066f30 msgr2=0x7ff9640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:18.550 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.549+0000 7ff97f1a0700 1 --2- 172.21.15.99:0/3944858087 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7ff964066f30 0x7ff9640693e0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7ff97809e1b0 tx=0x7ff968009040 comp rx=0 tx=0).stop 2023-10-08T00:22:18.551 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.549+0000 7ff97f1a0700 1 -- 172.21.15.99:0/3944858087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9780b46f0 msgr2=0x7ff9780d13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:18.551 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.549+0000 7ff97f1a0700 1 --2- 172.21.15.99:0/3944858087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9780b46f0 0x7ff9780d13f0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7ff97400e0e0 tx=0x7ff9740098d0 comp rx=0 tx=0).stop 2023-10-08T00:22:18.551 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.549+0000 7ff97d73d700 1 -- 172.21.15.99:0/3944858087 reap_dead start 2023-10-08T00:22:18.551 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.549+0000 7ff97f1a0700 1 -- 172.21.15.99:0/3944858087 shutdown_connections 2023-10-08T00:22:18.551 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.549+0000 7ff97f1a0700 1 -- 172.21.15.99:0/3944858087 >> 172.21.15.99:0/3944858087 conn(0x7ff97809a0a0 msgr2=0x7ff97815b150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:18.551 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.549+0000 7ff97f1a0700 1 -- 172.21.15.99:0/3944858087 shutdown_connections 2023-10-08T00:22:18.551 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.549+0000 7ff97f1a0700 1 -- 172.21.15.99:0/3944858087 wait complete. 2023-10-08T00:22:18.561 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-10-08T00:22:18.676 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.675+0000 7f0eeda91700 1 Processor -- start 2023-10-08T00:22:18.676 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.675+0000 7f0eeda91700 1 -- start start 2023-10-08T00:22:18.678 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.676+0000 7f0eeda91700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0ee8157c40 0x7f0ee8158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:18.678 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.676+0000 7f0eeda91700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ee815cbb0 con 0x7f0ee815a620 2023-10-08T00:22:18.678 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.676+0000 7f0eeda91700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ee815bfa0 con 0x7f0ee8157870 2023-10-08T00:22:18.678 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.676+0000 7f0eeda91700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ee815bdb0 con 0x7f0ee8157c40 2023-10-08T00:22:18.678 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.676+0000 7f0ee7fff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0ee8157c40 0x7f0ee8158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:18.678 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.676+0000 7f0ee6ffd700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f0ee815a620 0x7f0ee815d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:37406/0 (socket says 172.21.15.99:37406) 2023-10-08T00:22:18.679 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.676+0000 7f0ee6ffd700 1 -- 172.21.15.99:0/1346894905 learned_addr learned my addr 172.21.15.99:0/1346894905 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:22:18.679 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.676+0000 7f0ee7fff700 1 -- 172.21.15.99:0/1346894905 >> v1:172.21.15.191:6789/0 conn(0x7f0ee8157870 legacy=0x7f0ee815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:18.679 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.677+0000 7f0ee7fff700 1 -- 172.21.15.99:0/1346894905 >> v1:172.21.15.99:6789/0 conn(0x7f0ee815a620 legacy=0x7f0ee815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:18.679 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.677+0000 7f0ee7fff700 1 -- 172.21.15.99:0/1346894905 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee815c280 con 0x7f0ee8157c40 2023-10-08T00:22:18.679 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.677+0000 7f0ee67fc700 1 -- 172.21.15.99:0/1346894905 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 364126973 0 0) 0x7f0ee815cbb0 con 0x7f0ee815a620 2023-10-08T00:22:18.679 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.677+0000 7f0ee7fff700 1 --2- 172.21.15.99:0/1346894905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ee8157c40 0x7f0ee8158110 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f0ee00098e0 tx=0x7f0ee00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0fb2c50504c9f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:18.679 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.677+0000 7f0ee67fc700 1 -- 172.21.15.99:0/1346894905 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee0009e80 con 0x7f0ee8157c40 2023-10-08T00:22:18.679 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.677+0000 7f0ee67fc700 1 -- 172.21.15.99:0/1346894905 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ee000b860 con 0x7f0ee8157c40 2023-10-08T00:22:18.680 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.677+0000 7f0ee67fc700 1 -- 172.21.15.99:0/1346894905 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee000bbe0 con 0x7f0ee8157c40 2023-10-08T00:22:18.680 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.677+0000 7f0eeda91700 1 -- 172.21.15.99:0/1346894905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ee8157c40 msgr2=0x7f0ee8158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:18.680 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.677+0000 7f0eeda91700 1 --2- 172.21.15.99:0/1346894905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ee8157c40 0x7f0ee8158110 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f0ee00098e0 tx=0x7f0ee00091a0 comp rx=0 tx=0).stop 2023-10-08T00:22:18.680 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.677+0000 7f0eeda91700 1 -- 172.21.15.99:0/1346894905 shutdown_connections 2023-10-08T00:22:18.680 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.677+0000 7f0eeda91700 1 --2- 172.21.15.99:0/1346894905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ee8157c40 0x7f0ee8158110 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:18.680 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.677+0000 7f0eeda91700 1 -- 172.21.15.99:0/1346894905 >> 172.21.15.99:0/1346894905 conn(0x7f0ee8068ce0 msgr2=0x7f0ee80690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:18.680 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.677+0000 7f0eeda91700 1 -- 172.21.15.99:0/1346894905 shutdown_connections 2023-10-08T00:22:18.680 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.677+0000 7f0eeda91700 1 -- 172.21.15.99:0/1346894905 wait complete. 2023-10-08T00:22:18.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.679+0000 7f0eeda91700 1 Processor -- start 2023-10-08T00:22:18.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.679+0000 7f0eeda91700 1 -- start start 2023-10-08T00:22:18.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.679+0000 7f0eeda91700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ee8157870 0x7f0ee816a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:18.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.679+0000 7f0eeda91700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ee8157c40 0x7f0ee813f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:18.683 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.679+0000 7f0eeda91700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ee815a620 0x7f0ee80b2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:18.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.679+0000 7f0eeda91700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee814e6e0 con 0x7f0ee8157870 2023-10-08T00:22:18.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.679+0000 7f0eeda91700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee814ef80 con 0x7f0ee815a620 2023-10-08T00:22:18.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.679+0000 7f0eeda91700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee80b2eb0 con 0x7f0ee8157c40 2023-10-08T00:22:18.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.679+0000 7f0ee77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ee8157870 0x7f0ee816a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:18.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.679+0000 7f0ee77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ee8157870 0x7f0ee816a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33548/0 (socket says 172.21.15.99:33548) 2023-10-08T00:22:18.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.679+0000 7f0ee77fe700 1 -- 172.21.15.99:0/2056301328 learned_addr learned my addr 172.21.15.99:0/2056301328 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:18.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.679+0000 7f0ee7fff700 1 --2- 172.21.15.99:0/2056301328 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ee815a620 0x7f0ee80b2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:18.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.680+0000 7f0ee6ffd700 1 --2- 172.21.15.99:0/2056301328 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ee8157c40 0x7f0ee813f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:18.684 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.680+0000 7f0ee77fe700 1 -- 172.21.15.99:0/2056301328 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ee8157c40 msgr2=0x7f0ee813f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:18.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.680+0000 7f0ee77fe700 1 --2- 172.21.15.99:0/2056301328 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ee8157c40 0x7f0ee813f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:18.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.680+0000 7f0ee77fe700 1 -- 172.21.15.99:0/2056301328 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ee815a620 msgr2=0x7f0ee80b2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:18.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.680+0000 7f0ee77fe700 1 --2- 172.21.15.99:0/2056301328 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ee815a620 0x7f0ee80b2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:18.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.680+0000 7f0ee77fe700 1 -- 172.21.15.99:0/2056301328 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee0009420 con 0x7f0ee8157870 2023-10-08T00:22:18.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.680+0000 7f0ee77fe700 1 --2- 172.21.15.99:0/2056301328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ee8157870 0x7f0ee816a3d0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f0ed800cb40 tx=0x7f0ed80017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:18.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.680+0000 7f0ecf7fe700 1 -- 172.21.15.99:0/2056301328 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed800a7a0 con 0x7f0ee8157870 2023-10-08T00:22:18.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.680+0000 7f0ecf7fe700 1 -- 172.21.15.99:0/2056301328 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ed800a920 con 0x7f0ee8157870 2023-10-08T00:22:18.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.680+0000 7f0eeda91700 1 -- 172.21.15.99:0/2056301328 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ee814e930 con 0x7f0ee8157870 2023-10-08T00:22:18.685 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.680+0000 7f0ecf7fe700 1 -- 172.21.15.99:0/2056301328 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed800ace0 con 0x7f0ee8157870 2023-10-08T00:22:18.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.680+0000 7f0eeda91700 1 -- 172.21.15.99:0/2056301328 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ee813fd50 con 0x7f0ee8157870 2023-10-08T00:22:18.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.681+0000 7f0eeda91700 1 -- 172.21.15.99:0/2056301328 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ee804c3c0 con 0x7f0ee8157870 2023-10-08T00:22:18.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.681+0000 7f0ecf7fe700 1 -- 172.21.15.99:0/2056301328 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0ed8003030 con 0x7f0ee8157870 2023-10-08T00:22:18.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.682+0000 7f0ecf7fe700 1 --2- 172.21.15.99:0/2056301328 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0ed4066f30 0x7f0ed40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:18.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.682+0000 7f0ecf7fe700 1 -- 172.21.15.99:0/2056301328 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f0ed8084260 con 0x7f0ee8157870 2023-10-08T00:22:18.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.682+0000 7f0ee6ffd700 1 --2- 172.21.15.99:0/2056301328 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0ed4066f30 0x7f0ed40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:18.686 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.682+0000 7f0ee6ffd700 1 --2- 172.21.15.99:0/2056301328 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0ed4066f30 0x7f0ed40693e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8157090 tx=0x7f0edc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:18.687 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.686+0000 7f0ecf7fe700 1 -- 172.21.15.99:0/2056301328 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0ed80517d0 con 0x7f0ee8157870 2023-10-08T00:22:18.865 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.865+0000 7f0eeda91700 1 -- 172.21.15.99:0/2056301328 --> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ee814f0c0 con 0x7f0ed4066f30 2023-10-08T00:22:18.866 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.866+0000 7f0ecf7fe700 1 -- 172.21.15.99:0/2056301328 <== mgr.4099 v2:172.21.15.191:6832/85985 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f0edc00cd90 con 0x7f0ed4066f30 2023-10-08T00:22:18.869 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.867+0000 7f0eeda91700 1 -- 172.21.15.99:0/2056301328 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0ed4066f30 msgr2=0x7f0ed40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:18.869 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.867+0000 7f0eeda91700 1 --2- 172.21.15.99:0/2056301328 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0ed4066f30 0x7f0ed40693e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8157090 tx=0x7f0edc009040 comp rx=0 tx=0).stop 2023-10-08T00:22:18.870 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.867+0000 7f0eeda91700 1 -- 172.21.15.99:0/2056301328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ee8157870 msgr2=0x7f0ee816a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:18.870 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.867+0000 7f0eeda91700 1 --2- 172.21.15.99:0/2056301328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ee8157870 0x7f0ee816a3d0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f0ed800cb40 tx=0x7f0ed80017b0 comp rx=0 tx=0).stop 2023-10-08T00:22:18.870 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.867+0000 7f0eeda91700 1 -- 172.21.15.99:0/2056301328 shutdown_connections 2023-10-08T00:22:18.870 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.867+0000 7f0eeda91700 1 --2- 172.21.15.99:0/2056301328 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0ed4066f30 0x7f0ed40693e0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:18.870 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.867+0000 7f0eeda91700 1 --2- 172.21.15.99:0/2056301328 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ee815a620 0x7f0ee80b2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:18.870 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.867+0000 7f0eeda91700 1 --2- 172.21.15.99:0/2056301328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ee8157870 0x7f0ee816a3d0 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:18.871 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.867+0000 7f0eeda91700 1 --2- 172.21.15.99:0/2056301328 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ee8157c40 0x7f0ee813f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:18.871 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.867+0000 7f0eeda91700 1 -- 172.21.15.99:0/2056301328 >> 172.21.15.99:0/2056301328 conn(0x7f0ee8068ce0 msgr2=0x7f0ee80d0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:18.871 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.868+0000 7f0eeda91700 1 -- 172.21.15.99:0/2056301328 shutdown_connections 2023-10-08T00:22:18.872 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.868+0000 7f0eeda91700 1 -- 172.21.15.99:0/2056301328 wait complete. 2023-10-08T00:22:18.872 INFO:teuthology.orchestra.run.smithi099.stderr:instructed osd(s) 6 to deep-scrub 2023-10-08T00:22:18.880 INFO:tasks.ceph:Scrubbing osd.7 2023-10-08T00:22:18.880 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-10-08T00:22:18.998 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.997+0000 7f823d437700 1 Processor -- start 2023-10-08T00:22:18.998 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.997+0000 7f823d437700 1 -- start start 2023-10-08T00:22:19.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.998+0000 7f823d437700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8238068ec0 0x7f82380b1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:19.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.998+0000 7f823d437700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82380aff80 con 0x7f82380b1950 2023-10-08T00:22:19.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.998+0000 7f823d437700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82380b0350 con 0x7f8238068af0 2023-10-08T00:22:19.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.998+0000 7f823d437700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82380b15c0 con 0x7f8238068ec0 2023-10-08T00:22:19.000 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.998+0000 7f82367fc700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8238068ec0 0x7f82380b1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:19.001 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.998+0000 7f82377fe700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f82380b1950 0x7f82380af0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:37410/0 (socket says 172.21.15.99:37410) 2023-10-08T00:22:19.001 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.998+0000 7f82377fe700 1 -- 172.21.15.99:0/774813541 learned_addr learned my addr 172.21.15.99:0/774813541 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:22:19.001 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.999+0000 7f82367fc700 1 -- 172.21.15.99:0/774813541 >> v1:172.21.15.191:6789/0 conn(0x7f8238068af0 legacy=0x7f823815aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:19.001 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.999+0000 7f82367fc700 1 -- 172.21.15.99:0/774813541 >> v1:172.21.15.99:6789/0 conn(0x7f82380b1950 legacy=0x7f82380af0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:19.001 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.999+0000 7f82367fc700 1 -- 172.21.15.99:0/774813541 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82380b0bf0 con 0x7f8238068ec0 2023-10-08T00:22:19.001 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.999+0000 7f82367fc700 1 --2- 172.21.15.99:0/774813541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8238068ec0 0x7f82380b1080 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f822c0098e0 tx=0x7f822c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74db459e7c03fb6d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:19.001 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.999+0000 7f8235ffb700 1 -- 172.21.15.99:0/774813541 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c009e80 con 0x7f8238068ec0 2023-10-08T00:22:19.001 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.999+0000 7f8235ffb700 1 -- 172.21.15.99:0/774813541 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f822c00b820 con 0x7f8238068ec0 2023-10-08T00:22:19.001 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.999+0000 7f8235ffb700 1 -- 172.21.15.99:0/774813541 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c00bba0 con 0x7f8238068ec0 2023-10-08T00:22:19.001 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.999+0000 7f823d437700 1 -- 172.21.15.99:0/774813541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8238068ec0 msgr2=0x7f82380b1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:19.001 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.999+0000 7f823d437700 1 --2- 172.21.15.99:0/774813541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8238068ec0 0x7f82380b1080 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f822c0098e0 tx=0x7f822c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:22:19.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.999+0000 7f823d437700 1 -- 172.21.15.99:0/774813541 shutdown_connections 2023-10-08T00:22:19.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.999+0000 7f823d437700 1 --2- 172.21.15.99:0/774813541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8238068ec0 0x7f82380b1080 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:19.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.999+0000 7f823d437700 1 -- 172.21.15.99:0/774813541 >> 172.21.15.99:0/774813541 conn(0x7f8238069870 msgr2=0x7f8238069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:19.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.999+0000 7f823d437700 1 -- 172.21.15.99:0/774813541 shutdown_connections 2023-10-08T00:22:19.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:18.999+0000 7f823d437700 1 -- 172.21.15.99:0/774813541 wait complete. 2023-10-08T00:22:19.005 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.001+0000 7f823d437700 1 Processor -- start 2023-10-08T00:22:19.005 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.001+0000 7f823d437700 1 -- start start 2023-10-08T00:22:19.005 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.001+0000 7f823d437700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8238068af0 0x7f82380aceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:19.005 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.001+0000 7f823d437700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8238068ec0 0x7f82380b3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:19.005 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.001+0000 7f823d437700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82380b1950 0x7f82380b39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:19.005 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.001+0000 7f823d437700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823816a940 con 0x7f82380b1950 2023-10-08T00:22:19.005 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.001+0000 7f823d437700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82380c9040 con 0x7f8238068af0 2023-10-08T00:22:19.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.001+0000 7f823d437700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82380ca810 con 0x7f8238068ec0 2023-10-08T00:22:19.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.002+0000 7f82377fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82380b1950 0x7f82380b39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:19.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.002+0000 7f82377fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82380b1950 0x7f82380b39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33564/0 (socket says 172.21.15.99:33564) 2023-10-08T00:22:19.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.002+0000 7f82377fe700 1 -- 172.21.15.99:0/3671993201 learned_addr learned my addr 172.21.15.99:0/3671993201 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:19.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.002+0000 7f82377fe700 1 -- 172.21.15.99:0/3671993201 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8238068ec0 msgr2=0x7f82380b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:19.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.002+0000 7f82367fc700 1 --2- 172.21.15.99:0/3671993201 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8238068ec0 0x7f82380b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:19.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.002+0000 7f8236ffd700 1 --2- 172.21.15.99:0/3671993201 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8238068af0 0x7f82380aceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:19.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.002+0000 7f82377fe700 1 --2- 172.21.15.99:0/3671993201 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8238068ec0 0x7f82380b3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:19.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.002+0000 7f82377fe700 1 -- 172.21.15.99:0/3671993201 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8238068af0 msgr2=0x7f82380aceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:19.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.002+0000 7f82377fe700 1 --2- 172.21.15.99:0/3671993201 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8238068af0 0x7f82380aceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:19.006 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.002+0000 7f82377fe700 1 -- 172.21.15.99:0/3671993201 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f822c009420 con 0x7f82380b1950 2023-10-08T00:22:19.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.002+0000 7f82377fe700 1 --2- 172.21.15.99:0/3671993201 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82380b1950 0x7f82380b39d0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f823000c560 tx=0x7f8230007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:19.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.002+0000 7f821effd700 1 -- 172.21.15.99:0/3671993201 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8230005b30 con 0x7f82380b1950 2023-10-08T00:22:19.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.002+0000 7f821effd700 1 -- 172.21.15.99:0/3671993201 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8230005c90 con 0x7f82380b1950 2023-10-08T00:22:19.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.002+0000 7f823d437700 1 -- 172.21.15.99:0/3671993201 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82380caaf0 con 0x7f82380b1950 2023-10-08T00:22:19.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.002+0000 7f821effd700 1 -- 172.21.15.99:0/3671993201 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82300072e0 con 0x7f82380b1950 2023-10-08T00:22:19.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.002+0000 7f823d437700 1 -- 172.21.15.99:0/3671993201 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82380cb0b0 con 0x7f82380b1950 2023-10-08T00:22:19.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.003+0000 7f823d437700 1 -- 172.21.15.99:0/3671993201 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f82380cdfb0 con 0x7f82380b1950 2023-10-08T00:22:19.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.004+0000 7f821effd700 1 -- 172.21.15.99:0/3671993201 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8230007690 con 0x7f82380b1950 2023-10-08T00:22:19.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.004+0000 7f821effd700 1 --2- 172.21.15.99:0/3671993201 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8224066f30 0x7f82240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:19.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.004+0000 7f821effd700 1 -- 172.21.15.99:0/3671993201 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f8230083590 con 0x7f82380b1950 2023-10-08T00:22:19.007 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.004+0000 7f821effd700 1 --2- 172.21.15.99:0/3671993201 >> [v2:172.21.15.191:6817/86612,v1:172.21.15.191:6819/86612] conn(0x7f822406cd80 0x7f822406f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:19.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.004+0000 7f821effd700 1 -- 172.21.15.99:0/3671993201 --> [v2:172.21.15.191:6817/86612,v1:172.21.15.191:6819/86612] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f822406f860 con 0x7f822406cd80 2023-10-08T00:22:19.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.004+0000 7f821effd700 1 -- 172.21.15.99:0/3671993201 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=192) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8230085bd0 con 0x7f82380b1950 2023-10-08T00:22:19.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.004+0000 7f8236ffd700 1 --2- 172.21.15.99:0/3671993201 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8224066f30 0x7f82240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:19.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.004+0000 7f82367fc700 1 --2- 172.21.15.99:0/3671993201 >> [v2:172.21.15.191:6817/86612,v1:172.21.15.191:6819/86612] conn(0x7f822406cd80 0x7f822406f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:19.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.005+0000 7f8236ffd700 1 --2- 172.21.15.99:0/3671993201 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8224066f30 0x7f82240693e0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f8228007fd0 tx=0x7f8228009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:19.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.005+0000 7f82367fc700 1 --2- 172.21.15.99:0/3671993201 >> [v2:172.21.15.191:6817/86612,v1:172.21.15.191:6819/86612] conn(0x7f822406cd80 0x7f822406f190 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:19.008 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.005+0000 7f821effd700 1 -- 172.21.15.99:0/3671993201 <== osd.7 v2:172.21.15.191:6817/86612 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22340 (crc 0 0 0) 0x7f822406f860 con 0x7f822406cd80 2023-10-08T00:22:19.018 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.018+0000 7f823d437700 1 -- 172.21.15.99:0/3671993201 --> [v2:172.21.15.191:6817/86612,v1:172.21.15.191:6819/86612] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f82380ce440 con 0x7f822406cd80 2023-10-08T00:22:19.019 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.018+0000 7f821effd700 1 -- 172.21.15.99:0/3671993201 <== osd.7 v2:172.21.15.191:6817/86612 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+682 (crc 0 0 0) 0x7f82380ce440 con 0x7f822406cd80 2023-10-08T00:22:19.019 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T00:22:19.020 INFO:teuthology.orchestra.run.smithi099.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2023-10-08T00:22:19.020 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T00:22:19.021 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.019+0000 7f823d437700 1 -- 172.21.15.99:0/3671993201 >> [v2:172.21.15.191:6817/86612,v1:172.21.15.191:6819/86612] conn(0x7f822406cd80 msgr2=0x7f822406f190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:19.021 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.019+0000 7f823d437700 1 --2- 172.21.15.99:0/3671993201 >> [v2:172.21.15.191:6817/86612,v1:172.21.15.191:6819/86612] conn(0x7f822406cd80 0x7f822406f190 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:19.021 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.019+0000 7f823d437700 1 -- 172.21.15.99:0/3671993201 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8224066f30 msgr2=0x7f82240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:19.021 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.019+0000 7f823d437700 1 --2- 172.21.15.99:0/3671993201 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f8224066f30 0x7f82240693e0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f8228007fd0 tx=0x7f8228009040 comp rx=0 tx=0).stop 2023-10-08T00:22:19.021 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.019+0000 7f823d437700 1 -- 172.21.15.99:0/3671993201 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82380b1950 msgr2=0x7f82380b39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:19.021 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.019+0000 7f823d437700 1 --2- 172.21.15.99:0/3671993201 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82380b1950 0x7f82380b39d0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f823000c560 tx=0x7f8230007f60 comp rx=0 tx=0).stop 2023-10-08T00:22:19.021 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.019+0000 7f82377fe700 1 -- 172.21.15.99:0/3671993201 reap_dead start 2023-10-08T00:22:19.021 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.019+0000 7f823d437700 1 -- 172.21.15.99:0/3671993201 shutdown_connections 2023-10-08T00:22:19.021 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.019+0000 7f823d437700 1 -- 172.21.15.99:0/3671993201 >> 172.21.15.99:0/3671993201 conn(0x7f8238069870 msgr2=0x7f823806c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:19.021 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.019+0000 7f823d437700 1 -- 172.21.15.99:0/3671993201 shutdown_connections 2023-10-08T00:22:19.022 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.019+0000 7f823d437700 1 -- 172.21.15.99:0/3671993201 wait complete. 2023-10-08T00:22:19.031 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-10-08T00:22:19.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.145+0000 7f49b6b3f700 1 Processor -- start 2023-10-08T00:22:19.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.145+0000 7f49b6b3f700 1 -- start start 2023-10-08T00:22:19.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.146+0000 7f49b6b3f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49b0159da0 0x7f49b00b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:19.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.146+0000 7f49b6b3f700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f49b015a170 0x7f49b0068b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:19.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.146+0000 7f49b6b3f700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49b015ea70 con 0x7f49b0069040 2023-10-08T00:22:19.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.146+0000 7f49b6b3f700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49b00694e0 con 0x7f49b015a170 2023-10-08T00:22:19.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.146+0000 7f49b6b3f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49b00b3990 con 0x7f49b0159da0 2023-10-08T00:22:19.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.146+0000 7f49b533c700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f49b015a170 0x7f49b0068b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:19.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.146+0000 7f49b633e700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f49b0069040 0x7f49b00af4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:37418/0 (socket says 172.21.15.99:37418) 2023-10-08T00:22:19.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.146+0000 7f49b633e700 1 -- 172.21.15.99:0/527189394 learned_addr learned my addr 172.21.15.99:0/527189394 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:22:19.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.146+0000 7f49b5b3d700 1 --2- 172.21.15.99:0/527189394 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49b0159da0 0x7f49b00b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:19.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.146+0000 7f49b533c700 1 -- 172.21.15.99:0/527189394 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49b0159da0 msgr2=0x7f49b00b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:19.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.146+0000 7f49b533c700 1 --2- 172.21.15.99:0/527189394 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49b0159da0 0x7f49b00b2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:19.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.146+0000 7f49b533c700 1 -- 172.21.15.99:0/527189394 >> v1:172.21.15.99:6789/0 conn(0x7f49b0069040 legacy=0x7f49b00af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:19.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.146+0000 7f49b533c700 1 -- 172.21.15.99:0/527189394 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b00b0b70 con 0x7f49b015a170 2023-10-08T00:22:19.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.146+0000 7f49b4b3b700 1 -- 172.21.15.99:0/527189394 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3277044342 0 0) 0x7f49b015ea70 con 0x7f49b0069040 2023-10-08T00:22:19.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.146+0000 7f49b533c700 1 --2- 172.21.15.99:0/527189394 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49b015a170 0x7f49b0068b00 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f49a00098e0 tx=0x7f49a00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6168fc31e167e6f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:19.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.146+0000 7f49b4b3b700 1 -- 172.21.15.99:0/527189394 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a000b640 con 0x7f49b015a170 2023-10-08T00:22:19.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.146+0000 7f49b4b3b700 1 -- 172.21.15.99:0/527189394 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49a000b7a0 con 0x7f49b015a170 2023-10-08T00:22:19.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.146+0000 7f49b4b3b700 1 -- 172.21.15.99:0/527189394 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a000bb40 con 0x7f49b015a170 2023-10-08T00:22:19.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.147+0000 7f49b6b3f700 1 -- 172.21.15.99:0/527189394 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49b015a170 msgr2=0x7f49b0068b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:19.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.147+0000 7f49b6b3f700 1 --2- 172.21.15.99:0/527189394 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49b015a170 0x7f49b0068b00 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f49a00098e0 tx=0x7f49a00091a0 comp rx=0 tx=0).stop 2023-10-08T00:22:19.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.147+0000 7f49b6b3f700 1 -- 172.21.15.99:0/527189394 shutdown_connections 2023-10-08T00:22:19.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.147+0000 7f49b6b3f700 1 --2- 172.21.15.99:0/527189394 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49b0159da0 0x7f49b00b2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:19.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.147+0000 7f49b6b3f700 1 --2- 172.21.15.99:0/527189394 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49b015a170 0x7f49b0068b00 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:19.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.147+0000 7f49b6b3f700 1 -- 172.21.15.99:0/527189394 >> 172.21.15.99:0/527189394 conn(0x7f49b00d0bc0 msgr2=0x7f49b00d0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:19.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.147+0000 7f49b6b3f700 1 -- 172.21.15.99:0/527189394 shutdown_connections 2023-10-08T00:22:19.150 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.147+0000 7f49b6b3f700 1 -- 172.21.15.99:0/527189394 wait complete. 2023-10-08T00:22:19.152 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.148+0000 7f49b6b3f700 1 Processor -- start 2023-10-08T00:22:19.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.148+0000 7f49b6b3f700 1 -- start start 2023-10-08T00:22:19.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.149+0000 7f49b6b3f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49b0069040 0x7f49b0157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:19.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.149+0000 7f49b6b3f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49b0159da0 0x7f49b015ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:19.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.149+0000 7f49b6b3f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49b015a170 0x7f49b015f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:19.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.149+0000 7f49b6b3f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b00c6f70 con 0x7f49b0159da0 2023-10-08T00:22:19.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.149+0000 7f49b6b3f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b00c6360 con 0x7f49b015a170 2023-10-08T00:22:19.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.149+0000 7f49b6b3f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49b00c6640 con 0x7f49b0069040 2023-10-08T00:22:19.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.149+0000 7f49b533c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49b0159da0 0x7f49b015ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:19.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.149+0000 7f49b533c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49b0159da0 0x7f49b015ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33586/0 (socket says 172.21.15.99:33586) 2023-10-08T00:22:19.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.149+0000 7f49b533c700 1 -- 172.21.15.99:0/189633832 learned_addr learned my addr 172.21.15.99:0/189633832 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:19.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.149+0000 7f49b5b3d700 1 --2- 172.21.15.99:0/189633832 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49b0069040 0x7f49b0157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:19.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.149+0000 7f49b633e700 1 --2- 172.21.15.99:0/189633832 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49b015a170 0x7f49b015f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:19.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.149+0000 7f49b533c700 1 -- 172.21.15.99:0/189633832 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49b0069040 msgr2=0x7f49b0157150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:19.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.149+0000 7f49b533c700 1 --2- 172.21.15.99:0/189633832 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49b0069040 0x7f49b0157150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:19.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.149+0000 7f49b533c700 1 -- 172.21.15.99:0/189633832 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49b015a170 msgr2=0x7f49b015f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:19.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.149+0000 7f49b533c700 1 --2- 172.21.15.99:0/189633832 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49b015a170 0x7f49b015f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:19.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.149+0000 7f49b533c700 1 -- 172.21.15.99:0/189633832 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a0009420 con 0x7f49b0159da0 2023-10-08T00:22:19.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.149+0000 7f49b5b3d700 1 --2- 172.21.15.99:0/189633832 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49b0069040 0x7f49b0157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:22:19.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.149+0000 7f49b633e700 1 --2- 172.21.15.99:0/189633832 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49b015a170 0x7f49b015f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:22:19.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.149+0000 7f49b533c700 1 --2- 172.21.15.99:0/189633832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49b0159da0 0x7f49b015ee60 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f49a0009f80 tx=0x7f49a0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:19.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.150+0000 7f49a5ffb700 1 -- 172.21.15.99:0/189633832 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a0000d10 con 0x7f49b0159da0 2023-10-08T00:22:19.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.150+0000 7f49a5ffb700 1 -- 172.21.15.99:0/189633832 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49a000bb20 con 0x7f49b0159da0 2023-10-08T00:22:19.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.150+0000 7f49a5ffb700 1 -- 172.21.15.99:0/189633832 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a0011690 con 0x7f49b0159da0 2023-10-08T00:22:19.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.150+0000 7f49b6b3f700 1 -- 172.21.15.99:0/189633832 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49b00c6920 con 0x7f49b0159da0 2023-10-08T00:22:19.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.150+0000 7f49b6b3f700 1 -- 172.21.15.99:0/189633832 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49b0157690 con 0x7f49b0159da0 2023-10-08T00:22:19.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.151+0000 7f49b6b3f700 1 -- 172.21.15.99:0/189633832 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49b0089480 con 0x7f49b0159da0 2023-10-08T00:22:19.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.151+0000 7f49a5ffb700 1 -- 172.21.15.99:0/189633832 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f49a0011a80 con 0x7f49b0159da0 2023-10-08T00:22:19.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.151+0000 7f49a5ffb700 1 --2- 172.21.15.99:0/189633832 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4994066f30 0x7f49940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:19.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.151+0000 7f49a5ffb700 1 -- 172.21.15.99:0/189633832 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f49a0085c30 con 0x7f49b0159da0 2023-10-08T00:22:19.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.152+0000 7f49b5b3d700 1 --2- 172.21.15.99:0/189633832 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4994066f30 0x7f49940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:19.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.152+0000 7f49b5b3d700 1 --2- 172.21.15.99:0/189633832 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4994066f30 0x7f49940693e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f49ac009d80 tx=0x7f49ac008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:19.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.155+0000 7f49a5ffb700 1 -- 172.21.15.99:0/189633832 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f49a00531a0 con 0x7f49b0159da0 2023-10-08T00:22:19.337 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.336+0000 7f49b6b3f700 1 -- 172.21.15.99:0/189633832 --> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f49b00c70b0 con 0x7f4994066f30 2023-10-08T00:22:19.338 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.337+0000 7f49a5ffb700 1 -- 172.21.15.99:0/189633832 <== mgr.4099 v2:172.21.15.191:6832/85985 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f49b00c70b0 con 0x7f4994066f30 2023-10-08T00:22:19.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.339+0000 7f49b6b3f700 1 -- 172.21.15.99:0/189633832 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4994066f30 msgr2=0x7f49940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:19.340 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.339+0000 7f49b6b3f700 1 --2- 172.21.15.99:0/189633832 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4994066f30 0x7f49940693e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f49ac009d80 tx=0x7f49ac008040 comp rx=0 tx=0).stop 2023-10-08T00:22:19.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.339+0000 7f49b6b3f700 1 -- 172.21.15.99:0/189633832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49b0159da0 msgr2=0x7f49b015ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:19.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.339+0000 7f49b6b3f700 1 --2- 172.21.15.99:0/189633832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49b0159da0 0x7f49b015ee60 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f49a0009f80 tx=0x7f49a0017000 comp rx=0 tx=0).stop 2023-10-08T00:22:19.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.339+0000 7f49b6b3f700 1 -- 172.21.15.99:0/189633832 shutdown_connections 2023-10-08T00:22:19.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.339+0000 7f49b6b3f700 1 --2- 172.21.15.99:0/189633832 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f4994066f30 0x7f49940693e0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:19.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.339+0000 7f49b6b3f700 1 --2- 172.21.15.99:0/189633832 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49b015a170 0x7f49b015f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:19.341 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.339+0000 7f49b6b3f700 1 --2- 172.21.15.99:0/189633832 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49b0069040 0x7f49b0157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:19.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.339+0000 7f49b6b3f700 1 --2- 172.21.15.99:0/189633832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49b0159da0 0x7f49b015ee60 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:19.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.339+0000 7f49b6b3f700 1 -- 172.21.15.99:0/189633832 >> 172.21.15.99:0/189633832 conn(0x7f49b00d0bc0 msgr2=0x7f49b0158fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:19.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.339+0000 7f49b6b3f700 1 -- 172.21.15.99:0/189633832 shutdown_connections 2023-10-08T00:22:19.342 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.339+0000 7f49b6b3f700 1 -- 172.21.15.99:0/189633832 wait complete. 2023-10-08T00:22:19.342 INFO:teuthology.orchestra.run.smithi099.stderr:instructed osd(s) 7 to deep-scrub 2023-10-08T00:22:19.351 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T00:22:19.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.464+0000 7f936292a700 1 Processor -- start 2023-10-08T00:22:19.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.465+0000 7f936292a700 1 -- start start 2023-10-08T00:22:19.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.465+0000 7f936292a700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f935c1577d0 0x7f935c0cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:19.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.465+0000 7f936292a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f935c06ac30 0x7f935c15c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:19.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.465+0000 7f936292a700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f935c15c360 con 0x7f935c157ba0 2023-10-08T00:22:19.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.465+0000 7f936292a700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f935c15b790 con 0x7f935c1577d0 2023-10-08T00:22:19.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.465+0000 7f936292a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f935c15ba30 con 0x7f935c06ac30 2023-10-08T00:22:19.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.465+0000 7f935bfff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f935c1577d0 0x7f935c0cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:19.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.466+0000 7f935bfff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f935c1577d0 0x7f935c0cd770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33588/0 (socket says 172.21.15.99:33588) 2023-10-08T00:22:19.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.466+0000 7f935bfff700 1 -- 172.21.15.99:0/3203596790 learned_addr learned my addr 172.21.15.99:0/3203596790 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:19.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.466+0000 7f9360ec7700 1 --2- 172.21.15.99:0/3203596790 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f935c06ac30 0x7f935c15c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:19.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.466+0000 7f935bfff700 1 -- 172.21.15.99:0/3203596790 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f935c06ac30 msgr2=0x7f935c15c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:19.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.466+0000 7f935bfff700 1 --2- 172.21.15.99:0/3203596790 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f935c06ac30 0x7f935c15c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:19.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.466+0000 7f935bfff700 1 -- 172.21.15.99:0/3203596790 >> v1:172.21.15.191:6789/0 conn(0x7f935c157ba0 legacy=0x7f935c06a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:19.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.466+0000 7f935bfff700 1 -- 172.21.15.99:0/3203596790 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f935c15bd10 con 0x7f935c1577d0 2023-10-08T00:22:19.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.466+0000 7f935bfff700 1 --2- 172.21.15.99:0/3203596790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f935c1577d0 0x7f935c0cd770 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f934c0099b0 tx=0x7f934c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beff4463d5cbc21 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:19.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.466+0000 7f935affd700 1 -- 172.21.15.99:0/3203596790 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934c00b860 con 0x7f935c1577d0 2023-10-08T00:22:19.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.466+0000 7f935affd700 1 -- 172.21.15.99:0/3203596790 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f934c00b9c0 con 0x7f935c1577d0 2023-10-08T00:22:19.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.466+0000 7f935affd700 1 -- 172.21.15.99:0/3203596790 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934c00bd60 con 0x7f935c1577d0 2023-10-08T00:22:19.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.467+0000 7f936292a700 1 -- 172.21.15.99:0/3203596790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f935c1577d0 msgr2=0x7f935c0cd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:19.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.467+0000 7f936292a700 1 --2- 172.21.15.99:0/3203596790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f935c1577d0 0x7f935c0cd770 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f934c0099b0 tx=0x7f934c009250 comp rx=0 tx=0).stop 2023-10-08T00:22:19.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.467+0000 7f936292a700 1 -- 172.21.15.99:0/3203596790 shutdown_connections 2023-10-08T00:22:19.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.467+0000 7f936292a700 1 --2- 172.21.15.99:0/3203596790 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f935c06ac30 0x7f935c15c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:19.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.467+0000 7f936292a700 1 --2- 172.21.15.99:0/3203596790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f935c1577d0 0x7f935c0cd770 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:19.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.467+0000 7f936292a700 1 -- 172.21.15.99:0/3203596790 >> 172.21.15.99:0/3203596790 conn(0x7f935c068ce0 msgr2=0x7f935c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:19.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.467+0000 7f936292a700 1 -- 172.21.15.99:0/3203596790 shutdown_connections 2023-10-08T00:22:19.470 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.467+0000 7f936292a700 1 -- 172.21.15.99:0/3203596790 wait complete. 2023-10-08T00:22:19.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.468+0000 7f936292a700 1 Processor -- start 2023-10-08T00:22:19.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.468+0000 7f936292a700 1 -- start start 2023-10-08T00:22:19.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.469+0000 7f936292a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f935c06ac30 0x7f935c0aefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:19.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.469+0000 7f936292a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f935c1577d0 0x7f935c16a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:19.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.469+0000 7f936292a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f935c157ba0 0x7f935c0a8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:19.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.469+0000 7f936292a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935c0a85e0 con 0x7f935c1577d0 2023-10-08T00:22:19.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.469+0000 7f936292a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935c0a8880 con 0x7f935c157ba0 2023-10-08T00:22:19.474 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.469+0000 7f936292a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f935c16a970 con 0x7f935c06ac30 2023-10-08T00:22:19.474 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.469+0000 7f935b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f935c1577d0 0x7f935c16a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:19.474 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.469+0000 7f935b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f935c1577d0 0x7f935c16a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33594/0 (socket says 172.21.15.99:33594) 2023-10-08T00:22:19.474 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.469+0000 7f935b7fe700 1 -- 172.21.15.99:0/2961950452 learned_addr learned my addr 172.21.15.99:0/2961950452 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:19.474 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.469+0000 7f935bfff700 1 --2- 172.21.15.99:0/2961950452 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f935c06ac30 0x7f935c0aefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:19.474 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.469+0000 7f9360ec7700 1 --2- 172.21.15.99:0/2961950452 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f935c157ba0 0x7f935c0a8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:19.474 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.469+0000 7f935b7fe700 1 -- 172.21.15.99:0/2961950452 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f935c06ac30 msgr2=0x7f935c0aefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:19.474 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.469+0000 7f935b7fe700 1 --2- 172.21.15.99:0/2961950452 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f935c06ac30 0x7f935c0aefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:19.474 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.469+0000 7f935b7fe700 1 -- 172.21.15.99:0/2961950452 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f935c157ba0 msgr2=0x7f935c0a8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:19.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.469+0000 7f935b7fe700 1 --2- 172.21.15.99:0/2961950452 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f935c157ba0 0x7f935c0a8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:19.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.469+0000 7f935b7fe700 1 -- 172.21.15.99:0/2961950452 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f934c0094f0 con 0x7f935c1577d0 2023-10-08T00:22:19.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.469+0000 7f935b7fe700 1 --2- 172.21.15.99:0/2961950452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f935c1577d0 0x7f935c16a430 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f935000c4e0 tx=0x7f935000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:19.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.469+0000 7f9343fff700 1 -- 172.21.15.99:0/2961950452 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9350011590 con 0x7f935c1577d0 2023-10-08T00:22:19.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.469+0000 7f9343fff700 1 -- 172.21.15.99:0/2961950452 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9350011710 con 0x7f935c1577d0 2023-10-08T00:22:19.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.469+0000 7f936292a700 1 -- 172.21.15.99:0/2961950452 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f935c16ac40 con 0x7f935c1577d0 2023-10-08T00:22:19.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.469+0000 7f9343fff700 1 -- 172.21.15.99:0/2961950452 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9350011ad0 con 0x7f935c1577d0 2023-10-08T00:22:19.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.469+0000 7f936292a700 1 -- 172.21.15.99:0/2961950452 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f935c16b200 con 0x7f935c1577d0 2023-10-08T00:22:19.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.470+0000 7f936292a700 1 -- 172.21.15.99:0/2961950452 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f935c0a8a30 con 0x7f935c1577d0 2023-10-08T00:22:19.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.471+0000 7f9343fff700 1 -- 172.21.15.99:0/2961950452 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f935001b3b0 con 0x7f935c1577d0 2023-10-08T00:22:19.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.471+0000 7f9343fff700 1 --2- 172.21.15.99:0/2961950452 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9348066f30 0x7f93480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:19.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.471+0000 7f9343fff700 1 -- 172.21.15.99:0/2961950452 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f9350083650 con 0x7f935c1577d0 2023-10-08T00:22:19.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.471+0000 7f935bfff700 1 --2- 172.21.15.99:0/2961950452 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9348066f30 0x7f93480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:19.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.472+0000 7f935bfff700 1 --2- 172.21.15.99:0/2961950452 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9348066f30 0x7f93480693e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f934c0099b0 tx=0x7f934c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:19.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.475+0000 7f9343fff700 1 -- 172.21.15.99:0/2961950452 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f935c0a8a30 con 0x7f935c1577d0 2023-10-08T00:22:19.657 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.657+0000 7f936292a700 1 -- 172.21.15.99:0/2961950452 --> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f935c0a89f0 con 0x7f9348066f30 2023-10-08T00:22:19.660 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.659+0000 7f9343fff700 1 -- 172.21.15.99:0/2961950452 <== mgr.4099 v2:172.21.15.191:6832/85985 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50207 (secure 0 0 0) 0x7f935c0a89f0 con 0x7f9348066f30 2023-10-08T00:22:19.661 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:22:19.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.661+0000 7f936292a700 1 -- 172.21.15.99:0/2961950452 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9348066f30 msgr2=0x7f93480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:19.662 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.661+0000 7f936292a700 1 --2- 172.21.15.99:0/2961950452 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9348066f30 0x7f93480693e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f934c0099b0 tx=0x7f934c017000 comp rx=0 tx=0).stop 2023-10-08T00:22:19.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.661+0000 7f936292a700 1 -- 172.21.15.99:0/2961950452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f935c1577d0 msgr2=0x7f935c16a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:19.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.661+0000 7f936292a700 1 --2- 172.21.15.99:0/2961950452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f935c1577d0 0x7f935c16a430 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f935000c4e0 tx=0x7f935000c7f0 comp rx=0 tx=0).stop 2023-10-08T00:22:19.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.661+0000 7f936292a700 1 -- 172.21.15.99:0/2961950452 shutdown_connections 2023-10-08T00:22:19.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.661+0000 7f936292a700 1 --2- 172.21.15.99:0/2961950452 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f9348066f30 0x7f93480693e0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:19.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.661+0000 7f936292a700 1 --2- 172.21.15.99:0/2961950452 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f935c157ba0 0x7f935c0a8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:19.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.661+0000 7f936292a700 1 --2- 172.21.15.99:0/2961950452 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f935c06ac30 0x7f935c0aefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:19.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.661+0000 7f936292a700 1 --2- 172.21.15.99:0/2961950452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f935c1577d0 0x7f935c16a430 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:19.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.661+0000 7f936292a700 1 -- 172.21.15.99:0/2961950452 >> 172.21.15.99:0/2961950452 conn(0x7f935c068ce0 msgr2=0x7f935c158370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:19.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.662+0000 7f936292a700 1 -- 172.21.15.99:0/2961950452 shutdown_connections 2023-10-08T00:22:19.663 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:19.662+0000 7f936292a700 1 -- 172.21.15.99:0/2961950452 wait complete. 2023-10-08T00:22:19.664 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2023-10-08T00:22:19.674 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":530,"stamp":"2023-10-08T00:22:18.186025+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9605,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1552894,"statfs":{"total":1600000000,"available":1590163552,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52489239,"num_objects":-36,"num_object_clones":0,"num_object_copies":-72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-36,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-122,"num_write_kb":-85081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001282"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":313,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-08T00:22:11.250754+0000","last_change":"2023-10-08T00:10:43.728490+0000","last_active":"2023-10-08T00:22:11.250754+0000","last_peered":"2023-10-08T00:22:11.250754+0000","last_clean":"2023-10-08T00:22:11.250754+0000","last_became_active":"2023-10-08T00:10:43.728341+0000","last_became_peered":"2023-10-08T00:10:43.728341+0000","last_unstale":"2023-10-08T00:22:11.250754+0000","last_undegraded":"2023-10-08T00:22:11.250754+0000","last_fullsized":"2023-10-08T00:22:11.250754+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:10:42.716096+0000","last_clean_scrub_stamp":"2023-10-08T00:10:42.716096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574989,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1056,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194255,"statfs":{"total":200000000,"available":198918036,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.53000000000000003,"15min":0.52000000000000002},"min":{"1min":0.377,"5min":0.27800000000000002,"15min":0.26900000000000002},"max":{"1min":0.67500000000000004,"5min":0.77600000000000002,"15min":1.129},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.54900000000000004,"15min":0.54500000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.73799999999999999,"5min":0.754,"15min":1.0269999999999999},"last":0.42799999999999999}]},{"osd":1,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.57199999999999995,"15min":0.55500000000000005},"min":{"1min":0.372,"5min":0.32300000000000001,"15min":0.32000000000000001},"max":{"1min":0.70399999999999996,"5min":0.74299999999999999,"15min":0.82999999999999996},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.59799999999999998,"15min":0.58099999999999996},"min":{"1min":0.40500000000000003,"5min":0.39900000000000002,"15min":0.38200000000000001},"max":{"1min":0.746,"5min":0.79400000000000004,"15min":0.80100000000000005},"last":0.621}]},{"osd":2,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56699999999999995,"15min":0.55100000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.34200000000000003},"max":{"1min":0.748,"5min":0.76400000000000001,"15min":0.79900000000000004},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.54900000000000004,"15min":0.53800000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.72599999999999998,"5min":0.748,"15min":0.78500000000000003},"last":0.44700000000000001}]},{"osd":3,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.54000000000000004},"min":{"1min":0.42999999999999999,"5min":0.36699999999999999,"15min":0.33700000000000002},"max":{"1min":0.753,"5min":0.77400000000000002,"15min":0.79300000000000004},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57099999999999995,"15min":0.56499999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.75800000000000001,"5min":0.77600000000000002,"15min":0.82099999999999995},"last":0.496}]},{"osd":4,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.46600000000000003,"15min":0.496},"min":{"1min":0.36599999999999999,"5min":0.30399999999999999,"15min":0.26400000000000001},"max":{"1min":0.66800000000000004,"5min":0.71499999999999997,"15min":0.755},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.51300000000000001,"15min":0.51800000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.64500000000000002}]},{"osd":5,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.503,"15min":0.52500000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.73499999999999999,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.54100000000000004,"15min":0.56000000000000005},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.76500000000000001,"5min":0.80600000000000005,"15min":0.82799999999999996},"last":0.63}]},{"osd":6,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.41499999999999998,"5min":0.39600000000000002,"15min":0.33100000000000002},"max":{"1min":0.80400000000000005,"5min":0.84799999999999998,"15min":0.85399999999999998},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.58299999999999996,"15min":0.60399999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.27800000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.91500000000000004},"last":0.66100000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574989,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1056,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194255,"statfs":{"total":200000000,"available":198918036,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:22:15 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.52700000000000002,"15min":0.50700000000000001},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.69299999999999995,"5min":0.76300000000000001,"15min":1.012},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.53800000000000003,"15min":0.59299999999999997},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.29199999999999998},"max":{"1min":0.70099999999999996,"5min":0.77800000000000002,"15min":5.1230000000000002},"last":0.374}]},{"osd":1,"last update":"Sun Oct 8 00:22:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56499999999999995,"15min":0.54400000000000004},"min":{"1min":0.39600000000000002,"5min":0.314,"15min":0.28899999999999998},"max":{"1min":0.72099999999999997,"5min":0.754,"15min":1.1419999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.59199999999999997,"15min":0.57499999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.32100000000000001},"max":{"1min":0.69099999999999995,"5min":0.81799999999999995,"15min":1.1539999999999999},"last":0.59599999999999997}]},{"osd":2,"last update":"Sun Oct 8 00:22:15 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.56100000000000005,"15min":0.54300000000000004},"min":{"1min":0.41599999999999998,"5min":0.36599999999999999,"15min":0.32900000000000001},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":1.3149999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59199999999999997,"15min":0.57699999999999996},"min":{"1min":0.44800000000000001,"5min":0.374,"15min":0.29399999999999998},"max":{"1min":0.75700000000000001,"5min":0.83699999999999997,"15min":1.3280000000000001},"last":0.58199999999999996}]},{"osd":3,"last update":"Sun Oct 8 00:22:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57099999999999995,"15min":0.55600000000000005},"min":{"1min":0.41399999999999998,"5min":0.32000000000000001,"15min":0.25700000000000001},"max":{"1min":0.76900000000000002,"5min":0.78500000000000003,"15min":1.046},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.58699999999999997,"15min":0.56299999999999994},"min":{"1min":0.39500000000000002,"5min":0.38600000000000001,"15min":0.32900000000000001},"max":{"1min":0.74399999999999999,"5min":0.81799999999999995,"15min":1.4390000000000001},"last":0.49099999999999999}]},{"osd":4,"last update":"Sun Oct 8 00:22:15 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.45300000000000001,"15min":0.47799999999999998},"min":{"1min":0.30499999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.60699999999999998,"5min":0.70099999999999996,"15min":0.80100000000000005},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54600000000000004,"15min":0.55600000000000005},"min":{"1min":0.40000000000000002,"5min":0.375,"15min":0.33800000000000002},"max":{"1min":0.71199999999999997,"5min":0.77600000000000002,"15min":0.89100000000000001},"last":0.64000000000000001}]},{"osd":5,"last update":"Sun Oct 8 00:22:15 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.47699999999999998,"15min":0.48299999999999998},"min":{"1min":0.308,"5min":0.29299999999999998,"15min":0.29099999999999998},"max":{"1min":0.66000000000000003,"5min":0.73499999999999999,"15min":0.80500000000000005},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.497,"15min":0.52800000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.28699999999999998},"max":{"1min":0.70299999999999996,"5min":0.71099999999999997,"15min":0.84299999999999997},"last":0.65300000000000002}]},{"osd":7,"last update":"Sun Oct 8 00:22:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.50900000000000001,"15min":0.52600000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.307},"max":{"1min":0.71699999999999997,"5min":0.754,"15min":0.91500000000000004},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.53700000000000003,"15min":0.54200000000000004},"min":{"1min":0.37,"5min":0.34899999999999998,"15min":0.32400000000000001},"max":{"1min":0.71999999999999997,"5min":0.76400000000000001,"15min":0.79900000000000004},"last":0.623}]}]},{"osd":1,"up_from":13,"seq":55834574989,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1633,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193679,"statfs":{"total":200000000,"available":198327668,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.35999999999999999,"15min":0.33300000000000002},"min":{"1min":0.223,"5min":0.216,"15min":0.19500000000000001},"max":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.41999999999999998,"15min":0.39500000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.67300000000000004},"last":0.35199999999999998}]},{"osd":2,"last update":"Sun Oct 8 00:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.38200000000000001,"15min":0.35499999999999998},"min":{"1min":0.27400000000000002,"5min":0.25900000000000001,"15min":0.22600000000000001},"max":{"1min":0.56999999999999995,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39500000000000002,"15min":0.36899999999999999},"min":{"1min":0.28499999999999998,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.53400000000000003,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.32200000000000001}]},{"osd":3,"last update":"Sun Oct 8 00:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.42499999999999999,"15min":0.39300000000000002},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.58799999999999997,"5min":0.63,"15min":0.70999999999999996},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.375,"15min":0.35599999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.222},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.36399999999999999}]},{"osd":4,"last update":"Sun Oct 8 00:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57699999999999996,"15min":0.55600000000000005},"min":{"1min":0.42999999999999999,"5min":0.34999999999999998,"15min":0.32600000000000001},"max":{"1min":0.82199999999999995,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.498},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59899999999999998,"15min":0.57999999999999996},"min":{"1min":0.42199999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.76000000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.57499999999999996}]},{"osd":5,"last update":"Sun Oct 8 00:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57799999999999996,"15min":0.55900000000000005},"min":{"1min":0.39900000000000002,"5min":0.35099999999999998,"15min":0.33500000000000002},"max":{"1min":0.71799999999999997,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60299999999999998,"15min":0.58699999999999997},"min":{"1min":0.41799999999999998,"5min":0.39500000000000002,"15min":0.30399999999999999},"max":{"1min":0.76900000000000002,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.59899999999999998}]},{"osd":6,"last update":"Sun Oct 8 00:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.58299999999999996,"15min":0.57599999999999996},"min":{"1min":0.41499999999999998,"5min":0.37,"15min":0.317},"max":{"1min":0.755,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59299999999999997,"15min":0.57599999999999996},"min":{"1min":0.40799999999999997,"5min":0.377,"15min":0.33200000000000002},"max":{"1min":0.748,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.61399999999999999}]},{"osd":7,"last update":"Sun Oct 8 00:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58699999999999997,"15min":0.57599999999999996},"min":{"1min":0.439,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.746,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.63200000000000001,"15min":0.61099999999999999},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.39100000000000001},"max":{"1min":0.79400000000000004,"5min":0.872,"15min":0.96699999999999997},"last":0.63200000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574989,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1056,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194255,"statfs":{"total":200000000,"available":198918036,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 00:22:08 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.33900000000000002,"15min":0.33500000000000002},"min":{"1min":0.218,"5min":0.20399999999999999,"15min":0.193},"max":{"1min":0.501,"5min":0.54900000000000004,"15min":1.0640000000000001},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.36899999999999999,"15min":0.35499999999999998},"min":{"1min":0.23499999999999999,"5min":0.216,"15min":0.20300000000000001},"max":{"1min":0.48999999999999999,"5min":0.66500000000000004,"15min":0.87},"last":0.33900000000000002}]},{"osd":2,"last update":"Sun Oct 8 00:22:08 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38900000000000001,"15min":0.38700000000000001},"min":{"1min":0.25800000000000001,"5min":0.249,"15min":0.24099999999999999},"max":{"1min":0.51600000000000001,"5min":0.57199999999999995,"15min":0.93799999999999994},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.35899999999999999,"15min":0.52500000000000002},"min":{"1min":0.23599999999999999,"5min":0.23499999999999999,"15min":0.185},"max":{"1min":0.45700000000000002,"5min":0.59899999999999998,"15min":12.019},"last":0.26000000000000001}]},{"osd":3,"last update":"Sun Oct 8 00:22:08 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.39000000000000001,"15min":0.371},"min":{"1min":0.27300000000000002,"5min":0.25700000000000001,"15min":0.22500000000000001},"max":{"1min":0.46700000000000003,"5min":0.628,"15min":1.1020000000000001},"last":0.307},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.42099999999999999,"15min":0.40799999999999997},"min":{"1min":0.28499999999999998,"5min":0.26700000000000002,"15min":0.246},"max":{"1min":0.52000000000000002,"5min":0.66100000000000003,"15min":1.0409999999999999},"last":0.31900000000000001}]},{"osd":4,"last update":"Sun Oct 8 00:22:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.432,"5min":0.377,"15min":0.35899999999999999},"max":{"1min":0.72499999999999998,"5min":0.89700000000000002,"15min":1.1779999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.624,"15min":0.622},"min":{"1min":0.46899999999999997,"5min":0.41099999999999998,"15min":0.40300000000000002},"max":{"1min":0.78000000000000003,"5min":0.93899999999999995,"15min":0.96199999999999997},"last":0.59299999999999997}]},{"osd":5,"last update":"Sun Oct 8 00:22:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58999999999999997,"15min":0.75900000000000001},"min":{"1min":0.44700000000000001,"5min":0.378,"15min":0.32000000000000001},"max":{"1min":0.83299999999999996,"5min":0.90700000000000003,"15min":11.992000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.60799999999999998,"15min":0.59599999999999997},"min":{"1min":0.443,"5min":0.379,"15min":0.379},"max":{"1min":0.77100000000000002,"5min":0.91600000000000004,"15min":1.1299999999999999},"last":0.503}]},{"osd":6,"last update":"Sun Oct 8 00:22:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.60099999999999998,"15min":0.59299999999999997},"min":{"1min":0.442,"5min":0.372,"15min":0.372},"max":{"1min":0.73899999999999999,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.61299999999999999,"15min":0.76100000000000001},"min":{"1min":0.46899999999999997,"5min":0.39600000000000002,"15min":0.38100000000000001},"max":{"1min":0.85799999999999998,"5min":0.93999999999999995,"15min":11.927},"last":0.63400000000000001}]},{"osd":7,"last update":"Sun Oct 8 00:22:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.60099999999999998,"15min":0.76700000000000002},"min":{"1min":0.42199999999999999,"5min":0.38600000000000001,"15min":0.35699999999999998},"max":{"1min":0.76400000000000001,"5min":0.95099999999999996,"15min":11.965},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.63700000000000001,"15min":0.626},"min":{"1min":0.42399999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.79900000000000004,"5min":0.89800000000000002,"15min":1.1539999999999999},"last":0.61299999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574989,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1056,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194255,"statfs":{"total":200000000,"available":198918036,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.32900000000000001,"15min":0.35799999999999998},"min":{"1min":0.20100000000000001,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.47599999999999998,"5min":0.56899999999999995,"15min":2.5139999999999998},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.34699999999999998,"15min":0.36899999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.20599999999999999},"max":{"1min":0.51200000000000001,"5min":0.59099999999999997,"15min":2.1309999999999998},"last":0.36299999999999999}]},{"osd":1,"last update":"Sun Oct 8 00:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.35499999999999998,"15min":0.37},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":2.2639999999999998},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.374,"5min":0.38400000000000001,"15min":0.40300000000000002},"min":{"1min":0.254,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.54400000000000004,"5min":0.57199999999999995,"15min":2.5379999999999998},"last":0.42499999999999999}]},{"osd":3,"last update":"Sun Oct 8 00:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.40400000000000003,"15min":0.41099999999999998},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.64400000000000002,"5min":0.66900000000000004,"15min":2.2890000000000001},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.41899999999999998,"15min":0.435},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.55900000000000005,"5min":0.64300000000000002,"15min":2.5619999999999998},"last":0.32900000000000001}]},{"osd":4,"last update":"Sun Oct 8 00:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54400000000000004,"15min":0.57999999999999996},"min":{"1min":0.31,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.77300000000000002,"5min":0.80800000000000005,"15min":2.488},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58599999999999997,"15min":0.59599999999999997},"min":{"1min":0.42999999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.77300000000000002,"5min":0.92400000000000004,"15min":2.073},"last":0.40899999999999997}]},{"osd":5,"last update":"Sun Oct 8 00:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.60899999999999999,"15min":0.63},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.76300000000000001,"5min":0.94299999999999995,"15min":2.2069999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.61699999999999999,"15min":0.64000000000000001},"min":{"1min":0.42199999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77300000000000002,"5min":0.91000000000000003,"15min":2.456},"last":0.45100000000000001}]},{"osd":6,"last update":"Sun Oct 8 00:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59599999999999997,"15min":0.60699999999999998},"min":{"1min":0.39100000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.80200000000000005,"5min":0.872,"15min":2.0339999999999998},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59399999999999997,"15min":0.60499999999999998},"min":{"1min":0.40799999999999997,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.75,"5min":0.94099999999999995,"15min":2.1699999999999999},"last":0.52400000000000002}]},{"osd":7,"last update":"Sun Oct 8 00:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60099999999999998,"15min":0.61899999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82199999999999995,"5min":0.91500000000000004,"15min":2.1030000000000002},"last":0.441},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60699999999999998,"15min":0.61799999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.77700000000000002,"5min":1.0169999999999999,"15min":2.2349999999999999},"last":0.56399999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574989,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1056,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194255,"statfs":{"total":200000000,"available":198918036,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.33600000000000002,"15min":0.34699999999999998},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.192},"max":{"1min":0.58899999999999997,"5min":0.59299999999999997,"15min":1.46},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.31900000000000001,"5min":0.34999999999999998,"15min":0.34200000000000003},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.192},"max":{"1min":0.56000000000000005,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.33300000000000002}]},{"osd":1,"last update":"Sun Oct 8 00:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.36299999999999999,"15min":0.55500000000000005},"min":{"1min":0.25,"5min":0.247,"15min":0.20000000000000001},"max":{"1min":0.52700000000000002,"5min":0.57499999999999996,"15min":12.351000000000001},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38700000000000001,"15min":0.372},"min":{"1min":0.23200000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.54500000000000004,"5min":0.58199999999999996,"15min":0.71999999999999997},"last":0.44700000000000001}]},{"osd":2,"last update":"Sun Oct 8 00:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.40799999999999997,"15min":0.40600000000000003},"min":{"1min":0.248,"5min":0.248,"15min":0.23499999999999999},"max":{"1min":0.58099999999999996,"5min":0.59699999999999998,"15min":1.5489999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.41999999999999998,"15min":0.39800000000000002},"min":{"1min":0.27100000000000002,"5min":0.25700000000000001,"15min":0.246},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.753},"last":0.41099999999999998}]},{"osd":4,"last update":"Sun Oct 8 00:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.625,"15min":0.78400000000000003},"min":{"1min":0.39800000000000002,"5min":0.36899999999999999,"15min":0.34899999999999998},"max":{"1min":0.81399999999999995,"5min":0.96699999999999997,"15min":11.333},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.63400000000000001,"15min":0.80400000000000005},"min":{"1min":0.39100000000000001,"5min":0.38100000000000001,"15min":0.36099999999999999},"max":{"1min":0.89500000000000002,"5min":0.97699999999999998,"15min":12.321999999999999},"last":0.45900000000000002}]},{"osd":5,"last update":"Sun Oct 8 00:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.61199999999999999,"15min":0.79800000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35499999999999998},"max":{"1min":0.86299999999999999,"5min":0.93500000000000005,"15min":12.291},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.64400000000000002,"15min":0.621},"min":{"1min":0.44800000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80500000000000005,"5min":1.143,"15min":1.143},"last":0.47399999999999998}]},{"osd":6,"last update":"Sun Oct 8 00:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.59899999999999998,"15min":0.755},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.33700000000000002},"max":{"1min":0.91200000000000003,"5min":0.91500000000000004,"15min":11.288},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.62,"15min":0.79100000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.36599999999999999},"max":{"1min":0.84599999999999997,"5min":0.94599999999999995,"15min":12.253},"last":0.435}]},{"osd":7,"last update":"Sun Oct 8 00:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63200000000000001,"15min":0.60299999999999998},"min":{"1min":0.45700000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86399999999999999,"5min":1.0920000000000001,"15min":1.3109999999999999},"last":0.501},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61599999999999999,"15min":0.78400000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.38600000000000001},"max":{"1min":0.879,"5min":0.89300000000000002,"15min":11.363},"last":0.42799999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574989,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1056,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194255,"statfs":{"total":200000000,"available":198918036,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.56699999999999995,"15min":0.56100000000000005},"min":{"1min":0.36299999999999999,"5min":0.32300000000000001,"15min":0.193},"max":{"1min":0.80900000000000005,"5min":0.84799999999999998,"15min":0.93000000000000005},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.60099999999999998,"15min":0.58699999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.30399999999999999},"max":{"1min":0.70199999999999996,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.45100000000000001}]},{"osd":1,"last update":"Sun Oct 8 00:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.54300000000000004,"15min":0.52900000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.33700000000000002},"max":{"1min":0.749,"5min":0.751,"15min":0.78100000000000003},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57299999999999995,"15min":0.54600000000000004},"min":{"1min":0.46899999999999997,"5min":0.34300000000000003,"15min":0.30199999999999999},"max":{"1min":0.71799999999999997,"5min":0.80600000000000005,"15min":0.81299999999999994},"last":0.33500000000000002}]},{"osd":2,"last update":"Sun Oct 8 00:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.54800000000000004,"15min":0.52700000000000002},"min":{"1min":0.35999999999999999,"5min":0.32500000000000001,"15min":0.32100000000000001},"max":{"1min":0.78400000000000003,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.56499999999999995,"15min":0.53400000000000003},"min":{"1min":0.41599999999999998,"5min":0.35399999999999998,"15min":0.32600000000000001},"max":{"1min":0.65300000000000002,"5min":0.79600000000000004,"15min":0.81699999999999995},"last":0.37}]},{"osd":3,"last update":"Sun Oct 8 00:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.57999999999999996,"15min":0.55000000000000004},"min":{"1min":0.41099999999999998,"5min":0.35199999999999998,"15min":0.28799999999999998},"max":{"1min":0.77100000000000002,"5min":0.82299999999999995,"15min":0.85399999999999998},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.55200000000000005,"15min":0.54000000000000004},"min":{"1min":0.433,"5min":0.37,"15min":0.36199999999999999},"max":{"1min":0.68400000000000005,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.45800000000000002}]},{"osd":5,"last update":"Sun Oct 8 00:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.47299999999999998,"15min":0.47899999999999998},"min":{"1min":0.33900000000000002,"5min":0.314,"15min":0.27500000000000002},"max":{"1min":0.68600000000000005,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.501,"15min":0.504},"min":{"1min":0.34100000000000003,"5min":0.317,"15min":0.24399999999999999},"max":{"1min":0.72399999999999998,"5min":0.79100000000000004,"15min":0.94899999999999995},"last":0.46800000000000003}]},{"osd":6,"last update":"Sun Oct 8 00:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.48899999999999999,"15min":0.48599999999999999},"min":{"1min":0.371,"5min":0.32400000000000001,"15min":0.26400000000000001},"max":{"1min":0.70899999999999996,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52500000000000002,"15min":0.53700000000000003},"min":{"1min":0.36299999999999999,"5min":0.34799999999999998,"15min":0.29799999999999999},"max":{"1min":0.73699999999999999,"5min":0.76200000000000001,"15min":0.80100000000000005},"last":0.50600000000000001}]},{"osd":7,"last update":"Sun Oct 8 00:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.53900000000000003,"15min":0.54300000000000004},"min":{"1min":0.34999999999999998,"5min":0.33200000000000002,"15min":0.27900000000000003},"max":{"1min":0.76200000000000001,"5min":0.83599999999999997,"15min":0.90100000000000002},"last":0.442},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56999999999999995,"15min":0.57399999999999995},"min":{"1min":0.42999999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.751,"5min":0.88,"15min":0.88},"last":0.49199999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574989,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1633,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193679,"statfs":{"total":200000000,"available":198327668,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.54400000000000004,"15min":0.50600000000000001},"min":{"1min":0.38,"5min":0.36399999999999999,"15min":0.249},"max":{"1min":0.71599999999999997,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54100000000000004,"15min":0.54800000000000004},"min":{"1min":0.39000000000000001,"5min":0.29799999999999999,"15min":0.29399999999999998},"max":{"1min":0.75700000000000001,"5min":0.81699999999999995,"15min":0.88600000000000001},"last":0.50700000000000001}]},{"osd":1,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.55600000000000005,"15min":0.53600000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.32500000000000001},"max":{"1min":0.68300000000000005,"5min":0.72599999999999998,"15min":0.82999999999999996},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.53800000000000003,"15min":0.52500000000000002},"min":{"1min":0.34599999999999997,"5min":0.34499999999999997,"15min":0.33000000000000002},"max":{"1min":0.68700000000000006,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.47999999999999998}]},{"osd":2,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55700000000000005,"15min":0.53700000000000003},"min":{"1min":0.42999999999999999,"5min":0.39600000000000002,"15min":0.32400000000000001},"max":{"1min":0.71599999999999997,"5min":0.749,"15min":0.86199999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56100000000000005,"15min":0.56000000000000005},"min":{"1min":0.38100000000000001,"5min":0.33400000000000002,"15min":0.29599999999999999},"max":{"1min":0.749,"5min":0.80000000000000004,"15min":0.95399999999999996},"last":0.63200000000000001}]},{"osd":3,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57299999999999995,"15min":0.55700000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.316},"max":{"1min":0.71999999999999997,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56599999999999995,"15min":0.56899999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.72399999999999998,"5min":0.73499999999999999,"15min":1.0529999999999999},"last":0.64800000000000002}]},{"osd":4,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.47299999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.27700000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.504,"5min":0.51400000000000001,"15min":0.51700000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.28899999999999998},"max":{"1min":0.747,"5min":0.77500000000000002,"15min":0.91500000000000004},"last":0.56999999999999995}]},{"osd":6,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.33600000000000002,"5min":0.28199999999999997,"15min":0.27400000000000002},"max":{"1min":0.79400000000000004,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52300000000000002,"15min":0.51100000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.30499999999999999},"max":{"1min":0.81499999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.59499999999999997}]},{"osd":7,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.54200000000000004,"15min":0.53100000000000003},"min":{"1min":0.33500000000000002,"5min":0.318,"15min":0.29399999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81999999999999995},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56899999999999995,"15min":0.56200000000000006},"min":{"1min":0.38300000000000001,"5min":0.35599999999999998,"15min":0.32500000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.66500000000000004}]}]}],"pool_statfs":[]}} 2023-10-08T00:22:19.678 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-08T00:10:42.716096+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=8, tm_hour=0, tm_min=10, tm_sec=42, tm_wday=6, tm_yday=281, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=8, tm_hour=0, tm_min=22, tm_sec=14, tm_wday=6, tm_yday=281, tm_isdst=0) 2023-10-08T00:22:19.678 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-08T00:22:24.360 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:22:24.363 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:22:39.679 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T00:22:39.804 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.803+0000 7f2702bc8700 1 Processor -- start 2023-10-08T00:22:39.805 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.804+0000 7f2702bc8700 1 -- start start 2023-10-08T00:22:39.807 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.804+0000 7f2702bc8700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f26fc159cf0 0x7f26fc15e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:39.808 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.804+0000 7f2702bc8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26fc15a0c0 0x7f26fc0ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:39.808 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.804+0000 7f2702bc8700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f26fc0ce1a0 0x7f26fc0ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:39.808 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.804+0000 7f2702bc8700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26fc0cca20 con 0x7f26fc0ce1a0 2023-10-08T00:22:39.808 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.804+0000 7f2702bc8700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26fc15ea40 con 0x7f26fc159cf0 2023-10-08T00:22:39.808 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.804+0000 7f2702bc8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26fc067660 con 0x7f26fc15a0c0 2023-10-08T00:22:39.808 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.804+0000 7f2701165700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f26fc0ce1a0 0x7f26fc0ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:39.809 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.804+0000 7f2701165700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f26fc0ce1a0 0x7f26fc0ceff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41374/0 (socket says 172.21.15.99:41374) 2023-10-08T00:22:39.809 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.805+0000 7f2701165700 1 -- 172.21.15.99:0/2764066932 learned_addr learned my addr 172.21.15.99:0/2764066932 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:39.809 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.805+0000 7f26fbfff700 1 --2- 172.21.15.99:0/2764066932 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26fc15a0c0 0x7f26fc0ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:39.809 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.805+0000 7f2701165700 1 -- 172.21.15.99:0/2764066932 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26fc15a0c0 msgr2=0x7f26fc0ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:39.809 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.805+0000 7f2701165700 1 --2- 172.21.15.99:0/2764066932 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26fc15a0c0 0x7f26fc0ceab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:39.809 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.805+0000 7f2700964700 1 --2- 172.21.15.99:0/2764066932 >> v2:172.21.15.191:3300/0 conn(0x7f26fc159cf0 0x7f26fc15e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:39.809 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.805+0000 7f2701165700 1 -- 172.21.15.99:0/2764066932 >> v2:172.21.15.191:3300/0 conn(0x7f26fc159cf0 msgr2=0x7f26fc15e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:39.809 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.805+0000 7f2701165700 1 --2- 172.21.15.99:0/2764066932 >> v2:172.21.15.191:3300/0 conn(0x7f26fc159cf0 0x7f26fc15e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:39.809 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.805+0000 7f2701165700 1 -- 172.21.15.99:0/2764066932 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26fc067f00 con 0x7f26fc0ce1a0 2023-10-08T00:22:39.810 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.805+0000 7f26fbfff700 1 --2- 172.21.15.99:0/2764066932 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26fc15a0c0 0x7f26fc0ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:22:39.810 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.805+0000 7f2701165700 1 --2- 172.21.15.99:0/2764066932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26fc0ce1a0 0x7f26fc0ceff0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f26fc0777e0 tx=0x7f26f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81eb3b286d46e25e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:39.810 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.805+0000 7f26fb7fe700 1 -- 172.21.15.99:0/2764066932 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f400b860 con 0x7f26fc0ce1a0 2023-10-08T00:22:39.810 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.805+0000 7f26fb7fe700 1 -- 172.21.15.99:0/2764066932 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f400b9c0 con 0x7f26fc0ce1a0 2023-10-08T00:22:39.810 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.805+0000 7f26fb7fe700 1 -- 172.21.15.99:0/2764066932 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f400bd60 con 0x7f26fc0ce1a0 2023-10-08T00:22:39.810 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.806+0000 7f2702bc8700 1 -- 172.21.15.99:0/2764066932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26fc0ce1a0 msgr2=0x7f26fc0ceff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:39.810 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.806+0000 7f2702bc8700 1 --2- 172.21.15.99:0/2764066932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26fc0ce1a0 0x7f26fc0ceff0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f26fc0777e0 tx=0x7f26f4009250 comp rx=0 tx=0).stop 2023-10-08T00:22:39.810 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.806+0000 7f2702bc8700 1 -- 172.21.15.99:0/2764066932 shutdown_connections 2023-10-08T00:22:39.810 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.806+0000 7f2702bc8700 1 --2- 172.21.15.99:0/2764066932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26fc0ce1a0 0x7f26fc0ceff0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:39.811 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.806+0000 7f2702bc8700 1 --2- 172.21.15.99:0/2764066932 >> v2:172.21.15.191:3300/0 conn(0x7f26fc159cf0 0x7f26fc15e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:39.811 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.806+0000 7f2702bc8700 1 --2- 172.21.15.99:0/2764066932 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26fc15a0c0 0x7f26fc0ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:39.811 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.806+0000 7f2702bc8700 1 -- 172.21.15.99:0/2764066932 >> 172.21.15.99:0/2764066932 conn(0x7f26fc06c6e0 msgr2=0x7f26fc069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:39.811 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.806+0000 7f2702bc8700 1 -- 172.21.15.99:0/2764066932 shutdown_connections 2023-10-08T00:22:39.811 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.806+0000 7f2702bc8700 1 -- 172.21.15.99:0/2764066932 wait complete. 2023-10-08T00:22:39.811 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.807+0000 7f2702bc8700 1 Processor -- start 2023-10-08T00:22:39.811 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.807+0000 7f2702bc8700 1 -- start start 2023-10-08T00:22:39.814 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.808+0000 7f2702bc8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26fc0ce1a0 0x7f26fc0c77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:39.814 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.808+0000 7f2702bc8700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26fc159cf0 0x7f26fc16a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:39.814 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.808+0000 7f2702bc8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26fc15a0c0 0x7f26fc0c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:39.814 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.808+0000 7f2702bc8700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26fc0b1e60 con 0x7f26fc0ce1a0 2023-10-08T00:22:39.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.808+0000 7f2702bc8700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26fc16a930 con 0x7f26fc159cf0 2023-10-08T00:22:39.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.808+0000 7f2702bc8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26fc0c61c0 con 0x7f26fc15a0c0 2023-10-08T00:22:39.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.808+0000 7f2700964700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26fc0ce1a0 0x7f26fc0c77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:39.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.808+0000 7f2700964700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26fc0ce1a0 0x7f26fc0c77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41384/0 (socket says 172.21.15.99:41384) 2023-10-08T00:22:39.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.808+0000 7f2700964700 1 -- 172.21.15.99:0/453733995 learned_addr learned my addr 172.21.15.99:0/453733995 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:39.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.808+0000 7f2701165700 1 --2- 172.21.15.99:0/453733995 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26fc15a0c0 0x7f26fc0c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:39.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.808+0000 7f26fbfff700 1 --2- 172.21.15.99:0/453733995 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26fc159cf0 0x7f26fc16a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:39.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.808+0000 7f2700964700 1 -- 172.21.15.99:0/453733995 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26fc15a0c0 msgr2=0x7f26fc0c8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:39.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.808+0000 7f2700964700 1 --2- 172.21.15.99:0/453733995 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26fc15a0c0 0x7f26fc0c8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:39.815 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.808+0000 7f2700964700 1 -- 172.21.15.99:0/453733995 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26fc159cf0 msgr2=0x7f26fc16a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:39.816 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.808+0000 7f2700964700 1 --2- 172.21.15.99:0/453733995 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26fc159cf0 0x7f26fc16a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:39.816 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.808+0000 7f2700964700 1 -- 172.21.15.99:0/453733995 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f40094f0 con 0x7f26fc0ce1a0 2023-10-08T00:22:39.816 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.808+0000 7f2701165700 1 --2- 172.21.15.99:0/453733995 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26fc15a0c0 0x7f26fc0c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:22:39.816 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.808+0000 7f26fbfff700 1 --2- 172.21.15.99:0/453733995 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26fc159cf0 0x7f26fc16a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:22:39.816 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.808+0000 7f2700964700 1 --2- 172.21.15.99:0/453733995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26fc0ce1a0 0x7f26fc0c77c0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f26ec00cfc0 tx=0x7f26ec0078b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:39.816 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.808+0000 7f26f8ff9700 1 -- 172.21.15.99:0/453733995 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ec016430 con 0x7f26fc0ce1a0 2023-10-08T00:22:39.816 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.808+0000 7f26f8ff9700 1 -- 172.21.15.99:0/453733995 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26ec016590 con 0x7f26fc0ce1a0 2023-10-08T00:22:39.816 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.808+0000 7f2702bc8700 1 -- 172.21.15.99:0/453733995 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26fc0c64a0 con 0x7f26fc0ce1a0 2023-10-08T00:22:39.816 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.808+0000 7f26f8ff9700 1 -- 172.21.15.99:0/453733995 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ec016930 con 0x7f26fc0ce1a0 2023-10-08T00:22:39.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.809+0000 7f2702bc8700 1 -- 172.21.15.99:0/453733995 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26fc0c6a60 con 0x7f26fc0ce1a0 2023-10-08T00:22:39.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.809+0000 7f2702bc8700 1 -- 172.21.15.99:0/453733995 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26fc0b1fd0 con 0x7f26fc0ce1a0 2023-10-08T00:22:39.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.810+0000 7f26f8ff9700 1 -- 172.21.15.99:0/453733995 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f26ec003020 con 0x7f26fc0ce1a0 2023-10-08T00:22:39.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.810+0000 7f26f8ff9700 1 --2- 172.21.15.99:0/453733995 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f26e8066f30 0x7f26e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:39.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.810+0000 7f26f8ff9700 1 -- 172.21.15.99:0/453733995 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f26ec012070 con 0x7f26fc0ce1a0 2023-10-08T00:22:39.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.810+0000 7f26fbfff700 1 --2- 172.21.15.99:0/453733995 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f26e8066f30 0x7f26e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:39.817 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.811+0000 7f26fbfff700 1 --2- 172.21.15.99:0/453733995 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f26e8066f30 0x7f26e80693e0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f26f0006fd0 tx=0x7f26f0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:39.818 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.815+0000 7f26f8ff9700 1 -- 172.21.15.99:0/453733995 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f26fc0b1fd0 con 0x7f26fc0ce1a0 2023-10-08T00:22:39.996 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.996+0000 7f2702bc8700 1 -- 172.21.15.99:0/453733995 --> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f26fc0b1fa0 con 0x7f26e8066f30 2023-10-08T00:22:39.999 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:39.998+0000 7f26f8ff9700 1 -- 172.21.15.99:0/453733995 <== mgr.4099 v2:172.21.15.191:6832/85985 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50306 (secure 0 0 0) 0x7f26fc0b1fa0 con 0x7f26e8066f30 2023-10-08T00:22:40.000 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:22:40.001 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.000+0000 7f2702bc8700 1 -- 172.21.15.99:0/453733995 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f26e8066f30 msgr2=0x7f26e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:40.001 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.000+0000 7f2702bc8700 1 --2- 172.21.15.99:0/453733995 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f26e8066f30 0x7f26e80693e0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f26f0006fd0 tx=0x7f26f0008040 comp rx=0 tx=0).stop 2023-10-08T00:22:40.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.000+0000 7f2702bc8700 1 -- 172.21.15.99:0/453733995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26fc0ce1a0 msgr2=0x7f26fc0c77c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:40.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.000+0000 7f2702bc8700 1 --2- 172.21.15.99:0/453733995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26fc0ce1a0 0x7f26fc0c77c0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f26ec00cfc0 tx=0x7f26ec0078b0 comp rx=0 tx=0).stop 2023-10-08T00:22:40.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.000+0000 7f2702bc8700 1 -- 172.21.15.99:0/453733995 shutdown_connections 2023-10-08T00:22:40.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.000+0000 7f2702bc8700 1 --2- 172.21.15.99:0/453733995 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f26e8066f30 0x7f26e80693e0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:40.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.000+0000 7f2702bc8700 1 --2- 172.21.15.99:0/453733995 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26fc15a0c0 0x7f26fc0c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:40.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.000+0000 7f2702bc8700 1 --2- 172.21.15.99:0/453733995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26fc0ce1a0 0x7f26fc0c77c0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:40.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.000+0000 7f2702bc8700 1 --2- 172.21.15.99:0/453733995 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26fc159cf0 0x7f26fc16a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:40.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.000+0000 7f2702bc8700 1 -- 172.21.15.99:0/453733995 >> 172.21.15.99:0/453733995 conn(0x7f26fc06c6e0 msgr2=0x7f26fc0c38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:40.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.001+0000 7f2702bc8700 1 -- 172.21.15.99:0/453733995 shutdown_connections 2023-10-08T00:22:40.002 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.001+0000 7f2702bc8700 1 -- 172.21.15.99:0/453733995 wait complete. 2023-10-08T00:22:40.003 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2023-10-08T00:22:40.015 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":540,"stamp":"2023-10-08T00:22:38.189836+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9605,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1552894,"statfs":{"total":1600000000,"available":1590163552,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002370"},"pg_stats":[{"pgid":"1.0","version":"192'119","reported_seq":325,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-08T00:22:20.378180+0000","last_change":"2023-10-08T00:22:20.378158+0000","last_active":"2023-10-08T00:22:20.378180+0000","last_peered":"2023-10-08T00:22:20.378180+0000","last_clean":"2023-10-08T00:22:20.378180+0000","last_became_active":"2023-10-08T00:10:43.728341+0000","last_became_peered":"2023-10-08T00:10:43.728341+0000","last_unstale":"2023-10-08T00:22:20.378180+0000","last_undegraded":"2023-10-08T00:22:20.378180+0000","last_fullsized":"2023-10-08T00:22:20.378180+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"192'119","last_scrub_stamp":"2023-10-08T00:22:20.378109+0000","last_deep_scrub":"192'119","last_deep_scrub_stamp":"2023-10-08T00:22:20.378109+0000","last_clean_scrub_stamp":"2023-10-08T00:22:20.378109+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:28:02.074046+0000","scrub_duration":0.0024256249999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574993,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1056,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194255,"statfs":{"total":200000000,"available":198918036,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.53000000000000003,"15min":0.52000000000000002},"min":{"1min":0.377,"5min":0.27800000000000002,"15min":0.26900000000000002},"max":{"1min":0.67500000000000004,"5min":0.77600000000000002,"15min":1.129},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.54900000000000004,"15min":0.54500000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.73799999999999999,"5min":0.754,"15min":1.0269999999999999},"last":0.59199999999999997}]},{"osd":1,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.57199999999999995,"15min":0.55500000000000005},"min":{"1min":0.372,"5min":0.32300000000000001,"15min":0.32000000000000001},"max":{"1min":0.70399999999999996,"5min":0.74299999999999999,"15min":0.82999999999999996},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.59799999999999998,"15min":0.58099999999999996},"min":{"1min":0.40500000000000003,"5min":0.39900000000000002,"15min":0.38200000000000001},"max":{"1min":0.746,"5min":0.79400000000000004,"15min":0.80100000000000005},"last":0.63400000000000001}]},{"osd":2,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56699999999999995,"15min":0.55100000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.34200000000000003},"max":{"1min":0.748,"5min":0.76400000000000001,"15min":0.79900000000000004},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.54900000000000004,"15min":0.53800000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.72599999999999998,"5min":0.748,"15min":0.78500000000000003},"last":0.64600000000000002}]},{"osd":3,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.54000000000000004},"min":{"1min":0.42999999999999999,"5min":0.36699999999999999,"15min":0.33700000000000002},"max":{"1min":0.753,"5min":0.77400000000000002,"15min":0.79300000000000004},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57099999999999995,"15min":0.56499999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.75800000000000001,"5min":0.77600000000000002,"15min":0.82099999999999995},"last":0.60899999999999999}]},{"osd":4,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.46600000000000003,"15min":0.496},"min":{"1min":0.36599999999999999,"5min":0.30399999999999999,"15min":0.26400000000000001},"max":{"1min":0.66800000000000004,"5min":0.71499999999999997,"15min":0.755},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.51300000000000001,"15min":0.51800000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.45800000000000002}]},{"osd":5,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.503,"15min":0.52500000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.73499999999999999,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.54100000000000004,"15min":0.56000000000000005},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.76500000000000001,"5min":0.80600000000000005,"15min":0.82799999999999996},"last":0.47799999999999998}]},{"osd":6,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.41499999999999998,"5min":0.39600000000000002,"15min":0.33100000000000002},"max":{"1min":0.80400000000000005,"5min":0.84799999999999998,"15min":0.85399999999999998},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.58299999999999996,"15min":0.60399999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.27800000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.91500000000000004},"last":0.65800000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574993,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1056,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194255,"statfs":{"total":200000000,"available":198918036,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:22:15 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.52700000000000002,"15min":0.50700000000000001},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.69299999999999995,"5min":0.76300000000000001,"15min":1.012},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.53800000000000003,"15min":0.59299999999999997},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.29199999999999998},"max":{"1min":0.70099999999999996,"5min":0.77800000000000002,"15min":5.1230000000000002},"last":0.63200000000000001}]},{"osd":1,"last update":"Sun Oct 8 00:22:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56499999999999995,"15min":0.54400000000000004},"min":{"1min":0.39600000000000002,"5min":0.314,"15min":0.28899999999999998},"max":{"1min":0.72099999999999997,"5min":0.754,"15min":1.1419999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.59199999999999997,"15min":0.57499999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.32100000000000001},"max":{"1min":0.69099999999999995,"5min":0.81799999999999995,"15min":1.1539999999999999},"last":0.67900000000000005}]},{"osd":2,"last update":"Sun Oct 8 00:22:15 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.56100000000000005,"15min":0.54300000000000004},"min":{"1min":0.41599999999999998,"5min":0.36599999999999999,"15min":0.32900000000000001},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":1.3149999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59199999999999997,"15min":0.57699999999999996},"min":{"1min":0.44800000000000001,"5min":0.374,"15min":0.29399999999999998},"max":{"1min":0.75700000000000001,"5min":0.83699999999999997,"15min":1.3280000000000001},"last":0.72099999999999997}]},{"osd":3,"last update":"Sun Oct 8 00:22:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57099999999999995,"15min":0.55600000000000005},"min":{"1min":0.41399999999999998,"5min":0.32000000000000001,"15min":0.25700000000000001},"max":{"1min":0.76900000000000002,"5min":0.78500000000000003,"15min":1.046},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.58699999999999997,"15min":0.56299999999999994},"min":{"1min":0.39500000000000002,"5min":0.38600000000000001,"15min":0.32900000000000001},"max":{"1min":0.74399999999999999,"5min":0.81799999999999995,"15min":1.4390000000000001},"last":0.64800000000000002}]},{"osd":4,"last update":"Sun Oct 8 00:22:15 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.45300000000000001,"15min":0.47799999999999998},"min":{"1min":0.30499999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.60699999999999998,"5min":0.70099999999999996,"15min":0.80100000000000005},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54600000000000004,"15min":0.55600000000000005},"min":{"1min":0.40000000000000002,"5min":0.375,"15min":0.33800000000000002},"max":{"1min":0.71199999999999997,"5min":0.77600000000000002,"15min":0.89100000000000001},"last":0.70599999999999996}]},{"osd":5,"last update":"Sun Oct 8 00:22:15 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.47699999999999998,"15min":0.48299999999999998},"min":{"1min":0.308,"5min":0.29299999999999998,"15min":0.29099999999999998},"max":{"1min":0.66000000000000003,"5min":0.73499999999999999,"15min":0.80500000000000005},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.497,"15min":0.52800000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.28699999999999998},"max":{"1min":0.70299999999999996,"5min":0.71099999999999997,"15min":0.84299999999999997},"last":0.66400000000000003}]},{"osd":7,"last update":"Sun Oct 8 00:22:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.50900000000000001,"15min":0.52600000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.307},"max":{"1min":0.71699999999999997,"5min":0.754,"15min":0.91500000000000004},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.53700000000000003,"15min":0.54200000000000004},"min":{"1min":0.37,"5min":0.34899999999999998,"15min":0.32400000000000001},"max":{"1min":0.71999999999999997,"5min":0.76400000000000001,"15min":0.79900000000000004},"last":0.57699999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574993,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1633,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193679,"statfs":{"total":200000000,"available":198327668,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.35999999999999999,"15min":0.33300000000000002},"min":{"1min":0.223,"5min":0.216,"15min":0.19500000000000001},"max":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.41999999999999998,"15min":0.39500000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.67300000000000004},"last":0.437}]},{"osd":2,"last update":"Sun Oct 8 00:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.38200000000000001,"15min":0.35499999999999998},"min":{"1min":0.27400000000000002,"5min":0.25900000000000001,"15min":0.22600000000000001},"max":{"1min":0.56999999999999995,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39500000000000002,"15min":0.36899999999999999},"min":{"1min":0.28499999999999998,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.53400000000000003,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.38600000000000001}]},{"osd":3,"last update":"Sun Oct 8 00:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.42499999999999999,"15min":0.39300000000000002},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.58799999999999997,"5min":0.63,"15min":0.70999999999999996},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.375,"15min":0.35599999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.222},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.47899999999999998}]},{"osd":4,"last update":"Sun Oct 8 00:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57699999999999996,"15min":0.55600000000000005},"min":{"1min":0.42999999999999999,"5min":0.34999999999999998,"15min":0.32600000000000001},"max":{"1min":0.82199999999999995,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59899999999999998,"15min":0.57999999999999996},"min":{"1min":0.42199999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.76000000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.70299999999999996}]},{"osd":5,"last update":"Sun Oct 8 00:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57799999999999996,"15min":0.55900000000000005},"min":{"1min":0.39900000000000002,"5min":0.35099999999999998,"15min":0.33500000000000002},"max":{"1min":0.71799999999999997,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.755},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60299999999999998,"15min":0.58699999999999997},"min":{"1min":0.41799999999999998,"5min":0.39500000000000002,"15min":0.30399999999999999},"max":{"1min":0.76900000000000002,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.67000000000000004}]},{"osd":6,"last update":"Sun Oct 8 00:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.58299999999999996,"15min":0.57599999999999996},"min":{"1min":0.41499999999999998,"5min":0.37,"15min":0.317},"max":{"1min":0.755,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59299999999999997,"15min":0.57599999999999996},"min":{"1min":0.40799999999999997,"5min":0.377,"15min":0.33200000000000002},"max":{"1min":0.748,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.71499999999999997}]},{"osd":7,"last update":"Sun Oct 8 00:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58699999999999997,"15min":0.57599999999999996},"min":{"1min":0.439,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.746,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.623,"5min":0.63200000000000001,"15min":0.61099999999999999},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.39100000000000001},"max":{"1min":0.79400000000000004,"5min":0.872,"15min":0.96699999999999997},"last":0.77400000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574993,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1056,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194255,"statfs":{"total":200000000,"available":198918036,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 00:22:08 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.33900000000000002,"15min":0.33500000000000002},"min":{"1min":0.218,"5min":0.20399999999999999,"15min":0.193},"max":{"1min":0.501,"5min":0.54900000000000004,"15min":1.0640000000000001},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.36899999999999999,"15min":0.35499999999999998},"min":{"1min":0.23499999999999999,"5min":0.216,"15min":0.20300000000000001},"max":{"1min":0.48999999999999999,"5min":0.66500000000000004,"15min":0.87},"last":0.47999999999999998}]},{"osd":2,"last update":"Sun Oct 8 00:22:08 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38900000000000001,"15min":0.38700000000000001},"min":{"1min":0.25800000000000001,"5min":0.249,"15min":0.24099999999999999},"max":{"1min":0.51600000000000001,"5min":0.57199999999999995,"15min":0.93799999999999994},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.35899999999999999,"15min":0.52500000000000002},"min":{"1min":0.23599999999999999,"5min":0.23499999999999999,"15min":0.185},"max":{"1min":0.45700000000000002,"5min":0.59899999999999998,"15min":12.019},"last":0.46800000000000003}]},{"osd":3,"last update":"Sun Oct 8 00:22:08 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.39000000000000001,"15min":0.371},"min":{"1min":0.27300000000000002,"5min":0.25700000000000001,"15min":0.22500000000000001},"max":{"1min":0.46700000000000003,"5min":0.628,"15min":1.1020000000000001},"last":0.503},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.42099999999999999,"15min":0.40799999999999997},"min":{"1min":0.28499999999999998,"5min":0.26700000000000002,"15min":0.246},"max":{"1min":0.52000000000000002,"5min":0.66100000000000003,"15min":1.0409999999999999},"last":0.55200000000000005}]},{"osd":4,"last update":"Sun Oct 8 00:22:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.432,"5min":0.377,"15min":0.35899999999999999},"max":{"1min":0.72499999999999998,"5min":0.89700000000000002,"15min":1.1779999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.624,"15min":0.622},"min":{"1min":0.46899999999999997,"5min":0.41099999999999998,"15min":0.40300000000000002},"max":{"1min":0.78000000000000003,"5min":0.93899999999999995,"15min":0.96199999999999997},"last":0.752}]},{"osd":5,"last update":"Sun Oct 8 00:22:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58999999999999997,"15min":0.75900000000000001},"min":{"1min":0.44700000000000001,"5min":0.378,"15min":0.32000000000000001},"max":{"1min":0.83299999999999996,"5min":0.90700000000000003,"15min":11.992000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.60799999999999998,"15min":0.59599999999999997},"min":{"1min":0.443,"5min":0.379,"15min":0.379},"max":{"1min":0.77100000000000002,"5min":0.91600000000000004,"15min":1.1299999999999999},"last":0.63400000000000001}]},{"osd":6,"last update":"Sun Oct 8 00:22:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.60099999999999998,"15min":0.59299999999999997},"min":{"1min":0.442,"5min":0.372,"15min":0.372},"max":{"1min":0.73899999999999999,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.61299999999999999,"15min":0.76100000000000001},"min":{"1min":0.46899999999999997,"5min":0.39600000000000002,"15min":0.38100000000000001},"max":{"1min":0.85799999999999998,"5min":0.93999999999999995,"15min":11.927},"last":0.64900000000000002}]},{"osd":7,"last update":"Sun Oct 8 00:22:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.60099999999999998,"15min":0.76700000000000002},"min":{"1min":0.42199999999999999,"5min":0.38600000000000001,"15min":0.35699999999999998},"max":{"1min":0.76400000000000001,"5min":0.95099999999999996,"15min":11.965},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.63700000000000001,"15min":0.626},"min":{"1min":0.42399999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.79900000000000004,"5min":0.89800000000000002,"15min":1.1539999999999999},"last":0.79500000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574993,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1056,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194255,"statfs":{"total":200000000,"available":198918036,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.32900000000000001,"15min":0.35799999999999998},"min":{"1min":0.20100000000000001,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.47599999999999998,"5min":0.56899999999999995,"15min":2.5139999999999998},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.34699999999999998,"15min":0.36899999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.20599999999999999},"max":{"1min":0.51200000000000001,"5min":0.59099999999999997,"15min":2.1309999999999998},"last":0.41499999999999998}]},{"osd":1,"last update":"Sun Oct 8 00:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.35499999999999998,"15min":0.37},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":2.2639999999999998},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.374,"5min":0.38400000000000001,"15min":0.40300000000000002},"min":{"1min":0.254,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.54400000000000004,"5min":0.57199999999999995,"15min":2.5379999999999998},"last":0.54500000000000004}]},{"osd":3,"last update":"Sun Oct 8 00:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.40400000000000003,"15min":0.41099999999999998},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.64400000000000002,"5min":0.66900000000000004,"15min":2.2890000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.41899999999999998,"15min":0.435},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.55900000000000005,"5min":0.64300000000000002,"15min":2.5619999999999998},"last":0.57999999999999996}]},{"osd":4,"last update":"Sun Oct 8 00:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54400000000000004,"15min":0.57999999999999996},"min":{"1min":0.31,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.77300000000000002,"5min":0.80800000000000005,"15min":2.488},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58599999999999997,"15min":0.59599999999999997},"min":{"1min":0.42999999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.77300000000000002,"5min":0.92400000000000004,"15min":2.073},"last":0.73399999999999999}]},{"osd":5,"last update":"Sun Oct 8 00:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.60899999999999999,"15min":0.63},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.76300000000000001,"5min":0.94299999999999995,"15min":2.2069999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.61699999999999999,"15min":0.64000000000000001},"min":{"1min":0.42199999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77300000000000002,"5min":0.91000000000000003,"15min":2.456},"last":0.63100000000000001}]},{"osd":6,"last update":"Sun Oct 8 00:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59599999999999997,"15min":0.60699999999999998},"min":{"1min":0.39100000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.80200000000000005,"5min":0.872,"15min":2.0339999999999998},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59399999999999997,"15min":0.60499999999999998},"min":{"1min":0.40799999999999997,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.75,"5min":0.94099999999999995,"15min":2.1699999999999999},"last":0.79900000000000004}]},{"osd":7,"last update":"Sun Oct 8 00:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60099999999999998,"15min":0.61899999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82199999999999995,"5min":0.91500000000000004,"15min":2.1030000000000002},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60699999999999998,"15min":0.61799999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.77700000000000002,"5min":1.0169999999999999,"15min":2.2349999999999999},"last":0.70899999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574993,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1056,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194255,"statfs":{"total":200000000,"available":198918036,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.33600000000000002,"15min":0.34699999999999998},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.192},"max":{"1min":0.58899999999999997,"5min":0.59299999999999997,"15min":1.46},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.31900000000000001,"5min":0.34999999999999998,"15min":0.34200000000000003},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.192},"max":{"1min":0.56000000000000005,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.32700000000000001}]},{"osd":1,"last update":"Sun Oct 8 00:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.36299999999999999,"15min":0.55500000000000005},"min":{"1min":0.25,"5min":0.247,"15min":0.20000000000000001},"max":{"1min":0.52700000000000002,"5min":0.57499999999999996,"15min":12.351000000000001},"last":0.318},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38700000000000001,"15min":0.372},"min":{"1min":0.23200000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.54500000000000004,"5min":0.58199999999999996,"15min":0.71999999999999997},"last":0.34499999999999997}]},{"osd":2,"last update":"Sun Oct 8 00:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.40799999999999997,"15min":0.40600000000000003},"min":{"1min":0.248,"5min":0.248,"15min":0.23499999999999999},"max":{"1min":0.58099999999999996,"5min":0.59699999999999998,"15min":1.5489999999999999},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.41999999999999998,"15min":0.39800000000000002},"min":{"1min":0.27100000000000002,"5min":0.25700000000000001,"15min":0.246},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.753},"last":0.39400000000000002}]},{"osd":4,"last update":"Sun Oct 8 00:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.625,"15min":0.78400000000000003},"min":{"1min":0.39800000000000002,"5min":0.36899999999999999,"15min":0.34899999999999998},"max":{"1min":0.81399999999999995,"5min":0.96699999999999997,"15min":11.333},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.626,"5min":0.63400000000000001,"15min":0.80400000000000005},"min":{"1min":0.39100000000000001,"5min":0.38100000000000001,"15min":0.36099999999999999},"max":{"1min":0.89500000000000002,"5min":0.97699999999999998,"15min":12.321999999999999},"last":0.72199999999999998}]},{"osd":5,"last update":"Sun Oct 8 00:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.61199999999999999,"15min":0.79800000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35499999999999998},"max":{"1min":0.86299999999999999,"5min":0.93500000000000005,"15min":12.291},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.64400000000000002,"15min":0.621},"min":{"1min":0.44800000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80500000000000005,"5min":1.143,"15min":1.143},"last":0.49399999999999999}]},{"osd":6,"last update":"Sun Oct 8 00:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.59899999999999998,"15min":0.755},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.33700000000000002},"max":{"1min":0.91200000000000003,"5min":0.91500000000000004,"15min":11.288},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.621,"5min":0.62,"15min":0.79100000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.36599999999999999},"max":{"1min":0.84599999999999997,"5min":0.94599999999999995,"15min":12.253},"last":0.60099999999999998}]},{"osd":7,"last update":"Sun Oct 8 00:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63200000000000001,"15min":0.60299999999999998},"min":{"1min":0.45700000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86399999999999999,"5min":1.0920000000000001,"15min":1.3109999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61599999999999999,"15min":0.78400000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.38600000000000001},"max":{"1min":0.879,"5min":0.89300000000000002,"15min":11.363},"last":0.626}]}]},{"osd":4,"up_from":13,"seq":55834574993,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1056,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194255,"statfs":{"total":200000000,"available":198918036,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.56699999999999995,"15min":0.56100000000000005},"min":{"1min":0.36299999999999999,"5min":0.32300000000000001,"15min":0.193},"max":{"1min":0.80900000000000005,"5min":0.84799999999999998,"15min":0.93000000000000005},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.60099999999999998,"15min":0.58699999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.30399999999999999},"max":{"1min":0.70199999999999996,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.54600000000000004}]},{"osd":1,"last update":"Sun Oct 8 00:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.54300000000000004,"15min":0.52900000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.33700000000000002},"max":{"1min":0.749,"5min":0.751,"15min":0.78100000000000003},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57299999999999995,"15min":0.54600000000000004},"min":{"1min":0.46899999999999997,"5min":0.34300000000000003,"15min":0.30199999999999999},"max":{"1min":0.71799999999999997,"5min":0.80600000000000005,"15min":0.81299999999999994},"last":0.38300000000000001}]},{"osd":2,"last update":"Sun Oct 8 00:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.54800000000000004,"15min":0.52700000000000002},"min":{"1min":0.35999999999999999,"5min":0.32500000000000001,"15min":0.32100000000000001},"max":{"1min":0.78400000000000003,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.56499999999999995,"15min":0.53400000000000003},"min":{"1min":0.41599999999999998,"5min":0.35399999999999998,"15min":0.32600000000000001},"max":{"1min":0.65300000000000002,"5min":0.79600000000000004,"15min":0.81699999999999995},"last":0.45300000000000001}]},{"osd":3,"last update":"Sun Oct 8 00:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.57999999999999996,"15min":0.55000000000000004},"min":{"1min":0.41099999999999998,"5min":0.35199999999999998,"15min":0.28799999999999998},"max":{"1min":0.77100000000000002,"5min":0.82299999999999995,"15min":0.85399999999999998},"last":0.439},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.55200000000000005,"15min":0.54000000000000004},"min":{"1min":0.433,"5min":0.37,"15min":0.36199999999999999},"max":{"1min":0.68400000000000005,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.63100000000000001}]},{"osd":5,"last update":"Sun Oct 8 00:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.47299999999999998,"15min":0.47899999999999998},"min":{"1min":0.33900000000000002,"5min":0.314,"15min":0.27500000000000002},"max":{"1min":0.68600000000000005,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.501,"15min":0.504},"min":{"1min":0.34100000000000003,"5min":0.317,"15min":0.24399999999999999},"max":{"1min":0.72399999999999998,"5min":0.79100000000000004,"15min":0.94899999999999995},"last":0.47599999999999998}]},{"osd":6,"last update":"Sun Oct 8 00:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.48899999999999999,"15min":0.48599999999999999},"min":{"1min":0.371,"5min":0.32400000000000001,"15min":0.26400000000000001},"max":{"1min":0.70899999999999996,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52500000000000002,"15min":0.53700000000000003},"min":{"1min":0.36299999999999999,"5min":0.34799999999999998,"15min":0.29799999999999999},"max":{"1min":0.73699999999999999,"5min":0.76200000000000001,"15min":0.80100000000000005},"last":0.58299999999999996}]},{"osd":7,"last update":"Sun Oct 8 00:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.53900000000000003,"15min":0.54300000000000004},"min":{"1min":0.34999999999999998,"5min":0.33200000000000002,"15min":0.27900000000000003},"max":{"1min":0.76200000000000001,"5min":0.83599999999999997,"15min":0.90100000000000002},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56999999999999995,"15min":0.57399999999999995},"min":{"1min":0.42999999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.751,"5min":0.88,"15min":0.88},"last":0.65600000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574993,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1633,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193679,"statfs":{"total":200000000,"available":198327668,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.54400000000000004,"15min":0.50600000000000001},"min":{"1min":0.38,"5min":0.36399999999999999,"15min":0.249},"max":{"1min":0.71599999999999997,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54100000000000004,"15min":0.54800000000000004},"min":{"1min":0.39000000000000001,"5min":0.29799999999999999,"15min":0.29399999999999998},"max":{"1min":0.75700000000000001,"5min":0.81699999999999995,"15min":0.88600000000000001},"last":0.59499999999999997}]},{"osd":1,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.55600000000000005,"15min":0.53600000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.32500000000000001},"max":{"1min":0.68300000000000005,"5min":0.72599999999999998,"15min":0.82999999999999996},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.53800000000000003,"15min":0.52500000000000002},"min":{"1min":0.34599999999999997,"5min":0.34499999999999997,"15min":0.33000000000000002},"max":{"1min":0.68700000000000006,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.66000000000000003}]},{"osd":2,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55700000000000005,"15min":0.53700000000000003},"min":{"1min":0.42999999999999999,"5min":0.39600000000000002,"15min":0.32400000000000001},"max":{"1min":0.71599999999999997,"5min":0.749,"15min":0.86199999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56100000000000005,"15min":0.56000000000000005},"min":{"1min":0.38100000000000001,"5min":0.33400000000000002,"15min":0.29599999999999999},"max":{"1min":0.749,"5min":0.80000000000000004,"15min":0.95399999999999996},"last":0.64800000000000002}]},{"osd":3,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57299999999999995,"15min":0.55700000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.316},"max":{"1min":0.71999999999999997,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56599999999999995,"15min":0.56899999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.72399999999999998,"5min":0.73499999999999999,"15min":1.0529999999999999},"last":0.624}]},{"osd":4,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.47299999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.27700000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.504,"5min":0.51400000000000001,"15min":0.51700000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.28899999999999998},"max":{"1min":0.747,"5min":0.77500000000000002,"15min":0.91500000000000004},"last":0.67600000000000005}]},{"osd":6,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.33600000000000002,"5min":0.28199999999999997,"15min":0.27400000000000002},"max":{"1min":0.79400000000000004,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52300000000000002,"15min":0.51100000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.30499999999999999},"max":{"1min":0.81499999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.54200000000000004}]},{"osd":7,"last update":"Sun Oct 8 00:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.54200000000000004,"15min":0.53100000000000003},"min":{"1min":0.33500000000000002,"5min":0.318,"15min":0.29399999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81999999999999995},"last":0.504},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56899999999999995,"15min":0.56200000000000006},"min":{"1min":0.38300000000000001,"5min":0.35599999999999998,"15min":0.32500000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.57499999999999996}]}]}],"pool_statfs":[]}} 2023-10-08T00:22:40.019 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-10-08T00:22:40.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.133+0000 7f0fd759e700 1 Processor -- start 2023-10-08T00:22:40.134 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.133+0000 7f0fd759e700 1 -- start start 2023-10-08T00:22:40.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.134+0000 7f0fd759e700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0fd80b6580 0x7f0fd806a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:40.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.134+0000 7f0fd759e700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fd806ac90 con 0x7f0fd806a830 2023-10-08T00:22:40.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.134+0000 7f0fd759e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fd80b4d70 con 0x7f0fd80cce30 2023-10-08T00:22:40.136 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.134+0000 7f0fd759e700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fd815b3b0 con 0x7f0fd80b6580 2023-10-08T00:22:40.137 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.134+0000 7f0fcdd9b700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0fd80b6580 0x7f0fd806a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:40.137 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.134+0000 7f0fd6d9d700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f0fd806a830 0x7f0fd80b5200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59662/0 (socket says 172.21.15.99:59662) 2023-10-08T00:22:40.137 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.134+0000 7f0fd6d9d700 1 -- 172.21.15.99:0/1413802603 learned_addr learned my addr 172.21.15.99:0/1413802603 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:22:40.137 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.134+0000 7f0fcdd9b700 1 -- 172.21.15.99:0/1413802603 >> v1:172.21.15.191:6789/0 conn(0x7f0fd80cce30 legacy=0x7f0fd815aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:40.137 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.134+0000 7f0fcdd9b700 1 -- 172.21.15.99:0/1413802603 >> v1:172.21.15.99:6789/0 conn(0x7f0fd806a830 legacy=0x7f0fd80b5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:40.137 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.134+0000 7f0fcdd9b700 1 -- 172.21.15.99:0/1413802603 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd80cd8d0 con 0x7f0fd80b6580 2023-10-08T00:22:40.137 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.134+0000 7f0fcdd9b700 1 --2- 172.21.15.99:0/1413802603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fd80b6580 0x7f0fd806a2f0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f0fc00098e0 tx=0x7f0fc00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb99dfa1f7ed5873 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:40.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.134+0000 7f0fd5d9b700 1 -- 172.21.15.99:0/1413802603 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc0009e80 con 0x7f0fd80b6580 2023-10-08T00:22:40.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.135+0000 7f0fd5d9b700 1 -- 172.21.15.99:0/1413802603 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc000b820 con 0x7f0fd80b6580 2023-10-08T00:22:40.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.135+0000 7f0fd5d9b700 1 -- 172.21.15.99:0/1413802603 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc000bba0 con 0x7f0fd80b6580 2023-10-08T00:22:40.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.135+0000 7f0fd759e700 1 -- 172.21.15.99:0/1413802603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fd80b6580 msgr2=0x7f0fd806a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:40.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.135+0000 7f0fd759e700 1 --2- 172.21.15.99:0/1413802603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fd80b6580 0x7f0fd806a2f0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f0fc00098e0 tx=0x7f0fc00091a0 comp rx=0 tx=0).stop 2023-10-08T00:22:40.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.135+0000 7f0fd759e700 1 -- 172.21.15.99:0/1413802603 shutdown_connections 2023-10-08T00:22:40.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.135+0000 7f0fd759e700 1 --2- 172.21.15.99:0/1413802603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fd80b6580 0x7f0fd806a2f0 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:40.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.135+0000 7f0fd759e700 1 -- 172.21.15.99:0/1413802603 >> 172.21.15.99:0/1413802603 conn(0x7f0fd8068cb0 msgr2=0x7f0fd80690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:40.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.135+0000 7f0fd759e700 1 -- 172.21.15.99:0/1413802603 shutdown_connections 2023-10-08T00:22:40.139 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.135+0000 7f0fd759e700 1 -- 172.21.15.99:0/1413802603 wait complete. 2023-10-08T00:22:40.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.136+0000 7f0fd759e700 1 Processor -- start 2023-10-08T00:22:40.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.136+0000 7f0fd759e700 1 -- start start 2023-10-08T00:22:40.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.137+0000 7f0fd759e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0fd806a830 0x7f0fd814f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:40.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.137+0000 7f0fd759e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0fd80b6580 0x7f0fd816a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:40.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.137+0000 7f0fd759e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fd80cce30 0x7f0fd8152060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:40.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.137+0000 7f0fd759e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd80cc720 con 0x7f0fd80cce30 2023-10-08T00:22:40.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.137+0000 7f0fd759e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd816b400 con 0x7f0fd80b6580 2023-10-08T00:22:40.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.137+0000 7f0fd759e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd816a7d0 con 0x7f0fd806a830 2023-10-08T00:22:40.144 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.137+0000 7f0fd6d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fd80cce30 0x7f0fd8152060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:40.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.137+0000 7f0fd6d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fd80cce30 0x7f0fd8152060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41406/0 (socket says 172.21.15.99:41406) 2023-10-08T00:22:40.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.137+0000 7f0fd6d9d700 1 -- 172.21.15.99:0/3970226988 learned_addr learned my addr 172.21.15.99:0/3970226988 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:40.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.137+0000 7f0fcdd9b700 1 --2- 172.21.15.99:0/3970226988 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0fd80b6580 0x7f0fd816a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:40.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.137+0000 7f0fd659c700 1 --2- 172.21.15.99:0/3970226988 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0fd806a830 0x7f0fd814f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:40.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.137+0000 7f0fd6d9d700 1 -- 172.21.15.99:0/3970226988 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0fd806a830 msgr2=0x7f0fd814f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:40.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.137+0000 7f0fd6d9d700 1 --2- 172.21.15.99:0/3970226988 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0fd806a830 0x7f0fd814f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:40.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.137+0000 7f0fd6d9d700 1 -- 172.21.15.99:0/3970226988 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0fd80b6580 msgr2=0x7f0fd816a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:40.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.137+0000 7f0fd6d9d700 1 --2- 172.21.15.99:0/3970226988 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0fd80b6580 0x7f0fd816a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:40.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.137+0000 7f0fd6d9d700 1 -- 172.21.15.99:0/3970226988 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc0009420 con 0x7f0fd80cce30 2023-10-08T00:22:40.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.138+0000 7f0fd6d9d700 1 --2- 172.21.15.99:0/3970226988 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fd80cce30 0x7f0fd8152060 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f0fd000cc40 tx=0x7f0fd0001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:40.146 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.138+0000 7f0fceffd700 1 -- 172.21.15.99:0/3970226988 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd000a9c0 con 0x7f0fd80cce30 2023-10-08T00:22:40.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.138+0000 7f0fceffd700 1 -- 172.21.15.99:0/3970226988 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fd000ab20 con 0x7f0fd80cce30 2023-10-08T00:22:40.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.138+0000 7f0fd759e700 1 -- 172.21.15.99:0/3970226988 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fd816aab0 con 0x7f0fd80cce30 2023-10-08T00:22:40.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.138+0000 7f0fceffd700 1 -- 172.21.15.99:0/3970226988 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd00070f0 con 0x7f0fd80cce30 2023-10-08T00:22:40.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.138+0000 7f0fd759e700 1 -- 172.21.15.99:0/3970226988 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fd816b070 con 0x7f0fd80cce30 2023-10-08T00:22:40.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.139+0000 7f0fd759e700 1 -- 172.21.15.99:0/3970226988 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fd80ce290 con 0x7f0fd80cce30 2023-10-08T00:22:40.147 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.139+0000 7f0fceffd700 1 -- 172.21.15.99:0/3970226988 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0fd00074a0 con 0x7f0fd80cce30 2023-10-08T00:22:40.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.139+0000 7f0fceffd700 1 --2- 172.21.15.99:0/3970226988 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0fb8066f30 0x7f0fb80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:40.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.139+0000 7f0fceffd700 1 -- 172.21.15.99:0/3970226988 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f0fd0084480 con 0x7f0fd80cce30 2023-10-08T00:22:40.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.140+0000 7f0fd659c700 1 --2- 172.21.15.99:0/3970226988 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0fb8066f30 0x7f0fb80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:40.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.140+0000 7f0fd659c700 1 --2- 172.21.15.99:0/3970226988 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0fb8066f30 0x7f0fb80693e0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f0fc8006fd0 tx=0x7f0fc8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:40.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.143+0000 7f0fceffd700 1 -- 172.21.15.99:0/3970226988 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0fd0051a70 con 0x7f0fd80cce30 2023-10-08T00:22:40.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.324+0000 7f0fd759e700 1 -- 172.21.15.99:0/3970226988 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f0fd806ac90 con 0x7f0fd80cce30 2023-10-08T00:22:40.330 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.330+0000 7f0fceffd700 1 -- 172.21.15.99:0/3970226988 <== mon.0 v2:172.21.15.99:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f0fd0001270 con 0x7f0fd80cce30 2023-10-08T00:22:40.331 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.331+0000 7f0fceffd700 1 -- 172.21.15.99:0/3970226988 <== mon.0 v2:172.21.15.99:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v22)=0 v22) v1 ==== 123+0+0 (secure 0 0 0) 0x7f0fd806ac90 con 0x7f0fd80cce30 2023-10-08T00:22:40.334 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.333+0000 7f0fd759e700 1 -- 172.21.15.99:0/3970226988 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0fb8066f30 msgr2=0x7f0fb80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:40.334 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.333+0000 7f0fd759e700 1 --2- 172.21.15.99:0/3970226988 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0fb8066f30 0x7f0fb80693e0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f0fc8006fd0 tx=0x7f0fc8008040 comp rx=0 tx=0).stop 2023-10-08T00:22:40.334 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.333+0000 7f0fd759e700 1 -- 172.21.15.99:0/3970226988 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fd80cce30 msgr2=0x7f0fd8152060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:40.334 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.333+0000 7f0fd759e700 1 --2- 172.21.15.99:0/3970226988 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fd80cce30 0x7f0fd8152060 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f0fd000cc40 tx=0x7f0fd0001a50 comp rx=0 tx=0).stop 2023-10-08T00:22:40.334 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.333+0000 7f0fd759e700 1 -- 172.21.15.99:0/3970226988 shutdown_connections 2023-10-08T00:22:40.334 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.333+0000 7f0fd759e700 1 --2- 172.21.15.99:0/3970226988 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0fb8066f30 0x7f0fb80693e0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:40.335 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.333+0000 7f0fd759e700 1 --2- 172.21.15.99:0/3970226988 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fd80cce30 0x7f0fd8152060 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:40.335 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.333+0000 7f0fd759e700 1 --2- 172.21.15.99:0/3970226988 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0fd806a830 0x7f0fd814f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:40.335 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.333+0000 7f0fd759e700 1 --2- 172.21.15.99:0/3970226988 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0fd80b6580 0x7f0fd816a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:40.335 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.333+0000 7f0fd759e700 1 -- 172.21.15.99:0/3970226988 >> 172.21.15.99:0/3970226988 conn(0x7f0fd8068cb0 msgr2=0x7f0fd8157e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:40.335 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.333+0000 7f0fd759e700 1 -- 172.21.15.99:0/3970226988 shutdown_connections 2023-10-08T00:22:40.335 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.333+0000 7f0fd759e700 1 -- 172.21.15.99:0/3970226988 wait complete. 2023-10-08T00:22:40.345 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T00:22:40.345 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:22:40.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.458+0000 7f0ef2e13700 1 Processor -- start 2023-10-08T00:22:40.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.458+0000 7f0ef2e13700 1 -- start start 2023-10-08T00:22:40.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.459+0000 7f0ef2e13700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0eec0aa870 0x7f0eec0cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:40.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.459+0000 7f0ef2e13700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f0eec15a310 0x7f0eec0afd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:40.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.459+0000 7f0ef2e13700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0eec0ac650 con 0x7f0eec0aac40 2023-10-08T00:22:40.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.459+0000 7f0ef2e13700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0eec0ad4b0 con 0x7f0eec0aa870 2023-10-08T00:22:40.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.459+0000 7f0ef2e13700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0eec0ac8a0 con 0x7f0eec15a310 2023-10-08T00:22:40.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.459+0000 7f0ef0baf700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0eec0aa870 0x7f0eec0cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:40.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.459+0000 7f0ef13b0700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f0eec0aac40 0x7f0eec0ab0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59678/0 (socket says 172.21.15.99:59678) 2023-10-08T00:22:40.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.459+0000 7f0ef13b0700 1 -- 172.21.15.99:0/3647146801 learned_addr learned my addr 172.21.15.99:0/3647146801 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-08T00:22:40.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.460+0000 7f0ee3fff700 1 --2- 172.21.15.99:0/3647146801 >> v2:172.21.15.191:3300/0 conn(0x7f0eec15a310 0x7f0eec0afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:40.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.460+0000 7f0ee37fe700 1 -- 172.21.15.99:0/3647146801 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2172318553 0 0) 0x7f0eec0ac650 con 0x7f0eec0aac40 2023-10-08T00:22:40.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.460+0000 7f0ee37fe700 1 -- 172.21.15.99:0/3647146801 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ed4003570 con 0x7f0eec0aac40 2023-10-08T00:22:40.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.460+0000 7f0ef0baf700 1 -- 172.21.15.99:0/3647146801 >> v2:172.21.15.191:3300/0 conn(0x7f0eec15a310 msgr2=0x7f0eec0afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:22:40.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.460+0000 7f0ef0baf700 1 --2- 172.21.15.99:0/3647146801 >> v2:172.21.15.191:3300/0 conn(0x7f0eec15a310 0x7f0eec0afd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:40.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.460+0000 7f0ef0baf700 1 -- 172.21.15.99:0/3647146801 >> v1:172.21.15.99:6789/0 conn(0x7f0eec0aac40 legacy=0x7f0eec0ab0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:40.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.460+0000 7f0ef0baf700 1 -- 172.21.15.99:0/3647146801 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eec0acb80 con 0x7f0eec0aa870 2023-10-08T00:22:40.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.460+0000 7f0ef0baf700 1 --2- 172.21.15.99:0/3647146801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0eec0aa870 0x7f0eec0cd4b0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f0eec063e40 tx=0x7f0ee8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cd185f52c1ecfaf server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:40.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.460+0000 7f0ee37fe700 1 -- 172.21.15.99:0/3647146801 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee800e070 con 0x7f0eec0aa870 2023-10-08T00:22:40.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.460+0000 7f0ee37fe700 1 -- 172.21.15.99:0/3647146801 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f0ee800bbf0 con 0x7f0eec0aa870 2023-10-08T00:22:40.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.460+0000 7f0ee37fe700 1 -- 172.21.15.99:0/3647146801 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee8012430 con 0x7f0eec0aa870 2023-10-08T00:22:40.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.460+0000 7f0ef2e13700 1 -- 172.21.15.99:0/3647146801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0eec0aa870 msgr2=0x7f0eec0cd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:40.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.460+0000 7f0ef2e13700 1 --2- 172.21.15.99:0/3647146801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0eec0aa870 0x7f0eec0cd4b0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f0eec063e40 tx=0x7f0ee8009780 comp rx=0 tx=0).stop 2023-10-08T00:22:40.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.460+0000 7f0ef2e13700 1 -- 172.21.15.99:0/3647146801 shutdown_connections 2023-10-08T00:22:40.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.460+0000 7f0ef2e13700 1 --2- 172.21.15.99:0/3647146801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0eec0aa870 0x7f0eec0cd4b0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:40.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.460+0000 7f0ef2e13700 1 --2- 172.21.15.99:0/3647146801 >> v2:172.21.15.191:3300/0 conn(0x7f0eec15a310 0x7f0eec0afd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:40.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.460+0000 7f0ef2e13700 1 -- 172.21.15.99:0/3647146801 >> 172.21.15.99:0/3647146801 conn(0x7f0eec0680f0 msgr2=0x7f0eec0ccda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:40.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.460+0000 7f0ef2e13700 1 -- 172.21.15.99:0/3647146801 shutdown_connections 2023-10-08T00:22:40.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.460+0000 7f0ef2e13700 1 -- 172.21.15.99:0/3647146801 wait complete. 2023-10-08T00:22:40.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.461+0000 7f0ef2e13700 1 Processor -- start 2023-10-08T00:22:40.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.462+0000 7f0ef2e13700 1 -- start start 2023-10-08T00:22:40.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.462+0000 7f0ef2e13700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0eec0aa870 0x7f0eec16b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:40.469 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.462+0000 7f0ef2e13700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0eec0aac40 0x7f0eec16d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:40.472 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.462+0000 7f0ef2e13700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0eec15a310 0x7f0eec15ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:40.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.462+0000 7f0ef2e13700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eec16b570 con 0x7f0eec0aa870 2023-10-08T00:22:40.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.462+0000 7f0ef2e13700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eec0c1f00 con 0x7f0eec0aac40 2023-10-08T00:22:40.473 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.462+0000 7f0ef2e13700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eec0c21e0 con 0x7f0eec15a310 2023-10-08T00:22:40.474 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.463+0000 7f0ef0baf700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0eec0aa870 0x7f0eec16b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:40.474 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.463+0000 7f0ef0baf700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0eec0aa870 0x7f0eec16b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41432/0 (socket says 172.21.15.99:41432) 2023-10-08T00:22:40.474 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.463+0000 7f0ef0baf700 1 -- 172.21.15.99:0/596416595 learned_addr learned my addr 172.21.15.99:0/596416595 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T00:22:40.474 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.463+0000 7f0ee3fff700 1 --2- 172.21.15.99:0/596416595 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0eec0aac40 0x7f0eec16d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:40.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.463+0000 7f0ef13b0700 1 --2- 172.21.15.99:0/596416595 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0eec15a310 0x7f0eec15ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:40.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.463+0000 7f0ef0baf700 1 -- 172.21.15.99:0/596416595 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0eec15a310 msgr2=0x7f0eec15ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:40.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.463+0000 7f0ef0baf700 1 --2- 172.21.15.99:0/596416595 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0eec15a310 0x7f0eec15ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:40.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.463+0000 7f0ef0baf700 1 -- 172.21.15.99:0/596416595 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0eec0aac40 msgr2=0x7f0eec16d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:40.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.463+0000 7f0ef0baf700 1 --2- 172.21.15.99:0/596416595 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0eec0aac40 0x7f0eec16d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:40.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.463+0000 7f0ef0baf700 1 -- 172.21.15.99:0/596416595 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee8009a20 con 0x7f0eec0aa870 2023-10-08T00:22:40.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.463+0000 7f0ef0baf700 1 --2- 172.21.15.99:0/596416595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0eec0aa870 0x7f0eec16b030 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f0eec068c40 tx=0x7f0ee8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:40.475 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.463+0000 7f0ee0ff9700 1 -- 172.21.15.99:0/596416595 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee8012e40 con 0x7f0eec0aa870 2023-10-08T00:22:40.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.463+0000 7f0ee0ff9700 1 -- 172.21.15.99:0/596416595 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f0ee801d630 con 0x7f0eec0aa870 2023-10-08T00:22:40.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.463+0000 7f0ee0ff9700 1 -- 172.21.15.99:0/596416595 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee801da10 con 0x7f0eec0aa870 2023-10-08T00:22:40.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.463+0000 7f0ef2e13700 1 -- 172.21.15.99:0/596416595 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eec0c24c0 con 0x7f0eec0aa870 2023-10-08T00:22:40.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.463+0000 7f0ef2e13700 1 -- 172.21.15.99:0/596416595 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eec0c2a80 con 0x7f0eec0aa870 2023-10-08T00:22:40.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.464+0000 7f0ef2e13700 1 -- 172.21.15.99:0/596416595 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eec04c3c0 con 0x7f0eec0aa870 2023-10-08T00:22:40.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.465+0000 7f0ee0ff9700 1 -- 172.21.15.99:0/596416595 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0ee8024070 con 0x7f0eec0aa870 2023-10-08T00:22:40.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.465+0000 7f0ee0ff9700 1 --2- 172.21.15.99:0/596416595 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0ed8066f30 0x7f0ed80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:22:40.476 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.465+0000 7f0ee0ff9700 1 -- 172.21.15.99:0/596416595 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f0ee8086950 con 0x7f0eec0aa870 2023-10-08T00:22:40.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.465+0000 7f0ee3fff700 1 --2- 172.21.15.99:0/596416595 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0ed8066f30 0x7f0ed80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:22:40.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.466+0000 7f0ee3fff700 1 --2- 172.21.15.99:0/596416595 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0ed8066f30 0x7f0ed80693e0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f0edc006fd0 tx=0x7f0edc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:22:40.477 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.469+0000 7f0ee0ff9700 1 -- 172.21.15.99:0/596416595 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0ee8053f40 con 0x7f0eec0aa870 2023-10-08T00:22:40.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.694+0000 7f0ef2e13700 1 -- 172.21.15.99:0/596416595 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0eec064900 con 0x7f0eec0aa870 2023-10-08T00:22:40.695 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.695+0000 7f0ee0ff9700 1 -- 172.21.15.99:0/596416595 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f0ee802cde0 con 0x7f0eec0aa870 2023-10-08T00:22:40.695 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T00:22:40.696 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":86,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/3862684847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3862684847},{"type":"v1","addr":"172.21.15.191:6836","nonce":3862684847}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":7525,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/4137947128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":4137947128},{"type":"v1","addr":"172.21.15.191:6837","nonce":4137947128}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":8106,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/359623206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":359623206},{"type":"v1","addr":"172.21.15.99:6834","nonce":359623206}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":8109,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3064101777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":3064101777},{"type":"v1","addr":"172.21.15.99:6835","nonce":3064101777}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80}],"filesystems":[]} 2023-10-08T00:22:40.697 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.697+0000 7f0ef2e13700 1 -- 172.21.15.99:0/596416595 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0ed8066f30 msgr2=0x7f0ed80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:40.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.697+0000 7f0ef2e13700 1 --2- 172.21.15.99:0/596416595 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0ed8066f30 0x7f0ed80693e0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f0edc006fd0 tx=0x7f0edc008040 comp rx=0 tx=0).stop 2023-10-08T00:22:40.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.697+0000 7f0ef2e13700 1 -- 172.21.15.99:0/596416595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0eec0aa870 msgr2=0x7f0eec16b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:22:40.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.697+0000 7f0ef2e13700 1 --2- 172.21.15.99:0/596416595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0eec0aa870 0x7f0eec16b030 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f0eec068c40 tx=0x7f0ee8009780 comp rx=0 tx=0).stop 2023-10-08T00:22:40.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.697+0000 7f0ef2e13700 1 -- 172.21.15.99:0/596416595 shutdown_connections 2023-10-08T00:22:40.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.697+0000 7f0ef2e13700 1 --2- 172.21.15.99:0/596416595 >> [v2:172.21.15.191:6832/85985,v1:172.21.15.191:6833/85985] conn(0x7f0ed8066f30 0x7f0ed80693e0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:40.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.697+0000 7f0ef2e13700 1 --2- 172.21.15.99:0/596416595 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0eec15a310 0x7f0eec15ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:40.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.697+0000 7f0ef2e13700 1 --2- 172.21.15.99:0/596416595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0eec0aa870 0x7f0eec16b030 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:40.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.697+0000 7f0ef2e13700 1 --2- 172.21.15.99:0/596416595 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0eec0aac40 0x7f0eec16d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:22:40.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.697+0000 7f0ef2e13700 1 -- 172.21.15.99:0/596416595 >> 172.21.15.99:0/596416595 conn(0x7f0eec0680f0 msgr2=0x7f0eec159610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:22:40.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.697+0000 7f0ef2e13700 1 -- 172.21.15.99:0/596416595 shutdown_connections 2023-10-08T00:22:40.698 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T00:22:40.697+0000 7f0ef2e13700 1 -- 172.21.15.99:0/596416595 wait complete. 2023-10-08T00:22:40.699 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 86 2023-10-08T00:22:40.709 DEBUG:tasks.cephfs.filesystem:already dead... 2023-10-08T00:22:40.709 INFO:teuthology.misc:Shutting down mds daemons... 2023-10-08T00:22:40.710 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T00:22:40.710 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:22:40.718 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T00:22:40.718 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T00:22:40.718 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:22:40.767 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T00:22:40.767 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T00:22:40.767 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:22:40.777 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T00:22:40.778 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T00:22:40.778 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:22:40.828 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T00:22:40.828 INFO:teuthology.misc:Shutting down osd daemons... 2023-10-08T00:22:40.829 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-08T00:22:40.829 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:22:40.847 INFO:tasks.ceph.osd.0:Stopped 2023-10-08T00:22:40.847 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-08T00:22:40.847 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:22:40.906 INFO:tasks.ceph.osd.1:Stopped 2023-10-08T00:22:40.906 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-08T00:22:40.906 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:22:40.967 INFO:tasks.ceph.osd.2:Stopped 2023-10-08T00:22:40.967 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-10-08T00:22:40.968 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:22:41.027 INFO:tasks.ceph.osd.3:Stopped 2023-10-08T00:22:41.027 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-10-08T00:22:41.027 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:22:41.047 INFO:tasks.ceph.osd.4:Stopped 2023-10-08T00:22:41.048 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-10-08T00:22:41.048 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:22:41.114 INFO:tasks.ceph.osd.5:Stopped 2023-10-08T00:22:41.114 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-10-08T00:22:41.114 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:22:41.180 INFO:tasks.ceph.osd.6:Stopped 2023-10-08T00:22:41.181 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-10-08T00:22:41.181 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:22:41.244 INFO:tasks.ceph.osd.7:Stopped 2023-10-08T00:22:41.244 INFO:teuthology.misc:Shutting down mgr daemons... 2023-10-08T00:22:41.245 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-10-08T00:22:41.246 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:22:41.267 INFO:tasks.ceph.mgr.y:Stopped 2023-10-08T00:22:41.267 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-08T00:22:41.268 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:22:41.310 INFO:tasks.ceph.mgr.x:Stopped 2023-10-08T00:22:41.311 INFO:teuthology.misc:Shutting down mon daemons... 2023-10-08T00:22:41.311 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-08T00:22:41.311 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:22:41.321 INFO:tasks.ceph.mon.a:Stopped 2023-10-08T00:22:41.321 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-10-08T00:22:41.322 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:22:41.362 INFO:tasks.ceph.mon.b:Stopped 2023-10-08T00:22:41.362 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-10-08T00:22:41.362 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:22:41.412 INFO:tasks.ceph.mon.c:Stopped 2023-10-08T00:22:41.413 INFO:tasks.ceph:Checking cluster log for badness... 2023-10-08T00:22:41.413 DEBUG:teuthology.orchestra.run.smithi099:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | egrep -v PG_RECOVERY_FULL | egrep -v PG_DEGRADED | head -n 1 2023-10-08T00:22:41.443 INFO:teuthology.orchestra.run.smithi099.stdout:1696723840.7136083 osd.2 (osd.2) 3 : cluster [WRN] OSD bench result of 227531.837772 IOPS exceeded the threshold limit of 500.000000 IOPS for osd.2. IOPS capacity is unchanged at 315.000000 IOPS. The recommendation is to establish the osd's IOPS capacity using other benchmark tools (e.g. Fio) and then override osd_mclock_max_capacity_iops_[hdd|ssd]. 2023-10-08T00:22:41.444 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2023-10-08T00:22:41.444 DEBUG:teuthology.orchestra.run.smithi099:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | egrep -v PG_RECOVERY_FULL | egrep -v PG_DEGRADED | head -n 1 2023-10-08T00:22:41.513 DEBUG:teuthology.orchestra.run.smithi099:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | egrep -v PG_RECOVERY_FULL | egrep -v PG_DEGRADED | head -n 1 2023-10-08T00:22:41.582 DEBUG:teuthology.orchestra.run.smithi099:> sudo egrep '\[WRN\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | egrep -v PG_RECOVERY_FULL | egrep -v PG_DEGRADED | head -n 1 2023-10-08T00:22:41.651 INFO:teuthology.orchestra.run.smithi099.stdout:1696723840.7136083 osd.2 (osd.2) 3 : cluster [WRN] OSD bench result of 227531.837772 IOPS exceeded the threshold limit of 500.000000 IOPS for osd.2. IOPS capacity is unchanged at 315.000000 IOPS. The recommendation is to establish the osd's IOPS capacity using other benchmark tools (e.g. Fio) and then override osd_mclock_max_capacity_iops_[hdd|ssd]. 2023-10-08T00:22:41.652 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi099.front.sepia.ceph.com 2023-10-08T00:22:41.652 DEBUG:teuthology.orchestra.run.smithi099:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-10-08T00:22:41.949 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi099.front.sepia.ceph.com 2023-10-08T00:22:41.950 DEBUG:teuthology.orchestra.run.smithi099:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-10-08T00:22:42.024 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi099.front.sepia.ceph.com 2023-10-08T00:22:42.024 DEBUG:teuthology.orchestra.run.smithi099:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-10-08T00:22:42.129 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi099.front.sepia.ceph.com 2023-10-08T00:22:42.129 DEBUG:teuthology.orchestra.run.smithi099:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-10-08T00:22:42.198 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi191.front.sepia.ceph.com 2023-10-08T00:22:42.199 DEBUG:teuthology.orchestra.run.smithi191:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-10-08T00:22:42.375 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi191.front.sepia.ceph.com 2023-10-08T00:22:42.375 DEBUG:teuthology.orchestra.run.smithi191:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-10-08T00:22:42.445 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi191.front.sepia.ceph.com 2023-10-08T00:22:42.445 DEBUG:teuthology.orchestra.run.smithi191:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-10-08T00:22:42.513 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi191.front.sepia.ceph.com 2023-10-08T00:22:42.513 DEBUG:teuthology.orchestra.run.smithi191:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-10-08T00:22:42.579 INFO:tasks.ceph:Archiving mon data... 2023-10-08T00:22:42.580 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418110/data/mon.a.tgz 2023-10-08T00:22:42.580 DEBUG:teuthology.orchestra.run.smithi099:> mktemp 2023-10-08T00:22:42.595 INFO:teuthology.orchestra.run.smithi099.stdout:/tmp/tmp.hLhyyacuxu 2023-10-08T00:22:42.596 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.hLhyyacuxu 2023-10-08T00:22:42.740 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 0666 /tmp/tmp.hLhyyacuxu 2023-10-08T00:22:42.822 DEBUG:teuthology.orchestra.remote:smithi099:/tmp/tmp.hLhyyacuxu is 359KB 2023-10-08T00:22:42.918 DEBUG:teuthology.orchestra.run.smithi099:> rm -fr /tmp/tmp.hLhyyacuxu 2023-10-08T00:22:42.932 DEBUG:teuthology.misc:Transferring archived files from smithi191:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418110/data/mon.b.tgz 2023-10-08T00:22:42.932 DEBUG:teuthology.orchestra.run.smithi191:> mktemp 2023-10-08T00:22:42.949 INFO:teuthology.orchestra.run.smithi191.stdout:/tmp/tmp.0kRVu49rPD 2023-10-08T00:22:42.949 DEBUG:teuthology.orchestra.run.smithi191:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.0kRVu49rPD 2023-10-08T00:22:43.094 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 0666 /tmp/tmp.0kRVu49rPD 2023-10-08T00:22:43.175 DEBUG:teuthology.orchestra.remote:smithi191:/tmp/tmp.0kRVu49rPD is 360KB 2023-10-08T00:22:43.269 DEBUG:teuthology.orchestra.run.smithi191:> rm -fr /tmp/tmp.0kRVu49rPD 2023-10-08T00:22:43.284 DEBUG:teuthology.misc:Transferring archived files from smithi191:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418110/data/mon.c.tgz 2023-10-08T00:22:43.284 DEBUG:teuthology.orchestra.run.smithi191:> mktemp 2023-10-08T00:22:43.337 INFO:teuthology.orchestra.run.smithi191.stdout:/tmp/tmp.fZ11a69ic2 2023-10-08T00:22:43.338 DEBUG:teuthology.orchestra.run.smithi191:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.fZ11a69ic2 2023-10-08T00:22:43.465 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 0666 /tmp/tmp.fZ11a69ic2 2023-10-08T00:22:43.545 DEBUG:teuthology.orchestra.remote:smithi191:/tmp/tmp.fZ11a69ic2 is 360KB 2023-10-08T00:22:43.642 DEBUG:teuthology.orchestra.run.smithi191:> rm -fr /tmp/tmp.fZ11a69ic2 2023-10-08T00:22:43.656 INFO:tasks.ceph:Cleaning ceph cluster... 2023-10-08T00:22:43.656 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-08T00:22:43.658 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-08T00:22:43.720 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-10-08T00:22:43.720 DEBUG:teuthology.orchestra.run.smithi099:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-08T00:22:43.724 DEBUG:teuthology.orchestra.run.smithi191:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-08T00:22:43.748 INFO:teuthology.orchestra.run.smithi099.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-08T00:22:43.785 INFO:teuthology.orchestra.run.smithi191.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-08T00:22:43.791 INFO:tasks.ceph:Archiving crash dumps... 2023-10-08T00:22:43.792 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418110/remote/smithi099/crash 2023-10-08T00:22:43.794 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-08T00:22:43.819 DEBUG:teuthology.misc:Transferring archived files from smithi191:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418110/remote/smithi191/crash 2023-10-08T00:22:43.820 DEBUG:teuthology.orchestra.run.smithi191:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-08T00:22:43.854 INFO:tasks.ceph:Shutting down logrotate 2023-10-08T00:22:43.854 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:22:43.863 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:22:43.988 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-08T00:22:44.012 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-08T00:22:44.036 INFO:tasks.ceph:Compressing logs... 2023-10-08T00:22:44.036 DEBUG:teuthology.orchestra.run.smithi099:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-08T00:22:44.054 DEBUG:teuthology.orchestra.run.smithi191:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-08T00:22:44.124 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86178.log 2023-10-08T00:22:44.127 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.87862.log 2023-10-08T00:22:44.128 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.admin.86178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.92008.log 2023-10-08T00:22:44.128 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.3.87862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86218.log 88.6% 2023-10-08T00:22:44.128 INFO:teuthology.orchestra.run.smithi191.stderr: -- replaced with /var/log/ceph/ceph-client.admin.86178.log.gz 2023-10-08T00:22:44.128 INFO:teuthology.orchestra.run.smithi191.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.87862.log.gz 2023-10-08T00:22:44.128 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.92008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.91099.log 2023-10-08T00:22:44.128 INFO:teuthology.orchestra.run.smithi191.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.92008.log.gz 2023-10-08T00:22:44.128 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.admin.86218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.87504.log 2023-10-08T00:22:44.128 INFO:teuthology.orchestra.run.smithi191.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.86218.log.gz 2023-10-08T00:22:44.128 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.91099.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-08T00:22:44.129 INFO:teuthology.orchestra.run.smithi191.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.91099.log.gz 2023-10-08T00:22:44.129 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.87504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.89576.log 2023-10-08T00:22:44.129 INFO:teuthology.orchestra.run.smithi191.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.87504.log.gz 2023-10-08T00:22:44.129 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.89123.log 2023-10-08T00:22:44.129 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.89576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.87496.log 2023-10-08T00:22:44.129 INFO:teuthology.orchestra.run.smithi191.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.89576.log.gz 2023-10-08T00:22:44.129 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.89123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.91103.log 2023-10-08T00:22:44.129 INFO:teuthology.orchestra.run.smithi191.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.89123.log.gz 2023-10-08T00:22:44.129 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.87496.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-10-08T00:22:44.130 INFO:teuthology.orchestra.run.smithi191.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.87496.log.gz 2023-10-08T00:22:44.130 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.91103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.88273.log 2023-10-08T00:22:44.130 INFO:teuthology.orchestra.run.smithi191.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.91103.log.gz 2023-10-08T00:22:44.130 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-08T00:22:44.130 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.88273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.90345.log 2023-10-08T00:22:44.130 INFO:teuthology.orchestra.run.smithi191.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.88273.log.gz 2023-10-08T00:22:44.130 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.89580.log 2023-10-08T00:22:44.130 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.90345.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.90345.log.gz 2023-10-08T00:22:44.130 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-10-08T00:22:44.130 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-10-08T00:22:44.131 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.89580.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.89580.log.gz 2023-10-08T00:22:44.131 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-10-08T00:22:44.131 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.90349.log 2023-10-08T00:22:44.131 INFO:teuthology.orchestra.run.smithi191.stderr: 92.0% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-08T00:22:44.131 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86302.log 2023-10-08T00:22:44.131 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.90349.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.90349.log.gz 2023-10-08T00:22:44.131 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.admin.86302.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.86302.log.gz 2023-10-08T00:22:44.131 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-10-08T00:22:44.131 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86344.log 2023-10-08T00:22:44.131 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-10-08T00:22:44.132 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.admin.86344.log: /var/log/ceph/ceph-mds.b.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.86344.log.gz 2023-10-08T00:22:44.132 INFO:teuthology.orchestra.run.smithi191.stderr: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-08T00:22:44.132 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.88269.log 2023-10-08T00:22:44.132 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.92012.log 2023-10-08T00:22:44.132 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.88269.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.88269.log.gz 2023-10-08T00:22:44.132 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.92012.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.92012.log.gz 2023-10-08T00:22:44.132 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-10-08T00:22:44.132 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86259.log 2023-10-08T00:22:44.132 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-osd.6.log: /var/log/ceph/ceph-client.admin.86259.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.86259.log.gz 2023-10-08T00:22:44.132 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.88646.log 2023-10-08T00:22:44.132 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.88646.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.88646.log.gz 2023-10-08T00:22:44.132 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.90718.log 2023-10-08T00:22:44.133 INFO:teuthology.orchestra.run.smithi191.stderr: 92.7% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-10-08T00:22:44.133 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.90718.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.90718.log.gz 2023-10-08T00:22:44.133 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-10-08T00:22:44.133 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.87866.log 2023-10-08T00:22:44.133 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-osd.5.log: /var/log/ceph/ceph-client.3.87866.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.87866.log.gz 2023-10-08T00:22:44.133 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86388.log 2023-10-08T00:22:44.133 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.admin.86388.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.86388.log.gz 2023-10-08T00:22:44.133 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86431.log 2023-10-08T00:22:44.133 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.admin.86431.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.86431.log.gz 2023-10-08T00:22:44.133 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.89119.log 2023-10-08T00:22:44.134 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.89119.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.89119.log.gz 2023-10-08T00:22:44.134 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.88642.log 2023-10-08T00:22:44.134 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.88642.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.88642.log.gz 2023-10-08T00:22:44.134 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.89969.log 2023-10-08T00:22:44.134 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.89969.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.89969.log.gz 2023-10-08T00:22:44.134 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.89965.log 2023-10-08T00:22:44.134 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.89965.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.89965.log.gz 2023-10-08T00:22:44.134 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-10-08T00:22:44.135 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.90722.log 2023-10-08T00:22:44.154 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.90722.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.90722.log.gz 2023-10-08T00:22:44.154 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.91596.log 2023-10-08T00:22:44.179 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.91596.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.91596.log.gz 2023-10-08T00:22:44.179 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.91592.log 2023-10-08T00:22:44.191 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.91592.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.91592.log.gz 2023-10-08T00:22:44.191 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86474.log 2023-10-08T00:22:44.204 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.admin.86474.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.86474.log.gz 2023-10-08T00:22:44.247 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94731.log 2023-10-08T00:22:44.247 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115567.log 2023-10-08T00:22:44.247 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.94731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93266.log 2023-10-08T00:22:44.247 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.115567.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.94731.log.gz 2023-10-08T00:22:44.247 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108207.log 2023-10-08T00:22:44.248 INFO:teuthology.orchestra.run.smithi099.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.115567.log.gz 2023-10-08T00:22:44.248 INFO:teuthology.orchestra.run.smithi099.stderr:gzip /var/log/ceph/ceph-client.admin.93266.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.99591.log 2023-10-08T00:22:44.248 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.108207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.98996.log 2023-10-08T00:22:44.248 INFO:teuthology.orchestra.run.smithi099.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.93266.log.gz 2023-10-08T00:22:44.248 INFO:teuthology.orchestra.run.smithi099.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.108207.log.gz/var/log/ceph/ceph-client.admin.99591.log: 2023-10-08T00:22:44.248 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109829.log 2023-10-08T00:22:44.248 INFO:teuthology.orchestra.run.smithi099.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.99591.log.gz 2023-10-08T00:22:44.248 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.0.98996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98803.log 2023-10-08T00:22:44.248 INFO:teuthology.orchestra.run.smithi099.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.98996.log.gz 2023-10-08T00:22:44.249 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.109829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96458.log 2023-10-08T00:22:44.249 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.98803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115651.log 2023-10-08T00:22:44.249 INFO:teuthology.orchestra.run.smithi099.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.109829.log.gz 2023-10-08T00:22:44.249 INFO:teuthology.orchestra.run.smithi099.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.98803.log.gz 2023-10-08T00:22:44.249 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.96458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99929.log 2023-10-08T00:22:44.249 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.115651.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.96458.log.gz 2023-10-08T00:22:44.249 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.111685.log 2023-10-08T00:22:44.249 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.99929.log: gzip 88.6%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.115651.log.gz-- 2023-10-08T00:22:44.249 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.109191.log 2023-10-08T00:22:44.249 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.0.111685.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99929.log.gz 2023-10-08T00:22:44.249 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98653.log 2023-10-08T00:22:44.250 INFO:teuthology.orchestra.run.smithi099.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.111685.log.gz 2023-10-08T00:22:44.250 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.109191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112740.log 2023-10-08T00:22:44.250 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.98653.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.109191.log.gzgzip 2023-10-08T00:22:44.250 INFO:teuthology.orchestra.run.smithi099.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.115861.log 2023-10-08T00:22:44.250 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.112740.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.98653.log.gz 2023-10-08T00:22:44.250 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91926.log 2023-10-08T00:22:44.250 INFO:teuthology.orchestra.run.smithi099.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.112740.log.gz 2023-10-08T00:22:44.250 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.115861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111124.log 2023-10-08T00:22:44.250 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.91926.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.115861.log.gzgzip 2023-10-08T00:22:44.250 INFO:teuthology.orchestra.run.smithi099.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.89127.log 2023-10-08T00:22:44.250 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.111124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113355.log 2023-10-08T00:22:44.251 INFO:teuthology.orchestra.run.smithi099.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.91926.log.gz 2023-10-08T00:22:44.251 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.89127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88711.log 2023-10-08T00:22:44.251 INFO:teuthology.orchestra.run.smithi099.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.111124.log.gz 2023-10-08T00:22:44.251 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.113355.log: gzip 88.7%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.89127.log.gz-- 2023-10-08T00:22:44.251 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.92948.log 2023-10-08T00:22:44.251 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.88711.log: 88.8%gzip -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.113355.log.gz-- /var/log/ceph/ceph-client.admin.97214.log 2023-10-08T00:22:44.251 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-10-08T00:22:44.251 INFO:teuthology.orchestra.run.smithi099.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.88711.log.gz 2023-10-08T00:22:44.251 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.92948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109997.log 2023-10-08T00:22:44.251 INFO:teuthology.orchestra.run.smithi099.stderr: 88.2%/var/log/ceph/ceph-client.admin.97214.log: -- replaced with /var/log/ceph/ceph-client.admin.92948.log.gz 2023-10-08T00:22:44.252 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111730.log 2023-10-08T00:22:44.252 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.109997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89253.log 2023-10-08T00:22:44.252 INFO:teuthology.orchestra.run.smithi099.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.97214.log.gz 2023-10-08T00:22:44.252 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.111730.log: 88.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.109997.log.gz-5 --verbose 2023-10-08T00:22:44.252 INFO:teuthology.orchestra.run.smithi099.stderr:-- /var/log/ceph/ceph-client.admin.111492.log 2023-10-08T00:22:44.252 INFO:teuthology.orchestra.run.smithi099.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.111730.log.gz 2023-10-08T00:22:44.252 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.89253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102439.log 2023-10-08T00:22:44.252 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.111492.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.89253.log.gz 2023-10-08T00:22:44.252 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103005.log 2023-10-08T00:22:44.252 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.102439.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.111492.log.gz 2023-10-08T00:22:44.252 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110700.log 2023-10-08T00:22:44.253 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.103005.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.102439.log.gz 2023-10-08T00:22:44.253 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105456.log 2023-10-08T00:22:44.253 INFO:teuthology.orchestra.run.smithi099.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.103005.log.gz 2023-10-08T00:22:44.253 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.110700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115693.log 2023-10-08T00:22:44.253 INFO:teuthology.orchestra.run.smithi099.stderr: 87.6%/var/log/ceph/ceph-client.admin.105456.log: -- replaced with /var/log/ceph/ceph-client.admin.110700.log.gz 2023-10-08T00:22:44.253 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98527.log 2023-10-08T00:22:44.253 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.115693.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.105456.log.gz 2023-10-08T00:22:44.253 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112051.log 2023-10-08T00:22:44.253 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.98527.log: gzip -5 88.3%--verbose -- /var/log/ceph/ceph-client.admin.92130.log -- replaced with /var/log/ceph/ceph-client.admin.115693.log.gz 2023-10-08T00:22:44.253 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-10-08T00:22:44.253 INFO:teuthology.orchestra.run.smithi099.stderr: 87.8%/var/log/ceph/ceph-client.admin.112051.log: -- replaced with /var/log/ceph/ceph-client.admin.98527.log.gz 2023-10-08T00:22:44.254 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118493.log 2023-10-08T00:22:44.254 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.92130.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.112051.log.gz 2023-10-08T00:22:44.254 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95183.log 2023-10-08T00:22:44.254 INFO:teuthology.orchestra.run.smithi099.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.92130.log.gz/var/log/ceph/ceph-client.admin.118493.log: 2023-10-08T00:22:44.254 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106937.log 2023-10-08T00:22:44.254 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.95183.log: gzip 89.2%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.118493.log.gz-- /var/log/ceph/ceph-client.admin.88369.log 2023-10-08T00:22:44.254 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-10-08T00:22:44.254 INFO:teuthology.orchestra.run.smithi099.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.95183.log.gz 2023-10-08T00:22:44.254 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.106937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97340.log 2023-10-08T00:22:44.254 INFO:teuthology.orchestra.run.smithi099.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.106937.log.gz 2023-10-08T00:22:44.254 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.88369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.90495.log 2023-10-08T00:22:44.254 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.97340.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.88369.log.gz 2023-10-08T00:22:44.255 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97935.log 2023-10-08T00:22:44.255 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.2.90495.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.97340.log.gz 2023-10-08T00:22:44.255 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.111689.log 2023-10-08T00:22:44.255 INFO:teuthology.orchestra.run.smithi099.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.90495.log.gz 2023-10-08T00:22:44.255 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.97935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103987.log 2023-10-08T00:22:44.255 INFO:teuthology.orchestra.run.smithi099.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.97935.log.gz 2023-10-08T00:22:44.255 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.0.111689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88959.log 2023-10-08T00:22:44.255 INFO:teuthology.orchestra.run.smithi099.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.111689.log.gz 2023-10-08T00:22:44.255 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.103987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88714.log 2023-10-08T00:22:44.255 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.88959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116704.log 2023-10-08T00:22:44.255 INFO:teuthology.orchestra.run.smithi099.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.103987.log.gz 2023-10-08T00:22:44.255 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.88714.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-10-08T00:22:44.256 INFO:teuthology.orchestra.run.smithi099.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.88959.log.gz 2023-10-08T00:22:44.256 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.116704.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.88714.log.gz 2023-10-08T00:22:44.256 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102298.log 2023-10-08T00:22:44.256 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113609.log 2023-10-08T00:22:44.256 INFO:teuthology.orchestra.run.smithi099.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.116704.log.gz 2023-10-08T00:22:44.256 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.102298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87132.log 2023-10-08T00:22:44.256 INFO:teuthology.orchestra.run.smithi099.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.102298.log.gz 2023-10-08T00:22:44.256 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.113609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100611.log 2023-10-08T00:22:44.256 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.87132.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.113609.log.gz 2023-10-08T00:22:44.256 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104665.log 2023-10-08T00:22:44.256 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.100611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113313.log 2023-10-08T00:22:44.257 INFO:teuthology.orchestra.run.smithi099.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.87132.log.gz 2023-10-08T00:22:44.257 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.104665.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.100611.log.gz 2023-10-08T00:22:44.257 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116963.log 2023-10-08T00:22:44.257 INFO:teuthology.orchestra.run.smithi099.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.104665.log.gz 2023-10-08T00:22:44.257 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.113313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98063.log 2023-10-08T00:22:44.257 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.116963.log: gzip 88.1%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.113313.log.gz-- 2023-10-08T00:22:44.257 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.103835.log 2023-10-08T00:22:44.257 INFO:teuthology.orchestra.run.smithi099.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.116963.log.gz 2023-10-08T00:22:44.257 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.98063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.101250.log 2023-10-08T00:22:44.257 INFO:teuthology.orchestra.run.smithi099.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.98063.log.gz 2023-10-08T00:22:44.257 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.103835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98611.log 2023-10-08T00:22:44.258 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.0.101250.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.103835.log.gz 2023-10-08T00:22:44.258 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112530.log 2023-10-08T00:22:44.258 INFO:teuthology.orchestra.run.smithi099.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.101250.log.gz 2023-10-08T00:22:44.258 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.98611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94502.log 2023-10-08T00:22:44.258 INFO:teuthology.orchestra.run.smithi099.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.98611.log.gz 2023-10-08T00:22:44.258 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.112530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94376.log 2023-10-08T00:22:44.258 INFO:teuthology.orchestra.run.smithi099.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.112530.log.gz 2023-10-08T00:22:44.258 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.94502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105040.log 2023-10-08T00:22:44.258 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.94376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100823.log 2023-10-08T00:22:44.258 INFO:teuthology.orchestra.run.smithi099.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.94502.log.gz 2023-10-08T00:22:44.258 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.105040.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.94376.log.gz 2023-10-08T00:22:44.258 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92052.log 2023-10-08T00:22:44.259 INFO:teuthology.orchestra.run.smithi099.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.105040.log.gz 2023-10-08T00:22:44.259 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.100823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104894.log 2023-10-08T00:22:44.259 INFO:teuthology.orchestra.run.smithi099.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.100823.log.gz 2023-10-08T00:22:44.259 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.92052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92464.log 2023-10-08T00:22:44.259 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.104894.log: 88.3%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.92052.log.gz--verbose 2023-10-08T00:22:44.259 INFO:teuthology.orchestra.run.smithi099.stderr:-- /var/log/ceph/ceph-client.admin.101965.log 2023-10-08T00:22:44.259 INFO:teuthology.orchestra.run.smithi099.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.104894.log.gz 2023-10-08T00:22:44.259 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.92464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94460.log 2023-10-08T00:22:44.259 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.101965.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.92464.log.gz 2023-10-08T00:22:44.259 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111772.log 2023-10-08T00:22:44.259 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.94460.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.101965.log.gz 2023-10-08T00:22:44.260 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105945.log 2023-10-08T00:22:44.260 INFO:teuthology.orchestra.run.smithi099.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.94460.log.gz 2023-10-08T00:22:44.260 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.111772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114675.log 2023-10-08T00:22:44.260 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.105945.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.111772.log.gz 2023-10-08T00:22:44.260 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113219.log 2023-10-08T00:22:44.260 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.114675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114339.log 2023-10-08T00:22:44.260 INFO:teuthology.orchestra.run.smithi099.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.105945.log.gz 2023-10-08T00:22:44.260 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.113219.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.114675.log.gz 2023-10-08T00:22:44.260 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91187.log 2023-10-08T00:22:44.260 INFO:teuthology.orchestra.run.smithi099.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.113219.log.gz 2023-10-08T00:22:44.260 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.114339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87657.log 2023-10-08T00:22:44.260 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.91187.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.114339.log.gz 2023-10-08T00:22:44.261 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91758.log 2023-10-08T00:22:44.261 INFO:teuthology.orchestra.run.smithi099.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.91187.log.gz 2023-10-08T00:22:44.261 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.87657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105326.log 2023-10-08T00:22:44.261 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.91758.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.87657.log.gz 2023-10-08T00:22:44.261 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95023.log 2023-10-08T00:22:44.261 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.105326.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.91758.log.gz 2023-10-08T00:22:44.261 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115819.log 2023-10-08T00:22:44.261 INFO:teuthology.orchestra.run.smithi099.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.105326.log.gz 2023-10-08T00:22:44.261 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.95023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88373.log 2023-10-08T00:22:44.261 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.115819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92295.log 2023-10-08T00:22:44.261 INFO:teuthology.orchestra.run.smithi099.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.95023.log.gz 2023-10-08T00:22:44.261 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.88373.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.115819.log.gz 2023-10-08T00:22:44.262 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90582.log 2023-10-08T00:22:44.262 INFO:teuthology.orchestra.run.smithi099.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.88373.log.gz 2023-10-08T00:22:44.262 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.92295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99803.log 2023-10-08T00:22:44.262 INFO:teuthology.orchestra.run.smithi099.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.92295.log.gz 2023-10-08T00:22:44.262 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.90582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114862.log 2023-10-08T00:22:44.262 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.99803.log: gzip -5 88.2%--verbose -- /var/log/ceph/ceph-client.admin.98149.log -- replaced with /var/log/ceph/ceph-client.admin.90582.log.gz 2023-10-08T00:22:44.262 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-10-08T00:22:44.262 INFO:teuthology.orchestra.run.smithi099.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.99803.log.gz 2023-10-08T00:22:44.262 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.114862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115609.log 2023-10-08T00:22:44.262 INFO:teuthology.orchestra.run.smithi099.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.114862.log.gz 2023-10-08T00:22:44.262 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.98149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100519.log 2023-10-08T00:22:44.263 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.115609.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.98149.log.gz 2023-10-08T00:22:44.263 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97893.log 2023-10-08T00:22:44.263 INFO:teuthology.orchestra.run.smithi099.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.115609.log.gz 2023-10-08T00:22:44.263 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.100519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116562.log 2023-10-08T00:22:44.263 INFO:teuthology.orchestra.run.smithi099.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.100519.log.gz 2023-10-08T00:22:44.263 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.97893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111384.log 2023-10-08T00:22:44.263 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.116562.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.97893.log.gz 2023-10-08T00:22:44.263 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92253.log 2023-10-08T00:22:44.263 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.111384.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.116562.log.gz 2023-10-08T00:22:44.263 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88371.log 2023-10-08T00:22:44.263 INFO:teuthology.orchestra.run.smithi099.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.111384.log.gz 2023-10-08T00:22:44.263 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.92253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94857.log 2023-10-08T00:22:44.264 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.88371.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.92253.log.gz 2023-10-08T00:22:44.264 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92506.log 2023-10-08T00:22:44.264 INFO:teuthology.orchestra.run.smithi099.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.88371.log.gz 2023-10-08T00:22:44.264 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.94857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87867.log 2023-10-08T00:22:44.264 INFO:teuthology.orchestra.run.smithi099.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.94857.log.gz 2023-10-08T00:22:44.264 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.92506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112387.log 2023-10-08T00:22:44.264 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.87867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113693.log 2023-10-08T00:22:44.264 INFO:teuthology.orchestra.run.smithi099.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.92506.log.gz 2023-10-08T00:22:44.264 INFO:teuthology.orchestra.run.smithi099.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.87867.log.gz 2023-10-08T00:22:44.264 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.112387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100907.log 2023-10-08T00:22:44.264 INFO:teuthology.orchestra.run.smithi099.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.112387.log.gz 2023-10-08T00:22:44.265 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.113693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96500.log 2023-10-08T00:22:44.265 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.100907.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.113693.log.gz 2023-10-08T00:22:44.265 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114081.log 2023-10-08T00:22:44.265 INFO:teuthology.orchestra.run.smithi099.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.100907.log.gz 2023-10-08T00:22:44.265 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.96500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114507.log 2023-10-08T00:22:44.265 INFO:teuthology.orchestra.run.smithi099.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.96500.log.gz 2023-10-08T00:22:44.265 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.114081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89001.log 2023-10-08T00:22:44.265 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.114507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98569.log 2023-10-08T00:22:44.265 INFO:teuthology.orchestra.run.smithi099.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.114081.log.gz 2023-10-08T00:22:44.265 INFO:teuthology.orchestra.run.smithi099.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.114507.log.gz 2023-10-08T00:22:44.265 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.89001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117412.log 2023-10-08T00:22:44.265 INFO:teuthology.orchestra.run.smithi099.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.89001.log.gz 2023-10-08T00:22:44.266 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.98569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115151.log 2023-10-08T00:22:44.266 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.117412.log: 88.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.98569.log.gz-5 2023-10-08T00:22:44.266 INFO:teuthology.orchestra.run.smithi099.stderr:--verbose -- /var/log/ceph/ceph-client.admin.110868.log 2023-10-08T00:22:44.266 INFO:teuthology.orchestra.run.smithi099.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.117412.log.gz 2023-10-08T00:22:44.266 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.115151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113175.log 2023-10-08T00:22:44.266 INFO:teuthology.orchestra.run.smithi099.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.115151.log.gz 2023-10-08T00:22:44.266 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.110868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107417.log 2023-10-08T00:22:44.266 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.113175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110784.log 2023-10-08T00:22:44.266 INFO:teuthology.orchestra.run.smithi099.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.110868.log.gz 2023-10-08T00:22:44.266 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.107417.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.113175.log.gz 2023-10-08T00:22:44.266 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89421.log 2023-10-08T00:22:44.266 INFO:teuthology.orchestra.run.smithi099.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.107417.log.gz 2023-10-08T00:22:44.267 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.110784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116921.log 2023-10-08T00:22:44.267 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.89421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99083.log 2023-10-08T00:22:44.267 INFO:teuthology.orchestra.run.smithi099.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.110784.log.gz 2023-10-08T00:22:44.267 INFO:teuthology.orchestra.run.smithi099.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.89421.log.gz 2023-10-08T00:22:44.267 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.116921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93200.log 2023-10-08T00:22:44.267 INFO:teuthology.orchestra.run.smithi099.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.116921.log.gz 2023-10-08T00:22:44.267 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.99083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103215.log 2023-10-08T00:22:44.267 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.93200.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.99083.log.gz 2023-10-08T00:22:44.267 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106979.log 2023-10-08T00:22:44.267 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.103215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93771.log 2023-10-08T00:22:44.267 INFO:teuthology.orchestra.run.smithi099.stderr: 88.2% 91.0% -- replaced with /var/log/ceph/ceph-client.admin.103215.log.gz 2023-10-08T00:22:44.267 INFO:teuthology.orchestra.run.smithi099.stderr: -- replaced with /var/log/ceph/ceph-client.admin.93200.log.gz 2023-10-08T00:22:44.268 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.106979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106475.log 2023-10-08T00:22:44.268 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.93771.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.106979.log.gz 2023-10-08T00:22:44.268 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87048.log 2023-10-08T00:22:44.268 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.106475.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.93771.log.gz 2023-10-08T00:22:44.268 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88032.log 2023-10-08T00:22:44.268 INFO:teuthology.orchestra.run.smithi099.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.106475.log.gz 2023-10-08T00:22:44.268 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.87048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95395.log 2023-10-08T00:22:44.268 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.88032.log: 87.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.87048.log.gz-5 2023-10-08T00:22:44.268 INFO:teuthology.orchestra.run.smithi099.stderr:--verbose -- /var/log/ceph/ceph-client.admin.116029.log 2023-10-08T00:22:44.268 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.95395.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.88032.log.gz 2023-10-08T00:22:44.268 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117202.log 2023-10-08T00:22:44.268 INFO:teuthology.orchestra.run.smithi099.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.95395.log.gz 2023-10-08T00:22:44.269 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.116029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112345.log 2023-10-08T00:22:44.269 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.117202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86879.log 2023-10-08T00:22:44.269 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.112345.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.117202.log.gz 2023-10-08T00:22:44.269 INFO:teuthology.orchestra.run.smithi099.stderr: 91.1%gzip -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.116029.log.gz-- 2023-10-08T00:22:44.269 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.99761.log 2023-10-08T00:22:44.269 INFO:teuthology.orchestra.run.smithi099.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.112345.log.gz 2023-10-08T00:22:44.269 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.86879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99506.log 2023-10-08T00:22:44.269 INFO:teuthology.orchestra.run.smithi099.stderr: 89.4%/var/log/ceph/ceph-client.admin.99761.log: -- replaced with /var/log/ceph/ceph-client.admin.86879.log.gz 2023-10-08T00:22:44.269 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94899.log 2023-10-08T00:22:44.269 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.99506.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.99761.log.gz 2023-10-08T00:22:44.269 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111174.log 2023-10-08T00:22:44.270 INFO:teuthology.orchestra.run.smithi099.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.99506.log.gz 2023-10-08T00:22:44.270 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.94899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117947.log 2023-10-08T00:22:44.270 INFO:teuthology.orchestra.run.smithi099.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.94899.log.gz 2023-10-08T00:22:44.270 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.111174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108563.log 2023-10-08T00:22:44.270 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.117947.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.111174.log.gz 2023-10-08T00:22:44.270 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92338.log 2023-10-08T00:22:44.270 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.108563.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.117947.log.gz 2023-10-08T00:22:44.270 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92632.log 2023-10-08T00:22:44.271 INFO:teuthology.orchestra.run.smithi099.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.108563.log.gz 2023-10-08T00:22:44.271 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.92338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101057.log 2023-10-08T00:22:44.271 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.92632.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.92338.log.gz 2023-10-08T00:22:44.271 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.96164.log 2023-10-08T00:22:44.271 INFO:teuthology.orchestra.run.smithi099.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.92632.log.gz 2023-10-08T00:22:44.272 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.101057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117664.log 2023-10-08T00:22:44.272 INFO:teuthology.orchestra.run.smithi099.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.101057.log.gz 2023-10-08T00:22:44.272 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.0.96164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92990.log 2023-10-08T00:22:44.272 INFO:teuthology.orchestra.run.smithi099.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.96164.log.gz 2023-10-08T00:22:44.272 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.117664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112698.log 2023-10-08T00:22:44.272 INFO:teuthology.orchestra.run.smithi099.stderr: 88.2%/var/log/ceph/ceph-client.admin.92990.log: -- replaced with /var/log/ceph/ceph-client.admin.117664.log.gz 2023-10-08T00:22:44.272 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-08T00:22:44.272 INFO:teuthology.orchestra.run.smithi099.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.92990.log.gz 2023-10-08T00:22:44.272 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112698.log: /var/log/ceph/ceph-client.admin.91674.log 2023-10-08T00:22:44.272 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112844.log 2023-10-08T00:22:44.272 INFO:teuthology.orchestra.run.smithi099.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.112698.log.gz 2023-10-08T00:22:44.272 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.91674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94250.log 2023-10-08T00:22:44.273 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.112844.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.91674.log.gz 2023-10-08T00:22:44.273 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106559.log 2023-10-08T00:22:44.273 INFO:teuthology.orchestra.run.smithi099.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.112844.log.gz 2023-10-08T00:22:44.273 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.94250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109233.log 2023-10-08T00:22:44.273 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111216.log 2023-10-08T00:22:44.273 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.106559.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.94250.log.gz 2023-10-08T00:22:44.273 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.109233.log: 88.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.106559.log.gz-5 2023-10-08T00:22:44.273 INFO:teuthology.orchestra.run.smithi099.stderr:--verbose -- /var/log/ceph/ceph-client.admin.94292.log 2023-10-08T00:22:44.273 INFO:teuthology.orchestra.run.smithi099.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.109233.log.gz 2023-10-08T00:22:44.273 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91145.log 2023-10-08T00:22:44.273 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.111216.log: /var/log/ceph/ceph-client.admin.94292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104413.log 2023-10-08T00:22:44.274 INFO:teuthology.orchestra.run.smithi099.stderr: 85.5% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-08T00:22:44.274 INFO:teuthology.orchestra.run.smithi099.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.111216.log.gz 2023-10-08T00:22:44.274 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.91145.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.94292.log.gz 2023-10-08T00:22:44.274 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99887.log 2023-10-08T00:22:44.274 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.104413.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.91145.log.gz 2023-10-08T00:22:44.274 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100781.log 2023-10-08T00:22:44.274 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.99887.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.104413.log.gz 2023-10-08T00:22:44.274 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114423.log 2023-10-08T00:22:44.274 INFO:teuthology.orchestra.run.smithi099.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99887.log.gz 2023-10-08T00:22:44.275 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.100781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108479.log 2023-10-08T00:22:44.275 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.114423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104371.log 2023-10-08T00:22:44.275 INFO:teuthology.orchestra.run.smithi099.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.100781.log.gz 2023-10-08T00:22:44.275 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.108479.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.114423.log.gz 2023-10-08T00:22:44.275 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101923.log 2023-10-08T00:22:44.275 INFO:teuthology.orchestra.run.smithi099.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.108479.log.gz 2023-10-08T00:22:44.275 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.104371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98019.log 2023-10-08T00:22:44.275 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.101923.log: gzip 88.4%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.104371.log.gz-- 2023-10-08T00:22:44.275 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.87741.log 2023-10-08T00:22:44.275 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.98019.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.101923.log.gz 2023-10-08T00:22:44.275 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118661.log 2023-10-08T00:22:44.275 INFO:teuthology.orchestra.run.smithi099.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.98019.log.gz 2023-10-08T00:22:44.276 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.87741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88375.log 2023-10-08T00:22:44.276 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.118661.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.87741.log.gz 2023-10-08T00:22:44.276 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.93463.log 2023-10-08T00:22:44.276 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.88375.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.118661.log.gz 2023-10-08T00:22:44.276 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103323.log 2023-10-08T00:22:44.276 INFO:teuthology.orchestra.run.smithi099.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.88375.log.gz 2023-10-08T00:22:44.276 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.0.93463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99635.log 2023-10-08T00:22:44.276 INFO:teuthology.orchestra.run.smithi099.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.93463.log.gz 2023-10-08T00:22:44.276 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.103323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104978.log 2023-10-08T00:22:44.276 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.99635.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.103323.log.gz 2023-10-08T00:22:44.276 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.103516.log 2023-10-08T00:22:44.276 INFO:teuthology.orchestra.run.smithi099.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.99635.log.gz 2023-10-08T00:22:44.277 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.104978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103257.log 2023-10-08T00:22:44.277 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.0.103516.log: 89.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.104978.log.gz-5 2023-10-08T00:22:44.277 INFO:teuthology.orchestra.run.smithi099.stderr:--verbose -- /var/log/ceph/ceph-client.admin.99464.log 2023-10-08T00:22:44.277 INFO:teuthology.orchestra.run.smithi099.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.103516.log.gz 2023-10-08T00:22:44.277 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.103257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100467.log 2023-10-08T00:22:44.277 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.99464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110540.log 2023-10-08T00:22:44.277 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.100467.log: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.103257.log.gz 2023-10-08T00:22:44.277 INFO:teuthology.orchestra.run.smithi099.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99464.log.gz 2023-10-08T00:22:44.277 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109528.log 2023-10-08T00:22:44.277 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.110540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108336.log 88.9% 2023-10-08T00:22:44.277 INFO:teuthology.orchestra.run.smithi099.stderr: -- replaced with /var/log/ceph/ceph-client.admin.100467.log.gz 2023-10-08T00:22:44.278 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.109528.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.110540.log.gz 2023-10-08T00:22:44.278 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114465.log 2023-10-08T00:22:44.278 INFO:teuthology.orchestra.run.smithi099.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.109528.log.gz 2023-10-08T00:22:44.278 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.108336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97466.log 2023-10-08T00:22:44.278 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.114465.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.108336.log.gz 2023-10-08T00:22:44.278 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93116.log 2023-10-08T00:22:44.278 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.97466.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.114465.log.gz 2023-10-08T00:22:44.278 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118157.log 2023-10-08T00:22:44.278 INFO:teuthology.orchestra.run.smithi099.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.97466.log.gz 2023-10-08T00:22:44.278 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.93116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91968.log 2023-10-08T00:22:44.278 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.118157.log: gzip -5 --verbose -- 89.1%/var/log/ceph/ceph-client.admin.92906.log 2023-10-08T00:22:44.278 INFO:teuthology.orchestra.run.smithi099.stderr: -- replaced with /var/log/ceph/ceph-client.admin.93116.log.gz 2023-10-08T00:22:44.279 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.91968.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.118157.log.gz 2023-10-08T00:22:44.279 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92380.log 2023-10-08T00:22:44.279 INFO:teuthology.orchestra.run.smithi099.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.91968.log.gz 2023-10-08T00:22:44.279 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.92906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115353.log 2023-10-08T00:22:44.279 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.92380.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.92906.log.gz 2023-10-08T00:22:44.279 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95859.log 2023-10-08T00:22:44.279 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.115353.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.92380.log.gz 2023-10-08T00:22:44.279 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97088.log 2023-10-08T00:22:44.279 INFO:teuthology.orchestra.run.smithi099.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.115353.log.gz 2023-10-08T00:22:44.279 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.95859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98275.log 2023-10-08T00:22:44.279 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.97088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102911.log 2023-10-08T00:22:44.280 INFO:teuthology.orchestra.run.smithi099.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.95859.log.gz 2023-10-08T00:22:44.280 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.98275.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.97088.log.gz 2023-10-08T00:22:44.280 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107544.log 2023-10-08T00:22:44.280 INFO:teuthology.orchestra.run.smithi099.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.98275.log.gz 2023-10-08T00:22:44.280 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.102911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101839.log 2023-10-08T00:22:44.280 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.107544.log: 88.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.102911.log.gz-5 2023-10-08T00:22:44.280 INFO:teuthology.orchestra.run.smithi099.stderr:--verbose -- /var/log/ceph/ceph-client.admin.86096.log 2023-10-08T00:22:44.280 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.101839.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.107544.log.gz 2023-10-08T00:22:44.280 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.113998.log 2023-10-08T00:22:44.280 INFO:teuthology.orchestra.run.smithi099.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.101839.log.gz 2023-10-08T00:22:44.280 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.86096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87615.log 2023-10-08T00:22:44.280 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.0.113998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96758.log 2023-10-08T00:22:44.281 INFO:teuthology.orchestra.run.smithi099.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.113998.log.gz 2023-10-08T00:22:44.281 INFO:teuthology.orchestra.run.smithi099.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.86096.log.gz 2023-10-08T00:22:44.281 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.87615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91103.log 2023-10-08T00:22:44.281 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.96758.log: gzip -5 --verbose -- 88.7%/var/log/ceph/ceph-client.admin.105118.log 2023-10-08T00:22:44.281 INFO:teuthology.orchestra.run.smithi099.stderr: -- replaced with /var/log/ceph/ceph-client.admin.87615.log.gz 2023-10-08T00:22:44.281 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.91103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94418.log 2023-10-08T00:22:44.281 INFO:teuthology.orchestra.run.smithi099.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.96758.log.gz 2023-10-08T00:22:44.281 INFO:teuthology.orchestra.run.smithi099.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.91103.log.gz 2023-10-08T00:22:44.281 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.105118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110826.log 2023-10-08T00:22:44.281 INFO:teuthology.orchestra.run.smithi099.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.105118.log.gz 2023-10-08T00:22:44.281 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.94418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98191.log 2023-10-08T00:22:44.281 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.110826.log: gzip 89.0%-5 -- replaced with /var/log/ceph/ceph-client.admin.94418.log.gz--verbose 2023-10-08T00:22:44.282 INFO:teuthology.orchestra.run.smithi099.stderr:-- /var/log/ceph/ceph-client.admin.99380.log 2023-10-08T00:22:44.282 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.98191.log: gzip 87.6%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.110826.log.gz-- 2023-10-08T00:22:44.282 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.113049.log 2023-10-08T00:22:44.282 INFO:teuthology.orchestra.run.smithi099.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.98191.log.gz 2023-10-08T00:22:44.282 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.99380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88037.log 2023-10-08T00:22:44.282 INFO:teuthology.orchestra.run.smithi099.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.99380.log.gz 2023-10-08T00:22:44.282 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.113049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117370.log 2023-10-08T00:22:44.282 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.88037.log: 89.4%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.113049.log.gz--verbose 2023-10-08T00:22:44.282 INFO:teuthology.orchestra.run.smithi099.stderr:-- /var/log/ceph/ceph-client.admin.100949.log 2023-10-08T00:22:44.282 INFO:teuthology.orchestra.run.smithi099.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.88037.log.gz 2023-10-08T00:22:44.283 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.117370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87825.log 2023-10-08T00:22:44.283 INFO:teuthology.orchestra.run.smithi099.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.117370.log.gz 2023-10-08T00:22:44.283 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.100949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104936.log 2023-10-08T00:22:44.283 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.87825.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100949.log.gz 2023-10-08T00:22:44.283 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106011.log 2023-10-08T00:22:44.283 INFO:teuthology.orchestra.run.smithi099.stderr: 87.8%/var/log/ceph/ceph-client.admin.104936.log: -- replaced with /var/log/ceph/ceph-client.admin.87825.log.gz 2023-10-08T00:22:44.283 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113735.log 2023-10-08T00:22:44.283 INFO:teuthology.orchestra.run.smithi099.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.104936.log.gz 2023-10-08T00:22:44.283 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.106011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90876.log 2023-10-08T00:22:44.283 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.113735.log: 87.7%gzip -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.106011.log.gz-- 2023-10-08T00:22:44.283 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.96400.log 2023-10-08T00:22:44.283 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.90876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111300.log 2023-10-08T00:22:44.284 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.96400.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.113735.log.gz 2023-10-08T00:22:44.284 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109291.log 2023-10-08T00:22:44.284 INFO:teuthology.orchestra.run.smithi099.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.90876.log.gz 2023-10-08T00:22:44.284 INFO:teuthology.orchestra.run.smithi099.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.96400.log.gz 2023-10-08T00:22:44.284 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.111300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113441.log 2023-10-08T00:22:44.284 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.109291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106680.log 2023-10-08T00:22:44.284 INFO:teuthology.orchestra.run.smithi099.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.111300.log.gz 2023-10-08T00:22:44.284 INFO:teuthology.orchestra.run.smithi099.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.109291.log.gz 2023-10-08T00:22:44.284 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.113441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118451.log 2023-10-08T00:22:44.284 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.106680.log: gzip -5 89.3%--verbose -- -- replaced with /var/log/ceph/ceph-client.admin.113441.log.gz/var/log/ceph/ceph-client.admin.107315.log 2023-10-08T00:22:44.284 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-10-08T00:22:44.285 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.118451.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.106680.log.gz 2023-10-08T00:22:44.285 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115030.log 2023-10-08T00:22:44.285 INFO:teuthology.orchestra.run.smithi099.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.118451.log.gz 2023-10-08T00:22:44.285 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.107315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99338.log 2023-10-08T00:22:44.285 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.115030.log: gzip 88.3%-5 -- replaced with /var/log/ceph/ceph-client.admin.107315.log.gz--verbose 2023-10-08T00:22:44.285 INFO:teuthology.orchestra.run.smithi099.stderr:-- /var/log/ceph/ceph-client.admin.100697.log 2023-10-08T00:22:44.285 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.99338.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.115030.log.gz 2023-10-08T00:22:44.285 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-08T00:22:44.285 INFO:teuthology.orchestra.run.smithi099.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.99338.log.gz 2023-10-08T00:22:44.285 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.100697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112656.log 2023-10-08T00:22:44.285 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph.audit.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100697.log.gz 2023-10-08T00:22:44.285 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101629.log 2023-10-08T00:22:44.286 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.112656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118325.log 2023-10-08T00:22:44.286 INFO:teuthology.orchestra.run.smithi099.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.112656.log.gz 2023-10-08T00:22:44.286 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.101629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114143.log 2023-10-08T00:22:44.286 INFO:teuthology.orchestra.run.smithi099.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.101629.log.gz 2023-10-08T00:22:44.286 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.118325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95481.log 2023-10-08T00:22:44.286 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.114143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95691.log 2023-10-08T00:22:44.286 INFO:teuthology.orchestra.run.smithi099.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.118325.log.gz 2023-10-08T00:22:44.286 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.95481.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.114143.log.gz 2023-10-08T00:22:44.286 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97382.log 2023-10-08T00:22:44.286 INFO:teuthology.orchestra.run.smithi099.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.95481.log.gz 2023-10-08T00:22:44.286 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.95691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118283.log 2023-10-08T00:22:44.286 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89295.log 2023-10-08T00:22:44.287 INFO:teuthology.orchestra.run.smithi099.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.95691.log.gz 2023-10-08T00:22:44.287 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.97382.log: /var/log/ceph/ceph-client.admin.118283.log: gzip 88.6%-5 -- replaced with /var/log/ceph/ceph-client.admin.97382.log.gz--verbose 2023-10-08T00:22:44.287 INFO:teuthology.orchestra.run.smithi099.stderr:-- /var/log/ceph/ceph-client.admin.87006.log 2023-10-08T00:22:44.287 INFO:teuthology.orchestra.run.smithi099.stderr: 89.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-08T00:22:44.287 INFO:teuthology.orchestra.run.smithi099.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.118283.log.gz 2023-10-08T00:22:44.287 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88035.log 2023-10-08T00:22:44.287 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.89295.log: /var/log/ceph/ceph-client.admin.87006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115987.log 2023-10-08T00:22:44.287 INFO:teuthology.orchestra.run.smithi099.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.89295.log.gz 2023-10-08T00:22:44.287 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.88035.log: gzip -5 --verbose 89.2%-- /var/log/ceph/ceph-client.admin.88041.log -- replaced with /var/log/ceph/ceph-client.admin.87006.log.gz 2023-10-08T00:22:44.287 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-10-08T00:22:44.287 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.115987.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.88035.log.gzgzip 2023-10-08T00:22:44.287 INFO:teuthology.orchestra.run.smithi099.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.88033.log 2023-10-08T00:22:44.288 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.88041.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.115987.log.gz 2023-10-08T00:22:44.288 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93032.log 2023-10-08T00:22:44.288 INFO:teuthology.orchestra.run.smithi099.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.88041.log.gz 2023-10-08T00:22:44.288 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.88033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91314.log 2023-10-08T00:22:44.288 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.93032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107231.log 2023-10-08T00:22:44.288 INFO:teuthology.orchestra.run.smithi099.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.88033.log.gz 2023-10-08T00:22:44.288 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.91314.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.93032.log.gz 2023-10-08T00:22:44.288 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91632.log 2023-10-08T00:22:44.288 INFO:teuthology.orchestra.run.smithi099.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.91314.log.gz 2023-10-08T00:22:44.288 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.107231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103131.log 2023-10-08T00:22:44.288 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.91632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.108910.log 2023-10-08T00:22:44.289 INFO:teuthology.orchestra.run.smithi099.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.107231.log.gz 2023-10-08T00:22:44.289 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.103131.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.91632.log.gz 2023-10-08T00:22:44.289 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107594.log 2023-10-08T00:22:44.289 INFO:teuthology.orchestra.run.smithi099.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.103131.log.gz 2023-10-08T00:22:44.289 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.0.108910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89169.log 2023-10-08T00:22:44.289 INFO:teuthology.orchestra.run.smithi099.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.108910.log.gz 2023-10-08T00:22:44.289 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.107594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110436.log 2023-10-08T00:22:44.289 INFO:teuthology.orchestra.run.smithi099.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.107594.log.gz/var/log/ceph/ceph-client.admin.89169.log: 2023-10-08T00:22:44.289 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102733.log 2023-10-08T00:22:44.289 INFO:teuthology.orchestra.run.smithi099.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.89169.log.gz 2023-10-08T00:22:44.289 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.110436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89379.log 2023-10-08T00:22:44.289 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.102733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104245.log 2023-10-08T00:22:44.290 INFO:teuthology.orchestra.run.smithi099.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.110436.log.gz 2023-10-08T00:22:44.290 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.89379.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.102733.log.gz 2023-10-08T00:22:44.290 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102521.log 2023-10-08T00:22:44.290 INFO:teuthology.orchestra.run.smithi099.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.89379.log.gz 2023-10-08T00:22:44.290 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.104245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93608.log 2023-10-08T00:22:44.290 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.102521.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.104245.log.gz 2023-10-08T00:22:44.290 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98359.log 2023-10-08T00:22:44.290 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.93608.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.102521.log.gz 2023-10-08T00:22:44.290 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105242.log 2023-10-08T00:22:44.290 INFO:teuthology.orchestra.run.smithi099.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.93608.log.gz 2023-10-08T00:22:44.290 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.98359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95649.log 2023-10-08T00:22:44.291 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.105242.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.98359.log.gz 2023-10-08T00:22:44.291 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95733.log 2023-10-08T00:22:44.291 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.95649.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.105242.log.gz 2023-10-08T00:22:44.291 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-10-08T00:22:44.291 INFO:teuthology.orchestra.run.smithi099.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.95649.log.gz 2023-10-08T00:22:44.291 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.95733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114039.log 2023-10-08T00:22:44.291 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-mds.c.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.95733.log.gz 2023-10-08T00:22:44.291 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113525.log 2023-10-08T00:22:44.291 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.114039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105735.log 2023-10-08T00:22:44.291 INFO:teuthology.orchestra.run.smithi099.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.114039.log.gz 2023-10-08T00:22:44.291 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.113525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101881.log 2023-10-08T00:22:44.291 INFO:teuthology.orchestra.run.smithi099.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.113525.log.gz 2023-10-08T00:22:44.292 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95775.log 2023-10-08T00:22:44.292 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.105735.log: /var/log/ceph/ceph-client.admin.101881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95309.log 2023-10-08T00:22:44.292 INFO:teuthology.orchestra.run.smithi099.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.105735.log.gz 2023-10-08T00:22:44.292 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.95775.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.101881.log.gz 2023-10-08T00:22:44.292 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92422.log 2023-10-08T00:22:44.292 INFO:teuthology.orchestra.run.smithi099.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.95775.log.gz 2023-10-08T00:22:44.292 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.95309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105548.log 2023-10-08T00:22:44.292 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88600.log 2023-10-08T00:22:44.292 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.92422.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.95309.log.gz 2023-10-08T00:22:44.292 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.105548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118073.log 2023-10-08T00:22:44.292 INFO:teuthology.orchestra.run.smithi099.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.92422.log.gz 2023-10-08T00:22:44.293 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91002.log 2023-10-08T00:22:44.293 INFO:teuthology.orchestra.run.smithi099.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.105548.log.gz 2023-10-08T00:22:44.293 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.88600.log: /var/log/ceph/ceph-client.admin.118073.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.84646.log 2023-10-08T00:22:44.293 INFO:teuthology.orchestra.run.smithi099.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.88600.log.gz 2023-10-08T00:22:44.293 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.91002.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.118073.log.gz 2023-10-08T00:22:44.293 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108437.log 2023-10-08T00:22:44.293 INFO:teuthology.orchestra.run.smithi099.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.91002.log.gz 2023-10-08T00:22:44.293 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph.tmp-client.admin.84646.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.84646.log.gz 2023-10-08T00:22:44.293 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105200.log 2023-10-08T00:22:44.293 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105412.log 2023-10-08T00:22:44.293 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.108437.log: /var/log/ceph/ceph-client.admin.105200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112924.log 2023-10-08T00:22:44.293 INFO:teuthology.orchestra.run.smithi099.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.108437.log.gz 2023-10-08T00:22:44.294 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.105412.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.105200.log.gz 2023-10-08T00:22:44.294 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89043.log 2023-10-08T00:22:44.294 INFO:teuthology.orchestra.run.smithi099.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.105412.log.gz 2023-10-08T00:22:44.294 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.112924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91272.log 2023-10-08T00:22:44.294 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110912.log 2023-10-08T00:22:44.294 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.89043.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.112924.log.gz 2023-10-08T00:22:44.294 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.91272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115395.log 2023-10-08T00:22:44.294 INFO:teuthology.orchestra.run.smithi099.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.89043.log.gz 2023-10-08T00:22:44.294 INFO:teuthology.orchestra.run.smithi099.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.91272.log.gz 2023-10-08T00:22:44.294 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109745.log 2023-10-08T00:22:44.294 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.110912.log: /var/log/ceph/ceph-client.admin.115395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102025.log 2023-10-08T00:22:44.294 INFO:teuthology.orchestra.run.smithi099.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.110912.log.gz 2023-10-08T00:22:44.295 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.109745.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.115395.log.gz 2023-10-08T00:22:44.295 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88707.log 2023-10-08T00:22:44.295 INFO:teuthology.orchestra.run.smithi099.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.109745.log.gz 2023-10-08T00:22:44.295 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.102025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117989.log 2023-10-08T00:22:44.295 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117580.log 2023-10-08T00:22:44.295 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.88707.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.102025.log.gz 2023-10-08T00:22:44.295 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.117989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102605.log 87.6% 2023-10-08T00:22:44.295 INFO:teuthology.orchestra.run.smithi099.stderr: -- replaced with /var/log/ceph/ceph-client.admin.88707.log.gz 2023-10-08T00:22:44.295 INFO:teuthology.orchestra.run.smithi099.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.117989.log.gz 2023-10-08T00:22:44.295 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95523.log 2023-10-08T00:22:44.295 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.117580.log: /var/log/ceph/ceph-client.admin.102605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91376.log 2023-10-08T00:22:44.295 INFO:teuthology.orchestra.run.smithi099.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.117580.log.gz 2023-10-08T00:22:44.296 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.95523.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.102605.log.gz 2023-10-08T00:22:44.296 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.116292.log 2023-10-08T00:22:44.296 INFO:teuthology.orchestra.run.smithi099.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.95523.log.gz 2023-10-08T00:22:44.296 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.91376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116879.log 2023-10-08T00:22:44.296 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.106208.log 2023-10-08T00:22:44.296 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.0.116292.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.91376.log.gz 2023-10-08T00:22:44.296 INFO:teuthology.orchestra.run.smithi099.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.116292.log.gz 2023-10-08T00:22:44.296 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.116879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87321.log 2023-10-08T00:22:44.296 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114904.log 88.9% 2023-10-08T00:22:44.296 INFO:teuthology.orchestra.run.smithi099.stderr: -- replaced with /var/log/ceph/ceph-client.admin.116879.log.gz 2023-10-08T00:22:44.296 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.0.106208.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.106208.log.gz 2023-10-08T00:22:44.296 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.87321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108951.log 2023-10-08T00:22:44.297 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.114904.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.87321.log.gz 2023-10-08T00:22:44.297 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90750.log 2023-10-08T00:22:44.297 INFO:teuthology.orchestra.run.smithi099.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.114904.log.gz 2023-10-08T00:22:44.297 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.108951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97130.log 2023-10-08T00:22:44.297 INFO:teuthology.orchestra.run.smithi099.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.108951.log.gz 2023-10-08T00:22:44.297 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100255.log 2023-10-08T00:22:44.297 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.90750.log: /var/log/ceph/ceph-client.admin.97130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97172.log 2023-10-08T00:22:44.297 INFO:teuthology.orchestra.run.smithi099.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.90750.log.gz 2023-10-08T00:22:44.297 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.100255.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.97130.log.gz 2023-10-08T00:22:44.297 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95901.log 2023-10-08T00:22:44.297 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117286.log 2023-10-08T00:22:44.298 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.97172.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.100255.log.gz 2023-10-08T00:22:44.298 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.95901.log: gzip 88.7%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.97172.log.gz-- 2023-10-08T00:22:44.298 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.107901.log 2023-10-08T00:22:44.298 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90624.log 2023-10-08T00:22:44.298 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.117286.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.95901.log.gz 2023-10-08T00:22:44.298 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.107901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90792.log 2023-10-08T00:22:44.298 INFO:teuthology.orchestra.run.smithi099.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.117286.log.gz 2023-10-08T00:22:44.298 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104455.log 2023-10-08T00:22:44.298 INFO:teuthology.orchestra.run.smithi099.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.107901.log.gz 2023-10-08T00:22:44.298 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.90624.log: /var/log/ceph/ceph-client.admin.90792.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.90624.log.gz 2023-10-08T00:22:44.299 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96716.log 2023-10-08T00:22:44.299 INFO:teuthology.orchestra.run.smithi099.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.90792.log.gz 2023-10-08T00:22:44.299 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103047.log 2023-10-08T00:22:44.299 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.104455.log: /var/log/ceph/ceph-client.admin.96716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98737.log 2023-10-08T00:22:44.299 INFO:teuthology.orchestra.run.smithi099.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.104455.log.gz 2023-10-08T00:22:44.299 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.103047.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.96716.log.gz 2023-10-08T00:22:44.299 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112572.log 2023-10-08T00:22:44.299 INFO:teuthology.orchestra.run.smithi099.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.103047.log.gz 2023-10-08T00:22:44.299 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.98737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115089.log 2023-10-08T00:22:44.299 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118031.log 2023-10-08T00:22:44.299 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.112572.log: /var/log/ceph/ceph-client.admin.115089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87531.log 2023-10-08T00:22:44.300 INFO:teuthology.orchestra.run.smithi099.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.112572.log.gz 2023-10-08T00:22:44.300 INFO:teuthology.orchestra.run.smithi099.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.98737.log.gz 2023-10-08T00:22:44.300 INFO:teuthology.orchestra.run.smithi099.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.115089.log.gz 2023-10-08T00:22:44.300 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.118031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116376.log 2023-10-08T00:22:44.300 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96870.log 2023-10-08T00:22:44.300 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.87531.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.118031.log.gz 2023-10-08T00:22:44.300 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97298.log 2023-10-08T00:22:44.300 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.116376.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.87531.log.gz 2023-10-08T00:22:44.300 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.96870.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.116376.log.gz 2023-10-08T00:22:44.300 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115945.log 2023-10-08T00:22:44.300 INFO:teuthology.orchestra.run.smithi099.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.96870.log.gz 2023-10-08T00:22:44.300 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104329.log 2023-10-08T00:22:44.301 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.97298.log: /var/log/ceph/ceph-client.admin.115945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87237.log 2023-10-08T00:22:44.301 INFO:teuthology.orchestra.run.smithi099.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.97298.log.gz 2023-10-08T00:22:44.301 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.104329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110310.log 2023-10-08T00:22:44.301 INFO:teuthology.orchestra.run.smithi099.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.115945.log.gz 2023-10-08T00:22:44.301 INFO:teuthology.orchestra.run.smithi099.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.104329.log.gz 2023-10-08T00:22:44.301 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.87237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100173.log 2023-10-08T00:22:44.301 INFO:teuthology.orchestra.run.smithi099.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.87237.log.gz 2023-10-08T00:22:44.301 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112303.log 2023-10-08T00:22:44.301 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.110310.log: /var/log/ceph/ceph-client.admin.100173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101295.log 2023-10-08T00:22:44.301 INFO:teuthology.orchestra.run.smithi099.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.110310.log.gz 2023-10-08T00:22:44.301 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.112303.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.100173.log.gz 2023-10-08T00:22:44.301 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88370.log 2023-10-08T00:22:44.302 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105506.log 2023-10-08T00:22:44.302 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.101295.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.112303.log.gz 2023-10-08T00:22:44.302 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.88370.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.101295.log.gz 2023-10-08T00:22:44.302 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115229.log 2023-10-08T00:22:44.302 INFO:teuthology.orchestra.run.smithi099.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.88370.log.gz 2023-10-08T00:22:44.302 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90136.log 2023-10-08T00:22:44.302 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.105506.log: /var/log/ceph/ceph-client.admin.115229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104539.log 2023-10-08T00:22:44.302 INFO:teuthology.orchestra.run.smithi099.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.105506.log.gz 2023-10-08T00:22:44.302 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.90136.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.115229.log.gz 2023-10-08T00:22:44.302 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108605.log 2023-10-08T00:22:44.302 INFO:teuthology.orchestra.run.smithi099.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.90136.log.gz 2023-10-08T00:22:44.302 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.104539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94628.log 2023-10-08T00:22:44.303 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95225.log 2023-10-08T00:22:44.303 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.108605.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.104539.log.gz 2023-10-08T00:22:44.303 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.94628.log: gzip 88.9%-5 -- replaced with /var/log/ceph/ceph-client.admin.108605.log.gz--verbose 2023-10-08T00:22:44.303 INFO:teuthology.orchestra.run.smithi099.stderr:-- /var/log/ceph/ceph-client.admin.99041.log 2023-10-08T00:22:44.303 INFO:teuthology.orchestra.run.smithi099.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.94628.log.gz 2023-10-08T00:22:44.303 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112429.log 2023-10-08T00:22:44.303 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.95225.log: /var/log/ceph/ceph-client.admin.99041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98107.log 2023-10-08T00:22:44.303 INFO:teuthology.orchestra.run.smithi099.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.95225.log.gz 2023-10-08T00:22:44.303 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.112429.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.99041.log.gz 2023-10-08T00:22:44.303 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108113.log 2023-10-08T00:22:44.303 INFO:teuthology.orchestra.run.smithi099.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.112429.log.gz 2023-10-08T00:22:44.304 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.98107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104581.log 2023-10-08T00:22:44.304 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117076.log 2023-10-08T00:22:44.304 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.108113.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.98107.log.gz 2023-10-08T00:22:44.304 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.104581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105777.log 2023-10-08T00:22:44.304 INFO:teuthology.orchestra.run.smithi099.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.108113.log.gz 2023-10-08T00:22:44.304 INFO:teuthology.orchestra.run.smithi099.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.104581.log.gz 2023-10-08T00:22:44.304 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108069.log 2023-10-08T00:22:44.304 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.117076.log: /var/log/ceph/ceph-client.admin.105777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100297.log 2023-10-08T00:22:44.304 INFO:teuthology.orchestra.run.smithi099.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.117076.log.gz 2023-10-08T00:22:44.304 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.108069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108293.log 2023-10-08T00:22:44.304 INFO:teuthology.orchestra.run.smithi099.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.105777.log.gz 2023-10-08T00:22:44.304 INFO:teuthology.orchestra.run.smithi099.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.108069.log.gz/var/log/ceph/ceph-client.admin.100297.log: 2023-10-08T00:22:44.305 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96632.log 2023-10-08T00:22:44.305 INFO:teuthology.orchestra.run.smithi099.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.100297.log.gz 2023-10-08T00:22:44.305 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.90499.log 2023-10-08T00:22:44.305 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.108293.log: /var/log/ceph/ceph-client.admin.96632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110998.log 2023-10-08T00:22:44.305 INFO:teuthology.orchestra.run.smithi099.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.108293.log.gz 2023-10-08T00:22:44.305 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.2.90499.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.96632.log.gz 2023-10-08T00:22:44.305 INFO:teuthology.orchestra.run.smithi099.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.90499.log.gz 2023-10-08T00:22:44.305 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110352.log 2023-10-08T00:22:44.305 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.110998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110742.log 2023-10-08T00:22:44.305 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.110352.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.110998.log.gz 2023-10-08T00:22:44.305 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97550.log 2023-10-08T00:22:44.305 INFO:teuthology.orchestra.run.smithi099.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.110352.log.gz 2023-10-08T00:22:44.306 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.110742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91506.log 2023-10-08T00:22:44.306 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111258.log 2023-10-08T00:22:44.306 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.97550.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.110742.log.gz 2023-10-08T00:22:44.306 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.91506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117328.log 2023-10-08T00:22:44.306 INFO:teuthology.orchestra.run.smithi099.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.97550.log.gz 2023-10-08T00:22:44.306 INFO:teuthology.orchestra.run.smithi099.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.91506.log.gz 2023-10-08T00:22:44.306 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107985.log 2023-10-08T00:22:44.306 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.111258.log: /var/log/ceph/ceph-client.admin.117328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110956.log 2023-10-08T00:22:44.306 INFO:teuthology.orchestra.run.smithi099.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.111258.log.gz 2023-10-08T00:22:44.306 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.107985.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.117328.log.gz 2023-10-08T00:22:44.306 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117244.log 2023-10-08T00:22:44.307 INFO:teuthology.orchestra.run.smithi099.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.107985.log.gz 2023-10-08T00:22:44.307 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.110956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112261.log 2023-10-08T00:22:44.307 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97851.log 2023-10-08T00:22:44.307 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.117244.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.110956.log.gz 2023-10-08T00:22:44.307 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.112261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94334.log 2023-10-08T00:22:44.307 INFO:teuthology.orchestra.run.smithi099.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.117244.log.gz 2023-10-08T00:22:44.307 INFO:teuthology.orchestra.run.smithi099.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.112261.log.gz 2023-10-08T00:22:44.307 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107943.log 2023-10-08T00:22:44.307 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.97851.log: /var/log/ceph/ceph-client.admin.94334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99422.log 2023-10-08T00:22:44.307 INFO:teuthology.orchestra.run.smithi099.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.97851.log.gz 2023-10-08T00:22:44.307 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.107943.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.94334.log.gz 2023-10-08T00:22:44.307 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106517.log 2023-10-08T00:22:44.308 INFO:teuthology.orchestra.run.smithi099.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.107943.log.gz 2023-10-08T00:22:44.308 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.99422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113133.log 2023-10-08T00:22:44.308 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-10-08T00:22:44.308 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.106517.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.99422.log.gz 2023-10-08T00:22:44.308 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.113133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111814.log 2023-10-08T00:22:44.308 INFO:teuthology.orchestra.run.smithi099.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.106517.log.gz 2023-10-08T00:22:44.308 INFO:teuthology.orchestra.run.smithi099.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.113133.log.gz 2023-10-08T00:22:44.308 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98401.log 2023-10-08T00:22:44.308 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-osd.3.log: /var/log/ceph/ceph-client.admin.111814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108249.log 2023-10-08T00:22:44.308 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.98401.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.111814.log.gz 2023-10-08T00:22:44.309 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102827.log 2023-10-08T00:22:44.309 INFO:teuthology.orchestra.run.smithi099.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.98401.log.gz 2023-10-08T00:22:44.309 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.108249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107105.log 2023-10-08T00:22:44.309 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-10-08T00:22:44.309 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.102827.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.108249.log.gz 2023-10-08T00:22:44.309 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.107105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107740.log 2023-10-08T00:22:44.309 INFO:teuthology.orchestra.run.smithi099.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.102827.log.gz 2023-10-08T00:22:44.309 INFO:teuthology.orchestra.run.smithi099.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.107105.log.gz 2023-10-08T00:22:44.309 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.101254.log 2023-10-08T00:22:44.309 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.107740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88599.log 2023-10-08T00:22:44.310 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.0.101254.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.107740.log.gz 2023-10-08T00:22:44.310 INFO:teuthology.orchestra.run.smithi099.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.101254.log.gz 2023-10-08T00:22:44.310 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104287.log 2023-10-08T00:22:44.310 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.88599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91884.log 2023-10-08T00:22:44.310 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.104287.log: 88.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.88599.log.gz-5 2023-10-08T00:22:44.310 INFO:teuthology.orchestra.run.smithi099.stderr:--verbose -- /var/log/ceph/ceph-client.admin.103562.log 2023-10-08T00:22:44.310 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.91884.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.104287.log.gz 2023-10-08T00:22:44.310 INFO:teuthology.orchestra.run.smithi099.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.91884.log.gz 2023-10-08T00:22:44.310 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.103562.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.103562.log.gz 2023-10-08T00:22:44.311 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102869.log 2023-10-08T00:22:44.311 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88040.log 2023-10-08T00:22:44.311 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.102869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109055.log 2023-10-08T00:22:44.311 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.88040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92010.log 2023-10-08T00:22:44.311 INFO:teuthology.orchestra.run.smithi099.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.102869.log.gz 2023-10-08T00:22:44.311 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.109055.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.88040.log.gz 2023-10-08T00:22:44.311 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109469.log 2023-10-08T00:22:44.311 INFO:teuthology.orchestra.run.smithi099.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.109055.log.gz 2023-10-08T00:22:44.311 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.92010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117160.log 2023-10-08T00:22:44.311 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.106204.log 2023-10-08T00:22:44.311 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.109469.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.92010.log.gz 2023-10-08T00:22:44.311 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.117160.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.109469.log.gz 2023-10-08T00:22:44.312 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.0.106204.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.117160.log.gz 2023-10-08T00:22:44.312 INFO:teuthology.orchestra.run.smithi099.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.106204.log.gz 2023-10-08T00:22:44.312 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108713.log 2023-10-08T00:22:44.312 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91800.log 2023-10-08T00:22:44.312 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.108713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104029.log 2023-10-08T00:22:44.312 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.91800.log: gzip -5 --verbose -- 87.8%/var/log/ceph/ceph-client.admin.93729.log 2023-10-08T00:22:44.312 INFO:teuthology.orchestra.run.smithi099.stderr: -- replaced with /var/log/ceph/ceph-client.admin.108713.log.gz 2023-10-08T00:22:44.312 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.104029.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.91800.log.gz 2023-10-08T00:22:44.312 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93813.log 2023-10-08T00:22:44.312 INFO:teuthology.orchestra.run.smithi099.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.104029.log.gz 2023-10-08T00:22:44.312 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.93729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114717.log 2023-10-08T00:22:44.312 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96912.log 2023-10-08T00:22:44.313 INFO:teuthology.orchestra.run.smithi099.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.93729.log.gz 2023-10-08T00:22:44.313 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.93813.log: /var/log/ceph/ceph-client.admin.114717.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.93813.log.gz 2023-10-08T00:22:44.313 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.96912.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.114717.log.gz 2023-10-08T00:22:44.313 INFO:teuthology.orchestra.run.smithi099.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.96912.log.gz 2023-10-08T00:22:44.313 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113651.log 2023-10-08T00:22:44.313 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105861.log 2023-10-08T00:22:44.313 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.113651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106354.log 2023-10-08T00:22:44.313 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.105861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95439.log 2023-10-08T00:22:44.313 INFO:teuthology.orchestra.run.smithi099.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.113651.log.gz 2023-10-08T00:22:44.313 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.106354.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.105861.log.gz 2023-10-08T00:22:44.313 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100423.log 2023-10-08T00:22:44.313 INFO:teuthology.orchestra.run.smithi099.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.106354.log.gz 2023-10-08T00:22:44.314 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.95439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107189.log 2023-10-08T00:22:44.314 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94586.log 2023-10-08T00:22:44.314 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.100423.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.95439.log.gz 2023-10-08T00:22:44.314 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.107189.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100423.log.gz 2023-10-08T00:22:44.314 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.94586.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.107189.log.gz 2023-10-08T00:22:44.314 INFO:teuthology.orchestra.run.smithi099.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.94586.log.gz 2023-10-08T00:22:44.314 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.93459.log 2023-10-08T00:22:44.314 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88372.log 2023-10-08T00:22:44.314 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.0.93459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92676.log 2023-10-08T00:22:44.314 INFO:teuthology.orchestra.run.smithi099.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.93459.log.gz 2023-10-08T00:22:44.315 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.88372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104497.log 2023-10-08T00:22:44.315 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.92676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98443.log 2023-10-08T00:22:44.315 INFO:teuthology.orchestra.run.smithi099.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.88372.log.gz 2023-10-08T00:22:44.315 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.104497.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.92676.log.gz 2023-10-08T00:22:44.315 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106292.log 2023-10-08T00:22:44.315 INFO:teuthology.orchestra.run.smithi099.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.104497.log.gz 2023-10-08T00:22:44.315 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.98443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112488.log 2023-10-08T00:22:44.315 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98485.log 2023-10-08T00:22:44.315 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.106292.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.98443.log.gz 2023-10-08T00:22:44.315 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.112488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96205.log 2023-10-08T00:22:44.315 INFO:teuthology.orchestra.run.smithi099.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.106292.log.gz 2023-10-08T00:22:44.316 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118577.log 2023-10-08T00:22:44.316 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.98485.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.112488.log.gz 2023-10-08T00:22:44.316 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.96205.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.98485.log.gz 2023-10-08T00:22:44.316 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.118577.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.96205.log.gz 2023-10-08T00:22:44.316 INFO:teuthology.orchestra.run.smithi099.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.118577.log.gz 2023-10-08T00:22:44.316 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100739.log 2023-10-08T00:22:44.316 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-10-08T00:22:44.316 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.100739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114819.log 2023-10-08T00:22:44.316 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110123.log 2023-10-08T00:22:44.316 INFO:teuthology.orchestra.run.smithi099.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.100739.log.gz 2023-10-08T00:22:44.316 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.114819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94688.log 2023-10-08T00:22:44.316 INFO:teuthology.orchestra.run.smithi099.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.114819.log.gz 2023-10-08T00:22:44.317 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.110123.log: /var/log/ceph/ceph-client.admin.94688.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.110123.log.gz 2023-10-08T00:22:44.317 INFO:teuthology.orchestra.run.smithi099.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.94688.log.gz 2023-10-08T00:22:44.317 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111342.log 2023-10-08T00:22:44.317 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118846.log 2023-10-08T00:22:44.317 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.111342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110225.log 2023-10-08T00:22:44.317 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.118846.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.111342.log.gz 2023-10-08T00:22:44.317 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.110225.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.118846.log.gz 2023-10-08T00:22:44.317 INFO:teuthology.orchestra.run.smithi099.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.110225.log.gz 2023-10-08T00:22:44.317 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95267.log 2023-10-08T00:22:44.317 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96341.log 2023-10-08T00:22:44.317 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.95267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102194.log 2023-10-08T00:22:44.318 INFO:teuthology.orchestra.run.smithi099.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.95267.log.gz/var/log/ceph/ceph-client.admin.96341.log: 2023-10-08T00:22:44.318 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.102194.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.96341.log.gz 2023-10-08T00:22:44.318 INFO:teuthology.orchestra.run.smithi099.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.102194.log.gz 2023-10-08T00:22:44.318 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118888.log 2023-10-08T00:22:44.318 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107460.log 2023-10-08T00:22:44.318 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.118888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111040.log 2023-10-08T00:22:44.318 INFO:teuthology.orchestra.run.smithi099.stderr: 88.1%/var/log/ceph/ceph-client.admin.107460.log: -- replaced with /var/log/ceph/ceph-client.admin.118888.log.gz 2023-10-08T00:22:44.318 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.111040.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.107460.log.gz 2023-10-08T00:22:44.318 INFO:teuthology.orchestra.run.smithi099.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.111040.log.gz 2023-10-08T00:22:44.318 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96954.log 2023-10-08T00:22:44.319 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97612.log 2023-10-08T00:22:44.319 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.96954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87090.log 2023-10-08T00:22:44.319 INFO:teuthology.orchestra.run.smithi099.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.96954.log.gz 2023-10-08T00:22:44.319 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.97612.log: /var/log/ceph/ceph-client.admin.87090.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.97612.log.gz 2023-10-08T00:22:44.319 INFO:teuthology.orchestra.run.smithi099.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.87090.log.gz 2023-10-08T00:22:44.319 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116837.log 2023-10-08T00:22:44.319 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115735.log 2023-10-08T00:22:44.319 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.116837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90960.log 2023-10-08T00:22:44.319 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.115735.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.116837.log.gz 2023-10-08T00:22:44.319 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.90960.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.115735.log.gz 2023-10-08T00:22:44.319 INFO:teuthology.orchestra.run.smithi099.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.90960.log.gz 2023-10-08T00:22:44.319 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108993.log 2023-10-08T00:22:44.320 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99296.log 2023-10-08T00:22:44.320 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.108993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87489.log 2023-10-08T00:22:44.320 INFO:teuthology.orchestra.run.smithi099.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.108993.log.gz 2023-10-08T00:22:44.320 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.99296.log: /var/log/ceph/ceph-client.admin.87489.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.99296.log.gz 2023-10-08T00:22:44.320 INFO:teuthology.orchestra.run.smithi099.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.87489.log.gz 2023-10-08T00:22:44.320 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101797.log 2023-10-08T00:22:44.320 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104623.log 2023-10-08T00:22:44.320 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.101797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97674.log 2023-10-08T00:22:44.320 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.104623.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.101797.log.gz 2023-10-08T00:22:44.320 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.97674.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.104623.log.gz 2023-10-08T00:22:44.320 INFO:teuthology.orchestra.run.smithi099.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.97674.log.gz 2023-10-08T00:22:44.321 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102777.log 2023-10-08T00:22:44.321 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95607.log 2023-10-08T00:22:44.321 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.102777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108395.log 2023-10-08T00:22:44.321 INFO:teuthology.orchestra.run.smithi099.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.102777.log.gz 2023-10-08T00:22:44.321 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.95607.log: /var/log/ceph/ceph-client.admin.108395.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.95607.log.gz 2023-10-08T00:22:44.321 INFO:teuthology.orchestra.run.smithi099.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.108395.log.gz 2023-10-08T00:22:44.321 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101545.log 2023-10-08T00:22:44.321 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89337.log 2023-10-08T00:22:44.321 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.101545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104810.log 2023-10-08T00:22:44.321 INFO:teuthology.orchestra.run.smithi099.stderr: 88.0%/var/log/ceph/ceph-client.admin.89337.log: -- replaced with /var/log/ceph/ceph-client.admin.101545.log.gz 2023-10-08T00:22:44.321 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.104810.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.89337.log.gz 2023-10-08T00:22:44.322 INFO:teuthology.orchestra.run.smithi099.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.104810.log.gz 2023-10-08T00:22:44.322 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115311.log 2023-10-08T00:22:44.322 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114759.log 2023-10-08T00:22:44.322 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.115311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88710.log 2023-10-08T00:22:44.322 INFO:teuthology.orchestra.run.smithi099.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.115311.log.gz 2023-10-08T00:22:44.322 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.114759.log: /var/log/ceph/ceph-client.admin.88710.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.114759.log.gz 2023-10-08T00:22:44.322 INFO:teuthology.orchestra.run.smithi099.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.88710.log.gz 2023-10-08T00:22:44.322 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94544.log 2023-10-08T00:22:44.322 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86798.log 2023-10-08T00:22:44.322 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.94544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110268.log 2023-10-08T00:22:44.323 INFO:teuthology.orchestra.run.smithi099.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.94544.log.gz 2023-10-08T00:22:44.323 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.86798.log: /var/log/ceph/ceph-client.admin.110268.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.86798.log.gz 2023-10-08T00:22:44.323 INFO:teuthology.orchestra.run.smithi099.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110268.log.gz 2023-10-08T00:22:44.323 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.90095.log 2023-10-08T00:22:44.323 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105903.log 2023-10-08T00:22:44.323 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.0.90095.log: gzip -5 --verbose 0.0%-- /var/log/ceph/ceph-client.admin.86921.log -- replaced with /var/log/ceph/ceph-client.0.90095.log.gz 2023-10-08T00:22:44.323 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-10-08T00:22:44.323 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105590.log 2023-10-08T00:22:44.323 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.105903.log: /var/log/ceph/ceph-client.admin.86921.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.105903.log.gz 2023-10-08T00:22:44.323 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.105590.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.86921.log.gz 2023-10-08T00:22:44.323 INFO:teuthology.orchestra.run.smithi099.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.105590.log.gz 2023-10-08T00:22:44.324 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113483.log 2023-10-08T00:22:44.324 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109871.log 2023-10-08T00:22:44.324 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.113483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100653.log 2023-10-08T00:22:44.324 INFO:teuthology.orchestra.run.smithi099.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.113483.log.gz 2023-10-08T00:22:44.324 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.109871.log: /var/log/ceph/ceph-client.admin.100653.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.109871.log.gz 2023-10-08T00:22:44.324 INFO:teuthology.orchestra.run.smithi099.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.100653.log.gz 2023-10-08T00:22:44.324 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112093.log 2023-10-08T00:22:44.324 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101587.log 2023-10-08T00:22:44.324 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.112093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100381.log 2023-10-08T00:22:44.324 INFO:teuthology.orchestra.run.smithi099.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.112093.log.gz 2023-10-08T00:22:44.324 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.101587.log: /var/log/ceph/ceph-client.admin.100381.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.101587.log.gz 2023-10-08T00:22:44.324 INFO:teuthology.orchestra.run.smithi099.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.100381.log.gz 2023-10-08T00:22:44.325 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93546.log 2023-10-08T00:22:44.325 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99677.log 2023-10-08T00:22:44.325 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.93546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110478.log 2023-10-08T00:22:44.325 INFO:teuthology.orchestra.run.smithi099.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.93546.log.gz 2023-10-08T00:22:44.325 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.99677.log: /var/log/ceph/ceph-client.admin.110478.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.99677.log.gz 2023-10-08T00:22:44.325 INFO:teuthology.orchestra.run.smithi099.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.110478.log.gz 2023-10-08T00:22:44.325 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116095.log 2023-10-08T00:22:44.325 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98317.log 2023-10-08T00:22:44.325 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.116095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96674.log 2023-10-08T00:22:44.325 INFO:teuthology.orchestra.run.smithi099.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.116095.log.gz 2023-10-08T00:22:44.325 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.98317.log: /var/log/ceph/ceph-client.admin.96674.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.98317.log.gz 2023-10-08T00:22:44.325 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88704.log 2023-10-08T00:22:44.326 INFO:teuthology.orchestra.run.smithi099.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.96674.log.gz 2023-10-08T00:22:44.326 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114946.log 2023-10-08T00:22:44.326 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104071.log 2023-10-08T00:22:44.326 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.88704.log: /var/log/ceph/ceph-client.admin.114946.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.88704.log.gz 2023-10-08T00:22:44.326 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.104071.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.114946.log.gz 2023-10-08T00:22:44.326 INFO:teuthology.orchestra.run.smithi099.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.104071.log.gz 2023-10-08T00:22:44.326 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88042.log 2023-10-08T00:22:44.326 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110039.log 2023-10-08T00:22:44.326 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.88042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102152.log 2023-10-08T00:22:44.326 INFO:teuthology.orchestra.run.smithi099.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.88042.log.gz 2023-10-08T00:22:44.326 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.110039.log: /var/log/ceph/ceph-client.admin.102152.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.110039.log.gz 2023-10-08T00:22:44.326 INFO:teuthology.orchestra.run.smithi099.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.102152.log.gz 2023-10-08T00:22:44.327 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.99000.log 2023-10-08T00:22:44.327 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102068.log 2023-10-08T00:22:44.327 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.0.99000.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.99000.log.gzgzip 2023-10-08T00:22:44.327 INFO:teuthology.orchestra.run.smithi099.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.104768.log 2023-10-08T00:22:44.327 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98233.log 2023-10-08T00:22:44.327 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.102068.log: /var/log/ceph/ceph-client.admin.104768.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.102068.log.gz 2023-10-08T00:22:44.327 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.98233.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.104768.log.gz 2023-10-08T00:22:44.327 INFO:teuthology.orchestra.run.smithi099.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.98233.log.gz 2023-10-08T00:22:44.327 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116662.log 2023-10-08T00:22:44.327 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108647.log 2023-10-08T00:22:44.327 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.116662.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.116662.log.gz 2023-10-08T00:22:44.328 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.108647.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.108647.log.gz 2023-10-08T00:22:44.328 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109913.log 2023-10-08T00:22:44.328 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97752.log 2023-10-08T00:22:44.328 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.109913.log: /var/log/ceph/ceph-client.admin.97752.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.109913.log.gz 2023-10-08T00:22:44.328 INFO:teuthology.orchestra.run.smithi099.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.97752.log.gz 2023-10-08T00:22:44.328 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93074.log 2023-10-08T00:22:44.328 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111426.log 2023-10-08T00:22:44.328 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.93074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107021.log 2023-10-08T00:22:44.328 INFO:teuthology.orchestra.run.smithi099.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.93074.log.gz 2023-10-08T00:22:44.328 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.111426.log: /var/log/ceph/ceph-client.admin.107021.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.107021.log.gz 2023-10-08T00:22:44.328 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94961.log 2023-10-08T00:22:44.328 INFO:teuthology.orchestra.run.smithi099.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.111426.log.gz 2023-10-08T00:22:44.329 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93951.log 2023-10-08T00:22:44.329 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.94961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115437.log 2023-10-08T00:22:44.329 INFO:teuthology.orchestra.run.smithi099.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.94961.log.gz 2023-10-08T00:22:44.329 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.93951.log: /var/log/ceph/ceph-client.admin.115437.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.93951.log.gz 2023-10-08T00:22:44.329 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110165.log 2023-10-08T00:22:44.329 INFO:teuthology.orchestra.run.smithi099.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.115437.log.gz 2023-10-08T00:22:44.329 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110618.log 2023-10-08T00:22:44.329 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102236.log 2023-10-08T00:22:44.329 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.110165.log: /var/log/ceph/ceph-client.admin.110618.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.110165.log.gz 2023-10-08T00:22:44.329 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.102236.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110618.log.gz 2023-10-08T00:22:44.329 INFO:teuthology.orchestra.run.smithi099.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.102236.log.gz 2023-10-08T00:22:44.329 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118619.log 2023-10-08T00:22:44.330 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103173.log 2023-10-08T00:22:44.330 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.118619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107063.log 2023-10-08T00:22:44.330 INFO:teuthology.orchestra.run.smithi099.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.118619.log.gz 2023-10-08T00:22:44.330 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.103173.log: /var/log/ceph/ceph-client.admin.107063.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.103173.log.gz 2023-10-08T00:22:44.330 INFO:teuthology.orchestra.run.smithi099.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.107063.log.gz 2023-10-08T00:22:44.330 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101337.log 2023-10-08T00:22:44.330 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117454.log 2023-10-08T00:22:44.330 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.101337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87363.log 2023-10-08T00:22:44.330 INFO:teuthology.orchestra.run.smithi099.stderr: 88.3%/var/log/ceph/ceph-client.admin.117454.log: -- replaced with /var/log/ceph/ceph-client.admin.101337.log.gz 2023-10-08T00:22:44.330 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.87363.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.117454.log.gz 2023-10-08T00:22:44.330 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96996.log 2023-10-08T00:22:44.331 INFO:teuthology.orchestra.run.smithi099.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.87363.log.gz 2023-10-08T00:22:44.331 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92720.log 2023-10-08T00:22:44.331 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.96996.log: /var/log/ceph/ceph-client.admin.92720.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.96996.log.gz 2023-10-08T00:22:44.331 INFO:teuthology.orchestra.run.smithi099.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.92720.log.gz 2023-10-08T00:22:44.331 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86964.log 2023-10-08T00:22:44.331 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107678.log 2023-10-08T00:22:44.331 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.86964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107502.log 2023-10-08T00:22:44.331 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.107678.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.86964.log.gz 2023-10-08T00:22:44.331 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.107502.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.107678.log.gz 2023-10-08T00:22:44.331 INFO:teuthology.orchestra.run.smithi099.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.107502.log.gz 2023-10-08T00:22:44.331 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117538.log 2023-10-08T00:22:44.331 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117706.log 2023-10-08T00:22:44.332 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.117538.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.117538.log.gz 2023-10-08T00:22:44.332 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.117706.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.117706.log.gz 2023-10-08T00:22:44.332 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102963.log 2023-10-08T00:22:44.332 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110394.log 2023-10-08T00:22:44.332 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.102963.log: /var/log/ceph/ceph-client.admin.110394.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.102963.log.gz 2023-10-08T00:22:44.332 INFO:teuthology.orchestra.run.smithi099.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.110394.log.gz 2023-10-08T00:22:44.332 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99971.log 2023-10-08T00:22:44.332 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97256.log 2023-10-08T00:22:44.332 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.99971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102110.log 2023-10-08T00:22:44.332 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.97256.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.99971.log.gz 2023-10-08T00:22:44.332 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.102110.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.97256.log.gz 2023-10-08T00:22:44.332 INFO:teuthology.orchestra.run.smithi099.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.102110.log.gz 2023-10-08T00:22:44.333 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116418.log 2023-10-08T00:22:44.333 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118409.log 2023-10-08T00:22:44.333 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.116418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91842.log 2023-10-08T00:22:44.333 INFO:teuthology.orchestra.run.smithi099.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.116418.log.gz 2023-10-08T00:22:44.333 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.118409.log: /var/log/ceph/ceph-client.admin.91842.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.118409.log.gz 2023-10-08T00:22:44.333 INFO:teuthology.orchestra.run.smithi099.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.91842.log.gz 2023-10-08T00:22:44.333 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89505.log 2023-10-08T00:22:44.333 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109570.log 2023-10-08T00:22:44.333 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.89505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107357.log 2023-10-08T00:22:44.333 INFO:teuthology.orchestra.run.smithi099.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.89505.log.gz 2023-10-08T00:22:44.333 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.109570.log: /var/log/ceph/ceph-client.admin.107357.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.109570.log.gz 2023-10-08T00:22:44.333 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113263.log 2023-10-08T00:22:44.334 INFO:teuthology.orchestra.run.smithi099.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.107357.log.gz 2023-10-08T00:22:44.334 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112219.log 2023-10-08T00:22:44.334 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96590.log 2023-10-08T00:22:44.334 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.113263.log: /var/log/ceph/ceph-client.admin.112219.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.113263.log.gz 2023-10-08T00:22:44.334 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.96590.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.112219.log.gz 2023-10-08T00:22:44.334 INFO:teuthology.orchestra.run.smithi099.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.96590.log.gz 2023-10-08T00:22:44.334 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112135.log 2023-10-08T00:22:44.334 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102689.log 2023-10-08T00:22:44.334 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.112135.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-10-08T00:22:44.334 INFO:teuthology.orchestra.run.smithi099.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.112135.log.gz 2023-10-08T00:22:44.334 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.102689.log: /var/log/ceph/ceph-osd.0.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.102689.log.gz 2023-10-08T00:22:44.335 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114633.log 2023-10-08T00:22:44.335 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91230.log 2023-10-08T00:22:44.335 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.114633.log: /var/log/ceph/ceph-client.admin.91230.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.114633.log.gz 2023-10-08T00:22:44.335 INFO:teuthology.orchestra.run.smithi099.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.91230.log.gz 2023-10-08T00:22:44.335 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92770.log 2023-10-08T00:22:44.335 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107818.log 2023-10-08T00:22:44.335 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.92770.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.92770.log.gz 2023-10-08T00:22:44.335 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.107818.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.107818.log.gz 2023-10-08T00:22:44.335 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107273.log 2023-10-08T00:22:44.335 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88709.log 2023-10-08T00:22:44.335 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.107273.log: /var/log/ceph/ceph-client.admin.88709.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.107273.log.gz 2023-10-08T00:22:44.336 INFO:teuthology.orchestra.run.smithi099.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.88709.log.gz 2023-10-08T00:22:44.336 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99719.log 2023-10-08T00:22:44.336 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89681.log 2023-10-08T00:22:44.336 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.99719.log: /var/log/ceph/ceph-client.admin.89681.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.99719.log.gz 2023-10-08T00:22:44.336 INFO:teuthology.orchestra.run.smithi099.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.89681.log.gz 2023-10-08T00:22:44.336 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95565.log 2023-10-08T00:22:44.336 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.116288.log 2023-10-08T00:22:44.336 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.95565.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.95565.log.gz 2023-10-08T00:22:44.336 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.0.116288.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.116288.log.gz 2023-10-08T00:22:44.336 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87447.log 2023-10-08T00:22:44.336 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91716.log 2023-10-08T00:22:44.336 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.87447.log: /var/log/ceph/ceph-client.admin.91716.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.87447.log.gz 2023-10-08T00:22:44.337 INFO:teuthology.orchestra.run.smithi099.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.91716.log.gz 2023-10-08T00:22:44.337 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87573.log 2023-10-08T00:22:44.337 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99548.log 2023-10-08T00:22:44.337 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.87573.log: /var/log/ceph/ceph-client.admin.99548.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.87573.log.gz 2023-10-08T00:22:44.337 INFO:teuthology.orchestra.run.smithi099.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.99548.log.gz 2023-10-08T00:22:44.337 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95101.log 2023-10-08T00:22:44.337 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90666.log 2023-10-08T00:22:44.337 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.95101.log: /var/log/ceph/ceph-client.admin.90666.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.95101.log.gz 2023-10-08T00:22:44.337 INFO:teuthology.orchestra.run.smithi099.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.90666.log.gz 2023-10-08T00:22:44.337 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104725.log 2023-10-08T00:22:44.337 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94773.log 2023-10-08T00:22:44.337 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.104725.log: /var/log/ceph/ceph-client.admin.94773.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.104725.log.gz 2023-10-08T00:22:44.338 INFO:teuthology.orchestra.run.smithi099.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.94773.log.gz 2023-10-08T00:22:44.338 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115903.log 2023-10-08T00:22:44.338 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97046.log 2023-10-08T00:22:44.338 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.115903.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.115903.log.gz 2023-10-08T00:22:44.338 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.97046.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.97046.log.gz 2023-10-08T00:22:44.338 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97424.log 2023-10-08T00:22:44.338 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.96160.log 2023-10-08T00:22:44.338 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.97424.log: /var/log/ceph/ceph-client.0.96160.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.97424.log.gz 2023-10-08T00:22:44.338 INFO:teuthology.orchestra.run.smithi099.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.96160.log.gz 2023-10-08T00:22:44.338 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105693.log 2023-10-08T00:22:44.338 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109787.log 2023-10-08T00:22:44.339 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.105693.log: /var/log/ceph/ceph-client.admin.109787.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.105693.log.gz 2023-10-08T00:22:44.339 INFO:teuthology.orchestra.run.smithi099.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.109787.log.gz 2023-10-08T00:22:44.339 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92211.log 2023-10-08T00:22:44.339 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104113.log 2023-10-08T00:22:44.339 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.92211.log: /var/log/ceph/ceph-client.admin.104113.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.92211.log.gz 2023-10-08T00:22:44.339 INFO:teuthology.orchestra.run.smithi099.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.104113.log.gz 2023-10-08T00:22:44.339 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113567.log 2023-10-08T00:22:44.339 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117863.log 2023-10-08T00:22:44.340 INFO:teuthology.orchestra.run.smithi191.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-10-08T00:22:44.363 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.113567.log: /var/log/ceph/ceph-client.admin.117863.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.113567.log.gz 2023-10-08T00:22:44.363 INFO:teuthology.orchestra.run.smithi099.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.117863.log.gz 2023-10-08T00:22:44.363 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95817.log 2023-10-08T00:22:44.363 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90708.log 2023-10-08T00:22:44.364 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.95817.log: /var/log/ceph/ceph-client.admin.90708.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.95817.log.gz 2023-10-08T00:22:44.364 INFO:teuthology.orchestra.run.smithi099.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.90708.log.gz 2023-10-08T00:22:44.364 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101671.log 2023-10-08T00:22:44.364 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109612.log 2023-10-08T00:22:44.364 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.101671.log: /var/log/ceph/ceph-client.admin.109612.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.101671.log.gz 2023-10-08T00:22:44.364 INFO:teuthology.orchestra.run.smithi099.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.109612.log.gz 2023-10-08T00:22:44.364 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87279.log 2023-10-08T00:22:44.364 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99845.log 2023-10-08T00:22:44.364 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.87279.log: /var/log/ceph/ceph-client.admin.99845.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.87279.log.gz 2023-10-08T00:22:44.364 INFO:teuthology.orchestra.run.smithi099.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.99845.log.gz 2023-10-08T00:22:44.364 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118806.log 2023-10-08T00:22:44.365 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100991.log 2023-10-08T00:22:44.365 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.118806.log: /var/log/ceph/ceph-client.admin.100991.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.118806.log.gz 2023-10-08T00:22:44.365 INFO:teuthology.orchestra.run.smithi099.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.100991.log.gz 2023-10-08T00:22:44.365 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113397.log 2023-10-08T00:22:44.365 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113091.log 2023-10-08T00:22:44.365 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.113397.log: /var/log/ceph/ceph-client.admin.113091.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.113397.log.gz 2023-10-08T00:22:44.365 INFO:teuthology.orchestra.run.smithi099.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.113091.log.gz 2023-10-08T00:22:44.365 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100865.log 2023-10-08T00:22:44.365 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100033.log 2023-10-08T00:22:44.365 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.100865.log: /var/log/ceph/ceph-client.admin.100033.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.100865.log.gz 2023-10-08T00:22:44.365 INFO:teuthology.orchestra.run.smithi099.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.100033.log.gz 2023-10-08T00:22:44.365 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88917.log 2023-10-08T00:22:44.366 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90540.log 2023-10-08T00:22:44.366 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.88917.log: /var/log/ceph/ceph-client.admin.90540.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.88917.log.gz 2023-10-08T00:22:44.366 INFO:teuthology.orchestra.run.smithi099.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.90540.log.gz 2023-10-08T00:22:44.366 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86705.log 2023-10-08T00:22:44.366 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.90087.log 2023-10-08T00:22:44.366 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.86705.log: /var/log/ceph/ceph-client.0.90087.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.86705.log.gz 2023-10-08T00:22:44.366 INFO:teuthology.orchestra.run.smithi099.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.90087.log.gz 2023-10-08T00:22:44.366 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87405.log 2023-10-08T00:22:44.366 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94208.log 2023-10-08T00:22:44.366 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.87405.log: /var/log/ceph/ceph-client.admin.94208.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.87405.log.gz 2023-10-08T00:22:44.366 INFO:teuthology.orchestra.run.smithi099.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.94208.log.gz 2023-10-08T00:22:44.367 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101713.log 2023-10-08T00:22:44.367 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114381.log 2023-10-08T00:22:44.367 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.101713.log: /var/log/ceph/ceph-client.admin.114381.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.101713.log.gz 2023-10-08T00:22:44.367 INFO:teuthology.orchestra.run.smithi099.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.114381.log.gz 2023-10-08T00:22:44.367 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92548.log 2023-10-08T00:22:44.367 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116778.log 2023-10-08T00:22:44.367 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.92548.log: /var/log/ceph/ceph-client.admin.116778.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.92548.log.gz 2023-10-08T00:22:44.367 INFO:teuthology.orchestra.run.smithi099.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.116778.log.gz 2023-10-08T00:22:44.367 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115479.log 2023-10-08T00:22:44.367 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-10-08T00:22:44.367 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.115479.log: /var/log/ceph/ceph-mgr.y.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.115479.log.gz 2023-10-08T00:22:44.368 INFO:teuthology.orchestra.run.smithi099.stderr: 94.4% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-10-08T00:22:44.368 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98695.log 2023-10-08T00:22:44.368 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101755.log 2023-10-08T00:22:44.368 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.98695.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.98695.log.gz 2023-10-08T00:22:44.368 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94815.log 2023-10-08T00:22:44.368 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.101755.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.101755.log.gz 2023-10-08T00:22:44.368 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.94815.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.94815.log.gz 2023-10-08T00:22:44.368 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117622.log 2023-10-08T00:22:44.368 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92812.log 2023-10-08T00:22:44.368 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.117622.log: /var/log/ceph/ceph-client.admin.92812.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.117622.log.gz 2023-10-08T00:22:44.368 INFO:teuthology.orchestra.run.smithi099.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.92812.log.gz 2023-10-08T00:22:44.368 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92590.log 2023-10-08T00:22:44.369 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103089.log 2023-10-08T00:22:44.369 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.92590.log: /var/log/ceph/ceph-client.admin.103089.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.92590.log.gz 2023-10-08T00:22:44.369 INFO:teuthology.orchestra.run.smithi099.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.103089.log.gz 2023-10-08T00:22:44.369 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112614.log 2023-10-08T00:22:44.369 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87176.log 2023-10-08T00:22:44.369 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.112614.log: /var/log/ceph/ceph-client.admin.87176.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.112614.log.gz 2023-10-08T00:22:44.369 INFO:teuthology.orchestra.run.smithi099.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.87176.log.gz 2023-10-08T00:22:44.369 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86610.log 2023-10-08T00:22:44.369 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109427.log 2023-10-08T00:22:44.369 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.86610.log: /var/log/ceph/ceph-client.admin.109427.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.86610.log.gz 2023-10-08T00:22:44.369 INFO:teuthology.orchestra.run.smithi099.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.109427.log.gz 2023-10-08T00:22:44.369 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117905.log 2023-10-08T00:22:44.370 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102563.log 2023-10-08T00:22:44.370 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.117905.log: 89.0%/var/log/ceph/ceph-client.admin.102563.log: -- replaced with /var/log/ceph/ceph-client.admin.117905.log.gz 2023-10-08T00:22:44.370 INFO:teuthology.orchestra.run.smithi099.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.102563.log.gz 2023-10-08T00:22:44.370 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106250.log 2023-10-08T00:22:44.370 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93158.log 2023-10-08T00:22:44.370 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.106250.log: /var/log/ceph/ceph-client.admin.93158.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.106250.log.gz 2023-10-08T00:22:44.370 INFO:teuthology.orchestra.run.smithi099.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.93158.log.gz 2023-10-08T00:22:44.370 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118745.log 2023-10-08T00:22:44.370 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110081.log 2023-10-08T00:22:44.370 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.118745.log: /var/log/ceph/ceph-client.admin.110081.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.118745.log.gz 2023-10-08T00:22:44.370 INFO:teuthology.orchestra.run.smithi099.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.110081.log.gz 2023-10-08T00:22:44.370 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89463.log 2023-10-08T00:22:44.371 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95351.log 2023-10-08T00:22:44.371 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.89463.log: /var/log/ceph/ceph-client.admin.95351.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.89463.log.gz 2023-10-08T00:22:44.371 INFO:teuthology.orchestra.run.smithi099.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.95351.log.gz 2023-10-08T00:22:44.371 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95967.log 2023-10-08T00:22:44.371 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89085.log 2023-10-08T00:22:44.371 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.95967.log: 87.7%/var/log/ceph/ceph-client.admin.89085.log: -- replaced with /var/log/ceph/ceph-client.admin.95967.log.gz 2023-10-08T00:22:44.371 INFO:teuthology.orchestra.run.smithi099.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.89085.log.gz 2023-10-08T00:22:44.371 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93504.log 2023-10-08T00:22:44.371 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117496.log 2023-10-08T00:22:44.371 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.93504.log: /var/log/ceph/ceph-client.admin.117496.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.93504.log.gz 2023-10-08T00:22:44.372 INFO:teuthology.orchestra.run.smithi099.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.117496.log.gz 2023-10-08T00:22:44.372 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103604.log 2023-10-08T00:22:44.372 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115777.log 2023-10-08T00:22:44.372 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.103604.log: /var/log/ceph/ceph-client.admin.115777.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.103604.log.gz 2023-10-08T00:22:44.372 INFO:teuthology.orchestra.run.smithi099.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.115777.log.gz 2023-10-08T00:22:44.372 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118199.log 2023-10-08T00:22:44.372 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.118199.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.118199.log.gz 2023-10-08T00:22:44.372 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88038.log 2023-10-08T00:22:44.372 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118115.log 2023-10-08T00:22:44.372 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.88038.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.88038.log.gz 2023-10-08T00:22:44.372 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90918.log 2023-10-08T00:22:44.372 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.118115.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.118115.log.gz/var/log/ceph/ceph-client.admin.90918.log: 2023-10-08T00:22:44.373 INFO:teuthology.orchestra.run.smithi099.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.90918.log.gz 2023-10-08T00:22:44.373 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105819.log 2023-10-08T00:22:44.373 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112177.log 2023-10-08T00:22:44.373 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.105819.log: /var/log/ceph/ceph-client.admin.112177.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.105819.log.gz 2023-10-08T00:22:44.373 INFO:teuthology.orchestra.run.smithi099.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.112177.log.gz 2023-10-08T00:22:44.373 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116604.log 2023-10-08T00:22:44.373 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117118.log 2023-10-08T00:22:44.373 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.116604.log: /var/log/ceph/ceph-client.admin.117118.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.116604.log.gz 2023-10-08T00:22:44.373 INFO:teuthology.orchestra.run.smithi099.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.117118.log.gz 2023-10-08T00:22:44.373 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118703.log 2023-10-08T00:22:44.373 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87699.log 2023-10-08T00:22:44.373 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.118703.log: /var/log/ceph/ceph-client.admin.87699.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.118703.log.gz 2023-10-08T00:22:44.374 INFO:teuthology.orchestra.run.smithi099.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.87699.log.gz 2023-10-08T00:22:44.374 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97508.log 2023-10-08T00:22:44.374 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103718.log 2023-10-08T00:22:44.374 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.97508.log: /var/log/ceph/ceph-client.admin.103718.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.97508.log.gz 2023-10-08T00:22:44.374 INFO:teuthology.orchestra.run.smithi099.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.103718.log.gz 2023-10-08T00:22:44.374 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.108906.log 2023-10-08T00:22:44.374 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109333.log 2023-10-08T00:22:44.374 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.0.108906.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.108906.log.gz 2023-10-08T00:22:44.374 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.109333.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.109333.log.gz 2023-10-08T00:22:44.374 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100095.log 2023-10-08T00:22:44.374 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91548.log 2023-10-08T00:22:44.375 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.100095.log: /var/log/ceph/ceph-client.admin.91548.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.100095.log.gz 2023-10-08T00:22:44.375 INFO:teuthology.orchestra.run.smithi099.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.91548.log.gz 2023-10-08T00:22:44.375 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96247.log 2023-10-08T00:22:44.375 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87783.log 2023-10-08T00:22:44.375 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.96247.log: /var/log/ceph/ceph-client.admin.87783.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.96247.log.gz 2023-10-08T00:22:44.375 INFO:teuthology.orchestra.run.smithi099.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.87783.log.gz 2023-10-08T00:22:44.375 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.113994.log 2023-10-08T00:22:44.375 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111082.log 2023-10-08T00:22:44.375 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.0.113994.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.113994.log.gz 2023-10-08T00:22:44.375 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.111082.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.111082.log.gz 2023-10-08T00:22:44.375 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88374.log 2023-10-08T00:22:44.375 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103877.log 2023-10-08T00:22:44.376 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.88374.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.88374.log.gz 2023-10-08T00:22:44.376 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115523.log 2023-10-08T00:22:44.376 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.103877.log: /var/log/ceph/ceph-client.admin.115523.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.103877.log.gz 2023-10-08T00:22:44.376 INFO:teuthology.orchestra.run.smithi099.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.115523.log.gz 2023-10-08T00:22:44.376 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.103520.log 2023-10-08T00:22:44.376 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105368.log 2023-10-08T00:22:44.376 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.0.103520.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.103520.log.gz 2023-10-08T00:22:44.376 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.105368.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.105368.log.gz 2023-10-08T00:22:44.376 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108027.log 2023-10-08T00:22:44.376 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102647.log 2023-10-08T00:22:44.376 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.108027.log: /var/log/ceph/ceph-client.admin.102647.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.108027.log.gz 2023-10-08T00:22:44.376 INFO:teuthology.orchestra.run.smithi099.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.102647.log.gz 2023-10-08T00:22:44.377 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112782.log 2023-10-08T00:22:44.377 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114591.log 2023-10-08T00:22:44.377 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.112782.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.112782.log.gz 2023-10-08T00:22:44.377 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86364.log 2023-10-08T00:22:44.377 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.114591.log: /var/log/ceph/ceph-client.admin.86364.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.114591.log.gz 2023-10-08T00:22:44.377 INFO:teuthology.orchestra.run.smithi099.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.86364.log.gz 2023-10-08T00:22:44.377 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114549.log 2023-10-08T00:22:44.377 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113007.log 2023-10-08T00:22:44.377 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.114549.log: /var/log/ceph/ceph-client.admin.113007.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.114549.log.gz 2023-10-08T00:22:44.377 INFO:teuthology.orchestra.run.smithi099.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.113007.log.gz 2023-10-08T00:22:44.377 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108157.log 2023-10-08T00:22:44.377 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91044.log 2023-10-08T00:22:44.377 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.108157.log: /var/log/ceph/ceph-client.admin.91044.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.108157.log.gz 2023-10-08T00:22:44.378 INFO:teuthology.orchestra.run.smithi099.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.91044.log.gz 2023-10-08T00:22:44.378 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88378.log 2023-10-08T00:22:44.378 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105651.log 2023-10-08T00:22:44.378 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.88378.log: /var/log/ceph/ceph-client.admin.105651.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.88378.log.gz 2023-10-08T00:22:44.378 INFO:teuthology.orchestra.run.smithi099.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.105651.log.gz 2023-10-08T00:22:44.378 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107147.log 2023-10-08T00:22:44.378 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118367.log 2023-10-08T00:22:44.378 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.107147.log: /var/log/ceph/ceph-client.admin.118367.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.107147.log.gz 2023-10-08T00:22:44.378 INFO:teuthology.orchestra.run.smithi099.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.118367.log.gz 2023-10-08T00:22:44.378 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103777.log 2023-10-08T00:22:44.378 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89211.log 2023-10-08T00:22:44.379 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.103777.log: /var/log/ceph/ceph-client.admin.89211.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.103777.log.gz 2023-10-08T00:22:44.379 INFO:teuthology.orchestra.run.smithi099.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.89211.log.gz 2023-10-08T00:22:44.379 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92854.log 2023-10-08T00:22:44.379 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90834.log 2023-10-08T00:22:44.379 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.92854.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.92854.log.gz 2023-10-08T00:22:44.379 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.90834.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.90834.log.gz 2023-10-08T00:22:44.379 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100569.log 2023-10-08T00:22:44.379 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105284.log 2023-10-08T00:22:44.379 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.100569.log: /var/log/ceph/ceph-client.admin.105284.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.100569.log.gz 2023-10-08T00:22:44.379 INFO:teuthology.orchestra.run.smithi099.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.105284.log.gz 2023-10-08T00:22:44.379 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97977.log 2023-10-08T00:22:44.379 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107636.log 2023-10-08T00:22:44.380 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.97977.log: /var/log/ceph/ceph-client.admin.107636.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.97977.log.gz 2023-10-08T00:22:44.380 INFO:teuthology.orchestra.run.smithi099.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.107636.log.gz 2023-10-08T00:22:44.380 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108521.log 2023-10-08T00:22:44.380 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102361.log 2023-10-08T00:22:44.380 INFO:teuthology.orchestra.run.smithi099.stderr: 92.4%/var/log/ceph/ceph-client.admin.108521.log: -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-10-08T00:22:44.380 INFO:teuthology.orchestra.run.smithi099.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.108521.log.gz 2023-10-08T00:22:44.380 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.102361.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.102361.log.gz 2023-10-08T00:22:44.380 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109149.log 2023-10-08T00:22:44.380 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100339.log 2023-10-08T00:22:44.380 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.109149.log: /var/log/ceph/ceph-client.admin.100339.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.109149.log.gz 2023-10-08T00:22:44.380 INFO:teuthology.orchestra.run.smithi099.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.100339.log.gz 2023-10-08T00:22:44.380 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113801.log 2023-10-08T00:22:44.381 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91590.log 2023-10-08T00:22:44.381 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.113801.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-10-08T00:22:44.381 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.91590.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.113801.log.gz 2023-10-08T00:22:44.381 INFO:teuthology.orchestra.run.smithi099.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.91590.log.gz 2023-10-08T00:22:44.381 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-osd.2.log: 94.8% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-10-08T00:22:44.381 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89547.log 2023-10-08T00:22:44.381 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118535.log 2023-10-08T00:22:44.381 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.89547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109955.log 2023-10-08T00:22:44.381 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.118535.log: /var/log/ceph/ceph-client.admin.109955.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.118535.log.gz 2023-10-08T00:22:44.381 INFO:teuthology.orchestra.run.smithi099.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.89547.log.gz 2023-10-08T00:22:44.381 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104852.log 2023-10-08T00:22:44.381 INFO:teuthology.orchestra.run.smithi099.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.109955.log.gz 2023-10-08T00:22:44.382 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118241.log 2023-10-08T00:22:44.382 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.104852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114988.log 2023-10-08T00:22:44.382 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.118241.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.104852.log.gz 2023-10-08T00:22:44.382 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.114988.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.118241.log.gz 2023-10-08T00:22:44.382 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116334.log 2023-10-08T00:22:44.382 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116512.log 2023-10-08T00:22:44.382 INFO:teuthology.orchestra.run.smithi099.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.114988.log.gz 2023-10-08T00:22:44.382 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.116334.log: /var/log/ceph/ceph-client.admin.116512.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.116334.log.gz 2023-10-08T00:22:44.382 INFO:teuthology.orchestra.run.smithi099.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.116512.log.gz 2023-10-08T00:22:44.393 INFO:teuthology.orchestra.run.smithi191.stderr: 92.3% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-10-08T00:22:44.487 INFO:teuthology.orchestra.run.smithi191.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-10-08T00:22:44.519 INFO:teuthology.orchestra.run.smithi191.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-10-08T00:22:44.846 INFO:teuthology.orchestra.run.smithi099.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-10-08T00:22:44.859 INFO:teuthology.orchestra.run.smithi191.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-10-08T00:22:44.894 INFO:teuthology.orchestra.run.smithi099.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-10-08T00:22:44.906 INFO:teuthology.orchestra.run.smithi099.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-10-08T00:22:44.916 INFO:teuthology.orchestra.run.smithi191.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-10-08T00:22:44.920 INFO:teuthology.orchestra.run.smithi191.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-10-08T00:22:44.976 INFO:teuthology.orchestra.run.smithi191.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-10-08T00:22:44.977 INFO:teuthology.orchestra.run.smithi191.stderr: 2023-10-08T00:22:44.978 INFO:teuthology.orchestra.run.smithi191.stderr:real 0m0.883s 2023-10-08T00:22:44.978 INFO:teuthology.orchestra.run.smithi191.stderr:user 0m4.286s 2023-10-08T00:22:44.978 INFO:teuthology.orchestra.run.smithi191.stderr:sys 0m0.192s 2023-10-08T00:22:45.092 INFO:teuthology.orchestra.run.smithi099.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-10-08T00:22:45.186 INFO:teuthology.orchestra.run.smithi099.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-10-08T00:22:45.187 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-10-08T00:22:45.188 INFO:teuthology.orchestra.run.smithi099.stderr:real 0m1.121s 2023-10-08T00:22:45.188 INFO:teuthology.orchestra.run.smithi099.stderr:user 0m4.408s 2023-10-08T00:22:45.188 INFO:teuthology.orchestra.run.smithi099.stderr:sys 0m0.466s 2023-10-08T00:22:45.188 INFO:tasks.ceph:Archiving logs... 2023-10-08T00:22:45.188 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418110/remote/smithi099/log 2023-10-08T00:22:45.189 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-08T00:22:46.286 DEBUG:teuthology.misc:Transferring archived files from smithi191:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418110/remote/smithi191/log 2023-10-08T00:22:46.287 DEBUG:teuthology.orchestra.run.smithi191:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-08T00:22:46.921 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-08T00:22:46.931 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-08T00:22:46.931 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-08T00:22:46.933 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-08T00:22:47.020 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-08T00:22:47.030 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-08T00:22:47.030 DEBUG:teuthology.orchestra.run.smithi099:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-08T00:22:47.033 DEBUG:teuthology.orchestra.run.smithi191:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-08T00:22:47.045 INFO:teuthology.orchestra.run.smithi099.stderr:bash: ntpq: command not found 2023-10-08T00:22:47.048 INFO:teuthology.orchestra.run.smithi191.stderr:bash: ntpq: command not found 2023-10-08T00:22:47.049 INFO:teuthology.orchestra.run.smithi099.stdout:210 Number of sources = 4 2023-10-08T00:22:47.049 INFO:teuthology.orchestra.run.smithi099.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-08T00:22:47.049 INFO:teuthology.orchestra.run.smithi099.stdout:=============================================================================== 2023-10-08T00:22:47.049 INFO:teuthology.orchestra.run.smithi099.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 22 -150us[ -150us] +/- 77ms 2023-10-08T00:22:47.049 INFO:teuthology.orchestra.run.smithi099.stdout:^* hv02.front.sepia.ceph.com 3 6 377 27 -238us[ -231us] +/- 56ms 2023-10-08T00:22:47.049 INFO:teuthology.orchestra.run.smithi099.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 25 +451us[ +451us] +/- 71ms 2023-10-08T00:22:47.049 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T00:22:47.051 INFO:teuthology.orchestra.run.smithi191.stdout:210 Number of sources = 4 2023-10-08T00:22:47.051 INFO:teuthology.orchestra.run.smithi191.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-08T00:22:47.051 INFO:teuthology.orchestra.run.smithi191.stdout:=============================================================================== 2023-10-08T00:22:47.051 INFO:teuthology.orchestra.run.smithi191.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 24 -139us[ -139us] +/- 77ms 2023-10-08T00:22:47.051 INFO:teuthology.orchestra.run.smithi191.stdout:^* hv02.front.sepia.ceph.com 3 6 377 27 -245us[ -236us] +/- 56ms 2023-10-08T00:22:47.051 INFO:teuthology.orchestra.run.smithi191.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 26 +442us[ +442us] +/- 71ms 2023-10-08T00:22:47.051 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T00:22:47.052 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-08T00:22:47.061 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-08T00:22:47.062 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-08T00:22:47.078 DEBUG:teuthology.orchestra.run.smithi099:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-08T00:22:47.200 DEBUG:teuthology.orchestra.run.smithi191:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-08T00:22:47.302 DEBUG:teuthology.orchestra.run.smithi099:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-08T00:22:47.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T00:22:47.333 DEBUG:teuthology.orchestra.run.smithi191:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-08T00:22:47.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T00:22:47.365 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-08T00:22:47.377 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-08T00:22:47.462 INFO:teuthology.task.internal:Duration was 1365.526755 seconds 2023-10-08T00:22:47.462 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-08T00:22:47.490 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-08T00:22:47.490 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-08T00:22:47.493 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-08T00:22:47.534 INFO:teuthology.orchestra.run.smithi099.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-08T00:22:47.541 INFO:teuthology.orchestra.run.smithi191.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-08T00:22:47.917 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-08T00:22:47.917 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi099.front.sepia.ceph.com 2023-10-08T00:22:47.917 DEBUG:teuthology.orchestra.run.smithi099:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-08T00:22:47.937 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi191.front.sepia.ceph.com 2023-10-08T00:22:47.938 DEBUG:teuthology.orchestra.run.smithi191:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-08T00:22:47.958 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-08T00:22:47.958 DEBUG:teuthology.orchestra.run.smithi099:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-08T00:22:47.980 DEBUG:teuthology.orchestra.run.smithi191:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-08T00:22:48.040 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-08T00:22:48.040 DEBUG:teuthology.orchestra.run.smithi099:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-08T00:22:48.064 DEBUG:teuthology.orchestra.run.smithi191:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-08T00:22:48.156 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-08T00:22:48.167 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-08T00:22:48.167 DEBUG:teuthology.orchestra.run.smithi099:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-08T00:22:48.191 DEBUG:teuthology.orchestra.run.smithi191:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-08T00:22:48.216 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-08T00:22:48.226 DEBUG:teuthology.orchestra.run.smithi099:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-08T00:22:48.233 DEBUG:teuthology.orchestra.run.smithi191:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-08T00:22:48.254 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern = core 2023-10-08T00:22:48.281 INFO:teuthology.orchestra.run.smithi191.stdout:kernel.core_pattern = core 2023-10-08T00:22:48.295 DEBUG:teuthology.orchestra.run.smithi099:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-08T00:22:48.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T00:22:48.323 DEBUG:teuthology.orchestra.run.smithi191:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-08T00:22:48.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T00:22:48.349 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-08T00:22:48.359 INFO:teuthology.task.internal:Transferring archived files... 2023-10-08T00:22:48.360 DEBUG:teuthology.misc:Transferring archived files from smithi099:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418110/remote/smithi099 2023-10-08T00:22:48.361 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-08T00:22:48.401 DEBUG:teuthology.misc:Transferring archived files from smithi191:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418110/remote/smithi191 2023-10-08T00:22:48.401 DEBUG:teuthology.orchestra.run.smithi191:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-08T00:22:48.436 INFO:teuthology.task.internal:Removing archive directory... 2023-10-08T00:22:48.436 DEBUG:teuthology.orchestra.run.smithi099:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-08T00:22:48.438 DEBUG:teuthology.orchestra.run.smithi191:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-08T00:22:48.484 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-08T00:22:48.496 INFO:teuthology.task.internal:Not uploading archives. 2023-10-08T00:22:48.497 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-08T00:22:48.506 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-08T00:22:48.507 DEBUG:teuthology.orchestra.run.smithi099:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-08T00:22:48.509 DEBUG:teuthology.orchestra.run.smithi191:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-08T00:22:48.521 INFO:teuthology.orchestra.run.smithi099.stdout: 263007 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 8 00:22 /home/ubuntu/cephtest 2023-10-08T00:22:48.538 INFO:teuthology.orchestra.run.smithi191.stdout: 263007 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 8 00:22 /home/ubuntu/cephtest 2023-10-08T00:22:48.540 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-08T00:22:48.550 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-08T00:22:48.572 INFO:teuthology.nuke:Checking targets against current locks 2023-10-08T00:22:48.599 DEBUG:teuthology.nuke:shortname: smithi099 2023-10-08T00:22:48.600 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-08T00:22:48.622 DEBUG:teuthology.nuke:shortname: smithi191 2023-10-08T00:22:48.623 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-08T00:22:48.626 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi099.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418110', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-07 23:52:52.349362', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD5LwXEONYcXRmTEhW6npr5lkKPFXj2fVvjuLfb0XPQFOInHhkdkKYBEEFID0gPuAhmpfy5GOshhTa4hwoSPgBk='} 2023-10-08T00:22:48.647 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi191.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418110', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-07 23:52:52.347975', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDoCm6mEVQTmB6VkKOVpObIqmV+5V75Vd5lLvS4FoTXnIeMM/GF1r4oBcMw/DMRs5W3hz8Dp6kZyQGqVobIVqRk='} 2023-10-08T00:22:48.681 INFO:teuthology.orchestra.console.smithi099:Power off 2023-10-08T00:22:48.681 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-08T00:22:48.703 DEBUG:teuthology.orchestra.console.smithi099:power off output: Chassis Power Control: Down/Off 2023-10-08T00:22:48.704 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T00:22:48.718 INFO:teuthology.orchestra.console.smithi191:Power off 2023-10-08T00:22:48.718 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-08T00:22:48.729 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is on 2023-10-08T00:22:48.743 DEBUG:teuthology.orchestra.console.smithi191:power off output: Chassis Power Control: Down/Off 2023-10-08T00:22:48.744 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T00:22:48.765 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is on 2023-10-08T00:22:52.730 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T00:22:52.767 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T00:22:52.842 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is on 2023-10-08T00:22:52.880 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is on 2023-10-08T00:22:56.846 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T00:22:56.881 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T00:22:56.956 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is off 2023-10-08T00:22:56.990 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is off 2023-10-08T00:22:57.057 INFO:teuthology.orchestra.console.smithi099:Power off completed 2023-10-08T00:22:57.091 INFO:teuthology.orchestra.console.smithi191:Power off completed 2023-10-08T00:22:57.196 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full} duration: 1365.5267553329468 failure_reason: '"1696723840.7136083 osd.2 (osd.2) 3 : cluster [WRN] OSD bench result of 227531.837772 IOPS exceeded the threshold limit of 500.000000 IOPS for osd.2. IOPS capacity is unchanged at 315.000000 IOPS. The recommendation is to establish the osd''s IOPS capacity using other benchmark tools (e.g. Fio) and then override osd_mclock_max_capacity_iops_[hdd|ssd]." in cluster log' flavor: default owner: scheduled_yuriw@teuthology success: false 2023-10-08T00:22:57.196 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-08T00:22:57.276 INFO:teuthology.run:FAIL